2023-10-06T10:24:54.933 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-06T10:24:54.933 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi --archive /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412360 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/multimds_misc} --owner scheduled_yuriw@teuthology -- /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412360/orig.config.yaml 2023-10-06T10:24:54.977 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T10:24:55.078 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412360 branch: wip-yuri3-testing-2023-10-05-0943-quincy description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/multimds_misc} email: yweinste@redhat.com first_in_suite: false job_id: '7412360' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri3-testing-2023-10-05-0943-quincy ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Scrub error on inode sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri3-testing-2023-10-05-0943-quincy sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 4002 sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri3-testing-2023-10-05-0943-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 targets: smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF53OE4i5vcG3W2Q2lwwvr0uEhnNAo2G09ujsL4/0+SF1JifRhTX0X3iB1Zpi5s16tvOQdqI//q3UQa2B/6tRpQ= smithi124.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ/sH53ZqpmGXnu83Xkf3Q5s6Bg6EwuiibrtBaWjsLHpp52l0b1kwbr7opnFzmdmy/p6yF4gbYp7YLTzdj6jNUo= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_multimds_misc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-05_23:54:57 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-06T10:24:55.078 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa; will attempt to use it 2023-10-06T10:24:55.079 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa/tasks 2023-10-06T10:24:55.079 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-06T10:24:55.081 INFO:teuthology.task.internal:Checking packages... 2023-10-06T10:24:55.110 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564' 2023-10-06T10:24:55.110 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-06T10:24:55.110 INFO:teuthology.packaging:ref: None 2023-10-06T10:24:55.110 INFO:teuthology.packaging:tag: None 2023-10-06T10:24:55.110 INFO:teuthology.packaging:branch: wip-yuri3-testing-2023-10-05-0943-quincy 2023-10-06T10:24:55.110 INFO:teuthology.packaging:sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T10:24:55.110 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri3-testing-2023-10-05-0943-quincy 2023-10-06T10:24:55.275 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1274.g68e53a4a 2023-10-06T10:24:55.275 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-06T10:24:55.281 INFO:teuthology.task.internal:no buildpackages task found 2023-10-06T10:24:55.281 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-06T10:24:55.322 INFO:teuthology.task.internal:Saving configuration 2023-10-06T10:24:55.340 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-06T10:24:55.346 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T10:24:55.372 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412360', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 10:19:41.323619', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF53OE4i5vcG3W2Q2lwwvr0uEhnNAo2G09ujsL4/0+SF1JifRhTX0X3iB1Zpi5s16tvOQdqI//q3UQa2B/6tRpQ='} 2023-10-06T10:24:55.395 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi124.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412360', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 10:19:41.321956', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ/sH53ZqpmGXnu83Xkf3Q5s6Bg6EwuiibrtBaWjsLHpp52l0b1kwbr7opnFzmdmy/p6yF4gbYp7YLTzdj6jNUo='} 2023-10-06T10:24:55.395 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-06T10:24:55.401 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-06T10:24:55.402 INFO:teuthology.task.internal:roles: ubuntu@smithi124.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-06T10:24:55.402 INFO:teuthology.run_tasks:Running task console_log... 2023-10-06T10:24:55.476 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd69de68b80>, signals=[15]) 2023-10-06T10:24:55.476 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-06T10:24:55.482 INFO:teuthology.task.internal:Opening connections... 2023-10-06T10:24:55.482 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T10:24:55.484 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:24:55.551 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi124.front.sepia.ceph.com 2023-10-06T10:24:55.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:24:55.618 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-06T10:24:55.624 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2023-10-06T10:24:55.641 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2023-10-06T10:24:55.641 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2023-10-06T10:24:55.696 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T10:24:55.697 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="8.4 (Ootpa)" 2023-10-06T10:24:55.697 INFO:teuthology.orchestra.run.smithi092.stdout:ID="rhel" 2023-10-06T10:24:55.697 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="fedora" 2023-10-06T10:24:55.697 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="8.4" 2023-10-06T10:24:55.697 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el8" 2023-10-06T10:24:55.697 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-06T10:24:55.697 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2023-10-06T10:24:55.697 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-06T10:24:55.697 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T10:24:55.697 INFO:teuthology.orchestra.run.smithi092.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T10:24:55.698 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T10:24:55.698 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:24:55.698 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T10:24:55.698 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-06T10:24:55.698 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T10:24:55.698 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-06T10:24:55.699 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2023-10-06T10:24:55.723 DEBUG:teuthology.orchestra.run.smithi124:> uname -m 2023-10-06T10:24:55.739 INFO:teuthology.orchestra.run.smithi124.stdout:x86_64 2023-10-06T10:24:55.739 DEBUG:teuthology.orchestra.run.smithi124:> cat /etc/os-release 2023-10-06T10:24:55.795 INFO:teuthology.orchestra.run.smithi124.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T10:24:55.795 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION="8.4 (Ootpa)" 2023-10-06T10:24:55.795 INFO:teuthology.orchestra.run.smithi124.stdout:ID="rhel" 2023-10-06T10:24:55.795 INFO:teuthology.orchestra.run.smithi124.stdout:ID_LIKE="fedora" 2023-10-06T10:24:55.795 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_ID="8.4" 2023-10-06T10:24:55.795 INFO:teuthology.orchestra.run.smithi124.stdout:PLATFORM_ID="platform:el8" 2023-10-06T10:24:55.795 INFO:teuthology.orchestra.run.smithi124.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-06T10:24:55.795 INFO:teuthology.orchestra.run.smithi124.stdout:ANSI_COLOR="0;31" 2023-10-06T10:24:55.796 INFO:teuthology.orchestra.run.smithi124.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-06T10:24:55.796 INFO:teuthology.orchestra.run.smithi124.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T10:24:55.796 INFO:teuthology.orchestra.run.smithi124.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T10:24:55.796 INFO:teuthology.orchestra.run.smithi124.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T10:24:55.796 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:24:55.796 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T10:24:55.796 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-06T10:24:55.796 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T10:24:55.796 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-06T10:24:55.797 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server 2023-10-06T10:24:55.814 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-06T10:24:55.823 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-06T10:24:55.841 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-06T10:24:55.842 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2023-10-06T10:24:55.845 DEBUG:teuthology.orchestra.run.smithi124:> test '!' -e /home/ubuntu/cephtest 2023-10-06T10:24:55.860 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-06T10:24:55.865 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-06T10:24:55.866 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2023-10-06T10:24:55.900 DEBUG:teuthology.orchestra.run.smithi124:> test -z $(ls -A /var/lib/ceph) 2023-10-06T10:24:55.914 INFO:teuthology.orchestra.run.smithi092.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-06T10:24:55.916 INFO:teuthology.orchestra.run.smithi124.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-06T10:24:55.916 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-06T10:24:55.976 INFO:teuthology.run_tasks:Running task kernel... 2023-10-06T10:24:55.992 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-10-06T10:24:55.992 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-10-06T10:24:55.992 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-10-06T10:24:55.992 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T10:24:55.993 DEBUG:teuthology.orchestra.run.smithi124:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T10:24:56.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T10:24:56.008 DEBUG:teuthology.orchestra.run.smithi124:> uname -r 2023-10-06T10:24:56.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T10:24:56.010 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2023-10-06T10:24:56.064 INFO:teuthology.orchestra.run.smithi124.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-06T10:24:56.064 INFO:teuthology.task.kernel:Running kernel on smithi124: 4.18.0-305.25.1.el8_4.x86_64 2023-10-06T10:24:56.064 DEBUG:teuthology.orchestra.run.smithi124:> sudo yum install -y kernel 2023-10-06T10:24:56.068 INFO:teuthology.orchestra.run.smithi092.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-06T10:24:56.068 INFO:teuthology.task.kernel:Running kernel on smithi092: 4.18.0-305.25.1.el8_4.x86_64 2023-10-06T10:24:56.068 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum install -y kernel 2023-10-06T10:24:57.474 INFO:teuthology.orchestra.run.smithi124.stdout:Updating Subscription Management repositories. 2023-10-06T10:24:57.475 INFO:teuthology.orchestra.run.smithi124.stdout:Unable to read consumer identity 2023-10-06T10:24:57.532 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2023-10-06T10:24:57.532 INFO:teuthology.orchestra.run.smithi092.stdout:Unable to read consumer identity 2023-10-06T10:24:58.813 INFO:teuthology.orchestra.run.smithi124.stdout:Copr repo for python3-asyncssh owned by ceph 29 kB/s | 3.5 kB 00:00 2023-10-06T10:24:58.889 INFO:teuthology.orchestra.run.smithi092.stdout:Copr repo for python3-asyncssh owned by ceph 32 kB/s | 3.5 kB 00:00 2023-10-06T10:24:59.381 INFO:teuthology.orchestra.run.smithi124.stdout:Extra Packages for Enterprise Linux 30 MB/s | 16 MB 00:00 2023-10-06T10:25:00.034 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2023-10-06T10:25:02.645 INFO:teuthology.orchestra.run.smithi124.stdout:lab-extras 4.5 MB/s | 24 kB 00:00 2023-10-06T10:25:03.342 INFO:teuthology.orchestra.run.smithi124.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-10-06T10:25:03.342 INFO:teuthology.orchestra.run.smithi124.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-10-06T10:25:03.343 INFO:teuthology.orchestra.run.smithi124.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-10-06T10:25:03.343 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 4.6 MB/s | 24 kB 00:00 2023-10-06T10:25:03.500 INFO:teuthology.orchestra.run.smithi124.stdout:Dependencies resolved. 2023-10-06T10:25:03.501 INFO:teuthology.orchestra.run.smithi124.stdout:Nothing to do. 2023-10-06T10:25:03.501 INFO:teuthology.orchestra.run.smithi124.stdout:Complete! 2023-10-06T10:25:03.601 DEBUG:teuthology.orchestra.run.smithi124:> echo no | sudo yum reinstall kernel || true 2023-10-06T10:25:03.945 INFO:teuthology.orchestra.run.smithi124.stdout:Updating Subscription Management repositories. 2023-10-06T10:25:03.945 INFO:teuthology.orchestra.run.smithi124.stdout:Unable to read consumer identity 2023-10-06T10:25:04.044 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-10-06T10:25:04.044 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-10-06T10:25:04.044 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-10-06T10:25:04.204 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2023-10-06T10:25:04.205 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2023-10-06T10:25:04.205 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2023-10-06T10:25:04.233 INFO:teuthology.orchestra.run.smithi124.stdout:Last metadata expiration check: 0:00:02 ago on Fri 06 Oct 2023 10:25:02 AM UTC. 2023-10-06T10:25:04.308 DEBUG:teuthology.orchestra.run.smithi092:> echo no | sudo yum reinstall kernel || true 2023-10-06T10:25:04.371 INFO:teuthology.orchestra.run.smithi124.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-06T10:25:04.371 INFO:teuthology.orchestra.run.smithi124.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T10:25:04.371 INFO:teuthology.orchestra.run.smithi124.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T10:25:04.394 INFO:teuthology.orchestra.run.smithi124.stderr:Error: No packages marked for reinstall. 2023-10-06T10:25:04.443 DEBUG:teuthology.orchestra.run.smithi124:> sudo yum reinstall -y kernel || true 2023-10-06T10:25:04.690 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2023-10-06T10:25:04.690 INFO:teuthology.orchestra.run.smithi092.stdout:Unable to read consumer identity 2023-10-06T10:25:04.785 INFO:teuthology.orchestra.run.smithi124.stdout:Updating Subscription Management repositories. 2023-10-06T10:25:04.785 INFO:teuthology.orchestra.run.smithi124.stdout:Unable to read consumer identity 2023-10-06T10:25:04.986 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:01 ago on Fri 06 Oct 2023 10:25:03 AM UTC. 2023-10-06T10:25:05.073 INFO:teuthology.orchestra.run.smithi124.stdout:Last metadata expiration check: 0:00:03 ago on Fri 06 Oct 2023 10:25:02 AM UTC. 2023-10-06T10:25:05.116 INFO:teuthology.orchestra.run.smithi092.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-06T10:25:05.116 INFO:teuthology.orchestra.run.smithi092.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T10:25:05.116 INFO:teuthology.orchestra.run.smithi092.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T10:25:05.142 INFO:teuthology.orchestra.run.smithi092.stderr:Error: No packages marked for reinstall. 2023-10-06T10:25:05.192 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum reinstall -y kernel || true 2023-10-06T10:25:05.246 INFO:teuthology.orchestra.run.smithi124.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-06T10:25:05.247 INFO:teuthology.orchestra.run.smithi124.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T10:25:05.247 INFO:teuthology.orchestra.run.smithi124.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T10:25:05.259 INFO:teuthology.orchestra.run.smithi124.stderr:Error: No packages marked for reinstall. 2023-10-06T10:25:05.314 DEBUG:teuthology.orchestra.run.smithi124:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T10:25:05.439 INFO:teuthology.orchestra.run.smithi124.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-06T10:25:05.440 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-06T10:25:05.440 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-10-06T10:25:05.440 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-06T10:25:05.440 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-06T10:25:05.440 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2023-10-06T10:25:05.440 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi124.front.sepia.ceph.com, path=None, version=distro) 2023-10-06T10:25:05.440 DEBUG:teuthology.orchestra.run.smithi124:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T10:25:05.533 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2023-10-06T10:25:05.533 INFO:teuthology.orchestra.run.smithi092.stdout:Unable to read consumer identity 2023-10-06T10:25:05.565 INFO:teuthology.orchestra.run.smithi124.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-06T10:25:05.565 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-06T10:25:05.565 DEBUG:teuthology.orchestra.run.smithi124:> sudo rpm -qi grub2-tools 2023-10-06T10:25:05.591 INFO:teuthology.orchestra.run.smithi124.stdout:Name : grub2-tools 2023-10-06T10:25:05.592 INFO:teuthology.orchestra.run.smithi124.stdout:Epoch : 1 2023-10-06T10:25:05.592 INFO:teuthology.orchestra.run.smithi124.stdout:Version : 2.02 2023-10-06T10:25:05.592 INFO:teuthology.orchestra.run.smithi124.stdout:Release : 99.el8_4.1 2023-10-06T10:25:05.592 INFO:teuthology.orchestra.run.smithi124.stdout:Architecture: x86_64 2023-10-06T10:25:05.592 INFO:teuthology.orchestra.run.smithi124.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-06T10:25:05.592 INFO:teuthology.orchestra.run.smithi124.stdout:Group : System Environment/Base 2023-10-06T10:25:05.592 INFO:teuthology.orchestra.run.smithi124.stdout:Size : 9264631 2023-10-06T10:25:05.592 INFO:teuthology.orchestra.run.smithi124.stdout:License : GPLv3+ 2023-10-06T10:25:05.592 INFO:teuthology.orchestra.run.smithi124.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-06T10:25:05.592 INFO:teuthology.orchestra.run.smithi124.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-06T10:25:05.592 INFO:teuthology.orchestra.run.smithi124.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-06T10:25:05.593 INFO:teuthology.orchestra.run.smithi124.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-06T10:25:05.593 INFO:teuthology.orchestra.run.smithi124.stdout:Relocations : (not relocatable) 2023-10-06T10:25:05.593 INFO:teuthology.orchestra.run.smithi124.stdout:Packager : Red Hat, Inc. 2023-10-06T10:25:05.593 INFO:teuthology.orchestra.run.smithi124.stdout:Vendor : Red Hat, Inc. 2023-10-06T10:25:05.593 INFO:teuthology.orchestra.run.smithi124.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T10:25:05.593 INFO:teuthology.orchestra.run.smithi124.stdout:Summary : Support tools for GRUB. 2023-10-06T10:25:05.593 INFO:teuthology.orchestra.run.smithi124.stdout:Description : 2023-10-06T10:25:05.593 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:25:05.593 INFO:teuthology.orchestra.run.smithi124.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T10:25:05.593 INFO:teuthology.orchestra.run.smithi124.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T10:25:05.593 INFO:teuthology.orchestra.run.smithi124.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T10:25:05.593 INFO:teuthology.orchestra.run.smithi124.stdout:hardware devices. 2023-10-06T10:25:05.594 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:25:05.594 INFO:teuthology.orchestra.run.smithi124.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T10:25:05.594 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T10:25:05.594 INFO:teuthology.task.kernel:Updating grub on smithi124 to boot 4.18.0-425.3.1.el8.x86_64 2023-10-06T10:25:05.595 DEBUG:teuthology.orchestra.run.smithi124:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T10:25:05.814 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:02 ago on Fri 06 Oct 2023 10:25:03 AM UTC. 2023-10-06T10:25:05.948 INFO:teuthology.orchestra.run.smithi092.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-06T10:25:05.948 INFO:teuthology.orchestra.run.smithi092.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T10:25:05.949 INFO:teuthology.orchestra.run.smithi092.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T10:25:05.963 INFO:teuthology.orchestra.run.smithi092.stderr:Error: No packages marked for reinstall. 2023-10-06T10:25:06.011 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T10:25:06.136 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-06T10:25:06.136 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-06T10:25:06.136 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-10-06T10:25:06.136 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-06T10:25:06.136 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-06T10:25:06.137 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2023-10-06T10:25:06.137 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi092.front.sepia.ceph.com, path=None, version=distro) 2023-10-06T10:25:06.137 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T10:25:06.260 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-06T10:25:06.260 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-06T10:25:06.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo rpm -qi grub2-tools 2023-10-06T10:25:06.287 INFO:teuthology.orchestra.run.smithi092.stdout:Name : grub2-tools 2023-10-06T10:25:06.287 INFO:teuthology.orchestra.run.smithi092.stdout:Epoch : 1 2023-10-06T10:25:06.287 INFO:teuthology.orchestra.run.smithi092.stdout:Version : 2.02 2023-10-06T10:25:06.287 INFO:teuthology.orchestra.run.smithi092.stdout:Release : 99.el8_4.1 2023-10-06T10:25:06.287 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: x86_64 2023-10-06T10:25:06.287 INFO:teuthology.orchestra.run.smithi092.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-06T10:25:06.288 INFO:teuthology.orchestra.run.smithi092.stdout:Group : System Environment/Base 2023-10-06T10:25:06.288 INFO:teuthology.orchestra.run.smithi092.stdout:Size : 9264631 2023-10-06T10:25:06.288 INFO:teuthology.orchestra.run.smithi092.stdout:License : GPLv3+ 2023-10-06T10:25:06.288 INFO:teuthology.orchestra.run.smithi092.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-06T10:25:06.288 INFO:teuthology.orchestra.run.smithi092.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-06T10:25:06.288 INFO:teuthology.orchestra.run.smithi092.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-06T10:25:06.288 INFO:teuthology.orchestra.run.smithi092.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-06T10:25:06.288 INFO:teuthology.orchestra.run.smithi092.stdout:Relocations : (not relocatable) 2023-10-06T10:25:06.288 INFO:teuthology.orchestra.run.smithi092.stdout:Packager : Red Hat, Inc. 2023-10-06T10:25:06.288 INFO:teuthology.orchestra.run.smithi092.stdout:Vendor : Red Hat, Inc. 2023-10-06T10:25:06.288 INFO:teuthology.orchestra.run.smithi092.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T10:25:06.288 INFO:teuthology.orchestra.run.smithi092.stdout:Summary : Support tools for GRUB. 2023-10-06T10:25:06.288 INFO:teuthology.orchestra.run.smithi092.stdout:Description : 2023-10-06T10:25:06.289 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:25:06.289 INFO:teuthology.orchestra.run.smithi092.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T10:25:06.289 INFO:teuthology.orchestra.run.smithi092.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T10:25:06.289 INFO:teuthology.orchestra.run.smithi092.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T10:25:06.289 INFO:teuthology.orchestra.run.smithi092.stdout:hardware devices. 2023-10-06T10:25:06.289 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:25:06.289 INFO:teuthology.orchestra.run.smithi092.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T10:25:06.290 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T10:25:06.290 INFO:teuthology.task.kernel:Updating grub on smithi092 to boot 4.18.0-425.3.1.el8.x86_64 2023-10-06T10:25:06.290 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T10:25:06.828 INFO:teuthology.orchestra.run.smithi124.stderr:Generating grub configuration file ... 2023-10-06T10:25:07.616 INFO:teuthology.orchestra.run.smithi092.stderr:Generating grub configuration file ... 2023-10-06T10:25:08.974 INFO:teuthology.orchestra.run.smithi124.stderr:done 2023-10-06T10:25:08.976 DEBUG:teuthology.orchestra.run.smithi124:> mktemp 2023-10-06T10:25:08.991 INFO:teuthology.orchestra.run.smithi124.stdout:/tmp/tmp.RT13uEXx6b 2023-10-06T10:25:08.991 DEBUG:teuthology.orchestra.run.smithi124:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.RT13uEXx6b 2023-10-06T10:25:09.075 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 0666 /tmp/tmp.RT13uEXx6b 2023-10-06T10:25:09.228 DEBUG:teuthology.orchestra.remote:smithi124:/tmp/tmp.RT13uEXx6b is 6KB 2023-10-06T10:25:09.277 DEBUG:teuthology.orchestra.run.smithi124:> rm -fr /tmp/tmp.RT13uEXx6b 2023-10-06T10:25:09.291 DEBUG:teuthology.orchestra.run.smithi124:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T10:25:09.372 INFO:teuthology.orchestra.run.smithi124.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-06T10:25:09.372 INFO:teuthology.orchestra.run.smithi124.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-06T10:25:09.372 INFO:teuthology.orchestra.run.smithi124.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-06T10:25:09.372 INFO:teuthology.orchestra.run.smithi124.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-06T10:25:09.373 DEBUG:teuthology.orchestra.run.smithi124:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-10-06T10:25:09.563 DEBUG:teuthology.orchestra.run.smithi124:> sudo shutdown -r now 2023-10-06T10:25:09.728 INFO:teuthology.orchestra.run.smithi092.stderr:done 2023-10-06T10:25:09.730 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2023-10-06T10:25:09.744 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.BiN5TP0zGY 2023-10-06T10:25:09.744 DEBUG:teuthology.orchestra.run.smithi092:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.BiN5TP0zGY 2023-10-06T10:25:09.814 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.BiN5TP0zGY 2023-10-06T10:25:09.979 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.BiN5TP0zGY is 6KB 2023-10-06T10:25:10.028 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.BiN5TP0zGY 2023-10-06T10:25:10.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T10:25:10.104 INFO:teuthology.orchestra.run.smithi092.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-06T10:25:10.104 INFO:teuthology.orchestra.run.smithi092.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-06T10:25:10.104 INFO:teuthology.orchestra.run.smithi092.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-06T10:25:10.104 INFO:teuthology.orchestra.run.smithi092.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-06T10:25:10.105 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-10-06T10:25:10.257 DEBUG:teuthology.orchestra.run.smithi092:> sudo shutdown -r now 2023-10-06T10:25:39.594 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-10-06T10:25:39.595 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2023-10-06T10:25:39.595 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:25:40.260 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-06T10:25:40.263 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-10-06T10:25:40.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:25:58.138 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-10-06T10:25:58.650 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2023-10-06T10:26:07.148 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2023-10-06T10:26:07.149 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:26:07.652 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-10-06T10:26:07.653 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:26:10.202 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-10-06T10:26:10.714 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2023-10-06T10:26:22.214 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2023-10-06T10:26:22.215 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:26:22.785 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-10-06T10:26:22.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:26:40.638 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-10-06T10:26:54.591 DEBUG:teuthology.orchestra.run.smithi092:> true 2023-10-06T10:26:54.942 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-10-06T10:26:54.943 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "4.18.0-425.3.1.el8.x86_64"... 2023-10-06T10:26:54.943 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2023-10-06T10:26:54.956 INFO:teuthology.orchestra.run.smithi092.stdout:4.18.0-425.3.1.el8.x86_64 2023-10-06T10:26:54.957 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-10-06T10:26:54.957 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T10:26:54.957 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-06T10:26:55.639 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2023-10-06T10:26:55.640 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:26:55.958 DEBUG:teuthology.parallel:result is None 2023-10-06T10:26:56.137 DEBUG:teuthology.orchestra.run.smithi124:> true 2023-10-06T10:26:56.383 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi124.front.sepia.ceph.com' 2023-10-06T10:26:56.383 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "4.18.0-425.3.1.el8.x86_64"... 2023-10-06T10:26:56.384 DEBUG:teuthology.orchestra.run.smithi124:> uname -r 2023-10-06T10:26:56.435 INFO:teuthology.orchestra.run.smithi124.stdout:4.18.0-425.3.1.el8.x86_64 2023-10-06T10:26:56.435 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-10-06T10:26:56.435 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T10:26:56.435 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-06T10:26:57.436 DEBUG:teuthology.parallel:result is None 2023-10-06T10:26:57.437 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-06T10:26:57.444 INFO:teuthology.task.internal:Creating test directory... 2023-10-06T10:26:57.444 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T10:26:57.485 DEBUG:teuthology.orchestra.run.smithi124:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T10:26:57.744 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-06T10:26:57.751 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-06T10:26:57.777 INFO:teuthology.task.internal:Creating archive directory... 2023-10-06T10:26:57.777 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T10:26:57.821 DEBUG:teuthology.orchestra.run.smithi124:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T10:26:57.943 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-06T10:26:57.950 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-06T10:26:57.950 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T10:26:57.991 DEBUG:teuthology.orchestra.run.smithi124:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T10:26:58.208 INFO:teuthology.orchestra.run.smithi124.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T10:26:58.288 INFO:teuthology.orchestra.run.smithi124.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T10:26:58.561 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T10:26:58.783 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T10:26:58.803 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-06T10:26:58.810 INFO:teuthology.task.internal:Configuring sudo... 2023-10-06T10:26:58.811 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T10:26:58.865 DEBUG:teuthology.orchestra.run.smithi124:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T10:26:59.210 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-06T10:26:59.219 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-06T10:26:59.219 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T10:26:59.221 DEBUG:teuthology.orchestra.run.smithi124:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T10:26:59.530 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T10:26:59.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T10:26:59.686 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T10:26:59.726 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T10:26:59.769 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:26:59.769 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T10:26:59.835 DEBUG:teuthology.orchestra.run.smithi124:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T10:27:00.188 DEBUG:teuthology.orchestra.run.smithi124:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T10:27:00.623 DEBUG:teuthology.orchestra.run.smithi124:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T10:27:00.775 DEBUG:teuthology.orchestra.run.smithi124:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T10:27:00.798 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:27:00.798 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T10:27:00.862 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2023-10-06T10:27:00.864 DEBUG:teuthology.orchestra.run.smithi124:> sudo service rsyslog restart 2023-10-06T10:27:00.925 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T10:27:01.122 INFO:teuthology.orchestra.run.smithi124.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T10:27:01.158 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-06T10:27:01.165 INFO:teuthology.task.internal:Starting timer... 2023-10-06T10:27:01.165 INFO:teuthology.run_tasks:Running task pcp... 2023-10-06T10:27:01.213 INFO:teuthology.run_tasks:Running task selinux... 2023-10-06T10:27:01.245 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-10-06T10:27:01.245 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2023-10-06T10:27:01.314 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: [ OK ] 2023-10-06T10:27:01.316 DEBUG:teuthology.orchestra.run.smithi124:> sudo service auditd rotate 2023-10-06T10:27:01.476 INFO:teuthology.orchestra.run.smithi124.stdout:Rotating logs: [ OK ] 2023-10-06T10:27:01.478 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-06T10:27:01.478 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2023-10-06T10:27:01.504 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2023-10-06T10:27:01.504 DEBUG:teuthology.orchestra.run.smithi124:> /usr/sbin/getenforce 2023-10-06T10:27:01.601 INFO:teuthology.orchestra.run.smithi124.stdout:Permissive 2023-10-06T10:27:01.602 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi092.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi124.front.sepia.ceph.com': 'permissive'} 2023-10-06T10:27:01.602 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T10:27:01.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T10:27:01.627 DEBUG:teuthology.orchestra.run.smithi124:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T10:27:01.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T10:27:01.653 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-06T10:27:01.653 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2023-10-06T10:27:01.692 DEBUG:teuthology.orchestra.run.smithi124:> sudo /usr/sbin/setenforce permissive 2023-10-06T10:27:01.784 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-06T10:27:01.794 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-06T10:27:01.799 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-06T10:27:01.918 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-06T10:27:01.933 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-06T10:27:01.934 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi092.front.sepia.ceph.com,smithi124.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-06T10:35:18.695 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi124.front.sepia.ceph.com')] 2023-10-06T10:35:18.696 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-10-06T10:35:18.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:35:18.767 DEBUG:teuthology.orchestra.run.smithi092:> true 2023-10-06T10:35:18.833 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-10-06T10:35:18.834 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2023-10-06T10:35:18.834 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:35:18.901 DEBUG:teuthology.orchestra.run.smithi124:> true 2023-10-06T10:35:18.964 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi124.front.sepia.ceph.com' 2023-10-06T10:35:18.965 INFO:teuthology.run_tasks:Running task clock... 2023-10-06T10:35:18.981 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-06T10:35:18.982 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T10:35:18.982 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T10:35:18.985 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T10:35:18.985 DEBUG:teuthology.orchestra.run.smithi124:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T10:35:19.011 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T10:35:19.026 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T10:35:19.032 INFO:teuthology.orchestra.run.smithi124.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T10:35:19.045 INFO:teuthology.orchestra.run.smithi124.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T10:35:19.059 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2023-10-06T10:35:19.069 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2023-10-06T10:35:19.083 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T10:35:19.086 INFO:teuthology.orchestra.run.smithi124.stderr:sudo: ntpd: command not found 2023-10-06T10:35:19.096 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T10:35:19.097 INFO:teuthology.orchestra.run.smithi124.stdout:506 Cannot talk to daemon 2023-10-06T10:35:19.110 INFO:teuthology.orchestra.run.smithi124.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T10:35:19.124 INFO:teuthology.orchestra.run.smithi124.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T10:35:19.165 INFO:teuthology.orchestra.run.smithi092.stderr:bash: ntpq: command not found 2023-10-06T10:35:19.168 INFO:teuthology.orchestra.run.smithi092.stdout:210 Number of sources = 4 2023-10-06T10:35:19.168 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T10:35:19.168 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2023-10-06T10:35:19.168 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T10:35:19.168 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T10:35:19.168 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T10:35:19.168 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T10:35:19.182 INFO:teuthology.orchestra.run.smithi124.stderr:bash: ntpq: command not found 2023-10-06T10:35:19.184 INFO:teuthology.orchestra.run.smithi124.stdout:210 Number of sources = 4 2023-10-06T10:35:19.185 INFO:teuthology.orchestra.run.smithi124.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T10:35:19.185 INFO:teuthology.orchestra.run.smithi124.stdout:=============================================================================== 2023-10-06T10:35:19.185 INFO:teuthology.orchestra.run.smithi124.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T10:35:19.185 INFO:teuthology.orchestra.run.smithi124.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T10:35:19.185 INFO:teuthology.orchestra.run.smithi124.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T10:35:19.185 INFO:teuthology.orchestra.run.smithi124.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T10:35:19.186 INFO:teuthology.run_tasks:Running task install... 2023-10-06T10:35:19.194 DEBUG:teuthology.task.install:project ceph 2023-10-06T10:35:19.194 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}} 2023-10-06T10:35:19.194 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'} 2023-10-06T10:35:19.194 INFO:teuthology.task.install:Using flavor: default 2023-10-06T10:35:19.201 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-06T10:35:19.201 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-06T10:35:19.202 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'tag': None, 'wait_for_package': False} 2023-10-06T10:35:19.202 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T10:35:19.204 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'tag': None, 'wait_for_package': False} 2023-10-06T10:35:19.204 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T10:35:19.431 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/ 2023-10-06T10:35:19.431 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1274.g68e53a4a 2023-10-06T10:35:19.435 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/ 2023-10-06T10:35:19.436 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1274.g68e53a4a 2023-10-06T10:35:19.544 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T10:35:19.544 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:35:19.544 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T10:35:19.571 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T10:35:19.571 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T10:35:19.611 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T10:35:19.611 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:35:19.611 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T10:35:19.638 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T10:35:19.638 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T10:35:19.639 DEBUG:teuthology.orchestra.run.smithi124:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T10:35:19.703 DEBUG:teuthology.orchestra.run.smithi124:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T10:35:19.713 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T10:35:19.777 DEBUG:teuthology.orchestra.run.smithi124:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T10:35:19.778 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2023-10-06T10:35:19.778 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2023-10-06T10:35:19.841 INFO:teuthology.orchestra.run.smithi124.stdout:check_obsoletes = 1 2023-10-06T10:35:19.842 DEBUG:teuthology.orchestra.run.smithi124:> sudo yum clean all 2023-10-06T10:35:20.168 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2023-10-06T10:35:20.232 INFO:teuthology.orchestra.run.smithi124.stdout:Updating Subscription Management repositories. 2023-10-06T10:35:20.507 INFO:teuthology.orchestra.run.smithi092.stdout:44 files removed 2023-10-06T10:35:20.558 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T10:35:20.572 INFO:teuthology.orchestra.run.smithi124.stdout:44 files removed 2023-10-06T10:35:20.622 DEBUG:teuthology.orchestra.run.smithi124:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T10:35:20.906 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2023-10-06T10:35:20.971 INFO:teuthology.orchestra.run.smithi124.stdout:Updating Subscription Management repositories. 2023-10-06T10:35:21.683 INFO:teuthology.orchestra.run.smithi124.stdout:ceph packages for x86_64 280 kB/s | 78 kB 00:00 2023-10-06T10:35:21.714 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 207 kB/s | 78 kB 00:00 2023-10-06T10:35:21.966 INFO:teuthology.orchestra.run.smithi124.stdout:ceph noarch packages 38 kB/s | 9.2 kB 00:00 2023-10-06T10:35:22.000 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 37 kB/s | 9.2 kB 00:00 2023-10-06T10:35:22.236 INFO:teuthology.orchestra.run.smithi124.stdout:ceph source packages 6.0 kB/s | 1.4 kB 00:00 2023-10-06T10:35:22.331 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 4.8 kB/s | 1.4 kB 00:00 2023-10-06T10:35:22.342 INFO:teuthology.orchestra.run.smithi124.stdout:Copr repo for python3-asyncssh owned by ceph 45 kB/s | 3.5 kB 00:00 2023-10-06T10:35:22.437 INFO:teuthology.orchestra.run.smithi092.stdout:Copr repo for python3-asyncssh owned by ceph 45 kB/s | 3.5 kB 00:00 2023-10-06T10:35:23.362 INFO:teuthology.orchestra.run.smithi124.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:00 2023-10-06T10:35:23.814 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-10-06T10:35:26.645 INFO:teuthology.orchestra.run.smithi124.stdout:lab-extras 4.3 MB/s | 24 kB 00:00 2023-10-06T10:35:27.026 INFO:teuthology.orchestra.run.smithi124.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 127 MB/s | 38 MB 00:00 2023-10-06T10:35:27.199 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 4.3 MB/s | 24 kB 00:00 2023-10-06T10:35:27.577 INFO:teuthology.orchestra.run.smithi092.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 128 MB/s | 38 MB 00:00 2023-10-06T10:35:32.664 INFO:teuthology.orchestra.run.smithi124.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 121 MB/s | 33 MB 00:00 2023-10-06T10:35:33.239 INFO:teuthology.orchestra.run.smithi092.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 122 MB/s | 33 MB 00:00 2023-10-06T10:35:40.784 INFO:teuthology.orchestra.run.smithi124.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T10:35:40.784 INFO:teuthology.orchestra.run.smithi124.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T10:35:40.795 INFO:teuthology.orchestra.run.smithi124.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-06T10:35:40.796 INFO:teuthology.orchestra.run.smithi124.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-06T10:35:40.797 INFO:teuthology.orchestra.run.smithi124.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-06T10:35:40.798 INFO:teuthology.orchestra.run.smithi124.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T10:35:40.799 INFO:teuthology.orchestra.run.smithi124.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T10:35:40.802 INFO:teuthology.orchestra.run.smithi124.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-06T10:35:40.802 INFO:teuthology.orchestra.run.smithi124.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-06T10:35:41.100 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T10:35:41.101 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T10:35:41.112 INFO:teuthology.orchestra.run.smithi092.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-06T10:35:41.113 INFO:teuthology.orchestra.run.smithi092.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-06T10:35:41.114 INFO:teuthology.orchestra.run.smithi092.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-06T10:35:41.115 INFO:teuthology.orchestra.run.smithi092.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T10:35:41.117 INFO:teuthology.orchestra.run.smithi092.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T10:35:41.119 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-06T10:35:41.120 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-06T10:35:41.286 INFO:teuthology.orchestra.run.smithi124.stdout:Dependencies resolved. 2023-10-06T10:35:41.297 INFO:teuthology.orchestra.run.smithi124.stdout:====================================================================================================================== 2023-10-06T10:35:41.297 INFO:teuthology.orchestra.run.smithi124.stdout: Package Arch Version Repository Size 2023-10-06T10:35:41.297 INFO:teuthology.orchestra.run.smithi124.stdout:====================================================================================================================== 2023-10-06T10:35:41.297 INFO:teuthology.orchestra.run.smithi124.stdout:Installing: 2023-10-06T10:35:41.298 INFO:teuthology.orchestra.run.smithi124.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T10:35:41.298 INFO:teuthology.orchestra.run.smithi124.stdout: ceph x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 6.4 k 2023-10-06T10:35:41.298 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-base x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.2 M 2023-10-06T10:35:41.298 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 877 k 2023-10-06T10:35:41.298 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 162 k 2023-10-06T10:35:41.298 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 1.6 M 2023-10-06T10:35:41.298 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 121 k 2023-10-06T10:35:41.298 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 1.7 M 2023-10-06T10:35:41.298 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 7.4 M 2023-10-06T10:35:41.298 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-rook noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 47 k 2023-10-06T10:35:41.298 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-radosgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T10:35:41.299 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-test x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 52 M 2023-10-06T10:35:41.299 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-volume noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 267 k 2023-10-06T10:35:41.299 INFO:teuthology.orchestra.run.smithi124.stdout: cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 80 k 2023-10-06T10:35:41.299 INFO:teuthology.orchestra.run.smithi124.stdout: cephfs-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 255 k 2023-10-06T10:35:41.299 INFO:teuthology.orchestra.run.smithi124.stdout: cephfs-top noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 18 k 2023-10-06T10:35:41.299 INFO:teuthology.orchestra.run.smithi124.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-06T10:35:41.299 INFO:teuthology.orchestra.run.smithi124.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T10:35:41.299 INFO:teuthology.orchestra.run.smithi124.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T10:35:41.299 INFO:teuthology.orchestra.run.smithi124.stdout: libcephfs-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 25 k 2023-10-06T10:35:41.299 INFO:teuthology.orchestra.run.smithi124.stdout: libcephfs2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 797 k 2023-10-06T10:35:41.299 INFO:teuthology.orchestra.run.smithi124.stdout: librados-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 130 k 2023-10-06T10:35:41.299 INFO:teuthology.orchestra.run.smithi124.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T10:35:41.300 INFO:teuthology.orchestra.run.smithi124.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-06T10:35:41.300 INFO:teuthology.orchestra.run.smithi124.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T10:35:41.300 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cephfs x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 190 k 2023-10-06T10:35:41.300 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rados x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 371 k 2023-10-06T10:35:41.300 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 358 k 2023-10-06T10:35:41.300 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 107 k 2023-10-06T10:35:41.300 INFO:teuthology.orchestra.run.smithi124.stdout: rbd-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 91 k 2023-10-06T10:35:41.300 INFO:teuthology.orchestra.run.smithi124.stdout: rbd-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T10:35:41.300 INFO:teuthology.orchestra.run.smithi124.stdout: rbd-nbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 173 k 2023-10-06T10:35:41.300 INFO:teuthology.orchestra.run.smithi124.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-06T10:35:41.301 INFO:teuthology.orchestra.run.smithi124.stdout:Upgrading: 2023-10-06T10:35:41.301 INFO:teuthology.orchestra.run.smithi124.stdout: librados2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.4 M 2023-10-06T10:35:41.301 INFO:teuthology.orchestra.run.smithi124.stdout: librbd1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T10:35:41.301 INFO:teuthology.orchestra.run.smithi124.stdout:Installing dependencies: 2023-10-06T10:35:41.301 INFO:teuthology.orchestra.run.smithi124.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T10:35:41.301 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 23 M 2023-10-06T10:35:41.301 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 23 k 2023-10-06T10:35:41.301 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mds x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 2.3 M 2023-10-06T10:35:41.301 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 238 k 2023-10-06T10:35:41.301 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mon x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.9 M 2023-10-06T10:35:41.301 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-osd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T10:35:41.301 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 14 k 2023-10-06T10:35:41.302 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-selinux x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 24 k 2023-10-06T10:35:41.302 INFO:teuthology.orchestra.run.smithi124.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T10:35:41.302 INFO:teuthology.orchestra.run.smithi124.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T10:35:41.302 INFO:teuthology.orchestra.run.smithi124.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-06T10:35:41.302 INFO:teuthology.orchestra.run.smithi124.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T10:35:41.302 INFO:teuthology.orchestra.run.smithi124.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T10:35:41.302 INFO:teuthology.orchestra.run.smithi124.stdout: libcephsqlite x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 175 k 2023-10-06T10:35:41.302 INFO:teuthology.orchestra.run.smithi124.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-06T10:35:41.302 INFO:teuthology.orchestra.run.smithi124.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-06T10:35:41.302 INFO:teuthology.orchestra.run.smithi124.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T10:35:41.303 INFO:teuthology.orchestra.run.smithi124.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-06T10:35:41.303 INFO:teuthology.orchestra.run.smithi124.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T10:35:41.303 INFO:teuthology.orchestra.run.smithi124.stdout: libradosstriper1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 506 k 2023-10-06T10:35:41.303 INFO:teuthology.orchestra.run.smithi124.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-06T10:35:41.303 INFO:teuthology.orchestra.run.smithi124.stdout: librgw2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.6 M 2023-10-06T10:35:41.303 INFO:teuthology.orchestra.run.smithi124.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T10:35:41.303 INFO:teuthology.orchestra.run.smithi124.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-06T10:35:41.303 INFO:teuthology.orchestra.run.smithi124.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T10:35:41.303 INFO:teuthology.orchestra.run.smithi124.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T10:35:41.303 INFO:teuthology.orchestra.run.smithi124.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T10:35:41.303 INFO:teuthology.orchestra.run.smithi124.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T10:35:41.304 INFO:teuthology.orchestra.run.smithi124.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-06T10:35:41.304 INFO:teuthology.orchestra.run.smithi124.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T10:35:41.304 INFO:teuthology.orchestra.run.smithi124.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T10:35:41.304 INFO:teuthology.orchestra.run.smithi124.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T10:35:41.304 INFO:teuthology.orchestra.run.smithi124.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T10:35:41.304 INFO:teuthology.orchestra.run.smithi124.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T10:35:41.304 INFO:teuthology.orchestra.run.smithi124.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T10:35:41.304 INFO:teuthology.orchestra.run.smithi124.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T10:35:41.304 INFO:teuthology.orchestra.run.smithi124.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T10:35:41.304 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T10:35:41.304 INFO:teuthology.orchestra.run.smithi124.stdout: python3-ceph-argparse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 45 k 2023-10-06T10:35:41.305 INFO:teuthology.orchestra.run.smithi124.stdout: python3-ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 115 k 2023-10-06T10:35:41.305 INFO:teuthology.orchestra.run.smithi124.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T10:35:41.305 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T10:35:41.305 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T10:35:41.305 INFO:teuthology.orchestra.run.smithi124.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T10:35:41.305 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T10:35:41.305 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T10:35:41.305 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T10:35:41.305 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T10:35:41.305 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T10:35:41.305 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T10:35:41.306 INFO:teuthology.orchestra.run.smithi124.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T10:35:41.306 INFO:teuthology.orchestra.run.smithi124.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T10:35:41.306 INFO:teuthology.orchestra.run.smithi124.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T10:35:41.306 INFO:teuthology.orchestra.run.smithi124.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T10:35:41.306 INFO:teuthology.orchestra.run.smithi124.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T10:35:41.306 INFO:teuthology.orchestra.run.smithi124.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T10:35:41.306 INFO:teuthology.orchestra.run.smithi124.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T10:35:41.306 INFO:teuthology.orchestra.run.smithi124.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T10:35:41.306 INFO:teuthology.orchestra.run.smithi124.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T10:35:41.306 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T10:35:41.307 INFO:teuthology.orchestra.run.smithi124.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T10:35:41.307 INFO:teuthology.orchestra.run.smithi124.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T10:35:41.307 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T10:35:41.307 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T10:35:41.307 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T10:35:41.307 INFO:teuthology.orchestra.run.smithi124.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T10:35:41.307 INFO:teuthology.orchestra.run.smithi124.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T10:35:41.307 INFO:teuthology.orchestra.run.smithi124.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T10:35:41.307 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T10:35:41.307 INFO:teuthology.orchestra.run.smithi124.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-06T10:35:41.308 INFO:teuthology.orchestra.run.smithi124.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T10:35:41.308 INFO:teuthology.orchestra.run.smithi124.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T10:35:41.308 INFO:teuthology.orchestra.run.smithi124.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T10:35:41.308 INFO:teuthology.orchestra.run.smithi124.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T10:35:41.308 INFO:teuthology.orchestra.run.smithi124.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T10:35:41.308 INFO:teuthology.orchestra.run.smithi124.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T10:35:41.308 INFO:teuthology.orchestra.run.smithi124.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T10:35:41.308 INFO:teuthology.orchestra.run.smithi124.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T10:35:41.308 INFO:teuthology.orchestra.run.smithi124.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T10:35:41.308 INFO:teuthology.orchestra.run.smithi124.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T10:35:41.308 INFO:teuthology.orchestra.run.smithi124.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-06T10:35:41.309 INFO:teuthology.orchestra.run.smithi124.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T10:35:41.309 INFO:teuthology.orchestra.run.smithi124.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T10:35:41.309 INFO:teuthology.orchestra.run.smithi124.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T10:35:41.309 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:35:41.309 INFO:teuthology.orchestra.run.smithi124.stdout:Transaction Summary 2023-10-06T10:35:41.309 INFO:teuthology.orchestra.run.smithi124.stdout:====================================================================================================================== 2023-10-06T10:35:41.309 INFO:teuthology.orchestra.run.smithi124.stdout:Install 118 Packages 2023-10-06T10:35:41.309 INFO:teuthology.orchestra.run.smithi124.stdout:Upgrade 2 Packages 2023-10-06T10:35:41.309 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:35:41.310 INFO:teuthology.orchestra.run.smithi124.stdout:Total download size: 201 M 2023-10-06T10:35:41.310 INFO:teuthology.orchestra.run.smithi124.stdout:Downloading Packages: 2023-10-06T10:35:41.611 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2023-10-06T10:35:41.622 INFO:teuthology.orchestra.run.smithi092.stdout:====================================================================================================================== 2023-10-06T10:35:41.622 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2023-10-06T10:35:41.622 INFO:teuthology.orchestra.run.smithi092.stdout:====================================================================================================================== 2023-10-06T10:35:41.622 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2023-10-06T10:35:41.622 INFO:teuthology.orchestra.run.smithi092.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T10:35:41.622 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 6.4 k 2023-10-06T10:35:41.623 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.2 M 2023-10-06T10:35:41.623 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 877 k 2023-10-06T10:35:41.623 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 162 k 2023-10-06T10:35:41.623 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 1.6 M 2023-10-06T10:35:41.623 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 121 k 2023-10-06T10:35:41.623 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 1.7 M 2023-10-06T10:35:41.623 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 7.4 M 2023-10-06T10:35:41.623 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 47 k 2023-10-06T10:35:41.623 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T10:35:41.623 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 52 M 2023-10-06T10:35:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 267 k 2023-10-06T10:35:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 80 k 2023-10-06T10:35:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 255 k 2023-10-06T10:35:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 18 k 2023-10-06T10:35:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-06T10:35:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T10:35:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T10:35:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 25 k 2023-10-06T10:35:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 797 k 2023-10-06T10:35:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 130 k 2023-10-06T10:35:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T10:35:41.625 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-06T10:35:41.625 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T10:35:41.625 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 190 k 2023-10-06T10:35:41.625 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 371 k 2023-10-06T10:35:41.625 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 358 k 2023-10-06T10:35:41.625 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 107 k 2023-10-06T10:35:41.625 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 91 k 2023-10-06T10:35:41.625 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T10:35:41.625 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 173 k 2023-10-06T10:35:41.625 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-06T10:35:41.625 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2023-10-06T10:35:41.626 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.4 M 2023-10-06T10:35:41.626 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T10:35:41.626 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2023-10-06T10:35:41.626 INFO:teuthology.orchestra.run.smithi092.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T10:35:41.626 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 23 M 2023-10-06T10:35:41.626 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 23 k 2023-10-06T10:35:41.626 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 2.3 M 2023-10-06T10:35:41.626 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 238 k 2023-10-06T10:35:41.626 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.9 M 2023-10-06T10:35:41.627 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T10:35:41.627 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 14 k 2023-10-06T10:35:41.627 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 24 k 2023-10-06T10:35:41.627 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T10:35:41.627 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T10:35:41.627 INFO:teuthology.orchestra.run.smithi092.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-06T10:35:41.627 INFO:teuthology.orchestra.run.smithi092.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T10:35:41.627 INFO:teuthology.orchestra.run.smithi092.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T10:35:41.627 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 175 k 2023-10-06T10:35:41.627 INFO:teuthology.orchestra.run.smithi092.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-06T10:35:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-06T10:35:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T10:35:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-06T10:35:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T10:35:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 506 k 2023-10-06T10:35:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-06T10:35:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.6 M 2023-10-06T10:35:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T10:35:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-06T10:35:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T10:35:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T10:35:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T10:35:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T10:35:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-06T10:35:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T10:35:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T10:35:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T10:35:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T10:35:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T10:35:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T10:35:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T10:35:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T10:35:41.630 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T10:35:41.630 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 45 k 2023-10-06T10:35:41.630 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 115 k 2023-10-06T10:35:41.630 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T10:35:41.630 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T10:35:41.630 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T10:35:41.630 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T10:35:41.630 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T10:35:41.630 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T10:35:41.631 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T10:35:41.631 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T10:35:41.631 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T10:35:41.631 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T10:35:41.631 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T10:35:41.631 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T10:35:41.631 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T10:35:41.631 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T10:35:41.631 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T10:35:41.632 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T10:35:41.632 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T10:35:41.632 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T10:35:41.632 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T10:35:41.632 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T10:35:41.632 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T10:35:41.632 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T10:35:41.632 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T10:35:41.632 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T10:35:41.632 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T10:35:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T10:35:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T10:35:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T10:35:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T10:35:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-06T10:35:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T10:35:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T10:35:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T10:35:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T10:35:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T10:35:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T10:35:41.634 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T10:35:41.634 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T10:35:41.634 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T10:35:41.634 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T10:35:41.634 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-06T10:35:41.634 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T10:35:41.634 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T10:35:41.634 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T10:35:41.634 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:35:41.634 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2023-10-06T10:35:41.634 INFO:teuthology.orchestra.run.smithi092.stdout:====================================================================================================================== 2023-10-06T10:35:41.635 INFO:teuthology.orchestra.run.smithi092.stdout:Install 118 Packages 2023-10-06T10:35:41.635 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 2 Packages 2023-10-06T10:35:41.635 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:35:41.635 INFO:teuthology.orchestra.run.smithi124.stdout:(1/120): ceph-17.2.6-1274.g68e53a4a.el8.x86_64. 43 kB/s | 6.4 kB 00:00 2023-10-06T10:35:41.636 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 201 M 2023-10-06T10:35:41.636 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2023-10-06T10:35:42.052 INFO:teuthology.orchestra.run.smithi092.stdout:(1/120): ceph-17.2.6-1274.g68e53a4a.el8.x86_64. 24 kB/s | 6.4 kB 00:00 2023-10-06T10:35:42.480 INFO:teuthology.orchestra.run.smithi124.stdout:(2/120): ceph-fuse-17.2.6-1274.g68e53a4a.el8.x8 1.0 MB/s | 877 kB 00:00 2023-10-06T10:35:42.623 INFO:teuthology.orchestra.run.smithi124.stdout:(3/120): ceph-immutable-object-cache-17.2.6-127 1.1 MB/s | 162 kB 00:00 2023-10-06T10:35:42.860 INFO:teuthology.orchestra.run.smithi092.stdout:(2/120): ceph-fuse-17.2.6-1274.g68e53a4a.el8.x8 1.1 MB/s | 877 kB 00:00 2023-10-06T10:35:42.933 INFO:teuthology.orchestra.run.smithi092.stdout:(3/120): ceph-immutable-object-cache-17.2.6-127 2.2 MB/s | 162 kB 00:00 2023-10-06T10:35:43.015 INFO:teuthology.orchestra.run.smithi124.stdout:(4/120): ceph-common-17.2.6-1274.g68e53a4a.el8. 15 MB/s | 23 MB 00:01 2023-10-06T10:35:43.049 INFO:teuthology.orchestra.run.smithi092.stdout:(4/120): ceph-base-17.2.6-1274.g68e53a4a.el8.x8 4.1 MB/s | 5.2 MB 00:01 2023-10-06T10:35:43.189 INFO:teuthology.orchestra.run.smithi124.stdout:(5/120): ceph-mgr-17.2.6-1274.g68e53a4a.el8.x86 9.1 MB/s | 1.6 MB 00:00 2023-10-06T10:35:43.215 INFO:teuthology.orchestra.run.smithi092.stdout:(5/120): ceph-mgr-17.2.6-1274.g68e53a4a.el8.x86 9.5 MB/s | 1.6 MB 00:00 2023-10-06T10:35:43.369 INFO:teuthology.orchestra.run.smithi124.stdout:(6/120): ceph-base-17.2.6-1274.g68e53a4a.el8.x8 2.7 MB/s | 5.2 MB 00:01 2023-10-06T10:35:43.548 INFO:teuthology.orchestra.run.smithi124.stdout:(7/120): ceph-mon-17.2.6-1274.g68e53a4a.el8.x86 11 MB/s | 3.9 MB 00:00 2023-10-06T10:35:43.725 INFO:teuthology.orchestra.run.smithi092.stdout:(6/120): ceph-mon-17.2.6-1274.g68e53a4a.el8.x86 7.7 MB/s | 3.9 MB 00:00 2023-10-06T10:35:43.765 INFO:teuthology.orchestra.run.smithi124.stdout:(8/120): ceph-mds-17.2.6-1274.g68e53a4a.el8.x86 2.0 MB/s | 2.3 MB 00:01 2023-10-06T10:35:43.815 INFO:teuthology.orchestra.run.smithi092.stdout:(7/120): ceph-mds-17.2.6-1274.g68e53a4a.el8.x86 2.6 MB/s | 2.3 MB 00:00 2023-10-06T10:35:43.913 INFO:teuthology.orchestra.run.smithi124.stdout:(9/120): ceph-selinux-17.2.6-1274.g68e53a4a.el8 164 kB/s | 24 kB 00:00 2023-10-06T10:35:44.932 INFO:teuthology.orchestra.run.smithi124.stdout:(10/120): ceph-radosgw-17.2.6-1274.g68e53a4a.el 12 MB/s | 17 MB 00:01 2023-10-06T10:35:44.986 INFO:teuthology.orchestra.run.smithi124.stdout:(11/120): cephfs-mirror-17.2.6-1274.g68e53a4a.e 4.7 MB/s | 255 kB 00:00 2023-10-06T10:35:45.032 INFO:teuthology.orchestra.run.smithi124.stdout:(12/120): libcephfs-devel-17.2.6-1274.g68e53a4a 547 kB/s | 25 kB 00:00 2023-10-06T10:35:45.101 INFO:teuthology.orchestra.run.smithi124.stdout:(13/120): libcephfs2-17.2.6-1274.g68e53a4a.el8. 11 MB/s | 797 kB 00:00 2023-10-06T10:35:45.158 INFO:teuthology.orchestra.run.smithi124.stdout:(14/120): libcephsqlite-17.2.6-1274.g68e53a4a.e 3.0 MB/s | 175 kB 00:00 2023-10-06T10:35:45.209 INFO:teuthology.orchestra.run.smithi124.stdout:(15/120): librados-devel-17.2.6-1274.g68e53a4a. 2.5 MB/s | 130 kB 00:00 2023-10-06T10:35:45.274 INFO:teuthology.orchestra.run.smithi124.stdout:(16/120): libradosstriper1-17.2.6-1274.g68e53a4 7.7 MB/s | 506 kB 00:00 2023-10-06T10:35:45.923 INFO:teuthology.orchestra.run.smithi124.stdout:(17/120): librgw2-17.2.6-1274.g68e53a4a.el8.x86 8.6 MB/s | 5.6 MB 00:00 2023-10-06T10:35:45.973 INFO:teuthology.orchestra.run.smithi124.stdout:(18/120): python3-ceph-argparse-17.2.6-1274.g68 893 kB/s | 45 kB 00:00 2023-10-06T10:35:46.028 INFO:teuthology.orchestra.run.smithi124.stdout:(19/120): python3-ceph-common-17.2.6-1274.g68e5 2.0 MB/s | 115 kB 00:00 2023-10-06T10:35:46.090 INFO:teuthology.orchestra.run.smithi124.stdout:(20/120): python3-cephfs-17.2.6-1274.g68e53a4a. 3.0 MB/s | 190 kB 00:00 2023-10-06T10:35:46.156 INFO:teuthology.orchestra.run.smithi124.stdout:(21/120): python3-rados-17.2.6-1274.g68e53a4a.e 5.5 MB/s | 371 kB 00:00 2023-10-06T10:35:46.215 INFO:teuthology.orchestra.run.smithi124.stdout:(22/120): python3-rbd-17.2.6-1274.g68e53a4a.el8 6.0 MB/s | 358 kB 00:00 2023-10-06T10:35:46.237 INFO:teuthology.orchestra.run.smithi092.stdout:(8/120): ceph-osd-17.2.6-1274.g68e53a4a.el8.x86 6.7 MB/s | 17 MB 00:02 2023-10-06T10:35:46.263 INFO:teuthology.orchestra.run.smithi124.stdout:(23/120): python3-rgw-17.2.6-1274.g68e53a4a.el8 2.2 MB/s | 107 kB 00:00 2023-10-06T10:35:46.267 INFO:teuthology.orchestra.run.smithi092.stdout:(9/120): ceph-selinux-17.2.6-1274.g68e53a4a.el8 795 kB/s | 24 kB 00:00 2023-10-06T10:35:46.285 INFO:teuthology.orchestra.run.smithi124.stdout:(24/120): rbd-fuse-17.2.6-1274.g68e53a4a.el8.x8 3.9 MB/s | 91 kB 00:00 2023-10-06T10:35:46.382 INFO:teuthology.orchestra.run.smithi092.stdout:(10/120): ceph-radosgw-17.2.6-1274.g68e53a4a.el 6.5 MB/s | 17 MB 00:02 2023-10-06T10:35:46.463 INFO:teuthology.orchestra.run.smithi092.stdout:(11/120): cephfs-mirror-17.2.6-1274.g68e53a4a.e 3.1 MB/s | 255 kB 00:00 2023-10-06T10:35:46.532 INFO:teuthology.orchestra.run.smithi092.stdout:(12/120): libcephfs-devel-17.2.6-1274.g68e53a4a 361 kB/s | 25 kB 00:00 2023-10-06T10:35:46.569 INFO:teuthology.orchestra.run.smithi124.stdout:(25/120): rbd-mirror-17.2.6-1274.g68e53a4a.el8. 12 MB/s | 3.5 MB 00:00 2023-10-06T10:35:46.611 INFO:teuthology.orchestra.run.smithi092.stdout:(13/120): libcephfs2-17.2.6-1274.g68e53a4a.el8. 9.9 MB/s | 797 kB 00:00 2023-10-06T10:35:46.628 INFO:teuthology.orchestra.run.smithi124.stdout:(26/120): rbd-nbd-17.2.6-1274.g68e53a4a.el8.x86 2.9 MB/s | 173 kB 00:00 2023-10-06T10:35:46.663 INFO:teuthology.orchestra.run.smithi124.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1274.g 676 kB/s | 23 kB 00:00 2023-10-06T10:35:46.666 INFO:teuthology.orchestra.run.smithi092.stdout:(14/120): libcephsqlite-17.2.6-1274.g68e53a4a.e 3.1 MB/s | 175 kB 00:00 2023-10-06T10:35:46.693 INFO:teuthology.orchestra.run.smithi124.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1274.g68e53a4 3.9 MB/s | 121 kB 00:00 2023-10-06T10:35:46.715 INFO:teuthology.orchestra.run.smithi092.stdout:(15/120): librados-devel-17.2.6-1274.g68e53a4a. 2.6 MB/s | 130 kB 00:00 2023-10-06T10:35:46.799 INFO:teuthology.orchestra.run.smithi092.stdout:(16/120): libradosstriper1-17.2.6-1274.g68e53a4 6.0 MB/s | 506 kB 00:00 2023-10-06T10:35:46.973 INFO:teuthology.orchestra.run.smithi124.stdout:(29/120): ceph-mgr-dashboard-17.2.6-1274.g68e53 6.1 MB/s | 1.7 MB 00:00 2023-10-06T10:35:47.370 INFO:teuthology.orchestra.run.smithi092.stdout:(17/120): librgw2-17.2.6-1274.g68e53a4a.el8.x86 9.7 MB/s | 5.6 MB 00:00 2023-10-06T10:35:47.427 INFO:teuthology.orchestra.run.smithi092.stdout:(18/120): python3-ceph-argparse-17.2.6-1274.g68 774 kB/s | 45 kB 00:00 2023-10-06T10:35:47.490 INFO:teuthology.orchestra.run.smithi092.stdout:(19/120): python3-ceph-common-17.2.6-1274.g68e5 1.8 MB/s | 115 kB 00:00 2023-10-06T10:35:47.555 INFO:teuthology.orchestra.run.smithi092.stdout:(20/120): python3-cephfs-17.2.6-1274.g68e53a4a. 2.8 MB/s | 190 kB 00:00 2023-10-06T10:35:47.628 INFO:teuthology.orchestra.run.smithi092.stdout:(21/120): python3-rados-17.2.6-1274.g68e53a4a.e 4.9 MB/s | 371 kB 00:00 2023-10-06T10:35:47.703 INFO:teuthology.orchestra.run.smithi092.stdout:(22/120): python3-rbd-17.2.6-1274.g68e53a4a.el8 4.7 MB/s | 358 kB 00:00 2023-10-06T10:35:47.768 INFO:teuthology.orchestra.run.smithi092.stdout:(23/120): python3-rgw-17.2.6-1274.g68e53a4a.el8 1.6 MB/s | 107 kB 00:00 2023-10-06T10:35:47.833 INFO:teuthology.orchestra.run.smithi092.stdout:(24/120): rbd-fuse-17.2.6-1274.g68e53a4a.el8.x8 1.4 MB/s | 91 kB 00:00 2023-10-06T10:35:47.854 INFO:teuthology.orchestra.run.smithi124.stdout:(30/120): ceph-mgr-diskprediction-local-17.2.6- 8.4 MB/s | 7.4 MB 00:00 2023-10-06T10:35:47.916 INFO:teuthology.orchestra.run.smithi124.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1274.g68 3.7 MB/s | 238 kB 00:00 2023-10-06T10:35:47.973 INFO:teuthology.orchestra.run.smithi124.stdout:(32/120): ceph-mgr-rook-17.2.6-1274.g68e53a4a.e 833 kB/s | 47 kB 00:00 2023-10-06T10:35:48.026 INFO:teuthology.orchestra.run.smithi124.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1274.g6 275 kB/s | 14 kB 00:00 2023-10-06T10:35:48.088 INFO:teuthology.orchestra.run.smithi124.stdout:(34/120): ceph-volume-17.2.6-1274.g68e53a4a.el8 4.2 MB/s | 267 kB 00:00 2023-10-06T10:35:48.144 INFO:teuthology.orchestra.run.smithi124.stdout:(35/120): cephadm-17.2.6-1274.g68e53a4a.el8.noa 1.4 MB/s | 80 kB 00:00 2023-10-06T10:35:48.164 INFO:teuthology.orchestra.run.smithi092.stdout:(25/120): rbd-mirror-17.2.6-1274.g68e53a4a.el8. 10 MB/s | 3.5 MB 00:00 2023-10-06T10:35:48.175 INFO:teuthology.orchestra.run.smithi124.stdout:(36/120): cephfs-top-17.2.6-1274.g68e53a4a.el8. 579 kB/s | 18 kB 00:00 2023-10-06T10:35:48.216 INFO:teuthology.orchestra.run.smithi092.stdout:(26/120): rbd-nbd-17.2.6-1274.g68e53a4a.el8.x86 3.3 MB/s | 173 kB 00:00 2023-10-06T10:35:48.257 INFO:teuthology.orchestra.run.smithi092.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1274.g 557 kB/s | 23 kB 00:00 2023-10-06T10:35:48.303 INFO:teuthology.orchestra.run.smithi092.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1274.g68e53a4 2.6 MB/s | 121 kB 00:00 2023-10-06T10:35:48.474 INFO:teuthology.orchestra.run.smithi092.stdout:(29/120): ceph-mgr-dashboard-17.2.6-1274.g68e53 10 MB/s | 1.7 MB 00:00 2023-10-06T10:35:48.650 INFO:teuthology.orchestra.run.smithi124.stdout:(37/120): gperftools-libs-2.7-9.el8.x86_64.rpm 644 kB/s | 306 kB 00:00 2023-10-06T10:35:48.731 INFO:teuthology.orchestra.run.smithi124.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 735 kB/s | 59 kB 00:00 2023-10-06T10:35:48.811 INFO:teuthology.orchestra.run.smithi124.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 938 kB/s | 75 kB 00:00 2023-10-06T10:35:48.972 INFO:teuthology.orchestra.run.smithi124.stdout:(40/120): python3-asyncssh-2.7.0-2.el8.noarch.r 2.9 MB/s | 476 kB 00:00 2023-10-06T10:35:49.052 INFO:teuthology.orchestra.run.smithi124.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 546 kB/s | 44 kB 00:00 2023-10-06T10:35:49.103 INFO:teuthology.orchestra.run.smithi092.stdout:(30/120): ceph-mgr-diskprediction-local-17.2.6- 12 MB/s | 7.4 MB 00:00 2023-10-06T10:35:49.133 INFO:teuthology.orchestra.run.smithi124.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 2.2 MB/s | 185 kB 00:00 2023-10-06T10:35:49.157 INFO:teuthology.orchestra.run.smithi092.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1274.g68 4.3 MB/s | 238 kB 00:00 2023-10-06T10:35:49.204 INFO:teuthology.orchestra.run.smithi092.stdout:(32/120): ceph-mgr-rook-17.2.6-1274.g68e53a4a.e 1.0 MB/s | 47 kB 00:00 2023-10-06T10:35:49.213 INFO:teuthology.orchestra.run.smithi124.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 414 kB/s | 33 kB 00:00 2023-10-06T10:35:49.249 INFO:teuthology.orchestra.run.smithi092.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1274.g6 323 kB/s | 14 kB 00:00 2023-10-06T10:35:49.293 INFO:teuthology.orchestra.run.smithi124.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 205 kB/s | 16 kB 00:00 2023-10-06T10:35:49.304 INFO:teuthology.orchestra.run.smithi092.stdout:(34/120): ceph-volume-17.2.6-1274.g68e53a4a.el8 4.8 MB/s | 267 kB 00:00 2023-10-06T10:35:49.351 INFO:teuthology.orchestra.run.smithi092.stdout:(35/120): cephadm-17.2.6-1274.g68e53a4a.el8.noa 1.7 MB/s | 80 kB 00:00 2023-10-06T10:35:49.374 INFO:teuthology.orchestra.run.smithi124.stdout:(45/120): python3-cheroot-8.5.2-1.el8.noarch.rp 2.1 MB/s | 173 kB 00:00 2023-10-06T10:35:49.396 INFO:teuthology.orchestra.run.smithi092.stdout:(36/120): cephfs-top-17.2.6-1274.g68e53a4a.el8. 399 kB/s | 18 kB 00:00 2023-10-06T10:35:49.557 INFO:teuthology.orchestra.run.smithi124.stdout:(46/120): ceph-osd-17.2.6-1274.g68e53a4a.el8.x8 2.7 MB/s | 17 MB 00:06 2023-10-06T10:35:49.560 INFO:teuthology.orchestra.run.smithi124.stdout:(47/120): python3-cherrypy-18.4.0-1.el8.noarch. 2.0 MB/s | 384 kB 00:00 2023-10-06T10:35:49.640 INFO:teuthology.orchestra.run.smithi124.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 139 kB/s | 11 kB 00:00 2023-10-06T10:35:49.720 INFO:teuthology.orchestra.run.smithi124.stdout:(49/120): python3-jaraco-functools-2.0-4.el8.no 221 kB/s | 18 kB 00:00 2023-10-06T10:35:49.874 INFO:teuthology.orchestra.run.smithi124.stdout:(50/120): python3-google-auth-1.1.1-10.el8.noar 256 kB/s | 81 kB 00:00 2023-10-06T10:35:49.884 INFO:teuthology.orchestra.run.smithi124.stdout:(51/120): python3-kubernetes-11.0.0-6.el8.noarc 7.1 MB/s | 1.2 MB 00:00 2023-10-06T10:35:49.895 INFO:teuthology.orchestra.run.smithi092.stdout:(37/120): ceph-common-17.2.6-1274.g68e53a4a.el8 2.9 MB/s | 23 MB 00:08 2023-10-06T10:35:49.955 INFO:teuthology.orchestra.run.smithi124.stdout:(52/120): python3-logutils-0.3.5-11.el8.noarch. 610 kB/s | 49 kB 00:00 2023-10-06T10:35:49.964 INFO:teuthology.orchestra.run.smithi124.stdout:(53/120): python3-more-itertools-7.2.0-3.el8.no 734 kB/s | 59 kB 00:00 2023-10-06T10:35:50.034 INFO:teuthology.orchestra.run.smithi124.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 753 kB/s | 60 kB 00:00 2023-10-06T10:35:50.074 INFO:teuthology.orchestra.run.smithi124.stdout:(55/120): python3-pecan-1.3.2-9.el8.noarch.rpm 3.4 MB/s | 283 kB 00:00 2023-10-06T10:35:50.114 INFO:teuthology.orchestra.run.smithi124.stdout:(56/120): python3-portend-2.6-1.el8.noarch.rpm 196 kB/s | 16 kB 00:00 2023-10-06T10:35:50.122 INFO:teuthology.orchestra.run.smithi092.stdout:(38/120): gperftools-libs-2.7-9.el8.x86_64.rpm 422 kB/s | 306 kB 00:00 2023-10-06T10:35:50.126 INFO:teuthology.orchestra.run.smithi124.stdout:(57/120): python3-repoze-lru-0.7-6.el8.noarch.r 422 kB/s | 34 kB 00:00 2023-10-06T10:35:50.207 INFO:teuthology.orchestra.run.smithi124.stdout:(58/120): python3-rsa-4.9-2.el8.noarch.rpm 804 kB/s | 64 kB 00:00 2023-10-06T10:35:50.235 INFO:teuthology.orchestra.run.smithi092.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 666 kB/s | 75 kB 00:00 2023-10-06T10:35:50.273 INFO:teuthology.orchestra.run.smithi124.stdout:(59/120): python3-routes-2.4.1-12.el8.noarch.rp 1.2 MB/s | 196 kB 00:00 2023-10-06T10:35:50.286 INFO:teuthology.orchestra.run.smithi124.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 235 kB/s | 19 kB 00:00 2023-10-06T10:35:50.353 INFO:teuthology.orchestra.run.smithi124.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 300 kB/s | 24 kB 00:00 2023-10-06T10:35:50.367 INFO:teuthology.orchestra.run.smithi124.stdout:(62/120): python3-tempora-1.14.1-5.el8.noarch.r 366 kB/s | 29 kB 00:00 2023-10-06T10:35:50.392 INFO:teuthology.orchestra.run.smithi092.stdout:(40/120): liboath-2.6.2-3.el8.x86_64.rpm 118 kB/s | 59 kB 00:00 2023-10-06T10:35:50.433 INFO:teuthology.orchestra.run.smithi124.stdout:(63/120): python3-trustme-0.6.0-4.el8.noarch.rp 345 kB/s | 27 kB 00:00 2023-10-06T10:35:50.449 INFO:teuthology.orchestra.run.smithi124.stdout:(64/120): python3-waitress-1.4.3-1.el8.noarch.r 3.0 MB/s | 255 kB 00:00 2023-10-06T10:35:50.466 INFO:teuthology.orchestra.run.smithi092.stdout:(41/120): python3-asyncssh-2.7.0-2.el8.noarch.r 2.0 MB/s | 476 kB 00:00 2023-10-06T10:35:50.516 INFO:teuthology.orchestra.run.smithi124.stdout:(65/120): python3-webob-1.8.5-1.el8.1.noarch.rp 3.0 MB/s | 251 kB 00:00 2023-10-06T10:35:50.526 INFO:teuthology.orchestra.run.smithi092.stdout:(42/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 325 kB/s | 44 kB 00:00 2023-10-06T10:35:50.529 INFO:teuthology.orchestra.run.smithi124.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 766 kB/s | 61 kB 00:00 2023-10-06T10:35:50.581 INFO:teuthology.orchestra.run.smithi092.stdout:(43/120): python3-beautifulsoup4-4.6.3-2.el8.1. 1.6 MB/s | 185 kB 00:00 2023-10-06T10:35:50.596 INFO:teuthology.orchestra.run.smithi124.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 1.0 MB/s | 85 kB 00:00 2023-10-06T10:35:50.609 INFO:teuthology.orchestra.run.smithi124.stdout:(68/120): python3-zc-lockfile-2.0-2.el8.noarch. 289 kB/s | 23 kB 00:00 2023-10-06T10:35:50.622 INFO:teuthology.orchestra.run.smithi124.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 5.2 MB/s | 70 kB 00:00 2023-10-06T10:35:50.651 INFO:teuthology.orchestra.run.smithi092.stdout:(44/120): python3-cachetools-3.1.1-4.el8.noarch 264 kB/s | 33 kB 00:00 2023-10-06T10:35:50.653 INFO:teuthology.orchestra.run.smithi124.stdout:(70/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 598 kB/s | 19 kB 00:00 2023-10-06T10:35:50.668 INFO:teuthology.orchestra.run.smithi124.stdout:(71/120): python3-jwt-1.6.1-2.el8.noarch.rpm 3.0 MB/s | 43 kB 00:00 2023-10-06T10:35:50.680 INFO:teuthology.orchestra.run.smithi092.stdout:(45/120): python3-certifi-2018.10.15-7.el8.noar 165 kB/s | 16 kB 00:00 2023-10-06T10:35:50.683 INFO:teuthology.orchestra.run.smithi124.stdout:(72/120): python3-oauthlib-2.1.0-1.el8.noarch.r 10 MB/s | 155 kB 00:00 2023-10-06T10:35:50.698 INFO:teuthology.orchestra.run.smithi124.stdout:(73/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 14 MB/s | 220 kB 00:00 2023-10-06T10:35:50.712 INFO:teuthology.orchestra.run.smithi124.stdout:(74/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 3.3 MB/s | 48 kB 00:00 2023-10-06T10:35:50.727 INFO:teuthology.orchestra.run.smithi124.stdout:(75/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 15 MB/s | 229 kB 00:00 2023-10-06T10:35:50.740 INFO:teuthology.orchestra.run.smithi124.stdout:(76/120): python3-requests-oauthlib-1.0.0-1.el8 3.4 MB/s | 43 kB 00:00 2023-10-06T10:35:50.778 INFO:teuthology.orchestra.run.smithi124.stdout:(77/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 62 MB/s | 2.3 MB 00:00 2023-10-06T10:35:50.790 INFO:teuthology.orchestra.run.smithi124.stdout:(78/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 1.4 MB/s | 18 kB 00:00 2023-10-06T10:35:50.808 INFO:teuthology.orchestra.run.smithi124.stdout:(79/120): pcre2-devel-10.32-2.el8.x86_64.rpm 34 MB/s | 605 kB 00:00 2023-10-06T10:35:50.820 INFO:teuthology.orchestra.run.smithi124.stdout:(80/120): fuse-2.9.7-12.el8.x86_64.rpm 6.6 MB/s | 83 kB 00:00 2023-10-06T10:35:50.832 INFO:teuthology.orchestra.run.smithi124.stdout:(81/120): elfutils-libelf-devel-0.182-3.el8.x86 4.8 MB/s | 59 kB 00:00 2023-10-06T10:35:50.839 INFO:teuthology.orchestra.run.smithi124.stdout:(82/120): thrift-0.13.0-2.el8.x86_64.rpm 7.1 MB/s | 1.7 MB 00:00 2023-10-06T10:35:50.845 INFO:teuthology.orchestra.run.smithi124.stdout:(83/120): libquadmath-8.4.1-1.el8.x86_64.rpm 13 MB/s | 169 kB 00:00 2023-10-06T10:35:50.859 INFO:teuthology.orchestra.run.smithi124.stdout:(84/120): libsepol-devel-2.9-2.el8.x86_64.rpm 6.3 MB/s | 86 kB 00:00 2023-10-06T10:35:50.866 INFO:teuthology.orchestra.run.smithi124.stdout:(85/120): libcom_err-devel-1.45.6-1.el8.x86_64. 1.4 MB/s | 38 kB 00:00 2023-10-06T10:35:50.872 INFO:teuthology.orchestra.run.smithi124.stdout:(86/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 4.2 MB/s | 58 kB 00:00 2023-10-06T10:35:50.879 INFO:teuthology.orchestra.run.smithi124.stdout:(87/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.4 MB/s | 47 kB 00:00 2023-10-06T10:35:50.891 INFO:teuthology.orchestra.run.smithi124.stdout:(88/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 30 MB/s | 559 kB 00:00 2023-10-06T10:35:50.898 INFO:teuthology.orchestra.run.smithi124.stdout:(89/120): libgfortran-8.4.1-1.el8.x86_64.rpm 33 MB/s | 639 kB 00:00 2023-10-06T10:35:50.904 INFO:teuthology.orchestra.run.smithi124.stdout:(90/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 5.3 MB/s | 70 kB 00:00 2023-10-06T10:35:50.913 INFO:teuthology.orchestra.run.smithi124.stdout:(91/120): libselinux-devel-2.9-5.el8.x86_64.rpm 13 MB/s | 200 kB 00:00 2023-10-06T10:35:50.917 INFO:teuthology.orchestra.run.smithi124.stdout:(92/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 7.3 MB/s | 97 kB 00:00 2023-10-06T10:35:50.930 INFO:teuthology.orchestra.run.smithi124.stdout:(93/120): python3-prettytable-0.7.2-14.el8.noar 3.5 MB/s | 44 kB 00:00 2023-10-06T10:35:50.941 INFO:teuthology.orchestra.run.smithi124.stdout:(94/120): python3-jsonpatch-1.21-2.el8.noarch.r 2.2 MB/s | 27 kB 00:00 2023-10-06T10:35:50.955 INFO:teuthology.orchestra.run.smithi124.stdout:(95/120): jq-1.5-12.el8.x86_64.rpm 12 MB/s | 161 kB 00:00 2023-10-06T10:35:50.971 INFO:teuthology.orchestra.run.smithi124.stdout:(96/120): flex-2.6.1-9.el8.x86_64.rpm 19 MB/s | 320 kB 00:00 2023-10-06T10:35:50.989 INFO:teuthology.orchestra.run.smithi124.stdout:(97/120): python3-werkzeug-0.12.2-4.el8.noarch. 25 MB/s | 457 kB 00:00 2023-10-06T10:35:51.068 INFO:teuthology.orchestra.run.smithi124.stdout:(98/120): python3-babel-2.5.1-5.el8.noarch.rpm 62 MB/s | 4.8 MB 00:00 2023-10-06T10:35:51.080 INFO:teuthology.orchestra.run.smithi124.stdout:(99/120): python3-markupsafe-0.23-19.el8.x86_64 3.0 MB/s | 39 kB 00:00 2023-10-06T10:35:51.108 INFO:teuthology.orchestra.run.smithi092.stdout:(46/120): ceph-test-17.2.6-1274.g68e53a4a.el8.x 11 MB/s | 52 MB 00:04 2023-10-06T10:35:51.174 INFO:teuthology.orchestra.run.smithi124.stdout:(100/120): python3-scipy-1.0.0-20.module+el8.1. 56 MB/s | 15 MB 00:00 2023-10-06T10:35:51.182 INFO:teuthology.orchestra.run.smithi124.stdout:(101/120): bison-3.0.4-10.el8.x86_64.rpm 6.6 MB/s | 688 kB 00:00 2023-10-06T10:35:51.188 INFO:teuthology.orchestra.run.smithi124.stdout:(102/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 17 MB/s | 259 kB 00:00 2023-10-06T10:35:51.191 INFO:teuthology.orchestra.run.smithi092.stdout:(47/120): python3-cheroot-8.5.2-1.el8.noarch.rp 320 kB/s | 173 kB 00:00 2023-10-06T10:35:51.193 INFO:teuthology.orchestra.run.smithi092.stdout:(48/120): python3-cherrypy-18.4.0-1.el8.noarch. 749 kB/s | 384 kB 00:00 2023-10-06T10:35:51.197 INFO:teuthology.orchestra.run.smithi124.stdout:(103/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 15 MB/s | 225 kB 00:00 2023-10-06T10:35:51.203 INFO:teuthology.orchestra.run.smithi124.stdout:(104/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 8.8 MB/s | 126 kB 00:00 2023-10-06T10:35:51.218 INFO:teuthology.orchestra.run.smithi124.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 12 MB/s | 187 kB 00:00 2023-10-06T10:35:51.237 INFO:teuthology.orchestra.run.smithi124.stdout:(106/120): python3-jinja2-2.10.1-2.el8_0.noarch 29 MB/s | 538 kB 00:00 2023-10-06T10:35:51.255 INFO:teuthology.orchestra.run.smithi124.stdout:(107/120): python3-numpy-1.14.3-9.el8.x86_64.rp 65 MB/s | 3.7 MB 00:00 2023-10-06T10:35:51.259 INFO:teuthology.orchestra.run.smithi124.stdout:(108/120): socat-1.7.3.3-2.el8.x86_64.rpm 13 MB/s | 302 kB 00:00 2023-10-06T10:35:51.269 INFO:teuthology.orchestra.run.smithi124.stdout:(109/120): python3-mako-1.0.6-13.el8.noarch.rpm 12 MB/s | 157 kB 00:00 2023-10-06T10:35:51.272 INFO:teuthology.orchestra.run.smithi124.stdout:(110/120): python3-pyasn1-modules-0.3.7-6.el8.n 8.2 MB/s | 110 kB 00:00 2023-10-06T10:35:51.287 INFO:teuthology.orchestra.run.smithi124.stdout:(111/120): librdkafka-0.11.4-1.el8.x86_64.rpm 20 MB/s | 353 kB 00:00 2023-10-06T10:35:51.291 INFO:teuthology.orchestra.run.smithi092.stdout:(49/120): python3-jaraco-functools-2.0-4.el8.no 180 kB/s | 18 kB 00:00 2023-10-06T10:35:51.291 INFO:teuthology.orchestra.run.smithi124.stdout:(112/120): python3-jsonpointer-1.10-11.el8.noar 1.0 MB/s | 20 kB 00:00 2023-10-06T10:35:51.292 INFO:teuthology.orchestra.run.smithi092.stdout:(50/120): python3-jaraco-6.2-6.el8.noarch.rpm 110 kB/s | 11 kB 00:00 2023-10-06T10:35:51.305 INFO:teuthology.orchestra.run.smithi124.stdout:(113/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.8 MB/s | 54 kB 00:00 2023-10-06T10:35:51.310 INFO:teuthology.orchestra.run.smithi124.stdout:(114/120): libtool-2.4.6-25.el8.x86_64.rpm 30 MB/s | 709 kB 00:00 2023-10-06T10:35:51.382 INFO:teuthology.orchestra.run.smithi124.stdout:(115/120): openblas-threads-0.3.12-1.el8.x86_64 62 MB/s | 4.7 MB 00:00 2023-10-06T10:35:51.397 INFO:teuthology.orchestra.run.smithi124.stdout:(116/120): openblas-0.3.12-1.el8.x86_64.rpm 53 MB/s | 4.6 MB 00:00 2023-10-06T10:35:51.404 INFO:teuthology.orchestra.run.smithi124.stdout:(117/120): automake-1.16.1-7.el8.noarch.rpm 31 MB/s | 713 kB 00:00 2023-10-06T10:35:51.409 INFO:teuthology.orchestra.run.smithi092.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 418 kB/s | 49 kB 00:00 2023-10-06T10:35:51.490 INFO:teuthology.orchestra.run.smithi092.stdout:(52/120): python3-kubernetes-11.0.0-6.el8.noarc 5.9 MB/s | 1.2 MB 00:00 2023-10-06T10:35:51.529 INFO:teuthology.orchestra.run.smithi092.stdout:(53/120): python3-more-itertools-7.2.0-3.el8.no 491 kB/s | 59 kB 00:00 2023-10-06T10:35:51.604 INFO:teuthology.orchestra.run.smithi092.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 533 kB/s | 60 kB 00:00 2023-10-06T10:35:51.645 INFO:teuthology.orchestra.run.smithi092.stdout:(55/120): python3-google-auth-1.1.1-10.el8.noar 151 kB/s | 81 kB 00:00 2023-10-06T10:35:51.686 INFO:teuthology.orchestra.run.smithi124.stdout:(118/120): librbd1-17.2.6-1274.g68e53a4a.el8.x8 13 MB/s | 3.5 MB 00:00 2023-10-06T10:35:51.700 INFO:teuthology.orchestra.run.smithi092.stdout:(56/120): python3-portend-2.6-1.el8.noarch.rpm 162 kB/s | 16 kB 00:00 2023-10-06T10:35:51.740 INFO:teuthology.orchestra.run.smithi092.stdout:(57/120): python3-pecan-1.3.2-9.el8.noarch.rpm 1.3 MB/s | 283 kB 00:00 2023-10-06T10:35:51.766 INFO:teuthology.orchestra.run.smithi092.stdout:(58/120): python3-repoze-lru-0.7-6.el8.noarch.r 277 kB/s | 34 kB 00:00 2023-10-06T10:35:51.802 INFO:teuthology.orchestra.run.smithi092.stdout:(59/120): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2023-10-06T10:35:51.845 INFO:teuthology.orchestra.run.smithi092.stdout:(60/120): python3-rsa-4.9-2.el8.noarch.rpm 607 kB/s | 64 kB 00:00 2023-10-06T10:35:51.877 INFO:teuthology.orchestra.run.smithi092.stdout:(61/120): python3-simplegeneric-0.8.1-17.el8.no 168 kB/s | 19 kB 00:00 2023-10-06T10:35:51.899 INFO:teuthology.orchestra.run.smithi092.stdout:(62/120): python3-singledispatch-3.4.0.3-18.el8 247 kB/s | 24 kB 00:00 2023-10-06T10:35:51.946 INFO:teuthology.orchestra.run.smithi092.stdout:(63/120): python3-tempora-1.14.1-5.el8.noarch.r 290 kB/s | 29 kB 00:00 2023-10-06T10:35:51.995 INFO:teuthology.orchestra.run.smithi092.stdout:(64/120): python3-trustme-0.6.0-4.el8.noarch.rp 235 kB/s | 27 kB 00:00 2023-10-06T10:35:52.003 INFO:teuthology.orchestra.run.smithi092.stdout:(65/120): python3-waitress-1.4.3-1.el8.noarch.r 2.4 MB/s | 255 kB 00:00 2023-10-06T10:35:52.079 INFO:teuthology.orchestra.run.smithi092.stdout:(66/120): python3-webob-1.8.5-1.el8.1.noarch.rp 1.8 MB/s | 251 kB 00:00 2023-10-06T10:35:52.102 INFO:teuthology.orchestra.run.smithi092.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 868 kB/s | 85 kB 00:00 2023-10-06T10:35:52.137 INFO:teuthology.orchestra.run.smithi092.stdout:(68/120): python3-websocket-client-0.56.0-5.el8 430 kB/s | 61 kB 00:00 2023-10-06T10:35:52.142 INFO:teuthology.orchestra.run.smithi092.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 15 MB/s | 70 kB 00:00 2023-10-06T10:35:52.164 INFO:teuthology.orchestra.run.smithi124.stdout:(119/120): librados2-17.2.6-1274.g68e53a4a.el8. 4.4 MB/s | 3.4 MB 00:00 2023-10-06T10:35:52.172 INFO:teuthology.orchestra.run.smithi092.stdout:(70/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 618 kB/s | 19 kB 00:00 2023-10-06T10:35:52.180 INFO:teuthology.orchestra.run.smithi092.stdout:(71/120): python3-zc-lockfile-2.0-2.el8.noarch. 229 kB/s | 23 kB 00:00 2023-10-06T10:35:52.187 INFO:teuthology.orchestra.run.smithi092.stdout:(72/120): python3-jwt-1.6.1-2.el8.noarch.rpm 2.9 MB/s | 43 kB 00:00 2023-10-06T10:35:52.204 INFO:teuthology.orchestra.run.smithi092.stdout:(73/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 13 MB/s | 220 kB 00:00 2023-10-06T10:35:52.213 INFO:teuthology.orchestra.run.smithi092.stdout:(74/120): python3-oauthlib-2.1.0-1.el8.noarch.r 4.6 MB/s | 155 kB 00:00 2023-10-06T10:35:52.220 INFO:teuthology.orchestra.run.smithi092.stdout:(75/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 2.9 MB/s | 48 kB 00:00 2023-10-06T10:35:52.230 INFO:teuthology.orchestra.run.smithi092.stdout:(76/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 13 MB/s | 229 kB 00:00 2023-10-06T10:35:52.234 INFO:teuthology.orchestra.run.smithi092.stdout:(77/120): python3-requests-oauthlib-1.0.0-1.el8 3.2 MB/s | 43 kB 00:00 2023-10-06T10:35:52.248 INFO:teuthology.orchestra.run.smithi092.stdout:(78/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 1.3 MB/s | 18 kB 00:00 2023-10-06T10:35:52.267 INFO:teuthology.orchestra.run.smithi092.stdout:(79/120): thrift-0.13.0-2.el8.x86_64.rpm 10 MB/s | 1.7 MB 00:00 2023-10-06T10:35:52.276 INFO:teuthology.orchestra.run.smithi092.stdout:(80/120): pcre2-devel-10.32-2.el8.x86_64.rpm 21 MB/s | 605 kB 00:00 2023-10-06T10:35:52.292 INFO:teuthology.orchestra.run.smithi092.stdout:(81/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 38 MB/s | 2.3 MB 00:00 2023-10-06T10:35:52.297 INFO:teuthology.orchestra.run.smithi092.stdout:(82/120): elfutils-libelf-devel-0.182-3.el8.x86 3.7 MB/s | 59 kB 00:00 2023-10-06T10:35:52.301 INFO:teuthology.orchestra.run.smithi092.stdout:(83/120): fuse-2.9.7-12.el8.x86_64.rpm 2.4 MB/s | 83 kB 00:00 2023-10-06T10:35:52.310 INFO:teuthology.orchestra.run.smithi092.stdout:(84/120): libquadmath-8.4.1-1.el8.x86_64.rpm 9.3 MB/s | 169 kB 00:00 2023-10-06T10:35:52.314 INFO:teuthology.orchestra.run.smithi092.stdout:(85/120): libcom_err-devel-1.45.6-1.el8.x86_64. 2.3 MB/s | 38 kB 00:00 2023-10-06T10:35:52.321 INFO:teuthology.orchestra.run.smithi092.stdout:(86/120): libsepol-devel-2.9-2.el8.x86_64.rpm 4.2 MB/s | 86 kB 00:00 2023-10-06T10:35:52.326 INFO:teuthology.orchestra.run.smithi092.stdout:(87/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 3.7 MB/s | 58 kB 00:00 2023-10-06T10:35:52.334 INFO:teuthology.orchestra.run.smithi092.stdout:(88/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 2.3 MB/s | 47 kB 00:00 2023-10-06T10:35:52.341 INFO:teuthology.orchestra.run.smithi092.stdout:(89/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 28 MB/s | 559 kB 00:00 2023-10-06T10:35:52.348 INFO:teuthology.orchestra.run.smithi092.stdout:(90/120): libgfortran-8.4.1-1.el8.x86_64.rpm 28 MB/s | 639 kB 00:00 2023-10-06T10:35:52.350 INFO:teuthology.orchestra.run.smithi092.stdout:(91/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 4.5 MB/s | 70 kB 00:00 2023-10-06T10:35:52.360 INFO:teuthology.orchestra.run.smithi092.stdout:(92/120): libselinux-devel-2.9-5.el8.x86_64.rpm 10 MB/s | 200 kB 00:00 2023-10-06T10:35:52.364 INFO:teuthology.orchestra.run.smithi092.stdout:(93/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 6.4 MB/s | 97 kB 00:00 2023-10-06T10:35:52.374 INFO:teuthology.orchestra.run.smithi092.stdout:(94/120): python3-prettytable-0.7.2-14.el8.noar 3.1 MB/s | 44 kB 00:00 2023-10-06T10:35:52.378 INFO:teuthology.orchestra.run.smithi092.stdout:(95/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.8 MB/s | 27 kB 00:00 2023-10-06T10:35:52.389 INFO:teuthology.orchestra.run.smithi092.stdout:(96/120): jq-1.5-12.el8.x86_64.rpm 11 MB/s | 161 kB 00:00 2023-10-06T10:35:52.396 INFO:teuthology.orchestra.run.smithi092.stdout:(97/120): flex-2.6.1-9.el8.x86_64.rpm 18 MB/s | 320 kB 00:00 2023-10-06T10:35:52.407 INFO:teuthology.orchestra.run.smithi092.stdout:(98/120): python3-werkzeug-0.12.2-4.el8.noarch. 24 MB/s | 457 kB 00:00 2023-10-06T10:35:52.420 INFO:teuthology.orchestra.run.smithi092.stdout:(99/120): python3-markupsafe-0.23-19.el8.x86_64 3.1 MB/s | 39 kB 00:00 2023-10-06T10:35:52.443 INFO:teuthology.orchestra.run.smithi092.stdout:(100/120): bison-3.0.4-10.el8.x86_64.rpm 29 MB/s | 688 kB 00:00 2023-10-06T10:35:52.459 INFO:teuthology.orchestra.run.smithi092.stdout:(101/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 16 MB/s | 259 kB 00:00 2023-10-06T10:35:52.481 INFO:teuthology.orchestra.run.smithi092.stdout:(102/120): python3-babel-2.5.1-5.el8.noarch.rpm 56 MB/s | 4.8 MB 00:00 2023-10-06T10:35:52.484 INFO:teuthology.orchestra.run.smithi092.stdout:(103/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 8.9 MB/s | 225 kB 00:00 2023-10-06T10:35:52.496 INFO:teuthology.orchestra.run.smithi092.stdout:(104/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 8.1 MB/s | 126 kB 00:00 2023-10-06T10:35:52.512 INFO:teuthology.orchestra.run.smithi092.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 12 MB/s | 187 kB 00:00 2023-10-06T10:35:52.532 INFO:teuthology.orchestra.run.smithi092.stdout:(106/120): python3-jinja2-2.10.1-2.el8_0.noarch 27 MB/s | 538 kB 00:00 2023-10-06T10:35:52.554 INFO:teuthology.orchestra.run.smithi092.stdout:(107/120): python3-numpy-1.14.3-9.el8.x86_64.rp 54 MB/s | 3.7 MB 00:00 2023-10-06T10:35:52.558 INFO:teuthology.orchestra.run.smithi092.stdout:(108/120): socat-1.7.3.3-2.el8.x86_64.rpm 12 MB/s | 302 kB 00:00 2023-10-06T10:35:52.664 INFO:teuthology.orchestra.run.smithi092.stdout:(109/120): python3-scipy-1.0.0-20.module+el8.1. 46 MB/s | 15 MB 00:00 2023-10-06T10:35:52.665 INFO:teuthology.orchestra.run.smithi092.stdout:(110/120): python3-pyasn1-modules-0.3.7-6.el8.n 1.0 MB/s | 110 kB 00:00 2023-10-06T10:35:52.667 INFO:teuthology.orchestra.run.smithi092.stdout:(111/120): python3-mako-1.0.6-13.el8.noarch.rpm 1.4 MB/s | 157 kB 00:00 2023-10-06T10:35:52.681 INFO:teuthology.orchestra.run.smithi092.stdout:(112/120): librdkafka-0.11.4-1.el8.x86_64.rpm 20 MB/s | 353 kB 00:00 2023-10-06T10:35:52.681 INFO:teuthology.orchestra.run.smithi092.stdout:(113/120): python3-jsonpointer-1.10-11.el8.noar 1.2 MB/s | 20 kB 00:00 2023-10-06T10:35:52.689 INFO:teuthology.orchestra.run.smithi092.stdout:(114/120): libtool-2.4.6-25.el8.x86_64.rpm 31 MB/s | 709 kB 00:00 2023-10-06T10:35:52.697 INFO:teuthology.orchestra.run.smithi092.stdout:(115/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.4 MB/s | 54 kB 00:00 2023-10-06T10:35:52.723 INFO:teuthology.orchestra.run.smithi092.stdout:(116/120): automake-1.16.1-7.el8.noarch.rpm 26 MB/s | 713 kB 00:00 2023-10-06T10:35:52.762 INFO:teuthology.orchestra.run.smithi092.stdout:(117/120): openblas-threads-0.3.12-1.el8.x86_64 59 MB/s | 4.7 MB 00:00 2023-10-06T10:35:52.780 INFO:teuthology.orchestra.run.smithi092.stdout:(118/120): openblas-0.3.12-1.el8.x86_64.rpm 51 MB/s | 4.6 MB 00:00 2023-10-06T10:35:53.139 INFO:teuthology.orchestra.run.smithi092.stdout:(119/120): librbd1-17.2.6-1274.g68e53a4a.el8.x8 9.4 MB/s | 3.5 MB 00:00 2023-10-06T10:35:53.162 INFO:teuthology.orchestra.run.smithi092.stdout:(120/120): librados2-17.2.6-1274.g68e53a4a.el8. 7.8 MB/s | 3.4 MB 00:00 2023-10-06T10:35:53.167 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2023-10-06T10:35:53.167 INFO:teuthology.orchestra.run.smithi092.stdout:Total 17 MB/s | 201 MB 00:11 2023-10-06T10:35:53.214 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2023-10-06T10:35:53.326 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2023-10-06T10:35:53.326 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2023-10-06T10:35:55.000 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2023-10-06T10:35:55.004 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2023-10-06T10:35:56.057 INFO:teuthology.orchestra.run.smithi124.stdout:(120/120): ceph-test-17.2.6-1274.g68e53a4a.el8. 4.3 MB/s | 52 MB 00:12 2023-10-06T10:35:56.061 INFO:teuthology.orchestra.run.smithi124.stdout:-------------------------------------------------------------------------------- 2023-10-06T10:35:56.062 INFO:teuthology.orchestra.run.smithi124.stdout:Total 14 MB/s | 201 MB 00:14 2023-10-06T10:35:56.113 INFO:teuthology.orchestra.run.smithi124.stdout:Running transaction check 2023-10-06T10:35:56.228 INFO:teuthology.orchestra.run.smithi124.stdout:Transaction check succeeded. 2023-10-06T10:35:56.229 INFO:teuthology.orchestra.run.smithi124.stdout:Running transaction test 2023-10-06T10:35:57.929 INFO:teuthology.orchestra.run.smithi124.stdout:Transaction test succeeded. 2023-10-06T10:35:57.933 INFO:teuthology.orchestra.run.smithi124.stdout:Running transaction 2023-10-06T10:35:59.525 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2023-10-06T10:35:59.737 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-06T10:35:59.913 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-06T10:36:00.068 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-06T10:36:00.279 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-06T10:36:00.654 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-06T10:36:00.887 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/122 2023-10-06T10:36:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/122 2023-10-06T10:36:01.454 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 5/122 2023-10-06T10:36:01.569 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 5/122 2023-10-06T10:36:01.754 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T10:36:01.854 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T10:36:02.084 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T10:36:02.375 INFO:teuthology.orchestra.run.smithi124.stdout: Preparing : 1/1 2023-10-06T10:36:02.406 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-10-06T10:36:02.587 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-06T10:36:02.779 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-06T10:36:02.914 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-06T10:36:03.121 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-06T10:36:03.144 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-06T10:36:03.304 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T10:36:03.421 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T10:36:03.525 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-06T10:36:03.683 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-06T10:36:03.779 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/122 2023-10-06T10:36:03.860 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-06T10:36:04.037 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T10:36:04.129 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T10:36:04.173 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/122 2023-10-06T10:36:04.303 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 14/122 2023-10-06T10:36:04.365 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 14/122 2023-10-06T10:36:04.395 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 5/122 2023-10-06T10:36:04.527 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 5/122 2023-10-06T10:36:04.558 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-10-06T10:36:04.720 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T10:36:04.831 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T10:36:05.042 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T10:36:05.132 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-10-06T10:36:05.352 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-10-06T10:36:05.367 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-10-06T10:36:05.480 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-10-06T10:36:05.637 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-06T10:36:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-06T10:36:05.945 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-06T10:36:06.091 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-06T10:36:06.270 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T10:36:06.380 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T10:36:06.496 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-06T10:36:06.633 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-06T10:36:06.661 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-06T10:36:06.802 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-06T10:36:07.003 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T10:36:07.113 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T10:36:07.211 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-06T10:36:07.287 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 14/122 2023-10-06T10:36:07.367 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 14/122 2023-10-06T10:36:07.403 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-10-06T10:36:07.550 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-10-06T10:36:07.826 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-10-06T10:36:08.027 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-10-06T10:36:08.126 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-10-06T10:36:08.175 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-10-06T10:36:08.320 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-06T10:36:08.431 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-10-06T10:36:08.443 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-06T10:36:08.595 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-06T10:36:08.695 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-10-06T10:36:08.938 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-06T10:36:09.117 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 26/122 2023-10-06T10:36:09.127 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-06T10:36:09.246 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 27/122 2023-10-06T10:36:09.286 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-06T10:36:09.551 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 28/122 2023-10-06T10:36:09.774 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-10-06T10:36:09.826 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-06T10:36:10.076 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-10-06T10:36:10.086 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-10-06T10:36:10.777 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-06T10:36:10.809 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-10-06T10:36:11.056 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-10-06T10:36:11.312 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-10-06T10:36:11.556 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-06T10:36:11.718 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 26/122 2023-10-06T10:36:11.880 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 27/122 2023-10-06T10:36:12.127 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 28/122 2023-10-06T10:36:12.367 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-10-06T10:36:12.652 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-10-06T10:36:12.849 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-06T10:36:12.886 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-06T10:36:13.191 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-10-06T10:36:14.242 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-10-06T10:36:14.590 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-10-06T10:36:14.812 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-10-06T10:36:14.910 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-06T10:36:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-06T10:36:15.254 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-06T10:36:15.469 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 38/122 2023-10-06T10:36:15.634 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-10-06T10:36:15.676 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 39/122 2023-10-06T10:36:15.967 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 40/122 2023-10-06T10:36:16.208 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-10-06T10:36:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-10-06T10:36:16.630 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-10-06T10:36:16.705 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-10-06T10:36:16.845 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-10-06T10:36:17.039 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-10-06T10:36:17.050 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-06T10:36:17.113 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-06T10:36:17.284 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-10-06T10:36:17.322 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-10-06T10:36:17.437 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-10-06T10:36:17.617 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-06T10:36:17.632 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-10-06T10:36:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-10-06T10:36:17.830 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-06T10:36:17.958 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-10-06T10:36:18.046 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 38/122 2023-10-06T10:36:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-10-06T10:36:18.275 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 39/122 2023-10-06T10:36:18.288 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-10-06T10:36:18.509 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-10-06T10:36:18.536 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 40/122 2023-10-06T10:36:18.695 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-10-06T10:36:18.776 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-10-06T10:36:18.943 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-10-06T10:36:19.007 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-10-06T10:36:19.132 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-10-06T10:36:19.270 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-10-06T10:36:19.355 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-10-06T10:36:19.513 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-10-06T10:36:19.527 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-06T10:36:19.603 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-06T10:36:19.769 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-10-06T10:36:19.814 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-10-06T10:36:19.971 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-10-06T10:36:19.992 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-06T10:36:20.135 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-10-06T10:36:20.171 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-06T10:36:20.307 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-10-06T10:36:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-06T10:36:20.529 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-10-06T10:36:20.667 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-06T10:36:20.688 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-10-06T10:36:20.831 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-10-06T10:36:20.898 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-06T10:36:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-06T10:36:21.070 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-10-06T10:36:21.185 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-06T10:36:21.239 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-10-06T10:36:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-06T10:36:21.555 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-10-06T10:36:21.620 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-06T10:36:21.806 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-10-06T10:36:21.861 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-06T10:36:22.061 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-06T10:36:22.076 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-10-06T10:36:22.278 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-06T10:36:22.395 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-10-06T10:36:22.522 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-06T10:36:22.569 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-06T10:36:22.745 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-06T10:36:22.792 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-06T10:36:22.997 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-06T10:36:23.055 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-06T10:36:23.270 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-06T10:36:23.282 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-06T10:36:23.493 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-06T10:36:23.540 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-06T10:36:23.651 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-06T10:36:23.740 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-06T10:36:23.814 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-06T10:36:23.989 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-06T10:36:24.039 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-06T10:36:24.258 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-06T10:36:24.516 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-06T10:36:24.730 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-06T10:36:24.928 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-06T10:36:25.137 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-06T10:36:25.201 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-06T10:36:25.406 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-06T10:36:25.497 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-06T10:36:25.633 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-06T10:36:25.700 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-06T10:36:25.911 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-06T10:36:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-06T10:36:26.176 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-06T10:36:26.367 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-06T10:36:26.377 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-06T10:36:26.646 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-06T10:36:26.697 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-06T10:36:26.920 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-06T10:36:27.141 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-06T10:36:27.349 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-06T10:36:27.825 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-06T10:36:28.165 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-06T10:36:28.385 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-06T10:36:28.731 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-06T10:36:28.999 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-06T10:36:29.356 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-06T10:36:29.595 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-06T10:36:29.835 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-06T10:36:30.051 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-06T10:36:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T10:36:32.806 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T10:36:33.412 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T10:36:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 88/122 2023-10-06T10:36:33.820 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 88/122 2023-10-06T10:36:33.947 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 89/122 2023-10-06T10:36:35.150 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T10:36:35.425 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T10:36:36.003 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T10:36:36.954 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 88/122 2023-10-06T10:36:37.054 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 88/122 2023-10-06T10:36:37.233 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 89/122 2023-10-06T10:36:51.855 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 89/122 2023-10-06T10:36:51.855 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2023-10-06T10:36:51.855 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2023-10-06T10:36:51.855 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /dev 2023-10-06T10:36:51.855 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /run 2023-10-06T10:36:51.855 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2023-10-06T10:36:51.856 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2023-10-06T10:36:51.856 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2023-10-06T10:36:51.856 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2023-10-06T10:36:51.856 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /dev 2023-10-06T10:36:51.856 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:36:52.202 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 90/122 2023-10-06T10:36:53.682 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 90/122 2023-10-06T10:36:53.860 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 91/122 2023-10-06T10:36:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 91/122 2023-10-06T10:36:54.234 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 92/122 2023-10-06T10:36:54.612 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 89/122 2023-10-06T10:36:54.613 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /sys 2023-10-06T10:36:54.613 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /proc 2023-10-06T10:36:54.613 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /dev 2023-10-06T10:36:54.613 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /run 2023-10-06T10:36:54.613 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /mnt 2023-10-06T10:36:54.613 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /var/tmp 2023-10-06T10:36:54.613 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /home 2023-10-06T10:36:54.613 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /tmp 2023-10-06T10:36:54.613 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /dev 2023-10-06T10:36:54.613 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:36:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 92/122 2023-10-06T10:36:54.776 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 93/122 2023-10-06T10:36:54.793 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 93/122 2023-10-06T10:36:54.989 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 90/122 2023-10-06T10:36:56.476 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 90/122 2023-10-06T10:36:56.676 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 91/122 2023-10-06T10:36:56.897 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 91/122 2023-10-06T10:36:57.072 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 92/122 2023-10-06T10:36:57.470 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 92/122 2023-10-06T10:36:57.575 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 94/122 2023-10-06T10:36:57.656 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 93/122 2023-10-06T10:36:57.672 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 93/122 2023-10-06T10:36:57.745 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 94/122 2023-10-06T10:36:57.923 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 95/122 2023-10-06T10:36:58.156 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 96/122 2023-10-06T10:36:58.316 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 97/122 2023-10-06T10:36:58.530 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 97/122 2023-10-06T10:36:58.741 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 98/122 2023-10-06T10:36:59.709 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 98/122 2023-10-06T10:36:59.891 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 99/122 2023-10-06T10:36:59.990 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 99/122 2023-10-06T10:37:00.397 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 100/122 2023-10-06T10:37:00.507 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 94/122 2023-10-06T10:37:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/122 2023-10-06T10:37:00.620 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/122 2023-10-06T10:37:00.679 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 94/122 2023-10-06T10:37:00.816 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 102/122 2023-10-06T10:37:00.827 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 102/122 2023-10-06T10:37:00.870 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 95/122 2023-10-06T10:37:01.059 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 96/122 2023-10-06T10:37:01.238 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 97/122 2023-10-06T10:37:01.453 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 97/122 2023-10-06T10:37:01.692 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 98/122 2023-10-06T10:37:02.404 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 103/122 2023-10-06T10:37:02.625 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 104/122 2023-10-06T10:37:02.662 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 98/122 2023-10-06T10:37:02.714 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 104/122 2023-10-06T10:37:02.864 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 99/122 2023-10-06T10:37:02.874 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 105/122 2023-10-06T10:37:02.917 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1274.g68e53 105/122 2023-10-06T10:37:02.960 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 99/122 2023-10-06T10:37:03.058 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 106/122 2023-10-06T10:37:03.332 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 100/122 2023-10-06T10:37:03.405 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 106/122 2023-10-06T10:37:03.534 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/122 2023-10-06T10:37:03.571 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/122 2023-10-06T10:37:03.607 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 107/122 2023-10-06T10:37:03.729 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 102/122 2023-10-06T10:37:03.740 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 102/122 2023-10-06T10:37:05.334 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 103/122 2023-10-06T10:37:05.567 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 104/122 2023-10-06T10:37:05.645 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 104/122 2023-10-06T10:37:05.821 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 105/122 2023-10-06T10:37:05.869 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1274.g68e53 105/122 2023-10-06T10:37:06.021 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 106/122 2023-10-06T10:37:06.368 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 106/122 2023-10-06T10:37:06.554 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 107/122 2023-10-06T10:37:08.549 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 107/122 2023-10-06T10:37:09.476 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 108/122 2023-10-06T10:37:09.866 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-10-06T10:37:10.091 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 110/122 2023-10-06T10:37:10.339 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-10-06T10:37:10.522 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-10-06T10:37:10.533 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-10-06T10:37:10.676 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 113/122 2023-10-06T10:37:10.802 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-06T10:37:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 115/122 2023-10-06T10:37:11.155 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 116/122 2023-10-06T10:37:11.389 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 117/122 2023-10-06T10:37:11.509 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 107/122 2023-10-06T10:37:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-10-06T10:37:11.680 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-10-06T10:37:11.863 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-10-06T10:37:11.873 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-10-06T10:37:11.999 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-10-06T10:37:11.999 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T10:37:12.250 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T10:37:12.250 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T10:37:12.461 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 108/122 2023-10-06T10:37:12.863 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-10-06T10:37:13.101 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T10:37:13.102 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 1/122 2023-10-06T10:37:13.102 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2/122 2023-10-06T10:37:13.102 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 3/122 2023-10-06T10:37:13.102 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/122 2023-10-06T10:37:13.103 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 5/122 2023-10-06T10:37:13.103 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T10:37:13.103 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T10:37:13.103 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/122 2023-10-06T10:37:13.103 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 9/122 2023-10-06T10:37:13.104 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T10:37:13.104 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 11/122 2023-10-06T10:37:13.104 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 12/122 2023-10-06T10:37:13.104 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T10:37:13.105 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 14/122 2023-10-06T10:37:13.105 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 15/122 2023-10-06T10:37:13.105 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 16/122 2023-10-06T10:37:13.105 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 17/122 2023-10-06T10:37:13.105 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 18/122 2023-10-06T10:37:13.106 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 19/122 2023-10-06T10:37:13.106 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 20/122 2023-10-06T10:37:13.106 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 21/122 2023-10-06T10:37:13.106 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 22/122 2023-10-06T10:37:13.106 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 23/122 2023-10-06T10:37:13.107 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 24/122 2023-10-06T10:37:13.107 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 25/122 2023-10-06T10:37:13.107 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 26/122 2023-10-06T10:37:13.107 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 27/122 2023-10-06T10:37:13.107 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 28/122 2023-10-06T10:37:13.108 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 29/122 2023-10-06T10:37:13.108 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 30/122 2023-10-06T10:37:13.117 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 31/122 2023-10-06T10:37:13.117 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 32/122 2023-10-06T10:37:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 33/122 2023-10-06T10:37:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 34/122 2023-10-06T10:37:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 35/122 2023-10-06T10:37:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 36/122 2023-10-06T10:37:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 37/122 2023-10-06T10:37:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 38/122 2023-10-06T10:37:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-06T10:37:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-06T10:37:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-06T10:37:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-06T10:37:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-06T10:37:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-06T10:37:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-06T10:37:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-06T10:37:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-06T10:37:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-06T10:37:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-06T10:37:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-06T10:37:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-06T10:37:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-06T10:37:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-06T10:37:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-06T10:37:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-06T10:37:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-06T10:37:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-06T10:37:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-06T10:37:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-06T10:37:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-06T10:37:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-06T10:37:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-06T10:37:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-06T10:37:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-06T10:37:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-06T10:37:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-06T10:37:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-06T10:37:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-06T10:37:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-06T10:37:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-06T10:37:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-06T10:37:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-10-06T10:37:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-10-06T10:37:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-10-06T10:37:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-10-06T10:37:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-10-06T10:37:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-10-06T10:37:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-10-06T10:37:13.122 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-10-06T10:37:13.122 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-10-06T10:37:13.122 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-10-06T10:37:13.122 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-10-06T10:37:13.122 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-10-06T10:37:13.122 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-10-06T10:37:13.122 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-10-06T10:37:13.122 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-10-06T10:37:13.122 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-10-06T10:37:13.122 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-10-06T10:37:13.122 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-10-06T10:37:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-10-06T10:37:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-10-06T10:37:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-10-06T10:37:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-10-06T10:37:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-10-06T10:37:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-10-06T10:37:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-10-06T10:37:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-10-06T10:37:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-10-06T10:37:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-10-06T10:37:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-10-06T10:37:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-10-06T10:37:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-10-06T10:37:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-10-06T10:37:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-10-06T10:37:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-10-06T10:37:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-10-06T10:37:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-10-06T10:37:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-10-06T10:37:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-10-06T10:37:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-10-06T10:37:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-10-06T10:37:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-10-06T10:37:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-10-06T10:37:13.125 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-10-06T10:37:13.125 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-10-06T10:37:13.125 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-10-06T10:37:13.125 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-10-06T10:37:13.125 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-10-06T10:37:13.125 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 119/122 2023-10-06T10:37:13.125 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-06T10:37:13.125 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 121/122 2023-10-06T10:37:13.219 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 110/122 2023-10-06T10:37:13.451 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-10-06T10:37:13.603 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-10-06T10:37:13.614 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-10-06T10:37:13.824 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 113/122 2023-10-06T10:37:13.983 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-06T10:37:14.161 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 115/122 2023-10-06T10:37:14.377 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 116/122 2023-10-06T10:37:14.608 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 117/122 2023-10-06T10:37:14.811 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-10-06T10:37:14.891 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-10-06T10:37:15.070 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-10-06T10:37:15.078 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-10-06T10:37:15.205 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-10-06T10:37:15.205 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T10:37:15.263 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T10:37:15.265 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:42 ago on Fri 06 Oct 2023 10:35:33 AM UTC. 2023-10-06T10:37:15.414 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T10:37:15.414 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T10:37:16.318 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T10:37:16.319 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 1/122 2023-10-06T10:37:16.319 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2/122 2023-10-06T10:37:16.319 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 3/122 2023-10-06T10:37:16.319 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/122 2023-10-06T10:37:16.319 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 5/122 2023-10-06T10:37:16.319 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T10:37:16.319 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T10:37:16.319 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/122 2023-10-06T10:37:16.319 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 9/122 2023-10-06T10:37:16.319 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T10:37:16.319 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 11/122 2023-10-06T10:37:16.320 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 12/122 2023-10-06T10:37:16.320 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T10:37:16.320 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 14/122 2023-10-06T10:37:16.320 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 15/122 2023-10-06T10:37:16.320 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 16/122 2023-10-06T10:37:16.320 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 17/122 2023-10-06T10:37:16.320 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 18/122 2023-10-06T10:37:16.320 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 19/122 2023-10-06T10:37:16.320 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 20/122 2023-10-06T10:37:16.320 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 21/122 2023-10-06T10:37:16.321 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 22/122 2023-10-06T10:37:16.321 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 23/122 2023-10-06T10:37:16.321 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 24/122 2023-10-06T10:37:16.321 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 25/122 2023-10-06T10:37:16.321 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 26/122 2023-10-06T10:37:16.321 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 27/122 2023-10-06T10:37:16.321 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 28/122 2023-10-06T10:37:16.321 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 29/122 2023-10-06T10:37:16.321 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 30/122 2023-10-06T10:37:16.329 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 31/122 2023-10-06T10:37:16.329 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 32/122 2023-10-06T10:37:16.329 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 33/122 2023-10-06T10:37:16.329 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 34/122 2023-10-06T10:37:16.329 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 35/122 2023-10-06T10:37:16.329 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 36/122 2023-10-06T10:37:16.329 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 37/122 2023-10-06T10:37:16.330 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 38/122 2023-10-06T10:37:16.330 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-06T10:37:16.330 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-06T10:37:16.330 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-06T10:37:16.330 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-06T10:37:16.330 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-06T10:37:16.330 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-06T10:37:16.330 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-06T10:37:16.330 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-06T10:37:16.330 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-06T10:37:16.330 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-06T10:37:16.331 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-06T10:37:16.331 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-06T10:37:16.331 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-06T10:37:16.331 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-06T10:37:16.331 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-06T10:37:16.331 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-06T10:37:16.331 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-06T10:37:16.331 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-06T10:37:16.331 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-06T10:37:16.331 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-06T10:37:16.331 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-06T10:37:16.332 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-06T10:37:16.332 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-06T10:37:16.332 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-06T10:37:16.332 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-06T10:37:16.332 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-06T10:37:16.332 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-06T10:37:16.332 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-06T10:37:16.332 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-06T10:37:16.332 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-06T10:37:16.332 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-06T10:37:16.332 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-06T10:37:16.332 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-06T10:37:16.333 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-10-06T10:37:16.333 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-10-06T10:37:16.333 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-10-06T10:37:16.333 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-10-06T10:37:16.333 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-10-06T10:37:16.333 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-10-06T10:37:16.333 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-10-06T10:37:16.333 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-10-06T10:37:16.333 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-10-06T10:37:16.333 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-10-06T10:37:16.333 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-10-06T10:37:16.334 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-10-06T10:37:16.334 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-10-06T10:37:16.334 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-10-06T10:37:16.334 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-10-06T10:37:16.334 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-10-06T10:37:16.334 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-10-06T10:37:16.334 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-10-06T10:37:16.334 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-10-06T10:37:16.334 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-10-06T10:37:16.334 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-10-06T10:37:16.334 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-10-06T10:37:16.334 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-10-06T10:37:16.335 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-10-06T10:37:16.335 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-10-06T10:37:16.335 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-10-06T10:37:16.335 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-10-06T10:37:16.335 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-10-06T10:37:16.335 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-10-06T10:37:16.335 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-10-06T10:37:16.335 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-10-06T10:37:16.335 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-10-06T10:37:16.335 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-10-06T10:37:16.335 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-10-06T10:37:16.336 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-10-06T10:37:16.336 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-10-06T10:37:16.336 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-10-06T10:37:16.336 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-10-06T10:37:16.336 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-10-06T10:37:16.336 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-10-06T10:37:16.336 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-10-06T10:37:16.336 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-10-06T10:37:16.336 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-10-06T10:37:16.336 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-10-06T10:37:16.336 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-10-06T10:37:16.336 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-10-06T10:37:16.336 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-10-06T10:37:16.337 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 119/122 2023-10-06T10:37:16.337 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-06T10:37:16.337 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 121/122 2023-10-06T10:37:17.235 INFO:teuthology.orchestra.run.smithi092.stdout:Installed products updated. 2023-10-06T10:37:17.380 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:17.380 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2023-10-06T10:37:17.380 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.381 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.381 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2023-10-06T10:37:17.381 INFO:teuthology.orchestra.run.smithi092.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T10:37:17.381 INFO:teuthology.orchestra.run.smithi092.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T10:37:17.381 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.381 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.381 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.381 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.381 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T10:37:17.382 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.382 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.382 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.382 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T10:37:17.382 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T10:37:17.382 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T10:37:17.382 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T10:37:17.382 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T10:37:17.382 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.382 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.382 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T10:37:17.383 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.383 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.383 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.383 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T10:37:17.383 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T10:37:17.383 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.383 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T10:37:17.383 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-06T10:37:17.383 INFO:teuthology.orchestra.run.smithi092.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T10:37:17.383 INFO:teuthology.orchestra.run.smithi092.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-06T10:37:17.383 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T10:37:17.384 INFO:teuthology.orchestra.run.smithi092.stdout: jq-1.5-12.el8.x86_64 2023-10-06T10:37:17.384 INFO:teuthology.orchestra.run.smithi092.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-06T10:37:17.384 INFO:teuthology.orchestra.run.smithi092.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-06T10:37:17.384 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T10:37:17.384 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.384 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.384 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.384 INFO:teuthology.orchestra.run.smithi092.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-06T10:37:17.384 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-06T10:37:17.384 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T10:37:17.384 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-06T10:37:17.384 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T10:37:17.384 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.385 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.385 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-06T10:37:17.385 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.385 INFO:teuthology.orchestra.run.smithi092.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T10:37:17.385 INFO:teuthology.orchestra.run.smithi092.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-06T10:37:17.385 INFO:teuthology.orchestra.run.smithi092.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T10:37:17.385 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T10:37:17.385 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-06T10:37:17.385 INFO:teuthology.orchestra.run.smithi092.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T10:37:17.386 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T10:37:17.386 INFO:teuthology.orchestra.run.smithi092.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T10:37:17.386 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-06T10:37:17.386 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-06T10:37:17.386 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-06T10:37:17.386 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T10:37:17.386 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T10:37:17.386 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T10:37:17.386 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T10:37:17.386 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-06T10:37:17.386 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T10:37:17.387 INFO:teuthology.orchestra.run.smithi092.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T10:37:17.387 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T10:37:17.387 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.387 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.387 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.387 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T10:37:17.387 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T10:37:17.387 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T10:37:17.388 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T10:37:17.388 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T10:37:17.388 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T10:37:17.388 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-06T10:37:17.388 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T10:37:17.388 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T10:37:17.388 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T10:37:17.388 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T10:37:17.388 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T10:37:17.388 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T10:37:17.388 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T10:37:17.389 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T10:37:17.389 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T10:37:17.389 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-06T10:37:17.389 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-06T10:37:17.389 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T10:37:17.389 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T10:37:17.389 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T10:37:17.389 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T10:37:17.389 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T10:37:17.389 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T10:37:17.390 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T10:37:17.390 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.390 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.390 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T10:37:17.390 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T10:37:17.390 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.390 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T10:37:17.390 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T10:37:17.390 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-06T10:37:17.390 INFO:teuthology.orchestra.run.smithi092.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T10:37:17.391 INFO:teuthology.orchestra.run.smithi092.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T10:37:17.391 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T10:37:17.391 INFO:teuthology.orchestra.run.smithi092.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T10:37:17.391 INFO:teuthology.orchestra.run.smithi092.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T10:37:17.391 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T10:37:17.391 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T10:37:17.391 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T10:37:17.391 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T10:37:17.391 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T10:37:17.392 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.392 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.392 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:17.392 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-06T10:37:17.392 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T10:37:17.392 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-06T10:37:17.393 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T10:37:17.393 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-06T10:37:17.393 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:17.393 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2023-10-06T10:37:17.671 DEBUG:teuthology.parallel:result is None 2023-10-06T10:37:18.524 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T10:37:18.524 INFO:teuthology.orchestra.run.smithi124.stdout:Last metadata expiration check: 0:01:46 ago on Fri 06 Oct 2023 10:35:32 AM UTC. 2023-10-06T10:37:20.519 INFO:teuthology.orchestra.run.smithi124.stdout:Installed products updated. 2023-10-06T10:37:20.670 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:20.670 INFO:teuthology.orchestra.run.smithi124.stdout:Upgraded: 2023-10-06T10:37:20.670 INFO:teuthology.orchestra.run.smithi124.stdout: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.670 INFO:teuthology.orchestra.run.smithi124.stdout: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.670 INFO:teuthology.orchestra.run.smithi124.stdout:Installed: 2023-10-06T10:37:20.670 INFO:teuthology.orchestra.run.smithi124.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T10:37:20.670 INFO:teuthology.orchestra.run.smithi124.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T10:37:20.671 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.671 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.671 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.671 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.671 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T10:37:20.671 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-immutable-object-cache-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.671 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.671 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.671 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T10:37:20.671 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T10:37:20.671 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T10:37:20.671 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T10:37:20.672 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T10:37:20.672 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.672 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.672 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T10:37:20.672 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.672 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.672 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.672 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T10:37:20.672 INFO:teuthology.orchestra.run.smithi124.stdout: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T10:37:20.672 INFO:teuthology.orchestra.run.smithi124.stdout: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.673 INFO:teuthology.orchestra.run.smithi124.stdout: cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T10:37:20.673 INFO:teuthology.orchestra.run.smithi124.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-06T10:37:20.673 INFO:teuthology.orchestra.run.smithi124.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T10:37:20.673 INFO:teuthology.orchestra.run.smithi124.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-06T10:37:20.673 INFO:teuthology.orchestra.run.smithi124.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T10:37:20.673 INFO:teuthology.orchestra.run.smithi124.stdout: jq-1.5-12.el8.x86_64 2023-10-06T10:37:20.673 INFO:teuthology.orchestra.run.smithi124.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-06T10:37:20.673 INFO:teuthology.orchestra.run.smithi124.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-06T10:37:20.673 INFO:teuthology.orchestra.run.smithi124.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T10:37:20.673 INFO:teuthology.orchestra.run.smithi124.stdout: libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.673 INFO:teuthology.orchestra.run.smithi124.stdout: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.673 INFO:teuthology.orchestra.run.smithi124.stdout: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.674 INFO:teuthology.orchestra.run.smithi124.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-06T10:37:20.674 INFO:teuthology.orchestra.run.smithi124.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-06T10:37:20.674 INFO:teuthology.orchestra.run.smithi124.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T10:37:20.674 INFO:teuthology.orchestra.run.smithi124.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-06T10:37:20.674 INFO:teuthology.orchestra.run.smithi124.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T10:37:20.674 INFO:teuthology.orchestra.run.smithi124.stdout: librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.674 INFO:teuthology.orchestra.run.smithi124.stdout: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.674 INFO:teuthology.orchestra.run.smithi124.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-06T10:37:20.674 INFO:teuthology.orchestra.run.smithi124.stdout: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.674 INFO:teuthology.orchestra.run.smithi124.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T10:37:20.674 INFO:teuthology.orchestra.run.smithi124.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-06T10:37:20.675 INFO:teuthology.orchestra.run.smithi124.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T10:37:20.675 INFO:teuthology.orchestra.run.smithi124.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T10:37:20.675 INFO:teuthology.orchestra.run.smithi124.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-06T10:37:20.675 INFO:teuthology.orchestra.run.smithi124.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T10:37:20.675 INFO:teuthology.orchestra.run.smithi124.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T10:37:20.675 INFO:teuthology.orchestra.run.smithi124.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T10:37:20.675 INFO:teuthology.orchestra.run.smithi124.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-06T10:37:20.675 INFO:teuthology.orchestra.run.smithi124.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-06T10:37:20.675 INFO:teuthology.orchestra.run.smithi124.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-06T10:37:20.675 INFO:teuthology.orchestra.run.smithi124.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T10:37:20.675 INFO:teuthology.orchestra.run.smithi124.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T10:37:20.675 INFO:teuthology.orchestra.run.smithi124.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T10:37:20.676 INFO:teuthology.orchestra.run.smithi124.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T10:37:20.676 INFO:teuthology.orchestra.run.smithi124.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-06T10:37:20.676 INFO:teuthology.orchestra.run.smithi124.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T10:37:20.676 INFO:teuthology.orchestra.run.smithi124.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T10:37:20.676 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T10:37:20.676 INFO:teuthology.orchestra.run.smithi124.stdout: python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.676 INFO:teuthology.orchestra.run.smithi124.stdout: python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.676 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.676 INFO:teuthology.orchestra.run.smithi124.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T10:37:20.676 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T10:37:20.676 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T10:37:20.676 INFO:teuthology.orchestra.run.smithi124.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T10:37:20.677 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T10:37:20.677 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T10:37:20.677 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-06T10:37:20.677 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T10:37:20.677 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T10:37:20.677 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T10:37:20.677 INFO:teuthology.orchestra.run.smithi124.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T10:37:20.677 INFO:teuthology.orchestra.run.smithi124.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T10:37:20.677 INFO:teuthology.orchestra.run.smithi124.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T10:37:20.677 INFO:teuthology.orchestra.run.smithi124.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T10:37:20.677 INFO:teuthology.orchestra.run.smithi124.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T10:37:20.677 INFO:teuthology.orchestra.run.smithi124.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T10:37:20.678 INFO:teuthology.orchestra.run.smithi124.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-06T10:37:20.678 INFO:teuthology.orchestra.run.smithi124.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-06T10:37:20.678 INFO:teuthology.orchestra.run.smithi124.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T10:37:20.678 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T10:37:20.678 INFO:teuthology.orchestra.run.smithi124.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T10:37:20.678 INFO:teuthology.orchestra.run.smithi124.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T10:37:20.678 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T10:37:20.678 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T10:37:20.678 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T10:37:20.678 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.678 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.679 INFO:teuthology.orchestra.run.smithi124.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T10:37:20.679 INFO:teuthology.orchestra.run.smithi124.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T10:37:20.679 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.679 INFO:teuthology.orchestra.run.smithi124.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T10:37:20.679 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T10:37:20.679 INFO:teuthology.orchestra.run.smithi124.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-06T10:37:20.679 INFO:teuthology.orchestra.run.smithi124.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T10:37:20.679 INFO:teuthology.orchestra.run.smithi124.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T10:37:20.679 INFO:teuthology.orchestra.run.smithi124.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T10:37:20.679 INFO:teuthology.orchestra.run.smithi124.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T10:37:20.679 INFO:teuthology.orchestra.run.smithi124.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T10:37:20.679 INFO:teuthology.orchestra.run.smithi124.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T10:37:20.680 INFO:teuthology.orchestra.run.smithi124.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T10:37:20.680 INFO:teuthology.orchestra.run.smithi124.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T10:37:20.680 INFO:teuthology.orchestra.run.smithi124.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T10:37:20.680 INFO:teuthology.orchestra.run.smithi124.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T10:37:20.680 INFO:teuthology.orchestra.run.smithi124.stdout: rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.680 INFO:teuthology.orchestra.run.smithi124.stdout: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.680 INFO:teuthology.orchestra.run.smithi124.stdout: rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T10:37:20.680 INFO:teuthology.orchestra.run.smithi124.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-06T10:37:20.680 INFO:teuthology.orchestra.run.smithi124.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T10:37:20.680 INFO:teuthology.orchestra.run.smithi124.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-06T10:37:20.680 INFO:teuthology.orchestra.run.smithi124.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T10:37:20.681 INFO:teuthology.orchestra.run.smithi124.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-06T10:37:20.681 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:20.681 INFO:teuthology.orchestra.run.smithi124.stdout:Complete! 2023-10-06T10:37:20.931 DEBUG:teuthology.parallel:result is None 2023-10-06T10:37:20.932 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T10:37:21.078 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T10:37:21.100 INFO:teuthology.orchestra.run.smithi092.stdout:17.2.6-1274.g68e53a4a.el8 2023-10-06T10:37:21.101 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1274.g68e53a4a.el8 2023-10-06T10:37:21.101 INFO:teuthology.task.install:The correct ceph version 17.2.6-1274.g68e53a4a is installed. 2023-10-06T10:37:21.101 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T10:37:21.319 DEBUG:teuthology.orchestra.run.smithi124:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T10:37:21.340 INFO:teuthology.orchestra.run.smithi124.stdout:17.2.6-1274.g68e53a4a.el8 2023-10-06T10:37:21.341 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1274.g68e53a4a.el8 2023-10-06T10:37:21.341 INFO:teuthology.task.install:The correct ceph version 17.2.6-1274.g68e53a4a is installed. 2023-10-06T10:37:21.341 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-06T10:37:21.341 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:21.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T10:37:21.380 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:37:21.380 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T10:37:21.419 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-06T10:37:21.419 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:21.419 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T10:37:21.445 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T10:37:21.510 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:37:21.510 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T10:37:21.534 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T10:37:21.599 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-06T10:37:21.599 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:21.599 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T10:37:21.623 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T10:37:21.687 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:37:21.687 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T10:37:21.711 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T10:37:21.775 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-06T10:37:21.775 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:21.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T10:37:21.799 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T10:37:21.863 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:37:21.864 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T10:37:21.887 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T10:37:21.951 INFO:teuthology.run_tasks:Running task ceph... 2023-10-06T10:37:22.015 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-06T10:37:22.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /var/log/ceph 2023-10-06T10:37:22.017 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 777 /var/log/ceph 2023-10-06T10:37:22.040 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-06T10:37:22.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T10:37:22.082 DEBUG:teuthology.orchestra.run.smithi124:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T10:37:22.105 INFO:tasks.ceph:Creating extra log directories... 2023-10-06T10:37:22.106 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T10:37:22.146 DEBUG:teuthology.orchestra.run.smithi124:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T10:37:22.176 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-06T10:37:22.176 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-06T10:37:22.177 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-06T10:37:22.177 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:22.177 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T10:37:22.223 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T10:37:22.333 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T10:37:22.357 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:37:22.358 DEBUG:teuthology.orchestra.run.smithi124:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T10:37:22.372 DEBUG:teuthology.orchestra.run.smithi124:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T10:37:22.484 DEBUG:teuthology.orchestra.run.smithi124:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T10:37:22.514 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-06T10:37:22.515 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Scrub error on inode'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-06T10:37:22.515 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412360', 'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/multimds_misc}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7412360', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Scrub error on inode'], 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 4002, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'targets': {'smithi092.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF53OE4i5vcG3W2Q2lwwvr0uEhnNAo2G09ujsL4/0+SF1JifRhTX0X3iB1Zpi5s16tvOQdqI//q3UQa2B/6tRpQ=', 'smithi124.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ/sH53ZqpmGXnu83Xkf3Q5s6Bg6EwuiibrtBaWjsLHpp52l0b1kwbr7opnFzmdmy/p6yF4gbYp7YLTzdj6jNUo='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_multimds_misc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-05_23:54:57', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-06T10:37:22.515 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T10:37:22.517 DEBUG:teuthology.orchestra.run.smithi124:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T10:37:22.564 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T10:37:22.572 DEBUG:teuthology.orchestra.run.smithi124:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T10:37:22.628 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:22.628 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T10:37:22.648 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T10:37:22.648 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2023-10-06T10:37:22.703 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T10:37:22.769 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T10:37:22.769 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 219360 Links: 1 2023-10-06T10:37:22.769 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T10:37:22.769 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T10:37:22.769 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-10-06 10:37:12.699823294 +0000 2023-10-06T10:37:22.769 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-10-06 10:34:48.036118598 +0000 2023-10-06T10:37:22.769 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-10-06 10:34:48.036118598 +0000 2023-10-06T10:37:22.770 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2023-10-06T10:37:22.770 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T10:37:22.795 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-10-06T10:37:22.795 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-10-06T10:37:22.795 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000298176 s, 1.7 MB/s 2023-10-06T10:37:22.796 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T10:37:22.854 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2023-10-06T10:37:22.908 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T10:37:22.908 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T10:37:22.908 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 218614 Links: 1 2023-10-06T10:37:22.908 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T10:37:22.908 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T10:37:22.909 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-10-06 10:37:12.699823294 +0000 2023-10-06T10:37:22.909 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-10-06 10:34:48.527104202 +0000 2023-10-06T10:37:22.909 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-10-06 10:34:48.527104202 +0000 2023-10-06T10:37:22.909 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2023-10-06T10:37:22.909 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T10:37:22.971 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-10-06T10:37:22.971 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-10-06T10:37:22.971 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000251225 s, 2.0 MB/s 2023-10-06T10:37:22.972 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T10:37:23.028 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2023-10-06T10:37:23.082 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T10:37:23.082 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T10:37:23.082 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 223391 Links: 1 2023-10-06T10:37:23.083 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T10:37:23.083 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T10:37:23.083 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-10-06 10:37:12.699823294 +0000 2023-10-06T10:37:23.083 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-10-06 10:34:48.995090481 +0000 2023-10-06T10:37:23.083 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-10-06 10:34:48.995090481 +0000 2023-10-06T10:37:23.083 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2023-10-06T10:37:23.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T10:37:23.145 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-10-06T10:37:23.145 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-10-06T10:37:23.146 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000244808 s, 2.1 MB/s 2023-10-06T10:37:23.147 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T10:37:23.203 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2023-10-06T10:37:23.258 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T10:37:23.259 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T10:37:23.259 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 221424 Links: 1 2023-10-06T10:37:23.259 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T10:37:23.259 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T10:37:23.259 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-10-06 10:37:12.699823294 +0000 2023-10-06T10:37:23.259 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-10-06 10:34:49.470076554 +0000 2023-10-06T10:37:23.259 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-10-06 10:34:49.470076554 +0000 2023-10-06T10:37:23.259 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2023-10-06T10:37:23.259 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T10:37:23.321 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-10-06T10:37:23.321 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-10-06T10:37:23.321 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000207882 s, 2.5 MB/s 2023-10-06T10:37:23.322 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T10:37:23.378 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T10:37:23.378 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:37:23.378 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T10:37:23.395 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T10:37:23.395 DEBUG:teuthology.orchestra.run.smithi124:> stat /dev/vg_nvme/lv_1 2023-10-06T10:37:23.455 INFO:teuthology.orchestra.run.smithi124.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T10:37:23.456 INFO:teuthology.orchestra.run.smithi124.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T10:37:23.456 INFO:teuthology.orchestra.run.smithi124.stdout:Device: 6h/6d Inode: 215790 Links: 1 2023-10-06T10:37:23.456 INFO:teuthology.orchestra.run.smithi124.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T10:37:23.456 INFO:teuthology.orchestra.run.smithi124.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T10:37:23.456 INFO:teuthology.orchestra.run.smithi124.stdout:Access: 2023-10-06 10:37:15.884205349 +0000 2023-10-06T10:37:23.456 INFO:teuthology.orchestra.run.smithi124.stdout:Modify: 2023-10-06 10:34:48.040934658 +0000 2023-10-06T10:37:23.456 INFO:teuthology.orchestra.run.smithi124.stdout:Change: 2023-10-06 10:34:48.040934658 +0000 2023-10-06T10:37:23.456 INFO:teuthology.orchestra.run.smithi124.stdout: Birth: - 2023-10-06T10:37:23.457 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T10:37:23.519 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records in 2023-10-06T10:37:23.520 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records out 2023-10-06T10:37:23.520 INFO:teuthology.orchestra.run.smithi124.stderr:512 bytes copied, 0.000261996 s, 2.0 MB/s 2023-10-06T10:37:23.521 DEBUG:teuthology.orchestra.run.smithi124:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T10:37:23.578 DEBUG:teuthology.orchestra.run.smithi124:> stat /dev/vg_nvme/lv_2 2023-10-06T10:37:23.634 INFO:teuthology.orchestra.run.smithi124.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T10:37:23.634 INFO:teuthology.orchestra.run.smithi124.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T10:37:23.635 INFO:teuthology.orchestra.run.smithi124.stdout:Device: 6h/6d Inode: 214902 Links: 1 2023-10-06T10:37:23.635 INFO:teuthology.orchestra.run.smithi124.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T10:37:23.635 INFO:teuthology.orchestra.run.smithi124.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T10:37:23.635 INFO:teuthology.orchestra.run.smithi124.stdout:Access: 2023-10-06 10:37:15.884205349 +0000 2023-10-06T10:37:23.635 INFO:teuthology.orchestra.run.smithi124.stdout:Modify: 2023-10-06 10:34:48.541926384 +0000 2023-10-06T10:37:23.635 INFO:teuthology.orchestra.run.smithi124.stdout:Change: 2023-10-06 10:34:48.541926384 +0000 2023-10-06T10:37:23.635 INFO:teuthology.orchestra.run.smithi124.stdout: Birth: - 2023-10-06T10:37:23.635 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T10:37:23.700 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records in 2023-10-06T10:37:23.700 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records out 2023-10-06T10:37:23.701 INFO:teuthology.orchestra.run.smithi124.stderr:512 bytes copied, 0.00182128 s, 281 kB/s 2023-10-06T10:37:23.701 DEBUG:teuthology.orchestra.run.smithi124:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T10:37:23.758 DEBUG:teuthology.orchestra.run.smithi124:> stat /dev/vg_nvme/lv_3 2023-10-06T10:37:23.813 INFO:teuthology.orchestra.run.smithi124.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T10:37:23.813 INFO:teuthology.orchestra.run.smithi124.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T10:37:23.813 INFO:teuthology.orchestra.run.smithi124.stdout:Device: 6h/6d Inode: 220515 Links: 1 2023-10-06T10:37:23.814 INFO:teuthology.orchestra.run.smithi124.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T10:37:23.814 INFO:teuthology.orchestra.run.smithi124.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T10:37:23.814 INFO:teuthology.orchestra.run.smithi124.stdout:Access: 2023-10-06 10:37:15.884205349 +0000 2023-10-06T10:37:23.814 INFO:teuthology.orchestra.run.smithi124.stdout:Modify: 2023-10-06 10:34:49.033918259 +0000 2023-10-06T10:37:23.814 INFO:teuthology.orchestra.run.smithi124.stdout:Change: 2023-10-06 10:34:49.033918259 +0000 2023-10-06T10:37:23.814 INFO:teuthology.orchestra.run.smithi124.stdout: Birth: - 2023-10-06T10:37:23.814 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T10:37:23.877 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records in 2023-10-06T10:37:23.877 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records out 2023-10-06T10:37:23.877 INFO:teuthology.orchestra.run.smithi124.stderr:512 bytes copied, 0.000255771 s, 2.0 MB/s 2023-10-06T10:37:23.878 DEBUG:teuthology.orchestra.run.smithi124:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T10:37:23.935 DEBUG:teuthology.orchestra.run.smithi124:> stat /dev/vg_nvme/lv_4 2023-10-06T10:37:23.990 INFO:teuthology.orchestra.run.smithi124.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T10:37:23.990 INFO:teuthology.orchestra.run.smithi124.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T10:37:23.990 INFO:teuthology.orchestra.run.smithi124.stdout:Device: 6h/6d Inode: 220543 Links: 1 2023-10-06T10:37:23.990 INFO:teuthology.orchestra.run.smithi124.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T10:37:23.990 INFO:teuthology.orchestra.run.smithi124.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T10:37:23.990 INFO:teuthology.orchestra.run.smithi124.stdout:Access: 2023-10-06 10:37:15.884205349 +0000 2023-10-06T10:37:23.990 INFO:teuthology.orchestra.run.smithi124.stdout:Modify: 2023-10-06 10:34:49.517910265 +0000 2023-10-06T10:37:23.990 INFO:teuthology.orchestra.run.smithi124.stdout:Change: 2023-10-06 10:34:49.517910265 +0000 2023-10-06T10:37:23.990 INFO:teuthology.orchestra.run.smithi124.stdout: Birth: - 2023-10-06T10:37:23.991 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T10:37:24.053 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records in 2023-10-06T10:37:24.053 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records out 2023-10-06T10:37:24.054 INFO:teuthology.orchestra.run.smithi124.stderr:512 bytes copied, 0.000196417 s, 2.6 MB/s 2023-10-06T10:37:24.054 DEBUG:teuthology.orchestra.run.smithi124:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T10:37:24.110 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T10:37:24.110 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi124.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T10:37:24.111 INFO:tasks.ceph:Generating config... 2023-10-06T10:37:24.112 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-06T10:37:24.112 INFO:tasks.ceph:[client] debug client = 20 2023-10-06T10:37:24.112 INFO:tasks.ceph:[client] debug ms = 1 2023-10-06T10:37:24.112 INFO:tasks.ceph:[client] pid file = 2023-10-06T10:37:24.112 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-06T10:37:24.112 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-06T10:37:24.112 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-06T10:37:24.113 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-06T10:37:24.113 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-06T10:37:24.113 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-06T10:37:24.113 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-06T10:37:24.113 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-06T10:37:24.113 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-06T10:37:24.113 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-06T10:37:24.113 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-06T10:37:24.113 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-06T10:37:24.113 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-06T10:37:24.113 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-06T10:37:24.114 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-06T10:37:24.114 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-06T10:37:24.114 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-06T10:37:24.114 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-06T10:37:24.114 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-06T10:37:24.114 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-06T10:37:24.114 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-10-06T10:37:24.114 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-10-06T10:37:24.114 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-06T10:37:24.114 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-06T10:37:24.114 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-10-06T10:37:24.114 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-06T10:37:24.115 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-06T10:37:24.115 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-06T10:37:24.115 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-06T10:37:24.115 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-06T10:37:24.115 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-06T10:37:24.115 INFO:tasks.ceph:Setting up mon.a... 2023-10-06T10:37:24.115 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-06T10:37:24.161 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.keyring 2023-10-06T10:37:24.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-06T10:37:24.204 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T10:37:24.269 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.92'), ('mon.b', '172.21.15.124'), ('mon.c', '[v2:172.21.15.124:3301,v1:172.21.15.124:6790]')] 2023-10-06T10:37:24.269 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.92,172.21.15.124,[v2:172.21.15.124:3301,v1:172.21.15.124:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-06T10:37:24.270 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:24.270 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T10:37:24.323 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.92 --add b 172.21.15.124 --addv c '[v2:172.21.15.124:3301,v1:172.21.15.124:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-06T10:37:24.405 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-06T10:37:24.405 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: generated fsid 8f307bfb-c4cb-4f66-aade-46fffe0c6e0c 2023-10-06T10:37:24.405 INFO:teuthology.orchestra.run.smithi092.stdout:setting min_mon_release = octopus 2023-10-06T10:37:24.405 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 0 2023-10-06T10:37:24.405 INFO:teuthology.orchestra.run.smithi092.stdout:fsid 8f307bfb-c4cb-4f66-aade-46fffe0c6e0c 2023-10-06T10:37:24.405 INFO:teuthology.orchestra.run.smithi092.stdout:last_changed 2023-10-06T10:37:24.404917+0000 2023-10-06T10:37:24.406 INFO:teuthology.orchestra.run.smithi092.stdout:created 2023-10-06T10:37:24.404917+0000 2023-10-06T10:37:24.406 INFO:teuthology.orchestra.run.smithi092.stdout:min_mon_release 15 (octopus) 2023-10-06T10:37:24.406 INFO:teuthology.orchestra.run.smithi092.stdout:election_strategy: 1 2023-10-06T10:37:24.406 INFO:teuthology.orchestra.run.smithi092.stdout:0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2023-10-06T10:37:24.406 INFO:teuthology.orchestra.run.smithi092.stdout:1: [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] mon.b 2023-10-06T10:37:24.406 INFO:teuthology.orchestra.run.smithi092.stdout:2: [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] mon.c 2023-10-06T10:37:24.406 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-06T10:37:24.407 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T10:37:24.462 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 8f307bfb-c4cb-4f66-aade-46fffe0c6e0c... 2023-10-06T10:37:24.463 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T10:37:24.505 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T10:37:24.545 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2023-10-06T10:37:24.545 INFO:teuthology.orchestra.run.smithi092.stdout: chdir = "" 2023-10-06T10:37:24.545 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T10:37:24.545 INFO:teuthology.orchestra.run.smithi092.stdout: auth supported = cephx 2023-10-06T10:37:24.545 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.546 INFO:teuthology.orchestra.run.smithi092.stdout: filestore xattr use omap = true 2023-10-06T10:37:24.546 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.546 INFO:teuthology.orchestra.run.smithi092.stdout: mon clock drift allowed = 1.000 2023-10-06T10:37:24.546 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.546 INFO:teuthology.orchestra.run.smithi092.stdout: osd crush chooseleaf type = 0 2023-10-06T10:37:24.546 INFO:teuthology.orchestra.run.smithi092.stdout: auth debug = true 2023-10-06T10:37:24.546 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.546 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on old message = true 2023-10-06T10:37:24.546 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on bug = true 2023-10-06T10:37:24.546 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.546 INFO:teuthology.orchestra.run.smithi092.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T10:37:24.546 INFO:teuthology.orchestra.run.smithi092.stdout: mon pg warn max object skew = 0 2023-10-06T10:37:24.547 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.547 INFO:teuthology.orchestra.run.smithi092.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T10:37:24.547 INFO:teuthology.orchestra.run.smithi092.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T10:37:24.547 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.547 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default size = 2 2023-10-06T10:37:24.547 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.547 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow primary affinity = true 2023-10-06T10:37:24.547 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow pg remap = true 2023-10-06T10:37:24.548 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on legacy crush tunables = false 2023-10-06T10:37:24.548 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on crush straw calc version zero = false 2023-10-06T10:37:24.548 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on no sortbitwise = false 2023-10-06T10:37:24.548 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on osd down out interval zero = false 2023-10-06T10:37:24.548 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on too few osds = false 2023-10-06T10:37:24.548 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T10:37:24.548 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T10:37:24.548 INFO:teuthology.orchestra.run.smithi092.stdout: mon_allow_pool_size_one = true 2023-10-06T10:37:24.548 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.548 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T10:37:24.548 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.548 INFO:teuthology.orchestra.run.smithi092.stdout: osd default data pool replay window = 5 2023-10-06T10:37:24.549 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.549 INFO:teuthology.orchestra.run.smithi092.stdout: mon allow pool delete = true 2023-10-06T10:37:24.549 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.549 INFO:teuthology.orchestra.run.smithi092.stdout: mon cluster log file level = debug 2023-10-06T10:37:24.549 INFO:teuthology.orchestra.run.smithi092.stdout: debug asserts on shutdown = true 2023-10-06T10:37:24.549 INFO:teuthology.orchestra.run.smithi092.stdout: mon health detail to clog = false 2023-10-06T10:37:24.549 INFO:teuthology.orchestra.run.smithi092.stdout: mon host = "172.21.15.92,172.21.15.124,[v2:172.21.15.124:3301,v1:172.21.15.124:6790]" 2023-10-06T10:37:24.549 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on skipped message = False 2023-10-06T10:37:24.549 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = 8f307bfb-c4cb-4f66-aade-46fffe0c6e0c 2023-10-06T10:37:24.549 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.549 INFO:teuthology.orchestra.run.smithi092.stdout:[osd] 2023-10-06T10:37:24.549 INFO:teuthology.orchestra.run.smithi092.stdout: osd journal size = 100 2023-10-06T10:37:24.550 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.550 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub load threshold = 5.0 2023-10-06T10:37:24.550 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub max interval = 600 2023-10-06T10:37:24.550 INFO:teuthology.orchestra.run.smithi092.stdout: osd mclock profile = high_recovery_ops 2023-10-06T10:37:24.550 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.550 INFO:teuthology.orchestra.run.smithi092.stdout: osd recover clone overlap = true 2023-10-06T10:37:24.550 INFO:teuthology.orchestra.run.smithi092.stdout: osd recovery max chunk = 1048576 2023-10-06T10:37:24.550 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.550 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug shutdown = true 2023-10-06T10:37:24.550 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug op order = true 2023-10-06T10:37:24.550 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify stray on activate = true 2023-10-06T10:37:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: osd open classes on start = true 2023-10-06T10:37:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug pg log writeout = true 2023-10-06T10:37:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: osd deep scrub update digest min age = 30 2023-10-06T10:37:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: osd map max advance = 10 2023-10-06T10:37:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: journal zero on create = true 2023-10-06T10:37:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: filestore ondisk finisher threads = 3 2023-10-06T10:37:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: filestore apply finisher threads = 3 2023-10-06T10:37:24.552 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.552 INFO:teuthology.orchestra.run.smithi092.stdout: bdev debug aio = true 2023-10-06T10:37:24.552 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug misdirected ops = true 2023-10-06T10:37:24.552 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore block size = 96636764160 2023-10-06T10:37:24.552 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore fsck on mount = True 2023-10-06T10:37:24.552 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluefs = 20 2023-10-06T10:37:24.552 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluestore = 20 2023-10-06T10:37:24.552 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-10-06T10:37:24.552 INFO:teuthology.orchestra.run.smithi092.stdout: debug osd = 20 2023-10-06T10:37:24.552 INFO:teuthology.orchestra.run.smithi092.stdout: debug rocksdb = 10 2023-10-06T10:37:24.552 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T10:37:24.552 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd full ratio = 0.9 2023-10-06T10:37:24.552 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd nearfull ratio = 0.8 2023-10-06T10:37:24.553 INFO:teuthology.orchestra.run.smithi092.stdout: osd failsafe full ratio = 0.95 2023-10-06T10:37:24.553 INFO:teuthology.orchestra.run.smithi092.stdout: osd objectstore = bluestore 2023-10-06T10:37:24.553 INFO:teuthology.orchestra.run.smithi092.stdout: osd op complaint time = 180 2023-10-06T10:37:24.553 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.553 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr] 2023-10-06T10:37:24.553 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-10-06T10:37:24.553 INFO:teuthology.orchestra.run.smithi092.stdout: debug mgr = 20 2023-10-06T10:37:24.553 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2023-10-06T10:37:24.553 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2023-10-06T10:37:24.553 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2023-10-06T10:37:24.553 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2023-10-06T10:37:24.553 INFO:teuthology.orchestra.run.smithi092.stdout: mgr/telemetry/nag = false 2023-10-06T10:37:24.554 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.554 INFO:teuthology.orchestra.run.smithi092.stdout:[mon] 2023-10-06T10:37:24.554 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-10-06T10:37:24.554 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2023-10-06T10:37:24.554 INFO:teuthology.orchestra.run.smithi092.stdout: debug paxos = 20 2023-10-06T10:37:24.554 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2023-10-06T10:37:24.554 INFO:teuthology.orchestra.run.smithi092.stdout: mon data avail warn = 5 2023-10-06T10:37:24.554 INFO:teuthology.orchestra.run.smithi092.stdout: mon mgr mkfs grace = 240 2023-10-06T10:37:24.555 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2023-10-06T10:37:24.555 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd reporter subtree level = osd 2023-10-06T10:37:24.555 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd prime pg temp = true 2023-10-06T10:37:24.555 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2023-10-06T10:37:24.555 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.555 INFO:teuthology.orchestra.run.smithi092.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T10:37:24.555 INFO:teuthology.orchestra.run.smithi092.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T10:37:24.555 INFO:teuthology.orchestra.run.smithi092.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T10:37:24.555 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.555 INFO:teuthology.orchestra.run.smithi092.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T10:37:24.555 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T10:37:24.555 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T10:37:24.556 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.556 INFO:teuthology.orchestra.run.smithi092.stdout: # 1m isn't quite enough 2023-10-06T10:37:24.556 INFO:teuthology.orchestra.run.smithi092.stdout: mon_down_mkfs_grace = 2m 2023-10-06T10:37:24.556 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.556 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_filestore_osds = false 2023-10-06T10:37:24.556 INFO:teuthology.orchestra.run.smithi092.stdout: mon op complaint time = 120 2023-10-06T10:37:24.556 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:37:24.556 INFO:teuthology.orchestra.run.smithi092.stdout:[client] 2023-10-06T10:37:24.556 INFO:teuthology.orchestra.run.smithi092.stdout: rgw cache enabled = true 2023-10-06T10:37:24.556 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable ops log = true 2023-10-06T10:37:24.556 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable usage log = true 2023-10-06T10:37:24.556 INFO:teuthology.orchestra.run.smithi092.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T10:37:24.557 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T10:37:24.557 INFO:teuthology.orchestra.run.smithi092.stdout: client mount timeout = 600 2023-10-06T10:37:24.557 INFO:teuthology.orchestra.run.smithi092.stdout: debug client = 20 2023-10-06T10:37:24.557 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-10-06T10:37:24.557 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = "" 2023-10-06T10:37:24.557 INFO:teuthology.orchestra.run.smithi092.stdout: rados mon op timeout = 900 2023-10-06T10:37:24.557 INFO:teuthology.orchestra.run.smithi092.stdout: rados osd op timeout = 900 2023-10-06T10:37:24.557 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.a] 2023-10-06T10:37:24.557 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.b] 2023-10-06T10:37:24.557 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.c] 2023-10-06T10:37:24.558 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a] 2023-10-06T10:37:24.558 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c] 2023-10-06T10:37:24.558 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b] 2023-10-06T10:37:24.558 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d] 2023-10-06T10:37:24.558 INFO:teuthology.orchestra.run.smithi092.stdout:[mds] 2023-10-06T10:37:24.558 INFO:teuthology.orchestra.run.smithi092.stdout: debug mds = 20 2023-10-06T10:37:24.558 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-10-06T10:37:24.558 INFO:teuthology.orchestra.run.smithi092.stdout: mds debug frag = True 2023-10-06T10:37:24.558 INFO:teuthology.orchestra.run.smithi092.stdout: mds debug scatterstat = True 2023-10-06T10:37:24.559 INFO:teuthology.orchestra.run.smithi092.stdout: mds op complaint time = 180 2023-10-06T10:37:24.559 INFO:teuthology.orchestra.run.smithi092.stdout: mds verify scatter = True 2023-10-06T10:37:24.559 INFO:teuthology.orchestra.run.smithi092.stdout: osd op complaint time = 180 2023-10-06T10:37:24.559 INFO:teuthology.orchestra.run.smithi092.stdout: rados mon op timeout = 900 2023-10-06T10:37:24.559 INFO:teuthology.orchestra.run.smithi092.stdout: rados osd op timeout = 900 2023-10-06T10:37:24.560 INFO:teuthology.orchestra.run.smithi124.stdout:[global] 2023-10-06T10:37:24.560 INFO:teuthology.orchestra.run.smithi124.stdout: chdir = "" 2023-10-06T10:37:24.561 INFO:teuthology.orchestra.run.smithi124.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T10:37:24.561 INFO:teuthology.orchestra.run.smithi124.stdout: auth supported = cephx 2023-10-06T10:37:24.561 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.561 INFO:teuthology.orchestra.run.smithi124.stdout: filestore xattr use omap = true 2023-10-06T10:37:24.561 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.561 INFO:teuthology.orchestra.run.smithi124.stdout: mon clock drift allowed = 1.000 2023-10-06T10:37:24.561 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.561 INFO:teuthology.orchestra.run.smithi124.stdout: osd crush chooseleaf type = 0 2023-10-06T10:37:24.561 INFO:teuthology.orchestra.run.smithi124.stdout: auth debug = true 2023-10-06T10:37:24.561 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.561 INFO:teuthology.orchestra.run.smithi124.stdout: ms die on old message = true 2023-10-06T10:37:24.562 INFO:teuthology.orchestra.run.smithi124.stdout: ms die on bug = true 2023-10-06T10:37:24.562 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.562 INFO:teuthology.orchestra.run.smithi124.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T10:37:24.562 INFO:teuthology.orchestra.run.smithi124.stdout: mon pg warn max object skew = 0 2023-10-06T10:37:24.562 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.562 INFO:teuthology.orchestra.run.smithi124.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T10:37:24.562 INFO:teuthology.orchestra.run.smithi124.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T10:37:24.562 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.562 INFO:teuthology.orchestra.run.smithi124.stdout: osd pool default size = 2 2023-10-06T10:37:24.562 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.563 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd allow primary affinity = true 2023-10-06T10:37:24.563 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd allow pg remap = true 2023-10-06T10:37:24.563 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on legacy crush tunables = false 2023-10-06T10:37:24.563 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on crush straw calc version zero = false 2023-10-06T10:37:24.563 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on no sortbitwise = false 2023-10-06T10:37:24.563 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on osd down out interval zero = false 2023-10-06T10:37:24.563 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on too few osds = false 2023-10-06T10:37:24.563 INFO:teuthology.orchestra.run.smithi124.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T10:37:24.563 INFO:teuthology.orchestra.run.smithi124.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T10:37:24.563 INFO:teuthology.orchestra.run.smithi124.stdout: mon_allow_pool_size_one = true 2023-10-06T10:37:24.563 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.563 INFO:teuthology.orchestra.run.smithi124.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T10:37:24.563 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.564 INFO:teuthology.orchestra.run.smithi124.stdout: osd default data pool replay window = 5 2023-10-06T10:37:24.564 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.564 INFO:teuthology.orchestra.run.smithi124.stdout: mon allow pool delete = true 2023-10-06T10:37:24.564 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.564 INFO:teuthology.orchestra.run.smithi124.stdout: mon cluster log file level = debug 2023-10-06T10:37:24.564 INFO:teuthology.orchestra.run.smithi124.stdout: debug asserts on shutdown = true 2023-10-06T10:37:24.564 INFO:teuthology.orchestra.run.smithi124.stdout: mon health detail to clog = false 2023-10-06T10:37:24.564 INFO:teuthology.orchestra.run.smithi124.stdout: mon host = "172.21.15.92,172.21.15.124,[v2:172.21.15.124:3301,v1:172.21.15.124:6790]" 2023-10-06T10:37:24.564 INFO:teuthology.orchestra.run.smithi124.stdout: ms die on skipped message = False 2023-10-06T10:37:24.564 INFO:teuthology.orchestra.run.smithi124.stdout: fsid = 8f307bfb-c4cb-4f66-aade-46fffe0c6e0c 2023-10-06T10:37:24.564 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.565 INFO:teuthology.orchestra.run.smithi124.stdout:[osd] 2023-10-06T10:37:24.565 INFO:teuthology.orchestra.run.smithi124.stdout: osd journal size = 100 2023-10-06T10:37:24.565 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.565 INFO:teuthology.orchestra.run.smithi124.stdout: osd scrub load threshold = 5.0 2023-10-06T10:37:24.565 INFO:teuthology.orchestra.run.smithi124.stdout: osd scrub max interval = 600 2023-10-06T10:37:24.565 INFO:teuthology.orchestra.run.smithi124.stdout: osd mclock profile = high_recovery_ops 2023-10-06T10:37:24.565 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.565 INFO:teuthology.orchestra.run.smithi124.stdout: osd recover clone overlap = true 2023-10-06T10:37:24.565 INFO:teuthology.orchestra.run.smithi124.stdout: osd recovery max chunk = 1048576 2023-10-06T10:37:24.565 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.565 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug shutdown = true 2023-10-06T10:37:24.565 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug op order = true 2023-10-06T10:37:24.566 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug verify stray on activate = true 2023-10-06T10:37:24.566 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.566 INFO:teuthology.orchestra.run.smithi124.stdout: osd open classes on start = true 2023-10-06T10:37:24.566 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug pg log writeout = true 2023-10-06T10:37:24.566 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.566 INFO:teuthology.orchestra.run.smithi124.stdout: osd deep scrub update digest min age = 30 2023-10-06T10:37:24.566 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.566 INFO:teuthology.orchestra.run.smithi124.stdout: osd map max advance = 10 2023-10-06T10:37:24.566 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.566 INFO:teuthology.orchestra.run.smithi124.stdout: journal zero on create = true 2023-10-06T10:37:24.566 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.567 INFO:teuthology.orchestra.run.smithi124.stdout: filestore ondisk finisher threads = 3 2023-10-06T10:37:24.567 INFO:teuthology.orchestra.run.smithi124.stdout: filestore apply finisher threads = 3 2023-10-06T10:37:24.567 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.567 INFO:teuthology.orchestra.run.smithi124.stdout: bdev debug aio = true 2023-10-06T10:37:24.567 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug misdirected ops = true 2023-10-06T10:37:24.567 INFO:teuthology.orchestra.run.smithi124.stdout: bluestore block size = 96636764160 2023-10-06T10:37:24.567 INFO:teuthology.orchestra.run.smithi124.stdout: bluestore fsck on mount = True 2023-10-06T10:37:24.567 INFO:teuthology.orchestra.run.smithi124.stdout: debug bluefs = 20 2023-10-06T10:37:24.567 INFO:teuthology.orchestra.run.smithi124.stdout: debug bluestore = 20 2023-10-06T10:37:24.567 INFO:teuthology.orchestra.run.smithi124.stdout: debug ms = 1 2023-10-06T10:37:24.567 INFO:teuthology.orchestra.run.smithi124.stdout: debug osd = 20 2023-10-06T10:37:24.567 INFO:teuthology.orchestra.run.smithi124.stdout: debug rocksdb = 10 2023-10-06T10:37:24.567 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T10:37:24.568 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd full ratio = 0.9 2023-10-06T10:37:24.568 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd nearfull ratio = 0.8 2023-10-06T10:37:24.568 INFO:teuthology.orchestra.run.smithi124.stdout: osd failsafe full ratio = 0.95 2023-10-06T10:37:24.568 INFO:teuthology.orchestra.run.smithi124.stdout: osd objectstore = bluestore 2023-10-06T10:37:24.568 INFO:teuthology.orchestra.run.smithi124.stdout: osd op complaint time = 180 2023-10-06T10:37:24.568 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.568 INFO:teuthology.orchestra.run.smithi124.stdout:[mgr] 2023-10-06T10:37:24.568 INFO:teuthology.orchestra.run.smithi124.stdout: debug ms = 1 2023-10-06T10:37:24.568 INFO:teuthology.orchestra.run.smithi124.stdout: debug mgr = 20 2023-10-06T10:37:24.569 INFO:teuthology.orchestra.run.smithi124.stdout: debug mon = 20 2023-10-06T10:37:24.569 INFO:teuthology.orchestra.run.smithi124.stdout: debug auth = 20 2023-10-06T10:37:24.569 INFO:teuthology.orchestra.run.smithi124.stdout: mon reweight min pgs per osd = 4 2023-10-06T10:37:24.569 INFO:teuthology.orchestra.run.smithi124.stdout: mon reweight min bytes per osd = 10 2023-10-06T10:37:24.569 INFO:teuthology.orchestra.run.smithi124.stdout: mgr/telemetry/nag = false 2023-10-06T10:37:24.569 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.569 INFO:teuthology.orchestra.run.smithi124.stdout:[mon] 2023-10-06T10:37:24.569 INFO:teuthology.orchestra.run.smithi124.stdout: debug ms = 1 2023-10-06T10:37:24.569 INFO:teuthology.orchestra.run.smithi124.stdout: debug mon = 20 2023-10-06T10:37:24.569 INFO:teuthology.orchestra.run.smithi124.stdout: debug paxos = 20 2023-10-06T10:37:24.569 INFO:teuthology.orchestra.run.smithi124.stdout: debug auth = 20 2023-10-06T10:37:24.570 INFO:teuthology.orchestra.run.smithi124.stdout: mon data avail warn = 5 2023-10-06T10:37:24.570 INFO:teuthology.orchestra.run.smithi124.stdout: mon mgr mkfs grace = 240 2023-10-06T10:37:24.570 INFO:teuthology.orchestra.run.smithi124.stdout: mon reweight min pgs per osd = 4 2023-10-06T10:37:24.570 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd reporter subtree level = osd 2023-10-06T10:37:24.570 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd prime pg temp = true 2023-10-06T10:37:24.570 INFO:teuthology.orchestra.run.smithi124.stdout: mon reweight min bytes per osd = 10 2023-10-06T10:37:24.570 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.570 INFO:teuthology.orchestra.run.smithi124.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T10:37:24.571 INFO:teuthology.orchestra.run.smithi124.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T10:37:24.571 INFO:teuthology.orchestra.run.smithi124.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T10:37:24.571 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.571 INFO:teuthology.orchestra.run.smithi124.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T10:37:24.571 INFO:teuthology.orchestra.run.smithi124.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T10:37:24.571 INFO:teuthology.orchestra.run.smithi124.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T10:37:24.571 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.571 INFO:teuthology.orchestra.run.smithi124.stdout: # 1m isn't quite enough 2023-10-06T10:37:24.571 INFO:teuthology.orchestra.run.smithi124.stdout: mon_down_mkfs_grace = 2m 2023-10-06T10:37:24.571 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.571 INFO:teuthology.orchestra.run.smithi124.stdout: mon_warn_on_filestore_osds = false 2023-10-06T10:37:24.571 INFO:teuthology.orchestra.run.smithi124.stdout: mon op complaint time = 120 2023-10-06T10:37:24.572 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-06T10:37:24.572 INFO:teuthology.orchestra.run.smithi124.stdout:[client] 2023-10-06T10:37:24.572 INFO:teuthology.orchestra.run.smithi124.stdout: rgw cache enabled = true 2023-10-06T10:37:24.572 INFO:teuthology.orchestra.run.smithi124.stdout: rgw enable ops log = true 2023-10-06T10:37:24.572 INFO:teuthology.orchestra.run.smithi124.stdout: rgw enable usage log = true 2023-10-06T10:37:24.572 INFO:teuthology.orchestra.run.smithi124.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T10:37:24.572 INFO:teuthology.orchestra.run.smithi124.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T10:37:24.572 INFO:teuthology.orchestra.run.smithi124.stdout: client mount timeout = 600 2023-10-06T10:37:24.572 INFO:teuthology.orchestra.run.smithi124.stdout: debug client = 20 2023-10-06T10:37:24.572 INFO:teuthology.orchestra.run.smithi124.stdout: debug ms = 1 2023-10-06T10:37:24.572 INFO:teuthology.orchestra.run.smithi124.stdout: pid file = "" 2023-10-06T10:37:24.573 INFO:teuthology.orchestra.run.smithi124.stdout: rados mon op timeout = 900 2023-10-06T10:37:24.573 INFO:teuthology.orchestra.run.smithi124.stdout: rados osd op timeout = 900 2023-10-06T10:37:24.573 INFO:teuthology.orchestra.run.smithi124.stdout:[mon.a] 2023-10-06T10:37:24.573 INFO:teuthology.orchestra.run.smithi124.stdout:[mon.b] 2023-10-06T10:37:24.573 INFO:teuthology.orchestra.run.smithi124.stdout:[mon.c] 2023-10-06T10:37:24.573 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.a] 2023-10-06T10:37:24.573 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.c] 2023-10-06T10:37:24.574 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.b] 2023-10-06T10:37:24.574 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.d] 2023-10-06T10:37:24.574 INFO:teuthology.orchestra.run.smithi124.stdout:[mds] 2023-10-06T10:37:24.574 INFO:teuthology.orchestra.run.smithi124.stdout: debug mds = 20 2023-10-06T10:37:24.574 INFO:teuthology.orchestra.run.smithi124.stdout: debug ms = 1 2023-10-06T10:37:24.574 INFO:teuthology.orchestra.run.smithi124.stdout: mds debug frag = True 2023-10-06T10:37:24.574 INFO:teuthology.orchestra.run.smithi124.stdout: mds debug scatterstat = True 2023-10-06T10:37:24.574 INFO:teuthology.orchestra.run.smithi124.stdout: mds op complaint time = 180 2023-10-06T10:37:24.574 INFO:teuthology.orchestra.run.smithi124.stdout: mds verify scatter = True 2023-10-06T10:37:24.575 INFO:teuthology.orchestra.run.smithi124.stdout: osd op complaint time = 180 2023-10-06T10:37:24.575 INFO:teuthology.orchestra.run.smithi124.stdout: rados mon op timeout = 900 2023-10-06T10:37:24.575 INFO:teuthology.orchestra.run.smithi124.stdout: rados osd op timeout = 900 2023-10-06T10:37:24.575 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-06T10:37:24.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-06T10:37:24.651 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-06T10:37:24.651 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:24.651 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-06T10:37:24.665 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:24.666 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-06T10:37:24.720 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T10:37:24.721 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:24.721 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T10:37:24.721 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T10:37:24.793 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:24.794 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T10:37:24.848 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi124.front.sepia.ceph.com 2023-10-06T10:37:24.848 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:37:24.848 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T10:37:24.848 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T10:37:24.883 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:37:24.883 DEBUG:teuthology.orchestra.run.smithi124:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T10:37:24.938 INFO:tasks.ceph:Setting up mon nodes... 2023-10-06T10:37:24.938 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-06T10:37:24.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T10:37:24.986 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T10:37:24.988 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T10:37:25.086 INFO:teuthology.orchestra.run.smithi124.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T10:37:25.088 INFO:tasks.ceph:Setting up mds nodes... 2023-10-06T10:37:25.089 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T10:37:25.136 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T10:37:25.139 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-06T10:37:25.163 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T10:37:25.250 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T10:37:25.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-06T10:37:25.276 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T10:37:25.324 INFO:teuthology.orchestra.run.smithi124.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T10:37:25.326 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-06T10:37:25.350 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T10:37:25.437 INFO:teuthology.orchestra.run.smithi124.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T10:37:25.440 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-06T10:37:25.464 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-06T10:37:25.464 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-06T10:37:25.501 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-06T10:37:25.513 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-06T10:37:25.591 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-06T10:37:25.603 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-06T10:37:25.641 INFO:teuthology.orchestra.run.smithi124.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-06T10:37:25.653 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-06T10:37:25.731 INFO:teuthology.orchestra.run.smithi124.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-06T10:37:25.743 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-06T10:37:25.743 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi124.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T10:37:25.743 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-06T10:37:25.768 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T10:37:25.768 INFO:tasks.ceph:role: osd.0 2023-10-06T10:37:25.768 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T10:37:25.768 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T10:37:25.840 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T10:37:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T10:37:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T10:37:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 2023-10-06T10:37:25.840 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T10:37:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-10-06T10:37:25.841 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T10:37:25.841 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T10:37:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T10:37:25.841 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T10:37:26.268 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-10-06T10:37:26.269 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-10-06T10:37:26.270 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-06T10:37:26.440 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-06T10:37:26.471 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-06T10:37:26.536 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T10:37:26.536 INFO:tasks.ceph:role: osd.1 2023-10-06T10:37:26.536 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T10:37:26.536 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T10:37:26.608 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T10:37:26.608 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T10:37:26.608 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T10:37:26.608 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 2023-10-06T10:37:26.608 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T10:37:26.608 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-10-06T10:37:26.608 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T10:37:26.608 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T10:37:26.609 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T10:37:26.609 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T10:37:27.039 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-10-06T10:37:27.040 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-10-06T10:37:27.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-06T10:37:27.140 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-06T10:37:27.169 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-06T10:37:27.234 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T10:37:27.234 INFO:tasks.ceph:role: osd.2 2023-10-06T10:37:27.234 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T10:37:27.234 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T10:37:27.306 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T10:37:27.306 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T10:37:27.306 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T10:37:27.306 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 2023-10-06T10:37:27.306 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T10:37:27.306 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-10-06T10:37:27.307 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T10:37:27.307 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T10:37:27.307 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T10:37:27.307 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T10:37:27.756 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-10-06T10:37:27.756 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-10-06T10:37:27.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-06T10:37:27.913 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-06T10:37:27.942 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-06T10:37:28.006 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T10:37:28.006 INFO:tasks.ceph:role: osd.3 2023-10-06T10:37:28.006 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T10:37:28.006 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T10:37:28.079 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T10:37:28.079 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T10:37:28.079 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T10:37:28.079 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 2023-10-06T10:37:28.079 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T10:37:28.079 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-10-06T10:37:28.080 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T10:37:28.080 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T10:37:28.080 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T10:37:28.080 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T10:37:28.517 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-10-06T10:37:28.519 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-10-06T10:37:28.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-06T10:37:28.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-06T10:37:28.712 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T10:37:28.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:28.804+0000 7fcb418f9540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-06T10:37:28.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:28.806+0000 7fcb418f9540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-06T10:37:28.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:28.806+0000 7fcb418f9540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T10:37:28.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:28.806+0000 7fcb418f9540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T10:37:28.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:28.806+0000 7fcb418f9540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T10:37:28.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:28.806+0000 7fcb418f9540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-06T10:37:29.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:29.334+0000 7fcb418f9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:30.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:30.097+0000 7fcb418f9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:30.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:30.860+0000 7fcb418f9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:31.665 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-06T10:37:31.691 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T10:37:31.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:31.771+0000 7f3e38cd7540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-06T10:37:31.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:31.771+0000 7f3e38cd7540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-06T10:37:31.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:31.771+0000 7f3e38cd7540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T10:37:31.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:31.771+0000 7f3e38cd7540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T10:37:31.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:31.771+0000 7f3e38cd7540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T10:37:31.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:31.771+0000 7f3e38cd7540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-06T10:37:32.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:32.297+0000 7f3e38cd7540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:33.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:33.066+0000 7f3e38cd7540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:33.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:33.829+0000 7f3e38cd7540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:34.627 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-06T10:37:34.695 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T10:37:34.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:34.778+0000 7f88fd21c540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-06T10:37:34.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:34.778+0000 7f88fd21c540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-06T10:37:34.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:34.778+0000 7f88fd21c540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T10:37:34.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:34.778+0000 7f88fd21c540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T10:37:34.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:34.778+0000 7f88fd21c540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T10:37:34.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:34.779+0000 7f88fd21c540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-06T10:37:35.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:35.306+0000 7f88fd21c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:36.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:36.074+0000 7f88fd21c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:36.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:36.836+0000 7f88fd21c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:37.625 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-06T10:37:37.690 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T10:37:37.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:37.770+0000 7fa8fa083540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-06T10:37:37.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:37.770+0000 7fa8fa083540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-06T10:37:37.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:37.770+0000 7fa8fa083540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T10:37:37.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:37.770+0000 7fa8fa083540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T10:37:37.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:37.770+0000 7fa8fa083540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T10:37:37.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:37.770+0000 7fa8fa083540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-06T10:37:38.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:38.303+0000 7fa8fa083540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:39.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:39.065+0000 7fa8fa083540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:39.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:39.831+0000 7fa8fa083540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:40.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-06T10:37:40.694 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-06T10:37:40.722 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T10:37:40.722 INFO:tasks.ceph:role: osd.4 2023-10-06T10:37:40.722 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi124.front.sepia.ceph.com 2023-10-06T10:37:40.722 DEBUG:teuthology.orchestra.run.smithi124:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T10:37:40.795 INFO:teuthology.orchestra.run.smithi124.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T10:37:40.795 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T10:37:40.796 INFO:teuthology.orchestra.run.smithi124.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T10:37:40.796 INFO:teuthology.orchestra.run.smithi124.stdout: = reflink=1 2023-10-06T10:37:40.796 INFO:teuthology.orchestra.run.smithi124.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T10:37:40.796 INFO:teuthology.orchestra.run.smithi124.stdout: = sunit=0 swidth=0 blks 2023-10-06T10:37:40.796 INFO:teuthology.orchestra.run.smithi124.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T10:37:40.796 INFO:teuthology.orchestra.run.smithi124.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T10:37:40.796 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T10:37:40.796 INFO:teuthology.orchestra.run.smithi124.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T10:37:41.233 INFO:teuthology.orchestra.run.smithi124.stdout:Discarding blocks...Done. 2023-10-06T10:37:41.235 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi124.front.sepia.ceph.com -o noatime 2023-10-06T10:37:41.236 DEBUG:teuthology.orchestra.run.smithi124:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-06T10:37:41.378 DEBUG:teuthology.orchestra.run.smithi124:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-06T10:37:41.413 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-06T10:37:41.479 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T10:37:41.479 INFO:tasks.ceph:role: osd.5 2023-10-06T10:37:41.479 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi124.front.sepia.ceph.com 2023-10-06T10:37:41.479 DEBUG:teuthology.orchestra.run.smithi124:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T10:37:41.551 INFO:teuthology.orchestra.run.smithi124.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T10:37:41.551 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T10:37:41.551 INFO:teuthology.orchestra.run.smithi124.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T10:37:41.551 INFO:teuthology.orchestra.run.smithi124.stdout: = reflink=1 2023-10-06T10:37:41.551 INFO:teuthology.orchestra.run.smithi124.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T10:37:41.552 INFO:teuthology.orchestra.run.smithi124.stdout: = sunit=0 swidth=0 blks 2023-10-06T10:37:41.552 INFO:teuthology.orchestra.run.smithi124.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T10:37:41.552 INFO:teuthology.orchestra.run.smithi124.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T10:37:41.552 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T10:37:41.552 INFO:teuthology.orchestra.run.smithi124.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T10:37:41.982 INFO:teuthology.orchestra.run.smithi124.stdout:Discarding blocks...Done. 2023-10-06T10:37:41.983 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi124.front.sepia.ceph.com -o noatime 2023-10-06T10:37:41.984 DEBUG:teuthology.orchestra.run.smithi124:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-06T10:37:42.087 DEBUG:teuthology.orchestra.run.smithi124:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-06T10:37:42.117 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-06T10:37:42.181 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T10:37:42.182 INFO:tasks.ceph:role: osd.6 2023-10-06T10:37:42.182 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi124.front.sepia.ceph.com 2023-10-06T10:37:42.182 DEBUG:teuthology.orchestra.run.smithi124:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T10:37:42.254 INFO:teuthology.orchestra.run.smithi124.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T10:37:42.255 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T10:37:42.255 INFO:teuthology.orchestra.run.smithi124.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T10:37:42.255 INFO:teuthology.orchestra.run.smithi124.stdout: = reflink=1 2023-10-06T10:37:42.255 INFO:teuthology.orchestra.run.smithi124.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T10:37:42.255 INFO:teuthology.orchestra.run.smithi124.stdout: = sunit=0 swidth=0 blks 2023-10-06T10:37:42.255 INFO:teuthology.orchestra.run.smithi124.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T10:37:42.255 INFO:teuthology.orchestra.run.smithi124.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T10:37:42.256 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T10:37:42.256 INFO:teuthology.orchestra.run.smithi124.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T10:37:42.708 INFO:teuthology.orchestra.run.smithi124.stdout:Discarding blocks...Done. 2023-10-06T10:37:42.710 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi124.front.sepia.ceph.com -o noatime 2023-10-06T10:37:42.710 DEBUG:teuthology.orchestra.run.smithi124:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-06T10:37:42.867 DEBUG:teuthology.orchestra.run.smithi124:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-06T10:37:42.905 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-06T10:37:42.972 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T10:37:42.973 INFO:tasks.ceph:role: osd.7 2023-10-06T10:37:42.973 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi124.front.sepia.ceph.com 2023-10-06T10:37:42.973 DEBUG:teuthology.orchestra.run.smithi124:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T10:37:43.046 INFO:teuthology.orchestra.run.smithi124.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T10:37:43.047 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T10:37:43.047 INFO:teuthology.orchestra.run.smithi124.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T10:37:43.047 INFO:teuthology.orchestra.run.smithi124.stdout: = reflink=1 2023-10-06T10:37:43.047 INFO:teuthology.orchestra.run.smithi124.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T10:37:43.047 INFO:teuthology.orchestra.run.smithi124.stdout: = sunit=0 swidth=0 blks 2023-10-06T10:37:43.047 INFO:teuthology.orchestra.run.smithi124.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T10:37:43.047 INFO:teuthology.orchestra.run.smithi124.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T10:37:43.048 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T10:37:43.048 INFO:teuthology.orchestra.run.smithi124.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T10:37:43.479 INFO:teuthology.orchestra.run.smithi124.stdout:Discarding blocks...Done. 2023-10-06T10:37:43.481 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi124.front.sepia.ceph.com -o noatime 2023-10-06T10:37:43.481 DEBUG:teuthology.orchestra.run.smithi124:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-06T10:37:43.655 DEBUG:teuthology.orchestra.run.smithi124:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-06T10:37:43.691 DEBUG:teuthology.orchestra.run.smithi124:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T10:37:43.820 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:43.818+0000 7f59037e1540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-06T10:37:43.820 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:43.818+0000 7f59037e1540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-06T10:37:43.820 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:43.818+0000 7f59037e1540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T10:37:43.820 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:43.818+0000 7f59037e1540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T10:37:43.820 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:43.818+0000 7f59037e1540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T10:37:43.820 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:43.818+0000 7f59037e1540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-06T10:37:44.348 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:44.346+0000 7f59037e1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:45.112 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:45.110+0000 7f59037e1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:45.874 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:45.872+0000 7f59037e1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:46.666 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-06T10:37:46.693 DEBUG:teuthology.orchestra.run.smithi124:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T10:37:46.775 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:46.772+0000 7f4339699540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-06T10:37:46.775 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:46.773+0000 7f4339699540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-06T10:37:46.775 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:46.773+0000 7f4339699540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T10:37:46.775 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:46.773+0000 7f4339699540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T10:37:46.775 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:46.773+0000 7f4339699540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T10:37:46.775 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:46.773+0000 7f4339699540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-06T10:37:47.297 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:47.295+0000 7f4339699540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:48.060 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:48.058+0000 7f4339699540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:48.826 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:48.824+0000 7f4339699540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:49.620 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-06T10:37:49.690 DEBUG:teuthology.orchestra.run.smithi124:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T10:37:49.776 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:49.774+0000 7fd9abf6c540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-06T10:37:49.776 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:49.774+0000 7fd9abf6c540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-06T10:37:49.776 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:49.774+0000 7fd9abf6c540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T10:37:49.777 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:49.774+0000 7fd9abf6c540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T10:37:49.777 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:49.774+0000 7fd9abf6c540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T10:37:49.777 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:49.775+0000 7fd9abf6c540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-06T10:37:50.309 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:50.307+0000 7fd9abf6c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:51.081 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:51.080+0000 7fd9abf6c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:51.844 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:51.842+0000 7fd9abf6c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:52.517 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:37:52.520 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:37:52.648 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-06T10:37:52.673 DEBUG:teuthology.orchestra.run.smithi124:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T10:37:52.754 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:52.752+0000 7fc462ab9540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-06T10:37:52.754 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:52.752+0000 7fc462ab9540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-06T10:37:52.754 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:52.752+0000 7fc462ab9540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T10:37:52.754 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:52.752+0000 7fc462ab9540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T10:37:52.754 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:52.752+0000 7fc462ab9540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T10:37:52.754 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:52.752+0000 7fc462ab9540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-06T10:37:53.277 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:53.275+0000 7fc462ab9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:54.050 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:54.048+0000 7fc462ab9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:54.816 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:54.814+0000 7fc462ab9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T10:37:55.607 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-06T10:37:55.677 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-06T10:37:55.677 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:55.677 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-06T10:37:55.706 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:55.706 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-06T10:37:55.774 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:55.774 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-06T10:37:55.840 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:55.840 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-06T10:37:55.907 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:55.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-06T10:37:55.972 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:55.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-06T10:37:56.038 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:56.038 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-06T10:37:56.102 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:37:56.102 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-06T10:37:56.129 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:37:56.129 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-06T10:37:56.196 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:37:56.196 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-06T10:37:56.260 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:37:56.261 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-06T10:37:56.326 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:37:56.326 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-06T10:37:56.390 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:37:56.390 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-06T10:37:56.456 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:37:56.456 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-06T10:37:56.519 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:56.520 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-06T10:37:56.536 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:56.537 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-06T10:37:56.592 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:37:56.592 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-06T10:37:56.608 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:37:56.609 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-06T10:37:56.662 INFO:tasks.ceph:Adding keys to all mons... 2023-10-06T10:37:56.663 DEBUG:teuthology.orchestra.run.smithi092:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T10:37:56.665 DEBUG:teuthology.orchestra.run.smithi124:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T10:37:56.706 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.y] 2023-10-06T10:37:56.706 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBk4x9lgYa/OhAAyoI717Ga+Wv72srZ4FK+lQ== 2023-10-06T10:37:56.706 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a] 2023-10-06T10:37:56.706 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBl4x9lc28gCBAAVrjhXIawHuZEox0PE3/Rsg== 2023-10-06T10:37:56.707 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c] 2023-10-06T10:37:56.707 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBl4x9lrCrsDhAAvp++wvvnSpZRGzvRTrGWEw== 2023-10-06T10:37:56.707 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.0] 2023-10-06T10:37:56.707 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBo4x9lDEIFMBAAc3DY1fYpY2UiY7MX7qSP5A== 2023-10-06T10:37:56.707 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.1] 2023-10-06T10:37:56.707 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBr4x9lKzX8LRAApryLtFGkK6urkE8W70l5Cg== 2023-10-06T10:37:56.707 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.2] 2023-10-06T10:37:56.707 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBu4x9lCMZpLhAAEBPa40Wnb820p6j+lv+PsA== 2023-10-06T10:37:56.708 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.3] 2023-10-06T10:37:56.708 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBx4x9l4AHrLRAAFn6SnNkHnOiVXmmiU8qGyA== 2023-10-06T10:37:56.708 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.x] 2023-10-06T10:37:56.708 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBl4x9ll8wgBRAAQUrHERQ3Igv1tosa+wryIg== 2023-10-06T10:37:56.708 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b] 2023-10-06T10:37:56.708 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBl4x9lQpdOExAAm44s0n4Jekhy8AJk/o6Waw== 2023-10-06T10:37:56.708 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d] 2023-10-06T10:37:56.708 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBl4x9l/DgWGhAAy3JMA5ntn/UaRkuTCyPdkg== 2023-10-06T10:37:56.708 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.4] 2023-10-06T10:37:56.708 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQB34x9l7V7WMBAAvs5kvBP5n0Th478l/idjeQ== 2023-10-06T10:37:56.709 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.5] 2023-10-06T10:37:56.709 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQB64x9lfHIlLhAADCuqt+O/zPEETgoj//hwVg== 2023-10-06T10:37:56.709 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.6] 2023-10-06T10:37:56.709 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQB94x9lUmU7LhAAvxmtGmJxrI8TlOOD7taHOQ== 2023-10-06T10:37:56.709 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.7] 2023-10-06T10:37:56.709 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCA4x9lJE3nLBAAnREk9yNK+FhX4fnG+Za5Ug== 2023-10-06T10:37:56.709 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2023-10-06T10:37:56.709 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBl4x9lMePcHRAAMyftXWJWk7rwmWFkq18Rqg== 2023-10-06T10:37:56.710 INFO:teuthology.orchestra.run.smithi092.stdout:[client.2] 2023-10-06T10:37:56.710 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBl4x9lc6o9IxAAaOY7S5rYn+OdeMNWK1UYMA== 2023-10-06T10:37:56.710 INFO:teuthology.orchestra.run.smithi092.stdout:[client.1] 2023-10-06T10:37:56.710 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBl4x9lp1s0JhAAKvZ23MYjObxOK83q3UqJBQ== 2023-10-06T10:37:56.710 INFO:teuthology.orchestra.run.smithi092.stdout:[client.3] 2023-10-06T10:37:56.711 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBl4x9lwoyOKxAA811lehuQQ8MQQQzuTFAApg== 2023-10-06T10:37:56.725 INFO:teuthology.orchestra.run.smithi124.stdout:[mgr.y] 2023-10-06T10:37:56.725 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBk4x9lgYa/OhAAyoI717Ga+Wv72srZ4FK+lQ== 2023-10-06T10:37:56.725 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.a] 2023-10-06T10:37:56.725 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBl4x9lc28gCBAAVrjhXIawHuZEox0PE3/Rsg== 2023-10-06T10:37:56.725 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.c] 2023-10-06T10:37:56.726 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBl4x9lrCrsDhAAvp++wvvnSpZRGzvRTrGWEw== 2023-10-06T10:37:56.726 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.0] 2023-10-06T10:37:56.726 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBo4x9lDEIFMBAAc3DY1fYpY2UiY7MX7qSP5A== 2023-10-06T10:37:56.726 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.1] 2023-10-06T10:37:56.726 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBr4x9lKzX8LRAApryLtFGkK6urkE8W70l5Cg== 2023-10-06T10:37:56.726 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.2] 2023-10-06T10:37:56.726 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBu4x9lCMZpLhAAEBPa40Wnb820p6j+lv+PsA== 2023-10-06T10:37:56.727 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.3] 2023-10-06T10:37:56.727 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBx4x9l4AHrLRAAFn6SnNkHnOiVXmmiU8qGyA== 2023-10-06T10:37:56.727 INFO:teuthology.orchestra.run.smithi124.stdout:[mgr.x] 2023-10-06T10:37:56.727 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBl4x9ll8wgBRAAQUrHERQ3Igv1tosa+wryIg== 2023-10-06T10:37:56.728 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.b] 2023-10-06T10:37:56.728 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBl4x9lQpdOExAAm44s0n4Jekhy8AJk/o6Waw== 2023-10-06T10:37:56.728 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.d] 2023-10-06T10:37:56.728 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBl4x9l/DgWGhAAy3JMA5ntn/UaRkuTCyPdkg== 2023-10-06T10:37:56.728 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.4] 2023-10-06T10:37:56.728 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQB34x9l7V7WMBAAvs5kvBP5n0Th478l/idjeQ== 2023-10-06T10:37:56.728 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.5] 2023-10-06T10:37:56.728 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQB64x9lfHIlLhAADCuqt+O/zPEETgoj//hwVg== 2023-10-06T10:37:56.728 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.6] 2023-10-06T10:37:56.729 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQB94x9lUmU7LhAAvxmtGmJxrI8TlOOD7taHOQ== 2023-10-06T10:37:56.729 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.7] 2023-10-06T10:37:56.729 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQCA4x9lJE3nLBAAnREk9yNK+FhX4fnG+Za5Ug== 2023-10-06T10:37:56.729 INFO:teuthology.orchestra.run.smithi124.stdout:[client.0] 2023-10-06T10:37:56.729 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBl4x9lMePcHRAAMyftXWJWk7rwmWFkq18Rqg== 2023-10-06T10:37:56.729 INFO:teuthology.orchestra.run.smithi124.stdout:[client.2] 2023-10-06T10:37:56.729 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBl4x9lc6o9IxAAaOY7S5rYn+OdeMNWK1UYMA== 2023-10-06T10:37:56.729 INFO:teuthology.orchestra.run.smithi124.stdout:[client.1] 2023-10-06T10:37:56.729 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBl4x9lp1s0JhAAKvZ23MYjObxOK83q3UqJBQ== 2023-10-06T10:37:56.730 INFO:teuthology.orchestra.run.smithi124.stdout:[client.3] 2023-10-06T10:37:56.730 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBl4x9lwoyOKxAA811lehuQQ8MQQQzuTFAApg== 2023-10-06T10:37:56.731 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T10:37:56.755 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T10:37:56.818 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T10:37:56.821 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T10:37:56.870 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T10:37:56.873 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T10:37:56.922 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T10:37:56.924 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T10:37:56.971 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T10:37:56.973 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T10:37:57.020 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T10:37:57.022 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T10:37:57.069 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T10:37:57.071 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T10:37:57.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T10:37:57.120 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T10:37:57.167 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T10:37:57.169 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T10:37:57.219 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T10:37:57.221 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T10:37:57.271 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T10:37:57.273 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T10:37:57.320 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T10:37:57.322 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T10:37:57.370 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T10:37:57.372 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T10:37:57.418 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T10:37:57.421 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T10:37:57.467 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T10:37:57.469 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T10:37:57.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T10:37:57.521 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T10:37:57.571 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T10:37:57.573 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T10:37:57.623 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T10:37:57.625 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T10:37:57.676 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-06T10:37:57.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-06T10:37:57.700 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T10:37:57.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-06T10:37:57.815 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-06T10:37:57.839 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T10:37:57.930 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-06T10:37:57.955 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-06T10:37:58.019 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T10:37:58.110 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-06T10:37:58.134 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T10:37:58.137 DEBUG:teuthology.orchestra.run.smithi124:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T10:37:58.189 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-06T10:37:58.189 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-06T10:37:58.189 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-06T10:37:58.191 INFO:tasks.ceph.mon.a:Started 2023-10-06T10:37:58.192 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-06T10:37:58.192 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-06T10:37:58.230 INFO:tasks.ceph.mon.b:Started 2023-10-06T10:37:58.231 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-06T10:37:58.231 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-06T10:37:58.233 INFO:tasks.ceph.mon.c:Started 2023-10-06T10:37:58.233 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-06T10:37:58.233 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-06T10:37:58.233 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-06T10:37:58.235 INFO:tasks.ceph.mgr.y:Started 2023-10-06T10:37:58.235 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-06T10:37:58.236 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-06T10:37:58.237 INFO:tasks.ceph.mgr.x:Started 2023-10-06T10:37:58.237 INFO:tasks.ceph:Setting crush tunables to default 2023-10-06T10:37:58.238 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd crush tunables default 2023-10-06T10:37:58.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.352+0000 7f26ce72d700 1 Processor -- start 2023-10-06T10:37:58.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.352+0000 7f26ce72d700 1 -- start start 2023-10-06T10:37:58.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.353+0000 7f26ce72d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c81595b0 0x7f26c80af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:58.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.353+0000 7f26ce72d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f26c80c6f60 0x7f26c80c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:58.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.353+0000 7f26ce72d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f26c8068470 0x7f26c80688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:58.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.353+0000 7f26ce72d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26c80678f0 con 0x7f26c80c6f60 2023-10-06T10:37:58.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.353+0000 7f26ce72d700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26c80c7910 con 0x7f26c8068470 2023-10-06T10:37:58.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.353+0000 7f26ce72d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c8069d10 con 0x7f26c81595b0 2023-10-06T10:37:58.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.353+0000 7f26c77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f26c80c6f60 0x7f26c80c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:58.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.353+0000 7f26c77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f26c80c6f60 0x7f26c80c73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44396/0 (socket says 172.21.15.92:44396) 2023-10-06T10:37:58.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.353+0000 7f26c77fe700 1 -- 172.21.15.92:0/1264516703 learned_addr learned my addr 172.21.15.92:0/1264516703 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:37:58.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.354+0000 7f26c77fe700 1 -- 172.21.15.92:0/1264516703 >> v2:172.21.15.92:3300/0 conn(0x7f26c80c6f60 msgr2=0x7f26c80c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-06T10:37:58.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.354+0000 7f26c77fe700 1 -- 172.21.15.92:0/1264516703 >> v2:172.21.15.92:3300/0 conn(0x7f26c80c6f60 msgr2=0x7f26c80c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T10:37:58.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.354+0000 7f26c77fe700 1 --2- 172.21.15.92:0/1264516703 >> v2:172.21.15.92:3300/0 conn(0x7f26c80c6f60 0x7f26c80c73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T10:37:58.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.354+0000 7f26c77fe700 1 --2- 172.21.15.92:0/1264516703 >> v2:172.21.15.92:3300/0 conn(0x7f26c80c6f60 0x7f26c80c73d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T10:37:58.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.354+0000 7f26cccca700 1 --2- 172.21.15.92:0/1264516703 >> v2:172.21.15.124:3300/0 conn(0x7f26c8068470 0x7f26c80688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:58.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.354+0000 7f26c7fff700 1 --2- 172.21.15.92:0/1264516703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c81595b0 0x7f26c80af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:58.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.354+0000 7f26cccca700 1 -- 172.21.15.92:0/1264516703 >> v2:172.21.15.124:3300/0 conn(0x7f26c8068470 msgr2=0x7f26c80688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T10:37:58.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.354+0000 7f26cccca700 1 -- 172.21.15.92:0/1264516703 >> v2:172.21.15.124:3300/0 conn(0x7f26c8068470 msgr2=0x7f26c80688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T10:37:58.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.354+0000 7f26cccca700 1 --2- 172.21.15.92:0/1264516703 >> v2:172.21.15.124:3300/0 conn(0x7f26c8068470 0x7f26c80688e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T10:37:58.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.354+0000 7f26cccca700 1 --2- 172.21.15.92:0/1264516703 >> v2:172.21.15.124:3300/0 conn(0x7f26c8068470 0x7f26c80688e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T10:37:58.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.354+0000 7f26c7fff700 1 -- 172.21.15.92:0/1264516703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c81595b0 msgr2=0x7f26c80af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T10:37:58.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.354+0000 7f26c7fff700 1 -- 172.21.15.92:0/1264516703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c81595b0 msgr2=0x7f26c80af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T10:37:58.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.354+0000 7f26c7fff700 1 --2- 172.21.15.92:0/1264516703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c81595b0 0x7f26c80af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T10:37:58.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.354+0000 7f26c7fff700 1 --2- 172.21.15.92:0/1264516703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c81595b0 0x7f26c80af7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T10:37:58.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.555+0000 7f26c77fe700 1 --2- 172.21.15.92:0/1264516703 >> v2:172.21.15.92:3300/0 conn(0x7f26c80c6f60 0x7f26c80c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:58.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.555+0000 7f26c7fff700 1 --2- 172.21.15.92:0/1264516703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c81595b0 0x7f26c80af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:58.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.555+0000 7f26cccca700 1 --2- 172.21.15.92:0/1264516703 >> v2:172.21.15.124:3300/0 conn(0x7f26c8068470 0x7f26c80688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:58.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.555+0000 7f26c77fe700 1 -- 172.21.15.92:0/1264516703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c81595b0 msgr2=0x7f26c80af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:37:58.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.555+0000 7f26c77fe700 1 --2- 172.21.15.92:0/1264516703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c81595b0 0x7f26c80af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:58.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.555+0000 7f26c77fe700 1 -- 172.21.15.92:0/1264516703 >> v2:172.21.15.124:3300/0 conn(0x7f26c8068470 msgr2=0x7f26c80688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:37:58.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.555+0000 7f26c77fe700 1 --2- 172.21.15.92:0/1264516703 >> v2:172.21.15.124:3300/0 conn(0x7f26c8068470 0x7f26c80688e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:58.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.555+0000 7f26c77fe700 1 -- 172.21.15.92:0/1264516703 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c806a890 con 0x7f26c80c6f60 2023-10-06T10:37:58.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.555+0000 7f26c77fe700 1 --2- 172.21.15.92:0/1264516703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c80c6f60 0x7f26c80c73d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f26c815d180 tx=0x7f26bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177c3293d7c8b486 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:37:58.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.555+0000 7f26c6ffd700 1 -- 172.21.15.92:0/1264516703 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc00c070 con 0x7f26c80c6f60 2023-10-06T10:37:58.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.555+0000 7f26c6ffd700 1 -- 172.21.15.92:0/1264516703 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26bc013410 con 0x7f26c80c6f60 2023-10-06T10:37:58.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.555+0000 7f26c6ffd700 1 -- 172.21.15.92:0/1264516703 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc0137b0 con 0x7f26c80c6f60 2023-10-06T10:37:58.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.556+0000 7f26ce72d700 1 -- 172.21.15.92:0/1264516703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c80c6f60 msgr2=0x7f26c80c73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:58.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.556+0000 7f26ce72d700 1 --2- 172.21.15.92:0/1264516703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c80c6f60 0x7f26c80c73d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f26c815d180 tx=0x7f26bc008040 comp rx=0 tx=0).stop 2023-10-06T10:37:58.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.556+0000 7f26ce72d700 1 -- 172.21.15.92:0/1264516703 shutdown_connections 2023-10-06T10:37:58.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.556+0000 7f26ce72d700 1 --2- 172.21.15.92:0/1264516703 >> v2:172.21.15.124:3300/0 conn(0x7f26c8068470 0x7f26c80688e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:58.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.556+0000 7f26ce72d700 1 --2- 172.21.15.92:0/1264516703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c81595b0 0x7f26c80af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:58.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.556+0000 7f26ce72d700 1 --2- 172.21.15.92:0/1264516703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c80c6f60 0x7f26c80c73d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:58.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.556+0000 7f26ce72d700 1 -- 172.21.15.92:0/1264516703 >> 172.21.15.92:0/1264516703 conn(0x7f26c80cd3f0 msgr2=0x7f26c80c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:37:58.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.556+0000 7f26ce72d700 1 -- 172.21.15.92:0/1264516703 shutdown_connections 2023-10-06T10:37:58.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.556+0000 7f26ce72d700 1 -- 172.21.15.92:0/1264516703 wait complete. 2023-10-06T10:37:58.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.557+0000 7f26ce72d700 1 Processor -- start 2023-10-06T10:37:58.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.558+0000 7f26ce72d700 1 -- start start 2023-10-06T10:37:58.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.558+0000 7f26ce72d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26c8068470 0x7f26c814ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:58.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.558+0000 7f26ce72d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c80c6f60 0x7f26c815c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:58.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.558+0000 7f26ce72d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c81595b0 0x7f26c80aa610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:58.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.558+0000 7f26ce72d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c814d370 con 0x7f26c80c6f60 2023-10-06T10:37:58.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.558+0000 7f26ce72d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c8159980 con 0x7f26c8068470 2023-10-06T10:37:58.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.558+0000 7f26ce72d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c8159f40 con 0x7f26c81595b0 2023-10-06T10:37:58.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.558+0000 7f26c77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c80c6f60 0x7f26c815c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:58.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.559+0000 7f26c7fff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26c8068470 0x7f26c814ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:58.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.559+0000 7f26c77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c80c6f60 0x7f26c815c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44440/0 (socket says 172.21.15.92:44440) 2023-10-06T10:37:58.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.559+0000 7f26c77fe700 1 -- 172.21.15.92:0/505223278 learned_addr learned my addr 172.21.15.92:0/505223278 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:37:58.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.559+0000 7f26cccca700 1 --2- 172.21.15.92:0/505223278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c81595b0 0x7f26c80aa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:58.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.559+0000 7f26c77fe700 1 -- 172.21.15.92:0/505223278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c81595b0 msgr2=0x7f26c80aa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:58.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.559+0000 7f26c77fe700 1 --2- 172.21.15.92:0/505223278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c81595b0 0x7f26c80aa610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:58.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.559+0000 7f26c77fe700 1 -- 172.21.15.92:0/505223278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26c8068470 msgr2=0x7f26c814ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:58.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.559+0000 7f26c77fe700 1 --2- 172.21.15.92:0/505223278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26c8068470 0x7f26c814ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:58.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.559+0000 7f26c77fe700 1 -- 172.21.15.92:0/505223278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26bc009590 con 0x7f26c80c6f60 2023-10-06T10:37:58.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.559+0000 7f26c77fe700 1 --2- 172.21.15.92:0/505223278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c80c6f60 0x7f26c815c040 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009560 tx=0x7f26bc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:37:58.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.560+0000 7f26affff700 1 -- 172.21.15.92:0/505223278 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc021070 con 0x7f26c80c6f60 2023-10-06T10:37:58.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.560+0000 7f26affff700 1 -- 172.21.15.92:0/505223278 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26bc01c520 con 0x7f26c80c6f60 2023-10-06T10:37:58.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.560+0000 7f26ce72d700 1 -- 172.21.15.92:0/505223278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c815cb00 con 0x7f26c80c6f60 2023-10-06T10:37:58.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.560+0000 7f26affff700 1 -- 172.21.15.92:0/505223278 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc01c900 con 0x7f26c80c6f60 2023-10-06T10:37:58.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.560+0000 7f26ce72d700 1 -- 172.21.15.92:0/505223278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c8159bd0 con 0x7f26c80c6f60 2023-10-06T10:37:58.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.560+0000 7f26affff700 1 -- 172.21.15.92:0/505223278 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f26bc025070 con 0x7f26c80c6f60 2023-10-06T10:37:58.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.560+0000 7f26affff700 1 -- 172.21.15.92:0/505223278 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f26bc031750 con 0x7f26c80c6f60 2023-10-06T10:37:58.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.561+0000 7f26ce72d700 1 -- 172.21.15.92:0/505223278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26c815a0b0 con 0x7f26c80c6f60 2023-10-06T10:37:58.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.563+0000 7f26affff700 1 -- 172.21.15.92:0/505223278 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f26bc02c070 con 0x7f26c80c6f60 2023-10-06T10:37:58.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.638+0000 7f26ce72d700 1 -- 172.21.15.92:0/505223278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f26c8064a90 con 0x7f26c80c6f60 2023-10-06T10:37:58.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.644+0000 7f26affff700 1 -- 172.21.15.92:0/505223278 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f26bc017020 con 0x7f26c80c6f60 2023-10-06T10:37:58.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.645+0000 7f26ce72d700 1 -- 172.21.15.92:0/505223278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c80c6f60 msgr2=0x7f26c815c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:58.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.645+0000 7f26ce72d700 1 --2- 172.21.15.92:0/505223278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c80c6f60 0x7f26c815c040 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009560 tx=0x7f26bc01b000 comp rx=0 tx=0).stop 2023-10-06T10:37:58.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.645+0000 7f26ce72d700 1 -- 172.21.15.92:0/505223278 shutdown_connections 2023-10-06T10:37:58.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.645+0000 7f26ce72d700 1 --2- 172.21.15.92:0/505223278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c81595b0 0x7f26c80aa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:58.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.645+0000 7f26ce72d700 1 --2- 172.21.15.92:0/505223278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26c8068470 0x7f26c814ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:58.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.645+0000 7f26ce72d700 1 --2- 172.21.15.92:0/505223278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c80c6f60 0x7f26c815c040 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:58.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.645+0000 7f26ce72d700 1 -- 172.21.15.92:0/505223278 >> 172.21.15.92:0/505223278 conn(0x7f26c80cd3f0 msgr2=0x7f26c80caec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:37:58.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.645+0000 7f26ce72d700 1 -- 172.21.15.92:0/505223278 shutdown_connections 2023-10-06T10:37:58.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:37:58.646+0000 7f26ce72d700 1 -- 172.21.15.92:0/505223278 wait complete. 2023-10-06T10:37:58.649 INFO:teuthology.orchestra.run.smithi092.stderr:adjusted tunables profile to default 2023-10-06T10:37:58.659 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-06T10:37:58.659 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:58.659 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-06T10:37:58.683 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:58.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-06T10:37:58.747 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:58.748 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-06T10:37:58.813 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T10:37:58.813 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-06T10:37:58.878 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:37:58.878 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-06T10:37:58.903 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:37:58.903 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-06T10:37:58.968 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:37:58.968 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-06T10:37:59.032 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-06T10:37:59.033 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-06T10:37:59.096 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new 07b2c439-9b78-4ecf-baec-34cf38431e7a 0 2023-10-06T10:37:59.252 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.251+0000 7fdefbfff700 1 Processor -- start 2023-10-06T10:37:59.253 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.251+0000 7fdefbfff700 1 -- start start 2023-10-06T10:37:59.255 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.252+0000 7fdefbfff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdefc0b2620 0x7fdefc0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:59.255 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.252+0000 7fdefbfff700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdefc0cf1b0 0x7fdefc0cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:59.255 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.252+0000 7fdefbfff700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdefc0cc970 con 0x7fdefc0b29f0 2023-10-06T10:37:59.256 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.252+0000 7fdefbfff700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdefc0cbd60 con 0x7fdefc0cf1b0 2023-10-06T10:37:59.256 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.252+0000 7fdefbfff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0cc040 con 0x7fdefc0b2620 2023-10-06T10:37:59.256 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.252+0000 7fdefb7fe700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdefc0cf1b0 0x7fdefc0cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:59.256 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.252+0000 7fdefaffd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdefc0b2620 0x7fdefc0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:59.256 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.252+0000 7fdefb7fe700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdefc0cf1b0 0x7fdefc0cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:45762/0 (socket says 172.21.15.124:45762) 2023-10-06T10:37:59.256 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.252+0000 7fdefaffd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdefc0b2620 0x7fdefc0cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.124:41998/0 (socket says 172.21.15.124:41998) 2023-10-06T10:37:59.256 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.252+0000 7fdefb7fe700 1 -- 172.21.15.124:0/4108813382 learned_addr learned my addr 172.21.15.124:0/4108813382 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-06T10:37:59.256 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.252+0000 7fdefb7fe700 1 -- 172.21.15.124:0/4108813382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdefc0b2620 msgr2=0x7fdefc0cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:37:59.256 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.252+0000 7fdefb7fe700 1 --2- 172.21.15.124:0/4108813382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdefc0b2620 0x7fdefc0cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.256 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.252+0000 7fdefb7fe700 1 -- 172.21.15.124:0/4108813382 >> v1:172.21.15.124:6789/0 conn(0x7fdefc0b29f0 legacy=0x7fdefc0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:59.257 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.252+0000 7fdef9ffb700 1 -- 172.21.15.124:0/4108813382 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2428006875 0 0) 0x7fdefc0cc970 con 0x7fdefc0b29f0 2023-10-06T10:37:59.257 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.252+0000 7fdefb7fe700 1 -- 172.21.15.124:0/4108813382 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc0cc320 con 0x7fdefc0cf1b0 2023-10-06T10:37:59.257 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.252+0000 7fdefaffd700 1 --2- 172.21.15.124:0/4108813382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdefc0b2620 0x7fdefc0cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:37:59.257 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.252+0000 7fdefb7fe700 1 --2- 172.21.15.124:0/4108813382 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdefc0cf1b0 0x7fdefc0cd1f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009f20 tx=0x7fdef4009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f5ffacaaf15c81c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:37:59.257 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.252+0000 7fdef9ffb700 1 -- 172.21.15.124:0/4108813382 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef400e070 con 0x7fdefc0cf1b0 2023-10-06T10:37:59.257 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.252+0000 7fdef9ffb700 1 -- 172.21.15.124:0/4108813382 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef400b9d0 con 0x7fdefc0cf1b0 2023-10-06T10:37:59.257 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.252+0000 7fdef9ffb700 1 -- 172.21.15.124:0/4108813382 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef40092e0 con 0x7fdefc0cf1b0 2023-10-06T10:37:59.257 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.253+0000 7fdefbfff700 1 -- 172.21.15.124:0/4108813382 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdefc0cf1b0 msgr2=0x7fdefc0cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:59.257 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.253+0000 7fdefbfff700 1 --2- 172.21.15.124:0/4108813382 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdefc0cf1b0 0x7fdefc0cd1f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009f20 tx=0x7fdef4009840 comp rx=0 tx=0).stop 2023-10-06T10:37:59.257 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.253+0000 7fdefbfff700 1 -- 172.21.15.124:0/4108813382 shutdown_connections 2023-10-06T10:37:59.258 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.253+0000 7fdefbfff700 1 --2- 172.21.15.124:0/4108813382 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdefc0cf1b0 0x7fdefc0cd1f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.258 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.253+0000 7fdefbfff700 1 --2- 172.21.15.124:0/4108813382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdefc0b2620 0x7fdefc0cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.258 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.253+0000 7fdefbfff700 1 -- 172.21.15.124:0/4108813382 >> 172.21.15.124:0/4108813382 conn(0x7fdefc099520 msgr2=0x7fdefc06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:37:59.258 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.253+0000 7fdefbfff700 1 -- 172.21.15.124:0/4108813382 shutdown_connections 2023-10-06T10:37:59.258 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.253+0000 7fdefbfff700 1 -- 172.21.15.124:0/4108813382 wait complete. 2023-10-06T10:37:59.261 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.255+0000 7fdefbfff700 1 Processor -- start 2023-10-06T10:37:59.261 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.255+0000 7fdefbfff700 1 -- start start 2023-10-06T10:37:59.261 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.256+0000 7fdefbfff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdefc0b2620 0x7fdefc155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:59.261 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.256+0000 7fdefbfff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdefc0b29f0 0x7fdefc0b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:59.261 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.256+0000 7fdefbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdefc0cf1b0 0x7fdefc0b5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:59.261 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.256+0000 7fdefbfff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc155a50 con 0x7fdefc0cf1b0 2023-10-06T10:37:59.261 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.256+0000 7fdefbfff700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc16d8f0 con 0x7fdefc0b29f0 2023-10-06T10:37:59.261 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.256+0000 7fdefbfff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc15cd90 con 0x7fdefc0b2620 2023-10-06T10:37:59.261 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.256+0000 7fdefa7fc700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdefc0b29f0 0x7fdefc0b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:59.262 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.256+0000 7fdefa7fc700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdefc0b29f0 0x7fdefc0b51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:45768/0 (socket says 172.21.15.124:45768) 2023-10-06T10:37:59.262 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.256+0000 7fdefa7fc700 1 -- 172.21.15.124:0/2316186049 learned_addr learned my addr 172.21.15.124:0/2316186049 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-06T10:37:59.262 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.256+0000 7fdefaffd700 1 --2- 172.21.15.124:0/2316186049 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdefc0b2620 0x7fdefc155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:59.262 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.256+0000 7fdefb7fe700 1 --2- 172.21.15.124:0/2316186049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdefc0cf1b0 0x7fdefc0b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:59.262 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.257+0000 7fdefa7fc700 1 -- 172.21.15.124:0/2316186049 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdefc0b2620 msgr2=0x7fdefc155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:59.262 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.257+0000 7fdefa7fc700 1 --2- 172.21.15.124:0/2316186049 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdefc0b2620 0x7fdefc155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.262 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.257+0000 7fdefa7fc700 1 -- 172.21.15.124:0/2316186049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdefc0cf1b0 msgr2=0x7fdefc0b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:59.262 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.257+0000 7fdefa7fc700 1 --2- 172.21.15.124:0/2316186049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdefc0cf1b0 0x7fdefc0b5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.262 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.257+0000 7fdefa7fc700 1 -- 172.21.15.124:0/2316186049 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeec0093c0 con 0x7fdefc0b29f0 2023-10-06T10:37:59.262 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.257+0000 7fdefb7fe700 1 --2- 172.21.15.124:0/2316186049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdefc0cf1b0 0x7fdefc0b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:37:59.262 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.257+0000 7fdefa7fc700 1 --2- 172.21.15.124:0/2316186049 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdefc0b29f0 0x7fdefc0b51e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdef000c7e0 tx=0x7fdef0007df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:37:59.263 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.257+0000 7fdee2ffd700 1 -- 172.21.15.124:0/2316186049 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0011590 con 0x7fdefc0b29f0 2023-10-06T10:37:59.263 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.257+0000 7fdee2ffd700 1 -- 172.21.15.124:0/2316186049 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef0011710 con 0x7fdefc0b29f0 2023-10-06T10:37:59.263 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.257+0000 7fdee2ffd700 1 -- 172.21.15.124:0/2316186049 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0011ad0 con 0x7fdefc0b29f0 2023-10-06T10:37:59.263 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.257+0000 7fdefbfff700 1 -- 172.21.15.124:0/2316186049 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef4009ae0 con 0x7fdefc0b29f0 2023-10-06T10:37:59.263 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.257+0000 7fdefbfff700 1 -- 172.21.15.124:0/2316186049 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdefc15d350 con 0x7fdefc0b29f0 2023-10-06T10:37:59.263 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.257+0000 7fdee2ffd700 1 -- 172.21.15.124:0/2316186049 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fdef001c6c0 con 0x7fdefc0b29f0 2023-10-06T10:37:59.263 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.257+0000 7fdee2ffd700 1 -- 172.21.15.124:0/2316186049 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7fdef0012770 con 0x7fdefc0b29f0 2023-10-06T10:37:59.263 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.258+0000 7fdefbfff700 1 -- 172.21.15.124:0/2316186049 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdefc05f230 con 0x7fdefc0b29f0 2023-10-06T10:37:59.264 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.260+0000 7fdee2ffd700 1 -- 172.21.15.124:0/2316186049 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fdef0017070 con 0x7fdefc0b29f0 2023-10-06T10:37:59.342 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.341+0000 7fdefbfff700 1 -- 172.21.15.124:0/2316186049 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd new", "uuid": "07b2c439-9b78-4ecf-baec-34cf38431e7a", "id": 0} v 0) v1 -- 0x7fdefc0cc600 con 0x7fdefc0b29f0 2023-10-06T10:37:59.348 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.346+0000 7fdee2ffd700 1 -- 172.21.15.124:0/2316186049 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "07b2c439-9b78-4ecf-baec-34cf38431e7a", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdef001cb00 con 0x7fdefc0b29f0 2023-10-06T10:37:59.348 INFO:teuthology.orchestra.run.smithi124.stdout:0 2023-10-06T10:37:59.349 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.347+0000 7fdefbfff700 1 -- 172.21.15.124:0/2316186049 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdefc0b29f0 msgr2=0x7fdefc0b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:59.349 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.347+0000 7fdefbfff700 1 --2- 172.21.15.124:0/2316186049 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdefc0b29f0 0x7fdefc0b51e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdef000c7e0 tx=0x7fdef0007df0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.350 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.347+0000 7fdefbfff700 1 -- 172.21.15.124:0/2316186049 shutdown_connections 2023-10-06T10:37:59.350 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.347+0000 7fdefbfff700 1 --2- 172.21.15.124:0/2316186049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdefc0cf1b0 0x7fdefc0b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.350 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.347+0000 7fdefbfff700 1 --2- 172.21.15.124:0/2316186049 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdefc0b2620 0x7fdefc155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.350 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.347+0000 7fdefbfff700 1 --2- 172.21.15.124:0/2316186049 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdefc0b29f0 0x7fdefc0b51e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.350 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.347+0000 7fdefbfff700 1 -- 172.21.15.124:0/2316186049 >> 172.21.15.124:0/2316186049 conn(0x7fdefc099520 msgr2=0x7fdefc06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:37:59.350 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.347+0000 7fdefbfff700 1 -- 172.21.15.124:0/2316186049 shutdown_connections 2023-10-06T10:37:59.350 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.347+0000 7fdefbfff700 1 -- 172.21.15.124:0/2316186049 wait complete. 2023-10-06T10:37:59.360 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new c8a0b0c4-3d96-4679-b5c3-905d147b24fb 1 2023-10-06T10:37:59.475 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.473+0000 7faf61747700 1 Processor -- start 2023-10-06T10:37:59.475 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.473+0000 7faf61747700 1 -- start start 2023-10-06T10:37:59.477 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.474+0000 7faf61747700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7faf5c159830 0x7faf5c0ac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:59.478 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.474+0000 7faf61747700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7faf5c0ca7b0 0x7faf5c0675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:59.478 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.474+0000 7faf61747700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf5c0acf20 0x7faf5c067af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:59.478 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.474+0000 7faf61747700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5c0ca320 con 0x7faf5c0ca7b0 2023-10-06T10:37:59.478 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.474+0000 7faf61747700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5c159c00 con 0x7faf5c159830 2023-10-06T10:37:59.478 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.474+0000 7faf61747700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c0ad480 con 0x7faf5c0acf20 2023-10-06T10:37:59.478 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.474+0000 7faf5b7fe700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf5c0acf20 0x7faf5c067af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:59.478 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.474+0000 7faf5affd700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7faf5c159830 0x7faf5c0ac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:59.478 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.474+0000 7faf5b7fe700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf5c0acf20 0x7faf5c067af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.124:42016/0 (socket says 172.21.15.124:42016) 2023-10-06T10:37:59.478 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.474+0000 7faf5b7fe700 1 -- 172.21.15.124:0/598174408 learned_addr learned my addr 172.21.15.124:0/598174408 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-06T10:37:59.478 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.474+0000 7faf5a7fc700 1 --2- 172.21.15.124:0/598174408 >> v2:172.21.15.92:3300/0 conn(0x7faf5c0ca7b0 0x7faf5c0675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:59.479 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.474+0000 7faf5affd700 1 -- 172.21.15.124:0/598174408 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf5c0acf20 msgr2=0x7faf5c067af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:37:59.479 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.474+0000 7faf5affd700 1 --2- 172.21.15.124:0/598174408 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf5c0acf20 0x7faf5c067af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.479 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.474+0000 7faf5affd700 1 -- 172.21.15.124:0/598174408 >> v2:172.21.15.92:3300/0 conn(0x7faf5c0ca7b0 msgr2=0x7faf5c0675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:37:59.479 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.474+0000 7faf5affd700 1 --2- 172.21.15.124:0/598174408 >> v2:172.21.15.92:3300/0 conn(0x7faf5c0ca7b0 0x7faf5c0675b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.479 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.474+0000 7faf5affd700 1 -- 172.21.15.124:0/598174408 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5c0c99f0 con 0x7faf5c159830 2023-10-06T10:37:59.479 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.474+0000 7faf5b7fe700 1 --2- 172.21.15.124:0/598174408 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf5c0acf20 0x7faf5c067af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:37:59.479 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.475+0000 7faf5affd700 1 --2- 172.21.15.124:0/598174408 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf5c159830 0x7faf5c0ac9e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faf5c068690 tx=0x7faf500091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b29b6e6c2a71f39d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:37:59.479 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.475+0000 7faf59ffb700 1 -- 172.21.15.124:0/598174408 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf50011630 con 0x7faf5c159830 2023-10-06T10:37:59.479 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.475+0000 7faf59ffb700 1 -- 172.21.15.124:0/598174408 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf500117b0 con 0x7faf5c159830 2023-10-06T10:37:59.479 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.475+0000 7faf59ffb700 1 -- 172.21.15.124:0/598174408 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf50011b70 con 0x7faf5c159830 2023-10-06T10:37:59.479 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.475+0000 7faf61747700 1 -- 172.21.15.124:0/598174408 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf5c159830 msgr2=0x7faf5c0ac9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:59.480 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.475+0000 7faf61747700 1 --2- 172.21.15.124:0/598174408 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf5c159830 0x7faf5c0ac9e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faf5c068690 tx=0x7faf500091a0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.480 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.475+0000 7faf61747700 1 -- 172.21.15.124:0/598174408 shutdown_connections 2023-10-06T10:37:59.480 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.475+0000 7faf61747700 1 --2- 172.21.15.124:0/598174408 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf5c0acf20 0x7faf5c067af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.480 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.475+0000 7faf61747700 1 --2- 172.21.15.124:0/598174408 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf5c159830 0x7faf5c0ac9e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.480 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.475+0000 7faf61747700 1 --2- 172.21.15.124:0/598174408 >> v2:172.21.15.92:3300/0 conn(0x7faf5c0ca7b0 0x7faf5c0675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.480 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.475+0000 7faf61747700 1 -- 172.21.15.124:0/598174408 >> 172.21.15.124:0/598174408 conn(0x7faf5c069150 msgr2=0x7faf5c06a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:37:59.480 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.475+0000 7faf61747700 1 -- 172.21.15.124:0/598174408 shutdown_connections 2023-10-06T10:37:59.480 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.475+0000 7faf61747700 1 -- 172.21.15.124:0/598174408 wait complete. 2023-10-06T10:37:59.483 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.477+0000 7faf61747700 1 Processor -- start 2023-10-06T10:37:59.483 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.477+0000 7faf61747700 1 -- start start 2023-10-06T10:37:59.483 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.477+0000 7faf61747700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf5c0acf20 0x7faf5c0aac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:59.483 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.477+0000 7faf61747700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf5c0ca7b0 0x7faf5c0a9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:59.483 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.477+0000 7faf61747700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf5c159830 0x7faf5c14c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:59.483 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.477+0000 7faf61747700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c0ac470 con 0x7faf5c0ca7b0 2023-10-06T10:37:59.483 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.477+0000 7faf61747700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c0a9e50 con 0x7faf5c159830 2023-10-06T10:37:59.483 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.477+0000 7faf61747700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c14c970 con 0x7faf5c0acf20 2023-10-06T10:37:59.484 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.477+0000 7faf5b7fe700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf5c159830 0x7faf5c14c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:59.484 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.477+0000 7faf5affd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf5c0acf20 0x7faf5c0aac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:59.484 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.478+0000 7faf5b7fe700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf5c159830 0x7faf5c14c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:45798/0 (socket says 172.21.15.124:45798) 2023-10-06T10:37:59.484 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.478+0000 7faf5b7fe700 1 -- 172.21.15.124:0/3810607981 learned_addr learned my addr 172.21.15.124:0/3810607981 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-06T10:37:59.484 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.478+0000 7faf5a7fc700 1 --2- 172.21.15.124:0/3810607981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf5c0ca7b0 0x7faf5c0a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:59.484 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.478+0000 7faf5b7fe700 1 -- 172.21.15.124:0/3810607981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf5c0acf20 msgr2=0x7faf5c0aac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:59.484 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.478+0000 7faf5b7fe700 1 --2- 172.21.15.124:0/3810607981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf5c0acf20 0x7faf5c0aac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.484 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.478+0000 7faf5b7fe700 1 -- 172.21.15.124:0/3810607981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf5c0ca7b0 msgr2=0x7faf5c0a9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:59.484 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.478+0000 7faf5b7fe700 1 --2- 172.21.15.124:0/3810607981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf5c0ca7b0 0x7faf5c0a9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.484 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.478+0000 7faf5b7fe700 1 -- 172.21.15.124:0/3810607981 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf54009ae0 con 0x7faf5c159830 2023-10-06T10:37:59.484 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.478+0000 7faf5affd700 1 --2- 172.21.15.124:0/3810607981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf5c0acf20 0x7faf5c0aac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:37:59.485 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.478+0000 7faf5b7fe700 1 --2- 172.21.15.124:0/3810607981 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf5c159830 0x7faf5c14c430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faf54009f30 tx=0x7faf5400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:37:59.485 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.478+0000 7faf42ffd700 1 -- 172.21.15.124:0/3810607981 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf54018070 con 0x7faf5c159830 2023-10-06T10:37:59.485 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.478+0000 7faf42ffd700 1 -- 172.21.15.124:0/3810607981 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5400f040 con 0x7faf5c159830 2023-10-06T10:37:59.485 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.478+0000 7faf61747700 1 -- 172.21.15.124:0/3810607981 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf50009420 con 0x7faf5c159830 2023-10-06T10:37:59.485 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.478+0000 7faf42ffd700 1 -- 172.21.15.124:0/3810607981 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5400cb70 con 0x7faf5c159830 2023-10-06T10:37:59.485 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.478+0000 7faf61747700 1 -- 172.21.15.124:0/3810607981 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf5c0abb40 con 0x7faf5c159830 2023-10-06T10:37:59.485 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.478+0000 7faf42ffd700 1 -- 172.21.15.124:0/3810607981 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7faf54003020 con 0x7faf5c159830 2023-10-06T10:37:59.485 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.478+0000 7faf42ffd700 1 -- 172.21.15.124:0/3810607981 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7faf5401d410 con 0x7faf5c159830 2023-10-06T10:37:59.485 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.479+0000 7faf61747700 1 -- 172.21.15.124:0/3810607981 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5c0ac5e0 con 0x7faf5c159830 2023-10-06T10:37:59.485 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.481+0000 7faf42ffd700 1 -- 172.21.15.124:0/3810607981 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7faf54014480 con 0x7faf5c159830 2023-10-06T10:37:59.582 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.580+0000 7faf61747700 1 -- 172.21.15.124:0/3810607981 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c8a0b0c4-3d96-4679-b5c3-905d147b24fb", "id": 1} v 0) v1 -- 0x7faf5c05f230 con 0x7faf5c159830 2023-10-06T10:37:59.587 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.586+0000 7faf42ffd700 1 -- 172.21.15.124:0/3810607981 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c8a0b0c4-3d96-4679-b5c3-905d147b24fb", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7faf5402c0b0 con 0x7faf5c159830 2023-10-06T10:37:59.587 INFO:teuthology.orchestra.run.smithi124.stdout:1 2023-10-06T10:37:59.588 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.587+0000 7faf61747700 1 -- 172.21.15.124:0/3810607981 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf5c159830 msgr2=0x7faf5c14c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:59.589 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.587+0000 7faf61747700 1 --2- 172.21.15.124:0/3810607981 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf5c159830 0x7faf5c14c430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faf54009f30 tx=0x7faf5400b040 comp rx=0 tx=0).stop 2023-10-06T10:37:59.589 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.587+0000 7faf61747700 1 -- 172.21.15.124:0/3810607981 shutdown_connections 2023-10-06T10:37:59.589 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.587+0000 7faf61747700 1 --2- 172.21.15.124:0/3810607981 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf5c159830 0x7faf5c14c430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.589 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.587+0000 7faf61747700 1 --2- 172.21.15.124:0/3810607981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf5c0acf20 0x7faf5c0aac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.589 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.587+0000 7faf61747700 1 --2- 172.21.15.124:0/3810607981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf5c0ca7b0 0x7faf5c0a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.589 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.587+0000 7faf61747700 1 -- 172.21.15.124:0/3810607981 >> 172.21.15.124:0/3810607981 conn(0x7faf5c069150 msgr2=0x7faf5c06a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:37:59.589 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.587+0000 7faf61747700 1 -- 172.21.15.124:0/3810607981 shutdown_connections 2023-10-06T10:37:59.589 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.587+0000 7faf61747700 1 -- 172.21.15.124:0/3810607981 wait complete. 2023-10-06T10:37:59.600 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new d5138725-f0d1-4c57-99de-6dbda923cdd8 2 2023-10-06T10:37:59.713 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:37:59.712+0000 7f5645afc180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T10:37:59.720 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.715+0000 7f5da2ceb700 1 Processor -- start 2023-10-06T10:37:59.720 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.715+0000 7f5da2ceb700 1 -- start start 2023-10-06T10:37:59.720 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.716+0000 7f5da2ceb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5d9c150940 0x7f5d9c159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:59.720 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.716+0000 7f5da2ceb700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9c15ac30 con 0x7f5d9c0cd4d0 2023-10-06T10:37:59.721 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.716+0000 7f5da2ceb700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9c0cb6a0 con 0x7f5d9c15a160 2023-10-06T10:37:59.721 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.716+0000 7f5da2ceb700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d9c0cb090 con 0x7f5d9c150940 2023-10-06T10:37:59.721 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.716+0000 7f5d9bfff700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f5d9c15a160 0x7f5d9c0ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.124:60742/0 (socket says 172.21.15.124:60742) 2023-10-06T10:37:59.721 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.716+0000 7f5d9bfff700 1 -- 172.21.15.124:0/1271268687 learned_addr learned my addr 172.21.15.124:0/1271268687 (peer_addr_for_me v1:172.21.15.124:0/0) 2023-10-06T10:37:59.721 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.716+0000 7f5da0a87700 1 --2- 172.21.15.124:0/1271268687 >> v2:172.21.15.92:3300/0 conn(0x7f5d9c150940 0x7f5d9c159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:59.721 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.716+0000 7f5d9b7fe700 1 -- 172.21.15.124:0/1271268687 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685909875 0 0) 0x7f5d9c0cb6a0 con 0x7f5d9c15a160 2023-10-06T10:37:59.721 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.716+0000 7f5d9b7fe700 1 -- 172.21.15.124:0/1271268687 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d8c003570 con 0x7f5d9c15a160 2023-10-06T10:37:59.721 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.716+0000 7f5da0a87700 1 -- 172.21.15.124:0/1271268687 >> v1:172.21.15.124:6789/0 conn(0x7f5d9c15a160 legacy=0x7f5d9c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:59.721 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.716+0000 7f5da0a87700 1 -- 172.21.15.124:0/1271268687 >> v1:172.21.15.92:6789/0 conn(0x7f5d9c0cd4d0 legacy=0x7f5d9c0b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:59.722 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.716+0000 7f5da0a87700 1 -- 172.21.15.124:0/1271268687 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c15a670 con 0x7f5d9c150940 2023-10-06T10:37:59.722 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.717+0000 7f5da0a87700 1 --2- 172.21.15.124:0/1271268687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d9c150940 0x7f5d9c159c20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5d900098d0 tx=0x7f5d90009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24e1b8d02b0e870 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:37:59.722 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.717+0000 7f5d9b7fe700 1 -- 172.21.15.124:0/1271268687 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d90005c10 con 0x7f5d9c150940 2023-10-06T10:37:59.723 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.717+0000 7f5d9b7fe700 1 -- 172.21.15.124:0/1271268687 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d90005d70 con 0x7f5d9c150940 2023-10-06T10:37:59.723 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.717+0000 7f5d9b7fe700 1 -- 172.21.15.124:0/1271268687 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d90006280 con 0x7f5d9c150940 2023-10-06T10:37:59.723 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.717+0000 7f5da2ceb700 1 -- 172.21.15.124:0/1271268687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d9c150940 msgr2=0x7f5d9c159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:59.723 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.717+0000 7f5da2ceb700 1 --2- 172.21.15.124:0/1271268687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d9c150940 0x7f5d9c159c20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5d900098d0 tx=0x7f5d90009120 comp rx=0 tx=0).stop 2023-10-06T10:37:59.723 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.717+0000 7f5da2ceb700 1 -- 172.21.15.124:0/1271268687 shutdown_connections 2023-10-06T10:37:59.723 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.717+0000 7f5da2ceb700 1 --2- 172.21.15.124:0/1271268687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d9c150940 0x7f5d9c159c20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.723 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.717+0000 7f5da2ceb700 1 -- 172.21.15.124:0/1271268687 >> 172.21.15.124:0/1271268687 conn(0x7f5d9c099700 msgr2=0x7f5d9c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:37:59.723 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.717+0000 7f5da2ceb700 1 -- 172.21.15.124:0/1271268687 shutdown_connections 2023-10-06T10:37:59.723 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.717+0000 7f5da2ceb700 1 -- 172.21.15.124:0/1271268687 wait complete. 2023-10-06T10:37:59.723 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.719+0000 7f5da2ceb700 1 Processor -- start 2023-10-06T10:37:59.725 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.719+0000 7f5da2ceb700 1 -- start start 2023-10-06T10:37:59.726 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.719+0000 7f5da2ceb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d9c0cd4d0 0x7f5d9c0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:59.726 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.719+0000 7f5da2ceb700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d9c15a160 0x7f5d9c0b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:59.726 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.719+0000 7f5da2ceb700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d9c14fe00 0x7f5d9c151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:59.726 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.719+0000 7f5da2ceb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c1502a0 con 0x7f5d9c0cd4d0 2023-10-06T10:37:59.726 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.719+0000 7f5da2ceb700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0b4c30 con 0x7f5d9c14fe00 2023-10-06T10:37:59.726 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.719+0000 7f5da2ceb700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0b4f10 con 0x7f5d9c15a160 2023-10-06T10:37:59.726 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.720+0000 7f5da1288700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d9c14fe00 0x7f5d9c151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:59.726 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.720+0000 7f5d9bfff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d9c15a160 0x7f5d9c0b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:59.726 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.720+0000 7f5da1288700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d9c14fe00 0x7f5d9c151f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:45806/0 (socket says 172.21.15.124:45806) 2023-10-06T10:37:59.726 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.720+0000 7f5da1288700 1 -- 172.21.15.124:0/837684160 learned_addr learned my addr 172.21.15.124:0/837684160 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-06T10:37:59.726 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.720+0000 7f5da0a87700 1 --2- 172.21.15.124:0/837684160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d9c0cd4d0 0x7f5d9c0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:59.727 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.720+0000 7f5da1288700 1 -- 172.21.15.124:0/837684160 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d9c15a160 msgr2=0x7f5d9c0b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:59.727 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.720+0000 7f5da1288700 1 --2- 172.21.15.124:0/837684160 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d9c15a160 0x7f5d9c0b5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.727 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.720+0000 7f5da1288700 1 -- 172.21.15.124:0/837684160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d9c0cd4d0 msgr2=0x7f5d9c0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:59.727 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.720+0000 7f5da1288700 1 --2- 172.21.15.124:0/837684160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d9c0cd4d0 0x7f5d9c0b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.727 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.720+0000 7f5da1288700 1 -- 172.21.15.124:0/837684160 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d900093c0 con 0x7f5d9c14fe00 2023-10-06T10:37:59.727 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.720+0000 7f5da1288700 1 --2- 172.21.15.124:0/837684160 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d9c14fe00 0x7f5d9c151f10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400c8e0 tx=0x7f5d94001710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:37:59.727 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.720+0000 7f5d98ff9700 1 -- 172.21.15.124:0/837684160 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d94012630 con 0x7f5d9c14fe00 2023-10-06T10:37:59.727 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.720+0000 7f5d98ff9700 1 -- 172.21.15.124:0/837684160 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d94012790 con 0x7f5d9c14fe00 2023-10-06T10:37:59.727 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.720+0000 7f5da2ceb700 1 -- 172.21.15.124:0/837684160 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9c0b51f0 con 0x7f5d9c14fe00 2023-10-06T10:37:59.727 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.720+0000 7f5d98ff9700 1 -- 172.21.15.124:0/837684160 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d94012b50 con 0x7f5d9c14fe00 2023-10-06T10:37:59.727 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.720+0000 7f5da2ceb700 1 -- 172.21.15.124:0/837684160 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9c0b57b0 con 0x7f5d9c14fe00 2023-10-06T10:37:59.727 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.720+0000 7f5d98ff9700 1 -- 172.21.15.124:0/837684160 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f5d9401b400 con 0x7f5d9c14fe00 2023-10-06T10:37:59.728 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.720+0000 7f5d98ff9700 1 -- 172.21.15.124:0/837684160 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f5d9400a7e0 con 0x7f5d9c14fe00 2023-10-06T10:37:59.728 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.721+0000 7f5da2ceb700 1 -- 172.21.15.124:0/837684160 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9c0b6210 con 0x7f5d9c14fe00 2023-10-06T10:37:59.728 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.723+0000 7f5d98ff9700 1 -- 172.21.15.124:0/837684160 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f5d9c15ac30 con 0x7f5d9c14fe00 2023-10-06T10:37:59.754 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:37:59.752+0000 7f2f0258d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T10:37:59.807 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.806+0000 7f5da2ceb700 1 -- 172.21.15.124:0/837684160 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d5138725-f0d1-4c57-99de-6dbda923cdd8", "id": 2} v 0) v1 -- 0x7f5d9c064a90 con 0x7f5d9c14fe00 2023-10-06T10:37:59.811 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.810+0000 7f5d98ff9700 1 -- 172.21.15.124:0/837684160 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d5138725-f0d1-4c57-99de-6dbda923cdd8", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5d9401b890 con 0x7f5d9c14fe00 2023-10-06T10:37:59.811 INFO:teuthology.orchestra.run.smithi124.stdout:2 2023-10-06T10:37:59.812 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.810+0000 7f5da2ceb700 1 -- 172.21.15.124:0/837684160 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d9c14fe00 msgr2=0x7f5d9c151f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:59.812 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.810+0000 7f5da2ceb700 1 --2- 172.21.15.124:0/837684160 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d9c14fe00 0x7f5d9c151f10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400c8e0 tx=0x7f5d94001710 comp rx=0 tx=0).stop 2023-10-06T10:37:59.813 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.810+0000 7f5da2ceb700 1 -- 172.21.15.124:0/837684160 shutdown_connections 2023-10-06T10:37:59.813 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.810+0000 7f5da2ceb700 1 --2- 172.21.15.124:0/837684160 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d9c14fe00 0x7f5d9c151f10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.813 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.811+0000 7f5da2ceb700 1 --2- 172.21.15.124:0/837684160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d9c0cd4d0 0x7f5d9c0b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.813 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.811+0000 7f5da2ceb700 1 --2- 172.21.15.124:0/837684160 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d9c15a160 0x7f5d9c0b5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.813 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.811+0000 7f5da2ceb700 1 -- 172.21.15.124:0/837684160 >> 172.21.15.124:0/837684160 conn(0x7f5d9c099700 msgr2=0x7f5d9c159850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:37:59.813 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.811+0000 7f5da2ceb700 1 -- 172.21.15.124:0/837684160 shutdown_connections 2023-10-06T10:37:59.813 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.811+0000 7f5da2ceb700 1 -- 172.21.15.124:0/837684160 wait complete. 2023-10-06T10:37:59.824 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new c3210bd7-36b2-4ea4-969b-8bb51bb9ca76 3 2023-10-06T10:37:59.846 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:37:59.845+0000 7f5645afc180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T10:37:59.891 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:37:59.890+0000 7f2f0258d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T10:37:59.922 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:37:59.918+0000 7f5645afc180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T10:37:59.939 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.937+0000 7f79f6146700 1 Processor -- start 2023-10-06T10:37:59.939 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.937+0000 7f79f6146700 1 -- start start 2023-10-06T10:37:59.941 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.938+0000 7f79f6146700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f79f013c430 0x7f79f00ca950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:59.941 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.938+0000 7f79f6146700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f79f00cae90 0x7f79f0159b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:59.941 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.938+0000 7f79f6146700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f00ca5b0 con 0x7f79f015a080 2023-10-06T10:37:59.941 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.938+0000 7f79f6146700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f00cb330 con 0x7f79f00cae90 2023-10-06T10:37:59.941 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.938+0000 7f79f6146700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f015a8e0 con 0x7f79f013c430 2023-10-06T10:37:59.941 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.938+0000 7f79f5144700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f79f013c430 0x7f79f00ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:59.942 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.938+0000 7f79f5144700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f79f013c430 0x7f79f00ca950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:45820/0 (socket says 172.21.15.124:45820) 2023-10-06T10:37:59.942 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.938+0000 7f79f5144700 1 -- 172.21.15.124:0/3472727635 learned_addr learned my addr 172.21.15.124:0/3472727635 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-06T10:37:59.942 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.938+0000 7f79f4943700 1 --2- 172.21.15.124:0/3472727635 >> v2:172.21.15.92:3300/0 conn(0x7f79f00cae90 0x7f79f0159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:59.942 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.938+0000 7f79f5144700 1 -- 172.21.15.124:0/3472727635 >> v2:172.21.15.92:3300/0 conn(0x7f79f00cae90 msgr2=0x7f79f0159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:37:59.942 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.938+0000 7f79f5144700 1 --2- 172.21.15.124:0/3472727635 >> v2:172.21.15.92:3300/0 conn(0x7f79f00cae90 0x7f79f0159b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.942 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.938+0000 7f79f5144700 1 -- 172.21.15.124:0/3472727635 >> v1:172.21.15.124:6789/0 conn(0x7f79f015a080 legacy=0x7f79f015cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:59.942 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.938+0000 7f79f5144700 1 -- 172.21.15.124:0/3472727635 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f006d500 con 0x7f79f013c430 2023-10-06T10:37:59.942 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.938+0000 7f79f5144700 1 --2- 172.21.15.124:0/3472727635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79f013c430 0x7f79f00ca950 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0099b0 tx=0x7f79ec009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3e8a866ef6fd672 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:37:59.942 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.939+0000 7f79e7fff700 1 -- 172.21.15.124:0/3472727635 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec00b820 con 0x7f79f013c430 2023-10-06T10:37:59.942 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.939+0000 7f79e7fff700 1 -- 172.21.15.124:0/3472727635 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec00b980 con 0x7f79f013c430 2023-10-06T10:37:59.942 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.939+0000 7f79e7fff700 1 -- 172.21.15.124:0/3472727635 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec00bd20 con 0x7f79f013c430 2023-10-06T10:37:59.943 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.939+0000 7f79f6146700 1 -- 172.21.15.124:0/3472727635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79f013c430 msgr2=0x7f79f00ca950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:59.943 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.939+0000 7f79f6146700 1 --2- 172.21.15.124:0/3472727635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79f013c430 0x7f79f00ca950 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0099b0 tx=0x7f79ec009250 comp rx=0 tx=0).stop 2023-10-06T10:37:59.943 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.939+0000 7f79f6146700 1 -- 172.21.15.124:0/3472727635 shutdown_connections 2023-10-06T10:37:59.943 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.939+0000 7f79f6146700 1 --2- 172.21.15.124:0/3472727635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79f013c430 0x7f79f00ca950 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.943 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.939+0000 7f79f6146700 1 --2- 172.21.15.124:0/3472727635 >> v2:172.21.15.92:3300/0 conn(0x7f79f00cae90 0x7f79f0159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.943 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.939+0000 7f79f6146700 1 -- 172.21.15.124:0/3472727635 >> 172.21.15.124:0/3472727635 conn(0x7f79f006b330 msgr2=0x7f79f006b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:37:59.943 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.939+0000 7f79f6146700 1 -- 172.21.15.124:0/3472727635 shutdown_connections 2023-10-06T10:37:59.943 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.939+0000 7f79f6146700 1 -- 172.21.15.124:0/3472727635 wait complete. 2023-10-06T10:37:59.945 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.940+0000 7f79f6146700 1 Processor -- start 2023-10-06T10:37:59.945 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.940+0000 7f79f6146700 1 -- start start 2023-10-06T10:37:59.946 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.941+0000 7f79f6146700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79f00cae90 0x7f79f014f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:59.946 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.941+0000 7f79f6146700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79f013c430 0x7f79f014e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:59.946 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.941+0000 7f79f6146700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79f015a080 0x7f79f013da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:37:59.946 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.941+0000 7f79f6146700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f014fb00 con 0x7f79f013c430 2023-10-06T10:37:59.946 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.941+0000 7f79f6146700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00b1230 con 0x7f79f015a080 2023-10-06T10:37:59.946 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.941+0000 7f79f6146700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f014eb50 con 0x7f79f00cae90 2023-10-06T10:37:59.946 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.941+0000 7f79f5945700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79f015a080 0x7f79f013da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:59.946 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.941+0000 7f79f5144700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79f00cae90 0x7f79f014f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:59.946 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.941+0000 7f79f5945700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79f015a080 0x7f79f013da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:45828/0 (socket says 172.21.15.124:45828) 2023-10-06T10:37:59.946 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.941+0000 7f79f5144700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79f00cae90 0x7f79f014f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.124:42036/0 (socket says 172.21.15.124:42036) 2023-10-06T10:37:59.946 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.941+0000 7f79f5945700 1 -- 172.21.15.124:0/1686943383 learned_addr learned my addr 172.21.15.124:0/1686943383 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-06T10:37:59.946 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.941+0000 7f79f4943700 1 --2- 172.21.15.124:0/1686943383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79f013c430 0x7f79f014e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:37:59.947 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.942+0000 7f79f5144700 1 -- 172.21.15.124:0/1686943383 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79f015a080 msgr2=0x7f79f013da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:59.947 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.942+0000 7f79f5144700 1 --2- 172.21.15.124:0/1686943383 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79f015a080 0x7f79f013da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.947 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.942+0000 7f79f5144700 1 -- 172.21.15.124:0/1686943383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79f013c430 msgr2=0x7f79f014e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:37:59.947 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.942+0000 7f79f5144700 1 --2- 172.21.15.124:0/1686943383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79f013c430 0x7f79f014e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:37:59.947 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.942+0000 7f79f5144700 1 -- 172.21.15.124:0/1686943383 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec0094f0 con 0x7f79f00cae90 2023-10-06T10:37:59.947 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.942+0000 7f79f5945700 1 --2- 172.21.15.124:0/1686943383 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79f015a080 0x7f79f013da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:37:59.947 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.942+0000 7f79f5144700 1 --2- 172.21.15.124:0/1686943383 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79f00cae90 0x7f79f014f5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f79f00b0bd0 tx=0x7f79ec017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:37:59.947 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.942+0000 7f79e57fa700 1 -- 172.21.15.124:0/1686943383 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec00bd00 con 0x7f79f00cae90 2023-10-06T10:37:59.947 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.942+0000 7f79e57fa700 1 -- 172.21.15.124:0/1686943383 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec024440 con 0x7f79f00cae90 2023-10-06T10:37:59.947 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.942+0000 7f79f6146700 1 -- 172.21.15.124:0/1686943383 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f01521d0 con 0x7f79f00cae90 2023-10-06T10:37:59.947 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.942+0000 7f79e57fa700 1 -- 172.21.15.124:0/1686943383 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0247e0 con 0x7f79f00cae90 2023-10-06T10:37:59.948 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.942+0000 7f79f6146700 1 -- 172.21.15.124:0/1686943383 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f0152790 con 0x7f79f00cae90 2023-10-06T10:37:59.948 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.942+0000 7f79e57fa700 1 -- 172.21.15.124:0/1686943383 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f79ec02d440 con 0x7f79f00cae90 2023-10-06T10:37:59.948 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.942+0000 7f79e57fa700 1 -- 172.21.15.124:0/1686943383 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f79ec01a8c0 con 0x7f79f00cae90 2023-10-06T10:37:59.948 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.943+0000 7f79f6146700 1 -- 172.21.15.124:0/1686943383 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f005f230 con 0x7f79f00cae90 2023-10-06T10:37:59.948 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:37:59.947+0000 7f79e57fa700 1 -- 172.21.15.124:0/1686943383 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f79ec015020 con 0x7f79f00cae90 2023-10-06T10:37:59.971 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:37:59.969+0000 7f2f0258d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T10:38:00.030 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.029+0000 7f79f6146700 1 -- 172.21.15.124:0/1686943383 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c3210bd7-36b2-4ea4-969b-8bb51bb9ca76", "id": 3} v 0) v1 -- 0x7f79f013c800 con 0x7f79f00cae90 2023-10-06T10:38:00.034 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.033+0000 7f79e57fa700 1 -- 172.21.15.124:0/1686943383 <== mon.2 v2:172.21.15.124:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c3210bd7-36b2-4ea4-969b-8bb51bb9ca76", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f79ec024e50 con 0x7f79f00cae90 2023-10-06T10:38:00.034 INFO:teuthology.orchestra.run.smithi124.stdout:3 2023-10-06T10:38:00.036 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.034+0000 7f79f6146700 1 -- 172.21.15.124:0/1686943383 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79f00cae90 msgr2=0x7f79f014f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:00.036 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.034+0000 7f79f6146700 1 --2- 172.21.15.124:0/1686943383 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79f00cae90 0x7f79f014f5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f79f00b0bd0 tx=0x7f79ec017000 comp rx=0 tx=0).stop 2023-10-06T10:38:00.036 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.034+0000 7f79f6146700 1 -- 172.21.15.124:0/1686943383 shutdown_connections 2023-10-06T10:38:00.036 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.034+0000 7f79f6146700 1 --2- 172.21.15.124:0/1686943383 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79f015a080 0x7f79f013da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.036 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.034+0000 7f79f6146700 1 --2- 172.21.15.124:0/1686943383 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79f00cae90 0x7f79f014f5c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.036 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.034+0000 7f79f6146700 1 --2- 172.21.15.124:0/1686943383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79f013c430 0x7f79f014e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.036 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.034+0000 7f79f6146700 1 -- 172.21.15.124:0/1686943383 >> 172.21.15.124:0/1686943383 conn(0x7f79f006b330 msgr2=0x7f79f01560e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:00.036 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.034+0000 7f79f6146700 1 -- 172.21.15.124:0/1686943383 shutdown_connections 2023-10-06T10:38:00.036 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.034+0000 7f79f6146700 1 -- 172.21.15.124:0/1686943383 wait complete. 2023-10-06T10:38:00.047 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new 5d42097a-3896-4e09-b7e3-5a6702775f4c 4 2023-10-06T10:38:00.103 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:00.101+0000 7f5645afc180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T10:38:00.161 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:38:00.160+0000 7f2f0258d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T10:38:00.162 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.160+0000 7f2d4b38d700 1 Processor -- start 2023-10-06T10:38:00.162 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.160+0000 7f2d4b38d700 1 -- start start 2023-10-06T10:38:00.165 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.161+0000 7f2d4b38d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2d440b2620 0x7f2d440cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:00.165 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.161+0000 7f2d4b38d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d440b29f0 0x7f2d440cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:00.165 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.161+0000 7f2d4b38d700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d440cc970 con 0x7f2d440ceaa0 2023-10-06T10:38:00.165 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.161+0000 7f2d4b38d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d440cbd60 con 0x7f2d440b2620 2023-10-06T10:38:00.166 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.161+0000 7f2d4b38d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d440cc040 con 0x7f2d440b29f0 2023-10-06T10:38:00.166 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.161+0000 7f2d49b8a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d440b29f0 0x7f2d440cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:00.166 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.161+0000 7f2d4ab8c700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f2d440ceaa0 0x7f2d440cef40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.124:60760/0 (socket says 172.21.15.124:60760) 2023-10-06T10:38:00.166 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.161+0000 7f2d4ab8c700 1 -- 172.21.15.124:0/3535301948 learned_addr learned my addr 172.21.15.124:0/3535301948 (peer_addr_for_me v1:172.21.15.124:0/0) 2023-10-06T10:38:00.166 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.161+0000 7f2d4a38b700 1 --2- 172.21.15.124:0/3535301948 >> v2:172.21.15.92:3300/0 conn(0x7f2d440b2620 0x7f2d440cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:00.167 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.161+0000 7f2d49389700 1 -- 172.21.15.124:0/3535301948 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 982313037 0 0) 0x7f2d440cc970 con 0x7f2d440ceaa0 2023-10-06T10:38:00.167 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.161+0000 7f2d49389700 1 -- 172.21.15.124:0/3535301948 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d30003570 con 0x7f2d440ceaa0 2023-10-06T10:38:00.167 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.161+0000 7f2d49b8a700 1 -- 172.21.15.124:0/3535301948 >> v2:172.21.15.92:3300/0 conn(0x7f2d440b2620 msgr2=0x7f2d440cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:00.167 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.161+0000 7f2d49b8a700 1 --2- 172.21.15.124:0/3535301948 >> v2:172.21.15.92:3300/0 conn(0x7f2d440b2620 0x7f2d440cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.167 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.161+0000 7f2d49b8a700 1 -- 172.21.15.124:0/3535301948 >> v1:172.21.15.124:6789/0 conn(0x7f2d440ceaa0 legacy=0x7f2d440cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:00.167 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.161+0000 7f2d49b8a700 1 -- 172.21.15.124:0/3535301948 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d440cc320 con 0x7f2d440b29f0 2023-10-06T10:38:00.167 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.162+0000 7f2d49b8a700 1 --2- 172.21.15.124:0/3535301948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d440b29f0 0x7f2d440cd1f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2d340098e0 tx=0x7f2d340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b79e22d8ff8de743 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:00.167 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.162+0000 7f2d49389700 1 -- 172.21.15.124:0/3535301948 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d34011630 con 0x7f2d440b29f0 2023-10-06T10:38:00.167 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.162+0000 7f2d49389700 1 -- 172.21.15.124:0/3535301948 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d34011790 con 0x7f2d440b29f0 2023-10-06T10:38:00.167 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.162+0000 7f2d49389700 1 -- 172.21.15.124:0/3535301948 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d34011b30 con 0x7f2d440b29f0 2023-10-06T10:38:00.168 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.162+0000 7f2d4b38d700 1 -- 172.21.15.124:0/3535301948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d440b29f0 msgr2=0x7f2d440cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:00.168 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.162+0000 7f2d4b38d700 1 --2- 172.21.15.124:0/3535301948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d440b29f0 0x7f2d440cd1f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2d340098e0 tx=0x7f2d340091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.168 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.162+0000 7f2d4b38d700 1 -- 172.21.15.124:0/3535301948 shutdown_connections 2023-10-06T10:38:00.168 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.162+0000 7f2d4b38d700 1 --2- 172.21.15.124:0/3535301948 >> v2:172.21.15.92:3300/0 conn(0x7f2d440b2620 0x7f2d440cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.168 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.162+0000 7f2d4b38d700 1 --2- 172.21.15.124:0/3535301948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d440b29f0 0x7f2d440cd1f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.168 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.162+0000 7f2d4b38d700 1 -- 172.21.15.124:0/3535301948 >> 172.21.15.124:0/3535301948 conn(0x7f2d44099520 msgr2=0x7f2d4406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:00.168 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.162+0000 7f2d4b38d700 1 -- 172.21.15.124:0/3535301948 shutdown_connections 2023-10-06T10:38:00.168 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.162+0000 7f2d4b38d700 1 -- 172.21.15.124:0/3535301948 wait complete. 2023-10-06T10:38:00.168 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.163+0000 7f2d4b38d700 1 Processor -- start 2023-10-06T10:38:00.169 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.164+0000 7f2d4b38d700 1 -- start start 2023-10-06T10:38:00.171 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.164+0000 7f2d4b38d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d440b2620 0x7f2d4414dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:00.171 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.164+0000 7f2d4b38d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d440b29f0 0x7f2d440c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:00.171 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.164+0000 7f2d4b38d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d440ceaa0 0x7f2d440b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:00.171 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.164+0000 7f2d4b38d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d440c7b70 con 0x7f2d440ceaa0 2023-10-06T10:38:00.171 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.164+0000 7f2d4b38d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d44150950 con 0x7f2d440b29f0 2023-10-06T10:38:00.171 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.164+0000 7f2d4b38d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d44150c30 con 0x7f2d440b2620 2023-10-06T10:38:00.171 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.164+0000 7f2d4a38b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d440b2620 0x7f2d4414dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:00.171 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.164+0000 7f2d4a38b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d440b2620 0x7f2d4414dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.124:42050/0 (socket says 172.21.15.124:42050) 2023-10-06T10:38:00.171 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.164+0000 7f2d4a38b700 1 -- 172.21.15.124:0/2450684230 learned_addr learned my addr 172.21.15.124:0/2450684230 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-06T10:38:00.172 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.164+0000 7f2d49b8a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d440b29f0 0x7f2d440c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:00.172 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.164+0000 7f2d4ab8c700 1 --2- 172.21.15.124:0/2450684230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d440ceaa0 0x7f2d440b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:00.172 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.165+0000 7f2d4a38b700 1 -- 172.21.15.124:0/2450684230 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d440b29f0 msgr2=0x7f2d440c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:00.172 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.165+0000 7f2d4a38b700 1 --2- 172.21.15.124:0/2450684230 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d440b29f0 0x7f2d440c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.172 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.165+0000 7f2d4a38b700 1 -- 172.21.15.124:0/2450684230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d440ceaa0 msgr2=0x7f2d440b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:00.172 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.165+0000 7f2d4a38b700 1 --2- 172.21.15.124:0/2450684230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d440ceaa0 0x7f2d440b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.172 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.165+0000 7f2d4a38b700 1 -- 172.21.15.124:0/2450684230 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d34009420 con 0x7f2d440b2620 2023-10-06T10:38:00.172 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.165+0000 7f2d4a38b700 1 --2- 172.21.15.124:0/2450684230 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d440b2620 0x7f2d4414dcb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000ea00 tx=0x7f2d40009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:00.172 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.165+0000 7f2d3a7fc700 1 -- 172.21.15.124:0/2450684230 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4000c730 con 0x7f2d440b2620 2023-10-06T10:38:00.173 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.165+0000 7f2d3a7fc700 1 -- 172.21.15.124:0/2450684230 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4000c890 con 0x7f2d440b2620 2023-10-06T10:38:00.173 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.165+0000 7f2d4b38d700 1 -- 172.21.15.124:0/2450684230 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d44150f10 con 0x7f2d440b2620 2023-10-06T10:38:00.173 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.165+0000 7f2d3a7fc700 1 -- 172.21.15.124:0/2450684230 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4000cc10 con 0x7f2d440b2620 2023-10-06T10:38:00.173 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.165+0000 7f2d4b38d700 1 -- 172.21.15.124:0/2450684230 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d440c9e90 con 0x7f2d440b2620 2023-10-06T10:38:00.174 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.165+0000 7f2d3a7fc700 1 -- 172.21.15.124:0/2450684230 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f2d400136f0 con 0x7f2d440b2620 2023-10-06T10:38:00.174 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.165+0000 7f2d3a7fc700 1 -- 172.21.15.124:0/2450684230 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f2d4001ca50 con 0x7f2d440b2620 2023-10-06T10:38:00.174 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.166+0000 7f2d4b38d700 1 -- 172.21.15.124:0/2450684230 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d440c7e50 con 0x7f2d440b2620 2023-10-06T10:38:00.175 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.168+0000 7f2d3a7fc700 1 -- 172.21.15.124:0/2450684230 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f2d4003b6d0 con 0x7f2d440b2620 2023-10-06T10:38:00.252 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.250+0000 7f2d4b38d700 1 -- 172.21.15.124:0/2450684230 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5d42097a-3896-4e09-b7e3-5a6702775f4c", "id": 4} v 0) v1 -- 0x7f2d440ca080 con 0x7f2d440b2620 2023-10-06T10:38:00.255 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.254+0000 7f2d3a7fc700 1 -- 172.21.15.124:0/2450684230 <== mon.2 v2:172.21.15.124:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5d42097a-3896-4e09-b7e3-5a6702775f4c", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2d440ca080 con 0x7f2d440b2620 2023-10-06T10:38:00.256 INFO:teuthology.orchestra.run.smithi124.stdout:4 2023-10-06T10:38:00.257 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.255+0000 7f2d4b38d700 1 -- 172.21.15.124:0/2450684230 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d440b2620 msgr2=0x7f2d4414dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:00.258 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.255+0000 7f2d4b38d700 1 --2- 172.21.15.124:0/2450684230 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d440b2620 0x7f2d4414dcb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000ea00 tx=0x7f2d40009120 comp rx=0 tx=0).stop 2023-10-06T10:38:00.258 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.255+0000 7f2d4b38d700 1 -- 172.21.15.124:0/2450684230 shutdown_connections 2023-10-06T10:38:00.258 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.255+0000 7f2d4b38d700 1 --2- 172.21.15.124:0/2450684230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d440ceaa0 0x7f2d440b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.258 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.255+0000 7f2d4b38d700 1 --2- 172.21.15.124:0/2450684230 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d440b2620 0x7f2d4414dcb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.258 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.255+0000 7f2d4b38d700 1 --2- 172.21.15.124:0/2450684230 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d440b29f0 0x7f2d440c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.258 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.255+0000 7f2d4b38d700 1 -- 172.21.15.124:0/2450684230 >> 172.21.15.124:0/2450684230 conn(0x7f2d44099520 msgr2=0x7f2d44068bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:00.258 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.255+0000 7f2d4b38d700 1 -- 172.21.15.124:0/2450684230 shutdown_connections 2023-10-06T10:38:00.258 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.255+0000 7f2d4b38d700 1 -- 172.21.15.124:0/2450684230 wait complete. 2023-10-06T10:38:00.269 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new ac00cda9-44d6-432e-b24e-86fa42706227 5 2023-10-06T10:38:00.278 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:00.276+0000 7f5645afc180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T10:38:00.342 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:38:00.340+0000 7f2f0258d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T10:38:00.354 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:00.353+0000 7f5645afc180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T10:38:00.394 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.393+0000 7f42e1b53700 1 Processor -- start 2023-10-06T10:38:00.395 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.393+0000 7f42e1b53700 1 -- start start 2023-10-06T10:38:00.396 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.394+0000 7f42e1b53700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42dc0cd4d0 0x7f42dc0b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:00.396 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.394+0000 7f42e1b53700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42dc15ac30 con 0x7f42dc150940 2023-10-06T10:38:00.396 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.394+0000 7f42e1b53700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42dc0cb6a0 con 0x7f42dc0c5250 2023-10-06T10:38:00.396 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.394+0000 7f42e1b53700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc0cb090 con 0x7f42dc0cd4d0 2023-10-06T10:38:00.397 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.394+0000 7f42d3fff700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f42dc0c5250 0x7f42dc0ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.124:60766/0 (socket says 172.21.15.124:60766) 2023-10-06T10:38:00.397 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.394+0000 7f42d3fff700 1 -- 172.21.15.124:0/711638277 learned_addr learned my addr 172.21.15.124:0/711638277 (peer_addr_for_me v1:172.21.15.124:0/0) 2023-10-06T10:38:00.397 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.394+0000 7f42dbfff700 1 --2- 172.21.15.124:0/711638277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42dc0cd4d0 0x7f42dc0b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:00.397 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.394+0000 7f42daffd700 1 -- 172.21.15.124:0/711638277 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1954290513 0 0) 0x7f42dc0cb6a0 con 0x7f42dc0c5250 2023-10-06T10:38:00.397 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.394+0000 7f42daffd700 1 -- 172.21.15.124:0/711638277 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42c8003570 con 0x7f42dc0c5250 2023-10-06T10:38:00.397 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.394+0000 7f42dbfff700 1 -- 172.21.15.124:0/711638277 >> v1:172.21.15.124:6789/0 conn(0x7f42dc0c5250 legacy=0x7f42dc0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:00.397 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.394+0000 7f42dbfff700 1 -- 172.21.15.124:0/711638277 >> v1:172.21.15.92:6789/0 conn(0x7f42dc150940 legacy=0x7f42dc159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:00.397 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.394+0000 7f42dbfff700 1 -- 172.21.15.124:0/711638277 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42dc15a670 con 0x7f42dc0cd4d0 2023-10-06T10:38:00.397 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.394+0000 7f42dbfff700 1 --2- 172.21.15.124:0/711638277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42dc0cd4d0 0x7f42dc0b2050 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009e90 tx=0x7f42d4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91377a7cddb9bfa2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:00.397 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.394+0000 7f42daffd700 1 -- 172.21.15.124:0/711638277 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d400e070 con 0x7f42dc0cd4d0 2023-10-06T10:38:00.398 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.394+0000 7f42daffd700 1 -- 172.21.15.124:0/711638277 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d400bc30 con 0x7f42dc0cd4d0 2023-10-06T10:38:00.398 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.394+0000 7f42daffd700 1 -- 172.21.15.124:0/711638277 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4012430 con 0x7f42dc0cd4d0 2023-10-06T10:38:00.400 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.395+0000 7f42e1b53700 1 -- 172.21.15.124:0/711638277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42dc0cd4d0 msgr2=0x7f42dc0b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:00.400 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.395+0000 7f42e1b53700 1 --2- 172.21.15.124:0/711638277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42dc0cd4d0 0x7f42dc0b2050 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009e90 tx=0x7f42d4009780 comp rx=0 tx=0).stop 2023-10-06T10:38:00.400 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.395+0000 7f42e1b53700 1 -- 172.21.15.124:0/711638277 shutdown_connections 2023-10-06T10:38:00.401 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.395+0000 7f42e1b53700 1 --2- 172.21.15.124:0/711638277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42dc0cd4d0 0x7f42dc0b2050 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.401 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.395+0000 7f42e1b53700 1 -- 172.21.15.124:0/711638277 >> 172.21.15.124:0/711638277 conn(0x7f42dc099700 msgr2=0x7f42dc06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:00.401 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.395+0000 7f42e1b53700 1 -- 172.21.15.124:0/711638277 shutdown_connections 2023-10-06T10:38:00.401 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.395+0000 7f42e1b53700 1 -- 172.21.15.124:0/711638277 wait complete. 2023-10-06T10:38:00.401 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.396+0000 7f42e1b53700 1 Processor -- start 2023-10-06T10:38:00.401 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.396+0000 7f42e1b53700 1 -- start start 2023-10-06T10:38:00.401 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.397+0000 7f42e1b53700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42dc0c5250 0x7f42dc152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:00.401 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.397+0000 7f42e1b53700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42dc0cd4d0 0x7f42dc0ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:00.401 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.397+0000 7f42e1b53700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42dc150d20 0x7f42dc14fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:00.401 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.397+0000 7f42e1b53700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc151ed0 con 0x7f42dc0cd4d0 2023-10-06T10:38:00.401 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.397+0000 7f42e1b53700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc1512c0 con 0x7f42dc150d20 2023-10-06T10:38:00.401 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.397+0000 7f42e1b53700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc1515a0 con 0x7f42dc0c5250 2023-10-06T10:38:00.402 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.397+0000 7f42db7fe700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42dc0c5250 0x7f42dc152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:00.402 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.397+0000 7f42dbfff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42dc150d20 0x7f42dc14fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:00.402 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.397+0000 7f42db7fe700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42dc0c5250 0x7f42dc152360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.124:42060/0 (socket says 172.21.15.124:42060) 2023-10-06T10:38:00.402 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.397+0000 7f42db7fe700 1 -- 172.21.15.124:0/3915502606 learned_addr learned my addr 172.21.15.124:0/3915502606 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-06T10:38:00.402 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.397+0000 7f42d3fff700 1 --2- 172.21.15.124:0/3915502606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42dc0cd4d0 0x7f42dc0ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:00.402 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.397+0000 7f42db7fe700 1 -- 172.21.15.124:0/3915502606 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42dc150d20 msgr2=0x7f42dc14fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:00.402 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.397+0000 7f42db7fe700 1 --2- 172.21.15.124:0/3915502606 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42dc150d20 0x7f42dc14fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.402 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.397+0000 7f42db7fe700 1 -- 172.21.15.124:0/3915502606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42dc0cd4d0 msgr2=0x7f42dc0ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:00.402 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.397+0000 7f42db7fe700 1 --2- 172.21.15.124:0/3915502606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42dc0cd4d0 0x7f42dc0ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.402 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.397+0000 7f42db7fe700 1 -- 172.21.15.124:0/3915502606 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d4009a20 con 0x7f42dc0c5250 2023-10-06T10:38:00.402 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.397+0000 7f42db7fe700 1 --2- 172.21.15.124:0/3915502606 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42dc0c5250 0x7f42dc152360 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f42cc00c540 tx=0x7f42cc001070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:00.403 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.397+0000 7f42d37fe700 1 -- 172.21.15.124:0/3915502606 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc011590 con 0x7f42dc0c5250 2023-10-06T10:38:00.403 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.397+0000 7f42d37fe700 1 -- 172.21.15.124:0/3915502606 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42cc0116f0 con 0x7f42dc0c5250 2023-10-06T10:38:00.403 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.397+0000 7f42e1b53700 1 -- 172.21.15.124:0/3915502606 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42dc151880 con 0x7f42dc0c5250 2023-10-06T10:38:00.403 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.398+0000 7f42d37fe700 1 -- 172.21.15.124:0/3915502606 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc011ab0 con 0x7f42dc0c5250 2023-10-06T10:38:00.403 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.398+0000 7f42e1b53700 1 -- 172.21.15.124:0/3915502606 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42dc0caa50 con 0x7f42dc0c5250 2023-10-06T10:38:00.403 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.398+0000 7f42d37fe700 1 -- 172.21.15.124:0/3915502606 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f42cc01c6c0 con 0x7f42dc0c5250 2023-10-06T10:38:00.403 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.398+0000 7f42d37fe700 1 -- 172.21.15.124:0/3915502606 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f42cc0129a0 con 0x7f42dc0c5250 2023-10-06T10:38:00.403 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.398+0000 7f42e1b53700 1 -- 172.21.15.124:0/3915502606 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42dc064a90 con 0x7f42dc0c5250 2023-10-06T10:38:00.404 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.400+0000 7f42d37fe700 1 -- 172.21.15.124:0/3915502606 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f42cc017070 con 0x7f42dc0c5250 2023-10-06T10:38:00.432 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:38:00.430+0000 7f2f0258d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T10:38:00.484 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.483+0000 7f42e1b53700 1 -- 172.21.15.124:0/3915502606 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ac00cda9-44d6-432e-b24e-86fa42706227", "id": 5} v 0) v1 -- 0x7f42dc04bb40 con 0x7f42dc0c5250 2023-10-06T10:38:00.488 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.487+0000 7f42d37fe700 1 -- 172.21.15.124:0/3915502606 <== mon.2 v2:172.21.15.124:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ac00cda9-44d6-432e-b24e-86fa42706227", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f42dc15ac30 con 0x7f42dc0c5250 2023-10-06T10:38:00.488 INFO:teuthology.orchestra.run.smithi124.stdout:5 2023-10-06T10:38:00.489 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:00.487+0000 7f5645afc180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T10:38:00.489 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.488+0000 7f42e1b53700 1 -- 172.21.15.124:0/3915502606 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42dc0c5250 msgr2=0x7f42dc152360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:00.490 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.488+0000 7f42e1b53700 1 --2- 172.21.15.124:0/3915502606 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42dc0c5250 0x7f42dc152360 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f42cc00c540 tx=0x7f42cc001070 comp rx=0 tx=0).stop 2023-10-06T10:38:00.490 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.488+0000 7f42e1b53700 1 -- 172.21.15.124:0/3915502606 shutdown_connections 2023-10-06T10:38:00.490 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.488+0000 7f42e1b53700 1 --2- 172.21.15.124:0/3915502606 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42dc150d20 0x7f42dc14fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.490 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.488+0000 7f42e1b53700 1 --2- 172.21.15.124:0/3915502606 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42dc0c5250 0x7f42dc152360 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.490 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.488+0000 7f42e1b53700 1 --2- 172.21.15.124:0/3915502606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42dc0cd4d0 0x7f42dc0ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.490 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.488+0000 7f42e1b53700 1 -- 172.21.15.124:0/3915502606 >> 172.21.15.124:0/3915502606 conn(0x7f42dc099700 msgr2=0x7f42dc159da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:00.490 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.488+0000 7f42e1b53700 1 -- 172.21.15.124:0/3915502606 shutdown_connections 2023-10-06T10:38:00.490 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.488+0000 7f42e1b53700 1 -- 172.21.15.124:0/3915502606 wait complete. 2023-10-06T10:38:00.501 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01 6 2023-10-06T10:38:00.573 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:38:00.571+0000 7f2f0258d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T10:38:00.583 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:00.581+0000 7f5645afc180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T10:38:00.618 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.616+0000 7fce815ab700 1 Processor -- start 2023-10-06T10:38:00.618 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.616+0000 7fce815ab700 1 -- start start 2023-10-06T10:38:00.620 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.617+0000 7fce815ab700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c15da80 0x7fce7c0c8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:00.620 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.617+0000 7fce815ab700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce7c0685c0 0x7fce7c068a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:00.620 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.617+0000 7fce815ab700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7c0c8860 con 0x7fce7c1593b0 2023-10-06T10:38:00.621 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.617+0000 7fce815ab700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7c0c9050 con 0x7fce7c0685c0 2023-10-06T10:38:00.621 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.617+0000 7fce815ab700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c068f80 con 0x7fce7c15da80 2023-10-06T10:38:00.621 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.617+0000 7fce7a7fc700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c15da80 0x7fce7c0c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:00.621 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.617+0000 7fce7a7fc700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c15da80 0x7fce7c0c8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.124:42064/0 (socket says 172.21.15.124:42064) 2023-10-06T10:38:00.621 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.617+0000 7fce7a7fc700 1 -- 172.21.15.124:0/2463302634 learned_addr learned my addr 172.21.15.124:0/2463302634 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-06T10:38:00.621 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.617+0000 7fce7b7fe700 1 --2- 172.21.15.124:0/2463302634 >> v2:172.21.15.92:3300/0 conn(0x7fce7c0685c0 0x7fce7c068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:00.621 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.617+0000 7fce7a7fc700 1 -- 172.21.15.124:0/2463302634 >> v2:172.21.15.92:3300/0 conn(0x7fce7c0685c0 msgr2=0x7fce7c068a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:00.621 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.617+0000 7fce7a7fc700 1 --2- 172.21.15.124:0/2463302634 >> v2:172.21.15.92:3300/0 conn(0x7fce7c0685c0 0x7fce7c068a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.622 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.617+0000 7fce79ffb700 1 -- 172.21.15.124:0/2463302634 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875696060 0 0) 0x7fce7c0c8860 con 0x7fce7c1593b0 2023-10-06T10:38:00.622 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.617+0000 7fce7a7fc700 1 -- 172.21.15.124:0/2463302634 >> v1:172.21.15.92:6789/0 conn(0x7fce7c1593b0 legacy=0x7fce7c0c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:00.622 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.617+0000 7fce7a7fc700 1 -- 172.21.15.124:0/2463302634 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7c0c5f40 con 0x7fce7c15da80 2023-10-06T10:38:00.622 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.618+0000 7fce7a7fc700 1 --2- 172.21.15.124:0/2463302634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c15da80 0x7fce7c0c8320 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0099b0 tx=0x7fce6c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a2afffaf16dc8d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:00.622 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.618+0000 7fce79ffb700 1 -- 172.21.15.124:0/2463302634 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c009ea0 con 0x7fce7c15da80 2023-10-06T10:38:00.622 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.618+0000 7fce79ffb700 1 -- 172.21.15.124:0/2463302634 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c00b860 con 0x7fce7c15da80 2023-10-06T10:38:00.622 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.618+0000 7fce79ffb700 1 -- 172.21.15.124:0/2463302634 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c00bc00 con 0x7fce7c15da80 2023-10-06T10:38:00.622 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.618+0000 7fce815ab700 1 -- 172.21.15.124:0/2463302634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c15da80 msgr2=0x7fce7c0c8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:00.623 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.618+0000 7fce815ab700 1 --2- 172.21.15.124:0/2463302634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c15da80 0x7fce7c0c8320 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0099b0 tx=0x7fce6c009250 comp rx=0 tx=0).stop 2023-10-06T10:38:00.623 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.618+0000 7fce815ab700 1 -- 172.21.15.124:0/2463302634 shutdown_connections 2023-10-06T10:38:00.623 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.618+0000 7fce815ab700 1 --2- 172.21.15.124:0/2463302634 >> v2:172.21.15.92:3300/0 conn(0x7fce7c0685c0 0x7fce7c068a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.623 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.618+0000 7fce815ab700 1 --2- 172.21.15.124:0/2463302634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c15da80 0x7fce7c0c8320 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.623 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.618+0000 7fce815ab700 1 -- 172.21.15.124:0/2463302634 >> 172.21.15.124:0/2463302634 conn(0x7fce7c0c9c90 msgr2=0x7fce7c06da00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:00.623 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.618+0000 7fce815ab700 1 -- 172.21.15.124:0/2463302634 shutdown_connections 2023-10-06T10:38:00.623 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.618+0000 7fce815ab700 1 -- 172.21.15.124:0/2463302634 wait complete. 2023-10-06T10:38:00.626 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.620+0000 7fce815ab700 1 Processor -- start 2023-10-06T10:38:00.626 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.620+0000 7fce815ab700 1 -- start start 2023-10-06T10:38:00.626 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.620+0000 7fce815ab700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce7c0685c0 0x7fce7c169920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:00.627 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.620+0000 7fce815ab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c1593b0 0x7fce7c16c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:00.627 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.620+0000 7fce815ab700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c15da80 0x7fce7c15ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:00.627 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.620+0000 7fce815ab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c169e60 con 0x7fce7c1593b0 2023-10-06T10:38:00.627 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.620+0000 7fce815ab700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c15a150 con 0x7fce7c0685c0 2023-10-06T10:38:00.627 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.620+0000 7fce815ab700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c0adab0 con 0x7fce7c15da80 2023-10-06T10:38:00.627 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.621+0000 7fce7b7fe700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c15da80 0x7fce7c15ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:00.627 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.621+0000 7fce7affd700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce7c0685c0 0x7fce7c169920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:00.627 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.621+0000 7fce7b7fe700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c15da80 0x7fce7c15ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.124:42080/0 (socket says 172.21.15.124:42080) 2023-10-06T10:38:00.627 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.621+0000 7fce7b7fe700 1 -- 172.21.15.124:0/2396131058 learned_addr learned my addr 172.21.15.124:0/2396131058 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-06T10:38:00.627 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.621+0000 7fce7a7fc700 1 --2- 172.21.15.124:0/2396131058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c1593b0 0x7fce7c16c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:00.628 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.621+0000 7fce7b7fe700 1 -- 172.21.15.124:0/2396131058 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce7c0685c0 msgr2=0x7fce7c169920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:00.628 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.621+0000 7fce7b7fe700 1 --2- 172.21.15.124:0/2396131058 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce7c0685c0 0x7fce7c169920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.628 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.621+0000 7fce7b7fe700 1 -- 172.21.15.124:0/2396131058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c1593b0 msgr2=0x7fce7c16c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:00.628 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.621+0000 7fce7b7fe700 1 --2- 172.21.15.124:0/2396131058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c1593b0 0x7fce7c16c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.628 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.621+0000 7fce7b7fe700 1 -- 172.21.15.124:0/2396131058 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c0094f0 con 0x7fce7c15da80 2023-10-06T10:38:00.628 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.621+0000 7fce7affd700 1 --2- 172.21.15.124:0/2396131058 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce7c0685c0 0x7fce7c169920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:00.628 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.621+0000 7fce7b7fe700 1 --2- 172.21.15.124:0/2396131058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c15da80 0x7fce7c15ba30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fce74009e90 tx=0x7fce7400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:00.628 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.621+0000 7fce62ffd700 1 -- 172.21.15.124:0/2396131058 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7400cb70 con 0x7fce7c15da80 2023-10-06T10:38:00.628 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.621+0000 7fce62ffd700 1 -- 172.21.15.124:0/2396131058 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7400f040 con 0x7fce7c15da80 2023-10-06T10:38:00.628 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.621+0000 7fce815ab700 1 -- 172.21.15.124:0/2396131058 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7c159820 con 0x7fce7c15da80 2023-10-06T10:38:00.628 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.621+0000 7fce62ffd700 1 -- 172.21.15.124:0/2396131058 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce74014480 con 0x7fce7c15da80 2023-10-06T10:38:00.628 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.621+0000 7fce815ab700 1 -- 172.21.15.124:0/2396131058 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7c159de0 con 0x7fce7c15da80 2023-10-06T10:38:00.629 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.621+0000 7fce62ffd700 1 -- 172.21.15.124:0/2396131058 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fce74014830 con 0x7fce7c15da80 2023-10-06T10:38:00.629 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.621+0000 7fce62ffd700 1 -- 172.21.15.124:0/2396131058 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7fce740139e0 con 0x7fce7c15da80 2023-10-06T10:38:00.629 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.622+0000 7fce815ab700 1 -- 172.21.15.124:0/2396131058 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7c05f230 con 0x7fce7c15da80 2023-10-06T10:38:00.629 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.624+0000 7fce62ffd700 1 -- 172.21.15.124:0/2396131058 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fce74018020 con 0x7fce7c15da80 2023-10-06T10:38:00.673 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:38:00.671+0000 7f2f0258d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T10:38:00.708 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.707+0000 7fce815ab700 1 -- 172.21.15.124:0/2396131058 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01", "id": 6} v 0) v1 -- 0x7fce7c15a290 con 0x7fce7c15da80 2023-10-06T10:38:00.716 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.714+0000 7fce62ffd700 1 -- 172.21.15.124:0/2396131058 <== mon.2 v2:172.21.15.124:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fce74014d10 con 0x7fce7c15da80 2023-10-06T10:38:00.716 INFO:teuthology.orchestra.run.smithi124.stdout:6 2023-10-06T10:38:00.717 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.715+0000 7fce815ab700 1 -- 172.21.15.124:0/2396131058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c15da80 msgr2=0x7fce7c15ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:00.717 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.715+0000 7fce815ab700 1 --2- 172.21.15.124:0/2396131058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c15da80 0x7fce7c15ba30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fce74009e90 tx=0x7fce7400b040 comp rx=0 tx=0).stop 2023-10-06T10:38:00.718 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.715+0000 7fce815ab700 1 -- 172.21.15.124:0/2396131058 shutdown_connections 2023-10-06T10:38:00.718 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.715+0000 7fce815ab700 1 --2- 172.21.15.124:0/2396131058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c15da80 0x7fce7c15ba30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.718 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.716+0000 7fce815ab700 1 --2- 172.21.15.124:0/2396131058 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce7c0685c0 0x7fce7c169920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.718 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.716+0000 7fce815ab700 1 --2- 172.21.15.124:0/2396131058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c1593b0 0x7fce7c16c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.718 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.716+0000 7fce815ab700 1 -- 172.21.15.124:0/2396131058 >> 172.21.15.124:0/2396131058 conn(0x7fce7c0c9c90 msgr2=0x7fce7c1562c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:00.718 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.716+0000 7fce815ab700 1 -- 172.21.15.124:0/2396131058 shutdown_connections 2023-10-06T10:38:00.718 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.716+0000 7fce815ab700 1 -- 172.21.15.124:0/2396131058 wait complete. 2023-10-06T10:38:00.729 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new 3c94185e-03aa-4150-9860-9550f0651c4f 7 2023-10-06T10:38:00.844 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.842+0000 7fde92d79700 1 Processor -- start 2023-10-06T10:38:00.844 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.842+0000 7fde92d79700 1 -- start start 2023-10-06T10:38:00.847 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.843+0000 7fde92d79700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fde8c0cce20 0x7fde8c0c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:00.847 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.843+0000 7fde92d79700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fde8c0c93b0 0x7fde8c0c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:00.847 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.843+0000 7fde92d79700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde8c0cadd0 con 0x7fde8c0b05d0 2023-10-06T10:38:00.847 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.843+0000 7fde92d79700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde8c1577d0 con 0x7fde8c0cce20 2023-10-06T10:38:00.847 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.843+0000 7fde92d79700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde8c0c9ae0 con 0x7fde8c0c93b0 2023-10-06T10:38:00.847 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.843+0000 7fde92578700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fde8c0c93b0 0x7fde8c0c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:00.847 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.843+0000 7fde92578700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fde8c0c93b0 0x7fde8c0c2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:45854/0 (socket says 172.21.15.124:45854) 2023-10-06T10:38:00.848 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.843+0000 7fde92578700 1 -- 172.21.15.124:0/1261158051 learned_addr learned my addr 172.21.15.124:0/1261158051 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-06T10:38:00.848 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.843+0000 7fde91d77700 1 --1- 172.21.15.124:0/1261158051 >> v1:172.21.15.124:6789/0 conn(0x7fde8c0b05d0 0x7fde8c0cc710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.124:60776/0 (socket says 172.21.15.124:60776) 2023-10-06T10:38:00.848 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.843+0000 7fde91576700 1 --2- 172.21.15.124:0/1261158051 >> v2:172.21.15.92:3300/0 conn(0x7fde8c0cce20 0x7fde8c0c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:00.848 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.843+0000 7fde92578700 1 -- 172.21.15.124:0/1261158051 >> v2:172.21.15.92:3300/0 conn(0x7fde8c0cce20 msgr2=0x7fde8c0c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:00.848 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.843+0000 7fde92578700 1 --2- 172.21.15.124:0/1261158051 >> v2:172.21.15.92:3300/0 conn(0x7fde8c0cce20 0x7fde8c0c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.848 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.843+0000 7fde92578700 1 -- 172.21.15.124:0/1261158051 >> v1:172.21.15.124:6789/0 conn(0x7fde8c0b05d0 legacy=0x7fde8c0cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:00.848 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.843+0000 7fde92578700 1 -- 172.21.15.124:0/1261158051 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8c0c9dc0 con 0x7fde8c0c93b0 2023-10-06T10:38:00.848 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.844+0000 7fde92578700 1 --2- 172.21.15.124:0/1261158051 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde8c0c93b0 0x7fde8c0c2f40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fde8c06f250 tx=0x7fde84009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70061b5c01c5b7bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:00.848 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.844+0000 7fde90d75700 1 -- 172.21.15.124:0/1261158051 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8400e070 con 0x7fde8c0c93b0 2023-10-06T10:38:00.848 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.844+0000 7fde90d75700 1 -- 172.21.15.124:0/1261158051 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8400ba20 con 0x7fde8c0c93b0 2023-10-06T10:38:00.848 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.844+0000 7fde90d75700 1 -- 172.21.15.124:0/1261158051 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde840092e0 con 0x7fde8c0c93b0 2023-10-06T10:38:00.849 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.844+0000 7fde92d79700 1 -- 172.21.15.124:0/1261158051 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde8c0c93b0 msgr2=0x7fde8c0c2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:00.849 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.844+0000 7fde92d79700 1 --2- 172.21.15.124:0/1261158051 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde8c0c93b0 0x7fde8c0c2f40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fde8c06f250 tx=0x7fde84009840 comp rx=0 tx=0).stop 2023-10-06T10:38:00.849 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.844+0000 7fde92d79700 1 -- 172.21.15.124:0/1261158051 shutdown_connections 2023-10-06T10:38:00.849 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.844+0000 7fde92d79700 1 --2- 172.21.15.124:0/1261158051 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde8c0c93b0 0x7fde8c0c2f40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.849 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.844+0000 7fde92d79700 1 --2- 172.21.15.124:0/1261158051 >> v2:172.21.15.92:3300/0 conn(0x7fde8c0cce20 0x7fde8c0c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.849 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.844+0000 7fde92d79700 1 -- 172.21.15.124:0/1261158051 >> 172.21.15.124:0/1261158051 conn(0x7fde8c096a30 msgr2=0x7fde8c096e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:00.849 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.844+0000 7fde92d79700 1 -- 172.21.15.124:0/1261158051 shutdown_connections 2023-10-06T10:38:00.849 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.844+0000 7fde92d79700 1 -- 172.21.15.124:0/1261158051 wait complete. 2023-10-06T10:38:00.849 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.845+0000 7fde92d79700 1 Processor -- start 2023-10-06T10:38:00.849 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.845+0000 7fde92d79700 1 -- start start 2023-10-06T10:38:00.851 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.846+0000 7fde92d79700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde8c0b05d0 0x7fde8c151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:00.852 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.846+0000 7fde92d79700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde8c0c93b0 0x7fde8c154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:00.852 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.846+0000 7fde92d79700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde8c0cce20 0x7fde8c0996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:00.852 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.846+0000 7fde92d79700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c098bf0 con 0x7fde8c0b05d0 2023-10-06T10:38:00.852 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.846+0000 7fde92d79700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c152360 con 0x7fde8c0cce20 2023-10-06T10:38:00.852 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.846+0000 7fde92d79700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c154770 con 0x7fde8c0c93b0 2023-10-06T10:38:00.852 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.846+0000 7fde92578700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde8c0cce20 0x7fde8c0996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:00.852 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.846+0000 7fde91576700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde8c0c93b0 0x7fde8c154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:00.852 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.846+0000 7fde91576700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde8c0c93b0 0x7fde8c154230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.124:42092/0 (socket says 172.21.15.124:42092) 2023-10-06T10:38:00.852 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.846+0000 7fde92578700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde8c0cce20 0x7fde8c0996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:45864/0 (socket says 172.21.15.124:45864) 2023-10-06T10:38:00.852 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.846+0000 7fde91576700 1 -- 172.21.15.124:0/4195118989 learned_addr learned my addr 172.21.15.124:0/4195118989 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-06T10:38:00.852 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.846+0000 7fde91d77700 1 --2- 172.21.15.124:0/4195118989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde8c0b05d0 0x7fde8c151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:00.853 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.847+0000 7fde91576700 1 -- 172.21.15.124:0/4195118989 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde8c0cce20 msgr2=0x7fde8c0996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:00.853 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.847+0000 7fde91576700 1 --2- 172.21.15.124:0/4195118989 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde8c0cce20 0x7fde8c0996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.853 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.847+0000 7fde91576700 1 -- 172.21.15.124:0/4195118989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde8c0b05d0 msgr2=0x7fde8c151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:00.853 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.847+0000 7fde91576700 1 --2- 172.21.15.124:0/4195118989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde8c0b05d0 0x7fde8c151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.853 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.847+0000 7fde91576700 1 -- 172.21.15.124:0/4195118989 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde84009ae0 con 0x7fde8c0c93b0 2023-10-06T10:38:00.853 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.847+0000 7fde92578700 1 --2- 172.21.15.124:0/4195118989 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde8c0cce20 0x7fde8c0996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:00.853 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.847+0000 7fde91576700 1 --2- 172.21.15.124:0/4195118989 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde8c0c93b0 0x7fde8c154230 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00e9d0 tx=0x7fde7c009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:00.853 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.847+0000 7fde81ffb700 1 -- 172.21.15.124:0/4195118989 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c013680 con 0x7fde8c0c93b0 2023-10-06T10:38:00.853 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.847+0000 7fde81ffb700 1 -- 172.21.15.124:0/4195118989 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7c0137e0 con 0x7fde8c0c93b0 2023-10-06T10:38:00.853 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.847+0000 7fde92d79700 1 -- 172.21.15.124:0/4195118989 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde8c0c57a0 con 0x7fde8c0c93b0 2023-10-06T10:38:00.853 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.847+0000 7fde81ffb700 1 -- 172.21.15.124:0/4195118989 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c013b80 con 0x7fde8c0c93b0 2023-10-06T10:38:00.854 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.847+0000 7fde92d79700 1 -- 172.21.15.124:0/4195118989 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde8c0c5de0 con 0x7fde8c0c93b0 2023-10-06T10:38:00.854 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.847+0000 7fde81ffb700 1 -- 172.21.15.124:0/4195118989 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fde7c01c6d0 con 0x7fde8c0c93b0 2023-10-06T10:38:00.854 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.847+0000 7fde81ffb700 1 -- 172.21.15.124:0/4195118989 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7fde7c00cb00 con 0x7fde8c0c93b0 2023-10-06T10:38:00.854 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.848+0000 7fde92d79700 1 -- 172.21.15.124:0/4195118989 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde8c05f230 con 0x7fde8c0c93b0 2023-10-06T10:38:00.854 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.850+0000 7fde81ffb700 1 -- 172.21.15.124:0/4195118989 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fde7c021060 con 0x7fde8c0c93b0 2023-10-06T10:38:00.934 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.932+0000 7fde92d79700 1 -- 172.21.15.124:0/4195118989 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3c94185e-03aa-4150-9860-9550f0651c4f", "id": 7} v 0) v1 -- 0x7fde8c0ca0a0 con 0x7fde8c0c93b0 2023-10-06T10:38:00.942 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.940+0000 7fde81ffb700 1 -- 172.21.15.124:0/4195118989 <== mon.2 v2:172.21.15.124:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3c94185e-03aa-4150-9860-9550f0651c4f", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fde7c017070 con 0x7fde8c0c93b0 2023-10-06T10:38:00.942 INFO:teuthology.orchestra.run.smithi124.stdout:7 2023-10-06T10:38:00.943 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.941+0000 7fde92d79700 1 -- 172.21.15.124:0/4195118989 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde8c0c93b0 msgr2=0x7fde8c154230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:00.944 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.941+0000 7fde92d79700 1 --2- 172.21.15.124:0/4195118989 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde8c0c93b0 0x7fde8c154230 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00e9d0 tx=0x7fde7c009120 comp rx=0 tx=0).stop 2023-10-06T10:38:00.944 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.941+0000 7fde92d79700 1 -- 172.21.15.124:0/4195118989 shutdown_connections 2023-10-06T10:38:00.944 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.941+0000 7fde92d79700 1 --2- 172.21.15.124:0/4195118989 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde8c0cce20 0x7fde8c0996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.944 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.941+0000 7fde92d79700 1 --2- 172.21.15.124:0/4195118989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde8c0b05d0 0x7fde8c151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.944 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.941+0000 7fde92d79700 1 --2- 172.21.15.124:0/4195118989 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde8c0c93b0 0x7fde8c154230 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:00.944 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.941+0000 7fde92d79700 1 -- 172.21.15.124:0/4195118989 >> 172.21.15.124:0/4195118989 conn(0x7fde8c096a30 msgr2=0x7fde8c096e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:00.944 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.941+0000 7fde92d79700 1 -- 172.21.15.124:0/4195118989 shutdown_connections 2023-10-06T10:38:00.945 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-06T10:38:00.941+0000 7fde92d79700 1 -- 172.21.15.124:0/4195118989 wait complete. 2023-10-06T10:38:00.955 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-06T10:38:00.955 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-06T10:38:00.957 INFO:tasks.ceph.osd.0:Started 2023-10-06T10:38:00.957 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-06T10:38:00.957 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-06T10:38:00.959 INFO:tasks.ceph.osd.1:Started 2023-10-06T10:38:00.959 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-06T10:38:00.960 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-06T10:38:00.961 INFO:tasks.ceph.osd.2:Started 2023-10-06T10:38:00.962 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-06T10:38:00.962 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-06T10:38:00.964 INFO:tasks.ceph.osd.3:Started 2023-10-06T10:38:00.964 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-06T10:38:00.964 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-06T10:38:00.966 INFO:tasks.ceph.osd.4:Started 2023-10-06T10:38:00.966 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-06T10:38:00.966 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-06T10:38:00.968 INFO:tasks.ceph.osd.5:Started 2023-10-06T10:38:00.968 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-06T10:38:00.968 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-06T10:38:00.970 INFO:tasks.ceph.osd.6:Started 2023-10-06T10:38:00.970 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-06T10:38:00.971 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-06T10:38:00.972 INFO:tasks.ceph.osd.7:Started 2023-10-06T10:38:00.973 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:38:01.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.112+0000 7f1affec2700 1 Processor -- start 2023-10-06T10:38:01.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.113+0000 7f1affec2700 1 -- start start 2023-10-06T10:38:01.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.113+0000 7f1affec2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1af8159ad0 0x7f1af8159f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:01.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.113+0000 7f1affec2700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1af80ca670 0x7f1af80caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:01.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.113+0000 7f1affec2700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af80cafd0 con 0x7f1af80cb5f0 2023-10-06T10:38:01.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.113+0000 7f1affec2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af815a490 con 0x7f1af8159ad0 2023-10-06T10:38:01.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.113+0000 7f1affec2700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af8154020 con 0x7f1af80ca670 2023-10-06T10:38:01.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.114+0000 7f1afd45d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1af8159ad0 0x7f1af8159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:01.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.114+0000 7f1afd45d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1af8159ad0 0x7f1af8159f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44470/0 (socket says 172.21.15.92:44470) 2023-10-06T10:38:01.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.114+0000 7f1afd45d700 1 -- 172.21.15.92:0/76178722 learned_addr learned my addr 172.21.15.92:0/76178722 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:01.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.114+0000 7f1afe45f700 1 --2- 172.21.15.92:0/76178722 >> v2:172.21.15.124:3300/0 conn(0x7f1af80ca670 0x7f1af80caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:01.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.114+0000 7f1afd45d700 1 -- 172.21.15.92:0/76178722 >> v2:172.21.15.124:3300/0 conn(0x7f1af80ca670 msgr2=0x7f1af80caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:01.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.114+0000 7f1afd45d700 1 --2- 172.21.15.92:0/76178722 >> v2:172.21.15.124:3300/0 conn(0x7f1af80ca670 0x7f1af80caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:01.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.114+0000 7f1afd45d700 1 -- 172.21.15.92:0/76178722 >> v1:172.21.15.124:6789/0 conn(0x7f1af80cb5f0 legacy=0x7f1af80691a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:01.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.114+0000 7f1afd45d700 1 -- 172.21.15.92:0/76178722 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af8068d10 con 0x7f1af8159ad0 2023-10-06T10:38:01.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.114+0000 7f1afe45f700 1 --2- 172.21.15.92:0/76178722 >> v2:172.21.15.124:3300/0 conn(0x7f1af80ca670 0x7f1af80caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:01.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.114+0000 7f1afd45d700 1 --2- 172.21.15.92:0/76178722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1af8159ad0 0x7f1af8159f50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1af8068c80 tx=0x7f1aec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b673e7a60f490a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:01.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.114+0000 7f1afcc5c700 1 -- 172.21.15.92:0/76178722 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec00b820 con 0x7f1af8159ad0 2023-10-06T10:38:01.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.114+0000 7f1afcc5c700 1 -- 172.21.15.92:0/76178722 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aec00b980 con 0x7f1af8159ad0 2023-10-06T10:38:01.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.114+0000 7f1afcc5c700 1 -- 172.21.15.92:0/76178722 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec00bd20 con 0x7f1af8159ad0 2023-10-06T10:38:01.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.115+0000 7f1affec2700 1 -- 172.21.15.92:0/76178722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1af8159ad0 msgr2=0x7f1af8159f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:01.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.115+0000 7f1affec2700 1 --2- 172.21.15.92:0/76178722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1af8159ad0 0x7f1af8159f50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1af8068c80 tx=0x7f1aec009250 comp rx=0 tx=0).stop 2023-10-06T10:38:01.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.115+0000 7f1affec2700 1 -- 172.21.15.92:0/76178722 shutdown_connections 2023-10-06T10:38:01.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.115+0000 7f1affec2700 1 --2- 172.21.15.92:0/76178722 >> v2:172.21.15.124:3300/0 conn(0x7f1af80ca670 0x7f1af80caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:01.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.115+0000 7f1affec2700 1 --2- 172.21.15.92:0/76178722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1af8159ad0 0x7f1af8159f50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:01.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.115+0000 7f1affec2700 1 -- 172.21.15.92:0/76178722 >> 172.21.15.92:0/76178722 conn(0x7f1af80707d0 msgr2=0x7f1af8070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:01.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.115+0000 7f1affec2700 1 -- 172.21.15.92:0/76178722 shutdown_connections 2023-10-06T10:38:01.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.115+0000 7f1affec2700 1 -- 172.21.15.92:0/76178722 wait complete. 2023-10-06T10:38:01.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.116+0000 7f1affec2700 1 Processor -- start 2023-10-06T10:38:01.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.116+0000 7f1affec2700 1 -- start start 2023-10-06T10:38:01.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.117+0000 7f1affec2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1af80ca670 0x7f1af816ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:01.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.117+0000 7f1affec2700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1af80cb5f0 0x7f1af81586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:01.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.117+0000 7f1affec2700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1af8159ad0 0x7f1af8158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:01.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.117+0000 7f1affec2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af80b3ea0 con 0x7f1af80ca670 2023-10-06T10:38:01.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.117+0000 7f1affec2700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af80b3600 con 0x7f1af80cb5f0 2023-10-06T10:38:01.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.117+0000 7f1affec2700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af80b38e0 con 0x7f1af8159ad0 2023-10-06T10:38:01.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.117+0000 7f1afdc5e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1af80ca670 0x7f1af816ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:01.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.117+0000 7f1afdc5e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1af80ca670 0x7f1af816ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44476/0 (socket says 172.21.15.92:44476) 2023-10-06T10:38:01.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.117+0000 7f1afdc5e700 1 -- 172.21.15.92:0/2058033820 learned_addr learned my addr 172.21.15.92:0/2058033820 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:01.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.117+0000 7f1afd45d700 1 --2- 172.21.15.92:0/2058033820 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1af80cb5f0 0x7f1af81586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:01.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.117+0000 7f1afe45f700 1 --2- 172.21.15.92:0/2058033820 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1af8159ad0 0x7f1af8158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:01.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.117+0000 7f1afdc5e700 1 -- 172.21.15.92:0/2058033820 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1af8159ad0 msgr2=0x7f1af8158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:01.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.117+0000 7f1afdc5e700 1 --2- 172.21.15.92:0/2058033820 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1af8159ad0 0x7f1af8158c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:01.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.117+0000 7f1afdc5e700 1 -- 172.21.15.92:0/2058033820 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1af80cb5f0 msgr2=0x7f1af81586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:01.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.117+0000 7f1afdc5e700 1 --2- 172.21.15.92:0/2058033820 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1af80cb5f0 0x7f1af81586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:01.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.117+0000 7f1afdc5e700 1 -- 172.21.15.92:0/2058033820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aec0094f0 con 0x7f1af80ca670 2023-10-06T10:38:01.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.117+0000 7f1afdc5e700 1 --2- 172.21.15.92:0/2058033820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1af80ca670 0x7f1af816ab50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800cb10 tx=0x7f1ae80017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:01.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.117+0000 7f1af5ffb700 1 -- 172.21.15.92:0/2058033820 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae800ce20 con 0x7f1af80ca670 2023-10-06T10:38:01.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.117+0000 7f1af5ffb700 1 -- 172.21.15.92:0/2058033820 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae8006ba0 con 0x7f1af80ca670 2023-10-06T10:38:01.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.117+0000 7f1af5ffb700 1 -- 172.21.15.92:0/2058033820 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae800a7e0 con 0x7f1af80ca670 2023-10-06T10:38:01.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.117+0000 7f1affec2700 1 -- 172.21.15.92:0/2058033820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af80b3bc0 con 0x7f1af80ca670 2023-10-06T10:38:01.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.118+0000 7f1af5ffb700 1 -- 172.21.15.92:0/2058033820 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f1ae8007020 con 0x7f1af80ca670 2023-10-06T10:38:01.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.118+0000 7f1affec2700 1 -- 172.21.15.92:0/2058033820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af816b090 con 0x7f1af80ca670 2023-10-06T10:38:01.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.118+0000 7f1af5ffb700 1 -- 172.21.15.92:0/2058033820 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f1ae8007500 con 0x7f1af80ca670 2023-10-06T10:38:01.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.118+0000 7f1affec2700 1 -- 172.21.15.92:0/2058033820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af8064a90 con 0x7f1af80ca670 2023-10-06T10:38:01.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.120+0000 7f1af5ffb700 1 -- 172.21.15.92:0/2058033820 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f1ae8016030 con 0x7f1af80ca670 2023-10-06T10:38:01.154 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:01.153+0000 7f5645afc180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T10:38:01.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.188+0000 7f1affec2700 1 -- 172.21.15.92:0/2058033820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1af804bb40 con 0x7f1af80ca670 2023-10-06T10:38:01.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.189+0000 7f1af5ffb700 1 -- 172.21.15.92:0/2058033820 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f1af80cafd0 con 0x7f1af80ca670 2023-10-06T10:38:01.191 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:01.191 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":10,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:00.935065+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:38:01.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.190+0000 7f1affec2700 1 -- 172.21.15.92:0/2058033820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1af80ca670 msgr2=0x7f1af816ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:01.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.190+0000 7f1affec2700 1 --2- 172.21.15.92:0/2058033820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1af80ca670 0x7f1af816ab50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800cb10 tx=0x7f1ae80017b0 comp rx=0 tx=0).stop 2023-10-06T10:38:01.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.190+0000 7f1affec2700 1 -- 172.21.15.92:0/2058033820 shutdown_connections 2023-10-06T10:38:01.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.190+0000 7f1affec2700 1 --2- 172.21.15.92:0/2058033820 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1af8159ad0 0x7f1af8158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:01.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.190+0000 7f1affec2700 1 --2- 172.21.15.92:0/2058033820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1af80ca670 0x7f1af816ab50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:01.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.190+0000 7f1affec2700 1 --2- 172.21.15.92:0/2058033820 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1af80cb5f0 0x7f1af81586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:01.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.190+0000 7f1affec2700 1 -- 172.21.15.92:0/2058033820 >> 172.21.15.92:0/2058033820 conn(0x7f1af80707d0 msgr2=0x7f1af80ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:01.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.190+0000 7f1affec2700 1 -- 172.21.15.92:0/2058033820 shutdown_connections 2023-10-06T10:38:01.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:01.190+0000 7f1affec2700 1 -- 172.21.15.92:0/2058033820 wait complete. 2023-10-06T10:38:01.203 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-06T10:38:01.204 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-06T10:38:01.230 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:38:01.229+0000 7f2f0258d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T10:38:01.234 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:01.232+0000 7f5645afc180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T10:38:01.307 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:38:01.305+0000 7f2f0258d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T10:38:01.311 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:01.309+0000 7f5645afc180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T10:38:01.386 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:38:01.384+0000 7f2f0258d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T10:38:01.561 INFO:tasks.ceph.osd.4.smithi124.stderr:2023-10-06T10:38:01.559+0000 7f29a9fef540 -1 Falling back to public interface 2023-10-06T10:38:01.565 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-10-06T10:38:01.563+0000 7ffa9c669540 -1 Falling back to public interface 2023-10-06T10:38:01.566 INFO:tasks.ceph.osd.7.smithi124.stderr:2023-10-06T10:38:01.564+0000 7fd4d745a540 -1 Falling back to public interface 2023-10-06T10:38:01.568 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-10-06T10:38:01.566+0000 7fdcf408a540 -1 Falling back to public interface 2023-10-06T10:38:01.572 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-10-06T10:38:01.570+0000 7f6022822540 -1 Falling back to public interface 2023-10-06T10:38:01.578 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-10-06T10:38:01.576+0000 7fc4447c0540 -1 Falling back to public interface 2023-10-06T10:38:01.585 INFO:tasks.ceph.osd.5.smithi124.stderr:2023-10-06T10:38:01.583+0000 7fc8a9630540 -1 Falling back to public interface 2023-10-06T10:38:01.603 INFO:tasks.ceph.osd.6.smithi124.stderr:2023-10-06T10:38:01.601+0000 7f13cccbb540 -1 Falling back to public interface 2023-10-06T10:38:01.896 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:01.894+0000 7f5645afc180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T10:38:01.965 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:38:01.964+0000 7f2f0258d180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T10:38:02.009 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T10:38:02.034 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:02.033+0000 7f5645afc180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T10:38:02.102 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:38:02.100+0000 7f2f0258d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T10:38:02.117 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.115+0000 7fc9ebfff700 1 Processor -- start 2023-10-06T10:38:02.117 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.115+0000 7fc9ebfff700 1 -- start start 2023-10-06T10:38:02.119 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.116+0000 7fc9ebfff700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc9ec06c2b0 0x7fc9ec0c9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:02.119 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.116+0000 7fc9ebfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc9ec06c680 0x7fc9ec06cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:02.119 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.116+0000 7fc9ebfff700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ec06afe0 con 0x7fc9ec067920 2023-10-06T10:38:02.119 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.116+0000 7fc9ebfff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ec140ff0 con 0x7fc9ec06c680 2023-10-06T10:38:02.119 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.116+0000 7fc9ebfff700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ec0c7970 con 0x7fc9ec06c2b0 2023-10-06T10:38:02.120 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.116+0000 7fc9ea7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc9ec06c680 0x7fc9ec06cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:02.120 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.116+0000 7fc9ea7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc9ec06c680 0x7fc9ec06cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44484/0 (socket says 172.21.15.92:44484) 2023-10-06T10:38:02.120 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.116+0000 7fc9ea7fc700 1 -- 172.21.15.92:0/4119377747 learned_addr learned my addr 172.21.15.92:0/4119377747 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:02.120 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.116+0000 7fc9eaffd700 1 --2- 172.21.15.92:0/4119377747 >> v2:172.21.15.124:3300/0 conn(0x7fc9ec06c2b0 0x7fc9ec0c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:02.120 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.116+0000 7fc9ea7fc700 1 -- 172.21.15.92:0/4119377747 >> v2:172.21.15.124:3300/0 conn(0x7fc9ec06c2b0 msgr2=0x7fc9ec0c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:02.120 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.116+0000 7fc9ea7fc700 1 --2- 172.21.15.92:0/4119377747 >> v2:172.21.15.124:3300/0 conn(0x7fc9ec06c2b0 0x7fc9ec0c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:02.120 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.116+0000 7fc9ea7fc700 1 -- 172.21.15.92:0/4119377747 >> v1:172.21.15.124:6789/0 conn(0x7fc9ec067920 legacy=0x7fc9ec067d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:02.120 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.116+0000 7fc9ea7fc700 1 -- 172.21.15.92:0/4119377747 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ec06b280 con 0x7fc9ec06c680 2023-10-06T10:38:02.120 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.117+0000 7fc9ea7fc700 1 --2- 172.21.15.92:0/4119377747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9ec06c680 0x7fc9ec06cb20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec06e730 tx=0x7fc9e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c276c6befe14bcf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:02.120 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.117+0000 7fc9e9ffb700 1 -- 172.21.15.92:0/4119377747 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0009e70 con 0x7fc9ec06c680 2023-10-06T10:38:02.120 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.117+0000 7fc9e9ffb700 1 -- 172.21.15.92:0/4119377747 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e000b860 con 0x7fc9ec06c680 2023-10-06T10:38:02.121 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.117+0000 7fc9e9ffb700 1 -- 172.21.15.92:0/4119377747 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e000bc00 con 0x7fc9ec06c680 2023-10-06T10:38:02.121 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.117+0000 7fc9ebfff700 1 -- 172.21.15.92:0/4119377747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9ec06c680 msgr2=0x7fc9ec06cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:02.121 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.117+0000 7fc9ebfff700 1 --2- 172.21.15.92:0/4119377747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9ec06c680 0x7fc9ec06cb20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec06e730 tx=0x7fc9e0009250 comp rx=0 tx=0).stop 2023-10-06T10:38:02.121 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.117+0000 7fc9ebfff700 1 -- 172.21.15.92:0/4119377747 shutdown_connections 2023-10-06T10:38:02.121 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.117+0000 7fc9ebfff700 1 --2- 172.21.15.92:0/4119377747 >> v2:172.21.15.124:3300/0 conn(0x7fc9ec06c2b0 0x7fc9ec0c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:02.121 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.117+0000 7fc9ebfff700 1 --2- 172.21.15.92:0/4119377747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9ec06c680 0x7fc9ec06cb20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:02.121 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.117+0000 7fc9ebfff700 1 -- 172.21.15.92:0/4119377747 >> 172.21.15.92:0/4119377747 conn(0x7fc9ec06de90 msgr2=0x7fc9ec0ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:02.121 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.117+0000 7fc9ebfff700 1 -- 172.21.15.92:0/4119377747 shutdown_connections 2023-10-06T10:38:02.121 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.117+0000 7fc9ebfff700 1 -- 172.21.15.92:0/4119377747 wait complete. 2023-10-06T10:38:02.122 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.119+0000 7fc9ebfff700 1 Processor -- start 2023-10-06T10:38:02.122 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.119+0000 7fc9ebfff700 1 -- start start 2023-10-06T10:38:02.124 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9ebfff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9ec067920 0x7fc9ec15a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:02.124 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9ebfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9ec06c2b0 0x7fc9ec0c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:02.124 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9ebfff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9ec15a5e0 0x7fc9ec0b1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:02.124 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9ebfff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec0c5a70 con 0x7fc9ec06c2b0 2023-10-06T10:38:02.125 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9ebfff700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec0c4e60 con 0x7fc9ec067920 2023-10-06T10:38:02.125 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9ebfff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec0c5140 con 0x7fc9ec15a5e0 2023-10-06T10:38:02.125 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9ea7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9ec06c2b0 0x7fc9ec0c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:02.125 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9ea7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9ec06c2b0 0x7fc9ec0c6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44494/0 (socket says 172.21.15.92:44494) 2023-10-06T10:38:02.125 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9ea7fc700 1 -- 172.21.15.92:0/1292956862 learned_addr learned my addr 172.21.15.92:0/1292956862 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:02.125 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9eaffd700 1 --2- 172.21.15.92:0/1292956862 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9ec067920 0x7fc9ec15a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:02.125 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9eb7fe700 1 --2- 172.21.15.92:0/1292956862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9ec15a5e0 0x7fc9ec0b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:02.125 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9ea7fc700 1 -- 172.21.15.92:0/1292956862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9ec15a5e0 msgr2=0x7fc9ec0b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:02.125 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9ea7fc700 1 --2- 172.21.15.92:0/1292956862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9ec15a5e0 0x7fc9ec0b1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:02.125 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9ea7fc700 1 -- 172.21.15.92:0/1292956862 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9ec067920 msgr2=0x7fc9ec15a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:02.125 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9ea7fc700 1 --2- 172.21.15.92:0/1292956862 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9ec067920 0x7fc9ec15a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:02.125 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9ea7fc700 1 -- 172.21.15.92:0/1292956862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e00094f0 con 0x7fc9ec06c2b0 2023-10-06T10:38:02.126 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9eaffd700 1 --2- 172.21.15.92:0/1292956862 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9ec067920 0x7fc9ec15a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:02.126 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9ea7fc700 1 --2- 172.21.15.92:0/1292956862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9ec06c2b0 0x7fc9ec0c6710 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00094c0 tx=0x7fc9e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:02.126 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9d2ffd700 1 -- 172.21.15.92:0/1292956862 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0000bc0 con 0x7fc9ec06c2b0 2023-10-06T10:38:02.126 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9d2ffd700 1 -- 172.21.15.92:0/1292956862 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e000b9d0 con 0x7fc9ec06c2b0 2023-10-06T10:38:02.126 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9ebfff700 1 -- 172.21.15.92:0/1292956862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ec0c5420 con 0x7fc9ec06c2b0 2023-10-06T10:38:02.126 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9d2ffd700 1 -- 172.21.15.92:0/1292956862 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0025420 con 0x7fc9ec06c2b0 2023-10-06T10:38:02.126 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9ebfff700 1 -- 172.21.15.92:0/1292956862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ec0c4a20 con 0x7fc9ec06c2b0 2023-10-06T10:38:02.126 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.121+0000 7fc9d2ffd700 1 -- 172.21.15.92:0/1292956862 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fc9e0023070 con 0x7fc9ec06c2b0 2023-10-06T10:38:02.126 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.122+0000 7fc9d2ffd700 1 -- 172.21.15.92:0/1292956862 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fc9e0011b00 con 0x7fc9ec06c2b0 2023-10-06T10:38:02.126 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.122+0000 7fc9ebfff700 1 -- 172.21.15.92:0/1292956862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ec158250 con 0x7fc9ec06c2b0 2023-10-06T10:38:02.127 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.125+0000 7fc9d2ffd700 1 -- 172.21.15.92:0/1292956862 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fc9e002a070 con 0x7fc9ec06c2b0 2023-10-06T10:38:02.138 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:02.136+0000 7f5645afc180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T10:38:02.197 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:38:02.195+0000 7f2f0258d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T10:38:02.212 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.210+0000 7fc9ebfff700 1 -- 172.21.15.92:0/1292956862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9ec064a90 con 0x7fc9ec06c2b0 2023-10-06T10:38:02.213 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.211+0000 7fc9d2ffd700 1 -- 172.21.15.92:0/1292956862 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fc9e0015020 con 0x7fc9ec06c2b0 2023-10-06T10:38:02.213 INFO:teuthology.misc.health.smithi092.stdout: 2023-10-06T10:38:02.213 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":10,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:00.935065+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:38:02.214 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.212+0000 7fc9ebfff700 1 -- 172.21.15.92:0/1292956862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9ec06c2b0 msgr2=0x7fc9ec0c6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:02.214 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.212+0000 7fc9ebfff700 1 --2- 172.21.15.92:0/1292956862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9ec06c2b0 0x7fc9ec0c6710 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00094c0 tx=0x7fc9e0017000 comp rx=0 tx=0).stop 2023-10-06T10:38:02.214 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.212+0000 7fc9ebfff700 1 -- 172.21.15.92:0/1292956862 shutdown_connections 2023-10-06T10:38:02.214 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.212+0000 7fc9ebfff700 1 --2- 172.21.15.92:0/1292956862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9ec15a5e0 0x7fc9ec0b1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:02.214 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.212+0000 7fc9ebfff700 1 --2- 172.21.15.92:0/1292956862 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9ec067920 0x7fc9ec15a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:02.215 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.212+0000 7fc9ebfff700 1 --2- 172.21.15.92:0/1292956862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9ec06c2b0 0x7fc9ec0c6710 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:02.215 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.212+0000 7fc9ebfff700 1 -- 172.21.15.92:0/1292956862 >> 172.21.15.92:0/1292956862 conn(0x7fc9ec06de90 msgr2=0x7fc9ec13b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:02.215 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.212+0000 7fc9ebfff700 1 -- 172.21.15.92:0/1292956862 shutdown_connections 2023-10-06T10:38:02.215 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:02.212+0000 7fc9ebfff700 1 -- 172.21.15.92:0/1292956862 wait complete. 2023-10-06T10:38:02.225 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T10:38:02.400 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:02.398+0000 7f5645afc180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T10:38:02.442 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:38:02.440+0000 7f2f0258d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T10:38:02.478 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:02.477+0000 7f5645afc180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T10:38:02.518 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:38:02.516+0000 7f2f0258d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T10:38:03.310 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:03.308+0000 7f5645afc180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T10:38:03.344 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:38:03.343+0000 7f2f0258d180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T10:38:03.382 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:03.380+0000 7f5645afc180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T10:38:03.417 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:38:03.415+0000 7f2f0258d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T10:38:03.457 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:03.455+0000 7f5645afc180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T10:38:03.491 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:38:03.490+0000 7f2f0258d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T10:38:03.590 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:03.589+0000 7f5645afc180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T10:38:03.624 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:38:03.622+0000 7f2f0258d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T10:38:03.674 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:03.673+0000 7f5645afc180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T10:38:03.708 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:38:03.706+0000 7f2f0258d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T10:38:03.826 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:03.825+0000 7f5645afc180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T10:38:03.860 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:38:03.859+0000 7f2f0258d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T10:38:03.906 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:03.904+0000 7f5645afc180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T10:38:03.937 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:38:03.935+0000 7f2f0258d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T10:38:03.980 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:03.978+0000 7f5645afc180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T10:38:04.012 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:38:04.010+0000 7f2f0258d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T10:38:04.062 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:04.060+0000 7f5645afc180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T10:38:04.111 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-06T10:38:04.109+0000 7f2f0258d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T10:38:04.825 INFO:tasks.ceph.osd.4.smithi124.stderr:2023-10-06T10:38:04.824+0000 7f29a9fef540 -1 osd.4 0 log_to_monitors true 2023-10-06T10:38:05.063 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-10-06T10:38:05.063+0000 7ffa9c669540 -1 osd.0 0 log_to_monitors true 2023-10-06T10:38:05.086 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:05.085+0000 7f55feefc700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-06T10:38:05.086 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T10:38:05.085+0000 7f55feefc700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-06T10:38:05.087 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-10-06T10:38:05.086+0000 7fc4447c0540 -1 osd.2 0 log_to_monitors true 2023-10-06T10:38:05.100 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-10-06T10:38:05.100+0000 7fdcf408a540 -1 osd.3 0 log_to_monitors true 2023-10-06T10:38:05.110 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-10-06T10:38:05.110+0000 7f6022822540 -1 osd.1 0 log_to_monitors true 2023-10-06T10:38:05.115 INFO:tasks.ceph.osd.5.smithi124.stderr:2023-10-06T10:38:05.114+0000 7fc8a9630540 -1 osd.5 0 log_to_monitors true 2023-10-06T10:38:05.119 INFO:tasks.ceph.osd.7.smithi124.stderr:2023-10-06T10:38:05.117+0000 7fd4d745a540 -1 osd.7 0 log_to_monitors true 2023-10-06T10:38:05.123 INFO:tasks.ceph.osd.6.smithi124.stderr:2023-10-06T10:38:05.122+0000 7f13cccbb540 -1 osd.6 0 log_to_monitors true 2023-10-06T10:38:08.234 INFO:tasks.ceph.osd.6.smithi124.stderr:2023-10-06T10:38:08.233+0000 7f13c38dd700 -1 osd.6 0 waiting for initial osdmap 2023-10-06T10:38:08.238 INFO:tasks.ceph.osd.6.smithi124.stderr:2023-10-06T10:38:08.236+0000 7f13bc6d4700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T10:38:08.317 INFO:tasks.ceph.osd.5.smithi124.stderr:2023-10-06T10:38:08.316+0000 7fc8a1a55700 -1 osd.5 0 waiting for initial osdmap 2023-10-06T10:38:08.322 INFO:tasks.ceph.osd.5.smithi124.stderr:2023-10-06T10:38:08.320+0000 7fc899049700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T10:38:08.323 INFO:tasks.ceph.osd.4.smithi124.stderr:2023-10-06T10:38:08.320+0000 7f29a0c11700 -1 osd.4 0 waiting for initial osdmap 2023-10-06T10:38:08.326 INFO:tasks.ceph.osd.4.smithi124.stderr:2023-10-06T10:38:08.324+0000 7f2999a08700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T10:38:08.326 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-10-06T10:38:08.321+0000 7fc43b3e2700 -1 osd.2 0 waiting for initial osdmap 2023-10-06T10:38:08.329 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-10-06T10:38:08.328+0000 7f6019444700 -1 osd.1 0 waiting for initial osdmap 2023-10-06T10:38:08.333 INFO:tasks.ceph.osd.7.smithi124.stderr:2023-10-06T10:38:08.331+0000 7fd4ce07c700 -1 osd.7 0 waiting for initial osdmap 2023-10-06T10:38:08.334 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-10-06T10:38:08.332+0000 7f601223b700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T10:38:08.334 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-10-06T10:38:08.332+0000 7fc4341d9700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T10:38:08.335 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-10-06T10:38:08.330+0000 7fdceacac700 -1 osd.3 0 waiting for initial osdmap 2023-10-06T10:38:08.337 INFO:tasks.ceph.osd.7.smithi124.stderr:2023-10-06T10:38:08.335+0000 7fd4c6e73700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T10:38:08.337 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-10-06T10:38:08.337+0000 7fdce3aa3700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T10:38:08.367 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-10-06T10:38:08.366+0000 7ffa94a8e700 -1 osd.0 0 waiting for initial osdmap 2023-10-06T10:38:08.370 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-10-06T10:38:08.370+0000 7ffa8c082700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T10:38:09.031 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T10:38:09.137 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.136+0000 7f671c559700 1 Processor -- start 2023-10-06T10:38:09.137 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.136+0000 7f671c559700 1 -- start start 2023-10-06T10:38:09.140 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.137+0000 7f671c559700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6714142c80 0x7f671406a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:09.140 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.137+0000 7f671c559700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f671406ac70 0x7f67140cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:09.140 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.137+0000 7f671c559700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67140c77b0 con 0x7f67140cb280 2023-10-06T10:38:09.140 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.137+0000 7f671c559700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67140cb700 con 0x7f671406ac70 2023-10-06T10:38:09.140 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.137+0000 7f671c559700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671406b180 con 0x7f6714142c80 2023-10-06T10:38:09.140 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.138+0000 7f671a2f5700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6714142c80 0x7f671406a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:09.140 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.138+0000 7f6719af4700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f671406ac70 0x7f67140cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:09.140 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.138+0000 7f671aaf6700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f67140cb280 0x7f67140ccc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:48432/0 (socket says 172.21.15.92:48432) 2023-10-06T10:38:09.141 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.138+0000 7f671aaf6700 1 -- 172.21.15.92:0/3796127719 learned_addr learned my addr 172.21.15.92:0/3796127719 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:09.141 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.138+0000 7f6719af4700 1 --2- 172.21.15.92:0/3796127719 >> v2:172.21.15.124:3300/0 conn(0x7f671406ac70 0x7f67140cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:50176/0 (socket says 172.21.15.92:50176) 2023-10-06T10:38:09.141 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.138+0000 7f671a2f5700 1 -- 172.21.15.92:0/3796127719 >> v2:172.21.15.124:3300/0 conn(0x7f671406ac70 msgr2=0x7f67140cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:09.141 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.138+0000 7f671a2f5700 1 --2- 172.21.15.92:0/3796127719 >> v2:172.21.15.124:3300/0 conn(0x7f671406ac70 0x7f67140cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:09.141 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.138+0000 7f671a2f5700 1 -- 172.21.15.92:0/3796127719 >> v1:172.21.15.124:6789/0 conn(0x7f67140cb280 legacy=0x7f67140ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:09.141 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.138+0000 7f671a2f5700 1 -- 172.21.15.92:0/3796127719 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671406b460 con 0x7f6714142c80 2023-10-06T10:38:09.141 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.138+0000 7f6719af4700 1 --2- 172.21.15.92:0/3796127719 >> v2:172.21.15.124:3300/0 conn(0x7f671406ac70 0x7f67140cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:09.141 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.138+0000 7f671a2f5700 1 --2- 172.21.15.92:0/3796127719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6714142c80 0x7f671406a730 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f67100098e0 tx=0x7f67100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=344b40e09ff45b55 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:09.141 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.138+0000 7f67192f3700 1 -- 172.21.15.92:0/3796127719 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671000b640 con 0x7f6714142c80 2023-10-06T10:38:09.141 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.138+0000 7f67192f3700 1 -- 172.21.15.92:0/3796127719 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671000b7a0 con 0x7f6714142c80 2023-10-06T10:38:09.141 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.138+0000 7f67192f3700 1 -- 172.21.15.92:0/3796127719 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671000bb40 con 0x7f6714142c80 2023-10-06T10:38:09.142 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.139+0000 7f671c559700 1 -- 172.21.15.92:0/3796127719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6714142c80 msgr2=0x7f671406a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:09.142 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.139+0000 7f671c559700 1 --2- 172.21.15.92:0/3796127719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6714142c80 0x7f671406a730 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f67100098e0 tx=0x7f67100091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:09.142 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.139+0000 7f671c559700 1 -- 172.21.15.92:0/3796127719 shutdown_connections 2023-10-06T10:38:09.142 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.139+0000 7f671c559700 1 --2- 172.21.15.92:0/3796127719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6714142c80 0x7f671406a730 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:09.142 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.139+0000 7f671c559700 1 --2- 172.21.15.92:0/3796127719 >> v2:172.21.15.124:3300/0 conn(0x7f671406ac70 0x7f67140cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:09.142 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.139+0000 7f671c559700 1 -- 172.21.15.92:0/3796127719 >> 172.21.15.92:0/3796127719 conn(0x7f67140675b0 msgr2=0x7f67140679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:09.142 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.139+0000 7f671c559700 1 -- 172.21.15.92:0/3796127719 shutdown_connections 2023-10-06T10:38:09.142 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.139+0000 7f671c559700 1 -- 172.21.15.92:0/3796127719 wait complete. 2023-10-06T10:38:09.145 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.140+0000 7f671c559700 1 Processor -- start 2023-10-06T10:38:09.145 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.140+0000 7f671c559700 1 -- start start 2023-10-06T10:38:09.145 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.141+0000 7f671c559700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f671406ac70 0x7f67140c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:09.145 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.141+0000 7f671c559700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67140cb280 0x7f671415dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:09.145 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.141+0000 7f671c559700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6714142c80 0x7f67140c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:09.145 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.141+0000 7f671c559700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6714159c00 con 0x7f671406ac70 2023-10-06T10:38:09.145 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.141+0000 7f671c559700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6714159ee0 con 0x7f67140cb280 2023-10-06T10:38:09.146 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.141+0000 7f671c559700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67140c5a40 con 0x7f6714142c80 2023-10-06T10:38:09.146 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.141+0000 7f671a2f5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f671406ac70 0x7f67140c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:09.146 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.141+0000 7f671a2f5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f671406ac70 0x7f67140c51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51304/0 (socket says 172.21.15.92:51304) 2023-10-06T10:38:09.146 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.141+0000 7f671a2f5700 1 -- 172.21.15.92:0/399582869 learned_addr learned my addr 172.21.15.92:0/399582869 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:09.146 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.141+0000 7f671aaf6700 1 --2- 172.21.15.92:0/399582869 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6714142c80 0x7f67140c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:09.146 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.141+0000 7f6719af4700 1 --2- 172.21.15.92:0/399582869 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67140cb280 0x7f671415dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:09.146 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.141+0000 7f671a2f5700 1 -- 172.21.15.92:0/399582869 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6714142c80 msgr2=0x7f67140c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:09.146 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.141+0000 7f671a2f5700 1 --2- 172.21.15.92:0/399582869 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6714142c80 0x7f67140c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:09.146 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.141+0000 7f671a2f5700 1 -- 172.21.15.92:0/399582869 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67140cb280 msgr2=0x7f671415dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:09.146 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.141+0000 7f671a2f5700 1 --2- 172.21.15.92:0/399582869 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67140cb280 0x7f671415dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:09.146 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.141+0000 7f671a2f5700 1 -- 172.21.15.92:0/399582869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6704009420 con 0x7f671406ac70 2023-10-06T10:38:09.147 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.141+0000 7f671a2f5700 1 --2- 172.21.15.92:0/399582869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f671406ac70 0x7f67140c51f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f671406b2c0 tx=0x7f6710017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:09.147 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.141+0000 7f670a7fc700 1 -- 172.21.15.92:0/399582869 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671001bc40 con 0x7f671406ac70 2023-10-06T10:38:09.147 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.141+0000 7f670a7fc700 1 -- 172.21.15.92:0/399582869 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671001bdc0 con 0x7f671406ac70 2023-10-06T10:38:09.147 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.141+0000 7f671c559700 1 -- 172.21.15.92:0/399582869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6710009420 con 0x7f671406ac70 2023-10-06T10:38:09.147 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.141+0000 7f670a7fc700 1 -- 172.21.15.92:0/399582869 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6710011670 con 0x7f671406ac70 2023-10-06T10:38:09.147 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.141+0000 7f671c559700 1 -- 172.21.15.92:0/399582869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67140c6000 con 0x7f671406ac70 2023-10-06T10:38:09.147 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.142+0000 7f671c559700 1 -- 172.21.15.92:0/399582869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6714064a90 con 0x7f671406ac70 2023-10-06T10:38:09.147 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.143+0000 7f670a7fc700 1 -- 172.21.15.92:0/399582869 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f6710011a20 con 0x7f671406ac70 2023-10-06T10:38:09.147 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.143+0000 7f670a7fc700 1 --2- 172.21.15.92:0/399582869 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f66f8066ea0 0x7f66f8069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:09.147 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.143+0000 7f670a7fc700 1 -- 172.21.15.92:0/399582869 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2227+0+0 (secure 0 0 0) 0x7f6710084b70 con 0x7f671406ac70 2023-10-06T10:38:09.147 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.143+0000 7f6719af4700 1 --2- 172.21.15.92:0/399582869 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f66f8066ea0 0x7f66f8069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:09.147 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.143+0000 7f6719af4700 1 --2- 172.21.15.92:0/399582869 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f66f8066ea0 0x7f66f8069350 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f67140c6520 tx=0x7f67040091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:09.148 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.147+0000 7f670a7fc700 1 -- 172.21.15.92:0/399582869 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6710052c50 con 0x7f671406ac70 2023-10-06T10:38:09.342 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.341+0000 7f671c559700 1 -- 172.21.15.92:0/399582869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f671404bb40 con 0x7f671406ac70 2023-10-06T10:38:09.343 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.342+0000 7f670a7fc700 1 -- 172.21.15.92:0/399582869 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10396 (secure 0 0 0) 0x7f6710015090 con 0x7f671406ac70 2023-10-06T10:38:09.343 INFO:teuthology.misc.health.smithi092.stdout: 2023-10-06T10:38:09.343 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":13,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:09.254652+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:38:09.345 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.344+0000 7f671c559700 1 -- 172.21.15.92:0/399582869 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f66f8066ea0 msgr2=0x7f66f8069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:09.346 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.344+0000 7f671c559700 1 --2- 172.21.15.92:0/399582869 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f66f8066ea0 0x7f66f8069350 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f67140c6520 tx=0x7f67040091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:09.346 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.344+0000 7f671c559700 1 -- 172.21.15.92:0/399582869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f671406ac70 msgr2=0x7f67140c51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:09.346 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.344+0000 7f671c559700 1 --2- 172.21.15.92:0/399582869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f671406ac70 0x7f67140c51f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f671406b2c0 tx=0x7f6710017000 comp rx=0 tx=0).stop 2023-10-06T10:38:09.346 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.344+0000 7f671c559700 1 -- 172.21.15.92:0/399582869 shutdown_connections 2023-10-06T10:38:09.346 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.344+0000 7f671c559700 1 --2- 172.21.15.92:0/399582869 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f66f8066ea0 0x7f66f8069350 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:09.346 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.344+0000 7f671c559700 1 --2- 172.21.15.92:0/399582869 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6714142c80 0x7f67140c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:09.346 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.344+0000 7f671c559700 1 --2- 172.21.15.92:0/399582869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f671406ac70 0x7f67140c51f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:09.346 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.344+0000 7f671c559700 1 --2- 172.21.15.92:0/399582869 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67140cb280 0x7f671415dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:09.346 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.344+0000 7f671c559700 1 -- 172.21.15.92:0/399582869 >> 172.21.15.92:0/399582869 conn(0x7f67140675b0 msgr2=0x7f67140c2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:09.346 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.344+0000 7f671c559700 1 -- 172.21.15.92:0/399582869 shutdown_connections 2023-10-06T10:38:09.346 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T10:38:09.344+0000 7f671c559700 1 -- 172.21.15.92:0/399582869 wait complete. 2023-10-06T10:38:09.356 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-06T10:38:09.356 INFO:tasks.ceph:Creating RBD pool 2023-10-06T10:38:09.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-06T10:38:09.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.469+0000 7f616ffff700 1 Processor -- start 2023-10-06T10:38:09.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.469+0000 7f616ffff700 1 -- start start 2023-10-06T10:38:09.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.470+0000 7f616ffff700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6170159910 0x7f61700ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:09.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.470+0000 7f616ffff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61700ae210 con 0x7f61700c6020 2023-10-06T10:38:09.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.470+0000 7f616ffff700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61700ad600 con 0x7f6170159540 2023-10-06T10:38:09.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.470+0000 7f616ffff700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61700ad8e0 con 0x7f6170159910 2023-10-06T10:38:09.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.470+0000 7f616f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f61700c6020 0x7f61700ce850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37078/0 (socket says 172.21.15.92:37078) 2023-10-06T10:38:09.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.470+0000 7f616f7fe700 1 -- 172.21.15.92:0/1647420568 learned_addr learned my addr 172.21.15.92:0/1647420568 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:09.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.470+0000 7f61667fc700 1 --2- 172.21.15.92:0/1647420568 >> v2:172.21.15.124:3300/0 conn(0x7f6170159910 0x7f61700ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:09.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.470+0000 7f616e7fc700 1 -- 172.21.15.92:0/1647420568 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 366587300 0 0) 0x7f61700ae210 con 0x7f61700c6020 2023-10-06T10:38:09.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.470+0000 7f616e7fc700 1 -- 172.21.15.92:0/1647420568 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f615c003570 con 0x7f61700c6020 2023-10-06T10:38:09.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.470+0000 7f616e7fc700 1 -- 172.21.15.92:0/1647420568 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2119174723 0 0) 0x7f61700ad600 con 0x7f6170159540 2023-10-06T10:38:09.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.470+0000 7f616e7fc700 1 -- 172.21.15.92:0/1647420568 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61700ae210 con 0x7f6170159540 2023-10-06T10:38:09.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.470+0000 7f61667fc700 1 -- 172.21.15.92:0/1647420568 >> v1:172.21.15.124:6789/0 conn(0x7f6170159540 legacy=0x7f61700ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:09.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.470+0000 7f61667fc700 1 -- 172.21.15.92:0/1647420568 >> v1:172.21.15.92:6789/0 conn(0x7f61700c6020 legacy=0x7f61700ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:09.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.470+0000 7f61667fc700 1 -- 172.21.15.92:0/1647420568 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61700adbc0 con 0x7f6170159910 2023-10-06T10:38:09.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.471+0000 7f61667fc700 1 --2- 172.21.15.92:0/1647420568 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6170159910 0x7f61700ae6a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f61580098e0 tx=0x7f61580091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1d6791227f5f8cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:09.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.471+0000 7f616e7fc700 1 -- 172.21.15.92:0/1647420568 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6158011630 con 0x7f6170159910 2023-10-06T10:38:09.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.471+0000 7f616e7fc700 1 -- 172.21.15.92:0/1647420568 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6158011790 con 0x7f6170159910 2023-10-06T10:38:09.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.471+0000 7f616e7fc700 1 -- 172.21.15.92:0/1647420568 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6158011b30 con 0x7f6170159910 2023-10-06T10:38:09.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.471+0000 7f616ffff700 1 -- 172.21.15.92:0/1647420568 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6170159910 msgr2=0x7f61700ae6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:09.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.471+0000 7f616ffff700 1 --2- 172.21.15.92:0/1647420568 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6170159910 0x7f61700ae6a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f61580098e0 tx=0x7f61580091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:09.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.471+0000 7f616ffff700 1 -- 172.21.15.92:0/1647420568 shutdown_connections 2023-10-06T10:38:09.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.471+0000 7f616ffff700 1 --2- 172.21.15.92:0/1647420568 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6170159910 0x7f61700ae6a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:09.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.471+0000 7f616ffff700 1 -- 172.21.15.92:0/1647420568 >> 172.21.15.92:0/1647420568 conn(0x7f6170099520 msgr2=0x7f617006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:09.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.471+0000 7f616ffff700 1 -- 172.21.15.92:0/1647420568 shutdown_connections 2023-10-06T10:38:09.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.471+0000 7f616ffff700 1 -- 172.21.15.92:0/1647420568 wait complete. 2023-10-06T10:38:09.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.473+0000 7f616ffff700 1 Processor -- start 2023-10-06T10:38:09.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.473+0000 7f616ffff700 1 -- start start 2023-10-06T10:38:09.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.473+0000 7f616ffff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61700c6020 0x7f6170157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:09.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.473+0000 7f616ffff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6170159540 0x7f61700b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:09.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.473+0000 7f616ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6170159910 0x7f6170158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:09.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.473+0000 7f616ffff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61700c8740 con 0x7f6170159910 2023-10-06T10:38:09.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.473+0000 7f616ffff700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61700b39f0 con 0x7f6170159540 2023-10-06T10:38:09.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.473+0000 7f616ffff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61700b3cd0 con 0x7f61700c6020 2023-10-06T10:38:09.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.473+0000 7f616f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6170159910 0x7f6170158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:09.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.473+0000 7f616f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6170159910 0x7f6170158990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51316/0 (socket says 172.21.15.92:51316) 2023-10-06T10:38:09.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.473+0000 7f616f7fe700 1 -- 172.21.15.92:0/3308698315 learned_addr learned my addr 172.21.15.92:0/3308698315 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:09.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.474+0000 7f61667fc700 1 --2- 172.21.15.92:0/3308698315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6170159540 0x7f61700b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:09.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.474+0000 7f616effd700 1 --2- 172.21.15.92:0/3308698315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61700c6020 0x7f6170157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:09.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.474+0000 7f616f7fe700 1 -- 172.21.15.92:0/3308698315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61700c6020 msgr2=0x7f6170157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:09.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.474+0000 7f616f7fe700 1 --2- 172.21.15.92:0/3308698315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61700c6020 0x7f6170157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:09.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.474+0000 7f616f7fe700 1 -- 172.21.15.92:0/3308698315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6170159540 msgr2=0x7f61700b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:09.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.474+0000 7f616f7fe700 1 --2- 172.21.15.92:0/3308698315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6170159540 0x7f61700b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:09.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.474+0000 7f616f7fe700 1 -- 172.21.15.92:0/3308698315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6158009420 con 0x7f6170159910 2023-10-06T10:38:09.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.474+0000 7f616effd700 1 --2- 172.21.15.92:0/3308698315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61700c6020 0x7f6170157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:09.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.474+0000 7f61667fc700 1 --2- 172.21.15.92:0/3308698315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6170159540 0x7f61700b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:09.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.474+0000 7f616f7fe700 1 --2- 172.21.15.92:0/3308698315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6170159910 0x7f6170158990 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f616800cc20 tx=0x7f6168001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:09.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.474+0000 7f61677fe700 1 -- 172.21.15.92:0/3308698315 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616800a960 con 0x7f6170159910 2023-10-06T10:38:09.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.474+0000 7f61677fe700 1 -- 172.21.15.92:0/3308698315 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616800aac0 con 0x7f6170159910 2023-10-06T10:38:09.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.474+0000 7f61677fe700 1 -- 172.21.15.92:0/3308698315 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616800ae80 con 0x7f6170159910 2023-10-06T10:38:09.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.474+0000 7f616ffff700 1 -- 172.21.15.92:0/3308698315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61700b3fb0 con 0x7f6170159910 2023-10-06T10:38:09.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.474+0000 7f616ffff700 1 -- 172.21.15.92:0/3308698315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6170158450 con 0x7f6170159910 2023-10-06T10:38:09.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.475+0000 7f616ffff700 1 -- 172.21.15.92:0/3308698315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61700c8510 con 0x7f6170159910 2023-10-06T10:38:09.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.475+0000 7f61677fe700 1 -- 172.21.15.92:0/3308698315 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f6168007300 con 0x7f6170159910 2023-10-06T10:38:09.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.476+0000 7f61677fe700 1 --2- 172.21.15.92:0/3308698315 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6150066ea0 0x7f6150069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:09.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.476+0000 7f61677fe700 1 -- 172.21.15.92:0/3308698315 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f6168084310 con 0x7f6170159910 2023-10-06T10:38:09.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.476+0000 7f616effd700 1 --2- 172.21.15.92:0/3308698315 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6150066ea0 0x7f6150069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:09.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.476+0000 7f616effd700 1 --2- 172.21.15.92:0/3308698315 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6150066ea0 0x7f6150069350 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6160003940 tx=0x7f6160009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:09.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.480+0000 7f61677fe700 1 -- 172.21.15.92:0/3308698315 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61700c8510 con 0x7f6170159910 2023-10-06T10:38:09.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:09.663+0000 7f616ffff700 1 -- 172.21.15.92:0/3308698315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f61700c8510 con 0x7f6170159910 2023-10-06T10:38:10.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.313+0000 7f61677fe700 1 -- 172.21.15.92:0/3308698315 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7f61700c8510 con 0x7f6170159910 2023-10-06T10:38:10.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.315+0000 7f616ffff700 1 -- 172.21.15.92:0/3308698315 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6150066ea0 msgr2=0x7f6150069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:10.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.315+0000 7f616ffff700 1 --2- 172.21.15.92:0/3308698315 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6150066ea0 0x7f6150069350 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6160003940 tx=0x7f6160009040 comp rx=0 tx=0).stop 2023-10-06T10:38:10.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.315+0000 7f616ffff700 1 -- 172.21.15.92:0/3308698315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6170159910 msgr2=0x7f6170158990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:10.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.315+0000 7f616ffff700 1 --2- 172.21.15.92:0/3308698315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6170159910 0x7f6170158990 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f616800cc20 tx=0x7f6168001a40 comp rx=0 tx=0).stop 2023-10-06T10:38:10.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.315+0000 7f616ffff700 1 -- 172.21.15.92:0/3308698315 shutdown_connections 2023-10-06T10:38:10.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.315+0000 7f616ffff700 1 --2- 172.21.15.92:0/3308698315 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6150066ea0 0x7f6150069350 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:10.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.315+0000 7f616ffff700 1 --2- 172.21.15.92:0/3308698315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6170159910 0x7f6170158990 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:10.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.315+0000 7f616ffff700 1 --2- 172.21.15.92:0/3308698315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61700c6020 0x7f6170157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:10.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.315+0000 7f616ffff700 1 --2- 172.21.15.92:0/3308698315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6170159540 0x7f61700b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:10.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.315+0000 7f616ffff700 1 -- 172.21.15.92:0/3308698315 >> 172.21.15.92:0/3308698315 conn(0x7f6170099520 msgr2=0x7f61700cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:10.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.315+0000 7f616ffff700 1 -- 172.21.15.92:0/3308698315 shutdown_connections 2023-10-06T10:38:10.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.315+0000 7f616ffff700 1 -- 172.21.15.92:0/3308698315 wait complete. 2023-10-06T10:38:10.318 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'rbd' created 2023-10-06T10:38:10.328 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-06T10:38:10.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.447+0000 7fd91f34c700 1 Processor -- start 2023-10-06T10:38:10.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.448+0000 7fd91f34c700 1 -- start start 2023-10-06T10:38:10.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.448+0000 7fd91f34c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd918068cd0 0x7fd918069d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:10.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.448+0000 7fd91f34c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd91806b990 0x7fd91806a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:10.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.448+0000 7fd91f34c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9180c89e0 con 0x7fd91806b5c0 2023-10-06T10:38:10.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.448+0000 7fd91f34c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9180c7dd0 con 0x7fd91806b990 2023-10-06T10:38:10.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.449+0000 7fd91f34c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180c80b0 con 0x7fd918068cd0 2023-10-06T10:38:10.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.449+0000 7fd91d8e9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd91806b990 0x7fd91806a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:10.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.449+0000 7fd91d8e9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd91806b990 0x7fd91806a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51322/0 (socket says 172.21.15.92:51322) 2023-10-06T10:38:10.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.449+0000 7fd91d0e8700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd91806b5c0 0x7fd9180685c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37082/0 (socket says 172.21.15.92:37082) 2023-10-06T10:38:10.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.449+0000 7fd91d8e9700 1 -- 172.21.15.92:0/3672103123 learned_addr learned my addr 172.21.15.92:0/3672103123 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:10.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.449+0000 7fd91c8e7700 1 --2- 172.21.15.92:0/3672103123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd918068cd0 0x7fd918069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:10.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.449+0000 7fd91d8e9700 1 -- 172.21.15.92:0/3672103123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd918068cd0 msgr2=0x7fd918069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:10.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.449+0000 7fd91d8e9700 1 --2- 172.21.15.92:0/3672103123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd918068cd0 0x7fd918069d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:10.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.449+0000 7fd91d8e9700 1 -- 172.21.15.92:0/3672103123 >> v1:172.21.15.92:6789/0 conn(0x7fd91806b5c0 legacy=0x7fd9180685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:10.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.449+0000 7fd91d8e9700 1 -- 172.21.15.92:0/3672103123 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9180c8390 con 0x7fd91806b990 2023-10-06T10:38:10.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.449+0000 7fd90ffff700 1 -- 172.21.15.92:0/3672103123 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1169416352 0 0) 0x7fd9180c89e0 con 0x7fd91806b5c0 2023-10-06T10:38:10.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.449+0000 7fd91d8e9700 1 --2- 172.21.15.92:0/3672103123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd91806b990 0x7fd91806a260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd910009f20 tx=0x7fd910009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7ca4f8d874e4f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:10.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.449+0000 7fd90ffff700 1 -- 172.21.15.92:0/3672103123 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91000e070 con 0x7fd91806b990 2023-10-06T10:38:10.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.449+0000 7fd90ffff700 1 -- 172.21.15.92:0/3672103123 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91000ba80 con 0x7fd91806b990 2023-10-06T10:38:10.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.449+0000 7fd90ffff700 1 -- 172.21.15.92:0/3672103123 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9100092e0 con 0x7fd91806b990 2023-10-06T10:38:10.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.450+0000 7fd91f34c700 1 -- 172.21.15.92:0/3672103123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd91806b990 msgr2=0x7fd91806a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:10.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.450+0000 7fd91f34c700 1 --2- 172.21.15.92:0/3672103123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd91806b990 0x7fd91806a260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd910009f20 tx=0x7fd910009840 comp rx=0 tx=0).stop 2023-10-06T10:38:10.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.450+0000 7fd91f34c700 1 -- 172.21.15.92:0/3672103123 shutdown_connections 2023-10-06T10:38:10.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.450+0000 7fd91f34c700 1 --2- 172.21.15.92:0/3672103123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd91806b990 0x7fd91806a260 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:10.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.450+0000 7fd91f34c700 1 --2- 172.21.15.92:0/3672103123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd918068cd0 0x7fd918069d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:10.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.450+0000 7fd91f34c700 1 -- 172.21.15.92:0/3672103123 >> 172.21.15.92:0/3672103123 conn(0x7fd9180c9df0 msgr2=0x7fd91806db70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:10.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.450+0000 7fd91f34c700 1 -- 172.21.15.92:0/3672103123 shutdown_connections 2023-10-06T10:38:10.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.450+0000 7fd91f34c700 1 -- 172.21.15.92:0/3672103123 wait complete. 2023-10-06T10:38:10.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.451+0000 7fd91f34c700 1 Processor -- start 2023-10-06T10:38:10.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.451+0000 7fd91f34c700 1 -- start start 2023-10-06T10:38:10.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.452+0000 7fd91f34c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd918068cd0 0x7fd91814bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:10.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.452+0000 7fd91f34c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd91806b5c0 0x7fd9180c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:10.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.452+0000 7fd91f34c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd91806b990 0x7fd918069c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:10.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.452+0000 7fd91f34c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91815bd40 con 0x7fd91806b990 2023-10-06T10:38:10.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.452+0000 7fd91f34c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91814c260 con 0x7fd918068cd0 2023-10-06T10:38:10.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.452+0000 7fd91f34c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91815bf70 con 0x7fd91806b5c0 2023-10-06T10:38:10.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.452+0000 7fd91d8e9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd91806b990 0x7fd918069c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:10.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.452+0000 7fd91d8e9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd91806b990 0x7fd918069c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51330/0 (socket says 172.21.15.92:51330) 2023-10-06T10:38:10.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.452+0000 7fd91d8e9700 1 -- 172.21.15.92:0/4025603504 learned_addr learned my addr 172.21.15.92:0/4025603504 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:10.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.452+0000 7fd91d0e8700 1 --2- 172.21.15.92:0/4025603504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd918068cd0 0x7fd91814bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:10.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.452+0000 7fd91c8e7700 1 --2- 172.21.15.92:0/4025603504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd91806b5c0 0x7fd9180c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:10.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.452+0000 7fd91d8e9700 1 -- 172.21.15.92:0/4025603504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd91806b5c0 msgr2=0x7fd9180c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:10.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.452+0000 7fd91d8e9700 1 --2- 172.21.15.92:0/4025603504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd91806b5c0 0x7fd9180c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:10.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.452+0000 7fd91d8e9700 1 -- 172.21.15.92:0/4025603504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd918068cd0 msgr2=0x7fd91814bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:10.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.452+0000 7fd91d8e9700 1 --2- 172.21.15.92:0/4025603504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd918068cd0 0x7fd91814bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:10.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.452+0000 7fd91d8e9700 1 -- 172.21.15.92:0/4025603504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd910009ae0 con 0x7fd91806b990 2023-10-06T10:38:10.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.452+0000 7fd91d0e8700 1 --2- 172.21.15.92:0/4025603504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd918068cd0 0x7fd91814bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:10.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.452+0000 7fd91d8e9700 1 --2- 172.21.15.92:0/4025603504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd91806b990 0x7fd918069c40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd91000b6c0 tx=0x7fd910018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:10.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.453+0000 7fd90d7fa700 1 -- 172.21.15.92:0/4025603504 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910008e70 con 0x7fd91806b990 2023-10-06T10:38:10.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.453+0000 7fd90d7fa700 1 -- 172.21.15.92:0/4025603504 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd910026460 con 0x7fd91806b990 2023-10-06T10:38:10.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.453+0000 7fd91f34c700 1 -- 172.21.15.92:0/4025603504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd91815c250 con 0x7fd91806b990 2023-10-06T10:38:10.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.453+0000 7fd90d7fa700 1 -- 172.21.15.92:0/4025603504 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910026820 con 0x7fd91806b990 2023-10-06T10:38:10.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.453+0000 7fd91f34c700 1 -- 172.21.15.92:0/4025603504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91815c810 con 0x7fd91806b990 2023-10-06T10:38:10.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.454+0000 7fd91f34c700 1 -- 172.21.15.92:0/4025603504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd918064a90 con 0x7fd91806b990 2023-10-06T10:38:10.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.454+0000 7fd90d7fa700 1 -- 172.21.15.92:0/4025603504 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fd910003040 con 0x7fd91806b990 2023-10-06T10:38:10.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.454+0000 7fd90d7fa700 1 --2- 172.21.15.92:0/4025603504 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd8fc066ea0 0x7fd8fc069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:10.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.455+0000 7fd91d0e8700 1 --2- 172.21.15.92:0/4025603504 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd8fc066ea0 0x7fd8fc069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:10.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.455+0000 7fd90d7fa700 1 -- 172.21.15.92:0/4025603504 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 5184+0+0 (secure 0 0 0) 0x7fd910086370 con 0x7fd91806b990 2023-10-06T10:38:10.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.455+0000 7fd91d0e8700 1 --2- 172.21.15.92:0/4025603504 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd8fc066ea0 0x7fd8fc069350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd9140011e0 tx=0x7fd914007c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:10.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.459+0000 7fd90d7fa700 1 -- 172.21.15.92:0/4025603504 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9100538d0 con 0x7fd91806b990 2023-10-06T10:38:10.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:10.644+0000 7fd91f34c700 1 -- 172.21.15.92:0/4025603504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd91804bb40 con 0x7fd91806b990 2023-10-06T10:38:11.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.314+0000 7fd90d7fa700 1 -- 172.21.15.92:0/4025603504 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7fd910056ef0 con 0x7fd91806b990 2023-10-06T10:38:11.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.317+0000 7fd91f34c700 1 -- 172.21.15.92:0/4025603504 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd8fc066ea0 msgr2=0x7fd8fc069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:11.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.317+0000 7fd91f34c700 1 --2- 172.21.15.92:0/4025603504 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd8fc066ea0 0x7fd8fc069350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd9140011e0 tx=0x7fd914007c90 comp rx=0 tx=0).stop 2023-10-06T10:38:11.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.317+0000 7fd91f34c700 1 -- 172.21.15.92:0/4025603504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd91806b990 msgr2=0x7fd918069c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:11.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.317+0000 7fd91f34c700 1 --2- 172.21.15.92:0/4025603504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd91806b990 0x7fd918069c40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd91000b6c0 tx=0x7fd910018000 comp rx=0 tx=0).stop 2023-10-06T10:38:11.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.317+0000 7fd91f34c700 1 -- 172.21.15.92:0/4025603504 shutdown_connections 2023-10-06T10:38:11.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.317+0000 7fd91f34c700 1 --2- 172.21.15.92:0/4025603504 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd8fc066ea0 0x7fd8fc069350 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:11.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.317+0000 7fd91f34c700 1 --2- 172.21.15.92:0/4025603504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd91806b990 0x7fd918069c40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:11.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.317+0000 7fd91f34c700 1 --2- 172.21.15.92:0/4025603504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd918068cd0 0x7fd91814bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:11.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.317+0000 7fd91f34c700 1 --2- 172.21.15.92:0/4025603504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd91806b5c0 0x7fd9180c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:11.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.317+0000 7fd91f34c700 1 -- 172.21.15.92:0/4025603504 >> 172.21.15.92:0/4025603504 conn(0x7fd9180c9df0 msgr2=0x7fd91806adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:11.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.317+0000 7fd91f34c700 1 -- 172.21.15.92:0/4025603504 shutdown_connections 2023-10-06T10:38:11.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.317+0000 7fd91f34c700 1 -- 172.21.15.92:0/4025603504 wait complete. 2023-10-06T10:38:11.319 INFO:teuthology.orchestra.run.smithi092.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-06T10:38:11.332 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-06T10:38:11.332 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T10:38:11.332 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T10:38:11.334 INFO:tasks.ceph.mds.a:Started 2023-10-06T10:38:11.334 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T10:38:11.335 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T10:38:11.336 INFO:tasks.ceph.mds.c:Started 2023-10-06T10:38:11.337 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T10:38:11.337 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T10:38:11.339 INFO:tasks.ceph.mds.b:Started 2023-10-06T10:38:11.339 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T10:38:11.339 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T10:38:11.341 INFO:tasks.ceph.mds.d:Started 2023-10-06T10:38:11.341 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-06T10:38:11.342 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:38:11.415 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T10:38:11.416 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T10:38:11.428 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-06T10:38:11.429 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-06T10:38:11.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.471+0000 7f9d8b2af700 1 Processor -- start 2023-10-06T10:38:11.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.471+0000 7f9d8b2af700 1 -- start start 2023-10-06T10:38:11.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.472+0000 7f9d8b2af700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9d8406c330 0x7f9d84161050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:11.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.472+0000 7f9d8b2af700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d8406c9a0 con 0x7f9d840cc8e0 2023-10-06T10:38:11.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.472+0000 7f9d8b2af700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d84156b20 con 0x7f9d84156350 2023-10-06T10:38:11.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.472+0000 7f9d8b2af700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d8406c790 con 0x7f9d8406c330 2023-10-06T10:38:11.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.472+0000 7f9d8884a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9d840cc8e0 0x7f9d8406bc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37114/0 (socket says 172.21.15.92:37114) 2023-10-06T10:38:11.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.472+0000 7f9d8984c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9d8406c330 0x7f9d84161050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:11.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.472+0000 7f9d8884a700 1 -- 172.21.15.92:0/3993354183 learned_addr learned my addr 172.21.15.92:0/3993354183 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:11.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.472+0000 7f9d7bfff700 1 -- 172.21.15.92:0/3993354183 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 665395148 0 0) 0x7f9d8406c9a0 con 0x7f9d840cc8e0 2023-10-06T10:38:11.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.472+0000 7f9d7bfff700 1 -- 172.21.15.92:0/3993354183 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d70003570 con 0x7f9d840cc8e0 2023-10-06T10:38:11.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.472+0000 7f9d8984c700 1 -- 172.21.15.92:0/3993354183 >> v1:172.21.15.124:6789/0 conn(0x7f9d84156350 legacy=0x7f9d840cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:11.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.472+0000 7f9d8984c700 1 -- 172.21.15.92:0/3993354183 >> v1:172.21.15.92:6789/0 conn(0x7f9d840cc8e0 legacy=0x7f9d8406bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:11.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.472+0000 7f9d8984c700 1 -- 172.21.15.92:0/3993354183 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d84156e00 con 0x7f9d8406c330 2023-10-06T10:38:11.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.472+0000 7f9d8984c700 1 --2- 172.21.15.92:0/3993354183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d8406c330 0x7f9d84161050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009e60 tx=0x7f9d7c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c37a2b720950e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:11.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.473+0000 7f9d7bfff700 1 -- 172.21.15.92:0/3993354183 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c00e070 con 0x7f9d8406c330 2023-10-06T10:38:11.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.473+0000 7f9d7bfff700 1 -- 172.21.15.92:0/3993354183 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7c00bb60 con 0x7f9d8406c330 2023-10-06T10:38:11.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.473+0000 7f9d7bfff700 1 -- 172.21.15.92:0/3993354183 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c012430 con 0x7f9d8406c330 2023-10-06T10:38:11.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.473+0000 7f9d8b2af700 1 -- 172.21.15.92:0/3993354183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d8406c330 msgr2=0x7f9d84161050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:11.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.473+0000 7f9d8b2af700 1 --2- 172.21.15.92:0/3993354183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d8406c330 0x7f9d84161050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009e60 tx=0x7f9d7c009780 comp rx=0 tx=0).stop 2023-10-06T10:38:11.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.473+0000 7f9d8b2af700 1 -- 172.21.15.92:0/3993354183 shutdown_connections 2023-10-06T10:38:11.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.473+0000 7f9d8b2af700 1 --2- 172.21.15.92:0/3993354183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d8406c330 0x7f9d84161050 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:11.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.473+0000 7f9d8b2af700 1 -- 172.21.15.92:0/3993354183 >> 172.21.15.92:0/3993354183 conn(0x7f9d840cdeb0 msgr2=0x7f9d840c9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:11.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.473+0000 7f9d8b2af700 1 -- 172.21.15.92:0/3993354183 shutdown_connections 2023-10-06T10:38:11.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.473+0000 7f9d8b2af700 1 -- 172.21.15.92:0/3993354183 wait complete. 2023-10-06T10:38:11.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.474+0000 7f9d8b2af700 1 Processor -- start 2023-10-06T10:38:11.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.474+0000 7f9d8b2af700 1 -- start start 2023-10-06T10:38:11.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.475+0000 7f9d8b2af700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d8406c330 0x7f9d84151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:11.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.475+0000 7f9d8b2af700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d840cc8e0 0x7f9d840a5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:11.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.475+0000 7f9d8b2af700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d84156350 0x7f9d84168ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:11.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.475+0000 7f9d8b2af700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d840a5ae0 con 0x7f9d84156350 2023-10-06T10:38:11.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.475+0000 7f9d8b2af700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d840a7ff0 con 0x7f9d840cc8e0 2023-10-06T10:38:11.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.475+0000 7f9d8b2af700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d840a82d0 con 0x7f9d8406c330 2023-10-06T10:38:11.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.475+0000 7f9d8984c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d84156350 0x7f9d84168ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:11.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.475+0000 7f9d8984c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d84156350 0x7f9d84168ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51394/0 (socket says 172.21.15.92:51394) 2023-10-06T10:38:11.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.475+0000 7f9d8984c700 1 -- 172.21.15.92:0/2049066850 learned_addr learned my addr 172.21.15.92:0/2049066850 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:11.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.475+0000 7f9d8884a700 1 --2- 172.21.15.92:0/2049066850 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d840cc8e0 0x7f9d840a5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:11.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.475+0000 7f9d8904b700 1 --2- 172.21.15.92:0/2049066850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d8406c330 0x7f9d84151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:11.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.476+0000 7f9d8984c700 1 -- 172.21.15.92:0/2049066850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d8406c330 msgr2=0x7f9d84151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:11.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.476+0000 7f9d8984c700 1 --2- 172.21.15.92:0/2049066850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d8406c330 0x7f9d84151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:11.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.476+0000 7f9d8984c700 1 -- 172.21.15.92:0/2049066850 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d840cc8e0 msgr2=0x7f9d840a5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:11.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.476+0000 7f9d8984c700 1 --2- 172.21.15.92:0/2049066850 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d840cc8e0 0x7f9d840a5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:11.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.476+0000 7f9d8984c700 1 -- 172.21.15.92:0/2049066850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7c009a20 con 0x7f9d84156350 2023-10-06T10:38:11.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.476+0000 7f9d8904b700 1 --2- 172.21.15.92:0/2049066850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d8406c330 0x7f9d84151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:11.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.476+0000 7f9d8984c700 1 --2- 172.21.15.92:0/2049066850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d84156350 0x7f9d84168ce0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9d8406df20 tx=0x7f9d7c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:11.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.476+0000 7f9d797fa700 1 -- 172.21.15.92:0/2049066850 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c027470 con 0x7f9d84156350 2023-10-06T10:38:11.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.476+0000 7f9d797fa700 1 -- 172.21.15.92:0/2049066850 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7c0275f0 con 0x7f9d84156350 2023-10-06T10:38:11.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.476+0000 7f9d8b2af700 1 -- 172.21.15.92:0/2049066850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d840a5490 con 0x7f9d84156350 2023-10-06T10:38:11.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.476+0000 7f9d797fa700 1 -- 172.21.15.92:0/2049066850 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0279b0 con 0x7f9d84156350 2023-10-06T10:38:11.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.476+0000 7f9d8b2af700 1 -- 172.21.15.92:0/2049066850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d841520d0 con 0x7f9d84156350 2023-10-06T10:38:11.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.477+0000 7f9d8b2af700 1 -- 172.21.15.92:0/2049066850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d840a5c90 con 0x7f9d84156350 2023-10-06T10:38:11.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.477+0000 7f9d797fa700 1 -- 172.21.15.92:0/2049066850 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f9d7c024070 con 0x7f9d84156350 2023-10-06T10:38:11.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.477+0000 7f9d797fa700 1 --2- 172.21.15.92:0/2049066850 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9d68066ea0 0x7f9d68069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:11.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.478+0000 7f9d797fa700 1 -- 172.21.15.92:0/2049066850 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f9d7c086ce0 con 0x7f9d84156350 2023-10-06T10:38:11.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.478+0000 7f9d8904b700 1 --2- 172.21.15.92:0/2049066850 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9d68066ea0 0x7f9d68069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:11.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.478+0000 7f9d8904b700 1 --2- 172.21.15.92:0/2049066850 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9d68066ea0 0x7f9d68069350 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9d800017a0 tx=0x7f9d80009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:11.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.482+0000 7f9d797fa700 1 -- 172.21.15.92:0/2049066850 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d7c054220 con 0x7f9d84156350 2023-10-06T10:38:11.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.682+0000 7f9d8b2af700 1 -- 172.21.15.92:0/2049066850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d84064a90 con 0x7f9d84156350 2023-10-06T10:38:11.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.683+0000 7f9d797fa700 1 -- 172.21.15.92:0/2049066850 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+13674 (secure 0 0 0) 0x7f9d7c057840 con 0x7f9d84156350 2023-10-06T10:38:11.684 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:11.684 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":15,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:11.311311+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:38:11.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.685+0000 7f9d8b2af700 1 -- 172.21.15.92:0/2049066850 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9d68066ea0 msgr2=0x7f9d68069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:11.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.685+0000 7f9d8b2af700 1 --2- 172.21.15.92:0/2049066850 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9d68066ea0 0x7f9d68069350 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9d800017a0 tx=0x7f9d80009040 comp rx=0 tx=0).stop 2023-10-06T10:38:11.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.685+0000 7f9d8b2af700 1 -- 172.21.15.92:0/2049066850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d84156350 msgr2=0x7f9d84168ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:11.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.685+0000 7f9d8b2af700 1 --2- 172.21.15.92:0/2049066850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d84156350 0x7f9d84168ce0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9d8406df20 tx=0x7f9d7c019000 comp rx=0 tx=0).stop 2023-10-06T10:38:11.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.685+0000 7f9d8b2af700 1 -- 172.21.15.92:0/2049066850 shutdown_connections 2023-10-06T10:38:11.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.685+0000 7f9d8b2af700 1 --2- 172.21.15.92:0/2049066850 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9d68066ea0 0x7f9d68069350 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:11.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.685+0000 7f9d8b2af700 1 --2- 172.21.15.92:0/2049066850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d84156350 0x7f9d84168ce0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:11.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.685+0000 7f9d8b2af700 1 --2- 172.21.15.92:0/2049066850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d8406c330 0x7f9d84151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:11.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.685+0000 7f9d8b2af700 1 --2- 172.21.15.92:0/2049066850 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d840cc8e0 0x7f9d840a5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:11.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.685+0000 7f9d8b2af700 1 -- 172.21.15.92:0/2049066850 >> 172.21.15.92:0/2049066850 conn(0x7f9d840cdeb0 msgr2=0x7f9d84153a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:11.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.685+0000 7f9d8b2af700 1 -- 172.21.15.92:0/2049066850 shutdown_connections 2023-10-06T10:38:11.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.685+0000 7f9d8b2af700 1 -- 172.21.15.92:0/2049066850 wait complete. 2023-10-06T10:38:11.698 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T10:38:09.680559+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T10:38:10.120786+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T10:38:11.699 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T10:38:11.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.818+0000 7fc7b2576700 1 Processor -- start 2023-10-06T10:38:11.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.818+0000 7fc7b2576700 1 -- start start 2023-10-06T10:38:11.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.819+0000 7fc7b2576700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7ac0cd3c0 0x7fc7ac0ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:11.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.819+0000 7fc7b2576700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc7ac0b29f0 0x7fc7ac0cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.819+0000 7fc7b2576700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7ac0cc970 con 0x7fc7ac0b2620 2023-10-06T10:38:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.819+0000 7fc7b2576700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7ac0cf520 con 0x7fc7ac0b29f0 2023-10-06T10:38:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.819+0000 7fc7b2576700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac0cbd60 con 0x7fc7ac0cd3c0 2023-10-06T10:38:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.819+0000 7fc7b0d73700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7ac0cd3c0 0x7fc7ac0ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.819+0000 7fc7b1d75700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc7ac0b29f0 0x7fc7ac0cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.819+0000 7fc7b1574700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fc7ac0b2620 0x7fc7ac0cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:48468/0 (socket says 172.21.15.92:48468) 2023-10-06T10:38:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.819+0000 7fc7b0d73700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7ac0cd3c0 0x7fc7ac0ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:53790/0 (socket says 172.21.15.92:53790) 2023-10-06T10:38:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.819+0000 7fc7b1574700 1 -- 172.21.15.92:0/702514836 learned_addr learned my addr 172.21.15.92:0/702514836 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.819+0000 7fc7b1d75700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc7ac0b29f0 0x7fc7ac0cefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:50260/0 (socket says 172.21.15.92:50260) 2023-10-06T10:38:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.819+0000 7fc7b0d73700 1 -- 172.21.15.92:0/702514836 >> v2:172.21.15.124:3300/0 conn(0x7fc7ac0b29f0 msgr2=0x7fc7ac0cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.819+0000 7fc7b0d73700 1 --2- 172.21.15.92:0/702514836 >> v2:172.21.15.124:3300/0 conn(0x7fc7ac0b29f0 0x7fc7ac0cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:11.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.819+0000 7fc7b0d73700 1 -- 172.21.15.92:0/702514836 >> v1:172.21.15.124:6789/0 conn(0x7fc7ac0b2620 legacy=0x7fc7ac0cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:11.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.819+0000 7fc7b0d73700 1 -- 172.21.15.92:0/702514836 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ac0cc040 con 0x7fc7ac0cd3c0 2023-10-06T10:38:11.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.819+0000 7fc7abfff700 1 -- 172.21.15.92:0/702514836 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054673284 0 0) 0x7fc7ac0cc970 con 0x7fc7ac0b2620 2023-10-06T10:38:11.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.819+0000 7fc7b1d75700 1 --2- 172.21.15.92:0/702514836 >> v2:172.21.15.124:3300/0 conn(0x7fc7ac0b29f0 0x7fc7ac0cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:11.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.820+0000 7fc7b0d73700 1 --2- 172.21.15.92:0/702514836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7ac0cd3c0 0x7fc7ac0ceaa0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc7a00098e0 tx=0x7fc7a00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d28ea04d5945675f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:11.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.820+0000 7fc7abfff700 1 -- 172.21.15.92:0/702514836 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a000b640 con 0x7fc7ac0cd3c0 2023-10-06T10:38:11.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.820+0000 7fc7abfff700 1 -- 172.21.15.92:0/702514836 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a000b7a0 con 0x7fc7ac0cd3c0 2023-10-06T10:38:11.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.820+0000 7fc7abfff700 1 -- 172.21.15.92:0/702514836 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a000bb40 con 0x7fc7ac0cd3c0 2023-10-06T10:38:11.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.820+0000 7fc7b2576700 1 -- 172.21.15.92:0/702514836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7ac0cd3c0 msgr2=0x7fc7ac0ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:11.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.820+0000 7fc7b2576700 1 --2- 172.21.15.92:0/702514836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7ac0cd3c0 0x7fc7ac0ceaa0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc7a00098e0 tx=0x7fc7a00091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:11.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.820+0000 7fc7b2576700 1 -- 172.21.15.92:0/702514836 shutdown_connections 2023-10-06T10:38:11.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.820+0000 7fc7b2576700 1 --2- 172.21.15.92:0/702514836 >> v2:172.21.15.124:3300/0 conn(0x7fc7ac0b29f0 0x7fc7ac0cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:11.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.820+0000 7fc7b2576700 1 --2- 172.21.15.92:0/702514836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7ac0cd3c0 0x7fc7ac0ceaa0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:11.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.820+0000 7fc7b2576700 1 -- 172.21.15.92:0/702514836 >> 172.21.15.92:0/702514836 conn(0x7fc7ac099520 msgr2=0x7fc7ac06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:11.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.820+0000 7fc7b2576700 1 -- 172.21.15.92:0/702514836 shutdown_connections 2023-10-06T10:38:11.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.820+0000 7fc7b2576700 1 -- 172.21.15.92:0/702514836 wait complete. 2023-10-06T10:38:11.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.821+0000 7fc7b2576700 1 Processor -- start 2023-10-06T10:38:11.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.821+0000 7fc7b2576700 1 -- start start 2023-10-06T10:38:11.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.822+0000 7fc7b2576700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7ac0b2620 0x7fc7ac155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:11.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.822+0000 7fc7b2576700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ac0b29f0 0x7fc7ac0b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:11.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.822+0000 7fc7b2576700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7ac0cd3c0 0x7fc7ac0b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:11.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.822+0000 7fc7b2576700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac155c70 con 0x7fc7ac0b29f0 2023-10-06T10:38:11.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.822+0000 7fc7b2576700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac15b3f0 con 0x7fc7ac0b2620 2023-10-06T10:38:11.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.822+0000 7fc7b2576700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac0b5ea0 con 0x7fc7ac0cd3c0 2023-10-06T10:38:11.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.822+0000 7fc7b0d73700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ac0b29f0 0x7fc7ac0b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:11.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.822+0000 7fc7b0d73700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ac0b29f0 0x7fc7ac0b39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51400/0 (socket says 172.21.15.92:51400) 2023-10-06T10:38:11.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.822+0000 7fc7b0d73700 1 -- 172.21.15.92:0/754300741 learned_addr learned my addr 172.21.15.92:0/754300741 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:11.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.822+0000 7fc7b1d75700 1 --2- 172.21.15.92:0/754300741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7ac0cd3c0 0x7fc7ac0b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:11.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.822+0000 7fc7b1574700 1 --2- 172.21.15.92:0/754300741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7ac0b2620 0x7fc7ac155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:11.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.823+0000 7fc7b0d73700 1 -- 172.21.15.92:0/754300741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7ac0cd3c0 msgr2=0x7fc7ac0b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:11.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.823+0000 7fc7b0d73700 1 --2- 172.21.15.92:0/754300741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7ac0cd3c0 0x7fc7ac0b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:11.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.823+0000 7fc7b0d73700 1 -- 172.21.15.92:0/754300741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7ac0b2620 msgr2=0x7fc7ac155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:11.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.823+0000 7fc7b0d73700 1 --2- 172.21.15.92:0/754300741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7ac0b2620 0x7fc7ac155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:11.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.823+0000 7fc7b0d73700 1 -- 172.21.15.92:0/754300741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c009a20 con 0x7fc7ac0b29f0 2023-10-06T10:38:11.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.823+0000 7fc7b1d75700 1 --2- 172.21.15.92:0/754300741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7ac0cd3c0 0x7fc7ac0b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:11.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.823+0000 7fc7b0d73700 1 --2- 172.21.15.92:0/754300741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ac0b29f0 0x7fc7ac0b39c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009f80 tx=0x7fc7a0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:11.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.823+0000 7fc7a97fa700 1 -- 172.21.15.92:0/754300741 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0000d10 con 0x7fc7ac0b29f0 2023-10-06T10:38:11.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.823+0000 7fc7a97fa700 1 -- 172.21.15.92:0/754300741 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a000b910 con 0x7fc7ac0b29f0 2023-10-06T10:38:11.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.823+0000 7fc7a97fa700 1 -- 172.21.15.92:0/754300741 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0025650 con 0x7fc7ac0b29f0 2023-10-06T10:38:11.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.823+0000 7fc7b2576700 1 -- 172.21.15.92:0/754300741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a0009420 con 0x7fc7ac0b29f0 2023-10-06T10:38:11.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.823+0000 7fc7b2576700 1 -- 172.21.15.92:0/754300741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7ac0c7380 con 0x7fc7ac0b29f0 2023-10-06T10:38:11.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.824+0000 7fc7b2576700 1 -- 172.21.15.92:0/754300741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7ac05f230 con 0x7fc7ac0b29f0 2023-10-06T10:38:11.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.824+0000 7fc7a97fa700 1 -- 172.21.15.92:0/754300741 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fc7a0023070 con 0x7fc7ac0b29f0 2023-10-06T10:38:11.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.825+0000 7fc7a97fa700 1 --2- 172.21.15.92:0/754300741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc798066ea0 0x7fc798069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:11.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.825+0000 7fc7a97fa700 1 -- 172.21.15.92:0/754300741 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc7a0085750 con 0x7fc7ac0b29f0 2023-10-06T10:38:11.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.829+0000 7fc7a97fa700 1 -- 172.21.15.92:0/754300741 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7a0052d50 con 0x7fc7ac0b29f0 2023-10-06T10:38:11.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.840+0000 7fc7b1574700 1 --2- 172.21.15.92:0/754300741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc798066ea0 0x7fc798069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:11.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:11.840+0000 7fc7b1574700 1 --2- 172.21.15.92:0/754300741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc798066ea0 0x7fc798069350 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4007fd0 tx=0x7fc7a4007f20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:12.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.021+0000 7fc7b2576700 1 -- 172.21.15.92:0/754300741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc7ac0cc320 con 0x7fc7ac0b29f0 2023-10-06T10:38:12.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.021+0000 7fc7a97fa700 1 -- 172.21.15.92:0/754300741 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc7a0056370 con 0x7fc7ac0b29f0 2023-10-06T10:38:12.023 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T10:38:12.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.023+0000 7fc7b2576700 1 -- 172.21.15.92:0/754300741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc798066ea0 msgr2=0x7fc798069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:12.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.023+0000 7fc7b2576700 1 --2- 172.21.15.92:0/754300741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc798066ea0 0x7fc798069350 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4007fd0 tx=0x7fc7a4007f20 comp rx=0 tx=0).stop 2023-10-06T10:38:12.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.023+0000 7fc7b2576700 1 -- 172.21.15.92:0/754300741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ac0b29f0 msgr2=0x7fc7ac0b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:12.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.023+0000 7fc7b2576700 1 --2- 172.21.15.92:0/754300741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ac0b29f0 0x7fc7ac0b39c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009f80 tx=0x7fc7a0017000 comp rx=0 tx=0).stop 2023-10-06T10:38:12.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.023+0000 7fc7b2576700 1 -- 172.21.15.92:0/754300741 shutdown_connections 2023-10-06T10:38:12.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.023+0000 7fc7b2576700 1 --2- 172.21.15.92:0/754300741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc798066ea0 0x7fc798069350 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.023+0000 7fc7b2576700 1 --2- 172.21.15.92:0/754300741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7ac0cd3c0 0x7fc7ac0b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.023+0000 7fc7b2576700 1 --2- 172.21.15.92:0/754300741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7ac0b2620 0x7fc7ac155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.023+0000 7fc7b2576700 1 --2- 172.21.15.92:0/754300741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ac0b29f0 0x7fc7ac0b39c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.023+0000 7fc7b2576700 1 -- 172.21.15.92:0/754300741 >> 172.21.15.92:0/754300741 conn(0x7fc7ac099520 msgr2=0x7fc7ac0c5350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:12.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.024+0000 7fc7b2576700 1 -- 172.21.15.92:0/754300741 shutdown_connections 2023-10-06T10:38:12.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.024+0000 7fc7b2576700 1 -- 172.21.15.92:0/754300741 wait complete. 2023-10-06T10:38:12.036 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T10:38:12.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.157+0000 7fb4d45fd700 1 Processor -- start 2023-10-06T10:38:12.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.157+0000 7fb4d45fd700 1 -- start start 2023-10-06T10:38:12.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.158+0000 7fb4d45fd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4cc06de00 0x7fb4cc06e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:12.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.158+0000 7fb4d45fd700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4cc0cacc0 con 0x7fb4cc0c9f60 2023-10-06T10:38:12.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.158+0000 7fb4d45fd700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4cc0676e0 con 0x7fb4cc1592b0 2023-10-06T10:38:12.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.158+0000 7fb4d45fd700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc0ca420 con 0x7fb4cc06de00 2023-10-06T10:38:12.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.158+0000 7fb4d1b98700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb4cc0c9f60 0x7fb4cc158ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37128/0 (socket says 172.21.15.92:37128) 2023-10-06T10:38:12.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.158+0000 7fb4d1b98700 1 -- 172.21.15.92:0/3333667778 learned_addr learned my addr 172.21.15.92:0/3333667778 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:12.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.158+0000 7fb4d2399700 1 --2- 172.21.15.92:0/3333667778 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4cc06de00 0x7fb4cc06e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:12.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.158+0000 7fb4d1397700 1 -- 172.21.15.92:0/3333667778 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2522027887 0 0) 0x7fb4cc0676e0 con 0x7fb4cc1592b0 2023-10-06T10:38:12.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.158+0000 7fb4d1397700 1 -- 172.21.15.92:0/3333667778 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4b8003570 con 0x7fb4cc1592b0 2023-10-06T10:38:12.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.158+0000 7fb4d2399700 1 -- 172.21.15.92:0/3333667778 >> v1:172.21.15.124:6789/0 conn(0x7fb4cc1592b0 legacy=0x7fb4cc067b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:12.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.158+0000 7fb4d2399700 1 -- 172.21.15.92:0/3333667778 >> v1:172.21.15.92:6789/0 conn(0x7fb4cc0c9f60 legacy=0x7fb4cc158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:12.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.158+0000 7fb4d2399700 1 -- 172.21.15.92:0/3333667778 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc0ca700 con 0x7fb4cc06de00 2023-10-06T10:38:12.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.159+0000 7fb4d2399700 1 --2- 172.21.15.92:0/3333667778 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4cc06de00 0x7fb4cc06e1d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009910 tx=0x7fb4bc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=553e67c593256da0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:12.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.159+0000 7fb4d1397700 1 -- 172.21.15.92:0/3333667778 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc00b680 con 0x7fb4cc06de00 2023-10-06T10:38:12.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.159+0000 7fb4d1397700 1 -- 172.21.15.92:0/3333667778 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4bc00b7e0 con 0x7fb4cc06de00 2023-10-06T10:38:12.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.159+0000 7fb4d1397700 1 -- 172.21.15.92:0/3333667778 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc00bba0 con 0x7fb4cc06de00 2023-10-06T10:38:12.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.160+0000 7fb4d45fd700 1 -- 172.21.15.92:0/3333667778 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4cc06de00 msgr2=0x7fb4cc06e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:12.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.160+0000 7fb4d45fd700 1 --2- 172.21.15.92:0/3333667778 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4cc06de00 0x7fb4cc06e1d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009910 tx=0x7fb4bc0091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.160+0000 7fb4d45fd700 1 -- 172.21.15.92:0/3333667778 shutdown_connections 2023-10-06T10:38:12.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.160+0000 7fb4d45fd700 1 --2- 172.21.15.92:0/3333667778 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4cc06de00 0x7fb4cc06e1d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.160+0000 7fb4d45fd700 1 -- 172.21.15.92:0/3333667778 >> 172.21.15.92:0/3333667778 conn(0x7fb4cc0cd3d0 msgr2=0x7fb4cc0b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:12.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.160+0000 7fb4d45fd700 1 -- 172.21.15.92:0/3333667778 shutdown_connections 2023-10-06T10:38:12.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.160+0000 7fb4d45fd700 1 -- 172.21.15.92:0/3333667778 wait complete. 2023-10-06T10:38:12.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.161+0000 7fb4d45fd700 1 Processor -- start 2023-10-06T10:38:12.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.161+0000 7fb4d45fd700 1 -- start start 2023-10-06T10:38:12.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.162+0000 7fb4d45fd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4cc06de00 0x7fb4cc13e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:12.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.162+0000 7fb4d45fd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4cc0c9f60 0x7fb4cc14f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:12.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.162+0000 7fb4d45fd700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4cc1592b0 0x7fb4cc168970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:12.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.162+0000 7fb4d45fd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc151230 con 0x7fb4cc06de00 2023-10-06T10:38:12.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.162+0000 7fb4d45fd700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc150340 con 0x7fb4cc1592b0 2023-10-06T10:38:12.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.162+0000 7fb4d45fd700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc150620 con 0x7fb4cc0c9f60 2023-10-06T10:38:12.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.162+0000 7fb4d2399700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4cc06de00 0x7fb4cc13e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:12.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.162+0000 7fb4d2399700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4cc06de00 0x7fb4cc13e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51412/0 (socket says 172.21.15.92:51412) 2023-10-06T10:38:12.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.162+0000 7fb4d2399700 1 -- 172.21.15.92:0/949290698 learned_addr learned my addr 172.21.15.92:0/949290698 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:12.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.162+0000 7fb4d1b98700 1 --2- 172.21.15.92:0/949290698 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4cc0c9f60 0x7fb4cc14f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:12.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.162+0000 7fb4d2b9a700 1 --2- 172.21.15.92:0/949290698 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4cc1592b0 0x7fb4cc168970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:12.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.162+0000 7fb4d2399700 1 -- 172.21.15.92:0/949290698 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4cc0c9f60 msgr2=0x7fb4cc14f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:12.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.162+0000 7fb4d2399700 1 --2- 172.21.15.92:0/949290698 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4cc0c9f60 0x7fb4cc14f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.162+0000 7fb4d2399700 1 -- 172.21.15.92:0/949290698 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4cc1592b0 msgr2=0x7fb4cc168970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:12.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.162+0000 7fb4d2399700 1 --2- 172.21.15.92:0/949290698 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4cc1592b0 0x7fb4cc168970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.162+0000 7fb4d2399700 1 -- 172.21.15.92:0/949290698 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4bc009420 con 0x7fb4cc06de00 2023-10-06T10:38:12.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.162+0000 7fb4d2b9a700 1 --2- 172.21.15.92:0/949290698 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4cc1592b0 0x7fb4cc168970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:12.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.162+0000 7fb4d2399700 1 --2- 172.21.15.92:0/949290698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4cc06de00 0x7fb4cc13e440 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc069610 tx=0x7fb4bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:12.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.162+0000 7fb4c27fc700 1 -- 172.21.15.92:0/949290698 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc00bb60 con 0x7fb4cc06de00 2023-10-06T10:38:12.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.162+0000 7fb4c27fc700 1 -- 172.21.15.92:0/949290698 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4bc000c90 con 0x7fb4cc06de00 2023-10-06T10:38:12.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.163+0000 7fb4d45fd700 1 -- 172.21.15.92:0/949290698 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4cc150900 con 0x7fb4cc06de00 2023-10-06T10:38:12.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.163+0000 7fb4c27fc700 1 -- 172.21.15.92:0/949290698 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc01a660 con 0x7fb4cc06de00 2023-10-06T10:38:12.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.163+0000 7fb4d45fd700 1 -- 172.21.15.92:0/949290698 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4cc150ec0 con 0x7fb4cc06de00 2023-10-06T10:38:12.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.163+0000 7fb4d45fd700 1 -- 172.21.15.92:0/949290698 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4cc064a90 con 0x7fb4cc06de00 2023-10-06T10:38:12.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.164+0000 7fb4c27fc700 1 -- 172.21.15.92:0/949290698 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fb4bc01aa10 con 0x7fb4cc06de00 2023-10-06T10:38:12.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.164+0000 7fb4c27fc700 1 --2- 172.21.15.92:0/949290698 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb4b0066ea0 0x7fb4b0069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:12.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.164+0000 7fb4d1b98700 1 --2- 172.21.15.92:0/949290698 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb4b0066ea0 0x7fb4b0069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:12.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.164+0000 7fb4c27fc700 1 -- 172.21.15.92:0/949290698 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb4bc085ee0 con 0x7fb4cc06de00 2023-10-06T10:38:12.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.165+0000 7fb4d1b98700 1 --2- 172.21.15.92:0/949290698 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb4b0066ea0 0x7fb4b0069350 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0c8a80 tx=0x7fb4c8007eb0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:12.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.169+0000 7fb4c27fc700 1 -- 172.21.15.92:0/949290698 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4bc053380 con 0x7fb4cc06de00 2023-10-06T10:38:12.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.355+0000 7fb4d45fd700 1 -- 172.21.15.92:0/949290698 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4cc068f20 con 0x7fb4cc06de00 2023-10-06T10:38:12.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.355+0000 7fb4c27fc700 1 -- 172.21.15.92:0/949290698 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb4bc0569a0 con 0x7fb4cc06de00 2023-10-06T10:38:12.356 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T10:38:12.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.358+0000 7fb4d45fd700 1 -- 172.21.15.92:0/949290698 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb4b0066ea0 msgr2=0x7fb4b0069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:12.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.358+0000 7fb4d45fd700 1 --2- 172.21.15.92:0/949290698 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb4b0066ea0 0x7fb4b0069350 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0c8a80 tx=0x7fb4c8007eb0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.358+0000 7fb4d45fd700 1 -- 172.21.15.92:0/949290698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4cc06de00 msgr2=0x7fb4cc13e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:12.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.358+0000 7fb4d45fd700 1 --2- 172.21.15.92:0/949290698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4cc06de00 0x7fb4cc13e440 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc069610 tx=0x7fb4bc017000 comp rx=0 tx=0).stop 2023-10-06T10:38:12.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.358+0000 7fb4d45fd700 1 -- 172.21.15.92:0/949290698 shutdown_connections 2023-10-06T10:38:12.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.358+0000 7fb4d45fd700 1 --2- 172.21.15.92:0/949290698 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb4b0066ea0 0x7fb4b0069350 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.358+0000 7fb4d45fd700 1 --2- 172.21.15.92:0/949290698 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4cc1592b0 0x7fb4cc168970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.358+0000 7fb4d45fd700 1 --2- 172.21.15.92:0/949290698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4cc06de00 0x7fb4cc13e440 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.358+0000 7fb4d45fd700 1 --2- 172.21.15.92:0/949290698 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4cc0c9f60 0x7fb4cc14f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.358+0000 7fb4d45fd700 1 -- 172.21.15.92:0/949290698 >> 172.21.15.92:0/949290698 conn(0x7fb4cc0cd3d0 msgr2=0x7fb4cc0b17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:12.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.358+0000 7fb4d45fd700 1 -- 172.21.15.92:0/949290698 shutdown_connections 2023-10-06T10:38:12.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.358+0000 7fb4d45fd700 1 -- 172.21.15.92:0/949290698 wait complete. 2023-10-06T10:38:12.370 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:38:12.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.485+0000 7f4a09432700 1 Processor -- start 2023-10-06T10:38:12.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.485+0000 7f4a09432700 1 -- start start 2023-10-06T10:38:12.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.486+0000 7f4a09432700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4a04159560 0x7f4a040aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:12.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.486+0000 7f4a09432700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a040b0c60 con 0x7f4a040b0760 2023-10-06T10:38:12.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.486+0000 7f4a09432700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a040b0f40 con 0x7f4a040b49a0 2023-10-06T10:38:12.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.486+0000 7f4a09432700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a040b1220 con 0x7f4a04159560 2023-10-06T10:38:12.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.486+0000 7f4a02ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4a04159560 0x7f4a040aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:12.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.486+0000 7f4a02ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4a04159560 0x7f4a040aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51414/0 (socket says 172.21.15.92:51414) 2023-10-06T10:38:12.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.486+0000 7f4a02ffd700 1 -- 172.21.15.92:0/764653234 learned_addr learned my addr 172.21.15.92:0/764653234 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:12.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.487+0000 7f4a02ffd700 1 -- 172.21.15.92:0/764653234 >> v1:172.21.15.124:6789/0 conn(0x7f4a040b49a0 legacy=0x7f4a040aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:12.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.487+0000 7f4a02ffd700 1 -- 172.21.15.92:0/764653234 >> v1:172.21.15.92:6789/0 conn(0x7f4a040b0760 legacy=0x7f4a040b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:12.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.487+0000 7f4a02ffd700 1 -- 172.21.15.92:0/764653234 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a040b1500 con 0x7f4a04159560 2023-10-06T10:38:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.487+0000 7f4a027fc700 1 -- 172.21.15.92:0/764653234 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115832026 0 0) 0x7f4a040b0c60 con 0x7f4a040b0760 2023-10-06T10:38:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.487+0000 7f4a02ffd700 1 --2- 172.21.15.92:0/764653234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a04159560 0x7f4a040aa530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f49f40099c0 tx=0x7f49f4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e0458fde6f8711 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.487+0000 7f4a027fc700 1 -- 172.21.15.92:0/764653234 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f4005c10 con 0x7f4a04159560 2023-10-06T10:38:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.487+0000 7f4a027fc700 1 -- 172.21.15.92:0/764653234 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f4005d70 con 0x7f4a04159560 2023-10-06T10:38:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.487+0000 7f4a027fc700 1 -- 172.21.15.92:0/764653234 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f400b800 con 0x7f4a04159560 2023-10-06T10:38:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.487+0000 7f4a09432700 1 -- 172.21.15.92:0/764653234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a04159560 msgr2=0x7f4a040aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.487+0000 7f4a09432700 1 --2- 172.21.15.92:0/764653234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a04159560 0x7f4a040aa530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f49f40099c0 tx=0x7f49f4009220 comp rx=0 tx=0).stop 2023-10-06T10:38:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.487+0000 7f4a09432700 1 -- 172.21.15.92:0/764653234 shutdown_connections 2023-10-06T10:38:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.487+0000 7f4a09432700 1 --2- 172.21.15.92:0/764653234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a04159560 0x7f4a040aa530 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.487+0000 7f4a09432700 1 -- 172.21.15.92:0/764653234 >> 172.21.15.92:0/764653234 conn(0x7f4a040997f0 msgr2=0x7f4a0406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.487+0000 7f4a09432700 1 -- 172.21.15.92:0/764653234 shutdown_connections 2023-10-06T10:38:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.487+0000 7f4a09432700 1 -- 172.21.15.92:0/764653234 wait complete. 2023-10-06T10:38:12.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.489+0000 7f4a09432700 1 Processor -- start 2023-10-06T10:38:12.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.489+0000 7f4a09432700 1 -- start start 2023-10-06T10:38:12.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.489+0000 7f4a09432700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a040b0760 0x7f4a04169cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:12.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.489+0000 7f4a09432700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a040b49a0 0x7f4a0415b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:12.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.489+0000 7f4a09432700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a04159560 0x7f4a04158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:12.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.489+0000 7f4a09432700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0415bc40 con 0x7f4a040b49a0 2023-10-06T10:38:12.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.489+0000 7f4a09432700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0415bf20 con 0x7f4a04159560 2023-10-06T10:38:12.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.489+0000 7f4a09432700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0415c200 con 0x7f4a040b0760 2023-10-06T10:38:12.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.489+0000 7f49fa7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a040b49a0 0x7f4a0415b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:12.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.489+0000 7f49fa7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a040b49a0 0x7f4a0415b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51422/0 (socket says 172.21.15.92:51422) 2023-10-06T10:38:12.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.489+0000 7f49fa7fc700 1 -- 172.21.15.92:0/2849904784 learned_addr learned my addr 172.21.15.92:0/2849904784 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:12.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.490+0000 7f4a037fe700 1 --2- 172.21.15.92:0/2849904784 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a04159560 0x7f4a04158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:12.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.490+0000 7f49fa7fc700 1 -- 172.21.15.92:0/2849904784 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a040b0760 msgr2=0x7f4a04169cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:12.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.490+0000 7f4a02ffd700 1 --2- 172.21.15.92:0/2849904784 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a040b0760 0x7f4a04169cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:12.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.490+0000 7f49fa7fc700 1 --2- 172.21.15.92:0/2849904784 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a040b0760 0x7f4a04169cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.490+0000 7f49fa7fc700 1 -- 172.21.15.92:0/2849904784 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a04159560 msgr2=0x7f4a04158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:12.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.490+0000 7f49fa7fc700 1 --2- 172.21.15.92:0/2849904784 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a04159560 0x7f4a04158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.490+0000 7f49fa7fc700 1 -- 172.21.15.92:0/2849904784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f4009480 con 0x7f4a040b49a0 2023-10-06T10:38:12.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.490+0000 7f49fa7fc700 1 --2- 172.21.15.92:0/2849904784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a040b49a0 0x7f4a0415b340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00b5e0 tx=0x7f49ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:12.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.490+0000 7f49fb7fe700 1 -- 172.21.15.92:0/2849904784 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec00f510 con 0x7f4a040b49a0 2023-10-06T10:38:12.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.490+0000 7f49fb7fe700 1 -- 172.21.15.92:0/2849904784 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ec00f670 con 0x7f4a040b49a0 2023-10-06T10:38:12.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.490+0000 7f4a09432700 1 -- 172.21.15.92:0/2849904784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0415c4e0 con 0x7f4a040b49a0 2023-10-06T10:38:12.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.490+0000 7f49fb7fe700 1 -- 172.21.15.92:0/2849904784 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec00fa30 con 0x7f4a040b49a0 2023-10-06T10:38:12.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.490+0000 7f4a09432700 1 -- 172.21.15.92:0/2849904784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a04158dc0 con 0x7f4a040b49a0 2023-10-06T10:38:12.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.491+0000 7f4a09432700 1 -- 172.21.15.92:0/2849904784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a04064a90 con 0x7f4a040b49a0 2023-10-06T10:38:12.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.491+0000 7f49fb7fe700 1 -- 172.21.15.92:0/2849904784 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f49ec01a400 con 0x7f4a040b49a0 2023-10-06T10:38:12.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.492+0000 7f49fb7fe700 1 --2- 172.21.15.92:0/2849904784 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f49e4066ea0 0x7f49e4069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:12.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.492+0000 7f49fb7fe700 1 -- 172.21.15.92:0/2849904784 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f49ec083d20 con 0x7f4a040b49a0 2023-10-06T10:38:12.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.492+0000 7f4a02ffd700 1 --2- 172.21.15.92:0/2849904784 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f49e4066ea0 0x7f49e4069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:12.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.492+0000 7f4a02ffd700 1 --2- 172.21.15.92:0/2849904784 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f49e4066ea0 0x7f49e4069350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009990 tx=0x7f49f4015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:12.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.496+0000 7f49fb7fe700 1 -- 172.21.15.92:0/2849904784 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f49ec0512e0 con 0x7f4a040b49a0 2023-10-06T10:38:12.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.724+0000 7f4a09432700 1 -- 172.21.15.92:0/2849904784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a040b17e0 con 0x7f4a040b49a0 2023-10-06T10:38:12.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.725+0000 7f49fb7fe700 1 -- 172.21.15.92:0/2849904784 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7f49ec054900 con 0x7f4a040b49a0 2023-10-06T10:38:12.726 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:12.726 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2303424485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2303424485},{"type":"v1","addr":"172.21.15.92:6836","nonce":2303424485}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1027343065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1027343065},{"type":"v1","addr":"172.21.15.92:6837","nonce":1027343065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2667820379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2667820379},{"type":"v1","addr":"172.21.15.124:6834","nonce":2667820379}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1919704285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":1919704285},{"type":"v1","addr":"172.21.15.124:6835","nonce":1919704285}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-10-06T10:38:12.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.727+0000 7f4a09432700 1 -- 172.21.15.92:0/2849904784 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f49e4066ea0 msgr2=0x7f49e4069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:12.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.727+0000 7f4a09432700 1 --2- 172.21.15.92:0/2849904784 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f49e4066ea0 0x7f49e4069350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009990 tx=0x7f49f4015000 comp rx=0 tx=0).stop 2023-10-06T10:38:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.727+0000 7f4a09432700 1 -- 172.21.15.92:0/2849904784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a040b49a0 msgr2=0x7f4a0415b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.727+0000 7f4a09432700 1 --2- 172.21.15.92:0/2849904784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a040b49a0 0x7f4a0415b340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00b5e0 tx=0x7f49ec0091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.727+0000 7f4a09432700 1 -- 172.21.15.92:0/2849904784 shutdown_connections 2023-10-06T10:38:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.727+0000 7f4a09432700 1 --2- 172.21.15.92:0/2849904784 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f49e4066ea0 0x7f49e4069350 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.727+0000 7f4a09432700 1 --2- 172.21.15.92:0/2849904784 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a04159560 0x7f4a04158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.727+0000 7f4a09432700 1 --2- 172.21.15.92:0/2849904784 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a040b0760 0x7f4a04169cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.727+0000 7f4a09432700 1 --2- 172.21.15.92:0/2849904784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a040b49a0 0x7f4a0415b340 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.727+0000 7f4a09432700 1 -- 172.21.15.92:0/2849904784 >> 172.21.15.92:0/2849904784 conn(0x7f4a040997f0 msgr2=0x7f4a040b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.727+0000 7f4a09432700 1 -- 172.21.15.92:0/2849904784 shutdown_connections 2023-10-06T10:38:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.727+0000 7f4a09432700 1 -- 172.21.15.92:0/2849904784 wait complete. 2023-10-06T10:38:12.730 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 3 2023-10-06T10:38:12.740 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:38:12.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.853+0000 7fdaae2ec700 1 Processor -- start 2023-10-06T10:38:12.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.853+0000 7fdaae2ec700 1 -- start start 2023-10-06T10:38:12.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.853+0000 7fdaae2ec700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdaa80ad5e0 0x7fdaa80cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:12.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.853+0000 7fdaae2ec700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa80cecf0 con 0x7fdaa80ad210 2023-10-06T10:38:12.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.853+0000 7fdaae2ec700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa80cefd0 con 0x7fdaa815db10 2023-10-06T10:38:12.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.853+0000 7fdaae2ec700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaa80cca90 con 0x7fdaa80ad5e0 2023-10-06T10:38:12.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.854+0000 7fdaa7fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdaa80ad210 0x7fdaa815d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37156/0 (socket says 172.21.15.92:37156) 2023-10-06T10:38:12.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.854+0000 7fdaa7fff700 1 -- 172.21.15.92:0/2447694598 learned_addr learned my addr 172.21.15.92:0/2447694598 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:12.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.854+0000 7fdaac889700 1 --2- 172.21.15.92:0/2447694598 >> v2:172.21.15.124:3300/0 conn(0x7fdaa80ad5e0 0x7fdaa80cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:12.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.854+0000 7fdaa6ffd700 1 -- 172.21.15.92:0/2447694598 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1967180355 0 0) 0x7fdaa80cecf0 con 0x7fdaa80ad210 2023-10-06T10:38:12.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.854+0000 7fdaa6ffd700 1 -- 172.21.15.92:0/2447694598 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda90003570 con 0x7fdaa80ad210 2023-10-06T10:38:12.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.854+0000 7fdaa6ffd700 1 -- 172.21.15.92:0/2447694598 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1685242466 0 0) 0x7fda90003570 con 0x7fdaa80ad210 2023-10-06T10:38:12.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.854+0000 7fdaa6ffd700 1 -- 172.21.15.92:0/2447694598 >> v2:172.21.15.124:3300/0 conn(0x7fdaa80ad5e0 msgr2=0x7fdaa80cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:12.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.854+0000 7fdaa6ffd700 1 --2- 172.21.15.92:0/2447694598 >> v2:172.21.15.124:3300/0 conn(0x7fdaa80ad5e0 0x7fdaa80cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.854+0000 7fdaa6ffd700 1 -- 172.21.15.92:0/2447694598 >> v1:172.21.15.124:6789/0 conn(0x7fdaa815db10 legacy=0x7fdaa80ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:12.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.854+0000 7fdaa6ffd700 1 -- 172.21.15.92:0/2447694598 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa80ccd70 con 0x7fdaa80ad210 2023-10-06T10:38:12.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.854+0000 7fdaa6ffd700 1 -- 172.21.15.92:0/2447694598 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fda980015f0 con 0x7fdaa80ad210 2023-10-06T10:38:12.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.854+0000 7fdaa6ffd700 1 -- 172.21.15.92:0/2447694598 >> v1:172.21.15.92:6789/0 conn(0x7fdaa80ad210 legacy=0x7fdaa815d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:12.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.854+0000 7fdaa6ffd700 1 --2- 172.21.15.92:0/2447694598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda90003a60 0x7fda90003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:12.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.854+0000 7fdaa6ffd700 1 --2- 172.21.15.92:0/2447694598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda90004810 0x7fda90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:12.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.854+0000 7fdaa6ffd700 1 --2- 172.21.15.92:0/2447694598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda900051c0 0x7fda9000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:12.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.854+0000 7fdaa6ffd700 1 -- 172.21.15.92:0/2447694598 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa80cca90 con 0x7fda90003a60 2023-10-06T10:38:12.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.854+0000 7fdaa6ffd700 1 -- 172.21.15.92:0/2447694598 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9000c0d0 con 0x7fda900051c0 2023-10-06T10:38:12.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.854+0000 7fdaa6ffd700 1 -- 172.21.15.92:0/2447694598 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9000c290 con 0x7fda90004810 2023-10-06T10:38:12.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.854+0000 7fdaa7fff700 1 --2- 172.21.15.92:0/2447694598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda90003a60 0x7fda90003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:12.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.854+0000 7fdaac889700 1 --2- 172.21.15.92:0/2447694598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda900051c0 0x7fda9000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:12.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.854+0000 7fdaa77fe700 1 --2- 172.21.15.92:0/2447694598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda90004810 0x7fda90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:12.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.854+0000 7fdaa7fff700 1 -- 172.21.15.92:0/2447694598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda90004810 msgr2=0x7fda90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:12.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.855+0000 7fdaa7fff700 1 --2- 172.21.15.92:0/2447694598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda90004810 0x7fda90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.855+0000 7fdaa7fff700 1 -- 172.21.15.92:0/2447694598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda900051c0 msgr2=0x7fda9000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:12.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.855+0000 7fdaa7fff700 1 --2- 172.21.15.92:0/2447694598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda900051c0 0x7fda9000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.855+0000 7fdaa7fff700 1 -- 172.21.15.92:0/2447694598 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda90003570 con 0x7fda90003a60 2023-10-06T10:38:12.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.855+0000 7fdaac889700 1 -- 172.21.15.92:0/2447694598 reap_dead start 2023-10-06T10:38:12.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.855+0000 7fdaa7fff700 1 --2- 172.21.15.92:0/2447694598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda90003a60 0x7fda90003e70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fda9800c3c0 tx=0x7fda98007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db58ed79693d431b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:12.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.855+0000 7fdaa6ffd700 1 -- 172.21.15.92:0/2447694598 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9800cc10 con 0x7fda90003a60 2023-10-06T10:38:12.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.855+0000 7fdaa6ffd700 1 -- 172.21.15.92:0/2447694598 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9800cd90 con 0x7fda90003a60 2023-10-06T10:38:12.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.855+0000 7fdaa6ffd700 1 -- 172.21.15.92:0/2447694598 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98007320 con 0x7fda90003a60 2023-10-06T10:38:12.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.855+0000 7fdaae2ec700 1 -- 172.21.15.92:0/2447694598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda90003a60 msgr2=0x7fda90003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:12.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.855+0000 7fdaae2ec700 1 --2- 172.21.15.92:0/2447694598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda90003a60 0x7fda90003e70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fda9800c3c0 tx=0x7fda98007920 comp rx=0 tx=0).stop 2023-10-06T10:38:12.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.855+0000 7fdaae2ec700 1 -- 172.21.15.92:0/2447694598 shutdown_connections 2023-10-06T10:38:12.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.855+0000 7fdaae2ec700 1 --2- 172.21.15.92:0/2447694598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda90003a60 0x7fda90003e70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.855+0000 7fdaae2ec700 1 -- 172.21.15.92:0/2447694598 >> 172.21.15.92:0/2447694598 conn(0x7fdaa8067d90 msgr2=0x7fdaa80ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:12.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.855+0000 7fdaae2ec700 1 -- 172.21.15.92:0/2447694598 shutdown_connections 2023-10-06T10:38:12.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.855+0000 7fdaae2ec700 1 -- 172.21.15.92:0/2447694598 wait complete. 2023-10-06T10:38:12.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.856+0000 7fdaae2ec700 1 Processor -- start 2023-10-06T10:38:12.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.857+0000 7fdaae2ec700 1 -- start start 2023-10-06T10:38:12.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.857+0000 7fdaae2ec700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda90003a60 0x7fdaa80b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:12.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.857+0000 7fdaae2ec700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaa80b03b0 0x7fdaa80b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:12.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.857+0000 7fdaae2ec700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdaa815aae0 0x7fdaa815af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:12.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.857+0000 7fdaae2ec700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa816d670 con 0x7fdaa80b03b0 2023-10-06T10:38:12.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.857+0000 7fdaae2ec700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa816d950 con 0x7fda90003a60 2023-10-06T10:38:12.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.857+0000 7fdaae2ec700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa816dc30 con 0x7fdaa815aae0 2023-10-06T10:38:12.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.857+0000 7fdaa77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaa80b03b0 0x7fdaa80b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:12.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.857+0000 7fdaa77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaa80b03b0 0x7fdaa80b0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51442/0 (socket says 172.21.15.92:51442) 2023-10-06T10:38:12.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.857+0000 7fdaa77fe700 1 -- 172.21.15.92:0/193308680 learned_addr learned my addr 172.21.15.92:0/193308680 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:12.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.857+0000 7fdaa7fff700 1 --2- 172.21.15.92:0/193308680 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda90003a60 0x7fdaa80b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:12.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.857+0000 7fdaac889700 1 --2- 172.21.15.92:0/193308680 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdaa815aae0 0x7fdaa815af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:12.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.858+0000 7fdaa77fe700 1 -- 172.21.15.92:0/193308680 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdaa815aae0 msgr2=0x7fdaa815af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:12.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.858+0000 7fdaa77fe700 1 --2- 172.21.15.92:0/193308680 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdaa815aae0 0x7fdaa815af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.858+0000 7fdaa77fe700 1 -- 172.21.15.92:0/193308680 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda90003a60 msgr2=0x7fdaa80b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:12.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.858+0000 7fdaa77fe700 1 --2- 172.21.15.92:0/193308680 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda90003a60 0x7fdaa80b33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:12.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.858+0000 7fdaa77fe700 1 -- 172.21.15.92:0/193308680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa816df10 con 0x7fdaa80b03b0 2023-10-06T10:38:12.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.858+0000 7fdaa77fe700 1 --2- 172.21.15.92:0/193308680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaa80b03b0 0x7fdaa80b0820 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fda9c002950 tx=0x7fda9c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:12.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.858+0000 7fda8ffff700 1 -- 172.21.15.92:0/193308680 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c013600 con 0x7fdaa80b03b0 2023-10-06T10:38:12.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.858+0000 7fda8ffff700 1 -- 172.21.15.92:0/193308680 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9c013760 con 0x7fdaa80b03b0 2023-10-06T10:38:12.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.858+0000 7fda8ffff700 1 -- 172.21.15.92:0/193308680 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c013b00 con 0x7fdaa80b03b0 2023-10-06T10:38:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.858+0000 7fdaae2ec700 1 -- 172.21.15.92:0/193308680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa816e1f0 con 0x7fdaa80b03b0 2023-10-06T10:38:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.858+0000 7fdaae2ec700 1 -- 172.21.15.92:0/193308680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa815b490 con 0x7fdaa80b03b0 2023-10-06T10:38:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.859+0000 7fdaae2ec700 1 -- 172.21.15.92:0/193308680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa8064a90 con 0x7fdaa80b03b0 2023-10-06T10:38:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.859+0000 7fda8ffff700 1 -- 172.21.15.92:0/193308680 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fda9c01c400 con 0x7fdaa80b03b0 2023-10-06T10:38:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.860+0000 7fda8ffff700 1 --2- 172.21.15.92:0/193308680 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda94066ea0 0x7fda94069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.860+0000 7fda8ffff700 1 -- 172.21.15.92:0/193308680 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fda9c084690 con 0x7fdaa80b03b0 2023-10-06T10:38:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.860+0000 7fdaa7fff700 1 --2- 172.21.15.92:0/193308680 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda94066ea0 0x7fda94069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.860+0000 7fdaa7fff700 1 --2- 172.21.15.92:0/193308680 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda94066ea0 0x7fda94069350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fda98000ef0 tx=0x7fda98015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.864+0000 7fda8ffff700 1 -- 172.21.15.92:0/193308680 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda9c051be0 con 0x7fdaa80b03b0 2023-10-06T10:38:12.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:12.885+0000 7fda8ffff700 1 -- 172.21.15.92:0/193308680 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fda9c051fb0 con 0x7fdaa80b03b0 2023-10-06T10:38:13.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.049+0000 7fdaae2ec700 1 -- 172.21.15.92:0/193308680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdaa80cc760 con 0x7fdaa80b03b0 2023-10-06T10:38:13.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.050+0000 7fda8ffff700 1 -- 172.21.15.92:0/193308680 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+13648 (secure 0 0 0) 0x7fda9c0553e0 con 0x7fdaa80b03b0 2023-10-06T10:38:13.051 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:13.051 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":16,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:12.312384+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:38:13.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.052+0000 7fdaae2ec700 1 -- 172.21.15.92:0/193308680 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda94066ea0 msgr2=0x7fda94069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:13.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.052+0000 7fdaae2ec700 1 --2- 172.21.15.92:0/193308680 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda94066ea0 0x7fda94069350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fda98000ef0 tx=0x7fda98015000 comp rx=0 tx=0).stop 2023-10-06T10:38:13.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.052+0000 7fdaae2ec700 1 -- 172.21.15.92:0/193308680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaa80b03b0 msgr2=0x7fdaa80b0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:13.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.052+0000 7fdaae2ec700 1 --2- 172.21.15.92:0/193308680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaa80b03b0 0x7fdaa80b0820 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fda9c002950 tx=0x7fda9c007d30 comp rx=0 tx=0).stop 2023-10-06T10:38:13.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.052+0000 7fdaae2ec700 1 -- 172.21.15.92:0/193308680 shutdown_connections 2023-10-06T10:38:13.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.052+0000 7fdaae2ec700 1 --2- 172.21.15.92:0/193308680 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda94066ea0 0x7fda94069350 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.052+0000 7fdaae2ec700 1 --2- 172.21.15.92:0/193308680 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdaa815aae0 0x7fdaa815af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.052+0000 7fdaae2ec700 1 --2- 172.21.15.92:0/193308680 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda90003a60 0x7fdaa80b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.052+0000 7fdaae2ec700 1 --2- 172.21.15.92:0/193308680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaa80b03b0 0x7fdaa80b0820 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.052+0000 7fdaae2ec700 1 -- 172.21.15.92:0/193308680 >> 172.21.15.92:0/193308680 conn(0x7fdaa8067d90 msgr2=0x7fdaa8069480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:13.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.052+0000 7fdaae2ec700 1 -- 172.21.15.92:0/193308680 shutdown_connections 2023-10-06T10:38:13.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.053+0000 7fdaae2ec700 1 -- 172.21.15.92:0/193308680 wait complete. 2023-10-06T10:38:13.065 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T10:38:09.680559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T10:38:10.120786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T10:38:13.065 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T10:38:13.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.181+0000 7fbfbc929700 1 Processor -- start 2023-10-06T10:38:13.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.181+0000 7fbfbc929700 1 -- start start 2023-10-06T10:38:13.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.182+0000 7fbfbc929700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfb81594d0 0x7fbfb80cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:13.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.182+0000 7fbfbc929700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb80cbec0 con 0x7fbfb81598a0 2023-10-06T10:38:13.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.182+0000 7fbfbc929700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb80cf5d0 con 0x7fbfb80cbaa0 2023-10-06T10:38:13.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.182+0000 7fbfbc929700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb80cf8b0 con 0x7fbfb81594d0 2023-10-06T10:38:13.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.182+0000 7fbfadd9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbfb81598a0 0x7fbfb80ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37168/0 (socket says 172.21.15.92:37168) 2023-10-06T10:38:13.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.182+0000 7fbfadd9b700 1 -- 172.21.15.92:0/1497994957 learned_addr learned my addr 172.21.15.92:0/1497994957 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:13.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.182+0000 7fbfb659c700 1 --2- 172.21.15.92:0/1497994957 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfb81594d0 0x7fbfb80cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:13.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.182+0000 7fbfb5d9b700 1 -- 172.21.15.92:0/1497994957 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3952560223 0 0) 0x7fbfb80cbec0 con 0x7fbfb81598a0 2023-10-06T10:38:13.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.182+0000 7fbfb5d9b700 1 -- 172.21.15.92:0/1497994957 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfa4003570 con 0x7fbfb81598a0 2023-10-06T10:38:13.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.183+0000 7fbfb5d9b700 1 -- 172.21.15.92:0/1497994957 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648492512 0 0) 0x7fbfb80cf5d0 con 0x7fbfb80cbaa0 2023-10-06T10:38:13.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.183+0000 7fbfb659c700 1 -- 172.21.15.92:0/1497994957 >> v1:172.21.15.124:6789/0 conn(0x7fbfb80cbaa0 legacy=0x7fbfb80c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:13.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.183+0000 7fbfb659c700 1 -- 172.21.15.92:0/1497994957 >> v1:172.21.15.92:6789/0 conn(0x7fbfb81598a0 legacy=0x7fbfb80ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:13.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.183+0000 7fbfb659c700 1 -- 172.21.15.92:0/1497994957 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb80cfb90 con 0x7fbfb81594d0 2023-10-06T10:38:13.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.183+0000 7fbfb659c700 1 --2- 172.21.15.92:0/1497994957 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfb81594d0 0x7fbfb80cb560 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009e90 tx=0x7fbfa8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6e7af1c6d96fc8b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:13.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.183+0000 7fbfb5d9b700 1 -- 172.21.15.92:0/1497994957 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa800e070 con 0x7fbfb81594d0 2023-10-06T10:38:13.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.183+0000 7fbfb5d9b700 1 -- 172.21.15.92:0/1497994957 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa800bc30 con 0x7fbfb81594d0 2023-10-06T10:38:13.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.183+0000 7fbfb5d9b700 1 -- 172.21.15.92:0/1497994957 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8012430 con 0x7fbfb81594d0 2023-10-06T10:38:13.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.183+0000 7fbfbc929700 1 -- 172.21.15.92:0/1497994957 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfb81594d0 msgr2=0x7fbfb80cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:13.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.183+0000 7fbfbc929700 1 --2- 172.21.15.92:0/1497994957 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfb81594d0 0x7fbfb80cb560 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009e90 tx=0x7fbfa8009780 comp rx=0 tx=0).stop 2023-10-06T10:38:13.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.184+0000 7fbfbc929700 1 -- 172.21.15.92:0/1497994957 shutdown_connections 2023-10-06T10:38:13.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.184+0000 7fbfbc929700 1 --2- 172.21.15.92:0/1497994957 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfb81594d0 0x7fbfb80cb560 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.184+0000 7fbfbc929700 1 -- 172.21.15.92:0/1497994957 >> 172.21.15.92:0/1497994957 conn(0x7fbfb8089e50 msgr2=0x7fbfb8099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:13.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.184+0000 7fbfbc929700 1 -- 172.21.15.92:0/1497994957 shutdown_connections 2023-10-06T10:38:13.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.184+0000 7fbfbc929700 1 -- 172.21.15.92:0/1497994957 wait complete. 2023-10-06T10:38:13.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.185+0000 7fbfbc929700 1 Processor -- start 2023-10-06T10:38:13.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.185+0000 7fbfbc929700 1 -- start start 2023-10-06T10:38:13.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.185+0000 7fbfbc929700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfb80cbaa0 0x7fbfb8155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:13.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.185+0000 7fbfbc929700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfb81594d0 0x7fbfb814e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:13.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.185+0000 7fbfbc929700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfb81598a0 0x7fbfb80c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:13.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.186+0000 7fbfbc929700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb80b4310 con 0x7fbfb80cbaa0 2023-10-06T10:38:13.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.186+0000 7fbfbc929700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb80c7a70 con 0x7fbfb81598a0 2023-10-06T10:38:13.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.186+0000 7fbfbc929700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb81582d0 con 0x7fbfb81594d0 2023-10-06T10:38:13.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.186+0000 7fbfb659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfb80cbaa0 0x7fbfb8155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:13.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.186+0000 7fbfb659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfb80cbaa0 0x7fbfb8155920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51456/0 (socket says 172.21.15.92:51456) 2023-10-06T10:38:13.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.186+0000 7fbfb659c700 1 -- 172.21.15.92:0/3935729248 learned_addr learned my addr 172.21.15.92:0/3935729248 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:13.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.186+0000 7fbfadd9b700 1 --2- 172.21.15.92:0/3935729248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfb81594d0 0x7fbfb814e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:13.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.186+0000 7fbfb6d9d700 1 --2- 172.21.15.92:0/3935729248 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfb81598a0 0x7fbfb80c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:13.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.186+0000 7fbfb659c700 1 -- 172.21.15.92:0/3935729248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfb81594d0 msgr2=0x7fbfb814e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:13.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.186+0000 7fbfb659c700 1 --2- 172.21.15.92:0/3935729248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfb81594d0 0x7fbfb814e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.186+0000 7fbfb659c700 1 -- 172.21.15.92:0/3935729248 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfb81598a0 msgr2=0x7fbfb80c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:13.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.186+0000 7fbfb659c700 1 --2- 172.21.15.92:0/3935729248 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfb81598a0 0x7fbfb80c99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.186+0000 7fbfb659c700 1 -- 172.21.15.92:0/3935729248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa8009a20 con 0x7fbfb80cbaa0 2023-10-06T10:38:13.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.186+0000 7fbfb659c700 1 --2- 172.21.15.92:0/3935729248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfb80cbaa0 0x7fbfb8155920 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800b6c0 tx=0x7fbfa8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:13.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.186+0000 7fbfaeffd700 1 -- 172.21.15.92:0/3935729248 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8026480 con 0x7fbfb80cbaa0 2023-10-06T10:38:13.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.186+0000 7fbfaeffd700 1 -- 172.21.15.92:0/3935729248 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa8026600 con 0x7fbfb80cbaa0 2023-10-06T10:38:13.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.186+0000 7fbfbc929700 1 -- 172.21.15.92:0/3935729248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb81585b0 con 0x7fbfb80cbaa0 2023-10-06T10:38:13.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.186+0000 7fbfaeffd700 1 -- 172.21.15.92:0/3935729248 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa80269c0 con 0x7fbfb80cbaa0 2023-10-06T10:38:13.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.186+0000 7fbfbc929700 1 -- 172.21.15.92:0/3935729248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb8158b70 con 0x7fbfb80cbaa0 2023-10-06T10:38:13.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.187+0000 7fbfbc929700 1 -- 172.21.15.92:0/3935729248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb8155e60 con 0x7fbfb80cbaa0 2023-10-06T10:38:13.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.187+0000 7fbfaeffd700 1 -- 172.21.15.92:0/3935729248 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbfa801c400 con 0x7fbfb80cbaa0 2023-10-06T10:38:13.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.188+0000 7fbfaeffd700 1 --2- 172.21.15.92:0/3935729248 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbf98066f30 0x7fbf980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:13.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.188+0000 7fbfaeffd700 1 -- 172.21.15.92:0/3935729248 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fbfa8086570 con 0x7fbfb80cbaa0 2023-10-06T10:38:13.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.188+0000 7fbfadd9b700 1 --2- 172.21.15.92:0/3935729248 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbf98066f30 0x7fbf980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:13.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.188+0000 7fbfadd9b700 1 --2- 172.21.15.92:0/3935729248 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbf98066f30 0x7fbf980693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbfb809e330 tx=0x7fbfa0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:13.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.192+0000 7fbfaeffd700 1 -- 172.21.15.92:0/3935729248 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfa80539d0 con 0x7fbfb80cbaa0 2023-10-06T10:38:13.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.372+0000 7fbfbc929700 1 -- 172.21.15.92:0/3935729248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbfb8064a90 con 0x7fbfb80cbaa0 2023-10-06T10:38:13.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.372+0000 7fbfaeffd700 1 -- 172.21.15.92:0/3935729248 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbfa8056ff0 con 0x7fbfb80cbaa0 2023-10-06T10:38:13.373 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T10:38:13.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.374+0000 7fbfbc929700 1 -- 172.21.15.92:0/3935729248 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbf98066f30 msgr2=0x7fbf980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:13.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.374+0000 7fbfbc929700 1 --2- 172.21.15.92:0/3935729248 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbf98066f30 0x7fbf980693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbfb809e330 tx=0x7fbfa0009040 comp rx=0 tx=0).stop 2023-10-06T10:38:13.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.374+0000 7fbfbc929700 1 -- 172.21.15.92:0/3935729248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfb80cbaa0 msgr2=0x7fbfb8155920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:13.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.374+0000 7fbfbc929700 1 --2- 172.21.15.92:0/3935729248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfb80cbaa0 0x7fbfb8155920 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800b6c0 tx=0x7fbfa8019000 comp rx=0 tx=0).stop 2023-10-06T10:38:13.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.374+0000 7fbfbc929700 1 -- 172.21.15.92:0/3935729248 shutdown_connections 2023-10-06T10:38:13.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.374+0000 7fbfbc929700 1 --2- 172.21.15.92:0/3935729248 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbf98066f30 0x7fbf980693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.374+0000 7fbfbc929700 1 --2- 172.21.15.92:0/3935729248 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfb81598a0 0x7fbfb80c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.374+0000 7fbfbc929700 1 --2- 172.21.15.92:0/3935729248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfb80cbaa0 0x7fbfb8155920 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.374+0000 7fbfbc929700 1 --2- 172.21.15.92:0/3935729248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfb81594d0 0x7fbfb814e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.375+0000 7fbfbc929700 1 -- 172.21.15.92:0/3935729248 >> 172.21.15.92:0/3935729248 conn(0x7fbfb8089e50 msgr2=0x7fbfb8063410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:13.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.375+0000 7fbfbc929700 1 -- 172.21.15.92:0/3935729248 shutdown_connections 2023-10-06T10:38:13.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.375+0000 7fbfbc929700 1 -- 172.21.15.92:0/3935729248 wait complete. 2023-10-06T10:38:13.387 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T10:38:13.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.502+0000 7f71466c4700 1 Processor -- start 2023-10-06T10:38:13.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.502+0000 7f71466c4700 1 -- start start 2023-10-06T10:38:13.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.503+0000 7f71466c4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7140156370 0x7f714006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:13.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.503+0000 7f71466c4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7140156740 0x7f714006e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:13.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.503+0000 7f71466c4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71400c84a0 con 0x7f714006e8d0 2023-10-06T10:38:13.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.503+0000 7f71466c4700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f714006ed10 con 0x7f7140156740 2023-10-06T10:38:13.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.503+0000 7f71466c4700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71400c7890 con 0x7f7140156370 2023-10-06T10:38:13.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.503+0000 7f713f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7140156740 0x7f714006e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:13.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.503+0000 7f7144c61700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f714006e8d0 0x7f714006a710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38322/0 (socket says 172.21.15.92:38322) 2023-10-06T10:38:13.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.503+0000 7f7144c61700 1 -- 172.21.15.92:0/3948388293 learned_addr learned my addr 172.21.15.92:0/3948388293 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:13.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.503+0000 7f713ffff700 1 --2- 172.21.15.92:0/3948388293 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7140156370 0x7f714006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:13.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.503+0000 7f713f7fe700 1 -- 172.21.15.92:0/3948388293 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7140156370 msgr2=0x7f714006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:13.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.503+0000 7f713f7fe700 1 --2- 172.21.15.92:0/3948388293 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7140156370 0x7f714006f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.503+0000 7f713f7fe700 1 -- 172.21.15.92:0/3948388293 >> v1:172.21.15.92:6789/0 conn(0x7f714006e8d0 legacy=0x7f714006a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:13.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.503+0000 7f713f7fe700 1 -- 172.21.15.92:0/3948388293 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71400c7b70 con 0x7f7140156740 2023-10-06T10:38:13.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.503+0000 7f713effd700 1 -- 172.21.15.92:0/3948388293 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2143044852 0 0) 0x7f71400c84a0 con 0x7f714006e8d0 2023-10-06T10:38:13.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.503+0000 7f713ffff700 1 --2- 172.21.15.92:0/3948388293 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7140156370 0x7f714006f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:13.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.504+0000 7f713f7fe700 1 --2- 172.21.15.92:0/3948388293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7140156740 0x7f714006e390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f713400bb90 tx=0x7f713400b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1892e311c4a83410 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:13.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.504+0000 7f713effd700 1 -- 172.21.15.92:0/3948388293 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713400dba0 con 0x7f7140156740 2023-10-06T10:38:13.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.504+0000 7f713effd700 1 -- 172.21.15.92:0/3948388293 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713400dd00 con 0x7f7140156740 2023-10-06T10:38:13.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.504+0000 7f713effd700 1 -- 172.21.15.92:0/3948388293 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71340196b0 con 0x7f7140156740 2023-10-06T10:38:13.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.504+0000 7f71466c4700 1 -- 172.21.15.92:0/3948388293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7140156740 msgr2=0x7f714006e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:13.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.504+0000 7f71466c4700 1 --2- 172.21.15.92:0/3948388293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7140156740 0x7f714006e390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f713400bb90 tx=0x7f713400b430 comp rx=0 tx=0).stop 2023-10-06T10:38:13.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.504+0000 7f71466c4700 1 -- 172.21.15.92:0/3948388293 shutdown_connections 2023-10-06T10:38:13.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.504+0000 7f71466c4700 1 --2- 172.21.15.92:0/3948388293 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7140156370 0x7f714006f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.504+0000 7f71466c4700 1 --2- 172.21.15.92:0/3948388293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7140156740 0x7f714006e390 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.504+0000 7f71466c4700 1 -- 172.21.15.92:0/3948388293 >> 172.21.15.92:0/3948388293 conn(0x7f7140069150 msgr2=0x7f71400cd170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:13.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.504+0000 7f71466c4700 1 -- 172.21.15.92:0/3948388293 shutdown_connections 2023-10-06T10:38:13.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.504+0000 7f71466c4700 1 -- 172.21.15.92:0/3948388293 wait complete. 2023-10-06T10:38:13.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.505+0000 7f71466c4700 1 Processor -- start 2023-10-06T10:38:13.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.506+0000 7f71466c4700 1 -- start start 2023-10-06T10:38:13.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.506+0000 7f71466c4700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f714006e8d0 0x7f71400a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:13.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.506+0000 7f71466c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7140156370 0x7f7140169690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:13.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.506+0000 7f71466c4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7140156740 0x7f71400abc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:13.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.506+0000 7f71466c4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71400aa1f0 con 0x7f7140156370 2023-10-06T10:38:13.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.506+0000 7f71466c4700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71400aad70 con 0x7f714006e8d0 2023-10-06T10:38:13.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.506+0000 7f71466c4700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71400aa440 con 0x7f7140156740 2023-10-06T10:38:13.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.506+0000 7f713f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7140156370 0x7f7140169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:13.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.506+0000 7f713f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7140156370 0x7f7140169690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59678/0 (socket says 172.21.15.92:59678) 2023-10-06T10:38:13.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.506+0000 7f713f7fe700 1 -- 172.21.15.92:0/2563402059 learned_addr learned my addr 172.21.15.92:0/2563402059 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:13.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.506+0000 7f7144c61700 1 --2- 172.21.15.92:0/2563402059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7140156740 0x7f71400abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:13.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.506+0000 7f713ffff700 1 --2- 172.21.15.92:0/2563402059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f714006e8d0 0x7f71400a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:13.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.507+0000 7f713f7fe700 1 -- 172.21.15.92:0/2563402059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7140156740 msgr2=0x7f71400abc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:13.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.507+0000 7f713f7fe700 1 --2- 172.21.15.92:0/2563402059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7140156740 0x7f71400abc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.507+0000 7f713f7fe700 1 -- 172.21.15.92:0/2563402059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f714006e8d0 msgr2=0x7f71400a98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:13.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.507+0000 7f713f7fe700 1 --2- 172.21.15.92:0/2563402059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f714006e8d0 0x7f71400a98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.507+0000 7f713f7fe700 1 -- 172.21.15.92:0/2563402059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713400b6d0 con 0x7f7140156370 2023-10-06T10:38:13.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.507+0000 7f713f7fe700 1 --2- 172.21.15.92:0/2563402059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7140156370 0x7f7140169690 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f714004c530 tx=0x7f713401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:13.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.507+0000 7f7127fff700 1 -- 172.21.15.92:0/2563402059 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7134010470 con 0x7f7140156370 2023-10-06T10:38:13.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.507+0000 7f7127fff700 1 -- 172.21.15.92:0/2563402059 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71340105d0 con 0x7f7140156370 2023-10-06T10:38:13.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.507+0000 7f71466c4700 1 -- 172.21.15.92:0/2563402059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71400aa720 con 0x7f7140156370 2023-10-06T10:38:13.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.507+0000 7f7127fff700 1 -- 172.21.15.92:0/2563402059 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7134010970 con 0x7f7140156370 2023-10-06T10:38:13.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.507+0000 7f71466c4700 1 -- 172.21.15.92:0/2563402059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71400a9df0 con 0x7f7140156370 2023-10-06T10:38:13.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.508+0000 7f71466c4700 1 -- 172.21.15.92:0/2563402059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7140064a90 con 0x7f7140156370 2023-10-06T10:38:13.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.508+0000 7f7127fff700 1 -- 172.21.15.92:0/2563402059 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7134024420 con 0x7f7140156370 2023-10-06T10:38:13.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.508+0000 7f7127fff700 1 --2- 172.21.15.92:0/2563402059 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f712c066f30 0x7f712c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:13.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.509+0000 7f7127fff700 1 -- 172.21.15.92:0/2563402059 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7134020070 con 0x7f7140156370 2023-10-06T10:38:13.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.509+0000 7f713ffff700 1 --2- 172.21.15.92:0/2563402059 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f712c066f30 0x7f712c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:13.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.509+0000 7f713ffff700 1 --2- 172.21.15.92:0/2563402059 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f712c066f30 0x7f712c0693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7140157f70 tx=0x7f7130008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:13.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.513+0000 7f7127fff700 1 -- 172.21.15.92:0/2563402059 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7134053a70 con 0x7f7140156370 2023-10-06T10:38:13.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.698+0000 7f71466c4700 1 -- 172.21.15.92:0/2563402059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f71400aaeb0 con 0x7f7140156370 2023-10-06T10:38:13.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.699+0000 7f7127fff700 1 -- 172.21.15.92:0/2563402059 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7134057090 con 0x7f7140156370 2023-10-06T10:38:13.700 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T10:38:13.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.701+0000 7f71466c4700 1 -- 172.21.15.92:0/2563402059 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f712c066f30 msgr2=0x7f712c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:13.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.701+0000 7f71466c4700 1 --2- 172.21.15.92:0/2563402059 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f712c066f30 0x7f712c0693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7140157f70 tx=0x7f7130008040 comp rx=0 tx=0).stop 2023-10-06T10:38:13.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.701+0000 7f71466c4700 1 -- 172.21.15.92:0/2563402059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7140156370 msgr2=0x7f7140169690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:13.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.701+0000 7f71466c4700 1 --2- 172.21.15.92:0/2563402059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7140156370 0x7f7140169690 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f714004c530 tx=0x7f713401b000 comp rx=0 tx=0).stop 2023-10-06T10:38:13.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.701+0000 7f71466c4700 1 -- 172.21.15.92:0/2563402059 shutdown_connections 2023-10-06T10:38:13.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.701+0000 7f71466c4700 1 --2- 172.21.15.92:0/2563402059 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f712c066f30 0x7f712c0693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.701+0000 7f71466c4700 1 --2- 172.21.15.92:0/2563402059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7140156740 0x7f71400abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.701+0000 7f71466c4700 1 --2- 172.21.15.92:0/2563402059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f714006e8d0 0x7f71400a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.701+0000 7f71466c4700 1 --2- 172.21.15.92:0/2563402059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7140156370 0x7f7140169690 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.701+0000 7f71466c4700 1 -- 172.21.15.92:0/2563402059 >> 172.21.15.92:0/2563402059 conn(0x7f7140069150 msgr2=0x7f71400b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:13.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.701+0000 7f71466c4700 1 -- 172.21.15.92:0/2563402059 shutdown_connections 2023-10-06T10:38:13.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.701+0000 7f71466c4700 1 -- 172.21.15.92:0/2563402059 wait complete. 2023-10-06T10:38:13.714 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T10:38:13.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.827+0000 7f3729719700 1 Processor -- start 2023-10-06T10:38:13.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.827+0000 7f3729719700 1 -- start start 2023-10-06T10:38:13.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.827+0000 7f3729719700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f37240b49a0 0x7f37240aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:13.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.827+0000 7f3729719700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37240b0c60 con 0x7f3724159560 2023-10-06T10:38:13.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.828+0000 7f3729719700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37240b0f40 con 0x7f37240b0760 2023-10-06T10:38:13.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.828+0000 7f3729719700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37240b1220 con 0x7f37240b49a0 2023-10-06T10:38:13.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.828+0000 7f3722ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3724159560 0x7f37240aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38334/0 (socket says 172.21.15.92:38334) 2023-10-06T10:38:13.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.828+0000 7f3722ffd700 1 -- 172.21.15.92:0/2562570020 learned_addr learned my addr 172.21.15.92:0/2562570020 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:13.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.828+0000 7f37227fc700 1 --2- 172.21.15.92:0/2562570020 >> v2:172.21.15.124:3300/0 conn(0x7f37240b49a0 0x7f37240aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:13.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.828+0000 7f3721ffb700 1 -- 172.21.15.92:0/2562570020 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2679136669 0 0) 0x7f37240b0c60 con 0x7f3724159560 2023-10-06T10:38:13.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.828+0000 7f3721ffb700 1 -- 172.21.15.92:0/2562570020 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3710003570 con 0x7f3724159560 2023-10-06T10:38:13.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.828+0000 7f3721ffb700 1 -- 172.21.15.92:0/2562570020 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3510441349 0 0) 0x7f3710003570 con 0x7f3724159560 2023-10-06T10:38:13.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.828+0000 7f37227fc700 1 -- 172.21.15.92:0/2562570020 >> v1:172.21.15.124:6789/0 conn(0x7f37240b0760 legacy=0x7f37240b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:13.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.828+0000 7f37227fc700 1 -- 172.21.15.92:0/2562570020 >> v1:172.21.15.92:6789/0 conn(0x7f3724159560 legacy=0x7f37240aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:13.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.828+0000 7f37227fc700 1 -- 172.21.15.92:0/2562570020 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37240b1500 con 0x7f37240b49a0 2023-10-06T10:38:13.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.828+0000 7f37227fc700 1 --2- 172.21.15.92:0/2562570020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37240b49a0 0x7f37240aac40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f370c009910 tx=0x7f370c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfcef55399605c58 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:13.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.828+0000 7f3721ffb700 1 -- 172.21.15.92:0/2562570020 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c001970 con 0x7f37240b49a0 2023-10-06T10:38:13.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.829+0000 7f3721ffb700 1 -- 172.21.15.92:0/2562570020 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370c011420 con 0x7f37240b49a0 2023-10-06T10:38:13.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.829+0000 7f3721ffb700 1 -- 172.21.15.92:0/2562570020 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c0117c0 con 0x7f37240b49a0 2023-10-06T10:38:13.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.829+0000 7f3729719700 1 -- 172.21.15.92:0/2562570020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37240b49a0 msgr2=0x7f37240aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:13.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.829+0000 7f3729719700 1 --2- 172.21.15.92:0/2562570020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37240b49a0 0x7f37240aac40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f370c009910 tx=0x7f370c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.829+0000 7f3729719700 1 -- 172.21.15.92:0/2562570020 shutdown_connections 2023-10-06T10:38:13.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.829+0000 7f3729719700 1 --2- 172.21.15.92:0/2562570020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37240b49a0 0x7f37240aac40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.829+0000 7f3729719700 1 -- 172.21.15.92:0/2562570020 >> 172.21.15.92:0/2562570020 conn(0x7f37240997f0 msgr2=0x7f372406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:13.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.829+0000 7f3729719700 1 -- 172.21.15.92:0/2562570020 shutdown_connections 2023-10-06T10:38:13.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.829+0000 7f3729719700 1 -- 172.21.15.92:0/2562570020 wait complete. 2023-10-06T10:38:13.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.830+0000 7f3729719700 1 Processor -- start 2023-10-06T10:38:13.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.830+0000 7f3729719700 1 -- start start 2023-10-06T10:38:13.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.831+0000 7f3729719700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37240b0760 0x7f372409a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:13.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.831+0000 7f3729719700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37240b49a0 0x7f3724157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:13.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.831+0000 7f3729719700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3724159560 0x7f372409aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:13.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.831+0000 7f3729719700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37240aedb0 con 0x7f37240b0760 2023-10-06T10:38:13.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.831+0000 7f3729719700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37240ae4f0 con 0x7f37240b49a0 2023-10-06T10:38:13.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.831+0000 7f3729719700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37240aadc0 con 0x7f3724159560 2023-10-06T10:38:13.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.831+0000 7f3722ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37240b0760 0x7f372409a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:13.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.831+0000 7f3722ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37240b0760 0x7f372409a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59686/0 (socket says 172.21.15.92:59686) 2023-10-06T10:38:13.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.831+0000 7f3722ffd700 1 -- 172.21.15.92:0/114631645 learned_addr learned my addr 172.21.15.92:0/114631645 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:13.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.831+0000 7f3722ffd700 1 -- 172.21.15.92:0/114631645 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3724159560 msgr2=0x7f372409aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:13.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.831+0000 7f3722ffd700 1 --2- 172.21.15.92:0/114631645 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3724159560 0x7f372409aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.831+0000 7f3722ffd700 1 -- 172.21.15.92:0/114631645 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37240b49a0 msgr2=0x7f3724157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:13.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.831+0000 7f3722ffd700 1 --2- 172.21.15.92:0/114631645 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37240b49a0 0x7f3724157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:13.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.831+0000 7f3722ffd700 1 -- 172.21.15.92:0/114631645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c009420 con 0x7f37240b0760 2023-10-06T10:38:13.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.832+0000 7f3722ffd700 1 --2- 172.21.15.92:0/114631645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37240b0760 0x7f372409a050 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f371c00cbe0 tx=0x7f371c002e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:13.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.832+0000 7f371affd700 1 -- 172.21.15.92:0/114631645 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c005d80 con 0x7f37240b0760 2023-10-06T10:38:13.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.832+0000 7f371affd700 1 -- 172.21.15.92:0/114631645 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371c00a6d0 con 0x7f37240b0760 2023-10-06T10:38:13.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.832+0000 7f3729719700 1 -- 172.21.15.92:0/114631645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37240ae760 con 0x7f37240b0760 2023-10-06T10:38:13.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.832+0000 7f371affd700 1 -- 172.21.15.92:0/114631645 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c00aa70 con 0x7f37240b0760 2023-10-06T10:38:13.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.832+0000 7f3729719700 1 -- 172.21.15.92:0/114631645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37240ad770 con 0x7f37240b0760 2023-10-06T10:38:13.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.833+0000 7f3729719700 1 -- 172.21.15.92:0/114631645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3724064a90 con 0x7f37240b0760 2023-10-06T10:38:13.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.833+0000 7f371affd700 1 -- 172.21.15.92:0/114631645 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f371c0070f0 con 0x7f37240b0760 2023-10-06T10:38:13.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.833+0000 7f371affd700 1 --2- 172.21.15.92:0/114631645 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3704066f30 0x7f37040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:13.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.833+0000 7f371affd700 1 -- 172.21.15.92:0/114631645 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f371c084390 con 0x7f37240b0760 2023-10-06T10:38:13.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.833+0000 7f37227fc700 1 --2- 172.21.15.92:0/114631645 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3704066f30 0x7f37040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:13.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.834+0000 7f37227fc700 1 --2- 172.21.15.92:0/114631645 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3704066f30 0x7f37040693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f370c009910 tx=0x7f370c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:13.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:13.838+0000 7f371affd700 1 -- 172.21.15.92:0/114631645 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f371c051870 con 0x7f37240b0760 2023-10-06T10:38:14.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.022+0000 7f3729719700 1 -- 172.21.15.92:0/114631645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f37240b17e0 con 0x7f37240b0760 2023-10-06T10:38:14.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.023+0000 7f371affd700 1 -- 172.21.15.92:0/114631645 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+126 (secure 0 0 0) 0x7f371c00ae60 con 0x7f37240b0760 2023-10-06T10:38:14.024 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:14.024 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T10:38:14.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:38:14.024 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-10-06T10:38:14.024 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-10-06T10:38:14.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:38:14.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:38:14.025 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-10-06T10:38:14.025 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-10-06T10:38:14.025 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:38:14.025 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T10:38:14.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.024+0000 7f3729719700 1 -- 172.21.15.92:0/114631645 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3704066f30 msgr2=0x7f37040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:14.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.024+0000 7f3729719700 1 --2- 172.21.15.92:0/114631645 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3704066f30 0x7f37040693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f370c009910 tx=0x7f370c016000 comp rx=0 tx=0).stop 2023-10-06T10:38:14.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.025+0000 7f3729719700 1 -- 172.21.15.92:0/114631645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37240b0760 msgr2=0x7f372409a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:14.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.025+0000 7f3729719700 1 --2- 172.21.15.92:0/114631645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37240b0760 0x7f372409a050 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f371c00cbe0 tx=0x7f371c002e80 comp rx=0 tx=0).stop 2023-10-06T10:38:14.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.025+0000 7f3729719700 1 -- 172.21.15.92:0/114631645 shutdown_connections 2023-10-06T10:38:14.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.025+0000 7f3729719700 1 --2- 172.21.15.92:0/114631645 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3704066f30 0x7f37040693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:14.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.025+0000 7f3729719700 1 --2- 172.21.15.92:0/114631645 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3724159560 0x7f372409aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:14.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.025+0000 7f3729719700 1 --2- 172.21.15.92:0/114631645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37240b0760 0x7f372409a050 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:14.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.025+0000 7f3729719700 1 --2- 172.21.15.92:0/114631645 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37240b49a0 0x7f3724157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:14.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.025+0000 7f3729719700 1 -- 172.21.15.92:0/114631645 >> 172.21.15.92:0/114631645 conn(0x7f37240997f0 msgr2=0x7f37240cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:14.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.025+0000 7f3729719700 1 -- 172.21.15.92:0/114631645 shutdown_connections 2023-10-06T10:38:14.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.025+0000 7f3729719700 1 -- 172.21.15.92:0/114631645 wait complete. 2023-10-06T10:38:14.037 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T10:38:14.037 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T10:38:14.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.156+0000 7fdff70d9700 1 Processor -- start 2023-10-06T10:38:14.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.156+0000 7fdff70d9700 1 -- start start 2023-10-06T10:38:14.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.157+0000 7fdff70d9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdff00c32e0 0x7fdff00cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:14.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.157+0000 7fdff70d9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff00c8eb0 con 0x7fdff00c36b0 2023-10-06T10:38:14.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.157+0000 7fdff70d9700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff00cf230 con 0x7fdff00c3ae0 2023-10-06T10:38:14.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.157+0000 7fdff70d9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff00c47e0 con 0x7fdff00c32e0 2023-10-06T10:38:14.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.157+0000 7fdff4e75700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdff00c32e0 0x7fdff00cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:14.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.157+0000 7fdfe7fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdff00c36b0 0x7fdff015d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38336/0 (socket says 172.21.15.92:38336) 2023-10-06T10:38:14.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.157+0000 7fdfe7fff700 1 -- 172.21.15.92:0/1077117137 learned_addr learned my addr 172.21.15.92:0/1077117137 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:14.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.157+0000 7fdff4e75700 1 -- 172.21.15.92:0/1077117137 >> v1:172.21.15.124:6789/0 conn(0x7fdff00c3ae0 legacy=0x7fdff015dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:14.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.157+0000 7fdff4e75700 1 -- 172.21.15.92:0/1077117137 >> v1:172.21.15.92:6789/0 conn(0x7fdff00c36b0 legacy=0x7fdff015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:14.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.157+0000 7fdff4e75700 1 -- 172.21.15.92:0/1077117137 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff015e380 con 0x7fdff00c32e0 2023-10-06T10:38:14.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.157+0000 7fdfe77fe700 1 -- 172.21.15.92:0/1077117137 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2166806653 0 0) 0x7fdff00c8eb0 con 0x7fdff00c36b0 2023-10-06T10:38:14.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.158+0000 7fdff4e75700 1 --2- 172.21.15.92:0/1077117137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdff00c32e0 0x7fdff00cecf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009eb0 tx=0x7fdfec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35785eaf0d3fa5bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:14.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.158+0000 7fdfe77fe700 1 -- 172.21.15.92:0/1077117137 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec00e070 con 0x7fdff00c32e0 2023-10-06T10:38:14.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.158+0000 7fdfe77fe700 1 -- 172.21.15.92:0/1077117137 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfec00bcc0 con 0x7fdff00c32e0 2023-10-06T10:38:14.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.158+0000 7fdfe77fe700 1 -- 172.21.15.92:0/1077117137 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec012680 con 0x7fdff00c32e0 2023-10-06T10:38:14.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.158+0000 7fdff70d9700 1 -- 172.21.15.92:0/1077117137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdff00c32e0 msgr2=0x7fdff00cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:14.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.158+0000 7fdff70d9700 1 --2- 172.21.15.92:0/1077117137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdff00c32e0 0x7fdff00cecf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009eb0 tx=0x7fdfec009780 comp rx=0 tx=0).stop 2023-10-06T10:38:14.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.158+0000 7fdff70d9700 1 -- 172.21.15.92:0/1077117137 shutdown_connections 2023-10-06T10:38:14.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.158+0000 7fdff70d9700 1 --2- 172.21.15.92:0/1077117137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdff00c32e0 0x7fdff00cecf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:14.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.158+0000 7fdff70d9700 1 -- 172.21.15.92:0/1077117137 >> 172.21.15.92:0/1077117137 conn(0x7fdff0087fc0 msgr2=0x7fdff00ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:14.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.158+0000 7fdff70d9700 1 -- 172.21.15.92:0/1077117137 shutdown_connections 2023-10-06T10:38:14.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.158+0000 7fdff70d9700 1 -- 172.21.15.92:0/1077117137 wait complete. 2023-10-06T10:38:14.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.159+0000 7fdff70d9700 1 Processor -- start 2023-10-06T10:38:14.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.159+0000 7fdff70d9700 1 -- start start 2023-10-06T10:38:14.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.160+0000 7fdff70d9700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdff00c32e0 0x7fdff01582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:14.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.160+0000 7fdff70d9700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdff00c36b0 0x7fdff0069540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:14.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.160+0000 7fdff70d9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdff00c3ae0 0x7fdff0069a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:14.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.160+0000 7fdff70d9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff00c6050 con 0x7fdff00c3ae0 2023-10-06T10:38:14.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.160+0000 7fdff70d9700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff00c5440 con 0x7fdff00c36b0 2023-10-06T10:38:14.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.160+0000 7fdff70d9700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff00c5720 con 0x7fdff00c32e0 2023-10-06T10:38:14.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.160+0000 7fdff5676700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdff00c3ae0 0x7fdff0069a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:14.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.160+0000 7fdff5676700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdff00c3ae0 0x7fdff0069a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59698/0 (socket says 172.21.15.92:59698) 2023-10-06T10:38:14.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.160+0000 7fdff5676700 1 -- 172.21.15.92:0/260108863 learned_addr learned my addr 172.21.15.92:0/260108863 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:14.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.160+0000 7fdfe7fff700 1 --2- 172.21.15.92:0/260108863 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdff00c36b0 0x7fdff0069540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:14.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.161+0000 7fdff4e75700 1 --2- 172.21.15.92:0/260108863 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdff00c32e0 0x7fdff01582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:14.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.161+0000 7fdff5676700 1 -- 172.21.15.92:0/260108863 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdff00c32e0 msgr2=0x7fdff01582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:14.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.161+0000 7fdff5676700 1 --2- 172.21.15.92:0/260108863 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdff00c32e0 0x7fdff01582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:14.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.161+0000 7fdff5676700 1 -- 172.21.15.92:0/260108863 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdff00c36b0 msgr2=0x7fdff0069540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:14.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.161+0000 7fdff5676700 1 --2- 172.21.15.92:0/260108863 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdff00c36b0 0x7fdff0069540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:14.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.161+0000 7fdff5676700 1 -- 172.21.15.92:0/260108863 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec009a20 con 0x7fdff00c3ae0 2023-10-06T10:38:14.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.161+0000 7fdfe7fff700 1 --2- 172.21.15.92:0/260108863 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdff00c36b0 0x7fdff0069540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:14.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.161+0000 7fdff4e75700 1 --2- 172.21.15.92:0/260108863 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdff00c32e0 0x7fdff01582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:14.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.161+0000 7fdff5676700 1 --2- 172.21.15.92:0/260108863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdff00c3ae0 0x7fdff0069a80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800b4e0 tx=0x7fdfe8009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:14.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.161+0000 7fdfe4ff9700 1 -- 172.21.15.92:0/260108863 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe800f5e0 con 0x7fdff00c3ae0 2023-10-06T10:38:14.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.161+0000 7fdfe4ff9700 1 -- 172.21.15.92:0/260108863 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe800f760 con 0x7fdff00c3ae0 2023-10-06T10:38:14.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.161+0000 7fdff70d9700 1 -- 172.21.15.92:0/260108863 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff00c5a00 con 0x7fdff00c3ae0 2023-10-06T10:38:14.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.161+0000 7fdfe4ff9700 1 -- 172.21.15.92:0/260108863 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe800fb20 con 0x7fdff00c3ae0 2023-10-06T10:38:14.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.161+0000 7fdff70d9700 1 -- 172.21.15.92:0/260108863 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff0158820 con 0x7fdff00c3ae0 2023-10-06T10:38:14.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.162+0000 7fdff70d9700 1 -- 172.21.15.92:0/260108863 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff00c61c0 con 0x7fdff00c3ae0 2023-10-06T10:38:14.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.162+0000 7fdfe4ff9700 1 -- 172.21.15.92:0/260108863 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdfe801a400 con 0x7fdff00c3ae0 2023-10-06T10:38:14.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.163+0000 7fdfe4ff9700 1 --2- 172.21.15.92:0/260108863 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdfdc066f30 0x7fdfdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:14.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.163+0000 7fdff4e75700 1 --2- 172.21.15.92:0/260108863 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdfdc066f30 0x7fdfdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:14.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.163+0000 7fdfe4ff9700 1 -- 172.21.15.92:0/260108863 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fdfe8083eb0 con 0x7fdff00c3ae0 2023-10-06T10:38:14.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.163+0000 7fdff4e75700 1 --2- 172.21.15.92:0/260108863 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdfdc066f30 0x7fdfdc0693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0099a0 tx=0x7fdfec016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:14.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.167+0000 7fdfe4ff9700 1 -- 172.21.15.92:0/260108863 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdfe8051390 con 0x7fdff00c3ae0 2023-10-06T10:38:14.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:14.351+0000 7fdff70d9700 1 -- 172.21.15.92:0/260108863 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fdff0064a90 con 0x7fdff00c3ae0 2023-10-06T10:38:15.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.337+0000 7fdfe4ff9700 1 -- 172.21.15.92:0/260108863 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdfe800fed0 con 0x7fdff00c3ae0 2023-10-06T10:38:15.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.340+0000 7fdff70d9700 1 -- 172.21.15.92:0/260108863 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdfdc066f30 msgr2=0x7fdfdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:15.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.340+0000 7fdff70d9700 1 --2- 172.21.15.92:0/260108863 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdfdc066f30 0x7fdfdc0693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0099a0 tx=0x7fdfec016000 comp rx=0 tx=0).stop 2023-10-06T10:38:15.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.340+0000 7fdff70d9700 1 -- 172.21.15.92:0/260108863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdff00c3ae0 msgr2=0x7fdff0069a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:15.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.340+0000 7fdff70d9700 1 --2- 172.21.15.92:0/260108863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdff00c3ae0 0x7fdff0069a80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800b4e0 tx=0x7fdfe8009060 comp rx=0 tx=0).stop 2023-10-06T10:38:15.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.340+0000 7fdff70d9700 1 -- 172.21.15.92:0/260108863 shutdown_connections 2023-10-06T10:38:15.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.340+0000 7fdff70d9700 1 --2- 172.21.15.92:0/260108863 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdfdc066f30 0x7fdfdc0693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:15.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.340+0000 7fdff70d9700 1 --2- 172.21.15.92:0/260108863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdff00c3ae0 0x7fdff0069a80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:15.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.340+0000 7fdff70d9700 1 --2- 172.21.15.92:0/260108863 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdff00c32e0 0x7fdff01582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:15.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.340+0000 7fdff70d9700 1 --2- 172.21.15.92:0/260108863 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdff00c36b0 0x7fdff0069540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:15.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.340+0000 7fdff70d9700 1 -- 172.21.15.92:0/260108863 >> 172.21.15.92:0/260108863 conn(0x7fdff0087fc0 msgr2=0x7fdff015dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:15.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.340+0000 7fdff70d9700 1 -- 172.21.15.92:0/260108863 shutdown_connections 2023-10-06T10:38:15.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.340+0000 7fdff70d9700 1 -- 172.21.15.92:0/260108863 wait complete. 2023-10-06T10:38:15.343 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-10-06T10:38:15.358 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-10-06T10:38:15.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.480+0000 7f31f63f8700 1 Processor -- start 2023-10-06T10:38:15.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.480+0000 7f31f63f8700 1 -- start start 2023-10-06T10:38:15.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.481+0000 7f31f63f8700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f31f015dcf0 0x7f31f015a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:15.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.481+0000 7f31f63f8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f015d860 con 0x7f31f00cb480 2023-10-06T10:38:15.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.481+0000 7f31f63f8700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f015a7f0 con 0x7f31f0067bd0 2023-10-06T10:38:15.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.481+0000 7f31f63f8700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f015aad0 con 0x7f31f015dcf0 2023-10-06T10:38:15.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.481+0000 7f31ef7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f31f00cb480 0x7f31f0159ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38340/0 (socket says 172.21.15.92:38340) 2023-10-06T10:38:15.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.481+0000 7f31ef7fe700 1 -- 172.21.15.92:0/1872232789 learned_addr learned my addr 172.21.15.92:0/1872232789 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:15.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.481+0000 7f31f4995700 1 --2- 172.21.15.92:0/1872232789 >> v2:172.21.15.124:3300/0 conn(0x7f31f015dcf0 0x7f31f015a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:15.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.481+0000 7f31eeffd700 1 -- 172.21.15.92:0/1872232789 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409796607 0 0) 0x7f31f015d860 con 0x7f31f00cb480 2023-10-06T10:38:15.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.481+0000 7f31eeffd700 1 -- 172.21.15.92:0/1872232789 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31d8003570 con 0x7f31f00cb480 2023-10-06T10:38:15.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.481+0000 7f31eeffd700 1 -- 172.21.15.92:0/1872232789 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3564272838 0 0) 0x7f31d8003570 con 0x7f31f00cb480 2023-10-06T10:38:15.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.481+0000 7f31eeffd700 1 -- 172.21.15.92:0/1872232789 >> v2:172.21.15.124:3300/0 conn(0x7f31f015dcf0 msgr2=0x7f31f015a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:15.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.481+0000 7f31eeffd700 1 --2- 172.21.15.92:0/1872232789 >> v2:172.21.15.124:3300/0 conn(0x7f31f015dcf0 0x7f31f015a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:15.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.481+0000 7f31eeffd700 1 -- 172.21.15.92:0/1872232789 >> v1:172.21.15.124:6789/0 conn(0x7f31f0067bd0 legacy=0x7f31f00cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:15.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.481+0000 7f31eeffd700 1 -- 172.21.15.92:0/1872232789 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f015d210 con 0x7f31f00cb480 2023-10-06T10:38:15.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.481+0000 7f31eeffd700 1 -- 172.21.15.92:0/1872232789 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f31e4001470 con 0x7f31f00cb480 2023-10-06T10:38:15.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.481+0000 7f31eeffd700 1 -- 172.21.15.92:0/1872232789 >> v1:172.21.15.92:6789/0 conn(0x7f31f00cb480 legacy=0x7f31f0159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:15.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.481+0000 7f31eeffd700 1 --2- 172.21.15.92:0/1872232789 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31d8003a60 0x7f31d8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:15.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.481+0000 7f31eeffd700 1 --2- 172.21.15.92:0/1872232789 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31d8004810 0x7f31d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:15.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.481+0000 7f31eeffd700 1 --2- 172.21.15.92:0/1872232789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d80051c0 0x7f31d800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:15.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.481+0000 7f31eeffd700 1 -- 172.21.15.92:0/1872232789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f015aad0 con 0x7f31d80051c0 2023-10-06T10:38:15.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.481+0000 7f31eeffd700 1 -- 172.21.15.92:0/1872232789 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d800c160 con 0x7f31d8003a60 2023-10-06T10:38:15.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.482+0000 7f31eeffd700 1 -- 172.21.15.92:0/1872232789 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d800c320 con 0x7f31d8004810 2023-10-06T10:38:15.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.482+0000 7f31f4995700 1 --2- 172.21.15.92:0/1872232789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d80051c0 0x7f31d800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:15.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.482+0000 7f31effff700 1 --2- 172.21.15.92:0/1872232789 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31d8003a60 0x7f31d8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:15.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.482+0000 7f31ef7fe700 1 --2- 172.21.15.92:0/1872232789 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31d8004810 0x7f31d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:15.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.482+0000 7f31f4995700 1 -- 172.21.15.92:0/1872232789 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31d8004810 msgr2=0x7f31d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:15.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.482+0000 7f31f4995700 1 --2- 172.21.15.92:0/1872232789 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31d8004810 0x7f31d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:15.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.482+0000 7f31f4995700 1 -- 172.21.15.92:0/1872232789 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31d8003a60 msgr2=0x7f31d8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:15.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.482+0000 7f31f4995700 1 --2- 172.21.15.92:0/1872232789 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31d8003a60 0x7f31d8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:15.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.482+0000 7f31f4995700 1 -- 172.21.15.92:0/1872232789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d8003570 con 0x7f31d80051c0 2023-10-06T10:38:15.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.482+0000 7f31ef7fe700 1 --2- 172.21.15.92:0/1872232789 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31d8004810 0x7f31d8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:15.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.482+0000 7f31f4995700 1 -- 172.21.15.92:0/1872232789 reap_dead start 2023-10-06T10:38:15.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.482+0000 7f31f4995700 1 --2- 172.21.15.92:0/1872232789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d80051c0 0x7f31d800ba60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f31e800e210 tx=0x7f31e800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe79d38d66a6b69 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:15.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.482+0000 7f31eeffd700 1 -- 172.21.15.92:0/1872232789 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e800eac0 con 0x7f31d80051c0 2023-10-06T10:38:15.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.482+0000 7f31eeffd700 1 -- 172.21.15.92:0/1872232789 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e800f040 con 0x7f31d80051c0 2023-10-06T10:38:15.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.482+0000 7f31eeffd700 1 -- 172.21.15.92:0/1872232789 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e800ee70 con 0x7f31d80051c0 2023-10-06T10:38:15.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.483+0000 7f31f63f8700 1 -- 172.21.15.92:0/1872232789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d80051c0 msgr2=0x7f31d800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:15.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.483+0000 7f31f63f8700 1 --2- 172.21.15.92:0/1872232789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d80051c0 0x7f31d800ba60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f31e800e210 tx=0x7f31e800b040 comp rx=0 tx=0).stop 2023-10-06T10:38:15.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.483+0000 7f31f63f8700 1 -- 172.21.15.92:0/1872232789 shutdown_connections 2023-10-06T10:38:15.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.483+0000 7f31f63f8700 1 --2- 172.21.15.92:0/1872232789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d80051c0 0x7f31d800ba60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:15.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.483+0000 7f31f63f8700 1 -- 172.21.15.92:0/1872232789 >> 172.21.15.92:0/1872232789 conn(0x7f31f00cd1a0 msgr2=0x7f31f00cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:15.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.483+0000 7f31f63f8700 1 -- 172.21.15.92:0/1872232789 shutdown_connections 2023-10-06T10:38:15.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.483+0000 7f31f63f8700 1 -- 172.21.15.92:0/1872232789 wait complete. 2023-10-06T10:38:15.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.484+0000 7f31f63f8700 1 Processor -- start 2023-10-06T10:38:15.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.484+0000 7f31f63f8700 1 -- start start 2023-10-06T10:38:15.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31f63f8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d80051c0 0x7f31f016a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:15.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31f63f8700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31f0150560 0x7f31f016a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:15.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31f63f8700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31f00b2760 0x7f31f016aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:15.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31f63f8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f01586d0 con 0x7f31d80051c0 2023-10-06T10:38:15.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31f63f8700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f014b570 con 0x7f31f00b2760 2023-10-06T10:38:15.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31f63f8700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f015a360 con 0x7f31f0150560 2023-10-06T10:38:15.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31effff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d80051c0 0x7f31f016a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:15.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31effff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d80051c0 0x7f31f016a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59722/0 (socket says 172.21.15.92:59722) 2023-10-06T10:38:15.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31effff700 1 -- 172.21.15.92:0/4139427443 learned_addr learned my addr 172.21.15.92:0/4139427443 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:15.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31ef7fe700 1 --2- 172.21.15.92:0/4139427443 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31f0150560 0x7f31f016a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:15.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31f4995700 1 --2- 172.21.15.92:0/4139427443 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31f00b2760 0x7f31f016aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:15.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31effff700 1 -- 172.21.15.92:0/4139427443 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31f0150560 msgr2=0x7f31f016a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:15.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31effff700 1 --2- 172.21.15.92:0/4139427443 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31f0150560 0x7f31f016a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:15.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31effff700 1 -- 172.21.15.92:0/4139427443 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31f00b2760 msgr2=0x7f31f016aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:15.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31effff700 1 --2- 172.21.15.92:0/4139427443 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31f00b2760 0x7f31f016aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:15.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31effff700 1 -- 172.21.15.92:0/4139427443 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f00b2d20 con 0x7f31d80051c0 2023-10-06T10:38:15.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31ef7fe700 1 --2- 172.21.15.92:0/4139427443 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31f0150560 0x7f31f016a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:15.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31f4995700 1 --2- 172.21.15.92:0/4139427443 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31f00b2760 0x7f31f016aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:15.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31effff700 1 --2- 172.21.15.92:0/4139427443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d80051c0 0x7f31f016a000 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f31e0002950 tx=0x7f31e0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:15.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31d7fff700 1 -- 172.21.15.92:0/4139427443 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e0002e90 con 0x7f31d80051c0 2023-10-06T10:38:15.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31d7fff700 1 -- 172.21.15.92:0/4139427443 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e000f8d0 con 0x7f31d80051c0 2023-10-06T10:38:15.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31d7fff700 1 -- 172.21.15.92:0/4139427443 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e000fc70 con 0x7f31d80051c0 2023-10-06T10:38:15.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31f63f8700 1 -- 172.21.15.92:0/4139427443 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f0159a30 con 0x7f31d80051c0 2023-10-06T10:38:15.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.485+0000 7f31f63f8700 1 -- 172.21.15.92:0/4139427443 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f0159ff0 con 0x7f31d80051c0 2023-10-06T10:38:15.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.486+0000 7f31f63f8700 1 -- 172.21.15.92:0/4139427443 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f0064a90 con 0x7f31d80051c0 2023-10-06T10:38:15.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.487+0000 7f31d7fff700 1 -- 172.21.15.92:0/4139427443 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f31e001c400 con 0x7f31d80051c0 2023-10-06T10:38:15.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.487+0000 7f31d7fff700 1 --2- 172.21.15.92:0/4139427443 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f31dc066f30 0x7f31dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:15.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.487+0000 7f31ef7fe700 1 --2- 172.21.15.92:0/4139427443 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f31dc066f30 0x7f31dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:15.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.487+0000 7f31d7fff700 1 -- 172.21.15.92:0/4139427443 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f31e0084980 con 0x7f31d80051c0 2023-10-06T10:38:15.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.487+0000 7f31ef7fe700 1 --2- 172.21.15.92:0/4139427443 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f31dc066f30 0x7f31dc0693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f31f006efe0 tx=0x7f31e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:15.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.492+0000 7f31d7fff700 1 -- 172.21.15.92:0/4139427443 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31e0051d20 con 0x7f31d80051c0 2023-10-06T10:38:15.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:15.674+0000 7f31f63f8700 1 -- 172.21.15.92:0/4139427443 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f31f004bb40 con 0x7f31d80051c0 2023-10-06T10:38:16.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.343+0000 7f31d7fff700 1 -- 172.21.15.92:0/4139427443 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7f31e0055340 con 0x7f31d80051c0 2023-10-06T10:38:16.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.348+0000 7f31f63f8700 1 -- 172.21.15.92:0/4139427443 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f31dc066f30 msgr2=0x7f31dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:16.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.348+0000 7f31f63f8700 1 --2- 172.21.15.92:0/4139427443 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f31dc066f30 0x7f31dc0693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f31f006efe0 tx=0x7f31e4009040 comp rx=0 tx=0).stop 2023-10-06T10:38:16.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.348+0000 7f31f63f8700 1 -- 172.21.15.92:0/4139427443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d80051c0 msgr2=0x7f31f016a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:16.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.348+0000 7f31f63f8700 1 --2- 172.21.15.92:0/4139427443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d80051c0 0x7f31f016a000 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f31e0002950 tx=0x7f31e0007d30 comp rx=0 tx=0).stop 2023-10-06T10:38:16.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.349+0000 7f31f63f8700 1 -- 172.21.15.92:0/4139427443 shutdown_connections 2023-10-06T10:38:16.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.349+0000 7f31f63f8700 1 --2- 172.21.15.92:0/4139427443 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f31dc066f30 0x7f31dc0693e0 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f31f006efe0 tx=0x7f31e4009040 comp rx=0 tx=0).stop 2023-10-06T10:38:16.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.349+0000 7f31f63f8700 1 --2- 172.21.15.92:0/4139427443 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31f00b2760 0x7f31f016aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:16.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.349+0000 7f31f63f8700 1 --2- 172.21.15.92:0/4139427443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d80051c0 0x7f31f016a000 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:16.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.349+0000 7f31f63f8700 1 --2- 172.21.15.92:0/4139427443 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31f0150560 0x7f31f016a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:16.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.349+0000 7f31f63f8700 1 -- 172.21.15.92:0/4139427443 >> 172.21.15.92:0/4139427443 conn(0x7f31f00cd1a0 msgr2=0x7f31f00cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:16.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.351+0000 7f31f63f8700 1 -- 172.21.15.92:0/4139427443 shutdown_connections 2023-10-06T10:38:16.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.351+0000 7f31f63f8700 1 -- 172.21.15.92:0/4139427443 wait complete. 2023-10-06T10:38:16.357 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-10-06T10:38:16.372 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T10:38:16.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.488+0000 7fe38e5ad700 1 Processor -- start 2023-10-06T10:38:16.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.488+0000 7fe38e5ad700 1 -- start start 2023-10-06T10:38:16.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.489+0000 7fe38e5ad700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe38806ce80 0x7fe38815b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:16.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.489+0000 7fe38e5ad700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe3880c88e0 0x7fe3880cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:16.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.489+0000 7fe38e5ad700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe38806d250 0x7fe3880cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:16.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.489+0000 7fe38e5ad700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe38815be00 con 0x7fe3880c88e0 2023-10-06T10:38:16.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.489+0000 7fe38e5ad700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3880cb430 con 0x7fe38806d250 2023-10-06T10:38:16.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.489+0000 7fe38e5ad700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3880ca5c0 con 0x7fe38806ce80 2023-10-06T10:38:16.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.489+0000 7fe3877fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe3880c88e0 0x7fe3880cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:16.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.489+0000 7fe3877fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe3880c88e0 0x7fe3880cbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59726/0 (socket says 172.21.15.92:59726) 2023-10-06T10:38:16.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.489+0000 7fe3877fe700 1 -- 172.21.15.92:0/2082461356 learned_addr learned my addr 172.21.15.92:0/2082461356 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:16.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.489+0000 7fe387fff700 1 --2- 172.21.15.92:0/2082461356 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe38806ce80 0x7fe38815b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:16.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.489+0000 7fe38cb4a700 1 --2- 172.21.15.92:0/2082461356 >> v2:172.21.15.124:3300/0 conn(0x7fe38806d250 0x7fe3880cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:16.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.490+0000 7fe3877fe700 1 -- 172.21.15.92:0/2082461356 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe38806ce80 msgr2=0x7fe38815b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:16.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.490+0000 7fe3877fe700 1 --2- 172.21.15.92:0/2082461356 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe38806ce80 0x7fe38815b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:16.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.490+0000 7fe3877fe700 1 -- 172.21.15.92:0/2082461356 >> v2:172.21.15.124:3300/0 conn(0x7fe38806d250 msgr2=0x7fe3880cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:16.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.490+0000 7fe3877fe700 1 --2- 172.21.15.92:0/2082461356 >> v2:172.21.15.124:3300/0 conn(0x7fe38806d250 0x7fe3880cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:16.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.490+0000 7fe3877fe700 1 -- 172.21.15.92:0/2082461356 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3880ca820 con 0x7fe3880c88e0 2023-10-06T10:38:16.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.490+0000 7fe387fff700 1 --2- 172.21.15.92:0/2082461356 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe38806ce80 0x7fe38815b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:16.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.490+0000 7fe3877fe700 1 --2- 172.21.15.92:0/2082461356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3880c88e0 0x7fe3880cbe70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe38806dc10 tx=0x7fe37c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987385e0aee93df6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:16.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.490+0000 7fe386ffd700 1 -- 172.21.15.92:0/2082461356 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c009e70 con 0x7fe3880c88e0 2023-10-06T10:38:16.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.490+0000 7fe386ffd700 1 -- 172.21.15.92:0/2082461356 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37c00b860 con 0x7fe3880c88e0 2023-10-06T10:38:16.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.490+0000 7fe386ffd700 1 -- 172.21.15.92:0/2082461356 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c00bc00 con 0x7fe3880c88e0 2023-10-06T10:38:16.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.490+0000 7fe38e5ad700 1 -- 172.21.15.92:0/2082461356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3880c88e0 msgr2=0x7fe3880cbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:16.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.490+0000 7fe38e5ad700 1 --2- 172.21.15.92:0/2082461356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3880c88e0 0x7fe3880cbe70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe38806dc10 tx=0x7fe37c009250 comp rx=0 tx=0).stop 2023-10-06T10:38:16.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.490+0000 7fe38e5ad700 1 -- 172.21.15.92:0/2082461356 shutdown_connections 2023-10-06T10:38:16.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.490+0000 7fe38e5ad700 1 --2- 172.21.15.92:0/2082461356 >> v2:172.21.15.124:3300/0 conn(0x7fe38806d250 0x7fe3880cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:16.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.490+0000 7fe38e5ad700 1 --2- 172.21.15.92:0/2082461356 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe38806ce80 0x7fe38815b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:16.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.490+0000 7fe38e5ad700 1 --2- 172.21.15.92:0/2082461356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3880c88e0 0x7fe3880cbe70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:16.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.490+0000 7fe38e5ad700 1 -- 172.21.15.92:0/2082461356 >> 172.21.15.92:0/2082461356 conn(0x7fe38806b610 msgr2=0x7fe38806ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:16.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.491+0000 7fe38e5ad700 1 -- 172.21.15.92:0/2082461356 shutdown_connections 2023-10-06T10:38:16.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.491+0000 7fe38e5ad700 1 -- 172.21.15.92:0/2082461356 wait complete. 2023-10-06T10:38:16.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.492+0000 7fe38e5ad700 1 Processor -- start 2023-10-06T10:38:16.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.492+0000 7fe38e5ad700 1 -- start start 2023-10-06T10:38:16.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.492+0000 7fe38e5ad700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe38806ce80 0x7fe388141c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:16.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.492+0000 7fe38e5ad700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe38806d250 0x7fe388156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:16.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.492+0000 7fe38e5ad700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3880c88e0 0x7fe388157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:16.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.492+0000 7fe38e5ad700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe388158d30 con 0x7fe38806d250 2023-10-06T10:38:16.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.492+0000 7fe38e5ad700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3880afc20 con 0x7fe38806ce80 2023-10-06T10:38:16.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.492+0000 7fe38e5ad700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3880aff00 con 0x7fe3880c88e0 2023-10-06T10:38:16.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.493+0000 7fe3877fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe38806d250 0x7fe388156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:16.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.493+0000 7fe3877fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe38806d250 0x7fe388156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59730/0 (socket says 172.21.15.92:59730) 2023-10-06T10:38:16.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.493+0000 7fe3877fe700 1 -- 172.21.15.92:0/2846390486 learned_addr learned my addr 172.21.15.92:0/2846390486 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:16.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.493+0000 7fe387fff700 1 --2- 172.21.15.92:0/2846390486 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe38806ce80 0x7fe388141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:16.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.493+0000 7fe38cb4a700 1 --2- 172.21.15.92:0/2846390486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3880c88e0 0x7fe388157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:16.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.493+0000 7fe3877fe700 1 -- 172.21.15.92:0/2846390486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3880c88e0 msgr2=0x7fe388157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:16.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.493+0000 7fe3877fe700 1 --2- 172.21.15.92:0/2846390486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3880c88e0 0x7fe388157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:16.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.493+0000 7fe3877fe700 1 -- 172.21.15.92:0/2846390486 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe38806ce80 msgr2=0x7fe388141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:16.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.493+0000 7fe3877fe700 1 --2- 172.21.15.92:0/2846390486 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe38806ce80 0x7fe388141c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:16.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.493+0000 7fe3877fe700 1 -- 172.21.15.92:0/2846390486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c0094f0 con 0x7fe38806d250 2023-10-06T10:38:16.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.493+0000 7fe3877fe700 1 --2- 172.21.15.92:0/2846390486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe38806d250 0x7fe388156ad0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009290 tx=0x7fe37c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:16.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.493+0000 7fe36ffff700 1 -- 172.21.15.92:0/2846390486 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c00bbe0 con 0x7fe38806d250 2023-10-06T10:38:16.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.493+0000 7fe36ffff700 1 -- 172.21.15.92:0/2846390486 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37c000c90 con 0x7fe38806d250 2023-10-06T10:38:16.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.493+0000 7fe38e5ad700 1 -- 172.21.15.92:0/2846390486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3880b01e0 con 0x7fe38806d250 2023-10-06T10:38:16.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.493+0000 7fe36ffff700 1 -- 172.21.15.92:0/2846390486 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c01a680 con 0x7fe38806d250 2023-10-06T10:38:16.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.493+0000 7fe38e5ad700 1 -- 172.21.15.92:0/2846390486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3881421b0 con 0x7fe38806d250 2023-10-06T10:38:16.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.494+0000 7fe38e5ad700 1 -- 172.21.15.92:0/2846390486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe388158b00 con 0x7fe38806d250 2023-10-06T10:38:16.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.494+0000 7fe36ffff700 1 -- 172.21.15.92:0/2846390486 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe37c01aa30 con 0x7fe38806d250 2023-10-06T10:38:16.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.495+0000 7fe36ffff700 1 --2- 172.21.15.92:0/2846390486 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe374066f30 0x7fe3740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:16.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.495+0000 7fe387fff700 1 --2- 172.21.15.92:0/2846390486 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe374066f30 0x7fe3740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:16.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.495+0000 7fe36ffff700 1 -- 172.21.15.92:0/2846390486 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fe37c085aa0 con 0x7fe38806d250 2023-10-06T10:38:16.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.495+0000 7fe387fff700 1 --2- 172.21.15.92:0/2846390486 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe374066f30 0x7fe3740693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe378006f00 tx=0x7fe378008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:16.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.499+0000 7fe36ffff700 1 -- 172.21.15.92:0/2846390486 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe388158b00 con 0x7fe38806d250 2023-10-06T10:38:16.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:16.723+0000 7fe38e5ad700 1 -- 172.21.15.92:0/2846390486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe388158b00 con 0x7fe38806d250 2023-10-06T10:38:17.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.345+0000 7fe36ffff700 1 -- 172.21.15.92:0/2846390486 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe388158b00 con 0x7fe38806d250 2023-10-06T10:38:17.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.350+0000 7fe38e5ad700 1 -- 172.21.15.92:0/2846390486 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe374066f30 msgr2=0x7fe3740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:17.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.350+0000 7fe38e5ad700 1 --2- 172.21.15.92:0/2846390486 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe374066f30 0x7fe3740693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe378006f00 tx=0x7fe378008040 comp rx=0 tx=0).stop 2023-10-06T10:38:17.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.350+0000 7fe38e5ad700 1 -- 172.21.15.92:0/2846390486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe38806d250 msgr2=0x7fe388156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:17.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.350+0000 7fe38e5ad700 1 --2- 172.21.15.92:0/2846390486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe38806d250 0x7fe388156ad0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009290 tx=0x7fe37c017000 comp rx=0 tx=0).stop 2023-10-06T10:38:17.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.350+0000 7fe38e5ad700 1 -- 172.21.15.92:0/2846390486 shutdown_connections 2023-10-06T10:38:17.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.350+0000 7fe38e5ad700 1 --2- 172.21.15.92:0/2846390486 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe374066f30 0x7fe3740693e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:17.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.350+0000 7fe38e5ad700 1 --2- 172.21.15.92:0/2846390486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3880c88e0 0x7fe388157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:17.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.350+0000 7fe38e5ad700 1 --2- 172.21.15.92:0/2846390486 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe38806ce80 0x7fe388141c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:17.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.350+0000 7fe38e5ad700 1 --2- 172.21.15.92:0/2846390486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe38806d250 0x7fe388156ad0 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009290 tx=0x7fe37c017000 comp rx=0 tx=0).stop 2023-10-06T10:38:17.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.350+0000 7fe38e5ad700 1 -- 172.21.15.92:0/2846390486 >> 172.21.15.92:0/2846390486 conn(0x7fe38806b610 msgr2=0x7fe388153f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:17.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.359+0000 7fe38e5ad700 1 -- 172.21.15.92:0/2846390486 shutdown_connections 2023-10-06T10:38:17.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.359+0000 7fe38e5ad700 1 -- 172.21.15.92:0/2846390486 wait complete. 2023-10-06T10:38:17.369 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 3 and data pool 4 2023-10-06T10:38:17.383 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-10-06T10:38:17.383 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-10-06T10:38:17.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.500+0000 7ff9ff07e700 1 Processor -- start 2023-10-06T10:38:17.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.500+0000 7ff9ff07e700 1 -- start start 2023-10-06T10:38:17.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.501+0000 7ff9ff07e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff9f806b5c0 0x7ff9f80685c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:17.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.501+0000 7ff9ff07e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9f806b990 0x7ff9f8068b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:17.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.501+0000 7ff9ff07e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f80c89e0 con 0x7ff9f80c7860 2023-10-06T10:38:17.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.501+0000 7ff9ff07e700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f80c7dd0 con 0x7ff9f806b5c0 2023-10-06T10:38:17.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.501+0000 7ff9ff07e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f80c80b0 con 0x7ff9f806b990 2023-10-06T10:38:17.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.501+0000 7ff9fd61b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff9f80c7860 0x7ff9f80a98b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38354/0 (socket says 172.21.15.92:38354) 2023-10-06T10:38:17.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.501+0000 7ff9fd61b700 1 -- 172.21.15.92:0/260749864 learned_addr learned my addr 172.21.15.92:0/260749864 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:17.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.501+0000 7ff9fce1a700 1 --2- 172.21.15.92:0/260749864 >> v2:172.21.15.124:3300/0 conn(0x7ff9f806b5c0 0x7ff9f80685c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:17.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.501+0000 7ff9effff700 1 --2- 172.21.15.92:0/260749864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9f806b990 0x7ff9f8068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:17.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.502+0000 7ff9ef7fe700 1 -- 172.21.15.92:0/260749864 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1085851779 0 0) 0x7ff9f80c89e0 con 0x7ff9f80c7860 2023-10-06T10:38:17.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.502+0000 7ff9ef7fe700 1 -- 172.21.15.92:0/260749864 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9e0003570 con 0x7ff9f80c7860 2023-10-06T10:38:17.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.502+0000 7ff9effff700 1 -- 172.21.15.92:0/260749864 >> v2:172.21.15.124:3300/0 conn(0x7ff9f806b5c0 msgr2=0x7ff9f80685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:17.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.502+0000 7ff9effff700 1 --2- 172.21.15.92:0/260749864 >> v2:172.21.15.124:3300/0 conn(0x7ff9f806b5c0 0x7ff9f80685c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:17.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.502+0000 7ff9effff700 1 -- 172.21.15.92:0/260749864 >> v1:172.21.15.92:6789/0 conn(0x7ff9f80c7860 legacy=0x7ff9f80a98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:17.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.502+0000 7ff9effff700 1 -- 172.21.15.92:0/260749864 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f80c8390 con 0x7ff9f806b990 2023-10-06T10:38:17.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.502+0000 7ff9fce1a700 1 --2- 172.21.15.92:0/260749864 >> v2:172.21.15.124:3300/0 conn(0x7ff9f806b5c0 0x7ff9f80685c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:17.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.502+0000 7ff9effff700 1 --2- 172.21.15.92:0/260749864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9f806b990 0x7ff9f8068b00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80098e0 tx=0x7ff9e80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d5cb6826e31e8fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:17.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.502+0000 7ff9ef7fe700 1 -- 172.21.15.92:0/260749864 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8011630 con 0x7ff9f806b990 2023-10-06T10:38:17.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.502+0000 7ff9ef7fe700 1 -- 172.21.15.92:0/260749864 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e8011790 con 0x7ff9f806b990 2023-10-06T10:38:17.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.502+0000 7ff9ef7fe700 1 -- 172.21.15.92:0/260749864 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8011b10 con 0x7ff9f806b990 2023-10-06T10:38:17.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.503+0000 7ff9ff07e700 1 -- 172.21.15.92:0/260749864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9f806b990 msgr2=0x7ff9f8068b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:17.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.503+0000 7ff9ff07e700 1 --2- 172.21.15.92:0/260749864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9f806b990 0x7ff9f8068b00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80098e0 tx=0x7ff9e80091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:17.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.503+0000 7ff9ff07e700 1 -- 172.21.15.92:0/260749864 shutdown_connections 2023-10-06T10:38:17.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.503+0000 7ff9ff07e700 1 --2- 172.21.15.92:0/260749864 >> v2:172.21.15.124:3300/0 conn(0x7ff9f806b5c0 0x7ff9f80685c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:17.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.503+0000 7ff9ff07e700 1 --2- 172.21.15.92:0/260749864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9f806b990 0x7ff9f8068b00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:17.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.503+0000 7ff9ff07e700 1 -- 172.21.15.92:0/260749864 >> 172.21.15.92:0/260749864 conn(0x7ff9f80c9df0 msgr2=0x7ff9f806db70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:17.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.503+0000 7ff9ff07e700 1 -- 172.21.15.92:0/260749864 shutdown_connections 2023-10-06T10:38:17.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.503+0000 7ff9ff07e700 1 -- 172.21.15.92:0/260749864 wait complete. 2023-10-06T10:38:17.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.504+0000 7ff9ff07e700 1 Processor -- start 2023-10-06T10:38:17.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.504+0000 7ff9ff07e700 1 -- start start 2023-10-06T10:38:17.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.505+0000 7ff9ff07e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f806b5c0 0x7ff9f815e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:17.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.505+0000 7ff9ff07e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9f806b990 0x7ff9f815c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:17.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.505+0000 7ff9ff07e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9f80c7860 0x7ff9f8157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:17.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.505+0000 7ff9ff07e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f815d540 con 0x7ff9f806b5c0 2023-10-06T10:38:17.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.505+0000 7ff9ff07e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f815d7e0 con 0x7ff9f806b990 2023-10-06T10:38:17.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.505+0000 7ff9ff07e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f815dac0 con 0x7ff9f80c7860 2023-10-06T10:38:17.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.505+0000 7ff9fce1a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f806b5c0 0x7ff9f815e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:17.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.505+0000 7ff9fce1a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f806b5c0 0x7ff9f815e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59732/0 (socket says 172.21.15.92:59732) 2023-10-06T10:38:17.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.505+0000 7ff9fce1a700 1 -- 172.21.15.92:0/380452304 learned_addr learned my addr 172.21.15.92:0/380452304 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:17.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.505+0000 7ff9effff700 1 --2- 172.21.15.92:0/380452304 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9f806b990 0x7ff9f815c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:17.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.505+0000 7ff9fd61b700 1 --2- 172.21.15.92:0/380452304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9f80c7860 0x7ff9f8157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.505+0000 7ff9fce1a700 1 -- 172.21.15.92:0/380452304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9f80c7860 msgr2=0x7ff9f8157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.505+0000 7ff9fce1a700 1 --2- 172.21.15.92:0/380452304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9f80c7860 0x7ff9f8157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.505+0000 7ff9fce1a700 1 -- 172.21.15.92:0/380452304 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9f806b990 msgr2=0x7ff9f815c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.505+0000 7ff9fce1a700 1 --2- 172.21.15.92:0/380452304 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9f806b990 0x7ff9f815c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.505+0000 7ff9fce1a700 1 -- 172.21.15.92:0/380452304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f4009a20 con 0x7ff9f806b5c0 2023-10-06T10:38:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.505+0000 7ff9fce1a700 1 --2- 172.21.15.92:0/380452304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f806b5c0 0x7ff9f815e880 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400c0c0 tx=0x7ff9f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.505+0000 7ff9ecff9700 1 -- 172.21.15.92:0/380452304 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f400cc00 con 0x7ff9f806b5c0 2023-10-06T10:38:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.505+0000 7ff9ecff9700 1 -- 172.21.15.92:0/380452304 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f400f040 con 0x7ff9f806b5c0 2023-10-06T10:38:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.505+0000 7ff9ff07e700 1 -- 172.21.15.92:0/380452304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e8009420 con 0x7ff9f806b5c0 2023-10-06T10:38:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.505+0000 7ff9ecff9700 1 -- 172.21.15.92:0/380452304 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4014480 con 0x7ff9f806b5c0 2023-10-06T10:38:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.505+0000 7ff9ff07e700 1 -- 172.21.15.92:0/380452304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f815e080 con 0x7ff9f806b5c0 2023-10-06T10:38:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.506+0000 7ff9ff07e700 1 -- 172.21.15.92:0/380452304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f815e6d0 con 0x7ff9f806b5c0 2023-10-06T10:38:17.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.507+0000 7ff9ecff9700 1 -- 172.21.15.92:0/380452304 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff9f40147f0 con 0x7ff9f806b5c0 2023-10-06T10:38:17.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.507+0000 7ff9ecff9700 1 --2- 172.21.15.92:0/380452304 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff9e4066f30 0x7ff9e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:17.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.507+0000 7ff9ecff9700 1 -- 172.21.15.92:0/380452304 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff9f4085390 con 0x7ff9f806b5c0 2023-10-06T10:38:17.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.507+0000 7ff9effff700 1 --2- 172.21.15.92:0/380452304 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff9e4066f30 0x7ff9e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:17.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.507+0000 7ff9effff700 1 --2- 172.21.15.92:0/380452304 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff9e4066f30 0x7ff9e40693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff9f806bde0 tx=0x7ff9e8017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:17.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.511+0000 7ff9ecff9700 1 -- 172.21.15.92:0/380452304 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9f40524a0 con 0x7ff9f806b5c0 2023-10-06T10:38:17.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:17.707+0000 7ff9ff07e700 1 -- 172.21.15.92:0/380452304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7ff9f8064a90 con 0x7ff9f806b5c0 2023-10-06T10:38:18.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.360+0000 7ff9ecff9700 1 -- 172.21.15.92:0/380452304 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7ff9f4055ac0 con 0x7ff9f806b5c0 2023-10-06T10:38:18.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.362+0000 7ff9ff07e700 1 -- 172.21.15.92:0/380452304 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff9e4066f30 msgr2=0x7ff9e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:18.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.362+0000 7ff9ff07e700 1 --2- 172.21.15.92:0/380452304 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff9e4066f30 0x7ff9e40693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff9f806bde0 tx=0x7ff9e8017000 comp rx=0 tx=0).stop 2023-10-06T10:38:18.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.362+0000 7ff9ff07e700 1 -- 172.21.15.92:0/380452304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f806b5c0 msgr2=0x7ff9f815e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:18.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.362+0000 7ff9ff07e700 1 --2- 172.21.15.92:0/380452304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f806b5c0 0x7ff9f815e880 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400c0c0 tx=0x7ff9f400b040 comp rx=0 tx=0).stop 2023-10-06T10:38:18.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.362+0000 7ff9ff07e700 1 -- 172.21.15.92:0/380452304 shutdown_connections 2023-10-06T10:38:18.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.362+0000 7ff9ff07e700 1 --2- 172.21.15.92:0/380452304 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff9e4066f30 0x7ff9e40693e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:18.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.362+0000 7ff9ff07e700 1 --2- 172.21.15.92:0/380452304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9f80c7860 0x7ff9f8157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:18.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.362+0000 7ff9ff07e700 1 --2- 172.21.15.92:0/380452304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f806b5c0 0x7ff9f815e880 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:18.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.362+0000 7ff9ff07e700 1 --2- 172.21.15.92:0/380452304 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9f806b990 0x7ff9f815c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:18.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.362+0000 7ff9ff07e700 1 -- 172.21.15.92:0/380452304 >> 172.21.15.92:0/380452304 conn(0x7ff9f80c9df0 msgr2=0x7ff9f80ca3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:18.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.363+0000 7ff9ff07e700 1 -- 172.21.15.92:0/380452304 shutdown_connections 2023-10-06T10:38:18.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.363+0000 7ff9ff07e700 1 -- 172.21.15.92:0/380452304 wait complete. 2023-10-06T10:38:18.378 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T10:38:18.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.499+0000 7ff2093b7700 1 Processor -- start 2023-10-06T10:38:18.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.499+0000 7ff2093b7700 1 -- start start 2023-10-06T10:38:18.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.500+0000 7ff2093b7700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff2040b2620 0x7ff2040cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:18.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.500+0000 7ff2093b7700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2040b29f0 0x7ff2040cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:18.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.500+0000 7ff2093b7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2040cc970 con 0x7ff2040ceaa0 2023-10-06T10:38:18.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.500+0000 7ff2093b7700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2040cbd60 con 0x7ff2040b2620 2023-10-06T10:38:18.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.500+0000 7ff2093b7700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2040cc040 con 0x7ff2040b29f0 2023-10-06T10:38:18.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.500+0000 7ff208bb6700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff2040ceaa0 0x7ff2040cef40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38356/0 (socket says 172.21.15.92:38356) 2023-10-06T10:38:18.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.500+0000 7ff208bb6700 1 -- 172.21.15.92:0/1144183126 learned_addr learned my addr 172.21.15.92:0/1144183126 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:18.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.500+0000 7ff2037fe700 1 --2- 172.21.15.92:0/1144183126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2040b29f0 0x7ff2040cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:18.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.500+0000 7ff203fff700 1 --2- 172.21.15.92:0/1144183126 >> v2:172.21.15.124:3300/0 conn(0x7ff2040b2620 0x7ff2040cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:18.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.500+0000 7ff202ffd700 1 -- 172.21.15.92:0/1144183126 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1111311761 0 0) 0x7ff2040cc970 con 0x7ff2040ceaa0 2023-10-06T10:38:18.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.500+0000 7ff202ffd700 1 -- 172.21.15.92:0/1144183126 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1ec003570 con 0x7ff2040ceaa0 2023-10-06T10:38:18.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.500+0000 7ff202ffd700 1 -- 172.21.15.92:0/1144183126 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2976404060 0 0) 0x7ff1ec003570 con 0x7ff2040ceaa0 2023-10-06T10:38:18.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.500+0000 7ff2037fe700 1 -- 172.21.15.92:0/1144183126 >> v2:172.21.15.124:3300/0 conn(0x7ff2040b2620 msgr2=0x7ff2040cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:18.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.500+0000 7ff2037fe700 1 --2- 172.21.15.92:0/1144183126 >> v2:172.21.15.124:3300/0 conn(0x7ff2040b2620 0x7ff2040cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:18.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.500+0000 7ff2037fe700 1 -- 172.21.15.92:0/1144183126 >> v1:172.21.15.92:6789/0 conn(0x7ff2040ceaa0 legacy=0x7ff2040cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:18.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.500+0000 7ff2037fe700 1 -- 172.21.15.92:0/1144183126 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2040cc320 con 0x7ff2040b29f0 2023-10-06T10:38:18.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.500+0000 7ff203fff700 1 --2- 172.21.15.92:0/1144183126 >> v2:172.21.15.124:3300/0 conn(0x7ff2040b2620 0x7ff2040cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:18.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.501+0000 7ff2037fe700 1 --2- 172.21.15.92:0/1144183126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2040b29f0 0x7ff2040cd1f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff1f80098e0 tx=0x7ff1f80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5d2930750206d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:18.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.501+0000 7ff202ffd700 1 -- 172.21.15.92:0/1144183126 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc0017f0 con 0x7ff2040b29f0 2023-10-06T10:38:18.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.501+0000 7ff202ffd700 1 -- 172.21.15.92:0/1144183126 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f8011420 con 0x7ff2040b29f0 2023-10-06T10:38:18.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.501+0000 7ff202ffd700 1 -- 172.21.15.92:0/1144183126 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f80117c0 con 0x7ff2040b29f0 2023-10-06T10:38:18.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.501+0000 7ff2093b7700 1 -- 172.21.15.92:0/1144183126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2040b29f0 msgr2=0x7ff2040cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:18.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.501+0000 7ff2093b7700 1 --2- 172.21.15.92:0/1144183126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2040b29f0 0x7ff2040cd1f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff1f80098e0 tx=0x7ff1f80091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:18.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.501+0000 7ff2093b7700 1 -- 172.21.15.92:0/1144183126 shutdown_connections 2023-10-06T10:38:18.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.501+0000 7ff2093b7700 1 --2- 172.21.15.92:0/1144183126 >> v2:172.21.15.124:3300/0 conn(0x7ff2040b2620 0x7ff2040cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:18.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.501+0000 7ff2093b7700 1 --2- 172.21.15.92:0/1144183126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2040b29f0 0x7ff2040cd1f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:18.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.501+0000 7ff2093b7700 1 -- 172.21.15.92:0/1144183126 >> 172.21.15.92:0/1144183126 conn(0x7ff204099520 msgr2=0x7ff20406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:18.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.501+0000 7ff2093b7700 1 -- 172.21.15.92:0/1144183126 shutdown_connections 2023-10-06T10:38:18.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.501+0000 7ff2093b7700 1 -- 172.21.15.92:0/1144183126 wait complete. 2023-10-06T10:38:18.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.502+0000 7ff2093b7700 1 Processor -- start 2023-10-06T10:38:18.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.503+0000 7ff2093b7700 1 -- start start 2023-10-06T10:38:18.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.503+0000 7ff2093b7700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2040b2620 0x7ff20414e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:18.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.503+0000 7ff2093b7700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2040b29f0 0x7ff2040c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:18.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.503+0000 7ff2093b7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2040ceaa0 0x7ff20416d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:18.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.503+0000 7ff2093b7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2040b3d10 con 0x7ff2040ceaa0 2023-10-06T10:38:18.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.503+0000 7ff2093b7700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2040b6170 con 0x7ff2040b29f0 2023-10-06T10:38:18.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.503+0000 7ff2093b7700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2040c95b0 con 0x7ff2040b2620 2023-10-06T10:38:18.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.503+0000 7ff208bb6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2040ceaa0 0x7ff20416d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:18.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.504+0000 7ff208bb6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2040ceaa0 0x7ff20416d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59738/0 (socket says 172.21.15.92:59738) 2023-10-06T10:38:18.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.504+0000 7ff208bb6700 1 -- 172.21.15.92:0/605833152 learned_addr learned my addr 172.21.15.92:0/605833152 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:18.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.504+0000 7ff203fff700 1 --2- 172.21.15.92:0/605833152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2040b2620 0x7ff20414e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:18.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.504+0000 7ff2037fe700 1 --2- 172.21.15.92:0/605833152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2040b29f0 0x7ff2040c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:18.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.504+0000 7ff208bb6700 1 -- 172.21.15.92:0/605833152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2040b2620 msgr2=0x7ff20414e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:18.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.504+0000 7ff208bb6700 1 --2- 172.21.15.92:0/605833152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2040b2620 0x7ff20414e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:18.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.504+0000 7ff208bb6700 1 -- 172.21.15.92:0/605833152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2040b29f0 msgr2=0x7ff2040c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:18.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.504+0000 7ff208bb6700 1 --2- 172.21.15.92:0/605833152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2040b29f0 0x7ff2040c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:18.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.504+0000 7ff208bb6700 1 -- 172.21.15.92:0/605833152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f40093c0 con 0x7ff2040ceaa0 2023-10-06T10:38:18.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.504+0000 7ff2037fe700 1 --2- 172.21.15.92:0/605833152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2040b29f0 0x7ff2040c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:18.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.504+0000 7ff208bb6700 1 --2- 172.21.15.92:0/605833152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2040ceaa0 0x7ff20416d180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc001560 tx=0x7ff1fc001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:18.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.504+0000 7ff1ebfff700 1 -- 172.21.15.92:0/605833152 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc004dc0 con 0x7ff2040ceaa0 2023-10-06T10:38:18.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.504+0000 7ff2093b7700 1 -- 172.21.15.92:0/605833152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f8009420 con 0x7ff2040ceaa0 2023-10-06T10:38:18.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.504+0000 7ff1ebfff700 1 -- 172.21.15.92:0/605833152 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1fc00a750 con 0x7ff2040ceaa0 2023-10-06T10:38:18.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.504+0000 7ff1ebfff700 1 -- 172.21.15.92:0/605833152 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc00ab10 con 0x7ff2040ceaa0 2023-10-06T10:38:18.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.504+0000 7ff2093b7700 1 -- 172.21.15.92:0/605833152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2040c9b70 con 0x7ff2040ceaa0 2023-10-06T10:38:18.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.505+0000 7ff2093b7700 1 -- 172.21.15.92:0/605833152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff20405f230 con 0x7ff2040ceaa0 2023-10-06T10:38:18.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.506+0000 7ff1ebfff700 1 -- 172.21.15.92:0/605833152 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff1fc007150 con 0x7ff2040ceaa0 2023-10-06T10:38:18.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.506+0000 7ff1ebfff700 1 --2- 172.21.15.92:0/605833152 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff1f0066f30 0x7ff1f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:18.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.506+0000 7ff1ebfff700 1 -- 172.21.15.92:0/605833152 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6284+0+0 (secure 0 0 0) 0x7ff1fc083f70 con 0x7ff2040ceaa0 2023-10-06T10:38:18.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.506+0000 7ff203fff700 1 --2- 172.21.15.92:0/605833152 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff1f0066f30 0x7ff1f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:18.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.506+0000 7ff203fff700 1 --2- 172.21.15.92:0/605833152 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff1f0066f30 0x7ff1f00693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009390 tx=0x7ff1f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:18.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.510+0000 7ff1ebfff700 1 -- 172.21.15.92:0/605833152 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1fc050fa0 con 0x7ff2040ceaa0 2023-10-06T10:38:18.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:18.695+0000 7ff2093b7700 1 -- 172.21.15.92:0/605833152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff2040cc600 con 0x7ff2040ceaa0 2023-10-06T10:38:20.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.370+0000 7ff1ebfff700 1 -- 172.21.15.92:0/605833152 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7ff1fc018100 con 0x7ff2040ceaa0 2023-10-06T10:38:20.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.374+0000 7ff2093b7700 1 -- 172.21.15.92:0/605833152 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff1f0066f30 msgr2=0x7ff1f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:20.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.374+0000 7ff2093b7700 1 --2- 172.21.15.92:0/605833152 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff1f0066f30 0x7ff1f00693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009390 tx=0x7ff1f4008040 comp rx=0 tx=0).stop 2023-10-06T10:38:20.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.374+0000 7ff2093b7700 1 -- 172.21.15.92:0/605833152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2040ceaa0 msgr2=0x7ff20416d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:20.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.374+0000 7ff2093b7700 1 --2- 172.21.15.92:0/605833152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2040ceaa0 0x7ff20416d180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc001560 tx=0x7ff1fc001640 comp rx=0 tx=0).stop 2023-10-06T10:38:20.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.381+0000 7ff2093b7700 1 -- 172.21.15.92:0/605833152 shutdown_connections 2023-10-06T10:38:20.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.381+0000 7ff2093b7700 1 --2- 172.21.15.92:0/605833152 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff1f0066f30 0x7ff1f00693e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:20.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.381+0000 7ff2093b7700 1 --2- 172.21.15.92:0/605833152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2040ceaa0 0x7ff20416d180 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:20.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.381+0000 7ff2093b7700 1 --2- 172.21.15.92:0/605833152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2040b2620 0x7ff20414e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:20.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.381+0000 7ff2093b7700 1 --2- 172.21.15.92:0/605833152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2040b29f0 0x7ff2040c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:20.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.381+0000 7ff2093b7700 1 -- 172.21.15.92:0/605833152 >> 172.21.15.92:0/605833152 conn(0x7ff204099520 msgr2=0x7ff2040ccb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:20.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.381+0000 7ff2093b7700 1 -- 172.21.15.92:0/605833152 shutdown_connections 2023-10-06T10:38:20.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.381+0000 7ff2093b7700 1 -- 172.21.15.92:0/605833152 wait complete. 2023-10-06T10:38:20.384 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data_ec' created 2023-10-06T10:38:20.396 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-10-06T10:38:20.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.514+0000 7f5e82895700 1 Processor -- start 2023-10-06T10:38:20.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.514+0000 7f5e82895700 1 -- start start 2023-10-06T10:38:20.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.515+0000 7f5e82895700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5e7c0b0720 0x7f5e7c0b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:20.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.515+0000 7f5e82895700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e7c0c4cd0 0x7f5e7c0cc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:20.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.515+0000 7f5e82895700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e7c0c4130 con 0x7f5e7c0c3090 2023-10-06T10:38:20.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.515+0000 7f5e82895700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e7c0c3520 con 0x7f5e7c0b0720 2023-10-06T10:38:20.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.515+0000 7f5e82895700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c0c3800 con 0x7f5e7c0c4cd0 2023-10-06T10:38:20.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.515+0000 7f5e7bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5e7c0b0720 0x7f5e7c0b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:20.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.515+0000 7f5e7bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5e7c0b0720 0x7f5e7c0b10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59742/0 (socket says 172.21.15.92:59742) 2023-10-06T10:38:20.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.515+0000 7f5e7bfff700 1 -- 172.21.15.92:0/578542959 learned_addr learned my addr 172.21.15.92:0/578542959 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:20.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.515+0000 7f5e80e32700 1 --2- 172.21.15.92:0/578542959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e7c0c4cd0 0x7f5e7c0cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.516+0000 7f5e7bfff700 1 -- 172.21.15.92:0/578542959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e7c0c4cd0 msgr2=0x7f5e7c0cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.516+0000 7f5e7bfff700 1 --2- 172.21.15.92:0/578542959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e7c0c4cd0 0x7f5e7c0cc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.516+0000 7f5e7bfff700 1 -- 172.21.15.92:0/578542959 >> v1:172.21.15.124:6789/0 conn(0x7f5e7c0c3090 legacy=0x7f5e7c0c45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.516+0000 7f5e7bfff700 1 -- 172.21.15.92:0/578542959 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7c0c3ae0 con 0x7f5e7c0b0720 2023-10-06T10:38:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.516+0000 7f5e80e32700 1 --2- 172.21.15.92:0/578542959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e7c0c4cd0 0x7f5e7c0cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.516+0000 7f5e7bfff700 1 --2- 172.21.15.92:0/578542959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e7c0b0720 0x7f5e7c0b10b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009990 tx=0x7f5e6c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa782feee748fb24 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.516+0000 7f5e7affd700 1 -- 172.21.15.92:0/578542959 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c005c10 con 0x7f5e7c0b0720 2023-10-06T10:38:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.516+0000 7f5e7affd700 1 -- 172.21.15.92:0/578542959 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c005d70 con 0x7f5e7c0b0720 2023-10-06T10:38:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.516+0000 7f5e7affd700 1 -- 172.21.15.92:0/578542959 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c00b910 con 0x7f5e7c0b0720 2023-10-06T10:38:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.516+0000 7f5e82895700 1 -- 172.21.15.92:0/578542959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e7c0b0720 msgr2=0x7f5e7c0b10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.516+0000 7f5e82895700 1 --2- 172.21.15.92:0/578542959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e7c0b0720 0x7f5e7c0b10b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009990 tx=0x7f5e6c009220 comp rx=0 tx=0).stop 2023-10-06T10:38:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.516+0000 7f5e82895700 1 -- 172.21.15.92:0/578542959 shutdown_connections 2023-10-06T10:38:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.516+0000 7f5e82895700 1 --2- 172.21.15.92:0/578542959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e7c0c4cd0 0x7f5e7c0cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.516+0000 7f5e82895700 1 --2- 172.21.15.92:0/578542959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e7c0b0720 0x7f5e7c0b10b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.516+0000 7f5e82895700 1 -- 172.21.15.92:0/578542959 >> 172.21.15.92:0/578542959 conn(0x7f5e7c087fc0 msgr2=0x7f5e7c0ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.516+0000 7f5e82895700 1 -- 172.21.15.92:0/578542959 shutdown_connections 2023-10-06T10:38:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.516+0000 7f5e82895700 1 -- 172.21.15.92:0/578542959 wait complete. 2023-10-06T10:38:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.517+0000 7f5e82895700 1 Processor -- start 2023-10-06T10:38:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.518+0000 7f5e82895700 1 -- start start 2023-10-06T10:38:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.518+0000 7f5e82895700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e7c0b0720 0x7f5e7c0690d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.518+0000 7f5e82895700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e7c0c3090 0x7f5e7c0b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.518+0000 7f5e82895700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e7c0c4cd0 0x7f5e7c0c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.518+0000 7f5e82895700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c0b3bf0 con 0x7f5e7c0b0720 2023-10-06T10:38:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.518+0000 7f5e82895700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c0c8230 con 0x7f5e7c0c4cd0 2023-10-06T10:38:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.518+0000 7f5e82895700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c0696b0 con 0x7f5e7c0c3090 2023-10-06T10:38:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.518+0000 7f5e7bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e7c0b0720 0x7f5e7c0690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.518+0000 7f5e7bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e7c0b0720 0x7f5e7c0690d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59754/0 (socket says 172.21.15.92:59754) 2023-10-06T10:38:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.518+0000 7f5e7bfff700 1 -- 172.21.15.92:0/2541026302 learned_addr learned my addr 172.21.15.92:0/2541026302 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.518+0000 7f5e80e32700 1 --2- 172.21.15.92:0/2541026302 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e7c0c4cd0 0x7f5e7c0c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.518+0000 7f5e7b7fe700 1 --2- 172.21.15.92:0/2541026302 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e7c0c3090 0x7f5e7c0b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.519+0000 7f5e7bfff700 1 -- 172.21.15.92:0/2541026302 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e7c0c3090 msgr2=0x7f5e7c0b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.519+0000 7f5e7bfff700 1 --2- 172.21.15.92:0/2541026302 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e7c0c3090 0x7f5e7c0b36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.519+0000 7f5e7bfff700 1 -- 172.21.15.92:0/2541026302 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e7c0c4cd0 msgr2=0x7f5e7c0c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.519+0000 7f5e7bfff700 1 --2- 172.21.15.92:0/2541026302 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e7c0c4cd0 0x7f5e7c0c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.519+0000 7f5e7bfff700 1 -- 172.21.15.92:0/2541026302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6c009480 con 0x7f5e7c0b0720 2023-10-06T10:38:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.519+0000 7f5e7bfff700 1 --2- 172.21.15.92:0/2541026302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e7c0b0720 0x7f5e7c0690d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00b3c0 tx=0x7f5e6c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.519+0000 7f5e63fff700 1 -- 172.21.15.92:0/2541026302 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c00bee0 con 0x7f5e7c0b0720 2023-10-06T10:38:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.519+0000 7f5e63fff700 1 -- 172.21.15.92:0/2541026302 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c000d10 con 0x7f5e7c0b0720 2023-10-06T10:38:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.519+0000 7f5e82895700 1 -- 172.21.15.92:0/2541026302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7c0c6f30 con 0x7f5e7c0b0720 2023-10-06T10:38:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.519+0000 7f5e63fff700 1 -- 172.21.15.92:0/2541026302 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c00b8f0 con 0x7f5e7c0b0720 2023-10-06T10:38:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.519+0000 7f5e82895700 1 -- 172.21.15.92:0/2541026302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7c0c74f0 con 0x7f5e7c0b0720 2023-10-06T10:38:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.520+0000 7f5e82895700 1 -- 172.21.15.92:0/2541026302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e7c064a90 con 0x7f5e7c0b0720 2023-10-06T10:38:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.520+0000 7f5e63fff700 1 -- 172.21.15.92:0/2541026302 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5e6c01b640 con 0x7f5e7c0b0720 2023-10-06T10:38:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.521+0000 7f5e63fff700 1 --2- 172.21.15.92:0/2541026302 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5e68066f30 0x7f5e680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:20.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.521+0000 7f5e7b7fe700 1 --2- 172.21.15.92:0/2541026302 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5e68066f30 0x7f5e680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:20.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.521+0000 7f5e63fff700 1 -- 172.21.15.92:0/2541026302 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f5e6c086730 con 0x7f5e7c0b0720 2023-10-06T10:38:20.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.521+0000 7f5e7b7fe700 1 --2- 172.21.15.92:0/2541026302 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5e68066f30 0x7f5e680693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c069b40 tx=0x7f5e70007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:20.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.525+0000 7f5e63fff700 1 -- 172.21.15.92:0/2541026302 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e6c053570 con 0x7f5e7c0b0720 2023-10-06T10:38:20.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:20.708+0000 7f5e82895700 1 -- 172.21.15.92:0/2541026302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f5e7c0c3dc0 con 0x7f5e7c0b0720 2023-10-06T10:38:21.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.372+0000 7f5e63fff700 1 -- 172.21.15.92:0/2541026302 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5e6c056b90 con 0x7f5e7c0b0720 2023-10-06T10:38:21.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.376+0000 7f5e82895700 1 -- 172.21.15.92:0/2541026302 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5e68066f30 msgr2=0x7f5e680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:21.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.376+0000 7f5e82895700 1 --2- 172.21.15.92:0/2541026302 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5e68066f30 0x7f5e680693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c069b40 tx=0x7f5e70007ec0 comp rx=0 tx=0).stop 2023-10-06T10:38:21.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.376+0000 7f5e82895700 1 -- 172.21.15.92:0/2541026302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e7c0b0720 msgr2=0x7f5e7c0690d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:21.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.376+0000 7f5e82895700 1 --2- 172.21.15.92:0/2541026302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e7c0b0720 0x7f5e7c0690d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00b3c0 tx=0x7f5e6c016000 comp rx=0 tx=0).stop 2023-10-06T10:38:21.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.377+0000 7f5e82895700 1 -- 172.21.15.92:0/2541026302 shutdown_connections 2023-10-06T10:38:21.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.377+0000 7f5e82895700 1 --2- 172.21.15.92:0/2541026302 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5e68066f30 0x7f5e680693e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:21.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.377+0000 7f5e82895700 1 --2- 172.21.15.92:0/2541026302 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e7c0c4cd0 0x7f5e7c0c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:21.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.377+0000 7f5e82895700 1 --2- 172.21.15.92:0/2541026302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e7c0b0720 0x7f5e7c0690d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:21.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.377+0000 7f5e82895700 1 --2- 172.21.15.92:0/2541026302 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e7c0c3090 0x7f5e7c0b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:21.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.377+0000 7f5e82895700 1 -- 172.21.15.92:0/2541026302 >> 172.21.15.92:0/2541026302 conn(0x7f5e7c087fc0 msgr2=0x7f5e7c156b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:21.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.382+0000 7f5e82895700 1 -- 172.21.15.92:0/2541026302 shutdown_connections 2023-10-06T10:38:21.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.384+0000 7f5e82895700 1 -- 172.21.15.92:0/2541026302 wait complete. 2023-10-06T10:38:21.388 INFO:teuthology.orchestra.run.smithi092.stderr:set pool 5 allow_ec_overwrites to true 2023-10-06T10:38:21.407 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-10-06T10:38:21.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.522+0000 7effb50b1700 1 Processor -- start 2023-10-06T10:38:21.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.523+0000 7effb50b1700 1 -- start start 2023-10-06T10:38:21.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.523+0000 7effb50b1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7effb0156b70 0x7effb00c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:21.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.523+0000 7effb50b1700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7effb006ac40 0x7effb0157390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:21.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.523+0000 7effb50b1700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb01578d0 con 0x7effb0156f40 2023-10-06T10:38:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.523+0000 7effb50b1700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb006c180 con 0x7effb0156b70 2023-10-06T10:38:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.523+0000 7effb50b1700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb00c8af0 con 0x7effb006ac40 2023-10-06T10:38:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.523+0000 7effaed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7effb0156b70 0x7effb00c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.523+0000 7effaed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7effb0156b70 0x7effb00c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59764/0 (socket says 172.21.15.92:59764) 2023-10-06T10:38:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.523+0000 7effaed9d700 1 -- 172.21.15.92:0/3489130768 learned_addr learned my addr 172.21.15.92:0/3489130768 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.524+0000 7effaed9d700 1 -- 172.21.15.92:0/3489130768 >> v2:172.21.15.124:3300/0 conn(0x7effb006ac40 msgr2=0x7effb0157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.524+0000 7effaed9d700 1 --2- 172.21.15.92:0/3489130768 >> v2:172.21.15.124:3300/0 conn(0x7effb006ac40 0x7effb0157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.524+0000 7effaed9d700 1 -- 172.21.15.92:0/3489130768 >> v1:172.21.15.124:6789/0 conn(0x7effb0156f40 legacy=0x7effb006a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.524+0000 7effaed9d700 1 -- 172.21.15.92:0/3489130768 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb00c8dd0 con 0x7effb0156b70 2023-10-06T10:38:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.524+0000 7effaed9d700 1 --2- 172.21.15.92:0/3489130768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effb0156b70 0x7effb00c84a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7effa00099b0 tx=0x7effa0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622864b00b34d7ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.524+0000 7effae59c700 1 -- 172.21.15.92:0/3489130768 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa000b7e0 con 0x7effb0156b70 2023-10-06T10:38:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.524+0000 7effae59c700 1 -- 172.21.15.92:0/3489130768 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa000b940 con 0x7effb0156b70 2023-10-06T10:38:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.524+0000 7effae59c700 1 -- 172.21.15.92:0/3489130768 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa000bce0 con 0x7effb0156b70 2023-10-06T10:38:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.524+0000 7effb50b1700 1 -- 172.21.15.92:0/3489130768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effb0156b70 msgr2=0x7effb00c84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:21.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.524+0000 7effb50b1700 1 --2- 172.21.15.92:0/3489130768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effb0156b70 0x7effb00c84a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7effa00099b0 tx=0x7effa0009250 comp rx=0 tx=0).stop 2023-10-06T10:38:21.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.525+0000 7effb50b1700 1 -- 172.21.15.92:0/3489130768 shutdown_connections 2023-10-06T10:38:21.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.525+0000 7effb50b1700 1 --2- 172.21.15.92:0/3489130768 >> v2:172.21.15.124:3300/0 conn(0x7effb006ac40 0x7effb0157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:21.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.525+0000 7effb50b1700 1 --2- 172.21.15.92:0/3489130768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effb0156b70 0x7effb00c84a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:21.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.525+0000 7effb50b1700 1 -- 172.21.15.92:0/3489130768 >> 172.21.15.92:0/3489130768 conn(0x7effb00cd1a0 msgr2=0x7effb00cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:21.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.525+0000 7effb50b1700 1 -- 172.21.15.92:0/3489130768 shutdown_connections 2023-10-06T10:38:21.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.525+0000 7effb50b1700 1 -- 172.21.15.92:0/3489130768 wait complete. 2023-10-06T10:38:21.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.526+0000 7effb50b1700 1 Processor -- start 2023-10-06T10:38:21.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.526+0000 7effb50b1700 1 -- start start 2023-10-06T10:38:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effb50b1700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effb006ac40 0x7effb013f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effb50b1700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb0156b70 0x7effb0167480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effb50b1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effb0156f40 0x7effb016b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effb50b1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb016ae10 con 0x7effb0156f40 2023-10-06T10:38:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effb50b1700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb0167d80 con 0x7effb006ac40 2023-10-06T10:38:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effb50b1700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb0168060 con 0x7effb0156b70 2023-10-06T10:38:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effaf59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effb0156f40 0x7effb016b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effaf59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effb0156f40 0x7effb016b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59768/0 (socket says 172.21.15.92:59768) 2023-10-06T10:38:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effaf59e700 1 -- 172.21.15.92:0/1195652796 learned_addr learned my addr 172.21.15.92:0/1195652796 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effaed9d700 1 --2- 172.21.15.92:0/1195652796 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effb006ac40 0x7effb013f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effa659c700 1 --2- 172.21.15.92:0/1195652796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb0156b70 0x7effb0167480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effaf59e700 1 -- 172.21.15.92:0/1195652796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb0156b70 msgr2=0x7effb0167480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effaf59e700 1 --2- 172.21.15.92:0/1195652796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb0156b70 0x7effb0167480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effaf59e700 1 -- 172.21.15.92:0/1195652796 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effb006ac40 msgr2=0x7effb013f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effaf59e700 1 --2- 172.21.15.92:0/1195652796 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effb006ac40 0x7effb013f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effaf59e700 1 -- 172.21.15.92:0/1195652796 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa00094f0 con 0x7effb0156f40 2023-10-06T10:38:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effaed9d700 1 --2- 172.21.15.92:0/1195652796 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effb006ac40 0x7effb013f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effa659c700 1 --2- 172.21.15.92:0/1195652796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb0156b70 0x7effb0167480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effaf59e700 1 --2- 172.21.15.92:0/1195652796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effb0156f40 0x7effb016b2a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7effa800bb60 tx=0x7effa8009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effa77fe700 1 -- 172.21.15.92:0/1195652796 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa800c940 con 0x7effb0156f40 2023-10-06T10:38:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effa77fe700 1 -- 172.21.15.92:0/1195652796 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa800cac0 con 0x7effb0156f40 2023-10-06T10:38:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effb50b1700 1 -- 172.21.15.92:0/1195652796 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb0168340 con 0x7effb0156f40 2023-10-06T10:38:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effa77fe700 1 -- 172.21.15.92:0/1195652796 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa800ce80 con 0x7effb0156f40 2023-10-06T10:38:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.527+0000 7effb50b1700 1 -- 172.21.15.92:0/1195652796 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb013fcd0 con 0x7effb0156f40 2023-10-06T10:38:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.528+0000 7effb50b1700 1 -- 172.21.15.92:0/1195652796 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb0064a90 con 0x7effb0156f40 2023-10-06T10:38:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.529+0000 7effa77fe700 1 -- 172.21.15.92:0/1195652796 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7effa8014620 con 0x7effb0156f40 2023-10-06T10:38:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.529+0000 7effa77fe700 1 --2- 172.21.15.92:0/1195652796 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7eff90066f60 0x7eff90069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.529+0000 7effa77fe700 1 -- 172.21.15.92:0/1195652796 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6777+0+0 (secure 0 0 0) 0x7effa8084980 con 0x7effb0156f40 2023-10-06T10:38:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.529+0000 7effaed9d700 1 --2- 172.21.15.92:0/1195652796 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7eff90066f60 0x7eff90069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.530+0000 7effaed9d700 1 --2- 172.21.15.92:0/1195652796 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7eff90066f60 0x7eff90069410 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7effa00099b0 tx=0x7effa0016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:21.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.533+0000 7effa77fe700 1 -- 172.21.15.92:0/1195652796 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7effa804c5a0 con 0x7effb0156f40 2023-10-06T10:38:21.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:21.758+0000 7effb50b1700 1 -- 172.21.15.92:0/1195652796 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7effb00c90b0 con 0x7effb0156f40 2023-10-06T10:38:22.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.388+0000 7effa77fe700 1 -- 172.21.15.92:0/1195652796 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v6) v1 ==== 137+0+0 (secure 0 0 0) 0x7effa80517c0 con 0x7effb0156f40 2023-10-06T10:38:22.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.391+0000 7effb50b1700 1 -- 172.21.15.92:0/1195652796 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7eff90066f60 msgr2=0x7eff90069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:22.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.391+0000 7effb50b1700 1 --2- 172.21.15.92:0/1195652796 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7eff90066f60 0x7eff90069410 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7effa00099b0 tx=0x7effa0016000 comp rx=0 tx=0).stop 2023-10-06T10:38:22.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.391+0000 7effb50b1700 1 -- 172.21.15.92:0/1195652796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effb0156f40 msgr2=0x7effb016b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:22.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.391+0000 7effb50b1700 1 --2- 172.21.15.92:0/1195652796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effb0156f40 0x7effb016b2a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7effa800bb60 tx=0x7effa8009810 comp rx=0 tx=0).stop 2023-10-06T10:38:22.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.391+0000 7effb50b1700 1 -- 172.21.15.92:0/1195652796 shutdown_connections 2023-10-06T10:38:22.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.391+0000 7effb50b1700 1 --2- 172.21.15.92:0/1195652796 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7eff90066f60 0x7eff90069410 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:22.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.391+0000 7effb50b1700 1 --2- 172.21.15.92:0/1195652796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effb0156f40 0x7effb016b2a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:22.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.391+0000 7effb50b1700 1 --2- 172.21.15.92:0/1195652796 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effb006ac40 0x7effb013f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:22.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.391+0000 7effb50b1700 1 --2- 172.21.15.92:0/1195652796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb0156b70 0x7effb0167480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:22.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.391+0000 7effb50b1700 1 -- 172.21.15.92:0/1195652796 >> 172.21.15.92:0/1195652796 conn(0x7effb00cd1a0 msgr2=0x7effb00c1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:22.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.391+0000 7effb50b1700 1 -- 172.21.15.92:0/1195652796 shutdown_connections 2023-10-06T10:38:22.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.391+0000 7effb50b1700 1 -- 172.21.15.92:0/1195652796 wait complete. 2023-10-06T10:38:22.395 INFO:teuthology.orchestra.run.smithi092.stderr:added data pool 5 to fsmap 2023-10-06T10:38:22.406 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:38:22.430 INFO:tasks.ceph.mds.d.smithi124.stderr:2023-10-06T10:38:22.429+0000 7f1c17b7e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T10:38:22.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.523+0000 7fcc9a411700 1 Processor -- start 2023-10-06T10:38:22.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.523+0000 7fcc9a411700 1 -- start start 2023-10-06T10:38:22.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.524+0000 7fcc9a411700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcc94159440 0x7fcc940b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:22.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.524+0000 7fcc9a411700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc940b35d0 0x7fcc940cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:22.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.524+0000 7fcc9a411700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc940cb7a0 con 0x7fcc940cc170 2023-10-06T10:38:22.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.524+0000 7fcc9a411700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc940b0860 con 0x7fcc94159440 2023-10-06T10:38:22.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.524+0000 7fcc9a411700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc940cb150 con 0x7fcc940b35d0 2023-10-06T10:38:22.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.524+0000 7fcc9940f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcc94159440 0x7fcc940b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:22.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.524+0000 7fcc9940f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcc94159440 0x7fcc940b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59780/0 (socket says 172.21.15.92:59780) 2023-10-06T10:38:22.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.524+0000 7fcc9940f700 1 -- 172.21.15.92:0/4011490780 learned_addr learned my addr 172.21.15.92:0/4011490780 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:22.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.524+0000 7fcc98c0e700 1 --2- 172.21.15.92:0/4011490780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc940b35d0 0x7fcc940cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:22.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.524+0000 7fcc9940f700 1 -- 172.21.15.92:0/4011490780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc940b35d0 msgr2=0x7fcc940cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:22.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.524+0000 7fcc9940f700 1 --2- 172.21.15.92:0/4011490780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc940b35d0 0x7fcc940cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:22.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.524+0000 7fcc9940f700 1 -- 172.21.15.92:0/4011490780 >> v1:172.21.15.124:6789/0 conn(0x7fcc940cc170 legacy=0x7fcc940b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:22.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.524+0000 7fcc9940f700 1 -- 172.21.15.92:0/4011490780 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc940cb430 con 0x7fcc94159440 2023-10-06T10:38:22.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.524+0000 7fcc9940f700 1 --2- 172.21.15.92:0/4011490780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc94159440 0x7fcc940b3090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0099b0 tx=0x7fcc8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96c8ae6a4782dc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:22.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.524+0000 7fcc93fff700 1 -- 172.21.15.92:0/4011490780 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c00b820 con 0x7fcc94159440 2023-10-06T10:38:22.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.525+0000 7fcc93fff700 1 -- 172.21.15.92:0/4011490780 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8c00b980 con 0x7fcc94159440 2023-10-06T10:38:22.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.525+0000 7fcc93fff700 1 -- 172.21.15.92:0/4011490780 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c00bd20 con 0x7fcc94159440 2023-10-06T10:38:22.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.525+0000 7fcc9a411700 1 -- 172.21.15.92:0/4011490780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc94159440 msgr2=0x7fcc940b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:22.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.525+0000 7fcc9a411700 1 --2- 172.21.15.92:0/4011490780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc94159440 0x7fcc940b3090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0099b0 tx=0x7fcc8c009250 comp rx=0 tx=0).stop 2023-10-06T10:38:22.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.525+0000 7fcc9a411700 1 -- 172.21.15.92:0/4011490780 shutdown_connections 2023-10-06T10:38:22.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.525+0000 7fcc9a411700 1 --2- 172.21.15.92:0/4011490780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc94159440 0x7fcc940b3090 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:22.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.525+0000 7fcc9a411700 1 --2- 172.21.15.92:0/4011490780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc940b35d0 0x7fcc940cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:22.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.525+0000 7fcc9a411700 1 -- 172.21.15.92:0/4011490780 >> 172.21.15.92:0/4011490780 conn(0x7fcc9406ece0 msgr2=0x7fcc9406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:22.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.525+0000 7fcc9a411700 1 -- 172.21.15.92:0/4011490780 shutdown_connections 2023-10-06T10:38:22.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.525+0000 7fcc9a411700 1 -- 172.21.15.92:0/4011490780 wait complete. 2023-10-06T10:38:22.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.526+0000 7fcc9a411700 1 Processor -- start 2023-10-06T10:38:22.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.526+0000 7fcc9a411700 1 -- start start 2023-10-06T10:38:22.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.527+0000 7fcc9a411700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc940b35d0 0x7fcc94159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:22.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.527+0000 7fcc9a411700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc940cc170 0x7fcc94154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:22.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.527+0000 7fcc9a411700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc94159440 0x7fcc941548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:22.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.527+0000 7fcc9a411700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc94154e10 con 0x7fcc940cc170 2023-10-06T10:38:22.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.527+0000 7fcc9a411700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc940c9c10 con 0x7fcc94159440 2023-10-06T10:38:22.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.527+0000 7fcc9a411700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9415cd70 con 0x7fcc940b35d0 2023-10-06T10:38:22.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.527+0000 7fcc98c0e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc940cc170 0x7fcc94154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:22.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.527+0000 7fcc98c0e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc940cc170 0x7fcc94154390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59792/0 (socket says 172.21.15.92:59792) 2023-10-06T10:38:22.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.527+0000 7fcc98c0e700 1 -- 172.21.15.92:0/312744684 learned_addr learned my addr 172.21.15.92:0/312744684 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:22.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.527+0000 7fcc9940f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc940b35d0 0x7fcc94159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:22.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.527+0000 7fcc99c10700 1 --2- 172.21.15.92:0/312744684 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc94159440 0x7fcc941548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:22.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.528+0000 7fcc98c0e700 1 -- 172.21.15.92:0/312744684 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc940b35d0 msgr2=0x7fcc94159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:22.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.528+0000 7fcc98c0e700 1 --2- 172.21.15.92:0/312744684 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc940b35d0 0x7fcc94159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:22.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.528+0000 7fcc98c0e700 1 -- 172.21.15.92:0/312744684 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc94159440 msgr2=0x7fcc941548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:22.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.528+0000 7fcc98c0e700 1 --2- 172.21.15.92:0/312744684 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc94159440 0x7fcc941548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:22.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.528+0000 7fcc98c0e700 1 -- 172.21.15.92:0/312744684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8c0094f0 con 0x7fcc940cc170 2023-10-06T10:38:22.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.528+0000 7fcc98c0e700 1 --2- 172.21.15.92:0/312744684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc940cc170 0x7fcc94154390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800c7d0 tx=0x7fcc88008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:22.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.528+0000 7fcc917fa700 1 -- 172.21.15.92:0/312744684 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88011600 con 0x7fcc940cc170 2023-10-06T10:38:22.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.528+0000 7fcc917fa700 1 -- 172.21.15.92:0/312744684 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc88011760 con 0x7fcc940cc170 2023-10-06T10:38:22.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.528+0000 7fcc9a411700 1 -- 172.21.15.92:0/312744684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc940c8e50 con 0x7fcc940cc170 2023-10-06T10:38:22.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.528+0000 7fcc917fa700 1 -- 172.21.15.92:0/312744684 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88011b20 con 0x7fcc940cc170 2023-10-06T10:38:22.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.528+0000 7fcc9a411700 1 -- 172.21.15.92:0/312744684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc940c9410 con 0x7fcc940cc170 2023-10-06T10:38:22.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.529+0000 7fcc9a411700 1 -- 172.21.15.92:0/312744684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc9405f230 con 0x7fcc940cc170 2023-10-06T10:38:22.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.529+0000 7fcc917fa700 1 -- 172.21.15.92:0/312744684 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcc880193f0 con 0x7fcc940cc170 2023-10-06T10:38:22.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.529+0000 7fcc917fa700 1 --2- 172.21.15.92:0/312744684 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcc80066f30 0x7fcc800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:22.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.530+0000 7fcc9940f700 1 --2- 172.21.15.92:0/312744684 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcc80066f30 0x7fcc800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:22.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.530+0000 7fcc917fa700 1 -- 172.21.15.92:0/312744684 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fcc88084c20 con 0x7fcc940cc170 2023-10-06T10:38:22.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.530+0000 7fcc9940f700 1 --2- 172.21.15.92:0/312744684 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcc80066f30 0x7fcc800693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0099b0 tx=0x7fcc8c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:22.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.534+0000 7fcc917fa700 1 -- 172.21.15.92:0/312744684 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc88051a40 con 0x7fcc940cc170 2023-10-06T10:38:22.598 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:38:22.600 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:38:22.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.764+0000 7fcc9a411700 1 -- 172.21.15.92:0/312744684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc940b5610 con 0x7fcc940cc170 2023-10-06T10:38:22.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.765+0000 7fcc917fa700 1 -- 172.21.15.92:0/312744684 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4599 (secure 0 0 0) 0x7fcc88055060 con 0x7fcc940cc170 2023-10-06T10:38:22.767 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:22.767 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2303424485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2303424485},{"type":"v1","addr":"172.21.15.92:6836","nonce":2303424485}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1027343065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1027343065},{"type":"v1","addr":"172.21.15.92:6837","nonce":1027343065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2667820379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2667820379},{"type":"v1","addr":"172.21.15.124:6834","nonce":2667820379}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:38:16.725144+0000","modified":"2023-10-06T10:38:22.380999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/1919704285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":1919704285},{"type":"v1","addr":"172.21.15.124:6835","nonce":1919704285}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2023-10-06T10:38:22.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.767+0000 7fcc9a411700 1 -- 172.21.15.92:0/312744684 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcc80066f30 msgr2=0x7fcc800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:22.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.767+0000 7fcc9a411700 1 --2- 172.21.15.92:0/312744684 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcc80066f30 0x7fcc800693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0099b0 tx=0x7fcc8c016000 comp rx=0 tx=0).stop 2023-10-06T10:38:22.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.767+0000 7fcc9a411700 1 -- 172.21.15.92:0/312744684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc940cc170 msgr2=0x7fcc94154390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:22.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.767+0000 7fcc9a411700 1 --2- 172.21.15.92:0/312744684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc940cc170 0x7fcc94154390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800c7d0 tx=0x7fcc88008040 comp rx=0 tx=0).stop 2023-10-06T10:38:22.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.767+0000 7fcc9a411700 1 -- 172.21.15.92:0/312744684 shutdown_connections 2023-10-06T10:38:22.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.767+0000 7fcc9a411700 1 --2- 172.21.15.92:0/312744684 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcc80066f30 0x7fcc800693e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:22.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.767+0000 7fcc9a411700 1 --2- 172.21.15.92:0/312744684 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc94159440 0x7fcc941548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:22.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.767+0000 7fcc9a411700 1 --2- 172.21.15.92:0/312744684 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc940b35d0 0x7fcc94159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:22.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.767+0000 7fcc9a411700 1 --2- 172.21.15.92:0/312744684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc940cc170 0x7fcc94154390 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:22.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.767+0000 7fcc9a411700 1 -- 172.21.15.92:0/312744684 >> 172.21.15.92:0/312744684 conn(0x7fcc9406ece0 msgr2=0x7fcc940b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:22.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.767+0000 7fcc9a411700 1 -- 172.21.15.92:0/312744684 shutdown_connections 2023-10-06T10:38:22.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.767+0000 7fcc9a411700 1 -- 172.21.15.92:0/312744684 wait complete. 2023-10-06T10:38:22.770 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 6 2023-10-06T10:38:22.781 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:38:22.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.892+0000 7f3f2b595700 1 Processor -- start 2023-10-06T10:38:22.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.892+0000 7f3f2b595700 1 -- start start 2023-10-06T10:38:22.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.893+0000 7f3f2b595700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3f24159390 0x7f3f240ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:22.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.893+0000 7f3f2b595700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3f24159760 0x7f3f240aad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:22.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.893+0000 7f3f2b595700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f240686c0 con 0x7f3f240ab240 2023-10-06T10:38:22.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.893+0000 7f3f2b595700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f24068c80 con 0x7f3f24159760 2023-10-06T10:38:22.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.893+0000 7f3f2b595700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f240ab660 con 0x7f3f24159390 2023-10-06T10:38:22.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.893+0000 7f3f29d92700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3f24159760 0x7f3f240aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:22.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.893+0000 7f3f2ad94700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3f240ab240 0x7f3f24069110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38358/0 (socket says 172.21.15.92:38358) 2023-10-06T10:38:22.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.893+0000 7f3f2ad94700 1 -- 172.21.15.92:0/1040708732 learned_addr learned my addr 172.21.15.92:0/1040708732 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:22.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.893+0000 7f3f29d92700 1 --2- 172.21.15.92:0/1040708732 >> v2:172.21.15.92:3300/0 conn(0x7f3f24159760 0x7f3f240aad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59808/0 (socket says 172.21.15.92:59808) 2023-10-06T10:38:22.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.893+0000 7f3f2a593700 1 --2- 172.21.15.92:0/1040708732 >> v2:172.21.15.124:3300/0 conn(0x7f3f24159390 0x7f3f240ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:22.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.893+0000 7f3f29d92700 1 -- 172.21.15.92:0/1040708732 >> v2:172.21.15.124:3300/0 conn(0x7f3f24159390 msgr2=0x7f3f240ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:22.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.893+0000 7f3f29d92700 1 --2- 172.21.15.92:0/1040708732 >> v2:172.21.15.124:3300/0 conn(0x7f3f24159390 0x7f3f240ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:22.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.893+0000 7f3f29d92700 1 -- 172.21.15.92:0/1040708732 >> v1:172.21.15.92:6789/0 conn(0x7f3f240ab240 legacy=0x7f3f24069110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:22.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.893+0000 7f3f29d92700 1 -- 172.21.15.92:0/1040708732 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f240ace20 con 0x7f3f24159760 2023-10-06T10:38:22.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.893+0000 7f3f29591700 1 -- 172.21.15.92:0/1040708732 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 258848021 0 0) 0x7f3f240686c0 con 0x7f3f240ab240 2023-10-06T10:38:22.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.894+0000 7f3f29d92700 1 --2- 172.21.15.92:0/1040708732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f24159760 0x7f3f240aad00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3f2406eb90 tx=0x7f3f140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed50767bc5e737c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:22.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.894+0000 7f3f29591700 1 -- 172.21.15.92:0/1040708732 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f14011630 con 0x7f3f24159760 2023-10-06T10:38:22.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.894+0000 7f3f29591700 1 -- 172.21.15.92:0/1040708732 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f14011790 con 0x7f3f24159760 2023-10-06T10:38:22.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.894+0000 7f3f29591700 1 -- 172.21.15.92:0/1040708732 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f14011b30 con 0x7f3f24159760 2023-10-06T10:38:22.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.894+0000 7f3f2b595700 1 -- 172.21.15.92:0/1040708732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f24159760 msgr2=0x7f3f240aad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:22.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.894+0000 7f3f2b595700 1 --2- 172.21.15.92:0/1040708732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f24159760 0x7f3f240aad00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3f2406eb90 tx=0x7f3f140091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:22.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.894+0000 7f3f2b595700 1 -- 172.21.15.92:0/1040708732 shutdown_connections 2023-10-06T10:38:22.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.894+0000 7f3f2b595700 1 --2- 172.21.15.92:0/1040708732 >> v2:172.21.15.124:3300/0 conn(0x7f3f24159390 0x7f3f240ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:22.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.894+0000 7f3f2b595700 1 --2- 172.21.15.92:0/1040708732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f24159760 0x7f3f240aad00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:22.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.894+0000 7f3f2b595700 1 -- 172.21.15.92:0/1040708732 >> 172.21.15.92:0/1040708732 conn(0x7f3f24097410 msgr2=0x7f3f24067d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:22.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.894+0000 7f3f2b595700 1 -- 172.21.15.92:0/1040708732 shutdown_connections 2023-10-06T10:38:22.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.894+0000 7f3f2b595700 1 -- 172.21.15.92:0/1040708732 wait complete. 2023-10-06T10:38:22.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.895+0000 7f3f2b595700 1 Processor -- start 2023-10-06T10:38:22.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.895+0000 7f3f2b595700 1 -- start start 2023-10-06T10:38:22.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.896+0000 7f3f2b595700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f240ab240 0x7f3f2409bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:22.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.896+0000 7f3f2b595700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f24159390 0x7f3f24148b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:22.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.896+0000 7f3f2b595700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f24159760 0x7f3f2409a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:22.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.896+0000 7f3f2b595700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2409b680 con 0x7f3f240ab240 2023-10-06T10:38:22.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.896+0000 7f3f2b595700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2409aa70 con 0x7f3f24159760 2023-10-06T10:38:22.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.896+0000 7f3f2b595700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2409ad50 con 0x7f3f24159390 2023-10-06T10:38:22.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.896+0000 7f3f2a593700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f240ab240 0x7f3f2409bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:22.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.896+0000 7f3f2a593700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f240ab240 0x7f3f2409bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59812/0 (socket says 172.21.15.92:59812) 2023-10-06T10:38:22.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.896+0000 7f3f2a593700 1 -- 172.21.15.92:0/2572666653 learned_addr learned my addr 172.21.15.92:0/2572666653 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:22.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.896+0000 7f3f2a593700 1 -- 172.21.15.92:0/2572666653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f24159390 msgr2=0x7f3f24148b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:22.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.896+0000 7f3f2a593700 1 --2- 172.21.15.92:0/2572666653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f24159390 0x7f3f24148b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:22.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.896+0000 7f3f2a593700 1 -- 172.21.15.92:0/2572666653 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f24159760 msgr2=0x7f3f2409a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:22.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.896+0000 7f3f2a593700 1 --2- 172.21.15.92:0/2572666653 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f24159760 0x7f3f2409a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:22.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.896+0000 7f3f2a593700 1 -- 172.21.15.92:0/2572666653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f14009420 con 0x7f3f240ab240 2023-10-06T10:38:22.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.897+0000 7f3f2a593700 1 --2- 172.21.15.92:0/2572666653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f240ab240 0x7f3f2409bb10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3f2000ec80 tx=0x7f3f200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:22.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.897+0000 7f3f1a7fc700 1 -- 172.21.15.92:0/2572666653 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2000c9d0 con 0x7f3f240ab240 2023-10-06T10:38:22.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.897+0000 7f3f1a7fc700 1 -- 172.21.15.92:0/2572666653 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2000cb50 con 0x7f3f240ab240 2023-10-06T10:38:22.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.897+0000 7f3f2b595700 1 -- 172.21.15.92:0/2572666653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f2409b030 con 0x7f3f240ab240 2023-10-06T10:38:22.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.897+0000 7f3f1a7fc700 1 -- 172.21.15.92:0/2572666653 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f20013470 con 0x7f3f240ab240 2023-10-06T10:38:22.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.897+0000 7f3f2b595700 1 -- 172.21.15.92:0/2572666653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f24149040 con 0x7f3f240ab240 2023-10-06T10:38:22.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.898+0000 7f3f2b595700 1 -- 172.21.15.92:0/2572666653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f2405f230 con 0x7f3f240ab240 2023-10-06T10:38:22.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.898+0000 7f3f1a7fc700 1 -- 172.21.15.92:0/2572666653 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3f20013860 con 0x7f3f240ab240 2023-10-06T10:38:22.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.898+0000 7f3f1a7fc700 1 --2- 172.21.15.92:0/2572666653 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3f08066f30 0x7f3f080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:22.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.899+0000 7f3f1a7fc700 1 -- 172.21.15.92:0/2572666653 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f3f20084d20 con 0x7f3f240ab240 2023-10-06T10:38:22.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.899+0000 7f3f29d92700 1 --2- 172.21.15.92:0/2572666653 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3f08066f30 0x7f3f080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:22.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.899+0000 7f3f29d92700 1 --2- 172.21.15.92:0/2572666653 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3f08066f30 0x7f3f080693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f140091e0 tx=0x7f3f14017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:22.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:22.903+0000 7f3f1a7fc700 1 -- 172.21.15.92:0/2572666653 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f20051b40 con 0x7f3f240ab240 2023-10-06T10:38:23.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.083+0000 7f3f2b595700 1 -- 172.21.15.92:0/2572666653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f24157510 con 0x7f3f240ab240 2023-10-06T10:38:23.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.084+0000 7f3f1a7fc700 1 -- 172.21.15.92:0/2572666653 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18707 (secure 0 0 0) 0x7f3f20055160 con 0x7f3f240ab240 2023-10-06T10:38:23.086 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:23.086 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:21.760233+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:38:14.368713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T10:38:15.692344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T10:38:19.382553+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:38:23.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.086+0000 7f3f2b595700 1 -- 172.21.15.92:0/2572666653 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3f08066f30 msgr2=0x7f3f080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:23.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.086+0000 7f3f2b595700 1 --2- 172.21.15.92:0/2572666653 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3f08066f30 0x7f3f080693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f140091e0 tx=0x7f3f14017000 comp rx=0 tx=0).stop 2023-10-06T10:38:23.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.086+0000 7f3f2b595700 1 -- 172.21.15.92:0/2572666653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f240ab240 msgr2=0x7f3f2409bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:23.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.086+0000 7f3f2b595700 1 --2- 172.21.15.92:0/2572666653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f240ab240 0x7f3f2409bb10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3f2000ec80 tx=0x7f3f200091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.086+0000 7f3f2b595700 1 -- 172.21.15.92:0/2572666653 shutdown_connections 2023-10-06T10:38:23.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.086+0000 7f3f2b595700 1 --2- 172.21.15.92:0/2572666653 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3f08066f30 0x7f3f080693e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.086+0000 7f3f2b595700 1 --2- 172.21.15.92:0/2572666653 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f24159760 0x7f3f2409a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.086+0000 7f3f2b595700 1 --2- 172.21.15.92:0/2572666653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f240ab240 0x7f3f2409bb10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.086+0000 7f3f2b595700 1 --2- 172.21.15.92:0/2572666653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f24159390 0x7f3f24148b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.086+0000 7f3f2b595700 1 -- 172.21.15.92:0/2572666653 >> 172.21.15.92:0/2572666653 conn(0x7f3f24097410 msgr2=0x7f3f24157ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:23.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.086+0000 7f3f2b595700 1 -- 172.21.15.92:0/2572666653 shutdown_connections 2023-10-06T10:38:23.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.086+0000 7f3f2b595700 1 -- 172.21.15.92:0/2572666653 wait complete. 2023-10-06T10:38:23.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:38:23.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.212+0000 7f32bcb41700 1 Processor -- start 2023-10-06T10:38:23.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.212+0000 7f32bcb41700 1 -- start start 2023-10-06T10:38:23.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.213+0000 7f32bcb41700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f32b80b49a0 0x7f32b80aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:23.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.213+0000 7f32bcb41700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f32b80b0760 0x7f32b80b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:23.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.213+0000 7f32bcb41700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b80b0c60 con 0x7f32b8159560 2023-10-06T10:38:23.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.213+0000 7f32bcb41700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b80b0f40 con 0x7f32b80b49a0 2023-10-06T10:38:23.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.213+0000 7f32bcb41700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b80b1220 con 0x7f32b80b0760 2023-10-06T10:38:23.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.213+0000 7f32b5d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f32b80b49a0 0x7f32b80aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:23.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.213+0000 7f32b5d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f32b80b49a0 0x7f32b80aac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59818/0 (socket says 172.21.15.92:59818) 2023-10-06T10:38:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.213+0000 7f32b5d9b700 1 -- 172.21.15.92:0/1928034891 learned_addr learned my addr 172.21.15.92:0/1928034891 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.213+0000 7f32b6d9d700 1 --2- 172.21.15.92:0/1928034891 >> v2:172.21.15.124:3300/0 conn(0x7f32b80b0760 0x7f32b80b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.213+0000 7f32b5d9b700 1 -- 172.21.15.92:0/1928034891 >> v2:172.21.15.124:3300/0 conn(0x7f32b80b0760 msgr2=0x7f32b80b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.213+0000 7f32b5d9b700 1 --2- 172.21.15.92:0/1928034891 >> v2:172.21.15.124:3300/0 conn(0x7f32b80b0760 0x7f32b80b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.213+0000 7f32b5d9b700 1 -- 172.21.15.92:0/1928034891 >> v1:172.21.15.124:6789/0 conn(0x7f32b8159560 legacy=0x7f32b80aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.213+0000 7f32b5d9b700 1 -- 172.21.15.92:0/1928034891 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b80b1500 con 0x7f32b80b49a0 2023-10-06T10:38:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.213+0000 7f32b5d9b700 1 --2- 172.21.15.92:0/1928034891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32b80b49a0 0x7f32b80aac40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0099b0 tx=0x7f32ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a4ae135eafa67b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.213+0000 7f32b559a700 1 -- 172.21.15.92:0/1928034891 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac00b820 con 0x7f32b80b49a0 2023-10-06T10:38:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.213+0000 7f32b559a700 1 -- 172.21.15.92:0/1928034891 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac00b980 con 0x7f32b80b49a0 2023-10-06T10:38:23.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.213+0000 7f32b559a700 1 -- 172.21.15.92:0/1928034891 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac00bd20 con 0x7f32b80b49a0 2023-10-06T10:38:23.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.214+0000 7f32bcb41700 1 -- 172.21.15.92:0/1928034891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32b80b49a0 msgr2=0x7f32b80aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:23.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.214+0000 7f32bcb41700 1 --2- 172.21.15.92:0/1928034891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32b80b49a0 0x7f32b80aac40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0099b0 tx=0x7f32ac009250 comp rx=0 tx=0).stop 2023-10-06T10:38:23.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.214+0000 7f32bcb41700 1 -- 172.21.15.92:0/1928034891 shutdown_connections 2023-10-06T10:38:23.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.214+0000 7f32bcb41700 1 --2- 172.21.15.92:0/1928034891 >> v2:172.21.15.124:3300/0 conn(0x7f32b80b0760 0x7f32b80b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.214+0000 7f32bcb41700 1 --2- 172.21.15.92:0/1928034891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32b80b49a0 0x7f32b80aac40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.214+0000 7f32bcb41700 1 -- 172.21.15.92:0/1928034891 >> 172.21.15.92:0/1928034891 conn(0x7f32b80997f0 msgr2=0x7f32b806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:23.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.214+0000 7f32bcb41700 1 -- 172.21.15.92:0/1928034891 shutdown_connections 2023-10-06T10:38:23.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.214+0000 7f32bcb41700 1 -- 172.21.15.92:0/1928034891 wait complete. 2023-10-06T10:38:23.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.215+0000 7f32bcb41700 1 Processor -- start 2023-10-06T10:38:23.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.215+0000 7f32bcb41700 1 -- start start 2023-10-06T10:38:23.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.216+0000 7f32bcb41700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32b80b0760 0x7f32b8157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:23.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.216+0000 7f32bcb41700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32b80b49a0 0x7f32b809c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:23.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.216+0000 7f32bcb41700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32b8159560 0x7f32b80cec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:23.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.216+0000 7f32bcb41700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b8158280 con 0x7f32b8159560 2023-10-06T10:38:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.216+0000 7f32bcb41700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b80abea0 con 0x7f32b80b0760 2023-10-06T10:38:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.216+0000 7f32bcb41700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b80ac180 con 0x7f32b80b49a0 2023-10-06T10:38:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.216+0000 7f32b6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32b8159560 0x7f32b80cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.216+0000 7f32b6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32b8159560 0x7f32b80cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59834/0 (socket says 172.21.15.92:59834) 2023-10-06T10:38:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.216+0000 7f32b6d9d700 1 -- 172.21.15.92:0/942415165 learned_addr learned my addr 172.21.15.92:0/942415165 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.216+0000 7f32b6d9d700 1 -- 172.21.15.92:0/942415165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32b80b49a0 msgr2=0x7f32b809c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.216+0000 7f32b6d9d700 1 --2- 172.21.15.92:0/942415165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32b80b49a0 0x7f32b809c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.216+0000 7f32b6d9d700 1 -- 172.21.15.92:0/942415165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32b80b0760 msgr2=0x7f32b8157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.216+0000 7f32b659c700 1 --2- 172.21.15.92:0/942415165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32b80b0760 0x7f32b8157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.216+0000 7f32b6d9d700 1 --2- 172.21.15.92:0/942415165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32b80b0760 0x7f32b8157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.216+0000 7f32b6d9d700 1 -- 172.21.15.92:0/942415165 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac0094f0 con 0x7f32b8159560 2023-10-06T10:38:23.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.216+0000 7f32b6d9d700 1 --2- 172.21.15.92:0/942415165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32b8159560 0x7f32b80cec60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009fa0 tx=0x7f32b0009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:23.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.216+0000 7f32a67fc700 1 -- 172.21.15.92:0/942415165 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b000ed20 con 0x7f32b8159560 2023-10-06T10:38:23.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.216+0000 7f32a67fc700 1 -- 172.21.15.92:0/942415165 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b000ee80 con 0x7f32b8159560 2023-10-06T10:38:23.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.216+0000 7f32bcb41700 1 -- 172.21.15.92:0/942415165 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b80ad6f0 con 0x7f32b8159560 2023-10-06T10:38:23.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.216+0000 7f32a67fc700 1 -- 172.21.15.92:0/942415165 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b0014670 con 0x7f32b8159560 2023-10-06T10:38:23.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.216+0000 7f32bcb41700 1 -- 172.21.15.92:0/942415165 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b809cc80 con 0x7f32b8159560 2023-10-06T10:38:23.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.217+0000 7f32bcb41700 1 -- 172.21.15.92:0/942415165 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b80adc10 con 0x7f32b8159560 2023-10-06T10:38:23.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.218+0000 7f32a67fc700 1 -- 172.21.15.92:0/942415165 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32b00149e0 con 0x7f32b8159560 2023-10-06T10:38:23.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.218+0000 7f32a67fc700 1 --2- 172.21.15.92:0/942415165 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3298066f30 0x7f32980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:23.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.218+0000 7f32a67fc700 1 -- 172.21.15.92:0/942415165 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f32b0085210 con 0x7f32b8159560 2023-10-06T10:38:23.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.218+0000 7f32b659c700 1 --2- 172.21.15.92:0/942415165 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3298066f30 0x7f32980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:23.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.219+0000 7f32b659c700 1 --2- 172.21.15.92:0/942415165 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3298066f30 0x7f32980693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f32a8001710 tx=0x7f32a8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:23.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.222+0000 7f32a67fc700 1 -- 172.21.15.92:0/942415165 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32b80adc10 con 0x7f32b8159560 2023-10-06T10:38:23.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.410+0000 7f32bcb41700 1 -- 172.21.15.92:0/942415165 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32b80adc10 con 0x7f32b8159560 2023-10-06T10:38:23.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.411+0000 7f32a67fc700 1 -- 172.21.15.92:0/942415165 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18707 (secure 0 0 0) 0x7f32b80adc10 con 0x7f32b8159560 2023-10-06T10:38:23.413 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:23.413 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:21.760233+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:38:14.368713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T10:38:15.692344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T10:38:19.382553+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:38:23.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.413+0000 7f32bcb41700 1 -- 172.21.15.92:0/942415165 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3298066f30 msgr2=0x7f32980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:23.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.413+0000 7f32bcb41700 1 --2- 172.21.15.92:0/942415165 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3298066f30 0x7f32980693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f32a8001710 tx=0x7f32a8009040 comp rx=0 tx=0).stop 2023-10-06T10:38:23.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.413+0000 7f32bcb41700 1 -- 172.21.15.92:0/942415165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32b8159560 msgr2=0x7f32b80cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:23.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.413+0000 7f32bcb41700 1 --2- 172.21.15.92:0/942415165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32b8159560 0x7f32b80cec60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009fa0 tx=0x7f32b0009900 comp rx=0 tx=0).stop 2023-10-06T10:38:23.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.413+0000 7f32bcb41700 1 -- 172.21.15.92:0/942415165 shutdown_connections 2023-10-06T10:38:23.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.413+0000 7f32bcb41700 1 --2- 172.21.15.92:0/942415165 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3298066f30 0x7f32980693e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.413+0000 7f32bcb41700 1 --2- 172.21.15.92:0/942415165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32b8159560 0x7f32b80cec60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.413+0000 7f32bcb41700 1 --2- 172.21.15.92:0/942415165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32b80b0760 0x7f32b8157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.413+0000 7f32bcb41700 1 --2- 172.21.15.92:0/942415165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32b80b49a0 0x7f32b809c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.413+0000 7f32bcb41700 1 -- 172.21.15.92:0/942415165 >> 172.21.15.92:0/942415165 conn(0x7f32b80997f0 msgr2=0x7f32b80699f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:23.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.413+0000 7f32bcb41700 1 -- 172.21.15.92:0/942415165 shutdown_connections 2023-10-06T10:38:23.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.413+0000 7f32bcb41700 1 -- 172.21.15.92:0/942415165 wait complete. 2023-10-06T10:38:23.439 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:38:23.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.550+0000 7fc474996700 1 Processor -- start 2023-10-06T10:38:23.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.550+0000 7fc474996700 1 -- start start 2023-10-06T10:38:23.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.551+0000 7fc474996700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc4700cb5f0 0x7fc4700691a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:23.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.551+0000 7fc474996700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc4700696e0 0x7fc470159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:23.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.551+0000 7fc474996700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc47015a010 0x7fc4700ca670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:23.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.551+0000 7fc474996700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc470154020 con 0x7fc4700696e0 2023-10-06T10:38:23.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.551+0000 7fc474996700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc47015a450 con 0x7fc4700cb5f0 2023-10-06T10:38:23.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.551+0000 7fc474996700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc470068d10 con 0x7fc47015a010 2023-10-06T10:38:23.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.551+0000 7fc465d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc4700696e0 0x7fc470159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:23.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.551+0000 7fc465d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc4700696e0 0x7fc470159ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37702/0 (socket says 172.21.15.92:37702) 2023-10-06T10:38:23.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.551+0000 7fc465d9b700 1 -- 172.21.15.92:0/385086592 learned_addr learned my addr 172.21.15.92:0/385086592 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:23.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.552+0000 7fc46ed9d700 1 --2- 172.21.15.92:0/385086592 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc47015a010 0x7fc4700ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:23.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.552+0000 7fc46e59c700 1 --2- 172.21.15.92:0/385086592 >> v2:172.21.15.124:3300/0 conn(0x7fc4700cb5f0 0x7fc4700691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:23.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.552+0000 7fc465d9b700 1 -- 172.21.15.92:0/385086592 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc47015a010 msgr2=0x7fc4700ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:23.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.552+0000 7fc465d9b700 1 --2- 172.21.15.92:0/385086592 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc47015a010 0x7fc4700ca670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.552+0000 7fc465d9b700 1 -- 172.21.15.92:0/385086592 >> v2:172.21.15.124:3300/0 conn(0x7fc4700cb5f0 msgr2=0x7fc4700691a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:23.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.552+0000 7fc465d9b700 1 --2- 172.21.15.92:0/385086592 >> v2:172.21.15.124:3300/0 conn(0x7fc4700cb5f0 0x7fc4700691a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.552+0000 7fc465d9b700 1 -- 172.21.15.92:0/385086592 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47015a670 con 0x7fc4700696e0 2023-10-06T10:38:23.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.552+0000 7fc465d9b700 1 --2- 172.21.15.92:0/385086592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4700696e0 0x7fc470159ad0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc4700cdf70 tx=0x7fc458009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2c6bafebf4ec9f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:23.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.552+0000 7fc46dd9b700 1 -- 172.21.15.92:0/385086592 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45800b820 con 0x7fc4700696e0 2023-10-06T10:38:23.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.552+0000 7fc46dd9b700 1 -- 172.21.15.92:0/385086592 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45800b980 con 0x7fc4700696e0 2023-10-06T10:38:23.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.552+0000 7fc46dd9b700 1 -- 172.21.15.92:0/385086592 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45800bd20 con 0x7fc4700696e0 2023-10-06T10:38:23.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.552+0000 7fc474996700 1 -- 172.21.15.92:0/385086592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4700696e0 msgr2=0x7fc470159ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:23.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.552+0000 7fc474996700 1 --2- 172.21.15.92:0/385086592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4700696e0 0x7fc470159ad0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc4700cdf70 tx=0x7fc458009250 comp rx=0 tx=0).stop 2023-10-06T10:38:23.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.552+0000 7fc474996700 1 -- 172.21.15.92:0/385086592 shutdown_connections 2023-10-06T10:38:23.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.552+0000 7fc474996700 1 --2- 172.21.15.92:0/385086592 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc47015a010 0x7fc4700ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.552+0000 7fc474996700 1 --2- 172.21.15.92:0/385086592 >> v2:172.21.15.124:3300/0 conn(0x7fc4700cb5f0 0x7fc4700691a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.552+0000 7fc474996700 1 --2- 172.21.15.92:0/385086592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4700696e0 0x7fc470159ad0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.552+0000 7fc474996700 1 -- 172.21.15.92:0/385086592 >> 172.21.15.92:0/385086592 conn(0x7fc4700707d0 msgr2=0x7fc470070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:23.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.553+0000 7fc474996700 1 -- 172.21.15.92:0/385086592 shutdown_connections 2023-10-06T10:38:23.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.553+0000 7fc474996700 1 -- 172.21.15.92:0/385086592 wait complete. 2023-10-06T10:38:23.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.554+0000 7fc474996700 1 Processor -- start 2023-10-06T10:38:23.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.554+0000 7fc474996700 1 -- start start 2023-10-06T10:38:23.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.554+0000 7fc474996700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4700696e0 0x7fc470156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:23.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.554+0000 7fc474996700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc4700cb5f0 0x7fc470151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:23.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.554+0000 7fc474996700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc47015a010 0x7fc470154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:23.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.554+0000 7fc474996700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc470155cc0 con 0x7fc4700696e0 2023-10-06T10:38:23.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.554+0000 7fc474996700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc470155ed0 con 0x7fc4700cb5f0 2023-10-06T10:38:23.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.555+0000 7fc474996700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4701561b0 con 0x7fc47015a010 2023-10-06T10:38:23.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.555+0000 7fc46e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4700696e0 0x7fc470156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:23.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.555+0000 7fc46e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4700696e0 0x7fc470156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37708/0 (socket says 172.21.15.92:37708) 2023-10-06T10:38:23.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.555+0000 7fc46e59c700 1 -- 172.21.15.92:0/582740695 learned_addr learned my addr 172.21.15.92:0/582740695 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.555+0000 7fc46ed9d700 1 --2- 172.21.15.92:0/582740695 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc47015a010 0x7fc470154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.555+0000 7fc465d9b700 1 --2- 172.21.15.92:0/582740695 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc4700cb5f0 0x7fc470151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.555+0000 7fc46e59c700 1 -- 172.21.15.92:0/582740695 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc47015a010 msgr2=0x7fc470154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.555+0000 7fc46e59c700 1 --2- 172.21.15.92:0/582740695 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc47015a010 0x7fc470154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.555+0000 7fc46e59c700 1 -- 172.21.15.92:0/582740695 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc4700cb5f0 msgr2=0x7fc470151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.555+0000 7fc46e59c700 1 --2- 172.21.15.92:0/582740695 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc4700cb5f0 0x7fc470151ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.555+0000 7fc46e59c700 1 -- 172.21.15.92:0/582740695 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4580094f0 con 0x7fc4700696e0 2023-10-06T10:38:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.555+0000 7fc465d9b700 1 --2- 172.21.15.92:0/582740695 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc4700cb5f0 0x7fc470151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.555+0000 7fc46e59c700 1 --2- 172.21.15.92:0/582740695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4700696e0 0x7fc470156f70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc46000c7d0 tx=0x7fc460008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.555+0000 7fc466ffd700 1 -- 172.21.15.92:0/582740695 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc460011600 con 0x7fc4700696e0 2023-10-06T10:38:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.555+0000 7fc466ffd700 1 -- 172.21.15.92:0/582740695 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc460011760 con 0x7fc4700696e0 2023-10-06T10:38:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.555+0000 7fc474996700 1 -- 172.21.15.92:0/582740695 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc470156490 con 0x7fc4700696e0 2023-10-06T10:38:23.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.555+0000 7fc466ffd700 1 -- 172.21.15.92:0/582740695 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc460011b20 con 0x7fc4700696e0 2023-10-06T10:38:23.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.555+0000 7fc474996700 1 -- 172.21.15.92:0/582740695 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc470156ad0 con 0x7fc4700696e0 2023-10-06T10:38:23.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.556+0000 7fc474996700 1 -- 172.21.15.92:0/582740695 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc470156c60 con 0x7fc4700696e0 2023-10-06T10:38:23.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.556+0000 7fc466ffd700 1 -- 172.21.15.92:0/582740695 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc460018350 con 0x7fc4700696e0 2023-10-06T10:38:23.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.557+0000 7fc466ffd700 1 --2- 172.21.15.92:0/582740695 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc450066f30 0x7fc4500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:23.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.557+0000 7fc466ffd700 1 -- 172.21.15.92:0/582740695 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fc460084870 con 0x7fc4700696e0 2023-10-06T10:38:23.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.557+0000 7fc465d9b700 1 --2- 172.21.15.92:0/582740695 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc450066f30 0x7fc4500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:23.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.557+0000 7fc465d9b700 1 --2- 172.21.15.92:0/582740695 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc450066f30 0x7fc4500693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc4700cdf70 tx=0x7fc458017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:23.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.561+0000 7fc466ffd700 1 -- 172.21.15.92:0/582740695 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc460051690 con 0x7fc4700696e0 2023-10-06T10:38:23.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.789+0000 7fc474996700 1 -- 172.21.15.92:0/582740695 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc470064a90 con 0x7fc4700696e0 2023-10-06T10:38:23.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.789+0000 7fc466ffd700 1 -- 172.21.15.92:0/582740695 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4599 (secure 0 0 0) 0x7fc460054cb0 con 0x7fc4700696e0 2023-10-06T10:38:23.791 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:23.791 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2303424485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2303424485},{"type":"v1","addr":"172.21.15.92:6836","nonce":2303424485}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1027343065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1027343065},{"type":"v1","addr":"172.21.15.92:6837","nonce":1027343065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2667820379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2667820379},{"type":"v1","addr":"172.21.15.124:6834","nonce":2667820379}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:38:16.725144+0000","modified":"2023-10-06T10:38:22.380999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/1919704285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":1919704285},{"type":"v1","addr":"172.21.15.124:6835","nonce":1919704285}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2023-10-06T10:38:23.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.792+0000 7fc474996700 1 -- 172.21.15.92:0/582740695 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc450066f30 msgr2=0x7fc4500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:23.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.792+0000 7fc474996700 1 --2- 172.21.15.92:0/582740695 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc450066f30 0x7fc4500693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc4700cdf70 tx=0x7fc458017000 comp rx=0 tx=0).stop 2023-10-06T10:38:23.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.792+0000 7fc474996700 1 -- 172.21.15.92:0/582740695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4700696e0 msgr2=0x7fc470156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:23.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.792+0000 7fc474996700 1 --2- 172.21.15.92:0/582740695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4700696e0 0x7fc470156f70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc46000c7d0 tx=0x7fc460008040 comp rx=0 tx=0).stop 2023-10-06T10:38:23.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.792+0000 7fc474996700 1 -- 172.21.15.92:0/582740695 shutdown_connections 2023-10-06T10:38:23.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.792+0000 7fc474996700 1 --2- 172.21.15.92:0/582740695 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc450066f30 0x7fc4500693e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.792+0000 7fc474996700 1 --2- 172.21.15.92:0/582740695 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc47015a010 0x7fc470154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.792+0000 7fc474996700 1 --2- 172.21.15.92:0/582740695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4700696e0 0x7fc470156f70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.792+0000 7fc474996700 1 --2- 172.21.15.92:0/582740695 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc4700cb5f0 0x7fc470151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.792+0000 7fc474996700 1 -- 172.21.15.92:0/582740695 >> 172.21.15.92:0/582740695 conn(0x7fc4700707d0 msgr2=0x7fc4700ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:23.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.792+0000 7fc474996700 1 -- 172.21.15.92:0/582740695 shutdown_connections 2023-10-06T10:38:23.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.792+0000 7fc474996700 1 -- 172.21.15.92:0/582740695 wait complete. 2023-10-06T10:38:23.795 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 6 2023-10-06T10:38:23.805 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:38:23.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.916+0000 7fc163cb3700 1 Processor -- start 2023-10-06T10:38:23.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.917+0000 7fc163cb3700 1 -- start start 2023-10-06T10:38:23.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.917+0000 7fc163cb3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc15c0adb60 0x7fc15c0ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:23.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.917+0000 7fc163cb3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc15c157de0 con 0x7fc15c0ca370 2023-10-06T10:38:23.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.917+0000 7fc163cb3700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc15c0ceb10 con 0x7fc15c0ce6d0 2023-10-06T10:38:23.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.917+0000 7fc163cb3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc15c0cbb90 con 0x7fc15c0adb60 2023-10-06T10:38:23.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.918+0000 7fc161a4f700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc15c0ca370 0x7fc15c158270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59904/0 (socket says 172.21.15.92:59904) 2023-10-06T10:38:23.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.918+0000 7fc161a4f700 1 -- 172.21.15.92:0/895362254 learned_addr learned my addr 172.21.15.92:0/895362254 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:23.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.918+0000 7fc16124e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc15c0adb60 0x7fc15c0ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:23.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.918+0000 7fc16124e700 1 -- 172.21.15.92:0/895362254 >> v1:172.21.15.124:6789/0 conn(0x7fc15c0ce6d0 legacy=0x7fc15c15aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:23.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.918+0000 7fc16124e700 1 -- 172.21.15.92:0/895362254 >> v1:172.21.15.92:6789/0 conn(0x7fc15c0ca370 legacy=0x7fc15c158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:23.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.918+0000 7fc16124e700 1 -- 172.21.15.92:0/895362254 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c0cb5d0 con 0x7fc15c0adb60 2023-10-06T10:38:23.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.918+0000 7fc160a4d700 1 -- 172.21.15.92:0/895362254 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947061598 0 0) 0x7fc15c157de0 con 0x7fc15c0ca370 2023-10-06T10:38:23.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.918+0000 7fc16124e700 1 --2- 172.21.15.92:0/895362254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc15c0adb60 0x7fc15c0ce190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0098e0 tx=0x7fc14c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64b76b16f907fb3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:23.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.918+0000 7fc160a4d700 1 -- 172.21.15.92:0/895362254 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c009e80 con 0x7fc15c0adb60 2023-10-06T10:38:23.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.918+0000 7fc160a4d700 1 -- 172.21.15.92:0/895362254 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14c00b820 con 0x7fc15c0adb60 2023-10-06T10:38:23.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.918+0000 7fc160a4d700 1 -- 172.21.15.92:0/895362254 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c00bba0 con 0x7fc15c0adb60 2023-10-06T10:38:23.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.919+0000 7fc163cb3700 1 -- 172.21.15.92:0/895362254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc15c0adb60 msgr2=0x7fc15c0ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:23.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.919+0000 7fc163cb3700 1 --2- 172.21.15.92:0/895362254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc15c0adb60 0x7fc15c0ce190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0098e0 tx=0x7fc14c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.919+0000 7fc163cb3700 1 -- 172.21.15.92:0/895362254 shutdown_connections 2023-10-06T10:38:23.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.919+0000 7fc163cb3700 1 --2- 172.21.15.92:0/895362254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc15c0adb60 0x7fc15c0ce190 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.919+0000 7fc163cb3700 1 -- 172.21.15.92:0/895362254 >> 172.21.15.92:0/895362254 conn(0x7fc15c0707d0 msgr2=0x7fc15c070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:23.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.919+0000 7fc163cb3700 1 -- 172.21.15.92:0/895362254 shutdown_connections 2023-10-06T10:38:23.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.919+0000 7fc163cb3700 1 -- 172.21.15.92:0/895362254 wait complete. 2023-10-06T10:38:23.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.920+0000 7fc163cb3700 1 Processor -- start 2023-10-06T10:38:23.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.920+0000 7fc163cb3700 1 -- start start 2023-10-06T10:38:23.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc163cb3700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc15c0adb60 0x7fc15c0b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:23.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc163cb3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc15c0ca370 0x7fc15c1682e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:23.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc163cb3700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc15c0ce6d0 0x7fc15c159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:23.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc163cb3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c168820 con 0x7fc15c0ca370 2023-10-06T10:38:23.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc163cb3700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c156440 con 0x7fc15c0ce6d0 2023-10-06T10:38:23.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc163cb3700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c155830 con 0x7fc15c0adb60 2023-10-06T10:38:23.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc16124e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc15c0ca370 0x7fc15c1682e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:23.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc16124e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc15c0ca370 0x7fc15c1682e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37720/0 (socket says 172.21.15.92:37720) 2023-10-06T10:38:23.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc16124e700 1 -- 172.21.15.92:0/1844447847 learned_addr learned my addr 172.21.15.92:0/1844447847 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:23.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc161a4f700 1 --2- 172.21.15.92:0/1844447847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc15c0adb60 0x7fc15c0b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:23.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc162250700 1 --2- 172.21.15.92:0/1844447847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc15c0ce6d0 0x7fc15c159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:23.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc16124e700 1 -- 172.21.15.92:0/1844447847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc15c0adb60 msgr2=0x7fc15c0b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:23.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc16124e700 1 --2- 172.21.15.92:0/1844447847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc15c0adb60 0x7fc15c0b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc16124e700 1 -- 172.21.15.92:0/1844447847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc15c0ce6d0 msgr2=0x7fc15c159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:23.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc16124e700 1 --2- 172.21.15.92:0/1844447847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc15c0ce6d0 0x7fc15c159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:23.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc16124e700 1 -- 172.21.15.92:0/1844447847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc14c009420 con 0x7fc15c0ca370 2023-10-06T10:38:23.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc161a4f700 1 --2- 172.21.15.92:0/1844447847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc15c0adb60 0x7fc15c0b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:23.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc162250700 1 --2- 172.21.15.92:0/1844447847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc15c0ce6d0 0x7fc15c159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:23.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc16124e700 1 --2- 172.21.15.92:0/1844447847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc15c0ca370 0x7fc15c1682e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0030c0 tx=0x7fc14c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:23.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc151ffb700 1 -- 172.21.15.92:0/1844447847 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c00ba10 con 0x7fc15c0ca370 2023-10-06T10:38:23.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc151ffb700 1 -- 172.21.15.92:0/1844447847 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14c020070 con 0x7fc15c0ca370 2023-10-06T10:38:23.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc163cb3700 1 -- 172.21.15.92:0/1844447847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc15c155b10 con 0x7fc15c0ca370 2023-10-06T10:38:23.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc151ffb700 1 -- 172.21.15.92:0/1844447847 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c01b6a0 con 0x7fc15c0ca370 2023-10-06T10:38:23.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.921+0000 7fc163cb3700 1 -- 172.21.15.92:0/1844447847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15c1560d0 con 0x7fc15c0ca370 2023-10-06T10:38:23.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.922+0000 7fc163cb3700 1 -- 172.21.15.92:0/1844447847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc15c064a90 con 0x7fc15c0ca370 2023-10-06T10:38:23.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.923+0000 7fc151ffb700 1 -- 172.21.15.92:0/1844447847 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc14c022030 con 0x7fc15c0ca370 2023-10-06T10:38:23.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.923+0000 7fc151ffb700 1 --2- 172.21.15.92:0/1844447847 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc140066f30 0x7fc1400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:23.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.923+0000 7fc151ffb700 1 -- 172.21.15.92:0/1844447847 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fc14c086840 con 0x7fc15c0ca370 2023-10-06T10:38:23.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.923+0000 7fc161a4f700 1 --2- 172.21.15.92:0/1844447847 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc140066f30 0x7fc1400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:23.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.924+0000 7fc161a4f700 1 --2- 172.21.15.92:0/1844447847 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc140066f30 0x7fc1400693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc158001810 tx=0x7fc158009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:23.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:23.927+0000 7fc151ffb700 1 -- 172.21.15.92:0/1844447847 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc14c053660 con 0x7fc15c0ca370 2023-10-06T10:38:24.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.111+0000 7fc163cb3700 1 -- 172.21.15.92:0/1844447847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc15c064a90 con 0x7fc15c0ca370 2023-10-06T10:38:24.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.112+0000 7fc151ffb700 1 -- 172.21.15.92:0/1844447847 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18707 (secure 0 0 0) 0x7fc14c056c80 con 0x7fc15c0ca370 2023-10-06T10:38:24.115 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:24.115 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:21.760233+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:38:14.368713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T10:38:15.692344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T10:38:19.382553+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:38:24.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.114+0000 7fc163cb3700 1 -- 172.21.15.92:0/1844447847 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc140066f30 msgr2=0x7fc1400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:24.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.114+0000 7fc163cb3700 1 --2- 172.21.15.92:0/1844447847 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc140066f30 0x7fc1400693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc158001810 tx=0x7fc158009040 comp rx=0 tx=0).stop 2023-10-06T10:38:24.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.114+0000 7fc163cb3700 1 -- 172.21.15.92:0/1844447847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc15c0ca370 msgr2=0x7fc15c1682e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.114+0000 7fc163cb3700 1 --2- 172.21.15.92:0/1844447847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc15c0ca370 0x7fc15c1682e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0030c0 tx=0x7fc14c016000 comp rx=0 tx=0).stop 2023-10-06T10:38:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.114+0000 7fc163cb3700 1 -- 172.21.15.92:0/1844447847 shutdown_connections 2023-10-06T10:38:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.114+0000 7fc163cb3700 1 --2- 172.21.15.92:0/1844447847 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc140066f30 0x7fc1400693e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.114+0000 7fc163cb3700 1 --2- 172.21.15.92:0/1844447847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc15c0ce6d0 0x7fc15c159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.114+0000 7fc163cb3700 1 --2- 172.21.15.92:0/1844447847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc15c0adb60 0x7fc15c0b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.114+0000 7fc163cb3700 1 --2- 172.21.15.92:0/1844447847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc15c0ca370 0x7fc15c1682e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.114+0000 7fc163cb3700 1 -- 172.21.15.92:0/1844447847 >> 172.21.15.92:0/1844447847 conn(0x7fc15c0707d0 msgr2=0x7fc15c0ccf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:24.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.114+0000 7fc163cb3700 1 -- 172.21.15.92:0/1844447847 shutdown_connections 2023-10-06T10:38:24.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.114+0000 7fc163cb3700 1 -- 172.21.15.92:0/1844447847 wait complete. 2023-10-06T10:38:24.530 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T10:38:16.725144+0000', 'modified': '2023-10-06T10:38:22.380999+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4306}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4306': {'gid': 4306, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.124:6835/1919704285', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6833', 'nonce': 1919704285}, {'type': 'v1', 'addr': '172.21.15.124:6835', 'nonce': 1919704285}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 1} 2023-10-06T10:38:24.531 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T10:38:24.531 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T10:38:24.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.646+0000 7fdf4759e700 1 Processor -- start 2023-10-06T10:38:24.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.646+0000 7fdf4759e700 1 -- start start 2023-10-06T10:38:24.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.647+0000 7fdf4759e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf48159490 0x7fdf480a9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.647+0000 7fdf4759e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdf480af5c0 0x7fdf4806af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.647+0000 7fdf4759e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf4806aaa0 con 0x7fdf4806b470 2023-10-06T10:38:24.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.647+0000 7fdf4759e700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf480a9ff0 con 0x7fdf480af5c0 2023-10-06T10:38:24.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.647+0000 7fdf4759e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf480aa290 con 0x7fdf48159490 2023-10-06T10:38:24.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.647+0000 7fdf46d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdf4806b470 0x7fdf480675b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59916/0 (socket says 172.21.15.92:59916) 2023-10-06T10:38:24.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.647+0000 7fdf46d9d700 1 -- 172.21.15.92:0/1716691834 learned_addr learned my addr 172.21.15.92:0/1716691834 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:24.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.647+0000 7fdf45d9b700 1 --2- 172.21.15.92:0/1716691834 >> v2:172.21.15.124:3300/0 conn(0x7fdf480af5c0 0x7fdf4806af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.647+0000 7fdf4559a700 1 -- 172.21.15.92:0/1716691834 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3101567857 0 0) 0x7fdf4806aaa0 con 0x7fdf4806b470 2023-10-06T10:38:24.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.647+0000 7fdf4659c700 1 --2- 172.21.15.92:0/1716691834 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf48159490 0x7fdf480a9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.647+0000 7fdf4559a700 1 -- 172.21.15.92:0/1716691834 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf30003570 con 0x7fdf4806b470 2023-10-06T10:38:24.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.647+0000 7fdf4559a700 1 -- 172.21.15.92:0/1716691834 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 684667077 0 0) 0x7fdf30003570 con 0x7fdf4806b470 2023-10-06T10:38:24.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.647+0000 7fdf4559a700 1 -- 172.21.15.92:0/1716691834 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf48159490 msgr2=0x7fdf480a9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:24.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.647+0000 7fdf4559a700 1 --2- 172.21.15.92:0/1716691834 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf48159490 0x7fdf480a9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.647+0000 7fdf4559a700 1 -- 172.21.15.92:0/1716691834 >> v2:172.21.15.124:3300/0 conn(0x7fdf480af5c0 msgr2=0x7fdf4806af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:24.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.647+0000 7fdf4559a700 1 --2- 172.21.15.92:0/1716691834 >> v2:172.21.15.124:3300/0 conn(0x7fdf480af5c0 0x7fdf4806af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.647+0000 7fdf4559a700 1 -- 172.21.15.92:0/1716691834 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf480aa570 con 0x7fdf4806b470 2023-10-06T10:38:24.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.647+0000 7fdf4559a700 1 -- 172.21.15.92:0/1716691834 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fdf400017f0 con 0x7fdf4806b470 2023-10-06T10:38:24.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.648+0000 7fdf4559a700 1 -- 172.21.15.92:0/1716691834 >> v1:172.21.15.92:6789/0 conn(0x7fdf4806b470 legacy=0x7fdf480675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:24.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.648+0000 7fdf4559a700 1 --2- 172.21.15.92:0/1716691834 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf48159490 0x7fdf30003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.648+0000 7fdf4559a700 1 --2- 172.21.15.92:0/1716691834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf30003fe0 0x7fdf30004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.648+0000 7fdf4559a700 1 --2- 172.21.15.92:0/1716691834 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf30004d50 0x7fdf300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.648+0000 7fdf4559a700 1 -- 172.21.15.92:0/1716691834 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf480a9ff0 con 0x7fdf30003fe0 2023-10-06T10:38:24.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.648+0000 7fdf4559a700 1 -- 172.21.15.92:0/1716691834 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf480aa290 con 0x7fdf30004d50 2023-10-06T10:38:24.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.648+0000 7fdf4559a700 1 -- 172.21.15.92:0/1716691834 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3000ba90 con 0x7fdf48159490 2023-10-06T10:38:24.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.648+0000 7fdf45d9b700 1 --2- 172.21.15.92:0/1716691834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf30003fe0 0x7fdf30004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.648+0000 7fdf4659c700 1 --2- 172.21.15.92:0/1716691834 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf48159490 0x7fdf30003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.648+0000 7fdf46d9d700 1 --2- 172.21.15.92:0/1716691834 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf30004d50 0x7fdf300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.648+0000 7fdf45d9b700 1 -- 172.21.15.92:0/1716691834 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf48159490 msgr2=0x7fdf30003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:24.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.648+0000 7fdf45d9b700 1 --2- 172.21.15.92:0/1716691834 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf48159490 0x7fdf30003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.648+0000 7fdf45d9b700 1 -- 172.21.15.92:0/1716691834 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf30004d50 msgr2=0x7fdf300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:24.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.648+0000 7fdf45d9b700 1 --2- 172.21.15.92:0/1716691834 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf30004d50 0x7fdf300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.648+0000 7fdf45d9b700 1 -- 172.21.15.92:0/1716691834 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf30003570 con 0x7fdf30003fe0 2023-10-06T10:38:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.648+0000 7fdf46d9d700 1 --2- 172.21.15.92:0/1716691834 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf30004d50 0x7fdf300051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.648+0000 7fdf45d9b700 1 --2- 172.21.15.92:0/1716691834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf30003fe0 0x7fdf30004810 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009c00 tx=0x7fdf3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f885e1125c9204 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.648+0000 7fdf4559a700 1 -- 172.21.15.92:0/1716691834 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c00e6b0 con 0x7fdf30003fe0 2023-10-06T10:38:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.648+0000 7fdf4559a700 1 -- 172.21.15.92:0/1716691834 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3c00e810 con 0x7fdf30003fe0 2023-10-06T10:38:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.648+0000 7fdf4559a700 1 -- 172.21.15.92:0/1716691834 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c00ebd0 con 0x7fdf30003fe0 2023-10-06T10:38:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.649+0000 7fdf4759e700 1 -- 172.21.15.92:0/1716691834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf30003fe0 msgr2=0x7fdf30004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.649+0000 7fdf4759e700 1 --2- 172.21.15.92:0/1716691834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf30003fe0 0x7fdf30004810 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009c00 tx=0x7fdf3c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.649+0000 7fdf46d9d700 1 -- 172.21.15.92:0/1716691834 reap_dead start 2023-10-06T10:38:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.649+0000 7fdf4759e700 1 -- 172.21.15.92:0/1716691834 shutdown_connections 2023-10-06T10:38:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.649+0000 7fdf4759e700 1 -- 172.21.15.92:0/1716691834 >> 172.21.15.92:0/1716691834 conn(0x7fdf48069190 msgr2=0x7fdf480aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.649+0000 7fdf4759e700 1 -- 172.21.15.92:0/1716691834 shutdown_connections 2023-10-06T10:38:24.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.649+0000 7fdf4759e700 1 -- 172.21.15.92:0/1716691834 wait complete. 2023-10-06T10:38:24.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.650+0000 7fdf4759e700 1 Processor -- start 2023-10-06T10:38:24.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.650+0000 7fdf4759e700 1 -- start start 2023-10-06T10:38:24.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.651+0000 7fdf4759e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf30003fe0 0x7fdf480b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.651+0000 7fdf4759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf4815a6e0 0x7fdf480b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.651+0000 7fdf4759e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf480b0f30 0x7fdf48157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.651+0000 7fdf4759e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf481575f0 con 0x7fdf4815a6e0 2023-10-06T10:38:24.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.651+0000 7fdf4759e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4815c650 con 0x7fdf480b0f30 2023-10-06T10:38:24.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.651+0000 7fdf4759e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf480b2bf0 con 0x7fdf30003fe0 2023-10-06T10:38:24.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.651+0000 7fdf45d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf4815a6e0 0x7fdf480b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.651+0000 7fdf45d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf4815a6e0 0x7fdf480b09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37730/0 (socket says 172.21.15.92:37730) 2023-10-06T10:38:24.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.651+0000 7fdf45d9b700 1 -- 172.21.15.92:0/107192166 learned_addr learned my addr 172.21.15.92:0/107192166 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:24.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.651+0000 7fdf4659c700 1 --2- 172.21.15.92:0/107192166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf30003fe0 0x7fdf480b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.651+0000 7fdf46d9d700 1 --2- 172.21.15.92:0/107192166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf480b0f30 0x7fdf48157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.651+0000 7fdf45d9b700 1 -- 172.21.15.92:0/107192166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf30003fe0 msgr2=0x7fdf480b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:24.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.651+0000 7fdf45d9b700 1 --2- 172.21.15.92:0/107192166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf30003fe0 0x7fdf480b2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.651+0000 7fdf45d9b700 1 -- 172.21.15.92:0/107192166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf480b0f30 msgr2=0x7fdf48157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:24.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.651+0000 7fdf45d9b700 1 --2- 172.21.15.92:0/107192166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf480b0f30 0x7fdf48157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.651+0000 7fdf45d9b700 1 -- 172.21.15.92:0/107192166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf48156fa0 con 0x7fdf4815a6e0 2023-10-06T10:38:24.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.651+0000 7fdf45d9b700 1 --2- 172.21.15.92:0/107192166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf4815a6e0 0x7fdf480b09f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdf30004730 tx=0x7fdf3c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:24.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.651+0000 7fdf367fc700 1 -- 172.21.15.92:0/107192166 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c022070 con 0x7fdf4815a6e0 2023-10-06T10:38:24.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.651+0000 7fdf367fc700 1 -- 172.21.15.92:0/107192166 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3c00e980 con 0x7fdf4815a6e0 2023-10-06T10:38:24.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.651+0000 7fdf4759e700 1 -- 172.21.15.92:0/107192166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf48157280 con 0x7fdf4815a6e0 2023-10-06T10:38:24.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.651+0000 7fdf367fc700 1 -- 172.21.15.92:0/107192166 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c027470 con 0x7fdf4815a6e0 2023-10-06T10:38:24.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.651+0000 7fdf4759e700 1 -- 172.21.15.92:0/107192166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf48158200 con 0x7fdf4815a6e0 2023-10-06T10:38:24.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.653+0000 7fdf367fc700 1 -- 172.21.15.92:0/107192166 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdf3c01c450 con 0x7fdf4815a6e0 2023-10-06T10:38:24.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.653+0000 7fdf367fc700 1 --2- 172.21.15.92:0/107192166 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdf28066f30 0x7fdf280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.653+0000 7fdf367fc700 1 -- 172.21.15.92:0/107192166 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fdf3c086610 con 0x7fdf4815a6e0 2023-10-06T10:38:24.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.653+0000 7fdf4659c700 1 --2- 172.21.15.92:0/107192166 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdf28066f30 0x7fdf280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.653+0000 7fdf4659c700 1 --2- 172.21.15.92:0/107192166 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdf28066f30 0x7fdf280693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdf38006dc0 tx=0x7fdf38008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:24.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.655+0000 7fdf55c7cb80 1 Processor -- start 2023-10-06T10:38:24.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.655+0000 7fdf55c7cb80 1 -- start start 2023-10-06T10:38:24.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.656+0000 7fdf55c7cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc716950 0x5612bc716d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.656+0000 7fdf55c7cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5612bc760bc0 0x5612bc73f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.656+0000 7fdf55c7cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc73fe60 0x5612bc712990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.656+0000 7fdf55c7cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bc696c80 con 0x5612bc73fe60 2023-10-06T10:38:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.656+0000 7fdf55c7cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bc66da70 con 0x5612bc716950 2023-10-06T10:38:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.656+0000 7fdf55c7cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612bc66b270 con 0x5612bc760bc0 2023-10-06T10:38:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.656+0000 7fdf46d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc73fe60 0x5612bc712990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.656+0000 7fdf46d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc73fe60 0x5612bc712990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37736/0 (socket says 172.21.15.92:37736) 2023-10-06T10:38:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.656+0000 7fdf46d9d700 1 -- 172.21.15.92:0/2927869608 learned_addr learned my addr 172.21.15.92:0/2927869608 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.656+0000 7fdf4659c700 1 --2- 172.21.15.92:0/2927869608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc716950 0x5612bc716d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.656+0000 7fdf45d9b700 1 --2- 172.21.15.92:0/2927869608 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5612bc760bc0 0x5612bc73f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.656+0000 7fdf46d9d700 1 -- 172.21.15.92:0/2927869608 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5612bc760bc0 msgr2=0x5612bc73f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.656+0000 7fdf46d9d700 1 --2- 172.21.15.92:0/2927869608 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5612bc760bc0 0x5612bc73f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.656+0000 7fdf46d9d700 1 -- 172.21.15.92:0/2927869608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc716950 msgr2=0x5612bc716d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.656+0000 7fdf46d9d700 1 --2- 172.21.15.92:0/2927869608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc716950 0x5612bc716d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.656+0000 7fdf46d9d700 1 -- 172.21.15.92:0/2927869608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612bc501270 con 0x5612bc73fe60 2023-10-06T10:38:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.656+0000 7fdf4659c700 1 --2- 172.21.15.92:0/2927869608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc716950 0x5612bc716d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.656+0000 7fdf45d9b700 1 --2- 172.21.15.92:0/2927869608 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5612bc760bc0 0x5612bc73f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.656+0000 7fdf46d9d700 1 --2- 172.21.15.92:0/2927869608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc73fe60 0x5612bc712990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdf4806e180 tx=0x7fdf40009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686e8d97c26b27a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.657+0000 7fdf277fe700 1 -- 172.21.15.92:0/2927869608 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf400078a0 con 0x5612bc73fe60 2023-10-06T10:38:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.657+0000 7fdf277fe700 1 -- 172.21.15.92:0/2927869608 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf40007a00 con 0x5612bc73fe60 2023-10-06T10:38:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.657+0000 7fdf277fe700 1 -- 172.21.15.92:0/2927869608 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40014610 con 0x5612bc73fe60 2023-10-06T10:38:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.657+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/2927869608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc73fe60 msgr2=0x5612bc712990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.657+0000 7fdf55c7cb80 1 --2- 172.21.15.92:0/2927869608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc73fe60 0x5612bc712990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdf4806e180 tx=0x7fdf40009040 comp rx=0 tx=0).stop 2023-10-06T10:38:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.657+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/2927869608 shutdown_connections 2023-10-06T10:38:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.657+0000 7fdf55c7cb80 1 --2- 172.21.15.92:0/2927869608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc73fe60 0x5612bc712990 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.657+0000 7fdf55c7cb80 1 --2- 172.21.15.92:0/2927869608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc716950 0x5612bc716d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.657+0000 7fdf55c7cb80 1 --2- 172.21.15.92:0/2927869608 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5612bc760bc0 0x5612bc73f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.657+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/2927869608 >> 172.21.15.92:0/2927869608 conn(0x5612bc718270 msgr2=0x5612bc718640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:24.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.657+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/2927869608 shutdown_connections 2023-10-06T10:38:24.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.657+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/2927869608 wait complete. 2023-10-06T10:38:24.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.658+0000 7fdf55c7cb80 1 -- start start 2023-10-06T10:38:24.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.658+0000 7fdf55c7cb80 1 Processor -- start 2023-10-06T10:38:24.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.659+0000 7fdf55c7cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc716950 0x5612bc7568a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.659+0000 7fdf55c7cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5612bc73fe60 0x5612bc756de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.659+0000 7fdf55c7cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc760bc0 0x5612bc770520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.659+0000 7fdf55c7cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bc667060 con 0x5612bc760bc0 2023-10-06T10:38:24.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.659+0000 7fdf55c7cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bc612890 con 0x5612bc716950 2023-10-06T10:38:24.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.659+0000 7fdf55c7cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612bc62dea0 con 0x5612bc73fe60 2023-10-06T10:38:24.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.659+0000 7fdf55c7cb80 10 client.0 _ll_register_callbacks cb 0x5612bc503f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:38:24.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.659+0000 7fdf46d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc760bc0 0x5612bc770520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.659+0000 7fdf46d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc760bc0 0x5612bc770520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37750/0 (socket says 172.21.15.92:37750) 2023-10-06T10:38:24.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.659+0000 7fdf46d9d700 1 -- 172.21.15.92:0/2595552644 learned_addr learned my addr 172.21.15.92:0/2595552644 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:24.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.659+0000 7fdf4659c700 1 --2- 172.21.15.92:0/2595552644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc716950 0x5612bc7568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.659+0000 7fdf45d9b700 1 --2- 172.21.15.92:0/2595552644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5612bc73fe60 0x5612bc756de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.659+0000 7fdf46d9d700 1 -- 172.21.15.92:0/2595552644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5612bc73fe60 msgr2=0x5612bc756de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:24.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.659+0000 7fdf46d9d700 1 --2- 172.21.15.92:0/2595552644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5612bc73fe60 0x5612bc756de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.659+0000 7fdf46d9d700 1 -- 172.21.15.92:0/2595552644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc716950 msgr2=0x5612bc7568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:24.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.659+0000 7fdf46d9d700 1 --2- 172.21.15.92:0/2595552644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc716950 0x5612bc7568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.659+0000 7fdf46d9d700 1 -- 172.21.15.92:0/2595552644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4000f040 con 0x5612bc760bc0 2023-10-06T10:38:24.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.659+0000 7fdf4659c700 1 --2- 172.21.15.92:0/2595552644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc716950 0x5612bc7568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:24.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.659+0000 7fdf46d9d700 1 --2- 172.21.15.92:0/2595552644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc760bc0 0x5612bc770520 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdf4806e330 tx=0x7fdf40018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6d8af63bc70f03 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:24.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.659+0000 7fdf24ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:38:24.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.660+0000 7fdf24ff9700 1 -- 172.21.15.92:0/2595552644 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40014d50 con 0x5612bc760bc0 2023-10-06T10:38:24.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.660+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/2595552644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612bc70b690 con 0x5612bc760bc0 2023-10-06T10:38:24.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.660+0000 7fdf24ff9700 1 -- 172.21.15.92:0/2595552644 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fdf4001f420 con 0x5612bc760bc0 2023-10-06T10:38:24.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.660+0000 7fdf24ff9700 1 -- 172.21.15.92:0/2595552644 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf4001f7f0 con 0x5612bc760bc0 2023-10-06T10:38:24.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.660+0000 7fdf55c7cb80 10 client.4440 fetch_fsmap learned FSMap version 6 2023-10-06T10:38:24.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.660+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/2595552644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x5612bc4a3fe0 con 0x5612bc760bc0 2023-10-06T10:38:24.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.660+0000 7fdf24ff9700 1 -- 172.21.15.92:0/2595552644 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5612bc70b690 con 0x5612bc760bc0 2023-10-06T10:38:24.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.660+0000 7fdf24ff9700 1 -- 172.21.15.92:0/2595552644 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 6) v1 ==== 3108+0+0 (secure 0 0 0) 0x7fdf4001b070 con 0x5612bc760bc0 2023-10-06T10:38:24.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.660+0000 7fdf55c7cb80 10 client.4440 fetch_fsmap finished waiting for FSMap version 6 2023-10-06T10:38:24.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.660+0000 7fdf55c7cb80 10 client.4440 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T10:38:24.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.660+0000 7fdf55c7cb80 20 client.4440 populate_metadata read hostname 'smithi092' 2023-10-06T10:38:24.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.660+0000 7fdf55c7cb80 1 --2- 172.21.15.92:0/2595552644 >> [v2:172.21.15.124:6833/1919704285,v1:172.21.15.124:6835/1919704285] conn(0x5612bc73dab0 0x5612bc73de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.660+0000 7fdf55c7cb80 4 client.4440 mds_command: new command op to 4306 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:38:24.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.660+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/2595552644 --> [v2:172.21.15.124:6833/1919704285,v1:172.21.15.124:6835/1919704285] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612bc6e85c0 con 0x5612bc73dab0 2023-10-06T10:38:24.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.660+0000 7fdf4659c700 1 --2- 172.21.15.92:0/2595552644 >> [v2:172.21.15.124:6833/1919704285,v1:172.21.15.124:6835/1919704285] conn(0x5612bc73dab0 0x5612bc73de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.661+0000 7fdf24ff9700 10 client.4440 ms_handle_connect on v2:172.21.15.124:6833/1919704285 2023-10-06T10:38:24.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.661+0000 7fdf4659c700 1 --2- 172.21.15.92:0/2595552644 >> [v2:172.21.15.124:6833/1919704285,v1:172.21.15.124:6835/1919704285] conn(0x5612bc73dab0 0x5612bc73de80 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f4c7fea3194683b server_cookie=be151e03407947ae in_seq=0 out_seq=0 2023-10-06T10:38:24.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.661+0000 7fdf24ff9700 1 -- 172.21.15.92:0/2595552644 <== mds.0 v2:172.21.15.124:6833/1919704285 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5612bc6e85c0 con 0x5612bc73dab0 2023-10-06T10:38:24.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.661+0000 7fdf24ff9700 10 client.4440 handle_command_reply: tid=0 2023-10-06T10:38:24.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.661+0000 7fdf55c7cb80 1 client.4440 shutdown 2023-10-06T10:38:24.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.661+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/2595552644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc760bc0 msgr2=0x5612bc770520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:24.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.661+0000 7fdf55c7cb80 1 --2- 172.21.15.92:0/2595552644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc760bc0 0x5612bc770520 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdf4806e330 tx=0x7fdf40018000 comp rx=0 tx=0).stop 2023-10-06T10:38:24.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.661+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/2595552644 shutdown_connections 2023-10-06T10:38:24.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.661+0000 7fdf55c7cb80 1 --2- 172.21.15.92:0/2595552644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc760bc0 0x5612bc770520 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.661+0000 7fdf55c7cb80 1 --2- 172.21.15.92:0/2595552644 >> [v2:172.21.15.124:6833/1919704285,v1:172.21.15.124:6835/1919704285] conn(0x5612bc73dab0 0x5612bc73de80 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.661+0000 7fdf55c7cb80 1 --2- 172.21.15.92:0/2595552644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc716950 0x5612bc7568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.661+0000 7fdf24ff9700 0 client.4440 ms_handle_reset on v2:172.21.15.124:6833/1919704285 2023-10-06T10:38:24.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.661+0000 7fdf55c7cb80 1 --2- 172.21.15.92:0/2595552644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5612bc73fe60 0x5612bc756de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.661+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/2595552644 >> 172.21.15.92:0/2595552644 conn(0x5612bc718270 msgr2=0x5612bc73c760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:24.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.662+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/2595552644 shutdown_connections 2023-10-06T10:38:24.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.662+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/2595552644 wait complete. 2023-10-06T10:38:24.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.662+0000 7fdf55c7cb80 20 client.4440 trim_cache size 0 max 16384 2023-10-06T10:38:24.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.678+0000 7fdf55c7cb80 1 Processor -- start 2023-10-06T10:38:24.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.678+0000 7fdf55c7cb80 1 -- start start 2023-10-06T10:38:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.679+0000 7fdf55c7cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc716950 0x5612bc718de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.679+0000 7fdf55c7cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc73fe60 0x5612bc75e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.679+0000 7fdf55c7cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5612bc760bc0 0x5612bc756860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.679+0000 7fdf55c7cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bc4e9e30 con 0x5612bc73fe60 2023-10-06T10:38:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.679+0000 7fdf55c7cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bc75a320 con 0x5612bc716950 2023-10-06T10:38:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.679+0000 7fdf55c7cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612bc73cb50 con 0x5612bc760bc0 2023-10-06T10:38:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.679+0000 7fdf45d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc73fe60 0x5612bc75e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.679+0000 7fdf45d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc73fe60 0x5612bc75e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37756/0 (socket says 172.21.15.92:37756) 2023-10-06T10:38:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.679+0000 7fdf45d9b700 1 -- 172.21.15.92:0/1094595071 learned_addr learned my addr 172.21.15.92:0/1094595071 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.679+0000 7fdf46d9d700 1 --2- 172.21.15.92:0/1094595071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5612bc760bc0 0x5612bc756860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.679+0000 7fdf4659c700 1 --2- 172.21.15.92:0/1094595071 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc716950 0x5612bc718de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.679+0000 7fdf45d9b700 1 -- 172.21.15.92:0/1094595071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5612bc760bc0 msgr2=0x5612bc756860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.679+0000 7fdf45d9b700 1 --2- 172.21.15.92:0/1094595071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5612bc760bc0 0x5612bc756860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.680+0000 7fdf45d9b700 1 -- 172.21.15.92:0/1094595071 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc716950 msgr2=0x5612bc718de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.680+0000 7fdf45d9b700 1 --2- 172.21.15.92:0/1094595071 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc716950 0x5612bc718de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.680+0000 7fdf45d9b700 1 -- 172.21.15.92:0/1094595071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612bc635780 con 0x5612bc73fe60 2023-10-06T10:38:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.680+0000 7fdf46d9d700 1 --2- 172.21.15.92:0/1094595071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5612bc760bc0 0x5612bc756860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.680+0000 7fdf4659c700 1 --2- 172.21.15.92:0/1094595071 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc716950 0x5612bc718de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.680+0000 7fdf45d9b700 1 --2- 172.21.15.92:0/1094595071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc73fe60 0x5612bc75e140 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00ed00 tx=0x7fdf3c01f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87f818494260ed5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.680+0000 7fdf27fff700 1 -- 172.21.15.92:0/1094595071 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c0570f0 con 0x5612bc73fe60 2023-10-06T10:38:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.680+0000 7fdf27fff700 1 -- 172.21.15.92:0/1094595071 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3c05a340 con 0x5612bc73fe60 2023-10-06T10:38:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.680+0000 7fdf27fff700 1 -- 172.21.15.92:0/1094595071 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c05e090 con 0x5612bc73fe60 2023-10-06T10:38:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.680+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/1094595071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc73fe60 msgr2=0x5612bc75e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.680+0000 7fdf55c7cb80 1 --2- 172.21.15.92:0/1094595071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc73fe60 0x5612bc75e140 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00ed00 tx=0x7fdf3c01f040 comp rx=0 tx=0).stop 2023-10-06T10:38:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.680+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/1094595071 shutdown_connections 2023-10-06T10:38:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.680+0000 7fdf55c7cb80 1 --2- 172.21.15.92:0/1094595071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5612bc760bc0 0x5612bc756860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.680+0000 7fdf55c7cb80 1 --2- 172.21.15.92:0/1094595071 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc716950 0x5612bc718de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.680+0000 7fdf55c7cb80 1 --2- 172.21.15.92:0/1094595071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc73fe60 0x5612bc75e140 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.680+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/1094595071 >> 172.21.15.92:0/1094595071 conn(0x5612bc718270 msgr2=0x5612bc76a220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.680+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/1094595071 shutdown_connections 2023-10-06T10:38:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.680+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/1094595071 wait complete. 2023-10-06T10:38:24.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.681+0000 7fdf55c7cb80 1 -- start start 2023-10-06T10:38:24.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.681+0000 7fdf55c7cb80 1 Processor -- start 2023-10-06T10:38:24.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.682+0000 7fdf55c7cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5612bc716950 0x5612bc7bf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.682+0000 7fdf55c7cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc73fe60 0x5612bc75dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.682+0000 7fdf55c7cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc760bc0 0x5612bc75e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.682+0000 7fdf55c7cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bc73e830 con 0x5612bc760bc0 2023-10-06T10:38:24.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.682+0000 7fdf55c7cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bc757c80 con 0x5612bc73fe60 2023-10-06T10:38:24.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.682+0000 7fdf55c7cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612bc70d1a0 con 0x5612bc716950 2023-10-06T10:38:24.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.682+0000 7fdf55c7cb80 10 client.0 _ll_register_callbacks cb 0x5612bc503f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:38:24.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.682+0000 7fdf46d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc760bc0 0x5612bc75e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.682+0000 7fdf46d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc760bc0 0x5612bc75e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37768/0 (socket says 172.21.15.92:37768) 2023-10-06T10:38:24.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.682+0000 7fdf46d9d700 1 -- 172.21.15.92:0/3688465873 learned_addr learned my addr 172.21.15.92:0/3688465873 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:24.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.682+0000 7fdf45d9b700 1 --2- 172.21.15.92:0/3688465873 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc73fe60 0x5612bc75dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.682+0000 7fdf4659c700 1 --2- 172.21.15.92:0/3688465873 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5612bc716950 0x5612bc7bf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.682+0000 7fdf46d9d700 1 -- 172.21.15.92:0/3688465873 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5612bc716950 msgr2=0x5612bc7bf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:24.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.682+0000 7fdf46d9d700 1 --2- 172.21.15.92:0/3688465873 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5612bc716950 0x5612bc7bf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.682+0000 7fdf46d9d700 1 -- 172.21.15.92:0/3688465873 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc73fe60 msgr2=0x5612bc75dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:24.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.682+0000 7fdf46d9d700 1 --2- 172.21.15.92:0/3688465873 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc73fe60 0x5612bc75dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.682+0000 7fdf46d9d700 1 -- 172.21.15.92:0/3688465873 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf48158200 con 0x5612bc760bc0 2023-10-06T10:38:24.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.682+0000 7fdf46d9d700 1 --2- 172.21.15.92:0/3688465873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc760bc0 0x5612bc75e010 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x5612bc7194c0 tx=0x7fdf40020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bb797181599c0f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:24.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.682+0000 7fdf24ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:38:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.682+0000 7fdf24ff9700 1 -- 172.21.15.92:0/3688465873 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40033070 con 0x5612bc760bc0 2023-10-06T10:38:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.682+0000 7fdf24ff9700 1 -- 172.21.15.92:0/3688465873 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fdf4001f420 con 0x5612bc760bc0 2023-10-06T10:38:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.682+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/3688465873 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf40007a00 con 0x5612bc760bc0 2023-10-06T10:38:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.683+0000 7fdf24ff9700 1 -- 172.21.15.92:0/3688465873 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf40001300 con 0x5612bc760bc0 2023-10-06T10:38:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.683+0000 7fdf55c7cb80 10 client.4446 fetch_fsmap learned FSMap version 6 2023-10-06T10:38:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.683+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/3688465873 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x5612bc61c470 con 0x5612bc760bc0 2023-10-06T10:38:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.683+0000 7fdf24ff9700 1 -- 172.21.15.92:0/3688465873 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf40007a00 con 0x5612bc760bc0 2023-10-06T10:38:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.683+0000 7fdf24ff9700 1 -- 172.21.15.92:0/3688465873 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 6) v1 ==== 3108+0+0 (secure 0 0 0) 0x7fdf40036020 con 0x5612bc760bc0 2023-10-06T10:38:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.683+0000 7fdf55c7cb80 10 client.4446 fetch_fsmap finished waiting for FSMap version 6 2023-10-06T10:38:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.683+0000 7fdf55c7cb80 10 client.4446 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T10:38:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.683+0000 7fdf55c7cb80 20 client.4446 populate_metadata read hostname 'smithi092' 2023-10-06T10:38:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.683+0000 7fdf55c7cb80 1 --2- 172.21.15.92:0/3688465873 >> [v2:172.21.15.124:6833/1919704285,v1:172.21.15.124:6835/1919704285] conn(0x5612bc7c2300 0x5612bc7c26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.683+0000 7fdf55c7cb80 4 client.4446 mds_command: new command op to 4306 tid=0[{"prefix": "status"}] 2023-10-06T10:38:24.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.683+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/3688465873 --> [v2:172.21.15.124:6833/1919704285,v1:172.21.15.124:6835/1919704285] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5612bc609e00 con 0x5612bc7c2300 2023-10-06T10:38:24.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.683+0000 7fdf4659c700 1 --2- 172.21.15.92:0/3688465873 >> [v2:172.21.15.124:6833/1919704285,v1:172.21.15.124:6835/1919704285] conn(0x5612bc7c2300 0x5612bc7c26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.683+0000 7fdf4659c700 1 --2- 172.21.15.92:0/3688465873 >> [v2:172.21.15.124:6833/1919704285,v1:172.21.15.124:6835/1919704285] conn(0x5612bc7c2300 0x5612bc7c26d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8329e1c558a56407 server_cookie=a71097bcc8740a8b in_seq=0 out_seq=0 2023-10-06T10:38:24.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.683+0000 7fdf24ff9700 10 client.4446 ms_handle_connect on v2:172.21.15.124:6833/1919704285 2023-10-06T10:38:24.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf24ff9700 1 -- 172.21.15.92:0/3688465873 <== mds.0 v2:172.21.15.124:6833/1919704285 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5612bc609e00 con 0x5612bc7c2300 2023-10-06T10:38:24.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf24ff9700 10 client.4446 handle_command_reply: tid=0 2023-10-06T10:38:24.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf55c7cb80 1 client.4446 shutdown 2023-10-06T10:38:24.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/3688465873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc760bc0 msgr2=0x5612bc75e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:24.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf55c7cb80 1 --2- 172.21.15.92:0/3688465873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc760bc0 0x5612bc75e010 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x5612bc7194c0 tx=0x7fdf40020000 comp rx=0 tx=0).stop 2023-10-06T10:38:24.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/3688465873 shutdown_connections 2023-10-06T10:38:24.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf55c7cb80 1 --2- 172.21.15.92:0/3688465873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612bc760bc0 0x5612bc75e010 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf55c7cb80 1 --2- 172.21.15.92:0/3688465873 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5612bc716950 0x5612bc7bf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf55c7cb80 1 --2- 172.21.15.92:0/3688465873 >> [v2:172.21.15.124:6833/1919704285,v1:172.21.15.124:6835/1919704285] conn(0x5612bc7c2300 0x5612bc7c26d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf55c7cb80 1 --2- 172.21.15.92:0/3688465873 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5612bc73fe60 0x5612bc75dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf24ff9700 0 client.4446 ms_handle_reset on v2:172.21.15.124:6833/1919704285 2023-10-06T10:38:24.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/3688465873 >> 172.21.15.92:0/3688465873 conn(0x5612bc718270 msgr2=0x5612bc76a220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:24.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/3688465873 shutdown_connections 2023-10-06T10:38:24.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf55c7cb80 1 -- 172.21.15.92:0/3688465873 wait complete. 2023-10-06T10:38:24.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf55c7cb80 20 client.4446 trim_cache size 0 max 16384 2023-10-06T10:38:24.693 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:38:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:38:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T10:38:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4306, 2023-10-06T10:38:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:38:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:38:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:38:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 7.3416465630000003, 2023-10-06T10:38:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 6, 2023-10-06T10:38:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 24, 2023-10-06T10:38:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 24, 2023-10-06T10:38:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 13.255000759 2023-10-06T10:38:24.695 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:38:24.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf4759e700 1 -- 172.21.15.92:0/107192166 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdf28066f30 msgr2=0x7fdf280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:24.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf4759e700 1 --2- 172.21.15.92:0/107192166 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdf28066f30 0x7fdf280693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdf38006dc0 tx=0x7fdf38008040 comp rx=0 tx=0).stop 2023-10-06T10:38:24.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf4759e700 1 -- 172.21.15.92:0/107192166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf4815a6e0 msgr2=0x7fdf480b09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:24.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf4759e700 1 --2- 172.21.15.92:0/107192166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf4815a6e0 0x7fdf480b09f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdf30004730 tx=0x7fdf3c01a000 comp rx=0 tx=0).stop 2023-10-06T10:38:24.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf4759e700 1 -- 172.21.15.92:0/107192166 shutdown_connections 2023-10-06T10:38:24.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf4759e700 1 --2- 172.21.15.92:0/107192166 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdf28066f30 0x7fdf280693e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf4759e700 1 --2- 172.21.15.92:0/107192166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf480b0f30 0x7fdf48157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf4759e700 1 --2- 172.21.15.92:0/107192166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf30003fe0 0x7fdf480b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf4759e700 1 --2- 172.21.15.92:0/107192166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf4815a6e0 0x7fdf480b09f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf4759e700 1 -- 172.21.15.92:0/107192166 >> 172.21.15.92:0/107192166 conn(0x7fdf48069190 msgr2=0x7fdf480675b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:24.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf4759e700 1 -- 172.21.15.92:0/107192166 shutdown_connections 2023-10-06T10:38:24.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.684+0000 7fdf4759e700 1 -- 172.21.15.92:0/107192166 wait complete. 2023-10-06T10:38:24.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T10:38:24.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.809+0000 7f96f4a05700 1 Processor -- start 2023-10-06T10:38:24.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.809+0000 7f96f4a05700 1 -- start start 2023-10-06T10:38:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.810+0000 7f96f4a05700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f96f00cbc30 0x7f96f00cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.810+0000 7f96f4a05700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96f00b2100 0x7f96f00b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.810+0000 7f96f4a05700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f00cb7a0 con 0x7f96f0159440 2023-10-06T10:38:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.810+0000 7f96f4a05700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f00b0860 con 0x7f96f00cbc30 2023-10-06T10:38:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.810+0000 7f96f4a05700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f00cb150 con 0x7f96f00b2100 2023-10-06T10:38:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.810+0000 7f96e659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f96f00cbc30 0x7f96f00cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.810+0000 7f96e659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f96f00cbc30 0x7f96f00cc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37772/0 (socket says 172.21.15.92:37772) 2023-10-06T10:38:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.810+0000 7f96e659c700 1 -- 172.21.15.92:0/3111381710 learned_addr learned my addr 172.21.15.92:0/3111381710 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.810+0000 7f96ef59e700 1 --2- 172.21.15.92:0/3111381710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96f00b2100 0x7f96f00b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.810+0000 7f96e659c700 1 -- 172.21.15.92:0/3111381710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96f00b2100 msgr2=0x7f96f00b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.810+0000 7f96e659c700 1 --2- 172.21.15.92:0/3111381710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96f00b2100 0x7f96f00b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.810+0000 7f96e659c700 1 -- 172.21.15.92:0/3111381710 >> v1:172.21.15.124:6789/0 conn(0x7f96f0159440 legacy=0x7f96f00b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.810+0000 7f96e659c700 1 -- 172.21.15.92:0/3111381710 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f00cb430 con 0x7f96f00cbc30 2023-10-06T10:38:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.810+0000 7f96e659c700 1 --2- 172.21.15.92:0/3111381710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96f00cbc30 0x7f96f00cc080 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f96d80099b0 tx=0x7f96d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33caac212eab5cf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.810+0000 7f96ee59c700 1 -- 172.21.15.92:0/3111381710 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d800b820 con 0x7f96f00cbc30 2023-10-06T10:38:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.810+0000 7f96ee59c700 1 -- 172.21.15.92:0/3111381710 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d800b980 con 0x7f96f00cbc30 2023-10-06T10:38:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.810+0000 7f96ee59c700 1 -- 172.21.15.92:0/3111381710 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d800bd20 con 0x7f96f00cbc30 2023-10-06T10:38:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.811+0000 7f96f4a05700 1 -- 172.21.15.92:0/3111381710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96f00cbc30 msgr2=0x7f96f00cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.811+0000 7f96f4a05700 1 --2- 172.21.15.92:0/3111381710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96f00cbc30 0x7f96f00cc080 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f96d80099b0 tx=0x7f96d8009250 comp rx=0 tx=0).stop 2023-10-06T10:38:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.811+0000 7f96f4a05700 1 -- 172.21.15.92:0/3111381710 shutdown_connections 2023-10-06T10:38:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.811+0000 7f96f4a05700 1 --2- 172.21.15.92:0/3111381710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96f00b2100 0x7f96f00b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.811+0000 7f96f4a05700 1 --2- 172.21.15.92:0/3111381710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96f00cbc30 0x7f96f00cc080 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.811+0000 7f96f4a05700 1 -- 172.21.15.92:0/3111381710 >> 172.21.15.92:0/3111381710 conn(0x7f96f006ece0 msgr2=0x7f96f006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.811+0000 7f96f4a05700 1 -- 172.21.15.92:0/3111381710 shutdown_connections 2023-10-06T10:38:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.811+0000 7f96f4a05700 1 -- 172.21.15.92:0/3111381710 wait complete. 2023-10-06T10:38:24.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.812+0000 7f96f4a05700 1 Processor -- start 2023-10-06T10:38:24.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.812+0000 7f96f4a05700 1 -- start start 2023-10-06T10:38:24.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.813+0000 7f96f4a05700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96f00b2100 0x7f96f00c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.813+0000 7f96f4a05700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96f00cbc30 0x7f96f00c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.813+0000 7f96f4a05700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96f0159440 0x7f96f0154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.813+0000 7f96f4a05700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f00c4810 con 0x7f96f00b2100 2023-10-06T10:38:24.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.813+0000 7f96f4a05700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f00c6b70 con 0x7f96f0159440 2023-10-06T10:38:24.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.813+0000 7f96f4a05700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f00c3c00 con 0x7f96f00cbc30 2023-10-06T10:38:24.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.813+0000 7f96eed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96f00b2100 0x7f96f00c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.813+0000 7f96eed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96f00b2100 0x7f96f00c6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37786/0 (socket says 172.21.15.92:37786) 2023-10-06T10:38:24.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.813+0000 7f96eed9d700 1 -- 172.21.15.92:0/787263819 learned_addr learned my addr 172.21.15.92:0/787263819 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:24.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.813+0000 7f96ef59e700 1 --2- 172.21.15.92:0/787263819 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96f0159440 0x7f96f0154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.813+0000 7f96e659c700 1 --2- 172.21.15.92:0/787263819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96f00cbc30 0x7f96f00c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.813+0000 7f96eed9d700 1 -- 172.21.15.92:0/787263819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96f00cbc30 msgr2=0x7f96f00c9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:24.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.813+0000 7f96eed9d700 1 --2- 172.21.15.92:0/787263819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96f00cbc30 0x7f96f00c9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.813+0000 7f96eed9d700 1 -- 172.21.15.92:0/787263819 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96f0159440 msgr2=0x7f96f0154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:24.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.813+0000 7f96eed9d700 1 --2- 172.21.15.92:0/787263819 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96f0159440 0x7f96f0154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:24.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.813+0000 7f96eed9d700 1 -- 172.21.15.92:0/787263819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d80094f0 con 0x7f96f00b2100 2023-10-06T10:38:24.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.813+0000 7f96ef59e700 1 --2- 172.21.15.92:0/787263819 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96f0159440 0x7f96f0154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:24.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.813+0000 7f96e659c700 1 --2- 172.21.15.92:0/787263819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96f00cbc30 0x7f96f00c9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:24.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.813+0000 7f96eed9d700 1 --2- 172.21.15.92:0/787263819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96f00b2100 0x7f96f00c6630 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f96e000b610 tx=0x7f96e0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:24.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.814+0000 7f96e77fe700 1 -- 172.21.15.92:0/787263819 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e000f510 con 0x7f96f00b2100 2023-10-06T10:38:24.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.814+0000 7f96e77fe700 1 -- 172.21.15.92:0/787263819 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e000f670 con 0x7f96f00b2100 2023-10-06T10:38:24.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.814+0000 7f96f4a05700 1 -- 172.21.15.92:0/787263819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f00c3ee0 con 0x7f96f00b2100 2023-10-06T10:38:24.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.814+0000 7f96e77fe700 1 -- 172.21.15.92:0/787263819 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e000fa30 con 0x7f96f00b2100 2023-10-06T10:38:24.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.814+0000 7f96f4a05700 1 -- 172.21.15.92:0/787263819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f00c44a0 con 0x7f96f00b2100 2023-10-06T10:38:24.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.815+0000 7f96f4a05700 1 -- 172.21.15.92:0/787263819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f00ca820 con 0x7f96f00b2100 2023-10-06T10:38:24.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.815+0000 7f96e77fe700 1 -- 172.21.15.92:0/787263819 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f96e001a400 con 0x7f96f00b2100 2023-10-06T10:38:24.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.815+0000 7f96e77fe700 1 --2- 172.21.15.92:0/787263819 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f96d0066f30 0x7f96d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:24.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.815+0000 7f96e77fe700 1 -- 172.21.15.92:0/787263819 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f96e00844b0 con 0x7f96f00b2100 2023-10-06T10:38:24.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.815+0000 7f96e659c700 1 --2- 172.21.15.92:0/787263819 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f96d0066f30 0x7f96d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:24.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.816+0000 7f96e659c700 1 --2- 172.21.15.92:0/787263819 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f96d0066f30 0x7f96d00693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f96d80099b0 tx=0x7f96d8016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:24.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:24.819+0000 7f96e77fe700 1 -- 172.21.15.92:0/787263819 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96e0051350 con 0x7f96f00b2100 2023-10-06T10:38:25.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:25.043+0000 7f96f4a05700 1 -- 172.21.15.92:0/787263819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f96f00b4a10 con 0x7f96f00b2100 2023-10-06T10:38:25.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:25.044+0000 7f96e77fe700 1 -- 172.21.15.92:0/787263819 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+88 (secure 0 0 0) 0x7f96e0054970 con 0x7f96f00b2100 2023-10-06T10:38:25.046 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T10:38:25.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:25.046+0000 7f96f4a05700 1 -- 172.21.15.92:0/787263819 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f96d0066f30 msgr2=0x7f96d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:25.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:25.046+0000 7f96f4a05700 1 --2- 172.21.15.92:0/787263819 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f96d0066f30 0x7f96d00693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f96d80099b0 tx=0x7f96d8016000 comp rx=0 tx=0).stop 2023-10-06T10:38:25.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:25.046+0000 7f96f4a05700 1 -- 172.21.15.92:0/787263819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96f00b2100 msgr2=0x7f96f00c6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:25.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:25.046+0000 7f96f4a05700 1 --2- 172.21.15.92:0/787263819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96f00b2100 0x7f96f00c6630 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f96e000b610 tx=0x7f96e0009120 comp rx=0 tx=0).stop 2023-10-06T10:38:25.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:25.046+0000 7f96f4a05700 1 -- 172.21.15.92:0/787263819 shutdown_connections 2023-10-06T10:38:25.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:25.046+0000 7f96f4a05700 1 --2- 172.21.15.92:0/787263819 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f96d0066f30 0x7f96d00693e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:25.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:25.046+0000 7f96f4a05700 1 --2- 172.21.15.92:0/787263819 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96f0159440 0x7f96f0154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:25.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:25.046+0000 7f96f4a05700 1 --2- 172.21.15.92:0/787263819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96f00b2100 0x7f96f00c6630 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:25.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:25.046+0000 7f96f4a05700 1 --2- 172.21.15.92:0/787263819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96f00cbc30 0x7f96f00c9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:25.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:25.046+0000 7f96f4a05700 1 -- 172.21.15.92:0/787263819 >> 172.21.15.92:0/787263819 conn(0x7f96f006ece0 msgr2=0x7f96f00b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:25.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:25.046+0000 7f96f4a05700 1 -- 172.21.15.92:0/787263819 shutdown_connections 2023-10-06T10:38:25.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:25.046+0000 7f96f4a05700 1 -- 172.21.15.92:0/787263819 wait complete. 2023-10-06T10:38:25.059 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T10:38:25.059 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T10:38:25.060 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T10:38:25.060 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-10-06T10:38:25.060 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-06T10:38:25.060 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T10:38:25.060 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T10:38:25.060 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T10:38:25.060 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-10-06T10:38:25.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:25.060 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T10:38:25.075 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T10:38:25.075 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T10:38:25.075 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T10:38:25.075 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:25.075 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T10:38:25.075 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:25.075 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:38:25.075 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:38:25.075 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T10:38:25.076 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:38:25.076 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:38:25.076 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T10:38:25.076 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:38:25.076 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:38:25.076 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T10:38:25.076 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:38:25.076 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T10:38:25.076 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42504sec preferred_lft 42504sec 2023-10-06T10:38:25.076 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T10:38:25.076 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:25.077 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T10:38:25.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:25.077 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T10:38:25.077 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T10:38:25.077 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add name ceph-brx type bridge 2023-10-06T10:38:25.077 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr flush dev ceph-brx 2023-10-06T10:38:25.078 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set ceph-brx up 2023-10-06T10:38:25.078 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T10:38:25.078 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T10:38:25.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:25.237 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T10:38:25.277 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T10:38:25.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:25.278 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T10:38:25.351 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T10:38:25.351 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T10:38:25.351 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T10:38:25.351 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T10:38:25.351 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T10:38:25.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:25.352 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T10:38:25.352 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T10:38:25.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T10:38:25.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T10:38:25.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T10:38:25.352 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T10:38:25.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:25.597 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T10:38:25.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:25.611 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list-id 2023-10-06T10:38:25.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:25.666 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T10:38:25.666 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T10:38:25.667 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-06T10:38:25.667 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-10-06T10:38:25.667 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T10:38:25.756 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-10-06T10:38:25.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:25.756 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T10:38:25.757 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T10:38:25.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-10-06T10:38:25.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T10:38:25.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-10-06T10:38:25.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-10-06T10:38:25.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-10-06T10:38:25.757 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T10:38:25.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:25.963 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T10:38:25.963 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T10:38:25.964 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-10-06T10:38:25.964 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T10:38:25.964 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T10:38:26.056 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-10-06T10:38:26.056 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T10:38:26.056 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-10-06T10:38:26.070 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-10-06T10:38:26.071 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T10:38:26.071 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-10-06T10:38:26.125 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2023-10-06T10:38:26.190 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T10:38:26.190 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T10:38:26.254 INFO:teuthology.orchestra.run.smithi092.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T10:38:26.255 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T10:38:26.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:38:26.256 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2023-10-06T10:38:26.310 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T10:38:26.310 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-10-06T10:38:26.353 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2023-10-06T10:38:26.376 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T10:38:26.376 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T10:38:26.442 INFO:teuthology.orchestra.run.smithi092.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T10:38:26.444 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi092.stderr:2023-10-06T10:38:26.440+0000 7f038d102540 -1 init, newargv = 0x55a748346790 newargc=15 2023-10-06T10:38:26.444 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi092.stderr:ceph-fuse[98359]: starting ceph client 2023-10-06T10:38:26.444 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T10:38:26.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:38:26.445 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2023-10-06T10:38:26.448 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi092.stderr:ceph-fuse[98359]: starting fuse 2023-10-06T10:38:26.461 INFO:teuthology.orchestra.run.smithi092.stdout:46 2023-10-06T10:38:26.462 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T10:38:26.462 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:38:26.462 DEBUG:teuthology.orchestra.run.smithi092:> import glob 2023-10-06T10:38:26.462 DEBUG:teuthology.orchestra.run.smithi092:> import re 2023-10-06T10:38:26.462 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:38:26.462 DEBUG:teuthology.orchestra.run.smithi092:> import subprocess 2023-10-06T10:38:26.462 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:38:26.462 DEBUG:teuthology.orchestra.run.smithi092:> def _find_admin_socket(client_name): 2023-10-06T10:38:26.462 DEBUG:teuthology.orchestra.run.smithi092:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-10-06T10:38:26.463 DEBUG:teuthology.orchestra.run.smithi092:> files = glob.glob(asok_path) 2023-10-06T10:38:26.463 DEBUG:teuthology.orchestra.run.smithi092:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-10-06T10:38:26.463 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:38:26.463 DEBUG:teuthology.orchestra.run.smithi092:> # Given a non-glob path, it better be there 2023-10-06T10:38:26.463 DEBUG:teuthology.orchestra.run.smithi092:> if "*" not in asok_path: 2023-10-06T10:38:26.463 DEBUG:teuthology.orchestra.run.smithi092:> assert(len(files) == 1) 2023-10-06T10:38:26.463 DEBUG:teuthology.orchestra.run.smithi092:> return files[0] 2023-10-06T10:38:26.463 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:38:26.463 DEBUG:teuthology.orchestra.run.smithi092:> for f in files: 2023-10-06T10:38:26.463 DEBUG:teuthology.orchestra.run.smithi092:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T10:38:26.463 DEBUG:teuthology.orchestra.run.smithi092:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T10:38:26.463 DEBUG:teuthology.orchestra.run.smithi092:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T10:38:26.464 DEBUG:teuthology.orchestra.run.smithi092:> contents = proc_f.read() 2023-10-06T10:38:26.464 DEBUG:teuthology.orchestra.run.smithi092:> if mountpoint in contents: 2023-10-06T10:38:26.464 DEBUG:teuthology.orchestra.run.smithi092:> return f 2023-10-06T10:38:26.464 DEBUG:teuthology.orchestra.run.smithi092:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T10:38:26.464 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:38:26.464 DEBUG:teuthology.orchestra.run.smithi092:> print(_find_admin_socket("client.admin")) 2023-10-06T10:38:26.464 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:38:26.582 INFO:teuthology.orchestra.run.smithi092.stdout:/var/run/ceph/ceph-client.admin.98359.asok 2023-10-06T10:38:26.764 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.98359.asok 2023-10-06T10:38:26.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:38:26.765 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.98359.asok status 2023-10-06T10:38:26.915 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:38:26.915 INFO:teuthology.orchestra.run.smithi092.stdout: "metadata": { 2023-10-06T10:38:26.915 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T10:38:26.915 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T10:38:26.915 INFO:teuthology.orchestra.run.smithi092.stdout: "entity_id": "admin", 2023-10-06T10:38:26.915 INFO:teuthology.orchestra.run.smithi092.stdout: "hostname": "smithi092", 2023-10-06T10:38:26.915 INFO:teuthology.orchestra.run.smithi092.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-10-06T10:38:26.915 INFO:teuthology.orchestra.run.smithi092.stdout: "pid": "98359", 2023-10-06T10:38:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: "root": "/" 2023-10-06T10:38:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:38:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_count": 0, 2023-10-06T10:38:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_pinned_count": 0, 2023-10-06T10:38:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4461, 2023-10-06T10:38:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: "inst": { 2023-10-06T10:38:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: "name": { 2023-10-06T10:38:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "client", 2023-10-06T10:38:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: "num": 4461 2023-10-06T10:38:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:38:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2023-10-06T10:38:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2023-10-06T10:38:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.1:0", 2023-10-06T10:38:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2737237959 2023-10-06T10:38:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:38:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:38:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2023-10-06T10:38:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2023-10-06T10:38:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.1:0", 2023-10-06T10:38:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2737237959 2023-10-06T10:38:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:38:26.918 INFO:teuthology.orchestra.run.smithi092.stdout: "inst_str": "client.4461 192.168.0.1:0/2737237959", 2023-10-06T10:38:26.918 INFO:teuthology.orchestra.run.smithi092.stdout: "addr_str": "192.168.0.1:0/2737237959", 2023-10-06T10:38:26.918 INFO:teuthology.orchestra.run.smithi092.stdout: "inode_count": 1, 2023-10-06T10:38:26.918 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_epoch": 6, 2023-10-06T10:38:26.918 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch": 24, 2023-10-06T10:38:26.918 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch_barrier": 0, 2023-10-06T10:38:26.918 INFO:teuthology.orchestra.run.smithi092.stdout: "blocklisted": false, 2023-10-06T10:38:26.918 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs" 2023-10-06T10:38:26.918 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:38:26.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:38:26.925 DEBUG:teuthology.orchestra.run.smithi092:> stat --file-system '--printf=%T 2023-10-06T10:38:26.925 DEBUG:teuthology.orchestra.run.smithi092:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-10-06T10:38:26.940 INFO:teuthology.orchestra.run.smithi092.stdout:fuseblk 2023-10-06T10:38:26.940 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-10-06T10:38:26.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:38:26.942 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-10-06T10:38:27.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:38:27.006 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-10-06T10:38:27.099 INFO:teuthology.orchestra.run.smithi092.stdout:# file: . 2023-10-06T10:38:27.099 INFO:teuthology.orchestra.run.smithi092.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-10-06T10:38:27.099 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:27.099 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi092.front.sepia.ceph.com... 2023-10-06T10:38:27.099 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T10:38:27.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-10-06T10:38:27.136 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi092.stderr:ceph-fuse[98359]: fuse finished with error 0 and tester_r 0 2023-10-06T10:38:27.138 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T10:38:27.232 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T10:38:27.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:27.233 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-10-06T10:38:27.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:38:27.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.401+0000 7f172a1e4700 1 Processor -- start 2023-10-06T10:38:27.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.401+0000 7f172a1e4700 1 -- start start 2023-10-06T10:38:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f172a1e4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1724159ad0 0x7f1724159f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f172a1e4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1724068d10 con 0x7f17240cb5f0 2023-10-06T10:38:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f172a1e4700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f172415a670 con 0x7f17240ca670 2023-10-06T10:38:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f172a1e4700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1724154020 con 0x7f1724159ad0 2023-10-06T10:38:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f17237fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f17240cb5f0 0x7f17240691a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59924/0 (socket says 172.21.15.92:59924) 2023-10-06T10:38:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f17237fe700 1 -- 172.21.15.92:0/2774063431 learned_addr learned my addr 172.21.15.92:0/2774063431 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f1722ffd700 1 --2- 172.21.15.92:0/2774063431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1724159ad0 0x7f1724159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f17227fc700 1 -- 172.21.15.92:0/2774063431 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3634592806 0 0) 0x7f1724068d10 con 0x7f17240cb5f0 2023-10-06T10:38:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f17227fc700 1 -- 172.21.15.92:0/2774063431 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f170c003570 con 0x7f17240cb5f0 2023-10-06T10:38:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f17227fc700 1 -- 172.21.15.92:0/2774063431 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2688358831 0 0) 0x7f170c003570 con 0x7f17240cb5f0 2023-10-06T10:38:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f17227fc700 1 -- 172.21.15.92:0/2774063431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1724159ad0 msgr2=0x7f1724159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f17227fc700 1 --2- 172.21.15.92:0/2774063431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1724159ad0 0x7f1724159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f17227fc700 1 -- 172.21.15.92:0/2774063431 >> v1:172.21.15.124:6789/0 conn(0x7f17240ca670 legacy=0x7f17240ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f17227fc700 1 -- 172.21.15.92:0/2774063431 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172415a950 con 0x7f17240cb5f0 2023-10-06T10:38:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f17227fc700 1 -- 172.21.15.92:0/2774063431 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f1714001470 con 0x7f17240cb5f0 2023-10-06T10:38:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f1722ffd700 1 --2- 172.21.15.92:0/2774063431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1724159ad0 0x7f1724159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f17227fc700 1 -- 172.21.15.92:0/2774063431 >> v1:172.21.15.92:6789/0 conn(0x7f17240cb5f0 legacy=0x7f17240691a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f17227fc700 1 --2- 172.21.15.92:0/2774063431 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f170c003a90 0x7f170c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f17227fc700 1 --2- 172.21.15.92:0/2774063431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1724159ad0 0x7f170c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f17227fc700 1 --2- 172.21.15.92:0/2774063431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f170c004d50 0x7f170c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f17227fc700 1 -- 172.21.15.92:0/2774063431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1724154020 con 0x7f170c004d50 2023-10-06T10:38:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f17227fc700 1 -- 172.21.15.92:0/2774063431 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c00bc40 con 0x7f170c003a90 2023-10-06T10:38:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f17227fc700 1 -- 172.21.15.92:0/2774063431 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170c00be00 con 0x7f1724159ad0 2023-10-06T10:38:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f17237fe700 1 --2- 172.21.15.92:0/2774063431 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f170c003a90 0x7f170c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f1723fff700 1 --2- 172.21.15.92:0/2774063431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f170c004d50 0x7f170c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.402+0000 7f1722ffd700 1 --2- 172.21.15.92:0/2774063431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1724159ad0 0x7f170c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.403+0000 7f1723fff700 1 -- 172.21.15.92:0/2774063431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1724159ad0 msgr2=0x7f170c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.403+0000 7f1723fff700 1 --2- 172.21.15.92:0/2774063431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1724159ad0 0x7f170c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:27.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.403+0000 7f1723fff700 1 -- 172.21.15.92:0/2774063431 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f170c003a90 msgr2=0x7f170c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:27.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.403+0000 7f1723fff700 1 --2- 172.21.15.92:0/2774063431 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f170c003a90 0x7f170c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:27.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.403+0000 7f1723fff700 1 -- 172.21.15.92:0/2774063431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c003570 con 0x7f170c004d50 2023-10-06T10:38:27.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.403+0000 7f1723fff700 1 --2- 172.21.15.92:0/2774063431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f170c004d50 0x7f170c0051c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f171c001fc0 tx=0x7f171c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9121de5448ce676 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:27.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.403+0000 7f17227fc700 1 -- 172.21.15.92:0/2774063431 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c001950 con 0x7f170c004d50 2023-10-06T10:38:27.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.403+0000 7f17227fc700 1 -- 172.21.15.92:0/2774063431 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171c00c8b0 con 0x7f170c004d50 2023-10-06T10:38:27.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.403+0000 7f17227fc700 1 -- 172.21.15.92:0/2774063431 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c00cc50 con 0x7f170c004d50 2023-10-06T10:38:27.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.403+0000 7f172a1e4700 1 -- 172.21.15.92:0/2774063431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f170c004d50 msgr2=0x7f170c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:27.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.403+0000 7f172a1e4700 1 --2- 172.21.15.92:0/2774063431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f170c004d50 0x7f170c0051c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f171c001fc0 tx=0x7f171c001640 comp rx=0 tx=0).stop 2023-10-06T10:38:27.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.403+0000 7f1723fff700 1 -- 172.21.15.92:0/2774063431 reap_dead start 2023-10-06T10:38:27.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.403+0000 7f172a1e4700 1 -- 172.21.15.92:0/2774063431 shutdown_connections 2023-10-06T10:38:27.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.403+0000 7f172a1e4700 1 -- 172.21.15.92:0/2774063431 >> 172.21.15.92:0/2774063431 conn(0x7f17240707d0 msgr2=0x7f1724070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:27.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.404+0000 7f172a1e4700 1 -- 172.21.15.92:0/2774063431 shutdown_connections 2023-10-06T10:38:27.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.404+0000 7f172a1e4700 1 -- 172.21.15.92:0/2774063431 wait complete. 2023-10-06T10:38:27.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.405+0000 7f172a1e4700 1 Processor -- start 2023-10-06T10:38:27.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.405+0000 7f172a1e4700 1 -- start start 2023-10-06T10:38:27.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.405+0000 7f172a1e4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1724151110 0x7f17240b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:27.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.405+0000 7f172a1e4700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1724151810 0x7f17241544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:27.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.405+0000 7f172a1e4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17240b3540 0x7f17240b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:27.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.405+0000 7f172a1e4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17240b3f20 con 0x7f17240b3540 2023-10-06T10:38:27.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.405+0000 7f172a1e4700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1724151f90 con 0x7f1724151810 2023-10-06T10:38:27.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.405+0000 7f172a1e4700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1724154a30 con 0x7f1724151110 2023-10-06T10:38:27.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.406+0000 7f1723fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17240b3540 0x7f17240b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:27.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.406+0000 7f1723fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17240b3540 0x7f17240b39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37812/0 (socket says 172.21.15.92:37812) 2023-10-06T10:38:27.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.406+0000 7f1723fff700 1 -- 172.21.15.92:0/651543024 learned_addr learned my addr 172.21.15.92:0/651543024 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:27.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.406+0000 7f17237fe700 1 --2- 172.21.15.92:0/651543024 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1724151110 0x7f17240b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:27.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.406+0000 7f1722ffd700 1 --2- 172.21.15.92:0/651543024 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1724151810 0x7f17241544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:27.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.406+0000 7f1723fff700 1 -- 172.21.15.92:0/651543024 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1724151110 msgr2=0x7f17240b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:27.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.406+0000 7f1723fff700 1 --2- 172.21.15.92:0/651543024 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1724151110 0x7f17240b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:27.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.406+0000 7f1723fff700 1 -- 172.21.15.92:0/651543024 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1724151810 msgr2=0x7f17241544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:27.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.406+0000 7f1723fff700 1 --2- 172.21.15.92:0/651543024 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1724151810 0x7f17241544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:27.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.406+0000 7f1723fff700 1 -- 172.21.15.92:0/651543024 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17240c2040 con 0x7f17240b3540 2023-10-06T10:38:27.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.406+0000 7f1723fff700 1 --2- 172.21.15.92:0/651543024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17240b3540 0x7f17240b39e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f171c004000 tx=0x7f171c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:27.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.406+0000 7f170b7fe700 1 -- 172.21.15.92:0/651543024 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c001950 con 0x7f17240b3540 2023-10-06T10:38:27.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.406+0000 7f170b7fe700 1 -- 172.21.15.92:0/651543024 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171c00c6d0 con 0x7f17240b3540 2023-10-06T10:38:27.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.406+0000 7f172a1e4700 1 -- 172.21.15.92:0/651543024 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17240c2320 con 0x7f17240b3540 2023-10-06T10:38:27.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.406+0000 7f170b7fe700 1 -- 172.21.15.92:0/651543024 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c006dd0 con 0x7f17240b3540 2023-10-06T10:38:27.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.406+0000 7f172a1e4700 1 -- 172.21.15.92:0/651543024 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17240c28e0 con 0x7f17240b3540 2023-10-06T10:38:27.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.407+0000 7f172a1e4700 1 -- 172.21.15.92:0/651543024 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1724064a90 con 0x7f17240b3540 2023-10-06T10:38:27.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.407+0000 7f170b7fe700 1 -- 172.21.15.92:0/651543024 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f171c019070 con 0x7f17240b3540 2023-10-06T10:38:27.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.408+0000 7f170b7fe700 1 --2- 172.21.15.92:0/651543024 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1710066f30 0x7f17100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:27.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.408+0000 7f170b7fe700 1 -- 172.21.15.92:0/651543024 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f171c086110 con 0x7f17240b3540 2023-10-06T10:38:27.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.408+0000 7f17237fe700 1 --2- 172.21.15.92:0/651543024 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1710066f30 0x7f17100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:27.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.408+0000 7f17237fe700 1 --2- 172.21.15.92:0/651543024 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1710066f30 0x7f17100693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f17140010e0 tx=0x7f1714001e50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:27.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.412+0000 7f170b7fe700 1 -- 172.21.15.92:0/651543024 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f171c052e90 con 0x7f17240b3540 2023-10-06T10:38:27.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.597+0000 7f172a1e4700 1 -- 172.21.15.92:0/651543024 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17240686c0 con 0x7f17240b3540 2023-10-06T10:38:27.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.598+0000 7f170b7fe700 1 -- 172.21.15.92:0/651543024 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18707 (secure 0 0 0) 0x7f171c017020 con 0x7f17240b3540 2023-10-06T10:38:27.600 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:27.600 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:21.760233+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:38:14.368713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T10:38:15.692344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T10:38:19.382553+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:38:27.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.599+0000 7f172a1e4700 1 -- 172.21.15.92:0/651543024 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1710066f30 msgr2=0x7f17100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:27.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.600+0000 7f172a1e4700 1 --2- 172.21.15.92:0/651543024 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1710066f30 0x7f17100693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f17140010e0 tx=0x7f1714001e50 comp rx=0 tx=0).stop 2023-10-06T10:38:27.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.600+0000 7f172a1e4700 1 -- 172.21.15.92:0/651543024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17240b3540 msgr2=0x7f17240b39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:27.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.600+0000 7f172a1e4700 1 --2- 172.21.15.92:0/651543024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17240b3540 0x7f17240b39e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f171c004000 tx=0x7f171c015000 comp rx=0 tx=0).stop 2023-10-06T10:38:27.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.600+0000 7f172a1e4700 1 -- 172.21.15.92:0/651543024 shutdown_connections 2023-10-06T10:38:27.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.600+0000 7f172a1e4700 1 --2- 172.21.15.92:0/651543024 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1710066f30 0x7f17100693e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:27.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.600+0000 7f172a1e4700 1 --2- 172.21.15.92:0/651543024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17240b3540 0x7f17240b39e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:27.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.600+0000 7f172a1e4700 1 --2- 172.21.15.92:0/651543024 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1724151110 0x7f17240b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:27.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.600+0000 7f172a1e4700 1 --2- 172.21.15.92:0/651543024 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1724151810 0x7f17241544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:27.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.600+0000 7f172a1e4700 1 -- 172.21.15.92:0/651543024 >> 172.21.15.92:0/651543024 conn(0x7f17240707d0 msgr2=0x7f1724152900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:27.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.600+0000 7f172a1e4700 1 -- 172.21.15.92:0/651543024 shutdown_connections 2023-10-06T10:38:27.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.600+0000 7f172a1e4700 1 -- 172.21.15.92:0/651543024 wait complete. 2023-10-06T10:38:27.614 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:38:27.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.725+0000 7ff598028700 1 Processor -- start 2023-10-06T10:38:27.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.725+0000 7ff598028700 1 -- start start 2023-10-06T10:38:27.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.726+0000 7ff598028700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff59015cab0 0x7ff59006f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:27.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.726+0000 7ff598028700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff59006ecd0 con 0x7ff5900675b0 2023-10-06T10:38:27.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.726+0000 7ff598028700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5900cb870 con 0x7ff5900caa90 2023-10-06T10:38:27.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.726+0000 7ff598028700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5900cbb50 con 0x7ff59015cab0 2023-10-06T10:38:27.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.726+0000 7ff5965c5700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff5900675b0 0x7ff5900679a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59932/0 (socket says 172.21.15.92:59932) 2023-10-06T10:38:27.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.726+0000 7ff5965c5700 1 -- 172.21.15.92:0/3434222078 learned_addr learned my addr 172.21.15.92:0/3434222078 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:27.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff595dc4700 1 --2- 172.21.15.92:0/3434222078 >> v2:172.21.15.124:3300/0 conn(0x7ff59015cab0 0x7ff59006f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:27.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff594dc2700 1 -- 172.21.15.92:0/3434222078 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2856042660 0 0) 0x7ff59006ecd0 con 0x7ff5900675b0 2023-10-06T10:38:27.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff594dc2700 1 -- 172.21.15.92:0/3434222078 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff57c003570 con 0x7ff5900675b0 2023-10-06T10:38:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff594dc2700 1 -- 172.21.15.92:0/3434222078 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 369731358 0 0) 0x7ff57c003570 con 0x7ff5900675b0 2023-10-06T10:38:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff594dc2700 1 -- 172.21.15.92:0/3434222078 >> v2:172.21.15.124:3300/0 conn(0x7ff59015cab0 msgr2=0x7ff59006f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff594dc2700 1 --2- 172.21.15.92:0/3434222078 >> v2:172.21.15.124:3300/0 conn(0x7ff59015cab0 0x7ff59006f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff594dc2700 1 -- 172.21.15.92:0/3434222078 >> v1:172.21.15.124:6789/0 conn(0x7ff5900caa90 legacy=0x7ff5900cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff594dc2700 1 -- 172.21.15.92:0/3434222078 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59006e680 con 0x7ff5900675b0 2023-10-06T10:38:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff594dc2700 1 -- 172.21.15.92:0/3434222078 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7ff588001470 con 0x7ff5900675b0 2023-10-06T10:38:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff594dc2700 1 -- 172.21.15.92:0/3434222078 >> v1:172.21.15.92:6789/0 conn(0x7ff5900675b0 legacy=0x7ff5900679a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff594dc2700 1 --2- 172.21.15.92:0/3434222078 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff57c003a60 0x7ff57c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff594dc2700 1 --2- 172.21.15.92:0/3434222078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff57c004810 0x7ff57c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff594dc2700 1 --2- 172.21.15.92:0/3434222078 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff57c0051c0 0x7ff57c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff594dc2700 1 -- 172.21.15.92:0/3434222078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5900cbb50 con 0x7ff57c004810 2023-10-06T10:38:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff594dc2700 1 -- 172.21.15.92:0/3434222078 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c00c160 con 0x7ff57c0051c0 2023-10-06T10:38:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff594dc2700 1 -- 172.21.15.92:0/3434222078 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c00c320 con 0x7ff57c003a60 2023-10-06T10:38:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff5955c3700 1 --2- 172.21.15.92:0/3434222078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff57c004810 0x7ff57c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff595dc4700 1 --2- 172.21.15.92:0/3434222078 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff57c003a60 0x7ff57c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff5965c5700 1 --2- 172.21.15.92:0/3434222078 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff57c0051c0 0x7ff57c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff5955c3700 1 -- 172.21.15.92:0/3434222078 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff57c003a60 msgr2=0x7ff57c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff5955c3700 1 --2- 172.21.15.92:0/3434222078 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff57c003a60 0x7ff57c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff5955c3700 1 -- 172.21.15.92:0/3434222078 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff57c0051c0 msgr2=0x7ff57c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.727+0000 7ff5955c3700 1 --2- 172.21.15.92:0/3434222078 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff57c0051c0 0x7ff57c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:27.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.728+0000 7ff5965c5700 1 -- 172.21.15.92:0/3434222078 reap_dead start 2023-10-06T10:38:27.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.728+0000 7ff5955c3700 1 -- 172.21.15.92:0/3434222078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff57c003570 con 0x7ff57c004810 2023-10-06T10:38:27.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.728+0000 7ff5955c3700 1 --2- 172.21.15.92:0/3434222078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff57c004810 0x7ff57c004c80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff5800018b0 tx=0x7ff580007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700fe6dc07bd0ac9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:27.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.728+0000 7ff594dc2700 1 -- 172.21.15.92:0/3434222078 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff580012420 con 0x7ff57c004810 2023-10-06T10:38:27.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.728+0000 7ff594dc2700 1 -- 172.21.15.92:0/3434222078 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5800125a0 con 0x7ff57c004810 2023-10-06T10:38:27.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.728+0000 7ff594dc2700 1 -- 172.21.15.92:0/3434222078 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff580012960 con 0x7ff57c004810 2023-10-06T10:38:27.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.728+0000 7ff598028700 1 -- 172.21.15.92:0/3434222078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff57c004810 msgr2=0x7ff57c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:27.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.728+0000 7ff598028700 1 --2- 172.21.15.92:0/3434222078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff57c004810 0x7ff57c004c80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff5800018b0 tx=0x7ff580007d30 comp rx=0 tx=0).stop 2023-10-06T10:38:27.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.728+0000 7ff598028700 1 -- 172.21.15.92:0/3434222078 shutdown_connections 2023-10-06T10:38:27.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.728+0000 7ff598028700 1 --2- 172.21.15.92:0/3434222078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff57c004810 0x7ff57c004c80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:27.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.728+0000 7ff598028700 1 -- 172.21.15.92:0/3434222078 >> 172.21.15.92:0/3434222078 conn(0x7ff5900cdeb0 msgr2=0x7ff5900c9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:27.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.728+0000 7ff598028700 1 -- 172.21.15.92:0/3434222078 shutdown_connections 2023-10-06T10:38:27.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.728+0000 7ff598028700 1 -- 172.21.15.92:0/3434222078 wait complete. 2023-10-06T10:38:27.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.730+0000 7ff598028700 1 Processor -- start 2023-10-06T10:38:27.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.730+0000 7ff598028700 1 -- start start 2023-10-06T10:38:27.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.730+0000 7ff598028700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff57c004810 0x7ff5901543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:27.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.730+0000 7ff598028700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff5900b2610 0x7ff5900b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:27.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.730+0000 7ff598028700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5900b10d0 0x7ff59015e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:27.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.730+0000 7ff598028700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5900b2180 con 0x7ff5900b10d0 2023-10-06T10:38:27.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.730+0000 7ff598028700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5901548e0 con 0x7ff5900b2610 2023-10-06T10:38:27.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.730+0000 7ff598028700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5900b1570 con 0x7ff57c004810 2023-10-06T10:38:27.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.730+0000 7ff5965c5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5900b10d0 0x7ff59015e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:27.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.730+0000 7ff5965c5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5900b10d0 0x7ff59015e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37824/0 (socket says 172.21.15.92:37824) 2023-10-06T10:38:27.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.730+0000 7ff5965c5700 1 -- 172.21.15.92:0/3584628386 learned_addr learned my addr 172.21.15.92:0/3584628386 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:27.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.730+0000 7ff595dc4700 1 --2- 172.21.15.92:0/3584628386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff57c004810 0x7ff5901543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:27.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.730+0000 7ff5955c3700 1 --2- 172.21.15.92:0/3584628386 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff5900b2610 0x7ff5900b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:27.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.731+0000 7ff5965c5700 1 -- 172.21.15.92:0/3584628386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff57c004810 msgr2=0x7ff5901543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:27.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.731+0000 7ff5965c5700 1 --2- 172.21.15.92:0/3584628386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff57c004810 0x7ff5901543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:27.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.731+0000 7ff5965c5700 1 -- 172.21.15.92:0/3584628386 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff5900b2610 msgr2=0x7ff5900b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:27.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.731+0000 7ff5965c5700 1 --2- 172.21.15.92:0/3584628386 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff5900b2610 0x7ff5900b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:27.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.731+0000 7ff5965c5700 1 -- 172.21.15.92:0/3584628386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5900b1850 con 0x7ff5900b10d0 2023-10-06T10:38:27.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.731+0000 7ff5955c3700 1 --2- 172.21.15.92:0/3584628386 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff5900b2610 0x7ff5900b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:27.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.731+0000 7ff5965c5700 1 --2- 172.21.15.92:0/3584628386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5900b10d0 0x7ff59015e460 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff5900caf30 tx=0x7ff59006ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:27.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.731+0000 7ff585ffb700 1 -- 172.21.15.92:0/3584628386 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff588018030 con 0x7ff5900b10d0 2023-10-06T10:38:27.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.731+0000 7ff585ffb700 1 -- 172.21.15.92:0/3584628386 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58800f630 con 0x7ff5900b10d0 2023-10-06T10:38:27.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.731+0000 7ff598028700 1 -- 172.21.15.92:0/3584628386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5900b1b30 con 0x7ff5900b10d0 2023-10-06T10:38:27.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.731+0000 7ff585ffb700 1 -- 172.21.15.92:0/3584628386 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58800fa10 con 0x7ff5900b10d0 2023-10-06T10:38:27.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.731+0000 7ff598028700 1 -- 172.21.15.92:0/3584628386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5900aee60 con 0x7ff5900b10d0 2023-10-06T10:38:27.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.732+0000 7ff598028700 1 -- 172.21.15.92:0/3584628386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5900b22f0 con 0x7ff5900b10d0 2023-10-06T10:38:27.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.732+0000 7ff585ffb700 1 -- 172.21.15.92:0/3584628386 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff58801b070 con 0x7ff5900b10d0 2023-10-06T10:38:27.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.733+0000 7ff585ffb700 1 --2- 172.21.15.92:0/3584628386 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff574066f30 0x7ff5740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:27.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.733+0000 7ff585ffb700 1 -- 172.21.15.92:0/3584628386 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ff588019070 con 0x7ff5900b10d0 2023-10-06T10:38:27.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.733+0000 7ff595dc4700 1 --2- 172.21.15.92:0/3584628386 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff574066f30 0x7ff5740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:27.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.733+0000 7ff595dc4700 1 --2- 172.21.15.92:0/3584628386 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff574066f30 0x7ff5740693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0094d0 tx=0x7ff58c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:27.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.737+0000 7ff585ffb700 1 -- 172.21.15.92:0/3584628386 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff588051360 con 0x7ff5900b10d0 2023-10-06T10:38:27.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.921+0000 7ff598028700 1 -- 172.21.15.92:0/3584628386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff590064a90 con 0x7ff5900b10d0 2023-10-06T10:38:27.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.922+0000 7ff585ffb700 1 -- 172.21.15.92:0/3584628386 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18707 (secure 0 0 0) 0x7ff588054980 con 0x7ff5900b10d0 2023-10-06T10:38:27.925 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:27.925 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:21.760233+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:38:14.368713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T10:38:15.692344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T10:38:19.382553+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:38:27.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.925+0000 7ff598028700 1 -- 172.21.15.92:0/3584628386 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff574066f30 msgr2=0x7ff5740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:27.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.925+0000 7ff598028700 1 --2- 172.21.15.92:0/3584628386 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff574066f30 0x7ff5740693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0094d0 tx=0x7ff58c00b040 comp rx=0 tx=0).stop 2023-10-06T10:38:27.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.925+0000 7ff598028700 1 -- 172.21.15.92:0/3584628386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5900b10d0 msgr2=0x7ff59015e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:27.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.925+0000 7ff598028700 1 --2- 172.21.15.92:0/3584628386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5900b10d0 0x7ff59015e460 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff5900caf30 tx=0x7ff59006ec40 comp rx=0 tx=0).stop 2023-10-06T10:38:27.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.925+0000 7ff598028700 1 -- 172.21.15.92:0/3584628386 shutdown_connections 2023-10-06T10:38:27.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.925+0000 7ff598028700 1 --2- 172.21.15.92:0/3584628386 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff574066f30 0x7ff5740693e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:27.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.925+0000 7ff598028700 1 --2- 172.21.15.92:0/3584628386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5900b10d0 0x7ff59015e460 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:27.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.925+0000 7ff598028700 1 --2- 172.21.15.92:0/3584628386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff57c004810 0x7ff5901543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:27.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.925+0000 7ff598028700 1 --2- 172.21.15.92:0/3584628386 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff5900b2610 0x7ff5900b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:27.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.925+0000 7ff598028700 1 -- 172.21.15.92:0/3584628386 >> 172.21.15.92:0/3584628386 conn(0x7ff5900cdeb0 msgr2=0x7ff5900c2180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:27.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.925+0000 7ff598028700 1 -- 172.21.15.92:0/3584628386 shutdown_connections 2023-10-06T10:38:27.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:27.925+0000 7ff598028700 1 -- 172.21.15.92:0/3584628386 wait complete. 2023-10-06T10:38:27.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T10:38:28.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.055+0000 7f1c7a0aa700 1 Processor -- start 2023-10-06T10:38:28.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.055+0000 7f1c7a0aa700 1 -- start start 2023-10-06T10:38:28.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.056+0000 7f1c7a0aa700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1c741594d0 0x7f1c740cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:28.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.056+0000 7f1c7a0aa700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1c741598a0 0x7f1c740cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:28.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.056+0000 7f1c7a0aa700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c740cf330 con 0x7f1c740ce7e0 2023-10-06T10:38:28.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.056+0000 7f1c7a0aa700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c740cf5d0 con 0x7f1c741594d0 2023-10-06T10:38:28.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.056+0000 7f1c7a0aa700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c740cf8b0 con 0x7f1c741598a0 2023-10-06T10:38:28.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.056+0000 7f1c737fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1c741594d0 0x7f1c740cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:28.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.056+0000 7f1c737fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1c741594d0 0x7f1c740cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37828/0 (socket says 172.21.15.92:37828) 2023-10-06T10:38:28.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.056+0000 7f1c737fe700 1 -- 172.21.15.92:0/1409157949 learned_addr learned my addr 172.21.15.92:0/1409157949 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:28.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.056+0000 7f1c72ffd700 1 --2- 172.21.15.92:0/1409157949 >> v2:172.21.15.124:3300/0 conn(0x7f1c741598a0 0x7f1c740cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:28.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.056+0000 7f1c737fe700 1 -- 172.21.15.92:0/1409157949 >> v2:172.21.15.124:3300/0 conn(0x7f1c741598a0 msgr2=0x7f1c740cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:28.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.056+0000 7f1c737fe700 1 --2- 172.21.15.92:0/1409157949 >> v2:172.21.15.124:3300/0 conn(0x7f1c741598a0 0x7f1c740cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:28.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.057+0000 7f1c737fe700 1 -- 172.21.15.92:0/1409157949 >> v1:172.21.15.124:6789/0 conn(0x7f1c740ce7e0 legacy=0x7f1c740cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:28.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.057+0000 7f1c737fe700 1 -- 172.21.15.92:0/1409157949 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c740cfb90 con 0x7f1c741594d0 2023-10-06T10:38:28.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.057+0000 7f1c72ffd700 1 --2- 172.21.15.92:0/1409157949 >> v2:172.21.15.124:3300/0 conn(0x7f1c741598a0 0x7f1c740cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:28.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.057+0000 7f1c737fe700 1 --2- 172.21.15.92:0/1409157949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c741594d0 0x7f1c740cb560 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1c740c7370 tx=0x7f1c64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a600a23c5066f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:28.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.057+0000 7f1c727fc700 1 -- 172.21.15.92:0/1409157949 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6400b820 con 0x7f1c741594d0 2023-10-06T10:38:28.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.057+0000 7f1c727fc700 1 -- 172.21.15.92:0/1409157949 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6400b980 con 0x7f1c741594d0 2023-10-06T10:38:28.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.057+0000 7f1c727fc700 1 -- 172.21.15.92:0/1409157949 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6400bd20 con 0x7f1c741594d0 2023-10-06T10:38:28.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.057+0000 7f1c7a0aa700 1 -- 172.21.15.92:0/1409157949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c741594d0 msgr2=0x7f1c740cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:28.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.057+0000 7f1c7a0aa700 1 --2- 172.21.15.92:0/1409157949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c741594d0 0x7f1c740cb560 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1c740c7370 tx=0x7f1c64009250 comp rx=0 tx=0).stop 2023-10-06T10:38:28.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.057+0000 7f1c7a0aa700 1 -- 172.21.15.92:0/1409157949 shutdown_connections 2023-10-06T10:38:28.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.057+0000 7f1c7a0aa700 1 --2- 172.21.15.92:0/1409157949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c741594d0 0x7f1c740cb560 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:28.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.057+0000 7f1c7a0aa700 1 --2- 172.21.15.92:0/1409157949 >> v2:172.21.15.124:3300/0 conn(0x7f1c741598a0 0x7f1c740cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:28.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.057+0000 7f1c7a0aa700 1 -- 172.21.15.92:0/1409157949 >> 172.21.15.92:0/1409157949 conn(0x7f1c74089e50 msgr2=0x7f1c74099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:28.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.057+0000 7f1c7a0aa700 1 -- 172.21.15.92:0/1409157949 shutdown_connections 2023-10-06T10:38:28.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.057+0000 7f1c7a0aa700 1 -- 172.21.15.92:0/1409157949 wait complete. 2023-10-06T10:38:28.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.059+0000 7f1c7a0aa700 1 Processor -- start 2023-10-06T10:38:28.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.059+0000 7f1c7a0aa700 1 -- start start 2023-10-06T10:38:28.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.059+0000 7f1c7a0aa700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c740ce7e0 0x7f1c7414f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:28.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.059+0000 7f1c7a0aa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c741594d0 0x7f1c740c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:28.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.059+0000 7f1c7a0aa700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c741598a0 0x7f1c7409d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:28.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.059+0000 7f1c7a0aa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740c85b0 con 0x7f1c741594d0 2023-10-06T10:38:28.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.059+0000 7f1c7a0aa700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7409d050 con 0x7f1c741598a0 2023-10-06T10:38:28.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.059+0000 7f1c7a0aa700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7409c760 con 0x7f1c740ce7e0 2023-10-06T10:38:28.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.059+0000 7f1c72ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c741594d0 0x7f1c740c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:28.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.060+0000 7f1c72ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c741594d0 0x7f1c740c8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37842/0 (socket says 172.21.15.92:37842) 2023-10-06T10:38:28.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.060+0000 7f1c72ffd700 1 -- 172.21.15.92:0/3219856867 learned_addr learned my addr 172.21.15.92:0/3219856867 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:28.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.060+0000 7f1c73fff700 1 --2- 172.21.15.92:0/3219856867 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c741598a0 0x7f1c7409d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:28.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.060+0000 7f1c737fe700 1 --2- 172.21.15.92:0/3219856867 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c740ce7e0 0x7f1c7414f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:28.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.060+0000 7f1c72ffd700 1 -- 172.21.15.92:0/3219856867 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c740ce7e0 msgr2=0x7f1c7414f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:28.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.060+0000 7f1c72ffd700 1 --2- 172.21.15.92:0/3219856867 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c740ce7e0 0x7f1c7414f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:28.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.060+0000 7f1c72ffd700 1 -- 172.21.15.92:0/3219856867 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c741598a0 msgr2=0x7f1c7409d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:28.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.060+0000 7f1c72ffd700 1 --2- 172.21.15.92:0/3219856867 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c741598a0 0x7f1c7409d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:28.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.060+0000 7f1c72ffd700 1 -- 172.21.15.92:0/3219856867 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c640094f0 con 0x7f1c741594d0 2023-10-06T10:38:28.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.060+0000 7f1c72ffd700 1 --2- 172.21.15.92:0/3219856867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c741594d0 0x7f1c740c8950 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800c8b0 tx=0x7f1c68008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:28.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.060+0000 7f1c5b7fe700 1 -- 172.21.15.92:0/3219856867 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68011600 con 0x7f1c741594d0 2023-10-06T10:38:28.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.060+0000 7f1c5b7fe700 1 -- 172.21.15.92:0/3219856867 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c68011760 con 0x7f1c741594d0 2023-10-06T10:38:28.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.060+0000 7f1c7a0aa700 1 -- 172.21.15.92:0/3219856867 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c740c82d0 con 0x7f1c741594d0 2023-10-06T10:38:28.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.060+0000 7f1c5b7fe700 1 -- 172.21.15.92:0/3219856867 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68011b20 con 0x7f1c741594d0 2023-10-06T10:38:28.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.060+0000 7f1c7a0aa700 1 -- 172.21.15.92:0/3219856867 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7409cce0 con 0x7f1c741594d0 2023-10-06T10:38:28.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.061+0000 7f1c7a0aa700 1 -- 172.21.15.92:0/3219856867 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c74064a90 con 0x7f1c741594d0 2023-10-06T10:38:28.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.061+0000 7f1c5b7fe700 1 -- 172.21.15.92:0/3219856867 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1c680193b0 con 0x7f1c741594d0 2023-10-06T10:38:28.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.062+0000 7f1c5b7fe700 1 --2- 172.21.15.92:0/3219856867 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1c60066f30 0x7f1c600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:28.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.062+0000 7f1c5b7fe700 1 -- 172.21.15.92:0/3219856867 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f1c680847c0 con 0x7f1c741594d0 2023-10-06T10:38:28.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.062+0000 7f1c737fe700 1 --2- 172.21.15.92:0/3219856867 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1c60066f30 0x7f1c600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:28.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.062+0000 7f1c737fe700 1 --2- 172.21.15.92:0/3219856867 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1c60066f30 0x7f1c600693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1c740c7370 tx=0x7f1c64016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:28.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.066+0000 7f1c5b7fe700 1 -- 172.21.15.92:0/3219856867 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c68055070 con 0x7f1c741594d0 2023-10-06T10:38:28.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:28.291+0000 7f1c7a0aa700 1 -- 172.21.15.92:0/3219856867 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1c7409d190 con 0x7f1c741594d0 2023-10-06T10:38:29.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.278+0000 7f1c5b7fe700 1 -- 172.21.15.92:0/3219856867 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1c68054c00 con 0x7f1c741594d0 2023-10-06T10:38:29.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.280+0000 7f1c7a0aa700 1 -- 172.21.15.92:0/3219856867 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1c60066f30 msgr2=0x7f1c600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:29.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.280+0000 7f1c7a0aa700 1 --2- 172.21.15.92:0/3219856867 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1c60066f30 0x7f1c600693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1c740c7370 tx=0x7f1c64016000 comp rx=0 tx=0).stop 2023-10-06T10:38:29.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.281+0000 7f1c7a0aa700 1 -- 172.21.15.92:0/3219856867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c741594d0 msgr2=0x7f1c740c8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:29.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.281+0000 7f1c7a0aa700 1 --2- 172.21.15.92:0/3219856867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c741594d0 0x7f1c740c8950 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800c8b0 tx=0x7f1c68008040 comp rx=0 tx=0).stop 2023-10-06T10:38:29.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.281+0000 7f1c7a0aa700 1 -- 172.21.15.92:0/3219856867 shutdown_connections 2023-10-06T10:38:29.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.281+0000 7f1c7a0aa700 1 --2- 172.21.15.92:0/3219856867 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1c60066f30 0x7f1c600693e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:29.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.281+0000 7f1c7a0aa700 1 --2- 172.21.15.92:0/3219856867 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c741598a0 0x7f1c7409d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:29.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.281+0000 7f1c7a0aa700 1 --2- 172.21.15.92:0/3219856867 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c740ce7e0 0x7f1c7414f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:29.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.281+0000 7f1c7a0aa700 1 --2- 172.21.15.92:0/3219856867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c741594d0 0x7f1c740c8950 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:29.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.281+0000 7f1c7a0aa700 1 -- 172.21.15.92:0/3219856867 >> 172.21.15.92:0/3219856867 conn(0x7f1c74089e50 msgr2=0x7f1c740c6540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:29.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.281+0000 7f1c7a0aa700 1 -- 172.21.15.92:0/3219856867 shutdown_connections 2023-10-06T10:38:29.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.281+0000 7f1c7a0aa700 1 -- 172.21.15.92:0/3219856867 wait complete. 2023-10-06T10:38:29.297 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T10:38:29.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.420+0000 7fe3d2547700 1 Processor -- start 2023-10-06T10:38:29.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.420+0000 7fe3d2547700 1 -- start start 2023-10-06T10:38:29.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.421+0000 7fe3d2547700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe3cc153970 0x7fe3cc0b1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:29.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.421+0000 7fe3d2547700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3cc15a600 con 0x7fe3cc0b07c0 2023-10-06T10:38:29.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.421+0000 7fe3d2547700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3cc157600 con 0x7fe3cc0af180 2023-10-06T10:38:29.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.421+0000 7fe3d2547700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3cc06b0b0 con 0x7fe3cc153970 2023-10-06T10:38:29.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.421+0000 7fe3d0d44700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe3cc0b07c0 0x7fe3cc06a290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59942/0 (socket says 172.21.15.92:59942) 2023-10-06T10:38:29.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.421+0000 7fe3d0d44700 1 -- 172.21.15.92:0/2818809847 learned_addr learned my addr 172.21.15.92:0/2818809847 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:29.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.421+0000 7fe3d1545700 1 --2- 172.21.15.92:0/2818809847 >> v2:172.21.15.124:3300/0 conn(0x7fe3cc153970 0x7fe3cc0b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:29.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.421+0000 7fe3c3fff700 1 -- 172.21.15.92:0/2818809847 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890711332 0 0) 0x7fe3cc15a600 con 0x7fe3cc0b07c0 2023-10-06T10:38:29.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.421+0000 7fe3c3fff700 1 -- 172.21.15.92:0/2818809847 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3b8003570 con 0x7fe3cc0b07c0 2023-10-06T10:38:29.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.422+0000 7fe3c3fff700 1 -- 172.21.15.92:0/2818809847 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2490081685 0 0) 0x7fe3b8003570 con 0x7fe3cc0b07c0 2023-10-06T10:38:29.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.422+0000 7fe3d1545700 1 -- 172.21.15.92:0/2818809847 >> v1:172.21.15.124:6789/0 conn(0x7fe3cc0af180 legacy=0x7fe3cc06a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:29.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.422+0000 7fe3d1545700 1 -- 172.21.15.92:0/2818809847 >> v1:172.21.15.92:6789/0 conn(0x7fe3cc0b07c0 legacy=0x7fe3cc06a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:29.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.422+0000 7fe3d1545700 1 -- 172.21.15.92:0/2818809847 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc0b0cd0 con 0x7fe3cc153970 2023-10-06T10:38:29.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.422+0000 7fe3d1545700 1 --2- 172.21.15.92:0/2818809847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3cc153970 0x7fe3cc0b1120 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80098e0 tx=0x7fe3c80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e76c35fd44ebd1a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:29.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.422+0000 7fe3c3fff700 1 -- 172.21.15.92:0/2818809847 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc001470 con 0x7fe3cc153970 2023-10-06T10:38:29.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.422+0000 7fe3c3fff700 1 -- 172.21.15.92:0/2818809847 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c8011420 con 0x7fe3cc153970 2023-10-06T10:38:29.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.422+0000 7fe3c3fff700 1 -- 172.21.15.92:0/2818809847 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80117c0 con 0x7fe3cc153970 2023-10-06T10:38:29.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.422+0000 7fe3d2547700 1 -- 172.21.15.92:0/2818809847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3cc153970 msgr2=0x7fe3cc0b1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:29.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.422+0000 7fe3d2547700 1 --2- 172.21.15.92:0/2818809847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3cc153970 0x7fe3cc0b1120 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80098e0 tx=0x7fe3c80091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:29.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.423+0000 7fe3d2547700 1 -- 172.21.15.92:0/2818809847 shutdown_connections 2023-10-06T10:38:29.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.423+0000 7fe3d2547700 1 --2- 172.21.15.92:0/2818809847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3cc153970 0x7fe3cc0b1120 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:29.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.423+0000 7fe3d2547700 1 -- 172.21.15.92:0/2818809847 >> 172.21.15.92:0/2818809847 conn(0x7fe3cc069140 msgr2=0x7fe3cc0aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:29.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.423+0000 7fe3d2547700 1 -- 172.21.15.92:0/2818809847 shutdown_connections 2023-10-06T10:38:29.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.423+0000 7fe3d2547700 1 -- 172.21.15.92:0/2818809847 wait complete. 2023-10-06T10:38:29.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.424+0000 7fe3d2547700 1 Processor -- start 2023-10-06T10:38:29.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.424+0000 7fe3d2547700 1 -- start start 2023-10-06T10:38:29.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3d2547700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3cc0af180 0x7fe3cc153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:29.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3d2547700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3cc0b07c0 0x7fe3cc0cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:29.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3d2547700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3cc153970 0x7fe3cc0cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:29.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3d2547700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc148780 con 0x7fe3cc0b07c0 2023-10-06T10:38:29.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3d2547700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc0cb370 con 0x7fe3cc0af180 2023-10-06T10:38:29.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3d2547700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc1489d0 con 0x7fe3cc153970 2023-10-06T10:38:29.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3d0d44700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3cc0b07c0 0x7fe3cc0cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:29.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3d0d44700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3cc0b07c0 0x7fe3cc0cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37850/0 (socket says 172.21.15.92:37850) 2023-10-06T10:38:29.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3d0d44700 1 -- 172.21.15.92:0/325261436 learned_addr learned my addr 172.21.15.92:0/325261436 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:29.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3d1545700 1 --2- 172.21.15.92:0/325261436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3cc0af180 0x7fe3cc153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:29.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3d1d46700 1 --2- 172.21.15.92:0/325261436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3cc153970 0x7fe3cc0cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:29.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3d0d44700 1 -- 172.21.15.92:0/325261436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3cc153970 msgr2=0x7fe3cc0cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:29.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3d0d44700 1 --2- 172.21.15.92:0/325261436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3cc153970 0x7fe3cc0cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:29.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3d0d44700 1 -- 172.21.15.92:0/325261436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3cc0af180 msgr2=0x7fe3cc153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:29.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3d0d44700 1 --2- 172.21.15.92:0/325261436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3cc0af180 0x7fe3cc153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:29.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3d0d44700 1 -- 172.21.15.92:0/325261436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c8009420 con 0x7fe3cc0b07c0 2023-10-06T10:38:29.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3d1545700 1 --2- 172.21.15.92:0/325261436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3cc0af180 0x7fe3cc153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:29.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3d0d44700 1 --2- 172.21.15.92:0/325261436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3cc0b07c0 0x7fe3cc0cb800 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00cb90 tx=0x7fe3bc001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:29.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3c17fa700 1 -- 172.21.15.92:0/325261436 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc007150 con 0x7fe3cc0b07c0 2023-10-06T10:38:29.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3c17fa700 1 -- 172.21.15.92:0/325261436 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3bc0072d0 con 0x7fe3cc0b07c0 2023-10-06T10:38:29.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3d2547700 1 -- 172.21.15.92:0/325261436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3cc0cb000 con 0x7fe3cc0b07c0 2023-10-06T10:38:29.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3c17fa700 1 -- 172.21.15.92:0/325261436 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc007690 con 0x7fe3cc0b07c0 2023-10-06T10:38:29.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.425+0000 7fe3d2547700 1 -- 172.21.15.92:0/325261436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3cc1544d0 con 0x7fe3cc0b07c0 2023-10-06T10:38:29.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.426+0000 7fe3d2547700 1 -- 172.21.15.92:0/325261436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3cc0cae60 con 0x7fe3cc0b07c0 2023-10-06T10:38:29.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.427+0000 7fe3c17fa700 1 -- 172.21.15.92:0/325261436 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe3bc004a40 con 0x7fe3cc0b07c0 2023-10-06T10:38:29.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.427+0000 7fe3c17fa700 1 --2- 172.21.15.92:0/325261436 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe3b0066f30 0x7fe3b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:29.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.427+0000 7fe3d1545700 1 --2- 172.21.15.92:0/325261436 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe3b0066f30 0x7fe3b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:29.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.427+0000 7fe3c17fa700 1 -- 172.21.15.92:0/325261436 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe3bc083e00 con 0x7fe3cc0b07c0 2023-10-06T10:38:29.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.428+0000 7fe3d1545700 1 --2- 172.21.15.92:0/325261436 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe3b0066f30 0x7fe3b00693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80091e0 tx=0x7fe3c8016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:29.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.432+0000 7fe3c17fa700 1 -- 172.21.15.92:0/325261436 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3cc0cae60 con 0x7fe3cc0b07c0 2023-10-06T10:38:29.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:29.658+0000 7fe3d2547700 1 -- 172.21.15.92:0/325261436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe3cc0cae60 con 0x7fe3cc0b07c0 2023-10-06T10:38:30.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.402+0000 7fe3c17fa700 1 -- 172.21.15.92:0/325261436 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe3cc0cae60 con 0x7fe3cc0b07c0 2023-10-06T10:38:30.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.404+0000 7fe3d2547700 1 -- 172.21.15.92:0/325261436 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe3b0066f30 msgr2=0x7fe3b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:30.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.404+0000 7fe3d2547700 1 --2- 172.21.15.92:0/325261436 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe3b0066f30 0x7fe3b00693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80091e0 tx=0x7fe3c8016000 comp rx=0 tx=0).stop 2023-10-06T10:38:30.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.404+0000 7fe3d2547700 1 -- 172.21.15.92:0/325261436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3cc0b07c0 msgr2=0x7fe3cc0cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:30.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.404+0000 7fe3d2547700 1 --2- 172.21.15.92:0/325261436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3cc0b07c0 0x7fe3cc0cb800 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00cb90 tx=0x7fe3bc001ad0 comp rx=0 tx=0).stop 2023-10-06T10:38:30.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.404+0000 7fe3d2547700 1 -- 172.21.15.92:0/325261436 shutdown_connections 2023-10-06T10:38:30.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.404+0000 7fe3d2547700 1 --2- 172.21.15.92:0/325261436 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe3b0066f30 0x7fe3b00693e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:30.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.404+0000 7fe3d2547700 1 --2- 172.21.15.92:0/325261436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3cc153970 0x7fe3cc0cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:30.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.404+0000 7fe3d2547700 1 --2- 172.21.15.92:0/325261436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3cc0af180 0x7fe3cc153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:30.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.404+0000 7fe3d2547700 1 --2- 172.21.15.92:0/325261436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3cc0b07c0 0x7fe3cc0cb800 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:30.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.404+0000 7fe3d2547700 1 -- 172.21.15.92:0/325261436 >> 172.21.15.92:0/325261436 conn(0x7fe3cc069140 msgr2=0x7fe3cc0ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:30.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.405+0000 7fe3d2547700 1 -- 172.21.15.92:0/325261436 shutdown_connections 2023-10-06T10:38:30.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.405+0000 7fe3d2547700 1 -- 172.21.15.92:0/325261436 wait complete. 2023-10-06T10:38:30.419 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:38:30.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.533+0000 7ff471630700 1 Processor -- start 2023-10-06T10:38:30.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.533+0000 7ff471630700 1 -- start start 2023-10-06T10:38:30.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.534+0000 7ff471630700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff46c159560 0x7ff46c0aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:30.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.534+0000 7ff471630700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff46c0b49a0 0x7ff46c0aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:30.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.534+0000 7ff471630700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff46c0b0760 0x7ff46c0b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:30.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.534+0000 7ff471630700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff46c0b0c60 con 0x7ff46c0b0760 2023-10-06T10:38:30.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.534+0000 7ff471630700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff46c0b0f40 con 0x7ff46c159560 2023-10-06T10:38:30.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.534+0000 7ff471630700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c0b1220 con 0x7ff46c0b49a0 2023-10-06T10:38:30.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.534+0000 7ff46b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff46c0b0760 0x7ff46c0b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:30.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.534+0000 7ff46b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff46c0b0760 0x7ff46c0b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37864/0 (socket says 172.21.15.92:37864) 2023-10-06T10:38:30.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.534+0000 7ff46b7fe700 1 -- 172.21.15.92:0/1951125948 learned_addr learned my addr 172.21.15.92:0/1951125948 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:30.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.535+0000 7ff46affd700 1 --2- 172.21.15.92:0/1951125948 >> v2:172.21.15.124:3300/0 conn(0x7ff46c159560 0x7ff46c0aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:30.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.535+0000 7ff4627fc700 1 --2- 172.21.15.92:0/1951125948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff46c0b49a0 0x7ff46c0aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:30.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.535+0000 7ff46b7fe700 1 -- 172.21.15.92:0/1951125948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff46c0b49a0 msgr2=0x7ff46c0aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:30.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.535+0000 7ff46b7fe700 1 --2- 172.21.15.92:0/1951125948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff46c0b49a0 0x7ff46c0aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:30.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.535+0000 7ff46b7fe700 1 -- 172.21.15.92:0/1951125948 >> v2:172.21.15.124:3300/0 conn(0x7ff46c159560 msgr2=0x7ff46c0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:30.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.535+0000 7ff46b7fe700 1 --2- 172.21.15.92:0/1951125948 >> v2:172.21.15.124:3300/0 conn(0x7ff46c159560 0x7ff46c0aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:30.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.535+0000 7ff46b7fe700 1 -- 172.21.15.92:0/1951125948 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46c0b1500 con 0x7ff46c0b0760 2023-10-06T10:38:30.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.535+0000 7ff46b7fe700 1 --2- 172.21.15.92:0/1951125948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff46c0b0760 0x7ff46c0b3890 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff46c088c80 tx=0x7ff464009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c612a198ca171b3e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:30.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.535+0000 7ff46a7fc700 1 -- 172.21.15.92:0/1951125948 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46400e070 con 0x7ff46c0b0760 2023-10-06T10:38:30.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.535+0000 7ff46a7fc700 1 -- 172.21.15.92:0/1951125948 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46400ba50 con 0x7ff46c0b0760 2023-10-06T10:38:30.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.535+0000 7ff46a7fc700 1 -- 172.21.15.92:0/1951125948 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4640092e0 con 0x7ff46c0b0760 2023-10-06T10:38:30.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.535+0000 7ff471630700 1 -- 172.21.15.92:0/1951125948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff46c0b0760 msgr2=0x7ff46c0b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:30.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.535+0000 7ff471630700 1 --2- 172.21.15.92:0/1951125948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff46c0b0760 0x7ff46c0b3890 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff46c088c80 tx=0x7ff464009840 comp rx=0 tx=0).stop 2023-10-06T10:38:30.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.536+0000 7ff471630700 1 -- 172.21.15.92:0/1951125948 shutdown_connections 2023-10-06T10:38:30.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.536+0000 7ff471630700 1 --2- 172.21.15.92:0/1951125948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff46c0b0760 0x7ff46c0b3890 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:30.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.536+0000 7ff471630700 1 --2- 172.21.15.92:0/1951125948 >> v2:172.21.15.124:3300/0 conn(0x7ff46c159560 0x7ff46c0aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:30.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.536+0000 7ff471630700 1 --2- 172.21.15.92:0/1951125948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff46c0b49a0 0x7ff46c0aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:30.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.536+0000 7ff471630700 1 -- 172.21.15.92:0/1951125948 >> 172.21.15.92:0/1951125948 conn(0x7ff46c0997f0 msgr2=0x7ff46c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:30.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.536+0000 7ff471630700 1 -- 172.21.15.92:0/1951125948 shutdown_connections 2023-10-06T10:38:30.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.536+0000 7ff471630700 1 -- 172.21.15.92:0/1951125948 wait complete. 2023-10-06T10:38:30.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.537+0000 7ff471630700 1 Processor -- start 2023-10-06T10:38:30.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.537+0000 7ff471630700 1 -- start start 2023-10-06T10:38:30.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff471630700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff46c0b0760 0x7ff46c09b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:30.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff471630700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff46c0b49a0 0x7ff46c16d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:30.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff471630700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff46c159560 0x7ff46c09d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:30.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff471630700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c16d4e0 con 0x7ff46c0b0760 2023-10-06T10:38:30.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff471630700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c16deb0 con 0x7ff46c159560 2023-10-06T10:38:30.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff471630700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c09bd30 con 0x7ff46c0b49a0 2023-10-06T10:38:30.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff46affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff46c0b0760 0x7ff46c09b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:30.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff46affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff46c0b0760 0x7ff46c09b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37872/0 (socket says 172.21.15.92:37872) 2023-10-06T10:38:30.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff46affd700 1 -- 172.21.15.92:0/2727242181 learned_addr learned my addr 172.21.15.92:0/2727242181 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:30.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff46b7fe700 1 --2- 172.21.15.92:0/2727242181 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff46c159560 0x7ff46c09d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:30.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff4627fc700 1 --2- 172.21.15.92:0/2727242181 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff46c0b49a0 0x7ff46c16d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:30.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff46affd700 1 -- 172.21.15.92:0/2727242181 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff46c0b49a0 msgr2=0x7ff46c16d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:30.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff46affd700 1 --2- 172.21.15.92:0/2727242181 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff46c0b49a0 0x7ff46c16d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:30.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff46affd700 1 -- 172.21.15.92:0/2727242181 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff46c159560 msgr2=0x7ff46c09d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:30.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff46affd700 1 --2- 172.21.15.92:0/2727242181 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff46c159560 0x7ff46c09d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:30.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff46affd700 1 -- 172.21.15.92:0/2727242181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff464009ae0 con 0x7ff46c0b0760 2023-10-06T10:38:30.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff46b7fe700 1 --2- 172.21.15.92:0/2727242181 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff46c159560 0x7ff46c09d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:30.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff46affd700 1 --2- 172.21.15.92:0/2727242181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff46c0b0760 0x7ff46c09b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff45c00d840 tx=0x7ff45c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:30.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff4637fe700 1 -- 172.21.15.92:0/2727242181 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c014620 con 0x7ff46c0b0760 2023-10-06T10:38:30.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff4637fe700 1 -- 172.21.15.92:0/2727242181 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45c014780 con 0x7ff46c0b0760 2023-10-06T10:38:30.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff471630700 1 -- 172.21.15.92:0/2727242181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff46c09c010 con 0x7ff46c0b0760 2023-10-06T10:38:30.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff4637fe700 1 -- 172.21.15.92:0/2727242181 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c014b40 con 0x7ff46c0b0760 2023-10-06T10:38:30.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.538+0000 7ff471630700 1 -- 172.21.15.92:0/2727242181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff46c16d170 con 0x7ff46c0b0760 2023-10-06T10:38:30.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.539+0000 7ff471630700 1 -- 172.21.15.92:0/2727242181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff46c16cfd0 con 0x7ff46c0b0760 2023-10-06T10:38:30.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.540+0000 7ff4637fe700 1 -- 172.21.15.92:0/2727242181 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff45c01d400 con 0x7ff46c0b0760 2023-10-06T10:38:30.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.540+0000 7ff4637fe700 1 --2- 172.21.15.92:0/2727242181 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff44c066f30 0x7ff44c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:30.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.540+0000 7ff4627fc700 1 --2- 172.21.15.92:0/2727242181 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff44c066f30 0x7ff44c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:30.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.540+0000 7ff4637fe700 1 -- 172.21.15.92:0/2727242181 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ff45c0852e0 con 0x7ff46c0b0760 2023-10-06T10:38:30.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.541+0000 7ff4627fc700 1 --2- 172.21.15.92:0/2727242181 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff44c066f30 0x7ff44c0693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff46c15c410 tx=0x7ff454008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:30.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.544+0000 7ff4637fe700 1 -- 172.21.15.92:0/2727242181 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff46c16cfd0 con 0x7ff46c0b0760 2023-10-06T10:38:30.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.767+0000 7ff471630700 1 -- 172.21.15.92:0/2727242181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff46c16cfd0 con 0x7ff46c0b0760 2023-10-06T10:38:30.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.768+0000 7ff4637fe700 1 -- 172.21.15.92:0/2727242181 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4599 (secure 0 0 0) 0x7ff46c16cfd0 con 0x7ff46c0b0760 2023-10-06T10:38:30.770 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:30.770 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2303424485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2303424485},{"type":"v1","addr":"172.21.15.92:6836","nonce":2303424485}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1027343065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1027343065},{"type":"v1","addr":"172.21.15.92:6837","nonce":1027343065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2667820379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2667820379},{"type":"v1","addr":"172.21.15.124:6834","nonce":2667820379}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:38:16.725144+0000","modified":"2023-10-06T10:38:30.398242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/1919704285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":1919704285},{"type":"v1","addr":"172.21.15.124:6835","nonce":1919704285}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T10:38:30.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.770+0000 7ff471630700 1 -- 172.21.15.92:0/2727242181 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff44c066f30 msgr2=0x7ff44c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:30.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.770+0000 7ff471630700 1 --2- 172.21.15.92:0/2727242181 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff44c066f30 0x7ff44c0693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff46c15c410 tx=0x7ff454008040 comp rx=0 tx=0).stop 2023-10-06T10:38:30.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.770+0000 7ff471630700 1 -- 172.21.15.92:0/2727242181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff46c0b0760 msgr2=0x7ff46c09b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:30.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.770+0000 7ff471630700 1 --2- 172.21.15.92:0/2727242181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff46c0b0760 0x7ff46c09b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff45c00d840 tx=0x7ff45c008040 comp rx=0 tx=0).stop 2023-10-06T10:38:30.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.770+0000 7ff471630700 1 -- 172.21.15.92:0/2727242181 shutdown_connections 2023-10-06T10:38:30.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.770+0000 7ff471630700 1 --2- 172.21.15.92:0/2727242181 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff44c066f30 0x7ff44c0693e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:30.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.770+0000 7ff471630700 1 --2- 172.21.15.92:0/2727242181 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff46c159560 0x7ff46c09d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:30.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.770+0000 7ff471630700 1 --2- 172.21.15.92:0/2727242181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff46c0b0760 0x7ff46c09b750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:30.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.770+0000 7ff471630700 1 --2- 172.21.15.92:0/2727242181 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff46c0b49a0 0x7ff46c16d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:30.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.770+0000 7ff471630700 1 -- 172.21.15.92:0/2727242181 >> 172.21.15.92:0/2727242181 conn(0x7ff46c0997f0 msgr2=0x7ff46c0b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:30.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.770+0000 7ff471630700 1 -- 172.21.15.92:0/2727242181 shutdown_connections 2023-10-06T10:38:30.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.770+0000 7ff471630700 1 -- 172.21.15.92:0/2727242181 wait complete. 2023-10-06T10:38:30.774 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-10-06T10:38:30.784 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:38:30.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.896+0000 7f6fd172d700 1 Processor -- start 2023-10-06T10:38:30.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.896+0000 7f6fd172d700 1 -- start start 2023-10-06T10:38:30.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.896+0000 7f6fd172d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6fcc0b49a0 0x7f6fcc0aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:30.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.896+0000 7f6fd172d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fcc0b0c60 con 0x7f6fcc0b0760 2023-10-06T10:38:30.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.896+0000 7f6fd172d700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fcc0b0f40 con 0x7f6fcc159560 2023-10-06T10:38:30.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.896+0000 7f6fd172d700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fcc0b1220 con 0x7f6fcc0b49a0 2023-10-06T10:38:30.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.897+0000 7f6fcb7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6fcc0b0760 0x7f6fcc0b1bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59946/0 (socket says 172.21.15.92:59946) 2023-10-06T10:38:30.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.897+0000 7f6fcb7fe700 1 -- 172.21.15.92:0/939192777 learned_addr learned my addr 172.21.15.92:0/939192777 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:30.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.897+0000 7f6fc27fc700 1 --2- 172.21.15.92:0/939192777 >> v2:172.21.15.124:3300/0 conn(0x7f6fcc0b49a0 0x7f6fcc0aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:30.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.897+0000 7f6fca7fc700 1 -- 172.21.15.92:0/939192777 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3536109830 0 0) 0x7f6fcc0b0c60 con 0x7f6fcc0b0760 2023-10-06T10:38:30.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.897+0000 7f6fca7fc700 1 -- 172.21.15.92:0/939192777 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fb8003570 con 0x7f6fcc0b0760 2023-10-06T10:38:30.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.897+0000 7f6fca7fc700 1 -- 172.21.15.92:0/939192777 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1337477539 0 0) 0x7f6fb8003570 con 0x7f6fcc0b0760 2023-10-06T10:38:30.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.897+0000 7f6fca7fc700 1 -- 172.21.15.92:0/939192777 >> v2:172.21.15.124:3300/0 conn(0x7f6fcc0b49a0 msgr2=0x7f6fcc0aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:30.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.897+0000 7f6fca7fc700 1 --2- 172.21.15.92:0/939192777 >> v2:172.21.15.124:3300/0 conn(0x7f6fcc0b49a0 0x7f6fcc0aac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:30.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.897+0000 7f6fca7fc700 1 -- 172.21.15.92:0/939192777 >> v1:172.21.15.124:6789/0 conn(0x7f6fcc159560 legacy=0x7f6fcc0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:30.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.897+0000 7f6fca7fc700 1 -- 172.21.15.92:0/939192777 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fcc0b1500 con 0x7f6fcc0b0760 2023-10-06T10:38:30.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.897+0000 7f6fca7fc700 1 -- 172.21.15.92:0/939192777 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f6fc40017f0 con 0x7f6fcc0b0760 2023-10-06T10:38:30.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.897+0000 7f6fca7fc700 1 -- 172.21.15.92:0/939192777 >> v1:172.21.15.92:6789/0 conn(0x7f6fcc0b0760 legacy=0x7f6fcc0b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:30.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.897+0000 7f6fca7fc700 1 --2- 172.21.15.92:0/939192777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb8003a90 0x7f6fb8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:30.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.897+0000 7f6fca7fc700 1 --2- 172.21.15.92:0/939192777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fb8004810 0x7f6fb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.897+0000 7f6fca7fc700 1 --2- 172.21.15.92:0/939192777 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fb8005210 0x7f6fb800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.897+0000 7f6fca7fc700 1 -- 172.21.15.92:0/939192777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc0b1220 con 0x7f6fb8003a90 2023-10-06T10:38:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.897+0000 7f6fca7fc700 1 -- 172.21.15.92:0/939192777 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb800c130 con 0x7f6fb8005210 2023-10-06T10:38:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.897+0000 7f6fca7fc700 1 -- 172.21.15.92:0/939192777 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb800c2c0 con 0x7f6fb8004810 2023-10-06T10:38:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.897+0000 7f6fcaffd700 1 --2- 172.21.15.92:0/939192777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb8003a90 0x7f6fb8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.897+0000 7f6fc27fc700 1 --2- 172.21.15.92:0/939192777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fb8004810 0x7f6fb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.897+0000 7f6fcb7fe700 1 --2- 172.21.15.92:0/939192777 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fb8005210 0x7f6fb800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.898+0000 7f6fcaffd700 1 -- 172.21.15.92:0/939192777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fb8004810 msgr2=0x7f6fb8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.898+0000 7f6fcaffd700 1 --2- 172.21.15.92:0/939192777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fb8004810 0x7f6fb8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.898+0000 7f6fcaffd700 1 -- 172.21.15.92:0/939192777 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fb8005210 msgr2=0x7f6fb800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.898+0000 7f6fcaffd700 1 --2- 172.21.15.92:0/939192777 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fb8005210 0x7f6fb800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.898+0000 7f6fcb7fe700 1 -- 172.21.15.92:0/939192777 reap_dead start 2023-10-06T10:38:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.898+0000 7f6fcaffd700 1 -- 172.21.15.92:0/939192777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb8003570 con 0x7f6fb8003a90 2023-10-06T10:38:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.898+0000 7f6fcaffd700 1 --2- 172.21.15.92:0/939192777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb8003a90 0x7f6fb8003ea0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc001900 tx=0x7f6fbc007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e98f3347cbc6ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.898+0000 7f6fca7fc700 1 -- 172.21.15.92:0/939192777 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc001290 con 0x7f6fb8003a90 2023-10-06T10:38:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.898+0000 7f6fca7fc700 1 -- 172.21.15.92:0/939192777 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fbc012420 con 0x7f6fb8003a90 2023-10-06T10:38:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.898+0000 7f6fca7fc700 1 -- 172.21.15.92:0/939192777 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc0127e0 con 0x7f6fb8003a90 2023-10-06T10:38:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.898+0000 7f6fd172d700 1 -- 172.21.15.92:0/939192777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb8003a90 msgr2=0x7f6fb8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.898+0000 7f6fd172d700 1 --2- 172.21.15.92:0/939192777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb8003a90 0x7f6fb8003ea0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc001900 tx=0x7f6fbc007d80 comp rx=0 tx=0).stop 2023-10-06T10:38:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.898+0000 7f6fd172d700 1 -- 172.21.15.92:0/939192777 shutdown_connections 2023-10-06T10:38:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.898+0000 7f6fd172d700 1 --2- 172.21.15.92:0/939192777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb8003a90 0x7f6fb8003ea0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.898+0000 7f6fd172d700 1 -- 172.21.15.92:0/939192777 >> 172.21.15.92:0/939192777 conn(0x7f6fcc0997f0 msgr2=0x7f6fcc06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.899+0000 7f6fd172d700 1 -- 172.21.15.92:0/939192777 shutdown_connections 2023-10-06T10:38:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.899+0000 7f6fd172d700 1 -- 172.21.15.92:0/939192777 wait complete. 2023-10-06T10:38:30.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.900+0000 7f6fd172d700 1 Processor -- start 2023-10-06T10:38:30.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.900+0000 7f6fd172d700 1 -- start start 2023-10-06T10:38:30.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.900+0000 7f6fd172d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb8003a90 0x7f6fcc15a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:30.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.900+0000 7f6fd172d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fcc158a20 0x7f6fcc15c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:30.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.900+0000 7f6fd172d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fcc09bb30 0x7f6fcc09bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:30.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.900+0000 7f6fd172d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc1699d0 con 0x7f6fb8003a90 2023-10-06T10:38:30.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.900+0000 7f6fd172d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc09c4e0 con 0x7f6fcc158a20 2023-10-06T10:38:30.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.900+0000 7f6fd172d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc15ce00 con 0x7f6fcc09bb30 2023-10-06T10:38:30.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.901+0000 7f6fcaffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb8003a90 0x7f6fcc15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:30.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.901+0000 7f6fcaffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb8003a90 0x7f6fcc15a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37890/0 (socket says 172.21.15.92:37890) 2023-10-06T10:38:30.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.901+0000 7f6fcaffd700 1 -- 172.21.15.92:0/2914398056 learned_addr learned my addr 172.21.15.92:0/2914398056 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:30.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.901+0000 7f6fc27fc700 1 --2- 172.21.15.92:0/2914398056 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fcc158a20 0x7f6fcc15c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:30.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.901+0000 7f6fcb7fe700 1 --2- 172.21.15.92:0/2914398056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fcc09bb30 0x7f6fcc09bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:30.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.901+0000 7f6fcaffd700 1 -- 172.21.15.92:0/2914398056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fcc09bb30 msgr2=0x7f6fcc09bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:30.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.901+0000 7f6fcaffd700 1 --2- 172.21.15.92:0/2914398056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fcc09bb30 0x7f6fcc09bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:30.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.901+0000 7f6fcaffd700 1 -- 172.21.15.92:0/2914398056 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fcc158a20 msgr2=0x7f6fcc15c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:30.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.901+0000 7f6fcaffd700 1 --2- 172.21.15.92:0/2914398056 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fcc158a20 0x7f6fcc15c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:30.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.901+0000 7f6fcaffd700 1 -- 172.21.15.92:0/2914398056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fcc15a980 con 0x7f6fb8003a90 2023-10-06T10:38:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.901+0000 7f6fcaffd700 1 --2- 172.21.15.92:0/2914398056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb8003a90 0x7f6fcc15a370 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6fb80057b0 tx=0x7f6fbc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.901+0000 7f6fc37fe700 1 -- 172.21.15.92:0/2914398056 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc01e610 con 0x7f6fb8003a90 2023-10-06T10:38:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.901+0000 7f6fc37fe700 1 -- 172.21.15.92:0/2914398056 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fbc01e770 con 0x7f6fb8003a90 2023-10-06T10:38:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.901+0000 7f6fd172d700 1 -- 172.21.15.92:0/2914398056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fcc15ac60 con 0x7f6fb8003a90 2023-10-06T10:38:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.901+0000 7f6fc37fe700 1 -- 172.21.15.92:0/2914398056 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc01eb30 con 0x7f6fb8003a90 2023-10-06T10:38:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.901+0000 7f6fd172d700 1 -- 172.21.15.92:0/2914398056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fcc15b220 con 0x7f6fb8003a90 2023-10-06T10:38:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.902+0000 7f6fd172d700 1 -- 172.21.15.92:0/2914398056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fcc15b440 con 0x7f6fb8003a90 2023-10-06T10:38:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.902+0000 7f6fc37fe700 1 -- 172.21.15.92:0/2914398056 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6fbc019030 con 0x7f6fb8003a90 2023-10-06T10:38:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.903+0000 7f6fc37fe700 1 --2- 172.21.15.92:0/2914398056 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6fac066f30 0x7f6fac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.903+0000 7f6fc37fe700 1 -- 172.21.15.92:0/2914398056 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f6fbc085bf0 con 0x7f6fb8003a90 2023-10-06T10:38:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.903+0000 7f6fc27fc700 1 --2- 172.21.15.92:0/2914398056 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6fac066f30 0x7f6fac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.903+0000 7f6fc27fc700 1 --2- 172.21.15.92:0/2914398056 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6fac066f30 0x7f6fac0693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc0aff90 tx=0x7f6fb40091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:30.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:30.907+0000 7f6fc37fe700 1 -- 172.21.15.92:0/2914398056 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6fbc052ad0 con 0x7f6fb8003a90 2023-10-06T10:38:31.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.090+0000 7f6fd172d700 1 -- 172.21.15.92:0/2914398056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fcc169c70 con 0x7f6fb8003a90 2023-10-06T10:38:31.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.091+0000 7f6fc37fe700 1 -- 172.21.15.92:0/2914398056 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18707 (secure 0 0 0) 0x7f6fbc017070 con 0x7f6fb8003a90 2023-10-06T10:38:31.093 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:31.093 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:21.760233+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:38:14.368713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T10:38:15.692344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T10:38:19.382553+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:38:31.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.093+0000 7f6fd172d700 1 -- 172.21.15.92:0/2914398056 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6fac066f30 msgr2=0x7f6fac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:31.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.093+0000 7f6fd172d700 1 --2- 172.21.15.92:0/2914398056 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6fac066f30 0x7f6fac0693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc0aff90 tx=0x7f6fb40091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.093+0000 7f6fd172d700 1 -- 172.21.15.92:0/2914398056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb8003a90 msgr2=0x7f6fcc15a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:31.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.093+0000 7f6fd172d700 1 --2- 172.21.15.92:0/2914398056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb8003a90 0x7f6fcc15a370 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6fb80057b0 tx=0x7f6fbc015000 comp rx=0 tx=0).stop 2023-10-06T10:38:31.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.093+0000 7f6fd172d700 1 -- 172.21.15.92:0/2914398056 shutdown_connections 2023-10-06T10:38:31.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.093+0000 7f6fd172d700 1 --2- 172.21.15.92:0/2914398056 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6fac066f30 0x7f6fac0693e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.093+0000 7f6fd172d700 1 --2- 172.21.15.92:0/2914398056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fcc09bb30 0x7f6fcc09bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.093+0000 7f6fd172d700 1 --2- 172.21.15.92:0/2914398056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb8003a90 0x7f6fcc15a370 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.093+0000 7f6fd172d700 1 --2- 172.21.15.92:0/2914398056 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fcc158a20 0x7f6fcc15c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.093+0000 7f6fd172d700 1 -- 172.21.15.92:0/2914398056 >> 172.21.15.92:0/2914398056 conn(0x7f6fcc0997f0 msgr2=0x7f6fcc0cd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:31.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.093+0000 7f6fd172d700 1 -- 172.21.15.92:0/2914398056 shutdown_connections 2023-10-06T10:38:31.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.093+0000 7f6fd172d700 1 -- 172.21.15.92:0/2914398056 wait complete. 2023-10-06T10:38:31.107 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T10:38:31.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:38:31.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.219+0000 7f087bbde700 1 Processor -- start 2023-10-06T10:38:31.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.219+0000 7f087bbde700 1 -- start start 2023-10-06T10:38:31.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.220+0000 7f087bbde700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08740b49a0 0x7f08740aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:31.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.220+0000 7f087bbde700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f08740b0760 0x7f08740b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:31.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.220+0000 7f087bbde700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08740b0c60 con 0x7f0874159560 2023-10-06T10:38:31.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.220+0000 7f087bbde700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08740b0f40 con 0x7f08740b0760 2023-10-06T10:38:31.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.220+0000 7f087bbde700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08740b1220 con 0x7f08740b49a0 2023-10-06T10:38:31.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.220+0000 7f087a17b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f08740b0760 0x7f08740b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:31.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.220+0000 7f087997a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0874159560 0x7f08740aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59950/0 (socket says 172.21.15.92:59950) 2023-10-06T10:38:31.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.220+0000 7f087997a700 1 -- 172.21.15.92:0/3510312960 learned_addr learned my addr 172.21.15.92:0/3510312960 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:31.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.220+0000 7f087a17b700 1 -- 172.21.15.92:0/3510312960 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08740b49a0 msgr2=0x7f08740aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:31.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.220+0000 7f087a17b700 1 --2- 172.21.15.92:0/3510312960 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08740b49a0 0x7f08740aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.220+0000 7f087a17b700 1 -- 172.21.15.92:0/3510312960 >> v1:172.21.15.92:6789/0 conn(0x7f0874159560 legacy=0x7f08740aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:31.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.220+0000 7f0878978700 1 -- 172.21.15.92:0/3510312960 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466903620 0 0) 0x7f08740b0c60 con 0x7f0874159560 2023-10-06T10:38:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.221+0000 7f087a17b700 1 -- 172.21.15.92:0/3510312960 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08740b1500 con 0x7f08740b0760 2023-10-06T10:38:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.221+0000 7f087a17b700 1 --2- 172.21.15.92:0/3510312960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08740b0760 0x7f08740b1bc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f086c009e60 tx=0x7f086c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef55342dc85866ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.221+0000 7f0878978700 1 -- 172.21.15.92:0/3510312960 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c00e070 con 0x7f08740b0760 2023-10-06T10:38:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.221+0000 7f0878978700 1 -- 172.21.15.92:0/3510312960 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086c00bc30 con 0x7f08740b0760 2023-10-06T10:38:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.221+0000 7f0878978700 1 -- 172.21.15.92:0/3510312960 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c012430 con 0x7f08740b0760 2023-10-06T10:38:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.221+0000 7f087bbde700 1 -- 172.21.15.92:0/3510312960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08740b0760 msgr2=0x7f08740b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.221+0000 7f087bbde700 1 --2- 172.21.15.92:0/3510312960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08740b0760 0x7f08740b1bc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f086c009e60 tx=0x7f086c009780 comp rx=0 tx=0).stop 2023-10-06T10:38:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.221+0000 7f087bbde700 1 -- 172.21.15.92:0/3510312960 shutdown_connections 2023-10-06T10:38:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.221+0000 7f087bbde700 1 --2- 172.21.15.92:0/3510312960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08740b0760 0x7f08740b1bc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.221+0000 7f087bbde700 1 --2- 172.21.15.92:0/3510312960 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08740b49a0 0x7f08740aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.221+0000 7f087bbde700 1 -- 172.21.15.92:0/3510312960 >> 172.21.15.92:0/3510312960 conn(0x7f08740997f0 msgr2=0x7f087406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.221+0000 7f087bbde700 1 -- 172.21.15.92:0/3510312960 shutdown_connections 2023-10-06T10:38:31.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.221+0000 7f087bbde700 1 -- 172.21.15.92:0/3510312960 wait complete. 2023-10-06T10:38:31.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.222+0000 7f087bbde700 1 Processor -- start 2023-10-06T10:38:31.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.223+0000 7f087bbde700 1 -- start start 2023-10-06T10:38:31.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.223+0000 7f087bbde700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08740b0760 0x7f087409dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:31.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.223+0000 7f087bbde700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08740b49a0 0x7f0874157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:31.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.223+0000 7f087bbde700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0874159560 0x7f087415c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:31.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.223+0000 7f087bbde700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0874099f50 con 0x7f0874159560 2023-10-06T10:38:31.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.223+0000 7f087bbde700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087416dbd0 con 0x7f08740b0760 2023-10-06T10:38:31.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.223+0000 7f087bbde700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087409a160 con 0x7f08740b49a0 2023-10-06T10:38:31.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.223+0000 7f087a17b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0874159560 0x7f087415c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:31.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.223+0000 7f087a17b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0874159560 0x7f087415c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37902/0 (socket says 172.21.15.92:37902) 2023-10-06T10:38:31.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.223+0000 7f087a17b700 1 -- 172.21.15.92:0/3987798817 learned_addr learned my addr 172.21.15.92:0/3987798817 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:31.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.224+0000 7f087997a700 1 --2- 172.21.15.92:0/3987798817 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08740b0760 0x7f087409dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:31.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.224+0000 7f087a17b700 1 -- 172.21.15.92:0/3987798817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08740b49a0 msgr2=0x7f0874157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:31.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.224+0000 7f087a17b700 1 --2- 172.21.15.92:0/3987798817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08740b49a0 0x7f0874157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.224+0000 7f087a17b700 1 -- 172.21.15.92:0/3987798817 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08740b0760 msgr2=0x7f087409dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:31.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.224+0000 7f087a17b700 1 --2- 172.21.15.92:0/3987798817 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08740b0760 0x7f087409dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.224+0000 7f087a17b700 1 -- 172.21.15.92:0/3987798817 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086c009a20 con 0x7f0874159560 2023-10-06T10:38:31.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.224+0000 7f087a17b700 1 --2- 172.21.15.92:0/3987798817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0874159560 0x7f087415c460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f086c013000 tx=0x7f086c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:31.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.224+0000 7f0869ffb700 1 -- 172.21.15.92:0/3987798817 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c01e630 con 0x7f0874159560 2023-10-06T10:38:31.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.224+0000 7f0869ffb700 1 -- 172.21.15.92:0/3987798817 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086c01e7b0 con 0x7f0874159560 2023-10-06T10:38:31.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.224+0000 7f0869ffb700 1 -- 172.21.15.92:0/3987798817 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c01eb70 con 0x7f0874159560 2023-10-06T10:38:31.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.224+0000 7f087bbde700 1 -- 172.21.15.92:0/3987798817 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f087409a440 con 0x7f0874159560 2023-10-06T10:38:31.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.224+0000 7f087bbde700 1 -- 172.21.15.92:0/3987798817 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08741581a0 con 0x7f0874159560 2023-10-06T10:38:31.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.225+0000 7f087bbde700 1 -- 172.21.15.92:0/3987798817 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f087409a960 con 0x7f0874159560 2023-10-06T10:38:31.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.225+0000 7f0869ffb700 1 -- 172.21.15.92:0/3987798817 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f086c024070 con 0x7f0874159560 2023-10-06T10:38:31.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.226+0000 7f0869ffb700 1 --2- 172.21.15.92:0/3987798817 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0858066f30 0x7f08580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:31.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.226+0000 7f0869ffb700 1 -- 172.21.15.92:0/3987798817 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f086c0872b0 con 0x7f0874159560 2023-10-06T10:38:31.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.226+0000 7f087997a700 1 --2- 172.21.15.92:0/3987798817 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0858066f30 0x7f08580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:31.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.226+0000 7f087997a700 1 --2- 172.21.15.92:0/3987798817 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0858066f30 0x7f08580693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f08700016e0 tx=0x7f0870009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:31.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.230+0000 7f0869ffb700 1 -- 172.21.15.92:0/3987798817 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f087409a960 con 0x7f0874159560 2023-10-06T10:38:31.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.411+0000 7f087bbde700 1 -- 172.21.15.92:0/3987798817 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f087409a960 con 0x7f0858066f30 2023-10-06T10:38:31.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.421+0000 7f0869ffb700 1 -- 172.21.15.92:0/3987798817 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367548 (secure 0 0 0) 0x7f087409a960 con 0x7f0858066f30 2023-10-06T10:38:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:31.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.425+0000 7f087bbde700 1 -- 172.21.15.92:0/3987798817 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0858066f30 msgr2=0x7f08580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:31.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.425+0000 7f087bbde700 1 --2- 172.21.15.92:0/3987798817 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0858066f30 0x7f08580693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f08700016e0 tx=0x7f0870009040 comp rx=0 tx=0).stop 2023-10-06T10:38:31.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.425+0000 7f087bbde700 1 -- 172.21.15.92:0/3987798817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0874159560 msgr2=0x7f087415c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:31.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.425+0000 7f087bbde700 1 --2- 172.21.15.92:0/3987798817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0874159560 0x7f087415c460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f086c013000 tx=0x7f086c019000 comp rx=0 tx=0).stop 2023-10-06T10:38:31.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.425+0000 7f087bbde700 1 -- 172.21.15.92:0/3987798817 shutdown_connections 2023-10-06T10:38:31.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.425+0000 7f087bbde700 1 --2- 172.21.15.92:0/3987798817 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0858066f30 0x7f08580693e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.425+0000 7f087bbde700 1 --2- 172.21.15.92:0/3987798817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0874159560 0x7f087415c460 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.425+0000 7f087bbde700 1 --2- 172.21.15.92:0/3987798817 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08740b0760 0x7f087409dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.425+0000 7f087bbde700 1 --2- 172.21.15.92:0/3987798817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08740b49a0 0x7f0874157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.425+0000 7f087bbde700 1 -- 172.21.15.92:0/3987798817 >> 172.21.15.92:0/3987798817 conn(0x7f08740997f0 msgr2=0x7f08740b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:31.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.425+0000 7f087bbde700 1 -- 172.21.15.92:0/3987798817 shutdown_connections 2023-10-06T10:38:31.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.425+0000 7f087bbde700 1 -- 172.21.15.92:0/3987798817 wait complete. 2023-10-06T10:38:31.433 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:38:31.439 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-10-06T10:38:30.083664+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594852,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381508,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593212,"statfs":{"total":773094113280,"available":770655449088,"internally_reserved":0,"allocated":3280896,"data_stored":2023855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2178,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.712825"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384275+0000","last_change":"2023-10-06T10:38:17.365963+0000","last_active":"2023-10-06T10:38:21.384275+0000","last_peered":"2023-10-06T10:38:21.384275+0000","last_clean":"2023-10-06T10:38:21.384275+0000","last_became_active":"2023-10-06T10:38:17.365702+0000","last_became_peered":"2023-10-06T10:38:17.365702+0000","last_unstale":"2023-10-06T10:38:21.384275+0000","last_undegraded":"2023-10-06T10:38:21.384275+0000","last_fullsized":"2023-10-06T10:38:21.384275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:58.222423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378758+0000","last_change":"2023-10-06T10:38:17.362155+0000","last_active":"2023-10-06T10:38:21.378758+0000","last_peered":"2023-10-06T10:38:21.378758+0000","last_clean":"2023-10-06T10:38:21.378758+0000","last_became_active":"2023-10-06T10:38:17.361871+0000","last_became_peered":"2023-10-06T10:38:17.361871+0000","last_unstale":"2023-10-06T10:38:21.378758+0000","last_undegraded":"2023-10-06T10:38:21.378758+0000","last_fullsized":"2023-10-06T10:38:21.378758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:49.973815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377999+0000","last_change":"2023-10-06T10:38:17.361502+0000","last_active":"2023-10-06T10:38:21.377999+0000","last_peered":"2023-10-06T10:38:21.377999+0000","last_clean":"2023-10-06T10:38:21.377999+0000","last_became_active":"2023-10-06T10:38:17.360866+0000","last_became_peered":"2023-10-06T10:38:17.360866+0000","last_unstale":"2023-10-06T10:38:21.377999+0000","last_undegraded":"2023-10-06T10:38:21.377999+0000","last_fullsized":"2023-10-06T10:38:21.377999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:01.022639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384859+0000","last_change":"2023-10-06T10:38:17.366612+0000","last_active":"2023-10-06T10:38:21.384859+0000","last_peered":"2023-10-06T10:38:21.384859+0000","last_clean":"2023-10-06T10:38:21.384859+0000","last_became_active":"2023-10-06T10:38:17.366337+0000","last_became_peered":"2023-10-06T10:38:17.366337+0000","last_unstale":"2023-10-06T10:38:21.384859+0000","last_undegraded":"2023-10-06T10:38:21.384859+0000","last_fullsized":"2023-10-06T10:38:21.384859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:01:12.162890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377172+0000","last_change":"2023-10-06T10:38:17.363460+0000","last_active":"2023-10-06T10:38:21.377172+0000","last_peered":"2023-10-06T10:38:21.377172+0000","last_clean":"2023-10-06T10:38:21.377172+0000","last_became_active":"2023-10-06T10:38:17.363211+0000","last_became_peered":"2023-10-06T10:38:17.363211+0000","last_unstale":"2023-10-06T10:38:21.377172+0000","last_undegraded":"2023-10-06T10:38:21.377172+0000","last_fullsized":"2023-10-06T10:38:21.377172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.132681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378316+0000","last_change":"2023-10-06T10:38:17.364098+0000","last_active":"2023-10-06T10:38:21.378316+0000","last_peered":"2023-10-06T10:38:21.378316+0000","last_clean":"2023-10-06T10:38:21.378316+0000","last_became_active":"2023-10-06T10:38:17.363806+0000","last_became_peered":"2023-10-06T10:38:17.363806+0000","last_unstale":"2023-10-06T10:38:21.378316+0000","last_undegraded":"2023-10-06T10:38:21.378316+0000","last_fullsized":"2023-10-06T10:38:21.378316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:27.532414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391179+0000","last_change":"2023-10-06T10:38:17.372436+0000","last_active":"2023-10-06T10:38:21.391179+0000","last_peered":"2023-10-06T10:38:21.391179+0000","last_clean":"2023-10-06T10:38:21.391179+0000","last_became_active":"2023-10-06T10:38:17.371126+0000","last_became_peered":"2023-10-06T10:38:17.371126+0000","last_unstale":"2023-10-06T10:38:21.391179+0000","last_undegraded":"2023-10-06T10:38:21.391179+0000","last_fullsized":"2023-10-06T10:38:21.391179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:13.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375652+0000","last_change":"2023-10-06T10:38:17.354022+0000","last_active":"2023-10-06T10:38:21.375652+0000","last_peered":"2023-10-06T10:38:21.375652+0000","last_clean":"2023-10-06T10:38:21.375652+0000","last_became_active":"2023-10-06T10:38:17.353630+0000","last_became_peered":"2023-10-06T10:38:17.353630+0000","last_unstale":"2023-10-06T10:38:21.375652+0000","last_undegraded":"2023-10-06T10:38:21.375652+0000","last_fullsized":"2023-10-06T10:38:21.375652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:03.953719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378266+0000","last_change":"2023-10-06T10:38:17.362560+0000","last_active":"2023-10-06T10:38:21.378266+0000","last_peered":"2023-10-06T10:38:21.378266+0000","last_clean":"2023-10-06T10:38:21.378266+0000","last_became_active":"2023-10-06T10:38:17.361953+0000","last_became_peered":"2023-10-06T10:38:17.361953+0000","last_unstale":"2023-10-06T10:38:21.378266+0000","last_undegraded":"2023-10-06T10:38:21.378266+0000","last_fullsized":"2023-10-06T10:38:21.378266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:38.417248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378932+0000","last_change":"2023-10-06T10:38:17.364442+0000","last_active":"2023-10-06T10:38:21.378932+0000","last_peered":"2023-10-06T10:38:21.378932+0000","last_clean":"2023-10-06T10:38:21.378932+0000","last_became_active":"2023-10-06T10:38:17.363787+0000","last_became_peered":"2023-10-06T10:38:17.363787+0000","last_unstale":"2023-10-06T10:38:21.378932+0000","last_undegraded":"2023-10-06T10:38:21.378932+0000","last_fullsized":"2023-10-06T10:38:21.378932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:13.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377971+0000","last_change":"2023-10-06T10:38:17.366791+0000","last_active":"2023-10-06T10:38:21.377971+0000","last_peered":"2023-10-06T10:38:21.377971+0000","last_clean":"2023-10-06T10:38:21.377971+0000","last_became_active":"2023-10-06T10:38:17.366281+0000","last_became_peered":"2023-10-06T10:38:17.366281+0000","last_unstale":"2023-10-06T10:38:21.377971+0000","last_undegraded":"2023-10-06T10:38:21.377971+0000","last_fullsized":"2023-10-06T10:38:21.377971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:22.349323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.383809+0000","last_change":"2023-10-06T10:38:17.368900+0000","last_active":"2023-10-06T10:38:21.383809+0000","last_peered":"2023-10-06T10:38:21.383809+0000","last_clean":"2023-10-06T10:38:21.383809+0000","last_became_active":"2023-10-06T10:38:17.368488+0000","last_became_peered":"2023-10-06T10:38:17.368488+0000","last_unstale":"2023-10-06T10:38:21.383809+0000","last_undegraded":"2023-10-06T10:38:21.383809+0000","last_fullsized":"2023-10-06T10:38:21.383809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:07.365430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.383322+0000","last_change":"2023-10-06T10:38:17.365338+0000","last_active":"2023-10-06T10:38:21.383322+0000","last_peered":"2023-10-06T10:38:21.383322+0000","last_clean":"2023-10-06T10:38:21.383322+0000","last_became_active":"2023-10-06T10:38:17.365056+0000","last_became_peered":"2023-10-06T10:38:17.365056+0000","last_unstale":"2023-10-06T10:38:21.383322+0000","last_undegraded":"2023-10-06T10:38:21.383322+0000","last_fullsized":"2023-10-06T10:38:21.383322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:03.122773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375751+0000","last_change":"2023-10-06T10:38:17.366413+0000","last_active":"2023-10-06T10:38:21.375751+0000","last_peered":"2023-10-06T10:38:21.375751+0000","last_clean":"2023-10-06T10:38:21.375751+0000","last_became_active":"2023-10-06T10:38:17.366139+0000","last_became_peered":"2023-10-06T10:38:17.366139+0000","last_unstale":"2023-10-06T10:38:21.375751+0000","last_undegraded":"2023-10-06T10:38:21.375751+0000","last_fullsized":"2023-10-06T10:38:21.375751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:34:27.066675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391014+0000","last_change":"2023-10-06T10:38:17.371250+0000","last_active":"2023-10-06T10:38:21.391014+0000","last_peered":"2023-10-06T10:38:21.391014+0000","last_clean":"2023-10-06T10:38:21.391014+0000","last_became_active":"2023-10-06T10:38:17.370858+0000","last_became_peered":"2023-10-06T10:38:17.370858+0000","last_unstale":"2023-10-06T10:38:21.391014+0000","last_undegraded":"2023-10-06T10:38:21.391014+0000","last_fullsized":"2023-10-06T10:38:21.391014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:28.001417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378433+0000","last_change":"2023-10-06T10:38:17.364371+0000","last_active":"2023-10-06T10:38:21.378433+0000","last_peered":"2023-10-06T10:38:21.378433+0000","last_clean":"2023-10-06T10:38:21.378433+0000","last_became_active":"2023-10-06T10:38:17.364040+0000","last_became_peered":"2023-10-06T10:38:17.364040+0000","last_unstale":"2023-10-06T10:38:21.378433+0000","last_undegraded":"2023-10-06T10:38:21.378433+0000","last_fullsized":"2023-10-06T10:38:21.378433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:29.897092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378355+0000","last_change":"2023-10-06T10:38:17.363058+0000","last_active":"2023-10-06T10:38:21.378355+0000","last_peered":"2023-10-06T10:38:21.378355+0000","last_clean":"2023-10-06T10:38:21.378355+0000","last_became_active":"2023-10-06T10:38:17.359050+0000","last_became_peered":"2023-10-06T10:38:17.359050+0000","last_unstale":"2023-10-06T10:38:21.378355+0000","last_undegraded":"2023-10-06T10:38:21.378355+0000","last_fullsized":"2023-10-06T10:38:21.378355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:28.001417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377744+0000","last_change":"2023-10-06T10:38:17.365982+0000","last_active":"2023-10-06T10:38:21.377744+0000","last_peered":"2023-10-06T10:38:21.377744+0000","last_clean":"2023-10-06T10:38:21.377744+0000","last_became_active":"2023-10-06T10:38:17.364876+0000","last_became_peered":"2023-10-06T10:38:17.364876+0000","last_unstale":"2023-10-06T10:38:21.377744+0000","last_undegraded":"2023-10-06T10:38:21.377744+0000","last_fullsized":"2023-10-06T10:38:21.377744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:13.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.383516+0000","last_change":"2023-10-06T10:38:17.368880+0000","last_active":"2023-10-06T10:38:21.383516+0000","last_peered":"2023-10-06T10:38:21.383516+0000","last_clean":"2023-10-06T10:38:21.383516+0000","last_became_active":"2023-10-06T10:38:17.368645+0000","last_became_peered":"2023-10-06T10:38:17.368645+0000","last_unstale":"2023-10-06T10:38:21.383516+0000","last_undegraded":"2023-10-06T10:38:21.383516+0000","last_fullsized":"2023-10-06T10:38:21.383516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:42.844587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377767+0000","last_change":"2023-10-06T10:38:17.375213+0000","last_active":"2023-10-06T10:38:21.377767+0000","last_peered":"2023-10-06T10:38:21.377767+0000","last_clean":"2023-10-06T10:38:21.377767+0000","last_became_active":"2023-10-06T10:38:17.374836+0000","last_became_peered":"2023-10-06T10:38:17.374836+0000","last_unstale":"2023-10-06T10:38:21.377767+0000","last_undegraded":"2023-10-06T10:38:21.377767+0000","last_fullsized":"2023-10-06T10:38:21.377767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:01.727252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.386555+0000","last_change":"2023-10-06T10:38:17.370625+0000","last_active":"2023-10-06T10:38:21.386555+0000","last_peered":"2023-10-06T10:38:21.386555+0000","last_clean":"2023-10-06T10:38:21.386555+0000","last_became_active":"2023-10-06T10:38:17.370355+0000","last_became_peered":"2023-10-06T10:38:17.370355+0000","last_unstale":"2023-10-06T10:38:21.386555+0000","last_undegraded":"2023-10-06T10:38:21.386555+0000","last_fullsized":"2023-10-06T10:38:21.386555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:38.417248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375531+0000","last_change":"2023-10-06T10:38:17.360622+0000","last_active":"2023-10-06T10:38:21.375531+0000","last_peered":"2023-10-06T10:38:21.375531+0000","last_clean":"2023-10-06T10:38:21.375531+0000","last_became_active":"2023-10-06T10:38:17.359689+0000","last_became_peered":"2023-10-06T10:38:17.359689+0000","last_unstale":"2023-10-06T10:38:21.375531+0000","last_undegraded":"2023-10-06T10:38:21.375531+0000","last_fullsized":"2023-10-06T10:38:21.375531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:03.650819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372352+0000","last_change":"2023-10-06T10:38:17.362700+0000","last_active":"2023-10-06T10:38:21.372352+0000","last_peered":"2023-10-06T10:38:21.372352+0000","last_clean":"2023-10-06T10:38:21.372352+0000","last_became_active":"2023-10-06T10:38:17.362457+0000","last_became_peered":"2023-10-06T10:38:17.362457+0000","last_unstale":"2023-10-06T10:38:21.372352+0000","last_undegraded":"2023-10-06T10:38:21.372352+0000","last_fullsized":"2023-10-06T10:38:21.372352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:50.378097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.393253+0000","last_change":"2023-10-06T10:38:17.371996+0000","last_active":"2023-10-06T10:38:21.393253+0000","last_peered":"2023-10-06T10:38:21.393253+0000","last_clean":"2023-10-06T10:38:21.393253+0000","last_became_active":"2023-10-06T10:38:17.371731+0000","last_became_peered":"2023-10-06T10:38:17.371731+0000","last_unstale":"2023-10-06T10:38:21.393253+0000","last_undegraded":"2023-10-06T10:38:21.393253+0000","last_fullsized":"2023-10-06T10:38:21.393253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:06.214524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378923+0000","last_change":"2023-10-06T10:38:17.373079+0000","last_active":"2023-10-06T10:38:21.378923+0000","last_peered":"2023-10-06T10:38:21.378923+0000","last_clean":"2023-10-06T10:38:21.378923+0000","last_became_active":"2023-10-06T10:38:17.372799+0000","last_became_peered":"2023-10-06T10:38:17.372799+0000","last_unstale":"2023-10-06T10:38:21.378923+0000","last_undegraded":"2023-10-06T10:38:21.378923+0000","last_fullsized":"2023-10-06T10:38:21.378923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:03.953719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.379048+0000","last_change":"2023-10-06T10:38:17.366829+0000","last_active":"2023-10-06T10:38:21.379048+0000","last_peered":"2023-10-06T10:38:21.379048+0000","last_clean":"2023-10-06T10:38:21.379048+0000","last_became_active":"2023-10-06T10:38:17.366617+0000","last_became_peered":"2023-10-06T10:38:17.366617+0000","last_unstale":"2023-10-06T10:38:21.379048+0000","last_undegraded":"2023-10-06T10:38:21.379048+0000","last_fullsized":"2023-10-06T10:38:21.379048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:35.729655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378140+0000","last_change":"2023-10-06T10:38:17.371330+0000","last_active":"2023-10-06T10:38:21.378140+0000","last_peered":"2023-10-06T10:38:21.378140+0000","last_clean":"2023-10-06T10:38:21.378140+0000","last_became_active":"2023-10-06T10:38:17.370864+0000","last_became_peered":"2023-10-06T10:38:17.370864+0000","last_unstale":"2023-10-06T10:38:21.378140+0000","last_undegraded":"2023-10-06T10:38:21.378140+0000","last_fullsized":"2023-10-06T10:38:21.378140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:38.417248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391173+0000","last_change":"2023-10-06T10:38:21.389792+0000","last_active":"2023-10-06T10:38:21.391173+0000","last_peered":"2023-10-06T10:38:21.391173+0000","last_clean":"2023-10-06T10:38:21.391173+0000","last_became_active":"2023-10-06T10:38:21.389400+0000","last_became_peered":"2023-10-06T10:38:21.389400+0000","last_unstale":"2023-10-06T10:38:21.391173+0000","last_undegraded":"2023-10-06T10:38:21.391173+0000","last_fullsized":"2023-10-06T10:38:21.391173+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:55.438723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378102+0000","last_change":"2023-10-06T10:38:16.366893+0000","last_active":"2023-10-06T10:38:21.378102+0000","last_peered":"2023-10-06T10:38:21.378102+0000","last_clean":"2023-10-06T10:38:21.378102+0000","last_became_active":"2023-10-06T10:38:16.366637+0000","last_became_peered":"2023-10-06T10:38:16.366637+0000","last_unstale":"2023-10-06T10:38:21.378102+0000","last_undegraded":"2023-10-06T10:38:21.378102+0000","last_fullsized":"2023-10-06T10:38:21.378102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:06.358487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372687+0000","last_change":"2023-10-06T10:38:17.360671+0000","last_active":"2023-10-06T10:38:21.372687+0000","last_peered":"2023-10-06T10:38:21.372687+0000","last_clean":"2023-10-06T10:38:21.372687+0000","last_became_active":"2023-10-06T10:38:17.360439+0000","last_became_peered":"2023-10-06T10:38:17.360439+0000","last_unstale":"2023-10-06T10:38:21.372687+0000","last_undegraded":"2023-10-06T10:38:21.372687+0000","last_fullsized":"2023-10-06T10:38:21.372687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:45:42.362885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402648+0000","last_change":"2023-10-06T10:38:21.399346+0000","last_active":"2023-10-06T10:38:21.402648+0000","last_peered":"2023-10-06T10:38:21.402648+0000","last_clean":"2023-10-06T10:38:21.402648+0000","last_became_active":"2023-10-06T10:38:21.398877+0000","last_became_peered":"2023-10-06T10:38:21.398877+0000","last_unstale":"2023-10-06T10:38:21.402648+0000","last_undegraded":"2023-10-06T10:38:21.402648+0000","last_fullsized":"2023-10-06T10:38:21.402648+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:05.052271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.376684+0000","last_change":"2023-10-06T10:38:16.357015+0000","last_active":"2023-10-06T10:38:21.376684+0000","last_peered":"2023-10-06T10:38:21.376684+0000","last_clean":"2023-10-06T10:38:21.376684+0000","last_became_active":"2023-10-06T10:38:16.356733+0000","last_became_peered":"2023-10-06T10:38:16.356733+0000","last_unstale":"2023-10-06T10:38:21.376684+0000","last_undegraded":"2023-10-06T10:38:21.376684+0000","last_fullsized":"2023-10-06T10:38:21.376684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:06.358487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375991+0000","last_change":"2023-10-06T10:38:17.370483+0000","last_active":"2023-10-06T10:38:21.375991+0000","last_peered":"2023-10-06T10:38:21.375991+0000","last_clean":"2023-10-06T10:38:21.375991+0000","last_became_active":"2023-10-06T10:38:17.370196+0000","last_became_peered":"2023-10-06T10:38:17.370196+0000","last_unstale":"2023-10-06T10:38:21.375991+0000","last_undegraded":"2023-10-06T10:38:21.375991+0000","last_fullsized":"2023-10-06T10:38:21.375991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:49.973815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.398933+0000","last_change":"2023-10-06T10:38:21.397171+0000","last_active":"2023-10-06T10:38:21.398933+0000","last_peered":"2023-10-06T10:38:21.398933+0000","last_clean":"2023-10-06T10:38:21.398933+0000","last_became_active":"2023-10-06T10:38:21.396794+0000","last_became_peered":"2023-10-06T10:38:21.396794+0000","last_unstale":"2023-10-06T10:38:21.398933+0000","last_undegraded":"2023-10-06T10:38:21.398933+0000","last_fullsized":"2023-10-06T10:38:21.398933+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:30.495506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372622+0000","last_change":"2023-10-06T10:38:16.362088+0000","last_active":"2023-10-06T10:38:21.372622+0000","last_peered":"2023-10-06T10:38:21.372622+0000","last_clean":"2023-10-06T10:38:21.372622+0000","last_became_active":"2023-10-06T10:38:16.361790+0000","last_became_peered":"2023-10-06T10:38:16.361790+0000","last_unstale":"2023-10-06T10:38:21.372622+0000","last_undegraded":"2023-10-06T10:38:21.372622+0000","last_fullsized":"2023-10-06T10:38:21.372622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:20.696308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385166+0000","last_change":"2023-10-06T10:38:17.372313+0000","last_active":"2023-10-06T10:38:21.385166+0000","last_peered":"2023-10-06T10:38:21.385166+0000","last_clean":"2023-10-06T10:38:21.385166+0000","last_became_active":"2023-10-06T10:38:17.369061+0000","last_became_peered":"2023-10-06T10:38:17.369061+0000","last_unstale":"2023-10-06T10:38:21.385166+0000","last_undegraded":"2023-10-06T10:38:21.385166+0000","last_fullsized":"2023-10-06T10:38:21.385166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:25.807302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402371+0000","last_change":"2023-10-06T10:38:21.399352+0000","last_active":"2023-10-06T10:38:21.402371+0000","last_peered":"2023-10-06T10:38:21.402371+0000","last_clean":"2023-10-06T10:38:21.402371+0000","last_became_active":"2023-10-06T10:38:21.398679+0000","last_became_peered":"2023-10-06T10:38:21.398679+0000","last_unstale":"2023-10-06T10:38:21.402371+0000","last_undegraded":"2023-10-06T10:38:21.402371+0000","last_fullsized":"2023-10-06T10:38:21.402371+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:14.207234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378436+0000","last_change":"2023-10-06T10:38:16.361297+0000","last_active":"2023-10-06T10:38:21.378436+0000","last_peered":"2023-10-06T10:38:21.378436+0000","last_clean":"2023-10-06T10:38:21.378436+0000","last_became_active":"2023-10-06T10:38:16.361034+0000","last_became_peered":"2023-10-06T10:38:16.361034+0000","last_unstale":"2023-10-06T10:38:21.378436+0000","last_undegraded":"2023-10-06T10:38:21.378436+0000","last_fullsized":"2023-10-06T10:38:21.378436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:02.115831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372551+0000","last_change":"2023-10-06T10:38:17.361626+0000","last_active":"2023-10-06T10:38:21.372551+0000","last_peered":"2023-10-06T10:38:21.372551+0000","last_clean":"2023-10-06T10:38:21.372551+0000","last_became_active":"2023-10-06T10:38:17.361386+0000","last_became_peered":"2023-10-06T10:38:17.361386+0000","last_unstale":"2023-10-06T10:38:21.372551+0000","last_undegraded":"2023-10-06T10:38:21.372551+0000","last_fullsized":"2023-10-06T10:38:21.372551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.132681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.397814+0000","last_change":"2023-10-06T10:38:21.395253+0000","last_active":"2023-10-06T10:38:21.397814+0000","last_peered":"2023-10-06T10:38:21.397814+0000","last_clean":"2023-10-06T10:38:21.397814+0000","last_became_active":"2023-10-06T10:38:21.394839+0000","last_became_peered":"2023-10-06T10:38:21.394839+0000","last_unstale":"2023-10-06T10:38:21.397814+0000","last_undegraded":"2023-10-06T10:38:21.397814+0000","last_fullsized":"2023-10-06T10:38:21.397814+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:43.578698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378846+0000","last_change":"2023-10-06T10:38:16.359731+0000","last_active":"2023-10-06T10:38:21.378846+0000","last_peered":"2023-10-06T10:38:21.378846+0000","last_clean":"2023-10-06T10:38:21.378846+0000","last_became_active":"2023-10-06T10:38:16.359451+0000","last_became_peered":"2023-10-06T10:38:16.359451+0000","last_unstale":"2023-10-06T10:38:21.378846+0000","last_undegraded":"2023-10-06T10:38:21.378846+0000","last_fullsized":"2023-10-06T10:38:21.378846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:06.358487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372246+0000","last_change":"2023-10-06T10:38:17.371057+0000","last_active":"2023-10-06T10:38:21.372246+0000","last_peered":"2023-10-06T10:38:21.372246+0000","last_clean":"2023-10-06T10:38:21.372246+0000","last_became_active":"2023-10-06T10:38:17.370898+0000","last_became_peered":"2023-10-06T10:38:17.370898+0000","last_unstale":"2023-10-06T10:38:21.372246+0000","last_undegraded":"2023-10-06T10:38:21.372246+0000","last_fullsized":"2023-10-06T10:38:21.372246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:59.822053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.393290+0000","last_change":"2023-10-06T10:38:21.387382+0000","last_active":"2023-10-06T10:38:21.393290+0000","last_peered":"2023-10-06T10:38:21.393290+0000","last_clean":"2023-10-06T10:38:21.393290+0000","last_became_active":"2023-10-06T10:38:21.386886+0000","last_became_peered":"2023-10-06T10:38:21.386886+0000","last_unstale":"2023-10-06T10:38:21.393290+0000","last_undegraded":"2023-10-06T10:38:21.393290+0000","last_fullsized":"2023-10-06T10:38:21.393290+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:04:14.062486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.392484+0000","last_change":"2023-10-06T10:38:16.362467+0000","last_active":"2023-10-06T10:38:21.392484+0000","last_peered":"2023-10-06T10:38:21.392484+0000","last_clean":"2023-10-06T10:38:21.392484+0000","last_became_active":"2023-10-06T10:38:16.362094+0000","last_became_peered":"2023-10-06T10:38:16.362094+0000","last_unstale":"2023-10-06T10:38:21.392484+0000","last_undegraded":"2023-10-06T10:38:21.392484+0000","last_fullsized":"2023-10-06T10:38:21.392484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:24.800360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378505+0000","last_change":"2023-10-06T10:38:17.371049+0000","last_active":"2023-10-06T10:38:21.378505+0000","last_peered":"2023-10-06T10:38:21.378505+0000","last_clean":"2023-10-06T10:38:21.378505+0000","last_became_active":"2023-10-06T10:38:17.370698+0000","last_became_peered":"2023-10-06T10:38:17.370698+0000","last_unstale":"2023-10-06T10:38:21.378505+0000","last_undegraded":"2023-10-06T10:38:21.378505+0000","last_fullsized":"2023-10-06T10:38:21.378505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:28.001417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.400543+0000","last_change":"2023-10-06T10:38:21.395461+0000","last_active":"2023-10-06T10:38:21.400543+0000","last_peered":"2023-10-06T10:38:21.400543+0000","last_clean":"2023-10-06T10:38:21.400543+0000","last_became_active":"2023-10-06T10:38:21.395077+0000","last_became_peered":"2023-10-06T10:38:21.395077+0000","last_unstale":"2023-10-06T10:38:21.400543+0000","last_undegraded":"2023-10-06T10:38:21.400543+0000","last_fullsized":"2023-10-06T10:38:21.400543+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:43.578698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'9","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T10:38:27.096420+0000","last_change":"2023-10-06T10:38:16.359809+0000","last_active":"2023-10-06T10:38:27.096420+0000","last_peered":"2023-10-06T10:38:27.096420+0000","last_clean":"2023-10-06T10:38:27.096420+0000","last_became_active":"2023-10-06T10:38:16.359520+0000","last_became_peered":"2023-10-06T10:38:16.359520+0000","last_unstale":"2023-10-06T10:38:27.096420+0000","last_undegraded":"2023-10-06T10:38:27.096420+0000","last_fullsized":"2023-10-06T10:38:27.096420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:41.837644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3714,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377665+0000","last_change":"2023-10-06T10:38:17.375788+0000","last_active":"2023-10-06T10:38:21.377665+0000","last_peered":"2023-10-06T10:38:21.377665+0000","last_clean":"2023-10-06T10:38:21.377665+0000","last_became_active":"2023-10-06T10:38:17.375416+0000","last_became_peered":"2023-10-06T10:38:17.375416+0000","last_unstale":"2023-10-06T10:38:21.377665+0000","last_undegraded":"2023-10-06T10:38:21.377665+0000","last_fullsized":"2023-10-06T10:38:21.377665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.132681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.396799+0000","last_change":"2023-10-06T10:38:21.391788+0000","last_active":"2023-10-06T10:38:21.396799+0000","last_peered":"2023-10-06T10:38:21.396799+0000","last_clean":"2023-10-06T10:38:21.396799+0000","last_became_active":"2023-10-06T10:38:21.391376+0000","last_became_peered":"2023-10-06T10:38:21.391376+0000","last_unstale":"2023-10-06T10:38:21.396799+0000","last_undegraded":"2023-10-06T10:38:21.396799+0000","last_fullsized":"2023-10-06T10:38:21.396799+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:25:40.423524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384475+0000","last_change":"2023-10-06T10:38:16.363389+0000","last_active":"2023-10-06T10:38:21.384475+0000","last_peered":"2023-10-06T10:38:21.384475+0000","last_clean":"2023-10-06T10:38:21.384475+0000","last_became_active":"2023-10-06T10:38:16.363141+0000","last_became_peered":"2023-10-06T10:38:16.363141+0000","last_unstale":"2023-10-06T10:38:21.384475+0000","last_undegraded":"2023-10-06T10:38:21.384475+0000","last_fullsized":"2023-10-06T10:38:21.384475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:01:11.155948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372587+0000","last_change":"2023-10-06T10:38:17.361247+0000","last_active":"2023-10-06T10:38:21.372587+0000","last_peered":"2023-10-06T10:38:21.372587+0000","last_clean":"2023-10-06T10:38:21.372587+0000","last_became_active":"2023-10-06T10:38:17.361016+0000","last_became_peered":"2023-10-06T10:38:17.361016+0000","last_unstale":"2023-10-06T10:38:21.372587+0000","last_undegraded":"2023-10-06T10:38:21.372587+0000","last_fullsized":"2023-10-06T10:38:21.372587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:38.417248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.398908+0000","last_change":"2023-10-06T10:38:21.395222+0000","last_active":"2023-10-06T10:38:21.398908+0000","last_peered":"2023-10-06T10:38:21.398908+0000","last_clean":"2023-10-06T10:38:21.398908+0000","last_became_active":"2023-10-06T10:38:21.394695+0000","last_became_peered":"2023-10-06T10:38:21.394695+0000","last_unstale":"2023-10-06T10:38:21.398908+0000","last_undegraded":"2023-10-06T10:38:21.398908+0000","last_fullsized":"2023-10-06T10:38:21.398908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:36:21.761225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375670+0000","last_change":"2023-10-06T10:38:16.359510+0000","last_active":"2023-10-06T10:38:21.375670+0000","last_peered":"2023-10-06T10:38:21.375670+0000","last_clean":"2023-10-06T10:38:21.375670+0000","last_became_active":"2023-10-06T10:38:16.359236+0000","last_became_peered":"2023-10-06T10:38:16.359236+0000","last_unstale":"2023-10-06T10:38:21.375670+0000","last_undegraded":"2023-10-06T10:38:21.375670+0000","last_fullsized":"2023-10-06T10:38:21.375670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:16.528477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378073+0000","last_change":"2023-10-06T10:38:17.371135+0000","last_active":"2023-10-06T10:38:21.378073+0000","last_peered":"2023-10-06T10:38:21.378073+0000","last_clean":"2023-10-06T10:38:21.378073+0000","last_became_active":"2023-10-06T10:38:17.370494+0000","last_became_peered":"2023-10-06T10:38:17.370494+0000","last_unstale":"2023-10-06T10:38:21.378073+0000","last_undegraded":"2023-10-06T10:38:21.378073+0000","last_fullsized":"2023-10-06T10:38:21.378073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:28.001417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.398908+0000","last_change":"2023-10-06T10:38:21.395871+0000","last_active":"2023-10-06T10:38:21.398908+0000","last_peered":"2023-10-06T10:38:21.398908+0000","last_clean":"2023-10-06T10:38:21.398908+0000","last_became_active":"2023-10-06T10:38:21.395558+0000","last_became_peered":"2023-10-06T10:38:21.395558+0000","last_unstale":"2023-10-06T10:38:21.398908+0000","last_undegraded":"2023-10-06T10:38:21.398908+0000","last_fullsized":"2023-10-06T10:38:21.398908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:32:37.103555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391095+0000","last_change":"2023-10-06T10:38:16.363354+0000","last_active":"2023-10-06T10:38:21.391095+0000","last_peered":"2023-10-06T10:38:21.391095+0000","last_clean":"2023-10-06T10:38:21.391095+0000","last_became_active":"2023-10-06T10:38:16.363088+0000","last_became_peered":"2023-10-06T10:38:16.363088+0000","last_unstale":"2023-10-06T10:38:21.391095+0000","last_undegraded":"2023-10-06T10:38:21.391095+0000","last_fullsized":"2023-10-06T10:38:21.391095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:39.672304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.376233+0000","last_change":"2023-10-06T10:38:17.359300+0000","last_active":"2023-10-06T10:38:21.376233+0000","last_peered":"2023-10-06T10:38:21.376233+0000","last_clean":"2023-10-06T10:38:21.376233+0000","last_became_active":"2023-10-06T10:38:17.358652+0000","last_became_peered":"2023-10-06T10:38:17.358652+0000","last_unstale":"2023-10-06T10:38:21.376233+0000","last_undegraded":"2023-10-06T10:38:21.376233+0000","last_fullsized":"2023-10-06T10:38:21.376233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:45:42.362885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.400005+0000","last_change":"2023-10-06T10:38:21.394129+0000","last_active":"2023-10-06T10:38:21.400005+0000","last_peered":"2023-10-06T10:38:21.400005+0000","last_clean":"2023-10-06T10:38:21.400005+0000","last_became_active":"2023-10-06T10:38:21.393517+0000","last_became_peered":"2023-10-06T10:38:21.393517+0000","last_unstale":"2023-10-06T10:38:21.400005+0000","last_undegraded":"2023-10-06T10:38:21.400005+0000","last_fullsized":"2023-10-06T10:38:21.400005+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:55:52.407995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385053+0000","last_change":"2023-10-06T10:38:16.364742+0000","last_active":"2023-10-06T10:38:21.385053+0000","last_peered":"2023-10-06T10:38:21.385053+0000","last_clean":"2023-10-06T10:38:21.385053+0000","last_became_active":"2023-10-06T10:38:16.364410+0000","last_became_peered":"2023-10-06T10:38:16.364410+0000","last_unstale":"2023-10-06T10:38:21.385053+0000","last_undegraded":"2023-10-06T10:38:21.385053+0000","last_fullsized":"2023-10-06T10:38:21.385053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:06.358487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.373267+0000","last_change":"2023-10-06T10:38:17.374643+0000","last_active":"2023-10-06T10:38:21.373267+0000","last_peered":"2023-10-06T10:38:21.373267+0000","last_clean":"2023-10-06T10:38:21.373267+0000","last_became_active":"2023-10-06T10:38:17.374436+0000","last_became_peered":"2023-10-06T10:38:17.374436+0000","last_unstale":"2023-10-06T10:38:21.373267+0000","last_undegraded":"2023-10-06T10:38:21.373267+0000","last_fullsized":"2023-10-06T10:38:21.373267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:01.022639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402862+0000","last_change":"2023-10-06T10:38:21.401121+0000","last_active":"2023-10-06T10:38:21.402862+0000","last_peered":"2023-10-06T10:38:21.402862+0000","last_clean":"2023-10-06T10:38:21.402862+0000","last_became_active":"2023-10-06T10:38:21.400766+0000","last_became_peered":"2023-10-06T10:38:21.400766+0000","last_unstale":"2023-10-06T10:38:21.402862+0000","last_undegraded":"2023-10-06T10:38:21.402862+0000","last_fullsized":"2023-10-06T10:38:21.402862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:39.967976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.392673+0000","last_change":"2023-10-06T10:38:16.363994+0000","last_active":"2023-10-06T10:38:21.392673+0000","last_peered":"2023-10-06T10:38:21.392673+0000","last_clean":"2023-10-06T10:38:21.392673+0000","last_became_active":"2023-10-06T10:38:16.363712+0000","last_became_peered":"2023-10-06T10:38:16.363712+0000","last_unstale":"2023-10-06T10:38:21.392673+0000","last_undegraded":"2023-10-06T10:38:21.392673+0000","last_fullsized":"2023-10-06T10:38:21.392673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:43.249942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375546+0000","last_change":"2023-10-06T10:38:17.374214+0000","last_active":"2023-10-06T10:38:21.375546+0000","last_peered":"2023-10-06T10:38:21.375546+0000","last_clean":"2023-10-06T10:38:21.375546+0000","last_became_active":"2023-10-06T10:38:17.373998+0000","last_became_peered":"2023-10-06T10:38:17.373998+0000","last_unstale":"2023-10-06T10:38:21.375546+0000","last_undegraded":"2023-10-06T10:38:21.375546+0000","last_fullsized":"2023-10-06T10:38:21.375546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:06.214524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.396278+0000","last_change":"2023-10-06T10:38:21.393210+0000","last_active":"2023-10-06T10:38:21.396278+0000","last_peered":"2023-10-06T10:38:21.396278+0000","last_clean":"2023-10-06T10:38:21.396278+0000","last_became_active":"2023-10-06T10:38:21.392827+0000","last_became_peered":"2023-10-06T10:38:21.392827+0000","last_unstale":"2023-10-06T10:38:21.396278+0000","last_undegraded":"2023-10-06T10:38:21.396278+0000","last_fullsized":"2023-10-06T10:38:21.396278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:05.756884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.392711+0000","last_change":"2023-10-06T10:38:16.363468+0000","last_active":"2023-10-06T10:38:21.392711+0000","last_peered":"2023-10-06T10:38:21.392711+0000","last_clean":"2023-10-06T10:38:21.392711+0000","last_became_active":"2023-10-06T10:38:16.363186+0000","last_became_peered":"2023-10-06T10:38:16.363186+0000","last_unstale":"2023-10-06T10:38:21.392711+0000","last_undegraded":"2023-10-06T10:38:21.392711+0000","last_fullsized":"2023-10-06T10:38:21.392711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:32.527614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377663+0000","last_change":"2023-10-06T10:38:17.365862+0000","last_active":"2023-10-06T10:38:21.377663+0000","last_peered":"2023-10-06T10:38:21.377663+0000","last_clean":"2023-10-06T10:38:21.377663+0000","last_became_active":"2023-10-06T10:38:17.365657+0000","last_became_peered":"2023-10-06T10:38:17.365657+0000","last_unstale":"2023-10-06T10:38:21.377663+0000","last_undegraded":"2023-10-06T10:38:21.377663+0000","last_fullsized":"2023-10-06T10:38:21.377663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:48.983240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.395385+0000","last_change":"2023-10-06T10:38:21.392166+0000","last_active":"2023-10-06T10:38:21.395385+0000","last_peered":"2023-10-06T10:38:21.395385+0000","last_clean":"2023-10-06T10:38:21.395385+0000","last_became_active":"2023-10-06T10:38:21.391756+0000","last_became_peered":"2023-10-06T10:38:21.391756+0000","last_unstale":"2023-10-06T10:38:21.395385+0000","last_undegraded":"2023-10-06T10:38:21.395385+0000","last_fullsized":"2023-10-06T10:38:21.395385+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:05.052271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.386206+0000","last_change":"2023-10-06T10:38:16.362369+0000","last_active":"2023-10-06T10:38:21.386206+0000","last_peered":"2023-10-06T10:38:21.386206+0000","last_clean":"2023-10-06T10:38:21.386206+0000","last_became_active":"2023-10-06T10:38:16.362076+0000","last_became_peered":"2023-10-06T10:38:16.362076+0000","last_unstale":"2023-10-06T10:38:21.386206+0000","last_undegraded":"2023-10-06T10:38:21.386206+0000","last_fullsized":"2023-10-06T10:38:21.386206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:27.473452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384735+0000","last_change":"2023-10-06T10:38:17.368641+0000","last_active":"2023-10-06T10:38:21.384735+0000","last_peered":"2023-10-06T10:38:21.384735+0000","last_clean":"2023-10-06T10:38:21.384735+0000","last_became_active":"2023-10-06T10:38:17.368283+0000","last_became_peered":"2023-10-06T10:38:17.368283+0000","last_unstale":"2023-10-06T10:38:21.384735+0000","last_undegraded":"2023-10-06T10:38:21.384735+0000","last_fullsized":"2023-10-06T10:38:21.384735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:17.351581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.395567+0000","last_change":"2023-10-06T10:38:21.392954+0000","last_active":"2023-10-06T10:38:21.395567+0000","last_peered":"2023-10-06T10:38:21.395567+0000","last_clean":"2023-10-06T10:38:21.395567+0000","last_became_active":"2023-10-06T10:38:21.392585+0000","last_became_peered":"2023-10-06T10:38:21.392585+0000","last_unstale":"2023-10-06T10:38:21.395567+0000","last_undegraded":"2023-10-06T10:38:21.395567+0000","last_fullsized":"2023-10-06T10:38:21.395567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:54.407728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375768+0000","last_change":"2023-10-06T10:38:16.357199+0000","last_active":"2023-10-06T10:38:21.375768+0000","last_peered":"2023-10-06T10:38:21.375768+0000","last_clean":"2023-10-06T10:38:21.375768+0000","last_became_active":"2023-10-06T10:38:16.356936+0000","last_became_peered":"2023-10-06T10:38:16.356936+0000","last_unstale":"2023-10-06T10:38:21.375768+0000","last_undegraded":"2023-10-06T10:38:21.375768+0000","last_fullsized":"2023-10-06T10:38:21.375768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:02.115831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377449+0000","last_change":"2023-10-06T10:38:17.359930+0000","last_active":"2023-10-06T10:38:21.377449+0000","last_peered":"2023-10-06T10:38:21.377449+0000","last_clean":"2023-10-06T10:38:21.377449+0000","last_became_active":"2023-10-06T10:38:17.359677+0000","last_became_peered":"2023-10-06T10:38:17.359677+0000","last_unstale":"2023-10-06T10:38:21.377449+0000","last_undegraded":"2023-10-06T10:38:21.377449+0000","last_fullsized":"2023-10-06T10:38:21.377449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:13.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.399357+0000","last_change":"2023-10-06T10:38:21.396020+0000","last_active":"2023-10-06T10:38:21.399357+0000","last_peered":"2023-10-06T10:38:21.399357+0000","last_clean":"2023-10-06T10:38:21.399357+0000","last_became_active":"2023-10-06T10:38:21.395659+0000","last_became_peered":"2023-10-06T10:38:21.395659+0000","last_unstale":"2023-10-06T10:38:21.399357+0000","last_undegraded":"2023-10-06T10:38:21.399357+0000","last_fullsized":"2023-10-06T10:38:21.399357+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:45:46.392517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372543+0000","last_change":"2023-10-06T10:38:16.364249+0000","last_active":"2023-10-06T10:38:21.372543+0000","last_peered":"2023-10-06T10:38:21.372543+0000","last_clean":"2023-10-06T10:38:21.372543+0000","last_became_active":"2023-10-06T10:38:16.363959+0000","last_became_peered":"2023-10-06T10:38:16.363959+0000","last_unstale":"2023-10-06T10:38:21.372543+0000","last_undegraded":"2023-10-06T10:38:21.372543+0000","last_fullsized":"2023-10-06T10:38:21.372543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:27.473452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377610+0000","last_change":"2023-10-06T10:38:17.362439+0000","last_active":"2023-10-06T10:38:21.377610+0000","last_peered":"2023-10-06T10:38:21.377610+0000","last_clean":"2023-10-06T10:38:21.377610+0000","last_became_active":"2023-10-06T10:38:17.361921+0000","last_became_peered":"2023-10-06T10:38:17.361921+0000","last_unstale":"2023-10-06T10:38:21.377610+0000","last_undegraded":"2023-10-06T10:38:21.377610+0000","last_fullsized":"2023-10-06T10:38:21.377610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:30:01.208220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402322+0000","last_change":"2023-10-06T10:38:21.399211+0000","last_active":"2023-10-06T10:38:21.402322+0000","last_peered":"2023-10-06T10:38:21.402322+0000","last_clean":"2023-10-06T10:38:21.402322+0000","last_became_active":"2023-10-06T10:38:21.396487+0000","last_became_peered":"2023-10-06T10:38:21.396487+0000","last_unstale":"2023-10-06T10:38:21.402322+0000","last_undegraded":"2023-10-06T10:38:21.402322+0000","last_fullsized":"2023-10-06T10:38:21.402322+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:26.378955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384822+0000","last_change":"2023-10-06T10:38:16.362941+0000","last_active":"2023-10-06T10:38:21.384822+0000","last_peered":"2023-10-06T10:38:21.384822+0000","last_clean":"2023-10-06T10:38:21.384822+0000","last_became_active":"2023-10-06T10:38:16.362651+0000","last_became_peered":"2023-10-06T10:38:16.362651+0000","last_unstale":"2023-10-06T10:38:21.384822+0000","last_undegraded":"2023-10-06T10:38:21.384822+0000","last_fullsized":"2023-10-06T10:38:21.384822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:41.837644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377389+0000","last_change":"2023-10-06T10:38:17.367573+0000","last_active":"2023-10-06T10:38:21.377389+0000","last_peered":"2023-10-06T10:38:21.377389+0000","last_clean":"2023-10-06T10:38:21.377389+0000","last_became_active":"2023-10-06T10:38:17.367363+0000","last_became_peered":"2023-10-06T10:38:17.367363+0000","last_unstale":"2023-10-06T10:38:21.377389+0000","last_undegraded":"2023-10-06T10:38:21.377389+0000","last_fullsized":"2023-10-06T10:38:21.377389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:03.650819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.401267+0000","last_change":"2023-10-06T10:38:21.394943+0000","last_active":"2023-10-06T10:38:21.401267+0000","last_peered":"2023-10-06T10:38:21.401267+0000","last_clean":"2023-10-06T10:38:21.401267+0000","last_became_active":"2023-10-06T10:38:21.394604+0000","last_became_peered":"2023-10-06T10:38:21.394604+0000","last_unstale":"2023-10-06T10:38:21.401267+0000","last_undegraded":"2023-10-06T10:38:21.401267+0000","last_fullsized":"2023-10-06T10:38:21.401267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:17:59.362798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377491+0000","last_change":"2023-10-06T10:38:16.358983+0000","last_active":"2023-10-06T10:38:21.377491+0000","last_peered":"2023-10-06T10:38:21.377491+0000","last_clean":"2023-10-06T10:38:21.377491+0000","last_became_active":"2023-10-06T10:38:16.358338+0000","last_became_peered":"2023-10-06T10:38:16.358338+0000","last_unstale":"2023-10-06T10:38:21.377491+0000","last_undegraded":"2023-10-06T10:38:21.377491+0000","last_fullsized":"2023-10-06T10:38:21.377491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:02.115831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378341+0000","last_change":"2023-10-06T10:38:17.375549+0000","last_active":"2023-10-06T10:38:21.378341+0000","last_peered":"2023-10-06T10:38:21.378341+0000","last_clean":"2023-10-06T10:38:21.378341+0000","last_became_active":"2023-10-06T10:38:17.374707+0000","last_became_peered":"2023-10-06T10:38:17.374707+0000","last_unstale":"2023-10-06T10:38:21.378341+0000","last_undegraded":"2023-10-06T10:38:21.378341+0000","last_fullsized":"2023-10-06T10:38:21.378341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:50.378097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391124+0000","last_change":"2023-10-06T10:38:21.388787+0000","last_active":"2023-10-06T10:38:21.391124+0000","last_peered":"2023-10-06T10:38:21.391124+0000","last_clean":"2023-10-06T10:38:21.391124+0000","last_became_active":"2023-10-06T10:38:21.388432+0000","last_became_peered":"2023-10-06T10:38:21.388432+0000","last_unstale":"2023-10-06T10:38:21.391124+0000","last_undegraded":"2023-10-06T10:38:21.391124+0000","last_fullsized":"2023-10-06T10:38:21.391124+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:03.131588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372464+0000","last_change":"2023-10-06T10:38:16.366048+0000","last_active":"2023-10-06T10:38:21.372464+0000","last_peered":"2023-10-06T10:38:21.372464+0000","last_clean":"2023-10-06T10:38:21.372464+0000","last_became_active":"2023-10-06T10:38:16.365814+0000","last_became_peered":"2023-10-06T10:38:16.365814+0000","last_unstale":"2023-10-06T10:38:21.372464+0000","last_undegraded":"2023-10-06T10:38:21.372464+0000","last_fullsized":"2023-10-06T10:38:21.372464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:36.469014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.379002+0000","last_change":"2023-10-06T10:38:17.358727+0000","last_active":"2023-10-06T10:38:21.379002+0000","last_peered":"2023-10-06T10:38:21.379002+0000","last_clean":"2023-10-06T10:38:21.379002+0000","last_became_active":"2023-10-06T10:38:17.358442+0000","last_became_peered":"2023-10-06T10:38:17.358442+0000","last_unstale":"2023-10-06T10:38:21.379002+0000","last_undegraded":"2023-10-06T10:38:21.379002+0000","last_fullsized":"2023-10-06T10:38:21.379002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:17.351581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.401082+0000","last_change":"2023-10-06T10:38:21.397353+0000","last_active":"2023-10-06T10:38:21.401082+0000","last_peered":"2023-10-06T10:38:21.401082+0000","last_clean":"2023-10-06T10:38:21.401082+0000","last_became_active":"2023-10-06T10:38:21.396788+0000","last_became_peered":"2023-10-06T10:38:21.396788+0000","last_unstale":"2023-10-06T10:38:21.401082+0000","last_undegraded":"2023-10-06T10:38:21.401082+0000","last_fullsized":"2023-10-06T10:38:21.401082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:50:06.948087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.373072+0000","last_change":"2023-10-06T10:38:16.362604+0000","last_active":"2023-10-06T10:38:21.373072+0000","last_peered":"2023-10-06T10:38:21.373072+0000","last_clean":"2023-10-06T10:38:21.373072+0000","last_became_active":"2023-10-06T10:38:16.362444+0000","last_became_peered":"2023-10-06T10:38:16.362444+0000","last_unstale":"2023-10-06T10:38:21.373072+0000","last_undegraded":"2023-10-06T10:38:21.373072+0000","last_fullsized":"2023-10-06T10:38:21.373072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:57.215481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377559+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:38:21.377559+0000","last_peered":"2023-10-06T10:38:21.377559+0000","last_clean":"2023-10-06T10:38:21.377559+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:38:21.377559+0000","last_undegraded":"2023-10-06T10:38:21.377559+0000","last_fullsized":"2023-10-06T10:38:21.377559+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372292+0000","last_change":"2023-10-06T10:38:17.374346+0000","last_active":"2023-10-06T10:38:21.372292+0000","last_peered":"2023-10-06T10:38:21.372292+0000","last_clean":"2023-10-06T10:38:21.372292+0000","last_became_active":"2023-10-06T10:38:17.374068+0000","last_became_peered":"2023-10-06T10:38:17.374068+0000","last_unstale":"2023-10-06T10:38:21.372292+0000","last_undegraded":"2023-10-06T10:38:21.372292+0000","last_fullsized":"2023-10-06T10:38:21.372292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:29.897092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.393470+0000","last_change":"2023-10-06T10:38:21.387372+0000","last_active":"2023-10-06T10:38:21.393470+0000","last_peered":"2023-10-06T10:38:21.393470+0000","last_clean":"2023-10-06T10:38:21.393470+0000","last_became_active":"2023-10-06T10:38:21.386907+0000","last_became_peered":"2023-10-06T10:38:21.386907+0000","last_unstale":"2023-10-06T10:38:21.393470+0000","last_undegraded":"2023-10-06T10:38:21.393470+0000","last_fullsized":"2023-10-06T10:38:21.393470+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:36.144881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372792+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:38:21.372792+0000","last_peered":"2023-10-06T10:38:21.372792+0000","last_clean":"2023-10-06T10:38:21.372792+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:38:21.372792+0000","last_undegraded":"2023-10-06T10:38:21.372792+0000","last_fullsized":"2023-10-06T10:38:21.372792+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.433239+0000","last_change":"2023-10-06T10:38:16.366116+0000","last_active":"2023-10-06T10:38:21.433239+0000","last_peered":"2023-10-06T10:38:21.433239+0000","last_clean":"2023-10-06T10:38:21.433239+0000","last_became_active":"2023-10-06T10:38:16.365875+0000","last_became_peered":"2023-10-06T10:38:16.365875+0000","last_unstale":"2023-10-06T10:38:21.433239+0000","last_undegraded":"2023-10-06T10:38:21.433239+0000","last_fullsized":"2023-10-06T10:38:21.433239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:35.127396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377577+0000","last_change":"2023-10-06T10:38:17.358654+0000","last_active":"2023-10-06T10:38:21.377577+0000","last_peered":"2023-10-06T10:38:21.377577+0000","last_clean":"2023-10-06T10:38:21.377577+0000","last_became_active":"2023-10-06T10:38:17.358361+0000","last_became_peered":"2023-10-06T10:38:17.358361+0000","last_unstale":"2023-10-06T10:38:21.377577+0000","last_undegraded":"2023-10-06T10:38:21.377577+0000","last_fullsized":"2023-10-06T10:38:21.377577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:06.214524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.397881+0000","last_change":"2023-10-06T10:38:21.392569+0000","last_active":"2023-10-06T10:38:21.397881+0000","last_peered":"2023-10-06T10:38:21.397881+0000","last_clean":"2023-10-06T10:38:21.397881+0000","last_became_active":"2023-10-06T10:38:21.392210+0000","last_became_peered":"2023-10-06T10:38:21.392210+0000","last_unstale":"2023-10-06T10:38:21.397881+0000","last_undegraded":"2023-10-06T10:38:21.397881+0000","last_fullsized":"2023-10-06T10:38:21.397881+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:14:10.094621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377969+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:38:21.377969+0000","last_peered":"2023-10-06T10:38:21.377969+0000","last_clean":"2023-10-06T10:38:21.377969+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:38:21.377969+0000","last_undegraded":"2023-10-06T10:38:21.377969+0000","last_fullsized":"2023-10-06T10:38:21.377969+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.390711+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:38:21.390711+0000","last_peered":"2023-10-06T10:38:21.390711+0000","last_clean":"2023-10-06T10:38:21.390711+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:38:21.390711+0000","last_undegraded":"2023-10-06T10:38:21.390711+0000","last_fullsized":"2023-10-06T10:38:21.390711+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375378+0000","last_change":"2023-10-06T10:38:16.362718+0000","last_active":"2023-10-06T10:38:21.375378+0000","last_peered":"2023-10-06T10:38:21.375378+0000","last_clean":"2023-10-06T10:38:21.375378+0000","last_became_active":"2023-10-06T10:38:16.361895+0000","last_became_peered":"2023-10-06T10:38:16.361895+0000","last_unstale":"2023-10-06T10:38:21.375378+0000","last_undegraded":"2023-10-06T10:38:21.375378+0000","last_fullsized":"2023-10-06T10:38:21.375378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:45.092301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.376156+0000","last_change":"2023-10-06T10:38:17.359496+0000","last_active":"2023-10-06T10:38:21.376156+0000","last_peered":"2023-10-06T10:38:21.376156+0000","last_clean":"2023-10-06T10:38:21.376156+0000","last_became_active":"2023-10-06T10:38:17.359011+0000","last_became_peered":"2023-10-06T10:38:17.359011+0000","last_unstale":"2023-10-06T10:38:21.376156+0000","last_undegraded":"2023-10-06T10:38:21.376156+0000","last_fullsized":"2023-10-06T10:38:21.376156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:06.214524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402221+0000","last_change":"2023-10-06T10:38:21.396996+0000","last_active":"2023-10-06T10:38:21.402221+0000","last_peered":"2023-10-06T10:38:21.402221+0000","last_clean":"2023-10-06T10:38:21.402221+0000","last_became_active":"2023-10-06T10:38:21.396694+0000","last_became_peered":"2023-10-06T10:38:21.396694+0000","last_unstale":"2023-10-06T10:38:21.402221+0000","last_undegraded":"2023-10-06T10:38:21.402221+0000","last_fullsized":"2023-10-06T10:38:21.402221+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:14.207234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.386749+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:38:21.386749+0000","last_peered":"2023-10-06T10:38:21.386749+0000","last_clean":"2023-10-06T10:38:21.386749+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:38:21.386749+0000","last_undegraded":"2023-10-06T10:38:21.386749+0000","last_fullsized":"2023-10-06T10:38:21.386749+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.376477+0000","last_change":"2023-10-06T10:38:16.357154+0000","last_active":"2023-10-06T10:38:21.376477+0000","last_peered":"2023-10-06T10:38:21.376477+0000","last_clean":"2023-10-06T10:38:21.376477+0000","last_became_active":"2023-10-06T10:38:16.356942+0000","last_became_peered":"2023-10-06T10:38:16.356942+0000","last_unstale":"2023-10-06T10:38:21.376477+0000","last_undegraded":"2023-10-06T10:38:21.376477+0000","last_fullsized":"2023-10-06T10:38:21.376477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:16.528477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378971+0000","last_change":"2023-10-06T10:38:17.358788+0000","last_active":"2023-10-06T10:38:21.378971+0000","last_peered":"2023-10-06T10:38:21.378971+0000","last_clean":"2023-10-06T10:38:21.378971+0000","last_became_active":"2023-10-06T10:38:17.358514+0000","last_became_peered":"2023-10-06T10:38:17.358514+0000","last_unstale":"2023-10-06T10:38:21.378971+0000","last_undegraded":"2023-10-06T10:38:21.378971+0000","last_fullsized":"2023-10-06T10:38:21.378971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:49.973815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402838+0000","last_change":"2023-10-06T10:38:21.401259+0000","last_active":"2023-10-06T10:38:21.402838+0000","last_peered":"2023-10-06T10:38:21.402838+0000","last_clean":"2023-10-06T10:38:21.402838+0000","last_became_active":"2023-10-06T10:38:21.400865+0000","last_became_peered":"2023-10-06T10:38:21.400865+0000","last_unstale":"2023-10-06T10:38:21.402838+0000","last_undegraded":"2023-10-06T10:38:21.402838+0000","last_fullsized":"2023-10-06T10:38:21.402838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:30.495506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384378+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:38:21.384378+0000","last_peered":"2023-10-06T10:38:21.384378+0000","last_clean":"2023-10-06T10:38:21.384378+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:38:21.384378+0000","last_undegraded":"2023-10-06T10:38:21.384378+0000","last_fullsized":"2023-10-06T10:38:21.384378+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372330+0000","last_change":"2023-10-06T10:38:16.358939+0000","last_active":"2023-10-06T10:38:21.372330+0000","last_peered":"2023-10-06T10:38:21.372330+0000","last_clean":"2023-10-06T10:38:21.372330+0000","last_became_active":"2023-10-06T10:38:16.358685+0000","last_became_peered":"2023-10-06T10:38:16.358685+0000","last_unstale":"2023-10-06T10:38:21.372330+0000","last_undegraded":"2023-10-06T10:38:21.372330+0000","last_fullsized":"2023-10-06T10:38:21.372330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:41.837644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385109+0000","last_change":"2023-10-06T10:38:17.366729+0000","last_active":"2023-10-06T10:38:21.385109+0000","last_peered":"2023-10-06T10:38:21.385109+0000","last_clean":"2023-10-06T10:38:21.385109+0000","last_became_active":"2023-10-06T10:38:17.366408+0000","last_became_peered":"2023-10-06T10:38:17.366408+0000","last_unstale":"2023-10-06T10:38:21.385109+0000","last_undegraded":"2023-10-06T10:38:21.385109+0000","last_fullsized":"2023-10-06T10:38:21.385109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:35.729655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.395664+0000","last_change":"2023-10-06T10:38:21.392874+0000","last_active":"2023-10-06T10:38:21.395664+0000","last_peered":"2023-10-06T10:38:21.395664+0000","last_clean":"2023-10-06T10:38:21.395664+0000","last_became_active":"2023-10-06T10:38:21.392487+0000","last_became_peered":"2023-10-06T10:38:21.392487+0000","last_unstale":"2023-10-06T10:38:21.395664+0000","last_undegraded":"2023-10-06T10:38:21.395664+0000","last_fullsized":"2023-10-06T10:38:21.395664+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:39.967976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375754+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:38:21.375754+0000","last_peered":"2023-10-06T10:38:21.375754+0000","last_clean":"2023-10-06T10:38:21.375754+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:38:21.375754+0000","last_undegraded":"2023-10-06T10:38:21.375754+0000","last_fullsized":"2023-10-06T10:38:21.375754+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377566+0000","last_change":"2023-10-06T10:38:16.362244+0000","last_active":"2023-10-06T10:38:21.377566+0000","last_peered":"2023-10-06T10:38:21.377566+0000","last_clean":"2023-10-06T10:38:21.377566+0000","last_became_active":"2023-10-06T10:38:16.361869+0000","last_became_peered":"2023-10-06T10:38:16.361869+0000","last_unstale":"2023-10-06T10:38:21.377566+0000","last_undegraded":"2023-10-06T10:38:21.377566+0000","last_fullsized":"2023-10-06T10:38:21.377566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:57.215481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.379113+0000","last_change":"2023-10-06T10:38:17.367649+0000","last_active":"2023-10-06T10:38:21.379113+0000","last_peered":"2023-10-06T10:38:21.379113+0000","last_clean":"2023-10-06T10:38:21.379113+0000","last_became_active":"2023-10-06T10:38:17.366773+0000","last_became_peered":"2023-10-06T10:38:17.366773+0000","last_unstale":"2023-10-06T10:38:21.379113+0000","last_undegraded":"2023-10-06T10:38:21.379113+0000","last_fullsized":"2023-10-06T10:38:21.379113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:40.679246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.401815+0000","last_change":"2023-10-06T10:38:21.393988+0000","last_active":"2023-10-06T10:38:21.401815+0000","last_peered":"2023-10-06T10:38:21.401815+0000","last_clean":"2023-10-06T10:38:21.401815+0000","last_became_active":"2023-10-06T10:38:21.393668+0000","last_became_peered":"2023-10-06T10:38:21.393668+0000","last_unstale":"2023-10-06T10:38:21.401815+0000","last_undegraded":"2023-10-06T10:38:21.401815+0000","last_fullsized":"2023-10-06T10:38:21.401815+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:33.926723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385218+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:38:21.385218+0000","last_peered":"2023-10-06T10:38:21.385218+0000","last_clean":"2023-10-06T10:38:21.385218+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:38:21.385218+0000","last_undegraded":"2023-10-06T10:38:21.385218+0000","last_fullsized":"2023-10-06T10:38:21.385218+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.379391+0000","last_change":"2023-10-06T10:38:16.360214+0000","last_active":"2023-10-06T10:38:21.379391+0000","last_peered":"2023-10-06T10:38:21.379391+0000","last_clean":"2023-10-06T10:38:21.379391+0000","last_became_active":"2023-10-06T10:38:16.359912+0000","last_became_peered":"2023-10-06T10:38:16.359912+0000","last_unstale":"2023-10-06T10:38:21.379391+0000","last_undegraded":"2023-10-06T10:38:21.379391+0000","last_fullsized":"2023-10-06T10:38:21.379391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:41.837644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377275+0000","last_change":"2023-10-06T10:38:17.354930+0000","last_active":"2023-10-06T10:38:21.377275+0000","last_peered":"2023-10-06T10:38:21.377275+0000","last_clean":"2023-10-06T10:38:21.377275+0000","last_became_active":"2023-10-06T10:38:17.354637+0000","last_became_peered":"2023-10-06T10:38:17.354637+0000","last_unstale":"2023-10-06T10:38:21.377275+0000","last_undegraded":"2023-10-06T10:38:21.377275+0000","last_fullsized":"2023-10-06T10:38:21.377275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:03.953719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391646+0000","last_change":"2023-10-06T10:38:21.387944+0000","last_active":"2023-10-06T10:38:21.391646+0000","last_peered":"2023-10-06T10:38:21.391646+0000","last_clean":"2023-10-06T10:38:21.391646+0000","last_became_active":"2023-10-06T10:38:21.387580+0000","last_became_peered":"2023-10-06T10:38:21.387580+0000","last_unstale":"2023-10-06T10:38:21.391646+0000","last_undegraded":"2023-10-06T10:38:21.391646+0000","last_fullsized":"2023-10-06T10:38:21.391646+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:54:20.934943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385079+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:38:21.385079+0000","last_peered":"2023-10-06T10:38:21.385079+0000","last_clean":"2023-10-06T10:38:21.385079+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:38:21.385079+0000","last_undegraded":"2023-10-06T10:38:21.385079+0000","last_fullsized":"2023-10-06T10:38:21.385079+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377995+0000","last_change":"2023-10-06T10:38:16.358519+0000","last_active":"2023-10-06T10:38:21.377995+0000","last_peered":"2023-10-06T10:38:21.377995+0000","last_clean":"2023-10-06T10:38:21.377995+0000","last_became_active":"2023-10-06T10:38:16.357730+0000","last_became_peered":"2023-10-06T10:38:16.357730+0000","last_unstale":"2023-10-06T10:38:21.377995+0000","last_undegraded":"2023-10-06T10:38:21.377995+0000","last_fullsized":"2023-10-06T10:38:21.377995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:36.469014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378421+0000","last_change":"2023-10-06T10:38:17.358314+0000","last_active":"2023-10-06T10:38:21.378421+0000","last_peered":"2023-10-06T10:38:21.378421+0000","last_clean":"2023-10-06T10:38:21.378421+0000","last_became_active":"2023-10-06T10:38:17.358060+0000","last_became_peered":"2023-10-06T10:38:17.358060+0000","last_unstale":"2023-10-06T10:38:21.378421+0000","last_undegraded":"2023-10-06T10:38:21.378421+0000","last_fullsized":"2023-10-06T10:38:21.378421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:34:27.066675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.400514+0000","last_change":"2023-10-06T10:38:21.391294+0000","last_active":"2023-10-06T10:38:21.400514+0000","last_peered":"2023-10-06T10:38:21.400514+0000","last_clean":"2023-10-06T10:38:21.400514+0000","last_became_active":"2023-10-06T10:38:21.390927+0000","last_became_peered":"2023-10-06T10:38:21.390927+0000","last_unstale":"2023-10-06T10:38:21.400514+0000","last_undegraded":"2023-10-06T10:38:21.400514+0000","last_fullsized":"2023-10-06T10:38:21.400514+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:53.012871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377366+0000","last_change":"2023-10-06T10:38:16.360511+0000","last_active":"2023-10-06T10:38:21.377366+0000","last_peered":"2023-10-06T10:38:21.377366+0000","last_clean":"2023-10-06T10:38:21.377366+0000","last_became_active":"2023-10-06T10:38:16.360260+0000","last_became_peered":"2023-10-06T10:38:16.360260+0000","last_unstale":"2023-10-06T10:38:21.377366+0000","last_undegraded":"2023-10-06T10:38:21.377366+0000","last_fullsized":"2023-10-06T10:38:21.377366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:57.215481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.383881+0000","last_change":"2023-10-06T10:38:17.365370+0000","last_active":"2023-10-06T10:38:21.383881+0000","last_peered":"2023-10-06T10:38:21.383881+0000","last_clean":"2023-10-06T10:38:21.383881+0000","last_became_active":"2023-10-06T10:38:17.365110+0000","last_became_peered":"2023-10-06T10:38:17.365110+0000","last_unstale":"2023-10-06T10:38:21.383881+0000","last_undegraded":"2023-10-06T10:38:21.383881+0000","last_fullsized":"2023-10-06T10:38:21.383881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:37.475956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402373+0000","last_change":"2023-10-06T10:38:21.399284+0000","last_active":"2023-10-06T10:38:21.402373+0000","last_peered":"2023-10-06T10:38:21.402373+0000","last_clean":"2023-10-06T10:38:21.402373+0000","last_became_active":"2023-10-06T10:38:21.398931+0000","last_became_peered":"2023-10-06T10:38:21.398931+0000","last_unstale":"2023-10-06T10:38:21.402373+0000","last_undegraded":"2023-10-06T10:38:21.402373+0000","last_fullsized":"2023-10-06T10:38:21.402373+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:17:39.406756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375583+0000","last_change":"2023-10-06T10:38:16.353975+0000","last_active":"2023-10-06T10:38:21.375583+0000","last_peered":"2023-10-06T10:38:21.375583+0000","last_clean":"2023-10-06T10:38:21.375583+0000","last_became_active":"2023-10-06T10:38:16.353629+0000","last_became_peered":"2023-10-06T10:38:16.353629+0000","last_unstale":"2023-10-06T10:38:21.375583+0000","last_undegraded":"2023-10-06T10:38:21.375583+0000","last_fullsized":"2023-10-06T10:38:21.375583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:20.696308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377446+0000","last_change":"2023-10-06T10:38:17.363838+0000","last_active":"2023-10-06T10:38:21.377446+0000","last_peered":"2023-10-06T10:38:21.377446+0000","last_clean":"2023-10-06T10:38:21.377446+0000","last_became_active":"2023-10-06T10:38:17.363580+0000","last_became_peered":"2023-10-06T10:38:17.363580+0000","last_unstale":"2023-10-06T10:38:21.377446+0000","last_undegraded":"2023-10-06T10:38:21.377446+0000","last_fullsized":"2023-10-06T10:38:21.377446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:43.614842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.399586+0000","last_change":"2023-10-06T10:38:21.396452+0000","last_active":"2023-10-06T10:38:21.399586+0000","last_peered":"2023-10-06T10:38:21.399586+0000","last_clean":"2023-10-06T10:38:21.399586+0000","last_became_active":"2023-10-06T10:38:21.396052+0000","last_became_peered":"2023-10-06T10:38:21.396052+0000","last_unstale":"2023-10-06T10:38:21.399586+0000","last_undegraded":"2023-10-06T10:38:21.399586+0000","last_fullsized":"2023-10-06T10:38:21.399586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:17:39.406756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377909+0000","last_change":"2023-10-06T10:38:16.354639+0000","last_active":"2023-10-06T10:38:21.377909+0000","last_peered":"2023-10-06T10:38:21.377909+0000","last_clean":"2023-10-06T10:38:21.377909+0000","last_became_active":"2023-10-06T10:38:16.354367+0000","last_became_peered":"2023-10-06T10:38:16.354367+0000","last_unstale":"2023-10-06T10:38:21.377909+0000","last_undegraded":"2023-10-06T10:38:21.377909+0000","last_fullsized":"2023-10-06T10:38:21.377909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:57.215481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377353+0000","last_change":"2023-10-06T10:38:17.358682+0000","last_active":"2023-10-06T10:38:21.377353+0000","last_peered":"2023-10-06T10:38:21.377353+0000","last_clean":"2023-10-06T10:38:21.377353+0000","last_became_active":"2023-10-06T10:38:17.358409+0000","last_became_peered":"2023-10-06T10:38:17.358409+0000","last_unstale":"2023-10-06T10:38:21.377353+0000","last_undegraded":"2023-10-06T10:38:21.377353+0000","last_fullsized":"2023-10-06T10:38:21.377353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:52:14.320333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391815+0000","last_change":"2023-10-06T10:38:21.390566+0000","last_active":"2023-10-06T10:38:21.391815+0000","last_peered":"2023-10-06T10:38:21.391815+0000","last_clean":"2023-10-06T10:38:21.391815+0000","last_became_active":"2023-10-06T10:38:21.390172+0000","last_became_peered":"2023-10-06T10:38:21.390172+0000","last_unstale":"2023-10-06T10:38:21.391815+0000","last_undegraded":"2023-10-06T10:38:21.391815+0000","last_fullsized":"2023-10-06T10:38:21.391815+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:37.174852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372471+0000","last_change":"2023-10-06T10:38:16.353396+0000","last_active":"2023-10-06T10:38:21.372471+0000","last_peered":"2023-10-06T10:38:21.372471+0000","last_clean":"2023-10-06T10:38:21.372471+0000","last_became_active":"2023-10-06T10:38:16.353136+0000","last_became_peered":"2023-10-06T10:38:16.353136+0000","last_unstale":"2023-10-06T10:38:21.372471+0000","last_undegraded":"2023-10-06T10:38:21.372471+0000","last_fullsized":"2023-10-06T10:38:21.372471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:02.115831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372424+0000","last_change":"2023-10-06T10:38:17.364589+0000","last_active":"2023-10-06T10:38:21.372424+0000","last_peered":"2023-10-06T10:38:21.372424+0000","last_clean":"2023-10-06T10:38:21.372424+0000","last_became_active":"2023-10-06T10:38:17.364364+0000","last_became_peered":"2023-10-06T10:38:17.364364+0000","last_unstale":"2023-10-06T10:38:21.372424+0000","last_undegraded":"2023-10-06T10:38:21.372424+0000","last_fullsized":"2023-10-06T10:38:21.372424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:13.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.399883+0000","last_change":"2023-10-06T10:38:21.395180+0000","last_active":"2023-10-06T10:38:21.399883+0000","last_peered":"2023-10-06T10:38:21.399883+0000","last_clean":"2023-10-06T10:38:21.399883+0000","last_became_active":"2023-10-06T10:38:21.394859+0000","last_became_peered":"2023-10-06T10:38:21.394859+0000","last_unstale":"2023-10-06T10:38:21.399883+0000","last_undegraded":"2023-10-06T10:38:21.399883+0000","last_fullsized":"2023-10-06T10:38:21.399883+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:54:20.934943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378521+0000","last_change":"2023-10-06T10:38:16.361291+0000","last_active":"2023-10-06T10:38:21.378521+0000","last_peered":"2023-10-06T10:38:21.378521+0000","last_clean":"2023-10-06T10:38:21.378521+0000","last_became_active":"2023-10-06T10:38:16.361027+0000","last_became_peered":"2023-10-06T10:38:16.361027+0000","last_unstale":"2023-10-06T10:38:21.378521+0000","last_undegraded":"2023-10-06T10:38:21.378521+0000","last_fullsized":"2023-10-06T10:38:21.378521+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:36.469014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378387+0000","last_change":"2023-10-06T10:38:17.375274+0000","last_active":"2023-10-06T10:38:21.378387+0000","last_peered":"2023-10-06T10:38:21.378387+0000","last_clean":"2023-10-06T10:38:21.378387+0000","last_became_active":"2023-10-06T10:38:17.374982+0000","last_became_peered":"2023-10-06T10:38:17.374982+0000","last_unstale":"2023-10-06T10:38:21.378387+0000","last_undegraded":"2023-10-06T10:38:21.378387+0000","last_fullsized":"2023-10-06T10:38:21.378387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:59.822053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385168+0000","last_change":"2023-10-06T10:38:17.368598+0000","last_active":"2023-10-06T10:38:21.385168+0000","last_peered":"2023-10-06T10:38:21.385168+0000","last_clean":"2023-10-06T10:38:21.385168+0000","last_became_active":"2023-10-06T10:38:17.367720+0000","last_became_peered":"2023-10-06T10:38:17.367720+0000","last_unstale":"2023-10-06T10:38:21.385168+0000","last_undegraded":"2023-10-06T10:38:21.385168+0000","last_fullsized":"2023-10-06T10:38:21.385168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:40.679246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372285+0000","last_change":"2023-10-06T10:38:17.362818+0000","last_active":"2023-10-06T10:38:21.372285+0000","last_peered":"2023-10-06T10:38:21.372285+0000","last_clean":"2023-10-06T10:38:21.372285+0000","last_became_active":"2023-10-06T10:38:17.362585+0000","last_became_peered":"2023-10-06T10:38:17.362585+0000","last_unstale":"2023-10-06T10:38:21.372285+0000","last_undegraded":"2023-10-06T10:38:21.372285+0000","last_fullsized":"2023-10-06T10:38:21.372285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:03.650819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378216+0000","last_change":"2023-10-06T10:38:17.375510+0000","last_active":"2023-10-06T10:38:21.378216+0000","last_peered":"2023-10-06T10:38:21.378216+0000","last_clean":"2023-10-06T10:38:21.378216+0000","last_became_active":"2023-10-06T10:38:17.375023+0000","last_became_peered":"2023-10-06T10:38:17.375023+0000","last_unstale":"2023-10-06T10:38:21.378216+0000","last_undegraded":"2023-10-06T10:38:21.378216+0000","last_fullsized":"2023-10-06T10:38:21.378216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:10.177603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372410+0000","last_change":"2023-10-06T10:38:17.370520+0000","last_active":"2023-10-06T10:38:21.372410+0000","last_peered":"2023-10-06T10:38:21.372410+0000","last_clean":"2023-10-06T10:38:21.372410+0000","last_became_active":"2023-10-06T10:38:17.370265+0000","last_became_peered":"2023-10-06T10:38:17.370265+0000","last_unstale":"2023-10-06T10:38:21.372410+0000","last_undegraded":"2023-10-06T10:38:21.372410+0000","last_fullsized":"2023-10-06T10:38:21.372410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:06.214524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.379183+0000","last_change":"2023-10-06T10:38:17.372067+0000","last_active":"2023-10-06T10:38:21.379183+0000","last_peered":"2023-10-06T10:38:21.379183+0000","last_clean":"2023-10-06T10:38:21.379183+0000","last_became_active":"2023-10-06T10:38:17.371813+0000","last_became_peered":"2023-10-06T10:38:17.371813+0000","last_unstale":"2023-10-06T10:38:21.379183+0000","last_undegraded":"2023-10-06T10:38:21.379183+0000","last_fullsized":"2023-10-06T10:38:21.379183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:23:23.463492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4484,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":876544,"data_stored":698736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":103085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298116,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073724,"statfs":{"total":96636764160,"available":96331493376,"internally_reserved":0,"allocated":847872,"data_stored":694806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":258048,"data_stored":104992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":104438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":258048,"data_stored":104528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.248}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297556,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074284,"statfs":{"total":96636764160,"available":96332066816,"internally_reserved":0,"allocated":274432,"data_stored":108208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":105062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:38:31.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:38:31.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.594+0000 7f1689c61700 1 Processor -- start 2023-10-06T10:38:31.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.594+0000 7f1689c61700 1 -- start start 2023-10-06T10:38:31.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.595+0000 7f1689c61700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1684157360 0x7f168406ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:31.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.595+0000 7f1689c61700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f16840b2820 0x7f1684157730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:31.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.595+0000 7f1689c61700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f168406b470 con 0x7f16840af670 2023-10-06T10:38:31.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.595+0000 7f1689c61700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f168406ab70 con 0x7f1684157360 2023-10-06T10:38:31.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.595+0000 7f1689c61700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16840afaa0 con 0x7f16840b2820 2023-10-06T10:38:31.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.595+0000 7f1688c5f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1684157360 0x7f168406ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:31.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.595+0000 7f1688c5f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1684157360 0x7f168406ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37916/0 (socket says 172.21.15.92:37916) 2023-10-06T10:38:31.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.595+0000 7f167bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f16840af670 0x7f16840b2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59958/0 (socket says 172.21.15.92:59958) 2023-10-06T10:38:31.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.595+0000 7f1688c5f700 1 -- 172.21.15.92:0/1313441308 learned_addr learned my addr 172.21.15.92:0/1313441308 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:31.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.595+0000 7f1689460700 1 --2- 172.21.15.92:0/1313441308 >> v2:172.21.15.124:3300/0 conn(0x7f16840b2820 0x7f1684157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:31.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.595+0000 7f1688c5f700 1 -- 172.21.15.92:0/1313441308 >> v2:172.21.15.124:3300/0 conn(0x7f16840b2820 msgr2=0x7f1684157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:31.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.595+0000 7f1688c5f700 1 --2- 172.21.15.92:0/1313441308 >> v2:172.21.15.124:3300/0 conn(0x7f16840b2820 0x7f1684157730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.595+0000 7f1688c5f700 1 -- 172.21.15.92:0/1313441308 >> v1:172.21.15.92:6789/0 conn(0x7f16840af670 legacy=0x7f16840b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:31.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.595+0000 7f1688c5f700 1 -- 172.21.15.92:0/1313441308 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16840c97c0 con 0x7f1684157360 2023-10-06T10:38:31.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.595+0000 7f167b7fe700 1 -- 172.21.15.92:0/1313441308 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 207056238 0 0) 0x7f168406b470 con 0x7f16840af670 2023-10-06T10:38:31.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.596+0000 7f1688c5f700 1 --2- 172.21.15.92:0/1313441308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1684157360 0x7f168406ee90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f168406de30 tx=0x7f1680009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62defa072e6788af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:31.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.596+0000 7f167b7fe700 1 -- 172.21.15.92:0/1313441308 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168000e070 con 0x7f1684157360 2023-10-06T10:38:31.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.596+0000 7f167b7fe700 1 -- 172.21.15.92:0/1313441308 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168000ba40 con 0x7f1684157360 2023-10-06T10:38:31.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.596+0000 7f167b7fe700 1 -- 172.21.15.92:0/1313441308 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16800092e0 con 0x7f1684157360 2023-10-06T10:38:31.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.596+0000 7f1689c61700 1 -- 172.21.15.92:0/1313441308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1684157360 msgr2=0x7f168406ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:31.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.596+0000 7f1689c61700 1 --2- 172.21.15.92:0/1313441308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1684157360 0x7f168406ee90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f168406de30 tx=0x7f1680009840 comp rx=0 tx=0).stop 2023-10-06T10:38:31.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.596+0000 7f1689c61700 1 -- 172.21.15.92:0/1313441308 shutdown_connections 2023-10-06T10:38:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.596+0000 7f1689c61700 1 --2- 172.21.15.92:0/1313441308 >> v2:172.21.15.124:3300/0 conn(0x7f16840b2820 0x7f1684157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.596+0000 7f1689c61700 1 --2- 172.21.15.92:0/1313441308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1684157360 0x7f168406ee90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.596+0000 7f1689c61700 1 -- 172.21.15.92:0/1313441308 >> 172.21.15.92:0/1313441308 conn(0x7f1684069190 msgr2=0x7f16840aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.596+0000 7f1689c61700 1 -- 172.21.15.92:0/1313441308 shutdown_connections 2023-10-06T10:38:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.596+0000 7f1689c61700 1 -- 172.21.15.92:0/1313441308 wait complete. 2023-10-06T10:38:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.597+0000 7f1689c61700 1 Processor -- start 2023-10-06T10:38:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.598+0000 7f1689c61700 1 -- start start 2023-10-06T10:38:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.598+0000 7f1689c61700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16840af670 0x7f16841533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.598+0000 7f1689c61700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f16840b2820 0x7f16841538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.598+0000 7f1689c61700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1684157360 0x7f16841682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.598+0000 7f1689c61700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1684153e30 con 0x7f16840af670 2023-10-06T10:38:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.598+0000 7f1689c61700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16840a58a0 con 0x7f16840b2820 2023-10-06T10:38:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.598+0000 7f1689c61700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1684154030 con 0x7f1684157360 2023-10-06T10:38:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.598+0000 7f1688c5f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16840af670 0x7f16841533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.598+0000 7f1688c5f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16840af670 0x7f16841533b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37920/0 (socket says 172.21.15.92:37920) 2023-10-06T10:38:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.598+0000 7f1688c5f700 1 -- 172.21.15.92:0/294487373 learned_addr learned my addr 172.21.15.92:0/294487373 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.598+0000 7f167bfff700 1 --2- 172.21.15.92:0/294487373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f16840b2820 0x7f16841538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.598+0000 7f1689460700 1 --2- 172.21.15.92:0/294487373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1684157360 0x7f16841682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.599+0000 7f1688c5f700 1 -- 172.21.15.92:0/294487373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1684157360 msgr2=0x7f16841682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.599+0000 7f1688c5f700 1 --2- 172.21.15.92:0/294487373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1684157360 0x7f16841682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.599+0000 7f1688c5f700 1 -- 172.21.15.92:0/294487373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f16840b2820 msgr2=0x7f16841538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:31.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.599+0000 7f1688c5f700 1 --2- 172.21.15.92:0/294487373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f16840b2820 0x7f16841538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.599+0000 7f1688c5f700 1 -- 172.21.15.92:0/294487373 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1680009ae0 con 0x7f16840af670 2023-10-06T10:38:31.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.599+0000 7f1689460700 1 --2- 172.21.15.92:0/294487373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1684157360 0x7f16841682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:31.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.599+0000 7f1688c5f700 1 --2- 172.21.15.92:0/294487373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16840af670 0x7f16841533b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1684064f60 tx=0x7f1680018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:31.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.599+0000 7f1678ff9700 1 -- 172.21.15.92:0/294487373 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1680008a80 con 0x7f16840af670 2023-10-06T10:38:31.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.599+0000 7f1678ff9700 1 -- 172.21.15.92:0/294487373 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1680008be0 con 0x7f16840af670 2023-10-06T10:38:31.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.599+0000 7f1689c61700 1 -- 172.21.15.92:0/294487373 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1684154310 con 0x7f16840af670 2023-10-06T10:38:31.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.599+0000 7f1678ff9700 1 -- 172.21.15.92:0/294487373 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1680025460 con 0x7f16840af670 2023-10-06T10:38:31.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.599+0000 7f1689c61700 1 -- 172.21.15.92:0/294487373 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16840a5530 con 0x7f16840af670 2023-10-06T10:38:31.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.600+0000 7f1689c61700 1 -- 172.21.15.92:0/294487373 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f168405f230 con 0x7f16840af670 2023-10-06T10:38:31.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.600+0000 7f1678ff9700 1 -- 172.21.15.92:0/294487373 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1680003040 con 0x7f16840af670 2023-10-06T10:38:31.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.601+0000 7f1678ff9700 1 --2- 172.21.15.92:0/294487373 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1670066f30 0x7f16700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:31.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.601+0000 7f167bfff700 1 --2- 172.21.15.92:0/294487373 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1670066f30 0x7f16700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:31.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.601+0000 7f1678ff9700 1 -- 172.21.15.92:0/294487373 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f1680086470 con 0x7f16840af670 2023-10-06T10:38:31.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.601+0000 7f167bfff700 1 --2- 172.21.15.92:0/294487373 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1670066f30 0x7f16700693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f16841563d0 tx=0x7f1674007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:31.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.605+0000 7f1678ff9700 1 -- 172.21.15.92:0/294487373 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1680053290 con 0x7f16840af670 2023-10-06T10:38:31.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.785+0000 7f1689c61700 1 -- 172.21.15.92:0/294487373 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f168406a5d0 con 0x7f1670066f30 2023-10-06T10:38:31.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.794+0000 7f1678ff9700 1 -- 172.21.15.92:0/294487373 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367548 (secure 0 0 0) 0x7f168406a5d0 con 0x7f1670066f30 2023-10-06T10:38:31.803 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:31.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.798+0000 7f1689c61700 1 -- 172.21.15.92:0/294487373 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1670066f30 msgr2=0x7f16700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:31.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.798+0000 7f1689c61700 1 --2- 172.21.15.92:0/294487373 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1670066f30 0x7f16700693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f16841563d0 tx=0x7f1674007d50 comp rx=0 tx=0).stop 2023-10-06T10:38:31.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.798+0000 7f1689c61700 1 -- 172.21.15.92:0/294487373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16840af670 msgr2=0x7f16841533b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:31.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.798+0000 7f1689c61700 1 --2- 172.21.15.92:0/294487373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16840af670 0x7f16841533b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1684064f60 tx=0x7f1680018000 comp rx=0 tx=0).stop 2023-10-06T10:38:31.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.798+0000 7f1689c61700 1 -- 172.21.15.92:0/294487373 shutdown_connections 2023-10-06T10:38:31.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.798+0000 7f1689c61700 1 --2- 172.21.15.92:0/294487373 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1670066f30 0x7f16700693e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.798+0000 7f1689c61700 1 --2- 172.21.15.92:0/294487373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1684157360 0x7f16841682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.798+0000 7f1689c61700 1 --2- 172.21.15.92:0/294487373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16840af670 0x7f16841533b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.798+0000 7f1689c61700 1 --2- 172.21.15.92:0/294487373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f16840b2820 0x7f16841538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.798+0000 7f1689c61700 1 -- 172.21.15.92:0/294487373 >> 172.21.15.92:0/294487373 conn(0x7f1684069190 msgr2=0x7f168415aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:31.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.798+0000 7f1689c61700 1 -- 172.21.15.92:0/294487373 shutdown_connections 2023-10-06T10:38:31.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.798+0000 7f1689c61700 1 -- 172.21.15.92:0/294487373 wait complete. 2023-10-06T10:38:31.809 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:38:31.812 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-10-06T10:38:30.083664+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594852,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381508,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593212,"statfs":{"total":773094113280,"available":770655449088,"internally_reserved":0,"allocated":3280896,"data_stored":2023855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2178,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.712825"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384275+0000","last_change":"2023-10-06T10:38:17.365963+0000","last_active":"2023-10-06T10:38:21.384275+0000","last_peered":"2023-10-06T10:38:21.384275+0000","last_clean":"2023-10-06T10:38:21.384275+0000","last_became_active":"2023-10-06T10:38:17.365702+0000","last_became_peered":"2023-10-06T10:38:17.365702+0000","last_unstale":"2023-10-06T10:38:21.384275+0000","last_undegraded":"2023-10-06T10:38:21.384275+0000","last_fullsized":"2023-10-06T10:38:21.384275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:58.222423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378758+0000","last_change":"2023-10-06T10:38:17.362155+0000","last_active":"2023-10-06T10:38:21.378758+0000","last_peered":"2023-10-06T10:38:21.378758+0000","last_clean":"2023-10-06T10:38:21.378758+0000","last_became_active":"2023-10-06T10:38:17.361871+0000","last_became_peered":"2023-10-06T10:38:17.361871+0000","last_unstale":"2023-10-06T10:38:21.378758+0000","last_undegraded":"2023-10-06T10:38:21.378758+0000","last_fullsized":"2023-10-06T10:38:21.378758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:49.973815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377999+0000","last_change":"2023-10-06T10:38:17.361502+0000","last_active":"2023-10-06T10:38:21.377999+0000","last_peered":"2023-10-06T10:38:21.377999+0000","last_clean":"2023-10-06T10:38:21.377999+0000","last_became_active":"2023-10-06T10:38:17.360866+0000","last_became_peered":"2023-10-06T10:38:17.360866+0000","last_unstale":"2023-10-06T10:38:21.377999+0000","last_undegraded":"2023-10-06T10:38:21.377999+0000","last_fullsized":"2023-10-06T10:38:21.377999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:01.022639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384859+0000","last_change":"2023-10-06T10:38:17.366612+0000","last_active":"2023-10-06T10:38:21.384859+0000","last_peered":"2023-10-06T10:38:21.384859+0000","last_clean":"2023-10-06T10:38:21.384859+0000","last_became_active":"2023-10-06T10:38:17.366337+0000","last_became_peered":"2023-10-06T10:38:17.366337+0000","last_unstale":"2023-10-06T10:38:21.384859+0000","last_undegraded":"2023-10-06T10:38:21.384859+0000","last_fullsized":"2023-10-06T10:38:21.384859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:01:12.162890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377172+0000","last_change":"2023-10-06T10:38:17.363460+0000","last_active":"2023-10-06T10:38:21.377172+0000","last_peered":"2023-10-06T10:38:21.377172+0000","last_clean":"2023-10-06T10:38:21.377172+0000","last_became_active":"2023-10-06T10:38:17.363211+0000","last_became_peered":"2023-10-06T10:38:17.363211+0000","last_unstale":"2023-10-06T10:38:21.377172+0000","last_undegraded":"2023-10-06T10:38:21.377172+0000","last_fullsized":"2023-10-06T10:38:21.377172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.132681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378316+0000","last_change":"2023-10-06T10:38:17.364098+0000","last_active":"2023-10-06T10:38:21.378316+0000","last_peered":"2023-10-06T10:38:21.378316+0000","last_clean":"2023-10-06T10:38:21.378316+0000","last_became_active":"2023-10-06T10:38:17.363806+0000","last_became_peered":"2023-10-06T10:38:17.363806+0000","last_unstale":"2023-10-06T10:38:21.378316+0000","last_undegraded":"2023-10-06T10:38:21.378316+0000","last_fullsized":"2023-10-06T10:38:21.378316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:27.532414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391179+0000","last_change":"2023-10-06T10:38:17.372436+0000","last_active":"2023-10-06T10:38:21.391179+0000","last_peered":"2023-10-06T10:38:21.391179+0000","last_clean":"2023-10-06T10:38:21.391179+0000","last_became_active":"2023-10-06T10:38:17.371126+0000","last_became_peered":"2023-10-06T10:38:17.371126+0000","last_unstale":"2023-10-06T10:38:21.391179+0000","last_undegraded":"2023-10-06T10:38:21.391179+0000","last_fullsized":"2023-10-06T10:38:21.391179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:13.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375652+0000","last_change":"2023-10-06T10:38:17.354022+0000","last_active":"2023-10-06T10:38:21.375652+0000","last_peered":"2023-10-06T10:38:21.375652+0000","last_clean":"2023-10-06T10:38:21.375652+0000","last_became_active":"2023-10-06T10:38:17.353630+0000","last_became_peered":"2023-10-06T10:38:17.353630+0000","last_unstale":"2023-10-06T10:38:21.375652+0000","last_undegraded":"2023-10-06T10:38:21.375652+0000","last_fullsized":"2023-10-06T10:38:21.375652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:03.953719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378266+0000","last_change":"2023-10-06T10:38:17.362560+0000","last_active":"2023-10-06T10:38:21.378266+0000","last_peered":"2023-10-06T10:38:21.378266+0000","last_clean":"2023-10-06T10:38:21.378266+0000","last_became_active":"2023-10-06T10:38:17.361953+0000","last_became_peered":"2023-10-06T10:38:17.361953+0000","last_unstale":"2023-10-06T10:38:21.378266+0000","last_undegraded":"2023-10-06T10:38:21.378266+0000","last_fullsized":"2023-10-06T10:38:21.378266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:38.417248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378932+0000","last_change":"2023-10-06T10:38:17.364442+0000","last_active":"2023-10-06T10:38:21.378932+0000","last_peered":"2023-10-06T10:38:21.378932+0000","last_clean":"2023-10-06T10:38:21.378932+0000","last_became_active":"2023-10-06T10:38:17.363787+0000","last_became_peered":"2023-10-06T10:38:17.363787+0000","last_unstale":"2023-10-06T10:38:21.378932+0000","last_undegraded":"2023-10-06T10:38:21.378932+0000","last_fullsized":"2023-10-06T10:38:21.378932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:13.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377971+0000","last_change":"2023-10-06T10:38:17.366791+0000","last_active":"2023-10-06T10:38:21.377971+0000","last_peered":"2023-10-06T10:38:21.377971+0000","last_clean":"2023-10-06T10:38:21.377971+0000","last_became_active":"2023-10-06T10:38:17.366281+0000","last_became_peered":"2023-10-06T10:38:17.366281+0000","last_unstale":"2023-10-06T10:38:21.377971+0000","last_undegraded":"2023-10-06T10:38:21.377971+0000","last_fullsized":"2023-10-06T10:38:21.377971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:22.349323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.383809+0000","last_change":"2023-10-06T10:38:17.368900+0000","last_active":"2023-10-06T10:38:21.383809+0000","last_peered":"2023-10-06T10:38:21.383809+0000","last_clean":"2023-10-06T10:38:21.383809+0000","last_became_active":"2023-10-06T10:38:17.368488+0000","last_became_peered":"2023-10-06T10:38:17.368488+0000","last_unstale":"2023-10-06T10:38:21.383809+0000","last_undegraded":"2023-10-06T10:38:21.383809+0000","last_fullsized":"2023-10-06T10:38:21.383809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:07.365430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.383322+0000","last_change":"2023-10-06T10:38:17.365338+0000","last_active":"2023-10-06T10:38:21.383322+0000","last_peered":"2023-10-06T10:38:21.383322+0000","last_clean":"2023-10-06T10:38:21.383322+0000","last_became_active":"2023-10-06T10:38:17.365056+0000","last_became_peered":"2023-10-06T10:38:17.365056+0000","last_unstale":"2023-10-06T10:38:21.383322+0000","last_undegraded":"2023-10-06T10:38:21.383322+0000","last_fullsized":"2023-10-06T10:38:21.383322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:03.122773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375751+0000","last_change":"2023-10-06T10:38:17.366413+0000","last_active":"2023-10-06T10:38:21.375751+0000","last_peered":"2023-10-06T10:38:21.375751+0000","last_clean":"2023-10-06T10:38:21.375751+0000","last_became_active":"2023-10-06T10:38:17.366139+0000","last_became_peered":"2023-10-06T10:38:17.366139+0000","last_unstale":"2023-10-06T10:38:21.375751+0000","last_undegraded":"2023-10-06T10:38:21.375751+0000","last_fullsized":"2023-10-06T10:38:21.375751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:34:27.066675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391014+0000","last_change":"2023-10-06T10:38:17.371250+0000","last_active":"2023-10-06T10:38:21.391014+0000","last_peered":"2023-10-06T10:38:21.391014+0000","last_clean":"2023-10-06T10:38:21.391014+0000","last_became_active":"2023-10-06T10:38:17.370858+0000","last_became_peered":"2023-10-06T10:38:17.370858+0000","last_unstale":"2023-10-06T10:38:21.391014+0000","last_undegraded":"2023-10-06T10:38:21.391014+0000","last_fullsized":"2023-10-06T10:38:21.391014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:28.001417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378433+0000","last_change":"2023-10-06T10:38:17.364371+0000","last_active":"2023-10-06T10:38:21.378433+0000","last_peered":"2023-10-06T10:38:21.378433+0000","last_clean":"2023-10-06T10:38:21.378433+0000","last_became_active":"2023-10-06T10:38:17.364040+0000","last_became_peered":"2023-10-06T10:38:17.364040+0000","last_unstale":"2023-10-06T10:38:21.378433+0000","last_undegraded":"2023-10-06T10:38:21.378433+0000","last_fullsized":"2023-10-06T10:38:21.378433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:29.897092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378355+0000","last_change":"2023-10-06T10:38:17.363058+0000","last_active":"2023-10-06T10:38:21.378355+0000","last_peered":"2023-10-06T10:38:21.378355+0000","last_clean":"2023-10-06T10:38:21.378355+0000","last_became_active":"2023-10-06T10:38:17.359050+0000","last_became_peered":"2023-10-06T10:38:17.359050+0000","last_unstale":"2023-10-06T10:38:21.378355+0000","last_undegraded":"2023-10-06T10:38:21.378355+0000","last_fullsized":"2023-10-06T10:38:21.378355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:28.001417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377744+0000","last_change":"2023-10-06T10:38:17.365982+0000","last_active":"2023-10-06T10:38:21.377744+0000","last_peered":"2023-10-06T10:38:21.377744+0000","last_clean":"2023-10-06T10:38:21.377744+0000","last_became_active":"2023-10-06T10:38:17.364876+0000","last_became_peered":"2023-10-06T10:38:17.364876+0000","last_unstale":"2023-10-06T10:38:21.377744+0000","last_undegraded":"2023-10-06T10:38:21.377744+0000","last_fullsized":"2023-10-06T10:38:21.377744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:13.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.383516+0000","last_change":"2023-10-06T10:38:17.368880+0000","last_active":"2023-10-06T10:38:21.383516+0000","last_peered":"2023-10-06T10:38:21.383516+0000","last_clean":"2023-10-06T10:38:21.383516+0000","last_became_active":"2023-10-06T10:38:17.368645+0000","last_became_peered":"2023-10-06T10:38:17.368645+0000","last_unstale":"2023-10-06T10:38:21.383516+0000","last_undegraded":"2023-10-06T10:38:21.383516+0000","last_fullsized":"2023-10-06T10:38:21.383516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:42.844587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377767+0000","last_change":"2023-10-06T10:38:17.375213+0000","last_active":"2023-10-06T10:38:21.377767+0000","last_peered":"2023-10-06T10:38:21.377767+0000","last_clean":"2023-10-06T10:38:21.377767+0000","last_became_active":"2023-10-06T10:38:17.374836+0000","last_became_peered":"2023-10-06T10:38:17.374836+0000","last_unstale":"2023-10-06T10:38:21.377767+0000","last_undegraded":"2023-10-06T10:38:21.377767+0000","last_fullsized":"2023-10-06T10:38:21.377767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:01.727252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.386555+0000","last_change":"2023-10-06T10:38:17.370625+0000","last_active":"2023-10-06T10:38:21.386555+0000","last_peered":"2023-10-06T10:38:21.386555+0000","last_clean":"2023-10-06T10:38:21.386555+0000","last_became_active":"2023-10-06T10:38:17.370355+0000","last_became_peered":"2023-10-06T10:38:17.370355+0000","last_unstale":"2023-10-06T10:38:21.386555+0000","last_undegraded":"2023-10-06T10:38:21.386555+0000","last_fullsized":"2023-10-06T10:38:21.386555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:38.417248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375531+0000","last_change":"2023-10-06T10:38:17.360622+0000","last_active":"2023-10-06T10:38:21.375531+0000","last_peered":"2023-10-06T10:38:21.375531+0000","last_clean":"2023-10-06T10:38:21.375531+0000","last_became_active":"2023-10-06T10:38:17.359689+0000","last_became_peered":"2023-10-06T10:38:17.359689+0000","last_unstale":"2023-10-06T10:38:21.375531+0000","last_undegraded":"2023-10-06T10:38:21.375531+0000","last_fullsized":"2023-10-06T10:38:21.375531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:03.650819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372352+0000","last_change":"2023-10-06T10:38:17.362700+0000","last_active":"2023-10-06T10:38:21.372352+0000","last_peered":"2023-10-06T10:38:21.372352+0000","last_clean":"2023-10-06T10:38:21.372352+0000","last_became_active":"2023-10-06T10:38:17.362457+0000","last_became_peered":"2023-10-06T10:38:17.362457+0000","last_unstale":"2023-10-06T10:38:21.372352+0000","last_undegraded":"2023-10-06T10:38:21.372352+0000","last_fullsized":"2023-10-06T10:38:21.372352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:50.378097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.393253+0000","last_change":"2023-10-06T10:38:17.371996+0000","last_active":"2023-10-06T10:38:21.393253+0000","last_peered":"2023-10-06T10:38:21.393253+0000","last_clean":"2023-10-06T10:38:21.393253+0000","last_became_active":"2023-10-06T10:38:17.371731+0000","last_became_peered":"2023-10-06T10:38:17.371731+0000","last_unstale":"2023-10-06T10:38:21.393253+0000","last_undegraded":"2023-10-06T10:38:21.393253+0000","last_fullsized":"2023-10-06T10:38:21.393253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:06.214524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378923+0000","last_change":"2023-10-06T10:38:17.373079+0000","last_active":"2023-10-06T10:38:21.378923+0000","last_peered":"2023-10-06T10:38:21.378923+0000","last_clean":"2023-10-06T10:38:21.378923+0000","last_became_active":"2023-10-06T10:38:17.372799+0000","last_became_peered":"2023-10-06T10:38:17.372799+0000","last_unstale":"2023-10-06T10:38:21.378923+0000","last_undegraded":"2023-10-06T10:38:21.378923+0000","last_fullsized":"2023-10-06T10:38:21.378923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:03.953719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.379048+0000","last_change":"2023-10-06T10:38:17.366829+0000","last_active":"2023-10-06T10:38:21.379048+0000","last_peered":"2023-10-06T10:38:21.379048+0000","last_clean":"2023-10-06T10:38:21.379048+0000","last_became_active":"2023-10-06T10:38:17.366617+0000","last_became_peered":"2023-10-06T10:38:17.366617+0000","last_unstale":"2023-10-06T10:38:21.379048+0000","last_undegraded":"2023-10-06T10:38:21.379048+0000","last_fullsized":"2023-10-06T10:38:21.379048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:35.729655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378140+0000","last_change":"2023-10-06T10:38:17.371330+0000","last_active":"2023-10-06T10:38:21.378140+0000","last_peered":"2023-10-06T10:38:21.378140+0000","last_clean":"2023-10-06T10:38:21.378140+0000","last_became_active":"2023-10-06T10:38:17.370864+0000","last_became_peered":"2023-10-06T10:38:17.370864+0000","last_unstale":"2023-10-06T10:38:21.378140+0000","last_undegraded":"2023-10-06T10:38:21.378140+0000","last_fullsized":"2023-10-06T10:38:21.378140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:38.417248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391173+0000","last_change":"2023-10-06T10:38:21.389792+0000","last_active":"2023-10-06T10:38:21.391173+0000","last_peered":"2023-10-06T10:38:21.391173+0000","last_clean":"2023-10-06T10:38:21.391173+0000","last_became_active":"2023-10-06T10:38:21.389400+0000","last_became_peered":"2023-10-06T10:38:21.389400+0000","last_unstale":"2023-10-06T10:38:21.391173+0000","last_undegraded":"2023-10-06T10:38:21.391173+0000","last_fullsized":"2023-10-06T10:38:21.391173+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:55.438723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378102+0000","last_change":"2023-10-06T10:38:16.366893+0000","last_active":"2023-10-06T10:38:21.378102+0000","last_peered":"2023-10-06T10:38:21.378102+0000","last_clean":"2023-10-06T10:38:21.378102+0000","last_became_active":"2023-10-06T10:38:16.366637+0000","last_became_peered":"2023-10-06T10:38:16.366637+0000","last_unstale":"2023-10-06T10:38:21.378102+0000","last_undegraded":"2023-10-06T10:38:21.378102+0000","last_fullsized":"2023-10-06T10:38:21.378102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:06.358487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372687+0000","last_change":"2023-10-06T10:38:17.360671+0000","last_active":"2023-10-06T10:38:21.372687+0000","last_peered":"2023-10-06T10:38:21.372687+0000","last_clean":"2023-10-06T10:38:21.372687+0000","last_became_active":"2023-10-06T10:38:17.360439+0000","last_became_peered":"2023-10-06T10:38:17.360439+0000","last_unstale":"2023-10-06T10:38:21.372687+0000","last_undegraded":"2023-10-06T10:38:21.372687+0000","last_fullsized":"2023-10-06T10:38:21.372687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:45:42.362885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402648+0000","last_change":"2023-10-06T10:38:21.399346+0000","last_active":"2023-10-06T10:38:21.402648+0000","last_peered":"2023-10-06T10:38:21.402648+0000","last_clean":"2023-10-06T10:38:21.402648+0000","last_became_active":"2023-10-06T10:38:21.398877+0000","last_became_peered":"2023-10-06T10:38:21.398877+0000","last_unstale":"2023-10-06T10:38:21.402648+0000","last_undegraded":"2023-10-06T10:38:21.402648+0000","last_fullsized":"2023-10-06T10:38:21.402648+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:05.052271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.376684+0000","last_change":"2023-10-06T10:38:16.357015+0000","last_active":"2023-10-06T10:38:21.376684+0000","last_peered":"2023-10-06T10:38:21.376684+0000","last_clean":"2023-10-06T10:38:21.376684+0000","last_became_active":"2023-10-06T10:38:16.356733+0000","last_became_peered":"2023-10-06T10:38:16.356733+0000","last_unstale":"2023-10-06T10:38:21.376684+0000","last_undegraded":"2023-10-06T10:38:21.376684+0000","last_fullsized":"2023-10-06T10:38:21.376684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:06.358487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375991+0000","last_change":"2023-10-06T10:38:17.370483+0000","last_active":"2023-10-06T10:38:21.375991+0000","last_peered":"2023-10-06T10:38:21.375991+0000","last_clean":"2023-10-06T10:38:21.375991+0000","last_became_active":"2023-10-06T10:38:17.370196+0000","last_became_peered":"2023-10-06T10:38:17.370196+0000","last_unstale":"2023-10-06T10:38:21.375991+0000","last_undegraded":"2023-10-06T10:38:21.375991+0000","last_fullsized":"2023-10-06T10:38:21.375991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:49.973815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.398933+0000","last_change":"2023-10-06T10:38:21.397171+0000","last_active":"2023-10-06T10:38:21.398933+0000","last_peered":"2023-10-06T10:38:21.398933+0000","last_clean":"2023-10-06T10:38:21.398933+0000","last_became_active":"2023-10-06T10:38:21.396794+0000","last_became_peered":"2023-10-06T10:38:21.396794+0000","last_unstale":"2023-10-06T10:38:21.398933+0000","last_undegraded":"2023-10-06T10:38:21.398933+0000","last_fullsized":"2023-10-06T10:38:21.398933+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:30.495506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372622+0000","last_change":"2023-10-06T10:38:16.362088+0000","last_active":"2023-10-06T10:38:21.372622+0000","last_peered":"2023-10-06T10:38:21.372622+0000","last_clean":"2023-10-06T10:38:21.372622+0000","last_became_active":"2023-10-06T10:38:16.361790+0000","last_became_peered":"2023-10-06T10:38:16.361790+0000","last_unstale":"2023-10-06T10:38:21.372622+0000","last_undegraded":"2023-10-06T10:38:21.372622+0000","last_fullsized":"2023-10-06T10:38:21.372622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:20.696308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385166+0000","last_change":"2023-10-06T10:38:17.372313+0000","last_active":"2023-10-06T10:38:21.385166+0000","last_peered":"2023-10-06T10:38:21.385166+0000","last_clean":"2023-10-06T10:38:21.385166+0000","last_became_active":"2023-10-06T10:38:17.369061+0000","last_became_peered":"2023-10-06T10:38:17.369061+0000","last_unstale":"2023-10-06T10:38:21.385166+0000","last_undegraded":"2023-10-06T10:38:21.385166+0000","last_fullsized":"2023-10-06T10:38:21.385166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:25.807302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402371+0000","last_change":"2023-10-06T10:38:21.399352+0000","last_active":"2023-10-06T10:38:21.402371+0000","last_peered":"2023-10-06T10:38:21.402371+0000","last_clean":"2023-10-06T10:38:21.402371+0000","last_became_active":"2023-10-06T10:38:21.398679+0000","last_became_peered":"2023-10-06T10:38:21.398679+0000","last_unstale":"2023-10-06T10:38:21.402371+0000","last_undegraded":"2023-10-06T10:38:21.402371+0000","last_fullsized":"2023-10-06T10:38:21.402371+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:14.207234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378436+0000","last_change":"2023-10-06T10:38:16.361297+0000","last_active":"2023-10-06T10:38:21.378436+0000","last_peered":"2023-10-06T10:38:21.378436+0000","last_clean":"2023-10-06T10:38:21.378436+0000","last_became_active":"2023-10-06T10:38:16.361034+0000","last_became_peered":"2023-10-06T10:38:16.361034+0000","last_unstale":"2023-10-06T10:38:21.378436+0000","last_undegraded":"2023-10-06T10:38:21.378436+0000","last_fullsized":"2023-10-06T10:38:21.378436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:02.115831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372551+0000","last_change":"2023-10-06T10:38:17.361626+0000","last_active":"2023-10-06T10:38:21.372551+0000","last_peered":"2023-10-06T10:38:21.372551+0000","last_clean":"2023-10-06T10:38:21.372551+0000","last_became_active":"2023-10-06T10:38:17.361386+0000","last_became_peered":"2023-10-06T10:38:17.361386+0000","last_unstale":"2023-10-06T10:38:21.372551+0000","last_undegraded":"2023-10-06T10:38:21.372551+0000","last_fullsized":"2023-10-06T10:38:21.372551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.132681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.397814+0000","last_change":"2023-10-06T10:38:21.395253+0000","last_active":"2023-10-06T10:38:21.397814+0000","last_peered":"2023-10-06T10:38:21.397814+0000","last_clean":"2023-10-06T10:38:21.397814+0000","last_became_active":"2023-10-06T10:38:21.394839+0000","last_became_peered":"2023-10-06T10:38:21.394839+0000","last_unstale":"2023-10-06T10:38:21.397814+0000","last_undegraded":"2023-10-06T10:38:21.397814+0000","last_fullsized":"2023-10-06T10:38:21.397814+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:43.578698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378846+0000","last_change":"2023-10-06T10:38:16.359731+0000","last_active":"2023-10-06T10:38:21.378846+0000","last_peered":"2023-10-06T10:38:21.378846+0000","last_clean":"2023-10-06T10:38:21.378846+0000","last_became_active":"2023-10-06T10:38:16.359451+0000","last_became_peered":"2023-10-06T10:38:16.359451+0000","last_unstale":"2023-10-06T10:38:21.378846+0000","last_undegraded":"2023-10-06T10:38:21.378846+0000","last_fullsized":"2023-10-06T10:38:21.378846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:06.358487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372246+0000","last_change":"2023-10-06T10:38:17.371057+0000","last_active":"2023-10-06T10:38:21.372246+0000","last_peered":"2023-10-06T10:38:21.372246+0000","last_clean":"2023-10-06T10:38:21.372246+0000","last_became_active":"2023-10-06T10:38:17.370898+0000","last_became_peered":"2023-10-06T10:38:17.370898+0000","last_unstale":"2023-10-06T10:38:21.372246+0000","last_undegraded":"2023-10-06T10:38:21.372246+0000","last_fullsized":"2023-10-06T10:38:21.372246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:59.822053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.393290+0000","last_change":"2023-10-06T10:38:21.387382+0000","last_active":"2023-10-06T10:38:21.393290+0000","last_peered":"2023-10-06T10:38:21.393290+0000","last_clean":"2023-10-06T10:38:21.393290+0000","last_became_active":"2023-10-06T10:38:21.386886+0000","last_became_peered":"2023-10-06T10:38:21.386886+0000","last_unstale":"2023-10-06T10:38:21.393290+0000","last_undegraded":"2023-10-06T10:38:21.393290+0000","last_fullsized":"2023-10-06T10:38:21.393290+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:04:14.062486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.392484+0000","last_change":"2023-10-06T10:38:16.362467+0000","last_active":"2023-10-06T10:38:21.392484+0000","last_peered":"2023-10-06T10:38:21.392484+0000","last_clean":"2023-10-06T10:38:21.392484+0000","last_became_active":"2023-10-06T10:38:16.362094+0000","last_became_peered":"2023-10-06T10:38:16.362094+0000","last_unstale":"2023-10-06T10:38:21.392484+0000","last_undegraded":"2023-10-06T10:38:21.392484+0000","last_fullsized":"2023-10-06T10:38:21.392484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:24.800360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378505+0000","last_change":"2023-10-06T10:38:17.371049+0000","last_active":"2023-10-06T10:38:21.378505+0000","last_peered":"2023-10-06T10:38:21.378505+0000","last_clean":"2023-10-06T10:38:21.378505+0000","last_became_active":"2023-10-06T10:38:17.370698+0000","last_became_peered":"2023-10-06T10:38:17.370698+0000","last_unstale":"2023-10-06T10:38:21.378505+0000","last_undegraded":"2023-10-06T10:38:21.378505+0000","last_fullsized":"2023-10-06T10:38:21.378505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:28.001417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.400543+0000","last_change":"2023-10-06T10:38:21.395461+0000","last_active":"2023-10-06T10:38:21.400543+0000","last_peered":"2023-10-06T10:38:21.400543+0000","last_clean":"2023-10-06T10:38:21.400543+0000","last_became_active":"2023-10-06T10:38:21.395077+0000","last_became_peered":"2023-10-06T10:38:21.395077+0000","last_unstale":"2023-10-06T10:38:21.400543+0000","last_undegraded":"2023-10-06T10:38:21.400543+0000","last_fullsized":"2023-10-06T10:38:21.400543+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:43.578698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'9","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T10:38:27.096420+0000","last_change":"2023-10-06T10:38:16.359809+0000","last_active":"2023-10-06T10:38:27.096420+0000","last_peered":"2023-10-06T10:38:27.096420+0000","last_clean":"2023-10-06T10:38:27.096420+0000","last_became_active":"2023-10-06T10:38:16.359520+0000","last_became_peered":"2023-10-06T10:38:16.359520+0000","last_unstale":"2023-10-06T10:38:27.096420+0000","last_undegraded":"2023-10-06T10:38:27.096420+0000","last_fullsized":"2023-10-06T10:38:27.096420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:41.837644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3714,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377665+0000","last_change":"2023-10-06T10:38:17.375788+0000","last_active":"2023-10-06T10:38:21.377665+0000","last_peered":"2023-10-06T10:38:21.377665+0000","last_clean":"2023-10-06T10:38:21.377665+0000","last_became_active":"2023-10-06T10:38:17.375416+0000","last_became_peered":"2023-10-06T10:38:17.375416+0000","last_unstale":"2023-10-06T10:38:21.377665+0000","last_undegraded":"2023-10-06T10:38:21.377665+0000","last_fullsized":"2023-10-06T10:38:21.377665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.132681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.396799+0000","last_change":"2023-10-06T10:38:21.391788+0000","last_active":"2023-10-06T10:38:21.396799+0000","last_peered":"2023-10-06T10:38:21.396799+0000","last_clean":"2023-10-06T10:38:21.396799+0000","last_became_active":"2023-10-06T10:38:21.391376+0000","last_became_peered":"2023-10-06T10:38:21.391376+0000","last_unstale":"2023-10-06T10:38:21.396799+0000","last_undegraded":"2023-10-06T10:38:21.396799+0000","last_fullsized":"2023-10-06T10:38:21.396799+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:25:40.423524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384475+0000","last_change":"2023-10-06T10:38:16.363389+0000","last_active":"2023-10-06T10:38:21.384475+0000","last_peered":"2023-10-06T10:38:21.384475+0000","last_clean":"2023-10-06T10:38:21.384475+0000","last_became_active":"2023-10-06T10:38:16.363141+0000","last_became_peered":"2023-10-06T10:38:16.363141+0000","last_unstale":"2023-10-06T10:38:21.384475+0000","last_undegraded":"2023-10-06T10:38:21.384475+0000","last_fullsized":"2023-10-06T10:38:21.384475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:01:11.155948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372587+0000","last_change":"2023-10-06T10:38:17.361247+0000","last_active":"2023-10-06T10:38:21.372587+0000","last_peered":"2023-10-06T10:38:21.372587+0000","last_clean":"2023-10-06T10:38:21.372587+0000","last_became_active":"2023-10-06T10:38:17.361016+0000","last_became_peered":"2023-10-06T10:38:17.361016+0000","last_unstale":"2023-10-06T10:38:21.372587+0000","last_undegraded":"2023-10-06T10:38:21.372587+0000","last_fullsized":"2023-10-06T10:38:21.372587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:38.417248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.398908+0000","last_change":"2023-10-06T10:38:21.395222+0000","last_active":"2023-10-06T10:38:21.398908+0000","last_peered":"2023-10-06T10:38:21.398908+0000","last_clean":"2023-10-06T10:38:21.398908+0000","last_became_active":"2023-10-06T10:38:21.394695+0000","last_became_peered":"2023-10-06T10:38:21.394695+0000","last_unstale":"2023-10-06T10:38:21.398908+0000","last_undegraded":"2023-10-06T10:38:21.398908+0000","last_fullsized":"2023-10-06T10:38:21.398908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:36:21.761225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375670+0000","last_change":"2023-10-06T10:38:16.359510+0000","last_active":"2023-10-06T10:38:21.375670+0000","last_peered":"2023-10-06T10:38:21.375670+0000","last_clean":"2023-10-06T10:38:21.375670+0000","last_became_active":"2023-10-06T10:38:16.359236+0000","last_became_peered":"2023-10-06T10:38:16.359236+0000","last_unstale":"2023-10-06T10:38:21.375670+0000","last_undegraded":"2023-10-06T10:38:21.375670+0000","last_fullsized":"2023-10-06T10:38:21.375670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:16.528477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378073+0000","last_change":"2023-10-06T10:38:17.371135+0000","last_active":"2023-10-06T10:38:21.378073+0000","last_peered":"2023-10-06T10:38:21.378073+0000","last_clean":"2023-10-06T10:38:21.378073+0000","last_became_active":"2023-10-06T10:38:17.370494+0000","last_became_peered":"2023-10-06T10:38:17.370494+0000","last_unstale":"2023-10-06T10:38:21.378073+0000","last_undegraded":"2023-10-06T10:38:21.378073+0000","last_fullsized":"2023-10-06T10:38:21.378073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:28.001417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.398908+0000","last_change":"2023-10-06T10:38:21.395871+0000","last_active":"2023-10-06T10:38:21.398908+0000","last_peered":"2023-10-06T10:38:21.398908+0000","last_clean":"2023-10-06T10:38:21.398908+0000","last_became_active":"2023-10-06T10:38:21.395558+0000","last_became_peered":"2023-10-06T10:38:21.395558+0000","last_unstale":"2023-10-06T10:38:21.398908+0000","last_undegraded":"2023-10-06T10:38:21.398908+0000","last_fullsized":"2023-10-06T10:38:21.398908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:32:37.103555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391095+0000","last_change":"2023-10-06T10:38:16.363354+0000","last_active":"2023-10-06T10:38:21.391095+0000","last_peered":"2023-10-06T10:38:21.391095+0000","last_clean":"2023-10-06T10:38:21.391095+0000","last_became_active":"2023-10-06T10:38:16.363088+0000","last_became_peered":"2023-10-06T10:38:16.363088+0000","last_unstale":"2023-10-06T10:38:21.391095+0000","last_undegraded":"2023-10-06T10:38:21.391095+0000","last_fullsized":"2023-10-06T10:38:21.391095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:39.672304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.376233+0000","last_change":"2023-10-06T10:38:17.359300+0000","last_active":"2023-10-06T10:38:21.376233+0000","last_peered":"2023-10-06T10:38:21.376233+0000","last_clean":"2023-10-06T10:38:21.376233+0000","last_became_active":"2023-10-06T10:38:17.358652+0000","last_became_peered":"2023-10-06T10:38:17.358652+0000","last_unstale":"2023-10-06T10:38:21.376233+0000","last_undegraded":"2023-10-06T10:38:21.376233+0000","last_fullsized":"2023-10-06T10:38:21.376233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:45:42.362885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.400005+0000","last_change":"2023-10-06T10:38:21.394129+0000","last_active":"2023-10-06T10:38:21.400005+0000","last_peered":"2023-10-06T10:38:21.400005+0000","last_clean":"2023-10-06T10:38:21.400005+0000","last_became_active":"2023-10-06T10:38:21.393517+0000","last_became_peered":"2023-10-06T10:38:21.393517+0000","last_unstale":"2023-10-06T10:38:21.400005+0000","last_undegraded":"2023-10-06T10:38:21.400005+0000","last_fullsized":"2023-10-06T10:38:21.400005+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:55:52.407995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385053+0000","last_change":"2023-10-06T10:38:16.364742+0000","last_active":"2023-10-06T10:38:21.385053+0000","last_peered":"2023-10-06T10:38:21.385053+0000","last_clean":"2023-10-06T10:38:21.385053+0000","last_became_active":"2023-10-06T10:38:16.364410+0000","last_became_peered":"2023-10-06T10:38:16.364410+0000","last_unstale":"2023-10-06T10:38:21.385053+0000","last_undegraded":"2023-10-06T10:38:21.385053+0000","last_fullsized":"2023-10-06T10:38:21.385053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:06.358487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.373267+0000","last_change":"2023-10-06T10:38:17.374643+0000","last_active":"2023-10-06T10:38:21.373267+0000","last_peered":"2023-10-06T10:38:21.373267+0000","last_clean":"2023-10-06T10:38:21.373267+0000","last_became_active":"2023-10-06T10:38:17.374436+0000","last_became_peered":"2023-10-06T10:38:17.374436+0000","last_unstale":"2023-10-06T10:38:21.373267+0000","last_undegraded":"2023-10-06T10:38:21.373267+0000","last_fullsized":"2023-10-06T10:38:21.373267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:01.022639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402862+0000","last_change":"2023-10-06T10:38:21.401121+0000","last_active":"2023-10-06T10:38:21.402862+0000","last_peered":"2023-10-06T10:38:21.402862+0000","last_clean":"2023-10-06T10:38:21.402862+0000","last_became_active":"2023-10-06T10:38:21.400766+0000","last_became_peered":"2023-10-06T10:38:21.400766+0000","last_unstale":"2023-10-06T10:38:21.402862+0000","last_undegraded":"2023-10-06T10:38:21.402862+0000","last_fullsized":"2023-10-06T10:38:21.402862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:39.967976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.392673+0000","last_change":"2023-10-06T10:38:16.363994+0000","last_active":"2023-10-06T10:38:21.392673+0000","last_peered":"2023-10-06T10:38:21.392673+0000","last_clean":"2023-10-06T10:38:21.392673+0000","last_became_active":"2023-10-06T10:38:16.363712+0000","last_became_peered":"2023-10-06T10:38:16.363712+0000","last_unstale":"2023-10-06T10:38:21.392673+0000","last_undegraded":"2023-10-06T10:38:21.392673+0000","last_fullsized":"2023-10-06T10:38:21.392673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:43.249942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375546+0000","last_change":"2023-10-06T10:38:17.374214+0000","last_active":"2023-10-06T10:38:21.375546+0000","last_peered":"2023-10-06T10:38:21.375546+0000","last_clean":"2023-10-06T10:38:21.375546+0000","last_became_active":"2023-10-06T10:38:17.373998+0000","last_became_peered":"2023-10-06T10:38:17.373998+0000","last_unstale":"2023-10-06T10:38:21.375546+0000","last_undegraded":"2023-10-06T10:38:21.375546+0000","last_fullsized":"2023-10-06T10:38:21.375546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:06.214524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.396278+0000","last_change":"2023-10-06T10:38:21.393210+0000","last_active":"2023-10-06T10:38:21.396278+0000","last_peered":"2023-10-06T10:38:21.396278+0000","last_clean":"2023-10-06T10:38:21.396278+0000","last_became_active":"2023-10-06T10:38:21.392827+0000","last_became_peered":"2023-10-06T10:38:21.392827+0000","last_unstale":"2023-10-06T10:38:21.396278+0000","last_undegraded":"2023-10-06T10:38:21.396278+0000","last_fullsized":"2023-10-06T10:38:21.396278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:05.756884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.392711+0000","last_change":"2023-10-06T10:38:16.363468+0000","last_active":"2023-10-06T10:38:21.392711+0000","last_peered":"2023-10-06T10:38:21.392711+0000","last_clean":"2023-10-06T10:38:21.392711+0000","last_became_active":"2023-10-06T10:38:16.363186+0000","last_became_peered":"2023-10-06T10:38:16.363186+0000","last_unstale":"2023-10-06T10:38:21.392711+0000","last_undegraded":"2023-10-06T10:38:21.392711+0000","last_fullsized":"2023-10-06T10:38:21.392711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:32.527614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377663+0000","last_change":"2023-10-06T10:38:17.365862+0000","last_active":"2023-10-06T10:38:21.377663+0000","last_peered":"2023-10-06T10:38:21.377663+0000","last_clean":"2023-10-06T10:38:21.377663+0000","last_became_active":"2023-10-06T10:38:17.365657+0000","last_became_peered":"2023-10-06T10:38:17.365657+0000","last_unstale":"2023-10-06T10:38:21.377663+0000","last_undegraded":"2023-10-06T10:38:21.377663+0000","last_fullsized":"2023-10-06T10:38:21.377663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:48.983240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.395385+0000","last_change":"2023-10-06T10:38:21.392166+0000","last_active":"2023-10-06T10:38:21.395385+0000","last_peered":"2023-10-06T10:38:21.395385+0000","last_clean":"2023-10-06T10:38:21.395385+0000","last_became_active":"2023-10-06T10:38:21.391756+0000","last_became_peered":"2023-10-06T10:38:21.391756+0000","last_unstale":"2023-10-06T10:38:21.395385+0000","last_undegraded":"2023-10-06T10:38:21.395385+0000","last_fullsized":"2023-10-06T10:38:21.395385+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:05.052271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.386206+0000","last_change":"2023-10-06T10:38:16.362369+0000","last_active":"2023-10-06T10:38:21.386206+0000","last_peered":"2023-10-06T10:38:21.386206+0000","last_clean":"2023-10-06T10:38:21.386206+0000","last_became_active":"2023-10-06T10:38:16.362076+0000","last_became_peered":"2023-10-06T10:38:16.362076+0000","last_unstale":"2023-10-06T10:38:21.386206+0000","last_undegraded":"2023-10-06T10:38:21.386206+0000","last_fullsized":"2023-10-06T10:38:21.386206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:27.473452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384735+0000","last_change":"2023-10-06T10:38:17.368641+0000","last_active":"2023-10-06T10:38:21.384735+0000","last_peered":"2023-10-06T10:38:21.384735+0000","last_clean":"2023-10-06T10:38:21.384735+0000","last_became_active":"2023-10-06T10:38:17.368283+0000","last_became_peered":"2023-10-06T10:38:17.368283+0000","last_unstale":"2023-10-06T10:38:21.384735+0000","last_undegraded":"2023-10-06T10:38:21.384735+0000","last_fullsized":"2023-10-06T10:38:21.384735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:17.351581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.395567+0000","last_change":"2023-10-06T10:38:21.392954+0000","last_active":"2023-10-06T10:38:21.395567+0000","last_peered":"2023-10-06T10:38:21.395567+0000","last_clean":"2023-10-06T10:38:21.395567+0000","last_became_active":"2023-10-06T10:38:21.392585+0000","last_became_peered":"2023-10-06T10:38:21.392585+0000","last_unstale":"2023-10-06T10:38:21.395567+0000","last_undegraded":"2023-10-06T10:38:21.395567+0000","last_fullsized":"2023-10-06T10:38:21.395567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:54.407728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375768+0000","last_change":"2023-10-06T10:38:16.357199+0000","last_active":"2023-10-06T10:38:21.375768+0000","last_peered":"2023-10-06T10:38:21.375768+0000","last_clean":"2023-10-06T10:38:21.375768+0000","last_became_active":"2023-10-06T10:38:16.356936+0000","last_became_peered":"2023-10-06T10:38:16.356936+0000","last_unstale":"2023-10-06T10:38:21.375768+0000","last_undegraded":"2023-10-06T10:38:21.375768+0000","last_fullsized":"2023-10-06T10:38:21.375768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:02.115831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377449+0000","last_change":"2023-10-06T10:38:17.359930+0000","last_active":"2023-10-06T10:38:21.377449+0000","last_peered":"2023-10-06T10:38:21.377449+0000","last_clean":"2023-10-06T10:38:21.377449+0000","last_became_active":"2023-10-06T10:38:17.359677+0000","last_became_peered":"2023-10-06T10:38:17.359677+0000","last_unstale":"2023-10-06T10:38:21.377449+0000","last_undegraded":"2023-10-06T10:38:21.377449+0000","last_fullsized":"2023-10-06T10:38:21.377449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:13.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.399357+0000","last_change":"2023-10-06T10:38:21.396020+0000","last_active":"2023-10-06T10:38:21.399357+0000","last_peered":"2023-10-06T10:38:21.399357+0000","last_clean":"2023-10-06T10:38:21.399357+0000","last_became_active":"2023-10-06T10:38:21.395659+0000","last_became_peered":"2023-10-06T10:38:21.395659+0000","last_unstale":"2023-10-06T10:38:21.399357+0000","last_undegraded":"2023-10-06T10:38:21.399357+0000","last_fullsized":"2023-10-06T10:38:21.399357+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:45:46.392517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372543+0000","last_change":"2023-10-06T10:38:16.364249+0000","last_active":"2023-10-06T10:38:21.372543+0000","last_peered":"2023-10-06T10:38:21.372543+0000","last_clean":"2023-10-06T10:38:21.372543+0000","last_became_active":"2023-10-06T10:38:16.363959+0000","last_became_peered":"2023-10-06T10:38:16.363959+0000","last_unstale":"2023-10-06T10:38:21.372543+0000","last_undegraded":"2023-10-06T10:38:21.372543+0000","last_fullsized":"2023-10-06T10:38:21.372543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:27.473452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377610+0000","last_change":"2023-10-06T10:38:17.362439+0000","last_active":"2023-10-06T10:38:21.377610+0000","last_peered":"2023-10-06T10:38:21.377610+0000","last_clean":"2023-10-06T10:38:21.377610+0000","last_became_active":"2023-10-06T10:38:17.361921+0000","last_became_peered":"2023-10-06T10:38:17.361921+0000","last_unstale":"2023-10-06T10:38:21.377610+0000","last_undegraded":"2023-10-06T10:38:21.377610+0000","last_fullsized":"2023-10-06T10:38:21.377610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:30:01.208220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402322+0000","last_change":"2023-10-06T10:38:21.399211+0000","last_active":"2023-10-06T10:38:21.402322+0000","last_peered":"2023-10-06T10:38:21.402322+0000","last_clean":"2023-10-06T10:38:21.402322+0000","last_became_active":"2023-10-06T10:38:21.396487+0000","last_became_peered":"2023-10-06T10:38:21.396487+0000","last_unstale":"2023-10-06T10:38:21.402322+0000","last_undegraded":"2023-10-06T10:38:21.402322+0000","last_fullsized":"2023-10-06T10:38:21.402322+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:26.378955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384822+0000","last_change":"2023-10-06T10:38:16.362941+0000","last_active":"2023-10-06T10:38:21.384822+0000","last_peered":"2023-10-06T10:38:21.384822+0000","last_clean":"2023-10-06T10:38:21.384822+0000","last_became_active":"2023-10-06T10:38:16.362651+0000","last_became_peered":"2023-10-06T10:38:16.362651+0000","last_unstale":"2023-10-06T10:38:21.384822+0000","last_undegraded":"2023-10-06T10:38:21.384822+0000","last_fullsized":"2023-10-06T10:38:21.384822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:41.837644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377389+0000","last_change":"2023-10-06T10:38:17.367573+0000","last_active":"2023-10-06T10:38:21.377389+0000","last_peered":"2023-10-06T10:38:21.377389+0000","last_clean":"2023-10-06T10:38:21.377389+0000","last_became_active":"2023-10-06T10:38:17.367363+0000","last_became_peered":"2023-10-06T10:38:17.367363+0000","last_unstale":"2023-10-06T10:38:21.377389+0000","last_undegraded":"2023-10-06T10:38:21.377389+0000","last_fullsized":"2023-10-06T10:38:21.377389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:03.650819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.401267+0000","last_change":"2023-10-06T10:38:21.394943+0000","last_active":"2023-10-06T10:38:21.401267+0000","last_peered":"2023-10-06T10:38:21.401267+0000","last_clean":"2023-10-06T10:38:21.401267+0000","last_became_active":"2023-10-06T10:38:21.394604+0000","last_became_peered":"2023-10-06T10:38:21.394604+0000","last_unstale":"2023-10-06T10:38:21.401267+0000","last_undegraded":"2023-10-06T10:38:21.401267+0000","last_fullsized":"2023-10-06T10:38:21.401267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:17:59.362798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377491+0000","last_change":"2023-10-06T10:38:16.358983+0000","last_active":"2023-10-06T10:38:21.377491+0000","last_peered":"2023-10-06T10:38:21.377491+0000","last_clean":"2023-10-06T10:38:21.377491+0000","last_became_active":"2023-10-06T10:38:16.358338+0000","last_became_peered":"2023-10-06T10:38:16.358338+0000","last_unstale":"2023-10-06T10:38:21.377491+0000","last_undegraded":"2023-10-06T10:38:21.377491+0000","last_fullsized":"2023-10-06T10:38:21.377491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:02.115831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378341+0000","last_change":"2023-10-06T10:38:17.375549+0000","last_active":"2023-10-06T10:38:21.378341+0000","last_peered":"2023-10-06T10:38:21.378341+0000","last_clean":"2023-10-06T10:38:21.378341+0000","last_became_active":"2023-10-06T10:38:17.374707+0000","last_became_peered":"2023-10-06T10:38:17.374707+0000","last_unstale":"2023-10-06T10:38:21.378341+0000","last_undegraded":"2023-10-06T10:38:21.378341+0000","last_fullsized":"2023-10-06T10:38:21.378341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:50.378097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391124+0000","last_change":"2023-10-06T10:38:21.388787+0000","last_active":"2023-10-06T10:38:21.391124+0000","last_peered":"2023-10-06T10:38:21.391124+0000","last_clean":"2023-10-06T10:38:21.391124+0000","last_became_active":"2023-10-06T10:38:21.388432+0000","last_became_peered":"2023-10-06T10:38:21.388432+0000","last_unstale":"2023-10-06T10:38:21.391124+0000","last_undegraded":"2023-10-06T10:38:21.391124+0000","last_fullsized":"2023-10-06T10:38:21.391124+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:03.131588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372464+0000","last_change":"2023-10-06T10:38:16.366048+0000","last_active":"2023-10-06T10:38:21.372464+0000","last_peered":"2023-10-06T10:38:21.372464+0000","last_clean":"2023-10-06T10:38:21.372464+0000","last_became_active":"2023-10-06T10:38:16.365814+0000","last_became_peered":"2023-10-06T10:38:16.365814+0000","last_unstale":"2023-10-06T10:38:21.372464+0000","last_undegraded":"2023-10-06T10:38:21.372464+0000","last_fullsized":"2023-10-06T10:38:21.372464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:36.469014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.379002+0000","last_change":"2023-10-06T10:38:17.358727+0000","last_active":"2023-10-06T10:38:21.379002+0000","last_peered":"2023-10-06T10:38:21.379002+0000","last_clean":"2023-10-06T10:38:21.379002+0000","last_became_active":"2023-10-06T10:38:17.358442+0000","last_became_peered":"2023-10-06T10:38:17.358442+0000","last_unstale":"2023-10-06T10:38:21.379002+0000","last_undegraded":"2023-10-06T10:38:21.379002+0000","last_fullsized":"2023-10-06T10:38:21.379002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:17.351581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.401082+0000","last_change":"2023-10-06T10:38:21.397353+0000","last_active":"2023-10-06T10:38:21.401082+0000","last_peered":"2023-10-06T10:38:21.401082+0000","last_clean":"2023-10-06T10:38:21.401082+0000","last_became_active":"2023-10-06T10:38:21.396788+0000","last_became_peered":"2023-10-06T10:38:21.396788+0000","last_unstale":"2023-10-06T10:38:21.401082+0000","last_undegraded":"2023-10-06T10:38:21.401082+0000","last_fullsized":"2023-10-06T10:38:21.401082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:50:06.948087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.373072+0000","last_change":"2023-10-06T10:38:16.362604+0000","last_active":"2023-10-06T10:38:21.373072+0000","last_peered":"2023-10-06T10:38:21.373072+0000","last_clean":"2023-10-06T10:38:21.373072+0000","last_became_active":"2023-10-06T10:38:16.362444+0000","last_became_peered":"2023-10-06T10:38:16.362444+0000","last_unstale":"2023-10-06T10:38:21.373072+0000","last_undegraded":"2023-10-06T10:38:21.373072+0000","last_fullsized":"2023-10-06T10:38:21.373072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:57.215481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377559+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:38:21.377559+0000","last_peered":"2023-10-06T10:38:21.377559+0000","last_clean":"2023-10-06T10:38:21.377559+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:38:21.377559+0000","last_undegraded":"2023-10-06T10:38:21.377559+0000","last_fullsized":"2023-10-06T10:38:21.377559+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372292+0000","last_change":"2023-10-06T10:38:17.374346+0000","last_active":"2023-10-06T10:38:21.372292+0000","last_peered":"2023-10-06T10:38:21.372292+0000","last_clean":"2023-10-06T10:38:21.372292+0000","last_became_active":"2023-10-06T10:38:17.374068+0000","last_became_peered":"2023-10-06T10:38:17.374068+0000","last_unstale":"2023-10-06T10:38:21.372292+0000","last_undegraded":"2023-10-06T10:38:21.372292+0000","last_fullsized":"2023-10-06T10:38:21.372292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:29.897092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.393470+0000","last_change":"2023-10-06T10:38:21.387372+0000","last_active":"2023-10-06T10:38:21.393470+0000","last_peered":"2023-10-06T10:38:21.393470+0000","last_clean":"2023-10-06T10:38:21.393470+0000","last_became_active":"2023-10-06T10:38:21.386907+0000","last_became_peered":"2023-10-06T10:38:21.386907+0000","last_unstale":"2023-10-06T10:38:21.393470+0000","last_undegraded":"2023-10-06T10:38:21.393470+0000","last_fullsized":"2023-10-06T10:38:21.393470+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:36.144881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372792+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:38:21.372792+0000","last_peered":"2023-10-06T10:38:21.372792+0000","last_clean":"2023-10-06T10:38:21.372792+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:38:21.372792+0000","last_undegraded":"2023-10-06T10:38:21.372792+0000","last_fullsized":"2023-10-06T10:38:21.372792+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.433239+0000","last_change":"2023-10-06T10:38:16.366116+0000","last_active":"2023-10-06T10:38:21.433239+0000","last_peered":"2023-10-06T10:38:21.433239+0000","last_clean":"2023-10-06T10:38:21.433239+0000","last_became_active":"2023-10-06T10:38:16.365875+0000","last_became_peered":"2023-10-06T10:38:16.365875+0000","last_unstale":"2023-10-06T10:38:21.433239+0000","last_undegraded":"2023-10-06T10:38:21.433239+0000","last_fullsized":"2023-10-06T10:38:21.433239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:35.127396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377577+0000","last_change":"2023-10-06T10:38:17.358654+0000","last_active":"2023-10-06T10:38:21.377577+0000","last_peered":"2023-10-06T10:38:21.377577+0000","last_clean":"2023-10-06T10:38:21.377577+0000","last_became_active":"2023-10-06T10:38:17.358361+0000","last_became_peered":"2023-10-06T10:38:17.358361+0000","last_unstale":"2023-10-06T10:38:21.377577+0000","last_undegraded":"2023-10-06T10:38:21.377577+0000","last_fullsized":"2023-10-06T10:38:21.377577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:06.214524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.397881+0000","last_change":"2023-10-06T10:38:21.392569+0000","last_active":"2023-10-06T10:38:21.397881+0000","last_peered":"2023-10-06T10:38:21.397881+0000","last_clean":"2023-10-06T10:38:21.397881+0000","last_became_active":"2023-10-06T10:38:21.392210+0000","last_became_peered":"2023-10-06T10:38:21.392210+0000","last_unstale":"2023-10-06T10:38:21.397881+0000","last_undegraded":"2023-10-06T10:38:21.397881+0000","last_fullsized":"2023-10-06T10:38:21.397881+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:14:10.094621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377969+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:38:21.377969+0000","last_peered":"2023-10-06T10:38:21.377969+0000","last_clean":"2023-10-06T10:38:21.377969+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:38:21.377969+0000","last_undegraded":"2023-10-06T10:38:21.377969+0000","last_fullsized":"2023-10-06T10:38:21.377969+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.390711+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:38:21.390711+0000","last_peered":"2023-10-06T10:38:21.390711+0000","last_clean":"2023-10-06T10:38:21.390711+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:38:21.390711+0000","last_undegraded":"2023-10-06T10:38:21.390711+0000","last_fullsized":"2023-10-06T10:38:21.390711+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375378+0000","last_change":"2023-10-06T10:38:16.362718+0000","last_active":"2023-10-06T10:38:21.375378+0000","last_peered":"2023-10-06T10:38:21.375378+0000","last_clean":"2023-10-06T10:38:21.375378+0000","last_became_active":"2023-10-06T10:38:16.361895+0000","last_became_peered":"2023-10-06T10:38:16.361895+0000","last_unstale":"2023-10-06T10:38:21.375378+0000","last_undegraded":"2023-10-06T10:38:21.375378+0000","last_fullsized":"2023-10-06T10:38:21.375378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:45.092301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.376156+0000","last_change":"2023-10-06T10:38:17.359496+0000","last_active":"2023-10-06T10:38:21.376156+0000","last_peered":"2023-10-06T10:38:21.376156+0000","last_clean":"2023-10-06T10:38:21.376156+0000","last_became_active":"2023-10-06T10:38:17.359011+0000","last_became_peered":"2023-10-06T10:38:17.359011+0000","last_unstale":"2023-10-06T10:38:21.376156+0000","last_undegraded":"2023-10-06T10:38:21.376156+0000","last_fullsized":"2023-10-06T10:38:21.376156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:06.214524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402221+0000","last_change":"2023-10-06T10:38:21.396996+0000","last_active":"2023-10-06T10:38:21.402221+0000","last_peered":"2023-10-06T10:38:21.402221+0000","last_clean":"2023-10-06T10:38:21.402221+0000","last_became_active":"2023-10-06T10:38:21.396694+0000","last_became_peered":"2023-10-06T10:38:21.396694+0000","last_unstale":"2023-10-06T10:38:21.402221+0000","last_undegraded":"2023-10-06T10:38:21.402221+0000","last_fullsized":"2023-10-06T10:38:21.402221+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:14.207234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.386749+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:38:21.386749+0000","last_peered":"2023-10-06T10:38:21.386749+0000","last_clean":"2023-10-06T10:38:21.386749+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:38:21.386749+0000","last_undegraded":"2023-10-06T10:38:21.386749+0000","last_fullsized":"2023-10-06T10:38:21.386749+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.376477+0000","last_change":"2023-10-06T10:38:16.357154+0000","last_active":"2023-10-06T10:38:21.376477+0000","last_peered":"2023-10-06T10:38:21.376477+0000","last_clean":"2023-10-06T10:38:21.376477+0000","last_became_active":"2023-10-06T10:38:16.356942+0000","last_became_peered":"2023-10-06T10:38:16.356942+0000","last_unstale":"2023-10-06T10:38:21.376477+0000","last_undegraded":"2023-10-06T10:38:21.376477+0000","last_fullsized":"2023-10-06T10:38:21.376477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:16.528477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378971+0000","last_change":"2023-10-06T10:38:17.358788+0000","last_active":"2023-10-06T10:38:21.378971+0000","last_peered":"2023-10-06T10:38:21.378971+0000","last_clean":"2023-10-06T10:38:21.378971+0000","last_became_active":"2023-10-06T10:38:17.358514+0000","last_became_peered":"2023-10-06T10:38:17.358514+0000","last_unstale":"2023-10-06T10:38:21.378971+0000","last_undegraded":"2023-10-06T10:38:21.378971+0000","last_fullsized":"2023-10-06T10:38:21.378971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:49.973815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402838+0000","last_change":"2023-10-06T10:38:21.401259+0000","last_active":"2023-10-06T10:38:21.402838+0000","last_peered":"2023-10-06T10:38:21.402838+0000","last_clean":"2023-10-06T10:38:21.402838+0000","last_became_active":"2023-10-06T10:38:21.400865+0000","last_became_peered":"2023-10-06T10:38:21.400865+0000","last_unstale":"2023-10-06T10:38:21.402838+0000","last_undegraded":"2023-10-06T10:38:21.402838+0000","last_fullsized":"2023-10-06T10:38:21.402838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:30.495506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384378+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:38:21.384378+0000","last_peered":"2023-10-06T10:38:21.384378+0000","last_clean":"2023-10-06T10:38:21.384378+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:38:21.384378+0000","last_undegraded":"2023-10-06T10:38:21.384378+0000","last_fullsized":"2023-10-06T10:38:21.384378+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372330+0000","last_change":"2023-10-06T10:38:16.358939+0000","last_active":"2023-10-06T10:38:21.372330+0000","last_peered":"2023-10-06T10:38:21.372330+0000","last_clean":"2023-10-06T10:38:21.372330+0000","last_became_active":"2023-10-06T10:38:16.358685+0000","last_became_peered":"2023-10-06T10:38:16.358685+0000","last_unstale":"2023-10-06T10:38:21.372330+0000","last_undegraded":"2023-10-06T10:38:21.372330+0000","last_fullsized":"2023-10-06T10:38:21.372330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:41.837644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385109+0000","last_change":"2023-10-06T10:38:17.366729+0000","last_active":"2023-10-06T10:38:21.385109+0000","last_peered":"2023-10-06T10:38:21.385109+0000","last_clean":"2023-10-06T10:38:21.385109+0000","last_became_active":"2023-10-06T10:38:17.366408+0000","last_became_peered":"2023-10-06T10:38:17.366408+0000","last_unstale":"2023-10-06T10:38:21.385109+0000","last_undegraded":"2023-10-06T10:38:21.385109+0000","last_fullsized":"2023-10-06T10:38:21.385109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:35.729655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.395664+0000","last_change":"2023-10-06T10:38:21.392874+0000","last_active":"2023-10-06T10:38:21.395664+0000","last_peered":"2023-10-06T10:38:21.395664+0000","last_clean":"2023-10-06T10:38:21.395664+0000","last_became_active":"2023-10-06T10:38:21.392487+0000","last_became_peered":"2023-10-06T10:38:21.392487+0000","last_unstale":"2023-10-06T10:38:21.395664+0000","last_undegraded":"2023-10-06T10:38:21.395664+0000","last_fullsized":"2023-10-06T10:38:21.395664+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:39.967976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375754+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:38:21.375754+0000","last_peered":"2023-10-06T10:38:21.375754+0000","last_clean":"2023-10-06T10:38:21.375754+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:38:21.375754+0000","last_undegraded":"2023-10-06T10:38:21.375754+0000","last_fullsized":"2023-10-06T10:38:21.375754+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377566+0000","last_change":"2023-10-06T10:38:16.362244+0000","last_active":"2023-10-06T10:38:21.377566+0000","last_peered":"2023-10-06T10:38:21.377566+0000","last_clean":"2023-10-06T10:38:21.377566+0000","last_became_active":"2023-10-06T10:38:16.361869+0000","last_became_peered":"2023-10-06T10:38:16.361869+0000","last_unstale":"2023-10-06T10:38:21.377566+0000","last_undegraded":"2023-10-06T10:38:21.377566+0000","last_fullsized":"2023-10-06T10:38:21.377566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:57.215481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.379113+0000","last_change":"2023-10-06T10:38:17.367649+0000","last_active":"2023-10-06T10:38:21.379113+0000","last_peered":"2023-10-06T10:38:21.379113+0000","last_clean":"2023-10-06T10:38:21.379113+0000","last_became_active":"2023-10-06T10:38:17.366773+0000","last_became_peered":"2023-10-06T10:38:17.366773+0000","last_unstale":"2023-10-06T10:38:21.379113+0000","last_undegraded":"2023-10-06T10:38:21.379113+0000","last_fullsized":"2023-10-06T10:38:21.379113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:40.679246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.401815+0000","last_change":"2023-10-06T10:38:21.393988+0000","last_active":"2023-10-06T10:38:21.401815+0000","last_peered":"2023-10-06T10:38:21.401815+0000","last_clean":"2023-10-06T10:38:21.401815+0000","last_became_active":"2023-10-06T10:38:21.393668+0000","last_became_peered":"2023-10-06T10:38:21.393668+0000","last_unstale":"2023-10-06T10:38:21.401815+0000","last_undegraded":"2023-10-06T10:38:21.401815+0000","last_fullsized":"2023-10-06T10:38:21.401815+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:33.926723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385218+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:38:21.385218+0000","last_peered":"2023-10-06T10:38:21.385218+0000","last_clean":"2023-10-06T10:38:21.385218+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:38:21.385218+0000","last_undegraded":"2023-10-06T10:38:21.385218+0000","last_fullsized":"2023-10-06T10:38:21.385218+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.379391+0000","last_change":"2023-10-06T10:38:16.360214+0000","last_active":"2023-10-06T10:38:21.379391+0000","last_peered":"2023-10-06T10:38:21.379391+0000","last_clean":"2023-10-06T10:38:21.379391+0000","last_became_active":"2023-10-06T10:38:16.359912+0000","last_became_peered":"2023-10-06T10:38:16.359912+0000","last_unstale":"2023-10-06T10:38:21.379391+0000","last_undegraded":"2023-10-06T10:38:21.379391+0000","last_fullsized":"2023-10-06T10:38:21.379391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:41.837644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377275+0000","last_change":"2023-10-06T10:38:17.354930+0000","last_active":"2023-10-06T10:38:21.377275+0000","last_peered":"2023-10-06T10:38:21.377275+0000","last_clean":"2023-10-06T10:38:21.377275+0000","last_became_active":"2023-10-06T10:38:17.354637+0000","last_became_peered":"2023-10-06T10:38:17.354637+0000","last_unstale":"2023-10-06T10:38:21.377275+0000","last_undegraded":"2023-10-06T10:38:21.377275+0000","last_fullsized":"2023-10-06T10:38:21.377275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:03.953719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391646+0000","last_change":"2023-10-06T10:38:21.387944+0000","last_active":"2023-10-06T10:38:21.391646+0000","last_peered":"2023-10-06T10:38:21.391646+0000","last_clean":"2023-10-06T10:38:21.391646+0000","last_became_active":"2023-10-06T10:38:21.387580+0000","last_became_peered":"2023-10-06T10:38:21.387580+0000","last_unstale":"2023-10-06T10:38:21.391646+0000","last_undegraded":"2023-10-06T10:38:21.391646+0000","last_fullsized":"2023-10-06T10:38:21.391646+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:54:20.934943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385079+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:38:21.385079+0000","last_peered":"2023-10-06T10:38:21.385079+0000","last_clean":"2023-10-06T10:38:21.385079+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:38:21.385079+0000","last_undegraded":"2023-10-06T10:38:21.385079+0000","last_fullsized":"2023-10-06T10:38:21.385079+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377995+0000","last_change":"2023-10-06T10:38:16.358519+0000","last_active":"2023-10-06T10:38:21.377995+0000","last_peered":"2023-10-06T10:38:21.377995+0000","last_clean":"2023-10-06T10:38:21.377995+0000","last_became_active":"2023-10-06T10:38:16.357730+0000","last_became_peered":"2023-10-06T10:38:16.357730+0000","last_unstale":"2023-10-06T10:38:21.377995+0000","last_undegraded":"2023-10-06T10:38:21.377995+0000","last_fullsized":"2023-10-06T10:38:21.377995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:36.469014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378421+0000","last_change":"2023-10-06T10:38:17.358314+0000","last_active":"2023-10-06T10:38:21.378421+0000","last_peered":"2023-10-06T10:38:21.378421+0000","last_clean":"2023-10-06T10:38:21.378421+0000","last_became_active":"2023-10-06T10:38:17.358060+0000","last_became_peered":"2023-10-06T10:38:17.358060+0000","last_unstale":"2023-10-06T10:38:21.378421+0000","last_undegraded":"2023-10-06T10:38:21.378421+0000","last_fullsized":"2023-10-06T10:38:21.378421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:34:27.066675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.400514+0000","last_change":"2023-10-06T10:38:21.391294+0000","last_active":"2023-10-06T10:38:21.400514+0000","last_peered":"2023-10-06T10:38:21.400514+0000","last_clean":"2023-10-06T10:38:21.400514+0000","last_became_active":"2023-10-06T10:38:21.390927+0000","last_became_peered":"2023-10-06T10:38:21.390927+0000","last_unstale":"2023-10-06T10:38:21.400514+0000","last_undegraded":"2023-10-06T10:38:21.400514+0000","last_fullsized":"2023-10-06T10:38:21.400514+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:53.012871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377366+0000","last_change":"2023-10-06T10:38:16.360511+0000","last_active":"2023-10-06T10:38:21.377366+0000","last_peered":"2023-10-06T10:38:21.377366+0000","last_clean":"2023-10-06T10:38:21.377366+0000","last_became_active":"2023-10-06T10:38:16.360260+0000","last_became_peered":"2023-10-06T10:38:16.360260+0000","last_unstale":"2023-10-06T10:38:21.377366+0000","last_undegraded":"2023-10-06T10:38:21.377366+0000","last_fullsized":"2023-10-06T10:38:21.377366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:57.215481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.383881+0000","last_change":"2023-10-06T10:38:17.365370+0000","last_active":"2023-10-06T10:38:21.383881+0000","last_peered":"2023-10-06T10:38:21.383881+0000","last_clean":"2023-10-06T10:38:21.383881+0000","last_became_active":"2023-10-06T10:38:17.365110+0000","last_became_peered":"2023-10-06T10:38:17.365110+0000","last_unstale":"2023-10-06T10:38:21.383881+0000","last_undegraded":"2023-10-06T10:38:21.383881+0000","last_fullsized":"2023-10-06T10:38:21.383881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:37.475956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402373+0000","last_change":"2023-10-06T10:38:21.399284+0000","last_active":"2023-10-06T10:38:21.402373+0000","last_peered":"2023-10-06T10:38:21.402373+0000","last_clean":"2023-10-06T10:38:21.402373+0000","last_became_active":"2023-10-06T10:38:21.398931+0000","last_became_peered":"2023-10-06T10:38:21.398931+0000","last_unstale":"2023-10-06T10:38:21.402373+0000","last_undegraded":"2023-10-06T10:38:21.402373+0000","last_fullsized":"2023-10-06T10:38:21.402373+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:17:39.406756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375583+0000","last_change":"2023-10-06T10:38:16.353975+0000","last_active":"2023-10-06T10:38:21.375583+0000","last_peered":"2023-10-06T10:38:21.375583+0000","last_clean":"2023-10-06T10:38:21.375583+0000","last_became_active":"2023-10-06T10:38:16.353629+0000","last_became_peered":"2023-10-06T10:38:16.353629+0000","last_unstale":"2023-10-06T10:38:21.375583+0000","last_undegraded":"2023-10-06T10:38:21.375583+0000","last_fullsized":"2023-10-06T10:38:21.375583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:20.696308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377446+0000","last_change":"2023-10-06T10:38:17.363838+0000","last_active":"2023-10-06T10:38:21.377446+0000","last_peered":"2023-10-06T10:38:21.377446+0000","last_clean":"2023-10-06T10:38:21.377446+0000","last_became_active":"2023-10-06T10:38:17.363580+0000","last_became_peered":"2023-10-06T10:38:17.363580+0000","last_unstale":"2023-10-06T10:38:21.377446+0000","last_undegraded":"2023-10-06T10:38:21.377446+0000","last_fullsized":"2023-10-06T10:38:21.377446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:43.614842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.399586+0000","last_change":"2023-10-06T10:38:21.396452+0000","last_active":"2023-10-06T10:38:21.399586+0000","last_peered":"2023-10-06T10:38:21.399586+0000","last_clean":"2023-10-06T10:38:21.399586+0000","last_became_active":"2023-10-06T10:38:21.396052+0000","last_became_peered":"2023-10-06T10:38:21.396052+0000","last_unstale":"2023-10-06T10:38:21.399586+0000","last_undegraded":"2023-10-06T10:38:21.399586+0000","last_fullsized":"2023-10-06T10:38:21.399586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:17:39.406756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377909+0000","last_change":"2023-10-06T10:38:16.354639+0000","last_active":"2023-10-06T10:38:21.377909+0000","last_peered":"2023-10-06T10:38:21.377909+0000","last_clean":"2023-10-06T10:38:21.377909+0000","last_became_active":"2023-10-06T10:38:16.354367+0000","last_became_peered":"2023-10-06T10:38:16.354367+0000","last_unstale":"2023-10-06T10:38:21.377909+0000","last_undegraded":"2023-10-06T10:38:21.377909+0000","last_fullsized":"2023-10-06T10:38:21.377909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:57.215481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377353+0000","last_change":"2023-10-06T10:38:17.358682+0000","last_active":"2023-10-06T10:38:21.377353+0000","last_peered":"2023-10-06T10:38:21.377353+0000","last_clean":"2023-10-06T10:38:21.377353+0000","last_became_active":"2023-10-06T10:38:17.358409+0000","last_became_peered":"2023-10-06T10:38:17.358409+0000","last_unstale":"2023-10-06T10:38:21.377353+0000","last_undegraded":"2023-10-06T10:38:21.377353+0000","last_fullsized":"2023-10-06T10:38:21.377353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:52:14.320333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391815+0000","last_change":"2023-10-06T10:38:21.390566+0000","last_active":"2023-10-06T10:38:21.391815+0000","last_peered":"2023-10-06T10:38:21.391815+0000","last_clean":"2023-10-06T10:38:21.391815+0000","last_became_active":"2023-10-06T10:38:21.390172+0000","last_became_peered":"2023-10-06T10:38:21.390172+0000","last_unstale":"2023-10-06T10:38:21.391815+0000","last_undegraded":"2023-10-06T10:38:21.391815+0000","last_fullsized":"2023-10-06T10:38:21.391815+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:37.174852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372471+0000","last_change":"2023-10-06T10:38:16.353396+0000","last_active":"2023-10-06T10:38:21.372471+0000","last_peered":"2023-10-06T10:38:21.372471+0000","last_clean":"2023-10-06T10:38:21.372471+0000","last_became_active":"2023-10-06T10:38:16.353136+0000","last_became_peered":"2023-10-06T10:38:16.353136+0000","last_unstale":"2023-10-06T10:38:21.372471+0000","last_undegraded":"2023-10-06T10:38:21.372471+0000","last_fullsized":"2023-10-06T10:38:21.372471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:02.115831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372424+0000","last_change":"2023-10-06T10:38:17.364589+0000","last_active":"2023-10-06T10:38:21.372424+0000","last_peered":"2023-10-06T10:38:21.372424+0000","last_clean":"2023-10-06T10:38:21.372424+0000","last_became_active":"2023-10-06T10:38:17.364364+0000","last_became_peered":"2023-10-06T10:38:17.364364+0000","last_unstale":"2023-10-06T10:38:21.372424+0000","last_undegraded":"2023-10-06T10:38:21.372424+0000","last_fullsized":"2023-10-06T10:38:21.372424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:13.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.399883+0000","last_change":"2023-10-06T10:38:21.395180+0000","last_active":"2023-10-06T10:38:21.399883+0000","last_peered":"2023-10-06T10:38:21.399883+0000","last_clean":"2023-10-06T10:38:21.399883+0000","last_became_active":"2023-10-06T10:38:21.394859+0000","last_became_peered":"2023-10-06T10:38:21.394859+0000","last_unstale":"2023-10-06T10:38:21.399883+0000","last_undegraded":"2023-10-06T10:38:21.399883+0000","last_fullsized":"2023-10-06T10:38:21.399883+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:54:20.934943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378521+0000","last_change":"2023-10-06T10:38:16.361291+0000","last_active":"2023-10-06T10:38:21.378521+0000","last_peered":"2023-10-06T10:38:21.378521+0000","last_clean":"2023-10-06T10:38:21.378521+0000","last_became_active":"2023-10-06T10:38:16.361027+0000","last_became_peered":"2023-10-06T10:38:16.361027+0000","last_unstale":"2023-10-06T10:38:21.378521+0000","last_undegraded":"2023-10-06T10:38:21.378521+0000","last_fullsized":"2023-10-06T10:38:21.378521+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:36.469014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378387+0000","last_change":"2023-10-06T10:38:17.375274+0000","last_active":"2023-10-06T10:38:21.378387+0000","last_peered":"2023-10-06T10:38:21.378387+0000","last_clean":"2023-10-06T10:38:21.378387+0000","last_became_active":"2023-10-06T10:38:17.374982+0000","last_became_peered":"2023-10-06T10:38:17.374982+0000","last_unstale":"2023-10-06T10:38:21.378387+0000","last_undegraded":"2023-10-06T10:38:21.378387+0000","last_fullsized":"2023-10-06T10:38:21.378387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:59.822053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385168+0000","last_change":"2023-10-06T10:38:17.368598+0000","last_active":"2023-10-06T10:38:21.385168+0000","last_peered":"2023-10-06T10:38:21.385168+0000","last_clean":"2023-10-06T10:38:21.385168+0000","last_became_active":"2023-10-06T10:38:17.367720+0000","last_became_peered":"2023-10-06T10:38:17.367720+0000","last_unstale":"2023-10-06T10:38:21.385168+0000","last_undegraded":"2023-10-06T10:38:21.385168+0000","last_fullsized":"2023-10-06T10:38:21.385168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:40.679246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372285+0000","last_change":"2023-10-06T10:38:17.362818+0000","last_active":"2023-10-06T10:38:21.372285+0000","last_peered":"2023-10-06T10:38:21.372285+0000","last_clean":"2023-10-06T10:38:21.372285+0000","last_became_active":"2023-10-06T10:38:17.362585+0000","last_became_peered":"2023-10-06T10:38:17.362585+0000","last_unstale":"2023-10-06T10:38:21.372285+0000","last_undegraded":"2023-10-06T10:38:21.372285+0000","last_fullsized":"2023-10-06T10:38:21.372285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:03.650819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378216+0000","last_change":"2023-10-06T10:38:17.375510+0000","last_active":"2023-10-06T10:38:21.378216+0000","last_peered":"2023-10-06T10:38:21.378216+0000","last_clean":"2023-10-06T10:38:21.378216+0000","last_became_active":"2023-10-06T10:38:17.375023+0000","last_became_peered":"2023-10-06T10:38:17.375023+0000","last_unstale":"2023-10-06T10:38:21.378216+0000","last_undegraded":"2023-10-06T10:38:21.378216+0000","last_fullsized":"2023-10-06T10:38:21.378216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:10.177603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372410+0000","last_change":"2023-10-06T10:38:17.370520+0000","last_active":"2023-10-06T10:38:21.372410+0000","last_peered":"2023-10-06T10:38:21.372410+0000","last_clean":"2023-10-06T10:38:21.372410+0000","last_became_active":"2023-10-06T10:38:17.370265+0000","last_became_peered":"2023-10-06T10:38:17.370265+0000","last_unstale":"2023-10-06T10:38:21.372410+0000","last_undegraded":"2023-10-06T10:38:21.372410+0000","last_fullsized":"2023-10-06T10:38:21.372410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:06.214524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.379183+0000","last_change":"2023-10-06T10:38:17.372067+0000","last_active":"2023-10-06T10:38:21.379183+0000","last_peered":"2023-10-06T10:38:21.379183+0000","last_clean":"2023-10-06T10:38:21.379183+0000","last_became_active":"2023-10-06T10:38:17.371813+0000","last_became_peered":"2023-10-06T10:38:17.371813+0000","last_unstale":"2023-10-06T10:38:21.379183+0000","last_undegraded":"2023-10-06T10:38:21.379183+0000","last_fullsized":"2023-10-06T10:38:21.379183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:23:23.463492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4484,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":876544,"data_stored":698736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":103085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298116,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073724,"statfs":{"total":96636764160,"available":96331493376,"internally_reserved":0,"allocated":847872,"data_stored":694806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":258048,"data_stored":104992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":104438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":258048,"data_stored":104528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.248}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297556,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074284,"statfs":{"total":96636764160,"available":96332066816,"internally_reserved":0,"allocated":274432,"data_stored":108208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":105062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:38:31.823 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T10:38:31.824 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-06T10:38:31.824 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-06T10:38:31.824 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-06T10:38:31.824 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-06T10:38:31.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.966+0000 7f62fc235700 1 Processor -- start 2023-10-06T10:38:31.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.966+0000 7f62fc235700 1 -- start start 2023-10-06T10:38:31.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.967+0000 7f62fc235700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62f40adb60 0x7f62f40ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:31.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.967+0000 7f62fc235700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f62f40ce6d0 0x7f62f415aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:31.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.967+0000 7f62fc235700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62f4157de0 con 0x7f62f40ca370 2023-10-06T10:38:31.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.967+0000 7f62fc235700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62f40cbb90 con 0x7f62f40adb60 2023-10-06T10:38:31.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.967+0000 7f62fc235700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62f40ceb10 con 0x7f62f40ce6d0 2023-10-06T10:38:31.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.967+0000 7f62f97d0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62f40adb60 0x7f62f40ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:31.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.967+0000 7f62f97d0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62f40adb60 0x7f62f40ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37930/0 (socket says 172.21.15.92:37930) 2023-10-06T10:38:31.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.967+0000 7f62f97d0700 1 -- 172.21.15.92:0/1728652248 learned_addr learned my addr 172.21.15.92:0/1728652248 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:31.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.968+0000 7f62fa7d2700 1 --2- 172.21.15.92:0/1728652248 >> v2:172.21.15.124:3300/0 conn(0x7f62f40ce6d0 0x7f62f415aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:31.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.968+0000 7f62f97d0700 1 -- 172.21.15.92:0/1728652248 >> v2:172.21.15.124:3300/0 conn(0x7f62f40ce6d0 msgr2=0x7f62f415aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:31.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.968+0000 7f62f97d0700 1 --2- 172.21.15.92:0/1728652248 >> v2:172.21.15.124:3300/0 conn(0x7f62f40ce6d0 0x7f62f415aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.968+0000 7f62f97d0700 1 -- 172.21.15.92:0/1728652248 >> v1:172.21.15.124:6789/0 conn(0x7f62f40ca370 legacy=0x7f62f4158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:31.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.968+0000 7f62f97d0700 1 -- 172.21.15.92:0/1728652248 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f40cb5d0 con 0x7f62f40adb60 2023-10-06T10:38:31.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.968+0000 7f62f97d0700 1 --2- 172.21.15.92:0/1728652248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62f40adb60 0x7f62f40ce190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f62f41554c0 tx=0x7f62e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16509ddbb44db73 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:31.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.968+0000 7f62f8fcf700 1 -- 172.21.15.92:0/1728652248 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e400b860 con 0x7f62f40adb60 2023-10-06T10:38:31.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.968+0000 7f62f8fcf700 1 -- 172.21.15.92:0/1728652248 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e400b9c0 con 0x7f62f40adb60 2023-10-06T10:38:31.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.968+0000 7f62f8fcf700 1 -- 172.21.15.92:0/1728652248 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e400bd60 con 0x7f62f40adb60 2023-10-06T10:38:31.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.968+0000 7f62fc235700 1 -- 172.21.15.92:0/1728652248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62f40adb60 msgr2=0x7f62f40ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:31.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.968+0000 7f62fc235700 1 --2- 172.21.15.92:0/1728652248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62f40adb60 0x7f62f40ce190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f62f41554c0 tx=0x7f62e4009250 comp rx=0 tx=0).stop 2023-10-06T10:38:31.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.968+0000 7f62fc235700 1 -- 172.21.15.92:0/1728652248 shutdown_connections 2023-10-06T10:38:31.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.968+0000 7f62fc235700 1 --2- 172.21.15.92:0/1728652248 >> v2:172.21.15.124:3300/0 conn(0x7f62f40ce6d0 0x7f62f415aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.968+0000 7f62fc235700 1 --2- 172.21.15.92:0/1728652248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62f40adb60 0x7f62f40ce190 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.968+0000 7f62fc235700 1 -- 172.21.15.92:0/1728652248 >> 172.21.15.92:0/1728652248 conn(0x7f62f40707d0 msgr2=0x7f62f4070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:31.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.969+0000 7f62fc235700 1 -- 172.21.15.92:0/1728652248 shutdown_connections 2023-10-06T10:38:31.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.969+0000 7f62fc235700 1 -- 172.21.15.92:0/1728652248 wait complete. 2023-10-06T10:38:31.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.970+0000 7f62fc235700 1 Processor -- start 2023-10-06T10:38:31.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.970+0000 7f62fc235700 1 -- start start 2023-10-06T10:38:31.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.970+0000 7f62fc235700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62f40adb60 0x7f62f40c26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:31.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.970+0000 7f62fc235700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62f40ca370 0x7f62f4168ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:31.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.970+0000 7f62fc235700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62f40ce6d0 0x7f62f416ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:31.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.970+0000 7f62fc235700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f416b140 con 0x7f62f40ce6d0 2023-10-06T10:38:31.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.970+0000 7f62fc235700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f4168d50 con 0x7f62f40ca370 2023-10-06T10:38:31.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.970+0000 7f62fc235700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62f41681d0 con 0x7f62f40adb60 2023-10-06T10:38:31.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.970+0000 7f62fa7d2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62f40ce6d0 0x7f62f416ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:31.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.971+0000 7f62fa7d2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62f40ce6d0 0x7f62f416ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37944/0 (socket says 172.21.15.92:37944) 2023-10-06T10:38:31.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.971+0000 7f62fa7d2700 1 -- 172.21.15.92:0/2747869726 learned_addr learned my addr 172.21.15.92:0/2747869726 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:31.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.971+0000 7f62f97d0700 1 --2- 172.21.15.92:0/2747869726 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62f40ca370 0x7f62f4168ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:31.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.971+0000 7f62f9fd1700 1 --2- 172.21.15.92:0/2747869726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62f40adb60 0x7f62f40c26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:31.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.971+0000 7f62fa7d2700 1 -- 172.21.15.92:0/2747869726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62f40adb60 msgr2=0x7f62f40c26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:31.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.971+0000 7f62fa7d2700 1 --2- 172.21.15.92:0/2747869726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62f40adb60 0x7f62f40c26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.971+0000 7f62fa7d2700 1 -- 172.21.15.92:0/2747869726 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62f40ca370 msgr2=0x7f62f4168ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:31.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.971+0000 7f62fa7d2700 1 --2- 172.21.15.92:0/2747869726 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62f40ca370 0x7f62f4168ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:31.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.971+0000 7f62fa7d2700 1 -- 172.21.15.92:0/2747869726 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e40094f0 con 0x7f62f40ce6d0 2023-10-06T10:38:31.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.971+0000 7f62fa7d2700 1 --2- 172.21.15.92:0/2747869726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62f40ce6d0 0x7f62f416ac00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f62ec009fa0 tx=0x7f62ec009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:31.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.971+0000 7f62e9ffb700 1 -- 172.21.15.92:0/2747869726 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec00ebc0 con 0x7f62f40ce6d0 2023-10-06T10:38:31.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.971+0000 7f62e9ffb700 1 -- 172.21.15.92:0/2747869726 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ec00ed20 con 0x7f62f40ce6d0 2023-10-06T10:38:31.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.971+0000 7f62fc235700 1 -- 172.21.15.92:0/2747869726 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f41684b0 con 0x7f62f40ce6d0 2023-10-06T10:38:31.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.971+0000 7f62e9ffb700 1 -- 172.21.15.92:0/2747869726 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec014670 con 0x7f62f40ce6d0 2023-10-06T10:38:31.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.971+0000 7f62fc235700 1 -- 172.21.15.92:0/2747869726 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f4168a70 con 0x7f62f40ce6d0 2023-10-06T10:38:31.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.972+0000 7f62fc235700 1 -- 172.21.15.92:0/2747869726 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62f40ad920 con 0x7f62f40ce6d0 2023-10-06T10:38:31.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.973+0000 7f62e9ffb700 1 -- 172.21.15.92:0/2747869726 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f62ec0149e0 con 0x7f62f40ce6d0 2023-10-06T10:38:31.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.973+0000 7f62e9ffb700 1 --2- 172.21.15.92:0/2747869726 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f62d8066f30 0x7f62d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:31.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.973+0000 7f62e9ffb700 1 -- 172.21.15.92:0/2747869726 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f62ec085020 con 0x7f62f40ce6d0 2023-10-06T10:38:31.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.973+0000 7f62f9fd1700 1 --2- 172.21.15.92:0/2747869726 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f62d8066f30 0x7f62d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:31.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.973+0000 7f62f9fd1700 1 --2- 172.21.15.92:0/2747869726 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f62d8066f30 0x7f62d80693e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f62f00039e0 tx=0x7f62f0007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:31.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:31.977+0000 7f62e9ffb700 1 -- 172.21.15.92:0/2747869726 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f62ec051e40 con 0x7f62f40ce6d0 2023-10-06T10:38:32.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.211+0000 7f62fc235700 1 -- 172.21.15.92:0/2747869726 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f62f40b4240 con 0x7f62f40ce6d0 2023-10-06T10:38:32.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.214+0000 7f62e9ffb700 1 -- 172.21.15.92:0/2747869726 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163699 (secure 0 0 0) 0x7f62ec055460 con 0x7f62f40ce6d0 2023-10-06T10:38:32.219 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:32.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.217+0000 7f62fc235700 1 -- 172.21.15.92:0/2747869726 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f62d8066f30 msgr2=0x7f62d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:32.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.217+0000 7f62fc235700 1 --2- 172.21.15.92:0/2747869726 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f62d8066f30 0x7f62d80693e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f62f00039e0 tx=0x7f62f0007ec0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.217+0000 7f62fc235700 1 -- 172.21.15.92:0/2747869726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62f40ce6d0 msgr2=0x7f62f416ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:32.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.217+0000 7f62fc235700 1 --2- 172.21.15.92:0/2747869726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62f40ce6d0 0x7f62f416ac00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f62ec009fa0 tx=0x7f62ec009900 comp rx=0 tx=0).stop 2023-10-06T10:38:32.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.217+0000 7f62fc235700 1 -- 172.21.15.92:0/2747869726 shutdown_connections 2023-10-06T10:38:32.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.217+0000 7f62fc235700 1 --2- 172.21.15.92:0/2747869726 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f62d8066f30 0x7f62d80693e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.217+0000 7f62fc235700 1 --2- 172.21.15.92:0/2747869726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62f40ce6d0 0x7f62f416ac00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.217+0000 7f62fc235700 1 --2- 172.21.15.92:0/2747869726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62f40adb60 0x7f62f40c26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.217+0000 7f62fc235700 1 --2- 172.21.15.92:0/2747869726 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62f40ca370 0x7f62f4168ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.217+0000 7f62fc235700 1 -- 172.21.15.92:0/2747869726 >> 172.21.15.92:0/2747869726 conn(0x7f62f40707d0 msgr2=0x7f62f41593b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:32.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.217+0000 7f62fc235700 1 -- 172.21.15.92:0/2747869726 shutdown_connections 2023-10-06T10:38:32.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.217+0000 7f62fc235700 1 -- 172.21.15.92:0/2747869726 wait complete. 2023-10-06T10:38:32.230 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":5,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":90373},{"type":"v1","addr":"172.21.15.92:6833","nonce":90373}]},"active_addr":"172.21.15.92:6833/90373","active_change":"2023-10-06T10:38:04.074130+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4103,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":3170501874}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":2069808572}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":2869989983}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":3851058350}]}]} 2023-10-06T10:38:32.235 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-06T10:38:32.235 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-06T10:38:32.235 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:38:32.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.348+0000 7f189f59e700 1 Processor -- start 2023-10-06T10:38:32.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.348+0000 7f189f59e700 1 -- start start 2023-10-06T10:38:32.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.349+0000 7f189f59e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18a0159440 0x7f18a00b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:32.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.349+0000 7f189f59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f18a00b2100 0x7f18a00b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:32.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.349+0000 7f189f59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18a00cb7a0 con 0x7f18a00b35d0 2023-10-06T10:38:32.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.349+0000 7f189f59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a00b0860 con 0x7f18a00b2100 2023-10-06T10:38:32.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.349+0000 7f189f59e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a00cb150 con 0x7f18a0159440 2023-10-06T10:38:32.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.349+0000 7f189ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f18a00b2100 0x7f18a00b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:32.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.349+0000 7f1895d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f18a00b35d0 0x7f18a00cbc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59962/0 (socket says 172.21.15.92:59962) 2023-10-06T10:38:32.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.349+0000 7f1895d9b700 1 -- 172.21.15.92:0/3532628528 learned_addr learned my addr 172.21.15.92:0/3532628528 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:32.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.349+0000 7f189e59c700 1 --2- 172.21.15.92:0/3532628528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18a0159440 0x7f18a00b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:32.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.349+0000 7f189ed9d700 1 -- 172.21.15.92:0/3532628528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18a0159440 msgr2=0x7f18a00b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:32.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.349+0000 7f189ed9d700 1 --2- 172.21.15.92:0/3532628528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18a0159440 0x7f18a00b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.349+0000 7f189ed9d700 1 -- 172.21.15.92:0/3532628528 >> v1:172.21.15.92:6789/0 conn(0x7f18a00b35d0 legacy=0x7f18a00cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:32.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.349+0000 7f189dd9b700 1 -- 172.21.15.92:0/3532628528 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3490872490 0 0) 0x7f18a00cb7a0 con 0x7f18a00b35d0 2023-10-06T10:38:32.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.349+0000 7f189ed9d700 1 -- 172.21.15.92:0/3532628528 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a00cb430 con 0x7f18a00b2100 2023-10-06T10:38:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.349+0000 7f189ed9d700 1 --2- 172.21.15.92:0/3532628528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a00b2100 0x7f18a00b2580 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1898009e60 tx=0x7f1898009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea43c78a66b74ff8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.350+0000 7f189dd9b700 1 -- 172.21.15.92:0/3532628528 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189800e070 con 0x7f18a00b2100 2023-10-06T10:38:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.350+0000 7f189dd9b700 1 -- 172.21.15.92:0/3532628528 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189800bb60 con 0x7f18a00b2100 2023-10-06T10:38:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.350+0000 7f189dd9b700 1 -- 172.21.15.92:0/3532628528 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1898012430 con 0x7f18a00b2100 2023-10-06T10:38:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.350+0000 7f189f59e700 1 -- 172.21.15.92:0/3532628528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a00b2100 msgr2=0x7f18a00b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.350+0000 7f189f59e700 1 --2- 172.21.15.92:0/3532628528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a00b2100 0x7f18a00b2580 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1898009e60 tx=0x7f1898009780 comp rx=0 tx=0).stop 2023-10-06T10:38:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.350+0000 7f189f59e700 1 -- 172.21.15.92:0/3532628528 shutdown_connections 2023-10-06T10:38:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.350+0000 7f189f59e700 1 --2- 172.21.15.92:0/3532628528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18a0159440 0x7f18a00b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.350+0000 7f189f59e700 1 --2- 172.21.15.92:0/3532628528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a00b2100 0x7f18a00b2580 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.350+0000 7f189f59e700 1 -- 172.21.15.92:0/3532628528 >> 172.21.15.92:0/3532628528 conn(0x7f18a006ece0 msgr2=0x7f18a006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.350+0000 7f189f59e700 1 -- 172.21.15.92:0/3532628528 shutdown_connections 2023-10-06T10:38:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.350+0000 7f189f59e700 1 -- 172.21.15.92:0/3532628528 wait complete. 2023-10-06T10:38:32.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.351+0000 7f189f59e700 1 Processor -- start 2023-10-06T10:38:32.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.351+0000 7f189f59e700 1 -- start start 2023-10-06T10:38:32.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.352+0000 7f189f59e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18a00b2100 0x7f18a01537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:32.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.352+0000 7f189f59e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18a00b35d0 0x7f18a00c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:32.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.352+0000 7f189f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a0159440 0x7f18a00c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:32.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.352+0000 7f189f59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a00cfde0 con 0x7f18a0159440 2023-10-06T10:38:32.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.352+0000 7f189f59e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a00cf1d0 con 0x7f18a00b2100 2023-10-06T10:38:32.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.352+0000 7f189f59e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a00cf4b0 con 0x7f18a00b35d0 2023-10-06T10:38:32.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.352+0000 7f189ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a0159440 0x7f18a00c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:32.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.352+0000 7f189ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a0159440 0x7f18a00c3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37962/0 (socket says 172.21.15.92:37962) 2023-10-06T10:38:32.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.352+0000 7f189ed9d700 1 -- 172.21.15.92:0/1729945630 learned_addr learned my addr 172.21.15.92:0/1729945630 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:32.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.352+0000 7f189e59c700 1 --2- 172.21.15.92:0/1729945630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18a00b2100 0x7f18a01537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:32.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.352+0000 7f1895d9b700 1 --2- 172.21.15.92:0/1729945630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18a00b35d0 0x7f18a00c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:32.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.352+0000 7f189ed9d700 1 -- 172.21.15.92:0/1729945630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18a00b35d0 msgr2=0x7f18a00c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:32.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.352+0000 7f189ed9d700 1 --2- 172.21.15.92:0/1729945630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18a00b35d0 0x7f18a00c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.352+0000 7f189ed9d700 1 -- 172.21.15.92:0/1729945630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18a00b2100 msgr2=0x7f18a01537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:32.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.352+0000 7f189ed9d700 1 --2- 172.21.15.92:0/1729945630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18a00b2100 0x7f18a01537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.352+0000 7f189ed9d700 1 -- 172.21.15.92:0/1729945630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1898009a20 con 0x7f18a0159440 2023-10-06T10:38:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.353+0000 7f1895d9b700 1 --2- 172.21.15.92:0/1729945630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18a00b35d0 0x7f18a00c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.353+0000 7f189ed9d700 1 --2- 172.21.15.92:0/1729945630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a0159440 0x7f18a00c3e10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f189800b6c0 tx=0x7f1898019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.353+0000 7f1896ffd700 1 -- 172.21.15.92:0/1729945630 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1898009420 con 0x7f18a0159440 2023-10-06T10:38:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.353+0000 7f1896ffd700 1 -- 172.21.15.92:0/1729945630 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1898027470 con 0x7f18a0159440 2023-10-06T10:38:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.353+0000 7f189f59e700 1 -- 172.21.15.92:0/1729945630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a00cf790 con 0x7f18a0159440 2023-10-06T10:38:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.353+0000 7f1896ffd700 1 -- 172.21.15.92:0/1729945630 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1898027830 con 0x7f18a0159440 2023-10-06T10:38:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.353+0000 7f189f59e700 1 -- 172.21.15.92:0/1729945630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a00cec10 con 0x7f18a0159440 2023-10-06T10:38:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.354+0000 7f189f59e700 1 -- 172.21.15.92:0/1729945630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a005f230 con 0x7f18a0159440 2023-10-06T10:38:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.354+0000 7f1896ffd700 1 -- 172.21.15.92:0/1729945630 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1898024070 con 0x7f18a0159440 2023-10-06T10:38:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.355+0000 7f1896ffd700 1 --2- 172.21.15.92:0/1729945630 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1880066f30 0x7f18800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.355+0000 7f1896ffd700 1 -- 172.21.15.92:0/1729945630 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f1898087030 con 0x7f18a0159440 2023-10-06T10:38:32.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.355+0000 7f189e59c700 1 --2- 172.21.15.92:0/1729945630 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1880066f30 0x7f18800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:32.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.355+0000 7f189e59c700 1 --2- 172.21.15.92:0/1729945630 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1880066f30 0x7f18800693e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f18a00d0e40 tx=0x7f18900091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:32.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.359+0000 7f1896ffd700 1 -- 172.21.15.92:0/1729945630 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1898053e50 con 0x7f18a0159440 2023-10-06T10:38:32.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.546+0000 7f189f59e700 1 -- 172.21.15.92:0/1729945630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18a00b2ac0 con 0x7f18a0159440 2023-10-06T10:38:32.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.547+0000 7f1896ffd700 1 -- 172.21.15.92:0/1729945630 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18707 (secure 0 0 0) 0x7f1898057470 con 0x7f18a0159440 2023-10-06T10:38:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:32.549 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:21.760233+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:38:14.368713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T10:38:15.692344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T10:38:19.382553+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:38:32.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.549+0000 7f189f59e700 1 -- 172.21.15.92:0/1729945630 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1880066f30 msgr2=0x7f18800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:32.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.549+0000 7f189f59e700 1 --2- 172.21.15.92:0/1729945630 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1880066f30 0x7f18800693e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f18a00d0e40 tx=0x7f18900091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.549+0000 7f189f59e700 1 -- 172.21.15.92:0/1729945630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a0159440 msgr2=0x7f18a00c3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:32.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.549+0000 7f189f59e700 1 --2- 172.21.15.92:0/1729945630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a0159440 0x7f18a00c3e10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f189800b6c0 tx=0x7f1898019000 comp rx=0 tx=0).stop 2023-10-06T10:38:32.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.549+0000 7f189f59e700 1 -- 172.21.15.92:0/1729945630 shutdown_connections 2023-10-06T10:38:32.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.549+0000 7f189f59e700 1 --2- 172.21.15.92:0/1729945630 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1880066f30 0x7f18800693e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.549+0000 7f189f59e700 1 --2- 172.21.15.92:0/1729945630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a0159440 0x7f18a00c3e10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.549+0000 7f189f59e700 1 --2- 172.21.15.92:0/1729945630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18a00b2100 0x7f18a01537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.549+0000 7f189f59e700 1 --2- 172.21.15.92:0/1729945630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18a00b35d0 0x7f18a00c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.549+0000 7f189f59e700 1 -- 172.21.15.92:0/1729945630 >> 172.21.15.92:0/1729945630 conn(0x7f18a006ece0 msgr2=0x7f18a0156760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:32.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.549+0000 7f189f59e700 1 -- 172.21.15.92:0/1729945630 shutdown_connections 2023-10-06T10:38:32.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.549+0000 7f189f59e700 1 -- 172.21.15.92:0/1729945630 wait complete. 2023-10-06T10:38:32.563 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-06T10:38:32.563 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:38:32.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.676+0000 7f96131ab700 1 Processor -- start 2023-10-06T10:38:32.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.676+0000 7f96131ab700 1 -- start start 2023-10-06T10:38:32.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.677+0000 7f96131ab700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f960c0ad210 0x7f960c0cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:32.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.677+0000 7f96131ab700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f960c0cecf0 con 0x7f960c0ad5e0 2023-10-06T10:38:32.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.677+0000 7f96131ab700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f960c0cefd0 con 0x7f960c15db10 2023-10-06T10:38:32.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.677+0000 7f96131ab700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f960c0cca90 con 0x7f960c0ad210 2023-10-06T10:38:32.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.677+0000 7f960bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f960c0ad5e0 0x7f960c15d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59966/0 (socket says 172.21.15.92:59966) 2023-10-06T10:38:32.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.677+0000 7f960bfff700 1 -- 172.21.15.92:0/2196077375 learned_addr learned my addr 172.21.15.92:0/2196077375 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:32.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.677+0000 7f9610f47700 1 --2- 172.21.15.92:0/2196077375 >> v2:172.21.15.124:3300/0 conn(0x7f960c0ad210 0x7f960c0cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:32.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.677+0000 7f960b7fe700 1 -- 172.21.15.92:0/2196077375 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 399084305 0 0) 0x7f960c0cecf0 con 0x7f960c0ad5e0 2023-10-06T10:38:32.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.677+0000 7f960b7fe700 1 -- 172.21.15.92:0/2196077375 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95fc003570 con 0x7f960c0ad5e0 2023-10-06T10:38:32.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.677+0000 7f960b7fe700 1 -- 172.21.15.92:0/2196077375 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 348334879 0 0) 0x7f95fc003570 con 0x7f960c0ad5e0 2023-10-06T10:38:32.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.677+0000 7f960b7fe700 1 -- 172.21.15.92:0/2196077375 >> v2:172.21.15.124:3300/0 conn(0x7f960c0ad210 msgr2=0x7f960c0cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:32.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.677+0000 7f960b7fe700 1 --2- 172.21.15.92:0/2196077375 >> v2:172.21.15.124:3300/0 conn(0x7f960c0ad210 0x7f960c0cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.677+0000 7f960b7fe700 1 -- 172.21.15.92:0/2196077375 >> v1:172.21.15.124:6789/0 conn(0x7f960c15db10 legacy=0x7f960c0ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:32.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.677+0000 7f960b7fe700 1 -- 172.21.15.92:0/2196077375 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960c0ccd70 con 0x7f960c0ad5e0 2023-10-06T10:38:32.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.678+0000 7f960b7fe700 1 -- 172.21.15.92:0/2196077375 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f95f8001470 con 0x7f960c0ad5e0 2023-10-06T10:38:32.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.678+0000 7f960b7fe700 1 -- 172.21.15.92:0/2196077375 >> v1:172.21.15.92:6789/0 conn(0x7f960c0ad5e0 legacy=0x7f960c15d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:32.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.678+0000 7f960b7fe700 1 --2- 172.21.15.92:0/2196077375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95fc003a60 0x7f95fc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:32.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.678+0000 7f960b7fe700 1 --2- 172.21.15.92:0/2196077375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95fc004810 0x7f95fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:32.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.678+0000 7f960b7fe700 1 --2- 172.21.15.92:0/2196077375 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95fc0051c0 0x7f95fc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:32.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.678+0000 7f960b7fe700 1 -- 172.21.15.92:0/2196077375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c0cca90 con 0x7f95fc004810 2023-10-06T10:38:32.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.678+0000 7f960b7fe700 1 -- 172.21.15.92:0/2196077375 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc00c050 con 0x7f95fc0051c0 2023-10-06T10:38:32.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.678+0000 7f960b7fe700 1 -- 172.21.15.92:0/2196077375 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc00c210 con 0x7f95fc003a60 2023-10-06T10:38:32.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.678+0000 7f960bfff700 1 --2- 172.21.15.92:0/2196077375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95fc004810 0x7f95fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:32.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.678+0000 7f9610f47700 1 --2- 172.21.15.92:0/2196077375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95fc003a60 0x7f95fc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:32.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.678+0000 7f9611748700 1 --2- 172.21.15.92:0/2196077375 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95fc0051c0 0x7f95fc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:32.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.678+0000 7f960bfff700 1 -- 172.21.15.92:0/2196077375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95fc003a60 msgr2=0x7f95fc003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:32.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.678+0000 7f960bfff700 1 --2- 172.21.15.92:0/2196077375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95fc003a60 0x7f95fc003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.678+0000 7f960bfff700 1 -- 172.21.15.92:0/2196077375 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95fc0051c0 msgr2=0x7f95fc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:32.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.678+0000 7f960bfff700 1 --2- 172.21.15.92:0/2196077375 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95fc0051c0 0x7f95fc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.678+0000 7f9611748700 1 -- 172.21.15.92:0/2196077375 reap_dead start 2023-10-06T10:38:32.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.678+0000 7f960bfff700 1 -- 172.21.15.92:0/2196077375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc003570 con 0x7f95fc004810 2023-10-06T10:38:32.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.678+0000 7f9610f47700 1 --2- 172.21.15.92:0/2196077375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95fc003a60 0x7f95fc003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:32.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.678+0000 7f960bfff700 1 --2- 172.21.15.92:0/2196077375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95fc004810 0x7f95fc004c80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f95f800c390 tx=0x7f95f8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58ee12418864e6c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:32.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.678+0000 7f960b7fe700 1 -- 172.21.15.92:0/2196077375 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f800cc60 con 0x7f95fc004810 2023-10-06T10:38:32.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.678+0000 7f960b7fe700 1 -- 172.21.15.92:0/2196077375 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f800cdc0 con 0x7f95fc004810 2023-10-06T10:38:32.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.678+0000 7f960b7fe700 1 -- 172.21.15.92:0/2196077375 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8005b70 con 0x7f95fc004810 2023-10-06T10:38:32.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.679+0000 7f96131ab700 1 -- 172.21.15.92:0/2196077375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95fc004810 msgr2=0x7f95fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:32.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.679+0000 7f96131ab700 1 --2- 172.21.15.92:0/2196077375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95fc004810 0x7f95fc004c80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f95f800c390 tx=0x7f95f8001e70 comp rx=0 tx=0).stop 2023-10-06T10:38:32.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.679+0000 7f96131ab700 1 -- 172.21.15.92:0/2196077375 shutdown_connections 2023-10-06T10:38:32.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.679+0000 7f96131ab700 1 --2- 172.21.15.92:0/2196077375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95fc004810 0x7f95fc004c80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.679+0000 7f96131ab700 1 -- 172.21.15.92:0/2196077375 >> 172.21.15.92:0/2196077375 conn(0x7f960c067d90 msgr2=0x7f960c0ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:32.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.679+0000 7f96131ab700 1 -- 172.21.15.92:0/2196077375 shutdown_connections 2023-10-06T10:38:32.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.679+0000 7f96131ab700 1 -- 172.21.15.92:0/2196077375 wait complete. 2023-10-06T10:38:32.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.680+0000 7f96131ab700 1 Processor -- start 2023-10-06T10:38:32.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.680+0000 7f96131ab700 1 -- start start 2023-10-06T10:38:32.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.681+0000 7f96131ab700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95fc004810 0x7f960c0c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:32.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.681+0000 7f96131ab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f960c0c7d00 0x7f960c06c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:32.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.681+0000 7f96131ab700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f960c06c8f0 0x7f960c16a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:32.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.681+0000 7f96131ab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c0b1ff0 con 0x7f960c0c7d00 2023-10-06T10:38:32.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.681+0000 7f96131ab700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c06cd40 con 0x7f960c06c8f0 2023-10-06T10:38:32.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.681+0000 7f96131ab700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960c0b2200 con 0x7f95fc004810 2023-10-06T10:38:32.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.681+0000 7f960bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f960c0c7d00 0x7f960c06c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:32.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.681+0000 7f960bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f960c0c7d00 0x7f960c06c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37978/0 (socket says 172.21.15.92:37978) 2023-10-06T10:38:32.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.681+0000 7f960bfff700 1 -- 172.21.15.92:0/1639339748 learned_addr learned my addr 172.21.15.92:0/1639339748 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:32.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.681+0000 7f9610f47700 1 --2- 172.21.15.92:0/1639339748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95fc004810 0x7f960c0c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:32.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.681+0000 7f9611748700 1 --2- 172.21.15.92:0/1639339748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f960c06c8f0 0x7f960c16a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:32.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.681+0000 7f960bfff700 1 -- 172.21.15.92:0/1639339748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95fc004810 msgr2=0x7f960c0c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:32.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.681+0000 7f960bfff700 1 --2- 172.21.15.92:0/1639339748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95fc004810 0x7f960c0c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.681+0000 7f960bfff700 1 -- 172.21.15.92:0/1639339748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f960c06c8f0 msgr2=0x7f960c16a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:32.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.681+0000 7f960bfff700 1 --2- 172.21.15.92:0/1639339748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f960c06c8f0 0x7f960c16a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.681+0000 7f960bfff700 1 -- 172.21.15.92:0/1639339748 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960c0b24e0 con 0x7f960c0c7d00 2023-10-06T10:38:32.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.681+0000 7f9610f47700 1 --2- 172.21.15.92:0/1639339748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95fc004810 0x7f960c0c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:32.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.681+0000 7f9611748700 1 --2- 172.21.15.92:0/1639339748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f960c06c8f0 0x7f960c16a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:32.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.681+0000 7f960bfff700 1 --2- 172.21.15.92:0/1639339748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f960c0c7d00 0x7f960c06c3b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f95f800c980 tx=0x7f95f8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:32.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.682+0000 7f9608ff9700 1 -- 172.21.15.92:0/1639339748 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8022070 con 0x7f960c0c7d00 2023-10-06T10:38:32.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.682+0000 7f9608ff9700 1 -- 172.21.15.92:0/1639339748 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f8001470 con 0x7f960c0c7d00 2023-10-06T10:38:32.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.682+0000 7f96131ab700 1 -- 172.21.15.92:0/1639339748 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960c0b27c0 con 0x7f960c0c7d00 2023-10-06T10:38:32.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.682+0000 7f9608ff9700 1 -- 172.21.15.92:0/1639339748 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8007300 con 0x7f960c0c7d00 2023-10-06T10:38:32.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.682+0000 7f96131ab700 1 -- 172.21.15.92:0/1639339748 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960c0b2fc0 con 0x7f960c0c7d00 2023-10-06T10:38:32.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.683+0000 7f96131ab700 1 -- 172.21.15.92:0/1639339748 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f960c0b2ce0 con 0x7f960c0c7d00 2023-10-06T10:38:32.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.683+0000 7f9608ff9700 1 -- 172.21.15.92:0/1639339748 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f95f80076f0 con 0x7f960c0c7d00 2023-10-06T10:38:32.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.683+0000 7f9608ff9700 1 --2- 172.21.15.92:0/1639339748 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f95f0066f30 0x7f95f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:32.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.683+0000 7f9608ff9700 1 -- 172.21.15.92:0/1639339748 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f95f8085490 con 0x7f960c0c7d00 2023-10-06T10:38:32.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.683+0000 7f9610f47700 1 --2- 172.21.15.92:0/1639339748 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f95f0066f30 0x7f95f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:32.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.684+0000 7f9610f47700 1 --2- 172.21.15.92:0/1639339748 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f95f0066f30 0x7f95f00693e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9600006aa0 tx=0x7f9600008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:32.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.688+0000 7f9608ff9700 1 -- 172.21.15.92:0/1639339748 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f960c0b2ce0 con 0x7f960c0c7d00 2023-10-06T10:38:32.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.869+0000 7f96131ab700 1 -- 172.21.15.92:0/1639339748 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f960c0b2ce0 con 0x7f960c0c7d00 2023-10-06T10:38:32.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.870+0000 7f9608ff9700 1 -- 172.21.15.92:0/1639339748 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18707 (secure 0 0 0) 0x7f960c0b2ce0 con 0x7f960c0c7d00 2023-10-06T10:38:32.872 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:32.873 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:21.760233+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:38:14.368713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T10:38:15.692344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T10:38:19.382553+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:38:32.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.872+0000 7f96131ab700 1 -- 172.21.15.92:0/1639339748 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f95f0066f30 msgr2=0x7f95f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:32.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.872+0000 7f96131ab700 1 --2- 172.21.15.92:0/1639339748 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f95f0066f30 0x7f95f00693e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9600006aa0 tx=0x7f9600008040 comp rx=0 tx=0).stop 2023-10-06T10:38:32.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.872+0000 7f96131ab700 1 -- 172.21.15.92:0/1639339748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f960c0c7d00 msgr2=0x7f960c06c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:32.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.872+0000 7f96131ab700 1 --2- 172.21.15.92:0/1639339748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f960c0c7d00 0x7f960c06c3b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f95f800c980 tx=0x7f95f8015000 comp rx=0 tx=0).stop 2023-10-06T10:38:32.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.872+0000 7f96131ab700 1 -- 172.21.15.92:0/1639339748 shutdown_connections 2023-10-06T10:38:32.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.872+0000 7f96131ab700 1 --2- 172.21.15.92:0/1639339748 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f95f0066f30 0x7f95f00693e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.872+0000 7f96131ab700 1 --2- 172.21.15.92:0/1639339748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f960c06c8f0 0x7f960c16a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.872+0000 7f96131ab700 1 --2- 172.21.15.92:0/1639339748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95fc004810 0x7f960c0c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.872+0000 7f96131ab700 1 --2- 172.21.15.92:0/1639339748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f960c0c7d00 0x7f960c06c3b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:32.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.872+0000 7f96131ab700 1 -- 172.21.15.92:0/1639339748 >> 172.21.15.92:0/1639339748 conn(0x7f960c067d90 msgr2=0x7f960c15bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:32.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.872+0000 7f96131ab700 1 -- 172.21.15.92:0/1639339748 shutdown_connections 2023-10-06T10:38:32.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:32.872+0000 7f96131ab700 1 -- 172.21.15.92:0/1639339748 wait complete. 2023-10-06T10:38:32.887 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-06T10:38:32.887 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-06T10:38:32.887 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-06T10:38:32.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-06T10:38:32.889 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-06T10:38:32.889 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-06T10:38:32.890 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-06T10:38:32.890 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-06T10:38:33.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.084+0000 7f48d429b700 1 Processor -- start 2023-10-06T10:38:33.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.084+0000 7f48d429b700 1 -- start start 2023-10-06T10:38:33.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.084+0000 7fd096f71700 1 Processor -- start 2023-10-06T10:38:33.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.084+0000 7fd096f71700 1 -- start start 2023-10-06T10:38:33.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.085+0000 7f48d429b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f48cc0b29b0 0x7f48cc159120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.085+0000 7f48d429b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48cc159660 0x7f48cc0cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.085+0000 7f48d429b700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48cc0b0da0 con 0x7f48cc0b25e0 2023-10-06T10:38:33.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.085+0000 7f48d429b700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48cc0cbfa0 con 0x7f48cc0b29b0 2023-10-06T10:38:33.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.085+0000 7f48d429b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc0cb390 con 0x7f48cc159660 2023-10-06T10:38:33.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.085+0000 7fd096f71700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd090156280 0x7fd09015a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.085+0000 7fd096f71700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd090068cd0 0x7fd0900abb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.085+0000 7fd096f71700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0900c9700 con 0x7fd090156650 2023-10-06T10:38:33.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.085+0000 7fd096f71700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd09015b000 con 0x7fd090068cd0 2023-10-06T10:38:33.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.085+0000 7fd096f71700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0900ad810 con 0x7fd090156280 2023-10-06T10:38:33.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.085+0000 7f48d1836700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f48cc0b29b0 0x7f48cc159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.085+0000 7f48d2838700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48cc159660 0x7f48cc0cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.085+0000 7f48d2037700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f48cc0b25e0 0x7f48cc0b0690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:56852/0 (socket says 172.21.15.92:56852) 2023-10-06T10:38:33.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.085+0000 7f48d2037700 1 -- 172.21.15.92:0/3449701064 learned_addr learned my addr 172.21.15.92:0/3449701064 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:33.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.085+0000 7f48d2838700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48cc159660 0x7f48cc0cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:43780/0 (socket says 172.21.15.92:43780) 2023-10-06T10:38:33.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.086+0000 7fd094d0d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd090156280 0x7fd09015a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.086+0000 7fd094d0d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd090156280 0x7fd09015a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:43788/0 (socket says 172.21.15.92:43788) 2023-10-06T10:38:33.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.086+0000 7fd094d0d700 1 -- 172.21.15.92:0/3276116534 learned_addr learned my addr 172.21.15.92:0/3276116534 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.086+0000 7fd08ffff700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fd090156650 0x7fd0900685c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:56862/0 (socket says 172.21.15.92:56862) 2023-10-06T10:38:33.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.086+0000 7fd09550e700 1 --2- 172.21.15.92:0/3276116534 >> v2:172.21.15.124:3300/0 conn(0x7fd090068cd0 0x7fd0900abb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.086+0000 7f48d2838700 1 -- 172.21.15.92:0/3449701064 >> v2:172.21.15.124:3300/0 conn(0x7f48cc0b29b0 msgr2=0x7f48cc159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:33.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.086+0000 7f48d2838700 1 --2- 172.21.15.92:0/3449701064 >> v2:172.21.15.124:3300/0 conn(0x7f48cc0b29b0 0x7f48cc159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.086+0000 7f48d2838700 1 -- 172.21.15.92:0/3449701064 >> v1:172.21.15.124:6789/0 conn(0x7f48cc0b25e0 legacy=0x7f48cc0b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.086+0000 7f48d2838700 1 -- 172.21.15.92:0/3449701064 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48cc0cb670 con 0x7f48cc159660 2023-10-06T10:38:33.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.086+0000 7f48d1836700 1 --2- 172.21.15.92:0/3449701064 >> v2:172.21.15.124:3300/0 conn(0x7f48cc0b29b0 0x7f48cc159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:33.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.086+0000 7fd094d0d700 1 -- 172.21.15.92:0/3276116534 >> v2:172.21.15.124:3300/0 conn(0x7fd090068cd0 msgr2=0x7fd0900abb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:33.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.086+0000 7fd094d0d700 1 --2- 172.21.15.92:0/3276116534 >> v2:172.21.15.124:3300/0 conn(0x7fd090068cd0 0x7fd0900abb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.086+0000 7fd094d0d700 1 -- 172.21.15.92:0/3276116534 >> v1:172.21.15.124:6789/0 conn(0x7fd090156650 legacy=0x7fd0900685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.086+0000 7fd094d0d700 1 -- 172.21.15.92:0/3276116534 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09015a760 con 0x7fd090156280 2023-10-06T10:38:33.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.086+0000 7fd09550e700 1 --2- 172.21.15.92:0/3276116534 >> v2:172.21.15.124:3300/0 conn(0x7fd090068cd0 0x7fd0900abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:33.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.086+0000 7f48d2838700 1 --2- 172.21.15.92:0/3449701064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48cc159660 0x7f48cc0cc430 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f48c4009e60 tx=0x7f48c4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34564b9749a4f21 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.086+0000 7fd094d0d700 1 --2- 172.21.15.92:0/3276116534 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd090156280 0x7fd09015a060 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd084009990 tx=0x7fd084009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af99e60f0338eea6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.086+0000 7f48d1035700 1 -- 172.21.15.92:0/3449701064 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c400e070 con 0x7f48cc159660 2023-10-06T10:38:33.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.086+0000 7f48d1035700 1 -- 172.21.15.92:0/3449701064 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c400bb60 con 0x7f48cc159660 2023-10-06T10:38:33.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.086+0000 7f48d1035700 1 -- 172.21.15.92:0/3449701064 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4012430 con 0x7f48cc159660 2023-10-06T10:38:33.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.087+0000 7fd08f7fe700 1 -- 172.21.15.92:0/3276116534 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084009e80 con 0x7fd090156280 2023-10-06T10:38:33.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.087+0000 7fd08f7fe700 1 -- 172.21.15.92:0/3276116534 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd084005c10 con 0x7fd090156280 2023-10-06T10:38:33.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.087+0000 7fd08f7fe700 1 -- 172.21.15.92:0/3276116534 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08400b650 con 0x7fd090156280 2023-10-06T10:38:33.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.087+0000 7f48d429b700 1 -- 172.21.15.92:0/3449701064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48cc159660 msgr2=0x7f48cc0cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.087+0000 7f48d429b700 1 --2- 172.21.15.92:0/3449701064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48cc159660 0x7f48cc0cc430 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f48c4009e60 tx=0x7f48c4009780 comp rx=0 tx=0).stop 2023-10-06T10:38:33.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.087+0000 7f48d429b700 1 -- 172.21.15.92:0/3449701064 shutdown_connections 2023-10-06T10:38:33.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.087+0000 7f48d429b700 1 --2- 172.21.15.92:0/3449701064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48cc159660 0x7f48cc0cc430 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.087+0000 7f48d429b700 1 --2- 172.21.15.92:0/3449701064 >> v2:172.21.15.124:3300/0 conn(0x7f48cc0b29b0 0x7f48cc159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.087+0000 7f48d429b700 1 -- 172.21.15.92:0/3449701064 >> 172.21.15.92:0/3449701064 conn(0x7f48cc06ece0 msgr2=0x7f48cc06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.087+0000 7f48d429b700 1 -- 172.21.15.92:0/3449701064 shutdown_connections 2023-10-06T10:38:33.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.087+0000 7f48d429b700 1 -- 172.21.15.92:0/3449701064 wait complete. 2023-10-06T10:38:33.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.087+0000 7fd096f71700 1 -- 172.21.15.92:0/3276116534 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd090156280 msgr2=0x7fd09015a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.087+0000 7fd096f71700 1 --2- 172.21.15.92:0/3276116534 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd090156280 0x7fd09015a060 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd084009990 tx=0x7fd084009220 comp rx=0 tx=0).stop 2023-10-06T10:38:33.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.087+0000 7fd096f71700 1 -- 172.21.15.92:0/3276116534 shutdown_connections 2023-10-06T10:38:33.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.087+0000 7fd096f71700 1 --2- 172.21.15.92:0/3276116534 >> v2:172.21.15.124:3300/0 conn(0x7fd090068cd0 0x7fd0900abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.087+0000 7fd096f71700 1 --2- 172.21.15.92:0/3276116534 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd090156280 0x7fd09015a060 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.087+0000 7fd096f71700 1 -- 172.21.15.92:0/3276116534 >> 172.21.15.92:0/3276116534 conn(0x7fd09006de40 msgr2=0x7fd09006e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.088+0000 7fd096f71700 1 -- 172.21.15.92:0/3276116534 shutdown_connections 2023-10-06T10:38:33.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.088+0000 7fd096f71700 1 -- 172.21.15.92:0/3276116534 wait complete. 2023-10-06T10:38:33.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.088+0000 7f4210c26700 1 Processor -- start 2023-10-06T10:38:33.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.088+0000 7f4210c26700 1 -- start start 2023-10-06T10:38:33.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.089+0000 7f48d429b700 1 Processor -- start 2023-10-06T10:38:33.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.089+0000 7f48d429b700 1 -- start start 2023-10-06T10:38:33.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f4210c26700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f420c068cd0 0x7f420c15a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f4210c26700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f420c156650 0x7f420c0abb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f4210c26700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f420c0c9700 con 0x7f420c156280 2023-10-06T10:38:33.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f4210c26700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f420c15b000 con 0x7f420c156650 2023-10-06T10:38:33.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f4210c26700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420c0ad810 con 0x7f420c068cd0 2023-10-06T10:38:33.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f420a59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f420c156280 0x7f420c0685c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59972/0 (socket says 172.21.15.92:59972) 2023-10-06T10:38:33.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f420a59c700 1 -- 172.21.15.92:0/2340450262 learned_addr learned my addr 172.21.15.92:0/2340450262 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:33.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f48d429b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f48cc0b25e0 0x7f48cc0c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f48d429b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48cc0b29b0 0x7f48cc15d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f48d429b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48cc159660 0x7f48cc156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f48d429b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc15b3a0 con 0x7f48cc0b29b0 2023-10-06T10:38:33.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f48d429b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc0ca210 con 0x7f48cc0b25e0 2023-10-06T10:38:33.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f48d429b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc15ad50 con 0x7f48cc159660 2023-10-06T10:38:33.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f48d1836700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48cc0b29b0 0x7f48cc15d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f48d1836700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48cc0b29b0 0x7f48cc15d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37982/0 (socket says 172.21.15.92:37982) 2023-10-06T10:38:33.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f48d1836700 1 -- 172.21.15.92:0/1874764120 learned_addr learned my addr 172.21.15.92:0/1874764120 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f48d2037700 1 --2- 172.21.15.92:0/1874764120 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f48cc0b25e0 0x7f48cc0c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7fd096f71700 1 Processor -- start 2023-10-06T10:38:33.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7fd096f71700 1 -- start start 2023-10-06T10:38:33.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f420ad9d700 1 --2- 172.21.15.92:0/2340450262 >> v2:172.21.15.124:3300/0 conn(0x7f420c156650 0x7f420c0abb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f4209d9b700 1 --2- 172.21.15.92:0/2340450262 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f420c068cd0 0x7f420c15a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f420959a700 1 -- 172.21.15.92:0/2340450262 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1833820199 0 0) 0x7f420c0c9700 con 0x7f420c156280 2023-10-06T10:38:33.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f420959a700 1 -- 172.21.15.92:0/2340450262 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41f8003570 con 0x7f420c156280 2023-10-06T10:38:33.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f48d2838700 1 --2- 172.21.15.92:0/1874764120 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48cc159660 0x7f48cc156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f48d1836700 1 -- 172.21.15.92:0/1874764120 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48cc159660 msgr2=0x7f48cc156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f48d1836700 1 --2- 172.21.15.92:0/1874764120 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48cc159660 0x7f48cc156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f48d1836700 1 -- 172.21.15.92:0/1874764120 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f48cc0b25e0 msgr2=0x7f48cc0c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f48d1836700 1 --2- 172.21.15.92:0/1874764120 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f48cc0b25e0 0x7f48cc0c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f48d1836700 1 -- 172.21.15.92:0/1874764120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48bc009420 con 0x7f48cc0b29b0 2023-10-06T10:38:33.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f420ad9d700 1 -- 172.21.15.92:0/2340450262 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f420c068cd0 msgr2=0x7f420c15a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:33.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f420ad9d700 1 --2- 172.21.15.92:0/2340450262 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f420c068cd0 0x7f420c15a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f420ad9d700 1 -- 172.21.15.92:0/2340450262 >> v1:172.21.15.92:6789/0 conn(0x7f420c156280 legacy=0x7f420c0685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f420ad9d700 1 -- 172.21.15.92:0/2340450262 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420c15a760 con 0x7f420c156650 2023-10-06T10:38:33.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f48d2037700 1 --2- 172.21.15.92:0/1874764120 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f48cc0b25e0 0x7f48cc0c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:33.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.091+0000 7f48d1836700 1 --2- 172.21.15.92:0/1874764120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48cc0b29b0 0x7f48cc15d1c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00ec80 tx=0x7f48bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.091+0000 7f8f96e64700 1 Processor -- start 2023-10-06T10:38:33.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.091+0000 7f48c27fc700 1 -- 172.21.15.92:0/1874764120 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc00c920 con 0x7f48cc0b29b0 2023-10-06T10:38:33.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.091+0000 7f48c27fc700 1 -- 172.21.15.92:0/1874764120 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48bc00ca80 con 0x7f48cc0b29b0 2023-10-06T10:38:33.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.091+0000 7f48c27fc700 1 -- 172.21.15.92:0/1874764120 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc00ce00 con 0x7f48cc0b29b0 2023-10-06T10:38:33.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f959ca4b700 1 Processor -- start 2023-10-06T10:38:33.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.090+0000 7f959ca4b700 1 -- start start 2023-10-06T10:38:33.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7f420ad9d700 1 --2- 172.21.15.92:0/2340450262 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f420c156650 0x7f420c0abb10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f41f40098e0 tx=0x7f41f40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e51991154ea56c34 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7f48d429b700 1 -- 172.21.15.92:0/1874764120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c4009a20 con 0x7f48cc0b29b0 2023-10-06T10:38:33.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7f48d429b700 1 -- 172.21.15.92:0/1874764120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48cc0c8570 con 0x7f48cc0b29b0 2023-10-06T10:38:33.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.091+0000 7fd096f71700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd090068cd0 0x7fd090151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.091+0000 7fd096f71700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd090156280 0x7fd09014cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.091+0000 7fd096f71700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd090156650 0x7fd090154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.091+0000 7fd096f71700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0901517b0 con 0x7fd090156650 2023-10-06T10:38:33.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.091+0000 7fd096f71700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09014f840 con 0x7fd090156280 2023-10-06T10:38:33.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.091+0000 7fd096f71700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09014fb20 con 0x7fd090068cd0 2023-10-06T10:38:33.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.091+0000 7fd08ffff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd090156280 0x7fd09014cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.091+0000 7fd08ffff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd090156280 0x7fd09014cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:54490/0 (socket says 172.21.15.92:54490) 2023-10-06T10:38:33.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.091+0000 7fd08ffff700 1 -- 172.21.15.92:0/4101451074 learned_addr learned my addr 172.21.15.92:0/4101451074 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7fd08ffff700 1 -- 172.21.15.92:0/4101451074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd090068cd0 msgr2=0x7fd090151270 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T10:38:33.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7fd08ffff700 1 --2- 172.21.15.92:0/4101451074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd090068cd0 0x7fd090151270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7fd08ffff700 1 -- 172.21.15.92:0/4101451074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd090156650 msgr2=0x7fd090154b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7fd08ffff700 1 --2- 172.21.15.92:0/4101451074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd090156650 0x7fd090154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7fd08ffff700 1 -- 172.21.15.92:0/4101451074 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd088009a20 con 0x7fd090156280 2023-10-06T10:38:33.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7f420959a700 1 -- 172.21.15.92:0/2340450262 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f4011630 con 0x7f420c156650 2023-10-06T10:38:33.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7f420959a700 1 -- 172.21.15.92:0/2340450262 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f4011790 con 0x7f420c156650 2023-10-06T10:38:33.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7effb59eb700 1 Processor -- start 2023-10-06T10:38:33.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7f9afd9b0700 1 Processor -- start 2023-10-06T10:38:33.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7f9afd9b0700 1 -- start start 2023-10-06T10:38:33.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.093+0000 7f48d429b700 1 -- 172.21.15.92:0/1874764120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f48cc064a90 con 0x7f48cc0b29b0 2023-10-06T10:38:33.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7f420959a700 1 -- 172.21.15.92:0/2340450262 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f4011b30 con 0x7f420c156650 2023-10-06T10:38:33.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.093+0000 7f4210c26700 1 -- 172.21.15.92:0/2340450262 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f420c156650 msgr2=0x7f420c0abb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.093+0000 7f4210c26700 1 --2- 172.21.15.92:0/2340450262 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f420c156650 0x7f420c0abb10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f41f40098e0 tx=0x7f41f40091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.093+0000 7f4210c26700 1 -- 172.21.15.92:0/2340450262 shutdown_connections 2023-10-06T10:38:33.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.093+0000 7f4210c26700 1 --2- 172.21.15.92:0/2340450262 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f420c156650 0x7f420c0abb10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.093+0000 7f4210c26700 1 --2- 172.21.15.92:0/2340450262 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f420c068cd0 0x7f420c15a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.093+0000 7f4210c26700 1 -- 172.21.15.92:0/2340450262 >> 172.21.15.92:0/2340450262 conn(0x7f420c06de40 msgr2=0x7f420c06e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7effb59eb700 1 -- start start 2023-10-06T10:38:33.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7f9afd9b0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af80cc8e0 0x7f9af806bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7f9afd9b0700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9af806c160 0x7f9af80c8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7f9afd9b0700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af806c9a0 con 0x7f9af8156350 2023-10-06T10:38:33.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7f9afd9b0700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af8156b20 con 0x7f9af806c160 2023-10-06T10:38:33.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7f9afd9b0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af8156e00 con 0x7f9af80cc8e0 2023-10-06T10:38:33.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.093+0000 7fd08ffff700 1 --2- 172.21.15.92:0/4101451074 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd090156280 0x7fd09014cd10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00c590 tx=0x7fd07c007f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.093+0000 7fd08cff9700 1 -- 172.21.15.92:0/4101451074 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c005a70 con 0x7fd090156280 2023-10-06T10:38:33.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.093+0000 7fd08cff9700 1 -- 172.21.15.92:0/4101451074 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c005bd0 con 0x7fd090156280 2023-10-06T10:38:33.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.093+0000 7fd08cff9700 1 -- 172.21.15.92:0/4101451074 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0070d0 con 0x7fd090156280 2023-10-06T10:38:33.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.093+0000 7fd096f71700 1 -- 172.21.15.92:0/4101451074 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd084009480 con 0x7fd090156280 2023-10-06T10:38:33.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.093+0000 7fd096f71700 1 -- 172.21.15.92:0/4101451074 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0901500e0 con 0x7fd090156280 2023-10-06T10:38:33.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7f8f96e64700 1 -- start start 2023-10-06T10:38:33.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7f8f96e64700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8f900c8260 0x7f8f90069520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7f8f96e64700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f900c7dd0 con 0x7f8f9006e3d0 2023-10-06T10:38:33.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7f8f96e64700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f900c71c0 con 0x7f8f90155820 2023-10-06T10:38:33.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7f8f96e64700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f900c74a0 con 0x7f8f900c8260 2023-10-06T10:38:33.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7f8f94c00700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f8f90155820 0x7f8f900ae480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:56874/0 (socket says 172.21.15.92:56874) 2023-10-06T10:38:33.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7f8f94c00700 1 -- 172.21.15.92:0/2898624366 learned_addr learned my addr 172.21.15.92:0/2898624366 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:33.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.093+0000 7effb59eb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7effb00685c0 0x7effb0068a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7effb59eb700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb015efd0 0x7effb015f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7effb59eb700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb015eb20 con 0x7effb015b200 2023-10-06T10:38:33.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7effb59eb700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb01611b0 con 0x7effb00685c0 2023-10-06T10:38:33.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7f959ca4b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9598068810 0x7f95980afb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7f959ca4b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9598068be0 0x7f9598155d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7effb59eb700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb0068fa0 con 0x7effb015efd0 2023-10-06T10:38:33.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7effb51ea700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb015efd0 0x7effb015f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7effb51ea700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb015efd0 0x7effb015f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:43836/0 (socket says 172.21.15.92:43836) 2023-10-06T10:38:33.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7effb51ea700 1 -- 172.21.15.92:0/3300569662 learned_addr learned my addr 172.21.15.92:0/3300569662 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7f959ca4b700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95980ac8d0 con 0x7f95980ac440 2023-10-06T10:38:33.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7f959ca4b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95980acbb0 con 0x7f9598068810 2023-10-06T10:38:33.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7f959ca4b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95980ace90 con 0x7f9598068be0 2023-10-06T10:38:33.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7f9595d9b700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f95980ac440 0x7f9598155600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:56864/0 (socket says 172.21.15.92:56864) 2023-10-06T10:38:33.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7f9595d9b700 1 -- 172.21.15.92:0/2812614687 learned_addr learned my addr 172.21.15.92:0/2812614687 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:33.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.092+0000 7f9596d9d700 1 --2- 172.21.15.92:0/2812614687 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9598068be0 0x7f9598155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.093+0000 7f959659c700 1 --2- 172.21.15.92:0/2812614687 >> v2:172.21.15.92:3300/0 conn(0x7f9598068810 0x7f95980afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.093+0000 7f959659c700 1 -- 172.21.15.92:0/2812614687 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9598068be0 msgr2=0x7f9598155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:33.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.093+0000 7f959659c700 1 --2- 172.21.15.92:0/2812614687 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9598068be0 0x7f9598155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.093+0000 7f959659c700 1 -- 172.21.15.92:0/2812614687 >> v1:172.21.15.124:6789/0 conn(0x7f95980ac440 legacy=0x7f9598155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:33.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.093+0000 7f959659c700 1 -- 172.21.15.92:0/2812614687 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95980ad170 con 0x7f9598068810 2023-10-06T10:38:33.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7f959659c700 1 --2- 172.21.15.92:0/2812614687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9598068810 0x7f95980afb40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f95880098d0 tx=0x7f9588009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8c401885576f84 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7f9af6ffd700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f9af8156350 0x7f9af80cc1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:56896/0 (socket says 172.21.15.92:56896) 2023-10-06T10:38:33.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7f9af6ffd700 1 -- 172.21.15.92:0/316278468 learned_addr learned my addr 172.21.15.92:0/316278468 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:33.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7effaffff700 1 --2- 172.21.15.92:0/3300569662 >> v2:172.21.15.92:3300/0 conn(0x7effb00685c0 0x7effb0068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7f8f8f7fe700 1 -- 172.21.15.92:0/2898624366 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1640804952 0 0) 0x7f8f900c71c0 con 0x7f8f90155820 2023-10-06T10:38:33.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7f8f8f7fe700 1 -- 172.21.15.92:0/2898624366 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f80003570 con 0x7f8f90155820 2023-10-06T10:38:33.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7f9af5ffb700 1 -- 172.21.15.92:0/316278468 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3451024658 0 0) 0x7f9af806c9a0 con 0x7f9af8156350 2023-10-06T10:38:33.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7f9af5ffb700 1 -- 172.21.15.92:0/316278468 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ae0003570 con 0x7f9af8156350 2023-10-06T10:38:33.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7effaffff700 1 -- 172.21.15.92:0/3300569662 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb015efd0 msgr2=0x7effb015f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:33.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7effaffff700 1 --2- 172.21.15.92:0/3300569662 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb015efd0 0x7effb015f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7effaffff700 1 -- 172.21.15.92:0/3300569662 >> v1:172.21.15.124:6789/0 conn(0x7effb015b200 legacy=0x7effb006e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:33.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7effaffff700 1 -- 172.21.15.92:0/3300569662 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb006ed40 con 0x7effb00685c0 2023-10-06T10:38:33.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f9af77fe700 1 --2- 172.21.15.92:0/316278468 >> v2:172.21.15.124:3300/0 conn(0x7f9af806c160 0x7f9af80c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f8f8f7fe700 1 -- 172.21.15.92:0/2898624366 <== mon.1 v1:172.21.15.124:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3215414130 0 0) 0x7f8f80003570 con 0x7f8f90155820 2023-10-06T10:38:33.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f8f8f7fe700 1 -- 172.21.15.92:0/2898624366 >> v2:172.21.15.124:3300/0 conn(0x7f8f900c8260 msgr2=0x7f8f90069520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T10:38:33.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f8f8f7fe700 1 --2- 172.21.15.92:0/2898624366 >> v2:172.21.15.124:3300/0 conn(0x7f8f900c8260 0x7f8f90069520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f8f8f7fe700 1 -- 172.21.15.92:0/2898624366 >> v1:172.21.15.92:6789/0 conn(0x7f8f9006e3d0 legacy=0x7f8f900abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f9af5ffb700 1 -- 172.21.15.92:0/316278468 <== mon.1 v1:172.21.15.124:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1476204458 0 0) 0x7f9ae0003570 con 0x7f9af8156350 2023-10-06T10:38:33.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f9af67fc700 1 --2- 172.21.15.92:0/316278468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af80cc8e0 0x7f9af806bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f9af5ffb700 1 -- 172.21.15.92:0/316278468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af80cc8e0 msgr2=0x7f9af806bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:33.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7effaffff700 1 --2- 172.21.15.92:0/3300569662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effb00685c0 0x7effb0068a60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7effa40098e0 tx=0x7effa40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de5e2b70778f748 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f9af5ffb700 1 --2- 172.21.15.92:0/316278468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af80cc8e0 0x7f9af806bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f9af5ffb700 1 -- 172.21.15.92:0/316278468 >> v2:172.21.15.124:3300/0 conn(0x7f9af806c160 msgr2=0x7f9af80c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:33.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f9af5ffb700 1 --2- 172.21.15.92:0/316278468 >> v2:172.21.15.124:3300/0 conn(0x7f9af806c160 0x7f9af80c8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f9af5ffb700 1 -- 172.21.15.92:0/316278468 --> v1:172.21.15.124:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af81570e0 con 0x7f9af8156350 2023-10-06T10:38:33.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f9af5ffb700 1 -- 172.21.15.92:0/316278468 <== mon.1 v1:172.21.15.124:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f9af00017f0 con 0x7f9af8156350 2023-10-06T10:38:33.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f9af67fc700 1 --2- 172.21.15.92:0/316278468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af80cc8e0 0x7f9af806bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:33.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f8f8f7fe700 1 -- 172.21.15.92:0/2898624366 --> v1:172.21.15.124:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f900c7780 con 0x7f8f90155820 2023-10-06T10:38:33.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f8f8f7fe700 1 -- 172.21.15.92:0/2898624366 <== mon.1 v1:172.21.15.124:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f8f7c001470 con 0x7f8f90155820 2023-10-06T10:38:33.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f8f8f7fe700 1 -- 172.21.15.92:0/2898624366 >> v1:172.21.15.124:6789/0 conn(0x7f8f90155820 legacy=0x7f8f900ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f8f8f7fe700 1 --2- 172.21.15.92:0/2898624366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f80003a60 0x7f8f80003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f8f8f7fe700 1 --2- 172.21.15.92:0/2898624366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f80004810 0x7f8f80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f8f8f7fe700 1 --2- 172.21.15.92:0/2898624366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f800052e0 0x7f8f8000cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f8f8f7fe700 1 -- 172.21.15.92:0/2898624366 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900c74a0 con 0x7f8f800052e0 2023-10-06T10:38:33.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f8f8f7fe700 1 -- 172.21.15.92:0/2898624366 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8000d2f0 con 0x7f8f80003a60 2023-10-06T10:38:33.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f8f8f7fe700 1 -- 172.21.15.92:0/2898624366 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8000d4b0 con 0x7f8f80004810 2023-10-06T10:38:33.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f9af5ffb700 1 -- 172.21.15.92:0/316278468 >> v1:172.21.15.124:6789/0 conn(0x7f9af8156350 legacy=0x7f9af80cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f9af5ffb700 1 --2- 172.21.15.92:0/316278468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af80cc8e0 0x7f9ae0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f9af5ffb700 1 --2- 172.21.15.92:0/316278468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0003fa0 0x7f9ae0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f9af5ffb700 1 --2- 172.21.15.92:0/316278468 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ae0004d50 0x7f9ae00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f9af5ffb700 1 -- 172.21.15.92:0/316278468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af8156b20 con 0x7f9ae0003fa0 2023-10-06T10:38:33.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f9af5ffb700 1 -- 172.21.15.92:0/316278468 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af8156e00 con 0x7f9ae0004d50 2023-10-06T10:38:33.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f9af5ffb700 1 -- 172.21.15.92:0/316278468 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae0009c50 con 0x7f9af80cc8e0 2023-10-06T10:38:33.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f9af67fc700 1 --2- 172.21.15.92:0/316278468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0003fa0 0x7f9ae0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f9af6ffd700 1 --2- 172.21.15.92:0/316278468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af80cc8e0 0x7f9ae0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.094+0000 7f48c27fc700 1 -- 172.21.15.92:0/1874764120 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f48bc013680 con 0x7f48cc0b29b0 2023-10-06T10:38:33.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f48c27fc700 1 --2- 172.21.15.92:0/1874764120 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f48b0066fb0 0x7f48b0069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f48c27fc700 1 -- 172.21.15.92:0/1874764120 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f48bc084df0 con 0x7f48cc0b29b0 2023-10-06T10:38:33.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f48c27fc700 1 --2- 172.21.15.92:0/1874764120 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x7f48b0070c80 0x7f48b00730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f48c27fc700 1 -- 172.21.15.92:0/1874764120 --> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f48b0073780 con 0x7f48b0070c80 2023-10-06T10:38:33.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f48c27fc700 1 -- 172.21.15.92:0/1874764120 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48bc087390 con 0x7f48cc0b29b0 2023-10-06T10:38:33.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.095+0000 7f9af77fe700 1 --2- 172.21.15.92:0/316278468 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ae0004d50 0x7f9ae00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f9af67fc700 1 -- 172.21.15.92:0/316278468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af80cc8e0 msgr2=0x7f9ae0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:33.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f9af67fc700 1 --2- 172.21.15.92:0/316278468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af80cc8e0 0x7f9ae0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f9af67fc700 1 -- 172.21.15.92:0/316278468 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ae0004d50 msgr2=0x7f9ae00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:33.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f9af67fc700 1 --2- 172.21.15.92:0/316278468 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ae0004d50 0x7f9ae00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7fd08cff9700 1 -- 172.21.15.92:0/4101451074 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd07c007480 con 0x7fd090156280 2023-10-06T10:38:33.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7fd096f71700 1 -- 172.21.15.92:0/4101451074 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd070000fd0 con 0x7fd090156280 2023-10-06T10:38:33.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f8f95401700 1 --2- 172.21.15.92:0/2898624366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f800052e0 0x7f8f8000cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f8f94c00700 1 --2- 172.21.15.92:0/2898624366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f80003a60 0x7f8f80003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f4210c26700 1 -- 172.21.15.92:0/2340450262 shutdown_connections 2023-10-06T10:38:33.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7effaf7fe700 1 -- 172.21.15.92:0/3300569662 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa4009e80 con 0x7effb00685c0 2023-10-06T10:38:33.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7effaf7fe700 1 -- 172.21.15.92:0/3300569662 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa400b820 con 0x7effb00685c0 2023-10-06T10:38:33.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f4210c26700 1 -- 172.21.15.92:0/2340450262 wait complete. 2023-10-06T10:38:33.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7effaf7fe700 1 -- 172.21.15.92:0/3300569662 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa400bba0 con 0x7effb00685c0 2023-10-06T10:38:33.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f9af67fc700 1 -- 172.21.15.92:0/316278468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae0003570 con 0x7f9ae0003fa0 2023-10-06T10:38:33.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f9af6ffd700 1 --2- 172.21.15.92:0/316278468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af80cc8e0 0x7f9ae0003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:33.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f959559a700 1 -- 172.21.15.92:0/2812614687 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9588009e70 con 0x7f9598068810 2023-10-06T10:38:33.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f959559a700 1 -- 172.21.15.92:0/2812614687 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9588005c10 con 0x7f9598068810 2023-10-06T10:38:33.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f959559a700 1 -- 172.21.15.92:0/2812614687 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958800b610 con 0x7f9598068810 2023-10-06T10:38:33.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f9af67fc700 1 --2- 172.21.15.92:0/316278468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0003fa0 0x7f9ae0004810 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00abf0 tx=0x7f9aec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf3125ea0ac06f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7fd08cff9700 1 --2- 172.21.15.92:0/4101451074 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd074066fb0 0x7fd074069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7fd08cff9700 1 -- 172.21.15.92:0/4101451074 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fd07c083f20 con 0x7fd090156280 2023-10-06T10:38:33.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7fd08cff9700 1 --2- 172.21.15.92:0/4101451074 >> [v2:172.21.15.92:6808/90605,v1:172.21.15.92:6809/90605] conn(0x7fd074070c80 0x7fd0740730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7fd08cff9700 1 -- 172.21.15.92:0/4101451074 --> [v2:172.21.15.92:6808/90605,v1:172.21.15.92:6809/90605] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd074073780 con 0x7fd074070c80 2023-10-06T10:38:33.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7fd08cff9700 1 -- 172.21.15.92:0/4101451074 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd07c089430 con 0x7fd090156280 2023-10-06T10:38:33.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7fd09550e700 1 --2- 172.21.15.92:0/4101451074 >> [v2:172.21.15.92:6808/90605,v1:172.21.15.92:6809/90605] conn(0x7fd074070c80 0x7fd0740730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f48d2838700 1 --2- 172.21.15.92:0/1874764120 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x7f48b0070c80 0x7f48b00730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f48d2037700 1 --2- 172.21.15.92:0/1874764120 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f48b0066fb0 0x7f48b0069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.097+0000 7fd094d0d700 1 --2- 172.21.15.92:0/4101451074 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd074066fb0 0x7fd074069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.097+0000 7effb59eb700 1 -- 172.21.15.92:0/3300569662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effb00685c0 msgr2=0x7effb0068a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.097+0000 7effb59eb700 1 --2- 172.21.15.92:0/3300569662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effb00685c0 0x7effb0068a60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7effa40098e0 tx=0x7effa40091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.097+0000 7f9af5ffb700 1 -- 172.21.15.92:0/316278468 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec00a530 con 0x7f9ae0003fa0 2023-10-06T10:38:33.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.097+0000 7effb59eb700 1 -- 172.21.15.92:0/3300569662 shutdown_connections 2023-10-06T10:38:33.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.097+0000 7effb59eb700 1 --2- 172.21.15.92:0/3300569662 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb015efd0 0x7effb015f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.097+0000 7effb59eb700 1 --2- 172.21.15.92:0/3300569662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effb00685c0 0x7effb0068a60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.097+0000 7effb59eb700 1 -- 172.21.15.92:0/3300569662 >> 172.21.15.92:0/3300569662 conn(0x7effb00c9c10 msgr2=0x7effb006dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.097+0000 7fd094d0d700 1 --2- 172.21.15.92:0/4101451074 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd074066fb0 0x7fd074069460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd084009260 tx=0x7fd084016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f8f95401700 1 -- 172.21.15.92:0/2898624366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f80004810 msgr2=0x7f8f80004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T10:38:33.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f8f95401700 1 --2- 172.21.15.92:0/2898624366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f80004810 0x7f8f80004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f8f95401700 1 -- 172.21.15.92:0/2898624366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f80003a60 msgr2=0x7f8f80003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:33.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f8f95401700 1 --2- 172.21.15.92:0/2898624366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f80003a60 0x7f8f80003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f8f95401700 1 -- 172.21.15.92:0/2898624366 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f900c7780 con 0x7f8f800052e0 2023-10-06T10:38:33.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f8f95401700 1 -- 172.21.15.92:0/2898624366 reap_dead start 2023-10-06T10:38:33.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.096+0000 7f8f95401700 1 --2- 172.21.15.92:0/2898624366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f800052e0 0x7f8f8000cb70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8f90070190 tx=0x7f8f88001ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc247b03cd820d75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.097+0000 7f8f8f7fe700 1 -- 172.21.15.92:0/2898624366 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8800ee90 con 0x7f8f800052e0 2023-10-06T10:38:33.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.097+0000 7f8f8f7fe700 1 -- 172.21.15.92:0/2898624366 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f88011470 con 0x7f8f800052e0 2023-10-06T10:38:33.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.097+0000 7f8f8f7fe700 1 -- 172.21.15.92:0/2898624366 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88011830 con 0x7f8f800052e0 2023-10-06T10:38:33.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.097+0000 7f4210c26700 1 Processor -- start 2023-10-06T10:38:33.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.097+0000 7f4210c26700 1 -- start start 2023-10-06T10:38:33.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.097+0000 7effb59eb700 1 -- 172.21.15.92:0/3300569662 shutdown_connections 2023-10-06T10:38:33.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.097+0000 7f9af5ffb700 1 -- 172.21.15.92:0/316278468 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aec00c6d0 con 0x7f9ae0003fa0 2023-10-06T10:38:33.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.097+0000 7f9af5ffb700 1 -- 172.21.15.92:0/316278468 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec00ca70 con 0x7f9ae0003fa0 2023-10-06T10:38:33.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.098+0000 7f9afd9b0700 1 -- 172.21.15.92:0/316278468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0003fa0 msgr2=0x7f9ae0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.098+0000 7f9afd9b0700 1 --2- 172.21.15.92:0/316278468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0003fa0 0x7f9ae0004810 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00abf0 tx=0x7f9aec008040 comp rx=0 tx=0).stop 2023-10-06T10:38:33.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.098+0000 7f9af77fe700 1 -- 172.21.15.92:0/316278468 reap_dead start 2023-10-06T10:38:33.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.098+0000 7effb59eb700 1 -- 172.21.15.92:0/3300569662 wait complete. 2023-10-06T10:38:33.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.098+0000 7f9afd9b0700 1 -- 172.21.15.92:0/316278468 shutdown_connections 2023-10-06T10:38:33.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.098+0000 7f9afd9b0700 1 -- 172.21.15.92:0/316278468 >> 172.21.15.92:0/316278468 conn(0x7f9af80cdeb0 msgr2=0x7f9af80c9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.098+0000 7f959ca4b700 1 -- 172.21.15.92:0/2812614687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9598068810 msgr2=0x7f95980afb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.098+0000 7f959ca4b700 1 --2- 172.21.15.92:0/2812614687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9598068810 0x7f95980afb40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f95880098d0 tx=0x7f9588009120 comp rx=0 tx=0).stop 2023-10-06T10:38:33.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.098+0000 7f8f96e64700 1 -- 172.21.15.92:0/2898624366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f800052e0 msgr2=0x7f8f8000cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.098+0000 7f8f96e64700 1 --2- 172.21.15.92:0/2898624366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f800052e0 0x7f8f8000cb70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8f90070190 tx=0x7f8f88001ba0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.098+0000 7f8f96e64700 1 -- 172.21.15.92:0/2898624366 shutdown_connections 2023-10-06T10:38:33.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.098+0000 7f8f96e64700 1 --2- 172.21.15.92:0/2898624366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f800052e0 0x7f8f8000cb70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.098+0000 7f8f96e64700 1 -- 172.21.15.92:0/2898624366 >> 172.21.15.92:0/2898624366 conn(0x7f8f90068080 msgr2=0x7f8f900cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.098+0000 7f9afd9b0700 1 -- 172.21.15.92:0/316278468 shutdown_connections 2023-10-06T10:38:33.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.098+0000 7f9afd9b0700 1 -- 172.21.15.92:0/316278468 wait complete. 2023-10-06T10:38:33.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.098+0000 7f959ca4b700 1 -- 172.21.15.92:0/2812614687 shutdown_connections 2023-10-06T10:38:33.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.098+0000 7f959ca4b700 1 --2- 172.21.15.92:0/2812614687 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9598068be0 0x7f9598155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.098+0000 7f959ca4b700 1 --2- 172.21.15.92:0/2812614687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9598068810 0x7f95980afb40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.098+0000 7f959ca4b700 1 -- 172.21.15.92:0/2812614687 >> 172.21.15.92:0/2812614687 conn(0x7f959806de40 msgr2=0x7f959806e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.098+0000 7fd09550e700 1 --2- 172.21.15.92:0/4101451074 >> [v2:172.21.15.92:6808/90605,v1:172.21.15.92:6809/90605] conn(0x7fd074070c80 0x7fd0740730b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.099+0000 7f4210c26700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f420c068cd0 0x7f420c0ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.099+0000 7f4210c26700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f420c156280 0x7f420c171700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.099+0000 7f4210c26700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f420c156650 0x7f420c14ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.099+0000 7f4210c26700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c0ca4b0 con 0x7f420c068cd0 2023-10-06T10:38:33.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.099+0000 7f4210c26700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c0cae80 con 0x7f420c156280 2023-10-06T10:38:33.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.099+0000 7f4210c26700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420c0c98a0 con 0x7f420c156650 2023-10-06T10:38:33.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.099+0000 7f420a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f420c068cd0 0x7f420c0ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.099+0000 7f420a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f420c068cd0 0x7f420c0ca940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38032/0 (socket says 172.21.15.92:38032) 2023-10-06T10:38:33.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.099+0000 7f420a59c700 1 -- 172.21.15.92:0/2122361083 learned_addr learned my addr 172.21.15.92:0/2122361083 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.099+0000 7f4209d9b700 1 --2- 172.21.15.92:0/2122361083 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f420c156280 0x7f420c171700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.099+0000 7f420ad9d700 1 --2- 172.21.15.92:0/2122361083 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f420c156650 0x7f420c14ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.099+0000 7fd08cff9700 1 -- 172.21.15.92:0/4101451074 <== osd.3 v2:172.21.15.92:6808/90605 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fd074073780 con 0x7fd074070c80 2023-10-06T10:38:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.099+0000 7effb59eb700 1 Processor -- start 2023-10-06T10:38:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.099+0000 7f420a59c700 1 -- 172.21.15.92:0/2122361083 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f420c156650 msgr2=0x7f420c14ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.099+0000 7f420a59c700 1 --2- 172.21.15.92:0/2122361083 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f420c156650 0x7f420c14ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.099+0000 7f420a59c700 1 -- 172.21.15.92:0/2122361083 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f420c156280 msgr2=0x7f420c171700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.099+0000 7f420a59c700 1 --2- 172.21.15.92:0/2122361083 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f420c156280 0x7f420c171700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.099+0000 7f420a59c700 1 -- 172.21.15.92:0/2122361083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f4009420 con 0x7f420c068cd0 2023-10-06T10:38:33.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.099+0000 7f4209d9b700 1 --2- 172.21.15.92:0/2122361083 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f420c156280 0x7f420c171700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:33.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.099+0000 7f420a59c700 1 --2- 172.21.15.92:0/2122361083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f420c068cd0 0x7f420c0ca940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00cb10 tx=0x7f41fc0027e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.097+0000 7f48d2037700 1 --2- 172.21.15.92:0/1874764120 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f48b0066fb0 0x7f48b0069460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f48c8005920 tx=0x7f48c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.099+0000 7effb59eb700 1 -- start start 2023-10-06T10:38:33.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.100+0000 7f959ca4b700 1 -- 172.21.15.92:0/2812614687 shutdown_connections 2023-10-06T10:38:33.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.100+0000 7f9afd9b0700 1 Processor -- start 2023-10-06T10:38:33.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.100+0000 7f8f96e64700 1 -- 172.21.15.92:0/2898624366 shutdown_connections 2023-10-06T10:38:33.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.100+0000 7f959ca4b700 1 -- 172.21.15.92:0/2812614687 wait complete. 2023-10-06T10:38:33.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.100+0000 7f9afd9b0700 1 -- start start 2023-10-06T10:38:33.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.100+0000 7f48d2838700 1 --2- 172.21.15.92:0/1874764120 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x7f48b0070c80 0x7f48b00730b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.100+0000 7f8f96e64700 1 -- 172.21.15.92:0/2898624366 wait complete. 2023-10-06T10:38:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.100+0000 7effb59eb700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb00685c0 0x7effb00ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.100+0000 7effb59eb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effb015b200 0x7effb00c3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.100+0000 7effb59eb700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effb015efd0 0x7effb00c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.100+0000 7effb59eb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb00c4ed0 con 0x7effb015b200 2023-10-06T10:38:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.100+0000 7effb59eb700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb00b2250 con 0x7effb015efd0 2023-10-06T10:38:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.100+0000 7effb59eb700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb00c42c0 con 0x7effb00685c0 2023-10-06T10:38:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.100+0000 7f42027fc700 1 -- 172.21.15.92:0/2122361083 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc00ce20 con 0x7f420c068cd0 2023-10-06T10:38:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.101+0000 7effb51ea700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effb015efd0 0x7effb00c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.101+0000 7effb51ea700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effb015efd0 0x7effb00c3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:54564/0 (socket says 172.21.15.92:54564) 2023-10-06T10:38:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.101+0000 7effb51ea700 1 -- 172.21.15.92:0/3325554716 learned_addr learned my addr 172.21.15.92:0/3325554716 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.101+0000 7f959ca4b700 1 Processor -- start 2023-10-06T10:38:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.101+0000 7f9afd9b0700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ae0003fa0 0x7f9af814cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.101+0000 7f9afd9b0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9af8166170 0x7f9af80c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.101+0000 7f9afd9b0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af80c70d0 0x7f9af8154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.101+0000 7f9afd9b0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af814bb40 con 0x7f9af8166170 2023-10-06T10:38:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.101+0000 7f9afd9b0700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af814bd70 con 0x7f9ae0003fa0 2023-10-06T10:38:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.101+0000 7f9afd9b0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af814c050 con 0x7f9af80c70d0 2023-10-06T10:38:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.101+0000 7f9af67fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9af8166170 0x7f9af80c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7effb49e9700 1 --2- 172.21.15.92:0/3325554716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb00685c0 0x7effb00ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.101+0000 7f9af67fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9af8166170 0x7f9af80c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38036/0 (socket says 172.21.15.92:38036) 2023-10-06T10:38:33.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.101+0000 7f9af67fc700 1 -- 172.21.15.92:0/869372176 learned_addr learned my addr 172.21.15.92:0/869372176 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.101+0000 7f9af77fe700 1 --2- 172.21.15.92:0/869372176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af80c70d0 0x7f9af8154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.101+0000 7f9af6ffd700 1 --2- 172.21.15.92:0/869372176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ae0003fa0 0x7f9af814cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.101+0000 7f959ca4b700 1 -- start start 2023-10-06T10:38:33.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.101+0000 7f8f96e64700 1 Processor -- start 2023-10-06T10:38:33.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.100+0000 7f42027fc700 1 -- 172.21.15.92:0/2122361083 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41fc005d80 con 0x7f420c068cd0 2023-10-06T10:38:33.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.101+0000 7f42027fc700 1 -- 172.21.15.92:0/2122361083 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc00a820 con 0x7f420c068cd0 2023-10-06T10:38:33.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.101+0000 7f4210c26700 1 -- 172.21.15.92:0/2122361083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f420c0c9b80 con 0x7f420c068cd0 2023-10-06T10:38:33.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.101+0000 7f4210c26700 1 -- 172.21.15.92:0/2122361083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420c0ca140 con 0x7f420c068cd0 2023-10-06T10:38:33.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.101+0000 7f48c27fc700 1 -- 172.21.15.92:0/1874764120 <== osd.6 v2:172.21.15.124:6824/90914 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f48b0073780 con 0x7f48b0070c80 2023-10-06T10:38:33.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7f959ca4b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9598068810 0x7f9598162d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7f959ca4b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9598068be0 0x7f95980a5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7f959ca4b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95980ac440 0x7f9598168cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7f959ca4b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95980a7b90 con 0x7f9598068810 2023-10-06T10:38:33.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7f959ca4b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95980a6f80 con 0x7f95980ac440 2023-10-06T10:38:33.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7f959ca4b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95980a7260 con 0x7f9598068be0 2023-10-06T10:38:33.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7effb49e9700 1 -- 172.21.15.92:0/3325554716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effb015efd0 msgr2=0x7effb00c3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7effb49e9700 1 --2- 172.21.15.92:0/3325554716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effb015efd0 0x7effb00c3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7effb49e9700 1 -- 172.21.15.92:0/3325554716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effb015b200 msgr2=0x7effb00c3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7effb49e9700 1 --2- 172.21.15.92:0/3325554716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effb015b200 0x7effb00c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7effb49e9700 1 -- 172.21.15.92:0/3325554716 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa4009420 con 0x7effb00685c0 2023-10-06T10:38:33.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7f959659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9598068810 0x7f9598162d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7f959659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9598068810 0x7f9598162d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38040/0 (socket says 172.21.15.92:38040) 2023-10-06T10:38:33.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7f959659c700 1 -- 172.21.15.92:0/3107489805 learned_addr learned my addr 172.21.15.92:0/3107489805 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7f9af67fc700 1 -- 172.21.15.92:0/869372176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af80c70d0 msgr2=0x7f9af8154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7f9af67fc700 1 --2- 172.21.15.92:0/869372176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af80c70d0 0x7f9af8154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7f9af67fc700 1 -- 172.21.15.92:0/869372176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ae0003fa0 msgr2=0x7f9af814cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7f9af67fc700 1 --2- 172.21.15.92:0/869372176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ae0003fa0 0x7f9af814cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7f9af67fc700 1 -- 172.21.15.92:0/869372176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af814c330 con 0x7f9af8166170 2023-10-06T10:38:33.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7f9af77fe700 1 --2- 172.21.15.92:0/869372176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af80c70d0 0x7f9af8154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:33.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7f9596d9d700 1 --2- 172.21.15.92:0/3107489805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95980ac440 0x7f9598168cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7f959659c700 1 -- 172.21.15.92:0/3107489805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9598068be0 msgr2=0x7f95980a5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7f959659c700 1 --2- 172.21.15.92:0/3107489805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9598068be0 0x7f95980a5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7f959659c700 1 -- 172.21.15.92:0/3107489805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95980ac440 msgr2=0x7f9598168cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7f959659c700 1 --2- 172.21.15.92:0/3107489805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95980ac440 0x7f9598168cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7f959659c700 1 -- 172.21.15.92:0/3107489805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95880093c0 con 0x7f9598068810 2023-10-06T10:38:33.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.103+0000 7f9af67fc700 1 --2- 172.21.15.92:0/869372176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9af8166170 0x7f9af80c6b90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0004710 tx=0x7f9aec01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.103+0000 7f959659c700 1 --2- 172.21.15.92:0/3107489805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9598068810 0x7f9598162d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9588009160 tx=0x7f9588026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.103+0000 7effb49e9700 1 --2- 172.21.15.92:0/3325554716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb00685c0 0x7effb00ac1c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7effa800c8c0 tx=0x7effa80026a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.103+0000 7effacff9700 1 -- 172.21.15.92:0/3325554716 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8012630 con 0x7effb00685c0 2023-10-06T10:38:33.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.102+0000 7f8f96e64700 1 -- start start 2023-10-06T10:38:33.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.103+0000 7f8f96e64700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f800052e0 0x7f8f9014e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.103+0000 7f8f96e64700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f9014e990 0x7f8f901714b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.103+0000 7f8f96e64700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f901719f0 0x7f8f900b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.103+0000 7f8f96e64700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f90171e40 con 0x7f8f800052e0 2023-10-06T10:38:33.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.103+0000 7f8f96e64700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9014d3b0 con 0x7f8f901719f0 2023-10-06T10:38:33.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.103+0000 7f8f96e64700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9014d690 con 0x7f8f9014e990 2023-10-06T10:38:33.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.103+0000 7f8f94c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f800052e0 0x7f8f9014e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.103+0000 7f8f94c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f800052e0 0x7f8f9014e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38042/0 (socket says 172.21.15.92:38042) 2023-10-06T10:38:33.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.103+0000 7f8f94c00700 1 -- 172.21.15.92:0/1036361170 learned_addr learned my addr 172.21.15.92:0/1036361170 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.103+0000 7f8f95401700 1 --2- 172.21.15.92:0/1036361170 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f901719f0 0x7f8f900b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.103+0000 7f8f8ffff700 1 --2- 172.21.15.92:0/1036361170 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f9014e990 0x7f8f901714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.103+0000 7f95867fc700 1 -- 172.21.15.92:0/3107489805 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9588033070 con 0x7f9598068810 2023-10-06T10:38:33.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.104+0000 7f95867fc700 1 -- 172.21.15.92:0/3107489805 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9588022740 con 0x7f9598068810 2023-10-06T10:38:33.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.104+0000 7f959ca4b700 1 -- 172.21.15.92:0/3107489805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95980a7540 con 0x7f9598068810 2023-10-06T10:38:33.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.104+0000 7f95867fc700 1 -- 172.21.15.92:0/3107489805 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9588022b00 con 0x7f9598068810 2023-10-06T10:38:33.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.104+0000 7f959ca4b700 1 -- 172.21.15.92:0/3107489805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95980a6bc0 con 0x7f9598068810 2023-10-06T10:38:33.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.104+0000 7f8f95401700 1 -- 172.21.15.92:0/1036361170 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f9014e990 msgr2=0x7f8f901714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.104+0000 7f8f95401700 1 --2- 172.21.15.92:0/1036361170 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f9014e990 0x7f8f901714b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.104+0000 7f8f95401700 1 -- 172.21.15.92:0/1036361170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f800052e0 msgr2=0x7f8f9014e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.104+0000 7f8f95401700 1 --2- 172.21.15.92:0/1036361170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f800052e0 0x7f8f9014e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.104+0000 7f8f95401700 1 -- 172.21.15.92:0/1036361170 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9014d970 con 0x7f8f901719f0 2023-10-06T10:38:33.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.104+0000 7f8f95401700 1 --2- 172.21.15.92:0/1036361170 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f901719f0 0x7f8f900b11b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8f9006d140 tx=0x7f8f8801a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.103+0000 7effacff9700 1 -- 172.21.15.92:0/3325554716 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa80127b0 con 0x7effb00685c0 2023-10-06T10:38:33.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.103+0000 7effacff9700 1 -- 172.21.15.92:0/3325554716 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8012b70 con 0x7effb00685c0 2023-10-06T10:38:33.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.103+0000 7effb59eb700 1 -- 172.21.15.92:0/3325554716 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb00c45a0 con 0x7effb00685c0 2023-10-06T10:38:33.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.103+0000 7effb59eb700 1 -- 172.21.15.92:0/3325554716 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb00c4b60 con 0x7effb00685c0 2023-10-06T10:38:33.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.105+0000 7f9adeffd700 1 -- 172.21.15.92:0/869372176 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec000bc0 con 0x7f9af8166170 2023-10-06T10:38:33.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.105+0000 7f9adeffd700 1 -- 172.21.15.92:0/869372176 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aec00c840 con 0x7f9af8166170 2023-10-06T10:38:33.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.105+0000 7f9adeffd700 1 -- 172.21.15.92:0/869372176 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec000d20 con 0x7f9af8166170 2023-10-06T10:38:33.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.104+0000 7f8f8cff9700 1 -- 172.21.15.92:0/1036361170 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8800ee90 con 0x7f8f901719f0 2023-10-06T10:38:33.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.104+0000 7f8f8cff9700 1 -- 172.21.15.92:0/1036361170 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f88001320 con 0x7f8f901719f0 2023-10-06T10:38:33.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.104+0000 7f8f8cff9700 1 -- 172.21.15.92:0/1036361170 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88024690 con 0x7f8f901719f0 2023-10-06T10:38:33.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.104+0000 7f8f96e64700 1 -- 172.21.15.92:0/1036361170 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f9014dc50 con 0x7f8f901719f0 2023-10-06T10:38:33.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.104+0000 7f8f96e64700 1 -- 172.21.15.92:0/1036361170 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f900aefe0 con 0x7f8f901719f0 2023-10-06T10:38:33.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.105+0000 7f42027fc700 1 -- 172.21.15.92:0/2122361083 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f41fc00ac10 con 0x7f420c068cd0 2023-10-06T10:38:33.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.106+0000 7f42027fc700 1 --2- 172.21.15.92:0/2122361083 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f41ec067080 0x7f41ec069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.106+0000 7f42027fc700 1 -- 172.21.15.92:0/2122361083 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f41fc084670 con 0x7f420c068cd0 2023-10-06T10:38:33.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.106+0000 7f4209d9b700 1 --2- 172.21.15.92:0/2122361083 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f41ec067080 0x7f41ec069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.105+0000 7f9afd9b0700 1 -- 172.21.15.92:0/869372176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af814c610 con 0x7f9af8166170 2023-10-06T10:38:33.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.105+0000 7f9afd9b0700 1 -- 172.21.15.92:0/869372176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af81665e0 con 0x7f9af8166170 2023-10-06T10:38:33.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.106+0000 7f95867fc700 1 -- 172.21.15.92:0/3107489805 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f958802d450 con 0x7f9598068810 2023-10-06T10:38:33.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.106+0000 7f95867fc700 1 --2- 172.21.15.92:0/3107489805 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9580067120 0x7f95800695d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.106+0000 7f95867fc700 1 -- 172.21.15.92:0/3107489805 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9588097100 con 0x7f9598068810 2023-10-06T10:38:33.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.106+0000 7f9595d9b700 1 --2- 172.21.15.92:0/3107489805 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9580067120 0x7f95800695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.107+0000 7f4209d9b700 1 --2- 172.21.15.92:0/2122361083 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f41ec067080 0x7f41ec069530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f420400bfd0 tx=0x7f4204008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.106+0000 7f9afd9b0700 1 -- 172.21.15.92:0/869372176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9af814bee0 con 0x7f9af8166170 2023-10-06T10:38:33.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.107+0000 7f9595d9b700 1 --2- 172.21.15.92:0/3107489805 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9580067120 0x7f95800695d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f95981632c0 tx=0x7f958c006c60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.107+0000 7effacff9700 1 -- 172.21.15.92:0/3325554716 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7effa801a3b0 con 0x7effb00685c0 2023-10-06T10:38:33.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.108+0000 7effb59eb700 1 -- 172.21.15.92:0/3325554716 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7eff90000fd0 con 0x7effb00685c0 2023-10-06T10:38:33.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.108+0000 7f8f8cff9700 1 -- 172.21.15.92:0/1036361170 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8f88017030 con 0x7f8f901719f0 2023-10-06T10:38:33.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.108+0000 7f959ca4b700 1 --2- 172.21.15.92:0/3107489805 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x7f957c0015f0 0x7f957c003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.108+0000 7f959ca4b700 1 -- 172.21.15.92:0/3107489805 --> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f957c006b90 con 0x7f957c0015f0 2023-10-06T10:38:33.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.108+0000 7effacff9700 1 --2- 172.21.15.92:0/3325554716 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7eff9c066f30 0x7eff9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.108+0000 7effaffff700 1 --2- 172.21.15.92:0/3325554716 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7eff9c066f30 0x7eff9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.108+0000 7effacff9700 1 -- 172.21.15.92:0/3325554716 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7effa8083f80 con 0x7effb00685c0 2023-10-06T10:38:33.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.109+0000 7effacff9700 1 --2- 172.21.15.92:0/3325554716 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x7eff9c070b80 0x7eff9c072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.109+0000 7effacff9700 1 -- 172.21.15.92:0/3325554716 --> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7eff9c073680 con 0x7eff9c070b80 2023-10-06T10:38:33.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.108+0000 7f8f8cff9700 1 --2- 172.21.15.92:0/1036361170 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8f74067080 0x7f8f74069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.109+0000 7f8f8cff9700 1 -- 172.21.15.92:0/1036361170 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f8f88015070 con 0x7f8f901719f0 2023-10-06T10:38:33.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.109+0000 7f8f96e64700 1 --2- 172.21.15.92:0/1036361170 >> [v2:172.21.15.124:6800/90883,v1:172.21.15.124:6801/90883] conn(0x7f8f700015f0 0x7f8f70003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.109+0000 7f8f96e64700 1 -- 172.21.15.92:0/1036361170 --> [v2:172.21.15.124:6800/90883,v1:172.21.15.124:6801/90883] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8f70006b90 con 0x7f8f700015f0 2023-10-06T10:38:33.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.109+0000 7effacff9700 1 -- 172.21.15.92:0/3325554716 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7effa8050d10 con 0x7effb00685c0 2023-10-06T10:38:33.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.109+0000 7effaffff700 1 --2- 172.21.15.92:0/3325554716 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7eff9c066f30 0x7eff9c0693e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7effb00c3540 tx=0x7effa4016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.109+0000 7f8f94c00700 1 --2- 172.21.15.92:0/1036361170 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8f74067080 0x7f8f74069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.109+0000 7f8f94c00700 1 --2- 172.21.15.92:0/1036361170 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8f74067080 0x7f8f74069530 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0015a0 tx=0x7f8f7c004950 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.109+0000 7f9adeffd700 1 -- 172.21.15.92:0/869372176 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9aec023070 con 0x7f9af8166170 2023-10-06T10:38:33.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.109+0000 7f9b11bf9700 1 Processor -- start 2023-10-06T10:38:33.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.109+0000 7f9b11bf9700 1 -- start start 2023-10-06T10:38:33.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.110+0000 7f9adeffd700 1 --2- 172.21.15.92:0/869372176 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9ae4067080 0x7f9ae4069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.110+0000 7f9596d9d700 1 --2- 172.21.15.92:0/3107489805 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x7f957c0015f0 0x7f957c003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.110+0000 7effb51ea700 1 --2- 172.21.15.92:0/3325554716 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x7eff9c070b80 0x7eff9c072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.110+0000 7f9adeffd700 1 -- 172.21.15.92:0/869372176 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9aec0866a0 con 0x7f9af8166170 2023-10-06T10:38:33.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.110+0000 7f8f8ffff700 1 --2- 172.21.15.92:0/1036361170 >> [v2:172.21.15.124:6800/90883,v1:172.21.15.124:6801/90883] conn(0x7f8f700015f0 0x7f8f70003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.110+0000 7f9adeffd700 1 --2- 172.21.15.92:0/869372176 >> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] conn(0x7f9ae4070d50 0x7f9ae4073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.110+0000 7f9adeffd700 1 -- 172.21.15.92:0/869372176 --> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9ae4073850 con 0x7f9ae4070d50 2023-10-06T10:38:33.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.110+0000 7f9adeffd700 1 -- 172.21.15.92:0/869372176 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9aec0869d0 con 0x7f9af8166170 2023-10-06T10:38:33.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.110+0000 7f9b11bf9700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b0c169fb0 0x7f9b0c16a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.110+0000 7f9b11bf9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9b0c0af830 0x7f9b0c0afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.110+0000 7f9b11bf9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b0c16a960 con 0x7f9b0c0af460 2023-10-06T10:38:33.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.110+0000 7f9b11bf9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b0c16ae50 con 0x7f9b0c0af830 2023-10-06T10:38:33.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.110+0000 7f9b11bf9700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c0cc3f0 con 0x7f9b0c169fb0 2023-10-06T10:38:33.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.110+0000 7f9596d9d700 1 --2- 172.21.15.92:0/3107489805 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x7f957c0015f0 0x7f957c003aa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.110+0000 7effb51ea700 1 --2- 172.21.15.92:0/3325554716 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x7eff9c070b80 0x7eff9c072fb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.110+0000 7f9b0bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9b0c0af830 0x7f9b0c0afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.110+0000 7f9b0bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9b0c0af830 0x7f9b0c0afcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38058/0 (socket says 172.21.15.92:38058) 2023-10-06T10:38:33.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.110+0000 7f9b0bfff700 1 -- 172.21.15.92:0/545051666 learned_addr learned my addr 172.21.15.92:0/545051666 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.111+0000 7f9af6ffd700 1 --2- 172.21.15.92:0/869372176 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9ae4067080 0x7f9ae4069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.111+0000 7f9af77fe700 1 --2- 172.21.15.92:0/869372176 >> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] conn(0x7f9ae4070d50 0x7f9ae4073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.111+0000 7f9b0a7fc700 1 -- 172.21.15.92:0/545051666 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83684125 0 0) 0x7f9b0c16a960 con 0x7f9b0c0af460 2023-10-06T10:38:33.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.111+0000 7f9b0a7fc700 1 -- 172.21.15.92:0/545051666 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9af4003570 con 0x7f9b0c0af460 2023-10-06T10:38:33.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.111+0000 7f95867fc700 1 -- 172.21.15.92:0/3107489805 <== osd.2 v2:172.21.15.92:6824/90615 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f957c006b90 con 0x7f957c0015f0 2023-10-06T10:38:33.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.111+0000 7f8f8ffff700 1 --2- 172.21.15.92:0/1036361170 >> [v2:172.21.15.124:6800/90883,v1:172.21.15.124:6801/90883] conn(0x7f8f700015f0 0x7f8f70003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.111+0000 7f9af77fe700 1 --2- 172.21.15.92:0/869372176 >> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] conn(0x7f9ae4070d50 0x7f9ae4073180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.111+0000 7f9af6ffd700 1 --2- 172.21.15.92:0/869372176 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9ae4067080 0x7f9ae4069530 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9af0000f80 tx=0x7f9af0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.112+0000 7effacff9700 1 -- 172.21.15.92:0/3325554716 <== osd.0 v2:172.21.15.92:6800/90604 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7eff9c073680 con 0x7eff9c070b80 2023-10-06T10:38:33.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.111+0000 7f9b0a7fc700 1 -- 172.21.15.92:0/545051666 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3325220389 0 0) 0x7f9af4003570 con 0x7f9b0c0af460 2023-10-06T10:38:33.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.111+0000 7f9b0a7fc700 1 -- 172.21.15.92:0/545051666 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b0c169fb0 msgr2=0x7f9b0c16a420 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T10:38:33.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.111+0000 7f9b0a7fc700 1 --2- 172.21.15.92:0/545051666 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b0c169fb0 0x7f9b0c16a420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.111+0000 7f9b0a7fc700 1 -- 172.21.15.92:0/545051666 >> v2:172.21.15.92:3300/0 conn(0x7f9b0c0af830 msgr2=0x7f9b0c0afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:33.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.111+0000 7f9b0a7fc700 1 --2- 172.21.15.92:0/545051666 >> v2:172.21.15.92:3300/0 conn(0x7f9b0c0af830 0x7f9b0c0afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.111+0000 7f9b0a7fc700 1 -- 172.21.15.92:0/545051666 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0c0b63a0 con 0x7f9b0c0af460 2023-10-06T10:38:33.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.111+0000 7f9b0a7fc700 1 -- 172.21.15.92:0/545051666 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f9afc001470 con 0x7f9b0c0af460 2023-10-06T10:38:33.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.111+0000 7f9b0a7fc700 1 -- 172.21.15.92:0/545051666 >> v1:172.21.15.92:6789/0 conn(0x7f9b0c0af460 legacy=0x7f9b0c0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.111+0000 7f9b0a7fc700 1 --2- 172.21.15.92:0/545051666 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af4003ae0 0x7f9af4003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.112+0000 7f9b0a7fc700 1 --2- 172.21.15.92:0/545051666 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9af4004810 0x7f9af4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.112+0000 7f9b0a7fc700 1 --2- 172.21.15.92:0/545051666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9af40051c0 0x7f9af400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.112+0000 7f9b0a7fc700 1 -- 172.21.15.92:0/545051666 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c0cc3f0 con 0x7f9af40051c0 2023-10-06T10:38:33.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.112+0000 7f9b0a7fc700 1 -- 172.21.15.92:0/545051666 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af400c080 con 0x7f9af4004810 2023-10-06T10:38:33.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.112+0000 7f9b0a7fc700 1 -- 172.21.15.92:0/545051666 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af400c240 con 0x7f9af4003ae0 2023-10-06T10:38:33.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.112+0000 7f9b0bfff700 1 --2- 172.21.15.92:0/545051666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9af40051c0 0x7f9af400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.112+0000 7f9b0b7fe700 1 --2- 172.21.15.92:0/545051666 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af4003ae0 0x7f9af4003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.112+0000 7f8f8cff9700 1 -- 172.21.15.92:0/1036361170 <== osd.4 v2:172.21.15.124:6800/90883 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f8f70006b90 con 0x7f8f700015f0 2023-10-06T10:38:33.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.112+0000 7f9b0bfff700 1 -- 172.21.15.92:0/545051666 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af4003ae0 msgr2=0x7f9af4003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:33.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.112+0000 7f9b0bfff700 1 --2- 172.21.15.92:0/545051666 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af4003ae0 0x7f9af4003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.112+0000 7f9b0bfff700 1 -- 172.21.15.92:0/545051666 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9af4004810 msgr2=0x7f9af4004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T10:38:33.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.112+0000 7f9b0bfff700 1 --2- 172.21.15.92:0/545051666 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9af4004810 0x7f9af4004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.112+0000 7f9b0bfff700 1 -- 172.21.15.92:0/545051666 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0c0b63a0 con 0x7f9af40051c0 2023-10-06T10:38:33.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.112+0000 7f9b0bfff700 1 --2- 172.21.15.92:0/545051666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9af40051c0 0x7f9af400ba60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400e2d0 tx=0x7f9b0400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a327655c5244153 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.112+0000 7f9adeffd700 1 -- 172.21.15.92:0/869372176 <== osd.1 v2:172.21.15.92:6816/90606 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f9ae4073850 con 0x7f9ae4070d50 2023-10-06T10:38:33.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.112+0000 7f9b0a7fc700 1 -- 172.21.15.92:0/545051666 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b040090d0 con 0x7f9af40051c0 2023-10-06T10:38:33.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.112+0000 7f9b0a7fc700 1 -- 172.21.15.92:0/545051666 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0400f040 con 0x7f9af40051c0 2023-10-06T10:38:33.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.112+0000 7f9b0a7fc700 1 -- 172.21.15.92:0/545051666 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0400eb80 con 0x7f9af40051c0 2023-10-06T10:38:33.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.113+0000 7f9b11bf9700 1 -- 172.21.15.92:0/545051666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9af40051c0 msgr2=0x7f9af400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.113+0000 7f9b11bf9700 1 --2- 172.21.15.92:0/545051666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9af40051c0 0x7f9af400ba60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400e2d0 tx=0x7f9b0400b040 comp rx=0 tx=0).stop 2023-10-06T10:38:33.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.113+0000 7f9b0bfff700 1 -- 172.21.15.92:0/545051666 reap_dead start 2023-10-06T10:38:33.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.113+0000 7f9b11bf9700 1 -- 172.21.15.92:0/545051666 shutdown_connections 2023-10-06T10:38:33.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.113+0000 7f9b11bf9700 1 -- 172.21.15.92:0/545051666 >> 172.21.15.92:0/545051666 conn(0x7f9b0c0ccf30 msgr2=0x7f9b0c0cd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.126+0000 7f4210c26700 1 --2- 172.21.15.92:0/2122361083 >> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] conn(0x7f41f00015f0 0x7f41f0003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.126+0000 7f4210c26700 1 -- 172.21.15.92:0/2122361083 --> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f41f0006b90 con 0x7f41f00015f0 2023-10-06T10:38:33.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.126+0000 7f9b11bf9700 1 -- 172.21.15.92:0/545051666 shutdown_connections 2023-10-06T10:38:33.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.126+0000 7f420ad9d700 1 --2- 172.21.15.92:0/2122361083 >> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] conn(0x7f41f00015f0 0x7f41f0003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.126+0000 7f9b11bf9700 1 -- 172.21.15.92:0/545051666 wait complete. 2023-10-06T10:38:33.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.127+0000 7f420ad9d700 1 --2- 172.21.15.92:0/2122361083 >> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] conn(0x7f41f00015f0 0x7f41f0003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.128+0000 7f9b11bf9700 1 Processor -- start 2023-10-06T10:38:33.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.128+0000 7f9b11bf9700 1 -- start start 2023-10-06T10:38:33.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.128+0000 7f9b11bf9700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af40051c0 0x7f9b0c0c3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.128+0000 7f9b11bf9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b0c15dcc0 0x7f9b0c164410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.128+0000 7f9b11bf9700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b0c0c4240 0x7f9b0c0c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.128+0000 7f9b11bf9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c160ba0 con 0x7f9b0c15dcc0 2023-10-06T10:38:33.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.128+0000 7f9b11bf9700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c0c3750 con 0x7f9b0c0c4240 2023-10-06T10:38:33.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.128+0000 7f9b11bf9700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c163f80 con 0x7f9af40051c0 2023-10-06T10:38:33.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.129+0000 7f42027fc700 1 -- 172.21.15.92:0/2122361083 <== osd.5 v2:172.21.15.124:6816/90907 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f41f0006b90 con 0x7f41f00015f0 2023-10-06T10:38:33.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.129+0000 7f9b0bfff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b0c0c4240 0x7f9b0c0c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.129+0000 7f9b0affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b0c15dcc0 0x7f9b0c164410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.129+0000 7f9b0affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b0c15dcc0 0x7f9b0c164410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38080/0 (socket says 172.21.15.92:38080) 2023-10-06T10:38:33.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.129+0000 7f9b0affd700 1 -- 172.21.15.92:0/1981790319 learned_addr learned my addr 172.21.15.92:0/1981790319 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.129+0000 7f9b0affd700 1 -- 172.21.15.92:0/1981790319 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af40051c0 msgr2=0x7f9b0c0c3210 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T10:38:33.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.129+0000 7f9b0affd700 1 --2- 172.21.15.92:0/1981790319 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af40051c0 0x7f9b0c0c3210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.129+0000 7f9b0affd700 1 -- 172.21.15.92:0/1981790319 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b0c0c4240 msgr2=0x7f9b0c0c46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.129+0000 7f9b0affd700 1 --2- 172.21.15.92:0/1981790319 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b0c0c4240 0x7f9b0c0c46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.129+0000 7f9b0affd700 1 -- 172.21.15.92:0/1981790319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0c1636d0 con 0x7f9b0c15dcc0 2023-10-06T10:38:33.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.129+0000 7fd096f71700 1 -- 172.21.15.92:0/4101451074 --> [v2:172.21.15.92:6808/90605,v1:172.21.15.92:6809/90605] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd070002d30 con 0x7fd074070c80 2023-10-06T10:38:33.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.129+0000 7f9b0affd700 1 --2- 172.21.15.92:0/1981790319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b0c15dcc0 0x7f9b0c164410 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000b520 tx=0x7f9b00009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.130+0000 7f9af37fe700 1 -- 172.21.15.92:0/1981790319 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0000f510 con 0x7f9b0c15dcc0 2023-10-06T10:38:33.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.130+0000 7f9af37fe700 1 -- 172.21.15.92:0/1981790319 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0000f690 con 0x7f9b0c15dcc0 2023-10-06T10:38:33.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.130+0000 7f9af37fe700 1 -- 172.21.15.92:0/1981790319 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0000fa50 con 0x7f9b0c15dcc0 2023-10-06T10:38:33.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.130+0000 7fd08cff9700 1 -- 172.21.15.92:0/4101451074 <== osd.3 v2:172.21.15.92:6808/90605 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd070002d30 con 0x7fd074070c80 2023-10-06T10:38:33.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.130+0000 7f9b11bf9700 1 -- 172.21.15.92:0/1981790319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b0c1608c0 con 0x7f9b0c15dcc0 2023-10-06T10:38:33.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.130+0000 7f9b11bf9700 1 -- 172.21.15.92:0/1981790319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b0c163c10 con 0x7f9b0c15dcc0 2023-10-06T10:38:33.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.130+0000 7f959ca4b700 1 -- 172.21.15.92:0/3107489805 --> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f957c005c80 con 0x7f957c0015f0 2023-10-06T10:38:33.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.130+0000 7f48d429b700 1 -- 172.21.15.92:0/1874764120 --> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f48cc0cb950 con 0x7f48b0070c80 2023-10-06T10:38:33.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.131+0000 7f9b11bf9700 1 -- 172.21.15.92:0/1981790319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9aec000fd0 con 0x7f9b0c15dcc0 2023-10-06T10:38:33.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.131+0000 7f95867fc700 1 -- 172.21.15.92:0/3107489805 <== osd.2 v2:172.21.15.92:6824/90615 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f957c005c80 con 0x7f957c0015f0 2023-10-06T10:38:33.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.131+0000 7f48c27fc700 1 -- 172.21.15.92:0/1874764120 <== osd.6 v2:172.21.15.124:6824/90914 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f48cc0cb950 con 0x7f48b0070c80 2023-10-06T10:38:33.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.132+0000 7f9af37fe700 1 -- 172.21.15.92:0/1981790319 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9b00024430 con 0x7f9b0c15dcc0 2023-10-06T10:38:33.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.132+0000 7f9af37fe700 1 --2- 172.21.15.92:0/1981790319 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9af8066f30 0x7f9af80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.132+0000 7f9b0b7fe700 1 --2- 172.21.15.92:0/1981790319 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9af8066f30 0x7f9af80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.132+0000 7f9af37fe700 1 -- 172.21.15.92:0/1981790319 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9b00084d80 con 0x7f9b0c15dcc0 2023-10-06T10:38:33.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.132+0000 7f9af37fe700 1 --2- 172.21.15.92:0/1981790319 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x7f9af8070c00 0x7f9af8073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.132+0000 7f9af37fe700 1 -- 172.21.15.92:0/1981790319 --> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9af8073700 con 0x7f9af8070c00 2023-10-06T10:38:33.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.132+0000 7f9af37fe700 1 -- 172.21.15.92:0/1981790319 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b00085150 con 0x7f9b0c15dcc0 2023-10-06T10:38:33.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.133+0000 7fd096f71700 1 -- 172.21.15.92:0/4101451074 >> [v2:172.21.15.92:6808/90605,v1:172.21.15.92:6809/90605] conn(0x7fd074070c80 msgr2=0x7fd0740730b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.133+0000 7fd096f71700 1 --2- 172.21.15.92:0/4101451074 >> [v2:172.21.15.92:6808/90605,v1:172.21.15.92:6809/90605] conn(0x7fd074070c80 0x7fd0740730b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.133+0000 7fd096f71700 1 -- 172.21.15.92:0/4101451074 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd074066fb0 msgr2=0x7fd074069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.133+0000 7fd096f71700 1 --2- 172.21.15.92:0/4101451074 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd074066fb0 0x7fd074069460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd084009260 tx=0x7fd084016000 comp rx=0 tx=0).stop 2023-10-06T10:38:33.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.133+0000 7fd096f71700 1 -- 172.21.15.92:0/4101451074 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd090156280 msgr2=0x7fd09014cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.133+0000 7fd096f71700 1 --2- 172.21.15.92:0/4101451074 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd090156280 0x7fd09014cd10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00c590 tx=0x7fd07c007f70 comp rx=0 tx=0).stop 2023-10-06T10:38:33.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.133+0000 7f9b0b7fe700 1 --2- 172.21.15.92:0/1981790319 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9af8066f30 0x7f9af80693e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9afc001010 tx=0x7f9afc0078c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.133+0000 7fd09550e700 1 -- 172.21.15.92:0/4101451074 reap_dead start 2023-10-06T10:38:33.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.133+0000 7fd096f71700 1 -- 172.21.15.92:0/4101451074 shutdown_connections 2023-10-06T10:38:33.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.133+0000 7fd096f71700 1 -- 172.21.15.92:0/4101451074 >> 172.21.15.92:0/4101451074 conn(0x7fd09006de40 msgr2=0x7fd090068180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.137+0000 7f959ca4b700 1 -- 172.21.15.92:0/3107489805 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x7f957c0015f0 msgr2=0x7f957c003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.137+0000 7f959ca4b700 1 --2- 172.21.15.92:0/3107489805 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x7f957c0015f0 0x7f957c003aa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.137+0000 7f9b0bfff700 1 --2- 172.21.15.92:0/1981790319 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x7f9af8070c00 0x7f9af8073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.137+0000 7f959ca4b700 1 -- 172.21.15.92:0/3107489805 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9580067120 msgr2=0x7f95800695d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.137+0000 7f959ca4b700 1 --2- 172.21.15.92:0/3107489805 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9580067120 0x7f95800695d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f95981632c0 tx=0x7f958c006c60 comp rx=0 tx=0).stop 2023-10-06T10:38:33.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.137+0000 7f959ca4b700 1 -- 172.21.15.92:0/3107489805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9598068810 msgr2=0x7f9598162d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.137+0000 7f959ca4b700 1 --2- 172.21.15.92:0/3107489805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9598068810 0x7f9598162d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9588009160 tx=0x7f9588026000 comp rx=0 tx=0).stop 2023-10-06T10:38:33.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.137+0000 7f48d429b700 1 -- 172.21.15.92:0/1874764120 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x7f48b0070c80 msgr2=0x7f48b00730b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.137+0000 7f48d429b700 1 --2- 172.21.15.92:0/1874764120 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x7f48b0070c80 0x7f48b00730b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.137+0000 7f48d429b700 1 -- 172.21.15.92:0/1874764120 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f48b0066fb0 msgr2=0x7f48b0069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.137+0000 7f48d429b700 1 --2- 172.21.15.92:0/1874764120 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f48b0066fb0 0x7f48b0069460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f48c8005920 tx=0x7f48c8009040 comp rx=0 tx=0).stop 2023-10-06T10:38:33.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.137+0000 7f48d429b700 1 -- 172.21.15.92:0/1874764120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48cc0b29b0 msgr2=0x7f48cc15d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.137+0000 7f48d429b700 1 --2- 172.21.15.92:0/1874764120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48cc0b29b0 0x7f48cc15d1c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00ec80 tx=0x7f48bc0091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.137+0000 7f48d2838700 1 -- 172.21.15.92:0/1874764120 reap_dead start 2023-10-06T10:38:33.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.137+0000 7f48d429b700 1 -- 172.21.15.92:0/1874764120 shutdown_connections 2023-10-06T10:38:33.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.137+0000 7f48d429b700 1 -- 172.21.15.92:0/1874764120 >> 172.21.15.92:0/1874764120 conn(0x7f48cc06ece0 msgr2=0x7f48cc0b52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.137+0000 7f9596d9d700 1 -- 172.21.15.92:0/3107489805 reap_dead start 2023-10-06T10:38:33.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.138+0000 7f48d429b700 1 -- 172.21.15.92:0/1874764120 shutdown_connections 2023-10-06T10:38:33.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.138+0000 7f48d429b700 1 -- 172.21.15.92:0/1874764120 wait complete. 2023-10-06T10:38:33.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.143+0000 7f959ca4b700 1 -- 172.21.15.92:0/3107489805 shutdown_connections 2023-10-06T10:38:33.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.143+0000 7f959ca4b700 1 -- 172.21.15.92:0/3107489805 >> 172.21.15.92:0/3107489805 conn(0x7f959806de40 msgr2=0x7f95980c99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.143+0000 7fd096f71700 1 -- 172.21.15.92:0/4101451074 shutdown_connections 2023-10-06T10:38:33.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.137+0000 7effb59eb700 1 -- 172.21.15.92:0/3325554716 --> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7eff90002d30 con 0x7eff9c070b80 2023-10-06T10:38:33.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.138+0000 7effacff9700 1 -- 172.21.15.92:0/3325554716 <== osd.0 v2:172.21.15.92:6800/90604 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7eff90002d30 con 0x7eff9c070b80 2023-10-06T10:38:33.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.138+0000 7effb59eb700 1 -- 172.21.15.92:0/3325554716 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x7eff9c070b80 msgr2=0x7eff9c072fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.138+0000 7effb59eb700 1 --2- 172.21.15.92:0/3325554716 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x7eff9c070b80 0x7eff9c072fb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.143+0000 7effb59eb700 1 -- 172.21.15.92:0/3325554716 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7eff9c066f30 msgr2=0x7eff9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.143+0000 7effb59eb700 1 --2- 172.21.15.92:0/3325554716 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7eff9c066f30 0x7eff9c0693e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7effb00c3540 tx=0x7effa4016000 comp rx=0 tx=0).stop 2023-10-06T10:38:33.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.143+0000 7effb59eb700 1 -- 172.21.15.92:0/3325554716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb00685c0 msgr2=0x7effb00ac1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.143+0000 7effb59eb700 1 --2- 172.21.15.92:0/3325554716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb00685c0 0x7effb00ac1c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7effa800c8c0 tx=0x7effa80026a0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.143+0000 7f959ca4b700 1 -- 172.21.15.92:0/3107489805 shutdown_connections 2023-10-06T10:38:33.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.143+0000 7effb51ea700 1 -- 172.21.15.92:0/3325554716 reap_dead start 2023-10-06T10:38:33.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.143+0000 7f959ca4b700 1 -- 172.21.15.92:0/3107489805 wait complete. 2023-10-06T10:38:33.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.143+0000 7effb59eb700 1 -- 172.21.15.92:0/3325554716 shutdown_connections 2023-10-06T10:38:33.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.143+0000 7effb59eb700 1 -- 172.21.15.92:0/3325554716 >> 172.21.15.92:0/3325554716 conn(0x7effb00c9c10 msgr2=0x7effb014d320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.143+0000 7effb59eb700 1 -- 172.21.15.92:0/3325554716 shutdown_connections 2023-10-06T10:38:33.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.143+0000 7effb59eb700 1 -- 172.21.15.92:0/3325554716 wait complete. 2023-10-06T10:38:33.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.143+0000 7f9b0bfff700 1 --2- 172.21.15.92:0/1981790319 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x7f9af8070c00 0x7f9af8073030 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.143+0000 7f8f96e64700 1 -- 172.21.15.92:0/1036361170 --> [v2:172.21.15.124:6800/90883,v1:172.21.15.124:6801/90883] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8f70005c80 con 0x7f8f700015f0 2023-10-06T10:38:33.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.144+0000 7fd096f71700 1 -- 172.21.15.92:0/4101451074 wait complete. 2023-10-06T10:38:33.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.144+0000 7f8f8cff9700 1 -- 172.21.15.92:0/1036361170 <== osd.4 v2:172.21.15.124:6800/90883 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8f70005c80 con 0x7f8f700015f0 2023-10-06T10:38:33.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.144+0000 7f8f96e64700 1 -- 172.21.15.92:0/1036361170 >> [v2:172.21.15.124:6800/90883,v1:172.21.15.124:6801/90883] conn(0x7f8f700015f0 msgr2=0x7f8f70003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.144+0000 7f8f96e64700 1 --2- 172.21.15.92:0/1036361170 >> [v2:172.21.15.124:6800/90883,v1:172.21.15.124:6801/90883] conn(0x7f8f700015f0 0x7f8f70003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.144+0000 7f8f96e64700 1 -- 172.21.15.92:0/1036361170 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8f74067080 msgr2=0x7f8f74069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.144+0000 7f8f96e64700 1 --2- 172.21.15.92:0/1036361170 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8f74067080 0x7f8f74069530 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0015a0 tx=0x7f8f7c004950 comp rx=0 tx=0).stop 2023-10-06T10:38:33.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.144+0000 7f8f96e64700 1 -- 172.21.15.92:0/1036361170 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f901719f0 msgr2=0x7f8f900b11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.144+0000 7f8f96e64700 1 --2- 172.21.15.92:0/1036361170 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f901719f0 0x7f8f900b11b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8f9006d140 tx=0x7f8f8801a000 comp rx=0 tx=0).stop 2023-10-06T10:38:33.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.144+0000 7f8f95401700 1 -- 172.21.15.92:0/1036361170 reap_dead start 2023-10-06T10:38:33.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.150+0000 7f8f96e64700 1 -- 172.21.15.92:0/1036361170 shutdown_connections 2023-10-06T10:38:33.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.150+0000 7f8f96e64700 1 -- 172.21.15.92:0/1036361170 >> 172.21.15.92:0/1036361170 conn(0x7f8f90068080 msgr2=0x7f8f900cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.151+0000 7f9afd9b0700 1 -- 172.21.15.92:0/869372176 --> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9af814bb40 con 0x7f9ae4070d50 2023-10-06T10:38:33.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.152+0000 7f8f96e64700 1 -- 172.21.15.92:0/1036361170 shutdown_connections 2023-10-06T10:38:33.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.152+0000 7f8f96e64700 1 -- 172.21.15.92:0/1036361170 wait complete. 2023-10-06T10:38:33.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.144+0000 7f9af37fe700 1 -- 172.21.15.92:0/1981790319 <== osd.7 v2:172.21.15.124:6808/90884 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f9af8073700 con 0x7f9af8070c00 2023-10-06T10:38:33.201 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-10-06T10:38:33.202 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T10:38:33.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.154+0000 7f4210c26700 1 -- 172.21.15.92:0/2122361083 --> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f41f0005c20 con 0x7f41f00015f0 2023-10-06T10:38:33.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.155+0000 7f42027fc700 1 -- 172.21.15.92:0/2122361083 <== osd.5 v2:172.21.15.124:6816/90907 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f41f0005c20 con 0x7f41f00015f0 2023-10-06T10:38:33.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.155+0000 7f4210c26700 1 -- 172.21.15.92:0/2122361083 >> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] conn(0x7f41f00015f0 msgr2=0x7f41f0003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.155+0000 7f4210c26700 1 --2- 172.21.15.92:0/2122361083 >> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] conn(0x7f41f00015f0 0x7f41f0003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.155+0000 7f4210c26700 1 -- 172.21.15.92:0/2122361083 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f41ec067080 msgr2=0x7f41ec069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.155+0000 7f4210c26700 1 --2- 172.21.15.92:0/2122361083 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f41ec067080 0x7f41ec069530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f420400bfd0 tx=0x7f4204008040 comp rx=0 tx=0).stop 2023-10-06T10:38:33.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.155+0000 7f4210c26700 1 -- 172.21.15.92:0/2122361083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f420c068cd0 msgr2=0x7f420c0ca940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.155+0000 7f4210c26700 1 --2- 172.21.15.92:0/2122361083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f420c068cd0 0x7f420c0ca940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00cb10 tx=0x7f41fc0027e0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.156+0000 7f420ad9d700 1 -- 172.21.15.92:0/2122361083 reap_dead start 2023-10-06T10:38:33.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.156+0000 7f4210c26700 1 -- 172.21.15.92:0/2122361083 shutdown_connections 2023-10-06T10:38:33.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.156+0000 7f4210c26700 1 -- 172.21.15.92:0/2122361083 >> 172.21.15.92:0/2122361083 conn(0x7f420c06de40 msgr2=0x7f420c0aeee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.156+0000 7f4210c26700 1 -- 172.21.15.92:0/2122361083 shutdown_connections 2023-10-06T10:38:33.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.156+0000 7f4210c26700 1 -- 172.21.15.92:0/2122361083 wait complete. 2023-10-06T10:38:33.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.158+0000 7f9adeffd700 1 -- 172.21.15.92:0/869372176 <== osd.1 v2:172.21.15.92:6816/90606 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9af814bb40 con 0x7f9ae4070d50 2023-10-06T10:38:33.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.161+0000 7f9afd9b0700 1 -- 172.21.15.92:0/869372176 >> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] conn(0x7f9ae4070d50 msgr2=0x7f9ae4073180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.162+0000 7f9afd9b0700 1 --2- 172.21.15.92:0/869372176 >> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] conn(0x7f9ae4070d50 0x7f9ae4073180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.206 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-10-06T10:38:33.206 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T10:38:33.207 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-10-06T10:38:33.207 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T10:38:33.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.163+0000 7f9afd9b0700 1 -- 172.21.15.92:0/869372176 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9ae4067080 msgr2=0x7f9ae4069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.163+0000 7f9afd9b0700 1 --2- 172.21.15.92:0/869372176 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9ae4067080 0x7f9ae4069530 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9af0000f80 tx=0x7f9af0009040 comp rx=0 tx=0).stop 2023-10-06T10:38:33.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.163+0000 7f9afd9b0700 1 -- 172.21.15.92:0/869372176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9af8166170 msgr2=0x7f9af80c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.163+0000 7f9afd9b0700 1 --2- 172.21.15.92:0/869372176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9af8166170 0x7f9af80c6b90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0004710 tx=0x7f9aec01a000 comp rx=0 tx=0).stop 2023-10-06T10:38:33.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.163+0000 7f9af77fe700 1 -- 172.21.15.92:0/869372176 reap_dead start 2023-10-06T10:38:33.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.163+0000 7f9afd9b0700 1 -- 172.21.15.92:0/869372176 shutdown_connections 2023-10-06T10:38:33.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.163+0000 7f9afd9b0700 1 -- 172.21.15.92:0/869372176 >> 172.21.15.92:0/869372176 conn(0x7f9af80cdeb0 msgr2=0x7f9af80cbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.168+0000 7f9afd9b0700 1 -- 172.21.15.92:0/869372176 shutdown_connections 2023-10-06T10:38:33.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.168+0000 7f9afd9b0700 1 -- 172.21.15.92:0/869372176 wait complete. 2023-10-06T10:38:33.209 INFO:teuthology.orchestra.run.smithi092.stdout:55834574857 2023-10-06T10:38:33.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.170+0000 7f9b11bf9700 1 -- 172.21.15.92:0/1981790319 --> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9aec002d30 con 0x7f9af8070c00 2023-10-06T10:38:33.209 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T10:38:33.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.171+0000 7f9af37fe700 1 -- 172.21.15.92:0/1981790319 <== osd.7 v2:172.21.15.124:6808/90884 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9aec002d30 con 0x7f9af8070c00 2023-10-06T10:38:33.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.171+0000 7f9b11bf9700 1 -- 172.21.15.92:0/1981790319 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x7f9af8070c00 msgr2=0x7f9af8073030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.171+0000 7f9b11bf9700 1 --2- 172.21.15.92:0/1981790319 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x7f9af8070c00 0x7f9af8073030 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.171+0000 7f9b11bf9700 1 -- 172.21.15.92:0/1981790319 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9af8066f30 msgr2=0x7f9af80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.171+0000 7f9b11bf9700 1 --2- 172.21.15.92:0/1981790319 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9af8066f30 0x7f9af80693e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9afc001010 tx=0x7f9afc0078c0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.171+0000 7f9b11bf9700 1 -- 172.21.15.92:0/1981790319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b0c15dcc0 msgr2=0x7f9b0c164410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.171+0000 7f9b11bf9700 1 --2- 172.21.15.92:0/1981790319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b0c15dcc0 0x7f9b0c164410 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000b520 tx=0x7f9b00009060 comp rx=0 tx=0).stop 2023-10-06T10:38:33.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.171+0000 7f9b0bfff700 1 -- 172.21.15.92:0/1981790319 reap_dead start 2023-10-06T10:38:33.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.171+0000 7f9b11bf9700 1 -- 172.21.15.92:0/1981790319 shutdown_connections 2023-10-06T10:38:33.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.171+0000 7f9b11bf9700 1 -- 172.21.15.92:0/1981790319 >> 172.21.15.92:0/1981790319 conn(0x7f9b0c0ccf30 msgr2=0x7f9b0c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.171+0000 7f9b11bf9700 1 -- 172.21.15.92:0/1981790319 shutdown_connections 2023-10-06T10:38:33.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.171+0000 7f9b11bf9700 1 -- 172.21.15.92:0/1981790319 wait complete. 2023-10-06T10:38:33.212 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-10-06T10:38:33.212 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T10:38:33.213 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-10-06T10:38:33.213 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T10:38:33.214 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-10-06T10:38:33.214 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T10:38:33.215 INFO:teuthology.orchestra.run.smithi092.stdout:55834574857 2023-10-06T10:38:33.216 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T10:38:33.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.443+0000 7feef064f700 1 Processor -- start 2023-10-06T10:38:33.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.443+0000 7feef064f700 1 -- start start 2023-10-06T10:38:33.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.444+0000 7fb41a520700 1 Processor -- start 2023-10-06T10:38:33.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.444+0000 7feef064f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feee80707d0 0x7feee8068810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.444+0000 7feef064f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7feee8070c20 0x7feee8155600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.444+0000 7feef064f700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee80ac8d0 con 0x7feee80ac440 2023-10-06T10:38:33.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.444+0000 7feef064f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee80acbb0 con 0x7feee8070c20 2023-10-06T10:38:33.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.444+0000 7feef064f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee80ace90 con 0x7feee80707d0 2023-10-06T10:38:33.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.444+0000 7fb41a520700 1 -- start start 2023-10-06T10:38:33.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.445+0000 7fb41a520700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb41415d8b0 0x7fb41415dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.445+0000 7fb41a520700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb414068cd0 0x7fb4140b2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.445+0000 7fb41a520700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4140c9710 con 0x7fb4140b1d10 2023-10-06T10:38:33.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.445+0000 7fb41a520700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4140c8eb0 con 0x7fb41415d8b0 2023-10-06T10:38:33.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.445+0000 7fb41a520700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4140b26c0 con 0x7fb414068cd0 2023-10-06T10:38:33.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.445+0000 7fb4137fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb4140b1d10 0x7fb4140685c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40018/0 (socket says 172.21.15.92:40018) 2023-10-06T10:38:33.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.445+0000 7fb4137fe700 1 -- 172.21.15.92:0/1397186664 learned_addr learned my addr 172.21.15.92:0/1397186664 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:33.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.445+0000 7fb418abd700 1 --2- 172.21.15.92:0/1397186664 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb414068cd0 0x7fb4140b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.444+0000 7feeedbea700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feee80707d0 0x7feee8068810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.444+0000 7feeedbea700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feee80707d0 0x7feee8068810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:51778/0 (socket says 172.21.15.92:51778) 2023-10-06T10:38:33.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.444+0000 7feeedbea700 1 -- 172.21.15.92:0/2023237296 learned_addr learned my addr 172.21.15.92:0/2023237296 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.444+0000 7feeeebec700 1 --2- 172.21.15.92:0/2023237296 >> v2:172.21.15.92:3300/0 conn(0x7feee8070c20 0x7feee8155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.445+0000 7feeedbea700 1 -- 172.21.15.92:0/2023237296 >> v2:172.21.15.92:3300/0 conn(0x7feee8070c20 msgr2=0x7feee8155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:33.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.445+0000 7feeedbea700 1 --2- 172.21.15.92:0/2023237296 >> v2:172.21.15.92:3300/0 conn(0x7feee8070c20 0x7feee8155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.445+0000 7feeedbea700 1 -- 172.21.15.92:0/2023237296 >> v1:172.21.15.124:6789/0 conn(0x7feee80ac440 legacy=0x7feee80afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:33.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.445+0000 7feeedbea700 1 -- 172.21.15.92:0/2023237296 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee80ad170 con 0x7feee80707d0 2023-10-06T10:38:33.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.445+0000 7feeedbea700 1 --2- 172.21.15.92:0/2023237296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feee80707d0 0x7feee8068810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feed80099b0 tx=0x7feed8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdf1bfe41d1a500 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.445+0000 7feeed3e9700 1 -- 172.21.15.92:0/2023237296 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed8009ea0 con 0x7feee80707d0 2023-10-06T10:38:33.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.445+0000 7f626c760700 1 Processor -- start 2023-10-06T10:38:33.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.446+0000 7fb418abd700 1 -- 172.21.15.92:0/1397186664 >> v2:172.21.15.92:3300/0 conn(0x7fb41415d8b0 msgr2=0x7fb41415dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:33.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.446+0000 7fb418abd700 1 --2- 172.21.15.92:0/1397186664 >> v2:172.21.15.92:3300/0 conn(0x7fb41415d8b0 0x7fb41415dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.446+0000 7fb418abd700 1 -- 172.21.15.92:0/1397186664 >> v1:172.21.15.92:6789/0 conn(0x7fb4140b1d10 legacy=0x7fb4140685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.446+0000 7fb418abd700 1 -- 172.21.15.92:0/1397186664 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4140b28d0 con 0x7fb414068cd0 2023-10-06T10:38:33.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.445+0000 7f626c760700 1 -- start start 2023-10-06T10:38:33.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.446+0000 7f789d36b700 1 Processor -- start 2023-10-06T10:38:33.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.446+0000 7fb418abd700 1 --2- 172.21.15.92:0/1397186664 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb414068cd0 0x7fb4140b2180 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009e60 tx=0x7fb40c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e06dba2906201b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.446+0000 7fb412ffd700 1 -- 172.21.15.92:0/1397186664 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c00e070 con 0x7fb414068cd0 2023-10-06T10:38:33.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.446+0000 7fb412ffd700 1 -- 172.21.15.92:0/1397186664 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40c00bc30 con 0x7fb414068cd0 2023-10-06T10:38:33.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.446+0000 7fb412ffd700 1 -- 172.21.15.92:0/1397186664 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c012430 con 0x7fb414068cd0 2023-10-06T10:38:33.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7fb41a520700 1 -- 172.21.15.92:0/1397186664 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb414068cd0 msgr2=0x7fb4140b2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7fb41a520700 1 --2- 172.21.15.92:0/1397186664 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb414068cd0 0x7fb4140b2180 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009e60 tx=0x7fb40c009780 comp rx=0 tx=0).stop 2023-10-06T10:38:33.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.446+0000 7f789d36b700 1 -- start start 2023-10-06T10:38:33.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7feeed3e9700 1 -- 172.21.15.92:0/2023237296 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed800b820 con 0x7feee80707d0 2023-10-06T10:38:33.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7feeed3e9700 1 -- 172.21.15.92:0/2023237296 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed800bba0 con 0x7feee80707d0 2023-10-06T10:38:33.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.446+0000 7f626c760700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62640c8260 0x7f6264069520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f626c760700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62640c7dd0 con 0x7f62640cbc20 2023-10-06T10:38:33.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f626c760700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62640c71c0 con 0x7f6264155820 2023-10-06T10:38:33.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f626c760700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62640c74a0 con 0x7f62640c8260 2023-10-06T10:38:33.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f626a4fc700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f6264155820 0x7f62640ae480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:37304/0 (socket says 172.21.15.92:37304) 2023-10-06T10:38:33.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f626a4fc700 1 -- 172.21.15.92:0/4225089129 learned_addr learned my addr 172.21.15.92:0/4225089129 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:33.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f62694fa700 1 -- 172.21.15.92:0/4225089129 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2763925180 0 0) 0x7f62640c7dd0 con 0x7f62640cbc20 2023-10-06T10:38:33.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f62694fa700 1 -- 172.21.15.92:0/4225089129 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6250003570 con 0x7f62640cbc20 2023-10-06T10:38:33.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f62694fa700 1 -- 172.21.15.92:0/4225089129 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2427637041 0 0) 0x7f62640c71c0 con 0x7f6264155820 2023-10-06T10:38:33.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f62694fa700 1 -- 172.21.15.92:0/4225089129 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62640c7dd0 con 0x7f6264155820 2023-10-06T10:38:33.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7fb683ac4700 1 Processor -- start 2023-10-06T10:38:33.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f789d36b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f78980c8260 0x7f7898155820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f789d36b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f789806e3d0 0x7f7898069520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f789d36b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78980c7dd0 con 0x7f78980cbc20 2023-10-06T10:38:33.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f789d36b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78980acca0 con 0x7f78980c8260 2023-10-06T10:38:33.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f789d36b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7898155d60 con 0x7f789806e3d0 2023-10-06T10:38:33.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f62694fa700 1 -- 172.21.15.92:0/4225089129 <== mon.1 v1:172.21.15.124:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3152039833 0 0) 0x7f62640c7dd0 con 0x7f6264155820 2023-10-06T10:38:33.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f62694fa700 1 -- 172.21.15.92:0/4225089129 >> v2:172.21.15.92:3300/0 conn(0x7f62640c8260 msgr2=0x7f6264069520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T10:38:33.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f62694fa700 1 --2- 172.21.15.92:0/4225089129 >> v2:172.21.15.92:3300/0 conn(0x7f62640c8260 0x7f6264069520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f62694fa700 1 -- 172.21.15.92:0/4225089129 >> v1:172.21.15.92:6789/0 conn(0x7f62640cbc20 legacy=0x7f62640abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f62694fa700 1 -- 172.21.15.92:0/4225089129 --> v1:172.21.15.124:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62640c7780 con 0x7f6264155820 2023-10-06T10:38:33.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f62694fa700 1 -- 172.21.15.92:0/4225089129 <== mon.1 v1:172.21.15.124:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f625c0017f0 con 0x7f6264155820 2023-10-06T10:38:33.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f62694fa700 1 -- 172.21.15.92:0/4225089129 >> v1:172.21.15.124:6789/0 conn(0x7f6264155820 legacy=0x7f62640ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f62694fa700 1 --2- 172.21.15.92:0/4225089129 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6250004c50 0x7f6250005020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f62694fa700 1 --2- 172.21.15.92:0/4225089129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62500037b0 0x7f6250003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f62694fa700 1 --2- 172.21.15.92:0/4225089129 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6250004280 0x7f6250014270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f62694fa700 1 -- 172.21.15.92:0/4225089129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62640c74a0 con 0x7f62500037b0 2023-10-06T10:38:33.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f62694fa700 1 -- 172.21.15.92:0/4225089129 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62500149f0 con 0x7f6250004c50 2023-10-06T10:38:33.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7f62694fa700 1 -- 172.21.15.92:0/4225089129 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6250014bb0 con 0x7f6250004280 2023-10-06T10:38:33.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7fb41a520700 1 -- 172.21.15.92:0/1397186664 shutdown_connections 2023-10-06T10:38:33.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7fb41a520700 1 --2- 172.21.15.92:0/1397186664 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb414068cd0 0x7fb4140b2180 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7fb41a520700 1 --2- 172.21.15.92:0/1397186664 >> v2:172.21.15.92:3300/0 conn(0x7fb41415d8b0 0x7fb41415dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7fb41a520700 1 -- 172.21.15.92:0/1397186664 >> 172.21.15.92:0/1397186664 conn(0x7fb41406de40 msgr2=0x7fb41406e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.448+0000 7f7896ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f78980c8260 0x7f7898155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.448+0000 7f7896ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f78980c8260 0x7f7898155820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36666/0 (socket says 172.21.15.92:36666) 2023-10-06T10:38:33.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.448+0000 7f7896ffd700 1 -- 172.21.15.92:0/3846050477 learned_addr learned my addr 172.21.15.92:0/3846050477 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.448+0000 7f78977fe700 1 --2- 172.21.15.92:0/3846050477 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f789806e3d0 0x7f7898069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.448+0000 7f626a4fc700 1 --2- 172.21.15.92:0/4225089129 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6250004c50 0x7f6250005020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.448+0000 7fb41a520700 1 -- 172.21.15.92:0/1397186664 shutdown_connections 2023-10-06T10:38:33.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.448+0000 7fb41a520700 1 -- 172.21.15.92:0/1397186664 wait complete. 2023-10-06T10:38:33.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.448+0000 7feef064f700 1 -- 172.21.15.92:0/2023237296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feee80707d0 msgr2=0x7feee8068810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.448+0000 7feef064f700 1 --2- 172.21.15.92:0/2023237296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feee80707d0 0x7feee8068810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feed80099b0 tx=0x7feed8009250 comp rx=0 tx=0).stop 2023-10-06T10:38:33.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.448+0000 7f7896ffd700 1 -- 172.21.15.92:0/3846050477 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f789806e3d0 msgr2=0x7f7898069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:33.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.448+0000 7f7896ffd700 1 --2- 172.21.15.92:0/3846050477 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f789806e3d0 0x7f7898069520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.448+0000 7f7896ffd700 1 -- 172.21.15.92:0/3846050477 >> v1:172.21.15.92:6789/0 conn(0x7f78980cbc20 legacy=0x7f78980ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.448+0000 7f7896ffd700 1 -- 172.21.15.92:0/3846050477 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78980c71c0 con 0x7f78980c8260 2023-10-06T10:38:33.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.448+0000 7feef064f700 1 -- 172.21.15.92:0/2023237296 shutdown_connections 2023-10-06T10:38:33.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.448+0000 7feef064f700 1 --2- 172.21.15.92:0/2023237296 >> v2:172.21.15.92:3300/0 conn(0x7feee8070c20 0x7feee8155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.448+0000 7feef064f700 1 --2- 172.21.15.92:0/2023237296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feee80707d0 0x7feee8068810 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.448+0000 7feef064f700 1 -- 172.21.15.92:0/2023237296 >> 172.21.15.92:0/2023237296 conn(0x7feee806de40 msgr2=0x7feee806e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.449+0000 7f7896ffd700 1 --2- 172.21.15.92:0/3846050477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78980c8260 0x7f7898155820 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7890009f20 tx=0x7f7890009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc9b1ee5e5e3e70 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.448+0000 7f626a4fc700 1 -- 172.21.15.92:0/4225089129 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6250004280 msgr2=0x7f6250014270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T10:38:33.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.448+0000 7f626a4fc700 1 --2- 172.21.15.92:0/4225089129 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6250004280 0x7f6250014270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.448+0000 7f626a4fc700 1 -- 172.21.15.92:0/4225089129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62500037b0 msgr2=0x7f6250003c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T10:38:33.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.448+0000 7f626a4fc700 1 --2- 172.21.15.92:0/4225089129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62500037b0 0x7f6250003c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.448+0000 7f626a4fc700 1 -- 172.21.15.92:0/4225089129 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62640c7780 con 0x7f6250004c50 2023-10-06T10:38:33.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.447+0000 7fb683ac4700 1 -- start start 2023-10-06T10:38:33.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.449+0000 7fb683ac4700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb67c0b2420 0x7fb67c15dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.449+0000 7fb683ac4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb67c0685c0 0x7fb67c068a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.449+0000 7fb683ac4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67c0c9710 con 0x7fb67c15d8b0 2023-10-06T10:38:33.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.449+0000 7fb683ac4700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb67c068fa0 con 0x7fb67c0b2420 2023-10-06T10:38:33.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.449+0000 7fb683ac4700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c0c8eb0 con 0x7fb67c0685c0 2023-10-06T10:38:33.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.449+0000 7f7895ffb700 1 -- 172.21.15.92:0/3846050477 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789000e070 con 0x7f78980c8260 2023-10-06T10:38:33.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.449+0000 7f626acfd700 1 -- 172.21.15.92:0/4225089129 reap_dead start 2023-10-06T10:38:33.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.449+0000 7fb68105f700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb67c0b2420 0x7fb67c15dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.449+0000 7fb68105f700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb67c0b2420 0x7fb67c15dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:47328/0 (socket says 172.21.15.92:47328) 2023-10-06T10:38:33.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.449+0000 7fb68105f700 1 -- 172.21.15.92:0/1722225009 learned_addr learned my addr 172.21.15.92:0/1722225009 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.449+0000 7feef064f700 1 -- 172.21.15.92:0/2023237296 shutdown_connections 2023-10-06T10:38:33.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.449+0000 7fb68105f700 1 -- 172.21.15.92:0/1722225009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb67c0685c0 msgr2=0x7fb67c068a60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T10:38:33.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.449+0000 7fb68105f700 1 --2- 172.21.15.92:0/1722225009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb67c0685c0 0x7fb67c068a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.449+0000 7fb68105f700 1 -- 172.21.15.92:0/1722225009 >> v1:172.21.15.92:6789/0 conn(0x7fb67c15d8b0 legacy=0x7fb67c0b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.449+0000 7fb68105f700 1 -- 172.21.15.92:0/1722225009 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c0b28d0 con 0x7fb67c0b2420 2023-10-06T10:38:33.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.449+0000 7f626a4fc700 1 --2- 172.21.15.92:0/4225089129 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6250004c50 0x7f6250005020 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f625c00a960 tx=0x7f625c00ac70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46dbb1e9678d3947 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.449+0000 7fb1f32a1700 1 Processor -- start 2023-10-06T10:38:33.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.449+0000 7f1a7338c700 1 Processor -- start 2023-10-06T10:38:33.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.449+0000 7fb68105f700 1 --2- 172.21.15.92:0/1722225009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb67c0b2420 0x7fb67c15dc80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb6780099b0 tx=0x7fb678009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1332a6dc782feb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.450+0000 7f62694fa700 1 -- 172.21.15.92:0/4225089129 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c007110 con 0x7f6250004c50 2023-10-06T10:38:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.450+0000 7f62694fa700 1 -- 172.21.15.92:0/4225089129 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625c007270 con 0x7f6250004c50 2023-10-06T10:38:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.450+0000 7f62694fa700 1 -- 172.21.15.92:0/4225089129 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c007630 con 0x7f6250004c50 2023-10-06T10:38:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.450+0000 7fb1f32a1700 1 -- start start 2023-10-06T10:38:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.450+0000 7fb41a520700 1 Processor -- start 2023-10-06T10:38:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.450+0000 7f1a7338c700 1 -- start start 2023-10-06T10:38:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.449+0000 7f7895ffb700 1 -- 172.21.15.92:0/3846050477 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789000b980 con 0x7f78980c8260 2023-10-06T10:38:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.449+0000 7f7895ffb700 1 -- 172.21.15.92:0/3846050477 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78900092e0 con 0x7f78980c8260 2023-10-06T10:38:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.450+0000 7f789d36b700 1 -- 172.21.15.92:0/3846050477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78980c8260 msgr2=0x7f7898155820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.450+0000 7f789d36b700 1 --2- 172.21.15.92:0/3846050477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78980c8260 0x7f7898155820 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7890009f20 tx=0x7f7890009840 comp rx=0 tx=0).stop 2023-10-06T10:38:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.450+0000 7feef064f700 1 -- 172.21.15.92:0/2023237296 wait complete. 2023-10-06T10:38:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.450+0000 7fb68085e700 1 -- 172.21.15.92:0/1722225009 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67800b860 con 0x7fb67c0b2420 2023-10-06T10:38:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.450+0000 7fb68085e700 1 -- 172.21.15.92:0/1722225009 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67800b9c0 con 0x7fb67c0b2420 2023-10-06T10:38:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.450+0000 7fb68085e700 1 -- 172.21.15.92:0/1722225009 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67800bd60 con 0x7fb67c0b2420 2023-10-06T10:38:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.450+0000 7f789d36b700 1 -- 172.21.15.92:0/3846050477 shutdown_connections 2023-10-06T10:38:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.450+0000 7f789d36b700 1 --2- 172.21.15.92:0/3846050477 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f789806e3d0 0x7f7898069520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.450+0000 7f789d36b700 1 --2- 172.21.15.92:0/3846050477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78980c8260 0x7f7898155820 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.450+0000 7f789d36b700 1 -- 172.21.15.92:0/3846050477 >> 172.21.15.92:0/3846050477 conn(0x7f7898068080 msgr2=0x7f78980cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7f789d36b700 1 -- 172.21.15.92:0/3846050477 shutdown_connections 2023-10-06T10:38:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7f789d36b700 1 -- 172.21.15.92:0/3846050477 wait complete. 2023-10-06T10:38:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.450+0000 7f626c760700 1 -- 172.21.15.92:0/4225089129 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6250004c50 msgr2=0x7f6250005020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.450+0000 7f626c760700 1 --2- 172.21.15.92:0/4225089129 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6250004c50 0x7f6250005020 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f625c00a960 tx=0x7f625c00ac70 comp rx=0 tx=0).stop 2023-10-06T10:38:33.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.450+0000 7fb41a520700 1 -- start start 2023-10-06T10:38:33.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7fb1f32a1700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb1ec15dab0 0x7fb1ec164700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7fb1f32a1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1ec15b920 con 0x7fb1ec159a20 2023-10-06T10:38:33.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7fb1f32a1700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1ec15e2b0 con 0x7fb1ec069c30 2023-10-06T10:38:33.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7fb1f32a1700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1ec15a1e0 con 0x7fb1ec15dab0 2023-10-06T10:38:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7f1a7338c700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1a6c155820 0x7f1a6c0ae480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7f1a7338c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1a6c069c30 0x7f1a6c1604a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7f1a7338c700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6c0c7dd0 con 0x7f1a6c0c8260 2023-10-06T10:38:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7f1a7338c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6c0acca0 con 0x7f1a6c069c30 2023-10-06T10:38:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7f1a7338c700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6c0ae9c0 con 0x7f1a6c155820 2023-10-06T10:38:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7f626c760700 1 -- 172.21.15.92:0/4225089129 shutdown_connections 2023-10-06T10:38:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7f626c760700 1 --2- 172.21.15.92:0/4225089129 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6250004c50 0x7f6250005020 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7f626c760700 1 -- 172.21.15.92:0/4225089129 >> 172.21.15.92:0/4225089129 conn(0x7f6264068080 msgr2=0x7f62640cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7fb1f083c700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fb1ec069c30 0x7fb1ec15d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:37310/0 (socket says 172.21.15.92:37310) 2023-10-06T10:38:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7fb1f083c700 1 -- 172.21.15.92:0/1622032140 learned_addr learned my addr 172.21.15.92:0/1622032140 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7feef064f700 1 Processor -- start 2023-10-06T10:38:33.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7f1a71128700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1a6c155820 0x7f1a6c0ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7fb41a520700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb414068cd0 0x7fb41414b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7fb41a520700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4140b1d10 0x7fb41414b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7f1a71128700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1a6c155820 0x7f1a6c0ae480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:47332/0 (socket says 172.21.15.92:47332) 2023-10-06T10:38:33.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7f1a71128700 1 -- 172.21.15.92:0/3879634750 learned_addr learned my addr 172.21.15.92:0/3879634750 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7fb41a520700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb41415d8b0 0x7fb41414c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7fb41a520700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4140ad5d0 con 0x7fb41415d8b0 2023-10-06T10:38:33.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7fb41a520700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41414dd40 con 0x7fb414068cd0 2023-10-06T10:38:33.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.451+0000 7fb41a520700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41414afe0 con 0x7fb4140b1d10 2023-10-06T10:38:33.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7fb413fff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb414068cd0 0x7fb41414b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7fb413fff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb414068cd0 0x7fb41414b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:47338/0 (socket says 172.21.15.92:47338) 2023-10-06T10:38:33.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7fb413fff700 1 -- 172.21.15.92:0/180912183 learned_addr learned my addr 172.21.15.92:0/180912183 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7f626c760700 1 -- 172.21.15.92:0/4225089129 shutdown_connections 2023-10-06T10:38:33.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7feef064f700 1 -- start start 2023-10-06T10:38:33.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7fb4137fe700 1 --2- 172.21.15.92:0/180912183 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4140b1d10 0x7fb41414b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7fb683ac4700 1 -- 172.21.15.92:0/1722225009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb67c0b2420 msgr2=0x7fb67c15dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7fb683ac4700 1 --2- 172.21.15.92:0/1722225009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb67c0b2420 0x7fb67c15dc80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb6780099b0 tx=0x7fb678009250 comp rx=0 tx=0).stop 2023-10-06T10:38:33.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7fb1f183e700 1 --2- 172.21.15.92:0/1622032140 >> v2:172.21.15.124:3300/0 conn(0x7fb1ec15dab0 0x7fb1ec164700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7fb1ebfff700 1 -- 172.21.15.92:0/1622032140 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2662579534 0 0) 0x7fb1ec15e2b0 con 0x7fb1ec069c30 2023-10-06T10:38:33.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7fb1ebfff700 1 -- 172.21.15.92:0/1622032140 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1dc003570 con 0x7fb1ec069c30 2023-10-06T10:38:33.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7fb418abd700 1 --2- 172.21.15.92:0/180912183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb41415d8b0 0x7fb41414c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7f1a71929700 1 --2- 172.21.15.92:0/3879634750 >> v2:172.21.15.92:3300/0 conn(0x7f1a6c069c30 0x7f1a6c1604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7fb413fff700 1 -- 172.21.15.92:0/180912183 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4140b1d10 msgr2=0x7fb41414b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7fb413fff700 1 --2- 172.21.15.92:0/180912183 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4140b1d10 0x7fb41414b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7fb413fff700 1 -- 172.21.15.92:0/180912183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb41415d8b0 msgr2=0x7fb41414c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7fb413fff700 1 --2- 172.21.15.92:0/180912183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb41415d8b0 0x7fb41414c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7fb413fff700 1 -- 172.21.15.92:0/180912183 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40c009a20 con 0x7fb414068cd0 2023-10-06T10:38:33.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7fb1f183e700 1 -- 172.21.15.92:0/1622032140 >> v1:172.21.15.124:6789/0 conn(0x7fb1ec069c30 legacy=0x7fb1ec15d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7fb1f183e700 1 -- 172.21.15.92:0/1622032140 >> v1:172.21.15.92:6789/0 conn(0x7fb1ec159a20 legacy=0x7fb1ec069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7fb1f183e700 1 -- 172.21.15.92:0/1622032140 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ec15dfd0 con 0x7fb1ec15dab0 2023-10-06T10:38:33.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7f1a63fff700 1 -- 172.21.15.92:0/3879634750 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 893411850 0 0) 0x7f1a6c0c7dd0 con 0x7f1a6c0c8260 2023-10-06T10:38:33.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7f1a63fff700 1 -- 172.21.15.92:0/3879634750 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a58003570 con 0x7f1a6c0c8260 2023-10-06T10:38:33.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7f1a71929700 1 -- 172.21.15.92:0/3879634750 >> v2:172.21.15.124:3300/0 conn(0x7f1a6c155820 msgr2=0x7f1a6c0ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:33.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7f1a71929700 1 --2- 172.21.15.92:0/3879634750 >> v2:172.21.15.124:3300/0 conn(0x7f1a6c155820 0x7f1a6c0ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7f1a71929700 1 -- 172.21.15.92:0/3879634750 >> v1:172.21.15.124:6789/0 conn(0x7f1a6c0c8260 legacy=0x7f1a6c069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7f1a71929700 1 -- 172.21.15.92:0/3879634750 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6c0c71c0 con 0x7f1a6c069c30 2023-10-06T10:38:33.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7f789d36b700 1 Processor -- start 2023-10-06T10:38:33.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7f626c760700 1 -- 172.21.15.92:0/4225089129 wait complete. 2023-10-06T10:38:33.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.452+0000 7fb413fff700 1 --2- 172.21.15.92:0/180912183 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb414068cd0 0x7fb41414b410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb40800b600 tx=0x7fb4080091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7fb3fbfff700 1 -- 172.21.15.92:0/180912183 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40800f510 con 0x7fb414068cd0 2023-10-06T10:38:33.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7fb418abd700 1 --2- 172.21.15.92:0/180912183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb41415d8b0 0x7fb41414c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7fb41a520700 1 -- 172.21.15.92:0/180912183 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb414162970 con 0x7fb414068cd0 2023-10-06T10:38:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7f789d36b700 1 -- start start 2023-10-06T10:38:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7feef064f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feee80707d0 0x7feee816df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7feef064f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee8070c20 0x7feee8148770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7feef064f700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feee80ac440 0x7feee815deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7feef064f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee816e480 con 0x7feee8070c20 2023-10-06T10:38:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7feef064f700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee815f580 con 0x7feee80ac440 2023-10-06T10:38:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7feef064f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee815e970 con 0x7feee80707d0 2023-10-06T10:38:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7fb1f183e700 1 --2- 172.21.15.92:0/1622032140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1ec15dab0 0x7fb1ec164700 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4006fd0 tx=0x7fb1e40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c902661854f3299c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7f1a71929700 1 --2- 172.21.15.92:0/3879634750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a6c069c30 0x7f1a6c1604a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c06ab70 tx=0x7f1a680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2f0a51c3e47268 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7feeeebec700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feee80ac440 0x7feee815deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7feeeebec700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feee80ac440 0x7feee815deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:47350/0 (socket says 172.21.15.92:47350) 2023-10-06T10:38:33.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7feeeebec700 1 -- 172.21.15.92:0/735756471 learned_addr learned my addr 172.21.15.92:0/735756471 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7fb41a520700 1 -- 172.21.15.92:0/180912183 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb414162f30 con 0x7fb414068cd0 2023-10-06T10:38:33.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7fb3fbfff700 1 -- 172.21.15.92:0/180912183 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40800f670 con 0x7fb414068cd0 2023-10-06T10:38:33.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7fb3fbfff700 1 -- 172.21.15.92:0/180912183 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40800fa30 con 0x7fb414068cd0 2023-10-06T10:38:33.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7feeedbea700 1 --2- 172.21.15.92:0/735756471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee8070c20 0x7feee8148770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7f1a63fff700 1 -- 172.21.15.92:0/3879634750 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a68005c60 con 0x7f1a6c069c30 2023-10-06T10:38:33.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7f1a63fff700 1 -- 172.21.15.92:0/3879634750 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a68005dc0 con 0x7f1a6c069c30 2023-10-06T10:38:33.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7f1a63fff700 1 -- 172.21.15.92:0/3879634750 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a68006280 con 0x7f1a6c069c30 2023-10-06T10:38:33.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7feeeebec700 1 -- 172.21.15.92:0/735756471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feee80707d0 msgr2=0x7feee816df40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T10:38:33.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7feeeebec700 1 --2- 172.21.15.92:0/735756471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feee80707d0 0x7feee816df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7feeeebec700 1 -- 172.21.15.92:0/735756471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee8070c20 msgr2=0x7feee8148770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7feeeebec700 1 --2- 172.21.15.92:0/735756471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee8070c20 0x7feee8148770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7feeeebec700 1 -- 172.21.15.92:0/735756471 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed80094f0 con 0x7feee80ac440 2023-10-06T10:38:33.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.453+0000 7feeedbea700 1 --2- 172.21.15.92:0/735756471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee8070c20 0x7feee8148770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:33.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7fb1ebfff700 1 -- 172.21.15.92:0/1622032140 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4005c60 con 0x7fb1ec15dab0 2023-10-06T10:38:33.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7feeeebec700 1 --2- 172.21.15.92:0/735756471 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feee80ac440 0x7feee815deb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feee0009e90 tx=0x7feee000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7fb1ebfff700 1 -- 172.21.15.92:0/1622032140 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e4005dc0 con 0x7fb1ec15dab0 2023-10-06T10:38:33.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7fb1ebfff700 1 -- 172.21.15.92:0/1622032140 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4006260 con 0x7fb1ec15dab0 2023-10-06T10:38:33.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7fb683ac4700 1 -- 172.21.15.92:0/1722225009 shutdown_connections 2023-10-06T10:38:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7fb683ac4700 1 --2- 172.21.15.92:0/1722225009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb67c0685c0 0x7fb67c068a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7fb683ac4700 1 --2- 172.21.15.92:0/1722225009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb67c0b2420 0x7fb67c15dc80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7fb683ac4700 1 -- 172.21.15.92:0/1722225009 >> 172.21.15.92:0/1722225009 conn(0x7fb67c06de40 msgr2=0x7fb67c06e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7f789d36b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f789806e3d0 0x7f78980af510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7f789d36b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78980c8260 0x7f78980b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7f1a7338c700 1 -- 172.21.15.92:0/3879634750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a6c069c30 msgr2=0x7f1a6c1604a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7f1a7338c700 1 --2- 172.21.15.92:0/3879634750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a6c069c30 0x7f1a6c1604a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c06ab70 tx=0x7f1a680091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7f789d36b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78980cbc20 0x7f78980bebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7f789d36b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7898146610 con 0x7f78980c8260 2023-10-06T10:38:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7f789d36b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7898145d70 con 0x7f789806e3d0 2023-10-06T10:38:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7f789d36b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78980bf6d0 con 0x7f78980cbc20 2023-10-06T10:38:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7feede7fc700 1 -- 172.21.15.92:0/735756471 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee000cc20 con 0x7feee80ac440 2023-10-06T10:38:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7feede7fc700 1 -- 172.21.15.92:0/735756471 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee000f040 con 0x7feee80ac440 2023-10-06T10:38:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7feede7fc700 1 -- 172.21.15.92:0/735756471 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee0014480 con 0x7feee80ac440 2023-10-06T10:38:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7feef064f700 1 -- 172.21.15.92:0/735756471 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee815ec50 con 0x7feee80ac440 2023-10-06T10:38:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7feef064f700 1 -- 172.21.15.92:0/735756471 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee815f210 con 0x7feee80ac440 2023-10-06T10:38:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7f626c760700 1 Processor -- start 2023-10-06T10:38:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7f1a7338c700 1 -- 172.21.15.92:0/3879634750 shutdown_connections 2023-10-06T10:38:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7f1a7338c700 1 --2- 172.21.15.92:0/3879634750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a6c069c30 0x7f1a6c1604a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7f1a7338c700 1 --2- 172.21.15.92:0/3879634750 >> v2:172.21.15.124:3300/0 conn(0x7f1a6c155820 0x7f1a6c0ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7f1a7338c700 1 -- 172.21.15.92:0/3879634750 >> 172.21.15.92:0/3879634750 conn(0x7f1a6c068080 msgr2=0x7f1a6c0cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7f78967fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78980c8260 0x7f78980b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7f78967fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78980c8260 0x7f78980b1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36700/0 (socket says 172.21.15.92:36700) 2023-10-06T10:38:33.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7f78967fc700 1 -- 172.21.15.92:0/3928874897 learned_addr learned my addr 172.21.15.92:0/3928874897 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7f7896ffd700 1 --2- 172.21.15.92:0/3928874897 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f789806e3d0 0x7f78980af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7fb683ac4700 1 -- 172.21.15.92:0/1722225009 shutdown_connections 2023-10-06T10:38:33.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.454+0000 7f1a7338c700 1 -- 172.21.15.92:0/3879634750 shutdown_connections 2023-10-06T10:38:33.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.455+0000 7fb683ac4700 1 -- 172.21.15.92:0/1722225009 wait complete. 2023-10-06T10:38:33.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.455+0000 7fb1f32a1700 1 -- 172.21.15.92:0/1622032140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1ec15dab0 msgr2=0x7fb1ec164700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.455+0000 7fb1f32a1700 1 --2- 172.21.15.92:0/1622032140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1ec15dab0 0x7fb1ec164700 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4006fd0 tx=0x7fb1e40091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.455+0000 7f78967fc700 1 -- 172.21.15.92:0/3928874897 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78980cbc20 msgr2=0x7f78980bebe0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T10:38:33.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.455+0000 7f78967fc700 1 --2- 172.21.15.92:0/3928874897 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78980cbc20 0x7f78980bebe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.455+0000 7f78967fc700 1 -- 172.21.15.92:0/3928874897 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f789806e3d0 msgr2=0x7f78980af510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.455+0000 7f78967fc700 1 --2- 172.21.15.92:0/3928874897 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f789806e3d0 0x7f78980af510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.455+0000 7f78967fc700 1 -- 172.21.15.92:0/3928874897 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7890009ae0 con 0x7f78980c8260 2023-10-06T10:38:33.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.455+0000 7fb41a520700 1 -- 172.21.15.92:0/180912183 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f40052b0 con 0x7fb414068cd0 2023-10-06T10:38:33.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.455+0000 7f78967fc700 1 --2- 172.21.15.92:0/3928874897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78980c8260 0x7f78980b1520 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f788c00c810 tx=0x7f788c007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.455+0000 7fb3fbfff700 1 -- 172.21.15.92:0/180912183 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb40801a400 con 0x7fb414068cd0 2023-10-06T10:38:33.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.455+0000 7fb1f32a1700 1 -- 172.21.15.92:0/1622032140 shutdown_connections 2023-10-06T10:38:33.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.455+0000 7fb1f32a1700 1 --2- 172.21.15.92:0/1622032140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1ec15dab0 0x7fb1ec164700 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.456+0000 7fb1f32a1700 1 -- 172.21.15.92:0/1622032140 >> 172.21.15.92:0/1622032140 conn(0x7fb1ec068080 msgr2=0x7fb1ec0cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.456+0000 7fb3fbfff700 1 --2- 172.21.15.92:0/180912183 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb400066f30 0x7fb4000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.456+0000 7fb3fbfff700 1 -- 172.21.15.92:0/180912183 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb4080843d0 con 0x7fb414068cd0 2023-10-06T10:38:33.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.455+0000 7f626c760700 1 -- start start 2023-10-06T10:38:33.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.456+0000 7feede7fc700 1 -- 172.21.15.92:0/735756471 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feee00147f0 con 0x7feee80ac440 2023-10-06T10:38:33.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.456+0000 7f626c760700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6250004c50 0x7f62641548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.456+0000 7f626c760700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6264154df0 0x7f62641521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.456+0000 7f626c760700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6264152720 0x7f62640b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.456+0000 7f626c760700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6264155d20 con 0x7f6264154df0 2023-10-06T10:38:33.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.456+0000 7f626c760700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6264152b70 con 0x7f6250004c50 2023-10-06T10:38:33.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.456+0000 7f626c760700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62640b1410 con 0x7f6264152720 2023-10-06T10:38:33.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.456+0000 7f6269cfb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6264154df0 0x7f62641521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.456+0000 7fb683ac4700 1 Processor -- start 2023-10-06T10:38:33.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.456+0000 7f6269cfb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6264154df0 0x7f62641521e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36708/0 (socket says 172.21.15.92:36708) 2023-10-06T10:38:33.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.456+0000 7f6269cfb700 1 -- 172.21.15.92:0/2382776264 learned_addr learned my addr 172.21.15.92:0/2382776264 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.456+0000 7f626acfd700 1 --2- 172.21.15.92:0/2382776264 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6264152720 0x7f62640b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.456+0000 7f626a4fc700 1 --2- 172.21.15.92:0/2382776264 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6250004c50 0x7f62641548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.456+0000 7fb683ac4700 1 -- start start 2023-10-06T10:38:33.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.455+0000 7f787effd700 1 -- 172.21.15.92:0/3928874897 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c011590 con 0x7f78980c8260 2023-10-06T10:38:33.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.455+0000 7f787effd700 1 -- 172.21.15.92:0/3928874897 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788c011710 con 0x7f78980c8260 2023-10-06T10:38:33.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.455+0000 7f787effd700 1 -- 172.21.15.92:0/3928874897 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c011ad0 con 0x7f78980c8260 2023-10-06T10:38:33.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.456+0000 7f6269cfb700 1 -- 172.21.15.92:0/2382776264 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6264152720 msgr2=0x7f62640b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.456+0000 7f6269cfb700 1 --2- 172.21.15.92:0/2382776264 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6264152720 0x7f62640b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7f6269cfb700 1 -- 172.21.15.92:0/2382776264 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6250004c50 msgr2=0x7f62641548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7f6269cfb700 1 --2- 172.21.15.92:0/2382776264 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6250004c50 0x7f62641548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7f6269cfb700 1 -- 172.21.15.92:0/2382776264 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62640b16f0 con 0x7f6264154df0 2023-10-06T10:38:33.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7fb1f32a1700 1 -- 172.21.15.92:0/1622032140 shutdown_connections 2023-10-06T10:38:33.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7fb1f32a1700 1 -- 172.21.15.92:0/1622032140 wait complete. 2023-10-06T10:38:33.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7f6269cfb700 1 --2- 172.21.15.92:0/2382776264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6264154df0 0x7f62641521e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f626000b460 tx=0x7f6260006f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.455+0000 7f1a7338c700 1 -- 172.21.15.92:0/3879634750 wait complete. 2023-10-06T10:38:33.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7f1a7338c700 1 Processor -- start 2023-10-06T10:38:33.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7fb4137fe700 1 --2- 172.21.15.92:0/180912183 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb400066f30 0x7fb4000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7f625a7fc700 1 -- 172.21.15.92:0/2382776264 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260011400 con 0x7f6264154df0 2023-10-06T10:38:33.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.456+0000 7feede7fc700 1 --2- 172.21.15.92:0/735756471 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7feecc067080 0x7feecc069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7feef064f700 1 -- 172.21.15.92:0/735756471 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed00052b0 con 0x7feee80ac440 2023-10-06T10:38:33.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.458+0000 7feede7fc700 1 -- 172.21.15.92:0/735756471 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7feee0085400 con 0x7feee80ac440 2023-10-06T10:38:33.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.458+0000 7feeee3eb700 1 --2- 172.21.15.92:0/735756471 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7feecc067080 0x7feecc069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7f625a7fc700 1 -- 172.21.15.92:0/2382776264 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6260011560 con 0x7f6264154df0 2023-10-06T10:38:33.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7f625a7fc700 1 -- 172.21.15.92:0/2382776264 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260011920 con 0x7f6264154df0 2023-10-06T10:38:33.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7f626c760700 1 -- 172.21.15.92:0/2382776264 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62640b19d0 con 0x7f6264154df0 2023-10-06T10:38:33.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7f626c760700 1 -- 172.21.15.92:0/2382776264 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62640b0c90 con 0x7f6264154df0 2023-10-06T10:38:33.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7fb683ac4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb67c0685c0 0x7fb67c162840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7fb683ac4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb67c0b2420 0x7fb67c160660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7fb683ac4700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb67c15d8b0 0x7fb67c1402c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7fb683ac4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c1419d0 con 0x7fb67c0685c0 2023-10-06T10:38:33.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7fb683ac4700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c140dc0 con 0x7fb67c15d8b0 2023-10-06T10:38:33.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7fb683ac4700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c1410a0 con 0x7fb67c0b2420 2023-10-06T10:38:33.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7fb681860700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb67c0685c0 0x7fb67c162840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7fb681860700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb67c0685c0 0x7fb67c162840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36714/0 (socket says 172.21.15.92:36714) 2023-10-06T10:38:33.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.457+0000 7fb681860700 1 -- 172.21.15.92:0/562073311 learned_addr learned my addr 172.21.15.92:0/562073311 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.458+0000 7fb68105f700 1 --2- 172.21.15.92:0/562073311 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb67c0b2420 0x7fb67c160660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.458+0000 7fb681860700 1 -- 172.21.15.92:0/562073311 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb67c0b2420 msgr2=0x7fb67c160660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.458+0000 7fb681860700 1 --2- 172.21.15.92:0/562073311 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb67c0b2420 0x7fb67c160660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.458+0000 7fb681860700 1 -- 172.21.15.92:0/562073311 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb67c15d8b0 msgr2=0x7fb67c1402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.458+0000 7fb682061700 1 --2- 172.21.15.92:0/562073311 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb67c15d8b0 0x7fb67c1402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.458+0000 7fb681860700 1 --2- 172.21.15.92:0/562073311 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb67c15d8b0 0x7fb67c1402c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.458+0000 7fb681860700 1 -- 172.21.15.92:0/562073311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6780094f0 con 0x7fb67c0685c0 2023-10-06T10:38:33.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.458+0000 7feeee3eb700 1 --2- 172.21.15.92:0/735756471 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7feecc067080 0x7feecc069530 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feee4009680 tx=0x7feee4006ba0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.458+0000 7fb1f32a1700 1 Processor -- start 2023-10-06T10:38:33.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.458+0000 7fb681860700 1 --2- 172.21.15.92:0/562073311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb67c0685c0 0x7fb67c162840 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb67400c8c0 tx=0x7fb674001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.458+0000 7fb671ffb700 1 -- 172.21.15.92:0/562073311 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67400ce60 con 0x7fb67c0685c0 2023-10-06T10:38:33.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.458+0000 7fb683ac4700 1 -- 172.21.15.92:0/562073311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67c141380 con 0x7fb67c0685c0 2023-10-06T10:38:33.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.458+0000 7fb4137fe700 1 --2- 172.21.15.92:0/180912183 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb400066f30 0x7fb4000693e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb414168950 tx=0x7fb404007eb0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.458+0000 7fb671ffb700 1 -- 172.21.15.92:0/562073311 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67400a7e0 con 0x7fb67c0685c0 2023-10-06T10:38:33.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.459+0000 7fb671ffb700 1 -- 172.21.15.92:0/562073311 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67400aba0 con 0x7fb67c0685c0 2023-10-06T10:38:33.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.458+0000 7fb1f32a1700 1 -- start start 2023-10-06T10:38:33.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.458+0000 7f1a7338c700 1 -- start start 2023-10-06T10:38:33.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.459+0000 7f1a7338c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a6c069c30 0x7f1a6c154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.459+0000 7f1a7338c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a6c0c8260 0x7f1a6c179990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.459+0000 7f1a7338c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a6c155820 0x7f1a6c179ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.459+0000 7f1a7338c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0beb90 con 0x7f1a6c155820 2023-10-06T10:38:33.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.459+0000 7f1a7338c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c1550c0 con 0x7f1a6c0c8260 2023-10-06T10:38:33.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.459+0000 7f1a7338c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0c1c90 con 0x7f1a6c069c30 2023-10-06T10:38:33.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.459+0000 7f1a71128700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a6c069c30 0x7f1a6c154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.459+0000 7f1a71128700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a6c069c30 0x7f1a6c154b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:51846/0 (socket says 172.21.15.92:51846) 2023-10-06T10:38:33.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.459+0000 7f1a71128700 1 -- 172.21.15.92:0/4259662525 learned_addr learned my addr 172.21.15.92:0/4259662525 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.459+0000 7f626c760700 1 -- 172.21.15.92:0/2382776264 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f624c0052b0 con 0x7f6264154df0 2023-10-06T10:38:33.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7f1a70927700 1 --2- 172.21.15.92:0/4259662525 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a6c0c8260 0x7f1a6c179990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7fb683ac4700 1 -- 172.21.15.92:0/562073311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67c160ba0 con 0x7fb67c0685c0 2023-10-06T10:38:33.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7fb1f32a1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1ec069c30 0x7fb1ec0aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7fb1f32a1700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1ec159a20 0x7fb1ec0c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7fb1f32a1700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1ec15dab0 0x7fb1ec0c0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7fb1f32a1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ec0c6f40 con 0x7fb1ec069c30 2023-10-06T10:38:33.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7fb1f32a1700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ec153ec0 con 0x7fb1ec159a20 2023-10-06T10:38:33.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7fb1f32a1700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ec0b0370 con 0x7fb1ec15dab0 2023-10-06T10:38:33.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7f625a7fc700 1 -- 172.21.15.92:0/2382776264 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6260011cd0 con 0x7f6264154df0 2023-10-06T10:38:33.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7fb1f083c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1ec159a20 0x7fb1ec0c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7fb1f083c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1ec159a20 0x7fb1ec0c6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:47390/0 (socket says 172.21.15.92:47390) 2023-10-06T10:38:33.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7fb1f083c700 1 -- 172.21.15.92:0/524391032 learned_addr learned my addr 172.21.15.92:0/524391032 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7fb1f183e700 1 --2- 172.21.15.92:0/524391032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1ec15dab0 0x7fb1ec0c0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.461+0000 7fb1f083c700 1 -- 172.21.15.92:0/524391032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1ec15dab0 msgr2=0x7fb1ec0c0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.461+0000 7fb1f083c700 1 --2- 172.21.15.92:0/524391032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1ec15dab0 0x7fb1ec0c0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.461+0000 7fb1f083c700 1 -- 172.21.15.92:0/524391032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1ec069c30 msgr2=0x7fb1ec0aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.461+0000 7fb1f083c700 1 --2- 172.21.15.92:0/524391032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1ec069c30 0x7fb1ec0aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.461+0000 7fb1f083c700 1 -- 172.21.15.92:0/524391032 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e4009420 con 0x7fb1ec159a20 2023-10-06T10:38:33.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.461+0000 7f625a7fc700 1 --2- 172.21.15.92:0/2382776264 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6248066fb0 0x7f6248069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7f789d36b700 1 -- 172.21.15.92:0/3928874897 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78980b1c30 con 0x7f78980c8260 2023-10-06T10:38:33.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7f789d36b700 1 -- 172.21.15.92:0/3928874897 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78981462a0 con 0x7f78980c8260 2023-10-06T10:38:33.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.461+0000 7f625a7fc700 1 -- 172.21.15.92:0/2382776264 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f6260084ce0 con 0x7f6264154df0 2023-10-06T10:38:33.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.461+0000 7fb1f083c700 1 --2- 172.21.15.92:0/524391032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1ec159a20 0x7fb1ec0c6a00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800c560 tx=0x7fb1d8007dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.461+0000 7fb671ffb700 1 -- 172.21.15.92:0/562073311 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb674003030 con 0x7fb67c0685c0 2023-10-06T10:38:33.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.461+0000 7fb671ffb700 1 --2- 172.21.15.92:0/562073311 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb660066f30 0x7fb6600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.462+0000 7fb68105f700 1 --2- 172.21.15.92:0/562073311 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb660066f30 0x7fb6600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.461+0000 7fc730725700 1 Processor -- start 2023-10-06T10:38:33.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.461+0000 7fc730725700 1 -- start start 2023-10-06T10:38:33.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.462+0000 7f626a4fc700 1 --2- 172.21.15.92:0/2382776264 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6248066fb0 0x7f6248069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.462+0000 7fb3fbfff700 1 -- 172.21.15.92:0/180912183 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb408051270 con 0x7fb414068cd0 2023-10-06T10:38:33.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7f1a71128700 1 -- 172.21.15.92:0/4259662525 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a6c0c8260 msgr2=0x7f1a6c179990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7f1a71128700 1 --2- 172.21.15.92:0/4259662525 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a6c0c8260 0x7f1a6c179990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7f1a71128700 1 -- 172.21.15.92:0/4259662525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a6c155820 msgr2=0x7f1a6c179ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7f1a71929700 1 --2- 172.21.15.92:0/4259662525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a6c155820 0x7f1a6c179ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7f1a71128700 1 --2- 172.21.15.92:0/4259662525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a6c155820 0x7f1a6c179ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7f1a71128700 1 -- 172.21.15.92:0/4259662525 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a68009420 con 0x7f1a6c069c30 2023-10-06T10:38:33.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7f1a71929700 1 --2- 172.21.15.92:0/4259662525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a6c155820 0x7f1a6c179ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:33.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7f1a71128700 1 --2- 172.21.15.92:0/4259662525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a6c069c30 0x7f1a6c154b80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c00e980 tx=0x7f1a5c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7f1a617fa700 1 -- 172.21.15.92:0/4259662525 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c009520 con 0x7f1a6c069c30 2023-10-06T10:38:33.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7f1a617fa700 1 -- 172.21.15.92:0/4259662525 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5c00c850 con 0x7f1a6c069c30 2023-10-06T10:38:33.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7f1a617fa700 1 -- 172.21.15.92:0/4259662525 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c00cbf0 con 0x7f1a6c069c30 2023-10-06T10:38:33.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.460+0000 7f1a7338c700 1 -- 172.21.15.92:0/4259662525 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6c0c1f70 con 0x7f1a6c069c30 2023-10-06T10:38:33.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.461+0000 7f1a7338c700 1 -- 172.21.15.92:0/4259662525 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6c0c2530 con 0x7f1a6c069c30 2023-10-06T10:38:33.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.462+0000 7fb68105f700 1 --2- 172.21.15.92:0/562073311 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb660066f30 0x7fb6600693e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb67c15e1c0 tx=0x7fb678017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.462+0000 7f1a7338c700 1 -- 172.21.15.92:0/4259662525 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6c162970 con 0x7f1a6c069c30 2023-10-06T10:38:33.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.462+0000 7f789d36b700 1 -- 172.21.15.92:0/3928874897 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78981467c0 con 0x7f78980c8260 2023-10-06T10:38:33.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.462+0000 7f1a617fa700 1 -- 172.21.15.92:0/4259662525 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1a5c013470 con 0x7f1a6c069c30 2023-10-06T10:38:33.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.462+0000 7fc730725700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc7280878b0 0x7fc72806d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.462+0000 7fc730725700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc728069c30 0x7fc7280aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.462+0000 7fc730725700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7280c7230 con 0x7fc72806d890 2023-10-06T10:38:33.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.462+0000 7fc730725700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7280c89e0 con 0x7fc728069c30 2023-10-06T10:38:33.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.462+0000 7fc730725700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7280c7040 con 0x7fc7280878b0 2023-10-06T10:38:33.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.463+0000 7f626a4fc700 1 --2- 172.21.15.92:0/2382776264 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6248066fb0 0x7f6248069460 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f625c00acb0 tx=0x7f625c015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.462+0000 7fc72ecc2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc728069c30 0x7fc7280aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.463+0000 7fc72ecc2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc728069c30 0x7fc7280aeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36738/0 (socket says 172.21.15.92:36738) 2023-10-06T10:38:33.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.463+0000 7fc72ecc2700 1 -- 172.21.15.92:0/4234824420 learned_addr learned my addr 172.21.15.92:0/4234824420 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.463+0000 7fc72e4c1700 1 --2- 172.21.15.92:0/4234824420 >> v2:172.21.15.124:3300/0 conn(0x7fc7280878b0 0x7fc72806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.463+0000 7fb683ac4700 1 -- 172.21.15.92:0/562073311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb67c064a90 con 0x7fb67c0685c0 2023-10-06T10:38:33.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.463+0000 7fc72ecc2700 1 -- 172.21.15.92:0/4234824420 >> v2:172.21.15.124:3300/0 conn(0x7fc7280878b0 msgr2=0x7fc72806d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:33.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.463+0000 7fc72ecc2700 1 --2- 172.21.15.92:0/4234824420 >> v2:172.21.15.124:3300/0 conn(0x7fc7280878b0 0x7fc72806d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.463+0000 7fc72ecc2700 1 -- 172.21.15.92:0/4234824420 >> v1:172.21.15.92:6789/0 conn(0x7fc72806d890 legacy=0x7fc728069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.463+0000 7fc72ecc2700 1 -- 172.21.15.92:0/4234824420 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7280c7510 con 0x7fc728069c30 2023-10-06T10:38:33.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.463+0000 7f1a617fa700 1 --2- 172.21.15.92:0/4259662525 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1a50066f30 0x7f1a500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.463+0000 7fc72e4c1700 1 --2- 172.21.15.92:0/4234824420 >> v2:172.21.15.124:3300/0 conn(0x7fc7280878b0 0x7fc72806d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:33.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.463+0000 7f1a617fa700 1 -- 172.21.15.92:0/4259662525 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f1a5c084aa0 con 0x7f1a6c069c30 2023-10-06T10:38:33.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.463+0000 7fc72ecc2700 1 --2- 172.21.15.92:0/4234824420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc728069c30 0x7fc7280aeeb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc72806cf00 tx=0x7fc720009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2823dd39f4cd41d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.464+0000 7f1a70927700 1 --2- 172.21.15.92:0/4259662525 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1a50066f30 0x7f1a500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.464+0000 7f1a70927700 1 --2- 172.21.15.92:0/4259662525 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1a50066f30 0x7f1a500693e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c06d110 tx=0x7f1a64007f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.465+0000 7fb1e97fa700 1 -- 172.21.15.92:0/524391032 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8005c00 con 0x7fb1ec159a20 2023-10-06T10:38:33.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.465+0000 7fb1e97fa700 1 -- 172.21.15.92:0/524391032 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d8005d60 con 0x7fb1ec159a20 2023-10-06T10:38:33.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.465+0000 7fb1e97fa700 1 -- 172.21.15.92:0/524391032 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8007300 con 0x7fb1ec159a20 2023-10-06T10:38:33.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.465+0000 7fb1f32a1700 1 -- 172.21.15.92:0/524391032 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1ec0c0780 con 0x7fb1ec159a20 2023-10-06T10:38:33.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.465+0000 7fb1f32a1700 1 -- 172.21.15.92:0/524391032 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1ec0af7e0 con 0x7fb1ec159a20 2023-10-06T10:38:33.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.465+0000 7feede7fc700 1 -- 172.21.15.92:0/735756471 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feee0055c90 con 0x7feee80ac440 2023-10-06T10:38:33.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.466+0000 7f625a7fc700 1 -- 172.21.15.92:0/2382776264 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6260051b60 con 0x7f6264154df0 2023-10-06T10:38:33.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.466+0000 7fb1f32a1700 1 -- 172.21.15.92:0/524391032 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1ec064a90 con 0x7fb1ec159a20 2023-10-06T10:38:33.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.467+0000 7fb1e97fa700 1 -- 172.21.15.92:0/524391032 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb1d8007670 con 0x7fb1ec159a20 2023-10-06T10:38:33.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.467+0000 7fb1e97fa700 1 --2- 172.21.15.92:0/524391032 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb1d0066f30 0x7fb1d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.467+0000 7fb1e97fa700 1 -- 172.21.15.92:0/524391032 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb1d8083e20 con 0x7fb1ec159a20 2023-10-06T10:38:33.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.468+0000 7fb671ffb700 1 -- 172.21.15.92:0/562073311 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb674055350 con 0x7fb67c0685c0 2023-10-06T10:38:33.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.468+0000 7f1a617fa700 1 -- 172.21.15.92:0/4259662525 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a5c0518c0 con 0x7f1a6c069c30 2023-10-06T10:38:33.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.469+0000 7f787effd700 1 -- 172.21.15.92:0/3928874897 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f788c01c440 con 0x7f78980c8260 2023-10-06T10:38:33.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.470+0000 7f787effd700 1 --2- 172.21.15.92:0/3928874897 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7884066f30 0x7f78840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.470+0000 7f787effd700 1 -- 172.21.15.92:0/3928874897 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f788c084450 con 0x7f78980c8260 2023-10-06T10:38:33.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.470+0000 7f7896ffd700 1 --2- 172.21.15.92:0/3928874897 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7884066f30 0x7f78840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.470+0000 7f7896ffd700 1 --2- 172.21.15.92:0/3928874897 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7884066f30 0x7f78840693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7890009880 tx=0x7f7890017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.471+0000 7fb1f103d700 1 --2- 172.21.15.92:0/524391032 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb1d0066f30 0x7fb1d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.471+0000 7fb1f103d700 1 --2- 172.21.15.92:0/524391032 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb1d0066f30 0x7fb1d00693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0001cf0 tx=0x7fb1e0007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.472+0000 7fb1e97fa700 1 -- 172.21.15.92:0/524391032 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb1d8050c40 con 0x7fb1ec159a20 2023-10-06T10:38:33.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.472+0000 7fb671ffb700 1 -- 172.21.15.92:0/562073311 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb674089060 con 0x7fb67c0685c0 2023-10-06T10:38:33.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.480+0000 7f787effd700 1 -- 172.21.15.92:0/3928874897 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f788c051250 con 0x7f78980c8260 2023-10-06T10:38:33.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.470+0000 7fc72d4bf700 1 -- 172.21.15.92:0/4234824420 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72000e070 con 0x7fc728069c30 2023-10-06T10:38:33.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.480+0000 7fc72d4bf700 1 -- 172.21.15.92:0/4234824420 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72000ba20 con 0x7fc728069c30 2023-10-06T10:38:33.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.480+0000 7fc72d4bf700 1 -- 172.21.15.92:0/4234824420 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7200092e0 con 0x7fc728069c30 2023-10-06T10:38:33.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.481+0000 7fc730725700 1 -- 172.21.15.92:0/4234824420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc728069c30 msgr2=0x7fc7280aeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.481+0000 7fc730725700 1 --2- 172.21.15.92:0/4234824420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc728069c30 0x7fc7280aeeb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc72806cf00 tx=0x7fc720009840 comp rx=0 tx=0).stop 2023-10-06T10:38:33.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.481+0000 7fc730725700 1 -- 172.21.15.92:0/4234824420 shutdown_connections 2023-10-06T10:38:33.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.481+0000 7fc730725700 1 --2- 172.21.15.92:0/4234824420 >> v2:172.21.15.124:3300/0 conn(0x7fc7280878b0 0x7fc72806d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.481+0000 7fc730725700 1 --2- 172.21.15.92:0/4234824420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc728069c30 0x7fc7280aeeb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.481+0000 7fc730725700 1 -- 172.21.15.92:0/4234824420 >> 172.21.15.92:0/4234824420 conn(0x7fc728068080 msgr2=0x7fc7280cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.488+0000 7fc730725700 1 -- 172.21.15.92:0/4234824420 shutdown_connections 2023-10-06T10:38:33.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.491+0000 7fc730725700 1 -- 172.21.15.92:0/4234824420 wait complete. 2023-10-06T10:38:33.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.492+0000 7fc730725700 1 Processor -- start 2023-10-06T10:38:33.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.493+0000 7fc730725700 1 -- start start 2023-10-06T10:38:33.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.494+0000 7fc730725700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc728069c30 0x7fc728086aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.494+0000 7fc730725700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc72806d890 0x7fc72817b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.494+0000 7fc730725700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7280878b0 0x7fc7280c3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.494+0000 7fc730725700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72816a230 con 0x7fc7280878b0 2023-10-06T10:38:33.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.494+0000 7fc730725700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc728168f40 con 0x7fc72806d890 2023-10-06T10:38:33.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.494+0000 7fc730725700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc728169da0 con 0x7fc728069c30 2023-10-06T10:38:33.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.494+0000 7fc72dcc0700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc72806d890 0x7fc72817b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.494+0000 7fc72dcc0700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc72806d890 0x7fc72817b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:47406/0 (socket says 172.21.15.92:47406) 2023-10-06T10:38:33.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.494+0000 7fc72dcc0700 1 -- 172.21.15.92:0/2247725494 learned_addr learned my addr 172.21.15.92:0/2247725494 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:33.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.495+0000 7fc72dcc0700 1 -- 172.21.15.92:0/2247725494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc728069c30 msgr2=0x7fc728086aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.495+0000 7fc72dcc0700 1 --2- 172.21.15.92:0/2247725494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc728069c30 0x7fc728086aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.495+0000 7fc72dcc0700 1 -- 172.21.15.92:0/2247725494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7280878b0 msgr2=0x7fc7280c3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.495+0000 7fc72dcc0700 1 --2- 172.21.15.92:0/2247725494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7280878b0 0x7fc7280c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.495+0000 7fc72dcc0700 1 -- 172.21.15.92:0/2247725494 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72400d8d0 con 0x7fc72806d890 2023-10-06T10:38:33.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.495+0000 7fc72dcc0700 1 --2- 172.21.15.92:0/2247725494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc72806d890 0x7fc72817b210 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc71800c540 tx=0x7fc71800c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.496+0000 7fc71e7fc700 1 -- 172.21.15.92:0/2247725494 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc718011590 con 0x7fc72806d890 2023-10-06T10:38:33.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.496+0000 7fc71e7fc700 1 -- 172.21.15.92:0/2247725494 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc718011710 con 0x7fc72806d890 2023-10-06T10:38:33.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.496+0000 7fc71e7fc700 1 -- 172.21.15.92:0/2247725494 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc718011ad0 con 0x7fc72806d890 2023-10-06T10:38:33.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.496+0000 7fc730725700 1 -- 172.21.15.92:0/2247725494 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc720009ae0 con 0x7fc72806d890 2023-10-06T10:38:33.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.496+0000 7fc730725700 1 -- 172.21.15.92:0/2247725494 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc728169210 con 0x7fc72806d890 2023-10-06T10:38:33.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.499+0000 7fc71e7fc700 1 -- 172.21.15.92:0/2247725494 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc71801b3b0 con 0x7fc72806d890 2023-10-06T10:38:33.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.500+0000 7fc71e7fc700 1 --2- 172.21.15.92:0/2247725494 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc70c067080 0x7fc70c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:33.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.500+0000 7fc71e7fc700 1 -- 172.21.15.92:0/2247725494 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fc718083e70 con 0x7fc72806d890 2023-10-06T10:38:33.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.501+0000 7fc730725700 1 -- 172.21.15.92:0/2247725494 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7100052b0 con 0x7fc72806d890 2023-10-06T10:38:33.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.501+0000 7fc72e4c1700 1 --2- 172.21.15.92:0/2247725494 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc70c067080 0x7fc70c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:33.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.502+0000 7fc72e4c1700 1 --2- 172.21.15.92:0/2247725494 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc70c067080 0x7fc70c069530 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc72400df00 tx=0x7fc72400f040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:33.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.506+0000 7fc71e7fc700 1 -- 172.21.15.92:0/2247725494 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc718017080 con 0x7fc72806d890 2023-10-06T10:38:33.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.848+0000 7f1a7338c700 1 -- 172.21.15.92:0/4259662525 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f1a6c064a90 con 0x7f1a6c069c30 2023-10-06T10:38:33.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.849+0000 7f1a617fa700 1 -- 172.21.15.92:0/4259662525 <== mon.2 v2:172.21.15.124:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1a5c054ee0 con 0x7f1a6c069c30 2023-10-06T10:38:33.851 INFO:teuthology.orchestra.run.smithi092.stdout:55834574856 2023-10-06T10:38:33.854 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-10-06T10:38:33.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.856+0000 7fb683ac4700 1 -- 172.21.15.92:0/562073311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb67c0c90c0 con 0x7fb67c0685c0 2023-10-06T10:38:33.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.856+0000 7fb671ffb700 1 -- 172.21.15.92:0/562073311 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb674054a20 con 0x7fb67c0685c0 2023-10-06T10:38:33.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.856+0000 7f1a7338c700 1 -- 172.21.15.92:0/4259662525 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1a50066f30 msgr2=0x7f1a500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.856+0000 7f1a7338c700 1 --2- 172.21.15.92:0/4259662525 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1a50066f30 0x7f1a500693e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c06d110 tx=0x7f1a64007f90 comp rx=0 tx=0).stop 2023-10-06T10:38:33.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.856+0000 7f1a7338c700 1 -- 172.21.15.92:0/4259662525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a6c069c30 msgr2=0x7f1a6c154b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.856+0000 7f1a7338c700 1 --2- 172.21.15.92:0/4259662525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a6c069c30 0x7f1a6c154b80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c00e980 tx=0x7f1a5c009250 comp rx=0 tx=0).stop 2023-10-06T10:38:33.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.849+0000 7feef064f700 1 -- 172.21.15.92:0/735756471 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7feed0005560 con 0x7feee80ac440 2023-10-06T10:38:33.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.849+0000 7feede7fc700 1 -- 172.21.15.92:0/735756471 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7feee0018070 con 0x7feee80ac440 2023-10-06T10:38:33.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.856+0000 7feef064f700 1 -- 172.21.15.92:0/735756471 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7feecc067080 msgr2=0x7feecc069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.856+0000 7feef064f700 1 --2- 172.21.15.92:0/735756471 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7feecc067080 0x7feecc069530 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feee4009680 tx=0x7feee4006ba0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.856+0000 7feef064f700 1 -- 172.21.15.92:0/735756471 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feee80ac440 msgr2=0x7feee815deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.856+0000 7feef064f700 1 --2- 172.21.15.92:0/735756471 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feee80ac440 0x7feee815deb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feee0009e90 tx=0x7feee000b040 comp rx=0 tx=0).stop 2023-10-06T10:38:33.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.856+0000 7feef064f700 1 -- 172.21.15.92:0/735756471 shutdown_connections 2023-10-06T10:38:33.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.856+0000 7feef064f700 1 --2- 172.21.15.92:0/735756471 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7feecc067080 0x7feecc069530 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feee4009680 tx=0x7feee4006ba0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.856+0000 7feef064f700 1 --2- 172.21.15.92:0/735756471 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feee80ac440 0x7feee815deb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.856+0000 7feef064f700 1 --2- 172.21.15.92:0/735756471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feee80707d0 0x7feee816df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.856+0000 7feef064f700 1 --2- 172.21.15.92:0/735756471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee8070c20 0x7feee8148770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.856+0000 7feef064f700 1 -- 172.21.15.92:0/735756471 >> 172.21.15.92:0/735756471 conn(0x7feee806de40 msgr2=0x7feee81496e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.860 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-10-06T10:38:33.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.857+0000 7f1a7338c700 1 -- 172.21.15.92:0/4259662525 shutdown_connections 2023-10-06T10:38:33.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.857+0000 7f1a7338c700 1 --2- 172.21.15.92:0/4259662525 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1a50066f30 0x7f1a500693e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.857+0000 7f1a7338c700 1 --2- 172.21.15.92:0/4259662525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a6c155820 0x7f1a6c179ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.857+0000 7f1a7338c700 1 --2- 172.21.15.92:0/4259662525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a6c069c30 0x7f1a6c154b80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.857+0000 7f1a7338c700 1 --2- 172.21.15.92:0/4259662525 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a6c0c8260 0x7f1a6c179990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.857+0000 7f1a7338c700 1 -- 172.21.15.92:0/4259662525 >> 172.21.15.92:0/4259662525 conn(0x7f1a6c068080 msgr2=0x7f1a6c0adfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.857+0000 7f1a7338c700 1 -- 172.21.15.92:0/4259662525 shutdown_connections 2023-10-06T10:38:33.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.857+0000 7f626c760700 1 -- 172.21.15.92:0/2382776264 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f624c005830 con 0x7f6264154df0 2023-10-06T10:38:33.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.857+0000 7f1a7338c700 1 -- 172.21.15.92:0/4259662525 wait complete. 2023-10-06T10:38:33.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.857+0000 7fb41a520700 1 -- 172.21.15.92:0/180912183 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb3f4005830 con 0x7fb414068cd0 2023-10-06T10:38:33.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.858+0000 7f625a7fc700 1 -- 172.21.15.92:0/2382776264 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6260055180 con 0x7f6264154df0 2023-10-06T10:38:33.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.858+0000 7fb3fbfff700 1 -- 172.21.15.92:0/180912183 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb408054890 con 0x7fb414068cd0 2023-10-06T10:38:33.863 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-10-06T10:38:33.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.859+0000 7feef064f700 1 -- 172.21.15.92:0/735756471 shutdown_connections 2023-10-06T10:38:33.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.859+0000 7fb683ac4700 1 -- 172.21.15.92:0/562073311 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb660066f30 msgr2=0x7fb6600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.859+0000 7fb683ac4700 1 --2- 172.21.15.92:0/562073311 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb660066f30 0x7fb6600693e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb67c15e1c0 tx=0x7fb678017000 comp rx=0 tx=0).stop 2023-10-06T10:38:33.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.859+0000 7fb683ac4700 1 -- 172.21.15.92:0/562073311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb67c0685c0 msgr2=0x7fb67c162840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.859+0000 7fb683ac4700 1 --2- 172.21.15.92:0/562073311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb67c0685c0 0x7fb67c162840 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb67400c8c0 tx=0x7fb674001700 comp rx=0 tx=0).stop 2023-10-06T10:38:33.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.860+0000 7feef064f700 1 -- 172.21.15.92:0/735756471 wait complete. 2023-10-06T10:38:33.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.860+0000 7f626c760700 1 -- 172.21.15.92:0/2382776264 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6248066fb0 msgr2=0x7f6248069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.864 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-10-06T10:38:33.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.860+0000 7f626c760700 1 --2- 172.21.15.92:0/2382776264 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6248066fb0 0x7f6248069460 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f625c00acb0 tx=0x7f625c015000 comp rx=0 tx=0).stop 2023-10-06T10:38:33.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.860+0000 7f626c760700 1 -- 172.21.15.92:0/2382776264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6264154df0 msgr2=0x7f62641521e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.860+0000 7f626c760700 1 --2- 172.21.15.92:0/2382776264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6264154df0 0x7f62641521e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f626000b460 tx=0x7f6260006f70 comp rx=0 tx=0).stop 2023-10-06T10:38:33.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.860+0000 7f626c760700 1 -- 172.21.15.92:0/2382776264 shutdown_connections 2023-10-06T10:38:33.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.860+0000 7f626c760700 1 --2- 172.21.15.92:0/2382776264 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6248066fb0 0x7f6248069460 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.860+0000 7f626c760700 1 --2- 172.21.15.92:0/2382776264 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6264152720 0x7f62640b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.860+0000 7f626c760700 1 --2- 172.21.15.92:0/2382776264 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6250004c50 0x7f62641548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.860+0000 7f626c760700 1 --2- 172.21.15.92:0/2382776264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6264154df0 0x7f62641521e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.860+0000 7f626c760700 1 -- 172.21.15.92:0/2382776264 >> 172.21.15.92:0/2382776264 conn(0x7f6264068080 msgr2=0x7f6264153c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.860+0000 7f626c760700 1 -- 172.21.15.92:0/2382776264 shutdown_connections 2023-10-06T10:38:33.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.860+0000 7f626c760700 1 -- 172.21.15.92:0/2382776264 wait complete. 2023-10-06T10:38:33.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.863+0000 7fb1f32a1700 1 -- 172.21.15.92:0/524391032 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb1ec15b2d0 con 0x7fb1ec159a20 2023-10-06T10:38:33.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.863+0000 7fb1e97fa700 1 -- 172.21.15.92:0/524391032 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb1d8017020 con 0x7fb1ec159a20 2023-10-06T10:38:33.867 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-10-06T10:38:33.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.865+0000 7fb683ac4700 1 -- 172.21.15.92:0/562073311 shutdown_connections 2023-10-06T10:38:33.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.865+0000 7fb683ac4700 1 --2- 172.21.15.92:0/562073311 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb660066f30 0x7fb6600693e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.865+0000 7fb683ac4700 1 --2- 172.21.15.92:0/562073311 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb67c15d8b0 0x7fb67c1402c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.865+0000 7fb683ac4700 1 --2- 172.21.15.92:0/562073311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb67c0685c0 0x7fb67c162840 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.865+0000 7fb683ac4700 1 --2- 172.21.15.92:0/562073311 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb67c0b2420 0x7fb67c160660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.865+0000 7fb683ac4700 1 -- 172.21.15.92:0/562073311 >> 172.21.15.92:0/562073311 conn(0x7fb67c06de40 msgr2=0x7fb67c0cb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.867+0000 7fb41a520700 1 -- 172.21.15.92:0/180912183 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb400066f30 msgr2=0x7fb4000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.867+0000 7fb41a520700 1 --2- 172.21.15.92:0/180912183 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb400066f30 0x7fb4000693e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb414168950 tx=0x7fb404007eb0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.867+0000 7fb41a520700 1 -- 172.21.15.92:0/180912183 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb414068cd0 msgr2=0x7fb41414b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.867+0000 7fb41a520700 1 --2- 172.21.15.92:0/180912183 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb414068cd0 0x7fb41414b410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb40800b600 tx=0x7fb4080091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.868+0000 7fb41a520700 1 -- 172.21.15.92:0/180912183 shutdown_connections 2023-10-06T10:38:33.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.868+0000 7fb41a520700 1 --2- 172.21.15.92:0/180912183 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb400066f30 0x7fb4000693e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.868+0000 7fb41a520700 1 --2- 172.21.15.92:0/180912183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb41415d8b0 0x7fb41414c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.868+0000 7fb41a520700 1 --2- 172.21.15.92:0/180912183 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb414068cd0 0x7fb41414b410 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.868+0000 7fb41a520700 1 --2- 172.21.15.92:0/180912183 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4140b1d10 0x7fb41414b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.868+0000 7fb41a520700 1 -- 172.21.15.92:0/180912183 >> 172.21.15.92:0/180912183 conn(0x7fb41406de40 msgr2=0x7fb414069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.868+0000 7fb41a520700 1 -- 172.21.15.92:0/180912183 shutdown_connections 2023-10-06T10:38:33.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.868+0000 7fb41a520700 1 -- 172.21.15.92:0/180912183 wait complete. 2023-10-06T10:38:33.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.873+0000 7fb683ac4700 1 -- 172.21.15.92:0/562073311 shutdown_connections 2023-10-06T10:38:33.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.874+0000 7fb683ac4700 1 -- 172.21.15.92:0/562073311 wait complete. 2023-10-06T10:38:33.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.874+0000 7fb1f32a1700 1 -- 172.21.15.92:0/524391032 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb1d0066f30 msgr2=0x7fb1d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.874+0000 7fb1f32a1700 1 --2- 172.21.15.92:0/524391032 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb1d0066f30 0x7fb1d00693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0001cf0 tx=0x7fb1e0007ec0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.874+0000 7fb1f32a1700 1 -- 172.21.15.92:0/524391032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1ec159a20 msgr2=0x7fb1ec0c6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.874+0000 7fb1f32a1700 1 --2- 172.21.15.92:0/524391032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1ec159a20 0x7fb1ec0c6a00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800c560 tx=0x7fb1d8007dd0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.875+0000 7fc730725700 1 -- 172.21.15.92:0/2247725494 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fc710005560 con 0x7fc72806d890 2023-10-06T10:38:33.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.876+0000 7fb1f32a1700 1 -- 172.21.15.92:0/524391032 shutdown_connections 2023-10-06T10:38:33.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.876+0000 7fb1f32a1700 1 --2- 172.21.15.92:0/524391032 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb1d0066f30 0x7fb1d00693e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.876+0000 7fb1f32a1700 1 --2- 172.21.15.92:0/524391032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1ec15dab0 0x7fb1ec0c0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.876+0000 7fb1f32a1700 1 --2- 172.21.15.92:0/524391032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1ec069c30 0x7fb1ec0aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.876+0000 7fb1f32a1700 1 --2- 172.21.15.92:0/524391032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1ec159a20 0x7fb1ec0c6a00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.876+0000 7fb1f32a1700 1 -- 172.21.15.92:0/524391032 >> 172.21.15.92:0/524391032 conn(0x7fb1ec068080 msgr2=0x7fb1ec0c7340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.876+0000 7fc71e7fc700 1 -- 172.21.15.92:0/2247725494 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc718054700 con 0x7fc72806d890 2023-10-06T10:38:33.879 INFO:teuthology.orchestra.run.smithi092.stdout:55834574856 2023-10-06T10:38:33.880 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2023-10-06T10:38:33.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.879+0000 7f789d36b700 1 -- 172.21.15.92:0/3928874897 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f78981467c0 con 0x7f78980c8260 2023-10-06T10:38:33.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.880+0000 7f787effd700 1 -- 172.21.15.92:0/3928874897 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f78981467c0 con 0x7f78980c8260 2023-10-06T10:38:33.882 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-10-06T10:38:33.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.880+0000 7fc730725700 1 -- 172.21.15.92:0/2247725494 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc70c067080 msgr2=0x7fc70c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.880+0000 7fc730725700 1 --2- 172.21.15.92:0/2247725494 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc70c067080 0x7fc70c069530 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc72400df00 tx=0x7fc72400f040 comp rx=0 tx=0).stop 2023-10-06T10:38:33.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.880+0000 7fc730725700 1 -- 172.21.15.92:0/2247725494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc72806d890 msgr2=0x7fc72817b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.880+0000 7fc730725700 1 --2- 172.21.15.92:0/2247725494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc72806d890 0x7fc72817b210 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc71800c540 tx=0x7fc71800c900 comp rx=0 tx=0).stop 2023-10-06T10:38:33.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.880+0000 7fc730725700 1 -- 172.21.15.92:0/2247725494 shutdown_connections 2023-10-06T10:38:33.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.880+0000 7fc730725700 1 --2- 172.21.15.92:0/2247725494 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc70c067080 0x7fc70c069530 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.880+0000 7fc730725700 1 --2- 172.21.15.92:0/2247725494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7280878b0 0x7fc7280c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.880+0000 7fc730725700 1 --2- 172.21.15.92:0/2247725494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc728069c30 0x7fc728086aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.880+0000 7fc730725700 1 --2- 172.21.15.92:0/2247725494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc72806d890 0x7fc72817b210 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.880+0000 7fc730725700 1 -- 172.21.15.92:0/2247725494 >> 172.21.15.92:0/2247725494 conn(0x7fc728068080 msgr2=0x7fc7280c6620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.880+0000 7fc730725700 1 -- 172.21.15.92:0/2247725494 shutdown_connections 2023-10-06T10:38:33.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.880+0000 7fc730725700 1 -- 172.21.15.92:0/2247725494 wait complete. 2023-10-06T10:38:33.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.882+0000 7f789d36b700 1 -- 172.21.15.92:0/3928874897 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7884066f30 msgr2=0x7f78840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.882+0000 7f789d36b700 1 --2- 172.21.15.92:0/3928874897 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7884066f30 0x7f78840693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7890009880 tx=0x7f7890017000 comp rx=0 tx=0).stop 2023-10-06T10:38:33.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.882+0000 7f789d36b700 1 -- 172.21.15.92:0/3928874897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78980c8260 msgr2=0x7f78980b1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:33.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.882+0000 7f789d36b700 1 --2- 172.21.15.92:0/3928874897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78980c8260 0x7f78980b1520 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f788c00c810 tx=0x7f788c007ee0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.883+0000 7f789d36b700 1 -- 172.21.15.92:0/3928874897 shutdown_connections 2023-10-06T10:38:33.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.883+0000 7f789d36b700 1 --2- 172.21.15.92:0/3928874897 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7884066f30 0x7f78840693e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.883+0000 7f789d36b700 1 --2- 172.21.15.92:0/3928874897 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78980cbc20 0x7f78980bebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.883+0000 7f789d36b700 1 --2- 172.21.15.92:0/3928874897 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f789806e3d0 0x7f78980af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.883+0000 7f789d36b700 1 --2- 172.21.15.92:0/3928874897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78980c8260 0x7f78980b1520 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:33.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.883+0000 7f789d36b700 1 -- 172.21.15.92:0/3928874897 >> 172.21.15.92:0/3928874897 conn(0x7f7898068080 msgr2=0x7f789814b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:33.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.883+0000 7fb1f32a1700 1 -- 172.21.15.92:0/524391032 shutdown_connections 2023-10-06T10:38:33.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.883+0000 7f789d36b700 1 -- 172.21.15.92:0/3928874897 shutdown_connections 2023-10-06T10:38:33.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.883+0000 7f789d36b700 1 -- 172.21.15.92:0/3928874897 wait complete. 2023-10-06T10:38:33.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:33.885+0000 7fb1f32a1700 1 -- 172.21.15.92:0/524391032 wait complete. 2023-10-06T10:38:33.887 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2023-10-06T10:38:33.887 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2023-10-06T10:38:33.892 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2023-10-06T10:38:33.901 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2023-10-06T10:38:33.902 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2023-10-06T10:38:33.903 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2023-10-06T10:38:33.903 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.7 2023-10-06T10:38:34.880 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T10:38:34.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T10:38:34.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T10:38:34.893 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T10:38:34.902 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T10:38:34.902 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T10:38:34.904 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T10:38:34.904 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T10:38:35.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.062+0000 7f5eef67c700 1 Processor -- start 2023-10-06T10:38:35.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.062+0000 7f5eef67c700 1 -- start start 2023-10-06T10:38:35.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.063+0000 7f5eef67c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ee806d510 0x7f5ee806d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.063+0000 7f5eef67c700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5ee80c82e0 0x7f5ee80878b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.063+0000 7f5eef67c700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee8087df0 con 0x7f5ee8069520 2023-10-06T10:38:35.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.063+0000 7f5eef67c700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee80c89e0 con 0x7f5ee80c82e0 2023-10-06T10:38:35.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.063+0000 7f5eef67c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee80c7040 con 0x7f5ee806d510 2023-10-06T10:38:35.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.064+0000 7f5eecc17700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5ee80c82e0 0x7f5ee80878b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.064+0000 7f5eecc17700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5ee80c82e0 0x7f5ee80878b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:47412/0 (socket says 172.21.15.92:47412) 2023-10-06T10:38:35.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.064+0000 7f5eecc17700 1 -- 172.21.15.92:0/817734594 learned_addr learned my addr 172.21.15.92:0/817734594 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:35.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.064+0000 7f5eed418700 1 --2- 172.21.15.92:0/817734594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ee806d510 0x7f5ee806d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.065+0000 7f5edffff700 1 -- 172.21.15.92:0/817734594 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3538958703 0 0) 0x7f5ed8001160 con 0x7f5ee8069520 2023-10-06T10:38:35.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.065+0000 7f5edffff700 1 -- 172.21.15.92:0/817734594 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ed4003570 con 0x7f5ee8069520 2023-10-06T10:38:35.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.065+0000 7f5eecc17700 1 -- 172.21.15.92:0/817734594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ee806d510 msgr2=0x7f5ee806d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:35.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.065+0000 7f5eecc17700 1 --2- 172.21.15.92:0/817734594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ee806d510 0x7f5ee806d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.065+0000 7f5eecc17700 1 -- 172.21.15.92:0/817734594 >> v1:172.21.15.124:6789/0 conn(0x7f5ee8069520 legacy=0x7f5ee8069940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.065+0000 7f5eecc17700 1 -- 172.21.15.92:0/817734594 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee80c8750 con 0x7f5ee80c82e0 2023-10-06T10:38:35.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.065+0000 7f5eecc17700 1 --2- 172.21.15.92:0/817734594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ee80c82e0 0x7f5ee80878b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009f20 tx=0x7f5ee0007510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ada5eb98199b1e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.065+0000 7f5edffff700 1 -- 172.21.15.92:0/817734594 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee000e070 con 0x7f5ee80c82e0 2023-10-06T10:38:35.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.065+0000 7f5edffff700 1 -- 172.21.15.92:0/817734594 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee000bb80 con 0x7f5ee80c82e0 2023-10-06T10:38:35.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.065+0000 7f5edffff700 1 -- 172.21.15.92:0/817734594 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0007850 con 0x7f5ee80c82e0 2023-10-06T10:38:35.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.066+0000 7f5eef67c700 1 -- 172.21.15.92:0/817734594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ee80c82e0 msgr2=0x7f5ee80878b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.066+0000 7f5eef67c700 1 --2- 172.21.15.92:0/817734594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ee80c82e0 0x7f5ee80878b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009f20 tx=0x7f5ee0007510 comp rx=0 tx=0).stop 2023-10-06T10:38:35.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.067+0000 7f5eef67c700 1 -- 172.21.15.92:0/817734594 shutdown_connections 2023-10-06T10:38:35.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.067+0000 7f5eef67c700 1 --2- 172.21.15.92:0/817734594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ee806d510 0x7f5ee806d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.067+0000 7f5eef67c700 1 --2- 172.21.15.92:0/817734594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ee80c82e0 0x7f5ee80878b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.067+0000 7f5eef67c700 1 -- 172.21.15.92:0/817734594 >> 172.21.15.92:0/817734594 conn(0x7f5ee8068080 msgr2=0x7f5ee80cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:35.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.067+0000 7f5eef67c700 1 -- 172.21.15.92:0/817734594 shutdown_connections 2023-10-06T10:38:35.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.067+0000 7f5eef67c700 1 -- 172.21.15.92:0/817734594 wait complete. 2023-10-06T10:38:35.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.068+0000 7f5eef67c700 1 Processor -- start 2023-10-06T10:38:35.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.069+0000 7f5eef67c700 1 -- start start 2023-10-06T10:38:35.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.070+0000 7f5eef67c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ee8069520 0x7f5ee8166480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.070+0000 7f5eef67c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ee806d510 0x7f5ee8166de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.070+0000 7f5eef67c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ee80c82e0 0x7f5ee8167320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.070+0000 7f5eef67c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee816cf80 con 0x7f5ee8069520 2023-10-06T10:38:35.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.070+0000 7f5eef67c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee80c0b00 con 0x7f5ee80c82e0 2023-10-06T10:38:35.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.070+0000 7f5eef67c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee816cca0 con 0x7f5ee806d510 2023-10-06T10:38:35.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.071+0000 7f5eed418700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ee8069520 0x7f5ee8166480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.071+0000 7f5eed418700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ee8069520 0x7f5ee8166480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36746/0 (socket says 172.21.15.92:36746) 2023-10-06T10:38:35.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.071+0000 7f5eed418700 1 -- 172.21.15.92:0/1318102634 learned_addr learned my addr 172.21.15.92:0/1318102634 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:35.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.071+0000 7f5eed418700 1 -- 172.21.15.92:0/1318102634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ee806d510 msgr2=0x7f5ee8166de0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T10:38:35.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.071+0000 7f5eed418700 1 --2- 172.21.15.92:0/1318102634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ee806d510 0x7f5ee8166de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.071+0000 7f5eed418700 1 -- 172.21.15.92:0/1318102634 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ee80c82e0 msgr2=0x7f5ee8167320 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T10:38:35.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.071+0000 7f5eed418700 1 --2- 172.21.15.92:0/1318102634 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ee80c82e0 0x7f5ee8167320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.071+0000 7f5eed418700 1 -- 172.21.15.92:0/1318102634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee0009ae0 con 0x7f5ee8069520 2023-10-06T10:38:35.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.072+0000 7f5eed418700 1 --2- 172.21.15.92:0/1318102634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ee8069520 0x7f5ee8166480 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400cfa0 tx=0x7f5ee4005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.072+0000 7f5edd7fa700 1 -- 172.21.15.92:0/1318102634 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4011070 con 0x7f5ee8069520 2023-10-06T10:38:35.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.072+0000 7f5edd7fa700 1 -- 172.21.15.92:0/1318102634 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee4012040 con 0x7f5ee8069520 2023-10-06T10:38:35.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.072+0000 7f5edd7fa700 1 -- 172.21.15.92:0/1318102634 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee400c160 con 0x7f5ee8069520 2023-10-06T10:38:35.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.072+0000 7f5eef67c700 1 -- 172.21.15.92:0/1318102634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee80c04b0 con 0x7f5ee8069520 2023-10-06T10:38:35.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.073+0000 7f5eef67c700 1 -- 172.21.15.92:0/1318102634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee81669c0 con 0x7f5ee8069520 2023-10-06T10:38:35.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.077+0000 7f5edd7fa700 1 -- 172.21.15.92:0/1318102634 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ee40165c0 con 0x7f5ee8069520 2023-10-06T10:38:35.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.078+0000 7f5edd7fa700 1 --2- 172.21.15.92:0/1318102634 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5ecc067080 0x7f5ecc069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.078+0000 7f5edd7fa700 1 -- 172.21.15.92:0/1318102634 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f5ee4088e60 con 0x7f5ee8069520 2023-10-06T10:38:35.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.078+0000 7f5eef67c700 1 -- 172.21.15.92:0/1318102634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec80052b0 con 0x7f5ee8069520 2023-10-06T10:38:35.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.087+0000 7f5edd7fa700 1 -- 172.21.15.92:0/1318102634 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ee401a080 con 0x7f5ee8069520 2023-10-06T10:38:35.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.090+0000 7f5eecc17700 1 --2- 172.21.15.92:0/1318102634 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5ecc067080 0x7f5ecc069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.091+0000 7f5eecc17700 1 --2- 172.21.15.92:0/1318102634 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5ecc067080 0x7f5ecc069530 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5ee80cb470 tx=0x7f5ee0018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.123+0000 7f710ce09700 1 Processor -- start 2023-10-06T10:38:35.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.123+0000 7fb809f8d700 1 Processor -- start 2023-10-06T10:38:35.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.123+0000 7fb809f8d700 1 -- start start 2023-10-06T10:38:35.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.124+0000 7fc9432e5700 1 Processor -- start 2023-10-06T10:38:35.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.124+0000 7fc9432e5700 1 -- start start 2023-10-06T10:38:35.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.124+0000 7f710ce09700 1 -- start start 2023-10-06T10:38:35.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7f710ce09700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f710815d8b0 0x7f710815dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7fb809f8d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb80415d8b0 0x7fb80415dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7fb809f8d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb804068cd0 0x7fb8040b2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7fb809f8d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8040c9710 con 0x7fb8040b1d10 2023-10-06T10:38:35.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7fb809f8d700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8040c8eb0 con 0x7fb80415d8b0 2023-10-06T10:38:35.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7fb809f8d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040b26c0 con 0x7fb804068cd0 2023-10-06T10:38:35.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7f710ce09700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f71080b1d10 0x7f71080b2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7f710ce09700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71080c9710 con 0x7f71080685c0 2023-10-06T10:38:35.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7f710ce09700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71080c8eb0 con 0x7f71080b1d10 2023-10-06T10:38:35.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7f710ce09700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71080b26c0 con 0x7f710815d8b0 2023-10-06T10:38:35.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7fb802ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb8040b1d10 0x7fb8040685c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40086/0 (socket says 172.21.15.92:40086) 2023-10-06T10:38:35.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7fb802ffd700 1 -- 172.21.15.92:0/325973347 learned_addr learned my addr 172.21.15.92:0/325973347 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:35.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7fb8037fe700 1 --2- 172.21.15.92:0/325973347 >> v2:172.21.15.124:3300/0 conn(0x7fb80415d8b0 0x7fb80415dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7f552da45700 1 Processor -- start 2023-10-06T10:38:35.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7f710659c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f710815d8b0 0x7f710815dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7f710659c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f710815d8b0 0x7f710815dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:51896/0 (socket says 172.21.15.92:51896) 2023-10-06T10:38:35.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7f710659c700 1 -- 172.21.15.92:0/2074363103 learned_addr learned my addr 172.21.15.92:0/2074363103 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:35.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7fc9432e5700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc93c069c30 0x7fc93c15d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7fc9432e5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc93c15b920 con 0x7fc93c159a20 2023-10-06T10:38:35.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7fc9432e5700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc93c15e2b0 con 0x7fc93c15d8e0 2023-10-06T10:38:35.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7fc9432e5700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc93c15a1e0 con 0x7fc93c069c30 2023-10-06T10:38:35.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7fb803fff700 1 --2- 172.21.15.92:0/325973347 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb804068cd0 0x7fb8040b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7f552da45700 1 -- start start 2023-10-06T10:38:35.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7fc940880700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc93c069c30 0x7fc93c15d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7f5482d69700 1 Processor -- start 2023-10-06T10:38:35.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.126+0000 7f5482d69700 1 -- start start 2023-10-06T10:38:35.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.126+0000 7fc940880700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc93c069c30 0x7fc93c15d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:47440/0 (socket says 172.21.15.92:47440) 2023-10-06T10:38:35.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.126+0000 7fc940880700 1 -- 172.21.15.92:0/3112867145 learned_addr learned my addr 172.21.15.92:0/3112867145 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:35.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.126+0000 7f7105d9b700 1 --2- 172.21.15.92:0/2074363103 >> v2:172.21.15.92:3300/0 conn(0x7f71080b1d10 0x7f71080b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.126+0000 7f7105d9b700 1 -- 172.21.15.92:0/2074363103 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f710815d8b0 msgr2=0x7f710815dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:35.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.126+0000 7f7105d9b700 1 --2- 172.21.15.92:0/2074363103 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f710815d8b0 0x7f710815dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.126+0000 7f7105d9b700 1 -- 172.21.15.92:0/2074363103 >> v1:172.21.15.124:6789/0 conn(0x7f71080685c0 legacy=0x7f7108161780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.126+0000 7f7105d9b700 1 -- 172.21.15.92:0/2074363103 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71080b28d0 con 0x7f71080b1d10 2023-10-06T10:38:35.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.126+0000 7fc933fff700 1 -- 172.21.15.92:0/3112867145 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3449391600 0 0) 0x7fc93c15e2b0 con 0x7fc93c15d8e0 2023-10-06T10:38:35.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.126+0000 7fc933fff700 1 -- 172.21.15.92:0/3112867145 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc928003570 con 0x7fc93c15d8e0 2023-10-06T10:38:35.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.126+0000 7fc940880700 1 -- 172.21.15.92:0/3112867145 >> v1:172.21.15.124:6789/0 conn(0x7fc93c15d8e0 legacy=0x7fc93c15bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.126+0000 7fc940880700 1 -- 172.21.15.92:0/3112867145 >> v1:172.21.15.92:6789/0 conn(0x7fc93c159a20 legacy=0x7fc93c069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.126+0000 7fc940880700 1 -- 172.21.15.92:0/3112867145 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93c15dfd0 con 0x7fc93c069c30 2023-10-06T10:38:35.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.126+0000 7f7105d9b700 1 --2- 172.21.15.92:0/2074363103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71080b1d10 0x7f71080b2180 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f70f80098e0 tx=0x7f70f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9913f5c378c240 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.126+0000 7fc940880700 1 --2- 172.21.15.92:0/3112867145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc93c069c30 0x7fc93c15d3a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc92c0099b0 tx=0x7fc92c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2f4dea6a8bae818 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.125+0000 7fb8037fe700 1 -- 172.21.15.92:0/325973347 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb804068cd0 msgr2=0x7fb8040b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:35.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.126+0000 7fb8037fe700 1 --2- 172.21.15.92:0/325973347 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb804068cd0 0x7fb8040b2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.126+0000 7fb8037fe700 1 -- 172.21.15.92:0/325973347 >> v1:172.21.15.92:6789/0 conn(0x7fb8040b1d10 legacy=0x7fb8040685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:35.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.126+0000 7fb8037fe700 1 -- 172.21.15.92:0/325973347 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8040b28d0 con 0x7fb80415d8b0 2023-10-06T10:38:35.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.126+0000 7fb8037fe700 1 --2- 172.21.15.92:0/325973347 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb80415d8b0 0x7fb80415dc80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40098d0 tx=0x7fb7f4009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=155c38545d7d677 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.126+0000 7fb8027fc700 1 -- 172.21.15.92:0/325973347 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4009e70 con 0x7fb80415d8b0 2023-10-06T10:38:35.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.126+0000 7fb8027fc700 1 -- 172.21.15.92:0/325973347 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f4005c10 con 0x7fb80415d8b0 2023-10-06T10:38:35.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.126+0000 7fb8027fc700 1 -- 172.21.15.92:0/325973347 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f400b610 con 0x7fb80415d8b0 2023-10-06T10:38:35.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f552da45700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f55281563e0 0x7f55280c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f552da45700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55280b1120 0x7f55280b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f552da45700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f552815ad50 con 0x7f5528156010 2023-10-06T10:38:35.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f552da45700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55280b1ae0 con 0x7f55281563e0 2023-10-06T10:38:35.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f552da45700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5528159e60 con 0x7f55280b1120 2023-10-06T10:38:35.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7fc933fff700 1 -- 172.21.15.92:0/3112867145 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c009ea0 con 0x7fc93c069c30 2023-10-06T10:38:35.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7fc933fff700 1 -- 172.21.15.92:0/3112867145 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92c00b820 con 0x7fc93c069c30 2023-10-06T10:38:35.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7fc933fff700 1 -- 172.21.15.92:0/3112867145 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c00bbc0 con 0x7fc93c069c30 2023-10-06T10:38:35.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f552ca43700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f5528156010 0x7f55280b07b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:37362/0 (socket says 172.21.15.92:37362) 2023-10-06T10:38:35.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f552ca43700 1 -- 172.21.15.92:0/3695063737 learned_addr learned my addr 172.21.15.92:0/3695063737 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:35.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f552d244700 1 --2- 172.21.15.92:0/3695063737 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55280b1120 0x7f55280b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f710559a700 1 -- 172.21.15.92:0/2074363103 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f800b640 con 0x7f71080b1d10 2023-10-06T10:38:35.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f710559a700 1 -- 172.21.15.92:0/2074363103 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f800b7a0 con 0x7f71080b1d10 2023-10-06T10:38:35.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f710559a700 1 -- 172.21.15.92:0/2074363103 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f800bb40 con 0x7f71080b1d10 2023-10-06T10:38:35.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f5482d69700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f547c069c30 0x7f547c0c8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f5482d69700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f547c0c7fc0 con 0x7f547c0bf6f0 2023-10-06T10:38:35.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7fb809f8d700 1 -- 172.21.15.92:0/325973347 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb80415d8b0 msgr2=0x7fb80415dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7fb809f8d700 1 --2- 172.21.15.92:0/325973347 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb80415d8b0 0x7fb80415dc80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40098d0 tx=0x7fb7f4009120 comp rx=0 tx=0).stop 2023-10-06T10:38:35.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f5482d69700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f547c0c73b0 con 0x7f547c0bf320 2023-10-06T10:38:35.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f5482d69700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f547c15e190 con 0x7f547c069c30 2023-10-06T10:38:35.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f5481306700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f547c0bf6f0 0x7f547c0c6680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40102/0 (socket says 172.21.15.92:40102) 2023-10-06T10:38:35.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f5481306700 1 -- 172.21.15.92:0/1702321581 learned_addr learned my addr 172.21.15.92:0/1702321581 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:35.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f5527fff700 1 --2- 172.21.15.92:0/3695063737 >> v2:172.21.15.124:3300/0 conn(0x7f55281563e0 0x7f55280c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f55277fe700 1 -- 172.21.15.92:0/3695063737 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3361845610 0 0) 0x7f552815ad50 con 0x7f5528156010 2023-10-06T10:38:35.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f55277fe700 1 -- 172.21.15.92:0/3695063737 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5510003570 con 0x7f5528156010 2023-10-06T10:38:35.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7fb809f8d700 1 -- 172.21.15.92:0/325973347 shutdown_connections 2023-10-06T10:38:35.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7fb809f8d700 1 --2- 172.21.15.92:0/325973347 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb804068cd0 0x7fb8040b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7fb809f8d700 1 --2- 172.21.15.92:0/325973347 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb80415d8b0 0x7fb80415dc80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7fb809f8d700 1 -- 172.21.15.92:0/325973347 >> 172.21.15.92:0/325973347 conn(0x7fb80406de40 msgr2=0x7fb80406e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:35.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f552d244700 1 -- 172.21.15.92:0/3695063737 >> v2:172.21.15.124:3300/0 conn(0x7f55281563e0 msgr2=0x7f55280c9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:35.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f552d244700 1 --2- 172.21.15.92:0/3695063737 >> v2:172.21.15.124:3300/0 conn(0x7f55281563e0 0x7f55280c9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f552d244700 1 -- 172.21.15.92:0/3695063737 >> v1:172.21.15.124:6789/0 conn(0x7f5528156010 legacy=0x7f55280b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f552d244700 1 -- 172.21.15.92:0/3695063737 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552815a140 con 0x7f55280b1120 2023-10-06T10:38:35.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.127+0000 7f547bfff700 1 --2- 172.21.15.92:0/1702321581 >> v2:172.21.15.124:3300/0 conn(0x7f547c069c30 0x7f547c0c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f547b7fe700 1 -- 172.21.15.92:0/1702321581 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1311416425 0 0) 0x7f547c0c7fc0 con 0x7f547c0bf6f0 2023-10-06T10:38:35.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f547b7fe700 1 -- 172.21.15.92:0/1702321581 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5464003570 con 0x7f547c0bf6f0 2023-10-06T10:38:35.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f552d244700 1 --2- 172.21.15.92:0/3695063737 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55280b1120 0x7f55280b15a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5520009e60 tx=0x7f5520009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e254a8fcbf1ab63 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7fc9432e5700 1 -- 172.21.15.92:0/3112867145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc93c069c30 msgr2=0x7fc93c15d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7fc9432e5700 1 --2- 172.21.15.92:0/3112867145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc93c069c30 0x7fc93c15d3a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc92c0099b0 tx=0x7fc92c009250 comp rx=0 tx=0).stop 2023-10-06T10:38:35.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7fb809f8d700 1 -- 172.21.15.92:0/325973347 shutdown_connections 2023-10-06T10:38:35.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f55277fe700 1 -- 172.21.15.92:0/3695063737 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552000bb60 con 0x7f55280b1120 2023-10-06T10:38:35.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f55277fe700 1 -- 172.21.15.92:0/3695063737 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f552000bcc0 con 0x7f55280b1120 2023-10-06T10:38:35.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f55277fe700 1 -- 172.21.15.92:0/3695063737 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55200116b0 con 0x7f55280b1120 2023-10-06T10:38:35.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f547b7fe700 1 -- 172.21.15.92:0/1702321581 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3389269623 0 0) 0x7f5464003570 con 0x7f547c0bf6f0 2023-10-06T10:38:35.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f547b7fe700 1 -- 172.21.15.92:0/1702321581 >> v2:172.21.15.124:3300/0 conn(0x7f547c069c30 msgr2=0x7f547c0c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:35.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f547b7fe700 1 --2- 172.21.15.92:0/1702321581 >> v2:172.21.15.124:3300/0 conn(0x7f547c069c30 0x7f547c0c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f547b7fe700 1 -- 172.21.15.92:0/1702321581 >> v1:172.21.15.124:6789/0 conn(0x7f547c0bf320 legacy=0x7f547c069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f547b7fe700 1 -- 172.21.15.92:0/1702321581 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547c0c7690 con 0x7f547c0bf6f0 2023-10-06T10:38:35.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f547b7fe700 1 -- 172.21.15.92:0/1702321581 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f54740017f0 con 0x7f547c0bf6f0 2023-10-06T10:38:35.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f547b7fe700 1 -- 172.21.15.92:0/1702321581 >> v1:172.21.15.92:6789/0 conn(0x7f547c0bf6f0 legacy=0x7f547c0c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f547b7fe700 1 --2- 172.21.15.92:0/1702321581 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5464003a60 0x7f5464003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f547b7fe700 1 --2- 172.21.15.92:0/1702321581 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5464004810 0x7f5464004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f547b7fe700 1 --2- 172.21.15.92:0/1702321581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54640051c0 0x7f546400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f547b7fe700 1 -- 172.21.15.92:0/1702321581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c15e190 con 0x7f54640051c0 2023-10-06T10:38:35.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f547b7fe700 1 -- 172.21.15.92:0/1702321581 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546400c0d0 con 0x7f5464004810 2023-10-06T10:38:35.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f547b7fe700 1 -- 172.21.15.92:0/1702321581 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546400c290 con 0x7f5464003a60 2023-10-06T10:38:35.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7fb809f8d700 1 -- 172.21.15.92:0/325973347 wait complete. 2023-10-06T10:38:35.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f5481306700 1 --2- 172.21.15.92:0/1702321581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54640051c0 0x7f546400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f547bfff700 1 --2- 172.21.15.92:0/1702321581 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5464004810 0x7f5464004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f5481306700 1 -- 172.21.15.92:0/1702321581 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5464003a60 msgr2=0x7f5464003e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T10:38:35.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f5481306700 1 --2- 172.21.15.92:0/1702321581 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5464003a60 0x7f5464003e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f5481306700 1 -- 172.21.15.92:0/1702321581 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5464004810 msgr2=0x7f5464004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:35.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f5481306700 1 --2- 172.21.15.92:0/1702321581 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5464004810 0x7f5464004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f5481306700 1 -- 172.21.15.92:0/1702321581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5464003570 con 0x7f54640051c0 2023-10-06T10:38:35.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.128+0000 7f5481306700 1 -- 172.21.15.92:0/1702321581 reap_dead start 2023-10-06T10:38:35.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.129+0000 7f5481306700 1 --2- 172.21.15.92:0/1702321581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54640051c0 0x7f546400ba60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f547400a950 tx=0x7f54740021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b842d88f7849f41 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.129+0000 7f552da45700 1 -- 172.21.15.92:0/3695063737 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55280b1120 msgr2=0x7f55280b15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.129+0000 7f552da45700 1 --2- 172.21.15.92:0/3695063737 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55280b1120 0x7f55280b15a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5520009e60 tx=0x7f5520009780 comp rx=0 tx=0).stop 2023-10-06T10:38:35.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.129+0000 7f547b7fe700 1 -- 172.21.15.92:0/1702321581 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54740070d0 con 0x7f54640051c0 2023-10-06T10:38:35.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.129+0000 7f547b7fe700 1 -- 172.21.15.92:0/1702321581 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5474007250 con 0x7f54640051c0 2023-10-06T10:38:35.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.129+0000 7f547b7fe700 1 -- 172.21.15.92:0/1702321581 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474007610 con 0x7f54640051c0 2023-10-06T10:38:35.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.130+0000 7fb809f8d700 1 Processor -- start 2023-10-06T10:38:35.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.130+0000 7fc9432e5700 1 -- 172.21.15.92:0/3112867145 shutdown_connections 2023-10-06T10:38:35.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.130+0000 7fc9432e5700 1 --2- 172.21.15.92:0/3112867145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc93c069c30 0x7fc93c15d3a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.130+0000 7fc9432e5700 1 -- 172.21.15.92:0/3112867145 >> 172.21.15.92:0/3112867145 conn(0x7fc93c068080 msgr2=0x7fc93c0cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:35.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.130+0000 7fc9432e5700 1 -- 172.21.15.92:0/3112867145 shutdown_connections 2023-10-06T10:38:35.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.130+0000 7f5482d69700 1 -- 172.21.15.92:0/1702321581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54640051c0 msgr2=0x7f546400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.130+0000 7f5482d69700 1 --2- 172.21.15.92:0/1702321581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54640051c0 0x7f546400ba60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f547400a950 tx=0x7f54740021f0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.130+0000 7f5482d69700 1 -- 172.21.15.92:0/1702321581 shutdown_connections 2023-10-06T10:38:35.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.130+0000 7f5482d69700 1 --2- 172.21.15.92:0/1702321581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54640051c0 0x7f546400ba60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.130+0000 7f5482d69700 1 -- 172.21.15.92:0/1702321581 >> 172.21.15.92:0/1702321581 conn(0x7f547c068080 msgr2=0x7f547c0cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:35.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.130+0000 7fc9432e5700 1 -- 172.21.15.92:0/3112867145 wait complete. 2023-10-06T10:38:35.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.130+0000 7fb809f8d700 1 -- start start 2023-10-06T10:38:35.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.130+0000 7fb809f8d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb804068cd0 0x7fb8040acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.130+0000 7fb809f8d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8040b1d10 0x7fb80415ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.130+0000 7fb809f8d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb80415d8b0 0x7fb80415f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.130+0000 7fb809f8d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80415e6b0 con 0x7fb8040b1d10 2023-10-06T10:38:35.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.130+0000 7fb809f8d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040ad6e0 con 0x7fb804068cd0 2023-10-06T10:38:35.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.130+0000 7fb809f8d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040ad9c0 con 0x7fb80415d8b0 2023-10-06T10:38:35.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.131+0000 7fb802ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8040b1d10 0x7fb80415ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.131+0000 7fb803fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb80415d8b0 0x7fb80415f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.131+0000 7fb802ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8040b1d10 0x7fb80415ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36770/0 (socket says 172.21.15.92:36770) 2023-10-06T10:38:35.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.131+0000 7fb802ffd700 1 -- 172.21.15.92:0/710548967 learned_addr learned my addr 172.21.15.92:0/710548967 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:35.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.131+0000 7fb803fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb80415d8b0 0x7fb80415f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:51912/0 (socket says 172.21.15.92:51912) 2023-10-06T10:38:35.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.131+0000 7fb802ffd700 1 -- 172.21.15.92:0/710548967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb80415d8b0 msgr2=0x7fb80415f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.131+0000 7fb802ffd700 1 --2- 172.21.15.92:0/710548967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb80415d8b0 0x7fb80415f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.131+0000 7fb802ffd700 1 -- 172.21.15.92:0/710548967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb804068cd0 msgr2=0x7fb8040acdf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T10:38:35.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.131+0000 7fb802ffd700 1 --2- 172.21.15.92:0/710548967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb804068cd0 0x7fb8040acdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.131+0000 7fb802ffd700 1 -- 172.21.15.92:0/710548967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f40093c0 con 0x7fb8040b1d10 2023-10-06T10:38:35.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.131+0000 7fb803fff700 1 --2- 172.21.15.92:0/710548967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb80415d8b0 0x7fb80415f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:35.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.131+0000 7fb802ffd700 1 --2- 172.21.15.92:0/710548967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8040b1d10 0x7fb80415ee20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800c5b0 tx=0x7fb7f8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.131+0000 7fb7eb7fe700 1 -- 172.21.15.92:0/710548967 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8005b30 con 0x7fb8040b1d10 2023-10-06T10:38:35.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.130+0000 7f5482d69700 1 -- 172.21.15.92:0/1702321581 shutdown_connections 2023-10-06T10:38:35.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.130+0000 7f5482d69700 1 -- 172.21.15.92:0/1702321581 wait complete. 2023-10-06T10:38:35.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.131+0000 7fb7eb7fe700 1 -- 172.21.15.92:0/710548967 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f8005cb0 con 0x7fb8040b1d10 2023-10-06T10:38:35.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.131+0000 7fb7eb7fe700 1 -- 172.21.15.92:0/710548967 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8007320 con 0x7fb8040b1d10 2023-10-06T10:38:35.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.131+0000 7f5482d69700 1 Processor -- start 2023-10-06T10:38:35.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.131+0000 7f5482d69700 1 -- start start 2023-10-06T10:38:35.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.131+0000 7fc9432e5700 1 Processor -- start 2023-10-06T10:38:35.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.132+0000 7fc9432e5700 1 -- start start 2023-10-06T10:38:35.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.132+0000 7f5482d69700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54640051c0 0x7f547c15e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.132+0000 7f5482d69700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f547c0c3400 0x7f547c0c3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.132+0000 7f5482d69700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f547c154d10 0x7f547c15fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.132+0000 7f5482d69700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c0c4150 con 0x7f547c0c3400 2023-10-06T10:38:35.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.132+0000 7f5482d69700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c0c4430 con 0x7f547c154d10 2023-10-06T10:38:35.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.132+0000 7f5482d69700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547c0c4710 con 0x7f54640051c0 2023-10-06T10:38:35.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.132+0000 7fb809f8d700 1 -- 172.21.15.92:0/710548967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8040adca0 con 0x7fb8040b1d10 2023-10-06T10:38:35.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.132+0000 7fb809f8d700 1 -- 172.21.15.92:0/710548967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8040ad330 con 0x7fb8040b1d10 2023-10-06T10:38:35.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.132+0000 7f5480b05700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54640051c0 0x7f547c15e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.133+0000 7f5480b05700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54640051c0 0x7f547c15e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:51914/0 (socket says 172.21.15.92:51914) 2023-10-06T10:38:35.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.133+0000 7f5480b05700 1 -- 172.21.15.92:0/386230761 learned_addr learned my addr 172.21.15.92:0/386230761 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:35.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.133+0000 7f5481306700 1 --2- 172.21.15.92:0/386230761 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f547c154d10 0x7f547c15fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.133+0000 7f5480b05700 1 -- 172.21.15.92:0/386230761 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f547c154d10 msgr2=0x7f547c15fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.133+0000 7f5480b05700 1 --2- 172.21.15.92:0/386230761 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f547c154d10 0x7f547c15fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.133+0000 7f5480b05700 1 -- 172.21.15.92:0/386230761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f547c0c3400 msgr2=0x7f547c0c3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.133+0000 7f5480b05700 1 --2- 172.21.15.92:0/386230761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f547c0c3400 0x7f547c0c3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.133+0000 7f5480b05700 1 -- 172.21.15.92:0/386230761 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547c0c49f0 con 0x7f54640051c0 2023-10-06T10:38:35.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.133+0000 7fc9432e5700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc93c069c30 0x7fc93c0c23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.133+0000 7fc9432e5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc93c159a20 0x7fc93c154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.133+0000 7fc9432e5700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc93c15d8e0 0x7fc93c06eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.133+0000 7fc9432e5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c0c19c0 con 0x7fc93c159a20 2023-10-06T10:38:35.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.133+0000 7fc9432e5700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c0c0b10 con 0x7fc93c069c30 2023-10-06T10:38:35.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.133+0000 7fc9432e5700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c0c0db0 con 0x7fc93c15d8e0 2023-10-06T10:38:35.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.133+0000 7f710ce09700 1 -- 172.21.15.92:0/2074363103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71080b1d10 msgr2=0x7f71080b2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.133+0000 7f710ce09700 1 --2- 172.21.15.92:0/2074363103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71080b1d10 0x7f71080b2180 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f70f80098e0 tx=0x7f70f80091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.133+0000 7f5480b05700 1 --2- 172.21.15.92:0/386230761 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54640051c0 0x7f547c15e560 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f547c06fe00 tx=0x7f546c007ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.133+0000 7fb809f8d700 1 -- 172.21.15.92:0/710548967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb804064a90 con 0x7fb8040b1d10 2023-10-06T10:38:35.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.133+0000 7f5478ff9700 1 -- 172.21.15.92:0/386230761 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c011590 con 0x7f54640051c0 2023-10-06T10:38:35.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.133+0000 7f5478ff9700 1 -- 172.21.15.92:0/386230761 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546c011710 con 0x7f54640051c0 2023-10-06T10:38:35.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.134+0000 7f5478ff9700 1 -- 172.21.15.92:0/386230761 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c011ad0 con 0x7f54640051c0 2023-10-06T10:38:35.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.134+0000 7fc941081700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc93c069c30 0x7fc93c0c23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.134+0000 7fc941081700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc93c069c30 0x7fc93c0c23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:47482/0 (socket says 172.21.15.92:47482) 2023-10-06T10:38:35.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.134+0000 7fc941081700 1 -- 172.21.15.92:0/3825188607 learned_addr learned my addr 172.21.15.92:0/3825188607 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:35.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.134+0000 7f552da45700 1 -- 172.21.15.92:0/3695063737 shutdown_connections 2023-10-06T10:38:35.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.134+0000 7f552da45700 1 --2- 172.21.15.92:0/3695063737 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55280b1120 0x7f55280b15a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.134+0000 7f552da45700 1 --2- 172.21.15.92:0/3695063737 >> v2:172.21.15.124:3300/0 conn(0x7f55281563e0 0x7f55280c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.134+0000 7f552da45700 1 -- 172.21.15.92:0/3695063737 >> 172.21.15.92:0/3695063737 conn(0x7f5528069190 msgr2=0x7f55280aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:35.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.134+0000 7f552da45700 1 -- 172.21.15.92:0/3695063737 shutdown_connections 2023-10-06T10:38:35.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.134+0000 7f552da45700 1 -- 172.21.15.92:0/3695063737 wait complete. 2023-10-06T10:38:35.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.134+0000 7fc941081700 1 -- 172.21.15.92:0/3825188607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc93c15d8e0 msgr2=0x7fc93c06eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.134+0000 7fc941081700 1 --2- 172.21.15.92:0/3825188607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc93c15d8e0 0x7fc93c06eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.134+0000 7fc941081700 1 -- 172.21.15.92:0/3825188607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc93c159a20 msgr2=0x7fc93c154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.134+0000 7fc941081700 1 --2- 172.21.15.92:0/3825188607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc93c159a20 0x7fc93c154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.134+0000 7fc941081700 1 -- 172.21.15.92:0/3825188607 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c0094f0 con 0x7fc93c069c30 2023-10-06T10:38:35.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.134+0000 7f710ce09700 1 -- 172.21.15.92:0/2074363103 shutdown_connections 2023-10-06T10:38:35.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.134+0000 7f710ce09700 1 --2- 172.21.15.92:0/2074363103 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f710815d8b0 0x7f710815dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.134+0000 7f710ce09700 1 --2- 172.21.15.92:0/2074363103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71080b1d10 0x7f71080b2180 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.134+0000 7f710ce09700 1 -- 172.21.15.92:0/2074363103 >> 172.21.15.92:0/2074363103 conn(0x7f710806de40 msgr2=0x7f710806e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:35.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.134+0000 7f710ce09700 1 -- 172.21.15.92:0/2074363103 shutdown_connections 2023-10-06T10:38:35.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.134+0000 7f5482d69700 1 -- 172.21.15.92:0/386230761 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f547c0c4cd0 con 0x7f54640051c0 2023-10-06T10:38:35.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.135+0000 7f5482d69700 1 -- 172.21.15.92:0/386230761 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547c15eaa0 con 0x7f54640051c0 2023-10-06T10:38:35.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.136+0000 7f552da45700 1 Processor -- start 2023-10-06T10:38:35.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.134+0000 7f710ce09700 1 -- 172.21.15.92:0/2074363103 wait complete. 2023-10-06T10:38:35.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.136+0000 7f710ce09700 1 Processor -- start 2023-10-06T10:38:35.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.136+0000 7f552da45700 1 -- start start 2023-10-06T10:38:35.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.136+0000 7f5482d69700 1 -- 172.21.15.92:0/386230761 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547c15ecc0 con 0x7f54640051c0 2023-10-06T10:38:35.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.136+0000 7f5478ff9700 1 -- 172.21.15.92:0/386230761 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f546c01b3b0 con 0x7f54640051c0 2023-10-06T10:38:35.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f5478ff9700 1 --2- 172.21.15.92:0/386230761 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5464068900 0x7f546406adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f5478ff9700 1 -- 172.21.15.92:0/386230761 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f546c083d70 con 0x7f54640051c0 2023-10-06T10:38:35.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f552da45700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55280b1120 0x7f5528157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f552da45700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5528156010 0x7f5528168650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f552da45700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55281563e0 0x7f5528168b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f552da45700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55281583b0 con 0x7f55281563e0 2023-10-06T10:38:35.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f552da45700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552813cc20 con 0x7f55280b1120 2023-10-06T10:38:35.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f552da45700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552813c010 con 0x7f5528156010 2023-10-06T10:38:35.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.136+0000 7f710ce09700 1 -- start start 2023-10-06T10:38:35.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f710ce09700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71080685c0 0x7f710815c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f710ce09700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71080b1d10 0x7f71080abad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f710ce09700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f710815d8b0 0x7f71080a8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f710ce09700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71080ac010 con 0x7f71080b1d10 2023-10-06T10:38:35.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f710ce09700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71080ab640 con 0x7f710815d8b0 2023-10-06T10:38:35.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f710ce09700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71080aaa30 con 0x7f71080685c0 2023-10-06T10:38:35.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7fb7eb7fe700 1 -- 172.21.15.92:0/710548967 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb7f801b3b0 con 0x7fb8040b1d10 2023-10-06T10:38:35.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7fb7eb7fe700 1 --2- 172.21.15.92:0/710548967 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb7f0066f30 0x7fb7f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f710659c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71080685c0 0x7f710815c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f710659c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71080685c0 0x7f710815c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:51922/0 (socket says 172.21.15.92:51922) 2023-10-06T10:38:35.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f710659c700 1 -- 172.21.15.92:0/2527037044 learned_addr learned my addr 172.21.15.92:0/2527037044 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:35.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7fb7eb7fe700 1 -- 172.21.15.92:0/710548967 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb7f8084390 con 0x7fb8040b1d10 2023-10-06T10:38:35.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.138+0000 7fb8037fe700 1 --2- 172.21.15.92:0/710548967 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb7f0066f30 0x7fb7f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f7105d9b700 1 --2- 172.21.15.92:0/2527037044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71080b1d10 0x7f71080abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.138+0000 7f710659c700 1 -- 172.21.15.92:0/2527037044 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f710815d8b0 msgr2=0x7f71080a8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.138+0000 7f710659c700 1 --2- 172.21.15.92:0/2527037044 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f710815d8b0 0x7f71080a8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.138+0000 7f710659c700 1 -- 172.21.15.92:0/2527037044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71080b1d10 msgr2=0x7f71080abad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.138+0000 7f710659c700 1 --2- 172.21.15.92:0/2527037044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71080b1d10 0x7f71080abad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.138+0000 7f710659c700 1 -- 172.21.15.92:0/2527037044 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f8009420 con 0x7f71080685c0 2023-10-06T10:38:35.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.138+0000 7f710659c700 1 --2- 172.21.15.92:0/2527037044 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71080685c0 0x7f710815c250 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00e9a0 tx=0x7f70fc009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f5527fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5528156010 0x7f5528168650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f552ca43700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55280b1120 0x7f5528157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f5527fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5528156010 0x7f5528168650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:51938/0 (socket says 172.21.15.92:51938) 2023-10-06T10:38:35.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f5527fff700 1 -- 172.21.15.92:0/3086757281 learned_addr learned my addr 172.21.15.92:0/3086757281 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:35.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.138+0000 7f552d244700 1 --2- 172.21.15.92:0/3086757281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55281563e0 0x7f5528168b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.138+0000 7f552ca43700 1 -- 172.21.15.92:0/3086757281 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5528156010 msgr2=0x7f5528168650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.138+0000 7f552ca43700 1 --2- 172.21.15.92:0/3086757281 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5528156010 0x7f5528168650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.138+0000 7f552ca43700 1 -- 172.21.15.92:0/3086757281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55281563e0 msgr2=0x7f5528168b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.138+0000 7f552ca43700 1 --2- 172.21.15.92:0/3086757281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55281563e0 0x7f5528168b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.138+0000 7f552ca43700 1 -- 172.21.15.92:0/3086757281 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5520009a20 con 0x7f55280b1120 2023-10-06T10:38:35.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.138+0000 7f5527fff700 1 --2- 172.21.15.92:0/3086757281 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5528156010 0x7f5528168650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:35.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.138+0000 7f552ca43700 1 --2- 172.21.15.92:0/3086757281 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55280b1120 0x7f5528157e70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f551800c5b0 tx=0x7f5518007d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.138+0000 7f547bfff700 1 --2- 172.21.15.92:0/386230761 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5464068900 0x7f546406adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.138+0000 7f547bfff700 1 --2- 172.21.15.92:0/386230761 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5464068900 0x7f546406adb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5470009170 tx=0x7f54700091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.139+0000 7fb8037fe700 1 --2- 172.21.15.92:0/710548967 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb7f0066f30 0x7fb7f00693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009160 tx=0x7fb7f4015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f628ba8f700 1 Processor -- start 2023-10-06T10:38:35.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.137+0000 7f628ba8f700 1 -- start start 2023-10-06T10:38:35.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.139+0000 7f628ba8f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62840b0030 0x7f62840b2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.139+0000 7f628ba8f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628415eb70 0x7f62840cc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.139+0000 7f628ba8f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62840cd6a0 con 0x7f628415e7a0 2023-10-06T10:38:35.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.139+0000 7f628ba8f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62840b1e30 con 0x7f62840b0030 2023-10-06T10:38:35.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.139+0000 7f628ba8f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62840b2110 con 0x7f628415eb70 2023-10-06T10:38:35.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.139+0000 7f628902a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62840b0030 0x7f62840b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.139+0000 7f628a02c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628415eb70 0x7f62840cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.139+0000 7f628982b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f628415e7a0 0x7f62840af920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40104/0 (socket says 172.21.15.92:40104) 2023-10-06T10:38:35.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.139+0000 7f628902a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62840b0030 0x7f62840b2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36828/0 (socket says 172.21.15.92:36828) 2023-10-06T10:38:35.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.139+0000 7f628982b700 1 -- 172.21.15.92:0/2586465126 learned_addr learned my addr 172.21.15.92:0/2586465126 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:35.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.139+0000 7f628a02c700 1 -- 172.21.15.92:0/2586465126 >> v2:172.21.15.92:3300/0 conn(0x7f62840b0030 msgr2=0x7f62840b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:35.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.139+0000 7f628a02c700 1 --2- 172.21.15.92:0/2586465126 >> v2:172.21.15.92:3300/0 conn(0x7f62840b0030 0x7f62840b2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.139+0000 7f628a02c700 1 -- 172.21.15.92:0/2586465126 >> v1:172.21.15.92:6789/0 conn(0x7f628415e7a0 legacy=0x7f62840af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.139+0000 7f628a02c700 1 -- 172.21.15.92:0/2586465126 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62840b23f0 con 0x7f628415eb70 2023-10-06T10:38:35.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.140+0000 7f5524ff9700 1 -- 172.21.15.92:0/3086757281 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518005c00 con 0x7f55280b1120 2023-10-06T10:38:35.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.140+0000 7f5524ff9700 1 -- 172.21.15.92:0/3086757281 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5518005d60 con 0x7f55280b1120 2023-10-06T10:38:35.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.140+0000 7f552da45700 1 -- 172.21.15.92:0/3086757281 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552813c2f0 con 0x7f55280b1120 2023-10-06T10:38:35.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.140+0000 7f5524ff9700 1 -- 172.21.15.92:0/3086757281 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518007320 con 0x7f55280b1120 2023-10-06T10:38:35.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.140+0000 7f552da45700 1 -- 172.21.15.92:0/3086757281 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552813c8b0 con 0x7f55280b1120 2023-10-06T10:38:35.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.140+0000 7fb7eb7fe700 1 -- 172.21.15.92:0/710548967 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb7f80511b0 con 0x7fb8040b1d10 2023-10-06T10:38:35.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.141+0000 7fc941081700 1 --2- 172.21.15.92:0/3825188607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc93c069c30 0x7fc93c0c23f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc93800c590 tx=0x7fc938007f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.141+0000 7fc9317fa700 1 -- 172.21.15.92:0/3825188607 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938011590 con 0x7fc93c069c30 2023-10-06T10:38:35.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.141+0000 7fc9317fa700 1 -- 172.21.15.92:0/3825188607 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc938011730 con 0x7fc93c069c30 2023-10-06T10:38:35.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.141+0000 7fc9317fa700 1 -- 172.21.15.92:0/3825188607 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938011af0 con 0x7fc93c069c30 2023-10-06T10:38:35.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.141+0000 7f552da45700 1 -- 172.21.15.92:0/3086757281 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f552805f230 con 0x7f55280b1120 2023-10-06T10:38:35.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.141+0000 7fc9432e5700 1 -- 172.21.15.92:0/3825188607 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc93c0c1090 con 0x7fc93c069c30 2023-10-06T10:38:35.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.141+0000 7fc9432e5700 1 -- 172.21.15.92:0/3825188607 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc93c0c1650 con 0x7fc93c069c30 2023-10-06T10:38:35.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.141+0000 7f2166817700 1 Processor -- start 2023-10-06T10:38:35.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.142+0000 7f70f67fc700 1 -- 172.21.15.92:0/2527037044 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc00ee90 con 0x7f71080685c0 2023-10-06T10:38:35.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.142+0000 7f70f67fc700 1 -- 172.21.15.92:0/2527037044 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70fc00c850 con 0x7f71080685c0 2023-10-06T10:38:35.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.142+0000 7f70f67fc700 1 -- 172.21.15.92:0/2527037044 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc00cbf0 con 0x7f71080685c0 2023-10-06T10:38:35.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.142+0000 7f710ce09700 1 -- 172.21.15.92:0/2527037044 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71080aad10 con 0x7f71080685c0 2023-10-06T10:38:35.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.142+0000 7f710ce09700 1 -- 172.21.15.92:0/2527037044 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71080ab2d0 con 0x7f71080685c0 2023-10-06T10:38:35.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.142+0000 7f2166817700 1 -- start start 2023-10-06T10:38:35.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.142+0000 7f5478ff9700 1 -- 172.21.15.92:0/386230761 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f546c050b90 con 0x7f54640051c0 2023-10-06T10:38:35.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.142+0000 7f5524ff9700 1 -- 172.21.15.92:0/3086757281 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f551801b3b0 con 0x7f55280b1120 2023-10-06T10:38:35.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.142+0000 7f628a02c700 1 --2- 172.21.15.92:0/2586465126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628415eb70 0x7f62840cc970 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f627c009e60 tx=0x7f627c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38574ba8f4f416f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.142+0000 7f5524ff9700 1 --2- 172.21.15.92:0/3086757281 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f550c067050 0x7f550c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.142+0000 7f6288829700 1 -- 172.21.15.92:0/2586465126 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c00e070 con 0x7f628415eb70 2023-10-06T10:38:35.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.142+0000 7f6288829700 1 -- 172.21.15.92:0/2586465126 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627c00bb60 con 0x7f628415eb70 2023-10-06T10:38:35.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.142+0000 7f6288829700 1 -- 172.21.15.92:0/2586465126 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c012430 con 0x7f628415eb70 2023-10-06T10:38:35.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.142+0000 7fc9432e5700 1 -- 172.21.15.92:0/3825188607 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc91c0052b0 con 0x7fc93c069c30 2023-10-06T10:38:35.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.143+0000 7f2166817700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f216006a450 0x7f216015e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.143+0000 7f2166817700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f216006a8c0 0x7f216015ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.143+0000 7f2166817700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216015e100 con 0x7f216015da10 2023-10-06T10:38:35.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.143+0000 7f2166817700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216015dde0 con 0x7f216006a8c0 2023-10-06T10:38:35.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.143+0000 7f2166817700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21600b0c90 con 0x7f216006a450 2023-10-06T10:38:35.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.143+0000 7f2165815700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f216015da10 0x7f21600af180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:37372/0 (socket says 172.21.15.92:37372) 2023-10-06T10:38:35.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.143+0000 7f2165815700 1 -- 172.21.15.92:0/465007014 learned_addr learned my addr 172.21.15.92:0/465007014 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:35.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.143+0000 7f2165014700 1 --2- 172.21.15.92:0/465007014 >> v2:172.21.15.124:3300/0 conn(0x7f216006a450 0x7f216015e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.143+0000 7f2166016700 1 --2- 172.21.15.92:0/465007014 >> v2:172.21.15.92:3300/0 conn(0x7f216006a8c0 0x7f216015ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.143+0000 7f628ba8f700 1 -- 172.21.15.92:0/2586465126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628415eb70 msgr2=0x7f62840cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.144+0000 7f628ba8f700 1 --2- 172.21.15.92:0/2586465126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628415eb70 0x7f62840cc970 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f627c009e60 tx=0x7f627c009780 comp rx=0 tx=0).stop 2023-10-06T10:38:35.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.144+0000 7f70f67fc700 1 -- 172.21.15.92:0/2527037044 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f70fc013470 con 0x7f71080685c0 2023-10-06T10:38:35.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.144+0000 7f2165014700 1 -- 172.21.15.92:0/465007014 >> v2:172.21.15.92:3300/0 conn(0x7f216006a8c0 msgr2=0x7f216015ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:35.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.144+0000 7f2165014700 1 --2- 172.21.15.92:0/465007014 >> v2:172.21.15.92:3300/0 conn(0x7f216006a8c0 0x7f216015ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.144+0000 7f2165014700 1 -- 172.21.15.92:0/465007014 >> v1:172.21.15.124:6789/0 conn(0x7f216015da10 legacy=0x7f21600af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.144+0000 7f2165014700 1 -- 172.21.15.92:0/465007014 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21600b0110 con 0x7f216006a450 2023-10-06T10:38:35.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.144+0000 7f2166016700 1 --2- 172.21.15.92:0/465007014 >> v2:172.21.15.92:3300/0 conn(0x7f216006a8c0 0x7f216015ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:35.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.144+0000 7f2165014700 1 --2- 172.21.15.92:0/465007014 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f216006a450 0x7f216015e590 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2160064370 tx=0x7f215c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94f53f990e721190 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.144+0000 7f70f67fc700 1 --2- 172.21.15.92:0/2527037044 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f70e8066f30 0x7f70e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.144+0000 7f7105d9b700 1 --2- 172.21.15.92:0/2527037044 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f70e8066f30 0x7f70e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.144+0000 7f70f67fc700 1 -- 172.21.15.92:0/2527037044 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f70fc084e70 con 0x7f71080685c0 2023-10-06T10:38:35.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.145+0000 7f7105d9b700 1 --2- 172.21.15.92:0/2527037044 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f70e8066f30 0x7f70e80693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f70f80098e0 tx=0x7f70f8017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.146+0000 7f2164813700 1 -- 172.21.15.92:0/465007014 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c011630 con 0x7f216006a450 2023-10-06T10:38:35.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.146+0000 7f2164813700 1 -- 172.21.15.92:0/465007014 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215c011790 con 0x7f216006a450 2023-10-06T10:38:35.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.146+0000 7f2164813700 1 -- 172.21.15.92:0/465007014 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c011b30 con 0x7f216006a450 2023-10-06T10:38:35.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.147+0000 7f2166817700 1 -- 172.21.15.92:0/465007014 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f216006a450 msgr2=0x7f216015e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.147+0000 7f2166817700 1 --2- 172.21.15.92:0/465007014 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f216006a450 0x7f216015e590 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2160064370 tx=0x7f215c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.147+0000 7f2166817700 1 -- 172.21.15.92:0/465007014 shutdown_connections 2023-10-06T10:38:35.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.147+0000 7f2166817700 1 --2- 172.21.15.92:0/465007014 >> v2:172.21.15.92:3300/0 conn(0x7f216006a8c0 0x7f216015ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.147+0000 7f2166817700 1 --2- 172.21.15.92:0/465007014 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f216006a450 0x7f216015e590 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.147+0000 7f2166817700 1 -- 172.21.15.92:0/465007014 >> 172.21.15.92:0/465007014 conn(0x7f2160069190 msgr2=0x7f21600aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:35.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.147+0000 7f2166817700 1 -- 172.21.15.92:0/465007014 shutdown_connections 2023-10-06T10:38:35.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.147+0000 7f2166817700 1 -- 172.21.15.92:0/465007014 wait complete. 2023-10-06T10:38:35.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.142+0000 7f5524ff9700 1 -- 172.21.15.92:0/3086757281 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f5518083e80 con 0x7f55280b1120 2023-10-06T10:38:35.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.144+0000 7f5527fff700 1 --2- 172.21.15.92:0/3086757281 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f550c067050 0x7f550c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.144+0000 7f5527fff700 1 --2- 172.21.15.92:0/3086757281 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f550c067050 0x7f550c069500 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f55281685b0 tx=0x7f551c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.146+0000 7f5524ff9700 1 -- 172.21.15.92:0/3086757281 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5518050c80 con 0x7f55280b1120 2023-10-06T10:38:35.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.148+0000 7f628ba8f700 1 -- 172.21.15.92:0/2586465126 shutdown_connections 2023-10-06T10:38:35.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.148+0000 7f628ba8f700 1 --2- 172.21.15.92:0/2586465126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628415eb70 0x7f62840cc970 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.148+0000 7f628ba8f700 1 --2- 172.21.15.92:0/2586465126 >> v2:172.21.15.92:3300/0 conn(0x7f62840b0030 0x7f62840b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.148+0000 7f628ba8f700 1 -- 172.21.15.92:0/2586465126 >> 172.21.15.92:0/2586465126 conn(0x7f628406ece0 msgr2=0x7f628406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:35.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.148+0000 7f628ba8f700 1 -- 172.21.15.92:0/2586465126 shutdown_connections 2023-10-06T10:38:35.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.148+0000 7f2166817700 1 Processor -- start 2023-10-06T10:38:35.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.148+0000 7f2166817700 1 -- start start 2023-10-06T10:38:35.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.149+0000 7f2166817700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f216006a450 0x7f2160156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.149+0000 7f2166817700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f216006a8c0 0x7f216014f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.149+0000 7f2166817700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f216015da10 0x7f2160158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.149+0000 7f2166817700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2160155cd0 con 0x7f216006a8c0 2023-10-06T10:38:35.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.149+0000 7f2166817700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2160155fb0 con 0x7f216015da10 2023-10-06T10:38:35.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.149+0000 7f2166817700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2160156290 con 0x7f216006a450 2023-10-06T10:38:35.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.149+0000 7f2165014700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f216006a8c0 0x7f216014f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.149+0000 7f2165014700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f216006a8c0 0x7f216014f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36854/0 (socket says 172.21.15.92:36854) 2023-10-06T10:38:35.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.149+0000 7f2165014700 1 -- 172.21.15.92:0/2320070701 learned_addr learned my addr 172.21.15.92:0/2320070701 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:35.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.149+0000 7f2165815700 1 --2- 172.21.15.92:0/2320070701 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f216006a450 0x7f2160156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.149+0000 7f2166016700 1 --2- 172.21.15.92:0/2320070701 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f216015da10 0x7f2160158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.148+0000 7f628ba8f700 1 -- 172.21.15.92:0/2586465126 wait complete. 2023-10-06T10:38:35.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.150+0000 7f628ba8f700 1 Processor -- start 2023-10-06T10:38:35.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.150+0000 7f2166016700 1 -- 172.21.15.92:0/2320070701 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f216006a450 msgr2=0x7f2160156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.150+0000 7f2166016700 1 --2- 172.21.15.92:0/2320070701 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f216006a450 0x7f2160156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.150+0000 7f2166016700 1 -- 172.21.15.92:0/2320070701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f216006a8c0 msgr2=0x7f216014f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.150+0000 7f2166016700 1 --2- 172.21.15.92:0/2320070701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f216006a8c0 0x7f216014f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.150+0000 7f2166016700 1 -- 172.21.15.92:0/2320070701 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2158009a20 con 0x7f216015da10 2023-10-06T10:38:35.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.150+0000 7f2165014700 1 --2- 172.21.15.92:0/2320070701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f216006a8c0 0x7f216014f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:35.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.150+0000 7f2165815700 1 --2- 172.21.15.92:0/2320070701 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f216006a450 0x7f2160156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:35.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.150+0000 7f628ba8f700 1 -- start start 2023-10-06T10:38:35.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.151+0000 7f628ba8f700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62840b0030 0x7f628414aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.151+0000 7f628ba8f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628415e7a0 0x7f62840cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.151+0000 7f628ba8f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628415eb70 0x7f62841574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.151+0000 7f628ba8f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62840cbf50 con 0x7f628415eb70 2023-10-06T10:38:35.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.151+0000 7f628ba8f700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62840cb6f0 con 0x7f62840b0030 2023-10-06T10:38:35.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.151+0000 7f628ba8f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62840cb990 con 0x7f628415e7a0 2023-10-06T10:38:35.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.151+0000 7f628a02c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628415eb70 0x7f62841574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.151+0000 7f628a02c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628415eb70 0x7f62841574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36860/0 (socket says 172.21.15.92:36860) 2023-10-06T10:38:35.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.151+0000 7f628a02c700 1 -- 172.21.15.92:0/3973767351 learned_addr learned my addr 172.21.15.92:0/3973767351 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:35.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.151+0000 7f628982b700 1 --2- 172.21.15.92:0/3973767351 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62840b0030 0x7f628414aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.151+0000 7f628902a700 1 --2- 172.21.15.92:0/3973767351 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628415e7a0 0x7f62840cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.151+0000 7f628a02c700 1 -- 172.21.15.92:0/3973767351 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628415e7a0 msgr2=0x7f62840cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.151+0000 7f628a02c700 1 --2- 172.21.15.92:0/3973767351 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628415e7a0 0x7f62840cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.151+0000 7f628a02c700 1 -- 172.21.15.92:0/3973767351 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62840b0030 msgr2=0x7f628414aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.151+0000 7f628a02c700 1 --2- 172.21.15.92:0/3973767351 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62840b0030 0x7f628414aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.151+0000 7f628a02c700 1 -- 172.21.15.92:0/3973767351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627c009a20 con 0x7f628415eb70 2023-10-06T10:38:35.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.151+0000 7f628982b700 1 --2- 172.21.15.92:0/3973767351 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62840b0030 0x7f628414aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:35.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.152+0000 7f2166016700 1 --2- 172.21.15.92:0/2320070701 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f216015da10 0x7f2160158a90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2158009f10 tx=0x7f215800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.152+0000 7f628a02c700 1 --2- 172.21.15.92:0/3973767351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628415eb70 0x7f62841574a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f628415f050 tx=0x7f627c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.152+0000 7f2155ffb700 1 -- 172.21.15.92:0/2320070701 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215800cb70 con 0x7f216015da10 2023-10-06T10:38:35.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.152+0000 7f6279ffb700 1 -- 172.21.15.92:0/3973767351 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c012ea0 con 0x7f628415eb70 2023-10-06T10:38:35.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.152+0000 7f6279ffb700 1 -- 172.21.15.92:0/3973767351 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627c027470 con 0x7f628415eb70 2023-10-06T10:38:35.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.152+0000 7f6279ffb700 1 -- 172.21.15.92:0/3973767351 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c027850 con 0x7f628415eb70 2023-10-06T10:38:35.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.152+0000 7f628ba8f700 1 -- 172.21.15.92:0/3973767351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62840cbc70 con 0x7f628415eb70 2023-10-06T10:38:35.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.152+0000 7f628ba8f700 1 -- 172.21.15.92:0/3973767351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628406ad30 con 0x7f628415eb70 2023-10-06T10:38:35.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.152+0000 7f2155ffb700 1 -- 172.21.15.92:0/2320070701 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215800f040 con 0x7f216015da10 2023-10-06T10:38:35.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.152+0000 7f2155ffb700 1 -- 172.21.15.92:0/2320070701 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2158014480 con 0x7f216015da10 2023-10-06T10:38:35.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.152+0000 7f2166817700 1 -- 172.21.15.92:0/2320070701 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f215c009420 con 0x7f216015da10 2023-10-06T10:38:35.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.152+0000 7f2166817700 1 -- 172.21.15.92:0/2320070701 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2160161540 con 0x7f216015da10 2023-10-06T10:38:35.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.153+0000 7f2166817700 1 -- 172.21.15.92:0/2320070701 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2160161760 con 0x7f216015da10 2023-10-06T10:38:35.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.152+0000 7fc9317fa700 1 -- 172.21.15.92:0/3825188607 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc938012420 con 0x7fc93c069c30 2023-10-06T10:38:35.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.153+0000 7fc9317fa700 1 --2- 172.21.15.92:0/3825188607 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc920066f30 0x7fc9200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.153+0000 7fc9317fa700 1 -- 172.21.15.92:0/3825188607 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fc938083cb0 con 0x7fc93c069c30 2023-10-06T10:38:35.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.153+0000 7fc940880700 1 --2- 172.21.15.92:0/3825188607 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc920066f30 0x7fc9200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.154+0000 7fc940880700 1 --2- 172.21.15.92:0/3825188607 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc920066f30 0x7fc9200693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc92c0099b0 tx=0x7fc92c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.155+0000 7f6279ffb700 1 -- 172.21.15.92:0/3973767351 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f627c024070 con 0x7f628415eb70 2023-10-06T10:38:35.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.156+0000 7f6279ffb700 1 --2- 172.21.15.92:0/3973767351 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6268066f30 0x7f62680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.156+0000 7f6279ffb700 1 -- 172.21.15.92:0/3973767351 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f627c087330 con 0x7f628415eb70 2023-10-06T10:38:35.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.156+0000 7f628982b700 1 --2- 172.21.15.92:0/3973767351 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6268066f30 0x7f62680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.156+0000 7f628982b700 1 --2- 172.21.15.92:0/3973767351 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6268066f30 0x7f62680693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6274007fd0 tx=0x7f6274009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.157+0000 7f710ce09700 1 -- 172.21.15.92:0/2527037044 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70ec0052b0 con 0x7f71080685c0 2023-10-06T10:38:35.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.157+0000 7fc9317fa700 1 -- 172.21.15.92:0/3825188607 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc93804b5a0 con 0x7fc93c069c30 2023-10-06T10:38:35.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.158+0000 7f2155ffb700 1 -- 172.21.15.92:0/2320070701 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2158003020 con 0x7f216015da10 2023-10-06T10:38:35.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.159+0000 7f2155ffb700 1 --2- 172.21.15.92:0/2320070701 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2144066f30 0x7f21440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.159+0000 7f2155ffb700 1 -- 172.21.15.92:0/2320070701 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f2158085c40 con 0x7f216015da10 2023-10-06T10:38:35.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.159+0000 7f2165815700 1 --2- 172.21.15.92:0/2320070701 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2144066f30 0x7f21440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.160+0000 7f2165815700 1 --2- 172.21.15.92:0/2320070701 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2144066f30 0x7f21440693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2160159a00 tx=0x7f2150007f10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.162+0000 7f70f67fc700 1 -- 172.21.15.92:0/2527037044 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70fc051c90 con 0x7f71080685c0 2023-10-06T10:38:35.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.162+0000 7f2155ffb700 1 -- 172.21.15.92:0/2320070701 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2158052a40 con 0x7f216015da10 2023-10-06T10:38:35.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.172+0000 7f628ba8f700 1 -- 172.21.15.92:0/3973767351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62640052b0 con 0x7f628415eb70 2023-10-06T10:38:35.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.185+0000 7f6279ffb700 1 -- 172.21.15.92:0/3973767351 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f627c054150 con 0x7f628415eb70 2023-10-06T10:38:35.474 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-10-06T10:38:35.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.462+0000 7f5eef67c700 1 -- 172.21.15.92:0/1318102634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5ec8005560 con 0x7f5ee8069520 2023-10-06T10:38:35.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.470+0000 7f5edd7fa700 1 -- 172.21.15.92:0/1318102634 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5ee4055c60 con 0x7f5ee8069520 2023-10-06T10:38:35.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.473+0000 7f5eef67c700 1 -- 172.21.15.92:0/1318102634 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5ecc067080 msgr2=0x7f5ecc069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.473+0000 7f5eef67c700 1 --2- 172.21.15.92:0/1318102634 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5ecc067080 0x7f5ecc069530 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5ee80cb470 tx=0x7f5ee0018000 comp rx=0 tx=0).stop 2023-10-06T10:38:35.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.473+0000 7f5eef67c700 1 -- 172.21.15.92:0/1318102634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ee8069520 msgr2=0x7f5ee8166480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.473+0000 7f5eef67c700 1 --2- 172.21.15.92:0/1318102634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ee8069520 0x7f5ee8166480 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400cfa0 tx=0x7f5ee4005e40 comp rx=0 tx=0).stop 2023-10-06T10:38:35.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.473+0000 7f5eef67c700 1 -- 172.21.15.92:0/1318102634 shutdown_connections 2023-10-06T10:38:35.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.473+0000 7f5eef67c700 1 --2- 172.21.15.92:0/1318102634 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5ecc067080 0x7f5ecc069530 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.473+0000 7f5eef67c700 1 --2- 172.21.15.92:0/1318102634 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ee80c82e0 0x7f5ee8167320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.473+0000 7f5eef67c700 1 --2- 172.21.15.92:0/1318102634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ee8069520 0x7f5ee8166480 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.473+0000 7f5eef67c700 1 --2- 172.21.15.92:0/1318102634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ee806d510 0x7f5ee8166de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.473+0000 7f5eef67c700 1 -- 172.21.15.92:0/1318102634 >> 172.21.15.92:0/1318102634 conn(0x7f5ee8068080 msgr2=0x7f5ee8069940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:35.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.475+0000 7f5eef67c700 1 -- 172.21.15.92:0/1318102634 shutdown_connections 2023-10-06T10:38:35.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.475+0000 7f5eef67c700 1 -- 172.21.15.92:0/1318102634 wait complete. 2023-10-06T10:38:35.494 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2023-10-06T10:38:35.495 DEBUG:teuthology.parallel:result is None 2023-10-06T10:38:35.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.508+0000 7f5482d69700 1 -- 172.21.15.92:0/386230761 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f547c064a90 con 0x7f54640051c0 2023-10-06T10:38:35.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.509+0000 7f5478ff9700 1 -- 172.21.15.92:0/386230761 <== mon.2 v2:172.21.15.124:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f546c017070 con 0x7f54640051c0 2023-10-06T10:38:35.511 INFO:teuthology.orchestra.run.smithi092.stdout:55834574857 2023-10-06T10:38:35.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.513+0000 7f5482d69700 1 -- 172.21.15.92:0/386230761 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5464068900 msgr2=0x7f546406adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.513+0000 7f5482d69700 1 --2- 172.21.15.92:0/386230761 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5464068900 0x7f546406adb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5470009170 tx=0x7f54700091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.513+0000 7f5482d69700 1 -- 172.21.15.92:0/386230761 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54640051c0 msgr2=0x7f547c15e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.513+0000 7f5482d69700 1 --2- 172.21.15.92:0/386230761 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54640051c0 0x7f547c15e560 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f547c06fe00 tx=0x7f546c007ec0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.514+0000 7f5482d69700 1 -- 172.21.15.92:0/386230761 shutdown_connections 2023-10-06T10:38:35.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.514+0000 7f5482d69700 1 --2- 172.21.15.92:0/386230761 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5464068900 0x7f546406adb0 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5470009170 tx=0x7f54700091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.514+0000 7f5482d69700 1 --2- 172.21.15.92:0/386230761 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f547c154d10 0x7f547c15fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.514+0000 7f5482d69700 1 --2- 172.21.15.92:0/386230761 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54640051c0 0x7f547c15e560 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.514+0000 7f5482d69700 1 --2- 172.21.15.92:0/386230761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f547c0c3400 0x7f547c0c3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.514+0000 7f5482d69700 1 -- 172.21.15.92:0/386230761 >> 172.21.15.92:0/386230761 conn(0x7f547c068080 msgr2=0x7f547c06d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:35.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.515+0000 7f5482d69700 1 -- 172.21.15.92:0/386230761 shutdown_connections 2023-10-06T10:38:35.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.515+0000 7f5482d69700 1 -- 172.21.15.92:0/386230761 wait complete. 2023-10-06T10:38:35.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.521+0000 7f2166817700 1 -- 172.21.15.92:0/2320070701 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f216005f230 con 0x7f216015da10 2023-10-06T10:38:35.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.524+0000 7f2155ffb700 1 -- 172.21.15.92:0/2320070701 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2158056060 con 0x7f216015da10 2023-10-06T10:38:35.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.527+0000 7fb809f8d700 1 -- 172.21.15.92:0/710548967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb8040c90c0 con 0x7fb8040b1d10 2023-10-06T10:38:35.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.528+0000 7fb7eb7fe700 1 -- 172.21.15.92:0/710548967 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb7f8017020 con 0x7fb8040b1d10 2023-10-06T10:38:35.529 INFO:teuthology.orchestra.run.smithi092.stdout:55834574857 2023-10-06T10:38:35.530 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-10-06T10:38:35.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.531+0000 7f2166817700 1 -- 172.21.15.92:0/2320070701 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2144066f30 msgr2=0x7f21440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.531+0000 7f2166817700 1 --2- 172.21.15.92:0/2320070701 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2144066f30 0x7f21440693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2160159a00 tx=0x7f2150007f10 comp rx=0 tx=0).stop 2023-10-06T10:38:35.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.531+0000 7f2166817700 1 -- 172.21.15.92:0/2320070701 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f216015da10 msgr2=0x7f2160158a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.531+0000 7f2166817700 1 --2- 172.21.15.92:0/2320070701 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f216015da10 0x7f2160158a90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2158009f10 tx=0x7f215800b040 comp rx=0 tx=0).stop 2023-10-06T10:38:35.535 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.7 2023-10-06T10:38:35.536 DEBUG:teuthology.parallel:result is None 2023-10-06T10:38:35.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.534+0000 7fb809f8d700 1 -- 172.21.15.92:0/710548967 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb7f0066f30 msgr2=0x7fb7f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.534+0000 7fb809f8d700 1 --2- 172.21.15.92:0/710548967 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb7f0066f30 0x7fb7f00693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009160 tx=0x7fb7f4015000 comp rx=0 tx=0).stop 2023-10-06T10:38:35.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.534+0000 7fb809f8d700 1 -- 172.21.15.92:0/710548967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8040b1d10 msgr2=0x7fb80415ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.534+0000 7fb809f8d700 1 --2- 172.21.15.92:0/710548967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8040b1d10 0x7fb80415ee20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800c5b0 tx=0x7fb7f8007f90 comp rx=0 tx=0).stop 2023-10-06T10:38:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.534+0000 7fb809f8d700 1 -- 172.21.15.92:0/710548967 shutdown_connections 2023-10-06T10:38:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.534+0000 7fb809f8d700 1 --2- 172.21.15.92:0/710548967 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb7f0066f30 0x7fb7f00693e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.534+0000 7fb809f8d700 1 --2- 172.21.15.92:0/710548967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb80415d8b0 0x7fb80415f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.534+0000 7fb809f8d700 1 --2- 172.21.15.92:0/710548967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb804068cd0 0x7fb8040acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.534+0000 7fb809f8d700 1 --2- 172.21.15.92:0/710548967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8040b1d10 0x7fb80415ee20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.534+0000 7fb809f8d700 1 -- 172.21.15.92:0/710548967 >> 172.21.15.92:0/710548967 conn(0x7fb80406de40 msgr2=0x7fb8040aec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.534+0000 7fb809f8d700 1 -- 172.21.15.92:0/710548967 shutdown_connections 2023-10-06T10:38:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.534+0000 7fb809f8d700 1 -- 172.21.15.92:0/710548967 wait complete. 2023-10-06T10:38:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.535+0000 7f2166817700 1 -- 172.21.15.92:0/2320070701 shutdown_connections 2023-10-06T10:38:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.535+0000 7f2166817700 1 --2- 172.21.15.92:0/2320070701 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2144066f30 0x7f21440693e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.535+0000 7f2166817700 1 --2- 172.21.15.92:0/2320070701 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f216015da10 0x7f2160158a90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.535+0000 7f2166817700 1 --2- 172.21.15.92:0/2320070701 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f216006a450 0x7f2160156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.535+0000 7f2166817700 1 --2- 172.21.15.92:0/2320070701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f216006a8c0 0x7f216014f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.535+0000 7f2166817700 1 -- 172.21.15.92:0/2320070701 >> 172.21.15.92:0/2320070701 conn(0x7f2160069190 msgr2=0x7f21601527b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:35.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.535+0000 7f552da45700 1 -- 172.21.15.92:0/3086757281 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f55280af660 con 0x7f55280b1120 2023-10-06T10:38:35.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.536+0000 7f5524ff9700 1 -- 172.21.15.92:0/3086757281 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5518017020 con 0x7f55280b1120 2023-10-06T10:38:35.539 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-10-06T10:38:35.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.539+0000 7f552da45700 1 -- 172.21.15.92:0/3086757281 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f550c067050 msgr2=0x7f550c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.539+0000 7f552da45700 1 --2- 172.21.15.92:0/3086757281 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f550c067050 0x7f550c069500 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f55281685b0 tx=0x7f551c008040 comp rx=0 tx=0).stop 2023-10-06T10:38:35.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.539+0000 7f552da45700 1 -- 172.21.15.92:0/3086757281 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55280b1120 msgr2=0x7f5528157e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.539+0000 7f552da45700 1 --2- 172.21.15.92:0/3086757281 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55280b1120 0x7f5528157e70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f551800c5b0 tx=0x7f5518007d50 comp rx=0 tx=0).stop 2023-10-06T10:38:35.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.540+0000 7f552da45700 1 -- 172.21.15.92:0/3086757281 shutdown_connections 2023-10-06T10:38:35.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.540+0000 7f552da45700 1 --2- 172.21.15.92:0/3086757281 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f550c067050 0x7f550c069500 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.540+0000 7f552da45700 1 --2- 172.21.15.92:0/3086757281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55281563e0 0x7f5528168b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.540+0000 7f552da45700 1 --2- 172.21.15.92:0/3086757281 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55280b1120 0x7f5528157e70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.540+0000 7f552da45700 1 --2- 172.21.15.92:0/3086757281 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5528156010 0x7f5528168650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.540+0000 7f552da45700 1 -- 172.21.15.92:0/3086757281 >> 172.21.15.92:0/3086757281 conn(0x7f5528069190 msgr2=0x7f552806ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:35.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.540+0000 7f552da45700 1 -- 172.21.15.92:0/3086757281 shutdown_connections 2023-10-06T10:38:35.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.540+0000 7f552da45700 1 -- 172.21.15.92:0/3086757281 wait complete. 2023-10-06T10:38:35.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.541+0000 7f2166817700 1 -- 172.21.15.92:0/2320070701 shutdown_connections 2023-10-06T10:38:35.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.541+0000 7f2166817700 1 -- 172.21.15.92:0/2320070701 wait complete. 2023-10-06T10:38:35.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.545+0000 7fc9432e5700 1 -- 172.21.15.92:0/3825188607 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc91c005830 con 0x7fc93c069c30 2023-10-06T10:38:35.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.546+0000 7fc9317fa700 1 -- 172.21.15.92:0/3825188607 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc938017070 con 0x7fc93c069c30 2023-10-06T10:38:35.548 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-10-06T10:38:35.551 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2023-10-06T10:38:35.552 DEBUG:teuthology.parallel:result is None 2023-10-06T10:38:35.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.549+0000 7fc9432e5700 1 -- 172.21.15.92:0/3825188607 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc920066f30 msgr2=0x7fc9200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.549+0000 7fc9432e5700 1 --2- 172.21.15.92:0/3825188607 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc920066f30 0x7fc9200693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc92c0099b0 tx=0x7fc92c016000 comp rx=0 tx=0).stop 2023-10-06T10:38:35.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.549+0000 7fc9432e5700 1 -- 172.21.15.92:0/3825188607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc93c069c30 msgr2=0x7fc93c0c23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.549+0000 7fc9432e5700 1 --2- 172.21.15.92:0/3825188607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc93c069c30 0x7fc93c0c23f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc93800c590 tx=0x7fc938007f30 comp rx=0 tx=0).stop 2023-10-06T10:38:35.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.550+0000 7fc9432e5700 1 -- 172.21.15.92:0/3825188607 shutdown_connections 2023-10-06T10:38:35.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.550+0000 7fc9432e5700 1 --2- 172.21.15.92:0/3825188607 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc920066f30 0x7fc9200693e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.550+0000 7fc9432e5700 1 --2- 172.21.15.92:0/3825188607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc93c15d8e0 0x7fc93c06eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.550+0000 7fc9432e5700 1 --2- 172.21.15.92:0/3825188607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc93c069c30 0x7fc93c0c23f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.550+0000 7fc9432e5700 1 --2- 172.21.15.92:0/3825188607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc93c159a20 0x7fc93c154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.550+0000 7fc9432e5700 1 -- 172.21.15.92:0/3825188607 >> 172.21.15.92:0/3825188607 conn(0x7fc93c068080 msgr2=0x7fc93c06de90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:35.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.550+0000 7fc9432e5700 1 -- 172.21.15.92:0/3825188607 shutdown_connections 2023-10-06T10:38:35.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.550+0000 7fc9432e5700 1 -- 172.21.15.92:0/3825188607 wait complete. 2023-10-06T10:38:35.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.550+0000 7f710ce09700 1 -- 172.21.15.92:0/2527037044 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f70ec005830 con 0x7f71080685c0 2023-10-06T10:38:35.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.551+0000 7f70f67fc700 1 -- 172.21.15.92:0/2527037044 <== mon.2 v2:172.21.15.124:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f70fc021070 con 0x7f71080685c0 2023-10-06T10:38:35.554 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-10-06T10:38:35.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.553+0000 7f628ba8f700 1 -- 172.21.15.92:0/3973767351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6264005830 con 0x7f628415eb70 2023-10-06T10:38:35.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.553+0000 7f6279ffb700 1 -- 172.21.15.92:0/3973767351 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f627c02c020 con 0x7f628415eb70 2023-10-06T10:38:35.555 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-10-06T10:38:35.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.554+0000 7f710ce09700 1 -- 172.21.15.92:0/2527037044 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f70e8066f30 msgr2=0x7f70e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.554+0000 7f710ce09700 1 --2- 172.21.15.92:0/2527037044 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f70e8066f30 0x7f70e80693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f70f80098e0 tx=0x7f70f8017000 comp rx=0 tx=0).stop 2023-10-06T10:38:35.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.554+0000 7f710ce09700 1 -- 172.21.15.92:0/2527037044 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71080685c0 msgr2=0x7f710815c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.554+0000 7f710ce09700 1 --2- 172.21.15.92:0/2527037044 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71080685c0 0x7f710815c250 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00e9a0 tx=0x7f70fc009220 comp rx=0 tx=0).stop 2023-10-06T10:38:35.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.556+0000 7f710ce09700 1 -- 172.21.15.92:0/2527037044 shutdown_connections 2023-10-06T10:38:35.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.556+0000 7f710ce09700 1 --2- 172.21.15.92:0/2527037044 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f70e8066f30 0x7f70e80693e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.556+0000 7f710ce09700 1 --2- 172.21.15.92:0/2527037044 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f710815d8b0 0x7f71080a8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.556+0000 7f710ce09700 1 --2- 172.21.15.92:0/2527037044 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71080685c0 0x7f710815c250 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.556+0000 7f710ce09700 1 --2- 172.21.15.92:0/2527037044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71080b1d10 0x7f71080abad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.556+0000 7f710ce09700 1 -- 172.21.15.92:0/2527037044 >> 172.21.15.92:0/2527037044 conn(0x7f710806de40 msgr2=0x7f71080ac440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:35.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.556+0000 7f710ce09700 1 -- 172.21.15.92:0/2527037044 shutdown_connections 2023-10-06T10:38:35.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.556+0000 7f710ce09700 1 -- 172.21.15.92:0/2527037044 wait complete. 2023-10-06T10:38:35.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.556+0000 7f628ba8f700 1 -- 172.21.15.92:0/3973767351 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6268066f30 msgr2=0x7f62680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.556+0000 7f628ba8f700 1 --2- 172.21.15.92:0/3973767351 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6268066f30 0x7f62680693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6274007fd0 tx=0x7f6274009040 comp rx=0 tx=0).stop 2023-10-06T10:38:35.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.556+0000 7f628ba8f700 1 -- 172.21.15.92:0/3973767351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628415eb70 msgr2=0x7f62841574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.556+0000 7f628ba8f700 1 --2- 172.21.15.92:0/3973767351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628415eb70 0x7f62841574a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f628415f050 tx=0x7f627c019000 comp rx=0 tx=0).stop 2023-10-06T10:38:35.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.556+0000 7f628ba8f700 1 -- 172.21.15.92:0/3973767351 shutdown_connections 2023-10-06T10:38:35.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.556+0000 7f628ba8f700 1 --2- 172.21.15.92:0/3973767351 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6268066f30 0x7f62680693e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.556+0000 7f628ba8f700 1 --2- 172.21.15.92:0/3973767351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628415eb70 0x7f62841574a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.556+0000 7f628ba8f700 1 --2- 172.21.15.92:0/3973767351 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62840b0030 0x7f628414aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.556+0000 7f628ba8f700 1 --2- 172.21.15.92:0/3973767351 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628415e7a0 0x7f62840cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.556+0000 7f628ba8f700 1 -- 172.21.15.92:0/3973767351 >> 172.21.15.92:0/3973767351 conn(0x7f628406ece0 msgr2=0x7f62840c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:35.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.556+0000 7f628ba8f700 1 -- 172.21.15.92:0/3973767351 shutdown_connections 2023-10-06T10:38:35.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.556+0000 7f628ba8f700 1 -- 172.21.15.92:0/3973767351 wait complete. 2023-10-06T10:38:35.564 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2023-10-06T10:38:35.565 DEBUG:teuthology.parallel:result is None 2023-10-06T10:38:35.565 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2023-10-06T10:38:35.565 DEBUG:teuthology.parallel:result is None 2023-10-06T10:38:35.569 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-10-06T10:38:35.569 DEBUG:teuthology.parallel:result is None 2023-10-06T10:38:35.574 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2023-10-06T10:38:35.574 DEBUG:teuthology.parallel:result is None 2023-10-06T10:38:35.575 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2023-10-06T10:38:35.575 DEBUG:teuthology.parallel:result is None 2023-10-06T10:38:35.575 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T10:38:35.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:38:35.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.728+0000 7f7eafd48700 1 Processor -- start 2023-10-06T10:38:35.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.728+0000 7f7eafd48700 1 -- start start 2023-10-06T10:38:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.729+0000 7f7eafd48700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7ea8069ab0 0x7f7ea80ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.729+0000 7f7eafd48700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ea806d350 0x7f7ea806eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.729+0000 7f7eafd48700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea806e6d0 con 0x7f7ea80696e0 2023-10-06T10:38:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.729+0000 7f7eafd48700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea80cd000 con 0x7f7ea8069ab0 2023-10-06T10:38:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.729+0000 7f7eafd48700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea806dac0 con 0x7f7ea806d350 2023-10-06T10:38:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.729+0000 7f7ead2e3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7ea8069ab0 0x7f7ea80ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.729+0000 7f7eadae4700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7ea80696e0 0x7f7ea80aeeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40110/0 (socket says 172.21.15.92:40110) 2023-10-06T10:38:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.729+0000 7f7eadae4700 1 -- 172.21.15.92:0/2769701106 learned_addr learned my addr 172.21.15.92:0/2769701106 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.729+0000 7f7ead2e3700 1 -- 172.21.15.92:0/2769701106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ea806d350 msgr2=0x7f7ea806eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.729+0000 7f7ead2e3700 1 --2- 172.21.15.92:0/2769701106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ea806d350 0x7f7ea806eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.729+0000 7f7ead2e3700 1 -- 172.21.15.92:0/2769701106 >> v1:172.21.15.92:6789/0 conn(0x7f7ea80696e0 legacy=0x7f7ea80aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.729+0000 7f7ead2e3700 1 -- 172.21.15.92:0/2769701106 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea806dda0 con 0x7f7ea8069ab0 2023-10-06T10:38:35.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.729+0000 7f7eacae2700 1 -- 172.21.15.92:0/2769701106 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 603686903 0 0) 0x7f7ea806e6d0 con 0x7f7ea80696e0 2023-10-06T10:38:35.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.729+0000 7f7ead2e3700 1 --2- 172.21.15.92:0/2769701106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ea8069ab0 0x7f7ea80ada60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7e980098e0 tx=0x7f7e980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbb0fdb5d30981b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.730+0000 7f7eacae2700 1 -- 172.21.15.92:0/2769701106 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98009e80 con 0x7f7ea8069ab0 2023-10-06T10:38:35.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.730+0000 7f7eacae2700 1 -- 172.21.15.92:0/2769701106 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9800b860 con 0x7f7ea8069ab0 2023-10-06T10:38:35.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.730+0000 7f7eacae2700 1 -- 172.21.15.92:0/2769701106 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9800bbe0 con 0x7f7ea8069ab0 2023-10-06T10:38:35.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.730+0000 7f7eafd48700 1 -- 172.21.15.92:0/2769701106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ea8069ab0 msgr2=0x7f7ea80ada60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.730+0000 7f7eafd48700 1 --2- 172.21.15.92:0/2769701106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ea8069ab0 0x7f7ea80ada60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7e980098e0 tx=0x7f7e980091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.730+0000 7f7eafd48700 1 -- 172.21.15.92:0/2769701106 shutdown_connections 2023-10-06T10:38:35.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.730+0000 7f7eafd48700 1 --2- 172.21.15.92:0/2769701106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ea806d350 0x7f7ea806eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.730+0000 7f7eafd48700 1 --2- 172.21.15.92:0/2769701106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ea8069ab0 0x7f7ea80ada60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.730+0000 7f7eafd48700 1 -- 172.21.15.92:0/2769701106 >> 172.21.15.92:0/2769701106 conn(0x7f7ea8068080 msgr2=0x7f7ea80cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:35.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.730+0000 7f7eafd48700 1 -- 172.21.15.92:0/2769701106 shutdown_connections 2023-10-06T10:38:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.730+0000 7f7eafd48700 1 -- 172.21.15.92:0/2769701106 wait complete. 2023-10-06T10:38:35.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.731+0000 7f7eafd48700 1 Processor -- start 2023-10-06T10:38:35.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.731+0000 7f7eafd48700 1 -- start start 2023-10-06T10:38:35.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.732+0000 7f7eafd48700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ea80696e0 0x7f7ea8153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.732+0000 7f7eafd48700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ea8069ab0 0x7f7ea8154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.732+0000 7f7eafd48700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ea806d350 0x7f7ea8157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.732+0000 7f7eafd48700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea8167a30 con 0x7f7ea806d350 2023-10-06T10:38:35.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.732+0000 7f7eafd48700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea80c08d0 con 0x7f7ea8069ab0 2023-10-06T10:38:35.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.732+0000 7f7eafd48700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea8156ff0 con 0x7f7ea80696e0 2023-10-06T10:38:35.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.732+0000 7f7eae2e5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ea806d350 0x7f7ea8157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.732+0000 7f7eae2e5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ea806d350 0x7f7ea8157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36872/0 (socket says 172.21.15.92:36872) 2023-10-06T10:38:35.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.732+0000 7f7eae2e5700 1 -- 172.21.15.92:0/3207019272 learned_addr learned my addr 172.21.15.92:0/3207019272 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:35.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.732+0000 7f7ead2e3700 1 --2- 172.21.15.92:0/3207019272 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ea8069ab0 0x7f7ea8154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.732+0000 7f7eadae4700 1 --2- 172.21.15.92:0/3207019272 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ea80696e0 0x7f7ea8153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.732+0000 7f7eae2e5700 1 -- 172.21.15.92:0/3207019272 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ea80696e0 msgr2=0x7f7ea8153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.732+0000 7f7eae2e5700 1 --2- 172.21.15.92:0/3207019272 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ea80696e0 0x7f7ea8153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.732+0000 7f7eae2e5700 1 -- 172.21.15.92:0/3207019272 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ea8069ab0 msgr2=0x7f7ea8154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.732+0000 7f7eae2e5700 1 --2- 172.21.15.92:0/3207019272 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ea8069ab0 0x7f7ea8154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.732+0000 7f7eae2e5700 1 -- 172.21.15.92:0/3207019272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e98009420 con 0x7f7ea806d350 2023-10-06T10:38:35.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.733+0000 7f7eae2e5700 1 --2- 172.21.15.92:0/3207019272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ea806d350 0x7f7ea8157710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000bc30 tx=0x7f7ea0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.733+0000 7f7e9dffb700 1 -- 172.21.15.92:0/3207019272 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00092e0 con 0x7f7ea806d350 2023-10-06T10:38:35.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.733+0000 7f7e9dffb700 1 -- 172.21.15.92:0/3207019272 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea000c7a0 con 0x7f7ea806d350 2023-10-06T10:38:35.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.733+0000 7f7eafd48700 1 -- 172.21.15.92:0/3207019272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea80bfcc0 con 0x7f7ea806d350 2023-10-06T10:38:35.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.733+0000 7f7e9dffb700 1 -- 172.21.15.92:0/3207019272 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea000cb80 con 0x7f7ea806d350 2023-10-06T10:38:35.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.733+0000 7f7eafd48700 1 -- 172.21.15.92:0/3207019272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea80c0280 con 0x7f7ea806d350 2023-10-06T10:38:35.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.734+0000 7f7eafd48700 1 -- 172.21.15.92:0/3207019272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea8064a90 con 0x7f7ea806d350 2023-10-06T10:38:35.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.734+0000 7f7e9dffb700 1 -- 172.21.15.92:0/3207019272 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7ea0012070 con 0x7f7ea806d350 2023-10-06T10:38:35.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.735+0000 7f7e9dffb700 1 --2- 172.21.15.92:0/3207019272 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7e8c066f30 0x7f7e8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:35.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.735+0000 7f7e9dffb700 1 -- 172.21.15.92:0/3207019272 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f7ea00846b0 con 0x7f7ea806d350 2023-10-06T10:38:35.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.735+0000 7f7eadae4700 1 --2- 172.21.15.92:0/3207019272 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7e8c066f30 0x7f7e8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:35.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.735+0000 7f7eadae4700 1 --2- 172.21.15.92:0/3207019272 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7e8c066f30 0x7f7e8c0693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7ea40011e0 tx=0x7f7ea4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:35.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.739+0000 7f7e9dffb700 1 -- 172.21.15.92:0/3207019272 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ea0051430 con 0x7f7ea806d350 2023-10-06T10:38:35.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.917+0000 7f7eafd48700 1 -- 172.21.15.92:0/3207019272 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7ea80c0a10 con 0x7f7e8c066f30 2023-10-06T10:38:35.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.927+0000 7f7e9dffb700 1 -- 172.21.15.92:0/3207019272 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367636 (secure 0 0 0) 0x7f7ea40014f0 con 0x7f7e8c066f30 2023-10-06T10:38:35.935 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:35.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.930+0000 7f7eafd48700 1 -- 172.21.15.92:0/3207019272 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7e8c066f30 msgr2=0x7f7e8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.930+0000 7f7eafd48700 1 --2- 172.21.15.92:0/3207019272 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7e8c066f30 0x7f7e8c0693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7ea40011e0 tx=0x7f7ea4009040 comp rx=0 tx=0).stop 2023-10-06T10:38:35.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.930+0000 7f7eafd48700 1 -- 172.21.15.92:0/3207019272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ea806d350 msgr2=0x7f7ea8157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:35.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.930+0000 7f7eafd48700 1 --2- 172.21.15.92:0/3207019272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ea806d350 0x7f7ea8157710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000bc30 tx=0x7f7ea0009840 comp rx=0 tx=0).stop 2023-10-06T10:38:35.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.930+0000 7f7eafd48700 1 -- 172.21.15.92:0/3207019272 shutdown_connections 2023-10-06T10:38:35.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.930+0000 7f7eafd48700 1 --2- 172.21.15.92:0/3207019272 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7e8c066f30 0x7f7e8c0693e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.930+0000 7f7eafd48700 1 --2- 172.21.15.92:0/3207019272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ea806d350 0x7f7ea8157710 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.930+0000 7f7eafd48700 1 --2- 172.21.15.92:0/3207019272 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ea80696e0 0x7f7ea8153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.930+0000 7f7eafd48700 1 --2- 172.21.15.92:0/3207019272 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ea8069ab0 0x7f7ea8154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:35.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.931+0000 7f7eafd48700 1 -- 172.21.15.92:0/3207019272 >> 172.21.15.92:0/3207019272 conn(0x7f7ea8068080 msgr2=0x7f7ea80c8890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:35.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.931+0000 7f7eafd48700 1 -- 172.21.15.92:0/3207019272 shutdown_connections 2023-10-06T10:38:35.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:35.931+0000 7f7eafd48700 1 -- 172.21.15.92:0/3207019272 wait complete. 2023-10-06T10:38:35.939 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:38:35.945 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-10-06T10:38:34.084589+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594852,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381508,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593212,"statfs":{"total":773094113280,"available":770655449088,"internally_reserved":0,"allocated":3280896,"data_stored":2023855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2178,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.702429"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384275+0000","last_change":"2023-10-06T10:38:17.365963+0000","last_active":"2023-10-06T10:38:21.384275+0000","last_peered":"2023-10-06T10:38:21.384275+0000","last_clean":"2023-10-06T10:38:21.384275+0000","last_became_active":"2023-10-06T10:38:17.365702+0000","last_became_peered":"2023-10-06T10:38:17.365702+0000","last_unstale":"2023-10-06T10:38:21.384275+0000","last_undegraded":"2023-10-06T10:38:21.384275+0000","last_fullsized":"2023-10-06T10:38:21.384275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:58.222423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378758+0000","last_change":"2023-10-06T10:38:17.362155+0000","last_active":"2023-10-06T10:38:21.378758+0000","last_peered":"2023-10-06T10:38:21.378758+0000","last_clean":"2023-10-06T10:38:21.378758+0000","last_became_active":"2023-10-06T10:38:17.361871+0000","last_became_peered":"2023-10-06T10:38:17.361871+0000","last_unstale":"2023-10-06T10:38:21.378758+0000","last_undegraded":"2023-10-06T10:38:21.378758+0000","last_fullsized":"2023-10-06T10:38:21.378758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:49.973815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377999+0000","last_change":"2023-10-06T10:38:17.361502+0000","last_active":"2023-10-06T10:38:21.377999+0000","last_peered":"2023-10-06T10:38:21.377999+0000","last_clean":"2023-10-06T10:38:21.377999+0000","last_became_active":"2023-10-06T10:38:17.360866+0000","last_became_peered":"2023-10-06T10:38:17.360866+0000","last_unstale":"2023-10-06T10:38:21.377999+0000","last_undegraded":"2023-10-06T10:38:21.377999+0000","last_fullsized":"2023-10-06T10:38:21.377999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:01.022639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384859+0000","last_change":"2023-10-06T10:38:17.366612+0000","last_active":"2023-10-06T10:38:21.384859+0000","last_peered":"2023-10-06T10:38:21.384859+0000","last_clean":"2023-10-06T10:38:21.384859+0000","last_became_active":"2023-10-06T10:38:17.366337+0000","last_became_peered":"2023-10-06T10:38:17.366337+0000","last_unstale":"2023-10-06T10:38:21.384859+0000","last_undegraded":"2023-10-06T10:38:21.384859+0000","last_fullsized":"2023-10-06T10:38:21.384859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:01:12.162890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377172+0000","last_change":"2023-10-06T10:38:17.363460+0000","last_active":"2023-10-06T10:38:21.377172+0000","last_peered":"2023-10-06T10:38:21.377172+0000","last_clean":"2023-10-06T10:38:21.377172+0000","last_became_active":"2023-10-06T10:38:17.363211+0000","last_became_peered":"2023-10-06T10:38:17.363211+0000","last_unstale":"2023-10-06T10:38:21.377172+0000","last_undegraded":"2023-10-06T10:38:21.377172+0000","last_fullsized":"2023-10-06T10:38:21.377172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.132681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378316+0000","last_change":"2023-10-06T10:38:17.364098+0000","last_active":"2023-10-06T10:38:21.378316+0000","last_peered":"2023-10-06T10:38:21.378316+0000","last_clean":"2023-10-06T10:38:21.378316+0000","last_became_active":"2023-10-06T10:38:17.363806+0000","last_became_peered":"2023-10-06T10:38:17.363806+0000","last_unstale":"2023-10-06T10:38:21.378316+0000","last_undegraded":"2023-10-06T10:38:21.378316+0000","last_fullsized":"2023-10-06T10:38:21.378316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:27.532414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391179+0000","last_change":"2023-10-06T10:38:17.372436+0000","last_active":"2023-10-06T10:38:21.391179+0000","last_peered":"2023-10-06T10:38:21.391179+0000","last_clean":"2023-10-06T10:38:21.391179+0000","last_became_active":"2023-10-06T10:38:17.371126+0000","last_became_peered":"2023-10-06T10:38:17.371126+0000","last_unstale":"2023-10-06T10:38:21.391179+0000","last_undegraded":"2023-10-06T10:38:21.391179+0000","last_fullsized":"2023-10-06T10:38:21.391179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:13.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375652+0000","last_change":"2023-10-06T10:38:17.354022+0000","last_active":"2023-10-06T10:38:21.375652+0000","last_peered":"2023-10-06T10:38:21.375652+0000","last_clean":"2023-10-06T10:38:21.375652+0000","last_became_active":"2023-10-06T10:38:17.353630+0000","last_became_peered":"2023-10-06T10:38:17.353630+0000","last_unstale":"2023-10-06T10:38:21.375652+0000","last_undegraded":"2023-10-06T10:38:21.375652+0000","last_fullsized":"2023-10-06T10:38:21.375652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:03.953719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378266+0000","last_change":"2023-10-06T10:38:17.362560+0000","last_active":"2023-10-06T10:38:21.378266+0000","last_peered":"2023-10-06T10:38:21.378266+0000","last_clean":"2023-10-06T10:38:21.378266+0000","last_became_active":"2023-10-06T10:38:17.361953+0000","last_became_peered":"2023-10-06T10:38:17.361953+0000","last_unstale":"2023-10-06T10:38:21.378266+0000","last_undegraded":"2023-10-06T10:38:21.378266+0000","last_fullsized":"2023-10-06T10:38:21.378266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:38.417248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378932+0000","last_change":"2023-10-06T10:38:17.364442+0000","last_active":"2023-10-06T10:38:21.378932+0000","last_peered":"2023-10-06T10:38:21.378932+0000","last_clean":"2023-10-06T10:38:21.378932+0000","last_became_active":"2023-10-06T10:38:17.363787+0000","last_became_peered":"2023-10-06T10:38:17.363787+0000","last_unstale":"2023-10-06T10:38:21.378932+0000","last_undegraded":"2023-10-06T10:38:21.378932+0000","last_fullsized":"2023-10-06T10:38:21.378932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:13.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377971+0000","last_change":"2023-10-06T10:38:17.366791+0000","last_active":"2023-10-06T10:38:21.377971+0000","last_peered":"2023-10-06T10:38:21.377971+0000","last_clean":"2023-10-06T10:38:21.377971+0000","last_became_active":"2023-10-06T10:38:17.366281+0000","last_became_peered":"2023-10-06T10:38:17.366281+0000","last_unstale":"2023-10-06T10:38:21.377971+0000","last_undegraded":"2023-10-06T10:38:21.377971+0000","last_fullsized":"2023-10-06T10:38:21.377971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:22.349323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.383809+0000","last_change":"2023-10-06T10:38:17.368900+0000","last_active":"2023-10-06T10:38:21.383809+0000","last_peered":"2023-10-06T10:38:21.383809+0000","last_clean":"2023-10-06T10:38:21.383809+0000","last_became_active":"2023-10-06T10:38:17.368488+0000","last_became_peered":"2023-10-06T10:38:17.368488+0000","last_unstale":"2023-10-06T10:38:21.383809+0000","last_undegraded":"2023-10-06T10:38:21.383809+0000","last_fullsized":"2023-10-06T10:38:21.383809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:07.365430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.383322+0000","last_change":"2023-10-06T10:38:17.365338+0000","last_active":"2023-10-06T10:38:21.383322+0000","last_peered":"2023-10-06T10:38:21.383322+0000","last_clean":"2023-10-06T10:38:21.383322+0000","last_became_active":"2023-10-06T10:38:17.365056+0000","last_became_peered":"2023-10-06T10:38:17.365056+0000","last_unstale":"2023-10-06T10:38:21.383322+0000","last_undegraded":"2023-10-06T10:38:21.383322+0000","last_fullsized":"2023-10-06T10:38:21.383322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:03.122773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375751+0000","last_change":"2023-10-06T10:38:17.366413+0000","last_active":"2023-10-06T10:38:21.375751+0000","last_peered":"2023-10-06T10:38:21.375751+0000","last_clean":"2023-10-06T10:38:21.375751+0000","last_became_active":"2023-10-06T10:38:17.366139+0000","last_became_peered":"2023-10-06T10:38:17.366139+0000","last_unstale":"2023-10-06T10:38:21.375751+0000","last_undegraded":"2023-10-06T10:38:21.375751+0000","last_fullsized":"2023-10-06T10:38:21.375751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:34:27.066675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391014+0000","last_change":"2023-10-06T10:38:17.371250+0000","last_active":"2023-10-06T10:38:21.391014+0000","last_peered":"2023-10-06T10:38:21.391014+0000","last_clean":"2023-10-06T10:38:21.391014+0000","last_became_active":"2023-10-06T10:38:17.370858+0000","last_became_peered":"2023-10-06T10:38:17.370858+0000","last_unstale":"2023-10-06T10:38:21.391014+0000","last_undegraded":"2023-10-06T10:38:21.391014+0000","last_fullsized":"2023-10-06T10:38:21.391014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:28.001417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378433+0000","last_change":"2023-10-06T10:38:17.364371+0000","last_active":"2023-10-06T10:38:21.378433+0000","last_peered":"2023-10-06T10:38:21.378433+0000","last_clean":"2023-10-06T10:38:21.378433+0000","last_became_active":"2023-10-06T10:38:17.364040+0000","last_became_peered":"2023-10-06T10:38:17.364040+0000","last_unstale":"2023-10-06T10:38:21.378433+0000","last_undegraded":"2023-10-06T10:38:21.378433+0000","last_fullsized":"2023-10-06T10:38:21.378433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:29.897092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378355+0000","last_change":"2023-10-06T10:38:17.363058+0000","last_active":"2023-10-06T10:38:21.378355+0000","last_peered":"2023-10-06T10:38:21.378355+0000","last_clean":"2023-10-06T10:38:21.378355+0000","last_became_active":"2023-10-06T10:38:17.359050+0000","last_became_peered":"2023-10-06T10:38:17.359050+0000","last_unstale":"2023-10-06T10:38:21.378355+0000","last_undegraded":"2023-10-06T10:38:21.378355+0000","last_fullsized":"2023-10-06T10:38:21.378355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:28.001417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377744+0000","last_change":"2023-10-06T10:38:17.365982+0000","last_active":"2023-10-06T10:38:21.377744+0000","last_peered":"2023-10-06T10:38:21.377744+0000","last_clean":"2023-10-06T10:38:21.377744+0000","last_became_active":"2023-10-06T10:38:17.364876+0000","last_became_peered":"2023-10-06T10:38:17.364876+0000","last_unstale":"2023-10-06T10:38:21.377744+0000","last_undegraded":"2023-10-06T10:38:21.377744+0000","last_fullsized":"2023-10-06T10:38:21.377744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:13.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.383516+0000","last_change":"2023-10-06T10:38:17.368880+0000","last_active":"2023-10-06T10:38:21.383516+0000","last_peered":"2023-10-06T10:38:21.383516+0000","last_clean":"2023-10-06T10:38:21.383516+0000","last_became_active":"2023-10-06T10:38:17.368645+0000","last_became_peered":"2023-10-06T10:38:17.368645+0000","last_unstale":"2023-10-06T10:38:21.383516+0000","last_undegraded":"2023-10-06T10:38:21.383516+0000","last_fullsized":"2023-10-06T10:38:21.383516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:42.844587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377767+0000","last_change":"2023-10-06T10:38:17.375213+0000","last_active":"2023-10-06T10:38:21.377767+0000","last_peered":"2023-10-06T10:38:21.377767+0000","last_clean":"2023-10-06T10:38:21.377767+0000","last_became_active":"2023-10-06T10:38:17.374836+0000","last_became_peered":"2023-10-06T10:38:17.374836+0000","last_unstale":"2023-10-06T10:38:21.377767+0000","last_undegraded":"2023-10-06T10:38:21.377767+0000","last_fullsized":"2023-10-06T10:38:21.377767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:01.727252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.386555+0000","last_change":"2023-10-06T10:38:17.370625+0000","last_active":"2023-10-06T10:38:21.386555+0000","last_peered":"2023-10-06T10:38:21.386555+0000","last_clean":"2023-10-06T10:38:21.386555+0000","last_became_active":"2023-10-06T10:38:17.370355+0000","last_became_peered":"2023-10-06T10:38:17.370355+0000","last_unstale":"2023-10-06T10:38:21.386555+0000","last_undegraded":"2023-10-06T10:38:21.386555+0000","last_fullsized":"2023-10-06T10:38:21.386555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:38.417248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375531+0000","last_change":"2023-10-06T10:38:17.360622+0000","last_active":"2023-10-06T10:38:21.375531+0000","last_peered":"2023-10-06T10:38:21.375531+0000","last_clean":"2023-10-06T10:38:21.375531+0000","last_became_active":"2023-10-06T10:38:17.359689+0000","last_became_peered":"2023-10-06T10:38:17.359689+0000","last_unstale":"2023-10-06T10:38:21.375531+0000","last_undegraded":"2023-10-06T10:38:21.375531+0000","last_fullsized":"2023-10-06T10:38:21.375531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:03.650819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372352+0000","last_change":"2023-10-06T10:38:17.362700+0000","last_active":"2023-10-06T10:38:21.372352+0000","last_peered":"2023-10-06T10:38:21.372352+0000","last_clean":"2023-10-06T10:38:21.372352+0000","last_became_active":"2023-10-06T10:38:17.362457+0000","last_became_peered":"2023-10-06T10:38:17.362457+0000","last_unstale":"2023-10-06T10:38:21.372352+0000","last_undegraded":"2023-10-06T10:38:21.372352+0000","last_fullsized":"2023-10-06T10:38:21.372352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:50.378097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.393253+0000","last_change":"2023-10-06T10:38:17.371996+0000","last_active":"2023-10-06T10:38:21.393253+0000","last_peered":"2023-10-06T10:38:21.393253+0000","last_clean":"2023-10-06T10:38:21.393253+0000","last_became_active":"2023-10-06T10:38:17.371731+0000","last_became_peered":"2023-10-06T10:38:17.371731+0000","last_unstale":"2023-10-06T10:38:21.393253+0000","last_undegraded":"2023-10-06T10:38:21.393253+0000","last_fullsized":"2023-10-06T10:38:21.393253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:06.214524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378923+0000","last_change":"2023-10-06T10:38:17.373079+0000","last_active":"2023-10-06T10:38:21.378923+0000","last_peered":"2023-10-06T10:38:21.378923+0000","last_clean":"2023-10-06T10:38:21.378923+0000","last_became_active":"2023-10-06T10:38:17.372799+0000","last_became_peered":"2023-10-06T10:38:17.372799+0000","last_unstale":"2023-10-06T10:38:21.378923+0000","last_undegraded":"2023-10-06T10:38:21.378923+0000","last_fullsized":"2023-10-06T10:38:21.378923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:03.953719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.379048+0000","last_change":"2023-10-06T10:38:17.366829+0000","last_active":"2023-10-06T10:38:21.379048+0000","last_peered":"2023-10-06T10:38:21.379048+0000","last_clean":"2023-10-06T10:38:21.379048+0000","last_became_active":"2023-10-06T10:38:17.366617+0000","last_became_peered":"2023-10-06T10:38:17.366617+0000","last_unstale":"2023-10-06T10:38:21.379048+0000","last_undegraded":"2023-10-06T10:38:21.379048+0000","last_fullsized":"2023-10-06T10:38:21.379048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:35.729655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378140+0000","last_change":"2023-10-06T10:38:17.371330+0000","last_active":"2023-10-06T10:38:21.378140+0000","last_peered":"2023-10-06T10:38:21.378140+0000","last_clean":"2023-10-06T10:38:21.378140+0000","last_became_active":"2023-10-06T10:38:17.370864+0000","last_became_peered":"2023-10-06T10:38:17.370864+0000","last_unstale":"2023-10-06T10:38:21.378140+0000","last_undegraded":"2023-10-06T10:38:21.378140+0000","last_fullsized":"2023-10-06T10:38:21.378140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:38.417248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391173+0000","last_change":"2023-10-06T10:38:21.389792+0000","last_active":"2023-10-06T10:38:21.391173+0000","last_peered":"2023-10-06T10:38:21.391173+0000","last_clean":"2023-10-06T10:38:21.391173+0000","last_became_active":"2023-10-06T10:38:21.389400+0000","last_became_peered":"2023-10-06T10:38:21.389400+0000","last_unstale":"2023-10-06T10:38:21.391173+0000","last_undegraded":"2023-10-06T10:38:21.391173+0000","last_fullsized":"2023-10-06T10:38:21.391173+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:55.438723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378102+0000","last_change":"2023-10-06T10:38:16.366893+0000","last_active":"2023-10-06T10:38:21.378102+0000","last_peered":"2023-10-06T10:38:21.378102+0000","last_clean":"2023-10-06T10:38:21.378102+0000","last_became_active":"2023-10-06T10:38:16.366637+0000","last_became_peered":"2023-10-06T10:38:16.366637+0000","last_unstale":"2023-10-06T10:38:21.378102+0000","last_undegraded":"2023-10-06T10:38:21.378102+0000","last_fullsized":"2023-10-06T10:38:21.378102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:06.358487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372687+0000","last_change":"2023-10-06T10:38:17.360671+0000","last_active":"2023-10-06T10:38:21.372687+0000","last_peered":"2023-10-06T10:38:21.372687+0000","last_clean":"2023-10-06T10:38:21.372687+0000","last_became_active":"2023-10-06T10:38:17.360439+0000","last_became_peered":"2023-10-06T10:38:17.360439+0000","last_unstale":"2023-10-06T10:38:21.372687+0000","last_undegraded":"2023-10-06T10:38:21.372687+0000","last_fullsized":"2023-10-06T10:38:21.372687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:45:42.362885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402648+0000","last_change":"2023-10-06T10:38:21.399346+0000","last_active":"2023-10-06T10:38:21.402648+0000","last_peered":"2023-10-06T10:38:21.402648+0000","last_clean":"2023-10-06T10:38:21.402648+0000","last_became_active":"2023-10-06T10:38:21.398877+0000","last_became_peered":"2023-10-06T10:38:21.398877+0000","last_unstale":"2023-10-06T10:38:21.402648+0000","last_undegraded":"2023-10-06T10:38:21.402648+0000","last_fullsized":"2023-10-06T10:38:21.402648+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:05.052271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.376684+0000","last_change":"2023-10-06T10:38:16.357015+0000","last_active":"2023-10-06T10:38:21.376684+0000","last_peered":"2023-10-06T10:38:21.376684+0000","last_clean":"2023-10-06T10:38:21.376684+0000","last_became_active":"2023-10-06T10:38:16.356733+0000","last_became_peered":"2023-10-06T10:38:16.356733+0000","last_unstale":"2023-10-06T10:38:21.376684+0000","last_undegraded":"2023-10-06T10:38:21.376684+0000","last_fullsized":"2023-10-06T10:38:21.376684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:06.358487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375991+0000","last_change":"2023-10-06T10:38:17.370483+0000","last_active":"2023-10-06T10:38:21.375991+0000","last_peered":"2023-10-06T10:38:21.375991+0000","last_clean":"2023-10-06T10:38:21.375991+0000","last_became_active":"2023-10-06T10:38:17.370196+0000","last_became_peered":"2023-10-06T10:38:17.370196+0000","last_unstale":"2023-10-06T10:38:21.375991+0000","last_undegraded":"2023-10-06T10:38:21.375991+0000","last_fullsized":"2023-10-06T10:38:21.375991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:49.973815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.398933+0000","last_change":"2023-10-06T10:38:21.397171+0000","last_active":"2023-10-06T10:38:21.398933+0000","last_peered":"2023-10-06T10:38:21.398933+0000","last_clean":"2023-10-06T10:38:21.398933+0000","last_became_active":"2023-10-06T10:38:21.396794+0000","last_became_peered":"2023-10-06T10:38:21.396794+0000","last_unstale":"2023-10-06T10:38:21.398933+0000","last_undegraded":"2023-10-06T10:38:21.398933+0000","last_fullsized":"2023-10-06T10:38:21.398933+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:30.495506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372622+0000","last_change":"2023-10-06T10:38:16.362088+0000","last_active":"2023-10-06T10:38:21.372622+0000","last_peered":"2023-10-06T10:38:21.372622+0000","last_clean":"2023-10-06T10:38:21.372622+0000","last_became_active":"2023-10-06T10:38:16.361790+0000","last_became_peered":"2023-10-06T10:38:16.361790+0000","last_unstale":"2023-10-06T10:38:21.372622+0000","last_undegraded":"2023-10-06T10:38:21.372622+0000","last_fullsized":"2023-10-06T10:38:21.372622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:20.696308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385166+0000","last_change":"2023-10-06T10:38:17.372313+0000","last_active":"2023-10-06T10:38:21.385166+0000","last_peered":"2023-10-06T10:38:21.385166+0000","last_clean":"2023-10-06T10:38:21.385166+0000","last_became_active":"2023-10-06T10:38:17.369061+0000","last_became_peered":"2023-10-06T10:38:17.369061+0000","last_unstale":"2023-10-06T10:38:21.385166+0000","last_undegraded":"2023-10-06T10:38:21.385166+0000","last_fullsized":"2023-10-06T10:38:21.385166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:25.807302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402371+0000","last_change":"2023-10-06T10:38:21.399352+0000","last_active":"2023-10-06T10:38:21.402371+0000","last_peered":"2023-10-06T10:38:21.402371+0000","last_clean":"2023-10-06T10:38:21.402371+0000","last_became_active":"2023-10-06T10:38:21.398679+0000","last_became_peered":"2023-10-06T10:38:21.398679+0000","last_unstale":"2023-10-06T10:38:21.402371+0000","last_undegraded":"2023-10-06T10:38:21.402371+0000","last_fullsized":"2023-10-06T10:38:21.402371+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:14.207234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378436+0000","last_change":"2023-10-06T10:38:16.361297+0000","last_active":"2023-10-06T10:38:21.378436+0000","last_peered":"2023-10-06T10:38:21.378436+0000","last_clean":"2023-10-06T10:38:21.378436+0000","last_became_active":"2023-10-06T10:38:16.361034+0000","last_became_peered":"2023-10-06T10:38:16.361034+0000","last_unstale":"2023-10-06T10:38:21.378436+0000","last_undegraded":"2023-10-06T10:38:21.378436+0000","last_fullsized":"2023-10-06T10:38:21.378436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:02.115831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372551+0000","last_change":"2023-10-06T10:38:17.361626+0000","last_active":"2023-10-06T10:38:21.372551+0000","last_peered":"2023-10-06T10:38:21.372551+0000","last_clean":"2023-10-06T10:38:21.372551+0000","last_became_active":"2023-10-06T10:38:17.361386+0000","last_became_peered":"2023-10-06T10:38:17.361386+0000","last_unstale":"2023-10-06T10:38:21.372551+0000","last_undegraded":"2023-10-06T10:38:21.372551+0000","last_fullsized":"2023-10-06T10:38:21.372551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.132681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.397814+0000","last_change":"2023-10-06T10:38:21.395253+0000","last_active":"2023-10-06T10:38:21.397814+0000","last_peered":"2023-10-06T10:38:21.397814+0000","last_clean":"2023-10-06T10:38:21.397814+0000","last_became_active":"2023-10-06T10:38:21.394839+0000","last_became_peered":"2023-10-06T10:38:21.394839+0000","last_unstale":"2023-10-06T10:38:21.397814+0000","last_undegraded":"2023-10-06T10:38:21.397814+0000","last_fullsized":"2023-10-06T10:38:21.397814+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:43.578698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378846+0000","last_change":"2023-10-06T10:38:16.359731+0000","last_active":"2023-10-06T10:38:21.378846+0000","last_peered":"2023-10-06T10:38:21.378846+0000","last_clean":"2023-10-06T10:38:21.378846+0000","last_became_active":"2023-10-06T10:38:16.359451+0000","last_became_peered":"2023-10-06T10:38:16.359451+0000","last_unstale":"2023-10-06T10:38:21.378846+0000","last_undegraded":"2023-10-06T10:38:21.378846+0000","last_fullsized":"2023-10-06T10:38:21.378846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:06.358487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372246+0000","last_change":"2023-10-06T10:38:17.371057+0000","last_active":"2023-10-06T10:38:21.372246+0000","last_peered":"2023-10-06T10:38:21.372246+0000","last_clean":"2023-10-06T10:38:21.372246+0000","last_became_active":"2023-10-06T10:38:17.370898+0000","last_became_peered":"2023-10-06T10:38:17.370898+0000","last_unstale":"2023-10-06T10:38:21.372246+0000","last_undegraded":"2023-10-06T10:38:21.372246+0000","last_fullsized":"2023-10-06T10:38:21.372246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:59.822053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.393290+0000","last_change":"2023-10-06T10:38:21.387382+0000","last_active":"2023-10-06T10:38:21.393290+0000","last_peered":"2023-10-06T10:38:21.393290+0000","last_clean":"2023-10-06T10:38:21.393290+0000","last_became_active":"2023-10-06T10:38:21.386886+0000","last_became_peered":"2023-10-06T10:38:21.386886+0000","last_unstale":"2023-10-06T10:38:21.393290+0000","last_undegraded":"2023-10-06T10:38:21.393290+0000","last_fullsized":"2023-10-06T10:38:21.393290+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:04:14.062486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.392484+0000","last_change":"2023-10-06T10:38:16.362467+0000","last_active":"2023-10-06T10:38:21.392484+0000","last_peered":"2023-10-06T10:38:21.392484+0000","last_clean":"2023-10-06T10:38:21.392484+0000","last_became_active":"2023-10-06T10:38:16.362094+0000","last_became_peered":"2023-10-06T10:38:16.362094+0000","last_unstale":"2023-10-06T10:38:21.392484+0000","last_undegraded":"2023-10-06T10:38:21.392484+0000","last_fullsized":"2023-10-06T10:38:21.392484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:24.800360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378505+0000","last_change":"2023-10-06T10:38:17.371049+0000","last_active":"2023-10-06T10:38:21.378505+0000","last_peered":"2023-10-06T10:38:21.378505+0000","last_clean":"2023-10-06T10:38:21.378505+0000","last_became_active":"2023-10-06T10:38:17.370698+0000","last_became_peered":"2023-10-06T10:38:17.370698+0000","last_unstale":"2023-10-06T10:38:21.378505+0000","last_undegraded":"2023-10-06T10:38:21.378505+0000","last_fullsized":"2023-10-06T10:38:21.378505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:28.001417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.400543+0000","last_change":"2023-10-06T10:38:21.395461+0000","last_active":"2023-10-06T10:38:21.400543+0000","last_peered":"2023-10-06T10:38:21.400543+0000","last_clean":"2023-10-06T10:38:21.400543+0000","last_became_active":"2023-10-06T10:38:21.395077+0000","last_became_peered":"2023-10-06T10:38:21.395077+0000","last_unstale":"2023-10-06T10:38:21.400543+0000","last_undegraded":"2023-10-06T10:38:21.400543+0000","last_fullsized":"2023-10-06T10:38:21.400543+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:43.578698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'9","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T10:38:27.096420+0000","last_change":"2023-10-06T10:38:16.359809+0000","last_active":"2023-10-06T10:38:27.096420+0000","last_peered":"2023-10-06T10:38:27.096420+0000","last_clean":"2023-10-06T10:38:27.096420+0000","last_became_active":"2023-10-06T10:38:16.359520+0000","last_became_peered":"2023-10-06T10:38:16.359520+0000","last_unstale":"2023-10-06T10:38:27.096420+0000","last_undegraded":"2023-10-06T10:38:27.096420+0000","last_fullsized":"2023-10-06T10:38:27.096420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:41.837644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3714,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377665+0000","last_change":"2023-10-06T10:38:17.375788+0000","last_active":"2023-10-06T10:38:21.377665+0000","last_peered":"2023-10-06T10:38:21.377665+0000","last_clean":"2023-10-06T10:38:21.377665+0000","last_became_active":"2023-10-06T10:38:17.375416+0000","last_became_peered":"2023-10-06T10:38:17.375416+0000","last_unstale":"2023-10-06T10:38:21.377665+0000","last_undegraded":"2023-10-06T10:38:21.377665+0000","last_fullsized":"2023-10-06T10:38:21.377665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.132681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.396799+0000","last_change":"2023-10-06T10:38:21.391788+0000","last_active":"2023-10-06T10:38:21.396799+0000","last_peered":"2023-10-06T10:38:21.396799+0000","last_clean":"2023-10-06T10:38:21.396799+0000","last_became_active":"2023-10-06T10:38:21.391376+0000","last_became_peered":"2023-10-06T10:38:21.391376+0000","last_unstale":"2023-10-06T10:38:21.396799+0000","last_undegraded":"2023-10-06T10:38:21.396799+0000","last_fullsized":"2023-10-06T10:38:21.396799+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:25:40.423524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384475+0000","last_change":"2023-10-06T10:38:16.363389+0000","last_active":"2023-10-06T10:38:21.384475+0000","last_peered":"2023-10-06T10:38:21.384475+0000","last_clean":"2023-10-06T10:38:21.384475+0000","last_became_active":"2023-10-06T10:38:16.363141+0000","last_became_peered":"2023-10-06T10:38:16.363141+0000","last_unstale":"2023-10-06T10:38:21.384475+0000","last_undegraded":"2023-10-06T10:38:21.384475+0000","last_fullsized":"2023-10-06T10:38:21.384475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:01:11.155948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372587+0000","last_change":"2023-10-06T10:38:17.361247+0000","last_active":"2023-10-06T10:38:21.372587+0000","last_peered":"2023-10-06T10:38:21.372587+0000","last_clean":"2023-10-06T10:38:21.372587+0000","last_became_active":"2023-10-06T10:38:17.361016+0000","last_became_peered":"2023-10-06T10:38:17.361016+0000","last_unstale":"2023-10-06T10:38:21.372587+0000","last_undegraded":"2023-10-06T10:38:21.372587+0000","last_fullsized":"2023-10-06T10:38:21.372587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:38.417248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.398908+0000","last_change":"2023-10-06T10:38:21.395222+0000","last_active":"2023-10-06T10:38:21.398908+0000","last_peered":"2023-10-06T10:38:21.398908+0000","last_clean":"2023-10-06T10:38:21.398908+0000","last_became_active":"2023-10-06T10:38:21.394695+0000","last_became_peered":"2023-10-06T10:38:21.394695+0000","last_unstale":"2023-10-06T10:38:21.398908+0000","last_undegraded":"2023-10-06T10:38:21.398908+0000","last_fullsized":"2023-10-06T10:38:21.398908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:36:21.761225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375670+0000","last_change":"2023-10-06T10:38:16.359510+0000","last_active":"2023-10-06T10:38:21.375670+0000","last_peered":"2023-10-06T10:38:21.375670+0000","last_clean":"2023-10-06T10:38:21.375670+0000","last_became_active":"2023-10-06T10:38:16.359236+0000","last_became_peered":"2023-10-06T10:38:16.359236+0000","last_unstale":"2023-10-06T10:38:21.375670+0000","last_undegraded":"2023-10-06T10:38:21.375670+0000","last_fullsized":"2023-10-06T10:38:21.375670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:16.528477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378073+0000","last_change":"2023-10-06T10:38:17.371135+0000","last_active":"2023-10-06T10:38:21.378073+0000","last_peered":"2023-10-06T10:38:21.378073+0000","last_clean":"2023-10-06T10:38:21.378073+0000","last_became_active":"2023-10-06T10:38:17.370494+0000","last_became_peered":"2023-10-06T10:38:17.370494+0000","last_unstale":"2023-10-06T10:38:21.378073+0000","last_undegraded":"2023-10-06T10:38:21.378073+0000","last_fullsized":"2023-10-06T10:38:21.378073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:28.001417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.398908+0000","last_change":"2023-10-06T10:38:21.395871+0000","last_active":"2023-10-06T10:38:21.398908+0000","last_peered":"2023-10-06T10:38:21.398908+0000","last_clean":"2023-10-06T10:38:21.398908+0000","last_became_active":"2023-10-06T10:38:21.395558+0000","last_became_peered":"2023-10-06T10:38:21.395558+0000","last_unstale":"2023-10-06T10:38:21.398908+0000","last_undegraded":"2023-10-06T10:38:21.398908+0000","last_fullsized":"2023-10-06T10:38:21.398908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:32:37.103555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391095+0000","last_change":"2023-10-06T10:38:16.363354+0000","last_active":"2023-10-06T10:38:21.391095+0000","last_peered":"2023-10-06T10:38:21.391095+0000","last_clean":"2023-10-06T10:38:21.391095+0000","last_became_active":"2023-10-06T10:38:16.363088+0000","last_became_peered":"2023-10-06T10:38:16.363088+0000","last_unstale":"2023-10-06T10:38:21.391095+0000","last_undegraded":"2023-10-06T10:38:21.391095+0000","last_fullsized":"2023-10-06T10:38:21.391095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:39.672304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.376233+0000","last_change":"2023-10-06T10:38:17.359300+0000","last_active":"2023-10-06T10:38:21.376233+0000","last_peered":"2023-10-06T10:38:21.376233+0000","last_clean":"2023-10-06T10:38:21.376233+0000","last_became_active":"2023-10-06T10:38:17.358652+0000","last_became_peered":"2023-10-06T10:38:17.358652+0000","last_unstale":"2023-10-06T10:38:21.376233+0000","last_undegraded":"2023-10-06T10:38:21.376233+0000","last_fullsized":"2023-10-06T10:38:21.376233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:45:42.362885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.400005+0000","last_change":"2023-10-06T10:38:21.394129+0000","last_active":"2023-10-06T10:38:21.400005+0000","last_peered":"2023-10-06T10:38:21.400005+0000","last_clean":"2023-10-06T10:38:21.400005+0000","last_became_active":"2023-10-06T10:38:21.393517+0000","last_became_peered":"2023-10-06T10:38:21.393517+0000","last_unstale":"2023-10-06T10:38:21.400005+0000","last_undegraded":"2023-10-06T10:38:21.400005+0000","last_fullsized":"2023-10-06T10:38:21.400005+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:55:52.407995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385053+0000","last_change":"2023-10-06T10:38:16.364742+0000","last_active":"2023-10-06T10:38:21.385053+0000","last_peered":"2023-10-06T10:38:21.385053+0000","last_clean":"2023-10-06T10:38:21.385053+0000","last_became_active":"2023-10-06T10:38:16.364410+0000","last_became_peered":"2023-10-06T10:38:16.364410+0000","last_unstale":"2023-10-06T10:38:21.385053+0000","last_undegraded":"2023-10-06T10:38:21.385053+0000","last_fullsized":"2023-10-06T10:38:21.385053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:06.358487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.373267+0000","last_change":"2023-10-06T10:38:17.374643+0000","last_active":"2023-10-06T10:38:21.373267+0000","last_peered":"2023-10-06T10:38:21.373267+0000","last_clean":"2023-10-06T10:38:21.373267+0000","last_became_active":"2023-10-06T10:38:17.374436+0000","last_became_peered":"2023-10-06T10:38:17.374436+0000","last_unstale":"2023-10-06T10:38:21.373267+0000","last_undegraded":"2023-10-06T10:38:21.373267+0000","last_fullsized":"2023-10-06T10:38:21.373267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:01.022639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402862+0000","last_change":"2023-10-06T10:38:21.401121+0000","last_active":"2023-10-06T10:38:21.402862+0000","last_peered":"2023-10-06T10:38:21.402862+0000","last_clean":"2023-10-06T10:38:21.402862+0000","last_became_active":"2023-10-06T10:38:21.400766+0000","last_became_peered":"2023-10-06T10:38:21.400766+0000","last_unstale":"2023-10-06T10:38:21.402862+0000","last_undegraded":"2023-10-06T10:38:21.402862+0000","last_fullsized":"2023-10-06T10:38:21.402862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:39.967976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.392673+0000","last_change":"2023-10-06T10:38:16.363994+0000","last_active":"2023-10-06T10:38:21.392673+0000","last_peered":"2023-10-06T10:38:21.392673+0000","last_clean":"2023-10-06T10:38:21.392673+0000","last_became_active":"2023-10-06T10:38:16.363712+0000","last_became_peered":"2023-10-06T10:38:16.363712+0000","last_unstale":"2023-10-06T10:38:21.392673+0000","last_undegraded":"2023-10-06T10:38:21.392673+0000","last_fullsized":"2023-10-06T10:38:21.392673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:43.249942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375546+0000","last_change":"2023-10-06T10:38:17.374214+0000","last_active":"2023-10-06T10:38:21.375546+0000","last_peered":"2023-10-06T10:38:21.375546+0000","last_clean":"2023-10-06T10:38:21.375546+0000","last_became_active":"2023-10-06T10:38:17.373998+0000","last_became_peered":"2023-10-06T10:38:17.373998+0000","last_unstale":"2023-10-06T10:38:21.375546+0000","last_undegraded":"2023-10-06T10:38:21.375546+0000","last_fullsized":"2023-10-06T10:38:21.375546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:06.214524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.396278+0000","last_change":"2023-10-06T10:38:21.393210+0000","last_active":"2023-10-06T10:38:21.396278+0000","last_peered":"2023-10-06T10:38:21.396278+0000","last_clean":"2023-10-06T10:38:21.396278+0000","last_became_active":"2023-10-06T10:38:21.392827+0000","last_became_peered":"2023-10-06T10:38:21.392827+0000","last_unstale":"2023-10-06T10:38:21.396278+0000","last_undegraded":"2023-10-06T10:38:21.396278+0000","last_fullsized":"2023-10-06T10:38:21.396278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:05.756884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.392711+0000","last_change":"2023-10-06T10:38:16.363468+0000","last_active":"2023-10-06T10:38:21.392711+0000","last_peered":"2023-10-06T10:38:21.392711+0000","last_clean":"2023-10-06T10:38:21.392711+0000","last_became_active":"2023-10-06T10:38:16.363186+0000","last_became_peered":"2023-10-06T10:38:16.363186+0000","last_unstale":"2023-10-06T10:38:21.392711+0000","last_undegraded":"2023-10-06T10:38:21.392711+0000","last_fullsized":"2023-10-06T10:38:21.392711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:32.527614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377663+0000","last_change":"2023-10-06T10:38:17.365862+0000","last_active":"2023-10-06T10:38:21.377663+0000","last_peered":"2023-10-06T10:38:21.377663+0000","last_clean":"2023-10-06T10:38:21.377663+0000","last_became_active":"2023-10-06T10:38:17.365657+0000","last_became_peered":"2023-10-06T10:38:17.365657+0000","last_unstale":"2023-10-06T10:38:21.377663+0000","last_undegraded":"2023-10-06T10:38:21.377663+0000","last_fullsized":"2023-10-06T10:38:21.377663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:48.983240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.395385+0000","last_change":"2023-10-06T10:38:21.392166+0000","last_active":"2023-10-06T10:38:21.395385+0000","last_peered":"2023-10-06T10:38:21.395385+0000","last_clean":"2023-10-06T10:38:21.395385+0000","last_became_active":"2023-10-06T10:38:21.391756+0000","last_became_peered":"2023-10-06T10:38:21.391756+0000","last_unstale":"2023-10-06T10:38:21.395385+0000","last_undegraded":"2023-10-06T10:38:21.395385+0000","last_fullsized":"2023-10-06T10:38:21.395385+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:05.052271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.386206+0000","last_change":"2023-10-06T10:38:16.362369+0000","last_active":"2023-10-06T10:38:21.386206+0000","last_peered":"2023-10-06T10:38:21.386206+0000","last_clean":"2023-10-06T10:38:21.386206+0000","last_became_active":"2023-10-06T10:38:16.362076+0000","last_became_peered":"2023-10-06T10:38:16.362076+0000","last_unstale":"2023-10-06T10:38:21.386206+0000","last_undegraded":"2023-10-06T10:38:21.386206+0000","last_fullsized":"2023-10-06T10:38:21.386206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:27.473452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384735+0000","last_change":"2023-10-06T10:38:17.368641+0000","last_active":"2023-10-06T10:38:21.384735+0000","last_peered":"2023-10-06T10:38:21.384735+0000","last_clean":"2023-10-06T10:38:21.384735+0000","last_became_active":"2023-10-06T10:38:17.368283+0000","last_became_peered":"2023-10-06T10:38:17.368283+0000","last_unstale":"2023-10-06T10:38:21.384735+0000","last_undegraded":"2023-10-06T10:38:21.384735+0000","last_fullsized":"2023-10-06T10:38:21.384735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:17.351581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.395567+0000","last_change":"2023-10-06T10:38:21.392954+0000","last_active":"2023-10-06T10:38:21.395567+0000","last_peered":"2023-10-06T10:38:21.395567+0000","last_clean":"2023-10-06T10:38:21.395567+0000","last_became_active":"2023-10-06T10:38:21.392585+0000","last_became_peered":"2023-10-06T10:38:21.392585+0000","last_unstale":"2023-10-06T10:38:21.395567+0000","last_undegraded":"2023-10-06T10:38:21.395567+0000","last_fullsized":"2023-10-06T10:38:21.395567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:54.407728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375768+0000","last_change":"2023-10-06T10:38:16.357199+0000","last_active":"2023-10-06T10:38:21.375768+0000","last_peered":"2023-10-06T10:38:21.375768+0000","last_clean":"2023-10-06T10:38:21.375768+0000","last_became_active":"2023-10-06T10:38:16.356936+0000","last_became_peered":"2023-10-06T10:38:16.356936+0000","last_unstale":"2023-10-06T10:38:21.375768+0000","last_undegraded":"2023-10-06T10:38:21.375768+0000","last_fullsized":"2023-10-06T10:38:21.375768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:02.115831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377449+0000","last_change":"2023-10-06T10:38:17.359930+0000","last_active":"2023-10-06T10:38:21.377449+0000","last_peered":"2023-10-06T10:38:21.377449+0000","last_clean":"2023-10-06T10:38:21.377449+0000","last_became_active":"2023-10-06T10:38:17.359677+0000","last_became_peered":"2023-10-06T10:38:17.359677+0000","last_unstale":"2023-10-06T10:38:21.377449+0000","last_undegraded":"2023-10-06T10:38:21.377449+0000","last_fullsized":"2023-10-06T10:38:21.377449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:13.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.399357+0000","last_change":"2023-10-06T10:38:21.396020+0000","last_active":"2023-10-06T10:38:21.399357+0000","last_peered":"2023-10-06T10:38:21.399357+0000","last_clean":"2023-10-06T10:38:21.399357+0000","last_became_active":"2023-10-06T10:38:21.395659+0000","last_became_peered":"2023-10-06T10:38:21.395659+0000","last_unstale":"2023-10-06T10:38:21.399357+0000","last_undegraded":"2023-10-06T10:38:21.399357+0000","last_fullsized":"2023-10-06T10:38:21.399357+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:45:46.392517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372543+0000","last_change":"2023-10-06T10:38:16.364249+0000","last_active":"2023-10-06T10:38:21.372543+0000","last_peered":"2023-10-06T10:38:21.372543+0000","last_clean":"2023-10-06T10:38:21.372543+0000","last_became_active":"2023-10-06T10:38:16.363959+0000","last_became_peered":"2023-10-06T10:38:16.363959+0000","last_unstale":"2023-10-06T10:38:21.372543+0000","last_undegraded":"2023-10-06T10:38:21.372543+0000","last_fullsized":"2023-10-06T10:38:21.372543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:27.473452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377610+0000","last_change":"2023-10-06T10:38:17.362439+0000","last_active":"2023-10-06T10:38:21.377610+0000","last_peered":"2023-10-06T10:38:21.377610+0000","last_clean":"2023-10-06T10:38:21.377610+0000","last_became_active":"2023-10-06T10:38:17.361921+0000","last_became_peered":"2023-10-06T10:38:17.361921+0000","last_unstale":"2023-10-06T10:38:21.377610+0000","last_undegraded":"2023-10-06T10:38:21.377610+0000","last_fullsized":"2023-10-06T10:38:21.377610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:30:01.208220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402322+0000","last_change":"2023-10-06T10:38:21.399211+0000","last_active":"2023-10-06T10:38:21.402322+0000","last_peered":"2023-10-06T10:38:21.402322+0000","last_clean":"2023-10-06T10:38:21.402322+0000","last_became_active":"2023-10-06T10:38:21.396487+0000","last_became_peered":"2023-10-06T10:38:21.396487+0000","last_unstale":"2023-10-06T10:38:21.402322+0000","last_undegraded":"2023-10-06T10:38:21.402322+0000","last_fullsized":"2023-10-06T10:38:21.402322+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:26.378955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384822+0000","last_change":"2023-10-06T10:38:16.362941+0000","last_active":"2023-10-06T10:38:21.384822+0000","last_peered":"2023-10-06T10:38:21.384822+0000","last_clean":"2023-10-06T10:38:21.384822+0000","last_became_active":"2023-10-06T10:38:16.362651+0000","last_became_peered":"2023-10-06T10:38:16.362651+0000","last_unstale":"2023-10-06T10:38:21.384822+0000","last_undegraded":"2023-10-06T10:38:21.384822+0000","last_fullsized":"2023-10-06T10:38:21.384822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:41.837644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377389+0000","last_change":"2023-10-06T10:38:17.367573+0000","last_active":"2023-10-06T10:38:21.377389+0000","last_peered":"2023-10-06T10:38:21.377389+0000","last_clean":"2023-10-06T10:38:21.377389+0000","last_became_active":"2023-10-06T10:38:17.367363+0000","last_became_peered":"2023-10-06T10:38:17.367363+0000","last_unstale":"2023-10-06T10:38:21.377389+0000","last_undegraded":"2023-10-06T10:38:21.377389+0000","last_fullsized":"2023-10-06T10:38:21.377389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:03.650819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.401267+0000","last_change":"2023-10-06T10:38:21.394943+0000","last_active":"2023-10-06T10:38:21.401267+0000","last_peered":"2023-10-06T10:38:21.401267+0000","last_clean":"2023-10-06T10:38:21.401267+0000","last_became_active":"2023-10-06T10:38:21.394604+0000","last_became_peered":"2023-10-06T10:38:21.394604+0000","last_unstale":"2023-10-06T10:38:21.401267+0000","last_undegraded":"2023-10-06T10:38:21.401267+0000","last_fullsized":"2023-10-06T10:38:21.401267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:17:59.362798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377491+0000","last_change":"2023-10-06T10:38:16.358983+0000","last_active":"2023-10-06T10:38:21.377491+0000","last_peered":"2023-10-06T10:38:21.377491+0000","last_clean":"2023-10-06T10:38:21.377491+0000","last_became_active":"2023-10-06T10:38:16.358338+0000","last_became_peered":"2023-10-06T10:38:16.358338+0000","last_unstale":"2023-10-06T10:38:21.377491+0000","last_undegraded":"2023-10-06T10:38:21.377491+0000","last_fullsized":"2023-10-06T10:38:21.377491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:02.115831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378341+0000","last_change":"2023-10-06T10:38:17.375549+0000","last_active":"2023-10-06T10:38:21.378341+0000","last_peered":"2023-10-06T10:38:21.378341+0000","last_clean":"2023-10-06T10:38:21.378341+0000","last_became_active":"2023-10-06T10:38:17.374707+0000","last_became_peered":"2023-10-06T10:38:17.374707+0000","last_unstale":"2023-10-06T10:38:21.378341+0000","last_undegraded":"2023-10-06T10:38:21.378341+0000","last_fullsized":"2023-10-06T10:38:21.378341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:50.378097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391124+0000","last_change":"2023-10-06T10:38:21.388787+0000","last_active":"2023-10-06T10:38:21.391124+0000","last_peered":"2023-10-06T10:38:21.391124+0000","last_clean":"2023-10-06T10:38:21.391124+0000","last_became_active":"2023-10-06T10:38:21.388432+0000","last_became_peered":"2023-10-06T10:38:21.388432+0000","last_unstale":"2023-10-06T10:38:21.391124+0000","last_undegraded":"2023-10-06T10:38:21.391124+0000","last_fullsized":"2023-10-06T10:38:21.391124+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:03.131588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372464+0000","last_change":"2023-10-06T10:38:16.366048+0000","last_active":"2023-10-06T10:38:21.372464+0000","last_peered":"2023-10-06T10:38:21.372464+0000","last_clean":"2023-10-06T10:38:21.372464+0000","last_became_active":"2023-10-06T10:38:16.365814+0000","last_became_peered":"2023-10-06T10:38:16.365814+0000","last_unstale":"2023-10-06T10:38:21.372464+0000","last_undegraded":"2023-10-06T10:38:21.372464+0000","last_fullsized":"2023-10-06T10:38:21.372464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:36.469014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.379002+0000","last_change":"2023-10-06T10:38:17.358727+0000","last_active":"2023-10-06T10:38:21.379002+0000","last_peered":"2023-10-06T10:38:21.379002+0000","last_clean":"2023-10-06T10:38:21.379002+0000","last_became_active":"2023-10-06T10:38:17.358442+0000","last_became_peered":"2023-10-06T10:38:17.358442+0000","last_unstale":"2023-10-06T10:38:21.379002+0000","last_undegraded":"2023-10-06T10:38:21.379002+0000","last_fullsized":"2023-10-06T10:38:21.379002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:17.351581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.401082+0000","last_change":"2023-10-06T10:38:21.397353+0000","last_active":"2023-10-06T10:38:21.401082+0000","last_peered":"2023-10-06T10:38:21.401082+0000","last_clean":"2023-10-06T10:38:21.401082+0000","last_became_active":"2023-10-06T10:38:21.396788+0000","last_became_peered":"2023-10-06T10:38:21.396788+0000","last_unstale":"2023-10-06T10:38:21.401082+0000","last_undegraded":"2023-10-06T10:38:21.401082+0000","last_fullsized":"2023-10-06T10:38:21.401082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:50:06.948087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.373072+0000","last_change":"2023-10-06T10:38:16.362604+0000","last_active":"2023-10-06T10:38:21.373072+0000","last_peered":"2023-10-06T10:38:21.373072+0000","last_clean":"2023-10-06T10:38:21.373072+0000","last_became_active":"2023-10-06T10:38:16.362444+0000","last_became_peered":"2023-10-06T10:38:16.362444+0000","last_unstale":"2023-10-06T10:38:21.373072+0000","last_undegraded":"2023-10-06T10:38:21.373072+0000","last_fullsized":"2023-10-06T10:38:21.373072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:57.215481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377559+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:38:21.377559+0000","last_peered":"2023-10-06T10:38:21.377559+0000","last_clean":"2023-10-06T10:38:21.377559+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:38:21.377559+0000","last_undegraded":"2023-10-06T10:38:21.377559+0000","last_fullsized":"2023-10-06T10:38:21.377559+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372292+0000","last_change":"2023-10-06T10:38:17.374346+0000","last_active":"2023-10-06T10:38:21.372292+0000","last_peered":"2023-10-06T10:38:21.372292+0000","last_clean":"2023-10-06T10:38:21.372292+0000","last_became_active":"2023-10-06T10:38:17.374068+0000","last_became_peered":"2023-10-06T10:38:17.374068+0000","last_unstale":"2023-10-06T10:38:21.372292+0000","last_undegraded":"2023-10-06T10:38:21.372292+0000","last_fullsized":"2023-10-06T10:38:21.372292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:29.897092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.393470+0000","last_change":"2023-10-06T10:38:21.387372+0000","last_active":"2023-10-06T10:38:21.393470+0000","last_peered":"2023-10-06T10:38:21.393470+0000","last_clean":"2023-10-06T10:38:21.393470+0000","last_became_active":"2023-10-06T10:38:21.386907+0000","last_became_peered":"2023-10-06T10:38:21.386907+0000","last_unstale":"2023-10-06T10:38:21.393470+0000","last_undegraded":"2023-10-06T10:38:21.393470+0000","last_fullsized":"2023-10-06T10:38:21.393470+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:36.144881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372792+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:38:21.372792+0000","last_peered":"2023-10-06T10:38:21.372792+0000","last_clean":"2023-10-06T10:38:21.372792+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:38:21.372792+0000","last_undegraded":"2023-10-06T10:38:21.372792+0000","last_fullsized":"2023-10-06T10:38:21.372792+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.433239+0000","last_change":"2023-10-06T10:38:16.366116+0000","last_active":"2023-10-06T10:38:21.433239+0000","last_peered":"2023-10-06T10:38:21.433239+0000","last_clean":"2023-10-06T10:38:21.433239+0000","last_became_active":"2023-10-06T10:38:16.365875+0000","last_became_peered":"2023-10-06T10:38:16.365875+0000","last_unstale":"2023-10-06T10:38:21.433239+0000","last_undegraded":"2023-10-06T10:38:21.433239+0000","last_fullsized":"2023-10-06T10:38:21.433239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:35.127396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377577+0000","last_change":"2023-10-06T10:38:17.358654+0000","last_active":"2023-10-06T10:38:21.377577+0000","last_peered":"2023-10-06T10:38:21.377577+0000","last_clean":"2023-10-06T10:38:21.377577+0000","last_became_active":"2023-10-06T10:38:17.358361+0000","last_became_peered":"2023-10-06T10:38:17.358361+0000","last_unstale":"2023-10-06T10:38:21.377577+0000","last_undegraded":"2023-10-06T10:38:21.377577+0000","last_fullsized":"2023-10-06T10:38:21.377577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:06.214524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.397881+0000","last_change":"2023-10-06T10:38:21.392569+0000","last_active":"2023-10-06T10:38:21.397881+0000","last_peered":"2023-10-06T10:38:21.397881+0000","last_clean":"2023-10-06T10:38:21.397881+0000","last_became_active":"2023-10-06T10:38:21.392210+0000","last_became_peered":"2023-10-06T10:38:21.392210+0000","last_unstale":"2023-10-06T10:38:21.397881+0000","last_undegraded":"2023-10-06T10:38:21.397881+0000","last_fullsized":"2023-10-06T10:38:21.397881+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:14:10.094621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377969+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:38:21.377969+0000","last_peered":"2023-10-06T10:38:21.377969+0000","last_clean":"2023-10-06T10:38:21.377969+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:38:21.377969+0000","last_undegraded":"2023-10-06T10:38:21.377969+0000","last_fullsized":"2023-10-06T10:38:21.377969+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.390711+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:38:21.390711+0000","last_peered":"2023-10-06T10:38:21.390711+0000","last_clean":"2023-10-06T10:38:21.390711+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:38:21.390711+0000","last_undegraded":"2023-10-06T10:38:21.390711+0000","last_fullsized":"2023-10-06T10:38:21.390711+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375378+0000","last_change":"2023-10-06T10:38:16.362718+0000","last_active":"2023-10-06T10:38:21.375378+0000","last_peered":"2023-10-06T10:38:21.375378+0000","last_clean":"2023-10-06T10:38:21.375378+0000","last_became_active":"2023-10-06T10:38:16.361895+0000","last_became_peered":"2023-10-06T10:38:16.361895+0000","last_unstale":"2023-10-06T10:38:21.375378+0000","last_undegraded":"2023-10-06T10:38:21.375378+0000","last_fullsized":"2023-10-06T10:38:21.375378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:45.092301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.376156+0000","last_change":"2023-10-06T10:38:17.359496+0000","last_active":"2023-10-06T10:38:21.376156+0000","last_peered":"2023-10-06T10:38:21.376156+0000","last_clean":"2023-10-06T10:38:21.376156+0000","last_became_active":"2023-10-06T10:38:17.359011+0000","last_became_peered":"2023-10-06T10:38:17.359011+0000","last_unstale":"2023-10-06T10:38:21.376156+0000","last_undegraded":"2023-10-06T10:38:21.376156+0000","last_fullsized":"2023-10-06T10:38:21.376156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:06.214524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402221+0000","last_change":"2023-10-06T10:38:21.396996+0000","last_active":"2023-10-06T10:38:21.402221+0000","last_peered":"2023-10-06T10:38:21.402221+0000","last_clean":"2023-10-06T10:38:21.402221+0000","last_became_active":"2023-10-06T10:38:21.396694+0000","last_became_peered":"2023-10-06T10:38:21.396694+0000","last_unstale":"2023-10-06T10:38:21.402221+0000","last_undegraded":"2023-10-06T10:38:21.402221+0000","last_fullsized":"2023-10-06T10:38:21.402221+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:14.207234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.386749+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:38:21.386749+0000","last_peered":"2023-10-06T10:38:21.386749+0000","last_clean":"2023-10-06T10:38:21.386749+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:38:21.386749+0000","last_undegraded":"2023-10-06T10:38:21.386749+0000","last_fullsized":"2023-10-06T10:38:21.386749+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.376477+0000","last_change":"2023-10-06T10:38:16.357154+0000","last_active":"2023-10-06T10:38:21.376477+0000","last_peered":"2023-10-06T10:38:21.376477+0000","last_clean":"2023-10-06T10:38:21.376477+0000","last_became_active":"2023-10-06T10:38:16.356942+0000","last_became_peered":"2023-10-06T10:38:16.356942+0000","last_unstale":"2023-10-06T10:38:21.376477+0000","last_undegraded":"2023-10-06T10:38:21.376477+0000","last_fullsized":"2023-10-06T10:38:21.376477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:16.528477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378971+0000","last_change":"2023-10-06T10:38:17.358788+0000","last_active":"2023-10-06T10:38:21.378971+0000","last_peered":"2023-10-06T10:38:21.378971+0000","last_clean":"2023-10-06T10:38:21.378971+0000","last_became_active":"2023-10-06T10:38:17.358514+0000","last_became_peered":"2023-10-06T10:38:17.358514+0000","last_unstale":"2023-10-06T10:38:21.378971+0000","last_undegraded":"2023-10-06T10:38:21.378971+0000","last_fullsized":"2023-10-06T10:38:21.378971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:49.973815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402838+0000","last_change":"2023-10-06T10:38:21.401259+0000","last_active":"2023-10-06T10:38:21.402838+0000","last_peered":"2023-10-06T10:38:21.402838+0000","last_clean":"2023-10-06T10:38:21.402838+0000","last_became_active":"2023-10-06T10:38:21.400865+0000","last_became_peered":"2023-10-06T10:38:21.400865+0000","last_unstale":"2023-10-06T10:38:21.402838+0000","last_undegraded":"2023-10-06T10:38:21.402838+0000","last_fullsized":"2023-10-06T10:38:21.402838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:30.495506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384378+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:38:21.384378+0000","last_peered":"2023-10-06T10:38:21.384378+0000","last_clean":"2023-10-06T10:38:21.384378+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:38:21.384378+0000","last_undegraded":"2023-10-06T10:38:21.384378+0000","last_fullsized":"2023-10-06T10:38:21.384378+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372330+0000","last_change":"2023-10-06T10:38:16.358939+0000","last_active":"2023-10-06T10:38:21.372330+0000","last_peered":"2023-10-06T10:38:21.372330+0000","last_clean":"2023-10-06T10:38:21.372330+0000","last_became_active":"2023-10-06T10:38:16.358685+0000","last_became_peered":"2023-10-06T10:38:16.358685+0000","last_unstale":"2023-10-06T10:38:21.372330+0000","last_undegraded":"2023-10-06T10:38:21.372330+0000","last_fullsized":"2023-10-06T10:38:21.372330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:41.837644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385109+0000","last_change":"2023-10-06T10:38:17.366729+0000","last_active":"2023-10-06T10:38:21.385109+0000","last_peered":"2023-10-06T10:38:21.385109+0000","last_clean":"2023-10-06T10:38:21.385109+0000","last_became_active":"2023-10-06T10:38:17.366408+0000","last_became_peered":"2023-10-06T10:38:17.366408+0000","last_unstale":"2023-10-06T10:38:21.385109+0000","last_undegraded":"2023-10-06T10:38:21.385109+0000","last_fullsized":"2023-10-06T10:38:21.385109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:35.729655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.395664+0000","last_change":"2023-10-06T10:38:21.392874+0000","last_active":"2023-10-06T10:38:21.395664+0000","last_peered":"2023-10-06T10:38:21.395664+0000","last_clean":"2023-10-06T10:38:21.395664+0000","last_became_active":"2023-10-06T10:38:21.392487+0000","last_became_peered":"2023-10-06T10:38:21.392487+0000","last_unstale":"2023-10-06T10:38:21.395664+0000","last_undegraded":"2023-10-06T10:38:21.395664+0000","last_fullsized":"2023-10-06T10:38:21.395664+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:39.967976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375754+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:38:21.375754+0000","last_peered":"2023-10-06T10:38:21.375754+0000","last_clean":"2023-10-06T10:38:21.375754+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:38:21.375754+0000","last_undegraded":"2023-10-06T10:38:21.375754+0000","last_fullsized":"2023-10-06T10:38:21.375754+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377566+0000","last_change":"2023-10-06T10:38:16.362244+0000","last_active":"2023-10-06T10:38:21.377566+0000","last_peered":"2023-10-06T10:38:21.377566+0000","last_clean":"2023-10-06T10:38:21.377566+0000","last_became_active":"2023-10-06T10:38:16.361869+0000","last_became_peered":"2023-10-06T10:38:16.361869+0000","last_unstale":"2023-10-06T10:38:21.377566+0000","last_undegraded":"2023-10-06T10:38:21.377566+0000","last_fullsized":"2023-10-06T10:38:21.377566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:57.215481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.379113+0000","last_change":"2023-10-06T10:38:17.367649+0000","last_active":"2023-10-06T10:38:21.379113+0000","last_peered":"2023-10-06T10:38:21.379113+0000","last_clean":"2023-10-06T10:38:21.379113+0000","last_became_active":"2023-10-06T10:38:17.366773+0000","last_became_peered":"2023-10-06T10:38:17.366773+0000","last_unstale":"2023-10-06T10:38:21.379113+0000","last_undegraded":"2023-10-06T10:38:21.379113+0000","last_fullsized":"2023-10-06T10:38:21.379113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:40.679246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.401815+0000","last_change":"2023-10-06T10:38:21.393988+0000","last_active":"2023-10-06T10:38:21.401815+0000","last_peered":"2023-10-06T10:38:21.401815+0000","last_clean":"2023-10-06T10:38:21.401815+0000","last_became_active":"2023-10-06T10:38:21.393668+0000","last_became_peered":"2023-10-06T10:38:21.393668+0000","last_unstale":"2023-10-06T10:38:21.401815+0000","last_undegraded":"2023-10-06T10:38:21.401815+0000","last_fullsized":"2023-10-06T10:38:21.401815+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:33.926723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385218+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:38:21.385218+0000","last_peered":"2023-10-06T10:38:21.385218+0000","last_clean":"2023-10-06T10:38:21.385218+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:38:21.385218+0000","last_undegraded":"2023-10-06T10:38:21.385218+0000","last_fullsized":"2023-10-06T10:38:21.385218+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.379391+0000","last_change":"2023-10-06T10:38:16.360214+0000","last_active":"2023-10-06T10:38:21.379391+0000","last_peered":"2023-10-06T10:38:21.379391+0000","last_clean":"2023-10-06T10:38:21.379391+0000","last_became_active":"2023-10-06T10:38:16.359912+0000","last_became_peered":"2023-10-06T10:38:16.359912+0000","last_unstale":"2023-10-06T10:38:21.379391+0000","last_undegraded":"2023-10-06T10:38:21.379391+0000","last_fullsized":"2023-10-06T10:38:21.379391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:41.837644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377275+0000","last_change":"2023-10-06T10:38:17.354930+0000","last_active":"2023-10-06T10:38:21.377275+0000","last_peered":"2023-10-06T10:38:21.377275+0000","last_clean":"2023-10-06T10:38:21.377275+0000","last_became_active":"2023-10-06T10:38:17.354637+0000","last_became_peered":"2023-10-06T10:38:17.354637+0000","last_unstale":"2023-10-06T10:38:21.377275+0000","last_undegraded":"2023-10-06T10:38:21.377275+0000","last_fullsized":"2023-10-06T10:38:21.377275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:03.953719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391646+0000","last_change":"2023-10-06T10:38:21.387944+0000","last_active":"2023-10-06T10:38:21.391646+0000","last_peered":"2023-10-06T10:38:21.391646+0000","last_clean":"2023-10-06T10:38:21.391646+0000","last_became_active":"2023-10-06T10:38:21.387580+0000","last_became_peered":"2023-10-06T10:38:21.387580+0000","last_unstale":"2023-10-06T10:38:21.391646+0000","last_undegraded":"2023-10-06T10:38:21.391646+0000","last_fullsized":"2023-10-06T10:38:21.391646+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:54:20.934943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385079+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:38:21.385079+0000","last_peered":"2023-10-06T10:38:21.385079+0000","last_clean":"2023-10-06T10:38:21.385079+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:38:21.385079+0000","last_undegraded":"2023-10-06T10:38:21.385079+0000","last_fullsized":"2023-10-06T10:38:21.385079+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377995+0000","last_change":"2023-10-06T10:38:16.358519+0000","last_active":"2023-10-06T10:38:21.377995+0000","last_peered":"2023-10-06T10:38:21.377995+0000","last_clean":"2023-10-06T10:38:21.377995+0000","last_became_active":"2023-10-06T10:38:16.357730+0000","last_became_peered":"2023-10-06T10:38:16.357730+0000","last_unstale":"2023-10-06T10:38:21.377995+0000","last_undegraded":"2023-10-06T10:38:21.377995+0000","last_fullsized":"2023-10-06T10:38:21.377995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:36.469014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378421+0000","last_change":"2023-10-06T10:38:17.358314+0000","last_active":"2023-10-06T10:38:21.378421+0000","last_peered":"2023-10-06T10:38:21.378421+0000","last_clean":"2023-10-06T10:38:21.378421+0000","last_became_active":"2023-10-06T10:38:17.358060+0000","last_became_peered":"2023-10-06T10:38:17.358060+0000","last_unstale":"2023-10-06T10:38:21.378421+0000","last_undegraded":"2023-10-06T10:38:21.378421+0000","last_fullsized":"2023-10-06T10:38:21.378421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:34:27.066675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.400514+0000","last_change":"2023-10-06T10:38:21.391294+0000","last_active":"2023-10-06T10:38:21.400514+0000","last_peered":"2023-10-06T10:38:21.400514+0000","last_clean":"2023-10-06T10:38:21.400514+0000","last_became_active":"2023-10-06T10:38:21.390927+0000","last_became_peered":"2023-10-06T10:38:21.390927+0000","last_unstale":"2023-10-06T10:38:21.400514+0000","last_undegraded":"2023-10-06T10:38:21.400514+0000","last_fullsized":"2023-10-06T10:38:21.400514+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:53.012871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377366+0000","last_change":"2023-10-06T10:38:16.360511+0000","last_active":"2023-10-06T10:38:21.377366+0000","last_peered":"2023-10-06T10:38:21.377366+0000","last_clean":"2023-10-06T10:38:21.377366+0000","last_became_active":"2023-10-06T10:38:16.360260+0000","last_became_peered":"2023-10-06T10:38:16.360260+0000","last_unstale":"2023-10-06T10:38:21.377366+0000","last_undegraded":"2023-10-06T10:38:21.377366+0000","last_fullsized":"2023-10-06T10:38:21.377366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:57.215481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.383881+0000","last_change":"2023-10-06T10:38:17.365370+0000","last_active":"2023-10-06T10:38:21.383881+0000","last_peered":"2023-10-06T10:38:21.383881+0000","last_clean":"2023-10-06T10:38:21.383881+0000","last_became_active":"2023-10-06T10:38:17.365110+0000","last_became_peered":"2023-10-06T10:38:17.365110+0000","last_unstale":"2023-10-06T10:38:21.383881+0000","last_undegraded":"2023-10-06T10:38:21.383881+0000","last_fullsized":"2023-10-06T10:38:21.383881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:37.475956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402373+0000","last_change":"2023-10-06T10:38:21.399284+0000","last_active":"2023-10-06T10:38:21.402373+0000","last_peered":"2023-10-06T10:38:21.402373+0000","last_clean":"2023-10-06T10:38:21.402373+0000","last_became_active":"2023-10-06T10:38:21.398931+0000","last_became_peered":"2023-10-06T10:38:21.398931+0000","last_unstale":"2023-10-06T10:38:21.402373+0000","last_undegraded":"2023-10-06T10:38:21.402373+0000","last_fullsized":"2023-10-06T10:38:21.402373+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:17:39.406756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375583+0000","last_change":"2023-10-06T10:38:16.353975+0000","last_active":"2023-10-06T10:38:21.375583+0000","last_peered":"2023-10-06T10:38:21.375583+0000","last_clean":"2023-10-06T10:38:21.375583+0000","last_became_active":"2023-10-06T10:38:16.353629+0000","last_became_peered":"2023-10-06T10:38:16.353629+0000","last_unstale":"2023-10-06T10:38:21.375583+0000","last_undegraded":"2023-10-06T10:38:21.375583+0000","last_fullsized":"2023-10-06T10:38:21.375583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:20.696308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377446+0000","last_change":"2023-10-06T10:38:17.363838+0000","last_active":"2023-10-06T10:38:21.377446+0000","last_peered":"2023-10-06T10:38:21.377446+0000","last_clean":"2023-10-06T10:38:21.377446+0000","last_became_active":"2023-10-06T10:38:17.363580+0000","last_became_peered":"2023-10-06T10:38:17.363580+0000","last_unstale":"2023-10-06T10:38:21.377446+0000","last_undegraded":"2023-10-06T10:38:21.377446+0000","last_fullsized":"2023-10-06T10:38:21.377446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:43.614842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.399586+0000","last_change":"2023-10-06T10:38:21.396452+0000","last_active":"2023-10-06T10:38:21.399586+0000","last_peered":"2023-10-06T10:38:21.399586+0000","last_clean":"2023-10-06T10:38:21.399586+0000","last_became_active":"2023-10-06T10:38:21.396052+0000","last_became_peered":"2023-10-06T10:38:21.396052+0000","last_unstale":"2023-10-06T10:38:21.399586+0000","last_undegraded":"2023-10-06T10:38:21.399586+0000","last_fullsized":"2023-10-06T10:38:21.399586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:17:39.406756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377909+0000","last_change":"2023-10-06T10:38:16.354639+0000","last_active":"2023-10-06T10:38:21.377909+0000","last_peered":"2023-10-06T10:38:21.377909+0000","last_clean":"2023-10-06T10:38:21.377909+0000","last_became_active":"2023-10-06T10:38:16.354367+0000","last_became_peered":"2023-10-06T10:38:16.354367+0000","last_unstale":"2023-10-06T10:38:21.377909+0000","last_undegraded":"2023-10-06T10:38:21.377909+0000","last_fullsized":"2023-10-06T10:38:21.377909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:57.215481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377353+0000","last_change":"2023-10-06T10:38:17.358682+0000","last_active":"2023-10-06T10:38:21.377353+0000","last_peered":"2023-10-06T10:38:21.377353+0000","last_clean":"2023-10-06T10:38:21.377353+0000","last_became_active":"2023-10-06T10:38:17.358409+0000","last_became_peered":"2023-10-06T10:38:17.358409+0000","last_unstale":"2023-10-06T10:38:21.377353+0000","last_undegraded":"2023-10-06T10:38:21.377353+0000","last_fullsized":"2023-10-06T10:38:21.377353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:52:14.320333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391815+0000","last_change":"2023-10-06T10:38:21.390566+0000","last_active":"2023-10-06T10:38:21.391815+0000","last_peered":"2023-10-06T10:38:21.391815+0000","last_clean":"2023-10-06T10:38:21.391815+0000","last_became_active":"2023-10-06T10:38:21.390172+0000","last_became_peered":"2023-10-06T10:38:21.390172+0000","last_unstale":"2023-10-06T10:38:21.391815+0000","last_undegraded":"2023-10-06T10:38:21.391815+0000","last_fullsized":"2023-10-06T10:38:21.391815+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:37.174852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372471+0000","last_change":"2023-10-06T10:38:16.353396+0000","last_active":"2023-10-06T10:38:21.372471+0000","last_peered":"2023-10-06T10:38:21.372471+0000","last_clean":"2023-10-06T10:38:21.372471+0000","last_became_active":"2023-10-06T10:38:16.353136+0000","last_became_peered":"2023-10-06T10:38:16.353136+0000","last_unstale":"2023-10-06T10:38:21.372471+0000","last_undegraded":"2023-10-06T10:38:21.372471+0000","last_fullsized":"2023-10-06T10:38:21.372471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:02.115831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372424+0000","last_change":"2023-10-06T10:38:17.364589+0000","last_active":"2023-10-06T10:38:21.372424+0000","last_peered":"2023-10-06T10:38:21.372424+0000","last_clean":"2023-10-06T10:38:21.372424+0000","last_became_active":"2023-10-06T10:38:17.364364+0000","last_became_peered":"2023-10-06T10:38:17.364364+0000","last_unstale":"2023-10-06T10:38:21.372424+0000","last_undegraded":"2023-10-06T10:38:21.372424+0000","last_fullsized":"2023-10-06T10:38:21.372424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:13.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.399883+0000","last_change":"2023-10-06T10:38:21.395180+0000","last_active":"2023-10-06T10:38:21.399883+0000","last_peered":"2023-10-06T10:38:21.399883+0000","last_clean":"2023-10-06T10:38:21.399883+0000","last_became_active":"2023-10-06T10:38:21.394859+0000","last_became_peered":"2023-10-06T10:38:21.394859+0000","last_unstale":"2023-10-06T10:38:21.399883+0000","last_undegraded":"2023-10-06T10:38:21.399883+0000","last_fullsized":"2023-10-06T10:38:21.399883+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:54:20.934943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378521+0000","last_change":"2023-10-06T10:38:16.361291+0000","last_active":"2023-10-06T10:38:21.378521+0000","last_peered":"2023-10-06T10:38:21.378521+0000","last_clean":"2023-10-06T10:38:21.378521+0000","last_became_active":"2023-10-06T10:38:16.361027+0000","last_became_peered":"2023-10-06T10:38:16.361027+0000","last_unstale":"2023-10-06T10:38:21.378521+0000","last_undegraded":"2023-10-06T10:38:21.378521+0000","last_fullsized":"2023-10-06T10:38:21.378521+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:36.469014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378387+0000","last_change":"2023-10-06T10:38:17.375274+0000","last_active":"2023-10-06T10:38:21.378387+0000","last_peered":"2023-10-06T10:38:21.378387+0000","last_clean":"2023-10-06T10:38:21.378387+0000","last_became_active":"2023-10-06T10:38:17.374982+0000","last_became_peered":"2023-10-06T10:38:17.374982+0000","last_unstale":"2023-10-06T10:38:21.378387+0000","last_undegraded":"2023-10-06T10:38:21.378387+0000","last_fullsized":"2023-10-06T10:38:21.378387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:59.822053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385168+0000","last_change":"2023-10-06T10:38:17.368598+0000","last_active":"2023-10-06T10:38:21.385168+0000","last_peered":"2023-10-06T10:38:21.385168+0000","last_clean":"2023-10-06T10:38:21.385168+0000","last_became_active":"2023-10-06T10:38:17.367720+0000","last_became_peered":"2023-10-06T10:38:17.367720+0000","last_unstale":"2023-10-06T10:38:21.385168+0000","last_undegraded":"2023-10-06T10:38:21.385168+0000","last_fullsized":"2023-10-06T10:38:21.385168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:40.679246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372285+0000","last_change":"2023-10-06T10:38:17.362818+0000","last_active":"2023-10-06T10:38:21.372285+0000","last_peered":"2023-10-06T10:38:21.372285+0000","last_clean":"2023-10-06T10:38:21.372285+0000","last_became_active":"2023-10-06T10:38:17.362585+0000","last_became_peered":"2023-10-06T10:38:17.362585+0000","last_unstale":"2023-10-06T10:38:21.372285+0000","last_undegraded":"2023-10-06T10:38:21.372285+0000","last_fullsized":"2023-10-06T10:38:21.372285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:03.650819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378216+0000","last_change":"2023-10-06T10:38:17.375510+0000","last_active":"2023-10-06T10:38:21.378216+0000","last_peered":"2023-10-06T10:38:21.378216+0000","last_clean":"2023-10-06T10:38:21.378216+0000","last_became_active":"2023-10-06T10:38:17.375023+0000","last_became_peered":"2023-10-06T10:38:17.375023+0000","last_unstale":"2023-10-06T10:38:21.378216+0000","last_undegraded":"2023-10-06T10:38:21.378216+0000","last_fullsized":"2023-10-06T10:38:21.378216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:10.177603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372410+0000","last_change":"2023-10-06T10:38:17.370520+0000","last_active":"2023-10-06T10:38:21.372410+0000","last_peered":"2023-10-06T10:38:21.372410+0000","last_clean":"2023-10-06T10:38:21.372410+0000","last_became_active":"2023-10-06T10:38:17.370265+0000","last_became_peered":"2023-10-06T10:38:17.370265+0000","last_unstale":"2023-10-06T10:38:21.372410+0000","last_undegraded":"2023-10-06T10:38:21.372410+0000","last_fullsized":"2023-10-06T10:38:21.372410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:06.214524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.379183+0000","last_change":"2023-10-06T10:38:17.372067+0000","last_active":"2023-10-06T10:38:21.379183+0000","last_peered":"2023-10-06T10:38:21.379183+0000","last_clean":"2023-10-06T10:38:21.379183+0000","last_became_active":"2023-10-06T10:38:17.371813+0000","last_became_peered":"2023-10-06T10:38:17.371813+0000","last_unstale":"2023-10-06T10:38:21.379183+0000","last_undegraded":"2023-10-06T10:38:21.379183+0000","last_fullsized":"2023-10-06T10:38:21.379183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:23:23.463492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4484,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":876544,"data_stored":698736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":103085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298116,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073724,"statfs":{"total":96636764160,"available":96331493376,"internally_reserved":0,"allocated":847872,"data_stored":694806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":258048,"data_stored":104992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":104438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":258048,"data_stored":104528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297556,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074284,"statfs":{"total":96636764160,"available":96332066816,"internally_reserved":0,"allocated":274432,"data_stored":108208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":105062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:38:35.955 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:38:36.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.098+0000 7f4d7f59e700 1 Processor -- start 2023-10-06T10:38:36.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.098+0000 7f4d7f59e700 1 -- start start 2023-10-06T10:38:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.099+0000 7f4d7f59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4d80159490 0x7f4d800a9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.099+0000 7f4d7f59e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4d800af5c0 0x7f4d8006af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.099+0000 7f4d7f59e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d8006aaa0 con 0x7f4d8006b470 2023-10-06T10:38:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.099+0000 7f4d7f59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d800a9ff0 con 0x7f4d80159490 2023-10-06T10:38:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.099+0000 7f4d7f59e700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d800aa290 con 0x7f4d800af5c0 2023-10-06T10:38:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.099+0000 7f4d7e59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4d80159490 0x7f4d800a9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.099+0000 7f4d7e59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4d80159490 0x7f4d800a9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36878/0 (socket says 172.21.15.92:36878) 2023-10-06T10:38:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.099+0000 7f4d7e59c700 1 -- 172.21.15.92:0/3329285498 learned_addr learned my addr 172.21.15.92:0/3329285498 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.099+0000 7f4d75d9b700 1 --2- 172.21.15.92:0/3329285498 >> v2:172.21.15.124:3300/0 conn(0x7f4d800af5c0 0x7f4d8006af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.100+0000 7f4d7e59c700 1 -- 172.21.15.92:0/3329285498 >> v2:172.21.15.124:3300/0 conn(0x7f4d800af5c0 msgr2=0x7f4d8006af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.100+0000 7f4d7e59c700 1 --2- 172.21.15.92:0/3329285498 >> v2:172.21.15.124:3300/0 conn(0x7f4d800af5c0 0x7f4d8006af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.100+0000 7f4d7e59c700 1 -- 172.21.15.92:0/3329285498 >> v1:172.21.15.124:6789/0 conn(0x7f4d8006b470 legacy=0x7f4d800675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.100+0000 7f4d7e59c700 1 -- 172.21.15.92:0/3329285498 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d800aa570 con 0x7f4d80159490 2023-10-06T10:38:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.100+0000 7f4d75d9b700 1 --2- 172.21.15.92:0/3329285498 >> v2:172.21.15.124:3300/0 conn(0x7f4d800af5c0 0x7f4d8006af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.100+0000 7f4d7e59c700 1 --2- 172.21.15.92:0/3329285498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d80159490 0x7f4d800a9ab0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4d8004c3a0 tx=0x7f4d70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dffa16a537e4b293 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.100+0000 7f4d7dd9b700 1 -- 172.21.15.92:0/3329285498 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7000b820 con 0x7f4d80159490 2023-10-06T10:38:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.100+0000 7f4d7dd9b700 1 -- 172.21.15.92:0/3329285498 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7000b980 con 0x7f4d80159490 2023-10-06T10:38:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.100+0000 7f4d7dd9b700 1 -- 172.21.15.92:0/3329285498 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7000bd20 con 0x7f4d80159490 2023-10-06T10:38:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.100+0000 7f4d7f59e700 1 -- 172.21.15.92:0/3329285498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d80159490 msgr2=0x7f4d800a9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.100+0000 7f4d7f59e700 1 --2- 172.21.15.92:0/3329285498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d80159490 0x7f4d800a9ab0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4d8004c3a0 tx=0x7f4d70009250 comp rx=0 tx=0).stop 2023-10-06T10:38:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.100+0000 7f4d7f59e700 1 -- 172.21.15.92:0/3329285498 shutdown_connections 2023-10-06T10:38:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.100+0000 7f4d7f59e700 1 --2- 172.21.15.92:0/3329285498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d80159490 0x7f4d800a9ab0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.100+0000 7f4d7f59e700 1 --2- 172.21.15.92:0/3329285498 >> v2:172.21.15.124:3300/0 conn(0x7f4d800af5c0 0x7f4d8006af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.100+0000 7f4d7f59e700 1 -- 172.21.15.92:0/3329285498 >> 172.21.15.92:0/3329285498 conn(0x7f4d80069190 msgr2=0x7f4d800aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.100+0000 7f4d7f59e700 1 -- 172.21.15.92:0/3329285498 shutdown_connections 2023-10-06T10:38:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.100+0000 7f4d7f59e700 1 -- 172.21.15.92:0/3329285498 wait complete. 2023-10-06T10:38:36.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.102+0000 7f4d7f59e700 1 Processor -- start 2023-10-06T10:38:36.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.102+0000 7f4d7f59e700 1 -- start start 2023-10-06T10:38:36.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.102+0000 7f4d7f59e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d8006b470 0x7f4d8016cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:36.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.102+0000 7f4d7f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d800af5c0 0x7f4d8016a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:36.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.102+0000 7f4d7f59e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d80159490 0x7f4d8013dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:36.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.102+0000 7f4d7f59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8016a8a0 con 0x7f4d800af5c0 2023-10-06T10:38:36.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.102+0000 7f4d7f59e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d80152fa0 con 0x7f4d80159490 2023-10-06T10:38:36.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.102+0000 7f4d7f59e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8016d4c0 con 0x7f4d8006b470 2023-10-06T10:38:36.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.102+0000 7f4d75d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d800af5c0 0x7f4d8016a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.103+0000 7f4d75d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d800af5c0 0x7f4d8016a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36884/0 (socket says 172.21.15.92:36884) 2023-10-06T10:38:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.103+0000 7f4d75d9b700 1 -- 172.21.15.92:0/2279471903 learned_addr learned my addr 172.21.15.92:0/2279471903 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.103+0000 7f4d7ed9d700 1 --2- 172.21.15.92:0/2279471903 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d80159490 0x7f4d8013dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.103+0000 7f4d7e59c700 1 --2- 172.21.15.92:0/2279471903 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d8006b470 0x7f4d8016cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.103+0000 7f4d75d9b700 1 -- 172.21.15.92:0/2279471903 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d8006b470 msgr2=0x7f4d8016cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.103+0000 7f4d75d9b700 1 --2- 172.21.15.92:0/2279471903 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d8006b470 0x7f4d8016cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.103+0000 7f4d75d9b700 1 -- 172.21.15.92:0/2279471903 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d80159490 msgr2=0x7f4d8013dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.103+0000 7f4d75d9b700 1 --2- 172.21.15.92:0/2279471903 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d80159490 0x7f4d8013dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.103+0000 7f4d75d9b700 1 -- 172.21.15.92:0/2279471903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d700094f0 con 0x7f4d800af5c0 2023-10-06T10:38:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.103+0000 7f4d75d9b700 1 --2- 172.21.15.92:0/2279471903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d800af5c0 0x7f4d8016a360 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800c830 tx=0x7f4d68008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.103+0000 7f4d76ffd700 1 -- 172.21.15.92:0/2279471903 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d68011600 con 0x7f4d800af5c0 2023-10-06T10:38:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.103+0000 7f4d76ffd700 1 -- 172.21.15.92:0/2279471903 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d68011760 con 0x7f4d800af5c0 2023-10-06T10:38:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.103+0000 7f4d7f59e700 1 -- 172.21.15.92:0/2279471903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d8016d7a0 con 0x7f4d800af5c0 2023-10-06T10:38:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.103+0000 7f4d76ffd700 1 -- 172.21.15.92:0/2279471903 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d68011b20 con 0x7f4d800af5c0 2023-10-06T10:38:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.103+0000 7f4d7f59e700 1 -- 172.21.15.92:0/2279471903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d8016dd60 con 0x7f4d800af5c0 2023-10-06T10:38:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.104+0000 7f4d7f59e700 1 -- 172.21.15.92:0/2279471903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d8005f230 con 0x7f4d800af5c0 2023-10-06T10:38:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.104+0000 7f4d76ffd700 1 -- 172.21.15.92:0/2279471903 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4d68018360 con 0x7f4d800af5c0 2023-10-06T10:38:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.105+0000 7f4d76ffd700 1 --2- 172.21.15.92:0/2279471903 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4d60066f30 0x7f4d600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.105+0000 7f4d76ffd700 1 -- 172.21.15.92:0/2279471903 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4d68084910 con 0x7f4d800af5c0 2023-10-06T10:38:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.105+0000 7f4d7e59c700 1 --2- 172.21.15.92:0/2279471903 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4d60066f30 0x7f4d600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:36.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.105+0000 7f4d7e59c700 1 --2- 172.21.15.92:0/2279471903 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4d60066f30 0x7f4d600693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4d8004c3a0 tx=0x7f4d70016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:36.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.109+0000 7f4d76ffd700 1 -- 172.21.15.92:0/2279471903 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d68051730 con 0x7f4d800af5c0 2023-10-06T10:38:36.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.286+0000 7f4d7f59e700 1 -- 172.21.15.92:0/2279471903 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4d80071550 con 0x7f4d60066f30 2023-10-06T10:38:36.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.295+0000 7f4d76ffd700 1 -- 172.21.15.92:0/2279471903 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367593 (secure 0 0 0) 0x7f4d70006c50 con 0x7f4d60066f30 2023-10-06T10:38:36.301 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:36.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.299+0000 7f4d7f59e700 1 -- 172.21.15.92:0/2279471903 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4d60066f30 msgr2=0x7f4d600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:36.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.299+0000 7f4d7f59e700 1 --2- 172.21.15.92:0/2279471903 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4d60066f30 0x7f4d600693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4d8004c3a0 tx=0x7f4d70016000 comp rx=0 tx=0).stop 2023-10-06T10:38:36.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.299+0000 7f4d7f59e700 1 -- 172.21.15.92:0/2279471903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d800af5c0 msgr2=0x7f4d8016a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:36.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.299+0000 7f4d7f59e700 1 --2- 172.21.15.92:0/2279471903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d800af5c0 0x7f4d8016a360 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800c830 tx=0x7f4d68008040 comp rx=0 tx=0).stop 2023-10-06T10:38:36.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.299+0000 7f4d7f59e700 1 -- 172.21.15.92:0/2279471903 shutdown_connections 2023-10-06T10:38:36.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.299+0000 7f4d7f59e700 1 --2- 172.21.15.92:0/2279471903 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4d60066f30 0x7f4d600693e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.299+0000 7f4d7f59e700 1 --2- 172.21.15.92:0/2279471903 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d80159490 0x7f4d8013dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.299+0000 7f4d7f59e700 1 --2- 172.21.15.92:0/2279471903 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d8006b470 0x7f4d8016cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.299+0000 7f4d7f59e700 1 --2- 172.21.15.92:0/2279471903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d800af5c0 0x7f4d8016a360 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.299+0000 7f4d7f59e700 1 -- 172.21.15.92:0/2279471903 >> 172.21.15.92:0/2279471903 conn(0x7f4d80069190 msgr2=0x7f4d801586a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:36.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.299+0000 7f4d7f59e700 1 -- 172.21.15.92:0/2279471903 shutdown_connections 2023-10-06T10:38:36.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.299+0000 7f4d7f59e700 1 -- 172.21.15.92:0/2279471903 wait complete. 2023-10-06T10:38:36.306 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:38:36.314 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-10-06T10:38:36.085177+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594852,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381508,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593212,"statfs":{"total":773094113280,"available":770655449088,"internally_reserved":0,"allocated":3280896,"data_stored":2023855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2178,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003013"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384275+0000","last_change":"2023-10-06T10:38:17.365963+0000","last_active":"2023-10-06T10:38:21.384275+0000","last_peered":"2023-10-06T10:38:21.384275+0000","last_clean":"2023-10-06T10:38:21.384275+0000","last_became_active":"2023-10-06T10:38:17.365702+0000","last_became_peered":"2023-10-06T10:38:17.365702+0000","last_unstale":"2023-10-06T10:38:21.384275+0000","last_undegraded":"2023-10-06T10:38:21.384275+0000","last_fullsized":"2023-10-06T10:38:21.384275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:58.222423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378758+0000","last_change":"2023-10-06T10:38:17.362155+0000","last_active":"2023-10-06T10:38:21.378758+0000","last_peered":"2023-10-06T10:38:21.378758+0000","last_clean":"2023-10-06T10:38:21.378758+0000","last_became_active":"2023-10-06T10:38:17.361871+0000","last_became_peered":"2023-10-06T10:38:17.361871+0000","last_unstale":"2023-10-06T10:38:21.378758+0000","last_undegraded":"2023-10-06T10:38:21.378758+0000","last_fullsized":"2023-10-06T10:38:21.378758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:49.973815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377999+0000","last_change":"2023-10-06T10:38:17.361502+0000","last_active":"2023-10-06T10:38:21.377999+0000","last_peered":"2023-10-06T10:38:21.377999+0000","last_clean":"2023-10-06T10:38:21.377999+0000","last_became_active":"2023-10-06T10:38:17.360866+0000","last_became_peered":"2023-10-06T10:38:17.360866+0000","last_unstale":"2023-10-06T10:38:21.377999+0000","last_undegraded":"2023-10-06T10:38:21.377999+0000","last_fullsized":"2023-10-06T10:38:21.377999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:01.022639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384859+0000","last_change":"2023-10-06T10:38:17.366612+0000","last_active":"2023-10-06T10:38:21.384859+0000","last_peered":"2023-10-06T10:38:21.384859+0000","last_clean":"2023-10-06T10:38:21.384859+0000","last_became_active":"2023-10-06T10:38:17.366337+0000","last_became_peered":"2023-10-06T10:38:17.366337+0000","last_unstale":"2023-10-06T10:38:21.384859+0000","last_undegraded":"2023-10-06T10:38:21.384859+0000","last_fullsized":"2023-10-06T10:38:21.384859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:01:12.162890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377172+0000","last_change":"2023-10-06T10:38:17.363460+0000","last_active":"2023-10-06T10:38:21.377172+0000","last_peered":"2023-10-06T10:38:21.377172+0000","last_clean":"2023-10-06T10:38:21.377172+0000","last_became_active":"2023-10-06T10:38:17.363211+0000","last_became_peered":"2023-10-06T10:38:17.363211+0000","last_unstale":"2023-10-06T10:38:21.377172+0000","last_undegraded":"2023-10-06T10:38:21.377172+0000","last_fullsized":"2023-10-06T10:38:21.377172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.132681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378316+0000","last_change":"2023-10-06T10:38:17.364098+0000","last_active":"2023-10-06T10:38:21.378316+0000","last_peered":"2023-10-06T10:38:21.378316+0000","last_clean":"2023-10-06T10:38:21.378316+0000","last_became_active":"2023-10-06T10:38:17.363806+0000","last_became_peered":"2023-10-06T10:38:17.363806+0000","last_unstale":"2023-10-06T10:38:21.378316+0000","last_undegraded":"2023-10-06T10:38:21.378316+0000","last_fullsized":"2023-10-06T10:38:21.378316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:27.532414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391179+0000","last_change":"2023-10-06T10:38:17.372436+0000","last_active":"2023-10-06T10:38:21.391179+0000","last_peered":"2023-10-06T10:38:21.391179+0000","last_clean":"2023-10-06T10:38:21.391179+0000","last_became_active":"2023-10-06T10:38:17.371126+0000","last_became_peered":"2023-10-06T10:38:17.371126+0000","last_unstale":"2023-10-06T10:38:21.391179+0000","last_undegraded":"2023-10-06T10:38:21.391179+0000","last_fullsized":"2023-10-06T10:38:21.391179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:13.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375652+0000","last_change":"2023-10-06T10:38:17.354022+0000","last_active":"2023-10-06T10:38:21.375652+0000","last_peered":"2023-10-06T10:38:21.375652+0000","last_clean":"2023-10-06T10:38:21.375652+0000","last_became_active":"2023-10-06T10:38:17.353630+0000","last_became_peered":"2023-10-06T10:38:17.353630+0000","last_unstale":"2023-10-06T10:38:21.375652+0000","last_undegraded":"2023-10-06T10:38:21.375652+0000","last_fullsized":"2023-10-06T10:38:21.375652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:03.953719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378266+0000","last_change":"2023-10-06T10:38:17.362560+0000","last_active":"2023-10-06T10:38:21.378266+0000","last_peered":"2023-10-06T10:38:21.378266+0000","last_clean":"2023-10-06T10:38:21.378266+0000","last_became_active":"2023-10-06T10:38:17.361953+0000","last_became_peered":"2023-10-06T10:38:17.361953+0000","last_unstale":"2023-10-06T10:38:21.378266+0000","last_undegraded":"2023-10-06T10:38:21.378266+0000","last_fullsized":"2023-10-06T10:38:21.378266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:38.417248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378932+0000","last_change":"2023-10-06T10:38:17.364442+0000","last_active":"2023-10-06T10:38:21.378932+0000","last_peered":"2023-10-06T10:38:21.378932+0000","last_clean":"2023-10-06T10:38:21.378932+0000","last_became_active":"2023-10-06T10:38:17.363787+0000","last_became_peered":"2023-10-06T10:38:17.363787+0000","last_unstale":"2023-10-06T10:38:21.378932+0000","last_undegraded":"2023-10-06T10:38:21.378932+0000","last_fullsized":"2023-10-06T10:38:21.378932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:13.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377971+0000","last_change":"2023-10-06T10:38:17.366791+0000","last_active":"2023-10-06T10:38:21.377971+0000","last_peered":"2023-10-06T10:38:21.377971+0000","last_clean":"2023-10-06T10:38:21.377971+0000","last_became_active":"2023-10-06T10:38:17.366281+0000","last_became_peered":"2023-10-06T10:38:17.366281+0000","last_unstale":"2023-10-06T10:38:21.377971+0000","last_undegraded":"2023-10-06T10:38:21.377971+0000","last_fullsized":"2023-10-06T10:38:21.377971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:22.349323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.383809+0000","last_change":"2023-10-06T10:38:17.368900+0000","last_active":"2023-10-06T10:38:21.383809+0000","last_peered":"2023-10-06T10:38:21.383809+0000","last_clean":"2023-10-06T10:38:21.383809+0000","last_became_active":"2023-10-06T10:38:17.368488+0000","last_became_peered":"2023-10-06T10:38:17.368488+0000","last_unstale":"2023-10-06T10:38:21.383809+0000","last_undegraded":"2023-10-06T10:38:21.383809+0000","last_fullsized":"2023-10-06T10:38:21.383809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:07.365430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.383322+0000","last_change":"2023-10-06T10:38:17.365338+0000","last_active":"2023-10-06T10:38:21.383322+0000","last_peered":"2023-10-06T10:38:21.383322+0000","last_clean":"2023-10-06T10:38:21.383322+0000","last_became_active":"2023-10-06T10:38:17.365056+0000","last_became_peered":"2023-10-06T10:38:17.365056+0000","last_unstale":"2023-10-06T10:38:21.383322+0000","last_undegraded":"2023-10-06T10:38:21.383322+0000","last_fullsized":"2023-10-06T10:38:21.383322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:03.122773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375751+0000","last_change":"2023-10-06T10:38:17.366413+0000","last_active":"2023-10-06T10:38:21.375751+0000","last_peered":"2023-10-06T10:38:21.375751+0000","last_clean":"2023-10-06T10:38:21.375751+0000","last_became_active":"2023-10-06T10:38:17.366139+0000","last_became_peered":"2023-10-06T10:38:17.366139+0000","last_unstale":"2023-10-06T10:38:21.375751+0000","last_undegraded":"2023-10-06T10:38:21.375751+0000","last_fullsized":"2023-10-06T10:38:21.375751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:34:27.066675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391014+0000","last_change":"2023-10-06T10:38:17.371250+0000","last_active":"2023-10-06T10:38:21.391014+0000","last_peered":"2023-10-06T10:38:21.391014+0000","last_clean":"2023-10-06T10:38:21.391014+0000","last_became_active":"2023-10-06T10:38:17.370858+0000","last_became_peered":"2023-10-06T10:38:17.370858+0000","last_unstale":"2023-10-06T10:38:21.391014+0000","last_undegraded":"2023-10-06T10:38:21.391014+0000","last_fullsized":"2023-10-06T10:38:21.391014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:28.001417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378433+0000","last_change":"2023-10-06T10:38:17.364371+0000","last_active":"2023-10-06T10:38:21.378433+0000","last_peered":"2023-10-06T10:38:21.378433+0000","last_clean":"2023-10-06T10:38:21.378433+0000","last_became_active":"2023-10-06T10:38:17.364040+0000","last_became_peered":"2023-10-06T10:38:17.364040+0000","last_unstale":"2023-10-06T10:38:21.378433+0000","last_undegraded":"2023-10-06T10:38:21.378433+0000","last_fullsized":"2023-10-06T10:38:21.378433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:29.897092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378355+0000","last_change":"2023-10-06T10:38:17.363058+0000","last_active":"2023-10-06T10:38:21.378355+0000","last_peered":"2023-10-06T10:38:21.378355+0000","last_clean":"2023-10-06T10:38:21.378355+0000","last_became_active":"2023-10-06T10:38:17.359050+0000","last_became_peered":"2023-10-06T10:38:17.359050+0000","last_unstale":"2023-10-06T10:38:21.378355+0000","last_undegraded":"2023-10-06T10:38:21.378355+0000","last_fullsized":"2023-10-06T10:38:21.378355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:28.001417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377744+0000","last_change":"2023-10-06T10:38:17.365982+0000","last_active":"2023-10-06T10:38:21.377744+0000","last_peered":"2023-10-06T10:38:21.377744+0000","last_clean":"2023-10-06T10:38:21.377744+0000","last_became_active":"2023-10-06T10:38:17.364876+0000","last_became_peered":"2023-10-06T10:38:17.364876+0000","last_unstale":"2023-10-06T10:38:21.377744+0000","last_undegraded":"2023-10-06T10:38:21.377744+0000","last_fullsized":"2023-10-06T10:38:21.377744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:13.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.383516+0000","last_change":"2023-10-06T10:38:17.368880+0000","last_active":"2023-10-06T10:38:21.383516+0000","last_peered":"2023-10-06T10:38:21.383516+0000","last_clean":"2023-10-06T10:38:21.383516+0000","last_became_active":"2023-10-06T10:38:17.368645+0000","last_became_peered":"2023-10-06T10:38:17.368645+0000","last_unstale":"2023-10-06T10:38:21.383516+0000","last_undegraded":"2023-10-06T10:38:21.383516+0000","last_fullsized":"2023-10-06T10:38:21.383516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:42.844587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377767+0000","last_change":"2023-10-06T10:38:17.375213+0000","last_active":"2023-10-06T10:38:21.377767+0000","last_peered":"2023-10-06T10:38:21.377767+0000","last_clean":"2023-10-06T10:38:21.377767+0000","last_became_active":"2023-10-06T10:38:17.374836+0000","last_became_peered":"2023-10-06T10:38:17.374836+0000","last_unstale":"2023-10-06T10:38:21.377767+0000","last_undegraded":"2023-10-06T10:38:21.377767+0000","last_fullsized":"2023-10-06T10:38:21.377767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:01.727252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.386555+0000","last_change":"2023-10-06T10:38:17.370625+0000","last_active":"2023-10-06T10:38:21.386555+0000","last_peered":"2023-10-06T10:38:21.386555+0000","last_clean":"2023-10-06T10:38:21.386555+0000","last_became_active":"2023-10-06T10:38:17.370355+0000","last_became_peered":"2023-10-06T10:38:17.370355+0000","last_unstale":"2023-10-06T10:38:21.386555+0000","last_undegraded":"2023-10-06T10:38:21.386555+0000","last_fullsized":"2023-10-06T10:38:21.386555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:38.417248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375531+0000","last_change":"2023-10-06T10:38:17.360622+0000","last_active":"2023-10-06T10:38:21.375531+0000","last_peered":"2023-10-06T10:38:21.375531+0000","last_clean":"2023-10-06T10:38:21.375531+0000","last_became_active":"2023-10-06T10:38:17.359689+0000","last_became_peered":"2023-10-06T10:38:17.359689+0000","last_unstale":"2023-10-06T10:38:21.375531+0000","last_undegraded":"2023-10-06T10:38:21.375531+0000","last_fullsized":"2023-10-06T10:38:21.375531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:03.650819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372352+0000","last_change":"2023-10-06T10:38:17.362700+0000","last_active":"2023-10-06T10:38:21.372352+0000","last_peered":"2023-10-06T10:38:21.372352+0000","last_clean":"2023-10-06T10:38:21.372352+0000","last_became_active":"2023-10-06T10:38:17.362457+0000","last_became_peered":"2023-10-06T10:38:17.362457+0000","last_unstale":"2023-10-06T10:38:21.372352+0000","last_undegraded":"2023-10-06T10:38:21.372352+0000","last_fullsized":"2023-10-06T10:38:21.372352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:50.378097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.393253+0000","last_change":"2023-10-06T10:38:17.371996+0000","last_active":"2023-10-06T10:38:21.393253+0000","last_peered":"2023-10-06T10:38:21.393253+0000","last_clean":"2023-10-06T10:38:21.393253+0000","last_became_active":"2023-10-06T10:38:17.371731+0000","last_became_peered":"2023-10-06T10:38:17.371731+0000","last_unstale":"2023-10-06T10:38:21.393253+0000","last_undegraded":"2023-10-06T10:38:21.393253+0000","last_fullsized":"2023-10-06T10:38:21.393253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:06.214524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378923+0000","last_change":"2023-10-06T10:38:17.373079+0000","last_active":"2023-10-06T10:38:21.378923+0000","last_peered":"2023-10-06T10:38:21.378923+0000","last_clean":"2023-10-06T10:38:21.378923+0000","last_became_active":"2023-10-06T10:38:17.372799+0000","last_became_peered":"2023-10-06T10:38:17.372799+0000","last_unstale":"2023-10-06T10:38:21.378923+0000","last_undegraded":"2023-10-06T10:38:21.378923+0000","last_fullsized":"2023-10-06T10:38:21.378923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:03.953719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.379048+0000","last_change":"2023-10-06T10:38:17.366829+0000","last_active":"2023-10-06T10:38:21.379048+0000","last_peered":"2023-10-06T10:38:21.379048+0000","last_clean":"2023-10-06T10:38:21.379048+0000","last_became_active":"2023-10-06T10:38:17.366617+0000","last_became_peered":"2023-10-06T10:38:17.366617+0000","last_unstale":"2023-10-06T10:38:21.379048+0000","last_undegraded":"2023-10-06T10:38:21.379048+0000","last_fullsized":"2023-10-06T10:38:21.379048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:35.729655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378140+0000","last_change":"2023-10-06T10:38:17.371330+0000","last_active":"2023-10-06T10:38:21.378140+0000","last_peered":"2023-10-06T10:38:21.378140+0000","last_clean":"2023-10-06T10:38:21.378140+0000","last_became_active":"2023-10-06T10:38:17.370864+0000","last_became_peered":"2023-10-06T10:38:17.370864+0000","last_unstale":"2023-10-06T10:38:21.378140+0000","last_undegraded":"2023-10-06T10:38:21.378140+0000","last_fullsized":"2023-10-06T10:38:21.378140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:38.417248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391173+0000","last_change":"2023-10-06T10:38:21.389792+0000","last_active":"2023-10-06T10:38:21.391173+0000","last_peered":"2023-10-06T10:38:21.391173+0000","last_clean":"2023-10-06T10:38:21.391173+0000","last_became_active":"2023-10-06T10:38:21.389400+0000","last_became_peered":"2023-10-06T10:38:21.389400+0000","last_unstale":"2023-10-06T10:38:21.391173+0000","last_undegraded":"2023-10-06T10:38:21.391173+0000","last_fullsized":"2023-10-06T10:38:21.391173+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:55.438723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378102+0000","last_change":"2023-10-06T10:38:16.366893+0000","last_active":"2023-10-06T10:38:21.378102+0000","last_peered":"2023-10-06T10:38:21.378102+0000","last_clean":"2023-10-06T10:38:21.378102+0000","last_became_active":"2023-10-06T10:38:16.366637+0000","last_became_peered":"2023-10-06T10:38:16.366637+0000","last_unstale":"2023-10-06T10:38:21.378102+0000","last_undegraded":"2023-10-06T10:38:21.378102+0000","last_fullsized":"2023-10-06T10:38:21.378102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:06.358487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372687+0000","last_change":"2023-10-06T10:38:17.360671+0000","last_active":"2023-10-06T10:38:21.372687+0000","last_peered":"2023-10-06T10:38:21.372687+0000","last_clean":"2023-10-06T10:38:21.372687+0000","last_became_active":"2023-10-06T10:38:17.360439+0000","last_became_peered":"2023-10-06T10:38:17.360439+0000","last_unstale":"2023-10-06T10:38:21.372687+0000","last_undegraded":"2023-10-06T10:38:21.372687+0000","last_fullsized":"2023-10-06T10:38:21.372687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:45:42.362885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402648+0000","last_change":"2023-10-06T10:38:21.399346+0000","last_active":"2023-10-06T10:38:21.402648+0000","last_peered":"2023-10-06T10:38:21.402648+0000","last_clean":"2023-10-06T10:38:21.402648+0000","last_became_active":"2023-10-06T10:38:21.398877+0000","last_became_peered":"2023-10-06T10:38:21.398877+0000","last_unstale":"2023-10-06T10:38:21.402648+0000","last_undegraded":"2023-10-06T10:38:21.402648+0000","last_fullsized":"2023-10-06T10:38:21.402648+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:05.052271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.376684+0000","last_change":"2023-10-06T10:38:16.357015+0000","last_active":"2023-10-06T10:38:21.376684+0000","last_peered":"2023-10-06T10:38:21.376684+0000","last_clean":"2023-10-06T10:38:21.376684+0000","last_became_active":"2023-10-06T10:38:16.356733+0000","last_became_peered":"2023-10-06T10:38:16.356733+0000","last_unstale":"2023-10-06T10:38:21.376684+0000","last_undegraded":"2023-10-06T10:38:21.376684+0000","last_fullsized":"2023-10-06T10:38:21.376684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:06.358487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375991+0000","last_change":"2023-10-06T10:38:17.370483+0000","last_active":"2023-10-06T10:38:21.375991+0000","last_peered":"2023-10-06T10:38:21.375991+0000","last_clean":"2023-10-06T10:38:21.375991+0000","last_became_active":"2023-10-06T10:38:17.370196+0000","last_became_peered":"2023-10-06T10:38:17.370196+0000","last_unstale":"2023-10-06T10:38:21.375991+0000","last_undegraded":"2023-10-06T10:38:21.375991+0000","last_fullsized":"2023-10-06T10:38:21.375991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:49.973815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.398933+0000","last_change":"2023-10-06T10:38:21.397171+0000","last_active":"2023-10-06T10:38:21.398933+0000","last_peered":"2023-10-06T10:38:21.398933+0000","last_clean":"2023-10-06T10:38:21.398933+0000","last_became_active":"2023-10-06T10:38:21.396794+0000","last_became_peered":"2023-10-06T10:38:21.396794+0000","last_unstale":"2023-10-06T10:38:21.398933+0000","last_undegraded":"2023-10-06T10:38:21.398933+0000","last_fullsized":"2023-10-06T10:38:21.398933+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:30.495506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372622+0000","last_change":"2023-10-06T10:38:16.362088+0000","last_active":"2023-10-06T10:38:21.372622+0000","last_peered":"2023-10-06T10:38:21.372622+0000","last_clean":"2023-10-06T10:38:21.372622+0000","last_became_active":"2023-10-06T10:38:16.361790+0000","last_became_peered":"2023-10-06T10:38:16.361790+0000","last_unstale":"2023-10-06T10:38:21.372622+0000","last_undegraded":"2023-10-06T10:38:21.372622+0000","last_fullsized":"2023-10-06T10:38:21.372622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:20.696308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385166+0000","last_change":"2023-10-06T10:38:17.372313+0000","last_active":"2023-10-06T10:38:21.385166+0000","last_peered":"2023-10-06T10:38:21.385166+0000","last_clean":"2023-10-06T10:38:21.385166+0000","last_became_active":"2023-10-06T10:38:17.369061+0000","last_became_peered":"2023-10-06T10:38:17.369061+0000","last_unstale":"2023-10-06T10:38:21.385166+0000","last_undegraded":"2023-10-06T10:38:21.385166+0000","last_fullsized":"2023-10-06T10:38:21.385166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:25.807302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402371+0000","last_change":"2023-10-06T10:38:21.399352+0000","last_active":"2023-10-06T10:38:21.402371+0000","last_peered":"2023-10-06T10:38:21.402371+0000","last_clean":"2023-10-06T10:38:21.402371+0000","last_became_active":"2023-10-06T10:38:21.398679+0000","last_became_peered":"2023-10-06T10:38:21.398679+0000","last_unstale":"2023-10-06T10:38:21.402371+0000","last_undegraded":"2023-10-06T10:38:21.402371+0000","last_fullsized":"2023-10-06T10:38:21.402371+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:14.207234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378436+0000","last_change":"2023-10-06T10:38:16.361297+0000","last_active":"2023-10-06T10:38:21.378436+0000","last_peered":"2023-10-06T10:38:21.378436+0000","last_clean":"2023-10-06T10:38:21.378436+0000","last_became_active":"2023-10-06T10:38:16.361034+0000","last_became_peered":"2023-10-06T10:38:16.361034+0000","last_unstale":"2023-10-06T10:38:21.378436+0000","last_undegraded":"2023-10-06T10:38:21.378436+0000","last_fullsized":"2023-10-06T10:38:21.378436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:02.115831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372551+0000","last_change":"2023-10-06T10:38:17.361626+0000","last_active":"2023-10-06T10:38:21.372551+0000","last_peered":"2023-10-06T10:38:21.372551+0000","last_clean":"2023-10-06T10:38:21.372551+0000","last_became_active":"2023-10-06T10:38:17.361386+0000","last_became_peered":"2023-10-06T10:38:17.361386+0000","last_unstale":"2023-10-06T10:38:21.372551+0000","last_undegraded":"2023-10-06T10:38:21.372551+0000","last_fullsized":"2023-10-06T10:38:21.372551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.132681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.397814+0000","last_change":"2023-10-06T10:38:21.395253+0000","last_active":"2023-10-06T10:38:21.397814+0000","last_peered":"2023-10-06T10:38:21.397814+0000","last_clean":"2023-10-06T10:38:21.397814+0000","last_became_active":"2023-10-06T10:38:21.394839+0000","last_became_peered":"2023-10-06T10:38:21.394839+0000","last_unstale":"2023-10-06T10:38:21.397814+0000","last_undegraded":"2023-10-06T10:38:21.397814+0000","last_fullsized":"2023-10-06T10:38:21.397814+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:43.578698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378846+0000","last_change":"2023-10-06T10:38:16.359731+0000","last_active":"2023-10-06T10:38:21.378846+0000","last_peered":"2023-10-06T10:38:21.378846+0000","last_clean":"2023-10-06T10:38:21.378846+0000","last_became_active":"2023-10-06T10:38:16.359451+0000","last_became_peered":"2023-10-06T10:38:16.359451+0000","last_unstale":"2023-10-06T10:38:21.378846+0000","last_undegraded":"2023-10-06T10:38:21.378846+0000","last_fullsized":"2023-10-06T10:38:21.378846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:06.358487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372246+0000","last_change":"2023-10-06T10:38:17.371057+0000","last_active":"2023-10-06T10:38:21.372246+0000","last_peered":"2023-10-06T10:38:21.372246+0000","last_clean":"2023-10-06T10:38:21.372246+0000","last_became_active":"2023-10-06T10:38:17.370898+0000","last_became_peered":"2023-10-06T10:38:17.370898+0000","last_unstale":"2023-10-06T10:38:21.372246+0000","last_undegraded":"2023-10-06T10:38:21.372246+0000","last_fullsized":"2023-10-06T10:38:21.372246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:59.822053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.393290+0000","last_change":"2023-10-06T10:38:21.387382+0000","last_active":"2023-10-06T10:38:21.393290+0000","last_peered":"2023-10-06T10:38:21.393290+0000","last_clean":"2023-10-06T10:38:21.393290+0000","last_became_active":"2023-10-06T10:38:21.386886+0000","last_became_peered":"2023-10-06T10:38:21.386886+0000","last_unstale":"2023-10-06T10:38:21.393290+0000","last_undegraded":"2023-10-06T10:38:21.393290+0000","last_fullsized":"2023-10-06T10:38:21.393290+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:04:14.062486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.392484+0000","last_change":"2023-10-06T10:38:16.362467+0000","last_active":"2023-10-06T10:38:21.392484+0000","last_peered":"2023-10-06T10:38:21.392484+0000","last_clean":"2023-10-06T10:38:21.392484+0000","last_became_active":"2023-10-06T10:38:16.362094+0000","last_became_peered":"2023-10-06T10:38:16.362094+0000","last_unstale":"2023-10-06T10:38:21.392484+0000","last_undegraded":"2023-10-06T10:38:21.392484+0000","last_fullsized":"2023-10-06T10:38:21.392484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:24.800360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378505+0000","last_change":"2023-10-06T10:38:17.371049+0000","last_active":"2023-10-06T10:38:21.378505+0000","last_peered":"2023-10-06T10:38:21.378505+0000","last_clean":"2023-10-06T10:38:21.378505+0000","last_became_active":"2023-10-06T10:38:17.370698+0000","last_became_peered":"2023-10-06T10:38:17.370698+0000","last_unstale":"2023-10-06T10:38:21.378505+0000","last_undegraded":"2023-10-06T10:38:21.378505+0000","last_fullsized":"2023-10-06T10:38:21.378505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:28.001417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.400543+0000","last_change":"2023-10-06T10:38:21.395461+0000","last_active":"2023-10-06T10:38:21.400543+0000","last_peered":"2023-10-06T10:38:21.400543+0000","last_clean":"2023-10-06T10:38:21.400543+0000","last_became_active":"2023-10-06T10:38:21.395077+0000","last_became_peered":"2023-10-06T10:38:21.395077+0000","last_unstale":"2023-10-06T10:38:21.400543+0000","last_undegraded":"2023-10-06T10:38:21.400543+0000","last_fullsized":"2023-10-06T10:38:21.400543+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:43.578698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'9","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T10:38:27.096420+0000","last_change":"2023-10-06T10:38:16.359809+0000","last_active":"2023-10-06T10:38:27.096420+0000","last_peered":"2023-10-06T10:38:27.096420+0000","last_clean":"2023-10-06T10:38:27.096420+0000","last_became_active":"2023-10-06T10:38:16.359520+0000","last_became_peered":"2023-10-06T10:38:16.359520+0000","last_unstale":"2023-10-06T10:38:27.096420+0000","last_undegraded":"2023-10-06T10:38:27.096420+0000","last_fullsized":"2023-10-06T10:38:27.096420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:41.837644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3714,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377665+0000","last_change":"2023-10-06T10:38:17.375788+0000","last_active":"2023-10-06T10:38:21.377665+0000","last_peered":"2023-10-06T10:38:21.377665+0000","last_clean":"2023-10-06T10:38:21.377665+0000","last_became_active":"2023-10-06T10:38:17.375416+0000","last_became_peered":"2023-10-06T10:38:17.375416+0000","last_unstale":"2023-10-06T10:38:21.377665+0000","last_undegraded":"2023-10-06T10:38:21.377665+0000","last_fullsized":"2023-10-06T10:38:21.377665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.132681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.396799+0000","last_change":"2023-10-06T10:38:21.391788+0000","last_active":"2023-10-06T10:38:21.396799+0000","last_peered":"2023-10-06T10:38:21.396799+0000","last_clean":"2023-10-06T10:38:21.396799+0000","last_became_active":"2023-10-06T10:38:21.391376+0000","last_became_peered":"2023-10-06T10:38:21.391376+0000","last_unstale":"2023-10-06T10:38:21.396799+0000","last_undegraded":"2023-10-06T10:38:21.396799+0000","last_fullsized":"2023-10-06T10:38:21.396799+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:25:40.423524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384475+0000","last_change":"2023-10-06T10:38:16.363389+0000","last_active":"2023-10-06T10:38:21.384475+0000","last_peered":"2023-10-06T10:38:21.384475+0000","last_clean":"2023-10-06T10:38:21.384475+0000","last_became_active":"2023-10-06T10:38:16.363141+0000","last_became_peered":"2023-10-06T10:38:16.363141+0000","last_unstale":"2023-10-06T10:38:21.384475+0000","last_undegraded":"2023-10-06T10:38:21.384475+0000","last_fullsized":"2023-10-06T10:38:21.384475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:01:11.155948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372587+0000","last_change":"2023-10-06T10:38:17.361247+0000","last_active":"2023-10-06T10:38:21.372587+0000","last_peered":"2023-10-06T10:38:21.372587+0000","last_clean":"2023-10-06T10:38:21.372587+0000","last_became_active":"2023-10-06T10:38:17.361016+0000","last_became_peered":"2023-10-06T10:38:17.361016+0000","last_unstale":"2023-10-06T10:38:21.372587+0000","last_undegraded":"2023-10-06T10:38:21.372587+0000","last_fullsized":"2023-10-06T10:38:21.372587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:38.417248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.398908+0000","last_change":"2023-10-06T10:38:21.395222+0000","last_active":"2023-10-06T10:38:21.398908+0000","last_peered":"2023-10-06T10:38:21.398908+0000","last_clean":"2023-10-06T10:38:21.398908+0000","last_became_active":"2023-10-06T10:38:21.394695+0000","last_became_peered":"2023-10-06T10:38:21.394695+0000","last_unstale":"2023-10-06T10:38:21.398908+0000","last_undegraded":"2023-10-06T10:38:21.398908+0000","last_fullsized":"2023-10-06T10:38:21.398908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:36:21.761225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375670+0000","last_change":"2023-10-06T10:38:16.359510+0000","last_active":"2023-10-06T10:38:21.375670+0000","last_peered":"2023-10-06T10:38:21.375670+0000","last_clean":"2023-10-06T10:38:21.375670+0000","last_became_active":"2023-10-06T10:38:16.359236+0000","last_became_peered":"2023-10-06T10:38:16.359236+0000","last_unstale":"2023-10-06T10:38:21.375670+0000","last_undegraded":"2023-10-06T10:38:21.375670+0000","last_fullsized":"2023-10-06T10:38:21.375670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:16.528477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378073+0000","last_change":"2023-10-06T10:38:17.371135+0000","last_active":"2023-10-06T10:38:21.378073+0000","last_peered":"2023-10-06T10:38:21.378073+0000","last_clean":"2023-10-06T10:38:21.378073+0000","last_became_active":"2023-10-06T10:38:17.370494+0000","last_became_peered":"2023-10-06T10:38:17.370494+0000","last_unstale":"2023-10-06T10:38:21.378073+0000","last_undegraded":"2023-10-06T10:38:21.378073+0000","last_fullsized":"2023-10-06T10:38:21.378073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:28.001417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.398908+0000","last_change":"2023-10-06T10:38:21.395871+0000","last_active":"2023-10-06T10:38:21.398908+0000","last_peered":"2023-10-06T10:38:21.398908+0000","last_clean":"2023-10-06T10:38:21.398908+0000","last_became_active":"2023-10-06T10:38:21.395558+0000","last_became_peered":"2023-10-06T10:38:21.395558+0000","last_unstale":"2023-10-06T10:38:21.398908+0000","last_undegraded":"2023-10-06T10:38:21.398908+0000","last_fullsized":"2023-10-06T10:38:21.398908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:32:37.103555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391095+0000","last_change":"2023-10-06T10:38:16.363354+0000","last_active":"2023-10-06T10:38:21.391095+0000","last_peered":"2023-10-06T10:38:21.391095+0000","last_clean":"2023-10-06T10:38:21.391095+0000","last_became_active":"2023-10-06T10:38:16.363088+0000","last_became_peered":"2023-10-06T10:38:16.363088+0000","last_unstale":"2023-10-06T10:38:21.391095+0000","last_undegraded":"2023-10-06T10:38:21.391095+0000","last_fullsized":"2023-10-06T10:38:21.391095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:39.672304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.376233+0000","last_change":"2023-10-06T10:38:17.359300+0000","last_active":"2023-10-06T10:38:21.376233+0000","last_peered":"2023-10-06T10:38:21.376233+0000","last_clean":"2023-10-06T10:38:21.376233+0000","last_became_active":"2023-10-06T10:38:17.358652+0000","last_became_peered":"2023-10-06T10:38:17.358652+0000","last_unstale":"2023-10-06T10:38:21.376233+0000","last_undegraded":"2023-10-06T10:38:21.376233+0000","last_fullsized":"2023-10-06T10:38:21.376233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:45:42.362885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.400005+0000","last_change":"2023-10-06T10:38:21.394129+0000","last_active":"2023-10-06T10:38:21.400005+0000","last_peered":"2023-10-06T10:38:21.400005+0000","last_clean":"2023-10-06T10:38:21.400005+0000","last_became_active":"2023-10-06T10:38:21.393517+0000","last_became_peered":"2023-10-06T10:38:21.393517+0000","last_unstale":"2023-10-06T10:38:21.400005+0000","last_undegraded":"2023-10-06T10:38:21.400005+0000","last_fullsized":"2023-10-06T10:38:21.400005+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:55:52.407995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385053+0000","last_change":"2023-10-06T10:38:16.364742+0000","last_active":"2023-10-06T10:38:21.385053+0000","last_peered":"2023-10-06T10:38:21.385053+0000","last_clean":"2023-10-06T10:38:21.385053+0000","last_became_active":"2023-10-06T10:38:16.364410+0000","last_became_peered":"2023-10-06T10:38:16.364410+0000","last_unstale":"2023-10-06T10:38:21.385053+0000","last_undegraded":"2023-10-06T10:38:21.385053+0000","last_fullsized":"2023-10-06T10:38:21.385053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:06.358487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.373267+0000","last_change":"2023-10-06T10:38:17.374643+0000","last_active":"2023-10-06T10:38:21.373267+0000","last_peered":"2023-10-06T10:38:21.373267+0000","last_clean":"2023-10-06T10:38:21.373267+0000","last_became_active":"2023-10-06T10:38:17.374436+0000","last_became_peered":"2023-10-06T10:38:17.374436+0000","last_unstale":"2023-10-06T10:38:21.373267+0000","last_undegraded":"2023-10-06T10:38:21.373267+0000","last_fullsized":"2023-10-06T10:38:21.373267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:01.022639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402862+0000","last_change":"2023-10-06T10:38:21.401121+0000","last_active":"2023-10-06T10:38:21.402862+0000","last_peered":"2023-10-06T10:38:21.402862+0000","last_clean":"2023-10-06T10:38:21.402862+0000","last_became_active":"2023-10-06T10:38:21.400766+0000","last_became_peered":"2023-10-06T10:38:21.400766+0000","last_unstale":"2023-10-06T10:38:21.402862+0000","last_undegraded":"2023-10-06T10:38:21.402862+0000","last_fullsized":"2023-10-06T10:38:21.402862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:39.967976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.392673+0000","last_change":"2023-10-06T10:38:16.363994+0000","last_active":"2023-10-06T10:38:21.392673+0000","last_peered":"2023-10-06T10:38:21.392673+0000","last_clean":"2023-10-06T10:38:21.392673+0000","last_became_active":"2023-10-06T10:38:16.363712+0000","last_became_peered":"2023-10-06T10:38:16.363712+0000","last_unstale":"2023-10-06T10:38:21.392673+0000","last_undegraded":"2023-10-06T10:38:21.392673+0000","last_fullsized":"2023-10-06T10:38:21.392673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:43.249942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375546+0000","last_change":"2023-10-06T10:38:17.374214+0000","last_active":"2023-10-06T10:38:21.375546+0000","last_peered":"2023-10-06T10:38:21.375546+0000","last_clean":"2023-10-06T10:38:21.375546+0000","last_became_active":"2023-10-06T10:38:17.373998+0000","last_became_peered":"2023-10-06T10:38:17.373998+0000","last_unstale":"2023-10-06T10:38:21.375546+0000","last_undegraded":"2023-10-06T10:38:21.375546+0000","last_fullsized":"2023-10-06T10:38:21.375546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:06.214524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.396278+0000","last_change":"2023-10-06T10:38:21.393210+0000","last_active":"2023-10-06T10:38:21.396278+0000","last_peered":"2023-10-06T10:38:21.396278+0000","last_clean":"2023-10-06T10:38:21.396278+0000","last_became_active":"2023-10-06T10:38:21.392827+0000","last_became_peered":"2023-10-06T10:38:21.392827+0000","last_unstale":"2023-10-06T10:38:21.396278+0000","last_undegraded":"2023-10-06T10:38:21.396278+0000","last_fullsized":"2023-10-06T10:38:21.396278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:05.756884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.392711+0000","last_change":"2023-10-06T10:38:16.363468+0000","last_active":"2023-10-06T10:38:21.392711+0000","last_peered":"2023-10-06T10:38:21.392711+0000","last_clean":"2023-10-06T10:38:21.392711+0000","last_became_active":"2023-10-06T10:38:16.363186+0000","last_became_peered":"2023-10-06T10:38:16.363186+0000","last_unstale":"2023-10-06T10:38:21.392711+0000","last_undegraded":"2023-10-06T10:38:21.392711+0000","last_fullsized":"2023-10-06T10:38:21.392711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:32.527614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377663+0000","last_change":"2023-10-06T10:38:17.365862+0000","last_active":"2023-10-06T10:38:21.377663+0000","last_peered":"2023-10-06T10:38:21.377663+0000","last_clean":"2023-10-06T10:38:21.377663+0000","last_became_active":"2023-10-06T10:38:17.365657+0000","last_became_peered":"2023-10-06T10:38:17.365657+0000","last_unstale":"2023-10-06T10:38:21.377663+0000","last_undegraded":"2023-10-06T10:38:21.377663+0000","last_fullsized":"2023-10-06T10:38:21.377663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:48.983240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.395385+0000","last_change":"2023-10-06T10:38:21.392166+0000","last_active":"2023-10-06T10:38:21.395385+0000","last_peered":"2023-10-06T10:38:21.395385+0000","last_clean":"2023-10-06T10:38:21.395385+0000","last_became_active":"2023-10-06T10:38:21.391756+0000","last_became_peered":"2023-10-06T10:38:21.391756+0000","last_unstale":"2023-10-06T10:38:21.395385+0000","last_undegraded":"2023-10-06T10:38:21.395385+0000","last_fullsized":"2023-10-06T10:38:21.395385+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:05.052271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.386206+0000","last_change":"2023-10-06T10:38:16.362369+0000","last_active":"2023-10-06T10:38:21.386206+0000","last_peered":"2023-10-06T10:38:21.386206+0000","last_clean":"2023-10-06T10:38:21.386206+0000","last_became_active":"2023-10-06T10:38:16.362076+0000","last_became_peered":"2023-10-06T10:38:16.362076+0000","last_unstale":"2023-10-06T10:38:21.386206+0000","last_undegraded":"2023-10-06T10:38:21.386206+0000","last_fullsized":"2023-10-06T10:38:21.386206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:27.473452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384735+0000","last_change":"2023-10-06T10:38:17.368641+0000","last_active":"2023-10-06T10:38:21.384735+0000","last_peered":"2023-10-06T10:38:21.384735+0000","last_clean":"2023-10-06T10:38:21.384735+0000","last_became_active":"2023-10-06T10:38:17.368283+0000","last_became_peered":"2023-10-06T10:38:17.368283+0000","last_unstale":"2023-10-06T10:38:21.384735+0000","last_undegraded":"2023-10-06T10:38:21.384735+0000","last_fullsized":"2023-10-06T10:38:21.384735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:17.351581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.395567+0000","last_change":"2023-10-06T10:38:21.392954+0000","last_active":"2023-10-06T10:38:21.395567+0000","last_peered":"2023-10-06T10:38:21.395567+0000","last_clean":"2023-10-06T10:38:21.395567+0000","last_became_active":"2023-10-06T10:38:21.392585+0000","last_became_peered":"2023-10-06T10:38:21.392585+0000","last_unstale":"2023-10-06T10:38:21.395567+0000","last_undegraded":"2023-10-06T10:38:21.395567+0000","last_fullsized":"2023-10-06T10:38:21.395567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:54.407728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375768+0000","last_change":"2023-10-06T10:38:16.357199+0000","last_active":"2023-10-06T10:38:21.375768+0000","last_peered":"2023-10-06T10:38:21.375768+0000","last_clean":"2023-10-06T10:38:21.375768+0000","last_became_active":"2023-10-06T10:38:16.356936+0000","last_became_peered":"2023-10-06T10:38:16.356936+0000","last_unstale":"2023-10-06T10:38:21.375768+0000","last_undegraded":"2023-10-06T10:38:21.375768+0000","last_fullsized":"2023-10-06T10:38:21.375768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:02.115831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377449+0000","last_change":"2023-10-06T10:38:17.359930+0000","last_active":"2023-10-06T10:38:21.377449+0000","last_peered":"2023-10-06T10:38:21.377449+0000","last_clean":"2023-10-06T10:38:21.377449+0000","last_became_active":"2023-10-06T10:38:17.359677+0000","last_became_peered":"2023-10-06T10:38:17.359677+0000","last_unstale":"2023-10-06T10:38:21.377449+0000","last_undegraded":"2023-10-06T10:38:21.377449+0000","last_fullsized":"2023-10-06T10:38:21.377449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:13.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.399357+0000","last_change":"2023-10-06T10:38:21.396020+0000","last_active":"2023-10-06T10:38:21.399357+0000","last_peered":"2023-10-06T10:38:21.399357+0000","last_clean":"2023-10-06T10:38:21.399357+0000","last_became_active":"2023-10-06T10:38:21.395659+0000","last_became_peered":"2023-10-06T10:38:21.395659+0000","last_unstale":"2023-10-06T10:38:21.399357+0000","last_undegraded":"2023-10-06T10:38:21.399357+0000","last_fullsized":"2023-10-06T10:38:21.399357+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:45:46.392517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372543+0000","last_change":"2023-10-06T10:38:16.364249+0000","last_active":"2023-10-06T10:38:21.372543+0000","last_peered":"2023-10-06T10:38:21.372543+0000","last_clean":"2023-10-06T10:38:21.372543+0000","last_became_active":"2023-10-06T10:38:16.363959+0000","last_became_peered":"2023-10-06T10:38:16.363959+0000","last_unstale":"2023-10-06T10:38:21.372543+0000","last_undegraded":"2023-10-06T10:38:21.372543+0000","last_fullsized":"2023-10-06T10:38:21.372543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:27.473452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377610+0000","last_change":"2023-10-06T10:38:17.362439+0000","last_active":"2023-10-06T10:38:21.377610+0000","last_peered":"2023-10-06T10:38:21.377610+0000","last_clean":"2023-10-06T10:38:21.377610+0000","last_became_active":"2023-10-06T10:38:17.361921+0000","last_became_peered":"2023-10-06T10:38:17.361921+0000","last_unstale":"2023-10-06T10:38:21.377610+0000","last_undegraded":"2023-10-06T10:38:21.377610+0000","last_fullsized":"2023-10-06T10:38:21.377610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:30:01.208220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402322+0000","last_change":"2023-10-06T10:38:21.399211+0000","last_active":"2023-10-06T10:38:21.402322+0000","last_peered":"2023-10-06T10:38:21.402322+0000","last_clean":"2023-10-06T10:38:21.402322+0000","last_became_active":"2023-10-06T10:38:21.396487+0000","last_became_peered":"2023-10-06T10:38:21.396487+0000","last_unstale":"2023-10-06T10:38:21.402322+0000","last_undegraded":"2023-10-06T10:38:21.402322+0000","last_fullsized":"2023-10-06T10:38:21.402322+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:26.378955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384822+0000","last_change":"2023-10-06T10:38:16.362941+0000","last_active":"2023-10-06T10:38:21.384822+0000","last_peered":"2023-10-06T10:38:21.384822+0000","last_clean":"2023-10-06T10:38:21.384822+0000","last_became_active":"2023-10-06T10:38:16.362651+0000","last_became_peered":"2023-10-06T10:38:16.362651+0000","last_unstale":"2023-10-06T10:38:21.384822+0000","last_undegraded":"2023-10-06T10:38:21.384822+0000","last_fullsized":"2023-10-06T10:38:21.384822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:41.837644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377389+0000","last_change":"2023-10-06T10:38:17.367573+0000","last_active":"2023-10-06T10:38:21.377389+0000","last_peered":"2023-10-06T10:38:21.377389+0000","last_clean":"2023-10-06T10:38:21.377389+0000","last_became_active":"2023-10-06T10:38:17.367363+0000","last_became_peered":"2023-10-06T10:38:17.367363+0000","last_unstale":"2023-10-06T10:38:21.377389+0000","last_undegraded":"2023-10-06T10:38:21.377389+0000","last_fullsized":"2023-10-06T10:38:21.377389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:03.650819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.401267+0000","last_change":"2023-10-06T10:38:21.394943+0000","last_active":"2023-10-06T10:38:21.401267+0000","last_peered":"2023-10-06T10:38:21.401267+0000","last_clean":"2023-10-06T10:38:21.401267+0000","last_became_active":"2023-10-06T10:38:21.394604+0000","last_became_peered":"2023-10-06T10:38:21.394604+0000","last_unstale":"2023-10-06T10:38:21.401267+0000","last_undegraded":"2023-10-06T10:38:21.401267+0000","last_fullsized":"2023-10-06T10:38:21.401267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:17:59.362798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377491+0000","last_change":"2023-10-06T10:38:16.358983+0000","last_active":"2023-10-06T10:38:21.377491+0000","last_peered":"2023-10-06T10:38:21.377491+0000","last_clean":"2023-10-06T10:38:21.377491+0000","last_became_active":"2023-10-06T10:38:16.358338+0000","last_became_peered":"2023-10-06T10:38:16.358338+0000","last_unstale":"2023-10-06T10:38:21.377491+0000","last_undegraded":"2023-10-06T10:38:21.377491+0000","last_fullsized":"2023-10-06T10:38:21.377491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:02.115831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378341+0000","last_change":"2023-10-06T10:38:17.375549+0000","last_active":"2023-10-06T10:38:21.378341+0000","last_peered":"2023-10-06T10:38:21.378341+0000","last_clean":"2023-10-06T10:38:21.378341+0000","last_became_active":"2023-10-06T10:38:17.374707+0000","last_became_peered":"2023-10-06T10:38:17.374707+0000","last_unstale":"2023-10-06T10:38:21.378341+0000","last_undegraded":"2023-10-06T10:38:21.378341+0000","last_fullsized":"2023-10-06T10:38:21.378341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:50.378097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391124+0000","last_change":"2023-10-06T10:38:21.388787+0000","last_active":"2023-10-06T10:38:21.391124+0000","last_peered":"2023-10-06T10:38:21.391124+0000","last_clean":"2023-10-06T10:38:21.391124+0000","last_became_active":"2023-10-06T10:38:21.388432+0000","last_became_peered":"2023-10-06T10:38:21.388432+0000","last_unstale":"2023-10-06T10:38:21.391124+0000","last_undegraded":"2023-10-06T10:38:21.391124+0000","last_fullsized":"2023-10-06T10:38:21.391124+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:03.131588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372464+0000","last_change":"2023-10-06T10:38:16.366048+0000","last_active":"2023-10-06T10:38:21.372464+0000","last_peered":"2023-10-06T10:38:21.372464+0000","last_clean":"2023-10-06T10:38:21.372464+0000","last_became_active":"2023-10-06T10:38:16.365814+0000","last_became_peered":"2023-10-06T10:38:16.365814+0000","last_unstale":"2023-10-06T10:38:21.372464+0000","last_undegraded":"2023-10-06T10:38:21.372464+0000","last_fullsized":"2023-10-06T10:38:21.372464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:36.469014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.379002+0000","last_change":"2023-10-06T10:38:17.358727+0000","last_active":"2023-10-06T10:38:21.379002+0000","last_peered":"2023-10-06T10:38:21.379002+0000","last_clean":"2023-10-06T10:38:21.379002+0000","last_became_active":"2023-10-06T10:38:17.358442+0000","last_became_peered":"2023-10-06T10:38:17.358442+0000","last_unstale":"2023-10-06T10:38:21.379002+0000","last_undegraded":"2023-10-06T10:38:21.379002+0000","last_fullsized":"2023-10-06T10:38:21.379002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:17.351581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.401082+0000","last_change":"2023-10-06T10:38:21.397353+0000","last_active":"2023-10-06T10:38:21.401082+0000","last_peered":"2023-10-06T10:38:21.401082+0000","last_clean":"2023-10-06T10:38:21.401082+0000","last_became_active":"2023-10-06T10:38:21.396788+0000","last_became_peered":"2023-10-06T10:38:21.396788+0000","last_unstale":"2023-10-06T10:38:21.401082+0000","last_undegraded":"2023-10-06T10:38:21.401082+0000","last_fullsized":"2023-10-06T10:38:21.401082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:50:06.948087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.373072+0000","last_change":"2023-10-06T10:38:16.362604+0000","last_active":"2023-10-06T10:38:21.373072+0000","last_peered":"2023-10-06T10:38:21.373072+0000","last_clean":"2023-10-06T10:38:21.373072+0000","last_became_active":"2023-10-06T10:38:16.362444+0000","last_became_peered":"2023-10-06T10:38:16.362444+0000","last_unstale":"2023-10-06T10:38:21.373072+0000","last_undegraded":"2023-10-06T10:38:21.373072+0000","last_fullsized":"2023-10-06T10:38:21.373072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:57.215481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377559+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:38:21.377559+0000","last_peered":"2023-10-06T10:38:21.377559+0000","last_clean":"2023-10-06T10:38:21.377559+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:38:21.377559+0000","last_undegraded":"2023-10-06T10:38:21.377559+0000","last_fullsized":"2023-10-06T10:38:21.377559+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372292+0000","last_change":"2023-10-06T10:38:17.374346+0000","last_active":"2023-10-06T10:38:21.372292+0000","last_peered":"2023-10-06T10:38:21.372292+0000","last_clean":"2023-10-06T10:38:21.372292+0000","last_became_active":"2023-10-06T10:38:17.374068+0000","last_became_peered":"2023-10-06T10:38:17.374068+0000","last_unstale":"2023-10-06T10:38:21.372292+0000","last_undegraded":"2023-10-06T10:38:21.372292+0000","last_fullsized":"2023-10-06T10:38:21.372292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:29.897092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.393470+0000","last_change":"2023-10-06T10:38:21.387372+0000","last_active":"2023-10-06T10:38:21.393470+0000","last_peered":"2023-10-06T10:38:21.393470+0000","last_clean":"2023-10-06T10:38:21.393470+0000","last_became_active":"2023-10-06T10:38:21.386907+0000","last_became_peered":"2023-10-06T10:38:21.386907+0000","last_unstale":"2023-10-06T10:38:21.393470+0000","last_undegraded":"2023-10-06T10:38:21.393470+0000","last_fullsized":"2023-10-06T10:38:21.393470+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:36.144881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372792+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:38:21.372792+0000","last_peered":"2023-10-06T10:38:21.372792+0000","last_clean":"2023-10-06T10:38:21.372792+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:38:21.372792+0000","last_undegraded":"2023-10-06T10:38:21.372792+0000","last_fullsized":"2023-10-06T10:38:21.372792+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.433239+0000","last_change":"2023-10-06T10:38:16.366116+0000","last_active":"2023-10-06T10:38:21.433239+0000","last_peered":"2023-10-06T10:38:21.433239+0000","last_clean":"2023-10-06T10:38:21.433239+0000","last_became_active":"2023-10-06T10:38:16.365875+0000","last_became_peered":"2023-10-06T10:38:16.365875+0000","last_unstale":"2023-10-06T10:38:21.433239+0000","last_undegraded":"2023-10-06T10:38:21.433239+0000","last_fullsized":"2023-10-06T10:38:21.433239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:35.127396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377577+0000","last_change":"2023-10-06T10:38:17.358654+0000","last_active":"2023-10-06T10:38:21.377577+0000","last_peered":"2023-10-06T10:38:21.377577+0000","last_clean":"2023-10-06T10:38:21.377577+0000","last_became_active":"2023-10-06T10:38:17.358361+0000","last_became_peered":"2023-10-06T10:38:17.358361+0000","last_unstale":"2023-10-06T10:38:21.377577+0000","last_undegraded":"2023-10-06T10:38:21.377577+0000","last_fullsized":"2023-10-06T10:38:21.377577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:06.214524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.397881+0000","last_change":"2023-10-06T10:38:21.392569+0000","last_active":"2023-10-06T10:38:21.397881+0000","last_peered":"2023-10-06T10:38:21.397881+0000","last_clean":"2023-10-06T10:38:21.397881+0000","last_became_active":"2023-10-06T10:38:21.392210+0000","last_became_peered":"2023-10-06T10:38:21.392210+0000","last_unstale":"2023-10-06T10:38:21.397881+0000","last_undegraded":"2023-10-06T10:38:21.397881+0000","last_fullsized":"2023-10-06T10:38:21.397881+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:14:10.094621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377969+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:38:21.377969+0000","last_peered":"2023-10-06T10:38:21.377969+0000","last_clean":"2023-10-06T10:38:21.377969+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:38:21.377969+0000","last_undegraded":"2023-10-06T10:38:21.377969+0000","last_fullsized":"2023-10-06T10:38:21.377969+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.390711+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:38:21.390711+0000","last_peered":"2023-10-06T10:38:21.390711+0000","last_clean":"2023-10-06T10:38:21.390711+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:38:21.390711+0000","last_undegraded":"2023-10-06T10:38:21.390711+0000","last_fullsized":"2023-10-06T10:38:21.390711+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375378+0000","last_change":"2023-10-06T10:38:16.362718+0000","last_active":"2023-10-06T10:38:21.375378+0000","last_peered":"2023-10-06T10:38:21.375378+0000","last_clean":"2023-10-06T10:38:21.375378+0000","last_became_active":"2023-10-06T10:38:16.361895+0000","last_became_peered":"2023-10-06T10:38:16.361895+0000","last_unstale":"2023-10-06T10:38:21.375378+0000","last_undegraded":"2023-10-06T10:38:21.375378+0000","last_fullsized":"2023-10-06T10:38:21.375378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:45.092301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.376156+0000","last_change":"2023-10-06T10:38:17.359496+0000","last_active":"2023-10-06T10:38:21.376156+0000","last_peered":"2023-10-06T10:38:21.376156+0000","last_clean":"2023-10-06T10:38:21.376156+0000","last_became_active":"2023-10-06T10:38:17.359011+0000","last_became_peered":"2023-10-06T10:38:17.359011+0000","last_unstale":"2023-10-06T10:38:21.376156+0000","last_undegraded":"2023-10-06T10:38:21.376156+0000","last_fullsized":"2023-10-06T10:38:21.376156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:06.214524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402221+0000","last_change":"2023-10-06T10:38:21.396996+0000","last_active":"2023-10-06T10:38:21.402221+0000","last_peered":"2023-10-06T10:38:21.402221+0000","last_clean":"2023-10-06T10:38:21.402221+0000","last_became_active":"2023-10-06T10:38:21.396694+0000","last_became_peered":"2023-10-06T10:38:21.396694+0000","last_unstale":"2023-10-06T10:38:21.402221+0000","last_undegraded":"2023-10-06T10:38:21.402221+0000","last_fullsized":"2023-10-06T10:38:21.402221+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:14.207234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.386749+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:38:21.386749+0000","last_peered":"2023-10-06T10:38:21.386749+0000","last_clean":"2023-10-06T10:38:21.386749+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:38:21.386749+0000","last_undegraded":"2023-10-06T10:38:21.386749+0000","last_fullsized":"2023-10-06T10:38:21.386749+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.376477+0000","last_change":"2023-10-06T10:38:16.357154+0000","last_active":"2023-10-06T10:38:21.376477+0000","last_peered":"2023-10-06T10:38:21.376477+0000","last_clean":"2023-10-06T10:38:21.376477+0000","last_became_active":"2023-10-06T10:38:16.356942+0000","last_became_peered":"2023-10-06T10:38:16.356942+0000","last_unstale":"2023-10-06T10:38:21.376477+0000","last_undegraded":"2023-10-06T10:38:21.376477+0000","last_fullsized":"2023-10-06T10:38:21.376477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:16.528477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378971+0000","last_change":"2023-10-06T10:38:17.358788+0000","last_active":"2023-10-06T10:38:21.378971+0000","last_peered":"2023-10-06T10:38:21.378971+0000","last_clean":"2023-10-06T10:38:21.378971+0000","last_became_active":"2023-10-06T10:38:17.358514+0000","last_became_peered":"2023-10-06T10:38:17.358514+0000","last_unstale":"2023-10-06T10:38:21.378971+0000","last_undegraded":"2023-10-06T10:38:21.378971+0000","last_fullsized":"2023-10-06T10:38:21.378971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:49.973815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402838+0000","last_change":"2023-10-06T10:38:21.401259+0000","last_active":"2023-10-06T10:38:21.402838+0000","last_peered":"2023-10-06T10:38:21.402838+0000","last_clean":"2023-10-06T10:38:21.402838+0000","last_became_active":"2023-10-06T10:38:21.400865+0000","last_became_peered":"2023-10-06T10:38:21.400865+0000","last_unstale":"2023-10-06T10:38:21.402838+0000","last_undegraded":"2023-10-06T10:38:21.402838+0000","last_fullsized":"2023-10-06T10:38:21.402838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:30.495506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.384378+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:38:21.384378+0000","last_peered":"2023-10-06T10:38:21.384378+0000","last_clean":"2023-10-06T10:38:21.384378+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:38:21.384378+0000","last_undegraded":"2023-10-06T10:38:21.384378+0000","last_fullsized":"2023-10-06T10:38:21.384378+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372330+0000","last_change":"2023-10-06T10:38:16.358939+0000","last_active":"2023-10-06T10:38:21.372330+0000","last_peered":"2023-10-06T10:38:21.372330+0000","last_clean":"2023-10-06T10:38:21.372330+0000","last_became_active":"2023-10-06T10:38:16.358685+0000","last_became_peered":"2023-10-06T10:38:16.358685+0000","last_unstale":"2023-10-06T10:38:21.372330+0000","last_undegraded":"2023-10-06T10:38:21.372330+0000","last_fullsized":"2023-10-06T10:38:21.372330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:41.837644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385109+0000","last_change":"2023-10-06T10:38:17.366729+0000","last_active":"2023-10-06T10:38:21.385109+0000","last_peered":"2023-10-06T10:38:21.385109+0000","last_clean":"2023-10-06T10:38:21.385109+0000","last_became_active":"2023-10-06T10:38:17.366408+0000","last_became_peered":"2023-10-06T10:38:17.366408+0000","last_unstale":"2023-10-06T10:38:21.385109+0000","last_undegraded":"2023-10-06T10:38:21.385109+0000","last_fullsized":"2023-10-06T10:38:21.385109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:35.729655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.395664+0000","last_change":"2023-10-06T10:38:21.392874+0000","last_active":"2023-10-06T10:38:21.395664+0000","last_peered":"2023-10-06T10:38:21.395664+0000","last_clean":"2023-10-06T10:38:21.395664+0000","last_became_active":"2023-10-06T10:38:21.392487+0000","last_became_peered":"2023-10-06T10:38:21.392487+0000","last_unstale":"2023-10-06T10:38:21.395664+0000","last_undegraded":"2023-10-06T10:38:21.395664+0000","last_fullsized":"2023-10-06T10:38:21.395664+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:39.967976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375754+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:38:21.375754+0000","last_peered":"2023-10-06T10:38:21.375754+0000","last_clean":"2023-10-06T10:38:21.375754+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:38:21.375754+0000","last_undegraded":"2023-10-06T10:38:21.375754+0000","last_fullsized":"2023-10-06T10:38:21.375754+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377566+0000","last_change":"2023-10-06T10:38:16.362244+0000","last_active":"2023-10-06T10:38:21.377566+0000","last_peered":"2023-10-06T10:38:21.377566+0000","last_clean":"2023-10-06T10:38:21.377566+0000","last_became_active":"2023-10-06T10:38:16.361869+0000","last_became_peered":"2023-10-06T10:38:16.361869+0000","last_unstale":"2023-10-06T10:38:21.377566+0000","last_undegraded":"2023-10-06T10:38:21.377566+0000","last_fullsized":"2023-10-06T10:38:21.377566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:57.215481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.379113+0000","last_change":"2023-10-06T10:38:17.367649+0000","last_active":"2023-10-06T10:38:21.379113+0000","last_peered":"2023-10-06T10:38:21.379113+0000","last_clean":"2023-10-06T10:38:21.379113+0000","last_became_active":"2023-10-06T10:38:17.366773+0000","last_became_peered":"2023-10-06T10:38:17.366773+0000","last_unstale":"2023-10-06T10:38:21.379113+0000","last_undegraded":"2023-10-06T10:38:21.379113+0000","last_fullsized":"2023-10-06T10:38:21.379113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:40.679246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.401815+0000","last_change":"2023-10-06T10:38:21.393988+0000","last_active":"2023-10-06T10:38:21.401815+0000","last_peered":"2023-10-06T10:38:21.401815+0000","last_clean":"2023-10-06T10:38:21.401815+0000","last_became_active":"2023-10-06T10:38:21.393668+0000","last_became_peered":"2023-10-06T10:38:21.393668+0000","last_unstale":"2023-10-06T10:38:21.401815+0000","last_undegraded":"2023-10-06T10:38:21.401815+0000","last_fullsized":"2023-10-06T10:38:21.401815+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:33.926723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385218+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:38:21.385218+0000","last_peered":"2023-10-06T10:38:21.385218+0000","last_clean":"2023-10-06T10:38:21.385218+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:38:21.385218+0000","last_undegraded":"2023-10-06T10:38:21.385218+0000","last_fullsized":"2023-10-06T10:38:21.385218+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.379391+0000","last_change":"2023-10-06T10:38:16.360214+0000","last_active":"2023-10-06T10:38:21.379391+0000","last_peered":"2023-10-06T10:38:21.379391+0000","last_clean":"2023-10-06T10:38:21.379391+0000","last_became_active":"2023-10-06T10:38:16.359912+0000","last_became_peered":"2023-10-06T10:38:16.359912+0000","last_unstale":"2023-10-06T10:38:21.379391+0000","last_undegraded":"2023-10-06T10:38:21.379391+0000","last_fullsized":"2023-10-06T10:38:21.379391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:41.837644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377275+0000","last_change":"2023-10-06T10:38:17.354930+0000","last_active":"2023-10-06T10:38:21.377275+0000","last_peered":"2023-10-06T10:38:21.377275+0000","last_clean":"2023-10-06T10:38:21.377275+0000","last_became_active":"2023-10-06T10:38:17.354637+0000","last_became_peered":"2023-10-06T10:38:17.354637+0000","last_unstale":"2023-10-06T10:38:21.377275+0000","last_undegraded":"2023-10-06T10:38:21.377275+0000","last_fullsized":"2023-10-06T10:38:21.377275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:03.953719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391646+0000","last_change":"2023-10-06T10:38:21.387944+0000","last_active":"2023-10-06T10:38:21.391646+0000","last_peered":"2023-10-06T10:38:21.391646+0000","last_clean":"2023-10-06T10:38:21.391646+0000","last_became_active":"2023-10-06T10:38:21.387580+0000","last_became_peered":"2023-10-06T10:38:21.387580+0000","last_unstale":"2023-10-06T10:38:21.391646+0000","last_undegraded":"2023-10-06T10:38:21.391646+0000","last_fullsized":"2023-10-06T10:38:21.391646+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:54:20.934943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385079+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:38:21.385079+0000","last_peered":"2023-10-06T10:38:21.385079+0000","last_clean":"2023-10-06T10:38:21.385079+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:38:21.385079+0000","last_undegraded":"2023-10-06T10:38:21.385079+0000","last_fullsized":"2023-10-06T10:38:21.385079+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377995+0000","last_change":"2023-10-06T10:38:16.358519+0000","last_active":"2023-10-06T10:38:21.377995+0000","last_peered":"2023-10-06T10:38:21.377995+0000","last_clean":"2023-10-06T10:38:21.377995+0000","last_became_active":"2023-10-06T10:38:16.357730+0000","last_became_peered":"2023-10-06T10:38:16.357730+0000","last_unstale":"2023-10-06T10:38:21.377995+0000","last_undegraded":"2023-10-06T10:38:21.377995+0000","last_fullsized":"2023-10-06T10:38:21.377995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:36.469014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378421+0000","last_change":"2023-10-06T10:38:17.358314+0000","last_active":"2023-10-06T10:38:21.378421+0000","last_peered":"2023-10-06T10:38:21.378421+0000","last_clean":"2023-10-06T10:38:21.378421+0000","last_became_active":"2023-10-06T10:38:17.358060+0000","last_became_peered":"2023-10-06T10:38:17.358060+0000","last_unstale":"2023-10-06T10:38:21.378421+0000","last_undegraded":"2023-10-06T10:38:21.378421+0000","last_fullsized":"2023-10-06T10:38:21.378421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:34:27.066675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.400514+0000","last_change":"2023-10-06T10:38:21.391294+0000","last_active":"2023-10-06T10:38:21.400514+0000","last_peered":"2023-10-06T10:38:21.400514+0000","last_clean":"2023-10-06T10:38:21.400514+0000","last_became_active":"2023-10-06T10:38:21.390927+0000","last_became_peered":"2023-10-06T10:38:21.390927+0000","last_unstale":"2023-10-06T10:38:21.400514+0000","last_undegraded":"2023-10-06T10:38:21.400514+0000","last_fullsized":"2023-10-06T10:38:21.400514+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:53.012871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377366+0000","last_change":"2023-10-06T10:38:16.360511+0000","last_active":"2023-10-06T10:38:21.377366+0000","last_peered":"2023-10-06T10:38:21.377366+0000","last_clean":"2023-10-06T10:38:21.377366+0000","last_became_active":"2023-10-06T10:38:16.360260+0000","last_became_peered":"2023-10-06T10:38:16.360260+0000","last_unstale":"2023-10-06T10:38:21.377366+0000","last_undegraded":"2023-10-06T10:38:21.377366+0000","last_fullsized":"2023-10-06T10:38:21.377366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:57.215481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.383881+0000","last_change":"2023-10-06T10:38:17.365370+0000","last_active":"2023-10-06T10:38:21.383881+0000","last_peered":"2023-10-06T10:38:21.383881+0000","last_clean":"2023-10-06T10:38:21.383881+0000","last_became_active":"2023-10-06T10:38:17.365110+0000","last_became_peered":"2023-10-06T10:38:17.365110+0000","last_unstale":"2023-10-06T10:38:21.383881+0000","last_undegraded":"2023-10-06T10:38:21.383881+0000","last_fullsized":"2023-10-06T10:38:21.383881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:37.475956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.402373+0000","last_change":"2023-10-06T10:38:21.399284+0000","last_active":"2023-10-06T10:38:21.402373+0000","last_peered":"2023-10-06T10:38:21.402373+0000","last_clean":"2023-10-06T10:38:21.402373+0000","last_became_active":"2023-10-06T10:38:21.398931+0000","last_became_peered":"2023-10-06T10:38:21.398931+0000","last_unstale":"2023-10-06T10:38:21.402373+0000","last_undegraded":"2023-10-06T10:38:21.402373+0000","last_fullsized":"2023-10-06T10:38:21.402373+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:17:39.406756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.375583+0000","last_change":"2023-10-06T10:38:16.353975+0000","last_active":"2023-10-06T10:38:21.375583+0000","last_peered":"2023-10-06T10:38:21.375583+0000","last_clean":"2023-10-06T10:38:21.375583+0000","last_became_active":"2023-10-06T10:38:16.353629+0000","last_became_peered":"2023-10-06T10:38:16.353629+0000","last_unstale":"2023-10-06T10:38:21.375583+0000","last_undegraded":"2023-10-06T10:38:21.375583+0000","last_fullsized":"2023-10-06T10:38:21.375583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:20.696308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377446+0000","last_change":"2023-10-06T10:38:17.363838+0000","last_active":"2023-10-06T10:38:21.377446+0000","last_peered":"2023-10-06T10:38:21.377446+0000","last_clean":"2023-10-06T10:38:21.377446+0000","last_became_active":"2023-10-06T10:38:17.363580+0000","last_became_peered":"2023-10-06T10:38:17.363580+0000","last_unstale":"2023-10-06T10:38:21.377446+0000","last_undegraded":"2023-10-06T10:38:21.377446+0000","last_fullsized":"2023-10-06T10:38:21.377446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:43.614842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.399586+0000","last_change":"2023-10-06T10:38:21.396452+0000","last_active":"2023-10-06T10:38:21.399586+0000","last_peered":"2023-10-06T10:38:21.399586+0000","last_clean":"2023-10-06T10:38:21.399586+0000","last_became_active":"2023-10-06T10:38:21.396052+0000","last_became_peered":"2023-10-06T10:38:21.396052+0000","last_unstale":"2023-10-06T10:38:21.399586+0000","last_undegraded":"2023-10-06T10:38:21.399586+0000","last_fullsized":"2023-10-06T10:38:21.399586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:17:39.406756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377909+0000","last_change":"2023-10-06T10:38:16.354639+0000","last_active":"2023-10-06T10:38:21.377909+0000","last_peered":"2023-10-06T10:38:21.377909+0000","last_clean":"2023-10-06T10:38:21.377909+0000","last_became_active":"2023-10-06T10:38:16.354367+0000","last_became_peered":"2023-10-06T10:38:16.354367+0000","last_unstale":"2023-10-06T10:38:21.377909+0000","last_undegraded":"2023-10-06T10:38:21.377909+0000","last_fullsized":"2023-10-06T10:38:21.377909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:57.215481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.377353+0000","last_change":"2023-10-06T10:38:17.358682+0000","last_active":"2023-10-06T10:38:21.377353+0000","last_peered":"2023-10-06T10:38:21.377353+0000","last_clean":"2023-10-06T10:38:21.377353+0000","last_became_active":"2023-10-06T10:38:17.358409+0000","last_became_peered":"2023-10-06T10:38:17.358409+0000","last_unstale":"2023-10-06T10:38:21.377353+0000","last_undegraded":"2023-10-06T10:38:21.377353+0000","last_fullsized":"2023-10-06T10:38:21.377353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:52:14.320333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.391815+0000","last_change":"2023-10-06T10:38:21.390566+0000","last_active":"2023-10-06T10:38:21.391815+0000","last_peered":"2023-10-06T10:38:21.391815+0000","last_clean":"2023-10-06T10:38:21.391815+0000","last_became_active":"2023-10-06T10:38:21.390172+0000","last_became_peered":"2023-10-06T10:38:21.390172+0000","last_unstale":"2023-10-06T10:38:21.391815+0000","last_undegraded":"2023-10-06T10:38:21.391815+0000","last_fullsized":"2023-10-06T10:38:21.391815+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:37.174852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372471+0000","last_change":"2023-10-06T10:38:16.353396+0000","last_active":"2023-10-06T10:38:21.372471+0000","last_peered":"2023-10-06T10:38:21.372471+0000","last_clean":"2023-10-06T10:38:21.372471+0000","last_became_active":"2023-10-06T10:38:16.353136+0000","last_became_peered":"2023-10-06T10:38:16.353136+0000","last_unstale":"2023-10-06T10:38:21.372471+0000","last_undegraded":"2023-10-06T10:38:21.372471+0000","last_fullsized":"2023-10-06T10:38:21.372471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:02.115831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372424+0000","last_change":"2023-10-06T10:38:17.364589+0000","last_active":"2023-10-06T10:38:21.372424+0000","last_peered":"2023-10-06T10:38:21.372424+0000","last_clean":"2023-10-06T10:38:21.372424+0000","last_became_active":"2023-10-06T10:38:17.364364+0000","last_became_peered":"2023-10-06T10:38:17.364364+0000","last_unstale":"2023-10-06T10:38:21.372424+0000","last_undegraded":"2023-10-06T10:38:21.372424+0000","last_fullsized":"2023-10-06T10:38:21.372424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:13.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.399883+0000","last_change":"2023-10-06T10:38:21.395180+0000","last_active":"2023-10-06T10:38:21.399883+0000","last_peered":"2023-10-06T10:38:21.399883+0000","last_clean":"2023-10-06T10:38:21.399883+0000","last_became_active":"2023-10-06T10:38:21.394859+0000","last_became_peered":"2023-10-06T10:38:21.394859+0000","last_unstale":"2023-10-06T10:38:21.399883+0000","last_undegraded":"2023-10-06T10:38:21.399883+0000","last_fullsized":"2023-10-06T10:38:21.399883+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:20.364464+0000","last_clean_scrub_stamp":"2023-10-06T10:38:20.364464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:54:20.934943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378521+0000","last_change":"2023-10-06T10:38:16.361291+0000","last_active":"2023-10-06T10:38:21.378521+0000","last_peered":"2023-10-06T10:38:21.378521+0000","last_clean":"2023-10-06T10:38:21.378521+0000","last_became_active":"2023-10-06T10:38:16.361027+0000","last_became_peered":"2023-10-06T10:38:16.361027+0000","last_unstale":"2023-10-06T10:38:21.378521+0000","last_undegraded":"2023-10-06T10:38:21.378521+0000","last_fullsized":"2023-10-06T10:38:21.378521+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:15.327890+0000","last_clean_scrub_stamp":"2023-10-06T10:38:15.327890+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:36.469014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378387+0000","last_change":"2023-10-06T10:38:17.375274+0000","last_active":"2023-10-06T10:38:21.378387+0000","last_peered":"2023-10-06T10:38:21.378387+0000","last_clean":"2023-10-06T10:38:21.378387+0000","last_became_active":"2023-10-06T10:38:17.374982+0000","last_became_peered":"2023-10-06T10:38:17.374982+0000","last_unstale":"2023-10-06T10:38:21.378387+0000","last_undegraded":"2023-10-06T10:38:21.378387+0000","last_fullsized":"2023-10-06T10:38:21.378387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:59.822053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.385168+0000","last_change":"2023-10-06T10:38:17.368598+0000","last_active":"2023-10-06T10:38:21.385168+0000","last_peered":"2023-10-06T10:38:21.385168+0000","last_clean":"2023-10-06T10:38:21.385168+0000","last_became_active":"2023-10-06T10:38:17.367720+0000","last_became_peered":"2023-10-06T10:38:17.367720+0000","last_unstale":"2023-10-06T10:38:21.385168+0000","last_undegraded":"2023-10-06T10:38:21.385168+0000","last_fullsized":"2023-10-06T10:38:21.385168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:40.679246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372285+0000","last_change":"2023-10-06T10:38:17.362818+0000","last_active":"2023-10-06T10:38:21.372285+0000","last_peered":"2023-10-06T10:38:21.372285+0000","last_clean":"2023-10-06T10:38:21.372285+0000","last_became_active":"2023-10-06T10:38:17.362585+0000","last_became_peered":"2023-10-06T10:38:17.362585+0000","last_unstale":"2023-10-06T10:38:21.372285+0000","last_undegraded":"2023-10-06T10:38:21.372285+0000","last_fullsized":"2023-10-06T10:38:21.372285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:03.650819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.378216+0000","last_change":"2023-10-06T10:38:17.375510+0000","last_active":"2023-10-06T10:38:21.378216+0000","last_peered":"2023-10-06T10:38:21.378216+0000","last_clean":"2023-10-06T10:38:21.378216+0000","last_became_active":"2023-10-06T10:38:17.375023+0000","last_became_peered":"2023-10-06T10:38:17.375023+0000","last_unstale":"2023-10-06T10:38:21.378216+0000","last_undegraded":"2023-10-06T10:38:21.378216+0000","last_fullsized":"2023-10-06T10:38:21.378216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:10.177603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.372410+0000","last_change":"2023-10-06T10:38:17.370520+0000","last_active":"2023-10-06T10:38:21.372410+0000","last_peered":"2023-10-06T10:38:21.372410+0000","last_clean":"2023-10-06T10:38:21.372410+0000","last_became_active":"2023-10-06T10:38:17.370265+0000","last_became_peered":"2023-10-06T10:38:17.370265+0000","last_unstale":"2023-10-06T10:38:21.372410+0000","last_undegraded":"2023-10-06T10:38:21.372410+0000","last_fullsized":"2023-10-06T10:38:21.372410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:06.214524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T10:38:21.379183+0000","last_change":"2023-10-06T10:38:17.372067+0000","last_active":"2023-10-06T10:38:21.379183+0000","last_peered":"2023-10-06T10:38:21.379183+0000","last_clean":"2023-10-06T10:38:21.379183+0000","last_became_active":"2023-10-06T10:38:17.371813+0000","last_became_peered":"2023-10-06T10:38:17.371813+0000","last_unstale":"2023-10-06T10:38:21.379183+0000","last_undegraded":"2023-10-06T10:38:21.379183+0000","last_fullsized":"2023-10-06T10:38:21.379183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:16.334833+0000","last_clean_scrub_stamp":"2023-10-06T10:38:16.334833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:23:23.463492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4484,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":876544,"data_stored":698736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":103085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298116,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073724,"statfs":{"total":96636764160,"available":96331493376,"internally_reserved":0,"allocated":847872,"data_stored":694806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":258048,"data_stored":104992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":104438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":258048,"data_stored":104528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297556,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074284,"statfs":{"total":96636764160,"available":96332066816,"internally_reserved":0,"allocated":274432,"data_stored":108208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":105062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:38:36.437 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T10:38:36.437 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-06T10:38:36.437 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-06T10:38:36.437 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T10:38:36.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.550+0000 7fa7c1030700 1 Processor -- start 2023-10-06T10:38:36.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.550+0000 7fa7c1030700 1 -- start start 2023-10-06T10:38:36.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.551+0000 7fa7c1030700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa7bc0cad80 0x7fa7bc159f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:36.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.551+0000 7fa7c1030700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc068d10 con 0x7fa7bc0cb5f0 2023-10-06T10:38:36.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.551+0000 7fa7c1030700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc15a670 con 0x7fa7bc159ad0 2023-10-06T10:38:36.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.551+0000 7fa7c1030700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7bc15a490 con 0x7fa7bc0cad80 2023-10-06T10:38:36.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.551+0000 7fa7bb59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa7bc0cad80 0x7fa7bc159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:36.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.551+0000 7fa7bb59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa7bc0cad80 0x7fa7bc159f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36892/0 (socket says 172.21.15.92:36892) 2023-10-06T10:38:36.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.551+0000 7fa7bad9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa7bc0cb5f0 0x7fa7bc0691a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40118/0 (socket says 172.21.15.92:40118) 2023-10-06T10:38:36.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.551+0000 7fa7bb59e700 1 -- 172.21.15.92:0/4122191660 learned_addr learned my addr 172.21.15.92:0/4122191660 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:36.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.552+0000 7fa7bb59e700 1 -- 172.21.15.92:0/4122191660 >> v1:172.21.15.124:6789/0 conn(0x7fa7bc159ad0 legacy=0x7fa7bc0ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:36.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.552+0000 7fa7bb59e700 1 -- 172.21.15.92:0/4122191660 >> v1:172.21.15.92:6789/0 conn(0x7fa7bc0cb5f0 legacy=0x7fa7bc0691a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:36.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.552+0000 7fa7bb59e700 1 -- 172.21.15.92:0/4122191660 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc15a950 con 0x7fa7bc0cad80 2023-10-06T10:38:36.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.552+0000 7fa7ba59c700 1 -- 172.21.15.92:0/4122191660 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2597612206 0 0) 0x7fa7bc068d10 con 0x7fa7bc0cb5f0 2023-10-06T10:38:36.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.552+0000 7fa7bb59e700 1 --2- 172.21.15.92:0/4122191660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc0cad80 0x7fa7bc159f50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009f20 tx=0x7fa7b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b8d26c911bb65d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:36.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.552+0000 7fa7ba59c700 1 -- 172.21.15.92:0/4122191660 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b400e070 con 0x7fa7bc0cad80 2023-10-06T10:38:36.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.552+0000 7fa7ba59c700 1 -- 172.21.15.92:0/4122191660 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b400ba40 con 0x7fa7bc0cad80 2023-10-06T10:38:36.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.552+0000 7fa7ba59c700 1 -- 172.21.15.92:0/4122191660 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b40092e0 con 0x7fa7bc0cad80 2023-10-06T10:38:36.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.552+0000 7fa7c1030700 1 -- 172.21.15.92:0/4122191660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc0cad80 msgr2=0x7fa7bc159f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:36.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.552+0000 7fa7c1030700 1 --2- 172.21.15.92:0/4122191660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc0cad80 0x7fa7bc159f50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009f20 tx=0x7fa7b4009840 comp rx=0 tx=0).stop 2023-10-06T10:38:36.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.552+0000 7fa7c1030700 1 -- 172.21.15.92:0/4122191660 shutdown_connections 2023-10-06T10:38:36.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.552+0000 7fa7c1030700 1 --2- 172.21.15.92:0/4122191660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc0cad80 0x7fa7bc159f50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.552+0000 7fa7c1030700 1 -- 172.21.15.92:0/4122191660 >> 172.21.15.92:0/4122191660 conn(0x7fa7bc0707d0 msgr2=0x7fa7bc070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:36.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.552+0000 7fa7c1030700 1 -- 172.21.15.92:0/4122191660 shutdown_connections 2023-10-06T10:38:36.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.552+0000 7fa7c1030700 1 -- 172.21.15.92:0/4122191660 wait complete. 2023-10-06T10:38:36.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.554+0000 7fa7c1030700 1 Processor -- start 2023-10-06T10:38:36.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.554+0000 7fa7c1030700 1 -- start start 2023-10-06T10:38:36.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.554+0000 7fa7c1030700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7bc0cad80 0x7fa7bc158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:36.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.554+0000 7fa7c1030700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7bc0cb5f0 0x7fa7bc0c29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:36.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.554+0000 7fa7c1030700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc159ad0 0x7fa7bc0c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:36.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.554+0000 7fa7c1030700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc1594b0 con 0x7fa7bc159ad0 2023-10-06T10:38:36.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.554+0000 7fa7c1030700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc0c5bf0 con 0x7fa7bc0cb5f0 2023-10-06T10:38:36.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.554+0000 7fa7c1030700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc0c2ef0 con 0x7fa7bc0cad80 2023-10-06T10:38:36.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.554+0000 7fa7bb59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc159ad0 0x7fa7bc0c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:36.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.554+0000 7fa7bb59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc159ad0 0x7fa7bc0c6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36904/0 (socket says 172.21.15.92:36904) 2023-10-06T10:38:36.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.554+0000 7fa7bb59e700 1 -- 172.21.15.92:0/771239308 learned_addr learned my addr 172.21.15.92:0/771239308 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:36.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.555+0000 7fa7b259c700 1 --2- 172.21.15.92:0/771239308 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7bc0cb5f0 0x7fa7bc0c29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:36.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.555+0000 7fa7bad9d700 1 --2- 172.21.15.92:0/771239308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7bc0cad80 0x7fa7bc158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:36.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.555+0000 7fa7bb59e700 1 -- 172.21.15.92:0/771239308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7bc0cad80 msgr2=0x7fa7bc158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:36.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.555+0000 7fa7bb59e700 1 --2- 172.21.15.92:0/771239308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7bc0cad80 0x7fa7bc158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.555+0000 7fa7bb59e700 1 -- 172.21.15.92:0/771239308 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7bc0cb5f0 msgr2=0x7fa7bc0c29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:36.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.555+0000 7fa7bb59e700 1 --2- 172.21.15.92:0/771239308 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7bc0cb5f0 0x7fa7bc0c29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.555+0000 7fa7bb59e700 1 -- 172.21.15.92:0/771239308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b4009ae0 con 0x7fa7bc159ad0 2023-10-06T10:38:36.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.555+0000 7fa7bad9d700 1 --2- 172.21.15.92:0/771239308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7bc0cad80 0x7fa7bc158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:36.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.555+0000 7fa7bb59e700 1 --2- 172.21.15.92:0/771239308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc159ad0 0x7fa7bc0c6080 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009880 tx=0x7fa7b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:36.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.555+0000 7fa7b37fe700 1 -- 172.21.15.92:0/771239308 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4012db0 con 0x7fa7bc159ad0 2023-10-06T10:38:36.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.555+0000 7fa7b37fe700 1 -- 172.21.15.92:0/771239308 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b401e430 con 0x7fa7bc159ad0 2023-10-06T10:38:36.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.555+0000 7fa7c1030700 1 -- 172.21.15.92:0/771239308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7bc0c52e0 con 0x7fa7bc159ad0 2023-10-06T10:38:36.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.555+0000 7fa7b37fe700 1 -- 172.21.15.92:0/771239308 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b401e7f0 con 0x7fa7bc159ad0 2023-10-06T10:38:36.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.555+0000 7fa7c1030700 1 -- 172.21.15.92:0/771239308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7bc0c55a0 con 0x7fa7bc159ad0 2023-10-06T10:38:36.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.556+0000 7fa7c1030700 1 -- 172.21.15.92:0/771239308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7bc0c5d80 con 0x7fa7bc159ad0 2023-10-06T10:38:36.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.556+0000 7fa7b37fe700 1 -- 172.21.15.92:0/771239308 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa7b401ec20 con 0x7fa7bc159ad0 2023-10-06T10:38:36.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.557+0000 7fa7b37fe700 1 --2- 172.21.15.92:0/771239308 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa79c066f30 0x7fa79c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:36.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.557+0000 7fa7b37fe700 1 -- 172.21.15.92:0/771239308 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa7b40864d0 con 0x7fa7bc159ad0 2023-10-06T10:38:36.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.557+0000 7fa7bad9d700 1 --2- 172.21.15.92:0/771239308 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa79c066f30 0x7fa79c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:36.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.557+0000 7fa7bad9d700 1 --2- 172.21.15.92:0/771239308 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa79c066f30 0x7fa79c0693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac007fd0 tx=0x7fa7ac007e30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:36.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.561+0000 7fa7b37fe700 1 -- 172.21.15.92:0/771239308 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7b40532f0 con 0x7fa7bc159ad0 2023-10-06T10:38:36.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.785+0000 7fa7c1030700 1 -- 172.21.15.92:0/771239308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa7bc064a90 con 0x7fa7bc159ad0 2023-10-06T10:38:36.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.785+0000 7fa7b37fe700 1 -- 172.21.15.92:0/771239308 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa7b4056910 con 0x7fa7bc159ad0 2023-10-06T10:38:36.787 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:36.787 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T10:38:36.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.787+0000 7fa7c1030700 1 -- 172.21.15.92:0/771239308 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa79c066f30 msgr2=0x7fa79c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:36.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.787+0000 7fa7c1030700 1 --2- 172.21.15.92:0/771239308 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa79c066f30 0x7fa79c0693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac007fd0 tx=0x7fa7ac007e30 comp rx=0 tx=0).stop 2023-10-06T10:38:36.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.787+0000 7fa7c1030700 1 -- 172.21.15.92:0/771239308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc159ad0 msgr2=0x7fa7bc0c6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:36.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.787+0000 7fa7c1030700 1 --2- 172.21.15.92:0/771239308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc159ad0 0x7fa7bc0c6080 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009880 tx=0x7fa7b4017000 comp rx=0 tx=0).stop 2023-10-06T10:38:36.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.787+0000 7fa7c1030700 1 -- 172.21.15.92:0/771239308 shutdown_connections 2023-10-06T10:38:36.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.787+0000 7fa7c1030700 1 --2- 172.21.15.92:0/771239308 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa79c066f30 0x7fa79c0693e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.787+0000 7fa7c1030700 1 --2- 172.21.15.92:0/771239308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc159ad0 0x7fa7bc0c6080 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.787+0000 7fa7c1030700 1 --2- 172.21.15.92:0/771239308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7bc0cad80 0x7fa7bc158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.787+0000 7fa7c1030700 1 --2- 172.21.15.92:0/771239308 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7bc0cb5f0 0x7fa7bc0c29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.787+0000 7fa7c1030700 1 -- 172.21.15.92:0/771239308 >> 172.21.15.92:0/771239308 conn(0x7fa7bc0707d0 msgr2=0x7fa7bc1529b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:36.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.787+0000 7fa7c1030700 1 -- 172.21.15.92:0/771239308 shutdown_connections 2023-10-06T10:38:36.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.787+0000 7fa7c1030700 1 -- 172.21.15.92:0/771239308 wait complete. 2023-10-06T10:38:36.801 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-06T10:38:36.801 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:38:36.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.913+0000 7fac796e8700 1 Processor -- start 2023-10-06T10:38:36.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.913+0000 7fac796e8700 1 -- start start 2023-10-06T10:38:36.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.914+0000 7fac796e8700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac740ca370 0x7fac74158270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:36.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.914+0000 7fac796e8700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fac740ce8a0 0x7fac7415aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:36.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.914+0000 7fac796e8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac74157de0 con 0x7fac740adb60 2023-10-06T10:38:36.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.914+0000 7fac796e8700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac740cbb90 con 0x7fac740ce8a0 2023-10-06T10:38:36.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.914+0000 7fac796e8700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac740cb5d0 con 0x7fac740ca370 2023-10-06T10:38:36.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.914+0000 7fac6a7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fac740adb60 0x7fac740ce190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40132/0 (socket says 172.21.15.92:40132) 2023-10-06T10:38:36.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.914+0000 7fac6a7fc700 1 -- 172.21.15.92:0/4012316006 learned_addr learned my addr 172.21.15.92:0/4012316006 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:36.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac737fe700 1 --2- 172.21.15.92:0/4012316006 >> v2:172.21.15.124:3300/0 conn(0x7fac740ce8a0 0x7fac7415aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:36.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac727fc700 1 -- 172.21.15.92:0/4012316006 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104678342 0 0) 0x7fac74157de0 con 0x7fac740adb60 2023-10-06T10:38:36.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac72ffd700 1 --2- 172.21.15.92:0/4012316006 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac740ca370 0x7fac74158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:36.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac727fc700 1 -- 172.21.15.92:0/4012316006 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac60003570 con 0x7fac740adb60 2023-10-06T10:38:36.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac727fc700 1 -- 172.21.15.92:0/4012316006 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2794420763 0 0) 0x7fac60003570 con 0x7fac740adb60 2023-10-06T10:38:36.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac727fc700 1 -- 172.21.15.92:0/4012316006 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac740ca370 msgr2=0x7fac74158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:36.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac727fc700 1 --2- 172.21.15.92:0/4012316006 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac740ca370 0x7fac74158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac727fc700 1 -- 172.21.15.92:0/4012316006 >> v2:172.21.15.124:3300/0 conn(0x7fac740ce8a0 msgr2=0x7fac7415aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:36.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac727fc700 1 --2- 172.21.15.92:0/4012316006 >> v2:172.21.15.124:3300/0 conn(0x7fac740ce8a0 0x7fac7415aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac727fc700 1 -- 172.21.15.92:0/4012316006 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac740cb8b0 con 0x7fac740adb60 2023-10-06T10:38:36.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac727fc700 1 -- 172.21.15.92:0/4012316006 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fac5c001470 con 0x7fac740adb60 2023-10-06T10:38:36.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac727fc700 1 -- 172.21.15.92:0/4012316006 >> v1:172.21.15.92:6789/0 conn(0x7fac740adb60 legacy=0x7fac740ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:36.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac727fc700 1 --2- 172.21.15.92:0/4012316006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac60003a60 0x7fac60003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:36.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac727fc700 1 --2- 172.21.15.92:0/4012316006 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac740ca370 0x7fac60004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:36.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac727fc700 1 --2- 172.21.15.92:0/4012316006 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac60004d50 0x7fac600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:36.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac727fc700 1 -- 172.21.15.92:0/4012316006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac740cbb90 con 0x7fac60003a60 2023-10-06T10:38:36.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac727fc700 1 -- 172.21.15.92:0/4012316006 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac740cb5d0 con 0x7fac60004d50 2023-10-06T10:38:36.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac727fc700 1 -- 172.21.15.92:0/4012316006 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac6000ba60 con 0x7fac740ca370 2023-10-06T10:38:36.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac72ffd700 1 --2- 172.21.15.92:0/4012316006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac60003a60 0x7fac60003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:36.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac6a7fc700 1 --2- 172.21.15.92:0/4012316006 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac740ca370 0x7fac60004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:36.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac737fe700 1 --2- 172.21.15.92:0/4012316006 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac60004d50 0x7fac600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:36.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac72ffd700 1 -- 172.21.15.92:0/4012316006 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac740ca370 msgr2=0x7fac60004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:36.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac72ffd700 1 --2- 172.21.15.92:0/4012316006 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac740ca370 0x7fac60004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac72ffd700 1 -- 172.21.15.92:0/4012316006 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac60004d50 msgr2=0x7fac600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:36.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac72ffd700 1 --2- 172.21.15.92:0/4012316006 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac60004d50 0x7fac600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac72ffd700 1 -- 172.21.15.92:0/4012316006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac60003570 con 0x7fac60003a60 2023-10-06T10:38:36.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.915+0000 7fac6a7fc700 1 --2- 172.21.15.92:0/4012316006 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac740ca370 0x7fac60004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:36.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.916+0000 7fac72ffd700 1 --2- 172.21.15.92:0/4012316006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac60003a60 0x7fac60003e70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fac64009c00 tx=0x7fac640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67fd8b10cd75572e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:36.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.916+0000 7fac727fc700 1 -- 172.21.15.92:0/4012316006 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6400e6b0 con 0x7fac60003a60 2023-10-06T10:38:36.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.916+0000 7fac727fc700 1 -- 172.21.15.92:0/4012316006 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6400e810 con 0x7fac60003a60 2023-10-06T10:38:36.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.916+0000 7fac727fc700 1 -- 172.21.15.92:0/4012316006 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6400ebd0 con 0x7fac60003a60 2023-10-06T10:38:36.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.916+0000 7fac796e8700 1 -- 172.21.15.92:0/4012316006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac60003a60 msgr2=0x7fac60003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:36.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.916+0000 7fac796e8700 1 --2- 172.21.15.92:0/4012316006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac60003a60 0x7fac60003e70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fac64009c00 tx=0x7fac640091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.916+0000 7fac737fe700 1 -- 172.21.15.92:0/4012316006 reap_dead start 2023-10-06T10:38:36.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.916+0000 7fac796e8700 1 -- 172.21.15.92:0/4012316006 shutdown_connections 2023-10-06T10:38:36.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.916+0000 7fac796e8700 1 -- 172.21.15.92:0/4012316006 >> 172.21.15.92:0/4012316006 conn(0x7fac740707d0 msgr2=0x7fac74070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:36.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.916+0000 7fac796e8700 1 -- 172.21.15.92:0/4012316006 shutdown_connections 2023-10-06T10:38:36.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.916+0000 7fac796e8700 1 -- 172.21.15.92:0/4012316006 wait complete. 2023-10-06T10:38:36.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.918+0000 7fac796e8700 1 Processor -- start 2023-10-06T10:38:36.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.918+0000 7fac796e8700 1 -- start start 2023-10-06T10:38:36.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.918+0000 7fac796e8700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac60003a60 0x7fac740c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:36.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.918+0000 7fac796e8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac7416cee0 0x7fac741485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:36.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.918+0000 7fac796e8700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac740c96f0 0x7fac740c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:36.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.918+0000 7fac796e8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac740ca0b0 con 0x7fac7416cee0 2023-10-06T10:38:36.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.918+0000 7fac796e8700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7416d450 con 0x7fac740c96f0 2023-10-06T10:38:36.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.918+0000 7fac796e8700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac7416d730 con 0x7fac60003a60 2023-10-06T10:38:36.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.918+0000 7fac6a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac7416cee0 0x7fac741485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:36.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.918+0000 7fac6a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac7416cee0 0x7fac741485e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36930/0 (socket says 172.21.15.92:36930) 2023-10-06T10:38:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.918+0000 7fac6a7fc700 1 -- 172.21.15.92:0/2518314607 learned_addr learned my addr 172.21.15.92:0/2518314607 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.918+0000 7fac737fe700 1 --2- 172.21.15.92:0/2518314607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac740c96f0 0x7fac740c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.918+0000 7fac72ffd700 1 --2- 172.21.15.92:0/2518314607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac60003a60 0x7fac740c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.919+0000 7fac6a7fc700 1 -- 172.21.15.92:0/2518314607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac60003a60 msgr2=0x7fac740c44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.919+0000 7fac6a7fc700 1 --2- 172.21.15.92:0/2518314607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac60003a60 0x7fac740c44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.919+0000 7fac6a7fc700 1 -- 172.21.15.92:0/2518314607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac740c96f0 msgr2=0x7fac740c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:36.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.919+0000 7fac6a7fc700 1 --2- 172.21.15.92:0/2518314607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac740c96f0 0x7fac740c9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:36.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.919+0000 7fac6a7fc700 1 -- 172.21.15.92:0/2518314607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7416da10 con 0x7fac7416cee0 2023-10-06T10:38:36.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.919+0000 7fac6a7fc700 1 --2- 172.21.15.92:0/2518314607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac7416cee0 0x7fac741485e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0015e0 tx=0x7fac5c001570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:36.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.919+0000 7fac6b7fe700 1 -- 172.21.15.92:0/2518314607 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c00ee70 con 0x7fac7416cee0 2023-10-06T10:38:36.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.919+0000 7fac6b7fe700 1 -- 172.21.15.92:0/2518314607 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5c00c730 con 0x7fac7416cee0 2023-10-06T10:38:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.919+0000 7fac796e8700 1 -- 172.21.15.92:0/2518314607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac7416dcf0 con 0x7fac7416cee0 2023-10-06T10:38:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.919+0000 7fac6b7fe700 1 -- 172.21.15.92:0/2518314607 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c00cad0 con 0x7fac7416cee0 2023-10-06T10:38:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.919+0000 7fac796e8700 1 -- 172.21.15.92:0/2518314607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac740c4a10 con 0x7fac7416cee0 2023-10-06T10:38:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.920+0000 7fac796e8700 1 -- 172.21.15.92:0/2518314607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac740689a0 con 0x7fac7416cee0 2023-10-06T10:38:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.920+0000 7fac6b7fe700 1 -- 172.21.15.92:0/2518314607 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fac5c01d460 con 0x7fac7416cee0 2023-10-06T10:38:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.921+0000 7fac6b7fe700 1 --2- 172.21.15.92:0/2518314607 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fac54066f30 0x7fac540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.921+0000 7fac6b7fe700 1 -- 172.21.15.92:0/2518314607 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fac5c0848a0 con 0x7fac7416cee0 2023-10-06T10:38:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.921+0000 7fac72ffd700 1 --2- 172.21.15.92:0/2518314607 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fac54066f30 0x7fac540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.921+0000 7fac72ffd700 1 --2- 172.21.15.92:0/2518314607 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fac54066f30 0x7fac540693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fac7415aa30 tx=0x7fac6401a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:36.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:36.925+0000 7fac6b7fe700 1 -- 172.21.15.92:0/2518314607 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac740689a0 con 0x7fac7416cee0 2023-10-06T10:38:37.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.109+0000 7fac796e8700 1 -- 172.21.15.92:0/2518314607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac740689a0 con 0x7fac7416cee0 2023-10-06T10:38:37.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.110+0000 7fac6b7fe700 1 -- 172.21.15.92:0/2518314607 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18707 (secure 0 0 0) 0x7fac740689a0 con 0x7fac7416cee0 2023-10-06T10:38:37.112 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:37.112 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:21.760233+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:38:14.368713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T10:38:15.692344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T10:38:19.382553+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:38:37.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.111+0000 7fac796e8700 1 -- 172.21.15.92:0/2518314607 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fac54066f30 msgr2=0x7fac540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:37.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.111+0000 7fac796e8700 1 --2- 172.21.15.92:0/2518314607 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fac54066f30 0x7fac540693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fac7415aa30 tx=0x7fac6401a000 comp rx=0 tx=0).stop 2023-10-06T10:38:37.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.111+0000 7fac796e8700 1 -- 172.21.15.92:0/2518314607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac7416cee0 msgr2=0x7fac741485e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:37.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.111+0000 7fac796e8700 1 --2- 172.21.15.92:0/2518314607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac7416cee0 0x7fac741485e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0015e0 tx=0x7fac5c001570 comp rx=0 tx=0).stop 2023-10-06T10:38:37.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.112+0000 7fac796e8700 1 -- 172.21.15.92:0/2518314607 shutdown_connections 2023-10-06T10:38:37.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.112+0000 7fac796e8700 1 --2- 172.21.15.92:0/2518314607 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fac54066f30 0x7fac540693e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.112+0000 7fac796e8700 1 --2- 172.21.15.92:0/2518314607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac740c96f0 0x7fac740c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.112+0000 7fac796e8700 1 --2- 172.21.15.92:0/2518314607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac60003a60 0x7fac740c44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.112+0000 7fac796e8700 1 --2- 172.21.15.92:0/2518314607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac7416cee0 0x7fac741485e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.112+0000 7fac796e8700 1 -- 172.21.15.92:0/2518314607 >> 172.21.15.92:0/2518314607 conn(0x7fac740707d0 msgr2=0x7fac741534e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:37.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.112+0000 7fac796e8700 1 -- 172.21.15.92:0/2518314607 shutdown_connections 2023-10-06T10:38:37.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.112+0000 7fac796e8700 1 -- 172.21.15.92:0/2518314607 wait complete. 2023-10-06T10:38:37.126 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T10:38:09.680559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T10:38:10.120786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T10:38:14.368713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T10:38:15.692344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T10:38:19.382553+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T10:38:37.128 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T10:38:37.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.243+0000 7ff7835b9700 1 Processor -- start 2023-10-06T10:38:37.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.243+0000 7ff7835b9700 1 -- start start 2023-10-06T10:38:37.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.244+0000 7ff7835b9700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff77c0cbc70 0x7ff77c0ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:37.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.244+0000 7ff7835b9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77c0cf5d0 con 0x7ff77c1598a0 2023-10-06T10:38:37.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.244+0000 7ff7835b9700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77c0cf8b0 con 0x7ff77c1594d0 2023-10-06T10:38:37.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.244+0000 7ff7835b9700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff77c0cfb90 con 0x7ff77c0cbc70 2023-10-06T10:38:37.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.244+0000 7ff780b54700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff77c1598a0 0x7ff77c0ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40148/0 (socket says 172.21.15.92:40148) 2023-10-06T10:38:37.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.244+0000 7ff780b54700 1 -- 172.21.15.92:0/1560965281 learned_addr learned my addr 172.21.15.92:0/1560965281 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:37.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.244+0000 7ff781b56700 1 --2- 172.21.15.92:0/1560965281 >> v2:172.21.15.124:3300/0 conn(0x7ff77c0cbc70 0x7ff77c0ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:37.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.244+0000 7ff773fff700 1 -- 172.21.15.92:0/1560965281 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3538208186 0 0) 0x7ff77c0cf5d0 con 0x7ff77c1598a0 2023-10-06T10:38:37.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff773fff700 1 -- 172.21.15.92:0/1560965281 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff768003570 con 0x7ff77c1598a0 2023-10-06T10:38:37.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff773fff700 1 -- 172.21.15.92:0/1560965281 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 213022819 0 0) 0x7ff768003570 con 0x7ff77c1598a0 2023-10-06T10:38:37.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff773fff700 1 -- 172.21.15.92:0/1560965281 >> v2:172.21.15.124:3300/0 conn(0x7ff77c0cbc70 msgr2=0x7ff77c0ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:37.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff773fff700 1 --2- 172.21.15.92:0/1560965281 >> v2:172.21.15.124:3300/0 conn(0x7ff77c0cbc70 0x7ff77c0ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff773fff700 1 -- 172.21.15.92:0/1560965281 >> v1:172.21.15.124:6789/0 conn(0x7ff77c1594d0 legacy=0x7ff77c0cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:37.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff773fff700 1 -- 172.21.15.92:0/1560965281 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c0cfe70 con 0x7ff77c1598a0 2023-10-06T10:38:37.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff773fff700 1 -- 172.21.15.92:0/1560965281 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7ff76c001470 con 0x7ff77c1598a0 2023-10-06T10:38:37.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff773fff700 1 -- 172.21.15.92:0/1560965281 >> v1:172.21.15.92:6789/0 conn(0x7ff77c1598a0 legacy=0x7ff77c0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:37.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff773fff700 1 --2- 172.21.15.92:0/1560965281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768003a60 0x7ff768003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:37.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff773fff700 1 --2- 172.21.15.92:0/1560965281 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff768004810 0x7ff768004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:37.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff773fff700 1 --2- 172.21.15.92:0/1560965281 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7680051c0 0x7ff76800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:37.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff773fff700 1 -- 172.21.15.92:0/1560965281 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c0cfb90 con 0x7ff768003a60 2023-10-06T10:38:37.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff773fff700 1 -- 172.21.15.92:0/1560965281 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76800c0d0 con 0x7ff768004810 2023-10-06T10:38:37.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff773fff700 1 -- 172.21.15.92:0/1560965281 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76800c290 con 0x7ff7680051c0 2023-10-06T10:38:37.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff781355700 1 --2- 172.21.15.92:0/1560965281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768003a60 0x7ff768003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:37.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff780b54700 1 --2- 172.21.15.92:0/1560965281 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff768004810 0x7ff768004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:37.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff781b56700 1 --2- 172.21.15.92:0/1560965281 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7680051c0 0x7ff76800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:37.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff781355700 1 -- 172.21.15.92:0/1560965281 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7680051c0 msgr2=0x7ff76800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:37.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff781355700 1 --2- 172.21.15.92:0/1560965281 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7680051c0 0x7ff76800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff781355700 1 -- 172.21.15.92:0/1560965281 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff768004810 msgr2=0x7ff768004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:37.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff781355700 1 --2- 172.21.15.92:0/1560965281 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff768004810 0x7ff768004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff781355700 1 -- 172.21.15.92:0/1560965281 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff768003570 con 0x7ff768003a60 2023-10-06T10:38:37.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff781b56700 1 --2- 172.21.15.92:0/1560965281 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7680051c0 0x7ff76800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:37.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff781b56700 1 -- 172.21.15.92:0/1560965281 reap_dead start 2023-10-06T10:38:37.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff781355700 1 --2- 172.21.15.92:0/1560965281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768003a60 0x7ff768003e70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff7780018b0 tx=0x7ff778007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3acc34c97e4033a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:37.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff773fff700 1 -- 172.21.15.92:0/1560965281 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff778012420 con 0x7ff768003a60 2023-10-06T10:38:37.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff773fff700 1 -- 172.21.15.92:0/1560965281 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7780125a0 con 0x7ff768003a60 2023-10-06T10:38:37.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.245+0000 7ff773fff700 1 -- 172.21.15.92:0/1560965281 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff778012960 con 0x7ff768003a60 2023-10-06T10:38:37.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.246+0000 7ff7835b9700 1 -- 172.21.15.92:0/1560965281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768003a60 msgr2=0x7ff768003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:37.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.246+0000 7ff7835b9700 1 --2- 172.21.15.92:0/1560965281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768003a60 0x7ff768003e70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff7780018b0 tx=0x7ff778007d30 comp rx=0 tx=0).stop 2023-10-06T10:38:37.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.246+0000 7ff7835b9700 1 -- 172.21.15.92:0/1560965281 shutdown_connections 2023-10-06T10:38:37.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.246+0000 7ff7835b9700 1 --2- 172.21.15.92:0/1560965281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768003a60 0x7ff768003e70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.246+0000 7ff7835b9700 1 -- 172.21.15.92:0/1560965281 >> 172.21.15.92:0/1560965281 conn(0x7ff77c089e50 msgr2=0x7ff77c099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:37.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.246+0000 7ff7835b9700 1 -- 172.21.15.92:0/1560965281 shutdown_connections 2023-10-06T10:38:37.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.246+0000 7ff7835b9700 1 -- 172.21.15.92:0/1560965281 wait complete. 2023-10-06T10:38:37.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.247+0000 7ff7835b9700 1 Processor -- start 2023-10-06T10:38:37.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.247+0000 7ff7835b9700 1 -- start start 2023-10-06T10:38:37.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.248+0000 7ff7835b9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768003a60 0x7ff77c15b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:37.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.248+0000 7ff7835b9700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff77c159ca0 0x7ff77c16d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:37.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.248+0000 7ff7835b9700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff77c1679a0 0x7ff77c16dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:37.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.248+0000 7ff7835b9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c15ad20 con 0x7ff768003a60 2023-10-06T10:38:37.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.248+0000 7ff7835b9700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c15a110 con 0x7ff77c1679a0 2023-10-06T10:38:37.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.248+0000 7ff7835b9700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c15a3f0 con 0x7ff77c159ca0 2023-10-06T10:38:37.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.248+0000 7ff781355700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768003a60 0x7ff77c15b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:37.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.248+0000 7ff781355700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768003a60 0x7ff77c15b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36940/0 (socket says 172.21.15.92:36940) 2023-10-06T10:38:37.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.248+0000 7ff781355700 1 -- 172.21.15.92:0/2138762969 learned_addr learned my addr 172.21.15.92:0/2138762969 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:37.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.248+0000 7ff781b56700 1 --2- 172.21.15.92:0/2138762969 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff77c1679a0 0x7ff77c16dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:37.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.248+0000 7ff780b54700 1 --2- 172.21.15.92:0/2138762969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff77c159ca0 0x7ff77c16d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:37.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.248+0000 7ff781355700 1 -- 172.21.15.92:0/2138762969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff77c159ca0 msgr2=0x7ff77c16d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:37.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.248+0000 7ff781355700 1 --2- 172.21.15.92:0/2138762969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff77c159ca0 0x7ff77c16d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.248+0000 7ff781355700 1 -- 172.21.15.92:0/2138762969 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff77c1679a0 msgr2=0x7ff77c16dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:37.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.248+0000 7ff781355700 1 --2- 172.21.15.92:0/2138762969 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff77c1679a0 0x7ff77c16dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.248+0000 7ff781355700 1 -- 172.21.15.92:0/2138762969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c15a6d0 con 0x7ff768003a60 2023-10-06T10:38:37.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.248+0000 7ff780b54700 1 --2- 172.21.15.92:0/2138762969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff77c159ca0 0x7ff77c16d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:37.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.249+0000 7ff781355700 1 --2- 172.21.15.92:0/2138762969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768003a60 0x7ff77c15b1b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff77800c3f0 tx=0x7ff778015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:37.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.249+0000 7ff7717fa700 1 -- 172.21.15.92:0/2138762969 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff778001200 con 0x7ff768003a60 2023-10-06T10:38:37.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.249+0000 7ff7717fa700 1 -- 172.21.15.92:0/2138762969 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff778012cd0 con 0x7ff768003a60 2023-10-06T10:38:37.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.249+0000 7ff7835b9700 1 -- 172.21.15.92:0/2138762969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77c15a9b0 con 0x7ff768003a60 2023-10-06T10:38:37.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.249+0000 7ff7717fa700 1 -- 172.21.15.92:0/2138762969 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff778012940 con 0x7ff768003a60 2023-10-06T10:38:37.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.249+0000 7ff7835b9700 1 -- 172.21.15.92:0/2138762969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77c16b550 con 0x7ff768003a60 2023-10-06T10:38:37.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.250+0000 7ff7835b9700 1 -- 172.21.15.92:0/2138762969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff77c15b000 con 0x7ff768003a60 2023-10-06T10:38:37.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.250+0000 7ff7717fa700 1 -- 172.21.15.92:0/2138762969 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff77801f620 con 0x7ff768003a60 2023-10-06T10:38:37.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.250+0000 7ff7717fa700 1 --2- 172.21.15.92:0/2138762969 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff760066f30 0x7ff7600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:37.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.251+0000 7ff7717fa700 1 -- 172.21.15.92:0/2138762969 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ff778086440 con 0x7ff768003a60 2023-10-06T10:38:37.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.251+0000 7ff780b54700 1 --2- 172.21.15.92:0/2138762969 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff760066f30 0x7ff7600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:37.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.251+0000 7ff780b54700 1 --2- 172.21.15.92:0/2138762969 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff760066f30 0x7ff7600693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff76c001ce0 tx=0x7ff76c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:37.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.255+0000 7ff7717fa700 1 -- 172.21.15.92:0/2138762969 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff778053260 con 0x7ff768003a60 2023-10-06T10:38:37.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.439+0000 7ff7835b9700 1 -- 172.21.15.92:0/2138762969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff77c064a90 con 0x7ff768003a60 2023-10-06T10:38:37.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.439+0000 7ff7717fa700 1 -- 172.21.15.92:0/2138762969 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff778056880 con 0x7ff768003a60 2023-10-06T10:38:37.441 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T10:38:37.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.441+0000 7ff7835b9700 1 -- 172.21.15.92:0/2138762969 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff760066f30 msgr2=0x7ff7600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:37.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.441+0000 7ff7835b9700 1 --2- 172.21.15.92:0/2138762969 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff760066f30 0x7ff7600693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff76c001ce0 tx=0x7ff76c009040 comp rx=0 tx=0).stop 2023-10-06T10:38:37.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.441+0000 7ff7835b9700 1 -- 172.21.15.92:0/2138762969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768003a60 msgr2=0x7ff77c15b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:37.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.441+0000 7ff7835b9700 1 --2- 172.21.15.92:0/2138762969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768003a60 0x7ff77c15b1b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff77800c3f0 tx=0x7ff778015000 comp rx=0 tx=0).stop 2023-10-06T10:38:37.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.441+0000 7ff7835b9700 1 -- 172.21.15.92:0/2138762969 shutdown_connections 2023-10-06T10:38:37.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.441+0000 7ff7835b9700 1 --2- 172.21.15.92:0/2138762969 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff760066f30 0x7ff7600693e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.441+0000 7ff7835b9700 1 --2- 172.21.15.92:0/2138762969 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff77c1679a0 0x7ff77c16dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.441+0000 7ff7835b9700 1 --2- 172.21.15.92:0/2138762969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768003a60 0x7ff77c15b1b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.441+0000 7ff7835b9700 1 --2- 172.21.15.92:0/2138762969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff77c159ca0 0x7ff77c16d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.441+0000 7ff7835b9700 1 -- 172.21.15.92:0/2138762969 >> 172.21.15.92:0/2138762969 conn(0x7ff77c089e50 msgr2=0x7ff77c0ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:37.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.441+0000 7ff7835b9700 1 -- 172.21.15.92:0/2138762969 shutdown_connections 2023-10-06T10:38:37.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.441+0000 7ff7835b9700 1 -- 172.21.15.92:0/2138762969 wait complete. 2023-10-06T10:38:37.455 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T10:38:37.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.570+0000 7f43f694b700 1 Processor -- start 2023-10-06T10:38:37.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.570+0000 7f43f694b700 1 -- start start 2023-10-06T10:38:37.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.571+0000 7f43f694b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f43f01539c0 0x7f43f0069550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:37.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.571+0000 7f43f694b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f43f0069a90 0x7f43f015d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:37.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.571+0000 7f43f694b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f015db90 con 0x7f43f0153d90 2023-10-06T10:38:37.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.571+0000 7f43f694b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f015ab70 con 0x7f43f0069a90 2023-10-06T10:38:37.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.571+0000 7f43f694b700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f015de70 con 0x7f43f01539c0 2023-10-06T10:38:37.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.571+0000 7f43ef7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f43f0069a90 0x7f43f015d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:37.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.571+0000 7f43f4ee8700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f43f0153d90 0x7f43f00cbe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40162/0 (socket says 172.21.15.92:40162) 2023-10-06T10:38:37.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.571+0000 7f43f4ee8700 1 -- 172.21.15.92:0/1947247388 learned_addr learned my addr 172.21.15.92:0/1947247388 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:37.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.571+0000 7f43ef7fe700 1 --2- 172.21.15.92:0/1947247388 >> v2:172.21.15.92:3300/0 conn(0x7f43f0069a90 0x7f43f015d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36944/0 (socket says 172.21.15.92:36944) 2023-10-06T10:38:37.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.571+0000 7f43effff700 1 --2- 172.21.15.92:0/1947247388 >> v2:172.21.15.124:3300/0 conn(0x7f43f01539c0 0x7f43f0069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:37.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.572+0000 7f43ef7fe700 1 -- 172.21.15.92:0/1947247388 >> v2:172.21.15.124:3300/0 conn(0x7f43f01539c0 msgr2=0x7f43f0069550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:37.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.572+0000 7f43ef7fe700 1 --2- 172.21.15.92:0/1947247388 >> v2:172.21.15.124:3300/0 conn(0x7f43f01539c0 0x7f43f0069550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.572+0000 7f43ef7fe700 1 -- 172.21.15.92:0/1947247388 >> v1:172.21.15.92:6789/0 conn(0x7f43f0153d90 legacy=0x7f43f00cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:37.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.572+0000 7f43ef7fe700 1 -- 172.21.15.92:0/1947247388 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f0068d50 con 0x7f43f0069a90 2023-10-06T10:38:37.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.572+0000 7f43eeffd700 1 -- 172.21.15.92:0/1947247388 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2214179660 0 0) 0x7f43f015db90 con 0x7f43f0153d90 2023-10-06T10:38:37.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.572+0000 7f43ef7fe700 1 --2- 172.21.15.92:0/1947247388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43f0069a90 0x7f43f015d110 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f43f006af20 tx=0x7f43e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c07114457e1bbc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:37.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.572+0000 7f43eeffd700 1 -- 172.21.15.92:0/1947247388 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e4011630 con 0x7f43f0069a90 2023-10-06T10:38:37.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.572+0000 7f43eeffd700 1 -- 172.21.15.92:0/1947247388 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e4011790 con 0x7f43f0069a90 2023-10-06T10:38:37.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.572+0000 7f43eeffd700 1 -- 172.21.15.92:0/1947247388 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e4011b30 con 0x7f43f0069a90 2023-10-06T10:38:37.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.572+0000 7f43f694b700 1 -- 172.21.15.92:0/1947247388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43f0069a90 msgr2=0x7f43f015d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:37.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.572+0000 7f43f694b700 1 --2- 172.21.15.92:0/1947247388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43f0069a90 0x7f43f015d110 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f43f006af20 tx=0x7f43e40091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.572+0000 7f43f694b700 1 -- 172.21.15.92:0/1947247388 shutdown_connections 2023-10-06T10:38:37.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.572+0000 7f43f694b700 1 --2- 172.21.15.92:0/1947247388 >> v2:172.21.15.124:3300/0 conn(0x7f43f01539c0 0x7f43f0069550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.572+0000 7f43f694b700 1 --2- 172.21.15.92:0/1947247388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43f0069a90 0x7f43f015d110 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.572+0000 7f43f694b700 1 -- 172.21.15.92:0/1947247388 >> 172.21.15.92:0/1947247388 conn(0x7f43f006b610 msgr2=0x7f43f006ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:37.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.573+0000 7f43f694b700 1 -- 172.21.15.92:0/1947247388 shutdown_connections 2023-10-06T10:38:37.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.573+0000 7f43f694b700 1 -- 172.21.15.92:0/1947247388 wait complete. 2023-10-06T10:38:37.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.574+0000 7f43f694b700 1 Processor -- start 2023-10-06T10:38:37.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.574+0000 7f43f694b700 1 -- start start 2023-10-06T10:38:37.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.574+0000 7f43f694b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43f0069a90 0x7f43f016a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:37.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.574+0000 7f43f694b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43f01539c0 0x7f43f0147cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:37.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.575+0000 7f43f694b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43f0153d90 0x7f43f015c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:37.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.575+0000 7f43f694b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f016a390 con 0x7f43f0069a90 2023-10-06T10:38:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.575+0000 7f43f694b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f0169d80 con 0x7f43f01539c0 2023-10-06T10:38:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.575+0000 7f43f694b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f00b1e20 con 0x7f43f0153d90 2023-10-06T10:38:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.575+0000 7f43effff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43f0069a90 0x7f43f016a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.575+0000 7f43effff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43f0069a90 0x7f43f016a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36956/0 (socket says 172.21.15.92:36956) 2023-10-06T10:38:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.575+0000 7f43effff700 1 -- 172.21.15.92:0/3155823741 learned_addr learned my addr 172.21.15.92:0/3155823741 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.575+0000 7f43ef7fe700 1 --2- 172.21.15.92:0/3155823741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43f01539c0 0x7f43f0147cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.575+0000 7f43f4ee8700 1 --2- 172.21.15.92:0/3155823741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43f0153d90 0x7f43f015c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.575+0000 7f43effff700 1 -- 172.21.15.92:0/3155823741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43f0153d90 msgr2=0x7f43f015c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.575+0000 7f43effff700 1 --2- 172.21.15.92:0/3155823741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43f0153d90 0x7f43f015c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.575+0000 7f43effff700 1 -- 172.21.15.92:0/3155823741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43f01539c0 msgr2=0x7f43f0147cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.575+0000 7f43effff700 1 --2- 172.21.15.92:0/3155823741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43f01539c0 0x7f43f0147cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.575+0000 7f43effff700 1 -- 172.21.15.92:0/3155823741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e4009420 con 0x7f43f0069a90 2023-10-06T10:38:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.575+0000 7f43effff700 1 --2- 172.21.15.92:0/3155823741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43f0069a90 0x7f43f016a820 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f43e000ec80 tx=0x7f43e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.575+0000 7f43d7fff700 1 -- 172.21.15.92:0/3155823741 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e000c8e0 con 0x7f43f0069a90 2023-10-06T10:38:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.575+0000 7f43d7fff700 1 -- 172.21.15.92:0/3155823741 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e000ca40 con 0x7f43f0069a90 2023-10-06T10:38:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.575+0000 7f43f694b700 1 -- 172.21.15.92:0/3155823741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f016a020 con 0x7f43f0069a90 2023-10-06T10:38:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.575+0000 7f43d7fff700 1 -- 172.21.15.92:0/3155823741 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e000cde0 con 0x7f43f0069a90 2023-10-06T10:38:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.575+0000 7f43f694b700 1 -- 172.21.15.92:0/3155823741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f016ad60 con 0x7f43f0069a90 2023-10-06T10:38:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.576+0000 7f43f694b700 1 -- 172.21.15.92:0/3155823741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f0064a90 con 0x7f43f0069a90 2023-10-06T10:38:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.577+0000 7f43d7fff700 1 -- 172.21.15.92:0/3155823741 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f43e0013680 con 0x7f43f0069a90 2023-10-06T10:38:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.577+0000 7f43d7fff700 1 --2- 172.21.15.92:0/3155823741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f43dc066f30 0x7f43dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.577+0000 7f43d7fff700 1 -- 172.21.15.92:0/3155823741 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f43e0084c10 con 0x7f43f0069a90 2023-10-06T10:38:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.577+0000 7f43ef7fe700 1 --2- 172.21.15.92:0/3155823741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f43dc066f30 0x7f43dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.577+0000 7f43ef7fe700 1 --2- 172.21.15.92:0/3155823741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f43dc066f30 0x7f43dc0693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f43f00c5290 tx=0x7f43e4017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:37.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.581+0000 7f43d7fff700 1 -- 172.21.15.92:0/3155823741 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43e0051a30 con 0x7f43f0069a90 2023-10-06T10:38:37.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.762+0000 7f43f694b700 1 -- 172.21.15.92:0/3155823741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f43f004bb40 con 0x7f43f0069a90 2023-10-06T10:38:37.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.762+0000 7f43d7fff700 1 -- 172.21.15.92:0/3155823741 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f43e0055050 con 0x7f43f0069a90 2023-10-06T10:38:37.764 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T10:38:37.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.765+0000 7f43f694b700 1 -- 172.21.15.92:0/3155823741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f43dc066f30 msgr2=0x7f43dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:37.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.765+0000 7f43f694b700 1 --2- 172.21.15.92:0/3155823741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f43dc066f30 0x7f43dc0693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f43f00c5290 tx=0x7f43e4017000 comp rx=0 tx=0).stop 2023-10-06T10:38:37.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.765+0000 7f43f694b700 1 -- 172.21.15.92:0/3155823741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43f0069a90 msgr2=0x7f43f016a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:37.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.765+0000 7f43f694b700 1 --2- 172.21.15.92:0/3155823741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43f0069a90 0x7f43f016a820 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f43e000ec80 tx=0x7f43e00091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.765+0000 7f43f694b700 1 -- 172.21.15.92:0/3155823741 shutdown_connections 2023-10-06T10:38:37.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.765+0000 7f43f694b700 1 --2- 172.21.15.92:0/3155823741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f43dc066f30 0x7f43dc0693e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.765+0000 7f43f694b700 1 --2- 172.21.15.92:0/3155823741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43f0153d90 0x7f43f015c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.765+0000 7f43f694b700 1 --2- 172.21.15.92:0/3155823741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43f0069a90 0x7f43f016a820 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.765+0000 7f43f694b700 1 --2- 172.21.15.92:0/3155823741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43f01539c0 0x7f43f0147cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.765+0000 7f43f694b700 1 -- 172.21.15.92:0/3155823741 >> 172.21.15.92:0/3155823741 conn(0x7f43f006b610 msgr2=0x7f43f0159d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:37.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.765+0000 7f43f694b700 1 -- 172.21.15.92:0/3155823741 shutdown_connections 2023-10-06T10:38:37.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.765+0000 7f43f694b700 1 -- 172.21.15.92:0/3155823741 wait complete. 2023-10-06T10:38:37.779 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T10:38:37.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.892+0000 7f6aef54d700 1 Processor -- start 2023-10-06T10:38:37.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.893+0000 7f6aef54d700 1 -- start start 2023-10-06T10:38:37.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.893+0000 7f6aef54d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ae80cd940 0x7f6ae8159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:37.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.893+0000 7f6aef54d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6ae80b4a20 0x7f6ae80d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:37.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.894+0000 7f6aef54d700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae80b5ef0 con 0x7f6ae80d0810 2023-10-06T10:38:37.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.894+0000 7f6aef54d700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ae80b3600 con 0x7f6ae80b4a20 2023-10-06T10:38:37.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.894+0000 7f6aef54d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae80d0c50 con 0x7f6ae80cd940 2023-10-06T10:38:37.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.894+0000 7f6aecae8700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6ae80b4a20 0x7f6ae80d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:37.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.894+0000 7f6aed2e9700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ae80cd940 0x7f6ae8159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:37.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.894+0000 7f6aecae8700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6ae80b4a20 0x7f6ae80d02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:47614/0 (socket says 172.21.15.92:47614) 2023-10-06T10:38:37.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.894+0000 7f6aed2e9700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ae80cd940 0x7f6ae8159bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:52068/0 (socket says 172.21.15.92:52068) 2023-10-06T10:38:37.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.894+0000 7f6aecae8700 1 -- 172.21.15.92:0/2610394144 learned_addr learned my addr 172.21.15.92:0/2610394144 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:37.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.894+0000 7f6aecae8700 1 -- 172.21.15.92:0/2610394144 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ae80cd940 msgr2=0x7f6ae8159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:37.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.894+0000 7f6aecae8700 1 --2- 172.21.15.92:0/2610394144 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ae80cd940 0x7f6ae8159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.894+0000 7f6aecae8700 1 -- 172.21.15.92:0/2610394144 >> v1:172.21.15.124:6789/0 conn(0x7f6ae80d0810 legacy=0x7f6ae815ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:37.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.894+0000 7f6aecae8700 1 -- 172.21.15.92:0/2610394144 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae80b6380 con 0x7f6ae80b4a20 2023-10-06T10:38:37.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.895+0000 7f6aecae8700 1 --2- 172.21.15.92:0/2610394144 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ae80b4a20 0x7f6ae80d02d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80099b0 tx=0x7f6ad8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=401547b1e9d7d623 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:37.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.895+0000 7f6adffff700 1 -- 172.21.15.92:0/2610394144 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8009ea0 con 0x7f6ae80b4a20 2023-10-06T10:38:37.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.895+0000 7f6adffff700 1 -- 172.21.15.92:0/2610394144 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad800b860 con 0x7f6ae80b4a20 2023-10-06T10:38:37.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.895+0000 7f6adffff700 1 -- 172.21.15.92:0/2610394144 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad800bc00 con 0x7f6ae80b4a20 2023-10-06T10:38:37.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.895+0000 7f6aef54d700 1 -- 172.21.15.92:0/2610394144 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ae80b4a20 msgr2=0x7f6ae80d02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:37.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.895+0000 7f6aef54d700 1 --2- 172.21.15.92:0/2610394144 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ae80b4a20 0x7f6ae80d02d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80099b0 tx=0x7f6ad8009250 comp rx=0 tx=0).stop 2023-10-06T10:38:37.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.895+0000 7f6aef54d700 1 -- 172.21.15.92:0/2610394144 shutdown_connections 2023-10-06T10:38:37.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.895+0000 7f6aef54d700 1 --2- 172.21.15.92:0/2610394144 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ae80cd940 0x7f6ae8159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.895+0000 7f6aef54d700 1 --2- 172.21.15.92:0/2610394144 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ae80b4a20 0x7f6ae80d02d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.895+0000 7f6aef54d700 1 -- 172.21.15.92:0/2610394144 >> 172.21.15.92:0/2610394144 conn(0x7f6ae806ece0 msgr2=0x7f6ae806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:37.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.895+0000 7f6aef54d700 1 -- 172.21.15.92:0/2610394144 shutdown_connections 2023-10-06T10:38:37.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.895+0000 7f6aef54d700 1 -- 172.21.15.92:0/2610394144 wait complete. 2023-10-06T10:38:37.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.896+0000 7f6aef54d700 1 Processor -- start 2023-10-06T10:38:37.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.896+0000 7f6aef54d700 1 -- start start 2023-10-06T10:38:37.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.897+0000 7f6aef54d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ae80b4a20 0x7f6ae8156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:37.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.897+0000 7f6aef54d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ae80cd940 0x7f6ae81496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:37.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.897+0000 7f6aef54d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ae80d0810 0x7f6ae8151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:37.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.897+0000 7f6aef54d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae81569c0 con 0x7f6ae80d0810 2023-10-06T10:38:37.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.897+0000 7f6aef54d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae80b1180 con 0x7f6ae80cd940 2023-10-06T10:38:37.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.897+0000 7f6aef54d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae80b1460 con 0x7f6ae80b4a20 2023-10-06T10:38:37.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.897+0000 7f6aedaea700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ae80d0810 0x7f6ae8151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:37.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.897+0000 7f6aedaea700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ae80d0810 0x7f6ae8151b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36968/0 (socket says 172.21.15.92:36968) 2023-10-06T10:38:37.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.897+0000 7f6aedaea700 1 -- 172.21.15.92:0/4174516465 learned_addr learned my addr 172.21.15.92:0/4174516465 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:37.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.898+0000 7f6aecae8700 1 --2- 172.21.15.92:0/4174516465 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ae80cd940 0x7f6ae81496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:37.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.898+0000 7f6aed2e9700 1 --2- 172.21.15.92:0/4174516465 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ae80b4a20 0x7f6ae8156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:37.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.898+0000 7f6aedaea700 1 -- 172.21.15.92:0/4174516465 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ae80b4a20 msgr2=0x7f6ae8156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:37.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.898+0000 7f6aedaea700 1 --2- 172.21.15.92:0/4174516465 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ae80b4a20 0x7f6ae8156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.898+0000 7f6aedaea700 1 -- 172.21.15.92:0/4174516465 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ae80cd940 msgr2=0x7f6ae81496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:37.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.898+0000 7f6aedaea700 1 --2- 172.21.15.92:0/4174516465 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ae80cd940 0x7f6ae81496c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:37.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.898+0000 7f6aedaea700 1 -- 172.21.15.92:0/4174516465 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad80094f0 con 0x7f6ae80d0810 2023-10-06T10:38:37.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.898+0000 7f6aedaea700 1 --2- 172.21.15.92:0/4174516465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ae80d0810 0x7f6ae8151b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000c840 tx=0x7f6ae0001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:37.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.898+0000 7f6add7fa700 1 -- 172.21.15.92:0/4174516465 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0012630 con 0x7f6ae80d0810 2023-10-06T10:38:37.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.898+0000 7f6add7fa700 1 -- 172.21.15.92:0/4174516465 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae00127b0 con 0x7f6ae80d0810 2023-10-06T10:38:37.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.898+0000 7f6aef54d700 1 -- 172.21.15.92:0/4174516465 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae80b1740 con 0x7f6ae80d0810 2023-10-06T10:38:37.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.898+0000 7f6add7fa700 1 -- 172.21.15.92:0/4174516465 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0012b70 con 0x7f6ae80d0810 2023-10-06T10:38:37.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.898+0000 7f6aef54d700 1 -- 172.21.15.92:0/4174516465 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae80b1d00 con 0x7f6ae80d0810 2023-10-06T10:38:37.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.899+0000 7f6aef54d700 1 -- 172.21.15.92:0/4174516465 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae8064a90 con 0x7f6ae80d0810 2023-10-06T10:38:37.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.899+0000 7f6add7fa700 1 -- 172.21.15.92:0/4174516465 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ae001a3a0 con 0x7f6ae80d0810 2023-10-06T10:38:37.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.900+0000 7f6add7fa700 1 --2- 172.21.15.92:0/4174516465 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6acc066f30 0x7f6acc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:37.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.900+0000 7f6add7fa700 1 -- 172.21.15.92:0/4174516465 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f6ae0084ab0 con 0x7f6ae80d0810 2023-10-06T10:38:37.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.900+0000 7f6aed2e9700 1 --2- 172.21.15.92:0/4174516465 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6acc066f30 0x7f6acc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:37.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.900+0000 7f6aed2e9700 1 --2- 172.21.15.92:0/4174516465 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6acc066f30 0x7f6acc0693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009db0 tx=0x7f6ae40091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:37.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:37.904+0000 7f6add7fa700 1 -- 172.21.15.92:0/4174516465 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ae00518d0 con 0x7f6ae80d0810 2023-10-06T10:38:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.083+0000 7f6aef54d700 1 -- 172.21.15.92:0/4174516465 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ae80b52e0 con 0x7f6ae80d0810 2023-10-06T10:38:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.083+0000 7f6add7fa700 1 -- 172.21.15.92:0/4174516465 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6ae0054ef0 con 0x7f6ae80d0810 2023-10-06T10:38:38.085 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T10:38:38.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.085+0000 7f6aef54d700 1 -- 172.21.15.92:0/4174516465 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6acc066f30 msgr2=0x7f6acc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:38.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.085+0000 7f6aef54d700 1 --2- 172.21.15.92:0/4174516465 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6acc066f30 0x7f6acc0693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009db0 tx=0x7f6ae40091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.085+0000 7f6aef54d700 1 -- 172.21.15.92:0/4174516465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ae80d0810 msgr2=0x7f6ae8151b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:38.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.085+0000 7f6aef54d700 1 --2- 172.21.15.92:0/4174516465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ae80d0810 0x7f6ae8151b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000c840 tx=0x7f6ae0001670 comp rx=0 tx=0).stop 2023-10-06T10:38:38.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.085+0000 7f6aef54d700 1 -- 172.21.15.92:0/4174516465 shutdown_connections 2023-10-06T10:38:38.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.085+0000 7f6aef54d700 1 --2- 172.21.15.92:0/4174516465 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6acc066f30 0x7f6acc0693e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.085+0000 7f6aef54d700 1 --2- 172.21.15.92:0/4174516465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ae80d0810 0x7f6ae8151b00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.085+0000 7f6aef54d700 1 --2- 172.21.15.92:0/4174516465 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ae80b4a20 0x7f6ae8156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.085+0000 7f6aef54d700 1 --2- 172.21.15.92:0/4174516465 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ae80cd940 0x7f6ae81496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.085+0000 7f6aef54d700 1 -- 172.21.15.92:0/4174516465 >> 172.21.15.92:0/4174516465 conn(0x7f6ae806ece0 msgr2=0x7f6ae8153110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:38.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.086+0000 7f6aef54d700 1 -- 172.21.15.92:0/4174516465 shutdown_connections 2023-10-06T10:38:38.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.086+0000 7f6aef54d700 1 -- 172.21.15.92:0/4174516465 wait complete. 2023-10-06T10:38:38.099 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T10:38:38.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.214+0000 7f7488b95700 1 Processor -- start 2023-10-06T10:38:38.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.214+0000 7f7488b95700 1 -- start start 2023-10-06T10:38:38.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.215+0000 7f7488b95700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f74841594d0 0x7f74840cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:38.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.215+0000 7f7488b95700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f74840cbaa0 0x7f74840ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.215+0000 7f7488b95700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74840cbee0 con 0x7f74841598a0 2023-10-06T10:38:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.215+0000 7f7488b95700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74840cf5d0 con 0x7f74841594d0 2023-10-06T10:38:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.215+0000 7f7488b95700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74840cf8b0 con 0x7f74840cbaa0 2023-10-06T10:38:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.215+0000 7f748259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f74841594d0 0x7f74840cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.215+0000 7f748259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f74841594d0 0x7f74840cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36976/0 (socket says 172.21.15.92:36976) 2023-10-06T10:38:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.215+0000 7f748259c700 1 -- 172.21.15.92:0/1595226332 learned_addr learned my addr 172.21.15.92:0/1595226332 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.215+0000 7f7482d9d700 1 --2- 172.21.15.92:0/1595226332 >> v2:172.21.15.124:3300/0 conn(0x7f74840cbaa0 0x7f74840ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.215+0000 7f748259c700 1 -- 172.21.15.92:0/1595226332 >> v2:172.21.15.124:3300/0 conn(0x7f74840cbaa0 msgr2=0x7f74840ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.215+0000 7f748259c700 1 --2- 172.21.15.92:0/1595226332 >> v2:172.21.15.124:3300/0 conn(0x7f74840cbaa0 0x7f74840ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.215+0000 7f748259c700 1 -- 172.21.15.92:0/1595226332 >> v1:172.21.15.124:6789/0 conn(0x7f74841598a0 legacy=0x7f74840ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.215+0000 7f748259c700 1 -- 172.21.15.92:0/1595226332 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74840cfb90 con 0x7f74841594d0 2023-10-06T10:38:38.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.215+0000 7f7482d9d700 1 --2- 172.21.15.92:0/1595226332 >> v2:172.21.15.124:3300/0 conn(0x7f74840cbaa0 0x7f74840ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:38.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.216+0000 7f748259c700 1 --2- 172.21.15.92:0/1595226332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74841594d0 0x7f74840cb560 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f748406f950 tx=0x7f747c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b4b29887a1091e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:38.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.216+0000 7f748159a700 1 -- 172.21.15.92:0/1595226332 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c00b820 con 0x7f74841594d0 2023-10-06T10:38:38.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.216+0000 7f748159a700 1 -- 172.21.15.92:0/1595226332 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c00b980 con 0x7f74841594d0 2023-10-06T10:38:38.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.216+0000 7f748159a700 1 -- 172.21.15.92:0/1595226332 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c00bd20 con 0x7f74841594d0 2023-10-06T10:38:38.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.216+0000 7f7488b95700 1 -- 172.21.15.92:0/1595226332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74841594d0 msgr2=0x7f74840cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:38.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.216+0000 7f7488b95700 1 --2- 172.21.15.92:0/1595226332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74841594d0 0x7f74840cb560 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f748406f950 tx=0x7f747c009250 comp rx=0 tx=0).stop 2023-10-06T10:38:38.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.216+0000 7f7488b95700 1 -- 172.21.15.92:0/1595226332 shutdown_connections 2023-10-06T10:38:38.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.216+0000 7f7488b95700 1 --2- 172.21.15.92:0/1595226332 >> v2:172.21.15.124:3300/0 conn(0x7f74840cbaa0 0x7f74840ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.216+0000 7f7488b95700 1 --2- 172.21.15.92:0/1595226332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74841594d0 0x7f74840cb560 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.216+0000 7f7488b95700 1 -- 172.21.15.92:0/1595226332 >> 172.21.15.92:0/1595226332 conn(0x7f7484089e50 msgr2=0x7f7484099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:38.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.216+0000 7f7488b95700 1 -- 172.21.15.92:0/1595226332 shutdown_connections 2023-10-06T10:38:38.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.216+0000 7f7488b95700 1 -- 172.21.15.92:0/1595226332 wait complete. 2023-10-06T10:38:38.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.217+0000 7f7488b95700 1 Processor -- start 2023-10-06T10:38:38.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.217+0000 7f7488b95700 1 -- start start 2023-10-06T10:38:38.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.218+0000 7f7488b95700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74840cbaa0 0x7f748409ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:38.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.218+0000 7f7488b95700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74841594d0 0x7f74840b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:38.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.218+0000 7f7488b95700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f74841598a0 0x7f748409c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:38.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.218+0000 7f7488b95700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748409c380 con 0x7f74841594d0 2023-10-06T10:38:38.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.218+0000 7f7488b95700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748409b770 con 0x7f74841598a0 2023-10-06T10:38:38.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.218+0000 7f7488b95700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748409ba50 con 0x7f74840cbaa0 2023-10-06T10:38:38.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.218+0000 7f7481d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74841594d0 0x7f74840b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:38.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.218+0000 7f7481d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74841594d0 0x7f74840b32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36988/0 (socket says 172.21.15.92:36988) 2023-10-06T10:38:38.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.218+0000 7f7481d9b700 1 -- 172.21.15.92:0/4265732612 learned_addr learned my addr 172.21.15.92:0/4265732612 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:38.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.218+0000 7f7482d9d700 1 --2- 172.21.15.92:0/4265732612 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f74841598a0 0x7f748409c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:38.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.218+0000 7f748259c700 1 --2- 172.21.15.92:0/4265732612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74840cbaa0 0x7f748409ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:38.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.219+0000 7f7481d9b700 1 -- 172.21.15.92:0/4265732612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74840cbaa0 msgr2=0x7f748409ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:38.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.219+0000 7f7481d9b700 1 --2- 172.21.15.92:0/4265732612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74840cbaa0 0x7f748409ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.219+0000 7f7481d9b700 1 -- 172.21.15.92:0/4265732612 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f74841598a0 msgr2=0x7f748409c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.219+0000 7f7481d9b700 1 --2- 172.21.15.92:0/4265732612 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f74841598a0 0x7f748409c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.219+0000 7f7481d9b700 1 -- 172.21.15.92:0/4265732612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747c0094f0 con 0x7f74841594d0 2023-10-06T10:38:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.219+0000 7f7481d9b700 1 --2- 172.21.15.92:0/4265732612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74841594d0 0x7f74840b32b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f746c00b510 tx=0x7f746c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.219+0000 7f747a7fc700 1 -- 172.21.15.92:0/4265732612 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c00f510 con 0x7f74841594d0 2023-10-06T10:38:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.219+0000 7f747a7fc700 1 -- 172.21.15.92:0/4265732612 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746c00f670 con 0x7f74841594d0 2023-10-06T10:38:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.219+0000 7f7488b95700 1 -- 172.21.15.92:0/4265732612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748409bd30 con 0x7f74841594d0 2023-10-06T10:38:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.219+0000 7f747a7fc700 1 -- 172.21.15.92:0/4265732612 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c00fa30 con 0x7f74841594d0 2023-10-06T10:38:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.219+0000 7f7488b95700 1 -- 172.21.15.92:0/4265732612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748414d880 con 0x7f74841594d0 2023-10-06T10:38:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.220+0000 7f7488b95700 1 -- 172.21.15.92:0/4265732612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748409c530 con 0x7f74841594d0 2023-10-06T10:38:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.220+0000 7f747a7fc700 1 -- 172.21.15.92:0/4265732612 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f746c01a400 con 0x7f74841594d0 2023-10-06T10:38:38.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.221+0000 7f747a7fc700 1 --2- 172.21.15.92:0/4265732612 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7464066f30 0x7f74640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:38.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.221+0000 7f748259c700 1 --2- 172.21.15.92:0/4265732612 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7464066f30 0x7f74640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:38.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.221+0000 7f747a7fc700 1 -- 172.21.15.92:0/4265732612 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f746c0842c0 con 0x7f74841594d0 2023-10-06T10:38:38.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.221+0000 7f748259c700 1 --2- 172.21.15.92:0/4265732612 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7464066f30 0x7f74640693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f748406f950 tx=0x7f747c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:38.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.226+0000 7f747a7fc700 1 -- 172.21.15.92:0/4265732612 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f748409c530 con 0x7f74841594d0 2023-10-06T10:38:38.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.409+0000 7f7488b95700 1 -- 172.21.15.92:0/4265732612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7484064a90 con 0x7f74841594d0 2023-10-06T10:38:38.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.410+0000 7f747a7fc700 1 -- 172.21.15.92:0/4265732612 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f746c051160 con 0x7f74841594d0 2023-10-06T10:38:38.412 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T10:38:38.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.412+0000 7f7488b95700 1 -- 172.21.15.92:0/4265732612 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7464066f30 msgr2=0x7f74640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:38.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.412+0000 7f7488b95700 1 --2- 172.21.15.92:0/4265732612 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7464066f30 0x7f74640693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f748406f950 tx=0x7f747c016000 comp rx=0 tx=0).stop 2023-10-06T10:38:38.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.412+0000 7f7488b95700 1 -- 172.21.15.92:0/4265732612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74841594d0 msgr2=0x7f74840b32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:38.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.412+0000 7f7488b95700 1 --2- 172.21.15.92:0/4265732612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74841594d0 0x7f74840b32b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f746c00b510 tx=0x7f746c0090f0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.412+0000 7f7488b95700 1 -- 172.21.15.92:0/4265732612 shutdown_connections 2023-10-06T10:38:38.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.412+0000 7f7488b95700 1 --2- 172.21.15.92:0/4265732612 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7464066f30 0x7f74640693e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.412+0000 7f7488b95700 1 --2- 172.21.15.92:0/4265732612 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f74841598a0 0x7f748409c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.412+0000 7f7488b95700 1 --2- 172.21.15.92:0/4265732612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74840cbaa0 0x7f748409ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.412+0000 7f7488b95700 1 --2- 172.21.15.92:0/4265732612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74841594d0 0x7f74840b32b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.412+0000 7f7488b95700 1 -- 172.21.15.92:0/4265732612 >> 172.21.15.92:0/4265732612 conn(0x7f7484089e50 msgr2=0x7f748414f750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:38.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.412+0000 7f7488b95700 1 -- 172.21.15.92:0/4265732612 shutdown_connections 2023-10-06T10:38:38.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.412+0000 7f7488b95700 1 -- 172.21.15.92:0/4265732612 wait complete. 2023-10-06T10:38:38.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T10:38:38.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.542+0000 7f7d1d9ca700 1 Processor -- start 2023-10-06T10:38:38.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.542+0000 7f7d1d9ca700 1 -- start start 2023-10-06T10:38:38.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.543+0000 7f7d1d9ca700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7d18159470 0x7f7d180ce980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:38.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.543+0000 7f7d1d9ca700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7d18159840 0x7f7d180ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:38.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.543+0000 7f7d1d9ca700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d180c6170 0x7f7d180c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:38.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.543+0000 7f7d1d9ca700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d180cbb80 con 0x7f7d18159840 2023-10-06T10:38:38.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.543+0000 7f7d1d9ca700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d180c6f00 con 0x7f7d18159470 2023-10-06T10:38:38.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.543+0000 7f7d1d9ca700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180cb530 con 0x7f7d180c6170 2023-10-06T10:38:38.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.543+0000 7f7d0e7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7d18159840 0x7f7d180ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:38.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.543+0000 7f7d0e7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7d18159840 0x7f7d180ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36992/0 (socket says 172.21.15.92:36992) 2023-10-06T10:38:38.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.543+0000 7f7d0e7fc700 1 -- 172.21.15.92:0/333538474 learned_addr learned my addr 172.21.15.92:0/333538474 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:38.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.544+0000 7f7d16ffd700 1 --2- 172.21.15.92:0/333538474 >> v2:172.21.15.124:3300/0 conn(0x7f7d18159470 0x7f7d180ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:38.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.544+0000 7f7d177fe700 1 --2- 172.21.15.92:0/333538474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d180c6170 0x7f7d180c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:38.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.544+0000 7f7d0e7fc700 1 -- 172.21.15.92:0/333538474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d180c6170 msgr2=0x7f7d180c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:38.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.544+0000 7f7d0e7fc700 1 --2- 172.21.15.92:0/333538474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d180c6170 0x7f7d180c65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.544+0000 7f7d0e7fc700 1 -- 172.21.15.92:0/333538474 >> v2:172.21.15.124:3300/0 conn(0x7f7d18159470 msgr2=0x7f7d180ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:38.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.544+0000 7f7d0e7fc700 1 --2- 172.21.15.92:0/333538474 >> v2:172.21.15.124:3300/0 conn(0x7f7d18159470 0x7f7d180ce980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.544+0000 7f7d0e7fc700 1 -- 172.21.15.92:0/333538474 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d180cb810 con 0x7f7d18159840 2023-10-06T10:38:38.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.544+0000 7f7d16ffd700 1 --2- 172.21.15.92:0/333538474 >> v2:172.21.15.124:3300/0 conn(0x7f7d18159470 0x7f7d180ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:38.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.544+0000 7f7d177fe700 1 --2- 172.21.15.92:0/333538474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d180c6170 0x7f7d180c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:38.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.544+0000 7f7d0e7fc700 1 --2- 172.21.15.92:0/333538474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d18159840 0x7f7d180ceec0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7d18089d50 tx=0x7f7d00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91f39d1fda0fe60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:38.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.544+0000 7f7d167fc700 1 -- 172.21.15.92:0/333538474 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0000b860 con 0x7f7d18159840 2023-10-06T10:38:38.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.544+0000 7f7d167fc700 1 -- 172.21.15.92:0/333538474 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0000b9c0 con 0x7f7d18159840 2023-10-06T10:38:38.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.544+0000 7f7d167fc700 1 -- 172.21.15.92:0/333538474 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0000bd60 con 0x7f7d18159840 2023-10-06T10:38:38.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.544+0000 7f7d1d9ca700 1 -- 172.21.15.92:0/333538474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d18159840 msgr2=0x7f7d180ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:38.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.544+0000 7f7d1d9ca700 1 --2- 172.21.15.92:0/333538474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d18159840 0x7f7d180ceec0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7d18089d50 tx=0x7f7d00009250 comp rx=0 tx=0).stop 2023-10-06T10:38:38.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.545+0000 7f7d1d9ca700 1 -- 172.21.15.92:0/333538474 shutdown_connections 2023-10-06T10:38:38.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.545+0000 7f7d1d9ca700 1 --2- 172.21.15.92:0/333538474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d180c6170 0x7f7d180c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.545+0000 7f7d1d9ca700 1 --2- 172.21.15.92:0/333538474 >> v2:172.21.15.124:3300/0 conn(0x7f7d18159470 0x7f7d180ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.545+0000 7f7d1d9ca700 1 --2- 172.21.15.92:0/333538474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d18159840 0x7f7d180ceec0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.545+0000 7f7d1d9ca700 1 -- 172.21.15.92:0/333538474 >> 172.21.15.92:0/333538474 conn(0x7f7d180996e0 msgr2=0x7f7d1806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:38.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.545+0000 7f7d1d9ca700 1 -- 172.21.15.92:0/333538474 shutdown_connections 2023-10-06T10:38:38.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.545+0000 7f7d1d9ca700 1 -- 172.21.15.92:0/333538474 wait complete. 2023-10-06T10:38:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.546+0000 7f7d1d9ca700 1 Processor -- start 2023-10-06T10:38:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.546+0000 7f7d1d9ca700 1 -- start start 2023-10-06T10:38:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.547+0000 7f7d1d9ca700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d180c6170 0x7f7d18153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.547+0000 7f7d1d9ca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d18159470 0x7f7d1816d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.547+0000 7f7d1d9ca700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d18159840 0x7f7d180ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:38.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.547+0000 7f7d1d9ca700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1814baf0 con 0x7f7d18159470 2023-10-06T10:38:38.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.547+0000 7f7d1d9ca700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1814c0b0 con 0x7f7d180c6170 2023-10-06T10:38:38.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.547+0000 7f7d1d9ca700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1816db60 con 0x7f7d18159840 2023-10-06T10:38:38.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.547+0000 7f7d0e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d18159470 0x7f7d1816d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:38.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.547+0000 7f7d0e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d18159470 0x7f7d1816d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37002/0 (socket says 172.21.15.92:37002) 2023-10-06T10:38:38.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.547+0000 7f7d0e7fc700 1 -- 172.21.15.92:0/667355913 learned_addr learned my addr 172.21.15.92:0/667355913 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:38.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.547+0000 7f7d16ffd700 1 --2- 172.21.15.92:0/667355913 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d180c6170 0x7f7d18153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:38.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.547+0000 7f7d177fe700 1 --2- 172.21.15.92:0/667355913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d18159840 0x7f7d180ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:38.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.547+0000 7f7d0e7fc700 1 -- 172.21.15.92:0/667355913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d18159840 msgr2=0x7f7d180ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:38.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.547+0000 7f7d0e7fc700 1 --2- 172.21.15.92:0/667355913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d18159840 0x7f7d180ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.547+0000 7f7d0e7fc700 1 -- 172.21.15.92:0/667355913 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d180c6170 msgr2=0x7f7d18153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:38.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.547+0000 7f7d0e7fc700 1 --2- 172.21.15.92:0/667355913 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d180c6170 0x7f7d18153690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.547+0000 7f7d0e7fc700 1 -- 172.21.15.92:0/667355913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d000094f0 con 0x7f7d18159470 2023-10-06T10:38:38.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.547+0000 7f7d0e7fc700 1 --2- 172.21.15.92:0/667355913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d18159470 0x7f7d1816d0a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7d00006fd0 tx=0x7f7d00017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:38.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.547+0000 7f7d0f7fe700 1 -- 172.21.15.92:0/667355913 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d00011e00 con 0x7f7d18159470 2023-10-06T10:38:38.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.547+0000 7f7d0f7fe700 1 -- 172.21.15.92:0/667355913 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0001a450 con 0x7f7d18159470 2023-10-06T10:38:38.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.547+0000 7f7d1d9ca700 1 -- 172.21.15.92:0/667355913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1816de40 con 0x7f7d18159470 2023-10-06T10:38:38.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.547+0000 7f7d0f7fe700 1 -- 172.21.15.92:0/667355913 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0001a810 con 0x7f7d18159470 2023-10-06T10:38:38.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.547+0000 7f7d1d9ca700 1 -- 172.21.15.92:0/667355913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1814e2a0 con 0x7f7d18159470 2023-10-06T10:38:38.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.548+0000 7f7d1d9ca700 1 -- 172.21.15.92:0/667355913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d18064a90 con 0x7f7d18159470 2023-10-06T10:38:38.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.549+0000 7f7d0f7fe700 1 -- 172.21.15.92:0/667355913 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7d0001abc0 con 0x7f7d18159470 2023-10-06T10:38:38.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.549+0000 7f7d0f7fe700 1 --2- 172.21.15.92:0/667355913 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7cf8066f30 0x7f7cf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:38.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.549+0000 7f7d16ffd700 1 --2- 172.21.15.92:0/667355913 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7cf8066f30 0x7f7cf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:38.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.549+0000 7f7d0f7fe700 1 -- 172.21.15.92:0/667355913 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f7d00086340 con 0x7f7d18159470 2023-10-06T10:38:38.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.550+0000 7f7d16ffd700 1 --2- 172.21.15.92:0/667355913 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7cf8066f30 0x7f7cf80693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7d0800ad40 tx=0x7f7d08008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:38.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.555+0000 7f7d0f7fe700 1 -- 172.21.15.92:0/667355913 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d00053160 con 0x7f7d18159470 2023-10-06T10:38:38.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.734+0000 7f7d1d9ca700 1 -- 172.21.15.92:0/667355913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7d1814c1f0 con 0x7f7d18159470 2023-10-06T10:38:38.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.734+0000 7f7d0f7fe700 1 -- 172.21.15.92:0/667355913 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7d00056780 con 0x7f7d18159470 2023-10-06T10:38:38.737 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T10:38:38.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.737+0000 7f7d1d9ca700 1 -- 172.21.15.92:0/667355913 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7cf8066f30 msgr2=0x7f7cf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:38.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.737+0000 7f7d1d9ca700 1 --2- 172.21.15.92:0/667355913 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7cf8066f30 0x7f7cf80693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7d0800ad40 tx=0x7f7d08008040 comp rx=0 tx=0).stop 2023-10-06T10:38:38.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.737+0000 7f7d1d9ca700 1 -- 172.21.15.92:0/667355913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d18159470 msgr2=0x7f7d1816d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:38.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.737+0000 7f7d1d9ca700 1 --2- 172.21.15.92:0/667355913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d18159470 0x7f7d1816d0a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7d00006fd0 tx=0x7f7d00017000 comp rx=0 tx=0).stop 2023-10-06T10:38:38.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.737+0000 7f7d1d9ca700 1 -- 172.21.15.92:0/667355913 shutdown_connections 2023-10-06T10:38:38.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.737+0000 7f7d1d9ca700 1 --2- 172.21.15.92:0/667355913 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7cf8066f30 0x7f7cf80693e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.737+0000 7f7d1d9ca700 1 --2- 172.21.15.92:0/667355913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d18159840 0x7f7d180ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.737+0000 7f7d1d9ca700 1 --2- 172.21.15.92:0/667355913 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d180c6170 0x7f7d18153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.737+0000 7f7d1d9ca700 1 --2- 172.21.15.92:0/667355913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d18159470 0x7f7d1816d0a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.737+0000 7f7d1d9ca700 1 -- 172.21.15.92:0/667355913 >> 172.21.15.92:0/667355913 conn(0x7f7d180996e0 msgr2=0x7f7d180b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:38.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.737+0000 7f7d1d9ca700 1 -- 172.21.15.92:0/667355913 shutdown_connections 2023-10-06T10:38:38.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.737+0000 7f7d1d9ca700 1 -- 172.21.15.92:0/667355913 wait complete. 2023-10-06T10:38:38.751 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:38:38.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.862+0000 7f453b2f6700 1 Processor -- start 2023-10-06T10:38:38.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.862+0000 7f453b2f6700 1 -- start start 2023-10-06T10:38:38.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f453b2f6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4534159440 0x7f45340b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:38.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f453b2f6700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f45340b2100 0x7f45340b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:38.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f453b2f6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45340cb7a0 con 0x7f45340b35d0 2023-10-06T10:38:38.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f453b2f6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45340b0860 con 0x7f4534159440 2023-10-06T10:38:38.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f453b2f6700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45340cb150 con 0x7f45340b2100 2023-10-06T10:38:38.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f453a2f4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4534159440 0x7f45340b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:38.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f4539af3700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f45340b35d0 0x7f45340cbc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40178/0 (socket says 172.21.15.92:40178) 2023-10-06T10:38:38.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f453a2f4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4534159440 0x7f45340b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37006/0 (socket says 172.21.15.92:37006) 2023-10-06T10:38:38.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f4539af3700 1 -- 172.21.15.92:0/3722949312 learned_addr learned my addr 172.21.15.92:0/3722949312 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:38.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f453aaf5700 1 --2- 172.21.15.92:0/3722949312 >> v2:172.21.15.124:3300/0 conn(0x7f45340b2100 0x7f45340b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:38.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f453a2f4700 1 -- 172.21.15.92:0/3722949312 >> v2:172.21.15.124:3300/0 conn(0x7f45340b2100 msgr2=0x7f45340b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:38.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f453a2f4700 1 --2- 172.21.15.92:0/3722949312 >> v2:172.21.15.124:3300/0 conn(0x7f45340b2100 0x7f45340b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f453a2f4700 1 -- 172.21.15.92:0/3722949312 >> v1:172.21.15.92:6789/0 conn(0x7f45340b35d0 legacy=0x7f45340cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:38.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f453a2f4700 1 -- 172.21.15.92:0/3722949312 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45340cb430 con 0x7f4534159440 2023-10-06T10:38:38.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f45392f2700 1 -- 172.21.15.92:0/3722949312 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2305300944 0 0) 0x7f45340cb7a0 con 0x7f45340b35d0 2023-10-06T10:38:38.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f453aaf5700 1 --2- 172.21.15.92:0/3722949312 >> v2:172.21.15.124:3300/0 conn(0x7f45340b2100 0x7f45340b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:38.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f453a2f4700 1 --2- 172.21.15.92:0/3722949312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4534159440 0x7f45340b3090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f45340ca7f0 tx=0x7f4530009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc21b8e0a35e68cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:38.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f45392f2700 1 -- 172.21.15.92:0/3722949312 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453000e070 con 0x7f4534159440 2023-10-06T10:38:38.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f45392f2700 1 -- 172.21.15.92:0/3722949312 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453000bc30 con 0x7f4534159440 2023-10-06T10:38:38.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f45392f2700 1 -- 172.21.15.92:0/3722949312 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4530012430 con 0x7f4534159440 2023-10-06T10:38:38.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f453b2f6700 1 -- 172.21.15.92:0/3722949312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4534159440 msgr2=0x7f45340b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:38.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f453b2f6700 1 --2- 172.21.15.92:0/3722949312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4534159440 0x7f45340b3090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f45340ca7f0 tx=0x7f4530009780 comp rx=0 tx=0).stop 2023-10-06T10:38:38.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f453b2f6700 1 -- 172.21.15.92:0/3722949312 shutdown_connections 2023-10-06T10:38:38.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f453b2f6700 1 --2- 172.21.15.92:0/3722949312 >> v2:172.21.15.124:3300/0 conn(0x7f45340b2100 0x7f45340b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f453b2f6700 1 --2- 172.21.15.92:0/3722949312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4534159440 0x7f45340b3090 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f453b2f6700 1 -- 172.21.15.92:0/3722949312 >> 172.21.15.92:0/3722949312 conn(0x7f453406ece0 msgr2=0x7f453406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:38.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f453b2f6700 1 -- 172.21.15.92:0/3722949312 shutdown_connections 2023-10-06T10:38:38.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.864+0000 7f453b2f6700 1 -- 172.21.15.92:0/3722949312 wait complete. 2023-10-06T10:38:38.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.865+0000 7f453b2f6700 1 Processor -- start 2023-10-06T10:38:38.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.865+0000 7f453b2f6700 1 -- start start 2023-10-06T10:38:38.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.866+0000 7f453b2f6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45340b2100 0x7f453415b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:38.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.866+0000 7f453b2f6700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45340b35d0 0x7f45340b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:38.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.866+0000 7f453b2f6700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4534159440 0x7f4534157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:38.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.866+0000 7f453b2f6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453416a0a0 con 0x7f45340b2100 2023-10-06T10:38:38.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.866+0000 7f453b2f6700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45340b42a0 con 0x7f4534159440 2023-10-06T10:38:38.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.866+0000 7f453b2f6700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4534169dc0 con 0x7f45340b35d0 2023-10-06T10:38:38.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.866+0000 7f453a2f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45340b2100 0x7f453415b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:38.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.866+0000 7f453a2f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45340b2100 0x7f453415b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37022/0 (socket says 172.21.15.92:37022) 2023-10-06T10:38:38.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.866+0000 7f453a2f4700 1 -- 172.21.15.92:0/2930659570 learned_addr learned my addr 172.21.15.92:0/2930659570 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:38.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.866+0000 7f453aaf5700 1 --2- 172.21.15.92:0/2930659570 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4534159440 0x7f4534157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:38.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.866+0000 7f4539af3700 1 --2- 172.21.15.92:0/2930659570 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45340b35d0 0x7f45340b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:38.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.867+0000 7f453a2f4700 1 -- 172.21.15.92:0/2930659570 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45340b35d0 msgr2=0x7f45340b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:38.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.867+0000 7f453a2f4700 1 --2- 172.21.15.92:0/2930659570 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45340b35d0 0x7f45340b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.867+0000 7f453a2f4700 1 -- 172.21.15.92:0/2930659570 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4534159440 msgr2=0x7f4534157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:38.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.867+0000 7f453a2f4700 1 --2- 172.21.15.92:0/2930659570 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4534159440 0x7f4534157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:38.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.867+0000 7f453a2f4700 1 -- 172.21.15.92:0/2930659570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4530009a20 con 0x7f45340b2100 2023-10-06T10:38:38.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.867+0000 7f453aaf5700 1 --2- 172.21.15.92:0/2930659570 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4534159440 0x7f4534157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:38.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.867+0000 7f4539af3700 1 --2- 172.21.15.92:0/2930659570 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45340b35d0 0x7f45340b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:38.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.867+0000 7f453a2f4700 1 --2- 172.21.15.92:0/2930659570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45340b2100 0x7f453415b950 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f45300099f0 tx=0x7f4530019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:38.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.867+0000 7f452a7fc700 1 -- 172.21.15.92:0/2930659570 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4530012ea0 con 0x7f45340b2100 2023-10-06T10:38:38.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.867+0000 7f452a7fc700 1 -- 172.21.15.92:0/2930659570 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4530001090 con 0x7f45340b2100 2023-10-06T10:38:38.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.867+0000 7f453b2f6700 1 -- 172.21.15.92:0/2930659570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45340b3c50 con 0x7f45340b2100 2023-10-06T10:38:38.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.867+0000 7f452a7fc700 1 -- 172.21.15.92:0/2930659570 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453001d880 con 0x7f45340b2100 2023-10-06T10:38:38.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.867+0000 7f453b2f6700 1 -- 172.21.15.92:0/2930659570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45340b4c70 con 0x7f45340b2100 2023-10-06T10:38:38.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.868+0000 7f453b2f6700 1 -- 172.21.15.92:0/2930659570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45340b3ab0 con 0x7f45340b2100 2023-10-06T10:38:38.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.868+0000 7f452a7fc700 1 -- 172.21.15.92:0/2930659570 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4530024070 con 0x7f45340b2100 2023-10-06T10:38:38.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.869+0000 7f452a7fc700 1 --2- 172.21.15.92:0/2930659570 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4518066f30 0x7f45180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:38.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.869+0000 7f452a7fc700 1 -- 172.21.15.92:0/2930659570 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4530087060 con 0x7f45340b2100 2023-10-06T10:38:38.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.869+0000 7f4539af3700 1 --2- 172.21.15.92:0/2930659570 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4518066f30 0x7f45180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:38.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.869+0000 7f4539af3700 1 --2- 172.21.15.92:0/2930659570 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4518066f30 0x7f45180693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f453415ca60 tx=0x7f4524009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:38.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:38.873+0000 7f452a7fc700 1 -- 172.21.15.92:0/2930659570 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f45340b3ab0 con 0x7f45340b2100 2023-10-06T10:38:39.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.097+0000 7f453b2f6700 1 -- 172.21.15.92:0/2930659570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45340b3ab0 con 0x7f45340b2100 2023-10-06T10:38:39.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.097+0000 7f452a7fc700 1 -- 172.21.15.92:0/2930659570 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4599 (secure 0 0 0) 0x7f45340b3ab0 con 0x7f45340b2100 2023-10-06T10:38:39.098 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:39.099 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2303424485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2303424485},{"type":"v1","addr":"172.21.15.92:6836","nonce":2303424485}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1027343065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1027343065},{"type":"v1","addr":"172.21.15.92:6837","nonce":1027343065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2667820379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2667820379},{"type":"v1","addr":"172.21.15.124:6834","nonce":2667820379}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:38:16.725144+0000","modified":"2023-10-06T10:38:30.398242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/1919704285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":1919704285},{"type":"v1","addr":"172.21.15.124:6835","nonce":1919704285}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T10:38:39.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.099+0000 7f453b2f6700 1 -- 172.21.15.92:0/2930659570 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4518066f30 msgr2=0x7f45180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:39.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.099+0000 7f453b2f6700 1 --2- 172.21.15.92:0/2930659570 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4518066f30 0x7f45180693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f453415ca60 tx=0x7f4524009040 comp rx=0 tx=0).stop 2023-10-06T10:38:39.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.099+0000 7f453b2f6700 1 -- 172.21.15.92:0/2930659570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45340b2100 msgr2=0x7f453415b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:39.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.099+0000 7f453b2f6700 1 --2- 172.21.15.92:0/2930659570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45340b2100 0x7f453415b950 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f45300099f0 tx=0x7f4530019000 comp rx=0 tx=0).stop 2023-10-06T10:38:39.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.099+0000 7f453b2f6700 1 -- 172.21.15.92:0/2930659570 shutdown_connections 2023-10-06T10:38:39.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.100+0000 7f453b2f6700 1 --2- 172.21.15.92:0/2930659570 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4518066f30 0x7f45180693e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.100+0000 7f453b2f6700 1 --2- 172.21.15.92:0/2930659570 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4534159440 0x7f4534157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.100+0000 7f453b2f6700 1 --2- 172.21.15.92:0/2930659570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45340b2100 0x7f453415b950 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.100+0000 7f453b2f6700 1 --2- 172.21.15.92:0/2930659570 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45340b35d0 0x7f45340b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.100+0000 7f453b2f6700 1 -- 172.21.15.92:0/2930659570 >> 172.21.15.92:0/2930659570 conn(0x7f453406ece0 msgr2=0x7f45340cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:39.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.100+0000 7f453b2f6700 1 -- 172.21.15.92:0/2930659570 shutdown_connections 2023-10-06T10:38:39.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.100+0000 7f453b2f6700 1 -- 172.21.15.92:0/2930659570 wait complete. 2023-10-06T10:38:39.102 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-10-06T10:38:39.113 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:38:39.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.225+0000 7f129a906700 1 Processor -- start 2023-10-06T10:38:39.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.226+0000 7f129a906700 1 -- start start 2023-10-06T10:38:39.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.226+0000 7f129a906700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1294068810 0x7f12940afb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:39.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.226+0000 7f129a906700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f12940ac440 0x7f1294155600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:39.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.226+0000 7f129a906700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1294068be0 0x7f1294155b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:39.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.226+0000 7f129a906700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12940ac8d0 con 0x7f12940ac440 2023-10-06T10:38:39.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.226+0000 7f129a906700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12940acbb0 con 0x7f1294068be0 2023-10-06T10:38:39.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.226+0000 7f129a906700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12940ace90 con 0x7f1294068810 2023-10-06T10:38:39.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.226+0000 7f12937fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f12940ac440 0x7f1294155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:39.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.226+0000 7f12937fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f12940ac440 0x7f1294155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37036/0 (socket says 172.21.15.92:37036) 2023-10-06T10:38:39.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.226+0000 7f12937fe700 1 -- 172.21.15.92:0/3319071812 learned_addr learned my addr 172.21.15.92:0/3319071812 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:39.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.227+0000 7f1293fff700 1 --2- 172.21.15.92:0/3319071812 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1294068810 0x7f12940afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:39.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.227+0000 7f1298ea3700 1 --2- 172.21.15.92:0/3319071812 >> v2:172.21.15.124:3300/0 conn(0x7f1294068be0 0x7f1294155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:39.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.227+0000 7f12937fe700 1 -- 172.21.15.92:0/3319071812 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1294068810 msgr2=0x7f12940afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:39.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.227+0000 7f12937fe700 1 --2- 172.21.15.92:0/3319071812 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1294068810 0x7f12940afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.227+0000 7f12937fe700 1 -- 172.21.15.92:0/3319071812 >> v2:172.21.15.124:3300/0 conn(0x7f1294068be0 msgr2=0x7f1294155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:39.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.227+0000 7f12937fe700 1 --2- 172.21.15.92:0/3319071812 >> v2:172.21.15.124:3300/0 conn(0x7f1294068be0 0x7f1294155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.227+0000 7f12937fe700 1 -- 172.21.15.92:0/3319071812 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12940ad170 con 0x7f12940ac440 2023-10-06T10:38:39.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.227+0000 7f12937fe700 1 --2- 172.21.15.92:0/3319071812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12940ac440 0x7f1294155600 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f12940b1270 tx=0x7f128800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e379d6451dcae3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:39.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.227+0000 7f1292ffd700 1 -- 172.21.15.92:0/3319071812 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128800dc50 con 0x7f12940ac440 2023-10-06T10:38:39.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.227+0000 7f1292ffd700 1 -- 172.21.15.92:0/3319071812 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128800ddb0 con 0x7f12940ac440 2023-10-06T10:38:39.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.227+0000 7f1292ffd700 1 -- 172.21.15.92:0/3319071812 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12880196d0 con 0x7f12940ac440 2023-10-06T10:38:39.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.228+0000 7f129a906700 1 -- 172.21.15.92:0/3319071812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12940ac440 msgr2=0x7f1294155600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:39.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.228+0000 7f129a906700 1 --2- 172.21.15.92:0/3319071812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12940ac440 0x7f1294155600 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f12940b1270 tx=0x7f128800b4f0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.228+0000 7f129a906700 1 -- 172.21.15.92:0/3319071812 shutdown_connections 2023-10-06T10:38:39.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.228+0000 7f129a906700 1 --2- 172.21.15.92:0/3319071812 >> v2:172.21.15.124:3300/0 conn(0x7f1294068be0 0x7f1294155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.228+0000 7f129a906700 1 --2- 172.21.15.92:0/3319071812 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1294068810 0x7f12940afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.228+0000 7f129a906700 1 --2- 172.21.15.92:0/3319071812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12940ac440 0x7f1294155600 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.228+0000 7f129a906700 1 -- 172.21.15.92:0/3319071812 >> 172.21.15.92:0/3319071812 conn(0x7f129406de40 msgr2=0x7f129406e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:39.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.228+0000 7f129a906700 1 -- 172.21.15.92:0/3319071812 shutdown_connections 2023-10-06T10:38:39.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.228+0000 7f129a906700 1 -- 172.21.15.92:0/3319071812 wait complete. 2023-10-06T10:38:39.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.229+0000 7f129a906700 1 Processor -- start 2023-10-06T10:38:39.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.229+0000 7f129a906700 1 -- start start 2023-10-06T10:38:39.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f129a906700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1294068810 0x7f1294165e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:39.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f129a906700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1294068be0 0x7f1294168960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f129a906700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12940ac440 0x7f12940ab150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f129a906700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1294168ea0 con 0x7f1294068810 2023-10-06T10:38:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f129a906700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1294169120 con 0x7f1294068be0 2023-10-06T10:38:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f129a906700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1294169400 con 0x7f12940ac440 2023-10-06T10:38:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f1293fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1294068810 0x7f1294165e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:39.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f1293fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1294068810 0x7f1294165e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37042/0 (socket says 172.21.15.92:37042) 2023-10-06T10:38:39.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f1293fff700 1 -- 172.21.15.92:0/3421159125 learned_addr learned my addr 172.21.15.92:0/3421159125 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:39.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f12937fe700 1 --2- 172.21.15.92:0/3421159125 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1294068be0 0x7f1294168960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:39.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f1298ea3700 1 --2- 172.21.15.92:0/3421159125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12940ac440 0x7f12940ab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:39.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f1293fff700 1 -- 172.21.15.92:0/3421159125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12940ac440 msgr2=0x7f12940ab150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:39.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f1293fff700 1 --2- 172.21.15.92:0/3421159125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12940ac440 0x7f12940ab150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f1293fff700 1 -- 172.21.15.92:0/3421159125 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1294068be0 msgr2=0x7f1294168960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:39.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f1293fff700 1 --2- 172.21.15.92:0/3421159125 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1294068be0 0x7f1294168960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f1293fff700 1 -- 172.21.15.92:0/3421159125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128800b790 con 0x7f1294068810 2023-10-06T10:38:39.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f1298ea3700 1 --2- 172.21.15.92:0/3421159125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12940ac440 0x7f12940ab150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:39.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f12937fe700 1 --2- 172.21.15.92:0/3421159125 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1294068be0 0x7f1294168960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:39.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f1293fff700 1 --2- 172.21.15.92:0/3421159125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1294068810 0x7f1294165e90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f128400d810 tx=0x7f1284008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:39.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f127bfff700 1 -- 172.21.15.92:0/3421159125 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12840125f0 con 0x7f1294068810 2023-10-06T10:38:39.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f127bfff700 1 -- 172.21.15.92:0/3421159125 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1284012750 con 0x7f1294068810 2023-10-06T10:38:39.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f129a906700 1 -- 172.21.15.92:0/3421159125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12941696e0 con 0x7f1294068810 2023-10-06T10:38:39.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f127bfff700 1 -- 172.21.15.92:0/3421159125 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1284012b10 con 0x7f1294068810 2023-10-06T10:38:39.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.230+0000 7f129a906700 1 -- 172.21.15.92:0/3421159125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12940aac30 con 0x7f1294068810 2023-10-06T10:38:39.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.231+0000 7f129a906700 1 -- 172.21.15.92:0/3421159125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1294064a90 con 0x7f1294068810 2023-10-06T10:38:39.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.231+0000 7f127bfff700 1 -- 172.21.15.92:0/3421159125 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f128401e460 con 0x7f1294068810 2023-10-06T10:38:39.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.232+0000 7f127bfff700 1 --2- 172.21.15.92:0/3421159125 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1280066f30 0x7f12800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:39.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.232+0000 7f127bfff700 1 -- 172.21.15.92:0/3421159125 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f1284085010 con 0x7f1294068810 2023-10-06T10:38:39.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.232+0000 7f12937fe700 1 --2- 172.21.15.92:0/3421159125 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1280066f30 0x7f12800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:39.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.232+0000 7f12937fe700 1 --2- 172.21.15.92:0/3421159125 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1280066f30 0x7f12800693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f12940b1270 tx=0x7f128801b000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:39.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.236+0000 7f127bfff700 1 -- 172.21.15.92:0/3421159125 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1284051e30 con 0x7f1294068810 2023-10-06T10:38:39.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.421+0000 7f129a906700 1 -- 172.21.15.92:0/3421159125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12940aafa0 con 0x7f1294068810 2023-10-06T10:38:39.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.422+0000 7f127bfff700 1 -- 172.21.15.92:0/3421159125 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18707 (secure 0 0 0) 0x7f1284055450 con 0x7f1294068810 2023-10-06T10:38:39.423 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:39.423 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:21.760233+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:38:14.368713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T10:38:15.692344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T10:38:19.382553+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:38:39.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.424+0000 7f129a906700 1 -- 172.21.15.92:0/3421159125 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1280066f30 msgr2=0x7f12800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:39.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.424+0000 7f129a906700 1 --2- 172.21.15.92:0/3421159125 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1280066f30 0x7f12800693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f12940b1270 tx=0x7f128801b000 comp rx=0 tx=0).stop 2023-10-06T10:38:39.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.424+0000 7f129a906700 1 -- 172.21.15.92:0/3421159125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1294068810 msgr2=0x7f1294165e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:39.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.424+0000 7f129a906700 1 --2- 172.21.15.92:0/3421159125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1294068810 0x7f1294165e90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f128400d810 tx=0x7f1284008040 comp rx=0 tx=0).stop 2023-10-06T10:38:39.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.424+0000 7f129a906700 1 -- 172.21.15.92:0/3421159125 shutdown_connections 2023-10-06T10:38:39.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.424+0000 7f129a906700 1 --2- 172.21.15.92:0/3421159125 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1280066f30 0x7f12800693e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.424+0000 7f129a906700 1 --2- 172.21.15.92:0/3421159125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12940ac440 0x7f12940ab150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.424+0000 7f129a906700 1 --2- 172.21.15.92:0/3421159125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1294068810 0x7f1294165e90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.424+0000 7f129a906700 1 --2- 172.21.15.92:0/3421159125 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1294068be0 0x7f1294168960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.424+0000 7f129a906700 1 -- 172.21.15.92:0/3421159125 >> 172.21.15.92:0/3421159125 conn(0x7f129406de40 msgr2=0x7f12940ae000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:39.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.424+0000 7f129a906700 1 -- 172.21.15.92:0/3421159125 shutdown_connections 2023-10-06T10:38:39.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.424+0000 7f129a906700 1 -- 172.21.15.92:0/3421159125 wait complete. 2023-10-06T10:38:39.437 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T10:38:09.680559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T10:38:10.120786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T10:38:14.368713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T10:38:15.692344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T10:38:19.382553+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T10:38:39.438 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T10:38:39.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.556+0000 7fb652a56700 1 Processor -- start 2023-10-06T10:38:39.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.556+0000 7fb652a56700 1 -- start start 2023-10-06T10:38:39.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.556+0000 7fb652a56700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb64c159bd0 0x7fb64c0d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:39.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.556+0000 7fb652a56700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb64c0b5ef0 con 0x7fb64c0cd940 2023-10-06T10:38:39.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.556+0000 7fb652a56700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb64c0b3600 con 0x7fb64c0d0810 2023-10-06T10:38:39.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.556+0000 7fb652a56700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb64c0d0c20 con 0x7fb64c159bd0 2023-10-06T10:38:39.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.557+0000 7fb64bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb64c0cd940 0x7fb64c0b4a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40194/0 (socket says 172.21.15.92:40194) 2023-10-06T10:38:39.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.557+0000 7fb64b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb64c159bd0 0x7fb64c0d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:39.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.557+0000 7fb64bfff700 1 -- 172.21.15.92:0/1546467016 learned_addr learned my addr 172.21.15.92:0/1546467016 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:39.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.557+0000 7fb64b7fe700 1 -- 172.21.15.92:0/1546467016 >> v1:172.21.15.124:6789/0 conn(0x7fb64c0d0810 legacy=0x7fb64c163200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:39.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.557+0000 7fb64b7fe700 1 -- 172.21.15.92:0/1546467016 >> v1:172.21.15.92:6789/0 conn(0x7fb64c0cd940 legacy=0x7fb64c0b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:39.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.557+0000 7fb64affd700 1 -- 172.21.15.92:0/1546467016 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 448182017 0 0) 0x7fb64c0b5ef0 con 0x7fb64c0cd940 2023-10-06T10:38:39.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.557+0000 7fb64b7fe700 1 -- 172.21.15.92:0/1546467016 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64c0b6380 con 0x7fb64c159bd0 2023-10-06T10:38:39.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.557+0000 7fb64b7fe700 1 --2- 172.21.15.92:0/1546467016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c159bd0 0x7fb64c0d02d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb63c0098e0 tx=0x7fb63c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce8996429bacdb11 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:39.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.557+0000 7fb64affd700 1 -- 172.21.15.92:0/1546467016 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c009e80 con 0x7fb64c159bd0 2023-10-06T10:38:39.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.557+0000 7fb64affd700 1 -- 172.21.15.92:0/1546467016 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63c00b820 con 0x7fb64c159bd0 2023-10-06T10:38:39.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.557+0000 7fb64affd700 1 -- 172.21.15.92:0/1546467016 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c00bba0 con 0x7fb64c159bd0 2023-10-06T10:38:39.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.558+0000 7fb652a56700 1 -- 172.21.15.92:0/1546467016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c159bd0 msgr2=0x7fb64c0d02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:39.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.558+0000 7fb652a56700 1 --2- 172.21.15.92:0/1546467016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c159bd0 0x7fb64c0d02d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb63c0098e0 tx=0x7fb63c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.558+0000 7fb652a56700 1 -- 172.21.15.92:0/1546467016 shutdown_connections 2023-10-06T10:38:39.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.558+0000 7fb652a56700 1 --2- 172.21.15.92:0/1546467016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c159bd0 0x7fb64c0d02d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.558+0000 7fb652a56700 1 -- 172.21.15.92:0/1546467016 >> 172.21.15.92:0/1546467016 conn(0x7fb64c06ece0 msgr2=0x7fb64c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:39.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.558+0000 7fb652a56700 1 -- 172.21.15.92:0/1546467016 shutdown_connections 2023-10-06T10:38:39.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.558+0000 7fb652a56700 1 -- 172.21.15.92:0/1546467016 wait complete. 2023-10-06T10:38:39.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.559+0000 7fb652a56700 1 Processor -- start 2023-10-06T10:38:39.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.559+0000 7fb652a56700 1 -- start start 2023-10-06T10:38:39.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.560+0000 7fb652a56700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb64c0cd940 0x7fb64c152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:39.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.560+0000 7fb652a56700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb64c0d0810 0x7fb64c169980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:39.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.560+0000 7fb652a56700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c159bd0 0x7fb64c0c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:39.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.560+0000 7fb652a56700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c156240 con 0x7fb64c159bd0 2023-10-06T10:38:39.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.560+0000 7fb652a56700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c155630 con 0x7fb64c0d0810 2023-10-06T10:38:39.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.560+0000 7fb652a56700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c155910 con 0x7fb64c0cd940 2023-10-06T10:38:39.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.560+0000 7fb650ff3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c159bd0 0x7fb64c0c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:39.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.560+0000 7fb650ff3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c159bd0 0x7fb64c0c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37056/0 (socket says 172.21.15.92:37056) 2023-10-06T10:38:39.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.560+0000 7fb650ff3700 1 -- 172.21.15.92:0/2958074898 learned_addr learned my addr 172.21.15.92:0/2958074898 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:39.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.560+0000 7fb64bfff700 1 --2- 172.21.15.92:0/2958074898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb64c0cd940 0x7fb64c152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:39.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.560+0000 7fb650ff3700 1 -- 172.21.15.92:0/2958074898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb64c0cd940 msgr2=0x7fb64c152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:39.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.560+0000 7fb64b7fe700 1 --2- 172.21.15.92:0/2958074898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb64c0d0810 0x7fb64c169980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:39.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.560+0000 7fb650ff3700 1 --2- 172.21.15.92:0/2958074898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb64c0cd940 0x7fb64c152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.560+0000 7fb650ff3700 1 -- 172.21.15.92:0/2958074898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb64c0d0810 msgr2=0x7fb64c169980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:39.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.560+0000 7fb650ff3700 1 --2- 172.21.15.92:0/2958074898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb64c0d0810 0x7fb64c169980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.560+0000 7fb650ff3700 1 -- 172.21.15.92:0/2958074898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c009420 con 0x7fb64c159bd0 2023-10-06T10:38:39.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.560+0000 7fb64bfff700 1 --2- 172.21.15.92:0/2958074898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb64c0cd940 0x7fb64c152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.560+0000 7fb650ff3700 1 --2- 172.21.15.92:0/2958074898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c159bd0 0x7fb64c0c75c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb64400c930 tx=0x7fb6440026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.560+0000 7fb633fff700 1 -- 172.21.15.92:0/2958074898 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64400ce70 con 0x7fb64c159bd0 2023-10-06T10:38:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.560+0000 7fb633fff700 1 -- 172.21.15.92:0/2958074898 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64400a7e0 con 0x7fb64c159bd0 2023-10-06T10:38:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.560+0000 7fb633fff700 1 -- 172.21.15.92:0/2958074898 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64400aba0 con 0x7fb64c159bd0 2023-10-06T10:38:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.560+0000 7fb652a56700 1 -- 172.21.15.92:0/2958074898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb64c155bf0 con 0x7fb64c159bd0 2023-10-06T10:38:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.561+0000 7fb652a56700 1 -- 172.21.15.92:0/2958074898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb64c153080 con 0x7fb64c159bd0 2023-10-06T10:38:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.561+0000 7fb652a56700 1 -- 172.21.15.92:0/2958074898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb64c064a90 con 0x7fb64c159bd0 2023-10-06T10:38:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.562+0000 7fb633fff700 1 -- 172.21.15.92:0/2958074898 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb644012420 con 0x7fb64c159bd0 2023-10-06T10:38:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.562+0000 7fb633fff700 1 --2- 172.21.15.92:0/2958074898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb638066f30 0x7fb6380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.562+0000 7fb633fff700 1 -- 172.21.15.92:0/2958074898 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb644084dc0 con 0x7fb64c159bd0 2023-10-06T10:38:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.562+0000 7fb64bfff700 1 --2- 172.21.15.92:0/2958074898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb638066f30 0x7fb6380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.563+0000 7fb64bfff700 1 --2- 172.21.15.92:0/2958074898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb638066f30 0x7fb6380693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb64c1574b0 tx=0x7fb640009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.566+0000 7fb633fff700 1 -- 172.21.15.92:0/2958074898 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb644051c90 con 0x7fb64c159bd0 2023-10-06T10:38:39.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.747+0000 7fb652a56700 1 -- 172.21.15.92:0/2958074898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb64c156380 con 0x7fb64c159bd0 2023-10-06T10:38:39.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.748+0000 7fb633fff700 1 -- 172.21.15.92:0/2958074898 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb6440552b0 con 0x7fb64c159bd0 2023-10-06T10:38:39.749 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T10:38:39.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.750+0000 7fb652a56700 1 -- 172.21.15.92:0/2958074898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb638066f30 msgr2=0x7fb6380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:39.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.750+0000 7fb652a56700 1 --2- 172.21.15.92:0/2958074898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb638066f30 0x7fb6380693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb64c1574b0 tx=0x7fb640009040 comp rx=0 tx=0).stop 2023-10-06T10:38:39.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.750+0000 7fb652a56700 1 -- 172.21.15.92:0/2958074898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c159bd0 msgr2=0x7fb64c0c75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:39.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.750+0000 7fb652a56700 1 --2- 172.21.15.92:0/2958074898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c159bd0 0x7fb64c0c75c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb64400c930 tx=0x7fb6440026e0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.750+0000 7fb652a56700 1 -- 172.21.15.92:0/2958074898 shutdown_connections 2023-10-06T10:38:39.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.750+0000 7fb652a56700 1 --2- 172.21.15.92:0/2958074898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb638066f30 0x7fb6380693e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.750+0000 7fb652a56700 1 --2- 172.21.15.92:0/2958074898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c159bd0 0x7fb64c0c75c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.750+0000 7fb652a56700 1 --2- 172.21.15.92:0/2958074898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb64c0cd940 0x7fb64c152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.750+0000 7fb652a56700 1 --2- 172.21.15.92:0/2958074898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb64c0d0810 0x7fb64c169980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.750+0000 7fb652a56700 1 -- 172.21.15.92:0/2958074898 >> 172.21.15.92:0/2958074898 conn(0x7fb64c06ece0 msgr2=0x7fb64c0cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:39.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.750+0000 7fb652a56700 1 -- 172.21.15.92:0/2958074898 shutdown_connections 2023-10-06T10:38:39.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.750+0000 7fb652a56700 1 -- 172.21.15.92:0/2958074898 wait complete. 2023-10-06T10:38:39.764 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T10:38:39.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.881+0000 7fe9bb084700 1 Processor -- start 2023-10-06T10:38:39.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.881+0000 7fe9bb084700 1 -- start start 2023-10-06T10:38:39.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.881+0000 7fe9bb084700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe9b41594d0 0x7fe9b40cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:39.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.881+0000 7fe9bb084700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe9b41598a0 0x7fe9b40cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:39.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.882+0000 7fe9bb084700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b40cf330 con 0x7fe9b40ce7e0 2023-10-06T10:38:39.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.882+0000 7fe9bb084700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b40cf5d0 con 0x7fe9b41594d0 2023-10-06T10:38:39.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.882+0000 7fe9bb084700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b40cf8b0 con 0x7fe9b41598a0 2023-10-06T10:38:39.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.882+0000 7fe9b8e20700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe9b41594d0 0x7fe9b40cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:39.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.882+0000 7fe9b8e20700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe9b41594d0 0x7fe9b40cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37064/0 (socket says 172.21.15.92:37064) 2023-10-06T10:38:39.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.882+0000 7fe9b8e20700 1 -- 172.21.15.92:0/1021201277 learned_addr learned my addr 172.21.15.92:0/1021201277 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:39.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.882+0000 7fe9b3fff700 1 --2- 172.21.15.92:0/1021201277 >> v2:172.21.15.124:3300/0 conn(0x7fe9b41598a0 0x7fe9b40cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:39.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.882+0000 7fe9b8e20700 1 -- 172.21.15.92:0/1021201277 >> v2:172.21.15.124:3300/0 conn(0x7fe9b41598a0 msgr2=0x7fe9b40cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.882+0000 7fe9b8e20700 1 --2- 172.21.15.92:0/1021201277 >> v2:172.21.15.124:3300/0 conn(0x7fe9b41598a0 0x7fe9b40cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.882+0000 7fe9b8e20700 1 -- 172.21.15.92:0/1021201277 >> v1:172.21.15.124:6789/0 conn(0x7fe9b40ce7e0 legacy=0x7fe9b40cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.882+0000 7fe9b8e20700 1 -- 172.21.15.92:0/1021201277 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b40cfb90 con 0x7fe9b41594d0 2023-10-06T10:38:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.882+0000 7fe9b8e20700 1 --2- 172.21.15.92:0/1021201277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9b41594d0 0x7fe9b40cb560 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40c7370 tx=0x7fe9a4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a597424f44aa4ee1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.882+0000 7fe9b37fe700 1 -- 172.21.15.92:0/1021201277 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a400e070 con 0x7fe9b41594d0 2023-10-06T10:38:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.882+0000 7fe9b37fe700 1 -- 172.21.15.92:0/1021201277 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a400ba20 con 0x7fe9b41594d0 2023-10-06T10:38:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.882+0000 7fe9b37fe700 1 -- 172.21.15.92:0/1021201277 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a40092e0 con 0x7fe9b41594d0 2023-10-06T10:38:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.883+0000 7fe9bb084700 1 -- 172.21.15.92:0/1021201277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9b41594d0 msgr2=0x7fe9b40cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.883+0000 7fe9bb084700 1 --2- 172.21.15.92:0/1021201277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9b41594d0 0x7fe9b40cb560 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40c7370 tx=0x7fe9a4009840 comp rx=0 tx=0).stop 2023-10-06T10:38:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.883+0000 7fe9bb084700 1 -- 172.21.15.92:0/1021201277 shutdown_connections 2023-10-06T10:38:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.883+0000 7fe9bb084700 1 --2- 172.21.15.92:0/1021201277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9b41594d0 0x7fe9b40cb560 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.883+0000 7fe9bb084700 1 --2- 172.21.15.92:0/1021201277 >> v2:172.21.15.124:3300/0 conn(0x7fe9b41598a0 0x7fe9b40cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.883+0000 7fe9bb084700 1 -- 172.21.15.92:0/1021201277 >> 172.21.15.92:0/1021201277 conn(0x7fe9b4089e50 msgr2=0x7fe9b4099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.883+0000 7fe9bb084700 1 -- 172.21.15.92:0/1021201277 shutdown_connections 2023-10-06T10:38:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.883+0000 7fe9bb084700 1 -- 172.21.15.92:0/1021201277 wait complete. 2023-10-06T10:38:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.884+0000 7fe9bb084700 1 Processor -- start 2023-10-06T10:38:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.884+0000 7fe9bb084700 1 -- start start 2023-10-06T10:38:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.885+0000 7fe9bb084700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9b40ce7e0 0x7fe9b414f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.885+0000 7fe9bb084700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9b41594d0 0x7fe9b409caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.885+0000 7fe9bb084700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9b41598a0 0x7fe9b409cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.885+0000 7fe9bb084700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b409e180 con 0x7fe9b41598a0 2023-10-06T10:38:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.885+0000 7fe9bb084700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b409d600 con 0x7fe9b40ce7e0 2023-10-06T10:38:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.885+0000 7fe9bb084700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b409d8e0 con 0x7fe9b41594d0 2023-10-06T10:38:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.885+0000 7fe9b9621700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9b41598a0 0x7fe9b409cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.885+0000 7fe9b9621700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9b41598a0 0x7fe9b409cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37072/0 (socket says 172.21.15.92:37072) 2023-10-06T10:38:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.885+0000 7fe9b9621700 1 -- 172.21.15.92:0/1335532607 learned_addr learned my addr 172.21.15.92:0/1335532607 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.885+0000 7fe9b3fff700 1 --2- 172.21.15.92:0/1335532607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9b41594d0 0x7fe9b409caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.885+0000 7fe9b8e20700 1 --2- 172.21.15.92:0/1335532607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9b40ce7e0 0x7fe9b414f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.885+0000 7fe9b9621700 1 -- 172.21.15.92:0/1335532607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9b41594d0 msgr2=0x7fe9b409caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.885+0000 7fe9b9621700 1 --2- 172.21.15.92:0/1335532607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9b41594d0 0x7fe9b409caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.885+0000 7fe9b9621700 1 -- 172.21.15.92:0/1335532607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9b40ce7e0 msgr2=0x7fe9b414f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.885+0000 7fe9b9621700 1 --2- 172.21.15.92:0/1335532607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9b40ce7e0 0x7fe9b414f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.885+0000 7fe9b9621700 1 -- 172.21.15.92:0/1335532607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a4009ae0 con 0x7fe9b41598a0 2023-10-06T10:38:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.885+0000 7fe9b3fff700 1 --2- 172.21.15.92:0/1335532607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9b41594d0 0x7fe9b409caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.885+0000 7fe9b9621700 1 --2- 172.21.15.92:0/1335532607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9b41598a0 0x7fe9b409cfe0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00c560 tx=0x7fe9ac007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.886+0000 7fe9b0ff9700 1 -- 172.21.15.92:0/1335532607 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac011590 con 0x7fe9b41598a0 2023-10-06T10:38:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.886+0000 7fe9b0ff9700 1 -- 172.21.15.92:0/1335532607 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ac0116f0 con 0x7fe9b41598a0 2023-10-06T10:38:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.886+0000 7fe9bb084700 1 -- 172.21.15.92:0/1335532607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b409dbc0 con 0x7fe9b41598a0 2023-10-06T10:38:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.886+0000 7fe9b0ff9700 1 -- 172.21.15.92:0/1335532607 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac011ab0 con 0x7fe9b41598a0 2023-10-06T10:38:39.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.886+0000 7fe9bb084700 1 -- 172.21.15.92:0/1335532607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b414f680 con 0x7fe9b41598a0 2023-10-06T10:38:39.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.886+0000 7fe9bb084700 1 -- 172.21.15.92:0/1335532607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b4064a90 con 0x7fe9b41598a0 2023-10-06T10:38:39.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.887+0000 7fe9b0ff9700 1 -- 172.21.15.92:0/1335532607 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe9ac01b3b0 con 0x7fe9b41598a0 2023-10-06T10:38:39.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.887+0000 7fe9b0ff9700 1 --2- 172.21.15.92:0/1335532607 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe9a0066f30 0x7fe9a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:39.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.887+0000 7fe9b0ff9700 1 -- 172.21.15.92:0/1335532607 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe9ac083e60 con 0x7fe9b41598a0 2023-10-06T10:38:39.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.887+0000 7fe9b8e20700 1 --2- 172.21.15.92:0/1335532607 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe9a0066f30 0x7fe9a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:39.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.888+0000 7fe9b8e20700 1 --2- 172.21.15.92:0/1335532607 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe9a0066f30 0x7fe9a00693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009880 tx=0x7fe9a4018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:39.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:39.892+0000 7fe9b0ff9700 1 -- 172.21.15.92:0/1335532607 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe9ac050c80 con 0x7fe9b41598a0 2023-10-06T10:38:40.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.072+0000 7fe9bb084700 1 -- 172.21.15.92:0/1335532607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe9b40cfe70 con 0x7fe9b41598a0 2023-10-06T10:38:40.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.073+0000 7fe9b0ff9700 1 -- 172.21.15.92:0/1335532607 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe9ac017070 con 0x7fe9b41598a0 2023-10-06T10:38:40.074 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T10:38:40.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.075+0000 7fe9bb084700 1 -- 172.21.15.92:0/1335532607 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe9a0066f30 msgr2=0x7fe9a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:40.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.075+0000 7fe9bb084700 1 --2- 172.21.15.92:0/1335532607 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe9a0066f30 0x7fe9a00693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009880 tx=0x7fe9a4018000 comp rx=0 tx=0).stop 2023-10-06T10:38:40.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.075+0000 7fe9bb084700 1 -- 172.21.15.92:0/1335532607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9b41598a0 msgr2=0x7fe9b409cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:40.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.075+0000 7fe9bb084700 1 --2- 172.21.15.92:0/1335532607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9b41598a0 0x7fe9b409cfe0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00c560 tx=0x7fe9ac007ef0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.075+0000 7fe9bb084700 1 -- 172.21.15.92:0/1335532607 shutdown_connections 2023-10-06T10:38:40.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.075+0000 7fe9bb084700 1 --2- 172.21.15.92:0/1335532607 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe9a0066f30 0x7fe9a00693e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.075+0000 7fe9bb084700 1 --2- 172.21.15.92:0/1335532607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9b41598a0 0x7fe9b409cfe0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.075+0000 7fe9bb084700 1 --2- 172.21.15.92:0/1335532607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9b40ce7e0 0x7fe9b414f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.075+0000 7fe9bb084700 1 --2- 172.21.15.92:0/1335532607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9b41594d0 0x7fe9b409caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.075+0000 7fe9bb084700 1 -- 172.21.15.92:0/1335532607 >> 172.21.15.92:0/1335532607 conn(0x7fe9b4089e50 msgr2=0x7fe9b40c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:40.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.075+0000 7fe9bb084700 1 -- 172.21.15.92:0/1335532607 shutdown_connections 2023-10-06T10:38:40.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.075+0000 7fe9bb084700 1 -- 172.21.15.92:0/1335532607 wait complete. 2023-10-06T10:38:40.088 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T10:38:40.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.204+0000 7f2b5359e700 1 Processor -- start 2023-10-06T10:38:40.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.204+0000 7f2b5359e700 1 -- start start 2023-10-06T10:38:40.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.205+0000 7f2b5359e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2b540cd3c0 0x7f2b540ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:40.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.205+0000 7f2b5359e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b540b29f0 0x7f2b540cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:40.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.205+0000 7f2b5359e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b540cc970 con 0x7f2b540b2620 2023-10-06T10:38:40.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.205+0000 7f2b5359e700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b540cf520 con 0x7f2b540cd3c0 2023-10-06T10:38:40.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.205+0000 7f2b5359e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b540cbd60 con 0x7f2b540b29f0 2023-10-06T10:38:40.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.205+0000 7f2b5259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2b540b2620 0x7f2b540cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40200/0 (socket says 172.21.15.92:40200) 2023-10-06T10:38:40.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.205+0000 7f2b5259c700 1 -- 172.21.15.92:0/1218108415 learned_addr learned my addr 172.21.15.92:0/1218108415 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:40.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.205+0000 7f2b51d9b700 1 --2- 172.21.15.92:0/1218108415 >> v2:172.21.15.124:3300/0 conn(0x7f2b540cd3c0 0x7f2b540ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:40.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.205+0000 7f2b52d9d700 1 --2- 172.21.15.92:0/1218108415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b540b29f0 0x7f2b540cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:40.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.205+0000 7f2b5159a700 1 -- 172.21.15.92:0/1218108415 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134820295 0 0) 0x7f2b540cc970 con 0x7f2b540b2620 2023-10-06T10:38:40.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.205+0000 7f2b5159a700 1 -- 172.21.15.92:0/1218108415 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b3c003570 con 0x7f2b540b2620 2023-10-06T10:38:40.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.205+0000 7f2b5159a700 1 -- 172.21.15.92:0/1218108415 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2832973440 0 0) 0x7f2b3c003570 con 0x7f2b540b2620 2023-10-06T10:38:40.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.205+0000 7f2b5159a700 1 -- 172.21.15.92:0/1218108415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b540b29f0 msgr2=0x7f2b540cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:40.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.205+0000 7f2b5159a700 1 --2- 172.21.15.92:0/1218108415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b540b29f0 0x7f2b540cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.205+0000 7f2b5159a700 1 -- 172.21.15.92:0/1218108415 >> v2:172.21.15.124:3300/0 conn(0x7f2b540cd3c0 msgr2=0x7f2b540ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:40.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.205+0000 7f2b5159a700 1 --2- 172.21.15.92:0/1218108415 >> v2:172.21.15.124:3300/0 conn(0x7f2b540cd3c0 0x7f2b540ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.205+0000 7f2b5159a700 1 -- 172.21.15.92:0/1218108415 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b540cc040 con 0x7f2b540b2620 2023-10-06T10:38:40.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.206+0000 7f2b5159a700 1 -- 172.21.15.92:0/1218108415 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f2b440015f0 con 0x7f2b540b2620 2023-10-06T10:38:40.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.206+0000 7f2b5159a700 1 -- 172.21.15.92:0/1218108415 >> v1:172.21.15.92:6789/0 conn(0x7f2b540b2620 legacy=0x7f2b540cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:40.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.206+0000 7f2b5159a700 1 --2- 172.21.15.92:0/1218108415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b3c003a60 0x7f2b3c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:40.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.206+0000 7f2b5159a700 1 --2- 172.21.15.92:0/1218108415 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b3c004810 0x7f2b3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:40.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.206+0000 7f2b5159a700 1 --2- 172.21.15.92:0/1218108415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b540b29f0 0x7f2b3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:40.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.206+0000 7f2b5159a700 1 -- 172.21.15.92:0/1218108415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b540cbd60 con 0x7f2b3c003a60 2023-10-06T10:38:40.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.206+0000 7f2b5159a700 1 -- 172.21.15.92:0/1218108415 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c00ba60 con 0x7f2b3c004810 2023-10-06T10:38:40.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.206+0000 7f2b5159a700 1 -- 172.21.15.92:0/1218108415 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c00bba0 con 0x7f2b540b29f0 2023-10-06T10:38:40.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.206+0000 7f2b5259c700 1 --2- 172.21.15.92:0/1218108415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b3c003a60 0x7f2b3c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:40.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.206+0000 7f2b51d9b700 1 --2- 172.21.15.92:0/1218108415 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b3c004810 0x7f2b3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:40.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.206+0000 7f2b52d9d700 1 --2- 172.21.15.92:0/1218108415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b540b29f0 0x7f2b3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:40.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.206+0000 7f2b5259c700 1 -- 172.21.15.92:0/1218108415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b540b29f0 msgr2=0x7f2b3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:40.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.206+0000 7f2b5259c700 1 --2- 172.21.15.92:0/1218108415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b540b29f0 0x7f2b3c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.206+0000 7f2b5259c700 1 -- 172.21.15.92:0/1218108415 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b3c004810 msgr2=0x7f2b3c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:40.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.206+0000 7f2b5259c700 1 --2- 172.21.15.92:0/1218108415 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b3c004810 0x7f2b3c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.206+0000 7f2b5259c700 1 -- 172.21.15.92:0/1218108415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3c003570 con 0x7f2b3c003a60 2023-10-06T10:38:40.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.206+0000 7f2b52d9d700 1 --2- 172.21.15.92:0/1218108415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b540b29f0 0x7f2b3c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:40.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.206+0000 7f2b5259c700 1 --2- 172.21.15.92:0/1218108415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b3c003a60 0x7f2b3c003e70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400c3f0 tx=0x7f2b44007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ea5d39ad740a56 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:40.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.206+0000 7f2b5159a700 1 -- 172.21.15.92:0/1218108415 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4400cc80 con 0x7f2b3c003a60 2023-10-06T10:38:40.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.206+0000 7f2b5159a700 1 -- 172.21.15.92:0/1218108415 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4400cde0 con 0x7f2b3c003a60 2023-10-06T10:38:40.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.206+0000 7f2b5159a700 1 -- 172.21.15.92:0/1218108415 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44007320 con 0x7f2b3c003a60 2023-10-06T10:38:40.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.207+0000 7f2b5359e700 1 -- 172.21.15.92:0/1218108415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b3c003a60 msgr2=0x7f2b3c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:40.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.207+0000 7f2b5359e700 1 --2- 172.21.15.92:0/1218108415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b3c003a60 0x7f2b3c003e70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400c3f0 tx=0x7f2b44007920 comp rx=0 tx=0).stop 2023-10-06T10:38:40.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.207+0000 7f2b52d9d700 1 -- 172.21.15.92:0/1218108415 reap_dead start 2023-10-06T10:38:40.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.207+0000 7f2b5359e700 1 -- 172.21.15.92:0/1218108415 shutdown_connections 2023-10-06T10:38:40.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.207+0000 7f2b5359e700 1 -- 172.21.15.92:0/1218108415 >> 172.21.15.92:0/1218108415 conn(0x7f2b54099520 msgr2=0x7f2b5406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:40.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.207+0000 7f2b5359e700 1 -- 172.21.15.92:0/1218108415 shutdown_connections 2023-10-06T10:38:40.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.207+0000 7f2b5359e700 1 -- 172.21.15.92:0/1218108415 wait complete. 2023-10-06T10:38:40.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.208+0000 7f2b5359e700 1 Processor -- start 2023-10-06T10:38:40.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.208+0000 7f2b5359e700 1 -- start start 2023-10-06T10:38:40.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b5359e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b3c003a60 0x7f2b541548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:40.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b5359e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b540b3090 0x7f2b54152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:40.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b5359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b540b3500 0x7f2b54153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:40.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b5359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b540c8410 con 0x7f2b540b3500 2023-10-06T10:38:40.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b5359e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b540b4310 con 0x7f2b540b3090 2023-10-06T10:38:40.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b5359e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b540c80e0 con 0x7f2b3c003a60 2023-10-06T10:38:40.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b52d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b540b3500 0x7f2b54153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:40.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b52d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b540b3500 0x7f2b54153090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37090/0 (socket says 172.21.15.92:37090) 2023-10-06T10:38:40.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b52d9d700 1 -- 172.21.15.92:0/2400328985 learned_addr learned my addr 172.21.15.92:0/2400328985 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:40.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b5259c700 1 --2- 172.21.15.92:0/2400328985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b3c003a60 0x7f2b541548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:40.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b51d9b700 1 --2- 172.21.15.92:0/2400328985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b540b3090 0x7f2b54152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:40.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b52d9d700 1 -- 172.21.15.92:0/2400328985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b3c003a60 msgr2=0x7f2b541548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:40.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b52d9d700 1 --2- 172.21.15.92:0/2400328985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b3c003a60 0x7f2b541548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b52d9d700 1 -- 172.21.15.92:0/2400328985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b540b3090 msgr2=0x7f2b54152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:40.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b52d9d700 1 --2- 172.21.15.92:0/2400328985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b540b3090 0x7f2b54152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b52d9d700 1 -- 172.21.15.92:0/2400328985 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b540b3a70 con 0x7f2b540b3500 2023-10-06T10:38:40.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b51d9b700 1 --2- 172.21.15.92:0/2400328985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b540b3090 0x7f2b54152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:40.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b52d9d700 1 --2- 172.21.15.92:0/2400328985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b540b3500 0x7f2b54153090 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2b540b2e10 tx=0x7f2b4c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:40.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b427fc700 1 -- 172.21.15.92:0/2400328985 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c01b070 con 0x7f2b540b3500 2023-10-06T10:38:40.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b427fc700 1 -- 172.21.15.92:0/2400328985 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4c00f040 con 0x7f2b540b3500 2023-10-06T10:38:40.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b5359e700 1 -- 172.21.15.92:0/2400328985 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b540b3d50 con 0x7f2b540b3500 2023-10-06T10:38:40.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b427fc700 1 -- 172.21.15.92:0/2400328985 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c00e050 con 0x7f2b540b3500 2023-10-06T10:38:40.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.209+0000 7f2b5359e700 1 -- 172.21.15.92:0/2400328985 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b54158e30 con 0x7f2b540b3500 2023-10-06T10:38:40.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.210+0000 7f2b5359e700 1 -- 172.21.15.92:0/2400328985 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b540c8580 con 0x7f2b540b3500 2023-10-06T10:38:40.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.211+0000 7f2b427fc700 1 -- 172.21.15.92:0/2400328985 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2b4c003020 con 0x7f2b540b3500 2023-10-06T10:38:40.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.211+0000 7f2b427fc700 1 --2- 172.21.15.92:0/2400328985 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2b34066f60 0x7f2b34069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:40.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.211+0000 7f2b427fc700 1 -- 172.21.15.92:0/2400328985 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f2b4c084ed0 con 0x7f2b540b3500 2023-10-06T10:38:40.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.211+0000 7f2b5259c700 1 --2- 172.21.15.92:0/2400328985 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2b34066f60 0x7f2b34069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:40.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.211+0000 7f2b5259c700 1 --2- 172.21.15.92:0/2400328985 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2b34066f60 0x7f2b34069410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400c310 tx=0x7f2b44015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:40.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.215+0000 7f2b427fc700 1 -- 172.21.15.92:0/2400328985 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b540c8580 con 0x7f2b540b3500 2023-10-06T10:38:40.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.402+0000 7f2b5359e700 1 -- 172.21.15.92:0/2400328985 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2b540c8550 con 0x7f2b540b3500 2023-10-06T10:38:40.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.402+0000 7f2b427fc700 1 -- 172.21.15.92:0/2400328985 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2b4c051cf0 con 0x7f2b540b3500 2023-10-06T10:38:40.403 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T10:38:40.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.404+0000 7f2b5359e700 1 -- 172.21.15.92:0/2400328985 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2b34066f60 msgr2=0x7f2b34069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:40.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.404+0000 7f2b5359e700 1 --2- 172.21.15.92:0/2400328985 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2b34066f60 0x7f2b34069410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400c310 tx=0x7f2b44015000 comp rx=0 tx=0).stop 2023-10-06T10:38:40.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.404+0000 7f2b5359e700 1 -- 172.21.15.92:0/2400328985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b540b3500 msgr2=0x7f2b54153090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:40.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.404+0000 7f2b5359e700 1 --2- 172.21.15.92:0/2400328985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b540b3500 0x7f2b54153090 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2b540b2e10 tx=0x7f2b4c00b040 comp rx=0 tx=0).stop 2023-10-06T10:38:40.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.404+0000 7f2b5359e700 1 -- 172.21.15.92:0/2400328985 shutdown_connections 2023-10-06T10:38:40.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.404+0000 7f2b5359e700 1 --2- 172.21.15.92:0/2400328985 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2b34066f60 0x7f2b34069410 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.404+0000 7f2b5359e700 1 --2- 172.21.15.92:0/2400328985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b540b3500 0x7f2b54153090 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.404+0000 7f2b5359e700 1 --2- 172.21.15.92:0/2400328985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b3c003a60 0x7f2b541548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.404+0000 7f2b5359e700 1 --2- 172.21.15.92:0/2400328985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b540b3090 0x7f2b54152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.404+0000 7f2b5359e700 1 -- 172.21.15.92:0/2400328985 >> 172.21.15.92:0/2400328985 conn(0x7f2b54099520 msgr2=0x7f2b540cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:40.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.405+0000 7f2b5359e700 1 -- 172.21.15.92:0/2400328985 shutdown_connections 2023-10-06T10:38:40.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.405+0000 7f2b5359e700 1 -- 172.21.15.92:0/2400328985 wait complete. 2023-10-06T10:38:40.417 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T10:38:40.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.535+0000 7f73de079700 1 Processor -- start 2023-10-06T10:38:40.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.535+0000 7f73de079700 1 -- start start 2023-10-06T10:38:40.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.536+0000 7f73de079700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f73d80b0820 0x7f73d80686c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:40.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.536+0000 7f73de079700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f73d80c4450 0x7f73d80cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:40.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.536+0000 7f73de079700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d80c9670 con 0x7f73d80cc740 2023-10-06T10:38:40.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.536+0000 7f73de079700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d8159c00 con 0x7f73d80c4450 2023-10-06T10:38:40.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.536+0000 7f73de079700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d80c8a60 con 0x7f73d80b0820 2023-10-06T10:38:40.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.536+0000 7f73d6ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f73d80c4450 0x7f73d80cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:40.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.536+0000 7f73d6ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f73d80c4450 0x7f73d80cec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37098/0 (socket says 172.21.15.92:37098) 2023-10-06T10:38:40.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.536+0000 7f73d6ffd700 1 -- 172.21.15.92:0/3645486790 learned_addr learned my addr 172.21.15.92:0/3645486790 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:40.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.537+0000 7f73d77fe700 1 --2- 172.21.15.92:0/3645486790 >> v2:172.21.15.124:3300/0 conn(0x7f73d80b0820 0x7f73d80686c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:40.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.537+0000 7f73d6ffd700 1 -- 172.21.15.92:0/3645486790 >> v2:172.21.15.124:3300/0 conn(0x7f73d80b0820 msgr2=0x7f73d80686c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:40.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.537+0000 7f73d6ffd700 1 --2- 172.21.15.92:0/3645486790 >> v2:172.21.15.124:3300/0 conn(0x7f73d80b0820 0x7f73d80686c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.537+0000 7f73d6ffd700 1 -- 172.21.15.92:0/3645486790 >> v1:172.21.15.124:6789/0 conn(0x7f73d80cc740 legacy=0x7f73d80ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:40.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.537+0000 7f73d6ffd700 1 -- 172.21.15.92:0/3645486790 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d80c8d40 con 0x7f73d80c4450 2023-10-06T10:38:40.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.537+0000 7f73d6ffd700 1 --2- 172.21.15.92:0/3645486790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d80c4450 0x7f73d80cec30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f73d806e180 tx=0x7f73cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594d1daac67a76a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:40.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.537+0000 7f73d67fc700 1 -- 172.21.15.92:0/3645486790 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc00b820 con 0x7f73d80c4450 2023-10-06T10:38:40.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.537+0000 7f73d67fc700 1 -- 172.21.15.92:0/3645486790 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73cc00b980 con 0x7f73d80c4450 2023-10-06T10:38:40.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.537+0000 7f73d67fc700 1 -- 172.21.15.92:0/3645486790 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc00bd20 con 0x7f73d80c4450 2023-10-06T10:38:40.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.537+0000 7f73de079700 1 -- 172.21.15.92:0/3645486790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d80c4450 msgr2=0x7f73d80cec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:40.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.537+0000 7f73de079700 1 --2- 172.21.15.92:0/3645486790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d80c4450 0x7f73d80cec30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f73d806e180 tx=0x7f73cc009250 comp rx=0 tx=0).stop 2023-10-06T10:38:40.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.537+0000 7f73de079700 1 -- 172.21.15.92:0/3645486790 shutdown_connections 2023-10-06T10:38:40.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.537+0000 7f73de079700 1 --2- 172.21.15.92:0/3645486790 >> v2:172.21.15.124:3300/0 conn(0x7f73d80b0820 0x7f73d80686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.537+0000 7f73de079700 1 --2- 172.21.15.92:0/3645486790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d80c4450 0x7f73d80cec30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.537+0000 7f73de079700 1 -- 172.21.15.92:0/3645486790 >> 172.21.15.92:0/3645486790 conn(0x7f73d8087fc0 msgr2=0x7f73d80ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:40.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.538+0000 7f73de079700 1 -- 172.21.15.92:0/3645486790 shutdown_connections 2023-10-06T10:38:40.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.538+0000 7f73de079700 1 -- 172.21.15.92:0/3645486790 wait complete. 2023-10-06T10:38:40.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.539+0000 7f73de079700 1 Processor -- start 2023-10-06T10:38:40.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.539+0000 7f73de079700 1 -- start start 2023-10-06T10:38:40.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.539+0000 7f73de079700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73d80b0820 0x7f73d80c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:40.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.539+0000 7f73de079700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73d80c4450 0x7f73d8157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:40.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.539+0000 7f73de079700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d80cc740 0x7f73d80b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:40.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.540+0000 7f73de079700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d80c5f00 con 0x7f73d80cc740 2023-10-06T10:38:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.540+0000 7f73de079700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d815b450 con 0x7f73d80b0820 2023-10-06T10:38:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.540+0000 7f73de079700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d815c2b0 con 0x7f73d80c4450 2023-10-06T10:38:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.540+0000 7f73d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d80cc740 0x7f73d80b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.540+0000 7f73d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d80cc740 0x7f73d80b33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37110/0 (socket says 172.21.15.92:37110) 2023-10-06T10:38:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.540+0000 7f73d7fff700 1 -- 172.21.15.92:0/580716236 learned_addr learned my addr 172.21.15.92:0/580716236 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.540+0000 7f73d77fe700 1 --2- 172.21.15.92:0/580716236 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73d80b0820 0x7f73d80c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.540+0000 7f73d6ffd700 1 --2- 172.21.15.92:0/580716236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73d80c4450 0x7f73d8157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.540+0000 7f73d7fff700 1 -- 172.21.15.92:0/580716236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73d80c4450 msgr2=0x7f73d8157170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.540+0000 7f73d7fff700 1 --2- 172.21.15.92:0/580716236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73d80c4450 0x7f73d8157170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.540+0000 7f73d7fff700 1 -- 172.21.15.92:0/580716236 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73d80b0820 msgr2=0x7f73d80c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.540+0000 7f73d7fff700 1 --2- 172.21.15.92:0/580716236 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73d80b0820 0x7f73d80c59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.540+0000 7f73d7fff700 1 -- 172.21.15.92:0/580716236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73cc0094f0 con 0x7f73d80cc740 2023-10-06T10:38:40.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.540+0000 7f73d7fff700 1 --2- 172.21.15.92:0/580716236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d80cc740 0x7f73d80b33b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f73d000bc30 tx=0x7f73d0009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:40.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.540+0000 7f73bf7fe700 1 -- 172.21.15.92:0/580716236 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d000c840 con 0x7f73d80cc740 2023-10-06T10:38:40.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.540+0000 7f73bf7fe700 1 -- 172.21.15.92:0/580716236 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d000c9e0 con 0x7f73d80cc740 2023-10-06T10:38:40.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.540+0000 7f73de079700 1 -- 172.21.15.92:0/580716236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d81576b0 con 0x7f73d80cc740 2023-10-06T10:38:40.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.540+0000 7f73bf7fe700 1 -- 172.21.15.92:0/580716236 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d000cda0 con 0x7f73d80cc740 2023-10-06T10:38:40.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.540+0000 7f73de079700 1 -- 172.21.15.92:0/580716236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d815b980 con 0x7f73d80cc740 2023-10-06T10:38:40.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.541+0000 7f73de079700 1 -- 172.21.15.92:0/580716236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d8064a90 con 0x7f73d80cc740 2023-10-06T10:38:40.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.542+0000 7f73bf7fe700 1 -- 172.21.15.92:0/580716236 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f73d0014620 con 0x7f73d80cc740 2023-10-06T10:38:40.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.542+0000 7f73bf7fe700 1 --2- 172.21.15.92:0/580716236 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f73c4066f30 0x7f73c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:40.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.542+0000 7f73bf7fe700 1 -- 172.21.15.92:0/580716236 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f73d00848c0 con 0x7f73d80cc740 2023-10-06T10:38:40.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.542+0000 7f73d77fe700 1 --2- 172.21.15.92:0/580716236 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f73c4066f30 0x7f73c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:40.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.542+0000 7f73d77fe700 1 --2- 172.21.15.92:0/580716236 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f73c4066f30 0x7f73c40693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f73c8006aa0 tx=0x7f73c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:40.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.546+0000 7f73bf7fe700 1 -- 172.21.15.92:0/580716236 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73d004c5a0 con 0x7f73d80cc740 2023-10-06T10:38:40.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.729+0000 7f73de079700 1 -- 172.21.15.92:0/580716236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f73d804bb40 con 0x7f73d80cc740 2023-10-06T10:38:40.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.729+0000 7f73bf7fe700 1 -- 172.21.15.92:0/580716236 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f73d00516e0 con 0x7f73d80cc740 2023-10-06T10:38:40.730 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T10:38:40.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.731+0000 7f73de079700 1 -- 172.21.15.92:0/580716236 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f73c4066f30 msgr2=0x7f73c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:40.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.731+0000 7f73de079700 1 --2- 172.21.15.92:0/580716236 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f73c4066f30 0x7f73c40693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f73c8006aa0 tx=0x7f73c8008040 comp rx=0 tx=0).stop 2023-10-06T10:38:40.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.731+0000 7f73de079700 1 -- 172.21.15.92:0/580716236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d80cc740 msgr2=0x7f73d80b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.731+0000 7f73de079700 1 --2- 172.21.15.92:0/580716236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d80cc740 0x7f73d80b33b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f73d000bc30 tx=0x7f73d0009820 comp rx=0 tx=0).stop 2023-10-06T10:38:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.731+0000 7f73de079700 1 -- 172.21.15.92:0/580716236 shutdown_connections 2023-10-06T10:38:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.731+0000 7f73de079700 1 --2- 172.21.15.92:0/580716236 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f73c4066f30 0x7f73c40693e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.731+0000 7f73de079700 1 --2- 172.21.15.92:0/580716236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d80cc740 0x7f73d80b33b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.731+0000 7f73de079700 1 --2- 172.21.15.92:0/580716236 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73d80b0820 0x7f73d80c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.731+0000 7f73de079700 1 --2- 172.21.15.92:0/580716236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73d80c4450 0x7f73d8157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.731+0000 7f73de079700 1 -- 172.21.15.92:0/580716236 >> 172.21.15.92:0/580716236 conn(0x7f73d8087fc0 msgr2=0x7f73d8158f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.731+0000 7f73de079700 1 -- 172.21.15.92:0/580716236 shutdown_connections 2023-10-06T10:38:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.731+0000 7f73de079700 1 -- 172.21.15.92:0/580716236 wait complete. 2023-10-06T10:38:40.744 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T10:38:40.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.860+0000 7faea3f2a700 1 Processor -- start 2023-10-06T10:38:40.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.860+0000 7faea3f2a700 1 -- start start 2023-10-06T10:38:40.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.861+0000 7faea3f2a700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fae9c156740 0x7fae9c06e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:40.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.861+0000 7faea3f2a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fae9c06e8d0 0x7fae9c0adb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:40.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.861+0000 7faea3f2a700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9c0c84a0 con 0x7fae9c156370 2023-10-06T10:38:40.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.861+0000 7faea3f2a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae9c06f8d0 con 0x7fae9c06e8d0 2023-10-06T10:38:40.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.861+0000 7faea3f2a700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae9c06ed10 con 0x7fae9c156740 2023-10-06T10:38:40.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.861+0000 7faea24c7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fae9c06e8d0 0x7fae9c0adb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:40.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.861+0000 7faea24c7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fae9c06e8d0 0x7fae9c0adb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37122/0 (socket says 172.21.15.92:37122) 2023-10-06T10:38:40.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.861+0000 7faea24c7700 1 -- 172.21.15.92:0/3165477366 learned_addr learned my addr 172.21.15.92:0/3165477366 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:40.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.861+0000 7faea14c5700 1 --2- 172.21.15.92:0/3165477366 >> v2:172.21.15.124:3300/0 conn(0x7fae9c156740 0x7fae9c06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:40.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.862+0000 7faea24c7700 1 -- 172.21.15.92:0/3165477366 >> v2:172.21.15.124:3300/0 conn(0x7fae9c156740 msgr2=0x7fae9c06e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:40.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.862+0000 7faea24c7700 1 --2- 172.21.15.92:0/3165477366 >> v2:172.21.15.124:3300/0 conn(0x7fae9c156740 0x7fae9c06e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.862+0000 7faea24c7700 1 -- 172.21.15.92:0/3165477366 >> v1:172.21.15.124:6789/0 conn(0x7fae9c156370 legacy=0x7fae9c06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:40.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.862+0000 7faea24c7700 1 -- 172.21.15.92:0/3165477366 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c0c7890 con 0x7fae9c06e8d0 2023-10-06T10:38:40.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.862+0000 7faea14c5700 1 --2- 172.21.15.92:0/3165477366 >> v2:172.21.15.124:3300/0 conn(0x7fae9c156740 0x7fae9c06e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:40.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.862+0000 7faea24c7700 1 --2- 172.21.15.92:0/3165477366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae9c06e8d0 0x7fae9c0adb30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fae9c06e230 tx=0x7fae94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a0087aec9d2655 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:40.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.862+0000 7faea0cc4700 1 -- 172.21.15.92:0/3165477366 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9400b860 con 0x7fae9c06e8d0 2023-10-06T10:38:40.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.862+0000 7faea0cc4700 1 -- 172.21.15.92:0/3165477366 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9400b9c0 con 0x7fae9c06e8d0 2023-10-06T10:38:40.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.862+0000 7faea0cc4700 1 -- 172.21.15.92:0/3165477366 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9400bd60 con 0x7fae9c06e8d0 2023-10-06T10:38:40.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.862+0000 7faea3f2a700 1 -- 172.21.15.92:0/3165477366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae9c06e8d0 msgr2=0x7fae9c0adb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:40.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.862+0000 7faea3f2a700 1 --2- 172.21.15.92:0/3165477366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae9c06e8d0 0x7fae9c0adb30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fae9c06e230 tx=0x7fae94009250 comp rx=0 tx=0).stop 2023-10-06T10:38:40.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.863+0000 7faea3f2a700 1 -- 172.21.15.92:0/3165477366 shutdown_connections 2023-10-06T10:38:40.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.863+0000 7faea3f2a700 1 --2- 172.21.15.92:0/3165477366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae9c06e8d0 0x7fae9c0adb30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.863+0000 7faea3f2a700 1 --2- 172.21.15.92:0/3165477366 >> v2:172.21.15.124:3300/0 conn(0x7fae9c156740 0x7fae9c06e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.863+0000 7faea3f2a700 1 -- 172.21.15.92:0/3165477366 >> 172.21.15.92:0/3165477366 conn(0x7fae9c069150 msgr2=0x7fae9c0cd170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:40.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.863+0000 7faea3f2a700 1 -- 172.21.15.92:0/3165477366 shutdown_connections 2023-10-06T10:38:40.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.863+0000 7faea3f2a700 1 -- 172.21.15.92:0/3165477366 wait complete. 2023-10-06T10:38:40.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.864+0000 7faea3f2a700 1 Processor -- start 2023-10-06T10:38:40.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.864+0000 7faea3f2a700 1 -- start start 2023-10-06T10:38:40.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.864+0000 7faea3f2a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae9c06e8d0 0x7fae9c14c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:40.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.864+0000 7faea3f2a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae9c156370 0x7fae9c0c4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:40.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.864+0000 7faea3f2a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae9c156740 0x7fae9c0c2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:40.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.864+0000 7faea3f2a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c0caa80 con 0x7fae9c156740 2023-10-06T10:38:40.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.864+0000 7faea3f2a700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c0c14a0 con 0x7fae9c06e8d0 2023-10-06T10:38:40.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.864+0000 7faea3f2a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c0c1ce0 con 0x7fae9c156370 2023-10-06T10:38:40.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.865+0000 7faea24c7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae9c156740 0x7fae9c0c2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:40.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.865+0000 7faea24c7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae9c156740 0x7fae9c0c2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37128/0 (socket says 172.21.15.92:37128) 2023-10-06T10:38:40.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.865+0000 7faea24c7700 1 -- 172.21.15.92:0/3057187090 learned_addr learned my addr 172.21.15.92:0/3057187090 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:40.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.865+0000 7faea1cc6700 1 --2- 172.21.15.92:0/3057187090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae9c06e8d0 0x7fae9c14c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:40.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.865+0000 7faea14c5700 1 --2- 172.21.15.92:0/3057187090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae9c156370 0x7fae9c0c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:40.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.865+0000 7faea24c7700 1 -- 172.21.15.92:0/3057187090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae9c156370 msgr2=0x7fae9c0c4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:40.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.865+0000 7faea24c7700 1 --2- 172.21.15.92:0/3057187090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae9c156370 0x7fae9c0c4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.865+0000 7faea24c7700 1 -- 172.21.15.92:0/3057187090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae9c06e8d0 msgr2=0x7fae9c14c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:40.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.865+0000 7faea24c7700 1 --2- 172.21.15.92:0/3057187090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae9c06e8d0 0x7fae9c14c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:40.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.865+0000 7faea24c7700 1 -- 172.21.15.92:0/3057187090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae940094f0 con 0x7fae9c156740 2023-10-06T10:38:40.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.865+0000 7faea24c7700 1 --2- 172.21.15.92:0/3057187090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae9c156740 0x7fae9c0c2170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fae940090b0 tx=0x7fae94017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:40.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.865+0000 7fae91ffb700 1 -- 172.21.15.92:0/3057187090 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae94011e80 con 0x7fae9c156740 2023-10-06T10:38:40.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.865+0000 7fae91ffb700 1 -- 172.21.15.92:0/3057187090 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9401f070 con 0x7fae9c156740 2023-10-06T10:38:40.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.865+0000 7faea3f2a700 1 -- 172.21.15.92:0/3057187090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae9c0ca7e0 con 0x7fae9c156740 2023-10-06T10:38:40.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.865+0000 7fae91ffb700 1 -- 172.21.15.92:0/3057187090 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9401a850 con 0x7fae9c156740 2023-10-06T10:38:40.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.865+0000 7faea3f2a700 1 -- 172.21.15.92:0/3057187090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9c0c1690 con 0x7fae9c156740 2023-10-06T10:38:40.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.866+0000 7faea3f2a700 1 -- 172.21.15.92:0/3057187090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae9c064a90 con 0x7fae9c156740 2023-10-06T10:38:40.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.867+0000 7fae91ffb700 1 -- 172.21.15.92:0/3057187090 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fae9401ac00 con 0x7fae9c156740 2023-10-06T10:38:40.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.867+0000 7fae91ffb700 1 --2- 172.21.15.92:0/3057187090 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fae80066f30 0x7fae800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:40.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.867+0000 7fae91ffb700 1 -- 172.21.15.92:0/3057187090 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fae94021070 con 0x7fae9c156740 2023-10-06T10:38:40.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.867+0000 7faea1cc6700 1 --2- 172.21.15.92:0/3057187090 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fae80066f30 0x7fae800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:40.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.867+0000 7faea1cc6700 1 --2- 172.21.15.92:0/3057187090 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fae80066f30 0x7fae800693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fae9800c580 tx=0x7fae98001b20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:40.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:40.871+0000 7fae91ffb700 1 -- 172.21.15.92:0/3057187090 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae94053770 con 0x7fae9c156740 2023-10-06T10:38:41.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.052+0000 7faea3f2a700 1 -- 172.21.15.92:0/3057187090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fae9c0c1e20 con 0x7fae9c156740 2023-10-06T10:38:41.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.053+0000 7fae91ffb700 1 -- 172.21.15.92:0/3057187090 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fae94015020 con 0x7fae9c156740 2023-10-06T10:38:41.056 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T10:38:41.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.055+0000 7faea3f2a700 1 -- 172.21.15.92:0/3057187090 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fae80066f30 msgr2=0x7fae800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:41.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.055+0000 7faea3f2a700 1 --2- 172.21.15.92:0/3057187090 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fae80066f30 0x7fae800693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fae9800c580 tx=0x7fae98001b20 comp rx=0 tx=0).stop 2023-10-06T10:38:41.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.055+0000 7faea3f2a700 1 -- 172.21.15.92:0/3057187090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae9c156740 msgr2=0x7fae9c0c2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:41.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.055+0000 7faea3f2a700 1 --2- 172.21.15.92:0/3057187090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae9c156740 0x7fae9c0c2170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fae940090b0 tx=0x7fae94017000 comp rx=0 tx=0).stop 2023-10-06T10:38:41.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.055+0000 7faea3f2a700 1 -- 172.21.15.92:0/3057187090 shutdown_connections 2023-10-06T10:38:41.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.055+0000 7faea3f2a700 1 --2- 172.21.15.92:0/3057187090 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fae80066f30 0x7fae800693e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.055+0000 7faea3f2a700 1 --2- 172.21.15.92:0/3057187090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae9c156740 0x7fae9c0c2170 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.055+0000 7faea3f2a700 1 --2- 172.21.15.92:0/3057187090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae9c06e8d0 0x7fae9c14c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.055+0000 7faea3f2a700 1 --2- 172.21.15.92:0/3057187090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae9c156370 0x7fae9c0c4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.055+0000 7faea3f2a700 1 -- 172.21.15.92:0/3057187090 >> 172.21.15.92:0/3057187090 conn(0x7fae9c069150 msgr2=0x7fae9c0b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:41.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.055+0000 7faea3f2a700 1 -- 172.21.15.92:0/3057187090 shutdown_connections 2023-10-06T10:38:41.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.055+0000 7faea3f2a700 1 -- 172.21.15.92:0/3057187090 wait complete. 2023-10-06T10:38:41.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:38:41.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.180+0000 7f347cb39700 1 Processor -- start 2023-10-06T10:38:41.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.180+0000 7f347cb39700 1 -- start start 2023-10-06T10:38:41.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.181+0000 7f347cb39700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34780b49a0 0x7f34780aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:41.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.181+0000 7f347cb39700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f34780b0760 0x7f34780b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:41.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.181+0000 7f347cb39700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34780b0c60 con 0x7f3478159560 2023-10-06T10:38:41.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.181+0000 7f347cb39700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34780b0f40 con 0x7f34780b0760 2023-10-06T10:38:41.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.181+0000 7f347cb39700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34780b1220 con 0x7f34780b49a0 2023-10-06T10:38:41.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.181+0000 7f3476d9d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f34780b0760 0x7f34780b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:41.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.181+0000 7f346dd9b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34780b49a0 0x7f34780aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:41.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.181+0000 7f3476d9d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f34780b0760 0x7f34780b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:47756/0 (socket says 172.21.15.92:47756) 2023-10-06T10:38:41.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.181+0000 7f3476d9d700 1 -- 172.21.15.92:0/3366528799 learned_addr learned my addr 172.21.15.92:0/3366528799 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:41.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.181+0000 7f346dd9b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34780b49a0 0x7f34780aac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:52200/0 (socket says 172.21.15.92:52200) 2023-10-06T10:38:41.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.182+0000 7f346dd9b700 1 -- 172.21.15.92:0/3366528799 >> v2:172.21.15.124:3300/0 conn(0x7f34780b0760 msgr2=0x7f34780b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:41.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.182+0000 7f346dd9b700 1 --2- 172.21.15.92:0/3366528799 >> v2:172.21.15.124:3300/0 conn(0x7f34780b0760 0x7f34780b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.182+0000 7f346dd9b700 1 -- 172.21.15.92:0/3366528799 >> v1:172.21.15.124:6789/0 conn(0x7f3478159560 legacy=0x7f34780aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:41.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.182+0000 7f346dd9b700 1 -- 172.21.15.92:0/3366528799 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34780b1500 con 0x7f34780b49a0 2023-10-06T10:38:41.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.182+0000 7f346dd9b700 1 --2- 172.21.15.92:0/3366528799 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34780b49a0 0x7f34780aac40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f34600098e0 tx=0x7f34600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93689428a00deff4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:41.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.182+0000 7f3475d9b700 1 -- 172.21.15.92:0/3366528799 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346000b640 con 0x7f34780b49a0 2023-10-06T10:38:41.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.182+0000 7f3475d9b700 1 -- 172.21.15.92:0/3366528799 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346000b7a0 con 0x7f34780b49a0 2023-10-06T10:38:41.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.182+0000 7f3475d9b700 1 -- 172.21.15.92:0/3366528799 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346000bb40 con 0x7f34780b49a0 2023-10-06T10:38:41.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.183+0000 7f347cb39700 1 -- 172.21.15.92:0/3366528799 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34780b49a0 msgr2=0x7f34780aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:41.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.183+0000 7f347cb39700 1 --2- 172.21.15.92:0/3366528799 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34780b49a0 0x7f34780aac40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f34600098e0 tx=0x7f34600091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.183+0000 7f347cb39700 1 -- 172.21.15.92:0/3366528799 shutdown_connections 2023-10-06T10:38:41.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.183+0000 7f347cb39700 1 --2- 172.21.15.92:0/3366528799 >> v2:172.21.15.124:3300/0 conn(0x7f34780b0760 0x7f34780b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.183+0000 7f347cb39700 1 --2- 172.21.15.92:0/3366528799 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34780b49a0 0x7f34780aac40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.183+0000 7f347cb39700 1 -- 172.21.15.92:0/3366528799 >> 172.21.15.92:0/3366528799 conn(0x7f34780997f0 msgr2=0x7f347806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:41.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.183+0000 7f347cb39700 1 -- 172.21.15.92:0/3366528799 shutdown_connections 2023-10-06T10:38:41.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.183+0000 7f347cb39700 1 -- 172.21.15.92:0/3366528799 wait complete. 2023-10-06T10:38:41.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.184+0000 7f347cb39700 1 Processor -- start 2023-10-06T10:38:41.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.184+0000 7f347cb39700 1 -- start start 2023-10-06T10:38:41.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.185+0000 7f347cb39700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34780b0760 0x7f3478157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:41.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.185+0000 7f347cb39700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34780b49a0 0x7f347816a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:41.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.185+0000 7f347cb39700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3478159560 0x7f34780ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:41.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.185+0000 7f347cb39700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34780ac260 con 0x7f34780b0760 2023-10-06T10:38:41.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.185+0000 7f347cb39700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347809acb0 con 0x7f34780b49a0 2023-10-06T10:38:41.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.185+0000 7f347cb39700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3478157e50 con 0x7f3478159560 2023-10-06T10:38:41.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.185+0000 7f347659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34780b0760 0x7f3478157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:41.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.185+0000 7f347659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34780b0760 0x7f3478157910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37134/0 (socket says 172.21.15.92:37134) 2023-10-06T10:38:41.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.185+0000 7f347659c700 1 -- 172.21.15.92:0/1771313172 learned_addr learned my addr 172.21.15.92:0/1771313172 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:41.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.185+0000 7f3476d9d700 1 --2- 172.21.15.92:0/1771313172 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3478159560 0x7f34780ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:41.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.185+0000 7f346dd9b700 1 --2- 172.21.15.92:0/1771313172 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34780b49a0 0x7f347816a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:41.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.185+0000 7f347659c700 1 -- 172.21.15.92:0/1771313172 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3478159560 msgr2=0x7f34780ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:41.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.185+0000 7f347659c700 1 --2- 172.21.15.92:0/1771313172 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3478159560 0x7f34780ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.185+0000 7f347659c700 1 -- 172.21.15.92:0/1771313172 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34780b49a0 msgr2=0x7f347816a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:41.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.185+0000 7f347659c700 1 --2- 172.21.15.92:0/1771313172 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34780b49a0 0x7f347816a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.185+0000 7f347659c700 1 -- 172.21.15.92:0/1771313172 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3460009420 con 0x7f34780b0760 2023-10-06T10:38:41.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.185+0000 7f347659c700 1 --2- 172.21.15.92:0/1771313172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34780b0760 0x7f3478157910 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f346800c540 tx=0x7f3468001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:41.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.185+0000 7f346effd700 1 -- 172.21.15.92:0/1771313172 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468011590 con 0x7f34780b0760 2023-10-06T10:38:41.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.185+0000 7f346effd700 1 -- 172.21.15.92:0/1771313172 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34680116f0 con 0x7f34780b0760 2023-10-06T10:38:41.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.185+0000 7f347cb39700 1 -- 172.21.15.92:0/1771313172 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f347809c530 con 0x7f34780b0760 2023-10-06T10:38:41.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.185+0000 7f346effd700 1 -- 172.21.15.92:0/1771313172 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468011ab0 con 0x7f34780b0760 2023-10-06T10:38:41.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.186+0000 7f347cb39700 1 -- 172.21.15.92:0/1771313172 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34780abca0 con 0x7f34780b0760 2023-10-06T10:38:41.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.187+0000 7f347cb39700 1 -- 172.21.15.92:0/1771313172 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3478064a90 con 0x7f34780b0760 2023-10-06T10:38:41.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.187+0000 7f346effd700 1 -- 172.21.15.92:0/1771313172 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f346801b3b0 con 0x7f34780b0760 2023-10-06T10:38:41.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.187+0000 7f346effd700 1 --2- 172.21.15.92:0/1771313172 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3458066f30 0x7f34580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:41.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.187+0000 7f346effd700 1 -- 172.21.15.92:0/1771313172 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f3468083c70 con 0x7f34780b0760 2023-10-06T10:38:41.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.187+0000 7f346dd9b700 1 --2- 172.21.15.92:0/1771313172 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3458066f30 0x7f34580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:41.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.188+0000 7f346dd9b700 1 --2- 172.21.15.92:0/1771313172 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3458066f30 0x7f34580693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f34600093f0 tx=0x7f3460017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:41.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.191+0000 7f346effd700 1 -- 172.21.15.92:0/1771313172 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3468050b40 con 0x7f34780b0760 2023-10-06T10:38:41.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.424+0000 7f347cb39700 1 -- 172.21.15.92:0/1771313172 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34780b17e0 con 0x7f34780b0760 2023-10-06T10:38:41.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.425+0000 7f346effd700 1 -- 172.21.15.92:0/1771313172 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4599 (secure 0 0 0) 0x7f3468017020 con 0x7f34780b0760 2023-10-06T10:38:41.426 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:41.426 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2303424485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2303424485},{"type":"v1","addr":"172.21.15.92:6836","nonce":2303424485}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1027343065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1027343065},{"type":"v1","addr":"172.21.15.92:6837","nonce":1027343065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2667820379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2667820379},{"type":"v1","addr":"172.21.15.124:6834","nonce":2667820379}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:38:16.725144+0000","modified":"2023-10-06T10:38:30.398242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/1919704285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":1919704285},{"type":"v1","addr":"172.21.15.124:6835","nonce":1919704285}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T10:38:41.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.427+0000 7f347cb39700 1 -- 172.21.15.92:0/1771313172 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3458066f30 msgr2=0x7f34580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:41.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.427+0000 7f347cb39700 1 --2- 172.21.15.92:0/1771313172 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3458066f30 0x7f34580693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f34600093f0 tx=0x7f3460017000 comp rx=0 tx=0).stop 2023-10-06T10:38:41.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.427+0000 7f347cb39700 1 -- 172.21.15.92:0/1771313172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34780b0760 msgr2=0x7f3478157910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:41.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.427+0000 7f347cb39700 1 --2- 172.21.15.92:0/1771313172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34780b0760 0x7f3478157910 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f346800c540 tx=0x7f3468001070 comp rx=0 tx=0).stop 2023-10-06T10:38:41.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.427+0000 7f347cb39700 1 -- 172.21.15.92:0/1771313172 shutdown_connections 2023-10-06T10:38:41.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.427+0000 7f347cb39700 1 --2- 172.21.15.92:0/1771313172 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3458066f30 0x7f34580693e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.427+0000 7f347cb39700 1 --2- 172.21.15.92:0/1771313172 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3478159560 0x7f34780ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.427+0000 7f347cb39700 1 --2- 172.21.15.92:0/1771313172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34780b0760 0x7f3478157910 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.427+0000 7f347cb39700 1 --2- 172.21.15.92:0/1771313172 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34780b49a0 0x7f347816a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.427+0000 7f347cb39700 1 -- 172.21.15.92:0/1771313172 >> 172.21.15.92:0/1771313172 conn(0x7f34780997f0 msgr2=0x7f34780b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:41.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.427+0000 7f347cb39700 1 -- 172.21.15.92:0/1771313172 shutdown_connections 2023-10-06T10:38:41.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.427+0000 7f347cb39700 1 -- 172.21.15.92:0/1771313172 wait complete. 2023-10-06T10:38:41.430 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-10-06T10:38:41.440 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:38:41.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.554+0000 7fc97359e700 1 Processor -- start 2023-10-06T10:38:41.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.554+0000 7fc97359e700 1 -- start start 2023-10-06T10:38:41.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc97359e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc9740b2810 0x7fc9740cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:41.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc97359e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9740cb7a0 con 0x7fc974159440 2023-10-06T10:38:41.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc97359e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9740b0860 con 0x7fc9740cbc30 2023-10-06T10:38:41.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc97359e700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9740cb150 con 0x7fc9740b2810 2023-10-06T10:38:41.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc97259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc974159440 0x7fc9740b3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40210/0 (socket says 172.21.15.92:40210) 2023-10-06T10:38:41.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc97259c700 1 -- 172.21.15.92:0/647337988 learned_addr learned my addr 172.21.15.92:0/647337988 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:41.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc972d9d700 1 --2- 172.21.15.92:0/647337988 >> v2:172.21.15.124:3300/0 conn(0x7fc9740b2810 0x7fc9740cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:41.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc97159a700 1 -- 172.21.15.92:0/647337988 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3691385998 0 0) 0x7fc9740cb7a0 con 0x7fc974159440 2023-10-06T10:38:41.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc97159a700 1 -- 172.21.15.92:0/647337988 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc95c003570 con 0x7fc974159440 2023-10-06T10:38:41.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc97159a700 1 -- 172.21.15.92:0/647337988 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1623950210 0 0) 0x7fc95c003570 con 0x7fc974159440 2023-10-06T10:38:41.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc97159a700 1 -- 172.21.15.92:0/647337988 >> v2:172.21.15.124:3300/0 conn(0x7fc9740b2810 msgr2=0x7fc9740cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:41.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc97159a700 1 --2- 172.21.15.92:0/647337988 >> v2:172.21.15.124:3300/0 conn(0x7fc9740b2810 0x7fc9740cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc97159a700 1 -- 172.21.15.92:0/647337988 >> v1:172.21.15.124:6789/0 conn(0x7fc9740cbc30 legacy=0x7fc9740b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:41.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc97159a700 1 -- 172.21.15.92:0/647337988 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9740cb430 con 0x7fc974159440 2023-10-06T10:38:41.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc97159a700 1 -- 172.21.15.92:0/647337988 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fc964001470 con 0x7fc974159440 2023-10-06T10:38:41.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc97159a700 1 -- 172.21.15.92:0/647337988 >> v1:172.21.15.92:6789/0 conn(0x7fc974159440 legacy=0x7fc9740b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:41.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc97159a700 1 --2- 172.21.15.92:0/647337988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc95c003a60 0x7fc95c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc97159a700 1 --2- 172.21.15.92:0/647337988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc95c004810 0x7fc95c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc97159a700 1 --2- 172.21.15.92:0/647337988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc95c0051c0 0x7fc95c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc97159a700 1 -- 172.21.15.92:0/647337988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9740cb150 con 0x7fc95c004810 2023-10-06T10:38:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc97159a700 1 -- 172.21.15.92:0/647337988 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c00c0d0 con 0x7fc95c003a60 2023-10-06T10:38:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc97159a700 1 -- 172.21.15.92:0/647337988 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c00c290 con 0x7fc95c0051c0 2023-10-06T10:38:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.555+0000 7fc971d9b700 1 --2- 172.21.15.92:0/647337988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc95c004810 0x7fc95c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.556+0000 7fc97259c700 1 --2- 172.21.15.92:0/647337988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc95c003a60 0x7fc95c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.556+0000 7fc972d9d700 1 --2- 172.21.15.92:0/647337988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc95c0051c0 0x7fc95c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.556+0000 7fc971d9b700 1 -- 172.21.15.92:0/647337988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc95c0051c0 msgr2=0x7fc95c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.556+0000 7fc971d9b700 1 --2- 172.21.15.92:0/647337988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc95c0051c0 0x7fc95c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.556+0000 7fc971d9b700 1 -- 172.21.15.92:0/647337988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc95c003a60 msgr2=0x7fc95c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:41.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.556+0000 7fc971d9b700 1 --2- 172.21.15.92:0/647337988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc95c003a60 0x7fc95c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.556+0000 7fc972d9d700 1 -- 172.21.15.92:0/647337988 reap_dead start 2023-10-06T10:38:41.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.556+0000 7fc971d9b700 1 -- 172.21.15.92:0/647337988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95c003570 con 0x7fc95c004810 2023-10-06T10:38:41.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.556+0000 7fc971d9b700 1 --2- 172.21.15.92:0/647337988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc95c004810 0x7fc95c004c80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc9680018e0 tx=0x7fc968007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d0b37bb82d136c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:41.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.556+0000 7fc97159a700 1 -- 172.21.15.92:0/647337988 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc968012420 con 0x7fc95c004810 2023-10-06T10:38:41.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.556+0000 7fc97159a700 1 -- 172.21.15.92:0/647337988 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9680125a0 con 0x7fc95c004810 2023-10-06T10:38:41.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.556+0000 7fc97159a700 1 -- 172.21.15.92:0/647337988 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc968012960 con 0x7fc95c004810 2023-10-06T10:38:41.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.557+0000 7fc97359e700 1 -- 172.21.15.92:0/647337988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc95c004810 msgr2=0x7fc95c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:41.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.557+0000 7fc97359e700 1 --2- 172.21.15.92:0/647337988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc95c004810 0x7fc95c004c80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc9680018e0 tx=0x7fc968007d30 comp rx=0 tx=0).stop 2023-10-06T10:38:41.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.557+0000 7fc97359e700 1 -- 172.21.15.92:0/647337988 shutdown_connections 2023-10-06T10:38:41.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.557+0000 7fc97359e700 1 --2- 172.21.15.92:0/647337988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc95c004810 0x7fc95c004c80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.557+0000 7fc97359e700 1 -- 172.21.15.92:0/647337988 >> 172.21.15.92:0/647337988 conn(0x7fc97406ece0 msgr2=0x7fc97406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:41.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.557+0000 7fc97359e700 1 -- 172.21.15.92:0/647337988 shutdown_connections 2023-10-06T10:38:41.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.557+0000 7fc97359e700 1 -- 172.21.15.92:0/647337988 wait complete. 2023-10-06T10:38:41.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.558+0000 7fc97359e700 1 Processor -- start 2023-10-06T10:38:41.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.558+0000 7fc97359e700 1 -- start start 2023-10-06T10:38:41.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.558+0000 7fc97359e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc95c004810 0x7fc974169530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:41.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.558+0000 7fc97359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc974153aa0 0x7fc974169a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:41.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.558+0000 7fc97359e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9740b5a10 0x7fc974169fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:41.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.558+0000 7fc97359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97416a8c0 con 0x7fc974153aa0 2023-10-06T10:38:41.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.558+0000 7fc97359e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc974153800 con 0x7fc95c004810 2023-10-06T10:38:41.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.559+0000 7fc97359e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97415aae0 con 0x7fc9740b5a10 2023-10-06T10:38:41.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.559+0000 7fc971d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc974153aa0 0x7fc974169a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:41.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.559+0000 7fc971d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc974153aa0 0x7fc974169a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37154/0 (socket says 172.21.15.92:37154) 2023-10-06T10:38:41.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.559+0000 7fc971d9b700 1 -- 172.21.15.92:0/2319219266 learned_addr learned my addr 172.21.15.92:0/2319219266 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:41.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.559+0000 7fc97259c700 1 --2- 172.21.15.92:0/2319219266 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc95c004810 0x7fc974169530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:41.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.559+0000 7fc972d9d700 1 --2- 172.21.15.92:0/2319219266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9740b5a10 0x7fc974169fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:41.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.559+0000 7fc971d9b700 1 -- 172.21.15.92:0/2319219266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9740b5a10 msgr2=0x7fc974169fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:41.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.559+0000 7fc971d9b700 1 --2- 172.21.15.92:0/2319219266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9740b5a10 0x7fc974169fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.559+0000 7fc971d9b700 1 -- 172.21.15.92:0/2319219266 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc95c004810 msgr2=0x7fc974169530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:41.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.559+0000 7fc971d9b700 1 --2- 172.21.15.92:0/2319219266 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc95c004810 0x7fc974169530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.559+0000 7fc971d9b700 1 -- 172.21.15.92:0/2319219266 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97415adc0 con 0x7fc974153aa0 2023-10-06T10:38:41.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.559+0000 7fc971d9b700 1 --2- 172.21.15.92:0/2319219266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc974153aa0 0x7fc974169a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc96800c0e0 tx=0x7fc968015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:41.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.559+0000 7fc9627fc700 1 -- 172.21.15.92:0/2319219266 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc968001230 con 0x7fc974153aa0 2023-10-06T10:38:41.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.559+0000 7fc9627fc700 1 -- 172.21.15.92:0/2319219266 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96800ca90 con 0x7fc974153aa0 2023-10-06T10:38:41.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.559+0000 7fc97359e700 1 -- 172.21.15.92:0/2319219266 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc97415b0a0 con 0x7fc974153aa0 2023-10-06T10:38:41.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.559+0000 7fc9627fc700 1 -- 172.21.15.92:0/2319219266 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96800ce30 con 0x7fc974153aa0 2023-10-06T10:38:41.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.559+0000 7fc97359e700 1 -- 172.21.15.92:0/2319219266 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97415a720 con 0x7fc974153aa0 2023-10-06T10:38:41.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.560+0000 7fc97359e700 1 -- 172.21.15.92:0/2319219266 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc97415cca0 con 0x7fc974153aa0 2023-10-06T10:38:41.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.561+0000 7fc9627fc700 1 -- 172.21.15.92:0/2319219266 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc968019030 con 0x7fc974153aa0 2023-10-06T10:38:41.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.561+0000 7fc9627fc700 1 --2- 172.21.15.92:0/2319219266 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc954066f30 0x7fc9540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:41.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.561+0000 7fc9627fc700 1 -- 172.21.15.92:0/2319219266 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fc968085e20 con 0x7fc974153aa0 2023-10-06T10:38:41.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.561+0000 7fc97259c700 1 --2- 172.21.15.92:0/2319219266 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc954066f30 0x7fc9540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:41.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.561+0000 7fc97259c700 1 --2- 172.21.15.92:0/2319219266 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc954066f30 0x7fc9540693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc964001ce0 tx=0x7fc964001e50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:41.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.565+0000 7fc9627fc700 1 -- 172.21.15.92:0/2319219266 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc968052c40 con 0x7fc974153aa0 2023-10-06T10:38:41.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.750+0000 7fc97359e700 1 -- 172.21.15.92:0/2319219266 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc97405f230 con 0x7fc974153aa0 2023-10-06T10:38:41.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.751+0000 7fc9627fc700 1 -- 172.21.15.92:0/2319219266 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18707 (secure 0 0 0) 0x7fc968017090 con 0x7fc974153aa0 2023-10-06T10:38:41.752 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:41.752 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:21.760233+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:38:14.368713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T10:38:15.692344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T10:38:19.382553+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:38:41.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.753+0000 7fc97359e700 1 -- 172.21.15.92:0/2319219266 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc954066f30 msgr2=0x7fc9540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:41.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.753+0000 7fc97359e700 1 --2- 172.21.15.92:0/2319219266 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc954066f30 0x7fc9540693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc964001ce0 tx=0x7fc964001e50 comp rx=0 tx=0).stop 2023-10-06T10:38:41.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.753+0000 7fc97359e700 1 -- 172.21.15.92:0/2319219266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc974153aa0 msgr2=0x7fc974169a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:41.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.753+0000 7fc97359e700 1 --2- 172.21.15.92:0/2319219266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc974153aa0 0x7fc974169a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc96800c0e0 tx=0x7fc968015000 comp rx=0 tx=0).stop 2023-10-06T10:38:41.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.753+0000 7fc97359e700 1 -- 172.21.15.92:0/2319219266 shutdown_connections 2023-10-06T10:38:41.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.753+0000 7fc97359e700 1 --2- 172.21.15.92:0/2319219266 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc954066f30 0x7fc9540693e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.753+0000 7fc97359e700 1 --2- 172.21.15.92:0/2319219266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9740b5a10 0x7fc974169fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.753+0000 7fc97359e700 1 --2- 172.21.15.92:0/2319219266 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc95c004810 0x7fc974169530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.753+0000 7fc97359e700 1 --2- 172.21.15.92:0/2319219266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc974153aa0 0x7fc974169a70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.753+0000 7fc97359e700 1 -- 172.21.15.92:0/2319219266 >> 172.21.15.92:0/2319219266 conn(0x7fc97406ece0 msgr2=0x7fc9740cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:41.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.753+0000 7fc97359e700 1 -- 172.21.15.92:0/2319219266 shutdown_connections 2023-10-06T10:38:41.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.753+0000 7fc97359e700 1 -- 172.21.15.92:0/2319219266 wait complete. 2023-10-06T10:38:41.766 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:38:41.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.879+0000 7fba2d8b5700 1 Processor -- start 2023-10-06T10:38:41.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.879+0000 7fba2d8b5700 1 -- start start 2023-10-06T10:38:41.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.879+0000 7fba2d8b5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fba280b49a0 0x7fba280aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:41.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.879+0000 7fba2d8b5700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba280b0760 0x7fba280b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:41.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.879+0000 7fba2d8b5700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba280b0c60 con 0x7fba28159560 2023-10-06T10:38:41.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.879+0000 7fba2d8b5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba280b0f40 con 0x7fba280b49a0 2023-10-06T10:38:41.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.879+0000 7fba2d8b5700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba280b1220 con 0x7fba280b0760 2023-10-06T10:38:41.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.880+0000 7fba1e7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fba280b49a0 0x7fba280aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:41.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.880+0000 7fba1e7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fba280b49a0 0x7fba280aac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37168/0 (socket says 172.21.15.92:37168) 2023-10-06T10:38:41.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.880+0000 7fba1e7fc700 1 -- 172.21.15.92:0/2861766739 learned_addr learned my addr 172.21.15.92:0/2861766739 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:41.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.880+0000 7fba277fe700 1 --2- 172.21.15.92:0/2861766739 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba280b0760 0x7fba280b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:41.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.880+0000 7fba1e7fc700 1 -- 172.21.15.92:0/2861766739 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba280b0760 msgr2=0x7fba280b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:41.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.880+0000 7fba1e7fc700 1 --2- 172.21.15.92:0/2861766739 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba280b0760 0x7fba280b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.880+0000 7fba1e7fc700 1 -- 172.21.15.92:0/2861766739 >> v1:172.21.15.124:6789/0 conn(0x7fba28159560 legacy=0x7fba280aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:41.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.880+0000 7fba1e7fc700 1 -- 172.21.15.92:0/2861766739 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba280b1500 con 0x7fba280b49a0 2023-10-06T10:38:41.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.880+0000 7fba277fe700 1 --2- 172.21.15.92:0/2861766739 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba280b0760 0x7fba280b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:41.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.880+0000 7fba1e7fc700 1 --2- 172.21.15.92:0/2861766739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba280b49a0 0x7fba280aac40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fba100099e0 tx=0x7fba10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc7e02d9980c3a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:41.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.880+0000 7fba267fc700 1 -- 172.21.15.92:0/2861766739 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1000b7e0 con 0x7fba280b49a0 2023-10-06T10:38:41.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.880+0000 7fba267fc700 1 -- 172.21.15.92:0/2861766739 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1000b940 con 0x7fba280b49a0 2023-10-06T10:38:41.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.880+0000 7fba267fc700 1 -- 172.21.15.92:0/2861766739 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1000bce0 con 0x7fba280b49a0 2023-10-06T10:38:41.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.881+0000 7fba2d8b5700 1 -- 172.21.15.92:0/2861766739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba280b49a0 msgr2=0x7fba280aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:41.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.881+0000 7fba2d8b5700 1 --2- 172.21.15.92:0/2861766739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba280b49a0 0x7fba280aac40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fba100099e0 tx=0x7fba10009250 comp rx=0 tx=0).stop 2023-10-06T10:38:41.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.881+0000 7fba2d8b5700 1 -- 172.21.15.92:0/2861766739 shutdown_connections 2023-10-06T10:38:41.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.881+0000 7fba2d8b5700 1 --2- 172.21.15.92:0/2861766739 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba280b0760 0x7fba280b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.881+0000 7fba2d8b5700 1 --2- 172.21.15.92:0/2861766739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba280b49a0 0x7fba280aac40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.881+0000 7fba2d8b5700 1 -- 172.21.15.92:0/2861766739 >> 172.21.15.92:0/2861766739 conn(0x7fba280997f0 msgr2=0x7fba2806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:41.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.881+0000 7fba2d8b5700 1 -- 172.21.15.92:0/2861766739 shutdown_connections 2023-10-06T10:38:41.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.881+0000 7fba2d8b5700 1 -- 172.21.15.92:0/2861766739 wait complete. 2023-10-06T10:38:41.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.882+0000 7fba2d8b5700 1 Processor -- start 2023-10-06T10:38:41.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.882+0000 7fba2d8b5700 1 -- start start 2023-10-06T10:38:41.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.883+0000 7fba2d8b5700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba280b0760 0x7fba28157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:41.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.883+0000 7fba2d8b5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba280b49a0 0x7fba2815ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:41.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.883+0000 7fba2d8b5700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba28159560 0x7fba2815bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:41.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.883+0000 7fba2d8b5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba280aacc0 con 0x7fba280b49a0 2023-10-06T10:38:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.883+0000 7fba2d8b5700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba280acfa0 con 0x7fba280b0760 2023-10-06T10:38:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.883+0000 7fba2d8b5700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba280ad280 con 0x7fba28159560 2023-10-06T10:38:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.883+0000 7fba1e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba280b49a0 0x7fba2815ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.883+0000 7fba1e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba280b49a0 0x7fba2815ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37170/0 (socket says 172.21.15.92:37170) 2023-10-06T10:38:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.883+0000 7fba1e7fc700 1 -- 172.21.15.92:0/720464945 learned_addr learned my addr 172.21.15.92:0/720464945 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.883+0000 7fba26ffd700 1 --2- 172.21.15.92:0/720464945 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba280b0760 0x7fba28157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.883+0000 7fba277fe700 1 --2- 172.21.15.92:0/720464945 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba28159560 0x7fba2815bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.883+0000 7fba1e7fc700 1 -- 172.21.15.92:0/720464945 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba28159560 msgr2=0x7fba2815bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.883+0000 7fba1e7fc700 1 --2- 172.21.15.92:0/720464945 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba28159560 0x7fba2815bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.883+0000 7fba1e7fc700 1 -- 172.21.15.92:0/720464945 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba280b0760 msgr2=0x7fba28157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:41.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.883+0000 7fba1e7fc700 1 --2- 172.21.15.92:0/720464945 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba280b0760 0x7fba28157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:41.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.883+0000 7fba1e7fc700 1 -- 172.21.15.92:0/720464945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba100094f0 con 0x7fba280b49a0 2023-10-06T10:38:41.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.883+0000 7fba1e7fc700 1 --2- 172.21.15.92:0/720464945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba280b49a0 0x7fba2815ba80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fba10006fd0 tx=0x7fba10016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:41.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.883+0000 7fba1f7fe700 1 -- 172.21.15.92:0/720464945 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba10011bd0 con 0x7fba280b49a0 2023-10-06T10:38:41.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.884+0000 7fba1f7fe700 1 -- 172.21.15.92:0/720464945 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba10020070 con 0x7fba280b49a0 2023-10-06T10:38:41.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.884+0000 7fba2d8b5700 1 -- 172.21.15.92:0/720464945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2809d540 con 0x7fba280b49a0 2023-10-06T10:38:41.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.884+0000 7fba1f7fe700 1 -- 172.21.15.92:0/720464945 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1001b450 con 0x7fba280b49a0 2023-10-06T10:38:41.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.884+0000 7fba2d8b5700 1 -- 172.21.15.92:0/720464945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2809db00 con 0x7fba280b49a0 2023-10-06T10:38:41.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.885+0000 7fba2d8b5700 1 -- 172.21.15.92:0/720464945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba28064a90 con 0x7fba280b49a0 2023-10-06T10:38:41.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.885+0000 7fba1f7fe700 1 -- 172.21.15.92:0/720464945 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fba10022030 con 0x7fba280b49a0 2023-10-06T10:38:41.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.885+0000 7fba1f7fe700 1 --2- 172.21.15.92:0/720464945 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fba08066f30 0x7fba080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:41.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.885+0000 7fba26ffd700 1 --2- 172.21.15.92:0/720464945 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fba08066f30 0x7fba080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:41.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.886+0000 7fba1f7fe700 1 -- 172.21.15.92:0/720464945 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fba10086740 con 0x7fba280b49a0 2023-10-06T10:38:41.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.886+0000 7fba26ffd700 1 --2- 172.21.15.92:0/720464945 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fba08066f30 0x7fba080693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fba18006fd0 tx=0x7fba18009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:41.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:41.890+0000 7fba1f7fe700 1 -- 172.21.15.92:0/720464945 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba10053610 con 0x7fba280b49a0 2023-10-06T10:38:42.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.112+0000 7fba2d8b5700 1 -- 172.21.15.92:0/720464945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba28064a90 con 0x7fba280b49a0 2023-10-06T10:38:42.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.113+0000 7fba1f7fe700 1 -- 172.21.15.92:0/720464945 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4599 (secure 0 0 0) 0x7fba10056c30 con 0x7fba280b49a0 2023-10-06T10:38:42.114 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:42.114 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2303424485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2303424485},{"type":"v1","addr":"172.21.15.92:6836","nonce":2303424485}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1027343065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1027343065},{"type":"v1","addr":"172.21.15.92:6837","nonce":1027343065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2667820379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2667820379},{"type":"v1","addr":"172.21.15.124:6834","nonce":2667820379}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:38:16.725144+0000","modified":"2023-10-06T10:38:30.398242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/1919704285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":1919704285},{"type":"v1","addr":"172.21.15.124:6835","nonce":1919704285}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T10:38:42.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.115+0000 7fba2d8b5700 1 -- 172.21.15.92:0/720464945 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fba08066f30 msgr2=0x7fba080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:42.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.115+0000 7fba2d8b5700 1 --2- 172.21.15.92:0/720464945 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fba08066f30 0x7fba080693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fba18006fd0 tx=0x7fba18009060 comp rx=0 tx=0).stop 2023-10-06T10:38:42.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.115+0000 7fba2d8b5700 1 -- 172.21.15.92:0/720464945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba280b49a0 msgr2=0x7fba2815ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:42.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.115+0000 7fba2d8b5700 1 --2- 172.21.15.92:0/720464945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba280b49a0 0x7fba2815ba80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fba10006fd0 tx=0x7fba10016000 comp rx=0 tx=0).stop 2023-10-06T10:38:42.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.115+0000 7fba2d8b5700 1 -- 172.21.15.92:0/720464945 shutdown_connections 2023-10-06T10:38:42.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.115+0000 7fba2d8b5700 1 --2- 172.21.15.92:0/720464945 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fba08066f30 0x7fba080693e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.115+0000 7fba2d8b5700 1 --2- 172.21.15.92:0/720464945 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba28159560 0x7fba2815bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.115+0000 7fba2d8b5700 1 --2- 172.21.15.92:0/720464945 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba280b0760 0x7fba28157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.115+0000 7fba2d8b5700 1 --2- 172.21.15.92:0/720464945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba280b49a0 0x7fba2815ba80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.115+0000 7fba2d8b5700 1 -- 172.21.15.92:0/720464945 >> 172.21.15.92:0/720464945 conn(0x7fba280997f0 msgr2=0x7fba2815cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:42.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.115+0000 7fba2d8b5700 1 -- 172.21.15.92:0/720464945 shutdown_connections 2023-10-06T10:38:42.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.115+0000 7fba2d8b5700 1 -- 172.21.15.92:0/720464945 wait complete. 2023-10-06T10:38:42.118 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-10-06T10:38:42.531 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T10:38:16.725144+0000', 'modified': '2023-10-06T10:38:30.398242+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4306}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4306': {'gid': 4306, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.124:6835/1919704285', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6833', 'nonce': 1919704285}, {'type': 'v1', 'addr': '172.21.15.124:6835', 'nonce': 1919704285}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T10:38:42.531 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T10:38:42.531 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T10:38:42.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.646+0000 7f0f12205700 1 Processor -- start 2023-10-06T10:38:42.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.646+0000 7f0f12205700 1 -- start start 2023-10-06T10:38:42.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.647+0000 7f0f12205700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f0c0b29b0 0x7f0f0c159120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.647+0000 7f0f12205700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0f0c159660 0x7f0f0c0cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.647+0000 7f0f12205700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f0c0b0da0 con 0x7f0f0c0b25e0 2023-10-06T10:38:42.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.647+0000 7f0f12205700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f0c0cbfa0 con 0x7f0f0c159660 2023-10-06T10:38:42.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.647+0000 7f0f12205700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0cb390 con 0x7f0f0c0b29b0 2023-10-06T10:38:42.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.647+0000 7f0f0b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0f0c0b25e0 0x7f0f0c0b0690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40224/0 (socket says 172.21.15.92:40224) 2023-10-06T10:38:42.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.647+0000 7f0f0b7fe700 1 -- 172.21.15.92:0/3881746968 learned_addr learned my addr 172.21.15.92:0/3881746968 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:42.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.647+0000 7f0f0a7fc700 1 -- 172.21.15.92:0/3881746968 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1689511894 0 0) 0x7f0efc001610 con 0x7f0f0c0b25e0 2023-10-06T10:38:42.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.647+0000 7f0f0bfff700 1 --2- 172.21.15.92:0/3881746968 >> v2:172.21.15.124:3300/0 conn(0x7f0f0c159660 0x7f0f0c0cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.647+0000 7f0f0affd700 1 --2- 172.21.15.92:0/3881746968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f0c0b29b0 0x7f0f0c159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.647+0000 7f0f0a7fc700 1 -- 172.21.15.92:0/3881746968 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ef4003570 con 0x7f0f0c0b25e0 2023-10-06T10:38:42.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0a7fc700 1 -- 172.21.15.92:0/3881746968 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 294946844 0 0) 0x7f0ef4003570 con 0x7f0f0c0b25e0 2023-10-06T10:38:42.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0a7fc700 1 -- 172.21.15.92:0/3881746968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f0c0b29b0 msgr2=0x7f0f0c159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:42.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0a7fc700 1 --2- 172.21.15.92:0/3881746968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f0c0b29b0 0x7f0f0c159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0a7fc700 1 -- 172.21.15.92:0/3881746968 >> v2:172.21.15.124:3300/0 conn(0x7f0f0c159660 msgr2=0x7f0f0c0cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:42.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0a7fc700 1 --2- 172.21.15.92:0/3881746968 >> v2:172.21.15.124:3300/0 conn(0x7f0f0c159660 0x7f0f0c0cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0a7fc700 1 -- 172.21.15.92:0/3881746968 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c0cb670 con 0x7f0f0c0b25e0 2023-10-06T10:38:42.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0a7fc700 1 -- 172.21.15.92:0/3881746968 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f0efc001b00 con 0x7f0f0c0b25e0 2023-10-06T10:38:42.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0a7fc700 1 -- 172.21.15.92:0/3881746968 >> v1:172.21.15.92:6789/0 conn(0x7f0f0c0b25e0 legacy=0x7f0f0c0b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:42.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0a7fc700 1 --2- 172.21.15.92:0/3881746968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f0c0b29b0 0x7f0ef4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0a7fc700 1 --2- 172.21.15.92:0/3881746968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef4003fe0 0x7f0ef4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0a7fc700 1 --2- 172.21.15.92:0/3881746968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ef4004d50 0x7f0ef40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0a7fc700 1 -- 172.21.15.92:0/3881746968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0cbfa0 con 0x7f0ef4003fe0 2023-10-06T10:38:42.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0a7fc700 1 -- 172.21.15.92:0/3881746968 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0cb390 con 0x7f0ef4004d50 2023-10-06T10:38:42.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0a7fc700 1 -- 172.21.15.92:0/3881746968 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef400ba90 con 0x7f0f0c0b29b0 2023-10-06T10:38:42.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0affd700 1 --2- 172.21.15.92:0/3881746968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef4003fe0 0x7f0ef4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0b7fe700 1 --2- 172.21.15.92:0/3881746968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f0c0b29b0 0x7f0ef4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0bfff700 1 --2- 172.21.15.92:0/3881746968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ef4004d50 0x7f0ef40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0affd700 1 -- 172.21.15.92:0/3881746968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f0c0b29b0 msgr2=0x7f0ef4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:42.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0affd700 1 --2- 172.21.15.92:0/3881746968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f0c0b29b0 0x7f0ef4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0affd700 1 -- 172.21.15.92:0/3881746968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ef4004d50 msgr2=0x7f0ef40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:42.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0affd700 1 --2- 172.21.15.92:0/3881746968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ef4004d50 0x7f0ef40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0affd700 1 -- 172.21.15.92:0/3881746968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef4003570 con 0x7f0ef4003fe0 2023-10-06T10:38:42.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0bfff700 1 --2- 172.21.15.92:0/3881746968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ef4004d50 0x7f0ef40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:42.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0affd700 1 --2- 172.21.15.92:0/3881746968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef4003fe0 0x7f0ef4004810 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009c00 tx=0x7f0f000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff8038cc784ef73 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:42.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0a7fc700 1 -- 172.21.15.92:0/3881746968 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0000e6b0 con 0x7f0ef4003fe0 2023-10-06T10:38:42.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0a7fc700 1 -- 172.21.15.92:0/3881746968 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0000e810 con 0x7f0ef4003fe0 2023-10-06T10:38:42.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.648+0000 7f0f0a7fc700 1 -- 172.21.15.92:0/3881746968 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0000ebd0 con 0x7f0ef4003fe0 2023-10-06T10:38:42.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.649+0000 7f0f12205700 1 -- 172.21.15.92:0/3881746968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef4003fe0 msgr2=0x7f0ef4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:42.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.649+0000 7f0f12205700 1 --2- 172.21.15.92:0/3881746968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef4003fe0 0x7f0ef4004810 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009c00 tx=0x7f0f000091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.649+0000 7f0f0bfff700 1 -- 172.21.15.92:0/3881746968 reap_dead start 2023-10-06T10:38:42.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.649+0000 7f0f12205700 1 -- 172.21.15.92:0/3881746968 shutdown_connections 2023-10-06T10:38:42.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.649+0000 7f0f12205700 1 -- 172.21.15.92:0/3881746968 >> 172.21.15.92:0/3881746968 conn(0x7f0f0c06ece0 msgr2=0x7f0f0c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:42.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.649+0000 7f0f12205700 1 -- 172.21.15.92:0/3881746968 shutdown_connections 2023-10-06T10:38:42.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.649+0000 7f0f12205700 1 -- 172.21.15.92:0/3881746968 wait complete. 2023-10-06T10:38:42.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.650+0000 7f0f12205700 1 Processor -- start 2023-10-06T10:38:42.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.650+0000 7f0f12205700 1 -- start start 2023-10-06T10:38:42.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.651+0000 7f0f12205700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef4003fe0 0x7f0f0c151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.651+0000 7f0f12205700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0f0c0c3f20 0x7f0f0c0cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.651+0000 7f0f12205700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f0c16dd90 0x7f0f0c16d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.651+0000 7f0f12205700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0d0ce0 con 0x7f0ef4003fe0 2023-10-06T10:38:42.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.651+0000 7f0f12205700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0c9380 con 0x7f0f0c0c3f20 2023-10-06T10:38:42.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.651+0000 7f0f12205700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0d0720 con 0x7f0f0c16dd90 2023-10-06T10:38:42.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.651+0000 7f0f0b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef4003fe0 0x7f0f0c151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.651+0000 7f0f0b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef4003fe0 0x7f0f0c151850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37184/0 (socket says 172.21.15.92:37184) 2023-10-06T10:38:42.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.651+0000 7f0f0b7fe700 1 -- 172.21.15.92:0/2321896236 learned_addr learned my addr 172.21.15.92:0/2321896236 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:42.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.651+0000 7f0f0affd700 1 --2- 172.21.15.92:0/2321896236 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0f0c0c3f20 0x7f0f0c0cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.651+0000 7f0f0bfff700 1 --2- 172.21.15.92:0/2321896236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f0c16dd90 0x7f0f0c16d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.651+0000 7f0f0b7fe700 1 -- 172.21.15.92:0/2321896236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f0c16dd90 msgr2=0x7f0f0c16d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:42.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.651+0000 7f0f0b7fe700 1 --2- 172.21.15.92:0/2321896236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f0c16dd90 0x7f0f0c16d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.651+0000 7f0f0b7fe700 1 -- 172.21.15.92:0/2321896236 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0f0c0c3f20 msgr2=0x7f0f0c0cfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:42.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.651+0000 7f0f0b7fe700 1 --2- 172.21.15.92:0/2321896236 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0f0c0c3f20 0x7f0f0c0cfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.651+0000 7f0f0b7fe700 1 -- 172.21.15.92:0/2321896236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c0d0a00 con 0x7f0ef4003fe0 2023-10-06T10:38:42.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.651+0000 7f0f0affd700 1 --2- 172.21.15.92:0/2321896236 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0f0c0c3f20 0x7f0f0c0cfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:42.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.652+0000 7f0f0b7fe700 1 --2- 172.21.15.92:0/2321896236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef4003fe0 0x7f0f0c151850 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0efc001880 tx=0x7f0efc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:42.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.652+0000 7f0ef37fe700 1 -- 172.21.15.92:0/2321896236 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc005d80 con 0x7f0ef4003fe0 2023-10-06T10:38:42.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.652+0000 7f0ef37fe700 1 -- 172.21.15.92:0/2321896236 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0efc00c7b0 con 0x7f0ef4003fe0 2023-10-06T10:38:42.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.652+0000 7f0ef37fe700 1 -- 172.21.15.92:0/2321896236 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc00cb70 con 0x7f0ef4003fe0 2023-10-06T10:38:42.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.652+0000 7f0f12205700 1 -- 172.21.15.92:0/2321896236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f0c0c9010 con 0x7f0ef4003fe0 2023-10-06T10:38:42.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.652+0000 7f0f12205700 1 -- 172.21.15.92:0/2321896236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0c0d0310 con 0x7f0ef4003fe0 2023-10-06T10:38:42.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.653+0000 7f0ef37fe700 1 -- 172.21.15.92:0/2321896236 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0efc0070d0 con 0x7f0ef4003fe0 2023-10-06T10:38:42.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.653+0000 7f0ef37fe700 1 --2- 172.21.15.92:0/2321896236 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0ef8066f30 0x7f0ef80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.654+0000 7f0ef37fe700 1 -- 172.21.15.92:0/2321896236 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f0efc084d30 con 0x7f0ef4003fe0 2023-10-06T10:38:42.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.654+0000 7f0f0affd700 1 --2- 172.21.15.92:0/2321896236 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0ef8066f30 0x7f0ef80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.654+0000 7f0f0affd700 1 --2- 172.21.15.92:0/2321896236 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0ef8066f30 0x7f0ef80693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0f000091e0 tx=0x7f0f0001a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:42.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.655+0000 7f0f1b4fdb80 1 Processor -- start 2023-10-06T10:38:42.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.656+0000 7f0f1b4fdb80 1 -- start start 2023-10-06T10:38:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.656+0000 7f0f1b4fdb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce628a0a90 0x55ce628a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.656+0000 7f0f1b4fdb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ce6285c9a0 0x55ce62859580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.656+0000 7f0f1b4fdb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce6285abd0 0x55ce62859ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.656+0000 7f0f1b4fdb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce627d9c80 con 0x55ce6285abd0 2023-10-06T10:38:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.656+0000 7f0f1b4fdb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce627b0a70 con 0x55ce628a0a90 2023-10-06T10:38:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.656+0000 7f0f1b4fdb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce627ae270 con 0x55ce6285c9a0 2023-10-06T10:38:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.656+0000 7f0f0bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce6285abd0 0x55ce62859ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.656+0000 7f0f0bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce6285abd0 0x55ce62859ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37200/0 (socket says 172.21.15.92:37200) 2023-10-06T10:38:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.656+0000 7f0f0bfff700 1 -- 172.21.15.92:0/3185743173 learned_addr learned my addr 172.21.15.92:0/3185743173 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.657+0000 7f0f0b7fe700 1 --2- 172.21.15.92:0/3185743173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce628a0a90 0x55ce628a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.657+0000 7f0f0affd700 1 --2- 172.21.15.92:0/3185743173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ce6285c9a0 0x55ce62859580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.657+0000 7f0f0bfff700 1 -- 172.21.15.92:0/3185743173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ce6285c9a0 msgr2=0x55ce62859580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.657+0000 7f0f0bfff700 1 --2- 172.21.15.92:0/3185743173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ce6285c9a0 0x55ce62859580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.657+0000 7f0f0bfff700 1 -- 172.21.15.92:0/3185743173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce628a0a90 msgr2=0x55ce628a0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.657+0000 7f0f0bfff700 1 --2- 172.21.15.92:0/3185743173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce628a0a90 0x55ce628a0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.657+0000 7f0f0bfff700 1 -- 172.21.15.92:0/3185743173 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce62644270 con 0x55ce6285abd0 2023-10-06T10:38:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.657+0000 7f0f0b7fe700 1 --2- 172.21.15.92:0/3185743173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce628a0a90 0x55ce628a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.657+0000 7f0f0bfff700 1 --2- 172.21.15.92:0/3185743173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce6285abd0 0x55ce62859ac0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c068050 tx=0x7f0f04008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8795436b12a7948c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.657+0000 7f0ef0ff9700 1 -- 172.21.15.92:0/3185743173 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f04009590 con 0x55ce6285abd0 2023-10-06T10:38:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.657+0000 7f0ef0ff9700 1 -- 172.21.15.92:0/3185743173 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f04013750 con 0x55ce6285abd0 2023-10-06T10:38:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.657+0000 7f0ef0ff9700 1 -- 172.21.15.92:0/3185743173 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f04013b10 con 0x55ce6285abd0 2023-10-06T10:38:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.658+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/3185743173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce6285abd0 msgr2=0x55ce62859ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.658+0000 7f0f1b4fdb80 1 --2- 172.21.15.92:0/3185743173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce6285abd0 0x55ce62859ac0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c068050 tx=0x7f0f04008040 comp rx=0 tx=0).stop 2023-10-06T10:38:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.658+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/3185743173 shutdown_connections 2023-10-06T10:38:42.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.658+0000 7f0f1b4fdb80 1 --2- 172.21.15.92:0/3185743173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce6285abd0 0x55ce62859ac0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.658+0000 7f0f1b4fdb80 1 --2- 172.21.15.92:0/3185743173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce628a0a90 0x55ce628a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.658+0000 7f0f1b4fdb80 1 --2- 172.21.15.92:0/3185743173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ce6285c9a0 0x55ce62859580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.658+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/3185743173 >> 172.21.15.92:0/3185743173 conn(0x55ce6285b070 msgr2=0x55ce6285b440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:42.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.658+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/3185743173 shutdown_connections 2023-10-06T10:38:42.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.658+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/3185743173 wait complete. 2023-10-06T10:38:42.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.658+0000 7f0f1b4fdb80 1 -- start start 2023-10-06T10:38:42.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.658+0000 7f0f1b4fdb80 1 Processor -- start 2023-10-06T10:38:42.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.659+0000 7f0f1b4fdb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce6285abd0 0x55ce628b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.659+0000 7f0f1b4fdb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ce6285c9a0 0x55ce62854740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.659+0000 7f0f1b4fdb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce628a0a90 0x55ce62854c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.659+0000 7f0f1b4fdb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce627aa060 con 0x55ce6285abd0 2023-10-06T10:38:42.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.659+0000 7f0f1b4fdb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce62755890 con 0x55ce628a0a90 2023-10-06T10:38:42.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.659+0000 7f0f1b4fdb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce62770ea0 con 0x55ce6285c9a0 2023-10-06T10:38:42.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.659+0000 7f0f1b4fdb80 10 client.0 _ll_register_callbacks cb 0x55ce62646f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:38:42.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.659+0000 7f0f0b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce6285abd0 0x55ce628b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.659+0000 7f0f0b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce6285abd0 0x55ce628b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37202/0 (socket says 172.21.15.92:37202) 2023-10-06T10:38:42.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.659+0000 7f0f0b7fe700 1 -- 172.21.15.92:0/807572444 learned_addr learned my addr 172.21.15.92:0/807572444 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:42.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.659+0000 7f0f0affd700 1 --2- 172.21.15.92:0/807572444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ce6285c9a0 0x55ce62854740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.659+0000 7f0f0bfff700 1 --2- 172.21.15.92:0/807572444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce628a0a90 0x55ce62854c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.659+0000 7f0f0b7fe700 1 -- 172.21.15.92:0/807572444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ce6285c9a0 msgr2=0x55ce62854740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:42.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.659+0000 7f0f0b7fe700 1 --2- 172.21.15.92:0/807572444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ce6285c9a0 0x55ce62854740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.659+0000 7f0f0b7fe700 1 -- 172.21.15.92:0/807572444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce628a0a90 msgr2=0x55ce62854c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:42.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.659+0000 7f0f0b7fe700 1 --2- 172.21.15.92:0/807572444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce628a0a90 0x55ce62854c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.659+0000 7f0f0b7fe700 1 -- 172.21.15.92:0/807572444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f040096f0 con 0x55ce6285abd0 2023-10-06T10:38:42.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.659+0000 7f0f0bfff700 1 --2- 172.21.15.92:0/807572444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce628a0a90 0x55ce62854c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:42.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.659+0000 7f0f0affd700 1 --2- 172.21.15.92:0/807572444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ce6285c9a0 0x55ce62854740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:42.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.660+0000 7f0f0b7fe700 1 --2- 172.21.15.92:0/807572444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce6285abd0 0x55ce628b3ac0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00c470 tx=0x7f0efc082d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a870c3e06747a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:42.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.660+0000 7f0ee9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:38:42.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.660+0000 7f0ee9ffb700 1 -- 172.21.15.92:0/807572444 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc05c7b0 con 0x55ce6285abd0 2023-10-06T10:38:42.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.660+0000 7f0ee9ffb700 1 -- 172.21.15.92:0/807572444 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f0efc0712c0 con 0x55ce6285abd0 2023-10-06T10:38:42.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.660+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/807572444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce6284e690 con 0x55ce6285abd0 2023-10-06T10:38:42.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.660+0000 7f0ee9ffb700 1 -- 172.21.15.92:0/807572444 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0efc071480 con 0x55ce6285abd0 2023-10-06T10:38:42.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.660+0000 7f0f1b4fdb80 10 client.4782 fetch_fsmap learned FSMap version 8 2023-10-06T10:38:42.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.660+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/807572444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55ce625e6fe0 con 0x55ce6285abd0 2023-10-06T10:38:42.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.660+0000 7f0ee9ffb700 1 -- 172.21.15.92:0/807572444 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ce6284e690 con 0x55ce6285abd0 2023-10-06T10:38:42.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.660+0000 7f0ee9ffb700 1 -- 172.21.15.92:0/807572444 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 8) v1 ==== 3108+0+0 (secure 0 0 0) 0x7f0efc054d00 con 0x55ce6285abd0 2023-10-06T10:38:42.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.660+0000 7f0f1b4fdb80 10 client.4782 fetch_fsmap finished waiting for FSMap version 8 2023-10-06T10:38:42.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.660+0000 7f0f1b4fdb80 10 client.4782 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T10:38:42.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.660+0000 7f0f1b4fdb80 20 client.4782 populate_metadata read hostname 'smithi092' 2023-10-06T10:38:42.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.660+0000 7f0f1b4fdb80 1 --2- 172.21.15.92:0/807572444 >> [v2:172.21.15.124:6833/1919704285,v1:172.21.15.124:6835/1919704285] conn(0x55ce6289c040 0x55ce6289c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.660+0000 7f0f1b4fdb80 4 client.4782 mds_command: new command op to 4306 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:38:42.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.660+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/807572444 --> [v2:172.21.15.124:6833/1919704285,v1:172.21.15.124:6835/1919704285] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce6282b5c0 con 0x55ce6289c040 2023-10-06T10:38:42.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.660+0000 7f0f0affd700 1 --2- 172.21.15.92:0/807572444 >> [v2:172.21.15.124:6833/1919704285,v1:172.21.15.124:6835/1919704285] conn(0x55ce6289c040 0x55ce6289c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.661+0000 7f0ee9ffb700 10 client.4782 ms_handle_connect on v2:172.21.15.124:6833/1919704285 2023-10-06T10:38:42.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.661+0000 7f0f0affd700 1 --2- 172.21.15.92:0/807572444 >> [v2:172.21.15.124:6833/1919704285,v1:172.21.15.124:6835/1919704285] conn(0x55ce6289c040 0x55ce6289c410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6ff91b8ad7599f9 server_cookie=f368ec280d308760 in_seq=0 out_seq=0 2023-10-06T10:38:42.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.661+0000 7f0ee9ffb700 1 -- 172.21.15.92:0/807572444 <== mds.0 v2:172.21.15.124:6833/1919704285 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55ce6282b5c0 con 0x55ce6289c040 2023-10-06T10:38:42.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.661+0000 7f0ee9ffb700 10 client.4782 handle_command_reply: tid=0 2023-10-06T10:38:42.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.661+0000 7f0f1b4fdb80 1 client.4782 shutdown 2023-10-06T10:38:42.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.662+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/807572444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce6285abd0 msgr2=0x55ce628b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:42.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.662+0000 7f0f1b4fdb80 1 --2- 172.21.15.92:0/807572444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce6285abd0 0x55ce628b3ac0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00c470 tx=0x7f0efc082d60 comp rx=0 tx=0).stop 2023-10-06T10:38:42.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.662+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/807572444 shutdown_connections 2023-10-06T10:38:42.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.662+0000 7f0f1b4fdb80 1 --2- 172.21.15.92:0/807572444 >> [v2:172.21.15.124:6833/1919704285,v1:172.21.15.124:6835/1919704285] conn(0x55ce6289c040 0x55ce6289c410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.662+0000 7f0f1b4fdb80 1 --2- 172.21.15.92:0/807572444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce628a0a90 0x55ce62854c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.662+0000 7f0f1b4fdb80 1 --2- 172.21.15.92:0/807572444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce6285abd0 0x55ce628b3ac0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.662+0000 7f0ee9ffb700 0 client.4782 ms_handle_reset on v2:172.21.15.124:6833/1919704285 2023-10-06T10:38:42.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.662+0000 7f0f1b4fdb80 1 --2- 172.21.15.92:0/807572444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ce6285c9a0 0x55ce62854740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.662+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/807572444 >> 172.21.15.92:0/807572444 conn(0x55ce6285b070 msgr2=0x55ce6289e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:42.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.662+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/807572444 shutdown_connections 2023-10-06T10:38:42.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.662+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/807572444 wait complete. 2023-10-06T10:38:42.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.662+0000 7f0f1b4fdb80 20 client.4782 trim_cache size 0 max 16384 2023-10-06T10:38:42.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.678+0000 7f0f1b4fdb80 1 Processor -- start 2023-10-06T10:38:42.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.678+0000 7f0f1b4fdb80 1 -- start start 2023-10-06T10:38:42.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.679+0000 7f0f1b4fdb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ce6285abd0 0x55ce6289d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.679+0000 7f0f1b4fdb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce6285c9a0 0x55ce62885650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.679+0000 7f0f1b4fdb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce628a0a90 0x55ce6289b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.679+0000 7f0f1b4fdb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce6262ce30 con 0x55ce628a0a90 2023-10-06T10:38:42.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.679+0000 7f0f1b4fdb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce6289ed10 con 0x55ce6285c9a0 2023-10-06T10:38:42.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.679+0000 7f0f1b4fdb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce628540c0 con 0x55ce6285abd0 2023-10-06T10:38:42.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.679+0000 7f0f0bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce628a0a90 0x55ce6289b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.679+0000 7f0f0bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce628a0a90 0x55ce6289b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37210/0 (socket says 172.21.15.92:37210) 2023-10-06T10:38:42.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.679+0000 7f0f0bfff700 1 -- 172.21.15.92:0/1025712205 learned_addr learned my addr 172.21.15.92:0/1025712205 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:42.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.679+0000 7f0f0b7fe700 1 --2- 172.21.15.92:0/1025712205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ce6285abd0 0x55ce6289d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.679+0000 7f0f0affd700 1 --2- 172.21.15.92:0/1025712205 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce6285c9a0 0x55ce62885650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.679+0000 7f0f0bfff700 1 -- 172.21.15.92:0/1025712205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ce6285abd0 msgr2=0x55ce6289d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:42.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.679+0000 7f0f0bfff700 1 --2- 172.21.15.92:0/1025712205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ce6285abd0 0x55ce6289d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.679+0000 7f0f0bfff700 1 -- 172.21.15.92:0/1025712205 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce6285c9a0 msgr2=0x55ce62885650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:42.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.679+0000 7f0f0bfff700 1 --2- 172.21.15.92:0/1025712205 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce6285c9a0 0x55ce62885650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.679+0000 7f0f0bfff700 1 -- 172.21.15.92:0/1025712205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce62778780 con 0x55ce628a0a90 2023-10-06T10:38:42.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.680+0000 7f0f0bfff700 1 --2- 172.21.15.92:0/1025712205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce628a0a90 0x55ce6289b790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55ce6285cef0 tx=0x7f0f0401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504fb0cefe8461d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:42.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.680+0000 7f0ef17fa700 1 -- 172.21.15.92:0/1025712205 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f04009eb0 con 0x55ce628a0a90 2023-10-06T10:38:42.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.680+0000 7f0ef17fa700 1 -- 172.21.15.92:0/1025712205 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f04025040 con 0x55ce628a0a90 2023-10-06T10:38:42.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.680+0000 7f0ef17fa700 1 -- 172.21.15.92:0/1025712205 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f04013b10 con 0x55ce628a0a90 2023-10-06T10:38:42.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.680+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/1025712205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce628a0a90 msgr2=0x55ce6289b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:42.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.680+0000 7f0f1b4fdb80 1 --2- 172.21.15.92:0/1025712205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce628a0a90 0x55ce6289b790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55ce6285cef0 tx=0x7f0f0401d000 comp rx=0 tx=0).stop 2023-10-06T10:38:42.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.680+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/1025712205 shutdown_connections 2023-10-06T10:38:42.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.680+0000 7f0f1b4fdb80 1 --2- 172.21.15.92:0/1025712205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce628a0a90 0x55ce6289b790 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.680+0000 7f0f1b4fdb80 1 --2- 172.21.15.92:0/1025712205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ce6285abd0 0x55ce6289d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.680+0000 7f0f1b4fdb80 1 --2- 172.21.15.92:0/1025712205 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce6285c9a0 0x55ce62885650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.680+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/1025712205 >> 172.21.15.92:0/1025712205 conn(0x55ce6285b070 msgr2=0x55ce6285b440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:42.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.680+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/1025712205 shutdown_connections 2023-10-06T10:38:42.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.680+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/1025712205 wait complete. 2023-10-06T10:38:42.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.681+0000 7f0f1b4fdb80 1 -- start start 2023-10-06T10:38:42.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.681+0000 7f0f1b4fdb80 1 Processor -- start 2023-10-06T10:38:42.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.681+0000 7f0f1b4fdb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce6285abd0 0x55ce628b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0f1b4fdb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ce6285c9a0 0x55ce628a8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0f1b4fdb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce628a0a90 0x55ce628a8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0f1b4fdb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce62857360 con 0x55ce6285abd0 2023-10-06T10:38:42.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0f1b4fdb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce628b1160 con 0x55ce628a0a90 2023-10-06T10:38:42.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0f1b4fdb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce628896e0 con 0x55ce6285c9a0 2023-10-06T10:38:42.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0f1b4fdb80 10 client.0 _ll_register_callbacks cb 0x55ce62646f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:38:42.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0f0b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce6285abd0 0x55ce628b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0f0b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce6285abd0 0x55ce628b0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37224/0 (socket says 172.21.15.92:37224) 2023-10-06T10:38:42.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0f0b7fe700 1 -- 172.21.15.92:0/1250586579 learned_addr learned my addr 172.21.15.92:0/1250586579 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:42.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0f0affd700 1 --2- 172.21.15.92:0/1250586579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ce6285c9a0 0x55ce628a8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0f0bfff700 1 --2- 172.21.15.92:0/1250586579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce628a0a90 0x55ce628a8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0f0b7fe700 1 -- 172.21.15.92:0/1250586579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ce6285c9a0 msgr2=0x55ce628a8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:42.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0f0b7fe700 1 --2- 172.21.15.92:0/1250586579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ce6285c9a0 0x55ce628a8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0f0b7fe700 1 -- 172.21.15.92:0/1250586579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce628a0a90 msgr2=0x55ce628a8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:42.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0f0b7fe700 1 --2- 172.21.15.92:0/1250586579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce628a0a90 0x55ce628a8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0f0b7fe700 1 -- 172.21.15.92:0/1250586579 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce62644270 con 0x55ce6285abd0 2023-10-06T10:38:42.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0f0bfff700 1 --2- 172.21.15.92:0/1250586579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce628a0a90 0x55ce628a8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:42.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0f0b7fe700 1 --2- 172.21.15.92:0/1250586579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce6285abd0 0x55ce628b0c20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0efc082ee0 tx=0x7f0efc071000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c9cc10adc5813f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:42.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0ee9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:38:42.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0ee9ffb700 1 -- 172.21.15.92:0/1250586579 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc055cd0 con 0x55ce6285abd0 2023-10-06T10:38:42.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0ee9ffb700 1 -- 172.21.15.92:0/1250586579 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f0efc05c7b0 con 0x55ce6285abd0 2023-10-06T10:38:42.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/1250586579 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f04013750 con 0x55ce6285abd0 2023-10-06T10:38:42.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0ee9ffb700 1 -- 172.21.15.92:0/1250586579 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0efc05c320 con 0x55ce6285abd0 2023-10-06T10:38:42.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0f1b4fdb80 10 client.4788 fetch_fsmap learned FSMap version 8 2023-10-06T10:38:42.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/1250586579 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55ce6275f470 con 0x55ce6285abd0 2023-10-06T10:38:42.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.682+0000 7f0ee9ffb700 1 -- 172.21.15.92:0/1250586579 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f04013750 con 0x55ce6285abd0 2023-10-06T10:38:42.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.683+0000 7f0ee9ffb700 1 -- 172.21.15.92:0/1250586579 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 8) v1 ==== 3108+0+0 (secure 0 0 0) 0x7f0efc051670 con 0x55ce6285abd0 2023-10-06T10:38:42.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.683+0000 7f0f1b4fdb80 10 client.4788 fetch_fsmap finished waiting for FSMap version 8 2023-10-06T10:38:42.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.683+0000 7f0f1b4fdb80 10 client.4788 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T10:38:42.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.683+0000 7f0f1b4fdb80 20 client.4788 populate_metadata read hostname 'smithi092' 2023-10-06T10:38:42.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.683+0000 7f0f1b4fdb80 1 --2- 172.21.15.92:0/1250586579 >> [v2:172.21.15.124:6833/1919704285,v1:172.21.15.124:6835/1919704285] conn(0x55ce62905710 0x55ce6289f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:42.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.683+0000 7f0f1b4fdb80 4 client.4788 mds_command: new command op to 4306 tid=0[{"prefix": "status"}] 2023-10-06T10:38:42.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.683+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/1250586579 --> [v2:172.21.15.124:6833/1919704285,v1:172.21.15.124:6835/1919704285] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ce6274ce00 con 0x55ce62905710 2023-10-06T10:38:42.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.683+0000 7f0f0affd700 1 --2- 172.21.15.92:0/1250586579 >> [v2:172.21.15.124:6833/1919704285,v1:172.21.15.124:6835/1919704285] conn(0x55ce62905710 0x55ce6289f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:42.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.683+0000 7f0ee9ffb700 10 client.4788 ms_handle_connect on v2:172.21.15.124:6833/1919704285 2023-10-06T10:38:42.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.683+0000 7f0f0affd700 1 --2- 172.21.15.92:0/1250586579 >> [v2:172.21.15.124:6833/1919704285,v1:172.21.15.124:6835/1919704285] conn(0x55ce62905710 0x55ce6289f280 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=905237d94a5ad0f4 server_cookie=6f113e64edeb725e in_seq=0 out_seq=0 2023-10-06T10:38:42.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.683+0000 7f0ee9ffb700 1 -- 172.21.15.92:0/1250586579 <== mds.0 v2:172.21.15.124:6833/1919704285 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+317 (crc 0 0 0) 0x55ce6274ce00 con 0x55ce62905710 2023-10-06T10:38:42.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.683+0000 7f0ee9ffb700 10 client.4788 handle_command_reply: tid=0 2023-10-06T10:38:42.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.683+0000 7f0f1b4fdb80 1 client.4788 shutdown 2023-10-06T10:38:42.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.683+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/1250586579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce6285abd0 msgr2=0x55ce628b0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:42.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.683+0000 7f0f1b4fdb80 1 --2- 172.21.15.92:0/1250586579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce6285abd0 0x55ce628b0c20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0efc082ee0 tx=0x7f0efc071000 comp rx=0 tx=0).stop 2023-10-06T10:38:42.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.684+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/1250586579 shutdown_connections 2023-10-06T10:38:42.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.684+0000 7f0f1b4fdb80 1 --2- 172.21.15.92:0/1250586579 >> [v2:172.21.15.124:6833/1919704285,v1:172.21.15.124:6835/1919704285] conn(0x55ce62905710 0x55ce6289f280 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.684+0000 7f0f1b4fdb80 1 --2- 172.21.15.92:0/1250586579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ce628a0a90 0x55ce628a8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.684+0000 7f0ee9ffb700 0 client.4788 ms_handle_reset on v2:172.21.15.124:6833/1919704285 2023-10-06T10:38:42.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.684+0000 7f0f1b4fdb80 1 --2- 172.21.15.92:0/1250586579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce6285abd0 0x55ce628b0c20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.684+0000 7f0f1b4fdb80 1 --2- 172.21.15.92:0/1250586579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ce6285c9a0 0x55ce628a8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.684+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/1250586579 >> 172.21.15.92:0/1250586579 conn(0x55ce6285b070 msgr2=0x55ce62882630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:42.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.684+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/1250586579 shutdown_connections 2023-10-06T10:38:42.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.684+0000 7f0f1b4fdb80 1 -- 172.21.15.92:0/1250586579 wait complete. 2023-10-06T10:38:42.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.684+0000 7f0f1b4fdb80 20 client.4788 trim_cache size 0 max 16384 2023-10-06T10:38:42.694 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:38:42.694 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:38:42.694 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T10:38:42.694 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4306, 2023-10-06T10:38:42.694 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:38:42.695 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:38:42.695 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:38:42.695 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 25.340444197, 2023-10-06T10:38:42.695 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 8, 2023-10-06T10:38:42.695 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 24, 2023-10-06T10:38:42.695 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 24, 2023-10-06T10:38:42.695 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 31.25379839 2023-10-06T10:38:42.695 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:38:42.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.684+0000 7f0f12205700 1 -- 172.21.15.92:0/2321896236 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0ef8066f30 msgr2=0x7f0ef80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:42.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.684+0000 7f0f12205700 1 --2- 172.21.15.92:0/2321896236 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0ef8066f30 0x7f0ef80693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0f000091e0 tx=0x7f0f0001a000 comp rx=0 tx=0).stop 2023-10-06T10:38:42.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.684+0000 7f0f12205700 1 -- 172.21.15.92:0/2321896236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef4003fe0 msgr2=0x7f0f0c151850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:42.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.684+0000 7f0f12205700 1 --2- 172.21.15.92:0/2321896236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef4003fe0 0x7f0f0c151850 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0efc001880 tx=0x7f0efc009040 comp rx=0 tx=0).stop 2023-10-06T10:38:42.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.684+0000 7f0f12205700 1 -- 172.21.15.92:0/2321896236 shutdown_connections 2023-10-06T10:38:42.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.684+0000 7f0f12205700 1 --2- 172.21.15.92:0/2321896236 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0ef8066f30 0x7f0ef80693e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.684+0000 7f0f12205700 1 --2- 172.21.15.92:0/2321896236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f0c16dd90 0x7f0f0c16d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.684+0000 7f0f12205700 1 --2- 172.21.15.92:0/2321896236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef4003fe0 0x7f0f0c151850 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.684+0000 7f0f12205700 1 --2- 172.21.15.92:0/2321896236 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0f0c0c3f20 0x7f0f0c0cfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:42.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.684+0000 7f0f12205700 1 -- 172.21.15.92:0/2321896236 >> 172.21.15.92:0/2321896236 conn(0x7f0f0c06ece0 msgr2=0x7f0f0c157260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:42.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.684+0000 7f0f12205700 1 -- 172.21.15.92:0/2321896236 shutdown_connections 2023-10-06T10:38:42.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:42.684+0000 7f0f12205700 1 -- 172.21.15.92:0/2321896236 wait complete. 2023-10-06T10:38:42.698 INFO:teuthology.run_tasks:Running task kclient... 2023-10-06T10:38:42.712 INFO:tasks.kclient:Mounting kernel clients... 2023-10-06T10:38:42.712 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-06T10:38:42.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:42.712 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T10:38:42.728 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-10-06T10:38:42.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:42.728 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-06T10:38:42.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:42.802 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2023-10-06T10:38:42.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:42.892 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T10:38:42.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:42.908 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2023-10-06T10:38:42.974 INFO:teuthology.orchestra.run.smithi092.stderr:Cannot find device "ceph-brx" 2023-10-06T10:38:42.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T10:38:42.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:42.976 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2023-10-06T10:38:42.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:42.994 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link delete ceph-brx 2023-10-06T10:38:43.064 INFO:teuthology.orchestra.run.smithi124.stderr:Cannot find device "ceph-brx" 2023-10-06T10:38:43.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T10:38:43.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:43.066 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2023-10-06T10:38:43.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:43.123 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link delete ceph-brx 2023-10-06T10:38:43.190 INFO:teuthology.orchestra.run.smithi124.stderr:Cannot find device "ceph-brx" 2023-10-06T10:38:43.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T10:38:43.192 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-10-06T10:38:43.192 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T10:38:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.307+0000 7f1f91f8e700 1 Processor -- start 2023-10-06T10:38:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.307+0000 7f1f91f8e700 1 -- start start 2023-10-06T10:38:43.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.308+0000 7f1f91f8e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f8c0adf20 0x7f1f8c0ac000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:43.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.308+0000 7f1f91f8e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1f8c06e240 0x7f1f8c06e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:43.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.308+0000 7f1f91f8e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f8c0c7330 con 0x7f1f8c0cc7f0 2023-10-06T10:38:43.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.308+0000 7f1f91f8e700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f8c156110 con 0x7f1f8c06e240 2023-10-06T10:38:43.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.308+0000 7f1f91f8e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c0ac540 con 0x7f1f8c0adf20 2023-10-06T10:38:43.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.308+0000 7f1f8bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1f8c0cc7f0 0x7f1f8c069520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40234/0 (socket says 172.21.15.92:40234) 2023-10-06T10:38:43.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.308+0000 7f1f8bfff700 1 -- 172.21.15.92:0/3523450970 learned_addr learned my addr 172.21.15.92:0/3523450970 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:43.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.308+0000 7f1f8affd700 1 --2- 172.21.15.92:0/3523450970 >> v2:172.21.15.124:3300/0 conn(0x7f1f8c06e240 0x7f1f8c06e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:43.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.308+0000 7f1f8b7fe700 1 --2- 172.21.15.92:0/3523450970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f8c0adf20 0x7f1f8c0ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:43.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.308+0000 7f1f8a7fc700 1 -- 172.21.15.92:0/3523450970 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 100722772 0 0) 0x7f1f8c0c7330 con 0x7f1f8c0cc7f0 2023-10-06T10:38:43.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.308+0000 7f1f8a7fc700 1 -- 172.21.15.92:0/3523450970 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f74003570 con 0x7f1f8c0cc7f0 2023-10-06T10:38:43.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.308+0000 7f1f8b7fe700 1 -- 172.21.15.92:0/3523450970 >> v2:172.21.15.124:3300/0 conn(0x7f1f8c06e240 msgr2=0x7f1f8c06e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:43.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.308+0000 7f1f8b7fe700 1 --2- 172.21.15.92:0/3523450970 >> v2:172.21.15.124:3300/0 conn(0x7f1f8c06e240 0x7f1f8c06e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:43.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.308+0000 7f1f8b7fe700 1 -- 172.21.15.92:0/3523450970 >> v1:172.21.15.92:6789/0 conn(0x7f1f8c0cc7f0 legacy=0x7f1f8c069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:43.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.308+0000 7f1f8a7fc700 1 -- 172.21.15.92:0/3523450970 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1092856902 0 0) 0x7f1f74003570 con 0x7f1f8c0cc7f0 2023-10-06T10:38:43.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.308+0000 7f1f8b7fe700 1 -- 172.21.15.92:0/3523450970 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c0c6720 con 0x7f1f8c0adf20 2023-10-06T10:38:43.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.308+0000 7f1f8affd700 1 --2- 172.21.15.92:0/3523450970 >> v2:172.21.15.124:3300/0 conn(0x7f1f8c06e240 0x7f1f8c06e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:38:43.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.309+0000 7f1f8b7fe700 1 --2- 172.21.15.92:0/3523450970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f8c0adf20 0x7f1f8c0ac000 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0098e0 tx=0x7f1f7c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8ebefc0afe6e242 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:43.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.309+0000 7f1f8a7fc700 1 -- 172.21.15.92:0/3523450970 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f840017f0 con 0x7f1f8c0adf20 2023-10-06T10:38:43.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.309+0000 7f1f8a7fc700 1 -- 172.21.15.92:0/3523450970 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7c011420 con 0x7f1f8c0adf20 2023-10-06T10:38:43.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.309+0000 7f1f8a7fc700 1 -- 172.21.15.92:0/3523450970 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0117c0 con 0x7f1f8c0adf20 2023-10-06T10:38:43.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.309+0000 7f1f91f8e700 1 -- 172.21.15.92:0/3523450970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f8c0adf20 msgr2=0x7f1f8c0ac000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:43.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.309+0000 7f1f91f8e700 1 --2- 172.21.15.92:0/3523450970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f8c0adf20 0x7f1f8c0ac000 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0098e0 tx=0x7f1f7c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:43.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.309+0000 7f1f91f8e700 1 -- 172.21.15.92:0/3523450970 shutdown_connections 2023-10-06T10:38:43.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.309+0000 7f1f91f8e700 1 --2- 172.21.15.92:0/3523450970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f8c0adf20 0x7f1f8c0ac000 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:43.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.309+0000 7f1f91f8e700 1 --2- 172.21.15.92:0/3523450970 >> v2:172.21.15.124:3300/0 conn(0x7f1f8c06e240 0x7f1f8c06e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:43.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.309+0000 7f1f91f8e700 1 -- 172.21.15.92:0/3523450970 >> 172.21.15.92:0/3523450970 conn(0x7f1f8c068080 msgr2=0x7f1f8c0cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:43.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.309+0000 7f1f91f8e700 1 -- 172.21.15.92:0/3523450970 shutdown_connections 2023-10-06T10:38:43.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.309+0000 7f1f91f8e700 1 -- 172.21.15.92:0/3523450970 wait complete. 2023-10-06T10:38:43.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.311+0000 7f1f91f8e700 1 Processor -- start 2023-10-06T10:38:43.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.311+0000 7f1f91f8e700 1 -- start start 2023-10-06T10:38:43.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.311+0000 7f1f91f8e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c06e240 0x7f1f8c14e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:43.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.311+0000 7f1f91f8e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f8c0adf20 0x7f1f8c0c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:43.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.311+0000 7f1f91f8e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f8c0cc7f0 0x7f1f8c0c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:43.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.311+0000 7f1f91f8e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c0ab510 con 0x7f1f8c06e240 2023-10-06T10:38:43.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.311+0000 7f1f91f8e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c0c32b0 con 0x7f1f8c0cc7f0 2023-10-06T10:38:43.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.311+0000 7f1f91f8e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c14ec70 con 0x7f1f8c0adf20 2023-10-06T10:38:43.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.311+0000 7f1f8b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c06e240 0x7f1f8c14e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:43.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.311+0000 7f1f8b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c06e240 0x7f1f8c14e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37234/0 (socket says 172.21.15.92:37234) 2023-10-06T10:38:43.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.311+0000 7f1f8b7fe700 1 -- 172.21.15.92:0/2391424502 learned_addr learned my addr 172.21.15.92:0/2391424502 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:43.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.312+0000 7f1f8affd700 1 --2- 172.21.15.92:0/2391424502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f8c0adf20 0x7f1f8c0c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:43.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.312+0000 7f1f8bfff700 1 --2- 172.21.15.92:0/2391424502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f8c0cc7f0 0x7f1f8c0c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:43.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.312+0000 7f1f8b7fe700 1 -- 172.21.15.92:0/2391424502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f8c0adf20 msgr2=0x7f1f8c0c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:43.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.312+0000 7f1f8b7fe700 1 --2- 172.21.15.92:0/2391424502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f8c0adf20 0x7f1f8c0c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:43.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.312+0000 7f1f8b7fe700 1 -- 172.21.15.92:0/2391424502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f8c0cc7f0 msgr2=0x7f1f8c0c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:43.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.312+0000 7f1f8b7fe700 1 --2- 172.21.15.92:0/2391424502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f8c0cc7f0 0x7f1f8c0c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:43.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.312+0000 7f1f8b7fe700 1 -- 172.21.15.92:0/2391424502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f80009420 con 0x7f1f8c06e240 2023-10-06T10:38:43.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.312+0000 7f1f8bfff700 1 --2- 172.21.15.92:0/2391424502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f8c0cc7f0 0x7f1f8c0c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:43.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.312+0000 7f1f8b7fe700 1 --2- 172.21.15.92:0/2391424502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c06e240 0x7f1f8c14e730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0030c0 tx=0x7f1f7c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:43.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.312+0000 7f1f737fe700 1 -- 172.21.15.92:0/2391424502 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c01cbe0 con 0x7f1f8c06e240 2023-10-06T10:38:43.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.312+0000 7f1f737fe700 1 -- 172.21.15.92:0/2391424502 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7c01cd40 con 0x7f1f8c06e240 2023-10-06T10:38:43.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.312+0000 7f1f91f8e700 1 -- 172.21.15.92:0/2391424502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7c009420 con 0x7f1f8c06e240 2023-10-06T10:38:43.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.312+0000 7f1f737fe700 1 -- 172.21.15.92:0/2391424502 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c025680 con 0x7f1f8c06e240 2023-10-06T10:38:43.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.312+0000 7f1f91f8e700 1 -- 172.21.15.92:0/2391424502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8c0aaf50 con 0x7f1f8c06e240 2023-10-06T10:38:43.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.313+0000 7f1f91f8e700 1 -- 172.21.15.92:0/2391424502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8c064a90 con 0x7f1f8c06e240 2023-10-06T10:38:43.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.313+0000 7f1f737fe700 1 -- 172.21.15.92:0/2391424502 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1f7c0193f0 con 0x7f1f8c06e240 2023-10-06T10:38:43.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.314+0000 7f1f737fe700 1 --2- 172.21.15.92:0/2391424502 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1f78066f30 0x7f1f780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:43.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.314+0000 7f1f737fe700 1 -- 172.21.15.92:0/2391424502 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f1f7c085bf0 con 0x7f1f8c06e240 2023-10-06T10:38:43.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.314+0000 7f1f8affd700 1 --2- 172.21.15.92:0/2391424502 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1f78066f30 0x7f1f780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:43.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.314+0000 7f1f8affd700 1 --2- 172.21.15.92:0/2391424502 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1f78066f30 0x7f1f780693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c16cdb0 tx=0x7f1f800091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:43.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.318+0000 7f1f737fe700 1 -- 172.21.15.92:0/2391424502 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f7c052a90 con 0x7f1f8c06e240 2023-10-06T10:38:43.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.550+0000 7f1f91f8e700 1 -- 172.21.15.92:0/2391424502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1f8c0c6a00 con 0x7f1f8c06e240 2023-10-06T10:38:43.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.551+0000 7f1f737fe700 1 -- 172.21.15.92:0/2391424502 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f1f7c02a070 con 0x7f1f8c06e240 2023-10-06T10:38:43.552 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T10:38:43.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.553+0000 7f1f91f8e700 1 -- 172.21.15.92:0/2391424502 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1f78066f30 msgr2=0x7f1f780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:43.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.553+0000 7f1f91f8e700 1 --2- 172.21.15.92:0/2391424502 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1f78066f30 0x7f1f780693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c16cdb0 tx=0x7f1f800091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:43.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.553+0000 7f1f91f8e700 1 -- 172.21.15.92:0/2391424502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c06e240 msgr2=0x7f1f8c14e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:43.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.553+0000 7f1f91f8e700 1 --2- 172.21.15.92:0/2391424502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c06e240 0x7f1f8c14e730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0030c0 tx=0x7f1f7c016000 comp rx=0 tx=0).stop 2023-10-06T10:38:43.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.553+0000 7f1f91f8e700 1 -- 172.21.15.92:0/2391424502 shutdown_connections 2023-10-06T10:38:43.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.553+0000 7f1f91f8e700 1 --2- 172.21.15.92:0/2391424502 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1f78066f30 0x7f1f780693e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:43.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.553+0000 7f1f91f8e700 1 --2- 172.21.15.92:0/2391424502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f8c0cc7f0 0x7f1f8c0c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:43.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.553+0000 7f1f91f8e700 1 --2- 172.21.15.92:0/2391424502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c06e240 0x7f1f8c14e730 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:43.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.553+0000 7f1f91f8e700 1 --2- 172.21.15.92:0/2391424502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f8c0adf20 0x7f1f8c0c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:43.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.553+0000 7f1f91f8e700 1 -- 172.21.15.92:0/2391424502 >> 172.21.15.92:0/2391424502 conn(0x7f1f8c068080 msgr2=0x7f1f8c152c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:43.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.553+0000 7f1f91f8e700 1 -- 172.21.15.92:0/2391424502 shutdown_connections 2023-10-06T10:38:43.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:43.553+0000 7f1f91f8e700 1 -- 172.21.15.92:0/2391424502 wait complete. 2023-10-06T10:38:43.566 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T10:38:43.566 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T10:38:43.566 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T10:38:43.566 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T10:38:43.566 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T10:38:43.566 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T10:38:43.567 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T10:38:43.567 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T10:38:43.567 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T10:38:43.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:43.567 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T10:38:43.583 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T10:38:43.583 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T10:38:43.583 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T10:38:43.584 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:43.584 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T10:38:43.584 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:43.584 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:38:43.584 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:38:43.584 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T10:38:43.585 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:38:43.585 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:38:43.585 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T10:38:43.585 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:38:43.585 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:38:43.585 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T10:38:43.586 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:38:43.586 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T10:38:43.586 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42486sec preferred_lft 42486sec 2023-10-06T10:38:43.586 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T10:38:43.586 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:43.587 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T10:38:43.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:43.587 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T10:38:43.587 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T10:38:43.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add name ceph-brx type bridge 2023-10-06T10:38:43.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr flush dev ceph-brx 2023-10-06T10:38:43.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set ceph-brx up 2023-10-06T10:38:43.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T10:38:43.588 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T10:38:43.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:43.693 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T10:38:43.717 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T10:38:43.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:43.718 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T10:38:43.775 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T10:38:43.775 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T10:38:43.775 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T10:38:43.775 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T10:38:43.775 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T10:38:43.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:43.776 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T10:38:43.776 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T10:38:43.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T10:38:43.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T10:38:43.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T10:38:43.776 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T10:38:43.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:43.876 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T10:38:43.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:43.891 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list-id 2023-10-06T10:38:43.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:43.945 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T10:38:43.945 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T10:38:43.945 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T10:38:43.945 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-06T10:38:43.945 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T10:38:44.033 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-06T10:38:44.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:44.033 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T10:38:44.034 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T10:38:44.034 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-06T10:38:44.034 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T10:38:44.034 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-06T10:38:44.034 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-06T10:38:44.034 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-06T10:38:44.034 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T10:38:44.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:44.208 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T10:38:44.208 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T10:38:44.208 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-10-06T10:38:44.209 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T10:38:44.209 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T10:38:44.300 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T10:38:44.301 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T10:38:44.315 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T10:38:44.315 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T10:38:44.316 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T10:38:44.370 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T10:38:44.370 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T10:38:44.370 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:38:44.449 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:38:44.449 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:38:44.551 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:38:44.551 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:38:44.551 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-06T10:38:44.551 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@8f307bfb-c4cb-4f66-aade-46fffe0c6e0c.cephfs=/ 2023-10-06T10:38:44.552 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-06T10:38:44.552 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T10:38:44.552 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T10:38:44.552 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T10:38:44.573 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:38:44.574 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:38:44.574 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-06T10:38:44.574 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@8f307bfb-c4cb-4f66-aade-46fffe0c6e0c.cephfs=/ 2023-10-06T10:38:44.574 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-06T10:38:44.575 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T10:38:44.575 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T10:38:44.575 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T10:38:44.575 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.124:6789,172.21.15.124:6790" 2023-10-06T10:38:44.575 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.124:6789,172.21.15.124:6790:/ 2023-10-06T10:38:44.575 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=8f307bfb-c4cb-4f66-aade-46fffe0c6e0c" will pass to kernel 2023-10-06T10:38:44.576 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T10:38:44.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:44.576 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T10:38:44.641 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-10-06T10:38:44.641 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T10:38:44.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.794+0000 7fda2c797700 1 Processor -- start 2023-10-06T10:38:44.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.794+0000 7fda2c797700 1 -- start start 2023-10-06T10:38:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.795+0000 7fda2c797700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fda24156350 0x7fda240cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.795+0000 7fda2c797700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fda240cc710 0x7fda2406bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.795+0000 7fda2c797700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda2406c9a0 con 0x7fda2406c160 2023-10-06T10:38:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.795+0000 7fda2c797700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda240ccb90 con 0x7fda240cc710 2023-10-06T10:38:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.795+0000 7fda2c797700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda24156b20 con 0x7fda24156350 2023-10-06T10:38:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.795+0000 7fda29d32700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fda240cc710 0x7fda2406bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.796+0000 7fda29d32700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fda240cc710 0x7fda2406bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38446/0 (socket says 172.21.15.92:38446) 2023-10-06T10:38:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.796+0000 7fda29d32700 1 -- 172.21.15.92:0/1689281376 learned_addr learned my addr 172.21.15.92:0/1689281376 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.796+0000 7fda2a533700 1 --2- 172.21.15.92:0/1689281376 >> v2:172.21.15.124:3300/0 conn(0x7fda24156350 0x7fda240cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.796+0000 7fda29d32700 1 -- 172.21.15.92:0/1689281376 >> v2:172.21.15.124:3300/0 conn(0x7fda24156350 msgr2=0x7fda240cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:44.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.796+0000 7fda29d32700 1 --2- 172.21.15.92:0/1689281376 >> v2:172.21.15.124:3300/0 conn(0x7fda24156350 0x7fda240cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:44.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.796+0000 7fda29d32700 1 -- 172.21.15.92:0/1689281376 >> v1:172.21.15.124:6789/0 conn(0x7fda2406c160 legacy=0x7fda240cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:44.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.796+0000 7fda29d32700 1 -- 172.21.15.92:0/1689281376 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda24156e00 con 0x7fda240cc710 2023-10-06T10:38:44.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.796+0000 7fda29d32700 1 --2- 172.21.15.92:0/1689281376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda240cc710 0x7fda2406bc20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fda24069f50 tx=0x7fda18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675a95a63ac4e63e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:44.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.796+0000 7fda29531700 1 -- 172.21.15.92:0/1689281376 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18009e70 con 0x7fda240cc710 2023-10-06T10:38:44.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.796+0000 7fda29531700 1 -- 172.21.15.92:0/1689281376 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1800b860 con 0x7fda240cc710 2023-10-06T10:38:44.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.796+0000 7fda29531700 1 -- 172.21.15.92:0/1689281376 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1800bc00 con 0x7fda240cc710 2023-10-06T10:38:44.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.797+0000 7fda2c797700 1 -- 172.21.15.92:0/1689281376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda240cc710 msgr2=0x7fda2406bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:44.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.797+0000 7fda2c797700 1 --2- 172.21.15.92:0/1689281376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda240cc710 0x7fda2406bc20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fda24069f50 tx=0x7fda18009250 comp rx=0 tx=0).stop 2023-10-06T10:38:44.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.797+0000 7fda2c797700 1 -- 172.21.15.92:0/1689281376 shutdown_connections 2023-10-06T10:38:44.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.797+0000 7fda2c797700 1 --2- 172.21.15.92:0/1689281376 >> v2:172.21.15.124:3300/0 conn(0x7fda24156350 0x7fda240cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:44.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.797+0000 7fda2c797700 1 --2- 172.21.15.92:0/1689281376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda240cc710 0x7fda2406bc20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:44.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.797+0000 7fda2c797700 1 -- 172.21.15.92:0/1689281376 >> 172.21.15.92:0/1689281376 conn(0x7fda240cdeb0 msgr2=0x7fda240c9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:44.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.797+0000 7fda2c797700 1 -- 172.21.15.92:0/1689281376 shutdown_connections 2023-10-06T10:38:44.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.797+0000 7fda2c797700 1 -- 172.21.15.92:0/1689281376 wait complete. 2023-10-06T10:38:44.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.798+0000 7fda2c797700 1 Processor -- start 2023-10-06T10:38:44.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.798+0000 7fda2c797700 1 -- start start 2023-10-06T10:38:44.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.798+0000 7fda2c797700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda2406c160 0x7fda24151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:44.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.798+0000 7fda2c797700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda240cc710 0x7fda241581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:44.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.798+0000 7fda2c797700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda24156350 0x7fda240b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:44.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.799+0000 7fda2c797700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda24158ad0 con 0x7fda24156350 2023-10-06T10:38:44.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.799+0000 7fda2c797700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda24158db0 con 0x7fda2406c160 2023-10-06T10:38:44.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.799+0000 7fda2c797700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda24159090 con 0x7fda240cc710 2023-10-06T10:38:44.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.799+0000 7fda2ad34700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda24156350 0x7fda240b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:44.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.799+0000 7fda2a533700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda2406c160 0x7fda24151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:44.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.799+0000 7fda2ad34700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda24156350 0x7fda240b2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38452/0 (socket says 172.21.15.92:38452) 2023-10-06T10:38:44.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.799+0000 7fda2a533700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda2406c160 0x7fda24151af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:44824/0 (socket says 172.21.15.92:44824) 2023-10-06T10:38:44.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.799+0000 7fda2ad34700 1 -- 172.21.15.92:0/3939549843 learned_addr learned my addr 172.21.15.92:0/3939549843 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:44.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.799+0000 7fda29d32700 1 --2- 172.21.15.92:0/3939549843 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda240cc710 0x7fda241581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:44.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.799+0000 7fda2ad34700 1 -- 172.21.15.92:0/3939549843 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda240cc710 msgr2=0x7fda241581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:44.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.799+0000 7fda2ad34700 1 --2- 172.21.15.92:0/3939549843 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda240cc710 0x7fda241581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:44.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.799+0000 7fda2ad34700 1 -- 172.21.15.92:0/3939549843 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda2406c160 msgr2=0x7fda24151af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:44.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.799+0000 7fda2ad34700 1 --2- 172.21.15.92:0/3939549843 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda2406c160 0x7fda24151af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:44.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.799+0000 7fda2ad34700 1 -- 172.21.15.92:0/3939549843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda180094f0 con 0x7fda24156350 2023-10-06T10:38:44.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.799+0000 7fda2ad34700 1 --2- 172.21.15.92:0/3939549843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda24156350 0x7fda240b2530 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00c560 tx=0x7fda1c007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:44.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.799+0000 7fda227fc700 1 -- 172.21.15.92:0/3939549843 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c011590 con 0x7fda24156350 2023-10-06T10:38:44.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.799+0000 7fda227fc700 1 -- 172.21.15.92:0/3939549843 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1c0116f0 con 0x7fda24156350 2023-10-06T10:38:44.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.799+0000 7fda2c797700 1 -- 172.21.15.92:0/3939549843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda24159370 con 0x7fda24156350 2023-10-06T10:38:44.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.799+0000 7fda227fc700 1 -- 172.21.15.92:0/3939549843 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c011ab0 con 0x7fda24156350 2023-10-06T10:38:44.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.799+0000 7fda2c797700 1 -- 172.21.15.92:0/3939549843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda24158730 con 0x7fda24156350 2023-10-06T10:38:44.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.800+0000 7fda2c797700 1 -- 172.21.15.92:0/3939549843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda24159890 con 0x7fda24156350 2023-10-06T10:38:44.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.801+0000 7fda227fc700 1 -- 172.21.15.92:0/3939549843 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fda1c01b3b0 con 0x7fda24156350 2023-10-06T10:38:44.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.801+0000 7fda227fc700 1 --2- 172.21.15.92:0/3939549843 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda10066f30 0x7fda100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:44.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.801+0000 7fda227fc700 1 -- 172.21.15.92:0/3939549843 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fda1c083d40 con 0x7fda24156350 2023-10-06T10:38:44.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.801+0000 7fda2a533700 1 --2- 172.21.15.92:0/3939549843 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda10066f30 0x7fda100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:44.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.801+0000 7fda2a533700 1 --2- 172.21.15.92:0/3939549843 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda10066f30 0x7fda100693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda140128e0 tx=0x7fda14014040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:44.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:44.805+0000 7fda227fc700 1 -- 172.21.15.92:0/3939549843 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda24159890 con 0x7fda24156350 2023-10-06T10:38:45.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:45.031+0000 7fda2c797700 1 -- 172.21.15.92:0/3939549843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fda24159890 con 0x7fda24156350 2023-10-06T10:38:45.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:45.031+0000 7fda227fc700 1 -- 172.21.15.92:0/3939549843 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7fda24159890 con 0x7fda24156350 2023-10-06T10:38:45.032 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T10:38:45.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:45.033+0000 7fda2c797700 1 -- 172.21.15.92:0/3939549843 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda10066f30 msgr2=0x7fda100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:45.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:45.033+0000 7fda2c797700 1 --2- 172.21.15.92:0/3939549843 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda10066f30 0x7fda100693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda140128e0 tx=0x7fda14014040 comp rx=0 tx=0).stop 2023-10-06T10:38:45.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:45.033+0000 7fda2c797700 1 -- 172.21.15.92:0/3939549843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda24156350 msgr2=0x7fda240b2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:45.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:45.033+0000 7fda2c797700 1 --2- 172.21.15.92:0/3939549843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda24156350 0x7fda240b2530 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00c560 tx=0x7fda1c007ef0 comp rx=0 tx=0).stop 2023-10-06T10:38:45.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:45.033+0000 7fda2c797700 1 -- 172.21.15.92:0/3939549843 shutdown_connections 2023-10-06T10:38:45.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:45.033+0000 7fda2c797700 1 --2- 172.21.15.92:0/3939549843 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda10066f30 0x7fda100693e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:45.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:45.033+0000 7fda2c797700 1 --2- 172.21.15.92:0/3939549843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda24156350 0x7fda240b2530 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:45.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:45.033+0000 7fda2c797700 1 --2- 172.21.15.92:0/3939549843 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda2406c160 0x7fda24151af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:45.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:45.033+0000 7fda2c797700 1 --2- 172.21.15.92:0/3939549843 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda240cc710 0x7fda241581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:45.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:45.033+0000 7fda2c797700 1 -- 172.21.15.92:0/3939549843 >> 172.21.15.92:0/3939549843 conn(0x7fda240cdeb0 msgr2=0x7fda240c2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:45.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:45.033+0000 7fda2c797700 1 -- 172.21.15.92:0/3939549843 shutdown_connections 2023-10-06T10:38:45.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:45.033+0000 7fda2c797700 1 -- 172.21.15.92:0/3939549843 wait complete. 2023-10-06T10:38:45.046 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T10:38:45.046 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T10:38:45.046 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T10:38:45.046 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-06T10:38:45.046 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-06T10:38:45.046 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T10:38:45.046 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T10:38:45.046 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T10:38:45.046 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-06T10:38:45.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:45.047 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T10:38:45.061 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T10:38:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T10:38:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T10:38:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T10:38:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:45.062 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:38:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:38:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T10:38:45.062 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:38:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:38:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T10:38:45.063 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:38:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:38:45.063 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T10:38:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:38:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T10:38:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42484sec preferred_lft 42484sec 2023-10-06T10:38:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T10:38:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:45.063 INFO:teuthology.orchestra.run.smithi092.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T10:38:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 92:7f:88:24:4e:ca brd ff:ff:ff:ff:ff:ff 2023-10-06T10:38:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T10:38:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::9c9d:23ff:fee4:30ae/64 scope link 2023-10-06T10:38:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:45.064 INFO:teuthology.orchestra.run.smithi092.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T10:38:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 92:7f:88:24:4e:ca brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T10:38:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::907f:88ff:fe24:4eca/64 scope link tentative 2023-10-06T10:38:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:45.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:45.065 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T10:38:45.124 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T10:38:45.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:45.126 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T10:38:45.182 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T10:38:45.182 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T10:38:45.182 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T10:38:45.182 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T10:38:45.182 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T10:38:45.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:45.183 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T10:38:45.183 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T10:38:45.183 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T10:38:45.183 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T10:38:45.183 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T10:38:45.183 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T10:38:45.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:45.283 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T10:38:45.297 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T10:38:45.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:45.297 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list-id 2023-10-06T10:38:45.352 INFO:teuthology.orchestra.run.smithi092.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-06T10:38:45.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:45.352 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T10:38:45.353 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T10:38:45.353 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T10:38:45.353 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-06T10:38:45.353 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T10:38:45.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:45.441 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T10:38:45.473 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T10:38:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T10:38:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T10:38:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T10:38:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:45.474 INFO:teuthology.orchestra.run.smithi092.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T10:38:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 7a:78:fa:26:9c:52 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T10:38:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T10:38:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::7878:faff:fe26:9c52/64 scope link tentative 2023-10-06T10:38:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:45.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:45.483 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T10:38:45.555 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T10:38:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T10:38:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T10:38:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T10:38:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:45.556 INFO:teuthology.orchestra.run.smithi092.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T10:38:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 7a:78:fa:26:9c:52 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T10:38:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T10:38:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::7878:faff:fe26:9c52/64 scope link tentative 2023-10-06T10:38:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:45.574 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-06T10:38:45.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:45.574 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T10:38:45.574 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T10:38:45.574 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-06T10:38:45.574 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T10:38:45.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-06T10:38:45.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-06T10:38:45.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-06T10:38:45.575 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T10:38:45.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:45.801 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T10:38:45.802 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T10:38:45.802 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.1 up 2023-10-06T10:38:45.802 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T10:38:45.802 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T10:38:45.882 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T10:38:45.882 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-06T10:38:45.897 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-06T10:38:45.897 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T10:38:45.897 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-06T10:38:45.952 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-10-06T10:38:45.952 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T10:38:45.952 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:38:46.032 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:38:46.032 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:38:46.051 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:38:46.051 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:38:46.052 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-06T10:38:46.052 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 2@8f307bfb-c4cb-4f66-aade-46fffe0c6e0c.cephfs=/ 2023-10-06T10:38:46.052 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.92:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-06T10:38:46.052 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T10:38:46.052 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T10:38:46.052 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T10:38:46.074 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:38:46.074 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:38:46.074 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-06T10:38:46.074 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 2@8f307bfb-c4cb-4f66-aade-46fffe0c6e0c.cephfs=/ 2023-10-06T10:38:46.074 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.92:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-06T10:38:46.075 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T10:38:46.075 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T10:38:46.075 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T10:38:46.075 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.124:6789,172.21.15.124:6790" 2023-10-06T10:38:46.075 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.124:6789,172.21.15.124:6790:/ 2023-10-06T10:38:46.075 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,key=2,mds_namespace=cephfs,fsid=8f307bfb-c4cb-4f66-aade-46fffe0c6e0c" will pass to kernel 2023-10-06T10:38:46.076 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T10:38:46.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:46.077 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-06T10:38:46.141 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-10-06T10:38:46.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T10:38:46.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.294+0000 7f4e4ee78700 1 Processor -- start 2023-10-06T10:38:46.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.294+0000 7f4e4ee78700 1 -- start start 2023-10-06T10:38:46.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.295+0000 7f4e4ee78700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4e480adb60 0x7f4e480ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:46.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.295+0000 7f4e4ee78700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4e480ce6d0 0x7f4e4815aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:46.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.295+0000 7f4e4ee78700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e48157de0 con 0x7f4e480ca370 2023-10-06T10:38:46.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.295+0000 7f4e4ee78700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e480cbb90 con 0x7f4e480ce6d0 2023-10-06T10:38:46.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.295+0000 7f4e4ee78700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e480ceb10 con 0x7f4e480adb60 2023-10-06T10:38:46.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.295+0000 7f4e4d415700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4e480ce6d0 0x7f4e4815aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:46.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.295+0000 7f4e4cc14700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4e480ca370 0x7f4e48158270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45022/0 (socket says 172.21.15.92:45022) 2023-10-06T10:38:46.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.295+0000 7f4e4cc14700 1 -- 172.21.15.92:0/3290435759 learned_addr learned my addr 172.21.15.92:0/3290435759 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:46.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.295+0000 7f4e3ffff700 1 --2- 172.21.15.92:0/3290435759 >> v2:172.21.15.124:3300/0 conn(0x7f4e480adb60 0x7f4e480ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:46.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.295+0000 7f4e4d415700 1 -- 172.21.15.92:0/3290435759 >> v2:172.21.15.124:3300/0 conn(0x7f4e480adb60 msgr2=0x7f4e480ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:46.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.295+0000 7f4e4d415700 1 --2- 172.21.15.92:0/3290435759 >> v2:172.21.15.124:3300/0 conn(0x7f4e480adb60 0x7f4e480ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:46.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.295+0000 7f4e4d415700 1 -- 172.21.15.92:0/3290435759 >> v1:172.21.15.92:6789/0 conn(0x7f4e480ca370 legacy=0x7f4e48158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:46.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.295+0000 7f4e4d415700 1 -- 172.21.15.92:0/3290435759 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e480cb5d0 con 0x7f4e480ce6d0 2023-10-06T10:38:46.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.295+0000 7f4e3ffff700 1 --2- 172.21.15.92:0/3290435759 >> v2:172.21.15.124:3300/0 conn(0x7f4e480adb60 0x7f4e480ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:46.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.296+0000 7f4e4d415700 1 --2- 172.21.15.92:0/3290435759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e480ce6d0 0x7f4e4815aed0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4e4806c350 tx=0x7f4e400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f786f9fba1a598 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:46.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.296+0000 7f4e3f7fe700 1 -- 172.21.15.92:0/3290435759 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4000b6b0 con 0x7f4e480ce6d0 2023-10-06T10:38:46.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.296+0000 7f4e3f7fe700 1 -- 172.21.15.92:0/3290435759 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4000b810 con 0x7f4e480ce6d0 2023-10-06T10:38:46.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.296+0000 7f4e3f7fe700 1 -- 172.21.15.92:0/3290435759 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4000bbb0 con 0x7f4e480ce6d0 2023-10-06T10:38:46.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.296+0000 7f4e4ee78700 1 -- 172.21.15.92:0/3290435759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e480ce6d0 msgr2=0x7f4e4815aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:46.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.296+0000 7f4e4ee78700 1 --2- 172.21.15.92:0/3290435759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e480ce6d0 0x7f4e4815aed0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4e4806c350 tx=0x7f4e400091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:46.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.296+0000 7f4e4ee78700 1 -- 172.21.15.92:0/3290435759 shutdown_connections 2023-10-06T10:38:46.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.296+0000 7f4e4ee78700 1 --2- 172.21.15.92:0/3290435759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e480ce6d0 0x7f4e4815aed0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:46.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.296+0000 7f4e4ee78700 1 --2- 172.21.15.92:0/3290435759 >> v2:172.21.15.124:3300/0 conn(0x7f4e480adb60 0x7f4e480ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:46.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.296+0000 7f4e4ee78700 1 -- 172.21.15.92:0/3290435759 >> 172.21.15.92:0/3290435759 conn(0x7f4e480707d0 msgr2=0x7f4e48070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:46.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.296+0000 7f4e4ee78700 1 -- 172.21.15.92:0/3290435759 shutdown_connections 2023-10-06T10:38:46.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.296+0000 7f4e4ee78700 1 -- 172.21.15.92:0/3290435759 wait complete. 2023-10-06T10:38:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.297+0000 7f4e4ee78700 1 Processor -- start 2023-10-06T10:38:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.298+0000 7f4e4ee78700 1 -- start start 2023-10-06T10:38:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.298+0000 7f4e4ee78700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e480adb60 0x7f4e4814e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.298+0000 7f4e4ee78700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e480ca370 0x7f4e480b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:46.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.298+0000 7f4e4ee78700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e480ce6d0 0x7f4e48150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:46.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.298+0000 7f4e4ee78700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e480b2ab0 con 0x7f4e480adb60 2023-10-06T10:38:46.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.298+0000 7f4e4ee78700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e480b27d0 con 0x7f4e480ce6d0 2023-10-06T10:38:46.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.298+0000 7f4e4ee78700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e48151a50 con 0x7f4e480ca370 2023-10-06T10:38:46.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.298+0000 7f4e4cc14700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e480adb60 0x7f4e4814e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:46.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.298+0000 7f4e4cc14700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e480adb60 0x7f4e4814e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38468/0 (socket says 172.21.15.92:38468) 2023-10-06T10:38:46.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.298+0000 7f4e4cc14700 1 -- 172.21.15.92:0/1758429221 learned_addr learned my addr 172.21.15.92:0/1758429221 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:46.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.299+0000 7f4e4d415700 1 --2- 172.21.15.92:0/1758429221 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e480ce6d0 0x7f4e48150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:46.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.299+0000 7f4e4cc14700 1 -- 172.21.15.92:0/1758429221 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e480ca370 msgr2=0x7f4e480b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.299+0000 7f4e4cc14700 1 --2- 172.21.15.92:0/1758429221 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e480ca370 0x7f4e480b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.299+0000 7f4e4cc14700 1 -- 172.21.15.92:0/1758429221 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e480ce6d0 msgr2=0x7f4e48150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.299+0000 7f4e4cc14700 1 --2- 172.21.15.92:0/1758429221 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e480ce6d0 0x7f4e48150fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.299+0000 7f4e4cc14700 1 -- 172.21.15.92:0/1758429221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e40009420 con 0x7f4e480adb60 2023-10-06T10:38:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.299+0000 7f4e4cc14700 1 --2- 172.21.15.92:0/1758429221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e480adb60 0x7f4e4814e420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400cc70 tx=0x7f4e44001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.299+0000 7f4e3cff9700 1 -- 172.21.15.92:0/1758429221 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4400ce80 con 0x7f4e480adb60 2023-10-06T10:38:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.299+0000 7f4e3cff9700 1 -- 172.21.15.92:0/1758429221 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4400ab70 con 0x7f4e480adb60 2023-10-06T10:38:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.299+0000 7f4e4ee78700 1 -- 172.21.15.92:0/1758429221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e48151d30 con 0x7f4e480adb60 2023-10-06T10:38:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.299+0000 7f4e3cff9700 1 -- 172.21.15.92:0/1758429221 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e440070f0 con 0x7f4e480adb60 2023-10-06T10:38:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.299+0000 7f4e4ee78700 1 -- 172.21.15.92:0/1758429221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e480b24e0 con 0x7f4e480adb60 2023-10-06T10:38:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.300+0000 7f4e4ee78700 1 -- 172.21.15.92:0/1758429221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e48064a90 con 0x7f4e480adb60 2023-10-06T10:38:46.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.300+0000 7f4e3cff9700 1 -- 172.21.15.92:0/1758429221 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4e44007520 con 0x7f4e480adb60 2023-10-06T10:38:46.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.301+0000 7f4e3cff9700 1 --2- 172.21.15.92:0/1758429221 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4e34066f30 0x7f4e340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:46.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.301+0000 7f4e3ffff700 1 --2- 172.21.15.92:0/1758429221 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4e34066f30 0x7f4e340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:46.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.301+0000 7f4e3cff9700 1 -- 172.21.15.92:0/1758429221 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4e44084de0 con 0x7f4e480adb60 2023-10-06T10:38:46.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.301+0000 7f4e3ffff700 1 --2- 172.21.15.92:0/1758429221 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4e34066f30 0x7f4e340693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4e38006fd0 tx=0x7f4e38008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:46.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.305+0000 7f4e3cff9700 1 -- 172.21.15.92:0/1758429221 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e44051c80 con 0x7f4e480adb60 2023-10-06T10:38:46.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.532+0000 7f4e4ee78700 1 -- 172.21.15.92:0/1758429221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4e480cb8b0 con 0x7f4e480adb60 2023-10-06T10:38:46.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.532+0000 7f4e3cff9700 1 -- 172.21.15.92:0/1758429221 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f4e440552a0 con 0x7f4e480adb60 2023-10-06T10:38:46.533 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T10:38:46.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.534+0000 7f4e4ee78700 1 -- 172.21.15.92:0/1758429221 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4e34066f30 msgr2=0x7f4e340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:46.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.534+0000 7f4e4ee78700 1 --2- 172.21.15.92:0/1758429221 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4e34066f30 0x7f4e340693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4e38006fd0 tx=0x7f4e38008040 comp rx=0 tx=0).stop 2023-10-06T10:38:46.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.534+0000 7f4e4ee78700 1 -- 172.21.15.92:0/1758429221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e480adb60 msgr2=0x7f4e4814e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:46.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.534+0000 7f4e4ee78700 1 --2- 172.21.15.92:0/1758429221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e480adb60 0x7f4e4814e420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400cc70 tx=0x7f4e44001a30 comp rx=0 tx=0).stop 2023-10-06T10:38:46.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.534+0000 7f4e4ee78700 1 -- 172.21.15.92:0/1758429221 shutdown_connections 2023-10-06T10:38:46.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.534+0000 7f4e4ee78700 1 --2- 172.21.15.92:0/1758429221 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4e34066f30 0x7f4e340693e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:46.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.534+0000 7f4e4ee78700 1 --2- 172.21.15.92:0/1758429221 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e480ce6d0 0x7f4e48150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:46.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.534+0000 7f4e4ee78700 1 --2- 172.21.15.92:0/1758429221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e480adb60 0x7f4e4814e420 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:46.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.534+0000 7f4e4ee78700 1 --2- 172.21.15.92:0/1758429221 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e480ca370 0x7f4e480b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:46.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.534+0000 7f4e4ee78700 1 -- 172.21.15.92:0/1758429221 >> 172.21.15.92:0/1758429221 conn(0x7f4e480707d0 msgr2=0x7f4e48158a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:46.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.534+0000 7f4e4ee78700 1 -- 172.21.15.92:0/1758429221 shutdown_connections 2023-10-06T10:38:46.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:46.534+0000 7f4e4ee78700 1 -- 172.21.15.92:0/1758429221 wait complete. 2023-10-06T10:38:46.547 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T10:38:46.547 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T10:38:46.547 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi124.front.sepia.ceph.com 2023-10-06T10:38:46.547 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T10:38:46.547 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T10:38:46.547 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T10:38:46.547 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T10:38:46.547 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T10:38:46.547 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T10:38:46.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:46.548 DEBUG:teuthology.orchestra.run.smithi124:> ip addr 2023-10-06T10:38:46.565 INFO:teuthology.orchestra.run.smithi124.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T10:38:46.565 INFO:teuthology.orchestra.run.smithi124.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T10:38:46.565 INFO:teuthology.orchestra.run.smithi124.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T10:38:46.565 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:46.566 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 ::1/128 scope host 2023-10-06T10:38:46.566 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:46.566 INFO:teuthology.orchestra.run.smithi124.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:38:46.566 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:38:46.566 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f0 2023-10-06T10:38:46.566 INFO:teuthology.orchestra.run.smithi124.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:38:46.566 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:38:46.566 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f1 2023-10-06T10:38:46.566 INFO:teuthology.orchestra.run.smithi124.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:38:46.566 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c4 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:38:46.566 INFO:teuthology.orchestra.run.smithi124.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T10:38:46.567 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c5 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:38:46.567 INFO:teuthology.orchestra.run.smithi124.stdout: inet 172.21.15.124/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T10:38:46.567 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft 42484sec preferred_lft 42484sec 2023-10-06T10:38:46.567 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::ec4:7aff:fe8f:d5c5/64 scope link noprefixroute 2023-10-06T10:38:46.567 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:46.567 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T10:38:46.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:46.567 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2023-10-06T10:38:46.567 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2023-10-06T10:38:46.567 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link add name ceph-brx type bridge 2023-10-06T10:38:46.568 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip addr flush dev ceph-brx 2023-10-06T10:38:46.568 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link set ceph-brx up 2023-10-06T10:38:46.568 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T10:38:46.568 DEBUG:teuthology.orchestra.run.smithi124:> ') 2023-10-06T10:38:46.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:46.732 DEBUG:teuthology.orchestra.run.smithi124:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T10:38:46.773 INFO:teuthology.orchestra.run.smithi124.stdout:1 2023-10-06T10:38:46.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:46.774 DEBUG:teuthology.orchestra.run.smithi124:> route 2023-10-06T10:38:46.859 INFO:teuthology.orchestra.run.smithi124.stdout:Kernel IP routing table 2023-10-06T10:38:46.859 INFO:teuthology.orchestra.run.smithi124.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T10:38:46.859 INFO:teuthology.orchestra.run.smithi124.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T10:38:46.859 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T10:38:46.859 INFO:teuthology.orchestra.run.smithi124.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T10:38:46.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:46.860 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2023-10-06T10:38:46.860 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2023-10-06T10:38:46.860 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T10:38:46.860 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T10:38:46.860 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T10:38:46.860 DEBUG:teuthology.orchestra.run.smithi124:> ') 2023-10-06T10:38:47.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:47.085 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2023-10-06T10:38:47.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:47.100 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list-id 2023-10-06T10:38:47.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:47.154 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2023-10-06T10:38:47.154 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2023-10-06T10:38:47.155 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T10:38:47.155 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-06T10:38:47.155 DEBUG:teuthology.orchestra.run.smithi124:> ') 2023-10-06T10:38:47.243 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-06T10:38:47.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:47.243 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2023-10-06T10:38:47.243 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2023-10-06T10:38:47.243 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-06T10:38:47.244 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T10:38:47.244 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-06T10:38:47.244 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-06T10:38:47.244 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-06T10:38:47.244 DEBUG:teuthology.orchestra.run.smithi124:> ') 2023-10-06T10:38:47.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:47.440 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2023-10-06T10:38:47.440 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2023-10-06T10:38:47.441 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link set brx.0 up 2023-10-06T10:38:47.441 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T10:38:47.441 DEBUG:teuthology.orchestra.run.smithi124:> ') 2023-10-06T10:38:47.532 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T10:38:47.532 DEBUG:teuthology.orchestra.run.smithi124:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T10:38:47.547 INFO:teuthology.orchestra.run.smithi124.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T10:38:47.547 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T10:38:47.547 DEBUG:teuthology.orchestra.run.smithi124:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T10:38:47.601 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T10:38:47.602 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T10:38:47.602 DEBUG:teuthology.orchestra.run.smithi124:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:38:47.704 INFO:teuthology.orchestra.run.smithi124.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:38:47.704 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:38:47.792 INFO:teuthology.orchestra.run.smithi124.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:38:47.793 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:38:47.793 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-06T10:38:47.793 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: trying mount with new device syntax: 1@8f307bfb-c4cb-4f66-aade-46fffe0c6e0c.cephfs=/ 2023-10-06T10:38:47.793 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-06T10:38:47.793 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T10:38:47.793 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T10:38:47.793 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T10:38:47.814 INFO:teuthology.orchestra.run.smithi124.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:38:47.814 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:38:47.814 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-06T10:38:47.814 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: trying mount with new device syntax: 1@8f307bfb-c4cb-4f66-aade-46fffe0c6e0c.cephfs=/ 2023-10-06T10:38:47.815 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-06T10:38:47.815 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T10:38:47.815 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T10:38:47.815 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T10:38:47.815 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.124:6789,172.21.15.124:6790" 2023-10-06T10:38:47.815 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.124:6789,172.21.15.124:6790:/ 2023-10-06T10:38:47.815 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=8f307bfb-c4cb-4f66-aade-46fffe0c6e0c" will pass to kernel 2023-10-06T10:38:47.816 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T10:38:47.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:47.818 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T10:38:47.842 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-10-06T10:38:47.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T10:38:47.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.968+0000 7f66617fd700 1 Processor -- start 2023-10-06T10:38:47.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.968+0000 7f66617fd700 1 -- start start 2023-10-06T10:38:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.969+0000 7f66617fd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f665c0cbc30 0x7f665c0cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.969+0000 7f66617fd700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f665c0b2100 0x7f665c0b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.969+0000 7f66617fd700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665c0cb7a0 con 0x7f665c159440 2023-10-06T10:38:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.969+0000 7f66617fd700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f665c0b0860 con 0x7f665c0cbc30 2023-10-06T10:38:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.969+0000 7f66617fd700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f665c0cb150 con 0x7f665c0b2100 2023-10-06T10:38:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.969+0000 7f665b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f665c0cbc30 0x7f665c0cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.969+0000 7f665bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f665c159440 0x7f665c0b3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45028/0 (socket says 172.21.15.92:45028) 2023-10-06T10:38:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.969+0000 7f665bfff700 1 -- 172.21.15.92:0/859826697 learned_addr learned my addr 172.21.15.92:0/859826697 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.969+0000 7f6660ffc700 1 --2- 172.21.15.92:0/859826697 >> v2:172.21.15.124:3300/0 conn(0x7f665c0b2100 0x7f665c0b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.969+0000 7f665b7fe700 1 -- 172.21.15.92:0/859826697 >> v2:172.21.15.124:3300/0 conn(0x7f665c0b2100 msgr2=0x7f665c0b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.969+0000 7f665b7fe700 1 --2- 172.21.15.92:0/859826697 >> v2:172.21.15.124:3300/0 conn(0x7f665c0b2100 0x7f665c0b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.969+0000 7f665b7fe700 1 -- 172.21.15.92:0/859826697 >> v1:172.21.15.92:6789/0 conn(0x7f665c159440 legacy=0x7f665c0b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.970+0000 7f665b7fe700 1 -- 172.21.15.92:0/859826697 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c0cb430 con 0x7f665c0cbc30 2023-10-06T10:38:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.969+0000 7f665affd700 1 -- 172.21.15.92:0/859826697 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4232885853 0 0) 0x7f665c0cb7a0 con 0x7f665c159440 2023-10-06T10:38:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.970+0000 7f665b7fe700 1 --2- 172.21.15.92:0/859826697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f665c0cbc30 0x7f665c0cc080 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f665c0ca7f0 tx=0x7f66500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1fc1d3ff5e4754 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.970+0000 7f665affd700 1 -- 172.21.15.92:0/859826697 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665000b6b0 con 0x7f665c0cbc30 2023-10-06T10:38:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.970+0000 7f665affd700 1 -- 172.21.15.92:0/859826697 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665000b810 con 0x7f665c0cbc30 2023-10-06T10:38:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.970+0000 7f665affd700 1 -- 172.21.15.92:0/859826697 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665000bbb0 con 0x7f665c0cbc30 2023-10-06T10:38:47.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.970+0000 7f66617fd700 1 -- 172.21.15.92:0/859826697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f665c0cbc30 msgr2=0x7f665c0cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:47.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.970+0000 7f66617fd700 1 --2- 172.21.15.92:0/859826697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f665c0cbc30 0x7f665c0cc080 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f665c0ca7f0 tx=0x7f66500091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:47.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.970+0000 7f66617fd700 1 -- 172.21.15.92:0/859826697 shutdown_connections 2023-10-06T10:38:47.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.970+0000 7f66617fd700 1 --2- 172.21.15.92:0/859826697 >> v2:172.21.15.124:3300/0 conn(0x7f665c0b2100 0x7f665c0b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:47.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.970+0000 7f66617fd700 1 --2- 172.21.15.92:0/859826697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f665c0cbc30 0x7f665c0cc080 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:47.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.970+0000 7f66617fd700 1 -- 172.21.15.92:0/859826697 >> 172.21.15.92:0/859826697 conn(0x7f665c06ece0 msgr2=0x7f665c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.970+0000 7f66617fd700 1 -- 172.21.15.92:0/859826697 shutdown_connections 2023-10-06T10:38:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.970+0000 7f66617fd700 1 -- 172.21.15.92:0/859826697 wait complete. 2023-10-06T10:38:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.972+0000 7f66617fd700 1 Processor -- start 2023-10-06T10:38:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.972+0000 7f66617fd700 1 -- start start 2023-10-06T10:38:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.972+0000 7f66617fd700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f665c0b2100 0x7f665c153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.972+0000 7f66617fd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f665c0cbc30 0x7f665c0c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.972+0000 7f66617fd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f665c159440 0x7f665c0c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.972+0000 7f66617fd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c1541c0 con 0x7f665c0cbc30 2023-10-06T10:38:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.972+0000 7f66617fd700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c0ce440 con 0x7f665c0b2100 2023-10-06T10:38:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.972+0000 7f66617fd700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665c0c9730 con 0x7f665c159440 2023-10-06T10:38:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.973+0000 7f665b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f665c0cbc30 0x7f665c0c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.973+0000 7f665bfff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f665c0b2100 0x7f665c153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.973+0000 7f665b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f665c0cbc30 0x7f665c0c72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38486/0 (socket says 172.21.15.92:38486) 2023-10-06T10:38:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.973+0000 7f665b7fe700 1 -- 172.21.15.92:0/3999737674 learned_addr learned my addr 172.21.15.92:0/3999737674 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.973+0000 7f6660ffc700 1 --2- 172.21.15.92:0/3999737674 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f665c159440 0x7f665c0c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.973+0000 7f665b7fe700 1 -- 172.21.15.92:0/3999737674 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f665c159440 msgr2=0x7f665c0c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.973+0000 7f665b7fe700 1 --2- 172.21.15.92:0/3999737674 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f665c159440 0x7f665c0c7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.973+0000 7f665b7fe700 1 -- 172.21.15.92:0/3999737674 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f665c0b2100 msgr2=0x7f665c153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.973+0000 7f665b7fe700 1 --2- 172.21.15.92:0/3999737674 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f665c0b2100 0x7f665c153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.973+0000 7f665b7fe700 1 -- 172.21.15.92:0/3999737674 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6650009420 con 0x7f665c0cbc30 2023-10-06T10:38:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.973+0000 7f665b7fe700 1 --2- 172.21.15.92:0/3999737674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f665c0cbc30 0x7f665c0c72d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f665c069150 tx=0x7f6650017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.973+0000 7f6643fff700 1 -- 172.21.15.92:0/3999737674 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6650000d10 con 0x7f665c0cbc30 2023-10-06T10:38:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.973+0000 7f6643fff700 1 -- 172.21.15.92:0/3999737674 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665000b980 con 0x7f665c0cbc30 2023-10-06T10:38:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.973+0000 7f6643fff700 1 -- 172.21.15.92:0/3999737674 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6650025420 con 0x7f665c0cbc30 2023-10-06T10:38:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.973+0000 7f66617fd700 1 -- 172.21.15.92:0/3999737674 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665c0c9a10 con 0x7f665c0cbc30 2023-10-06T10:38:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.973+0000 7f66617fd700 1 -- 172.21.15.92:0/3999737674 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f665c0ce0d0 con 0x7f665c0cbc30 2023-10-06T10:38:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.974+0000 7f66617fd700 1 -- 172.21.15.92:0/3999737674 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f665c0ce5b0 con 0x7f665c0cbc30 2023-10-06T10:38:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.975+0000 7f6643fff700 1 -- 172.21.15.92:0/3999737674 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6650023070 con 0x7f665c0cbc30 2023-10-06T10:38:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.975+0000 7f6643fff700 1 --2- 172.21.15.92:0/3999737674 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6648066f30 0x7f66480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.975+0000 7f6643fff700 1 -- 172.21.15.92:0/3999737674 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f6650085e40 con 0x7f665c0cbc30 2023-10-06T10:38:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.975+0000 7f665bfff700 1 --2- 172.21.15.92:0/3999737674 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6648066f30 0x7f66480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.976+0000 7f665bfff700 1 --2- 172.21.15.92:0/3999737674 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6648066f30 0x7f66480693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f665c0cd6f0 tx=0x7f664c007f50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:47.979+0000 7f6643fff700 1 -- 172.21.15.92:0/3999737674 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6650052c60 con 0x7f665c0cbc30 2023-10-06T10:38:48.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:48.207+0000 7f66617fd700 1 -- 172.21.15.92:0/3999737674 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f665c05f230 con 0x7f665c0cbc30 2023-10-06T10:38:48.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:48.207+0000 7f6643fff700 1 -- 172.21.15.92:0/3999737674 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f6650015070 con 0x7f665c0cbc30 2023-10-06T10:38:48.208 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T10:38:48.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:48.209+0000 7f66617fd700 1 -- 172.21.15.92:0/3999737674 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6648066f30 msgr2=0x7f66480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:48.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:48.209+0000 7f66617fd700 1 --2- 172.21.15.92:0/3999737674 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6648066f30 0x7f66480693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f665c0cd6f0 tx=0x7f664c007f50 comp rx=0 tx=0).stop 2023-10-06T10:38:48.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:48.209+0000 7f66617fd700 1 -- 172.21.15.92:0/3999737674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f665c0cbc30 msgr2=0x7f665c0c72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:48.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:48.209+0000 7f66617fd700 1 --2- 172.21.15.92:0/3999737674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f665c0cbc30 0x7f665c0c72d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f665c069150 tx=0x7f6650017000 comp rx=0 tx=0).stop 2023-10-06T10:38:48.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:48.209+0000 7f66617fd700 1 -- 172.21.15.92:0/3999737674 shutdown_connections 2023-10-06T10:38:48.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:48.209+0000 7f66617fd700 1 --2- 172.21.15.92:0/3999737674 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6648066f30 0x7f66480693e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:48.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:48.209+0000 7f66617fd700 1 --2- 172.21.15.92:0/3999737674 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f665c159440 0x7f665c0c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:48.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:48.209+0000 7f66617fd700 1 --2- 172.21.15.92:0/3999737674 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f665c0b2100 0x7f665c153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:48.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:48.209+0000 7f66617fd700 1 --2- 172.21.15.92:0/3999737674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f665c0cbc30 0x7f665c0c72d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:48.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:48.209+0000 7f66617fd700 1 -- 172.21.15.92:0/3999737674 >> 172.21.15.92:0/3999737674 conn(0x7f665c06ece0 msgr2=0x7f665c158570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:48.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:48.210+0000 7f66617fd700 1 -- 172.21.15.92:0/3999737674 shutdown_connections 2023-10-06T10:38:48.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:48.210+0000 7f66617fd700 1 -- 172.21.15.92:0/3999737674 wait complete. 2023-10-06T10:38:48.222 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T10:38:48.222 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T10:38:48.222 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi124.front.sepia.ceph.com 2023-10-06T10:38:48.222 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-06T10:38:48.222 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-06T10:38:48.223 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T10:38:48.223 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T10:38:48.223 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T10:38:48.223 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-06T10:38:48.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:48.223 DEBUG:teuthology.orchestra.run.smithi124:> ip addr 2023-10-06T10:38:48.239 INFO:teuthology.orchestra.run.smithi124.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T10:38:48.239 INFO:teuthology.orchestra.run.smithi124.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T10:38:48.239 INFO:teuthology.orchestra.run.smithi124.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T10:38:48.239 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:48.240 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 ::1/128 scope host 2023-10-06T10:38:48.240 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:48.240 INFO:teuthology.orchestra.run.smithi124.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:38:48.240 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:38:48.240 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f0 2023-10-06T10:38:48.240 INFO:teuthology.orchestra.run.smithi124.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:38:48.240 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:38:48.240 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f1 2023-10-06T10:38:48.240 INFO:teuthology.orchestra.run.smithi124.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:38:48.240 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c4 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:38:48.240 INFO:teuthology.orchestra.run.smithi124.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T10:38:48.240 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c5 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:38:48.240 INFO:teuthology.orchestra.run.smithi124.stdout: inet 172.21.15.124/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T10:38:48.241 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft 42482sec preferred_lft 42482sec 2023-10-06T10:38:48.241 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::ec4:7aff:fe8f:d5c5/64 scope link noprefixroute 2023-10-06T10:38:48.241 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:48.241 INFO:teuthology.orchestra.run.smithi124.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T10:38:48.241 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 16:5c:38:fd:39:7b brd ff:ff:ff:ff:ff:ff 2023-10-06T10:38:48.241 INFO:teuthology.orchestra.run.smithi124.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T10:38:48.241 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:48.241 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::1cfd:a9ff:feec:8bb5/64 scope link 2023-10-06T10:38:48.241 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:48.241 INFO:teuthology.orchestra.run.smithi124.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T10:38:48.242 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 16:5c:38:fd:39:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T10:38:48.242 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::145c:38ff:fefd:397b/64 scope link tentative 2023-10-06T10:38:48.242 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:48.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:48.242 DEBUG:teuthology.orchestra.run.smithi124:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T10:38:48.304 INFO:teuthology.orchestra.run.smithi124.stdout:1 2023-10-06T10:38:48.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:48.306 DEBUG:teuthology.orchestra.run.smithi124:> route 2023-10-06T10:38:48.363 INFO:teuthology.orchestra.run.smithi124.stdout:Kernel IP routing table 2023-10-06T10:38:48.363 INFO:teuthology.orchestra.run.smithi124.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T10:38:48.363 INFO:teuthology.orchestra.run.smithi124.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T10:38:48.363 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T10:38:48.363 INFO:teuthology.orchestra.run.smithi124.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T10:38:48.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:48.364 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2023-10-06T10:38:48.364 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2023-10-06T10:38:48.364 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T10:38:48.364 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T10:38:48.364 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T10:38:48.364 DEBUG:teuthology.orchestra.run.smithi124:> ') 2023-10-06T10:38:48.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:48.464 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2023-10-06T10:38:48.479 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T10:38:48.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:48.479 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list-id 2023-10-06T10:38:48.533 INFO:teuthology.orchestra.run.smithi124.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-06T10:38:48.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:48.533 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2023-10-06T10:38:48.534 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2023-10-06T10:38:48.534 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T10:38:48.534 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-06T10:38:48.534 DEBUG:teuthology.orchestra.run.smithi124:> ') 2023-10-06T10:38:48.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:48.621 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T10:38:48.654 INFO:teuthology.orchestra.run.smithi124.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T10:38:48.655 INFO:teuthology.orchestra.run.smithi124.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T10:38:48.655 INFO:teuthology.orchestra.run.smithi124.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T10:38:48.655 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:48.655 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 ::1/128 scope host 2023-10-06T10:38:48.655 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:48.655 INFO:teuthology.orchestra.run.smithi124.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T10:38:48.655 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether f6:ea:7a:1d:de:09 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T10:38:48.655 INFO:teuthology.orchestra.run.smithi124.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T10:38:48.655 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:48.655 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::f4ea:7aff:fe1d:de09/64 scope link tentative 2023-10-06T10:38:48.655 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:48.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:48.669 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T10:38:48.745 INFO:teuthology.orchestra.run.smithi124.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T10:38:48.745 INFO:teuthology.orchestra.run.smithi124.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T10:38:48.745 INFO:teuthology.orchestra.run.smithi124.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T10:38:48.746 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:48.746 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 ::1/128 scope host 2023-10-06T10:38:48.746 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:48.746 INFO:teuthology.orchestra.run.smithi124.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T10:38:48.746 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether f6:ea:7a:1d:de:09 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T10:38:48.746 INFO:teuthology.orchestra.run.smithi124.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T10:38:48.746 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:48.746 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::f4ea:7aff:fe1d:de09/64 scope link tentative 2023-10-06T10:38:48.746 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:38:48.757 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-06T10:38:48.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:48.758 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2023-10-06T10:38:48.758 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2023-10-06T10:38:48.758 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-06T10:38:48.758 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T10:38:48.758 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-06T10:38:48.758 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-06T10:38:48.758 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-06T10:38:48.758 DEBUG:teuthology.orchestra.run.smithi124:> ') 2023-10-06T10:38:48.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:48.981 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2023-10-06T10:38:48.981 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2023-10-06T10:38:48.981 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link set brx.1 up 2023-10-06T10:38:48.982 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T10:38:48.982 DEBUG:teuthology.orchestra.run.smithi124:> ') 2023-10-06T10:38:49.060 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T10:38:49.061 DEBUG:teuthology.orchestra.run.smithi124:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-06T10:38:49.075 INFO:teuthology.orchestra.run.smithi124.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-06T10:38:49.076 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T10:38:49.076 DEBUG:teuthology.orchestra.run.smithi124:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-06T10:38:49.130 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-10-06T10:38:49.130 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T10:38:49.130 DEBUG:teuthology.orchestra.run.smithi124:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:38:49.209 INFO:teuthology.orchestra.run.smithi124.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:38:49.209 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:38:49.227 INFO:teuthology.orchestra.run.smithi124.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:38:49.227 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:38:49.227 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-06T10:38:49.227 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: trying mount with new device syntax: 3@8f307bfb-c4cb-4f66-aade-46fffe0c6e0c.cephfs=/ 2023-10-06T10:38:49.228 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.92:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-06T10:38:49.228 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T10:38:49.228 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T10:38:49.228 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T10:38:49.249 INFO:teuthology.orchestra.run.smithi124.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:38:49.249 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:38:49.249 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-06T10:38:49.249 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: trying mount with new device syntax: 3@8f307bfb-c4cb-4f66-aade-46fffe0c6e0c.cephfs=/ 2023-10-06T10:38:49.249 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.92:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-06T10:38:49.249 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T10:38:49.249 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T10:38:49.249 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T10:38:49.250 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.124:6789,172.21.15.124:6790" 2023-10-06T10:38:49.250 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.124:6789,172.21.15.124:6790:/ 2023-10-06T10:38:49.250 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,key=3,mds_namespace=cephfs,fsid=8f307bfb-c4cb-4f66-aade-46fffe0c6e0c" will pass to kernel 2023-10-06T10:38:49.251 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T10:38:49.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:49.251 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-06T10:38:49.316 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-06T10:38:49.333 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:38:49.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.450+0000 7f1986fd0700 1 Processor -- start 2023-10-06T10:38:49.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.450+0000 7f1986fd0700 1 -- start start 2023-10-06T10:38:49.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.451+0000 7f1986fd0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19800cbc30 0x7f19800cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:49.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.451+0000 7f1986fd0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f19800b2100 0x7f19800b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:49.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.451+0000 7f1986fd0700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19800cb7a0 con 0x7f1980159440 2023-10-06T10:38:49.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.451+0000 7f1986fd0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19800b0860 con 0x7f19800b2100 2023-10-06T10:38:49.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.451+0000 7f1986fd0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19800cb150 con 0x7f19800cbc30 2023-10-06T10:38:49.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.451+0000 7f19867cf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f19800b2100 0x7f19800b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:49.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.451+0000 7f19867cf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f19800b2100 0x7f19800b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38492/0 (socket says 172.21.15.92:38492) 2023-10-06T10:38:49.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.451+0000 7f19867cf700 1 -- 172.21.15.92:0/247408822 learned_addr learned my addr 172.21.15.92:0/247408822 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:49.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.451+0000 7f19857cd700 1 --2- 172.21.15.92:0/247408822 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19800cbc30 0x7f19800cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:49.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.451+0000 7f19867cf700 1 -- 172.21.15.92:0/247408822 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19800cbc30 msgr2=0x7f19800cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:49.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.451+0000 7f19867cf700 1 --2- 172.21.15.92:0/247408822 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19800cbc30 0x7f19800cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:49.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.451+0000 7f19867cf700 1 -- 172.21.15.92:0/247408822 >> v1:172.21.15.124:6789/0 conn(0x7f1980159440 legacy=0x7f19800b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:49.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.451+0000 7f19867cf700 1 -- 172.21.15.92:0/247408822 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19800cb430 con 0x7f19800b2100 2023-10-06T10:38:49.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.451+0000 7f19857cd700 1 --2- 172.21.15.92:0/247408822 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19800cbc30 0x7f19800cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:49.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.452+0000 7f19867cf700 1 --2- 172.21.15.92:0/247408822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19800b2100 0x7f19800b2580 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1978009f20 tx=0x7f1978009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32c92c4fa00f71b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:49.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.452+0000 7f1984fcc700 1 -- 172.21.15.92:0/247408822 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197800e070 con 0x7f19800b2100 2023-10-06T10:38:49.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.452+0000 7f1984fcc700 1 -- 172.21.15.92:0/247408822 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197800ba80 con 0x7f19800b2100 2023-10-06T10:38:49.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.452+0000 7f1984fcc700 1 -- 172.21.15.92:0/247408822 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19780092e0 con 0x7f19800b2100 2023-10-06T10:38:49.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.452+0000 7f1986fd0700 1 -- 172.21.15.92:0/247408822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19800b2100 msgr2=0x7f19800b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:49.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.452+0000 7f1986fd0700 1 --2- 172.21.15.92:0/247408822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19800b2100 0x7f19800b2580 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1978009f20 tx=0x7f1978009840 comp rx=0 tx=0).stop 2023-10-06T10:38:49.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.452+0000 7f1986fd0700 1 -- 172.21.15.92:0/247408822 shutdown_connections 2023-10-06T10:38:49.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.452+0000 7f1986fd0700 1 --2- 172.21.15.92:0/247408822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19800b2100 0x7f19800b2580 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:49.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.452+0000 7f1986fd0700 1 --2- 172.21.15.92:0/247408822 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19800cbc30 0x7f19800cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:49.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.452+0000 7f1986fd0700 1 -- 172.21.15.92:0/247408822 >> 172.21.15.92:0/247408822 conn(0x7f198006ece0 msgr2=0x7f198006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:49.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.452+0000 7f1986fd0700 1 -- 172.21.15.92:0/247408822 shutdown_connections 2023-10-06T10:38:49.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.452+0000 7f1986fd0700 1 -- 172.21.15.92:0/247408822 wait complete. 2023-10-06T10:38:49.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.453+0000 7f1986fd0700 1 Processor -- start 2023-10-06T10:38:49.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.454+0000 7f1986fd0700 1 -- start start 2023-10-06T10:38:49.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.454+0000 7f1986fd0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19800b2100 0x7f19801494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:49.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.454+0000 7f1986fd0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19800cbc30 0x7f19800cea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:49.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.454+0000 7f1986fd0700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1980159440 0x7f19800c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:49.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.454+0000 7f1986fd0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19800cfb00 con 0x7f19800cbc30 2023-10-06T10:38:49.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.454+0000 7f1986fd0700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19800c7f20 con 0x7f1980159440 2023-10-06T10:38:49.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.454+0000 7f1986fd0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19800cf540 con 0x7f19800b2100 2023-10-06T10:38:49.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.454+0000 7f19857cd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19800cbc30 0x7f19800cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:49.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.454+0000 7f19857cd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19800cbc30 0x7f19800cea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38508/0 (socket says 172.21.15.92:38508) 2023-10-06T10:38:49.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.454+0000 7f19857cd700 1 -- 172.21.15.92:0/2421441780 learned_addr learned my addr 172.21.15.92:0/2421441780 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:49.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.454+0000 7f1985fce700 1 --2- 172.21.15.92:0/2421441780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19800b2100 0x7f19801494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:49.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.454+0000 7f19867cf700 1 --2- 172.21.15.92:0/2421441780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1980159440 0x7f19800c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:49.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.455+0000 7f19857cd700 1 -- 172.21.15.92:0/2421441780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19800b2100 msgr2=0x7f19801494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:49.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.455+0000 7f19857cd700 1 --2- 172.21.15.92:0/2421441780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19800b2100 0x7f19801494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:49.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.455+0000 7f19857cd700 1 -- 172.21.15.92:0/2421441780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1980159440 msgr2=0x7f19800c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:49.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.455+0000 7f19857cd700 1 --2- 172.21.15.92:0/2421441780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1980159440 0x7f19800c9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:49.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.455+0000 7f19857cd700 1 -- 172.21.15.92:0/2421441780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1978009ae0 con 0x7f19800cbc30 2023-10-06T10:38:49.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.455+0000 7f19857cd700 1 --2- 172.21.15.92:0/2421441780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19800cbc30 0x7f19800cea40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f197000c830 tx=0x7f1970008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:49.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.455+0000 7f1975ffb700 1 -- 172.21.15.92:0/2421441780 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1970011600 con 0x7f19800cbc30 2023-10-06T10:38:49.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.455+0000 7f1975ffb700 1 -- 172.21.15.92:0/2421441780 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1970011760 con 0x7f19800cbc30 2023-10-06T10:38:49.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.455+0000 7f1986fd0700 1 -- 172.21.15.92:0/2421441780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19800cf820 con 0x7f19800cbc30 2023-10-06T10:38:49.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.455+0000 7f1975ffb700 1 -- 172.21.15.92:0/2421441780 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1970011b20 con 0x7f19800cbc30 2023-10-06T10:38:49.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.455+0000 7f1986fd0700 1 -- 172.21.15.92:0/2421441780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19800c7500 con 0x7f19800cbc30 2023-10-06T10:38:49.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.456+0000 7f1986fd0700 1 -- 172.21.15.92:0/2421441780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19800c8090 con 0x7f19800cbc30 2023-10-06T10:38:49.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.456+0000 7f1975ffb700 1 -- 172.21.15.92:0/2421441780 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1970018360 con 0x7f19800cbc30 2023-10-06T10:38:49.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.457+0000 7f1975ffb700 1 --2- 172.21.15.92:0/2421441780 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1964066f30 0x7f19640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:49.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.457+0000 7f1975ffb700 1 -- 172.21.15.92:0/2421441780 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f1970084910 con 0x7f19800cbc30 2023-10-06T10:38:49.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.457+0000 7f1985fce700 1 --2- 172.21.15.92:0/2421441780 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1964066f30 0x7f19640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:49.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.457+0000 7f1985fce700 1 --2- 172.21.15.92:0/2421441780 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1964066f30 0x7f19640693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f197c0039e0 tx=0x7f197c007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:49.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.461+0000 7f1975ffb700 1 -- 172.21.15.92:0/2421441780 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1970051730 con 0x7f19800cbc30 2023-10-06T10:38:49.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.648+0000 7f1986fd0700 1 -- 172.21.15.92:0/2421441780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f198005f230 con 0x7f19800cbc30 2023-10-06T10:38:49.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.649+0000 7f1975ffb700 1 -- 172.21.15.92:0/2421441780 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18707 (secure 0 0 0) 0x7f1970054d50 con 0x7f19800cbc30 2023-10-06T10:38:49.651 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:49.651 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:21.760233+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:38:14.368713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T10:38:15.692344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T10:38:19.382553+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:38:49.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.651+0000 7f1986fd0700 1 -- 172.21.15.92:0/2421441780 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1964066f30 msgr2=0x7f19640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:49.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.651+0000 7f1986fd0700 1 --2- 172.21.15.92:0/2421441780 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1964066f30 0x7f19640693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f197c0039e0 tx=0x7f197c007ec0 comp rx=0 tx=0).stop 2023-10-06T10:38:49.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.651+0000 7f1986fd0700 1 -- 172.21.15.92:0/2421441780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19800cbc30 msgr2=0x7f19800cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:49.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.651+0000 7f1986fd0700 1 --2- 172.21.15.92:0/2421441780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19800cbc30 0x7f19800cea40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f197000c830 tx=0x7f1970008040 comp rx=0 tx=0).stop 2023-10-06T10:38:49.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.651+0000 7f1986fd0700 1 -- 172.21.15.92:0/2421441780 shutdown_connections 2023-10-06T10:38:49.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.651+0000 7f1986fd0700 1 --2- 172.21.15.92:0/2421441780 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1964066f30 0x7f19640693e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:49.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.651+0000 7f1986fd0700 1 --2- 172.21.15.92:0/2421441780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1980159440 0x7f19800c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:49.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.651+0000 7f1986fd0700 1 --2- 172.21.15.92:0/2421441780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19800b2100 0x7f19801494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:49.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.651+0000 7f1986fd0700 1 --2- 172.21.15.92:0/2421441780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19800cbc30 0x7f19800cea40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:49.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.651+0000 7f1986fd0700 1 -- 172.21.15.92:0/2421441780 >> 172.21.15.92:0/2421441780 conn(0x7f198006ece0 msgr2=0x7f19800c6130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:49.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.651+0000 7f1986fd0700 1 -- 172.21.15.92:0/2421441780 shutdown_connections 2023-10-06T10:38:49.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.651+0000 7f1986fd0700 1 -- 172.21.15.92:0/2421441780 wait complete. 2023-10-06T10:38:49.665 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T10:38:09.680559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T10:38:10.120786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T10:38:14.368713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T10:38:15.692344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T10:38:19.382553+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T10:38:49.665 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T10:38:49.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.780+0000 7f8082a39700 1 Processor -- start 2023-10-06T10:38:49.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.780+0000 7f8082a39700 1 -- start start 2023-10-06T10:38:49.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.781+0000 7f8082a39700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f807c1598a0 0x7f807c0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:49.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.781+0000 7f8082a39700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f807c0cbc70 0x7f807c0ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:49.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.781+0000 7f8082a39700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f807c0cf5d0 con 0x7f807c1594d0 2023-10-06T10:38:49.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.781+0000 7f8082a39700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f807c0cf8b0 con 0x7f807c1598a0 2023-10-06T10:38:49.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.781+0000 7f8082a39700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f807c0cfb90 con 0x7f807c0cbc70 2023-10-06T10:38:49.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.781+0000 7f807b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f807c1598a0 0x7f807c0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:49.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.781+0000 7f807b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f807c1598a0 0x7f807c0ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38522/0 (socket says 172.21.15.92:38522) 2023-10-06T10:38:49.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.781+0000 7f807b7fe700 1 -- 172.21.15.92:0/614279734 learned_addr learned my addr 172.21.15.92:0/614279734 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:49.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.781+0000 7f8080fd6700 1 --2- 172.21.15.92:0/614279734 >> v2:172.21.15.124:3300/0 conn(0x7f807c0cbc70 0x7f807c0ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:49.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.781+0000 7f807b7fe700 1 -- 172.21.15.92:0/614279734 >> v2:172.21.15.124:3300/0 conn(0x7f807c0cbc70 msgr2=0x7f807c0ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:49.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.781+0000 7f807b7fe700 1 --2- 172.21.15.92:0/614279734 >> v2:172.21.15.124:3300/0 conn(0x7f807c0cbc70 0x7f807c0ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:49.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.781+0000 7f807b7fe700 1 -- 172.21.15.92:0/614279734 >> v1:172.21.15.124:6789/0 conn(0x7f807c1594d0 legacy=0x7f807c0cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:49.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.781+0000 7f807b7fe700 1 -- 172.21.15.92:0/614279734 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c0cfe70 con 0x7f807c1598a0 2023-10-06T10:38:49.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.782+0000 7f807b7fe700 1 --2- 172.21.15.92:0/614279734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f807c1598a0 0x7f807c0ce7e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f807c06f950 tx=0x7f8070009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff841685f3b8ecf7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:49.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.782+0000 7f807affd700 1 -- 172.21.15.92:0/614279734 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807000b820 con 0x7f807c1598a0 2023-10-06T10:38:49.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.782+0000 7f807affd700 1 -- 172.21.15.92:0/614279734 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807000b980 con 0x7f807c1598a0 2023-10-06T10:38:49.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.782+0000 7f807affd700 1 -- 172.21.15.92:0/614279734 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807000bd20 con 0x7f807c1598a0 2023-10-06T10:38:49.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.782+0000 7f8082a39700 1 -- 172.21.15.92:0/614279734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f807c1598a0 msgr2=0x7f807c0ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:49.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.782+0000 7f8082a39700 1 --2- 172.21.15.92:0/614279734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f807c1598a0 0x7f807c0ce7e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f807c06f950 tx=0x7f8070009250 comp rx=0 tx=0).stop 2023-10-06T10:38:49.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.782+0000 7f8082a39700 1 -- 172.21.15.92:0/614279734 shutdown_connections 2023-10-06T10:38:49.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.782+0000 7f8082a39700 1 --2- 172.21.15.92:0/614279734 >> v2:172.21.15.124:3300/0 conn(0x7f807c0cbc70 0x7f807c0ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:49.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.782+0000 7f8082a39700 1 --2- 172.21.15.92:0/614279734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f807c1598a0 0x7f807c0ce7e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:49.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.782+0000 7f8082a39700 1 -- 172.21.15.92:0/614279734 >> 172.21.15.92:0/614279734 conn(0x7f807c089e50 msgr2=0x7f807c099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:49.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.782+0000 7f8082a39700 1 -- 172.21.15.92:0/614279734 shutdown_connections 2023-10-06T10:38:49.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.782+0000 7f8082a39700 1 -- 172.21.15.92:0/614279734 wait complete. 2023-10-06T10:38:49.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.783+0000 7f8082a39700 1 Processor -- start 2023-10-06T10:38:49.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.784+0000 7f8082a39700 1 -- start start 2023-10-06T10:38:49.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.784+0000 7f8082a39700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f807c0cbc70 0x7f807c167b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:49.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.784+0000 7f8082a39700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f807c1594d0 0x7f807c169630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:49.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.784+0000 7f8082a39700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f807c1598a0 0x7f807c14e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:49.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.784+0000 7f8082a39700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c14e4c0 con 0x7f807c0cbc70 2023-10-06T10:38:49.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.784+0000 7f8082a39700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c14db90 con 0x7f807c1598a0 2023-10-06T10:38:49.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.784+0000 7f8082a39700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807c14de70 con 0x7f807c1594d0 2023-10-06T10:38:49.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.784+0000 7f807bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f807c0cbc70 0x7f807c167b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:49.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.784+0000 7f807bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f807c0cbc70 0x7f807c167b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38538/0 (socket says 172.21.15.92:38538) 2023-10-06T10:38:49.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.784+0000 7f807bfff700 1 -- 172.21.15.92:0/1703636098 learned_addr learned my addr 172.21.15.92:0/1703636098 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:49.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.785+0000 7f807b7fe700 1 --2- 172.21.15.92:0/1703636098 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f807c1594d0 0x7f807c169630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:49.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.785+0000 7f8080fd6700 1 --2- 172.21.15.92:0/1703636098 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f807c1598a0 0x7f807c14e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:49.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.785+0000 7f807bfff700 1 -- 172.21.15.92:0/1703636098 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f807c1594d0 msgr2=0x7f807c169630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:49.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.785+0000 7f807bfff700 1 --2- 172.21.15.92:0/1703636098 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f807c1594d0 0x7f807c169630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:49.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.785+0000 7f807bfff700 1 -- 172.21.15.92:0/1703636098 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f807c1598a0 msgr2=0x7f807c14e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:49.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.785+0000 7f807bfff700 1 --2- 172.21.15.92:0/1703636098 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f807c1598a0 0x7f807c14e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:49.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.785+0000 7f807bfff700 1 -- 172.21.15.92:0/1703636098 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80700094f0 con 0x7f807c0cbc70 2023-10-06T10:38:49.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.785+0000 7f807b7fe700 1 --2- 172.21.15.92:0/1703636098 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f807c1594d0 0x7f807c169630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:49.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.785+0000 7f8080fd6700 1 --2- 172.21.15.92:0/1703636098 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f807c1598a0 0x7f807c14e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:49.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.785+0000 7f807bfff700 1 --2- 172.21.15.92:0/1703636098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f807c0cbc70 0x7f807c167b80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f806c00b600 tx=0x7f806c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:49.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.785+0000 7f8063fff700 1 -- 172.21.15.92:0/1703636098 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c00f510 con 0x7f807c0cbc70 2023-10-06T10:38:49.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.785+0000 7f8063fff700 1 -- 172.21.15.92:0/1703636098 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f806c00f670 con 0x7f807c0cbc70 2023-10-06T10:38:49.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.785+0000 7f8082a39700 1 -- 172.21.15.92:0/1703636098 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807c14e150 con 0x7f807c0cbc70 2023-10-06T10:38:49.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.785+0000 7f8063fff700 1 -- 172.21.15.92:0/1703636098 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c00fa30 con 0x7f807c0cbc70 2023-10-06T10:38:49.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.785+0000 7f8082a39700 1 -- 172.21.15.92:0/1703636098 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f807c168280 con 0x7f807c0cbc70 2023-10-06T10:38:49.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.786+0000 7f8082a39700 1 -- 172.21.15.92:0/1703636098 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f807c14e7a0 con 0x7f807c0cbc70 2023-10-06T10:38:49.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.786+0000 7f8063fff700 1 -- 172.21.15.92:0/1703636098 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f806c01a400 con 0x7f807c0cbc70 2023-10-06T10:38:49.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.787+0000 7f8063fff700 1 --2- 172.21.15.92:0/1703636098 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8068066f30 0x7f80680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:49.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.787+0000 7f8063fff700 1 -- 172.21.15.92:0/1703636098 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f806c0844b0 con 0x7f807c0cbc70 2023-10-06T10:38:49.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.787+0000 7f807b7fe700 1 --2- 172.21.15.92:0/1703636098 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8068066f30 0x7f80680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:49.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.787+0000 7f807b7fe700 1 --2- 172.21.15.92:0/1703636098 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8068066f30 0x7f80680693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f807c08a830 tx=0x7f8070016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:49.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.791+0000 7f8063fff700 1 -- 172.21.15.92:0/1703636098 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f806c051350 con 0x7f807c0cbc70 2023-10-06T10:38:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.975+0000 7f8082a39700 1 -- 172.21.15.92:0/1703636098 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f807c064a90 con 0x7f807c0cbc70 2023-10-06T10:38:49.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.975+0000 7f8063fff700 1 -- 172.21.15.92:0/1703636098 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f806c054970 con 0x7f807c0cbc70 2023-10-06T10:38:49.977 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T10:38:49.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.977+0000 7f8082a39700 1 -- 172.21.15.92:0/1703636098 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8068066f30 msgr2=0x7f80680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:49.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.977+0000 7f8082a39700 1 --2- 172.21.15.92:0/1703636098 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8068066f30 0x7f80680693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f807c08a830 tx=0x7f8070016000 comp rx=0 tx=0).stop 2023-10-06T10:38:49.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.977+0000 7f8082a39700 1 -- 172.21.15.92:0/1703636098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f807c0cbc70 msgr2=0x7f807c167b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:49.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.977+0000 7f8082a39700 1 --2- 172.21.15.92:0/1703636098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f807c0cbc70 0x7f807c167b80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f806c00b600 tx=0x7f806c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:49.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.977+0000 7f8082a39700 1 -- 172.21.15.92:0/1703636098 shutdown_connections 2023-10-06T10:38:49.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.977+0000 7f8082a39700 1 --2- 172.21.15.92:0/1703636098 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8068066f30 0x7f80680693e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:49.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.977+0000 7f8082a39700 1 --2- 172.21.15.92:0/1703636098 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f807c1598a0 0x7f807c14e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:49.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.977+0000 7f8082a39700 1 --2- 172.21.15.92:0/1703636098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f807c0cbc70 0x7f807c167b80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:49.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.977+0000 7f8082a39700 1 --2- 172.21.15.92:0/1703636098 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f807c1594d0 0x7f807c169630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:49.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.977+0000 7f8082a39700 1 -- 172.21.15.92:0/1703636098 >> 172.21.15.92:0/1703636098 conn(0x7f807c089e50 msgr2=0x7f807c0c6260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:49.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.977+0000 7f8082a39700 1 -- 172.21.15.92:0/1703636098 shutdown_connections 2023-10-06T10:38:49.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:49.977+0000 7f8082a39700 1 -- 172.21.15.92:0/1703636098 wait complete. 2023-10-06T10:38:49.991 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T10:38:50.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.106+0000 7fe998640700 1 Processor -- start 2023-10-06T10:38:50.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.106+0000 7fe998640700 1 -- start start 2023-10-06T10:38:50.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.107+0000 7fe998640700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe9901594d0 0x7fe9900cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:50.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.107+0000 7fe998640700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9900cbaa0 0x7fe9900ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:50.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.107+0000 7fe998640700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9900cbeb0 con 0x7fe9901598a0 2023-10-06T10:38:50.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.107+0000 7fe998640700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9900cf5d0 con 0x7fe9901594d0 2023-10-06T10:38:50.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.107+0000 7fe998640700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9900cf8b0 con 0x7fe9900cbaa0 2023-10-06T10:38:50.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.107+0000 7fe9963dc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe9901594d0 0x7fe9900cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:50.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.107+0000 7fe995bdb700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe9901598a0 0x7fe9900ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45044/0 (socket says 172.21.15.92:45044) 2023-10-06T10:38:50.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.107+0000 7fe995bdb700 1 -- 172.21.15.92:0/4043605554 learned_addr learned my addr 172.21.15.92:0/4043605554 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:50.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.107+0000 7fe996bdd700 1 --2- 172.21.15.92:0/4043605554 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9900cbaa0 0x7fe9900ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:50.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.107+0000 7fe9963dc700 1 -- 172.21.15.92:0/4043605554 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9900cbaa0 msgr2=0x7fe9900ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:50.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.108+0000 7fe9963dc700 1 --2- 172.21.15.92:0/4043605554 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9900cbaa0 0x7fe9900ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.108+0000 7fe9963dc700 1 -- 172.21.15.92:0/4043605554 >> v1:172.21.15.92:6789/0 conn(0x7fe9901598a0 legacy=0x7fe9900ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:50.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.108+0000 7fe9963dc700 1 -- 172.21.15.92:0/4043605554 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9900cfb90 con 0x7fe9901594d0 2023-10-06T10:38:50.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.108+0000 7fe9953da700 1 -- 172.21.15.92:0/4043605554 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3239054428 0 0) 0x7fe9900cbeb0 con 0x7fe9901598a0 2023-10-06T10:38:50.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.108+0000 7fe996bdd700 1 --2- 172.21.15.92:0/4043605554 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9900cbaa0 0x7fe9900ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:50.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.108+0000 7fe9963dc700 1 --2- 172.21.15.92:0/4043605554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9901594d0 0x7fe9900cb560 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009e60 tx=0x7fe98c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e901683f144bb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:50.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.108+0000 7fe9953da700 1 -- 172.21.15.92:0/4043605554 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c00e070 con 0x7fe9901594d0 2023-10-06T10:38:50.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.108+0000 7fe9953da700 1 -- 172.21.15.92:0/4043605554 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98c00bc30 con 0x7fe9901594d0 2023-10-06T10:38:50.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.108+0000 7fe9953da700 1 -- 172.21.15.92:0/4043605554 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c012430 con 0x7fe9901594d0 2023-10-06T10:38:50.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.108+0000 7fe998640700 1 -- 172.21.15.92:0/4043605554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9901594d0 msgr2=0x7fe9900cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:50.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.108+0000 7fe998640700 1 --2- 172.21.15.92:0/4043605554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9901594d0 0x7fe9900cb560 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009e60 tx=0x7fe98c009780 comp rx=0 tx=0).stop 2023-10-06T10:38:50.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.108+0000 7fe998640700 1 -- 172.21.15.92:0/4043605554 shutdown_connections 2023-10-06T10:38:50.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.108+0000 7fe998640700 1 --2- 172.21.15.92:0/4043605554 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9900cbaa0 0x7fe9900ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.108+0000 7fe998640700 1 --2- 172.21.15.92:0/4043605554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9901594d0 0x7fe9900cb560 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.108+0000 7fe998640700 1 -- 172.21.15.92:0/4043605554 >> 172.21.15.92:0/4043605554 conn(0x7fe990089e50 msgr2=0x7fe990099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:50.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.108+0000 7fe998640700 1 -- 172.21.15.92:0/4043605554 shutdown_connections 2023-10-06T10:38:50.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.109+0000 7fe998640700 1 -- 172.21.15.92:0/4043605554 wait complete. 2023-10-06T10:38:50.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.110+0000 7fe998640700 1 Processor -- start 2023-10-06T10:38:50.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.110+0000 7fe998640700 1 -- start start 2023-10-06T10:38:50.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.110+0000 7fe998640700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9900cbaa0 0x7fe9900c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:50.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.110+0000 7fe998640700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9901594d0 0x7fe990155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:50.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.110+0000 7fe998640700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9901598a0 0x7fe9900b5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:50.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.110+0000 7fe998640700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9901563b0 con 0x7fe9901594d0 2023-10-06T10:38:50.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.110+0000 7fe998640700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe990155a60 con 0x7fe9901598a0 2023-10-06T10:38:50.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.110+0000 7fe998640700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe990155d40 con 0x7fe9900cbaa0 2023-10-06T10:38:50.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.110+0000 7fe995bdb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9901594d0 0x7fe990155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:50.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.110+0000 7fe995bdb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9901594d0 0x7fe990155210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38560/0 (socket says 172.21.15.92:38560) 2023-10-06T10:38:50.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.110+0000 7fe995bdb700 1 -- 172.21.15.92:0/939254678 learned_addr learned my addr 172.21.15.92:0/939254678 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:50.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.110+0000 7fe996bdd700 1 --2- 172.21.15.92:0/939254678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9901598a0 0x7fe9900b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:50.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.110+0000 7fe9963dc700 1 --2- 172.21.15.92:0/939254678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9900cbaa0 0x7fe9900c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:50.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.111+0000 7fe995bdb700 1 -- 172.21.15.92:0/939254678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9900cbaa0 msgr2=0x7fe9900c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:50.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.111+0000 7fe995bdb700 1 --2- 172.21.15.92:0/939254678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9900cbaa0 0x7fe9900c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.111+0000 7fe995bdb700 1 -- 172.21.15.92:0/939254678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9901598a0 msgr2=0x7fe9900b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:50.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.111+0000 7fe995bdb700 1 --2- 172.21.15.92:0/939254678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9901598a0 0x7fe9900b5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.111+0000 7fe995bdb700 1 -- 172.21.15.92:0/939254678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c009a20 con 0x7fe9901594d0 2023-10-06T10:38:50.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.111+0000 7fe995bdb700 1 --2- 172.21.15.92:0/939254678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9901594d0 0x7fe990155210 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe98000c510 tx=0x7fe98000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:50.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.111+0000 7fe9867fc700 1 -- 172.21.15.92:0/939254678 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980005b30 con 0x7fe9901594d0 2023-10-06T10:38:50.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.111+0000 7fe9867fc700 1 -- 172.21.15.92:0/939254678 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe980005c90 con 0x7fe9901594d0 2023-10-06T10:38:50.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.111+0000 7fe998640700 1 -- 172.21.15.92:0/939254678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe990156020 con 0x7fe9901594d0 2023-10-06T10:38:50.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.111+0000 7fe9867fc700 1 -- 172.21.15.92:0/939254678 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980007300 con 0x7fe9901594d0 2023-10-06T10:38:50.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.111+0000 7fe998640700 1 -- 172.21.15.92:0/939254678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe990155750 con 0x7fe9901594d0 2023-10-06T10:38:50.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.112+0000 7fe998640700 1 -- 172.21.15.92:0/939254678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe990156840 con 0x7fe9901594d0 2023-10-06T10:38:50.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.112+0000 7fe9867fc700 1 -- 172.21.15.92:0/939254678 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe98001b3b0 con 0x7fe9901594d0 2023-10-06T10:38:50.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.113+0000 7fe9867fc700 1 --2- 172.21.15.92:0/939254678 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe974066f30 0x7fe9740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:50.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.113+0000 7fe9867fc700 1 -- 172.21.15.92:0/939254678 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe980083ce0 con 0x7fe9901594d0 2023-10-06T10:38:50.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.113+0000 7fe9963dc700 1 --2- 172.21.15.92:0/939254678 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe974066f30 0x7fe9740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:50.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.113+0000 7fe9963dc700 1 --2- 172.21.15.92:0/939254678 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe974066f30 0x7fe9740693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009e60 tx=0x7fe98c019000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:50.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.117+0000 7fe9867fc700 1 -- 172.21.15.92:0/939254678 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe990156840 con 0x7fe9901594d0 2023-10-06T10:38:50.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.301+0000 7fe998640700 1 -- 172.21.15.92:0/939254678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe990156840 con 0x7fe9901594d0 2023-10-06T10:38:50.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.302+0000 7fe9867fc700 1 -- 172.21.15.92:0/939254678 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe990156840 con 0x7fe9901594d0 2023-10-06T10:38:50.304 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T10:38:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.303+0000 7fe998640700 1 -- 172.21.15.92:0/939254678 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe974066f30 msgr2=0x7fe9740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.303+0000 7fe998640700 1 --2- 172.21.15.92:0/939254678 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe974066f30 0x7fe9740693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009e60 tx=0x7fe98c019000 comp rx=0 tx=0).stop 2023-10-06T10:38:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.303+0000 7fe998640700 1 -- 172.21.15.92:0/939254678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9901594d0 msgr2=0x7fe990155210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.303+0000 7fe998640700 1 --2- 172.21.15.92:0/939254678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9901594d0 0x7fe990155210 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe98000c510 tx=0x7fe98000c820 comp rx=0 tx=0).stop 2023-10-06T10:38:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.303+0000 7fe998640700 1 -- 172.21.15.92:0/939254678 shutdown_connections 2023-10-06T10:38:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.303+0000 7fe998640700 1 --2- 172.21.15.92:0/939254678 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe974066f30 0x7fe9740693e0 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009e60 tx=0x7fe98c019000 comp rx=0 tx=0).stop 2023-10-06T10:38:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.303+0000 7fe998640700 1 --2- 172.21.15.92:0/939254678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9901598a0 0x7fe9900b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.303+0000 7fe998640700 1 --2- 172.21.15.92:0/939254678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9900cbaa0 0x7fe9900c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.303+0000 7fe998640700 1 --2- 172.21.15.92:0/939254678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9901594d0 0x7fe990155210 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.303+0000 7fe998640700 1 -- 172.21.15.92:0/939254678 >> 172.21.15.92:0/939254678 conn(0x7fe990089e50 msgr2=0x7fe9900c62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.304+0000 7fe998640700 1 -- 172.21.15.92:0/939254678 shutdown_connections 2023-10-06T10:38:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.304+0000 7fe998640700 1 -- 172.21.15.92:0/939254678 wait complete. 2023-10-06T10:38:50.317 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T10:38:50.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.432+0000 7f64e01ed700 1 Processor -- start 2023-10-06T10:38:50.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.432+0000 7f64e01ed700 1 -- start start 2023-10-06T10:38:50.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.433+0000 7f64e01ed700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f64d8156740 0x7f64d806e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:50.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.433+0000 7f64e01ed700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d80c84a0 con 0x7f64d8156370 2023-10-06T10:38:50.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.433+0000 7f64e01ed700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d806f8d0 con 0x7f64d806e8d0 2023-10-06T10:38:50.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.433+0000 7f64e01ed700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d806ed10 con 0x7f64d8156740 2023-10-06T10:38:50.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.433+0000 7f64ddf89700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f64d8156370 0x7f64d806f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45046/0 (socket says 172.21.15.92:45046) 2023-10-06T10:38:50.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.433+0000 7f64ddf89700 1 -- 172.21.15.92:0/839944721 learned_addr learned my addr 172.21.15.92:0/839944721 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:50.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.433+0000 7f64dd788700 1 --2- 172.21.15.92:0/839944721 >> v2:172.21.15.124:3300/0 conn(0x7f64d8156740 0x7f64d806e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:50.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.433+0000 7f64dcf87700 1 -- 172.21.15.92:0/839944721 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74483381 0 0) 0x7f64d80c84a0 con 0x7f64d8156370 2023-10-06T10:38:50.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.433+0000 7f64dcf87700 1 -- 172.21.15.92:0/839944721 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64c4003570 con 0x7f64d8156370 2023-10-06T10:38:50.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.434+0000 7f64dcf87700 1 -- 172.21.15.92:0/839944721 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507208061 0 0) 0x7f64d806f8d0 con 0x7f64d806e8d0 2023-10-06T10:38:50.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.434+0000 7f64dcf87700 1 -- 172.21.15.92:0/839944721 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64d80c84a0 con 0x7f64d806e8d0 2023-10-06T10:38:50.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.434+0000 7f64dd788700 1 -- 172.21.15.92:0/839944721 >> v1:172.21.15.124:6789/0 conn(0x7f64d806e8d0 legacy=0x7f64d80adb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:50.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.434+0000 7f64dd788700 1 -- 172.21.15.92:0/839944721 >> v1:172.21.15.92:6789/0 conn(0x7f64d8156370 legacy=0x7f64d806f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:50.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.434+0000 7f64dd788700 1 -- 172.21.15.92:0/839944721 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d80c7890 con 0x7f64d8156740 2023-10-06T10:38:50.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.434+0000 7f64dd788700 1 --2- 172.21.15.92:0/839944721 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64d8156740 0x7f64d806e390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f64c80098e0 tx=0x7f64c80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f95188ed4aa6ef07 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:50.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.434+0000 7f64dcf87700 1 -- 172.21.15.92:0/839944721 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8011630 con 0x7f64d8156740 2023-10-06T10:38:50.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.434+0000 7f64dcf87700 1 -- 172.21.15.92:0/839944721 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c8011790 con 0x7f64d8156740 2023-10-06T10:38:50.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.434+0000 7f64dcf87700 1 -- 172.21.15.92:0/839944721 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8011b30 con 0x7f64d8156740 2023-10-06T10:38:50.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.434+0000 7f64e01ed700 1 -- 172.21.15.92:0/839944721 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64d8156740 msgr2=0x7f64d806e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:50.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.434+0000 7f64e01ed700 1 --2- 172.21.15.92:0/839944721 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64d8156740 0x7f64d806e390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f64c80098e0 tx=0x7f64c80091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.434+0000 7f64e01ed700 1 -- 172.21.15.92:0/839944721 shutdown_connections 2023-10-06T10:38:50.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.434+0000 7f64e01ed700 1 --2- 172.21.15.92:0/839944721 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64d8156740 0x7f64d806e390 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.434+0000 7f64e01ed700 1 -- 172.21.15.92:0/839944721 >> 172.21.15.92:0/839944721 conn(0x7f64d8069150 msgr2=0x7f64d80cd170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:50.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.435+0000 7f64e01ed700 1 -- 172.21.15.92:0/839944721 shutdown_connections 2023-10-06T10:38:50.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.435+0000 7f64e01ed700 1 -- 172.21.15.92:0/839944721 wait complete. 2023-10-06T10:38:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.436+0000 7f64e01ed700 1 Processor -- start 2023-10-06T10:38:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.436+0000 7f64e01ed700 1 -- start start 2023-10-06T10:38:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.436+0000 7f64e01ed700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64d806e8d0 0x7f64d8155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.436+0000 7f64e01ed700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64d8156370 0x7f64d8159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.436+0000 7f64e01ed700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64d8156740 0x7f64d80b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.436+0000 7f64e01ed700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d815a110 con 0x7f64d8156370 2023-10-06T10:38:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.436+0000 7f64e01ed700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d815a3f0 con 0x7f64d8156740 2023-10-06T10:38:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.436+0000 7f64e01ed700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d815a6d0 con 0x7f64d806e8d0 2023-10-06T10:38:50.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.436+0000 7f64dd788700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64d8156370 0x7f64d8159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:50.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.436+0000 7f64dd788700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64d8156370 0x7f64d8159750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38566/0 (socket says 172.21.15.92:38566) 2023-10-06T10:38:50.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.436+0000 7f64dd788700 1 -- 172.21.15.92:0/3265400863 learned_addr learned my addr 172.21.15.92:0/3265400863 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:50.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.437+0000 7f64ddf89700 1 --2- 172.21.15.92:0/3265400863 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64d806e8d0 0x7f64d8155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:50.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.437+0000 7f64de78a700 1 --2- 172.21.15.92:0/3265400863 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64d8156740 0x7f64d80b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:50.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.437+0000 7f64dd788700 1 -- 172.21.15.92:0/3265400863 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64d806e8d0 msgr2=0x7f64d8155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:50.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.437+0000 7f64dd788700 1 --2- 172.21.15.92:0/3265400863 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64d806e8d0 0x7f64d8155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.437+0000 7f64dd788700 1 -- 172.21.15.92:0/3265400863 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64d8156740 msgr2=0x7f64d80b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:50.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.437+0000 7f64dd788700 1 --2- 172.21.15.92:0/3265400863 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64d8156740 0x7f64d80b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.437+0000 7f64dd788700 1 -- 172.21.15.92:0/3265400863 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c8009420 con 0x7f64d8156370 2023-10-06T10:38:50.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.437+0000 7f64dd788700 1 --2- 172.21.15.92:0/3265400863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64d8156370 0x7f64d8159750 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f64c8013000 tx=0x7f64c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:50.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.437+0000 7f64cdffb700 1 -- 172.21.15.92:0/3265400863 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c801f070 con 0x7f64d8156370 2023-10-06T10:38:50.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.437+0000 7f64cdffb700 1 -- 172.21.15.92:0/3265400863 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c8000bc0 con 0x7f64d8156370 2023-10-06T10:38:50.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.437+0000 7f64e01ed700 1 -- 172.21.15.92:0/3265400863 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d815a9b0 con 0x7f64d8156370 2023-10-06T10:38:50.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.437+0000 7f64cdffb700 1 -- 172.21.15.92:0/3265400863 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8025420 con 0x7f64d8156370 2023-10-06T10:38:50.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.437+0000 7f64e01ed700 1 -- 172.21.15.92:0/3265400863 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d80c6050 con 0x7f64d8156370 2023-10-06T10:38:50.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.438+0000 7f64e01ed700 1 -- 172.21.15.92:0/3265400863 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d80c6240 con 0x7f64d8156370 2023-10-06T10:38:50.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.439+0000 7f64cdffb700 1 -- 172.21.15.92:0/3265400863 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f64c8023070 con 0x7f64d8156370 2023-10-06T10:38:50.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.439+0000 7f64cdffb700 1 --2- 172.21.15.92:0/3265400863 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f64bc066f30 0x7f64bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:50.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.439+0000 7f64cdffb700 1 -- 172.21.15.92:0/3265400863 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f64c8085bc0 con 0x7f64d8156370 2023-10-06T10:38:50.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.439+0000 7f64ddf89700 1 --2- 172.21.15.92:0/3265400863 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f64bc066f30 0x7f64bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:50.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.439+0000 7f64ddf89700 1 --2- 172.21.15.92:0/3265400863 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f64bc066f30 0x7f64bc0693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f64d40016e0 tx=0x7f64d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:50.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.443+0000 7f64cdffb700 1 -- 172.21.15.92:0/3265400863 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64d80c6240 con 0x7f64d8156370 2023-10-06T10:38:50.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.625+0000 7f64e01ed700 1 -- 172.21.15.92:0/3265400863 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f64d80c6240 con 0x7f64d8156370 2023-10-06T10:38:50.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.625+0000 7f64cdffb700 1 -- 172.21.15.92:0/3265400863 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f64c802a020 con 0x7f64d8156370 2023-10-06T10:38:50.627 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T10:38:50.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.627+0000 7f64e01ed700 1 -- 172.21.15.92:0/3265400863 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f64bc066f30 msgr2=0x7f64bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:50.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.627+0000 7f64e01ed700 1 --2- 172.21.15.92:0/3265400863 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f64bc066f30 0x7f64bc0693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f64d40016e0 tx=0x7f64d4009040 comp rx=0 tx=0).stop 2023-10-06T10:38:50.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.627+0000 7f64e01ed700 1 -- 172.21.15.92:0/3265400863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64d8156370 msgr2=0x7f64d8159750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:50.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.627+0000 7f64e01ed700 1 --2- 172.21.15.92:0/3265400863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64d8156370 0x7f64d8159750 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f64c8013000 tx=0x7f64c8017000 comp rx=0 tx=0).stop 2023-10-06T10:38:50.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.627+0000 7f64e01ed700 1 -- 172.21.15.92:0/3265400863 shutdown_connections 2023-10-06T10:38:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.627+0000 7f64e01ed700 1 --2- 172.21.15.92:0/3265400863 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f64bc066f30 0x7f64bc0693e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.627+0000 7f64e01ed700 1 --2- 172.21.15.92:0/3265400863 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64d8156740 0x7f64d80b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.627+0000 7f64e01ed700 1 --2- 172.21.15.92:0/3265400863 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64d806e8d0 0x7f64d8155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.627+0000 7f64e01ed700 1 --2- 172.21.15.92:0/3265400863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64d8156370 0x7f64d8159750 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.627+0000 7f64e01ed700 1 -- 172.21.15.92:0/3265400863 >> 172.21.15.92:0/3265400863 conn(0x7f64d8069150 msgr2=0x7f64d80c9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.627+0000 7f64e01ed700 1 -- 172.21.15.92:0/3265400863 shutdown_connections 2023-10-06T10:38:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.627+0000 7f64e01ed700 1 -- 172.21.15.92:0/3265400863 wait complete. 2023-10-06T10:38:50.641 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T10:38:50.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.757+0000 7fc23183e700 1 Processor -- start 2023-10-06T10:38:50.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.757+0000 7fc23183e700 1 -- start start 2023-10-06T10:38:50.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.758+0000 7fc23183e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc22c0cd940 0x7fc22c159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:50.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.758+0000 7fc23183e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc22c0b5ef0 con 0x7fc22c0d09e0 2023-10-06T10:38:50.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.758+0000 7fc23183e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc22c0b3600 con 0x7fc22c0b4a20 2023-10-06T10:38:50.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.758+0000 7fc23183e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0b6380 con 0x7fc22c0cd940 2023-10-06T10:38:50.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.758+0000 7fc22b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc22c0d09e0 0x7fc22c163200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45052/0 (socket says 172.21.15.92:45052) 2023-10-06T10:38:50.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.758+0000 7fc22b7fe700 1 -- 172.21.15.92:0/3969863471 learned_addr learned my addr 172.21.15.92:0/3969863471 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:50.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.758+0000 7fc22affd700 1 --2- 172.21.15.92:0/3969863471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc22c0cd940 0x7fc22c159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:50.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.758+0000 7fc229ffb700 1 -- 172.21.15.92:0/3969863471 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 479053287 0 0) 0x7fc22c0b5ef0 con 0x7fc22c0d09e0 2023-10-06T10:38:50.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.758+0000 7fc229ffb700 1 -- 172.21.15.92:0/3969863471 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc214003570 con 0x7fc22c0d09e0 2023-10-06T10:38:50.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.758+0000 7fc229ffb700 1 -- 172.21.15.92:0/3969863471 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3356287928 0 0) 0x7fc22c0b3600 con 0x7fc22c0b4a20 2023-10-06T10:38:50.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.758+0000 7fc229ffb700 1 -- 172.21.15.92:0/3969863471 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc22c0b5ef0 con 0x7fc22c0b4a20 2023-10-06T10:38:50.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.758+0000 7fc22affd700 1 -- 172.21.15.92:0/3969863471 >> v1:172.21.15.124:6789/0 conn(0x7fc22c0b4a20 legacy=0x7fc22c0d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:50.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.758+0000 7fc22affd700 1 -- 172.21.15.92:0/3969863471 >> v1:172.21.15.92:6789/0 conn(0x7fc22c0d09e0 legacy=0x7fc22c163200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:50.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.758+0000 7fc22affd700 1 -- 172.21.15.92:0/3969863471 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22c0b52e0 con 0x7fc22c0cd940 2023-10-06T10:38:50.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.758+0000 7fc22affd700 1 --2- 172.21.15.92:0/3969863471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc22c0cd940 0x7fc22c159bd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009910 tx=0x7fc21c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88d87936549f590 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:50.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.759+0000 7fc229ffb700 1 -- 172.21.15.92:0/3969863471 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c011630 con 0x7fc22c0cd940 2023-10-06T10:38:50.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.759+0000 7fc229ffb700 1 -- 172.21.15.92:0/3969863471 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21c011790 con 0x7fc22c0cd940 2023-10-06T10:38:50.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.759+0000 7fc229ffb700 1 -- 172.21.15.92:0/3969863471 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c011b30 con 0x7fc22c0cd940 2023-10-06T10:38:50.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.759+0000 7fc23183e700 1 -- 172.21.15.92:0/3969863471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc22c0cd940 msgr2=0x7fc22c159bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:50.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.759+0000 7fc23183e700 1 --2- 172.21.15.92:0/3969863471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc22c0cd940 0x7fc22c159bd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009910 tx=0x7fc21c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.759+0000 7fc23183e700 1 -- 172.21.15.92:0/3969863471 shutdown_connections 2023-10-06T10:38:50.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.759+0000 7fc23183e700 1 --2- 172.21.15.92:0/3969863471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc22c0cd940 0x7fc22c159bd0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.759+0000 7fc23183e700 1 -- 172.21.15.92:0/3969863471 >> 172.21.15.92:0/3969863471 conn(0x7fc22c06ece0 msgr2=0x7fc22c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:50.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.759+0000 7fc23183e700 1 -- 172.21.15.92:0/3969863471 shutdown_connections 2023-10-06T10:38:50.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.759+0000 7fc23183e700 1 -- 172.21.15.92:0/3969863471 wait complete. 2023-10-06T10:38:50.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.760+0000 7fc23183e700 1 Processor -- start 2023-10-06T10:38:50.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.760+0000 7fc23183e700 1 -- start start 2023-10-06T10:38:50.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.761+0000 7fc23183e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc22c0b4a20 0x7fc22c154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:50.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.761+0000 7fc23183e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc22c0cd940 0x7fc22c143870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:50.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.761+0000 7fc23183e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc22c0d09e0 0x7fc22c1561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:50.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.761+0000 7fc23183e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c1551e0 con 0x7fc22c0d09e0 2023-10-06T10:38:50.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.761+0000 7fc23183e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c155d60 con 0x7fc22c0cd940 2023-10-06T10:38:50.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.761+0000 7fc23183e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c154d90 con 0x7fc22c0b4a20 2023-10-06T10:38:50.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.761+0000 7fc22b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc22c0d09e0 0x7fc22c1561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:50.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.761+0000 7fc22b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc22c0d09e0 0x7fc22c1561f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38576/0 (socket says 172.21.15.92:38576) 2023-10-06T10:38:50.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.761+0000 7fc22b7fe700 1 -- 172.21.15.92:0/2164629331 learned_addr learned my addr 172.21.15.92:0/2164629331 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:50.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.761+0000 7fc22affd700 1 --2- 172.21.15.92:0/2164629331 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc22c0b4a20 0x7fc22c154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:50.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.761+0000 7fc22a7fc700 1 --2- 172.21.15.92:0/2164629331 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc22c0cd940 0x7fc22c143870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:50.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.761+0000 7fc22b7fe700 1 -- 172.21.15.92:0/2164629331 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc22c0b4a20 msgr2=0x7fc22c154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:50.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.761+0000 7fc22b7fe700 1 --2- 172.21.15.92:0/2164629331 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc22c0b4a20 0x7fc22c154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.761+0000 7fc22b7fe700 1 -- 172.21.15.92:0/2164629331 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc22c0cd940 msgr2=0x7fc22c143870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:50.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.761+0000 7fc22b7fe700 1 --2- 172.21.15.92:0/2164629331 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc22c0cd940 0x7fc22c143870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.761+0000 7fc22b7fe700 1 -- 172.21.15.92:0/2164629331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c009420 con 0x7fc22c0d09e0 2023-10-06T10:38:50.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.762+0000 7fc22b7fe700 1 --2- 172.21.15.92:0/2164629331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc22c0d09e0 0x7fc22c1561f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc22400cc20 tx=0x7fc224001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:50.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.762+0000 7fc212ffd700 1 -- 172.21.15.92:0/2164629331 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22400a920 con 0x7fc22c0d09e0 2023-10-06T10:38:50.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.762+0000 7fc212ffd700 1 -- 172.21.15.92:0/2164629331 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22400aa80 con 0x7fc22c0d09e0 2023-10-06T10:38:50.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.762+0000 7fc212ffd700 1 -- 172.21.15.92:0/2164629331 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22400ae00 con 0x7fc22c0d09e0 2023-10-06T10:38:50.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.762+0000 7fc23183e700 1 -- 172.21.15.92:0/2164629331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22c155430 con 0x7fc22c0d09e0 2023-10-06T10:38:50.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.762+0000 7fc23183e700 1 -- 172.21.15.92:0/2164629331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22c1559f0 con 0x7fc22c0d09e0 2023-10-06T10:38:50.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.763+0000 7fc23183e700 1 -- 172.21.15.92:0/2164629331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22c0af010 con 0x7fc22c0d09e0 2023-10-06T10:38:50.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.763+0000 7fc212ffd700 1 -- 172.21.15.92:0/2164629331 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc224007300 con 0x7fc22c0d09e0 2023-10-06T10:38:50.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.763+0000 7fc212ffd700 1 --2- 172.21.15.92:0/2164629331 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc218066f30 0x7fc2180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:50.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.764+0000 7fc212ffd700 1 -- 172.21.15.92:0/2164629331 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fc224084e10 con 0x7fc22c0d09e0 2023-10-06T10:38:50.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.764+0000 7fc22affd700 1 --2- 172.21.15.92:0/2164629331 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc218066f30 0x7fc2180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:50.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.764+0000 7fc22affd700 1 --2- 172.21.15.92:0/2164629331 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc218066f30 0x7fc2180693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009860 tx=0x7fc21c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:50.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.768+0000 7fc212ffd700 1 -- 172.21.15.92:0/2164629331 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc224051c30 con 0x7fc22c0d09e0 2023-10-06T10:38:50.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.947+0000 7fc23183e700 1 -- 172.21.15.92:0/2164629331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc22c064a90 con 0x7fc22c0d09e0 2023-10-06T10:38:50.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.948+0000 7fc212ffd700 1 -- 172.21.15.92:0/2164629331 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc224055250 con 0x7fc22c0d09e0 2023-10-06T10:38:50.950 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T10:38:50.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.950+0000 7fc23183e700 1 -- 172.21.15.92:0/2164629331 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc218066f30 msgr2=0x7fc2180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:50.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.950+0000 7fc23183e700 1 --2- 172.21.15.92:0/2164629331 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc218066f30 0x7fc2180693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009860 tx=0x7fc21c017000 comp rx=0 tx=0).stop 2023-10-06T10:38:50.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.950+0000 7fc23183e700 1 -- 172.21.15.92:0/2164629331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc22c0d09e0 msgr2=0x7fc22c1561f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:50.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.950+0000 7fc23183e700 1 --2- 172.21.15.92:0/2164629331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc22c0d09e0 0x7fc22c1561f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc22400cc20 tx=0x7fc224001a20 comp rx=0 tx=0).stop 2023-10-06T10:38:50.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.950+0000 7fc23183e700 1 -- 172.21.15.92:0/2164629331 shutdown_connections 2023-10-06T10:38:50.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.950+0000 7fc23183e700 1 --2- 172.21.15.92:0/2164629331 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc218066f30 0x7fc2180693e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.950+0000 7fc23183e700 1 --2- 172.21.15.92:0/2164629331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc22c0d09e0 0x7fc22c1561f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.950+0000 7fc23183e700 1 --2- 172.21.15.92:0/2164629331 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc22c0b4a20 0x7fc22c154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.950+0000 7fc23183e700 1 --2- 172.21.15.92:0/2164629331 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc22c0cd940 0x7fc22c143870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:50.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.950+0000 7fc23183e700 1 -- 172.21.15.92:0/2164629331 >> 172.21.15.92:0/2164629331 conn(0x7fc22c06ece0 msgr2=0x7fc22c15eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:50.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.950+0000 7fc23183e700 1 -- 172.21.15.92:0/2164629331 shutdown_connections 2023-10-06T10:38:50.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:50.950+0000 7fc23183e700 1 -- 172.21.15.92:0/2164629331 wait complete. 2023-10-06T10:38:50.964 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T10:38:51.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.079+0000 7f5c15569700 1 Processor -- start 2023-10-06T10:38:51.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.079+0000 7f5c15569700 1 -- start start 2023-10-06T10:38:51.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c15569700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c100b0520 0x7f5c100cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:51.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c15569700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c100cdd30 con 0x7f5c100c4140 2023-10-06T10:38:51.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c15569700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c100cd120 con 0x7f5c100ccaa0 2023-10-06T10:38:51.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c15569700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c100ccf10 con 0x7f5c100b0520 2023-10-06T10:38:51.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c14d68700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5c100c4140 0x7f5c100c8a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45062/0 (socket says 172.21.15.92:45062) 2023-10-06T10:38:51.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c14d68700 1 -- 172.21.15.92:0/971476981 learned_addr learned my addr 172.21.15.92:0/971476981 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:51.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c0ffff700 1 --2- 172.21.15.92:0/971476981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c100b0520 0x7f5c100cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:51.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c0effd700 1 -- 172.21.15.92:0/971476981 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715814184 0 0) 0x7f5c100cdd30 con 0x7f5c100c4140 2023-10-06T10:38:51.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c0effd700 1 -- 172.21.15.92:0/971476981 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bf8003570 con 0x7f5c100c4140 2023-10-06T10:38:51.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c0effd700 1 -- 172.21.15.92:0/971476981 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 410154120 0 0) 0x7f5bf8003570 con 0x7f5c100c4140 2023-10-06T10:38:51.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c0effd700 1 -- 172.21.15.92:0/971476981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c100b0520 msgr2=0x7f5c100cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:51.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c0effd700 1 --2- 172.21.15.92:0/971476981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c100b0520 0x7f5c100cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c0effd700 1 -- 172.21.15.92:0/971476981 >> v1:172.21.15.124:6789/0 conn(0x7f5c100ccaa0 legacy=0x7f5c100c3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:51.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c0effd700 1 -- 172.21.15.92:0/971476981 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c100cd400 con 0x7f5c100c4140 2023-10-06T10:38:51.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c0effd700 1 -- 172.21.15.92:0/971476981 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f5c080017f0 con 0x7f5c100c4140 2023-10-06T10:38:51.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c0effd700 1 -- 172.21.15.92:0/971476981 >> v1:172.21.15.92:6789/0 conn(0x7f5c100c4140 legacy=0x7f5c100c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:51.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c0effd700 1 --2- 172.21.15.92:0/971476981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf8003a90 0x7f5bf8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:51.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c0effd700 1 --2- 172.21.15.92:0/971476981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c100b0520 0x7f5bf8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:51.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c0effd700 1 --2- 172.21.15.92:0/971476981 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5bf8004d50 0x7f5bf80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:51.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c0effd700 1 -- 172.21.15.92:0/971476981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c100ccf10 con 0x7f5bf8003a90 2023-10-06T10:38:51.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c0effd700 1 -- 172.21.15.92:0/971476981 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf800bb70 con 0x7f5bf8004d50 2023-10-06T10:38:51.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c0effd700 1 -- 172.21.15.92:0/971476981 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf800bd30 con 0x7f5c100b0520 2023-10-06T10:38:51.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.080+0000 7f5c0ffff700 1 --2- 172.21.15.92:0/971476981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf8003a90 0x7f5bf8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:51.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.081+0000 7f5c0f7fe700 1 --2- 172.21.15.92:0/971476981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c100b0520 0x7f5bf8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:51.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.081+0000 7f5c14d68700 1 --2- 172.21.15.92:0/971476981 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5bf8004d50 0x7f5bf80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:51.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.081+0000 7f5c0ffff700 1 -- 172.21.15.92:0/971476981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c100b0520 msgr2=0x7f5bf8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:51.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.081+0000 7f5c0ffff700 1 --2- 172.21.15.92:0/971476981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c100b0520 0x7f5bf8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.081+0000 7f5c0ffff700 1 -- 172.21.15.92:0/971476981 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5bf8004d50 msgr2=0x7f5bf80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:51.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.081+0000 7f5c0ffff700 1 --2- 172.21.15.92:0/971476981 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5bf8004d50 0x7f5bf80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.081+0000 7f5c0ffff700 1 -- 172.21.15.92:0/971476981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf8003570 con 0x7f5bf8003a90 2023-10-06T10:38:51.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.081+0000 7f5c0ffff700 1 --2- 172.21.15.92:0/971476981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf8003a90 0x7f5bf8003ea0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009c00 tx=0x7f5c000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bcb3536c51f7d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:51.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.081+0000 7f5c0effd700 1 -- 172.21.15.92:0/971476981 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0000e710 con 0x7f5bf8003a90 2023-10-06T10:38:51.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.081+0000 7f5c0effd700 1 -- 172.21.15.92:0/971476981 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0000e870 con 0x7f5bf8003a90 2023-10-06T10:38:51.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.081+0000 7f5c0effd700 1 -- 172.21.15.92:0/971476981 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0000ec30 con 0x7f5bf8003a90 2023-10-06T10:38:51.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.081+0000 7f5c15569700 1 -- 172.21.15.92:0/971476981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf8003a90 msgr2=0x7f5bf8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:51.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.081+0000 7f5c15569700 1 --2- 172.21.15.92:0/971476981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf8003a90 0x7f5bf8003ea0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009c00 tx=0x7f5c000091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.082+0000 7f5c14d68700 1 -- 172.21.15.92:0/971476981 reap_dead start 2023-10-06T10:38:51.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.082+0000 7f5c15569700 1 -- 172.21.15.92:0/971476981 shutdown_connections 2023-10-06T10:38:51.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.082+0000 7f5c15569700 1 -- 172.21.15.92:0/971476981 >> 172.21.15.92:0/971476981 conn(0x7f5c10096a30 msgr2=0x7f5c10096e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:51.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.082+0000 7f5c15569700 1 -- 172.21.15.92:0/971476981 shutdown_connections 2023-10-06T10:38:51.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.082+0000 7f5c15569700 1 -- 172.21.15.92:0/971476981 wait complete. 2023-10-06T10:38:51.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.083+0000 7f5c15569700 1 Processor -- start 2023-10-06T10:38:51.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.083+0000 7f5c15569700 1 -- start start 2023-10-06T10:38:51.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.083+0000 7f5c15569700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf8003a90 0x7f5c10150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:51.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.084+0000 7f5c15569700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c101535c0 0x7f5c100c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:51.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.084+0000 7f5c15569700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c1006c8d0 0x7f5c1006cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:51.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.084+0000 7f5c15569700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1006d280 con 0x7f5bf8003a90 2023-10-06T10:38:51.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.084+0000 7f5c15569700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c100c84d0 con 0x7f5c1006c8d0 2023-10-06T10:38:51.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.084+0000 7f5c15569700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c100b2660 con 0x7f5c101535c0 2023-10-06T10:38:51.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.084+0000 7f5c0ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf8003a90 0x7f5c10150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:51.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.084+0000 7f5c0ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf8003a90 0x7f5c10150230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38590/0 (socket says 172.21.15.92:38590) 2023-10-06T10:38:51.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.084+0000 7f5c0ffff700 1 -- 172.21.15.92:0/1531259399 learned_addr learned my addr 172.21.15.92:0/1531259399 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:51.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.084+0000 7f5c0f7fe700 1 --2- 172.21.15.92:0/1531259399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c101535c0 0x7f5c100c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:51.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.084+0000 7f5c14d68700 1 --2- 172.21.15.92:0/1531259399 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c1006c8d0 0x7f5c1006cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:51.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.084+0000 7f5c0ffff700 1 -- 172.21.15.92:0/1531259399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c101535c0 msgr2=0x7f5c100c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:51.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.084+0000 7f5c0ffff700 1 --2- 172.21.15.92:0/1531259399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c101535c0 0x7f5c100c7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.084+0000 7f5c0ffff700 1 -- 172.21.15.92:0/1531259399 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c1006c8d0 msgr2=0x7f5c1006cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:51.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.084+0000 7f5c0ffff700 1 --2- 172.21.15.92:0/1531259399 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c1006c8d0 0x7f5c1006cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.084+0000 7f5c0ffff700 1 -- 172.21.15.92:0/1531259399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c10150770 con 0x7f5bf8003a90 2023-10-06T10:38:51.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.084+0000 7f5c0ffff700 1 --2- 172.21.15.92:0/1531259399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf8003a90 0x7f5c10150230 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5c0000e4b0 tx=0x7f5c0001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:51.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.084+0000 7f5bf7fff700 1 -- 172.21.15.92:0/1531259399 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c00022070 con 0x7f5bf8003a90 2023-10-06T10:38:51.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.084+0000 7f5bf7fff700 1 -- 172.21.15.92:0/1531259399 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0000ebf0 con 0x7f5bf8003a90 2023-10-06T10:38:51.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.084+0000 7f5bf7fff700 1 -- 172.21.15.92:0/1531259399 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0001c6c0 con 0x7f5bf8003a90 2023-10-06T10:38:51.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.084+0000 7f5c15569700 1 -- 172.21.15.92:0/1531259399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c100c81f0 con 0x7f5bf8003a90 2023-10-06T10:38:51.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.084+0000 7f5c15569700 1 -- 172.21.15.92:0/1531259399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c100b22f0 con 0x7f5bf8003a90 2023-10-06T10:38:51.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.085+0000 7f5c15569700 1 -- 172.21.15.92:0/1531259399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c100cedb0 con 0x7f5bf8003a90 2023-10-06T10:38:51.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.086+0000 7f5bf7fff700 1 -- 172.21.15.92:0/1531259399 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5c0001ca70 con 0x7f5bf8003a90 2023-10-06T10:38:51.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.086+0000 7f5bf7fff700 1 --2- 172.21.15.92:0/1531259399 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5bfc066f30 0x7f5bfc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:51.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.086+0000 7f5bf7fff700 1 -- 172.21.15.92:0/1531259399 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f5c00086120 con 0x7f5bf8003a90 2023-10-06T10:38:51.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.086+0000 7f5c0f7fe700 1 --2- 172.21.15.92:0/1531259399 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5bfc066f30 0x7f5bfc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:51.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.086+0000 7f5c0f7fe700 1 --2- 172.21.15.92:0/1531259399 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5bfc066f30 0x7f5bfc0693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5c04000fb0 tx=0x7f5c04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:51.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.090+0000 7f5bf7fff700 1 -- 172.21.15.92:0/1531259399 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c00052f40 con 0x7f5bf8003a90 2023-10-06T10:38:51.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.272+0000 7f5c15569700 1 -- 172.21.15.92:0/1531259399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5c1005f230 con 0x7f5bf8003a90 2023-10-06T10:38:51.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.273+0000 7f5bf7fff700 1 -- 172.21.15.92:0/1531259399 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5c00056560 con 0x7f5bf8003a90 2023-10-06T10:38:51.275 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T10:38:51.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.275+0000 7f5c15569700 1 -- 172.21.15.92:0/1531259399 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5bfc066f30 msgr2=0x7f5bfc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:51.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.275+0000 7f5c15569700 1 --2- 172.21.15.92:0/1531259399 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5bfc066f30 0x7f5bfc0693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5c04000fb0 tx=0x7f5c04009040 comp rx=0 tx=0).stop 2023-10-06T10:38:51.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.275+0000 7f5c15569700 1 -- 172.21.15.92:0/1531259399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf8003a90 msgr2=0x7f5c10150230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:51.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.275+0000 7f5c15569700 1 --2- 172.21.15.92:0/1531259399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf8003a90 0x7f5c10150230 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5c0000e4b0 tx=0x7f5c0001a000 comp rx=0 tx=0).stop 2023-10-06T10:38:51.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.275+0000 7f5c15569700 1 -- 172.21.15.92:0/1531259399 shutdown_connections 2023-10-06T10:38:51.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.275+0000 7f5c15569700 1 --2- 172.21.15.92:0/1531259399 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5bfc066f30 0x7f5bfc0693e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.275+0000 7f5c15569700 1 --2- 172.21.15.92:0/1531259399 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c1006c8d0 0x7f5c1006cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.275+0000 7f5c15569700 1 --2- 172.21.15.92:0/1531259399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf8003a90 0x7f5c10150230 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.275+0000 7f5c15569700 1 --2- 172.21.15.92:0/1531259399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c101535c0 0x7f5c100c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.275+0000 7f5c15569700 1 -- 172.21.15.92:0/1531259399 >> 172.21.15.92:0/1531259399 conn(0x7f5c10096a30 msgr2=0x7f5c10156a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:51.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.275+0000 7f5c15569700 1 -- 172.21.15.92:0/1531259399 shutdown_connections 2023-10-06T10:38:51.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.275+0000 7f5c15569700 1 -- 172.21.15.92:0/1531259399 wait complete. 2023-10-06T10:38:51.289 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:38:51.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.402+0000 7fe689625700 1 Processor -- start 2023-10-06T10:38:51.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.402+0000 7fe689625700 1 -- start start 2023-10-06T10:38:51.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.403+0000 7fe689625700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe6840cd330 0x7fe6840ce980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:51.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.403+0000 7fe689625700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6840cc800 con 0x7fe684159470 2023-10-06T10:38:51.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.403+0000 7fe689625700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6840cbc30 con 0x7fe6840c6220 2023-10-06T10:38:51.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.403+0000 7fe689625700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6840cbed0 con 0x7fe6840cd330 2023-10-06T10:38:51.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.403+0000 7fe683fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe684159470 0x7fe6840ccc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45074/0 (socket says 172.21.15.92:45074) 2023-10-06T10:38:51.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.403+0000 7fe683fff700 1 -- 172.21.15.92:0/3108116737 learned_addr learned my addr 172.21.15.92:0/3108116737 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:51.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.403+0000 7fe682ffd700 1 -- 172.21.15.92:0/3108116737 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1189386834 0 0) 0x7fe6840cc800 con 0x7fe684159470 2023-10-06T10:38:51.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.403+0000 7fe682ffd700 1 -- 172.21.15.92:0/3108116737 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe66c003570 con 0x7fe684159470 2023-10-06T10:38:51.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.403+0000 7fe6837fe700 1 --2- 172.21.15.92:0/3108116737 >> v2:172.21.15.124:3300/0 conn(0x7fe6840cd330 0x7fe6840ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:51.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.403+0000 7fe682ffd700 1 -- 172.21.15.92:0/3108116737 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3443407038 0 0) 0x7fe66c003570 con 0x7fe684159470 2023-10-06T10:38:51.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.403+0000 7fe682ffd700 1 -- 172.21.15.92:0/3108116737 >> v2:172.21.15.124:3300/0 conn(0x7fe6840cd330 msgr2=0x7fe6840ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:51.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.403+0000 7fe682ffd700 1 --2- 172.21.15.92:0/3108116737 >> v2:172.21.15.124:3300/0 conn(0x7fe6840cd330 0x7fe6840ce980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.403+0000 7fe682ffd700 1 -- 172.21.15.92:0/3108116737 >> v1:172.21.15.124:6789/0 conn(0x7fe6840c6220 legacy=0x7fe6840ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:51.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.403+0000 7fe682ffd700 1 -- 172.21.15.92:0/3108116737 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6840cc1b0 con 0x7fe684159470 2023-10-06T10:38:51.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.403+0000 7fe682ffd700 1 -- 172.21.15.92:0/3108116737 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fe6740015f0 con 0x7fe684159470 2023-10-06T10:38:51.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.404+0000 7fe682ffd700 1 -- 172.21.15.92:0/3108116737 >> v1:172.21.15.92:6789/0 conn(0x7fe684159470 legacy=0x7fe6840ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:51.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.404+0000 7fe682ffd700 1 --2- 172.21.15.92:0/3108116737 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe66c003a60 0x7fe66c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:51.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.404+0000 7fe682ffd700 1 --2- 172.21.15.92:0/3108116737 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe66c004810 0x7fe66c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:51.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.404+0000 7fe682ffd700 1 --2- 172.21.15.92:0/3108116737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe66c0051c0 0x7fe66c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:51.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.404+0000 7fe682ffd700 1 -- 172.21.15.92:0/3108116737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6840cbed0 con 0x7fe66c0051c0 2023-10-06T10:38:51.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.404+0000 7fe682ffd700 1 -- 172.21.15.92:0/3108116737 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c00c1e0 con 0x7fe66c004810 2023-10-06T10:38:51.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.404+0000 7fe682ffd700 1 -- 172.21.15.92:0/3108116737 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c00c3a0 con 0x7fe66c003a60 2023-10-06T10:38:51.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.404+0000 7fe688e24700 1 --2- 172.21.15.92:0/3108116737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe66c0051c0 0x7fe66c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:51.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.404+0000 7fe6837fe700 1 --2- 172.21.15.92:0/3108116737 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe66c004810 0x7fe66c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:51.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.404+0000 7fe683fff700 1 --2- 172.21.15.92:0/3108116737 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe66c003a60 0x7fe66c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:51.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.404+0000 7fe688e24700 1 -- 172.21.15.92:0/3108116737 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe66c003a60 msgr2=0x7fe66c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:51.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.404+0000 7fe688e24700 1 --2- 172.21.15.92:0/3108116737 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe66c003a60 0x7fe66c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.404+0000 7fe688e24700 1 -- 172.21.15.92:0/3108116737 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe66c004810 msgr2=0x7fe66c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:51.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.404+0000 7fe688e24700 1 --2- 172.21.15.92:0/3108116737 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe66c004810 0x7fe66c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.404+0000 7fe688e24700 1 -- 172.21.15.92:0/3108116737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c003570 con 0x7fe66c0051c0 2023-10-06T10:38:51.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.404+0000 7fe688e24700 1 -- 172.21.15.92:0/3108116737 reap_dead start 2023-10-06T10:38:51.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.404+0000 7fe688e24700 1 --2- 172.21.15.92:0/3108116737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe66c0051c0 0x7fe66c00ba60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe67c001f90 tx=0x7fe67c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b973f059f153f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:51.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.404+0000 7fe682ffd700 1 -- 172.21.15.92:0/3108116737 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c001950 con 0x7fe66c0051c0 2023-10-06T10:38:51.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.404+0000 7fe682ffd700 1 -- 172.21.15.92:0/3108116737 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67c00c8b0 con 0x7fe66c0051c0 2023-10-06T10:38:51.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.404+0000 7fe682ffd700 1 -- 172.21.15.92:0/3108116737 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c00cc70 con 0x7fe66c0051c0 2023-10-06T10:38:51.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.405+0000 7fe689625700 1 -- 172.21.15.92:0/3108116737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe66c0051c0 msgr2=0x7fe66c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:51.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.405+0000 7fe689625700 1 --2- 172.21.15.92:0/3108116737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe66c0051c0 0x7fe66c00ba60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe67c001f90 tx=0x7fe67c001640 comp rx=0 tx=0).stop 2023-10-06T10:38:51.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.405+0000 7fe689625700 1 -- 172.21.15.92:0/3108116737 shutdown_connections 2023-10-06T10:38:51.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.405+0000 7fe689625700 1 --2- 172.21.15.92:0/3108116737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe66c0051c0 0x7fe66c00ba60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.405+0000 7fe689625700 1 -- 172.21.15.92:0/3108116737 >> 172.21.15.92:0/3108116737 conn(0x7fe6840995c0 msgr2=0x7fe68406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:51.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.405+0000 7fe689625700 1 -- 172.21.15.92:0/3108116737 shutdown_connections 2023-10-06T10:38:51.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.405+0000 7fe689625700 1 -- 172.21.15.92:0/3108116737 wait complete. 2023-10-06T10:38:51.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.406+0000 7fe689625700 1 Processor -- start 2023-10-06T10:38:51.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.406+0000 7fe689625700 1 -- start start 2023-10-06T10:38:51.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.407+0000 7fe689625700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe66c0051c0 0x7fe68409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:51.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.407+0000 7fe689625700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe68409a630 0x7fe684156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:51.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.407+0000 7fe689625700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe684169410 0x7fe68416ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:51.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.407+0000 7fe689625700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68409d770 con 0x7fe684169410 2023-10-06T10:38:51.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.407+0000 7fe689625700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6840c8de0 con 0x7fe66c0051c0 2023-10-06T10:38:51.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.407+0000 7fe689625700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6840c84b0 con 0x7fe68409a630 2023-10-06T10:38:51.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.407+0000 7fe688e24700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe684169410 0x7fe68416ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:51.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.407+0000 7fe688e24700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe684169410 0x7fe68416ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38610/0 (socket says 172.21.15.92:38610) 2023-10-06T10:38:51.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.407+0000 7fe688e24700 1 -- 172.21.15.92:0/411077982 learned_addr learned my addr 172.21.15.92:0/411077982 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:51.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.407+0000 7fe683fff700 1 --2- 172.21.15.92:0/411077982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe66c0051c0 0x7fe68409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:51.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.407+0000 7fe6837fe700 1 --2- 172.21.15.92:0/411077982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe68409a630 0x7fe684156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:51.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.407+0000 7fe688e24700 1 -- 172.21.15.92:0/411077982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe68409a630 msgr2=0x7fe684156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:51.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.407+0000 7fe688e24700 1 --2- 172.21.15.92:0/411077982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe68409a630 0x7fe684156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.407+0000 7fe688e24700 1 -- 172.21.15.92:0/411077982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe66c0051c0 msgr2=0x7fe68409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:51.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.407+0000 7fe688e24700 1 --2- 172.21.15.92:0/411077982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe66c0051c0 0x7fe68409cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.407+0000 7fe688e24700 1 -- 172.21.15.92:0/411077982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6840c8790 con 0x7fe684169410 2023-10-06T10:38:51.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.407+0000 7fe6837fe700 1 --2- 172.21.15.92:0/411077982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe68409a630 0x7fe684156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:51.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.407+0000 7fe688e24700 1 --2- 172.21.15.92:0/411077982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe684169410 0x7fe68416ca10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe6840b2e40 tx=0x7fe67c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:51.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.407+0000 7fe66bfff700 1 -- 172.21.15.92:0/411077982 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c001950 con 0x7fe684169410 2023-10-06T10:38:51.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.407+0000 7fe66bfff700 1 -- 172.21.15.92:0/411077982 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67c001250 con 0x7fe684169410 2023-10-06T10:38:51.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.408+0000 7fe689625700 1 -- 172.21.15.92:0/411077982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6840c8a70 con 0x7fe684169410 2023-10-06T10:38:51.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.408+0000 7fe66bfff700 1 -- 172.21.15.92:0/411077982 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c01e870 con 0x7fe684169410 2023-10-06T10:38:51.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.408+0000 7fe689625700 1 -- 172.21.15.92:0/411077982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe684156880 con 0x7fe684169410 2023-10-06T10:38:51.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.409+0000 7fe689625700 1 -- 172.21.15.92:0/411077982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6840c8f50 con 0x7fe684169410 2023-10-06T10:38:51.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.409+0000 7fe66bfff700 1 -- 172.21.15.92:0/411077982 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe67c01ec60 con 0x7fe684169410 2023-10-06T10:38:51.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.409+0000 7fe66bfff700 1 --2- 172.21.15.92:0/411077982 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe670066f30 0x7fe6700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:51.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.409+0000 7fe66bfff700 1 -- 172.21.15.92:0/411077982 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe67c086100 con 0x7fe684169410 2023-10-06T10:38:51.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.409+0000 7fe683fff700 1 --2- 172.21.15.92:0/411077982 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe670066f30 0x7fe6700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:51.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.410+0000 7fe683fff700 1 --2- 172.21.15.92:0/411077982 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe670066f30 0x7fe6700693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe674001ed0 tx=0x7fe674008be0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:51.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.413+0000 7fe66bfff700 1 -- 172.21.15.92:0/411077982 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe67c052f20 con 0x7fe684169410 2023-10-06T10:38:51.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.601+0000 7fe689625700 1 -- 172.21.15.92:0/411077982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe68405f230 con 0x7fe684169410 2023-10-06T10:38:51.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.602+0000 7fe66bfff700 1 -- 172.21.15.92:0/411077982 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18707 (secure 0 0 0) 0x7fe67c004020 con 0x7fe684169410 2023-10-06T10:38:51.604 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:51.604 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:21.760233+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:38:14.368713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T10:38:15.692344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T10:38:19.382553+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:38:51.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.604+0000 7fe689625700 1 -- 172.21.15.92:0/411077982 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe670066f30 msgr2=0x7fe6700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:51.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.604+0000 7fe689625700 1 --2- 172.21.15.92:0/411077982 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe670066f30 0x7fe6700693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe674001ed0 tx=0x7fe674008be0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.604+0000 7fe689625700 1 -- 172.21.15.92:0/411077982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe684169410 msgr2=0x7fe68416ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:51.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.604+0000 7fe689625700 1 --2- 172.21.15.92:0/411077982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe684169410 0x7fe68416ca10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe6840b2e40 tx=0x7fe67c015000 comp rx=0 tx=0).stop 2023-10-06T10:38:51.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.604+0000 7fe689625700 1 -- 172.21.15.92:0/411077982 shutdown_connections 2023-10-06T10:38:51.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.604+0000 7fe689625700 1 --2- 172.21.15.92:0/411077982 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe670066f30 0x7fe6700693e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.604+0000 7fe689625700 1 --2- 172.21.15.92:0/411077982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe684169410 0x7fe68416ca10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.604+0000 7fe689625700 1 --2- 172.21.15.92:0/411077982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe66c0051c0 0x7fe68409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.604+0000 7fe689625700 1 --2- 172.21.15.92:0/411077982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe68409a630 0x7fe684156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.604+0000 7fe689625700 1 -- 172.21.15.92:0/411077982 >> 172.21.15.92:0/411077982 conn(0x7fe6840995c0 msgr2=0x7fe6840cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:51.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.604+0000 7fe689625700 1 -- 172.21.15.92:0/411077982 shutdown_connections 2023-10-06T10:38:51.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.604+0000 7fe689625700 1 -- 172.21.15.92:0/411077982 wait complete. 2023-10-06T10:38:51.619 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T10:38:09.680559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T10:38:10.120786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T10:38:14.368713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T10:38:15.692344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T10:38:19.382553+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T10:38:51.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T10:38:51.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.735+0000 7ff46b06b700 1 Processor -- start 2023-10-06T10:38:51.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.735+0000 7ff46b06b700 1 -- start start 2023-10-06T10:38:51.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.736+0000 7ff46b06b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff4640cca70 0x7ff4640d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:51.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.736+0000 7ff46b06b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff4640d07f0 0x7ff4640ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:51.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.736+0000 7ff46b06b700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4640cdbf0 con 0x7ff464153990 2023-10-06T10:38:51.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.736+0000 7ff46b06b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4640cb4c0 con 0x7ff4640d07f0 2023-10-06T10:38:51.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.736+0000 7ff46b06b700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4640d0c10 con 0x7ff4640cca70 2023-10-06T10:38:51.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.736+0000 7ff46a86a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff4640d07f0 0x7ff4640ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:51.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.736+0000 7ff46a86a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff4640d07f0 0x7ff4640ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38622/0 (socket says 172.21.15.92:38622) 2023-10-06T10:38:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.736+0000 7ff46a86a700 1 -- 172.21.15.92:0/4082061057 learned_addr learned my addr 172.21.15.92:0/4082061057 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.736+0000 7ff469868700 1 --2- 172.21.15.92:0/4082061057 >> v2:172.21.15.124:3300/0 conn(0x7ff4640cca70 0x7ff4640d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.736+0000 7ff46a86a700 1 -- 172.21.15.92:0/4082061057 >> v2:172.21.15.124:3300/0 conn(0x7ff4640cca70 msgr2=0x7ff4640d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.736+0000 7ff46a86a700 1 --2- 172.21.15.92:0/4082061057 >> v2:172.21.15.124:3300/0 conn(0x7ff4640cca70 0x7ff4640d02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.736+0000 7ff46a86a700 1 -- 172.21.15.92:0/4082061057 >> v1:172.21.15.124:6789/0 conn(0x7ff464153990 legacy=0x7ff46415a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.736+0000 7ff46a86a700 1 -- 172.21.15.92:0/4082061057 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4640ccfe0 con 0x7ff4640d07f0 2023-10-06T10:38:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.736+0000 7ff46a86a700 1 --2- 172.21.15.92:0/4082061057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4640d07f0 0x7ff4640ce860 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff464088f40 tx=0x7ff45c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988ca989936cc644 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:51.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.736+0000 7ff469067700 1 -- 172.21.15.92:0/4082061057 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c00e070 con 0x7ff4640d07f0 2023-10-06T10:38:51.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.736+0000 7ff469067700 1 -- 172.21.15.92:0/4082061057 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45c00ba10 con 0x7ff4640d07f0 2023-10-06T10:38:51.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.736+0000 7ff469067700 1 -- 172.21.15.92:0/4082061057 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c0092e0 con 0x7ff4640d07f0 2023-10-06T10:38:51.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.737+0000 7ff46b06b700 1 -- 172.21.15.92:0/4082061057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4640d07f0 msgr2=0x7ff4640ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:51.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.737+0000 7ff46b06b700 1 --2- 172.21.15.92:0/4082061057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4640d07f0 0x7ff4640ce860 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff464088f40 tx=0x7ff45c009840 comp rx=0 tx=0).stop 2023-10-06T10:38:51.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.737+0000 7ff46b06b700 1 -- 172.21.15.92:0/4082061057 shutdown_connections 2023-10-06T10:38:51.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.737+0000 7ff46b06b700 1 --2- 172.21.15.92:0/4082061057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4640d07f0 0x7ff4640ce860 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.737+0000 7ff46b06b700 1 --2- 172.21.15.92:0/4082061057 >> v2:172.21.15.124:3300/0 conn(0x7ff4640cca70 0x7ff4640d02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.737+0000 7ff46b06b700 1 -- 172.21.15.92:0/4082061057 >> 172.21.15.92:0/4082061057 conn(0x7ff4640680c0 msgr2=0x7ff464068490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:51.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.737+0000 7ff46b06b700 1 -- 172.21.15.92:0/4082061057 shutdown_connections 2023-10-06T10:38:51.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.737+0000 7ff46b06b700 1 -- 172.21.15.92:0/4082061057 wait complete. 2023-10-06T10:38:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.738+0000 7ff46b06b700 1 Processor -- start 2023-10-06T10:38:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.738+0000 7ff46b06b700 1 -- start start 2023-10-06T10:38:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.739+0000 7ff46b06b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4640cca70 0x7ff464155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.739+0000 7ff46b06b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4640d07f0 0x7ff464158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.739+0000 7ff46b06b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff464153990 0x7ff46415e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.739+0000 7ff46b06b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4640b0e70 con 0x7ff464153990 2023-10-06T10:38:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.739+0000 7ff46b06b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4640b1e20 con 0x7ff4640d07f0 2023-10-06T10:38:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.739+0000 7ff46b06b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4640affc0 con 0x7ff4640cca70 2023-10-06T10:38:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.739+0000 7ff46a86a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff464153990 0x7ff46415e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.739+0000 7ff46a86a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff464153990 0x7ff46415e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38630/0 (socket says 172.21.15.92:38630) 2023-10-06T10:38:51.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.739+0000 7ff46a86a700 1 -- 172.21.15.92:0/224621944 learned_addr learned my addr 172.21.15.92:0/224621944 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:51.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.739+0000 7ff469868700 1 --2- 172.21.15.92:0/224621944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4640d07f0 0x7ff464158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:51.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.739+0000 7ff46a069700 1 --2- 172.21.15.92:0/224621944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4640cca70 0x7ff464155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:51.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.739+0000 7ff46a86a700 1 -- 172.21.15.92:0/224621944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4640cca70 msgr2=0x7ff464155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:51.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.739+0000 7ff46a86a700 1 --2- 172.21.15.92:0/224621944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4640cca70 0x7ff464155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.739+0000 7ff46a86a700 1 -- 172.21.15.92:0/224621944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4640d07f0 msgr2=0x7ff464158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:51.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.739+0000 7ff46a86a700 1 --2- 172.21.15.92:0/224621944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4640d07f0 0x7ff464158040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.739+0000 7ff46a86a700 1 -- 172.21.15.92:0/224621944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c009ae0 con 0x7ff464153990 2023-10-06T10:38:51.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.739+0000 7ff469868700 1 --2- 172.21.15.92:0/224621944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4640d07f0 0x7ff464158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:51.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.739+0000 7ff46a86a700 1 --2- 172.21.15.92:0/224621944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff464153990 0x7ff46415e760 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff4640890b0 tx=0x7ff45c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:51.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.739+0000 7ff45a7fc700 1 -- 172.21.15.92:0/224621944 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c00bee0 con 0x7ff464153990 2023-10-06T10:38:51.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.739+0000 7ff45a7fc700 1 -- 172.21.15.92:0/224621944 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45c0094e0 con 0x7ff464153990 2023-10-06T10:38:51.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.739+0000 7ff46b06b700 1 -- 172.21.15.92:0/224621944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4640b0260 con 0x7ff464153990 2023-10-06T10:38:51.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.740+0000 7ff45a7fc700 1 -- 172.21.15.92:0/224621944 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c01d650 con 0x7ff464153990 2023-10-06T10:38:51.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.740+0000 7ff46b06b700 1 -- 172.21.15.92:0/224621944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4640b0820 con 0x7ff464153990 2023-10-06T10:38:51.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.740+0000 7ff46b06b700 1 -- 172.21.15.92:0/224621944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff46405f230 con 0x7ff464153990 2023-10-06T10:38:51.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.741+0000 7ff45a7fc700 1 -- 172.21.15.92:0/224621944 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff45c01da80 con 0x7ff464153990 2023-10-06T10:38:51.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.741+0000 7ff45a7fc700 1 --2- 172.21.15.92:0/224621944 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff448066f30 0x7ff4480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:51.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.741+0000 7ff45a7fc700 1 -- 172.21.15.92:0/224621944 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ff45c0860d0 con 0x7ff464153990 2023-10-06T10:38:51.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.741+0000 7ff46a069700 1 --2- 172.21.15.92:0/224621944 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff448066f30 0x7ff4480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:51.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.742+0000 7ff46a069700 1 --2- 172.21.15.92:0/224621944 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff448066f30 0x7ff4480693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff4640b1fd0 tx=0x7ff460009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:51.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.745+0000 7ff45a7fc700 1 -- 172.21.15.92:0/224621944 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff45c052ef0 con 0x7ff464153990 2023-10-06T10:38:51.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.927+0000 7ff46b06b700 1 -- 172.21.15.92:0/224621944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff4640cd2c0 con 0x7ff464153990 2023-10-06T10:38:51.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.927+0000 7ff45a7fc700 1 -- 172.21.15.92:0/224621944 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff45c012e60 con 0x7ff464153990 2023-10-06T10:38:51.929 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T10:38:51.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.929+0000 7ff46b06b700 1 -- 172.21.15.92:0/224621944 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff448066f30 msgr2=0x7ff4480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:51.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.929+0000 7ff46b06b700 1 --2- 172.21.15.92:0/224621944 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff448066f30 0x7ff4480693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff4640b1fd0 tx=0x7ff460009040 comp rx=0 tx=0).stop 2023-10-06T10:38:51.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.929+0000 7ff46b06b700 1 -- 172.21.15.92:0/224621944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff464153990 msgr2=0x7ff46415e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:51.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.929+0000 7ff46b06b700 1 --2- 172.21.15.92:0/224621944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff464153990 0x7ff46415e760 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff4640890b0 tx=0x7ff45c017000 comp rx=0 tx=0).stop 2023-10-06T10:38:51.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.929+0000 7ff46b06b700 1 -- 172.21.15.92:0/224621944 shutdown_connections 2023-10-06T10:38:51.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.929+0000 7ff46b06b700 1 --2- 172.21.15.92:0/224621944 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff448066f30 0x7ff4480693e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.929+0000 7ff46b06b700 1 --2- 172.21.15.92:0/224621944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff464153990 0x7ff46415e760 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.929+0000 7ff46b06b700 1 --2- 172.21.15.92:0/224621944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4640cca70 0x7ff464155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.929+0000 7ff46b06b700 1 --2- 172.21.15.92:0/224621944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4640d07f0 0x7ff464158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:51.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.929+0000 7ff46b06b700 1 -- 172.21.15.92:0/224621944 >> 172.21.15.92:0/224621944 conn(0x7ff4640680c0 msgr2=0x7ff464151e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:51.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.929+0000 7ff46b06b700 1 -- 172.21.15.92:0/224621944 shutdown_connections 2023-10-06T10:38:51.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:51.929+0000 7ff46b06b700 1 -- 172.21.15.92:0/224621944 wait complete. 2023-10-06T10:38:51.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T10:38:52.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.058+0000 7f603e233700 1 Processor -- start 2023-10-06T10:38:52.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.058+0000 7f603e233700 1 -- start start 2023-10-06T10:38:52.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.059+0000 7f603e233700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f60381594d0 0x7f60380cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:52.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.059+0000 7f603e233700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60380cbaa0 0x7f60380ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:52.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.059+0000 7f603e233700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60380cbeb0 con 0x7f60381598a0 2023-10-06T10:38:52.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.059+0000 7f603e233700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60380cf5d0 con 0x7f60381594d0 2023-10-06T10:38:52.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.059+0000 7f603e233700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60380cf8b0 con 0x7f60380cbaa0 2023-10-06T10:38:52.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.059+0000 7f60377fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f60381594d0 0x7f60380cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:52.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.059+0000 7f60377fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f60381594d0 0x7f60380cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38632/0 (socket says 172.21.15.92:38632) 2023-10-06T10:38:52.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.059+0000 7f60377fe700 1 -- 172.21.15.92:0/2928350901 learned_addr learned my addr 172.21.15.92:0/2928350901 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:52.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.059+0000 7f6037fff700 1 --2- 172.21.15.92:0/2928350901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60380cbaa0 0x7f60380ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:52.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.059+0000 7f60377fe700 1 -- 172.21.15.92:0/2928350901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60380cbaa0 msgr2=0x7f60380ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:52.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.059+0000 7f60377fe700 1 --2- 172.21.15.92:0/2928350901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60380cbaa0 0x7f60380ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.059+0000 7f60377fe700 1 -- 172.21.15.92:0/2928350901 >> v1:172.21.15.124:6789/0 conn(0x7f60381598a0 legacy=0x7f60380ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:52.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.059+0000 7f60377fe700 1 -- 172.21.15.92:0/2928350901 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60380cfb90 con 0x7f60381594d0 2023-10-06T10:38:52.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.059+0000 7f60377fe700 1 --2- 172.21.15.92:0/2928350901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60381594d0 0x7f60380cb560 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6028009f20 tx=0x7f6028009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188c025ebdafe90 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:52.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.059+0000 7f60367fc700 1 -- 172.21.15.92:0/2928350901 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602800e070 con 0x7f60381594d0 2023-10-06T10:38:52.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.060+0000 7f60367fc700 1 -- 172.21.15.92:0/2928350901 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602800ba40 con 0x7f60381594d0 2023-10-06T10:38:52.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.060+0000 7f60367fc700 1 -- 172.21.15.92:0/2928350901 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60280092e0 con 0x7f60381594d0 2023-10-06T10:38:52.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.060+0000 7f603e233700 1 -- 172.21.15.92:0/2928350901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60381594d0 msgr2=0x7f60380cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:52.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.060+0000 7f603e233700 1 --2- 172.21.15.92:0/2928350901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60381594d0 0x7f60380cb560 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6028009f20 tx=0x7f6028009840 comp rx=0 tx=0).stop 2023-10-06T10:38:52.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.060+0000 7f603e233700 1 -- 172.21.15.92:0/2928350901 shutdown_connections 2023-10-06T10:38:52.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.060+0000 7f603e233700 1 --2- 172.21.15.92:0/2928350901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60380cbaa0 0x7f60380ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.060+0000 7f603e233700 1 --2- 172.21.15.92:0/2928350901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60381594d0 0x7f60380cb560 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.060+0000 7f603e233700 1 -- 172.21.15.92:0/2928350901 >> 172.21.15.92:0/2928350901 conn(0x7f6038089e50 msgr2=0x7f6038099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:52.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.060+0000 7f603e233700 1 -- 172.21.15.92:0/2928350901 shutdown_connections 2023-10-06T10:38:52.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.060+0000 7f603e233700 1 -- 172.21.15.92:0/2928350901 wait complete. 2023-10-06T10:38:52.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.061+0000 7f603e233700 1 Processor -- start 2023-10-06T10:38:52.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.061+0000 7f603e233700 1 -- start start 2023-10-06T10:38:52.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.062+0000 7f603e233700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60380cbaa0 0x7f603816b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:52.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.062+0000 7f603e233700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60381594d0 0x7f603816cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:52.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.062+0000 7f603e233700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60381598a0 0x7f603814a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:52.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.062+0000 7f603e233700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603814dd70 con 0x7f60381594d0 2023-10-06T10:38:52.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.062+0000 7f603e233700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603814da90 con 0x7f60381598a0 2023-10-06T10:38:52.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.062+0000 7f603e233700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60380b4420 con 0x7f60380cbaa0 2023-10-06T10:38:52.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.062+0000 7f6036ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60381594d0 0x7f603816cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:52.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.062+0000 7f6036ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60381594d0 0x7f603816cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38648/0 (socket says 172.21.15.92:38648) 2023-10-06T10:38:52.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.062+0000 7f6036ffd700 1 -- 172.21.15.92:0/3508284286 learned_addr learned my addr 172.21.15.92:0/3508284286 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:52.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.062+0000 7f60377fe700 1 --2- 172.21.15.92:0/3508284286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60380cbaa0 0x7f603816b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:52.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.062+0000 7f6037fff700 1 --2- 172.21.15.92:0/3508284286 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60381598a0 0x7f603814a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:52.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.062+0000 7f6036ffd700 1 -- 172.21.15.92:0/3508284286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60380cbaa0 msgr2=0x7f603816b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:52.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.062+0000 7f6036ffd700 1 --2- 172.21.15.92:0/3508284286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60380cbaa0 0x7f603816b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.062+0000 7f6036ffd700 1 -- 172.21.15.92:0/3508284286 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60381598a0 msgr2=0x7f603814a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:52.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.062+0000 7f6036ffd700 1 --2- 172.21.15.92:0/3508284286 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60381598a0 0x7f603814a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.062+0000 7f6036ffd700 1 -- 172.21.15.92:0/3508284286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6028009ae0 con 0x7f60381594d0 2023-10-06T10:38:52.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.062+0000 7f6037fff700 1 --2- 172.21.15.92:0/3508284286 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60381598a0 0x7f603814a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:52.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.062+0000 7f60377fe700 1 --2- 172.21.15.92:0/3508284286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60380cbaa0 0x7f603816b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:52.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.062+0000 7f6036ffd700 1 --2- 172.21.15.92:0/3508284286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60381594d0 0x7f603816cf60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f602c00b510 tx=0x7f602c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:52.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.063+0000 7f601f7fe700 1 -- 172.21.15.92:0/3508284286 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c00f510 con 0x7f60381594d0 2023-10-06T10:38:52.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.063+0000 7f601f7fe700 1 -- 172.21.15.92:0/3508284286 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602c00f670 con 0x7f60381594d0 2023-10-06T10:38:52.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.063+0000 7f603e233700 1 -- 172.21.15.92:0/3508284286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60380b4700 con 0x7f60381594d0 2023-10-06T10:38:52.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.063+0000 7f601f7fe700 1 -- 172.21.15.92:0/3508284286 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c00fa30 con 0x7f60381594d0 2023-10-06T10:38:52.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.063+0000 7f603e233700 1 -- 172.21.15.92:0/3508284286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603816d4a0 con 0x7f60381594d0 2023-10-06T10:38:52.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.064+0000 7f603e233700 1 -- 172.21.15.92:0/3508284286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60380b4280 con 0x7f60381594d0 2023-10-06T10:38:52.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.064+0000 7f601f7fe700 1 -- 172.21.15.92:0/3508284286 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f602c01a400 con 0x7f60381594d0 2023-10-06T10:38:52.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.064+0000 7f601f7fe700 1 --2- 172.21.15.92:0/3508284286 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6024066f30 0x7f60240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:52.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.065+0000 7f60377fe700 1 --2- 172.21.15.92:0/3508284286 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6024066f30 0x7f60240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:52.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.065+0000 7f601f7fe700 1 -- 172.21.15.92:0/3508284286 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f602c084510 con 0x7f60381594d0 2023-10-06T10:38:52.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.065+0000 7f60377fe700 1 --2- 172.21.15.92:0/3508284286 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6024066f30 0x7f60240693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6028009880 tx=0x7f6028017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:52.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.069+0000 7f601f7fe700 1 -- 172.21.15.92:0/3508284286 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f60380b4280 con 0x7f60381594d0 2023-10-06T10:38:52.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.251+0000 7f603e233700 1 -- 172.21.15.92:0/3508284286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f60380b4c20 con 0x7f60381594d0 2023-10-06T10:38:52.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.251+0000 7f601f7fe700 1 -- 172.21.15.92:0/3508284286 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f60380b4c20 con 0x7f60381594d0 2023-10-06T10:38:52.253 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T10:38:52.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.253+0000 7f603e233700 1 -- 172.21.15.92:0/3508284286 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6024066f30 msgr2=0x7f60240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:52.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.253+0000 7f603e233700 1 --2- 172.21.15.92:0/3508284286 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6024066f30 0x7f60240693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6028009880 tx=0x7f6028017000 comp rx=0 tx=0).stop 2023-10-06T10:38:52.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.253+0000 7f603e233700 1 -- 172.21.15.92:0/3508284286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60381594d0 msgr2=0x7f603816cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:52.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.253+0000 7f603e233700 1 --2- 172.21.15.92:0/3508284286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60381594d0 0x7f603816cf60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f602c00b510 tx=0x7f602c0090f0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.253+0000 7f603e233700 1 -- 172.21.15.92:0/3508284286 shutdown_connections 2023-10-06T10:38:52.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.253+0000 7f603e233700 1 --2- 172.21.15.92:0/3508284286 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6024066f30 0x7f60240693e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.253+0000 7f603e233700 1 --2- 172.21.15.92:0/3508284286 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60381598a0 0x7f603814a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.253+0000 7f603e233700 1 --2- 172.21.15.92:0/3508284286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60380cbaa0 0x7f603816b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.253+0000 7f603e233700 1 --2- 172.21.15.92:0/3508284286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60381594d0 0x7f603816cf60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.253+0000 7f603e233700 1 -- 172.21.15.92:0/3508284286 >> 172.21.15.92:0/3508284286 conn(0x7f6038089e50 msgr2=0x7f60380c6500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:52.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.253+0000 7f603e233700 1 -- 172.21.15.92:0/3508284286 shutdown_connections 2023-10-06T10:38:52.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.253+0000 7f603e233700 1 -- 172.21.15.92:0/3508284286 wait complete. 2023-10-06T10:38:52.267 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T10:38:52.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.382+0000 7f6733c19700 1 Processor -- start 2023-10-06T10:38:52.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.382+0000 7f6733c19700 1 -- start start 2023-10-06T10:38:52.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.383+0000 7f6733c19700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f672c1598a0 0x7f672c0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:52.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.383+0000 7f6733c19700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672c0cf5d0 con 0x7f672c0cbc70 2023-10-06T10:38:52.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.383+0000 7f6733c19700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672c0cf8b0 con 0x7f672c1594d0 2023-10-06T10:38:52.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.383+0000 7f6733c19700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672c0cfb90 con 0x7f672c1598a0 2023-10-06T10:38:52.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.383+0000 7f67321b6700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f672c0cbc70 0x7f672c0ced20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45076/0 (socket says 172.21.15.92:45076) 2023-10-06T10:38:52.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.383+0000 7f67321b6700 1 -- 172.21.15.92:0/4150885326 learned_addr learned my addr 172.21.15.92:0/4150885326 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:52.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.383+0000 7f67311b4700 1 --2- 172.21.15.92:0/4150885326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f672c1598a0 0x7f672c0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:52.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.383+0000 7f67309b3700 1 -- 172.21.15.92:0/4150885326 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3358450291 0 0) 0x7f672c0cf5d0 con 0x7f672c0cbc70 2023-10-06T10:38:52.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.383+0000 7f67309b3700 1 -- 172.21.15.92:0/4150885326 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6718003570 con 0x7f672c0cbc70 2023-10-06T10:38:52.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.383+0000 7f67309b3700 1 -- 172.21.15.92:0/4150885326 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1032765050 0 0) 0x7f672c0cf8b0 con 0x7f672c1594d0 2023-10-06T10:38:52.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.383+0000 7f67309b3700 1 -- 172.21.15.92:0/4150885326 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f672c0cf5d0 con 0x7f672c1594d0 2023-10-06T10:38:52.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.384+0000 7f67311b4700 1 -- 172.21.15.92:0/4150885326 >> v1:172.21.15.124:6789/0 conn(0x7f672c1594d0 legacy=0x7f672c0cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:52.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.384+0000 7f67311b4700 1 -- 172.21.15.92:0/4150885326 >> v1:172.21.15.92:6789/0 conn(0x7f672c0cbc70 legacy=0x7f672c0ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:52.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.384+0000 7f67311b4700 1 -- 172.21.15.92:0/4150885326 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672c0cfe70 con 0x7f672c1598a0 2023-10-06T10:38:52.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.384+0000 7f67311b4700 1 --2- 172.21.15.92:0/4150885326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f672c1598a0 0x7f672c0ce7e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f671c009910 tx=0x7f671c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4af13a151aa17b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:52.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.384+0000 7f67309b3700 1 -- 172.21.15.92:0/4150885326 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c011630 con 0x7f672c1598a0 2023-10-06T10:38:52.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.384+0000 7f67309b3700 1 -- 172.21.15.92:0/4150885326 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671c011790 con 0x7f672c1598a0 2023-10-06T10:38:52.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.384+0000 7f67309b3700 1 -- 172.21.15.92:0/4150885326 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c011b30 con 0x7f672c1598a0 2023-10-06T10:38:52.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.384+0000 7f6733c19700 1 -- 172.21.15.92:0/4150885326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f672c1598a0 msgr2=0x7f672c0ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:52.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.384+0000 7f6733c19700 1 --2- 172.21.15.92:0/4150885326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f672c1598a0 0x7f672c0ce7e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f671c009910 tx=0x7f671c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.384+0000 7f6733c19700 1 -- 172.21.15.92:0/4150885326 shutdown_connections 2023-10-06T10:38:52.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.384+0000 7f6733c19700 1 --2- 172.21.15.92:0/4150885326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f672c1598a0 0x7f672c0ce7e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.384+0000 7f6733c19700 1 -- 172.21.15.92:0/4150885326 >> 172.21.15.92:0/4150885326 conn(0x7f672c089e50 msgr2=0x7f672c099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:52.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.385+0000 7f6733c19700 1 -- 172.21.15.92:0/4150885326 shutdown_connections 2023-10-06T10:38:52.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.385+0000 7f6733c19700 1 -- 172.21.15.92:0/4150885326 wait complete. 2023-10-06T10:38:52.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.386+0000 7f6733c19700 1 Processor -- start 2023-10-06T10:38:52.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.386+0000 7f6733c19700 1 -- start start 2023-10-06T10:38:52.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.386+0000 7f6733c19700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f672c0cbc70 0x7f672c157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:52.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.386+0000 7f6733c19700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f672c1594d0 0x7f672c149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:52.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.386+0000 7f6733c19700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f672c1598a0 0x7f672c16e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:52.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.386+0000 7f6733c19700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c14a490 con 0x7f672c1598a0 2023-10-06T10:38:52.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.386+0000 7f6733c19700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c16aaf0 con 0x7f672c0cbc70 2023-10-06T10:38:52.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.386+0000 7f6733c19700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672c16add0 con 0x7f672c1594d0 2023-10-06T10:38:52.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.386+0000 7f67321b6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f672c1598a0 0x7f672c16e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:52.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.386+0000 7f67321b6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f672c1598a0 0x7f672c16e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38660/0 (socket says 172.21.15.92:38660) 2023-10-06T10:38:52.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.386+0000 7f67321b6700 1 -- 172.21.15.92:0/219092638 learned_addr learned my addr 172.21.15.92:0/219092638 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:52.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.386+0000 7f67319b5700 1 --2- 172.21.15.92:0/219092638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f672c0cbc70 0x7f672c157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:52.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.386+0000 7f67311b4700 1 --2- 172.21.15.92:0/219092638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f672c1594d0 0x7f672c149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:52.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.387+0000 7f67321b6700 1 -- 172.21.15.92:0/219092638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f672c1594d0 msgr2=0x7f672c149f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:52.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.387+0000 7f67321b6700 1 --2- 172.21.15.92:0/219092638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f672c1594d0 0x7f672c149f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.387+0000 7f67321b6700 1 -- 172.21.15.92:0/219092638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f672c0cbc70 msgr2=0x7f672c157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:52.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.387+0000 7f67321b6700 1 --2- 172.21.15.92:0/219092638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f672c0cbc70 0x7f672c157600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.387+0000 7f67321b6700 1 -- 172.21.15.92:0/219092638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671c009420 con 0x7f672c1598a0 2023-10-06T10:38:52.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.387+0000 7f67321b6700 1 --2- 172.21.15.92:0/219092638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f672c1598a0 0x7f672c16e110 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6724001460 tx=0x7f6724001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:52.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.387+0000 7f6721ffb700 1 -- 172.21.15.92:0/219092638 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672400a7e0 con 0x7f672c1598a0 2023-10-06T10:38:52.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.387+0000 7f6721ffb700 1 -- 172.21.15.92:0/219092638 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672400a980 con 0x7f672c1598a0 2023-10-06T10:38:52.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.387+0000 7f6733c19700 1 -- 172.21.15.92:0/219092638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f672c16d910 con 0x7f672c1598a0 2023-10-06T10:38:52.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.387+0000 7f6721ffb700 1 -- 172.21.15.92:0/219092638 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672400ad40 con 0x7f672c1598a0 2023-10-06T10:38:52.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.387+0000 7f6733c19700 1 -- 172.21.15.92:0/219092638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672c16e650 con 0x7f672c1598a0 2023-10-06T10:38:52.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.388+0000 7f6733c19700 1 -- 172.21.15.92:0/219092638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f672c064a90 con 0x7f672c1598a0 2023-10-06T10:38:52.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.389+0000 7f6721ffb700 1 -- 172.21.15.92:0/219092638 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6724007300 con 0x7f672c1598a0 2023-10-06T10:38:52.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.389+0000 7f6721ffb700 1 --2- 172.21.15.92:0/219092638 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6710066f30 0x7f67100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:52.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.389+0000 7f6721ffb700 1 -- 172.21.15.92:0/219092638 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f6724084bd0 con 0x7f672c1598a0 2023-10-06T10:38:52.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.389+0000 7f67319b5700 1 --2- 172.21.15.92:0/219092638 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6710066f30 0x7f67100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:52.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.390+0000 7f67319b5700 1 --2- 172.21.15.92:0/219092638 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6710066f30 0x7f67100693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f672c15b350 tx=0x7f6728007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:52.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.393+0000 7f6721ffb700 1 -- 172.21.15.92:0/219092638 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6724051a70 con 0x7f672c1598a0 2023-10-06T10:38:52.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.576+0000 7f6733c19700 1 -- 172.21.15.92:0/219092638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f672c0c66a0 con 0x7f672c1598a0 2023-10-06T10:38:52.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.576+0000 7f6721ffb700 1 -- 172.21.15.92:0/219092638 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6724055090 con 0x7f672c1598a0 2023-10-06T10:38:52.578 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T10:38:52.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.578+0000 7f6733c19700 1 -- 172.21.15.92:0/219092638 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6710066f30 msgr2=0x7f67100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:52.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.578+0000 7f6733c19700 1 --2- 172.21.15.92:0/219092638 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6710066f30 0x7f67100693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f672c15b350 tx=0x7f6728007d50 comp rx=0 tx=0).stop 2023-10-06T10:38:52.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.578+0000 7f6733c19700 1 -- 172.21.15.92:0/219092638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f672c1598a0 msgr2=0x7f672c16e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:52.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.578+0000 7f6733c19700 1 --2- 172.21.15.92:0/219092638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f672c1598a0 0x7f672c16e110 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6724001460 tx=0x7f6724001a00 comp rx=0 tx=0).stop 2023-10-06T10:38:52.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.578+0000 7f6733c19700 1 -- 172.21.15.92:0/219092638 shutdown_connections 2023-10-06T10:38:52.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.578+0000 7f6733c19700 1 --2- 172.21.15.92:0/219092638 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6710066f30 0x7f67100693e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.578+0000 7f6733c19700 1 --2- 172.21.15.92:0/219092638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f672c1598a0 0x7f672c16e110 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.578+0000 7f6733c19700 1 --2- 172.21.15.92:0/219092638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f672c0cbc70 0x7f672c157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.578+0000 7f6733c19700 1 --2- 172.21.15.92:0/219092638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f672c1594d0 0x7f672c149f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.578+0000 7f6733c19700 1 -- 172.21.15.92:0/219092638 >> 172.21.15.92:0/219092638 conn(0x7f672c089e50 msgr2=0x7f672c155fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:52.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.578+0000 7f6733c19700 1 -- 172.21.15.92:0/219092638 shutdown_connections 2023-10-06T10:38:52.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.578+0000 7f6733c19700 1 -- 172.21.15.92:0/219092638 wait complete. 2023-10-06T10:38:52.592 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T10:38:52.664 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:38:52.666 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:38:52.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.709+0000 7fb66153a700 1 Processor -- start 2023-10-06T10:38:52.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.709+0000 7fb66153a700 1 -- start start 2023-10-06T10:38:52.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.710+0000 7fb66153a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb65c0cce20 0x7fb65c0c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:52.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.710+0000 7fb66153a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb65c0c93b0 0x7fb65c0c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:52.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.710+0000 7fb66153a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb65c0cadd0 con 0x7fb65c0b05d0 2023-10-06T10:38:52.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.710+0000 7fb66153a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb65c1577d0 con 0x7fb65c0cce20 2023-10-06T10:38:52.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.710+0000 7fb66153a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c0c9ae0 con 0x7fb65c0c93b0 2023-10-06T10:38:52.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.710+0000 7fb65b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb65c0cce20 0x7fb65c0c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:52.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.710+0000 7fb65bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb65c0b05d0 0x7fb65c0cc710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45088/0 (socket says 172.21.15.92:45088) 2023-10-06T10:38:52.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.710+0000 7fb65bfff700 1 -- 172.21.15.92:0/4235062932 learned_addr learned my addr 172.21.15.92:0/4235062932 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:52.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.710+0000 7fb660d39700 1 --2- 172.21.15.92:0/4235062932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb65c0c93b0 0x7fb65c0c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:52.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.710+0000 7fb65b7fe700 1 -- 172.21.15.92:0/4235062932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb65c0c93b0 msgr2=0x7fb65c0c2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:52.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.710+0000 7fb65b7fe700 1 --2- 172.21.15.92:0/4235062932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb65c0c93b0 0x7fb65c0c2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.710+0000 7fb65b7fe700 1 -- 172.21.15.92:0/4235062932 >> v1:172.21.15.92:6789/0 conn(0x7fb65c0b05d0 legacy=0x7fb65c0cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:52.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.710+0000 7fb65affd700 1 -- 172.21.15.92:0/4235062932 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3501222238 0 0) 0x7fb65c0cadd0 con 0x7fb65c0b05d0 2023-10-06T10:38:52.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.710+0000 7fb65b7fe700 1 -- 172.21.15.92:0/4235062932 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c0c9dc0 con 0x7fb65c0cce20 2023-10-06T10:38:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.711+0000 7fb65b7fe700 1 --2- 172.21.15.92:0/4235062932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0cce20 0x7fb65c0c8e70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb6500098d0 tx=0x7fb650009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4606dde3c188dcb0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.711+0000 7fb65affd700 1 -- 172.21.15.92:0/4235062932 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb650009e70 con 0x7fb65c0cce20 2023-10-06T10:38:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.711+0000 7fb65affd700 1 -- 172.21.15.92:0/4235062932 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb650005c10 con 0x7fb65c0cce20 2023-10-06T10:38:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.711+0000 7fb65affd700 1 -- 172.21.15.92:0/4235062932 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65000b650 con 0x7fb65c0cce20 2023-10-06T10:38:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.711+0000 7fb66153a700 1 -- 172.21.15.92:0/4235062932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0cce20 msgr2=0x7fb65c0c8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.711+0000 7fb66153a700 1 --2- 172.21.15.92:0/4235062932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0cce20 0x7fb65c0c8e70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb6500098d0 tx=0x7fb650009120 comp rx=0 tx=0).stop 2023-10-06T10:38:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.711+0000 7fb66153a700 1 -- 172.21.15.92:0/4235062932 shutdown_connections 2023-10-06T10:38:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.711+0000 7fb66153a700 1 --2- 172.21.15.92:0/4235062932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb65c0c93b0 0x7fb65c0c2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.711+0000 7fb66153a700 1 --2- 172.21.15.92:0/4235062932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0cce20 0x7fb65c0c8e70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.711+0000 7fb66153a700 1 -- 172.21.15.92:0/4235062932 >> 172.21.15.92:0/4235062932 conn(0x7fb65c096a30 msgr2=0x7fb65c096e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.711+0000 7fb66153a700 1 -- 172.21.15.92:0/4235062932 shutdown_connections 2023-10-06T10:38:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.711+0000 7fb66153a700 1 -- 172.21.15.92:0/4235062932 wait complete. 2023-10-06T10:38:52.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.712+0000 7fb66153a700 1 Processor -- start 2023-10-06T10:38:52.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.713+0000 7fb66153a700 1 -- start start 2023-10-06T10:38:52.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.713+0000 7fb66153a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0b05d0 0x7fb65c0c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:52.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.713+0000 7fb66153a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb65c0c93b0 0x7fb65c154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:52.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.713+0000 7fb66153a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb65c0cce20 0x7fb65c0b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:52.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.713+0000 7fb66153a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c0c7e20 con 0x7fb65c0b05d0 2023-10-06T10:38:52.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.713+0000 7fb66153a700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c0c72a0 con 0x7fb65c0c93b0 2023-10-06T10:38:52.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.713+0000 7fb66153a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c0c7580 con 0x7fb65c0cce20 2023-10-06T10:38:52.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.713+0000 7fb65bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0b05d0 0x7fb65c0c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:52.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.713+0000 7fb65bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0b05d0 0x7fb65c0c6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38680/0 (socket says 172.21.15.92:38680) 2023-10-06T10:38:52.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.713+0000 7fb65bfff700 1 -- 172.21.15.92:0/1967142841 learned_addr learned my addr 172.21.15.92:0/1967142841 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:52.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.713+0000 7fb65b7fe700 1 --2- 172.21.15.92:0/1967142841 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb65c0c93b0 0x7fb65c154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:52.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.713+0000 7fb660d39700 1 --2- 172.21.15.92:0/1967142841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb65c0cce20 0x7fb65c0b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:52.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.714+0000 7fb65bfff700 1 -- 172.21.15.92:0/1967142841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb65c0cce20 msgr2=0x7fb65c0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:52.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.714+0000 7fb65bfff700 1 --2- 172.21.15.92:0/1967142841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb65c0cce20 0x7fb65c0b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.714+0000 7fb65bfff700 1 -- 172.21.15.92:0/1967142841 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb65c0c93b0 msgr2=0x7fb65c154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:52.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.714+0000 7fb65bfff700 1 --2- 172.21.15.92:0/1967142841 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb65c0c93b0 0x7fb65c154630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.714+0000 7fb65bfff700 1 -- 172.21.15.92:0/1967142841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6500093c0 con 0x7fb65c0b05d0 2023-10-06T10:38:52.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.714+0000 7fb660d39700 1 --2- 172.21.15.92:0/1967142841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb65c0cce20 0x7fb65c0b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:52.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.714+0000 7fb65b7fe700 1 --2- 172.21.15.92:0/1967142841 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb65c0c93b0 0x7fb65c154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:52.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.714+0000 7fb65bfff700 1 --2- 172.21.15.92:0/1967142841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0b05d0 0x7fb65c0c6a00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00c7e0 tx=0x7fb64c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:52.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.714+0000 7fb643fff700 1 -- 172.21.15.92:0/1967142841 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c005b30 con 0x7fb65c0b05d0 2023-10-06T10:38:52.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.714+0000 7fb643fff700 1 -- 172.21.15.92:0/1967142841 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64c005c90 con 0x7fb65c0b05d0 2023-10-06T10:38:52.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.714+0000 7fb66153a700 1 -- 172.21.15.92:0/1967142841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65c0c7860 con 0x7fb65c0b05d0 2023-10-06T10:38:52.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.714+0000 7fb643fff700 1 -- 172.21.15.92:0/1967142841 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c007300 con 0x7fb65c0b05d0 2023-10-06T10:38:52.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.714+0000 7fb66153a700 1 -- 172.21.15.92:0/1967142841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65c098d00 con 0x7fb65c0b05d0 2023-10-06T10:38:52.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.715+0000 7fb66153a700 1 -- 172.21.15.92:0/1967142841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65c0c7000 con 0x7fb65c0b05d0 2023-10-06T10:38:52.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.715+0000 7fb643fff700 1 -- 172.21.15.92:0/1967142841 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb64c01b400 con 0x7fb65c0b05d0 2023-10-06T10:38:52.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.715+0000 7fb643fff700 1 --2- 172.21.15.92:0/1967142841 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb648066f30 0x7fb6480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:52.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.716+0000 7fb643fff700 1 -- 172.21.15.92:0/1967142841 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb64c084480 con 0x7fb65c0b05d0 2023-10-06T10:38:52.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.716+0000 7fb65b7fe700 1 --2- 172.21.15.92:0/1967142841 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb648066f30 0x7fb6480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:52.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.716+0000 7fb65b7fe700 1 --2- 172.21.15.92:0/1967142841 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb648066f30 0x7fb6480693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0cd220 tx=0x7fb650016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:52.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.720+0000 7fb643fff700 1 -- 172.21.15.92:0/1967142841 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb64c051320 con 0x7fb65c0b05d0 2023-10-06T10:38:52.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.899+0000 7fb66153a700 1 -- 172.21.15.92:0/1967142841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb65c05f230 con 0x7fb65c0b05d0 2023-10-06T10:38:52.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.900+0000 7fb643fff700 1 -- 172.21.15.92:0/1967142841 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb64c054940 con 0x7fb65c0b05d0 2023-10-06T10:38:52.902 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T10:38:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.902+0000 7fb66153a700 1 -- 172.21.15.92:0/1967142841 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb648066f30 msgr2=0x7fb6480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.902+0000 7fb66153a700 1 --2- 172.21.15.92:0/1967142841 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb648066f30 0x7fb6480693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0cd220 tx=0x7fb650016000 comp rx=0 tx=0).stop 2023-10-06T10:38:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.902+0000 7fb66153a700 1 -- 172.21.15.92:0/1967142841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0b05d0 msgr2=0x7fb65c0c6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.902+0000 7fb66153a700 1 --2- 172.21.15.92:0/1967142841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0b05d0 0x7fb65c0c6a00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00c7e0 tx=0x7fb64c007df0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.902+0000 7fb66153a700 1 -- 172.21.15.92:0/1967142841 shutdown_connections 2023-10-06T10:38:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.902+0000 7fb66153a700 1 --2- 172.21.15.92:0/1967142841 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb648066f30 0x7fb6480693e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.902+0000 7fb66153a700 1 --2- 172.21.15.92:0/1967142841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb65c0cce20 0x7fb65c0b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.902+0000 7fb66153a700 1 --2- 172.21.15.92:0/1967142841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0b05d0 0x7fb65c0c6a00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.902+0000 7fb66153a700 1 --2- 172.21.15.92:0/1967142841 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb65c0c93b0 0x7fb65c154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:52.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.902+0000 7fb66153a700 1 -- 172.21.15.92:0/1967142841 >> 172.21.15.92:0/1967142841 conn(0x7fb65c096a30 msgr2=0x7fb65c096e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:52.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.902+0000 7fb66153a700 1 -- 172.21.15.92:0/1967142841 shutdown_connections 2023-10-06T10:38:52.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:52.902+0000 7fb66153a700 1 -- 172.21.15.92:0/1967142841 wait complete. 2023-10-06T10:38:52.916 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T10:38:53.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.032+0000 7f325a87b700 1 Processor -- start 2023-10-06T10:38:53.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.032+0000 7f325a87b700 1 -- start start 2023-10-06T10:38:53.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.033+0000 7f325a87b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f32540b0640 0x7f32540cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:53.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.033+0000 7f325a87b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32540cca70 con 0x7f32540b0a10 2023-10-06T10:38:53.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.033+0000 7f325a87b700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32540c53c0 con 0x7f32540cc5f0 2023-10-06T10:38:53.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.033+0000 7f325a87b700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32540ca790 con 0x7f32540b0640 2023-10-06T10:38:53.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.033+0000 7f32537fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f32540b0a10 0x7f32540cbee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45092/0 (socket says 172.21.15.92:45092) 2023-10-06T10:38:53.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.033+0000 7f32537fe700 1 -- 172.21.15.92:0/3240322704 learned_addr learned my addr 172.21.15.92:0/3240322704 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:53.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3253fff700 1 --2- 172.21.15.92:0/3240322704 >> v2:172.21.15.124:3300/0 conn(0x7f32540b0640 0x7f32540cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:53.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3252ffd700 1 -- 172.21.15.92:0/3240322704 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058271578 0 0) 0x7f32540cca70 con 0x7f32540b0a10 2023-10-06T10:38:53.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3252ffd700 1 -- 172.21.15.92:0/3240322704 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f323c003570 con 0x7f32540b0a10 2023-10-06T10:38:53.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3252ffd700 1 -- 172.21.15.92:0/3240322704 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3163204511 0 0) 0x7f323c003570 con 0x7f32540b0a10 2023-10-06T10:38:53.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3252ffd700 1 -- 172.21.15.92:0/3240322704 >> v2:172.21.15.124:3300/0 conn(0x7f32540b0640 msgr2=0x7f32540cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:53.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3252ffd700 1 --2- 172.21.15.92:0/3240322704 >> v2:172.21.15.124:3300/0 conn(0x7f32540b0640 0x7f32540cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3252ffd700 1 -- 172.21.15.92:0/3240322704 >> v1:172.21.15.124:6789/0 conn(0x7f32540cc5f0 legacy=0x7f32540c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:53.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3252ffd700 1 -- 172.21.15.92:0/3240322704 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32540ccd00 con 0x7f32540b0a10 2023-10-06T10:38:53.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3252ffd700 1 -- 172.21.15.92:0/3240322704 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f3248001470 con 0x7f32540b0a10 2023-10-06T10:38:53.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3252ffd700 1 -- 172.21.15.92:0/3240322704 >> v1:172.21.15.92:6789/0 conn(0x7f32540b0a10 legacy=0x7f32540cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:53.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3252ffd700 1 --2- 172.21.15.92:0/3240322704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c003a60 0x7f323c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:53.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3252ffd700 1 --2- 172.21.15.92:0/3240322704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f323c004810 0x7f323c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:53.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3252ffd700 1 --2- 172.21.15.92:0/3240322704 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f323c0051c0 0x7f323c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:53.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3252ffd700 1 -- 172.21.15.92:0/3240322704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32540ca790 con 0x7f323c003a60 2023-10-06T10:38:53.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3252ffd700 1 -- 172.21.15.92:0/3240322704 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c00c0e0 con 0x7f323c0051c0 2023-10-06T10:38:53.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3252ffd700 1 -- 172.21.15.92:0/3240322704 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323c00c2a0 con 0x7f323c004810 2023-10-06T10:38:53.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3253fff700 1 --2- 172.21.15.92:0/3240322704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c003a60 0x7f323c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:53.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f32537fe700 1 --2- 172.21.15.92:0/3240322704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f323c004810 0x7f323c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:53.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3258e18700 1 --2- 172.21.15.92:0/3240322704 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f323c0051c0 0x7f323c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:53.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3253fff700 1 -- 172.21.15.92:0/3240322704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f323c004810 msgr2=0x7f323c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:53.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3253fff700 1 --2- 172.21.15.92:0/3240322704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f323c004810 0x7f323c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3253fff700 1 -- 172.21.15.92:0/3240322704 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f323c0051c0 msgr2=0x7f323c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:53.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3253fff700 1 --2- 172.21.15.92:0/3240322704 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f323c0051c0 0x7f323c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3253fff700 1 -- 172.21.15.92:0/3240322704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c003570 con 0x7f323c003a60 2023-10-06T10:38:53.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3258e18700 1 --2- 172.21.15.92:0/3240322704 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f323c0051c0 0x7f323c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:53.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3258e18700 1 -- 172.21.15.92:0/3240322704 reap_dead start 2023-10-06T10:38:53.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.034+0000 7f3253fff700 1 --2- 172.21.15.92:0/3240322704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c003a60 0x7f323c003e30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3244009bf0 tx=0x7f3244009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4645f2581cb3d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:53.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.035+0000 7f3252ffd700 1 -- 172.21.15.92:0/3240322704 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3244005a00 con 0x7f323c003a60 2023-10-06T10:38:53.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.035+0000 7f3252ffd700 1 -- 172.21.15.92:0/3240322704 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3244005b60 con 0x7f323c003a60 2023-10-06T10:38:53.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.035+0000 7f3252ffd700 1 -- 172.21.15.92:0/3240322704 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324400e710 con 0x7f323c003a60 2023-10-06T10:38:53.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.035+0000 7f325a87b700 1 -- 172.21.15.92:0/3240322704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c003a60 msgr2=0x7f323c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:53.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.035+0000 7f325a87b700 1 --2- 172.21.15.92:0/3240322704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c003a60 0x7f323c003e30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3244009bf0 tx=0x7f3244009120 comp rx=0 tx=0).stop 2023-10-06T10:38:53.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.035+0000 7f325a87b700 1 -- 172.21.15.92:0/3240322704 shutdown_connections 2023-10-06T10:38:53.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.035+0000 7f325a87b700 1 --2- 172.21.15.92:0/3240322704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c003a60 0x7f323c003e30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.035+0000 7f325a87b700 1 -- 172.21.15.92:0/3240322704 >> 172.21.15.92:0/3240322704 conn(0x7f3254087fc0 msgr2=0x7f32540ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:53.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.035+0000 7f325a87b700 1 -- 172.21.15.92:0/3240322704 shutdown_connections 2023-10-06T10:38:53.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.035+0000 7f325a87b700 1 -- 172.21.15.92:0/3240322704 wait complete. 2023-10-06T10:38:53.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.036+0000 7f325a87b700 1 Processor -- start 2023-10-06T10:38:53.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.037+0000 7f325a87b700 1 -- start start 2023-10-06T10:38:53.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.037+0000 7f325a87b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f323c003a60 0x7f325416ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:53.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.037+0000 7f325a87b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3254155240 0x7f325415c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:53.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.037+0000 7f325a87b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3254169f30 0x7f325416a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:53.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.037+0000 7f325a87b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3254169820 con 0x7f3254169f30 2023-10-06T10:38:53.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.037+0000 7f325a87b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3254169aa0 con 0x7f323c003a60 2023-10-06T10:38:53.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.037+0000 7f325a87b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32540b3f00 con 0x7f3254155240 2023-10-06T10:38:53.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.037+0000 7f3258e18700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3254169f30 0x7f325416a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:53.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.037+0000 7f3258e18700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3254169f30 0x7f325416a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38688/0 (socket says 172.21.15.92:38688) 2023-10-06T10:38:53.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.037+0000 7f3258e18700 1 -- 172.21.15.92:0/2455865094 learned_addr learned my addr 172.21.15.92:0/2455865094 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:53.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.037+0000 7f3253fff700 1 --2- 172.21.15.92:0/2455865094 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f323c003a60 0x7f325416ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:53.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.037+0000 7f32537fe700 1 --2- 172.21.15.92:0/2455865094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3254155240 0x7f325415c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:53.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.038+0000 7f3258e18700 1 -- 172.21.15.92:0/2455865094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3254155240 msgr2=0x7f325415c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:53.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.038+0000 7f3258e18700 1 --2- 172.21.15.92:0/2455865094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3254155240 0x7f325415c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.038+0000 7f3258e18700 1 -- 172.21.15.92:0/2455865094 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f323c003a60 msgr2=0x7f325416ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:53.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.038+0000 7f3258e18700 1 --2- 172.21.15.92:0/2455865094 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f323c003a60 0x7f325416ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.038+0000 7f3258e18700 1 -- 172.21.15.92:0/2455865094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325416d500 con 0x7f3254169f30 2023-10-06T10:38:53.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.038+0000 7f3258e18700 1 --2- 172.21.15.92:0/2455865094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3254169f30 0x7f325416a3b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f325406ed10 tx=0x7f324c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:53.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.038+0000 7f323bfff700 1 -- 172.21.15.92:0/2455865094 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324c012660 con 0x7f3254169f30 2023-10-06T10:38:53.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.038+0000 7f323bfff700 1 -- 172.21.15.92:0/2455865094 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324c0127e0 con 0x7f3254169f30 2023-10-06T10:38:53.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.038+0000 7f325a87b700 1 -- 172.21.15.92:0/2455865094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f325416d7e0 con 0x7f3254169f30 2023-10-06T10:38:53.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.038+0000 7f323bfff700 1 -- 172.21.15.92:0/2455865094 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324c012ba0 con 0x7f3254169f30 2023-10-06T10:38:53.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.038+0000 7f325a87b700 1 -- 172.21.15.92:0/2455865094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325415ca50 con 0x7f3254169f30 2023-10-06T10:38:53.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.039+0000 7f325a87b700 1 -- 172.21.15.92:0/2455865094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3254169c50 con 0x7f3254169f30 2023-10-06T10:38:53.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.039+0000 7f323bfff700 1 -- 172.21.15.92:0/2455865094 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f324c003030 con 0x7f3254169f30 2023-10-06T10:38:53.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.040+0000 7f323bfff700 1 --2- 172.21.15.92:0/2455865094 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3240066f30 0x7f32400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:53.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.040+0000 7f323bfff700 1 -- 172.21.15.92:0/2455865094 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f324c084650 con 0x7f3254169f30 2023-10-06T10:38:53.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.040+0000 7f3253fff700 1 --2- 172.21.15.92:0/2455865094 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3240066f30 0x7f32400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:53.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.040+0000 7f3253fff700 1 --2- 172.21.15.92:0/2455865094 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3240066f30 0x7f32400693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3244009160 tx=0x7f3244019000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:53.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.044+0000 7f323bfff700 1 -- 172.21.15.92:0/2455865094 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3254169c50 con 0x7f3254169f30 2023-10-06T10:38:53.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.228+0000 7f325a87b700 1 -- 172.21.15.92:0/2455865094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3254169c50 con 0x7f3254169f30 2023-10-06T10:38:53.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.228+0000 7f323bfff700 1 -- 172.21.15.92:0/2455865094 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3254169c50 con 0x7f3254169f30 2023-10-06T10:38:53.230 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T10:38:53.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.230+0000 7f325a87b700 1 -- 172.21.15.92:0/2455865094 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3240066f30 msgr2=0x7f32400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:53.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.230+0000 7f325a87b700 1 --2- 172.21.15.92:0/2455865094 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3240066f30 0x7f32400693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3244009160 tx=0x7f3244019000 comp rx=0 tx=0).stop 2023-10-06T10:38:53.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.230+0000 7f325a87b700 1 -- 172.21.15.92:0/2455865094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3254169f30 msgr2=0x7f325416a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:53.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.230+0000 7f325a87b700 1 --2- 172.21.15.92:0/2455865094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3254169f30 0x7f325416a3b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f325406ed10 tx=0x7f324c009040 comp rx=0 tx=0).stop 2023-10-06T10:38:53.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.230+0000 7f325a87b700 1 -- 172.21.15.92:0/2455865094 shutdown_connections 2023-10-06T10:38:53.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.230+0000 7f325a87b700 1 --2- 172.21.15.92:0/2455865094 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3240066f30 0x7f32400693e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.230+0000 7f325a87b700 1 --2- 172.21.15.92:0/2455865094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3254169f30 0x7f325416a3b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.230+0000 7f325a87b700 1 --2- 172.21.15.92:0/2455865094 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f323c003a60 0x7f325416ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.230+0000 7f325a87b700 1 --2- 172.21.15.92:0/2455865094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3254155240 0x7f325415c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.230+0000 7f325a87b700 1 -- 172.21.15.92:0/2455865094 >> 172.21.15.92:0/2455865094 conn(0x7f3254087fc0 msgr2=0x7f3254155ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:53.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.230+0000 7f325a87b700 1 -- 172.21.15.92:0/2455865094 shutdown_connections 2023-10-06T10:38:53.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.230+0000 7f325a87b700 1 -- 172.21.15.92:0/2455865094 wait complete. 2023-10-06T10:38:53.244 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:38:53.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.356+0000 7f9a66959700 1 Processor -- start 2023-10-06T10:38:53.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.356+0000 7f9a66959700 1 -- start start 2023-10-06T10:38:53.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.357+0000 7f9a66959700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9a60156500 0x7f9a600c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:53.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.357+0000 7f9a66959700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a600c7570 0x7f9a6006e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:53.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.357+0000 7f9a66959700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a600acd20 con 0x7f9a6006eb90 2023-10-06T10:38:53.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.357+0000 7f9a66959700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a6006f430 con 0x7f9a60156500 2023-10-06T10:38:53.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.357+0000 7f9a66959700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6006f010 con 0x7f9a600c7570 2023-10-06T10:38:53.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.357+0000 7f9a5ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9a60156500 0x7f9a600c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:53.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.357+0000 7f9a5ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9a60156500 0x7f9a600c7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46976/0 (socket says 172.21.15.92:46976) 2023-10-06T10:38:53.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.357+0000 7f9a5ffff700 1 -- 172.21.15.92:0/1202972996 learned_addr learned my addr 172.21.15.92:0/1202972996 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:53.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.358+0000 7f9a5f7fe700 1 --2- 172.21.15.92:0/1202972996 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a600c7570 0x7f9a6006e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:53.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.358+0000 7f9a5ffff700 1 -- 172.21.15.92:0/1202972996 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a600c7570 msgr2=0x7f9a6006e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:53.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.358+0000 7f9a5ffff700 1 --2- 172.21.15.92:0/1202972996 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a600c7570 0x7f9a6006e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.358+0000 7f9a5ffff700 1 -- 172.21.15.92:0/1202972996 >> v1:172.21.15.124:6789/0 conn(0x7f9a6006eb90 legacy=0x7f9a6006d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:53.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.358+0000 7f9a5ffff700 1 -- 172.21.15.92:0/1202972996 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a601568d0 con 0x7f9a60156500 2023-10-06T10:38:53.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.358+0000 7f9a5ffff700 1 --2- 172.21.15.92:0/1202972996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a60156500 0x7f9a600c7030 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9a500099e0 tx=0x7f9a50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbdf06bc6eed35e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:53.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.358+0000 7f9a5effd700 1 -- 172.21.15.92:0/1202972996 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5000b820 con 0x7f9a60156500 2023-10-06T10:38:53.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.358+0000 7f9a5effd700 1 -- 172.21.15.92:0/1202972996 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5000b980 con 0x7f9a60156500 2023-10-06T10:38:53.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.358+0000 7f9a5effd700 1 -- 172.21.15.92:0/1202972996 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5000bd20 con 0x7f9a60156500 2023-10-06T10:38:53.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.359+0000 7f9a66959700 1 -- 172.21.15.92:0/1202972996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a60156500 msgr2=0x7f9a600c7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:53.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.359+0000 7f9a66959700 1 --2- 172.21.15.92:0/1202972996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a60156500 0x7f9a600c7030 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9a500099e0 tx=0x7f9a50009250 comp rx=0 tx=0).stop 2023-10-06T10:38:53.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.359+0000 7f9a66959700 1 -- 172.21.15.92:0/1202972996 shutdown_connections 2023-10-06T10:38:53.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.359+0000 7f9a66959700 1 --2- 172.21.15.92:0/1202972996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a60156500 0x7f9a600c7030 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.359+0000 7f9a66959700 1 --2- 172.21.15.92:0/1202972996 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a600c7570 0x7f9a6006e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.359+0000 7f9a66959700 1 -- 172.21.15.92:0/1202972996 >> 172.21.15.92:0/1202972996 conn(0x7f9a60068080 msgr2=0x7f9a600cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:53.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.359+0000 7f9a66959700 1 -- 172.21.15.92:0/1202972996 shutdown_connections 2023-10-06T10:38:53.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.359+0000 7f9a66959700 1 -- 172.21.15.92:0/1202972996 wait complete. 2023-10-06T10:38:53.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.360+0000 7f9a66959700 1 Processor -- start 2023-10-06T10:38:53.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.360+0000 7f9a66959700 1 -- start start 2023-10-06T10:38:53.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.361+0000 7f9a66959700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6006eb90 0x7f9a60154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:53.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.361+0000 7f9a66959700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a600c7570 0x7f9a600c21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:53.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.361+0000 7f9a66959700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a60156500 0x7f9a600aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:53.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.361+0000 7f9a66959700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a60158f30 con 0x7f9a6006eb90 2023-10-06T10:38:53.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.361+0000 7f9a66959700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a60159210 con 0x7f9a60156500 2023-10-06T10:38:53.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.361+0000 7f9a66959700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a601594f0 con 0x7f9a600c7570 2023-10-06T10:38:53.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.361+0000 7f9a5ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6006eb90 0x7f9a60154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:53.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.361+0000 7f9a5ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6006eb90 0x7f9a60154790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46978/0 (socket says 172.21.15.92:46978) 2023-10-06T10:38:53.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.361+0000 7f9a5ffff700 1 -- 172.21.15.92:0/883979925 learned_addr learned my addr 172.21.15.92:0/883979925 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:53.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.361+0000 7f9a5f7fe700 1 --2- 172.21.15.92:0/883979925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a600c7570 0x7f9a600c21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:53.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.361+0000 7f9a64ef6700 1 --2- 172.21.15.92:0/883979925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a60156500 0x7f9a600aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:53.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.361+0000 7f9a5ffff700 1 -- 172.21.15.92:0/883979925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a600c7570 msgr2=0x7f9a600c21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:53.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.361+0000 7f9a5ffff700 1 --2- 172.21.15.92:0/883979925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a600c7570 0x7f9a600c21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.361+0000 7f9a5ffff700 1 -- 172.21.15.92:0/883979925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a60156500 msgr2=0x7f9a600aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:53.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.361+0000 7f9a5ffff700 1 --2- 172.21.15.92:0/883979925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a60156500 0x7f9a600aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.361+0000 7f9a5ffff700 1 -- 172.21.15.92:0/883979925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a500094f0 con 0x7f9a6006eb90 2023-10-06T10:38:53.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.361+0000 7f9a5ffff700 1 --2- 172.21.15.92:0/883979925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6006eb90 0x7f9a60154790 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009290 tx=0x7f9a50017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:53.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.361+0000 7f9a47fff700 1 -- 172.21.15.92:0/883979925 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a50024440 con 0x7f9a6006eb90 2023-10-06T10:38:53.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.361+0000 7f9a47fff700 1 -- 172.21.15.92:0/883979925 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a500245a0 con 0x7f9a6006eb90 2023-10-06T10:38:53.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.361+0000 7f9a66959700 1 -- 172.21.15.92:0/883979925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a601597d0 con 0x7f9a6006eb90 2023-10-06T10:38:53.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.361+0000 7f9a47fff700 1 -- 172.21.15.92:0/883979925 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a50024960 con 0x7f9a6006eb90 2023-10-06T10:38:53.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.361+0000 7f9a66959700 1 -- 172.21.15.92:0/883979925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a60154cd0 con 0x7f9a6006eb90 2023-10-06T10:38:53.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.362+0000 7f9a66959700 1 -- 172.21.15.92:0/883979925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a60159cf0 con 0x7f9a6006eb90 2023-10-06T10:38:53.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.363+0000 7f9a47fff700 1 -- 172.21.15.92:0/883979925 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9a5001a420 con 0x7f9a6006eb90 2023-10-06T10:38:53.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.363+0000 7f9a47fff700 1 --2- 172.21.15.92:0/883979925 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a4c066f30 0x7f9a4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:53.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.363+0000 7f9a47fff700 1 -- 172.21.15.92:0/883979925 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9a50086880 con 0x7f9a6006eb90 2023-10-06T10:38:53.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.363+0000 7f9a5f7fe700 1 --2- 172.21.15.92:0/883979925 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a4c066f30 0x7f9a4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:53.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.363+0000 7f9a5f7fe700 1 --2- 172.21.15.92:0/883979925 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a4c066f30 0x7f9a4c0693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9a60155720 tx=0x7f9a54008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:53.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.367+0000 7f9a47fff700 1 -- 172.21.15.92:0/883979925 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a60159cf0 con 0x7f9a6006eb90 2023-10-06T10:38:53.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.556+0000 7f9a66959700 1 -- 172.21.15.92:0/883979925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a60064a90 con 0x7f9a6006eb90 2023-10-06T10:38:53.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.557+0000 7f9a47fff700 1 -- 172.21.15.92:0/883979925 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18707 (secure 0 0 0) 0x7f9a500536a0 con 0x7f9a6006eb90 2023-10-06T10:38:53.559 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:53.559 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:21.760233+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:38:14.368713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T10:38:15.692344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T10:38:19.382553+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:38:53.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.559+0000 7f9a66959700 1 -- 172.21.15.92:0/883979925 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a4c066f30 msgr2=0x7f9a4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:53.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.559+0000 7f9a66959700 1 --2- 172.21.15.92:0/883979925 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a4c066f30 0x7f9a4c0693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9a60155720 tx=0x7f9a54008040 comp rx=0 tx=0).stop 2023-10-06T10:38:53.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.559+0000 7f9a66959700 1 -- 172.21.15.92:0/883979925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6006eb90 msgr2=0x7f9a60154790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:53.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.559+0000 7f9a66959700 1 --2- 172.21.15.92:0/883979925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6006eb90 0x7f9a60154790 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009290 tx=0x7f9a50017000 comp rx=0 tx=0).stop 2023-10-06T10:38:53.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.559+0000 7f9a66959700 1 -- 172.21.15.92:0/883979925 shutdown_connections 2023-10-06T10:38:53.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.559+0000 7f9a66959700 1 --2- 172.21.15.92:0/883979925 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a4c066f30 0x7f9a4c0693e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.559+0000 7f9a66959700 1 --2- 172.21.15.92:0/883979925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a60156500 0x7f9a600aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.559+0000 7f9a66959700 1 --2- 172.21.15.92:0/883979925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6006eb90 0x7f9a60154790 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.559+0000 7f9a66959700 1 --2- 172.21.15.92:0/883979925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a600c7570 0x7f9a600c21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.559+0000 7f9a66959700 1 -- 172.21.15.92:0/883979925 >> 172.21.15.92:0/883979925 conn(0x7f9a60068080 msgr2=0x7f9a6006dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:53.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.559+0000 7f9a66959700 1 -- 172.21.15.92:0/883979925 shutdown_connections 2023-10-06T10:38:53.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.559+0000 7f9a66959700 1 -- 172.21.15.92:0/883979925 wait complete. 2023-10-06T10:38:53.573 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T10:38:09.680559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T10:38:10.120786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T10:38:14.368713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T10:38:15.692344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T10:38:19.382553+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T10:38:53.574 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T10:38:53.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.691+0000 7f6c4f56a700 1 Processor -- start 2023-10-06T10:38:53.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.691+0000 7f6c4f56a700 1 -- start start 2023-10-06T10:38:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.692+0000 7f6c4f56a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c48067cc0 0x7f6c4806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.692+0000 7f6c4f56a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c480cbf70 con 0x7f6c480cc820 2023-10-06T10:38:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.692+0000 7f6c4f56a700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c48159db0 con 0x7f6c480ccbf0 2023-10-06T10:38:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.692+0000 7f6c4f56a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c48159bc0 con 0x7f6c48067cc0 2023-10-06T10:38:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.692+0000 7f6c4d306700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6c480cc820 0x7f6c480c9220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50284/0 (socket says 172.21.15.92:50284) 2023-10-06T10:38:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.692+0000 7f6c4d306700 1 -- 172.21.15.92:0/2032633143 learned_addr learned my addr 172.21.15.92:0/2032633143 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c3ffff700 1 -- 172.21.15.92:0/2032633143 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3696554805 0 0) 0x7f6c480cbf70 con 0x7f6c480cc820 2023-10-06T10:38:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c3ffff700 1 -- 172.21.15.92:0/2032633143 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c34003570 con 0x7f6c480cc820 2023-10-06T10:38:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c4db07700 1 --2- 172.21.15.92:0/2032633143 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c48067cc0 0x7f6c4806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c3ffff700 1 -- 172.21.15.92:0/2032633143 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4047013778 0 0) 0x7f6c34003570 con 0x7f6c480cc820 2023-10-06T10:38:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c3ffff700 1 -- 172.21.15.92:0/2032633143 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c48067cc0 msgr2=0x7f6c4806efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c3ffff700 1 --2- 172.21.15.92:0/2032633143 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c48067cc0 0x7f6c4806efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c3ffff700 1 -- 172.21.15.92:0/2032633143 >> v1:172.21.15.124:6789/0 conn(0x7f6c480ccbf0 legacy=0x7f6c480675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c3ffff700 1 -- 172.21.15.92:0/2032633143 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4815a090 con 0x7f6c480cc820 2023-10-06T10:38:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c3ffff700 1 -- 172.21.15.92:0/2032633143 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f6c440017f0 con 0x7f6c480cc820 2023-10-06T10:38:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c3ffff700 1 -- 172.21.15.92:0/2032633143 >> v1:172.21.15.92:6789/0 conn(0x7f6c480cc820 legacy=0x7f6c480c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c3ffff700 1 --2- 172.21.15.92:0/2032633143 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c34003a90 0x7f6c34003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c3ffff700 1 --2- 172.21.15.92:0/2032633143 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c48067cc0 0x7f6c34004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c3ffff700 1 --2- 172.21.15.92:0/2032633143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c34004d50 0x7f6c340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c3ffff700 1 -- 172.21.15.92:0/2032633143 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3400bb40 con 0x7f6c34004d50 2023-10-06T10:38:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c3ffff700 1 -- 172.21.15.92:0/2032633143 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3400bcb0 con 0x7f6c34003a90 2023-10-06T10:38:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c3ffff700 1 -- 172.21.15.92:0/2032633143 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3400be70 con 0x7f6c48067cc0 2023-10-06T10:38:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c4db07700 1 --2- 172.21.15.92:0/2032633143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c34004d50 0x7f6c340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c4d306700 1 --2- 172.21.15.92:0/2032633143 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c34003a90 0x7f6c34003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c4cb05700 1 --2- 172.21.15.92:0/2032633143 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c48067cc0 0x7f6c34004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c4db07700 1 -- 172.21.15.92:0/2032633143 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c48067cc0 msgr2=0x7f6c34004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c4db07700 1 --2- 172.21.15.92:0/2032633143 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c48067cc0 0x7f6c34004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c4db07700 1 -- 172.21.15.92:0/2032633143 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c34003a90 msgr2=0x7f6c34003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c4db07700 1 --2- 172.21.15.92:0/2032633143 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c34003a90 0x7f6c34003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.693+0000 7f6c4db07700 1 -- 172.21.15.92:0/2032633143 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c34003570 con 0x7f6c34004d50 2023-10-06T10:38:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.694+0000 7f6c4db07700 1 --2- 172.21.15.92:0/2032633143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c34004d50 0x7f6c340051c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6c480c7770 tx=0x7f6c400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf531779d30b08b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.694+0000 7f6c3ffff700 1 -- 172.21.15.92:0/2032633143 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4000e6e0 con 0x7f6c34004d50 2023-10-06T10:38:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.694+0000 7f6c3ffff700 1 -- 172.21.15.92:0/2032633143 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4000e840 con 0x7f6c34004d50 2023-10-06T10:38:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.694+0000 7f6c3ffff700 1 -- 172.21.15.92:0/2032633143 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4000ec00 con 0x7f6c34004d50 2023-10-06T10:38:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.694+0000 7f6c4f56a700 1 -- 172.21.15.92:0/2032633143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c34004d50 msgr2=0x7f6c340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.694+0000 7f6c4f56a700 1 --2- 172.21.15.92:0/2032633143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c34004d50 0x7f6c340051c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6c480c7770 tx=0x7f6c400091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.694+0000 7f6c4db07700 1 -- 172.21.15.92:0/2032633143 reap_dead start 2023-10-06T10:38:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.694+0000 7f6c4f56a700 1 -- 172.21.15.92:0/2032633143 shutdown_connections 2023-10-06T10:38:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.694+0000 7f6c4f56a700 1 -- 172.21.15.92:0/2032633143 >> 172.21.15.92:0/2032633143 conn(0x7f6c48069190 msgr2=0x7f6c480cd180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.694+0000 7f6c4f56a700 1 -- 172.21.15.92:0/2032633143 shutdown_connections 2023-10-06T10:38:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.694+0000 7f6c4f56a700 1 -- 172.21.15.92:0/2032633143 wait complete. 2023-10-06T10:38:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.695+0000 7f6c4f56a700 1 Processor -- start 2023-10-06T10:38:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.695+0000 7f6c4f56a700 1 -- start start 2023-10-06T10:38:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.696+0000 7f6c4f56a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c34004d50 0x7f6c481536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.696+0000 7f6c4f56a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c480aff20 0x7f6c480b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.696+0000 7f6c4f56a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c4806ab20 0x7f6c4806afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.696+0000 7f6c4f56a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c480b1310 con 0x7f6c34004d50 2023-10-06T10:38:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.696+0000 7f6c4f56a700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c480b0a90 con 0x7f6c480aff20 2023-10-06T10:38:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.696+0000 7f6c4f56a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c480b0d70 con 0x7f6c4806ab20 2023-10-06T10:38:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.696+0000 7f6c4d306700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c34004d50 0x7f6c481536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.697+0000 7f6c4d306700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c34004d50 0x7f6c481536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46992/0 (socket says 172.21.15.92:46992) 2023-10-06T10:38:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.697+0000 7f6c4d306700 1 -- 172.21.15.92:0/1308160077 learned_addr learned my addr 172.21.15.92:0/1308160077 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.697+0000 7f6c4cb05700 1 --2- 172.21.15.92:0/1308160077 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c480aff20 0x7f6c480b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.697+0000 7f6c4db07700 1 --2- 172.21.15.92:0/1308160077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c4806ab20 0x7f6c4806afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.697+0000 7f6c4d306700 1 -- 172.21.15.92:0/1308160077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c4806ab20 msgr2=0x7f6c4806afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.697+0000 7f6c4d306700 1 --2- 172.21.15.92:0/1308160077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c4806ab20 0x7f6c4806afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.697+0000 7f6c4d306700 1 -- 172.21.15.92:0/1308160077 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c480aff20 msgr2=0x7f6c480b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.697+0000 7f6c4d306700 1 --2- 172.21.15.92:0/1308160077 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c480aff20 0x7f6c480b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.697+0000 7f6c4d306700 1 -- 172.21.15.92:0/1308160077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4806a040 con 0x7f6c34004d50 2023-10-06T10:38:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.697+0000 7f6c4d306700 1 --2- 172.21.15.92:0/1308160077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c34004d50 0x7f6c481536b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6c44002060 tx=0x7f6c440021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.697+0000 7f6c3d7fa700 1 -- 172.21.15.92:0/1308160077 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c440017f0 con 0x7f6c34004d50 2023-10-06T10:38:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.697+0000 7f6c3d7fa700 1 -- 172.21.15.92:0/1308160077 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4400ad70 con 0x7f6c34004d50 2023-10-06T10:38:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.697+0000 7f6c4f56a700 1 -- 172.21.15.92:0/1308160077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4806a320 con 0x7f6c34004d50 2023-10-06T10:38:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.697+0000 7f6c3d7fa700 1 -- 172.21.15.92:0/1308160077 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c44013660 con 0x7f6c34004d50 2023-10-06T10:38:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.697+0000 7f6c4f56a700 1 -- 172.21.15.92:0/1308160077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c480c4440 con 0x7f6c34004d50 2023-10-06T10:38:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.698+0000 7f6c4f56a700 1 -- 172.21.15.92:0/1308160077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c48064a90 con 0x7f6c34004d50 2023-10-06T10:38:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.698+0000 7f6c3d7fa700 1 -- 172.21.15.92:0/1308160077 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6c44013a50 con 0x7f6c34004d50 2023-10-06T10:38:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.699+0000 7f6c3d7fa700 1 --2- 172.21.15.92:0/1308160077 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6c2c066f30 0x7f6c2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.699+0000 7f6c3d7fa700 1 -- 172.21.15.92:0/1308160077 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f6c44085680 con 0x7f6c34004d50 2023-10-06T10:38:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.699+0000 7f6c4cb05700 1 --2- 172.21.15.92:0/1308160077 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6c2c066f30 0x7f6c2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.699+0000 7f6c4cb05700 1 --2- 172.21.15.92:0/1308160077 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6c2c066f30 0x7f6c2c0693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6c38007e40 tx=0x7f6c38007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.703+0000 7f6c3d7fa700 1 -- 172.21.15.92:0/1308160077 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c44052560 con 0x7f6c34004d50 2023-10-06T10:38:53.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.883+0000 7f6c4f56a700 1 -- 172.21.15.92:0/1308160077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6c4806b4e0 con 0x7f6c34004d50 2023-10-06T10:38:53.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.884+0000 7f6c3d7fa700 1 -- 172.21.15.92:0/1308160077 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6c44055b80 con 0x7f6c34004d50 2023-10-06T10:38:53.886 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T10:38:53.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.886+0000 7f6c4f56a700 1 -- 172.21.15.92:0/1308160077 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6c2c066f30 msgr2=0x7f6c2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:53.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.886+0000 7f6c4f56a700 1 --2- 172.21.15.92:0/1308160077 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6c2c066f30 0x7f6c2c0693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6c38007e40 tx=0x7f6c38007d30 comp rx=0 tx=0).stop 2023-10-06T10:38:53.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.886+0000 7f6c4f56a700 1 -- 172.21.15.92:0/1308160077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c34004d50 msgr2=0x7f6c481536b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:53.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.886+0000 7f6c4f56a700 1 --2- 172.21.15.92:0/1308160077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c34004d50 0x7f6c481536b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6c44002060 tx=0x7f6c440021d0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.886+0000 7f6c4f56a700 1 -- 172.21.15.92:0/1308160077 shutdown_connections 2023-10-06T10:38:53.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.886+0000 7f6c4f56a700 1 --2- 172.21.15.92:0/1308160077 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6c2c066f30 0x7f6c2c0693e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.886+0000 7f6c4f56a700 1 --2- 172.21.15.92:0/1308160077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c4806ab20 0x7f6c4806afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.886+0000 7f6c4f56a700 1 --2- 172.21.15.92:0/1308160077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c34004d50 0x7f6c481536b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.886+0000 7f6c4f56a700 1 --2- 172.21.15.92:0/1308160077 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c480aff20 0x7f6c480b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:53.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.886+0000 7f6c4f56a700 1 -- 172.21.15.92:0/1308160077 >> 172.21.15.92:0/1308160077 conn(0x7f6c48069190 msgr2=0x7f6c481545e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:53.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.886+0000 7f6c4f56a700 1 -- 172.21.15.92:0/1308160077 shutdown_connections 2023-10-06T10:38:53.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:53.886+0000 7f6c4f56a700 1 -- 172.21.15.92:0/1308160077 wait complete. 2023-10-06T10:38:53.899 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T10:38:54.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.015+0000 7fe0b08cd700 1 Processor -- start 2023-10-06T10:38:54.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.015+0000 7fe0b08cd700 1 -- start start 2023-10-06T10:38:54.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.016+0000 7fe0b08cd700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe0ac1598a0 0x7fe0ac0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:54.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.016+0000 7fe0b08cd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe0ac0cbc70 0x7fe0ac0ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:54.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.016+0000 7fe0b08cd700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ac0cf5d0 con 0x7fe0ac1594d0 2023-10-06T10:38:54.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.016+0000 7fe0b08cd700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0ac0cf8b0 con 0x7fe0ac1598a0 2023-10-06T10:38:54.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.016+0000 7fe0b08cd700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac0cfb90 con 0x7fe0ac0cbc70 2023-10-06T10:38:54.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.016+0000 7fe0aa59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe0ac1594d0 0x7fe0ac0cb560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50298/0 (socket says 172.21.15.92:50298) 2023-10-06T10:38:54.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.016+0000 7fe0aa59c700 1 -- 172.21.15.92:0/2266419953 learned_addr learned my addr 172.21.15.92:0/2266419953 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:54.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.016+0000 7fe0aad9d700 1 --2- 172.21.15.92:0/2266419953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe0ac0cbc70 0x7fe0ac0ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:54.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.016+0000 7fe0a1d9b700 1 --2- 172.21.15.92:0/2266419953 >> v2:172.21.15.124:3300/0 conn(0x7fe0ac1598a0 0x7fe0ac0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:54.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.016+0000 7fe0a9d9b700 1 -- 172.21.15.92:0/2266419953 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 190016285 0 0) 0x7fe0ac0cf5d0 con 0x7fe0ac1594d0 2023-10-06T10:38:54.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.016+0000 7fe0a9d9b700 1 -- 172.21.15.92:0/2266419953 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe098003570 con 0x7fe0ac1594d0 2023-10-06T10:38:54.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.017+0000 7fe0a1d9b700 1 -- 172.21.15.92:0/2266419953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe0ac0cbc70 msgr2=0x7fe0ac0ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.017+0000 7fe0a1d9b700 1 --2- 172.21.15.92:0/2266419953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe0ac0cbc70 0x7fe0ac0ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.017+0000 7fe0a1d9b700 1 -- 172.21.15.92:0/2266419953 >> v1:172.21.15.92:6789/0 conn(0x7fe0ac1594d0 legacy=0x7fe0ac0cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.017+0000 7fe0a1d9b700 1 -- 172.21.15.92:0/2266419953 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac0cfe70 con 0x7fe0ac1598a0 2023-10-06T10:38:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.017+0000 7fe0a1d9b700 1 --2- 172.21.15.92:0/2266419953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0ac1598a0 0x7fe0ac0ce7e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe0940098e0 tx=0x7fe0940091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed235776a1fcb122 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.017+0000 7fe0a9d9b700 1 -- 172.21.15.92:0/2266419953 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094011630 con 0x7fe0ac1598a0 2023-10-06T10:38:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.017+0000 7fe0a9d9b700 1 -- 172.21.15.92:0/2266419953 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe094011790 con 0x7fe0ac1598a0 2023-10-06T10:38:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.017+0000 7fe0a9d9b700 1 -- 172.21.15.92:0/2266419953 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094011b10 con 0x7fe0ac1598a0 2023-10-06T10:38:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.017+0000 7fe0b08cd700 1 -- 172.21.15.92:0/2266419953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0ac1598a0 msgr2=0x7fe0ac0ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.017+0000 7fe0b08cd700 1 --2- 172.21.15.92:0/2266419953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0ac1598a0 0x7fe0ac0ce7e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe0940098e0 tx=0x7fe0940091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.017+0000 7fe0b08cd700 1 -- 172.21.15.92:0/2266419953 shutdown_connections 2023-10-06T10:38:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.017+0000 7fe0b08cd700 1 --2- 172.21.15.92:0/2266419953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe0ac0cbc70 0x7fe0ac0ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.017+0000 7fe0b08cd700 1 --2- 172.21.15.92:0/2266419953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0ac1598a0 0x7fe0ac0ce7e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.017+0000 7fe0b08cd700 1 -- 172.21.15.92:0/2266419953 >> 172.21.15.92:0/2266419953 conn(0x7fe0ac089e50 msgr2=0x7fe0ac099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:54.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.018+0000 7fe0b08cd700 1 -- 172.21.15.92:0/2266419953 shutdown_connections 2023-10-06T10:38:54.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.018+0000 7fe0b08cd700 1 -- 172.21.15.92:0/2266419953 wait complete. 2023-10-06T10:38:54.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.019+0000 7fe0b08cd700 1 Processor -- start 2023-10-06T10:38:54.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.019+0000 7fe0b08cd700 1 -- start start 2023-10-06T10:38:54.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.019+0000 7fe0b08cd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0ac0cbc70 0x7fe0ac0b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:54.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.019+0000 7fe0b08cd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe0ac1594d0 0x7fe0ac14f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:54.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.019+0000 7fe0b08cd700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0ac1598a0 0x7fe0ac0c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:54.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.019+0000 7fe0b08cd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac14ca20 con 0x7fe0ac0cbc70 2023-10-06T10:38:54.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.019+0000 7fe0b08cd700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac0b4f00 con 0x7fe0ac1598a0 2023-10-06T10:38:54.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.019+0000 7fe0b08cd700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac0b4280 con 0x7fe0ac1594d0 2023-10-06T10:38:54.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.020+0000 7fe0aa59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0ac0cbc70 0x7fe0ac0b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:54.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.020+0000 7fe0aa59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0ac0cbc70 0x7fe0ac0b3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47008/0 (socket says 172.21.15.92:47008) 2023-10-06T10:38:54.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.020+0000 7fe0aa59c700 1 -- 172.21.15.92:0/3278305185 learned_addr learned my addr 172.21.15.92:0/3278305185 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:54.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.020+0000 7fe0a1d9b700 1 --2- 172.21.15.92:0/3278305185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe0ac1594d0 0x7fe0ac14f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:54.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.020+0000 7fe0aad9d700 1 --2- 172.21.15.92:0/3278305185 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0ac1598a0 0x7fe0ac0c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:54.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.020+0000 7fe0aa59c700 1 -- 172.21.15.92:0/3278305185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe0ac1594d0 msgr2=0x7fe0ac14f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:54.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.020+0000 7fe0aa59c700 1 --2- 172.21.15.92:0/3278305185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe0ac1594d0 0x7fe0ac14f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.020+0000 7fe0aa59c700 1 -- 172.21.15.92:0/3278305185 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0ac1598a0 msgr2=0x7fe0ac0c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:54.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.020+0000 7fe0aa59c700 1 --2- 172.21.15.92:0/3278305185 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0ac1598a0 0x7fe0ac0c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.020+0000 7fe0aa59c700 1 -- 172.21.15.92:0/3278305185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe094009420 con 0x7fe0ac0cbc70 2023-10-06T10:38:54.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.020+0000 7fe0aa59c700 1 --2- 172.21.15.92:0/3278305185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0ac0cbc70 0x7fe0ac0b3c70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0011e0 tx=0x7fe09c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.020+0000 7fe0a2ffd700 1 -- 172.21.15.92:0/3278305185 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c005b30 con 0x7fe0ac0cbc70 2023-10-06T10:38:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.020+0000 7fe0a2ffd700 1 -- 172.21.15.92:0/3278305185 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09c005c90 con 0x7fe0ac0cbc70 2023-10-06T10:38:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.020+0000 7fe0b08cd700 1 -- 172.21.15.92:0/3278305185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0ac0b4560 con 0x7fe0ac0cbc70 2023-10-06T10:38:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.020+0000 7fe0a2ffd700 1 -- 172.21.15.92:0/3278305185 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c007300 con 0x7fe0ac0cbc70 2023-10-06T10:38:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.020+0000 7fe0b08cd700 1 -- 172.21.15.92:0/3278305185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ac0b4b20 con 0x7fe0ac0cbc70 2023-10-06T10:38:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.021+0000 7fe0b08cd700 1 -- 172.21.15.92:0/3278305185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0ac064a90 con 0x7fe0ac0cbc70 2023-10-06T10:38:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.021+0000 7fe0a2ffd700 1 -- 172.21.15.92:0/3278305185 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe09c007670 con 0x7fe0ac0cbc70 2023-10-06T10:38:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.022+0000 7fe0a2ffd700 1 --2- 172.21.15.92:0/3278305185 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe08c066f30 0x7fe08c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.022+0000 7fe0a2ffd700 1 -- 172.21.15.92:0/3278305185 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe09c083d90 con 0x7fe0ac0cbc70 2023-10-06T10:38:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.022+0000 7fe0a1d9b700 1 --2- 172.21.15.92:0/3278305185 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe08c066f30 0x7fe08c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.022+0000 7fe0a1d9b700 1 --2- 172.21.15.92:0/3278305185 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe08c066f30 0x7fe08c0693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe0940098e0 tx=0x7fe094017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:54.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.026+0000 7fe0a2ffd700 1 -- 172.21.15.92:0/3278305185 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe09c050bb0 con 0x7fe0ac0cbc70 2023-10-06T10:38:54.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.208+0000 7fe0b08cd700 1 -- 172.21.15.92:0/3278305185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0ac0cf260 con 0x7fe0ac0cbc70 2023-10-06T10:38:54.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.208+0000 7fe0a2ffd700 1 -- 172.21.15.92:0/3278305185 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe09c017020 con 0x7fe0ac0cbc70 2023-10-06T10:38:54.210 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T10:38:54.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.211+0000 7fe0b08cd700 1 -- 172.21.15.92:0/3278305185 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe08c066f30 msgr2=0x7fe08c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:54.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.211+0000 7fe0b08cd700 1 --2- 172.21.15.92:0/3278305185 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe08c066f30 0x7fe08c0693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe0940098e0 tx=0x7fe094017000 comp rx=0 tx=0).stop 2023-10-06T10:38:54.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.211+0000 7fe0b08cd700 1 -- 172.21.15.92:0/3278305185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0ac0cbc70 msgr2=0x7fe0ac0b3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:54.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.211+0000 7fe0b08cd700 1 --2- 172.21.15.92:0/3278305185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0ac0cbc70 0x7fe0ac0b3c70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0011e0 tx=0x7fe09c007df0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.211+0000 7fe0b08cd700 1 -- 172.21.15.92:0/3278305185 shutdown_connections 2023-10-06T10:38:54.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.211+0000 7fe0b08cd700 1 --2- 172.21.15.92:0/3278305185 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe08c066f30 0x7fe08c0693e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.211+0000 7fe0b08cd700 1 --2- 172.21.15.92:0/3278305185 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0ac1598a0 0x7fe0ac0c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.211+0000 7fe0b08cd700 1 --2- 172.21.15.92:0/3278305185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0ac0cbc70 0x7fe0ac0b3c70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.211+0000 7fe0b08cd700 1 --2- 172.21.15.92:0/3278305185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe0ac1594d0 0x7fe0ac14f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.211+0000 7fe0b08cd700 1 -- 172.21.15.92:0/3278305185 >> 172.21.15.92:0/3278305185 conn(0x7fe0ac089e50 msgr2=0x7fe0ac068370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:54.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.211+0000 7fe0b08cd700 1 -- 172.21.15.92:0/3278305185 shutdown_connections 2023-10-06T10:38:54.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.211+0000 7fe0b08cd700 1 -- 172.21.15.92:0/3278305185 wait complete. 2023-10-06T10:38:54.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T10:38:54.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.341+0000 7f518aed7700 1 Processor -- start 2023-10-06T10:38:54.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.341+0000 7f518aed7700 1 -- start start 2023-10-06T10:38:54.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.342+0000 7f518aed7700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f51840ac450 0x7f51840a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:54.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.342+0000 7f518aed7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51840aa540 con 0x7f51840af380 2023-10-06T10:38:54.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.342+0000 7f518aed7700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f518415d3a0 con 0x7f51840ac080 2023-10-06T10:38:54.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.342+0000 7f518aed7700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51840af820 con 0x7f51840ac450 2023-10-06T10:38:54.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.342+0000 7f518a6d6700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f51840af380 0x7f51840675b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50302/0 (socket says 172.21.15.92:50302) 2023-10-06T10:38:54.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.342+0000 7f518a6d6700 1 -- 172.21.15.92:0/217204845 learned_addr learned my addr 172.21.15.92:0/217204845 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:54.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.342+0000 7f51896d4700 1 --2- 172.21.15.92:0/217204845 >> v2:172.21.15.124:3300/0 conn(0x7f51840ac450 0x7f51840a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:54.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.342+0000 7f5188ed3700 1 -- 172.21.15.92:0/217204845 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069541042 0 0) 0x7f51840aa540 con 0x7f51840af380 2023-10-06T10:38:54.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.342+0000 7f5188ed3700 1 -- 172.21.15.92:0/217204845 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5170003570 con 0x7f51840af380 2023-10-06T10:38:54.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.342+0000 7f5188ed3700 1 -- 172.21.15.92:0/217204845 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 623212313 0 0) 0x7f5170003570 con 0x7f51840af380 2023-10-06T10:38:54.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.342+0000 7f5188ed3700 1 -- 172.21.15.92:0/217204845 >> v2:172.21.15.124:3300/0 conn(0x7f51840ac450 msgr2=0x7f51840a98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:54.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.342+0000 7f5188ed3700 1 --2- 172.21.15.92:0/217204845 >> v2:172.21.15.124:3300/0 conn(0x7f51840ac450 0x7f51840a98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.342+0000 7f5188ed3700 1 -- 172.21.15.92:0/217204845 >> v1:172.21.15.124:6789/0 conn(0x7f51840ac080 legacy=0x7f518415d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:54.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.342+0000 7f5188ed3700 1 -- 172.21.15.92:0/217204845 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51840a9f80 con 0x7f51840af380 2023-10-06T10:38:54.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.342+0000 7f5188ed3700 1 -- 172.21.15.92:0/217204845 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f51800017f0 con 0x7f51840af380 2023-10-06T10:38:54.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.342+0000 7f5188ed3700 1 -- 172.21.15.92:0/217204845 >> v1:172.21.15.92:6789/0 conn(0x7f51840af380 legacy=0x7f51840675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:54.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.343+0000 7f5188ed3700 1 --2- 172.21.15.92:0/217204845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5170003a60 0x7f5170003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:54.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.343+0000 7f5188ed3700 1 --2- 172.21.15.92:0/217204845 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5170004810 0x7f5170004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:54.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.343+0000 7f5188ed3700 1 --2- 172.21.15.92:0/217204845 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51700051c0 0x7f517000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:54.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.343+0000 7f5188ed3700 1 -- 172.21.15.92:0/217204845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51840af820 con 0x7f5170003a60 2023-10-06T10:38:54.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.343+0000 7f5188ed3700 1 -- 172.21.15.92:0/217204845 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517000c0d0 con 0x7f51700051c0 2023-10-06T10:38:54.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.343+0000 7f5188ed3700 1 -- 172.21.15.92:0/217204845 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f517000c290 con 0x7f5170004810 2023-10-06T10:38:54.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.343+0000 7f5189ed5700 1 --2- 172.21.15.92:0/217204845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5170003a60 0x7f5170003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:54.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.343+0000 7f518a6d6700 1 --2- 172.21.15.92:0/217204845 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51700051c0 0x7f517000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:54.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.343+0000 7f51896d4700 1 --2- 172.21.15.92:0/217204845 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5170004810 0x7f5170004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:54.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.343+0000 7f5189ed5700 1 -- 172.21.15.92:0/217204845 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5170004810 msgr2=0x7f5170004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:54.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.343+0000 7f5189ed5700 1 --2- 172.21.15.92:0/217204845 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5170004810 0x7f5170004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.343+0000 7f5189ed5700 1 -- 172.21.15.92:0/217204845 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51700051c0 msgr2=0x7f517000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:54.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.343+0000 7f5189ed5700 1 --2- 172.21.15.92:0/217204845 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51700051c0 0x7f517000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.343+0000 7f518a6d6700 1 -- 172.21.15.92:0/217204845 reap_dead start 2023-10-06T10:38:54.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.343+0000 7f5189ed5700 1 -- 172.21.15.92:0/217204845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5170003570 con 0x7f5170003a60 2023-10-06T10:38:54.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.343+0000 7f5189ed5700 1 --2- 172.21.15.92:0/217204845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5170003a60 0x7f5170003e70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f517c0018b0 tx=0x7f517c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc46ee5cb8987af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:54.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.343+0000 7f5188ed3700 1 -- 172.21.15.92:0/217204845 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c012420 con 0x7f5170003a60 2023-10-06T10:38:54.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.343+0000 7f5188ed3700 1 -- 172.21.15.92:0/217204845 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517c0125a0 con 0x7f5170003a60 2023-10-06T10:38:54.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.343+0000 7f5188ed3700 1 -- 172.21.15.92:0/217204845 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c012960 con 0x7f5170003a60 2023-10-06T10:38:54.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.344+0000 7f518aed7700 1 -- 172.21.15.92:0/217204845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5170003a60 msgr2=0x7f5170003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:54.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.344+0000 7f518aed7700 1 --2- 172.21.15.92:0/217204845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5170003a60 0x7f5170003e70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f517c0018b0 tx=0x7f517c007d30 comp rx=0 tx=0).stop 2023-10-06T10:38:54.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.344+0000 7f518aed7700 1 -- 172.21.15.92:0/217204845 shutdown_connections 2023-10-06T10:38:54.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.344+0000 7f518aed7700 1 --2- 172.21.15.92:0/217204845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5170003a60 0x7f5170003e70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.344+0000 7f518aed7700 1 -- 172.21.15.92:0/217204845 >> 172.21.15.92:0/217204845 conn(0x7f5184069140 msgr2=0x7f51840aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:54.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.344+0000 7f518aed7700 1 -- 172.21.15.92:0/217204845 shutdown_connections 2023-10-06T10:38:54.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.344+0000 7f518aed7700 1 -- 172.21.15.92:0/217204845 wait complete. 2023-10-06T10:38:54.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.345+0000 7f518aed7700 1 Processor -- start 2023-10-06T10:38:54.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.345+0000 7f518aed7700 1 -- start start 2023-10-06T10:38:54.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.345+0000 7f518aed7700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5170003a60 0x7f518414eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:54.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.346+0000 7f518aed7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5184154530 0x7f5184169e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:54.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.346+0000 7f518aed7700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51840ada60 0x7f51840adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:54.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.346+0000 7f518aed7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51840ae440 con 0x7f5184154530 2023-10-06T10:38:54.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.346+0000 7f518aed7700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518414fcf0 con 0x7f5170003a60 2023-10-06T10:38:54.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.346+0000 7f518aed7700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518416a390 con 0x7f51840ada60 2023-10-06T10:38:54.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.346+0000 7f51896d4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5184154530 0x7f5184169e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:54.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.346+0000 7f51896d4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5184154530 0x7f5184169e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47030/0 (socket says 172.21.15.92:47030) 2023-10-06T10:38:54.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.346+0000 7f51896d4700 1 -- 172.21.15.92:0/1190709716 learned_addr learned my addr 172.21.15.92:0/1190709716 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:54.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.346+0000 7f5189ed5700 1 --2- 172.21.15.92:0/1190709716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5170003a60 0x7f518414eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:54.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.346+0000 7f518a6d6700 1 --2- 172.21.15.92:0/1190709716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51840ada60 0x7f51840adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:54.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.346+0000 7f51896d4700 1 -- 172.21.15.92:0/1190709716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51840ada60 msgr2=0x7f51840adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:54.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.346+0000 7f51896d4700 1 --2- 172.21.15.92:0/1190709716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51840ada60 0x7f51840adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.346+0000 7f51896d4700 1 -- 172.21.15.92:0/1190709716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5170003a60 msgr2=0x7f518414eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:54.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.346+0000 7f51896d4700 1 --2- 172.21.15.92:0/1190709716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5170003a60 0x7f518414eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.346+0000 7f51896d4700 1 -- 172.21.15.92:0/1190709716 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51841549a0 con 0x7f5184154530 2023-10-06T10:38:54.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.346+0000 7f51896d4700 1 --2- 172.21.15.92:0/1190709716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5184154530 0x7f5184169e50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5174011d40 tx=0x7f51740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:54.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.346+0000 7f5179ffb700 1 -- 172.21.15.92:0/1190709716 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517400fac0 con 0x7f5184154530 2023-10-06T10:38:54.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.346+0000 7f5179ffb700 1 -- 172.21.15.92:0/1190709716 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517400fc20 con 0x7f5184154530 2023-10-06T10:38:54.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.346+0000 7f518aed7700 1 -- 172.21.15.92:0/1190709716 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518414f0e0 con 0x7f5184154530 2023-10-06T10:38:54.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.346+0000 7f5179ffb700 1 -- 172.21.15.92:0/1190709716 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174014440 con 0x7f5184154530 2023-10-06T10:38:54.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.346+0000 7f518aed7700 1 -- 172.21.15.92:0/1190709716 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518414f6a0 con 0x7f5184154530 2023-10-06T10:38:54.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.347+0000 7f518aed7700 1 -- 172.21.15.92:0/1190709716 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f518415acb0 con 0x7f5184154530 2023-10-06T10:38:54.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.348+0000 7f5179ffb700 1 -- 172.21.15.92:0/1190709716 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5174014870 con 0x7f5184154530 2023-10-06T10:38:54.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.348+0000 7f5179ffb700 1 --2- 172.21.15.92:0/1190709716 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5168066f30 0x7f51680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:54.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.348+0000 7f5179ffb700 1 -- 172.21.15.92:0/1190709716 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f5174018030 con 0x7f5184154530 2023-10-06T10:38:54.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.348+0000 7f5189ed5700 1 --2- 172.21.15.92:0/1190709716 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5168066f30 0x7f51680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:54.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.348+0000 7f5189ed5700 1 --2- 172.21.15.92:0/1190709716 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5168066f30 0x7f51680693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f517c0018b0 tx=0x7f517c015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:54.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.352+0000 7f5179ffb700 1 -- 172.21.15.92:0/1190709716 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f518415acb0 con 0x7f5184154530 2023-10-06T10:38:54.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.535+0000 7f518aed7700 1 -- 172.21.15.92:0/1190709716 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f518415acb0 con 0x7f5184154530 2023-10-06T10:38:54.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.535+0000 7f5179ffb700 1 -- 172.21.15.92:0/1190709716 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f518415acb0 con 0x7f5184154530 2023-10-06T10:38:54.537 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T10:38:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.537+0000 7f518aed7700 1 -- 172.21.15.92:0/1190709716 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5168066f30 msgr2=0x7f51680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.537+0000 7f518aed7700 1 --2- 172.21.15.92:0/1190709716 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5168066f30 0x7f51680693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f517c0018b0 tx=0x7f517c015000 comp rx=0 tx=0).stop 2023-10-06T10:38:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.537+0000 7f518aed7700 1 -- 172.21.15.92:0/1190709716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5184154530 msgr2=0x7f5184169e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.537+0000 7f518aed7700 1 --2- 172.21.15.92:0/1190709716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5184154530 0x7f5184169e50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5174011d40 tx=0x7f51740091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.537+0000 7f518aed7700 1 -- 172.21.15.92:0/1190709716 shutdown_connections 2023-10-06T10:38:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.537+0000 7f518aed7700 1 --2- 172.21.15.92:0/1190709716 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5168066f30 0x7f51680693e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.537+0000 7f518aed7700 1 --2- 172.21.15.92:0/1190709716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51840ada60 0x7f51840adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.537+0000 7f518aed7700 1 --2- 172.21.15.92:0/1190709716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5170003a60 0x7f518414eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.537+0000 7f518aed7700 1 --2- 172.21.15.92:0/1190709716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5184154530 0x7f5184169e50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.537+0000 7f518aed7700 1 -- 172.21.15.92:0/1190709716 >> 172.21.15.92:0/1190709716 conn(0x7f5184069140 msgr2=0x7f518406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:54.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.538+0000 7f518aed7700 1 -- 172.21.15.92:0/1190709716 shutdown_connections 2023-10-06T10:38:54.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.538+0000 7f518aed7700 1 -- 172.21.15.92:0/1190709716 wait complete. 2023-10-06T10:38:54.551 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T10:38:54.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.666+0000 7fa977558700 1 Processor -- start 2023-10-06T10:38:54.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.667+0000 7fa977558700 1 -- start start 2023-10-06T10:38:54.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.667+0000 7fa977558700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa970159510 0x7fa9700675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:54.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.667+0000 7fa977558700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa97006eaa0 0x7fa970067af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:54.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.667+0000 7fa977558700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9700b0650 con 0x7fa9701598e0 2023-10-06T10:38:54.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.667+0000 7fa977558700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa97006efe0 con 0x7fa97006eaa0 2023-10-06T10:38:54.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.667+0000 7fa977558700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa97006f2c0 con 0x7fa970159510 2023-10-06T10:38:54.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.668+0000 7fa975af5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa97006eaa0 0x7fa970067af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:54.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.668+0000 7fa975af5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa97006eaa0 0x7fa970067af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47038/0 (socket says 172.21.15.92:47038) 2023-10-06T10:38:54.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.668+0000 7fa975af5700 1 -- 172.21.15.92:0/2715552854 learned_addr learned my addr 172.21.15.92:0/2715552854 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:54.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.668+0000 7fa9752f4700 1 --2- 172.21.15.92:0/2715552854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa970159510 0x7fa9700675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:54.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.668+0000 7fa975af5700 1 -- 172.21.15.92:0/2715552854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa970159510 msgr2=0x7fa9700675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:54.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.668+0000 7fa975af5700 1 --2- 172.21.15.92:0/2715552854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa970159510 0x7fa9700675b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.668+0000 7fa975af5700 1 -- 172.21.15.92:0/2715552854 >> v1:172.21.15.124:6789/0 conn(0x7fa9701598e0 legacy=0x7fa97006e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:54.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.668+0000 7fa96ffff700 1 -- 172.21.15.92:0/2715552854 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1854841195 0 0) 0x7fa9700b0650 con 0x7fa9701598e0 2023-10-06T10:38:54.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.668+0000 7fa975af5700 1 -- 172.21.15.92:0/2715552854 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9700b0000 con 0x7fa97006eaa0 2023-10-06T10:38:54.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.668+0000 7fa975af5700 1 --2- 172.21.15.92:0/2715552854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa97006eaa0 0x7fa970067af0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa968009f20 tx=0x7fa968009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597cca0039720809 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:54.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.668+0000 7fa96ffff700 1 -- 172.21.15.92:0/2715552854 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96800e070 con 0x7fa97006eaa0 2023-10-06T10:38:54.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.668+0000 7fa96ffff700 1 -- 172.21.15.92:0/2715552854 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96800b9d0 con 0x7fa97006eaa0 2023-10-06T10:38:54.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.668+0000 7fa96ffff700 1 -- 172.21.15.92:0/2715552854 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9680092e0 con 0x7fa97006eaa0 2023-10-06T10:38:54.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.669+0000 7fa977558700 1 -- 172.21.15.92:0/2715552854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa97006eaa0 msgr2=0x7fa970067af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:54.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.669+0000 7fa977558700 1 --2- 172.21.15.92:0/2715552854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa97006eaa0 0x7fa970067af0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa968009f20 tx=0x7fa968009840 comp rx=0 tx=0).stop 2023-10-06T10:38:54.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.669+0000 7fa977558700 1 -- 172.21.15.92:0/2715552854 shutdown_connections 2023-10-06T10:38:54.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.669+0000 7fa977558700 1 --2- 172.21.15.92:0/2715552854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa97006eaa0 0x7fa970067af0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.669+0000 7fa977558700 1 --2- 172.21.15.92:0/2715552854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa970159510 0x7fa9700675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.669+0000 7fa977558700 1 -- 172.21.15.92:0/2715552854 >> 172.21.15.92:0/2715552854 conn(0x7fa970069150 msgr2=0x7fa9700cd170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:54.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.669+0000 7fa977558700 1 -- 172.21.15.92:0/2715552854 shutdown_connections 2023-10-06T10:38:54.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.669+0000 7fa977558700 1 -- 172.21.15.92:0/2715552854 wait complete. 2023-10-06T10:38:54.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.670+0000 7fa977558700 1 Processor -- start 2023-10-06T10:38:54.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.670+0000 7fa977558700 1 -- start start 2023-10-06T10:38:54.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.671+0000 7fa977558700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa97006eaa0 0x7fa9700c2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.671+0000 7fa977558700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa970159510 0x7fa970144cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.671+0000 7fa977558700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9701598e0 0x7fa97015c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.671+0000 7fa977558700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa970157aa0 con 0x7fa9701598e0 2023-10-06T10:38:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.671+0000 7fa977558700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97016a6f0 con 0x7fa97006eaa0 2023-10-06T10:38:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.671+0000 7fa977558700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa970169ae0 con 0x7fa970159510 2023-10-06T10:38:54.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.671+0000 7fa975af5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9701598e0 0x7fa97015c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:54.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.671+0000 7fa975af5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9701598e0 0x7fa97015c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47042/0 (socket says 172.21.15.92:47042) 2023-10-06T10:38:54.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.671+0000 7fa975af5700 1 -- 172.21.15.92:0/4021492742 learned_addr learned my addr 172.21.15.92:0/4021492742 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:54.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.671+0000 7fa975af5700 1 -- 172.21.15.92:0/4021492742 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa970159510 msgr2=0x7fa970144cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:54.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.671+0000 7fa975af5700 1 --2- 172.21.15.92:0/4021492742 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa970159510 0x7fa970144cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.671+0000 7fa9752f4700 1 --2- 172.21.15.92:0/4021492742 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa97006eaa0 0x7fa9700c2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:54.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.671+0000 7fa975af5700 1 -- 172.21.15.92:0/4021492742 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa97006eaa0 msgr2=0x7fa9700c2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:54.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.671+0000 7fa975af5700 1 --2- 172.21.15.92:0/4021492742 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa97006eaa0 0x7fa9700c2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.671+0000 7fa975af5700 1 -- 172.21.15.92:0/4021492742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa968009ae0 con 0x7fa9701598e0 2023-10-06T10:38:54.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.671+0000 7fa975af5700 1 --2- 172.21.15.92:0/4021492742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9701598e0 0x7fa97015c790 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa96800b610 tx=0x7fa968018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.671+0000 7fa96d7fa700 1 -- 172.21.15.92:0/4021492742 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa968008c30 con 0x7fa9701598e0 2023-10-06T10:38:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.672+0000 7fa96d7fa700 1 -- 172.21.15.92:0/4021492742 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa968008d90 con 0x7fa9701598e0 2023-10-06T10:38:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.672+0000 7fa96d7fa700 1 -- 172.21.15.92:0/4021492742 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa968026670 con 0x7fa9701598e0 2023-10-06T10:38:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.672+0000 7fa977558700 1 -- 172.21.15.92:0/4021492742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa970169dc0 con 0x7fa9701598e0 2023-10-06T10:38:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.672+0000 7fa977558700 1 -- 172.21.15.92:0/4021492742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa97016a380 con 0x7fa9701598e0 2023-10-06T10:38:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.672+0000 7fa977558700 1 -- 172.21.15.92:0/4021492742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa97016cd90 con 0x7fa9701598e0 2023-10-06T10:38:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.673+0000 7fa96d7fa700 1 -- 172.21.15.92:0/4021492742 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa968003040 con 0x7fa9701598e0 2023-10-06T10:38:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.673+0000 7fa96d7fa700 1 --2- 172.21.15.92:0/4021492742 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa95c066f30 0x7fa95c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.673+0000 7fa96d7fa700 1 -- 172.21.15.92:0/4021492742 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa968086320 con 0x7fa9701598e0 2023-10-06T10:38:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.673+0000 7fa9752f4700 1 --2- 172.21.15.92:0/4021492742 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa95c066f30 0x7fa95c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.674+0000 7fa9752f4700 1 --2- 172.21.15.92:0/4021492742 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa95c066f30 0x7fa95c0693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa960009390 tx=0x7fa960008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.677+0000 7fa96d7fa700 1 -- 172.21.15.92:0/4021492742 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa97016cd90 con 0x7fa9701598e0 2023-10-06T10:38:54.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.858+0000 7fa977558700 1 -- 172.21.15.92:0/4021492742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa97016cd90 con 0x7fa9701598e0 2023-10-06T10:38:54.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.858+0000 7fa96d7fa700 1 -- 172.21.15.92:0/4021492742 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa97016cd90 con 0x7fa9701598e0 2023-10-06T10:38:54.892 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T10:38:54.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.860+0000 7fa977558700 1 -- 172.21.15.92:0/4021492742 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa95c066f30 msgr2=0x7fa95c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:54.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.860+0000 7fa977558700 1 --2- 172.21.15.92:0/4021492742 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa95c066f30 0x7fa95c0693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa960009390 tx=0x7fa960008040 comp rx=0 tx=0).stop 2023-10-06T10:38:54.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.860+0000 7fa977558700 1 -- 172.21.15.92:0/4021492742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9701598e0 msgr2=0x7fa97015c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:54.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.860+0000 7fa977558700 1 --2- 172.21.15.92:0/4021492742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9701598e0 0x7fa97015c790 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa96800b610 tx=0x7fa968018000 comp rx=0 tx=0).stop 2023-10-06T10:38:54.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.860+0000 7fa977558700 1 -- 172.21.15.92:0/4021492742 shutdown_connections 2023-10-06T10:38:54.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.860+0000 7fa977558700 1 --2- 172.21.15.92:0/4021492742 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa95c066f30 0x7fa95c0693e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.860+0000 7fa977558700 1 --2- 172.21.15.92:0/4021492742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9701598e0 0x7fa97015c790 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.860+0000 7fa977558700 1 --2- 172.21.15.92:0/4021492742 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa97006eaa0 0x7fa9700c2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.860+0000 7fa977558700 1 --2- 172.21.15.92:0/4021492742 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa970159510 0x7fa970144cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:54.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.860+0000 7fa977558700 1 -- 172.21.15.92:0/4021492742 >> 172.21.15.92:0/4021492742 conn(0x7fa970069150 msgr2=0x7fa97006e390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:54.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.860+0000 7fa977558700 1 -- 172.21.15.92:0/4021492742 shutdown_connections 2023-10-06T10:38:54.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:54.860+0000 7fa977558700 1 -- 172.21.15.92:0/4021492742 wait complete. 2023-10-06T10:38:54.895 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T10:38:55.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.010+0000 7f2c12a7b700 1 Processor -- start 2023-10-06T10:38:55.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.010+0000 7f2c12a7b700 1 -- start start 2023-10-06T10:38:55.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.011+0000 7f2c12a7b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2c0c1594d0 0x7f2c0c0cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.011+0000 7f2c12a7b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c0c0cbee0 con 0x7f2c0c0cbaa0 2023-10-06T10:38:55.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.011+0000 7f2c12a7b700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c0c0cf5d0 con 0x7f2c0c1598a0 2023-10-06T10:38:55.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.011+0000 7f2c12a7b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c0c0cf8b0 con 0x7f2c0c1594d0 2023-10-06T10:38:55.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.011+0000 7f2c10817700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2c0c1594d0 0x7f2c0c0cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.011+0000 7f2c11018700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2c0c0cbaa0 0x7f2c0c0c5690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50316/0 (socket says 172.21.15.92:50316) 2023-10-06T10:38:55.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.011+0000 7f2c11018700 1 -- 172.21.15.92:0/3320727618 learned_addr learned my addr 172.21.15.92:0/3320727618 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:55.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.012+0000 7f2c10817700 1 -- 172.21.15.92:0/3320727618 >> v1:172.21.15.124:6789/0 conn(0x7f2c0c1598a0 legacy=0x7f2c0c0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:55.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.012+0000 7f2c10817700 1 -- 172.21.15.92:0/3320727618 >> v1:172.21.15.92:6789/0 conn(0x7f2c0c0cbaa0 legacy=0x7f2c0c0c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:55.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.012+0000 7f2c10817700 1 -- 172.21.15.92:0/3320727618 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0c0cfb90 con 0x7f2c0c1594d0 2023-10-06T10:38:55.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.012+0000 7f2c0b7fe700 1 -- 172.21.15.92:0/3320727618 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3669797804 0 0) 0x7f2c0c0cbee0 con 0x7f2c0c0cbaa0 2023-10-06T10:38:55.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.012+0000 7f2c10817700 1 --2- 172.21.15.92:0/3320727618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c0c1594d0 0x7f2c0c0cb560 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009e60 tx=0x7f2bfc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a435d77704d4e307 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:55.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.012+0000 7f2c0b7fe700 1 -- 172.21.15.92:0/3320727618 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc00e070 con 0x7f2c0c1594d0 2023-10-06T10:38:55.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.012+0000 7f2c0b7fe700 1 -- 172.21.15.92:0/3320727618 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bfc00bbf0 con 0x7f2c0c1594d0 2023-10-06T10:38:55.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.012+0000 7f2c0b7fe700 1 -- 172.21.15.92:0/3320727618 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc012430 con 0x7f2c0c1594d0 2023-10-06T10:38:55.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.012+0000 7f2c12a7b700 1 -- 172.21.15.92:0/3320727618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c0c1594d0 msgr2=0x7f2c0c0cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:55.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.012+0000 7f2c12a7b700 1 --2- 172.21.15.92:0/3320727618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c0c1594d0 0x7f2c0c0cb560 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009e60 tx=0x7f2bfc009780 comp rx=0 tx=0).stop 2023-10-06T10:38:55.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.012+0000 7f2c12a7b700 1 -- 172.21.15.92:0/3320727618 shutdown_connections 2023-10-06T10:38:55.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.012+0000 7f2c12a7b700 1 --2- 172.21.15.92:0/3320727618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c0c1594d0 0x7f2c0c0cb560 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.013+0000 7f2c12a7b700 1 -- 172.21.15.92:0/3320727618 >> 172.21.15.92:0/3320727618 conn(0x7f2c0c089e50 msgr2=0x7f2c0c099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:55.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.013+0000 7f2c12a7b700 1 -- 172.21.15.92:0/3320727618 shutdown_connections 2023-10-06T10:38:55.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.013+0000 7f2c12a7b700 1 -- 172.21.15.92:0/3320727618 wait complete. 2023-10-06T10:38:55.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.014+0000 7f2c12a7b700 1 Processor -- start 2023-10-06T10:38:55.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.014+0000 7f2c12a7b700 1 -- start start 2023-10-06T10:38:55.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.014+0000 7f2c12a7b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c0c0cbaa0 0x7f2c0c0c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.014+0000 7f2c12a7b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c0c1594d0 0x7f2c0c16aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.014+0000 7f2c12a7b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c0c1598a0 0x7f2c0c0b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.015+0000 7f2c12a7b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c16b020 con 0x7f2c0c1598a0 2023-10-06T10:38:55.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.015+0000 7f2c12a7b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c0b4e10 con 0x7f2c0c0cbaa0 2023-10-06T10:38:55.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.015+0000 7f2c12a7b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c0b50b0 con 0x7f2c0c1594d0 2023-10-06T10:38:55.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.015+0000 7f2c11018700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c0c1598a0 0x7f2c0c0b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.015+0000 7f2c11018700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c0c1598a0 0x7f2c0c0b5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47066/0 (socket says 172.21.15.92:47066) 2023-10-06T10:38:55.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.015+0000 7f2c11018700 1 -- 172.21.15.92:0/815428693 learned_addr learned my addr 172.21.15.92:0/815428693 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:55.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.015+0000 7f2c10817700 1 --2- 172.21.15.92:0/815428693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c0c0cbaa0 0x7f2c0c0c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.015+0000 7f2c0bfff700 1 --2- 172.21.15.92:0/815428693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c0c1594d0 0x7f2c0c16aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.015+0000 7f2c11018700 1 -- 172.21.15.92:0/815428693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c0c1594d0 msgr2=0x7f2c0c16aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:55.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.015+0000 7f2c11018700 1 --2- 172.21.15.92:0/815428693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c0c1594d0 0x7f2c0c16aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.015+0000 7f2c11018700 1 -- 172.21.15.92:0/815428693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c0c0cbaa0 msgr2=0x7f2c0c0c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:55.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.015+0000 7f2c11018700 1 --2- 172.21.15.92:0/815428693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c0c0cbaa0 0x7f2c0c0c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.015+0000 7f2c11018700 1 -- 172.21.15.92:0/815428693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc009a20 con 0x7f2c0c1598a0 2023-10-06T10:38:55.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.015+0000 7f2c11018700 1 --2- 172.21.15.92:0/815428693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c0c1598a0 0x7f2c0c0b5b50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400c7e0 tx=0x7f2c04007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:55.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.015+0000 7f2c08ff9700 1 -- 172.21.15.92:0/815428693 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c04005c00 con 0x7f2c0c1598a0 2023-10-06T10:38:55.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.015+0000 7f2c08ff9700 1 -- 172.21.15.92:0/815428693 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c04005d80 con 0x7f2c0c1598a0 2023-10-06T10:38:55.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.015+0000 7f2c12a7b700 1 -- 172.21.15.92:0/815428693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0c0b3970 con 0x7f2c0c1598a0 2023-10-06T10:38:55.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.015+0000 7f2c08ff9700 1 -- 172.21.15.92:0/815428693 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c04007320 con 0x7f2c0c1598a0 2023-10-06T10:38:55.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.015+0000 7f2c12a7b700 1 -- 172.21.15.92:0/815428693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0c0b5350 con 0x7f2c0c1598a0 2023-10-06T10:38:55.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.016+0000 7f2c12a7b700 1 -- 172.21.15.92:0/815428693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c0c064a90 con 0x7f2c0c1598a0 2023-10-06T10:38:55.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.017+0000 7f2c08ff9700 1 -- 172.21.15.92:0/815428693 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2c0401b3b0 con 0x7f2c0c1598a0 2023-10-06T10:38:55.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.017+0000 7f2c08ff9700 1 --2- 172.21.15.92:0/815428693 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2bf8066f30 0x7f2bf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.017+0000 7f2c08ff9700 1 -- 172.21.15.92:0/815428693 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f2c04083de0 con 0x7f2c0c1598a0 2023-10-06T10:38:55.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.017+0000 7f2c10817700 1 --2- 172.21.15.92:0/815428693 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2bf8066f30 0x7f2bf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.017+0000 7f2c10817700 1 --2- 172.21.15.92:0/815428693 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2bf8066f30 0x7f2bf80693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0097c0 tx=0x7f2bfc018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:55.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.021+0000 7f2c08ff9700 1 -- 172.21.15.92:0/815428693 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c04050c00 con 0x7f2c0c1598a0 2023-10-06T10:38:55.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.204+0000 7f2c12a7b700 1 -- 172.21.15.92:0/815428693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2c0c0cfe70 con 0x7f2c0c1598a0 2023-10-06T10:38:55.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.204+0000 7f2c08ff9700 1 -- 172.21.15.92:0/815428693 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2c04017070 con 0x7f2c0c1598a0 2023-10-06T10:38:55.206 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T10:38:55.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.206+0000 7f2c12a7b700 1 -- 172.21.15.92:0/815428693 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2bf8066f30 msgr2=0x7f2bf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:55.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.206+0000 7f2c12a7b700 1 --2- 172.21.15.92:0/815428693 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2bf8066f30 0x7f2bf80693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0097c0 tx=0x7f2bfc018000 comp rx=0 tx=0).stop 2023-10-06T10:38:55.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.206+0000 7f2c12a7b700 1 -- 172.21.15.92:0/815428693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c0c1598a0 msgr2=0x7f2c0c0b5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:55.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.206+0000 7f2c12a7b700 1 --2- 172.21.15.92:0/815428693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c0c1598a0 0x7f2c0c0b5b50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400c7e0 tx=0x7f2c04007d90 comp rx=0 tx=0).stop 2023-10-06T10:38:55.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.206+0000 7f2c12a7b700 1 -- 172.21.15.92:0/815428693 shutdown_connections 2023-10-06T10:38:55.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.206+0000 7f2c12a7b700 1 --2- 172.21.15.92:0/815428693 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2bf8066f30 0x7f2bf80693e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.206+0000 7f2c12a7b700 1 --2- 172.21.15.92:0/815428693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c0c1598a0 0x7f2c0c0b5b50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.206+0000 7f2c12a7b700 1 --2- 172.21.15.92:0/815428693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c0c0cbaa0 0x7f2c0c0c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.206+0000 7f2c12a7b700 1 --2- 172.21.15.92:0/815428693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c0c1594d0 0x7f2c0c16aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.206+0000 7f2c12a7b700 1 -- 172.21.15.92:0/815428693 >> 172.21.15.92:0/815428693 conn(0x7f2c0c089e50 msgr2=0x7f2c0c068370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:55.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.206+0000 7f2c12a7b700 1 -- 172.21.15.92:0/815428693 shutdown_connections 2023-10-06T10:38:55.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.206+0000 7f2c12a7b700 1 -- 172.21.15.92:0/815428693 wait complete. 2023-10-06T10:38:55.220 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:38:55.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.332+0000 7fe4f76f3700 1 Processor -- start 2023-10-06T10:38:55.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.333+0000 7fe4f76f3700 1 -- start start 2023-10-06T10:38:55.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.333+0000 7fe4f76f3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe4f0159560 0x7fe4f00aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.333+0000 7fe4f76f3700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4f00b0760 0x7fe4f00b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.333+0000 7fe4f76f3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f00b0c60 con 0x7fe4f00b49a0 2023-10-06T10:38:55.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.333+0000 7fe4f76f3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4f00b0f40 con 0x7fe4f0159560 2023-10-06T10:38:55.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.333+0000 7fe4f76f3700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00b1220 con 0x7fe4f00b0760 2023-10-06T10:38:55.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.333+0000 7fe4f548f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe4f0159560 0x7fe4f00aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.333+0000 7fe4f4c8e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe4f00b49a0 0x7fe4f00aaa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50322/0 (socket says 172.21.15.92:50322) 2023-10-06T10:38:55.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.333+0000 7fe4f4c8e700 1 -- 172.21.15.92:0/81236380 learned_addr learned my addr 172.21.15.92:0/81236380 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:55.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.334+0000 7fe4f5c90700 1 --2- 172.21.15.92:0/81236380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4f00b0760 0x7fe4f00b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.334+0000 7fe4f548f700 1 -- 172.21.15.92:0/81236380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4f00b0760 msgr2=0x7fe4f00b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:55.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.334+0000 7fe4f548f700 1 --2- 172.21.15.92:0/81236380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4f00b0760 0x7fe4f00b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.334+0000 7fe4f548f700 1 -- 172.21.15.92:0/81236380 >> v1:172.21.15.92:6789/0 conn(0x7fe4f00b49a0 legacy=0x7fe4f00aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:55.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.334+0000 7fe4f548f700 1 -- 172.21.15.92:0/81236380 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f00b1500 con 0x7fe4f0159560 2023-10-06T10:38:55.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.334+0000 7fe4e7fff700 1 -- 172.21.15.92:0/81236380 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1509319377 0 0) 0x7fe4f00b0c60 con 0x7fe4f00b49a0 2023-10-06T10:38:55.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.334+0000 7fe4f5c90700 1 --2- 172.21.15.92:0/81236380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4f00b0760 0x7fe4f00b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:55.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.334+0000 7fe4f548f700 1 --2- 172.21.15.92:0/81236380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4f0159560 0x7fe4f00aa530 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009ee0 tx=0x7fe4ec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d97937e1e76d9a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:55.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.334+0000 7fe4e7fff700 1 -- 172.21.15.92:0/81236380 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec00e070 con 0x7fe4f0159560 2023-10-06T10:38:55.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.334+0000 7fe4e7fff700 1 -- 172.21.15.92:0/81236380 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ec00bc10 con 0x7fe4f0159560 2023-10-06T10:38:55.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.334+0000 7fe4e7fff700 1 -- 172.21.15.92:0/81236380 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec012430 con 0x7fe4f0159560 2023-10-06T10:38:55.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.335+0000 7fe4f76f3700 1 -- 172.21.15.92:0/81236380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4f0159560 msgr2=0x7fe4f00aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:55.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.335+0000 7fe4f76f3700 1 --2- 172.21.15.92:0/81236380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4f0159560 0x7fe4f00aa530 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009ee0 tx=0x7fe4ec009780 comp rx=0 tx=0).stop 2023-10-06T10:38:55.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.335+0000 7fe4f76f3700 1 -- 172.21.15.92:0/81236380 shutdown_connections 2023-10-06T10:38:55.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.335+0000 7fe4f76f3700 1 --2- 172.21.15.92:0/81236380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4f00b0760 0x7fe4f00b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.335+0000 7fe4f76f3700 1 --2- 172.21.15.92:0/81236380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4f0159560 0x7fe4f00aa530 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.335+0000 7fe4f76f3700 1 -- 172.21.15.92:0/81236380 >> 172.21.15.92:0/81236380 conn(0x7fe4f00997f0 msgr2=0x7fe4f006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:55.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.335+0000 7fe4f76f3700 1 -- 172.21.15.92:0/81236380 shutdown_connections 2023-10-06T10:38:55.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.335+0000 7fe4f76f3700 1 -- 172.21.15.92:0/81236380 wait complete. 2023-10-06T10:38:55.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.336+0000 7fe4f76f3700 1 Processor -- start 2023-10-06T10:38:55.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.336+0000 7fe4f76f3700 1 -- start start 2023-10-06T10:38:55.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.337+0000 7fe4f76f3700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4f00b0760 0x7fe4f015a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.337+0000 7fe4f76f3700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4f00b49a0 0x7fe4f009da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.337+0000 7fe4f76f3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4f0159560 0x7fe4f0168f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.337+0000 7fe4f76f3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f009df70 con 0x7fe4f0159560 2023-10-06T10:38:55.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.337+0000 7fe4f76f3700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f009a0e0 con 0x7fe4f00b49a0 2023-10-06T10:38:55.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.337+0000 7fe4f76f3700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f015aa20 con 0x7fe4f00b0760 2023-10-06T10:38:55.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.337+0000 7fe4f5c90700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4f0159560 0x7fe4f0168f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.337+0000 7fe4f5c90700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4f0159560 0x7fe4f0168f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47084/0 (socket says 172.21.15.92:47084) 2023-10-06T10:38:55.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.337+0000 7fe4f5c90700 1 -- 172.21.15.92:0/3232786770 learned_addr learned my addr 172.21.15.92:0/3232786770 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:55.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.337+0000 7fe4f548f700 1 --2- 172.21.15.92:0/3232786770 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4f00b0760 0x7fe4f015a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.337+0000 7fe4f4c8e700 1 --2- 172.21.15.92:0/3232786770 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4f00b49a0 0x7fe4f009da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.337+0000 7fe4f5c90700 1 -- 172.21.15.92:0/3232786770 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4f00b0760 msgr2=0x7fe4f015a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:55.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.337+0000 7fe4f5c90700 1 --2- 172.21.15.92:0/3232786770 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4f00b0760 0x7fe4f015a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.337+0000 7fe4f5c90700 1 -- 172.21.15.92:0/3232786770 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4f00b49a0 msgr2=0x7fe4f009da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:55.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.337+0000 7fe4f5c90700 1 --2- 172.21.15.92:0/3232786770 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4f00b49a0 0x7fe4f009da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.337+0000 7fe4f5c90700 1 -- 172.21.15.92:0/3232786770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ec009a20 con 0x7fe4f0159560 2023-10-06T10:38:55.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.337+0000 7fe4f5c90700 1 --2- 172.21.15.92:0/3232786770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4f0159560 0x7fe4f0168f00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800c880 tx=0x7fe4e8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:55.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.337+0000 7fe4e57fa700 1 -- 172.21.15.92:0/3232786770 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8011600 con 0x7fe4f0159560 2023-10-06T10:38:55.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.337+0000 7fe4e57fa700 1 -- 172.21.15.92:0/3232786770 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e8011780 con 0x7fe4f0159560 2023-10-06T10:38:55.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.337+0000 7fe4f76f3700 1 -- 172.21.15.92:0/3232786770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f015ad00 con 0x7fe4f0159560 2023-10-06T10:38:55.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.337+0000 7fe4e57fa700 1 -- 172.21.15.92:0/3232786770 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8011b40 con 0x7fe4f0159560 2023-10-06T10:38:55.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.338+0000 7fe4f76f3700 1 -- 172.21.15.92:0/3232786770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f015b2c0 con 0x7fe4f0159560 2023-10-06T10:38:55.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.338+0000 7fe4f76f3700 1 -- 172.21.15.92:0/3232786770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f0064a90 con 0x7fe4f0159560 2023-10-06T10:38:55.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.339+0000 7fe4e57fa700 1 -- 172.21.15.92:0/3232786770 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe4e8018350 con 0x7fe4f0159560 2023-10-06T10:38:55.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.339+0000 7fe4e57fa700 1 --2- 172.21.15.92:0/3232786770 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe4d4066f30 0x7fe4d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.339+0000 7fe4e57fa700 1 -- 172.21.15.92:0/3232786770 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe4e80848d0 con 0x7fe4f0159560 2023-10-06T10:38:55.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.339+0000 7fe4f548f700 1 --2- 172.21.15.92:0/3232786770 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe4d4066f30 0x7fe4d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.340+0000 7fe4f548f700 1 --2- 172.21.15.92:0/3232786770 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe4d4066f30 0x7fe4d40693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009ee0 tx=0x7fe4ec019000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:55.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.343+0000 7fe4e57fa700 1 -- 172.21.15.92:0/3232786770 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe4e8055180 con 0x7fe4f0159560 2023-10-06T10:38:55.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.527+0000 7fe4f76f3700 1 -- 172.21.15.92:0/3232786770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4f00b17e0 con 0x7fe4f0159560 2023-10-06T10:38:55.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.528+0000 7fe4e57fa700 1 -- 172.21.15.92:0/3232786770 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18707 (secure 0 0 0) 0x7fe4e8054d10 con 0x7fe4f0159560 2023-10-06T10:38:55.530 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:55.530 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:21.760233+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:38:14.368713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T10:38:15.692344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T10:38:19.382553+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:38:55.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.530+0000 7fe4f76f3700 1 -- 172.21.15.92:0/3232786770 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe4d4066f30 msgr2=0x7fe4d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:55.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.530+0000 7fe4f76f3700 1 --2- 172.21.15.92:0/3232786770 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe4d4066f30 0x7fe4d40693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009ee0 tx=0x7fe4ec019000 comp rx=0 tx=0).stop 2023-10-06T10:38:55.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.530+0000 7fe4f76f3700 1 -- 172.21.15.92:0/3232786770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4f0159560 msgr2=0x7fe4f0168f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:55.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.530+0000 7fe4f76f3700 1 --2- 172.21.15.92:0/3232786770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4f0159560 0x7fe4f0168f00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800c880 tx=0x7fe4e8008040 comp rx=0 tx=0).stop 2023-10-06T10:38:55.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.530+0000 7fe4f76f3700 1 -- 172.21.15.92:0/3232786770 shutdown_connections 2023-10-06T10:38:55.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.530+0000 7fe4f76f3700 1 --2- 172.21.15.92:0/3232786770 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe4d4066f30 0x7fe4d40693e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.530+0000 7fe4f76f3700 1 --2- 172.21.15.92:0/3232786770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4f0159560 0x7fe4f0168f00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.530+0000 7fe4f76f3700 1 --2- 172.21.15.92:0/3232786770 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4f00b0760 0x7fe4f015a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.530+0000 7fe4f76f3700 1 --2- 172.21.15.92:0/3232786770 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4f00b49a0 0x7fe4f009da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.530+0000 7fe4f76f3700 1 -- 172.21.15.92:0/3232786770 >> 172.21.15.92:0/3232786770 conn(0x7fe4f00997f0 msgr2=0x7fe4f00683c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:55.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.531+0000 7fe4f76f3700 1 -- 172.21.15.92:0/3232786770 shutdown_connections 2023-10-06T10:38:55.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.531+0000 7fe4f76f3700 1 -- 172.21.15.92:0/3232786770 wait complete. 2023-10-06T10:38:55.544 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T10:38:09.680559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T10:38:10.120786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T10:38:14.368713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T10:38:15.692344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T10:38:19.382553+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T10:38:55.545 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T10:38:55.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.660+0000 7fe3fc7bb700 1 Processor -- start 2023-10-06T10:38:55.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.660+0000 7fe3fc7bb700 1 -- start start 2023-10-06T10:38:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.661+0000 7fe3fc7bb700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe3f41594d0 0x7fe3f40cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.661+0000 7fe3fc7bb700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3f41598a0 0x7fe3f40cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.661+0000 7fe3fc7bb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe3f40ce7e0 0x7fe3f40cec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.661+0000 7fe3fc7bb700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f40cf5d0 con 0x7fe3f40ce7e0 2023-10-06T10:38:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.661+0000 7fe3fc7bb700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f40cf8b0 con 0x7fe3f41594d0 2023-10-06T10:38:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.661+0000 7fe3fc7bb700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f40cfb90 con 0x7fe3f41598a0 2023-10-06T10:38:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.661+0000 7fe3fad58700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe3f40ce7e0 0x7fe3f40cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.661+0000 7fe3fad58700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe3f40ce7e0 0x7fe3f40cec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47096/0 (socket says 172.21.15.92:47096) 2023-10-06T10:38:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.661+0000 7fe3fad58700 1 -- 172.21.15.92:0/1989779936 learned_addr learned my addr 172.21.15.92:0/1989779936 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.661+0000 7fe3f9d56700 1 --2- 172.21.15.92:0/1989779936 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3f41598a0 0x7fe3f40cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.661+0000 7fe3fad58700 1 -- 172.21.15.92:0/1989779936 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3f41598a0 msgr2=0x7fe3f40cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.661+0000 7fe3fa557700 1 --2- 172.21.15.92:0/1989779936 >> v2:172.21.15.124:3300/0 conn(0x7fe3f41594d0 0x7fe3f40cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.661+0000 7fe3fad58700 1 --2- 172.21.15.92:0/1989779936 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3f41598a0 0x7fe3f40cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.661+0000 7fe3fad58700 1 -- 172.21.15.92:0/1989779936 >> v2:172.21.15.124:3300/0 conn(0x7fe3f41594d0 msgr2=0x7fe3f40cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.661+0000 7fe3fad58700 1 --2- 172.21.15.92:0/1989779936 >> v2:172.21.15.124:3300/0 conn(0x7fe3f41594d0 0x7fe3f40cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.661+0000 7fe3fad58700 1 -- 172.21.15.92:0/1989779936 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f40cfe70 con 0x7fe3f40ce7e0 2023-10-06T10:38:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.661+0000 7fe3fad58700 1 --2- 172.21.15.92:0/1989779936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3f40ce7e0 0x7fe3f40cec00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4088f70 tx=0x7fe3ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa3aad973bd0c59 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:55.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.661+0000 7fe3f9555700 1 -- 172.21.15.92:0/1989779936 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec00b860 con 0x7fe3f40ce7e0 2023-10-06T10:38:55.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.661+0000 7fe3f9555700 1 -- 172.21.15.92:0/1989779936 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ec00b9c0 con 0x7fe3f40ce7e0 2023-10-06T10:38:55.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.661+0000 7fe3f9555700 1 -- 172.21.15.92:0/1989779936 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec00bd60 con 0x7fe3f40ce7e0 2023-10-06T10:38:55.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.662+0000 7fe3fc7bb700 1 -- 172.21.15.92:0/1989779936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3f40ce7e0 msgr2=0x7fe3f40cec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:55.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.662+0000 7fe3fc7bb700 1 --2- 172.21.15.92:0/1989779936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3f40ce7e0 0x7fe3f40cec00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4088f70 tx=0x7fe3ec009250 comp rx=0 tx=0).stop 2023-10-06T10:38:55.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.662+0000 7fe3fc7bb700 1 -- 172.21.15.92:0/1989779936 shutdown_connections 2023-10-06T10:38:55.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.662+0000 7fe3fc7bb700 1 --2- 172.21.15.92:0/1989779936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3f40ce7e0 0x7fe3f40cec00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.662+0000 7fe3fc7bb700 1 --2- 172.21.15.92:0/1989779936 >> v2:172.21.15.124:3300/0 conn(0x7fe3f41594d0 0x7fe3f40cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.662+0000 7fe3fc7bb700 1 --2- 172.21.15.92:0/1989779936 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3f41598a0 0x7fe3f40cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.662+0000 7fe3fc7bb700 1 -- 172.21.15.92:0/1989779936 >> 172.21.15.92:0/1989779936 conn(0x7fe3f4089e50 msgr2=0x7fe3f4099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:55.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.662+0000 7fe3fc7bb700 1 -- 172.21.15.92:0/1989779936 shutdown_connections 2023-10-06T10:38:55.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.662+0000 7fe3fc7bb700 1 -- 172.21.15.92:0/1989779936 wait complete. 2023-10-06T10:38:55.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.663+0000 7fe3fc7bb700 1 Processor -- start 2023-10-06T10:38:55.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.663+0000 7fe3fc7bb700 1 -- start start 2023-10-06T10:38:55.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.664+0000 7fe3fc7bb700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3f40ce7e0 0x7fe3f41582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.664+0000 7fe3fc7bb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3f41594d0 0x7fe3f414f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.664+0000 7fe3fc7bb700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3f41598a0 0x7fe3f4156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.664+0000 7fe3fc7bb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f4157010 con 0x7fe3f41594d0 2023-10-06T10:38:55.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.664+0000 7fe3fc7bb700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f4157e50 con 0x7fe3f40ce7e0 2023-10-06T10:38:55.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.664+0000 7fe3fc7bb700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f4157240 con 0x7fe3f41598a0 2023-10-06T10:38:55.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.664+0000 7fe3f9d56700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3f41594d0 0x7fe3f414f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.664+0000 7fe3f9d56700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3f41594d0 0x7fe3f414f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47106/0 (socket says 172.21.15.92:47106) 2023-10-06T10:38:55.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.664+0000 7fe3f9d56700 1 -- 172.21.15.92:0/2324520866 learned_addr learned my addr 172.21.15.92:0/2324520866 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:55.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.664+0000 7fe3f9d56700 1 -- 172.21.15.92:0/2324520866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3f41598a0 msgr2=0x7fe3f4156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:55.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.664+0000 7fe3fad58700 1 --2- 172.21.15.92:0/2324520866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3f41598a0 0x7fe3f4156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.664+0000 7fe3f9d56700 1 --2- 172.21.15.92:0/2324520866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3f41598a0 0x7fe3f4156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.664+0000 7fe3fa557700 1 --2- 172.21.15.92:0/2324520866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3f40ce7e0 0x7fe3f41582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.664+0000 7fe3f9d56700 1 -- 172.21.15.92:0/2324520866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3f40ce7e0 msgr2=0x7fe3f41582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:55.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.664+0000 7fe3f9d56700 1 --2- 172.21.15.92:0/2324520866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3f40ce7e0 0x7fe3f41582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.664+0000 7fe3f9d56700 1 -- 172.21.15.92:0/2324520866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec0094f0 con 0x7fe3f41594d0 2023-10-06T10:38:55.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.664+0000 7fe3f9d56700 1 --2- 172.21.15.92:0/2324520866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3f41594d0 0x7fe3f414f630 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400c880 tx=0x7fe3e4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:55.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.664+0000 7fe3ea7fc700 1 -- 172.21.15.92:0/2324520866 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4011600 con 0x7fe3f41594d0 2023-10-06T10:38:55.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.664+0000 7fe3ea7fc700 1 -- 172.21.15.92:0/2324520866 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e4011780 con 0x7fe3f41594d0 2023-10-06T10:38:55.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.664+0000 7fe3fc7bb700 1 -- 172.21.15.92:0/2324520866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f4157520 con 0x7fe3f41594d0 2023-10-06T10:38:55.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.664+0000 7fe3ea7fc700 1 -- 172.21.15.92:0/2324520866 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4011b40 con 0x7fe3f41594d0 2023-10-06T10:38:55.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.665+0000 7fe3fc7bb700 1 -- 172.21.15.92:0/2324520866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f4157ae0 con 0x7fe3f41594d0 2023-10-06T10:38:55.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.665+0000 7fe3fc7bb700 1 -- 172.21.15.92:0/2324520866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f4157fc0 con 0x7fe3f41594d0 2023-10-06T10:38:55.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.666+0000 7fe3ea7fc700 1 -- 172.21.15.92:0/2324520866 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe3e4010350 con 0x7fe3f41594d0 2023-10-06T10:38:55.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.666+0000 7fe3ea7fc700 1 --2- 172.21.15.92:0/2324520866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe3d8066f30 0x7fe3d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.666+0000 7fe3ea7fc700 1 -- 172.21.15.92:0/2324520866 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe3e4084770 con 0x7fe3f41594d0 2023-10-06T10:38:55.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.666+0000 7fe3fa557700 1 --2- 172.21.15.92:0/2324520866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe3d8066f30 0x7fe3d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.667+0000 7fe3fa557700 1 --2- 172.21.15.92:0/2324520866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe3d8066f30 0x7fe3d80693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4159fd0 tx=0x7fe3f0007530 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:55.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.671+0000 7fe3ea7fc700 1 -- 172.21.15.92:0/2324520866 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3e4051590 con 0x7fe3f41594d0 2023-10-06T10:38:55.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.852+0000 7fe3fc7bb700 1 -- 172.21.15.92:0/2324520866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe3f4064a90 con 0x7fe3f41594d0 2023-10-06T10:38:55.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.852+0000 7fe3ea7fc700 1 -- 172.21.15.92:0/2324520866 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe3e4054bb0 con 0x7fe3f41594d0 2023-10-06T10:38:55.854 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T10:38:55.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.854+0000 7fe3fc7bb700 1 -- 172.21.15.92:0/2324520866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe3d8066f30 msgr2=0x7fe3d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:55.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.854+0000 7fe3fc7bb700 1 --2- 172.21.15.92:0/2324520866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe3d8066f30 0x7fe3d80693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4159fd0 tx=0x7fe3f0007530 comp rx=0 tx=0).stop 2023-10-06T10:38:55.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.854+0000 7fe3fc7bb700 1 -- 172.21.15.92:0/2324520866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3f41594d0 msgr2=0x7fe3f414f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:55.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.854+0000 7fe3fc7bb700 1 --2- 172.21.15.92:0/2324520866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3f41594d0 0x7fe3f414f630 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400c880 tx=0x7fe3e4008040 comp rx=0 tx=0).stop 2023-10-06T10:38:55.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.854+0000 7fe3fc7bb700 1 -- 172.21.15.92:0/2324520866 shutdown_connections 2023-10-06T10:38:55.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.854+0000 7fe3fc7bb700 1 --2- 172.21.15.92:0/2324520866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe3d8066f30 0x7fe3d80693e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.854+0000 7fe3fc7bb700 1 --2- 172.21.15.92:0/2324520866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3f41598a0 0x7fe3f4156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.854+0000 7fe3fc7bb700 1 --2- 172.21.15.92:0/2324520866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3f40ce7e0 0x7fe3f41582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.854+0000 7fe3fc7bb700 1 --2- 172.21.15.92:0/2324520866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3f41594d0 0x7fe3f414f630 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.854+0000 7fe3fc7bb700 1 -- 172.21.15.92:0/2324520866 >> 172.21.15.92:0/2324520866 conn(0x7fe3f4089e50 msgr2=0x7fe3f414c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:55.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.854+0000 7fe3fc7bb700 1 -- 172.21.15.92:0/2324520866 shutdown_connections 2023-10-06T10:38:55.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.854+0000 7fe3fc7bb700 1 -- 172.21.15.92:0/2324520866 wait complete. 2023-10-06T10:38:55.868 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T10:38:55.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.983+0000 7f6915beb700 1 Processor -- start 2023-10-06T10:38:55.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.983+0000 7f6915beb700 1 -- start start 2023-10-06T10:38:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.984+0000 7f6915beb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f69100b2620 0x7f69100cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.984+0000 7f6915beb700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69100b29f0 0x7f69100cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.984+0000 7f6915beb700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f69100ceaa0 0x7f69100cef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.984+0000 7f6915beb700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69100cc970 con 0x7f69100b2620 2023-10-06T10:38:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.984+0000 7f6915beb700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69100cf480 con 0x7f69100ceaa0 2023-10-06T10:38:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.984+0000 7f6915beb700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69100cbd60 con 0x7f69100b29f0 2023-10-06T10:38:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.984+0000 7f6914be9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f69100b2620 0x7f69100cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.984+0000 7f6914be9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f69100b2620 0x7f69100cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47114/0 (socket says 172.21.15.92:47114) 2023-10-06T10:38:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.984+0000 7f6914be9700 1 -- 172.21.15.92:0/3843863474 learned_addr learned my addr 172.21.15.92:0/3843863474 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.984+0000 7f69153ea700 1 --2- 172.21.15.92:0/3843863474 >> v2:172.21.15.124:3300/0 conn(0x7f69100ceaa0 0x7f69100cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.984+0000 7f690ffff700 1 --2- 172.21.15.92:0/3843863474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69100b29f0 0x7f69100cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.984+0000 7f6914be9700 1 -- 172.21.15.92:0/3843863474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69100b29f0 msgr2=0x7f69100cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.984+0000 7f6914be9700 1 --2- 172.21.15.92:0/3843863474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69100b29f0 0x7f69100cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.984+0000 7f6914be9700 1 -- 172.21.15.92:0/3843863474 >> v2:172.21.15.124:3300/0 conn(0x7f69100ceaa0 msgr2=0x7f69100cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.984+0000 7f6914be9700 1 --2- 172.21.15.92:0/3843863474 >> v2:172.21.15.124:3300/0 conn(0x7f69100ceaa0 0x7f69100cef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.984+0000 7f6914be9700 1 -- 172.21.15.92:0/3843863474 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69100cc040 con 0x7f69100b2620 2023-10-06T10:38:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.984+0000 7f690ffff700 1 --2- 172.21.15.92:0/3843863474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69100b29f0 0x7f69100cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.984+0000 7f69153ea700 1 --2- 172.21.15.92:0/3843863474 >> v2:172.21.15.124:3300/0 conn(0x7f69100ceaa0 0x7f69100cef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.985+0000 7f6914be9700 1 --2- 172.21.15.92:0/3843863474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69100b2620 0x7f69100cccb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6910089650 tx=0x7f6908009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c96852989a88289 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.985+0000 7f690f7fe700 1 -- 172.21.15.92:0/3843863474 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6908005c10 con 0x7f69100b2620 2023-10-06T10:38:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.985+0000 7f690f7fe700 1 -- 172.21.15.92:0/3843863474 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6908005d70 con 0x7f69100b2620 2023-10-06T10:38:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.985+0000 7f690f7fe700 1 -- 172.21.15.92:0/3843863474 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690800b880 con 0x7f69100b2620 2023-10-06T10:38:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.985+0000 7f6915beb700 1 -- 172.21.15.92:0/3843863474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69100b2620 msgr2=0x7f69100cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.985+0000 7f6915beb700 1 --2- 172.21.15.92:0/3843863474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69100b2620 0x7f69100cccb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6910089650 tx=0x7f6908009220 comp rx=0 tx=0).stop 2023-10-06T10:38:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.985+0000 7f6915beb700 1 -- 172.21.15.92:0/3843863474 shutdown_connections 2023-10-06T10:38:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.985+0000 7f6915beb700 1 --2- 172.21.15.92:0/3843863474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69100b2620 0x7f69100cccb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.985+0000 7f6915beb700 1 --2- 172.21.15.92:0/3843863474 >> v2:172.21.15.124:3300/0 conn(0x7f69100ceaa0 0x7f69100cef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.985+0000 7f6915beb700 1 --2- 172.21.15.92:0/3843863474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69100b29f0 0x7f69100cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.985+0000 7f6915beb700 1 -- 172.21.15.92:0/3843863474 >> 172.21.15.92:0/3843863474 conn(0x7f6910099520 msgr2=0x7f691006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.985+0000 7f6915beb700 1 -- 172.21.15.92:0/3843863474 shutdown_connections 2023-10-06T10:38:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.985+0000 7f6915beb700 1 -- 172.21.15.92:0/3843863474 wait complete. 2023-10-06T10:38:55.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.986+0000 7f6915beb700 1 Processor -- start 2023-10-06T10:38:55.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.987+0000 7f6915beb700 1 -- start start 2023-10-06T10:38:55.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.987+0000 7f6915beb700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69100b2620 0x7f69100b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.987+0000 7f6915beb700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69100b29f0 0x7f6910153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.987+0000 7f6915beb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69100ceaa0 0x7f69100b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.987+0000 7f6915beb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69100c6230 con 0x7f69100ceaa0 2023-10-06T10:38:55.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.987+0000 7f6915beb700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69100b4640 con 0x7f69100b2620 2023-10-06T10:38:55.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.987+0000 7f6915beb700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69100b4920 con 0x7f69100b29f0 2023-10-06T10:38:55.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.987+0000 7f69153ea700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69100ceaa0 0x7f69100b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.987+0000 7f69153ea700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69100ceaa0 0x7f69100b56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47122/0 (socket says 172.21.15.92:47122) 2023-10-06T10:38:55.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.987+0000 7f69153ea700 1 -- 172.21.15.92:0/1504628296 learned_addr learned my addr 172.21.15.92:0/1504628296 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:55.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.988+0000 7f690ffff700 1 --2- 172.21.15.92:0/1504628296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69100b29f0 0x7f6910153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.988+0000 7f6914be9700 1 --2- 172.21.15.92:0/1504628296 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69100b2620 0x7f69100b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.988+0000 7f69153ea700 1 -- 172.21.15.92:0/1504628296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69100b29f0 msgr2=0x7f6910153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:55.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.988+0000 7f69153ea700 1 --2- 172.21.15.92:0/1504628296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69100b29f0 0x7f6910153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.988+0000 7f69153ea700 1 -- 172.21.15.92:0/1504628296 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69100b2620 msgr2=0x7f69100b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:55.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.988+0000 7f69153ea700 1 --2- 172.21.15.92:0/1504628296 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69100b2620 0x7f69100b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:55.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.988+0000 7f69153ea700 1 -- 172.21.15.92:0/1504628296 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6908009480 con 0x7f69100ceaa0 2023-10-06T10:38:55.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.988+0000 7f69153ea700 1 --2- 172.21.15.92:0/1504628296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69100ceaa0 0x7f69100b56e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f68f800c880 tx=0x7f68f8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:55.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.988+0000 7f690cff9700 1 -- 172.21.15.92:0/1504628296 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8011600 con 0x7f69100ceaa0 2023-10-06T10:38:55.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.988+0000 7f690cff9700 1 -- 172.21.15.92:0/1504628296 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f8011760 con 0x7f69100ceaa0 2023-10-06T10:38:55.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.988+0000 7f690cff9700 1 -- 172.21.15.92:0/1504628296 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8011b20 con 0x7f69100ceaa0 2023-10-06T10:38:55.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.988+0000 7f6915beb700 1 -- 172.21.15.92:0/1504628296 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69100b4c00 con 0x7f69100ceaa0 2023-10-06T10:38:55.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.988+0000 7f6915beb700 1 -- 172.21.15.92:0/1504628296 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69100b51c0 con 0x7f69100ceaa0 2023-10-06T10:38:55.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.989+0000 7f6915beb700 1 -- 172.21.15.92:0/1504628296 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69100b53e0 con 0x7f69100ceaa0 2023-10-06T10:38:55.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.989+0000 7f690cff9700 1 -- 172.21.15.92:0/1504628296 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f68f80193b0 con 0x7f69100ceaa0 2023-10-06T10:38:55.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.990+0000 7f690cff9700 1 --2- 172.21.15.92:0/1504628296 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f68f0066f30 0x7f68f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:55.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.990+0000 7f6914be9700 1 --2- 172.21.15.92:0/1504628296 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f68f0066f30 0x7f68f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:55.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.990+0000 7f690cff9700 1 -- 172.21.15.92:0/1504628296 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f68f8084930 con 0x7f69100ceaa0 2023-10-06T10:38:55.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.990+0000 7f6914be9700 1 --2- 172.21.15.92:0/1504628296 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f68f0066f30 0x7f68f00693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6908009260 tx=0x7f6908016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:55.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:55.994+0000 7f690cff9700 1 -- 172.21.15.92:0/1504628296 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68f80551e0 con 0x7f69100ceaa0 2023-10-06T10:38:56.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.176+0000 7f6915beb700 1 -- 172.21.15.92:0/1504628296 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f691005f230 con 0x7f69100ceaa0 2023-10-06T10:38:56.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.177+0000 7f690cff9700 1 -- 172.21.15.92:0/1504628296 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f68f8054d70 con 0x7f69100ceaa0 2023-10-06T10:38:56.179 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T10:38:56.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.179+0000 7f6915beb700 1 -- 172.21.15.92:0/1504628296 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f68f0066f30 msgr2=0x7f68f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:56.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.179+0000 7f6915beb700 1 --2- 172.21.15.92:0/1504628296 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f68f0066f30 0x7f68f00693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6908009260 tx=0x7f6908016000 comp rx=0 tx=0).stop 2023-10-06T10:38:56.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.179+0000 7f6915beb700 1 -- 172.21.15.92:0/1504628296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69100ceaa0 msgr2=0x7f69100b56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:56.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.179+0000 7f6915beb700 1 --2- 172.21.15.92:0/1504628296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69100ceaa0 0x7f69100b56e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f68f800c880 tx=0x7f68f8008040 comp rx=0 tx=0).stop 2023-10-06T10:38:56.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.179+0000 7f6915beb700 1 -- 172.21.15.92:0/1504628296 shutdown_connections 2023-10-06T10:38:56.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.179+0000 7f6915beb700 1 --2- 172.21.15.92:0/1504628296 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f68f0066f30 0x7f68f00693e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.179+0000 7f6915beb700 1 --2- 172.21.15.92:0/1504628296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69100ceaa0 0x7f69100b56e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.179+0000 7f6915beb700 1 --2- 172.21.15.92:0/1504628296 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69100b2620 0x7f69100b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.179+0000 7f6915beb700 1 --2- 172.21.15.92:0/1504628296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69100b29f0 0x7f6910153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.179+0000 7f6915beb700 1 -- 172.21.15.92:0/1504628296 >> 172.21.15.92:0/1504628296 conn(0x7f6910099520 msgr2=0x7f69100c5350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:56.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.179+0000 7f6915beb700 1 -- 172.21.15.92:0/1504628296 shutdown_connections 2023-10-06T10:38:56.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.179+0000 7f6915beb700 1 -- 172.21.15.92:0/1504628296 wait complete. 2023-10-06T10:38:56.192 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T10:38:56.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.307+0000 7f61bbc7b700 1 Processor -- start 2023-10-06T10:38:56.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.307+0000 7f61bbc7b700 1 -- start start 2023-10-06T10:38:56.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.308+0000 7f61bbc7b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f61b41594d0 0x7f61b40cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:56.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.308+0000 7f61bbc7b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f61b40cbaa0 0x7f61b40ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:56.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.308+0000 7f61bbc7b700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b40cbee0 con 0x7f61b41598a0 2023-10-06T10:38:56.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.308+0000 7f61bbc7b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b40cf5d0 con 0x7f61b40cbaa0 2023-10-06T10:38:56.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.308+0000 7f61bbc7b700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b40cf8b0 con 0x7f61b41594d0 2023-10-06T10:38:56.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.308+0000 7f61ba218700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f61b40cbaa0 0x7f61b40ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:56.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.308+0000 7f61ba218700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f61b40cbaa0 0x7f61b40ceef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47136/0 (socket says 172.21.15.92:47136) 2023-10-06T10:38:56.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.308+0000 7f61ba218700 1 -- 172.21.15.92:0/3161819339 learned_addr learned my addr 172.21.15.92:0/3161819339 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:56.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.308+0000 7f61b9a17700 1 --2- 172.21.15.92:0/3161819339 >> v2:172.21.15.124:3300/0 conn(0x7f61b41594d0 0x7f61b40cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:56.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.308+0000 7f61ba218700 1 -- 172.21.15.92:0/3161819339 >> v2:172.21.15.124:3300/0 conn(0x7f61b41594d0 msgr2=0x7f61b40cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:56.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.308+0000 7f61ba218700 1 --2- 172.21.15.92:0/3161819339 >> v2:172.21.15.124:3300/0 conn(0x7f61b41594d0 0x7f61b40cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.308+0000 7f61ba218700 1 -- 172.21.15.92:0/3161819339 >> v1:172.21.15.124:6789/0 conn(0x7f61b41598a0 legacy=0x7f61b40ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:56.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.308+0000 7f61ba218700 1 -- 172.21.15.92:0/3161819339 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b40cfb90 con 0x7f61b40cbaa0 2023-10-06T10:38:56.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.308+0000 7f61b9a17700 1 --2- 172.21.15.92:0/3161819339 >> v2:172.21.15.124:3300/0 conn(0x7f61b41594d0 0x7f61b40cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:56.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.308+0000 7f61ba218700 1 --2- 172.21.15.92:0/3161819339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61b40cbaa0 0x7f61b40ceef0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f61b4088f70 tx=0x7f61b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c99c4446be1f2f2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:56.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.308+0000 7f61b8a15700 1 -- 172.21.15.92:0/3161819339 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b000b860 con 0x7f61b40cbaa0 2023-10-06T10:38:56.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.309+0000 7f61b8a15700 1 -- 172.21.15.92:0/3161819339 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b000b9c0 con 0x7f61b40cbaa0 2023-10-06T10:38:56.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.309+0000 7f61b8a15700 1 -- 172.21.15.92:0/3161819339 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b000bd60 con 0x7f61b40cbaa0 2023-10-06T10:38:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.309+0000 7f61bbc7b700 1 -- 172.21.15.92:0/3161819339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61b40cbaa0 msgr2=0x7f61b40ceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.309+0000 7f61bbc7b700 1 --2- 172.21.15.92:0/3161819339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61b40cbaa0 0x7f61b40ceef0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f61b4088f70 tx=0x7f61b0009250 comp rx=0 tx=0).stop 2023-10-06T10:38:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.309+0000 7f61bbc7b700 1 -- 172.21.15.92:0/3161819339 shutdown_connections 2023-10-06T10:38:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.309+0000 7f61bbc7b700 1 --2- 172.21.15.92:0/3161819339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61b40cbaa0 0x7f61b40ceef0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.309+0000 7f61bbc7b700 1 --2- 172.21.15.92:0/3161819339 >> v2:172.21.15.124:3300/0 conn(0x7f61b41594d0 0x7f61b40cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.309+0000 7f61bbc7b700 1 -- 172.21.15.92:0/3161819339 >> 172.21.15.92:0/3161819339 conn(0x7f61b4089e50 msgr2=0x7f61b4099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.309+0000 7f61bbc7b700 1 -- 172.21.15.92:0/3161819339 shutdown_connections 2023-10-06T10:38:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.309+0000 7f61bbc7b700 1 -- 172.21.15.92:0/3161819339 wait complete. 2023-10-06T10:38:56.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.310+0000 7f61bbc7b700 1 Processor -- start 2023-10-06T10:38:56.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.310+0000 7f61bbc7b700 1 -- start start 2023-10-06T10:38:56.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.311+0000 7f61bbc7b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61b40cbaa0 0x7f61b4152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:56.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.311+0000 7f61bbc7b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61b41594d0 0x7f61b415d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:56.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.311+0000 7f61bbc7b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61b41598a0 0x7f61b415d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:56.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.311+0000 7f61bbc7b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b4159ca0 con 0x7f61b41594d0 2023-10-06T10:38:56.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.311+0000 7f61bbc7b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b40c8250 con 0x7f61b41598a0 2023-10-06T10:38:56.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.311+0000 7f61bbc7b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b40c8530 con 0x7f61b40cbaa0 2023-10-06T10:38:56.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.311+0000 7f61b9216700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61b41594d0 0x7f61b415d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:56.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.311+0000 7f61b9216700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61b41594d0 0x7f61b415d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47138/0 (socket says 172.21.15.92:47138) 2023-10-06T10:38:56.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.311+0000 7f61b9216700 1 -- 172.21.15.92:0/2267578338 learned_addr learned my addr 172.21.15.92:0/2267578338 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:56.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.311+0000 7f61ba218700 1 --2- 172.21.15.92:0/2267578338 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61b41598a0 0x7f61b415d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:56.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.311+0000 7f61b9a17700 1 --2- 172.21.15.92:0/2267578338 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61b40cbaa0 0x7f61b4152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:56.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.311+0000 7f61b9216700 1 -- 172.21.15.92:0/2267578338 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61b40cbaa0 msgr2=0x7f61b4152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:56.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.311+0000 7f61b9216700 1 --2- 172.21.15.92:0/2267578338 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61b40cbaa0 0x7f61b4152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.311+0000 7f61b9216700 1 -- 172.21.15.92:0/2267578338 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61b41598a0 msgr2=0x7f61b415d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:56.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.311+0000 7f61b9216700 1 --2- 172.21.15.92:0/2267578338 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61b41598a0 0x7f61b415d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.311+0000 7f61b9216700 1 -- 172.21.15.92:0/2267578338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b00094f0 con 0x7f61b41594d0 2023-10-06T10:38:56.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.312+0000 7f61b9216700 1 --2- 172.21.15.92:0/2267578338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61b41594d0 0x7f61b415d430 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f61a400c4c0 tx=0x7f61a4007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:56.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.312+0000 7f61a9ffb700 1 -- 172.21.15.92:0/2267578338 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a400ce70 con 0x7f61b41594d0 2023-10-06T10:38:56.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.312+0000 7f61a9ffb700 1 -- 172.21.15.92:0/2267578338 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a4011590 con 0x7f61b41594d0 2023-10-06T10:38:56.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.312+0000 7f61bbc7b700 1 -- 172.21.15.92:0/2267578338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b40c8810 con 0x7f61b41594d0 2023-10-06T10:38:56.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.312+0000 7f61a9ffb700 1 -- 172.21.15.92:0/2267578338 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a4011970 con 0x7f61b41594d0 2023-10-06T10:38:56.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.312+0000 7f61bbc7b700 1 -- 172.21.15.92:0/2267578338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b40c8dd0 con 0x7f61b41594d0 2023-10-06T10:38:56.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.313+0000 7f61bbc7b700 1 -- 172.21.15.92:0/2267578338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61b4064a90 con 0x7f61b41594d0 2023-10-06T10:38:56.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.313+0000 7f61a9ffb700 1 -- 172.21.15.92:0/2267578338 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f61a4011d20 con 0x7f61b41594d0 2023-10-06T10:38:56.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.314+0000 7f61a9ffb700 1 --2- 172.21.15.92:0/2267578338 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6198066f30 0x7f61980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:56.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.314+0000 7f61a9ffb700 1 -- 172.21.15.92:0/2267578338 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f61a4083c60 con 0x7f61b41594d0 2023-10-06T10:38:56.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.314+0000 7f61b9a17700 1 --2- 172.21.15.92:0/2267578338 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6198066f30 0x7f61980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:56.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.314+0000 7f61b9a17700 1 --2- 172.21.15.92:0/2267578338 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6198066f30 0x7f61980693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f61ac007740 tx=0x7f61ac0099d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:56.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.318+0000 7f61a9ffb700 1 -- 172.21.15.92:0/2267578338 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61a4050a80 con 0x7f61b41594d0 2023-10-06T10:38:56.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.503+0000 7f61bbc7b700 1 -- 172.21.15.92:0/2267578338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f61b40cfe70 con 0x7f61b41594d0 2023-10-06T10:38:56.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.503+0000 7f61a9ffb700 1 -- 172.21.15.92:0/2267578338 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f61a4017020 con 0x7f61b41594d0 2023-10-06T10:38:56.505 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T10:38:56.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.505+0000 7f61bbc7b700 1 -- 172.21.15.92:0/2267578338 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6198066f30 msgr2=0x7f61980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:56.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.505+0000 7f61bbc7b700 1 --2- 172.21.15.92:0/2267578338 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6198066f30 0x7f61980693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f61ac007740 tx=0x7f61ac0099d0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.505+0000 7f61bbc7b700 1 -- 172.21.15.92:0/2267578338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61b41594d0 msgr2=0x7f61b415d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:56.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.505+0000 7f61bbc7b700 1 --2- 172.21.15.92:0/2267578338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61b41594d0 0x7f61b415d430 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f61a400c4c0 tx=0x7f61a4007ef0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.505+0000 7f61bbc7b700 1 -- 172.21.15.92:0/2267578338 shutdown_connections 2023-10-06T10:38:56.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.505+0000 7f61bbc7b700 1 --2- 172.21.15.92:0/2267578338 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6198066f30 0x7f61980693e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.505+0000 7f61bbc7b700 1 --2- 172.21.15.92:0/2267578338 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61b41598a0 0x7f61b415d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.505+0000 7f61bbc7b700 1 --2- 172.21.15.92:0/2267578338 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61b40cbaa0 0x7f61b4152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.505+0000 7f61bbc7b700 1 --2- 172.21.15.92:0/2267578338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61b41594d0 0x7f61b415d430 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.505+0000 7f61bbc7b700 1 -- 172.21.15.92:0/2267578338 >> 172.21.15.92:0/2267578338 conn(0x7f61b4089e50 msgr2=0x7f61b4099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:56.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.505+0000 7f61bbc7b700 1 -- 172.21.15.92:0/2267578338 shutdown_connections 2023-10-06T10:38:56.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.505+0000 7f61bbc7b700 1 -- 172.21.15.92:0/2267578338 wait complete. 2023-10-06T10:38:56.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T10:38:56.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.636+0000 7f512ddce700 1 Processor -- start 2023-10-06T10:38:56.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.636+0000 7f512ddce700 1 -- start start 2023-10-06T10:38:56.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.637+0000 7f512ddce700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f51281594d0 0x7f51280cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:56.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.637+0000 7f512ddce700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f51280cbaa0 0x7f51280ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:56.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.637+0000 7f512ddce700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51280cbee0 con 0x7f51281598a0 2023-10-06T10:38:56.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.637+0000 7f512ddce700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51280cf5d0 con 0x7f51281594d0 2023-10-06T10:38:56.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.637+0000 7f512ddce700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51280cf8b0 con 0x7f51280cbaa0 2023-10-06T10:38:56.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.637+0000 7f51277fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f51281594d0 0x7f51280cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:56.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.637+0000 7f511ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f51281598a0 0x7f51280ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50330/0 (socket says 172.21.15.92:50330) 2023-10-06T10:38:56.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.637+0000 7f511ffff700 1 -- 172.21.15.92:0/1649401936 learned_addr learned my addr 172.21.15.92:0/1649401936 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:56.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.638+0000 7f51277fe700 1 -- 172.21.15.92:0/1649401936 >> v2:172.21.15.124:3300/0 conn(0x7f51280cbaa0 msgr2=0x7f51280ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:56.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.638+0000 7f5127fff700 1 --2- 172.21.15.92:0/1649401936 >> v2:172.21.15.124:3300/0 conn(0x7f51280cbaa0 0x7f51280ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:56.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.638+0000 7f51277fe700 1 --2- 172.21.15.92:0/1649401936 >> v2:172.21.15.124:3300/0 conn(0x7f51280cbaa0 0x7f51280ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.638+0000 7f51277fe700 1 -- 172.21.15.92:0/1649401936 >> v1:172.21.15.92:6789/0 conn(0x7f51281598a0 legacy=0x7f51280ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:56.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.638+0000 7f5126ffd700 1 -- 172.21.15.92:0/1649401936 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968950131 0 0) 0x7f51280cbee0 con 0x7f51281598a0 2023-10-06T10:38:56.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.638+0000 7f51277fe700 1 -- 172.21.15.92:0/1649401936 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51280cfb90 con 0x7f51281594d0 2023-10-06T10:38:56.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.638+0000 7f5127fff700 1 --2- 172.21.15.92:0/1649401936 >> v2:172.21.15.124:3300/0 conn(0x7f51280cbaa0 0x7f51280ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:56.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.638+0000 7f51277fe700 1 --2- 172.21.15.92:0/1649401936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51281594d0 0x7f51280cb560 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f512806f950 tx=0x7f51180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4ca080ee492a77 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:56.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.638+0000 7f5126ffd700 1 -- 172.21.15.92:0/1649401936 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511800b6b0 con 0x7f51281594d0 2023-10-06T10:38:56.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.638+0000 7f5126ffd700 1 -- 172.21.15.92:0/1649401936 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511800b810 con 0x7f51281594d0 2023-10-06T10:38:56.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.638+0000 7f5126ffd700 1 -- 172.21.15.92:0/1649401936 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511800bbb0 con 0x7f51281594d0 2023-10-06T10:38:56.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.639+0000 7f512ddce700 1 -- 172.21.15.92:0/1649401936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51281594d0 msgr2=0x7f51280cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:56.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.639+0000 7f512ddce700 1 --2- 172.21.15.92:0/1649401936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51281594d0 0x7f51280cb560 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f512806f950 tx=0x7f51180091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.639+0000 7f512ddce700 1 -- 172.21.15.92:0/1649401936 shutdown_connections 2023-10-06T10:38:56.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.639+0000 7f512ddce700 1 --2- 172.21.15.92:0/1649401936 >> v2:172.21.15.124:3300/0 conn(0x7f51280cbaa0 0x7f51280ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.639+0000 7f512ddce700 1 --2- 172.21.15.92:0/1649401936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51281594d0 0x7f51280cb560 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.639+0000 7f512ddce700 1 -- 172.21.15.92:0/1649401936 >> 172.21.15.92:0/1649401936 conn(0x7f5128089e50 msgr2=0x7f5128099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:56.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.639+0000 7f512ddce700 1 -- 172.21.15.92:0/1649401936 shutdown_connections 2023-10-06T10:38:56.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.639+0000 7f512ddce700 1 -- 172.21.15.92:0/1649401936 wait complete. 2023-10-06T10:38:56.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.640+0000 7f512ddce700 1 Processor -- start 2023-10-06T10:38:56.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.640+0000 7f512ddce700 1 -- start start 2023-10-06T10:38:56.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.640+0000 7f512ddce700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51280cbaa0 0x7f51281562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:56.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.640+0000 7f512ddce700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51281594d0 0x7f51280b5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:56.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.640+0000 7f512ddce700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51281598a0 0x7f5128069730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:56.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.640+0000 7f512ddce700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51281567f0 con 0x7f51280cbaa0 2023-10-06T10:38:56.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.640+0000 7f512ddce700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5128155e20 con 0x7f51281594d0 2023-10-06T10:38:56.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.640+0000 7f512ddce700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5128155210 con 0x7f51281598a0 2023-10-06T10:38:56.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.640+0000 7f51277fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51280cbaa0 0x7f51281562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:56.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.640+0000 7f51277fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51280cbaa0 0x7f51281562b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47152/0 (socket says 172.21.15.92:47152) 2023-10-06T10:38:56.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.640+0000 7f51277fe700 1 -- 172.21.15.92:0/2176637050 learned_addr learned my addr 172.21.15.92:0/2176637050 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:56.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.641+0000 7f511ffff700 1 --2- 172.21.15.92:0/2176637050 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51281594d0 0x7f51280b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:56.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.641+0000 7f5127fff700 1 --2- 172.21.15.92:0/2176637050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51281598a0 0x7f5128069730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:56.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.641+0000 7f51277fe700 1 -- 172.21.15.92:0/2176637050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51281598a0 msgr2=0x7f5128069730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:56.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.641+0000 7f51277fe700 1 --2- 172.21.15.92:0/2176637050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51281598a0 0x7f5128069730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.641+0000 7f51277fe700 1 -- 172.21.15.92:0/2176637050 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51281594d0 msgr2=0x7f51280b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:56.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.641+0000 7f51277fe700 1 --2- 172.21.15.92:0/2176637050 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51281594d0 0x7f51280b5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.641+0000 7f51277fe700 1 -- 172.21.15.92:0/2176637050 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5118009420 con 0x7f51280cbaa0 2023-10-06T10:38:56.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.641+0000 7f5127fff700 1 --2- 172.21.15.92:0/2176637050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51281598a0 0x7f5128069730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:56.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.641+0000 7f511ffff700 1 --2- 172.21.15.92:0/2176637050 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51281594d0 0x7f51280b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:56.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.641+0000 7f51277fe700 1 --2- 172.21.15.92:0/2176637050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51280cbaa0 0x7f51281562b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f51180091e0 tx=0x7f5118017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:56.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.641+0000 7f511f7fe700 1 -- 172.21.15.92:0/2176637050 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511800ba80 con 0x7f51280cbaa0 2023-10-06T10:38:56.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.641+0000 7f511f7fe700 1 -- 172.21.15.92:0/2176637050 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5118025420 con 0x7f51280cbaa0 2023-10-06T10:38:56.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.641+0000 7f512ddce700 1 -- 172.21.15.92:0/2176637050 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51281554f0 con 0x7f51280cbaa0 2023-10-06T10:38:56.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.641+0000 7f511f7fe700 1 -- 172.21.15.92:0/2176637050 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51180257e0 con 0x7f51280cbaa0 2023-10-06T10:38:56.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.641+0000 7f512ddce700 1 -- 172.21.15.92:0/2176637050 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5128155ab0 con 0x7f51280cbaa0 2023-10-06T10:38:56.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.642+0000 7f512ddce700 1 -- 172.21.15.92:0/2176637050 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5128064a90 con 0x7f51280cbaa0 2023-10-06T10:38:56.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.642+0000 7f511f7fe700 1 -- 172.21.15.92:0/2176637050 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5118023070 con 0x7f51280cbaa0 2023-10-06T10:38:56.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.643+0000 7f511f7fe700 1 --2- 172.21.15.92:0/2176637050 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5108066f30 0x7f51080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:56.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.643+0000 7f511f7fe700 1 -- 172.21.15.92:0/2176637050 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f5118086080 con 0x7f51280cbaa0 2023-10-06T10:38:56.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.643+0000 7f511ffff700 1 --2- 172.21.15.92:0/2176637050 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5108066f30 0x7f51080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:56.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.643+0000 7f511ffff700 1 --2- 172.21.15.92:0/2176637050 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5108066f30 0x7f51080693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f51281587d0 tx=0x7f5110009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:56.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.647+0000 7f511f7fe700 1 -- 172.21.15.92:0/2176637050 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5118052ea0 con 0x7f51280cbaa0 2023-10-06T10:38:56.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.828+0000 7f512ddce700 1 -- 172.21.15.92:0/2176637050 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f51280cfe70 con 0x7f51280cbaa0 2023-10-06T10:38:56.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.828+0000 7f511f7fe700 1 -- 172.21.15.92:0/2176637050 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f51180564c0 con 0x7f51280cbaa0 2023-10-06T10:38:56.830 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T10:38:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.830+0000 7f512ddce700 1 -- 172.21.15.92:0/2176637050 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5108066f30 msgr2=0x7f51080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.830+0000 7f512ddce700 1 --2- 172.21.15.92:0/2176637050 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5108066f30 0x7f51080693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f51281587d0 tx=0x7f5110009040 comp rx=0 tx=0).stop 2023-10-06T10:38:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.830+0000 7f512ddce700 1 -- 172.21.15.92:0/2176637050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51280cbaa0 msgr2=0x7f51281562b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.830+0000 7f512ddce700 1 --2- 172.21.15.92:0/2176637050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51280cbaa0 0x7f51281562b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f51180091e0 tx=0x7f5118017000 comp rx=0 tx=0).stop 2023-10-06T10:38:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.830+0000 7f512ddce700 1 -- 172.21.15.92:0/2176637050 shutdown_connections 2023-10-06T10:38:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.830+0000 7f512ddce700 1 --2- 172.21.15.92:0/2176637050 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5108066f30 0x7f51080693e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.830+0000 7f512ddce700 1 --2- 172.21.15.92:0/2176637050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51281598a0 0x7f5128069730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.830+0000 7f512ddce700 1 --2- 172.21.15.92:0/2176637050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51280cbaa0 0x7f51281562b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.830+0000 7f512ddce700 1 --2- 172.21.15.92:0/2176637050 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51281594d0 0x7f51280b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.830+0000 7f512ddce700 1 -- 172.21.15.92:0/2176637050 >> 172.21.15.92:0/2176637050 conn(0x7f5128089e50 msgr2=0x7f51280c6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.830+0000 7f512ddce700 1 -- 172.21.15.92:0/2176637050 shutdown_connections 2023-10-06T10:38:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.830+0000 7f512ddce700 1 -- 172.21.15.92:0/2176637050 wait complete. 2023-10-06T10:38:56.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T10:38:56.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.959+0000 7fc5b2c40700 1 Processor -- start 2023-10-06T10:38:56.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.959+0000 7fc5b2c40700 1 -- start start 2023-10-06T10:38:56.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.960+0000 7fc5b2c40700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc5ac1594d0 0x7fc5ac0cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:56.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.960+0000 7fc5b2c40700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc5ac1598a0 0x7fc5ac0cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:56.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.960+0000 7fc5b2c40700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5ac0cf330 con 0x7fc5ac0ce7e0 2023-10-06T10:38:56.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.960+0000 7fc5b2c40700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ac0cf5d0 con 0x7fc5ac1598a0 2023-10-06T10:38:56.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.960+0000 7fc5b2c40700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ac0cf8b0 con 0x7fc5ac1594d0 2023-10-06T10:38:56.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.960+0000 7fc5a3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc5ac1598a0 0x7fc5ac0cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:56.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.960+0000 7fc5a3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc5ac1598a0 0x7fc5ac0cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47158/0 (socket says 172.21.15.92:47158) 2023-10-06T10:38:56.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.960+0000 7fc5a3fff700 1 -- 172.21.15.92:0/1745588005 learned_addr learned my addr 172.21.15.92:0/1745588005 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:56.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.960+0000 7fc5b09dc700 1 --2- 172.21.15.92:0/1745588005 >> v2:172.21.15.124:3300/0 conn(0x7fc5ac1594d0 0x7fc5ac0cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:56.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.961+0000 7fc5a3fff700 1 -- 172.21.15.92:0/1745588005 >> v2:172.21.15.124:3300/0 conn(0x7fc5ac1594d0 msgr2=0x7fc5ac0cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:56.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.961+0000 7fc5a3fff700 1 --2- 172.21.15.92:0/1745588005 >> v2:172.21.15.124:3300/0 conn(0x7fc5ac1594d0 0x7fc5ac0cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.961+0000 7fc5a3fff700 1 -- 172.21.15.92:0/1745588005 >> v1:172.21.15.124:6789/0 conn(0x7fc5ac0ce7e0 legacy=0x7fc5ac0cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:56.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.961+0000 7fc5a3fff700 1 -- 172.21.15.92:0/1745588005 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ac0cfb90 con 0x7fc5ac1598a0 2023-10-06T10:38:56.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.961+0000 7fc5a3fff700 1 --2- 172.21.15.92:0/1745588005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ac1598a0 0x7fc5ac0cbaa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac088f70 tx=0x7fc59c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ecbe5b2fd7bc274 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:56.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.961+0000 7fc5a37fe700 1 -- 172.21.15.92:0/1745588005 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c00b860 con 0x7fc5ac1598a0 2023-10-06T10:38:56.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.961+0000 7fc5a37fe700 1 -- 172.21.15.92:0/1745588005 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59c00b9c0 con 0x7fc5ac1598a0 2023-10-06T10:38:56.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.961+0000 7fc5a37fe700 1 -- 172.21.15.92:0/1745588005 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c00bd60 con 0x7fc5ac1598a0 2023-10-06T10:38:56.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.961+0000 7fc5b2c40700 1 -- 172.21.15.92:0/1745588005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ac1598a0 msgr2=0x7fc5ac0cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:56.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.961+0000 7fc5b2c40700 1 --2- 172.21.15.92:0/1745588005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ac1598a0 0x7fc5ac0cbaa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac088f70 tx=0x7fc59c009250 comp rx=0 tx=0).stop 2023-10-06T10:38:56.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.961+0000 7fc5b2c40700 1 -- 172.21.15.92:0/1745588005 shutdown_connections 2023-10-06T10:38:56.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.961+0000 7fc5b2c40700 1 --2- 172.21.15.92:0/1745588005 >> v2:172.21.15.124:3300/0 conn(0x7fc5ac1594d0 0x7fc5ac0cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.961+0000 7fc5b2c40700 1 --2- 172.21.15.92:0/1745588005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ac1598a0 0x7fc5ac0cbaa0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.961+0000 7fc5b2c40700 1 -- 172.21.15.92:0/1745588005 >> 172.21.15.92:0/1745588005 conn(0x7fc5ac089e50 msgr2=0x7fc5ac099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:56.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.961+0000 7fc5b2c40700 1 -- 172.21.15.92:0/1745588005 shutdown_connections 2023-10-06T10:38:56.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.962+0000 7fc5b2c40700 1 -- 172.21.15.92:0/1745588005 wait complete. 2023-10-06T10:38:56.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.963+0000 7fc5b2c40700 1 Processor -- start 2023-10-06T10:38:56.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.963+0000 7fc5b2c40700 1 -- start start 2023-10-06T10:38:56.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.963+0000 7fc5b2c40700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5ac0ce7e0 0x7fc5ac157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:56.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.963+0000 7fc5b2c40700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ac1594d0 0x7fc5ac15c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:56.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.963+0000 7fc5b2c40700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5ac1598a0 0x7fc5ac16add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:56.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.963+0000 7fc5b2c40700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac15b700 con 0x7fc5ac1594d0 2023-10-06T10:38:56.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.963+0000 7fc5b2c40700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac156c70 con 0x7fc5ac0ce7e0 2023-10-06T10:38:56.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.964+0000 7fc5b2c40700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac156f50 con 0x7fc5ac1598a0 2023-10-06T10:38:56.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.964+0000 7fc5a3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ac1594d0 0x7fc5ac15c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:56.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.964+0000 7fc5a3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ac1594d0 0x7fc5ac15c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47162/0 (socket says 172.21.15.92:47162) 2023-10-06T10:38:56.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.964+0000 7fc5a3fff700 1 -- 172.21.15.92:0/3015450023 learned_addr learned my addr 172.21.15.92:0/3015450023 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:56.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.964+0000 7fc5b09dc700 1 --2- 172.21.15.92:0/3015450023 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5ac0ce7e0 0x7fc5ac157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:56.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.964+0000 7fc5b11dd700 1 --2- 172.21.15.92:0/3015450023 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5ac1598a0 0x7fc5ac16add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:56.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.964+0000 7fc5a3fff700 1 -- 172.21.15.92:0/3015450023 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5ac1598a0 msgr2=0x7fc5ac16add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:56.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.964+0000 7fc5a3fff700 1 --2- 172.21.15.92:0/3015450023 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5ac1598a0 0x7fc5ac16add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.964+0000 7fc5a3fff700 1 -- 172.21.15.92:0/3015450023 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5ac0ce7e0 msgr2=0x7fc5ac157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:56.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.964+0000 7fc5a3fff700 1 --2- 172.21.15.92:0/3015450023 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5ac0ce7e0 0x7fc5ac157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:56.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.964+0000 7fc5a3fff700 1 -- 172.21.15.92:0/3015450023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59c0094f0 con 0x7fc5ac1594d0 2023-10-06T10:38:56.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.964+0000 7fc5a3fff700 1 --2- 172.21.15.92:0/3015450023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ac1594d0 0x7fc5ac15c310 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac067c20 tx=0x7fc59c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:56.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.964+0000 7fc5a0ff9700 1 -- 172.21.15.92:0/3015450023 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c024440 con 0x7fc5ac1594d0 2023-10-06T10:38:56.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.964+0000 7fc5a0ff9700 1 -- 172.21.15.92:0/3015450023 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59c0245a0 con 0x7fc5ac1594d0 2023-10-06T10:38:56.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.964+0000 7fc5b2c40700 1 -- 172.21.15.92:0/3015450023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ac157230 con 0x7fc5ac1594d0 2023-10-06T10:38:56.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.964+0000 7fc5a0ff9700 1 -- 172.21.15.92:0/3015450023 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c024960 con 0x7fc5ac1594d0 2023-10-06T10:38:56.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.964+0000 7fc5b2c40700 1 -- 172.21.15.92:0/3015450023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ac1577f0 con 0x7fc5ac1594d0 2023-10-06T10:38:56.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.965+0000 7fc5b2c40700 1 -- 172.21.15.92:0/3015450023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ac064a90 con 0x7fc5ac1594d0 2023-10-06T10:38:56.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.965+0000 7fc5a0ff9700 1 -- 172.21.15.92:0/3015450023 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc59c01a420 con 0x7fc5ac1594d0 2023-10-06T10:38:56.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.966+0000 7fc5a0ff9700 1 --2- 172.21.15.92:0/3015450023 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc598066f30 0x7fc5980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:56.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.966+0000 7fc5b09dc700 1 --2- 172.21.15.92:0/3015450023 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc598066f30 0x7fc5980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:56.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.966+0000 7fc5a0ff9700 1 -- 172.21.15.92:0/3015450023 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fc59c0866d0 con 0x7fc5ac1594d0 2023-10-06T10:38:56.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.966+0000 7fc5b09dc700 1 --2- 172.21.15.92:0/3015450023 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc598066f30 0x7fc5980693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac069bc0 tx=0x7fc5a8009850 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:56.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:56.970+0000 7fc5a0ff9700 1 -- 172.21.15.92:0/3015450023 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc59c0534f0 con 0x7fc5ac1594d0 2023-10-06T10:38:57.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.154+0000 7fc5b2c40700 1 -- 172.21.15.92:0/3015450023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc5ac0cfe70 con 0x7fc5ac1594d0 2023-10-06T10:38:57.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.155+0000 7fc5a0ff9700 1 -- 172.21.15.92:0/3015450023 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc59c056b10 con 0x7fc5ac1594d0 2023-10-06T10:38:57.157 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T10:38:57.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.157+0000 7fc5b2c40700 1 -- 172.21.15.92:0/3015450023 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc598066f30 msgr2=0x7fc5980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:57.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.157+0000 7fc5b2c40700 1 --2- 172.21.15.92:0/3015450023 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc598066f30 0x7fc5980693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac069bc0 tx=0x7fc5a8009850 comp rx=0 tx=0).stop 2023-10-06T10:38:57.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.157+0000 7fc5b2c40700 1 -- 172.21.15.92:0/3015450023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ac1594d0 msgr2=0x7fc5ac15c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:57.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.157+0000 7fc5b2c40700 1 --2- 172.21.15.92:0/3015450023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ac1594d0 0x7fc5ac15c310 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac067c20 tx=0x7fc59c017000 comp rx=0 tx=0).stop 2023-10-06T10:38:57.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.157+0000 7fc5b2c40700 1 -- 172.21.15.92:0/3015450023 shutdown_connections 2023-10-06T10:38:57.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.157+0000 7fc5b2c40700 1 --2- 172.21.15.92:0/3015450023 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc598066f30 0x7fc5980693e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:57.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.157+0000 7fc5b2c40700 1 --2- 172.21.15.92:0/3015450023 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5ac1598a0 0x7fc5ac16add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:57.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.157+0000 7fc5b2c40700 1 --2- 172.21.15.92:0/3015450023 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5ac0ce7e0 0x7fc5ac157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:57.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.157+0000 7fc5b2c40700 1 --2- 172.21.15.92:0/3015450023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ac1594d0 0x7fc5ac15c310 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:57.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.157+0000 7fc5b2c40700 1 -- 172.21.15.92:0/3015450023 >> 172.21.15.92:0/3015450023 conn(0x7fc5ac089e50 msgr2=0x7fc5ac14bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:57.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.157+0000 7fc5b2c40700 1 -- 172.21.15.92:0/3015450023 shutdown_connections 2023-10-06T10:38:57.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.157+0000 7fc5b2c40700 1 -- 172.21.15.92:0/3015450023 wait complete. 2023-10-06T10:38:57.210 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_multimds_misc.TestScrub2) 2023-10-06T10:38:57.210 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag' 2023-10-06T10:38:57.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.322+0000 7f1268828700 1 Processor -- start 2023-10-06T10:38:57.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.323+0000 7f1268828700 1 -- start start 2023-10-06T10:38:57.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.323+0000 7f1268828700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f12640b49a0 0x7f12640aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:57.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.323+0000 7f1268828700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12640b0c60 con 0x7f1264159560 2023-10-06T10:38:57.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.323+0000 7f1268828700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12640b0f40 con 0x7f12640b0760 2023-10-06T10:38:57.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.323+0000 7f1268828700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12640b1220 con 0x7f12640b49a0 2023-10-06T10:38:57.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.323+0000 7f1259d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f12640b49a0 0x7f12640aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:57.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.323+0000 7f126259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1264159560 0x7f12640aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50332/0 (socket says 172.21.15.92:50332) 2023-10-06T10:38:57.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.323+0000 7f126259c700 1 -- 172.21.15.92:0/2269699589 learned_addr learned my addr 172.21.15.92:0/2269699589 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:57.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.324+0000 7f1259d9b700 1 -- 172.21.15.92:0/2269699589 >> v1:172.21.15.124:6789/0 conn(0x7f12640b0760 legacy=0x7f12640b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:57.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.324+0000 7f1259d9b700 1 -- 172.21.15.92:0/2269699589 >> v1:172.21.15.92:6789/0 conn(0x7f1264159560 legacy=0x7f12640aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:57.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.324+0000 7f1259d9b700 1 -- 172.21.15.92:0/2269699589 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12640b1500 con 0x7f12640b49a0 2023-10-06T10:38:57.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.324+0000 7f1261d9b700 1 -- 172.21.15.92:0/2269699589 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2684059875 0 0) 0x7f12640b0c60 con 0x7f1264159560 2023-10-06T10:38:57.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.324+0000 7f1259d9b700 1 --2- 172.21.15.92:0/2269699589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12640b49a0 0x7f12640aac40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f124c009910 tx=0x7f124c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c48ec6dbe126987 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:57.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.324+0000 7f1261d9b700 1 -- 172.21.15.92:0/2269699589 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c00b7e0 con 0x7f12640b49a0 2023-10-06T10:38:57.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.324+0000 7f1261d9b700 1 -- 172.21.15.92:0/2269699589 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124c00b940 con 0x7f12640b49a0 2023-10-06T10:38:57.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.324+0000 7f1261d9b700 1 -- 172.21.15.92:0/2269699589 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c00bce0 con 0x7f12640b49a0 2023-10-06T10:38:57.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.324+0000 7f1268828700 1 -- 172.21.15.92:0/2269699589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12640b49a0 msgr2=0x7f12640aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:57.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.324+0000 7f1268828700 1 --2- 172.21.15.92:0/2269699589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12640b49a0 0x7f12640aac40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f124c009910 tx=0x7f124c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:57.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.324+0000 7f1268828700 1 -- 172.21.15.92:0/2269699589 shutdown_connections 2023-10-06T10:38:57.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.324+0000 7f1268828700 1 --2- 172.21.15.92:0/2269699589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12640b49a0 0x7f12640aac40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:57.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.324+0000 7f1268828700 1 -- 172.21.15.92:0/2269699589 >> 172.21.15.92:0/2269699589 conn(0x7f12640997f0 msgr2=0x7f126406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:57.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.325+0000 7f1268828700 1 -- 172.21.15.92:0/2269699589 shutdown_connections 2023-10-06T10:38:57.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.325+0000 7f1268828700 1 -- 172.21.15.92:0/2269699589 wait complete. 2023-10-06T10:38:57.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.326+0000 7f1268828700 1 Processor -- start 2023-10-06T10:38:57.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.326+0000 7f1268828700 1 -- start start 2023-10-06T10:38:57.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.326+0000 7f1268828700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12640b0760 0x7f126409a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:57.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.326+0000 7f1268828700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12640b49a0 0x7f126409b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:57.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.326+0000 7f1268828700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1264159560 0x7f126409d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:57.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.326+0000 7f1268828700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126415be10 con 0x7f12640b49a0 2023-10-06T10:38:57.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.326+0000 7f1268828700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1264156f40 con 0x7f1264159560 2023-10-06T10:38:57.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.326+0000 7f1268828700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f126409afd0 con 0x7f12640b0760 2023-10-06T10:38:57.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.327+0000 7f1259d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12640b49a0 0x7f126409b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:57.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.327+0000 7f1259d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12640b49a0 0x7f126409b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47190/0 (socket says 172.21.15.92:47190) 2023-10-06T10:38:57.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.327+0000 7f1259d9b700 1 -- 172.21.15.92:0/1548483567 learned_addr learned my addr 172.21.15.92:0/1548483567 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:57.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.327+0000 7f1262d9d700 1 --2- 172.21.15.92:0/1548483567 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1264159560 0x7f126409d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:57.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.327+0000 7f126259c700 1 --2- 172.21.15.92:0/1548483567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12640b0760 0x7f126409a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:57.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.327+0000 7f1259d9b700 1 -- 172.21.15.92:0/1548483567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12640b0760 msgr2=0x7f126409a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:57.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.327+0000 7f1259d9b700 1 --2- 172.21.15.92:0/1548483567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12640b0760 0x7f126409a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:57.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.327+0000 7f1259d9b700 1 -- 172.21.15.92:0/1548483567 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1264159560 msgr2=0x7f126409d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:57.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.327+0000 7f1259d9b700 1 --2- 172.21.15.92:0/1548483567 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1264159560 0x7f126409d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:57.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.327+0000 7f1259d9b700 1 -- 172.21.15.92:0/1548483567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c009420 con 0x7f12640b49a0 2023-10-06T10:38:57.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.327+0000 7f1262d9d700 1 --2- 172.21.15.92:0/1548483567 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1264159560 0x7f126409d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:57.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.327+0000 7f1259d9b700 1 --2- 172.21.15.92:0/1548483567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12640b49a0 0x7f126409b960 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f124c0093f0 tx=0x7f124c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:57.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.327+0000 7f125affd700 1 -- 172.21.15.92:0/1548483567 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c011e30 con 0x7f12640b49a0 2023-10-06T10:38:57.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.327+0000 7f125affd700 1 -- 172.21.15.92:0/1548483567 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124c01b680 con 0x7f12640b49a0 2023-10-06T10:38:57.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.327+0000 7f1268828700 1 -- 172.21.15.92:0/1548483567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126409a730 con 0x7f12640b49a0 2023-10-06T10:38:57.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.327+0000 7f125affd700 1 -- 172.21.15.92:0/1548483567 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c01ba40 con 0x7f12640b49a0 2023-10-06T10:38:57.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.327+0000 7f1268828700 1 -- 172.21.15.92:0/1548483567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126409acf0 con 0x7f12640b49a0 2023-10-06T10:38:57.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.328+0000 7f1268828700 1 -- 172.21.15.92:0/1548483567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f126415bf80 con 0x7f12640b49a0 2023-10-06T10:38:57.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.329+0000 7f125affd700 1 -- 172.21.15.92:0/1548483567 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f124c01c450 con 0x7f12640b49a0 2023-10-06T10:38:57.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.329+0000 7f125affd700 1 --2- 172.21.15.92:0/1548483567 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1244066f30 0x7f12440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:57.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.329+0000 7f125affd700 1 -- 172.21.15.92:0/1548483567 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f124c086300 con 0x7f12640b49a0 2023-10-06T10:38:57.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.329+0000 7f126259c700 1 --2- 172.21.15.92:0/1548483567 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1244066f30 0x7f12440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:57.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.329+0000 7f126259c700 1 --2- 172.21.15.92:0/1548483567 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1244066f30 0x7f12440693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f12540029c0 tx=0x7f1254009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:57.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.333+0000 7f125affd700 1 -- 172.21.15.92:0/1548483567 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f126415bf80 con 0x7f12640b49a0 2023-10-06T10:38:57.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:57.594+0000 7f1268828700 1 -- 172.21.15.92:0/1548483567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]} v 0) v1 -- 0x7f126415bf50 con 0x7f12640b49a0 2023-10-06T10:38:58.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.535+0000 7f125affd700 1 -- 172.21.15.92:0/1548483567 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]}]=0 v58) v1 ==== 139+0+0 (secure 0 0 0) 0x7f124c0531d0 con 0x7f12640b49a0 2023-10-06T10:38:58.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.537+0000 7f1268828700 1 -- 172.21.15.92:0/1548483567 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1244066f30 msgr2=0x7f12440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.537+0000 7f1268828700 1 --2- 172.21.15.92:0/1548483567 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1244066f30 0x7f12440693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f12540029c0 tx=0x7f1254009040 comp rx=0 tx=0).stop 2023-10-06T10:38:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.537+0000 7f1268828700 1 -- 172.21.15.92:0/1548483567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12640b49a0 msgr2=0x7f126409b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.537+0000 7f1268828700 1 --2- 172.21.15.92:0/1548483567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12640b49a0 0x7f126409b960 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f124c0093f0 tx=0x7f124c016000 comp rx=0 tx=0).stop 2023-10-06T10:38:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.537+0000 7f1268828700 1 -- 172.21.15.92:0/1548483567 shutdown_connections 2023-10-06T10:38:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.537+0000 7f1268828700 1 --2- 172.21.15.92:0/1548483567 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1244066f30 0x7f12440693e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.537+0000 7f1268828700 1 --2- 172.21.15.92:0/1548483567 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1264159560 0x7f126409d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.537+0000 7f1268828700 1 --2- 172.21.15.92:0/1548483567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12640b0760 0x7f126409a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.537+0000 7f1268828700 1 --2- 172.21.15.92:0/1548483567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12640b49a0 0x7f126409b960 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.537+0000 7f1268828700 1 -- 172.21.15.92:0/1548483567 >> 172.21.15.92:0/1548483567 conn(0x7f12640997f0 msgr2=0x7f1264069580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.537+0000 7f1268828700 1 -- 172.21.15.92:0/1548483567 shutdown_connections 2023-10-06T10:38:58.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.537+0000 7f1268828700 1 -- 172.21.15.92:0/1548483567 wait complete. 2023-10-06T10:38:58.554 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T10:38:58.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.675+0000 7fb73f8fd700 1 Processor -- start 2023-10-06T10:38:58.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.675+0000 7fb73f8fd700 1 -- start start 2023-10-06T10:38:58.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.676+0000 7fb73f8fd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb7380c4450 0x7fb7380686c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:58.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.676+0000 7fb73f8fd700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7380c9670 con 0x7fb7380b0820 2023-10-06T10:38:58.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.676+0000 7fb73f8fd700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb738159c00 con 0x7fb7380cc740 2023-10-06T10:38:58.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.676+0000 7fb73f8fd700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7380c8a60 con 0x7fb7380c4450 2023-10-06T10:38:58.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.676+0000 7fb73ce98700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb7380c4450 0x7fb7380686c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:58.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.676+0000 7fb73d699700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb7380b0820 0x7fb7380cec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50334/0 (socket says 172.21.15.92:50334) 2023-10-06T10:38:58.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.676+0000 7fb73d699700 1 -- 172.21.15.92:0/2530066215 learned_addr learned my addr 172.21.15.92:0/2530066215 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:58.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.677+0000 7fb73ce98700 1 -- 172.21.15.92:0/2530066215 >> v1:172.21.15.124:6789/0 conn(0x7fb7380cc740 legacy=0x7fb7380ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:58.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.677+0000 7fb73ce98700 1 -- 172.21.15.92:0/2530066215 >> v1:172.21.15.92:6789/0 conn(0x7fb7380b0820 legacy=0x7fb7380cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:58.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.677+0000 7fb73ce98700 1 -- 172.21.15.92:0/2530066215 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7380c8d40 con 0x7fb7380c4450 2023-10-06T10:38:58.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.677+0000 7fb73ce98700 1 --2- 172.21.15.92:0/2530066215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7380c4450 0x7fb7380686c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb7280098e0 tx=0x7fb7280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25701264abcf6840 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:58.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.677+0000 7fb72ffff700 1 -- 172.21.15.92:0/2530066215 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb728009e80 con 0x7fb7380c4450 2023-10-06T10:38:58.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.677+0000 7fb72ffff700 1 -- 172.21.15.92:0/2530066215 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72800b860 con 0x7fb7380c4450 2023-10-06T10:38:58.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.677+0000 7fb72ffff700 1 -- 172.21.15.92:0/2530066215 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72800bbe0 con 0x7fb7380c4450 2023-10-06T10:38:58.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.677+0000 7fb73f8fd700 1 -- 172.21.15.92:0/2530066215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7380c4450 msgr2=0x7fb7380686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:58.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.677+0000 7fb73f8fd700 1 --2- 172.21.15.92:0/2530066215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7380c4450 0x7fb7380686c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb7280098e0 tx=0x7fb7280091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:58.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.678+0000 7fb73f8fd700 1 -- 172.21.15.92:0/2530066215 shutdown_connections 2023-10-06T10:38:58.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.678+0000 7fb73f8fd700 1 --2- 172.21.15.92:0/2530066215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7380c4450 0x7fb7380686c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:58.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.678+0000 7fb73f8fd700 1 -- 172.21.15.92:0/2530066215 >> 172.21.15.92:0/2530066215 conn(0x7fb738087fc0 msgr2=0x7fb7380ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:58.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.678+0000 7fb73f8fd700 1 -- 172.21.15.92:0/2530066215 shutdown_connections 2023-10-06T10:38:58.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.678+0000 7fb73f8fd700 1 -- 172.21.15.92:0/2530066215 wait complete. 2023-10-06T10:38:58.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.679+0000 7fb73f8fd700 1 Processor -- start 2023-10-06T10:38:58.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.679+0000 7fb73f8fd700 1 -- start start 2023-10-06T10:38:58.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.680+0000 7fb73f8fd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7380b0820 0x7fb738150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:58.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.680+0000 7fb73f8fd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7380c4450 0x7fb73815d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:58.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.680+0000 7fb73f8fd700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7380cc740 0x7fb73815b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:58.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.680+0000 7fb73f8fd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73815ee20 con 0x7fb7380b0820 2023-10-06T10:38:58.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.680+0000 7fb73f8fd700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73815e210 con 0x7fb7380cc740 2023-10-06T10:38:58.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.680+0000 7fb73f8fd700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb73815e4f0 con 0x7fb7380c4450 2023-10-06T10:38:58.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.680+0000 7fb73d699700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7380b0820 0x7fb738150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:58.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.680+0000 7fb73d699700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7380b0820 0x7fb738150340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47220/0 (socket says 172.21.15.92:47220) 2023-10-06T10:38:58.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.680+0000 7fb73d699700 1 -- 172.21.15.92:0/1653302350 learned_addr learned my addr 172.21.15.92:0/1653302350 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:58.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.680+0000 7fb73ce98700 1 --2- 172.21.15.92:0/1653302350 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7380c4450 0x7fb73815d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:58.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.680+0000 7fb73de9a700 1 --2- 172.21.15.92:0/1653302350 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7380cc740 0x7fb73815b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:58.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.680+0000 7fb73d699700 1 -- 172.21.15.92:0/1653302350 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7380c4450 msgr2=0x7fb73815d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:58.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.680+0000 7fb73d699700 1 --2- 172.21.15.92:0/1653302350 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7380c4450 0x7fb73815d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:58.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.680+0000 7fb73d699700 1 -- 172.21.15.92:0/1653302350 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7380cc740 msgr2=0x7fb73815b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:58.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.680+0000 7fb73d699700 1 --2- 172.21.15.92:0/1653302350 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7380cc740 0x7fb73815b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:58.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.680+0000 7fb73d699700 1 -- 172.21.15.92:0/1653302350 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb728009420 con 0x7fb7380b0820 2023-10-06T10:38:58.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.680+0000 7fb73d699700 1 --2- 172.21.15.92:0/1653302350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7380b0820 0x7fb738150340 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb73400cb40 tx=0x7fb734001790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:58.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.680+0000 7fb72d7fa700 1 -- 172.21.15.92:0/1653302350 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb734005d80 con 0x7fb7380b0820 2023-10-06T10:38:58.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.680+0000 7fb72d7fa700 1 -- 172.21.15.92:0/1653302350 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73400a5d0 con 0x7fb7380b0820 2023-10-06T10:38:58.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.680+0000 7fb73f8fd700 1 -- 172.21.15.92:0/1653302350 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb73815e7d0 con 0x7fb7380b0820 2023-10-06T10:38:58.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.680+0000 7fb72d7fa700 1 -- 172.21.15.92:0/1653302350 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73400a9d0 con 0x7fb7380b0820 2023-10-06T10:38:58.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.680+0000 7fb73f8fd700 1 -- 172.21.15.92:0/1653302350 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb738150880 con 0x7fb7380b0820 2023-10-06T10:38:58.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.681+0000 7fb73f8fd700 1 -- 172.21.15.92:0/1653302350 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb738064a90 con 0x7fb7380b0820 2023-10-06T10:38:58.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.682+0000 7fb72d7fa700 1 -- 172.21.15.92:0/1653302350 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb7340070f0 con 0x7fb7380b0820 2023-10-06T10:38:58.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.682+0000 7fb72d7fa700 1 --2- 172.21.15.92:0/1653302350 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb71c066f30 0x7fb71c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:58.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.682+0000 7fb72d7fa700 1 -- 172.21.15.92:0/1653302350 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb7340847f0 con 0x7fb7380b0820 2023-10-06T10:38:58.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.682+0000 7fb73ce98700 1 --2- 172.21.15.92:0/1653302350 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb71c066f30 0x7fb71c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:58.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.682+0000 7fb73ce98700 1 --2- 172.21.15.92:0/1653302350 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb71c066f30 0x7fb71c0693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb7280093f0 tx=0x7fb728017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:58.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.686+0000 7fb72d7fa700 1 -- 172.21.15.92:0/1653302350 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb734051610 con 0x7fb7380b0820 2023-10-06T10:38:58.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.869+0000 7fb73f8fd700 1 -- 172.21.15.92:0/1653302350 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb73804bb40 con 0x7fb7380b0820 2023-10-06T10:38:58.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.874+0000 7fb72d7fa700 1 -- 172.21.15.92:0/1653302350 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb734054c30 con 0x7fb7380b0820 2023-10-06T10:38:58.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.877+0000 7fb73f8fd700 1 -- 172.21.15.92:0/1653302350 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb71c066f30 msgr2=0x7fb71c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:58.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.877+0000 7fb73f8fd700 1 --2- 172.21.15.92:0/1653302350 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb71c066f30 0x7fb71c0693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb7280093f0 tx=0x7fb728017000 comp rx=0 tx=0).stop 2023-10-06T10:38:58.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.877+0000 7fb73f8fd700 1 -- 172.21.15.92:0/1653302350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7380b0820 msgr2=0x7fb738150340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:58.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.877+0000 7fb73f8fd700 1 --2- 172.21.15.92:0/1653302350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7380b0820 0x7fb738150340 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb73400cb40 tx=0x7fb734001790 comp rx=0 tx=0).stop 2023-10-06T10:38:58.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.877+0000 7fb73f8fd700 1 -- 172.21.15.92:0/1653302350 shutdown_connections 2023-10-06T10:38:58.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.877+0000 7fb73f8fd700 1 --2- 172.21.15.92:0/1653302350 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb71c066f30 0x7fb71c0693e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:58.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.877+0000 7fb73f8fd700 1 --2- 172.21.15.92:0/1653302350 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7380cc740 0x7fb73815b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:58.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.877+0000 7fb73f8fd700 1 --2- 172.21.15.92:0/1653302350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7380b0820 0x7fb738150340 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:58.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.877+0000 7fb73f8fd700 1 --2- 172.21.15.92:0/1653302350 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7380c4450 0x7fb73815d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:58.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.877+0000 7fb73f8fd700 1 -- 172.21.15.92:0/1653302350 >> 172.21.15.92:0/1653302350 conn(0x7fb738087fc0 msgr2=0x7fb7381590b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:58.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.877+0000 7fb73f8fd700 1 -- 172.21.15.92:0/1653302350 shutdown_connections 2023-10-06T10:38:58.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:58.877+0000 7fb73f8fd700 1 -- 172.21.15.92:0/1653302350 wait complete. 2023-10-06T10:38:58.891 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T10:38:58.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:38:58.937 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:38:58.976 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T10:38:58.976 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T10:38:58.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:38:58.977 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-06T10:38:59.015 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T10:38:59.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:59.016 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T10:38:59.071 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T10:38:59.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:38:59.071 DEBUG:teuthology.orchestra.run.smithi124:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-10-06T10:38:59.106 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi124.front.sepia.ceph.com 2023-10-06T10:38:59.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:59.107 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T10:38:59.162 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-10-06T10:38:59.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:38:59.163 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-10-06T10:38:59.204 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T10:38:59.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:59.205 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T10:38:59.259 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-10-06T10:38:59.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:38:59.260 DEBUG:teuthology.orchestra.run.smithi124:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-10-06T10:38:59.295 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi124.front.sepia.ceph.com 2023-10-06T10:38:59.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:38:59.296 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T10:38:59.351 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T10:38:59.351 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:38:59.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.463+0000 7f1da6cd2700 1 Processor -- start 2023-10-06T10:38:59.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.463+0000 7f1da6cd2700 1 -- start start 2023-10-06T10:38:59.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.464+0000 7f1da6cd2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1da00b49a0 0x7f1da00aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:59.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.464+0000 7f1da6cd2700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da00b0760 0x7f1da00b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:59.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.464+0000 7f1da6cd2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da00b0c60 con 0x7f1da0159560 2023-10-06T10:38:59.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.464+0000 7f1da6cd2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da00b0f40 con 0x7f1da00b49a0 2023-10-06T10:38:59.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.464+0000 7f1da6cd2700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da00b1220 con 0x7f1da00b0760 2023-10-06T10:38:59.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.464+0000 7f1d9ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1da00b49a0 0x7f1da00aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:59.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.464+0000 7f1da4a6e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1da0159560 0x7f1da00aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50336/0 (socket says 172.21.15.92:50336) 2023-10-06T10:38:59.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.464+0000 7f1da4a6e700 1 -- 172.21.15.92:0/2156596510 learned_addr learned my addr 172.21.15.92:0/2156596510 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:59.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.464+0000 7f1d9ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1da00b49a0 0x7f1da00aac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47226/0 (socket says 172.21.15.92:47226) 2023-10-06T10:38:59.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.465+0000 7f1da526f700 1 --2- 172.21.15.92:0/2156596510 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da00b0760 0x7f1da00b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:59.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.465+0000 7f1d9ffff700 1 -- 172.21.15.92:0/2156596510 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da00b0760 msgr2=0x7f1da00b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:38:59.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.465+0000 7f1d9ffff700 1 --2- 172.21.15.92:0/2156596510 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da00b0760 0x7f1da00b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:59.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.465+0000 7f1d9ffff700 1 -- 172.21.15.92:0/2156596510 >> v1:172.21.15.92:6789/0 conn(0x7f1da0159560 legacy=0x7f1da00aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:59.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.465+0000 7f1d9f7fe700 1 -- 172.21.15.92:0/2156596510 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1173342417 0 0) 0x7f1da00b0c60 con 0x7f1da0159560 2023-10-06T10:38:59.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.465+0000 7f1d9ffff700 1 -- 172.21.15.92:0/2156596510 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da00b1500 con 0x7f1da00b49a0 2023-10-06T10:38:59.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.465+0000 7f1da526f700 1 --2- 172.21.15.92:0/2156596510 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da00b0760 0x7f1da00b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:38:59.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.465+0000 7f1d9ffff700 1 --2- 172.21.15.92:0/2156596510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da00b49a0 0x7f1da00aac40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009910 tx=0x7f1d8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af15fead5db12770 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:59.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.465+0000 7f1d9f7fe700 1 -- 172.21.15.92:0/2156596510 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c00b820 con 0x7f1da00b49a0 2023-10-06T10:38:59.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.465+0000 7f1d9f7fe700 1 -- 172.21.15.92:0/2156596510 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c00b980 con 0x7f1da00b49a0 2023-10-06T10:38:59.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.465+0000 7f1d9f7fe700 1 -- 172.21.15.92:0/2156596510 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c00bd20 con 0x7f1da00b49a0 2023-10-06T10:38:59.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.465+0000 7f1da6cd2700 1 -- 172.21.15.92:0/2156596510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da00b49a0 msgr2=0x7f1da00aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:59.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.465+0000 7f1da6cd2700 1 --2- 172.21.15.92:0/2156596510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da00b49a0 0x7f1da00aac40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009910 tx=0x7f1d8c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:38:59.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.465+0000 7f1da6cd2700 1 -- 172.21.15.92:0/2156596510 shutdown_connections 2023-10-06T10:38:59.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.465+0000 7f1da6cd2700 1 --2- 172.21.15.92:0/2156596510 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da00b0760 0x7f1da00b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:59.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.465+0000 7f1da6cd2700 1 --2- 172.21.15.92:0/2156596510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da00b49a0 0x7f1da00aac40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:59.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.465+0000 7f1da6cd2700 1 -- 172.21.15.92:0/2156596510 >> 172.21.15.92:0/2156596510 conn(0x7f1da00997f0 msgr2=0x7f1da006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:59.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.465+0000 7f1da6cd2700 1 -- 172.21.15.92:0/2156596510 shutdown_connections 2023-10-06T10:38:59.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.465+0000 7f1da6cd2700 1 -- 172.21.15.92:0/2156596510 wait complete. 2023-10-06T10:38:59.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.467+0000 7f1da6cd2700 1 Processor -- start 2023-10-06T10:38:59.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.467+0000 7f1da6cd2700 1 -- start start 2023-10-06T10:38:59.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.467+0000 7f1da6cd2700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1da00b0760 0x7f1da0158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:59.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.467+0000 7f1da6cd2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da00b49a0 0x7f1da00ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:59.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.467+0000 7f1da6cd2700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da0159560 0x7f1da00cef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:59.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.467+0000 7f1da6cd2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da015ba40 con 0x7f1da00b49a0 2023-10-06T10:38:59.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.467+0000 7f1da6cd2700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da015b5b0 con 0x7f1da00b0760 2023-10-06T10:38:59.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.467+0000 7f1da6cd2700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da015a9a0 con 0x7f1da0159560 2023-10-06T10:38:59.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.468+0000 7f1d9ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da00b49a0 0x7f1da00ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:59.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.468+0000 7f1d9ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da00b49a0 0x7f1da00ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47242/0 (socket says 172.21.15.92:47242) 2023-10-06T10:38:59.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.468+0000 7f1d9ffff700 1 -- 172.21.15.92:0/439718079 learned_addr learned my addr 172.21.15.92:0/439718079 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:59.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.468+0000 7f1da526f700 1 --2- 172.21.15.92:0/439718079 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da0159560 0x7f1da00cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:59.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.468+0000 7f1da4a6e700 1 --2- 172.21.15.92:0/439718079 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1da00b0760 0x7f1da0158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:59.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.468+0000 7f1d9ffff700 1 -- 172.21.15.92:0/439718079 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da0159560 msgr2=0x7f1da00cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:59.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.468+0000 7f1d9ffff700 1 --2- 172.21.15.92:0/439718079 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da0159560 0x7f1da00cef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:59.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.468+0000 7f1d9ffff700 1 -- 172.21.15.92:0/439718079 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1da00b0760 msgr2=0x7f1da0158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:59.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.468+0000 7f1d9ffff700 1 --2- 172.21.15.92:0/439718079 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1da00b0760 0x7f1da0158b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:59.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.468+0000 7f1d9ffff700 1 -- 172.21.15.92:0/439718079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c009420 con 0x7f1da00b49a0 2023-10-06T10:38:59.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.468+0000 7f1da4a6e700 1 --2- 172.21.15.92:0/439718079 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1da00b0760 0x7f1da0158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:38:59.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.468+0000 7f1d9ffff700 1 --2- 172.21.15.92:0/439718079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da00b49a0 0x7f1da00ce9c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1da0067c20 tx=0x7f1d8c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:59.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.468+0000 7f1d9cff9700 1 -- 172.21.15.92:0/439718079 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c024440 con 0x7f1da00b49a0 2023-10-06T10:38:59.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.468+0000 7f1d9cff9700 1 -- 172.21.15.92:0/439718079 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c0245c0 con 0x7f1da00b49a0 2023-10-06T10:38:59.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.468+0000 7f1da6cd2700 1 -- 172.21.15.92:0/439718079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da015ac80 con 0x7f1da00b49a0 2023-10-06T10:38:59.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.468+0000 7f1d9cff9700 1 -- 172.21.15.92:0/439718079 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c024980 con 0x7f1da00b49a0 2023-10-06T10:38:59.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.468+0000 7f1da6cd2700 1 -- 172.21.15.92:0/439718079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da015b240 con 0x7f1da00b49a0 2023-10-06T10:38:59.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.469+0000 7f1da6cd2700 1 -- 172.21.15.92:0/439718079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da015b740 con 0x7f1da00b49a0 2023-10-06T10:38:59.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.469+0000 7f1d9cff9700 1 -- 172.21.15.92:0/439718079 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1d8c01a420 con 0x7f1da00b49a0 2023-10-06T10:38:59.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.470+0000 7f1d9cff9700 1 --2- 172.21.15.92:0/439718079 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1d84066f60 0x7f1d84069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:59.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.470+0000 7f1d9cff9700 1 -- 172.21.15.92:0/439718079 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f1d8c0868f0 con 0x7f1da00b49a0 2023-10-06T10:38:59.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.470+0000 7f1da4a6e700 1 --2- 172.21.15.92:0/439718079 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1d84066f60 0x7f1d84069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:59.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.470+0000 7f1da4a6e700 1 --2- 172.21.15.92:0/439718079 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1d84066f60 0x7f1d84069410 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1d94001890 tx=0x7f1d94009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:59.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.474+0000 7f1d9cff9700 1 -- 172.21.15.92:0/439718079 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d8c053670 con 0x7f1da00b49a0 2023-10-06T10:38:59.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.704+0000 7f1da6cd2700 1 -- 172.21.15.92:0/439718079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1da0064a90 con 0x7f1da00b49a0 2023-10-06T10:38:59.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.705+0000 7f1d9cff9700 1 -- 172.21.15.92:0/439718079 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4599 (secure 0 0 0) 0x7f1d8c056c90 con 0x7f1da00b49a0 2023-10-06T10:38:59.707 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:38:59.707 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2303424485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2303424485},{"type":"v1","addr":"172.21.15.92:6836","nonce":2303424485}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1027343065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1027343065},{"type":"v1","addr":"172.21.15.92:6837","nonce":1027343065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2667820379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2667820379},{"type":"v1","addr":"172.21.15.124:6834","nonce":2667820379}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:38:16.725144+0000","modified":"2023-10-06T10:38:30.398242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/1919704285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":1919704285},{"type":"v1","addr":"172.21.15.124:6835","nonce":1919704285}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T10:38:59.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.707+0000 7f1da6cd2700 1 -- 172.21.15.92:0/439718079 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1d84066f60 msgr2=0x7f1d84069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:59.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.707+0000 7f1da6cd2700 1 --2- 172.21.15.92:0/439718079 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1d84066f60 0x7f1d84069410 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1d94001890 tx=0x7f1d94009040 comp rx=0 tx=0).stop 2023-10-06T10:38:59.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.707+0000 7f1da6cd2700 1 -- 172.21.15.92:0/439718079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da00b49a0 msgr2=0x7f1da00ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:59.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.707+0000 7f1da6cd2700 1 --2- 172.21.15.92:0/439718079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da00b49a0 0x7f1da00ce9c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1da0067c20 tx=0x7f1d8c017000 comp rx=0 tx=0).stop 2023-10-06T10:38:59.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.707+0000 7f1da6cd2700 1 -- 172.21.15.92:0/439718079 shutdown_connections 2023-10-06T10:38:59.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.707+0000 7f1da6cd2700 1 --2- 172.21.15.92:0/439718079 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1d84066f60 0x7f1d84069410 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:59.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.707+0000 7f1da6cd2700 1 --2- 172.21.15.92:0/439718079 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da0159560 0x7f1da00cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:59.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.707+0000 7f1da6cd2700 1 --2- 172.21.15.92:0/439718079 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1da00b0760 0x7f1da0158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:59.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.707+0000 7f1da6cd2700 1 --2- 172.21.15.92:0/439718079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da00b49a0 0x7f1da00ce9c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:59.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.707+0000 7f1da6cd2700 1 -- 172.21.15.92:0/439718079 >> 172.21.15.92:0/439718079 conn(0x7f1da00997f0 msgr2=0x7f1da00b23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:59.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.708+0000 7f1da6cd2700 1 -- 172.21.15.92:0/439718079 shutdown_connections 2023-10-06T10:38:59.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.708+0000 7f1da6cd2700 1 -- 172.21.15.92:0/439718079 wait complete. 2023-10-06T10:38:59.711 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-10-06T10:38:59.721 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:38:59.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.833+0000 7f3710e90700 1 Processor -- start 2023-10-06T10:38:59.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.833+0000 7f3710e90700 1 -- start start 2023-10-06T10:38:59.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.834+0000 7f3710e90700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f370c0cd440 0x7f370c1645a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:59.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.834+0000 7f3710e90700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370c0b0da0 con 0x7f370c0b29b0 2023-10-06T10:38:59.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.834+0000 7f3710e90700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370c0cbfa0 con 0x7f370c0b25e0 2023-10-06T10:38:59.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.834+0000 7f3710e90700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370c0cb390 con 0x7f370c0cd440 2023-10-06T10:38:59.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.834+0000 7f3709d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f370c0b29b0 0x7f370c159120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50346/0 (socket says 172.21.15.92:50346) 2023-10-06T10:38:59.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.834+0000 7f3709d9b700 1 -- 172.21.15.92:0/3632910996 learned_addr learned my addr 172.21.15.92:0/3632910996 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:38:59.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.834+0000 7f370ad9d700 1 --2- 172.21.15.92:0/3632910996 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f370c0cd440 0x7f370c1645a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:59.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.835+0000 7f370959a700 1 -- 172.21.15.92:0/3632910996 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2586722791 0 0) 0x7f36f4001110 con 0x7f370c0b29b0 2023-10-06T10:38:59.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.835+0000 7f370959a700 1 -- 172.21.15.92:0/3632910996 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36f8003570 con 0x7f370c0b29b0 2023-10-06T10:38:59.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.835+0000 7f370959a700 1 -- 172.21.15.92:0/3632910996 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2549757891 0 0) 0x7f370c0cbfa0 con 0x7f370c0b25e0 2023-10-06T10:38:59.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.835+0000 7f370959a700 1 -- 172.21.15.92:0/3632910996 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36f4001110 con 0x7f370c0b25e0 2023-10-06T10:38:59.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.835+0000 7f370ad9d700 1 -- 172.21.15.92:0/3632910996 >> v1:172.21.15.124:6789/0 conn(0x7f370c0b25e0 legacy=0x7f370c0b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:59.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.835+0000 7f370959a700 1 -- 172.21.15.92:0/3632910996 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2226244723 0 0) 0x7f36f8003570 con 0x7f370c0b29b0 2023-10-06T10:38:59.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.835+0000 7f370ad9d700 1 -- 172.21.15.92:0/3632910996 >> v1:172.21.15.92:6789/0 conn(0x7f370c0b29b0 legacy=0x7f370c159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:59.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.835+0000 7f370ad9d700 1 -- 172.21.15.92:0/3632910996 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c0cb670 con 0x7f370c0cd440 2023-10-06T10:38:59.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.835+0000 7f370ad9d700 1 --2- 172.21.15.92:0/3632910996 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f370c0cd440 0x7f370c1645a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3704009e90 tx=0x7f3704009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1eac230ef3ead63 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:59.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.835+0000 7f370959a700 1 -- 172.21.15.92:0/3632910996 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370400e070 con 0x7f370c0cd440 2023-10-06T10:38:59.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.835+0000 7f370959a700 1 -- 172.21.15.92:0/3632910996 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370400b870 con 0x7f370c0cd440 2023-10-06T10:38:59.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.835+0000 7f370959a700 1 -- 172.21.15.92:0/3632910996 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f4001600 con 0x7f370c0cd440 2023-10-06T10:38:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.836+0000 7f3710e90700 1 -- 172.21.15.92:0/3632910996 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f370c0cd440 msgr2=0x7f370c1645a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.836+0000 7f3710e90700 1 --2- 172.21.15.92:0/3632910996 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f370c0cd440 0x7f370c1645a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3704009e90 tx=0x7f3704009780 comp rx=0 tx=0).stop 2023-10-06T10:38:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.836+0000 7f3710e90700 1 -- 172.21.15.92:0/3632910996 shutdown_connections 2023-10-06T10:38:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.836+0000 7f3710e90700 1 --2- 172.21.15.92:0/3632910996 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f370c0cd440 0x7f370c1645a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.836+0000 7f3710e90700 1 -- 172.21.15.92:0/3632910996 >> 172.21.15.92:0/3632910996 conn(0x7f370c06ece0 msgr2=0x7f370c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:38:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.836+0000 7f3710e90700 1 -- 172.21.15.92:0/3632910996 shutdown_connections 2023-10-06T10:38:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.836+0000 7f3710e90700 1 -- 172.21.15.92:0/3632910996 wait complete. 2023-10-06T10:38:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.837+0000 7f3710e90700 1 Processor -- start 2023-10-06T10:38:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.837+0000 7f3710e90700 1 -- start start 2023-10-06T10:38:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.838+0000 7f3710e90700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f370c0b25e0 0x7f370c0c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.838+0000 7f3710e90700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f370c0b29b0 0x7f370c14b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.838+0000 7f3710e90700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f370c0cd440 0x7f370c1697a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.838+0000 7f3710e90700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c14d9d0 con 0x7f370c0b25e0 2023-10-06T10:38:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.838+0000 7f3710e90700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c14dcb0 con 0x7f370c0b29b0 2023-10-06T10:38:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.838+0000 7f3710e90700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370c14df90 con 0x7f370c0cd440 2023-10-06T10:38:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.838+0000 7f370a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f370c0b25e0 0x7f370c0c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.838+0000 7f370a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f370c0b25e0 0x7f370c0c6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47244/0 (socket says 172.21.15.92:47244) 2023-10-06T10:38:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.838+0000 7f370a59c700 1 -- 172.21.15.92:0/701798795 learned_addr learned my addr 172.21.15.92:0/701798795 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:38:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.838+0000 7f370ad9d700 1 --2- 172.21.15.92:0/701798795 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f370c0cd440 0x7f370c1697a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.838+0000 7f3709d9b700 1 --2- 172.21.15.92:0/701798795 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f370c0b29b0 0x7f370c14b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.838+0000 7f370a59c700 1 -- 172.21.15.92:0/701798795 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f370c0cd440 msgr2=0x7f370c1697a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.838+0000 7f370a59c700 1 --2- 172.21.15.92:0/701798795 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f370c0cd440 0x7f370c1697a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:59.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.838+0000 7f370a59c700 1 -- 172.21.15.92:0/701798795 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f370c0b29b0 msgr2=0x7f370c14b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:38:59.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.838+0000 7f370a59c700 1 --2- 172.21.15.92:0/701798795 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f370c0b29b0 0x7f370c14b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:38:59.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.838+0000 7f370a59c700 1 -- 172.21.15.92:0/701798795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3704009a20 con 0x7f370c0b25e0 2023-10-06T10:38:59.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.838+0000 7f370a59c700 1 --2- 172.21.15.92:0/701798795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f370c0b25e0 0x7f370c0c6900 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00c7c0 tx=0x7f36fc007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:59.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.838+0000 7f37027fc700 1 -- 172.21.15.92:0/701798795 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc011680 con 0x7f370c0b25e0 2023-10-06T10:38:59.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.838+0000 7f37027fc700 1 -- 172.21.15.92:0/701798795 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc0117e0 con 0x7f370c0b25e0 2023-10-06T10:38:59.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.838+0000 7f3710e90700 1 -- 172.21.15.92:0/701798795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370c14e270 con 0x7f370c0b25e0 2023-10-06T10:38:59.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.838+0000 7f37027fc700 1 -- 172.21.15.92:0/701798795 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc011ba0 con 0x7f370c0b25e0 2023-10-06T10:38:59.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.838+0000 7f3710e90700 1 -- 172.21.15.92:0/701798795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f370c0c6e40 con 0x7f370c0b25e0 2023-10-06T10:38:59.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.839+0000 7f3710e90700 1 -- 172.21.15.92:0/701798795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370c064a90 con 0x7f370c0b25e0 2023-10-06T10:38:59.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.840+0000 7f37027fc700 1 -- 172.21.15.92:0/701798795 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f36fc012440 con 0x7f370c0b25e0 2023-10-06T10:38:59.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.840+0000 7f37027fc700 1 --2- 172.21.15.92:0/701798795 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f36ec066f30 0x7f36ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:38:59.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.840+0000 7f37027fc700 1 -- 172.21.15.92:0/701798795 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f36fc084870 con 0x7f370c0b25e0 2023-10-06T10:38:59.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.840+0000 7f3709d9b700 1 --2- 172.21.15.92:0/701798795 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f36ec066f30 0x7f36ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:38:59.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.840+0000 7f3709d9b700 1 --2- 172.21.15.92:0/701798795 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f36ec066f30 0x7f36ec0693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f370c0d0bf0 tx=0x7f36f4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:38:59.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:38:59.844+0000 7f37027fc700 1 -- 172.21.15.92:0/701798795 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36fc051690 con 0x7f370c0b25e0 2023-10-06T10:39:00.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.027+0000 7f3710e90700 1 -- 172.21.15.92:0/701798795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f370c0cefc0 con 0x7f370c0b25e0 2023-10-06T10:39:00.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.027+0000 7f37027fc700 1 -- 172.21.15.92:0/701798795 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18707 (secure 0 0 0) 0x7f36fc054cb0 con 0x7f370c0b25e0 2023-10-06T10:39:00.030 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:39:00.030 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:21.760233+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:38:14.368713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T10:38:15.692344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T10:38:19.382553+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:39:00.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.030+0000 7f3710e90700 1 -- 172.21.15.92:0/701798795 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f36ec066f30 msgr2=0x7f36ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:00.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.030+0000 7f3710e90700 1 --2- 172.21.15.92:0/701798795 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f36ec066f30 0x7f36ec0693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f370c0d0bf0 tx=0x7f36f4009040 comp rx=0 tx=0).stop 2023-10-06T10:39:00.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.030+0000 7f3710e90700 1 -- 172.21.15.92:0/701798795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f370c0b25e0 msgr2=0x7f370c0c6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:00.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.030+0000 7f3710e90700 1 --2- 172.21.15.92:0/701798795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f370c0b25e0 0x7f370c0c6900 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00c7c0 tx=0x7f36fc007e00 comp rx=0 tx=0).stop 2023-10-06T10:39:00.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.030+0000 7f3710e90700 1 -- 172.21.15.92:0/701798795 shutdown_connections 2023-10-06T10:39:00.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.030+0000 7f3710e90700 1 --2- 172.21.15.92:0/701798795 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f36ec066f30 0x7f36ec0693e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.030+0000 7f3710e90700 1 --2- 172.21.15.92:0/701798795 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f370c0cd440 0x7f370c1697a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.030+0000 7f3710e90700 1 --2- 172.21.15.92:0/701798795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f370c0b25e0 0x7f370c0c6900 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.030+0000 7f3710e90700 1 --2- 172.21.15.92:0/701798795 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f370c0b29b0 0x7f370c14b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.030+0000 7f3710e90700 1 -- 172.21.15.92:0/701798795 >> 172.21.15.92:0/701798795 conn(0x7f370c06ece0 msgr2=0x7f370c0b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:00.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.030+0000 7f3710e90700 1 -- 172.21.15.92:0/701798795 shutdown_connections 2023-10-06T10:39:00.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.030+0000 7f3710e90700 1 -- 172.21.15.92:0/701798795 wait complete. 2023-10-06T10:39:00.044 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T10:38:09.680559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T10:38:10.120786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T10:38:14.368713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T10:38:15.692344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T10:38:19.382553+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T10:39:00.045 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T10:39:00.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.160+0000 7f6762f55700 1 Processor -- start 2023-10-06T10:39:00.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.160+0000 7f6762f55700 1 -- start start 2023-10-06T10:39:00.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.161+0000 7f6762f55700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f675c1594d0 0x7f675c0cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:00.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.161+0000 7f6762f55700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f675c0cbaa0 0x7f675c0ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:00.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.161+0000 7f6762f55700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f675c0cbee0 con 0x7f675c1598a0 2023-10-06T10:39:00.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.161+0000 7f6762f55700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f675c0cf5d0 con 0x7f675c0cbaa0 2023-10-06T10:39:00.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.161+0000 7f6762f55700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f675c0cf8b0 con 0x7f675c1594d0 2023-10-06T10:39:00.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.161+0000 7f67614f2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f675c0cbaa0 0x7f675c0ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:00.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.161+0000 7f675bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f675c1598a0 0x7f675c0ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50350/0 (socket says 172.21.15.92:50350) 2023-10-06T10:39:00.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.161+0000 7f67614f2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f675c0cbaa0 0x7f675c0ceef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47258/0 (socket says 172.21.15.92:47258) 2023-10-06T10:39:00.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.161+0000 7f675bfff700 1 -- 172.21.15.92:0/2543145669 learned_addr learned my addr 172.21.15.92:0/2543145669 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:00.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.161+0000 7f6760cf1700 1 --2- 172.21.15.92:0/2543145669 >> v2:172.21.15.124:3300/0 conn(0x7f675c1594d0 0x7f675c0cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:00.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.161+0000 7f67614f2700 1 -- 172.21.15.92:0/2543145669 >> v2:172.21.15.124:3300/0 conn(0x7f675c1594d0 msgr2=0x7f675c0cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:00.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.161+0000 7f67614f2700 1 --2- 172.21.15.92:0/2543145669 >> v2:172.21.15.124:3300/0 conn(0x7f675c1594d0 0x7f675c0cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.161+0000 7f67614f2700 1 -- 172.21.15.92:0/2543145669 >> v1:172.21.15.92:6789/0 conn(0x7f675c1598a0 legacy=0x7f675c0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:00.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.161+0000 7f67614f2700 1 -- 172.21.15.92:0/2543145669 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675c0cfb90 con 0x7f675c0cbaa0 2023-10-06T10:39:00.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.161+0000 7f675b7fe700 1 -- 172.21.15.92:0/2543145669 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4039510547 0 0) 0x7f675c0cbee0 con 0x7f675c1598a0 2023-10-06T10:39:00.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.162+0000 7f67614f2700 1 --2- 172.21.15.92:0/2543145669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675c0cbaa0 0x7f675c0ceef0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f675c088f70 tx=0x7f67540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318f1f949088d48f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:00.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.162+0000 7f675b7fe700 1 -- 172.21.15.92:0/2543145669 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6754011630 con 0x7f675c0cbaa0 2023-10-06T10:39:00.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.162+0000 7f675b7fe700 1 -- 172.21.15.92:0/2543145669 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6754011790 con 0x7f675c0cbaa0 2023-10-06T10:39:00.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.162+0000 7f675b7fe700 1 -- 172.21.15.92:0/2543145669 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6754011b30 con 0x7f675c0cbaa0 2023-10-06T10:39:00.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.162+0000 7f6762f55700 1 -- 172.21.15.92:0/2543145669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675c0cbaa0 msgr2=0x7f675c0ceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:00.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.162+0000 7f6762f55700 1 --2- 172.21.15.92:0/2543145669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675c0cbaa0 0x7f675c0ceef0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f675c088f70 tx=0x7f67540091a0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.162+0000 7f6762f55700 1 -- 172.21.15.92:0/2543145669 shutdown_connections 2023-10-06T10:39:00.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.162+0000 7f6762f55700 1 --2- 172.21.15.92:0/2543145669 >> v2:172.21.15.124:3300/0 conn(0x7f675c1594d0 0x7f675c0cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.162+0000 7f6762f55700 1 --2- 172.21.15.92:0/2543145669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675c0cbaa0 0x7f675c0ceef0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.162+0000 7f6762f55700 1 -- 172.21.15.92:0/2543145669 >> 172.21.15.92:0/2543145669 conn(0x7f675c089e50 msgr2=0x7f675c099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:00.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.162+0000 7f6762f55700 1 -- 172.21.15.92:0/2543145669 shutdown_connections 2023-10-06T10:39:00.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.162+0000 7f6762f55700 1 -- 172.21.15.92:0/2543145669 wait complete. 2023-10-06T10:39:00.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.163+0000 7f6762f55700 1 Processor -- start 2023-10-06T10:39:00.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.163+0000 7f6762f55700 1 -- start start 2023-10-06T10:39:00.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.164+0000 7f6762f55700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675c0cbaa0 0x7f675c0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:00.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.164+0000 7f6762f55700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f675c1594d0 0x7f675c169ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:00.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.164+0000 7f6762f55700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f675c1598a0 0x7f675c16a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:00.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.164+0000 7f6762f55700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c14b8a0 con 0x7f675c0cbaa0 2023-10-06T10:39:00.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.164+0000 7f6762f55700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c0b32b0 con 0x7f675c1598a0 2023-10-06T10:39:00.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.164+0000 7f6762f55700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f675c0b4240 con 0x7f675c1594d0 2023-10-06T10:39:00.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.164+0000 7f6760cf1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675c0cbaa0 0x7f675c0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:00.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.164+0000 7f6760cf1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675c0cbaa0 0x7f675c0b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47266/0 (socket says 172.21.15.92:47266) 2023-10-06T10:39:00.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.164+0000 7f6760cf1700 1 -- 172.21.15.92:0/600368774 learned_addr learned my addr 172.21.15.92:0/600368774 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:00.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.164+0000 7f67614f2700 1 --2- 172.21.15.92:0/600368774 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f675c1598a0 0x7f675c16a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:00.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.164+0000 7f675bfff700 1 --2- 172.21.15.92:0/600368774 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f675c1594d0 0x7f675c169ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:00.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.165+0000 7f6760cf1700 1 -- 172.21.15.92:0/600368774 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f675c1594d0 msgr2=0x7f675c169ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:00.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.165+0000 7f6760cf1700 1 --2- 172.21.15.92:0/600368774 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f675c1594d0 0x7f675c169ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.165+0000 7f6760cf1700 1 -- 172.21.15.92:0/600368774 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f675c1598a0 msgr2=0x7f675c16a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:00.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.165+0000 7f6760cf1700 1 --2- 172.21.15.92:0/600368774 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f675c1598a0 0x7f675c16a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.165+0000 7f6760cf1700 1 -- 172.21.15.92:0/600368774 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6754009420 con 0x7f675c0cbaa0 2023-10-06T10:39:00.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.165+0000 7f6760cf1700 1 --2- 172.21.15.92:0/600368774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675c0cbaa0 0x7f675c0b3d00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f674c009ef0 tx=0x7f674c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:00.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.165+0000 7f6758ff9700 1 -- 172.21.15.92:0/600368774 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c00cc20 con 0x7f675c0cbaa0 2023-10-06T10:39:00.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.165+0000 7f6758ff9700 1 -- 172.21.15.92:0/600368774 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674c00f040 con 0x7f675c0cbaa0 2023-10-06T10:39:00.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.165+0000 7f6762f55700 1 -- 172.21.15.92:0/600368774 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f675c16a810 con 0x7f675c0cbaa0 2023-10-06T10:39:00.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.165+0000 7f6758ff9700 1 -- 172.21.15.92:0/600368774 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c014480 con 0x7f675c0cbaa0 2023-10-06T10:39:00.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.165+0000 7f6762f55700 1 -- 172.21.15.92:0/600368774 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675c0b3500 con 0x7f675c0cbaa0 2023-10-06T10:39:00.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.166+0000 7f6762f55700 1 -- 172.21.15.92:0/600368774 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f675c064a90 con 0x7f675c0cbaa0 2023-10-06T10:39:00.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.166+0000 7f6758ff9700 1 -- 172.21.15.92:0/600368774 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f674c0147f0 con 0x7f675c0cbaa0 2023-10-06T10:39:00.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.167+0000 7f6758ff9700 1 --2- 172.21.15.92:0/600368774 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6748066f30 0x7f67480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:00.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.167+0000 7f6758ff9700 1 -- 172.21.15.92:0/600368774 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f674c085320 con 0x7f675c0cbaa0 2023-10-06T10:39:00.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.167+0000 7f675bfff700 1 --2- 172.21.15.92:0/600368774 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6748066f30 0x7f67480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:00.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.167+0000 7f675bfff700 1 --2- 172.21.15.92:0/600368774 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6748066f30 0x7f67480693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f675c0cad00 tx=0x7f6750009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:00.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.171+0000 7f6758ff9700 1 -- 172.21.15.92:0/600368774 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f674c052140 con 0x7f675c0cbaa0 2023-10-06T10:39:00.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.353+0000 7f6762f55700 1 -- 172.21.15.92:0/600368774 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f675c0cfe70 con 0x7f675c0cbaa0 2023-10-06T10:39:00.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.353+0000 7f6758ff9700 1 -- 172.21.15.92:0/600368774 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f674c055760 con 0x7f675c0cbaa0 2023-10-06T10:39:00.356 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T10:39:00.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.356+0000 7f6762f55700 1 -- 172.21.15.92:0/600368774 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6748066f30 msgr2=0x7f67480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:00.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.356+0000 7f6762f55700 1 --2- 172.21.15.92:0/600368774 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6748066f30 0x7f67480693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f675c0cad00 tx=0x7f6750009040 comp rx=0 tx=0).stop 2023-10-06T10:39:00.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.356+0000 7f6762f55700 1 -- 172.21.15.92:0/600368774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675c0cbaa0 msgr2=0x7f675c0b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:00.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.356+0000 7f6762f55700 1 --2- 172.21.15.92:0/600368774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675c0cbaa0 0x7f675c0b3d00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f674c009ef0 tx=0x7f674c00b040 comp rx=0 tx=0).stop 2023-10-06T10:39:00.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.356+0000 7f6762f55700 1 -- 172.21.15.92:0/600368774 shutdown_connections 2023-10-06T10:39:00.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.356+0000 7f6762f55700 1 --2- 172.21.15.92:0/600368774 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6748066f30 0x7f67480693e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.356+0000 7f6762f55700 1 --2- 172.21.15.92:0/600368774 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f675c1598a0 0x7f675c16a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.356+0000 7f6762f55700 1 --2- 172.21.15.92:0/600368774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675c0cbaa0 0x7f675c0b3d00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.356+0000 7f6762f55700 1 --2- 172.21.15.92:0/600368774 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f675c1594d0 0x7f675c169ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.356+0000 7f6762f55700 1 -- 172.21.15.92:0/600368774 >> 172.21.15.92:0/600368774 conn(0x7f675c089e50 msgr2=0x7f675c14c150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:00.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.356+0000 7f6762f55700 1 -- 172.21.15.92:0/600368774 shutdown_connections 2023-10-06T10:39:00.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.356+0000 7f6762f55700 1 -- 172.21.15.92:0/600368774 wait complete. 2023-10-06T10:39:00.370 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T10:39:00.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.485+0000 7ffb148b2700 1 Processor -- start 2023-10-06T10:39:00.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.485+0000 7ffb148b2700 1 -- start start 2023-10-06T10:39:00.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb148b2700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ffb100cd3c0 0x7ffb100ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:00.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb148b2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb100cc970 con 0x7ffb100b2620 2023-10-06T10:39:00.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb148b2700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb100cbd60 con 0x7ffb100b29f0 2023-10-06T10:39:00.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb148b2700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb100cc040 con 0x7ffb100cd3c0 2023-10-06T10:39:00.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb0ed9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ffb100b2620 0x7ffb100cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50358/0 (socket says 172.21.15.92:50358) 2023-10-06T10:39:00.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb0ed9d700 1 -- 172.21.15.92:0/3994755818 learned_addr learned my addr 172.21.15.92:0/3994755818 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:00.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb0e59c700 1 --2- 172.21.15.92:0/3994755818 >> v2:172.21.15.124:3300/0 conn(0x7ffb100cd3c0 0x7ffb100ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:00.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb0dd9b700 1 -- 172.21.15.92:0/3994755818 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2195193407 0 0) 0x7ffb100cc970 con 0x7ffb100b2620 2023-10-06T10:39:00.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb0dd9b700 1 -- 172.21.15.92:0/3994755818 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffafc003570 con 0x7ffb100b2620 2023-10-06T10:39:00.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb0dd9b700 1 -- 172.21.15.92:0/3994755818 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 565067010 0 0) 0x7ffafc003570 con 0x7ffb100b2620 2023-10-06T10:39:00.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb0dd9b700 1 -- 172.21.15.92:0/3994755818 >> v2:172.21.15.124:3300/0 conn(0x7ffb100cd3c0 msgr2=0x7ffb100ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:00.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb0dd9b700 1 --2- 172.21.15.92:0/3994755818 >> v2:172.21.15.124:3300/0 conn(0x7ffb100cd3c0 0x7ffb100ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb0dd9b700 1 -- 172.21.15.92:0/3994755818 >> v1:172.21.15.124:6789/0 conn(0x7ffb100b29f0 legacy=0x7ffb100c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:00.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb0dd9b700 1 -- 172.21.15.92:0/3994755818 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb100cc320 con 0x7ffb100b2620 2023-10-06T10:39:00.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb0dd9b700 1 -- 172.21.15.92:0/3994755818 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7ffb08001970 con 0x7ffb100b2620 2023-10-06T10:39:00.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb0dd9b700 1 -- 172.21.15.92:0/3994755818 >> v1:172.21.15.92:6789/0 conn(0x7ffb100b2620 legacy=0x7ffb100cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:00.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb0dd9b700 1 --2- 172.21.15.92:0/3994755818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffafc003a60 0x7ffafc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:00.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb0dd9b700 1 --2- 172.21.15.92:0/3994755818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffafc004810 0x7ffafc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:00.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb0dd9b700 1 --2- 172.21.15.92:0/3994755818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffafc0051c0 0x7ffafc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:00.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb0dd9b700 1 -- 172.21.15.92:0/3994755818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb100cc040 con 0x7ffafc0051c0 2023-10-06T10:39:00.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb0dd9b700 1 -- 172.21.15.92:0/3994755818 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc00c0d0 con 0x7ffafc004810 2023-10-06T10:39:00.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb0dd9b700 1 -- 172.21.15.92:0/3994755818 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc00c290 con 0x7ffafc003a60 2023-10-06T10:39:00.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb0f59e700 1 --2- 172.21.15.92:0/3994755818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffafc0051c0 0x7ffafc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:00.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb0ed9d700 1 --2- 172.21.15.92:0/3994755818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffafc003a60 0x7ffafc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:00.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.486+0000 7ffb0e59c700 1 --2- 172.21.15.92:0/3994755818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffafc004810 0x7ffafc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:00.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.487+0000 7ffb0f59e700 1 -- 172.21.15.92:0/3994755818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffafc003a60 msgr2=0x7ffafc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:00.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.487+0000 7ffb0f59e700 1 --2- 172.21.15.92:0/3994755818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffafc003a60 0x7ffafc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.487+0000 7ffb0f59e700 1 -- 172.21.15.92:0/3994755818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffafc004810 msgr2=0x7ffafc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:00.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.487+0000 7ffb0f59e700 1 --2- 172.21.15.92:0/3994755818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffafc004810 0x7ffafc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.487+0000 7ffb0f59e700 1 -- 172.21.15.92:0/3994755818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc003570 con 0x7ffafc0051c0 2023-10-06T10:39:00.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.487+0000 7ffb0f59e700 1 -- 172.21.15.92:0/3994755818 reap_dead start 2023-10-06T10:39:00.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.487+0000 7ffb0f59e700 1 --2- 172.21.15.92:0/3994755818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffafc0051c0 0x7ffafc00ba60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffb00001880 tx=0x7ffb00007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af17f679e01cbd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:00.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.487+0000 7ffb0dd9b700 1 -- 172.21.15.92:0/3994755818 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb00012420 con 0x7ffafc0051c0 2023-10-06T10:39:00.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.487+0000 7ffb0dd9b700 1 -- 172.21.15.92:0/3994755818 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb000125a0 con 0x7ffafc0051c0 2023-10-06T10:39:00.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.487+0000 7ffb0dd9b700 1 -- 172.21.15.92:0/3994755818 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb00012960 con 0x7ffafc0051c0 2023-10-06T10:39:00.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.487+0000 7ffb148b2700 1 -- 172.21.15.92:0/3994755818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffafc0051c0 msgr2=0x7ffafc00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:00.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.487+0000 7ffb148b2700 1 --2- 172.21.15.92:0/3994755818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffafc0051c0 0x7ffafc00ba60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffb00001880 tx=0x7ffb00007d30 comp rx=0 tx=0).stop 2023-10-06T10:39:00.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.487+0000 7ffb148b2700 1 -- 172.21.15.92:0/3994755818 shutdown_connections 2023-10-06T10:39:00.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.487+0000 7ffb148b2700 1 --2- 172.21.15.92:0/3994755818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffafc0051c0 0x7ffafc00ba60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.487+0000 7ffb148b2700 1 -- 172.21.15.92:0/3994755818 >> 172.21.15.92:0/3994755818 conn(0x7ffb10099520 msgr2=0x7ffb1006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:00.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.487+0000 7ffb148b2700 1 -- 172.21.15.92:0/3994755818 shutdown_connections 2023-10-06T10:39:00.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.487+0000 7ffb148b2700 1 -- 172.21.15.92:0/3994755818 wait complete. 2023-10-06T10:39:00.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.489+0000 7ffb148b2700 1 Processor -- start 2023-10-06T10:39:00.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.489+0000 7ffb148b2700 1 -- start start 2023-10-06T10:39:00.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.489+0000 7ffb148b2700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffafc0051c0 0x7ffb10153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:00.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.489+0000 7ffb148b2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb100b3fb0 0x7ffb100b5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:00.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.489+0000 7ffb148b2700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb100b4420 0x7ffb100b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:00.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.489+0000 7ffb148b2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb100c92c0 con 0x7ffb100b3fb0 2023-10-06T10:39:00.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.489+0000 7ffb148b2700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb100c8450 con 0x7ffafc0051c0 2023-10-06T10:39:00.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.489+0000 7ffb148b2700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb100c86f0 con 0x7ffb100b4420 2023-10-06T10:39:00.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.489+0000 7ffb0e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb100b3fb0 0x7ffb100b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:00.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.489+0000 7ffb0e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb100b3fb0 0x7ffb100b5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47280/0 (socket says 172.21.15.92:47280) 2023-10-06T10:39:00.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.489+0000 7ffb0e59c700 1 -- 172.21.15.92:0/3796266755 learned_addr learned my addr 172.21.15.92:0/3796266755 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:00.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.490+0000 7ffb0ed9d700 1 --2- 172.21.15.92:0/3796266755 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffafc0051c0 0x7ffb10153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:00.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.490+0000 7ffb0f59e700 1 --2- 172.21.15.92:0/3796266755 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb100b4420 0x7ffb100b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:00.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.490+0000 7ffb0e59c700 1 -- 172.21.15.92:0/3796266755 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb100b4420 msgr2=0x7ffb100b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:00.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.490+0000 7ffb0e59c700 1 --2- 172.21.15.92:0/3796266755 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb100b4420 0x7ffb100b57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.490+0000 7ffb0e59c700 1 -- 172.21.15.92:0/3796266755 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffafc0051c0 msgr2=0x7ffb10153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:00.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.490+0000 7ffb0e59c700 1 --2- 172.21.15.92:0/3796266755 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffafc0051c0 0x7ffb10153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.490+0000 7ffb0e59c700 1 -- 172.21.15.92:0/3796266755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb100c8990 con 0x7ffb100b3fb0 2023-10-06T10:39:00.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.490+0000 7ffb0e59c700 1 --2- 172.21.15.92:0/3796266755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb100b3fb0 0x7ffb100b5290 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009fd0 tx=0x7ffaf80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:00.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.490+0000 7ffb06ffd700 1 -- 172.21.15.92:0/3796266755 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf800f940 con 0x7ffb100b3fb0 2023-10-06T10:39:00.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.490+0000 7ffb06ffd700 1 -- 172.21.15.92:0/3796266755 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf800faa0 con 0x7ffb100b3fb0 2023-10-06T10:39:00.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.490+0000 7ffb148b2700 1 -- 172.21.15.92:0/3796266755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb100c8c70 con 0x7ffb100b3fb0 2023-10-06T10:39:00.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.490+0000 7ffb06ffd700 1 -- 172.21.15.92:0/3796266755 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf800fe20 con 0x7ffb100b3fb0 2023-10-06T10:39:00.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.490+0000 7ffb148b2700 1 -- 172.21.15.92:0/3796266755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb10154040 con 0x7ffb100b3fb0 2023-10-06T10:39:00.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.491+0000 7ffb148b2700 1 -- 172.21.15.92:0/3796266755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb1005f230 con 0x7ffb100b3fb0 2023-10-06T10:39:00.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.491+0000 7ffb06ffd700 1 -- 172.21.15.92:0/3796266755 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffaf8014650 con 0x7ffb100b3fb0 2023-10-06T10:39:00.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.492+0000 7ffb06ffd700 1 --2- 172.21.15.92:0/3796266755 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ffaf0066f30 0x7ffaf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:00.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.492+0000 7ffb06ffd700 1 -- 172.21.15.92:0/3796266755 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ffaf8018030 con 0x7ffb100b3fb0 2023-10-06T10:39:00.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.492+0000 7ffb0ed9d700 1 --2- 172.21.15.92:0/3796266755 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ffaf0066f30 0x7ffaf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:00.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.492+0000 7ffb0ed9d700 1 --2- 172.21.15.92:0/3796266755 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ffaf0066f30 0x7ffaf00693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffb080031f0 tx=0x7ffb08009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:00.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.496+0000 7ffb06ffd700 1 -- 172.21.15.92:0/3796266755 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffaf80521a0 con 0x7ffb100b3fb0 2023-10-06T10:39:00.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.678+0000 7ffb148b2700 1 -- 172.21.15.92:0/3796266755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb100cc600 con 0x7ffb100b3fb0 2023-10-06T10:39:00.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.679+0000 7ffb06ffd700 1 -- 172.21.15.92:0/3796266755 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffaf80557c0 con 0x7ffb100b3fb0 2023-10-06T10:39:00.681 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T10:39:00.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.681+0000 7ffb148b2700 1 -- 172.21.15.92:0/3796266755 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ffaf0066f30 msgr2=0x7ffaf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:00.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.681+0000 7ffb148b2700 1 --2- 172.21.15.92:0/3796266755 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ffaf0066f30 0x7ffaf00693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffb080031f0 tx=0x7ffb08009040 comp rx=0 tx=0).stop 2023-10-06T10:39:00.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.681+0000 7ffb148b2700 1 -- 172.21.15.92:0/3796266755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb100b3fb0 msgr2=0x7ffb100b5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:00.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.681+0000 7ffb148b2700 1 --2- 172.21.15.92:0/3796266755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb100b3fb0 0x7ffb100b5290 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009fd0 tx=0x7ffaf80091a0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.681+0000 7ffb148b2700 1 -- 172.21.15.92:0/3796266755 shutdown_connections 2023-10-06T10:39:00.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.681+0000 7ffb148b2700 1 --2- 172.21.15.92:0/3796266755 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ffaf0066f30 0x7ffaf00693e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.681+0000 7ffb148b2700 1 --2- 172.21.15.92:0/3796266755 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb100b4420 0x7ffb100b57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.681+0000 7ffb148b2700 1 --2- 172.21.15.92:0/3796266755 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffafc0051c0 0x7ffb10153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.681+0000 7ffb148b2700 1 --2- 172.21.15.92:0/3796266755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb100b3fb0 0x7ffb100b5290 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.681+0000 7ffb148b2700 1 -- 172.21.15.92:0/3796266755 >> 172.21.15.92:0/3796266755 conn(0x7ffb10099520 msgr2=0x7ffb10068110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:00.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.681+0000 7ffb148b2700 1 -- 172.21.15.92:0/3796266755 shutdown_connections 2023-10-06T10:39:00.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.681+0000 7ffb148b2700 1 -- 172.21.15.92:0/3796266755 wait complete. 2023-10-06T10:39:00.695 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T10:39:00.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.809+0000 7f17ca7f5700 1 Processor -- start 2023-10-06T10:39:00.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.809+0000 7f17ca7f5700 1 -- start start 2023-10-06T10:39:00.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.810+0000 7f17ca7f5700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f17c41594d0 0x7f17c40cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:00.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.810+0000 7f17ca7f5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f17c40cbaa0 0x7f17c40ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:00.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.810+0000 7f17ca7f5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c40cbee0 con 0x7f17c41598a0 2023-10-06T10:39:00.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.810+0000 7f17ca7f5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c40cf5d0 con 0x7f17c40cbaa0 2023-10-06T10:39:00.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.810+0000 7f17ca7f5700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c40cf8b0 con 0x7f17c41594d0 2023-10-06T10:39:00.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.810+0000 7f17c8d92700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f17c40cbaa0 0x7f17c40ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:00.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.810+0000 7f17c8d92700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f17c40cbaa0 0x7f17c40ceef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47282/0 (socket says 172.21.15.92:47282) 2023-10-06T10:39:00.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.810+0000 7f17c8d92700 1 -- 172.21.15.92:0/323385487 learned_addr learned my addr 172.21.15.92:0/323385487 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:00.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.810+0000 7f17c37fe700 1 --1- 172.21.15.92:0/323385487 >> v1:172.21.15.92:6789/0 conn(0x7f17c41598a0 0x7f17c40ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50366/0 (socket says 172.21.15.92:50366) 2023-10-06T10:39:00.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.810+0000 7f17c3fff700 1 --2- 172.21.15.92:0/323385487 >> v2:172.21.15.124:3300/0 conn(0x7f17c41594d0 0x7f17c40cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:00.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.810+0000 7f17c8d92700 1 -- 172.21.15.92:0/323385487 >> v2:172.21.15.124:3300/0 conn(0x7f17c41594d0 msgr2=0x7f17c40cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:00.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.810+0000 7f17c8d92700 1 --2- 172.21.15.92:0/323385487 >> v2:172.21.15.124:3300/0 conn(0x7f17c41594d0 0x7f17c40cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.810+0000 7f17c8d92700 1 -- 172.21.15.92:0/323385487 >> v1:172.21.15.92:6789/0 conn(0x7f17c41598a0 legacy=0x7f17c40ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:00.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.810+0000 7f17c8d92700 1 -- 172.21.15.92:0/323385487 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c40cfb90 con 0x7f17c40cbaa0 2023-10-06T10:39:00.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.810+0000 7f17c2ffd700 1 -- 172.21.15.92:0/323385487 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478926372 0 0) 0x7f17c40cbee0 con 0x7f17c41598a0 2023-10-06T10:39:00.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.810+0000 7f17c3fff700 1 --2- 172.21.15.92:0/323385487 >> v2:172.21.15.124:3300/0 conn(0x7f17c41594d0 0x7f17c40cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:00.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.811+0000 7f17c8d92700 1 --2- 172.21.15.92:0/323385487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17c40cbaa0 0x7f17c40ceef0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f17c4088f70 tx=0x7f17bc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceed56de44121aea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:00.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.811+0000 7f17c2ffd700 1 -- 172.21.15.92:0/323385487 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc00e070 con 0x7f17c40cbaa0 2023-10-06T10:39:00.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.811+0000 7f17c2ffd700 1 -- 172.21.15.92:0/323385487 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc00ba50 con 0x7f17c40cbaa0 2023-10-06T10:39:00.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.811+0000 7f17c2ffd700 1 -- 172.21.15.92:0/323385487 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc0092e0 con 0x7f17c40cbaa0 2023-10-06T10:39:00.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.811+0000 7f17ca7f5700 1 -- 172.21.15.92:0/323385487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17c40cbaa0 msgr2=0x7f17c40ceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:00.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.811+0000 7f17ca7f5700 1 --2- 172.21.15.92:0/323385487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17c40cbaa0 0x7f17c40ceef0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f17c4088f70 tx=0x7f17bc009840 comp rx=0 tx=0).stop 2023-10-06T10:39:00.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.811+0000 7f17ca7f5700 1 -- 172.21.15.92:0/323385487 shutdown_connections 2023-10-06T10:39:00.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.811+0000 7f17ca7f5700 1 --2- 172.21.15.92:0/323385487 >> v2:172.21.15.124:3300/0 conn(0x7f17c41594d0 0x7f17c40cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.811+0000 7f17ca7f5700 1 --2- 172.21.15.92:0/323385487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17c40cbaa0 0x7f17c40ceef0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.811+0000 7f17ca7f5700 1 -- 172.21.15.92:0/323385487 >> 172.21.15.92:0/323385487 conn(0x7f17c4089e50 msgr2=0x7f17c4099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:00.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.811+0000 7f17ca7f5700 1 -- 172.21.15.92:0/323385487 shutdown_connections 2023-10-06T10:39:00.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.811+0000 7f17ca7f5700 1 -- 172.21.15.92:0/323385487 wait complete. 2023-10-06T10:39:00.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.812+0000 7f17ca7f5700 1 Processor -- start 2023-10-06T10:39:00.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.813+0000 7f17ca7f5700 1 -- start start 2023-10-06T10:39:00.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.813+0000 7f17ca7f5700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17c40cbaa0 0x7f17c40c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:00.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.813+0000 7f17ca7f5700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17c41594d0 0x7f17c414ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:00.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.813+0000 7f17ca7f5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17c41598a0 0x7f17c40c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:00.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.813+0000 7f17ca7f5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c414b750 con 0x7f17c41598a0 2023-10-06T10:39:00.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.813+0000 7f17ca7f5700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c4169760 con 0x7f17c40cbaa0 2023-10-06T10:39:00.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.813+0000 7f17ca7f5700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c40c86d0 con 0x7f17c41594d0 2023-10-06T10:39:00.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.813+0000 7f17c8d92700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17c41598a0 0x7f17c40c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:00.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.813+0000 7f17c8d92700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17c41598a0 0x7f17c40c5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47284/0 (socket says 172.21.15.92:47284) 2023-10-06T10:39:00.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.813+0000 7f17c8d92700 1 -- 172.21.15.92:0/3359095642 learned_addr learned my addr 172.21.15.92:0/3359095642 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:00.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.813+0000 7f17c3fff700 1 --2- 172.21.15.92:0/3359095642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17c40cbaa0 0x7f17c40c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:00.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.813+0000 7f17c37fe700 1 --2- 172.21.15.92:0/3359095642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17c41594d0 0x7f17c414ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:00.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.814+0000 7f17c8d92700 1 -- 172.21.15.92:0/3359095642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17c41594d0 msgr2=0x7f17c414ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:00.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.814+0000 7f17c8d92700 1 --2- 172.21.15.92:0/3359095642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17c41594d0 0x7f17c414ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.814+0000 7f17c8d92700 1 -- 172.21.15.92:0/3359095642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17c40cbaa0 msgr2=0x7f17c40c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:00.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.814+0000 7f17c8d92700 1 --2- 172.21.15.92:0/3359095642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17c40cbaa0 0x7f17c40c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:00.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.814+0000 7f17c8d92700 1 -- 172.21.15.92:0/3359095642 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17bc009ae0 con 0x7f17c41598a0 2023-10-06T10:39:00.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.814+0000 7f17c8d92700 1 --2- 172.21.15.92:0/3359095642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17c41598a0 0x7f17c40c5b10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f17c40cc070 tx=0x7f17bc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:00.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.814+0000 7f17abfff700 1 -- 172.21.15.92:0/3359095642 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc008d60 con 0x7f17c41598a0 2023-10-06T10:39:00.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.814+0000 7f17abfff700 1 -- 172.21.15.92:0/3359095642 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc026460 con 0x7f17c41598a0 2023-10-06T10:39:00.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.814+0000 7f17ca7f5700 1 -- 172.21.15.92:0/3359095642 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c4169a20 con 0x7f17c41598a0 2023-10-06T10:39:00.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.814+0000 7f17abfff700 1 -- 172.21.15.92:0/3359095642 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc026820 con 0x7f17c41598a0 2023-10-06T10:39:00.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.814+0000 7f17ca7f5700 1 -- 172.21.15.92:0/3359095642 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c4169fe0 con 0x7f17c41598a0 2023-10-06T10:39:00.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.815+0000 7f17ca7f5700 1 -- 172.21.15.92:0/3359095642 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c416a200 con 0x7f17c41598a0 2023-10-06T10:39:00.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.815+0000 7f17abfff700 1 -- 172.21.15.92:0/3359095642 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f17bc003040 con 0x7f17c41598a0 2023-10-06T10:39:00.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.816+0000 7f17abfff700 1 --2- 172.21.15.92:0/3359095642 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f17b0066f30 0x7f17b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:00.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.816+0000 7f17abfff700 1 -- 172.21.15.92:0/3359095642 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f17bc086de0 con 0x7f17c41598a0 2023-10-06T10:39:00.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.816+0000 7f17c3fff700 1 --2- 172.21.15.92:0/3359095642 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f17b0066f30 0x7f17b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:00.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.816+0000 7f17c3fff700 1 --2- 172.21.15.92:0/3359095642 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f17b0066f30 0x7f17b00693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f17c40ca870 tx=0x7f17b4006f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:00.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:00.820+0000 7f17abfff700 1 -- 172.21.15.92:0/3359095642 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17bc053c00 con 0x7f17c41598a0 2023-10-06T10:39:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.001+0000 7f17ca7f5700 1 -- 172.21.15.92:0/3359095642 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f17c4064a90 con 0x7f17c41598a0 2023-10-06T10:39:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.001+0000 7f17abfff700 1 -- 172.21.15.92:0/3359095642 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f17bc057220 con 0x7f17c41598a0 2023-10-06T10:39:01.003 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T10:39:01.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.003+0000 7f17ca7f5700 1 -- 172.21.15.92:0/3359095642 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f17b0066f30 msgr2=0x7f17b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:01.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.003+0000 7f17ca7f5700 1 --2- 172.21.15.92:0/3359095642 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f17b0066f30 0x7f17b00693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f17c40ca870 tx=0x7f17b4006f90 comp rx=0 tx=0).stop 2023-10-06T10:39:01.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.003+0000 7f17ca7f5700 1 -- 172.21.15.92:0/3359095642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17c41598a0 msgr2=0x7f17c40c5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:01.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.003+0000 7f17ca7f5700 1 --2- 172.21.15.92:0/3359095642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17c41598a0 0x7f17c40c5b10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f17c40cc070 tx=0x7f17bc018000 comp rx=0 tx=0).stop 2023-10-06T10:39:01.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.003+0000 7f17ca7f5700 1 -- 172.21.15.92:0/3359095642 shutdown_connections 2023-10-06T10:39:01.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.003+0000 7f17ca7f5700 1 --2- 172.21.15.92:0/3359095642 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f17b0066f30 0x7f17b00693e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.003+0000 7f17ca7f5700 1 --2- 172.21.15.92:0/3359095642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17c41598a0 0x7f17c40c5b10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.003+0000 7f17ca7f5700 1 --2- 172.21.15.92:0/3359095642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17c40cbaa0 0x7f17c40c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.003+0000 7f17ca7f5700 1 --2- 172.21.15.92:0/3359095642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17c41594d0 0x7f17c414ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.003+0000 7f17ca7f5700 1 -- 172.21.15.92:0/3359095642 >> 172.21.15.92:0/3359095642 conn(0x7f17c4089e50 msgr2=0x7f17c4068ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:01.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.003+0000 7f17ca7f5700 1 -- 172.21.15.92:0/3359095642 shutdown_connections 2023-10-06T10:39:01.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.003+0000 7f17ca7f5700 1 -- 172.21.15.92:0/3359095642 wait complete. 2023-10-06T10:39:01.017 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T10:39:01.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.132+0000 7f1b3f59e700 1 Processor -- start 2023-10-06T10:39:01.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.132+0000 7f1b3f59e700 1 -- start start 2023-10-06T10:39:01.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.133+0000 7f1b3f59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1b400b2620 0x7f1b400cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:01.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.133+0000 7f1b3f59e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1b400b29f0 0x7f1b400cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:01.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.133+0000 7f1b3f59e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b400cc970 con 0x7f1b400ceaa0 2023-10-06T10:39:01.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.133+0000 7f1b3f59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b400cbd60 con 0x7f1b400b2620 2023-10-06T10:39:01.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.133+0000 7f1b3f59e700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b400cc040 con 0x7f1b400b29f0 2023-10-06T10:39:01.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.133+0000 7f1b3e59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1b400b2620 0x7f1b400cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:01.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.133+0000 7f1b3e59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1b400b2620 0x7f1b400cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47298/0 (socket says 172.21.15.92:47298) 2023-10-06T10:39:01.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.133+0000 7f1b3e59c700 1 -- 172.21.15.92:0/3778073099 learned_addr learned my addr 172.21.15.92:0/3778073099 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:01.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.133+0000 7f1b35d9b700 1 --2- 172.21.15.92:0/3778073099 >> v2:172.21.15.124:3300/0 conn(0x7f1b400b29f0 0x7f1b400cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:01.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.133+0000 7f1b3e59c700 1 -- 172.21.15.92:0/3778073099 >> v2:172.21.15.124:3300/0 conn(0x7f1b400b29f0 msgr2=0x7f1b400cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:01.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.133+0000 7f1b3e59c700 1 --2- 172.21.15.92:0/3778073099 >> v2:172.21.15.124:3300/0 conn(0x7f1b400b29f0 0x7f1b400cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.133+0000 7f1b3e59c700 1 -- 172.21.15.92:0/3778073099 >> v1:172.21.15.124:6789/0 conn(0x7f1b400ceaa0 legacy=0x7f1b400cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:01.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.133+0000 7f1b3e59c700 1 -- 172.21.15.92:0/3778073099 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b400cc320 con 0x7f1b400b2620 2023-10-06T10:39:01.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.133+0000 7f1b3e59c700 1 --2- 172.21.15.92:0/3778073099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b400b2620 0x7f1b400cccb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1b400b62d0 tx=0x7f1b30009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c58341fd42cd692 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:01.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.133+0000 7f1b3dd9b700 1 -- 172.21.15.92:0/3778073099 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b30005c10 con 0x7f1b400b2620 2023-10-06T10:39:01.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.134+0000 7f1b3dd9b700 1 -- 172.21.15.92:0/3778073099 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b30005d70 con 0x7f1b400b2620 2023-10-06T10:39:01.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.134+0000 7f1b3dd9b700 1 -- 172.21.15.92:0/3778073099 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3000b840 con 0x7f1b400b2620 2023-10-06T10:39:01.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.134+0000 7f1b3f59e700 1 -- 172.21.15.92:0/3778073099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b400b2620 msgr2=0x7f1b400cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:01.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.134+0000 7f1b3f59e700 1 --2- 172.21.15.92:0/3778073099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b400b2620 0x7f1b400cccb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1b400b62d0 tx=0x7f1b30009220 comp rx=0 tx=0).stop 2023-10-06T10:39:01.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.134+0000 7f1b3f59e700 1 -- 172.21.15.92:0/3778073099 shutdown_connections 2023-10-06T10:39:01.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.134+0000 7f1b3f59e700 1 --2- 172.21.15.92:0/3778073099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b400b2620 0x7f1b400cccb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.134+0000 7f1b3f59e700 1 --2- 172.21.15.92:0/3778073099 >> v2:172.21.15.124:3300/0 conn(0x7f1b400b29f0 0x7f1b400cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.134+0000 7f1b3f59e700 1 -- 172.21.15.92:0/3778073099 >> 172.21.15.92:0/3778073099 conn(0x7f1b40099520 msgr2=0x7f1b4006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:01.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.134+0000 7f1b3f59e700 1 -- 172.21.15.92:0/3778073099 shutdown_connections 2023-10-06T10:39:01.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.134+0000 7f1b3f59e700 1 -- 172.21.15.92:0/3778073099 wait complete. 2023-10-06T10:39:01.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.135+0000 7f1b3f59e700 1 Processor -- start 2023-10-06T10:39:01.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.135+0000 7f1b3f59e700 1 -- start start 2023-10-06T10:39:01.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.136+0000 7f1b3f59e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b400b2620 0x7f1b400c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:01.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.136+0000 7f1b3f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b400b29f0 0x7f1b400c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:01.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.136+0000 7f1b3f59e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b400ceaa0 0x7f1b400c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:01.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.136+0000 7f1b3f59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b40159bc0 con 0x7f1b400b29f0 2023-10-06T10:39:01.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.136+0000 7f1b3f59e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b400c87e0 con 0x7f1b400b2620 2023-10-06T10:39:01.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.136+0000 7f1b3f59e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b400c8ac0 con 0x7f1b400ceaa0 2023-10-06T10:39:01.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.136+0000 7f1b35d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b400b29f0 0x7f1b400c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:01.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.136+0000 7f1b35d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b400b29f0 0x7f1b400c92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47306/0 (socket says 172.21.15.92:47306) 2023-10-06T10:39:01.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.136+0000 7f1b35d9b700 1 -- 172.21.15.92:0/930558372 learned_addr learned my addr 172.21.15.92:0/930558372 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:01.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.136+0000 7f1b35d9b700 1 -- 172.21.15.92:0/930558372 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b400ceaa0 msgr2=0x7f1b400c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:01.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.136+0000 7f1b35d9b700 1 --2- 172.21.15.92:0/930558372 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b400ceaa0 0x7f1b400c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.136+0000 7f1b35d9b700 1 -- 172.21.15.92:0/930558372 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b400b2620 msgr2=0x7f1b400c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:01.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.136+0000 7f1b35d9b700 1 --2- 172.21.15.92:0/930558372 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b400b2620 0x7f1b400c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.136+0000 7f1b35d9b700 1 -- 172.21.15.92:0/930558372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b30009480 con 0x7f1b400b29f0 2023-10-06T10:39:01.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.136+0000 7f1b35d9b700 1 --2- 172.21.15.92:0/930558372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b400b29f0 0x7f1b400c92f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800d860 tx=0x7f1b28008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:01.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.137+0000 7f1b36ffd700 1 -- 172.21.15.92:0/930558372 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b280125f0 con 0x7f1b400b29f0 2023-10-06T10:39:01.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.137+0000 7f1b36ffd700 1 -- 172.21.15.92:0/930558372 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b28012750 con 0x7f1b400b29f0 2023-10-06T10:39:01.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.137+0000 7f1b3f59e700 1 -- 172.21.15.92:0/930558372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b400c8da0 con 0x7f1b400b29f0 2023-10-06T10:39:01.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.137+0000 7f1b36ffd700 1 -- 172.21.15.92:0/930558372 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b28012ad0 con 0x7f1b400b29f0 2023-10-06T10:39:01.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.137+0000 7f1b3f59e700 1 -- 172.21.15.92:0/930558372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b40169ec0 con 0x7f1b400b29f0 2023-10-06T10:39:01.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.138+0000 7f1b3f59e700 1 -- 172.21.15.92:0/930558372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b40159fd0 con 0x7f1b400b29f0 2023-10-06T10:39:01.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.138+0000 7f1b36ffd700 1 -- 172.21.15.92:0/930558372 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1b2801e460 con 0x7f1b400b29f0 2023-10-06T10:39:01.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.138+0000 7f1b36ffd700 1 --2- 172.21.15.92:0/930558372 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1b20066f30 0x7f1b200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:01.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.138+0000 7f1b36ffd700 1 -- 172.21.15.92:0/930558372 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f1b28085040 con 0x7f1b400b29f0 2023-10-06T10:39:01.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.138+0000 7f1b3e59c700 1 --2- 172.21.15.92:0/930558372 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1b20066f30 0x7f1b200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:01.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.139+0000 7f1b3e59c700 1 --2- 172.21.15.92:0/930558372 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1b20066f30 0x7f1b200693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009260 tx=0x7f1b30015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:01.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.143+0000 7f1b36ffd700 1 -- 172.21.15.92:0/930558372 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b280558f0 con 0x7f1b400b29f0 2023-10-06T10:39:01.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.325+0000 7f1b3f59e700 1 -- 172.21.15.92:0/930558372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1b4005f230 con 0x7f1b400b29f0 2023-10-06T10:39:01.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.325+0000 7f1b36ffd700 1 -- 172.21.15.92:0/930558372 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1b28055480 con 0x7f1b400b29f0 2023-10-06T10:39:01.327 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T10:39:01.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.327+0000 7f1b3f59e700 1 -- 172.21.15.92:0/930558372 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1b20066f30 msgr2=0x7f1b200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:01.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.327+0000 7f1b3f59e700 1 --2- 172.21.15.92:0/930558372 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1b20066f30 0x7f1b200693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009260 tx=0x7f1b30015000 comp rx=0 tx=0).stop 2023-10-06T10:39:01.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.327+0000 7f1b3f59e700 1 -- 172.21.15.92:0/930558372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b400b29f0 msgr2=0x7f1b400c92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:01.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.327+0000 7f1b3f59e700 1 --2- 172.21.15.92:0/930558372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b400b29f0 0x7f1b400c92f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800d860 tx=0x7f1b28008040 comp rx=0 tx=0).stop 2023-10-06T10:39:01.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.327+0000 7f1b3f59e700 1 -- 172.21.15.92:0/930558372 shutdown_connections 2023-10-06T10:39:01.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.327+0000 7f1b3f59e700 1 --2- 172.21.15.92:0/930558372 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1b20066f30 0x7f1b200693e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.327+0000 7f1b3f59e700 1 --2- 172.21.15.92:0/930558372 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b400ceaa0 0x7f1b400c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.327+0000 7f1b3f59e700 1 --2- 172.21.15.92:0/930558372 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b400b2620 0x7f1b400c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.327+0000 7f1b3f59e700 1 --2- 172.21.15.92:0/930558372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b400b29f0 0x7f1b400c92f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.327+0000 7f1b3f59e700 1 -- 172.21.15.92:0/930558372 >> 172.21.15.92:0/930558372 conn(0x7f1b40099520 msgr2=0x7f1b40069010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:01.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.327+0000 7f1b3f59e700 1 -- 172.21.15.92:0/930558372 shutdown_connections 2023-10-06T10:39:01.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.327+0000 7f1b3f59e700 1 -- 172.21.15.92:0/930558372 wait complete. 2023-10-06T10:39:01.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T10:39:01.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.459+0000 7f0e23455700 1 Processor -- start 2023-10-06T10:39:01.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.459+0000 7f0e23455700 1 -- start start 2023-10-06T10:39:01.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.460+0000 7f0e23455700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0e1c1598a0 0x7f0e1c0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:01.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.460+0000 7f0e23455700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1c0cf5d0 con 0x7f0e1c1594d0 2023-10-06T10:39:01.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.460+0000 7f0e23455700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1c0cf8b0 con 0x7f0e1c0cbc70 2023-10-06T10:39:01.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.460+0000 7f0e23455700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e1c0cfb90 con 0x7f0e1c1598a0 2023-10-06T10:39:01.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.460+0000 7f0e211f1700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0e1c1594d0 0x7f0e1c0cb560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50376/0 (socket says 172.21.15.92:50376) 2023-10-06T10:39:01.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.460+0000 7f0e211f1700 1 -- 172.21.15.92:0/386825302 learned_addr learned my addr 172.21.15.92:0/386825302 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:01.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.460+0000 7f0e209f0700 1 --2- 172.21.15.92:0/386825302 >> v2:172.21.15.124:3300/0 conn(0x7f0e1c1598a0 0x7f0e1c0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:01.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.460+0000 7f0e13fff700 1 -- 172.21.15.92:0/386825302 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803321833 0 0) 0x7f0e1c0cf5d0 con 0x7f0e1c1594d0 2023-10-06T10:39:01.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.460+0000 7f0e13fff700 1 -- 172.21.15.92:0/386825302 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e08003570 con 0x7f0e1c1594d0 2023-10-06T10:39:01.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.460+0000 7f0e209f0700 1 -- 172.21.15.92:0/386825302 >> v1:172.21.15.124:6789/0 conn(0x7f0e1c0cbc70 legacy=0x7f0e1c0ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:01.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.460+0000 7f0e209f0700 1 -- 172.21.15.92:0/386825302 >> v1:172.21.15.92:6789/0 conn(0x7f0e1c1594d0 legacy=0x7f0e1c0cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:01.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.460+0000 7f0e209f0700 1 -- 172.21.15.92:0/386825302 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1c0cfe70 con 0x7f0e1c1598a0 2023-10-06T10:39:01.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.461+0000 7f0e209f0700 1 --2- 172.21.15.92:0/386825302 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e1c1598a0 0x7f0e1c0ce7e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c0098e0 tx=0x7f0e0c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4773a52054ee6b23 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:01.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.461+0000 7f0e13fff700 1 -- 172.21.15.92:0/386825302 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c00b640 con 0x7f0e1c1598a0 2023-10-06T10:39:01.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.461+0000 7f0e13fff700 1 -- 172.21.15.92:0/386825302 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0c00b7a0 con 0x7f0e1c1598a0 2023-10-06T10:39:01.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.461+0000 7f0e13fff700 1 -- 172.21.15.92:0/386825302 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c00bb20 con 0x7f0e1c1598a0 2023-10-06T10:39:01.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.461+0000 7f0e23455700 1 -- 172.21.15.92:0/386825302 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e1c1598a0 msgr2=0x7f0e1c0ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:01.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.461+0000 7f0e23455700 1 --2- 172.21.15.92:0/386825302 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e1c1598a0 0x7f0e1c0ce7e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c0098e0 tx=0x7f0e0c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.461+0000 7f0e23455700 1 -- 172.21.15.92:0/386825302 shutdown_connections 2023-10-06T10:39:01.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.461+0000 7f0e23455700 1 --2- 172.21.15.92:0/386825302 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e1c1598a0 0x7f0e1c0ce7e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.461+0000 7f0e23455700 1 -- 172.21.15.92:0/386825302 >> 172.21.15.92:0/386825302 conn(0x7f0e1c089e50 msgr2=0x7f0e1c099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:01.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.461+0000 7f0e23455700 1 -- 172.21.15.92:0/386825302 shutdown_connections 2023-10-06T10:39:01.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.461+0000 7f0e23455700 1 -- 172.21.15.92:0/386825302 wait complete. 2023-10-06T10:39:01.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.462+0000 7f0e23455700 1 Processor -- start 2023-10-06T10:39:01.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.462+0000 7f0e23455700 1 -- start start 2023-10-06T10:39:01.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.463+0000 7f0e23455700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e1c0cbc70 0x7f0e1c16d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:01.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.463+0000 7f0e23455700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e1c1594d0 0x7f0e1c157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:01.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.463+0000 7f0e23455700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e1c1598a0 0x7f0e1c150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:01.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.463+0000 7f0e23455700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c1579d0 con 0x7f0e1c0cbc70 2023-10-06T10:39:01.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.463+0000 7f0e23455700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c16e290 con 0x7f0e1c1598a0 2023-10-06T10:39:01.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.463+0000 7f0e23455700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c16d9f0 con 0x7f0e1c1594d0 2023-10-06T10:39:01.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.463+0000 7f0e211f1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e1c0cbc70 0x7f0e1c16d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:01.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.463+0000 7f0e211f1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e1c0cbc70 0x7f0e1c16d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47308/0 (socket says 172.21.15.92:47308) 2023-10-06T10:39:01.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.463+0000 7f0e211f1700 1 -- 172.21.15.92:0/2995412866 learned_addr learned my addr 172.21.15.92:0/2995412866 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:01.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.464+0000 7f0e209f0700 1 --2- 172.21.15.92:0/2995412866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e1c1594d0 0x7f0e1c157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:01.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.464+0000 7f0e219f2700 1 --2- 172.21.15.92:0/2995412866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e1c1598a0 0x7f0e1c150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:01.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.464+0000 7f0e211f1700 1 -- 172.21.15.92:0/2995412866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e1c1594d0 msgr2=0x7f0e1c157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:01.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.464+0000 7f0e211f1700 1 --2- 172.21.15.92:0/2995412866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e1c1594d0 0x7f0e1c157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.464+0000 7f0e211f1700 1 -- 172.21.15.92:0/2995412866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e1c1598a0 msgr2=0x7f0e1c150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:01.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.464+0000 7f0e211f1700 1 --2- 172.21.15.92:0/2995412866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e1c1598a0 0x7f0e1c150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.464+0000 7f0e211f1700 1 -- 172.21.15.92:0/2995412866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c009420 con 0x7f0e1c0cbc70 2023-10-06T10:39:01.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.464+0000 7f0e219f2700 1 --2- 172.21.15.92:0/2995412866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e1c1598a0 0x7f0e1c150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:01.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.464+0000 7f0e211f1700 1 --2- 172.21.15.92:0/2995412866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e1c0cbc70 0x7f0e1c16d330 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800c7c0 tx=0x7f0e18007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:01.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.464+0000 7f0e117fa700 1 -- 172.21.15.92:0/2995412866 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e18005b30 con 0x7f0e1c0cbc70 2023-10-06T10:39:01.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.464+0000 7f0e117fa700 1 -- 172.21.15.92:0/2995412866 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e18005c90 con 0x7f0e1c0cbc70 2023-10-06T10:39:01.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.464+0000 7f0e23455700 1 -- 172.21.15.92:0/2995412866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1c16dcd0 con 0x7f0e1c0cbc70 2023-10-06T10:39:01.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.464+0000 7f0e117fa700 1 -- 172.21.15.92:0/2995412866 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e18007320 con 0x7f0e1c0cbc70 2023-10-06T10:39:01.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.464+0000 7f0e23455700 1 -- 172.21.15.92:0/2995412866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1c0c8f10 con 0x7f0e1c0cbc70 2023-10-06T10:39:01.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.465+0000 7f0e23455700 1 -- 172.21.15.92:0/2995412866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e1c064a90 con 0x7f0e1c0cbc70 2023-10-06T10:39:01.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.465+0000 7f0e117fa700 1 -- 172.21.15.92:0/2995412866 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0e18007690 con 0x7f0e1c0cbc70 2023-10-06T10:39:01.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.466+0000 7f0e117fa700 1 --2- 172.21.15.92:0/2995412866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0e00066f30 0x7f0e000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:01.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.466+0000 7f0e117fa700 1 -- 172.21.15.92:0/2995412866 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f0e18083df0 con 0x7f0e1c0cbc70 2023-10-06T10:39:01.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.466+0000 7f0e209f0700 1 --2- 172.21.15.92:0/2995412866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0e00066f30 0x7f0e000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:01.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.466+0000 7f0e209f0700 1 --2- 172.21.15.92:0/2995412866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0e00066f30 0x7f0e000693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c0098e0 tx=0x7f0e0c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:01.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.470+0000 7f0e117fa700 1 -- 172.21.15.92:0/2995412866 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e18050c10 con 0x7f0e1c0cbc70 2023-10-06T10:39:01.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.654+0000 7f0e23455700 1 -- 172.21.15.92:0/2995412866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0e1c0c66a0 con 0x7f0e1c0cbc70 2023-10-06T10:39:01.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.654+0000 7f0e117fa700 1 -- 172.21.15.92:0/2995412866 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0e18017020 con 0x7f0e1c0cbc70 2023-10-06T10:39:01.656 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T10:39:01.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.656+0000 7f0e23455700 1 -- 172.21.15.92:0/2995412866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0e00066f30 msgr2=0x7f0e000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:01.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.656+0000 7f0e23455700 1 --2- 172.21.15.92:0/2995412866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0e00066f30 0x7f0e000693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c0098e0 tx=0x7f0e0c017000 comp rx=0 tx=0).stop 2023-10-06T10:39:01.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.656+0000 7f0e23455700 1 -- 172.21.15.92:0/2995412866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e1c0cbc70 msgr2=0x7f0e1c16d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:01.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.656+0000 7f0e23455700 1 --2- 172.21.15.92:0/2995412866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e1c0cbc70 0x7f0e1c16d330 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800c7c0 tx=0x7f0e18007e00 comp rx=0 tx=0).stop 2023-10-06T10:39:01.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.656+0000 7f0e23455700 1 -- 172.21.15.92:0/2995412866 shutdown_connections 2023-10-06T10:39:01.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.656+0000 7f0e23455700 1 --2- 172.21.15.92:0/2995412866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0e00066f30 0x7f0e000693e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.656+0000 7f0e23455700 1 --2- 172.21.15.92:0/2995412866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e1c1598a0 0x7f0e1c150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.656+0000 7f0e23455700 1 --2- 172.21.15.92:0/2995412866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e1c0cbc70 0x7f0e1c16d330 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.656+0000 7f0e23455700 1 --2- 172.21.15.92:0/2995412866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e1c1594d0 0x7f0e1c157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.656+0000 7f0e23455700 1 -- 172.21.15.92:0/2995412866 >> 172.21.15.92:0/2995412866 conn(0x7f0e1c089e50 msgr2=0x7f0e1c099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:01.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.656+0000 7f0e23455700 1 -- 172.21.15.92:0/2995412866 shutdown_connections 2023-10-06T10:39:01.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.656+0000 7f0e23455700 1 -- 172.21.15.92:0/2995412866 wait complete. 2023-10-06T10:39:01.670 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:39:01.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.782+0000 7fe04d0bd700 1 Processor -- start 2023-10-06T10:39:01.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.782+0000 7fe04d0bd700 1 -- start start 2023-10-06T10:39:01.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.783+0000 7fe04d0bd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe048153e00 0x7fe04806df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:01.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.783+0000 7fe04d0bd700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0480c9cc0 con 0x7fe04806a960 2023-10-06T10:39:01.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.783+0000 7fe04d0bd700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0480c6710 con 0x7fe04806a590 2023-10-06T10:39:01.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.783+0000 7fe04d0bd700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04806f560 con 0x7fe048153e00 2023-10-06T10:39:01.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.783+0000 7fe04c8bc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe04806a960 0x7fe04806e490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50388/0 (socket says 172.21.15.92:50388) 2023-10-06T10:39:01.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.783+0000 7fe04c8bc700 1 -- 172.21.15.92:0/2424749133 learned_addr learned my addr 172.21.15.92:0/2424749133 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:01.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.783+0000 7fe0477fe700 1 --2- 172.21.15.92:0/2424749133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe048153e00 0x7fe04806df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:01.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.784+0000 7fe046ffd700 1 -- 172.21.15.92:0/2424749133 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007317910 0 0) 0x7fe0480c9cc0 con 0x7fe04806a960 2023-10-06T10:39:01.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.784+0000 7fe046ffd700 1 -- 172.21.15.92:0/2424749133 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe030003570 con 0x7fe04806a960 2023-10-06T10:39:01.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.784+0000 7fe046ffd700 1 -- 172.21.15.92:0/2424749133 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048719602 0 0) 0x7fe0480c6710 con 0x7fe04806a590 2023-10-06T10:39:01.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.784+0000 7fe046ffd700 1 -- 172.21.15.92:0/2424749133 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0480c9cc0 con 0x7fe04806a590 2023-10-06T10:39:01.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.784+0000 7fe0477fe700 1 -- 172.21.15.92:0/2424749133 >> v1:172.21.15.124:6789/0 conn(0x7fe04806a590 legacy=0x7fe0481536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:01.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.784+0000 7fe0477fe700 1 -- 172.21.15.92:0/2424749133 >> v1:172.21.15.92:6789/0 conn(0x7fe04806a960 legacy=0x7fe04806e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:01.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.784+0000 7fe046ffd700 1 -- 172.21.15.92:0/2424749133 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2059251628 0 0) 0x7fe030003570 con 0x7fe04806a960 2023-10-06T10:39:01.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.784+0000 7fe0477fe700 1 -- 172.21.15.92:0/2424749133 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04806ecc0 con 0x7fe048153e00 2023-10-06T10:39:01.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.784+0000 7fe0477fe700 1 --2- 172.21.15.92:0/2424749133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe048153e00 0x7fe04806df50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009940 tx=0x7fe03c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b31e4bb94784950b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:01.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.784+0000 7fe046ffd700 1 -- 172.21.15.92:0/2424749133 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0400017f0 con 0x7fe048153e00 2023-10-06T10:39:01.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.784+0000 7fe046ffd700 1 -- 172.21.15.92:0/2424749133 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03c011420 con 0x7fe048153e00 2023-10-06T10:39:01.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.784+0000 7fe046ffd700 1 -- 172.21.15.92:0/2424749133 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c0117c0 con 0x7fe048153e00 2023-10-06T10:39:01.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.785+0000 7fe04d0bd700 1 -- 172.21.15.92:0/2424749133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe048153e00 msgr2=0x7fe04806df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:01.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.785+0000 7fe04d0bd700 1 --2- 172.21.15.92:0/2424749133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe048153e00 0x7fe04806df50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009940 tx=0x7fe03c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.785+0000 7fe04d0bd700 1 -- 172.21.15.92:0/2424749133 shutdown_connections 2023-10-06T10:39:01.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.785+0000 7fe04d0bd700 1 --2- 172.21.15.92:0/2424749133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe048153e00 0x7fe04806df50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.785+0000 7fe04d0bd700 1 -- 172.21.15.92:0/2424749133 >> 172.21.15.92:0/2424749133 conn(0x7fe048068080 msgr2=0x7fe0480cbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:01.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.785+0000 7fe04d0bd700 1 -- 172.21.15.92:0/2424749133 shutdown_connections 2023-10-06T10:39:01.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.785+0000 7fe04d0bd700 1 -- 172.21.15.92:0/2424749133 wait complete. 2023-10-06T10:39:01.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.786+0000 7fe04d0bd700 1 Processor -- start 2023-10-06T10:39:01.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.786+0000 7fe04d0bd700 1 -- start start 2023-10-06T10:39:01.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.787+0000 7fe04d0bd700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe04806a590 0x7fe0480c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:01.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.787+0000 7fe04d0bd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe04806a960 0x7fe048158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:01.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.787+0000 7fe04d0bd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe048153e00 0x7fe048157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:01.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.787+0000 7fe04d0bd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe048158ac0 con 0x7fe04806a960 2023-10-06T10:39:01.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.787+0000 7fe04d0bd700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe048157eb0 con 0x7fe04806a590 2023-10-06T10:39:01.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.787+0000 7fe04d0bd700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe048158190 con 0x7fe048153e00 2023-10-06T10:39:01.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.787+0000 7fe0477fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe04806a960 0x7fe048158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:01.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.787+0000 7fe0477fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe04806a960 0x7fe048158f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47314/0 (socket says 172.21.15.92:47314) 2023-10-06T10:39:01.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.787+0000 7fe0477fe700 1 -- 172.21.15.92:0/1274220641 learned_addr learned my addr 172.21.15.92:0/1274220641 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:01.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.787+0000 7fe0477fe700 1 -- 172.21.15.92:0/1274220641 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe048153e00 msgr2=0x7fe048157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:01.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.787+0000 7fe0477fe700 1 --2- 172.21.15.92:0/1274220641 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe048153e00 0x7fe048157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.787+0000 7fe0477fe700 1 -- 172.21.15.92:0/1274220641 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe04806a590 msgr2=0x7fe0480c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:01.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.787+0000 7fe047fff700 1 --2- 172.21.15.92:0/1274220641 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe04806a590 0x7fe0480c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:01.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.787+0000 7fe0477fe700 1 --2- 172.21.15.92:0/1274220641 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe04806a590 0x7fe0480c57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:01.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.787+0000 7fe0477fe700 1 -- 172.21.15.92:0/1274220641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c009420 con 0x7fe04806a960 2023-10-06T10:39:01.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.787+0000 7fe0477fe700 1 --2- 172.21.15.92:0/1274220641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe04806a960 0x7fe048158f50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe04806c780 tx=0x7fe03c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:01.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.788+0000 7fe02ffff700 1 -- 172.21.15.92:0/1274220641 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c01ba30 con 0x7fe04806a960 2023-10-06T10:39:01.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.788+0000 7fe02ffff700 1 -- 172.21.15.92:0/1274220641 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03c020070 con 0x7fe04806a960 2023-10-06T10:39:01.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.788+0000 7fe02ffff700 1 -- 172.21.15.92:0/1274220641 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c01bde0 con 0x7fe04806a960 2023-10-06T10:39:01.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.788+0000 7fe04d0bd700 1 -- 172.21.15.92:0/1274220641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe048158470 con 0x7fe04806a960 2023-10-06T10:39:01.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.788+0000 7fe04d0bd700 1 -- 172.21.15.92:0/1274220641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe048159490 con 0x7fe04806a960 2023-10-06T10:39:01.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.788+0000 7fe04d0bd700 1 -- 172.21.15.92:0/1274220641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe048158c30 con 0x7fe04806a960 2023-10-06T10:39:01.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.789+0000 7fe02ffff700 1 -- 172.21.15.92:0/1274220641 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe03c022030 con 0x7fe04806a960 2023-10-06T10:39:01.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.789+0000 7fe02ffff700 1 --2- 172.21.15.92:0/1274220641 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe034066f60 0x7fe034069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:01.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.789+0000 7fe02ffff700 1 -- 172.21.15.92:0/1274220641 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe03c085f80 con 0x7fe04806a960 2023-10-06T10:39:01.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.789+0000 7fe047fff700 1 --2- 172.21.15.92:0/1274220641 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe034066f60 0x7fe034069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:01.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.790+0000 7fe047fff700 1 --2- 172.21.15.92:0/1274220641 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe034066f60 0x7fe034069410 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe038002730 tx=0x7fe038009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:01.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:01.794+0000 7fe02ffff700 1 -- 172.21.15.92:0/1274220641 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe03c0b75a0 con 0x7fe04806a960 2023-10-06T10:39:02.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.017+0000 7fe04d0bd700 1 -- 172.21.15.92:0/1274220641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe04805f230 con 0x7fe04806a960 2023-10-06T10:39:02.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.017+0000 7fe02ffff700 1 -- 172.21.15.92:0/1274220641 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4599 (secure 0 0 0) 0x7fe03c052da0 con 0x7fe04806a960 2023-10-06T10:39:02.019 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:39:02.019 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2303424485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2303424485},{"type":"v1","addr":"172.21.15.92:6836","nonce":2303424485}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1027343065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1027343065},{"type":"v1","addr":"172.21.15.92:6837","nonce":1027343065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2667820379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2667820379},{"type":"v1","addr":"172.21.15.124:6834","nonce":2667820379}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:38:16.725144+0000","modified":"2023-10-06T10:38:30.398242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/1919704285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":1919704285},{"type":"v1","addr":"172.21.15.124:6835","nonce":1919704285}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T10:39:02.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.019+0000 7fe04d0bd700 1 -- 172.21.15.92:0/1274220641 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe034066f60 msgr2=0x7fe034069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:02.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.019+0000 7fe04d0bd700 1 --2- 172.21.15.92:0/1274220641 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe034066f60 0x7fe034069410 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe038002730 tx=0x7fe038009040 comp rx=0 tx=0).stop 2023-10-06T10:39:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.019+0000 7fe04d0bd700 1 -- 172.21.15.92:0/1274220641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe04806a960 msgr2=0x7fe048158f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.019+0000 7fe04d0bd700 1 --2- 172.21.15.92:0/1274220641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe04806a960 0x7fe048158f50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe04806c780 tx=0x7fe03c016000 comp rx=0 tx=0).stop 2023-10-06T10:39:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.019+0000 7fe04d0bd700 1 -- 172.21.15.92:0/1274220641 shutdown_connections 2023-10-06T10:39:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.019+0000 7fe04d0bd700 1 --2- 172.21.15.92:0/1274220641 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe034066f60 0x7fe034069410 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.019+0000 7fe04d0bd700 1 --2- 172.21.15.92:0/1274220641 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe048153e00 0x7fe048157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.019+0000 7fe04d0bd700 1 --2- 172.21.15.92:0/1274220641 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe04806a590 0x7fe0480c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.019+0000 7fe04d0bd700 1 --2- 172.21.15.92:0/1274220641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe04806a960 0x7fe048158f50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.019+0000 7fe04d0bd700 1 -- 172.21.15.92:0/1274220641 >> 172.21.15.92:0/1274220641 conn(0x7fe048068080 msgr2=0x7fe048149da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.020+0000 7fe04d0bd700 1 -- 172.21.15.92:0/1274220641 shutdown_connections 2023-10-06T10:39:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.020+0000 7fe04d0bd700 1 -- 172.21.15.92:0/1274220641 wait complete. 2023-10-06T10:39:02.023 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-10-06T10:39:02.033 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:39:02.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.146+0000 7f55d1cf3700 1 Processor -- start 2023-10-06T10:39:02.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.147+0000 7f55d1cf3700 1 -- start start 2023-10-06T10:39:02.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.147+0000 7f55d1cf3700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f55cc0ad320 0x7f55cc0ac230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:02.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.147+0000 7f55d1cf3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f55cc068cd0 0x7f55cc15b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:02.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.147+0000 7f55d1cf3700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55cc159f40 con 0x7f55cc0ad6f0 2023-10-06T10:39:02.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.147+0000 7f55d1cf3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55cc15a220 con 0x7f55cc068cd0 2023-10-06T10:39:02.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.147+0000 7f55d1cf3700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55cc15a500 con 0x7f55cc0ad320 2023-10-06T10:39:02.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.148+0000 7f55cbfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f55cc068cd0 0x7f55cc15b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:02.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.148+0000 7f55cbfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f55cc068cd0 0x7f55cc15b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47320/0 (socket says 172.21.15.92:47320) 2023-10-06T10:39:02.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.148+0000 7f55cbfff700 1 -- 172.21.15.92:0/2688800470 learned_addr learned my addr 172.21.15.92:0/2688800470 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:02.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.148+0000 7f55cb7fe700 1 --2- 172.21.15.92:0/2688800470 >> v2:172.21.15.124:3300/0 conn(0x7f55cc0ad320 0x7f55cc0ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:02.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.148+0000 7f55cbfff700 1 -- 172.21.15.92:0/2688800470 >> v2:172.21.15.124:3300/0 conn(0x7f55cc0ad320 msgr2=0x7f55cc0ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:02.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.148+0000 7f55cbfff700 1 --2- 172.21.15.92:0/2688800470 >> v2:172.21.15.124:3300/0 conn(0x7f55cc0ad320 0x7f55cc0ac230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.148+0000 7f55cbfff700 1 -- 172.21.15.92:0/2688800470 >> v1:172.21.15.124:6789/0 conn(0x7f55cc0ad6f0 legacy=0x7f55cc0685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:02.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.148+0000 7f55cbfff700 1 -- 172.21.15.92:0/2688800470 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc15a7e0 con 0x7f55cc068cd0 2023-10-06T10:39:02.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.148+0000 7f55cbfff700 1 --2- 172.21.15.92:0/2688800470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55cc068cd0 0x7f55cc15b050 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f55cc063db0 tx=0x7f55c4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf92550c41d4cccf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:02.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.148+0000 7f55caffd700 1 -- 172.21.15.92:0/2688800470 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c400e070 con 0x7f55cc068cd0 2023-10-06T10:39:02.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.148+0000 7f55caffd700 1 -- 172.21.15.92:0/2688800470 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c400ba20 con 0x7f55cc068cd0 2023-10-06T10:39:02.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.148+0000 7f55caffd700 1 -- 172.21.15.92:0/2688800470 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c40092e0 con 0x7f55cc068cd0 2023-10-06T10:39:02.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.149+0000 7f55d1cf3700 1 -- 172.21.15.92:0/2688800470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55cc068cd0 msgr2=0x7f55cc15b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:02.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.149+0000 7f55d1cf3700 1 --2- 172.21.15.92:0/2688800470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55cc068cd0 0x7f55cc15b050 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f55cc063db0 tx=0x7f55c4009840 comp rx=0 tx=0).stop 2023-10-06T10:39:02.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.149+0000 7f55d1cf3700 1 -- 172.21.15.92:0/2688800470 shutdown_connections 2023-10-06T10:39:02.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.149+0000 7f55d1cf3700 1 --2- 172.21.15.92:0/2688800470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55cc068cd0 0x7f55cc15b050 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.149+0000 7f55d1cf3700 1 --2- 172.21.15.92:0/2688800470 >> v2:172.21.15.124:3300/0 conn(0x7f55cc0ad320 0x7f55cc0ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.149+0000 7f55d1cf3700 1 -- 172.21.15.92:0/2688800470 >> 172.21.15.92:0/2688800470 conn(0x7f55cc06dcd0 msgr2=0x7f55cc06e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:02.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.149+0000 7f55d1cf3700 1 -- 172.21.15.92:0/2688800470 shutdown_connections 2023-10-06T10:39:02.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.149+0000 7f55d1cf3700 1 -- 172.21.15.92:0/2688800470 wait complete. 2023-10-06T10:39:02.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.150+0000 7f55d1cf3700 1 Processor -- start 2023-10-06T10:39:02.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.150+0000 7f55d1cf3700 1 -- start start 2023-10-06T10:39:02.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.151+0000 7f55d1cf3700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55cc068cd0 0x7f55cc14e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:02.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.151+0000 7f55d1cf3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55cc0ad320 0x7f55cc0aac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:02.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.151+0000 7f55d1cf3700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55cc0ad6f0 0x7f55cc0ab160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:02.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.151+0000 7f55d1cf3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc14ec10 con 0x7f55cc0ad320 2023-10-06T10:39:02.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.151+0000 7f55d1cf3700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc16b910 con 0x7f55cc068cd0 2023-10-06T10:39:02.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.151+0000 7f55d1cf3700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc0c1710 con 0x7f55cc0ad6f0 2023-10-06T10:39:02.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.151+0000 7f55c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55cc0ad320 0x7f55cc0aac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:02.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.151+0000 7f55c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55cc0ad320 0x7f55cc0aac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47328/0 (socket says 172.21.15.92:47328) 2023-10-06T10:39:02.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.151+0000 7f55c3fff700 1 -- 172.21.15.92:0/3489814526 learned_addr learned my addr 172.21.15.92:0/3489814526 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:02.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.151+0000 7f55c3fff700 1 -- 172.21.15.92:0/3489814526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55cc0ad6f0 msgr2=0x7f55cc0ab160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:02.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.151+0000 7f55c3fff700 1 --2- 172.21.15.92:0/3489814526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55cc0ad6f0 0x7f55cc0ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.151+0000 7f55c3fff700 1 -- 172.21.15.92:0/3489814526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55cc068cd0 msgr2=0x7f55cc14e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:02.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.151+0000 7f55cb7fe700 1 --2- 172.21.15.92:0/3489814526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55cc068cd0 0x7f55cc14e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:02.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.151+0000 7f55c3fff700 1 --2- 172.21.15.92:0/3489814526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55cc068cd0 0x7f55cc14e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.151+0000 7f55c3fff700 1 -- 172.21.15.92:0/3489814526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c4009ae0 con 0x7f55cc0ad320 2023-10-06T10:39:02.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.151+0000 7f55cb7fe700 1 --2- 172.21.15.92:0/3489814526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55cc068cd0 0x7f55cc14e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:02.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.151+0000 7f55c3fff700 1 --2- 172.21.15.92:0/3489814526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55cc0ad320 0x7f55cc0aac20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f55b400c4e0 tx=0x7f55b400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:02.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.151+0000 7f55c37fe700 1 -- 172.21.15.92:0/3489814526 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b4011590 con 0x7f55cc0ad320 2023-10-06T10:39:02.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.151+0000 7f55c37fe700 1 -- 172.21.15.92:0/3489814526 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b4011710 con 0x7f55cc0ad320 2023-10-06T10:39:02.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.152+0000 7f55c37fe700 1 -- 172.21.15.92:0/3489814526 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b4011ad0 con 0x7f55cc0ad320 2023-10-06T10:39:02.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.152+0000 7f55d1cf3700 1 -- 172.21.15.92:0/3489814526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55cc16ac00 con 0x7f55cc0ad320 2023-10-06T10:39:02.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.152+0000 7f55d1cf3700 1 -- 172.21.15.92:0/3489814526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55cc16b2c0 con 0x7f55cc0ad320 2023-10-06T10:39:02.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.152+0000 7f55d1cf3700 1 -- 172.21.15.92:0/3489814526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55cc16ba80 con 0x7f55cc0ad320 2023-10-06T10:39:02.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.153+0000 7f55c37fe700 1 -- 172.21.15.92:0/3489814526 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f55b401b3b0 con 0x7f55cc0ad320 2023-10-06T10:39:02.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.153+0000 7f55c37fe700 1 --2- 172.21.15.92:0/3489814526 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f55ac066f30 0x7f55ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:02.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.153+0000 7f55c37fe700 1 -- 172.21.15.92:0/3489814526 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f55b4083eb0 con 0x7f55cc0ad320 2023-10-06T10:39:02.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.153+0000 7f55cb7fe700 1 --2- 172.21.15.92:0/3489814526 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f55ac066f30 0x7f55ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:02.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.154+0000 7f55cb7fe700 1 --2- 172.21.15.92:0/3489814526 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f55ac066f30 0x7f55ac0693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f55bc006f00 tx=0x7f55bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:02.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.157+0000 7f55c37fe700 1 -- 172.21.15.92:0/3489814526 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55b4050cd0 con 0x7f55cc0ad320 2023-10-06T10:39:02.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.341+0000 7f55d1cf3700 1 -- 172.21.15.92:0/3489814526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55cc064a90 con 0x7f55cc0ad320 2023-10-06T10:39:02.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.341+0000 7f55c37fe700 1 -- 172.21.15.92:0/3489814526 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18707 (secure 0 0 0) 0x7f55b4017070 con 0x7f55cc0ad320 2023-10-06T10:39:02.344 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:39:02.344 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:21.760233+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:38:14.368713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T10:38:15.692344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T10:38:19.382553+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:39:02.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.343+0000 7f55d1cf3700 1 -- 172.21.15.92:0/3489814526 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f55ac066f30 msgr2=0x7f55ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.343+0000 7f55d1cf3700 1 --2- 172.21.15.92:0/3489814526 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f55ac066f30 0x7f55ac0693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f55bc006f00 tx=0x7f55bc008040 comp rx=0 tx=0).stop 2023-10-06T10:39:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.343+0000 7f55d1cf3700 1 -- 172.21.15.92:0/3489814526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55cc0ad320 msgr2=0x7f55cc0aac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.343+0000 7f55d1cf3700 1 --2- 172.21.15.92:0/3489814526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55cc0ad320 0x7f55cc0aac20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f55b400c4e0 tx=0x7f55b400c8a0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.343+0000 7f55d1cf3700 1 -- 172.21.15.92:0/3489814526 shutdown_connections 2023-10-06T10:39:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.343+0000 7f55d1cf3700 1 --2- 172.21.15.92:0/3489814526 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f55ac066f30 0x7f55ac0693e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.343+0000 7f55d1cf3700 1 --2- 172.21.15.92:0/3489814526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55cc0ad6f0 0x7f55cc0ab160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.343+0000 7f55d1cf3700 1 --2- 172.21.15.92:0/3489814526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55cc068cd0 0x7f55cc14e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.343+0000 7f55d1cf3700 1 --2- 172.21.15.92:0/3489814526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55cc0ad320 0x7f55cc0aac20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.344+0000 7f55d1cf3700 1 -- 172.21.15.92:0/3489814526 >> 172.21.15.92:0/3489814526 conn(0x7f55cc06dcd0 msgr2=0x7f55cc0aa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.344+0000 7f55d1cf3700 1 -- 172.21.15.92:0/3489814526 shutdown_connections 2023-10-06T10:39:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.344+0000 7f55d1cf3700 1 -- 172.21.15.92:0/3489814526 wait complete. 2023-10-06T10:39:02.358 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T10:39:02.358 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:39:02.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.470+0000 7f786bd9f700 1 Processor -- start 2023-10-06T10:39:02.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.470+0000 7f786bd9f700 1 -- start start 2023-10-06T10:39:02.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.471+0000 7f786bd9f700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7864157600 0x7f786406e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:02.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.471+0000 7f786bd9f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78640ca940 0x7f78640b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:02.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.471+0000 7f786bd9f700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786406bcc0 con 0x7f78640675b0 2023-10-06T10:39:02.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.471+0000 7f786bd9f700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78640b06e0 con 0x7f7864157600 2023-10-06T10:39:02.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.471+0000 7f786bd9f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78640ca6f0 con 0x7f78640ca940 2023-10-06T10:39:02.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.472+0000 7f7869b3b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7864157600 0x7f786406e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:02.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.472+0000 7f786933a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78640ca940 0x7f78640b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:02.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.472+0000 7f786a33c700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f78640675b0 0x7f786406c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:36418/0 (socket says 172.21.15.92:36418) 2023-10-06T10:39:02.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.472+0000 7f786a33c700 1 -- 172.21.15.92:0/603597341 learned_addr learned my addr 172.21.15.92:0/603597341 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:02.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.472+0000 7f7868b39700 1 -- 172.21.15.92:0/603597341 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 434982467 0 0) 0x7f786406bcc0 con 0x7f78640675b0 2023-10-06T10:39:02.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.472+0000 7f7868b39700 1 -- 172.21.15.92:0/603597341 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f784c003570 con 0x7f78640675b0 2023-10-06T10:39:02.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.472+0000 7f786933a700 1 -- 172.21.15.92:0/603597341 >> v2:172.21.15.124:3300/0 conn(0x7f7864157600 msgr2=0x7f786406e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:02.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.472+0000 7f786933a700 1 --2- 172.21.15.92:0/603597341 >> v2:172.21.15.124:3300/0 conn(0x7f7864157600 0x7f786406e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.472+0000 7f786933a700 1 -- 172.21.15.92:0/603597341 >> v1:172.21.15.124:6789/0 conn(0x7f78640675b0 legacy=0x7f786406c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:02.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.472+0000 7f786933a700 1 -- 172.21.15.92:0/603597341 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786406ecd0 con 0x7f78640ca940 2023-10-06T10:39:02.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.472+0000 7f7869b3b700 1 --2- 172.21.15.92:0/603597341 >> v2:172.21.15.124:3300/0 conn(0x7f7864157600 0x7f786406e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:39:02.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.472+0000 7f786933a700 1 --2- 172.21.15.92:0/603597341 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78640ca940 0x7f78640b01a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f78580098e0 tx=0x7f78580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59c6f8f7c4ec4260 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:02.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.472+0000 7f7868b39700 1 -- 172.21.15.92:0/603597341 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858011630 con 0x7f78640ca940 2023-10-06T10:39:02.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.472+0000 7f7868b39700 1 -- 172.21.15.92:0/603597341 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7858011790 con 0x7f78640ca940 2023-10-06T10:39:02.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.472+0000 7f7868b39700 1 -- 172.21.15.92:0/603597341 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858011b10 con 0x7f78640ca940 2023-10-06T10:39:02.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.473+0000 7f786bd9f700 1 -- 172.21.15.92:0/603597341 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78640ca940 msgr2=0x7f78640b01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:02.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.473+0000 7f786bd9f700 1 --2- 172.21.15.92:0/603597341 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78640ca940 0x7f78640b01a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f78580098e0 tx=0x7f78580091a0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.473+0000 7f786bd9f700 1 -- 172.21.15.92:0/603597341 shutdown_connections 2023-10-06T10:39:02.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.473+0000 7f786bd9f700 1 --2- 172.21.15.92:0/603597341 >> v2:172.21.15.124:3300/0 conn(0x7f7864157600 0x7f786406e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.473+0000 7f786bd9f700 1 --2- 172.21.15.92:0/603597341 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78640ca940 0x7f78640b01a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.473+0000 7f786bd9f700 1 -- 172.21.15.92:0/603597341 >> 172.21.15.92:0/603597341 conn(0x7f78640cdeb0 msgr2=0x7f78640c9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:02.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.473+0000 7f786bd9f700 1 -- 172.21.15.92:0/603597341 shutdown_connections 2023-10-06T10:39:02.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.473+0000 7f786bd9f700 1 -- 172.21.15.92:0/603597341 wait complete. 2023-10-06T10:39:02.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.474+0000 7f786bd9f700 1 Processor -- start 2023-10-06T10:39:02.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.474+0000 7f786bd9f700 1 -- start start 2023-10-06T10:39:02.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.475+0000 7f786bd9f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78640675b0 0x7f786416a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:02.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.475+0000 7f786bd9f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78640ca940 0x7f78640cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:02.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.475+0000 7f786bd9f700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7864157600 0x7f78640a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:02.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.475+0000 7f786bd9f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78640cca50 con 0x7f78640675b0 2023-10-06T10:39:02.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.475+0000 7f786bd9f700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786416a430 con 0x7f7864157600 2023-10-06T10:39:02.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.475+0000 7f786bd9f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78640aa210 con 0x7f78640ca940 2023-10-06T10:39:02.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.475+0000 7f7869b3b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78640675b0 0x7f786416a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:02.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.475+0000 7f7869b3b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78640675b0 0x7f786416a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47342/0 (socket says 172.21.15.92:47342) 2023-10-06T10:39:02.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.475+0000 7f7869b3b700 1 -- 172.21.15.92:0/2170487054 learned_addr learned my addr 172.21.15.92:0/2170487054 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:02.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.475+0000 7f786933a700 1 --2- 172.21.15.92:0/2170487054 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78640ca940 0x7f78640cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:02.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.475+0000 7f786a33c700 1 --2- 172.21.15.92:0/2170487054 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7864157600 0x7f78640a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:02.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.475+0000 7f7869b3b700 1 -- 172.21.15.92:0/2170487054 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78640ca940 msgr2=0x7f78640cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:02.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.475+0000 7f7869b3b700 1 --2- 172.21.15.92:0/2170487054 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78640ca940 0x7f78640cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.475+0000 7f7869b3b700 1 -- 172.21.15.92:0/2170487054 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7864157600 msgr2=0x7f78640a98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:02.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.475+0000 7f7869b3b700 1 --2- 172.21.15.92:0/2170487054 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7864157600 0x7f78640a98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.475+0000 7f7869b3b700 1 -- 172.21.15.92:0/2170487054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785c009a20 con 0x7f78640675b0 2023-10-06T10:39:02.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.475+0000 7f7869b3b700 1 --2- 172.21.15.92:0/2170487054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78640675b0 0x7f786416a8c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f785c00c170 tx=0x7f785c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:02.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.476+0000 7f7861ffb700 1 -- 172.21.15.92:0/2170487054 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c00ccb0 con 0x7f78640675b0 2023-10-06T10:39:02.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.476+0000 7f7861ffb700 1 -- 172.21.15.92:0/2170487054 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785c00f040 con 0x7f78640675b0 2023-10-06T10:39:02.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.476+0000 7f786bd9f700 1 -- 172.21.15.92:0/2170487054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7858009420 con 0x7f78640675b0 2023-10-06T10:39:02.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.476+0000 7f7861ffb700 1 -- 172.21.15.92:0/2170487054 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c014480 con 0x7f78640675b0 2023-10-06T10:39:02.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.476+0000 7f786bd9f700 1 -- 172.21.15.92:0/2170487054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786416a0c0 con 0x7f78640675b0 2023-10-06T10:39:02.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.477+0000 7f786bd9f700 1 -- 172.21.15.92:0/2170487054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786416a710 con 0x7f78640675b0 2023-10-06T10:39:02.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.477+0000 7f7861ffb700 1 -- 172.21.15.92:0/2170487054 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f785c0147f0 con 0x7f78640675b0 2023-10-06T10:39:02.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.478+0000 7f7861ffb700 1 --2- 172.21.15.92:0/2170487054 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7850066f30 0x7f78500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:02.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.478+0000 7f7861ffb700 1 -- 172.21.15.92:0/2170487054 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f785c085700 con 0x7f78640675b0 2023-10-06T10:39:02.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.478+0000 7f786933a700 1 --2- 172.21.15.92:0/2170487054 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7850066f30 0x7f78500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:02.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.478+0000 7f786933a700 1 --2- 172.21.15.92:0/2170487054 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7850066f30 0x7f78500693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f78580091e0 tx=0x7f7858017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:02.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.482+0000 7f7861ffb700 1 -- 172.21.15.92:0/2170487054 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f785c052520 con 0x7f78640675b0 2023-10-06T10:39:02.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.711+0000 7f786bd9f700 1 -- 172.21.15.92:0/2170487054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7864064a90 con 0x7f78640675b0 2023-10-06T10:39:02.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.712+0000 7f7861ffb700 1 -- 172.21.15.92:0/2170487054 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4599 (secure 0 0 0) 0x7f785c05b030 con 0x7f78640675b0 2023-10-06T10:39:02.714 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:39:02.714 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2303424485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2303424485},{"type":"v1","addr":"172.21.15.92:6836","nonce":2303424485}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1027343065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1027343065},{"type":"v1","addr":"172.21.15.92:6837","nonce":1027343065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2667820379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2667820379},{"type":"v1","addr":"172.21.15.124:6834","nonce":2667820379}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:38:16.725144+0000","modified":"2023-10-06T10:38:30.398242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/1919704285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":1919704285},{"type":"v1","addr":"172.21.15.124:6835","nonce":1919704285}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T10:39:02.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.714+0000 7f786bd9f700 1 -- 172.21.15.92:0/2170487054 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7850066f30 msgr2=0x7f78500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:02.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.714+0000 7f786bd9f700 1 --2- 172.21.15.92:0/2170487054 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7850066f30 0x7f78500693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f78580091e0 tx=0x7f7858017000 comp rx=0 tx=0).stop 2023-10-06T10:39:02.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.714+0000 7f786bd9f700 1 -- 172.21.15.92:0/2170487054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78640675b0 msgr2=0x7f786416a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:02.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.714+0000 7f786bd9f700 1 --2- 172.21.15.92:0/2170487054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78640675b0 0x7f786416a8c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f785c00c170 tx=0x7f785c00b040 comp rx=0 tx=0).stop 2023-10-06T10:39:02.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.714+0000 7f786bd9f700 1 -- 172.21.15.92:0/2170487054 shutdown_connections 2023-10-06T10:39:02.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.714+0000 7f786bd9f700 1 --2- 172.21.15.92:0/2170487054 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7850066f30 0x7f78500693e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.714+0000 7f786bd9f700 1 --2- 172.21.15.92:0/2170487054 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7864157600 0x7f78640a98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.714+0000 7f786bd9f700 1 --2- 172.21.15.92:0/2170487054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78640675b0 0x7f786416a8c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.714+0000 7f786bd9f700 1 --2- 172.21.15.92:0/2170487054 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78640ca940 0x7f78640cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.714+0000 7f786bd9f700 1 -- 172.21.15.92:0/2170487054 >> 172.21.15.92:0/2170487054 conn(0x7f78640cdeb0 msgr2=0x7f7864148e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:02.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.714+0000 7f786bd9f700 1 -- 172.21.15.92:0/2170487054 shutdown_connections 2023-10-06T10:39:02.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.714+0000 7f786bd9f700 1 -- 172.21.15.92:0/2170487054 wait complete. 2023-10-06T10:39:02.717 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-10-06T10:39:02.728 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:39:02.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.840+0000 7f3813fff700 1 Processor -- start 2023-10-06T10:39:02.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.840+0000 7f3813fff700 1 -- start start 2023-10-06T10:39:02.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.841+0000 7f3813fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3814159440 0x7f38140b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:02.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.841+0000 7f3813fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140b2100 0x7f38140b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.841+0000 7f3813fff700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38140cb7a0 con 0x7f38140b35d0 2023-10-06T10:39:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.841+0000 7f3813fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38140b0860 con 0x7f3814159440 2023-10-06T10:39:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.841+0000 7f3813fff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38140cb150 con 0x7f38140b2100 2023-10-06T10:39:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.841+0000 7f3812ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3814159440 0x7f38140b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.841+0000 7f3812ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3814159440 0x7f38140b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47354/0 (socket says 172.21.15.92:47354) 2023-10-06T10:39:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.841+0000 7f3812ffd700 1 -- 172.21.15.92:0/2028826093 learned_addr learned my addr 172.21.15.92:0/2028826093 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.841+0000 7f38137fe700 1 --2- 172.21.15.92:0/2028826093 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140b2100 0x7f38140b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.841+0000 7f3812ffd700 1 -- 172.21.15.92:0/2028826093 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140b2100 msgr2=0x7f38140b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.841+0000 7f3812ffd700 1 --2- 172.21.15.92:0/2028826093 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140b2100 0x7f38140b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.841+0000 7f3812ffd700 1 -- 172.21.15.92:0/2028826093 >> v1:172.21.15.124:6789/0 conn(0x7f38140b35d0 legacy=0x7f38140cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.841+0000 7f3812ffd700 1 -- 172.21.15.92:0/2028826093 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38140cb430 con 0x7f3814159440 2023-10-06T10:39:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.842+0000 7f3812ffd700 1 --2- 172.21.15.92:0/2028826093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3814159440 0x7f38140b3090 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f38040099b0 tx=0x7f3804009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bb356e22761534 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:02.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.842+0000 7f38127fc700 1 -- 172.21.15.92:0/2028826093 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380400b860 con 0x7f3814159440 2023-10-06T10:39:02.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.842+0000 7f38127fc700 1 -- 172.21.15.92:0/2028826093 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380400b9c0 con 0x7f3814159440 2023-10-06T10:39:02.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.842+0000 7f38127fc700 1 -- 172.21.15.92:0/2028826093 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380400bd60 con 0x7f3814159440 2023-10-06T10:39:02.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.842+0000 7f3813fff700 1 -- 172.21.15.92:0/2028826093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3814159440 msgr2=0x7f38140b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:02.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.842+0000 7f3813fff700 1 --2- 172.21.15.92:0/2028826093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3814159440 0x7f38140b3090 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f38040099b0 tx=0x7f3804009250 comp rx=0 tx=0).stop 2023-10-06T10:39:02.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.842+0000 7f3813fff700 1 -- 172.21.15.92:0/2028826093 shutdown_connections 2023-10-06T10:39:02.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.842+0000 7f3813fff700 1 --2- 172.21.15.92:0/2028826093 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140b2100 0x7f38140b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.842+0000 7f3813fff700 1 --2- 172.21.15.92:0/2028826093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3814159440 0x7f38140b3090 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.842+0000 7f3813fff700 1 -- 172.21.15.92:0/2028826093 >> 172.21.15.92:0/2028826093 conn(0x7f381406ece0 msgr2=0x7f381406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:02.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.842+0000 7f3813fff700 1 -- 172.21.15.92:0/2028826093 shutdown_connections 2023-10-06T10:39:02.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.842+0000 7f3813fff700 1 -- 172.21.15.92:0/2028826093 wait complete. 2023-10-06T10:39:02.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.843+0000 7f3813fff700 1 Processor -- start 2023-10-06T10:39:02.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.844+0000 7f3813fff700 1 -- start start 2023-10-06T10:39:02.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.844+0000 7f3813fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140b2100 0x7f3814154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:02.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.844+0000 7f3813fff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38140b35d0 0x7f381415af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:02.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.844+0000 7f3813fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3814159440 0x7f381415b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:02.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.844+0000 7f3813fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3814159b40 con 0x7f3814159440 2023-10-06T10:39:02.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.844+0000 7f3813fff700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38140c8610 con 0x7f38140b35d0 2023-10-06T10:39:02.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.844+0000 7f3813fff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381415bb20 con 0x7f38140b2100 2023-10-06T10:39:02.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.844+0000 7f38137fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3814159440 0x7f381415b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:02.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.844+0000 7f38137fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3814159440 0x7f381415b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47360/0 (socket says 172.21.15.92:47360) 2023-10-06T10:39:02.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.844+0000 7f38137fe700 1 -- 172.21.15.92:0/2888098401 learned_addr learned my addr 172.21.15.92:0/2888098401 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:02.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.844+0000 7f3812ffd700 1 --2- 172.21.15.92:0/2888098401 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140b2100 0x7f3814154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:02.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.844+0000 7f380a7fc700 1 --2- 172.21.15.92:0/2888098401 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38140b35d0 0x7f381415af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:02.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.845+0000 7f38137fe700 1 -- 172.21.15.92:0/2888098401 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140b2100 msgr2=0x7f3814154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:02.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.845+0000 7f38137fe700 1 --2- 172.21.15.92:0/2888098401 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140b2100 0x7f3814154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.845+0000 7f38137fe700 1 -- 172.21.15.92:0/2888098401 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38140b35d0 msgr2=0x7f381415af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:02.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.845+0000 7f38137fe700 1 --2- 172.21.15.92:0/2888098401 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38140b35d0 0x7f381415af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:02.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.845+0000 7f38137fe700 1 -- 172.21.15.92:0/2888098401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38040094f0 con 0x7f3814159440 2023-10-06T10:39:02.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.845+0000 7f38137fe700 1 --2- 172.21.15.92:0/2888098401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3814159440 0x7f381415b480 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f380c009fa0 tx=0x7f380c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:02.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.845+0000 7f380b7fe700 1 -- 172.21.15.92:0/2888098401 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c00ebc0 con 0x7f3814159440 2023-10-06T10:39:02.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.845+0000 7f380b7fe700 1 -- 172.21.15.92:0/2888098401 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380c00ed20 con 0x7f3814159440 2023-10-06T10:39:02.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.845+0000 7f3813fff700 1 -- 172.21.15.92:0/2888098401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381415be00 con 0x7f3814159440 2023-10-06T10:39:02.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.845+0000 7f380b7fe700 1 -- 172.21.15.92:0/2888098401 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c014670 con 0x7f3814159440 2023-10-06T10:39:02.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.845+0000 7f3813fff700 1 -- 172.21.15.92:0/2888098401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381415c190 con 0x7f3814159440 2023-10-06T10:39:02.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.846+0000 7f3813fff700 1 -- 172.21.15.92:0/2888098401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38140c8450 con 0x7f3814159440 2023-10-06T10:39:02.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.846+0000 7f380b7fe700 1 -- 172.21.15.92:0/2888098401 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f380c0149e0 con 0x7f3814159440 2023-10-06T10:39:02.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.847+0000 7f380b7fe700 1 --2- 172.21.15.92:0/2888098401 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f37f4066f30 0x7f37f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:02.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.847+0000 7f380b7fe700 1 -- 172.21.15.92:0/2888098401 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f380c085010 con 0x7f3814159440 2023-10-06T10:39:02.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.847+0000 7f3812ffd700 1 --2- 172.21.15.92:0/2888098401 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f37f4066f30 0x7f37f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:02.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.847+0000 7f3812ffd700 1 --2- 172.21.15.92:0/2888098401 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f37f4066f30 0x7f37f40693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f38040099b0 tx=0x7f3804017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:02.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:02.851+0000 7f380b7fe700 1 -- 172.21.15.92:0/2888098401 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38140c8450 con 0x7f3814159440 2023-10-06T10:39:03.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.076+0000 7f3813fff700 1 -- 172.21.15.92:0/2888098401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38140c8450 con 0x7f3814159440 2023-10-06T10:39:03.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.077+0000 7f380b7fe700 1 -- 172.21.15.92:0/2888098401 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4599 (secure 0 0 0) 0x7f38140c8450 con 0x7f3814159440 2023-10-06T10:39:03.079 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:39:03.079 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2303424485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2303424485},{"type":"v1","addr":"172.21.15.92:6836","nonce":2303424485}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1027343065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1027343065},{"type":"v1","addr":"172.21.15.92:6837","nonce":1027343065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2667820379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2667820379},{"type":"v1","addr":"172.21.15.124:6834","nonce":2667820379}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:38:16.725144+0000","modified":"2023-10-06T10:38:30.398242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/1919704285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":1919704285},{"type":"v1","addr":"172.21.15.124:6835","nonce":1919704285}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T10:39:03.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.079+0000 7f3813fff700 1 -- 172.21.15.92:0/2888098401 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f37f4066f30 msgr2=0x7f37f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:03.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.079+0000 7f3813fff700 1 --2- 172.21.15.92:0/2888098401 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f37f4066f30 0x7f37f40693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f38040099b0 tx=0x7f3804017000 comp rx=0 tx=0).stop 2023-10-06T10:39:03.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.079+0000 7f3813fff700 1 -- 172.21.15.92:0/2888098401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3814159440 msgr2=0x7f381415b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:03.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.079+0000 7f3813fff700 1 --2- 172.21.15.92:0/2888098401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3814159440 0x7f381415b480 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f380c009fa0 tx=0x7f380c009900 comp rx=0 tx=0).stop 2023-10-06T10:39:03.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.079+0000 7f3813fff700 1 -- 172.21.15.92:0/2888098401 shutdown_connections 2023-10-06T10:39:03.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.079+0000 7f3813fff700 1 --2- 172.21.15.92:0/2888098401 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f37f4066f30 0x7f37f40693e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.079+0000 7f3813fff700 1 --2- 172.21.15.92:0/2888098401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3814159440 0x7f381415b480 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.079+0000 7f3813fff700 1 --2- 172.21.15.92:0/2888098401 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140b2100 0x7f3814154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.079+0000 7f3813fff700 1 --2- 172.21.15.92:0/2888098401 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38140b35d0 0x7f381415af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.079+0000 7f3813fff700 1 -- 172.21.15.92:0/2888098401 >> 172.21.15.92:0/2888098401 conn(0x7f381406ece0 msgr2=0x7f38140cf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:03.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.079+0000 7f3813fff700 1 -- 172.21.15.92:0/2888098401 shutdown_connections 2023-10-06T10:39:03.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.079+0000 7f3813fff700 1 -- 172.21.15.92:0/2888098401 wait complete. 2023-10-06T10:39:03.083 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-10-06T10:39:03.093 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:39:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.205+0000 7fba03a58700 1 Processor -- start 2023-10-06T10:39:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.205+0000 7fba03a58700 1 -- start start 2023-10-06T10:39:03.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.206+0000 7fba03a58700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9fc159560 0x7fb9fc0aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:03.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.206+0000 7fba03a58700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9fc0b0c60 con 0x7fb9fc0b0760 2023-10-06T10:39:03.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.206+0000 7fba03a58700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9fc0b0f40 con 0x7fb9fc0b49a0 2023-10-06T10:39:03.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.206+0000 7fba03a58700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc0b1220 con 0x7fb9fc159560 2023-10-06T10:39:03.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.206+0000 7fba01ff5700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb9fc0b0760 0x7fb9fc0b1bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50400/0 (socket says 172.21.15.92:50400) 2023-10-06T10:39:03.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.206+0000 7fba01ff5700 1 -- 172.21.15.92:0/2266240274 learned_addr learned my addr 172.21.15.92:0/2266240274 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:03.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.206+0000 7fba017f4700 1 --2- 172.21.15.92:0/2266240274 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9fc159560 0x7fb9fc0aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:03.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.206+0000 7fb9f3fff700 1 -- 172.21.15.92:0/2266240274 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623717982 0 0) 0x7fb9fc0b0c60 con 0x7fb9fc0b0760 2023-10-06T10:39:03.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.206+0000 7fb9f3fff700 1 -- 172.21.15.92:0/2266240274 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9e8003570 con 0x7fb9fc0b0760 2023-10-06T10:39:03.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fb9f3fff700 1 -- 172.21.15.92:0/2266240274 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2199436817 0 0) 0x7fb9e8003570 con 0x7fb9fc0b0760 2023-10-06T10:39:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fb9f3fff700 1 -- 172.21.15.92:0/2266240274 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9fc159560 msgr2=0x7fb9fc0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fb9f3fff700 1 --2- 172.21.15.92:0/2266240274 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9fc159560 0x7fb9fc0aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fb9f3fff700 1 -- 172.21.15.92:0/2266240274 >> v1:172.21.15.124:6789/0 conn(0x7fb9fc0b49a0 legacy=0x7fb9fc0aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fb9f3fff700 1 -- 172.21.15.92:0/2266240274 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9fc0b1500 con 0x7fb9fc0b0760 2023-10-06T10:39:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fb9f3fff700 1 -- 172.21.15.92:0/2266240274 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fb9f40017f0 con 0x7fb9fc0b0760 2023-10-06T10:39:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fb9f3fff700 1 -- 172.21.15.92:0/2266240274 >> v1:172.21.15.92:6789/0 conn(0x7fb9fc0b0760 legacy=0x7fb9fc0b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fb9f3fff700 1 --2- 172.21.15.92:0/2266240274 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9fc159560 0x7fb9e8003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fb9f3fff700 1 --2- 172.21.15.92:0/2266240274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9e8004010 0x7fb9e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fb9f3fff700 1 --2- 172.21.15.92:0/2266240274 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9e8004d50 0x7fb9e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fb9f3fff700 1 -- 172.21.15.92:0/2266240274 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc0b1220 con 0x7fb9e8004010 2023-10-06T10:39:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fb9f3fff700 1 -- 172.21.15.92:0/2266240274 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e800bbf0 con 0x7fb9e8004d50 2023-10-06T10:39:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fb9f3fff700 1 -- 172.21.15.92:0/2266240274 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e800bdb0 con 0x7fb9fc159560 2023-10-06T10:39:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fba00ff3700 1 --2- 172.21.15.92:0/2266240274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9e8004010 0x7fb9e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fba017f4700 1 --2- 172.21.15.92:0/2266240274 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9fc159560 0x7fb9e8003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fba01ff5700 1 --2- 172.21.15.92:0/2266240274 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9e8004d50 0x7fb9e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fba00ff3700 1 -- 172.21.15.92:0/2266240274 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9fc159560 msgr2=0x7fb9e8003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fba00ff3700 1 --2- 172.21.15.92:0/2266240274 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9fc159560 0x7fb9e8003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fba00ff3700 1 -- 172.21.15.92:0/2266240274 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9e8004d50 msgr2=0x7fb9e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fba00ff3700 1 --2- 172.21.15.92:0/2266240274 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9e8004d50 0x7fb9e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fba00ff3700 1 -- 172.21.15.92:0/2266240274 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e8003570 con 0x7fb9e8004010 2023-10-06T10:39:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fba01ff5700 1 --2- 172.21.15.92:0/2266240274 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9e8004d50 0x7fb9e80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fba00ff3700 1 --2- 172.21.15.92:0/2266240274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9e8004010 0x7fb9e8004810 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0b4db0 tx=0x7fb9ec007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624ed53e41ea0b01 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fb9f3fff700 1 -- 172.21.15.92:0/2266240274 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec012420 con 0x7fb9e8004010 2023-10-06T10:39:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fb9f3fff700 1 -- 172.21.15.92:0/2266240274 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ec012580 con 0x7fb9e8004010 2023-10-06T10:39:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.207+0000 7fb9f3fff700 1 -- 172.21.15.92:0/2266240274 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec012920 con 0x7fb9e8004010 2023-10-06T10:39:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.208+0000 7fba03a58700 1 -- 172.21.15.92:0/2266240274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9e8004010 msgr2=0x7fb9e8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.208+0000 7fba03a58700 1 --2- 172.21.15.92:0/2266240274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9e8004010 0x7fb9e8004810 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0b4db0 tx=0x7fb9ec007d30 comp rx=0 tx=0).stop 2023-10-06T10:39:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.208+0000 7fba01ff5700 1 -- 172.21.15.92:0/2266240274 reap_dead start 2023-10-06T10:39:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.208+0000 7fba03a58700 1 -- 172.21.15.92:0/2266240274 shutdown_connections 2023-10-06T10:39:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.208+0000 7fba03a58700 1 -- 172.21.15.92:0/2266240274 >> 172.21.15.92:0/2266240274 conn(0x7fb9fc0997f0 msgr2=0x7fb9fc06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.208+0000 7fba03a58700 1 -- 172.21.15.92:0/2266240274 shutdown_connections 2023-10-06T10:39:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.208+0000 7fba03a58700 1 -- 172.21.15.92:0/2266240274 wait complete. 2023-10-06T10:39:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.209+0000 7fba03a58700 1 Processor -- start 2023-10-06T10:39:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.209+0000 7fba03a58700 1 -- start start 2023-10-06T10:39:03.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.210+0000 7fba03a58700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9e8004010 0x7fb9fc156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:03.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.210+0000 7fba03a58700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9fc09d5d0 0x7fb9fc15a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:03.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.210+0000 7fba03a58700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9fc15ab60 0x7fb9fc15afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:03.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.210+0000 7fba03a58700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc09c320 con 0x7fb9e8004010 2023-10-06T10:39:03.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.210+0000 7fba03a58700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc09d140 con 0x7fb9fc15ab60 2023-10-06T10:39:03.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.210+0000 7fba03a58700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc09c530 con 0x7fb9fc09d5d0 2023-10-06T10:39:03.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.210+0000 7fba017f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9e8004010 0x7fb9fc156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:03.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.210+0000 7fba017f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9e8004010 0x7fb9fc156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47376/0 (socket says 172.21.15.92:47376) 2023-10-06T10:39:03.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.210+0000 7fba017f4700 1 -- 172.21.15.92:0/156874866 learned_addr learned my addr 172.21.15.92:0/156874866 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:03.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.210+0000 7fba017f4700 1 -- 172.21.15.92:0/156874866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9fc09d5d0 msgr2=0x7fb9fc15a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:03.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.210+0000 7fba017f4700 1 --2- 172.21.15.92:0/156874866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9fc09d5d0 0x7fb9fc15a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.210+0000 7fba017f4700 1 -- 172.21.15.92:0/156874866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9fc15ab60 msgr2=0x7fb9fc15afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:03.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.210+0000 7fba017f4700 1 --2- 172.21.15.92:0/156874866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9fc15ab60 0x7fb9fc15afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.210+0000 7fba017f4700 1 -- 172.21.15.92:0/156874866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9fc09c810 con 0x7fb9e8004010 2023-10-06T10:39:03.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.210+0000 7fba017f4700 1 --2- 172.21.15.92:0/156874866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9e8004010 0x7fb9fc156bc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8011b60 tx=0x7fb9f8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:03.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.211+0000 7fb9f17fa700 1 -- 172.21.15.92:0/156874866 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8005c10 con 0x7fb9e8004010 2023-10-06T10:39:03.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.211+0000 7fb9f17fa700 1 -- 172.21.15.92:0/156874866 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f8005d70 con 0x7fb9e8004010 2023-10-06T10:39:03.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.211+0000 7fba03a58700 1 -- 172.21.15.92:0/156874866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9fc09caf0 con 0x7fb9e8004010 2023-10-06T10:39:03.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.211+0000 7fb9f17fa700 1 -- 172.21.15.92:0/156874866 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f800fa80 con 0x7fb9e8004010 2023-10-06T10:39:03.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.211+0000 7fba03a58700 1 -- 172.21.15.92:0/156874866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9fc157100 con 0x7fb9e8004010 2023-10-06T10:39:03.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.212+0000 7fba03a58700 1 -- 172.21.15.92:0/156874866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9fc064a90 con 0x7fb9e8004010 2023-10-06T10:39:03.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.212+0000 7fb9f17fa700 1 -- 172.21.15.92:0/156874866 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb9f8006050 con 0x7fb9e8004010 2023-10-06T10:39:03.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.212+0000 7fb9f17fa700 1 --2- 172.21.15.92:0/156874866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb9e0066f30 0x7fb9e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:03.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.212+0000 7fb9f17fa700 1 -- 172.21.15.92:0/156874866 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb9f8085620 con 0x7fb9e8004010 2023-10-06T10:39:03.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.212+0000 7fba00ff3700 1 --2- 172.21.15.92:0/156874866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb9e0066f30 0x7fb9e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:03.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.213+0000 7fba00ff3700 1 --2- 172.21.15.92:0/156874866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb9e0066f30 0x7fb9e00693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc15c230 tx=0x7fb9ec015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:03.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.216+0000 7fb9f17fa700 1 -- 172.21.15.92:0/156874866 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9f8052440 con 0x7fb9e8004010 2023-10-06T10:39:03.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.404+0000 7fba03a58700 1 -- 172.21.15.92:0/156874866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9fc0b17e0 con 0x7fb9e8004010 2023-10-06T10:39:03.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.405+0000 7fb9f17fa700 1 -- 172.21.15.92:0/156874866 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18707 (secure 0 0 0) 0x7fb9f8055a60 con 0x7fb9e8004010 2023-10-06T10:39:03.407 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:39:03.407 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:38:21.760233+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:38:14.368713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T10:38:15.692344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T10:38:19.382553+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:39:03.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.407+0000 7fba03a58700 1 -- 172.21.15.92:0/156874866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb9e0066f30 msgr2=0x7fb9e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:03.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.407+0000 7fba03a58700 1 --2- 172.21.15.92:0/156874866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb9e0066f30 0x7fb9e00693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc15c230 tx=0x7fb9ec015000 comp rx=0 tx=0).stop 2023-10-06T10:39:03.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.407+0000 7fba03a58700 1 -- 172.21.15.92:0/156874866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9e8004010 msgr2=0x7fb9fc156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:03.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.407+0000 7fba03a58700 1 --2- 172.21.15.92:0/156874866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9e8004010 0x7fb9fc156bc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8011b60 tx=0x7fb9f8009120 comp rx=0 tx=0).stop 2023-10-06T10:39:03.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.407+0000 7fba03a58700 1 -- 172.21.15.92:0/156874866 shutdown_connections 2023-10-06T10:39:03.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.407+0000 7fba03a58700 1 --2- 172.21.15.92:0/156874866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb9e0066f30 0x7fb9e00693e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.407+0000 7fba03a58700 1 --2- 172.21.15.92:0/156874866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9fc15ab60 0x7fb9fc15afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.407+0000 7fba03a58700 1 --2- 172.21.15.92:0/156874866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9e8004010 0x7fb9fc156bc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.407+0000 7fba03a58700 1 --2- 172.21.15.92:0/156874866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9fc09d5d0 0x7fb9fc15a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.407+0000 7fba03a58700 1 -- 172.21.15.92:0/156874866 >> 172.21.15.92:0/156874866 conn(0x7fb9fc0997f0 msgr2=0x7fb9fc1575b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:03.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.407+0000 7fba03a58700 1 -- 172.21.15.92:0/156874866 shutdown_connections 2023-10-06T10:39:03.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.407+0000 7fba03a58700 1 -- 172.21.15.92:0/156874866 wait complete. 2023-10-06T10:39:03.421 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T10:39:03.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.534+0000 7f1b5ffc0700 1 Processor -- start 2023-10-06T10:39:03.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.534+0000 7f1b5ffc0700 1 -- start start 2023-10-06T10:39:03.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.535+0000 7f1b5ffc0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b58159560 0x7f1b580aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:03.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.535+0000 7f1b5ffc0700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1b580b49a0 0x7f1b580aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:03.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.535+0000 7f1b5ffc0700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b580b0c60 con 0x7f1b580b0760 2023-10-06T10:39:03.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.535+0000 7f1b5ffc0700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b580b0f40 con 0x7f1b580b49a0 2023-10-06T10:39:03.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.535+0000 7f1b5ffc0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b580b1220 con 0x7f1b58159560 2023-10-06T10:39:03.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.536+0000 7f1b5d55b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1b580b49a0 0x7f1b580aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:03.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.536+0000 7f1b5dd5c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b58159560 0x7f1b580aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:03.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.536+0000 7f1b5d55b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1b580b49a0 0x7f1b580aaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:34380/0 (socket says 172.21.15.92:34380) 2023-10-06T10:39:03.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.536+0000 7f1b5dd5c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b58159560 0x7f1b580aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:33018/0 (socket says 172.21.15.92:33018) 2023-10-06T10:39:03.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.536+0000 7f1b5d55b700 1 -- 172.21.15.92:0/1395401094 learned_addr learned my addr 172.21.15.92:0/1395401094 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:03.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.536+0000 7f1b5e55d700 1 --1- 172.21.15.92:0/1395401094 >> v1:172.21.15.124:6789/0 conn(0x7f1b580b0760 0x7f1b580b3890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:50232/0 (socket says 172.21.15.92:50232) 2023-10-06T10:39:03.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.536+0000 7f1b5dd5c700 1 -- 172.21.15.92:0/1395401094 >> v2:172.21.15.124:3300/0 conn(0x7f1b580b49a0 msgr2=0x7f1b580aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:03.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.536+0000 7f1b5dd5c700 1 --2- 172.21.15.92:0/1395401094 >> v2:172.21.15.124:3300/0 conn(0x7f1b580b49a0 0x7f1b580aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.536+0000 7f1b5dd5c700 1 -- 172.21.15.92:0/1395401094 >> v1:172.21.15.124:6789/0 conn(0x7f1b580b0760 legacy=0x7f1b580b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:03.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.536+0000 7f1b5dd5c700 1 -- 172.21.15.92:0/1395401094 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b580b1500 con 0x7f1b58159560 2023-10-06T10:39:03.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.536+0000 7f1b5d55b700 1 --2- 172.21.15.92:0/1395401094 >> v2:172.21.15.124:3300/0 conn(0x7f1b580b49a0 0x7f1b580aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:39:03.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.536+0000 7f1b5dd5c700 1 --2- 172.21.15.92:0/1395401094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b58159560 0x7f1b580aa530 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1b500098d0 tx=0x7f1b50009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a6617403f6ad201 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:03.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.536+0000 7f1b5cd5a700 1 -- 172.21.15.92:0/1395401094 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50005c10 con 0x7f1b58159560 2023-10-06T10:39:03.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.536+0000 7f1b5cd5a700 1 -- 172.21.15.92:0/1395401094 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b50005d70 con 0x7f1b58159560 2023-10-06T10:39:03.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.536+0000 7f1b5cd5a700 1 -- 172.21.15.92:0/1395401094 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50006280 con 0x7f1b58159560 2023-10-06T10:39:03.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.537+0000 7f1b5ffc0700 1 -- 172.21.15.92:0/1395401094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b58159560 msgr2=0x7f1b580aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:03.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.537+0000 7f1b5ffc0700 1 --2- 172.21.15.92:0/1395401094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b58159560 0x7f1b580aa530 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1b500098d0 tx=0x7f1b50009120 comp rx=0 tx=0).stop 2023-10-06T10:39:03.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.537+0000 7f1b5ffc0700 1 -- 172.21.15.92:0/1395401094 shutdown_connections 2023-10-06T10:39:03.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.537+0000 7f1b5ffc0700 1 --2- 172.21.15.92:0/1395401094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b58159560 0x7f1b580aa530 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.537+0000 7f1b5ffc0700 1 --2- 172.21.15.92:0/1395401094 >> v2:172.21.15.124:3300/0 conn(0x7f1b580b49a0 0x7f1b580aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.537+0000 7f1b5ffc0700 1 -- 172.21.15.92:0/1395401094 >> 172.21.15.92:0/1395401094 conn(0x7f1b580997f0 msgr2=0x7f1b5806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:03.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.537+0000 7f1b5ffc0700 1 -- 172.21.15.92:0/1395401094 shutdown_connections 2023-10-06T10:39:03.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.537+0000 7f1b5ffc0700 1 -- 172.21.15.92:0/1395401094 wait complete. 2023-10-06T10:39:03.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.538+0000 7f1b5ffc0700 1 Processor -- start 2023-10-06T10:39:03.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.538+0000 7f1b5ffc0700 1 -- start start 2023-10-06T10:39:03.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.539+0000 7f1b5ffc0700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b580b0760 0x7f1b58157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:03.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.539+0000 7f1b5ffc0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b580b49a0 0x7f1b58169dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:03.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.539+0000 7f1b5ffc0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b58159560 0x7f1b580c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:03.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.539+0000 7f1b5ffc0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b580c7450 con 0x7f1b58159560 2023-10-06T10:39:03.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.539+0000 7f1b5ffc0700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b580c5cf0 con 0x7f1b580b0760 2023-10-06T10:39:03.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.539+0000 7f1b5ffc0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b580c5fd0 con 0x7f1b580b49a0 2023-10-06T10:39:03.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.539+0000 7f1b5e55d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b58159560 0x7f1b580c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:03.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.539+0000 7f1b5e55d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b58159560 0x7f1b580c5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46514/0 (socket says 172.21.15.92:46514) 2023-10-06T10:39:03.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.539+0000 7f1b5e55d700 1 -- 172.21.15.92:0/2053758615 learned_addr learned my addr 172.21.15.92:0/2053758615 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:03.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.539+0000 7f1b5dd5c700 1 --2- 172.21.15.92:0/2053758615 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b580b0760 0x7f1b58157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:03.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.539+0000 7f1b5d55b700 1 --2- 172.21.15.92:0/2053758615 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b580b49a0 0x7f1b58169dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:03.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.539+0000 7f1b5e55d700 1 -- 172.21.15.92:0/2053758615 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b580b49a0 msgr2=0x7f1b58169dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:03.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.539+0000 7f1b5e55d700 1 --2- 172.21.15.92:0/2053758615 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b580b49a0 0x7f1b58169dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.539+0000 7f1b5e55d700 1 -- 172.21.15.92:0/2053758615 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b580b0760 msgr2=0x7f1b58157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:03.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.539+0000 7f1b5e55d700 1 --2- 172.21.15.92:0/2053758615 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b580b0760 0x7f1b58157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.539+0000 7f1b5e55d700 1 -- 172.21.15.92:0/2053758615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4c0094f0 con 0x7f1b58159560 2023-10-06T10:39:03.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.539+0000 7f1b5dd5c700 1 --2- 172.21.15.92:0/2053758615 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b580b0760 0x7f1b58157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:03.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.540+0000 7f1b5e55d700 1 --2- 172.21.15.92:0/2053758615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b58159560 0x7f1b580c5230 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800cd00 tx=0x7f1b48001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:03.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.540+0000 7f1b55ffb700 1 -- 172.21.15.92:0/2053758615 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b48005df0 con 0x7f1b58159560 2023-10-06T10:39:03.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.540+0000 7f1b55ffb700 1 -- 172.21.15.92:0/2053758615 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4800a960 con 0x7f1b58159560 2023-10-06T10:39:03.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.540+0000 7f1b5ffc0700 1 -- 172.21.15.92:0/2053758615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b500093c0 con 0x7f1b58159560 2023-10-06T10:39:03.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.540+0000 7f1b55ffb700 1 -- 172.21.15.92:0/2053758615 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4800ad00 con 0x7f1b58159560 2023-10-06T10:39:03.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.540+0000 7f1b5ffc0700 1 -- 172.21.15.92:0/2053758615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b580c70e0 con 0x7f1b58159560 2023-10-06T10:39:03.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.541+0000 7f1b5ffc0700 1 -- 172.21.15.92:0/2053758615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b580c75c0 con 0x7f1b58159560 2023-10-06T10:39:03.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.541+0000 7f1b55ffb700 1 -- 172.21.15.92:0/2053758615 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1b48007300 con 0x7f1b58159560 2023-10-06T10:39:03.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.541+0000 7f1b55ffb700 1 --2- 172.21.15.92:0/2053758615 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1b44066f60 0x7f1b44069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:03.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.541+0000 7f1b55ffb700 1 -- 172.21.15.92:0/2053758615 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f1b48084ac0 con 0x7f1b58159560 2023-10-06T10:39:03.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.542+0000 7f1b5dd5c700 1 --2- 172.21.15.92:0/2053758615 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1b44066f60 0x7f1b44069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:03.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.542+0000 7f1b5dd5c700 1 --2- 172.21.15.92:0/2053758615 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1b44066f60 0x7f1b44069410 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009160 tx=0x7f1b50016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:03.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.546+0000 7f1b55ffb700 1 -- 172.21.15.92:0/2053758615 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b48051960 con 0x7f1b58159560 2023-10-06T10:39:03.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.769+0000 7f1b5ffc0700 1 -- 172.21.15.92:0/2053758615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1b58064a90 con 0x7f1b58159560 2023-10-06T10:39:03.771 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T10:39:03.769+0000 7f9e7f3ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T10:39:03.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.774+0000 7f1b55ffb700 1 -- 172.21.15.92:0/2053758615 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1b48084e60 con 0x7f1b58159560 2023-10-06T10:39:03.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.776+0000 7f1b5ffc0700 1 -- 172.21.15.92:0/2053758615 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1b44066f60 msgr2=0x7f1b44069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:03.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.776+0000 7f1b5ffc0700 1 --2- 172.21.15.92:0/2053758615 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1b44066f60 0x7f1b44069410 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009160 tx=0x7f1b50016000 comp rx=0 tx=0).stop 2023-10-06T10:39:03.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.776+0000 7f1b5ffc0700 1 -- 172.21.15.92:0/2053758615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b58159560 msgr2=0x7f1b580c5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:03.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.776+0000 7f1b5ffc0700 1 --2- 172.21.15.92:0/2053758615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b58159560 0x7f1b580c5230 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800cd00 tx=0x7f1b48001b10 comp rx=0 tx=0).stop 2023-10-06T10:39:03.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.776+0000 7f1b5ffc0700 1 -- 172.21.15.92:0/2053758615 shutdown_connections 2023-10-06T10:39:03.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.776+0000 7f1b5ffc0700 1 --2- 172.21.15.92:0/2053758615 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1b44066f60 0x7f1b44069410 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.776+0000 7f1b5ffc0700 1 --2- 172.21.15.92:0/2053758615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b58159560 0x7f1b580c5230 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.776+0000 7f1b5ffc0700 1 --2- 172.21.15.92:0/2053758615 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b580b0760 0x7f1b58157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.776+0000 7f1b5ffc0700 1 --2- 172.21.15.92:0/2053758615 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b580b49a0 0x7f1b58169dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.776+0000 7f1b5ffc0700 1 -- 172.21.15.92:0/2053758615 >> 172.21.15.92:0/2053758615 conn(0x7f1b580997f0 msgr2=0x7f1b580d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:03.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.776+0000 7f1b5ffc0700 1 -- 172.21.15.92:0/2053758615 shutdown_connections 2023-10-06T10:39:03.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.776+0000 7f1b5ffc0700 1 -- 172.21.15.92:0/2053758615 wait complete. 2023-10-06T10:39:03.781 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T10:39:03.781 INFO:tasks.ceph.mds.d.smithi124.stderr: -3664> 2023-10-06T10:38:22.429+0000 7f1c17b7e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T10:39:03.790 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T10:39:03.805 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-06T10:39:03.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.905+0000 7f9e71778700 1 Processor -- start 2023-10-06T10:39:03.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.905+0000 7f9e71778700 1 -- start start 2023-10-06T10:39:03.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.906+0000 7f9e71778700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9e6c0cd4d0 0x7f9e6c0b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:03.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.906+0000 7f9e71778700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e6c15ac30 con 0x7f9e6c0c5250 2023-10-06T10:39:03.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.906+0000 7f9e71778700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e6c0cb6a0 con 0x7f9e6c150940 2023-10-06T10:39:03.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.906+0000 7f9e71778700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e6c0cb090 con 0x7f9e6c0cd4d0 2023-10-06T10:39:03.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.906+0000 7f9e627fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9e6c0c5250 0x7f9e6c0ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33104/0 (socket says 172.21.15.92:33104) 2023-10-06T10:39:03.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.906+0000 7f9e627fc700 1 -- 172.21.15.92:0/1673513315 learned_addr learned my addr 172.21.15.92:0/1673513315 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:03.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.907+0000 7f9e6b7fe700 1 --2- 172.21.15.92:0/1673513315 >> v2:172.21.15.124:3300/0 conn(0x7f9e6c0cd4d0 0x7f9e6c0b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:03.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.907+0000 7f9e6a7fc700 1 -- 172.21.15.92:0/1673513315 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229627163 0 0) 0x7f9e54001110 con 0x7f9e6c0c5250 2023-10-06T10:39:03.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.907+0000 7f9e6a7fc700 1 -- 172.21.15.92:0/1673513315 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e58003570 con 0x7f9e6c0c5250 2023-10-06T10:39:03.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.907+0000 7f9e6b7fe700 1 -- 172.21.15.92:0/1673513315 >> v1:172.21.15.124:6789/0 conn(0x7f9e6c150940 legacy=0x7f9e6c159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:03.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.907+0000 7f9e6a7fc700 1 -- 172.21.15.92:0/1673513315 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1667658980 0 0) 0x7f9e58003570 con 0x7f9e6c0c5250 2023-10-06T10:39:03.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.907+0000 7f9e6b7fe700 1 -- 172.21.15.92:0/1673513315 >> v1:172.21.15.92:6789/0 conn(0x7f9e6c0c5250 legacy=0x7f9e6c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:03.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.907+0000 7f9e6b7fe700 1 -- 172.21.15.92:0/1673513315 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c15a670 con 0x7f9e6c0cd4d0 2023-10-06T10:39:03.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.907+0000 7f9e6b7fe700 1 --2- 172.21.15.92:0/1673513315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e6c0cd4d0 0x7f9e6c0b2050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009e60 tx=0x7f9e64009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cfd9e5ea95ac3da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:03.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.907+0000 7f9e6a7fc700 1 -- 172.21.15.92:0/1673513315 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6400e070 con 0x7f9e6c0cd4d0 2023-10-06T10:39:03.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.907+0000 7f9e6a7fc700 1 -- 172.21.15.92:0/1673513315 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6400b850 con 0x7f9e6c0cd4d0 2023-10-06T10:39:03.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.907+0000 7f9e6a7fc700 1 -- 172.21.15.92:0/1673513315 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e54001600 con 0x7f9e6c0cd4d0 2023-10-06T10:39:03.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.908+0000 7f9e71778700 1 -- 172.21.15.92:0/1673513315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e6c0cd4d0 msgr2=0x7f9e6c0b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:03.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.908+0000 7f9e71778700 1 --2- 172.21.15.92:0/1673513315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e6c0cd4d0 0x7f9e6c0b2050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009e60 tx=0x7f9e64009780 comp rx=0 tx=0).stop 2023-10-06T10:39:03.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.908+0000 7f9e71778700 1 -- 172.21.15.92:0/1673513315 shutdown_connections 2023-10-06T10:39:03.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.908+0000 7f9e71778700 1 --2- 172.21.15.92:0/1673513315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e6c0cd4d0 0x7f9e6c0b2050 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.908+0000 7f9e71778700 1 -- 172.21.15.92:0/1673513315 >> 172.21.15.92:0/1673513315 conn(0x7f9e6c099700 msgr2=0x7f9e6c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:03.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.908+0000 7f9e71778700 1 -- 172.21.15.92:0/1673513315 shutdown_connections 2023-10-06T10:39:03.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.908+0000 7f9e71778700 1 -- 172.21.15.92:0/1673513315 wait complete. 2023-10-06T10:39:03.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.909+0000 7f9e71778700 1 Processor -- start 2023-10-06T10:39:03.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.909+0000 7f9e71778700 1 -- start start 2023-10-06T10:39:03.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.910+0000 7f9e71778700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e6c0c5250 0x7f9e6c15c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:03.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.910+0000 7f9e71778700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e6c0cd4d0 0x7f9e6c09d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:03.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.910+0000 7f9e71778700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e6c0b3600 0x7f9e6c0b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:03.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.910+0000 7f9e71778700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c0b3fa0 con 0x7f9e6c0cd4d0 2023-10-06T10:39:03.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.910+0000 7f9e71778700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c0c8e60 con 0x7f9e6c0b3600 2023-10-06T10:39:03.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.910+0000 7f9e71778700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c15ced0 con 0x7f9e6c0c5250 2023-10-06T10:39:03.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.910+0000 7f9e627fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e6c0cd4d0 0x7f9e6c09d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:03.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.910+0000 7f9e627fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e6c0cd4d0 0x7f9e6c09d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46518/0 (socket says 172.21.15.92:46518) 2023-10-06T10:39:03.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.910+0000 7f9e627fc700 1 -- 172.21.15.92:0/2291456305 learned_addr learned my addr 172.21.15.92:0/2291456305 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:03.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.910+0000 7f9e627fc700 1 -- 172.21.15.92:0/2291456305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e6c0c5250 msgr2=0x7f9e6c15c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:03.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.910+0000 7f9e627fc700 1 --2- 172.21.15.92:0/2291456305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e6c0c5250 0x7f9e6c15c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.910+0000 7f9e627fc700 1 -- 172.21.15.92:0/2291456305 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e6c0b3600 msgr2=0x7f9e6c0b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:03.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.910+0000 7f9e6b7fe700 1 --2- 172.21.15.92:0/2291456305 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e6c0b3600 0x7f9e6c0b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:03.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.910+0000 7f9e627fc700 1 --2- 172.21.15.92:0/2291456305 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e6c0b3600 0x7f9e6c0b3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:03.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.910+0000 7f9e627fc700 1 -- 172.21.15.92:0/2291456305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e64009a20 con 0x7f9e6c0cd4d0 2023-10-06T10:39:03.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.910+0000 7f9e627fc700 1 --2- 172.21.15.92:0/2291456305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e6c0cd4d0 0x7f9e6c09d520 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9e5400c810 tx=0x7f9e54001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:03.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.911+0000 7f9e637fe700 1 -- 172.21.15.92:0/2291456305 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e54007360 con 0x7f9e6c0cd4d0 2023-10-06T10:39:03.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.911+0000 7f9e637fe700 1 -- 172.21.15.92:0/2291456305 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e540074e0 con 0x7f9e6c0cd4d0 2023-10-06T10:39:03.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.911+0000 7f9e71778700 1 -- 172.21.15.92:0/2291456305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e6c16b100 con 0x7f9e6c0cd4d0 2023-10-06T10:39:03.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.911+0000 7f9e637fe700 1 -- 172.21.15.92:0/2291456305 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e54004830 con 0x7f9e6c0cd4d0 2023-10-06T10:39:03.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.911+0000 7f9e71778700 1 -- 172.21.15.92:0/2291456305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6c0c8810 con 0x7f9e6c0cd4d0 2023-10-06T10:39:03.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.911+0000 7f9e71778700 1 -- 172.21.15.92:0/2291456305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e6c15f130 con 0x7f9e6c0cd4d0 2023-10-06T10:39:03.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.912+0000 7f9e637fe700 1 -- 172.21.15.92:0/2291456305 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9e54004be0 con 0x7f9e6c0cd4d0 2023-10-06T10:39:03.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.912+0000 7f9e637fe700 1 --2- 172.21.15.92:0/2291456305 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9e4c066f30 0x7f9e4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:03.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.912+0000 7f9e637fe700 1 -- 172.21.15.92:0/2291456305 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6895+0+0 (secure 0 0 0) 0x7f9e540840b0 con 0x7f9e6c0cd4d0 2023-10-06T10:39:03.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.912+0000 7f9e6affd700 1 --2- 172.21.15.92:0/2291456305 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9e4c066f30 0x7f9e4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:03.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.913+0000 7f9e6affd700 1 --2- 172.21.15.92:0/2291456305 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9e4c066f30 0x7f9e4c0693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0017c0 tx=0x7f9e5c007a60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:03.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:03.917+0000 7f9e637fe700 1 -- 172.21.15.92:0/2291456305 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e54050e60 con 0x7f9e6c0cd4d0 2023-10-06T10:39:04.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.143+0000 7f9e71778700 1 -- 172.21.15.92:0/2291456305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9e6c15ac30 con 0x7f9e6c0cd4d0 2023-10-06T10:39:04.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.777+0000 7f9e637fe700 1 -- 172.21.15.92:0/2291456305 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9e6c15ac30 con 0x7f9e6c0cd4d0 2023-10-06T10:39:04.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.780+0000 7f9e71778700 1 -- 172.21.15.92:0/2291456305 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9e4c066f30 msgr2=0x7f9e4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:04.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.780+0000 7f9e71778700 1 --2- 172.21.15.92:0/2291456305 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9e4c066f30 0x7f9e4c0693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0017c0 tx=0x7f9e5c007a60 comp rx=0 tx=0).stop 2023-10-06T10:39:04.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.780+0000 7f9e71778700 1 -- 172.21.15.92:0/2291456305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e6c0cd4d0 msgr2=0x7f9e6c09d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:04.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.780+0000 7f9e71778700 1 --2- 172.21.15.92:0/2291456305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e6c0cd4d0 0x7f9e6c09d520 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9e5400c810 tx=0x7f9e54001a30 comp rx=0 tx=0).stop 2023-10-06T10:39:04.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.781+0000 7f9e71778700 1 -- 172.21.15.92:0/2291456305 shutdown_connections 2023-10-06T10:39:04.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.781+0000 7f9e71778700 1 --2- 172.21.15.92:0/2291456305 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9e4c066f30 0x7f9e4c0693e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:04.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.781+0000 7f9e71778700 1 --2- 172.21.15.92:0/2291456305 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e6c0b3600 0x7f9e6c0b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:04.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.781+0000 7f9e71778700 1 --2- 172.21.15.92:0/2291456305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e6c0c5250 0x7f9e6c15c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:04.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.781+0000 7f9e71778700 1 --2- 172.21.15.92:0/2291456305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e6c0cd4d0 0x7f9e6c09d520 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:04.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.781+0000 7f9e71778700 1 -- 172.21.15.92:0/2291456305 >> 172.21.15.92:0/2291456305 conn(0x7f9e6c099700 msgr2=0x7f9e6c0cbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:04.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.781+0000 7f9e71778700 1 -- 172.21.15.92:0/2291456305 shutdown_connections 2023-10-06T10:39:04.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.781+0000 7f9e71778700 1 -- 172.21.15.92:0/2291456305 wait complete. 2023-10-06T10:39:04.796 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T10:39:04.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.912+0000 7fad1c7a9700 1 Processor -- start 2023-10-06T10:39:04.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.912+0000 7fad1c7a9700 1 -- start start 2023-10-06T10:39:04.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.913+0000 7fad1c7a9700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fad1415ad30 0x7fad1415b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:04.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.913+0000 7fad1c7a9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad1415c400 con 0x7fad14153950 2023-10-06T10:39:04.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.913+0000 7fad1c7a9700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad14158200 con 0x7fad14153d20 2023-10-06T10:39:04.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.913+0000 7fad1c7a9700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1415cfa0 con 0x7fad1415ad30 2023-10-06T10:39:04.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.913+0000 7fad1a545700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fad14153950 0x7fad1415c890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33110/0 (socket says 172.21.15.92:33110) 2023-10-06T10:39:04.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.913+0000 7fad1a545700 1 -- 172.21.15.92:0/4287323448 learned_addr learned my addr 172.21.15.92:0/4287323448 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:04.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.913+0000 7fad19543700 1 -- 172.21.15.92:0/4287323448 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1301255519 0 0) 0x7fad1415c400 con 0x7fad14153950 2023-10-06T10:39:04.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.913+0000 7fad19543700 1 -- 172.21.15.92:0/4287323448 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad00003570 con 0x7fad14153950 2023-10-06T10:39:04.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.913+0000 7fad1ad46700 1 --2- 172.21.15.92:0/4287323448 >> v2:172.21.15.124:3300/0 conn(0x7fad1415ad30 0x7fad1415b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:04.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad19543700 1 -- 172.21.15.92:0/4287323448 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 233480806 0 0) 0x7fad00003570 con 0x7fad14153950 2023-10-06T10:39:04.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad19543700 1 -- 172.21.15.92:0/4287323448 >> v2:172.21.15.124:3300/0 conn(0x7fad1415ad30 msgr2=0x7fad1415b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:04.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad19543700 1 --2- 172.21.15.92:0/4287323448 >> v2:172.21.15.124:3300/0 conn(0x7fad1415ad30 0x7fad1415b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad19543700 1 -- 172.21.15.92:0/4287323448 >> v1:172.21.15.124:6789/0 conn(0x7fad14153d20 legacy=0x7fad141577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad19543700 1 -- 172.21.15.92:0/4287323448 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1415b7f0 con 0x7fad14153950 2023-10-06T10:39:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad19543700 1 -- 172.21.15.92:0/4287323448 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fad10001970 con 0x7fad14153950 2023-10-06T10:39:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad19543700 1 -- 172.21.15.92:0/4287323448 >> v1:172.21.15.92:6789/0 conn(0x7fad14153950 legacy=0x7fad1415c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad19543700 1 --2- 172.21.15.92:0/4287323448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad00003a60 0x7fad00003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad19543700 1 --2- 172.21.15.92:0/4287323448 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad00004810 0x7fad00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad19543700 1 --2- 172.21.15.92:0/4287323448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad000052e0 0x7fad0000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad19543700 1 -- 172.21.15.92:0/4287323448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0000c160 con 0x7fad00003a60 2023-10-06T10:39:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad19543700 1 -- 172.21.15.92:0/4287323448 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0000c2d0 con 0x7fad00004810 2023-10-06T10:39:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad19543700 1 -- 172.21.15.92:0/4287323448 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0000c490 con 0x7fad000052e0 2023-10-06T10:39:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad1ad46700 1 --2- 172.21.15.92:0/4287323448 >> v2:172.21.15.124:3300/0 conn(0x7fad1415ad30 0x7fad1415b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad1a545700 1 --2- 172.21.15.92:0/4287323448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad00003a60 0x7fad00003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad19d44700 1 --2- 172.21.15.92:0/4287323448 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad00004810 0x7fad00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:04.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad1ad46700 1 --2- 172.21.15.92:0/4287323448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad000052e0 0x7fad0000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:04.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad1a545700 1 -- 172.21.15.92:0/4287323448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad000052e0 msgr2=0x7fad0000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:04.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad1a545700 1 --2- 172.21.15.92:0/4287323448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad000052e0 0x7fad0000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:04.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad1a545700 1 -- 172.21.15.92:0/4287323448 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad00004810 msgr2=0x7fad00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:04.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad1a545700 1 --2- 172.21.15.92:0/4287323448 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad00004810 0x7fad00004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:04.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad1a545700 1 -- 172.21.15.92:0/4287323448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad00003570 con 0x7fad00003a60 2023-10-06T10:39:04.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad1ad46700 1 -- 172.21.15.92:0/4287323448 reap_dead start 2023-10-06T10:39:04.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad1a545700 1 --2- 172.21.15.92:0/4287323448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad00003a60 0x7fad00003e70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fad1000c670 tx=0x7fad10007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd44087f263b251 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:04.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad19543700 1 -- 172.21.15.92:0/4287323448 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10005b70 con 0x7fad00003a60 2023-10-06T10:39:04.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad19543700 1 -- 172.21.15.92:0/4287323448 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad10005cf0 con 0x7fad00003a60 2023-10-06T10:39:04.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.914+0000 7fad19543700 1 -- 172.21.15.92:0/4287323448 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10007360 con 0x7fad00003a60 2023-10-06T10:39:04.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.915+0000 7fad1c7a9700 1 -- 172.21.15.92:0/4287323448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad00003a60 msgr2=0x7fad00003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:04.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.915+0000 7fad1c7a9700 1 --2- 172.21.15.92:0/4287323448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad00003a60 0x7fad00003e70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fad1000c670 tx=0x7fad10007ca0 comp rx=0 tx=0).stop 2023-10-06T10:39:04.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.915+0000 7fad1c7a9700 1 -- 172.21.15.92:0/4287323448 shutdown_connections 2023-10-06T10:39:04.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.915+0000 7fad1c7a9700 1 --2- 172.21.15.92:0/4287323448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad00003a60 0x7fad00003e70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:04.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.915+0000 7fad1c7a9700 1 -- 172.21.15.92:0/4287323448 >> 172.21.15.92:0/4287323448 conn(0x7fad1406ece0 msgr2=0x7fad1406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:04.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.915+0000 7fad1c7a9700 1 -- 172.21.15.92:0/4287323448 shutdown_connections 2023-10-06T10:39:04.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.915+0000 7fad1c7a9700 1 -- 172.21.15.92:0/4287323448 wait complete. 2023-10-06T10:39:04.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.916+0000 7fad1c7a9700 1 Processor -- start 2023-10-06T10:39:04.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.916+0000 7fad1c7a9700 1 -- start start 2023-10-06T10:39:04.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.917+0000 7fad1c7a9700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad00003a60 0x7fad1413f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:04.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.917+0000 7fad1c7a9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad14154140 0x7fad140cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:04.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.917+0000 7fad1c7a9700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad140d01f0 0x7fad14156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:04.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.917+0000 7fad1c7a9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad140d0640 con 0x7fad14154140 2023-10-06T10:39:04.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.917+0000 7fad1c7a9700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1416b4c0 con 0x7fad140d01f0 2023-10-06T10:39:04.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.917+0000 7fad1c7a9700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1416b710 con 0x7fad00003a60 2023-10-06T10:39:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.917+0000 7fad19d44700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad14154140 0x7fad140cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.917+0000 7fad19d44700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad14154140 0x7fad140cfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46532/0 (socket says 172.21.15.92:46532) 2023-10-06T10:39:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.917+0000 7fad19d44700 1 -- 172.21.15.92:0/2605105805 learned_addr learned my addr 172.21.15.92:0/2605105805 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.917+0000 7fad1a545700 1 --2- 172.21.15.92:0/2605105805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad00003a60 0x7fad1413f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.917+0000 7fad1ad46700 1 --2- 172.21.15.92:0/2605105805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad140d01f0 0x7fad14156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.917+0000 7fad19d44700 1 -- 172.21.15.92:0/2605105805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad00003a60 msgr2=0x7fad1413f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.917+0000 7fad19d44700 1 --2- 172.21.15.92:0/2605105805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad00003a60 0x7fad1413f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.917+0000 7fad19d44700 1 -- 172.21.15.92:0/2605105805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad140d01f0 msgr2=0x7fad14156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.917+0000 7fad19d44700 1 --2- 172.21.15.92:0/2605105805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad140d01f0 0x7fad14156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.917+0000 7fad19d44700 1 -- 172.21.15.92:0/2605105805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1416b9f0 con 0x7fad14154140 2023-10-06T10:39:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.917+0000 7fad1ad46700 1 --2- 172.21.15.92:0/2605105805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad140d01f0 0x7fad14156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.917+0000 7fad19d44700 1 --2- 172.21.15.92:0/2605105805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad14154140 0x7fad140cfcb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fad04002950 tx=0x7fad04007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.918+0000 7fad0a7fc700 1 -- 172.21.15.92:0/2605105805 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad040033b0 con 0x7fad14154140 2023-10-06T10:39:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.918+0000 7fad0a7fc700 1 -- 172.21.15.92:0/2605105805 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad04002e90 con 0x7fad14154140 2023-10-06T10:39:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.918+0000 7fad1c7a9700 1 -- 172.21.15.92:0/2605105805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1416bcd0 con 0x7fad14154140 2023-10-06T10:39:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.918+0000 7fad0a7fc700 1 -- 172.21.15.92:0/2605105805 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad04006c80 con 0x7fad14154140 2023-10-06T10:39:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.918+0000 7fad1c7a9700 1 -- 172.21.15.92:0/2605105805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1416c290 con 0x7fad14154140 2023-10-06T10:39:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.919+0000 7fad1c7a9700 1 -- 172.21.15.92:0/2605105805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad14064a90 con 0x7fad14154140 2023-10-06T10:39:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.919+0000 7fad0a7fc700 1 -- 172.21.15.92:0/2605105805 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fad04007260 con 0x7fad14154140 2023-10-06T10:39:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.919+0000 7fad0a7fc700 1 --2- 172.21.15.92:0/2605105805 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7facf8066f30 0x7facf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.919+0000 7fad0a7fc700 1 -- 172.21.15.92:0/2605105805 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6953+0+0 (secure 0 0 0) 0x7fad04084ac0 con 0x7fad14154140 2023-10-06T10:39:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.919+0000 7fad1a545700 1 --2- 172.21.15.92:0/2605105805 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7facf8066f30 0x7facf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.920+0000 7fad1a545700 1 --2- 172.21.15.92:0/2605105805 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7facf8066f30 0x7facf80693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fad1000c670 tx=0x7fad10015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:04.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:04.924+0000 7fad0a7fc700 1 -- 172.21.15.92:0/2605105805 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad04051850 con 0x7fad14154140 2023-10-06T10:39:05.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.106+0000 7fad1c7a9700 1 -- 172.21.15.92:0/2605105805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fad1415bad0 con 0x7fad14154140 2023-10-06T10:39:05.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.778+0000 7fad0a7fc700 1 -- 172.21.15.92:0/2605105805 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v27) v1 ==== 181+0+0 (secure 0 0 0) 0x7fad04054e70 con 0x7fad14154140 2023-10-06T10:39:05.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.780+0000 7fad1c7a9700 1 -- 172.21.15.92:0/2605105805 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7facf8066f30 msgr2=0x7facf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:05.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.780+0000 7fad1c7a9700 1 --2- 172.21.15.92:0/2605105805 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7facf8066f30 0x7facf80693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fad1000c670 tx=0x7fad10015000 comp rx=0 tx=0).stop 2023-10-06T10:39:05.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.780+0000 7fad1c7a9700 1 -- 172.21.15.92:0/2605105805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad14154140 msgr2=0x7fad140cfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:05.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.780+0000 7fad1c7a9700 1 --2- 172.21.15.92:0/2605105805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad14154140 0x7fad140cfcb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fad04002950 tx=0x7fad04007d30 comp rx=0 tx=0).stop 2023-10-06T10:39:05.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.780+0000 7fad1c7a9700 1 -- 172.21.15.92:0/2605105805 shutdown_connections 2023-10-06T10:39:05.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.780+0000 7fad1c7a9700 1 --2- 172.21.15.92:0/2605105805 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7facf8066f30 0x7facf80693e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:05.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.781+0000 7fad1c7a9700 1 --2- 172.21.15.92:0/2605105805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad140d01f0 0x7fad14156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:05.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.781+0000 7fad1c7a9700 1 --2- 172.21.15.92:0/2605105805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad00003a60 0x7fad1413f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:05.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.781+0000 7fad1c7a9700 1 --2- 172.21.15.92:0/2605105805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad14154140 0x7fad140cfcb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:05.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.781+0000 7fad1c7a9700 1 -- 172.21.15.92:0/2605105805 >> 172.21.15.92:0/2605105805 conn(0x7fad1406ece0 msgr2=0x7fad1415c790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:05.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.781+0000 7fad1c7a9700 1 -- 172.21.15.92:0/2605105805 shutdown_connections 2023-10-06T10:39:05.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.781+0000 7fad1c7a9700 1 -- 172.21.15.92:0/2605105805 wait complete. 2023-10-06T10:39:05.784 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-10-06T10:39:05.795 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T10:39:05.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.913+0000 7fc1de9f1700 1 Processor -- start 2023-10-06T10:39:05.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.913+0000 7fc1de9f1700 1 -- start start 2023-10-06T10:39:05.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.913+0000 7fc1de9f1700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1d806ece0 0x7fc1d80b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:05.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.913+0000 7fc1de9f1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc1d806f0b0 0x7fc1d80b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:05.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.913+0000 7fc1de9f1700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d80b2ce0 con 0x7fc1d80aede0 2023-10-06T10:39:05.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.913+0000 7fc1de9f1700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1d806fb40 con 0x7fc1d806f0b0 2023-10-06T10:39:05.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.913+0000 7fc1de9f1700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d80b0410 con 0x7fc1d806ece0 2023-10-06T10:39:05.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.914+0000 7fc1d77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc1d806f0b0 0x7fc1d80b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:05.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.914+0000 7fc1d77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc1d806f0b0 0x7fc1d80b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46546/0 (socket says 172.21.15.92:46546) 2023-10-06T10:39:05.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.914+0000 7fc1d77fe700 1 -- 172.21.15.92:0/3127132317 learned_addr learned my addr 172.21.15.92:0/3127132317 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:05.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.914+0000 7fc1d7fff700 1 --2- 172.21.15.92:0/3127132317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1d806ece0 0x7fc1d80b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:05.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.914+0000 7fc1d77fe700 1 -- 172.21.15.92:0/3127132317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1d806ece0 msgr2=0x7fc1d80b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:05.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.914+0000 7fc1d77fe700 1 --2- 172.21.15.92:0/3127132317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1d806ece0 0x7fc1d80b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:05.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.914+0000 7fc1d77fe700 1 -- 172.21.15.92:0/3127132317 >> v1:172.21.15.124:6789/0 conn(0x7fc1d80aede0 legacy=0x7fc1d80af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:05.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.914+0000 7fc1d77fe700 1 -- 172.21.15.92:0/3127132317 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d80b2a00 con 0x7fc1d806f0b0 2023-10-06T10:39:05.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.914+0000 7fc1d7fff700 1 --2- 172.21.15.92:0/3127132317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1d806ece0 0x7fc1d80b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:05.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.914+0000 7fc1d77fe700 1 --2- 172.21.15.92:0/3127132317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1d806f0b0 0x7fc1d80b08a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0099e0 tx=0x7fc1cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1cc4b5f6fccfd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:05.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.914+0000 7fc1d6ffd700 1 -- 172.21.15.92:0/3127132317 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc00b7e0 con 0x7fc1d806f0b0 2023-10-06T10:39:05.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.914+0000 7fc1d6ffd700 1 -- 172.21.15.92:0/3127132317 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1cc00b940 con 0x7fc1d806f0b0 2023-10-06T10:39:05.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.914+0000 7fc1d6ffd700 1 -- 172.21.15.92:0/3127132317 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc00bce0 con 0x7fc1d806f0b0 2023-10-06T10:39:05.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.915+0000 7fc1de9f1700 1 -- 172.21.15.92:0/3127132317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1d806f0b0 msgr2=0x7fc1d80b08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:05.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.915+0000 7fc1de9f1700 1 --2- 172.21.15.92:0/3127132317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1d806f0b0 0x7fc1d80b08a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0099e0 tx=0x7fc1cc009250 comp rx=0 tx=0).stop 2023-10-06T10:39:05.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.915+0000 7fc1de9f1700 1 -- 172.21.15.92:0/3127132317 shutdown_connections 2023-10-06T10:39:05.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.915+0000 7fc1de9f1700 1 --2- 172.21.15.92:0/3127132317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1d806ece0 0x7fc1d80b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:05.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.915+0000 7fc1de9f1700 1 --2- 172.21.15.92:0/3127132317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1d806f0b0 0x7fc1d80b08a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:05.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.915+0000 7fc1de9f1700 1 -- 172.21.15.92:0/3127132317 >> 172.21.15.92:0/3127132317 conn(0x7fc1d80682f0 msgr2=0x7fc1d80686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:05.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.915+0000 7fc1de9f1700 1 -- 172.21.15.92:0/3127132317 shutdown_connections 2023-10-06T10:39:05.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.915+0000 7fc1de9f1700 1 -- 172.21.15.92:0/3127132317 wait complete. 2023-10-06T10:39:05.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.916+0000 7fc1de9f1700 1 Processor -- start 2023-10-06T10:39:05.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.916+0000 7fc1de9f1700 1 -- start start 2023-10-06T10:39:05.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.917+0000 7fc1de9f1700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1d806ece0 0x7fc1d81478f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:05.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.917+0000 7fc1de9f1700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1d806f0b0 0x7fc1d80b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:05.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.917+0000 7fc1de9f1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1d80aede0 0x7fc1d81496f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:05.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.917+0000 7fc1de9f1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d8149260 con 0x7fc1d80aede0 2023-10-06T10:39:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.917+0000 7fc1de9f1700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d8148840 con 0x7fc1d806f0b0 2023-10-06T10:39:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.917+0000 7fc1de9f1700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d8148280 con 0x7fc1d806ece0 2023-10-06T10:39:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.917+0000 7fc1dcf8e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1d80aede0 0x7fc1d81496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.917+0000 7fc1dcf8e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1d80aede0 0x7fc1d81496f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46552/0 (socket says 172.21.15.92:46552) 2023-10-06T10:39:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.917+0000 7fc1dcf8e700 1 -- 172.21.15.92:0/3188736933 learned_addr learned my addr 172.21.15.92:0/3188736933 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.917+0000 7fc1d77fe700 1 --2- 172.21.15.92:0/3188736933 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1d806f0b0 0x7fc1d80b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.917+0000 7fc1d7fff700 1 --2- 172.21.15.92:0/3188736933 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1d806ece0 0x7fc1d81478f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.917+0000 7fc1dcf8e700 1 -- 172.21.15.92:0/3188736933 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1d806ece0 msgr2=0x7fc1d81478f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.917+0000 7fc1dcf8e700 1 --2- 172.21.15.92:0/3188736933 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1d806ece0 0x7fc1d81478f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:05.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.917+0000 7fc1dcf8e700 1 -- 172.21.15.92:0/3188736933 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1d806f0b0 msgr2=0x7fc1d80b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:05.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.917+0000 7fc1dcf8e700 1 --2- 172.21.15.92:0/3188736933 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1d806f0b0 0x7fc1d80b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:05.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.917+0000 7fc1dcf8e700 1 -- 172.21.15.92:0/3188736933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1cc0094f0 con 0x7fc1d80aede0 2023-10-06T10:39:05.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.917+0000 7fc1dcf8e700 1 --2- 172.21.15.92:0/3188736933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1d80aede0 0x7fc1d81496f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc1d000c960 tx=0x7fc1d00026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:05.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.918+0000 7fc1bffff700 1 -- 172.21.15.92:0/3188736933 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0012630 con 0x7fc1d80aede0 2023-10-06T10:39:05.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.918+0000 7fc1bffff700 1 -- 172.21.15.92:0/3188736933 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d0012790 con 0x7fc1d80aede0 2023-10-06T10:39:05.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.918+0000 7fc1de9f1700 1 -- 172.21.15.92:0/3188736933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d8148560 con 0x7fc1d80aede0 2023-10-06T10:39:05.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.918+0000 7fc1bffff700 1 -- 172.21.15.92:0/3188736933 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0012b50 con 0x7fc1d80aede0 2023-10-06T10:39:05.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.918+0000 7fc1de9f1700 1 -- 172.21.15.92:0/3188736933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d8147e30 con 0x7fc1d80aede0 2023-10-06T10:39:05.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.919+0000 7fc1de9f1700 1 -- 172.21.15.92:0/3188736933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1d8149410 con 0x7fc1d80aede0 2023-10-06T10:39:05.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.919+0000 7fc1bffff700 1 -- 172.21.15.92:0/3188736933 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc1d001b400 con 0x7fc1d80aede0 2023-10-06T10:39:05.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.919+0000 7fc1bffff700 1 --2- 172.21.15.92:0/3188736933 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc1c4066f30 0x7fc1c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:05.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.919+0000 7fc1bffff700 1 -- 172.21.15.92:0/3188736933 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6515+0+0 (secure 0 0 0) 0x7fc1d0084b20 con 0x7fc1d80aede0 2023-10-06T10:39:05.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.919+0000 7fc1d7fff700 1 --2- 172.21.15.92:0/3188736933 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc1c4066f30 0x7fc1c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:05.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.920+0000 7fc1d7fff700 1 --2- 172.21.15.92:0/3188736933 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc1c4066f30 0x7fc1c40693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8006fd0 tx=0x7fc1c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:05.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:05.923+0000 7fc1bffff700 1 -- 172.21.15.92:0/3188736933 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1d8149410 con 0x7fc1d80aede0 2023-10-06T10:39:06.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.110+0000 7fc1de9f1700 1 -- 172.21.15.92:0/3188736933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc1d8149410 con 0x7fc1d80aede0 2023-10-06T10:39:06.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.780+0000 7fc1bffff700 1 -- 172.21.15.92:0/3188736933 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v28) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc1d8149410 con 0x7fc1d80aede0 2023-10-06T10:39:06.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.785+0000 7fc1de9f1700 1 -- 172.21.15.92:0/3188736933 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc1c4066f30 msgr2=0x7fc1c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:06.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.785+0000 7fc1de9f1700 1 --2- 172.21.15.92:0/3188736933 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc1c4066f30 0x7fc1c40693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8006fd0 tx=0x7fc1c8008040 comp rx=0 tx=0).stop 2023-10-06T10:39:06.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.785+0000 7fc1de9f1700 1 -- 172.21.15.92:0/3188736933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1d80aede0 msgr2=0x7fc1d81496f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:06.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.785+0000 7fc1de9f1700 1 --2- 172.21.15.92:0/3188736933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1d80aede0 0x7fc1d81496f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc1d000c960 tx=0x7fc1d00026e0 comp rx=0 tx=0).stop 2023-10-06T10:39:06.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.785+0000 7fc1de9f1700 1 -- 172.21.15.92:0/3188736933 shutdown_connections 2023-10-06T10:39:06.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.785+0000 7fc1de9f1700 1 --2- 172.21.15.92:0/3188736933 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc1c4066f30 0x7fc1c40693e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:06.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.785+0000 7fc1de9f1700 1 --2- 172.21.15.92:0/3188736933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1d80aede0 0x7fc1d81496f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:06.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.785+0000 7fc1de9f1700 1 --2- 172.21.15.92:0/3188736933 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1d806ece0 0x7fc1d81478f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:06.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.785+0000 7fc1de9f1700 1 --2- 172.21.15.92:0/3188736933 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1d806f0b0 0x7fc1d80b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:06.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.785+0000 7fc1de9f1700 1 -- 172.21.15.92:0/3188736933 >> 172.21.15.92:0/3188736933 conn(0x7fc1d80682f0 msgr2=0x7fc1d8156940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:06.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.785+0000 7fc1de9f1700 1 -- 172.21.15.92:0/3188736933 shutdown_connections 2023-10-06T10:39:06.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.785+0000 7fc1de9f1700 1 -- 172.21.15.92:0/3188736933 wait complete. 2023-10-06T10:39:06.788 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-10-06T10:39:06.801 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-10-06T10:39:06.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.921+0000 7f0fc4c6f700 1 Processor -- start 2023-10-06T10:39:06.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.921+0000 7f0fc4c6f700 1 -- start start 2023-10-06T10:39:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.922+0000 7f0fc4c6f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0fc00c3090 0x7f0fc00c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.922+0000 7f0fc4c6f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0fc00c4b00 0x7f0fc00cc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.922+0000 7f0fc4c6f700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc00c4130 con 0x7f0fc00b0720 2023-10-06T10:39:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.922+0000 7f0fc4c6f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc00b17c0 con 0x7f0fc00c3090 2023-10-06T10:39:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.922+0000 7f0fc4c6f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc00c3520 con 0x7f0fc00c4b00 2023-10-06T10:39:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.922+0000 7f0fb5d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0fc00c3090 0x7f0fc00c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.922+0000 7f0fb5d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0fc00c3090 0x7f0fc00c45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46564/0 (socket says 172.21.15.92:46564) 2023-10-06T10:39:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.922+0000 7f0fb5d9b700 1 -- 172.21.15.92:0/2863764166 learned_addr learned my addr 172.21.15.92:0/2863764166 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.922+0000 7f0fb5d9b700 1 -- 172.21.15.92:0/2863764166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0fc00c4b00 msgr2=0x7f0fc00cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.922+0000 7f0fbed9d700 1 --2- 172.21.15.92:0/2863764166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0fc00c4b00 0x7f0fc00cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.923+0000 7f0fb5d9b700 1 --2- 172.21.15.92:0/2863764166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0fc00c4b00 0x7f0fc00cc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.923+0000 7f0fb5d9b700 1 -- 172.21.15.92:0/2863764166 >> v1:172.21.15.124:6789/0 conn(0x7f0fc00b0720 legacy=0x7f0fc00b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.923+0000 7f0fb5d9b700 1 -- 172.21.15.92:0/2863764166 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc00c3800 con 0x7f0fc00c3090 2023-10-06T10:39:06.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.923+0000 7f0fb5d9b700 1 --2- 172.21.15.92:0/2863764166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fc00c3090 0x7f0fc00c45c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80099b0 tx=0x7f0fa8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3559ec79700141a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:06.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.923+0000 7f0fbdd9b700 1 -- 172.21.15.92:0/2863764166 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa800b8b0 con 0x7f0fc00c3090 2023-10-06T10:39:06.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.923+0000 7f0fbdd9b700 1 -- 172.21.15.92:0/2863764166 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa800ba10 con 0x7f0fc00c3090 2023-10-06T10:39:06.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.923+0000 7f0fbdd9b700 1 -- 172.21.15.92:0/2863764166 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa800bdb0 con 0x7f0fc00c3090 2023-10-06T10:39:06.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.923+0000 7f0fc4c6f700 1 -- 172.21.15.92:0/2863764166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fc00c3090 msgr2=0x7f0fc00c45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:06.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.923+0000 7f0fc4c6f700 1 --2- 172.21.15.92:0/2863764166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fc00c3090 0x7f0fc00c45c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80099b0 tx=0x7f0fa8009250 comp rx=0 tx=0).stop 2023-10-06T10:39:06.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.923+0000 7f0fc4c6f700 1 -- 172.21.15.92:0/2863764166 shutdown_connections 2023-10-06T10:39:06.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.923+0000 7f0fc4c6f700 1 --2- 172.21.15.92:0/2863764166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0fc00c4b00 0x7f0fc00cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:06.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.923+0000 7f0fc4c6f700 1 --2- 172.21.15.92:0/2863764166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fc00c3090 0x7f0fc00c45c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:06.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.923+0000 7f0fc4c6f700 1 -- 172.21.15.92:0/2863764166 >> 172.21.15.92:0/2863764166 conn(0x7f0fc0087fc0 msgr2=0x7f0fc00ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:06.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.923+0000 7f0fc4c6f700 1 -- 172.21.15.92:0/2863764166 shutdown_connections 2023-10-06T10:39:06.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.923+0000 7f0fc4c6f700 1 -- 172.21.15.92:0/2863764166 wait complete. 2023-10-06T10:39:06.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.925+0000 7f0fc4c6f700 1 Processor -- start 2023-10-06T10:39:06.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.925+0000 7f0fc4c6f700 1 -- start start 2023-10-06T10:39:06.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.925+0000 7f0fc4c6f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fc00b0720 0x7f0fc0156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:06.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.925+0000 7f0fc4c6f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0fc00c3090 0x7f0fc00c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:06.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.925+0000 7f0fc4c6f700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0fc00c4b00 0x7f0fc00cc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:06.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.925+0000 7f0fc4c6f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc00c84a0 con 0x7f0fc00b0720 2023-10-06T10:39:06.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.925+0000 7f0fc4c6f700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc0099220 con 0x7f0fc00c4b00 2023-10-06T10:39:06.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.925+0000 7f0fc4c6f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc00c7890 con 0x7f0fc00c3090 2023-10-06T10:39:06.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.925+0000 7f0fbe59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fc00b0720 0x7f0fc0156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:06.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.926+0000 7f0fbe59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fc00b0720 0x7f0fc0156350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46570/0 (socket says 172.21.15.92:46570) 2023-10-06T10:39:06.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.926+0000 7f0fbe59c700 1 -- 172.21.15.92:0/14905239 learned_addr learned my addr 172.21.15.92:0/14905239 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:06.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.926+0000 7f0fbed9d700 1 --2- 172.21.15.92:0/14905239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0fc00c4b00 0x7f0fc00cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:06.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.926+0000 7f0fb5d9b700 1 --2- 172.21.15.92:0/14905239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0fc00c3090 0x7f0fc00c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:06.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.926+0000 7f0fbe59c700 1 -- 172.21.15.92:0/14905239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0fc00c3090 msgr2=0x7f0fc00c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:06.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.926+0000 7f0fbe59c700 1 --2- 172.21.15.92:0/14905239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0fc00c3090 0x7f0fc00c6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:06.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.926+0000 7f0fbe59c700 1 -- 172.21.15.92:0/14905239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0fc00c4b00 msgr2=0x7f0fc00cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:06.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.926+0000 7f0fbe59c700 1 --2- 172.21.15.92:0/14905239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0fc00c4b00 0x7f0fc00cc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:06.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.926+0000 7f0fbe59c700 1 -- 172.21.15.92:0/14905239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa80094f0 con 0x7f0fc00b0720 2023-10-06T10:39:06.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.926+0000 7f0fbe59c700 1 --2- 172.21.15.92:0/14905239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fc00b0720 0x7f0fc0156350 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000b5e0 tx=0x7f0fb00090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:06.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.926+0000 7f0fb6ffd700 1 -- 172.21.15.92:0/14905239 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb000f510 con 0x7f0fc00b0720 2023-10-06T10:39:06.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.926+0000 7f0fb6ffd700 1 -- 172.21.15.92:0/14905239 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb000f670 con 0x7f0fc00b0720 2023-10-06T10:39:06.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.926+0000 7f0fc4c6f700 1 -- 172.21.15.92:0/14905239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc00c7b70 con 0x7f0fc00b0720 2023-10-06T10:39:06.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.926+0000 7f0fb6ffd700 1 -- 172.21.15.92:0/14905239 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb000fa30 con 0x7f0fc00b0720 2023-10-06T10:39:06.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.926+0000 7f0fc4c6f700 1 -- 172.21.15.92:0/14905239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc00c8130 con 0x7f0fc00b0720 2023-10-06T10:39:06.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.927+0000 7f0fc4c6f700 1 -- 172.21.15.92:0/14905239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc015d250 con 0x7f0fc00b0720 2023-10-06T10:39:06.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.927+0000 7f0fb6ffd700 1 -- 172.21.15.92:0/14905239 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0fb001a400 con 0x7f0fc00b0720 2023-10-06T10:39:06.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.928+0000 7f0fb6ffd700 1 --2- 172.21.15.92:0/14905239 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0fa0066f30 0x7f0fa00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:06.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.928+0000 7f0fb5d9b700 1 --2- 172.21.15.92:0/14905239 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0fa0066f30 0x7f0fa00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:06.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.928+0000 7f0fb6ffd700 1 -- 172.21.15.92:0/14905239 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6043+0+0 (secure 0 0 0) 0x7f0fb0083fd0 con 0x7f0fc00b0720 2023-10-06T10:39:06.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.928+0000 7f0fb5d9b700 1 --2- 172.21.15.92:0/14905239 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0fa0066f30 0x7f0fa00693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80099b0 tx=0x7f0fa8016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:06.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:06.932+0000 7f0fb6ffd700 1 -- 172.21.15.92:0/14905239 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0fb0051170 con 0x7f0fc00b0720 2023-10-06T10:39:07.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.115+0000 7f0fc4c6f700 1 -- 172.21.15.92:0/14905239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0fc0064a90 con 0x7f0fc00b0720 2023-10-06T10:39:07.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.792+0000 7f0fb6ffd700 1 -- 172.21.15.92:0/14905239 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v29) v1 ==== 178+0+0 (secure 0 0 0) 0x7f0fb0054790 con 0x7f0fc00b0720 2023-10-06T10:39:07.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.795+0000 7f0fc4c6f700 1 -- 172.21.15.92:0/14905239 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0fa0066f30 msgr2=0x7f0fa00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:07.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.795+0000 7f0fc4c6f700 1 --2- 172.21.15.92:0/14905239 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0fa0066f30 0x7f0fa00693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80099b0 tx=0x7f0fa8016000 comp rx=0 tx=0).stop 2023-10-06T10:39:07.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.795+0000 7f0fc4c6f700 1 -- 172.21.15.92:0/14905239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fc00b0720 msgr2=0x7f0fc0156350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:07.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.795+0000 7f0fc4c6f700 1 --2- 172.21.15.92:0/14905239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fc00b0720 0x7f0fc0156350 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000b5e0 tx=0x7f0fb00090f0 comp rx=0 tx=0).stop 2023-10-06T10:39:07.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.796+0000 7f0fc4c6f700 1 -- 172.21.15.92:0/14905239 shutdown_connections 2023-10-06T10:39:07.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.796+0000 7f0fc4c6f700 1 --2- 172.21.15.92:0/14905239 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0fa0066f30 0x7f0fa00693e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:07.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.796+0000 7f0fc4c6f700 1 --2- 172.21.15.92:0/14905239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0fc00c4b00 0x7f0fc00cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:07.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.796+0000 7f0fc4c6f700 1 --2- 172.21.15.92:0/14905239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fc00b0720 0x7f0fc0156350 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:07.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.796+0000 7f0fc4c6f700 1 --2- 172.21.15.92:0/14905239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0fc00c3090 0x7f0fc00c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:07.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.796+0000 7f0fc4c6f700 1 -- 172.21.15.92:0/14905239 >> 172.21.15.92:0/14905239 conn(0x7f0fc0087fc0 msgr2=0x7f0fc0156c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:07.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.796+0000 7f0fc4c6f700 1 -- 172.21.15.92:0/14905239 shutdown_connections 2023-10-06T10:39:07.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.796+0000 7f0fc4c6f700 1 -- 172.21.15.92:0/14905239 wait complete. 2023-10-06T10:39:07.799 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data_ec' removed 2023-10-06T10:39:07.811 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T10:39:07.811 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T10:39:07.811 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T10:39:07.812 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:39:07.812 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T10:39:07.812 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T10:39:07.812 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T10:39:07.812 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:39:07.812 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T10:39:07.812 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T10:39:07.813 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T10:39:07.813 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:39:07.813 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T10:39:07.813 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T10:39:07.813 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T10:39:07.813 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:39:07.820 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T10:39:07.821 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T10:39:07.822 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T10:39:07.822 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T10:39:07.823 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T10:39:07.823 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T10:39:07.824 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T10:39:07.824 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T10:39:07.863 INFO:tasks.ceph.mds.a:Started 2023-10-06T10:39:07.864 DEBUG:teuthology.parallel:result is None 2023-10-06T10:39:07.864 INFO:tasks.ceph.mds.c:Started 2023-10-06T10:39:07.864 DEBUG:teuthology.parallel:result is None 2023-10-06T10:39:07.866 INFO:tasks.ceph.mds.b:Started 2023-10-06T10:39:07.866 DEBUG:teuthology.parallel:result is None 2023-10-06T10:39:07.867 INFO:tasks.ceph.mds.d:Started 2023-10-06T10:39:07.867 DEBUG:teuthology.parallel:result is None 2023-10-06T10:39:07.868 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T10:39:07.937 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T10:39:07.937 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T10:39:07.945 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-06T10:39:07.945 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-06T10:39:07.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.985+0000 7f31f9985700 1 Processor -- start 2023-10-06T10:39:07.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.985+0000 7f31f9985700 1 -- start start 2023-10-06T10:39:07.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.986+0000 7f31f9985700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f31f4159560 0x7f31f40aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:07.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.986+0000 7f31f9985700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f31f40b0760 0x7f31f40b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:07.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.986+0000 7f31f9985700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f40b0c60 con 0x7f31f40b49a0 2023-10-06T10:39:07.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.986+0000 7f31f9985700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f40b0f40 con 0x7f31f4159560 2023-10-06T10:39:07.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.986+0000 7f31f9985700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f40b1220 con 0x7f31f40b0760 2023-10-06T10:39:07.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.986+0000 7f31f2ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f31f4159560 0x7f31f40aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:07.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.986+0000 7f31ebfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f31f40b49a0 0x7f31f40aaa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33118/0 (socket says 172.21.15.92:33118) 2023-10-06T10:39:07.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.986+0000 7f31ebfff700 1 -- 172.21.15.92:0/3413162951 learned_addr learned my addr 172.21.15.92:0/3413162951 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:07.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.986+0000 7f31f37fe700 1 --2- 172.21.15.92:0/3413162951 >> v2:172.21.15.124:3300/0 conn(0x7f31f40b0760 0x7f31f40b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:07.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.987+0000 7f31f2ffd700 1 -- 172.21.15.92:0/3413162951 >> v2:172.21.15.124:3300/0 conn(0x7f31f40b0760 msgr2=0x7f31f40b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:07.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.987+0000 7f31f2ffd700 1 --2- 172.21.15.92:0/3413162951 >> v2:172.21.15.124:3300/0 conn(0x7f31f40b0760 0x7f31f40b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:07.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.987+0000 7f31f2ffd700 1 -- 172.21.15.92:0/3413162951 >> v1:172.21.15.92:6789/0 conn(0x7f31f40b49a0 legacy=0x7f31f40aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:07.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.987+0000 7f31f2ffd700 1 -- 172.21.15.92:0/3413162951 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f40b1500 con 0x7f31f4159560 2023-10-06T10:39:07.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.987+0000 7f31f27fc700 1 -- 172.21.15.92:0/3413162951 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2501885697 0 0) 0x7f31f40b0c60 con 0x7f31f40b49a0 2023-10-06T10:39:07.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.987+0000 7f31f37fe700 1 --2- 172.21.15.92:0/3413162951 >> v2:172.21.15.124:3300/0 conn(0x7f31f40b0760 0x7f31f40b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:07.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.987+0000 7f31f2ffd700 1 --2- 172.21.15.92:0/3413162951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31f4159560 0x7f31f40aa530 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f31e40098d0 tx=0x7f31e4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d265d5b96dd11c6d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:07.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.987+0000 7f31f27fc700 1 -- 172.21.15.92:0/3413162951 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e4009e70 con 0x7f31f4159560 2023-10-06T10:39:07.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.987+0000 7f31f27fc700 1 -- 172.21.15.92:0/3413162951 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e4005c10 con 0x7f31f4159560 2023-10-06T10:39:07.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.987+0000 7f31f27fc700 1 -- 172.21.15.92:0/3413162951 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e400b610 con 0x7f31f4159560 2023-10-06T10:39:07.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.987+0000 7f31f9985700 1 -- 172.21.15.92:0/3413162951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31f4159560 msgr2=0x7f31f40aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:07.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.987+0000 7f31f9985700 1 --2- 172.21.15.92:0/3413162951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31f4159560 0x7f31f40aa530 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f31e40098d0 tx=0x7f31e4009120 comp rx=0 tx=0).stop 2023-10-06T10:39:07.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.987+0000 7f31f9985700 1 -- 172.21.15.92:0/3413162951 shutdown_connections 2023-10-06T10:39:07.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.987+0000 7f31f9985700 1 --2- 172.21.15.92:0/3413162951 >> v2:172.21.15.124:3300/0 conn(0x7f31f40b0760 0x7f31f40b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:07.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.987+0000 7f31f9985700 1 --2- 172.21.15.92:0/3413162951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31f4159560 0x7f31f40aa530 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:07.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.987+0000 7f31f9985700 1 -- 172.21.15.92:0/3413162951 >> 172.21.15.92:0/3413162951 conn(0x7f31f40997f0 msgr2=0x7f31f406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:07.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.987+0000 7f31f9985700 1 -- 172.21.15.92:0/3413162951 shutdown_connections 2023-10-06T10:39:07.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.987+0000 7f31f9985700 1 -- 172.21.15.92:0/3413162951 wait complete. 2023-10-06T10:39:07.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.988+0000 7f31f9985700 1 Processor -- start 2023-10-06T10:39:07.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.989+0000 7f31f9985700 1 -- start start 2023-10-06T10:39:07.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.989+0000 7f31f9985700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31f40b0760 0x7f31f4147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:07.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.989+0000 7f31f9985700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31f40b49a0 0x7f31f415ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:07.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.989+0000 7f31f9985700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31f4159560 0x7f31f415bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:07.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.989+0000 7f31f9985700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f40aef80 con 0x7f31f40b49a0 2023-10-06T10:39:07.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.989+0000 7f31f9985700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f40ae370 con 0x7f31f40b0760 2023-10-06T10:39:07.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.989+0000 7f31f9985700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f40ae650 con 0x7f31f4159560 2023-10-06T10:39:07.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.989+0000 7f31ebfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31f40b49a0 0x7f31f415ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:07.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.989+0000 7f31ebfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31f40b49a0 0x7f31f415ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46602/0 (socket says 172.21.15.92:46602) 2023-10-06T10:39:07.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.989+0000 7f31ebfff700 1 -- 172.21.15.92:0/487997498 learned_addr learned my addr 172.21.15.92:0/487997498 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:07.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.989+0000 7f31f37fe700 1 --2- 172.21.15.92:0/487997498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31f4159560 0x7f31f415bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:07.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.989+0000 7f31f2ffd700 1 --2- 172.21.15.92:0/487997498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31f40b0760 0x7f31f4147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:07.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.990+0000 7f31ebfff700 1 -- 172.21.15.92:0/487997498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31f4159560 msgr2=0x7f31f415bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:07.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.990+0000 7f31ebfff700 1 --2- 172.21.15.92:0/487997498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31f4159560 0x7f31f415bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:07.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.990+0000 7f31ebfff700 1 -- 172.21.15.92:0/487997498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31f40b0760 msgr2=0x7f31f4147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:07.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.990+0000 7f31ebfff700 1 --2- 172.21.15.92:0/487997498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31f40b0760 0x7f31f4147210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:07.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.990+0000 7f31ebfff700 1 -- 172.21.15.92:0/487997498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e40093c0 con 0x7f31f40b49a0 2023-10-06T10:39:07.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.990+0000 7f31f2ffd700 1 --2- 172.21.15.92:0/487997498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31f40b0760 0x7f31f4147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:07.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.990+0000 7f31ebfff700 1 --2- 172.21.15.92:0/487997498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31f40b49a0 0x7f31f415ba80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00c7e0 tx=0x7f31dc007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:07.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.990+0000 7f31eaffd700 1 -- 172.21.15.92:0/487997498 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc005c00 con 0x7f31f40b49a0 2023-10-06T10:39:07.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.990+0000 7f31eaffd700 1 -- 172.21.15.92:0/487997498 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31dc005d60 con 0x7f31f40b49a0 2023-10-06T10:39:07.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.990+0000 7f31f9985700 1 -- 172.21.15.92:0/487997498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f40ae930 con 0x7f31f40b49a0 2023-10-06T10:39:07.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.990+0000 7f31eaffd700 1 -- 172.21.15.92:0/487997498 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc0072e0 con 0x7f31f40b49a0 2023-10-06T10:39:07.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.990+0000 7f31f9985700 1 -- 172.21.15.92:0/487997498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f4147750 con 0x7f31f40b49a0 2023-10-06T10:39:07.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.991+0000 7f31f9985700 1 -- 172.21.15.92:0/487997498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f4064a90 con 0x7f31f40b49a0 2023-10-06T10:39:07.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.991+0000 7f31eaffd700 1 -- 172.21.15.92:0/487997498 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f31dc01b3b0 con 0x7f31f40b49a0 2023-10-06T10:39:07.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.992+0000 7f31eaffd700 1 --2- 172.21.15.92:0/487997498 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f31d4066f30 0x7f31d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:07.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.992+0000 7f31f2ffd700 1 --2- 172.21.15.92:0/487997498 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f31d4066f30 0x7f31d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:07.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.992+0000 7f31eaffd700 1 -- 172.21.15.92:0/487997498 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f31dc0838a0 con 0x7f31f40b49a0 2023-10-06T10:39:07.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.992+0000 7f31f2ffd700 1 --2- 172.21.15.92:0/487997498 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f31d4066f30 0x7f31d40693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f31e4009160 tx=0x7f31e4016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:07.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:07.996+0000 7f31eaffd700 1 -- 172.21.15.92:0/487997498 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31dc04b5a0 con 0x7f31f40b49a0 2023-10-06T10:39:08.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:08.196+0000 7f31f9985700 1 -- 172.21.15.92:0/487997498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f31f40af0c0 con 0x7f31f40b49a0 2023-10-06T10:39:09.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.307+0000 7f31eaffd700 1 -- 172.21.15.92:0/487997498 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) v1 ==== 97+0+0 (secure 0 0 0) 0x7f31dc017020 con 0x7f31f40b49a0 2023-10-06T10:39:09.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.310+0000 7f31f9985700 1 -- 172.21.15.92:0/487997498 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f31d4066f30 msgr2=0x7f31d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:09.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.310+0000 7f31f9985700 1 --2- 172.21.15.92:0/487997498 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f31d4066f30 0x7f31d40693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f31e4009160 tx=0x7f31e4016000 comp rx=0 tx=0).stop 2023-10-06T10:39:09.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.310+0000 7f31f9985700 1 -- 172.21.15.92:0/487997498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31f40b49a0 msgr2=0x7f31f415ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:09.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.310+0000 7f31f9985700 1 --2- 172.21.15.92:0/487997498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31f40b49a0 0x7f31f415ba80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00c7e0 tx=0x7f31dc007d90 comp rx=0 tx=0).stop 2023-10-06T10:39:09.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.310+0000 7f31f9985700 1 -- 172.21.15.92:0/487997498 shutdown_connections 2023-10-06T10:39:09.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.310+0000 7f31f9985700 1 --2- 172.21.15.92:0/487997498 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f31d4066f30 0x7f31d40693e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:09.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.310+0000 7f31f9985700 1 --2- 172.21.15.92:0/487997498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31f4159560 0x7f31f415bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:09.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.310+0000 7f31f9985700 1 --2- 172.21.15.92:0/487997498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31f40b0760 0x7f31f4147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:09.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.310+0000 7f31f9985700 1 --2- 172.21.15.92:0/487997498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31f40b49a0 0x7f31f415ba80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:09.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.310+0000 7f31f9985700 1 -- 172.21.15.92:0/487997498 >> 172.21.15.92:0/487997498 conn(0x7f31f40997f0 msgr2=0x7f31f40683c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:09.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.310+0000 7f31f9985700 1 -- 172.21.15.92:0/487997498 shutdown_connections 2023-10-06T10:39:09.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.310+0000 7f31f9985700 1 -- 172.21.15.92:0/487997498 wait complete. 2023-10-06T10:39:09.313 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-10-06T10:39:09.325 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T10:39:09.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.441+0000 7f1d3f69c700 1 Processor -- start 2023-10-06T10:39:09.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.441+0000 7f1d3f69c700 1 -- start start 2023-10-06T10:39:09.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.442+0000 7f1d3f69c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1d38159560 0x7f1d380aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:09.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.442+0000 7f1d3f69c700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1d380b49a0 0x7f1d380aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:09.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.442+0000 7f1d3f69c700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d380b0c60 con 0x7f1d380b0760 2023-10-06T10:39:09.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.442+0000 7f1d3f69c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d380b0f40 con 0x7f1d38159560 2023-10-06T10:39:09.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.442+0000 7f1d3f69c700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d380b1220 con 0x7f1d380b49a0 2023-10-06T10:39:09.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.442+0000 7f1d3d438700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1d38159560 0x7f1d380aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:09.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.442+0000 7f1d3d438700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1d38159560 0x7f1d380aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46618/0 (socket says 172.21.15.92:46618) 2023-10-06T10:39:09.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.442+0000 7f1d3d438700 1 -- 172.21.15.92:0/4093634640 learned_addr learned my addr 172.21.15.92:0/4093634640 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:09.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.442+0000 7f1d3cc37700 1 --2- 172.21.15.92:0/4093634640 >> v2:172.21.15.124:3300/0 conn(0x7f1d380b49a0 0x7f1d380aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:09.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.442+0000 7f1d3d438700 1 -- 172.21.15.92:0/4093634640 >> v2:172.21.15.124:3300/0 conn(0x7f1d380b49a0 msgr2=0x7f1d380aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:09.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.442+0000 7f1d3d438700 1 --2- 172.21.15.92:0/4093634640 >> v2:172.21.15.124:3300/0 conn(0x7f1d380b49a0 0x7f1d380aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:09.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.442+0000 7f1d3d438700 1 -- 172.21.15.92:0/4093634640 >> v1:172.21.15.124:6789/0 conn(0x7f1d380b0760 legacy=0x7f1d380b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:09.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.442+0000 7f1d3d438700 1 -- 172.21.15.92:0/4093634640 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d380b1500 con 0x7f1d38159560 2023-10-06T10:39:09.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.442+0000 7f1d3cc37700 1 --2- 172.21.15.92:0/4093634640 >> v2:172.21.15.124:3300/0 conn(0x7f1d380b49a0 0x7f1d380aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:09.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.442+0000 7f1d3d438700 1 --2- 172.21.15.92:0/4093634640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d38159560 0x7f1d380aa530 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009990 tx=0x7f1d34009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb3f4255ff78b5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:09.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.442+0000 7f1d2ffff700 1 -- 172.21.15.92:0/4093634640 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34005c10 con 0x7f1d38159560 2023-10-06T10:39:09.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.443+0000 7f1d2ffff700 1 -- 172.21.15.92:0/4093634640 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d34005d70 con 0x7f1d38159560 2023-10-06T10:39:09.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.443+0000 7f1d2ffff700 1 -- 172.21.15.92:0/4093634640 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3400b800 con 0x7f1d38159560 2023-10-06T10:39:09.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.443+0000 7f1d3f69c700 1 -- 172.21.15.92:0/4093634640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d38159560 msgr2=0x7f1d380aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:09.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.443+0000 7f1d3f69c700 1 --2- 172.21.15.92:0/4093634640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d38159560 0x7f1d380aa530 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009990 tx=0x7f1d34009220 comp rx=0 tx=0).stop 2023-10-06T10:39:09.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.443+0000 7f1d3f69c700 1 -- 172.21.15.92:0/4093634640 shutdown_connections 2023-10-06T10:39:09.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.443+0000 7f1d3f69c700 1 --2- 172.21.15.92:0/4093634640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d38159560 0x7f1d380aa530 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:09.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.443+0000 7f1d3f69c700 1 --2- 172.21.15.92:0/4093634640 >> v2:172.21.15.124:3300/0 conn(0x7f1d380b49a0 0x7f1d380aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:09.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.443+0000 7f1d3f69c700 1 -- 172.21.15.92:0/4093634640 >> 172.21.15.92:0/4093634640 conn(0x7f1d380997f0 msgr2=0x7f1d3806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:09.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.443+0000 7f1d3f69c700 1 -- 172.21.15.92:0/4093634640 shutdown_connections 2023-10-06T10:39:09.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.443+0000 7f1d3f69c700 1 -- 172.21.15.92:0/4093634640 wait complete. 2023-10-06T10:39:09.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.444+0000 7f1d3f69c700 1 Processor -- start 2023-10-06T10:39:09.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.444+0000 7f1d3f69c700 1 -- start start 2023-10-06T10:39:09.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.445+0000 7f1d3f69c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d380b0760 0x7f1d380aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:09.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.445+0000 7f1d3f69c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d380b49a0 0x7f1d3809a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:09.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.445+0000 7f1d3f69c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d38159560 0x7f1d3809a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:09.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.445+0000 7f1d3f69c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3809dd00 con 0x7f1d38159560 2023-10-06T10:39:09.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.445+0000 7f1d3f69c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d380cdd40 con 0x7f1d380b0760 2023-10-06T10:39:09.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.445+0000 7f1d3f69c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3809d740 con 0x7f1d380b49a0 2023-10-06T10:39:09.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.445+0000 7f1d3dc39700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d38159560 0x7f1d3809a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:09.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.445+0000 7f1d3dc39700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d38159560 0x7f1d3809a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46634/0 (socket says 172.21.15.92:46634) 2023-10-06T10:39:09.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.445+0000 7f1d3dc39700 1 -- 172.21.15.92:0/3853209116 learned_addr learned my addr 172.21.15.92:0/3853209116 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:09.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.445+0000 7f1d3d438700 1 --2- 172.21.15.92:0/3853209116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d380b0760 0x7f1d380aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:09.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.445+0000 7f1d3cc37700 1 --2- 172.21.15.92:0/3853209116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d380b49a0 0x7f1d3809a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:09.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.446+0000 7f1d3dc39700 1 -- 172.21.15.92:0/3853209116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d380b49a0 msgr2=0x7f1d3809a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:09.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.446+0000 7f1d3dc39700 1 --2- 172.21.15.92:0/3853209116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d380b49a0 0x7f1d3809a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:09.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.446+0000 7f1d3dc39700 1 -- 172.21.15.92:0/3853209116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d380b0760 msgr2=0x7f1d380aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:09.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.446+0000 7f1d3dc39700 1 --2- 172.21.15.92:0/3853209116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d380b0760 0x7f1d380aeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:09.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.446+0000 7f1d3dc39700 1 -- 172.21.15.92:0/3853209116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d34009480 con 0x7f1d38159560 2023-10-06T10:39:09.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.446+0000 7f1d3dc39700 1 --2- 172.21.15.92:0/3853209116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d38159560 0x7f1d3809a590 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000c960 tx=0x7f1d300026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:09.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.446+0000 7f1d2d7fa700 1 -- 172.21.15.92:0/3853209116 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30012630 con 0x7f1d38159560 2023-10-06T10:39:09.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.446+0000 7f1d2d7fa700 1 -- 172.21.15.92:0/3853209116 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d30012790 con 0x7f1d38159560 2023-10-06T10:39:09.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.446+0000 7f1d3f69c700 1 -- 172.21.15.92:0/3853209116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d3809da20 con 0x7f1d38159560 2023-10-06T10:39:09.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.446+0000 7f1d2d7fa700 1 -- 172.21.15.92:0/3853209116 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30012b50 con 0x7f1d38159560 2023-10-06T10:39:09.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.446+0000 7f1d3f69c700 1 -- 172.21.15.92:0/3853209116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d3809caa0 con 0x7f1d38159560 2023-10-06T10:39:09.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.447+0000 7f1d3f69c700 1 -- 172.21.15.92:0/3853209116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d380cded0 con 0x7f1d38159560 2023-10-06T10:39:09.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.447+0000 7f1d2d7fa700 1 -- 172.21.15.92:0/3853209116 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1d3001b400 con 0x7f1d38159560 2023-10-06T10:39:09.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.448+0000 7f1d2d7fa700 1 --2- 172.21.15.92:0/3853209116 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1d1c066f30 0x7f1d1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:09.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.448+0000 7f1d2d7fa700 1 -- 172.21.15.92:0/3853209116 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f1d300846f0 con 0x7f1d38159560 2023-10-06T10:39:09.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.448+0000 7f1d3d438700 1 --2- 172.21.15.92:0/3853209116 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1d1c066f30 0x7f1d1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:09.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.448+0000 7f1d3d438700 1 --2- 172.21.15.92:0/3853209116 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1d1c066f30 0x7f1d1c0693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1d34007fd0 tx=0x7f1d34015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:09.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.452+0000 7f1d2d7fa700 1 -- 172.21.15.92:0/3853209116 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d300554a0 con 0x7f1d38159560 2023-10-06T10:39:09.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.713+0000 7f1d3f69c700 1 -- 172.21.15.92:0/3853209116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1d38064a90 con 0x7f1d38159560 2023-10-06T10:39:09.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.713+0000 7f1d2d7fa700 1 -- 172.21.15.92:0/3853209116 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f1d30055030 con 0x7f1d38159560 2023-10-06T10:39:09.715 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:39:09.715 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:39:09.715 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-10-06T10:39:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-10-06T10:39:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lc28gCBAAVrjhXIawHuZEox0PE3/Rsg==", 2023-10-06T10:39:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:39:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:39:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:39:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:39:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-10-06T10:39:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lQpdOExAAm44s0n4Jekhy8AJk/o6Waw==", 2023-10-06T10:39:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:39:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:39:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:39:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:39:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-10-06T10:39:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lrCrsDhAAvp++wvvnSpZRGzvRTrGWEw==", 2023-10-06T10:39:09.717 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:39:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:39:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:39:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:39:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-10-06T10:39:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9l/DgWGhAAy3JMA5ntn/UaRkuTCyPdkg==", 2023-10-06T10:39:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:39:09.718 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:39:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:39:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:39:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-10-06T10:39:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBo4x9lDEIFMBAAc3DY1fYpY2UiY7MX7qSP5A==", 2023-10-06T10:39:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:39:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:39:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:39:09.719 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-10-06T10:39:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBr4x9lKzX8LRAApryLtFGkK6urkE8W70l5Cg==", 2023-10-06T10:39:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:39:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:39:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:39:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-10-06T10:39:09.720 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBu4x9lCMZpLhAAEBPa40Wnb820p6j+lv+PsA==", 2023-10-06T10:39:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:39:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:39:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:39:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-10-06T10:39:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBx4x9l4AHrLRAAFn6SnNkHnOiVXmmiU8qGyA==", 2023-10-06T10:39:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:39:09.721 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:39:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:39:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-10-06T10:39:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB34x9l7V7WMBAAvs5kvBP5n0Th478l/idjeQ==", 2023-10-06T10:39:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:39:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:39:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:39:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-10-06T10:39:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB64x9lfHIlLhAADCuqt+O/zPEETgoj//hwVg==", 2023-10-06T10:39:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:39:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:39:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:39:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-10-06T10:39:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB94x9lUmU7LhAAvxmtGmJxrI8TlOOD7taHOQ==", 2023-10-06T10:39:09.723 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:39:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:39:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:39:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-10-06T10:39:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQCA4x9lJE3nLBAAnREk9yNK+FhX4fnG+Za5Ug==", 2023-10-06T10:39:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:39:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:39:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:39:09.724 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-10-06T10:39:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lMePcHRAAMyftXWJWk7rwmWFkq18Rqg==", 2023-10-06T10:39:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:39:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T10:39:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T10:39:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T10:39:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-10-06T10:39:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lp1s0JhAAKvZ23MYjObxOK83q3UqJBQ==", 2023-10-06T10:39:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:39:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T10:39:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T10:39:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T10:39:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-10-06T10:39:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lc6o9IxAAaOY7S5rYn+OdeMNWK1UYMA==", 2023-10-06T10:39:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:39:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T10:39:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T10:39:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T10:39:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-10-06T10:39:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lwoyOKxAA811lehuQQ8MQQQzuTFAApg==", 2023-10-06T10:39:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.727 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:39:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T10:39:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T10:39:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T10:39:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-10-06T10:39:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBk4x9lONCoJhAAepF/sou/5M2GBvvfNAs5wQ==", 2023-10-06T10:39:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.728 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T10:39:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:39:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:39:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:39:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-mds", 2023-10-06T10:39:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQCG4x9l5rKkHxAAI56v4J5UGSYUi16Sqbh5Lw==", 2023-10-06T10:39:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-mds" 2023-10-06T10:39:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-mgr", 2023-10-06T10:39:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQCG4x9llfSkHxAA0B2celAtXRwVAZPyjAsy8w==", 2023-10-06T10:39:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-06T10:39:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-osd", 2023-10-06T10:39:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQCG4x9lI/6kHxAAPbYN1apIVjndpVZsxdmpFA==", 2023-10-06T10:39:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.730 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-osd" 2023-10-06T10:39:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-rbd", 2023-10-06T10:39:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQCG4x9lrAWlHxAAVuYQaAOnGliaW//1Na6v+g==", 2023-10-06T10:39:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-06T10:39:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-06T10:39:09.731 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQCG4x9l2QylHxAAdA704Nn36RS4IJMBNRjIDQ==", 2023-10-06T10:39:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-06T10:39:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-rgw", 2023-10-06T10:39:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQCG4x9lEBSlHxAAuEc7cXEFRbQin6U9u5NmCQ==", 2023-10-06T10:39:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-06T10:39:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-10-06T10:39:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9ll8wgBRAAQUrHERQ3Igv1tosa+wryIg==", 2023-10-06T10:39:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T10:39:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T10:39:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:39:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-10-06T10:39:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBk4x9lgYa/OhAAyoI717Ga+Wv72srZ4FK+lQ==", 2023-10-06T10:39:09.733 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:39:09.734 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T10:39:09.734 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T10:39:09.734 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:39:09.734 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.734 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:09.734 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T10:39:09.734 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:39:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.715+0000 7f1d3f69c700 1 -- 172.21.15.92:0/3853209116 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1d1c066f30 msgr2=0x7f1d1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.715+0000 7f1d3f69c700 1 --2- 172.21.15.92:0/3853209116 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1d1c066f30 0x7f1d1c0693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1d34007fd0 tx=0x7f1d34015000 comp rx=0 tx=0).stop 2023-10-06T10:39:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.715+0000 7f1d3f69c700 1 -- 172.21.15.92:0/3853209116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d38159560 msgr2=0x7f1d3809a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.715+0000 7f1d3f69c700 1 --2- 172.21.15.92:0/3853209116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d38159560 0x7f1d3809a590 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000c960 tx=0x7f1d300026e0 comp rx=0 tx=0).stop 2023-10-06T10:39:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.716+0000 7f1d3f69c700 1 -- 172.21.15.92:0/3853209116 shutdown_connections 2023-10-06T10:39:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.716+0000 7f1d3f69c700 1 --2- 172.21.15.92:0/3853209116 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1d1c066f30 0x7f1d1c0693e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.716+0000 7f1d3f69c700 1 --2- 172.21.15.92:0/3853209116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d38159560 0x7f1d3809a590 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.716+0000 7f1d3f69c700 1 --2- 172.21.15.92:0/3853209116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d380b0760 0x7f1d380aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.716+0000 7f1d3f69c700 1 --2- 172.21.15.92:0/3853209116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d380b49a0 0x7f1d3809a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.716+0000 7f1d3f69c700 1 -- 172.21.15.92:0/3853209116 >> 172.21.15.92:0/3853209116 conn(0x7f1d380997f0 msgr2=0x7f1d380cc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:09.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.716+0000 7f1d3f69c700 1 -- 172.21.15.92:0/3853209116 shutdown_connections 2023-10-06T10:39:09.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.716+0000 7f1d3f69c700 1 -- 172.21.15.92:0/3853209116 wait complete. 2023-10-06T10:39:09.737 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-06T10:39:09.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.851+0000 7f260a29a700 1 Processor -- start 2023-10-06T10:39:09.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.851+0000 7f260a29a700 1 -- start start 2023-10-06T10:39:09.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.852+0000 7f260a29a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2604159560 0x7f26040aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:09.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.852+0000 7f260a29a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f26040b49a0 0x7f26040aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:09.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.852+0000 7f260a29a700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f26040b0760 0x7f26040b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:09.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.852+0000 7f260a29a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26040b0c60 con 0x7f26040b49a0 2023-10-06T10:39:09.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.852+0000 7f260a29a700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26040b0f40 con 0x7f26040b0760 2023-10-06T10:39:09.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.852+0000 7f260a29a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26040b1220 con 0x7f2604159560 2023-10-06T10:39:09.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.852+0000 7f26037fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f26040b49a0 0x7f26040aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:09.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.852+0000 7f26037fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f26040b49a0 0x7f26040aaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46644/0 (socket says 172.21.15.92:46644) 2023-10-06T10:39:09.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.852+0000 7f26037fe700 1 -- 172.21.15.92:0/4074635916 learned_addr learned my addr 172.21.15.92:0/4074635916 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:09.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.853+0000 7f2603fff700 1 --2- 172.21.15.92:0/4074635916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2604159560 0x7f26040aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:09.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.853+0000 7f2608837700 1 --2- 172.21.15.92:0/4074635916 >> v2:172.21.15.124:3300/0 conn(0x7f26040b0760 0x7f26040b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:09.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.853+0000 7f26037fe700 1 -- 172.21.15.92:0/4074635916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2604159560 msgr2=0x7f26040aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:09.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.853+0000 7f26037fe700 1 --2- 172.21.15.92:0/4074635916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2604159560 0x7f26040aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:09.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.853+0000 7f26037fe700 1 -- 172.21.15.92:0/4074635916 >> v2:172.21.15.124:3300/0 conn(0x7f26040b0760 msgr2=0x7f26040b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:09.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.853+0000 7f26037fe700 1 --2- 172.21.15.92:0/4074635916 >> v2:172.21.15.124:3300/0 conn(0x7f26040b0760 0x7f26040b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:09.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.853+0000 7f26037fe700 1 -- 172.21.15.92:0/4074635916 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26040b1500 con 0x7f26040b49a0 2023-10-06T10:39:09.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.853+0000 7f2608837700 1 --2- 172.21.15.92:0/4074635916 >> v2:172.21.15.124:3300/0 conn(0x7f26040b0760 0x7f26040b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:09.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.853+0000 7f26037fe700 1 --2- 172.21.15.92:0/4074635916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26040b49a0 0x7f26040aaa70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f26040b2f10 tx=0x7f25f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad183e89711d085 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:09.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.853+0000 7f2602ffd700 1 -- 172.21.15.92:0/4074635916 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f800b860 con 0x7f26040b49a0 2023-10-06T10:39:09.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.853+0000 7f2602ffd700 1 -- 172.21.15.92:0/4074635916 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f800b9c0 con 0x7f26040b49a0 2023-10-06T10:39:09.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.853+0000 7f2602ffd700 1 -- 172.21.15.92:0/4074635916 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f800bd60 con 0x7f26040b49a0 2023-10-06T10:39:09.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.853+0000 7f260a29a700 1 -- 172.21.15.92:0/4074635916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26040b49a0 msgr2=0x7f26040aaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:09.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.853+0000 7f260a29a700 1 --2- 172.21.15.92:0/4074635916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26040b49a0 0x7f26040aaa70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f26040b2f10 tx=0x7f25f8009250 comp rx=0 tx=0).stop 2023-10-06T10:39:09.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.853+0000 7f260a29a700 1 -- 172.21.15.92:0/4074635916 shutdown_connections 2023-10-06T10:39:09.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.853+0000 7f260a29a700 1 --2- 172.21.15.92:0/4074635916 >> v2:172.21.15.124:3300/0 conn(0x7f26040b0760 0x7f26040b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:09.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.853+0000 7f260a29a700 1 --2- 172.21.15.92:0/4074635916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2604159560 0x7f26040aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:09.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.853+0000 7f260a29a700 1 --2- 172.21.15.92:0/4074635916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26040b49a0 0x7f26040aaa70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:09.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.853+0000 7f260a29a700 1 -- 172.21.15.92:0/4074635916 >> 172.21.15.92:0/4074635916 conn(0x7f26040997f0 msgr2=0x7f260406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:09.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.854+0000 7f260a29a700 1 -- 172.21.15.92:0/4074635916 shutdown_connections 2023-10-06T10:39:09.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.854+0000 7f260a29a700 1 -- 172.21.15.92:0/4074635916 wait complete. 2023-10-06T10:39:09.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.855+0000 7f260a29a700 1 Processor -- start 2023-10-06T10:39:09.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.855+0000 7f260a29a700 1 -- start start 2023-10-06T10:39:09.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.855+0000 7f260a29a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26040b0760 0x7f260416d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:09.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.855+0000 7f260a29a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26040b49a0 0x7f260409d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:09.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.856+0000 7f260a29a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2604159560 0x7f26040aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:09.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.856+0000 7f260a29a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260415bb90 con 0x7f2604159560 2023-10-06T10:39:09.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.856+0000 7f260a29a700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260416dc60 con 0x7f26040b0760 2023-10-06T10:39:09.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.856+0000 7f260a29a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260415af80 con 0x7f26040b49a0 2023-10-06T10:39:09.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.856+0000 7f2608837700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2604159560 0x7f26040aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:09.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.856+0000 7f2608837700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2604159560 0x7f26040aec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46650/0 (socket says 172.21.15.92:46650) 2023-10-06T10:39:09.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.856+0000 7f2608837700 1 -- 172.21.15.92:0/1343087178 learned_addr learned my addr 172.21.15.92:0/1343087178 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:09.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.856+0000 7f2603fff700 1 --2- 172.21.15.92:0/1343087178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26040b0760 0x7f260416d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:09.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.856+0000 7f26037fe700 1 --2- 172.21.15.92:0/1343087178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26040b49a0 0x7f260409d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:09.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.856+0000 7f2608837700 1 -- 172.21.15.92:0/1343087178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26040b49a0 msgr2=0x7f260409d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.856+0000 7f2608837700 1 --2- 172.21.15.92:0/1343087178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26040b49a0 0x7f260409d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.856+0000 7f2608837700 1 -- 172.21.15.92:0/1343087178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26040b0760 msgr2=0x7f260416d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.856+0000 7f2608837700 1 --2- 172.21.15.92:0/1343087178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26040b0760 0x7f260416d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.856+0000 7f2608837700 1 -- 172.21.15.92:0/1343087178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f80094f0 con 0x7f2604159560 2023-10-06T10:39:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.856+0000 7f2608837700 1 --2- 172.21.15.92:0/1343087178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2604159560 0x7f26040aec00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00a380 tx=0x7f25fc007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.856+0000 7f25ebfff700 1 -- 172.21.15.92:0/1343087178 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc015440 con 0x7f2604159560 2023-10-06T10:39:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.856+0000 7f25ebfff700 1 -- 172.21.15.92:0/1343087178 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25fc0155a0 con 0x7f2604159560 2023-10-06T10:39:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.856+0000 7f260a29a700 1 -- 172.21.15.92:0/1343087178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f260415b260 con 0x7f2604159560 2023-10-06T10:39:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.856+0000 7f25ebfff700 1 -- 172.21.15.92:0/1343087178 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc015960 con 0x7f2604159560 2023-10-06T10:39:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.856+0000 7f260a29a700 1 -- 172.21.15.92:0/1343087178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f260415b820 con 0x7f2604159560 2023-10-06T10:39:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.857+0000 7f260a29a700 1 -- 172.21.15.92:0/1343087178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f260415bd40 con 0x7f2604159560 2023-10-06T10:39:09.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.858+0000 7f25ebfff700 1 -- 172.21.15.92:0/1343087178 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f25fc00d440 con 0x7f2604159560 2023-10-06T10:39:09.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.858+0000 7f25ebfff700 1 --2- 172.21.15.92:0/1343087178 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f25f0066f30 0x7f25f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:09.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.858+0000 7f25ebfff700 1 -- 172.21.15.92:0/1343087178 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f25fc084f40 con 0x7f2604159560 2023-10-06T10:39:09.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.858+0000 7f2603fff700 1 --2- 172.21.15.92:0/1343087178 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f25f0066f30 0x7f25f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:09.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.858+0000 7f2603fff700 1 --2- 172.21.15.92:0/1343087178 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f25f0066f30 0x7f25f00693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f25f400ac00 tx=0x7f25f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:09.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:09.862+0000 7f25ebfff700 1 -- 172.21.15.92:0/1343087178 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f260415bd40 con 0x7f2604159560 2023-10-06T10:39:10.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.119+0000 7f260a29a700 1 -- 172.21.15.92:0/1343087178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f2604064a90 con 0x7f2604159560 2023-10-06T10:39:10.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.123+0000 7f25ebfff700 1 -- 172.21.15.92:0/1343087178 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f25fc01a0a0 con 0x7f2604159560 2023-10-06T10:39:10.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.125+0000 7f260a29a700 1 -- 172.21.15.92:0/1343087178 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f25f0066f30 msgr2=0x7f25f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:10.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.125+0000 7f260a29a700 1 --2- 172.21.15.92:0/1343087178 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f25f0066f30 0x7f25f00693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f25f400ac00 tx=0x7f25f4008040 comp rx=0 tx=0).stop 2023-10-06T10:39:10.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.125+0000 7f260a29a700 1 -- 172.21.15.92:0/1343087178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2604159560 msgr2=0x7f26040aec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:10.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.125+0000 7f260a29a700 1 --2- 172.21.15.92:0/1343087178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2604159560 0x7f26040aec00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00a380 tx=0x7f25fc007900 comp rx=0 tx=0).stop 2023-10-06T10:39:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.125+0000 7f260a29a700 1 -- 172.21.15.92:0/1343087178 shutdown_connections 2023-10-06T10:39:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.125+0000 7f260a29a700 1 --2- 172.21.15.92:0/1343087178 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f25f0066f30 0x7f25f00693e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.125+0000 7f260a29a700 1 --2- 172.21.15.92:0/1343087178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2604159560 0x7f26040aec00 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.125+0000 7f260a29a700 1 --2- 172.21.15.92:0/1343087178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26040b0760 0x7f260416d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.125+0000 7f260a29a700 1 --2- 172.21.15.92:0/1343087178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26040b49a0 0x7f260409d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.125+0000 7f260a29a700 1 -- 172.21.15.92:0/1343087178 >> 172.21.15.92:0/1343087178 conn(0x7f26040997f0 msgr2=0x7f26040b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.125+0000 7f260a29a700 1 -- 172.21.15.92:0/1343087178 shutdown_connections 2023-10-06T10:39:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.125+0000 7f260a29a700 1 -- 172.21.15.92:0/1343087178 wait complete. 2023-10-06T10:39:10.139 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-06T10:39:10.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.252+0000 7f6601500700 1 Processor -- start 2023-10-06T10:39:10.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.252+0000 7f6601500700 1 -- start start 2023-10-06T10:39:10.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.253+0000 7f6601500700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f65fc159560 0x7f65fc0aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:10.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.253+0000 7f6601500700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f65fc0b0760 0x7f65fc0b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:10.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.253+0000 7f6601500700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65fc0b0c60 con 0x7f65fc0b49a0 2023-10-06T10:39:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.253+0000 7f6601500700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65fc0b0f40 con 0x7f65fc159560 2023-10-06T10:39:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.253+0000 7f6601500700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65fc0b1220 con 0x7f65fc0b0760 2023-10-06T10:39:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.253+0000 7f65faffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f65fc159560 0x7f65fc0aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.253+0000 7f65f27fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f65fc0b49a0 0x7f65fc0aaa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33128/0 (socket says 172.21.15.92:33128) 2023-10-06T10:39:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.253+0000 7f65faffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f65fc159560 0x7f65fc0aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46652/0 (socket says 172.21.15.92:46652) 2023-10-06T10:39:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.253+0000 7f65f27fc700 1 -- 172.21.15.92:0/3712050897 learned_addr learned my addr 172.21.15.92:0/3712050897 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.253+0000 7f65fb7fe700 1 --2- 172.21.15.92:0/3712050897 >> v2:172.21.15.124:3300/0 conn(0x7f65fc0b0760 0x7f65fc0b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.253+0000 7f65faffd700 1 -- 172.21.15.92:0/3712050897 >> v2:172.21.15.124:3300/0 conn(0x7f65fc0b0760 msgr2=0x7f65fc0b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.253+0000 7f65faffd700 1 --2- 172.21.15.92:0/3712050897 >> v2:172.21.15.124:3300/0 conn(0x7f65fc0b0760 0x7f65fc0b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.253+0000 7f65faffd700 1 -- 172.21.15.92:0/3712050897 >> v1:172.21.15.92:6789/0 conn(0x7f65fc0b49a0 legacy=0x7f65fc0aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.253+0000 7f65faffd700 1 -- 172.21.15.92:0/3712050897 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65fc0b1500 con 0x7f65fc159560 2023-10-06T10:39:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.253+0000 7f65fa7fc700 1 -- 172.21.15.92:0/3712050897 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701648810 0 0) 0x7f65fc0b0c60 con 0x7f65fc0b49a0 2023-10-06T10:39:10.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.254+0000 7f65faffd700 1 --2- 172.21.15.92:0/3712050897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65fc159560 0x7f65fc0aa530 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f65ec0098d0 tx=0x7f65ec009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25093dac6f3bc2a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:10.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.254+0000 7f65fa7fc700 1 -- 172.21.15.92:0/3712050897 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec009e70 con 0x7f65fc159560 2023-10-06T10:39:10.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.254+0000 7f65fa7fc700 1 -- 172.21.15.92:0/3712050897 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ec005c10 con 0x7f65fc159560 2023-10-06T10:39:10.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.254+0000 7f65fa7fc700 1 -- 172.21.15.92:0/3712050897 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec00b610 con 0x7f65fc159560 2023-10-06T10:39:10.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.254+0000 7f6601500700 1 -- 172.21.15.92:0/3712050897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65fc159560 msgr2=0x7f65fc0aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:10.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.254+0000 7f6601500700 1 --2- 172.21.15.92:0/3712050897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65fc159560 0x7f65fc0aa530 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f65ec0098d0 tx=0x7f65ec009120 comp rx=0 tx=0).stop 2023-10-06T10:39:10.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.254+0000 7f6601500700 1 -- 172.21.15.92:0/3712050897 shutdown_connections 2023-10-06T10:39:10.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.254+0000 7f6601500700 1 --2- 172.21.15.92:0/3712050897 >> v2:172.21.15.124:3300/0 conn(0x7f65fc0b0760 0x7f65fc0b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.254+0000 7f6601500700 1 --2- 172.21.15.92:0/3712050897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65fc159560 0x7f65fc0aa530 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.254+0000 7f6601500700 1 -- 172.21.15.92:0/3712050897 >> 172.21.15.92:0/3712050897 conn(0x7f65fc0997f0 msgr2=0x7f65fc06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:10.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.254+0000 7f6601500700 1 -- 172.21.15.92:0/3712050897 shutdown_connections 2023-10-06T10:39:10.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.254+0000 7f6601500700 1 -- 172.21.15.92:0/3712050897 wait complete. 2023-10-06T10:39:10.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.256+0000 7f6601500700 1 Processor -- start 2023-10-06T10:39:10.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.256+0000 7f6601500700 1 -- start start 2023-10-06T10:39:10.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.256+0000 7f6601500700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65fc0b0760 0x7f65fc144a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:10.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.256+0000 7f6601500700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f65fc0b49a0 0x7f65fc15c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:10.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.256+0000 7f6601500700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65fc159560 0x7f65fc15bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.256+0000 7f6601500700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc09a420 con 0x7f65fc159560 2023-10-06T10:39:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.256+0000 7f6601500700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc09a0f0 con 0x7f65fc0b49a0 2023-10-06T10:39:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.256+0000 7f6601500700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc15ad20 con 0x7f65fc0b0760 2023-10-06T10:39:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.256+0000 7f65fb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65fc159560 0x7f65fc15bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.256+0000 7f65fb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65fc159560 0x7f65fc15bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46664/0 (socket says 172.21.15.92:46664) 2023-10-06T10:39:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.256+0000 7f65fb7fe700 1 -- 172.21.15.92:0/1643321354 learned_addr learned my addr 172.21.15.92:0/1643321354 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.256+0000 7f65f27fc700 1 --2- 172.21.15.92:0/1643321354 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f65fc0b49a0 0x7f65fc15c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.256+0000 7f65faffd700 1 --2- 172.21.15.92:0/1643321354 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65fc0b0760 0x7f65fc144a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.257+0000 7f65fb7fe700 1 -- 172.21.15.92:0/1643321354 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65fc0b0760 msgr2=0x7f65fc144a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.257+0000 7f65fb7fe700 1 --2- 172.21.15.92:0/1643321354 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65fc0b0760 0x7f65fc144a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.257+0000 7f65fb7fe700 1 -- 172.21.15.92:0/1643321354 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f65fc0b49a0 msgr2=0x7f65fc15c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.257+0000 7f65fb7fe700 1 --2- 172.21.15.92:0/1643321354 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f65fc0b49a0 0x7f65fc15c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.257+0000 7f65fb7fe700 1 -- 172.21.15.92:0/1643321354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ec0093c0 con 0x7f65fc159560 2023-10-06T10:39:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.257+0000 7f65fb7fe700 1 --2- 172.21.15.92:0/1643321354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65fc159560 0x7f65fc15bda0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f65f4009fa0 tx=0x7f65f4009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.257+0000 7f65f37fe700 1 -- 172.21.15.92:0/1643321354 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f400ebc0 con 0x7f65fc159560 2023-10-06T10:39:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.257+0000 7f65f37fe700 1 -- 172.21.15.92:0/1643321354 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f400ed20 con 0x7f65fc159560 2023-10-06T10:39:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.257+0000 7f6601500700 1 -- 172.21.15.92:0/1643321354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65fc15b000 con 0x7f65fc159560 2023-10-06T10:39:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.257+0000 7f65f37fe700 1 -- 172.21.15.92:0/1643321354 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f4014670 con 0x7f65fc159560 2023-10-06T10:39:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.257+0000 7f6601500700 1 -- 172.21.15.92:0/1643321354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65fc144f50 con 0x7f65fc159560 2023-10-06T10:39:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.258+0000 7f6601500700 1 -- 172.21.15.92:0/1643321354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65fc09a590 con 0x7f65fc159560 2023-10-06T10:39:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.259+0000 7f65f37fe700 1 -- 172.21.15.92:0/1643321354 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f65f40149e0 con 0x7f65fc159560 2023-10-06T10:39:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.259+0000 7f65f37fe700 1 --2- 172.21.15.92:0/1643321354 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f65dc066f30 0x7f65dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.259+0000 7f65f37fe700 1 -- 172.21.15.92:0/1643321354 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f65f4084b20 con 0x7f65fc159560 2023-10-06T10:39:10.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.259+0000 7f65faffd700 1 --2- 172.21.15.92:0/1643321354 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f65dc066f30 0x7f65dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:10.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.259+0000 7f65faffd700 1 --2- 172.21.15.92:0/1643321354 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f65dc066f30 0x7f65dc0693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f65ec009160 tx=0x7f65ec016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:10.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.263+0000 7f65f37fe700 1 -- 172.21.15.92:0/1643321354 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f65f4051e40 con 0x7f65fc159560 2023-10-06T10:39:10.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.520+0000 7f6601500700 1 -- 172.21.15.92:0/1643321354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f65fc064a90 con 0x7f65fc159560 2023-10-06T10:39:10.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.523+0000 7f65f37fe700 1 -- 172.21.15.92:0/1643321354 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f65f4055460 con 0x7f65fc159560 2023-10-06T10:39:10.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.525+0000 7f6601500700 1 -- 172.21.15.92:0/1643321354 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f65dc066f30 msgr2=0x7f65dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:10.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.525+0000 7f6601500700 1 --2- 172.21.15.92:0/1643321354 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f65dc066f30 0x7f65dc0693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f65ec009160 tx=0x7f65ec016000 comp rx=0 tx=0).stop 2023-10-06T10:39:10.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.525+0000 7f6601500700 1 -- 172.21.15.92:0/1643321354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65fc159560 msgr2=0x7f65fc15bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:10.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.525+0000 7f6601500700 1 --2- 172.21.15.92:0/1643321354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65fc159560 0x7f65fc15bda0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f65f4009fa0 tx=0x7f65f4009900 comp rx=0 tx=0).stop 2023-10-06T10:39:10.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.525+0000 7f6601500700 1 -- 172.21.15.92:0/1643321354 shutdown_connections 2023-10-06T10:39:10.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.525+0000 7f6601500700 1 --2- 172.21.15.92:0/1643321354 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f65dc066f30 0x7f65dc0693e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.525+0000 7f6601500700 1 --2- 172.21.15.92:0/1643321354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65fc159560 0x7f65fc15bda0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.525+0000 7f6601500700 1 --2- 172.21.15.92:0/1643321354 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65fc0b0760 0x7f65fc144a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.525+0000 7f6601500700 1 --2- 172.21.15.92:0/1643321354 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f65fc0b49a0 0x7f65fc15c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.525+0000 7f6601500700 1 -- 172.21.15.92:0/1643321354 >> 172.21.15.92:0/1643321354 conn(0x7f65fc0997f0 msgr2=0x7f65fc069850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:10.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.525+0000 7f6601500700 1 -- 172.21.15.92:0/1643321354 shutdown_connections 2023-10-06T10:39:10.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.525+0000 7f6601500700 1 -- 172.21.15.92:0/1643321354 wait complete. 2023-10-06T10:39:10.539 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-06T10:39:10.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.652+0000 7fa94619e700 1 Processor -- start 2023-10-06T10:39:10.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.652+0000 7fa94619e700 1 -- start start 2023-10-06T10:39:10.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.653+0000 7fa94619e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa940159390 0x7fa9400ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:10.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.653+0000 7fa94619e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa940159760 0x7fa9400aad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:10.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.653+0000 7fa94619e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9400ab240 0x7fa940069110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:10.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.653+0000 7fa94619e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9400ab6a0 con 0x7fa940159760 2023-10-06T10:39:10.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.653+0000 7fa94619e700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9400686c0 con 0x7fa940159390 2023-10-06T10:39:10.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.653+0000 7fa94619e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa940068c80 con 0x7fa9400ab240 2023-10-06T10:39:10.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.653+0000 7fa94499b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa940159760 0x7fa9400aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:10.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.653+0000 7fa94499b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa940159760 0x7fa9400aad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46668/0 (socket says 172.21.15.92:46668) 2023-10-06T10:39:10.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.653+0000 7fa94499b700 1 -- 172.21.15.92:0/2623402915 learned_addr learned my addr 172.21.15.92:0/2623402915 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:10.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.653+0000 7fa94519c700 1 --2- 172.21.15.92:0/2623402915 >> v2:172.21.15.124:3300/0 conn(0x7fa940159390 0x7fa9400ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:10.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.653+0000 7fa94599d700 1 --2- 172.21.15.92:0/2623402915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9400ab240 0x7fa940069110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:10.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.653+0000 7fa94499b700 1 -- 172.21.15.92:0/2623402915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9400ab240 msgr2=0x7fa940069110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:10.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.653+0000 7fa94499b700 1 --2- 172.21.15.92:0/2623402915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9400ab240 0x7fa940069110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.654+0000 7fa94499b700 1 -- 172.21.15.92:0/2623402915 >> v2:172.21.15.124:3300/0 conn(0x7fa940159390 msgr2=0x7fa9400ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:10.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.654+0000 7fa94499b700 1 --2- 172.21.15.92:0/2623402915 >> v2:172.21.15.124:3300/0 conn(0x7fa940159390 0x7fa9400ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.654+0000 7fa94499b700 1 -- 172.21.15.92:0/2623402915 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9400ace20 con 0x7fa940159760 2023-10-06T10:39:10.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.654+0000 7fa94499b700 1 --2- 172.21.15.92:0/2623402915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa940159760 0x7fa9400aad00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa94006eb90 tx=0x7fa930009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3391019f1bbf8d27 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:10.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.654+0000 7fa937fff700 1 -- 172.21.15.92:0/2623402915 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93000b640 con 0x7fa940159760 2023-10-06T10:39:10.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.654+0000 7fa937fff700 1 -- 172.21.15.92:0/2623402915 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93000b7a0 con 0x7fa940159760 2023-10-06T10:39:10.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.654+0000 7fa937fff700 1 -- 172.21.15.92:0/2623402915 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93000bb60 con 0x7fa940159760 2023-10-06T10:39:10.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.654+0000 7fa94619e700 1 -- 172.21.15.92:0/2623402915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa940159760 msgr2=0x7fa9400aad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:10.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.654+0000 7fa94619e700 1 --2- 172.21.15.92:0/2623402915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa940159760 0x7fa9400aad00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa94006eb90 tx=0x7fa930009250 comp rx=0 tx=0).stop 2023-10-06T10:39:10.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.654+0000 7fa94619e700 1 -- 172.21.15.92:0/2623402915 shutdown_connections 2023-10-06T10:39:10.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.654+0000 7fa94619e700 1 --2- 172.21.15.92:0/2623402915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9400ab240 0x7fa940069110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.654+0000 7fa94619e700 1 --2- 172.21.15.92:0/2623402915 >> v2:172.21.15.124:3300/0 conn(0x7fa940159390 0x7fa9400ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.654+0000 7fa94619e700 1 --2- 172.21.15.92:0/2623402915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa940159760 0x7fa9400aad00 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.654+0000 7fa94619e700 1 -- 172.21.15.92:0/2623402915 >> 172.21.15.92:0/2623402915 conn(0x7fa940097410 msgr2=0x7fa940067d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:10.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.654+0000 7fa94619e700 1 -- 172.21.15.92:0/2623402915 shutdown_connections 2023-10-06T10:39:10.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.654+0000 7fa94619e700 1 -- 172.21.15.92:0/2623402915 wait complete. 2023-10-06T10:39:10.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.655+0000 7fa94619e700 1 Processor -- start 2023-10-06T10:39:10.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.655+0000 7fa94619e700 1 -- start start 2023-10-06T10:39:10.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.656+0000 7fa94619e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9400ab240 0x7fa940154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:10.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.656+0000 7fa94619e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa940159390 0x7fa9401574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:10.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.656+0000 7fa94619e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa940159760 0x7fa940157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:10.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.656+0000 7fa94619e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9400b1260 con 0x7fa9400ab240 2023-10-06T10:39:10.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.656+0000 7fa94619e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9401699c0 con 0x7fa940159390 2023-10-06T10:39:10.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.656+0000 7fa94619e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9401580c0 con 0x7fa940159760 2023-10-06T10:39:10.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.656+0000 7fa94519c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9400ab240 0x7fa940154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:10.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.656+0000 7fa94519c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9400ab240 0x7fa940154cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46676/0 (socket says 172.21.15.92:46676) 2023-10-06T10:39:10.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.656+0000 7fa94519c700 1 -- 172.21.15.92:0/649253968 learned_addr learned my addr 172.21.15.92:0/649253968 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:10.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.657+0000 7fa94599d700 1 --2- 172.21.15.92:0/649253968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa940159760 0x7fa940157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:10.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.657+0000 7fa94499b700 1 --2- 172.21.15.92:0/649253968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa940159390 0x7fa9401574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:10.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.657+0000 7fa94519c700 1 -- 172.21.15.92:0/649253968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa940159760 msgr2=0x7fa940157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:10.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.657+0000 7fa94519c700 1 --2- 172.21.15.92:0/649253968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa940159760 0x7fa940157a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.657+0000 7fa94519c700 1 -- 172.21.15.92:0/649253968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa940159390 msgr2=0x7fa9401574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:10.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.657+0000 7fa94519c700 1 --2- 172.21.15.92:0/649253968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa940159390 0x7fa9401574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.657+0000 7fa94519c700 1 -- 172.21.15.92:0/649253968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9300094f0 con 0x7fa9400ab240 2023-10-06T10:39:10.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.657+0000 7fa94599d700 1 --2- 172.21.15.92:0/649253968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa940159760 0x7fa940157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:10.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.657+0000 7fa94519c700 1 --2- 172.21.15.92:0/649253968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9400ab240 0x7fa940154cd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00ec80 tx=0x7fa93c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:10.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.657+0000 7fa9357fa700 1 -- 172.21.15.92:0/649253968 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c00c850 con 0x7fa9400ab240 2023-10-06T10:39:10.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.657+0000 7fa9357fa700 1 -- 172.21.15.92:0/649253968 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c00c9b0 con 0x7fa9400ab240 2023-10-06T10:39:10.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.657+0000 7fa94619e700 1 -- 172.21.15.92:0/649253968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9401583a0 con 0x7fa9400ab240 2023-10-06T10:39:10.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.657+0000 7fa9357fa700 1 -- 172.21.15.92:0/649253968 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c00cd50 con 0x7fa9400ab240 2023-10-06T10:39:10.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.657+0000 7fa94619e700 1 -- 172.21.15.92:0/649253968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa940158960 con 0x7fa9400ab240 2023-10-06T10:39:10.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.658+0000 7fa94619e700 1 -- 172.21.15.92:0/649253968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa940169ca0 con 0x7fa9400ab240 2023-10-06T10:39:10.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.658+0000 7fa9357fa700 1 -- 172.21.15.92:0/649253968 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa93c013680 con 0x7fa9400ab240 2023-10-06T10:39:10.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.659+0000 7fa9357fa700 1 --2- 172.21.15.92:0/649253968 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa924066f30 0x7fa9240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:10.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.659+0000 7fa9357fa700 1 -- 172.21.15.92:0/649253968 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fa93c084720 con 0x7fa9400ab240 2023-10-06T10:39:10.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.659+0000 7fa94499b700 1 --2- 172.21.15.92:0/649253968 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa924066f30 0x7fa9240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:10.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.659+0000 7fa94499b700 1 --2- 172.21.15.92:0/649253968 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa924066f30 0x7fa9240693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa94006eb90 tx=0x7fa930017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:10.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.663+0000 7fa9357fa700 1 -- 172.21.15.92:0/649253968 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa93c051a40 con 0x7fa9400ab240 2023-10-06T10:39:10.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.921+0000 7fa94619e700 1 -- 172.21.15.92:0/649253968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fa94005f230 con 0x7fa9400ab240 2023-10-06T10:39:10.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.924+0000 7fa9357fa700 1 -- 172.21.15.92:0/649253968 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa93c055060 con 0x7fa9400ab240 2023-10-06T10:39:10.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.927+0000 7fa94619e700 1 -- 172.21.15.92:0/649253968 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa924066f30 msgr2=0x7fa9240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:10.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.927+0000 7fa94619e700 1 --2- 172.21.15.92:0/649253968 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa924066f30 0x7fa9240693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa94006eb90 tx=0x7fa930017000 comp rx=0 tx=0).stop 2023-10-06T10:39:10.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.927+0000 7fa94619e700 1 -- 172.21.15.92:0/649253968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9400ab240 msgr2=0x7fa940154cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:10.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.927+0000 7fa94619e700 1 --2- 172.21.15.92:0/649253968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9400ab240 0x7fa940154cd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00ec80 tx=0x7fa93c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.927+0000 7fa94619e700 1 -- 172.21.15.92:0/649253968 shutdown_connections 2023-10-06T10:39:10.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.927+0000 7fa94619e700 1 --2- 172.21.15.92:0/649253968 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa924066f30 0x7fa9240693e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.927+0000 7fa94619e700 1 --2- 172.21.15.92:0/649253968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa940159760 0x7fa940157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.927+0000 7fa94619e700 1 --2- 172.21.15.92:0/649253968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9400ab240 0x7fa940154cd0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.927+0000 7fa94619e700 1 --2- 172.21.15.92:0/649253968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa940159390 0x7fa9401574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:10.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.927+0000 7fa94619e700 1 -- 172.21.15.92:0/649253968 >> 172.21.15.92:0/649253968 conn(0x7fa940097410 msgr2=0x7fa94015d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:10.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.927+0000 7fa94619e700 1 -- 172.21.15.92:0/649253968 shutdown_connections 2023-10-06T10:39:10.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:10.927+0000 7fa94619e700 1 -- 172.21.15.92:0/649253968 wait complete. 2023-10-06T10:39:10.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-06T10:39:11.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.054+0000 7fd26e54a700 1 Processor -- start 2023-10-06T10:39:11.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.055+0000 7fd26e54a700 1 -- start start 2023-10-06T10:39:11.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.055+0000 7fd26e54a700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd2680ca370 0x7fd268158270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:11.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.055+0000 7fd26e54a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd2680adb60 0x7fd2680ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:11.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.055+0000 7fd26e54a700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd268157de0 con 0x7fd2680ce6d0 2023-10-06T10:39:11.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.055+0000 7fd26e54a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2680cbb90 con 0x7fd2680adb60 2023-10-06T10:39:11.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.055+0000 7fd26e54a700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2680ceb10 con 0x7fd2680ca370 2023-10-06T10:39:11.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.055+0000 7fd2677fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd2680adb60 0x7fd2680ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:11.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.055+0000 7fd2677fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd2680adb60 0x7fd2680ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46688/0 (socket says 172.21.15.92:46688) 2023-10-06T10:39:11.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.056+0000 7fd2677fe700 1 -- 172.21.15.92:0/1596423856 learned_addr learned my addr 172.21.15.92:0/1596423856 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:11.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.056+0000 7fd2677fe700 1 -- 172.21.15.92:0/1596423856 >> v2:172.21.15.124:3300/0 conn(0x7fd2680ca370 msgr2=0x7fd268158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:11.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.056+0000 7fd2677fe700 1 --2- 172.21.15.92:0/1596423856 >> v2:172.21.15.124:3300/0 conn(0x7fd2680ca370 0x7fd268158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.056+0000 7fd2677fe700 1 -- 172.21.15.92:0/1596423856 >> v1:172.21.15.124:6789/0 conn(0x7fd2680ce6d0 legacy=0x7fd26815aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:11.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.056+0000 7fd2677fe700 1 -- 172.21.15.92:0/1596423856 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2680cb5d0 con 0x7fd2680adb60 2023-10-06T10:39:11.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.056+0000 7fd2677fe700 1 --2- 172.21.15.92:0/1596423856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2680adb60 0x7fd2680ce190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd26806f2c0 tx=0x7fd25c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feab1b50ac742fc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:11.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.056+0000 7fd266ffd700 1 -- 172.21.15.92:0/1596423856 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c00b820 con 0x7fd2680adb60 2023-10-06T10:39:11.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.056+0000 7fd266ffd700 1 -- 172.21.15.92:0/1596423856 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25c00b980 con 0x7fd2680adb60 2023-10-06T10:39:11.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.056+0000 7fd266ffd700 1 -- 172.21.15.92:0/1596423856 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c00bd20 con 0x7fd2680adb60 2023-10-06T10:39:11.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.057+0000 7fd26e54a700 1 -- 172.21.15.92:0/1596423856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2680adb60 msgr2=0x7fd2680ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:11.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.057+0000 7fd26e54a700 1 --2- 172.21.15.92:0/1596423856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2680adb60 0x7fd2680ce190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd26806f2c0 tx=0x7fd25c009250 comp rx=0 tx=0).stop 2023-10-06T10:39:11.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.057+0000 7fd26e54a700 1 -- 172.21.15.92:0/1596423856 shutdown_connections 2023-10-06T10:39:11.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.057+0000 7fd26e54a700 1 --2- 172.21.15.92:0/1596423856 >> v2:172.21.15.124:3300/0 conn(0x7fd2680ca370 0x7fd268158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.057+0000 7fd26e54a700 1 --2- 172.21.15.92:0/1596423856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2680adb60 0x7fd2680ce190 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.057+0000 7fd26e54a700 1 -- 172.21.15.92:0/1596423856 >> 172.21.15.92:0/1596423856 conn(0x7fd2680707d0 msgr2=0x7fd268070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:11.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.057+0000 7fd26e54a700 1 -- 172.21.15.92:0/1596423856 shutdown_connections 2023-10-06T10:39:11.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.057+0000 7fd26e54a700 1 -- 172.21.15.92:0/1596423856 wait complete. 2023-10-06T10:39:11.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.058+0000 7fd26e54a700 1 Processor -- start 2023-10-06T10:39:11.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.058+0000 7fd26e54a700 1 -- start start 2023-10-06T10:39:11.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd26e54a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2680adb60 0x7fd2680c2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:11.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd26e54a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2680ca370 0x7fd2680b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:11.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd26e54a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2680ce6d0 0x7fd2680b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:11.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd26e54a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2680b11b0 con 0x7fd2680ce6d0 2023-10-06T10:39:11.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd26e54a700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2680c3910 con 0x7fd2680ca370 2023-10-06T10:39:11.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd26e54a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2680c3bf0 con 0x7fd2680adb60 2023-10-06T10:39:11.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd26cae7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2680ce6d0 0x7fd2680b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:11.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd26cae7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2680ce6d0 0x7fd2680b3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46700/0 (socket says 172.21.15.92:46700) 2023-10-06T10:39:11.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd26cae7700 1 -- 172.21.15.92:0/2732177807 learned_addr learned my addr 172.21.15.92:0/2732177807 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:11.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd26cae7700 1 -- 172.21.15.92:0/2732177807 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2680adb60 msgr2=0x7fd2680c2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:11.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd267fff700 1 --2- 172.21.15.92:0/2732177807 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2680adb60 0x7fd2680c2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:11.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd2677fe700 1 --2- 172.21.15.92:0/2732177807 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2680ca370 0x7fd2680b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:11.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd26cae7700 1 --2- 172.21.15.92:0/2732177807 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2680adb60 0x7fd2680c2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd26cae7700 1 -- 172.21.15.92:0/2732177807 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2680ca370 msgr2=0x7fd2680b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:11.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd26cae7700 1 --2- 172.21.15.92:0/2732177807 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2680ca370 0x7fd2680b33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd26cae7700 1 -- 172.21.15.92:0/2732177807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c0094f0 con 0x7fd2680ce6d0 2023-10-06T10:39:11.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd2677fe700 1 --2- 172.21.15.92:0/2732177807 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2680ca370 0x7fd2680b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:11.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd26cae7700 1 --2- 172.21.15.92:0/2732177807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2680ce6d0 0x7fd2680b3930 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd26000bbb0 tx=0x7fd260009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:11.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd24ffff700 1 -- 172.21.15.92:0/2732177807 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2600092e0 con 0x7fd2680ce6d0 2023-10-06T10:39:11.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd24ffff700 1 -- 172.21.15.92:0/2732177807 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26000c7a0 con 0x7fd2680ce6d0 2023-10-06T10:39:11.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd26e54a700 1 -- 172.21.15.92:0/2732177807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2680c3ed0 con 0x7fd2680ce6d0 2023-10-06T10:39:11.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd24ffff700 1 -- 172.21.15.92:0/2732177807 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26000cb40 con 0x7fd2680ce6d0 2023-10-06T10:39:11.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.059+0000 7fd26e54a700 1 -- 172.21.15.92:0/2732177807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd268167cf0 con 0x7fd2680ce6d0 2023-10-06T10:39:11.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.060+0000 7fd26e54a700 1 -- 172.21.15.92:0/2732177807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2680b1360 con 0x7fd2680ce6d0 2023-10-06T10:39:11.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.061+0000 7fd24ffff700 1 -- 172.21.15.92:0/2732177807 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd260012070 con 0x7fd2680ce6d0 2023-10-06T10:39:11.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.061+0000 7fd24ffff700 1 --2- 172.21.15.92:0/2732177807 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd254066f30 0x7fd2540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:11.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.061+0000 7fd267fff700 1 --2- 172.21.15.92:0/2732177807 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd254066f30 0x7fd2540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:11.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.061+0000 7fd24ffff700 1 -- 172.21.15.92:0/2732177807 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fd260084200 con 0x7fd2680ce6d0 2023-10-06T10:39:11.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.061+0000 7fd267fff700 1 --2- 172.21.15.92:0/2732177807 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd254066f30 0x7fd2540693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd258006fd0 tx=0x7fd258008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:11.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.065+0000 7fd24ffff700 1 -- 172.21.15.92:0/2732177807 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2680b1360 con 0x7fd2680ce6d0 2023-10-06T10:39:11.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.321+0000 7fd26e54a700 1 -- 172.21.15.92:0/2732177807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fd268064a90 con 0x7fd2680ce6d0 2023-10-06T10:39:11.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.325+0000 7fd24ffff700 1 -- 172.21.15.92:0/2732177807 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd26005a030 con 0x7fd2680ce6d0 2023-10-06T10:39:11.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.327+0000 7fd26e54a700 1 -- 172.21.15.92:0/2732177807 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd254066f30 msgr2=0x7fd2540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:11.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.327+0000 7fd26e54a700 1 --2- 172.21.15.92:0/2732177807 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd254066f30 0x7fd2540693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd258006fd0 tx=0x7fd258008040 comp rx=0 tx=0).stop 2023-10-06T10:39:11.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.327+0000 7fd26e54a700 1 -- 172.21.15.92:0/2732177807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2680ce6d0 msgr2=0x7fd2680b3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:11.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.327+0000 7fd26e54a700 1 --2- 172.21.15.92:0/2732177807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2680ce6d0 0x7fd2680b3930 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd26000bbb0 tx=0x7fd260009840 comp rx=0 tx=0).stop 2023-10-06T10:39:11.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.327+0000 7fd26e54a700 1 -- 172.21.15.92:0/2732177807 shutdown_connections 2023-10-06T10:39:11.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.327+0000 7fd26e54a700 1 --2- 172.21.15.92:0/2732177807 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd254066f30 0x7fd2540693e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.327+0000 7fd26e54a700 1 --2- 172.21.15.92:0/2732177807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2680ce6d0 0x7fd2680b3930 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.327+0000 7fd26e54a700 1 --2- 172.21.15.92:0/2732177807 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2680adb60 0x7fd2680c2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.327+0000 7fd26e54a700 1 --2- 172.21.15.92:0/2732177807 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2680ca370 0x7fd2680b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.327+0000 7fd26e54a700 1 -- 172.21.15.92:0/2732177807 >> 172.21.15.92:0/2732177807 conn(0x7fd2680707d0 msgr2=0x7fd268154500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:11.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.327+0000 7fd26e54a700 1 -- 172.21.15.92:0/2732177807 shutdown_connections 2023-10-06T10:39:11.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.327+0000 7fd26e54a700 1 -- 172.21.15.92:0/2732177807 wait complete. 2023-10-06T10:39:11.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-06T10:39:11.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.453+0000 7f0e94fb7700 1 Processor -- start 2023-10-06T10:39:11.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.453+0000 7f0e94fb7700 1 -- start start 2023-10-06T10:39:11.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.454+0000 7f0e94fb7700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0e900b49a0 0x7f0e900aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:11.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.454+0000 7f0e94fb7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0e900b0760 0x7f0e900b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:11.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.454+0000 7f0e94fb7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e900b0c60 con 0x7f0e90159560 2023-10-06T10:39:11.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.454+0000 7f0e94fb7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e900b0f40 con 0x7f0e900b0760 2023-10-06T10:39:11.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.454+0000 7f0e94fb7700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e900b1220 con 0x7f0e900b49a0 2023-10-06T10:39:11.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.454+0000 7f0e8ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0e900b0760 0x7f0e900b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:11.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.454+0000 7f0e8e59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0e90159560 0x7f0e900aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33144/0 (socket says 172.21.15.92:33144) 2023-10-06T10:39:11.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.454+0000 7f0e8e59c700 1 -- 172.21.15.92:0/3417962146 learned_addr learned my addr 172.21.15.92:0/3417962146 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:11.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.455+0000 7f0e87fff700 1 --2- 172.21.15.92:0/3417962146 >> v2:172.21.15.124:3300/0 conn(0x7f0e900b49a0 0x7f0e900aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:11.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.455+0000 7f0e8ed9d700 1 -- 172.21.15.92:0/3417962146 >> v2:172.21.15.124:3300/0 conn(0x7f0e900b49a0 msgr2=0x7f0e900aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:11.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.455+0000 7f0e8ed9d700 1 --2- 172.21.15.92:0/3417962146 >> v2:172.21.15.124:3300/0 conn(0x7f0e900b49a0 0x7f0e900aac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.455+0000 7f0e8dd9b700 1 -- 172.21.15.92:0/3417962146 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 523421236 0 0) 0x7f0e900b0c60 con 0x7f0e90159560 2023-10-06T10:39:11.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.455+0000 7f0e8ed9d700 1 -- 172.21.15.92:0/3417962146 >> v1:172.21.15.92:6789/0 conn(0x7f0e90159560 legacy=0x7f0e900aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:11.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.455+0000 7f0e8ed9d700 1 -- 172.21.15.92:0/3417962146 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e900b1500 con 0x7f0e900b0760 2023-10-06T10:39:11.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.455+0000 7f0e8ed9d700 1 --2- 172.21.15.92:0/3417962146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e900b0760 0x7f0e900b1bc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009e60 tx=0x7f0e88009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b81bca7174057 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:11.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.455+0000 7f0e8dd9b700 1 -- 172.21.15.92:0/3417962146 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8800e070 con 0x7f0e900b0760 2023-10-06T10:39:11.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.455+0000 7f0e8dd9b700 1 -- 172.21.15.92:0/3417962146 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8800bbf0 con 0x7f0e900b0760 2023-10-06T10:39:11.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.455+0000 7f0e8dd9b700 1 -- 172.21.15.92:0/3417962146 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e88012430 con 0x7f0e900b0760 2023-10-06T10:39:11.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.455+0000 7f0e94fb7700 1 -- 172.21.15.92:0/3417962146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e900b0760 msgr2=0x7f0e900b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:11.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.455+0000 7f0e94fb7700 1 --2- 172.21.15.92:0/3417962146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e900b0760 0x7f0e900b1bc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009e60 tx=0x7f0e88009780 comp rx=0 tx=0).stop 2023-10-06T10:39:11.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.455+0000 7f0e94fb7700 1 -- 172.21.15.92:0/3417962146 shutdown_connections 2023-10-06T10:39:11.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.455+0000 7f0e94fb7700 1 --2- 172.21.15.92:0/3417962146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e900b0760 0x7f0e900b1bc0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.455+0000 7f0e94fb7700 1 --2- 172.21.15.92:0/3417962146 >> v2:172.21.15.124:3300/0 conn(0x7f0e900b49a0 0x7f0e900aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.455+0000 7f0e94fb7700 1 -- 172.21.15.92:0/3417962146 >> 172.21.15.92:0/3417962146 conn(0x7f0e900997f0 msgr2=0x7f0e9006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:11.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.456+0000 7f0e94fb7700 1 -- 172.21.15.92:0/3417962146 shutdown_connections 2023-10-06T10:39:11.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.456+0000 7f0e94fb7700 1 -- 172.21.15.92:0/3417962146 wait complete. 2023-10-06T10:39:11.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.457+0000 7f0e94fb7700 1 Processor -- start 2023-10-06T10:39:11.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.457+0000 7f0e94fb7700 1 -- start start 2023-10-06T10:39:11.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.457+0000 7f0e94fb7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e900b0760 0x7f0e900aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:11.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.457+0000 7f0e94fb7700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e900b49a0 0x7f0e90169980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:11.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.457+0000 7f0e94fb7700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e90159560 0x7f0e90157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:11.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.457+0000 7f0e94fb7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e90169ec0 con 0x7f0e900b0760 2023-10-06T10:39:11.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.457+0000 7f0e94fb7700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e90159930 con 0x7f0e900b49a0 2023-10-06T10:39:11.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.457+0000 7f0e94fb7700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9015a1d0 con 0x7f0e90159560 2023-10-06T10:39:11.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.457+0000 7f0e8e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e900b0760 0x7f0e900aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:11.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.457+0000 7f0e8e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e900b0760 0x7f0e900aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46714/0 (socket says 172.21.15.92:46714) 2023-10-06T10:39:11.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.457+0000 7f0e8e59c700 1 -- 172.21.15.92:0/1194722277 learned_addr learned my addr 172.21.15.92:0/1194722277 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:11.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.457+0000 7f0e87fff700 1 --2- 172.21.15.92:0/1194722277 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e900b49a0 0x7f0e90169980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:11.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.458+0000 7f0e8ed9d700 1 --2- 172.21.15.92:0/1194722277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e90159560 0x7f0e90157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:11.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.458+0000 7f0e8e59c700 1 -- 172.21.15.92:0/1194722277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e90159560 msgr2=0x7f0e90157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:11.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.458+0000 7f0e8e59c700 1 --2- 172.21.15.92:0/1194722277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e90159560 0x7f0e90157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.458+0000 7f0e8e59c700 1 -- 172.21.15.92:0/1194722277 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e900b49a0 msgr2=0x7f0e90169980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:11.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.458+0000 7f0e8e59c700 1 --2- 172.21.15.92:0/1194722277 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e900b49a0 0x7f0e90169980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.458+0000 7f0e8e59c700 1 -- 172.21.15.92:0/1194722277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e88009a20 con 0x7f0e900b0760 2023-10-06T10:39:11.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.458+0000 7f0e87fff700 1 --2- 172.21.15.92:0/1194722277 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e900b49a0 0x7f0e90169980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:11.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.458+0000 7f0e8e59c700 1 --2- 172.21.15.92:0/1194722277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e900b0760 0x7f0e900aece0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000c5c0 tx=0x7f0e80007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:11.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.458+0000 7f0e867fc700 1 -- 172.21.15.92:0/1194722277 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e800072e0 con 0x7f0e900b0760 2023-10-06T10:39:11.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.458+0000 7f0e867fc700 1 -- 172.21.15.92:0/1194722277 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e80007480 con 0x7f0e900b0760 2023-10-06T10:39:11.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.458+0000 7f0e94fb7700 1 -- 172.21.15.92:0/1194722277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e90158280 con 0x7f0e900b0760 2023-10-06T10:39:11.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.458+0000 7f0e867fc700 1 -- 172.21.15.92:0/1194722277 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e800057f0 con 0x7f0e900b0760 2023-10-06T10:39:11.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.458+0000 7f0e94fb7700 1 -- 172.21.15.92:0/1194722277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e900ad650 con 0x7f0e900b0760 2023-10-06T10:39:11.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.459+0000 7f0e94fb7700 1 -- 172.21.15.92:0/1194722277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e90064a90 con 0x7f0e900b0760 2023-10-06T10:39:11.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.459+0000 7f0e867fc700 1 -- 172.21.15.92:0/1194722277 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0e80005ba0 con 0x7f0e900b0760 2023-10-06T10:39:11.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.459+0000 7f0e867fc700 1 --2- 172.21.15.92:0/1194722277 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0e70066f30 0x7f0e700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:11.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.460+0000 7f0e867fc700 1 -- 172.21.15.92:0/1194722277 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f0e80083870 con 0x7f0e900b0760 2023-10-06T10:39:11.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.460+0000 7f0e87fff700 1 --2- 172.21.15.92:0/1194722277 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0e70066f30 0x7f0e700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:11.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.460+0000 7f0e87fff700 1 --2- 172.21.15.92:0/1194722277 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0e70066f30 0x7f0e700693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0e9015baf0 tx=0x7f0e78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:11.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.464+0000 7f0e867fc700 1 -- 172.21.15.92:0/1194722277 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e8004b5a0 con 0x7f0e900b0760 2023-10-06T10:39:11.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.723+0000 7f0e94fb7700 1 -- 172.21.15.92:0/1194722277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f0e900b17e0 con 0x7f0e900b0760 2023-10-06T10:39:11.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.726+0000 7f0e867fc700 1 -- 172.21.15.92:0/1194722277 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0e80017020 con 0x7f0e900b0760 2023-10-06T10:39:11.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.728+0000 7f0e94fb7700 1 -- 172.21.15.92:0/1194722277 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0e70066f30 msgr2=0x7f0e700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.728+0000 7f0e94fb7700 1 --2- 172.21.15.92:0/1194722277 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0e70066f30 0x7f0e700693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0e9015baf0 tx=0x7f0e78008040 comp rx=0 tx=0).stop 2023-10-06T10:39:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.728+0000 7f0e94fb7700 1 -- 172.21.15.92:0/1194722277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e900b0760 msgr2=0x7f0e900aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.728+0000 7f0e94fb7700 1 --2- 172.21.15.92:0/1194722277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e900b0760 0x7f0e900aece0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000c5c0 tx=0x7f0e80007e80 comp rx=0 tx=0).stop 2023-10-06T10:39:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.728+0000 7f0e94fb7700 1 -- 172.21.15.92:0/1194722277 shutdown_connections 2023-10-06T10:39:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.728+0000 7f0e94fb7700 1 --2- 172.21.15.92:0/1194722277 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0e70066f30 0x7f0e700693e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.728+0000 7f0e94fb7700 1 --2- 172.21.15.92:0/1194722277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e90159560 0x7f0e90157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.728+0000 7f0e94fb7700 1 --2- 172.21.15.92:0/1194722277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e900b0760 0x7f0e900aece0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.728+0000 7f0e94fb7700 1 --2- 172.21.15.92:0/1194722277 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e900b49a0 0x7f0e90169980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.728+0000 7f0e94fb7700 1 -- 172.21.15.92:0/1194722277 >> 172.21.15.92:0/1194722277 conn(0x7f0e900997f0 msgr2=0x7f0e900cd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.728+0000 7f0e94fb7700 1 -- 172.21.15.92:0/1194722277 shutdown_connections 2023-10-06T10:39:11.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.728+0000 7f0e94fb7700 1 -- 172.21.15.92:0/1194722277 wait complete. 2023-10-06T10:39:11.742 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-06T10:39:11.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.855+0000 7fa2fbc0d700 1 Processor -- start 2023-10-06T10:39:11.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.855+0000 7fa2fbc0d700 1 -- start start 2023-10-06T10:39:11.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.856+0000 7fa2fbc0d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa2f4159560 0x7fa2f40aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:11.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.856+0000 7fa2fbc0d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2f40b49a0 0x7fa2f40aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:11.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.856+0000 7fa2fbc0d700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2f40b0c60 con 0x7fa2f40b0760 2023-10-06T10:39:11.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.856+0000 7fa2fbc0d700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2f40b0f40 con 0x7fa2f4159560 2023-10-06T10:39:11.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.856+0000 7fa2fbc0d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f40b1220 con 0x7fa2f40b49a0 2023-10-06T10:39:11.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.856+0000 7fa2f91a8700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2f40b49a0 0x7fa2f40aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:11.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.856+0000 7fa2f99a9700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa2f4159560 0x7fa2f40aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:11.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.856+0000 7fa2f91a8700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2f40b49a0 0x7fa2f40aaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:33228/0 (socket says 172.21.15.92:33228) 2023-10-06T10:39:11.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.856+0000 7fa2f99a9700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa2f4159560 0x7fa2f40aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:34648/0 (socket says 172.21.15.92:34648) 2023-10-06T10:39:11.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.857+0000 7fa2f91a8700 1 -- 172.21.15.92:0/1823351720 learned_addr learned my addr 172.21.15.92:0/1823351720 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:11.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.856+0000 7fa2fa1aa700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fa2f40b0760 0x7fa2f40b3890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:50312/0 (socket says 172.21.15.92:50312) 2023-10-06T10:39:11.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.857+0000 7fa2f99a9700 1 -- 172.21.15.92:0/1823351720 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2f40b49a0 msgr2=0x7fa2f40aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:11.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.857+0000 7fa2f99a9700 1 --2- 172.21.15.92:0/1823351720 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2f40b49a0 0x7fa2f40aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.857+0000 7fa2f99a9700 1 -- 172.21.15.92:0/1823351720 >> v1:172.21.15.124:6789/0 conn(0x7fa2f40b0760 legacy=0x7fa2f40b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:11.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.857+0000 7fa2f99a9700 1 -- 172.21.15.92:0/1823351720 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f40b1500 con 0x7fa2f4159560 2023-10-06T10:39:11.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.857+0000 7fa2f91a8700 1 --2- 172.21.15.92:0/1823351720 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2f40b49a0 0x7fa2f40aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:39:11.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.857+0000 7fa2f89a7700 1 -- 172.21.15.92:0/1823351720 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3663183377 0 0) 0x7fa2f40b0c60 con 0x7fa2f40b0760 2023-10-06T10:39:11.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.857+0000 7fa2f99a9700 1 --2- 172.21.15.92:0/1823351720 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2f4159560 0x7fa2f40aa530 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0009eb0 tx=0x7fa2f0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe773ac5118de4bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:11.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.857+0000 7fa2f89a7700 1 -- 172.21.15.92:0/1823351720 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f000e070 con 0x7fa2f4159560 2023-10-06T10:39:11.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.857+0000 7fa2f89a7700 1 -- 172.21.15.92:0/1823351720 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f000bc30 con 0x7fa2f4159560 2023-10-06T10:39:11.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.857+0000 7fa2f89a7700 1 -- 172.21.15.92:0/1823351720 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0012430 con 0x7fa2f4159560 2023-10-06T10:39:11.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.858+0000 7fa2fbc0d700 1 -- 172.21.15.92:0/1823351720 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2f4159560 msgr2=0x7fa2f40aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:11.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.858+0000 7fa2fbc0d700 1 --2- 172.21.15.92:0/1823351720 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2f4159560 0x7fa2f40aa530 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0009eb0 tx=0x7fa2f0009780 comp rx=0 tx=0).stop 2023-10-06T10:39:11.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.858+0000 7fa2fbc0d700 1 -- 172.21.15.92:0/1823351720 shutdown_connections 2023-10-06T10:39:11.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.858+0000 7fa2fbc0d700 1 --2- 172.21.15.92:0/1823351720 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2f4159560 0x7fa2f40aa530 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.858+0000 7fa2fbc0d700 1 --2- 172.21.15.92:0/1823351720 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2f40b49a0 0x7fa2f40aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.858+0000 7fa2fbc0d700 1 -- 172.21.15.92:0/1823351720 >> 172.21.15.92:0/1823351720 conn(0x7fa2f40997f0 msgr2=0x7fa2f406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:11.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.858+0000 7fa2fbc0d700 1 -- 172.21.15.92:0/1823351720 shutdown_connections 2023-10-06T10:39:11.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.858+0000 7fa2fbc0d700 1 -- 172.21.15.92:0/1823351720 wait complete. 2023-10-06T10:39:11.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.859+0000 7fa2fbc0d700 1 Processor -- start 2023-10-06T10:39:11.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.859+0000 7fa2fbc0d700 1 -- start start 2023-10-06T10:39:11.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.860+0000 7fa2fbc0d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2f40b0760 0x7fa2f4156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:11.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.860+0000 7fa2fbc0d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2f40b49a0 0x7fa2f409bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:11.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.860+0000 7fa2fbc0d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2f4159560 0x7fa2f409a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:11.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.860+0000 7fa2fbc0d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f409aad0 con 0x7fa2f4159560 2023-10-06T10:39:11.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.860+0000 7fa2fbc0d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f409adb0 con 0x7fa2f40b0760 2023-10-06T10:39:11.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.860+0000 7fa2fbc0d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f409b090 con 0x7fa2f40b49a0 2023-10-06T10:39:11.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.860+0000 7fa2fa1aa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2f4159560 0x7fa2f409a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:11.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.860+0000 7fa2fa1aa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2f4159560 0x7fa2f409a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46728/0 (socket says 172.21.15.92:46728) 2023-10-06T10:39:11.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.860+0000 7fa2fa1aa700 1 -- 172.21.15.92:0/2287113273 learned_addr learned my addr 172.21.15.92:0/2287113273 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:11.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.860+0000 7fa2f99a9700 1 --2- 172.21.15.92:0/2287113273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2f40b0760 0x7fa2f4156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:11.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.860+0000 7fa2f91a8700 1 --2- 172.21.15.92:0/2287113273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2f40b49a0 0x7fa2f409bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:11.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.860+0000 7fa2fa1aa700 1 -- 172.21.15.92:0/2287113273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2f40b49a0 msgr2=0x7fa2f409bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:11.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.860+0000 7fa2fa1aa700 1 --2- 172.21.15.92:0/2287113273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2f40b49a0 0x7fa2f409bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.860+0000 7fa2fa1aa700 1 -- 172.21.15.92:0/2287113273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2f40b0760 msgr2=0x7fa2f4156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:11.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.860+0000 7fa2fa1aa700 1 --2- 172.21.15.92:0/2287113273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2f40b0760 0x7fa2f4156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:11.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.860+0000 7fa2fa1aa700 1 -- 172.21.15.92:0/2287113273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e40094f0 con 0x7fa2f4159560 2023-10-06T10:39:11.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.860+0000 7fa2fa1aa700 1 --2- 172.21.15.92:0/2287113273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2f4159560 0x7fa2f409a050 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec00c5e0 tx=0x7fa2ec007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:11.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.860+0000 7fa2e9ffb700 1 -- 172.21.15.92:0/2287113273 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec005a70 con 0x7fa2f4159560 2023-10-06T10:39:11.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.860+0000 7fa2e9ffb700 1 -- 172.21.15.92:0/2287113273 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2ec005bd0 con 0x7fa2f4159560 2023-10-06T10:39:11.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.860+0000 7fa2fbc0d700 1 -- 172.21.15.92:0/2287113273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f0009a20 con 0x7fa2f4159560 2023-10-06T10:39:11.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.860+0000 7fa2e9ffb700 1 -- 172.21.15.92:0/2287113273 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec0070d0 con 0x7fa2f4159560 2023-10-06T10:39:11.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.861+0000 7fa2fbc0d700 1 -- 172.21.15.92:0/2287113273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f409b650 con 0x7fa2f4159560 2023-10-06T10:39:11.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.861+0000 7fa2fbc0d700 1 -- 172.21.15.92:0/2287113273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f4064a90 con 0x7fa2f4159560 2023-10-06T10:39:11.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.862+0000 7fa2e9ffb700 1 -- 172.21.15.92:0/2287113273 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa2ec007440 con 0x7fa2f4159560 2023-10-06T10:39:11.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.862+0000 7fa2e9ffb700 1 --2- 172.21.15.92:0/2287113273 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa2d8066f30 0x7fa2d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:11.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.862+0000 7fa2e9ffb700 1 -- 172.21.15.92:0/2287113273 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fa2ec0838e0 con 0x7fa2f4159560 2023-10-06T10:39:11.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.862+0000 7fa2f99a9700 1 --2- 172.21.15.92:0/2287113273 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa2d8066f30 0x7fa2d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:11.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.863+0000 7fa2f99a9700 1 --2- 172.21.15.92:0/2287113273 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa2d8066f30 0x7fa2d80693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0009eb0 tx=0x7fa2f0019000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:11.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:11.866+0000 7fa2e9ffb700 1 -- 172.21.15.92:0/2287113273 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa2ec050c00 con 0x7fa2f4159560 2023-10-06T10:39:12.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.126+0000 7fa2fbc0d700 1 -- 172.21.15.92:0/2287113273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fa2f40b17e0 con 0x7fa2f4159560 2023-10-06T10:39:12.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.130+0000 7fa2e9ffb700 1 -- 172.21.15.92:0/2287113273 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa2ec017070 con 0x7fa2f4159560 2023-10-06T10:39:12.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.132+0000 7fa2fbc0d700 1 -- 172.21.15.92:0/2287113273 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa2d8066f30 msgr2=0x7fa2d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:12.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.132+0000 7fa2fbc0d700 1 --2- 172.21.15.92:0/2287113273 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa2d8066f30 0x7fa2d80693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0009eb0 tx=0x7fa2f0019000 comp rx=0 tx=0).stop 2023-10-06T10:39:12.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.132+0000 7fa2fbc0d700 1 -- 172.21.15.92:0/2287113273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2f4159560 msgr2=0x7fa2f409a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:12.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.132+0000 7fa2fbc0d700 1 --2- 172.21.15.92:0/2287113273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2f4159560 0x7fa2f409a050 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec00c5e0 tx=0x7fa2ec007d60 comp rx=0 tx=0).stop 2023-10-06T10:39:12.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.132+0000 7fa2fbc0d700 1 -- 172.21.15.92:0/2287113273 shutdown_connections 2023-10-06T10:39:12.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.132+0000 7fa2fbc0d700 1 --2- 172.21.15.92:0/2287113273 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa2d8066f30 0x7fa2d80693e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.132+0000 7fa2fbc0d700 1 --2- 172.21.15.92:0/2287113273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2f4159560 0x7fa2f409a050 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.132+0000 7fa2fbc0d700 1 --2- 172.21.15.92:0/2287113273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2f40b0760 0x7fa2f4156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.132+0000 7fa2fbc0d700 1 --2- 172.21.15.92:0/2287113273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2f40b49a0 0x7fa2f409bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.132+0000 7fa2fbc0d700 1 -- 172.21.15.92:0/2287113273 >> 172.21.15.92:0/2287113273 conn(0x7fa2f40997f0 msgr2=0x7fa2f4068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:12.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.132+0000 7fa2fbc0d700 1 -- 172.21.15.92:0/2287113273 shutdown_connections 2023-10-06T10:39:12.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.132+0000 7fa2fbc0d700 1 -- 172.21.15.92:0/2287113273 wait complete. 2023-10-06T10:39:12.147 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:39:12.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.259+0000 7f68b1a53700 1 Processor -- start 2023-10-06T10:39:12.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.259+0000 7f68b1a53700 1 -- start start 2023-10-06T10:39:12.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.260+0000 7f68b1a53700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f68ac159560 0x7f68ac0aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:12.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.260+0000 7f68b1a53700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f68ac0b0760 0x7f68ac0b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:12.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.260+0000 7f68b1a53700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68ac0b0c60 con 0x7f68ac0b49a0 2023-10-06T10:39:12.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.260+0000 7f68b1a53700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68ac0b0f40 con 0x7f68ac0b0760 2023-10-06T10:39:12.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.260+0000 7f68b1a53700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68ac0b1220 con 0x7f68ac159560 2023-10-06T10:39:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.260+0000 7f68ab7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f68ac0b0760 0x7f68ac0b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.260+0000 7f68a27fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f68ac0b49a0 0x7f68ac0aaa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33152/0 (socket says 172.21.15.92:33152) 2023-10-06T10:39:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.260+0000 7f68a27fc700 1 -- 172.21.15.92:0/1344390544 learned_addr learned my addr 172.21.15.92:0/1344390544 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.261+0000 7f68aaffd700 1 --2- 172.21.15.92:0/1344390544 >> v2:172.21.15.124:3300/0 conn(0x7f68ac159560 0x7f68ac0aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.261+0000 7f68ab7fe700 1 -- 172.21.15.92:0/1344390544 >> v2:172.21.15.124:3300/0 conn(0x7f68ac159560 msgr2=0x7f68ac0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.261+0000 7f68ab7fe700 1 --2- 172.21.15.92:0/1344390544 >> v2:172.21.15.124:3300/0 conn(0x7f68ac159560 0x7f68ac0aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.261+0000 7f68ab7fe700 1 -- 172.21.15.92:0/1344390544 >> v1:172.21.15.92:6789/0 conn(0x7f68ac0b49a0 legacy=0x7f68ac0aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.261+0000 7f68ab7fe700 1 -- 172.21.15.92:0/1344390544 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac0b1500 con 0x7f68ac0b0760 2023-10-06T10:39:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.261+0000 7f68aa7fc700 1 -- 172.21.15.92:0/1344390544 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2787486735 0 0) 0x7f68ac0b0c60 con 0x7f68ac0b49a0 2023-10-06T10:39:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.261+0000 7f68ab7fe700 1 --2- 172.21.15.92:0/1344390544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac0b0760 0x7f68ac0b1bc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009e60 tx=0x7f68a4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef14e83ecf0acbc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.261+0000 7f68aa7fc700 1 -- 172.21.15.92:0/1344390544 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a400e070 con 0x7f68ac0b0760 2023-10-06T10:39:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.261+0000 7f68aa7fc700 1 -- 172.21.15.92:0/1344390544 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a400bbf0 con 0x7f68ac0b0760 2023-10-06T10:39:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.261+0000 7f68aa7fc700 1 -- 172.21.15.92:0/1344390544 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4012430 con 0x7f68ac0b0760 2023-10-06T10:39:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.261+0000 7f68b1a53700 1 -- 172.21.15.92:0/1344390544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac0b0760 msgr2=0x7f68ac0b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.261+0000 7f68b1a53700 1 --2- 172.21.15.92:0/1344390544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac0b0760 0x7f68ac0b1bc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009e60 tx=0x7f68a4009780 comp rx=0 tx=0).stop 2023-10-06T10:39:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.261+0000 7f68b1a53700 1 -- 172.21.15.92:0/1344390544 shutdown_connections 2023-10-06T10:39:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.261+0000 7f68b1a53700 1 --2- 172.21.15.92:0/1344390544 >> v2:172.21.15.124:3300/0 conn(0x7f68ac159560 0x7f68ac0aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.261+0000 7f68b1a53700 1 --2- 172.21.15.92:0/1344390544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac0b0760 0x7f68ac0b1bc0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.261+0000 7f68b1a53700 1 -- 172.21.15.92:0/1344390544 >> 172.21.15.92:0/1344390544 conn(0x7f68ac0997f0 msgr2=0x7f68ac06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.261+0000 7f68b1a53700 1 -- 172.21.15.92:0/1344390544 shutdown_connections 2023-10-06T10:39:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.262+0000 7f68b1a53700 1 -- 172.21.15.92:0/1344390544 wait complete. 2023-10-06T10:39:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.263+0000 7f68b1a53700 1 Processor -- start 2023-10-06T10:39:12.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.263+0000 7f68b1a53700 1 -- start start 2023-10-06T10:39:12.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.263+0000 7f68b1a53700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68ac0b0760 0x7f68ac158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:12.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.263+0000 7f68b1a53700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68ac0b49a0 0x7f68ac15ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:12.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.263+0000 7f68b1a53700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac159560 0x7f68ac09a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:12.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.263+0000 7f68b1a53700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac169780 con 0x7f68ac159560 2023-10-06T10:39:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.263+0000 7f68b1a53700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac15aad0 con 0x7f68ac0b49a0 2023-10-06T10:39:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.263+0000 7f68b1a53700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac15adb0 con 0x7f68ac0b0760 2023-10-06T10:39:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.264+0000 7f68ab7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac159560 0x7f68ac09a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.264+0000 7f68ab7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac159560 0x7f68ac09a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46748/0 (socket says 172.21.15.92:46748) 2023-10-06T10:39:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.264+0000 7f68ab7fe700 1 -- 172.21.15.92:0/3530173968 learned_addr learned my addr 172.21.15.92:0/3530173968 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.264+0000 7f68a27fc700 1 --2- 172.21.15.92:0/3530173968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68ac0b49a0 0x7f68ac15ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.264+0000 7f68aaffd700 1 --2- 172.21.15.92:0/3530173968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68ac0b0760 0x7f68ac158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.264+0000 7f68ab7fe700 1 -- 172.21.15.92:0/3530173968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68ac0b0760 msgr2=0x7f68ac158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.264+0000 7f68ab7fe700 1 --2- 172.21.15.92:0/3530173968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68ac0b0760 0x7f68ac158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.264+0000 7f68ab7fe700 1 -- 172.21.15.92:0/3530173968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68ac0b49a0 msgr2=0x7f68ac15ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.264+0000 7f68ab7fe700 1 --2- 172.21.15.92:0/3530173968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68ac0b49a0 0x7f68ac15ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.264+0000 7f68ab7fe700 1 -- 172.21.15.92:0/3530173968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a4009a20 con 0x7f68ac159560 2023-10-06T10:39:12.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.264+0000 7f68ab7fe700 1 --2- 172.21.15.92:0/3530173968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac159560 0x7f68ac09a520 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f68a400b6c0 tx=0x7f68a4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:12.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.264+0000 7f68a37fe700 1 -- 172.21.15.92:0/3530173968 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4012e20 con 0x7f68ac159560 2023-10-06T10:39:12.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.264+0000 7f68a37fe700 1 -- 172.21.15.92:0/3530173968 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a401d630 con 0x7f68ac159560 2023-10-06T10:39:12.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.264+0000 7f68b1a53700 1 -- 172.21.15.92:0/3530173968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ac15b090 con 0x7f68ac159560 2023-10-06T10:39:12.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.264+0000 7f68a37fe700 1 -- 172.21.15.92:0/3530173968 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a401d9d0 con 0x7f68ac159560 2023-10-06T10:39:12.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.264+0000 7f68b1a53700 1 -- 172.21.15.92:0/3530173968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ac15cfd0 con 0x7f68ac159560 2023-10-06T10:39:12.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.265+0000 7f68b1a53700 1 -- 172.21.15.92:0/3530173968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68ac1698d0 con 0x7f68ac159560 2023-10-06T10:39:12.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.265+0000 7f68a37fe700 1 -- 172.21.15.92:0/3530173968 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f68a4024070 con 0x7f68ac159560 2023-10-06T10:39:12.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.266+0000 7f68a37fe700 1 --2- 172.21.15.92:0/3530173968 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f688c066f30 0x7f688c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:12.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.266+0000 7f68a37fe700 1 -- 172.21.15.92:0/3530173968 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f68a4086d10 con 0x7f68ac159560 2023-10-06T10:39:12.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.266+0000 7f68aaffd700 1 --2- 172.21.15.92:0/3530173968 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f688c066f30 0x7f688c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:12.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.266+0000 7f68aaffd700 1 --2- 172.21.15.92:0/3530173968 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f688c066f30 0x7f688c0693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f689c00ac00 tx=0x7f689c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:12.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.270+0000 7f68a37fe700 1 -- 172.21.15.92:0/3530173968 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68ac1698d0 con 0x7f68ac159560 2023-10-06T10:39:12.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.458+0000 7f68b1a53700 1 -- 172.21.15.92:0/3530173968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68ac1698d0 con 0x7f68ac159560 2023-10-06T10:39:12.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.459+0000 7f68a37fe700 1 -- 172.21.15.92:0/3530173968 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+13855 (secure 0 0 0) 0x7f68ac1698d0 con 0x7f68ac159560 2023-10-06T10:39:12.461 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:39:12.461 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":34,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:39:09.305540+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:39:12.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.461+0000 7f68b1a53700 1 -- 172.21.15.92:0/3530173968 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f688c066f30 msgr2=0x7f688c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:12.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.461+0000 7f68b1a53700 1 --2- 172.21.15.92:0/3530173968 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f688c066f30 0x7f688c0693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f689c00ac00 tx=0x7f689c008040 comp rx=0 tx=0).stop 2023-10-06T10:39:12.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.461+0000 7f68b1a53700 1 -- 172.21.15.92:0/3530173968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac159560 msgr2=0x7f68ac09a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:12.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.461+0000 7f68b1a53700 1 --2- 172.21.15.92:0/3530173968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac159560 0x7f68ac09a520 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f68a400b6c0 tx=0x7f68a4019000 comp rx=0 tx=0).stop 2023-10-06T10:39:12.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.461+0000 7f68b1a53700 1 -- 172.21.15.92:0/3530173968 shutdown_connections 2023-10-06T10:39:12.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.461+0000 7f68b1a53700 1 --2- 172.21.15.92:0/3530173968 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f688c066f30 0x7f688c0693e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.461+0000 7f68b1a53700 1 --2- 172.21.15.92:0/3530173968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac159560 0x7f68ac09a520 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.461+0000 7f68b1a53700 1 --2- 172.21.15.92:0/3530173968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68ac0b0760 0x7f68ac158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.461+0000 7f68b1a53700 1 --2- 172.21.15.92:0/3530173968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68ac0b49a0 0x7f68ac15ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.461+0000 7f68b1a53700 1 -- 172.21.15.92:0/3530173968 >> 172.21.15.92:0/3530173968 conn(0x7f68ac0997f0 msgr2=0x7f68ac0b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:12.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.461+0000 7f68b1a53700 1 -- 172.21.15.92:0/3530173968 shutdown_connections 2023-10-06T10:39:12.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.461+0000 7f68b1a53700 1 -- 172.21.15.92:0/3530173968 wait complete. 2023-10-06T10:39:12.475 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T10:38:09.680559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T10:38:10.120786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T10:39:12.475 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T10:39:12.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.591+0000 7f93e050b700 1 Processor -- start 2023-10-06T10:39:12.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.591+0000 7f93e050b700 1 -- start start 2023-10-06T10:39:12.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.592+0000 7f93e050b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93d81598a0 0x7f93d80ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:12.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.592+0000 7f93e050b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f93d80cbc70 0x7f93d80ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:12.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.592+0000 7f93e050b700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93d80cf5d0 con 0x7f93d81594d0 2023-10-06T10:39:12.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.592+0000 7f93e050b700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93d80cf8b0 con 0x7f93d80cbc70 2023-10-06T10:39:12.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.592+0000 7f93e050b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d80cfb90 con 0x7f93d81598a0 2023-10-06T10:39:12.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.592+0000 7f93ddaa6700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93d81598a0 0x7f93d80ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:12.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.592+0000 7f93deaa8700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f93d80cbc70 0x7f93d80ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:12.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.592+0000 7f93de2a7700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f93d81594d0 0x7f93d80cb560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:50320/0 (socket says 172.21.15.92:50320) 2023-10-06T10:39:12.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.592+0000 7f93de2a7700 1 -- 172.21.15.92:0/3509231095 learned_addr learned my addr 172.21.15.92:0/3509231095 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:12.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.592+0000 7f93deaa8700 1 -- 172.21.15.92:0/3509231095 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93d81598a0 msgr2=0x7f93d80ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:12.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.592+0000 7f93deaa8700 1 --2- 172.21.15.92:0/3509231095 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93d81598a0 0x7f93d80ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.592+0000 7f93deaa8700 1 -- 172.21.15.92:0/3509231095 >> v1:172.21.15.124:6789/0 conn(0x7f93d81594d0 legacy=0x7f93d80cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:12.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.592+0000 7f93deaa8700 1 -- 172.21.15.92:0/3509231095 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d80cfe70 con 0x7f93d80cbc70 2023-10-06T10:39:12.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.592+0000 7f93deaa8700 1 --2- 172.21.15.92:0/3509231095 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93d80cbc70 0x7f93d80ced20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009e60 tx=0x7f93d4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a96b95d906e3384 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:12.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.593+0000 7f93dd2a5700 1 -- 172.21.15.92:0/3509231095 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d400e070 con 0x7f93d80cbc70 2023-10-06T10:39:12.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.593+0000 7f93dd2a5700 1 -- 172.21.15.92:0/3509231095 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d400bb60 con 0x7f93d80cbc70 2023-10-06T10:39:12.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.593+0000 7f93dd2a5700 1 -- 172.21.15.92:0/3509231095 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d4012430 con 0x7f93d80cbc70 2023-10-06T10:39:12.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.593+0000 7f93e050b700 1 -- 172.21.15.92:0/3509231095 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93d80cbc70 msgr2=0x7f93d80ced20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:12.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.593+0000 7f93e050b700 1 --2- 172.21.15.92:0/3509231095 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93d80cbc70 0x7f93d80ced20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009e60 tx=0x7f93d4009780 comp rx=0 tx=0).stop 2023-10-06T10:39:12.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.593+0000 7f93e050b700 1 -- 172.21.15.92:0/3509231095 shutdown_connections 2023-10-06T10:39:12.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.593+0000 7f93e050b700 1 --2- 172.21.15.92:0/3509231095 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93d80cbc70 0x7f93d80ced20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.593+0000 7f93e050b700 1 --2- 172.21.15.92:0/3509231095 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93d81598a0 0x7f93d80ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.593+0000 7f93e050b700 1 -- 172.21.15.92:0/3509231095 >> 172.21.15.92:0/3509231095 conn(0x7f93d8089e50 msgr2=0x7f93d8099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:12.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.593+0000 7f93e050b700 1 -- 172.21.15.92:0/3509231095 shutdown_connections 2023-10-06T10:39:12.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.593+0000 7f93e050b700 1 -- 172.21.15.92:0/3509231095 wait complete. 2023-10-06T10:39:12.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.594+0000 7f93e050b700 1 Processor -- start 2023-10-06T10:39:12.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.594+0000 7f93e050b700 1 -- start start 2023-10-06T10:39:12.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.595+0000 7f93e050b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93d80cbc70 0x7f93d8149580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:12.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.595+0000 7f93e050b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93d81594d0 0x7f93d816a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:12.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.595+0000 7f93e050b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93d81598a0 0x7f93d80692d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:12.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.595+0000 7f93e050b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d80b5cf0 con 0x7f93d80cbc70 2023-10-06T10:39:12.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.595+0000 7f93e050b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d80b6180 con 0x7f93d81598a0 2023-10-06T10:39:12.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.595+0000 7f93e050b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d8149ac0 con 0x7f93d81594d0 2023-10-06T10:39:12.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.595+0000 7f93de2a7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93d80cbc70 0x7f93d8149580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:12.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.595+0000 7f93de2a7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93d80cbc70 0x7f93d8149580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46752/0 (socket says 172.21.15.92:46752) 2023-10-06T10:39:12.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.595+0000 7f93de2a7700 1 -- 172.21.15.92:0/224667808 learned_addr learned my addr 172.21.15.92:0/224667808 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:12.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.595+0000 7f93deaa8700 1 --2- 172.21.15.92:0/224667808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93d81598a0 0x7f93d80692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:12.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.595+0000 7f93ddaa6700 1 --2- 172.21.15.92:0/224667808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93d81594d0 0x7f93d816a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:12.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.596+0000 7f93de2a7700 1 -- 172.21.15.92:0/224667808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93d81594d0 msgr2=0x7f93d816a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:12.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.596+0000 7f93de2a7700 1 --2- 172.21.15.92:0/224667808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93d81594d0 0x7f93d816a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.596+0000 7f93de2a7700 1 -- 172.21.15.92:0/224667808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93d81598a0 msgr2=0x7f93d80692d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:12.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.596+0000 7f93de2a7700 1 --2- 172.21.15.92:0/224667808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93d81598a0 0x7f93d80692d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.596+0000 7f93de2a7700 1 -- 172.21.15.92:0/224667808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d4009a20 con 0x7f93d80cbc70 2023-10-06T10:39:12.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.596+0000 7f93deaa8700 1 --2- 172.21.15.92:0/224667808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93d81598a0 0x7f93d80692d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:12.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.596+0000 7f93de2a7700 1 --2- 172.21.15.92:0/224667808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93d80cbc70 0x7f93d8149580 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f93d000c7c0 tx=0x7f93d0007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:12.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.596+0000 7f93ce7fc700 1 -- 172.21.15.92:0/224667808 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d0005b30 con 0x7f93d80cbc70 2023-10-06T10:39:12.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.596+0000 7f93ce7fc700 1 -- 172.21.15.92:0/224667808 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d0005cb0 con 0x7f93d80cbc70 2023-10-06T10:39:12.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.596+0000 7f93e050b700 1 -- 172.21.15.92:0/224667808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d80b50e0 con 0x7f93d80cbc70 2023-10-06T10:39:12.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.596+0000 7f93ce7fc700 1 -- 172.21.15.92:0/224667808 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d0004a80 con 0x7f93d80cbc70 2023-10-06T10:39:12.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.596+0000 7f93e050b700 1 -- 172.21.15.92:0/224667808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d80b56a0 con 0x7f93d80cbc70 2023-10-06T10:39:12.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.597+0000 7f93e050b700 1 -- 172.21.15.92:0/224667808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d80b5fd0 con 0x7f93d80cbc70 2023-10-06T10:39:12.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.597+0000 7f93ce7fc700 1 -- 172.21.15.92:0/224667808 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f93d00070f0 con 0x7f93d80cbc70 2023-10-06T10:39:12.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.597+0000 7f93ce7fc700 1 --2- 172.21.15.92:0/224667808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f93bc066f30 0x7f93bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:12.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.598+0000 7f93ce7fc700 1 -- 172.21.15.92:0/224667808 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f93d0083980 con 0x7f93d80cbc70 2023-10-06T10:39:12.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.598+0000 7f93ddaa6700 1 --2- 172.21.15.92:0/224667808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f93bc066f30 0x7f93bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:12.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.598+0000 7f93ddaa6700 1 --2- 172.21.15.92:0/224667808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f93bc066f30 0x7f93bc0693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f93c8006f00 tx=0x7f93c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:12.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.602+0000 7f93ce7fc700 1 -- 172.21.15.92:0/224667808 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93d0050ca0 con 0x7f93d80cbc70 2023-10-06T10:39:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.783+0000 7f93e050b700 1 -- 172.21.15.92:0/224667808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f93d806f730 con 0x7f93d80cbc70 2023-10-06T10:39:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.783+0000 7f93ce7fc700 1 -- 172.21.15.92:0/224667808 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v34) v1 ==== 92+0+10 (secure 0 0 0) 0x7f93d0018030 con 0x7f93d80cbc70 2023-10-06T10:39:12.785 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T10:39:12.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.785+0000 7f93e050b700 1 -- 172.21.15.92:0/224667808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f93bc066f30 msgr2=0x7f93bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:12.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.785+0000 7f93e050b700 1 --2- 172.21.15.92:0/224667808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f93bc066f30 0x7f93bc0693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f93c8006f00 tx=0x7f93c8008040 comp rx=0 tx=0).stop 2023-10-06T10:39:12.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.785+0000 7f93e050b700 1 -- 172.21.15.92:0/224667808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93d80cbc70 msgr2=0x7f93d8149580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:12.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.785+0000 7f93e050b700 1 --2- 172.21.15.92:0/224667808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93d80cbc70 0x7f93d8149580 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f93d000c7c0 tx=0x7f93d0007f70 comp rx=0 tx=0).stop 2023-10-06T10:39:12.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.785+0000 7f93e050b700 1 -- 172.21.15.92:0/224667808 shutdown_connections 2023-10-06T10:39:12.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.785+0000 7f93e050b700 1 --2- 172.21.15.92:0/224667808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f93bc066f30 0x7f93bc0693e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.785+0000 7f93e050b700 1 --2- 172.21.15.92:0/224667808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93d81598a0 0x7f93d80692d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.785+0000 7f93e050b700 1 --2- 172.21.15.92:0/224667808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93d80cbc70 0x7f93d8149580 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.785+0000 7f93e050b700 1 --2- 172.21.15.92:0/224667808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93d81594d0 0x7f93d816a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.785+0000 7f93e050b700 1 -- 172.21.15.92:0/224667808 >> 172.21.15.92:0/224667808 conn(0x7f93d8089e50 msgr2=0x7f93d815cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:12.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.785+0000 7f93e050b700 1 -- 172.21.15.92:0/224667808 shutdown_connections 2023-10-06T10:39:12.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.785+0000 7f93e050b700 1 -- 172.21.15.92:0/224667808 wait complete. 2023-10-06T10:39:12.799 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T10:39:12.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.914+0000 7f6aad996700 1 Processor -- start 2023-10-06T10:39:12.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.914+0000 7f6aad996700 1 -- start start 2023-10-06T10:39:12.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.915+0000 7f6aad996700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6aa81594d0 0x7f6aa80cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:12.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.915+0000 7f6aad996700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa80cbec0 con 0x7f6aa81598a0 2023-10-06T10:39:12.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.915+0000 7f6aad996700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa80cf5d0 con 0x7f6aa80cbaa0 2023-10-06T10:39:12.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.915+0000 7f6aad996700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa80cf8b0 con 0x7f6aa81594d0 2023-10-06T10:39:12.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.915+0000 7f6a9e7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6aa81598a0 0x7f6aa80ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33156/0 (socket says 172.21.15.92:33156) 2023-10-06T10:39:12.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.915+0000 7f6a9e7fc700 1 -- 172.21.15.92:0/374751219 learned_addr learned my addr 172.21.15.92:0/374751219 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:12.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.915+0000 7f6aa6ffd700 1 --2- 172.21.15.92:0/374751219 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6aa81594d0 0x7f6aa80cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:12.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.915+0000 7f6aa67fc700 1 -- 172.21.15.92:0/374751219 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2872315617 0 0) 0x7f6aa80cbec0 con 0x7f6aa81598a0 2023-10-06T10:39:12.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.915+0000 7f6aa67fc700 1 -- 172.21.15.92:0/374751219 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a94003570 con 0x7f6aa81598a0 2023-10-06T10:39:12.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.915+0000 7f6aa67fc700 1 -- 172.21.15.92:0/374751219 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3881145945 0 0) 0x7f6a94003570 con 0x7f6aa81598a0 2023-10-06T10:39:12.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.915+0000 7f6aa67fc700 1 -- 172.21.15.92:0/374751219 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6aa81594d0 msgr2=0x7f6aa80cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:12.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.915+0000 7f6aa67fc700 1 --2- 172.21.15.92:0/374751219 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6aa81594d0 0x7f6aa80cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.915+0000 7f6aa67fc700 1 -- 172.21.15.92:0/374751219 >> v1:172.21.15.124:6789/0 conn(0x7f6aa80cbaa0 legacy=0x7f6aa80c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:12.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.915+0000 7f6aa67fc700 1 -- 172.21.15.92:0/374751219 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa80cfb90 con 0x7f6aa81598a0 2023-10-06T10:39:12.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.915+0000 7f6aa67fc700 1 -- 172.21.15.92:0/374751219 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f6a90001470 con 0x7f6aa81598a0 2023-10-06T10:39:12.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.915+0000 7f6aa67fc700 1 -- 172.21.15.92:0/374751219 >> v1:172.21.15.92:6789/0 conn(0x7f6aa81598a0 legacy=0x7f6aa80ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:12.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.916+0000 7f6aa67fc700 1 --2- 172.21.15.92:0/374751219 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a94003a90 0x7f6a94003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:12.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.916+0000 7f6aa67fc700 1 --2- 172.21.15.92:0/374751219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a94004810 0x7f6a94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:12.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.916+0000 7f6aa67fc700 1 --2- 172.21.15.92:0/374751219 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6aa81594d0 0x7f6a940051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:12.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.916+0000 7f6aa67fc700 1 -- 172.21.15.92:0/374751219 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa80cf8b0 con 0x7f6a94004810 2023-10-06T10:39:12.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.916+0000 7f6aa67fc700 1 -- 172.21.15.92:0/374751219 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9400bb60 con 0x7f6a94003a90 2023-10-06T10:39:12.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.916+0000 7f6aa67fc700 1 -- 172.21.15.92:0/374751219 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9400bd20 con 0x7f6aa81594d0 2023-10-06T10:39:12.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.916+0000 7f6a9e7fc700 1 --2- 172.21.15.92:0/374751219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a94004810 0x7f6a94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:12.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.916+0000 7f6aa6ffd700 1 --2- 172.21.15.92:0/374751219 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a94003a90 0x7f6a94003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:12.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.916+0000 7f6aa77fe700 1 --2- 172.21.15.92:0/374751219 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6aa81594d0 0x7f6a940051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:12.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.916+0000 7f6a9e7fc700 1 -- 172.21.15.92:0/374751219 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6aa81594d0 msgr2=0x7f6a940051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:12.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.916+0000 7f6a9e7fc700 1 --2- 172.21.15.92:0/374751219 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6aa81594d0 0x7f6a940051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.916+0000 7f6a9e7fc700 1 -- 172.21.15.92:0/374751219 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a94003a90 msgr2=0x7f6a94003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:12.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.916+0000 7f6a9e7fc700 1 --2- 172.21.15.92:0/374751219 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a94003a90 0x7f6a94003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.916+0000 7f6a9e7fc700 1 -- 172.21.15.92:0/374751219 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a94003570 con 0x7f6a94004810 2023-10-06T10:39:12.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.916+0000 7f6a9e7fc700 1 --2- 172.21.15.92:0/374751219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a94004810 0x7f6a94004c80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000c390 tx=0x7f6a90001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a151e08d0795c26c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:12.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.916+0000 7f6aa67fc700 1 -- 172.21.15.92:0/374751219 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9000cc60 con 0x7f6a94004810 2023-10-06T10:39:12.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.916+0000 7f6aa67fc700 1 -- 172.21.15.92:0/374751219 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9000cdc0 con 0x7f6a94004810 2023-10-06T10:39:12.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.916+0000 7f6aa67fc700 1 -- 172.21.15.92:0/374751219 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90005b50 con 0x7f6a94004810 2023-10-06T10:39:12.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.917+0000 7f6aad996700 1 -- 172.21.15.92:0/374751219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a94004810 msgr2=0x7f6a94004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:12.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.917+0000 7f6aad996700 1 --2- 172.21.15.92:0/374751219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a94004810 0x7f6a94004c80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000c390 tx=0x7f6a90001e50 comp rx=0 tx=0).stop 2023-10-06T10:39:12.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.917+0000 7f6aa77fe700 1 -- 172.21.15.92:0/374751219 reap_dead start 2023-10-06T10:39:12.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.917+0000 7f6aad996700 1 -- 172.21.15.92:0/374751219 shutdown_connections 2023-10-06T10:39:12.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.917+0000 7f6aad996700 1 -- 172.21.15.92:0/374751219 >> 172.21.15.92:0/374751219 conn(0x7f6aa8089e50 msgr2=0x7f6aa8099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:12.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.917+0000 7f6aad996700 1 -- 172.21.15.92:0/374751219 shutdown_connections 2023-10-06T10:39:12.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.917+0000 7f6aad996700 1 -- 172.21.15.92:0/374751219 wait complete. 2023-10-06T10:39:12.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.918+0000 7f6aad996700 1 Processor -- start 2023-10-06T10:39:12.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.918+0000 7f6aad996700 1 -- start start 2023-10-06T10:39:12.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6aad996700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a94004810 0x7f6aa8158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:12.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6aad996700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6aa8152270 0x7f6aa80c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:12.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6aad996700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa816a5a0 0x7f6aa8068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:12.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6aad996700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa8169b60 con 0x7f6aa816a5a0 2023-10-06T10:39:12.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6aad996700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa816a110 con 0x7f6a94004810 2023-10-06T10:39:12.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6aad996700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa80c9d80 con 0x7f6aa8152270 2023-10-06T10:39:12.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6aa77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa816a5a0 0x7f6aa8068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:12.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6aa77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa816a5a0 0x7f6aa8068370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46778/0 (socket says 172.21.15.92:46778) 2023-10-06T10:39:12.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6aa77fe700 1 -- 172.21.15.92:0/2141045383 learned_addr learned my addr 172.21.15.92:0/2141045383 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:12.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6aa6ffd700 1 --2- 172.21.15.92:0/2141045383 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a94004810 0x7f6aa8158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:12.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6aa77fe700 1 -- 172.21.15.92:0/2141045383 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6aa8152270 msgr2=0x7f6aa80c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:12.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6a9e7fc700 1 --2- 172.21.15.92:0/2141045383 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6aa8152270 0x7f6aa80c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:12.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6aa77fe700 1 --2- 172.21.15.92:0/2141045383 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6aa8152270 0x7f6aa80c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6aa77fe700 1 -- 172.21.15.92:0/2141045383 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a94004810 msgr2=0x7f6aa8158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:12.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6aa77fe700 1 --2- 172.21.15.92:0/2141045383 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a94004810 0x7f6aa8158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:12.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6aa77fe700 1 -- 172.21.15.92:0/2141045383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa80c9fe0 con 0x7f6aa816a5a0 2023-10-06T10:39:12.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6aa6ffd700 1 --2- 172.21.15.92:0/2141045383 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a94004810 0x7f6aa8158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:12.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6a9e7fc700 1 --2- 172.21.15.92:0/2141045383 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6aa8152270 0x7f6aa80c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:12.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6aa77fe700 1 --2- 172.21.15.92:0/2141045383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa816a5a0 0x7f6aa8068370 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8088cb0 tx=0x7f6aa0001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:12.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6a9f7fe700 1 -- 172.21.15.92:0/2141045383 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0019070 con 0x7f6aa816a5a0 2023-10-06T10:39:12.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6a9f7fe700 1 -- 172.21.15.92:0/2141045383 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa0006ba0 con 0x7f6aa816a5a0 2023-10-06T10:39:12.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6aad996700 1 -- 172.21.15.92:0/2141045383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa80ca2c0 con 0x7f6aa816a5a0 2023-10-06T10:39:12.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6a9f7fe700 1 -- 172.21.15.92:0/2141045383 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa000c7b0 con 0x7f6aa816a5a0 2023-10-06T10:39:12.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6aad996700 1 -- 172.21.15.92:0/2141045383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa816aa40 con 0x7f6aa816a5a0 2023-10-06T10:39:12.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.920+0000 7f6aad996700 1 -- 172.21.15.92:0/2141045383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa80ca4b0 con 0x7f6aa816a5a0 2023-10-06T10:39:12.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.921+0000 7f6a9f7fe700 1 -- 172.21.15.92:0/2141045383 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6aa000cba0 con 0x7f6aa816a5a0 2023-10-06T10:39:12.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.921+0000 7f6a9f7fe700 1 --2- 172.21.15.92:0/2141045383 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6a88066f30 0x7f6a880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:12.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.921+0000 7f6a9f7fe700 1 -- 172.21.15.92:0/2141045383 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f6aa00845c0 con 0x7f6aa816a5a0 2023-10-06T10:39:12.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.921+0000 7f6aa6ffd700 1 --2- 172.21.15.92:0/2141045383 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6a88066f30 0x7f6a880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:12.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.922+0000 7f6aa6ffd700 1 --2- 172.21.15.92:0/2141045383 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6a88066f30 0x7f6a880693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6a980093f0 tx=0x7f6a98008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:12.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:12.926+0000 7f6a9f7fe700 1 -- 172.21.15.92:0/2141045383 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6aa00518e0 con 0x7f6aa816a5a0 2023-10-06T10:39:13.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.108+0000 7f6aad996700 1 -- 172.21.15.92:0/2141045383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6aa8064a90 con 0x7f6aa816a5a0 2023-10-06T10:39:13.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.108+0000 7f6a9f7fe700 1 -- 172.21.15.92:0/2141045383 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6aa0054f00 con 0x7f6aa816a5a0 2023-10-06T10:39:13.110 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T10:39:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.111+0000 7f6aad996700 1 -- 172.21.15.92:0/2141045383 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6a88066f30 msgr2=0x7f6a880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.111+0000 7f6aad996700 1 --2- 172.21.15.92:0/2141045383 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6a88066f30 0x7f6a880693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6a980093f0 tx=0x7f6a98008040 comp rx=0 tx=0).stop 2023-10-06T10:39:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.111+0000 7f6aad996700 1 -- 172.21.15.92:0/2141045383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa816a5a0 msgr2=0x7f6aa8068370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.111+0000 7f6aad996700 1 --2- 172.21.15.92:0/2141045383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa816a5a0 0x7f6aa8068370 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8088cb0 tx=0x7f6aa0001640 comp rx=0 tx=0).stop 2023-10-06T10:39:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.111+0000 7f6aad996700 1 -- 172.21.15.92:0/2141045383 shutdown_connections 2023-10-06T10:39:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.111+0000 7f6aad996700 1 --2- 172.21.15.92:0/2141045383 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6a88066f30 0x7f6a880693e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.111+0000 7f6aad996700 1 --2- 172.21.15.92:0/2141045383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa816a5a0 0x7f6aa8068370 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.111+0000 7f6aad996700 1 --2- 172.21.15.92:0/2141045383 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a94004810 0x7f6aa8158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.111+0000 7f6aad996700 1 --2- 172.21.15.92:0/2141045383 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6aa8152270 0x7f6aa80c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.111+0000 7f6aad996700 1 -- 172.21.15.92:0/2141045383 >> 172.21.15.92:0/2141045383 conn(0x7f6aa8089e50 msgr2=0x7f6aa815bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.111+0000 7f6aad996700 1 -- 172.21.15.92:0/2141045383 shutdown_connections 2023-10-06T10:39:13.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.111+0000 7f6aad996700 1 -- 172.21.15.92:0/2141045383 wait complete. 2023-10-06T10:39:13.124 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T10:39:13.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.238+0000 7fe79649b700 1 Processor -- start 2023-10-06T10:39:13.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.238+0000 7fe79649b700 1 -- start start 2023-10-06T10:39:13.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.239+0000 7fe79649b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe790159440 0x7fe7900b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:13.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.239+0000 7fe79649b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe7900b35d0 0x7fe7900cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:13.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.239+0000 7fe79649b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7900cc170 0x7fe7900b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:13.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.239+0000 7fe79649b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7900cb7a0 con 0x7fe790159440 2023-10-06T10:39:13.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.239+0000 7fe79649b700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7900b0860 con 0x7fe7900b35d0 2023-10-06T10:39:13.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.239+0000 7fe79649b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7900cb150 con 0x7fe7900cc170 2023-10-06T10:39:13.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.239+0000 7fe795499700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe790159440 0x7fe7900b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:13.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.239+0000 7fe795499700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe790159440 0x7fe7900b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46788/0 (socket says 172.21.15.92:46788) 2023-10-06T10:39:13.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.239+0000 7fe795499700 1 -- 172.21.15.92:0/352649573 learned_addr learned my addr 172.21.15.92:0/352649573 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:13.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.239+0000 7fe794c98700 1 --2- 172.21.15.92:0/352649573 >> v2:172.21.15.124:3300/0 conn(0x7fe7900b35d0 0x7fe7900cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:13.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.239+0000 7fe795c9a700 1 --2- 172.21.15.92:0/352649573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7900cc170 0x7fe7900b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:13.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.239+0000 7fe795499700 1 -- 172.21.15.92:0/352649573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7900cc170 msgr2=0x7fe7900b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:13.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.239+0000 7fe795499700 1 --2- 172.21.15.92:0/352649573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7900cc170 0x7fe7900b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:13.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.239+0000 7fe795499700 1 -- 172.21.15.92:0/352649573 >> v2:172.21.15.124:3300/0 conn(0x7fe7900b35d0 msgr2=0x7fe7900cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:13.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.239+0000 7fe795499700 1 --2- 172.21.15.92:0/352649573 >> v2:172.21.15.124:3300/0 conn(0x7fe7900b35d0 0x7fe7900cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:13.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.239+0000 7fe795499700 1 -- 172.21.15.92:0/352649573 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7900cb430 con 0x7fe790159440 2023-10-06T10:39:13.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.239+0000 7fe794c98700 1 --2- 172.21.15.92:0/352649573 >> v2:172.21.15.124:3300/0 conn(0x7fe7900b35d0 0x7fe7900cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:13.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.240+0000 7fe795499700 1 --2- 172.21.15.92:0/352649573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe790159440 0x7fe7900b3090 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe7900d0220 tx=0x7fe78c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51f112920c6ae02 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:13.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.240+0000 7fe787fff700 1 -- 172.21.15.92:0/352649573 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c00b860 con 0x7fe790159440 2023-10-06T10:39:13.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.240+0000 7fe787fff700 1 -- 172.21.15.92:0/352649573 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78c00b9c0 con 0x7fe790159440 2023-10-06T10:39:13.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.240+0000 7fe787fff700 1 -- 172.21.15.92:0/352649573 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c00bd60 con 0x7fe790159440 2023-10-06T10:39:13.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.240+0000 7fe79649b700 1 -- 172.21.15.92:0/352649573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe790159440 msgr2=0x7fe7900b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:13.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.240+0000 7fe79649b700 1 --2- 172.21.15.92:0/352649573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe790159440 0x7fe7900b3090 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe7900d0220 tx=0x7fe78c009250 comp rx=0 tx=0).stop 2023-10-06T10:39:13.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.240+0000 7fe79649b700 1 -- 172.21.15.92:0/352649573 shutdown_connections 2023-10-06T10:39:13.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.240+0000 7fe79649b700 1 --2- 172.21.15.92:0/352649573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe790159440 0x7fe7900b3090 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:13.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.240+0000 7fe79649b700 1 --2- 172.21.15.92:0/352649573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7900cc170 0x7fe7900b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:13.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.240+0000 7fe79649b700 1 --2- 172.21.15.92:0/352649573 >> v2:172.21.15.124:3300/0 conn(0x7fe7900b35d0 0x7fe7900cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:13.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.240+0000 7fe79649b700 1 -- 172.21.15.92:0/352649573 >> 172.21.15.92:0/352649573 conn(0x7fe79006ece0 msgr2=0x7fe79006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:13.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.240+0000 7fe79649b700 1 -- 172.21.15.92:0/352649573 shutdown_connections 2023-10-06T10:39:13.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.240+0000 7fe79649b700 1 -- 172.21.15.92:0/352649573 wait complete. 2023-10-06T10:39:13.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.241+0000 7fe79649b700 1 Processor -- start 2023-10-06T10:39:13.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.241+0000 7fe79649b700 1 -- start start 2023-10-06T10:39:13.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.242+0000 7fe79649b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7900b35d0 0x7fe790153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:13.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.242+0000 7fe79649b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7900cc170 0x7fe7900c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:13.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.242+0000 7fe79649b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe790159440 0x7fe7900c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:13.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.242+0000 7fe79649b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7900c8080 con 0x7fe7900cc170 2023-10-06T10:39:13.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.242+0000 7fe79649b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe790154000 con 0x7fe7900b35d0 2023-10-06T10:39:13.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.242+0000 7fe79649b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7900c7470 con 0x7fe790159440 2023-10-06T10:39:13.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.242+0000 7fe794c98700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7900cc170 0x7fe7900c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:13.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.242+0000 7fe794c98700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7900cc170 0x7fe7900c5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46792/0 (socket says 172.21.15.92:46792) 2023-10-06T10:39:13.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.242+0000 7fe794c98700 1 -- 172.21.15.92:0/871312946 learned_addr learned my addr 172.21.15.92:0/871312946 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:13.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.242+0000 7fe795499700 1 --2- 172.21.15.92:0/871312946 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7900b35d0 0x7fe790153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:13.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.242+0000 7fe795c9a700 1 --2- 172.21.15.92:0/871312946 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe790159440 0x7fe7900c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:13.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.243+0000 7fe794c98700 1 -- 172.21.15.92:0/871312946 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe790159440 msgr2=0x7fe7900c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:13.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.243+0000 7fe794c98700 1 --2- 172.21.15.92:0/871312946 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe790159440 0x7fe7900c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:13.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.243+0000 7fe794c98700 1 -- 172.21.15.92:0/871312946 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7900b35d0 msgr2=0x7fe790153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:13.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.243+0000 7fe794c98700 1 --2- 172.21.15.92:0/871312946 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7900b35d0 0x7fe790153ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:13.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.243+0000 7fe794c98700 1 -- 172.21.15.92:0/871312946 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78c0094f0 con 0x7fe7900cc170 2023-10-06T10:39:13.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.243+0000 7fe795499700 1 --2- 172.21.15.92:0/871312946 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7900b35d0 0x7fe790153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:13.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.243+0000 7fe794c98700 1 --2- 172.21.15.92:0/871312946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7900cc170 0x7fe7900c5de0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe78000e900 tx=0x7fe780008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:13.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.243+0000 7fe7857fa700 1 -- 172.21.15.92:0/871312946 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78000eea0 con 0x7fe7900cc170 2023-10-06T10:39:13.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.243+0000 7fe7857fa700 1 -- 172.21.15.92:0/871312946 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78000c5e0 con 0x7fe7900cc170 2023-10-06T10:39:13.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.243+0000 7fe79649b700 1 -- 172.21.15.92:0/871312946 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7900c7750 con 0x7fe7900cc170 2023-10-06T10:39:13.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.243+0000 7fe7857fa700 1 -- 172.21.15.92:0/871312946 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78000c980 con 0x7fe7900cc170 2023-10-06T10:39:13.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.243+0000 7fe79649b700 1 -- 172.21.15.92:0/871312946 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7900c7d10 con 0x7fe7900cc170 2023-10-06T10:39:13.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.244+0000 7fe79649b700 1 -- 172.21.15.92:0/871312946 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7900c8230 con 0x7fe7900cc170 2023-10-06T10:39:13.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.244+0000 7fe7857fa700 1 -- 172.21.15.92:0/871312946 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe78000cd30 con 0x7fe7900cc170 2023-10-06T10:39:13.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.245+0000 7fe7857fa700 1 --2- 172.21.15.92:0/871312946 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe774066f30 0x7fe7740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:13.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.245+0000 7fe795499700 1 --2- 172.21.15.92:0/871312946 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe774066f30 0x7fe7740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:13.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.245+0000 7fe7857fa700 1 -- 172.21.15.92:0/871312946 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fe7800842c0 con 0x7fe7900cc170 2023-10-06T10:39:13.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.245+0000 7fe795499700 1 --2- 172.21.15.92:0/871312946 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe774066f30 0x7fe7740693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe7900d0220 tx=0x7fe78c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:13.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.249+0000 7fe7857fa700 1 -- 172.21.15.92:0/871312946 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7900c8230 con 0x7fe7900cc170 2023-10-06T10:39:13.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.437+0000 7fe79649b700 1 -- 172.21.15.92:0/871312946 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe79005f230 con 0x7fe7900cc170 2023-10-06T10:39:13.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.438+0000 7fe7857fa700 1 -- 172.21.15.92:0/871312946 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) v1 ==== 84+0+126 (secure 0 0 0) 0x7fe780054fd0 con 0x7fe7900cc170 2023-10-06T10:39:13.439 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:39:13.439 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T10:39:13.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:13.440 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-10-06T10:39:13.440 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-10-06T10:39:13.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:39:13.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:39:13.440 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-10-06T10:39:13.440 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-10-06T10:39:13.440 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:13.440 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T10:39:13.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.439+0000 7fe79649b700 1 -- 172.21.15.92:0/871312946 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe774066f30 msgr2=0x7fe7740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:13.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.439+0000 7fe79649b700 1 --2- 172.21.15.92:0/871312946 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe774066f30 0x7fe7740693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe7900d0220 tx=0x7fe78c017000 comp rx=0 tx=0).stop 2023-10-06T10:39:13.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.439+0000 7fe79649b700 1 -- 172.21.15.92:0/871312946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7900cc170 msgr2=0x7fe7900c5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:13.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.439+0000 7fe79649b700 1 --2- 172.21.15.92:0/871312946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7900cc170 0x7fe7900c5de0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe78000e900 tx=0x7fe780008040 comp rx=0 tx=0).stop 2023-10-06T10:39:13.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.440+0000 7fe79649b700 1 -- 172.21.15.92:0/871312946 shutdown_connections 2023-10-06T10:39:13.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.440+0000 7fe79649b700 1 --2- 172.21.15.92:0/871312946 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe774066f30 0x7fe7740693e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:13.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.440+0000 7fe79649b700 1 --2- 172.21.15.92:0/871312946 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe790159440 0x7fe7900c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:13.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.440+0000 7fe79649b700 1 --2- 172.21.15.92:0/871312946 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7900b35d0 0x7fe790153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:13.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.440+0000 7fe79649b700 1 --2- 172.21.15.92:0/871312946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7900cc170 0x7fe7900c5de0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:13.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.440+0000 7fe79649b700 1 -- 172.21.15.92:0/871312946 >> 172.21.15.92:0/871312946 conn(0x7fe79006ece0 msgr2=0x7fe790155170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:13.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.440+0000 7fe79649b700 1 -- 172.21.15.92:0/871312946 shutdown_connections 2023-10-06T10:39:13.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.440+0000 7fe79649b700 1 -- 172.21.15.92:0/871312946 wait complete. 2023-10-06T10:39:13.452 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T10:39:13.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T10:39:13.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.570+0000 7fed24e07700 1 Processor -- start 2023-10-06T10:39:13.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.570+0000 7fed24e07700 1 -- start start 2023-10-06T10:39:13.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.571+0000 7fed24e07700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fed200c32e0 0x7fed200cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:13.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.571+0000 7fed24e07700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed200c3ae0 0x7fed2015dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:13.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.571+0000 7fed24e07700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed200c8eb0 con 0x7fed200c36b0 2023-10-06T10:39:13.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.571+0000 7fed24e07700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed200c47e0 con 0x7fed200c32e0 2023-10-06T10:39:13.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.571+0000 7fed24e07700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed200c8930 con 0x7fed200c3ae0 2023-10-06T10:39:13.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.571+0000 7fed15d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fed200c36b0 0x7fed2015d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60168/0 (socket says 172.21.15.92:60168) 2023-10-06T10:39:13.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.571+0000 7fed15d9b700 1 -- 172.21.15.92:0/799859924 learned_addr learned my addr 172.21.15.92:0/799859924 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:13.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.571+0000 7fed1ed9d700 1 --2- 172.21.15.92:0/799859924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed200c3ae0 0x7fed2015dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:13.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.571+0000 7fed1e59c700 1 --2- 172.21.15.92:0/799859924 >> v2:172.21.15.124:3300/0 conn(0x7fed200c32e0 0x7fed200cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:13.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1dd9b700 1 -- 172.21.15.92:0/799859924 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 836221827 0 0) 0x7fed200c8eb0 con 0x7fed200c36b0 2023-10-06T10:39:13.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1dd9b700 1 -- 172.21.15.92:0/799859924 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed0c003570 con 0x7fed200c36b0 2023-10-06T10:39:13.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1dd9b700 1 -- 172.21.15.92:0/799859924 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3030972827 0 0) 0x7fed0c003570 con 0x7fed200c36b0 2023-10-06T10:39:13.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1dd9b700 1 -- 172.21.15.92:0/799859924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed200c3ae0 msgr2=0x7fed2015dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:13.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1dd9b700 1 --2- 172.21.15.92:0/799859924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed200c3ae0 0x7fed2015dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:13.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1dd9b700 1 -- 172.21.15.92:0/799859924 >> v2:172.21.15.124:3300/0 conn(0x7fed200c32e0 msgr2=0x7fed200cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:13.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1dd9b700 1 --2- 172.21.15.92:0/799859924 >> v2:172.21.15.124:3300/0 conn(0x7fed200c32e0 0x7fed200cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:13.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1dd9b700 1 -- 172.21.15.92:0/799859924 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed200cf230 con 0x7fed200c36b0 2023-10-06T10:39:13.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1dd9b700 1 -- 172.21.15.92:0/799859924 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fed08001470 con 0x7fed200c36b0 2023-10-06T10:39:13.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1ed9d700 1 --2- 172.21.15.92:0/799859924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed200c3ae0 0x7fed2015dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:39:13.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1dd9b700 1 -- 172.21.15.92:0/799859924 >> v1:172.21.15.92:6789/0 conn(0x7fed200c36b0 legacy=0x7fed2015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:13.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1dd9b700 1 --2- 172.21.15.92:0/799859924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed200c3ae0 0x7fed0c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:13.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1dd9b700 1 --2- 172.21.15.92:0/799859924 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed0c003fa0 0x7fed0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:13.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1dd9b700 1 --2- 172.21.15.92:0/799859924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed0c004d50 0x7fed0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:13.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1dd9b700 1 -- 172.21.15.92:0/799859924 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed200c8930 con 0x7fed0c004d50 2023-10-06T10:39:13.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1dd9b700 1 -- 172.21.15.92:0/799859924 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c00ba60 con 0x7fed0c003fa0 2023-10-06T10:39:13.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1dd9b700 1 -- 172.21.15.92:0/799859924 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c00bba0 con 0x7fed200c3ae0 2023-10-06T10:39:13.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1e59c700 1 --2- 172.21.15.92:0/799859924 >> v2:172.21.15.124:3300/0 conn(0x7fed200c32e0 0x7fed200cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:39:13.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1ed9d700 1 --2- 172.21.15.92:0/799859924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed0c004d50 0x7fed0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:13.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed15d9b700 1 --2- 172.21.15.92:0/799859924 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed0c003fa0 0x7fed0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:13.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1e59c700 1 --2- 172.21.15.92:0/799859924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed200c3ae0 0x7fed0c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:13.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1ed9d700 1 -- 172.21.15.92:0/799859924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed200c3ae0 msgr2=0x7fed0c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:13.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1ed9d700 1 --2- 172.21.15.92:0/799859924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed200c3ae0 0x7fed0c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:13.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1ed9d700 1 -- 172.21.15.92:0/799859924 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed0c003fa0 msgr2=0x7fed0c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:13.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1ed9d700 1 --2- 172.21.15.92:0/799859924 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed0c003fa0 0x7fed0c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:13.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1ed9d700 1 -- 172.21.15.92:0/799859924 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed18009a20 con 0x7fed0c004d50 2023-10-06T10:39:13.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.572+0000 7fed1ed9d700 1 --2- 172.21.15.92:0/799859924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed0c004d50 0x7fed0c0051c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fed1800e3c0 tx=0x7fed1800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a0cfd8a8d9e3b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:13.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.573+0000 7fed1dd9b700 1 -- 172.21.15.92:0/799859924 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18009c50 con 0x7fed0c004d50 2023-10-06T10:39:13.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.573+0000 7fed1dd9b700 1 -- 172.21.15.92:0/799859924 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1800f040 con 0x7fed0c004d50 2023-10-06T10:39:13.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.573+0000 7fed1dd9b700 1 -- 172.21.15.92:0/799859924 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1800eea0 con 0x7fed0c004d50 2023-10-06T10:39:13.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.573+0000 7fed24e07700 1 -- 172.21.15.92:0/799859924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed0c004d50 msgr2=0x7fed0c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:13.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.573+0000 7fed24e07700 1 --2- 172.21.15.92:0/799859924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed0c004d50 0x7fed0c0051c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fed1800e3c0 tx=0x7fed1800b040 comp rx=0 tx=0).stop 2023-10-06T10:39:13.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.573+0000 7fed1ed9d700 1 -- 172.21.15.92:0/799859924 reap_dead start 2023-10-06T10:39:13.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.573+0000 7fed24e07700 1 -- 172.21.15.92:0/799859924 shutdown_connections 2023-10-06T10:39:13.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.573+0000 7fed24e07700 1 -- 172.21.15.92:0/799859924 >> 172.21.15.92:0/799859924 conn(0x7fed20087fc0 msgr2=0x7fed200ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:13.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.573+0000 7fed24e07700 1 -- 172.21.15.92:0/799859924 shutdown_connections 2023-10-06T10:39:13.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.573+0000 7fed24e07700 1 -- 172.21.15.92:0/799859924 wait complete. 2023-10-06T10:39:13.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.574+0000 7fed24e07700 1 Processor -- start 2023-10-06T10:39:13.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.574+0000 7fed24e07700 1 -- start start 2023-10-06T10:39:13.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.575+0000 7fed24e07700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed20157670 0x7fed200b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:13.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.575+0000 7fed24e07700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed2016a710 0x7fed200c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:13.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.575+0000 7fed24e07700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed2016ab80 0x7fed200c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:13.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.575+0000 7fed24e07700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed200c5540 con 0x7fed2016ab80 2023-10-06T10:39:13.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.575+0000 7fed24e07700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed200c5820 con 0x7fed2016a710 2023-10-06T10:39:13.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.575+0000 7fed24e07700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed200c5b00 con 0x7fed20157670 2023-10-06T10:39:13.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.575+0000 7fed1ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed2016ab80 0x7fed200c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:13.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.575+0000 7fed1ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed2016ab80 0x7fed200c7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46064/0 (socket says 172.21.15.92:46064) 2023-10-06T10:39:13.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.575+0000 7fed1ed9d700 1 -- 172.21.15.92:0/158278449 learned_addr learned my addr 172.21.15.92:0/158278449 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:13.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.575+0000 7fed1e59c700 1 --2- 172.21.15.92:0/158278449 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed20157670 0x7fed200b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:13.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.575+0000 7fed15d9b700 1 --2- 172.21.15.92:0/158278449 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed2016a710 0x7fed200c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:13.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.575+0000 7fed1ed9d700 1 -- 172.21.15.92:0/158278449 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed20157670 msgr2=0x7fed200b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:13.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.575+0000 7fed1ed9d700 1 --2- 172.21.15.92:0/158278449 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed20157670 0x7fed200b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:13.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.575+0000 7fed1ed9d700 1 -- 172.21.15.92:0/158278449 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed2016a710 msgr2=0x7fed200c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:13.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.575+0000 7fed1ed9d700 1 --2- 172.21.15.92:0/158278449 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed2016a710 0x7fed200c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:13.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.575+0000 7fed1ed9d700 1 -- 172.21.15.92:0/158278449 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed200c5de0 con 0x7fed2016ab80 2023-10-06T10:39:13.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.576+0000 7fed15d9b700 1 --2- 172.21.15.92:0/158278449 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed2016a710 0x7fed200c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:13.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.576+0000 7fed1ed9d700 1 --2- 172.21.15.92:0/158278449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed2016ab80 0x7fed200c7bc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fed18016000 tx=0x7fed1801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:13.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.576+0000 7fed16ffd700 1 -- 172.21.15.92:0/158278449 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18009cd0 con 0x7fed2016ab80 2023-10-06T10:39:13.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.576+0000 7fed16ffd700 1 -- 172.21.15.92:0/158278449 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1801d630 con 0x7fed2016ab80 2023-10-06T10:39:13.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.576+0000 7fed24e07700 1 -- 172.21.15.92:0/158278449 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed200c60c0 con 0x7fed2016ab80 2023-10-06T10:39:13.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.576+0000 7fed16ffd700 1 -- 172.21.15.92:0/158278449 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1801d9f0 con 0x7fed2016ab80 2023-10-06T10:39:13.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.576+0000 7fed24e07700 1 -- 172.21.15.92:0/158278449 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed20157a40 con 0x7fed2016ab80 2023-10-06T10:39:13.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.577+0000 7fed24e07700 1 -- 172.21.15.92:0/158278449 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed200c62b0 con 0x7fed2016ab80 2023-10-06T10:39:13.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.577+0000 7fed16ffd700 1 -- 172.21.15.92:0/158278449 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fed18003040 con 0x7fed2016ab80 2023-10-06T10:39:13.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.577+0000 7fed16ffd700 1 --2- 172.21.15.92:0/158278449 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fed00066f30 0x7fed000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:13.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.578+0000 7fed16ffd700 1 -- 172.21.15.92:0/158278449 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fed18085650 con 0x7fed2016ab80 2023-10-06T10:39:13.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.578+0000 7fed1e59c700 1 --2- 172.21.15.92:0/158278449 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fed00066f30 0x7fed000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:13.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.578+0000 7fed1e59c700 1 --2- 172.21.15.92:0/158278449 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fed00066f30 0x7fed000693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fed10009390 tx=0x7fed10008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:13.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.582+0000 7fed16ffd700 1 -- 172.21.15.92:0/158278449 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed180529f0 con 0x7fed2016ab80 2023-10-06T10:39:13.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:13.764+0000 7fed24e07700 1 -- 172.21.15.92:0/158278449 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fed20064a90 con 0x7fed2016ab80 2023-10-06T10:39:14.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.290+0000 7fed16ffd700 1 -- 172.21.15.92:0/158278449 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7fed18056010 con 0x7fed2016ab80 2023-10-06T10:39:14.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.293+0000 7fed24e07700 1 -- 172.21.15.92:0/158278449 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fed00066f30 msgr2=0x7fed000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:14.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.293+0000 7fed24e07700 1 --2- 172.21.15.92:0/158278449 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fed00066f30 0x7fed000693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fed10009390 tx=0x7fed10008040 comp rx=0 tx=0).stop 2023-10-06T10:39:14.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.293+0000 7fed24e07700 1 -- 172.21.15.92:0/158278449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed2016ab80 msgr2=0x7fed200c7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:14.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.293+0000 7fed24e07700 1 --2- 172.21.15.92:0/158278449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed2016ab80 0x7fed200c7bc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fed18016000 tx=0x7fed1801a000 comp rx=0 tx=0).stop 2023-10-06T10:39:14.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.294+0000 7fed24e07700 1 -- 172.21.15.92:0/158278449 shutdown_connections 2023-10-06T10:39:14.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.294+0000 7fed24e07700 1 --2- 172.21.15.92:0/158278449 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fed00066f30 0x7fed000693e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:14.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.294+0000 7fed24e07700 1 --2- 172.21.15.92:0/158278449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed2016ab80 0x7fed200c7bc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:14.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.294+0000 7fed24e07700 1 --2- 172.21.15.92:0/158278449 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed20157670 0x7fed200b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:14.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.294+0000 7fed24e07700 1 --2- 172.21.15.92:0/158278449 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed2016a710 0x7fed200c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:14.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.294+0000 7fed24e07700 1 -- 172.21.15.92:0/158278449 >> 172.21.15.92:0/158278449 conn(0x7fed20087fc0 msgr2=0x7fed2015a800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:14.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.294+0000 7fed24e07700 1 -- 172.21.15.92:0/158278449 shutdown_connections 2023-10-06T10:39:14.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.294+0000 7fed24e07700 1 -- 172.21.15.92:0/158278449 wait complete. 2023-10-06T10:39:14.296 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-10-06T10:39:14.309 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T10:39:14.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.431+0000 7f2e95a3f700 1 Processor -- start 2023-10-06T10:39:14.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.431+0000 7f2e95a3f700 1 -- start start 2023-10-06T10:39:14.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.432+0000 7f2e95a3f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e900b2620 0x7f2e900cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:14.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.432+0000 7f2e95a3f700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2e900b29f0 0x7f2e900cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:14.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.432+0000 7f2e95a3f700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e900cc970 con 0x7f2e900ceaa0 2023-10-06T10:39:14.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.432+0000 7f2e95a3f700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e900cbd60 con 0x7f2e900b29f0 2023-10-06T10:39:14.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.432+0000 7f2e95a3f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e900cc040 con 0x7f2e900b2620 2023-10-06T10:39:14.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.432+0000 7f2e94a3d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e900b2620 0x7f2e900cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:14.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.432+0000 7f2e87fff700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2e900b29f0 0x7f2e900cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:14.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.432+0000 7f2e94a3d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e900b2620 0x7f2e900cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:49174/0 (socket says 172.21.15.92:49174) 2023-10-06T10:39:14.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.432+0000 7f2e87fff700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2e900b29f0 0x7f2e900cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:55060/0 (socket says 172.21.15.92:55060) 2023-10-06T10:39:14.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.432+0000 7f2e9523e700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f2e900ceaa0 0x7f2e900cef40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:36252/0 (socket says 172.21.15.92:36252) 2023-10-06T10:39:14.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.433+0000 7f2e94a3d700 1 -- 172.21.15.92:0/901486376 learned_addr learned my addr 172.21.15.92:0/901486376 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:14.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.433+0000 7f2e94a3d700 1 -- 172.21.15.92:0/901486376 >> v2:172.21.15.124:3300/0 conn(0x7f2e900b29f0 msgr2=0x7f2e900cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:14.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.433+0000 7f2e94a3d700 1 --2- 172.21.15.92:0/901486376 >> v2:172.21.15.124:3300/0 conn(0x7f2e900b29f0 0x7f2e900cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:14.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.433+0000 7f2e94a3d700 1 -- 172.21.15.92:0/901486376 >> v1:172.21.15.124:6789/0 conn(0x7f2e900ceaa0 legacy=0x7f2e900cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:14.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.433+0000 7f2e94a3d700 1 -- 172.21.15.92:0/901486376 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e900cc320 con 0x7f2e900b2620 2023-10-06T10:39:14.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.433+0000 7f2e87fff700 1 --2- 172.21.15.92:0/901486376 >> v2:172.21.15.124:3300/0 conn(0x7f2e900b29f0 0x7f2e900cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:39:14.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.433+0000 7f2e94a3d700 1 --2- 172.21.15.92:0/901486376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e900b2620 0x7f2e900cccb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009f70 tx=0x7f2e8c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90fc04b6df456229 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:14.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.433+0000 7f2e877fe700 1 -- 172.21.15.92:0/901486376 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c00e070 con 0x7f2e900b2620 2023-10-06T10:39:14.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.433+0000 7f2e877fe700 1 -- 172.21.15.92:0/901486376 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8c00ba20 con 0x7f2e900b2620 2023-10-06T10:39:14.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.433+0000 7f2e877fe700 1 -- 172.21.15.92:0/901486376 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c0092e0 con 0x7f2e900b2620 2023-10-06T10:39:14.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.434+0000 7f2e95a3f700 1 -- 172.21.15.92:0/901486376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e900b2620 msgr2=0x7f2e900cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:14.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.434+0000 7f2e95a3f700 1 --2- 172.21.15.92:0/901486376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e900b2620 0x7f2e900cccb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009f70 tx=0x7f2e8c009840 comp rx=0 tx=0).stop 2023-10-06T10:39:14.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.434+0000 7f2e95a3f700 1 -- 172.21.15.92:0/901486376 shutdown_connections 2023-10-06T10:39:14.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.434+0000 7f2e95a3f700 1 --2- 172.21.15.92:0/901486376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e900b2620 0x7f2e900cccb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:14.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.434+0000 7f2e95a3f700 1 --2- 172.21.15.92:0/901486376 >> v2:172.21.15.124:3300/0 conn(0x7f2e900b29f0 0x7f2e900cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:14.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.434+0000 7f2e95a3f700 1 -- 172.21.15.92:0/901486376 >> 172.21.15.92:0/901486376 conn(0x7f2e90099520 msgr2=0x7f2e9006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:14.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.434+0000 7f2e95a3f700 1 -- 172.21.15.92:0/901486376 shutdown_connections 2023-10-06T10:39:14.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.434+0000 7f2e95a3f700 1 -- 172.21.15.92:0/901486376 wait complete. 2023-10-06T10:39:14.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.435+0000 7f2e95a3f700 1 Processor -- start 2023-10-06T10:39:14.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.435+0000 7f2e95a3f700 1 -- start start 2023-10-06T10:39:14.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e95a3f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e900b2620 0x7f2e9015d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:14.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e95a3f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e900b29f0 0x7f2e9009c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:14.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e95a3f700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e900ceaa0 0x7f2e9009da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:14.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e95a3f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9009c770 con 0x7f2e900b29f0 2023-10-06T10:39:14.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e95a3f700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9009d570 con 0x7f2e900ceaa0 2023-10-06T10:39:14.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e95a3f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9009c960 con 0x7f2e900b2620 2023-10-06T10:39:14.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e87fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e900b29f0 0x7f2e9009c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:14.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e87fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e900b29f0 0x7f2e9009c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46076/0 (socket says 172.21.15.92:46076) 2023-10-06T10:39:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e87fff700 1 -- 172.21.15.92:0/2224491657 learned_addr learned my addr 172.21.15.92:0/2224491657 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e94a3d700 1 --2- 172.21.15.92:0/2224491657 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e900b2620 0x7f2e9015d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e9523e700 1 --2- 172.21.15.92:0/2224491657 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e900ceaa0 0x7f2e9009da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e87fff700 1 -- 172.21.15.92:0/2224491657 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e900b2620 msgr2=0x7f2e9015d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e87fff700 1 --2- 172.21.15.92:0/2224491657 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e900b2620 0x7f2e9015d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e87fff700 1 -- 172.21.15.92:0/2224491657 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e900ceaa0 msgr2=0x7f2e9009da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e87fff700 1 --2- 172.21.15.92:0/2224491657 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e900ceaa0 0x7f2e9009da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e87fff700 1 -- 172.21.15.92:0/2224491657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e80009420 con 0x7f2e900b29f0 2023-10-06T10:39:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e94a3d700 1 --2- 172.21.15.92:0/2224491657 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e900b2620 0x7f2e9015d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:39:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e87fff700 1 --2- 172.21.15.92:0/2224491657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e900b29f0 0x7f2e9009c230 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000ec80 tx=0x7f2e800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e84ff9700 1 -- 172.21.15.92:0/2224491657 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8000c920 con 0x7f2e900b29f0 2023-10-06T10:39:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e84ff9700 1 -- 172.21.15.92:0/2224491657 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8000ca80 con 0x7f2e900b29f0 2023-10-06T10:39:14.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e95a3f700 1 -- 172.21.15.92:0/2224491657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8c009ae0 con 0x7f2e900b29f0 2023-10-06T10:39:14.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e84ff9700 1 -- 172.21.15.92:0/2224491657 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8000ce00 con 0x7f2e900b29f0 2023-10-06T10:39:14.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.436+0000 7f2e95a3f700 1 -- 172.21.15.92:0/2224491657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9009cf20 con 0x7f2e900b29f0 2023-10-06T10:39:14.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.437+0000 7f2e95a3f700 1 -- 172.21.15.92:0/2224491657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9005f230 con 0x7f2e900b29f0 2023-10-06T10:39:14.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.438+0000 7f2e84ff9700 1 -- 172.21.15.92:0/2224491657 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2e80013680 con 0x7f2e900b29f0 2023-10-06T10:39:14.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.438+0000 7f2e84ff9700 1 --2- 172.21.15.92:0/2224491657 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2e7c066f30 0x7f2e7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:14.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.438+0000 7f2e84ff9700 1 -- 172.21.15.92:0/2224491657 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5896+0+0 (secure 0 0 0) 0x7f2e80084d10 con 0x7f2e900b29f0 2023-10-06T10:39:14.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.438+0000 7f2e94a3d700 1 --2- 172.21.15.92:0/2224491657 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2e7c066f30 0x7f2e7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:14.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.439+0000 7f2e94a3d700 1 --2- 172.21.15.92:0/2224491657 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2e7c066f30 0x7f2e7c0693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009880 tx=0x7f2e8c018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:14.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.443+0000 7f2e84ff9700 1 -- 172.21.15.92:0/2224491657 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e80051ec0 con 0x7f2e900b29f0 2023-10-06T10:39:14.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:14.627+0000 7f2e95a3f700 1 -- 172.21.15.92:0/2224491657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2e900b54d0 con 0x7f2e900b29f0 2023-10-06T10:39:15.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.289+0000 7f2e84ff9700 1 -- 172.21.15.92:0/2224491657 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2e800554e0 con 0x7f2e900b29f0 2023-10-06T10:39:15.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.295+0000 7f2e95a3f700 1 -- 172.21.15.92:0/2224491657 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2e7c066f30 msgr2=0x7f2e7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:15.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.295+0000 7f2e95a3f700 1 --2- 172.21.15.92:0/2224491657 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2e7c066f30 0x7f2e7c0693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009880 tx=0x7f2e8c018000 comp rx=0 tx=0).stop 2023-10-06T10:39:15.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.295+0000 7f2e95a3f700 1 -- 172.21.15.92:0/2224491657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e900b29f0 msgr2=0x7f2e9009c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:15.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.295+0000 7f2e95a3f700 1 --2- 172.21.15.92:0/2224491657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e900b29f0 0x7f2e9009c230 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000ec80 tx=0x7f2e800091a0 comp rx=0 tx=0).stop 2023-10-06T10:39:15.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.295+0000 7f2e95a3f700 1 -- 172.21.15.92:0/2224491657 shutdown_connections 2023-10-06T10:39:15.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.296+0000 7f2e95a3f700 1 --2- 172.21.15.92:0/2224491657 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2e7c066f30 0x7f2e7c0693e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:15.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.296+0000 7f2e95a3f700 1 --2- 172.21.15.92:0/2224491657 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e900ceaa0 0x7f2e9009da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:15.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.296+0000 7f2e95a3f700 1 --2- 172.21.15.92:0/2224491657 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e900b2620 0x7f2e9015d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:15.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.296+0000 7f2e95a3f700 1 --2- 172.21.15.92:0/2224491657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e900b29f0 0x7f2e9009c230 secure :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000ec80 tx=0x7f2e800091a0 comp rx=0 tx=0).stop 2023-10-06T10:39:15.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.296+0000 7f2e95a3f700 1 -- 172.21.15.92:0/2224491657 >> 172.21.15.92:0/2224491657 conn(0x7f2e90099520 msgr2=0x7f2e90068bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:15.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.302+0000 7f2e95a3f700 1 -- 172.21.15.92:0/2224491657 shutdown_connections 2023-10-06T10:39:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.303+0000 7f2e95a3f700 1 -- 172.21.15.92:0/2224491657 wait complete. 2023-10-06T10:39:15.307 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-10-06T10:39:15.322 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T10:39:15.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.438+0000 7f52ffc26700 1 Processor -- start 2023-10-06T10:39:15.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.438+0000 7f52ffc26700 1 -- start start 2023-10-06T10:39:15.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.439+0000 7f52ffc26700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f52f81598a0 0x7f52f80ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:15.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.439+0000 7f52ffc26700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f80cf5d0 con 0x7f52f81594d0 2023-10-06T10:39:15.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.439+0000 7f52ffc26700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f80cf8b0 con 0x7f52f80cbc70 2023-10-06T10:39:15.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.439+0000 7f52ffc26700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f80cfb90 con 0x7f52f81598a0 2023-10-06T10:39:15.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.439+0000 7f52fd9c2700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f52f81594d0 0x7f52f80cb560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60176/0 (socket says 172.21.15.92:60176) 2023-10-06T10:39:15.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.439+0000 7f52fd9c2700 1 -- 172.21.15.92:0/4116838182 learned_addr learned my addr 172.21.15.92:0/4116838182 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:15.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.439+0000 7f52fd1c1700 1 --2- 172.21.15.92:0/4116838182 >> v2:172.21.15.124:3300/0 conn(0x7f52f81598a0 0x7f52f80ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:15.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.440+0000 7f52fc9c0700 1 -- 172.21.15.92:0/4116838182 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1785402063 0 0) 0x7f52f80cf5d0 con 0x7f52f81594d0 2023-10-06T10:39:15.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.440+0000 7f52fc9c0700 1 -- 172.21.15.92:0/4116838182 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52e4003570 con 0x7f52f81594d0 2023-10-06T10:39:15.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.440+0000 7f52fd1c1700 1 -- 172.21.15.92:0/4116838182 >> v1:172.21.15.124:6789/0 conn(0x7f52f80cbc70 legacy=0x7f52f80ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:15.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.440+0000 7f52fd1c1700 1 -- 172.21.15.92:0/4116838182 >> v1:172.21.15.92:6789/0 conn(0x7f52f81594d0 legacy=0x7f52f80cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:15.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.440+0000 7f52fd1c1700 1 -- 172.21.15.92:0/4116838182 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f80cfe70 con 0x7f52f81598a0 2023-10-06T10:39:15.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.440+0000 7f52fd1c1700 1 --2- 172.21.15.92:0/4116838182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52f81598a0 0x7f52f80ce7e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f52e80098e0 tx=0x7f52e80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b00958dfab331fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:15.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.440+0000 7f52fc9c0700 1 -- 172.21.15.92:0/4116838182 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e800b640 con 0x7f52f81598a0 2023-10-06T10:39:15.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.440+0000 7f52fc9c0700 1 -- 172.21.15.92:0/4116838182 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e800b7a0 con 0x7f52f81598a0 2023-10-06T10:39:15.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.440+0000 7f52fc9c0700 1 -- 172.21.15.92:0/4116838182 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e800bb20 con 0x7f52f81598a0 2023-10-06T10:39:15.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.441+0000 7f52ffc26700 1 -- 172.21.15.92:0/4116838182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52f81598a0 msgr2=0x7f52f80ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:15.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.441+0000 7f52ffc26700 1 --2- 172.21.15.92:0/4116838182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52f81598a0 0x7f52f80ce7e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f52e80098e0 tx=0x7f52e80091a0 comp rx=0 tx=0).stop 2023-10-06T10:39:15.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.441+0000 7f52ffc26700 1 -- 172.21.15.92:0/4116838182 shutdown_connections 2023-10-06T10:39:15.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.441+0000 7f52ffc26700 1 --2- 172.21.15.92:0/4116838182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52f81598a0 0x7f52f80ce7e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:15.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.441+0000 7f52ffc26700 1 -- 172.21.15.92:0/4116838182 >> 172.21.15.92:0/4116838182 conn(0x7f52f8089e50 msgr2=0x7f52f8099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:15.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.441+0000 7f52ffc26700 1 -- 172.21.15.92:0/4116838182 shutdown_connections 2023-10-06T10:39:15.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.441+0000 7f52ffc26700 1 -- 172.21.15.92:0/4116838182 wait complete. 2023-10-06T10:39:15.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.442+0000 7f52ffc26700 1 Processor -- start 2023-10-06T10:39:15.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.442+0000 7f52ffc26700 1 -- start start 2023-10-06T10:39:15.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.442+0000 7f52ffc26700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f80cbc70 0x7f52f81490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:15.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.442+0000 7f52ffc26700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52f81594d0 0x7f52f80c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:15.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.442+0000 7f52ffc26700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52f81598a0 0x7f52f80d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:15.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.442+0000 7f52ffc26700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f8151fe0 con 0x7f52f80cbc70 2023-10-06T10:39:15.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.442+0000 7f52ffc26700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f81513d0 con 0x7f52f81594d0 2023-10-06T10:39:15.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.442+0000 7f52ffc26700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f81516b0 con 0x7f52f81598a0 2023-10-06T10:39:15.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.443+0000 7f52fd9c2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f80cbc70 0x7f52f81490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:15.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.443+0000 7f52fd9c2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f80cbc70 0x7f52f81490b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46080/0 (socket says 172.21.15.92:46080) 2023-10-06T10:39:15.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.443+0000 7f52fd9c2700 1 -- 172.21.15.92:0/2074486756 learned_addr learned my addr 172.21.15.92:0/2074486756 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:15.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.443+0000 7f52fe1c3700 1 --2- 172.21.15.92:0/2074486756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52f81598a0 0x7f52f80d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:15.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.443+0000 7f52fd1c1700 1 --2- 172.21.15.92:0/2074486756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52f81594d0 0x7f52f80c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:15.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.443+0000 7f52fd9c2700 1 -- 172.21.15.92:0/2074486756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52f81598a0 msgr2=0x7f52f80d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:15.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.443+0000 7f52fd9c2700 1 --2- 172.21.15.92:0/2074486756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52f81598a0 0x7f52f80d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:15.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.443+0000 7f52fd9c2700 1 -- 172.21.15.92:0/2074486756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52f81594d0 msgr2=0x7f52f80c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:15.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.443+0000 7f52fd9c2700 1 --2- 172.21.15.92:0/2074486756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52f81594d0 0x7f52f80c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:15.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.443+0000 7f52fd9c2700 1 -- 172.21.15.92:0/2074486756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e8009420 con 0x7f52f80cbc70 2023-10-06T10:39:15.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.443+0000 7f52fd9c2700 1 --2- 172.21.15.92:0/2074486756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f80cbc70 0x7f52f81490b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f52f400cd30 tx=0x7f52f4001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:15.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.443+0000 7f52edffb700 1 -- 172.21.15.92:0/2074486756 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f400ce90 con 0x7f52f80cbc70 2023-10-06T10:39:15.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.443+0000 7f52edffb700 1 -- 172.21.15.92:0/2074486756 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f400aa60 con 0x7f52f80cbc70 2023-10-06T10:39:15.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.443+0000 7f52edffb700 1 -- 172.21.15.92:0/2074486756 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f400ae40 con 0x7f52f80cbc70 2023-10-06T10:39:15.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.443+0000 7f52ffc26700 1 -- 172.21.15.92:0/2074486756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f8151990 con 0x7f52f80cbc70 2023-10-06T10:39:15.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.444+0000 7f52ffc26700 1 -- 172.21.15.92:0/2074486756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f81495f0 con 0x7f52f80cbc70 2023-10-06T10:39:15.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.444+0000 7f52ffc26700 1 -- 172.21.15.92:0/2074486756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f8064a90 con 0x7f52f80cbc70 2023-10-06T10:39:15.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.445+0000 7f52edffb700 1 -- 172.21.15.92:0/2074486756 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f52f4007300 con 0x7f52f80cbc70 2023-10-06T10:39:15.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.445+0000 7f52edffb700 1 --2- 172.21.15.92:0/2074486756 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f52dc066f60 0x7f52dc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:15.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.445+0000 7f52edffb700 1 -- 172.21.15.92:0/2074486756 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6278+0+0 (secure 0 0 0) 0x7f52f4084bd0 con 0x7f52f80cbc70 2023-10-06T10:39:15.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.445+0000 7f52fd1c1700 1 --2- 172.21.15.92:0/2074486756 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f52dc066f60 0x7f52dc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:15.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.446+0000 7f52fd1c1700 1 --2- 172.21.15.92:0/2074486756 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f52dc066f60 0x7f52dc069410 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f52e80098e0 tx=0x7f52e8017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:15.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.450+0000 7f52edffb700 1 -- 172.21.15.92:0/2074486756 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52f4051b60 con 0x7f52f80cbc70 2023-10-06T10:39:15.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:15.674+0000 7f52ffc26700 1 -- 172.21.15.92:0/2074486756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f52f80c66a0 con 0x7f52f80cbc70 2023-10-06T10:39:16.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.319+0000 7f52edffb700 1 -- 172.21.15.92:0/2074486756 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f52f4055180 con 0x7f52f80cbc70 2023-10-06T10:39:16.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.323+0000 7f52ffc26700 1 -- 172.21.15.92:0/2074486756 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f52dc066f60 msgr2=0x7f52dc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:16.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.323+0000 7f52ffc26700 1 --2- 172.21.15.92:0/2074486756 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f52dc066f60 0x7f52dc069410 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f52e80098e0 tx=0x7f52e8017000 comp rx=0 tx=0).stop 2023-10-06T10:39:16.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.323+0000 7f52ffc26700 1 -- 172.21.15.92:0/2074486756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f80cbc70 msgr2=0x7f52f81490b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:16.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.323+0000 7f52ffc26700 1 --2- 172.21.15.92:0/2074486756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f80cbc70 0x7f52f81490b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f52f400cd30 tx=0x7f52f4001b10 comp rx=0 tx=0).stop 2023-10-06T10:39:16.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.324+0000 7f52ffc26700 1 -- 172.21.15.92:0/2074486756 shutdown_connections 2023-10-06T10:39:16.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.324+0000 7f52ffc26700 1 --2- 172.21.15.92:0/2074486756 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f52dc066f60 0x7f52dc069410 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:16.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.324+0000 7f52ffc26700 1 --2- 172.21.15.92:0/2074486756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52f81598a0 0x7f52f80d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:16.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.324+0000 7f52ffc26700 1 --2- 172.21.15.92:0/2074486756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f80cbc70 0x7f52f81490b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:16.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.324+0000 7f52ffc26700 1 --2- 172.21.15.92:0/2074486756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52f81594d0 0x7f52f80c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:16.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.324+0000 7f52ffc26700 1 -- 172.21.15.92:0/2074486756 >> 172.21.15.92:0/2074486756 conn(0x7f52f8089e50 msgr2=0x7f52f814a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:16.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.325+0000 7f52ffc26700 1 -- 172.21.15.92:0/2074486756 shutdown_connections 2023-10-06T10:39:16.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.326+0000 7f52ffc26700 1 -- 172.21.15.92:0/2074486756 wait complete. 2023-10-06T10:39:16.329 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 6 and data pool 7 2023-10-06T10:39:16.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:39:16.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.454+0000 7f90e5174700 1 Processor -- start 2023-10-06T10:39:16.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.454+0000 7f90e5174700 1 -- start start 2023-10-06T10:39:16.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.455+0000 7f90e5174700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f90e00b1830 0x7f90e00c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:16.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.455+0000 7f90e5174700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e015ad50 con 0x7f90e01563e0 2023-10-06T10:39:16.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.455+0000 7f90e5174700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e0159e60 con 0x7f90e0156010 2023-10-06T10:39:16.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.455+0000 7f90e5174700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e015a140 con 0x7f90e00b1830 2023-10-06T10:39:16.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.455+0000 7f90df7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f90e01563e0 0x7f90e00b1120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60186/0 (socket says 172.21.15.92:60186) 2023-10-06T10:39:16.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.455+0000 7f90df7fe700 1 -- 172.21.15.92:0/2372398155 learned_addr learned my addr 172.21.15.92:0/2372398155 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:16.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.455+0000 7f90e4973700 1 --2- 172.21.15.92:0/2372398155 >> v2:172.21.15.124:3300/0 conn(0x7f90e00b1830 0x7f90e00c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:16.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.455+0000 7f90deffd700 1 -- 172.21.15.92:0/2372398155 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3193527839 0 0) 0x7f90e015ad50 con 0x7f90e01563e0 2023-10-06T10:39:16.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.455+0000 7f90deffd700 1 -- 172.21.15.92:0/2372398155 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90c8003570 con 0x7f90e01563e0 2023-10-06T10:39:16.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.456+0000 7f90deffd700 1 -- 172.21.15.92:0/2372398155 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 427051328 0 0) 0x7f90e0159e60 con 0x7f90e0156010 2023-10-06T10:39:16.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.456+0000 7f90deffd700 1 -- 172.21.15.92:0/2372398155 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90e015ad50 con 0x7f90e0156010 2023-10-06T10:39:16.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.456+0000 7f90deffd700 1 -- 172.21.15.92:0/2372398155 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3637871621 0 0) 0x7f90c8003570 con 0x7f90e01563e0 2023-10-06T10:39:16.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.456+0000 7f90e4973700 1 -- 172.21.15.92:0/2372398155 >> v1:172.21.15.124:6789/0 conn(0x7f90e0156010 legacy=0x7f90e00b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:16.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.456+0000 7f90e4973700 1 -- 172.21.15.92:0/2372398155 >> v1:172.21.15.92:6789/0 conn(0x7f90e01563e0 legacy=0x7f90e00b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:16.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.456+0000 7f90e4973700 1 -- 172.21.15.92:0/2372398155 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e015a420 con 0x7f90e00b1830 2023-10-06T10:39:16.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.456+0000 7f90e4973700 1 --2- 172.21.15.92:0/2372398155 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90e00b1830 0x7f90e00c9c00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009e60 tx=0x7f90d8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e7e2e89824a833c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:16.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.456+0000 7f90deffd700 1 -- 172.21.15.92:0/2372398155 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d800e070 con 0x7f90e00b1830 2023-10-06T10:39:16.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.456+0000 7f90deffd700 1 -- 172.21.15.92:0/2372398155 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d800b7a0 con 0x7f90e00b1830 2023-10-06T10:39:16.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.456+0000 7f90deffd700 1 -- 172.21.15.92:0/2372398155 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4001470 con 0x7f90e00b1830 2023-10-06T10:39:16.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.456+0000 7f90e5174700 1 -- 172.21.15.92:0/2372398155 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90e00b1830 msgr2=0x7f90e00c9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:16.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.456+0000 7f90e5174700 1 --2- 172.21.15.92:0/2372398155 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90e00b1830 0x7f90e00c9c00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009e60 tx=0x7f90d8009780 comp rx=0 tx=0).stop 2023-10-06T10:39:16.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.457+0000 7f90e5174700 1 -- 172.21.15.92:0/2372398155 shutdown_connections 2023-10-06T10:39:16.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.457+0000 7f90e5174700 1 --2- 172.21.15.92:0/2372398155 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90e00b1830 0x7f90e00c9c00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:16.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.457+0000 7f90e5174700 1 -- 172.21.15.92:0/2372398155 >> 172.21.15.92:0/2372398155 conn(0x7f90e0069190 msgr2=0x7f90e00aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:16.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.457+0000 7f90e5174700 1 -- 172.21.15.92:0/2372398155 shutdown_connections 2023-10-06T10:39:16.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.457+0000 7f90e5174700 1 -- 172.21.15.92:0/2372398155 wait complete. 2023-10-06T10:39:16.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.458+0000 7f90e5174700 1 Processor -- start 2023-10-06T10:39:16.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.458+0000 7f90e5174700 1 -- start start 2023-10-06T10:39:16.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.458+0000 7f90e5174700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e00b1830 0x7f90e016a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:16.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.458+0000 7f90e5174700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90e0156010 0x7f90e0156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.458+0000 7f90e5174700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90e01563e0 0x7f90e0152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.458+0000 7f90e5174700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e0157150 con 0x7f90e00b1830 2023-10-06T10:39:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.458+0000 7f90e5174700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e0157430 con 0x7f90e0156010 2023-10-06T10:39:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.458+0000 7f90e5174700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e0157710 con 0x7f90e01563e0 2023-10-06T10:39:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.459+0000 7f90dffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e00b1830 0x7f90e016a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.459+0000 7f90dffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e00b1830 0x7f90e016a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46084/0 (socket says 172.21.15.92:46084) 2023-10-06T10:39:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.459+0000 7f90dffff700 1 -- 172.21.15.92:0/929107717 learned_addr learned my addr 172.21.15.92:0/929107717 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.459+0000 7f90df7fe700 1 --2- 172.21.15.92:0/929107717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90e0156010 0x7f90e0156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.459+0000 7f90e4973700 1 --2- 172.21.15.92:0/929107717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90e01563e0 0x7f90e0152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.459+0000 7f90dffff700 1 -- 172.21.15.92:0/929107717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90e01563e0 msgr2=0x7f90e0152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.459+0000 7f90dffff700 1 --2- 172.21.15.92:0/929107717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90e01563e0 0x7f90e0152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.459+0000 7f90dffff700 1 -- 172.21.15.92:0/929107717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90e0156010 msgr2=0x7f90e0156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.459+0000 7f90dffff700 1 --2- 172.21.15.92:0/929107717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90e0156010 0x7f90e0156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.459+0000 7f90dffff700 1 -- 172.21.15.92:0/929107717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d8009a20 con 0x7f90e00b1830 2023-10-06T10:39:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.459+0000 7f90dffff700 1 --2- 172.21.15.92:0/929107717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e00b1830 0x7f90e016a310 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f90d000c790 tx=0x7f90d0007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.459+0000 7f90c7fff700 1 -- 172.21.15.92:0/929107717 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d0010510 con 0x7f90e00b1830 2023-10-06T10:39:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.459+0000 7f90c7fff700 1 -- 172.21.15.92:0/929107717 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d0010670 con 0x7f90e00b1830 2023-10-06T10:39:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.459+0000 7f90c7fff700 1 -- 172.21.15.92:0/929107717 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d0010a10 con 0x7f90e00b1830 2023-10-06T10:39:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.459+0000 7f90e5174700 1 -- 172.21.15.92:0/929107717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e01579f0 con 0x7f90e00b1830 2023-10-06T10:39:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.459+0000 7f90e5174700 1 -- 172.21.15.92:0/929107717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e016a850 con 0x7f90e00b1830 2023-10-06T10:39:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.460+0000 7f90e5174700 1 -- 172.21.15.92:0/929107717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e0157f10 con 0x7f90e00b1830 2023-10-06T10:39:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.461+0000 7f90c7fff700 1 -- 172.21.15.92:0/929107717 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f90d001b3b0 con 0x7f90e00b1830 2023-10-06T10:39:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.461+0000 7f90c7fff700 1 --2- 172.21.15.92:0/929107717 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f90cc066f30 0x7f90cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.461+0000 7f90c7fff700 1 -- 172.21.15.92:0/929107717 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f90d0084070 con 0x7f90e00b1830 2023-10-06T10:39:16.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.461+0000 7f90df7fe700 1 --2- 172.21.15.92:0/929107717 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f90cc066f30 0x7f90cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:16.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.461+0000 7f90df7fe700 1 --2- 172.21.15.92:0/929107717 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f90cc066f30 0x7f90cc0693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f90e01506e0 tx=0x7f90d4000bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:16.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.465+0000 7f90c7fff700 1 -- 172.21.15.92:0/929107717 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90e0157f10 con 0x7f90e00b1830 2023-10-06T10:39:16.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.652+0000 7f90e5174700 1 -- 172.21.15.92:0/929107717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90e0157f10 con 0x7f90e00b1830 2023-10-06T10:39:16.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.653+0000 7f90c7fff700 1 -- 172.21.15.92:0/929107717 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+17084 (secure 0 0 0) 0x7f90e0157f10 con 0x7f90e00b1830 2023-10-06T10:39:16.654 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:39:16.654 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":37,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:39:15.674694+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:39:13.780983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T10:39:14.644706+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:39:16.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.655+0000 7f90e5174700 1 -- 172.21.15.92:0/929107717 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f90cc066f30 msgr2=0x7f90cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:16.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.655+0000 7f90e5174700 1 --2- 172.21.15.92:0/929107717 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f90cc066f30 0x7f90cc0693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f90e01506e0 tx=0x7f90d4000bc0 comp rx=0 tx=0).stop 2023-10-06T10:39:16.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.655+0000 7f90e5174700 1 -- 172.21.15.92:0/929107717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e00b1830 msgr2=0x7f90e016a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:16.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.655+0000 7f90e5174700 1 --2- 172.21.15.92:0/929107717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e00b1830 0x7f90e016a310 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f90d000c790 tx=0x7f90d0007e10 comp rx=0 tx=0).stop 2023-10-06T10:39:16.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.655+0000 7f90e5174700 1 -- 172.21.15.92:0/929107717 shutdown_connections 2023-10-06T10:39:16.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.655+0000 7f90e5174700 1 --2- 172.21.15.92:0/929107717 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f90cc066f30 0x7f90cc0693e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:16.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.655+0000 7f90e5174700 1 --2- 172.21.15.92:0/929107717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90e01563e0 0x7f90e0152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:16.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.655+0000 7f90e5174700 1 --2- 172.21.15.92:0/929107717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e00b1830 0x7f90e016a310 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:16.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.655+0000 7f90e5174700 1 --2- 172.21.15.92:0/929107717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90e0156010 0x7f90e0156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:16.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.655+0000 7f90e5174700 1 -- 172.21.15.92:0/929107717 >> 172.21.15.92:0/929107717 conn(0x7f90e0069190 msgr2=0x7f90e006a450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:16.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.655+0000 7f90e5174700 1 -- 172.21.15.92:0/929107717 shutdown_connections 2023-10-06T10:39:16.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.655+0000 7f90e5174700 1 -- 172.21.15.92:0/929107717 wait complete. 2023-10-06T10:39:16.668 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:39:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.781+0000 7f729f13d700 1 Processor -- start 2023-10-06T10:39:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.781+0000 7f729f13d700 1 -- start start 2023-10-06T10:39:16.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f729f13d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f72980cb5f0 0x7f72980691a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:16.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f729f13d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72980696e0 0x7f7298159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:16.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f729f13d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7298068d10 con 0x7f729815a010 2023-10-06T10:39:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f729f13d700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7298069b30 con 0x7f72980cb5f0 2023-10-06T10:39:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f729f13d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7298154020 con 0x7f72980696e0 2023-10-06T10:39:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f729d6da700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f729815a010 0x7f72980ca670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60190/0 (socket says 172.21.15.92:60190) 2023-10-06T10:39:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f729d6da700 1 -- 172.21.15.92:0/1718501736 learned_addr learned my addr 172.21.15.92:0/1718501736 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f729ced9700 1 --2- 172.21.15.92:0/1718501736 >> v2:172.21.15.124:3300/0 conn(0x7f72980cb5f0 0x7f72980691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f72977fe700 1 -- 172.21.15.92:0/1718501736 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657993226 0 0) 0x7f7298068d10 con 0x7f729815a010 2023-10-06T10:39:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f7297fff700 1 --2- 172.21.15.92:0/1718501736 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72980696e0 0x7f7298159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f72977fe700 1 -- 172.21.15.92:0/1718501736 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7288003570 con 0x7f729815a010 2023-10-06T10:39:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f72977fe700 1 -- 172.21.15.92:0/1718501736 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3353039534 0 0) 0x7f7288003570 con 0x7f729815a010 2023-10-06T10:39:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f72977fe700 1 -- 172.21.15.92:0/1718501736 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72980696e0 msgr2=0x7f7298159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f72977fe700 1 --2- 172.21.15.92:0/1718501736 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72980696e0 0x7f7298159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f72977fe700 1 -- 172.21.15.92:0/1718501736 >> v2:172.21.15.124:3300/0 conn(0x7f72980cb5f0 msgr2=0x7f72980691a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f72977fe700 1 --2- 172.21.15.92:0/1718501736 >> v2:172.21.15.124:3300/0 conn(0x7f72980cb5f0 0x7f72980691a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f72977fe700 1 -- 172.21.15.92:0/1718501736 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729815a670 con 0x7f729815a010 2023-10-06T10:39:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f72977fe700 1 -- 172.21.15.92:0/1718501736 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f72900017f0 con 0x7f729815a010 2023-10-06T10:39:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f72977fe700 1 -- 172.21.15.92:0/1718501736 >> v1:172.21.15.92:6789/0 conn(0x7f729815a010 legacy=0x7f72980ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f72977fe700 1 --2- 172.21.15.92:0/1718501736 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72980696e0 0x7f7288003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f72977fe700 1 --2- 172.21.15.92:0/1718501736 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7288003fe0 0x7f7288004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f72977fe700 1 --2- 172.21.15.92:0/1718501736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7288004d50 0x7f72880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f72977fe700 1 -- 172.21.15.92:0/1718501736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7298069b30 con 0x7f7288004d50 2023-10-06T10:39:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f72977fe700 1 -- 172.21.15.92:0/1718501736 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7298154020 con 0x7f7288003fe0 2023-10-06T10:39:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f72977fe700 1 -- 172.21.15.92:0/1718501736 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728800ba90 con 0x7f72980696e0 2023-10-06T10:39:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.782+0000 7f729ced9700 1 --2- 172.21.15.92:0/1718501736 >> v2:172.21.15.124:3300/0 conn(0x7f72980cb5f0 0x7f72980691a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:39:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.783+0000 7f729d6da700 1 --2- 172.21.15.92:0/1718501736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7288004d50 0x7f72880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.783+0000 7f7297fff700 1 --2- 172.21.15.92:0/1718501736 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7288003fe0 0x7f7288004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.783+0000 7f729ced9700 1 --2- 172.21.15.92:0/1718501736 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72980696e0 0x7f7288003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.783+0000 7f729d6da700 1 -- 172.21.15.92:0/1718501736 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72980696e0 msgr2=0x7f7288003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.783+0000 7f729d6da700 1 --2- 172.21.15.92:0/1718501736 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72980696e0 0x7f7288003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.783+0000 7f729d6da700 1 -- 172.21.15.92:0/1718501736 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7288003fe0 msgr2=0x7f7288004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:16.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.783+0000 7f729d6da700 1 --2- 172.21.15.92:0/1718501736 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7288003fe0 0x7f7288004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:16.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.783+0000 7f729d6da700 1 -- 172.21.15.92:0/1718501736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7288003570 con 0x7f7288004d50 2023-10-06T10:39:16.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.783+0000 7f729d6da700 1 --2- 172.21.15.92:0/1718501736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7288004d50 0x7f72880051c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f729000a970 tx=0x7f7290002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76651623f38e9418 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:16.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.783+0000 7f72977fe700 1 -- 172.21.15.92:0/1718501736 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7290012440 con 0x7f7288004d50 2023-10-06T10:39:16.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.783+0000 7f72977fe700 1 -- 172.21.15.92:0/1718501736 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72900125c0 con 0x7f7288004d50 2023-10-06T10:39:16.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.783+0000 7f72977fe700 1 -- 172.21.15.92:0/1718501736 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7290012980 con 0x7f7288004d50 2023-10-06T10:39:16.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.784+0000 7f729f13d700 1 -- 172.21.15.92:0/1718501736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7288004d50 msgr2=0x7f72880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:16.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.784+0000 7f729f13d700 1 --2- 172.21.15.92:0/1718501736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7288004d50 0x7f72880051c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f729000a970 tx=0x7f7290002210 comp rx=0 tx=0).stop 2023-10-06T10:39:16.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.784+0000 7f729d6da700 1 -- 172.21.15.92:0/1718501736 reap_dead start 2023-10-06T10:39:16.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.784+0000 7f729f13d700 1 -- 172.21.15.92:0/1718501736 shutdown_connections 2023-10-06T10:39:16.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.784+0000 7f729f13d700 1 -- 172.21.15.92:0/1718501736 >> 172.21.15.92:0/1718501736 conn(0x7f72980707d0 msgr2=0x7f7298070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:16.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.784+0000 7f729f13d700 1 -- 172.21.15.92:0/1718501736 shutdown_connections 2023-10-06T10:39:16.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.784+0000 7f729f13d700 1 -- 172.21.15.92:0/1718501736 wait complete. 2023-10-06T10:39:16.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.785+0000 7f729f13d700 1 Processor -- start 2023-10-06T10:39:16.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.785+0000 7f729f13d700 1 -- start start 2023-10-06T10:39:16.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f729f13d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7298152ae0 0x7f7298155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:16.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f729f13d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7298152eb0 0x7f7298156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:16.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f729f13d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7298156e50 0x7f7298156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:16.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f729f13d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72980b3710 con 0x7f7298152ae0 2023-10-06T10:39:16.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f729f13d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72980b2b00 con 0x7f7298156e50 2023-10-06T10:39:16.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f729f13d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72980b2de0 con 0x7f7298152eb0 2023-10-06T10:39:16.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f729ced9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7298152ae0 0x7f7298155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:16.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f729ced9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7298152ae0 0x7f7298155cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46096/0 (socket says 172.21.15.92:46096) 2023-10-06T10:39:16.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f729ced9700 1 -- 172.21.15.92:0/105365382 learned_addr learned my addr 172.21.15.92:0/105365382 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:16.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f729d6da700 1 --2- 172.21.15.92:0/105365382 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7298156e50 0x7f7298156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:16.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f7297fff700 1 --2- 172.21.15.92:0/105365382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7298152eb0 0x7f7298156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:16.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f729ced9700 1 -- 172.21.15.92:0/105365382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7298152eb0 msgr2=0x7f7298156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:16.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f729ced9700 1 --2- 172.21.15.92:0/105365382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7298152eb0 0x7f7298156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:16.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f729ced9700 1 -- 172.21.15.92:0/105365382 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7298156e50 msgr2=0x7f7298156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:16.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f729ced9700 1 --2- 172.21.15.92:0/105365382 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7298156e50 0x7f7298156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:16.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f729ced9700 1 -- 172.21.15.92:0/105365382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72980b30c0 con 0x7f7298152ae0 2023-10-06T10:39:16.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f729d6da700 1 --2- 172.21.15.92:0/105365382 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7298156e50 0x7f7298156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:16.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f729ced9700 1 --2- 172.21.15.92:0/105365382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7298152ae0 0x7f7298155cf0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f728c011960 tx=0x7f728c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:16.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f7294ff9700 1 -- 172.21.15.92:0/105365382 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c011ea0 con 0x7f7298152ae0 2023-10-06T10:39:16.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f7294ff9700 1 -- 172.21.15.92:0/105365382 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728c00f970 con 0x7f7298152ae0 2023-10-06T10:39:16.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f729f13d700 1 -- 172.21.15.92:0/105365382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72980b33a0 con 0x7f7298152ae0 2023-10-06T10:39:16.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f7294ff9700 1 -- 172.21.15.92:0/105365382 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c00fcf0 con 0x7f7298152ae0 2023-10-06T10:39:16.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.786+0000 7f729f13d700 1 -- 172.21.15.92:0/105365382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7298154860 con 0x7f7298152ae0 2023-10-06T10:39:16.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.787+0000 7f729f13d700 1 -- 172.21.15.92:0/105365382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72980b3530 con 0x7f7298152ae0 2023-10-06T10:39:16.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.788+0000 7f7294ff9700 1 -- 172.21.15.92:0/105365382 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f728c015670 con 0x7f7298152ae0 2023-10-06T10:39:16.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.788+0000 7f7294ff9700 1 --2- 172.21.15.92:0/105365382 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f727c066f30 0x7f727c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:16.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.788+0000 7f7297fff700 1 --2- 172.21.15.92:0/105365382 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f727c066f30 0x7f727c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:16.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.788+0000 7f7294ff9700 1 -- 172.21.15.92:0/105365382 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f728c0851a0 con 0x7f7298152ae0 2023-10-06T10:39:16.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.789+0000 7f7297fff700 1 --2- 172.21.15.92:0/105365382 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f727c066f30 0x7f727c0693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7284006f00 tx=0x7f7284008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:16.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.792+0000 7f7294ff9700 1 -- 172.21.15.92:0/105365382 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f728c052220 con 0x7f7298152ae0 2023-10-06T10:39:16.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.976+0000 7f729f13d700 1 -- 172.21.15.92:0/105365382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7298064a90 con 0x7f7298152ae0 2023-10-06T10:39:16.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.977+0000 7f7294ff9700 1 -- 172.21.15.92:0/105365382 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+17084 (secure 0 0 0) 0x7f728c055840 con 0x7f7298152ae0 2023-10-06T10:39:16.978 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:39:16.978 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":37,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:39:15.674694+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:39:13.780983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T10:39:14.644706+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:39:16.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.979+0000 7f729f13d700 1 -- 172.21.15.92:0/105365382 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f727c066f30 msgr2=0x7f727c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:16.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.979+0000 7f729f13d700 1 --2- 172.21.15.92:0/105365382 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f727c066f30 0x7f727c0693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7284006f00 tx=0x7f7284008040 comp rx=0 tx=0).stop 2023-10-06T10:39:16.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.979+0000 7f729f13d700 1 -- 172.21.15.92:0/105365382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7298152ae0 msgr2=0x7f7298155cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:16.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.979+0000 7f729f13d700 1 --2- 172.21.15.92:0/105365382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7298152ae0 0x7f7298155cf0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f728c011960 tx=0x7f728c008040 comp rx=0 tx=0).stop 2023-10-06T10:39:16.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.979+0000 7f729f13d700 1 -- 172.21.15.92:0/105365382 shutdown_connections 2023-10-06T10:39:16.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.979+0000 7f729f13d700 1 --2- 172.21.15.92:0/105365382 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f727c066f30 0x7f727c0693e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:16.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.979+0000 7f729f13d700 1 --2- 172.21.15.92:0/105365382 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7298156e50 0x7f7298156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:16.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.979+0000 7f729f13d700 1 --2- 172.21.15.92:0/105365382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7298152ae0 0x7f7298155cf0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:16.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.979+0000 7f729f13d700 1 --2- 172.21.15.92:0/105365382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7298152eb0 0x7f7298156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:16.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.979+0000 7f729f13d700 1 -- 172.21.15.92:0/105365382 >> 172.21.15.92:0/105365382 conn(0x7f72980707d0 msgr2=0x7f72980cbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:16.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.979+0000 7f729f13d700 1 -- 172.21.15.92:0/105365382 shutdown_connections 2023-10-06T10:39:16.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:16.979+0000 7f729f13d700 1 -- 172.21.15.92:0/105365382 wait complete. 2023-10-06T10:39:16.992 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T10:39:17.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.107+0000 7f78a3453700 1 Processor -- start 2023-10-06T10:39:17.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.107+0000 7f78a3453700 1 -- start start 2023-10-06T10:39:17.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.108+0000 7f78a3453700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f789c1594d0 0x7f789c0cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:17.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.108+0000 7f78a3453700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f789c1598a0 0x7f789c0cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:17.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.108+0000 7f78a3453700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f789c0cf310 con 0x7f789c0ce7e0 2023-10-06T10:39:17.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.108+0000 7f78a3453700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f789c0cf5d0 con 0x7f789c1598a0 2023-10-06T10:39:17.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.108+0000 7f78a3453700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f789c0cf8b0 con 0x7f789c1594d0 2023-10-06T10:39:17.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.108+0000 7f78a09ee700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f789c1598a0 0x7f789c0cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:17.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.108+0000 7f78a09ee700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f789c1598a0 0x7f789c0cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46100/0 (socket says 172.21.15.92:46100) 2023-10-06T10:39:17.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.108+0000 7f78a09ee700 1 -- 172.21.15.92:0/978647574 learned_addr learned my addr 172.21.15.92:0/978647574 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:17.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.108+0000 7f78a11ef700 1 --2- 172.21.15.92:0/978647574 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f789c1594d0 0x7f789c0cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:17.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.108+0000 7f78a09ee700 1 -- 172.21.15.92:0/978647574 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f789c1594d0 msgr2=0x7f789c0cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:17.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.108+0000 7f78a09ee700 1 --2- 172.21.15.92:0/978647574 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f789c1594d0 0x7f789c0cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:17.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.109+0000 7f78a09ee700 1 -- 172.21.15.92:0/978647574 >> v1:172.21.15.124:6789/0 conn(0x7f789c0ce7e0 legacy=0x7f789c0cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:17.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.109+0000 7f78a09ee700 1 -- 172.21.15.92:0/978647574 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789c0cfb90 con 0x7f789c1598a0 2023-10-06T10:39:17.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.109+0000 7f78a09ee700 1 --2- 172.21.15.92:0/978647574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f789c1598a0 0x7f789c0cbaa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f788c0099b0 tx=0x7f788c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f49f4934390b34 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:17.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.109+0000 7f7893fff700 1 -- 172.21.15.92:0/978647574 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c00b860 con 0x7f789c1598a0 2023-10-06T10:39:17.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.109+0000 7f7893fff700 1 -- 172.21.15.92:0/978647574 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788c00b9c0 con 0x7f789c1598a0 2023-10-06T10:39:17.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.109+0000 7f7893fff700 1 -- 172.21.15.92:0/978647574 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c00bd60 con 0x7f789c1598a0 2023-10-06T10:39:17.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.109+0000 7f78a3453700 1 -- 172.21.15.92:0/978647574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f789c1598a0 msgr2=0x7f789c0cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:17.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.109+0000 7f78a3453700 1 --2- 172.21.15.92:0/978647574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f789c1598a0 0x7f789c0cbaa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f788c0099b0 tx=0x7f788c009250 comp rx=0 tx=0).stop 2023-10-06T10:39:17.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.109+0000 7f78a3453700 1 -- 172.21.15.92:0/978647574 shutdown_connections 2023-10-06T10:39:17.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.109+0000 7f78a3453700 1 --2- 172.21.15.92:0/978647574 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f789c1594d0 0x7f789c0cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:17.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.109+0000 7f78a3453700 1 --2- 172.21.15.92:0/978647574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f789c1598a0 0x7f789c0cbaa0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:17.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.109+0000 7f78a3453700 1 -- 172.21.15.92:0/978647574 >> 172.21.15.92:0/978647574 conn(0x7f789c089e50 msgr2=0x7f789c099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:17.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.109+0000 7f78a3453700 1 -- 172.21.15.92:0/978647574 shutdown_connections 2023-10-06T10:39:17.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.109+0000 7f78a3453700 1 -- 172.21.15.92:0/978647574 wait complete. 2023-10-06T10:39:17.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.110+0000 7f78a3453700 1 Processor -- start 2023-10-06T10:39:17.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.111+0000 7f78a3453700 1 -- start start 2023-10-06T10:39:17.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.111+0000 7f78a3453700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f789c0ce7e0 0x7f789c15c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:17.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.111+0000 7f78a3453700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f789c1594d0 0x7f789c146f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:17.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.111+0000 7f78a3453700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f789c1598a0 0x7f789c0b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:17.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.111+0000 7f78a3453700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789c158970 con 0x7f789c0ce7e0 2023-10-06T10:39:17.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.111+0000 7f78a3453700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789c0c8390 con 0x7f789c1594d0 2023-10-06T10:39:17.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.111+0000 7f78a3453700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f789c0c8670 con 0x7f789c1598a0 2023-10-06T10:39:17.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.111+0000 7f78a11ef700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f789c0ce7e0 0x7f789c15c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:17.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.112+0000 7f78a11ef700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f789c0ce7e0 0x7f789c15c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46108/0 (socket says 172.21.15.92:46108) 2023-10-06T10:39:17.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.112+0000 7f78a11ef700 1 -- 172.21.15.92:0/4169039122 learned_addr learned my addr 172.21.15.92:0/4169039122 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:17.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.112+0000 7f78a11ef700 1 -- 172.21.15.92:0/4169039122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f789c1598a0 msgr2=0x7f789c0b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:17.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.112+0000 7f78a11ef700 1 --2- 172.21.15.92:0/4169039122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f789c1598a0 0x7f789c0b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:17.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.112+0000 7f78a09ee700 1 --2- 172.21.15.92:0/4169039122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f789c1594d0 0x7f789c146f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:17.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.112+0000 7f78a11ef700 1 -- 172.21.15.92:0/4169039122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f789c1594d0 msgr2=0x7f789c146f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:17.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.112+0000 7f78a11ef700 1 --2- 172.21.15.92:0/4169039122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f789c1594d0 0x7f789c146f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:17.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.112+0000 7f78a11ef700 1 -- 172.21.15.92:0/4169039122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c0094f0 con 0x7f789c0ce7e0 2023-10-06T10:39:17.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.112+0000 7f78a11ef700 1 --2- 172.21.15.92:0/4169039122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f789c0ce7e0 0x7f789c15c480 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f789800d890 tx=0x7f7898008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:17.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.112+0000 7f78917fa700 1 -- 172.21.15.92:0/4169039122 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789800b600 con 0x7f789c0ce7e0 2023-10-06T10:39:17.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.112+0000 7f78917fa700 1 -- 172.21.15.92:0/4169039122 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789800b760 con 0x7f789c0ce7e0 2023-10-06T10:39:17.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.112+0000 7f78a3453700 1 -- 172.21.15.92:0/4169039122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789c0c8950 con 0x7f789c0ce7e0 2023-10-06T10:39:17.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.112+0000 7f78917fa700 1 -- 172.21.15.92:0/4169039122 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789800bae0 con 0x7f789c0ce7e0 2023-10-06T10:39:17.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.112+0000 7f78a3453700 1 -- 172.21.15.92:0/4169039122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f789c0c8f10 con 0x7f789c0ce7e0 2023-10-06T10:39:17.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.113+0000 7f78a3453700 1 -- 172.21.15.92:0/4169039122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f789c0c90a0 con 0x7f789c0ce7e0 2023-10-06T10:39:17.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.113+0000 7f78917fa700 1 -- 172.21.15.92:0/4169039122 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f789801e460 con 0x7f789c0ce7e0 2023-10-06T10:39:17.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.114+0000 7f78917fa700 1 --2- 172.21.15.92:0/4169039122 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7880066f30 0x7f78800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:17.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.114+0000 7f78917fa700 1 -- 172.21.15.92:0/4169039122 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7898084e80 con 0x7f789c0ce7e0 2023-10-06T10:39:17.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.114+0000 7f78a09ee700 1 --2- 172.21.15.92:0/4169039122 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7880066f30 0x7f78800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:17.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.114+0000 7f78a09ee700 1 --2- 172.21.15.92:0/4169039122 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7880066f30 0x7f78800693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f788c0099b0 tx=0x7f788c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:17.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.118+0000 7f78917fa700 1 -- 172.21.15.92:0/4169039122 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7898051e50 con 0x7f789c0ce7e0 2023-10-06T10:39:17.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:17.348+0000 7f78a3453700 1 -- 172.21.15.92:0/4169039122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f789c064ac0 con 0x7f789c0ce7e0 2023-10-06T10:39:18.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.324+0000 7f78917fa700 1 -- 172.21.15.92:0/4169039122 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7898055470 con 0x7f789c0ce7e0 2023-10-06T10:39:18.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.326+0000 7f78a3453700 1 -- 172.21.15.92:0/4169039122 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7880066f30 msgr2=0x7f78800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:18.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.326+0000 7f78a3453700 1 --2- 172.21.15.92:0/4169039122 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7880066f30 0x7f78800693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f788c0099b0 tx=0x7f788c017000 comp rx=0 tx=0).stop 2023-10-06T10:39:18.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.326+0000 7f78a3453700 1 -- 172.21.15.92:0/4169039122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f789c0ce7e0 msgr2=0x7f789c15c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:18.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.326+0000 7f78a3453700 1 --2- 172.21.15.92:0/4169039122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f789c0ce7e0 0x7f789c15c480 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f789800d890 tx=0x7f7898008040 comp rx=0 tx=0).stop 2023-10-06T10:39:18.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.326+0000 7f78a3453700 1 -- 172.21.15.92:0/4169039122 shutdown_connections 2023-10-06T10:39:18.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.326+0000 7f78a3453700 1 --2- 172.21.15.92:0/4169039122 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7880066f30 0x7f78800693e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:18.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.326+0000 7f78a3453700 1 --2- 172.21.15.92:0/4169039122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f789c1598a0 0x7f789c0b3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:18.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.326+0000 7f78a3453700 1 --2- 172.21.15.92:0/4169039122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f789c0ce7e0 0x7f789c15c480 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:18.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.326+0000 7f78a3453700 1 --2- 172.21.15.92:0/4169039122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f789c1594d0 0x7f789c146f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:18.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.326+0000 7f78a3453700 1 -- 172.21.15.92:0/4169039122 >> 172.21.15.92:0/4169039122 conn(0x7f789c089e50 msgr2=0x7f789c0d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:18.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.327+0000 7f78a3453700 1 -- 172.21.15.92:0/4169039122 shutdown_connections 2023-10-06T10:39:18.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.327+0000 7f78a3453700 1 -- 172.21.15.92:0/4169039122 wait complete. 2023-10-06T10:39:18.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T10:39:18.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.461+0000 7f714fc56700 1 Processor -- start 2023-10-06T10:39:18.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.461+0000 7f714fc56700 1 -- start start 2023-10-06T10:39:18.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.462+0000 7f714fc56700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7148159bd0 0x7f71480d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:18.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.462+0000 7f714fc56700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71480d0810 0x7f7148163200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:18.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.462+0000 7f714fc56700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71480b5ef0 con 0x7f71480cd940 2023-10-06T10:39:18.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.462+0000 7f714fc56700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71480b3600 con 0x7f7148159bd0 2023-10-06T10:39:18.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.462+0000 7f714fc56700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71480d0c20 con 0x7f71480d0810 2023-10-06T10:39:18.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.462+0000 7f714d9f2700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f71480cd940 0x7f71480b4a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60206/0 (socket says 172.21.15.92:60206) 2023-10-06T10:39:18.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.462+0000 7f714d9f2700 1 -- 172.21.15.92:0/2281333334 learned_addr learned my addr 172.21.15.92:0/2281333334 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:18.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714c9f0700 1 -- 172.21.15.92:0/2281333334 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902214343 0 0) 0x7f71480b5ef0 con 0x7f71480cd940 2023-10-06T10:39:18.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714c9f0700 1 -- 172.21.15.92:0/2281333334 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7134003570 con 0x7f71480cd940 2023-10-06T10:39:18.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714e1f3700 1 --2- 172.21.15.92:0/2281333334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71480d0810 0x7f7148163200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:18.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714d1f1700 1 --2- 172.21.15.92:0/2281333334 >> v2:172.21.15.124:3300/0 conn(0x7f7148159bd0 0x7f71480d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:18.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714c9f0700 1 -- 172.21.15.92:0/2281333334 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2046987860 0 0) 0x7f7134003570 con 0x7f71480cd940 2023-10-06T10:39:18.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714c9f0700 1 -- 172.21.15.92:0/2281333334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71480d0810 msgr2=0x7f7148163200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:18.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714c9f0700 1 --2- 172.21.15.92:0/2281333334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71480d0810 0x7f7148163200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:18.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714c9f0700 1 -- 172.21.15.92:0/2281333334 >> v2:172.21.15.124:3300/0 conn(0x7f7148159bd0 msgr2=0x7f71480d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:18.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714c9f0700 1 --2- 172.21.15.92:0/2281333334 >> v2:172.21.15.124:3300/0 conn(0x7f7148159bd0 0x7f71480d02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714c9f0700 1 -- 172.21.15.92:0/2281333334 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71480b6380 con 0x7f71480cd940 2023-10-06T10:39:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714c9f0700 1 -- 172.21.15.92:0/2281333334 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f7144001470 con 0x7f71480cd940 2023-10-06T10:39:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714e1f3700 1 --2- 172.21.15.92:0/2281333334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71480d0810 0x7f7148163200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714c9f0700 1 -- 172.21.15.92:0/2281333334 >> v1:172.21.15.92:6789/0 conn(0x7f71480cd940 legacy=0x7f71480b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714c9f0700 1 --2- 172.21.15.92:0/2281333334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71480d0810 0x7f7134003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714c9f0700 1 --2- 172.21.15.92:0/2281333334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7134004050 0x7f7134004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714c9f0700 1 --2- 172.21.15.92:0/2281333334 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7134004d50 0x7f71340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714c9f0700 1 -- 172.21.15.92:0/2281333334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71480b3600 con 0x7f7134004050 2023-10-06T10:39:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714c9f0700 1 -- 172.21.15.92:0/2281333334 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71480d0c20 con 0x7f7134004d50 2023-10-06T10:39:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714c9f0700 1 -- 172.21.15.92:0/2281333334 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713400bbc0 con 0x7f71480d0810 2023-10-06T10:39:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714d1f1700 1 --2- 172.21.15.92:0/2281333334 >> v2:172.21.15.124:3300/0 conn(0x7f7148159bd0 0x7f71480d02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714d9f2700 1 --2- 172.21.15.92:0/2281333334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71480d0810 0x7f7134003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714d1f1700 1 --2- 172.21.15.92:0/2281333334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7134004050 0x7f7134004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:18.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.463+0000 7f714e1f3700 1 --2- 172.21.15.92:0/2281333334 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7134004d50 0x7f71340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:18.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.464+0000 7f714d1f1700 1 -- 172.21.15.92:0/2281333334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71480d0810 msgr2=0x7f7134003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:18.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.464+0000 7f714d1f1700 1 --2- 172.21.15.92:0/2281333334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71480d0810 0x7f7134003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:18.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.464+0000 7f714d1f1700 1 -- 172.21.15.92:0/2281333334 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7134004d50 msgr2=0x7f71340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:18.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.464+0000 7f714d1f1700 1 --2- 172.21.15.92:0/2281333334 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7134004d50 0x7f71340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:18.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.464+0000 7f714d1f1700 1 -- 172.21.15.92:0/2281333334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7134003570 con 0x7f7134004050 2023-10-06T10:39:18.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.464+0000 7f714d1f1700 1 --2- 172.21.15.92:0/2281333334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7134004050 0x7f7134004810 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7138006fd0 tx=0x7f7138008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58286518276a239c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:18.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.464+0000 7f714c9f0700 1 -- 172.21.15.92:0/2281333334 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713800c070 con 0x7f7134004050 2023-10-06T10:39:18.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.464+0000 7f714c9f0700 1 -- 172.21.15.92:0/2281333334 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713800e5e0 con 0x7f7134004050 2023-10-06T10:39:18.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.464+0000 7f714c9f0700 1 -- 172.21.15.92:0/2281333334 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713800e790 con 0x7f7134004050 2023-10-06T10:39:18.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.464+0000 7f714fc56700 1 -- 172.21.15.92:0/2281333334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7134004050 msgr2=0x7f7134004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:18.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.464+0000 7f714fc56700 1 --2- 172.21.15.92:0/2281333334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7134004050 0x7f7134004810 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7138006fd0 tx=0x7f7138008040 comp rx=0 tx=0).stop 2023-10-06T10:39:18.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.464+0000 7f714e1f3700 1 -- 172.21.15.92:0/2281333334 reap_dead start 2023-10-06T10:39:18.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.464+0000 7f714fc56700 1 -- 172.21.15.92:0/2281333334 shutdown_connections 2023-10-06T10:39:18.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.464+0000 7f714fc56700 1 -- 172.21.15.92:0/2281333334 >> 172.21.15.92:0/2281333334 conn(0x7f714806ece0 msgr2=0x7f714806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:18.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.465+0000 7f714fc56700 1 -- 172.21.15.92:0/2281333334 shutdown_connections 2023-10-06T10:39:18.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.465+0000 7f714fc56700 1 -- 172.21.15.92:0/2281333334 wait complete. 2023-10-06T10:39:18.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.466+0000 7f714fc56700 1 Processor -- start 2023-10-06T10:39:18.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.466+0000 7f714fc56700 1 -- start start 2023-10-06T10:39:18.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.466+0000 7f714fc56700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7134004050 0x7f71480ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:18.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.466+0000 7f714fc56700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7148154ff0 0x7f7148150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.466+0000 7f714fc56700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71480b2660 0x7f714815afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.466+0000 7f714fc56700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7148155b00 con 0x7f7148154ff0 2023-10-06T10:39:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.466+0000 7f714fc56700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71480b1970 con 0x7f7134004050 2023-10-06T10:39:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.466+0000 7f714fc56700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7148155540 con 0x7f71480b2660 2023-10-06T10:39:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.466+0000 7f714d1f1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7148154ff0 0x7f7148150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.466+0000 7f714d1f1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7148154ff0 0x7f7148150540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46126/0 (socket says 172.21.15.92:46126) 2023-10-06T10:39:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.466+0000 7f714d1f1700 1 -- 172.21.15.92:0/3530036138 learned_addr learned my addr 172.21.15.92:0/3530036138 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.467+0000 7f714d9f2700 1 --2- 172.21.15.92:0/3530036138 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7134004050 0x7f71480ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.467+0000 7f714e1f3700 1 --2- 172.21.15.92:0/3530036138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71480b2660 0x7f714815afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.467+0000 7f714d1f1700 1 -- 172.21.15.92:0/3530036138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71480b2660 msgr2=0x7f714815afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.467+0000 7f714d1f1700 1 --2- 172.21.15.92:0/3530036138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71480b2660 0x7f714815afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.467+0000 7f714d1f1700 1 -- 172.21.15.92:0/3530036138 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7134004050 msgr2=0x7f71480ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.467+0000 7f714d1f1700 1 --2- 172.21.15.92:0/3530036138 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7134004050 0x7f71480ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.467+0000 7f714d1f1700 1 -- 172.21.15.92:0/3530036138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7148155820 con 0x7f7148154ff0 2023-10-06T10:39:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.467+0000 7f714d9f2700 1 --2- 172.21.15.92:0/3530036138 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7134004050 0x7f71480ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.467+0000 7f714d1f1700 1 --2- 172.21.15.92:0/3530036138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7148154ff0 0x7f7148150540 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f713400bb10 tx=0x7f713801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.467+0000 7f713dffb700 1 -- 172.21.15.92:0/3530036138 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138013c50 con 0x7f7148154ff0 2023-10-06T10:39:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.467+0000 7f713dffb700 1 -- 172.21.15.92:0/3530036138 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7138013db0 con 0x7f7148154ff0 2023-10-06T10:39:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.467+0000 7f713dffb700 1 -- 172.21.15.92:0/3530036138 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713801c690 con 0x7f7148154ff0 2023-10-06T10:39:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.467+0000 7f714fc56700 1 -- 172.21.15.92:0/3530036138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71480b1e60 con 0x7f7148154ff0 2023-10-06T10:39:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.467+0000 7f714fc56700 1 -- 172.21.15.92:0/3530036138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7148150a80 con 0x7f7148154ff0 2023-10-06T10:39:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.468+0000 7f714fc56700 1 -- 172.21.15.92:0/3530036138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7148155a60 con 0x7f7148154ff0 2023-10-06T10:39:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.469+0000 7f713dffb700 1 -- 172.21.15.92:0/3530036138 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f713801ca40 con 0x7f7148154ff0 2023-10-06T10:39:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.469+0000 7f713dffb700 1 --2- 172.21.15.92:0/3530036138 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f712c066f60 0x7f712c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:18.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.469+0000 7f714d9f2700 1 --2- 172.21.15.92:0/3530036138 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f712c066f60 0x7f712c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:18.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.469+0000 7f713dffb700 1 -- 172.21.15.92:0/3530036138 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7138085ea0 con 0x7f7148154ff0 2023-10-06T10:39:18.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.469+0000 7f714d9f2700 1 --2- 172.21.15.92:0/3530036138 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f712c066f60 0x7f712c069410 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7144001100 tx=0x7f7144009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:18.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.473+0000 7f713dffb700 1 -- 172.21.15.92:0/3530036138 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7138052e70 con 0x7f7148154ff0 2023-10-06T10:39:18.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:18.702+0000 7f714fc56700 1 -- 172.21.15.92:0/3530036138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7148064a90 con 0x7f7148154ff0 2023-10-06T10:39:18.805 INFO:tasks.ceph.mds.b.smithi124.stderr:2023-10-06T10:39:18.804+0000 7f2384ed0700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T10:39:19.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.325+0000 7f713dffb700 1 -- 172.21.15.92:0/3530036138 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7138056490 con 0x7f7148154ff0 2023-10-06T10:39:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.327+0000 7f714fc56700 1 -- 172.21.15.92:0/3530036138 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f712c066f60 msgr2=0x7f712c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.327+0000 7f714fc56700 1 --2- 172.21.15.92:0/3530036138 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f712c066f60 0x7f712c069410 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7144001100 tx=0x7f7144009040 comp rx=0 tx=0).stop 2023-10-06T10:39:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.327+0000 7f714fc56700 1 -- 172.21.15.92:0/3530036138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7148154ff0 msgr2=0x7f7148150540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.327+0000 7f714fc56700 1 --2- 172.21.15.92:0/3530036138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7148154ff0 0x7f7148150540 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f713400bb10 tx=0x7f713801a000 comp rx=0 tx=0).stop 2023-10-06T10:39:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.327+0000 7f714fc56700 1 -- 172.21.15.92:0/3530036138 shutdown_connections 2023-10-06T10:39:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.327+0000 7f714fc56700 1 --2- 172.21.15.92:0/3530036138 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f712c066f60 0x7f712c069410 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.327+0000 7f714fc56700 1 --2- 172.21.15.92:0/3530036138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71480b2660 0x7f714815afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.327+0000 7f714fc56700 1 --2- 172.21.15.92:0/3530036138 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7134004050 0x7f71480ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.327+0000 7f714fc56700 1 --2- 172.21.15.92:0/3530036138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7148154ff0 0x7f7148150540 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.327+0000 7f714fc56700 1 -- 172.21.15.92:0/3530036138 >> 172.21.15.92:0/3530036138 conn(0x7f714806ece0 msgr2=0x7f714815c220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.327+0000 7f714fc56700 1 -- 172.21.15.92:0/3530036138 shutdown_connections 2023-10-06T10:39:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.327+0000 7f714fc56700 1 -- 172.21.15.92:0/3530036138 wait complete. 2023-10-06T10:39:19.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:39:19.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.456+0000 7f93fffff700 1 Processor -- start 2023-10-06T10:39:19.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.456+0000 7f93fffff700 1 -- start start 2023-10-06T10:39:19.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.457+0000 7f93fffff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9400159440 0x7f94000b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:19.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.457+0000 7f93fffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f94000b2100 0x7f94000b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:19.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.457+0000 7f93fffff700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94000cb7a0 con 0x7f94000b35d0 2023-10-06T10:39:19.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.457+0000 7f93fffff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94000b0860 con 0x7f94000b2100 2023-10-06T10:39:19.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.457+0000 7f93fffff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94000cb150 con 0x7f9400159440 2023-10-06T10:39:19.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.457+0000 7f93ff7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f94000b2100 0x7f94000b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:19.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.457+0000 7f93ff7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f94000b2100 0x7f94000b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46134/0 (socket says 172.21.15.92:46134) 2023-10-06T10:39:19.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.457+0000 7f93ff7fe700 1 -- 172.21.15.92:0/4269877754 learned_addr learned my addr 172.21.15.92:0/4269877754 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:19.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.457+0000 7f93feffd700 1 --2- 172.21.15.92:0/4269877754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9400159440 0x7f94000b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:19.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.458+0000 7f93ff7fe700 1 -- 172.21.15.92:0/4269877754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9400159440 msgr2=0x7f94000b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:19.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.458+0000 7f93ff7fe700 1 --2- 172.21.15.92:0/4269877754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9400159440 0x7f94000b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:19.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.458+0000 7f93ff7fe700 1 -- 172.21.15.92:0/4269877754 >> v1:172.21.15.124:6789/0 conn(0x7f94000b35d0 legacy=0x7f94000cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:19.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.458+0000 7f93ff7fe700 1 -- 172.21.15.92:0/4269877754 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94000cb430 con 0x7f94000b2100 2023-10-06T10:39:19.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.458+0000 7f93ff7fe700 1 --2- 172.21.15.92:0/4269877754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94000b2100 0x7f94000b2580 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009f20 tx=0x7f93f8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8148f891364c5d12 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:19.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.458+0000 7f93fe7fc700 1 -- 172.21.15.92:0/4269877754 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f800e070 con 0x7f94000b2100 2023-10-06T10:39:19.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.458+0000 7f93fe7fc700 1 -- 172.21.15.92:0/4269877754 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f800b9d0 con 0x7f94000b2100 2023-10-06T10:39:19.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.458+0000 7f93fe7fc700 1 -- 172.21.15.92:0/4269877754 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f80092e0 con 0x7f94000b2100 2023-10-06T10:39:19.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.458+0000 7f93fffff700 1 -- 172.21.15.92:0/4269877754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94000b2100 msgr2=0x7f94000b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:19.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.458+0000 7f93fffff700 1 --2- 172.21.15.92:0/4269877754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94000b2100 0x7f94000b2580 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009f20 tx=0x7f93f8009840 comp rx=0 tx=0).stop 2023-10-06T10:39:19.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.458+0000 7f93fffff700 1 -- 172.21.15.92:0/4269877754 shutdown_connections 2023-10-06T10:39:19.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.458+0000 7f93fffff700 1 --2- 172.21.15.92:0/4269877754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94000b2100 0x7f94000b2580 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:19.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.458+0000 7f93fffff700 1 --2- 172.21.15.92:0/4269877754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9400159440 0x7f94000b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:19.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.458+0000 7f93fffff700 1 -- 172.21.15.92:0/4269877754 >> 172.21.15.92:0/4269877754 conn(0x7f940006ece0 msgr2=0x7f940006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:19.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.458+0000 7f93fffff700 1 -- 172.21.15.92:0/4269877754 shutdown_connections 2023-10-06T10:39:19.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.459+0000 7f93fffff700 1 -- 172.21.15.92:0/4269877754 wait complete. 2023-10-06T10:39:19.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.460+0000 7f93fffff700 1 Processor -- start 2023-10-06T10:39:19.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.460+0000 7f93fffff700 1 -- start start 2023-10-06T10:39:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.460+0000 7f93fffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94000b2100 0x7f940016bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.460+0000 7f93fffff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94000b35d0 0x7f94000cf780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.460+0000 7f93fffff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9400159440 0x7f94000c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.460+0000 7f93fffff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9400153b90 con 0x7f94000b2100 2023-10-06T10:39:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.460+0000 7f93fffff700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94000d0220 con 0x7f94000b35d0 2023-10-06T10:39:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.460+0000 7f93fffff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94000d0500 con 0x7f9400159440 2023-10-06T10:39:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.461+0000 7f93feffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94000b2100 0x7f940016bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.461+0000 7f93feffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94000b2100 0x7f940016bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46140/0 (socket says 172.21.15.92:46140) 2023-10-06T10:39:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.461+0000 7f93feffd700 1 -- 172.21.15.92:0/3721409380 learned_addr learned my addr 172.21.15.92:0/3721409380 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.461+0000 7f93ff7fe700 1 --2- 172.21.15.92:0/3721409380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9400159440 0x7f94000c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.461+0000 7f93f67fc700 1 --2- 172.21.15.92:0/3721409380 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94000b35d0 0x7f94000cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.461+0000 7f93feffd700 1 -- 172.21.15.92:0/3721409380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9400159440 msgr2=0x7f94000c72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.461+0000 7f93feffd700 1 --2- 172.21.15.92:0/3721409380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9400159440 0x7f94000c72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.461+0000 7f93feffd700 1 -- 172.21.15.92:0/3721409380 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94000b35d0 msgr2=0x7f94000cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.461+0000 7f93feffd700 1 --2- 172.21.15.92:0/3721409380 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94000b35d0 0x7f94000cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.461+0000 7f93feffd700 1 -- 172.21.15.92:0/3721409380 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f8009ae0 con 0x7f94000b2100 2023-10-06T10:39:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.461+0000 7f93f67fc700 1 --2- 172.21.15.92:0/3721409380 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94000b35d0 0x7f94000cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.461+0000 7f93ff7fe700 1 --2- 172.21.15.92:0/3721409380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9400159440 0x7f94000c72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.461+0000 7f93feffd700 1 --2- 172.21.15.92:0/3721409380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94000b2100 0x7f940016bee0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f93f000e930 tx=0x7f93f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.461+0000 7f93f77fe700 1 -- 172.21.15.92:0/3721409380 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f000eed0 con 0x7f94000b2100 2023-10-06T10:39:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.461+0000 7f93f77fe700 1 -- 172.21.15.92:0/3721409380 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f000c850 con 0x7f94000b2100 2023-10-06T10:39:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.461+0000 7f93f77fe700 1 -- 172.21.15.92:0/3721409380 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f000cbf0 con 0x7f94000b2100 2023-10-06T10:39:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.461+0000 7f93fffff700 1 -- 172.21.15.92:0/3721409380 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94000c44f0 con 0x7f94000b2100 2023-10-06T10:39:19.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.461+0000 7f93fffff700 1 -- 172.21.15.92:0/3721409380 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9400153f20 con 0x7f94000b2100 2023-10-06T10:39:19.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.462+0000 7f93fffff700 1 -- 172.21.15.92:0/3721409380 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94000c4b40 con 0x7f94000b2100 2023-10-06T10:39:19.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.462+0000 7f93f77fe700 1 -- 172.21.15.92:0/3721409380 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f93f0013470 con 0x7f94000b2100 2023-10-06T10:39:19.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.463+0000 7f93f77fe700 1 --2- 172.21.15.92:0/3721409380 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f93e0066f30 0x7f93e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:19.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.463+0000 7f93f77fe700 1 -- 172.21.15.92:0/3721409380 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f93f0084a00 con 0x7f94000b2100 2023-10-06T10:39:19.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.463+0000 7f93f67fc700 1 --2- 172.21.15.92:0/3721409380 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f93e0066f30 0x7f93e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:19.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.463+0000 7f93f67fc700 1 --2- 172.21.15.92:0/3721409380 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f93e0066f30 0x7f93e00693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f93e8007fd0 tx=0x7f93e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:19.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.467+0000 7f93f77fe700 1 -- 172.21.15.92:0/3721409380 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93f00519d0 con 0x7f94000b2100 2023-10-06T10:39:19.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.694+0000 7f93fffff700 1 -- 172.21.15.92:0/3721409380 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94000c4350 con 0x7f94000b2100 2023-10-06T10:39:19.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.695+0000 7f93f77fe700 1 -- 172.21.15.92:0/3721409380 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f94000c4350 con 0x7f94000b2100 2023-10-06T10:39:19.696 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:39:19.696 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2525196781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2525196781},{"type":"v1","addr":"172.21.15.124:6835","nonce":2525196781}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5118,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2409440810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2409440810},{"type":"v1","addr":"172.21.15.92:6836","nonce":2409440810}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5121,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/4010629505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4010629505},{"type":"v1","addr":"172.21.15.92:6837","nonce":4010629505}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:39:15.675013+0000","modified":"2023-10-06T10:39:19.322877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5133":{"gid":5133,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.124:6834/484869337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":484869337},{"type":"v1","addr":"172.21.15.124:6834","nonce":484869337}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T10:39:19.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.697+0000 7f93fffff700 1 -- 172.21.15.92:0/3721409380 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f93e0066f30 msgr2=0x7f93e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:19.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.697+0000 7f93fffff700 1 --2- 172.21.15.92:0/3721409380 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f93e0066f30 0x7f93e00693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f93e8007fd0 tx=0x7f93e8009040 comp rx=0 tx=0).stop 2023-10-06T10:39:19.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.697+0000 7f93fffff700 1 -- 172.21.15.92:0/3721409380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94000b2100 msgr2=0x7f940016bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:19.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.697+0000 7f93fffff700 1 --2- 172.21.15.92:0/3721409380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94000b2100 0x7f940016bee0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f93f000e930 tx=0x7f93f00091a0 comp rx=0 tx=0).stop 2023-10-06T10:39:19.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.697+0000 7f93fffff700 1 -- 172.21.15.92:0/3721409380 shutdown_connections 2023-10-06T10:39:19.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.697+0000 7f93fffff700 1 --2- 172.21.15.92:0/3721409380 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f93e0066f30 0x7f93e00693e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:19.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.697+0000 7f93fffff700 1 --2- 172.21.15.92:0/3721409380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9400159440 0x7f94000c72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:19.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.697+0000 7f93fffff700 1 --2- 172.21.15.92:0/3721409380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94000b2100 0x7f940016bee0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:19.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.697+0000 7f93fffff700 1 --2- 172.21.15.92:0/3721409380 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94000b35d0 0x7f94000cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:19.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.697+0000 7f93fffff700 1 -- 172.21.15.92:0/3721409380 >> 172.21.15.92:0/3721409380 conn(0x7f940006ece0 msgr2=0x7f9400156cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:19.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.697+0000 7f93fffff700 1 -- 172.21.15.92:0/3721409380 shutdown_connections 2023-10-06T10:39:19.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.697+0000 7f93fffff700 1 -- 172.21.15.92:0/3721409380 wait complete. 2023-10-06T10:39:19.699 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 16 2023-10-06T10:39:19.710 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:39:19.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.823+0000 7ff7cf0bd700 1 Processor -- start 2023-10-06T10:39:19.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.823+0000 7ff7cf0bd700 1 -- start start 2023-10-06T10:39:19.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.824+0000 7ff7cf0bd700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff7c815cab0 0x7ff7c806f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:19.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.824+0000 7ff7cf0bd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7c80caa90 0x7ff7c80cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:19.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.824+0000 7ff7cf0bd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff7c80675b0 0x7ff7c80679d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:19.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.824+0000 7ff7cf0bd700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c806ecd0 con 0x7ff7c80675b0 2023-10-06T10:39:19.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.824+0000 7ff7cf0bd700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c8067f10 con 0x7ff7c815cab0 2023-10-06T10:39:19.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.824+0000 7ff7cf0bd700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c80cb870 con 0x7ff7c80caa90 2023-10-06T10:39:19.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.824+0000 7ff7cd65a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff7c80675b0 0x7ff7c80679d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:19.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.824+0000 7ff7cd65a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff7c80675b0 0x7ff7c80679d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46154/0 (socket says 172.21.15.92:46154) 2023-10-06T10:39:19.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.824+0000 7ff7cd65a700 1 -- 172.21.15.92:0/3600588498 learned_addr learned my addr 172.21.15.92:0/3600588498 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:19.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.825+0000 7ff7c7fff700 1 --2- 172.21.15.92:0/3600588498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7c80caa90 0x7ff7c80cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:19.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.825+0000 7ff7cce59700 1 --2- 172.21.15.92:0/3600588498 >> v2:172.21.15.124:3300/0 conn(0x7ff7c815cab0 0x7ff7c806f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:19.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.825+0000 7ff7cd65a700 1 -- 172.21.15.92:0/3600588498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7c80caa90 msgr2=0x7ff7c80cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:19.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.825+0000 7ff7cd65a700 1 --2- 172.21.15.92:0/3600588498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7c80caa90 0x7ff7c80cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:19.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.825+0000 7ff7cd65a700 1 -- 172.21.15.92:0/3600588498 >> v2:172.21.15.124:3300/0 conn(0x7ff7c815cab0 msgr2=0x7ff7c806f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:19.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.825+0000 7ff7cd65a700 1 --2- 172.21.15.92:0/3600588498 >> v2:172.21.15.124:3300/0 conn(0x7ff7c815cab0 0x7ff7c806f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:19.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.825+0000 7ff7cd65a700 1 -- 172.21.15.92:0/3600588498 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c80cbb50 con 0x7ff7c80675b0 2023-10-06T10:39:19.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.825+0000 7ff7cd65a700 1 --2- 172.21.15.92:0/3600588498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c80675b0 0x7ff7c80679d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff7c806e480 tx=0x7ff7c0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2596cd8908c52e75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:19.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.825+0000 7ff7c77fe700 1 -- 172.21.15.92:0/3600588498 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c000e070 con 0x7ff7c80675b0 2023-10-06T10:39:19.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.825+0000 7ff7c77fe700 1 -- 172.21.15.92:0/3600588498 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c000b970 con 0x7ff7c80675b0 2023-10-06T10:39:19.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.825+0000 7ff7c77fe700 1 -- 172.21.15.92:0/3600588498 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c00092e0 con 0x7ff7c80675b0 2023-10-06T10:39:19.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.826+0000 7ff7cf0bd700 1 -- 172.21.15.92:0/3600588498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c80675b0 msgr2=0x7ff7c80679d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:19.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.826+0000 7ff7cf0bd700 1 --2- 172.21.15.92:0/3600588498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c80675b0 0x7ff7c80679d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff7c806e480 tx=0x7ff7c0009840 comp rx=0 tx=0).stop 2023-10-06T10:39:19.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.826+0000 7ff7cf0bd700 1 -- 172.21.15.92:0/3600588498 shutdown_connections 2023-10-06T10:39:19.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.826+0000 7ff7cf0bd700 1 --2- 172.21.15.92:0/3600588498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c80675b0 0x7ff7c80679d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:19.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.826+0000 7ff7cf0bd700 1 --2- 172.21.15.92:0/3600588498 >> v2:172.21.15.124:3300/0 conn(0x7ff7c815cab0 0x7ff7c806f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:19.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.826+0000 7ff7cf0bd700 1 --2- 172.21.15.92:0/3600588498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7c80caa90 0x7ff7c80cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:19.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.826+0000 7ff7cf0bd700 1 -- 172.21.15.92:0/3600588498 >> 172.21.15.92:0/3600588498 conn(0x7ff7c80cdeb0 msgr2=0x7ff7c80c9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:19.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.826+0000 7ff7cf0bd700 1 -- 172.21.15.92:0/3600588498 shutdown_connections 2023-10-06T10:39:19.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.826+0000 7ff7cf0bd700 1 -- 172.21.15.92:0/3600588498 wait complete. 2023-10-06T10:39:19.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.827+0000 7ff7cf0bd700 1 Processor -- start 2023-10-06T10:39:19.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.827+0000 7ff7cf0bd700 1 -- start start 2023-10-06T10:39:19.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.828+0000 7ff7cf0bd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7c80675b0 0x7ff7c80acf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:19.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.828+0000 7ff7cf0bd700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7c80caa90 0x7ff7c80af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:19.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.828+0000 7ff7cf0bd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c815cab0 0x7ff7c80ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:19.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.828+0000 7ff7cf0bd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c80aa910 con 0x7ff7c815cab0 2023-10-06T10:39:19.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.828+0000 7ff7cf0bd700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c80a9fe0 con 0x7ff7c80caa90 2023-10-06T10:39:19.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.828+0000 7ff7cf0bd700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c80aa2c0 con 0x7ff7c80675b0 2023-10-06T10:39:19.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.828+0000 7ff7cd65a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c815cab0 0x7ff7c80ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:19.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.828+0000 7ff7cd65a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c815cab0 0x7ff7c80ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46160/0 (socket says 172.21.15.92:46160) 2023-10-06T10:39:19.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.828+0000 7ff7cd65a700 1 -- 172.21.15.92:0/2219071307 learned_addr learned my addr 172.21.15.92:0/2219071307 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:19.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.828+0000 7ff7cce59700 1 --2- 172.21.15.92:0/2219071307 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7c80675b0 0x7ff7c80acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:19.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.828+0000 7ff7c7fff700 1 --2- 172.21.15.92:0/2219071307 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7c80caa90 0x7ff7c80af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:19.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.828+0000 7ff7cd65a700 1 -- 172.21.15.92:0/2219071307 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7c80675b0 msgr2=0x7ff7c80acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:19.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.828+0000 7ff7cd65a700 1 --2- 172.21.15.92:0/2219071307 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7c80675b0 0x7ff7c80acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:19.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.828+0000 7ff7cd65a700 1 -- 172.21.15.92:0/2219071307 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7c80caa90 msgr2=0x7ff7c80af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:19.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.828+0000 7ff7cd65a700 1 --2- 172.21.15.92:0/2219071307 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7c80caa90 0x7ff7c80af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:19.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.828+0000 7ff7cd65a700 1 -- 172.21.15.92:0/2219071307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c0009ae0 con 0x7ff7c815cab0 2023-10-06T10:39:19.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.828+0000 7ff7cd65a700 1 --2- 172.21.15.92:0/2219071307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c815cab0 0x7ff7c80ad600 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000b610 tx=0x7ff7c0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:19.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.828+0000 7ff7c4ff9700 1 -- 172.21.15.92:0/2219071307 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c00092e0 con 0x7ff7c815cab0 2023-10-06T10:39:19.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.828+0000 7ff7c4ff9700 1 -- 172.21.15.92:0/2219071307 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c0007b00 con 0x7ff7c815cab0 2023-10-06T10:39:19.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.828+0000 7ff7cf0bd700 1 -- 172.21.15.92:0/2219071307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c80aa5a0 con 0x7ff7c815cab0 2023-10-06T10:39:19.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.828+0000 7ff7c4ff9700 1 -- 172.21.15.92:0/2219071307 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c001d420 con 0x7ff7c815cab0 2023-10-06T10:39:19.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.828+0000 7ff7cf0bd700 1 -- 172.21.15.92:0/2219071307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c8155fb0 con 0x7ff7c815cab0 2023-10-06T10:39:19.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.829+0000 7ff7cf0bd700 1 -- 172.21.15.92:0/2219071307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c80aaac0 con 0x7ff7c815cab0 2023-10-06T10:39:19.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.830+0000 7ff7c4ff9700 1 -- 172.21.15.92:0/2219071307 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff7c0003050 con 0x7ff7c815cab0 2023-10-06T10:39:19.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.830+0000 7ff7c4ff9700 1 --2- 172.21.15.92:0/2219071307 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff7a8066f30 0x7ff7a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:19.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.830+0000 7ff7c4ff9700 1 -- 172.21.15.92:0/2219071307 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff7c00862f0 con 0x7ff7c815cab0 2023-10-06T10:39:19.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.830+0000 7ff7cce59700 1 --2- 172.21.15.92:0/2219071307 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff7a8066f30 0x7ff7a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:19.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.831+0000 7ff7cce59700 1 --2- 172.21.15.92:0/2219071307 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff7a8066f30 0x7ff7a80693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009d10 tx=0x7ff7b80091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:19.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:19.834+0000 7ff7c4ff9700 1 -- 172.21.15.92:0/2219071307 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7c80aaac0 con 0x7ff7c815cab0 2023-10-06T10:39:20.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.019+0000 7ff7cf0bd700 1 -- 172.21.15.92:0/2219071307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7c8064a90 con 0x7ff7c815cab0 2023-10-06T10:39:20.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.020+0000 7ff7c4ff9700 1 -- 172.21.15.92:0/2219071307 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+17071 (secure 0 0 0) 0x7ff7c00532c0 con 0x7ff7c815cab0 2023-10-06T10:39:20.021 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:39:20.021 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":38,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:39:17.309624+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:39:13.780983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T10:39:14.644706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:39:20.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.022+0000 7ff7cf0bd700 1 -- 172.21.15.92:0/2219071307 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff7a8066f30 msgr2=0x7ff7a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:20.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.022+0000 7ff7cf0bd700 1 --2- 172.21.15.92:0/2219071307 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff7a8066f30 0x7ff7a80693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009d10 tx=0x7ff7b80091a0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.022+0000 7ff7cf0bd700 1 -- 172.21.15.92:0/2219071307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c815cab0 msgr2=0x7ff7c80ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:20.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.022+0000 7ff7cf0bd700 1 --2- 172.21.15.92:0/2219071307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c815cab0 0x7ff7c80ad600 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000b610 tx=0x7ff7c0018000 comp rx=0 tx=0).stop 2023-10-06T10:39:20.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.022+0000 7ff7cf0bd700 1 -- 172.21.15.92:0/2219071307 shutdown_connections 2023-10-06T10:39:20.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.022+0000 7ff7cf0bd700 1 --2- 172.21.15.92:0/2219071307 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff7a8066f30 0x7ff7a80693e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.022+0000 7ff7cf0bd700 1 --2- 172.21.15.92:0/2219071307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c815cab0 0x7ff7c80ad600 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.022+0000 7ff7cf0bd700 1 --2- 172.21.15.92:0/2219071307 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7c80675b0 0x7ff7c80acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.022+0000 7ff7cf0bd700 1 --2- 172.21.15.92:0/2219071307 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7c80caa90 0x7ff7c80af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.022+0000 7ff7cf0bd700 1 -- 172.21.15.92:0/2219071307 >> 172.21.15.92:0/2219071307 conn(0x7ff7c80cdeb0 msgr2=0x7ff7c80c2180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:20.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.022+0000 7ff7cf0bd700 1 -- 172.21.15.92:0/2219071307 shutdown_connections 2023-10-06T10:39:20.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.022+0000 7ff7cf0bd700 1 -- 172.21.15.92:0/2219071307 wait complete. 2023-10-06T10:39:20.035 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T10:39:20.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:39:20.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.149+0000 7ff66d95d700 1 Processor -- start 2023-10-06T10:39:20.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.150+0000 7ff66d95d700 1 -- start start 2023-10-06T10:39:20.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.150+0000 7ff66d95d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff668157460 0x7ff66806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:20.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.150+0000 7ff66d95d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff6680cdb00 0x7ff6680675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:20.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.150+0000 7ff66d95d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6680aec90 con 0x7ff668067af0 2023-10-06T10:39:20.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.150+0000 7ff66d95d700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6680c4030 con 0x7ff6680cdb00 2023-10-06T10:39:20.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.150+0000 7ff66d95d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66806f3b0 con 0x7ff668157460 2023-10-06T10:39:20.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.150+0000 7ff6677fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff668067af0 0x7ff66806a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60222/0 (socket says 172.21.15.92:60222) 2023-10-06T10:39:20.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.151+0000 7ff6677fe700 1 -- 172.21.15.92:0/1173201763 learned_addr learned my addr 172.21.15.92:0/1173201763 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:20.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.151+0000 7ff666ffd700 1 --2- 172.21.15.92:0/1173201763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff668157460 0x7ff66806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:20.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.151+0000 7ff65e7fc700 1 --2- 172.21.15.92:0/1173201763 >> v2:172.21.15.124:3300/0 conn(0x7ff6680cdb00 0x7ff6680675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:20.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.151+0000 7ff6667fc700 1 -- 172.21.15.92:0/1173201763 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2203376958 0 0) 0x7ff6680aec90 con 0x7ff668067af0 2023-10-06T10:39:20.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.151+0000 7ff6667fc700 1 -- 172.21.15.92:0/1173201763 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff654003570 con 0x7ff668067af0 2023-10-06T10:39:20.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.151+0000 7ff65e7fc700 1 -- 172.21.15.92:0/1173201763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff668157460 msgr2=0x7ff66806ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:20.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.151+0000 7ff65e7fc700 1 --2- 172.21.15.92:0/1173201763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff668157460 0x7ff66806ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.151+0000 7ff65e7fc700 1 -- 172.21.15.92:0/1173201763 >> v1:172.21.15.92:6789/0 conn(0x7ff668067af0 legacy=0x7ff66806a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:20.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.151+0000 7ff65e7fc700 1 -- 172.21.15.92:0/1173201763 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6680aef70 con 0x7ff6680cdb00 2023-10-06T10:39:20.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.151+0000 7ff666ffd700 1 --2- 172.21.15.92:0/1173201763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff668157460 0x7ff66806ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:39:20.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.151+0000 7ff65e7fc700 1 --2- 172.21.15.92:0/1173201763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6680cdb00 0x7ff6680675b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff6500098e0 tx=0x7ff6500091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f965c5452b6b273 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:20.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.151+0000 7ff6667fc700 1 -- 172.21.15.92:0/1173201763 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff650011630 con 0x7ff6680cdb00 2023-10-06T10:39:20.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.151+0000 7ff6667fc700 1 -- 172.21.15.92:0/1173201763 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff650011790 con 0x7ff6680cdb00 2023-10-06T10:39:20.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.151+0000 7ff6667fc700 1 -- 172.21.15.92:0/1173201763 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff650011b10 con 0x7ff6680cdb00 2023-10-06T10:39:20.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.152+0000 7ff66d95d700 1 -- 172.21.15.92:0/1173201763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6680cdb00 msgr2=0x7ff6680675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:20.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.152+0000 7ff66d95d700 1 --2- 172.21.15.92:0/1173201763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6680cdb00 0x7ff6680675b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff6500098e0 tx=0x7ff6500091a0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.152+0000 7ff66d95d700 1 -- 172.21.15.92:0/1173201763 shutdown_connections 2023-10-06T10:39:20.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.152+0000 7ff66d95d700 1 --2- 172.21.15.92:0/1173201763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff668157460 0x7ff66806ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.152+0000 7ff66d95d700 1 --2- 172.21.15.92:0/1173201763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6680cdb00 0x7ff6680675b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.152+0000 7ff66d95d700 1 -- 172.21.15.92:0/1173201763 >> 172.21.15.92:0/1173201763 conn(0x7ff668069140 msgr2=0x7ff6680cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:20.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.152+0000 7ff66d95d700 1 -- 172.21.15.92:0/1173201763 shutdown_connections 2023-10-06T10:39:20.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.152+0000 7ff66d95d700 1 -- 172.21.15.92:0/1173201763 wait complete. 2023-10-06T10:39:20.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.153+0000 7ff66d95d700 1 Processor -- start 2023-10-06T10:39:20.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.153+0000 7ff66d95d700 1 -- start start 2023-10-06T10:39:20.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.154+0000 7ff66d95d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff668067af0 0x7ff6680aa750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:20.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.154+0000 7ff66d95d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6680cdb00 0x7ff6680b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:20.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.154+0000 7ff66d95d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff668157460 0x7ff6680b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:20.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.154+0000 7ff66d95d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6680acfc0 con 0x7ff668157460 2023-10-06T10:39:20.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.154+0000 7ff66d95d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6680aff90 con 0x7ff6680cdb00 2023-10-06T10:39:20.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.154+0000 7ff66d95d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6680b0b10 con 0x7ff668067af0 2023-10-06T10:39:20.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.154+0000 7ff6677fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff668157460 0x7ff6680b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:20.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.154+0000 7ff6677fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff668157460 0x7ff6680b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46162/0 (socket says 172.21.15.92:46162) 2023-10-06T10:39:20.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.154+0000 7ff6677fe700 1 -- 172.21.15.92:0/1187389884 learned_addr learned my addr 172.21.15.92:0/1187389884 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:20.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.154+0000 7ff65e7fc700 1 --2- 172.21.15.92:0/1187389884 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6680cdb00 0x7ff6680b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:20.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.154+0000 7ff666ffd700 1 --2- 172.21.15.92:0/1187389884 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff668067af0 0x7ff6680aa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:20.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.154+0000 7ff6677fe700 1 -- 172.21.15.92:0/1187389884 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff668067af0 msgr2=0x7ff6680aa750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:20.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.154+0000 7ff6677fe700 1 --2- 172.21.15.92:0/1187389884 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff668067af0 0x7ff6680aa750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.154+0000 7ff6677fe700 1 -- 172.21.15.92:0/1187389884 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6680cdb00 msgr2=0x7ff6680b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:20.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.154+0000 7ff6677fe700 1 --2- 172.21.15.92:0/1187389884 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6680cdb00 0x7ff6680b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.154+0000 7ff6677fe700 1 -- 172.21.15.92:0/1187389884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff658009420 con 0x7ff668157460 2023-10-06T10:39:20.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.155+0000 7ff6677fe700 1 --2- 172.21.15.92:0/1187389884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff668157460 0x7ff6680b14e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff66000cbc0 tx=0x7ff660001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:20.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.155+0000 7ff65f7fe700 1 -- 172.21.15.92:0/1187389884 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66000a7e0 con 0x7ff668157460 2023-10-06T10:39:20.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.155+0000 7ff65f7fe700 1 -- 172.21.15.92:0/1187389884 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66000a940 con 0x7ff668157460 2023-10-06T10:39:20.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.155+0000 7ff66d95d700 1 -- 172.21.15.92:0/1187389884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff650009420 con 0x7ff668157460 2023-10-06T10:39:20.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.155+0000 7ff65f7fe700 1 -- 172.21.15.92:0/1187389884 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66000acc0 con 0x7ff668157460 2023-10-06T10:39:20.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.155+0000 7ff66d95d700 1 -- 172.21.15.92:0/1187389884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6680b04c0 con 0x7ff668157460 2023-10-06T10:39:20.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.156+0000 7ff66d95d700 1 -- 172.21.15.92:0/1187389884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6680b0c80 con 0x7ff668157460 2023-10-06T10:39:20.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.156+0000 7ff65f7fe700 1 -- 172.21.15.92:0/1187389884 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff660012610 con 0x7ff668157460 2023-10-06T10:39:20.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.156+0000 7ff65f7fe700 1 --2- 172.21.15.92:0/1187389884 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff648066f30 0x7ff6480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:20.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.156+0000 7ff65f7fe700 1 -- 172.21.15.92:0/1187389884 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff660084a40 con 0x7ff668157460 2023-10-06T10:39:20.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.156+0000 7ff666ffd700 1 --2- 172.21.15.92:0/1187389884 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff648066f30 0x7ff6480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:20.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.157+0000 7ff666ffd700 1 --2- 172.21.15.92:0/1187389884 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff648066f30 0x7ff6480693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff658009ea0 tx=0x7ff6580091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:20.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.160+0000 7ff65f7fe700 1 -- 172.21.15.92:0/1187389884 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff660086e60 con 0x7ff668157460 2023-10-06T10:39:20.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.344+0000 7ff66d95d700 1 -- 172.21.15.92:0/1187389884 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff668064a90 con 0x7ff648066f30 2023-10-06T10:39:20.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.353+0000 7ff65f7fe700 1 -- 172.21.15.92:0/1187389884 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311807 (secure 0 0 0) 0x7ff668064a90 con 0x7ff648066f30 2023-10-06T10:39:20.357 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:39:20.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.356+0000 7ff66d95d700 1 -- 172.21.15.92:0/1187389884 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff648066f30 msgr2=0x7ff6480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:20.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.356+0000 7ff66d95d700 1 --2- 172.21.15.92:0/1187389884 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff648066f30 0x7ff6480693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff658009ea0 tx=0x7ff6580091a0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.356+0000 7ff66d95d700 1 -- 172.21.15.92:0/1187389884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff668157460 msgr2=0x7ff6680b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:20.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.356+0000 7ff66d95d700 1 --2- 172.21.15.92:0/1187389884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff668157460 0x7ff6680b14e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff66000cbc0 tx=0x7ff660001a70 comp rx=0 tx=0).stop 2023-10-06T10:39:20.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.356+0000 7ff66d95d700 1 -- 172.21.15.92:0/1187389884 shutdown_connections 2023-10-06T10:39:20.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.356+0000 7ff66d95d700 1 --2- 172.21.15.92:0/1187389884 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff648066f30 0x7ff6480693e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.356+0000 7ff66d95d700 1 --2- 172.21.15.92:0/1187389884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff668157460 0x7ff6680b14e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.356+0000 7ff66d95d700 1 --2- 172.21.15.92:0/1187389884 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff668067af0 0x7ff6680aa750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.356+0000 7ff66d95d700 1 --2- 172.21.15.92:0/1187389884 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6680cdb00 0x7ff6680b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.356+0000 7ff66d95d700 1 -- 172.21.15.92:0/1187389884 >> 172.21.15.92:0/1187389884 conn(0x7ff668069140 msgr2=0x7ff6681564c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:20.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.356+0000 7ff66d95d700 1 -- 172.21.15.92:0/1187389884 shutdown_connections 2023-10-06T10:39:20.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.356+0000 7ff66d95d700 1 -- 172.21.15.92:0/1187389884 wait complete. 2023-10-06T10:39:20.362 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:39:20.369 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2023-10-06T10:39:20.094690+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382948,"kb_used_data":4644,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752591772,"statfs":{"total":773094113280,"available":770653974528,"internally_reserved":0,"allocated":4755456,"data_stored":2751419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.805649"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.324564+0000","last_change":"2023-10-06T10:39:16.322743+0000","last_active":"2023-10-06T10:39:16.324564+0000","last_peered":"2023-10-06T10:39:16.324564+0000","last_clean":"2023-10-06T10:39:16.324564+0000","last_became_active":"2023-10-06T10:39:16.322467+0000","last_became_peered":"2023-10-06T10:39:16.322467+0000","last_unstale":"2023-10-06T10:39:16.324564+0000","last_undegraded":"2023-10-06T10:39:16.324564+0000","last_fullsized":"2023-10-06T10:39:16.324564+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:16:40.720156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.323776+0000","last_change":"2023-10-06T10:39:16.315729+0000","last_active":"2023-10-06T10:39:16.323776+0000","last_peered":"2023-10-06T10:39:16.323776+0000","last_clean":"2023-10-06T10:39:16.323776+0000","last_became_active":"2023-10-06T10:39:16.315440+0000","last_became_peered":"2023-10-06T10:39:16.315440+0000","last_unstale":"2023-10-06T10:39:16.323776+0000","last_undegraded":"2023-10-06T10:39:16.323776+0000","last_fullsized":"2023-10-06T10:39:16.323776+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:27:28.759803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.321105+0000","last_change":"2023-10-06T10:39:16.306759+0000","last_active":"2023-10-06T10:39:16.321105+0000","last_peered":"2023-10-06T10:39:16.321105+0000","last_clean":"2023-10-06T10:39:16.321105+0000","last_became_active":"2023-10-06T10:39:16.306466+0000","last_became_peered":"2023-10-06T10:39:16.306466+0000","last_unstale":"2023-10-06T10:39:16.321105+0000","last_undegraded":"2023-10-06T10:39:16.321105+0000","last_fullsized":"2023-10-06T10:39:16.321105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:35:29.567370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.321649+0000","last_change":"2023-10-06T10:39:16.317613+0000","last_active":"2023-10-06T10:39:16.321649+0000","last_peered":"2023-10-06T10:39:16.321649+0000","last_clean":"2023-10-06T10:39:16.321649+0000","last_became_active":"2023-10-06T10:39:16.317197+0000","last_became_peered":"2023-10-06T10:39:16.317197+0000","last_unstale":"2023-10-06T10:39:16.321649+0000","last_undegraded":"2023-10-06T10:39:16.321649+0000","last_fullsized":"2023-10-06T10:39:16.321649+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:46:44.354855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.323813+0000","last_change":"2023-10-06T10:39:16.315614+0000","last_active":"2023-10-06T10:39:16.323813+0000","last_peered":"2023-10-06T10:39:16.323813+0000","last_clean":"2023-10-06T10:39:16.323813+0000","last_became_active":"2023-10-06T10:39:16.315349+0000","last_became_peered":"2023-10-06T10:39:16.315349+0000","last_unstale":"2023-10-06T10:39:16.323813+0000","last_undegraded":"2023-10-06T10:39:16.323813+0000","last_fullsized":"2023-10-06T10:39:16.323813+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:37:00.510491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.318187+0000","last_change":"2023-10-06T10:39:16.316533+0000","last_active":"2023-10-06T10:39:16.318187+0000","last_peered":"2023-10-06T10:39:16.318187+0000","last_clean":"2023-10-06T10:39:16.318187+0000","last_became_active":"2023-10-06T10:39:16.316145+0000","last_became_peered":"2023-10-06T10:39:16.316145+0000","last_unstale":"2023-10-06T10:39:16.318187+0000","last_undegraded":"2023-10-06T10:39:16.318187+0000","last_fullsized":"2023-10-06T10:39:16.318187+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:14:44.803599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.322763+0000","last_change":"2023-10-06T10:39:16.314876+0000","last_active":"2023-10-06T10:39:16.322763+0000","last_peered":"2023-10-06T10:39:16.322763+0000","last_clean":"2023-10-06T10:39:16.322763+0000","last_became_active":"2023-10-06T10:39:16.314617+0000","last_became_peered":"2023-10-06T10:39:16.314617+0000","last_unstale":"2023-10-06T10:39:16.322763+0000","last_undegraded":"2023-10-06T10:39:16.322763+0000","last_fullsized":"2023-10-06T10:39:16.322763+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:35:13.124890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.329662+0000","last_change":"2023-10-06T10:39:16.323233+0000","last_active":"2023-10-06T10:39:16.329662+0000","last_peered":"2023-10-06T10:39:16.329662+0000","last_clean":"2023-10-06T10:39:16.329662+0000","last_became_active":"2023-10-06T10:39:16.322975+0000","last_became_peered":"2023-10-06T10:39:16.322975+0000","last_unstale":"2023-10-06T10:39:16.329662+0000","last_undegraded":"2023-10-06T10:39:16.329662+0000","last_fullsized":"2023-10-06T10:39:16.329662+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:37.593925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.319634+0000","last_change":"2023-10-06T10:39:16.312877+0000","last_active":"2023-10-06T10:39:16.319634+0000","last_peered":"2023-10-06T10:39:16.319634+0000","last_clean":"2023-10-06T10:39:16.319634+0000","last_became_active":"2023-10-06T10:39:16.312625+0000","last_became_peered":"2023-10-06T10:39:16.312625+0000","last_unstale":"2023-10-06T10:39:16.319634+0000","last_undegraded":"2023-10-06T10:39:16.319634+0000","last_fullsized":"2023-10-06T10:39:16.319634+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:05.869985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.329702+0000","last_change":"2023-10-06T10:39:16.323513+0000","last_active":"2023-10-06T10:39:16.329702+0000","last_peered":"2023-10-06T10:39:16.329702+0000","last_clean":"2023-10-06T10:39:16.329702+0000","last_became_active":"2023-10-06T10:39:16.322879+0000","last_became_peered":"2023-10-06T10:39:16.322879+0000","last_unstale":"2023-10-06T10:39:16.329702+0000","last_undegraded":"2023-10-06T10:39:16.329702+0000","last_fullsized":"2023-10-06T10:39:16.329702+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:36:44.908262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.317494+0000","last_change":"2023-10-06T10:39:16.312470+0000","last_active":"2023-10-06T10:39:16.317494+0000","last_peered":"2023-10-06T10:39:16.317494+0000","last_clean":"2023-10-06T10:39:16.317494+0000","last_became_active":"2023-10-06T10:39:16.312215+0000","last_became_peered":"2023-10-06T10:39:16.312215+0000","last_unstale":"2023-10-06T10:39:16.317494+0000","last_undegraded":"2023-10-06T10:39:16.317494+0000","last_fullsized":"2023-10-06T10:39:16.317494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:27:28.759803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.317493+0000","last_change":"2023-10-06T10:39:16.310935+0000","last_active":"2023-10-06T10:39:16.317493+0000","last_peered":"2023-10-06T10:39:16.317493+0000","last_clean":"2023-10-06T10:39:16.317493+0000","last_became_active":"2023-10-06T10:39:16.310671+0000","last_became_peered":"2023-10-06T10:39:16.310671+0000","last_unstale":"2023-10-06T10:39:16.317493+0000","last_undegraded":"2023-10-06T10:39:16.317493+0000","last_fullsized":"2023-10-06T10:39:16.317493+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:16:40.720156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.323817+0000","last_change":"2023-10-06T10:39:16.321749+0000","last_active":"2023-10-06T10:39:16.323817+0000","last_peered":"2023-10-06T10:39:16.323817+0000","last_clean":"2023-10-06T10:39:16.323817+0000","last_became_active":"2023-10-06T10:39:16.321421+0000","last_became_peered":"2023-10-06T10:39:16.321421+0000","last_unstale":"2023-10-06T10:39:16.323817+0000","last_undegraded":"2023-10-06T10:39:16.323817+0000","last_fullsized":"2023-10-06T10:39:16.323817+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:11:40.614228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.319527+0000","last_change":"2023-10-06T10:39:16.315307+0000","last_active":"2023-10-06T10:39:16.319527+0000","last_peered":"2023-10-06T10:39:16.319527+0000","last_clean":"2023-10-06T10:39:16.319527+0000","last_became_active":"2023-10-06T10:39:16.315075+0000","last_became_peered":"2023-10-06T10:39:16.315075+0000","last_unstale":"2023-10-06T10:39:16.319527+0000","last_undegraded":"2023-10-06T10:39:16.319527+0000","last_fullsized":"2023-10-06T10:39:16.319527+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:18:17.504520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.323277+0000","last_change":"2023-10-06T10:39:16.321160+0000","last_active":"2023-10-06T10:39:16.323277+0000","last_peered":"2023-10-06T10:39:16.323277+0000","last_clean":"2023-10-06T10:39:16.323277+0000","last_became_active":"2023-10-06T10:39:16.320904+0000","last_became_peered":"2023-10-06T10:39:16.320904+0000","last_unstale":"2023-10-06T10:39:16.323277+0000","last_undegraded":"2023-10-06T10:39:16.323277+0000","last_fullsized":"2023-10-06T10:39:16.323277+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:18.818780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.324013+0000","last_change":"2023-10-06T10:39:16.316387+0000","last_active":"2023-10-06T10:39:16.324013+0000","last_peered":"2023-10-06T10:39:16.324013+0000","last_clean":"2023-10-06T10:39:16.324013+0000","last_became_active":"2023-10-06T10:39:16.316103+0000","last_became_peered":"2023-10-06T10:39:16.316103+0000","last_unstale":"2023-10-06T10:39:16.324013+0000","last_undegraded":"2023-10-06T10:39:16.324013+0000","last_fullsized":"2023-10-06T10:39:16.324013+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:04.259427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.321282+0000","last_change":"2023-10-06T10:39:16.312435+0000","last_active":"2023-10-06T10:39:16.321282+0000","last_peered":"2023-10-06T10:39:16.321282+0000","last_clean":"2023-10-06T10:39:16.321282+0000","last_became_active":"2023-10-06T10:39:16.311654+0000","last_became_peered":"2023-10-06T10:39:16.311654+0000","last_unstale":"2023-10-06T10:39:16.321282+0000","last_undegraded":"2023-10-06T10:39:16.321282+0000","last_fullsized":"2023-10-06T10:39:16.321282+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:09.527903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.321283+0000","last_change":"2023-10-06T10:39:16.315547+0000","last_active":"2023-10-06T10:39:16.321283+0000","last_peered":"2023-10-06T10:39:16.321283+0000","last_clean":"2023-10-06T10:39:16.321283+0000","last_became_active":"2023-10-06T10:39:16.315293+0000","last_became_peered":"2023-10-06T10:39:16.315293+0000","last_unstale":"2023-10-06T10:39:16.321283+0000","last_undegraded":"2023-10-06T10:39:16.321283+0000","last_fullsized":"2023-10-06T10:39:16.321283+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:14:44.803599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.325843+0000","last_change":"2023-10-06T10:39:16.320957+0000","last_active":"2023-10-06T10:39:16.325843+0000","last_peered":"2023-10-06T10:39:16.325843+0000","last_clean":"2023-10-06T10:39:16.325843+0000","last_became_active":"2023-10-06T10:39:16.320691+0000","last_became_peered":"2023-10-06T10:39:16.320691+0000","last_unstale":"2023-10-06T10:39:16.325843+0000","last_undegraded":"2023-10-06T10:39:16.325843+0000","last_fullsized":"2023-10-06T10:39:16.325843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:47.503939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.320814+0000","last_change":"2023-10-06T10:39:16.314690+0000","last_active":"2023-10-06T10:39:16.320814+0000","last_peered":"2023-10-06T10:39:16.320814+0000","last_clean":"2023-10-06T10:39:16.320814+0000","last_became_active":"2023-10-06T10:39:16.314453+0000","last_became_peered":"2023-10-06T10:39:16.314453+0000","last_unstale":"2023-10-06T10:39:16.320814+0000","last_undegraded":"2023-10-06T10:39:16.320814+0000","last_fullsized":"2023-10-06T10:39:16.320814+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:55.246713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.318143+0000","last_change":"2023-10-06T10:39:16.316143+0000","last_active":"2023-10-06T10:39:16.318143+0000","last_peered":"2023-10-06T10:39:16.318143+0000","last_clean":"2023-10-06T10:39:16.318143+0000","last_became_active":"2023-10-06T10:39:16.315879+0000","last_became_peered":"2023-10-06T10:39:16.315879+0000","last_unstale":"2023-10-06T10:39:16.318143+0000","last_undegraded":"2023-10-06T10:39:16.318143+0000","last_fullsized":"2023-10-06T10:39:16.318143+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:10.131667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.327978+0000","last_change":"2023-10-06T10:39:16.322605+0000","last_active":"2023-10-06T10:39:16.327978+0000","last_peered":"2023-10-06T10:39:16.327978+0000","last_clean":"2023-10-06T10:39:16.327978+0000","last_became_active":"2023-10-06T10:39:16.322107+0000","last_became_peered":"2023-10-06T10:39:16.322107+0000","last_unstale":"2023-10-06T10:39:16.327978+0000","last_undegraded":"2023-10-06T10:39:16.327978+0000","last_fullsized":"2023-10-06T10:39:16.327978+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:35:29.567370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.325551+0000","last_change":"2023-10-06T10:39:16.320784+0000","last_active":"2023-10-06T10:39:16.325551+0000","last_peered":"2023-10-06T10:39:16.325551+0000","last_clean":"2023-10-06T10:39:16.325551+0000","last_became_active":"2023-10-06T10:39:16.320514+0000","last_became_peered":"2023-10-06T10:39:16.320514+0000","last_unstale":"2023-10-06T10:39:16.325551+0000","last_undegraded":"2023-10-06T10:39:16.325551+0000","last_fullsized":"2023-10-06T10:39:16.325551+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:55.246713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.325569+0000","last_change":"2023-10-06T10:39:16.320837+0000","last_active":"2023-10-06T10:39:16.325569+0000","last_peered":"2023-10-06T10:39:16.325569+0000","last_clean":"2023-10-06T10:39:16.325569+0000","last_became_active":"2023-10-06T10:39:16.320581+0000","last_became_peered":"2023-10-06T10:39:16.320581+0000","last_unstale":"2023-10-06T10:39:16.325569+0000","last_undegraded":"2023-10-06T10:39:16.325569+0000","last_fullsized":"2023-10-06T10:39:16.325569+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:37:59.945308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.325588+0000","last_change":"2023-10-06T10:39:16.320249+0000","last_active":"2023-10-06T10:39:16.325588+0000","last_peered":"2023-10-06T10:39:16.325588+0000","last_clean":"2023-10-06T10:39:16.325588+0000","last_became_active":"2023-10-06T10:39:16.320049+0000","last_became_peered":"2023-10-06T10:39:16.320049+0000","last_unstale":"2023-10-06T10:39:16.325588+0000","last_undegraded":"2023-10-06T10:39:16.325588+0000","last_fullsized":"2023-10-06T10:39:16.325588+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:14:53.280400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.323149+0000","last_change":"2023-10-06T10:39:16.319646+0000","last_active":"2023-10-06T10:39:16.323149+0000","last_peered":"2023-10-06T10:39:16.323149+0000","last_clean":"2023-10-06T10:39:16.323149+0000","last_became_active":"2023-10-06T10:39:16.319358+0000","last_became_peered":"2023-10-06T10:39:16.319358+0000","last_unstale":"2023-10-06T10:39:16.323149+0000","last_undegraded":"2023-10-06T10:39:16.323149+0000","last_fullsized":"2023-10-06T10:39:16.323149+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:47.503939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.321124+0000","last_change":"2023-10-06T10:39:16.309618+0000","last_active":"2023-10-06T10:39:16.321124+0000","last_peered":"2023-10-06T10:39:16.321124+0000","last_clean":"2023-10-06T10:39:16.321124+0000","last_became_active":"2023-10-06T10:39:16.309376+0000","last_became_peered":"2023-10-06T10:39:16.309376+0000","last_unstale":"2023-10-06T10:39:16.321124+0000","last_undegraded":"2023-10-06T10:39:16.321124+0000","last_fullsized":"2023-10-06T10:39:16.321124+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:09.420170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.322329+0000","last_change":"2023-10-06T10:39:16.314282+0000","last_active":"2023-10-06T10:39:16.322329+0000","last_peered":"2023-10-06T10:39:16.322329+0000","last_clean":"2023-10-06T10:39:16.322329+0000","last_became_active":"2023-10-06T10:39:16.314001+0000","last_became_peered":"2023-10-06T10:39:16.314001+0000","last_unstale":"2023-10-06T10:39:16.322329+0000","last_undegraded":"2023-10-06T10:39:16.322329+0000","last_fullsized":"2023-10-06T10:39:16.322329+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:27.557844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.325472+0000","last_change":"2023-10-06T10:39:16.320179+0000","last_active":"2023-10-06T10:39:16.325472+0000","last_peered":"2023-10-06T10:39:16.325472+0000","last_clean":"2023-10-06T10:39:16.325472+0000","last_became_active":"2023-10-06T10:39:16.319907+0000","last_became_peered":"2023-10-06T10:39:16.319907+0000","last_unstale":"2023-10-06T10:39:16.325472+0000","last_undegraded":"2023-10-06T10:39:16.325472+0000","last_fullsized":"2023-10-06T10:39:16.325472+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:15.030656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.321317+0000","last_change":"2023-10-06T10:39:16.312104+0000","last_active":"2023-10-06T10:39:16.321317+0000","last_peered":"2023-10-06T10:39:16.321317+0000","last_clean":"2023-10-06T10:39:16.321317+0000","last_became_active":"2023-10-06T10:39:16.311823+0000","last_became_peered":"2023-10-06T10:39:16.311823+0000","last_unstale":"2023-10-06T10:39:16.321317+0000","last_undegraded":"2023-10-06T10:39:16.321317+0000","last_fullsized":"2023-10-06T10:39:16.321317+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:36:44.908262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.318076+0000","last_change":"2023-10-06T10:39:16.313390+0000","last_active":"2023-10-06T10:39:16.318076+0000","last_peered":"2023-10-06T10:39:16.318076+0000","last_clean":"2023-10-06T10:39:16.318076+0000","last_became_active":"2023-10-06T10:39:16.313129+0000","last_became_peered":"2023-10-06T10:39:16.313129+0000","last_unstale":"2023-10-06T10:39:16.318076+0000","last_undegraded":"2023-10-06T10:39:16.318076+0000","last_fullsized":"2023-10-06T10:39:16.318076+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:46:44.354855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.318942+0000","last_change":"2023-10-06T10:39:16.313745+0000","last_active":"2023-10-06T10:39:16.318942+0000","last_peered":"2023-10-06T10:39:16.318942+0000","last_clean":"2023-10-06T10:39:16.318942+0000","last_became_active":"2023-10-06T10:39:16.313461+0000","last_became_peered":"2023-10-06T10:39:16.313461+0000","last_unstale":"2023-10-06T10:39:16.318942+0000","last_undegraded":"2023-10-06T10:39:16.318942+0000","last_fullsized":"2023-10-06T10:39:16.318942+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:13.217482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.318388+0000","last_change":"2023-10-06T10:39:16.313811+0000","last_active":"2023-10-06T10:39:16.318388+0000","last_peered":"2023-10-06T10:39:16.318388+0000","last_clean":"2023-10-06T10:39:16.318388+0000","last_became_active":"2023-10-06T10:39:16.313573+0000","last_became_peered":"2023-10-06T10:39:16.313573+0000","last_unstale":"2023-10-06T10:39:16.318388+0000","last_undegraded":"2023-10-06T10:39:16.318388+0000","last_fullsized":"2023-10-06T10:39:16.318388+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:24.577057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"37'4","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.331021+0000","last_change":"2023-10-06T10:39:15.303875+0000","last_active":"2023-10-06T10:39:16.331021+0000","last_peered":"2023-10-06T10:39:16.331021+0000","last_clean":"2023-10-06T10:39:16.331021+0000","last_became_active":"2023-10-06T10:39:15.303626+0000","last_became_peered":"2023-10-06T10:39:15.303626+0000","last_unstale":"2023-10-06T10:39:16.331021+0000","last_undegraded":"2023-10-06T10:39:16.331021+0000","last_fullsized":"2023-10-06T10:39:16.331021+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:48.263358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.325782+0000","last_change":"2023-10-06T10:39:16.323413+0000","last_active":"2023-10-06T10:39:16.325782+0000","last_peered":"2023-10-06T10:39:16.325782+0000","last_clean":"2023-10-06T10:39:16.325782+0000","last_became_active":"2023-10-06T10:39:16.323094+0000","last_became_peered":"2023-10-06T10:39:16.323094+0000","last_unstale":"2023-10-06T10:39:16.325782+0000","last_undegraded":"2023-10-06T10:39:16.325782+0000","last_fullsized":"2023-10-06T10:39:16.325782+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:24.577057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.332974+0000","last_change":"2023-10-06T10:39:15.310615+0000","last_active":"2023-10-06T10:39:16.332974+0000","last_peered":"2023-10-06T10:39:16.332974+0000","last_clean":"2023-10-06T10:39:16.332974+0000","last_became_active":"2023-10-06T10:39:15.310305+0000","last_became_peered":"2023-10-06T10:39:15.310305+0000","last_unstale":"2023-10-06T10:39:16.332974+0000","last_undegraded":"2023-10-06T10:39:16.332974+0000","last_fullsized":"2023-10-06T10:39:16.332974+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:16:37.935892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.323043+0000","last_change":"2023-10-06T10:39:16.316459+0000","last_active":"2023-10-06T10:39:16.323043+0000","last_peered":"2023-10-06T10:39:16.323043+0000","last_clean":"2023-10-06T10:39:16.323043+0000","last_became_active":"2023-10-06T10:39:16.316232+0000","last_became_peered":"2023-10-06T10:39:16.316232+0000","last_unstale":"2023-10-06T10:39:16.323043+0000","last_undegraded":"2023-10-06T10:39:16.323043+0000","last_fullsized":"2023-10-06T10:39:16.323043+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:05.869985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.330501+0000","last_change":"2023-10-06T10:39:15.299806+0000","last_active":"2023-10-06T10:39:16.330501+0000","last_peered":"2023-10-06T10:39:16.330501+0000","last_clean":"2023-10-06T10:39:16.330501+0000","last_became_active":"2023-10-06T10:39:15.299579+0000","last_became_peered":"2023-10-06T10:39:15.299579+0000","last_unstale":"2023-10-06T10:39:16.330501+0000","last_undegraded":"2023-10-06T10:39:16.330501+0000","last_fullsized":"2023-10-06T10:39:16.330501+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:14.029154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.319861+0000","last_change":"2023-10-06T10:39:16.318730+0000","last_active":"2023-10-06T10:39:16.319861+0000","last_peered":"2023-10-06T10:39:16.319861+0000","last_clean":"2023-10-06T10:39:16.319861+0000","last_became_active":"2023-10-06T10:39:16.318513+0000","last_became_peered":"2023-10-06T10:39:16.318513+0000","last_unstale":"2023-10-06T10:39:16.319861+0000","last_undegraded":"2023-10-06T10:39:16.319861+0000","last_fullsized":"2023-10-06T10:39:16.319861+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:04.259427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.330978+0000","last_change":"2023-10-06T10:39:15.307706+0000","last_active":"2023-10-06T10:39:16.330978+0000","last_peered":"2023-10-06T10:39:16.330978+0000","last_clean":"2023-10-06T10:39:16.330978+0000","last_became_active":"2023-10-06T10:39:15.307425+0000","last_became_peered":"2023-10-06T10:39:15.307425+0000","last_unstale":"2023-10-06T10:39:16.330978+0000","last_undegraded":"2023-10-06T10:39:16.330978+0000","last_fullsized":"2023-10-06T10:39:16.330978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:46.502437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.321328+0000","last_change":"2023-10-06T10:39:16.317415+0000","last_active":"2023-10-06T10:39:16.321328+0000","last_peered":"2023-10-06T10:39:16.321328+0000","last_clean":"2023-10-06T10:39:16.321328+0000","last_became_active":"2023-10-06T10:39:16.317178+0000","last_became_peered":"2023-10-06T10:39:16.317178+0000","last_unstale":"2023-10-06T10:39:16.321328+0000","last_undegraded":"2023-10-06T10:39:16.321328+0000","last_fullsized":"2023-10-06T10:39:16.321328+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:18.818780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.331333+0000","last_change":"2023-10-06T10:39:15.305959+0000","last_active":"2023-10-06T10:39:16.331333+0000","last_peered":"2023-10-06T10:39:16.331333+0000","last_clean":"2023-10-06T10:39:16.331333+0000","last_became_active":"2023-10-06T10:39:15.305700+0000","last_became_peered":"2023-10-06T10:39:15.305700+0000","last_unstale":"2023-10-06T10:39:16.331333+0000","last_undegraded":"2023-10-06T10:39:16.331333+0000","last_fullsized":"2023-10-06T10:39:16.331333+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:43:46.247836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.319525+0000","last_change":"2023-10-06T10:39:16.313841+0000","last_active":"2023-10-06T10:39:16.319525+0000","last_peered":"2023-10-06T10:39:16.319525+0000","last_clean":"2023-10-06T10:39:16.319525+0000","last_became_active":"2023-10-06T10:39:16.313587+0000","last_became_peered":"2023-10-06T10:39:16.313587+0000","last_unstale":"2023-10-06T10:39:16.319525+0000","last_undegraded":"2023-10-06T10:39:16.319525+0000","last_fullsized":"2023-10-06T10:39:16.319525+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:37.849449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.330534+0000","last_change":"2023-10-06T10:39:15.304964+0000","last_active":"2023-10-06T10:39:16.330534+0000","last_peered":"2023-10-06T10:39:16.330534+0000","last_clean":"2023-10-06T10:39:16.330534+0000","last_became_active":"2023-10-06T10:39:15.304284+0000","last_became_peered":"2023-10-06T10:39:15.304284+0000","last_unstale":"2023-10-06T10:39:16.330534+0000","last_undegraded":"2023-10-06T10:39:16.330534+0000","last_fullsized":"2023-10-06T10:39:16.330534+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:43:46.247836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.319776+0000","last_change":"2023-10-06T10:39:16.317078+0000","last_active":"2023-10-06T10:39:16.319776+0000","last_peered":"2023-10-06T10:39:16.319776+0000","last_clean":"2023-10-06T10:39:16.319776+0000","last_became_active":"2023-10-06T10:39:16.316806+0000","last_became_peered":"2023-10-06T10:39:16.316806+0000","last_unstale":"2023-10-06T10:39:16.319776+0000","last_undegraded":"2023-10-06T10:39:16.319776+0000","last_fullsized":"2023-10-06T10:39:16.319776+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:39.271089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.304432+0000","last_change":"2023-10-06T10:39:15.296453+0000","last_active":"2023-10-06T10:39:16.304432+0000","last_peered":"2023-10-06T10:39:16.304432+0000","last_clean":"2023-10-06T10:39:16.304432+0000","last_became_active":"2023-10-06T10:39:15.296109+0000","last_became_peered":"2023-10-06T10:39:15.296109+0000","last_unstale":"2023-10-06T10:39:16.304432+0000","last_undegraded":"2023-10-06T10:39:16.304432+0000","last_fullsized":"2023-10-06T10:39:16.304432+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:00.467864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.325794+0000","last_change":"2023-10-06T10:39:16.320973+0000","last_active":"2023-10-06T10:39:16.325794+0000","last_peered":"2023-10-06T10:39:16.325794+0000","last_clean":"2023-10-06T10:39:16.325794+0000","last_became_active":"2023-10-06T10:39:16.320156+0000","last_became_peered":"2023-10-06T10:39:16.320156+0000","last_unstale":"2023-10-06T10:39:16.325794+0000","last_undegraded":"2023-10-06T10:39:16.325794+0000","last_fullsized":"2023-10-06T10:39:16.325794+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:22.495874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.305343+0000","last_change":"2023-10-06T10:39:15.302908+0000","last_active":"2023-10-06T10:39:16.305343+0000","last_peered":"2023-10-06T10:39:16.305343+0000","last_clean":"2023-10-06T10:39:16.305343+0000","last_became_active":"2023-10-06T10:39:15.302669+0000","last_became_peered":"2023-10-06T10:39:15.302669+0000","last_unstale":"2023-10-06T10:39:16.305343+0000","last_undegraded":"2023-10-06T10:39:16.305343+0000","last_fullsized":"2023-10-06T10:39:16.305343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:21.494371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.321309+0000","last_change":"2023-10-06T10:39:16.315648+0000","last_active":"2023-10-06T10:39:16.321309+0000","last_peered":"2023-10-06T10:39:16.321309+0000","last_clean":"2023-10-06T10:39:16.321309+0000","last_became_active":"2023-10-06T10:39:16.315340+0000","last_became_peered":"2023-10-06T10:39:16.315340+0000","last_unstale":"2023-10-06T10:39:16.321309+0000","last_undegraded":"2023-10-06T10:39:16.321309+0000","last_fullsized":"2023-10-06T10:39:16.321309+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:37.202391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.305278+0000","last_change":"2023-10-06T10:39:15.304944+0000","last_active":"2023-10-06T10:39:16.305278+0000","last_peered":"2023-10-06T10:39:16.305278+0000","last_clean":"2023-10-06T10:39:16.305278+0000","last_became_active":"2023-10-06T10:39:15.304656+0000","last_became_peered":"2023-10-06T10:39:15.304656+0000","last_unstale":"2023-10-06T10:39:16.305278+0000","last_undegraded":"2023-10-06T10:39:16.305278+0000","last_fullsized":"2023-10-06T10:39:16.305278+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:33.419655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.323326+0000","last_change":"2023-10-06T10:39:16.318161+0000","last_active":"2023-10-06T10:39:16.323326+0000","last_peered":"2023-10-06T10:39:16.323326+0000","last_clean":"2023-10-06T10:39:16.323326+0000","last_became_active":"2023-10-06T10:39:16.317892+0000","last_became_peered":"2023-10-06T10:39:16.317892+0000","last_unstale":"2023-10-06T10:39:16.323326+0000","last_undegraded":"2023-10-06T10:39:16.323326+0000","last_fullsized":"2023-10-06T10:39:16.323326+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:43:47.249338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.309114+0000","last_change":"2023-10-06T10:39:15.305389+0000","last_active":"2023-10-06T10:39:16.309114+0000","last_peered":"2023-10-06T10:39:16.309114+0000","last_clean":"2023-10-06T10:39:16.309114+0000","last_became_active":"2023-10-06T10:39:15.305101+0000","last_became_peered":"2023-10-06T10:39:15.305101+0000","last_unstale":"2023-10-06T10:39:16.309114+0000","last_undegraded":"2023-10-06T10:39:16.309114+0000","last_fullsized":"2023-10-06T10:39:16.309114+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:14.029154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.318167+0000","last_change":"2023-10-06T10:39:16.314462+0000","last_active":"2023-10-06T10:39:16.318167+0000","last_peered":"2023-10-06T10:39:16.318167+0000","last_clean":"2023-10-06T10:39:16.318167+0000","last_became_active":"2023-10-06T10:39:16.314119+0000","last_became_peered":"2023-10-06T10:39:16.314119+0000","last_unstale":"2023-10-06T10:39:16.318167+0000","last_undegraded":"2023-10-06T10:39:16.318167+0000","last_fullsized":"2023-10-06T10:39:16.318167+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:37.593925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"37'3","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.330885+0000","last_change":"2023-10-06T10:39:15.310013+0000","last_active":"2023-10-06T10:39:16.330885+0000","last_peered":"2023-10-06T10:39:16.330885+0000","last_clean":"2023-10-06T10:39:16.330885+0000","last_became_active":"2023-10-06T10:39:15.309671+0000","last_became_peered":"2023-10-06T10:39:15.309671+0000","last_unstale":"2023-10-06T10:39:16.330885+0000","last_undegraded":"2023-10-06T10:39:16.330885+0000","last_fullsized":"2023-10-06T10:39:16.330885+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:23.575555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.322586+0000","last_change":"2023-10-06T10:39:16.312815+0000","last_active":"2023-10-06T10:39:16.322586+0000","last_peered":"2023-10-06T10:39:16.322586+0000","last_clean":"2023-10-06T10:39:16.322586+0000","last_became_active":"2023-10-06T10:39:16.312579+0000","last_became_peered":"2023-10-06T10:39:16.312579+0000","last_unstale":"2023-10-06T10:39:16.322586+0000","last_undegraded":"2023-10-06T10:39:16.322586+0000","last_fullsized":"2023-10-06T10:39:16.322586+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:34.421157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"37'6","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.335534+0000","last_change":"2023-10-06T10:39:15.303225+0000","last_active":"2023-10-06T10:39:16.335534+0000","last_peered":"2023-10-06T10:39:16.335534+0000","last_clean":"2023-10-06T10:39:16.335534+0000","last_became_active":"2023-10-06T10:39:15.302931+0000","last_became_peered":"2023-10-06T10:39:15.302931+0000","last_unstale":"2023-10-06T10:39:16.335534+0000","last_undegraded":"2023-10-06T10:39:16.335534+0000","last_fullsized":"2023-10-06T10:39:16.335534+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:45:40.374013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.325891+0000","last_change":"2023-10-06T10:39:16.318415+0000","last_active":"2023-10-06T10:39:16.325891+0000","last_peered":"2023-10-06T10:39:16.325891+0000","last_clean":"2023-10-06T10:39:16.325891+0000","last_became_active":"2023-10-06T10:39:16.318163+0000","last_became_peered":"2023-10-06T10:39:16.318163+0000","last_unstale":"2023-10-06T10:39:16.325891+0000","last_undegraded":"2023-10-06T10:39:16.325891+0000","last_fullsized":"2023-10-06T10:39:16.325891+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:11:40.614228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.335426+0000","last_change":"2023-10-06T10:39:15.307620+0000","last_active":"2023-10-06T10:39:16.335426+0000","last_peered":"2023-10-06T10:39:16.335426+0000","last_clean":"2023-10-06T10:39:16.335426+0000","last_became_active":"2023-10-06T10:39:15.307406+0000","last_became_peered":"2023-10-06T10:39:15.307406+0000","last_unstale":"2023-10-06T10:39:16.335426+0000","last_undegraded":"2023-10-06T10:39:16.335426+0000","last_fullsized":"2023-10-06T10:39:16.335426+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:18:16.503018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.319543+0000","last_change":"2023-10-06T10:39:16.314298+0000","last_active":"2023-10-06T10:39:16.319543+0000","last_peered":"2023-10-06T10:39:16.319543+0000","last_clean":"2023-10-06T10:39:16.319543+0000","last_became_active":"2023-10-06T10:39:16.314055+0000","last_became_peered":"2023-10-06T10:39:16.314055+0000","last_unstale":"2023-10-06T10:39:16.319543+0000","last_undegraded":"2023-10-06T10:39:16.319543+0000","last_fullsized":"2023-10-06T10:39:16.319543+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:01:06.914043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.311319+0000","last_change":"2023-10-06T10:39:15.305805+0000","last_active":"2023-10-06T10:39:16.311319+0000","last_peered":"2023-10-06T10:39:16.311319+0000","last_clean":"2023-10-06T10:39:16.311319+0000","last_became_active":"2023-10-06T10:39:15.305554+0000","last_became_peered":"2023-10-06T10:39:15.305554+0000","last_unstale":"2023-10-06T10:39:16.311319+0000","last_undegraded":"2023-10-06T10:39:16.311319+0000","last_fullsized":"2023-10-06T10:39:16.311319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:03.257925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.317529+0000","last_change":"2023-10-06T10:39:16.312332+0000","last_active":"2023-10-06T10:39:16.317529+0000","last_peered":"2023-10-06T10:39:16.317529+0000","last_clean":"2023-10-06T10:39:16.317529+0000","last_became_active":"2023-10-06T10:39:16.312115+0000","last_became_peered":"2023-10-06T10:39:16.312115+0000","last_unstale":"2023-10-06T10:39:16.317529+0000","last_undegraded":"2023-10-06T10:39:16.317529+0000","last_fullsized":"2023-10-06T10:39:16.317529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:39.271089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.305493+0000","last_change":"2023-10-06T10:39:15.306640+0000","last_active":"2023-10-06T10:39:16.305493+0000","last_peered":"2023-10-06T10:39:16.305493+0000","last_clean":"2023-10-06T10:39:16.305493+0000","last_became_active":"2023-10-06T10:39:15.306414+0000","last_became_peered":"2023-10-06T10:39:15.306414+0000","last_unstale":"2023-10-06T10:39:16.305493+0000","last_undegraded":"2023-10-06T10:39:16.305493+0000","last_fullsized":"2023-10-06T10:39:16.305493+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:35:12.123388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.323403+0000","last_change":"2023-10-06T10:39:16.321137+0000","last_active":"2023-10-06T10:39:16.323403+0000","last_peered":"2023-10-06T10:39:16.323403+0000","last_clean":"2023-10-06T10:39:16.323403+0000","last_became_active":"2023-10-06T10:39:16.320781+0000","last_became_peered":"2023-10-06T10:39:16.320781+0000","last_unstale":"2023-10-06T10:39:16.323403+0000","last_undegraded":"2023-10-06T10:39:16.323403+0000","last_fullsized":"2023-10-06T10:39:16.323403+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:27.557844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"37'8","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.335484+0000","last_change":"2023-10-06T10:39:15.302166+0000","last_active":"2023-10-06T10:39:16.335484+0000","last_peered":"2023-10-06T10:39:16.335484+0000","last_clean":"2023-10-06T10:39:16.335484+0000","last_became_active":"2023-10-06T10:39:15.301014+0000","last_became_peered":"2023-10-06T10:39:15.301014+0000","last_unstale":"2023-10-06T10:39:16.335484+0000","last_undegraded":"2023-10-06T10:39:16.335484+0000","last_fullsized":"2023-10-06T10:39:16.335484+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:12.215980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.310559+0000","last_change":"2023-10-06T10:39:16.308161+0000","last_active":"2023-10-06T10:39:16.310559+0000","last_peered":"2023-10-06T10:39:16.310559+0000","last_clean":"2023-10-06T10:39:16.310559+0000","last_became_active":"2023-10-06T10:39:16.307897+0000","last_became_peered":"2023-10-06T10:39:16.307897+0000","last_unstale":"2023-10-06T10:39:16.310559+0000","last_undegraded":"2023-10-06T10:39:16.310559+0000","last_fullsized":"2023-10-06T10:39:16.310559+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:37:59.945308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.327092+0000","last_change":"2023-10-06T10:39:15.304372+0000","last_active":"2023-10-06T10:39:16.327092+0000","last_peered":"2023-10-06T10:39:16.327092+0000","last_clean":"2023-10-06T10:39:16.327092+0000","last_became_active":"2023-10-06T10:39:15.304115+0000","last_became_peered":"2023-10-06T10:39:15.304115+0000","last_unstale":"2023-10-06T10:39:16.327092+0000","last_undegraded":"2023-10-06T10:39:16.327092+0000","last_fullsized":"2023-10-06T10:39:16.327092+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:39.434527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.322741+0000","last_change":"2023-10-06T10:39:16.320403+0000","last_active":"2023-10-06T10:39:16.322741+0000","last_peered":"2023-10-06T10:39:16.322741+0000","last_clean":"2023-10-06T10:39:16.322741+0000","last_became_active":"2023-10-06T10:39:16.319843+0000","last_became_peered":"2023-10-06T10:39:16.319843+0000","last_unstale":"2023-10-06T10:39:16.322741+0000","last_undegraded":"2023-10-06T10:39:16.322741+0000","last_fullsized":"2023-10-06T10:39:16.322741+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:37:00.510491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.333067+0000","last_change":"2023-10-06T10:39:15.307246+0000","last_active":"2023-10-06T10:39:16.333067+0000","last_peered":"2023-10-06T10:39:16.333067+0000","last_clean":"2023-10-06T10:39:16.333067+0000","last_became_active":"2023-10-06T10:39:15.306977+0000","last_became_peered":"2023-10-06T10:39:15.306977+0000","last_unstale":"2023-10-06T10:39:16.333067+0000","last_undegraded":"2023-10-06T10:39:16.333067+0000","last_fullsized":"2023-10-06T10:39:16.333067+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:18:16.503018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.324074+0000","last_change":"2023-10-06T10:39:16.316484+0000","last_active":"2023-10-06T10:39:16.324074+0000","last_peered":"2023-10-06T10:39:16.324074+0000","last_clean":"2023-10-06T10:39:16.324074+0000","last_became_active":"2023-10-06T10:39:16.316256+0000","last_became_peered":"2023-10-06T10:39:16.316256+0000","last_unstale":"2023-10-06T10:39:16.324074+0000","last_undegraded":"2023-10-06T10:39:16.324074+0000","last_fullsized":"2023-10-06T10:39:16.324074+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:13.217482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.330336+0000","last_change":"2023-10-06T10:39:15.295445+0000","last_active":"2023-10-06T10:39:16.330336+0000","last_peered":"2023-10-06T10:39:16.330336+0000","last_clean":"2023-10-06T10:39:16.330336+0000","last_became_active":"2023-10-06T10:39:15.295187+0000","last_became_peered":"2023-10-06T10:39:15.295187+0000","last_unstale":"2023-10-06T10:39:16.330336+0000","last_undegraded":"2023-10-06T10:39:16.330336+0000","last_fullsized":"2023-10-06T10:39:16.330336+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:01:08.443520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.323905+0000","last_change":"2023-10-06T10:39:16.313889+0000","last_active":"2023-10-06T10:39:16.323905+0000","last_peered":"2023-10-06T10:39:16.323905+0000","last_clean":"2023-10-06T10:39:16.323905+0000","last_became_active":"2023-10-06T10:39:16.313650+0000","last_became_peered":"2023-10-06T10:39:16.313650+0000","last_unstale":"2023-10-06T10:39:16.323905+0000","last_undegraded":"2023-10-06T10:39:16.323905+0000","last_fullsized":"2023-10-06T10:39:16.323905+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:10.202434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.305959+0000","last_change":"2023-10-06T10:39:15.305692+0000","last_active":"2023-10-06T10:39:16.305959+0000","last_peered":"2023-10-06T10:39:16.305959+0000","last_clean":"2023-10-06T10:39:16.305959+0000","last_became_active":"2023-10-06T10:39:15.305442+0000","last_became_peered":"2023-10-06T10:39:15.305442+0000","last_unstale":"2023-10-06T10:39:16.305959+0000","last_undegraded":"2023-10-06T10:39:16.305959+0000","last_fullsized":"2023-10-06T10:39:16.305959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:53.257606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.318038+0000","last_change":"2023-10-06T10:39:16.313925+0000","last_active":"2023-10-06T10:39:16.318038+0000","last_peered":"2023-10-06T10:39:16.318038+0000","last_clean":"2023-10-06T10:39:16.318038+0000","last_became_active":"2023-10-06T10:39:16.313679+0000","last_became_peered":"2023-10-06T10:39:16.313679+0000","last_unstale":"2023-10-06T10:39:16.318038+0000","last_undegraded":"2023-10-06T10:39:16.318038+0000","last_fullsized":"2023-10-06T10:39:16.318038+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:09.527903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.331399+0000","last_change":"2023-10-06T10:39:15.305483+0000","last_active":"2023-10-06T10:39:16.331399+0000","last_peered":"2023-10-06T10:39:16.331399+0000","last_clean":"2023-10-06T10:39:16.331399+0000","last_became_active":"2023-10-06T10:39:15.305222+0000","last_became_peered":"2023-10-06T10:39:15.305222+0000","last_unstale":"2023-10-06T10:39:16.331399+0000","last_undegraded":"2023-10-06T10:39:16.331399+0000","last_fullsized":"2023-10-06T10:39:16.331399+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:16:37.935892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.318312+0000","last_change":"2023-10-06T10:39:16.313915+0000","last_active":"2023-10-06T10:39:16.318312+0000","last_peered":"2023-10-06T10:39:16.318312+0000","last_clean":"2023-10-06T10:39:16.318312+0000","last_became_active":"2023-10-06T10:39:16.313678+0000","last_became_peered":"2023-10-06T10:39:16.313678+0000","last_unstale":"2023-10-06T10:39:16.318312+0000","last_undegraded":"2023-10-06T10:39:16.318312+0000","last_fullsized":"2023-10-06T10:39:16.318312+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:11:40.614228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.304606+0000","last_change":"2023-10-06T10:39:15.303138+0000","last_active":"2023-10-06T10:39:16.304606+0000","last_peered":"2023-10-06T10:39:16.304606+0000","last_clean":"2023-10-06T10:39:16.304606+0000","last_became_active":"2023-10-06T10:39:15.302847+0000","last_became_peered":"2023-10-06T10:39:15.302847+0000","last_unstale":"2023-10-06T10:39:16.304606+0000","last_undegraded":"2023-10-06T10:39:16.304606+0000","last_fullsized":"2023-10-06T10:39:16.304606+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:53.257606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.310661+0000","last_change":"2023-10-06T10:39:16.308674+0000","last_active":"2023-10-06T10:39:16.310661+0000","last_peered":"2023-10-06T10:39:16.310661+0000","last_clean":"2023-10-06T10:39:16.310661+0000","last_became_active":"2023-10-06T10:39:16.308434+0000","last_became_peered":"2023-10-06T10:39:16.308434+0000","last_unstale":"2023-10-06T10:39:16.310661+0000","last_undegraded":"2023-10-06T10:39:16.310661+0000","last_fullsized":"2023-10-06T10:39:16.310661+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:22.495874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.306585+0000","last_change":"2023-10-06T10:39:15.307533+0000","last_active":"2023-10-06T10:39:16.306585+0000","last_peered":"2023-10-06T10:39:16.306585+0000","last_clean":"2023-10-06T10:39:16.306585+0000","last_became_active":"2023-10-06T10:39:15.307133+0000","last_became_peered":"2023-10-06T10:39:15.307133+0000","last_unstale":"2023-10-06T10:39:16.306585+0000","last_undegraded":"2023-10-06T10:39:16.306585+0000","last_fullsized":"2023-10-06T10:39:16.306585+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:09.200932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.319480+0000","last_change":"2023-10-06T10:39:16.313614+0000","last_active":"2023-10-06T10:39:16.319480+0000","last_peered":"2023-10-06T10:39:16.319480+0000","last_clean":"2023-10-06T10:39:16.319480+0000","last_became_active":"2023-10-06T10:39:16.313354+0000","last_became_peered":"2023-10-06T10:39:16.313354+0000","last_unstale":"2023-10-06T10:39:16.319480+0000","last_undegraded":"2023-10-06T10:39:16.319480+0000","last_fullsized":"2023-10-06T10:39:16.319480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:35:13.124890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.306962+0000","last_change":"2023-10-06T10:39:15.307579+0000","last_active":"2023-10-06T10:39:16.306962+0000","last_peered":"2023-10-06T10:39:16.306962+0000","last_clean":"2023-10-06T10:39:16.306962+0000","last_became_active":"2023-10-06T10:39:15.306595+0000","last_became_peered":"2023-10-06T10:39:15.306595+0000","last_unstale":"2023-10-06T10:39:16.306962+0000","last_undegraded":"2023-10-06T10:39:16.306962+0000","last_fullsized":"2023-10-06T10:39:16.306962+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:53.257606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.317359+0000","last_change":"2023-10-06T10:39:16.312200+0000","last_active":"2023-10-06T10:39:16.317359+0000","last_peered":"2023-10-06T10:39:16.317359+0000","last_clean":"2023-10-06T10:39:16.317359+0000","last_became_active":"2023-10-06T10:39:16.311942+0000","last_became_peered":"2023-10-06T10:39:16.311942+0000","last_unstale":"2023-10-06T10:39:16.317359+0000","last_undegraded":"2023-10-06T10:39:16.317359+0000","last_fullsized":"2023-10-06T10:39:16.317359+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:35:29.567370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.303846+0000","last_change":"2023-10-06T10:39:15.309096+0000","last_active":"2023-10-06T10:39:16.303846+0000","last_peered":"2023-10-06T10:39:16.303846+0000","last_clean":"2023-10-06T10:39:16.303846+0000","last_became_active":"2023-10-06T10:39:15.308781+0000","last_became_peered":"2023-10-06T10:39:15.308781+0000","last_unstale":"2023-10-06T10:39:16.303846+0000","last_undegraded":"2023-10-06T10:39:16.303846+0000","last_fullsized":"2023-10-06T10:39:16.303846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:35:12.123388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.305169+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:39:16.305169+0000","last_peered":"2023-10-06T10:39:16.305169+0000","last_clean":"2023-10-06T10:39:16.305169+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:39:16.305169+0000","last_undegraded":"2023-10-06T10:39:16.305169+0000","last_fullsized":"2023-10-06T10:39:16.305169+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.318171+0000","last_change":"2023-10-06T10:39:16.315971+0000","last_active":"2023-10-06T10:39:16.318171+0000","last_peered":"2023-10-06T10:39:16.318171+0000","last_clean":"2023-10-06T10:39:16.318171+0000","last_became_active":"2023-10-06T10:39:16.315630+0000","last_became_peered":"2023-10-06T10:39:16.315630+0000","last_unstale":"2023-10-06T10:39:16.318171+0000","last_undegraded":"2023-10-06T10:39:16.318171+0000","last_fullsized":"2023-10-06T10:39:16.318171+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:09.420170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.307655+0000","last_change":"2023-10-06T10:39:15.307128+0000","last_active":"2023-10-06T10:39:16.307655+0000","last_peered":"2023-10-06T10:39:16.307655+0000","last_clean":"2023-10-06T10:39:16.307655+0000","last_became_active":"2023-10-06T10:39:15.306872+0000","last_became_peered":"2023-10-06T10:39:15.306872+0000","last_unstale":"2023-10-06T10:39:16.307655+0000","last_undegraded":"2023-10-06T10:39:16.307655+0000","last_fullsized":"2023-10-06T10:39:16.307655+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:21.494371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":26,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.305957+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:39:16.305957+0000","last_peered":"2023-10-06T10:39:16.305957+0000","last_clean":"2023-10-06T10:39:16.305957+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:39:16.305957+0000","last_undegraded":"2023-10-06T10:39:16.305957+0000","last_fullsized":"2023-10-06T10:39:16.305957+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.322680+0000","last_change":"2023-10-06T10:39:16.315838+0000","last_active":"2023-10-06T10:39:16.322680+0000","last_peered":"2023-10-06T10:39:16.322680+0000","last_clean":"2023-10-06T10:39:16.322680+0000","last_became_active":"2023-10-06T10:39:16.314901+0000","last_became_peered":"2023-10-06T10:39:16.314901+0000","last_unstale":"2023-10-06T10:39:16.322680+0000","last_undegraded":"2023-10-06T10:39:16.322680+0000","last_fullsized":"2023-10-06T10:39:16.322680+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:18:17.504520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.304733+0000","last_change":"2023-10-06T10:39:15.299404+0000","last_active":"2023-10-06T10:39:16.304733+0000","last_peered":"2023-10-06T10:39:16.304733+0000","last_clean":"2023-10-06T10:39:16.304733+0000","last_became_active":"2023-10-06T10:39:15.299143+0000","last_became_peered":"2023-10-06T10:39:15.299143+0000","last_unstale":"2023-10-06T10:39:16.304733+0000","last_undegraded":"2023-10-06T10:39:16.304733+0000","last_fullsized":"2023-10-06T10:39:16.304733+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:09.200932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":29,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.305945+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:39:16.305945+0000","last_peered":"2023-10-06T10:39:16.305945+0000","last_clean":"2023-10-06T10:39:16.305945+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:39:16.305945+0000","last_undegraded":"2023-10-06T10:39:16.305945+0000","last_fullsized":"2023-10-06T10:39:16.305945+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":175,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.307079+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:39:16.307079+0000","last_peered":"2023-10-06T10:39:16.307079+0000","last_clean":"2023-10-06T10:39:16.307079+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:39:16.307079+0000","last_undegraded":"2023-10-06T10:39:16.307079+0000","last_fullsized":"2023-10-06T10:39:16.307079+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.329635+0000","last_change":"2023-10-06T10:39:16.323035+0000","last_active":"2023-10-06T10:39:16.329635+0000","last_peered":"2023-10-06T10:39:16.329635+0000","last_clean":"2023-10-06T10:39:16.329635+0000","last_became_active":"2023-10-06T10:39:16.321484+0000","last_became_peered":"2023-10-06T10:39:16.321484+0000","last_unstale":"2023-10-06T10:39:16.329635+0000","last_undegraded":"2023-10-06T10:39:16.329635+0000","last_fullsized":"2023-10-06T10:39:16.329635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:16:40.720156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.310098+0000","last_change":"2023-10-06T10:39:15.310293+0000","last_active":"2023-10-06T10:39:16.310098+0000","last_peered":"2023-10-06T10:39:16.310098+0000","last_clean":"2023-10-06T10:39:16.310098+0000","last_became_active":"2023-10-06T10:39:15.310044+0000","last_became_peered":"2023-10-06T10:39:15.310044+0000","last_unstale":"2023-10-06T10:39:16.310098+0000","last_undegraded":"2023-10-06T10:39:16.310098+0000","last_fullsized":"2023-10-06T10:39:16.310098+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:11:39.612726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":25,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.311402+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:39:16.311402+0000","last_peered":"2023-10-06T10:39:16.311402+0000","last_clean":"2023-10-06T10:39:16.311402+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:39:16.311402+0000","last_undegraded":"2023-10-06T10:39:16.311402+0000","last_fullsized":"2023-10-06T10:39:16.311402+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.323263+0000","last_change":"2023-10-06T10:39:16.320245+0000","last_active":"2023-10-06T10:39:16.323263+0000","last_peered":"2023-10-06T10:39:16.323263+0000","last_clean":"2023-10-06T10:39:16.323263+0000","last_became_active":"2023-10-06T10:39:16.320019+0000","last_became_peered":"2023-10-06T10:39:16.320019+0000","last_unstale":"2023-10-06T10:39:16.323263+0000","last_undegraded":"2023-10-06T10:39:16.323263+0000","last_fullsized":"2023-10-06T10:39:16.323263+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:37.202391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.330559+0000","last_change":"2023-10-06T10:39:15.304330+0000","last_active":"2023-10-06T10:39:16.330559+0000","last_peered":"2023-10-06T10:39:16.330559+0000","last_clean":"2023-10-06T10:39:16.330559+0000","last_became_active":"2023-10-06T10:39:15.304075+0000","last_became_peered":"2023-10-06T10:39:15.304075+0000","last_unstale":"2023-10-06T10:39:16.330559+0000","last_undegraded":"2023-10-06T10:39:16.330559+0000","last_fullsized":"2023-10-06T10:39:16.330559+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:54.214826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":25,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.306962+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:39:16.306962+0000","last_peered":"2023-10-06T10:39:16.306962+0000","last_clean":"2023-10-06T10:39:16.306962+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:39:16.306962+0000","last_undegraded":"2023-10-06T10:39:16.306962+0000","last_fullsized":"2023-10-06T10:39:16.306962+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.327231+0000","last_change":"2023-10-06T10:39:16.324174+0000","last_active":"2023-10-06T10:39:16.327231+0000","last_peered":"2023-10-06T10:39:16.327231+0000","last_clean":"2023-10-06T10:39:16.327231+0000","last_became_active":"2023-10-06T10:39:16.323465+0000","last_became_peered":"2023-10-06T10:39:16.323465+0000","last_unstale":"2023-10-06T10:39:16.327231+0000","last_undegraded":"2023-10-06T10:39:16.327231+0000","last_fullsized":"2023-10-06T10:39:16.327231+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:54.259108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T10:39:17.330104+0000","last_change":"2023-10-06T10:39:15.309146+0000","last_active":"2023-10-06T10:39:17.330104+0000","last_peered":"2023-10-06T10:39:17.330104+0000","last_clean":"2023-10-06T10:39:17.330104+0000","last_became_active":"2023-10-06T10:39:15.308885+0000","last_became_peered":"2023-10-06T10:39:15.308885+0000","last_unstale":"2023-10-06T10:39:17.330104+0000","last_undegraded":"2023-10-06T10:39:17.330104+0000","last_fullsized":"2023-10-06T10:39:17.330104+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:25:48.384684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":31,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.304497+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:39:16.304497+0000","last_peered":"2023-10-06T10:39:16.304497+0000","last_clean":"2023-10-06T10:39:16.304497+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:39:16.304497+0000","last_undegraded":"2023-10-06T10:39:16.304497+0000","last_fullsized":"2023-10-06T10:39:16.304497+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.323266+0000","last_change":"2023-10-06T10:39:16.314100+0000","last_active":"2023-10-06T10:39:16.323266+0000","last_peered":"2023-10-06T10:39:16.323266+0000","last_clean":"2023-10-06T10:39:16.323266+0000","last_became_active":"2023-10-06T10:39:16.313841+0000","last_became_peered":"2023-10-06T10:39:16.313841+0000","last_unstale":"2023-10-06T10:39:16.323266+0000","last_undegraded":"2023-10-06T10:39:16.323266+0000","last_fullsized":"2023-10-06T10:39:16.323266+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:35:13.124890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.304709+0000","last_change":"2023-10-06T10:39:15.302089+0000","last_active":"2023-10-06T10:39:16.304709+0000","last_peered":"2023-10-06T10:39:16.304709+0000","last_clean":"2023-10-06T10:39:16.304709+0000","last_became_active":"2023-10-06T10:39:15.301841+0000","last_became_peered":"2023-10-06T10:39:15.301841+0000","last_unstale":"2023-10-06T10:39:16.304709+0000","last_undegraded":"2023-10-06T10:39:16.304709+0000","last_fullsized":"2023-10-06T10:39:16.304709+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:14:52.278898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":26,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.309203+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:39:16.309203+0000","last_peered":"2023-10-06T10:39:16.309203+0000","last_clean":"2023-10-06T10:39:16.309203+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:39:16.309203+0000","last_undegraded":"2023-10-06T10:39:16.309203+0000","last_fullsized":"2023-10-06T10:39:16.309203+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.321234+0000","last_change":"2023-10-06T10:39:16.313765+0000","last_active":"2023-10-06T10:39:16.321234+0000","last_peered":"2023-10-06T10:39:16.321234+0000","last_clean":"2023-10-06T10:39:16.321234+0000","last_became_active":"2023-10-06T10:39:16.313502+0000","last_became_peered":"2023-10-06T10:39:16.313502+0000","last_unstale":"2023-10-06T10:39:16.321234+0000","last_undegraded":"2023-10-06T10:39:16.321234+0000","last_fullsized":"2023-10-06T10:39:16.321234+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:10.131667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.306285+0000","last_change":"2023-10-06T10:39:15.305824+0000","last_active":"2023-10-06T10:39:16.306285+0000","last_peered":"2023-10-06T10:39:16.306285+0000","last_clean":"2023-10-06T10:39:16.306285+0000","last_became_active":"2023-10-06T10:39:15.305587+0000","last_became_peered":"2023-10-06T10:39:15.305587+0000","last_unstale":"2023-10-06T10:39:16.306285+0000","last_undegraded":"2023-10-06T10:39:16.306285+0000","last_fullsized":"2023-10-06T10:39:16.306285+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:54.245211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":26,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.311588+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:39:16.311588+0000","last_peered":"2023-10-06T10:39:16.311588+0000","last_clean":"2023-10-06T10:39:16.311588+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:39:16.311588+0000","last_undegraded":"2023-10-06T10:39:16.311588+0000","last_fullsized":"2023-10-06T10:39:16.311588+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298300,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073540,"statfs":{"total":96636764160,"available":96331304960,"internally_reserved":0,"allocated":1036288,"data_stored":786386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.436}]},{"osd":1,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.375},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.47099999999999997}]},{"osd":2,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.32400000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.41399999999999998}]},{"osd":4,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.38600000000000001}]},{"osd":5,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.40400000000000003}]},{"osd":6,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.46000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297720,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074120,"statfs":{"total":96636764160,"available":96331898880,"internally_reserved":0,"allocated":442368,"data_stored":195219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.436},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.60499999999999998}]},{"osd":2,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.48099999999999998}]},{"osd":3,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.57199999999999995}]},{"osd":4,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.39000000000000001}]},{"osd":5,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.5}]},{"osd":7,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.61699999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298300,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073540,"statfs":{"total":96636764160,"available":96331304960,"internally_reserved":0,"allocated":1036288,"data_stored":786366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.27400000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.316,"15min":0.316},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.32400000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.31}]},{"osd":4,"last update":"Fri Oct 6 10:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.48899999999999999}]},{"osd":5,"last update":"Fri Oct 6 10:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.499},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.44500000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.51700000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.53600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297720,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074120,"statfs":{"total":96636764160,"available":96331898880,"internally_reserved":0,"allocated":442368,"data_stored":195962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.316,"15min":0.316},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.432,"5min":0.432,"15min":0.432},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.373}]},{"osd":2,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.42799999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"last":0.40100000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.60299999999999998}]},{"osd":5,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.56399999999999995}]},{"osd":6,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.627}]},{"osd":7,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.63800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297732,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074108,"statfs":{"total":96636764160,"available":96331886592,"internally_reserved":0,"allocated":454656,"data_stored":197486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:39:11 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.214},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.20100000000000001}]},{"osd":1,"last update":"Fri Oct 6 10:39:11 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.26400000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:39:11 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.28899999999999998}]},{"osd":4,"last update":"Fri Oct 6 10:39:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.47299999999999998}]},{"osd":5,"last update":"Fri Oct 6 10:39:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.502},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.46700000000000003}]},{"osd":6,"last update":"Fri Oct 6 10:39:11 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.54800000000000004}]},{"osd":7,"last update":"Fri Oct 6 10:39:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297728,"kb_used_data":440,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074112,"statfs":{"total":96636764160,"available":96331890688,"internally_reserved":0,"allocated":450560,"data_stored":196538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":3.3210000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.28899999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.307},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":3.359,"5min":3.359,"15min":3.359},"last":0.27800000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":7.069,"5min":7.069,"15min":7.069},"last":0.34499999999999997}]},{"osd":4,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.52700000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.4140000000000001,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.437}]},{"osd":6,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.3879999999999999,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.49299999999999999}]},{"osd":7,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.54900000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297728,"kb_used_data":440,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074112,"statfs":{"total":96636764160,"available":96331890688,"internally_reserved":0,"allocated":450560,"data_stored":197464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.46999999999999997}]},{"osd":1,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.33100000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.36799999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.499},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.42499999999999999}]},{"osd":5,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.44600000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.312},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.38900000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.48899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297720,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074120,"statfs":{"total":96636764160,"available":96331898880,"internally_reserved":0,"allocated":442368,"data_stored":195998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.76600000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.59999999999999998}]},{"osd":3,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.65200000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.48399999999999999}]},{"osd":6,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.498}]},{"osd":7,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.46400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:39:20.381 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:39:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.523+0000 7ffab930b700 1 Processor -- start 2023-10-06T10:39:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.523+0000 7ffab930b700 1 -- start start 2023-10-06T10:39:20.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.524+0000 7ffab930b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffab40cd440 0x7ffab41645a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:20.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.524+0000 7ffab930b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab40b0da0 con 0x7ffab40b25e0 2023-10-06T10:39:20.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.524+0000 7ffab930b700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab40cbfa0 con 0x7ffab40b29b0 2023-10-06T10:39:20.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.524+0000 7ffab930b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab40cb390 con 0x7ffab40cd440 2023-10-06T10:39:20.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.524+0000 7ffab37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffab40cd440 0x7ffab41645a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:20.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.524+0000 7ffab2ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ffab40b25e0 0x7ffab40b0690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60224/0 (socket says 172.21.15.92:60224) 2023-10-06T10:39:20.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.524+0000 7ffab37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffab40cd440 0x7ffab41645a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46174/0 (socket says 172.21.15.92:46174) 2023-10-06T10:39:20.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.524+0000 7ffab2ffd700 1 -- 172.21.15.92:0/1189916564 learned_addr learned my addr 172.21.15.92:0/1189916564 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:20.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.525+0000 7ffab37fe700 1 -- 172.21.15.92:0/1189916564 >> v1:172.21.15.124:6789/0 conn(0x7ffab40b29b0 legacy=0x7ffab4159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:20.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.525+0000 7ffab37fe700 1 -- 172.21.15.92:0/1189916564 >> v1:172.21.15.92:6789/0 conn(0x7ffab40b25e0 legacy=0x7ffab40b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:20.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.525+0000 7ffab37fe700 1 -- 172.21.15.92:0/1189916564 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab40cb670 con 0x7ffab40cd440 2023-10-06T10:39:20.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.525+0000 7ffab37fe700 1 --2- 172.21.15.92:0/1189916564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab40cd440 0x7ffab41645a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009e60 tx=0x7ffaac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c309222c10b948 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:20.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.525+0000 7ffab27fc700 1 -- 172.21.15.92:0/1189916564 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac00e070 con 0x7ffab40cd440 2023-10-06T10:39:20.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.525+0000 7ffab27fc700 1 -- 172.21.15.92:0/1189916564 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaac00bbf0 con 0x7ffab40cd440 2023-10-06T10:39:20.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.525+0000 7ffab27fc700 1 -- 172.21.15.92:0/1189916564 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac012430 con 0x7ffab40cd440 2023-10-06T10:39:20.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.525+0000 7ffab930b700 1 -- 172.21.15.92:0/1189916564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab40cd440 msgr2=0x7ffab41645a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:20.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.525+0000 7ffab930b700 1 --2- 172.21.15.92:0/1189916564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab40cd440 0x7ffab41645a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009e60 tx=0x7ffaac009780 comp rx=0 tx=0).stop 2023-10-06T10:39:20.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.525+0000 7ffab930b700 1 -- 172.21.15.92:0/1189916564 shutdown_connections 2023-10-06T10:39:20.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.525+0000 7ffab930b700 1 --2- 172.21.15.92:0/1189916564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab40cd440 0x7ffab41645a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.525+0000 7ffab930b700 1 -- 172.21.15.92:0/1189916564 >> 172.21.15.92:0/1189916564 conn(0x7ffab406ece0 msgr2=0x7ffab406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:20.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.525+0000 7ffab930b700 1 -- 172.21.15.92:0/1189916564 shutdown_connections 2023-10-06T10:39:20.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.525+0000 7ffab930b700 1 -- 172.21.15.92:0/1189916564 wait complete. 2023-10-06T10:39:20.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.527+0000 7ffab930b700 1 Processor -- start 2023-10-06T10:39:20.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.527+0000 7ffab930b700 1 -- start start 2023-10-06T10:39:20.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.527+0000 7ffab930b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffab40b25e0 0x7ffab4150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:20.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.527+0000 7ffab930b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab40b29b0 0x7ffab40c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:20.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.527+0000 7ffab930b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffab40cd440 0x7ffab416a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:20.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.527+0000 7ffab930b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab40c9550 con 0x7ffab40b29b0 2023-10-06T10:39:20.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.527+0000 7ffab930b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab40c8940 con 0x7ffab40b25e0 2023-10-06T10:39:20.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.527+0000 7ffab930b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab40c8c20 con 0x7ffab40cd440 2023-10-06T10:39:20.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.528+0000 7ffaaa7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab40b29b0 0x7ffab40c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:20.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.528+0000 7ffaaa7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab40b29b0 0x7ffab40c7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46178/0 (socket says 172.21.15.92:46178) 2023-10-06T10:39:20.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.528+0000 7ffaaa7fc700 1 -- 172.21.15.92:0/2513358288 learned_addr learned my addr 172.21.15.92:0/2513358288 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:20.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.528+0000 7ffab2ffd700 1 --2- 172.21.15.92:0/2513358288 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffab40b25e0 0x7ffab4150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:20.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.528+0000 7ffab37fe700 1 --2- 172.21.15.92:0/2513358288 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffab40cd440 0x7ffab416a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:20.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.528+0000 7ffaaa7fc700 1 -- 172.21.15.92:0/2513358288 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffab40cd440 msgr2=0x7ffab416a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:20.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.528+0000 7ffaaa7fc700 1 --2- 172.21.15.92:0/2513358288 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffab40cd440 0x7ffab416a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.528+0000 7ffaaa7fc700 1 -- 172.21.15.92:0/2513358288 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffab40b25e0 msgr2=0x7ffab4150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:20.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.528+0000 7ffaaa7fc700 1 --2- 172.21.15.92:0/2513358288 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffab40b25e0 0x7ffab4150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.528+0000 7ffaaa7fc700 1 -- 172.21.15.92:0/2513358288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac009a20 con 0x7ffab40b29b0 2023-10-06T10:39:20.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.528+0000 7ffab37fe700 1 --2- 172.21.15.92:0/2513358288 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffab40cd440 0x7ffab416a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:20.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.528+0000 7ffaaa7fc700 1 --2- 172.21.15.92:0/2513358288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab40b29b0 0x7ffab40c7ef0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00b600 tx=0x7ffa9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:20.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.528+0000 7ffaab7fe700 1 -- 172.21.15.92:0/2513358288 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c00f510 con 0x7ffab40b29b0 2023-10-06T10:39:20.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.528+0000 7ffaab7fe700 1 -- 172.21.15.92:0/2513358288 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9c00f670 con 0x7ffab40b29b0 2023-10-06T10:39:20.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.528+0000 7ffab930b700 1 -- 172.21.15.92:0/2513358288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab40c8f00 con 0x7ffab40b29b0 2023-10-06T10:39:20.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.528+0000 7ffaab7fe700 1 -- 172.21.15.92:0/2513358288 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c00fa30 con 0x7ffab40b29b0 2023-10-06T10:39:20.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.528+0000 7ffab930b700 1 -- 172.21.15.92:0/2513358288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab41510d0 con 0x7ffab40b29b0 2023-10-06T10:39:20.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.529+0000 7ffab930b700 1 -- 172.21.15.92:0/2513358288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab4064a90 con 0x7ffab40b29b0 2023-10-06T10:39:20.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.530+0000 7ffaab7fe700 1 -- 172.21.15.92:0/2513358288 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffa9c01a400 con 0x7ffab40b29b0 2023-10-06T10:39:20.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.530+0000 7ffaab7fe700 1 --2- 172.21.15.92:0/2513358288 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ffa94066f30 0x7ffa940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:20.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.530+0000 7ffaab7fe700 1 -- 172.21.15.92:0/2513358288 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ffa9c0842f0 con 0x7ffab40b29b0 2023-10-06T10:39:20.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.530+0000 7ffab2ffd700 1 --2- 172.21.15.92:0/2513358288 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ffa94066f30 0x7ffa940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:20.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.530+0000 7ffab2ffd700 1 --2- 172.21.15.92:0/2513358288 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ffa94066f30 0x7ffa940693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffaa40010e0 tx=0x7ffaa4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:20.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.534+0000 7ffaab7fe700 1 -- 172.21.15.92:0/2513358288 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa9c051340 con 0x7ffab40b29b0 2023-10-06T10:39:20.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.713+0000 7ffab930b700 1 -- 172.21.15.92:0/2513358288 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffab40cb950 con 0x7ffa94066f30 2023-10-06T10:39:20.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.722+0000 7ffaab7fe700 1 -- 172.21.15.92:0/2513358288 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311807 (secure 0 0 0) 0x7ffab40cb950 con 0x7ffa94066f30 2023-10-06T10:39:20.727 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:39:20.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.725+0000 7ffab930b700 1 -- 172.21.15.92:0/2513358288 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ffa94066f30 msgr2=0x7ffa940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:20.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.725+0000 7ffab930b700 1 --2- 172.21.15.92:0/2513358288 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ffa94066f30 0x7ffa940693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffaa40010e0 tx=0x7ffaa4009040 comp rx=0 tx=0).stop 2023-10-06T10:39:20.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.725+0000 7ffab930b700 1 -- 172.21.15.92:0/2513358288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab40b29b0 msgr2=0x7ffab40c7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:20.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.725+0000 7ffab930b700 1 --2- 172.21.15.92:0/2513358288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab40b29b0 0x7ffab40c7ef0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00b600 tx=0x7ffa9c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.725+0000 7ffab930b700 1 -- 172.21.15.92:0/2513358288 shutdown_connections 2023-10-06T10:39:20.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.725+0000 7ffab930b700 1 --2- 172.21.15.92:0/2513358288 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ffa94066f30 0x7ffa940693e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.725+0000 7ffab930b700 1 --2- 172.21.15.92:0/2513358288 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffab40cd440 0x7ffab416a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.725+0000 7ffab930b700 1 --2- 172.21.15.92:0/2513358288 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffab40b25e0 0x7ffab4150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.725+0000 7ffab930b700 1 --2- 172.21.15.92:0/2513358288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab40b29b0 0x7ffab40c7ef0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.725+0000 7ffab930b700 1 -- 172.21.15.92:0/2513358288 >> 172.21.15.92:0/2513358288 conn(0x7ffab406ece0 msgr2=0x7ffab4153bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:20.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.726+0000 7ffab930b700 1 -- 172.21.15.92:0/2513358288 shutdown_connections 2023-10-06T10:39:20.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.726+0000 7ffab930b700 1 -- 172.21.15.92:0/2513358288 wait complete. 2023-10-06T10:39:20.731 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:39:20.738 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2023-10-06T10:39:20.094690+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382948,"kb_used_data":4644,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752591772,"statfs":{"total":773094113280,"available":770653974528,"internally_reserved":0,"allocated":4755456,"data_stored":2751419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.805649"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.324564+0000","last_change":"2023-10-06T10:39:16.322743+0000","last_active":"2023-10-06T10:39:16.324564+0000","last_peered":"2023-10-06T10:39:16.324564+0000","last_clean":"2023-10-06T10:39:16.324564+0000","last_became_active":"2023-10-06T10:39:16.322467+0000","last_became_peered":"2023-10-06T10:39:16.322467+0000","last_unstale":"2023-10-06T10:39:16.324564+0000","last_undegraded":"2023-10-06T10:39:16.324564+0000","last_fullsized":"2023-10-06T10:39:16.324564+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:16:40.720156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.323776+0000","last_change":"2023-10-06T10:39:16.315729+0000","last_active":"2023-10-06T10:39:16.323776+0000","last_peered":"2023-10-06T10:39:16.323776+0000","last_clean":"2023-10-06T10:39:16.323776+0000","last_became_active":"2023-10-06T10:39:16.315440+0000","last_became_peered":"2023-10-06T10:39:16.315440+0000","last_unstale":"2023-10-06T10:39:16.323776+0000","last_undegraded":"2023-10-06T10:39:16.323776+0000","last_fullsized":"2023-10-06T10:39:16.323776+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:27:28.759803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.321105+0000","last_change":"2023-10-06T10:39:16.306759+0000","last_active":"2023-10-06T10:39:16.321105+0000","last_peered":"2023-10-06T10:39:16.321105+0000","last_clean":"2023-10-06T10:39:16.321105+0000","last_became_active":"2023-10-06T10:39:16.306466+0000","last_became_peered":"2023-10-06T10:39:16.306466+0000","last_unstale":"2023-10-06T10:39:16.321105+0000","last_undegraded":"2023-10-06T10:39:16.321105+0000","last_fullsized":"2023-10-06T10:39:16.321105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:35:29.567370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.321649+0000","last_change":"2023-10-06T10:39:16.317613+0000","last_active":"2023-10-06T10:39:16.321649+0000","last_peered":"2023-10-06T10:39:16.321649+0000","last_clean":"2023-10-06T10:39:16.321649+0000","last_became_active":"2023-10-06T10:39:16.317197+0000","last_became_peered":"2023-10-06T10:39:16.317197+0000","last_unstale":"2023-10-06T10:39:16.321649+0000","last_undegraded":"2023-10-06T10:39:16.321649+0000","last_fullsized":"2023-10-06T10:39:16.321649+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:46:44.354855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.323813+0000","last_change":"2023-10-06T10:39:16.315614+0000","last_active":"2023-10-06T10:39:16.323813+0000","last_peered":"2023-10-06T10:39:16.323813+0000","last_clean":"2023-10-06T10:39:16.323813+0000","last_became_active":"2023-10-06T10:39:16.315349+0000","last_became_peered":"2023-10-06T10:39:16.315349+0000","last_unstale":"2023-10-06T10:39:16.323813+0000","last_undegraded":"2023-10-06T10:39:16.323813+0000","last_fullsized":"2023-10-06T10:39:16.323813+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:37:00.510491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.318187+0000","last_change":"2023-10-06T10:39:16.316533+0000","last_active":"2023-10-06T10:39:16.318187+0000","last_peered":"2023-10-06T10:39:16.318187+0000","last_clean":"2023-10-06T10:39:16.318187+0000","last_became_active":"2023-10-06T10:39:16.316145+0000","last_became_peered":"2023-10-06T10:39:16.316145+0000","last_unstale":"2023-10-06T10:39:16.318187+0000","last_undegraded":"2023-10-06T10:39:16.318187+0000","last_fullsized":"2023-10-06T10:39:16.318187+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:14:44.803599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.322763+0000","last_change":"2023-10-06T10:39:16.314876+0000","last_active":"2023-10-06T10:39:16.322763+0000","last_peered":"2023-10-06T10:39:16.322763+0000","last_clean":"2023-10-06T10:39:16.322763+0000","last_became_active":"2023-10-06T10:39:16.314617+0000","last_became_peered":"2023-10-06T10:39:16.314617+0000","last_unstale":"2023-10-06T10:39:16.322763+0000","last_undegraded":"2023-10-06T10:39:16.322763+0000","last_fullsized":"2023-10-06T10:39:16.322763+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:35:13.124890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.329662+0000","last_change":"2023-10-06T10:39:16.323233+0000","last_active":"2023-10-06T10:39:16.329662+0000","last_peered":"2023-10-06T10:39:16.329662+0000","last_clean":"2023-10-06T10:39:16.329662+0000","last_became_active":"2023-10-06T10:39:16.322975+0000","last_became_peered":"2023-10-06T10:39:16.322975+0000","last_unstale":"2023-10-06T10:39:16.329662+0000","last_undegraded":"2023-10-06T10:39:16.329662+0000","last_fullsized":"2023-10-06T10:39:16.329662+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:37.593925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.319634+0000","last_change":"2023-10-06T10:39:16.312877+0000","last_active":"2023-10-06T10:39:16.319634+0000","last_peered":"2023-10-06T10:39:16.319634+0000","last_clean":"2023-10-06T10:39:16.319634+0000","last_became_active":"2023-10-06T10:39:16.312625+0000","last_became_peered":"2023-10-06T10:39:16.312625+0000","last_unstale":"2023-10-06T10:39:16.319634+0000","last_undegraded":"2023-10-06T10:39:16.319634+0000","last_fullsized":"2023-10-06T10:39:16.319634+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:05.869985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.329702+0000","last_change":"2023-10-06T10:39:16.323513+0000","last_active":"2023-10-06T10:39:16.329702+0000","last_peered":"2023-10-06T10:39:16.329702+0000","last_clean":"2023-10-06T10:39:16.329702+0000","last_became_active":"2023-10-06T10:39:16.322879+0000","last_became_peered":"2023-10-06T10:39:16.322879+0000","last_unstale":"2023-10-06T10:39:16.329702+0000","last_undegraded":"2023-10-06T10:39:16.329702+0000","last_fullsized":"2023-10-06T10:39:16.329702+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:36:44.908262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.317494+0000","last_change":"2023-10-06T10:39:16.312470+0000","last_active":"2023-10-06T10:39:16.317494+0000","last_peered":"2023-10-06T10:39:16.317494+0000","last_clean":"2023-10-06T10:39:16.317494+0000","last_became_active":"2023-10-06T10:39:16.312215+0000","last_became_peered":"2023-10-06T10:39:16.312215+0000","last_unstale":"2023-10-06T10:39:16.317494+0000","last_undegraded":"2023-10-06T10:39:16.317494+0000","last_fullsized":"2023-10-06T10:39:16.317494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:27:28.759803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.317493+0000","last_change":"2023-10-06T10:39:16.310935+0000","last_active":"2023-10-06T10:39:16.317493+0000","last_peered":"2023-10-06T10:39:16.317493+0000","last_clean":"2023-10-06T10:39:16.317493+0000","last_became_active":"2023-10-06T10:39:16.310671+0000","last_became_peered":"2023-10-06T10:39:16.310671+0000","last_unstale":"2023-10-06T10:39:16.317493+0000","last_undegraded":"2023-10-06T10:39:16.317493+0000","last_fullsized":"2023-10-06T10:39:16.317493+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:16:40.720156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.323817+0000","last_change":"2023-10-06T10:39:16.321749+0000","last_active":"2023-10-06T10:39:16.323817+0000","last_peered":"2023-10-06T10:39:16.323817+0000","last_clean":"2023-10-06T10:39:16.323817+0000","last_became_active":"2023-10-06T10:39:16.321421+0000","last_became_peered":"2023-10-06T10:39:16.321421+0000","last_unstale":"2023-10-06T10:39:16.323817+0000","last_undegraded":"2023-10-06T10:39:16.323817+0000","last_fullsized":"2023-10-06T10:39:16.323817+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:11:40.614228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.319527+0000","last_change":"2023-10-06T10:39:16.315307+0000","last_active":"2023-10-06T10:39:16.319527+0000","last_peered":"2023-10-06T10:39:16.319527+0000","last_clean":"2023-10-06T10:39:16.319527+0000","last_became_active":"2023-10-06T10:39:16.315075+0000","last_became_peered":"2023-10-06T10:39:16.315075+0000","last_unstale":"2023-10-06T10:39:16.319527+0000","last_undegraded":"2023-10-06T10:39:16.319527+0000","last_fullsized":"2023-10-06T10:39:16.319527+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:18:17.504520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.323277+0000","last_change":"2023-10-06T10:39:16.321160+0000","last_active":"2023-10-06T10:39:16.323277+0000","last_peered":"2023-10-06T10:39:16.323277+0000","last_clean":"2023-10-06T10:39:16.323277+0000","last_became_active":"2023-10-06T10:39:16.320904+0000","last_became_peered":"2023-10-06T10:39:16.320904+0000","last_unstale":"2023-10-06T10:39:16.323277+0000","last_undegraded":"2023-10-06T10:39:16.323277+0000","last_fullsized":"2023-10-06T10:39:16.323277+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:18.818780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.324013+0000","last_change":"2023-10-06T10:39:16.316387+0000","last_active":"2023-10-06T10:39:16.324013+0000","last_peered":"2023-10-06T10:39:16.324013+0000","last_clean":"2023-10-06T10:39:16.324013+0000","last_became_active":"2023-10-06T10:39:16.316103+0000","last_became_peered":"2023-10-06T10:39:16.316103+0000","last_unstale":"2023-10-06T10:39:16.324013+0000","last_undegraded":"2023-10-06T10:39:16.324013+0000","last_fullsized":"2023-10-06T10:39:16.324013+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:04.259427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.321282+0000","last_change":"2023-10-06T10:39:16.312435+0000","last_active":"2023-10-06T10:39:16.321282+0000","last_peered":"2023-10-06T10:39:16.321282+0000","last_clean":"2023-10-06T10:39:16.321282+0000","last_became_active":"2023-10-06T10:39:16.311654+0000","last_became_peered":"2023-10-06T10:39:16.311654+0000","last_unstale":"2023-10-06T10:39:16.321282+0000","last_undegraded":"2023-10-06T10:39:16.321282+0000","last_fullsized":"2023-10-06T10:39:16.321282+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:09.527903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.321283+0000","last_change":"2023-10-06T10:39:16.315547+0000","last_active":"2023-10-06T10:39:16.321283+0000","last_peered":"2023-10-06T10:39:16.321283+0000","last_clean":"2023-10-06T10:39:16.321283+0000","last_became_active":"2023-10-06T10:39:16.315293+0000","last_became_peered":"2023-10-06T10:39:16.315293+0000","last_unstale":"2023-10-06T10:39:16.321283+0000","last_undegraded":"2023-10-06T10:39:16.321283+0000","last_fullsized":"2023-10-06T10:39:16.321283+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:14:44.803599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.325843+0000","last_change":"2023-10-06T10:39:16.320957+0000","last_active":"2023-10-06T10:39:16.325843+0000","last_peered":"2023-10-06T10:39:16.325843+0000","last_clean":"2023-10-06T10:39:16.325843+0000","last_became_active":"2023-10-06T10:39:16.320691+0000","last_became_peered":"2023-10-06T10:39:16.320691+0000","last_unstale":"2023-10-06T10:39:16.325843+0000","last_undegraded":"2023-10-06T10:39:16.325843+0000","last_fullsized":"2023-10-06T10:39:16.325843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:47.503939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.320814+0000","last_change":"2023-10-06T10:39:16.314690+0000","last_active":"2023-10-06T10:39:16.320814+0000","last_peered":"2023-10-06T10:39:16.320814+0000","last_clean":"2023-10-06T10:39:16.320814+0000","last_became_active":"2023-10-06T10:39:16.314453+0000","last_became_peered":"2023-10-06T10:39:16.314453+0000","last_unstale":"2023-10-06T10:39:16.320814+0000","last_undegraded":"2023-10-06T10:39:16.320814+0000","last_fullsized":"2023-10-06T10:39:16.320814+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:55.246713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.318143+0000","last_change":"2023-10-06T10:39:16.316143+0000","last_active":"2023-10-06T10:39:16.318143+0000","last_peered":"2023-10-06T10:39:16.318143+0000","last_clean":"2023-10-06T10:39:16.318143+0000","last_became_active":"2023-10-06T10:39:16.315879+0000","last_became_peered":"2023-10-06T10:39:16.315879+0000","last_unstale":"2023-10-06T10:39:16.318143+0000","last_undegraded":"2023-10-06T10:39:16.318143+0000","last_fullsized":"2023-10-06T10:39:16.318143+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:10.131667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.327978+0000","last_change":"2023-10-06T10:39:16.322605+0000","last_active":"2023-10-06T10:39:16.327978+0000","last_peered":"2023-10-06T10:39:16.327978+0000","last_clean":"2023-10-06T10:39:16.327978+0000","last_became_active":"2023-10-06T10:39:16.322107+0000","last_became_peered":"2023-10-06T10:39:16.322107+0000","last_unstale":"2023-10-06T10:39:16.327978+0000","last_undegraded":"2023-10-06T10:39:16.327978+0000","last_fullsized":"2023-10-06T10:39:16.327978+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:35:29.567370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.325551+0000","last_change":"2023-10-06T10:39:16.320784+0000","last_active":"2023-10-06T10:39:16.325551+0000","last_peered":"2023-10-06T10:39:16.325551+0000","last_clean":"2023-10-06T10:39:16.325551+0000","last_became_active":"2023-10-06T10:39:16.320514+0000","last_became_peered":"2023-10-06T10:39:16.320514+0000","last_unstale":"2023-10-06T10:39:16.325551+0000","last_undegraded":"2023-10-06T10:39:16.325551+0000","last_fullsized":"2023-10-06T10:39:16.325551+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:55.246713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.325569+0000","last_change":"2023-10-06T10:39:16.320837+0000","last_active":"2023-10-06T10:39:16.325569+0000","last_peered":"2023-10-06T10:39:16.325569+0000","last_clean":"2023-10-06T10:39:16.325569+0000","last_became_active":"2023-10-06T10:39:16.320581+0000","last_became_peered":"2023-10-06T10:39:16.320581+0000","last_unstale":"2023-10-06T10:39:16.325569+0000","last_undegraded":"2023-10-06T10:39:16.325569+0000","last_fullsized":"2023-10-06T10:39:16.325569+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:37:59.945308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.325588+0000","last_change":"2023-10-06T10:39:16.320249+0000","last_active":"2023-10-06T10:39:16.325588+0000","last_peered":"2023-10-06T10:39:16.325588+0000","last_clean":"2023-10-06T10:39:16.325588+0000","last_became_active":"2023-10-06T10:39:16.320049+0000","last_became_peered":"2023-10-06T10:39:16.320049+0000","last_unstale":"2023-10-06T10:39:16.325588+0000","last_undegraded":"2023-10-06T10:39:16.325588+0000","last_fullsized":"2023-10-06T10:39:16.325588+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:14:53.280400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.323149+0000","last_change":"2023-10-06T10:39:16.319646+0000","last_active":"2023-10-06T10:39:16.323149+0000","last_peered":"2023-10-06T10:39:16.323149+0000","last_clean":"2023-10-06T10:39:16.323149+0000","last_became_active":"2023-10-06T10:39:16.319358+0000","last_became_peered":"2023-10-06T10:39:16.319358+0000","last_unstale":"2023-10-06T10:39:16.323149+0000","last_undegraded":"2023-10-06T10:39:16.323149+0000","last_fullsized":"2023-10-06T10:39:16.323149+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:47.503939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.321124+0000","last_change":"2023-10-06T10:39:16.309618+0000","last_active":"2023-10-06T10:39:16.321124+0000","last_peered":"2023-10-06T10:39:16.321124+0000","last_clean":"2023-10-06T10:39:16.321124+0000","last_became_active":"2023-10-06T10:39:16.309376+0000","last_became_peered":"2023-10-06T10:39:16.309376+0000","last_unstale":"2023-10-06T10:39:16.321124+0000","last_undegraded":"2023-10-06T10:39:16.321124+0000","last_fullsized":"2023-10-06T10:39:16.321124+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:09.420170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.322329+0000","last_change":"2023-10-06T10:39:16.314282+0000","last_active":"2023-10-06T10:39:16.322329+0000","last_peered":"2023-10-06T10:39:16.322329+0000","last_clean":"2023-10-06T10:39:16.322329+0000","last_became_active":"2023-10-06T10:39:16.314001+0000","last_became_peered":"2023-10-06T10:39:16.314001+0000","last_unstale":"2023-10-06T10:39:16.322329+0000","last_undegraded":"2023-10-06T10:39:16.322329+0000","last_fullsized":"2023-10-06T10:39:16.322329+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:27.557844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.325472+0000","last_change":"2023-10-06T10:39:16.320179+0000","last_active":"2023-10-06T10:39:16.325472+0000","last_peered":"2023-10-06T10:39:16.325472+0000","last_clean":"2023-10-06T10:39:16.325472+0000","last_became_active":"2023-10-06T10:39:16.319907+0000","last_became_peered":"2023-10-06T10:39:16.319907+0000","last_unstale":"2023-10-06T10:39:16.325472+0000","last_undegraded":"2023-10-06T10:39:16.325472+0000","last_fullsized":"2023-10-06T10:39:16.325472+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:15.030656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.321317+0000","last_change":"2023-10-06T10:39:16.312104+0000","last_active":"2023-10-06T10:39:16.321317+0000","last_peered":"2023-10-06T10:39:16.321317+0000","last_clean":"2023-10-06T10:39:16.321317+0000","last_became_active":"2023-10-06T10:39:16.311823+0000","last_became_peered":"2023-10-06T10:39:16.311823+0000","last_unstale":"2023-10-06T10:39:16.321317+0000","last_undegraded":"2023-10-06T10:39:16.321317+0000","last_fullsized":"2023-10-06T10:39:16.321317+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:36:44.908262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.318076+0000","last_change":"2023-10-06T10:39:16.313390+0000","last_active":"2023-10-06T10:39:16.318076+0000","last_peered":"2023-10-06T10:39:16.318076+0000","last_clean":"2023-10-06T10:39:16.318076+0000","last_became_active":"2023-10-06T10:39:16.313129+0000","last_became_peered":"2023-10-06T10:39:16.313129+0000","last_unstale":"2023-10-06T10:39:16.318076+0000","last_undegraded":"2023-10-06T10:39:16.318076+0000","last_fullsized":"2023-10-06T10:39:16.318076+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:46:44.354855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.318942+0000","last_change":"2023-10-06T10:39:16.313745+0000","last_active":"2023-10-06T10:39:16.318942+0000","last_peered":"2023-10-06T10:39:16.318942+0000","last_clean":"2023-10-06T10:39:16.318942+0000","last_became_active":"2023-10-06T10:39:16.313461+0000","last_became_peered":"2023-10-06T10:39:16.313461+0000","last_unstale":"2023-10-06T10:39:16.318942+0000","last_undegraded":"2023-10-06T10:39:16.318942+0000","last_fullsized":"2023-10-06T10:39:16.318942+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:13.217482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.318388+0000","last_change":"2023-10-06T10:39:16.313811+0000","last_active":"2023-10-06T10:39:16.318388+0000","last_peered":"2023-10-06T10:39:16.318388+0000","last_clean":"2023-10-06T10:39:16.318388+0000","last_became_active":"2023-10-06T10:39:16.313573+0000","last_became_peered":"2023-10-06T10:39:16.313573+0000","last_unstale":"2023-10-06T10:39:16.318388+0000","last_undegraded":"2023-10-06T10:39:16.318388+0000","last_fullsized":"2023-10-06T10:39:16.318388+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:24.577057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"37'4","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.331021+0000","last_change":"2023-10-06T10:39:15.303875+0000","last_active":"2023-10-06T10:39:16.331021+0000","last_peered":"2023-10-06T10:39:16.331021+0000","last_clean":"2023-10-06T10:39:16.331021+0000","last_became_active":"2023-10-06T10:39:15.303626+0000","last_became_peered":"2023-10-06T10:39:15.303626+0000","last_unstale":"2023-10-06T10:39:16.331021+0000","last_undegraded":"2023-10-06T10:39:16.331021+0000","last_fullsized":"2023-10-06T10:39:16.331021+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:48.263358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.325782+0000","last_change":"2023-10-06T10:39:16.323413+0000","last_active":"2023-10-06T10:39:16.325782+0000","last_peered":"2023-10-06T10:39:16.325782+0000","last_clean":"2023-10-06T10:39:16.325782+0000","last_became_active":"2023-10-06T10:39:16.323094+0000","last_became_peered":"2023-10-06T10:39:16.323094+0000","last_unstale":"2023-10-06T10:39:16.325782+0000","last_undegraded":"2023-10-06T10:39:16.325782+0000","last_fullsized":"2023-10-06T10:39:16.325782+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:24.577057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.332974+0000","last_change":"2023-10-06T10:39:15.310615+0000","last_active":"2023-10-06T10:39:16.332974+0000","last_peered":"2023-10-06T10:39:16.332974+0000","last_clean":"2023-10-06T10:39:16.332974+0000","last_became_active":"2023-10-06T10:39:15.310305+0000","last_became_peered":"2023-10-06T10:39:15.310305+0000","last_unstale":"2023-10-06T10:39:16.332974+0000","last_undegraded":"2023-10-06T10:39:16.332974+0000","last_fullsized":"2023-10-06T10:39:16.332974+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:16:37.935892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.323043+0000","last_change":"2023-10-06T10:39:16.316459+0000","last_active":"2023-10-06T10:39:16.323043+0000","last_peered":"2023-10-06T10:39:16.323043+0000","last_clean":"2023-10-06T10:39:16.323043+0000","last_became_active":"2023-10-06T10:39:16.316232+0000","last_became_peered":"2023-10-06T10:39:16.316232+0000","last_unstale":"2023-10-06T10:39:16.323043+0000","last_undegraded":"2023-10-06T10:39:16.323043+0000","last_fullsized":"2023-10-06T10:39:16.323043+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:05.869985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.330501+0000","last_change":"2023-10-06T10:39:15.299806+0000","last_active":"2023-10-06T10:39:16.330501+0000","last_peered":"2023-10-06T10:39:16.330501+0000","last_clean":"2023-10-06T10:39:16.330501+0000","last_became_active":"2023-10-06T10:39:15.299579+0000","last_became_peered":"2023-10-06T10:39:15.299579+0000","last_unstale":"2023-10-06T10:39:16.330501+0000","last_undegraded":"2023-10-06T10:39:16.330501+0000","last_fullsized":"2023-10-06T10:39:16.330501+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:14.029154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.319861+0000","last_change":"2023-10-06T10:39:16.318730+0000","last_active":"2023-10-06T10:39:16.319861+0000","last_peered":"2023-10-06T10:39:16.319861+0000","last_clean":"2023-10-06T10:39:16.319861+0000","last_became_active":"2023-10-06T10:39:16.318513+0000","last_became_peered":"2023-10-06T10:39:16.318513+0000","last_unstale":"2023-10-06T10:39:16.319861+0000","last_undegraded":"2023-10-06T10:39:16.319861+0000","last_fullsized":"2023-10-06T10:39:16.319861+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:04.259427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.330978+0000","last_change":"2023-10-06T10:39:15.307706+0000","last_active":"2023-10-06T10:39:16.330978+0000","last_peered":"2023-10-06T10:39:16.330978+0000","last_clean":"2023-10-06T10:39:16.330978+0000","last_became_active":"2023-10-06T10:39:15.307425+0000","last_became_peered":"2023-10-06T10:39:15.307425+0000","last_unstale":"2023-10-06T10:39:16.330978+0000","last_undegraded":"2023-10-06T10:39:16.330978+0000","last_fullsized":"2023-10-06T10:39:16.330978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:46.502437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.321328+0000","last_change":"2023-10-06T10:39:16.317415+0000","last_active":"2023-10-06T10:39:16.321328+0000","last_peered":"2023-10-06T10:39:16.321328+0000","last_clean":"2023-10-06T10:39:16.321328+0000","last_became_active":"2023-10-06T10:39:16.317178+0000","last_became_peered":"2023-10-06T10:39:16.317178+0000","last_unstale":"2023-10-06T10:39:16.321328+0000","last_undegraded":"2023-10-06T10:39:16.321328+0000","last_fullsized":"2023-10-06T10:39:16.321328+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:18.818780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.331333+0000","last_change":"2023-10-06T10:39:15.305959+0000","last_active":"2023-10-06T10:39:16.331333+0000","last_peered":"2023-10-06T10:39:16.331333+0000","last_clean":"2023-10-06T10:39:16.331333+0000","last_became_active":"2023-10-06T10:39:15.305700+0000","last_became_peered":"2023-10-06T10:39:15.305700+0000","last_unstale":"2023-10-06T10:39:16.331333+0000","last_undegraded":"2023-10-06T10:39:16.331333+0000","last_fullsized":"2023-10-06T10:39:16.331333+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:43:46.247836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.319525+0000","last_change":"2023-10-06T10:39:16.313841+0000","last_active":"2023-10-06T10:39:16.319525+0000","last_peered":"2023-10-06T10:39:16.319525+0000","last_clean":"2023-10-06T10:39:16.319525+0000","last_became_active":"2023-10-06T10:39:16.313587+0000","last_became_peered":"2023-10-06T10:39:16.313587+0000","last_unstale":"2023-10-06T10:39:16.319525+0000","last_undegraded":"2023-10-06T10:39:16.319525+0000","last_fullsized":"2023-10-06T10:39:16.319525+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:37.849449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.330534+0000","last_change":"2023-10-06T10:39:15.304964+0000","last_active":"2023-10-06T10:39:16.330534+0000","last_peered":"2023-10-06T10:39:16.330534+0000","last_clean":"2023-10-06T10:39:16.330534+0000","last_became_active":"2023-10-06T10:39:15.304284+0000","last_became_peered":"2023-10-06T10:39:15.304284+0000","last_unstale":"2023-10-06T10:39:16.330534+0000","last_undegraded":"2023-10-06T10:39:16.330534+0000","last_fullsized":"2023-10-06T10:39:16.330534+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:43:46.247836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.319776+0000","last_change":"2023-10-06T10:39:16.317078+0000","last_active":"2023-10-06T10:39:16.319776+0000","last_peered":"2023-10-06T10:39:16.319776+0000","last_clean":"2023-10-06T10:39:16.319776+0000","last_became_active":"2023-10-06T10:39:16.316806+0000","last_became_peered":"2023-10-06T10:39:16.316806+0000","last_unstale":"2023-10-06T10:39:16.319776+0000","last_undegraded":"2023-10-06T10:39:16.319776+0000","last_fullsized":"2023-10-06T10:39:16.319776+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:39.271089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.304432+0000","last_change":"2023-10-06T10:39:15.296453+0000","last_active":"2023-10-06T10:39:16.304432+0000","last_peered":"2023-10-06T10:39:16.304432+0000","last_clean":"2023-10-06T10:39:16.304432+0000","last_became_active":"2023-10-06T10:39:15.296109+0000","last_became_peered":"2023-10-06T10:39:15.296109+0000","last_unstale":"2023-10-06T10:39:16.304432+0000","last_undegraded":"2023-10-06T10:39:16.304432+0000","last_fullsized":"2023-10-06T10:39:16.304432+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:00.467864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.325794+0000","last_change":"2023-10-06T10:39:16.320973+0000","last_active":"2023-10-06T10:39:16.325794+0000","last_peered":"2023-10-06T10:39:16.325794+0000","last_clean":"2023-10-06T10:39:16.325794+0000","last_became_active":"2023-10-06T10:39:16.320156+0000","last_became_peered":"2023-10-06T10:39:16.320156+0000","last_unstale":"2023-10-06T10:39:16.325794+0000","last_undegraded":"2023-10-06T10:39:16.325794+0000","last_fullsized":"2023-10-06T10:39:16.325794+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:22.495874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.305343+0000","last_change":"2023-10-06T10:39:15.302908+0000","last_active":"2023-10-06T10:39:16.305343+0000","last_peered":"2023-10-06T10:39:16.305343+0000","last_clean":"2023-10-06T10:39:16.305343+0000","last_became_active":"2023-10-06T10:39:15.302669+0000","last_became_peered":"2023-10-06T10:39:15.302669+0000","last_unstale":"2023-10-06T10:39:16.305343+0000","last_undegraded":"2023-10-06T10:39:16.305343+0000","last_fullsized":"2023-10-06T10:39:16.305343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:21.494371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.321309+0000","last_change":"2023-10-06T10:39:16.315648+0000","last_active":"2023-10-06T10:39:16.321309+0000","last_peered":"2023-10-06T10:39:16.321309+0000","last_clean":"2023-10-06T10:39:16.321309+0000","last_became_active":"2023-10-06T10:39:16.315340+0000","last_became_peered":"2023-10-06T10:39:16.315340+0000","last_unstale":"2023-10-06T10:39:16.321309+0000","last_undegraded":"2023-10-06T10:39:16.321309+0000","last_fullsized":"2023-10-06T10:39:16.321309+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:37.202391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.305278+0000","last_change":"2023-10-06T10:39:15.304944+0000","last_active":"2023-10-06T10:39:16.305278+0000","last_peered":"2023-10-06T10:39:16.305278+0000","last_clean":"2023-10-06T10:39:16.305278+0000","last_became_active":"2023-10-06T10:39:15.304656+0000","last_became_peered":"2023-10-06T10:39:15.304656+0000","last_unstale":"2023-10-06T10:39:16.305278+0000","last_undegraded":"2023-10-06T10:39:16.305278+0000","last_fullsized":"2023-10-06T10:39:16.305278+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:33.419655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.323326+0000","last_change":"2023-10-06T10:39:16.318161+0000","last_active":"2023-10-06T10:39:16.323326+0000","last_peered":"2023-10-06T10:39:16.323326+0000","last_clean":"2023-10-06T10:39:16.323326+0000","last_became_active":"2023-10-06T10:39:16.317892+0000","last_became_peered":"2023-10-06T10:39:16.317892+0000","last_unstale":"2023-10-06T10:39:16.323326+0000","last_undegraded":"2023-10-06T10:39:16.323326+0000","last_fullsized":"2023-10-06T10:39:16.323326+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:43:47.249338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.309114+0000","last_change":"2023-10-06T10:39:15.305389+0000","last_active":"2023-10-06T10:39:16.309114+0000","last_peered":"2023-10-06T10:39:16.309114+0000","last_clean":"2023-10-06T10:39:16.309114+0000","last_became_active":"2023-10-06T10:39:15.305101+0000","last_became_peered":"2023-10-06T10:39:15.305101+0000","last_unstale":"2023-10-06T10:39:16.309114+0000","last_undegraded":"2023-10-06T10:39:16.309114+0000","last_fullsized":"2023-10-06T10:39:16.309114+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:14.029154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.318167+0000","last_change":"2023-10-06T10:39:16.314462+0000","last_active":"2023-10-06T10:39:16.318167+0000","last_peered":"2023-10-06T10:39:16.318167+0000","last_clean":"2023-10-06T10:39:16.318167+0000","last_became_active":"2023-10-06T10:39:16.314119+0000","last_became_peered":"2023-10-06T10:39:16.314119+0000","last_unstale":"2023-10-06T10:39:16.318167+0000","last_undegraded":"2023-10-06T10:39:16.318167+0000","last_fullsized":"2023-10-06T10:39:16.318167+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:37.593925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"37'3","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.330885+0000","last_change":"2023-10-06T10:39:15.310013+0000","last_active":"2023-10-06T10:39:16.330885+0000","last_peered":"2023-10-06T10:39:16.330885+0000","last_clean":"2023-10-06T10:39:16.330885+0000","last_became_active":"2023-10-06T10:39:15.309671+0000","last_became_peered":"2023-10-06T10:39:15.309671+0000","last_unstale":"2023-10-06T10:39:16.330885+0000","last_undegraded":"2023-10-06T10:39:16.330885+0000","last_fullsized":"2023-10-06T10:39:16.330885+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:23.575555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.322586+0000","last_change":"2023-10-06T10:39:16.312815+0000","last_active":"2023-10-06T10:39:16.322586+0000","last_peered":"2023-10-06T10:39:16.322586+0000","last_clean":"2023-10-06T10:39:16.322586+0000","last_became_active":"2023-10-06T10:39:16.312579+0000","last_became_peered":"2023-10-06T10:39:16.312579+0000","last_unstale":"2023-10-06T10:39:16.322586+0000","last_undegraded":"2023-10-06T10:39:16.322586+0000","last_fullsized":"2023-10-06T10:39:16.322586+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:34.421157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"37'6","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.335534+0000","last_change":"2023-10-06T10:39:15.303225+0000","last_active":"2023-10-06T10:39:16.335534+0000","last_peered":"2023-10-06T10:39:16.335534+0000","last_clean":"2023-10-06T10:39:16.335534+0000","last_became_active":"2023-10-06T10:39:15.302931+0000","last_became_peered":"2023-10-06T10:39:15.302931+0000","last_unstale":"2023-10-06T10:39:16.335534+0000","last_undegraded":"2023-10-06T10:39:16.335534+0000","last_fullsized":"2023-10-06T10:39:16.335534+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:45:40.374013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.325891+0000","last_change":"2023-10-06T10:39:16.318415+0000","last_active":"2023-10-06T10:39:16.325891+0000","last_peered":"2023-10-06T10:39:16.325891+0000","last_clean":"2023-10-06T10:39:16.325891+0000","last_became_active":"2023-10-06T10:39:16.318163+0000","last_became_peered":"2023-10-06T10:39:16.318163+0000","last_unstale":"2023-10-06T10:39:16.325891+0000","last_undegraded":"2023-10-06T10:39:16.325891+0000","last_fullsized":"2023-10-06T10:39:16.325891+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:11:40.614228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.335426+0000","last_change":"2023-10-06T10:39:15.307620+0000","last_active":"2023-10-06T10:39:16.335426+0000","last_peered":"2023-10-06T10:39:16.335426+0000","last_clean":"2023-10-06T10:39:16.335426+0000","last_became_active":"2023-10-06T10:39:15.307406+0000","last_became_peered":"2023-10-06T10:39:15.307406+0000","last_unstale":"2023-10-06T10:39:16.335426+0000","last_undegraded":"2023-10-06T10:39:16.335426+0000","last_fullsized":"2023-10-06T10:39:16.335426+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:18:16.503018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.319543+0000","last_change":"2023-10-06T10:39:16.314298+0000","last_active":"2023-10-06T10:39:16.319543+0000","last_peered":"2023-10-06T10:39:16.319543+0000","last_clean":"2023-10-06T10:39:16.319543+0000","last_became_active":"2023-10-06T10:39:16.314055+0000","last_became_peered":"2023-10-06T10:39:16.314055+0000","last_unstale":"2023-10-06T10:39:16.319543+0000","last_undegraded":"2023-10-06T10:39:16.319543+0000","last_fullsized":"2023-10-06T10:39:16.319543+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:01:06.914043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.311319+0000","last_change":"2023-10-06T10:39:15.305805+0000","last_active":"2023-10-06T10:39:16.311319+0000","last_peered":"2023-10-06T10:39:16.311319+0000","last_clean":"2023-10-06T10:39:16.311319+0000","last_became_active":"2023-10-06T10:39:15.305554+0000","last_became_peered":"2023-10-06T10:39:15.305554+0000","last_unstale":"2023-10-06T10:39:16.311319+0000","last_undegraded":"2023-10-06T10:39:16.311319+0000","last_fullsized":"2023-10-06T10:39:16.311319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:03.257925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.317529+0000","last_change":"2023-10-06T10:39:16.312332+0000","last_active":"2023-10-06T10:39:16.317529+0000","last_peered":"2023-10-06T10:39:16.317529+0000","last_clean":"2023-10-06T10:39:16.317529+0000","last_became_active":"2023-10-06T10:39:16.312115+0000","last_became_peered":"2023-10-06T10:39:16.312115+0000","last_unstale":"2023-10-06T10:39:16.317529+0000","last_undegraded":"2023-10-06T10:39:16.317529+0000","last_fullsized":"2023-10-06T10:39:16.317529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:39.271089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.305493+0000","last_change":"2023-10-06T10:39:15.306640+0000","last_active":"2023-10-06T10:39:16.305493+0000","last_peered":"2023-10-06T10:39:16.305493+0000","last_clean":"2023-10-06T10:39:16.305493+0000","last_became_active":"2023-10-06T10:39:15.306414+0000","last_became_peered":"2023-10-06T10:39:15.306414+0000","last_unstale":"2023-10-06T10:39:16.305493+0000","last_undegraded":"2023-10-06T10:39:16.305493+0000","last_fullsized":"2023-10-06T10:39:16.305493+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:35:12.123388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.323403+0000","last_change":"2023-10-06T10:39:16.321137+0000","last_active":"2023-10-06T10:39:16.323403+0000","last_peered":"2023-10-06T10:39:16.323403+0000","last_clean":"2023-10-06T10:39:16.323403+0000","last_became_active":"2023-10-06T10:39:16.320781+0000","last_became_peered":"2023-10-06T10:39:16.320781+0000","last_unstale":"2023-10-06T10:39:16.323403+0000","last_undegraded":"2023-10-06T10:39:16.323403+0000","last_fullsized":"2023-10-06T10:39:16.323403+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:27.557844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"37'8","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.335484+0000","last_change":"2023-10-06T10:39:15.302166+0000","last_active":"2023-10-06T10:39:16.335484+0000","last_peered":"2023-10-06T10:39:16.335484+0000","last_clean":"2023-10-06T10:39:16.335484+0000","last_became_active":"2023-10-06T10:39:15.301014+0000","last_became_peered":"2023-10-06T10:39:15.301014+0000","last_unstale":"2023-10-06T10:39:16.335484+0000","last_undegraded":"2023-10-06T10:39:16.335484+0000","last_fullsized":"2023-10-06T10:39:16.335484+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:12.215980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.310559+0000","last_change":"2023-10-06T10:39:16.308161+0000","last_active":"2023-10-06T10:39:16.310559+0000","last_peered":"2023-10-06T10:39:16.310559+0000","last_clean":"2023-10-06T10:39:16.310559+0000","last_became_active":"2023-10-06T10:39:16.307897+0000","last_became_peered":"2023-10-06T10:39:16.307897+0000","last_unstale":"2023-10-06T10:39:16.310559+0000","last_undegraded":"2023-10-06T10:39:16.310559+0000","last_fullsized":"2023-10-06T10:39:16.310559+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:37:59.945308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.327092+0000","last_change":"2023-10-06T10:39:15.304372+0000","last_active":"2023-10-06T10:39:16.327092+0000","last_peered":"2023-10-06T10:39:16.327092+0000","last_clean":"2023-10-06T10:39:16.327092+0000","last_became_active":"2023-10-06T10:39:15.304115+0000","last_became_peered":"2023-10-06T10:39:15.304115+0000","last_unstale":"2023-10-06T10:39:16.327092+0000","last_undegraded":"2023-10-06T10:39:16.327092+0000","last_fullsized":"2023-10-06T10:39:16.327092+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:39.434527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.322741+0000","last_change":"2023-10-06T10:39:16.320403+0000","last_active":"2023-10-06T10:39:16.322741+0000","last_peered":"2023-10-06T10:39:16.322741+0000","last_clean":"2023-10-06T10:39:16.322741+0000","last_became_active":"2023-10-06T10:39:16.319843+0000","last_became_peered":"2023-10-06T10:39:16.319843+0000","last_unstale":"2023-10-06T10:39:16.322741+0000","last_undegraded":"2023-10-06T10:39:16.322741+0000","last_fullsized":"2023-10-06T10:39:16.322741+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:37:00.510491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.333067+0000","last_change":"2023-10-06T10:39:15.307246+0000","last_active":"2023-10-06T10:39:16.333067+0000","last_peered":"2023-10-06T10:39:16.333067+0000","last_clean":"2023-10-06T10:39:16.333067+0000","last_became_active":"2023-10-06T10:39:15.306977+0000","last_became_peered":"2023-10-06T10:39:15.306977+0000","last_unstale":"2023-10-06T10:39:16.333067+0000","last_undegraded":"2023-10-06T10:39:16.333067+0000","last_fullsized":"2023-10-06T10:39:16.333067+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:18:16.503018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.324074+0000","last_change":"2023-10-06T10:39:16.316484+0000","last_active":"2023-10-06T10:39:16.324074+0000","last_peered":"2023-10-06T10:39:16.324074+0000","last_clean":"2023-10-06T10:39:16.324074+0000","last_became_active":"2023-10-06T10:39:16.316256+0000","last_became_peered":"2023-10-06T10:39:16.316256+0000","last_unstale":"2023-10-06T10:39:16.324074+0000","last_undegraded":"2023-10-06T10:39:16.324074+0000","last_fullsized":"2023-10-06T10:39:16.324074+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:13.217482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.330336+0000","last_change":"2023-10-06T10:39:15.295445+0000","last_active":"2023-10-06T10:39:16.330336+0000","last_peered":"2023-10-06T10:39:16.330336+0000","last_clean":"2023-10-06T10:39:16.330336+0000","last_became_active":"2023-10-06T10:39:15.295187+0000","last_became_peered":"2023-10-06T10:39:15.295187+0000","last_unstale":"2023-10-06T10:39:16.330336+0000","last_undegraded":"2023-10-06T10:39:16.330336+0000","last_fullsized":"2023-10-06T10:39:16.330336+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:01:08.443520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.323905+0000","last_change":"2023-10-06T10:39:16.313889+0000","last_active":"2023-10-06T10:39:16.323905+0000","last_peered":"2023-10-06T10:39:16.323905+0000","last_clean":"2023-10-06T10:39:16.323905+0000","last_became_active":"2023-10-06T10:39:16.313650+0000","last_became_peered":"2023-10-06T10:39:16.313650+0000","last_unstale":"2023-10-06T10:39:16.323905+0000","last_undegraded":"2023-10-06T10:39:16.323905+0000","last_fullsized":"2023-10-06T10:39:16.323905+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:10.202434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.305959+0000","last_change":"2023-10-06T10:39:15.305692+0000","last_active":"2023-10-06T10:39:16.305959+0000","last_peered":"2023-10-06T10:39:16.305959+0000","last_clean":"2023-10-06T10:39:16.305959+0000","last_became_active":"2023-10-06T10:39:15.305442+0000","last_became_peered":"2023-10-06T10:39:15.305442+0000","last_unstale":"2023-10-06T10:39:16.305959+0000","last_undegraded":"2023-10-06T10:39:16.305959+0000","last_fullsized":"2023-10-06T10:39:16.305959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:53.257606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.318038+0000","last_change":"2023-10-06T10:39:16.313925+0000","last_active":"2023-10-06T10:39:16.318038+0000","last_peered":"2023-10-06T10:39:16.318038+0000","last_clean":"2023-10-06T10:39:16.318038+0000","last_became_active":"2023-10-06T10:39:16.313679+0000","last_became_peered":"2023-10-06T10:39:16.313679+0000","last_unstale":"2023-10-06T10:39:16.318038+0000","last_undegraded":"2023-10-06T10:39:16.318038+0000","last_fullsized":"2023-10-06T10:39:16.318038+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:09.527903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.331399+0000","last_change":"2023-10-06T10:39:15.305483+0000","last_active":"2023-10-06T10:39:16.331399+0000","last_peered":"2023-10-06T10:39:16.331399+0000","last_clean":"2023-10-06T10:39:16.331399+0000","last_became_active":"2023-10-06T10:39:15.305222+0000","last_became_peered":"2023-10-06T10:39:15.305222+0000","last_unstale":"2023-10-06T10:39:16.331399+0000","last_undegraded":"2023-10-06T10:39:16.331399+0000","last_fullsized":"2023-10-06T10:39:16.331399+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:16:37.935892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.318312+0000","last_change":"2023-10-06T10:39:16.313915+0000","last_active":"2023-10-06T10:39:16.318312+0000","last_peered":"2023-10-06T10:39:16.318312+0000","last_clean":"2023-10-06T10:39:16.318312+0000","last_became_active":"2023-10-06T10:39:16.313678+0000","last_became_peered":"2023-10-06T10:39:16.313678+0000","last_unstale":"2023-10-06T10:39:16.318312+0000","last_undegraded":"2023-10-06T10:39:16.318312+0000","last_fullsized":"2023-10-06T10:39:16.318312+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:11:40.614228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.304606+0000","last_change":"2023-10-06T10:39:15.303138+0000","last_active":"2023-10-06T10:39:16.304606+0000","last_peered":"2023-10-06T10:39:16.304606+0000","last_clean":"2023-10-06T10:39:16.304606+0000","last_became_active":"2023-10-06T10:39:15.302847+0000","last_became_peered":"2023-10-06T10:39:15.302847+0000","last_unstale":"2023-10-06T10:39:16.304606+0000","last_undegraded":"2023-10-06T10:39:16.304606+0000","last_fullsized":"2023-10-06T10:39:16.304606+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:53.257606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.310661+0000","last_change":"2023-10-06T10:39:16.308674+0000","last_active":"2023-10-06T10:39:16.310661+0000","last_peered":"2023-10-06T10:39:16.310661+0000","last_clean":"2023-10-06T10:39:16.310661+0000","last_became_active":"2023-10-06T10:39:16.308434+0000","last_became_peered":"2023-10-06T10:39:16.308434+0000","last_unstale":"2023-10-06T10:39:16.310661+0000","last_undegraded":"2023-10-06T10:39:16.310661+0000","last_fullsized":"2023-10-06T10:39:16.310661+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:22.495874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.306585+0000","last_change":"2023-10-06T10:39:15.307533+0000","last_active":"2023-10-06T10:39:16.306585+0000","last_peered":"2023-10-06T10:39:16.306585+0000","last_clean":"2023-10-06T10:39:16.306585+0000","last_became_active":"2023-10-06T10:39:15.307133+0000","last_became_peered":"2023-10-06T10:39:15.307133+0000","last_unstale":"2023-10-06T10:39:16.306585+0000","last_undegraded":"2023-10-06T10:39:16.306585+0000","last_fullsized":"2023-10-06T10:39:16.306585+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:09.200932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.319480+0000","last_change":"2023-10-06T10:39:16.313614+0000","last_active":"2023-10-06T10:39:16.319480+0000","last_peered":"2023-10-06T10:39:16.319480+0000","last_clean":"2023-10-06T10:39:16.319480+0000","last_became_active":"2023-10-06T10:39:16.313354+0000","last_became_peered":"2023-10-06T10:39:16.313354+0000","last_unstale":"2023-10-06T10:39:16.319480+0000","last_undegraded":"2023-10-06T10:39:16.319480+0000","last_fullsized":"2023-10-06T10:39:16.319480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:35:13.124890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.306962+0000","last_change":"2023-10-06T10:39:15.307579+0000","last_active":"2023-10-06T10:39:16.306962+0000","last_peered":"2023-10-06T10:39:16.306962+0000","last_clean":"2023-10-06T10:39:16.306962+0000","last_became_active":"2023-10-06T10:39:15.306595+0000","last_became_peered":"2023-10-06T10:39:15.306595+0000","last_unstale":"2023-10-06T10:39:16.306962+0000","last_undegraded":"2023-10-06T10:39:16.306962+0000","last_fullsized":"2023-10-06T10:39:16.306962+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:53.257606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.317359+0000","last_change":"2023-10-06T10:39:16.312200+0000","last_active":"2023-10-06T10:39:16.317359+0000","last_peered":"2023-10-06T10:39:16.317359+0000","last_clean":"2023-10-06T10:39:16.317359+0000","last_became_active":"2023-10-06T10:39:16.311942+0000","last_became_peered":"2023-10-06T10:39:16.311942+0000","last_unstale":"2023-10-06T10:39:16.317359+0000","last_undegraded":"2023-10-06T10:39:16.317359+0000","last_fullsized":"2023-10-06T10:39:16.317359+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:35:29.567370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.303846+0000","last_change":"2023-10-06T10:39:15.309096+0000","last_active":"2023-10-06T10:39:16.303846+0000","last_peered":"2023-10-06T10:39:16.303846+0000","last_clean":"2023-10-06T10:39:16.303846+0000","last_became_active":"2023-10-06T10:39:15.308781+0000","last_became_peered":"2023-10-06T10:39:15.308781+0000","last_unstale":"2023-10-06T10:39:16.303846+0000","last_undegraded":"2023-10-06T10:39:16.303846+0000","last_fullsized":"2023-10-06T10:39:16.303846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:35:12.123388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.305169+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:39:16.305169+0000","last_peered":"2023-10-06T10:39:16.305169+0000","last_clean":"2023-10-06T10:39:16.305169+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:39:16.305169+0000","last_undegraded":"2023-10-06T10:39:16.305169+0000","last_fullsized":"2023-10-06T10:39:16.305169+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.318171+0000","last_change":"2023-10-06T10:39:16.315971+0000","last_active":"2023-10-06T10:39:16.318171+0000","last_peered":"2023-10-06T10:39:16.318171+0000","last_clean":"2023-10-06T10:39:16.318171+0000","last_became_active":"2023-10-06T10:39:16.315630+0000","last_became_peered":"2023-10-06T10:39:16.315630+0000","last_unstale":"2023-10-06T10:39:16.318171+0000","last_undegraded":"2023-10-06T10:39:16.318171+0000","last_fullsized":"2023-10-06T10:39:16.318171+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:09.420170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.307655+0000","last_change":"2023-10-06T10:39:15.307128+0000","last_active":"2023-10-06T10:39:16.307655+0000","last_peered":"2023-10-06T10:39:16.307655+0000","last_clean":"2023-10-06T10:39:16.307655+0000","last_became_active":"2023-10-06T10:39:15.306872+0000","last_became_peered":"2023-10-06T10:39:15.306872+0000","last_unstale":"2023-10-06T10:39:16.307655+0000","last_undegraded":"2023-10-06T10:39:16.307655+0000","last_fullsized":"2023-10-06T10:39:16.307655+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:21.494371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":26,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.305957+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:39:16.305957+0000","last_peered":"2023-10-06T10:39:16.305957+0000","last_clean":"2023-10-06T10:39:16.305957+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:39:16.305957+0000","last_undegraded":"2023-10-06T10:39:16.305957+0000","last_fullsized":"2023-10-06T10:39:16.305957+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.322680+0000","last_change":"2023-10-06T10:39:16.315838+0000","last_active":"2023-10-06T10:39:16.322680+0000","last_peered":"2023-10-06T10:39:16.322680+0000","last_clean":"2023-10-06T10:39:16.322680+0000","last_became_active":"2023-10-06T10:39:16.314901+0000","last_became_peered":"2023-10-06T10:39:16.314901+0000","last_unstale":"2023-10-06T10:39:16.322680+0000","last_undegraded":"2023-10-06T10:39:16.322680+0000","last_fullsized":"2023-10-06T10:39:16.322680+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:18:17.504520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.304733+0000","last_change":"2023-10-06T10:39:15.299404+0000","last_active":"2023-10-06T10:39:16.304733+0000","last_peered":"2023-10-06T10:39:16.304733+0000","last_clean":"2023-10-06T10:39:16.304733+0000","last_became_active":"2023-10-06T10:39:15.299143+0000","last_became_peered":"2023-10-06T10:39:15.299143+0000","last_unstale":"2023-10-06T10:39:16.304733+0000","last_undegraded":"2023-10-06T10:39:16.304733+0000","last_fullsized":"2023-10-06T10:39:16.304733+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:09.200932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":29,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.305945+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:39:16.305945+0000","last_peered":"2023-10-06T10:39:16.305945+0000","last_clean":"2023-10-06T10:39:16.305945+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:39:16.305945+0000","last_undegraded":"2023-10-06T10:39:16.305945+0000","last_fullsized":"2023-10-06T10:39:16.305945+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":175,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.307079+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:39:16.307079+0000","last_peered":"2023-10-06T10:39:16.307079+0000","last_clean":"2023-10-06T10:39:16.307079+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:39:16.307079+0000","last_undegraded":"2023-10-06T10:39:16.307079+0000","last_fullsized":"2023-10-06T10:39:16.307079+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.329635+0000","last_change":"2023-10-06T10:39:16.323035+0000","last_active":"2023-10-06T10:39:16.329635+0000","last_peered":"2023-10-06T10:39:16.329635+0000","last_clean":"2023-10-06T10:39:16.329635+0000","last_became_active":"2023-10-06T10:39:16.321484+0000","last_became_peered":"2023-10-06T10:39:16.321484+0000","last_unstale":"2023-10-06T10:39:16.329635+0000","last_undegraded":"2023-10-06T10:39:16.329635+0000","last_fullsized":"2023-10-06T10:39:16.329635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:16:40.720156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.310098+0000","last_change":"2023-10-06T10:39:15.310293+0000","last_active":"2023-10-06T10:39:16.310098+0000","last_peered":"2023-10-06T10:39:16.310098+0000","last_clean":"2023-10-06T10:39:16.310098+0000","last_became_active":"2023-10-06T10:39:15.310044+0000","last_became_peered":"2023-10-06T10:39:15.310044+0000","last_unstale":"2023-10-06T10:39:16.310098+0000","last_undegraded":"2023-10-06T10:39:16.310098+0000","last_fullsized":"2023-10-06T10:39:16.310098+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:11:39.612726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":25,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.311402+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:39:16.311402+0000","last_peered":"2023-10-06T10:39:16.311402+0000","last_clean":"2023-10-06T10:39:16.311402+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:39:16.311402+0000","last_undegraded":"2023-10-06T10:39:16.311402+0000","last_fullsized":"2023-10-06T10:39:16.311402+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.323263+0000","last_change":"2023-10-06T10:39:16.320245+0000","last_active":"2023-10-06T10:39:16.323263+0000","last_peered":"2023-10-06T10:39:16.323263+0000","last_clean":"2023-10-06T10:39:16.323263+0000","last_became_active":"2023-10-06T10:39:16.320019+0000","last_became_peered":"2023-10-06T10:39:16.320019+0000","last_unstale":"2023-10-06T10:39:16.323263+0000","last_undegraded":"2023-10-06T10:39:16.323263+0000","last_fullsized":"2023-10-06T10:39:16.323263+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:37.202391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.330559+0000","last_change":"2023-10-06T10:39:15.304330+0000","last_active":"2023-10-06T10:39:16.330559+0000","last_peered":"2023-10-06T10:39:16.330559+0000","last_clean":"2023-10-06T10:39:16.330559+0000","last_became_active":"2023-10-06T10:39:15.304075+0000","last_became_peered":"2023-10-06T10:39:15.304075+0000","last_unstale":"2023-10-06T10:39:16.330559+0000","last_undegraded":"2023-10-06T10:39:16.330559+0000","last_fullsized":"2023-10-06T10:39:16.330559+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:54.214826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":25,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.306962+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:39:16.306962+0000","last_peered":"2023-10-06T10:39:16.306962+0000","last_clean":"2023-10-06T10:39:16.306962+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:39:16.306962+0000","last_undegraded":"2023-10-06T10:39:16.306962+0000","last_fullsized":"2023-10-06T10:39:16.306962+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.327231+0000","last_change":"2023-10-06T10:39:16.324174+0000","last_active":"2023-10-06T10:39:16.327231+0000","last_peered":"2023-10-06T10:39:16.327231+0000","last_clean":"2023-10-06T10:39:16.327231+0000","last_became_active":"2023-10-06T10:39:16.323465+0000","last_became_peered":"2023-10-06T10:39:16.323465+0000","last_unstale":"2023-10-06T10:39:16.327231+0000","last_undegraded":"2023-10-06T10:39:16.327231+0000","last_fullsized":"2023-10-06T10:39:16.327231+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:54.259108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T10:39:17.330104+0000","last_change":"2023-10-06T10:39:15.309146+0000","last_active":"2023-10-06T10:39:17.330104+0000","last_peered":"2023-10-06T10:39:17.330104+0000","last_clean":"2023-10-06T10:39:17.330104+0000","last_became_active":"2023-10-06T10:39:15.308885+0000","last_became_peered":"2023-10-06T10:39:15.308885+0000","last_unstale":"2023-10-06T10:39:17.330104+0000","last_undegraded":"2023-10-06T10:39:17.330104+0000","last_fullsized":"2023-10-06T10:39:17.330104+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:25:48.384684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":31,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.304497+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:39:16.304497+0000","last_peered":"2023-10-06T10:39:16.304497+0000","last_clean":"2023-10-06T10:39:16.304497+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:39:16.304497+0000","last_undegraded":"2023-10-06T10:39:16.304497+0000","last_fullsized":"2023-10-06T10:39:16.304497+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.323266+0000","last_change":"2023-10-06T10:39:16.314100+0000","last_active":"2023-10-06T10:39:16.323266+0000","last_peered":"2023-10-06T10:39:16.323266+0000","last_clean":"2023-10-06T10:39:16.323266+0000","last_became_active":"2023-10-06T10:39:16.313841+0000","last_became_peered":"2023-10-06T10:39:16.313841+0000","last_unstale":"2023-10-06T10:39:16.323266+0000","last_undegraded":"2023-10-06T10:39:16.323266+0000","last_fullsized":"2023-10-06T10:39:16.323266+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:35:13.124890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.304709+0000","last_change":"2023-10-06T10:39:15.302089+0000","last_active":"2023-10-06T10:39:16.304709+0000","last_peered":"2023-10-06T10:39:16.304709+0000","last_clean":"2023-10-06T10:39:16.304709+0000","last_became_active":"2023-10-06T10:39:15.301841+0000","last_became_peered":"2023-10-06T10:39:15.301841+0000","last_unstale":"2023-10-06T10:39:16.304709+0000","last_undegraded":"2023-10-06T10:39:16.304709+0000","last_fullsized":"2023-10-06T10:39:16.304709+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:14:52.278898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":26,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.309203+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:39:16.309203+0000","last_peered":"2023-10-06T10:39:16.309203+0000","last_clean":"2023-10-06T10:39:16.309203+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:39:16.309203+0000","last_undegraded":"2023-10-06T10:39:16.309203+0000","last_fullsized":"2023-10-06T10:39:16.309203+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.321234+0000","last_change":"2023-10-06T10:39:16.313765+0000","last_active":"2023-10-06T10:39:16.321234+0000","last_peered":"2023-10-06T10:39:16.321234+0000","last_clean":"2023-10-06T10:39:16.321234+0000","last_became_active":"2023-10-06T10:39:16.313502+0000","last_became_peered":"2023-10-06T10:39:16.313502+0000","last_unstale":"2023-10-06T10:39:16.321234+0000","last_undegraded":"2023-10-06T10:39:16.321234+0000","last_fullsized":"2023-10-06T10:39:16.321234+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:15.286180+0000","last_clean_scrub_stamp":"2023-10-06T10:39:15.286180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:10.131667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.306285+0000","last_change":"2023-10-06T10:39:15.305824+0000","last_active":"2023-10-06T10:39:16.306285+0000","last_peered":"2023-10-06T10:39:16.306285+0000","last_clean":"2023-10-06T10:39:16.306285+0000","last_became_active":"2023-10-06T10:39:15.305587+0000","last_became_peered":"2023-10-06T10:39:15.305587+0000","last_unstale":"2023-10-06T10:39:16.306285+0000","last_undegraded":"2023-10-06T10:39:16.306285+0000","last_fullsized":"2023-10-06T10:39:16.306285+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:39:14.284678+0000","last_clean_scrub_stamp":"2023-10-06T10:39:14.284678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:54.245211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":26,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T10:39:16.311588+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:39:16.311588+0000","last_peered":"2023-10-06T10:39:16.311588+0000","last_clean":"2023-10-06T10:39:16.311588+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:39:16.311588+0000","last_undegraded":"2023-10-06T10:39:16.311588+0000","last_fullsized":"2023-10-06T10:39:16.311588+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298300,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073540,"statfs":{"total":96636764160,"available":96331304960,"internally_reserved":0,"allocated":1036288,"data_stored":786386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.436}]},{"osd":1,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.375},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.47099999999999997}]},{"osd":2,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.32400000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.41399999999999998}]},{"osd":4,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.38600000000000001}]},{"osd":5,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.40400000000000003}]},{"osd":6,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.46000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297720,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074120,"statfs":{"total":96636764160,"available":96331898880,"internally_reserved":0,"allocated":442368,"data_stored":195219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.436},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.60499999999999998}]},{"osd":2,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.48099999999999998}]},{"osd":3,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.57199999999999995}]},{"osd":4,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.39000000000000001}]},{"osd":5,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.5}]},{"osd":7,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.61699999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298300,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073540,"statfs":{"total":96636764160,"available":96331304960,"internally_reserved":0,"allocated":1036288,"data_stored":786366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.27400000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.316,"15min":0.316},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.32400000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.31}]},{"osd":4,"last update":"Fri Oct 6 10:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.48899999999999999}]},{"osd":5,"last update":"Fri Oct 6 10:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.499},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.44500000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.51700000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.53600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297720,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074120,"statfs":{"total":96636764160,"available":96331898880,"internally_reserved":0,"allocated":442368,"data_stored":195962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.316,"15min":0.316},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.432,"5min":0.432,"15min":0.432},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.373}]},{"osd":2,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.42799999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"last":0.40100000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.60299999999999998}]},{"osd":5,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.56399999999999995}]},{"osd":6,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.627}]},{"osd":7,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.63800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297732,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074108,"statfs":{"total":96636764160,"available":96331886592,"internally_reserved":0,"allocated":454656,"data_stored":197486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:39:11 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.214},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.20100000000000001}]},{"osd":1,"last update":"Fri Oct 6 10:39:11 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.26400000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:39:11 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.28899999999999998}]},{"osd":4,"last update":"Fri Oct 6 10:39:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.47299999999999998}]},{"osd":5,"last update":"Fri Oct 6 10:39:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.502},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.46700000000000003}]},{"osd":6,"last update":"Fri Oct 6 10:39:11 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.54800000000000004}]},{"osd":7,"last update":"Fri Oct 6 10:39:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297728,"kb_used_data":440,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074112,"statfs":{"total":96636764160,"available":96331890688,"internally_reserved":0,"allocated":450560,"data_stored":196538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":3.3210000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.28899999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.307},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":3.359,"5min":3.359,"15min":3.359},"last":0.27800000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":7.069,"5min":7.069,"15min":7.069},"last":0.34499999999999997}]},{"osd":4,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.52700000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.4140000000000001,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.437}]},{"osd":6,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.3879999999999999,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.49299999999999999}]},{"osd":7,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.54900000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297728,"kb_used_data":440,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074112,"statfs":{"total":96636764160,"available":96331890688,"internally_reserved":0,"allocated":450560,"data_stored":197464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.46999999999999997}]},{"osd":1,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.33100000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.36799999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.499},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.42499999999999999}]},{"osd":5,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.44600000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.312},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.38900000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:39:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.48899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297720,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074120,"statfs":{"total":96636764160,"available":96331898880,"internally_reserved":0,"allocated":442368,"data_stored":195998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.76600000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.59999999999999998}]},{"osd":3,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.65200000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.48399999999999999}]},{"osd":6,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.498}]},{"osd":7,"last update":"Fri Oct 6 10:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.46400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:39:20.745 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T10:39:20.746 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T10:39:20.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.897+0000 7f4a3759e700 1 Processor -- start 2023-10-06T10:39:20.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.897+0000 7f4a3759e700 1 -- start start 2023-10-06T10:39:20.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.898+0000 7f4a3759e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a380b0520 0x7f4a380cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:20.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.898+0000 7f4a3759e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4a380ccaa0 0x7f4a380c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:20.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.898+0000 7f4a3759e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a380c43d0 con 0x7f4a380c3f70 2023-10-06T10:39:20.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.898+0000 7f4a3759e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a380ccf10 con 0x7f4a380ccaa0 2023-10-06T10:39:20.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.898+0000 7f4a3759e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a380cdd30 con 0x7f4a380b0520 2023-10-06T10:39:20.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.898+0000 7f4a2dd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4a380ccaa0 0x7f4a380c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:20.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.898+0000 7f4a36d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4a380c3f70 0x7f4a3815d0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60226/0 (socket says 172.21.15.92:60226) 2023-10-06T10:39:20.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.899+0000 7f4a36d9d700 1 -- 172.21.15.92:0/832475623 learned_addr learned my addr 172.21.15.92:0/832475623 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:20.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.899+0000 7f4a3659c700 1 --2- 172.21.15.92:0/832475623 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a380b0520 0x7f4a380cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:20.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.899+0000 7f4a2dd9b700 1 -- 172.21.15.92:0/832475623 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a380b0520 msgr2=0x7f4a380cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:20.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.899+0000 7f4a2dd9b700 1 --2- 172.21.15.92:0/832475623 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a380b0520 0x7f4a380cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.899+0000 7f4a35d9b700 1 -- 172.21.15.92:0/832475623 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2830734858 0 0) 0x7f4a30001490 con 0x7f4a380c3f70 2023-10-06T10:39:20.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.899+0000 7f4a2dd9b700 1 -- 172.21.15.92:0/832475623 >> v1:172.21.15.92:6789/0 conn(0x7f4a380c3f70 legacy=0x7f4a3815d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:20.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.899+0000 7f4a2dd9b700 1 -- 172.21.15.92:0/832475623 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a380cd120 con 0x7f4a380ccaa0 2023-10-06T10:39:20.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.899+0000 7f4a2dd9b700 1 --2- 172.21.15.92:0/832475623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a380ccaa0 0x7f4a380c3a30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4a200098d0 tx=0x7f4a20009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b2a463a7c31520 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:20.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.899+0000 7f4a35d9b700 1 -- 172.21.15.92:0/832475623 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a20009e70 con 0x7f4a380ccaa0 2023-10-06T10:39:20.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.899+0000 7f4a35d9b700 1 -- 172.21.15.92:0/832475623 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a20005c10 con 0x7f4a380ccaa0 2023-10-06T10:39:20.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.899+0000 7f4a35d9b700 1 -- 172.21.15.92:0/832475623 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2000b650 con 0x7f4a380ccaa0 2023-10-06T10:39:20.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.900+0000 7f4a3759e700 1 -- 172.21.15.92:0/832475623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a380ccaa0 msgr2=0x7f4a380c3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:20.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.900+0000 7f4a3759e700 1 --2- 172.21.15.92:0/832475623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a380ccaa0 0x7f4a380c3a30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4a200098d0 tx=0x7f4a20009120 comp rx=0 tx=0).stop 2023-10-06T10:39:20.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.900+0000 7f4a3759e700 1 -- 172.21.15.92:0/832475623 shutdown_connections 2023-10-06T10:39:20.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.900+0000 7f4a3759e700 1 --2- 172.21.15.92:0/832475623 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a380b0520 0x7f4a380cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.900+0000 7f4a3759e700 1 --2- 172.21.15.92:0/832475623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a380ccaa0 0x7f4a380c3a30 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.900+0000 7f4a3759e700 1 -- 172.21.15.92:0/832475623 >> 172.21.15.92:0/832475623 conn(0x7f4a38096a30 msgr2=0x7f4a38096e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:20.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.900+0000 7f4a3759e700 1 -- 172.21.15.92:0/832475623 shutdown_connections 2023-10-06T10:39:20.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.900+0000 7f4a3759e700 1 -- 172.21.15.92:0/832475623 wait complete. 2023-10-06T10:39:20.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.901+0000 7f4a3759e700 1 Processor -- start 2023-10-06T10:39:20.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.901+0000 7f4a3759e700 1 -- start start 2023-10-06T10:39:20.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.902+0000 7f4a3759e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a380b0520 0x7f4a380b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:20.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.902+0000 7f4a3759e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a380c3f70 0x7f4a381533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:20.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.902+0000 7f4a3759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a380ccaa0 0x7f4a38153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:20.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.902+0000 7f4a3759e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a38154b30 con 0x7f4a380ccaa0 2023-10-06T10:39:20.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.902+0000 7f4a3759e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a38153f20 con 0x7f4a380b0520 2023-10-06T10:39:20.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.902+0000 7f4a3759e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a38154200 con 0x7f4a380c3f70 2023-10-06T10:39:20.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.902+0000 7f4a36d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a380ccaa0 0x7f4a38153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:20.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.902+0000 7f4a36d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a380ccaa0 0x7f4a38153910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46204/0 (socket says 172.21.15.92:46204) 2023-10-06T10:39:20.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.902+0000 7f4a36d9d700 1 -- 172.21.15.92:0/1905195780 learned_addr learned my addr 172.21.15.92:0/1905195780 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:20.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.902+0000 7f4a2dd9b700 1 --2- 172.21.15.92:0/1905195780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a380c3f70 0x7f4a381533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:20.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.902+0000 7f4a3659c700 1 --2- 172.21.15.92:0/1905195780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a380b0520 0x7f4a380b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:20.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.902+0000 7f4a36d9d700 1 -- 172.21.15.92:0/1905195780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a380c3f70 msgr2=0x7f4a381533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:20.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.902+0000 7f4a36d9d700 1 --2- 172.21.15.92:0/1905195780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a380c3f70 0x7f4a381533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.902+0000 7f4a36d9d700 1 -- 172.21.15.92:0/1905195780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a380b0520 msgr2=0x7f4a380b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:20.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.902+0000 7f4a36d9d700 1 --2- 172.21.15.92:0/1905195780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a380b0520 0x7f4a380b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:20.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.902+0000 7f4a36d9d700 1 -- 172.21.15.92:0/1905195780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a200093c0 con 0x7f4a380ccaa0 2023-10-06T10:39:20.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.902+0000 7f4a36d9d700 1 --2- 172.21.15.92:0/1905195780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a380ccaa0 0x7f4a38153910 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000cc90 tx=0x7f4a30001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:20.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.902+0000 7f4a2effd700 1 -- 172.21.15.92:0/1905195780 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3000cea0 con 0x7f4a380ccaa0 2023-10-06T10:39:20.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.902+0000 7f4a2effd700 1 -- 172.21.15.92:0/1905195780 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a30004bb0 con 0x7f4a380ccaa0 2023-10-06T10:39:20.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.902+0000 7f4a2effd700 1 -- 172.21.15.92:0/1905195780 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3000ab70 con 0x7f4a380ccaa0 2023-10-06T10:39:20.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.902+0000 7f4a3759e700 1 -- 172.21.15.92:0/1905195780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a381544e0 con 0x7f4a380ccaa0 2023-10-06T10:39:20.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.903+0000 7f4a3759e700 1 -- 172.21.15.92:0/1905195780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a380b30a0 con 0x7f4a380ccaa0 2023-10-06T10:39:20.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.904+0000 7f4a3759e700 1 -- 172.21.15.92:0/1905195780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a38154ca0 con 0x7f4a380ccaa0 2023-10-06T10:39:20.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.904+0000 7f4a2effd700 1 -- 172.21.15.92:0/1905195780 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4a300070f0 con 0x7f4a380ccaa0 2023-10-06T10:39:20.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.904+0000 7f4a2effd700 1 --2- 172.21.15.92:0/1905195780 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4a18066f30 0x7f4a180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:20.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.904+0000 7f4a2effd700 1 -- 172.21.15.92:0/1905195780 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f4a30084550 con 0x7f4a380ccaa0 2023-10-06T10:39:20.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.904+0000 7f4a3659c700 1 --2- 172.21.15.92:0/1905195780 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4a18066f30 0x7f4a180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:20.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.905+0000 7f4a3659c700 1 --2- 172.21.15.92:0/1905195780 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4a18066f30 0x7f4a180693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4a28006fd0 tx=0x7f4a28008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:20.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:20.908+0000 7f4a2effd700 1 -- 172.21.15.92:0/1905195780 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a300515a0 con 0x7f4a380ccaa0 2023-10-06T10:39:21.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.165+0000 7f4a3759e700 1 -- 172.21.15.92:0/1905195780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4a380c43d0 con 0x7f4a380ccaa0 2023-10-06T10:39:21.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.169+0000 7f4a2effd700 1 -- 172.21.15.92:0/1905195780 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f4a380c43d0 con 0x7f4a380ccaa0 2023-10-06T10:39:21.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.171+0000 7f4a3759e700 1 -- 172.21.15.92:0/1905195780 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4a18066f30 msgr2=0x7f4a180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:21.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.171+0000 7f4a3759e700 1 --2- 172.21.15.92:0/1905195780 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4a18066f30 0x7f4a180693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4a28006fd0 tx=0x7f4a28008040 comp rx=0 tx=0).stop 2023-10-06T10:39:21.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.171+0000 7f4a3759e700 1 -- 172.21.15.92:0/1905195780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a380ccaa0 msgr2=0x7f4a38153910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:21.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.171+0000 7f4a3759e700 1 --2- 172.21.15.92:0/1905195780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a380ccaa0 0x7f4a38153910 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000cc90 tx=0x7f4a30001a50 comp rx=0 tx=0).stop 2023-10-06T10:39:21.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.171+0000 7f4a3759e700 1 -- 172.21.15.92:0/1905195780 shutdown_connections 2023-10-06T10:39:21.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.171+0000 7f4a3759e700 1 --2- 172.21.15.92:0/1905195780 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4a18066f30 0x7f4a180693e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:21.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.171+0000 7f4a3759e700 1 --2- 172.21.15.92:0/1905195780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a380ccaa0 0x7f4a38153910 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:21.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.171+0000 7f4a3759e700 1 --2- 172.21.15.92:0/1905195780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a380b0520 0x7f4a380b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:21.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.171+0000 7f4a3759e700 1 --2- 172.21.15.92:0/1905195780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a380c3f70 0x7f4a381533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:21.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.171+0000 7f4a3759e700 1 -- 172.21.15.92:0/1905195780 >> 172.21.15.92:0/1905195780 conn(0x7f4a38096a30 msgr2=0x7f4a380c9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:21.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.171+0000 7f4a3759e700 1 -- 172.21.15.92:0/1905195780 shutdown_connections 2023-10-06T10:39:21.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.171+0000 7f4a3759e700 1 -- 172.21.15.92:0/1905195780 wait complete. 2023-10-06T10:39:21.173 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-10-06T10:39:21.183 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:39:21.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.296+0000 7f3f748af700 1 Processor -- start 2023-10-06T10:39:21.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.296+0000 7f3f748af700 1 -- start start 2023-10-06T10:39:21.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.297+0000 7f3f748af700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f7006e8b0 0x7f3f700685c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:21.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.297+0000 7f3f748af700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f70068f40 con 0x7f3f70068b00 2023-10-06T10:39:21.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.297+0000 7f3f748af700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f700cad20 con 0x7f3f701570f0 2023-10-06T10:39:21.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.297+0000 7f3f748af700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7006a560 con 0x7f3f7006e8b0 2023-10-06T10:39:21.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.297+0000 7f3f6ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3f70068b00 0x7f3f700cb4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60228/0 (socket says 172.21.15.92:60228) 2023-10-06T10:39:21.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.297+0000 7f3f6ffff700 1 -- 172.21.15.92:0/699258595 learned_addr learned my addr 172.21.15.92:0/699258595 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:21.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.298+0000 7f3f66ffd700 1 --2- 172.21.15.92:0/699258595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f7006e8b0 0x7f3f700685c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:21.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.298+0000 7f3f6effd700 1 -- 172.21.15.92:0/699258595 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765604080 0 0) 0x7f3f70068f40 con 0x7f3f70068b00 2023-10-06T10:39:21.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.298+0000 7f3f6effd700 1 -- 172.21.15.92:0/699258595 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f5c003570 con 0x7f3f70068b00 2023-10-06T10:39:21.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.298+0000 7f3f6effd700 1 -- 172.21.15.92:0/699258595 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2898307816 0 0) 0x7f3f5c003570 con 0x7f3f70068b00 2023-10-06T10:39:21.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.298+0000 7f3f6effd700 1 -- 172.21.15.92:0/699258595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f7006e8b0 msgr2=0x7f3f700685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:21.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.298+0000 7f3f6effd700 1 --2- 172.21.15.92:0/699258595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f7006e8b0 0x7f3f700685c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:21.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.298+0000 7f3f6effd700 1 -- 172.21.15.92:0/699258595 >> v1:172.21.15.124:6789/0 conn(0x7f3f701570f0 legacy=0x7f3f70069ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:21.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.298+0000 7f3f6effd700 1 -- 172.21.15.92:0/699258595 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f7006a840 con 0x7f3f70068b00 2023-10-06T10:39:21.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.298+0000 7f3f6effd700 1 -- 172.21.15.92:0/699258595 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f3f680017f0 con 0x7f3f70068b00 2023-10-06T10:39:21.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.298+0000 7f3f6effd700 1 -- 172.21.15.92:0/699258595 >> v1:172.21.15.92:6789/0 conn(0x7f3f70068b00 legacy=0x7f3f700cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:21.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.298+0000 7f3f6effd700 1 --2- 172.21.15.92:0/699258595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f7006e8b0 0x7f3f5c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:21.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.298+0000 7f3f6effd700 1 --2- 172.21.15.92:0/699258595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f5c004010 0x7f3f5c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:21.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.298+0000 7f3f6effd700 1 --2- 172.21.15.92:0/699258595 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f5c004d50 0x7f3f5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:21.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.298+0000 7f3f6effd700 1 -- 172.21.15.92:0/699258595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7006a560 con 0x7f3f5c004010 2023-10-06T10:39:21.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.298+0000 7f3f6effd700 1 -- 172.21.15.92:0/699258595 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c00bc70 con 0x7f3f5c004d50 2023-10-06T10:39:21.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.298+0000 7f3f6effd700 1 -- 172.21.15.92:0/699258595 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c00be30 con 0x7f3f7006e8b0 2023-10-06T10:39:21.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.298+0000 7f3f66ffd700 1 --2- 172.21.15.92:0/699258595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f5c004010 0x7f3f5c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:21.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.298+0000 7f3f6f7fe700 1 --2- 172.21.15.92:0/699258595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f7006e8b0 0x7f3f5c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:21.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.298+0000 7f3f6ffff700 1 --2- 172.21.15.92:0/699258595 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f5c004d50 0x7f3f5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:21.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.298+0000 7f3f66ffd700 1 -- 172.21.15.92:0/699258595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f7006e8b0 msgr2=0x7f3f5c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:21.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.299+0000 7f3f66ffd700 1 --2- 172.21.15.92:0/699258595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f7006e8b0 0x7f3f5c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:21.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.299+0000 7f3f66ffd700 1 -- 172.21.15.92:0/699258595 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f5c004d50 msgr2=0x7f3f5c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:21.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.299+0000 7f3f66ffd700 1 --2- 172.21.15.92:0/699258595 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f5c004d50 0x7f3f5c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:21.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.299+0000 7f3f66ffd700 1 -- 172.21.15.92:0/699258595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5c003570 con 0x7f3f5c004010 2023-10-06T10:39:21.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.299+0000 7f3f66ffd700 1 --2- 172.21.15.92:0/699258595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f5c004010 0x7f3f5c004810 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3f58006f00 tx=0x7f3f580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c41650fe1a02e53 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:21.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.299+0000 7f3f6effd700 1 -- 172.21.15.92:0/699258595 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5800e710 con 0x7f3f5c004010 2023-10-06T10:39:21.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.299+0000 7f3f6effd700 1 -- 172.21.15.92:0/699258595 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5800e870 con 0x7f3f5c004010 2023-10-06T10:39:21.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.299+0000 7f3f6effd700 1 -- 172.21.15.92:0/699258595 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5800ec30 con 0x7f3f5c004010 2023-10-06T10:39:21.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.299+0000 7f3f748af700 1 -- 172.21.15.92:0/699258595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f5c004010 msgr2=0x7f3f5c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:21.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.299+0000 7f3f748af700 1 --2- 172.21.15.92:0/699258595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f5c004010 0x7f3f5c004810 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3f58006f00 tx=0x7f3f580091a0 comp rx=0 tx=0).stop 2023-10-06T10:39:21.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.299+0000 7f3f6ffff700 1 -- 172.21.15.92:0/699258595 reap_dead start 2023-10-06T10:39:21.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.299+0000 7f3f748af700 1 -- 172.21.15.92:0/699258595 shutdown_connections 2023-10-06T10:39:21.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.299+0000 7f3f748af700 1 -- 172.21.15.92:0/699258595 >> 172.21.15.92:0/699258595 conn(0x7f3f700c9c10 msgr2=0x7f3f7006dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:21.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.299+0000 7f3f748af700 1 -- 172.21.15.92:0/699258595 shutdown_connections 2023-10-06T10:39:21.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.299+0000 7f3f748af700 1 -- 172.21.15.92:0/699258595 wait complete. 2023-10-06T10:39:21.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.300+0000 7f3f748af700 1 Processor -- start 2023-10-06T10:39:21.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.301+0000 7f3f748af700 1 -- start start 2023-10-06T10:39:21.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.301+0000 7f3f748af700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f5c004010 0x7f3f701417a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:21.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.301+0000 7f3f748af700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f701526d0 0x7f3f700c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:21.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.301+0000 7f3f748af700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f701550c0 0x7f3f70155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:21.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.301+0000 7f3f748af700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f70155aa0 con 0x7f3f5c004010 2023-10-06T10:39:21.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.301+0000 7f3f748af700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f70141ce0 con 0x7f3f701550c0 2023-10-06T10:39:21.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.301+0000 7f3f748af700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f70152c40 con 0x7f3f701526d0 2023-10-06T10:39:21.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.301+0000 7f3f6f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f5c004010 0x7f3f701417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:21.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.302+0000 7f3f6f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f5c004010 0x7f3f701417a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46224/0 (socket says 172.21.15.92:46224) 2023-10-06T10:39:21.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.302+0000 7f3f6f7fe700 1 -- 172.21.15.92:0/990871741 learned_addr learned my addr 172.21.15.92:0/990871741 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:21.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.302+0000 7f3f66ffd700 1 --2- 172.21.15.92:0/990871741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f701526d0 0x7f3f700c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:21.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.302+0000 7f3f6ffff700 1 --2- 172.21.15.92:0/990871741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f701550c0 0x7f3f70155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:21.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.302+0000 7f3f6f7fe700 1 -- 172.21.15.92:0/990871741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f701526d0 msgr2=0x7f3f700c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:21.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.302+0000 7f3f6f7fe700 1 --2- 172.21.15.92:0/990871741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f701526d0 0x7f3f700c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:21.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.302+0000 7f3f6f7fe700 1 -- 172.21.15.92:0/990871741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f701550c0 msgr2=0x7f3f70155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:21.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.302+0000 7f3f6f7fe700 1 --2- 172.21.15.92:0/990871741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f701550c0 0x7f3f70155560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:21.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.302+0000 7f3f6f7fe700 1 -- 172.21.15.92:0/990871741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f70152f20 con 0x7f3f5c004010 2023-10-06T10:39:21.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.302+0000 7f3f66ffd700 1 --2- 172.21.15.92:0/990871741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f701526d0 0x7f3f700c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:21.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.302+0000 7f3f6ffff700 1 --2- 172.21.15.92:0/990871741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f701550c0 0x7f3f70155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:21.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.302+0000 7f3f6f7fe700 1 --2- 172.21.15.92:0/990871741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f5c004010 0x7f3f701417a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3f600029d0 tx=0x7f3f60007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:21.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.302+0000 7f3f67fff700 1 -- 172.21.15.92:0/990871741 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6000f870 con 0x7f3f5c004010 2023-10-06T10:39:21.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.302+0000 7f3f67fff700 1 -- 172.21.15.92:0/990871741 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6000f9d0 con 0x7f3f5c004010 2023-10-06T10:39:21.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.302+0000 7f3f748af700 1 -- 172.21.15.92:0/990871741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f70153200 con 0x7f3f5c004010 2023-10-06T10:39:21.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.302+0000 7f3f67fff700 1 -- 172.21.15.92:0/990871741 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6000fd90 con 0x7f3f5c004010 2023-10-06T10:39:21.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.302+0000 7f3f748af700 1 -- 172.21.15.92:0/990871741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f701537c0 con 0x7f3f5c004010 2023-10-06T10:39:21.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.303+0000 7f3f748af700 1 -- 172.21.15.92:0/990871741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f7005f230 con 0x7f3f5c004010 2023-10-06T10:39:21.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.303+0000 7f3f67fff700 1 -- 172.21.15.92:0/990871741 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3f60013600 con 0x7f3f5c004010 2023-10-06T10:39:21.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.304+0000 7f3f67fff700 1 --2- 172.21.15.92:0/990871741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3f50066f30 0x7f3f500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:21.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.304+0000 7f3f67fff700 1 -- 172.21.15.92:0/990871741 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3f60084d60 con 0x7f3f5c004010 2023-10-06T10:39:21.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.304+0000 7f3f66ffd700 1 --2- 172.21.15.92:0/990871741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3f50066f30 0x7f3f500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:21.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.304+0000 7f3f66ffd700 1 --2- 172.21.15.92:0/990871741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3f50066f30 0x7f3f500693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3f700ae720 tx=0x7f3f5801a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:21.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.308+0000 7f3f67fff700 1 -- 172.21.15.92:0/990871741 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f60051d30 con 0x7f3f5c004010 2023-10-06T10:39:21.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.531+0000 7f3f748af700 1 -- 172.21.15.92:0/990871741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f700cafc0 con 0x7f3f5c004010 2023-10-06T10:39:21.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.532+0000 7f3f67fff700 1 -- 172.21.15.92:0/990871741 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f3f60055350 con 0x7f3f5c004010 2023-10-06T10:39:21.533 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:39:21.533 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2525196781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2525196781},{"type":"v1","addr":"172.21.15.124:6835","nonce":2525196781}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5118,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2409440810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2409440810},{"type":"v1","addr":"172.21.15.92:6836","nonce":2409440810}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5121,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/4010629505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4010629505},{"type":"v1","addr":"172.21.15.92:6837","nonce":4010629505}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:39:15.675013+0000","modified":"2023-10-06T10:39:19.322877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5133":{"gid":5133,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.124:6834/484869337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":484869337},{"type":"v1","addr":"172.21.15.124:6834","nonce":484869337}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T10:39:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.534+0000 7f3f748af700 1 -- 172.21.15.92:0/990871741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3f50066f30 msgr2=0x7f3f500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.534+0000 7f3f748af700 1 --2- 172.21.15.92:0/990871741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3f50066f30 0x7f3f500693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3f700ae720 tx=0x7f3f5801a000 comp rx=0 tx=0).stop 2023-10-06T10:39:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.534+0000 7f3f748af700 1 -- 172.21.15.92:0/990871741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f5c004010 msgr2=0x7f3f701417a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.534+0000 7f3f748af700 1 --2- 172.21.15.92:0/990871741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f5c004010 0x7f3f701417a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3f600029d0 tx=0x7f3f60007d30 comp rx=0 tx=0).stop 2023-10-06T10:39:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.534+0000 7f3f748af700 1 -- 172.21.15.92:0/990871741 shutdown_connections 2023-10-06T10:39:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.534+0000 7f3f748af700 1 --2- 172.21.15.92:0/990871741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3f50066f30 0x7f3f500693e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.534+0000 7f3f748af700 1 --2- 172.21.15.92:0/990871741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f701550c0 0x7f3f70155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.534+0000 7f3f748af700 1 --2- 172.21.15.92:0/990871741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f5c004010 0x7f3f701417a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.534+0000 7f3f748af700 1 --2- 172.21.15.92:0/990871741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f701526d0 0x7f3f700c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.534+0000 7f3f748af700 1 -- 172.21.15.92:0/990871741 >> 172.21.15.92:0/990871741 conn(0x7f3f700c9c10 msgr2=0x7f3f700b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.534+0000 7f3f748af700 1 -- 172.21.15.92:0/990871741 shutdown_connections 2023-10-06T10:39:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:21.534+0000 7f3f748af700 1 -- 172.21.15.92:0/990871741 wait complete. 2023-10-06T10:39:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 16 2023-10-06T10:39:21.949 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T10:39:15.675013+0000', 'modified': '2023-10-06T10:39:19.322877+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5133}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5133': {'gid': 5133, 'name': 'b', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.124:6834/484869337', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6832', 'nonce': 484869337}, {'type': 'v1', 'addr': '172.21.15.124:6834', 'nonce': 484869337}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T10:39:21.949 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T10:39:21.950 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T10:39:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.066+0000 7f63dc8a0700 1 Processor -- start 2023-10-06T10:39:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.066+0000 7f63dc8a0700 1 -- start start 2023-10-06T10:39:22.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.067+0000 7f63dc8a0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63d80b49a0 0x7f63d80aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.067+0000 7f63dc8a0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f63d80b0760 0x7f63d80b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.067+0000 7f63dc8a0700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d80b0c60 con 0x7f63d8159560 2023-10-06T10:39:22.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.067+0000 7f63dc8a0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d80b0f40 con 0x7f63d80b0760 2023-10-06T10:39:22.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.067+0000 7f63dc8a0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d80b1220 con 0x7f63d80b49a0 2023-10-06T10:39:22.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.067+0000 7f63d6d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f63d80b0760 0x7f63d80b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.067+0000 7f63d6d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f63d80b0760 0x7f63d80b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46230/0 (socket says 172.21.15.92:46230) 2023-10-06T10:39:22.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.067+0000 7f63d6d9d700 1 -- 172.21.15.92:0/1701294639 learned_addr learned my addr 172.21.15.92:0/1701294639 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:22.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.067+0000 7f63cdd9b700 1 --2- 172.21.15.92:0/1701294639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63d80b49a0 0x7f63d80aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.067+0000 7f63d6d9d700 1 -- 172.21.15.92:0/1701294639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63d80b49a0 msgr2=0x7f63d80aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:22.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.067+0000 7f63d6d9d700 1 --2- 172.21.15.92:0/1701294639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63d80b49a0 0x7f63d80aac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.067+0000 7f63d6d9d700 1 -- 172.21.15.92:0/1701294639 >> v1:172.21.15.124:6789/0 conn(0x7f63d8159560 legacy=0x7f63d80aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:22.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.067+0000 7f63d6d9d700 1 -- 172.21.15.92:0/1701294639 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d80b1500 con 0x7f63d80b0760 2023-10-06T10:39:22.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.067+0000 7f63cdd9b700 1 --2- 172.21.15.92:0/1701294639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63d80b49a0 0x7f63d80aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:22.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.068+0000 7f63d6d9d700 1 --2- 172.21.15.92:0/1701294639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63d80b0760 0x7f63d80b1bc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009f20 tx=0x7f63d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434a53297c0b921e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:22.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.068+0000 7f63d5d9b700 1 -- 172.21.15.92:0/1701294639 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d000e070 con 0x7f63d80b0760 2023-10-06T10:39:22.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.068+0000 7f63d5d9b700 1 -- 172.21.15.92:0/1701294639 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d000ba80 con 0x7f63d80b0760 2023-10-06T10:39:22.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.068+0000 7f63d5d9b700 1 -- 172.21.15.92:0/1701294639 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d00092e0 con 0x7f63d80b0760 2023-10-06T10:39:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.068+0000 7f63dc8a0700 1 -- 172.21.15.92:0/1701294639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63d80b0760 msgr2=0x7f63d80b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.068+0000 7f63dc8a0700 1 --2- 172.21.15.92:0/1701294639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63d80b0760 0x7f63d80b1bc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009f20 tx=0x7f63d0009840 comp rx=0 tx=0).stop 2023-10-06T10:39:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.068+0000 7f63dc8a0700 1 -- 172.21.15.92:0/1701294639 shutdown_connections 2023-10-06T10:39:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.068+0000 7f63dc8a0700 1 --2- 172.21.15.92:0/1701294639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63d80b0760 0x7f63d80b1bc0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.068+0000 7f63dc8a0700 1 --2- 172.21.15.92:0/1701294639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63d80b49a0 0x7f63d80aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.068+0000 7f63dc8a0700 1 -- 172.21.15.92:0/1701294639 >> 172.21.15.92:0/1701294639 conn(0x7f63d80997f0 msgr2=0x7f63d806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:22.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.068+0000 7f63dc8a0700 1 -- 172.21.15.92:0/1701294639 shutdown_connections 2023-10-06T10:39:22.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.068+0000 7f63dc8a0700 1 -- 172.21.15.92:0/1701294639 wait complete. 2023-10-06T10:39:22.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.069+0000 7f63dc8a0700 1 Processor -- start 2023-10-06T10:39:22.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.070+0000 7f63dc8a0700 1 -- start start 2023-10-06T10:39:22.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.070+0000 7f63dc8a0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63d80b0760 0x7f63d809b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.070+0000 7f63dc8a0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63d80b49a0 0x7f63d815bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.070+0000 7f63dc8a0700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63d8159560 0x7f63d80b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.070+0000 7f63dc8a0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d815a410 con 0x7f63d80b49a0 2023-10-06T10:39:22.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.070+0000 7f63dc8a0700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d815c7b0 con 0x7f63d8159560 2023-10-06T10:39:22.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.070+0000 7f63dc8a0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d815ca90 con 0x7f63d80b0760 2023-10-06T10:39:22.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.070+0000 7f63cdd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63d80b49a0 0x7f63d815bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.070+0000 7f63cdd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63d80b49a0 0x7f63d815bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46234/0 (socket says 172.21.15.92:46234) 2023-10-06T10:39:22.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.070+0000 7f63cdd9b700 1 -- 172.21.15.92:0/3352308924 learned_addr learned my addr 172.21.15.92:0/3352308924 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:22.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.070+0000 7f63d6d9d700 1 --2- 172.21.15.92:0/3352308924 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63d8159560 0x7f63d80b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.070+0000 7f63d659c700 1 --2- 172.21.15.92:0/3352308924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63d80b0760 0x7f63d809b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.071+0000 7f63cdd9b700 1 -- 172.21.15.92:0/3352308924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63d80b0760 msgr2=0x7f63d809b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:22.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.071+0000 7f63cdd9b700 1 --2- 172.21.15.92:0/3352308924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63d80b0760 0x7f63d809b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.071+0000 7f63cdd9b700 1 -- 172.21.15.92:0/3352308924 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63d8159560 msgr2=0x7f63d80b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:22.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.071+0000 7f63cdd9b700 1 --2- 172.21.15.92:0/3352308924 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63d8159560 0x7f63d80b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.071+0000 7f63cdd9b700 1 -- 172.21.15.92:0/3352308924 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d0009ae0 con 0x7f63d80b49a0 2023-10-06T10:39:22.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.071+0000 7f63d6d9d700 1 --2- 172.21.15.92:0/3352308924 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63d8159560 0x7f63d80b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:22.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.071+0000 7f63cdd9b700 1 --2- 172.21.15.92:0/3352308924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63d80b49a0 0x7f63d815bdd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f63c000e8d0 tx=0x7f63c0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:22.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.071+0000 7f63ceffd700 1 -- 172.21.15.92:0/3352308924 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c000ee70 con 0x7f63d80b49a0 2023-10-06T10:39:22.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.071+0000 7f63ceffd700 1 -- 172.21.15.92:0/3352308924 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c000c5e0 con 0x7f63d80b49a0 2023-10-06T10:39:22.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.071+0000 7f63ceffd700 1 -- 172.21.15.92:0/3352308924 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c000c980 con 0x7f63d80b49a0 2023-10-06T10:39:22.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.071+0000 7f63dc8a0700 1 -- 172.21.15.92:0/3352308924 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d815cd70 con 0x7f63d80b49a0 2023-10-06T10:39:22.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.071+0000 7f63dc8a0700 1 -- 172.21.15.92:0/3352308924 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d80ce190 con 0x7f63d80b49a0 2023-10-06T10:39:22.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.072+0000 7f63ceffd700 1 -- 172.21.15.92:0/3352308924 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f63c0013410 con 0x7f63d80b49a0 2023-10-06T10:39:22.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.073+0000 7f63ceffd700 1 --2- 172.21.15.92:0/3352308924 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f63b8066fb0 0x7f63b8069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.073+0000 7f63ceffd700 1 -- 172.21.15.92:0/3352308924 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f63c0084660 con 0x7f63d80b49a0 2023-10-06T10:39:22.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.073+0000 7f63d659c700 1 --2- 172.21.15.92:0/3352308924 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f63b8066fb0 0x7f63b8069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.073+0000 7f63d659c700 1 --2- 172.21.15.92:0/3352308924 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f63b8066fb0 0x7f63b8069460 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f63c80037f0 tx=0x7f63c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:22.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.075+0000 7f63e5b98b80 1 Processor -- start 2023-10-06T10:39:22.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.075+0000 7f63e5b98b80 1 -- start start 2023-10-06T10:39:22.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.075+0000 7f63e5b98b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d5ef569a90 0x55d5ef569e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.075+0000 7f63e5b98b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef5259a0 0x55d5ef522580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.075+0000 7f63e5b98b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef523bd0 0x55d5ef522ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.075+0000 7f63e5b98b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ef4a2c80 con 0x55d5ef5259a0 2023-10-06T10:39:22.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.075+0000 7f63e5b98b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ef479a70 con 0x55d5ef523bd0 2023-10-06T10:39:22.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.075+0000 7f63e5b98b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5ef477270 con 0x55d5ef569a90 2023-10-06T10:39:22.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.076+0000 7f63cdd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef5259a0 0x55d5ef522580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.076+0000 7f63cdd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef5259a0 0x55d5ef522580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46240/0 (socket says 172.21.15.92:46240) 2023-10-06T10:39:22.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.076+0000 7f63cdd9b700 1 -- 172.21.15.92:0/1776100794 learned_addr learned my addr 172.21.15.92:0/1776100794 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:22.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.076+0000 7f63d659c700 1 --2- 172.21.15.92:0/1776100794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d5ef569a90 0x55d5ef569e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.076+0000 7f63d6d9d700 1 --2- 172.21.15.92:0/1776100794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef523bd0 0x55d5ef522ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.076+0000 7f63cdd9b700 1 -- 172.21.15.92:0/1776100794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d5ef569a90 msgr2=0x55d5ef569e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:22.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.076+0000 7f63cdd9b700 1 --2- 172.21.15.92:0/1776100794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d5ef569a90 0x55d5ef569e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.076+0000 7f63cdd9b700 1 -- 172.21.15.92:0/1776100794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef523bd0 msgr2=0x55d5ef522ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:22.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.076+0000 7f63cdd9b700 1 --2- 172.21.15.92:0/1776100794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef523bd0 0x55d5ef522ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.076+0000 7f63cdd9b700 1 -- 172.21.15.92:0/1776100794 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5ef30d270 con 0x55d5ef5259a0 2023-10-06T10:39:22.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.076+0000 7f63d6d9d700 1 --2- 172.21.15.92:0/1776100794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef523bd0 0x55d5ef522ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:22.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.076+0000 7f63cdd9b700 1 --2- 172.21.15.92:0/1776100794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef5259a0 0x55d5ef522580 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f63c0051750 tx=0x7f63c00827a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba8531e41321758 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:22.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.076+0000 7f63b77fe700 1 -- 172.21.15.92:0/1776100794 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0054d90 con 0x55d5ef5259a0 2023-10-06T10:39:22.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.076+0000 7f63b77fe700 1 -- 172.21.15.92:0/1776100794 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c0055250 con 0x55d5ef5259a0 2023-10-06T10:39:22.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.076+0000 7f63b77fe700 1 -- 172.21.15.92:0/1776100794 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c00584a0 con 0x55d5ef5259a0 2023-10-06T10:39:22.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.077+0000 7f63e5b98b80 1 -- 172.21.15.92:0/1776100794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef5259a0 msgr2=0x55d5ef522580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:22.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.077+0000 7f63e5b98b80 1 --2- 172.21.15.92:0/1776100794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef5259a0 0x55d5ef522580 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f63c0051750 tx=0x7f63c00827a0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.077+0000 7f63e5b98b80 1 -- 172.21.15.92:0/1776100794 shutdown_connections 2023-10-06T10:39:22.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.077+0000 7f63e5b98b80 1 --2- 172.21.15.92:0/1776100794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef523bd0 0x55d5ef522ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.077+0000 7f63e5b98b80 1 --2- 172.21.15.92:0/1776100794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d5ef569a90 0x55d5ef569e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.077+0000 7f63e5b98b80 1 --2- 172.21.15.92:0/1776100794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef5259a0 0x55d5ef522580 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.077+0000 7f63e5b98b80 1 -- 172.21.15.92:0/1776100794 >> 172.21.15.92:0/1776100794 conn(0x55d5ef524070 msgr2=0x55d5ef524440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:22.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.077+0000 7f63e5b98b80 1 -- 172.21.15.92:0/1776100794 shutdown_connections 2023-10-06T10:39:22.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.077+0000 7f63e5b98b80 1 -- 172.21.15.92:0/1776100794 wait complete. 2023-10-06T10:39:22.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.077+0000 7f63e5b98b80 1 -- start start 2023-10-06T10:39:22.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.078+0000 7f63e5b98b80 1 Processor -- start 2023-10-06T10:39:22.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.078+0000 7f63e5b98b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef523bd0 0x55d5ef5494c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.078+0000 7f63e5b98b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d5ef5259a0 0x55d5ef54c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.078+0000 7f63e5b98b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef569a90 0x55d5ef54df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.078+0000 7f63e5b98b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ef473060 con 0x55d5ef523bd0 2023-10-06T10:39:22.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.078+0000 7f63e5b98b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ef41e890 con 0x55d5ef569a90 2023-10-06T10:39:22.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.078+0000 7f63e5b98b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5ef439ea0 con 0x55d5ef5259a0 2023-10-06T10:39:22.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.078+0000 7f63e5b98b80 10 client.0 _ll_register_callbacks cb 0x55d5ef30ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:39:22.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.078+0000 7f63d659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef523bd0 0x55d5ef5494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.078+0000 7f63d659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef523bd0 0x55d5ef5494c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46252/0 (socket says 172.21.15.92:46252) 2023-10-06T10:39:22.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.079+0000 7f63d659c700 1 -- 172.21.15.92:0/408051305 learned_addr learned my addr 172.21.15.92:0/408051305 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:22.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.079+0000 7f63cdd9b700 1 --2- 172.21.15.92:0/408051305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d5ef5259a0 0x55d5ef54c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.079+0000 7f63d6d9d700 1 --2- 172.21.15.92:0/408051305 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef569a90 0x55d5ef54df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.079+0000 7f63d659c700 1 -- 172.21.15.92:0/408051305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d5ef5259a0 msgr2=0x55d5ef54c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:22.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.079+0000 7f63d659c700 1 --2- 172.21.15.92:0/408051305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d5ef5259a0 0x55d5ef54c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.079+0000 7f63d659c700 1 -- 172.21.15.92:0/408051305 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef569a90 msgr2=0x55d5ef54df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:22.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.079+0000 7f63d659c700 1 --2- 172.21.15.92:0/408051305 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef569a90 0x55d5ef54df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.079+0000 7f63d659c700 1 -- 172.21.15.92:0/408051305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d80ce190 con 0x55d5ef523bd0 2023-10-06T10:39:22.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.079+0000 7f63d659c700 1 --2- 172.21.15.92:0/408051305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef523bd0 0x55d5ef5494c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f63c8013f20 tx=0x7f63c8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ebfc14083544c86 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:22.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.079+0000 7f63b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:39:22.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.079+0000 7f63b4ff9700 1 -- 172.21.15.92:0/408051305 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8024070 con 0x55d5ef523bd0 2023-10-06T10:39:22.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.079+0000 7f63b4ff9700 1 -- 172.21.15.92:0/408051305 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f63c801b040 con 0x55d5ef523bd0 2023-10-06T10:39:22.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.079+0000 7f63e5b98b80 1 -- 172.21.15.92:0/408051305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5ef517690 con 0x55d5ef523bd0 2023-10-06T10:39:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.079+0000 7f63b4ff9700 1 -- 172.21.15.92:0/408051305 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63c8006870 con 0x55d5ef523bd0 2023-10-06T10:39:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.079+0000 7f63e5b98b80 10 client.5304 fetch_fsmap learned FSMap version 16 2023-10-06T10:39:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.079+0000 7f63e5b98b80 1 -- 172.21.15.92:0/408051305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55d5ef2affe0 con 0x55d5ef523bd0 2023-10-06T10:39:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.079+0000 7f63b4ff9700 1 -- 172.21.15.92:0/408051305 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d5ef517690 con 0x55d5ef523bd0 2023-10-06T10:39:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.079+0000 7f63b4ff9700 1 -- 172.21.15.92:0/408051305 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 16) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f63c8027020 con 0x55d5ef523bd0 2023-10-06T10:39:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.079+0000 7f63e5b98b80 10 client.5304 fetch_fsmap finished waiting for FSMap version 16 2023-10-06T10:39:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.079+0000 7f63e5b98b80 10 client.5304 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T10:39:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.079+0000 7f63e5b98b80 20 client.5304 populate_metadata read hostname 'smithi092' 2023-10-06T10:39:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.079+0000 7f63e5b98b80 1 --2- 172.21.15.92:0/408051305 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x55d5ef54af50 0x55d5ef54b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.080+0000 7f63e5b98b80 4 client.5304 mds_command: new command op to 5133 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:39:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.080+0000 7f63e5b98b80 1 -- 172.21.15.92:0/408051305 --> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5ef4f45c0 con 0x55d5ef54af50 2023-10-06T10:39:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.080+0000 7f63cdd9b700 1 --2- 172.21.15.92:0/408051305 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x55d5ef54af50 0x55d5ef54b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.080+0000 7f63cdd9b700 1 --2- 172.21.15.92:0/408051305 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x55d5ef54af50 0x55d5ef54b320 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=471c86eaa74a4ed3 server_cookie=f73254a6b5e1bcf9 in_seq=0 out_seq=0 2023-10-06T10:39:22.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.080+0000 7f63b4ff9700 10 client.5304 ms_handle_connect on v2:172.21.15.124:6832/484869337 2023-10-06T10:39:22.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.081+0000 7f63b4ff9700 1 -- 172.21.15.92:0/408051305 <== mds.0 v2:172.21.15.124:6832/484869337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55d5ef4f45c0 con 0x55d5ef54af50 2023-10-06T10:39:22.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.081+0000 7f63b4ff9700 10 client.5304 handle_command_reply: tid=0 2023-10-06T10:39:22.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.081+0000 7f63e5b98b80 1 client.5304 shutdown 2023-10-06T10:39:22.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.081+0000 7f63e5b98b80 1 -- 172.21.15.92:0/408051305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef523bd0 msgr2=0x55d5ef5494c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:22.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.081+0000 7f63e5b98b80 1 --2- 172.21.15.92:0/408051305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef523bd0 0x55d5ef5494c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f63c8013f20 tx=0x7f63c8014000 comp rx=0 tx=0).stop 2023-10-06T10:39:22.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.081+0000 7f63e5b98b80 1 -- 172.21.15.92:0/408051305 shutdown_connections 2023-10-06T10:39:22.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.081+0000 7f63e5b98b80 1 --2- 172.21.15.92:0/408051305 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x55d5ef54af50 0x55d5ef54b320 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.081+0000 7f63e5b98b80 1 --2- 172.21.15.92:0/408051305 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef569a90 0x55d5ef54df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.081+0000 7f63e5b98b80 1 --2- 172.21.15.92:0/408051305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef523bd0 0x55d5ef5494c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.081+0000 7f63b4ff9700 0 client.5304 ms_handle_reset on v2:172.21.15.124:6832/484869337 2023-10-06T10:39:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.081+0000 7f63e5b98b80 1 --2- 172.21.15.92:0/408051305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d5ef5259a0 0x55d5ef54c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.081+0000 7f63e5b98b80 1 -- 172.21.15.92:0/408051305 >> 172.21.15.92:0/408051305 conn(0x55d5ef524070 msgr2=0x55d5ef567560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.081+0000 7f63e5b98b80 1 -- 172.21.15.92:0/408051305 shutdown_connections 2023-10-06T10:39:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.081+0000 7f63e5b98b80 1 -- 172.21.15.92:0/408051305 wait complete. 2023-10-06T10:39:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.081+0000 7f63e5b98b80 20 client.5304 trim_cache size 0 max 16384 2023-10-06T10:39:22.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.098+0000 7f63e5b98b80 1 Processor -- start 2023-10-06T10:39:22.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.098+0000 7f63e5b98b80 1 -- start start 2023-10-06T10:39:22.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.099+0000 7f63e5b98b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef523bd0 0x55d5ef51dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.099+0000 7f63e5b98b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d5ef5259a0 0x55d5ef54ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.099+0000 7f63e5b98b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef569a90 0x55d5ef57b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.099+0000 7f63e5b98b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ef2f5e30 con 0x55d5ef523bd0 2023-10-06T10:39:22.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.099+0000 7f63e5b98b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ef569f60 con 0x55d5ef569a90 2023-10-06T10:39:22.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.099+0000 7f63e5b98b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5ef524440 con 0x55d5ef5259a0 2023-10-06T10:39:22.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.099+0000 7f63d659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef523bd0 0x55d5ef51dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.099+0000 7f63d659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef523bd0 0x55d5ef51dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46268/0 (socket says 172.21.15.92:46268) 2023-10-06T10:39:22.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.099+0000 7f63d659c700 1 -- 172.21.15.92:0/2690790234 learned_addr learned my addr 172.21.15.92:0/2690790234 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:22.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.099+0000 7f63d6d9d700 1 --2- 172.21.15.92:0/2690790234 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef569a90 0x55d5ef57b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.099+0000 7f63cdd9b700 1 --2- 172.21.15.92:0/2690790234 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d5ef5259a0 0x55d5ef54ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.099+0000 7f63d659c700 1 -- 172.21.15.92:0/2690790234 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d5ef5259a0 msgr2=0x55d5ef54ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:22.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.099+0000 7f63d659c700 1 --2- 172.21.15.92:0/2690790234 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d5ef5259a0 0x55d5ef54ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.099+0000 7f63d659c700 1 -- 172.21.15.92:0/2690790234 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef569a90 msgr2=0x55d5ef57b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:22.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.099+0000 7f63d659c700 1 --2- 172.21.15.92:0/2690790234 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef569a90 0x55d5ef57b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.099+0000 7f63d659c700 1 -- 172.21.15.92:0/2690790234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5ef441780 con 0x55d5ef523bd0 2023-10-06T10:39:22.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.099+0000 7f63d6d9d700 1 --2- 172.21.15.92:0/2690790234 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef569a90 0x55d5ef57b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:22.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.099+0000 7f63cdd9b700 1 --2- 172.21.15.92:0/2690790234 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d5ef5259a0 0x55d5ef54ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:22.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.099+0000 7f63d659c700 1 --2- 172.21.15.92:0/2690790234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef523bd0 0x55d5ef51dd70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f63c80030c0 tx=0x7f63c8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abd57b7be9ff746 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:22.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.099+0000 7f63b7fff700 1 -- 172.21.15.92:0/2690790234 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8024040 con 0x55d5ef523bd0 2023-10-06T10:39:22.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.099+0000 7f63b7fff700 1 -- 172.21.15.92:0/2690790234 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c8006870 con 0x55d5ef523bd0 2023-10-06T10:39:22.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.099+0000 7f63b7fff700 1 -- 172.21.15.92:0/2690790234 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8027570 con 0x55d5ef523bd0 2023-10-06T10:39:22.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.100+0000 7f63e5b98b80 1 -- 172.21.15.92:0/2690790234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef523bd0 msgr2=0x55d5ef51dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:22.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.100+0000 7f63e5b98b80 1 --2- 172.21.15.92:0/2690790234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef523bd0 0x55d5ef51dd70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f63c80030c0 tx=0x7f63c8020000 comp rx=0 tx=0).stop 2023-10-06T10:39:22.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.100+0000 7f63e5b98b80 1 -- 172.21.15.92:0/2690790234 shutdown_connections 2023-10-06T10:39:22.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.100+0000 7f63e5b98b80 1 --2- 172.21.15.92:0/2690790234 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef569a90 0x55d5ef57b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.100+0000 7f63e5b98b80 1 --2- 172.21.15.92:0/2690790234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef523bd0 0x55d5ef51dd70 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.100+0000 7f63e5b98b80 1 --2- 172.21.15.92:0/2690790234 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d5ef5259a0 0x55d5ef54ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.100+0000 7f63e5b98b80 1 -- 172.21.15.92:0/2690790234 >> 172.21.15.92:0/2690790234 conn(0x55d5ef524070 msgr2=0x55d5ef565ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:22.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.100+0000 7f63e5b98b80 1 -- 172.21.15.92:0/2690790234 shutdown_connections 2023-10-06T10:39:22.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.100+0000 7f63e5b98b80 1 -- 172.21.15.92:0/2690790234 wait complete. 2023-10-06T10:39:22.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.100+0000 7f63e5b98b80 1 -- start start 2023-10-06T10:39:22.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.100+0000 7f63e5b98b80 1 Processor -- start 2023-10-06T10:39:22.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.101+0000 7f63e5b98b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef523bd0 0x55d5ef568350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.101+0000 7f63e5b98b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d5ef5259a0 0x55d5ef582b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.101+0000 7f63e5b98b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef569a90 0x55d5ef5cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.101+0000 7f63e5b98b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ef568ba0 con 0x55d5ef569a90 2023-10-06T10:39:22.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.101+0000 7f63e5b98b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ef548bc0 con 0x55d5ef523bd0 2023-10-06T10:39:22.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.101+0000 7f63e5b98b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5ef5830b0 con 0x55d5ef5259a0 2023-10-06T10:39:22.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.101+0000 7f63e5b98b80 10 client.0 _ll_register_callbacks cb 0x55d5ef30ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:39:22.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.101+0000 7f63d6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef569a90 0x55d5ef5cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.101+0000 7f63d6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef569a90 0x55d5ef5cf4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46280/0 (socket says 172.21.15.92:46280) 2023-10-06T10:39:22.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.101+0000 7f63d6d9d700 1 -- 172.21.15.92:0/3026148725 learned_addr learned my addr 172.21.15.92:0/3026148725 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:22.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63cdd9b700 1 --2- 172.21.15.92:0/3026148725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d5ef5259a0 0x55d5ef582b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63d659c700 1 --2- 172.21.15.92:0/3026148725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef523bd0 0x55d5ef568350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63d6d9d700 1 -- 172.21.15.92:0/3026148725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d5ef5259a0 msgr2=0x55d5ef582b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:22.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63d6d9d700 1 --2- 172.21.15.92:0/3026148725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d5ef5259a0 0x55d5ef582b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63d6d9d700 1 -- 172.21.15.92:0/3026148725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef523bd0 msgr2=0x55d5ef568350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:22.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63d6d9d700 1 --2- 172.21.15.92:0/3026148725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef523bd0 0x55d5ef568350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63d6d9d700 1 -- 172.21.15.92:0/3026148725 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5ef2affe0 con 0x55d5ef569a90 2023-10-06T10:39:22.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63cdd9b700 1 --2- 172.21.15.92:0/3026148725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d5ef5259a0 0x55d5ef582b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:22.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63d659c700 1 --2- 172.21.15.92:0/3026148725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef523bd0 0x55d5ef568350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:22.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63d6d9d700 1 --2- 172.21.15.92:0/3026148725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef569a90 0x55d5ef5cf4c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f63d0012000 tx=0x7f63d0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d142e4bdb8667e6a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:22.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:39:22.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63b4ff9700 1 -- 172.21.15.92:0/3026148725 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d0025070 con 0x55d5ef569a90 2023-10-06T10:39:22.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63b4ff9700 1 -- 172.21.15.92:0/3026148725 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f63d0007a50 con 0x55d5ef569a90 2023-10-06T10:39:22.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63e5b98b80 1 -- 172.21.15.92:0/3026148725 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63c0055250 con 0x55d5ef569a90 2023-10-06T10:39:22.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63b4ff9700 1 -- 172.21.15.92:0/3026148725 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63d0007e20 con 0x55d5ef569a90 2023-10-06T10:39:22.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63e5b98b80 10 client.5310 fetch_fsmap learned FSMap version 16 2023-10-06T10:39:22.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63e5b98b80 1 -- 172.21.15.92:0/3026148725 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55d5ef428470 con 0x55d5ef569a90 2023-10-06T10:39:22.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63b4ff9700 1 -- 172.21.15.92:0/3026148725 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c0055250 con 0x55d5ef569a90 2023-10-06T10:39:22.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63b4ff9700 1 -- 172.21.15.92:0/3026148725 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 16) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f63d000b050 con 0x55d5ef569a90 2023-10-06T10:39:22.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63e5b98b80 10 client.5310 fetch_fsmap finished waiting for FSMap version 16 2023-10-06T10:39:22.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63e5b98b80 10 client.5310 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T10:39:22.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63e5b98b80 20 client.5310 populate_metadata read hostname 'smithi092' 2023-10-06T10:39:22.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63e5b98b80 1 --2- 172.21.15.92:0/3026148725 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x55d5ef549c20 0x55d5ef549ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63e5b98b80 4 client.5310 mds_command: new command op to 5133 tid=0[{"prefix": "status"}] 2023-10-06T10:39:22.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.102+0000 7f63e5b98b80 1 -- 172.21.15.92:0/3026148725 --> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d5ef415e00 con 0x55d5ef549c20 2023-10-06T10:39:22.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.103+0000 7f63d659c700 1 --2- 172.21.15.92:0/3026148725 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x55d5ef549c20 0x55d5ef549ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.103+0000 7f63b4ff9700 10 client.5310 ms_handle_connect on v2:172.21.15.124:6832/484869337 2023-10-06T10:39:22.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.103+0000 7f63d659c700 1 --2- 172.21.15.92:0/3026148725 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x55d5ef549c20 0x55d5ef549ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b80d81479d11bf26 server_cookie=4ca5a37c6a8b4cb5 in_seq=0 out_seq=0 2023-10-06T10:39:22.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.103+0000 7f63b4ff9700 1 -- 172.21.15.92:0/3026148725 <== mds.0 v2:172.21.15.124:6832/484869337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55d5ef415e00 con 0x55d5ef549c20 2023-10-06T10:39:22.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.103+0000 7f63b4ff9700 10 client.5310 handle_command_reply: tid=0 2023-10-06T10:39:22.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.103+0000 7f63e5b98b80 1 client.5310 shutdown 2023-10-06T10:39:22.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.103+0000 7f63e5b98b80 1 -- 172.21.15.92:0/3026148725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef569a90 msgr2=0x55d5ef5cf4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:22.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.103+0000 7f63e5b98b80 1 --2- 172.21.15.92:0/3026148725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef569a90 0x55d5ef5cf4c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f63d0012000 tx=0x7f63d0018000 comp rx=0 tx=0).stop 2023-10-06T10:39:22.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.103+0000 7f63e5b98b80 1 -- 172.21.15.92:0/3026148725 shutdown_connections 2023-10-06T10:39:22.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.103+0000 7f63e5b98b80 1 --2- 172.21.15.92:0/3026148725 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x55d5ef549c20 0x55d5ef549ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.103+0000 7f63e5b98b80 1 --2- 172.21.15.92:0/3026148725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5ef569a90 0x55d5ef5cf4c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.103+0000 7f63e5b98b80 1 --2- 172.21.15.92:0/3026148725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d5ef523bd0 0x55d5ef568350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.103+0000 7f63e5b98b80 1 --2- 172.21.15.92:0/3026148725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d5ef5259a0 0x55d5ef582b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.103+0000 7f63b4ff9700 0 client.5310 ms_handle_reset on v2:172.21.15.124:6832/484869337 2023-10-06T10:39:22.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.103+0000 7f63e5b98b80 1 -- 172.21.15.92:0/3026148725 >> 172.21.15.92:0/3026148725 conn(0x55d5ef524070 msgr2=0x55d5ef565ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:22.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.103+0000 7f63e5b98b80 1 -- 172.21.15.92:0/3026148725 shutdown_connections 2023-10-06T10:39:22.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.103+0000 7f63e5b98b80 1 -- 172.21.15.92:0/3026148725 wait complete. 2023-10-06T10:39:22.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.104+0000 7f63e5b98b80 20 client.5310 trim_cache size 0 max 16384 2023-10-06T10:39:22.123 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:39:22.124 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:39:22.124 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T10:39:22.124 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 5133, 2023-10-06T10:39:22.124 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:39:22.124 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:39:22.124 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:39:22.125 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 5.7922209010000003, 2023-10-06T10:39:22.125 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 16, 2023-10-06T10:39:22.125 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 38, 2023-10-06T10:39:22.125 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 38, 2023-10-06T10:39:22.125 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 14.157558559 2023-10-06T10:39:22.126 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:39:22.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.104+0000 7f63dc8a0700 1 -- 172.21.15.92:0/3352308924 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f63b8066fb0 msgr2=0x7f63b8069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:22.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.104+0000 7f63dc8a0700 1 --2- 172.21.15.92:0/3352308924 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f63b8066fb0 0x7f63b8069460 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f63c80037f0 tx=0x7f63c8009040 comp rx=0 tx=0).stop 2023-10-06T10:39:22.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.104+0000 7f63dc8a0700 1 -- 172.21.15.92:0/3352308924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63d80b49a0 msgr2=0x7f63d815bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:22.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.104+0000 7f63dc8a0700 1 --2- 172.21.15.92:0/3352308924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63d80b49a0 0x7f63d815bdd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f63c000e8d0 tx=0x7f63c0008040 comp rx=0 tx=0).stop 2023-10-06T10:39:22.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.104+0000 7f63dc8a0700 1 -- 172.21.15.92:0/3352308924 shutdown_connections 2023-10-06T10:39:22.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.104+0000 7f63dc8a0700 1 --2- 172.21.15.92:0/3352308924 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f63b8066fb0 0x7f63b8069460 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.104+0000 7f63dc8a0700 1 --2- 172.21.15.92:0/3352308924 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63d8159560 0x7f63d80b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.104+0000 7f63dc8a0700 1 --2- 172.21.15.92:0/3352308924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63d80b0760 0x7f63d809b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.104+0000 7f63dc8a0700 1 --2- 172.21.15.92:0/3352308924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63d80b49a0 0x7f63d815bdd0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.104+0000 7f63dc8a0700 1 -- 172.21.15.92:0/3352308924 >> 172.21.15.92:0/3352308924 conn(0x7f63d80997f0 msgr2=0x7f63d8158a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:22.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.104+0000 7f63dc8a0700 1 -- 172.21.15.92:0/3352308924 shutdown_connections 2023-10-06T10:39:22.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.104+0000 7f63dc8a0700 1 -- 172.21.15.92:0/3352308924 wait complete. 2023-10-06T10:39:22.129 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T10:39:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.241+0000 7fe4480ce700 1 Processor -- start 2023-10-06T10:39:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.242+0000 7fe4480ce700 1 -- start start 2023-10-06T10:39:22.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.242+0000 7fe4480ce700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4400b0760 0x7fe4400ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.242+0000 7fe4480ce700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4400b0c60 con 0x7fe440159560 2023-10-06T10:39:22.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.242+0000 7fe4480ce700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4400b0f40 con 0x7fe4400b49a0 2023-10-06T10:39:22.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.242+0000 7fe4480ce700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4400ab350 con 0x7fe4400b0760 2023-10-06T10:39:22.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.243+0000 7fe445e6a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe440159560 0x7fe4400aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60236/0 (socket says 172.21.15.92:60236) 2023-10-06T10:39:22.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.243+0000 7fe445e6a700 1 -- 172.21.15.92:0/1735634893 learned_addr learned my addr 172.21.15.92:0/1735634893 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:22.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.243+0000 7fe44666b700 1 --2- 172.21.15.92:0/1735634893 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4400b0760 0x7fe4400ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.243+0000 7fe444e68700 1 -- 172.21.15.92:0/1735634893 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 547094206 0 0) 0x7fe4400b0c60 con 0x7fe440159560 2023-10-06T10:39:22.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.243+0000 7fe444e68700 1 -- 172.21.15.92:0/1735634893 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe42c003570 con 0x7fe440159560 2023-10-06T10:39:22.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.243+0000 7fe444e68700 1 -- 172.21.15.92:0/1735634893 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1470305698 0 0) 0x7fe4400b0f40 con 0x7fe4400b49a0 2023-10-06T10:39:22.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.243+0000 7fe444e68700 1 -- 172.21.15.92:0/1735634893 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4400b0c60 con 0x7fe4400b49a0 2023-10-06T10:39:22.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.243+0000 7fe44666b700 1 -- 172.21.15.92:0/1735634893 >> v1:172.21.15.124:6789/0 conn(0x7fe4400b49a0 legacy=0x7fe4400aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:22.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.243+0000 7fe44666b700 1 -- 172.21.15.92:0/1735634893 >> v1:172.21.15.92:6789/0 conn(0x7fe440159560 legacy=0x7fe4400aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:22.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.243+0000 7fe44666b700 1 -- 172.21.15.92:0/1735634893 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4400b1220 con 0x7fe4400b0760 2023-10-06T10:39:22.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.243+0000 7fe44666b700 1 --2- 172.21.15.92:0/1735634893 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4400b0760 0x7fe4400ccf90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe438009ec0 tx=0x7fe438009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cf433b5d50d73f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:22.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.243+0000 7fe444e68700 1 -- 172.21.15.92:0/1735634893 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43800e070 con 0x7fe4400b0760 2023-10-06T10:39:22.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.243+0000 7fe444e68700 1 -- 172.21.15.92:0/1735634893 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43800bb40 con 0x7fe4400b0760 2023-10-06T10:39:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.243+0000 7fe444e68700 1 -- 172.21.15.92:0/1735634893 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe438012430 con 0x7fe4400b0760 2023-10-06T10:39:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.244+0000 7fe4480ce700 1 -- 172.21.15.92:0/1735634893 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4400b0760 msgr2=0x7fe4400ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.244+0000 7fe4480ce700 1 --2- 172.21.15.92:0/1735634893 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4400b0760 0x7fe4400ccf90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe438009ec0 tx=0x7fe438009780 comp rx=0 tx=0).stop 2023-10-06T10:39:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.244+0000 7fe4480ce700 1 -- 172.21.15.92:0/1735634893 shutdown_connections 2023-10-06T10:39:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.244+0000 7fe4480ce700 1 --2- 172.21.15.92:0/1735634893 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4400b0760 0x7fe4400ccf90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.244+0000 7fe4480ce700 1 -- 172.21.15.92:0/1735634893 >> 172.21.15.92:0/1735634893 conn(0x7fe4400997f0 msgr2=0x7fe44006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.244+0000 7fe4480ce700 1 -- 172.21.15.92:0/1735634893 shutdown_connections 2023-10-06T10:39:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.244+0000 7fe4480ce700 1 -- 172.21.15.92:0/1735634893 wait complete. 2023-10-06T10:39:22.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.245+0000 7fe4480ce700 1 Processor -- start 2023-10-06T10:39:22.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.245+0000 7fe4480ce700 1 -- start start 2023-10-06T10:39:22.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.246+0000 7fe4480ce700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4400b0760 0x7fe440157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.246+0000 7fe4480ce700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4400b49a0 0x7fe4400abf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.246+0000 7fe4480ce700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe440159560 0x7fe4400b3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.246+0000 7fe4480ce700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4400af410 con 0x7fe4400b0760 2023-10-06T10:39:22.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.246+0000 7fe4480ce700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4400abae0 con 0x7fe440159560 2023-10-06T10:39:22.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.246+0000 7fe4480ce700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe440158180 con 0x7fe4400b49a0 2023-10-06T10:39:22.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.246+0000 7fe445e6a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4400b0760 0x7fe440157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.246+0000 7fe445e6a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4400b0760 0x7fe440157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46292/0 (socket says 172.21.15.92:46292) 2023-10-06T10:39:22.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.246+0000 7fe445e6a700 1 -- 172.21.15.92:0/232502397 learned_addr learned my addr 172.21.15.92:0/232502397 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:22.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.246+0000 7fe44666b700 1 --2- 172.21.15.92:0/232502397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe440159560 0x7fe4400b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.246+0000 7fe445e6a700 1 -- 172.21.15.92:0/232502397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4400b49a0 msgr2=0x7fe4400abf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:22.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.246+0000 7fe445669700 1 --2- 172.21.15.92:0/232502397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4400b49a0 0x7fe4400abf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.246+0000 7fe445e6a700 1 --2- 172.21.15.92:0/232502397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4400b49a0 0x7fe4400abf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.246+0000 7fe445e6a700 1 -- 172.21.15.92:0/232502397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe440159560 msgr2=0x7fe4400b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:22.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.246+0000 7fe445e6a700 1 --2- 172.21.15.92:0/232502397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe440159560 0x7fe4400b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.246+0000 7fe445e6a700 1 -- 172.21.15.92:0/232502397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe438009a20 con 0x7fe4400b0760 2023-10-06T10:39:22.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.246+0000 7fe44666b700 1 --2- 172.21.15.92:0/232502397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe440159560 0x7fe4400b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:22.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.246+0000 7fe445669700 1 --2- 172.21.15.92:0/232502397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4400b49a0 0x7fe4400abf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:22.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.246+0000 7fe445e6a700 1 --2- 172.21.15.92:0/232502397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4400b0760 0x7fe440157c40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00c790 tx=0x7fe43c007ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:22.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.246+0000 7fe435ffb700 1 -- 172.21.15.92:0/232502397 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c0072e0 con 0x7fe4400b0760 2023-10-06T10:39:22.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.247+0000 7fe435ffb700 1 -- 172.21.15.92:0/232502397 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43c007440 con 0x7fe4400b0760 2023-10-06T10:39:22.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.247+0000 7fe4480ce700 1 -- 172.21.15.92:0/232502397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe44009ab60 con 0x7fe4400b0760 2023-10-06T10:39:22.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.247+0000 7fe435ffb700 1 -- 172.21.15.92:0/232502397 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c0057f0 con 0x7fe4400b0760 2023-10-06T10:39:22.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.247+0000 7fe4480ce700 1 -- 172.21.15.92:0/232502397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4400ab490 con 0x7fe4400b0760 2023-10-06T10:39:22.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.248+0000 7fe4480ce700 1 -- 172.21.15.92:0/232502397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4400abdc0 con 0x7fe4400b0760 2023-10-06T10:39:22.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.248+0000 7fe435ffb700 1 -- 172.21.15.92:0/232502397 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe43c005b60 con 0x7fe4400b0760 2023-10-06T10:39:22.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.248+0000 7fe435ffb700 1 --2- 172.21.15.92:0/232502397 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe424066f30 0x7fe4240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:22.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.248+0000 7fe435ffb700 1 -- 172.21.15.92:0/232502397 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe43c083bf0 con 0x7fe4400b0760 2023-10-06T10:39:22.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.248+0000 7fe445669700 1 --2- 172.21.15.92:0/232502397 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe424066f30 0x7fe4240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:22.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.249+0000 7fe445669700 1 --2- 172.21.15.92:0/232502397 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe424066f30 0x7fe4240693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe430007fd0 tx=0x7fe430009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:22.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.252+0000 7fe435ffb700 1 -- 172.21.15.92:0/232502397 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe43c04b5a0 con 0x7fe4400b0760 2023-10-06T10:39:22.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.479+0000 7fe4480ce700 1 -- 172.21.15.92:0/232502397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe440064a90 con 0x7fe4400b0760 2023-10-06T10:39:22.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.479+0000 7fe435ffb700 1 -- 172.21.15.92:0/232502397 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe43c017020 con 0x7fe4400b0760 2023-10-06T10:39:22.480 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T10:39:22.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.481+0000 7fe4480ce700 1 -- 172.21.15.92:0/232502397 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe424066f30 msgr2=0x7fe4240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:22.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.481+0000 7fe4480ce700 1 --2- 172.21.15.92:0/232502397 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe424066f30 0x7fe4240693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe430007fd0 tx=0x7fe430009040 comp rx=0 tx=0).stop 2023-10-06T10:39:22.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.481+0000 7fe4480ce700 1 -- 172.21.15.92:0/232502397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4400b0760 msgr2=0x7fe440157c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:22.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.481+0000 7fe4480ce700 1 --2- 172.21.15.92:0/232502397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4400b0760 0x7fe440157c40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00c790 tx=0x7fe43c007ce0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.481+0000 7fe4480ce700 1 -- 172.21.15.92:0/232502397 shutdown_connections 2023-10-06T10:39:22.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.481+0000 7fe4480ce700 1 --2- 172.21.15.92:0/232502397 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe424066f30 0x7fe4240693e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.481+0000 7fe4480ce700 1 --2- 172.21.15.92:0/232502397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe440159560 0x7fe4400b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.481+0000 7fe4480ce700 1 --2- 172.21.15.92:0/232502397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4400b0760 0x7fe440157c40 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.481+0000 7fe4480ce700 1 --2- 172.21.15.92:0/232502397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4400b49a0 0x7fe4400abf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:22.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.481+0000 7fe4480ce700 1 -- 172.21.15.92:0/232502397 >> 172.21.15.92:0/232502397 conn(0x7fe4400997f0 msgr2=0x7fe440158a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:22.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.481+0000 7fe4480ce700 1 -- 172.21.15.92:0/232502397 shutdown_connections 2023-10-06T10:39:22.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:22.481+0000 7fe4480ce700 1 -- 172.21.15.92:0/232502397 wait complete. 2023-10-06T10:39:22.494 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T10:39:22.494 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T10:39:22.494 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T10:39:22.494 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T10:39:22.494 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T10:39:22.494 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T10:39:22.494 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T10:39:22.495 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T10:39:22.495 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T10:39:22.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:39:22.495 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T10:39:22.510 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T10:39:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T10:39:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T10:39:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:39:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T10:39:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:39:22.511 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:39:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:39:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T10:39:22.511 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:39:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:39:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T10:39:22.511 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:39:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:39:22.511 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T10:39:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:39:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T10:39:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42447sec preferred_lft 42447sec 2023-10-06T10:39:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T10:39:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:39:22.512 INFO:teuthology.orchestra.run.smithi092.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T10:39:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 92:7f:88:24:4e:ca brd ff:ff:ff:ff:ff:ff 2023-10-06T10:39:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T10:39:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:39:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::9c9d:23ff:fee4:30ae/64 scope link 2023-10-06T10:39:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:39:22.513 INFO:teuthology.orchestra.run.smithi092.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T10:39:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 92:7f:88:24:4e:ca brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T10:39:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::907f:88ff:fe24:4eca/64 scope link 2023-10-06T10:39:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:39:22.513 INFO:teuthology.orchestra.run.smithi092.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T10:39:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether b6:32:f9:d9:96:1b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T10:39:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::b432:f9ff:fed9:961b/64 scope link 2023-10-06T10:39:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:39:22.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:39:22.514 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T10:39:22.573 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T10:39:22.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:39:22.575 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T10:39:22.631 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T10:39:22.631 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T10:39:22.631 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T10:39:22.631 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T10:39:22.631 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T10:39:22.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:39:22.632 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T10:39:22.632 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T10:39:22.632 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T10:39:22.633 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T10:39:22.633 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T10:39:22.633 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T10:39:22.728 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:39:22.731 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:39:22.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:39:22.733 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T10:39:22.788 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T10:39:22.788 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T10:39:22.789 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T10:39:22.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:39:22.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-10-06T10:39:22.855 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T10:39:22.855 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T10:39:22.910 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T10:39:22.910 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T10:39:22.910 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T10:39:22.965 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T10:39:22.965 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T10:39:22.965 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:39:23.045 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:39:23.045 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:39:23.063 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:39:23.063 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:39:23.064 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-06T10:39:23.064 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@8f307bfb-c4cb-4f66-aade-46fffe0c6e0c.cephfs=/ 2023-10-06T10:39:23.064 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-06T10:39:23.064 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T10:39:23.064 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T10:39:23.064 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T10:39:23.087 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:39:23.087 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:39:23.088 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-06T10:39:23.088 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@8f307bfb-c4cb-4f66-aade-46fffe0c6e0c.cephfs=/ 2023-10-06T10:39:23.088 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-06T10:39:23.088 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T10:39:23.088 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T10:39:23.088 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T10:39:23.088 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.124:6789,172.21.15.124:6790" 2023-10-06T10:39:23.088 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.124:6789,172.21.15.124:6790:/ 2023-10-06T10:39:23.088 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=8f307bfb-c4cb-4f66-aade-46fffe0c6e0c" will pass to kernel 2023-10-06T10:39:23.089 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T10:39:23.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:39:23.090 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T10:39:23.155 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-10-06T10:39:23.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.312+0000 7fd263fff700 1 Processor -- start 2023-10-06T10:39:23.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.312+0000 7fd263fff700 1 -- start start 2023-10-06T10:39:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.313+0000 7fd263fff700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd2640b2620 0x7fd2640cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.313+0000 7fd263fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2640cc970 con 0x7fd2640b29f0 2023-10-06T10:39:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.313+0000 7fd263fff700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2640cbd60 con 0x7fd2640cf1b0 2023-10-06T10:39:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.313+0000 7fd263fff700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2640cc040 con 0x7fd2640b2620 2023-10-06T10:39:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.313+0000 7fd25a7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd2640b29f0 0x7fd2640ceaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60240/0 (socket says 172.21.15.92:60240) 2023-10-06T10:39:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.313+0000 7fd25a7fc700 1 -- 172.21.15.92:0/2632439089 learned_addr learned my addr 172.21.15.92:0/2632439089 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.313+0000 7fd262ffd700 1 --2- 172.21.15.92:0/2632439089 >> v2:172.21.15.124:3300/0 conn(0x7fd2640b2620 0x7fd2640cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.313+0000 7fd2627fc700 1 -- 172.21.15.92:0/2632439089 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1630865326 0 0) 0x7fd2640cc970 con 0x7fd2640b29f0 2023-10-06T10:39:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.313+0000 7fd2627fc700 1 -- 172.21.15.92:0/2632439089 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd250003570 con 0x7fd2640b29f0 2023-10-06T10:39:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.314+0000 7fd2627fc700 1 -- 172.21.15.92:0/2632439089 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 41353043 0 0) 0x7fd2640cbd60 con 0x7fd2640cf1b0 2023-10-06T10:39:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.314+0000 7fd2627fc700 1 -- 172.21.15.92:0/2632439089 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2640cc970 con 0x7fd2640cf1b0 2023-10-06T10:39:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.314+0000 7fd2627fc700 1 -- 172.21.15.92:0/2632439089 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1041779411 0 0) 0x7fd250003570 con 0x7fd2640b29f0 2023-10-06T10:39:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.314+0000 7fd262ffd700 1 -- 172.21.15.92:0/2632439089 >> v1:172.21.15.124:6789/0 conn(0x7fd2640cf1b0 legacy=0x7fd2640c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.314+0000 7fd262ffd700 1 -- 172.21.15.92:0/2632439089 >> v1:172.21.15.92:6789/0 conn(0x7fd2640b29f0 legacy=0x7fd2640ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.314+0000 7fd262ffd700 1 -- 172.21.15.92:0/2632439089 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2640cc320 con 0x7fd2640b2620 2023-10-06T10:39:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.314+0000 7fd262ffd700 1 --2- 172.21.15.92:0/2632439089 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2640b2620 0x7fd2640cccb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd2540098d0 tx=0x7fd254009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb3d2094475aea36 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.314+0000 7fd2627fc700 1 -- 172.21.15.92:0/2632439089 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c001470 con 0x7fd2640b2620 2023-10-06T10:39:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.314+0000 7fd2627fc700 1 -- 172.21.15.92:0/2632439089 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd254011420 con 0x7fd2640b2620 2023-10-06T10:39:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.314+0000 7fd2627fc700 1 -- 172.21.15.92:0/2632439089 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2540117c0 con 0x7fd2640b2620 2023-10-06T10:39:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.314+0000 7fd263fff700 1 -- 172.21.15.92:0/2632439089 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2640b2620 msgr2=0x7fd2640cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.314+0000 7fd263fff700 1 --2- 172.21.15.92:0/2632439089 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2640b2620 0x7fd2640cccb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd2540098d0 tx=0x7fd254009120 comp rx=0 tx=0).stop 2023-10-06T10:39:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.315+0000 7fd263fff700 1 -- 172.21.15.92:0/2632439089 shutdown_connections 2023-10-06T10:39:23.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.315+0000 7fd263fff700 1 --2- 172.21.15.92:0/2632439089 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2640b2620 0x7fd2640cccb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:23.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.315+0000 7fd263fff700 1 -- 172.21.15.92:0/2632439089 >> 172.21.15.92:0/2632439089 conn(0x7fd264099520 msgr2=0x7fd26406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:23.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.315+0000 7fd263fff700 1 -- 172.21.15.92:0/2632439089 shutdown_connections 2023-10-06T10:39:23.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.315+0000 7fd263fff700 1 -- 172.21.15.92:0/2632439089 wait complete. 2023-10-06T10:39:23.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.316+0000 7fd263fff700 1 Processor -- start 2023-10-06T10:39:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.316+0000 7fd263fff700 1 -- start start 2023-10-06T10:39:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.316+0000 7fd263fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2640b2620 0x7fd26416d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.316+0000 7fd263fff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2640b29f0 0x7fd26415cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.316+0000 7fd263fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2640cf1b0 0x7fd264157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.316+0000 7fd263fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26415da00 con 0x7fd2640cf1b0 2023-10-06T10:39:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.316+0000 7fd263fff700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26415b4b0 con 0x7fd2640b29f0 2023-10-06T10:39:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.316+0000 7fd263fff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26415d4a0 con 0x7fd2640b2620 2023-10-06T10:39:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.317+0000 7fd2637fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2640cf1b0 0x7fd264157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.317+0000 7fd262ffd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2640b2620 0x7fd26416d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.317+0000 7fd2637fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2640cf1b0 0x7fd264157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46300/0 (socket says 172.21.15.92:46300) 2023-10-06T10:39:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.317+0000 7fd25a7fc700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2640b29f0 0x7fd26415cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.317+0000 7fd2637fe700 1 -- 172.21.15.92:0/910898659 learned_addr learned my addr 172.21.15.92:0/910898659 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.317+0000 7fd2637fe700 1 -- 172.21.15.92:0/910898659 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2640b2620 msgr2=0x7fd26416d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.317+0000 7fd2637fe700 1 --2- 172.21.15.92:0/910898659 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2640b2620 0x7fd26416d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.317+0000 7fd2637fe700 1 -- 172.21.15.92:0/910898659 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2640b29f0 msgr2=0x7fd26415cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.317+0000 7fd2637fe700 1 --2- 172.21.15.92:0/910898659 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2640b29f0 0x7fd26415cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.317+0000 7fd2637fe700 1 -- 172.21.15.92:0/910898659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2540093c0 con 0x7fd2640cf1b0 2023-10-06T10:39:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.317+0000 7fd2637fe700 1 --2- 172.21.15.92:0/910898659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2640cf1b0 0x7fd264157d40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00c940 tx=0x7fd25c0027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.317+0000 7fd25b7fe700 1 -- 172.21.15.92:0/910898659 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c012630 con 0x7fd2640cf1b0 2023-10-06T10:39:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.317+0000 7fd25b7fe700 1 -- 172.21.15.92:0/910898659 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25c0127b0 con 0x7fd2640cf1b0 2023-10-06T10:39:23.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.317+0000 7fd263fff700 1 -- 172.21.15.92:0/910898659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd26415d720 con 0x7fd2640cf1b0 2023-10-06T10:39:23.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.317+0000 7fd25b7fe700 1 -- 172.21.15.92:0/910898659 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c012b70 con 0x7fd2640cf1b0 2023-10-06T10:39:23.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.317+0000 7fd263fff700 1 -- 172.21.15.92:0/910898659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd26415b140 con 0x7fd2640cf1b0 2023-10-06T10:39:23.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.318+0000 7fd263fff700 1 -- 172.21.15.92:0/910898659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd26405f230 con 0x7fd2640cf1b0 2023-10-06T10:39:23.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.319+0000 7fd25b7fe700 1 -- 172.21.15.92:0/910898659 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd25c01b400 con 0x7fd2640cf1b0 2023-10-06T10:39:23.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.319+0000 7fd25b7fe700 1 --2- 172.21.15.92:0/910898659 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd244066f30 0x7fd2440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:23.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.319+0000 7fd25b7fe700 1 -- 172.21.15.92:0/910898659 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fd25c084b30 con 0x7fd2640cf1b0 2023-10-06T10:39:23.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.319+0000 7fd262ffd700 1 --2- 172.21.15.92:0/910898659 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd244066f30 0x7fd2440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:23.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.319+0000 7fd262ffd700 1 --2- 172.21.15.92:0/910898659 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd244066f30 0x7fd2440693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd254009390 tx=0x7fd254016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:23.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.323+0000 7fd25b7fe700 1 -- 172.21.15.92:0/910898659 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd25c051b00 con 0x7fd2640cf1b0 2023-10-06T10:39:23.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:23.551+0000 7fd263fff700 1 -- 172.21.15.92:0/910898659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7fd2640cc600 con 0x7fd2640cf1b0 2023-10-06T10:39:24.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.342+0000 7fd25b7fe700 1 -- 172.21.15.92:0/910898659 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v17) v1 ==== 105+0+0 (secure 0 0 0) 0x7fd25c055120 con 0x7fd2640cf1b0 2023-10-06T10:39:24.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.344+0000 7fd263fff700 1 -- 172.21.15.92:0/910898659 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd244066f30 msgr2=0x7fd2440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:24.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.344+0000 7fd263fff700 1 --2- 172.21.15.92:0/910898659 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd244066f30 0x7fd2440693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd254009390 tx=0x7fd254016000 comp rx=0 tx=0).stop 2023-10-06T10:39:24.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.344+0000 7fd263fff700 1 -- 172.21.15.92:0/910898659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2640cf1b0 msgr2=0x7fd264157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:24.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.344+0000 7fd263fff700 1 --2- 172.21.15.92:0/910898659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2640cf1b0 0x7fd264157d40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00c940 tx=0x7fd25c0027a0 comp rx=0 tx=0).stop 2023-10-06T10:39:24.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.344+0000 7fd263fff700 1 -- 172.21.15.92:0/910898659 shutdown_connections 2023-10-06T10:39:24.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.344+0000 7fd263fff700 1 --2- 172.21.15.92:0/910898659 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd244066f30 0x7fd2440693e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:24.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.344+0000 7fd263fff700 1 --2- 172.21.15.92:0/910898659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2640cf1b0 0x7fd264157d40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:24.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.344+0000 7fd263fff700 1 --2- 172.21.15.92:0/910898659 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2640b2620 0x7fd26416d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:24.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.344+0000 7fd263fff700 1 --2- 172.21.15.92:0/910898659 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2640b29f0 0x7fd26415cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:24.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.344+0000 7fd263fff700 1 -- 172.21.15.92:0/910898659 >> 172.21.15.92:0/910898659 conn(0x7fd264099520 msgr2=0x7fd2640b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:24.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.344+0000 7fd263fff700 1 -- 172.21.15.92:0/910898659 shutdown_connections 2023-10-06T10:39:24.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.344+0000 7fd263fff700 1 -- 172.21.15.92:0/910898659 wait complete. 2023-10-06T10:39:24.364 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:39:24.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.479+0000 7f202a9e0700 1 Processor -- start 2023-10-06T10:39:24.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.479+0000 7f202a9e0700 1 -- start start 2023-10-06T10:39:24.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.480+0000 7f202a9e0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2024159440 0x7f20240b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:24.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.480+0000 7f202a9e0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20240b35d0 0x7f20240cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:24.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.480+0000 7f202a9e0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20240cb7a0 con 0x7f20240cc170 2023-10-06T10:39:24.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.480+0000 7f202a9e0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20240b0860 con 0x7f2024159440 2023-10-06T10:39:24.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.480+0000 7f202a9e0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20240cb150 con 0x7f20240b35d0 2023-10-06T10:39:24.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.480+0000 7f20299de700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2024159440 0x7f20240b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:24.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.480+0000 7f202a1df700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f20240cc170 0x7f20240b2100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35324/0 (socket says 172.21.15.92:35324) 2023-10-06T10:39:24.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.480+0000 7f202a1df700 1 -- 172.21.15.92:0/2061915382 learned_addr learned my addr 172.21.15.92:0/2061915382 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:24.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.480+0000 7f20299de700 1 -- 172.21.15.92:0/2061915382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20240b35d0 msgr2=0x7f20240cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:24.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.480+0000 7f20299de700 1 --2- 172.21.15.92:0/2061915382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20240b35d0 0x7f20240cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:24.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.480+0000 7f20299de700 1 -- 172.21.15.92:0/2061915382 >> v1:172.21.15.92:6789/0 conn(0x7f20240cc170 legacy=0x7f20240b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:24.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.480+0000 7f20299de700 1 -- 172.21.15.92:0/2061915382 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20240cb430 con 0x7f2024159440 2023-10-06T10:39:24.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.480+0000 7f20289dc700 1 -- 172.21.15.92:0/2061915382 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1886779697 0 0) 0x7f20240cb7a0 con 0x7f20240cc170 2023-10-06T10:39:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.481+0000 7f20299de700 1 --2- 172.21.15.92:0/2061915382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2024159440 0x7f20240b3090 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f20200098e0 tx=0x7f20200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1eed523264f4a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.481+0000 7f20289dc700 1 -- 172.21.15.92:0/2061915382 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020009e80 con 0x7f2024159440 2023-10-06T10:39:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.481+0000 7f20289dc700 1 -- 172.21.15.92:0/2061915382 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202000b860 con 0x7f2024159440 2023-10-06T10:39:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.481+0000 7f20289dc700 1 -- 172.21.15.92:0/2061915382 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202000bbe0 con 0x7f2024159440 2023-10-06T10:39:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.481+0000 7f202a9e0700 1 -- 172.21.15.92:0/2061915382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2024159440 msgr2=0x7f20240b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.481+0000 7f202a9e0700 1 --2- 172.21.15.92:0/2061915382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2024159440 0x7f20240b3090 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f20200098e0 tx=0x7f20200091a0 comp rx=0 tx=0).stop 2023-10-06T10:39:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.481+0000 7f202a9e0700 1 -- 172.21.15.92:0/2061915382 shutdown_connections 2023-10-06T10:39:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.481+0000 7f202a9e0700 1 --2- 172.21.15.92:0/2061915382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2024159440 0x7f20240b3090 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.481+0000 7f202a9e0700 1 --2- 172.21.15.92:0/2061915382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20240b35d0 0x7f20240cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.481+0000 7f202a9e0700 1 -- 172.21.15.92:0/2061915382 >> 172.21.15.92:0/2061915382 conn(0x7f202406ece0 msgr2=0x7f202406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.481+0000 7f202a9e0700 1 -- 172.21.15.92:0/2061915382 shutdown_connections 2023-10-06T10:39:24.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.481+0000 7f202a9e0700 1 -- 172.21.15.92:0/2061915382 wait complete. 2023-10-06T10:39:24.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.482+0000 7f202a9e0700 1 Processor -- start 2023-10-06T10:39:24.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.482+0000 7f202a9e0700 1 -- start start 2023-10-06T10:39:24.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.483+0000 7f202a9e0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20240b35d0 0x7f202416a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:24.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.483+0000 7f202a9e0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20240cc170 0x7f2024169c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:24.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.483+0000 7f202a9e0700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2024159440 0x7f20240c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:24.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.483+0000 7f202a9e0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2024159a90 con 0x7f20240cc170 2023-10-06T10:39:24.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.483+0000 7f202a9e0700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202415a330 con 0x7f2024159440 2023-10-06T10:39:24.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.483+0000 7f202a9e0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202415ca40 con 0x7f20240b35d0 2023-10-06T10:39:24.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.483+0000 7f20291dd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20240cc170 0x7f2024169c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:24.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.483+0000 7f20291dd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20240cc170 0x7f2024169c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47814/0 (socket says 172.21.15.92:47814) 2023-10-06T10:39:24.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.483+0000 7f20291dd700 1 -- 172.21.15.92:0/522582569 learned_addr learned my addr 172.21.15.92:0/522582569 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:24.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.484+0000 7f20299de700 1 --2- 172.21.15.92:0/522582569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20240b35d0 0x7f202416a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:24.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.484+0000 7f20291dd700 1 -- 172.21.15.92:0/522582569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20240b35d0 msgr2=0x7f202416a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:24.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.484+0000 7f202a1df700 1 --2- 172.21.15.92:0/522582569 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2024159440 0x7f20240c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:24.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.484+0000 7f20291dd700 1 --2- 172.21.15.92:0/522582569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20240b35d0 0x7f202416a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:24.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.484+0000 7f20291dd700 1 -- 172.21.15.92:0/522582569 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2024159440 msgr2=0x7f20240c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:24.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.484+0000 7f20291dd700 1 --2- 172.21.15.92:0/522582569 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2024159440 0x7f20240c89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:24.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.484+0000 7f20291dd700 1 -- 172.21.15.92:0/522582569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2020009420 con 0x7f20240cc170 2023-10-06T10:39:24.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.484+0000 7f20291dd700 1 --2- 172.21.15.92:0/522582569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20240cc170 0x7f2024169c20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f201400b810 tx=0x7f2014009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:24.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.484+0000 7f2019ffb700 1 -- 172.21.15.92:0/522582569 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201400f5f0 con 0x7f20240cc170 2023-10-06T10:39:24.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.484+0000 7f2019ffb700 1 -- 172.21.15.92:0/522582569 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201400f750 con 0x7f20240cc170 2023-10-06T10:39:24.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.484+0000 7f202a9e0700 1 -- 172.21.15.92:0/522582569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202416a160 con 0x7f20240cc170 2023-10-06T10:39:24.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.484+0000 7f2019ffb700 1 -- 172.21.15.92:0/522582569 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201400fb10 con 0x7f20240cc170 2023-10-06T10:39:24.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.484+0000 7f202a9e0700 1 -- 172.21.15.92:0/522582569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2024159ce0 con 0x7f20240cc170 2023-10-06T10:39:24.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.485+0000 7f202a9e0700 1 -- 172.21.15.92:0/522582569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f202415a4a0 con 0x7f20240cc170 2023-10-06T10:39:24.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.485+0000 7f2019ffb700 1 -- 172.21.15.92:0/522582569 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2014010480 con 0x7f20240cc170 2023-10-06T10:39:24.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.485+0000 7f2019ffb700 1 --2- 172.21.15.92:0/522582569 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2008066f30 0x7f20080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:24.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.486+0000 7f2019ffb700 1 -- 172.21.15.92:0/522582569 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2014084680 con 0x7f20240cc170 2023-10-06T10:39:24.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.486+0000 7f20299de700 1 --2- 172.21.15.92:0/522582569 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2008066f30 0x7f20080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:24.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.486+0000 7f20299de700 1 --2- 172.21.15.92:0/522582569 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2008066f30 0x7f20080693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f202416a570 tx=0x7f2020017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:24.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.490+0000 7f2019ffb700 1 -- 172.21.15.92:0/522582569 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f20140516d0 con 0x7f20240cc170 2023-10-06T10:39:24.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.715+0000 7f202a9e0700 1 -- 172.21.15.92:0/522582569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f202405f230 con 0x7f20240cc170 2023-10-06T10:39:24.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.715+0000 7f2019ffb700 1 -- 172.21.15.92:0/522582569 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4616 (secure 0 0 0) 0x7f2014054cf0 con 0x7f20240cc170 2023-10-06T10:39:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:39:24.717 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2525196781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2525196781},{"type":"v1","addr":"172.21.15.124:6835","nonce":2525196781}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5118,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2409440810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2409440810},{"type":"v1","addr":"172.21.15.92:6836","nonce":2409440810}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:39:15.675013+0000","modified":"2023-10-06T10:39:24.343636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":5133,"mds_1":5121},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5121":{"gid":5121,"name":"c","rank":1,"incarnation":18,"state":"up:creating","state_seq":1,"addr":"172.21.15.92:6837/4010629505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4010629505},{"type":"v1","addr":"172.21.15.92:6837","nonce":4010629505}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5133":{"gid":5133,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.124:6834/484869337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":484869337},{"type":"v1","addr":"172.21.15.124:6834","nonce":484869337}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T10:39:24.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.717+0000 7f202a9e0700 1 -- 172.21.15.92:0/522582569 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2008066f30 msgr2=0x7f20080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:24.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.717+0000 7f202a9e0700 1 --2- 172.21.15.92:0/522582569 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2008066f30 0x7f20080693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f202416a570 tx=0x7f2020017000 comp rx=0 tx=0).stop 2023-10-06T10:39:24.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.717+0000 7f202a9e0700 1 -- 172.21.15.92:0/522582569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20240cc170 msgr2=0x7f2024169c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:24.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.717+0000 7f202a9e0700 1 --2- 172.21.15.92:0/522582569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20240cc170 0x7f2024169c20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f201400b810 tx=0x7f2014009250 comp rx=0 tx=0).stop 2023-10-06T10:39:24.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.717+0000 7f202a9e0700 1 -- 172.21.15.92:0/522582569 shutdown_connections 2023-10-06T10:39:24.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.717+0000 7f202a9e0700 1 --2- 172.21.15.92:0/522582569 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2008066f30 0x7f20080693e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:24.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.717+0000 7f202a9e0700 1 --2- 172.21.15.92:0/522582569 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2024159440 0x7f20240c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:24.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.717+0000 7f202a9e0700 1 --2- 172.21.15.92:0/522582569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20240b35d0 0x7f202416a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:24.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.717+0000 7f202a9e0700 1 --2- 172.21.15.92:0/522582569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20240cc170 0x7f2024169c20 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:24.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.717+0000 7f202a9e0700 1 -- 172.21.15.92:0/522582569 >> 172.21.15.92:0/522582569 conn(0x7f202406ece0 msgr2=0x7f20241585b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:24.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.717+0000 7f202a9e0700 1 -- 172.21.15.92:0/522582569 shutdown_connections 2023-10-06T10:39:24.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:24.717+0000 7f202a9e0700 1 -- 172.21.15.92:0/522582569 wait complete. 2023-10-06T10:39:24.721 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 18 2023-10-06T10:39:25.133 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T10:39:15.675013+0000', 'modified': '2023-10-06T10:39:24.343636+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 5133, 'mds_1': 5121}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5121': {'gid': 5121, 'name': 'c', 'rank': 1, 'incarnation': 18, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.92:6837/4010629505', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 4010629505}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 4010629505}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_5133': {'gid': 5133, 'name': 'b', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.124:6834/484869337', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6832', 'nonce': 484869337}, {'type': 'v1', 'addr': '172.21.15.124:6834', 'nonce': 484869337}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T10:39:25.133 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5121:up:creating 2023-10-06T10:39:26.134 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:39:26.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.256+0000 7f838f8d3700 1 Processor -- start 2023-10-06T10:39:26.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.257+0000 7f838f8d3700 1 -- start start 2023-10-06T10:39:26.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.257+0000 7f838f8d3700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f83880b25e0 0x7f83880b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:26.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.257+0000 7f838f8d3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f83880b29b0 0x7f8388159120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:26.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.257+0000 7f838f8d3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83880cbfa0 con 0x7f8388159660 2023-10-06T10:39:26.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.257+0000 7f838f8d3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83880cb390 con 0x7f83880b29b0 2023-10-06T10:39:26.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.257+0000 7f838f8d3700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83880cb670 con 0x7f83880b25e0 2023-10-06T10:39:26.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.258+0000 7f838ce6e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f83880b29b0 0x7f8388159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:26.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.258+0000 7f838de70700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8388159660 0x7f8388068540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35330/0 (socket says 172.21.15.92:35330) 2023-10-06T10:39:26.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.258+0000 7f838de70700 1 -- 172.21.15.92:0/528515019 learned_addr learned my addr 172.21.15.92:0/528515019 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:26.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.258+0000 7f838d66f700 1 --2- 172.21.15.92:0/528515019 >> v2:172.21.15.124:3300/0 conn(0x7f83880b25e0 0x7f83880b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:26.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.258+0000 7f838ce6e700 1 -- 172.21.15.92:0/528515019 >> v2:172.21.15.124:3300/0 conn(0x7f83880b25e0 msgr2=0x7f83880b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:26.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.258+0000 7f838ce6e700 1 --2- 172.21.15.92:0/528515019 >> v2:172.21.15.124:3300/0 conn(0x7f83880b25e0 0x7f83880b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:26.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.258+0000 7f838ce6e700 1 -- 172.21.15.92:0/528515019 >> v1:172.21.15.92:6789/0 conn(0x7f8388159660 legacy=0x7f8388068540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:26.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.258+0000 7f838ce6e700 1 -- 172.21.15.92:0/528515019 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83880cb950 con 0x7f83880b29b0 2023-10-06T10:39:26.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.258+0000 7f837ffff700 1 -- 172.21.15.92:0/528515019 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1301845143 0 0) 0x7f83880cbfa0 con 0x7f8388159660 2023-10-06T10:39:26.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.258+0000 7f838d66f700 1 --2- 172.21.15.92:0/528515019 >> v2:172.21.15.124:3300/0 conn(0x7f83880b25e0 0x7f83880b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:26.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.258+0000 7f838ce6e700 1 --2- 172.21.15.92:0/528515019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83880b29b0 0x7f8388159120 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f83880d0420 tx=0x7f83780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f6f1c1d71ba474 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:26.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.258+0000 7f837ffff700 1 -- 172.21.15.92:0/528515019 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837800b6b0 con 0x7f83880b29b0 2023-10-06T10:39:26.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.258+0000 7f837ffff700 1 -- 172.21.15.92:0/528515019 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837800b810 con 0x7f83880b29b0 2023-10-06T10:39:26.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.258+0000 7f837ffff700 1 -- 172.21.15.92:0/528515019 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837800bbb0 con 0x7f83880b29b0 2023-10-06T10:39:26.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.259+0000 7f838f8d3700 1 -- 172.21.15.92:0/528515019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83880b29b0 msgr2=0x7f8388159120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:26.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.259+0000 7f838f8d3700 1 --2- 172.21.15.92:0/528515019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83880b29b0 0x7f8388159120 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f83880d0420 tx=0x7f83780091a0 comp rx=0 tx=0).stop 2023-10-06T10:39:26.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.259+0000 7f838f8d3700 1 -- 172.21.15.92:0/528515019 shutdown_connections 2023-10-06T10:39:26.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.259+0000 7f838f8d3700 1 --2- 172.21.15.92:0/528515019 >> v2:172.21.15.124:3300/0 conn(0x7f83880b25e0 0x7f83880b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:26.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.259+0000 7f838f8d3700 1 --2- 172.21.15.92:0/528515019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83880b29b0 0x7f8388159120 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:26.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.259+0000 7f838f8d3700 1 -- 172.21.15.92:0/528515019 >> 172.21.15.92:0/528515019 conn(0x7f838806ece0 msgr2=0x7f838806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:26.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.259+0000 7f838f8d3700 1 -- 172.21.15.92:0/528515019 shutdown_connections 2023-10-06T10:39:26.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.259+0000 7f838f8d3700 1 -- 172.21.15.92:0/528515019 wait complete. 2023-10-06T10:39:26.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.260+0000 7f838f8d3700 1 Processor -- start 2023-10-06T10:39:26.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.260+0000 7f838f8d3700 1 -- start start 2023-10-06T10:39:26.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.261+0000 7f838f8d3700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83880b25e0 0x7f83880c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:26.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.261+0000 7f838f8d3700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f83880b29b0 0x7f83880c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:26.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.261+0000 7f838f8d3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388159660 0x7f838815c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:26.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.261+0000 7f838f8d3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83880c8ba0 con 0x7f8388159660 2023-10-06T10:39:26.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.261+0000 7f838f8d3700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83880c7cc0 con 0x7f83880b29b0 2023-10-06T10:39:26.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.261+0000 7f838f8d3700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83880c7f90 con 0x7f83880b25e0 2023-10-06T10:39:26.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.261+0000 7f838de70700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388159660 0x7f838815c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:26.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.261+0000 7f838de70700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388159660 0x7f838815c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47834/0 (socket says 172.21.15.92:47834) 2023-10-06T10:39:26.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.261+0000 7f838de70700 1 -- 172.21.15.92:0/1369954121 learned_addr learned my addr 172.21.15.92:0/1369954121 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:26.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.261+0000 7f838ce6e700 1 --2- 172.21.15.92:0/1369954121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f83880b29b0 0x7f83880c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:26.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.261+0000 7f838de70700 1 -- 172.21.15.92:0/1369954121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83880b25e0 msgr2=0x7f83880c4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:26.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.261+0000 7f838de70700 1 --2- 172.21.15.92:0/1369954121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83880b25e0 0x7f83880c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:26.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.261+0000 7f838de70700 1 -- 172.21.15.92:0/1369954121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f83880b29b0 msgr2=0x7f83880c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:26.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.261+0000 7f838de70700 1 --2- 172.21.15.92:0/1369954121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f83880b29b0 0x7f83880c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:26.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.261+0000 7f838de70700 1 -- 172.21.15.92:0/1369954121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8378009420 con 0x7f8388159660 2023-10-06T10:39:26.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.261+0000 7f838ce6e700 1 --2- 172.21.15.92:0/1369954121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f83880b29b0 0x7f83880c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:26.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.261+0000 7f838de70700 1 --2- 172.21.15.92:0/1369954121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388159660 0x7f838815c2e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f838000c4f0 tx=0x7f8380007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:26.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.261+0000 7f837d7fa700 1 -- 172.21.15.92:0/1369954121 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380005b30 con 0x7f8388159660 2023-10-06T10:39:26.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.261+0000 7f837d7fa700 1 -- 172.21.15.92:0/1369954121 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8380005cb0 con 0x7f8388159660 2023-10-06T10:39:26.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.261+0000 7f838f8d3700 1 -- 172.21.15.92:0/1369954121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83880c8270 con 0x7f8388159660 2023-10-06T10:39:26.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.261+0000 7f837d7fa700 1 -- 172.21.15.92:0/1369954121 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380007320 con 0x7f8388159660 2023-10-06T10:39:26.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.262+0000 7f838f8d3700 1 -- 172.21.15.92:0/1369954121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83880c8830 con 0x7f8388159660 2023-10-06T10:39:26.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.263+0000 7f838f8d3700 1 -- 172.21.15.92:0/1369954121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83880c8e80 con 0x7f8388159660 2023-10-06T10:39:26.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.263+0000 7f837d7fa700 1 -- 172.21.15.92:0/1369954121 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f838001b3b0 con 0x7f8388159660 2023-10-06T10:39:26.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.263+0000 7f837d7fa700 1 --2- 172.21.15.92:0/1369954121 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f836c066f30 0x7f836c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:26.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.263+0000 7f837d7fa700 1 -- 172.21.15.92:0/1369954121 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8380083ca0 con 0x7f8388159660 2023-10-06T10:39:26.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.263+0000 7f838d66f700 1 --2- 172.21.15.92:0/1369954121 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f836c066f30 0x7f836c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:26.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.264+0000 7f838d66f700 1 --2- 172.21.15.92:0/1369954121 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f836c066f30 0x7f836c0693e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f83840094d0 tx=0x7f8384009700 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:26.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.267+0000 7f837d7fa700 1 -- 172.21.15.92:0/1369954121 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8380050c70 con 0x7f8388159660 2023-10-06T10:39:26.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.495+0000 7f838f8d3700 1 -- 172.21.15.92:0/1369954121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83880c8ce0 con 0x7f8388159660 2023-10-06T10:39:26.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.496+0000 7f837d7fa700 1 -- 172.21.15.92:0/1369954121 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4628 (secure 0 0 0) 0x7f83880c8ce0 con 0x7f8388159660 2023-10-06T10:39:26.498 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:39:26.498 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2525196781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2525196781},{"type":"v1","addr":"172.21.15.124:6835","nonce":2525196781}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:39:15.675013+0000","modified":"2023-10-06T10:39:26.352599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5133,"mds_1":5121,"mds_2":5118},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5118":{"gid":5118,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6836/2409440810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2409440810},{"type":"v1","addr":"172.21.15.92:6836","nonce":2409440810}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5121":{"gid":5121,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/4010629505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4010629505},{"type":"v1","addr":"172.21.15.92:6837","nonce":4010629505}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5133":{"gid":5133,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.124:6834/484869337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":484869337},{"type":"v1","addr":"172.21.15.124:6834","nonce":484869337}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T10:39:26.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.498+0000 7f838f8d3700 1 -- 172.21.15.92:0/1369954121 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f836c066f30 msgr2=0x7f836c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:26.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.498+0000 7f838f8d3700 1 --2- 172.21.15.92:0/1369954121 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f836c066f30 0x7f836c0693e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f83840094d0 tx=0x7f8384009700 comp rx=0 tx=0).stop 2023-10-06T10:39:26.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.498+0000 7f838f8d3700 1 -- 172.21.15.92:0/1369954121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388159660 msgr2=0x7f838815c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:26.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.498+0000 7f838f8d3700 1 --2- 172.21.15.92:0/1369954121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388159660 0x7f838815c2e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f838000c4f0 tx=0x7f8380007f50 comp rx=0 tx=0).stop 2023-10-06T10:39:26.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.498+0000 7f838f8d3700 1 -- 172.21.15.92:0/1369954121 shutdown_connections 2023-10-06T10:39:26.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.498+0000 7f838f8d3700 1 --2- 172.21.15.92:0/1369954121 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f836c066f30 0x7f836c0693e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:26.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.498+0000 7f838f8d3700 1 --2- 172.21.15.92:0/1369954121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388159660 0x7f838815c2e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:26.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.498+0000 7f838f8d3700 1 --2- 172.21.15.92:0/1369954121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83880b25e0 0x7f83880c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:26.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.498+0000 7f838f8d3700 1 --2- 172.21.15.92:0/1369954121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f83880b29b0 0x7f83880c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:26.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.498+0000 7f838f8d3700 1 -- 172.21.15.92:0/1369954121 >> 172.21.15.92:0/1369954121 conn(0x7f838806ece0 msgr2=0x7f8388153ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:26.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.498+0000 7f838f8d3700 1 -- 172.21.15.92:0/1369954121 shutdown_connections 2023-10-06T10:39:26.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:26.498+0000 7f838f8d3700 1 -- 172.21.15.92:0/1369954121 wait complete. 2023-10-06T10:39:26.501 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 21 2023-10-06T10:39:26.914 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T10:39:15.675013+0000', 'modified': '2023-10-06T10:39:26.352599+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 5133, 'mds_1': 5121, 'mds_2': 5118}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5118': {'gid': 5118, 'name': 'a', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.92:6836/2409440810', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 2409440810}, {'type': 'v1', 'addr': '172.21.15.92:6836', 'nonce': 2409440810}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_5121': {'gid': 5121, 'name': 'c', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.92:6837/4010629505', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 4010629505}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 4010629505}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_5133': {'gid': 5133, 'name': 'b', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.124:6834/484869337', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6832', 'nonce': 484869337}, {'type': 'v1', 'addr': '172.21.15.124:6834', 'nonce': 484869337}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T10:39:26.915 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-10-06T10:39:26.915 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T10:39:27.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.029+0000 7f0a65100700 1 Processor -- start 2023-10-06T10:39:27.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.029+0000 7f0a65100700 1 -- start start 2023-10-06T10:39:27.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.030+0000 7f0a65100700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a600b49a0 0x7f0a600aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.030+0000 7f0a65100700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0a600b0760 0x7f0a600b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.030+0000 7f0a65100700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a600b0c60 con 0x7f0a60159560 2023-10-06T10:39:27.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.030+0000 7f0a65100700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a600b0f40 con 0x7f0a600b0760 2023-10-06T10:39:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.030+0000 7f0a65100700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a600b1220 con 0x7f0a600b49a0 2023-10-06T10:39:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.030+0000 7f0a5659c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a600b49a0 0x7f0a600aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.030+0000 7f0a5f59e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0a600b0760 0x7f0a600b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.030+0000 7f0a5ed9d700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f0a60159560 0x7f0a600aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:47820/0 (socket says 172.21.15.92:47820) 2023-10-06T10:39:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.030+0000 7f0a5659c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a600b49a0 0x7f0a600aac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:60586/0 (socket says 172.21.15.92:60586) 2023-10-06T10:39:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.030+0000 7f0a5f59e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0a600b0760 0x7f0a600b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:39942/0 (socket says 172.21.15.92:39942) 2023-10-06T10:39:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.030+0000 7f0a5659c700 1 -- 172.21.15.92:0/2367983233 learned_addr learned my addr 172.21.15.92:0/2367983233 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.031+0000 7f0a5659c700 1 -- 172.21.15.92:0/2367983233 >> v2:172.21.15.124:3300/0 conn(0x7f0a600b0760 msgr2=0x7f0a600b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.031+0000 7f0a5659c700 1 --2- 172.21.15.92:0/2367983233 >> v2:172.21.15.124:3300/0 conn(0x7f0a600b0760 0x7f0a600b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.031+0000 7f0a5659c700 1 -- 172.21.15.92:0/2367983233 >> v1:172.21.15.124:6789/0 conn(0x7f0a60159560 legacy=0x7f0a600aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.031+0000 7f0a5659c700 1 -- 172.21.15.92:0/2367983233 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a600b1500 con 0x7f0a600b49a0 2023-10-06T10:39:27.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.031+0000 7f0a5f59e700 1 --2- 172.21.15.92:0/2367983233 >> v2:172.21.15.124:3300/0 conn(0x7f0a600b0760 0x7f0a600b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:39:27.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.031+0000 7f0a5659c700 1 --2- 172.21.15.92:0/2367983233 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a600b49a0 0x7f0a600aac40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0a480099b0 tx=0x7f0a48009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78da9f3eceb1096c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.031+0000 7f0a5e59c700 1 -- 172.21.15.92:0/2367983233 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a48009ea0 con 0x7f0a600b49a0 2023-10-06T10:39:27.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.031+0000 7f0a5e59c700 1 -- 172.21.15.92:0/2367983233 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4800b860 con 0x7f0a600b49a0 2023-10-06T10:39:27.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.031+0000 7f0a5e59c700 1 -- 172.21.15.92:0/2367983233 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4800bc00 con 0x7f0a600b49a0 2023-10-06T10:39:27.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.031+0000 7f0a65100700 1 -- 172.21.15.92:0/2367983233 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a600b49a0 msgr2=0x7f0a600aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.031+0000 7f0a65100700 1 --2- 172.21.15.92:0/2367983233 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a600b49a0 0x7f0a600aac40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0a480099b0 tx=0x7f0a48009250 comp rx=0 tx=0).stop 2023-10-06T10:39:27.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.031+0000 7f0a65100700 1 -- 172.21.15.92:0/2367983233 shutdown_connections 2023-10-06T10:39:27.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.031+0000 7f0a65100700 1 --2- 172.21.15.92:0/2367983233 >> v2:172.21.15.124:3300/0 conn(0x7f0a600b0760 0x7f0a600b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.031+0000 7f0a65100700 1 --2- 172.21.15.92:0/2367983233 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a600b49a0 0x7f0a600aac40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.031+0000 7f0a65100700 1 -- 172.21.15.92:0/2367983233 >> 172.21.15.92:0/2367983233 conn(0x7f0a600997f0 msgr2=0x7f0a6006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:27.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.031+0000 7f0a65100700 1 -- 172.21.15.92:0/2367983233 shutdown_connections 2023-10-06T10:39:27.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.031+0000 7f0a65100700 1 -- 172.21.15.92:0/2367983233 wait complete. 2023-10-06T10:39:27.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.033+0000 7f0a65100700 1 Processor -- start 2023-10-06T10:39:27.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.033+0000 7f0a65100700 1 -- start start 2023-10-06T10:39:27.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.033+0000 7f0a65100700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a600b0760 0x7f0a6009dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.033+0000 7f0a65100700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a600b49a0 0x7f0a600ab200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.033+0000 7f0a65100700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a60159560 0x7f0a600aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.033+0000 7f0a65100700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6009c800 con 0x7f0a600b49a0 2023-10-06T10:39:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.033+0000 7f0a65100700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a600af0c0 con 0x7f0a600b0760 2023-10-06T10:39:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.033+0000 7f0a65100700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a600ac460 con 0x7f0a60159560 2023-10-06T10:39:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.034+0000 7f0a5659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a600b49a0 0x7f0a600ab200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.034+0000 7f0a5659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a600b49a0 0x7f0a600ab200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47846/0 (socket says 172.21.15.92:47846) 2023-10-06T10:39:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.034+0000 7f0a5659c700 1 -- 172.21.15.92:0/455676269 learned_addr learned my addr 172.21.15.92:0/455676269 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.034+0000 7f0a5ed9d700 1 --2- 172.21.15.92:0/455676269 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a600b0760 0x7f0a6009dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.034+0000 7f0a5f59e700 1 --2- 172.21.15.92:0/455676269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a60159560 0x7f0a600aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.034+0000 7f0a5659c700 1 -- 172.21.15.92:0/455676269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a60159560 msgr2=0x7f0a600aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.034+0000 7f0a5659c700 1 --2- 172.21.15.92:0/455676269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a60159560 0x7f0a600aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.034+0000 7f0a5659c700 1 -- 172.21.15.92:0/455676269 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a600b0760 msgr2=0x7f0a6009dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.034+0000 7f0a5659c700 1 --2- 172.21.15.92:0/455676269 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a600b0760 0x7f0a6009dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.034+0000 7f0a5659c700 1 -- 172.21.15.92:0/455676269 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a58009a20 con 0x7f0a600b49a0 2023-10-06T10:39:27.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.034+0000 7f0a5f59e700 1 --2- 172.21.15.92:0/455676269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a60159560 0x7f0a600aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:27.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.034+0000 7f0a5659c700 1 --2- 172.21.15.92:0/455676269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a600b49a0 0x7f0a600ab200 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0a48006fd0 tx=0x7f0a48017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.034+0000 7f0a577fe700 1 -- 172.21.15.92:0/455676269 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a48000d10 con 0x7f0a600b49a0 2023-10-06T10:39:27.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.034+0000 7f0a577fe700 1 -- 172.21.15.92:0/455676269 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4800bbe0 con 0x7f0a600b49a0 2023-10-06T10:39:27.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.034+0000 7f0a65100700 1 -- 172.21.15.92:0/455676269 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a480094f0 con 0x7f0a600b49a0 2023-10-06T10:39:27.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.034+0000 7f0a577fe700 1 -- 172.21.15.92:0/455676269 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a48011650 con 0x7f0a600b49a0 2023-10-06T10:39:27.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.034+0000 7f0a65100700 1 -- 172.21.15.92:0/455676269 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a600ab850 con 0x7f0a600b49a0 2023-10-06T10:39:27.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.035+0000 7f0a577fe700 1 -- 172.21.15.92:0/455676269 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0a48011a80 con 0x7f0a600b49a0 2023-10-06T10:39:27.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.036+0000 7f0a577fe700 1 --2- 172.21.15.92:0/455676269 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0a40066f30 0x7f0a400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.036+0000 7f0a577fe700 1 -- 172.21.15.92:0/455676269 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0a48086110 con 0x7f0a600b49a0 2023-10-06T10:39:27.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.036+0000 7f0a5ed9d700 1 --2- 172.21.15.92:0/455676269 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0a40066f30 0x7f0a400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.036+0000 7f0a5ed9d700 1 --2- 172.21.15.92:0/455676269 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0a40066f30 0x7f0a400693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0a50007fd0 tx=0x7f0a50007f00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.038+0000 7f0a6e3f8b80 1 Processor -- start 2023-10-06T10:39:27.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.038+0000 7f0a6e3f8b80 1 -- start start 2023-10-06T10:39:27.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.039+0000 7f0a6e3f8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2dba90 0x55924d2dbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.039+0000 7f0a6e3f8b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55924d2979a0 0x55924d294580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.039+0000 7f0a6e3f8b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55924d295bd0 0x55924d294ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.039+0000 7f0a6e3f8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55924d214c80 con 0x55924d2dba90 2023-10-06T10:39:27.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.039+0000 7f0a6e3f8b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55924d1eba70 con 0x55924d295bd0 2023-10-06T10:39:27.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.039+0000 7f0a6e3f8b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55924d1e9270 con 0x55924d2979a0 2023-10-06T10:39:27.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.039+0000 7f0a5ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2dba90 0x55924d2dbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.039+0000 7f0a5ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2dba90 0x55924d2dbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47858/0 (socket says 172.21.15.92:47858) 2023-10-06T10:39:27.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.039+0000 7f0a5ed9d700 1 -- 172.21.15.92:0/3908602531 learned_addr learned my addr 172.21.15.92:0/3908602531 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:27.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.039+0000 7f0a5659c700 1 --2- 172.21.15.92:0/3908602531 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55924d2979a0 0x55924d294580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.039+0000 7f0a5f59e700 1 --2- 172.21.15.92:0/3908602531 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55924d295bd0 0x55924d294ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.039+0000 7f0a5ed9d700 1 -- 172.21.15.92:0/3908602531 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55924d2979a0 msgr2=0x55924d294580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.039+0000 7f0a5ed9d700 1 --2- 172.21.15.92:0/3908602531 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55924d2979a0 0x55924d294580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.039+0000 7f0a5ed9d700 1 -- 172.21.15.92:0/3908602531 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55924d295bd0 msgr2=0x55924d294ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.039+0000 7f0a5ed9d700 1 --2- 172.21.15.92:0/3908602531 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55924d295bd0 0x55924d294ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.039+0000 7f0a5ed9d700 1 -- 172.21.15.92:0/3908602531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55924d07f270 con 0x55924d2dba90 2023-10-06T10:39:27.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.039+0000 7f0a5659c700 1 --2- 172.21.15.92:0/3908602531 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55924d2979a0 0x55924d294580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:27.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.039+0000 7f0a5ed9d700 1 --2- 172.21.15.92:0/3908602531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2dba90 0x55924d2dbe60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0a50013730 tx=0x7f0a50014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac33aba7b7c912be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.040+0000 7f0a3ffff700 1 -- 172.21.15.92:0/3908602531 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a50013e20 con 0x55924d2dba90 2023-10-06T10:39:27.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.040+0000 7f0a3ffff700 1 -- 172.21.15.92:0/3908602531 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a500014d0 con 0x55924d2dba90 2023-10-06T10:39:27.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.040+0000 7f0a3ffff700 1 -- 172.21.15.92:0/3908602531 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a50001850 con 0x55924d2dba90 2023-10-06T10:39:27.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.040+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/3908602531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2dba90 msgr2=0x55924d2dbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.040+0000 7f0a6e3f8b80 1 --2- 172.21.15.92:0/3908602531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2dba90 0x55924d2dbe60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0a50013730 tx=0x7f0a50014000 comp rx=0 tx=0).stop 2023-10-06T10:39:27.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.040+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/3908602531 shutdown_connections 2023-10-06T10:39:27.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.040+0000 7f0a6e3f8b80 1 --2- 172.21.15.92:0/3908602531 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55924d295bd0 0x55924d294ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.040+0000 7f0a6e3f8b80 1 --2- 172.21.15.92:0/3908602531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2dba90 0x55924d2dbe60 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.040+0000 7f0a6e3f8b80 1 --2- 172.21.15.92:0/3908602531 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55924d2979a0 0x55924d294580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.040+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/3908602531 >> 172.21.15.92:0/3908602531 conn(0x55924d296070 msgr2=0x55924d296440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:27.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.040+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/3908602531 shutdown_connections 2023-10-06T10:39:27.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.040+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/3908602531 wait complete. 2023-10-06T10:39:27.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.041+0000 7f0a6e3f8b80 1 -- start start 2023-10-06T10:39:27.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.041+0000 7f0a6e3f8b80 1 Processor -- start 2023-10-06T10:39:27.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.042+0000 7f0a6e3f8b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55924d295bd0 0x55924d2eb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.042+0000 7f0a6e3f8b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55924d2979a0 0x55924d2b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.042+0000 7f0a6e3f8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2dba90 0x55924d2deed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.042+0000 7f0a6e3f8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55924d1e5060 con 0x55924d2dba90 2023-10-06T10:39:27.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.042+0000 7f0a6e3f8b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55924d190890 con 0x55924d295bd0 2023-10-06T10:39:27.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.042+0000 7f0a6e3f8b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55924d1abea0 con 0x55924d2979a0 2023-10-06T10:39:27.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.042+0000 7f0a6e3f8b80 10 client.0 _ll_register_callbacks cb 0x55924d081f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:39:27.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.042+0000 7f0a5f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2dba90 0x55924d2deed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.042+0000 7f0a5f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2dba90 0x55924d2deed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47874/0 (socket says 172.21.15.92:47874) 2023-10-06T10:39:27.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.042+0000 7f0a5f59e700 1 -- 172.21.15.92:0/3653542882 learned_addr learned my addr 172.21.15.92:0/3653542882 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:27.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.042+0000 7f0a5ed9d700 1 --2- 172.21.15.92:0/3653542882 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55924d295bd0 0x55924d2eb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.042+0000 7f0a5f59e700 1 -- 172.21.15.92:0/3653542882 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55924d2979a0 msgr2=0x55924d2b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.042+0000 7f0a5f59e700 1 --2- 172.21.15.92:0/3653542882 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55924d2979a0 0x55924d2b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.042+0000 7f0a5f59e700 1 -- 172.21.15.92:0/3653542882 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55924d295bd0 msgr2=0x55924d2eb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.042+0000 7f0a5f59e700 1 --2- 172.21.15.92:0/3653542882 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55924d295bd0 0x55924d2eb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.042+0000 7f0a5f59e700 1 -- 172.21.15.92:0/3653542882 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a600b0c60 con 0x55924d2dba90 2023-10-06T10:39:27.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.042+0000 7f0a5f59e700 1 --2- 172.21.15.92:0/3653542882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2dba90 0x55924d2deed0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0a58007fc0 tx=0x7f0a5800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4200e3ed891d5b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.042+0000 7f0a3d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:39:27.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.042+0000 7f0a3d7fa700 1 -- 172.21.15.92:0/3653542882 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5800e040 con 0x55924d2dba90 2023-10-06T10:39:27.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.043+0000 7f0a3d7fa700 1 -- 172.21.15.92:0/3653542882 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0a58012e50 con 0x55924d2dba90 2023-10-06T10:39:27.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.043+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/3653542882 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55924d289690 con 0x55924d2dba90 2023-10-06T10:39:27.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.043+0000 7f0a3d7fa700 1 -- 172.21.15.92:0/3653542882 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a58018680 con 0x55924d2dba90 2023-10-06T10:39:27.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.043+0000 7f0a6e3f8b80 10 client.5349 fetch_fsmap learned FSMap version 21 2023-10-06T10:39:27.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.043+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/3653542882 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55924d021fe0 con 0x55924d2dba90 2023-10-06T10:39:27.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.043+0000 7f0a3d7fa700 1 -- 172.21.15.92:0/3653542882 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55924d289690 con 0x55924d2dba90 2023-10-06T10:39:27.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.043+0000 7f0a3d7fa700 1 -- 172.21.15.92:0/3653542882 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 21) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f0a58014040 con 0x55924d2dba90 2023-10-06T10:39:27.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.043+0000 7f0a6e3f8b80 10 client.5349 fetch_fsmap finished waiting for FSMap version 21 2023-10-06T10:39:27.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.043+0000 7f0a6e3f8b80 10 client.5349 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T10:39:27.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.043+0000 7f0a6e3f8b80 20 client.5349 populate_metadata read hostname 'smithi092' 2023-10-06T10:39:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.043+0000 7f0a6e3f8b80 1 --2- 172.21.15.92:0/3653542882 >> [v2:172.21.15.92:6834/2409440810,v1:172.21.15.92:6836/2409440810] conn(0x55924d2babf0 0x55924d2bafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.043+0000 7f0a6e3f8b80 4 client.5349 mds_command: new command op to 5118 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:39:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.043+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/3653542882 --> [v2:172.21.15.92:6834/2409440810,v1:172.21.15.92:6836/2409440810] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55924d2665c0 con 0x55924d2babf0 2023-10-06T10:39:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.043+0000 7f0a5ed9d700 1 --2- 172.21.15.92:0/3653542882 >> [v2:172.21.15.92:6834/2409440810,v1:172.21.15.92:6836/2409440810] conn(0x55924d2babf0 0x55924d2bafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.043+0000 7f0a3d7fa700 10 client.5349 ms_handle_connect on v2:172.21.15.92:6834/2409440810 2023-10-06T10:39:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.043+0000 7f0a5ed9d700 1 --2- 172.21.15.92:0/3653542882 >> [v2:172.21.15.92:6834/2409440810,v1:172.21.15.92:6836/2409440810] conn(0x55924d2babf0 0x55924d2bafc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b8da19848a1cf89a server_cookie=be466c5509cde195 in_seq=0 out_seq=0 2023-10-06T10:39:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.044+0000 7f0a3d7fa700 1 -- 172.21.15.92:0/3653542882 <== mds.2 v2:172.21.15.92:6834/2409440810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55924d2665c0 con 0x55924d2babf0 2023-10-06T10:39:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.044+0000 7f0a3d7fa700 10 client.5349 handle_command_reply: tid=0 2023-10-06T10:39:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.044+0000 7f0a6e3f8b80 1 client.5349 shutdown 2023-10-06T10:39:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.044+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/3653542882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2dba90 msgr2=0x55924d2deed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.044+0000 7f0a6e3f8b80 1 --2- 172.21.15.92:0/3653542882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2dba90 0x55924d2deed0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0a58007fc0 tx=0x7f0a5800b040 comp rx=0 tx=0).stop 2023-10-06T10:39:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.044+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/3653542882 shutdown_connections 2023-10-06T10:39:27.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.044+0000 7f0a6e3f8b80 1 --2- 172.21.15.92:0/3653542882 >> [v2:172.21.15.92:6834/2409440810,v1:172.21.15.92:6836/2409440810] conn(0x55924d2babf0 0x55924d2bafc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.044+0000 7f0a6e3f8b80 1 --2- 172.21.15.92:0/3653542882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2dba90 0x55924d2deed0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.044+0000 7f0a6e3f8b80 1 --2- 172.21.15.92:0/3653542882 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55924d295bd0 0x55924d2eb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.044+0000 7f0a6e3f8b80 1 --2- 172.21.15.92:0/3653542882 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55924d2979a0 0x55924d2b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.044+0000 7f0a3d7fa700 0 client.5349 ms_handle_reset on v2:172.21.15.92:6834/2409440810 2023-10-06T10:39:27.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.044+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/3653542882 >> 172.21.15.92:0/3653542882 conn(0x55924d296070 msgr2=0x55924d2d92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:27.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.044+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/3653542882 shutdown_connections 2023-10-06T10:39:27.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.044+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/3653542882 wait complete. 2023-10-06T10:39:27.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.044+0000 7f0a6e3f8b80 20 client.5349 trim_cache size 0 max 16384 2023-10-06T10:39:27.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.061+0000 7f0a6e3f8b80 1 Processor -- start 2023-10-06T10:39:27.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.061+0000 7f0a6e3f8b80 1 -- start start 2023-10-06T10:39:27.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.062+0000 7f0a6e3f8b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55924d295bd0 0x55924d2bc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.062+0000 7f0a6e3f8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2979a0 0x55924d2c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.062+0000 7f0a6e3f8b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55924d2dba90 0x55924d2c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.062+0000 7f0a6e3f8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55924d067e30 con 0x55924d2979a0 2023-10-06T10:39:27.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.062+0000 7f0a6e3f8b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55924d2740d0 con 0x55924d295bd0 2023-10-06T10:39:27.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.062+0000 7f0a6e3f8b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55924d296e80 con 0x55924d2dba90 2023-10-06T10:39:27.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.062+0000 7f0a5659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2979a0 0x55924d2c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.062+0000 7f0a5659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2979a0 0x55924d2c3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47888/0 (socket says 172.21.15.92:47888) 2023-10-06T10:39:27.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.062+0000 7f0a5659c700 1 -- 172.21.15.92:0/515748905 learned_addr learned my addr 172.21.15.92:0/515748905 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:27.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.062+0000 7f0a5ed9d700 1 --2- 172.21.15.92:0/515748905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55924d295bd0 0x55924d2bc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.062+0000 7f0a5f59e700 1 --2- 172.21.15.92:0/515748905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55924d2dba90 0x55924d2c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.062+0000 7f0a5659c700 1 -- 172.21.15.92:0/515748905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55924d2dba90 msgr2=0x55924d2c3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.062+0000 7f0a5659c700 1 --2- 172.21.15.92:0/515748905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55924d2dba90 0x55924d2c3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.062+0000 7f0a5659c700 1 -- 172.21.15.92:0/515748905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55924d295bd0 msgr2=0x55924d2bc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.062+0000 7f0a5659c700 1 --2- 172.21.15.92:0/515748905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55924d295bd0 0x55924d2bc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.062+0000 7f0a5659c700 1 -- 172.21.15.92:0/515748905 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55924d1b3780 con 0x55924d2979a0 2023-10-06T10:39:27.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.062+0000 7f0a5f59e700 1 --2- 172.21.15.92:0/515748905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55924d2dba90 0x55924d2c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:27.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.062+0000 7f0a5659c700 1 --2- 172.21.15.92:0/515748905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2979a0 0x55924d2c3a50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800b3c0 tx=0x7f0a48086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b9b34ad06f44ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.062+0000 7f0a54d99700 1 -- 172.21.15.92:0/515748905 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a48056da0 con 0x55924d2979a0 2023-10-06T10:39:27.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.062+0000 7f0a54d99700 1 -- 172.21.15.92:0/515748905 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a48059ff0 con 0x55924d2979a0 2023-10-06T10:39:27.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.062+0000 7f0a54d99700 1 -- 172.21.15.92:0/515748905 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4805dd40 con 0x55924d2979a0 2023-10-06T10:39:27.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.063+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/515748905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2979a0 msgr2=0x55924d2c3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.063+0000 7f0a6e3f8b80 1 --2- 172.21.15.92:0/515748905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2979a0 0x55924d2c3a50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800b3c0 tx=0x7f0a48086040 comp rx=0 tx=0).stop 2023-10-06T10:39:27.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.063+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/515748905 shutdown_connections 2023-10-06T10:39:27.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.063+0000 7f0a6e3f8b80 1 --2- 172.21.15.92:0/515748905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55924d2dba90 0x55924d2c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.063+0000 7f0a6e3f8b80 1 --2- 172.21.15.92:0/515748905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55924d295bd0 0x55924d2bc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.063+0000 7f0a6e3f8b80 1 --2- 172.21.15.92:0/515748905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2979a0 0x55924d2c3a50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.063+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/515748905 >> 172.21.15.92:0/515748905 conn(0x55924d296070 msgr2=0x55924d296440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:27.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.063+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/515748905 shutdown_connections 2023-10-06T10:39:27.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.063+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/515748905 wait complete. 2023-10-06T10:39:27.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.064+0000 7f0a6e3f8b80 1 -- start start 2023-10-06T10:39:27.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.064+0000 7f0a6e3f8b80 1 Processor -- start 2023-10-06T10:39:27.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.064+0000 7f0a6e3f8b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55924d295bd0 0x55924d2c3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.064+0000 7f0a6e3f8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2979a0 0x55924d2b6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.064+0000 7f0a6e3f8b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55924d2dba90 0x55924d3333f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.064+0000 7f0a6e3f8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55924d294f40 con 0x55924d2979a0 2023-10-06T10:39:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.064+0000 7f0a6e3f8b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55924d341f40 con 0x55924d2dba90 2023-10-06T10:39:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.064+0000 7f0a6e3f8b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55924d2d5e40 con 0x55924d295bd0 2023-10-06T10:39:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.064+0000 7f0a6e3f8b80 10 client.0 _ll_register_callbacks cb 0x55924d081f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:39:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a5659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2979a0 0x55924d2b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a5659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2979a0 0x55924d2b6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47896/0 (socket says 172.21.15.92:47896) 2023-10-06T10:39:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a5659c700 1 -- 172.21.15.92:0/2402170679 learned_addr learned my addr 172.21.15.92:0/2402170679 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a5ed9d700 1 --2- 172.21.15.92:0/2402170679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55924d295bd0 0x55924d2c3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a5f59e700 1 --2- 172.21.15.92:0/2402170679 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55924d2dba90 0x55924d3333f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a5659c700 1 -- 172.21.15.92:0/2402170679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55924d295bd0 msgr2=0x55924d2c3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a5659c700 1 --2- 172.21.15.92:0/2402170679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55924d295bd0 0x55924d2c3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a5659c700 1 -- 172.21.15.92:0/2402170679 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55924d2dba90 msgr2=0x55924d3333f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a5659c700 1 --2- 172.21.15.92:0/2402170679 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55924d2dba90 0x55924d3333f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a5659c700 1 -- 172.21.15.92:0/2402170679 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a600ab850 con 0x55924d2979a0 2023-10-06T10:39:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a5659c700 1 --2- 172.21.15.92:0/2402170679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2979a0 0x55924d2b6010 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009470 tx=0x7f0a4802f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed125444e86f7042 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a3d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:39:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a3d7fa700 1 -- 172.21.15.92:0/2402170679 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4806d1e0 con 0x55924d2979a0 2023-10-06T10:39:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a3d7fa700 1 -- 172.21.15.92:0/2402170679 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0a4805dd40 con 0x55924d2979a0 2023-10-06T10:39:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/2402170679 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55924d2be280 con 0x55924d2979a0 2023-10-06T10:39:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a3d7fa700 1 -- 172.21.15.92:0/2402170679 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a48084020 con 0x55924d2979a0 2023-10-06T10:39:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a6e3f8b80 10 client.5355 fetch_fsmap learned FSMap version 21 2023-10-06T10:39:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/2402170679 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55924d19a470 con 0x55924d2979a0 2023-10-06T10:39:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a3d7fa700 1 -- 172.21.15.92:0/2402170679 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55924d2be280 con 0x55924d2979a0 2023-10-06T10:39:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a3d7fa700 1 -- 172.21.15.92:0/2402170679 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 21) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f0a48056290 con 0x55924d2979a0 2023-10-06T10:39:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a6e3f8b80 10 client.5355 fetch_fsmap finished waiting for FSMap version 21 2023-10-06T10:39:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a6e3f8b80 10 client.5355 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T10:39:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a6e3f8b80 20 client.5355 populate_metadata read hostname 'smithi092' 2023-10-06T10:39:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a6e3f8b80 1 --2- 172.21.15.92:0/2402170679 >> [v2:172.21.15.92:6834/2409440810,v1:172.21.15.92:6836/2409440810] conn(0x55924d2bb8c0 0x55924d2bbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a6e3f8b80 4 client.5355 mds_command: new command op to 5118 tid=0[{"prefix": "status"}] 2023-10-06T10:39:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.065+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/2402170679 --> [v2:172.21.15.92:6834/2409440810,v1:172.21.15.92:6836/2409440810] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f0a500014d0 con 0x55924d2bb8c0 2023-10-06T10:39:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.066+0000 7f0a5ed9d700 1 --2- 172.21.15.92:0/2402170679 >> [v2:172.21.15.92:6834/2409440810,v1:172.21.15.92:6836/2409440810] conn(0x55924d2bb8c0 0x55924d2bbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.066+0000 7f0a3d7fa700 10 client.5355 ms_handle_connect on v2:172.21.15.92:6834/2409440810 2023-10-06T10:39:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.066+0000 7f0a5ed9d700 1 --2- 172.21.15.92:0/2402170679 >> [v2:172.21.15.92:6834/2409440810,v1:172.21.15.92:6836/2409440810] conn(0x55924d2bb8c0 0x55924d2bbc90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=53cecb650bdcd6c2 server_cookie=6e3e54bec363b3e4 in_seq=0 out_seq=0 2023-10-06T10:39:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.066+0000 7f0a3d7fa700 1 -- 172.21.15.92:0/2402170679 <== mds.2 v2:172.21.15.92:6834/2409440810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x7f0a500014d0 con 0x55924d2bb8c0 2023-10-06T10:39:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.066+0000 7f0a3d7fa700 10 client.5355 handle_command_reply: tid=0 2023-10-06T10:39:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.066+0000 7f0a6e3f8b80 1 client.5355 shutdown 2023-10-06T10:39:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.066+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/2402170679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2979a0 msgr2=0x55924d2b6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.066+0000 7f0a6e3f8b80 1 --2- 172.21.15.92:0/2402170679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2979a0 0x55924d2b6010 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009470 tx=0x7f0a4802f000 comp rx=0 tx=0).stop 2023-10-06T10:39:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.066+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/2402170679 shutdown_connections 2023-10-06T10:39:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.066+0000 7f0a6e3f8b80 1 --2- 172.21.15.92:0/2402170679 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55924d2dba90 0x55924d3333f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.066+0000 7f0a6e3f8b80 1 --2- 172.21.15.92:0/2402170679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55924d295bd0 0x55924d2c3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.066+0000 7f0a6e3f8b80 1 --2- 172.21.15.92:0/2402170679 >> [v2:172.21.15.92:6834/2409440810,v1:172.21.15.92:6836/2409440810] conn(0x55924d2bb8c0 0x55924d2bbc90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.066+0000 7f0a6e3f8b80 1 --2- 172.21.15.92:0/2402170679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55924d2979a0 0x55924d2b6010 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.066+0000 7f0a3d7fa700 0 client.5355 ms_handle_reset on v2:172.21.15.92:6834/2409440810 2023-10-06T10:39:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.066+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/2402170679 >> 172.21.15.92:0/2402170679 conn(0x55924d296070 msgr2=0x55924d2d99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.066+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/2402170679 shutdown_connections 2023-10-06T10:39:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.066+0000 7f0a6e3f8b80 1 -- 172.21.15.92:0/2402170679 wait complete. 2023-10-06T10:39:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.066+0000 7f0a6e3f8b80 20 client.5355 trim_cache size 0 max 16384 2023-10-06T10:39:27.076 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:39:27.076 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:39:27.076 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 2, 2023-10-06T10:39:27.076 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 5118, 2023-10-06T10:39:27.077 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:39:27.077 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:39:27.077 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:39:27.077 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 1.7149078929999999, 2023-10-06T10:39:27.077 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 21, 2023-10-06T10:39:27.077 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 38, 2023-10-06T10:39:27.077 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 38, 2023-10-06T10:39:27.077 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 19.129739045000001 2023-10-06T10:39:27.077 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:39:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.067+0000 7f0a65100700 1 -- 172.21.15.92:0/455676269 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0a40066f30 msgr2=0x7f0a400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.067+0000 7f0a65100700 1 --2- 172.21.15.92:0/455676269 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0a40066f30 0x7f0a400693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0a50007fd0 tx=0x7f0a50007f00 comp rx=0 tx=0).stop 2023-10-06T10:39:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.067+0000 7f0a65100700 1 -- 172.21.15.92:0/455676269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a600b49a0 msgr2=0x7f0a600ab200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.067+0000 7f0a65100700 1 --2- 172.21.15.92:0/455676269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a600b49a0 0x7f0a600ab200 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0a48006fd0 tx=0x7f0a48017000 comp rx=0 tx=0).stop 2023-10-06T10:39:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.067+0000 7f0a65100700 1 -- 172.21.15.92:0/455676269 shutdown_connections 2023-10-06T10:39:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.067+0000 7f0a65100700 1 --2- 172.21.15.92:0/455676269 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0a40066f30 0x7f0a400693e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.067+0000 7f0a65100700 1 --2- 172.21.15.92:0/455676269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a60159560 0x7f0a600aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.067+0000 7f0a65100700 1 --2- 172.21.15.92:0/455676269 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a600b0760 0x7f0a6009dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.067+0000 7f0a65100700 1 --2- 172.21.15.92:0/455676269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a600b49a0 0x7f0a600ab200 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.067+0000 7f0a65100700 1 -- 172.21.15.92:0/455676269 >> 172.21.15.92:0/455676269 conn(0x7f0a600997f0 msgr2=0x7f0a6015cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.067+0000 7f0a65100700 1 -- 172.21.15.92:0/455676269 shutdown_connections 2023-10-06T10:39:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.067+0000 7f0a65100700 1 -- 172.21.15.92:0/455676269 wait complete. 2023-10-06T10:39:27.080 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T10:39:27.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.194+0000 7f574457b700 1 Processor -- start 2023-10-06T10:39:27.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.194+0000 7f574457b700 1 -- start start 2023-10-06T10:39:27.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.195+0000 7f574457b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f573c0b49a0 0x7f573c0aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.195+0000 7f574457b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573c0b0c60 con 0x7f573c159560 2023-10-06T10:39:27.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.195+0000 7f574457b700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573c0b0f40 con 0x7f573c0b0760 2023-10-06T10:39:27.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.195+0000 7f574457b700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f573c0b1220 con 0x7f573c0b49a0 2023-10-06T10:39:27.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.195+0000 7f5742317700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f573c159560 0x7f573c0aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35334/0 (socket says 172.21.15.92:35334) 2023-10-06T10:39:27.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.195+0000 7f5742317700 1 -- 172.21.15.92:0/3258513547 learned_addr learned my addr 172.21.15.92:0/3258513547 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:27.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.195+0000 7f5741b16700 1 --2- 172.21.15.92:0/3258513547 >> v2:172.21.15.124:3300/0 conn(0x7f573c0b49a0 0x7f573c0aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.195+0000 7f5741315700 1 -- 172.21.15.92:0/3258513547 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 235377257 0 0) 0x7f573c0b0c60 con 0x7f573c159560 2023-10-06T10:39:27.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.195+0000 7f5741315700 1 -- 172.21.15.92:0/3258513547 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5728003570 con 0x7f573c159560 2023-10-06T10:39:27.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.196+0000 7f5741b16700 1 -- 172.21.15.92:0/3258513547 >> v1:172.21.15.124:6789/0 conn(0x7f573c0b0760 legacy=0x7f573c0b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.196+0000 7f5741b16700 1 -- 172.21.15.92:0/3258513547 >> v1:172.21.15.92:6789/0 conn(0x7f573c159560 legacy=0x7f573c0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.196+0000 7f5741b16700 1 -- 172.21.15.92:0/3258513547 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c0b1500 con 0x7f573c0b49a0 2023-10-06T10:39:27.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.196+0000 7f5741b16700 1 --2- 172.21.15.92:0/3258513547 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f573c0b49a0 0x7f573c0aac40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f572c009910 tx=0x7f572c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b79be449d8e5ab9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.196+0000 7f5741315700 1 -- 172.21.15.92:0/3258513547 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c00b640 con 0x7f573c0b49a0 2023-10-06T10:39:27.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.196+0000 7f5741315700 1 -- 172.21.15.92:0/3258513547 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572c00b7a0 con 0x7f573c0b49a0 2023-10-06T10:39:27.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.196+0000 7f5741315700 1 -- 172.21.15.92:0/3258513547 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c00bb20 con 0x7f573c0b49a0 2023-10-06T10:39:27.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.196+0000 7f574457b700 1 -- 172.21.15.92:0/3258513547 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f573c0b49a0 msgr2=0x7f573c0aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.196+0000 7f574457b700 1 --2- 172.21.15.92:0/3258513547 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f573c0b49a0 0x7f573c0aac40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f572c009910 tx=0x7f572c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.197+0000 7f574457b700 1 -- 172.21.15.92:0/3258513547 shutdown_connections 2023-10-06T10:39:27.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.197+0000 7f574457b700 1 --2- 172.21.15.92:0/3258513547 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f573c0b49a0 0x7f573c0aac40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.197+0000 7f574457b700 1 -- 172.21.15.92:0/3258513547 >> 172.21.15.92:0/3258513547 conn(0x7f573c0997f0 msgr2=0x7f573c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:27.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.197+0000 7f574457b700 1 -- 172.21.15.92:0/3258513547 shutdown_connections 2023-10-06T10:39:27.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.197+0000 7f574457b700 1 -- 172.21.15.92:0/3258513547 wait complete. 2023-10-06T10:39:27.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.198+0000 7f574457b700 1 Processor -- start 2023-10-06T10:39:27.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.198+0000 7f574457b700 1 -- start start 2023-10-06T10:39:27.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.198+0000 7f574457b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f573c0b0760 0x7f573c157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.198+0000 7f574457b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f573c0b49a0 0x7f573c0adc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.198+0000 7f574457b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f573c159560 0x7f573c0ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.198+0000 7f574457b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573c0ae810 con 0x7f573c159560 2023-10-06T10:39:27.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.198+0000 7f574457b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573c0aeaf0 con 0x7f573c0b49a0 2023-10-06T10:39:27.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.198+0000 7f574457b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573c0aedd0 con 0x7f573c0b0760 2023-10-06T10:39:27.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.198+0000 7f5742b18700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f573c159560 0x7f573c0ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.198+0000 7f5742b18700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f573c159560 0x7f573c0ae190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47904/0 (socket says 172.21.15.92:47904) 2023-10-06T10:39:27.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.199+0000 7f5742b18700 1 -- 172.21.15.92:0/3200935371 learned_addr learned my addr 172.21.15.92:0/3200935371 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:27.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.199+0000 7f5741b16700 1 --2- 172.21.15.92:0/3200935371 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f573c0b49a0 0x7f573c0adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.199+0000 7f5742317700 1 --2- 172.21.15.92:0/3200935371 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f573c0b0760 0x7f573c157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.199+0000 7f5742b18700 1 -- 172.21.15.92:0/3200935371 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f573c0b0760 msgr2=0x7f573c157c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.199+0000 7f5742b18700 1 --2- 172.21.15.92:0/3200935371 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f573c0b0760 0x7f573c157c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.199+0000 7f5742b18700 1 -- 172.21.15.92:0/3200935371 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f573c0b49a0 msgr2=0x7f573c0adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.199+0000 7f5742b18700 1 --2- 172.21.15.92:0/3200935371 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f573c0b49a0 0x7f573c0adc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.199+0000 7f5742b18700 1 -- 172.21.15.92:0/3200935371 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f572c009420 con 0x7f573c159560 2023-10-06T10:39:27.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.199+0000 7f5741b16700 1 --2- 172.21.15.92:0/3200935371 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f573c0b49a0 0x7f573c0adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:27.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.199+0000 7f5742b18700 1 --2- 172.21.15.92:0/3200935371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f573c159560 0x7f573c0ae190 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f573400c8e0 tx=0x7f5734001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.199+0000 7f57327fc700 1 -- 172.21.15.92:0/3200935371 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573400ce80 con 0x7f573c159560 2023-10-06T10:39:27.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.199+0000 7f57327fc700 1 -- 172.21.15.92:0/3200935371 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573400a7e0 con 0x7f573c159560 2023-10-06T10:39:27.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.199+0000 7f574457b700 1 -- 172.21.15.92:0/3200935371 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f573c0af0b0 con 0x7f573c159560 2023-10-06T10:39:27.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.199+0000 7f57327fc700 1 -- 172.21.15.92:0/3200935371 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573400ab80 con 0x7f573c159560 2023-10-06T10:39:27.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.199+0000 7f574457b700 1 -- 172.21.15.92:0/3200935371 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f573c0aaa20 con 0x7f573c159560 2023-10-06T10:39:27.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.201+0000 7f57327fc700 1 -- 172.21.15.92:0/3200935371 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5734012420 con 0x7f573c159560 2023-10-06T10:39:27.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.201+0000 7f57327fc700 1 --2- 172.21.15.92:0/3200935371 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5720066f30 0x7f57200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.201+0000 7f57327fc700 1 -- 172.21.15.92:0/3200935371 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5734084b10 con 0x7f573c159560 2023-10-06T10:39:27.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.201+0000 7f5742317700 1 --2- 172.21.15.92:0/3200935371 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5720066f30 0x7f57200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.201+0000 7f5742317700 1 --2- 172.21.15.92:0/3200935371 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5720066f30 0x7f57200693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5738001360 tx=0x7f5738009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.203+0000 7f574d873b80 1 Processor -- start 2023-10-06T10:39:27.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.203+0000 7f574d873b80 1 -- start start 2023-10-06T10:39:27.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.204+0000 7f574d873b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e52ba90 0x55e36e52be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.204+0000 7f574d873b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e36e4e79a0 0x55e36e4e4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.204+0000 7f574d873b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e36e4e5bd0 0x55e36e4e4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.204+0000 7f574d873b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e36e464c80 con 0x55e36e52ba90 2023-10-06T10:39:27.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.204+0000 7f574d873b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e36e43ba70 con 0x55e36e4e79a0 2023-10-06T10:39:27.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.204+0000 7f574d873b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e36e439270 con 0x55e36e4e5bd0 2023-10-06T10:39:27.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.204+0000 7f5742317700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e52ba90 0x55e36e52be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.204+0000 7f5742317700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e52ba90 0x55e36e52be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47908/0 (socket says 172.21.15.92:47908) 2023-10-06T10:39:27.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.204+0000 7f5742317700 1 -- 172.21.15.92:0/3798772680 learned_addr learned my addr 172.21.15.92:0/3798772680 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:27.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.204+0000 7f5741b16700 1 --2- 172.21.15.92:0/3798772680 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e36e4e79a0 0x55e36e4e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.204+0000 7f5742b18700 1 --2- 172.21.15.92:0/3798772680 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e36e4e5bd0 0x55e36e4e4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.204+0000 7f5742317700 1 -- 172.21.15.92:0/3798772680 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e36e4e5bd0 msgr2=0x55e36e4e4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.204+0000 7f5742317700 1 --2- 172.21.15.92:0/3798772680 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e36e4e5bd0 0x55e36e4e4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.204+0000 7f5742317700 1 -- 172.21.15.92:0/3798772680 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e36e4e79a0 msgr2=0x55e36e4e4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.204+0000 7f5742317700 1 --2- 172.21.15.92:0/3798772680 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e36e4e79a0 0x55e36e4e4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.204+0000 7f5742317700 1 -- 172.21.15.92:0/3798772680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e36e2cf270 con 0x55e36e52ba90 2023-10-06T10:39:27.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.204+0000 7f5742b18700 1 --2- 172.21.15.92:0/3798772680 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e36e4e5bd0 0x55e36e4e4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:27.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.204+0000 7f5742317700 1 --2- 172.21.15.92:0/3798772680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e52ba90 0x55e36e52be60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5738007600 tx=0x7f5738013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ac047f8c0838d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.204+0000 7f571f7fe700 1 -- 172.21.15.92:0/3798772680 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57380152e0 con 0x55e36e52ba90 2023-10-06T10:39:27.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.204+0000 7f571f7fe700 1 -- 172.21.15.92:0/3798772680 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5738015510 con 0x55e36e52ba90 2023-10-06T10:39:27.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.204+0000 7f571f7fe700 1 -- 172.21.15.92:0/3798772680 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57380158b0 con 0x55e36e52ba90 2023-10-06T10:39:27.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.205+0000 7f574d873b80 1 -- 172.21.15.92:0/3798772680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e52ba90 msgr2=0x55e36e52be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.205+0000 7f574d873b80 1 --2- 172.21.15.92:0/3798772680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e52ba90 0x55e36e52be60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5738007600 tx=0x7f5738013000 comp rx=0 tx=0).stop 2023-10-06T10:39:27.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.205+0000 7f574d873b80 1 -- 172.21.15.92:0/3798772680 shutdown_connections 2023-10-06T10:39:27.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.205+0000 7f574d873b80 1 --2- 172.21.15.92:0/3798772680 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e36e4e5bd0 0x55e36e4e4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.205+0000 7f574d873b80 1 --2- 172.21.15.92:0/3798772680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e52ba90 0x55e36e52be60 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.205+0000 7f574d873b80 1 --2- 172.21.15.92:0/3798772680 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e36e4e79a0 0x55e36e4e4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.205+0000 7f574d873b80 1 -- 172.21.15.92:0/3798772680 >> 172.21.15.92:0/3798772680 conn(0x55e36e4e6070 msgr2=0x55e36e4e6440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:27.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.205+0000 7f574d873b80 1 -- 172.21.15.92:0/3798772680 shutdown_connections 2023-10-06T10:39:27.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.205+0000 7f574d873b80 1 -- 172.21.15.92:0/3798772680 wait complete. 2023-10-06T10:39:27.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.206+0000 7f574d873b80 1 -- start start 2023-10-06T10:39:27.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.206+0000 7f574d873b80 1 Processor -- start 2023-10-06T10:39:27.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f574d873b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e36e4e5bd0 0x55e36e50b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f574d873b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e4e79a0 0x55e36e510370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f574d873b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e36e52ba90 0x55e36e5229d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f574d873b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e36e435060 con 0x55e36e4e79a0 2023-10-06T10:39:27.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f574d873b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e36e3e0890 con 0x55e36e4e5bd0 2023-10-06T10:39:27.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f574d873b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e36e3fbea0 con 0x55e36e52ba90 2023-10-06T10:39:27.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f574d873b80 10 client.0 _ll_register_callbacks cb 0x55e36e2d1f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:39:27.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f5741b16700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e4e79a0 0x55e36e510370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f5741b16700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e4e79a0 0x55e36e510370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47918/0 (socket says 172.21.15.92:47918) 2023-10-06T10:39:27.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f5741b16700 1 -- 172.21.15.92:0/3330858119 learned_addr learned my addr 172.21.15.92:0/3330858119 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:27.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f5742317700 1 --2- 172.21.15.92:0/3330858119 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e36e4e5bd0 0x55e36e50b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f5742b18700 1 --2- 172.21.15.92:0/3330858119 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e36e52ba90 0x55e36e5229d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f5741b16700 1 -- 172.21.15.92:0/3330858119 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e36e52ba90 msgr2=0x55e36e5229d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f5741b16700 1 --2- 172.21.15.92:0/3330858119 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e36e52ba90 0x55e36e5229d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f5741b16700 1 -- 172.21.15.92:0/3330858119 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e36e4e5bd0 msgr2=0x55e36e50b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f5741b16700 1 --2- 172.21.15.92:0/3330858119 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e36e4e5bd0 0x55e36e50b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f5741b16700 1 -- 172.21.15.92:0/3330858119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f573800f040 con 0x55e36e4e79a0 2023-10-06T10:39:27.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f5742b18700 1 --2- 172.21.15.92:0/3330858119 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e36e52ba90 0x55e36e5229d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:27.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f5741b16700 1 --2- 172.21.15.92:0/3330858119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e4e79a0 0x55e36e510370 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f572c009fb0 tx=0x7f572c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a744a6d3fb396e0c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f571cff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:39:27.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f571cff9700 1 -- 172.21.15.92:0/3330858119 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c022070 con 0x55e36e4e79a0 2023-10-06T10:39:27.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f571cff9700 1 -- 172.21.15.92:0/3330858119 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f572c027460 con 0x55e36e4e79a0 2023-10-06T10:39:27.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f574d873b80 1 -- 172.21.15.92:0/3330858119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e36e4d9690 con 0x55e36e4e79a0 2023-10-06T10:39:27.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f571cff9700 1 -- 172.21.15.92:0/3330858119 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f572c027830 con 0x55e36e4e79a0 2023-10-06T10:39:27.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f574d873b80 10 client.5367 fetch_fsmap learned FSMap version 21 2023-10-06T10:39:27.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f574d873b80 1 -- 172.21.15.92:0/3330858119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55e36e271fe0 con 0x55e36e4e79a0 2023-10-06T10:39:27.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.207+0000 7f571cff9700 1 -- 172.21.15.92:0/3330858119 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e36e4d9690 con 0x55e36e4e79a0 2023-10-06T10:39:27.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.208+0000 7f571cff9700 1 -- 172.21.15.92:0/3330858119 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 21) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f572c027990 con 0x55e36e4e79a0 2023-10-06T10:39:27.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.208+0000 7f574d873b80 10 client.5367 fetch_fsmap finished waiting for FSMap version 21 2023-10-06T10:39:27.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.208+0000 7f574d873b80 10 client.5367 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:39:27.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.208+0000 7f574d873b80 20 client.5367 populate_metadata read hostname 'smithi092' 2023-10-06T10:39:27.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.208+0000 7f574d873b80 1 --2- 172.21.15.92:0/3330858119 >> [v2:172.21.15.92:6835/4010629505,v1:172.21.15.92:6837/4010629505] conn(0x55e36e50ca00 0x55e36e50cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.208+0000 7f574d873b80 4 client.5367 mds_command: new command op to 5121 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:39:27.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.208+0000 7f574d873b80 1 -- 172.21.15.92:0/3330858119 --> [v2:172.21.15.92:6835/4010629505,v1:172.21.15.92:6837/4010629505] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e36e4b65c0 con 0x55e36e50ca00 2023-10-06T10:39:27.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.208+0000 7f5742317700 1 --2- 172.21.15.92:0/3330858119 >> [v2:172.21.15.92:6835/4010629505,v1:172.21.15.92:6837/4010629505] conn(0x55e36e50ca00 0x55e36e50cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.208+0000 7f5742317700 1 --2- 172.21.15.92:0/3330858119 >> [v2:172.21.15.92:6835/4010629505,v1:172.21.15.92:6837/4010629505] conn(0x55e36e50ca00 0x55e36e50cdd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fae1233c3a1ff877 server_cookie=773d87c6f445d714 in_seq=0 out_seq=0 2023-10-06T10:39:27.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.208+0000 7f571cff9700 10 client.5367 ms_handle_connect on v2:172.21.15.92:6835/4010629505 2023-10-06T10:39:27.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.209+0000 7f571cff9700 1 -- 172.21.15.92:0/3330858119 <== mds.1 v2:172.21.15.92:6835/4010629505 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55e36e4b65c0 con 0x55e36e50ca00 2023-10-06T10:39:27.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.209+0000 7f571cff9700 10 client.5367 handle_command_reply: tid=0 2023-10-06T10:39:27.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.209+0000 7f574d873b80 1 client.5367 shutdown 2023-10-06T10:39:27.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.209+0000 7f574d873b80 1 -- 172.21.15.92:0/3330858119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e4e79a0 msgr2=0x55e36e510370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.209+0000 7f574d873b80 1 --2- 172.21.15.92:0/3330858119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e4e79a0 0x55e36e510370 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f572c009fb0 tx=0x7f572c017000 comp rx=0 tx=0).stop 2023-10-06T10:39:27.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.209+0000 7f574d873b80 1 -- 172.21.15.92:0/3330858119 shutdown_connections 2023-10-06T10:39:27.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.209+0000 7f574d873b80 1 --2- 172.21.15.92:0/3330858119 >> [v2:172.21.15.92:6835/4010629505,v1:172.21.15.92:6837/4010629505] conn(0x55e36e50ca00 0x55e36e50cdd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.209+0000 7f574d873b80 1 --2- 172.21.15.92:0/3330858119 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e36e52ba90 0x55e36e5229d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.209+0000 7f571cff9700 0 client.5367 ms_handle_reset on v2:172.21.15.92:6835/4010629505 2023-10-06T10:39:27.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.209+0000 7f574d873b80 1 --2- 172.21.15.92:0/3330858119 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e36e4e5bd0 0x55e36e50b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.209+0000 7f574d873b80 1 --2- 172.21.15.92:0/3330858119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e4e79a0 0x55e36e510370 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.209+0000 7f574d873b80 1 -- 172.21.15.92:0/3330858119 >> 172.21.15.92:0/3330858119 conn(0x55e36e4e6070 msgr2=0x55e36e529440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:27.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.209+0000 7f574d873b80 1 -- 172.21.15.92:0/3330858119 shutdown_connections 2023-10-06T10:39:27.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.209+0000 7f574d873b80 1 -- 172.21.15.92:0/3330858119 wait complete. 2023-10-06T10:39:27.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.209+0000 7f574d873b80 20 client.5367 trim_cache size 0 max 16384 2023-10-06T10:39:27.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.226+0000 7f574d873b80 1 Processor -- start 2023-10-06T10:39:27.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.226+0000 7f574d873b80 1 -- start start 2023-10-06T10:39:27.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.226+0000 7f574d873b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e36e4e5bd0 0x55e36e4e6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.226+0000 7f574d873b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e4e79a0 0x55e36e4e6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.226+0000 7f574d873b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e36e52ba90 0x55e36e4e17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.226+0000 7f574d873b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e36e2b7e30 con 0x55e36e4e79a0 2023-10-06T10:39:27.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.226+0000 7f574d873b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e36e54d540 con 0x55e36e4e5bd0 2023-10-06T10:39:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.226+0000 7f574d873b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e36e4df0c0 con 0x55e36e52ba90 2023-10-06T10:39:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.226+0000 7f5741b16700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e4e79a0 0x55e36e4e6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.226+0000 7f5741b16700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e4e79a0 0x55e36e4e6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47932/0 (socket says 172.21.15.92:47932) 2023-10-06T10:39:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.226+0000 7f5741b16700 1 -- 172.21.15.92:0/221732354 learned_addr learned my addr 172.21.15.92:0/221732354 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.227+0000 7f5742317700 1 --2- 172.21.15.92:0/221732354 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e36e4e5bd0 0x55e36e4e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.227+0000 7f5742b18700 1 --2- 172.21.15.92:0/221732354 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e36e52ba90 0x55e36e4e17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.227+0000 7f5741b16700 1 -- 172.21.15.92:0/221732354 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e36e52ba90 msgr2=0x55e36e4e17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.227+0000 7f5741b16700 1 --2- 172.21.15.92:0/221732354 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e36e52ba90 0x55e36e4e17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.227+0000 7f5741b16700 1 -- 172.21.15.92:0/221732354 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e36e4e5bd0 msgr2=0x55e36e4e6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.227+0000 7f5741b16700 1 --2- 172.21.15.92:0/221732354 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e36e4e5bd0 0x55e36e4e6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.227+0000 7f5741b16700 1 -- 172.21.15.92:0/221732354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e36e403780 con 0x55e36e4e79a0 2023-10-06T10:39:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.227+0000 7f5742317700 1 --2- 172.21.15.92:0/221732354 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e36e4e5bd0 0x55e36e4e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.227+0000 7f5741b16700 1 --2- 172.21.15.92:0/221732354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e4e79a0 0x55e36e4e6980 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f572c0091e0 tx=0x7f572c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1664256db85196e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.227+0000 7f571ffff700 1 -- 172.21.15.92:0/221732354 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c002f30 con 0x55e36e4e79a0 2023-10-06T10:39:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.227+0000 7f571ffff700 1 -- 172.21.15.92:0/221732354 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572c00b430 con 0x55e36e4e79a0 2023-10-06T10:39:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.227+0000 7f571ffff700 1 -- 172.21.15.92:0/221732354 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c027a80 con 0x55e36e4e79a0 2023-10-06T10:39:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.227+0000 7f574d873b80 1 -- 172.21.15.92:0/221732354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e4e79a0 msgr2=0x55e36e4e6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.227+0000 7f574d873b80 1 --2- 172.21.15.92:0/221732354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e4e79a0 0x55e36e4e6980 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f572c0091e0 tx=0x7f572c022000 comp rx=0 tx=0).stop 2023-10-06T10:39:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.227+0000 7f574d873b80 1 -- 172.21.15.92:0/221732354 shutdown_connections 2023-10-06T10:39:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.227+0000 7f574d873b80 1 --2- 172.21.15.92:0/221732354 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e36e52ba90 0x55e36e4e17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.227+0000 7f574d873b80 1 --2- 172.21.15.92:0/221732354 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e36e4e5bd0 0x55e36e4e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.227+0000 7f574d873b80 1 --2- 172.21.15.92:0/221732354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e4e79a0 0x55e36e4e6980 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.227+0000 7f574d873b80 1 -- 172.21.15.92:0/221732354 >> 172.21.15.92:0/221732354 conn(0x55e36e4e6070 msgr2=0x55e36e5280d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.227+0000 7f574d873b80 1 -- 172.21.15.92:0/221732354 shutdown_connections 2023-10-06T10:39:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.227+0000 7f574d873b80 1 -- 172.21.15.92:0/221732354 wait complete. 2023-10-06T10:39:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.228+0000 7f574d873b80 1 -- start start 2023-10-06T10:39:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.228+0000 7f574d873b80 1 Processor -- start 2023-10-06T10:39:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.229+0000 7f574d873b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e36e4e5bd0 0x55e36e58fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.229+0000 7f574d873b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e36e52ba90 0x55e36e4e6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.229+0000 7f574d873b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e529da0 0x55e36e4e6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.229+0000 7f574d873b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e36e521570 con 0x55e36e529da0 2023-10-06T10:39:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.229+0000 7f574d873b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e36e4e1900 con 0x55e36e4e5bd0 2023-10-06T10:39:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.229+0000 7f574d873b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e36e4e1e30 con 0x55e36e52ba90 2023-10-06T10:39:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.229+0000 7f574d873b80 10 client.0 _ll_register_callbacks cb 0x55e36e2d1f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:39:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.229+0000 7f5742b18700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e529da0 0x55e36e4e6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.229+0000 7f5742b18700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e529da0 0x55e36e4e6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47936/0 (socket says 172.21.15.92:47936) 2023-10-06T10:39:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.229+0000 7f5742b18700 1 -- 172.21.15.92:0/2162959609 learned_addr learned my addr 172.21.15.92:0/2162959609 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.229+0000 7f5741b16700 1 --2- 172.21.15.92:0/2162959609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e36e52ba90 0x55e36e4e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.229+0000 7f5742b18700 1 -- 172.21.15.92:0/2162959609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e36e52ba90 msgr2=0x55e36e4e6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.229+0000 7f5742317700 1 --2- 172.21.15.92:0/2162959609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e36e4e5bd0 0x55e36e58fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.229+0000 7f5742b18700 1 --2- 172.21.15.92:0/2162959609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e36e52ba90 0x55e36e4e6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.229+0000 7f5742b18700 1 -- 172.21.15.92:0/2162959609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e36e4e5bd0 msgr2=0x55e36e58fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.229+0000 7f5742b18700 1 --2- 172.21.15.92:0/2162959609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e36e4e5bd0 0x55e36e58fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.229+0000 7f5742b18700 1 -- 172.21.15.92:0/2162959609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e36e271fe0 con 0x55e36e529da0 2023-10-06T10:39:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.229+0000 7f5742b18700 1 --2- 172.21.15.92:0/2162959609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e529da0 0x55e36e4e6980 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5734082dc0 tx=0x7f5734082cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edfd15544d162e2e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.229+0000 7f571cff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:39:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.230+0000 7f571cff9700 1 -- 172.21.15.92:0/2162959609 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573405c7c0 con 0x55e36e529da0 2023-10-06T10:39:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.230+0000 7f571cff9700 1 -- 172.21.15.92:0/2162959609 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5734053b40 con 0x55e36e529da0 2023-10-06T10:39:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.230+0000 7f574d873b80 1 -- 172.21.15.92:0/2162959609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5738015510 con 0x55e36e529da0 2023-10-06T10:39:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.230+0000 7f571cff9700 1 -- 172.21.15.92:0/2162959609 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5734055a00 con 0x55e36e529da0 2023-10-06T10:39:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.230+0000 7f574d873b80 10 client.5373 fetch_fsmap learned FSMap version 21 2023-10-06T10:39:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.230+0000 7f574d873b80 1 -- 172.21.15.92:0/2162959609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55e36e3ea470 con 0x55e36e529da0 2023-10-06T10:39:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.230+0000 7f571cff9700 1 -- 172.21.15.92:0/2162959609 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5734060070 con 0x55e36e529da0 2023-10-06T10:39:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.230+0000 7f571cff9700 1 -- 172.21.15.92:0/2162959609 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 21) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f5734054c90 con 0x55e36e529da0 2023-10-06T10:39:27.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.230+0000 7f574d873b80 10 client.5373 fetch_fsmap finished waiting for FSMap version 21 2023-10-06T10:39:27.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.230+0000 7f574d873b80 10 client.5373 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:39:27.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.230+0000 7f574d873b80 20 client.5373 populate_metadata read hostname 'smithi092' 2023-10-06T10:39:27.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.230+0000 7f574d873b80 1 --2- 172.21.15.92:0/2162959609 >> [v2:172.21.15.92:6835/4010629505,v1:172.21.15.92:6837/4010629505] conn(0x55e36e52f740 0x55e36e5902a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.230+0000 7f574d873b80 4 client.5373 mds_command: new command op to 5121 tid=0[{"prefix": "status"}] 2023-10-06T10:39:27.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.230+0000 7f574d873b80 1 -- 172.21.15.92:0/2162959609 --> [v2:172.21.15.92:6835/4010629505,v1:172.21.15.92:6837/4010629505] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e36e3d7e00 con 0x55e36e52f740 2023-10-06T10:39:27.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.230+0000 7f5742317700 1 --2- 172.21.15.92:0/2162959609 >> [v2:172.21.15.92:6835/4010629505,v1:172.21.15.92:6837/4010629505] conn(0x55e36e52f740 0x55e36e5902a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.230+0000 7f571cff9700 10 client.5373 ms_handle_connect on v2:172.21.15.92:6835/4010629505 2023-10-06T10:39:27.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.230+0000 7f5742317700 1 --2- 172.21.15.92:0/2162959609 >> [v2:172.21.15.92:6835/4010629505,v1:172.21.15.92:6837/4010629505] conn(0x55e36e52f740 0x55e36e5902a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de4616061d8b25de server_cookie=a5873c9d0d1e935d in_seq=0 out_seq=0 2023-10-06T10:39:27.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f571cff9700 1 -- 172.21.15.92:0/2162959609 <== mds.1 v2:172.21.15.92:6835/4010629505 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x55e36e3d7e00 con 0x55e36e52f740 2023-10-06T10:39:27.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f571cff9700 10 client.5373 handle_command_reply: tid=0 2023-10-06T10:39:27.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f574d873b80 1 client.5373 shutdown 2023-10-06T10:39:27.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f574d873b80 1 -- 172.21.15.92:0/2162959609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e529da0 msgr2=0x55e36e4e6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f574d873b80 1 --2- 172.21.15.92:0/2162959609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e529da0 0x55e36e4e6980 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5734082dc0 tx=0x7f5734082cf0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f574d873b80 1 -- 172.21.15.92:0/2162959609 shutdown_connections 2023-10-06T10:39:27.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f574d873b80 1 --2- 172.21.15.92:0/2162959609 >> [v2:172.21.15.92:6835/4010629505,v1:172.21.15.92:6837/4010629505] conn(0x55e36e52f740 0x55e36e5902a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f574d873b80 1 --2- 172.21.15.92:0/2162959609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e36e529da0 0x55e36e4e6980 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f574d873b80 1 --2- 172.21.15.92:0/2162959609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e36e4e5bd0 0x55e36e58fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f574d873b80 1 --2- 172.21.15.92:0/2162959609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e36e52ba90 0x55e36e4e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f574d873b80 1 -- 172.21.15.92:0/2162959609 >> 172.21.15.92:0/2162959609 conn(0x55e36e4e6070 msgr2=0x55e36e511150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:27.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f571cff9700 0 client.5373 ms_handle_reset on v2:172.21.15.92:6835/4010629505 2023-10-06T10:39:27.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f574d873b80 1 -- 172.21.15.92:0/2162959609 shutdown_connections 2023-10-06T10:39:27.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f574d873b80 1 -- 172.21.15.92:0/2162959609 wait complete. 2023-10-06T10:39:27.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f574d873b80 20 client.5373 trim_cache size 0 max 16384 2023-10-06T10:39:27.245 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:39:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:39:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 1, 2023-10-06T10:39:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 5121, 2023-10-06T10:39:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:39:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:39:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:39:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 2.884064403, 2023-10-06T10:39:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 21, 2023-10-06T10:39:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 38, 2023-10-06T10:39:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 38, 2023-10-06T10:39:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 19.293942067 2023-10-06T10:39:27.246 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:39:27.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f574457b700 1 -- 172.21.15.92:0/3200935371 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5720066f30 msgr2=0x7f57200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f574457b700 1 --2- 172.21.15.92:0/3200935371 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5720066f30 0x7f57200693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5738001360 tx=0x7f5738009040 comp rx=0 tx=0).stop 2023-10-06T10:39:27.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f574457b700 1 -- 172.21.15.92:0/3200935371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f573c159560 msgr2=0x7f573c0ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f574457b700 1 --2- 172.21.15.92:0/3200935371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f573c159560 0x7f573c0ae190 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f573400c8e0 tx=0x7f5734001710 comp rx=0 tx=0).stop 2023-10-06T10:39:27.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f574457b700 1 -- 172.21.15.92:0/3200935371 shutdown_connections 2023-10-06T10:39:27.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f574457b700 1 --2- 172.21.15.92:0/3200935371 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5720066f30 0x7f57200693e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f574457b700 1 --2- 172.21.15.92:0/3200935371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f573c159560 0x7f573c0ae190 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f574457b700 1 --2- 172.21.15.92:0/3200935371 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f573c0b0760 0x7f573c157c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f574457b700 1 --2- 172.21.15.92:0/3200935371 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f573c0b49a0 0x7f573c0adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.231+0000 7f574457b700 1 -- 172.21.15.92:0/3200935371 >> 172.21.15.92:0/3200935371 conn(0x7f573c0997f0 msgr2=0x7f573c158b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:27.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.232+0000 7f574457b700 1 -- 172.21.15.92:0/3200935371 shutdown_connections 2023-10-06T10:39:27.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.232+0000 7f574457b700 1 -- 172.21.15.92:0/3200935371 wait complete. 2023-10-06T10:39:27.249 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T10:39:27.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.362+0000 7f666d0fb700 1 Processor -- start 2023-10-06T10:39:27.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.362+0000 7f666d0fb700 1 -- start start 2023-10-06T10:39:27.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.362+0000 7f666d0fb700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f66680b29b0 0x7f6668159120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.362+0000 7f666d0fb700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66680b0da0 con 0x7f66680b25e0 2023-10-06T10:39:27.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.362+0000 7f666d0fb700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66680cbfa0 con 0x7f6668159660 2023-10-06T10:39:27.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.362+0000 7f666d0fb700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66680cb390 con 0x7f66680b29b0 2023-10-06T10:39:27.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.363+0000 7f6666d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f66680b25e0 0x7f66680b0690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35350/0 (socket says 172.21.15.92:35350) 2023-10-06T10:39:27.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.363+0000 7f6666d9d700 1 -- 172.21.15.92:0/3385504017 learned_addr learned my addr 172.21.15.92:0/3385504017 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:27.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.363+0000 7f666659c700 1 --2- 172.21.15.92:0/3385504017 >> v2:172.21.15.124:3300/0 conn(0x7f66680b29b0 0x7f6668159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.363+0000 7f6665d9b700 1 -- 172.21.15.92:0/3385504017 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1351117582 0 0) 0x7f6658001290 con 0x7f66680b25e0 2023-10-06T10:39:27.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.363+0000 7f6665d9b700 1 -- 172.21.15.92:0/3385504017 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6650003570 con 0x7f66680b25e0 2023-10-06T10:39:27.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.363+0000 7f666659c700 1 -- 172.21.15.92:0/3385504017 >> v1:172.21.15.124:6789/0 conn(0x7f6668159660 legacy=0x7f66680cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.363+0000 7f666659c700 1 -- 172.21.15.92:0/3385504017 >> v1:172.21.15.92:6789/0 conn(0x7f66680b25e0 legacy=0x7f66680b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.363+0000 7f666659c700 1 -- 172.21.15.92:0/3385504017 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66680cb670 con 0x7f66680b29b0 2023-10-06T10:39:27.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.363+0000 7f666659c700 1 --2- 172.21.15.92:0/3385504017 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66680b29b0 0x7f6668159120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f665c0098e0 tx=0x7f665c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aba07952ba977d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.363+0000 7f6665d9b700 1 -- 172.21.15.92:0/3385504017 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c00b640 con 0x7f66680b29b0 2023-10-06T10:39:27.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.363+0000 7f6665d9b700 1 -- 172.21.15.92:0/3385504017 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665c00b7a0 con 0x7f66680b29b0 2023-10-06T10:39:27.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.363+0000 7f6665d9b700 1 -- 172.21.15.92:0/3385504017 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c00bb20 con 0x7f66680b29b0 2023-10-06T10:39:27.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.364+0000 7f666d0fb700 1 -- 172.21.15.92:0/3385504017 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66680b29b0 msgr2=0x7f6668159120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.364+0000 7f666d0fb700 1 --2- 172.21.15.92:0/3385504017 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66680b29b0 0x7f6668159120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f665c0098e0 tx=0x7f665c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.364+0000 7f666d0fb700 1 -- 172.21.15.92:0/3385504017 shutdown_connections 2023-10-06T10:39:27.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.364+0000 7f666d0fb700 1 --2- 172.21.15.92:0/3385504017 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66680b29b0 0x7f6668159120 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.364+0000 7f666d0fb700 1 -- 172.21.15.92:0/3385504017 >> 172.21.15.92:0/3385504017 conn(0x7f666806ece0 msgr2=0x7f666806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:27.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.364+0000 7f666d0fb700 1 -- 172.21.15.92:0/3385504017 shutdown_connections 2023-10-06T10:39:27.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.364+0000 7f666d0fb700 1 -- 172.21.15.92:0/3385504017 wait complete. 2023-10-06T10:39:27.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.365+0000 7f666d0fb700 1 Processor -- start 2023-10-06T10:39:27.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.365+0000 7f666d0fb700 1 -- start start 2023-10-06T10:39:27.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.366+0000 7f666d0fb700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66680b25e0 0x7f66680c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.366+0000 7f666d0fb700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66680b29b0 0x7f66681578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.366+0000 7f666d0fb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6668159660 0x7f6668157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.366+0000 7f666d0fb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666815b400 con 0x7f6668159660 2023-10-06T10:39:27.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.366+0000 7f666d0fb700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6668156ae0 con 0x7f66680b25e0 2023-10-06T10:39:27.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.366+0000 7f666d0fb700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66681587a0 con 0x7f66680b29b0 2023-10-06T10:39:27.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.366+0000 7f666759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6668159660 0x7f6668157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.366+0000 7f666759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6668159660 0x7f6668157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47940/0 (socket says 172.21.15.92:47940) 2023-10-06T10:39:27.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.366+0000 7f666759e700 1 -- 172.21.15.92:0/263611526 learned_addr learned my addr 172.21.15.92:0/263611526 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:27.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.366+0000 7f6666d9d700 1 --2- 172.21.15.92:0/263611526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66680b25e0 0x7f66680c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.366+0000 7f666659c700 1 --2- 172.21.15.92:0/263611526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66680b29b0 0x7f66681578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.366+0000 7f666759e700 1 -- 172.21.15.92:0/263611526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66680b29b0 msgr2=0x7f66681578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.366+0000 7f666759e700 1 --2- 172.21.15.92:0/263611526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66680b29b0 0x7f66681578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.366+0000 7f666759e700 1 -- 172.21.15.92:0/263611526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66680b25e0 msgr2=0x7f66680c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.366+0000 7f666759e700 1 --2- 172.21.15.92:0/263611526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66680b25e0 0x7f66680c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.366+0000 7f666759e700 1 -- 172.21.15.92:0/263611526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c009420 con 0x7f6668159660 2023-10-06T10:39:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.367+0000 7f666759e700 1 --2- 172.21.15.92:0/263611526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6668159660 0x7f6668157df0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f666000c870 tx=0x7f6660001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.367+0000 7f664effd700 1 -- 172.21.15.92:0/263611526 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6660012630 con 0x7f6668159660 2023-10-06T10:39:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.367+0000 7f664effd700 1 -- 172.21.15.92:0/263611526 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66600127b0 con 0x7f6668159660 2023-10-06T10:39:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.367+0000 7f666d0fb700 1 -- 172.21.15.92:0/263611526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66680c8c00 con 0x7f6668159660 2023-10-06T10:39:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.367+0000 7f664effd700 1 -- 172.21.15.92:0/263611526 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6660012b70 con 0x7f6668159660 2023-10-06T10:39:27.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.367+0000 7f666d0fb700 1 -- 172.21.15.92:0/263611526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666815adb0 con 0x7f6668159660 2023-10-06T10:39:27.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.368+0000 7f664effd700 1 -- 172.21.15.92:0/263611526 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6660003030 con 0x7f6668159660 2023-10-06T10:39:27.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.368+0000 7f664effd700 1 --2- 172.21.15.92:0/263611526 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6654066f30 0x7f66540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.369+0000 7f6666d9d700 1 --2- 172.21.15.92:0/263611526 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6654066f30 0x7f66540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.369+0000 7f664effd700 1 -- 172.21.15.92:0/263611526 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6660084710 con 0x7f6668159660 2023-10-06T10:39:27.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.369+0000 7f6666d9d700 1 --2- 172.21.15.92:0/263611526 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6654066f30 0x7f66540693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f66580013f0 tx=0x7f6658009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.370+0000 7f66763f3b80 1 Processor -- start 2023-10-06T10:39:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.371+0000 7f66763f3b80 1 -- start start 2023-10-06T10:39:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.371+0000 7f66763f3b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c487c7aa90 0x55c487c7ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.371+0000 7f66763f3b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c487c369a0 0x55c487c33580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.371+0000 7f66763f3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c34bd0 0x55c487c33ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.371+0000 7f66763f3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c487bb3c80 con 0x55c487c34bd0 2023-10-06T10:39:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.371+0000 7f66763f3b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c487b8aa70 con 0x55c487c369a0 2023-10-06T10:39:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.371+0000 7f66763f3b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c487b88270 con 0x55c487c7aa90 2023-10-06T10:39:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.371+0000 7f666759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c34bd0 0x55c487c33ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.371+0000 7f666759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c34bd0 0x55c487c33ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47952/0 (socket says 172.21.15.92:47952) 2023-10-06T10:39:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.371+0000 7f666759e700 1 -- 172.21.15.92:0/3149728306 learned_addr learned my addr 172.21.15.92:0/3149728306 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:27.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.372+0000 7f666759e700 1 -- 172.21.15.92:0/3149728306 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c487c7aa90 msgr2=0x55c487c7ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.372+0000 7f6666d9d700 1 --2- 172.21.15.92:0/3149728306 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c487c7aa90 0x55c487c7ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.372+0000 7f666659c700 1 --2- 172.21.15.92:0/3149728306 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c487c369a0 0x55c487c33580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.372+0000 7f666759e700 1 --2- 172.21.15.92:0/3149728306 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c487c7aa90 0x55c487c7ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.372+0000 7f666759e700 1 -- 172.21.15.92:0/3149728306 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c487c369a0 msgr2=0x55c487c33580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.372+0000 7f666759e700 1 --2- 172.21.15.92:0/3149728306 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c487c369a0 0x55c487c33580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.372+0000 7f666759e700 1 -- 172.21.15.92:0/3149728306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c487a1e270 con 0x55c487c34bd0 2023-10-06T10:39:27.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.372+0000 7f666759e700 1 --2- 172.21.15.92:0/3149728306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c34bd0 0x55c487c33ac0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6660082990 tx=0x7f66600828f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545de3396e6b7842 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.372+0000 7f6647fff700 1 -- 172.21.15.92:0/3149728306 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6660054ee0 con 0x55c487c34bd0 2023-10-06T10:39:27.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.372+0000 7f6647fff700 1 -- 172.21.15.92:0/3149728306 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666005c340 con 0x55c487c34bd0 2023-10-06T10:39:27.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.372+0000 7f6647fff700 1 -- 172.21.15.92:0/3149728306 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6660053740 con 0x55c487c34bd0 2023-10-06T10:39:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.373+0000 7f66763f3b80 1 -- 172.21.15.92:0/3149728306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c34bd0 msgr2=0x55c487c33ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.373+0000 7f66763f3b80 1 --2- 172.21.15.92:0/3149728306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c34bd0 0x55c487c33ac0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6660082990 tx=0x7f66600828f0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.373+0000 7f66763f3b80 1 -- 172.21.15.92:0/3149728306 shutdown_connections 2023-10-06T10:39:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.373+0000 7f66763f3b80 1 --2- 172.21.15.92:0/3149728306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c34bd0 0x55c487c33ac0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.373+0000 7f66763f3b80 1 --2- 172.21.15.92:0/3149728306 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c487c7aa90 0x55c487c7ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.373+0000 7f66763f3b80 1 --2- 172.21.15.92:0/3149728306 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c487c369a0 0x55c487c33580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.373+0000 7f66763f3b80 1 -- 172.21.15.92:0/3149728306 >> 172.21.15.92:0/3149728306 conn(0x55c487c35070 msgr2=0x55c487c35440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.373+0000 7f66763f3b80 1 -- 172.21.15.92:0/3149728306 shutdown_connections 2023-10-06T10:39:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.373+0000 7f66763f3b80 1 -- 172.21.15.92:0/3149728306 wait complete. 2023-10-06T10:39:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.373+0000 7f66763f3b80 1 -- start start 2023-10-06T10:39:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.373+0000 7f66763f3b80 1 Processor -- start 2023-10-06T10:39:27.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.374+0000 7f66763f3b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c487c34bd0 0x55c487c5a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.374+0000 7f66763f3b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c487c369a0 0x55c487c77a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.374+0000 7f66763f3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c7aa90 0x55c487c5d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.374+0000 7f66763f3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c487b84060 con 0x55c487c7aa90 2023-10-06T10:39:27.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.374+0000 7f66763f3b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c487b2f890 con 0x55c487c369a0 2023-10-06T10:39:27.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.374+0000 7f66763f3b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c487b4aea0 con 0x55c487c34bd0 2023-10-06T10:39:27.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.374+0000 7f66763f3b80 10 client.0 _ll_register_callbacks cb 0x55c487a20f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:39:27.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.374+0000 7f666759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c7aa90 0x55c487c5d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f666759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c7aa90 0x55c487c5d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47956/0 (socket says 172.21.15.92:47956) 2023-10-06T10:39:27.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f666759e700 1 -- 172.21.15.92:0/2465024273 learned_addr learned my addr 172.21.15.92:0/2465024273 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:27.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f666659c700 1 --2- 172.21.15.92:0/2465024273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c487c369a0 0x55c487c77a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f6666d9d700 1 --2- 172.21.15.92:0/2465024273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c487c34bd0 0x55c487c5a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f666759e700 1 -- 172.21.15.92:0/2465024273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c487c34bd0 msgr2=0x55c487c5a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f666759e700 1 --2- 172.21.15.92:0/2465024273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c487c34bd0 0x55c487c5a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f666759e700 1 -- 172.21.15.92:0/2465024273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c487c369a0 msgr2=0x55c487c77a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f666759e700 1 --2- 172.21.15.92:0/2465024273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c487c369a0 0x55c487c77a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f666759e700 1 -- 172.21.15.92:0/2465024273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666815adb0 con 0x55c487c7aa90 2023-10-06T10:39:27.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f666759e700 1 --2- 172.21.15.92:0/2465024273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c7aa90 0x55c487c5d670 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f66600518a0 tx=0x7f666006f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b83fc525e79838 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f66457fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:39:27.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f66457fa700 1 -- 172.21.15.92:0/2465024273 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66600534e0 con 0x55c487c7aa90 2023-10-06T10:39:27.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f66457fa700 1 -- 172.21.15.92:0/2465024273 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6660053740 con 0x55c487c7aa90 2023-10-06T10:39:27.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f66763f3b80 1 -- 172.21.15.92:0/2465024273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c487c28690 con 0x55c487c7aa90 2023-10-06T10:39:27.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f66457fa700 1 -- 172.21.15.92:0/2465024273 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6660054aa0 con 0x55c487c7aa90 2023-10-06T10:39:27.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f66763f3b80 10 client.5385 fetch_fsmap learned FSMap version 21 2023-10-06T10:39:27.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f66763f3b80 1 -- 172.21.15.92:0/2465024273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55c4879c0fe0 con 0x55c487c7aa90 2023-10-06T10:39:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f66457fa700 1 -- 172.21.15.92:0/2465024273 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c487c28690 con 0x55c487c7aa90 2023-10-06T10:39:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f66457fa700 1 -- 172.21.15.92:0/2465024273 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 21) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f666005beb0 con 0x55c487c7aa90 2023-10-06T10:39:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f66763f3b80 10 client.5385 fetch_fsmap finished waiting for FSMap version 21 2023-10-06T10:39:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f66763f3b80 10 client.5385 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T10:39:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f66763f3b80 20 client.5385 populate_metadata read hostname 'smithi092' 2023-10-06T10:39:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f66763f3b80 1 --2- 172.21.15.92:0/2465024273 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x55c487c8da20 0x55c487c33a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f66763f3b80 4 client.5385 mds_command: new command op to 5133 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:39:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.375+0000 7f66763f3b80 1 -- 172.21.15.92:0/2465024273 --> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c487c055c0 con 0x55c487c8da20 2023-10-06T10:39:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.376+0000 7f6666d9d700 1 --2- 172.21.15.92:0/2465024273 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x55c487c8da20 0x55c487c33a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.376+0000 7f6666d9d700 1 --2- 172.21.15.92:0/2465024273 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x55c487c8da20 0x55c487c33a70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5596cd9268cac23d server_cookie=21ec8b0159825762 in_seq=0 out_seq=0 2023-10-06T10:39:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.376+0000 7f66457fa700 10 client.5385 ms_handle_connect on v2:172.21.15.124:6832/484869337 2023-10-06T10:39:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.377+0000 7f66457fa700 1 -- 172.21.15.92:0/2465024273 <== mds.0 v2:172.21.15.124:6832/484869337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55c487c055c0 con 0x55c487c8da20 2023-10-06T10:39:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.377+0000 7f66457fa700 10 client.5385 handle_command_reply: tid=0 2023-10-06T10:39:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.377+0000 7f66763f3b80 1 client.5385 shutdown 2023-10-06T10:39:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.377+0000 7f66763f3b80 1 -- 172.21.15.92:0/2465024273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c7aa90 msgr2=0x55c487c5d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.377+0000 7f66763f3b80 1 --2- 172.21.15.92:0/2465024273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c7aa90 0x55c487c5d670 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f66600518a0 tx=0x7f666006f000 comp rx=0 tx=0).stop 2023-10-06T10:39:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.377+0000 7f66763f3b80 1 -- 172.21.15.92:0/2465024273 shutdown_connections 2023-10-06T10:39:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.377+0000 7f66763f3b80 1 --2- 172.21.15.92:0/2465024273 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x55c487c8da20 0x55c487c33a70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.377+0000 7f66763f3b80 1 --2- 172.21.15.92:0/2465024273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c7aa90 0x55c487c5d670 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.377+0000 7f66457fa700 0 client.5385 ms_handle_reset on v2:172.21.15.124:6832/484869337 2023-10-06T10:39:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.377+0000 7f66763f3b80 1 --2- 172.21.15.92:0/2465024273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c487c34bd0 0x55c487c5a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.377+0000 7f66763f3b80 1 --2- 172.21.15.92:0/2465024273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c487c369a0 0x55c487c77a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.377+0000 7f66763f3b80 1 -- 172.21.15.92:0/2465024273 >> 172.21.15.92:0/2465024273 conn(0x55c487c35070 msgr2=0x55c487c78560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:27.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.377+0000 7f66763f3b80 1 -- 172.21.15.92:0/2465024273 shutdown_connections 2023-10-06T10:39:27.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.377+0000 7f66763f3b80 1 -- 172.21.15.92:0/2465024273 wait complete. 2023-10-06T10:39:27.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.377+0000 7f66763f3b80 20 client.5385 trim_cache size 0 max 16384 2023-10-06T10:39:27.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.394+0000 7f66763f3b80 1 Processor -- start 2023-10-06T10:39:27.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.394+0000 7f66763f3b80 1 -- start start 2023-10-06T10:39:27.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.394+0000 7f66763f3b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c487c34bd0 0x55c487c5e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.394+0000 7f66763f3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c369a0 0x55c487c5e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.394+0000 7f66763f3b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c487c7aa90 0x55c487c30620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.394+0000 7f66763f3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c487a06e30 con 0x55c487c369a0 2023-10-06T10:39:27.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.394+0000 7f66763f3b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c487c2f100 con 0x55c487c7aa90 2023-10-06T10:39:27.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.394+0000 7f66763f3b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c487c9c5a0 con 0x55c487c34bd0 2023-10-06T10:39:27.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.395+0000 7f666659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c369a0 0x55c487c5e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.395+0000 7f666659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c369a0 0x55c487c5e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47964/0 (socket says 172.21.15.92:47964) 2023-10-06T10:39:27.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.395+0000 7f666659c700 1 -- 172.21.15.92:0/1614750033 learned_addr learned my addr 172.21.15.92:0/1614750033 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:27.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.395+0000 7f666759e700 1 --2- 172.21.15.92:0/1614750033 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c487c7aa90 0x55c487c30620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.395+0000 7f6666d9d700 1 --2- 172.21.15.92:0/1614750033 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c487c34bd0 0x55c487c5e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.395+0000 7f666659c700 1 -- 172.21.15.92:0/1614750033 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c487c34bd0 msgr2=0x55c487c5e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.395+0000 7f666659c700 1 --2- 172.21.15.92:0/1614750033 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c487c34bd0 0x55c487c5e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.395+0000 7f666659c700 1 -- 172.21.15.92:0/1614750033 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c487c7aa90 msgr2=0x55c487c30620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.395+0000 7f666659c700 1 --2- 172.21.15.92:0/1614750033 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c487c7aa90 0x55c487c30620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.395+0000 7f666659c700 1 -- 172.21.15.92:0/1614750033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c487b52780 con 0x55c487c369a0 2023-10-06T10:39:27.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.395+0000 7f666759e700 1 --2- 172.21.15.92:0/1614750033 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c487c7aa90 0x55c487c30620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:27.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.395+0000 7f6666d9d700 1 --2- 172.21.15.92:0/1614750033 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c487c34bd0 0x55c487c5e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:27.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.395+0000 7f666659c700 1 --2- 172.21.15.92:0/1614750033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c369a0 0x55c487c5e830 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f665c013000 tx=0x7f665c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117a3706977d125 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.395+0000 7f664cff9700 1 -- 172.21.15.92:0/1614750033 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c000c90 con 0x55c487c369a0 2023-10-06T10:39:27.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.395+0000 7f664cff9700 1 -- 172.21.15.92:0/1614750033 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665c020040 con 0x55c487c369a0 2023-10-06T10:39:27.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.395+0000 7f664cff9700 1 -- 172.21.15.92:0/1614750033 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c026440 con 0x55c487c369a0 2023-10-06T10:39:27.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.396+0000 7f66763f3b80 1 -- 172.21.15.92:0/1614750033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c369a0 msgr2=0x55c487c5e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.396+0000 7f66763f3b80 1 --2- 172.21.15.92:0/1614750033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c369a0 0x55c487c5e830 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f665c013000 tx=0x7f665c017000 comp rx=0 tx=0).stop 2023-10-06T10:39:27.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.396+0000 7f66763f3b80 1 -- 172.21.15.92:0/1614750033 shutdown_connections 2023-10-06T10:39:27.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.396+0000 7f66763f3b80 1 --2- 172.21.15.92:0/1614750033 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c487c7aa90 0x55c487c30620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.396+0000 7f66763f3b80 1 --2- 172.21.15.92:0/1614750033 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c487c34bd0 0x55c487c5e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.396+0000 7f66763f3b80 1 --2- 172.21.15.92:0/1614750033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c369a0 0x55c487c5e830 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.396+0000 7f66763f3b80 1 -- 172.21.15.92:0/1614750033 >> 172.21.15.92:0/1614750033 conn(0x55c487c35070 msgr2=0x55c487c2fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:27.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.396+0000 7f66763f3b80 1 -- 172.21.15.92:0/1614750033 shutdown_connections 2023-10-06T10:39:27.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.396+0000 7f66763f3b80 1 -- 172.21.15.92:0/1614750033 wait complete. 2023-10-06T10:39:27.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.396+0000 7f66763f3b80 1 -- start start 2023-10-06T10:39:27.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.396+0000 7f66763f3b80 1 Processor -- start 2023-10-06T10:39:27.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.397+0000 7f66763f3b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c487c34bd0 0x55c487cdca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.397+0000 7f66763f3b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c487c369a0 0x55c487c8b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.397+0000 7f66763f3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c7aa90 0x55c487c5c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.397+0000 7f66763f3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c487c5a380 con 0x55c487c7aa90 2023-10-06T10:39:27.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.397+0000 7f66763f3b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c487c8b700 con 0x55c487c369a0 2023-10-06T10:39:27.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.397+0000 7f66763f3b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c487cdf4b0 con 0x55c487c34bd0 2023-10-06T10:39:27.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.397+0000 7f66763f3b80 10 client.0 _ll_register_callbacks cb 0x55c487a20f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:39:27.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.397+0000 7f666759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c7aa90 0x55c487c5c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.397+0000 7f666759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c7aa90 0x55c487c5c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47976/0 (socket says 172.21.15.92:47976) 2023-10-06T10:39:27.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.397+0000 7f666759e700 1 -- 172.21.15.92:0/941016366 learned_addr learned my addr 172.21.15.92:0/941016366 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:27.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.398+0000 7f666759e700 1 -- 172.21.15.92:0/941016366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c487c34bd0 msgr2=0x55c487cdca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.398+0000 7f666759e700 1 --2- 172.21.15.92:0/941016366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c487c34bd0 0x55c487cdca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.398+0000 7f666759e700 1 -- 172.21.15.92:0/941016366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c487c369a0 msgr2=0x55c487c8b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.398+0000 7f666759e700 1 --2- 172.21.15.92:0/941016366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c487c369a0 0x55c487c8b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.398+0000 7f666759e700 1 -- 172.21.15.92:0/941016366 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66680cb670 con 0x55c487c7aa90 2023-10-06T10:39:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.398+0000 7f666759e700 1 --2- 172.21.15.92:0/941016366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c7aa90 0x55c487c5c1d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6660055330 tx=0x7f666005d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4b303f936fe142 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.398+0000 7f66457fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:39:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.398+0000 7f66457fa700 1 -- 172.21.15.92:0/941016366 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6660082a70 con 0x55c487c7aa90 2023-10-06T10:39:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.398+0000 7f66457fa700 1 -- 172.21.15.92:0/941016366 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f66600534e0 con 0x55c487c7aa90 2023-10-06T10:39:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.398+0000 7f66763f3b80 1 -- 172.21.15.92:0/941016366 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f666005c340 con 0x55c487c7aa90 2023-10-06T10:39:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.398+0000 7f66457fa700 1 -- 172.21.15.92:0/941016366 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f666005beb0 con 0x55c487c7aa90 2023-10-06T10:39:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.398+0000 7f66763f3b80 10 client.5391 fetch_fsmap learned FSMap version 21 2023-10-06T10:39:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.398+0000 7f66763f3b80 1 -- 172.21.15.92:0/941016366 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55c487b39470 con 0x55c487c7aa90 2023-10-06T10:39:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.398+0000 7f66457fa700 1 -- 172.21.15.92:0/941016366 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666005c340 con 0x55c487c7aa90 2023-10-06T10:39:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.398+0000 7f66457fa700 1 -- 172.21.15.92:0/941016366 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 21) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f666005a5c0 con 0x55c487c7aa90 2023-10-06T10:39:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.398+0000 7f66763f3b80 10 client.5391 fetch_fsmap finished waiting for FSMap version 21 2023-10-06T10:39:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.398+0000 7f66763f3b80 10 client.5391 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T10:39:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.398+0000 7f66763f3b80 20 client.5391 populate_metadata read hostname 'smithi092' 2023-10-06T10:39:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.398+0000 7f66763f3b80 1 --2- 172.21.15.92:0/941016366 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x55c487c8da20 0x55c487cdf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.398+0000 7f66763f3b80 4 client.5391 mds_command: new command op to 5133 tid=0[{"prefix": "status"}] 2023-10-06T10:39:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.398+0000 7f66763f3b80 1 -- 172.21.15.92:0/941016366 --> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c487b26e00 con 0x55c487c8da20 2023-10-06T10:39:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.398+0000 7f6666d9d700 1 --2- 172.21.15.92:0/941016366 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x55c487c8da20 0x55c487cdf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.399+0000 7f6666d9d700 1 --2- 172.21.15.92:0/941016366 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x55c487c8da20 0x55c487cdf900 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4c77437722358e1 server_cookie=a6f742eccabb1651 in_seq=0 out_seq=0 2023-10-06T10:39:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.399+0000 7f66457fa700 10 client.5391 ms_handle_connect on v2:172.21.15.124:6832/484869337 2023-10-06T10:39:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.399+0000 7f66457fa700 1 -- 172.21.15.92:0/941016366 <== mds.0 v2:172.21.15.124:6832/484869337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55c487b26e00 con 0x55c487c8da20 2023-10-06T10:39:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.399+0000 7f66457fa700 10 client.5391 handle_command_reply: tid=0 2023-10-06T10:39:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.399+0000 7f66763f3b80 1 client.5391 shutdown 2023-10-06T10:39:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.399+0000 7f66763f3b80 1 -- 172.21.15.92:0/941016366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c7aa90 msgr2=0x55c487c5c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.399+0000 7f66763f3b80 1 --2- 172.21.15.92:0/941016366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c7aa90 0x55c487c5c1d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6660055330 tx=0x7f666005d000 comp rx=0 tx=0).stop 2023-10-06T10:39:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.399+0000 7f66763f3b80 1 -- 172.21.15.92:0/941016366 shutdown_connections 2023-10-06T10:39:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.399+0000 7f66763f3b80 1 --2- 172.21.15.92:0/941016366 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x55c487c8da20 0x55c487cdf900 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.399+0000 7f66763f3b80 1 --2- 172.21.15.92:0/941016366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c487c7aa90 0x55c487c5c1d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.399+0000 7f66457fa700 0 client.5391 ms_handle_reset on v2:172.21.15.124:6832/484869337 2023-10-06T10:39:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.399+0000 7f66763f3b80 1 --2- 172.21.15.92:0/941016366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c487c34bd0 0x55c487cdca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.399+0000 7f66763f3b80 1 --2- 172.21.15.92:0/941016366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c487c369a0 0x55c487c8b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.399+0000 7f66763f3b80 1 -- 172.21.15.92:0/941016366 >> 172.21.15.92:0/941016366 conn(0x55c487c35070 msgr2=0x55c487c305b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:27.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.399+0000 7f66763f3b80 1 -- 172.21.15.92:0/941016366 shutdown_connections 2023-10-06T10:39:27.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.399+0000 7f66763f3b80 1 -- 172.21.15.92:0/941016366 wait complete. 2023-10-06T10:39:27.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.399+0000 7f66763f3b80 20 client.5391 trim_cache size 0 max 16384 2023-10-06T10:39:27.409 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:39:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:39:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T10:39:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 5133, 2023-10-06T10:39:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 11.089077058000001, 2023-10-06T10:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 21, 2023-10-06T10:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 38, 2023-10-06T10:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 38, 2023-10-06T10:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 19.454416563999999 2023-10-06T10:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:39:27.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.400+0000 7f666d0fb700 1 -- 172.21.15.92:0/263611526 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6654066f30 msgr2=0x7f66540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.400+0000 7f666d0fb700 1 --2- 172.21.15.92:0/263611526 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6654066f30 0x7f66540693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f66580013f0 tx=0x7f6658009040 comp rx=0 tx=0).stop 2023-10-06T10:39:27.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.400+0000 7f666d0fb700 1 -- 172.21.15.92:0/263611526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6668159660 msgr2=0x7f6668157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.400+0000 7f666d0fb700 1 --2- 172.21.15.92:0/263611526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6668159660 0x7f6668157df0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f666000c870 tx=0x7f6660001670 comp rx=0 tx=0).stop 2023-10-06T10:39:27.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.400+0000 7f666d0fb700 1 -- 172.21.15.92:0/263611526 shutdown_connections 2023-10-06T10:39:27.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.400+0000 7f666d0fb700 1 --2- 172.21.15.92:0/263611526 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6654066f30 0x7f66540693e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.400+0000 7f666d0fb700 1 --2- 172.21.15.92:0/263611526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6668159660 0x7f6668157df0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.400+0000 7f666d0fb700 1 --2- 172.21.15.92:0/263611526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66680b25e0 0x7f66680c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.400+0000 7f666d0fb700 1 --2- 172.21.15.92:0/263611526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66680b29b0 0x7f66681578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.400+0000 7f666d0fb700 1 -- 172.21.15.92:0/263611526 >> 172.21.15.92:0/263611526 conn(0x7f666806ece0 msgr2=0x7f66680d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:27.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.400+0000 7f666d0fb700 1 -- 172.21.15.92:0/263611526 shutdown_connections 2023-10-06T10:39:27.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.400+0000 7f666d0fb700 1 -- 172.21.15.92:0/263611526 wait complete. 2023-10-06T10:39:27.413 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:39:27.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.527+0000 7ff10359e700 1 Processor -- start 2023-10-06T10:39:27.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.527+0000 7ff10359e700 1 -- start start 2023-10-06T10:39:27.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.528+0000 7ff10359e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff104157040 0x7ff1040cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.528+0000 7ff10359e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff104157410 0x7ff10406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.528+0000 7ff10359e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10415b1d0 con 0x7ff10415b660 2023-10-06T10:39:27.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.528+0000 7ff10359e700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff10406dae0 con 0x7ff104157410 2023-10-06T10:39:27.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.528+0000 7ff10359e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1040ce4b0 con 0x7ff104157040 2023-10-06T10:39:27.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.528+0000 7ff10259c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff104157040 0x7ff1040cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.528+0000 7ff101d9b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff104157410 0x7ff10406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.528+0000 7ff102d9d700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7ff10415b660 0x7ff1040c9f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:47854/0 (socket says 172.21.15.92:47854) 2023-10-06T10:39:27.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.528+0000 7ff102d9d700 1 -- 172.21.15.92:0/2062534231 learned_addr learned my addr 172.21.15.92:0/2062534231 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:27.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.529+0000 7ff10259c700 1 -- 172.21.15.92:0/2062534231 >> v2:172.21.15.124:3300/0 conn(0x7ff104157410 msgr2=0x7ff10406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:27.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.529+0000 7ff10259c700 1 --2- 172.21.15.92:0/2062534231 >> v2:172.21.15.124:3300/0 conn(0x7ff104157410 0x7ff10406d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.529+0000 7ff10259c700 1 -- 172.21.15.92:0/2062534231 >> v1:172.21.15.124:6789/0 conn(0x7ff10415b660 legacy=0x7ff1040c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.529+0000 7ff10259c700 1 -- 172.21.15.92:0/2062534231 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1040ce790 con 0x7ff104157040 2023-10-06T10:39:27.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.529+0000 7ff101d9b700 1 --2- 172.21.15.92:0/2062534231 >> v2:172.21.15.124:3300/0 conn(0x7ff104157410 0x7ff10406d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:39:27.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.529+0000 7ff10259c700 1 --2- 172.21.15.92:0/2062534231 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff104157040 0x7ff1040cde60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0098e0 tx=0x7ff0fc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3ab1264d9df739f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.529+0000 7ff10159a700 1 -- 172.21.15.92:0/2062534231 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc00b640 con 0x7ff104157040 2023-10-06T10:39:27.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.529+0000 7ff10159a700 1 -- 172.21.15.92:0/2062534231 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0fc00b7a0 con 0x7ff104157040 2023-10-06T10:39:27.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.529+0000 7ff10159a700 1 -- 172.21.15.92:0/2062534231 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc00bb40 con 0x7ff104157040 2023-10-06T10:39:27.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.529+0000 7ff10359e700 1 -- 172.21.15.92:0/2062534231 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff104157040 msgr2=0x7ff1040cde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.529+0000 7ff10359e700 1 --2- 172.21.15.92:0/2062534231 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff104157040 0x7ff1040cde60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0098e0 tx=0x7ff0fc0091a0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.530+0000 7ff10359e700 1 -- 172.21.15.92:0/2062534231 shutdown_connections 2023-10-06T10:39:27.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.530+0000 7ff10359e700 1 --2- 172.21.15.92:0/2062534231 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff104157040 0x7ff1040cde60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.530+0000 7ff10359e700 1 --2- 172.21.15.92:0/2062534231 >> v2:172.21.15.124:3300/0 conn(0x7ff104157410 0x7ff10406d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.530+0000 7ff10359e700 1 -- 172.21.15.92:0/2062534231 >> 172.21.15.92:0/2062534231 conn(0x7ff10406b500 msgr2=0x7ff10406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:27.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.530+0000 7ff10359e700 1 -- 172.21.15.92:0/2062534231 shutdown_connections 2023-10-06T10:39:27.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.530+0000 7ff10359e700 1 -- 172.21.15.92:0/2062534231 wait complete. 2023-10-06T10:39:27.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.531+0000 7ff10359e700 1 Processor -- start 2023-10-06T10:39:27.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.531+0000 7ff10359e700 1 -- start start 2023-10-06T10:39:27.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.531+0000 7ff10359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff104157040 0x7ff1040c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.531+0000 7ff10359e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff104157410 0x7ff10414ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.531+0000 7ff10359e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff10415b660 0x7ff104158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.531+0000 7ff10359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff104069ee0 con 0x7ff104157040 2023-10-06T10:39:27.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.531+0000 7ff10359e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1040c7840 con 0x7ff10415b660 2023-10-06T10:39:27.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.531+0000 7ff10359e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1040c7b20 con 0x7ff104157410 2023-10-06T10:39:27.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.531+0000 7ff10259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff104157040 0x7ff1040c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.532+0000 7ff10259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff104157040 0x7ff1040c95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47978/0 (socket says 172.21.15.92:47978) 2023-10-06T10:39:27.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.532+0000 7ff10259c700 1 -- 172.21.15.92:0/3429503190 learned_addr learned my addr 172.21.15.92:0/3429503190 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:27.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.532+0000 7ff102d9d700 1 --2- 172.21.15.92:0/3429503190 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff10415b660 0x7ff104158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.532+0000 7ff101d9b700 1 --2- 172.21.15.92:0/3429503190 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff104157410 0x7ff10414ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.532+0000 7ff10259c700 1 -- 172.21.15.92:0/3429503190 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff104157410 msgr2=0x7ff10414ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.532+0000 7ff10259c700 1 --2- 172.21.15.92:0/3429503190 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff104157410 0x7ff10414ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.532+0000 7ff10259c700 1 -- 172.21.15.92:0/3429503190 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff10415b660 msgr2=0x7ff104158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.532+0000 7ff10259c700 1 --2- 172.21.15.92:0/3429503190 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff10415b660 0x7ff104158f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.532+0000 7ff10259c700 1 -- 172.21.15.92:0/3429503190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec009420 con 0x7ff104157040 2023-10-06T10:39:27.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.532+0000 7ff10259c700 1 --2- 172.21.15.92:0/3429503190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff104157040 0x7ff1040c95c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff1040ce5f0 tx=0x7ff0fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.532+0000 7ff0fa7fc700 1 -- 172.21.15.92:0/3429503190 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc01bc40 con 0x7ff104157040 2023-10-06T10:39:27.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.532+0000 7ff0fa7fc700 1 -- 172.21.15.92:0/3429503190 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0fc01bdc0 con 0x7ff104157040 2023-10-06T10:39:27.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.532+0000 7ff0fa7fc700 1 -- 172.21.15.92:0/3429503190 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc011670 con 0x7ff104157040 2023-10-06T10:39:27.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.532+0000 7ff10359e700 1 -- 172.21.15.92:0/3429503190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0fc009420 con 0x7ff104157040 2023-10-06T10:39:27.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.532+0000 7ff10359e700 1 -- 172.21.15.92:0/3429503190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1040c80e0 con 0x7ff104157040 2023-10-06T10:39:27.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.533+0000 7ff10359e700 1 -- 172.21.15.92:0/3429503190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff10405f230 con 0x7ff104157040 2023-10-06T10:39:27.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.533+0000 7ff0fa7fc700 1 -- 172.21.15.92:0/3429503190 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff0fc011a20 con 0x7ff104157040 2023-10-06T10:39:27.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.534+0000 7ff0fa7fc700 1 --2- 172.21.15.92:0/3429503190 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff0e4067000 0x7ff0e40694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:27.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.534+0000 7ff0fa7fc700 1 -- 172.21.15.92:0/3429503190 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff0fc085bf0 con 0x7ff104157040 2023-10-06T10:39:27.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.534+0000 7ff101d9b700 1 --2- 172.21.15.92:0/3429503190 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff0e4067000 0x7ff0e40694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:27.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.534+0000 7ff101d9b700 1 --2- 172.21.15.92:0/3429503190 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff0e4067000 0x7ff0e40694b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff104157880 tx=0x7ff0ec0091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:27.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.538+0000 7ff0fa7fc700 1 -- 172.21.15.92:0/3429503190 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff0fc052c40 con 0x7ff104157040 2023-10-06T10:39:27.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.761+0000 7ff10359e700 1 -- 172.21.15.92:0/3429503190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff10415ae60 con 0x7ff104157040 2023-10-06T10:39:27.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.762+0000 7ff0fa7fc700 1 -- 172.21.15.92:0/3429503190 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4628 (secure 0 0 0) 0x7ff0fc056260 con 0x7ff104157040 2023-10-06T10:39:27.764 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:39:27.764 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2525196781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2525196781},{"type":"v1","addr":"172.21.15.124:6835","nonce":2525196781}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:39:15.675013+0000","modified":"2023-10-06T10:39:26.352599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5133,"mds_1":5121,"mds_2":5118},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5118":{"gid":5118,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6836/2409440810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2409440810},{"type":"v1","addr":"172.21.15.92:6836","nonce":2409440810}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5121":{"gid":5121,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/4010629505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4010629505},{"type":"v1","addr":"172.21.15.92:6837","nonce":4010629505}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5133":{"gid":5133,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.124:6834/484869337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":484869337},{"type":"v1","addr":"172.21.15.124:6834","nonce":484869337}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T10:39:27.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.764+0000 7ff10359e700 1 -- 172.21.15.92:0/3429503190 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff0e4067000 msgr2=0x7ff0e40694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.764+0000 7ff10359e700 1 --2- 172.21.15.92:0/3429503190 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff0e4067000 0x7ff0e40694b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff104157880 tx=0x7ff0ec0091a0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.764+0000 7ff10359e700 1 -- 172.21.15.92:0/3429503190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff104157040 msgr2=0x7ff1040c95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:27.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.764+0000 7ff10359e700 1 --2- 172.21.15.92:0/3429503190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff104157040 0x7ff1040c95c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff1040ce5f0 tx=0x7ff0fc017000 comp rx=0 tx=0).stop 2023-10-06T10:39:27.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.764+0000 7ff10359e700 1 -- 172.21.15.92:0/3429503190 shutdown_connections 2023-10-06T10:39:27.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.764+0000 7ff10359e700 1 --2- 172.21.15.92:0/3429503190 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff0e4067000 0x7ff0e40694b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.764+0000 7ff10359e700 1 --2- 172.21.15.92:0/3429503190 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff10415b660 0x7ff104158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.764+0000 7ff10359e700 1 --2- 172.21.15.92:0/3429503190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff104157040 0x7ff1040c95c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.764+0000 7ff10359e700 1 --2- 172.21.15.92:0/3429503190 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff104157410 0x7ff10414ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:27.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.764+0000 7ff10359e700 1 -- 172.21.15.92:0/3429503190 >> 172.21.15.92:0/3429503190 conn(0x7ff10406b500 msgr2=0x7ff104152140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:27.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.764+0000 7ff10359e700 1 -- 172.21.15.92:0/3429503190 shutdown_connections 2023-10-06T10:39:27.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:27.764+0000 7ff10359e700 1 -- 172.21.15.92:0/3429503190 wait complete. 2023-10-06T10:39:27.769 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 21 2023-10-06T10:39:27.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:39:27.778 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2023-10-06T10:39:27.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:39:27.806 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2023-10-06T10:39:27.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:39:27.867 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2023-10-06T10:39:27.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:39:27.923 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2023-10-06T10:39:27.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:39:27.980 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2023-10-06T10:39:28.036 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-10-06T10:39:28.177 INFO:teuthology.orchestra.run.smithi124.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:39:28.178 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:39:28.178 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:39:30.179 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-10-06T10:39:30.318 INFO:teuthology.orchestra.run.smithi124.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:39:30.318 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:39:30.319 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:39:32.320 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-10-06T10:39:32.461 INFO:teuthology.orchestra.run.smithi124.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:39:32.462 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:39:32.462 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:39:34.464 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-10-06T10:39:34.605 INFO:teuthology.orchestra.run.smithi124.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2023-10-06T10:39:34.606 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:39:34.606 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:39:36.607 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-10-06T10:39:36.747 INFO:teuthology.orchestra.run.smithi124.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2023-10-06T10:39:36.747 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:39:36.748 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:39:38.748 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-10-06T10:39:38.886 INFO:teuthology.orchestra.run.smithi124.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2023-10-06T10:39:38.887 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2023-10-06T10:39:38.887 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:39:38.887 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:39:39.028 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-10-06T10:39:39.028 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:39:39.028 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:39:41.029 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:39:41.165 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-10-06T10:39:41.166 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:39:41.166 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:39:43.167 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:39:43.311 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}}] 2023-10-06T10:39:43.312 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/d1/d2/d3/d4", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:39:43.312 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:39:45.313 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:39:45.457 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"16","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2023-10-06T10:39:45.458 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2023-10-06T10:39:45.462 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:39:45.462 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-10-06T10:39:45.592 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"17","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-10-06T10:39:45.593 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "17" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:39:45.594 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:39:45.594 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 flush journal 2023-10-06T10:39:45.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.707+0000 7f501f1c7700 1 Processor -- start 2023-10-06T10:39:45.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.707+0000 7f501f1c7700 1 -- start start 2023-10-06T10:39:45.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.708+0000 7f501f1c7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f50180b2620 0x7f50180cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.708+0000 7f501f1c7700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f50180cf1b0 0x7f50180cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.708+0000 7f501f1c7700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50180cc970 con 0x7f50180b29f0 2023-10-06T10:39:45.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.708+0000 7f501f1c7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50180cbd60 con 0x7f50180b2620 2023-10-06T10:39:45.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.708+0000 7f501f1c7700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50180cc040 con 0x7f50180cf1b0 2023-10-06T10:39:45.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.708+0000 7f501e1c5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f50180b2620 0x7f50180cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.708+0000 7f501e1c5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f50180b2620 0x7f50180cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53708/0 (socket says 172.21.15.92:53708) 2023-10-06T10:39:45.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.708+0000 7f501e1c5700 1 -- 172.21.15.92:0/1221417941 learned_addr learned my addr 172.21.15.92:0/1221417941 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:45.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.708+0000 7f501e1c5700 1 -- 172.21.15.92:0/1221417941 >> v2:172.21.15.124:3300/0 conn(0x7f50180cf1b0 msgr2=0x7f50180cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:45.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.708+0000 7f501e1c5700 1 --2- 172.21.15.92:0/1221417941 >> v2:172.21.15.124:3300/0 conn(0x7f50180cf1b0 0x7f50180cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.708+0000 7f501e1c5700 1 -- 172.21.15.92:0/1221417941 >> v1:172.21.15.124:6789/0 conn(0x7f50180b29f0 legacy=0x7f50180ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:45.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.708+0000 7f501e1c5700 1 -- 172.21.15.92:0/1221417941 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50180cc320 con 0x7f50180b2620 2023-10-06T10:39:45.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.709+0000 7f501e1c5700 1 --2- 172.21.15.92:0/1221417941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50180b2620 0x7f50180cccb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5018089470 tx=0x7f5014009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcef22dcbbfa9dff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:45.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.709+0000 7f501d1c3700 1 -- 172.21.15.92:0/1221417941 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5014005c10 con 0x7f50180b2620 2023-10-06T10:39:45.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.709+0000 7f501d1c3700 1 -- 172.21.15.92:0/1221417941 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5014005d70 con 0x7f50180b2620 2023-10-06T10:39:45.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.709+0000 7f501d1c3700 1 -- 172.21.15.92:0/1221417941 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501400b840 con 0x7f50180b2620 2023-10-06T10:39:45.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.709+0000 7f501f1c7700 1 -- 172.21.15.92:0/1221417941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50180b2620 msgr2=0x7f50180cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:45.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.709+0000 7f501f1c7700 1 --2- 172.21.15.92:0/1221417941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50180b2620 0x7f50180cccb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5018089470 tx=0x7f5014009220 comp rx=0 tx=0).stop 2023-10-06T10:39:45.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.709+0000 7f501f1c7700 1 -- 172.21.15.92:0/1221417941 shutdown_connections 2023-10-06T10:39:45.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.709+0000 7f501f1c7700 1 --2- 172.21.15.92:0/1221417941 >> v2:172.21.15.124:3300/0 conn(0x7f50180cf1b0 0x7f50180cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.709+0000 7f501f1c7700 1 --2- 172.21.15.92:0/1221417941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50180b2620 0x7f50180cccb0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.709+0000 7f501f1c7700 1 -- 172.21.15.92:0/1221417941 >> 172.21.15.92:0/1221417941 conn(0x7f5018099520 msgr2=0x7f501806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:45.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.709+0000 7f501f1c7700 1 -- 172.21.15.92:0/1221417941 shutdown_connections 2023-10-06T10:39:45.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.709+0000 7f501f1c7700 1 -- 172.21.15.92:0/1221417941 wait complete. 2023-10-06T10:39:45.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.710+0000 7f501f1c7700 1 Processor -- start 2023-10-06T10:39:45.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.710+0000 7f501f1c7700 1 -- start start 2023-10-06T10:39:45.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.711+0000 7f501f1c7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50180b2620 0x7f501815d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.711+0000 7f501f1c7700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50180b29f0 0x7f501816d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.711+0000 7f501f1c7700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50180cf1b0 0x7f501816a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.711+0000 7f501f1c7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501815da80 con 0x7f50180b2620 2023-10-06T10:39:45.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.711+0000 7f501f1c7700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501816db40 con 0x7f50180cf1b0 2023-10-06T10:39:45.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.711+0000 7f501f1c7700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f501816de20 con 0x7f50180b29f0 2023-10-06T10:39:45.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.711+0000 7f501e1c5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50180b2620 0x7f501815d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.711+0000 7f501e1c5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50180b2620 0x7f501815d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53710/0 (socket says 172.21.15.92:53710) 2023-10-06T10:39:45.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.711+0000 7f501e1c5700 1 -- 172.21.15.92:0/485880668 learned_addr learned my addr 172.21.15.92:0/485880668 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:45.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.712+0000 7f501e9c6700 1 --2- 172.21.15.92:0/485880668 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50180cf1b0 0x7f501816a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.712+0000 7f501e1c5700 1 -- 172.21.15.92:0/485880668 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50180b29f0 msgr2=0x7f501816d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:45.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.712+0000 7f501e1c5700 1 --2- 172.21.15.92:0/485880668 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50180b29f0 0x7f501816d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.712+0000 7f501e1c5700 1 -- 172.21.15.92:0/485880668 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50180cf1b0 msgr2=0x7f501816a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:45.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.712+0000 7f501e1c5700 1 --2- 172.21.15.92:0/485880668 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50180cf1b0 0x7f501816a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.712+0000 7f501e1c5700 1 -- 172.21.15.92:0/485880668 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5014009480 con 0x7f50180b2620 2023-10-06T10:39:45.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.712+0000 7f501e9c6700 1 --2- 172.21.15.92:0/485880668 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50180cf1b0 0x7f501816a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:45.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.712+0000 7f501e1c5700 1 --2- 172.21.15.92:0/485880668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50180b2620 0x7f501815d540 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5014009260 tx=0x7f5014015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:45.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.712+0000 7f500e7fc700 1 -- 172.21.15.92:0/485880668 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5014025070 con 0x7f50180b2620 2023-10-06T10:39:45.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.712+0000 7f500e7fc700 1 -- 172.21.15.92:0/485880668 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5014011630 con 0x7f50180b2620 2023-10-06T10:39:45.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.712+0000 7f501f1c7700 1 -- 172.21.15.92:0/485880668 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f501816e100 con 0x7f50180b2620 2023-10-06T10:39:45.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.712+0000 7f500e7fc700 1 -- 172.21.15.92:0/485880668 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50140119d0 con 0x7f50180b2620 2023-10-06T10:39:45.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.712+0000 7f501f1c7700 1 -- 172.21.15.92:0/485880668 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f501816d7e0 con 0x7f50180b2620 2023-10-06T10:39:45.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.713+0000 7f500e7fc700 1 -- 172.21.15.92:0/485880668 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5014005ee0 con 0x7f50180b2620 2023-10-06T10:39:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.713+0000 7f500e7fc700 1 --2- 172.21.15.92:0/485880668 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4ffc066f30 0x7f4ffc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.714+0000 7f500e7fc700 1 -- 172.21.15.92:0/485880668 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f501408e0e0 con 0x7f50180b2620 2023-10-06T10:39:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.714+0000 7f501d9c4700 1 --2- 172.21.15.92:0/485880668 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4ffc066f30 0x7f4ffc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.714+0000 7f501d9c4700 1 --2- 172.21.15.92:0/485880668 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4ffc066f30 0x7f4ffc0693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f50180b2e10 tx=0x7f5008006ce0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.716+0000 7f5029721b80 1 Processor -- start 2023-10-06T10:39:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.716+0000 7f5029721b80 1 -- start start 2023-10-06T10:39:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.716+0000 7f5029721b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a7dc909a0 0x559a7dc90d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.716+0000 7f5029721b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dcdac10 0x559a7dcb9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.716+0000 7f5029721b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dc8c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.716+0000 7f5029721b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7dc10cd0 con 0x559a7dcb9eb0 2023-10-06T10:39:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.716+0000 7f5029721b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7dbe7ac0 con 0x559a7dcdac10 2023-10-06T10:39:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.716+0000 7f5029721b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a7dbe52c0 con 0x559a7dc909a0 2023-10-06T10:39:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.717+0000 7f501e9c6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dc8c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.717+0000 7f501e9c6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dc8c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53712/0 (socket says 172.21.15.92:53712) 2023-10-06T10:39:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.717+0000 7f501e9c6700 1 -- 172.21.15.92:0/751591954 learned_addr learned my addr 172.21.15.92:0/751591954 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.717+0000 7f501d9c4700 1 --2- 172.21.15.92:0/751591954 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dcdac10 0x559a7dcb9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.717+0000 7f501e1c5700 1 --2- 172.21.15.92:0/751591954 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a7dc909a0 0x559a7dc90d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.717+0000 7f501e9c6700 1 -- 172.21.15.92:0/751591954 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a7dc909a0 msgr2=0x559a7dc90d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.717+0000 7f501e9c6700 1 --2- 172.21.15.92:0/751591954 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a7dc909a0 0x559a7dc90d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.717+0000 7f501e9c6700 1 -- 172.21.15.92:0/751591954 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dcdac10 msgr2=0x559a7dcb9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.717+0000 7f501e9c6700 1 --2- 172.21.15.92:0/751591954 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dcdac10 0x559a7dcb9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.717+0000 7f501e9c6700 1 -- 172.21.15.92:0/751591954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a7da7b2c0 con 0x559a7dcb9eb0 2023-10-06T10:39:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.717+0000 7f501d9c4700 1 --2- 172.21.15.92:0/751591954 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dcdac10 0x559a7dcb9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.717+0000 7f501e9c6700 1 --2- 172.21.15.92:0/751591954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dc8c9e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f501000c400 tx=0x7f5010007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da3729914e5c80e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.717+0000 7f4ffb7fe700 1 -- 172.21.15.92:0/751591954 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501000e640 con 0x559a7dcb9eb0 2023-10-06T10:39:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.717+0000 7f4ffb7fe700 1 -- 172.21.15.92:0/751591954 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501000e7a0 con 0x559a7dcb9eb0 2023-10-06T10:39:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.717+0000 7f4ffb7fe700 1 -- 172.21.15.92:0/751591954 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501000eb40 con 0x559a7dcb9eb0 2023-10-06T10:39:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.718+0000 7f5029721b80 1 -- 172.21.15.92:0/751591954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 msgr2=0x559a7dc8c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.718+0000 7f5029721b80 1 --2- 172.21.15.92:0/751591954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dc8c9e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f501000c400 tx=0x7f5010007930 comp rx=0 tx=0).stop 2023-10-06T10:39:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.718+0000 7f5029721b80 1 -- 172.21.15.92:0/751591954 shutdown_connections 2023-10-06T10:39:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.718+0000 7f5029721b80 1 --2- 172.21.15.92:0/751591954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dc8c9e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.718+0000 7f5029721b80 1 --2- 172.21.15.92:0/751591954 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a7dc909a0 0x559a7dc90d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.718+0000 7f5029721b80 1 --2- 172.21.15.92:0/751591954 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dcdac10 0x559a7dcb9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.718+0000 7f5029721b80 1 -- 172.21.15.92:0/751591954 >> 172.21.15.92:0/751591954 conn(0x559a7dc922c0 msgr2=0x559a7dc92690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.718+0000 7f5029721b80 1 -- 172.21.15.92:0/751591954 shutdown_connections 2023-10-06T10:39:45.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.718+0000 7f5029721b80 1 -- 172.21.15.92:0/751591954 wait complete. 2023-10-06T10:39:45.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.718+0000 7f5029721b80 1 -- start start 2023-10-06T10:39:45.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.719+0000 7f5029721b80 1 Processor -- start 2023-10-06T10:39:45.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.719+0000 7f5029721b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dc909a0 0x559a7dce6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.719+0000 7f5029721b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dcd2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.719+0000 7f5029721b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a7dcdac10 0x559a7dcd17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.719+0000 7f5029721b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7dbe10b0 con 0x559a7dcb9eb0 2023-10-06T10:39:45.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.719+0000 7f5029721b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7db8c8e0 con 0x559a7dc909a0 2023-10-06T10:39:45.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.719+0000 7f5029721b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a7dba7ef0 con 0x559a7dcdac10 2023-10-06T10:39:45.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.720+0000 7f5029721b80 10 client.0 _ll_register_callbacks cb 0x559a7da7df80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:39:45.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.720+0000 7f501d9c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dcd2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.720+0000 7f501d9c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dcd2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53716/0 (socket says 172.21.15.92:53716) 2023-10-06T10:39:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.720+0000 7f501d9c4700 1 -- 172.21.15.92:0/59555441 learned_addr learned my addr 172.21.15.92:0/59555441 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.720+0000 7f501e1c5700 1 --2- 172.21.15.92:0/59555441 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dc909a0 0x559a7dce6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.720+0000 7f501e9c6700 1 --2- 172.21.15.92:0/59555441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a7dcdac10 0x559a7dcd17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.720+0000 7f501d9c4700 1 -- 172.21.15.92:0/59555441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a7dcdac10 msgr2=0x559a7dcd17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.720+0000 7f501d9c4700 1 --2- 172.21.15.92:0/59555441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a7dcdac10 0x559a7dcd17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.720+0000 7f501d9c4700 1 -- 172.21.15.92:0/59555441 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dc909a0 msgr2=0x559a7dce6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.720+0000 7f501d9c4700 1 --2- 172.21.15.92:0/59555441 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dc909a0 0x559a7dce6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.720+0000 7f501d9c4700 1 -- 172.21.15.92:0/59555441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5010007c10 con 0x559a7dcb9eb0 2023-10-06T10:39:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.720+0000 7f501e9c6700 1 --2- 172.21.15.92:0/59555441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a7dcdac10 0x559a7dcd17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.720+0000 7f501d9c4700 1 --2- 172.21.15.92:0/59555441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dcd2ab0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5008012510 tx=0x7f5008010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3f70ef2b21f5db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.720+0000 7f4ff8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:39:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.720+0000 7f4ff8ff9700 1 -- 172.21.15.92:0/59555441 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50080103a0 con 0x559a7dcb9eb0 2023-10-06T10:39:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.720+0000 7f4ff8ff9700 1 -- 172.21.15.92:0/59555441 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f500801fd60 con 0x559a7dcb9eb0 2023-10-06T10:39:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.720+0000 7f5029721b80 1 -- 172.21.15.92:0/59555441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a7dc856e0 con 0x559a7dcb9eb0 2023-10-06T10:39:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.720+0000 7f4ff8ff9700 1 -- 172.21.15.92:0/59555441 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5008010710 con 0x559a7dcb9eb0 2023-10-06T10:39:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.720+0000 7f5029721b80 10 client.5406 fetch_fsmap learned FSMap version 23 2023-10-06T10:39:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.720+0000 7f5029721b80 1 -- 172.21.15.92:0/59555441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x559a7da1e030 con 0x559a7dcb9eb0 2023-10-06T10:39:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.720+0000 7f4ff8ff9700 1 -- 172.21.15.92:0/59555441 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559a7dc856e0 con 0x559a7dcb9eb0 2023-10-06T10:39:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.721+0000 7f4ff8ff9700 1 -- 172.21.15.92:0/59555441 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 23) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f5008010870 con 0x559a7dcb9eb0 2023-10-06T10:39:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.721+0000 7f5029721b80 10 client.5406 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T10:39:45.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.721+0000 7f5029721b80 10 client.5406 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2023-10-06T10:39:45.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.721+0000 7f5029721b80 20 client.5406 populate_metadata read hostname 'smithi092' 2023-10-06T10:39:45.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.721+0000 7f5029721b80 1 --2- 172.21.15.92:0/59555441 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x559a7dccf690 0x559a7dc87760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.721+0000 7f5029721b80 4 client.5406 mds_command: new command op to 5133 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:39:45.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.721+0000 7f5029721b80 1 -- 172.21.15.92:0/59555441 --> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a7dc62610 con 0x559a7dccf690 2023-10-06T10:39:45.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.721+0000 7f501e1c5700 1 --2- 172.21.15.92:0/59555441 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x559a7dccf690 0x559a7dc87760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.721+0000 7f4ff8ff9700 10 client.5406 ms_handle_connect on v2:172.21.15.124:6832/484869337 2023-10-06T10:39:45.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.721+0000 7f501e1c5700 1 --2- 172.21.15.92:0/59555441 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x559a7dccf690 0x559a7dc87760 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58980a23e924bbe1 server_cookie=30a5dd09700e7672 in_seq=0 out_seq=0 2023-10-06T10:39:45.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.722+0000 7f4ff8ff9700 1 -- 172.21.15.92:0/59555441 <== mds.0 v2:172.21.15.124:6832/484869337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x559a7dc62610 con 0x559a7dccf690 2023-10-06T10:39:45.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.722+0000 7f4ff8ff9700 10 client.5406 handle_command_reply: tid=0 2023-10-06T10:39:45.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.722+0000 7f5029721b80 1 client.5406 shutdown 2023-10-06T10:39:45.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.722+0000 7f5029721b80 1 -- 172.21.15.92:0/59555441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 msgr2=0x559a7dcd2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:45.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.722+0000 7f5029721b80 1 --2- 172.21.15.92:0/59555441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dcd2ab0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5008012510 tx=0x7f5008010000 comp rx=0 tx=0).stop 2023-10-06T10:39:45.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.722+0000 7f5029721b80 1 -- 172.21.15.92:0/59555441 shutdown_connections 2023-10-06T10:39:45.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.722+0000 7f5029721b80 1 --2- 172.21.15.92:0/59555441 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x559a7dccf690 0x559a7dc87760 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.722+0000 7f5029721b80 1 --2- 172.21.15.92:0/59555441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a7dcdac10 0x559a7dcd17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.722+0000 7f4ff8ff9700 0 client.5406 ms_handle_reset on v2:172.21.15.124:6832/484869337 2023-10-06T10:39:45.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.722+0000 7f5029721b80 1 --2- 172.21.15.92:0/59555441 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dc909a0 0x559a7dce6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.722+0000 7f5029721b80 1 --2- 172.21.15.92:0/59555441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dcd2ab0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.722+0000 7f5029721b80 1 -- 172.21.15.92:0/59555441 >> 172.21.15.92:0/59555441 conn(0x559a7dc922c0 msgr2=0x559a7dc92690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:45.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.722+0000 7f5029721b80 1 -- 172.21.15.92:0/59555441 shutdown_connections 2023-10-06T10:39:45.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.722+0000 7f5029721b80 1 -- 172.21.15.92:0/59555441 wait complete. 2023-10-06T10:39:45.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.722+0000 7f5029721b80 20 client.5406 trim_cache size 0 max 16384 2023-10-06T10:39:45.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.736+0000 7f5029721b80 1 Processor -- start 2023-10-06T10:39:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.736+0000 7f5029721b80 1 -- start start 2023-10-06T10:39:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.737+0000 7f5029721b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a7dc909a0 0x559a7dcd2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.737+0000 7f5029721b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dc91080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.737+0000 7f5029721b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dccf690 0x559a7dcc02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.737+0000 7f5029721b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7da63e80 con 0x559a7dcb9eb0 2023-10-06T10:39:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.737+0000 7f5029721b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7dce61b0 con 0x559a7dccf690 2023-10-06T10:39:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.737+0000 7f5029721b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a7dcb9bc0 con 0x559a7dc909a0 2023-10-06T10:39:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.737+0000 7f501d9c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dc91080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.737+0000 7f501d9c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dc91080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53720/0 (socket says 172.21.15.92:53720) 2023-10-06T10:39:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.737+0000 7f501d9c4700 1 -- 172.21.15.92:0/826404095 learned_addr learned my addr 172.21.15.92:0/826404095 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.737+0000 7f501e9c6700 1 --2- 172.21.15.92:0/826404095 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dccf690 0x559a7dcc02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.737+0000 7f501d9c4700 1 -- 172.21.15.92:0/826404095 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a7dc909a0 msgr2=0x559a7dcd2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.737+0000 7f501e1c5700 1 --2- 172.21.15.92:0/826404095 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a7dc909a0 0x559a7dcd2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.737+0000 7f501d9c4700 1 --2- 172.21.15.92:0/826404095 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a7dc909a0 0x559a7dcd2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.737+0000 7f501d9c4700 1 -- 172.21.15.92:0/826404095 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dccf690 msgr2=0x559a7dcc02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.737+0000 7f501d9c4700 1 --2- 172.21.15.92:0/826404095 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dccf690 0x559a7dcc02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.737+0000 7f501d9c4700 1 -- 172.21.15.92:0/826404095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a7dbaf7d0 con 0x559a7dcb9eb0 2023-10-06T10:39:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.737+0000 7f501e9c6700 1 --2- 172.21.15.92:0/826404095 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dccf690 0x559a7dcc02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.737+0000 7f501d9c4700 1 --2- 172.21.15.92:0/826404095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dc91080 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5008000fd0 tx=0x7f5008015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c9313eee5ce316 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.737+0000 7f4ffbfff700 1 -- 172.21.15.92:0/826404095 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5008012bb0 con 0x559a7dcb9eb0 2023-10-06T10:39:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.737+0000 7f4ffbfff700 1 -- 172.21.15.92:0/826404095 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5008000c30 con 0x559a7dcb9eb0 2023-10-06T10:39:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.737+0000 7f4ffbfff700 1 -- 172.21.15.92:0/826404095 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5008010710 con 0x559a7dcb9eb0 2023-10-06T10:39:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.738+0000 7f5029721b80 1 -- 172.21.15.92:0/826404095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 msgr2=0x559a7dc91080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.738+0000 7f5029721b80 1 --2- 172.21.15.92:0/826404095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dc91080 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5008000fd0 tx=0x7f5008015000 comp rx=0 tx=0).stop 2023-10-06T10:39:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.738+0000 7f5029721b80 1 -- 172.21.15.92:0/826404095 shutdown_connections 2023-10-06T10:39:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.738+0000 7f5029721b80 1 --2- 172.21.15.92:0/826404095 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dccf690 0x559a7dcc02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.738+0000 7f5029721b80 1 --2- 172.21.15.92:0/826404095 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a7dc909a0 0x559a7dcd2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.738+0000 7f5029721b80 1 --2- 172.21.15.92:0/826404095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dc91080 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.738+0000 7f5029721b80 1 -- 172.21.15.92:0/826404095 >> 172.21.15.92:0/826404095 conn(0x559a7dc922c0 msgr2=0x559a7dc92690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.738+0000 7f5029721b80 1 -- 172.21.15.92:0/826404095 shutdown_connections 2023-10-06T10:39:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.738+0000 7f5029721b80 1 -- 172.21.15.92:0/826404095 wait complete. 2023-10-06T10:39:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.738+0000 7f5029721b80 1 -- start start 2023-10-06T10:39:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.739+0000 7f5029721b80 1 Processor -- start 2023-10-06T10:39:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.739+0000 7f5029721b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dc909a0 0x559a7dcb6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.739+0000 7f5029721b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dcf5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.739+0000 7f5029721b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a7dccf690 0x559a7dcc02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.739+0000 7f5029721b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7dcf9830 con 0x559a7dcb9eb0 2023-10-06T10:39:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.739+0000 7f5029721b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7dcf6620 con 0x559a7dc909a0 2023-10-06T10:39:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.739+0000 7f5029721b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a7dcb5cf0 con 0x559a7dccf690 2023-10-06T10:39:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.740+0000 7f5029721b80 10 client.0 _ll_register_callbacks cb 0x7f5008013a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:39:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.740+0000 7f501d9c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dcf5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.740+0000 7f501d9c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dcf5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53736/0 (socket says 172.21.15.92:53736) 2023-10-06T10:39:45.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.740+0000 7f501d9c4700 1 -- 172.21.15.92:0/4149234578 learned_addr learned my addr 172.21.15.92:0/4149234578 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:45.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.740+0000 7f501e9c6700 1 --2- 172.21.15.92:0/4149234578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a7dccf690 0x559a7dcc02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.740+0000 7f501e1c5700 1 --2- 172.21.15.92:0/4149234578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dc909a0 0x559a7dcb6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.740+0000 7f501d9c4700 1 -- 172.21.15.92:0/4149234578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a7dccf690 msgr2=0x559a7dcc02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:45.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.740+0000 7f501d9c4700 1 --2- 172.21.15.92:0/4149234578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a7dccf690 0x559a7dcc02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.740+0000 7f501d9c4700 1 -- 172.21.15.92:0/4149234578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dc909a0 msgr2=0x559a7dcb6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:45.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.740+0000 7f501d9c4700 1 --2- 172.21.15.92:0/4149234578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dc909a0 0x559a7dcb6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.740+0000 7f501d9c4700 1 -- 172.21.15.92:0/4149234578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a7da1e030 con 0x559a7dcb9eb0 2023-10-06T10:39:45.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.740+0000 7f501e1c5700 1 --2- 172.21.15.92:0/4149234578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dc909a0 0x559a7dcb6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:45.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.740+0000 7f501d9c4700 1 --2- 172.21.15.92:0/4149234578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dcf5a80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f50080094e0 tx=0x7f5008031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1338416357bb33e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:45.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.740+0000 7f4ff8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:39:45.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.740+0000 7f4ff8ff9700 1 -- 172.21.15.92:0/4149234578 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5008012dc0 con 0x559a7dcb9eb0 2023-10-06T10:39:45.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.740+0000 7f4ff8ff9700 1 -- 172.21.15.92:0/4149234578 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f500803cd60 con 0x559a7dcb9eb0 2023-10-06T10:39:45.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.740+0000 7f5029721b80 1 -- 172.21.15.92:0/4149234578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f501000e7a0 con 0x559a7dcb9eb0 2023-10-06T10:39:45.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.740+0000 7f4ff8ff9700 1 -- 172.21.15.92:0/4149234578 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f500802a640 con 0x559a7dcb9eb0 2023-10-06T10:39:45.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.740+0000 7f5029721b80 10 client.5412 fetch_fsmap learned FSMap version 23 2023-10-06T10:39:45.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.740+0000 7f5029721b80 1 -- 172.21.15.92:0/4149234578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x559a7db964c0 con 0x559a7dcb9eb0 2023-10-06T10:39:45.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.740+0000 7f4ff8ff9700 1 -- 172.21.15.92:0/4149234578 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501000e7a0 con 0x559a7dcb9eb0 2023-10-06T10:39:45.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.741+0000 7f4ff8ff9700 1 -- 172.21.15.92:0/4149234578 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 23) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f5008031070 con 0x559a7dcb9eb0 2023-10-06T10:39:45.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.741+0000 7f5029721b80 10 client.5412 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T10:39:45.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.741+0000 7f5029721b80 10 client.5412 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2023-10-06T10:39:45.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.741+0000 7f5029721b80 20 client.5412 populate_metadata read hostname 'smithi092' 2023-10-06T10:39:45.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.741+0000 7f5029721b80 1 --2- 172.21.15.92:0/4149234578 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x559a7dcdac10 0x559a7dcc07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.741+0000 7f5029721b80 4 client.5412 mds_command: new command op to 5133 tid=0[{"prefix": "flush journal"}] 2023-10-06T10:39:45.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.741+0000 7f5029721b80 1 -- 172.21.15.92:0/4149234578 --> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559a7db83e50 con 0x559a7dcdac10 2023-10-06T10:39:45.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.741+0000 7f501e1c5700 1 --2- 172.21.15.92:0/4149234578 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x559a7dcdac10 0x559a7dcc07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.741+0000 7f4ff8ff9700 10 client.5412 ms_handle_connect on v2:172.21.15.124:6832/484869337 2023-10-06T10:39:45.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.741+0000 7f501e1c5700 1 --2- 172.21.15.92:0/4149234578 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x559a7dcdac10 0x559a7dcc07f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cd4ad3a6a22130 server_cookie=445f207f7121f016 in_seq=0 out_seq=0 2023-10-06T10:39:45.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.754+0000 7f4ff8ff9700 1 -- 172.21.15.92:0/4149234578 <== mds.0 v2:172.21.15.124:6832/484869337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559a7db83e50 con 0x559a7dcdac10 2023-10-06T10:39:45.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.754+0000 7f4ff8ff9700 10 client.5412 handle_command_reply: tid=0 2023-10-06T10:39:45.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.754+0000 7f5029721b80 1 client.5412 shutdown 2023-10-06T10:39:45.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f5029721b80 1 -- 172.21.15.92:0/4149234578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 msgr2=0x559a7dcf5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:45.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f5029721b80 1 --2- 172.21.15.92:0/4149234578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dcf5a80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f50080094e0 tx=0x7f5008031000 comp rx=0 tx=0).stop 2023-10-06T10:39:45.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f5029721b80 1 -- 172.21.15.92:0/4149234578 shutdown_connections 2023-10-06T10:39:45.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f5029721b80 1 --2- 172.21.15.92:0/4149234578 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x559a7dcdac10 0x559a7dcc07f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f5029721b80 1 --2- 172.21.15.92:0/4149234578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a7dccf690 0x559a7dcc02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f5029721b80 1 --2- 172.21.15.92:0/4149234578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a7dc909a0 0x559a7dcb6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f5029721b80 1 --2- 172.21.15.92:0/4149234578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7dcb9eb0 0x559a7dcf5a80 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f4ff8ff9700 0 client.5412 ms_handle_reset on v2:172.21.15.124:6832/484869337 2023-10-06T10:39:45.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f5029721b80 1 -- 172.21.15.92:0/4149234578 >> 172.21.15.92:0/4149234578 conn(0x559a7dc922c0 msgr2=0x559a7dc92690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:45.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f5029721b80 1 -- 172.21.15.92:0/4149234578 shutdown_connections 2023-10-06T10:39:45.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f5029721b80 1 -- 172.21.15.92:0/4149234578 wait complete. 2023-10-06T10:39:45.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f5029721b80 20 client.5412 trim_cache size 0 max 16384 2023-10-06T10:39:45.761 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:39:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2023-10-06T10:39:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:39:45.762 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:39:45.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f501f1c7700 1 -- 172.21.15.92:0/485880668 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4ffc066f30 msgr2=0x7f4ffc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:45.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f501f1c7700 1 --2- 172.21.15.92:0/485880668 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4ffc066f30 0x7f4ffc0693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f50180b2e10 tx=0x7f5008006ce0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f501f1c7700 1 -- 172.21.15.92:0/485880668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50180b2620 msgr2=0x7f501815d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:45.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f501f1c7700 1 --2- 172.21.15.92:0/485880668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50180b2620 0x7f501815d540 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5014009260 tx=0x7f5014015000 comp rx=0 tx=0).stop 2023-10-06T10:39:45.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f501f1c7700 1 -- 172.21.15.92:0/485880668 shutdown_connections 2023-10-06T10:39:45.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f501f1c7700 1 --2- 172.21.15.92:0/485880668 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4ffc066f30 0x7f4ffc0693e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f501f1c7700 1 --2- 172.21.15.92:0/485880668 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50180cf1b0 0x7f501816a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f501f1c7700 1 --2- 172.21.15.92:0/485880668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50180b2620 0x7f501815d540 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f501f1c7700 1 --2- 172.21.15.92:0/485880668 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50180b29f0 0x7f501816d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f501f1c7700 1 -- 172.21.15.92:0/485880668 >> 172.21.15.92:0/485880668 conn(0x7f5018099520 msgr2=0x7f50180cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:45.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f501f1c7700 1 -- 172.21.15.92:0/485880668 shutdown_connections 2023-10-06T10:39:45.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.755+0000 7f501f1c7700 1 -- 172.21.15.92:0/485880668 wait complete. 2023-10-06T10:39:45.769 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:1 flush journal 2023-10-06T10:39:45.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.883+0000 7f0b0e337700 1 Processor -- start 2023-10-06T10:39:45.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.883+0000 7f0b0e337700 1 -- start start 2023-10-06T10:39:45.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.884+0000 7f0b0e337700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0b08150940 0x7f0b08159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.884+0000 7f0b0e337700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b080cd4d0 0x7f0b080b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.884+0000 7f0b0e337700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b0815ac30 con 0x7f0b0815a160 2023-10-06T10:39:45.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.884+0000 7f0b0e337700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b080cb6a0 con 0x7f0b08150940 2023-10-06T10:39:45.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.884+0000 7f0b0e337700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b080cb090 con 0x7f0b080cd4d0 2023-10-06T10:39:45.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.884+0000 7f0b077fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0b0815a160 0x7f0b080ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44966/0 (socket says 172.21.15.92:44966) 2023-10-06T10:39:45.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.884+0000 7f0b077fe700 1 -- 172.21.15.92:0/1118231696 learned_addr learned my addr 172.21.15.92:0/1118231696 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:45.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.884+0000 7f0b07fff700 1 --2- 172.21.15.92:0/1118231696 >> v2:172.21.15.124:3300/0 conn(0x7f0b08150940 0x7f0b08159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.884+0000 7f0b0c8d4700 1 --2- 172.21.15.92:0/1118231696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b080cd4d0 0x7f0b080b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.885+0000 7f0b06ffd700 1 -- 172.21.15.92:0/1118231696 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3185245711 0 0) 0x7f0afc001110 con 0x7f0b0815a160 2023-10-06T10:39:45.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.885+0000 7f0b06ffd700 1 -- 172.21.15.92:0/1118231696 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0af0003570 con 0x7f0b0815a160 2023-10-06T10:39:45.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.885+0000 7f0b07fff700 1 -- 172.21.15.92:0/1118231696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b080cd4d0 msgr2=0x7f0b080b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:45.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.885+0000 7f0b07fff700 1 --2- 172.21.15.92:0/1118231696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b080cd4d0 0x7f0b080b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.885+0000 7f0b07fff700 1 -- 172.21.15.92:0/1118231696 >> v1:172.21.15.92:6789/0 conn(0x7f0b0815a160 legacy=0x7f0b080ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:45.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.885+0000 7f0b06ffd700 1 -- 172.21.15.92:0/1118231696 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2988511742 0 0) 0x7f0af0003570 con 0x7f0b0815a160 2023-10-06T10:39:45.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.885+0000 7f0b07fff700 1 -- 172.21.15.92:0/1118231696 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0815a670 con 0x7f0b08150940 2023-10-06T10:39:45.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.885+0000 7f0b0c8d4700 1 --2- 172.21.15.92:0/1118231696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b080cd4d0 0x7f0b080b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:39:45.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.885+0000 7f0b07fff700 1 --2- 172.21.15.92:0/1118231696 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b08150940 0x7f0b08159c20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0af80098d0 tx=0x7f0af8009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48ae0c6b92abeab9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:45.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.885+0000 7f0b06ffd700 1 -- 172.21.15.92:0/1118231696 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc001600 con 0x7f0b08150940 2023-10-06T10:39:45.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.885+0000 7f0b06ffd700 1 -- 172.21.15.92:0/1118231696 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af8011420 con 0x7f0b08150940 2023-10-06T10:39:45.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.885+0000 7f0b06ffd700 1 -- 172.21.15.92:0/1118231696 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af80117c0 con 0x7f0b08150940 2023-10-06T10:39:45.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.886+0000 7f0b0e337700 1 -- 172.21.15.92:0/1118231696 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b08150940 msgr2=0x7f0b08159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:45.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.886+0000 7f0b0e337700 1 --2- 172.21.15.92:0/1118231696 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b08150940 0x7f0b08159c20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0af80098d0 tx=0x7f0af8009120 comp rx=0 tx=0).stop 2023-10-06T10:39:45.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.886+0000 7f0b0e337700 1 -- 172.21.15.92:0/1118231696 shutdown_connections 2023-10-06T10:39:45.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.886+0000 7f0b0e337700 1 --2- 172.21.15.92:0/1118231696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b080cd4d0 0x7f0b080b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.886+0000 7f0b0e337700 1 --2- 172.21.15.92:0/1118231696 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b08150940 0x7f0b08159c20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.886+0000 7f0b0e337700 1 -- 172.21.15.92:0/1118231696 >> 172.21.15.92:0/1118231696 conn(0x7f0b08099700 msgr2=0x7f0b0806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:45.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.886+0000 7f0b0e337700 1 -- 172.21.15.92:0/1118231696 shutdown_connections 2023-10-06T10:39:45.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.886+0000 7f0b0e337700 1 -- 172.21.15.92:0/1118231696 wait complete. 2023-10-06T10:39:45.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.887+0000 7f0b0e337700 1 Processor -- start 2023-10-06T10:39:45.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.887+0000 7f0b0e337700 1 -- start start 2023-10-06T10:39:45.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.888+0000 7f0b0e337700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b080cd4d0 0x7f0b0816d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.888+0000 7f0b0e337700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b0815a160 0x7f0b0814f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.888+0000 7f0b0e337700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b08151320 0x7f0b08158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.888+0000 7f0b0e337700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0816de60 con 0x7f0b08151320 2023-10-06T10:39:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.888+0000 7f0b0e337700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b081518e0 con 0x7f0b080cd4d0 2023-10-06T10:39:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.888+0000 7f0b0e337700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b08151bc0 con 0x7f0b0815a160 2023-10-06T10:39:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.888+0000 7f0b0c8d4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b08151320 0x7f0b08158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.888+0000 7f0b0c8d4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b08151320 0x7f0b08158910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53746/0 (socket says 172.21.15.92:53746) 2023-10-06T10:39:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.888+0000 7f0b0c8d4700 1 -- 172.21.15.92:0/931079775 learned_addr learned my addr 172.21.15.92:0/931079775 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.888+0000 7f0b07fff700 1 --2- 172.21.15.92:0/931079775 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b080cd4d0 0x7f0b0816d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.888+0000 7f0b077fe700 1 --2- 172.21.15.92:0/931079775 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b0815a160 0x7f0b0814f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.888+0000 7f0b0c8d4700 1 -- 172.21.15.92:0/931079775 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b0815a160 msgr2=0x7f0b0814f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.888+0000 7f0b0c8d4700 1 --2- 172.21.15.92:0/931079775 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b0815a160 0x7f0b0814f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.888+0000 7f0b0c8d4700 1 -- 172.21.15.92:0/931079775 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b080cd4d0 msgr2=0x7f0b0816d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.888+0000 7f0b0c8d4700 1 --2- 172.21.15.92:0/931079775 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b080cd4d0 0x7f0b0816d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.888+0000 7f0b0c8d4700 1 -- 172.21.15.92:0/931079775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b00009a20 con 0x7f0b08151320 2023-10-06T10:39:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.888+0000 7f0b07fff700 1 --2- 172.21.15.92:0/931079775 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b080cd4d0 0x7f0b0816d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.888+0000 7f0b077fe700 1 --2- 172.21.15.92:0/931079775 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b0815a160 0x7f0b0814f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.888+0000 7f0b0c8d4700 1 --2- 172.21.15.92:0/931079775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b08151320 0x7f0b08158910 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0b0000ef90 tx=0x7f0b0000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.888+0000 7f0aeffff700 1 -- 172.21.15.92:0/931079775 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b00009d00 con 0x7f0b08151320 2023-10-06T10:39:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.888+0000 7f0aeffff700 1 -- 172.21.15.92:0/931079775 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0000f040 con 0x7f0b08151320 2023-10-06T10:39:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.888+0000 7f0b0e337700 1 -- 172.21.15.92:0/931079775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af80093c0 con 0x7f0b08151320 2023-10-06T10:39:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.889+0000 7f0aeffff700 1 -- 172.21.15.92:0/931079775 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b00014480 con 0x7f0b08151320 2023-10-06T10:39:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.889+0000 7f0b0e337700 1 -- 172.21.15.92:0/931079775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b08152180 con 0x7f0b08151320 2023-10-06T10:39:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.890+0000 7f0aeffff700 1 -- 172.21.15.92:0/931079775 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0b000148b0 con 0x7f0b08151320 2023-10-06T10:39:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.890+0000 7f0aeffff700 1 --2- 172.21.15.92:0/931079775 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0af4066f30 0x7f0af40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.890+0000 7f0aeffff700 1 -- 172.21.15.92:0/931079775 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0b00085560 con 0x7f0b08151320 2023-10-06T10:39:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.890+0000 7f0b07fff700 1 --2- 172.21.15.92:0/931079775 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0af4066f30 0x7f0af40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.891+0000 7f0b07fff700 1 --2- 172.21.15.92:0/931079775 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0af4066f30 0x7f0af40693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0af8009390 tx=0x7f0af8016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:45.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.892+0000 7f0b1762fb80 1 Processor -- start 2023-10-06T10:39:45.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.892+0000 7f0b1762fb80 1 -- start start 2023-10-06T10:39:45.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.893+0000 7f0b1762fb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4e15ae0 0x5632d4e15eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.893+0000 7f0b1762fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dd19f0 0x5632d4dce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.893+0000 7f0b1762fb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5632d4dcfc20 0x5632d4dceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.893+0000 7f0b1762fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d4d4ecd0 con 0x5632d4dd19f0 2023-10-06T10:39:45.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.893+0000 7f0b1762fb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d4d25ac0 con 0x5632d4dcfc20 2023-10-06T10:39:45.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.893+0000 7f0b1762fb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632d4d232c0 con 0x5632d4e15ae0 2023-10-06T10:39:45.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.893+0000 7f0b077fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dd19f0 0x5632d4dce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.893+0000 7f0b077fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dd19f0 0x5632d4dce5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53756/0 (socket says 172.21.15.92:53756) 2023-10-06T10:39:45.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.893+0000 7f0b077fe700 1 -- 172.21.15.92:0/3120131754 learned_addr learned my addr 172.21.15.92:0/3120131754 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:45.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.893+0000 7f0b0c8d4700 1 --2- 172.21.15.92:0/3120131754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5632d4dcfc20 0x5632d4dceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.893+0000 7f0b07fff700 1 --2- 172.21.15.92:0/3120131754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4e15ae0 0x5632d4e15eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.893+0000 7f0b077fe700 1 -- 172.21.15.92:0/3120131754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4e15ae0 msgr2=0x5632d4e15eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:45.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.893+0000 7f0b077fe700 1 --2- 172.21.15.92:0/3120131754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4e15ae0 0x5632d4e15eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.893+0000 7f0b077fe700 1 -- 172.21.15.92:0/3120131754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5632d4dcfc20 msgr2=0x5632d4dceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:45.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.893+0000 7f0b077fe700 1 --2- 172.21.15.92:0/3120131754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5632d4dcfc20 0x5632d4dceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.893+0000 7f0b077fe700 1 -- 172.21.15.92:0/3120131754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632d4bb92c0 con 0x5632d4dd19f0 2023-10-06T10:39:45.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.893+0000 7f0b07fff700 1 --2- 172.21.15.92:0/3120131754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4e15ae0 0x5632d4e15eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:45.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.894+0000 7f0b077fe700 1 --2- 172.21.15.92:0/3120131754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dd19f0 0x5632d4dce5d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0afc007b40 tx=0x7f0afc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ebe00c93d6eb57 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:45.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.894+0000 7f0aed7fa700 1 -- 172.21.15.92:0/3120131754 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc005b10 con 0x5632d4dd19f0 2023-10-06T10:39:45.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.894+0000 7f0aed7fa700 1 -- 172.21.15.92:0/3120131754 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0afc005c70 con 0x5632d4dd19f0 2023-10-06T10:39:45.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.894+0000 7f0aed7fa700 1 -- 172.21.15.92:0/3120131754 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc0070d0 con 0x5632d4dd19f0 2023-10-06T10:39:45.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.894+0000 7f0b1762fb80 1 -- 172.21.15.92:0/3120131754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dd19f0 msgr2=0x5632d4dce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:45.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.894+0000 7f0b1762fb80 1 --2- 172.21.15.92:0/3120131754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dd19f0 0x5632d4dce5d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0afc007b40 tx=0x7f0afc009040 comp rx=0 tx=0).stop 2023-10-06T10:39:45.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.894+0000 7f0b1762fb80 1 -- 172.21.15.92:0/3120131754 shutdown_connections 2023-10-06T10:39:45.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.894+0000 7f0b1762fb80 1 --2- 172.21.15.92:0/3120131754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5632d4dcfc20 0x5632d4dceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.894+0000 7f0b1762fb80 1 --2- 172.21.15.92:0/3120131754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4e15ae0 0x5632d4e15eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.894+0000 7f0b1762fb80 1 --2- 172.21.15.92:0/3120131754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dd19f0 0x5632d4dce5d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.894+0000 7f0b1762fb80 1 -- 172.21.15.92:0/3120131754 >> 172.21.15.92:0/3120131754 conn(0x5632d4dd00c0 msgr2=0x5632d4dd0490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:45.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.895+0000 7f0b1762fb80 1 -- 172.21.15.92:0/3120131754 shutdown_connections 2023-10-06T10:39:45.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.895+0000 7f0b1762fb80 1 -- 172.21.15.92:0/3120131754 wait complete. 2023-10-06T10:39:45.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.895+0000 7f0b1762fb80 1 -- start start 2023-10-06T10:39:45.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.895+0000 7f0b1762fb80 1 Processor -- start 2023-10-06T10:39:45.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.896+0000 7f0b1762fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dcfc20 0x5632d4e11330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.896+0000 7f0b1762fb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5632d4dd19f0 0x5632d4df5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.896+0000 7f0b1762fb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4e15ae0 0x5632d4dfa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.896+0000 7f0b1762fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d4d1f0b0 con 0x5632d4dcfc20 2023-10-06T10:39:45.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.896+0000 7f0b1762fb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d4cca8e0 con 0x5632d4dd19f0 2023-10-06T10:39:45.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.896+0000 7f0b1762fb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632d4ce5ef0 con 0x5632d4e15ae0 2023-10-06T10:39:45.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.896+0000 7f0b1762fb80 10 client.0 _ll_register_callbacks cb 0x5632d4bbbf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:39:45.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.896+0000 7f0b07fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dcfc20 0x5632d4e11330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.896+0000 7f0b07fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dcfc20 0x5632d4e11330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53768/0 (socket says 172.21.15.92:53768) 2023-10-06T10:39:45.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.896+0000 7f0b07fff700 1 -- 172.21.15.92:0/4210067852 learned_addr learned my addr 172.21.15.92:0/4210067852 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:45.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.896+0000 7f0b0c8d4700 1 --2- 172.21.15.92:0/4210067852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4e15ae0 0x5632d4dfa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.896+0000 7f0b077fe700 1 --2- 172.21.15.92:0/4210067852 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5632d4dd19f0 0x5632d4df5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.896+0000 7f0b07fff700 1 -- 172.21.15.92:0/4210067852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4e15ae0 msgr2=0x5632d4dfa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:45.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.896+0000 7f0b07fff700 1 --2- 172.21.15.92:0/4210067852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4e15ae0 0x5632d4dfa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.896+0000 7f0b07fff700 1 -- 172.21.15.92:0/4210067852 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5632d4dd19f0 msgr2=0x5632d4df5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:45.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.896+0000 7f0b07fff700 1 --2- 172.21.15.92:0/4210067852 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5632d4dd19f0 0x5632d4df5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.896+0000 7f0b07fff700 1 -- 172.21.15.92:0/4210067852 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0afc00f040 con 0x5632d4dcfc20 2023-10-06T10:39:45.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.896+0000 7f0b0c8d4700 1 --2- 172.21.15.92:0/4210067852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4e15ae0 0x5632d4dfa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:45.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.896+0000 7f0b077fe700 1 --2- 172.21.15.92:0/4210067852 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5632d4dd19f0 0x5632d4df5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:45.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.896+0000 7f0b07fff700 1 --2- 172.21.15.92:0/4210067852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dcfc20 0x5632d4e11330 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0b080d14c0 tx=0x7f0af8021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d309eba014746536 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:45.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.896+0000 7f0ae67fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:39:45.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.896+0000 7f0ae67fc700 1 -- 172.21.15.92:0/4210067852 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af802d070 con 0x5632d4dcfc20 2023-10-06T10:39:45.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.897+0000 7f0ae67fc700 1 -- 172.21.15.92:0/4210067852 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0af801cad0 con 0x5632d4dcfc20 2023-10-06T10:39:45.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.897+0000 7f0b1762fb80 1 -- 172.21.15.92:0/4210067852 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632d4dc36e0 con 0x5632d4dcfc20 2023-10-06T10:39:45.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.897+0000 7f0ae67fc700 1 -- 172.21.15.92:0/4210067852 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0af8021070 con 0x5632d4dcfc20 2023-10-06T10:39:45.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.897+0000 7f0b1762fb80 10 client.5424 fetch_fsmap learned FSMap version 23 2023-10-06T10:39:45.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.897+0000 7f0b1762fb80 1 -- 172.21.15.92:0/4210067852 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x5632d4b5c030 con 0x5632d4dcfc20 2023-10-06T10:39:45.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.897+0000 7f0ae67fc700 1 -- 172.21.15.92:0/4210067852 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5632d4dc36e0 con 0x5632d4dcfc20 2023-10-06T10:39:45.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.897+0000 7f0ae67fc700 1 -- 172.21.15.92:0/4210067852 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 23) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f0af8032470 con 0x5632d4dcfc20 2023-10-06T10:39:45.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.897+0000 7f0b1762fb80 10 client.5424 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T10:39:45.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.897+0000 7f0b1762fb80 10 client.5424 resolve_mds: resolved 2:1 to role '2:1' aka daemon mds.c 2023-10-06T10:39:45.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.897+0000 7f0b1762fb80 20 client.5424 populate_metadata read hostname 'smithi092' 2023-10-06T10:39:45.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.897+0000 7f0b1762fb80 1 --2- 172.21.15.92:0/4210067852 >> [v2:172.21.15.92:6835/4010629505,v1:172.21.15.92:6837/4010629505] conn(0x5632d4dfab40 0x5632d4dfbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.897+0000 7f0b1762fb80 4 client.5424 mds_command: new command op to 5121 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:39:45.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.897+0000 7f0b1762fb80 1 -- 172.21.15.92:0/4210067852 --> [v2:172.21.15.92:6835/4010629505,v1:172.21.15.92:6837/4010629505] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632d4da0610 con 0x5632d4dfab40 2023-10-06T10:39:45.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.897+0000 7f0b077fe700 1 --2- 172.21.15.92:0/4210067852 >> [v2:172.21.15.92:6835/4010629505,v1:172.21.15.92:6837/4010629505] conn(0x5632d4dfab40 0x5632d4dfbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.898+0000 7f0ae67fc700 10 client.5424 ms_handle_connect on v2:172.21.15.92:6835/4010629505 2023-10-06T10:39:45.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.898+0000 7f0b077fe700 1 --2- 172.21.15.92:0/4210067852 >> [v2:172.21.15.92:6835/4010629505,v1:172.21.15.92:6837/4010629505] conn(0x5632d4dfab40 0x5632d4dfbf80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ed0f8e8c7c74df6a server_cookie=af42fd1a70e47a20 in_seq=0 out_seq=0 2023-10-06T10:39:45.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.898+0000 7f0ae67fc700 1 -- 172.21.15.92:0/4210067852 <== mds.1 v2:172.21.15.92:6835/4010629505 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5632d4da0610 con 0x5632d4dfab40 2023-10-06T10:39:45.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.898+0000 7f0ae67fc700 10 client.5424 handle_command_reply: tid=0 2023-10-06T10:39:45.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.898+0000 7f0b1762fb80 1 client.5424 shutdown 2023-10-06T10:39:45.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.898+0000 7f0b1762fb80 1 -- 172.21.15.92:0/4210067852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dcfc20 msgr2=0x5632d4e11330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:45.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.898+0000 7f0b1762fb80 1 --2- 172.21.15.92:0/4210067852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dcfc20 0x5632d4e11330 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0b080d14c0 tx=0x7f0af8021000 comp rx=0 tx=0).stop 2023-10-06T10:39:45.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.898+0000 7f0b1762fb80 1 -- 172.21.15.92:0/4210067852 shutdown_connections 2023-10-06T10:39:45.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.898+0000 7f0b1762fb80 1 --2- 172.21.15.92:0/4210067852 >> [v2:172.21.15.92:6835/4010629505,v1:172.21.15.92:6837/4010629505] conn(0x5632d4dfab40 0x5632d4dfbf80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.898+0000 7f0b1762fb80 1 --2- 172.21.15.92:0/4210067852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4e15ae0 0x5632d4dfa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.898+0000 7f0ae67fc700 0 client.5424 ms_handle_reset on v2:172.21.15.92:6835/4010629505 2023-10-06T10:39:45.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.898+0000 7f0b1762fb80 1 --2- 172.21.15.92:0/4210067852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dcfc20 0x5632d4e11330 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.898+0000 7f0b1762fb80 1 --2- 172.21.15.92:0/4210067852 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5632d4dd19f0 0x5632d4df5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.898+0000 7f0b1762fb80 1 -- 172.21.15.92:0/4210067852 >> 172.21.15.92:0/4210067852 conn(0x5632d4dd00c0 msgr2=0x5632d4e135b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:45.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.898+0000 7f0b1762fb80 1 -- 172.21.15.92:0/4210067852 shutdown_connections 2023-10-06T10:39:45.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.898+0000 7f0b1762fb80 1 -- 172.21.15.92:0/4210067852 wait complete. 2023-10-06T10:39:45.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.898+0000 7f0b1762fb80 20 client.5424 trim_cache size 0 max 16384 2023-10-06T10:39:45.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.912+0000 7f0b1762fb80 1 Processor -- start 2023-10-06T10:39:45.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.912+0000 7f0b1762fb80 1 -- start start 2023-10-06T10:39:45.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.913+0000 7f0b1762fb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5632d4dd00c0 0x5632d4dca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.913+0000 7f0b1762fb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4dcfc20 0x5632d4dcad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.913+0000 7f0b1762fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dd19f0 0x5632d4df4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.913+0000 7f0b1762fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d4ba1e80 con 0x5632d4dd19f0 2023-10-06T10:39:45.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.913+0000 7f0b1762fb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d4dfc3e0 con 0x5632d4dd00c0 2023-10-06T10:39:45.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.913+0000 7f0b1762fb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632d4dc9110 con 0x5632d4dcfc20 2023-10-06T10:39:45.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.913+0000 7f0b0c8d4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dd19f0 0x5632d4df4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.913+0000 7f0b0c8d4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dd19f0 0x5632d4df4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53774/0 (socket says 172.21.15.92:53774) 2023-10-06T10:39:45.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.913+0000 7f0b0c8d4700 1 -- 172.21.15.92:0/1468291081 learned_addr learned my addr 172.21.15.92:0/1468291081 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:45.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.913+0000 7f0b07fff700 1 --2- 172.21.15.92:0/1468291081 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5632d4dd00c0 0x5632d4dca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.913+0000 7f0b077fe700 1 --2- 172.21.15.92:0/1468291081 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4dcfc20 0x5632d4dcad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.913+0000 7f0b0c8d4700 1 -- 172.21.15.92:0/1468291081 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4dcfc20 msgr2=0x5632d4dcad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.913+0000 7f0b0c8d4700 1 --2- 172.21.15.92:0/1468291081 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4dcfc20 0x5632d4dcad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.913+0000 7f0b0c8d4700 1 -- 172.21.15.92:0/1468291081 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5632d4dd00c0 msgr2=0x5632d4dca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.913+0000 7f0b0c8d4700 1 --2- 172.21.15.92:0/1468291081 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5632d4dd00c0 0x5632d4dca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.913+0000 7f0b0c8d4700 1 -- 172.21.15.92:0/1468291081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632d4ced7d0 con 0x5632d4dd19f0 2023-10-06T10:39:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.913+0000 7f0b077fe700 1 --2- 172.21.15.92:0/1468291081 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4dcfc20 0x5632d4dcad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.913+0000 7f0b07fff700 1 --2- 172.21.15.92:0/1468291081 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5632d4dd00c0 0x5632d4dca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.914+0000 7f0b0c8d4700 1 --2- 172.21.15.92:0/1468291081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dd19f0 0x5632d4df4080 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0b00083810 tx=0x7f0b00083740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7e80b544702cec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.914+0000 7f0aedffb700 1 -- 172.21.15.92:0/1468291081 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0005d190 con 0x5632d4dd19f0 2023-10-06T10:39:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.914+0000 7f0aedffb700 1 -- 172.21.15.92:0/1468291081 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b00054330 con 0x5632d4dd19f0 2023-10-06T10:39:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.914+0000 7f0aedffb700 1 -- 172.21.15.92:0/1468291081 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b00024db0 con 0x5632d4dd19f0 2023-10-06T10:39:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.914+0000 7f0b1762fb80 1 -- 172.21.15.92:0/1468291081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dd19f0 msgr2=0x5632d4df4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.914+0000 7f0b1762fb80 1 --2- 172.21.15.92:0/1468291081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dd19f0 0x5632d4df4080 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0b00083810 tx=0x7f0b00083740 comp rx=0 tx=0).stop 2023-10-06T10:39:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.914+0000 7f0b1762fb80 1 -- 172.21.15.92:0/1468291081 shutdown_connections 2023-10-06T10:39:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.914+0000 7f0b1762fb80 1 --2- 172.21.15.92:0/1468291081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dd19f0 0x5632d4df4080 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.914+0000 7f0b1762fb80 1 --2- 172.21.15.92:0/1468291081 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5632d4dd00c0 0x5632d4dca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.914+0000 7f0b1762fb80 1 --2- 172.21.15.92:0/1468291081 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4dcfc20 0x5632d4dcad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.914+0000 7f0b1762fb80 1 -- 172.21.15.92:0/1468291081 >> 172.21.15.92:0/1468291081 conn(0x5632d4e2ee60 msgr2=0x5632d4dcb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.914+0000 7f0b1762fb80 1 -- 172.21.15.92:0/1468291081 shutdown_connections 2023-10-06T10:39:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.914+0000 7f0b1762fb80 1 -- 172.21.15.92:0/1468291081 wait complete. 2023-10-06T10:39:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.915+0000 7f0b1762fb80 1 -- start start 2023-10-06T10:39:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.915+0000 7f0b1762fb80 1 Processor -- start 2023-10-06T10:39:45.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.916+0000 7f0b1762fb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4dcfc20 0x5632d4e1b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.916+0000 7f0b1762fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dd00c0 0x5632d4dcefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.916+0000 7f0b1762fb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5632d4dd19f0 0x5632d4e14c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.916+0000 7f0b1762fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d4e0ed40 con 0x5632d4dd00c0 2023-10-06T10:39:45.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.916+0000 7f0b1762fb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d4e1bb60 con 0x5632d4dd19f0 2023-10-06T10:39:45.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.916+0000 7f0b1762fb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632d4df4a90 con 0x5632d4dcfc20 2023-10-06T10:39:45.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.916+0000 7f0b1762fb80 10 client.0 _ll_register_callbacks cb 0x5632d4bbbf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:39:45.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.916+0000 7f0b077fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dd00c0 0x5632d4dcefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.916+0000 7f0b077fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dd00c0 0x5632d4dcefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53778/0 (socket says 172.21.15.92:53778) 2023-10-06T10:39:45.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.916+0000 7f0b077fe700 1 -- 172.21.15.92:0/3772321578 learned_addr learned my addr 172.21.15.92:0/3772321578 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:45.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.916+0000 7f0b0c8d4700 1 --2- 172.21.15.92:0/3772321578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5632d4dd19f0 0x5632d4e14c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.916+0000 7f0b07fff700 1 --2- 172.21.15.92:0/3772321578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4dcfc20 0x5632d4e1b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.916+0000 7f0b077fe700 1 -- 172.21.15.92:0/3772321578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4dcfc20 msgr2=0x5632d4e1b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:45.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.916+0000 7f0b077fe700 1 --2- 172.21.15.92:0/3772321578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4dcfc20 0x5632d4e1b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.916+0000 7f0b077fe700 1 -- 172.21.15.92:0/3772321578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5632d4dd19f0 msgr2=0x5632d4e14c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:45.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.916+0000 7f0b077fe700 1 --2- 172.21.15.92:0/3772321578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5632d4dd19f0 0x5632d4e14c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.916+0000 7f0b077fe700 1 -- 172.21.15.92:0/3772321578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b08152180 con 0x5632d4dd00c0 2023-10-06T10:39:45.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.916+0000 7f0b077fe700 1 --2- 172.21.15.92:0/3772321578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dd00c0 0x5632d4dcefe0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x5632d4dd1f40 tx=0x7f0afc026250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e3cf2870dd5d67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:45.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.916+0000 7f0ae67fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:39:45.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.916+0000 7f0ae67fc700 1 -- 172.21.15.92:0/3772321578 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc005b10 con 0x5632d4dd00c0 2023-10-06T10:39:45.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.916+0000 7f0ae67fc700 1 -- 172.21.15.92:0/3772321578 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0afc0074c0 con 0x5632d4dd00c0 2023-10-06T10:39:45.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.916+0000 7f0b1762fb80 1 -- 172.21.15.92:0/3772321578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0afc005c70 con 0x5632d4dd00c0 2023-10-06T10:39:45.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.917+0000 7f0ae67fc700 1 -- 172.21.15.92:0/3772321578 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0afc004830 con 0x5632d4dd00c0 2023-10-06T10:39:45.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.917+0000 7f0b1762fb80 10 client.5430 fetch_fsmap learned FSMap version 23 2023-10-06T10:39:45.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.917+0000 7f0b1762fb80 1 -- 172.21.15.92:0/3772321578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x5632d4cd44c0 con 0x5632d4dd00c0 2023-10-06T10:39:45.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.917+0000 7f0ae67fc700 1 -- 172.21.15.92:0/3772321578 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0afc005c70 con 0x5632d4dd00c0 2023-10-06T10:39:45.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.917+0000 7f0ae67fc700 1 -- 172.21.15.92:0/3772321578 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 23) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f0afc007c20 con 0x5632d4dd00c0 2023-10-06T10:39:45.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.917+0000 7f0b1762fb80 10 client.5430 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T10:39:45.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.917+0000 7f0b1762fb80 10 client.5430 resolve_mds: resolved 2:1 to role '2:1' aka daemon mds.c 2023-10-06T10:39:45.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.917+0000 7f0b1762fb80 20 client.5430 populate_metadata read hostname 'smithi092' 2023-10-06T10:39:45.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.917+0000 7f0b1762fb80 1 --2- 172.21.15.92:0/3772321578 >> [v2:172.21.15.92:6835/4010629505,v1:172.21.15.92:6837/4010629505] conn(0x5632d4e15ae0 0x5632d4e15140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:45.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.917+0000 7f0b1762fb80 4 client.5430 mds_command: new command op to 5121 tid=0[{"prefix": "flush journal"}] 2023-10-06T10:39:45.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.917+0000 7f0b1762fb80 1 -- 172.21.15.92:0/3772321578 --> [v2:172.21.15.92:6835/4010629505,v1:172.21.15.92:6837/4010629505] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5632d4cc1e50 con 0x5632d4e15ae0 2023-10-06T10:39:45.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.917+0000 7f0b07fff700 1 --2- 172.21.15.92:0/3772321578 >> [v2:172.21.15.92:6835/4010629505,v1:172.21.15.92:6837/4010629505] conn(0x5632d4e15ae0 0x5632d4e15140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:45.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.917+0000 7f0b07fff700 1 --2- 172.21.15.92:0/3772321578 >> [v2:172.21.15.92:6835/4010629505,v1:172.21.15.92:6837/4010629505] conn(0x5632d4e15ae0 0x5632d4e15140 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4024e0c1e9ad44d3 server_cookie=37144921cd4a45a0 in_seq=0 out_seq=0 2023-10-06T10:39:45.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.917+0000 7f0ae67fc700 10 client.5430 ms_handle_connect on v2:172.21.15.92:6835/4010629505 2023-10-06T10:39:45.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.926+0000 7f0ae67fc700 1 -- 172.21.15.92:0/3772321578 <== mds.1 v2:172.21.15.92:6835/4010629505 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5632d4cc1e50 con 0x5632d4e15ae0 2023-10-06T10:39:45.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.926+0000 7f0ae67fc700 10 client.5430 handle_command_reply: tid=0 2023-10-06T10:39:45.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.926+0000 7f0b1762fb80 1 client.5430 shutdown 2023-10-06T10:39:45.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.926+0000 7f0b1762fb80 1 -- 172.21.15.92:0/3772321578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dd00c0 msgr2=0x5632d4dcefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:45.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.926+0000 7f0b1762fb80 1 --2- 172.21.15.92:0/3772321578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dd00c0 0x5632d4dcefe0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x5632d4dd1f40 tx=0x7f0afc026250 comp rx=0 tx=0).stop 2023-10-06T10:39:45.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.926+0000 7f0b1762fb80 1 -- 172.21.15.92:0/3772321578 shutdown_connections 2023-10-06T10:39:45.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.926+0000 7f0b1762fb80 1 --2- 172.21.15.92:0/3772321578 >> [v2:172.21.15.92:6835/4010629505,v1:172.21.15.92:6837/4010629505] conn(0x5632d4e15ae0 0x5632d4e15140 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.926+0000 7f0b1762fb80 1 --2- 172.21.15.92:0/3772321578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5632d4dd19f0 0x5632d4e14c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.926+0000 7f0b1762fb80 1 --2- 172.21.15.92:0/3772321578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632d4dcfc20 0x5632d4e1b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.926+0000 7f0ae67fc700 0 client.5430 ms_handle_reset on v2:172.21.15.92:6835/4010629505 2023-10-06T10:39:45.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.926+0000 7f0b1762fb80 1 --2- 172.21.15.92:0/3772321578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5632d4dd00c0 0x5632d4dcefe0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.926+0000 7f0b1762fb80 1 -- 172.21.15.92:0/3772321578 >> 172.21.15.92:0/3772321578 conn(0x5632d4e2ee60 msgr2=0x5632d4df7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:45.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.927+0000 7f0b1762fb80 1 -- 172.21.15.92:0/3772321578 shutdown_connections 2023-10-06T10:39:45.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.927+0000 7f0b1762fb80 1 -- 172.21.15.92:0/3772321578 wait complete. 2023-10-06T10:39:45.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.927+0000 7f0b1762fb80 20 client.5430 trim_cache size 0 max 16384 2023-10-06T10:39:45.930 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:39:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2023-10-06T10:39:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:39:45.930 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:39:45.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.927+0000 7f0b0e337700 1 -- 172.21.15.92:0/931079775 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0af4066f30 msgr2=0x7f0af40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:45.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.927+0000 7f0b0e337700 1 --2- 172.21.15.92:0/931079775 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0af4066f30 0x7f0af40693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0af8009390 tx=0x7f0af8016000 comp rx=0 tx=0).stop 2023-10-06T10:39:45.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.927+0000 7f0b0e337700 1 -- 172.21.15.92:0/931079775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b08151320 msgr2=0x7f0b08158910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:45.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.927+0000 7f0b0e337700 1 --2- 172.21.15.92:0/931079775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b08151320 0x7f0b08158910 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0b0000ef90 tx=0x7f0b0000b040 comp rx=0 tx=0).stop 2023-10-06T10:39:45.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.927+0000 7f0b0e337700 1 -- 172.21.15.92:0/931079775 shutdown_connections 2023-10-06T10:39:45.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.927+0000 7f0b0e337700 1 --2- 172.21.15.92:0/931079775 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0af4066f30 0x7f0af40693e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.927+0000 7f0b0e337700 1 --2- 172.21.15.92:0/931079775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b08151320 0x7f0b08158910 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.927+0000 7f0b0e337700 1 --2- 172.21.15.92:0/931079775 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b080cd4d0 0x7f0b0816d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.927+0000 7f0b0e337700 1 --2- 172.21.15.92:0/931079775 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b0815a160 0x7f0b0814f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:45.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.927+0000 7f0b0e337700 1 -- 172.21.15.92:0/931079775 >> 172.21.15.92:0/931079775 conn(0x7f0b08099700 msgr2=0x7f0b080cbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:45.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.927+0000 7f0b0e337700 1 -- 172.21.15.92:0/931079775 shutdown_connections 2023-10-06T10:39:45.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:45.927+0000 7f0b0e337700 1 -- 172.21.15.92:0/931079775 wait complete. 2023-10-06T10:39:45.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:2 flush journal 2023-10-06T10:39:46.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.055+0000 7f3f5a738700 1 Processor -- start 2023-10-06T10:39:46.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.055+0000 7f3f5a738700 1 -- start start 2023-10-06T10:39:46.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.055+0000 7f3f5a738700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3f54150940 0x7f3f54159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:46.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.055+0000 7f3f5a738700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f5415ac30 con 0x7f3f540cd4d0 2023-10-06T10:39:46.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.055+0000 7f3f5a738700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f540cb6a0 con 0x7f3f5415a160 2023-10-06T10:39:46.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.055+0000 7f3f5a738700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f540cb090 con 0x7f3f54150940 2023-10-06T10:39:46.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.056+0000 7f3f58cd5700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3f540cd4d0 0x7f3f540b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44976/0 (socket says 172.21.15.92:44976) 2023-10-06T10:39:46.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.056+0000 7f3f58cd5700 1 -- 172.21.15.92:0/3414279807 learned_addr learned my addr 172.21.15.92:0/3414279807 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:39:46.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.056+0000 7f3f53fff700 1 --2- 172.21.15.92:0/3414279807 >> v2:172.21.15.124:3300/0 conn(0x7f3f54150940 0x7f3f54159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:46.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.056+0000 7f3f52ffd700 1 -- 172.21.15.92:0/3414279807 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3026551649 0 0) 0x7f3f4c001490 con 0x7f3f540cd4d0 2023-10-06T10:39:46.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.056+0000 7f3f52ffd700 1 -- 172.21.15.92:0/3414279807 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f3c003570 con 0x7f3f540cd4d0 2023-10-06T10:39:46.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.056+0000 7f3f52ffd700 1 -- 172.21.15.92:0/3414279807 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4001544980 0 0) 0x7f3f3c003570 con 0x7f3f540cd4d0 2023-10-06T10:39:46.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.056+0000 7f3f52ffd700 1 -- 172.21.15.92:0/3414279807 >> v2:172.21.15.124:3300/0 conn(0x7f3f54150940 msgr2=0x7f3f54159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:46.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.056+0000 7f3f52ffd700 1 --2- 172.21.15.92:0/3414279807 >> v2:172.21.15.124:3300/0 conn(0x7f3f54150940 0x7f3f54159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.056+0000 7f3f52ffd700 1 -- 172.21.15.92:0/3414279807 >> v1:172.21.15.124:6789/0 conn(0x7f3f5415a160 legacy=0x7f3f540ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:46.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.056+0000 7f3f52ffd700 1 -- 172.21.15.92:0/3414279807 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5415a670 con 0x7f3f540cd4d0 2023-10-06T10:39:46.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.056+0000 7f3f52ffd700 1 -- 172.21.15.92:0/3414279807 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f3f4c001980 con 0x7f3f540cd4d0 2023-10-06T10:39:46.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.056+0000 7f3f52ffd700 1 -- 172.21.15.92:0/3414279807 >> v1:172.21.15.92:6789/0 conn(0x7f3f540cd4d0 legacy=0x7f3f540b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:46.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.056+0000 7f3f52ffd700 1 --2- 172.21.15.92:0/3414279807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c003a60 0x7f3f3c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:46.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.056+0000 7f3f52ffd700 1 --2- 172.21.15.92:0/3414279807 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f3c004810 0x7f3f3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:46.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.056+0000 7f3f52ffd700 1 --2- 172.21.15.92:0/3414279807 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f3c0051c0 0x7f3f3c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:46.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.056+0000 7f3f52ffd700 1 -- 172.21.15.92:0/3414279807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540cb090 con 0x7f3f3c003a60 2023-10-06T10:39:46.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.056+0000 7f3f52ffd700 1 -- 172.21.15.92:0/3414279807 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c00c0d0 con 0x7f3f3c0051c0 2023-10-06T10:39:46.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.056+0000 7f3f52ffd700 1 -- 172.21.15.92:0/3414279807 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c00c290 con 0x7f3f3c004810 2023-10-06T10:39:46.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.056+0000 7f3f53fff700 1 --2- 172.21.15.92:0/3414279807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c003a60 0x7f3f3c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:46.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.056+0000 7f3f58cd5700 1 --2- 172.21.15.92:0/3414279807 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f3c0051c0 0x7f3f3c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:46.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.056+0000 7f3f537fe700 1 --2- 172.21.15.92:0/3414279807 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f3c004810 0x7f3f3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:46.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.057+0000 7f3f53fff700 1 -- 172.21.15.92:0/3414279807 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f3c004810 msgr2=0x7f3f3c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:46.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.057+0000 7f3f53fff700 1 --2- 172.21.15.92:0/3414279807 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f3c004810 0x7f3f3c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.057+0000 7f3f53fff700 1 -- 172.21.15.92:0/3414279807 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f3c0051c0 msgr2=0x7f3f3c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:46.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.057+0000 7f3f53fff700 1 --2- 172.21.15.92:0/3414279807 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f3c0051c0 0x7f3f3c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.057+0000 7f3f58cd5700 1 -- 172.21.15.92:0/3414279807 reap_dead start 2023-10-06T10:39:46.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.057+0000 7f3f53fff700 1 -- 172.21.15.92:0/3414279807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3c003570 con 0x7f3f3c003a60 2023-10-06T10:39:46.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.057+0000 7f3f53fff700 1 --2- 172.21.15.92:0/3414279807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c003a60 0x7f3f3c003e70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009bf0 tx=0x7f3f44009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae02442b759840f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:46.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.057+0000 7f3f52ffd700 1 -- 172.21.15.92:0/3414279807 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44005a00 con 0x7f3f3c003a60 2023-10-06T10:39:46.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.057+0000 7f3f52ffd700 1 -- 172.21.15.92:0/3414279807 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f44005b60 con 0x7f3f3c003a60 2023-10-06T10:39:46.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.057+0000 7f3f52ffd700 1 -- 172.21.15.92:0/3414279807 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4400e710 con 0x7f3f3c003a60 2023-10-06T10:39:46.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.057+0000 7f3f5a738700 1 -- 172.21.15.92:0/3414279807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c003a60 msgr2=0x7f3f3c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:46.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.057+0000 7f3f5a738700 1 --2- 172.21.15.92:0/3414279807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c003a60 0x7f3f3c003e70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009bf0 tx=0x7f3f44009120 comp rx=0 tx=0).stop 2023-10-06T10:39:46.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.057+0000 7f3f5a738700 1 -- 172.21.15.92:0/3414279807 shutdown_connections 2023-10-06T10:39:46.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.057+0000 7f3f5a738700 1 --2- 172.21.15.92:0/3414279807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c003a60 0x7f3f3c003e70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.057+0000 7f3f5a738700 1 -- 172.21.15.92:0/3414279807 >> 172.21.15.92:0/3414279807 conn(0x7f3f54099700 msgr2=0x7f3f5406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:46.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.057+0000 7f3f5a738700 1 -- 172.21.15.92:0/3414279807 shutdown_connections 2023-10-06T10:39:46.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.057+0000 7f3f5a738700 1 -- 172.21.15.92:0/3414279807 wait complete. 2023-10-06T10:39:46.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.058+0000 7f3f5a738700 1 Processor -- start 2023-10-06T10:39:46.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.059+0000 7f3f5a738700 1 -- start start 2023-10-06T10:39:46.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.059+0000 7f3f5a738700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f3c003a60 0x7f3f5416a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:46.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.059+0000 7f3f5a738700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f540b3250 0x7f3f54069830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:46.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.059+0000 7f3f5a738700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f540c8170 0x7f3f540c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:46.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.059+0000 7f3f5a738700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540c8b20 con 0x7f3f540b3250 2023-10-06T10:39:46.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.059+0000 7f3f5a738700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5416a1b0 con 0x7f3f540c8170 2023-10-06T10:39:46.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.059+0000 7f3f5a738700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540c77e0 con 0x7f3f3c003a60 2023-10-06T10:39:46.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.059+0000 7f3f537fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f540b3250 0x7f3f54069830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:46.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.059+0000 7f3f537fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f540b3250 0x7f3f54069830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53796/0 (socket says 172.21.15.92:53796) 2023-10-06T10:39:46.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.059+0000 7f3f537fe700 1 -- 172.21.15.92:0/3360741037 learned_addr learned my addr 172.21.15.92:0/3360741037 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:46.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.059+0000 7f3f58cd5700 1 --2- 172.21.15.92:0/3360741037 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f540c8170 0x7f3f540c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:46.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.060+0000 7f3f53fff700 1 --2- 172.21.15.92:0/3360741037 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f3c003a60 0x7f3f5416a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:46.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.060+0000 7f3f537fe700 1 -- 172.21.15.92:0/3360741037 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f3c003a60 msgr2=0x7f3f5416a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:46.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.060+0000 7f3f537fe700 1 --2- 172.21.15.92:0/3360741037 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f3c003a60 0x7f3f5416a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.060+0000 7f3f537fe700 1 -- 172.21.15.92:0/3360741037 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f540c8170 msgr2=0x7f3f540c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:46.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.060+0000 7f3f537fe700 1 --2- 172.21.15.92:0/3360741037 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f540c8170 0x7f3f540c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.060+0000 7f3f537fe700 1 -- 172.21.15.92:0/3360741037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f540c7ac0 con 0x7f3f540b3250 2023-10-06T10:39:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.060+0000 7f3f537fe700 1 --2- 172.21.15.92:0/3360741037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f540b3250 0x7f3f54069830 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800e510 tx=0x7f3f48009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.060+0000 7f3f3bfff700 1 -- 172.21.15.92:0/3360741037 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4800ee40 con 0x7f3f540b3250 2023-10-06T10:39:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.060+0000 7f3f3bfff700 1 -- 172.21.15.92:0/3360741037 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f48012590 con 0x7f3f540b3250 2023-10-06T10:39:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.060+0000 7f3f3bfff700 1 -- 172.21.15.92:0/3360741037 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48012910 con 0x7f3f540b3250 2023-10-06T10:39:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.060+0000 7f3f5a738700 1 -- 172.21.15.92:0/3360741037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f540c7da0 con 0x7f3f540b3250 2023-10-06T10:39:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.060+0000 7f3f5a738700 1 -- 172.21.15.92:0/3360741037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f54169e40 con 0x7f3f540b3250 2023-10-06T10:39:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.061+0000 7f3f3bfff700 1 -- 172.21.15.92:0/3360741037 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3f48012d00 con 0x7f3f540b3250 2023-10-06T10:39:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.062+0000 7f3f3bfff700 1 --2- 172.21.15.92:0/3360741037 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3f40066f30 0x7f3f400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.062+0000 7f3f3bfff700 1 -- 172.21.15.92:0/3360741037 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3f48084640 con 0x7f3f540b3250 2023-10-06T10:39:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.062+0000 7f3f53fff700 1 --2- 172.21.15.92:0/3360741037 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3f40066f30 0x7f3f400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.062+0000 7f3f53fff700 1 --2- 172.21.15.92:0/3360741037 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3f40066f30 0x7f3f400693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009160 tx=0x7f3f44019000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.064+0000 7f3f63a30b80 1 Processor -- start 2023-10-06T10:39:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.064+0000 7f3f63a30b80 1 -- start start 2023-10-06T10:39:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.064+0000 7f3f63a30b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d67fb5cae0 0x55d67fb5ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.064+0000 7f3f63a30b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d67fb189f0 0x55d67fb155d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.064+0000 7f3f63a30b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb16c20 0x55d67fb15b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.064+0000 7f3f63a30b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d67fa95cd0 con 0x55d67fb16c20 2023-10-06T10:39:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.064+0000 7f3f63a30b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d67fa6cac0 con 0x55d67fb189f0 2023-10-06T10:39:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.064+0000 7f3f63a30b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d67fa6a2c0 con 0x55d67fb5cae0 2023-10-06T10:39:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.065+0000 7f3f58cd5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb16c20 0x55d67fb15b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.065+0000 7f3f58cd5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb16c20 0x55d67fb15b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53806/0 (socket says 172.21.15.92:53806) 2023-10-06T10:39:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.065+0000 7f3f58cd5700 1 -- 172.21.15.92:0/2088327099 learned_addr learned my addr 172.21.15.92:0/2088327099 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.065+0000 7f3f537fe700 1 --2- 172.21.15.92:0/2088327099 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d67fb189f0 0x55d67fb155d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.065+0000 7f3f53fff700 1 --2- 172.21.15.92:0/2088327099 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d67fb5cae0 0x55d67fb5ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.065+0000 7f3f58cd5700 1 -- 172.21.15.92:0/2088327099 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d67fb5cae0 msgr2=0x55d67fb5ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.065+0000 7f3f58cd5700 1 --2- 172.21.15.92:0/2088327099 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d67fb5cae0 0x55d67fb5ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.065+0000 7f3f58cd5700 1 -- 172.21.15.92:0/2088327099 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d67fb189f0 msgr2=0x55d67fb155d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.065+0000 7f3f58cd5700 1 --2- 172.21.15.92:0/2088327099 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d67fb189f0 0x55d67fb155d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.065+0000 7f3f58cd5700 1 -- 172.21.15.92:0/2088327099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d67f9002c0 con 0x55d67fb16c20 2023-10-06T10:39:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.065+0000 7f3f58cd5700 1 --2- 172.21.15.92:0/2088327099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb16c20 0x55d67fb15b10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c002250 tx=0x7f3f4c006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9261220204afc11 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.065+0000 7f3f397fa700 1 -- 172.21.15.92:0/2088327099 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c00ad80 con 0x55d67fb16c20 2023-10-06T10:39:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.065+0000 7f3f397fa700 1 -- 172.21.15.92:0/2088327099 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c007130 con 0x55d67fb16c20 2023-10-06T10:39:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.065+0000 7f3f397fa700 1 -- 172.21.15.92:0/2088327099 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c0074b0 con 0x55d67fb16c20 2023-10-06T10:39:46.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.066+0000 7f3f63a30b80 1 -- 172.21.15.92:0/2088327099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb16c20 msgr2=0x55d67fb15b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:46.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.066+0000 7f3f63a30b80 1 --2- 172.21.15.92:0/2088327099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb16c20 0x55d67fb15b10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c002250 tx=0x7f3f4c006040 comp rx=0 tx=0).stop 2023-10-06T10:39:46.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.066+0000 7f3f63a30b80 1 -- 172.21.15.92:0/2088327099 shutdown_connections 2023-10-06T10:39:46.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.066+0000 7f3f63a30b80 1 --2- 172.21.15.92:0/2088327099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb16c20 0x55d67fb15b10 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.066+0000 7f3f63a30b80 1 --2- 172.21.15.92:0/2088327099 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d67fb5cae0 0x55d67fb5ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.066+0000 7f3f63a30b80 1 --2- 172.21.15.92:0/2088327099 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d67fb189f0 0x55d67fb155d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.066+0000 7f3f63a30b80 1 -- 172.21.15.92:0/2088327099 >> 172.21.15.92:0/2088327099 conn(0x55d67fb170c0 msgr2=0x55d67fb17490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:46.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.066+0000 7f3f63a30b80 1 -- 172.21.15.92:0/2088327099 shutdown_connections 2023-10-06T10:39:46.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.066+0000 7f3f63a30b80 1 -- 172.21.15.92:0/2088327099 wait complete. 2023-10-06T10:39:46.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.066+0000 7f3f63a30b80 1 -- start start 2023-10-06T10:39:46.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.067+0000 7f3f63a30b80 1 Processor -- start 2023-10-06T10:39:46.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.067+0000 7f3f63a30b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb16c20 0x55d67fb41880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:46.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.067+0000 7f3f63a30b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d67fb189f0 0x55d67fb3dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:46.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.067+0000 7f3f63a30b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d67fb5cae0 0x55d67fb3e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:46.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.067+0000 7f3f63a30b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d67fa660b0 con 0x55d67fb16c20 2023-10-06T10:39:46.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.067+0000 7f3f63a30b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d67fa118e0 con 0x55d67fb189f0 2023-10-06T10:39:46.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.067+0000 7f3f63a30b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d67fa2cef0 con 0x55d67fb5cae0 2023-10-06T10:39:46.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.067+0000 7f3f63a30b80 10 client.0 _ll_register_callbacks cb 0x55d67f902f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:39:46.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.067+0000 7f3f53fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb16c20 0x55d67fb41880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:46.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.067+0000 7f3f53fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb16c20 0x55d67fb41880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53818/0 (socket says 172.21.15.92:53818) 2023-10-06T10:39:46.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.067+0000 7f3f53fff700 1 -- 172.21.15.92:0/4101778693 learned_addr learned my addr 172.21.15.92:0/4101778693 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:46.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.067+0000 7f3f537fe700 1 --2- 172.21.15.92:0/4101778693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d67fb189f0 0x55d67fb3dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:46.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.067+0000 7f3f58cd5700 1 --2- 172.21.15.92:0/4101778693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d67fb5cae0 0x55d67fb3e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:46.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.068+0000 7f3f53fff700 1 -- 172.21.15.92:0/4101778693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d67fb5cae0 msgr2=0x55d67fb3e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:46.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.068+0000 7f3f53fff700 1 --2- 172.21.15.92:0/4101778693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d67fb5cae0 0x55d67fb3e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.068+0000 7f3f53fff700 1 -- 172.21.15.92:0/4101778693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d67fb189f0 msgr2=0x55d67fb3dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:46.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.068+0000 7f3f53fff700 1 --2- 172.21.15.92:0/4101778693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d67fb189f0 0x55d67fb3dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.068+0000 7f3f53fff700 1 -- 172.21.15.92:0/4101778693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f5415a670 con 0x55d67fb16c20 2023-10-06T10:39:46.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.068+0000 7f3f53fff700 1 --2- 172.21.15.92:0/4101778693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb16c20 0x55d67fb41880 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400eb80 tx=0x7f3f44025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a1af80cb8f875 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:46.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.068+0000 7f3f327fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:39:46.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.068+0000 7f3f327fc700 1 -- 172.21.15.92:0/4101778693 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44006490 con 0x55d67fb16c20 2023-10-06T10:39:46.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.068+0000 7f3f327fc700 1 -- 172.21.15.92:0/4101778693 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3f440065f0 con 0x55d67fb16c20 2023-10-06T10:39:46.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.068+0000 7f3f63a30b80 1 -- 172.21.15.92:0/4101778693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d67fb0a6e0 con 0x55d67fb16c20 2023-10-06T10:39:46.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.068+0000 7f3f327fc700 1 -- 172.21.15.92:0/4101778693 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f44013470 con 0x55d67fb16c20 2023-10-06T10:39:46.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.068+0000 7f3f63a30b80 10 client.5442 fetch_fsmap learned FSMap version 23 2023-10-06T10:39:46.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.068+0000 7f3f63a30b80 1 -- 172.21.15.92:0/4101778693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55d67f8a3030 con 0x55d67fb16c20 2023-10-06T10:39:46.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.068+0000 7f3f327fc700 1 -- 172.21.15.92:0/4101778693 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d67fb0a6e0 con 0x55d67fb16c20 2023-10-06T10:39:46.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.068+0000 7f3f327fc700 1 -- 172.21.15.92:0/4101778693 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 23) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f3f440135d0 con 0x55d67fb16c20 2023-10-06T10:39:46.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.068+0000 7f3f63a30b80 10 client.5442 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T10:39:46.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.068+0000 7f3f63a30b80 10 client.5442 resolve_mds: resolved 2:2 to role '2:2' aka daemon mds.a 2023-10-06T10:39:46.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.068+0000 7f3f63a30b80 20 client.5442 populate_metadata read hostname 'smithi092' 2023-10-06T10:39:46.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.068+0000 7f3f63a30b80 1 --2- 172.21.15.92:0/4101778693 >> [v2:172.21.15.92:6834/2409440810,v1:172.21.15.92:6836/2409440810] conn(0x55d67fb584c0 0x55d67fb42a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:46.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.068+0000 7f3f63a30b80 4 client.5442 mds_command: new command op to 5118 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:39:46.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.068+0000 7f3f63a30b80 1 -- 172.21.15.92:0/4101778693 --> [v2:172.21.15.92:6834/2409440810,v1:172.21.15.92:6836/2409440810] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d67fae7610 con 0x55d67fb584c0 2023-10-06T10:39:46.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.068+0000 7f3f537fe700 1 --2- 172.21.15.92:0/4101778693 >> [v2:172.21.15.92:6834/2409440810,v1:172.21.15.92:6836/2409440810] conn(0x55d67fb584c0 0x55d67fb42a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:46.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.069+0000 7f3f327fc700 10 client.5442 ms_handle_connect on v2:172.21.15.92:6834/2409440810 2023-10-06T10:39:46.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.069+0000 7f3f537fe700 1 --2- 172.21.15.92:0/4101778693 >> [v2:172.21.15.92:6834/2409440810,v1:172.21.15.92:6836/2409440810] conn(0x55d67fb584c0 0x55d67fb42a90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d7229754e397919 server_cookie=9d0b752142426cd1 in_seq=0 out_seq=0 2023-10-06T10:39:46.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.069+0000 7f3f327fc700 1 -- 172.21.15.92:0/4101778693 <== mds.2 v2:172.21.15.92:6834/2409440810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55d67fae7610 con 0x55d67fb584c0 2023-10-06T10:39:46.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.069+0000 7f3f327fc700 10 client.5442 handle_command_reply: tid=0 2023-10-06T10:39:46.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.069+0000 7f3f63a30b80 1 client.5442 shutdown 2023-10-06T10:39:46.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.069+0000 7f3f63a30b80 1 -- 172.21.15.92:0/4101778693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb16c20 msgr2=0x55d67fb41880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:46.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.069+0000 7f3f63a30b80 1 --2- 172.21.15.92:0/4101778693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb16c20 0x55d67fb41880 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400eb80 tx=0x7f3f44025000 comp rx=0 tx=0).stop 2023-10-06T10:39:46.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.069+0000 7f3f63a30b80 1 -- 172.21.15.92:0/4101778693 shutdown_connections 2023-10-06T10:39:46.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.069+0000 7f3f63a30b80 1 --2- 172.21.15.92:0/4101778693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d67fb5cae0 0x55d67fb3e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.069+0000 7f3f63a30b80 1 --2- 172.21.15.92:0/4101778693 >> [v2:172.21.15.92:6834/2409440810,v1:172.21.15.92:6836/2409440810] conn(0x55d67fb584c0 0x55d67fb42a90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.069+0000 7f3f63a30b80 1 --2- 172.21.15.92:0/4101778693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb16c20 0x55d67fb41880 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.069+0000 7f3f327fc700 0 client.5442 ms_handle_reset on v2:172.21.15.92:6834/2409440810 2023-10-06T10:39:46.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.069+0000 7f3f63a30b80 1 --2- 172.21.15.92:0/4101778693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d67fb189f0 0x55d67fb3dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.069+0000 7f3f63a30b80 1 -- 172.21.15.92:0/4101778693 >> 172.21.15.92:0/4101778693 conn(0x55d67fb170c0 msgr2=0x55d67fb5c030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:46.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.069+0000 7f3f63a30b80 1 -- 172.21.15.92:0/4101778693 shutdown_connections 2023-10-06T10:39:46.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.069+0000 7f3f63a30b80 1 -- 172.21.15.92:0/4101778693 wait complete. 2023-10-06T10:39:46.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.070+0000 7f3f63a30b80 20 client.5442 trim_cache size 0 max 16384 2023-10-06T10:39:46.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.083+0000 7f3f63a30b80 1 Processor -- start 2023-10-06T10:39:46.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.083+0000 7f3f63a30b80 1 -- start start 2023-10-06T10:39:46.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.084+0000 7f3f63a30b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d67fb16c20 0x55d67fb129e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:46.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.084+0000 7f3f63a30b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb189f0 0x55d67fb42940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:46.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.084+0000 7f3f63a30b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d67fb5cae0 0x55d67fb3de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:46.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.084+0000 7f3f63a30b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d67f8e8e80 con 0x55d67fb189f0 2023-10-06T10:39:46.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.084+0000 7f3f63a30b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d67fb3f310 con 0x55d67fb16c20 2023-10-06T10:39:46.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.084+0000 7f3f63a30b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d67fb571d0 con 0x55d67fb5cae0 2023-10-06T10:39:46.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.084+0000 7f3f537fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb189f0 0x55d67fb42940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:46.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.084+0000 7f3f537fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb189f0 0x55d67fb42940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53826/0 (socket says 172.21.15.92:53826) 2023-10-06T10:39:46.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.084+0000 7f3f537fe700 1 -- 172.21.15.92:0/2763740263 learned_addr learned my addr 172.21.15.92:0/2763740263 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.084+0000 7f3f537fe700 1 -- 172.21.15.92:0/2763740263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d67fb5cae0 msgr2=0x55d67fb3de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.084+0000 7f3f537fe700 1 --2- 172.21.15.92:0/2763740263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d67fb5cae0 0x55d67fb3de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.084+0000 7f3f537fe700 1 -- 172.21.15.92:0/2763740263 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d67fb16c20 msgr2=0x55d67fb129e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.084+0000 7f3f537fe700 1 --2- 172.21.15.92:0/2763740263 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d67fb16c20 0x55d67fb129e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.084+0000 7f3f537fe700 1 -- 172.21.15.92:0/2763740263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d67fa347d0 con 0x55d67fb189f0 2023-10-06T10:39:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.084+0000 7f3f537fe700 1 --2- 172.21.15.92:0/2763740263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb189f0 0x55d67fb42940 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3f48007ae0 tx=0x7f3f48069040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4ffcb57d7141d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.085+0000 7f3f39ffb700 1 -- 172.21.15.92:0/2763740263 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48058520 con 0x55d67fb189f0 2023-10-06T10:39:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.085+0000 7f3f39ffb700 1 -- 172.21.15.92:0/2763740263 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4805c270 con 0x55d67fb189f0 2023-10-06T10:39:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.085+0000 7f3f39ffb700 1 -- 172.21.15.92:0/2763740263 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f480829d0 con 0x55d67fb189f0 2023-10-06T10:39:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.085+0000 7f3f63a30b80 1 -- 172.21.15.92:0/2763740263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb189f0 msgr2=0x55d67fb42940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:46.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.085+0000 7f3f63a30b80 1 --2- 172.21.15.92:0/2763740263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb189f0 0x55d67fb42940 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3f48007ae0 tx=0x7f3f48069040 comp rx=0 tx=0).stop 2023-10-06T10:39:46.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.085+0000 7f3f63a30b80 1 -- 172.21.15.92:0/2763740263 shutdown_connections 2023-10-06T10:39:46.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.085+0000 7f3f63a30b80 1 --2- 172.21.15.92:0/2763740263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d67fb5cae0 0x55d67fb3de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.085+0000 7f3f63a30b80 1 --2- 172.21.15.92:0/2763740263 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d67fb16c20 0x55d67fb129e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.085+0000 7f3f63a30b80 1 --2- 172.21.15.92:0/2763740263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb189f0 0x55d67fb42940 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.085+0000 7f3f63a30b80 1 -- 172.21.15.92:0/2763740263 >> 172.21.15.92:0/2763740263 conn(0x55d67fb170c0 msgr2=0x55d67fb58fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:46.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.085+0000 7f3f63a30b80 1 -- 172.21.15.92:0/2763740263 shutdown_connections 2023-10-06T10:39:46.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.085+0000 7f3f63a30b80 1 -- 172.21.15.92:0/2763740263 wait complete. 2023-10-06T10:39:46.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.086+0000 7f3f63a30b80 1 -- start start 2023-10-06T10:39:46.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.086+0000 7f3f63a30b80 1 Processor -- start 2023-10-06T10:39:46.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f63a30b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d67fb16c20 0x55d67fb3bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:46.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f63a30b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb189f0 0x55d67fbb81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:46.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f63a30b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d67fb5cae0 0x55d67fb12d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:46.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f63a30b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d67fb3d650 con 0x55d67fb189f0 2023-10-06T10:39:46.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f63a30b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d67fb3c070 con 0x55d67fb16c20 2023-10-06T10:39:46.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f63a30b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d67fb3b120 con 0x55d67fb5cae0 2023-10-06T10:39:46.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f63a30b80 10 client.0 _ll_register_callbacks cb 0x55d67f902f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:39:46.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f537fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb189f0 0x55d67fbb81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:46.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f537fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb189f0 0x55d67fbb81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53832/0 (socket says 172.21.15.92:53832) 2023-10-06T10:39:46.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f537fe700 1 -- 172.21.15.92:0/1236611981 learned_addr learned my addr 172.21.15.92:0/1236611981 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:46.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f537fe700 1 -- 172.21.15.92:0/1236611981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d67fb5cae0 msgr2=0x55d67fb12d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:46.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f58cd5700 1 --2- 172.21.15.92:0/1236611981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d67fb5cae0 0x55d67fb12d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:46.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f53fff700 1 --2- 172.21.15.92:0/1236611981 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d67fb16c20 0x55d67fb3bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:46.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f537fe700 1 --2- 172.21.15.92:0/1236611981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d67fb5cae0 0x55d67fb12d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f537fe700 1 -- 172.21.15.92:0/1236611981 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d67fb16c20 msgr2=0x55d67fb3bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:46.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f537fe700 1 --2- 172.21.15.92:0/1236611981 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d67fb16c20 0x55d67fb3bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f537fe700 1 -- 172.21.15.92:0/1236611981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f54169e40 con 0x55d67fb189f0 2023-10-06T10:39:46.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f537fe700 1 --2- 172.21.15.92:0/1236611981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb189f0 0x55d67fbb81c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3f48084010 tx=0x7f3f4806a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3032fc6a85e82b06 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:46.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f327fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:39:46.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f327fc700 1 -- 172.21.15.92:0/1236611981 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f480829a0 con 0x55d67fb189f0 2023-10-06T10:39:46.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f327fc700 1 -- 172.21.15.92:0/1236611981 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3f48053670 con 0x55d67fb189f0 2023-10-06T10:39:46.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f63a30b80 1 -- 172.21.15.92:0/1236611981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f4c007130 con 0x55d67fb189f0 2023-10-06T10:39:46.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f327fc700 1 -- 172.21.15.92:0/1236611981 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f48055740 con 0x55d67fb189f0 2023-10-06T10:39:46.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f63a30b80 10 client.5448 fetch_fsmap learned FSMap version 23 2023-10-06T10:39:46.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f63a30b80 1 -- 172.21.15.92:0/1236611981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55d67fa1b4c0 con 0x55d67fb189f0 2023-10-06T10:39:46.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.087+0000 7f3f327fc700 1 -- 172.21.15.92:0/1236611981 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c007130 con 0x55d67fb189f0 2023-10-06T10:39:46.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.088+0000 7f3f327fc700 1 -- 172.21.15.92:0/1236611981 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 23) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f3f480547c0 con 0x55d67fb189f0 2023-10-06T10:39:46.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.088+0000 7f3f63a30b80 10 client.5448 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T10:39:46.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.088+0000 7f3f63a30b80 10 client.5448 resolve_mds: resolved 2:2 to role '2:2' aka daemon mds.a 2023-10-06T10:39:46.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.088+0000 7f3f63a30b80 20 client.5448 populate_metadata read hostname 'smithi092' 2023-10-06T10:39:46.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.088+0000 7f3f63a30b80 1 --2- 172.21.15.92:0/1236611981 >> [v2:172.21.15.92:6834/2409440810,v1:172.21.15.92:6836/2409440810] conn(0x55d67fb3cb10 0x55d67fbb8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:46.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.088+0000 7f3f63a30b80 4 client.5448 mds_command: new command op to 5118 tid=0[{"prefix": "flush journal"}] 2023-10-06T10:39:46.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.088+0000 7f3f63a30b80 1 -- 172.21.15.92:0/1236611981 --> [v2:172.21.15.92:6834/2409440810,v1:172.21.15.92:6836/2409440810] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d67fa08e50 con 0x55d67fb3cb10 2023-10-06T10:39:46.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.088+0000 7f3f53fff700 1 --2- 172.21.15.92:0/1236611981 >> [v2:172.21.15.92:6834/2409440810,v1:172.21.15.92:6836/2409440810] conn(0x55d67fb3cb10 0x55d67fbb8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:46.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.088+0000 7f3f327fc700 10 client.5448 ms_handle_connect on v2:172.21.15.92:6834/2409440810 2023-10-06T10:39:46.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.088+0000 7f3f53fff700 1 --2- 172.21.15.92:0/1236611981 >> [v2:172.21.15.92:6834/2409440810,v1:172.21.15.92:6836/2409440810] conn(0x55d67fb3cb10 0x55d67fbb8f00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7198623b50529477 server_cookie=25881aab7efe21df in_seq=0 out_seq=0 2023-10-06T10:39:46.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.099+0000 7f3f327fc700 1 -- 172.21.15.92:0/1236611981 <== mds.2 v2:172.21.15.92:6834/2409440810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d67fa08e50 con 0x55d67fb3cb10 2023-10-06T10:39:46.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.099+0000 7f3f327fc700 10 client.5448 handle_command_reply: tid=0 2023-10-06T10:39:46.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.099+0000 7f3f63a30b80 1 client.5448 shutdown 2023-10-06T10:39:46.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.099+0000 7f3f63a30b80 1 -- 172.21.15.92:0/1236611981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb189f0 msgr2=0x55d67fbb81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:46.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.099+0000 7f3f63a30b80 1 --2- 172.21.15.92:0/1236611981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb189f0 0x55d67fbb81c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3f48084010 tx=0x7f3f4806a000 comp rx=0 tx=0).stop 2023-10-06T10:39:46.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.099+0000 7f3f63a30b80 1 -- 172.21.15.92:0/1236611981 shutdown_connections 2023-10-06T10:39:46.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.099+0000 7f3f63a30b80 1 --2- 172.21.15.92:0/1236611981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d67fb5cae0 0x55d67fb12d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.099+0000 7f3f63a30b80 1 --2- 172.21.15.92:0/1236611981 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d67fb16c20 0x55d67fb3bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.099+0000 7f3f63a30b80 1 --2- 172.21.15.92:0/1236611981 >> [v2:172.21.15.92:6834/2409440810,v1:172.21.15.92:6836/2409440810] conn(0x55d67fb3cb10 0x55d67fbb8f00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.099+0000 7f3f63a30b80 1 --2- 172.21.15.92:0/1236611981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d67fb189f0 0x55d67fbb81c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.099+0000 7f3f327fc700 0 client.5448 ms_handle_reset on v2:172.21.15.92:6834/2409440810 2023-10-06T10:39:46.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.099+0000 7f3f63a30b80 1 -- 172.21.15.92:0/1236611981 >> 172.21.15.92:0/1236611981 conn(0x55d67fb170c0 msgr2=0x55d67fb5bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:46.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.099+0000 7f3f63a30b80 1 -- 172.21.15.92:0/1236611981 shutdown_connections 2023-10-06T10:39:46.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.099+0000 7f3f63a30b80 1 -- 172.21.15.92:0/1236611981 wait complete. 2023-10-06T10:39:46.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.100+0000 7f3f63a30b80 20 client.5448 trim_cache size 0 max 16384 2023-10-06T10:39:46.103 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:39:46.103 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2023-10-06T10:39:46.103 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:39:46.104 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:39:46.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.100+0000 7f3f5a738700 1 -- 172.21.15.92:0/3360741037 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3f40066f30 msgr2=0x7f3f400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:46.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.100+0000 7f3f5a738700 1 --2- 172.21.15.92:0/3360741037 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3f40066f30 0x7f3f400693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009160 tx=0x7f3f44019000 comp rx=0 tx=0).stop 2023-10-06T10:39:46.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.100+0000 7f3f5a738700 1 -- 172.21.15.92:0/3360741037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f540b3250 msgr2=0x7f3f54069830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:46.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.100+0000 7f3f5a738700 1 --2- 172.21.15.92:0/3360741037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f540b3250 0x7f3f54069830 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800e510 tx=0x7f3f48009040 comp rx=0 tx=0).stop 2023-10-06T10:39:46.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.100+0000 7f3f5a738700 1 -- 172.21.15.92:0/3360741037 shutdown_connections 2023-10-06T10:39:46.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.100+0000 7f3f5a738700 1 --2- 172.21.15.92:0/3360741037 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3f40066f30 0x7f3f400693e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.100+0000 7f3f5a738700 1 --2- 172.21.15.92:0/3360741037 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f540c8170 0x7f3f540c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.100+0000 7f3f5a738700 1 --2- 172.21.15.92:0/3360741037 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f3c003a60 0x7f3f5416a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.100+0000 7f3f5a738700 1 --2- 172.21.15.92:0/3360741037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f540b3250 0x7f3f54069830 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:46.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.100+0000 7f3f5a738700 1 -- 172.21.15.92:0/3360741037 >> 172.21.15.92:0/3360741037 conn(0x7f3f54099700 msgr2=0x7f3f540cec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:46.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.100+0000 7f3f5a738700 1 -- 172.21.15.92:0/3360741037 shutdown_connections 2023-10-06T10:39:46.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:46.100+0000 7f3f5a738700 1 -- 172.21.15.92:0/3360741037 wait complete. 2023-10-06T10:39:46.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:39:46.114 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2023-10-06T10:39:46.135 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/ 2023-10-06T10:39:46.136 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4 2023-10-06T10:39:46.136 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4/d5 2023-10-06T10:39:46.136 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4/d5/d6 2023-10-06T10:39:46.136 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4/d5/d6/d7 2023-10-06T10:39:46.136 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2023-10-06T10:39:46.136 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:39:46.136 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:39:46.137 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:39:46.137 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:39:46.137 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2023-10-06T10:39:46.137 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:39:46.238 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627778 2023-10-06T10:39:46.428 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:39:46.428 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:39:46.428 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:39:46.428 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:39:46.428 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2023-10-06T10:39:46.428 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:39:46.532 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627779 2023-10-06T10:39:46.722 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:39:46.722 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:39:46.722 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:39:46.722 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:39:46.722 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2023-10-06T10:39:46.722 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:39:46.825 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627780 2023-10-06T10:39:47.015 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:39:47.016 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:39:47.016 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:39:47.016 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:39:47.016 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2023-10-06T10:39:47.016 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:39:47.120 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627781 2023-10-06T10:39:47.310 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:39:47.310 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:39:47.310 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:39:47.310 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:39:47.310 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2023-10-06T10:39:47.310 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:39:47.414 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627782 2023-10-06T10:39:47.603 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:39:47.604 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:39:47.604 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:39:47.604 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:39:47.604 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2023-10-06T10:39:47.604 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:39:47.707 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627783 2023-10-06T10:39:47.897 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 tag path /d1/d2/d3 tag123 2023-10-06T10:39:48.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.055+0000 7f872aec0700 1 Processor -- start 2023-10-06T10:39:48.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.055+0000 7f872aec0700 1 -- start start 2023-10-06T10:39:48.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.056+0000 7f872aec0700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f87240b2360 0x7f87240b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.056+0000 7f872aec0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87240aede0 0x7f87240af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.056+0000 7f872aec0700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87240b2ce0 con 0x7f872406ece0 2023-10-06T10:39:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.056+0000 7f872aec0700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f872406fb40 con 0x7f87240b2360 2023-10-06T10:39:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.056+0000 7f872aec0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87240b2800 con 0x7f87240aede0 2023-10-06T10:39:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.056+0000 7f872945d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87240aede0 0x7f87240af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.056+0000 7f8723fff700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f87240b2360 0x7f87240b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.056+0000 7f8728c5c700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f872406ece0 0x7f872406f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:54944/0 (socket says 172.21.15.92:54944) 2023-10-06T10:39:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.056+0000 7f872945d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87240aede0 0x7f87240af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:40680/0 (socket says 172.21.15.92:40680) 2023-10-06T10:39:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.056+0000 7f872945d700 1 -- 172.21.15.92:0/3483327479 learned_addr learned my addr 172.21.15.92:0/3483327479 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:48.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.057+0000 7f872945d700 1 -- 172.21.15.92:0/3483327479 >> v2:172.21.15.124:3300/0 conn(0x7f87240b2360 msgr2=0x7f87240b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:48.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.057+0000 7f872945d700 1 --2- 172.21.15.92:0/3483327479 >> v2:172.21.15.124:3300/0 conn(0x7f87240b2360 0x7f87240b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.057+0000 7f872945d700 1 -- 172.21.15.92:0/3483327479 >> v1:172.21.15.124:6789/0 conn(0x7f872406ece0 legacy=0x7f872406f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:48.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.057+0000 7f872945d700 1 -- 172.21.15.92:0/3483327479 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87240b0410 con 0x7f87240aede0 2023-10-06T10:39:48.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.057+0000 7f8723fff700 1 --2- 172.21.15.92:0/3483327479 >> v2:172.21.15.124:3300/0 conn(0x7f87240b2360 0x7f87240b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:39:48.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.057+0000 7f87237fe700 1 -- 172.21.15.92:0/3483327479 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3600433572 0 0) 0x7f87240b2ce0 con 0x7f872406ece0 2023-10-06T10:39:48.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.057+0000 7f872945d700 1 --2- 172.21.15.92:0/3483327479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87240aede0 0x7f87240af240 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f871c009f20 tx=0x7f871c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cf7e5683a0e2fab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:48.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.057+0000 7f87237fe700 1 -- 172.21.15.92:0/3483327479 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c00e070 con 0x7f87240aede0 2023-10-06T10:39:48.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.057+0000 7f87237fe700 1 -- 172.21.15.92:0/3483327479 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871c00b9d0 con 0x7f87240aede0 2023-10-06T10:39:48.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.057+0000 7f87237fe700 1 -- 172.21.15.92:0/3483327479 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c0092e0 con 0x7f87240aede0 2023-10-06T10:39:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.057+0000 7f872aec0700 1 -- 172.21.15.92:0/3483327479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87240aede0 msgr2=0x7f87240af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.057+0000 7f872aec0700 1 --2- 172.21.15.92:0/3483327479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87240aede0 0x7f87240af240 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f871c009f20 tx=0x7f871c009840 comp rx=0 tx=0).stop 2023-10-06T10:39:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.058+0000 7f872aec0700 1 -- 172.21.15.92:0/3483327479 shutdown_connections 2023-10-06T10:39:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.058+0000 7f872aec0700 1 --2- 172.21.15.92:0/3483327479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87240aede0 0x7f87240af240 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.058+0000 7f872aec0700 1 --2- 172.21.15.92:0/3483327479 >> v2:172.21.15.124:3300/0 conn(0x7f87240b2360 0x7f87240b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.058+0000 7f872aec0700 1 -- 172.21.15.92:0/3483327479 >> 172.21.15.92:0/3483327479 conn(0x7f87240682f0 msgr2=0x7f87240686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.058+0000 7f872aec0700 1 -- 172.21.15.92:0/3483327479 shutdown_connections 2023-10-06T10:39:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.058+0000 7f872aec0700 1 -- 172.21.15.92:0/3483327479 wait complete. 2023-10-06T10:39:48.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.059+0000 7f872aec0700 1 Processor -- start 2023-10-06T10:39:48.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.059+0000 7f872aec0700 1 -- start start 2023-10-06T10:39:48.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.060+0000 7f872aec0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f872406ece0 0x7f87240c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.060+0000 7f872aec0700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87240aede0 0x7f8724149790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.060+0000 7f872aec0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87240b2360 0x7f87240cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.060+0000 7f872aec0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872415bbf0 con 0x7f87240b2360 2023-10-06T10:39:48.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.060+0000 7f872aec0700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87240c89b0 con 0x7f87240aede0 2023-10-06T10:39:48.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.060+0000 7f872aec0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872415b8e0 con 0x7f872406ece0 2023-10-06T10:39:48.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.060+0000 7f872945d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87240b2360 0x7f87240cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.060+0000 7f872945d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87240b2360 0x7f87240cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53848/0 (socket says 172.21.15.92:53848) 2023-10-06T10:39:48.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.060+0000 7f872945d700 1 -- 172.21.15.92:0/3798789691 learned_addr learned my addr 172.21.15.92:0/3798789691 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:48.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.060+0000 7f8723fff700 1 --2- 172.21.15.92:0/3798789691 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87240aede0 0x7f8724149790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.060+0000 7f8728c5c700 1 --2- 172.21.15.92:0/3798789691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f872406ece0 0x7f87240c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.060+0000 7f872945d700 1 -- 172.21.15.92:0/3798789691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f872406ece0 msgr2=0x7f87240c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:48.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.060+0000 7f872945d700 1 --2- 172.21.15.92:0/3798789691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f872406ece0 0x7f87240c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.060+0000 7f872945d700 1 -- 172.21.15.92:0/3798789691 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87240aede0 msgr2=0x7f8724149790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:48.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.060+0000 7f872945d700 1 --2- 172.21.15.92:0/3798789691 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87240aede0 0x7f8724149790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.060+0000 7f872945d700 1 -- 172.21.15.92:0/3798789691 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8710009420 con 0x7f87240b2360 2023-10-06T10:39:48.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.060+0000 7f872945d700 1 --2- 172.21.15.92:0/3798789691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87240b2360 0x7f87240cc950 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f871c009430 tx=0x7f871c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:48.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.060+0000 7f8720ff9700 1 -- 172.21.15.92:0/3798789691 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c007af0 con 0x7f87240b2360 2023-10-06T10:39:48.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.060+0000 7f8720ff9700 1 -- 172.21.15.92:0/3798789691 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871c007c50 con 0x7f87240b2360 2023-10-06T10:39:48.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.060+0000 7f8720ff9700 1 -- 172.21.15.92:0/3798789691 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c01c410 con 0x7f87240b2360 2023-10-06T10:39:48.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.060+0000 7f872aec0700 1 -- 172.21.15.92:0/3798789691 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f871c009ae0 con 0x7f87240b2360 2023-10-06T10:39:48.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.060+0000 7f872aec0700 1 -- 172.21.15.92:0/3798789691 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87240c8ef0 con 0x7f87240b2360 2023-10-06T10:39:48.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.062+0000 7f8720ff9700 1 -- 172.21.15.92:0/3798789691 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f871c01c780 con 0x7f87240b2360 2023-10-06T10:39:48.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.062+0000 7f8720ff9700 1 --2- 172.21.15.92:0/3798789691 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8708066f30 0x7f87080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.063+0000 7f8720ff9700 1 -- 172.21.15.92:0/3798789691 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f871c085fe0 con 0x7f87240b2360 2023-10-06T10:39:48.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.063+0000 7f8728c5c700 1 --2- 172.21.15.92:0/3798789691 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8708066f30 0x7f87080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.063+0000 7f8728c5c700 1 --2- 172.21.15.92:0/3798789691 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8708066f30 0x7f87080693e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8724063c10 tx=0x7f8718007ef0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:48.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.064+0000 7f87341b8b80 1 Processor -- start 2023-10-06T10:39:48.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.064+0000 7f87341b8b80 1 -- start start 2023-10-06T10:39:48.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.065+0000 7f87341b8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00bc8b50 0x560b00bc8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.065+0000 7f87341b8b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560b00b84a60 0x560b00b81640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.065+0000 7f87341b8b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560b00b82c90 0x560b00b81b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.065+0000 7f87341b8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b00b01d40 con 0x560b00bc8b50 2023-10-06T10:39:48.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.065+0000 7f87341b8b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b00ad8b30 con 0x560b00b84a60 2023-10-06T10:39:48.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.065+0000 7f87341b8b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b00ad6330 con 0x560b00b82c90 2023-10-06T10:39:48.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.065+0000 7f8728c5c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00bc8b50 0x560b00bc8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.065+0000 7f8728c5c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00bc8b50 0x560b00bc8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53854/0 (socket says 172.21.15.92:53854) 2023-10-06T10:39:48.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.065+0000 7f8728c5c700 1 -- 172.21.15.92:0/825355697 learned_addr learned my addr 172.21.15.92:0/825355697 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:48.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.065+0000 7f8723fff700 1 --2- 172.21.15.92:0/825355697 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560b00b84a60 0x560b00b81640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.065+0000 7f872945d700 1 --2- 172.21.15.92:0/825355697 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560b00b82c90 0x560b00b81b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.065+0000 7f8728c5c700 1 -- 172.21.15.92:0/825355697 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560b00b82c90 msgr2=0x560b00b81b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:48.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.065+0000 7f8728c5c700 1 --2- 172.21.15.92:0/825355697 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560b00b82c90 0x560b00b81b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.065+0000 7f8728c5c700 1 -- 172.21.15.92:0/825355697 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560b00b84a60 msgr2=0x560b00b81640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:48.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.065+0000 7f8728c5c700 1 --2- 172.21.15.92:0/825355697 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560b00b84a60 0x560b00b81640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.065+0000 7f8728c5c700 1 -- 172.21.15.92:0/825355697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b0096c330 con 0x560b00bc8b50 2023-10-06T10:39:48.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.065+0000 7f8723fff700 1 --2- 172.21.15.92:0/825355697 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560b00b84a60 0x560b00b81640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:48.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.065+0000 7f8728c5c700 1 --2- 172.21.15.92:0/825355697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00bc8b50 0x560b00bc8f20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8718001780 tx=0x7f8718014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37a4e51577296e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:48.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.066+0000 7f870dffb700 1 -- 172.21.15.92:0/825355697 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8718013600 con 0x560b00bc8b50 2023-10-06T10:39:48.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.066+0000 7f870dffb700 1 -- 172.21.15.92:0/825355697 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8718013760 con 0x560b00bc8b50 2023-10-06T10:39:48.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.066+0000 7f870dffb700 1 -- 172.21.15.92:0/825355697 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8718013ae0 con 0x560b00bc8b50 2023-10-06T10:39:48.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.066+0000 7f87341b8b80 1 -- 172.21.15.92:0/825355697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00bc8b50 msgr2=0x560b00bc8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:48.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.066+0000 7f87341b8b80 1 --2- 172.21.15.92:0/825355697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00bc8b50 0x560b00bc8f20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8718001780 tx=0x7f8718014000 comp rx=0 tx=0).stop 2023-10-06T10:39:48.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.066+0000 7f87341b8b80 1 -- 172.21.15.92:0/825355697 shutdown_connections 2023-10-06T10:39:48.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.066+0000 7f87341b8b80 1 --2- 172.21.15.92:0/825355697 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560b00b82c90 0x560b00b81b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.066+0000 7f87341b8b80 1 --2- 172.21.15.92:0/825355697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00bc8b50 0x560b00bc8f20 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.066+0000 7f87341b8b80 1 --2- 172.21.15.92:0/825355697 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560b00b84a60 0x560b00b81640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.066+0000 7f87341b8b80 1 -- 172.21.15.92:0/825355697 >> 172.21.15.92:0/825355697 conn(0x560b00b83130 msgr2=0x560b00b83500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:48.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.066+0000 7f87341b8b80 1 -- 172.21.15.92:0/825355697 shutdown_connections 2023-10-06T10:39:48.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.066+0000 7f87341b8b80 1 -- 172.21.15.92:0/825355697 wait complete. 2023-10-06T10:39:48.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.067+0000 7f87341b8b80 1 -- start start 2023-10-06T10:39:48.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.067+0000 7f87341b8b80 1 Processor -- start 2023-10-06T10:39:48.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f87341b8b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560b00b82c90 0x560b00bdb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f87341b8b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560b00b84a60 0x560b00b7c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f87341b8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00bc8b50 0x560b00b7cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f87341b8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b00ad2120 con 0x560b00bc8b50 2023-10-06T10:39:48.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f87341b8b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b00a7d950 con 0x560b00b84a60 2023-10-06T10:39:48.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f87341b8b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b00a98f60 con 0x560b00b82c90 2023-10-06T10:39:48.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f87341b8b80 10 client.0 _ll_register_callbacks cb 0x560b0096eff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:39:48.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f872945d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00bc8b50 0x560b00b7cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f872945d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00bc8b50 0x560b00b7cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53862/0 (socket says 172.21.15.92:53862) 2023-10-06T10:39:48.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f872945d700 1 -- 172.21.15.92:0/2197114982 learned_addr learned my addr 172.21.15.92:0/2197114982 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:48.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f8723fff700 1 --2- 172.21.15.92:0/2197114982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560b00b84a60 0x560b00b7c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f8728c5c700 1 --2- 172.21.15.92:0/2197114982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560b00b82c90 0x560b00bdb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f872945d700 1 -- 172.21.15.92:0/2197114982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560b00b82c90 msgr2=0x560b00bdb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:48.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f872945d700 1 --2- 172.21.15.92:0/2197114982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560b00b82c90 0x560b00bdb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f872945d700 1 -- 172.21.15.92:0/2197114982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560b00b84a60 msgr2=0x560b00b7c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:48.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f872945d700 1 --2- 172.21.15.92:0/2197114982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560b00b84a60 0x560b00b7c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f872945d700 1 -- 172.21.15.92:0/2197114982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87180013a0 con 0x560b00bc8b50 2023-10-06T10:39:48.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f8728c5c700 1 --2- 172.21.15.92:0/2197114982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560b00b82c90 0x560b00bdb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:48.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f872945d700 1 --2- 172.21.15.92:0/2197114982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00bc8b50 0x560b00b7cda0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f871c007730 tx=0x7f871c05c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b5c27877115751 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:48.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f8702ffd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:39:48.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f8702ffd700 1 -- 172.21.15.92:0/2197114982 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c059e60 con 0x560b00bc8b50 2023-10-06T10:39:48.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f8702ffd700 1 -- 172.21.15.92:0/2197114982 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f871c05dbb0 con 0x560b00bc8b50 2023-10-06T10:39:48.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f87341b8b80 1 -- 172.21.15.92:0/2197114982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b00b76750 con 0x560b00bc8b50 2023-10-06T10:39:48.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f8702ffd700 1 -- 172.21.15.92:0/2197114982 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f871c056310 con 0x560b00bc8b50 2023-10-06T10:39:48.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f87341b8b80 10 client.5457 fetch_fsmap learned FSMap version 23 2023-10-06T10:39:48.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.068+0000 7f87341b8b80 1 -- 172.21.15.92:0/2197114982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x560b0090f0a0 con 0x560b00bc8b50 2023-10-06T10:39:48.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.069+0000 7f8702ffd700 1 -- 172.21.15.92:0/2197114982 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b00b76750 con 0x560b00bc8b50 2023-10-06T10:39:48.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.069+0000 7f8702ffd700 1 -- 172.21.15.92:0/2197114982 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 23) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f871c05d720 con 0x560b00bc8b50 2023-10-06T10:39:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.069+0000 7f87341b8b80 10 client.5457 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T10:39:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.069+0000 7f87341b8b80 10 client.5457 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2023-10-06T10:39:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.069+0000 7f87341b8b80 20 client.5457 populate_metadata read hostname 'smithi092' 2023-10-06T10:39:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.069+0000 7f87341b8b80 1 --2- 172.21.15.92:0/2197114982 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x560b00ba8360 0x560b00bc4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.069+0000 7f87341b8b80 4 client.5457 mds_command: new command op to 5133 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:39:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.069+0000 7f87341b8b80 1 -- 172.21.15.92:0/2197114982 --> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b00b53680 con 0x560b00ba8360 2023-10-06T10:39:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.069+0000 7f8728c5c700 1 --2- 172.21.15.92:0/2197114982 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x560b00ba8360 0x560b00bc4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.069+0000 7f8702ffd700 10 client.5457 ms_handle_connect on v2:172.21.15.124:6832/484869337 2023-10-06T10:39:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.069+0000 7f8728c5c700 1 --2- 172.21.15.92:0/2197114982 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x560b00ba8360 0x560b00bc4700 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eac19aeeee42d906 server_cookie=bbc1abf260fb9961 in_seq=0 out_seq=0 2023-10-06T10:39:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.070+0000 7f8702ffd700 1 -- 172.21.15.92:0/2197114982 <== mds.0 v2:172.21.15.124:6832/484869337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x560b00b53680 con 0x560b00ba8360 2023-10-06T10:39:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.070+0000 7f8702ffd700 10 client.5457 handle_command_reply: tid=0 2023-10-06T10:39:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.070+0000 7f87341b8b80 1 client.5457 shutdown 2023-10-06T10:39:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.070+0000 7f87341b8b80 1 -- 172.21.15.92:0/2197114982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00bc8b50 msgr2=0x560b00b7cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.070+0000 7f87341b8b80 1 --2- 172.21.15.92:0/2197114982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00bc8b50 0x560b00b7cda0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f871c007730 tx=0x7f871c05c040 comp rx=0 tx=0).stop 2023-10-06T10:39:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.070+0000 7f87341b8b80 1 -- 172.21.15.92:0/2197114982 shutdown_connections 2023-10-06T10:39:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.070+0000 7f87341b8b80 1 --2- 172.21.15.92:0/2197114982 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x560b00ba8360 0x560b00bc4700 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.070+0000 7f87341b8b80 1 --2- 172.21.15.92:0/2197114982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00bc8b50 0x560b00b7cda0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.070+0000 7f8702ffd700 0 client.5457 ms_handle_reset on v2:172.21.15.124:6832/484869337 2023-10-06T10:39:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.070+0000 7f87341b8b80 1 --2- 172.21.15.92:0/2197114982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560b00b82c90 0x560b00bdb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.070+0000 7f87341b8b80 1 --2- 172.21.15.92:0/2197114982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560b00b84a60 0x560b00b7c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.070+0000 7f87341b8b80 1 -- 172.21.15.92:0/2197114982 >> 172.21.15.92:0/2197114982 conn(0x560b00b83130 msgr2=0x560b00bc7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.070+0000 7f87341b8b80 1 -- 172.21.15.92:0/2197114982 shutdown_connections 2023-10-06T10:39:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.070+0000 7f87341b8b80 1 -- 172.21.15.92:0/2197114982 wait complete. 2023-10-06T10:39:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.070+0000 7f87341b8b80 20 client.5457 trim_cache size 0 max 16384 2023-10-06T10:39:48.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.087+0000 7f87341b8b80 1 Processor -- start 2023-10-06T10:39:48.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.087+0000 7f87341b8b80 1 -- start start 2023-10-06T10:39:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.088+0000 7f87341b8b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560b00b82c90 0x560b00bc6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.088+0000 7f87341b8b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560b00b84a60 0x560b00bc7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.088+0000 7f87341b8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00bc8b50 0x560b00b7c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.088+0000 7f87341b8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b00bacf00 con 0x560b00bc8b50 2023-10-06T10:39:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.088+0000 7f87341b8b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b00954ef0 con 0x560b00b84a60 2023-10-06T10:39:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.088+0000 7f87341b8b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b00b7c180 con 0x560b00b82c90 2023-10-06T10:39:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.088+0000 7f872945d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00bc8b50 0x560b00b7c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.088+0000 7f872945d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00bc8b50 0x560b00b7c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53878/0 (socket says 172.21.15.92:53878) 2023-10-06T10:39:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.088+0000 7f872945d700 1 -- 172.21.15.92:0/3354339582 learned_addr learned my addr 172.21.15.92:0/3354339582 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.088+0000 7f8728c5c700 1 --2- 172.21.15.92:0/3354339582 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560b00b82c90 0x560b00bc6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.088+0000 7f872945d700 1 -- 172.21.15.92:0/3354339582 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560b00b82c90 msgr2=0x560b00bc6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.088+0000 7f8723fff700 1 --2- 172.21.15.92:0/3354339582 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560b00b84a60 0x560b00bc7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.088+0000 7f872945d700 1 --2- 172.21.15.92:0/3354339582 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560b00b82c90 0x560b00bc6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.088+0000 7f872945d700 1 -- 172.21.15.92:0/3354339582 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560b00b84a60 msgr2=0x560b00bc7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.088+0000 7f872945d700 1 --2- 172.21.15.92:0/3354339582 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560b00b84a60 0x560b00bc7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.088+0000 7f872945d700 1 -- 172.21.15.92:0/3354339582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b00bae6e0 con 0x560b00bc8b50 2023-10-06T10:39:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.089+0000 7f872945d700 1 --2- 172.21.15.92:0/3354339582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00bc8b50 0x560b00b7c800 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f871c056ba0 tx=0x7f871c028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc39c2f27788b00 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.089+0000 7f870e7fc700 1 -- 172.21.15.92:0/3354339582 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c059e60 con 0x560b00bc8b50 2023-10-06T10:39:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.089+0000 7f870e7fc700 1 -- 172.21.15.92:0/3354339582 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871c056c70 con 0x560b00bc8b50 2023-10-06T10:39:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.089+0000 7f870e7fc700 1 -- 172.21.15.92:0/3354339582 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c05dbb0 con 0x560b00bc8b50 2023-10-06T10:39:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.089+0000 7f87341b8b80 1 -- 172.21.15.92:0/3354339582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00bc8b50 msgr2=0x560b00b7c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.089+0000 7f87341b8b80 1 --2- 172.21.15.92:0/3354339582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00bc8b50 0x560b00b7c800 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f871c056ba0 tx=0x7f871c028000 comp rx=0 tx=0).stop 2023-10-06T10:39:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.089+0000 7f87341b8b80 1 -- 172.21.15.92:0/3354339582 shutdown_connections 2023-10-06T10:39:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.089+0000 7f87341b8b80 1 --2- 172.21.15.92:0/3354339582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00bc8b50 0x560b00b7c800 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.089+0000 7f87341b8b80 1 --2- 172.21.15.92:0/3354339582 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560b00b82c90 0x560b00bc6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.089+0000 7f87341b8b80 1 --2- 172.21.15.92:0/3354339582 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560b00b84a60 0x560b00bc7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.089+0000 7f87341b8b80 1 -- 172.21.15.92:0/3354339582 >> 172.21.15.92:0/3354339582 conn(0x560b00b83130 msgr2=0x560b00bc3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.089+0000 7f87341b8b80 1 -- 172.21.15.92:0/3354339582 shutdown_connections 2023-10-06T10:39:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.089+0000 7f87341b8b80 1 -- 172.21.15.92:0/3354339582 wait complete. 2023-10-06T10:39:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.090+0000 7f87341b8b80 1 -- start start 2023-10-06T10:39:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.090+0000 7f87341b8b80 1 Processor -- start 2023-10-06T10:39:48.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f87341b8b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560b00b82c90 0x560b00bb0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f87341b8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00b84a60 0x560b00bc4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f87341b8b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560b00bc8b50 0x560b00bc5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f87341b8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b00c31f70 con 0x560b00b84a60 2023-10-06T10:39:48.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f87341b8b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b00bb1260 con 0x560b00b82c90 2023-10-06T10:39:48.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f87341b8b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b00bc3560 con 0x560b00bc8b50 2023-10-06T10:39:48.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f87341b8b80 10 client.0 _ll_register_callbacks cb 0x560b0096eff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:39:48.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f8723fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00b84a60 0x560b00bc4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f8723fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00b84a60 0x560b00bc4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53888/0 (socket says 172.21.15.92:53888) 2023-10-06T10:39:48.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f8723fff700 1 -- 172.21.15.92:0/3683062892 learned_addr learned my addr 172.21.15.92:0/3683062892 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:48.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f8728c5c700 1 --2- 172.21.15.92:0/3683062892 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560b00b82c90 0x560b00bb0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f872945d700 1 --2- 172.21.15.92:0/3683062892 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560b00bc8b50 0x560b00bc5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f8723fff700 1 -- 172.21.15.92:0/3683062892 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560b00bc8b50 msgr2=0x560b00bc5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:48.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f8723fff700 1 --2- 172.21.15.92:0/3683062892 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560b00bc8b50 0x560b00bc5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f8723fff700 1 -- 172.21.15.92:0/3683062892 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560b00b82c90 msgr2=0x560b00bb0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:48.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f8723fff700 1 --2- 172.21.15.92:0/3683062892 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560b00b82c90 0x560b00bb0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f8723fff700 1 -- 172.21.15.92:0/3683062892 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b0090f0a0 con 0x560b00b84a60 2023-10-06T10:39:48.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f872945d700 1 --2- 172.21.15.92:0/3683062892 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560b00bc8b50 0x560b00bc5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:48.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f8723fff700 1 --2- 172.21.15.92:0/3683062892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00b84a60 0x560b00bc4d00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f872415f0f0 tx=0x7f8710008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cdc39d34941b9f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:48.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f8702ffd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:39:48.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f8702ffd700 1 -- 172.21.15.92:0/3683062892 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8710016aa0 con 0x560b00b84a60 2023-10-06T10:39:48.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f8702ffd700 1 -- 172.21.15.92:0/3683062892 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8710016c00 con 0x560b00b84a60 2023-10-06T10:39:48.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.091+0000 7f87341b8b80 1 -- 172.21.15.92:0/3683062892 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8718013760 con 0x560b00b84a60 2023-10-06T10:39:48.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.092+0000 7f8702ffd700 1 -- 172.21.15.92:0/3683062892 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8710023450 con 0x560b00b84a60 2023-10-06T10:39:48.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.092+0000 7f87341b8b80 10 client.5463 fetch_fsmap learned FSMap version 23 2023-10-06T10:39:48.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.092+0000 7f87341b8b80 1 -- 172.21.15.92:0/3683062892 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x560b00aa0840 con 0x560b00b84a60 2023-10-06T10:39:48.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.092+0000 7f8702ffd700 1 -- 172.21.15.92:0/3683062892 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8718013760 con 0x560b00b84a60 2023-10-06T10:39:48.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.092+0000 7f8702ffd700 1 -- 172.21.15.92:0/3683062892 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 23) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f87100123a0 con 0x560b00b84a60 2023-10-06T10:39:48.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.092+0000 7f87341b8b80 10 client.5463 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T10:39:48.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.092+0000 7f87341b8b80 10 client.5463 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2023-10-06T10:39:48.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.092+0000 7f87341b8b80 20 client.5463 populate_metadata read hostname 'smithi092' 2023-10-06T10:39:48.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.092+0000 7f87341b8b80 1 --2- 172.21.15.92:0/3683062892 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x560b00bab8c0 0x560b00c2e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.092+0000 7f87341b8b80 4 client.5463 mds_command: new command op to 5133 tid=0[{"prefix": "tag path", "path": "/d1/d2/d3", "tag": "tag123"}] 2023-10-06T10:39:48.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.092+0000 7f87341b8b80 1 -- 172.21.15.92:0/3683062892 --> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] -- command(tid 0: {"prefix": "tag path", "path": "/d1/d2/d3", "tag": "tag123"}) v1 -- 0x560b00a74ec0 con 0x560b00bab8c0 2023-10-06T10:39:48.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.092+0000 7f8728c5c700 1 --2- 172.21.15.92:0/3683062892 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x560b00bab8c0 0x560b00c2e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.092+0000 7f8702ffd700 10 client.5463 ms_handle_connect on v2:172.21.15.124:6832/484869337 2023-10-06T10:39:48.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.092+0000 7f8728c5c700 1 --2- 172.21.15.92:0/3683062892 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x560b00bab8c0 0x560b00c2e570 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=199efbe642dc2010 server_cookie=b1c88f558554f2c4 in_seq=0 out_seq=0 2023-10-06T10:39:48.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.093+0000 7f8702ffd700 1 -- 172.21.15.92:0/3683062892 <== mds.0 v2:172.21.15.124:6832/484869337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+80 (crc 0 0 0) 0x560b00a74ec0 con 0x560b00bab8c0 2023-10-06T10:39:48.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.093+0000 7f8702ffd700 10 client.5463 handle_command_reply: tid=0 2023-10-06T10:39:48.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.093+0000 7f87341b8b80 1 client.5463 shutdown 2023-10-06T10:39:48.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.093+0000 7f87341b8b80 1 -- 172.21.15.92:0/3683062892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00b84a60 msgr2=0x560b00bc4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:48.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.093+0000 7f87341b8b80 1 --2- 172.21.15.92:0/3683062892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00b84a60 0x560b00bc4d00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f872415f0f0 tx=0x7f8710008040 comp rx=0 tx=0).stop 2023-10-06T10:39:48.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.093+0000 7f87341b8b80 1 -- 172.21.15.92:0/3683062892 shutdown_connections 2023-10-06T10:39:48.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.093+0000 7f87341b8b80 1 --2- 172.21.15.92:0/3683062892 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x560b00bab8c0 0x560b00c2e570 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.093+0000 7f87341b8b80 1 --2- 172.21.15.92:0/3683062892 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560b00bc8b50 0x560b00bc5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.093+0000 7f87341b8b80 1 --2- 172.21.15.92:0/3683062892 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560b00b82c90 0x560b00bb0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.093+0000 7f8702ffd700 0 client.5463 ms_handle_reset on v2:172.21.15.124:6832/484869337 2023-10-06T10:39:48.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.093+0000 7f87341b8b80 1 --2- 172.21.15.92:0/3683062892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b00b84a60 0x560b00bc4d00 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.093+0000 7f87341b8b80 1 -- 172.21.15.92:0/3683062892 >> 172.21.15.92:0/3683062892 conn(0x560b00b83130 msgr2=0x560b00bc3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:48.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.093+0000 7f87341b8b80 1 -- 172.21.15.92:0/3683062892 shutdown_connections 2023-10-06T10:39:48.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.093+0000 7f87341b8b80 1 -- 172.21.15.92:0/3683062892 wait complete. 2023-10-06T10:39:48.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.093+0000 7f87341b8b80 20 client.5463 trim_cache size 0 max 16384 2023-10-06T10:39:48.101 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:39:48.102 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0, 2023-10-06T10:39:48.102 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_tag": "tag123", 2023-10-06T10:39:48.102 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": "asynchronous" 2023-10-06T10:39:48.102 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:39:48.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.094+0000 7f872aec0700 1 -- 172.21.15.92:0/3798789691 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8708066f30 msgr2=0x7f87080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.094+0000 7f872aec0700 1 --2- 172.21.15.92:0/3798789691 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8708066f30 0x7f87080693e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8724063c10 tx=0x7f8718007ef0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.094+0000 7f872aec0700 1 -- 172.21.15.92:0/3798789691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87240b2360 msgr2=0x7f87240cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.094+0000 7f872aec0700 1 --2- 172.21.15.92:0/3798789691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87240b2360 0x7f87240cc950 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f871c009430 tx=0x7f871c018000 comp rx=0 tx=0).stop 2023-10-06T10:39:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.094+0000 7f872aec0700 1 -- 172.21.15.92:0/3798789691 shutdown_connections 2023-10-06T10:39:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.094+0000 7f872aec0700 1 --2- 172.21.15.92:0/3798789691 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8708066f30 0x7f87080693e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.094+0000 7f872aec0700 1 --2- 172.21.15.92:0/3798789691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87240b2360 0x7f87240cc950 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.094+0000 7f872aec0700 1 --2- 172.21.15.92:0/3798789691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f872406ece0 0x7f87240c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.094+0000 7f872aec0700 1 --2- 172.21.15.92:0/3798789691 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87240aede0 0x7f8724149790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.094+0000 7f872aec0700 1 -- 172.21.15.92:0/3798789691 >> 172.21.15.92:0/3798789691 conn(0x7f87240682f0 msgr2=0x7f87240b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.095+0000 7f872aec0700 1 -- 172.21.15.92:0/3798789691 shutdown_connections 2023-10-06T10:39:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.095+0000 7f872aec0700 1 -- 172.21.15.92:0/3798789691 wait complete. 2023-10-06T10:39:48.108 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub status 2023-10-06T10:39:48.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.222+0000 7f7a83fff700 1 Processor -- start 2023-10-06T10:39:48.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.222+0000 7f7a83fff700 1 -- start start 2023-10-06T10:39:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.223+0000 7f7a83fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7a840b29f0 0x7f7a84164320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.223+0000 7f7a83fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a840cc970 con 0x7f7a840cd3c0 2023-10-06T10:39:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.223+0000 7f7a83fff700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a840cbd60 con 0x7f7a840b2620 2023-10-06T10:39:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.223+0000 7f7a83fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a840cc040 con 0x7f7a840b29f0 2023-10-06T10:39:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.223+0000 7f7a837fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7a840b29f0 0x7f7a84164320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.223+0000 7f7a837fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7a840b29f0 0x7f7a84164320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53902/0 (socket says 172.21.15.92:53902) 2023-10-06T10:39:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.223+0000 7f7a837fe700 1 -- 172.21.15.92:0/2924508139 learned_addr learned my addr 172.21.15.92:0/2924508139 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.224+0000 7f7a837fe700 1 -- 172.21.15.92:0/2924508139 >> v1:172.21.15.124:6789/0 conn(0x7f7a840b2620 legacy=0x7f7a840cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.224+0000 7f7a837fe700 1 -- 172.21.15.92:0/2924508139 >> v1:172.21.15.92:6789/0 conn(0x7f7a840cd3c0 legacy=0x7f7a840ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:48.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.224+0000 7f7a837fe700 1 -- 172.21.15.92:0/2924508139 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a840cc320 con 0x7f7a840b29f0 2023-10-06T10:39:48.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.224+0000 7f7a837fe700 1 --2- 172.21.15.92:0/2924508139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a840b29f0 0x7f7a84164320 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009f20 tx=0x7f7a7c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c13b7b894ed306 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:48.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.224+0000 7f7a827fc700 1 -- 172.21.15.92:0/2924508139 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c00e070 con 0x7f7a840b29f0 2023-10-06T10:39:48.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.224+0000 7f7a827fc700 1 -- 172.21.15.92:0/2924508139 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c00ba80 con 0x7f7a840b29f0 2023-10-06T10:39:48.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.224+0000 7f7a827fc700 1 -- 172.21.15.92:0/2924508139 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c0092e0 con 0x7f7a840b29f0 2023-10-06T10:39:48.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.224+0000 7f7a83fff700 1 -- 172.21.15.92:0/2924508139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a840b29f0 msgr2=0x7f7a84164320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:48.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.224+0000 7f7a83fff700 1 --2- 172.21.15.92:0/2924508139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a840b29f0 0x7f7a84164320 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009f20 tx=0x7f7a7c009840 comp rx=0 tx=0).stop 2023-10-06T10:39:48.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.224+0000 7f7a83fff700 1 -- 172.21.15.92:0/2924508139 shutdown_connections 2023-10-06T10:39:48.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.224+0000 7f7a83fff700 1 --2- 172.21.15.92:0/2924508139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a840b29f0 0x7f7a84164320 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.224+0000 7f7a83fff700 1 -- 172.21.15.92:0/2924508139 >> 172.21.15.92:0/2924508139 conn(0x7f7a84099520 msgr2=0x7f7a8406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:48.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.225+0000 7f7a83fff700 1 -- 172.21.15.92:0/2924508139 shutdown_connections 2023-10-06T10:39:48.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.225+0000 7f7a83fff700 1 -- 172.21.15.92:0/2924508139 wait complete. 2023-10-06T10:39:48.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.226+0000 7f7a83fff700 1 Processor -- start 2023-10-06T10:39:48.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.226+0000 7f7a83fff700 1 -- start start 2023-10-06T10:39:48.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.226+0000 7f7a83fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a840b2620 0x7f7a840b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.226+0000 7f7a83fff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a840b29f0 0x7f7a84158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.226+0000 7f7a83fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a840cd3c0 0x7f7a84158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.226+0000 7f7a83fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a84169a20 con 0x7f7a840b2620 2023-10-06T10:39:48.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.226+0000 7f7a83fff700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a84169cc0 con 0x7f7a840b29f0 2023-10-06T10:39:48.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.226+0000 7f7a83fff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a84169fa0 con 0x7f7a840cd3c0 2023-10-06T10:39:48.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.226+0000 7f7a82ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a840b2620 0x7f7a840b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.226+0000 7f7a82ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a840b2620 0x7f7a840b54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53918/0 (socket says 172.21.15.92:53918) 2023-10-06T10:39:48.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.226+0000 7f7a82ffd700 1 -- 172.21.15.92:0/637828813 learned_addr learned my addr 172.21.15.92:0/637828813 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:48.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.227+0000 7f7a837fe700 1 --2- 172.21.15.92:0/637828813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a840cd3c0 0x7f7a84158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.227+0000 7f7a7bfff700 1 --2- 172.21.15.92:0/637828813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a840b29f0 0x7f7a84158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.227+0000 7f7a82ffd700 1 -- 172.21.15.92:0/637828813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a840cd3c0 msgr2=0x7f7a84158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:48.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.227+0000 7f7a82ffd700 1 --2- 172.21.15.92:0/637828813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a840cd3c0 0x7f7a84158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.227+0000 7f7a82ffd700 1 -- 172.21.15.92:0/637828813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a840b29f0 msgr2=0x7f7a84158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:48.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.227+0000 7f7a82ffd700 1 --2- 172.21.15.92:0/637828813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a840b29f0 0x7f7a84158140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.227+0000 7f7a82ffd700 1 -- 172.21.15.92:0/637828813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7c009ae0 con 0x7f7a840b2620 2023-10-06T10:39:48.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.227+0000 7f7a837fe700 1 --2- 172.21.15.92:0/637828813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a840cd3c0 0x7f7a84158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:48.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.227+0000 7f7a82ffd700 1 --2- 172.21.15.92:0/637828813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a840b2620 0x7f7a840b54b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400c540 tx=0x7f7a74001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:48.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.227+0000 7f7a7affd700 1 -- 172.21.15.92:0/637828813 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a74011590 con 0x7f7a840b2620 2023-10-06T10:39:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.227+0000 7f7a7affd700 1 -- 172.21.15.92:0/637828813 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a740116f0 con 0x7f7a840b2620 2023-10-06T10:39:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.227+0000 7f7a83fff700 1 -- 172.21.15.92:0/637828813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8416a280 con 0x7f7a840b2620 2023-10-06T10:39:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.227+0000 7f7a7affd700 1 -- 172.21.15.92:0/637828813 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a74011ab0 con 0x7f7a840b2620 2023-10-06T10:39:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.227+0000 7f7a83fff700 1 -- 172.21.15.92:0/637828813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8416aa80 con 0x7f7a840b2620 2023-10-06T10:39:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.228+0000 7f7a7affd700 1 -- 172.21.15.92:0/637828813 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a7401b3b0 con 0x7f7a840b2620 2023-10-06T10:39:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.229+0000 7f7a7affd700 1 --2- 172.21.15.92:0/637828813 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7a64066f30 0x7f7a640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.229+0000 7f7a7affd700 1 -- 172.21.15.92:0/637828813 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7a74083c90 con 0x7f7a840b2620 2023-10-06T10:39:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.229+0000 7f7a7bfff700 1 --2- 172.21.15.92:0/637828813 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7a64066f30 0x7f7a640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.229+0000 7f7a7bfff700 1 --2- 172.21.15.92:0/637828813 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7a64066f30 0x7f7a640693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7a8415f810 tx=0x7f7a6c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.231+0000 7f7a927ccb80 1 Processor -- start 2023-10-06T10:39:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.231+0000 7f7a927ccb80 1 -- start start 2023-10-06T10:39:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.231+0000 7f7a927ccb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562ec46ac9a0 0x562ec46acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.231+0000 7f7a927ccb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46f6c10 0x562ec46d5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.232+0000 7f7a927ccb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46d5eb0 0x562ec46a89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.232+0000 7f7a927ccb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec462ccd0 con 0x562ec46d5eb0 2023-10-06T10:39:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.232+0000 7f7a927ccb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec4603ac0 con 0x562ec46f6c10 2023-10-06T10:39:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.232+0000 7f7a927ccb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ec46012c0 con 0x562ec46ac9a0 2023-10-06T10:39:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.232+0000 7f7a837fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46d5eb0 0x562ec46a89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.232+0000 7f7a837fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46d5eb0 0x562ec46a89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53924/0 (socket says 172.21.15.92:53924) 2023-10-06T10:39:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.232+0000 7f7a837fe700 1 -- 172.21.15.92:0/543797721 learned_addr learned my addr 172.21.15.92:0/543797721 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.232+0000 7f7a82ffd700 1 --2- 172.21.15.92:0/543797721 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562ec46ac9a0 0x562ec46acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.232+0000 7f7a7bfff700 1 --2- 172.21.15.92:0/543797721 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46f6c10 0x562ec46d5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.232+0000 7f7a837fe700 1 -- 172.21.15.92:0/543797721 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562ec46ac9a0 msgr2=0x562ec46acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.232+0000 7f7a837fe700 1 --2- 172.21.15.92:0/543797721 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562ec46ac9a0 0x562ec46acd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.232+0000 7f7a837fe700 1 -- 172.21.15.92:0/543797721 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46f6c10 msgr2=0x562ec46d5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:48.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.232+0000 7f7a837fe700 1 --2- 172.21.15.92:0/543797721 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46f6c10 0x562ec46d5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.232+0000 7f7a837fe700 1 -- 172.21.15.92:0/543797721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ec44972c0 con 0x562ec46d5eb0 2023-10-06T10:39:48.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.232+0000 7f7a7bfff700 1 --2- 172.21.15.92:0/543797721 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46f6c10 0x562ec46d5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:48.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.232+0000 7f7a837fe700 1 --2- 172.21.15.92:0/543797721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46d5eb0 0x562ec46a89e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7a8406fcc0 tx=0x7f7a7c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e8bc9a0fc9a790 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:48.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.232+0000 7f7a63fff700 1 -- 172.21.15.92:0/543797721 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c00be90 con 0x562ec46d5eb0 2023-10-06T10:39:48.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.232+0000 7f7a63fff700 1 -- 172.21.15.92:0/543797721 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c00bbf0 con 0x562ec46d5eb0 2023-10-06T10:39:48.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.232+0000 7f7a63fff700 1 -- 172.21.15.92:0/543797721 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c007840 con 0x562ec46d5eb0 2023-10-06T10:39:48.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.233+0000 7f7a927ccb80 1 -- 172.21.15.92:0/543797721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46d5eb0 msgr2=0x562ec46a89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:48.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.233+0000 7f7a927ccb80 1 --2- 172.21.15.92:0/543797721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46d5eb0 0x562ec46a89e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7a8406fcc0 tx=0x7f7a7c018000 comp rx=0 tx=0).stop 2023-10-06T10:39:48.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.233+0000 7f7a927ccb80 1 -- 172.21.15.92:0/543797721 shutdown_connections 2023-10-06T10:39:48.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.233+0000 7f7a927ccb80 1 --2- 172.21.15.92:0/543797721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46d5eb0 0x562ec46a89e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.233+0000 7f7a927ccb80 1 --2- 172.21.15.92:0/543797721 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562ec46ac9a0 0x562ec46acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.233+0000 7f7a927ccb80 1 --2- 172.21.15.92:0/543797721 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46f6c10 0x562ec46d5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.233+0000 7f7a927ccb80 1 -- 172.21.15.92:0/543797721 >> 172.21.15.92:0/543797721 conn(0x562ec46ae2c0 msgr2=0x562ec46ae690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:48.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.233+0000 7f7a927ccb80 1 -- 172.21.15.92:0/543797721 shutdown_connections 2023-10-06T10:39:48.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.233+0000 7f7a927ccb80 1 -- 172.21.15.92:0/543797721 wait complete. 2023-10-06T10:39:48.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.234+0000 7f7a927ccb80 1 -- start start 2023-10-06T10:39:48.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.234+0000 7f7a927ccb80 1 Processor -- start 2023-10-06T10:39:48.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.234+0000 7f7a927ccb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46ac9a0 0x562ec46d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.234+0000 7f7a927ccb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46d5eb0 0x562ec46d3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.234+0000 7f7a927ccb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562ec46f6c10 0x562ec46f0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a927ccb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec45fd0b0 con 0x562ec46ac9a0 2023-10-06T10:39:48.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a927ccb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec45a88e0 con 0x562ec46d5eb0 2023-10-06T10:39:48.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a927ccb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ec45c3ef0 con 0x562ec46f6c10 2023-10-06T10:39:48.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a927ccb80 10 client.0 _ll_register_callbacks cb 0x562ec4499f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:39:48.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a82ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46ac9a0 0x562ec46d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a82ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46ac9a0 0x562ec46d2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53926/0 (socket says 172.21.15.92:53926) 2023-10-06T10:39:48.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a82ffd700 1 -- 172.21.15.92:0/3498415292 learned_addr learned my addr 172.21.15.92:0/3498415292 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:48.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a7bfff700 1 --2- 172.21.15.92:0/3498415292 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46d5eb0 0x562ec46d3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a837fe700 1 --2- 172.21.15.92:0/3498415292 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562ec46f6c10 0x562ec46f0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a82ffd700 1 -- 172.21.15.92:0/3498415292 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562ec46f6c10 msgr2=0x562ec46f0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:48.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a82ffd700 1 --2- 172.21.15.92:0/3498415292 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562ec46f6c10 0x562ec46f0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a82ffd700 1 -- 172.21.15.92:0/3498415292 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46d5eb0 msgr2=0x562ec46d3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:48.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a82ffd700 1 --2- 172.21.15.92:0/3498415292 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46d5eb0 0x562ec46d3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a82ffd700 1 -- 172.21.15.92:0/3498415292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a840cc320 con 0x562ec46ac9a0 2023-10-06T10:39:48.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a837fe700 1 --2- 172.21.15.92:0/3498415292 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562ec46f6c10 0x562ec46f0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:48.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a82ffd700 1 --2- 172.21.15.92:0/3498415292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46ac9a0 0x562ec46d2330 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400cc90 tx=0x7f7a7401e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79455dcd8d36052 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:48.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a617fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:39:48.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a617fa700 1 -- 172.21.15.92:0/3498415292 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7406aea0 con 0x562ec46ac9a0 2023-10-06T10:39:48.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a617fa700 1 -- 172.21.15.92:0/3498415292 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7a74052a70 con 0x562ec46ac9a0 2023-10-06T10:39:48.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a927ccb80 1 -- 172.21.15.92:0/3498415292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ec46a16e0 con 0x562ec46ac9a0 2023-10-06T10:39:48.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a617fa700 1 -- 172.21.15.92:0/3498415292 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a74053e20 con 0x562ec46ac9a0 2023-10-06T10:39:48.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a927ccb80 10 client.5478 fetch_fsmap learned FSMap version 23 2023-10-06T10:39:48.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a927ccb80 1 -- 172.21.15.92:0/3498415292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x562ec443a030 con 0x562ec46ac9a0 2023-10-06T10:39:48.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a617fa700 1 -- 172.21.15.92:0/3498415292 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562ec46a16e0 con 0x562ec46ac9a0 2023-10-06T10:39:48.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.235+0000 7f7a617fa700 1 -- 172.21.15.92:0/3498415292 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 23) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f7a74070250 con 0x562ec46ac9a0 2023-10-06T10:39:48.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.236+0000 7f7a927ccb80 10 client.5478 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T10:39:48.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.236+0000 7f7a927ccb80 10 client.5478 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2023-10-06T10:39:48.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.236+0000 7f7a927ccb80 20 client.5478 populate_metadata read hostname 'smithi092' 2023-10-06T10:39:48.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.236+0000 7f7a927ccb80 1 --2- 172.21.15.92:0/3498415292 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x562ec46efe30 0x562ec46eab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.236+0000 7f7a927ccb80 4 client.5478 mds_command: new command op to 5133 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:39:48.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.236+0000 7f7a927ccb80 1 -- 172.21.15.92:0/3498415292 --> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ec467e610 con 0x562ec46efe30 2023-10-06T10:39:48.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.236+0000 7f7a7bfff700 1 --2- 172.21.15.92:0/3498415292 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x562ec46efe30 0x562ec46eab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.236+0000 7f7a617fa700 10 client.5478 ms_handle_connect on v2:172.21.15.124:6832/484869337 2023-10-06T10:39:48.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.236+0000 7f7a7bfff700 1 --2- 172.21.15.92:0/3498415292 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x562ec46efe30 0x562ec46eab30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8ef8a546909f467 server_cookie=42a1ad2efe5c6788 in_seq=0 out_seq=0 2023-10-06T10:39:48.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.237+0000 7f7a617fa700 1 -- 172.21.15.92:0/3498415292 <== mds.0 v2:172.21.15.124:6832/484869337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x562ec467e610 con 0x562ec46efe30 2023-10-06T10:39:48.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.237+0000 7f7a617fa700 10 client.5478 handle_command_reply: tid=0 2023-10-06T10:39:48.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.237+0000 7f7a927ccb80 1 client.5478 shutdown 2023-10-06T10:39:48.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.237+0000 7f7a927ccb80 1 -- 172.21.15.92:0/3498415292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46ac9a0 msgr2=0x562ec46d2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:48.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.237+0000 7f7a927ccb80 1 --2- 172.21.15.92:0/3498415292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46ac9a0 0x562ec46d2330 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400cc90 tx=0x7f7a7401e040 comp rx=0 tx=0).stop 2023-10-06T10:39:48.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.237+0000 7f7a927ccb80 1 -- 172.21.15.92:0/3498415292 shutdown_connections 2023-10-06T10:39:48.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.237+0000 7f7a927ccb80 1 --2- 172.21.15.92:0/3498415292 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x562ec46efe30 0x562ec46eab30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.237+0000 7f7a927ccb80 1 --2- 172.21.15.92:0/3498415292 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562ec46f6c10 0x562ec46f0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.237+0000 7f7a927ccb80 1 --2- 172.21.15.92:0/3498415292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46ac9a0 0x562ec46d2330 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.237+0000 7f7a617fa700 0 client.5478 ms_handle_reset on v2:172.21.15.124:6832/484869337 2023-10-06T10:39:48.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.237+0000 7f7a927ccb80 1 --2- 172.21.15.92:0/3498415292 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46d5eb0 0x562ec46d3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.237+0000 7f7a927ccb80 1 -- 172.21.15.92:0/3498415292 >> 172.21.15.92:0/3498415292 conn(0x562ec46ae2c0 msgr2=0x562ec46a31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:48.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.237+0000 7f7a927ccb80 1 -- 172.21.15.92:0/3498415292 shutdown_connections 2023-10-06T10:39:48.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.237+0000 7f7a927ccb80 1 -- 172.21.15.92:0/3498415292 wait complete. 2023-10-06T10:39:48.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.237+0000 7f7a927ccb80 20 client.5478 trim_cache size 0 max 16384 2023-10-06T10:39:48.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.253+0000 7f7a927ccb80 1 Processor -- start 2023-10-06T10:39:48.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.253+0000 7f7a927ccb80 1 -- start start 2023-10-06T10:39:48.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.254+0000 7f7a927ccb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46ac9a0 0x562ec46ed400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.254+0000 7f7a927ccb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562ec46d5eb0 0x562ec46d2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.254+0000 7f7a927ccb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46efe30 0x562ec46d3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.254+0000 7f7a927ccb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec447fe80 con 0x562ec46ac9a0 2023-10-06T10:39:48.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.254+0000 7f7a927ccb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec46eb240 con 0x562ec46efe30 2023-10-06T10:39:48.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.254+0000 7f7a927ccb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ec46f4930 con 0x562ec46d5eb0 2023-10-06T10:39:48.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.254+0000 7f7a82ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46ac9a0 0x562ec46ed400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.254+0000 7f7a82ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46ac9a0 0x562ec46ed400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53940/0 (socket says 172.21.15.92:53940) 2023-10-06T10:39:48.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.254+0000 7f7a82ffd700 1 -- 172.21.15.92:0/703597313 learned_addr learned my addr 172.21.15.92:0/703597313 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:48.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.254+0000 7f7a7bfff700 1 --2- 172.21.15.92:0/703597313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562ec46d5eb0 0x562ec46d2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.254+0000 7f7a837fe700 1 --2- 172.21.15.92:0/703597313 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46efe30 0x562ec46d3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.254+0000 7f7a82ffd700 1 -- 172.21.15.92:0/703597313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562ec46d5eb0 msgr2=0x562ec46d2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:48.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.254+0000 7f7a82ffd700 1 --2- 172.21.15.92:0/703597313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562ec46d5eb0 0x562ec46d2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.254+0000 7f7a82ffd700 1 -- 172.21.15.92:0/703597313 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46efe30 msgr2=0x562ec46d3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:48.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.254+0000 7f7a82ffd700 1 --2- 172.21.15.92:0/703597313 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46efe30 0x562ec46d3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.254+0000 7f7a82ffd700 1 -- 172.21.15.92:0/703597313 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ec45cb7d0 con 0x562ec46ac9a0 2023-10-06T10:39:48.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.254+0000 7f7a837fe700 1 --2- 172.21.15.92:0/703597313 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46efe30 0x562ec46d3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:48.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.254+0000 7f7a7bfff700 1 --2- 172.21.15.92:0/703597313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562ec46d5eb0 0x562ec46d2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:39:48.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.254+0000 7f7a82ffd700 1 --2- 172.21.15.92:0/703597313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46ac9a0 0x562ec46ed400 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7a74054490 tx=0x7f7a74003000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f8b08c7e3cac4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:48.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.254+0000 7f7a78ff9700 1 -- 172.21.15.92:0/703597313 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7406aea0 con 0x562ec46ac9a0 2023-10-06T10:39:48.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.255+0000 7f7a78ff9700 1 -- 172.21.15.92:0/703597313 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a74069070 con 0x562ec46ac9a0 2023-10-06T10:39:48.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.255+0000 7f7a78ff9700 1 -- 172.21.15.92:0/703597313 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a74011c10 con 0x562ec46ac9a0 2023-10-06T10:39:48.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.255+0000 7f7a927ccb80 1 -- 172.21.15.92:0/703597313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46ac9a0 msgr2=0x562ec46ed400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:48.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.255+0000 7f7a927ccb80 1 --2- 172.21.15.92:0/703597313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46ac9a0 0x562ec46ed400 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7a74054490 tx=0x7f7a74003000 comp rx=0 tx=0).stop 2023-10-06T10:39:48.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.255+0000 7f7a927ccb80 1 -- 172.21.15.92:0/703597313 shutdown_connections 2023-10-06T10:39:48.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.255+0000 7f7a927ccb80 1 --2- 172.21.15.92:0/703597313 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46efe30 0x562ec46d3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.255+0000 7f7a927ccb80 1 --2- 172.21.15.92:0/703597313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46ac9a0 0x562ec46ed400 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.255+0000 7f7a927ccb80 1 --2- 172.21.15.92:0/703597313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562ec46d5eb0 0x562ec46d2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.255+0000 7f7a927ccb80 1 -- 172.21.15.92:0/703597313 >> 172.21.15.92:0/703597313 conn(0x562ec46ae2c0 msgr2=0x562ec46eab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:48.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.255+0000 7f7a927ccb80 1 -- 172.21.15.92:0/703597313 shutdown_connections 2023-10-06T10:39:48.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.255+0000 7f7a927ccb80 1 -- 172.21.15.92:0/703597313 wait complete. 2023-10-06T10:39:48.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.256+0000 7f7a927ccb80 1 -- start start 2023-10-06T10:39:48.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.256+0000 7f7a927ccb80 1 Processor -- start 2023-10-06T10:39:48.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.256+0000 7f7a927ccb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46ac9a0 0x562ec4758730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.256+0000 7f7a927ccb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562ec46d5eb0 0x562ec4710860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.256+0000 7f7a927ccb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46efe30 0x562ec46f3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.256+0000 7f7a927ccb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec4705cd0 con 0x562ec46ac9a0 2023-10-06T10:39:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.256+0000 7f7a927ccb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec4710da0 con 0x562ec46efe30 2023-10-06T10:39:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.256+0000 7f7a927ccb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ec47111d0 con 0x562ec46d5eb0 2023-10-06T10:39:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a927ccb80 10 client.0 _ll_register_callbacks cb 0x562ec4499f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:39:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a82ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46ac9a0 0x562ec4758730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a82ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46ac9a0 0x562ec4758730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53956/0 (socket says 172.21.15.92:53956) 2023-10-06T10:39:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a82ffd700 1 -- 172.21.15.92:0/3402637893 learned_addr learned my addr 172.21.15.92:0/3402637893 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:39:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a7bfff700 1 --2- 172.21.15.92:0/3402637893 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562ec46d5eb0 0x562ec4710860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a837fe700 1 --2- 172.21.15.92:0/3402637893 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46efe30 0x562ec46f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a82ffd700 1 -- 172.21.15.92:0/3402637893 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562ec46d5eb0 msgr2=0x562ec4710860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a82ffd700 1 --2- 172.21.15.92:0/3402637893 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562ec46d5eb0 0x562ec4710860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a82ffd700 1 -- 172.21.15.92:0/3402637893 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46efe30 msgr2=0x562ec46f3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:39:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a82ffd700 1 --2- 172.21.15.92:0/3402637893 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46efe30 0x562ec46f3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a82ffd700 1 -- 172.21.15.92:0/3402637893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ec443a030 con 0x562ec46ac9a0 2023-10-06T10:39:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a837fe700 1 --2- 172.21.15.92:0/3402637893 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46efe30 0x562ec46f3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:39:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a82ffd700 1 --2- 172.21.15.92:0/3402637893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46ac9a0 0x562ec4758730 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7a74007fd0 tx=0x7f7a7406c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92a447b2746cfeb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:39:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a617fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:39:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a617fa700 1 -- 172.21.15.92:0/3402637893 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7406aea0 con 0x562ec46ac9a0 2023-10-06T10:39:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a617fa700 1 -- 172.21.15.92:0/3402637893 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7a74053890 con 0x562ec46ac9a0 2023-10-06T10:39:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a927ccb80 1 -- 172.21.15.92:0/3402637893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a7c00bbf0 con 0x562ec46ac9a0 2023-10-06T10:39:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a617fa700 1 -- 172.21.15.92:0/3402637893 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a7405b5a0 con 0x562ec46ac9a0 2023-10-06T10:39:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a927ccb80 10 client.5484 fetch_fsmap learned FSMap version 23 2023-10-06T10:39:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a927ccb80 1 -- 172.21.15.92:0/3402637893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x562ec45b24c0 con 0x562ec46ac9a0 2023-10-06T10:39:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a617fa700 1 -- 172.21.15.92:0/3402637893 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c00bbf0 con 0x562ec46ac9a0 2023-10-06T10:39:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a617fa700 1 -- 172.21.15.92:0/3402637893 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 23) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f7a74011c10 con 0x562ec46ac9a0 2023-10-06T10:39:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a927ccb80 10 client.5484 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T10:39:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a927ccb80 10 client.5484 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2023-10-06T10:39:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.257+0000 7f7a927ccb80 20 client.5484 populate_metadata read hostname 'smithi092' 2023-10-06T10:39:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.258+0000 7f7a927ccb80 1 --2- 172.21.15.92:0/3402637893 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x562ec46f6c10 0x562ec46ed220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:39:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.258+0000 7f7a927ccb80 4 client.5484 mds_command: new command op to 5133 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:39:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.258+0000 7f7a927ccb80 1 -- 172.21.15.92:0/3402637893 --> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562ec459fe50 con 0x562ec46f6c10 2023-10-06T10:39:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.258+0000 7f7a7bfff700 1 --2- 172.21.15.92:0/3402637893 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x562ec46f6c10 0x562ec46ed220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:39:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.258+0000 7f7a617fa700 10 client.5484 ms_handle_connect on v2:172.21.15.124:6832/484869337 2023-10-06T10:39:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.258+0000 7f7a7bfff700 1 --2- 172.21.15.92:0/3402637893 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x562ec46f6c10 0x562ec46ed220 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8803e45dcefe3296 server_cookie=f6c8bf480fecf216 in_seq=0 out_seq=0 2023-10-06T10:39:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.258+0000 7f7a617fa700 1 -- 172.21.15.92:0/3402637893 <== mds.0 v2:172.21.15.124:6832/484869337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+207 (crc 0 0 0) 0x562ec459fe50 con 0x562ec46f6c10 2023-10-06T10:39:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.258+0000 7f7a617fa700 10 client.5484 handle_command_reply: tid=0 2023-10-06T10:39:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.258+0000 7f7a927ccb80 1 client.5484 shutdown 2023-10-06T10:39:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.258+0000 7f7a927ccb80 1 -- 172.21.15.92:0/3402637893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46ac9a0 msgr2=0x562ec4758730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.258+0000 7f7a927ccb80 1 --2- 172.21.15.92:0/3402637893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46ac9a0 0x562ec4758730 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7a74007fd0 tx=0x7f7a7406c040 comp rx=0 tx=0).stop 2023-10-06T10:39:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.258+0000 7f7a927ccb80 1 -- 172.21.15.92:0/3402637893 shutdown_connections 2023-10-06T10:39:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.258+0000 7f7a927ccb80 1 --2- 172.21.15.92:0/3402637893 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x562ec46f6c10 0x562ec46ed220 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.258+0000 7f7a927ccb80 1 --2- 172.21.15.92:0/3402637893 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562ec46efe30 0x562ec46f3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.258+0000 7f7a927ccb80 1 --2- 172.21.15.92:0/3402637893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ec46ac9a0 0x562ec4758730 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.258+0000 7f7a927ccb80 1 --2- 172.21.15.92:0/3402637893 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562ec46d5eb0 0x562ec4710860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.258+0000 7f7a617fa700 0 client.5484 ms_handle_reset on v2:172.21.15.124:6832/484869337 2023-10-06T10:39:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.258+0000 7f7a927ccb80 1 -- 172.21.15.92:0/3402637893 >> 172.21.15.92:0/3402637893 conn(0x562ec46ae2c0 msgr2=0x562ec46a39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.258+0000 7f7a927ccb80 1 -- 172.21.15.92:0/3402637893 shutdown_connections 2023-10-06T10:39:48.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.258+0000 7f7a927ccb80 1 -- 172.21.15.92:0/3402637893 wait complete. 2023-10-06T10:39:48.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.259+0000 7f7a927ccb80 20 client.5484 trim_cache size 0 max 16384 2023-10-06T10:39:48.268 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:39:48.268 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T10:39:48.268 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:39:48.268 INFO:teuthology.orchestra.run.smithi092.stdout: "tag123": { 2023-10-06T10:39:48.268 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:39:48.268 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "tag123", 2023-10-06T10:39:48.268 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:39:48.269 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:48.269 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:39:48.269 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:39:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.259+0000 7f7a83fff700 1 -- 172.21.15.92:0/637828813 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7a64066f30 msgr2=0x7f7a640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.259+0000 7f7a83fff700 1 --2- 172.21.15.92:0/637828813 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7a64066f30 0x7f7a640693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7a8415f810 tx=0x7f7a6c009040 comp rx=0 tx=0).stop 2023-10-06T10:39:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.259+0000 7f7a83fff700 1 -- 172.21.15.92:0/637828813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a840b2620 msgr2=0x7f7a840b54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:39:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.259+0000 7f7a83fff700 1 --2- 172.21.15.92:0/637828813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a840b2620 0x7f7a840b54b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400c540 tx=0x7f7a74001070 comp rx=0 tx=0).stop 2023-10-06T10:39:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.259+0000 7f7a83fff700 1 -- 172.21.15.92:0/637828813 shutdown_connections 2023-10-06T10:39:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.259+0000 7f7a83fff700 1 --2- 172.21.15.92:0/637828813 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7a64066f30 0x7f7a640693e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.259+0000 7f7a83fff700 1 --2- 172.21.15.92:0/637828813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a840cd3c0 0x7f7a84158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.259+0000 7f7a83fff700 1 --2- 172.21.15.92:0/637828813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a840b2620 0x7f7a840b54b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.259+0000 7f7a83fff700 1 --2- 172.21.15.92:0/637828813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a840b29f0 0x7f7a84158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:39:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.259+0000 7f7a83fff700 1 -- 172.21.15.92:0/637828813 >> 172.21.15.92:0/637828813 conn(0x7f7a84099520 msgr2=0x7f7a840c5250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:39:48.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.259+0000 7f7a83fff700 1 -- 172.21.15.92:0/637828813 shutdown_connections 2023-10-06T10:39:48.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:39:48.259+0000 7f7a83fff700 1 -- 172.21.15.92:0/637828813 wait complete. 2023-10-06T10:39:48.272 INFO:tasks.cephfs.filesystem:scrub status for tag:tag123 - {'path': '/d1/d2/d3', 'tag': 'tag123', 'options': 'recursive,force'} 2023-10-06T10:39:52.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:39:52.794 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:40:18.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub status 2023-10-06T10:40:18.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.398+0000 7fd1d80f2700 1 Processor -- start 2023-10-06T10:40:18.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.399+0000 7fd1d80f2700 1 -- start start 2023-10-06T10:40:18.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.399+0000 7fd1d80f2700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1d006d610 0x7fd1d00c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.399+0000 7fd1d80f2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd1d0157bf0 0x7fd1d00ad950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.399+0000 7fd1d80f2700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd1d006d9e0 0x7fd1d00ade90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.399+0000 7fd1d80f2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1d00c7040 con 0x7fd1d0157bf0 2023-10-06T10:40:18.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.400+0000 7fd1d80f2700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1d00c7320 con 0x7fd1d006d9e0 2023-10-06T10:40:18.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.400+0000 7fd1d80f2700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d00c7600 con 0x7fd1d006d610 2023-10-06T10:40:18.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.400+0000 7fd1d568d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd1d0157bf0 0x7fd1d00ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.400+0000 7fd1d568d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd1d0157bf0 0x7fd1d00ad950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58048/0 (socket says 172.21.15.92:58048) 2023-10-06T10:40:18.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.400+0000 7fd1d568d700 1 -- 172.21.15.92:0/1662890432 learned_addr learned my addr 172.21.15.92:0/1662890432 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:18.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.400+0000 7fd1d668f700 1 --2- 172.21.15.92:0/1662890432 >> v2:172.21.15.124:3300/0 conn(0x7fd1d006d9e0 0x7fd1d00ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.400+0000 7fd1d568d700 1 -- 172.21.15.92:0/1662890432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1d006d610 msgr2=0x7fd1d00c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:18.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.400+0000 7fd1d568d700 1 --2- 172.21.15.92:0/1662890432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1d006d610 0x7fd1d00c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.400+0000 7fd1d568d700 1 -- 172.21.15.92:0/1662890432 >> v2:172.21.15.124:3300/0 conn(0x7fd1d006d9e0 msgr2=0x7fd1d00ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:18.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.400+0000 7fd1d568d700 1 --2- 172.21.15.92:0/1662890432 >> v2:172.21.15.124:3300/0 conn(0x7fd1d006d9e0 0x7fd1d00ade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.400+0000 7fd1d568d700 1 -- 172.21.15.92:0/1662890432 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d00c78e0 con 0x7fd1d0157bf0 2023-10-06T10:40:18.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.400+0000 7fd1d568d700 1 --2- 172.21.15.92:0/1662890432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1d0157bf0 0x7fd1d00ad950 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd1d00cd1a0 tx=0x7fd1c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5e08cf85b37a87 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.400+0000 7fd1d4e8c700 1 -- 172.21.15.92:0/1662890432 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c000b860 con 0x7fd1d0157bf0 2023-10-06T10:40:18.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.400+0000 7fd1d4e8c700 1 -- 172.21.15.92:0/1662890432 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c000b9c0 con 0x7fd1d0157bf0 2023-10-06T10:40:18.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.400+0000 7fd1d4e8c700 1 -- 172.21.15.92:0/1662890432 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c000bd60 con 0x7fd1d0157bf0 2023-10-06T10:40:18.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.401+0000 7fd1d80f2700 1 -- 172.21.15.92:0/1662890432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1d0157bf0 msgr2=0x7fd1d00ad950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.401+0000 7fd1d80f2700 1 --2- 172.21.15.92:0/1662890432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1d0157bf0 0x7fd1d00ad950 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd1d00cd1a0 tx=0x7fd1c0009250 comp rx=0 tx=0).stop 2023-10-06T10:40:18.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.401+0000 7fd1d80f2700 1 -- 172.21.15.92:0/1662890432 shutdown_connections 2023-10-06T10:40:18.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.401+0000 7fd1d80f2700 1 --2- 172.21.15.92:0/1662890432 >> v2:172.21.15.124:3300/0 conn(0x7fd1d006d9e0 0x7fd1d00ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.401+0000 7fd1d80f2700 1 --2- 172.21.15.92:0/1662890432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1d006d610 0x7fd1d00c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.401+0000 7fd1d80f2700 1 --2- 172.21.15.92:0/1662890432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1d0157bf0 0x7fd1d00ad950 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.401+0000 7fd1d80f2700 1 -- 172.21.15.92:0/1662890432 >> 172.21.15.92:0/1662890432 conn(0x7fd1d00cc090 msgr2=0x7fd1d00cc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:18.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.401+0000 7fd1d80f2700 1 -- 172.21.15.92:0/1662890432 shutdown_connections 2023-10-06T10:40:18.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.401+0000 7fd1d80f2700 1 -- 172.21.15.92:0/1662890432 wait complete. 2023-10-06T10:40:18.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.402+0000 7fd1d80f2700 1 Processor -- start 2023-10-06T10:40:18.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.402+0000 7fd1d80f2700 1 -- start start 2023-10-06T10:40:18.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.403+0000 7fd1d80f2700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1d006d610 0x7fd1d0156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.403+0000 7fd1d80f2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1d006d9e0 0x7fd1d0157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.403+0000 7fd1d80f2700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1d0157bf0 0x7fd1d00a8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.403+0000 7fd1d80f2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d00c5150 con 0x7fd1d006d9e0 2023-10-06T10:40:18.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.403+0000 7fd1d80f2700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d00c5430 con 0x7fd1d0157bf0 2023-10-06T10:40:18.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.403+0000 7fd1d80f2700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d00c5710 con 0x7fd1d006d610 2023-10-06T10:40:18.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.403+0000 7fd1d568d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1d006d9e0 0x7fd1d0157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.403+0000 7fd1d568d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1d006d9e0 0x7fd1d0157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58058/0 (socket says 172.21.15.92:58058) 2023-10-06T10:40:18.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.403+0000 7fd1d568d700 1 -- 172.21.15.92:0/3915998288 learned_addr learned my addr 172.21.15.92:0/3915998288 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:18.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.403+0000 7fd1d5e8e700 1 --2- 172.21.15.92:0/3915998288 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1d006d610 0x7fd1d0156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.403+0000 7fd1d668f700 1 --2- 172.21.15.92:0/3915998288 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1d0157bf0 0x7fd1d00a8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.403+0000 7fd1d568d700 1 -- 172.21.15.92:0/3915998288 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1d006d610 msgr2=0x7fd1d0156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.403+0000 7fd1d568d700 1 --2- 172.21.15.92:0/3915998288 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1d006d610 0x7fd1d0156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.403+0000 7fd1d568d700 1 -- 172.21.15.92:0/3915998288 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1d0157bf0 msgr2=0x7fd1d00a8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.403+0000 7fd1d568d700 1 --2- 172.21.15.92:0/3915998288 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1d0157bf0 0x7fd1d00a8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.403+0000 7fd1d568d700 1 -- 172.21.15.92:0/3915998288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c00094f0 con 0x7fd1d006d9e0 2023-10-06T10:40:18.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.403+0000 7fd1d568d700 1 --2- 172.21.15.92:0/3915998288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1d006d9e0 0x7fd1d0157380 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0009290 tx=0x7fd1c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.403+0000 7fd1c5ffb700 1 -- 172.21.15.92:0/3915998288 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0011e40 con 0x7fd1d006d9e0 2023-10-06T10:40:18.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.403+0000 7fd1c5ffb700 1 -- 172.21.15.92:0/3915998288 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c001f070 con 0x7fd1d006d9e0 2023-10-06T10:40:18.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.403+0000 7fd1d80f2700 1 -- 172.21.15.92:0/3915998288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d00c59f0 con 0x7fd1d006d9e0 2023-10-06T10:40:18.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.403+0000 7fd1c5ffb700 1 -- 172.21.15.92:0/3915998288 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c00266d0 con 0x7fd1d006d9e0 2023-10-06T10:40:18.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.404+0000 7fd1d80f2700 1 -- 172.21.15.92:0/3915998288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d016b420 con 0x7fd1d006d9e0 2023-10-06T10:40:18.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.405+0000 7fd1c5ffb700 1 -- 172.21.15.92:0/3915998288 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd1c0026a40 con 0x7fd1d006d9e0 2023-10-06T10:40:18.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.405+0000 7fd1c5ffb700 1 --2- 172.21.15.92:0/3915998288 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd1b4066f30 0x7fd1b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.405+0000 7fd1c5ffb700 1 -- 172.21.15.92:0/3915998288 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fd1c0021070 con 0x7fd1d006d9e0 2023-10-06T10:40:18.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.405+0000 7fd1d5e8e700 1 --2- 172.21.15.92:0/3915998288 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd1b4066f30 0x7fd1b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.406+0000 7fd1d5e8e700 1 --2- 172.21.15.92:0/3915998288 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd1b4066f30 0x7fd1b40693e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc007580 tx=0x7fd1cc009780 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.407+0000 7fd1e13eab80 1 Processor -- start 2023-10-06T10:40:18.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.407+0000 7fd1e13eab80 1 -- start start 2023-10-06T10:40:18.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.408+0000 7fd1e13eab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb1724eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.408+0000 7fd1e13eab80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16e09f0 0x563fb16dd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.408+0000 7fd1e13eab80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16dec20 0x563fb16ddb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.408+0000 7fd1e13eab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fb165dcd0 con 0x563fb1724ae0 2023-10-06T10:40:18.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.408+0000 7fd1e13eab80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fb1634ac0 con 0x563fb16dec20 2023-10-06T10:40:18.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.408+0000 7fd1e13eab80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563fb16322c0 con 0x563fb16e09f0 2023-10-06T10:40:18.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.408+0000 7fd1d5e8e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb1724eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.408+0000 7fd1d5e8e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb1724eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58074/0 (socket says 172.21.15.92:58074) 2023-10-06T10:40:18.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.408+0000 7fd1d5e8e700 1 -- 172.21.15.92:0/971891828 learned_addr learned my addr 172.21.15.92:0/971891828 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:18.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.408+0000 7fd1d668f700 1 --2- 172.21.15.92:0/971891828 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16dec20 0x563fb16ddb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.408+0000 7fd1d568d700 1 --2- 172.21.15.92:0/971891828 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16e09f0 0x563fb16dd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.408+0000 7fd1d5e8e700 1 -- 172.21.15.92:0/971891828 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16e09f0 msgr2=0x563fb16dd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:18.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.408+0000 7fd1d5e8e700 1 --2- 172.21.15.92:0/971891828 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16e09f0 0x563fb16dd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.408+0000 7fd1d5e8e700 1 -- 172.21.15.92:0/971891828 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16dec20 msgr2=0x563fb16ddb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:18.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.408+0000 7fd1d5e8e700 1 --2- 172.21.15.92:0/971891828 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16dec20 0x563fb16ddb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.408+0000 7fd1d5e8e700 1 -- 172.21.15.92:0/971891828 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fb14c82c0 con 0x563fb1724ae0 2023-10-06T10:40:18.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.408+0000 7fd1d668f700 1 --2- 172.21.15.92:0/971891828 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16dec20 0x563fb16ddb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:18.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.408+0000 7fd1d568d700 1 --2- 172.21.15.92:0/971891828 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16e09f0 0x563fb16dd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:18.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.408+0000 7fd1d5e8e700 1 --2- 172.21.15.92:0/971891828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb1724eb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0137b0 tx=0x7fd1cc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6308a2c00e5727 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.408+0000 7fd1baffd700 1 -- 172.21.15.92:0/971891828 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc009e70 con 0x563fb1724ae0 2023-10-06T10:40:18.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.409+0000 7fd1baffd700 1 -- 172.21.15.92:0/971891828 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1cc00f040 con 0x563fb1724ae0 2023-10-06T10:40:18.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.409+0000 7fd1baffd700 1 -- 172.21.15.92:0/971891828 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc0128c0 con 0x563fb1724ae0 2023-10-06T10:40:18.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.409+0000 7fd1e13eab80 1 -- 172.21.15.92:0/971891828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 msgr2=0x563fb1724eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.409+0000 7fd1e13eab80 1 --2- 172.21.15.92:0/971891828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb1724eb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0137b0 tx=0x7fd1cc014000 comp rx=0 tx=0).stop 2023-10-06T10:40:18.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.409+0000 7fd1e13eab80 1 -- 172.21.15.92:0/971891828 shutdown_connections 2023-10-06T10:40:18.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.409+0000 7fd1e13eab80 1 --2- 172.21.15.92:0/971891828 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16dec20 0x563fb16ddb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.409+0000 7fd1e13eab80 1 --2- 172.21.15.92:0/971891828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb1724eb0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.409+0000 7fd1e13eab80 1 --2- 172.21.15.92:0/971891828 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16e09f0 0x563fb16dd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.409+0000 7fd1e13eab80 1 -- 172.21.15.92:0/971891828 >> 172.21.15.92:0/971891828 conn(0x563fb16df0c0 msgr2=0x563fb16df490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:18.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.409+0000 7fd1e13eab80 1 -- 172.21.15.92:0/971891828 shutdown_connections 2023-10-06T10:40:18.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.409+0000 7fd1e13eab80 1 -- 172.21.15.92:0/971891828 wait complete. 2023-10-06T10:40:18.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.410+0000 7fd1e13eab80 1 -- start start 2023-10-06T10:40:18.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.410+0000 7fd1e13eab80 1 Processor -- start 2023-10-06T10:40:18.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1e13eab80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16dec20 0x563fb170a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1e13eab80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16e09f0 0x563fb1721570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1e13eab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb1705af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1e13eab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fb162e0b0 con 0x563fb1724ae0 2023-10-06T10:40:18.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1e13eab80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fb15d98e0 con 0x563fb16e09f0 2023-10-06T10:40:18.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1e13eab80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563fb15f4ef0 con 0x563fb16dec20 2023-10-06T10:40:18.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1e13eab80 10 client.0 _ll_register_callbacks cb 0x563fb14caf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:40:18.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1d668f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb1705af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1d668f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb1705af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58076/0 (socket says 172.21.15.92:58076) 2023-10-06T10:40:18.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1d668f700 1 -- 172.21.15.92:0/1735771982 learned_addr learned my addr 172.21.15.92:0/1735771982 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:18.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1d5e8e700 1 --2- 172.21.15.92:0/1735771982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16dec20 0x563fb170a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1d568d700 1 --2- 172.21.15.92:0/1735771982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16e09f0 0x563fb1721570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1d668f700 1 -- 172.21.15.92:0/1735771982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16dec20 msgr2=0x563fb170a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:18.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1d668f700 1 --2- 172.21.15.92:0/1735771982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16dec20 0x563fb170a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1d668f700 1 -- 172.21.15.92:0/1735771982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16e09f0 msgr2=0x563fb1721570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:18.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1d668f700 1 --2- 172.21.15.92:0/1735771982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16e09f0 0x563fb1721570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1d668f700 1 -- 172.21.15.92:0/1735771982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1cc013400 con 0x563fb1724ae0 2023-10-06T10:40:18.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1d568d700 1 --2- 172.21.15.92:0/1735771982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16e09f0 0x563fb1721570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:18.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1d668f700 1 --2- 172.21.15.92:0/1735771982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb1705af0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8011390 tx=0x7fd1c8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6560463191ea56d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1affff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:40:18.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1affff700 1 -- 172.21.15.92:0/1735771982 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8015460 con 0x563fb1724ae0 2023-10-06T10:40:18.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1e13eab80 1 -- 172.21.15.92:0/1735771982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563fb16d26e0 con 0x563fb1724ae0 2023-10-06T10:40:18.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.411+0000 7fd1affff700 1 -- 172.21.15.92:0/1735771982 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fd1c80155c0 con 0x563fb1724ae0 2023-10-06T10:40:18.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.412+0000 7fd1affff700 1 -- 172.21.15.92:0/1735771982 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1c8015990 con 0x563fb1724ae0 2023-10-06T10:40:18.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.412+0000 7fd1e13eab80 10 client.5496 fetch_fsmap learned FSMap version 24 2023-10-06T10:40:18.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.412+0000 7fd1e13eab80 1 -- 172.21.15.92:0/1735771982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x563fb146b030 con 0x563fb1724ae0 2023-10-06T10:40:18.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.412+0000 7fd1affff700 1 -- 172.21.15.92:0/1735771982 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563fb16d26e0 con 0x563fb1724ae0 2023-10-06T10:40:18.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.412+0000 7fd1affff700 1 -- 172.21.15.92:0/1735771982 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 24) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fd1c8015af0 con 0x563fb1724ae0 2023-10-06T10:40:18.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.412+0000 7fd1e13eab80 10 client.5496 fetch_fsmap finished waiting for FSMap version 24 2023-10-06T10:40:18.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.412+0000 7fd1e13eab80 10 client.5496 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2023-10-06T10:40:18.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.412+0000 7fd1e13eab80 20 client.5496 populate_metadata read hostname 'smithi092' 2023-10-06T10:40:18.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.412+0000 7fd1e13eab80 1 --2- 172.21.15.92:0/1735771982 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x563fb171da40 0x563fb171de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.412+0000 7fd1e13eab80 4 client.5496 mds_command: new command op to 5133 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:40:18.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.412+0000 7fd1e13eab80 1 -- 172.21.15.92:0/1735771982 --> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563fb16af610 con 0x563fb171da40 2023-10-06T10:40:18.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.412+0000 7fd1d5e8e700 1 --2- 172.21.15.92:0/1735771982 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x563fb171da40 0x563fb171de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.412+0000 7fd1affff700 10 client.5496 ms_handle_connect on v2:172.21.15.124:6832/484869337 2023-10-06T10:40:18.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.412+0000 7fd1d5e8e700 1 --2- 172.21.15.92:0/1735771982 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x563fb171da40 0x563fb171de40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37b4ad210ed2aafa server_cookie=db916a67e17f6bc6 in_seq=0 out_seq=0 2023-10-06T10:40:18.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.413+0000 7fd1affff700 1 -- 172.21.15.92:0/1735771982 <== mds.0 v2:172.21.15.124:6832/484869337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x563fb16af610 con 0x563fb171da40 2023-10-06T10:40:18.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.413+0000 7fd1affff700 10 client.5496 handle_command_reply: tid=0 2023-10-06T10:40:18.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.413+0000 7fd1e13eab80 1 client.5496 shutdown 2023-10-06T10:40:18.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.413+0000 7fd1e13eab80 1 -- 172.21.15.92:0/1735771982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 msgr2=0x563fb1705af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.413+0000 7fd1e13eab80 1 --2- 172.21.15.92:0/1735771982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb1705af0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8011390 tx=0x7fd1c8008040 comp rx=0 tx=0).stop 2023-10-06T10:40:18.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.413+0000 7fd1e13eab80 1 -- 172.21.15.92:0/1735771982 shutdown_connections 2023-10-06T10:40:18.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.413+0000 7fd1e13eab80 1 --2- 172.21.15.92:0/1735771982 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x563fb171da40 0x563fb171de40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.413+0000 7fd1e13eab80 1 --2- 172.21.15.92:0/1735771982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb1705af0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.413+0000 7fd1e13eab80 1 --2- 172.21.15.92:0/1735771982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16dec20 0x563fb170a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.413+0000 7fd1e13eab80 1 --2- 172.21.15.92:0/1735771982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16e09f0 0x563fb1721570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.413+0000 7fd1affff700 0 client.5496 ms_handle_reset on v2:172.21.15.124:6832/484869337 2023-10-06T10:40:18.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.413+0000 7fd1e13eab80 1 -- 172.21.15.92:0/1735771982 >> 172.21.15.92:0/1735771982 conn(0x563fb16df0c0 msgr2=0x563fb17225b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:18.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.413+0000 7fd1e13eab80 1 -- 172.21.15.92:0/1735771982 shutdown_connections 2023-10-06T10:40:18.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.413+0000 7fd1e13eab80 1 -- 172.21.15.92:0/1735771982 wait complete. 2023-10-06T10:40:18.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.413+0000 7fd1e13eab80 20 client.5496 trim_cache size 0 max 16384 2023-10-06T10:40:18.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.429+0000 7fd1e13eab80 1 Processor -- start 2023-10-06T10:40:18.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.429+0000 7fd1e13eab80 1 -- start start 2023-10-06T10:40:18.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.430+0000 7fd1e13eab80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16dec20 0x563fb17043b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.430+0000 7fd1e13eab80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16e09f0 0x563fb171e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.430+0000 7fd1e13eab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb16d6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.430+0000 7fd1e13eab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fb14b0e80 con 0x563fb1724ae0 2023-10-06T10:40:18.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.430+0000 7fd1e13eab80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fb16d6560 con 0x563fb16dec20 2023-10-06T10:40:18.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.430+0000 7fd1e13eab80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563fb16df490 con 0x563fb16e09f0 2023-10-06T10:40:18.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.430+0000 7fd1d668f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb16d6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.430+0000 7fd1d668f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb16d6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58092/0 (socket says 172.21.15.92:58092) 2023-10-06T10:40:18.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.430+0000 7fd1d668f700 1 -- 172.21.15.92:0/2443292875 learned_addr learned my addr 172.21.15.92:0/2443292875 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:18.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.430+0000 7fd1d668f700 1 -- 172.21.15.92:0/2443292875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16e09f0 msgr2=0x563fb171e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:18.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.430+0000 7fd1d568d700 1 --2- 172.21.15.92:0/2443292875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16e09f0 0x563fb171e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.430+0000 7fd1d5e8e700 1 --2- 172.21.15.92:0/2443292875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16dec20 0x563fb17043b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.430+0000 7fd1d668f700 1 --2- 172.21.15.92:0/2443292875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16e09f0 0x563fb171e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.430+0000 7fd1d668f700 1 -- 172.21.15.92:0/2443292875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16dec20 msgr2=0x563fb17043b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:18.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.430+0000 7fd1d668f700 1 --2- 172.21.15.92:0/2443292875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16dec20 0x563fb17043b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.430+0000 7fd1d668f700 1 -- 172.21.15.92:0/2443292875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fb17088f0 con 0x563fb1724ae0 2023-10-06T10:40:18.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.430+0000 7fd1d568d700 1 --2- 172.21.15.92:0/2443292875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16e09f0 0x563fb171e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:18.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.430+0000 7fd1d5e8e700 1 --2- 172.21.15.92:0/2443292875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16dec20 0x563fb17043b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:18.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.430+0000 7fd1d668f700 1 --2- 172.21.15.92:0/2443292875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb16d6020 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8001eb0 tx=0x7fd1c8022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c8dfc3d692b520 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.431+0000 7fd1bb7fe700 1 -- 172.21.15.92:0/2443292875 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8015460 con 0x563fb1724ae0 2023-10-06T10:40:18.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.431+0000 7fd1bb7fe700 1 -- 172.21.15.92:0/2443292875 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c800c040 con 0x563fb1724ae0 2023-10-06T10:40:18.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.431+0000 7fd1bb7fe700 1 -- 172.21.15.92:0/2443292875 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8015be0 con 0x563fb1724ae0 2023-10-06T10:40:18.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.431+0000 7fd1e13eab80 1 -- 172.21.15.92:0/2443292875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 msgr2=0x563fb16d6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.431+0000 7fd1e13eab80 1 --2- 172.21.15.92:0/2443292875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb16d6020 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8001eb0 tx=0x7fd1c8022000 comp rx=0 tx=0).stop 2023-10-06T10:40:18.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.431+0000 7fd1e13eab80 1 -- 172.21.15.92:0/2443292875 shutdown_connections 2023-10-06T10:40:18.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.431+0000 7fd1e13eab80 1 --2- 172.21.15.92:0/2443292875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb16d6020 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.431+0000 7fd1e13eab80 1 --2- 172.21.15.92:0/2443292875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16dec20 0x563fb17043b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.431+0000 7fd1e13eab80 1 --2- 172.21.15.92:0/2443292875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16e09f0 0x563fb171e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.431+0000 7fd1e13eab80 1 -- 172.21.15.92:0/2443292875 >> 172.21.15.92:0/2443292875 conn(0x563fb16df0c0 msgr2=0x563fb1720c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:18.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.431+0000 7fd1e13eab80 1 -- 172.21.15.92:0/2443292875 shutdown_connections 2023-10-06T10:40:18.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.431+0000 7fd1e13eab80 1 -- 172.21.15.92:0/2443292875 wait complete. 2023-10-06T10:40:18.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.432+0000 7fd1e13eab80 1 -- start start 2023-10-06T10:40:18.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.432+0000 7fd1e13eab80 1 Processor -- start 2023-10-06T10:40:18.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1e13eab80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16dec20 0x563fb1707d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1e13eab80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16e09f0 0x563fb1733b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1e13eab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb1721e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1e13eab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fb1704e40 con 0x563fb1724ae0 2023-10-06T10:40:18.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1e13eab80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fb16dea00 con 0x563fb16dec20 2023-10-06T10:40:18.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1e13eab80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563fb16ddb90 con 0x563fb16e09f0 2023-10-06T10:40:18.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1e13eab80 10 client.0 _ll_register_callbacks cb 0x563fb14caf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:40:18.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1d668f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb1721e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1d668f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb1721e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58108/0 (socket says 172.21.15.92:58108) 2023-10-06T10:40:18.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1d668f700 1 -- 172.21.15.92:0/3788994552 learned_addr learned my addr 172.21.15.92:0/3788994552 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:18.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1d5e8e700 1 --2- 172.21.15.92:0/3788994552 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16dec20 0x563fb1707d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1d568d700 1 --2- 172.21.15.92:0/3788994552 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16e09f0 0x563fb1733b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1d668f700 1 -- 172.21.15.92:0/3788994552 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16e09f0 msgr2=0x563fb1733b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:18.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1d668f700 1 --2- 172.21.15.92:0/3788994552 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16e09f0 0x563fb1733b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1d668f700 1 -- 172.21.15.92:0/3788994552 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16dec20 msgr2=0x563fb1707d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:18.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1d668f700 1 --2- 172.21.15.92:0/3788994552 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16dec20 0x563fb1707d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1d668f700 1 -- 172.21.15.92:0/3788994552 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563fb146b030 con 0x563fb1724ae0 2023-10-06T10:40:18.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1d5e8e700 1 --2- 172.21.15.92:0/3788994552 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16dec20 0x563fb1707d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:18.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1d568d700 1 --2- 172.21.15.92:0/3788994552 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16e09f0 0x563fb1733b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:18.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1d668f700 1 --2- 172.21.15.92:0/3788994552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb1721e70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8021010 tx=0x7fd1c8003050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e1662c72741067 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1affff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:40:18.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1affff700 1 -- 172.21.15.92:0/3788994552 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c801e720 con 0x563fb1724ae0 2023-10-06T10:40:18.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1affff700 1 -- 172.21.15.92:0/3788994552 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fd1c8038070 con 0x563fb1724ae0 2023-10-06T10:40:18.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1e13eab80 1 -- 172.21.15.92:0/3788994552 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1cc00f040 con 0x563fb1724ae0 2023-10-06T10:40:18.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1affff700 1 -- 172.21.15.92:0/3788994552 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1c801e880 con 0x563fb1724ae0 2023-10-06T10:40:18.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1e13eab80 10 client.5502 fetch_fsmap learned FSMap version 24 2023-10-06T10:40:18.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1e13eab80 1 -- 172.21.15.92:0/3788994552 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x563fb15fc7d0 con 0x563fb1724ae0 2023-10-06T10:40:18.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.433+0000 7fd1affff700 1 -- 172.21.15.92:0/3788994552 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1cc00f040 con 0x563fb1724ae0 2023-10-06T10:40:18.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.434+0000 7fd1affff700 1 -- 172.21.15.92:0/3788994552 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 24) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fd1c801e9e0 con 0x563fb1724ae0 2023-10-06T10:40:18.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.434+0000 7fd1e13eab80 10 client.5502 fetch_fsmap finished waiting for FSMap version 24 2023-10-06T10:40:18.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.434+0000 7fd1e13eab80 10 client.5502 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2023-10-06T10:40:18.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.434+0000 7fd1e13eab80 20 client.5502 populate_metadata read hostname 'smithi092' 2023-10-06T10:40:18.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.434+0000 7fd1e13eab80 1 --2- 172.21.15.92:0/3788994552 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x563fb17223b0 0x563fb1789750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.434+0000 7fd1e13eab80 4 client.5502 mds_command: new command op to 5133 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:40:18.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.434+0000 7fd1e13eab80 1 -- 172.21.15.92:0/3788994552 --> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563fb15d0e50 con 0x563fb17223b0 2023-10-06T10:40:18.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.434+0000 7fd1d5e8e700 1 --2- 172.21.15.92:0/3788994552 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x563fb17223b0 0x563fb1789750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.434+0000 7fd1affff700 10 client.5502 ms_handle_connect on v2:172.21.15.124:6832/484869337 2023-10-06T10:40:18.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.434+0000 7fd1d5e8e700 1 --2- 172.21.15.92:0/3788994552 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x563fb17223b0 0x563fb1789750 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9df0b33fd3524cf server_cookie=2b97949caaddbac0 in_seq=0 out_seq=0 2023-10-06T10:40:18.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.434+0000 7fd1affff700 1 -- 172.21.15.92:0/3788994552 <== mds.0 v2:172.21.15.124:6832/484869337 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563fb15d0e50 con 0x563fb17223b0 2023-10-06T10:40:18.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.434+0000 7fd1affff700 10 client.5502 handle_command_reply: tid=0 2023-10-06T10:40:18.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.434+0000 7fd1e13eab80 1 client.5502 shutdown 2023-10-06T10:40:18.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.434+0000 7fd1e13eab80 1 -- 172.21.15.92:0/3788994552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 msgr2=0x563fb1721e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.434+0000 7fd1e13eab80 1 --2- 172.21.15.92:0/3788994552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb1721e70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8021010 tx=0x7fd1c8003050 comp rx=0 tx=0).stop 2023-10-06T10:40:18.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.434+0000 7fd1e13eab80 1 -- 172.21.15.92:0/3788994552 shutdown_connections 2023-10-06T10:40:18.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.434+0000 7fd1e13eab80 1 --2- 172.21.15.92:0/3788994552 >> [v2:172.21.15.124:6832/484869337,v1:172.21.15.124:6834/484869337] conn(0x563fb17223b0 0x563fb1789750 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.434+0000 7fd1e13eab80 1 --2- 172.21.15.92:0/3788994552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fb1724ae0 0x563fb1721e70 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.434+0000 7fd1affff700 0 client.5502 ms_handle_reset on v2:172.21.15.124:6832/484869337 2023-10-06T10:40:18.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.434+0000 7fd1e13eab80 1 --2- 172.21.15.92:0/3788994552 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563fb16dec20 0x563fb1707d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.434+0000 7fd1e13eab80 1 --2- 172.21.15.92:0/3788994552 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563fb16e09f0 0x563fb1733b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.434+0000 7fd1e13eab80 1 -- 172.21.15.92:0/3788994552 >> 172.21.15.92:0/3788994552 conn(0x563fb16df0c0 msgr2=0x563fb1737140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:18.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.435+0000 7fd1e13eab80 1 -- 172.21.15.92:0/3788994552 shutdown_connections 2023-10-06T10:40:18.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.435+0000 7fd1e13eab80 1 -- 172.21.15.92:0/3788994552 wait complete. 2023-10-06T10:40:18.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.435+0000 7fd1e13eab80 20 client.5502 trim_cache size 0 max 16384 2023-10-06T10:40:18.445 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:40:18.445 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "no active scrubs running", 2023-10-06T10:40:18.445 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": {} 2023-10-06T10:40:18.445 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:40:18.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.435+0000 7fd1d80f2700 1 -- 172.21.15.92:0/3915998288 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd1b4066f30 msgr2=0x7fd1b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.435+0000 7fd1d80f2700 1 --2- 172.21.15.92:0/3915998288 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd1b4066f30 0x7fd1b40693e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc007580 tx=0x7fd1cc009780 comp rx=0 tx=0).stop 2023-10-06T10:40:18.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.435+0000 7fd1d80f2700 1 -- 172.21.15.92:0/3915998288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1d006d9e0 msgr2=0x7fd1d0157380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.435+0000 7fd1d80f2700 1 --2- 172.21.15.92:0/3915998288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1d006d9e0 0x7fd1d0157380 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0009290 tx=0x7fd1c0017000 comp rx=0 tx=0).stop 2023-10-06T10:40:18.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.435+0000 7fd1d80f2700 1 -- 172.21.15.92:0/3915998288 shutdown_connections 2023-10-06T10:40:18.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.435+0000 7fd1d80f2700 1 --2- 172.21.15.92:0/3915998288 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd1b4066f30 0x7fd1b40693e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.435+0000 7fd1d80f2700 1 --2- 172.21.15.92:0/3915998288 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1d0157bf0 0x7fd1d00a8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.435+0000 7fd1d80f2700 1 --2- 172.21.15.92:0/3915998288 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1d006d610 0x7fd1d0156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.435+0000 7fd1d80f2700 1 --2- 172.21.15.92:0/3915998288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1d006d9e0 0x7fd1d0157380 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.435+0000 7fd1d80f2700 1 -- 172.21.15.92:0/3915998288 >> 172.21.15.92:0/3915998288 conn(0x7fd1d00cc090 msgr2=0x7fd1d0158b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:18.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.435+0000 7fd1d80f2700 1 -- 172.21.15.92:0/3915998288 shutdown_connections 2023-10-06T10:40:18.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.435+0000 7fd1d80f2700 1 -- 172.21.15.92:0/3915998288 wait complete. 2023-10-06T10:40:18.449 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:40:18.449 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000002.00000000 scrub_tag 2023-10-06T10:40:18.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.485+0000 7f54ae6dc000 1 Processor -- start 2023-10-06T10:40:18.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.485+0000 7f54ae6dc000 1 -- start start 2023-10-06T10:40:18.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.485+0000 7f54ae6dc000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560aabc279c0 0x560aabd0c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.485+0000 7f54ae6dc000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560aabc27dc0 0x560aabd17b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.485+0000 7f54ae6dc000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560aabd0dc10 0x560aabc64080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.485+0000 7f54ae6dc000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aabbe0320 con 0x560aabc27dc0 2023-10-06T10:40:18.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.485+0000 7f54ae6dc000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aabb84980 con 0x560aabd0dc10 2023-10-06T10:40:18.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.485+0000 7f54ae6dc000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x560aabb84620 con 0x560aabc279c0 2023-10-06T10:40:18.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.486+0000 7f549ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560aabc27dc0 0x560aabd17b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.486+0000 7f549ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560aabc27dc0 0x560aabd17b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58116/0 (socket says 172.21.15.92:58116) 2023-10-06T10:40:18.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.486+0000 7f549ffff700 1 -- 172.21.15.92:0/2740816870 learned_addr learned my addr 172.21.15.92:0/2740816870 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:18.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.486+0000 7f54a8118700 1 --2- 172.21.15.92:0/2740816870 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560aabd0dc10 0x560aabc64080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.486+0000 7f54a7917700 1 --2- 172.21.15.92:0/2740816870 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560aabc279c0 0x560aabd0c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.486+0000 7f549ffff700 1 -- 172.21.15.92:0/2740816870 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560aabc279c0 msgr2=0x560aabd0c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:18.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.486+0000 7f549ffff700 1 --2- 172.21.15.92:0/2740816870 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560aabc279c0 0x560aabd0c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.486+0000 7f549ffff700 1 -- 172.21.15.92:0/2740816870 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560aabd0dc10 msgr2=0x560aabc64080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:18.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.486+0000 7f549ffff700 1 --2- 172.21.15.92:0/2740816870 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560aabd0dc10 0x560aabc64080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.486+0000 7f549ffff700 1 -- 172.21.15.92:0/2740816870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a000a0e0 con 0x560aabc27dc0 2023-10-06T10:40:18.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.486+0000 7f549ffff700 1 --2- 172.21.15.92:0/2740816870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560aabc27dc0 0x560aabd17b20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5490001e90 tx=0x7f5490001900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b731e8697e6a84e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.486+0000 7f54a6915700 1 -- 172.21.15.92:0/2740816870 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5490014070 con 0x560aabc27dc0 2023-10-06T10:40:18.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.486+0000 7f54a6915700 1 -- 172.21.15.92:0/2740816870 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5490008030 con 0x560aabc27dc0 2023-10-06T10:40:18.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.486+0000 7f54a6915700 1 -- 172.21.15.92:0/2740816870 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549000bd40 con 0x560aabc27dc0 2023-10-06T10:40:18.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.487+0000 7f54ae6dc000 1 -- 172.21.15.92:0/2740816870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560aabc27dc0 msgr2=0x560aabd17b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.487+0000 7f54ae6dc000 1 --2- 172.21.15.92:0/2740816870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560aabc27dc0 0x560aabd17b20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5490001e90 tx=0x7f5490001900 comp rx=0 tx=0).stop 2023-10-06T10:40:18.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.487+0000 7f54ae6dc000 1 -- 172.21.15.92:0/2740816870 shutdown_connections 2023-10-06T10:40:18.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.487+0000 7f54ae6dc000 1 --2- 172.21.15.92:0/2740816870 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560aabd0dc10 0x560aabc64080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.487+0000 7f54ae6dc000 1 --2- 172.21.15.92:0/2740816870 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560aabc279c0 0x560aabd0c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.487+0000 7f54ae6dc000 1 --2- 172.21.15.92:0/2740816870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560aabc27dc0 0x560aabd17b20 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.487+0000 7f54ae6dc000 1 -- 172.21.15.92:0/2740816870 >> 172.21.15.92:0/2740816870 conn(0x560aabc3a750 msgr2=0x560aabc7f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:18.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.487+0000 7f54ae6dc000 1 -- 172.21.15.92:0/2740816870 shutdown_connections 2023-10-06T10:40:18.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.487+0000 7f54ae6dc000 1 -- 172.21.15.92:0/2740816870 wait complete. 2023-10-06T10:40:18.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.487+0000 7f54ae6dc000 1 Processor -- start 2023-10-06T10:40:18.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.487+0000 7f54ae6dc000 1 -- start start 2023-10-06T10:40:18.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.488+0000 7f54ae6dc000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560aabc279c0 0x560aabc7e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.488+0000 7f54ae6dc000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560aabc27dc0 0x560aabd0c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.488+0000 7f54ae6dc000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560aabd0dc10 0x560aabc792d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.488+0000 7f54ae6dc000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aabb7b400 con 0x560aabd0dc10 2023-10-06T10:40:18.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.488+0000 7f54ae6dc000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aabd152d0 con 0x560aabc27dc0 2023-10-06T10:40:18.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.488+0000 7f54ae6dc000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x560aabd14020 con 0x560aabc279c0 2023-10-06T10:40:18.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.488+0000 7f54a8118700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560aabd0dc10 0x560aabc792d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.488+0000 7f54a8118700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560aabd0dc10 0x560aabc792d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58124/0 (socket says 172.21.15.92:58124) 2023-10-06T10:40:18.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.488+0000 7f54a8118700 1 -- 172.21.15.92:0/831037915 learned_addr learned my addr 172.21.15.92:0/831037915 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:18.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.488+0000 7f549ffff700 1 --2- 172.21.15.92:0/831037915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560aabc27dc0 0x560aabd0c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.488+0000 7f54a7917700 1 --2- 172.21.15.92:0/831037915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560aabc279c0 0x560aabc7e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.488+0000 7f54a8118700 1 -- 172.21.15.92:0/831037915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560aabc279c0 msgr2=0x560aabc7e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.488+0000 7f54a8118700 1 --2- 172.21.15.92:0/831037915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560aabc279c0 0x560aabc7e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.488+0000 7f54a8118700 1 -- 172.21.15.92:0/831037915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560aabc27dc0 msgr2=0x560aabd0c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.488+0000 7f54a8118700 1 --2- 172.21.15.92:0/831037915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560aabc27dc0 0x560aabd0c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.488+0000 7f54a8118700 1 -- 172.21.15.92:0/831037915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549000a040 con 0x560aabd0dc10 2023-10-06T10:40:18.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.489+0000 7f54a8118700 1 --2- 172.21.15.92:0/831037915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560aabd0dc10 0x560aabc792d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x560aabc28370 tx=0x7f54a0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.489+0000 7f54a5112700 1 -- 172.21.15.92:0/831037915 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a0021070 con 0x560aabd0dc10 2023-10-06T10:40:18.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.489+0000 7f54a5112700 1 -- 172.21.15.92:0/831037915 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a0005870 con 0x560aabd0dc10 2023-10-06T10:40:18.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.489+0000 7f54ae6dc000 1 -- 172.21.15.92:0/831037915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560aabb86ae0 con 0x560aabd0dc10 2023-10-06T10:40:18.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.489+0000 7f54a5112700 1 -- 172.21.15.92:0/831037915 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a001cd00 con 0x560aabd0dc10 2023-10-06T10:40:18.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.489+0000 7f54ae6dc000 1 -- 172.21.15.92:0/831037915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560aabb86540 con 0x560aabd0dc10 2023-10-06T10:40:18.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.490+0000 7f54a5112700 1 -- 172.21.15.92:0/831037915 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f54a0019440 con 0x560aabd0dc10 2023-10-06T10:40:18.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.490+0000 7f54a5112700 1 --2- 172.21.15.92:0/831037915 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5488065890 0x7f5488067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.490+0000 7f54a5112700 1 -- 172.21.15.92:0/831037915 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f54a00856a0 con 0x560aabd0dc10 2023-10-06T10:40:18.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.490+0000 7f54a7917700 1 --2- 172.21.15.92:0/831037915 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5488065890 0x7f5488067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.491+0000 7f54ae6dc000 1 --2- 172.21.15.92:0/831037915 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x560aabd0cb20 0x560aabd15ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.491+0000 7f54ae6dc000 1 -- 172.21.15.92:0/831037915 --> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x560aabd13330 con 0x560aabd0cb20 2023-10-06T10:40:18.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.491+0000 7f54a7917700 1 --2- 172.21.15.92:0/831037915 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5488065890 0x7f5488067d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5498001620 tx=0x7f5498000bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.491+0000 7f549ffff700 1 --2- 172.21.15.92:0/831037915 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x560aabd0cb20 0x560aabd15ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.491+0000 7f549ffff700 1 --2- 172.21.15.92:0/831037915 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x560aabd0cb20 0x560aabd15ee0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.492+0000 7f549ffff700 1 -- 172.21.15.92:0/831037915 <== osd.7 v2:172.21.15.124:6808/90884 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (10) out=10b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7f5490016070 con 0x560aabd0cb20 2023-10-06T10:40:18.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.492+0000 7f54ae6dc000 1 -- 172.21.15.92:0/831037915 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x560aabd0cb20 msgr2=0x560aabd15ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.492+0000 7f54ae6dc000 1 --2- 172.21.15.92:0/831037915 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x560aabd0cb20 0x560aabd15ee0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.492+0000 7f54ae6dc000 1 -- 172.21.15.92:0/831037915 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5488065890 msgr2=0x7f5488067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.492+0000 7f54ae6dc000 1 --2- 172.21.15.92:0/831037915 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5488065890 0x7f5488067d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5498001620 tx=0x7f5498000bc0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.492+0000 7f54ae6dc000 1 -- 172.21.15.92:0/831037915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560aabd0dc10 msgr2=0x560aabc792d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.492+0000 7f54ae6dc000 1 --2- 172.21.15.92:0/831037915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560aabd0dc10 0x560aabc792d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x560aabc28370 tx=0x7f54a0014000 comp rx=0 tx=0).stop 2023-10-06T10:40:18.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.492+0000 7f54a8118700 1 -- 172.21.15.92:0/831037915 reap_dead start 2023-10-06T10:40:18.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.492+0000 7f54ae6dc000 1 -- 172.21.15.92:0/831037915 shutdown_connections 2023-10-06T10:40:18.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.492+0000 7f54ae6dc000 1 -- 172.21.15.92:0/831037915 >> 172.21.15.92:0/831037915 conn(0x560aabc3a750 msgr2=0x560aabc65a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:18.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.492+0000 7f54ae6dc000 1 -- 172.21.15.92:0/831037915 shutdown_connections 2023-10-06T10:40:18.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.492+0000 7f54ae6dc000 1 -- 172.21.15.92:0/831037915 wait complete. 2023-10-06T10:40:18.500 INFO:teuthology.orchestra.run.smithi092.stdout:tag123 2023-10-06T10:40:18.500 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000003.00000000 scrub_tag 2023-10-06T10:40:18.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.576+0000 7f034eb5b000 1 Processor -- start 2023-10-06T10:40:18.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.576+0000 7f034eb5b000 1 -- start start 2023-10-06T10:40:18.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.577+0000 7f034eb5b000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5598c673c270 0x5598c67693d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.577+0000 7f034eb5b000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5598c6782600 0x5598c675afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.577+0000 7f034eb5b000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598c67829d0 0x5598c6811990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.577+0000 7f034eb5b000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c6684980 con 0x5598c67829d0 2023-10-06T10:40:18.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.577+0000 7f034eb5b000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c6684620 con 0x5598c6782600 2023-10-06T10:40:18.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.577+0000 7f034eb5b000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598c667b400 con 0x5598c673c270 2023-10-06T10:40:18.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.577+0000 7f0348597700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598c67829d0 0x5598c6811990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.577+0000 7f0348597700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598c67829d0 0x5598c6811990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58144/0 (socket says 172.21.15.92:58144) 2023-10-06T10:40:18.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.577+0000 7f0348597700 1 -- 172.21.15.92:0/3561987525 learned_addr learned my addr 172.21.15.92:0/3561987525 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:18.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.577+0000 7f0347d96700 1 --2- 172.21.15.92:0/3561987525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5598c673c270 0x5598c67693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.577+0000 7f0347595700 1 --2- 172.21.15.92:0/3561987525 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5598c6782600 0x5598c675afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.577+0000 7f0348597700 1 -- 172.21.15.92:0/3561987525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5598c673c270 msgr2=0x5598c67693d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:18.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.577+0000 7f0348597700 1 --2- 172.21.15.92:0/3561987525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5598c673c270 0x5598c67693d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.577+0000 7f0348597700 1 -- 172.21.15.92:0/3561987525 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5598c6782600 msgr2=0x5598c675afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:18.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.577+0000 7f0348597700 1 --2- 172.21.15.92:0/3561987525 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5598c6782600 0x5598c675afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.577+0000 7f0348597700 1 -- 172.21.15.92:0/3561987525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0338005db0 con 0x5598c67829d0 2023-10-06T10:40:18.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.577+0000 7f0348597700 1 --2- 172.21.15.92:0/3561987525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598c67829d0 0x5598c6811990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f034000a5a0 tx=0x7f0340007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab4c4ba98eda0a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.577+0000 7f0346593700 1 -- 172.21.15.92:0/3561987525 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034000aed0 con 0x5598c67829d0 2023-10-06T10:40:18.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.577+0000 7f0346593700 1 -- 172.21.15.92:0/3561987525 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03400115f0 con 0x5598c67829d0 2023-10-06T10:40:18.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.577+0000 7f0346593700 1 -- 172.21.15.92:0/3561987525 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0340011990 con 0x5598c67829d0 2023-10-06T10:40:18.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.578+0000 7f034eb5b000 1 -- 172.21.15.92:0/3561987525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598c67829d0 msgr2=0x5598c6811990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.578+0000 7f034eb5b000 1 --2- 172.21.15.92:0/3561987525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598c67829d0 0x5598c6811990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f034000a5a0 tx=0x7f0340007040 comp rx=0 tx=0).stop 2023-10-06T10:40:18.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.578+0000 7f034eb5b000 1 -- 172.21.15.92:0/3561987525 shutdown_connections 2023-10-06T10:40:18.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.578+0000 7f034eb5b000 1 --2- 172.21.15.92:0/3561987525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598c67829d0 0x5598c6811990 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.578+0000 7f034eb5b000 1 --2- 172.21.15.92:0/3561987525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5598c673c270 0x5598c67693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.578+0000 7f034eb5b000 1 --2- 172.21.15.92:0/3561987525 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5598c6782600 0x5598c675afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.578+0000 7f034eb5b000 1 -- 172.21.15.92:0/3561987525 >> 172.21.15.92:0/3561987525 conn(0x5598c673a750 msgr2=0x5598c677cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:18.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.578+0000 7f034eb5b000 1 -- 172.21.15.92:0/3561987525 shutdown_connections 2023-10-06T10:40:18.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.578+0000 7f034eb5b000 1 -- 172.21.15.92:0/3561987525 wait complete. 2023-10-06T10:40:18.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.578+0000 7f034eb5b000 1 Processor -- start 2023-10-06T10:40:18.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.578+0000 7f034eb5b000 1 -- start start 2023-10-06T10:40:18.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.579+0000 7f034eb5b000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598c673c270 0x5598c680d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.579+0000 7f034eb5b000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5598c6782600 0x5598c675d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.579+0000 7f034eb5b000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5598c67829d0 0x5598c677f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.579+0000 7f034eb5b000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c675e270 con 0x5598c673c270 2023-10-06T10:40:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.579+0000 7f034eb5b000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c675e510 con 0x5598c6782600 2023-10-06T10:40:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.579+0000 7f034eb5b000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598c675e7b0 con 0x5598c67829d0 2023-10-06T10:40:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.579+0000 7f0347d96700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598c673c270 0x5598c680d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.579+0000 7f0347d96700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598c673c270 0x5598c680d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58156/0 (socket says 172.21.15.92:58156) 2023-10-06T10:40:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.579+0000 7f0347d96700 1 -- 172.21.15.92:0/318330392 learned_addr learned my addr 172.21.15.92:0/318330392 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.579+0000 7f0347595700 1 --2- 172.21.15.92:0/318330392 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5598c6782600 0x5598c675d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.580+0000 7f0348597700 1 --2- 172.21.15.92:0/318330392 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5598c67829d0 0x5598c677f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.580+0000 7f0347d96700 1 -- 172.21.15.92:0/318330392 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5598c67829d0 msgr2=0x5598c677f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.580+0000 7f0347d96700 1 --2- 172.21.15.92:0/318330392 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5598c67829d0 0x5598c677f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.580+0000 7f0347d96700 1 -- 172.21.15.92:0/318330392 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5598c6782600 msgr2=0x5598c675d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.580+0000 7f0347d96700 1 --2- 172.21.15.92:0/318330392 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5598c6782600 0x5598c675d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.580+0000 7f0347d96700 1 -- 172.21.15.92:0/318330392 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034000d040 con 0x5598c673c270 2023-10-06T10:40:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.580+0000 7f0347d96700 1 --2- 172.21.15.92:0/318330392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598c673c270 0x5598c680d000 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0338012000 tx=0x7f0338016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.580+0000 7f0344d90700 1 -- 172.21.15.92:0/318330392 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0338012a80 con 0x5598c673c270 2023-10-06T10:40:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.580+0000 7f0344d90700 1 -- 172.21.15.92:0/318330392 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0338012be0 con 0x5598c673c270 2023-10-06T10:40:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.580+0000 7f034eb5b000 1 -- 172.21.15.92:0/318330392 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5598c66933d0 con 0x5598c673c270 2023-10-06T10:40:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.580+0000 7f0344d90700 1 -- 172.21.15.92:0/318330392 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033801d420 con 0x5598c673c270 2023-10-06T10:40:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.580+0000 7f034eb5b000 1 -- 172.21.15.92:0/318330392 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598c6686ae0 con 0x5598c673c270 2023-10-06T10:40:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.581+0000 7f0344d90700 1 -- 172.21.15.92:0/318330392 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f033801d810 con 0x5598c673c270 2023-10-06T10:40:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.582+0000 7f0344d90700 1 --2- 172.21.15.92:0/318330392 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0334065890 0x7f0334067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.582+0000 7f0344d90700 1 -- 172.21.15.92:0/318330392 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f0338085970 con 0x5598c673c270 2023-10-06T10:40:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.582+0000 7f0347595700 1 --2- 172.21.15.92:0/318330392 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0334065890 0x7f0334067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.582+0000 7f034eb5b000 1 --2- 172.21.15.92:0/318330392 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x5598c675ddd0 0x5598c6821290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.582+0000 7f034eb5b000 1 -- 172.21.15.92:0/318330392 --> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] -- osd_op(unknown.0.0:1 6.12 6:4def72d4:::10000000003.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x5598c68217d0 con 0x5598c675ddd0 2023-10-06T10:40:18.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.582+0000 7f0348597700 1 --2- 172.21.15.92:0/318330392 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x5598c675ddd0 0x5598c6821290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.582+0000 7f0347595700 1 --2- 172.21.15.92:0/318330392 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0334065890 0x7f0334067d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f033c0116f0 tx=0x7f033c0017a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.583+0000 7f0348597700 1 --2- 172.21.15.92:0/318330392 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x5598c675ddd0 0x5598c6821290 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.583+0000 7f0348597700 1 -- 172.21.15.92:0/318330392 <== osd.7 v2:172.21.15.124:6808/90884 1 ==== osd_op_reply(1 10000000003.00000000 [getxattr (10) out=10b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7f0340011760 con 0x5598c675ddd0 2023-10-06T10:40:18.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.583+0000 7f034eb5b000 1 -- 172.21.15.92:0/318330392 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x5598c675ddd0 msgr2=0x5598c6821290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.583+0000 7f034eb5b000 1 --2- 172.21.15.92:0/318330392 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x5598c675ddd0 0x5598c6821290 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.583+0000 7f034eb5b000 1 -- 172.21.15.92:0/318330392 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0334065890 msgr2=0x7f0334067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.583+0000 7f034eb5b000 1 --2- 172.21.15.92:0/318330392 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0334065890 0x7f0334067d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f033c0116f0 tx=0x7f033c0017a0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.583+0000 7f034eb5b000 1 -- 172.21.15.92:0/318330392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598c673c270 msgr2=0x5598c680d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.583+0000 7f034eb5b000 1 --2- 172.21.15.92:0/318330392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598c673c270 0x5598c680d000 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0338012000 tx=0x7f0338016000 comp rx=0 tx=0).stop 2023-10-06T10:40:18.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.583+0000 7f0348597700 1 -- 172.21.15.92:0/318330392 reap_dead start 2023-10-06T10:40:18.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.583+0000 7f034eb5b000 1 -- 172.21.15.92:0/318330392 shutdown_connections 2023-10-06T10:40:18.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.583+0000 7f034eb5b000 1 -- 172.21.15.92:0/318330392 >> 172.21.15.92:0/318330392 conn(0x5598c673a750 msgr2=0x5598c680ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:18.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.583+0000 7f034eb5b000 1 -- 172.21.15.92:0/318330392 shutdown_connections 2023-10-06T10:40:18.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.583+0000 7f034eb5b000 1 -- 172.21.15.92:0/318330392 wait complete. 2023-10-06T10:40:18.591 INFO:teuthology.orchestra.run.smithi092.stdout:tag123 2023-10-06T10:40:18.591 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000004.00000000 scrub_tag 2023-10-06T10:40:18.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.666+0000 7f4b293cd000 1 Processor -- start 2023-10-06T10:40:18.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.667+0000 7f4b293cd000 1 -- start start 2023-10-06T10:40:18.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.667+0000 7f4b293cd000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a2cb824270 0x55a2cb8f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.667+0000 7f4b293cd000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a2cb8f4c80 0x55a2cb8f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.667+0000 7f4b293cd000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2cb8f5050 0x55a2cb8ec9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.667+0000 7f4b293cd000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2cb7c8320 con 0x55a2cb8f5050 2023-10-06T10:40:18.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.667+0000 7f4b293cd000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2cb76c980 con 0x55a2cb824270 2023-10-06T10:40:18.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.667+0000 7f4b293cd000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a2cb76c620 con 0x55a2cb8f4c80 2023-10-06T10:40:18.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.667+0000 7f4b22e09700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2cb8f5050 0x55a2cb8ec9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.667+0000 7f4b22e09700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2cb8f5050 0x55a2cb8ec9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58158/0 (socket says 172.21.15.92:58158) 2023-10-06T10:40:18.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.667+0000 7f4b22e09700 1 -- 172.21.15.92:0/4208327598 learned_addr learned my addr 172.21.15.92:0/4208327598 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:18.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.667+0000 7f4b22608700 1 --2- 172.21.15.92:0/4208327598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a2cb824270 0x55a2cb8f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.667+0000 7f4b21e07700 1 --2- 172.21.15.92:0/4208327598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a2cb8f4c80 0x55a2cb8f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.668+0000 7f4b22e09700 1 -- 172.21.15.92:0/4208327598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a2cb8f4c80 msgr2=0x55a2cb8f78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:18.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.668+0000 7f4b22e09700 1 --2- 172.21.15.92:0/4208327598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a2cb8f4c80 0x55a2cb8f78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.668+0000 7f4b22e09700 1 -- 172.21.15.92:0/4208327598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a2cb824270 msgr2=0x55a2cb8f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:18.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.668+0000 7f4b22e09700 1 --2- 172.21.15.92:0/4208327598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a2cb824270 0x55a2cb8f73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.668+0000 7f4b22e09700 1 -- 172.21.15.92:0/4208327598 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1400a0e0 con 0x55a2cb8f5050 2023-10-06T10:40:18.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.668+0000 7f4b21e07700 1 --2- 172.21.15.92:0/4208327598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a2cb8f4c80 0x55a2cb8f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:18.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.668+0000 7f4b22e09700 1 --2- 172.21.15.92:0/4208327598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2cb8f5050 0x55a2cb8ec9d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c001cf0 tx=0x7f4b1c001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a5f345cb75e1a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.668+0000 7f4b20e05700 1 -- 172.21.15.92:0/4208327598 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c00bc00 con 0x55a2cb8f5050 2023-10-06T10:40:18.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.668+0000 7f4b20e05700 1 -- 172.21.15.92:0/4208327598 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1c00bd80 con 0x55a2cb8f5050 2023-10-06T10:40:18.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.668+0000 7f4b20e05700 1 -- 172.21.15.92:0/4208327598 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c00e6c0 con 0x55a2cb8f5050 2023-10-06T10:40:18.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.668+0000 7f4b293cd000 1 -- 172.21.15.92:0/4208327598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2cb8f5050 msgr2=0x55a2cb8ec9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.668+0000 7f4b293cd000 1 --2- 172.21.15.92:0/4208327598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2cb8f5050 0x55a2cb8ec9d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c001cf0 tx=0x7f4b1c001770 comp rx=0 tx=0).stop 2023-10-06T10:40:18.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.668+0000 7f4b293cd000 1 -- 172.21.15.92:0/4208327598 shutdown_connections 2023-10-06T10:40:18.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.668+0000 7f4b293cd000 1 --2- 172.21.15.92:0/4208327598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2cb8f5050 0x55a2cb8ec9d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.668+0000 7f4b293cd000 1 --2- 172.21.15.92:0/4208327598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a2cb824270 0x55a2cb8f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.668+0000 7f4b293cd000 1 --2- 172.21.15.92:0/4208327598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a2cb8f4c80 0x55a2cb8f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.668+0000 7f4b293cd000 1 -- 172.21.15.92:0/4208327598 >> 172.21.15.92:0/4208327598 conn(0x55a2cb822750 msgr2=0x55a2cb8507a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.668+0000 7f4b293cd000 1 -- 172.21.15.92:0/4208327598 shutdown_connections 2023-10-06T10:40:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.668+0000 7f4b293cd000 1 -- 172.21.15.92:0/4208327598 wait complete. 2023-10-06T10:40:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.669+0000 7f4b293cd000 1 Processor -- start 2023-10-06T10:40:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.669+0000 7f4b293cd000 1 -- start start 2023-10-06T10:40:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.670+0000 7f4b293cd000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a2cb824270 0x55a2cb8f11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.670+0000 7f4b293cd000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a2cb8f4c80 0x55a2cb8eb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.670+0000 7f4b293cd000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2cb8f5050 0x55a2cb8eb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.670+0000 7f4b293cd000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2cb763400 con 0x55a2cb8f5050 2023-10-06T10:40:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.670+0000 7f4b293cd000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2cb84ca90 con 0x55a2cb8f4c80 2023-10-06T10:40:18.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.670+0000 7f4b293cd000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a2cb8f19a0 con 0x55a2cb824270 2023-10-06T10:40:18.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.670+0000 7f4b22e09700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2cb8f5050 0x55a2cb8eb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.670+0000 7f4b22e09700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2cb8f5050 0x55a2cb8eb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58160/0 (socket says 172.21.15.92:58160) 2023-10-06T10:40:18.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.670+0000 7f4b22e09700 1 -- 172.21.15.92:0/3267989953 learned_addr learned my addr 172.21.15.92:0/3267989953 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:18.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.670+0000 7f4b21e07700 1 --2- 172.21.15.92:0/3267989953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a2cb8f4c80 0x55a2cb8eb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.670+0000 7f4b22608700 1 --2- 172.21.15.92:0/3267989953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a2cb824270 0x55a2cb8f11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.670+0000 7f4b22e09700 1 -- 172.21.15.92:0/3267989953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a2cb824270 msgr2=0x55a2cb8f11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.670+0000 7f4b22e09700 1 --2- 172.21.15.92:0/3267989953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a2cb824270 0x55a2cb8f11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.670+0000 7f4b22e09700 1 -- 172.21.15.92:0/3267989953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a2cb8f4c80 msgr2=0x55a2cb8eb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.670+0000 7f4b22e09700 1 --2- 172.21.15.92:0/3267989953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a2cb8f4c80 0x55a2cb8eb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.670+0000 7f4b22e09700 1 -- 172.21.15.92:0/3267989953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b10001e10 con 0x55a2cb8f5050 2023-10-06T10:40:18.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.670+0000 7f4b22e09700 1 --2- 172.21.15.92:0/3267989953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2cb8f5050 0x55a2cb8eb830 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c001120 tx=0x7f4b1c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.670+0000 7f4b1affd700 1 -- 172.21.15.92:0/3267989953 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c01f070 con 0x55a2cb8f5050 2023-10-06T10:40:18.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.670+0000 7f4b1affd700 1 -- 172.21.15.92:0/3267989953 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1c00bc00 con 0x55a2cb8f5050 2023-10-06T10:40:18.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.670+0000 7f4b293cd000 1 -- 172.21.15.92:0/3267989953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a2cb76eae0 con 0x55a2cb8f5050 2023-10-06T10:40:18.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.670+0000 7f4b1affd700 1 -- 172.21.15.92:0/3267989953 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c01ab70 con 0x55a2cb8f5050 2023-10-06T10:40:18.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.670+0000 7f4b293cd000 1 -- 172.21.15.92:0/3267989953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2cb76e540 con 0x55a2cb8f5050 2023-10-06T10:40:18.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.672+0000 7f4b1affd700 1 -- 172.21.15.92:0/3267989953 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4b1c018430 con 0x55a2cb8f5050 2023-10-06T10:40:18.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.672+0000 7f4b1affd700 1 --2- 172.21.15.92:0/3267989953 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4b040658c0 0x7f4b04067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.672+0000 7f4b1affd700 1 -- 172.21.15.92:0/3267989953 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f4b1c0850c0 con 0x55a2cb8f5050 2023-10-06T10:40:18.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.672+0000 7f4b22608700 1 --2- 172.21.15.92:0/3267989953 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4b040658c0 0x7f4b04067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.672+0000 7f4b293cd000 1 --2- 172.21.15.92:0/3267989953 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x55a2cb8f3010 0x55a2cb8f3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.672+0000 7f4b293cd000 1 -- 172.21.15.92:0/3267989953 --> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] -- osd_op(unknown.0.0:1 6.5 6:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x55a2cb8f3950 con 0x55a2cb8f3010 2023-10-06T10:40:18.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.672+0000 7f4b22608700 1 --2- 172.21.15.92:0/3267989953 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4b040658c0 0x7f4b04067d70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400a5a0 tx=0x7f4b14015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.672+0000 7f4b21e07700 1 --2- 172.21.15.92:0/3267989953 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x55a2cb8f3010 0x55a2cb8f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.673+0000 7f4b21e07700 1 --2- 172.21.15.92:0/3267989953 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x55a2cb8f3010 0x55a2cb8f3410 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.673+0000 7f4b21e07700 1 -- 172.21.15.92:0/3267989953 <== osd.7 v2:172.21.15.124:6808/90884 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (10) out=10b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7f4b0c00f730 con 0x55a2cb8f3010 2023-10-06T10:40:18.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.673+0000 7f4b293cd000 1 -- 172.21.15.92:0/3267989953 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x55a2cb8f3010 msgr2=0x55a2cb8f3410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.673+0000 7f4b293cd000 1 --2- 172.21.15.92:0/3267989953 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x55a2cb8f3010 0x55a2cb8f3410 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.673+0000 7f4b293cd000 1 -- 172.21.15.92:0/3267989953 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4b040658c0 msgr2=0x7f4b04067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.673+0000 7f4b293cd000 1 --2- 172.21.15.92:0/3267989953 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4b040658c0 0x7f4b04067d70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400a5a0 tx=0x7f4b14015000 comp rx=0 tx=0).stop 2023-10-06T10:40:18.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.673+0000 7f4b293cd000 1 -- 172.21.15.92:0/3267989953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2cb8f5050 msgr2=0x55a2cb8eb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.673+0000 7f4b293cd000 1 --2- 172.21.15.92:0/3267989953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2cb8f5050 0x55a2cb8eb830 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c001120 tx=0x7f4b1c014000 comp rx=0 tx=0).stop 2023-10-06T10:40:18.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.673+0000 7f4b22e09700 1 -- 172.21.15.92:0/3267989953 reap_dead start 2023-10-06T10:40:18.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.673+0000 7f4b293cd000 1 -- 172.21.15.92:0/3267989953 shutdown_connections 2023-10-06T10:40:18.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.673+0000 7f4b293cd000 1 -- 172.21.15.92:0/3267989953 >> 172.21.15.92:0/3267989953 conn(0x55a2cb822750 msgr2=0x55a2cb8f0270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:18.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.673+0000 7f4b293cd000 1 -- 172.21.15.92:0/3267989953 shutdown_connections 2023-10-06T10:40:18.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.673+0000 7f4b293cd000 1 -- 172.21.15.92:0/3267989953 wait complete. 2023-10-06T10:40:18.683 INFO:teuthology.orchestra.run.smithi092.stdout:tag123 2023-10-06T10:40:18.683 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000005.00000000 scrub_tag 2023-10-06T10:40:18.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.759+0000 7f9e0e85d000 1 Processor -- start 2023-10-06T10:40:18.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.759+0000 7f9e0e85d000 1 -- start start 2023-10-06T10:40:18.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.760+0000 7f9e0e85d000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c9724709c0 0x55c972559f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.760+0000 7f9e0e85d000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c972470d90 0x55c972484a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.760+0000 7f9e0e85d000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c972556c10 0x55c9724c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.760+0000 7f9e0e85d000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9723cd980 con 0x55c972556c10 2023-10-06T10:40:18.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.760+0000 7f9e0e85d000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9723cd620 con 0x55c972470d90 2023-10-06T10:40:18.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.760+0000 7f9e0e85d000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c9723c4400 con 0x55c9724709c0 2023-10-06T10:40:18.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.760+0000 7f9e08299700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c972556c10 0x55c9724c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.760+0000 7f9e08299700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c972556c10 0x55c9724c89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58166/0 (socket says 172.21.15.92:58166) 2023-10-06T10:40:18.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.760+0000 7f9e08299700 1 -- 172.21.15.92:0/3973698320 learned_addr learned my addr 172.21.15.92:0/3973698320 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:18.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.760+0000 7f9e07297700 1 --2- 172.21.15.92:0/3973698320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c972470d90 0x55c972484a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.760+0000 7f9e08299700 1 -- 172.21.15.92:0/3973698320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c9724709c0 msgr2=0x55c972559f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:18.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.760+0000 7f9e08299700 1 --2- 172.21.15.92:0/3973698320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c9724709c0 0x55c972559f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.760+0000 7f9e08299700 1 -- 172.21.15.92:0/3973698320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c972470d90 msgr2=0x55c972484a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:18.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.760+0000 7f9e08299700 1 --2- 172.21.15.92:0/3973698320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c972470d90 0x55c972484a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.760+0000 7f9e08299700 1 -- 172.21.15.92:0/3973698320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0000a0e0 con 0x55c972556c10 2023-10-06T10:40:18.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.760+0000 7f9e07297700 1 --2- 172.21.15.92:0/3973698320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c972470d90 0x55c972484a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:18.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.760+0000 7f9e08299700 1 --2- 172.21.15.92:0/3973698320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c972556c10 0x55c9724c89f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x55c972471370 tx=0x7f9e00015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6516c1afddb27b2a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.760+0000 7f9e06295700 1 -- 172.21.15.92:0/3973698320 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0000f860 con 0x55c972556c10 2023-10-06T10:40:18.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.760+0000 7f9e06295700 1 -- 172.21.15.92:0/3973698320 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0000fa00 con 0x55c972556c10 2023-10-06T10:40:18.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.760+0000 7f9e06295700 1 -- 172.21.15.92:0/3973698320 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e000246c0 con 0x55c972556c10 2023-10-06T10:40:18.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.761+0000 7f9e0e85d000 1 -- 172.21.15.92:0/3973698320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c972556c10 msgr2=0x55c9724c89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.761+0000 7f9e0e85d000 1 --2- 172.21.15.92:0/3973698320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c972556c10 0x55c9724c89f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x55c972471370 tx=0x7f9e00015000 comp rx=0 tx=0).stop 2023-10-06T10:40:18.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.761+0000 7f9e0e85d000 1 -- 172.21.15.92:0/3973698320 shutdown_connections 2023-10-06T10:40:18.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.761+0000 7f9e0e85d000 1 --2- 172.21.15.92:0/3973698320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c972556c10 0x55c9724c89f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.761+0000 7f9e0e85d000 1 --2- 172.21.15.92:0/3973698320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c9724709c0 0x55c972559f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.761+0000 7f9e0e85d000 1 --2- 172.21.15.92:0/3973698320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c972470d90 0x55c972484a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.761+0000 7f9e0e85d000 1 -- 172.21.15.92:0/3973698320 >> 172.21.15.92:0/3973698320 conn(0x55c972483750 msgr2=0x55c972558750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:18.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.761+0000 7f9e0e85d000 1 -- 172.21.15.92:0/3973698320 shutdown_connections 2023-10-06T10:40:18.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.761+0000 7f9e0e85d000 1 -- 172.21.15.92:0/3973698320 wait complete. 2023-10-06T10:40:18.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.762+0000 7f9e0e85d000 1 Processor -- start 2023-10-06T10:40:18.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.762+0000 7f9e0e85d000 1 -- start start 2023-10-06T10:40:18.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.762+0000 7f9e0e85d000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c9724709c0 0x55c972544aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.762+0000 7f9e0e85d000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c972470d90 0x55c9725519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.762+0000 7f9e0e85d000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c972556c10 0x55c9724acd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.762+0000 7f9e0e85d000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c972544fe0 con 0x55c972556c10 2023-10-06T10:40:18.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.762+0000 7f9e0e85d000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c972552450 con 0x55c9724709c0 2023-10-06T10:40:18.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.762+0000 7f9e0e85d000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c9725526f0 con 0x55c972470d90 2023-10-06T10:40:18.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.762+0000 7f9e08299700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c972556c10 0x55c9724acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.762+0000 7f9e08299700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c972556c10 0x55c9724acd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58174/0 (socket says 172.21.15.92:58174) 2023-10-06T10:40:18.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.762+0000 7f9e08299700 1 -- 172.21.15.92:0/2839399403 learned_addr learned my addr 172.21.15.92:0/2839399403 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:18.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.762+0000 7f9e07a98700 1 --2- 172.21.15.92:0/2839399403 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c9724709c0 0x55c972544aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.762+0000 7f9e07297700 1 --2- 172.21.15.92:0/2839399403 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c972470d90 0x55c9725519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.763+0000 7f9e08299700 1 -- 172.21.15.92:0/2839399403 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c972470d90 msgr2=0x55c9725519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.763+0000 7f9e08299700 1 --2- 172.21.15.92:0/2839399403 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c972470d90 0x55c9725519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.763+0000 7f9e08299700 1 -- 172.21.15.92:0/2839399403 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c9724709c0 msgr2=0x55c972544aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.763+0000 7f9e08299700 1 --2- 172.21.15.92:0/2839399403 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c9724709c0 0x55c972544aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.763+0000 7f9e08299700 1 -- 172.21.15.92:0/2839399403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9723f61c0 con 0x55c972556c10 2023-10-06T10:40:18.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.763+0000 7f9e08299700 1 --2- 172.21.15.92:0/2839399403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c972556c10 0x55c9724acd20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9e00005d80 tx=0x7f9e0001d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.763+0000 7f9e04a92700 1 -- 172.21.15.92:0/2839399403 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0002f420 con 0x55c972556c10 2023-10-06T10:40:18.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.763+0000 7f9e04a92700 1 -- 172.21.15.92:0/2839399403 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0002f580 con 0x55c972556c10 2023-10-06T10:40:18.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.763+0000 7f9e0e85d000 1 -- 172.21.15.92:0/2839399403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c9723dc3d0 con 0x55c972556c10 2023-10-06T10:40:18.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.763+0000 7f9e04a92700 1 -- 172.21.15.92:0/2839399403 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0002f940 con 0x55c972556c10 2023-10-06T10:40:18.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.763+0000 7f9e0e85d000 1 -- 172.21.15.92:0/2839399403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9723cfae0 con 0x55c972556c10 2023-10-06T10:40:18.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.764+0000 7f9e04a92700 1 -- 172.21.15.92:0/2839399403 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9e00021070 con 0x55c972556c10 2023-10-06T10:40:18.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.765+0000 7f9e04a92700 1 --2- 172.21.15.92:0/2839399403 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9df4065890 0x7f9df4067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.765+0000 7f9e04a92700 1 -- 172.21.15.92:0/2839399403 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f9e0002b070 con 0x55c972556c10 2023-10-06T10:40:18.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.765+0000 7f9e07a98700 1 --2- 172.21.15.92:0/2839399403 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9df4065890 0x7f9df4067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.765+0000 7f9e0e85d000 1 --2- 172.21.15.92:0/2839399403 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x55c972551f30 0x55c972564970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.765+0000 7f9e0e85d000 1 -- 172.21.15.92:0/2839399403 --> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] -- osd_op(unknown.0.0:1 6.9 6:9258b67d:::10000000005.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x55c972564eb0 con 0x55c972551f30 2023-10-06T10:40:18.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.765+0000 7f9e07297700 1 --2- 172.21.15.92:0/2839399403 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x55c972551f30 0x55c972564970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.765+0000 7f9e07a98700 1 --2- 172.21.15.92:0/2839399403 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9df4065890 0x7f9df4067d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9df80057a0 tx=0x7f9df8005a30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.765+0000 7f9e07297700 1 --2- 172.21.15.92:0/2839399403 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x55c972551f30 0x55c972564970 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.766+0000 7f9e07297700 1 -- 172.21.15.92:0/2839399403 <== osd.0 v2:172.21.15.92:6800/90604 1 ==== osd_op_reply(1 10000000005.00000000 [getxattr (10) out=10b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7f9dfc00f7e0 con 0x55c972551f30 2023-10-06T10:40:18.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.766+0000 7f9e0e85d000 1 -- 172.21.15.92:0/2839399403 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x55c972551f30 msgr2=0x55c972564970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.766+0000 7f9e0e85d000 1 --2- 172.21.15.92:0/2839399403 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x55c972551f30 0x55c972564970 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.766+0000 7f9e0e85d000 1 -- 172.21.15.92:0/2839399403 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9df4065890 msgr2=0x7f9df4067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.766+0000 7f9e0e85d000 1 --2- 172.21.15.92:0/2839399403 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9df4065890 0x7f9df4067d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9df80057a0 tx=0x7f9df8005a30 comp rx=0 tx=0).stop 2023-10-06T10:40:18.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.766+0000 7f9e0e85d000 1 -- 172.21.15.92:0/2839399403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c972556c10 msgr2=0x55c9724acd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.766+0000 7f9e0e85d000 1 --2- 172.21.15.92:0/2839399403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c972556c10 0x55c9724acd20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9e00005d80 tx=0x7f9e0001d070 comp rx=0 tx=0).stop 2023-10-06T10:40:18.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.766+0000 7f9e08299700 1 -- 172.21.15.92:0/2839399403 reap_dead start 2023-10-06T10:40:18.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.766+0000 7f9e0e85d000 1 -- 172.21.15.92:0/2839399403 shutdown_connections 2023-10-06T10:40:18.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.766+0000 7f9e0e85d000 1 -- 172.21.15.92:0/2839399403 >> 172.21.15.92:0/2839399403 conn(0x55c972483750 msgr2=0x55c972558580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:18.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.766+0000 7f9e0e85d000 1 -- 172.21.15.92:0/2839399403 shutdown_connections 2023-10-06T10:40:18.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.766+0000 7f9e0e85d000 1 -- 172.21.15.92:0/2839399403 wait complete. 2023-10-06T10:40:18.774 INFO:teuthology.orchestra.run.smithi092.stdout:tag123 2023-10-06T10:40:18.774 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000006.00000000 scrub_tag 2023-10-06T10:40:18.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.850+0000 7f8fbd846000 1 Processor -- start 2023-10-06T10:40:18.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.850+0000 7f8fbd846000 1 -- start start 2023-10-06T10:40:18.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.851+0000 7f8fbd846000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3c3298270 0x55a3c3369780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.851+0000 7f8fbd846000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a3c3368c80 0x55a3c3367430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.851+0000 7f8fbd846000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a3c3369050 0x55a3c3296b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.851+0000 7f8fbd846000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c31e0980 con 0x55a3c3298270 2023-10-06T10:40:18.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.851+0000 7f8fbd846000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c31e0620 con 0x55a3c3369050 2023-10-06T10:40:18.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.851+0000 7f8fbd846000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3c31d7400 con 0x55a3c3368c80 2023-10-06T10:40:18.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.851+0000 7f8fb6a81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3c3298270 0x55a3c3369780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.851+0000 7f8fb6a81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3c3298270 0x55a3c3369780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58182/0 (socket says 172.21.15.92:58182) 2023-10-06T10:40:18.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.851+0000 7f8fb6a81700 1 -- 172.21.15.92:0/2676175061 learned_addr learned my addr 172.21.15.92:0/2676175061 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:18.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.851+0000 7f8fb6280700 1 --2- 172.21.15.92:0/2676175061 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a3c3368c80 0x55a3c3367430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.851+0000 7f8fb7282700 1 --2- 172.21.15.92:0/2676175061 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a3c3369050 0x55a3c3296b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.851+0000 7f8fb6a81700 1 -- 172.21.15.92:0/2676175061 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a3c3368c80 msgr2=0x55a3c3367430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:18.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.851+0000 7f8fb6a81700 1 --2- 172.21.15.92:0/2676175061 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a3c3368c80 0x55a3c3367430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.851+0000 7f8fb6a81700 1 -- 172.21.15.92:0/2676175061 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a3c3369050 msgr2=0x55a3c3296b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:18.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.851+0000 7f8fb6a81700 1 --2- 172.21.15.92:0/2676175061 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a3c3369050 0x55a3c3296b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.851+0000 7f8fb6a81700 1 -- 172.21.15.92:0/2676175061 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa800a0e0 con 0x55a3c3298270 2023-10-06T10:40:18.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.851+0000 7f8fb6a81700 1 --2- 172.21.15.92:0/2676175061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3c3298270 0x55a3c3369780 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x55a3c3284370 tx=0x7f8fa8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61050e2fa1ba13fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.852+0000 7f8fb527e700 1 -- 172.21.15.92:0/2676175061 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8000f40 con 0x55a3c3298270 2023-10-06T10:40:18.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.852+0000 7f8fb527e700 1 -- 172.21.15.92:0/2676175061 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa800f860 con 0x55a3c3298270 2023-10-06T10:40:18.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.852+0000 7f8fb527e700 1 -- 172.21.15.92:0/2676175061 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8024470 con 0x55a3c3298270 2023-10-06T10:40:18.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.852+0000 7f8fbd846000 1 -- 172.21.15.92:0/2676175061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3c3298270 msgr2=0x55a3c3369780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.852+0000 7f8fbd846000 1 --2- 172.21.15.92:0/2676175061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3c3298270 0x55a3c3369780 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x55a3c3284370 tx=0x7f8fa8015000 comp rx=0 tx=0).stop 2023-10-06T10:40:18.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.852+0000 7f8fbd846000 1 -- 172.21.15.92:0/2676175061 shutdown_connections 2023-10-06T10:40:18.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.852+0000 7f8fbd846000 1 --2- 172.21.15.92:0/2676175061 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a3c3369050 0x55a3c3296b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.852+0000 7f8fbd846000 1 --2- 172.21.15.92:0/2676175061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3c3298270 0x55a3c3369780 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.852+0000 7f8fbd846000 1 --2- 172.21.15.92:0/2676175061 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a3c3368c80 0x55a3c3367430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.852+0000 7f8fbd846000 1 -- 172.21.15.92:0/2676175061 >> 172.21.15.92:0/2676175061 conn(0x55a3c3296750 msgr2=0x55a3c336bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:18.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.852+0000 7f8fbd846000 1 -- 172.21.15.92:0/2676175061 shutdown_connections 2023-10-06T10:40:18.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.852+0000 7f8fbd846000 1 -- 172.21.15.92:0/2676175061 wait complete. 2023-10-06T10:40:18.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.853+0000 7f8fbd846000 1 Processor -- start 2023-10-06T10:40:18.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.853+0000 7f8fbd846000 1 -- start start 2023-10-06T10:40:18.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.853+0000 7f8fbd846000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3c3298270 0x55a3c32db290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.853+0000 7f8fbd846000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a3c3368c80 0x55a3c3366ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.853+0000 7f8fbd846000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a3c3369050 0x55a3c337b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.853+0000 7f8fbd846000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c32dbb20 con 0x55a3c3298270 2023-10-06T10:40:18.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.853+0000 7f8fbd846000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c32db880 con 0x55a3c3368c80 2023-10-06T10:40:18.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.853+0000 7f8fbd846000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3c337a7a0 con 0x55a3c3369050 2023-10-06T10:40:18.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.853+0000 7f8fb6a81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3c3298270 0x55a3c32db290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.854+0000 7f8fb6a81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3c3298270 0x55a3c32db290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58186/0 (socket says 172.21.15.92:58186) 2023-10-06T10:40:18.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.854+0000 7f8fb6a81700 1 -- 172.21.15.92:0/1944673491 learned_addr learned my addr 172.21.15.92:0/1944673491 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:18.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.854+0000 7f8fb6280700 1 --2- 172.21.15.92:0/1944673491 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a3c3368c80 0x55a3c3366ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.854+0000 7f8fb7282700 1 --2- 172.21.15.92:0/1944673491 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a3c3369050 0x55a3c337b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.854+0000 7f8fb6a81700 1 -- 172.21.15.92:0/1944673491 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a3c3369050 msgr2=0x55a3c337b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.854+0000 7f8fb6a81700 1 --2- 172.21.15.92:0/1944673491 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a3c3369050 0x55a3c337b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.854+0000 7f8fb6a81700 1 -- 172.21.15.92:0/1944673491 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a3c3368c80 msgr2=0x55a3c3366ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.854+0000 7f8fb6a81700 1 --2- 172.21.15.92:0/1944673491 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a3c3368c80 0x55a3c3366ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.854+0000 7f8fb6a81700 1 -- 172.21.15.92:0/1944673491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3c32091c0 con 0x55a3c3298270 2023-10-06T10:40:18.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.854+0000 7f8fb7282700 1 --2- 172.21.15.92:0/1944673491 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a3c3369050 0x55a3c337b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:18.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.854+0000 7f8fb6a81700 1 --2- 172.21.15.92:0/1944673491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3c3298270 0x55a3c32db290 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800a520 tx=0x7f8fa801d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.854+0000 7f8fa77fe700 1 -- 172.21.15.92:0/1944673491 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8024c00 con 0x55a3c3298270 2023-10-06T10:40:18.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.854+0000 7f8fa77fe700 1 -- 172.21.15.92:0/1944673491 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa8024d60 con 0x55a3c3298270 2023-10-06T10:40:18.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.854+0000 7f8fbd846000 1 -- 172.21.15.92:0/1944673491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a3c31ef3d0 con 0x55a3c3298270 2023-10-06T10:40:18.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.854+0000 7f8fa77fe700 1 -- 172.21.15.92:0/1944673491 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8010650 con 0x55a3c3298270 2023-10-06T10:40:18.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.854+0000 7f8fbd846000 1 -- 172.21.15.92:0/1944673491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3c31e2ae0 con 0x55a3c3298270 2023-10-06T10:40:18.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.855+0000 7f8fa77fe700 1 -- 172.21.15.92:0/1944673491 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8fa8021070 con 0x55a3c3298270 2023-10-06T10:40:18.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.856+0000 7f8fa77fe700 1 --2- 172.21.15.92:0/1944673491 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8f9c065890 0x7f8f9c067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.856+0000 7f8fa77fe700 1 -- 172.21.15.92:0/1944673491 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f8fa802b070 con 0x55a3c3298270 2023-10-06T10:40:18.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.856+0000 7f8fb6280700 1 --2- 172.21.15.92:0/1944673491 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8f9c065890 0x7f8f9c067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.856+0000 7f8fbd846000 1 --2- 172.21.15.92:0/1944673491 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x55a3c336bb60 0x55a3c32d8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.856+0000 7f8fbd846000 1 -- 172.21.15.92:0/1944673491 --> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] -- osd_op(unknown.0.0:1 6.5 6:a5feb6f3:::10000000006.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x55a3c32d94e0 con 0x55a3c336bb60 2023-10-06T10:40:18.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.856+0000 7f8fb6280700 1 --2- 172.21.15.92:0/1944673491 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8f9c065890 0x7f8f9c067d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8fac000c00 tx=0x7f8fac0069b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.856+0000 7f8fb7282700 1 --2- 172.21.15.92:0/1944673491 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x55a3c336bb60 0x55a3c32d8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.856+0000 7f8fb7282700 1 --2- 172.21.15.92:0/1944673491 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x55a3c336bb60 0x55a3c32d8fa0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.857+0000 7f8fb7282700 1 -- 172.21.15.92:0/1944673491 <== osd.7 v2:172.21.15.124:6808/90884 1 ==== osd_op_reply(1 10000000006.00000000 [getxattr (10) out=10b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7f8fb0007950 con 0x55a3c336bb60 2023-10-06T10:40:18.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.857+0000 7f8fbd846000 1 -- 172.21.15.92:0/1944673491 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x55a3c336bb60 msgr2=0x55a3c32d8fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.857+0000 7f8fbd846000 1 --2- 172.21.15.92:0/1944673491 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x55a3c336bb60 0x55a3c32d8fa0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.857+0000 7f8fbd846000 1 -- 172.21.15.92:0/1944673491 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8f9c065890 msgr2=0x7f8f9c067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.857+0000 7f8fbd846000 1 --2- 172.21.15.92:0/1944673491 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8f9c065890 0x7f8f9c067d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8fac000c00 tx=0x7f8fac0069b0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.857+0000 7f8fbd846000 1 -- 172.21.15.92:0/1944673491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3c3298270 msgr2=0x55a3c32db290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.857+0000 7f8fbd846000 1 --2- 172.21.15.92:0/1944673491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3c3298270 0x55a3c32db290 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800a520 tx=0x7f8fa801d070 comp rx=0 tx=0).stop 2023-10-06T10:40:18.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.857+0000 7f8fb7282700 1 -- 172.21.15.92:0/1944673491 reap_dead start 2023-10-06T10:40:18.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.857+0000 7f8fbd846000 1 -- 172.21.15.92:0/1944673491 shutdown_connections 2023-10-06T10:40:18.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.857+0000 7f8fbd846000 1 -- 172.21.15.92:0/1944673491 >> 172.21.15.92:0/1944673491 conn(0x55a3c3296750 msgr2=0x55a3c32bfe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:18.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.857+0000 7f8fbd846000 1 -- 172.21.15.92:0/1944673491 shutdown_connections 2023-10-06T10:40:18.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.857+0000 7f8fbd846000 1 -- 172.21.15.92:0/1944673491 wait complete. 2023-10-06T10:40:18.868 INFO:teuthology.orchestra.run.smithi092.stdout:tag123 2023-10-06T10:40:18.868 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000007.00000000 scrub_tag 2023-10-06T10:40:18.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.945+0000 7f34f6e59000 1 Processor -- start 2023-10-06T10:40:18.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.945+0000 7f34f6e59000 1 -- start start 2023-10-06T10:40:18.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.945+0000 7f34f6e59000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5651d7127670 0x5651d70dfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.945+0000 7f34f6e59000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5651d719b040 0x5651d711eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.945+0000 7f34f6e59000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651d719b410 0x5651d71b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.945+0000 7f34f6e59000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d7029620 con 0x5651d719b410 2023-10-06T10:40:18.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.945+0000 7f34f6e59000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d7020400 con 0x5651d7127670 2023-10-06T10:40:18.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.945+0000 7f34f6e59000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5651d7127ce0 con 0x5651d719b040 2023-10-06T10:40:18.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.946+0000 7f34f0895700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651d719b410 0x5651d71b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.946+0000 7f34f0895700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651d719b410 0x5651d71b33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58208/0 (socket says 172.21.15.92:58208) 2023-10-06T10:40:18.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.946+0000 7f34f0895700 1 -- 172.21.15.92:0/1164651666 learned_addr learned my addr 172.21.15.92:0/1164651666 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:18.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.946+0000 7f34f0094700 1 --2- 172.21.15.92:0/1164651666 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5651d7127670 0x5651d70dfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.946+0000 7f34ef893700 1 --2- 172.21.15.92:0/1164651666 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5651d719b040 0x5651d711eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.946+0000 7f34f0895700 1 -- 172.21.15.92:0/1164651666 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5651d719b040 msgr2=0x5651d711eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:18.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.946+0000 7f34f0895700 1 --2- 172.21.15.92:0/1164651666 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5651d719b040 0x5651d711eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.946+0000 7f34f0895700 1 -- 172.21.15.92:0/1164651666 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5651d7127670 msgr2=0x5651d70dfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:18.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.946+0000 7f34f0895700 1 --2- 172.21.15.92:0/1164651666 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5651d7127670 0x5651d70dfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.946+0000 7f34f0895700 1 -- 172.21.15.92:0/1164651666 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e000a180 con 0x5651d719b410 2023-10-06T10:40:18.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.946+0000 7f34f0895700 1 --2- 172.21.15.92:0/1164651666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651d719b410 0x5651d71b33e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f34e800a7f0 tx=0x7f34e8005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df15aa54126fef3f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.946+0000 7f34ee891700 1 -- 172.21.15.92:0/1164651666 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e800f070 con 0x5651d719b410 2023-10-06T10:40:18.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.946+0000 7f34ee891700 1 -- 172.21.15.92:0/1164651666 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e8008760 con 0x5651d719b410 2023-10-06T10:40:18.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.946+0000 7f34ee891700 1 -- 172.21.15.92:0/1164651666 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e8008b00 con 0x5651d719b410 2023-10-06T10:40:18.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.947+0000 7f34f6e59000 1 -- 172.21.15.92:0/1164651666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651d719b410 msgr2=0x5651d71b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.947+0000 7f34f6e59000 1 --2- 172.21.15.92:0/1164651666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651d719b410 0x5651d71b33e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f34e800a7f0 tx=0x7f34e8005c60 comp rx=0 tx=0).stop 2023-10-06T10:40:18.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.947+0000 7f34f6e59000 1 -- 172.21.15.92:0/1164651666 shutdown_connections 2023-10-06T10:40:18.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.947+0000 7f34f6e59000 1 --2- 172.21.15.92:0/1164651666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651d719b410 0x5651d71b33e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.947+0000 7f34f6e59000 1 --2- 172.21.15.92:0/1164651666 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5651d7127670 0x5651d70dfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.947+0000 7f34f6e59000 1 --2- 172.21.15.92:0/1164651666 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5651d719b040 0x5651d711eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.947+0000 7f34f6e59000 1 -- 172.21.15.92:0/1164651666 >> 172.21.15.92:0/1164651666 conn(0x5651d70df750 msgr2=0x5651d7125d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:18.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.947+0000 7f34f6e59000 1 -- 172.21.15.92:0/1164651666 shutdown_connections 2023-10-06T10:40:18.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.947+0000 7f34f6e59000 1 -- 172.21.15.92:0/1164651666 wait complete. 2023-10-06T10:40:18.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.947+0000 7f34f6e59000 1 Processor -- start 2023-10-06T10:40:18.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.947+0000 7f34f6e59000 1 -- start start 2023-10-06T10:40:18.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.948+0000 7f34f6e59000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5651d7127670 0x5651d7199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.948+0000 7f34f6e59000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651d719b040 0x5651d7104d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.948+0000 7f34f6e59000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5651d719b410 0x5651d71b3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.948+0000 7f34f6e59000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d7105670 con 0x5651d719b040 2023-10-06T10:40:18.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.948+0000 7f34f6e59000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d7105910 con 0x5651d719b410 2023-10-06T10:40:18.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.948+0000 7f34f6e59000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5651d7105bb0 con 0x5651d7127670 2023-10-06T10:40:18.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.948+0000 7f34ef893700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651d719b040 0x5651d7104d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.948+0000 7f34ef893700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651d719b040 0x5651d7104d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58224/0 (socket says 172.21.15.92:58224) 2023-10-06T10:40:18.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.948+0000 7f34ef893700 1 -- 172.21.15.92:0/3912074369 learned_addr learned my addr 172.21.15.92:0/3912074369 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:18.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.948+0000 7f34f0094700 1 --2- 172.21.15.92:0/3912074369 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5651d7127670 0x5651d7199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.948+0000 7f34f0895700 1 --2- 172.21.15.92:0/3912074369 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5651d719b410 0x5651d71b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.948+0000 7f34ef893700 1 -- 172.21.15.92:0/3912074369 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5651d7127670 msgr2=0x5651d7199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.948+0000 7f34ef893700 1 --2- 172.21.15.92:0/3912074369 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5651d7127670 0x5651d7199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.948+0000 7f34ef893700 1 -- 172.21.15.92:0/3912074369 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5651d719b410 msgr2=0x5651d71b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.948+0000 7f34ef893700 1 --2- 172.21.15.92:0/3912074369 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5651d719b410 0x5651d71b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.948+0000 7f34ef893700 1 -- 172.21.15.92:0/3912074369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e800a440 con 0x5651d719b040 2023-10-06T10:40:18.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.948+0000 7f34ef893700 1 --2- 172.21.15.92:0/3912074369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651d719b040 0x5651d7104d80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f34d8008920 tx=0x7f34d8005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.948+0000 7f34ed08e700 1 -- 172.21.15.92:0/3912074369 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d800f050 con 0x5651d719b040 2023-10-06T10:40:18.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.949+0000 7f34ed08e700 1 -- 172.21.15.92:0/3912074369 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d8010040 con 0x5651d719b040 2023-10-06T10:40:18.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.949+0000 7f34f6e59000 1 -- 172.21.15.92:0/3912074369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5651d70521c0 con 0x5651d719b040 2023-10-06T10:40:18.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.949+0000 7f34ed08e700 1 -- 172.21.15.92:0/3912074369 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d8006cb0 con 0x5651d719b040 2023-10-06T10:40:18.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.949+0000 7f34f6e59000 1 -- 172.21.15.92:0/3912074369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651d70383d0 con 0x5651d719b040 2023-10-06T10:40:18.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.950+0000 7f34ed08e700 1 -- 172.21.15.92:0/3912074369 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f34d8015670 con 0x5651d719b040 2023-10-06T10:40:18.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.950+0000 7f34ed08e700 1 --2- 172.21.15.92:0/3912074369 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f34d0065890 0x7f34d0067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.950+0000 7f34ed08e700 1 -- 172.21.15.92:0/3912074369 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f34d8083c20 con 0x5651d719b040 2023-10-06T10:40:18.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.950+0000 7f34f0094700 1 --2- 172.21.15.92:0/3912074369 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f34d0065890 0x7f34d0067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.950+0000 7f34f6e59000 1 --2- 172.21.15.92:0/3912074369 >> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] conn(0x5651d7122130 0x5651d7107e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:18.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.950+0000 7f34f6e59000 1 -- 172.21.15.92:0/3912074369 --> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] -- osd_op(unknown.0.0:1 6.1d 6:bbf5850a:::10000000007.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x5651d7108390 con 0x5651d7122130 2023-10-06T10:40:18.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.951+0000 7f34f0895700 1 --2- 172.21.15.92:0/3912074369 >> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] conn(0x5651d7122130 0x5651d7107e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:18.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.951+0000 7f34f0094700 1 --2- 172.21.15.92:0/3912074369 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f34d0065890 0x7f34d0067d40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f34e000a150 tx=0x7f34e0015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.951+0000 7f34f0895700 1 --2- 172.21.15.92:0/3912074369 >> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] conn(0x5651d7122130 0x5651d7107e50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:18.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.951+0000 7f34f0895700 1 -- 172.21.15.92:0/3912074369 <== osd.1 v2:172.21.15.92:6816/90606 1 ==== osd_op_reply(1 10000000007.00000000 [getxattr (10) out=10b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7f34e801e420 con 0x5651d7122130 2023-10-06T10:40:18.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.951+0000 7f34f6e59000 1 -- 172.21.15.92:0/3912074369 >> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] conn(0x5651d7122130 msgr2=0x5651d7107e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.951+0000 7f34f6e59000 1 --2- 172.21.15.92:0/3912074369 >> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] conn(0x5651d7122130 0x5651d7107e50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:18.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.951+0000 7f34f6e59000 1 -- 172.21.15.92:0/3912074369 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f34d0065890 msgr2=0x7f34d0067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.951+0000 7f34f6e59000 1 --2- 172.21.15.92:0/3912074369 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f34d0065890 0x7f34d0067d40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f34e000a150 tx=0x7f34e0015000 comp rx=0 tx=0).stop 2023-10-06T10:40:18.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.951+0000 7f34f6e59000 1 -- 172.21.15.92:0/3912074369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651d719b040 msgr2=0x5651d7104d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:18.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.951+0000 7f34f6e59000 1 --2- 172.21.15.92:0/3912074369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651d719b040 0x5651d7104d80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f34d8008920 tx=0x7f34d8005b40 comp rx=0 tx=0).stop 2023-10-06T10:40:18.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.951+0000 7f34f0895700 1 -- 172.21.15.92:0/3912074369 reap_dead start 2023-10-06T10:40:18.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.951+0000 7f34f6e59000 1 -- 172.21.15.92:0/3912074369 shutdown_connections 2023-10-06T10:40:18.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.951+0000 7f34f6e59000 1 -- 172.21.15.92:0/3912074369 >> 172.21.15.92:0/3912074369 conn(0x5651d70df750 msgr2=0x5651d7122e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:18.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.951+0000 7f34f6e59000 1 -- 172.21.15.92:0/3912074369 shutdown_connections 2023-10-06T10:40:18.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:18.951+0000 7f34f6e59000 1 -- 172.21.15.92:0/3912074369 wait complete. 2023-10-06T10:40:18.960 INFO:teuthology.orchestra.run.smithi092.stdout:tag123 2023-10-06T10:40:18.960 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T10:40:18.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:40:19.040 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:40:19.079 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T10:40:19.080 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T10:40:19.080 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:40:19.080 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T10:40:19.115 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T10:40:19.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:40:19.115 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T10:40:19.170 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:40:19.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.324+0000 7f43b3560700 1 Processor -- start 2023-10-06T10:40:19.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.324+0000 7f43b3560700 1 -- start start 2023-10-06T10:40:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.325+0000 7f43b3560700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f43ac0b2820 0x7f43ac06ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.325+0000 7f43b3560700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43ac06b470 con 0x7f43ac157360 2023-10-06T10:40:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.325+0000 7f43b3560700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43ac0c97c0 con 0x7f43ac0af670 2023-10-06T10:40:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.325+0000 7f43b3560700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43ac06ab70 con 0x7f43ac0b2820 2023-10-06T10:40:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.325+0000 7f43b1d5d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f43ac0b2820 0x7f43ac06ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.325+0000 7f43b255e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f43ac157360 0x7f43ac0b2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54140/0 (socket says 172.21.15.92:54140) 2023-10-06T10:40:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.325+0000 7f43b255e700 1 -- 172.21.15.92:0/2305937226 learned_addr learned my addr 172.21.15.92:0/2305937226 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.325+0000 7f43b1d5d700 1 -- 172.21.15.92:0/2305937226 >> v1:172.21.15.124:6789/0 conn(0x7f43ac0af670 legacy=0x7f43ac157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.325+0000 7f43b1d5d700 1 -- 172.21.15.92:0/2305937226 >> v1:172.21.15.92:6789/0 conn(0x7f43ac157360 legacy=0x7f43ac0b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.325+0000 7f43b1d5d700 1 -- 172.21.15.92:0/2305937226 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ac06a5d0 con 0x7f43ac0b2820 2023-10-06T10:40:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.325+0000 7f43b155c700 1 -- 172.21.15.92:0/2305937226 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069013255 0 0) 0x7f43ac06b470 con 0x7f43ac157360 2023-10-06T10:40:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.326+0000 7f43b1d5d700 1 --2- 172.21.15.92:0/2305937226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43ac0b2820 0x7f43ac06ee90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f439c0098e0 tx=0x7f439c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3785695b51ec9e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.326+0000 7f43b155c700 1 -- 172.21.15.92:0/2305937226 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c009e80 con 0x7f43ac0b2820 2023-10-06T10:40:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.326+0000 7f43b155c700 1 -- 172.21.15.92:0/2305937226 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439c00b860 con 0x7f43ac0b2820 2023-10-06T10:40:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.326+0000 7f43b155c700 1 -- 172.21.15.92:0/2305937226 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c00bbe0 con 0x7f43ac0b2820 2023-10-06T10:40:19.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.326+0000 7f43b3560700 1 -- 172.21.15.92:0/2305937226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43ac0b2820 msgr2=0x7f43ac06ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:19.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.326+0000 7f43b3560700 1 --2- 172.21.15.92:0/2305937226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43ac0b2820 0x7f43ac06ee90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f439c0098e0 tx=0x7f439c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:19.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.326+0000 7f43b3560700 1 -- 172.21.15.92:0/2305937226 shutdown_connections 2023-10-06T10:40:19.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.326+0000 7f43b3560700 1 --2- 172.21.15.92:0/2305937226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43ac0b2820 0x7f43ac06ee90 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:19.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.326+0000 7f43b3560700 1 -- 172.21.15.92:0/2305937226 >> 172.21.15.92:0/2305937226 conn(0x7f43ac069190 msgr2=0x7f43ac0aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:19.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.326+0000 7f43b3560700 1 -- 172.21.15.92:0/2305937226 shutdown_connections 2023-10-06T10:40:19.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.326+0000 7f43b3560700 1 -- 172.21.15.92:0/2305937226 wait complete. 2023-10-06T10:40:19.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.327+0000 7f43b3560700 1 Processor -- start 2023-10-06T10:40:19.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.327+0000 7f43b3560700 1 -- start start 2023-10-06T10:40:19.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.328+0000 7f43b3560700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43ac0af670 0x7f43ac156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:19.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.328+0000 7f43b3560700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43ac0b2820 0x7f43ac1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:19.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.328+0000 7f43b3560700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43ac157360 0x7f43ac0a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:19.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.328+0000 7f43b3560700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac1506f0 con 0x7f43ac0af670 2023-10-06T10:40:19.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.328+0000 7f43b3560700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac0aacc0 con 0x7f43ac157360 2023-10-06T10:40:19.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.328+0000 7f43b3560700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac0a9de0 con 0x7f43ac0b2820 2023-10-06T10:40:19.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.328+0000 7f43b255e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43ac0af670 0x7f43ac156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:19.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.328+0000 7f43b255e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43ac0af670 0x7f43ac156490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58248/0 (socket says 172.21.15.92:58248) 2023-10-06T10:40:19.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.328+0000 7f43b255e700 1 -- 172.21.15.92:0/3170472781 learned_addr learned my addr 172.21.15.92:0/3170472781 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:19.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.328+0000 7f43b2d5f700 1 --2- 172.21.15.92:0/3170472781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43ac157360 0x7f43ac0a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:19.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.328+0000 7f43b1d5d700 1 --2- 172.21.15.92:0/3170472781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43ac0b2820 0x7f43ac1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:19.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.329+0000 7f43b255e700 1 -- 172.21.15.92:0/3170472781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43ac0b2820 msgr2=0x7f43ac1501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:19.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.329+0000 7f43b255e700 1 --2- 172.21.15.92:0/3170472781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43ac0b2820 0x7f43ac1501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:19.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.329+0000 7f43b255e700 1 -- 172.21.15.92:0/3170472781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43ac157360 msgr2=0x7f43ac0a98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:19.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.329+0000 7f43b255e700 1 --2- 172.21.15.92:0/3170472781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43ac157360 0x7f43ac0a98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:19.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.329+0000 7f43b255e700 1 -- 172.21.15.92:0/3170472781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439c009420 con 0x7f43ac0af670 2023-10-06T10:40:19.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.329+0000 7f43b2d5f700 1 --2- 172.21.15.92:0/3170472781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43ac157360 0x7f43ac0a98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:19.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.329+0000 7f43b255e700 1 --2- 172.21.15.92:0/3170472781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43ac0af670 0x7f43ac156490 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f43a800c7e0 tx=0x7f43a8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:19.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.329+0000 7f43a27fc700 1 -- 172.21.15.92:0/3170472781 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8005b30 con 0x7f43ac0af670 2023-10-06T10:40:19.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.329+0000 7f43a27fc700 1 -- 172.21.15.92:0/3170472781 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a8005c90 con 0x7f43ac0af670 2023-10-06T10:40:19.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.329+0000 7f43b3560700 1 -- 172.21.15.92:0/3170472781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43ac0aa0b0 con 0x7f43ac0af670 2023-10-06T10:40:19.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.329+0000 7f43a27fc700 1 -- 172.21.15.92:0/3170472781 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8007300 con 0x7f43ac0af670 2023-10-06T10:40:19.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.329+0000 7f43b3560700 1 -- 172.21.15.92:0/3170472781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43ac0aa670 con 0x7f43ac0af670 2023-10-06T10:40:19.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.330+0000 7f43b3560700 1 -- 172.21.15.92:0/3170472781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43ac05f230 con 0x7f43ac0af670 2023-10-06T10:40:19.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.330+0000 7f43a27fc700 1 -- 172.21.15.92:0/3170472781 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f43a80076b0 con 0x7f43ac0af670 2023-10-06T10:40:19.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.330+0000 7f43a27fc700 1 --2- 172.21.15.92:0/3170472781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4390066f30 0x7f43900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:19.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.331+0000 7f43a27fc700 1 -- 172.21.15.92:0/3170472781 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f43a80842b0 con 0x7f43ac0af670 2023-10-06T10:40:19.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.331+0000 7f43b1d5d700 1 --2- 172.21.15.92:0/3170472781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4390066f30 0x7f43900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:19.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.331+0000 7f43b1d5d700 1 --2- 172.21.15.92:0/3170472781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4390066f30 0x7f43900693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f439c0091e0 tx=0x7f439c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:19.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.335+0000 7f43a27fc700 1 -- 172.21.15.92:0/3170472781 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43a80512c0 con 0x7f43ac0af670 2023-10-06T10:40:19.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.562+0000 7f43b3560700 1 -- 172.21.15.92:0/3170472781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43ac0af450 con 0x7f43ac0af670 2023-10-06T10:40:19.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.563+0000 7f43a27fc700 1 -- 172.21.15.92:0/3170472781 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4634 (secure 0 0 0) 0x7f43a80548e0 con 0x7f43ac0af670 2023-10-06T10:40:19.565 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:19.565 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2525196781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2525196781},{"type":"v1","addr":"172.21.15.124:6835","nonce":2525196781}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:39:15.675013+0000","modified":"2023-10-06T10:39:48.393721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5133,"mds_1":5121,"mds_2":5118},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5118":{"gid":5118,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6836/2409440810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2409440810},{"type":"v1","addr":"172.21.15.92:6836","nonce":2409440810}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5121":{"gid":5121,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/4010629505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4010629505},{"type":"v1","addr":"172.21.15.92:6837","nonce":4010629505}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5133":{"gid":5133,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.124:6834/484869337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":484869337},{"type":"v1","addr":"172.21.15.124:6834","nonce":484869337}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T10:40:19.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.565+0000 7f43b3560700 1 -- 172.21.15.92:0/3170472781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4390066f30 msgr2=0x7f43900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:19.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.565+0000 7f43b3560700 1 --2- 172.21.15.92:0/3170472781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4390066f30 0x7f43900693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f439c0091e0 tx=0x7f439c016000 comp rx=0 tx=0).stop 2023-10-06T10:40:19.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.565+0000 7f43b3560700 1 -- 172.21.15.92:0/3170472781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43ac0af670 msgr2=0x7f43ac156490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:19.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.565+0000 7f43b3560700 1 --2- 172.21.15.92:0/3170472781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43ac0af670 0x7f43ac156490 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f43a800c7e0 tx=0x7f43a8007df0 comp rx=0 tx=0).stop 2023-10-06T10:40:19.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.565+0000 7f43b3560700 1 -- 172.21.15.92:0/3170472781 shutdown_connections 2023-10-06T10:40:19.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.565+0000 7f43b3560700 1 --2- 172.21.15.92:0/3170472781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4390066f30 0x7f43900693e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:19.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.565+0000 7f43b3560700 1 --2- 172.21.15.92:0/3170472781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43ac157360 0x7f43ac0a98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:19.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.565+0000 7f43b3560700 1 --2- 172.21.15.92:0/3170472781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43ac0af670 0x7f43ac156490 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:19.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.565+0000 7f43b3560700 1 --2- 172.21.15.92:0/3170472781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43ac0b2820 0x7f43ac1501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:19.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.565+0000 7f43b3560700 1 -- 172.21.15.92:0/3170472781 >> 172.21.15.92:0/3170472781 conn(0x7f43ac069190 msgr2=0x7f43ac157730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:19.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.565+0000 7f43b3560700 1 -- 172.21.15.92:0/3170472781 shutdown_connections 2023-10-06T10:40:19.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.565+0000 7f43b3560700 1 -- 172.21.15.92:0/3170472781 wait complete. 2023-10-06T10:40:19.568 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 24 2023-10-06T10:40:19.578 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:40:19.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.690+0000 7fe666c59700 1 Processor -- start 2023-10-06T10:40:19.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.690+0000 7fe666c59700 1 -- start start 2023-10-06T10:40:19.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.691+0000 7fe666c59700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe660159560 0x7fe6600aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:19.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.691+0000 7fe666c59700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe6600b0760 0x7fe6600b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:19.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.691+0000 7fe666c59700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6600b0c60 con 0x7fe6600b49a0 2023-10-06T10:40:19.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.691+0000 7fe666c59700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6600b0f40 con 0x7fe660159560 2023-10-06T10:40:19.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.691+0000 7fe666c59700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6600b1220 con 0x7fe6600b0760 2023-10-06T10:40:19.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.691+0000 7fe6649f5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe660159560 0x7fe6600aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:19.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.691+0000 7fe657fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe6600b49a0 0x7fe6600aaa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54146/0 (socket says 172.21.15.92:54146) 2023-10-06T10:40:19.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.691+0000 7fe657fff700 1 -- 172.21.15.92:0/12179588 learned_addr learned my addr 172.21.15.92:0/12179588 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:19.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.691+0000 7fe6649f5700 1 -- 172.21.15.92:0/12179588 >> v2:172.21.15.124:3300/0 conn(0x7fe6600b0760 msgr2=0x7fe6600b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:19.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.691+0000 7fe6649f5700 1 --2- 172.21.15.92:0/12179588 >> v2:172.21.15.124:3300/0 conn(0x7fe6600b0760 0x7fe6600b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:19.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.691+0000 7fe6649f5700 1 -- 172.21.15.92:0/12179588 >> v1:172.21.15.92:6789/0 conn(0x7fe6600b49a0 legacy=0x7fe6600aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:19.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.692+0000 7fe6649f5700 1 -- 172.21.15.92:0/12179588 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6600b1500 con 0x7fe660159560 2023-10-06T10:40:19.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.691+0000 7fe6577fe700 1 -- 172.21.15.92:0/12179588 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483543735 0 0) 0x7fe6600b0c60 con 0x7fe6600b49a0 2023-10-06T10:40:19.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.692+0000 7fe6649f5700 1 --2- 172.21.15.92:0/12179588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe660159560 0x7fe6600aa530 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0098d0 tx=0x7fe65c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a46e2e0be97c92e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:19.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.692+0000 7fe6577fe700 1 -- 172.21.15.92:0/12179588 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c009e70 con 0x7fe660159560 2023-10-06T10:40:19.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.692+0000 7fe6577fe700 1 -- 172.21.15.92:0/12179588 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65c005c10 con 0x7fe660159560 2023-10-06T10:40:19.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.692+0000 7fe6577fe700 1 -- 172.21.15.92:0/12179588 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c00b610 con 0x7fe660159560 2023-10-06T10:40:19.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.692+0000 7fe666c59700 1 -- 172.21.15.92:0/12179588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe660159560 msgr2=0x7fe6600aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:19.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.692+0000 7fe666c59700 1 --2- 172.21.15.92:0/12179588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe660159560 0x7fe6600aa530 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0098d0 tx=0x7fe65c009120 comp rx=0 tx=0).stop 2023-10-06T10:40:19.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.692+0000 7fe666c59700 1 -- 172.21.15.92:0/12179588 shutdown_connections 2023-10-06T10:40:19.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.692+0000 7fe666c59700 1 --2- 172.21.15.92:0/12179588 >> v2:172.21.15.124:3300/0 conn(0x7fe6600b0760 0x7fe6600b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:19.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.692+0000 7fe666c59700 1 --2- 172.21.15.92:0/12179588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe660159560 0x7fe6600aa530 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:19.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.692+0000 7fe666c59700 1 -- 172.21.15.92:0/12179588 >> 172.21.15.92:0/12179588 conn(0x7fe6600997f0 msgr2=0x7fe66006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:19.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.692+0000 7fe666c59700 1 -- 172.21.15.92:0/12179588 shutdown_connections 2023-10-06T10:40:19.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.692+0000 7fe666c59700 1 -- 172.21.15.92:0/12179588 wait complete. 2023-10-06T10:40:19.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.693+0000 7fe666c59700 1 Processor -- start 2023-10-06T10:40:19.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.694+0000 7fe666c59700 1 -- start start 2023-10-06T10:40:19.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.694+0000 7fe666c59700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6600b0760 0x7fe66014d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:19.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.694+0000 7fe666c59700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6600b49a0 0x7fe66016cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:19.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.694+0000 7fe666c59700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe660159560 0x7fe660069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:19.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.694+0000 7fe666c59700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66009c8a0 con 0x7fe6600b0760 2023-10-06T10:40:19.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.694+0000 7fe666c59700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66016d910 con 0x7fe6600b49a0 2023-10-06T10:40:19.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.694+0000 7fe666c59700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66016dbf0 con 0x7fe660159560 2023-10-06T10:40:19.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.694+0000 7fe6649f5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6600b0760 0x7fe66014d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:19.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.694+0000 7fe6649f5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6600b0760 0x7fe66014d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58258/0 (socket says 172.21.15.92:58258) 2023-10-06T10:40:19.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.694+0000 7fe6649f5700 1 -- 172.21.15.92:0/3633700341 learned_addr learned my addr 172.21.15.92:0/3633700341 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:19.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.695+0000 7fe657fff700 1 --2- 172.21.15.92:0/3633700341 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6600b49a0 0x7fe66016cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:19.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.695+0000 7fe6651f6700 1 --2- 172.21.15.92:0/3633700341 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe660159560 0x7fe660069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:19.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.695+0000 7fe6649f5700 1 -- 172.21.15.92:0/3633700341 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe660159560 msgr2=0x7fe660069670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:19.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.695+0000 7fe6649f5700 1 --2- 172.21.15.92:0/3633700341 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe660159560 0x7fe660069670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:19.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.695+0000 7fe6649f5700 1 -- 172.21.15.92:0/3633700341 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6600b49a0 msgr2=0x7fe66016cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:19.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.695+0000 7fe6649f5700 1 --2- 172.21.15.92:0/3633700341 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6600b49a0 0x7fe66016cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:19.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.695+0000 7fe6649f5700 1 -- 172.21.15.92:0/3633700341 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65c0093c0 con 0x7fe6600b0760 2023-10-06T10:40:19.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.695+0000 7fe657fff700 1 --2- 172.21.15.92:0/3633700341 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6600b49a0 0x7fe66016cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:19.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.695+0000 7fe6651f6700 1 --2- 172.21.15.92:0/3633700341 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe660159560 0x7fe660069670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:19.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.695+0000 7fe6649f5700 1 --2- 172.21.15.92:0/3633700341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6600b0760 0x7fe66014d6d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009160 tx=0x7fe65c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:19.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.695+0000 7fe654ff9700 1 -- 172.21.15.92:0/3633700341 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c006310 con 0x7fe6600b0760 2023-10-06T10:40:19.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.695+0000 7fe654ff9700 1 -- 172.21.15.92:0/3633700341 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65c006470 con 0x7fe6600b0760 2023-10-06T10:40:19.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.695+0000 7fe666c59700 1 -- 172.21.15.92:0/3633700341 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe66009c530 con 0x7fe6600b0760 2023-10-06T10:40:19.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.695+0000 7fe654ff9700 1 -- 172.21.15.92:0/3633700341 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c006830 con 0x7fe6600b0760 2023-10-06T10:40:19.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.695+0000 7fe666c59700 1 -- 172.21.15.92:0/3633700341 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe660157be0 con 0x7fe6600b0760 2023-10-06T10:40:19.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.696+0000 7fe666c59700 1 -- 172.21.15.92:0/3633700341 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe660064a90 con 0x7fe6600b0760 2023-10-06T10:40:19.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.696+0000 7fe654ff9700 1 -- 172.21.15.92:0/3633700341 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe65c01b690 con 0x7fe6600b0760 2023-10-06T10:40:19.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.697+0000 7fe654ff9700 1 --2- 172.21.15.92:0/3633700341 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe64c066f30 0x7fe64c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:19.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.697+0000 7fe657fff700 1 --2- 172.21.15.92:0/3633700341 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe64c066f30 0x7fe64c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:19.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.697+0000 7fe654ff9700 1 -- 172.21.15.92:0/3633700341 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fe65c086a70 con 0x7fe6600b0760 2023-10-06T10:40:19.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.697+0000 7fe657fff700 1 --2- 172.21.15.92:0/3633700341 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe64c066f30 0x7fe64c0693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe6600cfc80 tx=0x7fe650009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:19.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.701+0000 7fe654ff9700 1 -- 172.21.15.92:0/3633700341 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe65c053a00 con 0x7fe6600b0760 2023-10-06T10:40:19.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.884+0000 7fe666c59700 1 -- 172.21.15.92:0/3633700341 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6600b17e0 con 0x7fe6600b0760 2023-10-06T10:40:19.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.885+0000 7fe654ff9700 1 -- 172.21.15.92:0/3633700341 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+17166 (secure 0 0 0) 0x7fe65c057020 con 0x7fe6600b0760 2023-10-06T10:40:19.887 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:19.887 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":40,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:40:06.432393+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:39:13.780983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T10:39:14.644706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":4}]},{"pgid":"7.2a","mappings":[{"from":0,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:40:19.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.887+0000 7fe666c59700 1 -- 172.21.15.92:0/3633700341 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe64c066f30 msgr2=0x7fe64c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:19.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.887+0000 7fe666c59700 1 --2- 172.21.15.92:0/3633700341 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe64c066f30 0x7fe64c0693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe6600cfc80 tx=0x7fe650009040 comp rx=0 tx=0).stop 2023-10-06T10:40:19.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.887+0000 7fe666c59700 1 -- 172.21.15.92:0/3633700341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6600b0760 msgr2=0x7fe66014d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:19.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.887+0000 7fe666c59700 1 --2- 172.21.15.92:0/3633700341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6600b0760 0x7fe66014d6d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009160 tx=0x7fe65c016000 comp rx=0 tx=0).stop 2023-10-06T10:40:19.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.887+0000 7fe666c59700 1 -- 172.21.15.92:0/3633700341 shutdown_connections 2023-10-06T10:40:19.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.887+0000 7fe666c59700 1 --2- 172.21.15.92:0/3633700341 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe64c066f30 0x7fe64c0693e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:19.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.887+0000 7fe666c59700 1 --2- 172.21.15.92:0/3633700341 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe660159560 0x7fe660069670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:19.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.887+0000 7fe666c59700 1 --2- 172.21.15.92:0/3633700341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6600b0760 0x7fe66014d6d0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:19.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.887+0000 7fe666c59700 1 --2- 172.21.15.92:0/3633700341 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6600b49a0 0x7fe66016cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:19.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.887+0000 7fe666c59700 1 -- 172.21.15.92:0/3633700341 >> 172.21.15.92:0/3633700341 conn(0x7fe6600997f0 msgr2=0x7fe660158960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:19.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.887+0000 7fe666c59700 1 -- 172.21.15.92:0/3633700341 shutdown_connections 2023-10-06T10:40:19.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:19.887+0000 7fe666c59700 1 -- 172.21.15.92:0/3633700341 wait complete. 2023-10-06T10:40:19.901 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T10:38:09.680559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T10:38:10.120786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T10:39:13.780983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T10:39:14.644706+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T10:40:19.901 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T10:40:20.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.015+0000 7f9bfe5e1700 1 Processor -- start 2023-10-06T10:40:20.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.015+0000 7f9bfe5e1700 1 -- start start 2023-10-06T10:40:20.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.016+0000 7f9bfe5e1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9bf81598a0 0x7f9bf80ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:20.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.016+0000 7f9bfe5e1700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9bf80cbc70 0x7f9bf80ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:20.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.016+0000 7f9bfe5e1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf80cf5d0 con 0x7f9bf81594d0 2023-10-06T10:40:20.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.016+0000 7f9bfe5e1700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf80cf8b0 con 0x7f9bf81598a0 2023-10-06T10:40:20.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.016+0000 7f9bfe5e1700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf80cfb90 con 0x7f9bf80cbc70 2023-10-06T10:40:20.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.016+0000 7f9bf77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9bf81598a0 0x7f9bf80ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:20.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.016+0000 7f9bf7fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9bf81594d0 0x7f9bf80cb560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54148/0 (socket says 172.21.15.92:54148) 2023-10-06T10:40:20.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.016+0000 7f9bf7fff700 1 -- 172.21.15.92:0/1809731145 learned_addr learned my addr 172.21.15.92:0/1809731145 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:20.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.016+0000 7f9bfcb7e700 1 --2- 172.21.15.92:0/1809731145 >> v2:172.21.15.124:3300/0 conn(0x7f9bf80cbc70 0x7f9bf80ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:20.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.017+0000 7f9bf77fe700 1 -- 172.21.15.92:0/1809731145 >> v2:172.21.15.124:3300/0 conn(0x7f9bf80cbc70 msgr2=0x7f9bf80ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:20.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.017+0000 7f9bf77fe700 1 --2- 172.21.15.92:0/1809731145 >> v2:172.21.15.124:3300/0 conn(0x7f9bf80cbc70 0x7f9bf80ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.017+0000 7f9bf77fe700 1 -- 172.21.15.92:0/1809731145 >> v1:172.21.15.92:6789/0 conn(0x7f9bf81594d0 legacy=0x7f9bf80cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:20.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.017+0000 7f9bf77fe700 1 -- 172.21.15.92:0/1809731145 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf80cfe70 con 0x7f9bf81598a0 2023-10-06T10:40:20.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.017+0000 7f9bf77fe700 1 --2- 172.21.15.92:0/1809731145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bf81598a0 0x7f9bf80ce7e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9bf806f950 tx=0x7f9bec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0798e61b72f745 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:20.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.017+0000 7f9bf6ffd700 1 -- 172.21.15.92:0/1809731145 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec011630 con 0x7f9bf81598a0 2023-10-06T10:40:20.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.017+0000 7f9bf6ffd700 1 -- 172.21.15.92:0/1809731145 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec011790 con 0x7f9bf81598a0 2023-10-06T10:40:20.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.017+0000 7f9bf6ffd700 1 -- 172.21.15.92:0/1809731145 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec011b30 con 0x7f9bf81598a0 2023-10-06T10:40:20.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.017+0000 7f9bfe5e1700 1 -- 172.21.15.92:0/1809731145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bf81598a0 msgr2=0x7f9bf80ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:20.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.017+0000 7f9bfe5e1700 1 --2- 172.21.15.92:0/1809731145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bf81598a0 0x7f9bf80ce7e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9bf806f950 tx=0x7f9bec0091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.017+0000 7f9bfe5e1700 1 -- 172.21.15.92:0/1809731145 shutdown_connections 2023-10-06T10:40:20.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.017+0000 7f9bfe5e1700 1 --2- 172.21.15.92:0/1809731145 >> v2:172.21.15.124:3300/0 conn(0x7f9bf80cbc70 0x7f9bf80ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.017+0000 7f9bfe5e1700 1 --2- 172.21.15.92:0/1809731145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bf81598a0 0x7f9bf80ce7e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.017+0000 7f9bfe5e1700 1 -- 172.21.15.92:0/1809731145 >> 172.21.15.92:0/1809731145 conn(0x7f9bf8089e50 msgr2=0x7f9bf8099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:20.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.017+0000 7f9bfe5e1700 1 -- 172.21.15.92:0/1809731145 shutdown_connections 2023-10-06T10:40:20.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.017+0000 7f9bfe5e1700 1 -- 172.21.15.92:0/1809731145 wait complete. 2023-10-06T10:40:20.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.018+0000 7f9bfe5e1700 1 Processor -- start 2023-10-06T10:40:20.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.019+0000 7f9bfe5e1700 1 -- start start 2023-10-06T10:40:20.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.019+0000 7f9bfe5e1700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bf80cbc70 0x7f9bf8159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:20.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.019+0000 7f9bfe5e1700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bf81594d0 0x7f9bf815be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:20.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.019+0000 7f9bfe5e1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bf81598a0 0x7f9bf80c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:20.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.019+0000 7f9bfe5e1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80b5410 con 0x7f9bf81598a0 2023-10-06T10:40:20.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.019+0000 7f9bfe5e1700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80b4b00 con 0x7f9bf81594d0 2023-10-06T10:40:20.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.019+0000 7f9bfe5e1700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf815c910 con 0x7f9bf80cbc70 2023-10-06T10:40:20.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.019+0000 7f9bfcb7e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bf81598a0 0x7f9bf80c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:20.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.020+0000 7f9bfcb7e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bf81598a0 0x7f9bf80c6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58284/0 (socket says 172.21.15.92:58284) 2023-10-06T10:40:20.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.020+0000 7f9bfcb7e700 1 -- 172.21.15.92:0/747449242 learned_addr learned my addr 172.21.15.92:0/747449242 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:20.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.020+0000 7f9bf77fe700 1 --2- 172.21.15.92:0/747449242 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bf81594d0 0x7f9bf815be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:20.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.020+0000 7f9bf7fff700 1 --2- 172.21.15.92:0/747449242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bf80cbc70 0x7f9bf8159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:20.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.020+0000 7f9bfcb7e700 1 -- 172.21.15.92:0/747449242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bf80cbc70 msgr2=0x7f9bf8159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:20.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.020+0000 7f9bfcb7e700 1 --2- 172.21.15.92:0/747449242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bf80cbc70 0x7f9bf8159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.020+0000 7f9bfcb7e700 1 -- 172.21.15.92:0/747449242 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bf81594d0 msgr2=0x7f9bf815be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:20.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.020+0000 7f9bfcb7e700 1 --2- 172.21.15.92:0/747449242 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bf81594d0 0x7f9bf815be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.020+0000 7f9bfcb7e700 1 -- 172.21.15.92:0/747449242 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec009420 con 0x7f9bf81598a0 2023-10-06T10:40:20.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.020+0000 7f9bf7fff700 1 --2- 172.21.15.92:0/747449242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bf80cbc70 0x7f9bf8159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:20.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.020+0000 7f9bf77fe700 1 --2- 172.21.15.92:0/747449242 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bf81594d0 0x7f9bf815be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:20.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.020+0000 7f9bfcb7e700 1 --2- 172.21.15.92:0/747449242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bf81598a0 0x7f9bf80c6720 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0007f20 tx=0x7f9bf00098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:20.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.020+0000 7f9bdffff700 1 -- 172.21.15.92:0/747449242 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf000eca0 con 0x7f9bf81598a0 2023-10-06T10:40:20.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.020+0000 7f9bdffff700 1 -- 172.21.15.92:0/747449242 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf000ee20 con 0x7f9bf81598a0 2023-10-06T10:40:20.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.020+0000 7f9bfe5e1700 1 -- 172.21.15.92:0/747449242 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf80b4dc0 con 0x7f9bf81598a0 2023-10-06T10:40:20.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.020+0000 7f9bdffff700 1 -- 172.21.15.92:0/747449242 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0014670 con 0x7f9bf81598a0 2023-10-06T10:40:20.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.020+0000 7f9bfe5e1700 1 -- 172.21.15.92:0/747449242 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf80b34d0 con 0x7f9bf81598a0 2023-10-06T10:40:20.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.021+0000 7f9bfe5e1700 1 -- 172.21.15.92:0/747449242 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf8064a90 con 0x7f9bf81598a0 2023-10-06T10:40:20.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.021+0000 7f9bdffff700 1 -- 172.21.15.92:0/747449242 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9bf0014a60 con 0x7f9bf81598a0 2023-10-06T10:40:20.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.022+0000 7f9bdffff700 1 --2- 172.21.15.92:0/747449242 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9be4066f30 0x7f9be40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:20.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.022+0000 7f9bdffff700 1 -- 172.21.15.92:0/747449242 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f9bf0084f90 con 0x7f9bf81598a0 2023-10-06T10:40:20.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.022+0000 7f9bf7fff700 1 --2- 172.21.15.92:0/747449242 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9be4066f30 0x7f9be40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:20.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.022+0000 7f9bf7fff700 1 --2- 172.21.15.92:0/747449242 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9be4066f30 0x7f9be40693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9be8007fd0 tx=0x7f9be8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:20.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.026+0000 7f9bdffff700 1 -- 172.21.15.92:0/747449242 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bf0051f20 con 0x7f9bf81598a0 2023-10-06T10:40:20.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.209+0000 7f9bfe5e1700 1 -- 172.21.15.92:0/747449242 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9bf80cf260 con 0x7f9bf81598a0 2023-10-06T10:40:20.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.209+0000 7f9bdffff700 1 -- 172.21.15.92:0/747449242 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9bf0055540 con 0x7f9bf81598a0 2023-10-06T10:40:20.211 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T10:40:20.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.211+0000 7f9bfe5e1700 1 -- 172.21.15.92:0/747449242 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9be4066f30 msgr2=0x7f9be40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:20.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.211+0000 7f9bfe5e1700 1 --2- 172.21.15.92:0/747449242 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9be4066f30 0x7f9be40693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9be8007fd0 tx=0x7f9be8009040 comp rx=0 tx=0).stop 2023-10-06T10:40:20.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.211+0000 7f9bfe5e1700 1 -- 172.21.15.92:0/747449242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bf81598a0 msgr2=0x7f9bf80c6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:20.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.211+0000 7f9bfe5e1700 1 --2- 172.21.15.92:0/747449242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bf81598a0 0x7f9bf80c6720 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0007f20 tx=0x7f9bf00098a0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.211+0000 7f9bfe5e1700 1 -- 172.21.15.92:0/747449242 shutdown_connections 2023-10-06T10:40:20.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.211+0000 7f9bfe5e1700 1 --2- 172.21.15.92:0/747449242 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9be4066f30 0x7f9be40693e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.211+0000 7f9bfe5e1700 1 --2- 172.21.15.92:0/747449242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bf81598a0 0x7f9bf80c6720 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.211+0000 7f9bfe5e1700 1 --2- 172.21.15.92:0/747449242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bf80cbc70 0x7f9bf8159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.211+0000 7f9bfe5e1700 1 --2- 172.21.15.92:0/747449242 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bf81594d0 0x7f9bf815be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.211+0000 7f9bfe5e1700 1 -- 172.21.15.92:0/747449242 >> 172.21.15.92:0/747449242 conn(0x7f9bf8089e50 msgr2=0x7f9bf814b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:20.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.211+0000 7f9bfe5e1700 1 -- 172.21.15.92:0/747449242 shutdown_connections 2023-10-06T10:40:20.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.211+0000 7f9bfe5e1700 1 -- 172.21.15.92:0/747449242 wait complete. 2023-10-06T10:40:20.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T10:40:20.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.341+0000 7f6c824b6700 1 Processor -- start 2023-10-06T10:40:20.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.342+0000 7f6c824b6700 1 -- start start 2023-10-06T10:40:20.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.342+0000 7f6c824b6700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6c7c1594d0 0x7f6c7c0cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:20.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.342+0000 7f6c824b6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6c7c1598a0 0x7f6c7c0cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:20.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.342+0000 7f6c824b6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c7c0cf330 con 0x7f6c7c0ce7e0 2023-10-06T10:40:20.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.342+0000 7f6c824b6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c7c0cf5d0 con 0x7f6c7c1598a0 2023-10-06T10:40:20.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.342+0000 7f6c824b6700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c7c0cf8b0 con 0x7f6c7c1594d0 2023-10-06T10:40:20.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.342+0000 7f6c7b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6c7c1598a0 0x7f6c7c0cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:20.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.342+0000 7f6c80a53700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6c7c0ce7e0 0x7f6c7c0cec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54156/0 (socket says 172.21.15.92:54156) 2023-10-06T10:40:20.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.342+0000 7f6c7b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6c7c1598a0 0x7f6c7c0cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58286/0 (socket says 172.21.15.92:58286) 2023-10-06T10:40:20.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.342+0000 7f6c80a53700 1 -- 172.21.15.92:0/1799846557 learned_addr learned my addr 172.21.15.92:0/1799846557 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:20.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.343+0000 7f6c7bfff700 1 --2- 172.21.15.92:0/1799846557 >> v2:172.21.15.124:3300/0 conn(0x7f6c7c1594d0 0x7f6c7c0cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:20.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.343+0000 7f6c7b7fe700 1 -- 172.21.15.92:0/1799846557 >> v2:172.21.15.124:3300/0 conn(0x7f6c7c1594d0 msgr2=0x7f6c7c0cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:20.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.343+0000 7f6c7b7fe700 1 --2- 172.21.15.92:0/1799846557 >> v2:172.21.15.124:3300/0 conn(0x7f6c7c1594d0 0x7f6c7c0cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.343+0000 7f6c7b7fe700 1 -- 172.21.15.92:0/1799846557 >> v1:172.21.15.92:6789/0 conn(0x7f6c7c0ce7e0 legacy=0x7f6c7c0cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:20.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.343+0000 7f6c7b7fe700 1 -- 172.21.15.92:0/1799846557 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7c0cfb90 con 0x7f6c7c1598a0 2023-10-06T10:40:20.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.343+0000 7f6c7affd700 1 -- 172.21.15.92:0/1799846557 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2430618296 0 0) 0x7f6c7c0cf330 con 0x7f6c7c0ce7e0 2023-10-06T10:40:20.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.343+0000 7f6c7bfff700 1 --2- 172.21.15.92:0/1799846557 >> v2:172.21.15.124:3300/0 conn(0x7f6c7c1594d0 0x7f6c7c0cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:20.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.343+0000 7f6c7b7fe700 1 --2- 172.21.15.92:0/1799846557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c7c1598a0 0x7f6c7c0cbaa0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c088f70 tx=0x7f6c700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dab9b680422dccd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:20.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.343+0000 7f6c7affd700 1 -- 172.21.15.92:0/1799846557 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7000b6b0 con 0x7f6c7c1598a0 2023-10-06T10:40:20.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.343+0000 7f6c7affd700 1 -- 172.21.15.92:0/1799846557 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7000b810 con 0x7f6c7c1598a0 2023-10-06T10:40:20.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.343+0000 7f6c7affd700 1 -- 172.21.15.92:0/1799846557 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7000bbb0 con 0x7f6c7c1598a0 2023-10-06T10:40:20.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.344+0000 7f6c824b6700 1 -- 172.21.15.92:0/1799846557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c7c1598a0 msgr2=0x7f6c7c0cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:20.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.344+0000 7f6c824b6700 1 --2- 172.21.15.92:0/1799846557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c7c1598a0 0x7f6c7c0cbaa0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c088f70 tx=0x7f6c700091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.344+0000 7f6c824b6700 1 -- 172.21.15.92:0/1799846557 shutdown_connections 2023-10-06T10:40:20.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.344+0000 7f6c824b6700 1 --2- 172.21.15.92:0/1799846557 >> v2:172.21.15.124:3300/0 conn(0x7f6c7c1594d0 0x7f6c7c0cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.344+0000 7f6c824b6700 1 --2- 172.21.15.92:0/1799846557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c7c1598a0 0x7f6c7c0cbaa0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.344+0000 7f6c824b6700 1 -- 172.21.15.92:0/1799846557 >> 172.21.15.92:0/1799846557 conn(0x7f6c7c089e50 msgr2=0x7f6c7c099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:20.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.344+0000 7f6c824b6700 1 -- 172.21.15.92:0/1799846557 shutdown_connections 2023-10-06T10:40:20.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.344+0000 7f6c824b6700 1 -- 172.21.15.92:0/1799846557 wait complete. 2023-10-06T10:40:20.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.345+0000 7f6c824b6700 1 Processor -- start 2023-10-06T10:40:20.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.345+0000 7f6c824b6700 1 -- start start 2023-10-06T10:40:20.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.346+0000 7f6c824b6700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c7c0ce7e0 0x7f6c7c0b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:20.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.346+0000 7f6c824b6700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c7c1594d0 0x7f6c7c14a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:20.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.346+0000 7f6c824b6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c7c1598a0 0x7f6c7c0b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:20.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.346+0000 7f6c824b6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c14a5e0 con 0x7f6c7c1598a0 2023-10-06T10:40:20.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.346+0000 7f6c824b6700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c0b5380 con 0x7f6c7c0ce7e0 2023-10-06T10:40:20.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.346+0000 7f6c824b6700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c0b3890 con 0x7f6c7c1594d0 2023-10-06T10:40:20.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.346+0000 7f6c80a53700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c7c1598a0 0x7f6c7c0b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:20.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.346+0000 7f6c80a53700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c7c1598a0 0x7f6c7c0b4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58294/0 (socket says 172.21.15.92:58294) 2023-10-06T10:40:20.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.346+0000 7f6c80a53700 1 -- 172.21.15.92:0/3920402396 learned_addr learned my addr 172.21.15.92:0/3920402396 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:20.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.346+0000 7f6c80a53700 1 -- 172.21.15.92:0/3920402396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c7c1594d0 msgr2=0x7f6c7c14a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:20.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.346+0000 7f6c80a53700 1 --2- 172.21.15.92:0/3920402396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c7c1594d0 0x7f6c7c14a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.346+0000 7f6c80a53700 1 -- 172.21.15.92:0/3920402396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c7c0ce7e0 msgr2=0x7f6c7c0b32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:20.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.346+0000 7f6c7bfff700 1 --2- 172.21.15.92:0/3920402396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c7c0ce7e0 0x7f6c7c0b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:20.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.346+0000 7f6c80a53700 1 --2- 172.21.15.92:0/3920402396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c7c0ce7e0 0x7f6c7c0b32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.346+0000 7f6c80a53700 1 -- 172.21.15.92:0/3920402396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c70009420 con 0x7f6c7c1598a0 2023-10-06T10:40:20.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.346+0000 7f6c80a53700 1 --2- 172.21.15.92:0/3920402396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c7c1598a0 0x7f6c7c0b4e40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400cc90 tx=0x7f6c74001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:20.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.346+0000 7f6c63fff700 1 -- 172.21.15.92:0/3920402396 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7400cea0 con 0x7f6c7c1598a0 2023-10-06T10:40:20.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.346+0000 7f6c63fff700 1 -- 172.21.15.92:0/3920402396 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7400ab70 con 0x7f6c7c1598a0 2023-10-06T10:40:20.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.346+0000 7f6c824b6700 1 -- 172.21.15.92:0/3920402396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7c0b3b70 con 0x7f6c7c1598a0 2023-10-06T10:40:20.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.346+0000 7f6c63fff700 1 -- 172.21.15.92:0/3920402396 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c740070f0 con 0x7f6c7c1598a0 2023-10-06T10:40:20.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.347+0000 7f6c824b6700 1 -- 172.21.15.92:0/3920402396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7c0b4130 con 0x7f6c7c1598a0 2023-10-06T10:40:20.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.347+0000 7f6c824b6700 1 -- 172.21.15.92:0/3920402396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c7c064a90 con 0x7f6c7c1598a0 2023-10-06T10:40:20.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.348+0000 7f6c63fff700 1 -- 172.21.15.92:0/3920402396 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6c74007460 con 0x7f6c7c1598a0 2023-10-06T10:40:20.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.348+0000 7f6c63fff700 1 --2- 172.21.15.92:0/3920402396 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6c68066f30 0x7f6c680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:20.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.348+0000 7f6c63fff700 1 -- 172.21.15.92:0/3920402396 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f6c74084d40 con 0x7f6c7c1598a0 2023-10-06T10:40:20.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.348+0000 7f6c7bfff700 1 --2- 172.21.15.92:0/3920402396 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6c68066f30 0x7f6c680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:20.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.349+0000 7f6c7bfff700 1 --2- 172.21.15.92:0/3920402396 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6c68066f30 0x7f6c680693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c006f00 tx=0x7f6c6c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:20.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.352+0000 7f6c63fff700 1 -- 172.21.15.92:0/3920402396 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c74051cd0 con 0x7f6c7c1598a0 2023-10-06T10:40:20.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.537+0000 7f6c824b6700 1 -- 172.21.15.92:0/3920402396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6c7c0cfe70 con 0x7f6c7c1598a0 2023-10-06T10:40:20.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.537+0000 7f6c63fff700 1 -- 172.21.15.92:0/3920402396 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6c740552f0 con 0x7f6c7c1598a0 2023-10-06T10:40:20.539 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T10:40:20.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.539+0000 7f6c824b6700 1 -- 172.21.15.92:0/3920402396 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6c68066f30 msgr2=0x7f6c680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:20.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.539+0000 7f6c824b6700 1 --2- 172.21.15.92:0/3920402396 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6c68066f30 0x7f6c680693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c006f00 tx=0x7f6c6c008040 comp rx=0 tx=0).stop 2023-10-06T10:40:20.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.539+0000 7f6c824b6700 1 -- 172.21.15.92:0/3920402396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c7c1598a0 msgr2=0x7f6c7c0b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:20.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.539+0000 7f6c824b6700 1 --2- 172.21.15.92:0/3920402396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c7c1598a0 0x7f6c7c0b4e40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400cc90 tx=0x7f6c74001a80 comp rx=0 tx=0).stop 2023-10-06T10:40:20.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.539+0000 7f6c824b6700 1 -- 172.21.15.92:0/3920402396 shutdown_connections 2023-10-06T10:40:20.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.539+0000 7f6c824b6700 1 --2- 172.21.15.92:0/3920402396 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6c68066f30 0x7f6c680693e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.539+0000 7f6c824b6700 1 --2- 172.21.15.92:0/3920402396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c7c1598a0 0x7f6c7c0b4e40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.539+0000 7f6c824b6700 1 --2- 172.21.15.92:0/3920402396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c7c0ce7e0 0x7f6c7c0b32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.539+0000 7f6c824b6700 1 --2- 172.21.15.92:0/3920402396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c7c1594d0 0x7f6c7c14a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.539+0000 7f6c824b6700 1 -- 172.21.15.92:0/3920402396 >> 172.21.15.92:0/3920402396 conn(0x7f6c7c089e50 msgr2=0x7f6c7c0c65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:20.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.539+0000 7f6c824b6700 1 -- 172.21.15.92:0/3920402396 shutdown_connections 2023-10-06T10:40:20.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.539+0000 7f6c824b6700 1 -- 172.21.15.92:0/3920402396 wait complete. 2023-10-06T10:40:20.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T10:40:20.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.667+0000 7fb8f101c700 1 Processor -- start 2023-10-06T10:40:20.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.667+0000 7fb8f101c700 1 -- start start 2023-10-06T10:40:20.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.668+0000 7fb8f101c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb8ec159530 0x7fb8ec069db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:20.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.668+0000 7fb8f101c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8ec06a750 con 0x7fb8ec06a2f0 2023-10-06T10:40:20.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.668+0000 7fb8f101c700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8ec06f020 con 0x7fb8ec155800 2023-10-06T10:40:20.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.668+0000 7fb8f101c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8ec1599c0 con 0x7fb8ec159530 2023-10-06T10:40:20.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.668+0000 7fb8e259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb8ec159530 0x7fb8ec069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:20.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.668+0000 7fb8eb59e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb8ec06a2f0 0x7fb8ec155bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54164/0 (socket says 172.21.15.92:54164) 2023-10-06T10:40:20.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.668+0000 7fb8eb59e700 1 -- 172.21.15.92:0/686727674 learned_addr learned my addr 172.21.15.92:0/686727674 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:20.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.669+0000 7fb8e259c700 1 -- 172.21.15.92:0/686727674 >> v1:172.21.15.124:6789/0 conn(0x7fb8ec155800 legacy=0x7fb8ec06b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:20.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.669+0000 7fb8e259c700 1 -- 172.21.15.92:0/686727674 >> v1:172.21.15.92:6789/0 conn(0x7fb8ec06a2f0 legacy=0x7fb8ec155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:20.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.669+0000 7fb8e259c700 1 -- 172.21.15.92:0/686727674 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ec159ca0 con 0x7fb8ec159530 2023-10-06T10:40:20.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.669+0000 7fb8e259c700 1 --2- 172.21.15.92:0/686727674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8ec159530 0x7fb8ec069db0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009910 tx=0x7fb8d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75ecd12c45b6de6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:20.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.669+0000 7fb8ea59c700 1 -- 172.21.15.92:0/686727674 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d400b7e0 con 0x7fb8ec159530 2023-10-06T10:40:20.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.669+0000 7fb8ea59c700 1 -- 172.21.15.92:0/686727674 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d400b940 con 0x7fb8ec159530 2023-10-06T10:40:20.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.669+0000 7fb8ea59c700 1 -- 172.21.15.92:0/686727674 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d400bce0 con 0x7fb8ec159530 2023-10-06T10:40:20.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.669+0000 7fb8f101c700 1 -- 172.21.15.92:0/686727674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8ec159530 msgr2=0x7fb8ec069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:20.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.669+0000 7fb8f101c700 1 --2- 172.21.15.92:0/686727674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8ec159530 0x7fb8ec069db0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009910 tx=0x7fb8d40091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.669+0000 7fb8f101c700 1 -- 172.21.15.92:0/686727674 shutdown_connections 2023-10-06T10:40:20.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.669+0000 7fb8f101c700 1 --2- 172.21.15.92:0/686727674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8ec159530 0x7fb8ec069db0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.669+0000 7fb8f101c700 1 -- 172.21.15.92:0/686727674 >> 172.21.15.92:0/686727674 conn(0x7fb8ec0cc0a0 msgr2=0x7fb8ec0cc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:20.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.669+0000 7fb8f101c700 1 -- 172.21.15.92:0/686727674 shutdown_connections 2023-10-06T10:40:20.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.669+0000 7fb8f101c700 1 -- 172.21.15.92:0/686727674 wait complete. 2023-10-06T10:40:20.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.671+0000 7fb8f101c700 1 Processor -- start 2023-10-06T10:40:20.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.671+0000 7fb8f101c700 1 -- start start 2023-10-06T10:40:20.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.671+0000 7fb8f101c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8ec06a2f0 0x7fb8ec14fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:20.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.671+0000 7fb8f101c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8ec155800 0x7fb8ec0b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:20.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.671+0000 7fb8f101c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8ec159530 0x7fb8ec157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:20.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.671+0000 7fb8f101c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec0b1350 con 0x7fb8ec155800 2023-10-06T10:40:20.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.671+0000 7fb8f101c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec0b1d20 con 0x7fb8ec06a2f0 2023-10-06T10:40:20.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.671+0000 7fb8f101c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec157fb0 con 0x7fb8ec159530 2023-10-06T10:40:20.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.671+0000 7fb8e259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8ec155800 0x7fb8ec0b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:20.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.671+0000 7fb8e259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8ec155800 0x7fb8ec0b17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58316/0 (socket says 172.21.15.92:58316) 2023-10-06T10:40:20.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.671+0000 7fb8e259c700 1 -- 172.21.15.92:0/1754413655 learned_addr learned my addr 172.21.15.92:0/1754413655 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:20.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.672+0000 7fb8eb59e700 1 --2- 172.21.15.92:0/1754413655 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8ec159530 0x7fb8ec157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:20.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.672+0000 7fb8ead9d700 1 --2- 172.21.15.92:0/1754413655 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8ec06a2f0 0x7fb8ec14fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:20.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.672+0000 7fb8e259c700 1 -- 172.21.15.92:0/1754413655 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8ec159530 msgr2=0x7fb8ec157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:20.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.672+0000 7fb8e259c700 1 --2- 172.21.15.92:0/1754413655 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8ec159530 0x7fb8ec157a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.672+0000 7fb8e259c700 1 -- 172.21.15.92:0/1754413655 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8ec06a2f0 msgr2=0x7fb8ec14fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:20.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.672+0000 7fb8e259c700 1 --2- 172.21.15.92:0/1754413655 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8ec06a2f0 0x7fb8ec14fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.672+0000 7fb8e259c700 1 -- 172.21.15.92:0/1754413655 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d4009420 con 0x7fb8ec155800 2023-10-06T10:40:20.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.672+0000 7fb8e259c700 1 --2- 172.21.15.92:0/1754413655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8ec155800 0x7fb8ec0b17e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40093f0 tx=0x7fb8d4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:20.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.672+0000 7fb8e37fe700 1 -- 172.21.15.92:0/1754413655 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d401b450 con 0x7fb8ec155800 2023-10-06T10:40:20.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.672+0000 7fb8e37fe700 1 -- 172.21.15.92:0/1754413655 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d4020070 con 0x7fb8ec155800 2023-10-06T10:40:20.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.672+0000 7fb8f101c700 1 -- 172.21.15.92:0/1754413655 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8ec0af2d0 con 0x7fb8ec155800 2023-10-06T10:40:20.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.672+0000 7fb8e37fe700 1 -- 172.21.15.92:0/1754413655 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d401b820 con 0x7fb8ec155800 2023-10-06T10:40:20.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.672+0000 7fb8f101c700 1 -- 172.21.15.92:0/1754413655 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ec0b0d00 con 0x7fb8ec155800 2023-10-06T10:40:20.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.673+0000 7fb8f101c700 1 -- 172.21.15.92:0/1754413655 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8ec0b0b60 con 0x7fb8ec155800 2023-10-06T10:40:20.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.673+0000 7fb8e37fe700 1 -- 172.21.15.92:0/1754413655 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb8d4022030 con 0x7fb8ec155800 2023-10-06T10:40:20.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.674+0000 7fb8e37fe700 1 --2- 172.21.15.92:0/1754413655 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb8cc066f30 0x7fb8cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:20.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.674+0000 7fb8e37fe700 1 -- 172.21.15.92:0/1754413655 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fb8d40867d0 con 0x7fb8ec155800 2023-10-06T10:40:20.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.674+0000 7fb8ead9d700 1 --2- 172.21.15.92:0/1754413655 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb8cc066f30 0x7fb8cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:20.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.674+0000 7fb8ead9d700 1 --2- 172.21.15.92:0/1754413655 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb8cc066f30 0x7fb8cc0693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc003870 tx=0x7fb8dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:20.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.678+0000 7fb8e37fe700 1 -- 172.21.15.92:0/1754413655 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8ec0b0b60 con 0x7fb8ec155800 2023-10-06T10:40:20.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.858+0000 7fb8f101c700 1 -- 172.21.15.92:0/1754413655 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb8ec0b0b60 con 0x7fb8ec155800 2023-10-06T10:40:20.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.858+0000 7fb8e37fe700 1 -- 172.21.15.92:0/1754413655 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb8ec0b0b60 con 0x7fb8ec155800 2023-10-06T10:40:20.860 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T10:40:20.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.860+0000 7fb8f101c700 1 -- 172.21.15.92:0/1754413655 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb8cc066f30 msgr2=0x7fb8cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:20.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.860+0000 7fb8f101c700 1 --2- 172.21.15.92:0/1754413655 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb8cc066f30 0x7fb8cc0693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc003870 tx=0x7fb8dc009040 comp rx=0 tx=0).stop 2023-10-06T10:40:20.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.860+0000 7fb8f101c700 1 -- 172.21.15.92:0/1754413655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8ec155800 msgr2=0x7fb8ec0b17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:20.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.860+0000 7fb8f101c700 1 --2- 172.21.15.92:0/1754413655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8ec155800 0x7fb8ec0b17e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40093f0 tx=0x7fb8d4016000 comp rx=0 tx=0).stop 2023-10-06T10:40:20.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.861+0000 7fb8f101c700 1 -- 172.21.15.92:0/1754413655 shutdown_connections 2023-10-06T10:40:20.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.861+0000 7fb8f101c700 1 --2- 172.21.15.92:0/1754413655 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb8cc066f30 0x7fb8cc0693e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.861+0000 7fb8f101c700 1 --2- 172.21.15.92:0/1754413655 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8ec159530 0x7fb8ec157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.861+0000 7fb8f101c700 1 --2- 172.21.15.92:0/1754413655 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8ec06a2f0 0x7fb8ec14fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.861+0000 7fb8f101c700 1 --2- 172.21.15.92:0/1754413655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8ec155800 0x7fb8ec0b17e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.861+0000 7fb8f101c700 1 -- 172.21.15.92:0/1754413655 >> 172.21.15.92:0/1754413655 conn(0x7fb8ec0cc0a0 msgr2=0x7fb8ec0ca5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:20.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.861+0000 7fb8f101c700 1 -- 172.21.15.92:0/1754413655 shutdown_connections 2023-10-06T10:40:20.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.861+0000 7fb8f101c700 1 -- 172.21.15.92:0/1754413655 wait complete. 2023-10-06T10:40:20.875 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T10:40:20.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.990+0000 7f0971938700 1 Processor -- start 2023-10-06T10:40:20.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.990+0000 7f0971938700 1 -- start start 2023-10-06T10:40:20.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f0971938700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0cd3c0 0x7f096c0ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:20.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f0971938700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f096c0cc970 con 0x7f096c0b2620 2023-10-06T10:40:20.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f0971938700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f096c0cbd60 con 0x7f096c0b29f0 2023-10-06T10:40:20.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f0971938700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096c0cc040 con 0x7f096c0cd3c0 2023-10-06T10:40:20.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f0970936700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f096c0b2620 0x7f096c0cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54176/0 (socket says 172.21.15.92:54176) 2023-10-06T10:40:20.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f0970936700 1 -- 172.21.15.92:0/2315462987 learned_addr learned my addr 172.21.15.92:0/2315462987 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:20.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f0963fff700 1 --2- 172.21.15.92:0/2315462987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0cd3c0 0x7f096c0ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:20.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f09637fe700 1 -- 172.21.15.92:0/2315462987 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3175602820 0 0) 0x7f096c0cc970 con 0x7f096c0b2620 2023-10-06T10:40:20.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f09637fe700 1 -- 172.21.15.92:0/2315462987 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0954003570 con 0x7f096c0b2620 2023-10-06T10:40:20.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f09637fe700 1 -- 172.21.15.92:0/2315462987 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2014223378 0 0) 0x7f096c0cbd60 con 0x7f096c0b29f0 2023-10-06T10:40:20.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f09637fe700 1 -- 172.21.15.92:0/2315462987 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f096c0cc970 con 0x7f096c0b29f0 2023-10-06T10:40:20.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f09637fe700 1 -- 172.21.15.92:0/2315462987 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1567447904 0 0) 0x7f0954003570 con 0x7f096c0b2620 2023-10-06T10:40:20.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f09637fe700 1 -- 172.21.15.92:0/2315462987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0cd3c0 msgr2=0x7f096c0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:20.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f09637fe700 1 --2- 172.21.15.92:0/2315462987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0cd3c0 0x7f096c0ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f09637fe700 1 -- 172.21.15.92:0/2315462987 >> v1:172.21.15.124:6789/0 conn(0x7f096c0b29f0 legacy=0x7f096c0c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:20.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f09637fe700 1 -- 172.21.15.92:0/2315462987 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c0cc320 con 0x7f096c0b2620 2023-10-06T10:40:20.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f09637fe700 1 -- 172.21.15.92:0/2315462987 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f0968001970 con 0x7f096c0b2620 2023-10-06T10:40:20.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f09637fe700 1 -- 172.21.15.92:0/2315462987 >> v1:172.21.15.92:6789/0 conn(0x7f096c0b2620 legacy=0x7f096c0cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:20.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f09637fe700 1 --2- 172.21.15.92:0/2315462987 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0954003c20 0x7f0954004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:20.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f09637fe700 1 --2- 172.21.15.92:0/2315462987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0954004810 0x7f0954004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:20.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f09637fe700 1 --2- 172.21.15.92:0/2315462987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0cd3c0 0x7f09540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:20.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f09637fe700 1 -- 172.21.15.92:0/2315462987 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c0cc040 con 0x7f0954004810 2023-10-06T10:40:20.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f09637fe700 1 -- 172.21.15.92:0/2315462987 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095400bb40 con 0x7f0954003c20 2023-10-06T10:40:20.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.991+0000 7f09637fe700 1 -- 172.21.15.92:0/2315462987 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095400bd00 con 0x7f096c0cd3c0 2023-10-06T10:40:20.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.992+0000 7f0963fff700 1 --2- 172.21.15.92:0/2315462987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0954004810 0x7f0954004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:20.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.992+0000 7f0970936700 1 --2- 172.21.15.92:0/2315462987 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0954003c20 0x7f0954004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:20.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.992+0000 7f0971137700 1 --2- 172.21.15.92:0/2315462987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0cd3c0 0x7f09540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:20.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.992+0000 7f0963fff700 1 -- 172.21.15.92:0/2315462987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0cd3c0 msgr2=0x7f09540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:20.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.992+0000 7f0963fff700 1 --2- 172.21.15.92:0/2315462987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0cd3c0 0x7f09540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.992+0000 7f0963fff700 1 -- 172.21.15.92:0/2315462987 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0954003c20 msgr2=0x7f0954004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:20.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.992+0000 7f0963fff700 1 --2- 172.21.15.92:0/2315462987 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0954003c20 0x7f0954004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:20.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.992+0000 7f0963fff700 1 -- 172.21.15.92:0/2315462987 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0954003570 con 0x7f0954004810 2023-10-06T10:40:20.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.992+0000 7f0963fff700 1 --2- 172.21.15.92:0/2315462987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0954004810 0x7f0954004c80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f095c009c00 tx=0x7f095c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aec7723a953f4ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:20.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.992+0000 7f09637fe700 1 -- 172.21.15.92:0/2315462987 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c00e710 con 0x7f0954004810 2023-10-06T10:40:20.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.992+0000 7f09637fe700 1 -- 172.21.15.92:0/2315462987 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095c00e870 con 0x7f0954004810 2023-10-06T10:40:20.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.992+0000 7f09637fe700 1 -- 172.21.15.92:0/2315462987 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c00ec30 con 0x7f0954004810 2023-10-06T10:40:20.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.993+0000 7f0971938700 1 -- 172.21.15.92:0/2315462987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0954004810 msgr2=0x7f0954004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:20.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.993+0000 7f0971938700 1 --2- 172.21.15.92:0/2315462987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0954004810 0x7f0954004c80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f095c009c00 tx=0x7f095c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.993+0000 7f0971137700 1 -- 172.21.15.92:0/2315462987 reap_dead start 2023-10-06T10:40:21.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.993+0000 7f0971938700 1 -- 172.21.15.92:0/2315462987 shutdown_connections 2023-10-06T10:40:21.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.993+0000 7f0971938700 1 -- 172.21.15.92:0/2315462987 >> 172.21.15.92:0/2315462987 conn(0x7f096c099520 msgr2=0x7f096c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:21.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.993+0000 7f0971938700 1 -- 172.21.15.92:0/2315462987 shutdown_connections 2023-10-06T10:40:21.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.993+0000 7f0971938700 1 -- 172.21.15.92:0/2315462987 wait complete. 2023-10-06T10:40:21.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.994+0000 7f0971938700 1 Processor -- start 2023-10-06T10:40:21.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.994+0000 7f0971938700 1 -- start start 2023-10-06T10:40:21.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0971938700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0954004810 0x7f096c15a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:21.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0971938700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0c8ec0 0x7f096c0c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:21.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0971938700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f096c15bb70 0x7f096c15c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:21.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0971938700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c09b9c0 con 0x7f096c15bb70 2023-10-06T10:40:21.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0971938700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c0c9870 con 0x7f0954004810 2023-10-06T10:40:21.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0971938700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096c15c8c0 con 0x7f096c0c8ec0 2023-10-06T10:40:21.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0971137700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f096c15bb70 0x7f096c15c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:21.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0971137700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f096c15bb70 0x7f096c15c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58330/0 (socket says 172.21.15.92:58330) 2023-10-06T10:40:21.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0971137700 1 -- 172.21.15.92:0/3745929871 learned_addr learned my addr 172.21.15.92:0/3745929871 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:21.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0970936700 1 --2- 172.21.15.92:0/3745929871 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0954004810 0x7f096c15a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:21.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0963fff700 1 --2- 172.21.15.92:0/3745929871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0c8ec0 0x7f096c0c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:21.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0971137700 1 -- 172.21.15.92:0/3745929871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0c8ec0 msgr2=0x7f096c0c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:21.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0971137700 1 --2- 172.21.15.92:0/3745929871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0c8ec0 0x7f096c0c9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0971137700 1 -- 172.21.15.92:0/3745929871 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0954004810 msgr2=0x7f096c15a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:21.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0971137700 1 --2- 172.21.15.92:0/3745929871 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0954004810 0x7f096c15a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0971137700 1 -- 172.21.15.92:0/3745929871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c15cba0 con 0x7f096c15bb70 2023-10-06T10:40:21.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0970936700 1 --2- 172.21.15.92:0/3745929871 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0954004810 0x7f096c15a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:21.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0963fff700 1 --2- 172.21.15.92:0/3745929871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0c8ec0 0x7f096c0c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:21.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0971137700 1 --2- 172.21.15.92:0/3745929871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f096c15bb70 0x7f096c15c010 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f096c0b2e10 tx=0x7f0964007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:21.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0960ff9700 1 -- 172.21.15.92:0/3745929871 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0964003440 con 0x7f096c15bb70 2023-10-06T10:40:21.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0960ff9700 1 -- 172.21.15.92:0/3745929871 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096400f8d0 con 0x7f096c15bb70 2023-10-06T10:40:21.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0960ff9700 1 -- 172.21.15.92:0/3745929871 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096400fc70 con 0x7f096c15bb70 2023-10-06T10:40:21.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0971938700 1 -- 172.21.15.92:0/3745929871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096c15ce80 con 0x7f096c15bb70 2023-10-06T10:40:21.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.995+0000 7f0971938700 1 -- 172.21.15.92:0/3745929871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096c15d440 con 0x7f096c15bb70 2023-10-06T10:40:21.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.996+0000 7f0971938700 1 -- 172.21.15.92:0/3745929871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096c05f230 con 0x7f096c15bb70 2023-10-06T10:40:21.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.997+0000 7f0960ff9700 1 -- 172.21.15.92:0/3745929871 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f096401e460 con 0x7f096c15bb70 2023-10-06T10:40:21.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.997+0000 7f0960ff9700 1 --2- 172.21.15.92:0/3745929871 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0958066f30 0x7f09580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:21.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.997+0000 7f0960ff9700 1 -- 172.21.15.92:0/3745929871 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f09640848a0 con 0x7f096c15bb70 2023-10-06T10:40:21.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.997+0000 7f0970936700 1 --2- 172.21.15.92:0/3745929871 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0958066f30 0x7f09580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:21.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:20.997+0000 7f0970936700 1 --2- 172.21.15.92:0/3745929871 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0958066f30 0x7f09580693e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0968001270 tx=0x7f0968009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:21.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.001+0000 7f0960ff9700 1 -- 172.21.15.92:0/3745929871 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0964051830 con 0x7f096c15bb70 2023-10-06T10:40:21.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.182+0000 7f0971938700 1 -- 172.21.15.92:0/3745929871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f096c0cc600 con 0x7f096c15bb70 2023-10-06T10:40:21.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.183+0000 7f0960ff9700 1 -- 172.21.15.92:0/3745929871 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0964054e50 con 0x7f096c15bb70 2023-10-06T10:40:21.185 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T10:40:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.185+0000 7f0971938700 1 -- 172.21.15.92:0/3745929871 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0958066f30 msgr2=0x7f09580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.185+0000 7f0971938700 1 --2- 172.21.15.92:0/3745929871 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0958066f30 0x7f09580693e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0968001270 tx=0x7f0968009040 comp rx=0 tx=0).stop 2023-10-06T10:40:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.185+0000 7f0971938700 1 -- 172.21.15.92:0/3745929871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f096c15bb70 msgr2=0x7f096c15c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.185+0000 7f0971938700 1 --2- 172.21.15.92:0/3745929871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f096c15bb70 0x7f096c15c010 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f096c0b2e10 tx=0x7f0964007d30 comp rx=0 tx=0).stop 2023-10-06T10:40:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.185+0000 7f0971938700 1 -- 172.21.15.92:0/3745929871 shutdown_connections 2023-10-06T10:40:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.185+0000 7f0971938700 1 --2- 172.21.15.92:0/3745929871 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0958066f30 0x7f09580693e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.185+0000 7f0971938700 1 --2- 172.21.15.92:0/3745929871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f096c15bb70 0x7f096c15c010 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.185+0000 7f0971938700 1 --2- 172.21.15.92:0/3745929871 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0954004810 0x7f096c15a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.185+0000 7f0971938700 1 --2- 172.21.15.92:0/3745929871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0c8ec0 0x7f096c0c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.185+0000 7f0971938700 1 -- 172.21.15.92:0/3745929871 >> 172.21.15.92:0/3745929871 conn(0x7f096c099520 msgr2=0x7f096c0cccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:21.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.185+0000 7f0971938700 1 -- 172.21.15.92:0/3745929871 shutdown_connections 2023-10-06T10:40:21.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.185+0000 7f0971938700 1 -- 172.21.15.92:0/3745929871 wait complete. 2023-10-06T10:40:21.198 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:40:21.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.310+0000 7f9246bb7700 1 Processor -- start 2023-10-06T10:40:21.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.310+0000 7f9246bb7700 1 -- start start 2023-10-06T10:40:21.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.310+0000 7f9246bb7700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92400b0760 0x7f92400ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:21.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.310+0000 7f9246bb7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92400b0c60 con 0x7f9240159560 2023-10-06T10:40:21.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.310+0000 7f9246bb7700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92400b0f40 con 0x7f92400b49a0 2023-10-06T10:40:21.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.310+0000 7f9246bb7700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92400ab350 con 0x7f92400b0760 2023-10-06T10:40:21.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.311+0000 7f9244953700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9240159560 0x7f92400aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54190/0 (socket says 172.21.15.92:54190) 2023-10-06T10:40:21.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.311+0000 7f9244953700 1 -- 172.21.15.92:0/4229796361 learned_addr learned my addr 172.21.15.92:0/4229796361 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:21.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.311+0000 7f9245154700 1 --2- 172.21.15.92:0/4229796361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92400b0760 0x7f92400ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:21.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.311+0000 7f92377fe700 1 -- 172.21.15.92:0/4229796361 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2399440427 0 0) 0x7f92400b0c60 con 0x7f9240159560 2023-10-06T10:40:21.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.311+0000 7f92377fe700 1 -- 172.21.15.92:0/4229796361 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9228003570 con 0x7f9240159560 2023-10-06T10:40:21.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.311+0000 7f9245154700 1 -- 172.21.15.92:0/4229796361 >> v1:172.21.15.124:6789/0 conn(0x7f92400b49a0 legacy=0x7f92400aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:21.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.311+0000 7f92377fe700 1 -- 172.21.15.92:0/4229796361 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2692161857 0 0) 0x7f92400b0f40 con 0x7f92400b49a0 2023-10-06T10:40:21.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.311+0000 7f9245154700 1 -- 172.21.15.92:0/4229796361 >> v1:172.21.15.92:6789/0 conn(0x7f9240159560 legacy=0x7f92400aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:21.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.311+0000 7f9245154700 1 -- 172.21.15.92:0/4229796361 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92400b1220 con 0x7f92400b0760 2023-10-06T10:40:21.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.311+0000 7f9245154700 1 --2- 172.21.15.92:0/4229796361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92400b0760 0x7f92400ccf90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9238009940 tx=0x7f92380091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d173cbc4ccd6a13 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:21.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.311+0000 7f92377fe700 1 -- 172.21.15.92:0/4229796361 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923800b670 con 0x7f92400b0760 2023-10-06T10:40:21.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.311+0000 7f92377fe700 1 -- 172.21.15.92:0/4229796361 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923800b7d0 con 0x7f92400b0760 2023-10-06T10:40:21.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.311+0000 7f92377fe700 1 -- 172.21.15.92:0/4229796361 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923800bb50 con 0x7f92400b0760 2023-10-06T10:40:21.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.312+0000 7f9246bb7700 1 -- 172.21.15.92:0/4229796361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92400b0760 msgr2=0x7f92400ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:21.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.312+0000 7f9246bb7700 1 --2- 172.21.15.92:0/4229796361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92400b0760 0x7f92400ccf90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9238009940 tx=0x7f92380091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.312+0000 7f9246bb7700 1 -- 172.21.15.92:0/4229796361 shutdown_connections 2023-10-06T10:40:21.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.312+0000 7f9246bb7700 1 --2- 172.21.15.92:0/4229796361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92400b0760 0x7f92400ccf90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.312+0000 7f9246bb7700 1 -- 172.21.15.92:0/4229796361 >> 172.21.15.92:0/4229796361 conn(0x7f92400997f0 msgr2=0x7f924006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:21.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.312+0000 7f9246bb7700 1 -- 172.21.15.92:0/4229796361 shutdown_connections 2023-10-06T10:40:21.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.312+0000 7f9246bb7700 1 -- 172.21.15.92:0/4229796361 wait complete. 2023-10-06T10:40:21.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.313+0000 7f9246bb7700 1 Processor -- start 2023-10-06T10:40:21.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.313+0000 7f9246bb7700 1 -- start start 2023-10-06T10:40:21.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.314+0000 7f9246bb7700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f92400b0760 0x7f924009c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:21.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.314+0000 7f9246bb7700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92400b49a0 0x7f924015b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:21.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.314+0000 7f9246bb7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9240159560 0x7f924016c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:21.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.314+0000 7f9246bb7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92401584a0 con 0x7f9240159560 2023-10-06T10:40:21.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.314+0000 7f9246bb7700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9240169c80 con 0x7f92400b0760 2023-10-06T10:40:21.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.314+0000 7f9246bb7700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924015b820 con 0x7f92400b49a0 2023-10-06T10:40:21.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.314+0000 7f9245154700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9240159560 0x7f924016c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:21.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.314+0000 7f9245154700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9240159560 0x7f924016c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58336/0 (socket says 172.21.15.92:58336) 2023-10-06T10:40:21.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.314+0000 7f9245154700 1 -- 172.21.15.92:0/322901630 learned_addr learned my addr 172.21.15.92:0/322901630 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:21.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.314+0000 7f9237fff700 1 --2- 172.21.15.92:0/322901630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92400b49a0 0x7f924015b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:21.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.314+0000 7f9244953700 1 --2- 172.21.15.92:0/322901630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f92400b0760 0x7f924009c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:21.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.314+0000 7f9245154700 1 -- 172.21.15.92:0/322901630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92400b49a0 msgr2=0x7f924015b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:21.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.314+0000 7f9245154700 1 --2- 172.21.15.92:0/322901630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92400b49a0 0x7f924015b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.314+0000 7f9245154700 1 -- 172.21.15.92:0/322901630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f92400b0760 msgr2=0x7f924009c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:21.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.314+0000 7f9245154700 1 --2- 172.21.15.92:0/322901630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f92400b0760 0x7f924009c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.314+0000 7f9245154700 1 -- 172.21.15.92:0/322901630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9238009420 con 0x7f9240159560 2023-10-06T10:40:21.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.315+0000 7f9245154700 1 --2- 172.21.15.92:0/322901630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9240159560 0x7f924016c940 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9238013000 tx=0x7f9238017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:21.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.315+0000 7f9234ff9700 1 -- 172.21.15.92:0/322901630 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923801f070 con 0x7f9240159560 2023-10-06T10:40:21.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.315+0000 7f9234ff9700 1 -- 172.21.15.92:0/322901630 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9238011420 con 0x7f9240159560 2023-10-06T10:40:21.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.315+0000 7f9246bb7700 1 -- 172.21.15.92:0/322901630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f924015bab0 con 0x7f9240159560 2023-10-06T10:40:21.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.315+0000 7f9234ff9700 1 -- 172.21.15.92:0/322901630 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923800bb50 con 0x7f9240159560 2023-10-06T10:40:21.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.315+0000 7f9246bb7700 1 -- 172.21.15.92:0/322901630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f924015c070 con 0x7f9240159560 2023-10-06T10:40:21.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.316+0000 7f9246bb7700 1 -- 172.21.15.92:0/322901630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9240064a90 con 0x7f9240159560 2023-10-06T10:40:21.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.316+0000 7f9234ff9700 1 -- 172.21.15.92:0/322901630 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f92380119f0 con 0x7f9240159560 2023-10-06T10:40:21.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.316+0000 7f9234ff9700 1 --2- 172.21.15.92:0/322901630 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f922c066f30 0x7f922c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:21.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.316+0000 7f9234ff9700 1 -- 172.21.15.92:0/322901630 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f9238085da0 con 0x7f9240159560 2023-10-06T10:40:21.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.316+0000 7f9244953700 1 --2- 172.21.15.92:0/322901630 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f922c066f30 0x7f922c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:21.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.317+0000 7f9244953700 1 --2- 172.21.15.92:0/322901630 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f922c066f30 0x7f922c0693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f923c001460 tx=0x7f923c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:21.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.321+0000 7f9234ff9700 1 -- 172.21.15.92:0/322901630 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9238052d30 con 0x7f9240159560 2023-10-06T10:40:21.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.546+0000 7f9246bb7700 1 -- 172.21.15.92:0/322901630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92400b61d0 con 0x7f9240159560 2023-10-06T10:40:21.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.547+0000 7f9234ff9700 1 -- 172.21.15.92:0/322901630 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4634 (secure 0 0 0) 0x7f9238056350 con 0x7f9240159560 2023-10-06T10:40:21.549 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:21.549 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2525196781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2525196781},{"type":"v1","addr":"172.21.15.124:6835","nonce":2525196781}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:39:15.675013+0000","modified":"2023-10-06T10:39:48.393721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5133,"mds_1":5121,"mds_2":5118},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5118":{"gid":5118,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6836/2409440810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2409440810},{"type":"v1","addr":"172.21.15.92:6836","nonce":2409440810}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5121":{"gid":5121,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/4010629505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4010629505},{"type":"v1","addr":"172.21.15.92:6837","nonce":4010629505}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5133":{"gid":5133,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.124:6834/484869337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":484869337},{"type":"v1","addr":"172.21.15.124:6834","nonce":484869337}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T10:40:21.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.549+0000 7f9246bb7700 1 -- 172.21.15.92:0/322901630 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f922c066f30 msgr2=0x7f922c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:21.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.549+0000 7f9246bb7700 1 --2- 172.21.15.92:0/322901630 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f922c066f30 0x7f922c0693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f923c001460 tx=0x7f923c009040 comp rx=0 tx=0).stop 2023-10-06T10:40:21.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.549+0000 7f9246bb7700 1 -- 172.21.15.92:0/322901630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9240159560 msgr2=0x7f924016c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:21.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.549+0000 7f9246bb7700 1 --2- 172.21.15.92:0/322901630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9240159560 0x7f924016c940 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9238013000 tx=0x7f9238017000 comp rx=0 tx=0).stop 2023-10-06T10:40:21.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.549+0000 7f9246bb7700 1 -- 172.21.15.92:0/322901630 shutdown_connections 2023-10-06T10:40:21.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.549+0000 7f9246bb7700 1 --2- 172.21.15.92:0/322901630 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f922c066f30 0x7f922c0693e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.549+0000 7f9246bb7700 1 --2- 172.21.15.92:0/322901630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9240159560 0x7f924016c940 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.549+0000 7f9246bb7700 1 --2- 172.21.15.92:0/322901630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f92400b0760 0x7f924009c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.549+0000 7f9246bb7700 1 --2- 172.21.15.92:0/322901630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92400b49a0 0x7f924015b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.549+0000 7f9246bb7700 1 -- 172.21.15.92:0/322901630 >> 172.21.15.92:0/322901630 conn(0x7f92400997f0 msgr2=0x7f92400afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:21.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.549+0000 7f9246bb7700 1 -- 172.21.15.92:0/322901630 shutdown_connections 2023-10-06T10:40:21.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.549+0000 7f9246bb7700 1 -- 172.21.15.92:0/322901630 wait complete. 2023-10-06T10:40:21.552 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 24 2023-10-06T10:40:21.562 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:40:21.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.674+0000 7f105001b700 1 Processor -- start 2023-10-06T10:40:21.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.674+0000 7f105001b700 1 -- start start 2023-10-06T10:40:21.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.675+0000 7f105001b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1048159560 0x7f10480aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:21.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.675+0000 7f105001b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10480b0760 0x7f10480b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:21.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.675+0000 7f105001b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10480b0c60 con 0x7f10480b49a0 2023-10-06T10:40:21.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.675+0000 7f105001b700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10480b0f40 con 0x7f1048159560 2023-10-06T10:40:21.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.675+0000 7f105001b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10480b1220 con 0x7f10480b0760 2023-10-06T10:40:21.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.675+0000 7f104d5b6700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f10480b49a0 0x7f10480aaa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54202/0 (socket says 172.21.15.92:54202) 2023-10-06T10:40:21.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.675+0000 7f104d5b6700 1 -- 172.21.15.92:0/977042315 learned_addr learned my addr 172.21.15.92:0/977042315 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:21.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.675+0000 7f104ddb7700 1 --2- 172.21.15.92:0/977042315 >> v2:172.21.15.124:3300/0 conn(0x7f1048159560 0x7f10480aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:21.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.675+0000 7f104e5b8700 1 --2- 172.21.15.92:0/977042315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10480b0760 0x7f10480b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:21.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.675+0000 7f104cdb5700 1 -- 172.21.15.92:0/977042315 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491791930 0 0) 0x7f10480b0c60 con 0x7f10480b49a0 2023-10-06T10:40:21.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.675+0000 7f104cdb5700 1 -- 172.21.15.92:0/977042315 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1034003570 con 0x7f10480b49a0 2023-10-06T10:40:21.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104cdb5700 1 -- 172.21.15.92:0/977042315 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2931411086 0 0) 0x7f1034003570 con 0x7f10480b49a0 2023-10-06T10:40:21.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104cdb5700 1 -- 172.21.15.92:0/977042315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10480b0760 msgr2=0x7f10480b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:21.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104cdb5700 1 --2- 172.21.15.92:0/977042315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10480b0760 0x7f10480b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104cdb5700 1 -- 172.21.15.92:0/977042315 >> v2:172.21.15.124:3300/0 conn(0x7f1048159560 msgr2=0x7f10480aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:21.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104cdb5700 1 --2- 172.21.15.92:0/977042315 >> v2:172.21.15.124:3300/0 conn(0x7f1048159560 0x7f10480aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104cdb5700 1 -- 172.21.15.92:0/977042315 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10480b1500 con 0x7f10480b49a0 2023-10-06T10:40:21.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104cdb5700 1 -- 172.21.15.92:0/977042315 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f1038001470 con 0x7f10480b49a0 2023-10-06T10:40:21.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104cdb5700 1 -- 172.21.15.92:0/977042315 >> v1:172.21.15.92:6789/0 conn(0x7f10480b49a0 legacy=0x7f10480aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:21.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104cdb5700 1 --2- 172.21.15.92:0/977042315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1034003a60 0x7f1034003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:21.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104cdb5700 1 --2- 172.21.15.92:0/977042315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1034004810 0x7f1034004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:21.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104cdb5700 1 --2- 172.21.15.92:0/977042315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10480b0760 0x7f10340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:21.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104cdb5700 1 -- 172.21.15.92:0/977042315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10480b0f40 con 0x7f1034003a60 2023-10-06T10:40:21.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104cdb5700 1 -- 172.21.15.92:0/977042315 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10480b1220 con 0x7f1034004810 2023-10-06T10:40:21.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104cdb5700 1 -- 172.21.15.92:0/977042315 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103400ba60 con 0x7f10480b0760 2023-10-06T10:40:21.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104ddb7700 1 --2- 172.21.15.92:0/977042315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1034003a60 0x7f1034003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:21.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104d5b6700 1 --2- 172.21.15.92:0/977042315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1034004810 0x7f1034004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:21.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104e5b8700 1 --2- 172.21.15.92:0/977042315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10480b0760 0x7f10340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:21.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104ddb7700 1 -- 172.21.15.92:0/977042315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10480b0760 msgr2=0x7f10340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:21.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104ddb7700 1 --2- 172.21.15.92:0/977042315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10480b0760 0x7f10340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104ddb7700 1 -- 172.21.15.92:0/977042315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1034004810 msgr2=0x7f1034004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:21.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104ddb7700 1 --2- 172.21.15.92:0/977042315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1034004810 0x7f1034004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104ddb7700 1 -- 172.21.15.92:0/977042315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1034003570 con 0x7f1034003a60 2023-10-06T10:40:21.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104ddb7700 1 --2- 172.21.15.92:0/977042315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1034003a60 0x7f1034003e70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1044009c20 tx=0x7f1044009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1a565597cd7604 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:21.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.676+0000 7f104cdb5700 1 -- 172.21.15.92:0/977042315 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1044005a00 con 0x7f1034003a60 2023-10-06T10:40:21.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.677+0000 7f104cdb5700 1 -- 172.21.15.92:0/977042315 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1044005b60 con 0x7f1034003a60 2023-10-06T10:40:21.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.677+0000 7f104cdb5700 1 -- 172.21.15.92:0/977042315 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104400e710 con 0x7f1034003a60 2023-10-06T10:40:21.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.677+0000 7f105001b700 1 -- 172.21.15.92:0/977042315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1034003a60 msgr2=0x7f1034003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:21.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.677+0000 7f105001b700 1 --2- 172.21.15.92:0/977042315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1034003a60 0x7f1034003e70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1044009c20 tx=0x7f1044009120 comp rx=0 tx=0).stop 2023-10-06T10:40:21.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.677+0000 7f104e5b8700 1 -- 172.21.15.92:0/977042315 reap_dead start 2023-10-06T10:40:21.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.677+0000 7f105001b700 1 -- 172.21.15.92:0/977042315 shutdown_connections 2023-10-06T10:40:21.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.677+0000 7f105001b700 1 -- 172.21.15.92:0/977042315 >> 172.21.15.92:0/977042315 conn(0x7f10480997f0 msgr2=0x7f104806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:21.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.677+0000 7f105001b700 1 -- 172.21.15.92:0/977042315 shutdown_connections 2023-10-06T10:40:21.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.677+0000 7f105001b700 1 -- 172.21.15.92:0/977042315 wait complete. 2023-10-06T10:40:21.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.678+0000 7f105001b700 1 Processor -- start 2023-10-06T10:40:21.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.678+0000 7f105001b700 1 -- start start 2023-10-06T10:40:21.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.679+0000 7f105001b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1034003a60 0x7f104809ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:21.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.679+0000 7f105001b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f104816aa30 0x7f10480696f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:21.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.679+0000 7f105001b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1048069c30 0x7f10480ab200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:21.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.679+0000 7f105001b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104809b180 con 0x7f1034003a60 2023-10-06T10:40:21.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.679+0000 7f105001b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104806a0a0 con 0x7f104816aa30 2023-10-06T10:40:21.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.679+0000 7f105001b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f104816a5a0 con 0x7f1048069c30 2023-10-06T10:40:21.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.679+0000 7f104ddb7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1034003a60 0x7f104809ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:21.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.679+0000 7f104ddb7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1034003a60 0x7f104809ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58348/0 (socket says 172.21.15.92:58348) 2023-10-06T10:40:21.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.679+0000 7f104ddb7700 1 -- 172.21.15.92:0/3581364764 learned_addr learned my addr 172.21.15.92:0/3581364764 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:21.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.679+0000 7f104d5b6700 1 --2- 172.21.15.92:0/3581364764 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f104816aa30 0x7f10480696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:21.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.679+0000 7f104e5b8700 1 --2- 172.21.15.92:0/3581364764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1048069c30 0x7f10480ab200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:21.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.679+0000 7f104ddb7700 1 -- 172.21.15.92:0/3581364764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1048069c30 msgr2=0x7f10480ab200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:21.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.679+0000 7f104ddb7700 1 --2- 172.21.15.92:0/3581364764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1048069c30 0x7f10480ab200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.679+0000 7f104ddb7700 1 -- 172.21.15.92:0/3581364764 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f104816aa30 msgr2=0x7f10480696f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:21.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.679+0000 7f104ddb7700 1 --2- 172.21.15.92:0/3581364764 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f104816aa30 0x7f10480696f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.679+0000 7f104ddb7700 1 -- 172.21.15.92:0/3581364764 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10480abe50 con 0x7f1034003a60 2023-10-06T10:40:21.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.680+0000 7f104ddb7700 1 --2- 172.21.15.92:0/3581364764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1034003a60 0x7f104809ac40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f104400e4b0 tx=0x7f1044019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.680+0000 7f103dffb700 1 -- 172.21.15.92:0/3581364764 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104400e710 con 0x7f1034003a60 2023-10-06T10:40:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.680+0000 7f103dffb700 1 -- 172.21.15.92:0/3581364764 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1044006240 con 0x7f1034003a60 2023-10-06T10:40:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.680+0000 7f103dffb700 1 -- 172.21.15.92:0/3581364764 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10440065e0 con 0x7f1034003a60 2023-10-06T10:40:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.680+0000 7f105001b700 1 -- 172.21.15.92:0/3581364764 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10480ac130 con 0x7f1034003a60 2023-10-06T10:40:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.680+0000 7f105001b700 1 -- 172.21.15.92:0/3581364764 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f104816a230 con 0x7f1034003a60 2023-10-06T10:40:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.682+0000 7f105001b700 1 -- 172.21.15.92:0/3581364764 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1048064a90 con 0x7f1034003a60 2023-10-06T10:40:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.682+0000 7f103dffb700 1 -- 172.21.15.92:0/3581364764 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1044013470 con 0x7f1034003a60 2023-10-06T10:40:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.682+0000 7f103dffb700 1 --2- 172.21.15.92:0/3581364764 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f102c066f30 0x7f102c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.682+0000 7f103dffb700 1 -- 172.21.15.92:0/3581364764 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f1044085d70 con 0x7f1034003a60 2023-10-06T10:40:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.682+0000 7f104d5b6700 1 --2- 172.21.15.92:0/3581364764 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f102c066f30 0x7f102c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.682+0000 7f104d5b6700 1 --2- 172.21.15.92:0/3581364764 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f102c066f30 0x7f102c0693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1038001050 tx=0x7f1038001e50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.687+0000 7f103dffb700 1 -- 172.21.15.92:0/3581364764 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1044052d00 con 0x7f1034003a60 2023-10-06T10:40:21.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.870+0000 7f105001b700 1 -- 172.21.15.92:0/3581364764 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10480b17e0 con 0x7f1034003a60 2023-10-06T10:40:21.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.871+0000 7f103dffb700 1 -- 172.21.15.92:0/3581364764 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+17166 (secure 0 0 0) 0x7f1044056320 con 0x7f1034003a60 2023-10-06T10:40:21.872 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:21.872 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":40,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:40:06.432393+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:39:13.780983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T10:39:14.644706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":4}]},{"pgid":"7.2a","mappings":[{"from":0,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:40:21.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.873+0000 7f105001b700 1 -- 172.21.15.92:0/3581364764 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f102c066f30 msgr2=0x7f102c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:21.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.873+0000 7f105001b700 1 --2- 172.21.15.92:0/3581364764 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f102c066f30 0x7f102c0693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1038001050 tx=0x7f1038001e50 comp rx=0 tx=0).stop 2023-10-06T10:40:21.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.873+0000 7f105001b700 1 -- 172.21.15.92:0/3581364764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1034003a60 msgr2=0x7f104809ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:21.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.873+0000 7f105001b700 1 --2- 172.21.15.92:0/3581364764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1034003a60 0x7f104809ac40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f104400e4b0 tx=0x7f1044019000 comp rx=0 tx=0).stop 2023-10-06T10:40:21.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.873+0000 7f105001b700 1 -- 172.21.15.92:0/3581364764 shutdown_connections 2023-10-06T10:40:21.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.873+0000 7f105001b700 1 --2- 172.21.15.92:0/3581364764 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f102c066f30 0x7f102c0693e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.873+0000 7f105001b700 1 --2- 172.21.15.92:0/3581364764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1048069c30 0x7f10480ab200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.873+0000 7f105001b700 1 --2- 172.21.15.92:0/3581364764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1034003a60 0x7f104809ac40 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.873+0000 7f105001b700 1 --2- 172.21.15.92:0/3581364764 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f104816aa30 0x7f10480696f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:21.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.873+0000 7f105001b700 1 -- 172.21.15.92:0/3581364764 >> 172.21.15.92:0/3581364764 conn(0x7f10480997f0 msgr2=0x7f1048158960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:21.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.873+0000 7f105001b700 1 -- 172.21.15.92:0/3581364764 shutdown_connections 2023-10-06T10:40:21.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.873+0000 7f105001b700 1 -- 172.21.15.92:0/3581364764 wait complete. 2023-10-06T10:40:21.886 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T10:40:21.886 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:40:22.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.999+0000 7ff7b191f700 1 Processor -- start 2023-10-06T10:40:22.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:21.999+0000 7ff7b191f700 1 -- start start 2023-10-06T10:40:22.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.000+0000 7ff7b191f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff7ac15a870 0x7ff7ac0ce980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:22.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.000+0000 7ff7b191f700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff7ac15ac40 0x7ff7ac0c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:22.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.000+0000 7ff7b191f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7ac0cdb40 con 0x7ff7ac0ceec0 2023-10-06T10:40:22.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.000+0000 7ff7b191f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7ac0ccf30 con 0x7ff7ac15a870 2023-10-06T10:40:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.000+0000 7ff7b191f700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7ac0cd210 con 0x7ff7ac15ac40 2023-10-06T10:40:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.000+0000 7ff7aaffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff7ac15a870 0x7ff7ac0ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.000+0000 7ff7ab7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff7ac0ceec0 0x7ff7ac0cbda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54208/0 (socket says 172.21.15.92:54208) 2023-10-06T10:40:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.000+0000 7ff7ab7fe700 1 -- 172.21.15.92:0/304674041 learned_addr learned my addr 172.21.15.92:0/304674041 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.000+0000 7ff7aa7fc700 1 --2- 172.21.15.92:0/304674041 >> v2:172.21.15.124:3300/0 conn(0x7ff7ac15ac40 0x7ff7ac0c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.000+0000 7ff7a9ffb700 1 -- 172.21.15.92:0/304674041 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1298896869 0 0) 0x7ff7ac0cdb40 con 0x7ff7ac0ceec0 2023-10-06T10:40:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.000+0000 7ff7a9ffb700 1 -- 172.21.15.92:0/304674041 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff794003570 con 0x7ff7ac0ceec0 2023-10-06T10:40:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.000+0000 7ff7aaffd700 1 -- 172.21.15.92:0/304674041 >> v2:172.21.15.124:3300/0 conn(0x7ff7ac15ac40 msgr2=0x7ff7ac0c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.000+0000 7ff7aaffd700 1 --2- 172.21.15.92:0/304674041 >> v2:172.21.15.124:3300/0 conn(0x7ff7ac15ac40 0x7ff7ac0c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.000+0000 7ff7aaffd700 1 -- 172.21.15.92:0/304674041 >> v1:172.21.15.92:6789/0 conn(0x7ff7ac0ceec0 legacy=0x7ff7ac0cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.000+0000 7ff7aaffd700 1 -- 172.21.15.92:0/304674041 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac0cd4f0 con 0x7ff7ac15a870 2023-10-06T10:40:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.000+0000 7ff7aa7fc700 1 --2- 172.21.15.92:0/304674041 >> v2:172.21.15.124:3300/0 conn(0x7ff7ac15ac40 0x7ff7ac0c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.000+0000 7ff7aaffd700 1 --2- 172.21.15.92:0/304674041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac15a870 0x7ff7ac0ce980 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac06f410 tx=0x7ff79c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e24b1cd5dbf1a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.000+0000 7ff7a9ffb700 1 -- 172.21.15.92:0/304674041 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c005c10 con 0x7ff7ac15a870 2023-10-06T10:40:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.000+0000 7ff7a9ffb700 1 -- 172.21.15.92:0/304674041 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79c005d70 con 0x7ff7ac15a870 2023-10-06T10:40:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.000+0000 7ff7a9ffb700 1 -- 172.21.15.92:0/304674041 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c006260 con 0x7ff7ac15a870 2023-10-06T10:40:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.001+0000 7ff7b191f700 1 -- 172.21.15.92:0/304674041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac15a870 msgr2=0x7ff7ac0ce980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.001+0000 7ff7b191f700 1 --2- 172.21.15.92:0/304674041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac15a870 0x7ff7ac0ce980 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac06f410 tx=0x7ff79c009120 comp rx=0 tx=0).stop 2023-10-06T10:40:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.001+0000 7ff7b191f700 1 -- 172.21.15.92:0/304674041 shutdown_connections 2023-10-06T10:40:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.001+0000 7ff7b191f700 1 --2- 172.21.15.92:0/304674041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac15a870 0x7ff7ac0ce980 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.001+0000 7ff7b191f700 1 --2- 172.21.15.92:0/304674041 >> v2:172.21.15.124:3300/0 conn(0x7ff7ac15ac40 0x7ff7ac0c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.001+0000 7ff7b191f700 1 -- 172.21.15.92:0/304674041 >> 172.21.15.92:0/304674041 conn(0x7ff7ac067d90 msgr2=0x7ff7ac0ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.001+0000 7ff7b191f700 1 -- 172.21.15.92:0/304674041 shutdown_connections 2023-10-06T10:40:22.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.001+0000 7ff7b191f700 1 -- 172.21.15.92:0/304674041 wait complete. 2023-10-06T10:40:22.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.002+0000 7ff7b191f700 1 Processor -- start 2023-10-06T10:40:22.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.002+0000 7ff7b191f700 1 -- start start 2023-10-06T10:40:22.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.003+0000 7ff7b191f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7ac0ceec0 0x7ff7ac169880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:22.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.003+0000 7ff7b191f700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7ac15a870 0x7ff7ac0ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:22.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.003+0000 7ff7b191f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac15ac40 0x7ff7ac155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:22.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.003+0000 7ff7b191f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac0ae230 con 0x7ff7ac15ac40 2023-10-06T10:40:22.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.003+0000 7ff7b191f700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac0b37e0 con 0x7ff7ac15a870 2023-10-06T10:40:22.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.003+0000 7ff7b191f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac0b3ac0 con 0x7ff7ac0ceec0 2023-10-06T10:40:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.003+0000 7ff7ab7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac15ac40 0x7ff7ac155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.003+0000 7ff7ab7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac15ac40 0x7ff7ac155950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58360/0 (socket says 172.21.15.92:58360) 2023-10-06T10:40:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.003+0000 7ff7ab7fe700 1 -- 172.21.15.92:0/1245944107 learned_addr learned my addr 172.21.15.92:0/1245944107 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.003+0000 7ff7ab7fe700 1 -- 172.21.15.92:0/1245944107 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7ac0ceec0 msgr2=0x7ff7ac169880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.003+0000 7ff7ab7fe700 1 --2- 172.21.15.92:0/1245944107 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7ac0ceec0 0x7ff7ac169880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.003+0000 7ff7ab7fe700 1 -- 172.21.15.92:0/1245944107 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7ac15a870 msgr2=0x7ff7ac0ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.003+0000 7ff7ab7fe700 1 --2- 172.21.15.92:0/1245944107 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7ac15a870 0x7ff7ac0ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.003+0000 7ff7ab7fe700 1 -- 172.21.15.92:0/1245944107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79c0093c0 con 0x7ff7ac15ac40 2023-10-06T10:40:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.003+0000 7ff7ab7fe700 1 --2- 172.21.15.92:0/1245944107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac15ac40 0x7ff7ac155950 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4001460 tx=0x7ff7a4001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.004+0000 7ff792ffd700 1 -- 172.21.15.92:0/1245944107 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a400ce80 con 0x7ff7ac15ac40 2023-10-06T10:40:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.004+0000 7ff792ffd700 1 -- 172.21.15.92:0/1245944107 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a400ab70 con 0x7ff7ac15ac40 2023-10-06T10:40:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.004+0000 7ff792ffd700 1 -- 172.21.15.92:0/1245944107 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a40070f0 con 0x7ff7ac15ac40 2023-10-06T10:40:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.004+0000 7ff7b191f700 1 -- 172.21.15.92:0/1245944107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ac0b3da0 con 0x7ff7ac15ac40 2023-10-06T10:40:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.004+0000 7ff7b191f700 1 -- 172.21.15.92:0/1245944107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ac0b3490 con 0x7ff7ac15ac40 2023-10-06T10:40:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.005+0000 7ff7b191f700 1 -- 172.21.15.92:0/1245944107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7ac0ae710 con 0x7ff7ac15ac40 2023-10-06T10:40:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.005+0000 7ff792ffd700 1 -- 172.21.15.92:0/1245944107 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff7a4007520 con 0x7ff7ac15ac40 2023-10-06T10:40:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.005+0000 7ff792ffd700 1 --2- 172.21.15.92:0/1245944107 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff798066f30 0x7ff7980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.005+0000 7ff792ffd700 1 -- 172.21.15.92:0/1245944107 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6436+0+0 (secure 0 0 0) 0x7ff7a4084d30 con 0x7ff7ac15ac40 2023-10-06T10:40:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.005+0000 7ff7aaffd700 1 --2- 172.21.15.92:0/1245944107 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff798066f30 0x7ff7980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.006+0000 7ff7aaffd700 1 --2- 172.21.15.92:0/1245944107 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff798066f30 0x7ff7980693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff79c009160 tx=0x7ff79c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:22.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.009+0000 7ff792ffd700 1 -- 172.21.15.92:0/1245944107 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7ac0ae710 con 0x7ff7ac15ac40 2023-10-06T10:40:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.235+0000 7ff7b191f700 1 -- 172.21.15.92:0/1245944107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7ac0ae710 con 0x7ff7ac15ac40 2023-10-06T10:40:22.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.235+0000 7ff792ffd700 1 -- 172.21.15.92:0/1245944107 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4634 (secure 0 0 0) 0x7ff7ac0ae710 con 0x7ff7ac15ac40 2023-10-06T10:40:22.236 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:22.236 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2525196781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2525196781},{"type":"v1","addr":"172.21.15.124:6835","nonce":2525196781}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:39:15.675013+0000","modified":"2023-10-06T10:39:48.393721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5133,"mds_1":5121,"mds_2":5118},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5118":{"gid":5118,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6836/2409440810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2409440810},{"type":"v1","addr":"172.21.15.92:6836","nonce":2409440810}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5121":{"gid":5121,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/4010629505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4010629505},{"type":"v1","addr":"172.21.15.92:6837","nonce":4010629505}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5133":{"gid":5133,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.124:6834/484869337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":484869337},{"type":"v1","addr":"172.21.15.124:6834","nonce":484869337}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T10:40:22.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.237+0000 7ff7b191f700 1 -- 172.21.15.92:0/1245944107 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff798066f30 msgr2=0x7ff7980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:22.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.237+0000 7ff7b191f700 1 --2- 172.21.15.92:0/1245944107 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff798066f30 0x7ff7980693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff79c009160 tx=0x7ff79c016000 comp rx=0 tx=0).stop 2023-10-06T10:40:22.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.237+0000 7ff7b191f700 1 -- 172.21.15.92:0/1245944107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac15ac40 msgr2=0x7ff7ac155950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:22.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.237+0000 7ff7b191f700 1 --2- 172.21.15.92:0/1245944107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac15ac40 0x7ff7ac155950 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4001460 tx=0x7ff7a4001ae0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.237+0000 7ff7b191f700 1 -- 172.21.15.92:0/1245944107 shutdown_connections 2023-10-06T10:40:22.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.237+0000 7ff7b191f700 1 --2- 172.21.15.92:0/1245944107 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff798066f30 0x7ff7980693e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.237+0000 7ff7b191f700 1 --2- 172.21.15.92:0/1245944107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac15ac40 0x7ff7ac155950 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.237+0000 7ff7b191f700 1 --2- 172.21.15.92:0/1245944107 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7ac0ceec0 0x7ff7ac169880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.237+0000 7ff7b191f700 1 --2- 172.21.15.92:0/1245944107 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7ac15a870 0x7ff7ac0ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.237+0000 7ff7b191f700 1 -- 172.21.15.92:0/1245944107 >> 172.21.15.92:0/1245944107 conn(0x7ff7ac067d90 msgr2=0x7ff7ac157cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:22.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.237+0000 7ff7b191f700 1 -- 172.21.15.92:0/1245944107 shutdown_connections 2023-10-06T10:40:22.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.237+0000 7ff7b191f700 1 -- 172.21.15.92:0/1245944107 wait complete. 2023-10-06T10:40:22.240 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 24 2023-10-06T10:40:22.250 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:40:22.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.364+0000 7f7f1e4d4700 1 Processor -- start 2023-10-06T10:40:22.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.364+0000 7f7f1e4d4700 1 -- start start 2023-10-06T10:40:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.365+0000 7f7f1e4d4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f18159560 0x7f7f180aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.365+0000 7f7f1e4d4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7f180b49a0 0x7f7f180aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.365+0000 7f7f1e4d4700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f180b0c60 con 0x7f7f180b0760 2023-10-06T10:40:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.365+0000 7f7f1e4d4700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f180b0f40 con 0x7f7f180b49a0 2023-10-06T10:40:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.365+0000 7f7f1e4d4700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f180b1220 con 0x7f7f18159560 2023-10-06T10:40:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.365+0000 7f7f177fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7f180b49a0 0x7f7f180aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.365+0000 7f7f177fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7f180b49a0 0x7f7f180aaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58368/0 (socket says 172.21.15.92:58368) 2023-10-06T10:40:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.365+0000 7f7f177fe700 1 -- 172.21.15.92:0/2236255703 learned_addr learned my addr 172.21.15.92:0/2236255703 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.365+0000 7f7f17fff700 1 --2- 172.21.15.92:0/2236255703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f18159560 0x7f7f180aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.365+0000 7f7f177fe700 1 -- 172.21.15.92:0/2236255703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f18159560 msgr2=0x7f7f180aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.365+0000 7f7f177fe700 1 --2- 172.21.15.92:0/2236255703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f18159560 0x7f7f180aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.365+0000 7f7f177fe700 1 -- 172.21.15.92:0/2236255703 >> v1:172.21.15.124:6789/0 conn(0x7f7f180b0760 legacy=0x7f7f180b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.365+0000 7f7f177fe700 1 -- 172.21.15.92:0/2236255703 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f180b1500 con 0x7f7f180b49a0 2023-10-06T10:40:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.366+0000 7f7f177fe700 1 --2- 172.21.15.92:0/2236255703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f180b49a0 0x7f7f180aaa70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0099b0 tx=0x7f7f0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b630cdd323714010 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.366+0000 7f7f16ffd700 1 -- 172.21.15.92:0/2236255703 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c009ea0 con 0x7f7f180b49a0 2023-10-06T10:40:22.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.366+0000 7f7f16ffd700 1 -- 172.21.15.92:0/2236255703 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0c00b860 con 0x7f7f180b49a0 2023-10-06T10:40:22.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.366+0000 7f7f16ffd700 1 -- 172.21.15.92:0/2236255703 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c00bc00 con 0x7f7f180b49a0 2023-10-06T10:40:22.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.366+0000 7f7f1e4d4700 1 -- 172.21.15.92:0/2236255703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f180b49a0 msgr2=0x7f7f180aaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:22.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.366+0000 7f7f1e4d4700 1 --2- 172.21.15.92:0/2236255703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f180b49a0 0x7f7f180aaa70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0099b0 tx=0x7f7f0c009250 comp rx=0 tx=0).stop 2023-10-06T10:40:22.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.366+0000 7f7f1e4d4700 1 -- 172.21.15.92:0/2236255703 shutdown_connections 2023-10-06T10:40:22.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.366+0000 7f7f1e4d4700 1 --2- 172.21.15.92:0/2236255703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f18159560 0x7f7f180aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.366+0000 7f7f1e4d4700 1 --2- 172.21.15.92:0/2236255703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f180b49a0 0x7f7f180aaa70 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.366+0000 7f7f1e4d4700 1 -- 172.21.15.92:0/2236255703 >> 172.21.15.92:0/2236255703 conn(0x7f7f180997f0 msgr2=0x7f7f1806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:22.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.366+0000 7f7f1e4d4700 1 -- 172.21.15.92:0/2236255703 shutdown_connections 2023-10-06T10:40:22.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.366+0000 7f7f1e4d4700 1 -- 172.21.15.92:0/2236255703 wait complete. 2023-10-06T10:40:22.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.367+0000 7f7f1e4d4700 1 Processor -- start 2023-10-06T10:40:22.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.367+0000 7f7f1e4d4700 1 -- start start 2023-10-06T10:40:22.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.368+0000 7f7f1e4d4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f180b0760 0x7f7f1814b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:22.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.368+0000 7f7f1e4d4700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f180b49a0 0x7f7f1814ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:22.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.368+0000 7f7f1e4d4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f18159560 0x7f7f180c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:22.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.368+0000 7f7f1e4d4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f180c8320 con 0x7f7f180b0760 2023-10-06T10:40:22.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.368+0000 7f7f1e4d4700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f180c7710 con 0x7f7f180b49a0 2023-10-06T10:40:22.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.368+0000 7f7f1e4d4700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f180c79f0 con 0x7f7f18159560 2023-10-06T10:40:22.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.368+0000 7f7f17fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f180b0760 0x7f7f1814b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:22.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.368+0000 7f7f17fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f180b0760 0x7f7f1814b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58374/0 (socket says 172.21.15.92:58374) 2023-10-06T10:40:22.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.368+0000 7f7f17fff700 1 -- 172.21.15.92:0/3670005675 learned_addr learned my addr 172.21.15.92:0/3670005675 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:22.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.368+0000 7f7f1ca71700 1 --2- 172.21.15.92:0/3670005675 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f18159560 0x7f7f180c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:22.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.368+0000 7f7f177fe700 1 --2- 172.21.15.92:0/3670005675 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f180b49a0 0x7f7f1814ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:22.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.369+0000 7f7f17fff700 1 -- 172.21.15.92:0/3670005675 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f18159560 msgr2=0x7f7f180c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:22.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.369+0000 7f7f17fff700 1 --2- 172.21.15.92:0/3670005675 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f18159560 0x7f7f180c6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.369+0000 7f7f17fff700 1 -- 172.21.15.92:0/3670005675 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f180b49a0 msgr2=0x7f7f1814ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:22.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.369+0000 7f7f17fff700 1 --2- 172.21.15.92:0/3670005675 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f180b49a0 0x7f7f1814ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.369+0000 7f7f17fff700 1 -- 172.21.15.92:0/3670005675 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0c0094f0 con 0x7f7f180b0760 2023-10-06T10:40:22.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.369+0000 7f7f1ca71700 1 --2- 172.21.15.92:0/3670005675 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f18159560 0x7f7f180c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:22.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.369+0000 7f7f17fff700 1 --2- 172.21.15.92:0/3670005675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f180b0760 0x7f7f1814b4e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7f0800ea00 tx=0x7f7f08009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:22.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.369+0000 7f7efffff700 1 -- 172.21.15.92:0/3670005675 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0800c850 con 0x7f7f180b0760 2023-10-06T10:40:22.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.369+0000 7f7efffff700 1 -- 172.21.15.92:0/3670005675 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0800c9b0 con 0x7f7f180b0760 2023-10-06T10:40:22.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.369+0000 7f7f1e4d4700 1 -- 172.21.15.92:0/3670005675 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f180c7cd0 con 0x7f7f180b0760 2023-10-06T10:40:22.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.369+0000 7f7efffff700 1 -- 172.21.15.92:0/3670005675 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0800cd30 con 0x7f7f180b0760 2023-10-06T10:40:22.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.369+0000 7f7f1e4d4700 1 -- 172.21.15.92:0/3670005675 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1814b140 con 0x7f7f180b0760 2023-10-06T10:40:22.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.370+0000 7f7f1e4d4700 1 -- 172.21.15.92:0/3670005675 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f180c8600 con 0x7f7f180b0760 2023-10-06T10:40:22.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.370+0000 7f7efffff700 1 -- 172.21.15.92:0/3670005675 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7f08013680 con 0x7f7f180b0760 2023-10-06T10:40:22.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.370+0000 7f7efffff700 1 --2- 172.21.15.92:0/3670005675 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7f04066f30 0x7f7f040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:22.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.370+0000 7f7efffff700 1 -- 172.21.15.92:0/3670005675 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f7f08084f10 con 0x7f7f180b0760 2023-10-06T10:40:22.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.371+0000 7f7f177fe700 1 --2- 172.21.15.92:0/3670005675 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7f04066f30 0x7f7f040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:22.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.371+0000 7f7f177fe700 1 --2- 172.21.15.92:0/3670005675 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7f04066f30 0x7f7f040693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0099b0 tx=0x7f7f0c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:22.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.375+0000 7f7efffff700 1 -- 172.21.15.92:0/3670005675 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f08051ea0 con 0x7f7f180b0760 2023-10-06T10:40:22.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.600+0000 7f7f1e4d4700 1 -- 172.21.15.92:0/3670005675 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f18064a90 con 0x7f7f180b0760 2023-10-06T10:40:22.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.600+0000 7f7efffff700 1 -- 172.21.15.92:0/3670005675 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4634 (secure 0 0 0) 0x7f7f08021090 con 0x7f7f180b0760 2023-10-06T10:40:22.601 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:22.602 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2525196781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2525196781},{"type":"v1","addr":"172.21.15.124:6835","nonce":2525196781}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:39:15.675013+0000","modified":"2023-10-06T10:39:48.393721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5133,"mds_1":5121,"mds_2":5118},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5118":{"gid":5118,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6836/2409440810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2409440810},{"type":"v1","addr":"172.21.15.92:6836","nonce":2409440810}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5121":{"gid":5121,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/4010629505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4010629505},{"type":"v1","addr":"172.21.15.92:6837","nonce":4010629505}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5133":{"gid":5133,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.124:6834/484869337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":484869337},{"type":"v1","addr":"172.21.15.124:6834","nonce":484869337}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T10:40:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.602+0000 7f7f1e4d4700 1 -- 172.21.15.92:0/3670005675 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7f04066f30 msgr2=0x7f7f040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.602+0000 7f7f1e4d4700 1 --2- 172.21.15.92:0/3670005675 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7f04066f30 0x7f7f040693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0099b0 tx=0x7f7f0c017000 comp rx=0 tx=0).stop 2023-10-06T10:40:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.602+0000 7f7f1e4d4700 1 -- 172.21.15.92:0/3670005675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f180b0760 msgr2=0x7f7f1814b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.602+0000 7f7f1e4d4700 1 --2- 172.21.15.92:0/3670005675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f180b0760 0x7f7f1814b4e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7f0800ea00 tx=0x7f7f08009120 comp rx=0 tx=0).stop 2023-10-06T10:40:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.602+0000 7f7f1e4d4700 1 -- 172.21.15.92:0/3670005675 shutdown_connections 2023-10-06T10:40:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.602+0000 7f7f1e4d4700 1 --2- 172.21.15.92:0/3670005675 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7f04066f30 0x7f7f040693e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.602+0000 7f7f1e4d4700 1 --2- 172.21.15.92:0/3670005675 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f18159560 0x7f7f180c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.602+0000 7f7f1e4d4700 1 --2- 172.21.15.92:0/3670005675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f180b0760 0x7f7f1814b4e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.602+0000 7f7f1e4d4700 1 --2- 172.21.15.92:0/3670005675 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f180b49a0 0x7f7f1814ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.602+0000 7f7f1e4d4700 1 -- 172.21.15.92:0/3670005675 >> 172.21.15.92:0/3670005675 conn(0x7f7f180997f0 msgr2=0x7f7f180cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:22.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.602+0000 7f7f1e4d4700 1 -- 172.21.15.92:0/3670005675 shutdown_connections 2023-10-06T10:40:22.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.602+0000 7f7f1e4d4700 1 -- 172.21.15.92:0/3670005675 wait complete. 2023-10-06T10:40:22.605 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 24 2023-10-06T10:40:22.616 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:40:22.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.731+0000 7f7db80f9700 1 Processor -- start 2023-10-06T10:40:22.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.732+0000 7f7db80f9700 1 -- start start 2023-10-06T10:40:22.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.732+0000 7f7db80f9700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7db00ca370 0x7f7db0158270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:22.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.732+0000 7f7db80f9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db0157de0 con 0x7f7db00ce8a0 2023-10-06T10:40:22.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.732+0000 7f7db80f9700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db00cb5d0 con 0x7f7db00adb60 2023-10-06T10:40:22.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.732+0000 7f7db80f9700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db00cbb90 con 0x7f7db00ca370 2023-10-06T10:40:22.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.732+0000 7f7db6696700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7db00ce8a0 0x7f7db015aed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54212/0 (socket says 172.21.15.92:54212) 2023-10-06T10:40:22.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.732+0000 7f7db6696700 1 -- 172.21.15.92:0/494410138 learned_addr learned my addr 172.21.15.92:0/494410138 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:22.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db4e93700 1 -- 172.21.15.92:0/494410138 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 716538741 0 0) 0x7f7db0157de0 con 0x7f7db00ce8a0 2023-10-06T10:40:22.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db4e93700 1 -- 172.21.15.92:0/494410138 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d9c003570 con 0x7f7db00ce8a0 2023-10-06T10:40:22.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db5e95700 1 --2- 172.21.15.92:0/494410138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7db00ca370 0x7f7db0158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:22.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db4e93700 1 -- 172.21.15.92:0/494410138 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 665510644 0 0) 0x7f7d9c003570 con 0x7f7db00ce8a0 2023-10-06T10:40:22.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db4e93700 1 -- 172.21.15.92:0/494410138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7db00ca370 msgr2=0x7f7db0158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:22.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db4e93700 1 --2- 172.21.15.92:0/494410138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7db00ca370 0x7f7db0158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db4e93700 1 -- 172.21.15.92:0/494410138 >> v1:172.21.15.124:6789/0 conn(0x7f7db00adb60 legacy=0x7f7db00ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:22.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db4e93700 1 -- 172.21.15.92:0/494410138 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db00cb8b0 con 0x7f7db00ce8a0 2023-10-06T10:40:22.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db4e93700 1 -- 172.21.15.92:0/494410138 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f7da80017f0 con 0x7f7db00ce8a0 2023-10-06T10:40:22.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db4e93700 1 -- 172.21.15.92:0/494410138 >> v1:172.21.15.92:6789/0 conn(0x7f7db00ce8a0 legacy=0x7f7db015aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:22.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db4e93700 1 --2- 172.21.15.92:0/494410138 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d9c003a90 0x7f7d9c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:22.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db4e93700 1 --2- 172.21.15.92:0/494410138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7db00ca370 0x7f7d9c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:22.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db4e93700 1 --2- 172.21.15.92:0/494410138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d9c004d50 0x7f7d9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:22.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db4e93700 1 -- 172.21.15.92:0/494410138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db00cbb90 con 0x7f7d9c004d50 2023-10-06T10:40:22.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db4e93700 1 -- 172.21.15.92:0/494410138 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c00bb70 con 0x7f7d9c003a90 2023-10-06T10:40:22.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db4e93700 1 -- 172.21.15.92:0/494410138 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c00bd30 con 0x7f7db00ca370 2023-10-06T10:40:22.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db6696700 1 --2- 172.21.15.92:0/494410138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d9c004d50 0x7f7d9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:22.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db5e95700 1 --2- 172.21.15.92:0/494410138 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d9c003a90 0x7f7d9c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:22.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db5694700 1 --2- 172.21.15.92:0/494410138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7db00ca370 0x7f7d9c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:22.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db6696700 1 -- 172.21.15.92:0/494410138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7db00ca370 msgr2=0x7f7d9c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:22.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db6696700 1 --2- 172.21.15.92:0/494410138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7db00ca370 0x7f7d9c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db6696700 1 -- 172.21.15.92:0/494410138 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d9c003a90 msgr2=0x7f7d9c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:22.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db6696700 1 --2- 172.21.15.92:0/494410138 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d9c003a90 0x7f7d9c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db6696700 1 -- 172.21.15.92:0/494410138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9c003570 con 0x7f7d9c004d50 2023-10-06T10:40:22.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db5e95700 1 --2- 172.21.15.92:0/494410138 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d9c003a90 0x7f7d9c003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:22.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.733+0000 7f7db5694700 1 --2- 172.21.15.92:0/494410138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7db00ca370 0x7f7d9c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:22.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.734+0000 7f7db6696700 1 --2- 172.21.15.92:0/494410138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d9c004d50 0x7f7d9c0051c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7da800a900 tx=0x7f7da80021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832c329b88ae858b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:22.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.734+0000 7f7db4e93700 1 -- 172.21.15.92:0/494410138 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da80070d0 con 0x7f7d9c004d50 2023-10-06T10:40:22.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.734+0000 7f7db4e93700 1 -- 172.21.15.92:0/494410138 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da8007230 con 0x7f7d9c004d50 2023-10-06T10:40:22.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.734+0000 7f7db4e93700 1 -- 172.21.15.92:0/494410138 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da80075d0 con 0x7f7d9c004d50 2023-10-06T10:40:22.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.734+0000 7f7db80f9700 1 -- 172.21.15.92:0/494410138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d9c004d50 msgr2=0x7f7d9c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:22.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.734+0000 7f7db80f9700 1 --2- 172.21.15.92:0/494410138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d9c004d50 0x7f7d9c0051c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7da800a900 tx=0x7f7da80021d0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.734+0000 7f7db6696700 1 -- 172.21.15.92:0/494410138 reap_dead start 2023-10-06T10:40:22.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.734+0000 7f7db80f9700 1 -- 172.21.15.92:0/494410138 shutdown_connections 2023-10-06T10:40:22.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.734+0000 7f7db80f9700 1 -- 172.21.15.92:0/494410138 >> 172.21.15.92:0/494410138 conn(0x7f7db00707d0 msgr2=0x7f7db0070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:22.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.734+0000 7f7db80f9700 1 -- 172.21.15.92:0/494410138 shutdown_connections 2023-10-06T10:40:22.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.734+0000 7f7db80f9700 1 -- 172.21.15.92:0/494410138 wait complete. 2023-10-06T10:40:22.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.735+0000 7f7db80f9700 1 Processor -- start 2023-10-06T10:40:22.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.735+0000 7f7db80f9700 1 -- start start 2023-10-06T10:40:22.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.736+0000 7f7db80f9700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7db00c9940 0x7f7db00b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:22.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.736+0000 7f7db80f9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7db00b3d50 0x7f7db00c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:22.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.736+0000 7f7db80f9700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7db00c47f0 0x7f7db00c1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:22.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.736+0000 7f7db80f9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db00b3380 con 0x7f7db00b3d50 2023-10-06T10:40:22.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.736+0000 7f7db80f9700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db00b2a50 con 0x7f7db00c9940 2023-10-06T10:40:22.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.736+0000 7f7db80f9700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db00b2d30 con 0x7f7db00c47f0 2023-10-06T10:40:22.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.736+0000 7f7db5694700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7db00b3d50 0x7f7db00c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:22.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.736+0000 7f7db5694700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7db00b3d50 0x7f7db00c42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58388/0 (socket says 172.21.15.92:58388) 2023-10-06T10:40:22.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.736+0000 7f7db5694700 1 -- 172.21.15.92:0/3665739904 learned_addr learned my addr 172.21.15.92:0/3665739904 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:22.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.736+0000 7f7db6696700 1 --2- 172.21.15.92:0/3665739904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7db00c47f0 0x7f7db00c1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:22.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.736+0000 7f7db5e95700 1 --2- 172.21.15.92:0/3665739904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7db00c9940 0x7f7db00b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:22.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.737+0000 7f7db5694700 1 -- 172.21.15.92:0/3665739904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7db00c47f0 msgr2=0x7f7db00c1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:22.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.737+0000 7f7db5694700 1 --2- 172.21.15.92:0/3665739904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7db00c47f0 0x7f7db00c1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.737+0000 7f7db5694700 1 -- 172.21.15.92:0/3665739904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7db00c9940 msgr2=0x7f7db00b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:22.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.737+0000 7f7db5694700 1 --2- 172.21.15.92:0/3665739904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7db00c9940 0x7f7db00b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.737+0000 7f7db5694700 1 -- 172.21.15.92:0/3665739904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db00b3010 con 0x7f7db00b3d50 2023-10-06T10:40:22.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.737+0000 7f7db6696700 1 --2- 172.21.15.92:0/3665739904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7db00c47f0 0x7f7db00c1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:22.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.737+0000 7f7db5694700 1 --2- 172.21.15.92:0/3665739904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7db00b3d50 0x7f7db00c42b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7da000e7b0 tx=0x7f7da0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:22.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.737+0000 7f7da5ffb700 1 -- 172.21.15.92:0/3665739904 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da0006820 con 0x7f7db00b3d50 2023-10-06T10:40:22.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.737+0000 7f7da5ffb700 1 -- 172.21.15.92:0/3665739904 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da0010040 con 0x7f7db00b3d50 2023-10-06T10:40:22.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.737+0000 7f7da5ffb700 1 -- 172.21.15.92:0/3665739904 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da0006bf0 con 0x7f7db00b3d50 2023-10-06T10:40:22.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.737+0000 7f7db80f9700 1 -- 172.21.15.92:0/3665739904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db016dfe0 con 0x7f7db00b3d50 2023-10-06T10:40:22.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.737+0000 7f7db80f9700 1 -- 172.21.15.92:0/3665739904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db00c9d10 con 0x7f7db00b3d50 2023-10-06T10:40:22.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.738+0000 7f7db80f9700 1 -- 172.21.15.92:0/3665739904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db00ced40 con 0x7f7db00b3d50 2023-10-06T10:40:22.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.738+0000 7f7da5ffb700 1 -- 172.21.15.92:0/3665739904 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7da0006d50 con 0x7f7db00b3d50 2023-10-06T10:40:22.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.738+0000 7f7da5ffb700 1 --2- 172.21.15.92:0/3665739904 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7d94066f30 0x7f7d940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:22.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.739+0000 7f7da5ffb700 1 -- 172.21.15.92:0/3665739904 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f7da0083a50 con 0x7f7db00b3d50 2023-10-06T10:40:22.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.739+0000 7f7db5e95700 1 --2- 172.21.15.92:0/3665739904 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7d94066f30 0x7f7d940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:22.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.739+0000 7f7db5e95700 1 --2- 172.21.15.92:0/3665739904 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7d94066f30 0x7f7d940693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0093f0 tx=0x7f7dac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:22.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.743+0000 7f7da5ffb700 1 -- 172.21.15.92:0/3665739904 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7da004d5a0 con 0x7f7db00b3d50 2023-10-06T10:40:22.854 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:40:22.857 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:40:22.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.931+0000 7f7db80f9700 1 -- 172.21.15.92:0/3665739904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7db0064a90 con 0x7f7db00b3d50 2023-10-06T10:40:22.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.932+0000 7f7da5ffb700 1 -- 172.21.15.92:0/3665739904 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+17166 (secure 0 0 0) 0x7f7da00509e0 con 0x7f7db00b3d50 2023-10-06T10:40:22.933 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:22.933 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":40,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:40:06.432393+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:39:13.780983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T10:39:14.644706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":4}]},{"pgid":"7.2a","mappings":[{"from":0,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:40:22.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.934+0000 7f7db80f9700 1 -- 172.21.15.92:0/3665739904 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7d94066f30 msgr2=0x7f7d940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:22.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.934+0000 7f7db80f9700 1 --2- 172.21.15.92:0/3665739904 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7d94066f30 0x7f7d940693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0093f0 tx=0x7f7dac008040 comp rx=0 tx=0).stop 2023-10-06T10:40:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.934+0000 7f7db80f9700 1 -- 172.21.15.92:0/3665739904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7db00b3d50 msgr2=0x7f7db00c42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.934+0000 7f7db80f9700 1 --2- 172.21.15.92:0/3665739904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7db00b3d50 0x7f7db00c42b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7da000e7b0 tx=0x7f7da0009040 comp rx=0 tx=0).stop 2023-10-06T10:40:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.934+0000 7f7db80f9700 1 -- 172.21.15.92:0/3665739904 shutdown_connections 2023-10-06T10:40:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.934+0000 7f7db80f9700 1 --2- 172.21.15.92:0/3665739904 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7d94066f30 0x7f7d940693e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.934+0000 7f7db80f9700 1 --2- 172.21.15.92:0/3665739904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7db00c47f0 0x7f7db00c1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.934+0000 7f7db80f9700 1 --2- 172.21.15.92:0/3665739904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7db00c9940 0x7f7db00b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.934+0000 7f7db80f9700 1 --2- 172.21.15.92:0/3665739904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7db00b3d50 0x7f7db00c42b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.934+0000 7f7db80f9700 1 -- 172.21.15.92:0/3665739904 >> 172.21.15.92:0/3665739904 conn(0x7f7db00707d0 msgr2=0x7f7db015a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.934+0000 7f7db80f9700 1 -- 172.21.15.92:0/3665739904 shutdown_connections 2023-10-06T10:40:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:22.934+0000 7f7db80f9700 1 -- 172.21.15.92:0/3665739904 wait complete. 2023-10-06T10:40:22.947 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T10:40:23.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.101+0000 7fc8e49cf700 1 Processor -- start 2023-10-06T10:40:23.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.102+0000 7fc8e49cf700 1 -- start start 2023-10-06T10:40:23.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.102+0000 7fc8e49cf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc8e00ca370 0x7fc8e0158270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:23.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.102+0000 7fc8e49cf700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8e00adb60 0x7fc8e00ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:23.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.102+0000 7fc8e49cf700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc8e00ce6d0 0x7fc8e015aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:23.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.102+0000 7fc8e49cf700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e00cbb90 con 0x7fc8e00ca370 2023-10-06T10:40:23.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.102+0000 7fc8e49cf700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e00ceb30 con 0x7fc8e00ce6d0 2023-10-06T10:40:23.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.102+0000 7fc8e49cf700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e0157de0 con 0x7fc8e00adb60 2023-10-06T10:40:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.102+0000 7fc8de59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc8e00ca370 0x7fc8e0158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.102+0000 7fc8de59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc8e00ca370 0x7fc8e0158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58404/0 (socket says 172.21.15.92:58404) 2023-10-06T10:40:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.102+0000 7fc8de59c700 1 -- 172.21.15.92:0/1795773094 learned_addr learned my addr 172.21.15.92:0/1795773094 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.103+0000 7fc8de59c700 1 -- 172.21.15.92:0/1795773094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8e00adb60 msgr2=0x7fc8e00ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.103+0000 7fc8d5d9b700 1 --2- 172.21.15.92:0/1795773094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8e00adb60 0x7fc8e00ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.103+0000 7fc8ded9d700 1 --2- 172.21.15.92:0/1795773094 >> v2:172.21.15.124:3300/0 conn(0x7fc8e00ce6d0 0x7fc8e015aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.103+0000 7fc8de59c700 1 --2- 172.21.15.92:0/1795773094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8e00adb60 0x7fc8e00ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.103+0000 7fc8de59c700 1 -- 172.21.15.92:0/1795773094 >> v2:172.21.15.124:3300/0 conn(0x7fc8e00ce6d0 msgr2=0x7fc8e015aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.103+0000 7fc8de59c700 1 --2- 172.21.15.92:0/1795773094 >> v2:172.21.15.124:3300/0 conn(0x7fc8e00ce6d0 0x7fc8e015aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.103+0000 7fc8de59c700 1 -- 172.21.15.92:0/1795773094 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e00cb5d0 con 0x7fc8e00ca370 2023-10-06T10:40:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.103+0000 7fc8d5d9b700 1 --2- 172.21.15.92:0/1795773094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8e00adb60 0x7fc8e00ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.103+0000 7fc8ded9d700 1 --2- 172.21.15.92:0/1795773094 >> v2:172.21.15.124:3300/0 conn(0x7fc8e00ce6d0 0x7fc8e015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.103+0000 7fc8de59c700 1 --2- 172.21.15.92:0/1795773094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8e00ca370 0x7fc8e0158270 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00c5270 tx=0x7fc8d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f74f68fb200996 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.103+0000 7fc8ddd9b700 1 -- 172.21.15.92:0/1795773094 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d000b860 con 0x7fc8e00ca370 2023-10-06T10:40:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.103+0000 7fc8ddd9b700 1 -- 172.21.15.92:0/1795773094 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d000b9c0 con 0x7fc8e00ca370 2023-10-06T10:40:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.103+0000 7fc8ddd9b700 1 -- 172.21.15.92:0/1795773094 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d000bd60 con 0x7fc8e00ca370 2023-10-06T10:40:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.103+0000 7fc8e49cf700 1 -- 172.21.15.92:0/1795773094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8e00ca370 msgr2=0x7fc8e0158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.103+0000 7fc8e49cf700 1 --2- 172.21.15.92:0/1795773094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8e00ca370 0x7fc8e0158270 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00c5270 tx=0x7fc8d0009250 comp rx=0 tx=0).stop 2023-10-06T10:40:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.104+0000 7fc8e49cf700 1 -- 172.21.15.92:0/1795773094 shutdown_connections 2023-10-06T10:40:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.104+0000 7fc8e49cf700 1 --2- 172.21.15.92:0/1795773094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8e00ca370 0x7fc8e0158270 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.104+0000 7fc8e49cf700 1 --2- 172.21.15.92:0/1795773094 >> v2:172.21.15.124:3300/0 conn(0x7fc8e00ce6d0 0x7fc8e015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.104+0000 7fc8e49cf700 1 --2- 172.21.15.92:0/1795773094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8e00adb60 0x7fc8e00ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.104+0000 7fc8e49cf700 1 -- 172.21.15.92:0/1795773094 >> 172.21.15.92:0/1795773094 conn(0x7fc8e00707d0 msgr2=0x7fc8e0070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.104+0000 7fc8e49cf700 1 -- 172.21.15.92:0/1795773094 shutdown_connections 2023-10-06T10:40:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.104+0000 7fc8e49cf700 1 -- 172.21.15.92:0/1795773094 wait complete. 2023-10-06T10:40:23.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.105+0000 7fc8e49cf700 1 Processor -- start 2023-10-06T10:40:23.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.105+0000 7fc8e49cf700 1 -- start start 2023-10-06T10:40:23.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.105+0000 7fc8e49cf700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8e00adb60 0x7fc8e00b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:23.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.105+0000 7fc8e49cf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8e00ca370 0x7fc8e00c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:23.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.106+0000 7fc8e49cf700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc8e00ce6d0 0x7fc8e01598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:23.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.106+0000 7fc8e49cf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e00b2270 con 0x7fc8e00ca370 2023-10-06T10:40:23.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.106+0000 7fc8e49cf700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e00b1660 con 0x7fc8e00ce6d0 2023-10-06T10:40:23.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.106+0000 7fc8e49cf700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e00b1940 con 0x7fc8e00adb60 2023-10-06T10:40:23.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.106+0000 7fc8d5d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8e00ca370 0x7fc8e00c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:23.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.106+0000 7fc8d5d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8e00ca370 0x7fc8e00c24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58420/0 (socket says 172.21.15.92:58420) 2023-10-06T10:40:23.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.106+0000 7fc8d5d9b700 1 -- 172.21.15.92:0/3704384542 learned_addr learned my addr 172.21.15.92:0/3704384542 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:23.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.106+0000 7fc8de59c700 1 --2- 172.21.15.92:0/3704384542 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8e00adb60 0x7fc8e00b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:23.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.106+0000 7fc8ded9d700 1 --2- 172.21.15.92:0/3704384542 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc8e00ce6d0 0x7fc8e01598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:23.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.106+0000 7fc8d5d9b700 1 -- 172.21.15.92:0/3704384542 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8e00adb60 msgr2=0x7fc8e00b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:23.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.106+0000 7fc8d5d9b700 1 --2- 172.21.15.92:0/3704384542 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8e00adb60 0x7fc8e00b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:23.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.106+0000 7fc8d5d9b700 1 -- 172.21.15.92:0/3704384542 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc8e00ce6d0 msgr2=0x7fc8e01598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:23.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.106+0000 7fc8d5d9b700 1 --2- 172.21.15.92:0/3704384542 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc8e00ce6d0 0x7fc8e01598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:23.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.106+0000 7fc8d5d9b700 1 -- 172.21.15.92:0/3704384542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d00094f0 con 0x7fc8e00ca370 2023-10-06T10:40:23.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.106+0000 7fc8d5d9b700 1 --2- 172.21.15.92:0/3704384542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8e00ca370 0x7fc8e00c24b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800c850 tx=0x7fc8c8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:23.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.106+0000 7fc8d6ffd700 1 -- 172.21.15.92:0/3704384542 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8011600 con 0x7fc8e00ca370 2023-10-06T10:40:23.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.106+0000 7fc8d6ffd700 1 -- 172.21.15.92:0/3704384542 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c8011760 con 0x7fc8e00ca370 2023-10-06T10:40:23.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.106+0000 7fc8e49cf700 1 -- 172.21.15.92:0/3704384542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e00b1c20 con 0x7fc8e00ca370 2023-10-06T10:40:23.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.106+0000 7fc8d6ffd700 1 -- 172.21.15.92:0/3704384542 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8011b00 con 0x7fc8e00ca370 2023-10-06T10:40:23.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.106+0000 7fc8e49cf700 1 -- 172.21.15.92:0/3704384542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e0159e20 con 0x7fc8e00ca370 2023-10-06T10:40:23.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.107+0000 7fc8e49cf700 1 -- 172.21.15.92:0/3704384542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e00b2420 con 0x7fc8e00ca370 2023-10-06T10:40:23.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.108+0000 7fc8d6ffd700 1 -- 172.21.15.92:0/3704384542 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc8c8018350 con 0x7fc8e00ca370 2023-10-06T10:40:23.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.108+0000 7fc8d6ffd700 1 --2- 172.21.15.92:0/3704384542 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc8c0066f30 0x7fc8c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:23.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.108+0000 7fc8d6ffd700 1 -- 172.21.15.92:0/3704384542 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fc8c80846b0 con 0x7fc8e00ca370 2023-10-06T10:40:23.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.108+0000 7fc8de59c700 1 --2- 172.21.15.92:0/3704384542 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc8c0066f30 0x7fc8c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:23.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.108+0000 7fc8de59c700 1 --2- 172.21.15.92:0/3704384542 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc8c0066f30 0x7fc8c00693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00c5270 tx=0x7fc8d0017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:23.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.112+0000 7fc8d6ffd700 1 -- 172.21.15.92:0/3704384542 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc8e00b2420 con 0x7fc8e00ca370 2023-10-06T10:40:23.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.336+0000 7fc8e49cf700 1 -- 172.21.15.92:0/3704384542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc8e0064a90 con 0x7fc8e00ca370 2023-10-06T10:40:23.337 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T10:40:23.337+0000 7f9e7f3ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T10:40:23.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.341+0000 7fc8d6ffd700 1 -- 172.21.15.92:0/3704384542 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc8c8054e40 con 0x7fc8e00ca370 2023-10-06T10:40:23.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.343+0000 7fc8e49cf700 1 -- 172.21.15.92:0/3704384542 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc8c0066f30 msgr2=0x7fc8c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:23.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.343+0000 7fc8e49cf700 1 --2- 172.21.15.92:0/3704384542 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc8c0066f30 0x7fc8c00693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00c5270 tx=0x7fc8d0017000 comp rx=0 tx=0).stop 2023-10-06T10:40:23.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.343+0000 7fc8e49cf700 1 -- 172.21.15.92:0/3704384542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8e00ca370 msgr2=0x7fc8e00c24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:23.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.343+0000 7fc8e49cf700 1 --2- 172.21.15.92:0/3704384542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8e00ca370 0x7fc8e00c24b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800c850 tx=0x7fc8c8008040 comp rx=0 tx=0).stop 2023-10-06T10:40:23.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.344+0000 7fc8e49cf700 1 -- 172.21.15.92:0/3704384542 shutdown_connections 2023-10-06T10:40:23.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.344+0000 7fc8e49cf700 1 --2- 172.21.15.92:0/3704384542 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc8c0066f30 0x7fc8c00693e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:23.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.344+0000 7fc8e49cf700 1 --2- 172.21.15.92:0/3704384542 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc8e00ce6d0 0x7fc8e01598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:23.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.344+0000 7fc8e49cf700 1 --2- 172.21.15.92:0/3704384542 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8e00adb60 0x7fc8e00b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:23.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.344+0000 7fc8e49cf700 1 --2- 172.21.15.92:0/3704384542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8e00ca370 0x7fc8e00c24b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:23.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.344+0000 7fc8e49cf700 1 -- 172.21.15.92:0/3704384542 >> 172.21.15.92:0/3704384542 conn(0x7fc8e00707d0 msgr2=0x7fc8e0153cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:23.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.344+0000 7fc8e49cf700 1 -- 172.21.15.92:0/3704384542 shutdown_connections 2023-10-06T10:40:23.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.344+0000 7fc8e49cf700 1 -- 172.21.15.92:0/3704384542 wait complete. 2023-10-06T10:40:23.346 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T10:40:23.358 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T10:40:23.369 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T10:40:23.370 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T10:40:23.370 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-10-06T10:40:23.369+0000 7f61c3e33700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-06T10:40:23.377 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-06T10:40:23.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.475+0000 7f4b54eb3700 1 Processor -- start 2023-10-06T10:40:23.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.475+0000 7f4b54eb3700 1 -- start start 2023-10-06T10:40:23.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.476+0000 7f4b54eb3700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b50150940 0x7f4b50159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:23.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.476+0000 7f4b54eb3700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4b5015a160 0x7f4b500ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:23.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.476+0000 7f4b54eb3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4b500c5250 0x7f4b500cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:23.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.476+0000 7f4b54eb3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5015ac30 con 0x7f4b500c5250 2023-10-06T10:40:23.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.476+0000 7f4b54eb3700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b500cb6a0 con 0x7f4b5015a160 2023-10-06T10:40:23.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.476+0000 7f4b54eb3700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b500cb090 con 0x7f4b50150940 2023-10-06T10:40:23.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.476+0000 7f4b4ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4b500c5250 0x7f4b500cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:23.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.476+0000 7f4b4ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4b500c5250 0x7f4b500cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60394/0 (socket says 172.21.15.92:60394) 2023-10-06T10:40:23.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.476+0000 7f4b4ed9d700 1 -- 172.21.15.92:0/2275981846 learned_addr learned my addr 172.21.15.92:0/2275981846 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:23.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.476+0000 7f4b4dd9b700 1 --2- 172.21.15.92:0/2275981846 >> v2:172.21.15.124:3300/0 conn(0x7f4b5015a160 0x7f4b500ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:23.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.476+0000 7f4b4e59c700 1 --2- 172.21.15.92:0/2275981846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b50150940 0x7f4b50159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:23.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.476+0000 7f4b4ed9d700 1 -- 172.21.15.92:0/2275981846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b50150940 msgr2=0x7f4b50159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:23.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.476+0000 7f4b4ed9d700 1 --2- 172.21.15.92:0/2275981846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b50150940 0x7f4b50159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:23.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.476+0000 7f4b4ed9d700 1 -- 172.21.15.92:0/2275981846 >> v2:172.21.15.124:3300/0 conn(0x7f4b5015a160 msgr2=0x7f4b500ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:23.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.476+0000 7f4b4ed9d700 1 --2- 172.21.15.92:0/2275981846 >> v2:172.21.15.124:3300/0 conn(0x7f4b5015a160 0x7f4b500ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:23.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.476+0000 7f4b4ed9d700 1 -- 172.21.15.92:0/2275981846 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5015a670 con 0x7f4b500c5250 2023-10-06T10:40:23.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.476+0000 7f4b4e59c700 1 --2- 172.21.15.92:0/2275981846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b50150940 0x7f4b50159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:23.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.476+0000 7f4b4ed9d700 1 --2- 172.21.15.92:0/2275981846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b500c5250 0x7f4b500cd300 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4b50089590 tx=0x7f4b48009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9174942bb38777d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:23.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.476+0000 7f4b4d59a700 1 -- 172.21.15.92:0/2275981846 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4800e070 con 0x7f4b500c5250 2023-10-06T10:40:23.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.476+0000 7f4b4d59a700 1 -- 172.21.15.92:0/2275981846 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4800ba20 con 0x7f4b500c5250 2023-10-06T10:40:23.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.476+0000 7f4b4d59a700 1 -- 172.21.15.92:0/2275981846 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b480092e0 con 0x7f4b500c5250 2023-10-06T10:40:23.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.477+0000 7f4b54eb3700 1 -- 172.21.15.92:0/2275981846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b500c5250 msgr2=0x7f4b500cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:23.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.477+0000 7f4b54eb3700 1 --2- 172.21.15.92:0/2275981846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b500c5250 0x7f4b500cd300 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4b50089590 tx=0x7f4b48009840 comp rx=0 tx=0).stop 2023-10-06T10:40:23.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.477+0000 7f4b54eb3700 1 -- 172.21.15.92:0/2275981846 shutdown_connections 2023-10-06T10:40:23.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.477+0000 7f4b54eb3700 1 --2- 172.21.15.92:0/2275981846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b500c5250 0x7f4b500cd300 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:23.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.477+0000 7f4b54eb3700 1 --2- 172.21.15.92:0/2275981846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b50150940 0x7f4b50159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:23.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.477+0000 7f4b54eb3700 1 --2- 172.21.15.92:0/2275981846 >> v2:172.21.15.124:3300/0 conn(0x7f4b5015a160 0x7f4b500ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:23.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.477+0000 7f4b54eb3700 1 -- 172.21.15.92:0/2275981846 >> 172.21.15.92:0/2275981846 conn(0x7f4b50099700 msgr2=0x7f4b5006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:23.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.477+0000 7f4b54eb3700 1 -- 172.21.15.92:0/2275981846 shutdown_connections 2023-10-06T10:40:23.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.477+0000 7f4b54eb3700 1 -- 172.21.15.92:0/2275981846 wait complete. 2023-10-06T10:40:23.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.478+0000 7f4b54eb3700 1 Processor -- start 2023-10-06T10:40:23.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.478+0000 7f4b54eb3700 1 -- start start 2023-10-06T10:40:23.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.479+0000 7f4b54eb3700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b500c5250 0x7f4b5013d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:23.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.479+0000 7f4b54eb3700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b5015a160 0x7f4b500cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:23.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.479+0000 7f4b54eb3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b50155130 0x7f4b500cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:23.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.479+0000 7f4b54eb3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5013db00 con 0x7f4b50155130 2023-10-06T10:40:23.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.479+0000 7f4b54eb3700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b50154ca0 con 0x7f4b5015a160 2023-10-06T10:40:23.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.479+0000 7f4b54eb3700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b500b41d0 con 0x7f4b500c5250 2023-10-06T10:40:23.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.479+0000 7f4b4ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b50155130 0x7f4b500cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:23.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.479+0000 7f4b4ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b50155130 0x7f4b500cc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60402/0 (socket says 172.21.15.92:60402) 2023-10-06T10:40:23.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.479+0000 7f4b4ed9d700 1 -- 172.21.15.92:0/1171808504 learned_addr learned my addr 172.21.15.92:0/1171808504 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:23.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.479+0000 7f4b4dd9b700 1 --2- 172.21.15.92:0/1171808504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b5015a160 0x7f4b500cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:23.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.479+0000 7f4b4e59c700 1 --2- 172.21.15.92:0/1171808504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b500c5250 0x7f4b5013d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:23.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.479+0000 7f4b4ed9d700 1 -- 172.21.15.92:0/1171808504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b500c5250 msgr2=0x7f4b5013d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:23.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.479+0000 7f4b4ed9d700 1 --2- 172.21.15.92:0/1171808504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b500c5250 0x7f4b5013d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:23.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.479+0000 7f4b4ed9d700 1 -- 172.21.15.92:0/1171808504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b5015a160 msgr2=0x7f4b500cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:23.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.479+0000 7f4b4ed9d700 1 --2- 172.21.15.92:0/1171808504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b5015a160 0x7f4b500cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:23.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.479+0000 7f4b4ed9d700 1 -- 172.21.15.92:0/1171808504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b48009ae0 con 0x7f4b50155130 2023-10-06T10:40:23.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.479+0000 7f4b4dd9b700 1 --2- 172.21.15.92:0/1171808504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b5015a160 0x7f4b500cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:23.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.480+0000 7f4b4ed9d700 1 --2- 172.21.15.92:0/1171808504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b50155130 0x7f4b500cc070 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4b500d14c0 tx=0x7f4b48018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:23.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.480+0000 7f4b3e7fc700 1 -- 172.21.15.92:0/1171808504 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b480032f0 con 0x7f4b50155130 2023-10-06T10:40:23.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.480+0000 7f4b3e7fc700 1 -- 172.21.15.92:0/1171808504 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4801c410 con 0x7f4b50155130 2023-10-06T10:40:23.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.480+0000 7f4b3e7fc700 1 -- 172.21.15.92:0/1171808504 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4801c7b0 con 0x7f4b50155130 2023-10-06T10:40:23.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.480+0000 7f4b54eb3700 1 -- 172.21.15.92:0/1171808504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b50154370 con 0x7f4b50155130 2023-10-06T10:40:23.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.480+0000 7f4b54eb3700 1 -- 172.21.15.92:0/1171808504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b50154930 con 0x7f4b50155130 2023-10-06T10:40:23.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.481+0000 7f4b54eb3700 1 -- 172.21.15.92:0/1171808504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b50154e30 con 0x7f4b50155130 2023-10-06T10:40:23.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.481+0000 7f4b3e7fc700 1 -- 172.21.15.92:0/1171808504 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4b48003040 con 0x7f4b50155130 2023-10-06T10:40:23.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.481+0000 7f4b3e7fc700 1 --2- 172.21.15.92:0/1171808504 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4b30066f30 0x7f4b300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:23.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.481+0000 7f4b3e7fc700 1 -- 172.21.15.92:0/1171808504 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f4b480862e0 con 0x7f4b50155130 2023-10-06T10:40:23.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.482+0000 7f4b4e59c700 1 --2- 172.21.15.92:0/1171808504 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4b30066f30 0x7f4b300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:23.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.482+0000 7f4b4e59c700 1 --2- 172.21.15.92:0/1171808504 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4b30066f30 0x7f4b300693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4b40006ea0 tx=0x7f4b40008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:23.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.486+0000 7f4b3e7fc700 1 -- 172.21.15.92:0/1171808504 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b48053170 con 0x7f4b50155130 2023-10-06T10:40:23.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:23.713+0000 7f4b54eb3700 1 -- 172.21.15.92:0/1171808504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4b50064a90 con 0x7f4b50155130 2023-10-06T10:40:24.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.341+0000 7f4b3e7fc700 1 -- 172.21.15.92:0/1171808504 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4b48056790 con 0x7f4b50155130 2023-10-06T10:40:24.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.344+0000 7f4b54eb3700 1 -- 172.21.15.92:0/1171808504 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4b30066f30 msgr2=0x7f4b300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:24.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.344+0000 7f4b54eb3700 1 --2- 172.21.15.92:0/1171808504 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4b30066f30 0x7f4b300693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4b40006ea0 tx=0x7f4b40008040 comp rx=0 tx=0).stop 2023-10-06T10:40:24.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.344+0000 7f4b54eb3700 1 -- 172.21.15.92:0/1171808504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b50155130 msgr2=0x7f4b500cc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:24.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.344+0000 7f4b54eb3700 1 --2- 172.21.15.92:0/1171808504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b50155130 0x7f4b500cc070 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4b500d14c0 tx=0x7f4b48018000 comp rx=0 tx=0).stop 2023-10-06T10:40:24.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.344+0000 7f4b54eb3700 1 -- 172.21.15.92:0/1171808504 shutdown_connections 2023-10-06T10:40:24.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.344+0000 7f4b54eb3700 1 --2- 172.21.15.92:0/1171808504 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4b30066f30 0x7f4b300693e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:24.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.344+0000 7f4b54eb3700 1 --2- 172.21.15.92:0/1171808504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b50155130 0x7f4b500cc070 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:24.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.344+0000 7f4b54eb3700 1 --2- 172.21.15.92:0/1171808504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b500c5250 0x7f4b5013d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:24.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.344+0000 7f4b54eb3700 1 --2- 172.21.15.92:0/1171808504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b5015a160 0x7f4b500cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:24.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.345+0000 7f4b54eb3700 1 -- 172.21.15.92:0/1171808504 >> 172.21.15.92:0/1171808504 conn(0x7f4b50099700 msgr2=0x7f4b500ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:24.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.345+0000 7f4b54eb3700 1 -- 172.21.15.92:0/1171808504 shutdown_connections 2023-10-06T10:40:24.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.345+0000 7f4b54eb3700 1 -- 172.21.15.92:0/1171808504 wait complete. 2023-10-06T10:40:24.358 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T10:40:24.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.517+0000 7fa45695c700 1 Processor -- start 2023-10-06T10:40:24.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.518+0000 7fa45695c700 1 -- start start 2023-10-06T10:40:24.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.518+0000 7fa45695c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa450157230 0x7fa450159c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:24.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.518+0000 7fa45695c700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa45015d250 0x7fa4500c8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:24.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.518+0000 7fa45695c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4500c3a90 0x7fa4500c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:24.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.518+0000 7fa45695c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4500cc830 con 0x7fa450157230 2023-10-06T10:40:24.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.518+0000 7fa45695c700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa45015e1d0 con 0x7fa45015d250 2023-10-06T10:40:24.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.518+0000 7fa45695c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45015d6a0 con 0x7fa4500c3a90 2023-10-06T10:40:24.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.519+0000 7fa44ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa450157230 0x7fa450159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:24.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.519+0000 7fa44ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa450157230 0x7fa450159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60412/0 (socket says 172.21.15.92:60412) 2023-10-06T10:40:24.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.519+0000 7fa44ffff700 1 -- 172.21.15.92:0/3816418490 learned_addr learned my addr 172.21.15.92:0/3816418490 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:24.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.519+0000 7fa44f7fe700 1 --2- 172.21.15.92:0/3816418490 >> v2:172.21.15.124:3300/0 conn(0x7fa45015d250 0x7fa4500c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:24.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.519+0000 7fa454ef9700 1 --2- 172.21.15.92:0/3816418490 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4500c3a90 0x7fa4500c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:24.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.519+0000 7fa44ffff700 1 -- 172.21.15.92:0/3816418490 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4500c3a90 msgr2=0x7fa4500c3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:24.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.519+0000 7fa44ffff700 1 --2- 172.21.15.92:0/3816418490 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4500c3a90 0x7fa4500c3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:24.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.519+0000 7fa44ffff700 1 -- 172.21.15.92:0/3816418490 >> v2:172.21.15.124:3300/0 conn(0x7fa45015d250 msgr2=0x7fa4500c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:24.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.519+0000 7fa44ffff700 1 --2- 172.21.15.92:0/3816418490 >> v2:172.21.15.124:3300/0 conn(0x7fa45015d250 0x7fa4500c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:24.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.519+0000 7fa44ffff700 1 -- 172.21.15.92:0/3816418490 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45015d930 con 0x7fa450157230 2023-10-06T10:40:24.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.519+0000 7fa44f7fe700 1 --2- 172.21.15.92:0/3816418490 >> v2:172.21.15.124:3300/0 conn(0x7fa45015d250 0x7fa4500c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:24.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.519+0000 7fa454ef9700 1 --2- 172.21.15.92:0/3816418490 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4500c3a90 0x7fa4500c3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:24.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.519+0000 7fa44ffff700 1 --2- 172.21.15.92:0/3816418490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa450157230 0x7fa450159c50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa45006f2e0 tx=0x7fa440009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571536eb073d80da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:24.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.519+0000 7fa44effd700 1 -- 172.21.15.92:0/3816418490 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa440005c10 con 0x7fa450157230 2023-10-06T10:40:24.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.519+0000 7fa44effd700 1 -- 172.21.15.92:0/3816418490 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa440005d70 con 0x7fa450157230 2023-10-06T10:40:24.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.519+0000 7fa44effd700 1 -- 172.21.15.92:0/3816418490 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44000b880 con 0x7fa450157230 2023-10-06T10:40:24.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.520+0000 7fa45695c700 1 -- 172.21.15.92:0/3816418490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa450157230 msgr2=0x7fa450159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:24.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.520+0000 7fa45695c700 1 --2- 172.21.15.92:0/3816418490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa450157230 0x7fa450159c50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa45006f2e0 tx=0x7fa440009220 comp rx=0 tx=0).stop 2023-10-06T10:40:24.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.520+0000 7fa45695c700 1 -- 172.21.15.92:0/3816418490 shutdown_connections 2023-10-06T10:40:24.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.520+0000 7fa45695c700 1 --2- 172.21.15.92:0/3816418490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa450157230 0x7fa450159c50 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:24.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.520+0000 7fa45695c700 1 --2- 172.21.15.92:0/3816418490 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4500c3a90 0x7fa4500c3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:24.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.520+0000 7fa45695c700 1 --2- 172.21.15.92:0/3816418490 >> v2:172.21.15.124:3300/0 conn(0x7fa45015d250 0x7fa4500c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:24.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.520+0000 7fa45695c700 1 -- 172.21.15.92:0/3816418490 >> 172.21.15.92:0/3816418490 conn(0x7fa450087fc0 msgr2=0x7fa450067d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:24.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.520+0000 7fa45695c700 1 -- 172.21.15.92:0/3816418490 shutdown_connections 2023-10-06T10:40:24.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.520+0000 7fa45695c700 1 -- 172.21.15.92:0/3816418490 wait complete. 2023-10-06T10:40:24.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.521+0000 7fa45695c700 1 Processor -- start 2023-10-06T10:40:24.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.521+0000 7fa45695c700 1 -- start start 2023-10-06T10:40:24.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.522+0000 7fa45695c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4500c3a90 0x7fa45016e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:24.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.522+0000 7fa45695c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa450157230 0x7fa4500c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:24.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.522+0000 7fa45695c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa45015d250 0x7fa450158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:24.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.522+0000 7fa45695c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa450159380 con 0x7fa450157230 2023-10-06T10:40:24.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.522+0000 7fa45695c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45016dcb0 con 0x7fa45015d250 2023-10-06T10:40:24.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.522+0000 7fa45695c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45016d6a0 con 0x7fa4500c3a90 2023-10-06T10:40:24.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.522+0000 7fa44f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa450157230 0x7fa4500c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:24.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.522+0000 7fa44f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa450157230 0x7fa4500c7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60424/0 (socket says 172.21.15.92:60424) 2023-10-06T10:40:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.522+0000 7fa44f7fe700 1 -- 172.21.15.92:0/1075279925 learned_addr learned my addr 172.21.15.92:0/1075279925 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.522+0000 7fa454ef9700 1 --2- 172.21.15.92:0/1075279925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa45015d250 0x7fa450158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.522+0000 7fa44ffff700 1 --2- 172.21.15.92:0/1075279925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4500c3a90 0x7fa45016e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.522+0000 7fa44f7fe700 1 -- 172.21.15.92:0/1075279925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4500c3a90 msgr2=0x7fa45016e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.522+0000 7fa44f7fe700 1 --2- 172.21.15.92:0/1075279925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4500c3a90 0x7fa45016e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.522+0000 7fa44f7fe700 1 -- 172.21.15.92:0/1075279925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa45015d250 msgr2=0x7fa450158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.522+0000 7fa44f7fe700 1 --2- 172.21.15.92:0/1075279925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa45015d250 0x7fa450158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.522+0000 7fa44f7fe700 1 -- 172.21.15.92:0/1075279925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa440009480 con 0x7fa450157230 2023-10-06T10:40:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.522+0000 7fa454ef9700 1 --2- 172.21.15.92:0/1075279925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa45015d250 0x7fa450158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.522+0000 7fa44ffff700 1 --2- 172.21.15.92:0/1075279925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4500c3a90 0x7fa45016e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.522+0000 7fa44f7fe700 1 --2- 172.21.15.92:0/1075279925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa450157230 0x7fa4500c7e70 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa44400c830 tx=0x7fa444008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.522+0000 7fa437fff700 1 -- 172.21.15.92:0/1075279925 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444011600 con 0x7fa450157230 2023-10-06T10:40:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.522+0000 7fa437fff700 1 -- 172.21.15.92:0/1075279925 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa444011760 con 0x7fa450157230 2023-10-06T10:40:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.523+0000 7fa45695c700 1 -- 172.21.15.92:0/1075279925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4501595b0 con 0x7fa450157230 2023-10-06T10:40:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.523+0000 7fa437fff700 1 -- 172.21.15.92:0/1075279925 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444011b20 con 0x7fa450157230 2023-10-06T10:40:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.523+0000 7fa45695c700 1 -- 172.21.15.92:0/1075279925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa450159890 con 0x7fa450157230 2023-10-06T10:40:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.523+0000 7fa45695c700 1 -- 172.21.15.92:0/1075279925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa45016de60 con 0x7fa450157230 2023-10-06T10:40:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.524+0000 7fa437fff700 1 -- 172.21.15.92:0/1075279925 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa44400a3e0 con 0x7fa450157230 2023-10-06T10:40:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.524+0000 7fa437fff700 1 --2- 172.21.15.92:0/1075279925 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa43c066f30 0x7fa43c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.524+0000 7fa437fff700 1 -- 172.21.15.92:0/1075279925 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6694+0+0 (secure 0 0 0) 0x7fa444084da0 con 0x7fa450157230 2023-10-06T10:40:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.524+0000 7fa44ffff700 1 --2- 172.21.15.92:0/1075279925 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa43c066f30 0x7fa43c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.525+0000 7fa44ffff700 1 --2- 172.21.15.92:0/1075279925 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa43c066f30 0x7fa43c0693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa440009260 tx=0x7fa440016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.528+0000 7fa437fff700 1 -- 172.21.15.92:0/1075279925 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa45016de60 con 0x7fa450157230 2023-10-06T10:40:24.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:24.709+0000 7fa45695c700 1 -- 172.21.15.92:0/1075279925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa450070b90 con 0x7fa450157230 2023-10-06T10:40:25.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.475+0000 7fa437fff700 1 -- 172.21.15.92:0/1075279925 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v42) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa4440150d0 con 0x7fa450157230 2023-10-06T10:40:25.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.477+0000 7fa45695c700 1 -- 172.21.15.92:0/1075279925 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa43c066f30 msgr2=0x7fa43c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:25.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.477+0000 7fa45695c700 1 --2- 172.21.15.92:0/1075279925 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa43c066f30 0x7fa43c0693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa440009260 tx=0x7fa440016000 comp rx=0 tx=0).stop 2023-10-06T10:40:25.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.477+0000 7fa45695c700 1 -- 172.21.15.92:0/1075279925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa450157230 msgr2=0x7fa4500c7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:25.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.477+0000 7fa45695c700 1 --2- 172.21.15.92:0/1075279925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa450157230 0x7fa4500c7e70 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa44400c830 tx=0x7fa444008040 comp rx=0 tx=0).stop 2023-10-06T10:40:25.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.477+0000 7fa45695c700 1 -- 172.21.15.92:0/1075279925 shutdown_connections 2023-10-06T10:40:25.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.477+0000 7fa45695c700 1 --2- 172.21.15.92:0/1075279925 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa43c066f30 0x7fa43c0693e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:25.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.477+0000 7fa45695c700 1 --2- 172.21.15.92:0/1075279925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa45015d250 0x7fa450158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:25.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.477+0000 7fa45695c700 1 --2- 172.21.15.92:0/1075279925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4500c3a90 0x7fa45016e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:25.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.477+0000 7fa45695c700 1 --2- 172.21.15.92:0/1075279925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa450157230 0x7fa4500c7e70 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:25.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.477+0000 7fa45695c700 1 -- 172.21.15.92:0/1075279925 >> 172.21.15.92:0/1075279925 conn(0x7fa450087fc0 msgr2=0x7fa4501565d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:25.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.477+0000 7fa45695c700 1 -- 172.21.15.92:0/1075279925 shutdown_connections 2023-10-06T10:40:25.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.477+0000 7fa45695c700 1 -- 172.21.15.92:0/1075279925 wait complete. 2023-10-06T10:40:25.480 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-10-06T10:40:25.491 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T10:40:25.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.608+0000 7f0cdcbeb700 1 Processor -- start 2023-10-06T10:40:25.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.608+0000 7f0cdcbeb700 1 -- start start 2023-10-06T10:40:25.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.609+0000 7f0cdcbeb700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0cd80d09c0 0x7f0cd81642f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:25.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.609+0000 7f0cdcbeb700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd80cdbf0 con 0x7f0cd8153990 2023-10-06T10:40:25.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.609+0000 7f0cdcbeb700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd80cb4c0 con 0x7f0cd80cca70 2023-10-06T10:40:25.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.609+0000 7f0cdcbeb700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd80ccfe0 con 0x7f0cd80d09c0 2023-10-06T10:40:25.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.609+0000 7f0cd77fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0cd8153990 0x7f0cd815a370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50938/0 (socket says 172.21.15.92:50938) 2023-10-06T10:40:25.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.609+0000 7f0cd77fe700 1 -- 172.21.15.92:0/2567989630 learned_addr learned my addr 172.21.15.92:0/2567989630 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:25.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.609+0000 7f0cd67fc700 1 -- 172.21.15.92:0/2567989630 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1494738825 0 0) 0x7f0cd80cdbf0 con 0x7f0cd8153990 2023-10-06T10:40:25.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.609+0000 7f0cd67fc700 1 -- 172.21.15.92:0/2567989630 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cc0003570 con 0x7f0cd8153990 2023-10-06T10:40:25.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd7fff700 1 --2- 172.21.15.92:0/2567989630 >> v2:172.21.15.124:3300/0 conn(0x7f0cd80d09c0 0x7f0cd81642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:25.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd67fc700 1 -- 172.21.15.92:0/2567989630 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 622540489 0 0) 0x7f0cc0003570 con 0x7f0cd8153990 2023-10-06T10:40:25.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd67fc700 1 -- 172.21.15.92:0/2567989630 >> v2:172.21.15.124:3300/0 conn(0x7f0cd80d09c0 msgr2=0x7f0cd81642f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:25.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd67fc700 1 --2- 172.21.15.92:0/2567989630 >> v2:172.21.15.124:3300/0 conn(0x7f0cd80d09c0 0x7f0cd81642f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:25.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd67fc700 1 -- 172.21.15.92:0/2567989630 >> v1:172.21.15.124:6789/0 conn(0x7f0cd80cca70 legacy=0x7f0cd80d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:25.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd67fc700 1 -- 172.21.15.92:0/2567989630 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd80cd2c0 con 0x7f0cd8153990 2023-10-06T10:40:25.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd67fc700 1 -- 172.21.15.92:0/2567989630 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f0cc80017f0 con 0x7f0cd8153990 2023-10-06T10:40:25.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd67fc700 1 -- 172.21.15.92:0/2567989630 >> v1:172.21.15.92:6789/0 conn(0x7f0cd8153990 legacy=0x7f0cd815a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:25.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd67fc700 1 --2- 172.21.15.92:0/2567989630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cc0003a60 0x7f0cc0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:25.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd67fc700 1 --2- 172.21.15.92:0/2567989630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cc0004810 0x7f0cc0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:25.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd67fc700 1 --2- 172.21.15.92:0/2567989630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cc00051c0 0x7f0cc000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:25.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd67fc700 1 -- 172.21.15.92:0/2567989630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd80ccfe0 con 0x7f0cc0004810 2023-10-06T10:40:25.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd67fc700 1 -- 172.21.15.92:0/2567989630 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc000c160 con 0x7f0cc0003a60 2023-10-06T10:40:25.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd67fc700 1 -- 172.21.15.92:0/2567989630 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc000c320 con 0x7f0cc00051c0 2023-10-06T10:40:25.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd6ffd700 1 --2- 172.21.15.92:0/2567989630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cc0004810 0x7f0cc0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:25.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd77fe700 1 --2- 172.21.15.92:0/2567989630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cc0003a60 0x7f0cc0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:25.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd7fff700 1 --2- 172.21.15.92:0/2567989630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cc00051c0 0x7f0cc000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:25.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd6ffd700 1 -- 172.21.15.92:0/2567989630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cc00051c0 msgr2=0x7f0cc000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:25.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd6ffd700 1 --2- 172.21.15.92:0/2567989630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cc00051c0 0x7f0cc000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:25.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd6ffd700 1 -- 172.21.15.92:0/2567989630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cc0003a60 msgr2=0x7f0cc0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:25.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd6ffd700 1 --2- 172.21.15.92:0/2567989630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cc0003a60 0x7f0cc0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:25.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd6ffd700 1 -- 172.21.15.92:0/2567989630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc0003570 con 0x7f0cc0004810 2023-10-06T10:40:25.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd7fff700 1 --2- 172.21.15.92:0/2567989630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cc00051c0 0x7f0cc000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:25.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd7fff700 1 -- 172.21.15.92:0/2567989630 reap_dead start 2023-10-06T10:40:25.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.610+0000 7f0cd6ffd700 1 --2- 172.21.15.92:0/2567989630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cc0004810 0x7f0cc0004c80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0018b0 tx=0x7f0ccc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446df9b08e90cb4f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:25.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.611+0000 7f0cd67fc700 1 -- 172.21.15.92:0/2567989630 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc012420 con 0x7f0cc0004810 2023-10-06T10:40:25.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.611+0000 7f0cd67fc700 1 -- 172.21.15.92:0/2567989630 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ccc0125a0 con 0x7f0cc0004810 2023-10-06T10:40:25.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.611+0000 7f0cd67fc700 1 -- 172.21.15.92:0/2567989630 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc012960 con 0x7f0cc0004810 2023-10-06T10:40:25.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.611+0000 7f0cdcbeb700 1 -- 172.21.15.92:0/2567989630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cc0004810 msgr2=0x7f0cc0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:25.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.611+0000 7f0cdcbeb700 1 --2- 172.21.15.92:0/2567989630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cc0004810 0x7f0cc0004c80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0018b0 tx=0x7f0ccc007d30 comp rx=0 tx=0).stop 2023-10-06T10:40:25.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.611+0000 7f0cdcbeb700 1 -- 172.21.15.92:0/2567989630 shutdown_connections 2023-10-06T10:40:25.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.611+0000 7f0cdcbeb700 1 --2- 172.21.15.92:0/2567989630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cc0004810 0x7f0cc0004c80 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:25.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.611+0000 7f0cdcbeb700 1 -- 172.21.15.92:0/2567989630 >> 172.21.15.92:0/2567989630 conn(0x7f0cd80680c0 msgr2=0x7f0cd8068490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:25.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.611+0000 7f0cdcbeb700 1 -- 172.21.15.92:0/2567989630 shutdown_connections 2023-10-06T10:40:25.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.611+0000 7f0cdcbeb700 1 -- 172.21.15.92:0/2567989630 wait complete. 2023-10-06T10:40:25.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.612+0000 7f0cdcbeb700 1 Processor -- start 2023-10-06T10:40:25.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.612+0000 7f0cdcbeb700 1 -- start start 2023-10-06T10:40:25.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.613+0000 7f0cdcbeb700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cc0004810 0x7f0cd8158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:25.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.613+0000 7f0cdcbeb700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cd8151280 0x7f0cd80ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:25.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.613+0000 7f0cdcbeb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd80b0f90 0x7f0cd80b1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:25.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.613+0000 7f0cdcbeb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd80c8d80 con 0x7f0cd80b0f90 2023-10-06T10:40:25.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.613+0000 7f0cdcbeb700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd80b1940 con 0x7f0cc0004810 2023-10-06T10:40:25.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.613+0000 7f0cdcbeb700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd80cac40 con 0x7f0cd8151280 2023-10-06T10:40:25.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.613+0000 7f0cd7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd80b0f90 0x7f0cd80b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:25.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.613+0000 7f0cd7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd80b0f90 0x7f0cd80b1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60438/0 (socket says 172.21.15.92:60438) 2023-10-06T10:40:25.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.613+0000 7f0cd7fff700 1 -- 172.21.15.92:0/1463714487 learned_addr learned my addr 172.21.15.92:0/1463714487 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:25.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.613+0000 7f0cd77fe700 1 --2- 172.21.15.92:0/1463714487 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cc0004810 0x7f0cd8158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:25.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.613+0000 7f0cd6ffd700 1 --2- 172.21.15.92:0/1463714487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cd8151280 0x7f0cd80ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:25.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.613+0000 7f0cd7fff700 1 -- 172.21.15.92:0/1463714487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cd8151280 msgr2=0x7f0cd80ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:25.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.613+0000 7f0cd7fff700 1 --2- 172.21.15.92:0/1463714487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cd8151280 0x7f0cd80ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:25.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.613+0000 7f0cd7fff700 1 -- 172.21.15.92:0/1463714487 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cc0004810 msgr2=0x7f0cd8158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:25.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.613+0000 7f0cd7fff700 1 --2- 172.21.15.92:0/1463714487 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cc0004810 0x7f0cd8158fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:25.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.614+0000 7f0cd7fff700 1 -- 172.21.15.92:0/1463714487 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd80c88f0 con 0x7f0cd80b0f90 2023-10-06T10:40:25.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.614+0000 7f0cd77fe700 1 --2- 172.21.15.92:0/1463714487 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cc0004810 0x7f0cd8158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:25.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.614+0000 7f0cd6ffd700 1 --2- 172.21.15.92:0/1463714487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cd8151280 0x7f0cd80ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:25.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.614+0000 7f0cd7fff700 1 --2- 172.21.15.92:0/1463714487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd80b0f90 0x7f0cd80b1400 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80d0dc0 tx=0x7f0cd0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:25.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.614+0000 7f0cbf7fe700 1 -- 172.21.15.92:0/1463714487 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0015740 con 0x7f0cd80b0f90 2023-10-06T10:40:25.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.614+0000 7f0cbf7fe700 1 -- 172.21.15.92:0/1463714487 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd00158a0 con 0x7f0cd80b0f90 2023-10-06T10:40:25.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.614+0000 7f0cdcbeb700 1 -- 172.21.15.92:0/1463714487 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd80b30c0 con 0x7f0cd80b0f90 2023-10-06T10:40:25.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.614+0000 7f0cbf7fe700 1 -- 172.21.15.92:0/1463714487 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0015c60 con 0x7f0cd80b0f90 2023-10-06T10:40:25.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.614+0000 7f0cdcbeb700 1 -- 172.21.15.92:0/1463714487 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd80ca960 con 0x7f0cd80b0f90 2023-10-06T10:40:25.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.615+0000 7f0cdcbeb700 1 -- 172.21.15.92:0/1463714487 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd80c8a80 con 0x7f0cd80b0f90 2023-10-06T10:40:25.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.615+0000 7f0cbf7fe700 1 -- 172.21.15.92:0/1463714487 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0cd0020460 con 0x7f0cd80b0f90 2023-10-06T10:40:25.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.616+0000 7f0cbf7fe700 1 --2- 172.21.15.92:0/1463714487 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0cc4066f30 0x7f0cc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:25.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.616+0000 7f0cbf7fe700 1 -- 172.21.15.92:0/1463714487 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6256+0+0 (secure 0 0 0) 0x7f0cd0085630 con 0x7f0cd80b0f90 2023-10-06T10:40:25.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.616+0000 7f0cd77fe700 1 --2- 172.21.15.92:0/1463714487 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0cc4066f30 0x7f0cc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:25.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.616+0000 7f0cd77fe700 1 --2- 172.21.15.92:0/1463714487 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0cc4066f30 0x7f0cc40693e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8002060 tx=0x7f0cc8006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:25.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.620+0000 7f0cbf7fe700 1 -- 172.21.15.92:0/1463714487 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0cd0052700 con 0x7f0cd80b0f90 2023-10-06T10:40:25.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:25.800+0000 7f0cdcbeb700 1 -- 172.21.15.92:0/1463714487 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0cd805f290 con 0x7f0cd80b0f90 2023-10-06T10:40:26.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.489+0000 7f0cbf7fe700 1 -- 172.21.15.92:0/1463714487 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v43) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0cd0055d20 con 0x7f0cd80b0f90 2023-10-06T10:40:26.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.492+0000 7f0cdcbeb700 1 -- 172.21.15.92:0/1463714487 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0cc4066f30 msgr2=0x7f0cc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:26.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.492+0000 7f0cdcbeb700 1 --2- 172.21.15.92:0/1463714487 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0cc4066f30 0x7f0cc40693e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8002060 tx=0x7f0cc8006040 comp rx=0 tx=0).stop 2023-10-06T10:40:26.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.492+0000 7f0cdcbeb700 1 -- 172.21.15.92:0/1463714487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd80b0f90 msgr2=0x7f0cd80b1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:26.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.492+0000 7f0cdcbeb700 1 --2- 172.21.15.92:0/1463714487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd80b0f90 0x7f0cd80b1400 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80d0dc0 tx=0x7f0cd0008040 comp rx=0 tx=0).stop 2023-10-06T10:40:26.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.492+0000 7f0cdcbeb700 1 -- 172.21.15.92:0/1463714487 shutdown_connections 2023-10-06T10:40:26.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.492+0000 7f0cdcbeb700 1 --2- 172.21.15.92:0/1463714487 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0cc4066f30 0x7f0cc40693e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:26.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.492+0000 7f0cdcbeb700 1 --2- 172.21.15.92:0/1463714487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd80b0f90 0x7f0cd80b1400 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:26.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.492+0000 7f0cdcbeb700 1 --2- 172.21.15.92:0/1463714487 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cc0004810 0x7f0cd8158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:26.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.492+0000 7f0cdcbeb700 1 --2- 172.21.15.92:0/1463714487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cd8151280 0x7f0cd80ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:26.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.492+0000 7f0cdcbeb700 1 -- 172.21.15.92:0/1463714487 >> 172.21.15.92:0/1463714487 conn(0x7f0cd80680c0 msgr2=0x7f0cd80c50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:26.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.492+0000 7f0cdcbeb700 1 -- 172.21.15.92:0/1463714487 shutdown_connections 2023-10-06T10:40:26.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.492+0000 7f0cdcbeb700 1 -- 172.21.15.92:0/1463714487 wait complete. 2023-10-06T10:40:26.494 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-10-06T10:40:26.506 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T10:40:26.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.623+0000 7ff5e1447700 1 Processor -- start 2023-10-06T10:40:26.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.623+0000 7ff5e1447700 1 -- start start 2023-10-06T10:40:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.623+0000 7ff5e1447700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff5dc151980 0x7ff5dc06b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.623+0000 7ff5e1447700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5dc0b0500 con 0x7ff5dc15b900 2023-10-06T10:40:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.624+0000 7ff5e1447700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5dc15bd00 con 0x7ff5dc06bde0 2023-10-06T10:40:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.624+0000 7ff5e1447700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5dc06c260 con 0x7ff5dc151980 2023-10-06T10:40:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.624+0000 7ff5daffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff5dc151980 0x7ff5dc06b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.624+0000 7ff5db7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff5dc15b900 0x7ff5dc158b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50952/0 (socket says 172.21.15.92:50952) 2023-10-06T10:40:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.624+0000 7ff5db7fe700 1 -- 172.21.15.92:0/3884972468 learned_addr learned my addr 172.21.15.92:0/3884972468 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.624+0000 7ff5daffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff5dc151980 0x7ff5dc06b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60448/0 (socket says 172.21.15.92:60448) 2023-10-06T10:40:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.624+0000 7ff5daffd700 1 -- 172.21.15.92:0/3884972468 >> v1:172.21.15.124:6789/0 conn(0x7ff5dc06bde0 legacy=0x7ff5dc15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.624+0000 7ff5daffd700 1 -- 172.21.15.92:0/3884972468 >> v1:172.21.15.92:6789/0 conn(0x7ff5dc15b900 legacy=0x7ff5dc158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.624+0000 7ff5daffd700 1 -- 172.21.15.92:0/3884972468 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5dc0afbd0 con 0x7ff5dc151980 2023-10-06T10:40:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.624+0000 7ff5daffd700 1 --2- 172.21.15.92:0/3884972468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5dc151980 0x7ff5dc06b8a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc0098e0 tx=0x7ff5cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee904adccc324e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.624+0000 7ff5da7fc700 1 -- 172.21.15.92:0/3884972468 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc009e80 con 0x7ff5dc151980 2023-10-06T10:40:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.624+0000 7ff5da7fc700 1 -- 172.21.15.92:0/3884972468 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5cc00b820 con 0x7ff5dc151980 2023-10-06T10:40:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.624+0000 7ff5da7fc700 1 -- 172.21.15.92:0/3884972468 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc00bba0 con 0x7ff5dc151980 2023-10-06T10:40:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.625+0000 7ff5e1447700 1 -- 172.21.15.92:0/3884972468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5dc151980 msgr2=0x7ff5dc06b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.625+0000 7ff5e1447700 1 --2- 172.21.15.92:0/3884972468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5dc151980 0x7ff5dc06b8a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc0098e0 tx=0x7ff5cc0091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.625+0000 7ff5e1447700 1 -- 172.21.15.92:0/3884972468 shutdown_connections 2023-10-06T10:40:26.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.625+0000 7ff5e1447700 1 --2- 172.21.15.92:0/3884972468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5dc151980 0x7ff5dc06b8a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:26.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.625+0000 7ff5e1447700 1 -- 172.21.15.92:0/3884972468 >> 172.21.15.92:0/3884972468 conn(0x7ff5dc0ce2f0 msgr2=0x7ff5dc0ce6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:26.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.625+0000 7ff5e1447700 1 -- 172.21.15.92:0/3884972468 shutdown_connections 2023-10-06T10:40:26.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.625+0000 7ff5e1447700 1 -- 172.21.15.92:0/3884972468 wait complete. 2023-10-06T10:40:26.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.626+0000 7ff5e1447700 1 Processor -- start 2023-10-06T10:40:26.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.626+0000 7ff5e1447700 1 -- start start 2023-10-06T10:40:26.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.627+0000 7ff5e1447700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff5dc06bde0 0x7ff5dc0c0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:26.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.627+0000 7ff5e1447700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5dc151980 0x7ff5dc1591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:26.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.627+0000 7ff5e1447700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5dc15b900 0x7ff5dc159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:26.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.627+0000 7ff5e1447700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc14f410 con 0x7ff5dc151980 2023-10-06T10:40:26.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.627+0000 7ff5e1447700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc14e800 con 0x7ff5dc06bde0 2023-10-06T10:40:26.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.627+0000 7ff5e1447700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc14eae0 con 0x7ff5dc15b900 2023-10-06T10:40:26.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.627+0000 7ff5d3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5dc151980 0x7ff5dc1591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:26.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.627+0000 7ff5d3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5dc151980 0x7ff5dc1591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60456/0 (socket says 172.21.15.92:60456) 2023-10-06T10:40:26.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.627+0000 7ff5d3fff700 1 -- 172.21.15.92:0/815081993 learned_addr learned my addr 172.21.15.92:0/815081993 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:26.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.627+0000 7ff5d3fff700 1 -- 172.21.15.92:0/815081993 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5dc15b900 msgr2=0x7ff5dc159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:26.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.627+0000 7ff5d3fff700 1 --2- 172.21.15.92:0/815081993 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5dc15b900 0x7ff5dc159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:26.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.627+0000 7ff5d3fff700 1 -- 172.21.15.92:0/815081993 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff5dc06bde0 msgr2=0x7ff5dc0c0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:26.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.627+0000 7ff5d3fff700 1 --2- 172.21.15.92:0/815081993 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff5dc06bde0 0x7ff5dc0c0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:26.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.627+0000 7ff5d3fff700 1 -- 172.21.15.92:0/815081993 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5cc009420 con 0x7ff5dc151980 2023-10-06T10:40:26.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.627+0000 7ff5d3fff700 1 --2- 172.21.15.92:0/815081993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5dc151980 0x7ff5dc1591f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff5c400b600 tx=0x7ff5c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:26.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.627+0000 7ff5d2ffd700 1 -- 172.21.15.92:0/815081993 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c400f510 con 0x7ff5dc151980 2023-10-06T10:40:26.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.627+0000 7ff5d2ffd700 1 -- 172.21.15.92:0/815081993 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c400f690 con 0x7ff5dc151980 2023-10-06T10:40:26.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.627+0000 7ff5e1447700 1 -- 172.21.15.92:0/815081993 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5dc14edc0 con 0x7ff5dc151980 2023-10-06T10:40:26.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.627+0000 7ff5d2ffd700 1 -- 172.21.15.92:0/815081993 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c400fa50 con 0x7ff5dc151980 2023-10-06T10:40:26.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.627+0000 7ff5e1447700 1 -- 172.21.15.92:0/815081993 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5dc168760 con 0x7ff5dc151980 2023-10-06T10:40:26.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.628+0000 7ff5e1447700 1 -- 172.21.15.92:0/815081993 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5dc06db30 con 0x7ff5dc151980 2023-10-06T10:40:26.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.629+0000 7ff5d2ffd700 1 -- 172.21.15.92:0/815081993 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff5c401a400 con 0x7ff5dc151980 2023-10-06T10:40:26.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.629+0000 7ff5d2ffd700 1 --2- 172.21.15.92:0/815081993 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff5bc066f30 0x7ff5bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:26.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.629+0000 7ff5d2ffd700 1 -- 172.21.15.92:0/815081993 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5784+0+0 (secure 0 0 0) 0x7ff5c4084000 con 0x7ff5dc151980 2023-10-06T10:40:26.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.629+0000 7ff5daffd700 1 --2- 172.21.15.92:0/815081993 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff5bc066f30 0x7ff5bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:26.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.629+0000 7ff5daffd700 1 --2- 172.21.15.92:0/815081993 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff5bc066f30 0x7ff5bc0693e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc0caaf0 tx=0x7ff5cc016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:26.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.633+0000 7ff5d2ffd700 1 -- 172.21.15.92:0/815081993 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5c40512a0 con 0x7ff5dc151980 2023-10-06T10:40:26.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.818+0000 7ff5e1447700 1 -- 172.21.15.92:0/815081993 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff5dc0cc8b0 con 0x7ff5dc151980 2023-10-06T10:40:26.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.821+0000 7ff5d2ffd700 1 -- 172.21.15.92:0/815081993 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff5c40548c0 con 0x7ff5dc151980 2023-10-06T10:40:26.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.824+0000 7ff5e1447700 1 -- 172.21.15.92:0/815081993 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff5bc066f30 msgr2=0x7ff5bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:26.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.824+0000 7ff5e1447700 1 --2- 172.21.15.92:0/815081993 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff5bc066f30 0x7ff5bc0693e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc0caaf0 tx=0x7ff5cc016000 comp rx=0 tx=0).stop 2023-10-06T10:40:26.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.824+0000 7ff5e1447700 1 -- 172.21.15.92:0/815081993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5dc151980 msgr2=0x7ff5dc1591f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:26.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.824+0000 7ff5e1447700 1 --2- 172.21.15.92:0/815081993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5dc151980 0x7ff5dc1591f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff5c400b600 tx=0x7ff5c40091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:26.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.824+0000 7ff5e1447700 1 -- 172.21.15.92:0/815081993 shutdown_connections 2023-10-06T10:40:26.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.824+0000 7ff5e1447700 1 --2- 172.21.15.92:0/815081993 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff5bc066f30 0x7ff5bc0693e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:26.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.824+0000 7ff5e1447700 1 --2- 172.21.15.92:0/815081993 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5dc15b900 0x7ff5dc159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:26.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.824+0000 7ff5e1447700 1 --2- 172.21.15.92:0/815081993 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff5dc06bde0 0x7ff5dc0c0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:26.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.824+0000 7ff5e1447700 1 --2- 172.21.15.92:0/815081993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5dc151980 0x7ff5dc1591f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:26.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.824+0000 7ff5e1447700 1 -- 172.21.15.92:0/815081993 >> 172.21.15.92:0/815081993 conn(0x7ff5dc0ce2f0 msgr2=0x7ff5dc0c2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:26.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.824+0000 7ff5e1447700 1 -- 172.21.15.92:0/815081993 shutdown_connections 2023-10-06T10:40:26.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.824+0000 7ff5e1447700 1 -- 172.21.15.92:0/815081993 wait complete. 2023-10-06T10:40:26.836 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag' 2023-10-06T10:40:26.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.949+0000 7f913ab0a700 1 Processor -- start 2023-10-06T10:40:26.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.949+0000 7f913ab0a700 1 -- start start 2023-10-06T10:40:26.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.950+0000 7f913ab0a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9134159950 0x7f91340675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:26.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.950+0000 7f913ab0a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9134067af0 0x7f91340b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:26.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.950+0000 7f913ab0a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9134067f30 con 0x7f9134159580 2023-10-06T10:40:26.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.950+0000 7f913ab0a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91340cde30 con 0x7f9134067af0 2023-10-06T10:40:26.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.950+0000 7f913ab0a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913406ef20 con 0x7f9134159950 2023-10-06T10:40:26.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.950+0000 7f91390a7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9134067af0 0x7f91340b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:26.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.950+0000 7f91388a6700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9134159580 0x7f913406e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50968/0 (socket says 172.21.15.92:50968) 2023-10-06T10:40:26.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.950+0000 7f91388a6700 1 -- 172.21.15.92:0/4235530180 learned_addr learned my addr 172.21.15.92:0/4235530180 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:26.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.951+0000 7f912bfff700 1 --2- 172.21.15.92:0/4235530180 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9134159950 0x7f91340675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:26.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.951+0000 7f91390a7700 1 -- 172.21.15.92:0/4235530180 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9134159950 msgr2=0x7f91340675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:26.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.951+0000 7f91390a7700 1 --2- 172.21.15.92:0/4235530180 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9134159950 0x7f91340675b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:26.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.951+0000 7f91390a7700 1 -- 172.21.15.92:0/4235530180 >> v1:172.21.15.92:6789/0 conn(0x7f9134159580 legacy=0x7f913406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:26.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.951+0000 7f912b7fe700 1 -- 172.21.15.92:0/4235530180 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2793475119 0 0) 0x7f9134067f30 con 0x7f9134159580 2023-10-06T10:40:26.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.951+0000 7f91390a7700 1 -- 172.21.15.92:0/4235530180 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913406f200 con 0x7f9134067af0 2023-10-06T10:40:26.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.951+0000 7f91390a7700 1 --2- 172.21.15.92:0/4235530180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9134067af0 0x7f91340b0950 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f912c006fd0 tx=0x7f912c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fcc76e5f2145cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:26.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.951+0000 7f912b7fe700 1 -- 172.21.15.92:0/4235530180 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c005c60 con 0x7f9134067af0 2023-10-06T10:40:26.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.951+0000 7f912b7fe700 1 -- 172.21.15.92:0/4235530180 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c005dc0 con 0x7f9134067af0 2023-10-06T10:40:26.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.951+0000 7f912b7fe700 1 -- 172.21.15.92:0/4235530180 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c006280 con 0x7f9134067af0 2023-10-06T10:40:26.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.951+0000 7f913ab0a700 1 -- 172.21.15.92:0/4235530180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9134067af0 msgr2=0x7f91340b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:26.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.951+0000 7f913ab0a700 1 --2- 172.21.15.92:0/4235530180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9134067af0 0x7f91340b0950 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f912c006fd0 tx=0x7f912c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:26.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.952+0000 7f913ab0a700 1 -- 172.21.15.92:0/4235530180 shutdown_connections 2023-10-06T10:40:26.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.952+0000 7f913ab0a700 1 --2- 172.21.15.92:0/4235530180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9134067af0 0x7f91340b0950 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:26.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.952+0000 7f913ab0a700 1 --2- 172.21.15.92:0/4235530180 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9134159950 0x7f91340675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:26.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.952+0000 7f913ab0a700 1 -- 172.21.15.92:0/4235530180 >> 172.21.15.92:0/4235530180 conn(0x7f9134069140 msgr2=0x7f91340cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:26.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.952+0000 7f913ab0a700 1 -- 172.21.15.92:0/4235530180 shutdown_connections 2023-10-06T10:40:26.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.952+0000 7f913ab0a700 1 -- 172.21.15.92:0/4235530180 wait complete. 2023-10-06T10:40:26.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.953+0000 7f913ab0a700 1 Processor -- start 2023-10-06T10:40:26.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.953+0000 7f913ab0a700 1 -- start start 2023-10-06T10:40:26.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.953+0000 7f913ab0a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9134067af0 0x7f91341475b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:26.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.953+0000 7f913ab0a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9134159580 0x7f9134169f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:26.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.953+0000 7f913ab0a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9134159950 0x7f91340ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:26.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.954+0000 7f913ab0a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913416afb0 con 0x7f9134159950 2023-10-06T10:40:26.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.954+0000 7f913ab0a700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913416a9f0 con 0x7f9134067af0 2023-10-06T10:40:26.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.954+0000 7f913ab0a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913416acd0 con 0x7f9134159580 2023-10-06T10:40:26.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.954+0000 7f91390a7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9134159950 0x7f91340ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:26.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.954+0000 7f91390a7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9134159950 0x7f91340ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60482/0 (socket says 172.21.15.92:60482) 2023-10-06T10:40:26.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.954+0000 7f91390a7700 1 -- 172.21.15.92:0/2273908412 learned_addr learned my addr 172.21.15.92:0/2273908412 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:26.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.954+0000 7f912bfff700 1 --2- 172.21.15.92:0/2273908412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9134159580 0x7f9134169f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:26.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.954+0000 7f91388a6700 1 --2- 172.21.15.92:0/2273908412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9134067af0 0x7f91341475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:26.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.954+0000 7f91390a7700 1 -- 172.21.15.92:0/2273908412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9134159580 msgr2=0x7f9134169f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:26.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.954+0000 7f91390a7700 1 --2- 172.21.15.92:0/2273908412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9134159580 0x7f9134169f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:26.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.954+0000 7f91390a7700 1 -- 172.21.15.92:0/2273908412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9134067af0 msgr2=0x7f91341475b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:26.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.954+0000 7f91390a7700 1 --2- 172.21.15.92:0/2273908412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9134067af0 0x7f91341475b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:26.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.954+0000 7f91390a7700 1 -- 172.21.15.92:0/2273908412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c009420 con 0x7f9134159950 2023-10-06T10:40:26.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.954+0000 7f91390a7700 1 --2- 172.21.15.92:0/2273908412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9134159950 0x7f91340ccdb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f912c002000 tx=0x7f912c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:26.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.954+0000 7f9128ff9700 1 -- 172.21.15.92:0/2273908412 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c006290 con 0x7f9134159950 2023-10-06T10:40:26.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.954+0000 7f9128ff9700 1 -- 172.21.15.92:0/2273908412 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c00bba0 con 0x7f9134159950 2023-10-06T10:40:26.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.954+0000 7f913ab0a700 1 -- 172.21.15.92:0/2273908412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91340c4840 con 0x7f9134159950 2023-10-06T10:40:26.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.954+0000 7f9128ff9700 1 -- 172.21.15.92:0/2273908412 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c00bd30 con 0x7f9134159950 2023-10-06T10:40:26.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.954+0000 7f913ab0a700 1 -- 172.21.15.92:0/2273908412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9134147af0 con 0x7f9134159950 2023-10-06T10:40:26.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.955+0000 7f913ab0a700 1 -- 172.21.15.92:0/2273908412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91340c46a0 con 0x7f9134159950 2023-10-06T10:40:26.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.956+0000 7f9128ff9700 1 -- 172.21.15.92:0/2273908412 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f912c023020 con 0x7f9134159950 2023-10-06T10:40:26.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.956+0000 7f9128ff9700 1 --2- 172.21.15.92:0/2273908412 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9120066f30 0x7f91200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:26.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.956+0000 7f9128ff9700 1 -- 172.21.15.92:0/2273908412 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f912c086240 con 0x7f9134159950 2023-10-06T10:40:26.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.956+0000 7f91388a6700 1 --2- 172.21.15.92:0/2273908412 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9120066f30 0x7f91200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:26.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.957+0000 7f91388a6700 1 --2- 172.21.15.92:0/2273908412 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9120066f30 0x7f91200693e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9130002230 tx=0x7f9130001a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:26.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:26.960+0000 7f9128ff9700 1 -- 172.21.15.92:0/2273908412 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91340c46a0 con 0x7f9134159950 2023-10-06T10:40:27.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.221+0000 7f913ab0a700 1 -- 172.21.15.92:0/2273908412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]} v 0) v1 -- 0x7f91340c46a0 con 0x7f9134159950 2023-10-06T10:40:27.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.492+0000 7f9128ff9700 1 -- 172.21.15.92:0/2273908412 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]}]=0 v125) v1 ==== 136+0+0 (secure 0 0 0) 0x7f91340c46a0 con 0x7f9134159950 2023-10-06T10:40:27.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.494+0000 7f913ab0a700 1 -- 172.21.15.92:0/2273908412 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9120066f30 msgr2=0x7f91200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:27.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.494+0000 7f913ab0a700 1 --2- 172.21.15.92:0/2273908412 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9120066f30 0x7f91200693e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9130002230 tx=0x7f9130001a20 comp rx=0 tx=0).stop 2023-10-06T10:40:27.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.494+0000 7f913ab0a700 1 -- 172.21.15.92:0/2273908412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9134159950 msgr2=0x7f91340ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:27.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.494+0000 7f913ab0a700 1 --2- 172.21.15.92:0/2273908412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9134159950 0x7f91340ccdb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f912c002000 tx=0x7f912c015000 comp rx=0 tx=0).stop 2023-10-06T10:40:27.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.494+0000 7f913ab0a700 1 -- 172.21.15.92:0/2273908412 shutdown_connections 2023-10-06T10:40:27.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.494+0000 7f913ab0a700 1 --2- 172.21.15.92:0/2273908412 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9120066f30 0x7f91200693e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:27.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.494+0000 7f913ab0a700 1 --2- 172.21.15.92:0/2273908412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9134159950 0x7f91340ccdb0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:27.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.494+0000 7f913ab0a700 1 --2- 172.21.15.92:0/2273908412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9134067af0 0x7f91341475b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:27.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.494+0000 7f913ab0a700 1 --2- 172.21.15.92:0/2273908412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9134159580 0x7f9134169f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:27.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.494+0000 7f913ab0a700 1 -- 172.21.15.92:0/2273908412 >> 172.21.15.92:0/2273908412 conn(0x7f9134069140 msgr2=0x7f9134156f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:27.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.494+0000 7f913ab0a700 1 -- 172.21.15.92:0/2273908412 shutdown_connections 2023-10-06T10:40:27.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.494+0000 7f913ab0a700 1 -- 172.21.15.92:0/2273908412 wait complete. 2023-10-06T10:40:27.507 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2023-10-06T10:40:27.507 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2023-10-06T10:40:27.507 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0' 2023-10-06T10:40:27.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.628+0000 7f60df06f700 1 Processor -- start 2023-10-06T10:40:27.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.629+0000 7f60df06f700 1 -- start start 2023-10-06T10:40:27.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.629+0000 7f60df06f700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f60d815cab0 0x7f60d806f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:27.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.629+0000 7f60df06f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f60d80caa90 0x7f60d80cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:27.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.629+0000 7f60df06f700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60d806ecd0 con 0x7f60d80675b0 2023-10-06T10:40:27.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.629+0000 7f60df06f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60d80cb870 con 0x7f60d80caa90 2023-10-06T10:40:27.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.629+0000 7f60df06f700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60d80cbb50 con 0x7f60d815cab0 2023-10-06T10:40:27.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.629+0000 7f60d7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f60d80caa90 0x7f60d80cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:27.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.630+0000 7f60d7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f60d80caa90 0x7f60d80cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60494/0 (socket says 172.21.15.92:60494) 2023-10-06T10:40:27.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.630+0000 7f60d7fff700 1 -- 172.21.15.92:0/49370338 learned_addr learned my addr 172.21.15.92:0/49370338 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:27.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.630+0000 7f60dce0b700 1 --2- 172.21.15.92:0/49370338 >> v2:172.21.15.124:3300/0 conn(0x7f60d815cab0 0x7f60d806f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:27.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.630+0000 7f60d7fff700 1 -- 172.21.15.92:0/49370338 >> v2:172.21.15.124:3300/0 conn(0x7f60d815cab0 msgr2=0x7f60d806f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:27.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.630+0000 7f60d7fff700 1 --2- 172.21.15.92:0/49370338 >> v2:172.21.15.124:3300/0 conn(0x7f60d815cab0 0x7f60d806f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:27.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.630+0000 7f60d7fff700 1 -- 172.21.15.92:0/49370338 >> v1:172.21.15.124:6789/0 conn(0x7f60d80675b0 legacy=0x7f60d80679a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:27.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.630+0000 7f60d7fff700 1 -- 172.21.15.92:0/49370338 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d806e680 con 0x7f60d80caa90 2023-10-06T10:40:27.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.630+0000 7f60dce0b700 1 --2- 172.21.15.92:0/49370338 >> v2:172.21.15.124:3300/0 conn(0x7f60d815cab0 0x7f60d806f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:27.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.630+0000 7f60d7fff700 1 --2- 172.21.15.92:0/49370338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60d80caa90 0x7f60d80cafb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f60d806e480 tx=0x7f60c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83f31b198b4fbfd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:27.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.630+0000 7f60d77fe700 1 -- 172.21.15.92:0/49370338 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c800b860 con 0x7f60d80caa90 2023-10-06T10:40:27.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.630+0000 7f60d77fe700 1 -- 172.21.15.92:0/49370338 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c800b9c0 con 0x7f60d80caa90 2023-10-06T10:40:27.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.630+0000 7f60d77fe700 1 -- 172.21.15.92:0/49370338 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c800bd60 con 0x7f60d80caa90 2023-10-06T10:40:27.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.631+0000 7f60df06f700 1 -- 172.21.15.92:0/49370338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60d80caa90 msgr2=0x7f60d80cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:27.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.631+0000 7f60df06f700 1 --2- 172.21.15.92:0/49370338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60d80caa90 0x7f60d80cafb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f60d806e480 tx=0x7f60c8009250 comp rx=0 tx=0).stop 2023-10-06T10:40:27.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.631+0000 7f60df06f700 1 -- 172.21.15.92:0/49370338 shutdown_connections 2023-10-06T10:40:27.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.631+0000 7f60df06f700 1 --2- 172.21.15.92:0/49370338 >> v2:172.21.15.124:3300/0 conn(0x7f60d815cab0 0x7f60d806f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:27.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.631+0000 7f60df06f700 1 --2- 172.21.15.92:0/49370338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60d80caa90 0x7f60d80cafb0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:27.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.631+0000 7f60df06f700 1 -- 172.21.15.92:0/49370338 >> 172.21.15.92:0/49370338 conn(0x7f60d80cdeb0 msgr2=0x7f60d80c9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:27.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.631+0000 7f60df06f700 1 -- 172.21.15.92:0/49370338 shutdown_connections 2023-10-06T10:40:27.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.631+0000 7f60df06f700 1 -- 172.21.15.92:0/49370338 wait complete. 2023-10-06T10:40:27.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.632+0000 7f60df06f700 1 Processor -- start 2023-10-06T10:40:27.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.632+0000 7f60df06f700 1 -- start start 2023-10-06T10:40:27.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.633+0000 7f60df06f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60d80675b0 0x7f60d81556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:27.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.633+0000 7f60df06f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60d80caa90 0x7f60d806bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:27.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.633+0000 7f60df06f700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60d815cab0 0x7f60d806c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:27.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.633+0000 7f60df06f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d8169fe0 con 0x7f60d80caa90 2023-10-06T10:40:27.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.633+0000 7f60df06f700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d816a2c0 con 0x7f60d815cab0 2023-10-06T10:40:27.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.633+0000 7f60df06f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d816a5a0 con 0x7f60d80675b0 2023-10-06T10:40:27.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.633+0000 7f60d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60d80caa90 0x7f60d806bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:27.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.633+0000 7f60d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60d80caa90 0x7f60d806bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60506/0 (socket says 172.21.15.92:60506) 2023-10-06T10:40:27.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.633+0000 7f60d7fff700 1 -- 172.21.15.92:0/555538283 learned_addr learned my addr 172.21.15.92:0/555538283 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:27.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.633+0000 7f60dce0b700 1 --2- 172.21.15.92:0/555538283 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60d80675b0 0x7f60d81556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:27.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.633+0000 7f60dd60c700 1 --2- 172.21.15.92:0/555538283 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60d815cab0 0x7f60d806c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:27.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.633+0000 7f60d7fff700 1 -- 172.21.15.92:0/555538283 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60d80675b0 msgr2=0x7f60d81556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:27.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.633+0000 7f60d7fff700 1 --2- 172.21.15.92:0/555538283 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60d80675b0 0x7f60d81556b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:27.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.633+0000 7f60d7fff700 1 -- 172.21.15.92:0/555538283 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60d815cab0 msgr2=0x7f60d806c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:27.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.633+0000 7f60d7fff700 1 --2- 172.21.15.92:0/555538283 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60d815cab0 0x7f60d806c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:27.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.633+0000 7f60d7fff700 1 -- 172.21.15.92:0/555538283 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c80094f0 con 0x7f60d80caa90 2023-10-06T10:40:27.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.634+0000 7f60d7fff700 1 --2- 172.21.15.92:0/555538283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60d80caa90 0x7f60d806bb90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f60d806df20 tx=0x7f60c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:27.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.634+0000 7f60d4ff9700 1 -- 172.21.15.92:0/555538283 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c8011e60 con 0x7f60d80caa90 2023-10-06T10:40:27.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.634+0000 7f60d4ff9700 1 -- 172.21.15.92:0/555538283 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c801a630 con 0x7f60d80caa90 2023-10-06T10:40:27.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.634+0000 7f60d4ff9700 1 -- 172.21.15.92:0/555538283 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c801a9f0 con 0x7f60d80caa90 2023-10-06T10:40:27.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.634+0000 7f60df06f700 1 -- 172.21.15.92:0/555538283 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d816a880 con 0x7f60d80caa90 2023-10-06T10:40:27.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.634+0000 7f60df06f700 1 -- 172.21.15.92:0/555538283 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d8155bf0 con 0x7f60d80caa90 2023-10-06T10:40:27.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.635+0000 7f60df06f700 1 -- 172.21.15.92:0/555538283 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d8064a90 con 0x7f60d80caa90 2023-10-06T10:40:27.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.635+0000 7f60d4ff9700 1 -- 172.21.15.92:0/555538283 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f60c802d440 con 0x7f60d80caa90 2023-10-06T10:40:27.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.635+0000 7f60d4ff9700 1 --2- 172.21.15.92:0/555538283 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f60b8066fb0 0x7f60b8069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:27.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.636+0000 7f60d4ff9700 1 -- 172.21.15.92:0/555538283 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f60c80860b0 con 0x7f60d80caa90 2023-10-06T10:40:27.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.636+0000 7f60dce0b700 1 --2- 172.21.15.92:0/555538283 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f60b8066fb0 0x7f60b8069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:27.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.636+0000 7f60dce0b700 1 --2- 172.21.15.92:0/555538283 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f60b8066fb0 0x7f60b8069460 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f60d000d380 tx=0x7f60d0007500 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:27.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.640+0000 7f60d4ff9700 1 -- 172.21.15.92:0/555538283 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f60c80532b0 con 0x7f60d80caa90 2023-10-06T10:40:27.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:27.901+0000 7f60df06f700 1 -- 172.21.15.92:0/555538283 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]} v 0) v1 -- 0x7f60d806e960 con 0x7f60d80caa90 2023-10-06T10:40:28.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.499+0000 7f60d4ff9700 1 -- 172.21.15.92:0/555538283 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]}]=0 v126) v1 ==== 146+0+0 (secure 0 0 0) 0x7f60c80568d0 con 0x7f60d80caa90 2023-10-06T10:40:28.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.501+0000 7f60df06f700 1 -- 172.21.15.92:0/555538283 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f60b8066fb0 msgr2=0x7f60b8069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:28.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.501+0000 7f60df06f700 1 --2- 172.21.15.92:0/555538283 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f60b8066fb0 0x7f60b8069460 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f60d000d380 tx=0x7f60d0007500 comp rx=0 tx=0).stop 2023-10-06T10:40:28.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.501+0000 7f60df06f700 1 -- 172.21.15.92:0/555538283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60d80caa90 msgr2=0x7f60d806bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:28.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.501+0000 7f60df06f700 1 --2- 172.21.15.92:0/555538283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60d80caa90 0x7f60d806bb90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f60d806df20 tx=0x7f60c8017000 comp rx=0 tx=0).stop 2023-10-06T10:40:28.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.501+0000 7f60df06f700 1 -- 172.21.15.92:0/555538283 shutdown_connections 2023-10-06T10:40:28.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.501+0000 7f60df06f700 1 --2- 172.21.15.92:0/555538283 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f60b8066fb0 0x7f60b8069460 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:28.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.501+0000 7f60df06f700 1 --2- 172.21.15.92:0/555538283 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60d815cab0 0x7f60d806c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:28.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.501+0000 7f60df06f700 1 --2- 172.21.15.92:0/555538283 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60d80675b0 0x7f60d81556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:28.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.501+0000 7f60df06f700 1 --2- 172.21.15.92:0/555538283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60d80caa90 0x7f60d806bb90 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:28.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.501+0000 7f60df06f700 1 -- 172.21.15.92:0/555538283 >> 172.21.15.92:0/555538283 conn(0x7f60d80cdeb0 msgr2=0x7f60d80c28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:28.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.501+0000 7f60df06f700 1 -- 172.21.15.92:0/555538283 shutdown_connections 2023-10-06T10:40:28.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.502+0000 7f60df06f700 1 -- 172.21.15.92:0/555538283 wait complete. 2023-10-06T10:40:28.515 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T10:40:28.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.632+0000 7f431e55a700 1 Processor -- start 2023-10-06T10:40:28.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.632+0000 7f431e55a700 1 -- start start 2023-10-06T10:40:28.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.633+0000 7f431e55a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43180ccc70 0x7f43180c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:28.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.633+0000 7f431e55a700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f43180c3f70 0x7f43180c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:28.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.633+0000 7f431e55a700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43180c43d0 con 0x7f43180b0520 2023-10-06T10:40:28.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.633+0000 7f431e55a700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43180cdd30 con 0x7f43180c3f70 2023-10-06T10:40:28.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.633+0000 7f431e55a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43180cd120 con 0x7f43180ccc70 2023-10-06T10:40:28.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.633+0000 7f431dd59700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f43180c3f70 0x7f43180c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:28.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.633+0000 7f431cd57700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43180ccc70 0x7f43180c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:28.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.633+0000 7f431d558700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f43180b0520 0x7f43180cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:50996/0 (socket says 172.21.15.92:50996) 2023-10-06T10:40:28.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.633+0000 7f431dd59700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f43180c3f70 0x7f43180c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:46084/0 (socket says 172.21.15.92:46084) 2023-10-06T10:40:28.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.633+0000 7f431d558700 1 -- 172.21.15.92:0/1890725629 learned_addr learned my addr 172.21.15.92:0/1890725629 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:28.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.633+0000 7f431cd57700 1 -- 172.21.15.92:0/1890725629 >> v2:172.21.15.124:3300/0 conn(0x7f43180c3f70 msgr2=0x7f43180c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:28.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.633+0000 7f431cd57700 1 --2- 172.21.15.92:0/1890725629 >> v2:172.21.15.124:3300/0 conn(0x7f43180c3f70 0x7f43180c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:28.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.633+0000 7f431cd57700 1 -- 172.21.15.92:0/1890725629 >> v1:172.21.15.124:6789/0 conn(0x7f43180b0520 legacy=0x7f43180cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:28.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.633+0000 7f431cd57700 1 -- 172.21.15.92:0/1890725629 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43180cd400 con 0x7f43180ccc70 2023-10-06T10:40:28.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.633+0000 7f431dd59700 1 --2- 172.21.15.92:0/1890725629 >> v2:172.21.15.124:3300/0 conn(0x7f43180c3f70 0x7f43180c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:40:28.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.634+0000 7f431cd57700 1 --2- 172.21.15.92:0/1890725629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43180ccc70 0x7f43180c3a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f430c0098d0 tx=0x7f430c009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec67fd360ca5983d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:28.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.634+0000 7f4317fff700 1 -- 172.21.15.92:0/1890725629 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c005c10 con 0x7f43180ccc70 2023-10-06T10:40:28.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.634+0000 7f4317fff700 1 -- 172.21.15.92:0/1890725629 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430c005d70 con 0x7f43180ccc70 2023-10-06T10:40:28.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.634+0000 7f4317fff700 1 -- 172.21.15.92:0/1890725629 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c006280 con 0x7f43180ccc70 2023-10-06T10:40:28.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.634+0000 7f431e55a700 1 -- 172.21.15.92:0/1890725629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43180ccc70 msgr2=0x7f43180c3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:28.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.634+0000 7f431e55a700 1 --2- 172.21.15.92:0/1890725629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43180ccc70 0x7f43180c3a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f430c0098d0 tx=0x7f430c009120 comp rx=0 tx=0).stop 2023-10-06T10:40:28.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.634+0000 7f431e55a700 1 -- 172.21.15.92:0/1890725629 shutdown_connections 2023-10-06T10:40:28.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.634+0000 7f431e55a700 1 --2- 172.21.15.92:0/1890725629 >> v2:172.21.15.124:3300/0 conn(0x7f43180c3f70 0x7f43180c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:28.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.634+0000 7f431e55a700 1 --2- 172.21.15.92:0/1890725629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43180ccc70 0x7f43180c3a30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:28.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.634+0000 7f431e55a700 1 -- 172.21.15.92:0/1890725629 >> 172.21.15.92:0/1890725629 conn(0x7f4318096a30 msgr2=0x7f4318096e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:28.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.634+0000 7f431e55a700 1 -- 172.21.15.92:0/1890725629 shutdown_connections 2023-10-06T10:40:28.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.634+0000 7f431e55a700 1 -- 172.21.15.92:0/1890725629 wait complete. 2023-10-06T10:40:28.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.636+0000 7f431e55a700 1 Processor -- start 2023-10-06T10:40:28.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.636+0000 7f431e55a700 1 -- start start 2023-10-06T10:40:28.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.636+0000 7f431e55a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43180b0520 0x7f431816e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:28.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.636+0000 7f431e55a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43180c3f70 0x7f43181697b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:28.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.636+0000 7f431e55a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43180ccc70 0x7f43180b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:28.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.636+0000 7f431e55a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431815ac00 con 0x7f43180c3f70 2023-10-06T10:40:28.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.636+0000 7f431e55a700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431816e570 con 0x7f43180b0520 2023-10-06T10:40:28.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.636+0000 7f431e55a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43180c65c0 con 0x7f43180ccc70 2023-10-06T10:40:28.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.637+0000 7f431d558700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43180b0520 0x7f431816e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:28.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.637+0000 7f431cd57700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43180c3f70 0x7f43181697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:28.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.637+0000 7f431d558700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43180b0520 0x7f431816e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:46086/0 (socket says 172.21.15.92:46086) 2023-10-06T10:40:28.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.637+0000 7f431d558700 1 -- 172.21.15.92:0/3851972790 learned_addr learned my addr 172.21.15.92:0/3851972790 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:28.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.637+0000 7f431dd59700 1 --2- 172.21.15.92:0/3851972790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43180ccc70 0x7f43180b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:28.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.637+0000 7f431d558700 1 -- 172.21.15.92:0/3851972790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43180ccc70 msgr2=0x7f43180b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:28.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.637+0000 7f431d558700 1 --2- 172.21.15.92:0/3851972790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43180ccc70 0x7f43180b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:28.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.637+0000 7f431d558700 1 -- 172.21.15.92:0/3851972790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43180c3f70 msgr2=0x7f43181697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:28.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.637+0000 7f431d558700 1 --2- 172.21.15.92:0/3851972790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43180c3f70 0x7f43181697b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:28.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.637+0000 7f431d558700 1 -- 172.21.15.92:0/3851972790 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4308009a20 con 0x7f43180b0520 2023-10-06T10:40:28.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.637+0000 7f431cd57700 1 --2- 172.21.15.92:0/3851972790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43180c3f70 0x7f43181697b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:40:28.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.637+0000 7f431d558700 1 --2- 172.21.15.92:0/3851972790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43180b0520 0x7f431816e030 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f431000c560 tx=0x7f4310001020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:28.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.637+0000 7f43157fa700 1 -- 172.21.15.92:0/3851972790 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4310005b30 con 0x7f43180b0520 2023-10-06T10:40:28.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.637+0000 7f43157fa700 1 -- 172.21.15.92:0/3851972790 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4310005cb0 con 0x7f43180b0520 2023-10-06T10:40:28.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.637+0000 7f431e55a700 1 -- 172.21.15.92:0/3851972790 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430c0093c0 con 0x7f43180b0520 2023-10-06T10:40:28.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.637+0000 7f43157fa700 1 -- 172.21.15.92:0/3851972790 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4310007320 con 0x7f43180b0520 2023-10-06T10:40:28.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.637+0000 7f431e55a700 1 -- 172.21.15.92:0/3851972790 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431815a350 con 0x7f43180b0520 2023-10-06T10:40:28.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.638+0000 7f431e55a700 1 -- 172.21.15.92:0/3851972790 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431805f230 con 0x7f43180b0520 2023-10-06T10:40:28.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.639+0000 7f43157fa700 1 -- 172.21.15.92:0/3851972790 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4310007690 con 0x7f43180b0520 2023-10-06T10:40:28.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.639+0000 7f43157fa700 1 --2- 172.21.15.92:0/3851972790 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4304066f30 0x7f43040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:28.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.639+0000 7f43157fa700 1 -- 172.21.15.92:0/3851972790 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f43100837c0 con 0x7f43180b0520 2023-10-06T10:40:28.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.639+0000 7f431cd57700 1 --2- 172.21.15.92:0/3851972790 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4304066f30 0x7f43040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:28.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.640+0000 7f431cd57700 1 --2- 172.21.15.92:0/3851972790 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4304066f30 0x7f43040693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f430c009160 tx=0x7f430c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:28.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.644+0000 7f43157fa700 1 -- 172.21.15.92:0/3851972790 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4310050aa0 con 0x7f43180b0520 2023-10-06T10:40:28.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.827+0000 7f431e55a700 1 -- 172.21.15.92:0/3851972790 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f43180c43d0 con 0x7f43180b0520 2023-10-06T10:40:28.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.834+0000 7f43157fa700 1 -- 172.21.15.92:0/3851972790 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f43180c43d0 con 0x7f43180b0520 2023-10-06T10:40:28.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.836+0000 7f431e55a700 1 -- 172.21.15.92:0/3851972790 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4304066f30 msgr2=0x7f43040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:28.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.836+0000 7f431e55a700 1 --2- 172.21.15.92:0/3851972790 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4304066f30 0x7f43040693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f430c009160 tx=0x7f430c016000 comp rx=0 tx=0).stop 2023-10-06T10:40:28.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.836+0000 7f431e55a700 1 -- 172.21.15.92:0/3851972790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43180b0520 msgr2=0x7f431816e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:28.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.836+0000 7f431e55a700 1 --2- 172.21.15.92:0/3851972790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43180b0520 0x7f431816e030 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f431000c560 tx=0x7f4310001020 comp rx=0 tx=0).stop 2023-10-06T10:40:28.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.836+0000 7f431e55a700 1 -- 172.21.15.92:0/3851972790 shutdown_connections 2023-10-06T10:40:28.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.836+0000 7f431e55a700 1 --2- 172.21.15.92:0/3851972790 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4304066f30 0x7f43040693e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:28.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.836+0000 7f431e55a700 1 --2- 172.21.15.92:0/3851972790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43180ccc70 0x7f43180b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:28.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.836+0000 7f431e55a700 1 --2- 172.21.15.92:0/3851972790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43180b0520 0x7f431816e030 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:28.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.836+0000 7f431e55a700 1 --2- 172.21.15.92:0/3851972790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43180c3f70 0x7f43181697b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:28.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.836+0000 7f431e55a700 1 -- 172.21.15.92:0/3851972790 >> 172.21.15.92:0/3851972790 conn(0x7f4318096a30 msgr2=0x7f43180c9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:28.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.836+0000 7f431e55a700 1 -- 172.21.15.92:0/3851972790 shutdown_connections 2023-10-06T10:40:28.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:28.836+0000 7f431e55a700 1 -- 172.21.15.92:0/3851972790 wait complete. 2023-10-06T10:40:28.849 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T10:40:28.849 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:40:28.884 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:40:28.927 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T10:40:28.927 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T10:40:28.927 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:40:29.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.041+0000 7f96b07f1700 1 Processor -- start 2023-10-06T10:40:29.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.041+0000 7f96b07f1700 1 -- start start 2023-10-06T10:40:29.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.042+0000 7f96b07f1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f96a8159580 0x7f96a806e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.042+0000 7f96b07f1700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f96a8067cc0 0x7f96a80b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.042+0000 7f96b07f1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96a80cde30 con 0x7f96a806e9a0 2023-10-06T10:40:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.042+0000 7f96b07f1700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96a806ef20 con 0x7f96a8159580 2023-10-06T10:40:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.042+0000 7f96b07f1700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96a806f200 con 0x7f96a8067cc0 2023-10-06T10:40:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.042+0000 7f96ae58d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f96a8159580 0x7f96a806e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.042+0000 7f96add8c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f96a806e9a0 0x7f96a80675b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50984/0 (socket says 172.21.15.92:50984) 2023-10-06T10:40:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.042+0000 7f96add8c700 1 -- 172.21.15.92:0/2379238165 learned_addr learned my addr 172.21.15.92:0/2379238165 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.043+0000 7f96aed8e700 1 --2- 172.21.15.92:0/2379238165 >> v2:172.21.15.124:3300/0 conn(0x7f96a8067cc0 0x7f96a80b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.043+0000 7f96ae58d700 1 -- 172.21.15.92:0/2379238165 >> v2:172.21.15.124:3300/0 conn(0x7f96a8067cc0 msgr2=0x7f96a80b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.043+0000 7f96ae58d700 1 --2- 172.21.15.92:0/2379238165 >> v2:172.21.15.124:3300/0 conn(0x7f96a8067cc0 0x7f96a80b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.043+0000 7f96ae58d700 1 -- 172.21.15.92:0/2379238165 >> v1:172.21.15.92:6789/0 conn(0x7f96a806e9a0 legacy=0x7f96a80675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.043+0000 7f96ae58d700 1 -- 172.21.15.92:0/2379238165 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a80b0150 con 0x7f96a8159580 2023-10-06T10:40:29.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.043+0000 7f96ad58b700 1 -- 172.21.15.92:0/2379238165 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810434445 0 0) 0x7f96a80cde30 con 0x7f96a806e9a0 2023-10-06T10:40:29.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.043+0000 7f96aed8e700 1 --2- 172.21.15.92:0/2379238165 >> v2:172.21.15.124:3300/0 conn(0x7f96a8067cc0 0x7f96a80b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:29.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.043+0000 7f96ae58d700 1 --2- 172.21.15.92:0/2379238165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96a8159580 0x7f96a806e460 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f96a80b3290 tx=0x7f96a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbf2ab2c21e6d7b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:29.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.043+0000 7f96ad58b700 1 -- 172.21.15.92:0/2379238165 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a400b6b0 con 0x7f96a8159580 2023-10-06T10:40:29.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.043+0000 7f96ad58b700 1 -- 172.21.15.92:0/2379238165 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a400b810 con 0x7f96a8159580 2023-10-06T10:40:29.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.043+0000 7f96ad58b700 1 -- 172.21.15.92:0/2379238165 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a400bbb0 con 0x7f96a8159580 2023-10-06T10:40:29.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.043+0000 7f96b07f1700 1 -- 172.21.15.92:0/2379238165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96a8159580 msgr2=0x7f96a806e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:29.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.043+0000 7f96b07f1700 1 --2- 172.21.15.92:0/2379238165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96a8159580 0x7f96a806e460 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f96a80b3290 tx=0x7f96a40091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:29.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.044+0000 7f96b07f1700 1 -- 172.21.15.92:0/2379238165 shutdown_connections 2023-10-06T10:40:29.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.044+0000 7f96b07f1700 1 --2- 172.21.15.92:0/2379238165 >> v2:172.21.15.124:3300/0 conn(0x7f96a8067cc0 0x7f96a80b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:29.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.044+0000 7f96b07f1700 1 --2- 172.21.15.92:0/2379238165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96a8159580 0x7f96a806e460 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:29.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.044+0000 7f96b07f1700 1 -- 172.21.15.92:0/2379238165 >> 172.21.15.92:0/2379238165 conn(0x7f96a8069140 msgr2=0x7f96a80cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:29.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.044+0000 7f96b07f1700 1 -- 172.21.15.92:0/2379238165 shutdown_connections 2023-10-06T10:40:29.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.044+0000 7f96b07f1700 1 -- 172.21.15.92:0/2379238165 wait complete. 2023-10-06T10:40:29.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.045+0000 7f96b07f1700 1 Processor -- start 2023-10-06T10:40:29.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.045+0000 7f96b07f1700 1 -- start start 2023-10-06T10:40:29.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.045+0000 7f96b07f1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96a8067cc0 0x7f96a816a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:29.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.045+0000 7f96b07f1700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96a806e9a0 0x7f96a815aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:29.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.045+0000 7f96b07f1700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96a8159580 0x7f96a80c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:29.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.045+0000 7f96b07f1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a8157be0 con 0x7f96a8067cc0 2023-10-06T10:40:29.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.045+0000 7f96b07f1700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a815c470 con 0x7f96a8159580 2023-10-06T10:40:29.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.045+0000 7f96b07f1700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a815b3d0 con 0x7f96a806e9a0 2023-10-06T10:40:29.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.045+0000 7f96ae58d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96a8067cc0 0x7f96a816a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:29.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.045+0000 7f96ae58d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96a8067cc0 0x7f96a816a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60540/0 (socket says 172.21.15.92:60540) 2023-10-06T10:40:29.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.045+0000 7f96ae58d700 1 -- 172.21.15.92:0/3720158904 learned_addr learned my addr 172.21.15.92:0/3720158904 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:29.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.046+0000 7f96aed8e700 1 --2- 172.21.15.92:0/3720158904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96a8159580 0x7f96a80c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:29.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.046+0000 7f96add8c700 1 --2- 172.21.15.92:0/3720158904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96a806e9a0 0x7f96a815aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:29.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.046+0000 7f96ae58d700 1 -- 172.21.15.92:0/3720158904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96a806e9a0 msgr2=0x7f96a815aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:29.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.046+0000 7f96ae58d700 1 --2- 172.21.15.92:0/3720158904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96a806e9a0 0x7f96a815aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:29.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.046+0000 7f96ae58d700 1 -- 172.21.15.92:0/3720158904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96a8159580 msgr2=0x7f96a80c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:29.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.046+0000 7f96ae58d700 1 --2- 172.21.15.92:0/3720158904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96a8159580 0x7f96a80c65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:29.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.046+0000 7f96ae58d700 1 -- 172.21.15.92:0/3720158904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a4009420 con 0x7f96a8067cc0 2023-10-06T10:40:29.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.046+0000 7f96add8c700 1 --2- 172.21.15.92:0/3720158904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96a806e9a0 0x7f96a815aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:29.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.046+0000 7f96ae58d700 1 --2- 172.21.15.92:0/3720158904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96a8067cc0 0x7f96a816a4b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f96a400b0e0 tx=0x7f96a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:29.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.046+0000 7f969e7fc700 1 -- 172.21.15.92:0/3720158904 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a400bb90 con 0x7f96a8067cc0 2023-10-06T10:40:29.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.046+0000 7f969e7fc700 1 -- 172.21.15.92:0/3720158904 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a4011420 con 0x7f96a8067cc0 2023-10-06T10:40:29.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.046+0000 7f96b07f1700 1 -- 172.21.15.92:0/3720158904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a815b6b0 con 0x7f96a8067cc0 2023-10-06T10:40:29.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.046+0000 7f969e7fc700 1 -- 172.21.15.92:0/3720158904 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a40117a0 con 0x7f96a8067cc0 2023-10-06T10:40:29.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.046+0000 7f96b07f1700 1 -- 172.21.15.92:0/3720158904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a815bcf0 con 0x7f96a8067cc0 2023-10-06T10:40:29.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.047+0000 7f96b07f1700 1 -- 172.21.15.92:0/3720158904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a8064a90 con 0x7f96a8067cc0 2023-10-06T10:40:29.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.047+0000 7f969e7fc700 1 -- 172.21.15.92:0/3720158904 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f96a4011b50 con 0x7f96a8067cc0 2023-10-06T10:40:29.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.048+0000 7f969e7fc700 1 --2- 172.21.15.92:0/3720158904 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f968c066fb0 0x7f968c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:29.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.048+0000 7f969e7fc700 1 -- 172.21.15.92:0/3720158904 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f96a4086170 con 0x7f96a8067cc0 2023-10-06T10:40:29.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.048+0000 7f96add8c700 1 --2- 172.21.15.92:0/3720158904 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f968c066fb0 0x7f968c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:29.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.048+0000 7f96add8c700 1 --2- 172.21.15.92:0/3720158904 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f968c066fb0 0x7f968c069460 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f96980011e0 tx=0x7f9698009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:29.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.052+0000 7f969e7fc700 1 -- 172.21.15.92:0/3720158904 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96a4053370 con 0x7f96a8067cc0 2023-10-06T10:40:29.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.275+0000 7f96b07f1700 1 -- 172.21.15.92:0/3720158904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96a80b0430 con 0x7f96a8067cc0 2023-10-06T10:40:29.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.275+0000 7f969e7fc700 1 -- 172.21.15.92:0/3720158904 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f96a4015de0 con 0x7f96a8067cc0 2023-10-06T10:40:29.276 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:29.276 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2525196781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2525196781},{"type":"v1","addr":"172.21.15.124:6835","nonce":2525196781}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4991,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1457627327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1457627327},{"type":"v1","addr":"172.21.15.92:6837","nonce":1457627327}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5413,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2183145276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2183145276},{"type":"v1","addr":"172.21.15.124:6834","nonce":2183145276}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5661,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1056787971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1056787971},{"type":"v1","addr":"172.21.15.92:6836","nonce":1056787971}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[]} 2023-10-06T10:40:29.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.277+0000 7f96b07f1700 1 -- 172.21.15.92:0/3720158904 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f968c066fb0 msgr2=0x7f968c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:29.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.277+0000 7f96b07f1700 1 --2- 172.21.15.92:0/3720158904 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f968c066fb0 0x7f968c069460 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f96980011e0 tx=0x7f9698009040 comp rx=0 tx=0).stop 2023-10-06T10:40:29.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.277+0000 7f96b07f1700 1 -- 172.21.15.92:0/3720158904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96a8067cc0 msgr2=0x7f96a816a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:29.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.277+0000 7f96b07f1700 1 --2- 172.21.15.92:0/3720158904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96a8067cc0 0x7f96a816a4b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f96a400b0e0 tx=0x7f96a4017000 comp rx=0 tx=0).stop 2023-10-06T10:40:29.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.277+0000 7f96b07f1700 1 -- 172.21.15.92:0/3720158904 shutdown_connections 2023-10-06T10:40:29.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.277+0000 7f96b07f1700 1 --2- 172.21.15.92:0/3720158904 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f968c066fb0 0x7f968c069460 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:29.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.277+0000 7f96b07f1700 1 --2- 172.21.15.92:0/3720158904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96a8159580 0x7f96a80c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:29.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.277+0000 7f96b07f1700 1 --2- 172.21.15.92:0/3720158904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96a8067cc0 0x7f96a816a4b0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:29.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.277+0000 7f96b07f1700 1 --2- 172.21.15.92:0/3720158904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96a806e9a0 0x7f96a815aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:29.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.277+0000 7f96b07f1700 1 -- 172.21.15.92:0/3720158904 >> 172.21.15.92:0/3720158904 conn(0x7f96a8069140 msgr2=0x7f96a81585e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:29.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.277+0000 7f96b07f1700 1 -- 172.21.15.92:0/3720158904 shutdown_connections 2023-10-06T10:40:29.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.277+0000 7f96b07f1700 1 -- 172.21.15.92:0/3720158904 wait complete. 2023-10-06T10:40:29.280 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 26 2023-10-06T10:40:29.291 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T10:40:29.291 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T10:40:29.291 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T10:40:29.291 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:40:29.292 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T10:40:29.292 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T10:40:29.292 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T10:40:29.293 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:40:29.293 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T10:40:29.293 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T10:40:29.293 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T10:40:29.294 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:40:29.294 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T10:40:29.294 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T10:40:29.294 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T10:40:29.295 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:40:29.301 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T10:40:29.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T10:40:29.302 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T10:40:29.302 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T10:40:29.304 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T10:40:29.304 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T10:40:29.305 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T10:40:29.305 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T10:40:29.344 INFO:tasks.ceph.mds.a:Started 2023-10-06T10:40:29.344 DEBUG:teuthology.parallel:result is None 2023-10-06T10:40:29.345 INFO:tasks.ceph.mds.c:Started 2023-10-06T10:40:29.345 DEBUG:teuthology.parallel:result is None 2023-10-06T10:40:29.347 INFO:tasks.ceph.mds.b:Started 2023-10-06T10:40:29.347 DEBUG:teuthology.parallel:result is None 2023-10-06T10:40:29.348 INFO:tasks.ceph.mds.d:Started 2023-10-06T10:40:29.348 DEBUG:teuthology.parallel:result is None 2023-10-06T10:40:29.348 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T10:40:29.416 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T10:40:29.417 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T10:40:29.425 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-06T10:40:29.425 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-06T10:40:29.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.468+0000 7fafa28fe700 1 Processor -- start 2023-10-06T10:40:29.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.468+0000 7fafa28fe700 1 -- start start 2023-10-06T10:40:29.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.469+0000 7fafa28fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7faf9c15cab0 0x7faf9c06f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:29.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.469+0000 7fafa28fe700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9c06ecd0 con 0x7faf9c0caa90 2023-10-06T10:40:29.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.469+0000 7fafa28fe700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9c0cb870 con 0x7faf9c0675b0 2023-10-06T10:40:29.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.469+0000 7fafa28fe700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9c0cbb50 con 0x7faf9c15cab0 2023-10-06T10:40:29.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.469+0000 7faf9bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7faf9c15cab0 0x7faf9c06f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:29.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.469+0000 7faf9b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7faf9c0caa90 0x7faf9c0cafb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51002/0 (socket says 172.21.15.92:51002) 2023-10-06T10:40:29.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.469+0000 7faf9b7fe700 1 -- 172.21.15.92:0/3969165244 learned_addr learned my addr 172.21.15.92:0/3969165244 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:29.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.470+0000 7faf9bfff700 1 -- 172.21.15.92:0/3969165244 >> v1:172.21.15.124:6789/0 conn(0x7faf9c0675b0 legacy=0x7faf9c0679a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:29.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.470+0000 7faf9bfff700 1 -- 172.21.15.92:0/3969165244 >> v1:172.21.15.92:6789/0 conn(0x7faf9c0caa90 legacy=0x7faf9c0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:29.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.470+0000 7faf9bfff700 1 -- 172.21.15.92:0/3969165244 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9c06e680 con 0x7faf9c15cab0 2023-10-06T10:40:29.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.470+0000 7faf9bfff700 1 --2- 172.21.15.92:0/3969165244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9c15cab0 0x7faf9c06f160 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7faf8c0098e0 tx=0x7faf8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5b7e79842e2e24 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:29.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.470+0000 7faf9affd700 1 -- 172.21.15.92:0/3969165244 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c009e80 con 0x7faf9c15cab0 2023-10-06T10:40:29.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.470+0000 7faf9affd700 1 -- 172.21.15.92:0/3969165244 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c00b820 con 0x7faf9c15cab0 2023-10-06T10:40:29.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.470+0000 7faf9affd700 1 -- 172.21.15.92:0/3969165244 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c00bba0 con 0x7faf9c15cab0 2023-10-06T10:40:29.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.470+0000 7fafa28fe700 1 -- 172.21.15.92:0/3969165244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9c15cab0 msgr2=0x7faf9c06f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:29.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.470+0000 7fafa28fe700 1 --2- 172.21.15.92:0/3969165244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9c15cab0 0x7faf9c06f160 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7faf8c0098e0 tx=0x7faf8c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:29.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.471+0000 7fafa28fe700 1 -- 172.21.15.92:0/3969165244 shutdown_connections 2023-10-06T10:40:29.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.471+0000 7fafa28fe700 1 --2- 172.21.15.92:0/3969165244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9c15cab0 0x7faf9c06f160 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:29.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.471+0000 7fafa28fe700 1 -- 172.21.15.92:0/3969165244 >> 172.21.15.92:0/3969165244 conn(0x7faf9c0cdeb0 msgr2=0x7faf9c0c9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:29.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.471+0000 7fafa28fe700 1 -- 172.21.15.92:0/3969165244 shutdown_connections 2023-10-06T10:40:29.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.471+0000 7fafa28fe700 1 -- 172.21.15.92:0/3969165244 wait complete. 2023-10-06T10:40:29.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.472+0000 7fafa28fe700 1 Processor -- start 2023-10-06T10:40:29.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.472+0000 7fafa28fe700 1 -- start start 2023-10-06T10:40:29.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.472+0000 7fafa28fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9c0675b0 0x7faf9c144f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:29.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.473+0000 7fafa28fe700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf9c0caa90 0x7faf9c0acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:29.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.473+0000 7fafa28fe700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf9c15cab0 0x7faf9c0b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:29.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.473+0000 7fafa28fe700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c16adf0 con 0x7faf9c0675b0 2023-10-06T10:40:29.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.473+0000 7fafa28fe700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0b0900 con 0x7faf9c15cab0 2023-10-06T10:40:29.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.473+0000 7fafa28fe700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0ad020 con 0x7faf9c0caa90 2023-10-06T10:40:29.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.473+0000 7faf9bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9c0675b0 0x7faf9c144f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:29.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.473+0000 7faf9bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9c0675b0 0x7faf9c144f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60572/0 (socket says 172.21.15.92:60572) 2023-10-06T10:40:29.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.473+0000 7faf9bfff700 1 -- 172.21.15.92:0/2748729393 learned_addr learned my addr 172.21.15.92:0/2748729393 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:29.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.473+0000 7faf9b7fe700 1 --2- 172.21.15.92:0/2748729393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf9c0caa90 0x7faf9c0acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:29.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.473+0000 7fafa0e9b700 1 --2- 172.21.15.92:0/2748729393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf9c15cab0 0x7faf9c0b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:29.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.473+0000 7faf9bfff700 1 -- 172.21.15.92:0/2748729393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf9c0caa90 msgr2=0x7faf9c0acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:29.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.473+0000 7faf9bfff700 1 --2- 172.21.15.92:0/2748729393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf9c0caa90 0x7faf9c0acae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:29.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.473+0000 7faf9bfff700 1 -- 172.21.15.92:0/2748729393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf9c15cab0 msgr2=0x7faf9c0b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:29.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.473+0000 7faf9bfff700 1 --2- 172.21.15.92:0/2748729393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf9c15cab0 0x7faf9c0b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:29.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.473+0000 7faf9bfff700 1 -- 172.21.15.92:0/2748729393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8c009420 con 0x7faf9c0675b0 2023-10-06T10:40:29.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.473+0000 7fafa0e9b700 1 --2- 172.21.15.92:0/2748729393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf9c15cab0 0x7faf9c0b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:29.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.473+0000 7faf9bfff700 1 --2- 172.21.15.92:0/2748729393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9c0675b0 0x7faf9c144f80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7faf8c0030c0 tx=0x7faf8c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:29.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.473+0000 7faf83fff700 1 -- 172.21.15.92:0/2748729393 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c01b470 con 0x7faf9c0675b0 2023-10-06T10:40:29.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.473+0000 7faf83fff700 1 -- 172.21.15.92:0/2748729393 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c01b5f0 con 0x7faf9c0675b0 2023-10-06T10:40:29.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.473+0000 7fafa28fe700 1 -- 172.21.15.92:0/2748729393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9c16ab10 con 0x7faf9c0675b0 2023-10-06T10:40:29.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.473+0000 7faf83fff700 1 -- 172.21.15.92:0/2748729393 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c01b9b0 con 0x7faf9c0675b0 2023-10-06T10:40:29.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.473+0000 7fafa28fe700 1 -- 172.21.15.92:0/2748729393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9c0b0590 con 0x7faf9c0675b0 2023-10-06T10:40:29.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.474+0000 7fafa28fe700 1 -- 172.21.15.92:0/2748729393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf9c064a90 con 0x7faf9c0675b0 2023-10-06T10:40:29.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.475+0000 7faf83fff700 1 -- 172.21.15.92:0/2748729393 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faf8c01c450 con 0x7faf9c0675b0 2023-10-06T10:40:29.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.475+0000 7faf83fff700 1 --2- 172.21.15.92:0/2748729393 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7faf88066f30 0x7faf880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:29.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.475+0000 7faf83fff700 1 -- 172.21.15.92:0/2748729393 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5784+0+0 (secure 0 0 0) 0x7faf8c0862e0 con 0x7faf9c0675b0 2023-10-06T10:40:29.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.475+0000 7faf9b7fe700 1 --2- 172.21.15.92:0/2748729393 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7faf88066f30 0x7faf880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:29.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.475+0000 7faf9b7fe700 1 --2- 172.21.15.92:0/2748729393 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7faf88066f30 0x7faf880693e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faf9c154000 tx=0x7faf90009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:29.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.479+0000 7faf83fff700 1 -- 172.21.15.92:0/2748729393 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf8c053500 con 0x7faf9c0675b0 2023-10-06T10:40:29.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:29.679+0000 7fafa28fe700 1 -- 172.21.15.92:0/2748729393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7faf9c04bb40 con 0x7faf9c0675b0 2023-10-06T10:40:30.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.524+0000 7faf83fff700 1 -- 172.21.15.92:0/2748729393 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v48) v1 ==== 97+0+0 (secure 0 0 0) 0x7faf8c02a080 con 0x7faf9c0675b0 2023-10-06T10:40:30.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.527+0000 7fafa28fe700 1 -- 172.21.15.92:0/2748729393 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7faf88066f30 msgr2=0x7faf880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:30.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.527+0000 7fafa28fe700 1 --2- 172.21.15.92:0/2748729393 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7faf88066f30 0x7faf880693e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faf9c154000 tx=0x7faf90009040 comp rx=0 tx=0).stop 2023-10-06T10:40:30.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.527+0000 7fafa28fe700 1 -- 172.21.15.92:0/2748729393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9c0675b0 msgr2=0x7faf9c144f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:30.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.527+0000 7fafa28fe700 1 --2- 172.21.15.92:0/2748729393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9c0675b0 0x7faf9c144f80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7faf8c0030c0 tx=0x7faf8c016000 comp rx=0 tx=0).stop 2023-10-06T10:40:30.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.527+0000 7fafa28fe700 1 -- 172.21.15.92:0/2748729393 shutdown_connections 2023-10-06T10:40:30.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.527+0000 7fafa28fe700 1 --2- 172.21.15.92:0/2748729393 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7faf88066f30 0x7faf880693e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:30.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.527+0000 7fafa28fe700 1 --2- 172.21.15.92:0/2748729393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf9c15cab0 0x7faf9c0b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:30.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.527+0000 7fafa28fe700 1 --2- 172.21.15.92:0/2748729393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9c0675b0 0x7faf9c144f80 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:30.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.527+0000 7fafa28fe700 1 --2- 172.21.15.92:0/2748729393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf9c0caa90 0x7faf9c0acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:30.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.527+0000 7fafa28fe700 1 -- 172.21.15.92:0/2748729393 >> 172.21.15.92:0/2748729393 conn(0x7faf9c0cdeb0 msgr2=0x7faf9c156cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:30.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.527+0000 7fafa28fe700 1 -- 172.21.15.92:0/2748729393 shutdown_connections 2023-10-06T10:40:30.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.527+0000 7fafa28fe700 1 -- 172.21.15.92:0/2748729393 wait complete. 2023-10-06T10:40:30.529 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-10-06T10:40:30.541 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T10:40:30.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.654+0000 7f7a4acab700 1 Processor -- start 2023-10-06T10:40:30.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.655+0000 7f7a4acab700 1 -- start start 2023-10-06T10:40:30.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.655+0000 7f7a4acab700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7a44159ad0 0x7f7a44159f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:30.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.655+0000 7f7a4acab700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7a440ca670 0x7f7a440caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:30.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.655+0000 7f7a4acab700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a440cafd0 con 0x7f7a440cb5f0 2023-10-06T10:40:30.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.655+0000 7f7a4acab700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a4415a490 con 0x7f7a440ca670 2023-10-06T10:40:30.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.655+0000 7f7a4acab700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a44154020 con 0x7f7a44159ad0 2023-10-06T10:40:30.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.655+0000 7f7a49248700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7a440ca670 0x7f7a440caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:30.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.655+0000 7f7a49248700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7a440ca670 0x7f7a440caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60580/0 (socket says 172.21.15.92:60580) 2023-10-06T10:40:30.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.656+0000 7f7a49248700 1 -- 172.21.15.92:0/855979662 learned_addr learned my addr 172.21.15.92:0/855979662 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:30.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.656+0000 7f7a43fff700 1 --2- 172.21.15.92:0/855979662 >> v2:172.21.15.124:3300/0 conn(0x7f7a44159ad0 0x7f7a44159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:30.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.656+0000 7f7a49248700 1 -- 172.21.15.92:0/855979662 >> v2:172.21.15.124:3300/0 conn(0x7f7a44159ad0 msgr2=0x7f7a44159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:30.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.656+0000 7f7a49248700 1 --2- 172.21.15.92:0/855979662 >> v2:172.21.15.124:3300/0 conn(0x7f7a44159ad0 0x7f7a44159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:30.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.656+0000 7f7a49248700 1 -- 172.21.15.92:0/855979662 >> v1:172.21.15.124:6789/0 conn(0x7f7a440cb5f0 legacy=0x7f7a440691a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:30.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.656+0000 7f7a49248700 1 -- 172.21.15.92:0/855979662 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a44068d10 con 0x7f7a440ca670 2023-10-06T10:40:30.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.656+0000 7f7a49248700 1 --2- 172.21.15.92:0/855979662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a440ca670 0x7f7a440caa90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7a4406f2c0 tx=0x7f7a3c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354c5104758572c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:30.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.656+0000 7f7a437fe700 1 -- 172.21.15.92:0/855979662 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c00e070 con 0x7f7a440ca670 2023-10-06T10:40:30.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.656+0000 7f7a437fe700 1 -- 172.21.15.92:0/855979662 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3c00ba40 con 0x7f7a440ca670 2023-10-06T10:40:30.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.656+0000 7f7a437fe700 1 -- 172.21.15.92:0/855979662 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c0092e0 con 0x7f7a440ca670 2023-10-06T10:40:30.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.657+0000 7f7a4acab700 1 -- 172.21.15.92:0/855979662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a440ca670 msgr2=0x7f7a440caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:30.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.657+0000 7f7a4acab700 1 --2- 172.21.15.92:0/855979662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a440ca670 0x7f7a440caa90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7a4406f2c0 tx=0x7f7a3c009840 comp rx=0 tx=0).stop 2023-10-06T10:40:30.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.657+0000 7f7a4acab700 1 -- 172.21.15.92:0/855979662 shutdown_connections 2023-10-06T10:40:30.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.657+0000 7f7a4acab700 1 --2- 172.21.15.92:0/855979662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a440ca670 0x7f7a440caa90 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:30.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.657+0000 7f7a4acab700 1 --2- 172.21.15.92:0/855979662 >> v2:172.21.15.124:3300/0 conn(0x7f7a44159ad0 0x7f7a44159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:30.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.657+0000 7f7a4acab700 1 -- 172.21.15.92:0/855979662 >> 172.21.15.92:0/855979662 conn(0x7f7a440707d0 msgr2=0x7f7a44070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:30.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.657+0000 7f7a4acab700 1 -- 172.21.15.92:0/855979662 shutdown_connections 2023-10-06T10:40:30.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.657+0000 7f7a4acab700 1 -- 172.21.15.92:0/855979662 wait complete. 2023-10-06T10:40:30.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.658+0000 7f7a4acab700 1 Processor -- start 2023-10-06T10:40:30.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.658+0000 7f7a4acab700 1 -- start start 2023-10-06T10:40:30.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a4acab700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a440ca670 0x7f7a44151d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:30.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a4acab700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a440cb5f0 0x7f7a440c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:30.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a4acab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a44159ad0 0x7f7a440c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:30.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a4acab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a441522b0 con 0x7f7a44159ad0 2023-10-06T10:40:30.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a4acab700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a440c4cf0 con 0x7f7a440cb5f0 2023-10-06T10:40:30.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a4acab700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a440c7610 con 0x7f7a440ca670 2023-10-06T10:40:30.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a49248700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a44159ad0 0x7f7a440c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:30.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a49248700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a44159ad0 0x7f7a440c6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60588/0 (socket says 172.21.15.92:60588) 2023-10-06T10:40:30.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a49248700 1 -- 172.21.15.92:0/3263915362 learned_addr learned my addr 172.21.15.92:0/3263915362 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:30.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a48a47700 1 --2- 172.21.15.92:0/3263915362 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a440ca670 0x7f7a44151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:30.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a43fff700 1 --2- 172.21.15.92:0/3263915362 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a440cb5f0 0x7f7a440c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:30.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a49248700 1 -- 172.21.15.92:0/3263915362 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a440ca670 msgr2=0x7f7a44151d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:30.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a49248700 1 --2- 172.21.15.92:0/3263915362 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a440ca670 0x7f7a44151d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:30.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a49248700 1 -- 172.21.15.92:0/3263915362 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a440cb5f0 msgr2=0x7f7a440c6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:30.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a49248700 1 --2- 172.21.15.92:0/3263915362 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a440cb5f0 0x7f7a440c6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:30.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a49248700 1 -- 172.21.15.92:0/3263915362 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3c009ae0 con 0x7f7a44159ad0 2023-10-06T10:40:30.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a48a47700 1 --2- 172.21.15.92:0/3263915362 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a440ca670 0x7f7a44151d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:30.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a43fff700 1 --2- 172.21.15.92:0/3263915362 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a440cb5f0 0x7f7a440c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:30.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a49248700 1 --2- 172.21.15.92:0/3263915362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a44159ad0 0x7f7a440c6f70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00b6c0 tx=0x7f7a3c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:30.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a40ff9700 1 -- 172.21.15.92:0/3263915362 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c008e40 con 0x7f7a44159ad0 2023-10-06T10:40:30.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a40ff9700 1 -- 172.21.15.92:0/3263915362 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3c025460 con 0x7f7a44159ad0 2023-10-06T10:40:30.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a4acab700 1 -- 172.21.15.92:0/3263915362 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4416cfb0 con 0x7f7a44159ad0 2023-10-06T10:40:30.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a40ff9700 1 -- 172.21.15.92:0/3263915362 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c025820 con 0x7f7a44159ad0 2023-10-06T10:40:30.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.659+0000 7f7a4acab700 1 -- 172.21.15.92:0/3263915362 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4416db30 con 0x7f7a44159ad0 2023-10-06T10:40:30.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.660+0000 7f7a4acab700 1 -- 172.21.15.92:0/3263915362 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4416dec0 con 0x7f7a44159ad0 2023-10-06T10:40:30.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.661+0000 7f7a40ff9700 1 -- 172.21.15.92:0/3263915362 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a3c003040 con 0x7f7a44159ad0 2023-10-06T10:40:30.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.661+0000 7f7a40ff9700 1 --2- 172.21.15.92:0/3263915362 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7a30066f30 0x7f7a300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:30.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.661+0000 7f7a40ff9700 1 -- 172.21.15.92:0/3263915362 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f7a3c085b20 con 0x7f7a44159ad0 2023-10-06T10:40:30.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.661+0000 7f7a48a47700 1 --2- 172.21.15.92:0/3263915362 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7a30066f30 0x7f7a300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:30.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.662+0000 7f7a48a47700 1 --2- 172.21.15.92:0/3263915362 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7a30066f30 0x7f7a300693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7a34001840 tx=0x7f7a34009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:30.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.665+0000 7f7a40ff9700 1 -- 172.21.15.92:0/3263915362 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a3c052e40 con 0x7f7a44159ad0 2023-10-06T10:40:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.921+0000 7f7a4acab700 1 -- 172.21.15.92:0/3263915362 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7a4416dcc0 con 0x7f7a44159ad0 2023-10-06T10:40:30.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.922+0000 7f7a40ff9700 1 -- 172.21.15.92:0/3263915362 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7a3c056460 con 0x7f7a44159ad0 2023-10-06T10:40:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:30.923 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:40:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-10-06T10:40:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:40:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-10-06T10:40:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lc28gCBAAVrjhXIawHuZEox0PE3/Rsg==", 2023-10-06T10:40:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:40:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:40:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:40:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:40:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:40:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:40:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:40:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:40:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-10-06T10:40:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lQpdOExAAm44s0n4Jekhy8AJk/o6Waw==", 2023-10-06T10:40:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:40:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:40:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:40:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:40:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:40:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:40:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:40:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:40:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-10-06T10:40:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lrCrsDhAAvp++wvvnSpZRGzvRTrGWEw==", 2023-10-06T10:40:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:40:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:40:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:40:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:40:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:40:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:40:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:40:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:40:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-10-06T10:40:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9l/DgWGhAAy3JMA5ntn/UaRkuTCyPdkg==", 2023-10-06T10:40:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:40:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:40:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:40:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:40:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:40:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:40:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:40:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:40:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-10-06T10:40:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBo4x9lDEIFMBAAc3DY1fYpY2UiY7MX7qSP5A==", 2023-10-06T10:40:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:40:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:40:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:40:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:40:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:40:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:40:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:40:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-10-06T10:40:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBr4x9lKzX8LRAApryLtFGkK6urkE8W70l5Cg==", 2023-10-06T10:40:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:40:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:40:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:40:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:40:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:40:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:40:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:40:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-10-06T10:40:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBu4x9lCMZpLhAAEBPa40Wnb820p6j+lv+PsA==", 2023-10-06T10:40:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:40:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:40:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:40:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:40:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:40:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:40:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:40:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-10-06T10:40:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBx4x9l4AHrLRAAFn6SnNkHnOiVXmmiU8qGyA==", 2023-10-06T10:40:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:40:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:40:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:40:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:40:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:40:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:40:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:40:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-10-06T10:40:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB34x9l7V7WMBAAvs5kvBP5n0Th478l/idjeQ==", 2023-10-06T10:40:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:40:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:40:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:40:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:40:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:40:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:40:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:40:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-10-06T10:40:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB64x9lfHIlLhAADCuqt+O/zPEETgoj//hwVg==", 2023-10-06T10:40:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:40:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:40:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:40:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:40:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:40:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:40:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:40:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-10-06T10:40:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB94x9lUmU7LhAAvxmtGmJxrI8TlOOD7taHOQ==", 2023-10-06T10:40:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:40:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:40:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:40:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:40:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:40:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:40:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:40:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-10-06T10:40:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQCA4x9lJE3nLBAAnREk9yNK+FhX4fnG+Za5Ug==", 2023-10-06T10:40:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:40:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:40:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:40:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:40:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:40:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:40:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:40:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-10-06T10:40:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lMePcHRAAMyftXWJWk7rwmWFkq18Rqg==", 2023-10-06T10:40:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:40:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:40:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-10-06T10:40:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T10:40:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:40:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:40:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:40:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-10-06T10:40:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lp1s0JhAAKvZ23MYjObxOK83q3UqJBQ==", 2023-10-06T10:40:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:40:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:40:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T10:40:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T10:40:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T10:40:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:40:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:40:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:40:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-10-06T10:40:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lc6o9IxAAaOY7S5rYn+OdeMNWK1UYMA==", 2023-10-06T10:40:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:40:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:40:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T10:40:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T10:40:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T10:40:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:40:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:40:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:40:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-10-06T10:40:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lwoyOKxAA811lehuQQ8MQQQzuTFAApg==", 2023-10-06T10:40:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:40:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:40:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T10:40:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T10:40:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T10:40:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:40:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:40:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:40:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-10-06T10:40:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBk4x9lONCoJhAAepF/sou/5M2GBvvfNAs5wQ==", 2023-10-06T10:40:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:40:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T10:40:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:40:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:40:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:40:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:40:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:40:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:40:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-10-06T10:40:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9ll8wgBRAAQUrHERQ3Igv1tosa+wryIg==", 2023-10-06T10:40:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:40:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T10:40:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T10:40:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:40:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:40:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:40:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:40:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-10-06T10:40:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBk4x9lgYa/OhAAyoI717Ga+Wv72srZ4FK+lQ==", 2023-10-06T10:40:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:40:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T10:40:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T10:40:30.938 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:40:30.938 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:40:30.938 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:40:30.938 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T10:40:30.938 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:40:30.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.924+0000 7f7a4acab700 1 -- 172.21.15.92:0/3263915362 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7a30066f30 msgr2=0x7f7a300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:30.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.924+0000 7f7a4acab700 1 --2- 172.21.15.92:0/3263915362 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7a30066f30 0x7f7a300693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7a34001840 tx=0x7f7a34009040 comp rx=0 tx=0).stop 2023-10-06T10:40:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.924+0000 7f7a4acab700 1 -- 172.21.15.92:0/3263915362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a44159ad0 msgr2=0x7f7a440c6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.924+0000 7f7a4acab700 1 --2- 172.21.15.92:0/3263915362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a44159ad0 0x7f7a440c6f70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00b6c0 tx=0x7f7a3c018000 comp rx=0 tx=0).stop 2023-10-06T10:40:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.924+0000 7f7a4acab700 1 -- 172.21.15.92:0/3263915362 shutdown_connections 2023-10-06T10:40:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.924+0000 7f7a4acab700 1 --2- 172.21.15.92:0/3263915362 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7a30066f30 0x7f7a300693e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.924+0000 7f7a4acab700 1 --2- 172.21.15.92:0/3263915362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a44159ad0 0x7f7a440c6f70 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.924+0000 7f7a4acab700 1 --2- 172.21.15.92:0/3263915362 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a440ca670 0x7f7a44151d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.924+0000 7f7a4acab700 1 --2- 172.21.15.92:0/3263915362 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a440cb5f0 0x7f7a440c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.924+0000 7f7a4acab700 1 -- 172.21.15.92:0/3263915362 >> 172.21.15.92:0/3263915362 conn(0x7f7a440707d0 msgr2=0x7f7a44152ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.924+0000 7f7a4acab700 1 -- 172.21.15.92:0/3263915362 shutdown_connections 2023-10-06T10:40:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:30.924+0000 7f7a4acab700 1 -- 172.21.15.92:0/3263915362 wait complete. 2023-10-06T10:40:30.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:40:31.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.054+0000 7f2d6ee1d700 1 Processor -- start 2023-10-06T10:40:31.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.054+0000 7f2d6ee1d700 1 -- start start 2023-10-06T10:40:31.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.055+0000 7f2d6ee1d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d6806e650 0x7f2d6806eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:31.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.055+0000 7f2d6ee1d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6806f000 con 0x7f2d68156500 2023-10-06T10:40:31.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.055+0000 7f2d6ee1d700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d680acd20 con 0x7f2d680ad1b0 2023-10-06T10:40:31.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.055+0000 7f2d6ee1d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6806f430 con 0x7f2d6806e650 2023-10-06T10:40:31.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.055+0000 7f2d6cbb9700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2d68156500 0x7f2d680c7030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51006/0 (socket says 172.21.15.92:51006) 2023-10-06T10:40:31.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.055+0000 7f2d6cbb9700 1 -- 172.21.15.92:0/2597635332 learned_addr learned my addr 172.21.15.92:0/2597635332 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:31.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d67fff700 1 --2- 172.21.15.92:0/2597635332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d6806e650 0x7f2d6806eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:31.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d677fe700 1 -- 172.21.15.92:0/2597635332 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389607641 0 0) 0x7f2d6806f000 con 0x7f2d68156500 2023-10-06T10:40:31.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d677fe700 1 -- 172.21.15.92:0/2597635332 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d58003570 con 0x7f2d68156500 2023-10-06T10:40:31.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d677fe700 1 -- 172.21.15.92:0/2597635332 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1009108202 0 0) 0x7f2d58003570 con 0x7f2d68156500 2023-10-06T10:40:31.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d677fe700 1 -- 172.21.15.92:0/2597635332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d6806e650 msgr2=0x7f2d6806eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:31.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d677fe700 1 --2- 172.21.15.92:0/2597635332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d6806e650 0x7f2d6806eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d677fe700 1 -- 172.21.15.92:0/2597635332 >> v1:172.21.15.124:6789/0 conn(0x7f2d680ad1b0 legacy=0x7f2d680ad630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:31.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d677fe700 1 -- 172.21.15.92:0/2597635332 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d681568d0 con 0x7f2d68156500 2023-10-06T10:40:31.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d677fe700 1 -- 172.21.15.92:0/2597635332 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f2d5c001470 con 0x7f2d68156500 2023-10-06T10:40:31.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d677fe700 1 -- 172.21.15.92:0/2597635332 >> v1:172.21.15.92:6789/0 conn(0x7f2d68156500 legacy=0x7f2d680c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:31.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d677fe700 1 --2- 172.21.15.92:0/2597635332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d58003a90 0x7f2d58003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:31.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d677fe700 1 --2- 172.21.15.92:0/2597635332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d6806e650 0x7f2d58004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:31.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d677fe700 1 --2- 172.21.15.92:0/2597635332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d58004d50 0x7f2d580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:31.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d677fe700 1 -- 172.21.15.92:0/2597635332 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6806f430 con 0x7f2d58003a90 2023-10-06T10:40:31.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d677fe700 1 -- 172.21.15.92:0/2597635332 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5800bb70 con 0x7f2d58004d50 2023-10-06T10:40:31.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d677fe700 1 -- 172.21.15.92:0/2597635332 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5800df40 con 0x7f2d6806e650 2023-10-06T10:40:31.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d6cbb9700 1 --2- 172.21.15.92:0/2597635332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d58003a90 0x7f2d58003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:31.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d6d3ba700 1 --2- 172.21.15.92:0/2597635332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d58004d50 0x7f2d580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:31.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d67fff700 1 --2- 172.21.15.92:0/2597635332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d6806e650 0x7f2d58004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:31.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d6cbb9700 1 -- 172.21.15.92:0/2597635332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d6806e650 msgr2=0x7f2d58004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:31.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d6cbb9700 1 --2- 172.21.15.92:0/2597635332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d6806e650 0x7f2d58004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d6cbb9700 1 -- 172.21.15.92:0/2597635332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d58004d50 msgr2=0x7f2d580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:31.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d6cbb9700 1 --2- 172.21.15.92:0/2597635332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d58004d50 0x7f2d580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.056+0000 7f2d6cbb9700 1 -- 172.21.15.92:0/2597635332 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d58003570 con 0x7f2d58003a90 2023-10-06T10:40:31.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.057+0000 7f2d6cbb9700 1 --2- 172.21.15.92:0/2597635332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d58003a90 0x7f2d58003ea0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00c3c0 tx=0x7f2d5c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5dc87adab333eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:31.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.057+0000 7f2d677fe700 1 -- 172.21.15.92:0/2597635332 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c00cc90 con 0x7f2d58003a90 2023-10-06T10:40:31.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.057+0000 7f2d677fe700 1 -- 172.21.15.92:0/2597635332 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5c00cdf0 con 0x7f2d58003a90 2023-10-06T10:40:31.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.057+0000 7f2d677fe700 1 -- 172.21.15.92:0/2597635332 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c005b50 con 0x7f2d58003a90 2023-10-06T10:40:31.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.057+0000 7f2d6ee1d700 1 -- 172.21.15.92:0/2597635332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d58003a90 msgr2=0x7f2d58003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:31.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.057+0000 7f2d6ee1d700 1 --2- 172.21.15.92:0/2597635332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d58003a90 0x7f2d58003ea0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00c3c0 tx=0x7f2d5c001e50 comp rx=0 tx=0).stop 2023-10-06T10:40:31.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.057+0000 7f2d6d3ba700 1 -- 172.21.15.92:0/2597635332 reap_dead start 2023-10-06T10:40:31.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.057+0000 7f2d6ee1d700 1 -- 172.21.15.92:0/2597635332 shutdown_connections 2023-10-06T10:40:31.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.057+0000 7f2d6ee1d700 1 -- 172.21.15.92:0/2597635332 >> 172.21.15.92:0/2597635332 conn(0x7f2d68068080 msgr2=0x7f2d680cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:31.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.057+0000 7f2d6ee1d700 1 -- 172.21.15.92:0/2597635332 shutdown_connections 2023-10-06T10:40:31.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.057+0000 7f2d6ee1d700 1 -- 172.21.15.92:0/2597635332 wait complete. 2023-10-06T10:40:31.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.058+0000 7f2d6ee1d700 1 Processor -- start 2023-10-06T10:40:31.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.058+0000 7f2d6ee1d700 1 -- start start 2023-10-06T10:40:31.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.059+0000 7f2d6ee1d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d58003a90 0x7f2d680a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:31.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.059+0000 7f2d6ee1d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d6816d4e0 0x7f2d68169e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:31.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.059+0000 7f2d6ee1d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d6814d2d0 0x7f2d680ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:31.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.059+0000 7f2d6ee1d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d680c3430 con 0x7f2d6816d4e0 2023-10-06T10:40:31.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.059+0000 7f2d6ee1d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6816a530 con 0x7f2d58003a90 2023-10-06T10:40:31.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.059+0000 7f2d6ee1d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6816a810 con 0x7f2d6814d2d0 2023-10-06T10:40:31.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.059+0000 7f2d67fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d6816d4e0 0x7f2d68169e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:31.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.059+0000 7f2d67fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d6816d4e0 0x7f2d68169e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60616/0 (socket says 172.21.15.92:60616) 2023-10-06T10:40:31.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.059+0000 7f2d67fff700 1 -- 172.21.15.92:0/76410273 learned_addr learned my addr 172.21.15.92:0/76410273 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:31.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.059+0000 7f2d6cbb9700 1 --2- 172.21.15.92:0/76410273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d58003a90 0x7f2d680a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:31.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.059+0000 7f2d6d3ba700 1 --2- 172.21.15.92:0/76410273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d6814d2d0 0x7f2d680ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:31.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.060+0000 7f2d67fff700 1 -- 172.21.15.92:0/76410273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d6814d2d0 msgr2=0x7f2d680ad5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:31.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.060+0000 7f2d67fff700 1 --2- 172.21.15.92:0/76410273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d6814d2d0 0x7f2d680ad5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.060+0000 7f2d67fff700 1 -- 172.21.15.92:0/76410273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d58003a90 msgr2=0x7f2d680a98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:31.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.060+0000 7f2d67fff700 1 --2- 172.21.15.92:0/76410273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d58003a90 0x7f2d680a98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.060+0000 7f2d67fff700 1 -- 172.21.15.92:0/76410273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6816aaf0 con 0x7f2d6816d4e0 2023-10-06T10:40:31.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.060+0000 7f2d6cbb9700 1 --2- 172.21.15.92:0/76410273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d58003a90 0x7f2d680a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:31.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.060+0000 7f2d6d3ba700 1 --2- 172.21.15.92:0/76410273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d6814d2d0 0x7f2d680ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:31.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.060+0000 7f2d67fff700 1 --2- 172.21.15.92:0/76410273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d6816d4e0 0x7f2d68169e80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2d54011990 tx=0x7f2d54008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:31.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.060+0000 7f2d64ff9700 1 -- 172.21.15.92:0/76410273 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5400f6d0 con 0x7f2d6816d4e0 2023-10-06T10:40:31.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.060+0000 7f2d64ff9700 1 -- 172.21.15.92:0/76410273 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5400f830 con 0x7f2d6816d4e0 2023-10-06T10:40:31.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.060+0000 7f2d6ee1d700 1 -- 172.21.15.92:0/76410273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6816add0 con 0x7f2d6816d4e0 2023-10-06T10:40:31.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.060+0000 7f2d64ff9700 1 -- 172.21.15.92:0/76410273 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5400fbf0 con 0x7f2d6816d4e0 2023-10-06T10:40:31.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.060+0000 7f2d6ee1d700 1 -- 172.21.15.92:0/76410273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6814d740 con 0x7f2d6816d4e0 2023-10-06T10:40:31.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.061+0000 7f2d6ee1d700 1 -- 172.21.15.92:0/76410273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d68064a90 con 0x7f2d6816d4e0 2023-10-06T10:40:31.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.061+0000 7f2d64ff9700 1 -- 172.21.15.92:0/76410273 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2d54015460 con 0x7f2d6816d4e0 2023-10-06T10:40:31.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.062+0000 7f2d64ff9700 1 --2- 172.21.15.92:0/76410273 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2d4c066f30 0x7f2d4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:31.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.062+0000 7f2d64ff9700 1 -- 172.21.15.92:0/76410273 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f2d54084c10 con 0x7f2d6816d4e0 2023-10-06T10:40:31.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.062+0000 7f2d6cbb9700 1 --2- 172.21.15.92:0/76410273 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2d4c066f30 0x7f2d4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:31.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.062+0000 7f2d6cbb9700 1 --2- 172.21.15.92:0/76410273 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2d4c066f30 0x7f2d4c0693e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2d681512e0 tx=0x7f2d5c015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:31.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.066+0000 7f2d64ff9700 1 -- 172.21.15.92:0/76410273 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d54051fe0 con 0x7f2d6816d4e0 2023-10-06T10:40:31.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.248+0000 7f2d6ee1d700 1 -- 172.21.15.92:0/76410273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d680c7b30 con 0x7f2d6816d4e0 2023-10-06T10:40:31.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.249+0000 7f2d64ff9700 1 -- 172.21.15.92:0/76410273 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+13855 (secure 0 0 0) 0x7f2d54055600 con 0x7f2d6816d4e0 2023-10-06T10:40:31.250 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:31.250 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":48,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:40:30.522007+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:40:31.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.252+0000 7f2d6ee1d700 1 -- 172.21.15.92:0/76410273 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2d4c066f30 msgr2=0x7f2d4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:31.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.252+0000 7f2d6ee1d700 1 --2- 172.21.15.92:0/76410273 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2d4c066f30 0x7f2d4c0693e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2d681512e0 tx=0x7f2d5c015000 comp rx=0 tx=0).stop 2023-10-06T10:40:31.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.252+0000 7f2d6ee1d700 1 -- 172.21.15.92:0/76410273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d6816d4e0 msgr2=0x7f2d68169e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:31.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.252+0000 7f2d6ee1d700 1 --2- 172.21.15.92:0/76410273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d6816d4e0 0x7f2d68169e80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2d54011990 tx=0x7f2d54008040 comp rx=0 tx=0).stop 2023-10-06T10:40:31.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.252+0000 7f2d6ee1d700 1 -- 172.21.15.92:0/76410273 shutdown_connections 2023-10-06T10:40:31.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.252+0000 7f2d6ee1d700 1 --2- 172.21.15.92:0/76410273 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2d4c066f30 0x7f2d4c0693e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.252+0000 7f2d6ee1d700 1 --2- 172.21.15.92:0/76410273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d6814d2d0 0x7f2d680ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.252+0000 7f2d6ee1d700 1 --2- 172.21.15.92:0/76410273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d58003a90 0x7f2d680a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.252+0000 7f2d6ee1d700 1 --2- 172.21.15.92:0/76410273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d6816d4e0 0x7f2d68169e80 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.252+0000 7f2d6ee1d700 1 -- 172.21.15.92:0/76410273 >> 172.21.15.92:0/76410273 conn(0x7f2d68068080 msgr2=0x7f2d680af8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:31.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.252+0000 7f2d6ee1d700 1 -- 172.21.15.92:0/76410273 shutdown_connections 2023-10-06T10:40:31.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.252+0000 7f2d6ee1d700 1 -- 172.21.15.92:0/76410273 wait complete. 2023-10-06T10:40:31.265 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T10:38:09.680559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T10:38:10.120786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T10:40:31.266 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T10:40:31.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.382+0000 7fae6892c700 1 Processor -- start 2023-10-06T10:40:31.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.382+0000 7fae6892c700 1 -- start start 2023-10-06T10:40:31.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.383+0000 7fae6892c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae641594d0 0x7fae640cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:31.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.383+0000 7fae6892c700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fae641598a0 0x7fae640cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:31.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.383+0000 7fae6892c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fae640ce7e0 0x7fae640cec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:31.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.383+0000 7fae6892c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae640cf5d0 con 0x7fae640ce7e0 2023-10-06T10:40:31.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.383+0000 7fae6892c700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae640cf8b0 con 0x7fae641598a0 2023-10-06T10:40:31.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.383+0000 7fae6892c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae640cfb90 con 0x7fae641594d0 2023-10-06T10:40:31.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.383+0000 7fae62d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fae640ce7e0 0x7fae640cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:31.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.383+0000 7fae62d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fae640ce7e0 0x7fae640cec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60626/0 (socket says 172.21.15.92:60626) 2023-10-06T10:40:31.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.383+0000 7fae62d9d700 1 -- 172.21.15.92:0/3799094016 learned_addr learned my addr 172.21.15.92:0/3799094016 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:31.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.383+0000 7fae6259c700 1 --2- 172.21.15.92:0/3799094016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae641594d0 0x7fae640cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:31.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.383+0000 7fae59d9b700 1 --2- 172.21.15.92:0/3799094016 >> v2:172.21.15.124:3300/0 conn(0x7fae641598a0 0x7fae640cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:31.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.383+0000 7fae62d9d700 1 -- 172.21.15.92:0/3799094016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae641594d0 msgr2=0x7fae640cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:31.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.383+0000 7fae62d9d700 1 --2- 172.21.15.92:0/3799094016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae641594d0 0x7fae640cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.383+0000 7fae62d9d700 1 -- 172.21.15.92:0/3799094016 >> v2:172.21.15.124:3300/0 conn(0x7fae641598a0 msgr2=0x7fae640cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:31.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.383+0000 7fae62d9d700 1 --2- 172.21.15.92:0/3799094016 >> v2:172.21.15.124:3300/0 conn(0x7fae641598a0 0x7fae640cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.383+0000 7fae62d9d700 1 -- 172.21.15.92:0/3799094016 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae640cfe70 con 0x7fae640ce7e0 2023-10-06T10:40:31.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.383+0000 7fae6259c700 1 --2- 172.21.15.92:0/3799094016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae641594d0 0x7fae640cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:31.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.383+0000 7fae59d9b700 1 --2- 172.21.15.92:0/3799094016 >> v2:172.21.15.124:3300/0 conn(0x7fae641598a0 0x7fae640cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:31.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.383+0000 7fae62d9d700 1 --2- 172.21.15.92:0/3799094016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae640ce7e0 0x7fae640cec00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fae64089320 tx=0x7fae5c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7cb68a6575c8f69 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:31.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.384+0000 7fae61d9b700 1 -- 172.21.15.92:0/3799094016 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c00e070 con 0x7fae640ce7e0 2023-10-06T10:40:31.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.384+0000 7fae61d9b700 1 -- 172.21.15.92:0/3799094016 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5c00ba50 con 0x7fae640ce7e0 2023-10-06T10:40:31.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.384+0000 7fae61d9b700 1 -- 172.21.15.92:0/3799094016 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c0092e0 con 0x7fae640ce7e0 2023-10-06T10:40:31.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.384+0000 7fae6892c700 1 -- 172.21.15.92:0/3799094016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae640ce7e0 msgr2=0x7fae640cec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:31.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.384+0000 7fae6892c700 1 --2- 172.21.15.92:0/3799094016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae640ce7e0 0x7fae640cec00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fae64089320 tx=0x7fae5c009840 comp rx=0 tx=0).stop 2023-10-06T10:40:31.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.384+0000 7fae6892c700 1 -- 172.21.15.92:0/3799094016 shutdown_connections 2023-10-06T10:40:31.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.384+0000 7fae6892c700 1 --2- 172.21.15.92:0/3799094016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae640ce7e0 0x7fae640cec00 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.384+0000 7fae6892c700 1 --2- 172.21.15.92:0/3799094016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae641594d0 0x7fae640cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.384+0000 7fae6892c700 1 --2- 172.21.15.92:0/3799094016 >> v2:172.21.15.124:3300/0 conn(0x7fae641598a0 0x7fae640cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.384+0000 7fae6892c700 1 -- 172.21.15.92:0/3799094016 >> 172.21.15.92:0/3799094016 conn(0x7fae64089e50 msgr2=0x7fae64099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:31.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.384+0000 7fae6892c700 1 -- 172.21.15.92:0/3799094016 shutdown_connections 2023-10-06T10:40:31.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.384+0000 7fae6892c700 1 -- 172.21.15.92:0/3799094016 wait complete. 2023-10-06T10:40:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.385+0000 7fae6892c700 1 Processor -- start 2023-10-06T10:40:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.385+0000 7fae6892c700 1 -- start start 2023-10-06T10:40:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.386+0000 7fae6892c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae640ce7e0 0x7fae64149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.386+0000 7fae6892c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae641594d0 0x7fae6415bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.386+0000 7fae6892c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae641598a0 0x7fae640c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.386+0000 7fae6892c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6414a680 con 0x7fae640ce7e0 2023-10-06T10:40:31.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.386+0000 7fae6892c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6415bb30 con 0x7fae641598a0 2023-10-06T10:40:31.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.386+0000 7fae6892c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae64149580 con 0x7fae641594d0 2023-10-06T10:40:31.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.386+0000 7fae6259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae640ce7e0 0x7fae64149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:31.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.386+0000 7fae6259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae640ce7e0 0x7fae64149040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60636/0 (socket says 172.21.15.92:60636) 2023-10-06T10:40:31.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.386+0000 7fae6259c700 1 -- 172.21.15.92:0/3741102815 learned_addr learned my addr 172.21.15.92:0/3741102815 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:31.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.386+0000 7fae59d9b700 1 --2- 172.21.15.92:0/3741102815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae641594d0 0x7fae6415bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:31.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.386+0000 7fae62d9d700 1 --2- 172.21.15.92:0/3741102815 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae641598a0 0x7fae640c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:31.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.386+0000 7fae6259c700 1 -- 172.21.15.92:0/3741102815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae641594d0 msgr2=0x7fae6415bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:31.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.386+0000 7fae6259c700 1 --2- 172.21.15.92:0/3741102815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae641594d0 0x7fae6415bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.387+0000 7fae6259c700 1 -- 172.21.15.92:0/3741102815 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae641598a0 msgr2=0x7fae640c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:31.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.387+0000 7fae6259c700 1 --2- 172.21.15.92:0/3741102815 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae641598a0 0x7fae640c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.387+0000 7fae6259c700 1 -- 172.21.15.92:0/3741102815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c009ae0 con 0x7fae640ce7e0 2023-10-06T10:40:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.387+0000 7fae59d9b700 1 --2- 172.21.15.92:0/3741102815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae641594d0 0x7fae6415bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.387+0000 7fae62d9d700 1 --2- 172.21.15.92:0/3741102815 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae641598a0 0x7fae640c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.387+0000 7fae6259c700 1 --2- 172.21.15.92:0/3741102815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae640ce7e0 0x7fae64149040 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fae5400e900 tx=0x7fae54008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.387+0000 7fae5affd700 1 -- 172.21.15.92:0/3741102815 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5400eea0 con 0x7fae640ce7e0 2023-10-06T10:40:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.387+0000 7fae5affd700 1 -- 172.21.15.92:0/3741102815 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5400c5e0 con 0x7fae640ce7e0 2023-10-06T10:40:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.387+0000 7fae6892c700 1 -- 172.21.15.92:0/3741102815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae6414b5d0 con 0x7fae640ce7e0 2023-10-06T10:40:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.387+0000 7fae5affd700 1 -- 172.21.15.92:0/3741102815 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5400c980 con 0x7fae640ce7e0 2023-10-06T10:40:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.387+0000 7fae6892c700 1 -- 172.21.15.92:0/3741102815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae6415b7c0 con 0x7fae640ce7e0 2023-10-06T10:40:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.388+0000 7fae6892c700 1 -- 172.21.15.92:0/3741102815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae6415be10 con 0x7fae640ce7e0 2023-10-06T10:40:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.388+0000 7fae5affd700 1 -- 172.21.15.92:0/3741102815 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fae5400cd30 con 0x7fae640ce7e0 2023-10-06T10:40:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.389+0000 7fae5affd700 1 --2- 172.21.15.92:0/3741102815 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fae44066f30 0x7fae440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.389+0000 7fae59d9b700 1 --2- 172.21.15.92:0/3741102815 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fae44066f30 0x7fae440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.389+0000 7fae5affd700 1 -- 172.21.15.92:0/3741102815 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fae540842d0 con 0x7fae640ce7e0 2023-10-06T10:40:31.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.389+0000 7fae59d9b700 1 --2- 172.21.15.92:0/3741102815 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fae44066f30 0x7fae440693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fae640c6130 tx=0x7fae4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:31.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.393+0000 7fae5affd700 1 -- 172.21.15.92:0/3741102815 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae54055080 con 0x7fae640ce7e0 2023-10-06T10:40:31.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.573+0000 7fae6892c700 1 -- 172.21.15.92:0/3741102815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fae64064a90 con 0x7fae640ce7e0 2023-10-06T10:40:31.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.574+0000 7fae5affd700 1 -- 172.21.15.92:0/3741102815 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v48) v1 ==== 92+0+10 (secure 0 0 0) 0x7fae54054c10 con 0x7fae640ce7e0 2023-10-06T10:40:31.575 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T10:40:31.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.575+0000 7fae6892c700 1 -- 172.21.15.92:0/3741102815 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fae44066f30 msgr2=0x7fae440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:31.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.575+0000 7fae6892c700 1 --2- 172.21.15.92:0/3741102815 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fae44066f30 0x7fae440693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fae640c6130 tx=0x7fae4c008040 comp rx=0 tx=0).stop 2023-10-06T10:40:31.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.575+0000 7fae6892c700 1 -- 172.21.15.92:0/3741102815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae640ce7e0 msgr2=0x7fae64149040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:31.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.575+0000 7fae6892c700 1 --2- 172.21.15.92:0/3741102815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae640ce7e0 0x7fae64149040 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fae5400e900 tx=0x7fae54008040 comp rx=0 tx=0).stop 2023-10-06T10:40:31.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.576+0000 7fae6892c700 1 -- 172.21.15.92:0/3741102815 shutdown_connections 2023-10-06T10:40:31.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.576+0000 7fae6892c700 1 --2- 172.21.15.92:0/3741102815 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fae44066f30 0x7fae440693e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.576+0000 7fae6892c700 1 --2- 172.21.15.92:0/3741102815 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae641598a0 0x7fae640c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.576+0000 7fae6892c700 1 --2- 172.21.15.92:0/3741102815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae640ce7e0 0x7fae64149040 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.576+0000 7fae6892c700 1 --2- 172.21.15.92:0/3741102815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae641594d0 0x7fae6415bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.576+0000 7fae6892c700 1 -- 172.21.15.92:0/3741102815 >> 172.21.15.92:0/3741102815 conn(0x7fae64089e50 msgr2=0x7fae6414c260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:31.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.576+0000 7fae6892c700 1 -- 172.21.15.92:0/3741102815 shutdown_connections 2023-10-06T10:40:31.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.576+0000 7fae6892c700 1 -- 172.21.15.92:0/3741102815 wait complete. 2023-10-06T10:40:31.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T10:40:31.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.705+0000 7f4c4147c700 1 Processor -- start 2023-10-06T10:40:31.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.705+0000 7f4c4147c700 1 -- start start 2023-10-06T10:40:31.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.706+0000 7f4c4147c700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4c3c0b0a10 0x7f4c3c0cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:31.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.706+0000 7f4c4147c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c3c0cc420 0x7f4c3c0c9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:31.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.706+0000 7f4c4147c700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3c0c53c0 con 0x7f4c3c0b0640 2023-10-06T10:40:31.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.706+0000 7f4c4147c700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c3c0ca790 con 0x7f4c3c0b0a10 2023-10-06T10:40:31.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.706+0000 7f4c4147c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c0ccd00 con 0x7f4c3c0cc420 2023-10-06T10:40:31.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.706+0000 7f4c327fc700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4c3c0b0a10 0x7f4c3c0cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:31.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.706+0000 7f4c3b7fe700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c3c0cc420 0x7f4c3c0c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:31.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.706+0000 7f4c3affd700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f4c3c0b0640 0x7f4c3c0cecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:51050/0 (socket says 172.21.15.92:51050) 2023-10-06T10:40:31.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.707+0000 7f4c3affd700 1 -- 172.21.15.92:0/3144540223 learned_addr learned my addr 172.21.15.92:0/3144540223 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:31.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.707+0000 7f4c327fc700 1 -- 172.21.15.92:0/3144540223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c3c0cc420 msgr2=0x7f4c3c0c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:31.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.707+0000 7f4c327fc700 1 --2- 172.21.15.92:0/3144540223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c3c0cc420 0x7f4c3c0c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.707+0000 7f4c327fc700 1 -- 172.21.15.92:0/3144540223 >> v1:172.21.15.124:6789/0 conn(0x7f4c3c0b0640 legacy=0x7f4c3c0cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:31.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.707+0000 7f4c327fc700 1 -- 172.21.15.92:0/3144540223 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c0ccfe0 con 0x7f4c3c0b0a10 2023-10-06T10:40:31.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.707+0000 7f4c3b7fe700 1 --2- 172.21.15.92:0/3144540223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c3c0cc420 0x7f4c3c0c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:40:31.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.707+0000 7f4c3a7fc700 1 -- 172.21.15.92:0/3144540223 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1174633744 0 0) 0x7f4c3c0c53c0 con 0x7f4c3c0b0640 2023-10-06T10:40:31.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.707+0000 7f4c327fc700 1 --2- 172.21.15.92:0/3144540223 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c3c0b0a10 0x7f4c3c0cbee0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4c240098e0 tx=0x7f4c240091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=455bd8a18c6890ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:31.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.707+0000 7f4c3a7fc700 1 -- 172.21.15.92:0/3144540223 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400b640 con 0x7f4c3c0b0a10 2023-10-06T10:40:31.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.707+0000 7f4c3a7fc700 1 -- 172.21.15.92:0/3144540223 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2400b7a0 con 0x7f4c3c0b0a10 2023-10-06T10:40:31.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.707+0000 7f4c3a7fc700 1 -- 172.21.15.92:0/3144540223 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400bb40 con 0x7f4c3c0b0a10 2023-10-06T10:40:31.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.708+0000 7f4c4147c700 1 -- 172.21.15.92:0/3144540223 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c3c0b0a10 msgr2=0x7f4c3c0cbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:31.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.708+0000 7f4c4147c700 1 --2- 172.21.15.92:0/3144540223 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c3c0b0a10 0x7f4c3c0cbee0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4c240098e0 tx=0x7f4c240091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.708+0000 7f4c4147c700 1 -- 172.21.15.92:0/3144540223 shutdown_connections 2023-10-06T10:40:31.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.708+0000 7f4c4147c700 1 --2- 172.21.15.92:0/3144540223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c3c0cc420 0x7f4c3c0c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.708+0000 7f4c4147c700 1 --2- 172.21.15.92:0/3144540223 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c3c0b0a10 0x7f4c3c0cbee0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.708+0000 7f4c4147c700 1 -- 172.21.15.92:0/3144540223 >> 172.21.15.92:0/3144540223 conn(0x7f4c3c087fc0 msgr2=0x7f4c3c0ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:31.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.708+0000 7f4c4147c700 1 -- 172.21.15.92:0/3144540223 shutdown_connections 2023-10-06T10:40:31.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.708+0000 7f4c4147c700 1 -- 172.21.15.92:0/3144540223 wait complete. 2023-10-06T10:40:31.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.709+0000 7f4c4147c700 1 Processor -- start 2023-10-06T10:40:31.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.709+0000 7f4c4147c700 1 -- start start 2023-10-06T10:40:31.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.710+0000 7f4c4147c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c3c0b0640 0x7f4c3c154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:31.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.710+0000 7f4c4147c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c3c0b0a10 0x7f4c3c0b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:31.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.710+0000 7f4c4147c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c3c0cc420 0x7f4c3c0b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:31.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.710+0000 7f4c4147c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c0b1b70 con 0x7f4c3c0b0640 2023-10-06T10:40:31.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.710+0000 7f4c4147c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c0b1e50 con 0x7f4c3c0b0a10 2023-10-06T10:40:31.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.710+0000 7f4c4147c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c15c250 con 0x7f4c3c0cc420 2023-10-06T10:40:31.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.710+0000 7f4c3affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c3c0b0640 0x7f4c3c154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:31.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.710+0000 7f4c3affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c3c0b0640 0x7f4c3c154de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60644/0 (socket says 172.21.15.92:60644) 2023-10-06T10:40:31.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.710+0000 7f4c3affd700 1 -- 172.21.15.92:0/1667912169 learned_addr learned my addr 172.21.15.92:0/1667912169 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:31.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.710+0000 7f4c327fc700 1 --2- 172.21.15.92:0/1667912169 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c3c0b0a10 0x7f4c3c0b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:31.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.710+0000 7f4c3b7fe700 1 --2- 172.21.15.92:0/1667912169 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c3c0cc420 0x7f4c3c0b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:31.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.710+0000 7f4c3affd700 1 -- 172.21.15.92:0/1667912169 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c3c0cc420 msgr2=0x7f4c3c0b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:31.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.710+0000 7f4c3affd700 1 --2- 172.21.15.92:0/1667912169 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c3c0cc420 0x7f4c3c0b21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.710+0000 7f4c3affd700 1 -- 172.21.15.92:0/1667912169 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c3c0b0a10 msgr2=0x7f4c3c0b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:31.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.710+0000 7f4c3affd700 1 --2- 172.21.15.92:0/1667912169 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c3c0b0a10 0x7f4c3c0b10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.710+0000 7f4c3affd700 1 -- 172.21.15.92:0/1667912169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c34009a20 con 0x7f4c3c0b0640 2023-10-06T10:40:31.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.710+0000 7f4c3affd700 1 --2- 172.21.15.92:0/1667912169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c3c0b0640 0x7f4c3c154de0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00c5e0 tx=0x7f4c2c001030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:31.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.711+0000 7f4c337fe700 1 -- 172.21.15.92:0/1667912169 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c005b80 con 0x7f4c3c0b0640 2023-10-06T10:40:31.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.711+0000 7f4c337fe700 1 -- 172.21.15.92:0/1667912169 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2c005d00 con 0x7f4c3c0b0640 2023-10-06T10:40:31.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.711+0000 7f4c4147c700 1 -- 172.21.15.92:0/1667912169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c24009420 con 0x7f4c3c0b0640 2023-10-06T10:40:31.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.711+0000 7f4c337fe700 1 -- 172.21.15.92:0/1667912169 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c007320 con 0x7f4c3c0b0640 2023-10-06T10:40:31.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.711+0000 7f4c4147c700 1 -- 172.21.15.92:0/1667912169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3c0cbcc0 con 0x7f4c3c0b0640 2023-10-06T10:40:31.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.712+0000 7f4c4147c700 1 -- 172.21.15.92:0/1667912169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c3c064a90 con 0x7f4c3c0b0640 2023-10-06T10:40:31.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.712+0000 7f4c337fe700 1 -- 172.21.15.92:0/1667912169 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4c2c01b3b0 con 0x7f4c3c0b0640 2023-10-06T10:40:31.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.712+0000 7f4c337fe700 1 --2- 172.21.15.92:0/1667912169 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4c1c066f30 0x7f4c1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:31.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.712+0000 7f4c337fe700 1 -- 172.21.15.92:0/1667912169 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f4c2c0838b0 con 0x7f4c3c0b0640 2023-10-06T10:40:31.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.712+0000 7f4c327fc700 1 --2- 172.21.15.92:0/1667912169 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4c1c066f30 0x7f4c1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:31.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.713+0000 7f4c327fc700 1 --2- 172.21.15.92:0/1667912169 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4c1c066f30 0x7f4c1c0693e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4c240093f0 tx=0x7f4c24017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:31.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.717+0000 7f4c337fe700 1 -- 172.21.15.92:0/1667912169 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c2c050c90 con 0x7f4c3c0b0640 2023-10-06T10:40:31.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.901+0000 7f4c4147c700 1 -- 172.21.15.92:0/1667912169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4c3c0ca3b0 con 0x7f4c3c0b0640 2023-10-06T10:40:31.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.901+0000 7f4c337fe700 1 -- 172.21.15.92:0/1667912169 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4c2c017070 con 0x7f4c3c0b0640 2023-10-06T10:40:31.902 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T10:40:31.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.903+0000 7f4c4147c700 1 -- 172.21.15.92:0/1667912169 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4c1c066f30 msgr2=0x7f4c1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:31.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.903+0000 7f4c4147c700 1 --2- 172.21.15.92:0/1667912169 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4c1c066f30 0x7f4c1c0693e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4c240093f0 tx=0x7f4c24017000 comp rx=0 tx=0).stop 2023-10-06T10:40:31.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.903+0000 7f4c4147c700 1 -- 172.21.15.92:0/1667912169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c3c0b0640 msgr2=0x7f4c3c154de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:31.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.903+0000 7f4c4147c700 1 --2- 172.21.15.92:0/1667912169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c3c0b0640 0x7f4c3c154de0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00c5e0 tx=0x7f4c2c001030 comp rx=0 tx=0).stop 2023-10-06T10:40:31.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.903+0000 7f4c4147c700 1 -- 172.21.15.92:0/1667912169 shutdown_connections 2023-10-06T10:40:31.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.903+0000 7f4c4147c700 1 --2- 172.21.15.92:0/1667912169 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4c1c066f30 0x7f4c1c0693e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.903+0000 7f4c4147c700 1 --2- 172.21.15.92:0/1667912169 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c3c0cc420 0x7f4c3c0b21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.903+0000 7f4c4147c700 1 --2- 172.21.15.92:0/1667912169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c3c0b0640 0x7f4c3c154de0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.903+0000 7f4c4147c700 1 --2- 172.21.15.92:0/1667912169 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c3c0b0a10 0x7f4c3c0b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:31.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.903+0000 7f4c4147c700 1 -- 172.21.15.92:0/1667912169 >> 172.21.15.92:0/1667912169 conn(0x7f4c3c087fc0 msgr2=0x7f4c3c156bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:31.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.904+0000 7f4c4147c700 1 -- 172.21.15.92:0/1667912169 shutdown_connections 2023-10-06T10:40:31.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:31.904+0000 7f4c4147c700 1 -- 172.21.15.92:0/1667912169 wait complete. 2023-10-06T10:40:31.916 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T10:40:32.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.029+0000 7f361eb25700 1 Processor -- start 2023-10-06T10:40:32.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.029+0000 7f361eb25700 1 -- start start 2023-10-06T10:40:32.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.030+0000 7f361eb25700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f36180b49a0 0x7f36180aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:32.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.030+0000 7f361eb25700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36180b0c60 con 0x7f3618159560 2023-10-06T10:40:32.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.030+0000 7f361eb25700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36180b0f40 con 0x7f36180b0760 2023-10-06T10:40:32.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.030+0000 7f361eb25700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36180b1220 con 0x7f36180b49a0 2023-10-06T10:40:32.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.030+0000 7f360ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f36180b49a0 0x7f36180aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:32.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.030+0000 7f361c8c1700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3618159560 0x7f36180aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51008/0 (socket says 172.21.15.92:51008) 2023-10-06T10:40:32.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.030+0000 7f360ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f36180b49a0 0x7f36180aac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60646/0 (socket says 172.21.15.92:60646) 2023-10-06T10:40:32.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.030+0000 7f361c8c1700 1 -- 172.21.15.92:0/2234315950 learned_addr learned my addr 172.21.15.92:0/2234315950 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:32.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.030+0000 7f360ffff700 1 -- 172.21.15.92:0/2234315950 >> v1:172.21.15.124:6789/0 conn(0x7f36180b0760 legacy=0x7f36180b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:32.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.030+0000 7f360ffff700 1 -- 172.21.15.92:0/2234315950 >> v1:172.21.15.92:6789/0 conn(0x7f3618159560 legacy=0x7f36180aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:32.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.031+0000 7f360ffff700 1 -- 172.21.15.92:0/2234315950 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36180b1500 con 0x7f36180b49a0 2023-10-06T10:40:32.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.030+0000 7f360f7fe700 1 -- 172.21.15.92:0/2234315950 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 125954342 0 0) 0x7f36180b0c60 con 0x7f3618159560 2023-10-06T10:40:32.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.031+0000 7f360ffff700 1 --2- 172.21.15.92:0/2234315950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36180b49a0 0x7f36180aac40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3608009910 tx=0x7f36080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3442b2fb3f3b13 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:32.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.031+0000 7f360f7fe700 1 -- 172.21.15.92:0/2234315950 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360800b820 con 0x7f36180b49a0 2023-10-06T10:40:32.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.031+0000 7f360f7fe700 1 -- 172.21.15.92:0/2234315950 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360800b980 con 0x7f36180b49a0 2023-10-06T10:40:32.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.031+0000 7f360f7fe700 1 -- 172.21.15.92:0/2234315950 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360800bd20 con 0x7f36180b49a0 2023-10-06T10:40:32.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.031+0000 7f361eb25700 1 -- 172.21.15.92:0/2234315950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36180b49a0 msgr2=0x7f36180aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:32.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.031+0000 7f361eb25700 1 --2- 172.21.15.92:0/2234315950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36180b49a0 0x7f36180aac40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3608009910 tx=0x7f36080091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:32.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.031+0000 7f361eb25700 1 -- 172.21.15.92:0/2234315950 shutdown_connections 2023-10-06T10:40:32.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.031+0000 7f361eb25700 1 --2- 172.21.15.92:0/2234315950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36180b49a0 0x7f36180aac40 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:32.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.031+0000 7f361eb25700 1 -- 172.21.15.92:0/2234315950 >> 172.21.15.92:0/2234315950 conn(0x7f36180997f0 msgr2=0x7f361806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:32.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.031+0000 7f361eb25700 1 -- 172.21.15.92:0/2234315950 shutdown_connections 2023-10-06T10:40:32.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.031+0000 7f361eb25700 1 -- 172.21.15.92:0/2234315950 wait complete. 2023-10-06T10:40:32.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.032+0000 7f361eb25700 1 Processor -- start 2023-10-06T10:40:32.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.033+0000 7f361eb25700 1 -- start start 2023-10-06T10:40:32.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.033+0000 7f361eb25700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36180b0760 0x7f36180abe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:32.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.033+0000 7f361eb25700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36180b49a0 0x7f361809a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:32.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.033+0000 7f361eb25700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3618159560 0x7f361809bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:32.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.033+0000 7f361eb25700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3618157490 con 0x7f3618159560 2023-10-06T10:40:32.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.033+0000 7f361eb25700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361809aea0 con 0x7f36180b0760 2023-10-06T10:40:32.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.033+0000 7f361eb25700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361809b0d0 con 0x7f36180b49a0 2023-10-06T10:40:32.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.033+0000 7f361d0c2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3618159560 0x7f361809bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:32.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.033+0000 7f361d0c2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3618159560 0x7f361809bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60662/0 (socket says 172.21.15.92:60662) 2023-10-06T10:40:32.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.033+0000 7f361d0c2700 1 -- 172.21.15.92:0/2042626152 learned_addr learned my addr 172.21.15.92:0/2042626152 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:32.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.033+0000 7f360ffff700 1 --2- 172.21.15.92:0/2042626152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36180b49a0 0x7f361809a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:32.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.033+0000 7f361c8c1700 1 --2- 172.21.15.92:0/2042626152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36180b0760 0x7f36180abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:32.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.034+0000 7f361d0c2700 1 -- 172.21.15.92:0/2042626152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36180b49a0 msgr2=0x7f361809a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:32.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.034+0000 7f361d0c2700 1 --2- 172.21.15.92:0/2042626152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36180b49a0 0x7f361809a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:32.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.034+0000 7f361d0c2700 1 -- 172.21.15.92:0/2042626152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36180b0760 msgr2=0x7f36180abe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:32.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.034+0000 7f361d0c2700 1 --2- 172.21.15.92:0/2042626152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36180b0760 0x7f36180abe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:32.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.034+0000 7f361d0c2700 1 -- 172.21.15.92:0/2042626152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3608009420 con 0x7f3618159560 2023-10-06T10:40:32.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.034+0000 7f361d0c2700 1 --2- 172.21.15.92:0/2042626152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3618159560 0x7f361809bc60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f361000c560 tx=0x7f3610007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:32.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.034+0000 7f360cff9700 1 -- 172.21.15.92:0/2042626152 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3610011590 con 0x7f3618159560 2023-10-06T10:40:32.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.034+0000 7f360cff9700 1 -- 172.21.15.92:0/2042626152 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36100116f0 con 0x7f3618159560 2023-10-06T10:40:32.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.034+0000 7f361eb25700 1 -- 172.21.15.92:0/2042626152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36180ae260 con 0x7f3618159560 2023-10-06T10:40:32.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.034+0000 7f360cff9700 1 -- 172.21.15.92:0/2042626152 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3610011ab0 con 0x7f3618159560 2023-10-06T10:40:32.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.034+0000 7f361eb25700 1 -- 172.21.15.92:0/2042626152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36180adc10 con 0x7f3618159560 2023-10-06T10:40:32.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.035+0000 7f361eb25700 1 -- 172.21.15.92:0/2042626152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3618064a90 con 0x7f3618159560 2023-10-06T10:40:32.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.035+0000 7f360cff9700 1 -- 172.21.15.92:0/2042626152 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f361001b3b0 con 0x7f3618159560 2023-10-06T10:40:32.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.036+0000 7f360cff9700 1 --2- 172.21.15.92:0/2042626152 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3604066f30 0x7f36040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:32.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.036+0000 7f360cff9700 1 -- 172.21.15.92:0/2042626152 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f3610083840 con 0x7f3618159560 2023-10-06T10:40:32.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.036+0000 7f361c8c1700 1 --2- 172.21.15.92:0/2042626152 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3604066f30 0x7f36040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:32.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.036+0000 7f361c8c1700 1 --2- 172.21.15.92:0/2042626152 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3604066f30 0x7f36040693e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f361815d150 tx=0x7f3614009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:32.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.039+0000 7f360cff9700 1 -- 172.21.15.92:0/2042626152 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3610050b60 con 0x7f3618159560 2023-10-06T10:40:32.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.224+0000 7f361eb25700 1 -- 172.21.15.92:0/2042626152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f36180b17e0 con 0x7f3618159560 2023-10-06T10:40:32.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.225+0000 7f360cff9700 1 -- 172.21.15.92:0/2042626152 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v48) v1 ==== 84+0+126 (secure 0 0 0) 0x7f3610017070 con 0x7f3618159560 2023-10-06T10:40:32.227 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:32.227 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T10:40:32.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:40:32.227 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-10-06T10:40:32.227 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-10-06T10:40:32.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:40:32.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:40:32.228 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-10-06T10:40:32.228 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-10-06T10:40:32.228 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:40:32.228 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T10:40:32.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.226+0000 7f361eb25700 1 -- 172.21.15.92:0/2042626152 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3604066f30 msgr2=0x7f36040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:32.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.226+0000 7f361eb25700 1 --2- 172.21.15.92:0/2042626152 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3604066f30 0x7f36040693e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f361815d150 tx=0x7f3614009040 comp rx=0 tx=0).stop 2023-10-06T10:40:32.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.226+0000 7f361eb25700 1 -- 172.21.15.92:0/2042626152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3618159560 msgr2=0x7f361809bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:32.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.226+0000 7f361eb25700 1 --2- 172.21.15.92:0/2042626152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3618159560 0x7f361809bc60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f361000c560 tx=0x7f3610007ef0 comp rx=0 tx=0).stop 2023-10-06T10:40:32.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.226+0000 7f361eb25700 1 -- 172.21.15.92:0/2042626152 shutdown_connections 2023-10-06T10:40:32.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.226+0000 7f361eb25700 1 --2- 172.21.15.92:0/2042626152 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3604066f30 0x7f36040693e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:32.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.226+0000 7f361eb25700 1 --2- 172.21.15.92:0/2042626152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3618159560 0x7f361809bc60 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:32.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.226+0000 7f361eb25700 1 --2- 172.21.15.92:0/2042626152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36180b0760 0x7f36180abe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:32.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.226+0000 7f361eb25700 1 --2- 172.21.15.92:0/2042626152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36180b49a0 0x7f361809a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:32.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.226+0000 7f361eb25700 1 -- 172.21.15.92:0/2042626152 >> 172.21.15.92:0/2042626152 conn(0x7f36180997f0 msgr2=0x7f36180afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:32.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.227+0000 7f361eb25700 1 -- 172.21.15.92:0/2042626152 shutdown_connections 2023-10-06T10:40:32.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.227+0000 7f361eb25700 1 -- 172.21.15.92:0/2042626152 wait complete. 2023-10-06T10:40:32.240 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T10:40:32.240 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T10:40:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.357+0000 7f9789c1c700 1 Processor -- start 2023-10-06T10:40:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.357+0000 7f9789c1c700 1 -- start start 2023-10-06T10:40:32.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.357+0000 7f9789c1c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f978406ece0 0x7f97840b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:32.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.357+0000 7f9789c1c700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f978406f0b0 0x7f97840b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:32.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.358+0000 7f9789c1c700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97840b2ce0 con 0x7f97840aede0 2023-10-06T10:40:32.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.358+0000 7f9789c1c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f978406fb40 con 0x7f978406ece0 2023-10-06T10:40:32.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.358+0000 7f9789c1c700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97840b0410 con 0x7f978406f0b0 2023-10-06T10:40:32.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.358+0000 7f97837fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f978406ece0 0x7f97840b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.358+0000 7f97837fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f978406ece0 0x7f97840b2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60668/0 (socket says 172.21.15.92:60668) 2023-10-06T10:40:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.358+0000 7f97837fe700 1 -- 172.21.15.92:0/3400206687 learned_addr learned my addr 172.21.15.92:0/3400206687 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.358+0000 7f9782ffd700 1 --2- 172.21.15.92:0/3400206687 >> v2:172.21.15.124:3300/0 conn(0x7f978406f0b0 0x7f97840b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.358+0000 7f97837fe700 1 -- 172.21.15.92:0/3400206687 >> v2:172.21.15.124:3300/0 conn(0x7f978406f0b0 msgr2=0x7f97840b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.358+0000 7f97837fe700 1 --2- 172.21.15.92:0/3400206687 >> v2:172.21.15.124:3300/0 conn(0x7f978406f0b0 0x7f97840b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.358+0000 7f97837fe700 1 -- 172.21.15.92:0/3400206687 >> v1:172.21.15.124:6789/0 conn(0x7f97840aede0 legacy=0x7f97840af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.358+0000 7f97837fe700 1 -- 172.21.15.92:0/3400206687 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97840b2a00 con 0x7f978406ece0 2023-10-06T10:40:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.358+0000 7f9782ffd700 1 --2- 172.21.15.92:0/3400206687 >> v2:172.21.15.124:3300/0 conn(0x7f978406f0b0 0x7f97840b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.358+0000 7f97837fe700 1 --2- 172.21.15.92:0/3400206687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f978406ece0 0x7f97840b2360 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9774009f20 tx=0x7f9774009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47df23327584546b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:32.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.358+0000 7f97827fc700 1 -- 172.21.15.92:0/3400206687 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977400e070 con 0x7f978406ece0 2023-10-06T10:40:32.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.358+0000 7f97827fc700 1 -- 172.21.15.92:0/3400206687 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977400ba00 con 0x7f978406ece0 2023-10-06T10:40:32.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.358+0000 7f97827fc700 1 -- 172.21.15.92:0/3400206687 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97740092e0 con 0x7f978406ece0 2023-10-06T10:40:32.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.359+0000 7f9789c1c700 1 -- 172.21.15.92:0/3400206687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f978406ece0 msgr2=0x7f97840b2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:32.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.359+0000 7f9789c1c700 1 --2- 172.21.15.92:0/3400206687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f978406ece0 0x7f97840b2360 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9774009f20 tx=0x7f9774009840 comp rx=0 tx=0).stop 2023-10-06T10:40:32.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.359+0000 7f9789c1c700 1 -- 172.21.15.92:0/3400206687 shutdown_connections 2023-10-06T10:40:32.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.359+0000 7f9789c1c700 1 --2- 172.21.15.92:0/3400206687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f978406ece0 0x7f97840b2360 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:32.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.359+0000 7f9789c1c700 1 --2- 172.21.15.92:0/3400206687 >> v2:172.21.15.124:3300/0 conn(0x7f978406f0b0 0x7f97840b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:32.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.359+0000 7f9789c1c700 1 -- 172.21.15.92:0/3400206687 >> 172.21.15.92:0/3400206687 conn(0x7f97840682f0 msgr2=0x7f97840686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:32.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.359+0000 7f9789c1c700 1 -- 172.21.15.92:0/3400206687 shutdown_connections 2023-10-06T10:40:32.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.359+0000 7f9789c1c700 1 -- 172.21.15.92:0/3400206687 wait complete. 2023-10-06T10:40:32.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.360+0000 7f9789c1c700 1 Processor -- start 2023-10-06T10:40:32.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.360+0000 7f9789c1c700 1 -- start start 2023-10-06T10:40:32.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.361+0000 7f9789c1c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f978406ece0 0x7f9784169540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:32.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.361+0000 7f9789c1c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f978406f0b0 0x7f97840a6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:32.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.361+0000 7f9789c1c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97840aede0 0x7f97840ab650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:32.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.361+0000 7f9789c1c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97840a83f0 con 0x7f97840aede0 2023-10-06T10:40:32.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.361+0000 7f9789c1c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97840a77e0 con 0x7f978406f0b0 2023-10-06T10:40:32.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.361+0000 7f9789c1c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97840a7ac0 con 0x7f978406ece0 2023-10-06T10:40:32.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.361+0000 7f9783fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97840aede0 0x7f97840ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:32.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.361+0000 7f9783fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97840aede0 0x7f97840ab650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60684/0 (socket says 172.21.15.92:60684) 2023-10-06T10:40:32.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.361+0000 7f9783fff700 1 -- 172.21.15.92:0/1989131788 learned_addr learned my addr 172.21.15.92:0/1989131788 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:32.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.361+0000 7f9782ffd700 1 --2- 172.21.15.92:0/1989131788 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f978406f0b0 0x7f97840a6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:32.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.361+0000 7f97837fe700 1 --2- 172.21.15.92:0/1989131788 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f978406ece0 0x7f9784169540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:32.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.361+0000 7f9783fff700 1 -- 172.21.15.92:0/1989131788 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f978406ece0 msgr2=0x7f9784169540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:32.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.361+0000 7f9783fff700 1 --2- 172.21.15.92:0/1989131788 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f978406ece0 0x7f9784169540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:32.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.361+0000 7f9783fff700 1 -- 172.21.15.92:0/1989131788 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f978406f0b0 msgr2=0x7f97840a6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:32.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.361+0000 7f9783fff700 1 --2- 172.21.15.92:0/1989131788 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f978406f0b0 0x7f97840a6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:32.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.361+0000 7f9783fff700 1 -- 172.21.15.92:0/1989131788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9774009ae0 con 0x7f97840aede0 2023-10-06T10:40:32.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.362+0000 7f9783fff700 1 --2- 172.21.15.92:0/1989131788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97840aede0 0x7f97840ab650 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f977c00c5b0 tx=0x7f977c007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:32.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.362+0000 7f976b7fe700 1 -- 172.21.15.92:0/1989131788 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c011590 con 0x7f97840aede0 2023-10-06T10:40:32.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.362+0000 7f976b7fe700 1 -- 172.21.15.92:0/1989131788 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977c011710 con 0x7f97840aede0 2023-10-06T10:40:32.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.362+0000 7f9789c1c700 1 -- 172.21.15.92:0/1989131788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97840a7da0 con 0x7f97840aede0 2023-10-06T10:40:32.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.362+0000 7f976b7fe700 1 -- 172.21.15.92:0/1989131788 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c011ad0 con 0x7f97840aede0 2023-10-06T10:40:32.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.362+0000 7f9789c1c700 1 -- 172.21.15.92:0/1989131788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97840a8880 con 0x7f97840aede0 2023-10-06T10:40:32.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.363+0000 7f9789c1c700 1 -- 172.21.15.92:0/1989131788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9784064a90 con 0x7f97840aede0 2023-10-06T10:40:32.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.363+0000 7f976b7fe700 1 -- 172.21.15.92:0/1989131788 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f977c01c460 con 0x7f97840aede0 2023-10-06T10:40:32.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.363+0000 7f976b7fe700 1 --2- 172.21.15.92:0/1989131788 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9770066f30 0x7f97700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:32.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.364+0000 7f976b7fe700 1 -- 172.21.15.92:0/1989131788 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f977c084040 con 0x7f97840aede0 2023-10-06T10:40:32.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.364+0000 7f97837fe700 1 --2- 172.21.15.92:0/1989131788 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9770066f30 0x7f97700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:32.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.364+0000 7f97837fe700 1 --2- 172.21.15.92:0/1989131788 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9770066f30 0x7f97700693e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9774009880 tx=0x7f9774017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:32.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.368+0000 7f976b7fe700 1 -- 172.21.15.92:0/1989131788 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f977c051360 con 0x7f97840aede0 2023-10-06T10:40:32.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:32.552+0000 7f9789c1c700 1 -- 172.21.15.92:0/1989131788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f97840a8530 con 0x7f97840aede0 2023-10-06T10:40:33.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.536+0000 7f976b7fe700 1 -- 172.21.15.92:0/1989131788 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v49) v1 ==== 138+0+0 (secure 0 0 0) 0x7f977c054980 con 0x7f97840aede0 2023-10-06T10:40:33.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.540+0000 7f9789c1c700 1 -- 172.21.15.92:0/1989131788 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9770066f30 msgr2=0x7f97700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:33.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.540+0000 7f9789c1c700 1 --2- 172.21.15.92:0/1989131788 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9770066f30 0x7f97700693e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9774009880 tx=0x7f9774017000 comp rx=0 tx=0).stop 2023-10-06T10:40:33.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.540+0000 7f9789c1c700 1 -- 172.21.15.92:0/1989131788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97840aede0 msgr2=0x7f97840ab650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:33.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.540+0000 7f9789c1c700 1 --2- 172.21.15.92:0/1989131788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97840aede0 0x7f97840ab650 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f977c00c5b0 tx=0x7f977c007f60 comp rx=0 tx=0).stop 2023-10-06T10:40:33.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.540+0000 7f9789c1c700 1 -- 172.21.15.92:0/1989131788 shutdown_connections 2023-10-06T10:40:33.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.540+0000 7f9789c1c700 1 --2- 172.21.15.92:0/1989131788 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9770066f30 0x7f97700693e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:33.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.540+0000 7f9789c1c700 1 --2- 172.21.15.92:0/1989131788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97840aede0 0x7f97840ab650 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:33.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.540+0000 7f9789c1c700 1 --2- 172.21.15.92:0/1989131788 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f978406ece0 0x7f9784169540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:33.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.540+0000 7f9789c1c700 1 --2- 172.21.15.92:0/1989131788 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f978406f0b0 0x7f97840a6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:33.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.540+0000 7f9789c1c700 1 -- 172.21.15.92:0/1989131788 >> 172.21.15.92:0/1989131788 conn(0x7f97840682f0 msgr2=0x7f97840b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:33.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.541+0000 7f9789c1c700 1 -- 172.21.15.92:0/1989131788 shutdown_connections 2023-10-06T10:40:33.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.541+0000 7f9789c1c700 1 -- 172.21.15.92:0/1989131788 wait complete. 2023-10-06T10:40:33.544 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-10-06T10:40:33.556 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T10:40:33.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.675+0000 7fa94f59e700 1 Processor -- start 2023-10-06T10:40:33.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.675+0000 7fa94f59e700 1 -- start start 2023-10-06T10:40:33.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.676+0000 7fa94f59e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9500b2620 0x7fa9500cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:33.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.676+0000 7fa94f59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9500cc970 con 0x7fa9500b29f0 2023-10-06T10:40:33.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.676+0000 7fa94f59e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9500cbd60 con 0x7fa9500cf1b0 2023-10-06T10:40:33.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.676+0000 7fa94f59e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9500cc040 con 0x7fa9500b2620 2023-10-06T10:40:33.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.676+0000 7fa94dd9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa9500b29f0 0x7fa9500ceaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50286/0 (socket says 172.21.15.92:50286) 2023-10-06T10:40:33.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.676+0000 7fa94dd9b700 1 -- 172.21.15.92:0/1650005280 learned_addr learned my addr 172.21.15.92:0/1650005280 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:33.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.676+0000 7fa94e59c700 1 --2- 172.21.15.92:0/1650005280 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9500b2620 0x7fa9500cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:33.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.676+0000 7fa94d59a700 1 -- 172.21.15.92:0/1650005280 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 505997085 0 0) 0x7fa9500cc970 con 0x7fa9500b29f0 2023-10-06T10:40:33.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.676+0000 7fa94d59a700 1 -- 172.21.15.92:0/1650005280 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa93c003570 con 0x7fa9500b29f0 2023-10-06T10:40:33.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.676+0000 7fa94d59a700 1 -- 172.21.15.92:0/1650005280 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3352071 0 0) 0x7fa93c003570 con 0x7fa9500b29f0 2023-10-06T10:40:33.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.676+0000 7fa94d59a700 1 -- 172.21.15.92:0/1650005280 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9500b2620 msgr2=0x7fa9500cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:33.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.676+0000 7fa94d59a700 1 --2- 172.21.15.92:0/1650005280 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9500b2620 0x7fa9500cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:33.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.676+0000 7fa94d59a700 1 -- 172.21.15.92:0/1650005280 >> v1:172.21.15.124:6789/0 conn(0x7fa9500cf1b0 legacy=0x7fa9500c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:33.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.676+0000 7fa94d59a700 1 -- 172.21.15.92:0/1650005280 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9500cc320 con 0x7fa9500b29f0 2023-10-06T10:40:33.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.676+0000 7fa94d59a700 1 -- 172.21.15.92:0/1650005280 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fa938001470 con 0x7fa9500b29f0 2023-10-06T10:40:33.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.676+0000 7fa94d59a700 1 -- 172.21.15.92:0/1650005280 >> v1:172.21.15.92:6789/0 conn(0x7fa9500b29f0 legacy=0x7fa9500ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:33.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.676+0000 7fa94d59a700 1 --2- 172.21.15.92:0/1650005280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa93c003a90 0x7fa93c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:33.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.676+0000 7fa94d59a700 1 --2- 172.21.15.92:0/1650005280 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa93c004810 0x7fa93c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:33.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.677+0000 7fa94d59a700 1 --2- 172.21.15.92:0/1650005280 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9500b2620 0x7fa93c0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:33.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.677+0000 7fa94d59a700 1 -- 172.21.15.92:0/1650005280 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9500cc040 con 0x7fa93c003a90 2023-10-06T10:40:33.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.677+0000 7fa94d59a700 1 -- 172.21.15.92:0/1650005280 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c00bb60 con 0x7fa93c004810 2023-10-06T10:40:33.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.677+0000 7fa94d59a700 1 -- 172.21.15.92:0/1650005280 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c00bd20 con 0x7fa9500b2620 2023-10-06T10:40:33.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.677+0000 7fa94e59c700 1 --2- 172.21.15.92:0/1650005280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa93c003a90 0x7fa93c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:33.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.677+0000 7fa94dd9b700 1 --2- 172.21.15.92:0/1650005280 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa93c004810 0x7fa93c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:33.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.677+0000 7fa94ed9d700 1 --2- 172.21.15.92:0/1650005280 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9500b2620 0x7fa93c0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:33.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.677+0000 7fa94e59c700 1 -- 172.21.15.92:0/1650005280 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9500b2620 msgr2=0x7fa93c0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:33.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.677+0000 7fa94e59c700 1 --2- 172.21.15.92:0/1650005280 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9500b2620 0x7fa93c0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:33.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.677+0000 7fa94e59c700 1 -- 172.21.15.92:0/1650005280 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa93c004810 msgr2=0x7fa93c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:33.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.677+0000 7fa94e59c700 1 --2- 172.21.15.92:0/1650005280 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa93c004810 0x7fa93c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:33.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.677+0000 7fa94e59c700 1 -- 172.21.15.92:0/1650005280 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c003570 con 0x7fa93c003a90 2023-10-06T10:40:33.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.677+0000 7fa94e59c700 1 --2- 172.21.15.92:0/1650005280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa93c003a90 0x7fa93c003ea0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa94800e2d0 tx=0x7fa94800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cda03c466f510bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:33.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.677+0000 7fa94d59a700 1 -- 172.21.15.92:0/1650005280 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94800eb20 con 0x7fa93c003a90 2023-10-06T10:40:33.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.677+0000 7fa94d59a700 1 -- 172.21.15.92:0/1650005280 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94800f040 con 0x7fa93c003a90 2023-10-06T10:40:33.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.677+0000 7fa94d59a700 1 -- 172.21.15.92:0/1650005280 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94800eed0 con 0x7fa93c003a90 2023-10-06T10:40:33.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.678+0000 7fa94f59e700 1 -- 172.21.15.92:0/1650005280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa93c003a90 msgr2=0x7fa93c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:33.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.678+0000 7fa94f59e700 1 --2- 172.21.15.92:0/1650005280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa93c003a90 0x7fa93c003ea0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa94800e2d0 tx=0x7fa94800b040 comp rx=0 tx=0).stop 2023-10-06T10:40:33.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.678+0000 7fa94ed9d700 1 -- 172.21.15.92:0/1650005280 reap_dead start 2023-10-06T10:40:33.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.678+0000 7fa94f59e700 1 -- 172.21.15.92:0/1650005280 shutdown_connections 2023-10-06T10:40:33.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.678+0000 7fa94f59e700 1 -- 172.21.15.92:0/1650005280 >> 172.21.15.92:0/1650005280 conn(0x7fa950099520 msgr2=0x7fa95006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:33.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.678+0000 7fa94f59e700 1 -- 172.21.15.92:0/1650005280 shutdown_connections 2023-10-06T10:40:33.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.678+0000 7fa94f59e700 1 -- 172.21.15.92:0/1650005280 wait complete. 2023-10-06T10:40:33.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.679+0000 7fa94f59e700 1 Processor -- start 2023-10-06T10:40:33.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.679+0000 7fa94f59e700 1 -- start start 2023-10-06T10:40:33.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.680+0000 7fa94f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa93c003a90 0x7fa950169820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:33.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.680+0000 7fa94f59e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9500c7d30 0x7fa9500c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:33.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.680+0000 7fa94f59e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9500c81a0 0x7fa9500c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:33.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.680+0000 7fa94f59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9500b39c0 con 0x7fa93c003a90 2023-10-06T10:40:33.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.680+0000 7fa94f59e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9500c8770 con 0x7fa9500c7d30 2023-10-06T10:40:33.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.680+0000 7fa94f59e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9500b3090 con 0x7fa9500c81a0 2023-10-06T10:40:33.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.680+0000 7fa94e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa93c003a90 0x7fa950169820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:33.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.680+0000 7fa94e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa93c003a90 0x7fa950169820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60534/0 (socket says 172.21.15.92:60534) 2023-10-06T10:40:33.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.680+0000 7fa94e59c700 1 -- 172.21.15.92:0/3616951055 learned_addr learned my addr 172.21.15.92:0/3616951055 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:33.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.680+0000 7fa94e59c700 1 -- 172.21.15.92:0/3616951055 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9500c81a0 msgr2=0x7fa9500c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:33.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.680+0000 7fa94ed9d700 1 --2- 172.21.15.92:0/3616951055 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9500c81a0 0x7fa9500c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:33.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.680+0000 7fa94dd9b700 1 --2- 172.21.15.92:0/3616951055 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9500c7d30 0x7fa9500c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:33.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.680+0000 7fa94e59c700 1 --2- 172.21.15.92:0/3616951055 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9500c81a0 0x7fa9500c95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:33.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.680+0000 7fa94e59c700 1 -- 172.21.15.92:0/3616951055 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9500c7d30 msgr2=0x7fa9500c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:33.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.680+0000 7fa94e59c700 1 --2- 172.21.15.92:0/3616951055 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9500c7d30 0x7fa9500c9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:33.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.680+0000 7fa94e59c700 1 -- 172.21.15.92:0/3616951055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9500b3370 con 0x7fa93c003a90 2023-10-06T10:40:33.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.680+0000 7fa94e59c700 1 --2- 172.21.15.92:0/3616951055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa93c003a90 0x7fa950169820 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00bb30 tx=0x7fa94801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:33.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.680+0000 7fa9467fc700 1 -- 172.21.15.92:0/3616951055 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94801d420 con 0x7fa93c003a90 2023-10-06T10:40:33.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.680+0000 7fa9467fc700 1 -- 172.21.15.92:0/3616951055 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94801d5a0 con 0x7fa93c003a90 2023-10-06T10:40:33.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.680+0000 7fa94f59e700 1 -- 172.21.15.92:0/3616951055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9500b3650 con 0x7fa93c003a90 2023-10-06T10:40:33.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.680+0000 7fa9467fc700 1 -- 172.21.15.92:0/3616951055 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94801d960 con 0x7fa93c003a90 2023-10-06T10:40:33.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.680+0000 7fa94f59e700 1 -- 172.21.15.92:0/3616951055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa950155640 con 0x7fa93c003a90 2023-10-06T10:40:33.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.681+0000 7fa94f59e700 1 -- 172.21.15.92:0/3616951055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9500b3ca0 con 0x7fa93c003a90 2023-10-06T10:40:33.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.682+0000 7fa9467fc700 1 -- 172.21.15.92:0/3616951055 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa948003040 con 0x7fa93c003a90 2023-10-06T10:40:33.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.682+0000 7fa9467fc700 1 --2- 172.21.15.92:0/3616951055 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa930066f30 0x7fa9300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:33.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.682+0000 7fa9467fc700 1 -- 172.21.15.92:0/3616951055 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5896+0+0 (secure 0 0 0) 0x7fa948086b00 con 0x7fa93c003a90 2023-10-06T10:40:33.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.682+0000 7fa94dd9b700 1 --2- 172.21.15.92:0/3616951055 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa930066f30 0x7fa9300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:33.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.683+0000 7fa94dd9b700 1 --2- 172.21.15.92:0/3616951055 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa930066f30 0x7fa9300693e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa9500b62a0 tx=0x7fa938009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:33.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.687+0000 7fa9467fc700 1 -- 172.21.15.92:0/3616951055 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa948053cb0 con 0x7fa93c003a90 2023-10-06T10:40:33.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:33.868+0000 7fa94f59e700 1 -- 172.21.15.92:0/3616951055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa95005f230 con 0x7fa93c003a90 2023-10-06T10:40:34.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.543+0000 7fa9467fc700 1 -- 172.21.15.92:0/3616951055 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v50) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa9480180d0 con 0x7fa93c003a90 2023-10-06T10:40:34.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.550+0000 7fa94f59e700 1 -- 172.21.15.92:0/3616951055 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa930066f30 msgr2=0x7fa9300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:34.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.550+0000 7fa94f59e700 1 --2- 172.21.15.92:0/3616951055 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa930066f30 0x7fa9300693e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa9500b62a0 tx=0x7fa938009040 comp rx=0 tx=0).stop 2023-10-06T10:40:34.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.550+0000 7fa94f59e700 1 -- 172.21.15.92:0/3616951055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa93c003a90 msgr2=0x7fa950169820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:34.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.550+0000 7fa94f59e700 1 --2- 172.21.15.92:0/3616951055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa93c003a90 0x7fa950169820 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00bb30 tx=0x7fa94801a000 comp rx=0 tx=0).stop 2023-10-06T10:40:34.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.552+0000 7fa94f59e700 1 -- 172.21.15.92:0/3616951055 shutdown_connections 2023-10-06T10:40:34.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.552+0000 7fa94f59e700 1 --2- 172.21.15.92:0/3616951055 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa930066f30 0x7fa9300693e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:34.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.552+0000 7fa94f59e700 1 --2- 172.21.15.92:0/3616951055 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9500c81a0 0x7fa9500c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:34.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.552+0000 7fa94f59e700 1 --2- 172.21.15.92:0/3616951055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa93c003a90 0x7fa950169820 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:34.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.552+0000 7fa94f59e700 1 --2- 172.21.15.92:0/3616951055 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9500c7d30 0x7fa9500c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:34.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.552+0000 7fa94f59e700 1 -- 172.21.15.92:0/3616951055 >> 172.21.15.92:0/3616951055 conn(0x7fa950099520 msgr2=0x7fa9500ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:34.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.556+0000 7fa94f59e700 1 -- 172.21.15.92:0/3616951055 shutdown_connections 2023-10-06T10:40:34.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.557+0000 7fa94f59e700 1 -- 172.21.15.92:0/3616951055 wait complete. 2023-10-06T10:40:34.563 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-10-06T10:40:34.576 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T10:40:34.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.693+0000 7ff8f3853700 1 Processor -- start 2023-10-06T10:40:34.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.694+0000 7ff8f3853700 1 -- start start 2023-10-06T10:40:34.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.694+0000 7ff8f3853700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8ec1594d0 0x7ff8ec0cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:34.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.694+0000 7ff8f3853700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8ec0cbec0 con 0x7ff8ec1598a0 2023-10-06T10:40:34.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.694+0000 7ff8f3853700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8ec0cf5d0 con 0x7ff8ec0cbaa0 2023-10-06T10:40:34.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.694+0000 7ff8f3853700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec0cf8b0 con 0x7ff8ec1594d0 2023-10-06T10:40:34.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.695+0000 7ff8f0dee700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff8ec1598a0 0x7ff8ec0ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50294/0 (socket says 172.21.15.92:50294) 2023-10-06T10:40:34.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.695+0000 7ff8f0dee700 1 -- 172.21.15.92:0/101482719 learned_addr learned my addr 172.21.15.92:0/101482719 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:34.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.695+0000 7ff8f15ef700 1 --2- 172.21.15.92:0/101482719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8ec1594d0 0x7ff8ec0cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:34.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.695+0000 7ff8e3fff700 1 -- 172.21.15.92:0/101482719 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 615873737 0 0) 0x7ff8ec0cbec0 con 0x7ff8ec1598a0 2023-10-06T10:40:34.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.695+0000 7ff8e3fff700 1 -- 172.21.15.92:0/101482719 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8d8003570 con 0x7ff8ec1598a0 2023-10-06T10:40:34.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.695+0000 7ff8e3fff700 1 -- 172.21.15.92:0/101482719 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2404177902 0 0) 0x7ff8ec0cf5d0 con 0x7ff8ec0cbaa0 2023-10-06T10:40:34.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.695+0000 7ff8e3fff700 1 -- 172.21.15.92:0/101482719 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8ec0cbec0 con 0x7ff8ec0cbaa0 2023-10-06T10:40:34.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.695+0000 7ff8f15ef700 1 -- 172.21.15.92:0/101482719 >> v1:172.21.15.124:6789/0 conn(0x7ff8ec0cbaa0 legacy=0x7ff8ec0c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:34.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.695+0000 7ff8f15ef700 1 -- 172.21.15.92:0/101482719 >> v1:172.21.15.92:6789/0 conn(0x7ff8ec1598a0 legacy=0x7ff8ec0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:34.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.695+0000 7ff8f15ef700 1 -- 172.21.15.92:0/101482719 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ec0cfb90 con 0x7ff8ec1594d0 2023-10-06T10:40:34.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.695+0000 7ff8f15ef700 1 --2- 172.21.15.92:0/101482719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8ec1594d0 0x7ff8ec0cb560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009e90 tx=0x7ff8e8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff8b716c395d2720 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:34.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.695+0000 7ff8e3fff700 1 -- 172.21.15.92:0/101482719 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e800e070 con 0x7ff8ec1594d0 2023-10-06T10:40:34.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.695+0000 7ff8e3fff700 1 -- 172.21.15.92:0/101482719 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e800bb80 con 0x7ff8ec1594d0 2023-10-06T10:40:34.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.695+0000 7ff8e3fff700 1 -- 172.21.15.92:0/101482719 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8012430 con 0x7ff8ec1594d0 2023-10-06T10:40:34.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.696+0000 7ff8f3853700 1 -- 172.21.15.92:0/101482719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8ec1594d0 msgr2=0x7ff8ec0cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:34.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.696+0000 7ff8f3853700 1 --2- 172.21.15.92:0/101482719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8ec1594d0 0x7ff8ec0cb560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009e90 tx=0x7ff8e8009780 comp rx=0 tx=0).stop 2023-10-06T10:40:34.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.696+0000 7ff8f3853700 1 -- 172.21.15.92:0/101482719 shutdown_connections 2023-10-06T10:40:34.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.696+0000 7ff8f3853700 1 --2- 172.21.15.92:0/101482719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8ec1594d0 0x7ff8ec0cb560 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:34.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.696+0000 7ff8f3853700 1 -- 172.21.15.92:0/101482719 >> 172.21.15.92:0/101482719 conn(0x7ff8ec089e50 msgr2=0x7ff8ec099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:34.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.696+0000 7ff8f3853700 1 -- 172.21.15.92:0/101482719 shutdown_connections 2023-10-06T10:40:34.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.696+0000 7ff8f3853700 1 -- 172.21.15.92:0/101482719 wait complete. 2023-10-06T10:40:34.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.697+0000 7ff8f3853700 1 Processor -- start 2023-10-06T10:40:34.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.697+0000 7ff8f3853700 1 -- start start 2023-10-06T10:40:34.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.698+0000 7ff8f3853700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8ec0cbaa0 0x7ff8ec14ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:34.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.698+0000 7ff8f3853700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8ec1594d0 0x7ff8ec069780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:34.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.698+0000 7ff8f3853700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8ec1598a0 0x7ff8ec069cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:34.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.698+0000 7ff8f3853700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec0c9f20 con 0x7ff8ec1594d0 2023-10-06T10:40:34.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.698+0000 7ff8f3853700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec14f3b0 con 0x7ff8ec0cbaa0 2023-10-06T10:40:34.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.698+0000 7ff8f3853700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec09ae30 con 0x7ff8ec1598a0 2023-10-06T10:40:34.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.698+0000 7ff8f0dee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8ec1594d0 0x7ff8ec069780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:34.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.698+0000 7ff8f0dee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8ec1594d0 0x7ff8ec069780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60548/0 (socket says 172.21.15.92:60548) 2023-10-06T10:40:34.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.698+0000 7ff8f0dee700 1 -- 172.21.15.92:0/115085988 learned_addr learned my addr 172.21.15.92:0/115085988 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:34.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.698+0000 7ff8f1df0700 1 --2- 172.21.15.92:0/115085988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8ec1598a0 0x7ff8ec069cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:34.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.698+0000 7ff8f15ef700 1 --2- 172.21.15.92:0/115085988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8ec0cbaa0 0x7ff8ec14ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:34.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.698+0000 7ff8f0dee700 1 -- 172.21.15.92:0/115085988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8ec1598a0 msgr2=0x7ff8ec069cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:34.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.698+0000 7ff8f0dee700 1 --2- 172.21.15.92:0/115085988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8ec1598a0 0x7ff8ec069cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:34.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.698+0000 7ff8f0dee700 1 -- 172.21.15.92:0/115085988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8ec0cbaa0 msgr2=0x7ff8ec14ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:34.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.698+0000 7ff8f0dee700 1 --2- 172.21.15.92:0/115085988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8ec0cbaa0 0x7ff8ec14ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:34.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.698+0000 7ff8f0dee700 1 -- 172.21.15.92:0/115085988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e8009a20 con 0x7ff8ec1594d0 2023-10-06T10:40:34.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.699+0000 7ff8f0dee700 1 --2- 172.21.15.92:0/115085988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8ec1594d0 0x7ff8ec069780 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00c580 tx=0x7ff8dc007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:34.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.699+0000 7ff8e17fa700 1 -- 172.21.15.92:0/115085988 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc005b30 con 0x7ff8ec1594d0 2023-10-06T10:40:34.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.699+0000 7ff8e17fa700 1 -- 172.21.15.92:0/115085988 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8dc005c90 con 0x7ff8ec1594d0 2023-10-06T10:40:34.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.699+0000 7ff8e17fa700 1 -- 172.21.15.92:0/115085988 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc007300 con 0x7ff8ec1594d0 2023-10-06T10:40:34.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.699+0000 7ff8f3853700 1 -- 172.21.15.92:0/115085988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ec09b110 con 0x7ff8ec1594d0 2023-10-06T10:40:34.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.699+0000 7ff8f3853700 1 -- 172.21.15.92:0/115085988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ec0c9bb0 con 0x7ff8ec1594d0 2023-10-06T10:40:34.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.700+0000 7ff8f3853700 1 -- 172.21.15.92:0/115085988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8ec064a90 con 0x7ff8ec1594d0 2023-10-06T10:40:34.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.701+0000 7ff8e17fa700 1 -- 172.21.15.92:0/115085988 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff8dc01b3b0 con 0x7ff8ec1594d0 2023-10-06T10:40:34.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.701+0000 7ff8e17fa700 1 --2- 172.21.15.92:0/115085988 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff8d0066f30 0x7ff8d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:34.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.701+0000 7ff8f15ef700 1 --2- 172.21.15.92:0/115085988 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff8d0066f30 0x7ff8d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:34.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.701+0000 7ff8e17fa700 1 -- 172.21.15.92:0/115085988 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6278+0+0 (secure 0 0 0) 0x7ff8dc084140 con 0x7ff8ec1594d0 2023-10-06T10:40:34.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.701+0000 7ff8f15ef700 1 --2- 172.21.15.92:0/115085988 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff8d0066f30 0x7ff8d00693e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009de0 tx=0x7ff8e8019000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:34.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.705+0000 7ff8e17fa700 1 -- 172.21.15.92:0/115085988 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff8dc0511f0 con 0x7ff8ec1594d0 2023-10-06T10:40:34.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:34.930+0000 7ff8f3853700 1 -- 172.21.15.92:0/115085988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff8ec0ca060 con 0x7ff8ec1594d0 2023-10-06T10:40:35.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.544+0000 7ff8e17fa700 1 -- 172.21.15.92:0/115085988 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v28) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff8dc054810 con 0x7ff8ec1594d0 2023-10-06T10:40:35.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.548+0000 7ff8f3853700 1 -- 172.21.15.92:0/115085988 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff8d0066f30 msgr2=0x7ff8d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:35.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.548+0000 7ff8f3853700 1 --2- 172.21.15.92:0/115085988 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff8d0066f30 0x7ff8d00693e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009de0 tx=0x7ff8e8019000 comp rx=0 tx=0).stop 2023-10-06T10:40:35.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.548+0000 7ff8f3853700 1 -- 172.21.15.92:0/115085988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8ec1594d0 msgr2=0x7ff8ec069780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:35.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.548+0000 7ff8f3853700 1 --2- 172.21.15.92:0/115085988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8ec1594d0 0x7ff8ec069780 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00c580 tx=0x7ff8dc007e40 comp rx=0 tx=0).stop 2023-10-06T10:40:35.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.548+0000 7ff8f3853700 1 -- 172.21.15.92:0/115085988 shutdown_connections 2023-10-06T10:40:35.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.548+0000 7ff8f3853700 1 --2- 172.21.15.92:0/115085988 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff8d0066f30 0x7ff8d00693e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:35.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.548+0000 7ff8f3853700 1 --2- 172.21.15.92:0/115085988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8ec1598a0 0x7ff8ec069cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:35.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.548+0000 7ff8f3853700 1 --2- 172.21.15.92:0/115085988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8ec0cbaa0 0x7ff8ec14ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:35.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.548+0000 7ff8f3853700 1 --2- 172.21.15.92:0/115085988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8ec1594d0 0x7ff8ec069780 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:35.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.548+0000 7ff8f3853700 1 -- 172.21.15.92:0/115085988 >> 172.21.15.92:0/115085988 conn(0x7ff8ec089e50 msgr2=0x7ff8ec068370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:35.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.550+0000 7ff8f3853700 1 -- 172.21.15.92:0/115085988 shutdown_connections 2023-10-06T10:40:35.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.550+0000 7ff8f3853700 1 -- 172.21.15.92:0/115085988 wait complete. 2023-10-06T10:40:35.558 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 8 and data pool 9 2023-10-06T10:40:35.582 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:40:35.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.694+0000 7fd65c63a700 1 Processor -- start 2023-10-06T10:40:35.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.694+0000 7fd65c63a700 1 -- start start 2023-10-06T10:40:35.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.695+0000 7fd65c63a700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd654069ab0 0x7fd6540ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:35.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.695+0000 7fd65c63a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd65406d350 0x7fd65406eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:35.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.695+0000 7fd65c63a700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65406e6d0 con 0x7fd6540696e0 2023-10-06T10:40:35.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.695+0000 7fd65c63a700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6540cd000 con 0x7fd654069ab0 2023-10-06T10:40:35.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.695+0000 7fd65c63a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65406dac0 con 0x7fd65406d350 2023-10-06T10:40:35.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.695+0000 7fd65abd7700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd65406d350 0x7fd65406eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:35.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.695+0000 7fd659bd5700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd654069ab0 0x7fd6540ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:35.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.695+0000 7fd65abd7700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd65406d350 0x7fd65406eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:47088/0 (socket says 172.21.15.92:47088) 2023-10-06T10:40:35.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.695+0000 7fd65a3d6700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fd6540696e0 0x7fd6540aeeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:38028/0 (socket says 172.21.15.92:38028) 2023-10-06T10:40:35.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.695+0000 7fd65abd7700 1 -- 172.21.15.92:0/796170244 learned_addr learned my addr 172.21.15.92:0/796170244 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.695+0000 7fd65abd7700 1 -- 172.21.15.92:0/796170244 >> v2:172.21.15.124:3300/0 conn(0x7fd654069ab0 msgr2=0x7fd6540ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.695+0000 7fd65abd7700 1 --2- 172.21.15.92:0/796170244 >> v2:172.21.15.124:3300/0 conn(0x7fd654069ab0 0x7fd6540ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.695+0000 7fd65abd7700 1 -- 172.21.15.92:0/796170244 >> v1:172.21.15.124:6789/0 conn(0x7fd6540696e0 legacy=0x7fd6540aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.695+0000 7fd65abd7700 1 -- 172.21.15.92:0/796170244 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65406dda0 con 0x7fd65406d350 2023-10-06T10:40:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.695+0000 7fd659bd5700 1 --2- 172.21.15.92:0/796170244 >> v2:172.21.15.124:3300/0 conn(0x7fd654069ab0 0x7fd6540ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:40:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.696+0000 7fd65abd7700 1 --2- 172.21.15.92:0/796170244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd65406d350 0x7fd65406eb60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009f20 tx=0x7fd64c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abdcda51b1918d89 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.696+0000 7fd6593d4700 1 -- 172.21.15.92:0/796170244 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c00e070 con 0x7fd65406d350 2023-10-06T10:40:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.696+0000 7fd6593d4700 1 -- 172.21.15.92:0/796170244 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64c00b9d0 con 0x7fd65406d350 2023-10-06T10:40:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.696+0000 7fd6593d4700 1 -- 172.21.15.92:0/796170244 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c0092e0 con 0x7fd65406d350 2023-10-06T10:40:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.696+0000 7fd65c63a700 1 -- 172.21.15.92:0/796170244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd65406d350 msgr2=0x7fd65406eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.696+0000 7fd65c63a700 1 --2- 172.21.15.92:0/796170244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd65406d350 0x7fd65406eb60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009f20 tx=0x7fd64c009840 comp rx=0 tx=0).stop 2023-10-06T10:40:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.696+0000 7fd65c63a700 1 -- 172.21.15.92:0/796170244 shutdown_connections 2023-10-06T10:40:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.696+0000 7fd65c63a700 1 --2- 172.21.15.92:0/796170244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd65406d350 0x7fd65406eb60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.696+0000 7fd65c63a700 1 --2- 172.21.15.92:0/796170244 >> v2:172.21.15.124:3300/0 conn(0x7fd654069ab0 0x7fd6540ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.696+0000 7fd65c63a700 1 -- 172.21.15.92:0/796170244 >> 172.21.15.92:0/796170244 conn(0x7fd654068080 msgr2=0x7fd6540cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.696+0000 7fd65c63a700 1 -- 172.21.15.92:0/796170244 shutdown_connections 2023-10-06T10:40:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.696+0000 7fd65c63a700 1 -- 172.21.15.92:0/796170244 wait complete. 2023-10-06T10:40:35.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.698+0000 7fd65c63a700 1 Processor -- start 2023-10-06T10:40:35.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.698+0000 7fd65c63a700 1 -- start start 2023-10-06T10:40:35.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.698+0000 7fd65c63a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6540696e0 0x7fd6541490d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:35.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.698+0000 7fd65c63a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd654069ab0 0x7fd6541484b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:35.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.698+0000 7fd65c63a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd65406d350 0x7fd65415aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:35.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.698+0000 7fd65c63a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65415c0f0 con 0x7fd6540696e0 2023-10-06T10:40:35.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.698+0000 7fd65c63a700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65415b4e0 con 0x7fd654069ab0 2023-10-06T10:40:35.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.698+0000 7fd65c63a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65415b7c0 con 0x7fd65406d350 2023-10-06T10:40:35.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.698+0000 7fd65a3d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6540696e0 0x7fd6541490d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:35.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.698+0000 7fd65a3d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6540696e0 0x7fd6541490d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60562/0 (socket says 172.21.15.92:60562) 2023-10-06T10:40:35.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.698+0000 7fd65a3d6700 1 -- 172.21.15.92:0/442566944 learned_addr learned my addr 172.21.15.92:0/442566944 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.698+0000 7fd65abd7700 1 --2- 172.21.15.92:0/442566944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd65406d350 0x7fd65415aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.699+0000 7fd659bd5700 1 --2- 172.21.15.92:0/442566944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd654069ab0 0x7fd6541484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.699+0000 7fd65a3d6700 1 -- 172.21.15.92:0/442566944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd65406d350 msgr2=0x7fd65415aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.699+0000 7fd65a3d6700 1 --2- 172.21.15.92:0/442566944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd65406d350 0x7fd65415aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.699+0000 7fd65a3d6700 1 -- 172.21.15.92:0/442566944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd654069ab0 msgr2=0x7fd6541484b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.699+0000 7fd65a3d6700 1 --2- 172.21.15.92:0/442566944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd654069ab0 0x7fd6541484b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.699+0000 7fd65a3d6700 1 -- 172.21.15.92:0/442566944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd644009420 con 0x7fd6540696e0 2023-10-06T10:40:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.699+0000 7fd65a3d6700 1 --2- 172.21.15.92:0/442566944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6540696e0 0x7fd6541490d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd65000c7c0 tx=0x7fd650007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.699+0000 7fd64a7fc700 1 -- 172.21.15.92:0/442566944 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd650005a70 con 0x7fd6540696e0 2023-10-06T10:40:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.699+0000 7fd64a7fc700 1 -- 172.21.15.92:0/442566944 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd650005bf0 con 0x7fd6540696e0 2023-10-06T10:40:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.699+0000 7fd64a7fc700 1 -- 172.21.15.92:0/442566944 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6500070d0 con 0x7fd6540696e0 2023-10-06T10:40:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.699+0000 7fd65c63a700 1 -- 172.21.15.92:0/442566944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64c009ae0 con 0x7fd6540696e0 2023-10-06T10:40:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.699+0000 7fd65c63a700 1 -- 172.21.15.92:0/442566944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65415bd80 con 0x7fd6540696e0 2023-10-06T10:40:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.700+0000 7fd65c63a700 1 -- 172.21.15.92:0/442566944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd654064a90 con 0x7fd6540696e0 2023-10-06T10:40:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.700+0000 7fd64a7fc700 1 -- 172.21.15.92:0/442566944 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd650007480 con 0x7fd6540696e0 2023-10-06T10:40:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.701+0000 7fd64a7fc700 1 --2- 172.21.15.92:0/442566944 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd638066f30 0x7fd6380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.701+0000 7fd64a7fc700 1 -- 172.21.15.92:0/442566944 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fd650083b30 con 0x7fd6540696e0 2023-10-06T10:40:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.701+0000 7fd659bd5700 1 --2- 172.21.15.92:0/442566944 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd638066f30 0x7fd6380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.701+0000 7fd659bd5700 1 --2- 172.21.15.92:0/442566944 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd638066f30 0x7fd6380693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd65416b020 tx=0x7fd6440091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.705+0000 7fd64a7fc700 1 -- 172.21.15.92:0/442566944 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd650050b00 con 0x7fd6540696e0 2023-10-06T10:40:35.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.888+0000 7fd65c63a700 1 -- 172.21.15.92:0/442566944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd65404bb40 con 0x7fd6540696e0 2023-10-06T10:40:35.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.889+0000 7fd64a7fc700 1 -- 172.21.15.92:0/442566944 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+17084 (secure 0 0 0) 0x7fd650017020 con 0x7fd6540696e0 2023-10-06T10:40:35.891 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:35.891 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":51,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:40:34.932239+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:40:32.570535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-06T10:40:33.886794+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:40:35.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.891+0000 7fd65c63a700 1 -- 172.21.15.92:0/442566944 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd638066f30 msgr2=0x7fd6380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:35.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.891+0000 7fd65c63a700 1 --2- 172.21.15.92:0/442566944 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd638066f30 0x7fd6380693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd65416b020 tx=0x7fd6440091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:35.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.891+0000 7fd65c63a700 1 -- 172.21.15.92:0/442566944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6540696e0 msgr2=0x7fd6541490d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:35.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.891+0000 7fd65c63a700 1 --2- 172.21.15.92:0/442566944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6540696e0 0x7fd6541490d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd65000c7c0 tx=0x7fd650007f70 comp rx=0 tx=0).stop 2023-10-06T10:40:35.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.891+0000 7fd65c63a700 1 -- 172.21.15.92:0/442566944 shutdown_connections 2023-10-06T10:40:35.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.891+0000 7fd65c63a700 1 --2- 172.21.15.92:0/442566944 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd638066f30 0x7fd6380693e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:35.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.891+0000 7fd65c63a700 1 --2- 172.21.15.92:0/442566944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd65406d350 0x7fd65415aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:35.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.891+0000 7fd65c63a700 1 --2- 172.21.15.92:0/442566944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6540696e0 0x7fd6541490d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:35.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.891+0000 7fd65c63a700 1 --2- 172.21.15.92:0/442566944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd654069ab0 0x7fd6541484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:35.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.891+0000 7fd65c63a700 1 -- 172.21.15.92:0/442566944 >> 172.21.15.92:0/442566944 conn(0x7fd654068080 msgr2=0x7fd6540c30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:35.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.891+0000 7fd65c63a700 1 -- 172.21.15.92:0/442566944 shutdown_connections 2023-10-06T10:40:35.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:35.891+0000 7fd65c63a700 1 -- 172.21.15.92:0/442566944 wait complete. 2023-10-06T10:40:35.905 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:40:36.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.016+0000 7fc25edbd700 1 Processor -- start 2023-10-06T10:40:36.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.016+0000 7fc25edbd700 1 -- start start 2023-10-06T10:40:36.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.017+0000 7fc25edbd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc258159440 0x7fc2580b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:36.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.017+0000 7fc25edbd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2580b35d0 0x7fc2580cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:36.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.017+0000 7fc25edbd700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2580cb7a0 con 0x7fc2580cc170 2023-10-06T10:40:36.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.017+0000 7fc25edbd700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2580b0860 con 0x7fc258159440 2023-10-06T10:40:36.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.017+0000 7fc25edbd700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2580cb150 con 0x7fc2580b35d0 2023-10-06T10:40:36.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.017+0000 7fc25ddbb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc258159440 0x7fc2580b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:36.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.017+0000 7fc25ddbb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc258159440 0x7fc2580b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60570/0 (socket says 172.21.15.92:60570) 2023-10-06T10:40:36.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.017+0000 7fc25ddbb700 1 -- 172.21.15.92:0/2178495990 learned_addr learned my addr 172.21.15.92:0/2178495990 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:36.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.018+0000 7fc25d5ba700 1 --2- 172.21.15.92:0/2178495990 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2580b35d0 0x7fc2580cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:36.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.018+0000 7fc25ddbb700 1 -- 172.21.15.92:0/2178495990 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2580b35d0 msgr2=0x7fc2580cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:36.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.018+0000 7fc25ddbb700 1 --2- 172.21.15.92:0/2178495990 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2580b35d0 0x7fc2580cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:36.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.018+0000 7fc25ddbb700 1 -- 172.21.15.92:0/2178495990 >> v1:172.21.15.124:6789/0 conn(0x7fc2580cc170 legacy=0x7fc2580b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:36.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.018+0000 7fc25ddbb700 1 -- 172.21.15.92:0/2178495990 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2580cb430 con 0x7fc258159440 2023-10-06T10:40:36.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.018+0000 7fc25d5ba700 1 --2- 172.21.15.92:0/2178495990 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2580b35d0 0x7fc2580cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:36.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.018+0000 7fc25ddbb700 1 --2- 172.21.15.92:0/2178495990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc258159440 0x7fc2580b3090 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc2540099b0 tx=0x7fc254009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e324809b56f6c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:36.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.018+0000 7fc25cdb9700 1 -- 172.21.15.92:0/2178495990 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25400b820 con 0x7fc258159440 2023-10-06T10:40:36.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.018+0000 7fc25cdb9700 1 -- 172.21.15.92:0/2178495990 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25400b980 con 0x7fc258159440 2023-10-06T10:40:36.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.018+0000 7fc25cdb9700 1 -- 172.21.15.92:0/2178495990 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25400bd20 con 0x7fc258159440 2023-10-06T10:40:36.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.018+0000 7fc25edbd700 1 -- 172.21.15.92:0/2178495990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc258159440 msgr2=0x7fc2580b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:36.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.018+0000 7fc25edbd700 1 --2- 172.21.15.92:0/2178495990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc258159440 0x7fc2580b3090 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc2540099b0 tx=0x7fc254009250 comp rx=0 tx=0).stop 2023-10-06T10:40:36.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.018+0000 7fc25edbd700 1 -- 172.21.15.92:0/2178495990 shutdown_connections 2023-10-06T10:40:36.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.018+0000 7fc25edbd700 1 --2- 172.21.15.92:0/2178495990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc258159440 0x7fc2580b3090 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:36.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.018+0000 7fc25edbd700 1 --2- 172.21.15.92:0/2178495990 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2580b35d0 0x7fc2580cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:36.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.018+0000 7fc25edbd700 1 -- 172.21.15.92:0/2178495990 >> 172.21.15.92:0/2178495990 conn(0x7fc25806ece0 msgr2=0x7fc25806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:36.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.019+0000 7fc25edbd700 1 -- 172.21.15.92:0/2178495990 shutdown_connections 2023-10-06T10:40:36.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.019+0000 7fc25edbd700 1 -- 172.21.15.92:0/2178495990 wait complete. 2023-10-06T10:40:36.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.020+0000 7fc25edbd700 1 Processor -- start 2023-10-06T10:40:36.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.020+0000 7fc25edbd700 1 -- start start 2023-10-06T10:40:36.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.020+0000 7fc25edbd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2580b35d0 0x7fc258154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:36.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.020+0000 7fc25edbd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2580cc170 0x7fc25815aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:36.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.020+0000 7fc25edbd700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc258159440 0x7fc25815afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:36.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.020+0000 7fc25edbd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2580c6f40 con 0x7fc2580cc170 2023-10-06T10:40:36.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.020+0000 7fc25edbd700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2580c7690 con 0x7fc258159440 2023-10-06T10:40:36.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.020+0000 7fc25edbd700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2580c7970 con 0x7fc2580b35d0 2023-10-06T10:40:36.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.020+0000 7fc25d5ba700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2580cc170 0x7fc25815aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:36.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.020+0000 7fc25d5ba700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2580cc170 0x7fc25815aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60582/0 (socket says 172.21.15.92:60582) 2023-10-06T10:40:36.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.020+0000 7fc25d5ba700 1 -- 172.21.15.92:0/2822394954 learned_addr learned my addr 172.21.15.92:0/2822394954 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:36.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.021+0000 7fc25d5ba700 1 -- 172.21.15.92:0/2822394954 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2580b35d0 msgr2=0x7fc258154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:36.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.021+0000 7fc25ddbb700 1 --2- 172.21.15.92:0/2822394954 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2580b35d0 0x7fc258154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:36.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.021+0000 7fc25e5bc700 1 --2- 172.21.15.92:0/2822394954 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc258159440 0x7fc25815afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:36.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.021+0000 7fc25d5ba700 1 --2- 172.21.15.92:0/2822394954 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2580b35d0 0x7fc258154a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:36.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.021+0000 7fc25d5ba700 1 -- 172.21.15.92:0/2822394954 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc258159440 msgr2=0x7fc25815afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:36.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.021+0000 7fc25d5ba700 1 --2- 172.21.15.92:0/2822394954 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc258159440 0x7fc25815afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:36.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.021+0000 7fc25d5ba700 1 -- 172.21.15.92:0/2822394954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2540094f0 con 0x7fc2580cc170 2023-10-06T10:40:36.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.021+0000 7fc25d5ba700 1 --2- 172.21.15.92:0/2822394954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2580cc170 0x7fc25815aa60 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c830 tx=0x7fc248008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:36.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.021+0000 7fc24dffb700 1 -- 172.21.15.92:0/2822394954 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc248011600 con 0x7fc2580cc170 2023-10-06T10:40:36.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.021+0000 7fc24dffb700 1 -- 172.21.15.92:0/2822394954 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc248011760 con 0x7fc2580cc170 2023-10-06T10:40:36.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.021+0000 7fc25edbd700 1 -- 172.21.15.92:0/2822394954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2580c7c50 con 0x7fc2580cc170 2023-10-06T10:40:36.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.021+0000 7fc24dffb700 1 -- 172.21.15.92:0/2822394954 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc248011b20 con 0x7fc2580cc170 2023-10-06T10:40:36.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.021+0000 7fc25edbd700 1 -- 172.21.15.92:0/2822394954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2580c41f0 con 0x7fc2580cc170 2023-10-06T10:40:36.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.022+0000 7fc25edbd700 1 -- 172.21.15.92:0/2822394954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2580c8140 con 0x7fc2580cc170 2023-10-06T10:40:36.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.023+0000 7fc24dffb700 1 -- 172.21.15.92:0/2822394954 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc24800a3e0 con 0x7fc2580cc170 2023-10-06T10:40:36.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.023+0000 7fc24dffb700 1 --2- 172.21.15.92:0/2822394954 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc23c066f30 0x7fc23c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:36.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.023+0000 7fc24dffb700 1 -- 172.21.15.92:0/2822394954 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc248084c60 con 0x7fc2580cc170 2023-10-06T10:40:36.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.023+0000 7fc25ddbb700 1 --2- 172.21.15.92:0/2822394954 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc23c066f30 0x7fc23c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:36.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.023+0000 7fc25ddbb700 1 --2- 172.21.15.92:0/2822394954 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc23c066f30 0x7fc23c0693e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc2540099b0 tx=0x7fc254016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:36.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.027+0000 7fc24dffb700 1 -- 172.21.15.92:0/2822394954 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc248051c30 con 0x7fc2580cc170 2023-10-06T10:40:36.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.213+0000 7fc25edbd700 1 -- 172.21.15.92:0/2822394954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc25805f230 con 0x7fc2580cc170 2023-10-06T10:40:36.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.214+0000 7fc24dffb700 1 -- 172.21.15.92:0/2822394954 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+17084 (secure 0 0 0) 0x7fc248015090 con 0x7fc2580cc170 2023-10-06T10:40:36.217 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:36.217 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":51,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:40:34.932239+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:40:32.570535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-06T10:40:33.886794+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:40:36.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.216+0000 7fc25edbd700 1 -- 172.21.15.92:0/2822394954 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc23c066f30 msgr2=0x7fc23c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.216+0000 7fc25edbd700 1 --2- 172.21.15.92:0/2822394954 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc23c066f30 0x7fc23c0693e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc2540099b0 tx=0x7fc254016000 comp rx=0 tx=0).stop 2023-10-06T10:40:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.216+0000 7fc25edbd700 1 -- 172.21.15.92:0/2822394954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2580cc170 msgr2=0x7fc25815aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.216+0000 7fc25edbd700 1 --2- 172.21.15.92:0/2822394954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2580cc170 0x7fc25815aa60 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c830 tx=0x7fc248008040 comp rx=0 tx=0).stop 2023-10-06T10:40:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.216+0000 7fc25edbd700 1 -- 172.21.15.92:0/2822394954 shutdown_connections 2023-10-06T10:40:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.216+0000 7fc25edbd700 1 --2- 172.21.15.92:0/2822394954 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc23c066f30 0x7fc23c0693e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.216+0000 7fc25edbd700 1 --2- 172.21.15.92:0/2822394954 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc258159440 0x7fc25815afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.216+0000 7fc25edbd700 1 --2- 172.21.15.92:0/2822394954 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2580b35d0 0x7fc258154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.216+0000 7fc25edbd700 1 --2- 172.21.15.92:0/2822394954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2580cc170 0x7fc25815aa60 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.216+0000 7fc25edbd700 1 -- 172.21.15.92:0/2822394954 >> 172.21.15.92:0/2822394954 conn(0x7fc25806ece0 msgr2=0x7fc2580b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:36.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.216+0000 7fc25edbd700 1 -- 172.21.15.92:0/2822394954 shutdown_connections 2023-10-06T10:40:36.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.216+0000 7fc25edbd700 1 -- 172.21.15.92:0/2822394954 wait complete. 2023-10-06T10:40:36.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T10:40:36.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.346+0000 7f7766d15700 1 Processor -- start 2023-10-06T10:40:36.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.346+0000 7f7766d15700 1 -- start start 2023-10-06T10:40:36.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.347+0000 7f7766d15700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77601598a0 0x7f77600ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:36.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.347+0000 7f7766d15700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f77600cbc70 0x7f77600ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:36.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.347+0000 7f7766d15700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77600cf5d0 con 0x7f77601594d0 2023-10-06T10:40:36.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.347+0000 7f7766d15700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77600cf8b0 con 0x7f77600cbc70 2023-10-06T10:40:36.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.347+0000 7f7766d15700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77600cfb90 con 0x7f77601598a0 2023-10-06T10:40:36.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.347+0000 7f7764ab1700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f77601594d0 0x7f77600cb560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50302/0 (socket says 172.21.15.92:50302) 2023-10-06T10:40:36.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.347+0000 7f7764ab1700 1 -- 172.21.15.92:0/3712330663 learned_addr learned my addr 172.21.15.92:0/3712330663 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:36.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.347+0000 7f77577fe700 1 -- 172.21.15.92:0/3712330663 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1035651706 0 0) 0x7f77600cf5d0 con 0x7f77601594d0 2023-10-06T10:40:36.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.347+0000 7f77577fe700 1 -- 172.21.15.92:0/3712330663 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7748003570 con 0x7f77601594d0 2023-10-06T10:40:36.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.347+0000 7f7757fff700 1 --2- 172.21.15.92:0/3712330663 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77601598a0 0x7f77600ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:36.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.347+0000 7f77652b2700 1 --2- 172.21.15.92:0/3712330663 >> v2:172.21.15.124:3300/0 conn(0x7f77600cbc70 0x7f77600ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:36.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f77577fe700 1 -- 172.21.15.92:0/3712330663 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2322766177 0 0) 0x7f7748003570 con 0x7f77601594d0 2023-10-06T10:40:36.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f77577fe700 1 -- 172.21.15.92:0/3712330663 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77601598a0 msgr2=0x7f77600ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:36.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f77577fe700 1 --2- 172.21.15.92:0/3712330663 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77601598a0 0x7f77600ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:36.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f77577fe700 1 -- 172.21.15.92:0/3712330663 >> v2:172.21.15.124:3300/0 conn(0x7f77600cbc70 msgr2=0x7f77600ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:36.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f77577fe700 1 --2- 172.21.15.92:0/3712330663 >> v2:172.21.15.124:3300/0 conn(0x7f77600cbc70 0x7f77600ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:36.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f77577fe700 1 -- 172.21.15.92:0/3712330663 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77600cfe70 con 0x7f77601594d0 2023-10-06T10:40:36.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f77577fe700 1 -- 172.21.15.92:0/3712330663 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f775c001470 con 0x7f77601594d0 2023-10-06T10:40:36.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f77577fe700 1 -- 172.21.15.92:0/3712330663 >> v1:172.21.15.92:6789/0 conn(0x7f77601594d0 legacy=0x7f77600cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:36.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f77577fe700 1 --2- 172.21.15.92:0/3712330663 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77601598a0 0x7f7748003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:36.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f77577fe700 1 --2- 172.21.15.92:0/3712330663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7748003fe0 0x7f7748004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:36.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f77577fe700 1 --2- 172.21.15.92:0/3712330663 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7748004d50 0x7f77480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:36.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f77577fe700 1 -- 172.21.15.92:0/3712330663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77600cf8b0 con 0x7f7748003fe0 2023-10-06T10:40:36.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f77577fe700 1 -- 172.21.15.92:0/3712330663 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77600cfb90 con 0x7f7748004d50 2023-10-06T10:40:36.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f77577fe700 1 -- 172.21.15.92:0/3712330663 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f774800ba90 con 0x7f77601598a0 2023-10-06T10:40:36.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f7757fff700 1 --2- 172.21.15.92:0/3712330663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7748003fe0 0x7f7748004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:36.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f7764ab1700 1 --2- 172.21.15.92:0/3712330663 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77601598a0 0x7f7748003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:36.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f77652b2700 1 --2- 172.21.15.92:0/3712330663 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7748004d50 0x7f77480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:36.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f7757fff700 1 -- 172.21.15.92:0/3712330663 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77601598a0 msgr2=0x7f7748003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:36.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f7757fff700 1 --2- 172.21.15.92:0/3712330663 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77601598a0 0x7f7748003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:36.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f7757fff700 1 -- 172.21.15.92:0/3712330663 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7748004d50 msgr2=0x7f77480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:36.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f7757fff700 1 --2- 172.21.15.92:0/3712330663 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7748004d50 0x7f77480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:36.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f7757fff700 1 -- 172.21.15.92:0/3712330663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7748003570 con 0x7f7748003fe0 2023-10-06T10:40:36.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f7757fff700 1 --2- 172.21.15.92:0/3712330663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7748003fe0 0x7f7748004810 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7760159c90 tx=0x7f77500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c766afbf312ee6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:36.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f77577fe700 1 -- 172.21.15.92:0/3712330663 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775000e6b0 con 0x7f7748003fe0 2023-10-06T10:40:36.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f77577fe700 1 -- 172.21.15.92:0/3712330663 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775000e810 con 0x7f7748003fe0 2023-10-06T10:40:36.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.348+0000 7f77577fe700 1 -- 172.21.15.92:0/3712330663 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775000ebd0 con 0x7f7748003fe0 2023-10-06T10:40:36.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.349+0000 7f7766d15700 1 -- 172.21.15.92:0/3712330663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7748003fe0 msgr2=0x7f7748004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:36.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.349+0000 7f7766d15700 1 --2- 172.21.15.92:0/3712330663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7748003fe0 0x7f7748004810 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7760159c90 tx=0x7f77500091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:36.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.349+0000 7f77652b2700 1 -- 172.21.15.92:0/3712330663 reap_dead start 2023-10-06T10:40:36.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.349+0000 7f7766d15700 1 -- 172.21.15.92:0/3712330663 shutdown_connections 2023-10-06T10:40:36.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.349+0000 7f7766d15700 1 -- 172.21.15.92:0/3712330663 >> 172.21.15.92:0/3712330663 conn(0x7f7760089e50 msgr2=0x7f7760099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:36.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.349+0000 7f7766d15700 1 -- 172.21.15.92:0/3712330663 shutdown_connections 2023-10-06T10:40:36.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.349+0000 7f7766d15700 1 -- 172.21.15.92:0/3712330663 wait complete. 2023-10-06T10:40:36.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.350+0000 7f7766d15700 1 Processor -- start 2023-10-06T10:40:36.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.350+0000 7f7766d15700 1 -- start start 2023-10-06T10:40:36.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.351+0000 7f7766d15700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7748003fe0 0x7f7760146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:36.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.351+0000 7f7766d15700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7760159cc0 0x7f77600d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:36.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.351+0000 7f7766d15700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f776015a170 0x7f77600d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:36.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.351+0000 7f7766d15700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776016ac50 con 0x7f776015a170 2023-10-06T10:40:36.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.351+0000 7f7766d15700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77600b46f0 con 0x7f7748003fe0 2023-10-06T10:40:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.351+0000 7f7766d15700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776016a040 con 0x7f7760159cc0 2023-10-06T10:40:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.351+0000 7f77652b2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f776015a170 0x7f77600d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.351+0000 7f77652b2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f776015a170 0x7f77600d09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60596/0 (socket says 172.21.15.92:60596) 2023-10-06T10:40:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.351+0000 7f77652b2700 1 -- 172.21.15.92:0/3004238944 learned_addr learned my addr 172.21.15.92:0/3004238944 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.351+0000 7f7757fff700 1 --2- 172.21.15.92:0/3004238944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7760159cc0 0x7f77600d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.351+0000 7f7764ab1700 1 --2- 172.21.15.92:0/3004238944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7748003fe0 0x7f7760146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.351+0000 7f77652b2700 1 -- 172.21.15.92:0/3004238944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7760159cc0 msgr2=0x7f77600d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.351+0000 7f77652b2700 1 --2- 172.21.15.92:0/3004238944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7760159cc0 0x7f77600d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.351+0000 7f77652b2700 1 -- 172.21.15.92:0/3004238944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7748003fe0 msgr2=0x7f7760146d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.351+0000 7f77652b2700 1 --2- 172.21.15.92:0/3004238944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7748003fe0 0x7f7760146d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.351+0000 7f77652b2700 1 -- 172.21.15.92:0/3004238944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776016a320 con 0x7f776015a170 2023-10-06T10:40:36.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.351+0000 7f77652b2700 1 --2- 172.21.15.92:0/3004238944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f776015a170 0x7f77600d09e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f77600cc070 tx=0x7f775800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:36.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.352+0000 7f7754ff9700 1 -- 172.21.15.92:0/3004238944 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775800e040 con 0x7f776015a170 2023-10-06T10:40:36.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.352+0000 7f7754ff9700 1 -- 172.21.15.92:0/3004238944 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775800f040 con 0x7f776015a170 2023-10-06T10:40:36.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.352+0000 7f7766d15700 1 -- 172.21.15.92:0/3004238944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f776016a600 con 0x7f776015a170 2023-10-06T10:40:36.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.352+0000 7f7754ff9700 1 -- 172.21.15.92:0/3004238944 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775800fe50 con 0x7f776015a170 2023-10-06T10:40:36.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.352+0000 7f7766d15700 1 -- 172.21.15.92:0/3004238944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7760169cc0 con 0x7f776015a170 2023-10-06T10:40:36.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.352+0000 7f7766d15700 1 -- 172.21.15.92:0/3004238944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7760064a90 con 0x7f776015a170 2023-10-06T10:40:36.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.353+0000 7f7754ff9700 1 -- 172.21.15.92:0/3004238944 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7758015640 con 0x7f776015a170 2023-10-06T10:40:36.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.353+0000 7f7754ff9700 1 --2- 172.21.15.92:0/3004238944 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f774c066f30 0x7f774c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:36.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.353+0000 7f7754ff9700 1 -- 172.21.15.92:0/3004238944 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7758085110 con 0x7f776015a170 2023-10-06T10:40:36.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.353+0000 7f7764ab1700 1 --2- 172.21.15.92:0/3004238944 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f774c066f30 0x7f774c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:36.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.354+0000 7f7764ab1700 1 --2- 172.21.15.92:0/3004238944 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f774c066f30 0x7f774c0693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f775c001d20 tx=0x7f775c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:36.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.358+0000 7f7754ff9700 1 -- 172.21.15.92:0/3004238944 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f77580520e0 con 0x7f776015a170 2023-10-06T10:40:36.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:36.589+0000 7f7766d15700 1 -- 172.21.15.92:0/3004238944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f77600cf260 con 0x7f776015a170 2023-10-06T10:40:37.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.573+0000 7f7754ff9700 1 -- 172.21.15.92:0/3004238944 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7758055700 con 0x7f776015a170 2023-10-06T10:40:37.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.576+0000 7f7766d15700 1 -- 172.21.15.92:0/3004238944 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f774c066f30 msgr2=0x7f774c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:37.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.576+0000 7f7766d15700 1 --2- 172.21.15.92:0/3004238944 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f774c066f30 0x7f774c0693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f775c001d20 tx=0x7f775c009040 comp rx=0 tx=0).stop 2023-10-06T10:40:37.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.576+0000 7f7766d15700 1 -- 172.21.15.92:0/3004238944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f776015a170 msgr2=0x7f77600d09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:37.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.576+0000 7f7766d15700 1 --2- 172.21.15.92:0/3004238944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f776015a170 0x7f77600d09e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f77600cc070 tx=0x7f775800b040 comp rx=0 tx=0).stop 2023-10-06T10:40:37.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.576+0000 7f7766d15700 1 -- 172.21.15.92:0/3004238944 shutdown_connections 2023-10-06T10:40:37.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.576+0000 7f7766d15700 1 --2- 172.21.15.92:0/3004238944 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f774c066f30 0x7f774c0693e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:37.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.576+0000 7f7766d15700 1 --2- 172.21.15.92:0/3004238944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f776015a170 0x7f77600d09e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:37.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.576+0000 7f7766d15700 1 --2- 172.21.15.92:0/3004238944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7748003fe0 0x7f7760146d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:37.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.576+0000 7f7766d15700 1 --2- 172.21.15.92:0/3004238944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7760159cc0 0x7f77600d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:37.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.576+0000 7f7766d15700 1 -- 172.21.15.92:0/3004238944 >> 172.21.15.92:0/3004238944 conn(0x7f7760089e50 msgr2=0x7f7760099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:37.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.576+0000 7f7766d15700 1 -- 172.21.15.92:0/3004238944 shutdown_connections 2023-10-06T10:40:37.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.576+0000 7f7766d15700 1 -- 172.21.15.92:0/3004238944 wait complete. 2023-10-06T10:40:37.591 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T10:40:37.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.708+0000 7f1e7759e700 1 Processor -- start 2023-10-06T10:40:37.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.708+0000 7f1e7759e700 1 -- start start 2023-10-06T10:40:37.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.709+0000 7f1e7759e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1e780b2620 0x7f1e780cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:37.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.709+0000 7f1e7759e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1e780b29f0 0x7f1e780cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:37.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.709+0000 7f1e7759e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e780ceaa0 0x7f1e780cef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:37.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.709+0000 7f1e7759e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e780cc970 con 0x7f1e780b2620 2023-10-06T10:40:37.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.709+0000 7f1e7759e700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e780cf480 con 0x7f1e780b29f0 2023-10-06T10:40:37.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.709+0000 7f1e7759e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e780cbd60 con 0x7f1e780ceaa0 2023-10-06T10:40:37.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.709+0000 7f1e7659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1e780b2620 0x7f1e780cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:37.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.709+0000 7f1e7659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1e780b2620 0x7f1e780cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60608/0 (socket says 172.21.15.92:60608) 2023-10-06T10:40:37.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.709+0000 7f1e7659c700 1 -- 172.21.15.92:0/2355228717 learned_addr learned my addr 172.21.15.92:0/2355228717 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:37.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.709+0000 7f1e75d9b700 1 --2- 172.21.15.92:0/2355228717 >> v2:172.21.15.124:3300/0 conn(0x7f1e780b29f0 0x7f1e780cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:37.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.709+0000 7f1e76d9d700 1 --2- 172.21.15.92:0/2355228717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e780ceaa0 0x7f1e780cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:37.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.709+0000 7f1e7659c700 1 -- 172.21.15.92:0/2355228717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e780ceaa0 msgr2=0x7f1e780cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:37.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.709+0000 7f1e7659c700 1 --2- 172.21.15.92:0/2355228717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e780ceaa0 0x7f1e780cef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:37.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.709+0000 7f1e7659c700 1 -- 172.21.15.92:0/2355228717 >> v2:172.21.15.124:3300/0 conn(0x7f1e780b29f0 msgr2=0x7f1e780cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:37.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.709+0000 7f1e7659c700 1 --2- 172.21.15.92:0/2355228717 >> v2:172.21.15.124:3300/0 conn(0x7f1e780b29f0 0x7f1e780cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:37.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.709+0000 7f1e7659c700 1 -- 172.21.15.92:0/2355228717 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e780cc040 con 0x7f1e780b2620 2023-10-06T10:40:37.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.709+0000 7f1e75d9b700 1 --2- 172.21.15.92:0/2355228717 >> v2:172.21.15.124:3300/0 conn(0x7f1e780b29f0 0x7f1e780cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:37.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.709+0000 7f1e76d9d700 1 --2- 172.21.15.92:0/2355228717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e780ceaa0 0x7f1e780cef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:37.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.709+0000 7f1e7659c700 1 --2- 172.21.15.92:0/2355228717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e780b2620 0x7f1e780cccb0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1e780b62d0 tx=0x7f1e68009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e13173d566b8d8e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:37.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.710+0000 7f1e7559a700 1 -- 172.21.15.92:0/2355228717 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68005c10 con 0x7f1e780b2620 2023-10-06T10:40:37.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.710+0000 7f1e7559a700 1 -- 172.21.15.92:0/2355228717 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e68005d70 con 0x7f1e780b2620 2023-10-06T10:40:37.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.710+0000 7f1e7559a700 1 -- 172.21.15.92:0/2355228717 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6800b880 con 0x7f1e780b2620 2023-10-06T10:40:37.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.710+0000 7f1e7759e700 1 -- 172.21.15.92:0/2355228717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e780b2620 msgr2=0x7f1e780cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:37.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.710+0000 7f1e7759e700 1 --2- 172.21.15.92:0/2355228717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e780b2620 0x7f1e780cccb0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1e780b62d0 tx=0x7f1e68009220 comp rx=0 tx=0).stop 2023-10-06T10:40:37.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.710+0000 7f1e7759e700 1 -- 172.21.15.92:0/2355228717 shutdown_connections 2023-10-06T10:40:37.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.710+0000 7f1e7759e700 1 --2- 172.21.15.92:0/2355228717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e780b2620 0x7f1e780cccb0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:37.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.710+0000 7f1e7759e700 1 --2- 172.21.15.92:0/2355228717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e780ceaa0 0x7f1e780cef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:37.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.710+0000 7f1e7759e700 1 --2- 172.21.15.92:0/2355228717 >> v2:172.21.15.124:3300/0 conn(0x7f1e780b29f0 0x7f1e780cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:37.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.710+0000 7f1e7759e700 1 -- 172.21.15.92:0/2355228717 >> 172.21.15.92:0/2355228717 conn(0x7f1e78099520 msgr2=0x7f1e7806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:37.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.710+0000 7f1e7759e700 1 -- 172.21.15.92:0/2355228717 shutdown_connections 2023-10-06T10:40:37.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.710+0000 7f1e7759e700 1 -- 172.21.15.92:0/2355228717 wait complete. 2023-10-06T10:40:37.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.711+0000 7f1e7759e700 1 Processor -- start 2023-10-06T10:40:37.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.711+0000 7f1e7759e700 1 -- start start 2023-10-06T10:40:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.712+0000 7f1e7759e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e780b2620 0x7f1e780c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.712+0000 7f1e7759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e780b29f0 0x7f1e78154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.712+0000 7f1e7759e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1e780ceaa0 0x7f1e78153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.712+0000 7f1e7759e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e780b3710 con 0x7f1e780b29f0 2023-10-06T10:40:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.712+0000 7f1e7759e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e780c78c0 con 0x7f1e780ceaa0 2023-10-06T10:40:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.712+0000 7f1e7759e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e780c9fe0 con 0x7f1e780b2620 2023-10-06T10:40:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.712+0000 7f1e75d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e780b29f0 0x7f1e78154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.712+0000 7f1e75d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e780b29f0 0x7f1e78154c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60610/0 (socket says 172.21.15.92:60610) 2023-10-06T10:40:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.712+0000 7f1e75d9b700 1 -- 172.21.15.92:0/1930220074 learned_addr learned my addr 172.21.15.92:0/1930220074 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.712+0000 7f1e75d9b700 1 -- 172.21.15.92:0/1930220074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e780b2620 msgr2=0x7f1e780c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.712+0000 7f1e75d9b700 1 --2- 172.21.15.92:0/1930220074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e780b2620 0x7f1e780c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:37.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.712+0000 7f1e75d9b700 1 -- 172.21.15.92:0/1930220074 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1e780ceaa0 msgr2=0x7f1e78153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:37.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.712+0000 7f1e75d9b700 1 --2- 172.21.15.92:0/1930220074 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1e780ceaa0 0x7f1e78153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:37.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.712+0000 7f1e75d9b700 1 -- 172.21.15.92:0/1930220074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e68009480 con 0x7f1e780b29f0 2023-10-06T10:40:37.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.713+0000 7f1e75d9b700 1 --2- 172.21.15.92:0/1930220074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e780b29f0 0x7f1e78154c90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00e8a0 tx=0x7f1e6c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:37.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.713+0000 7f1e667fc700 1 -- 172.21.15.92:0/1930220074 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c00c430 con 0x7f1e780b29f0 2023-10-06T10:40:37.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.713+0000 7f1e667fc700 1 -- 172.21.15.92:0/1930220074 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6c00c590 con 0x7f1e780b29f0 2023-10-06T10:40:37.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.713+0000 7f1e7759e700 1 -- 172.21.15.92:0/1930220074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e780ca2c0 con 0x7f1e780b29f0 2023-10-06T10:40:37.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.713+0000 7f1e667fc700 1 -- 172.21.15.92:0/1930220074 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c00c950 con 0x7f1e780b29f0 2023-10-06T10:40:37.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.713+0000 7f1e7759e700 1 -- 172.21.15.92:0/1930220074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e780b33a0 con 0x7f1e780b29f0 2023-10-06T10:40:37.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.714+0000 7f1e7759e700 1 -- 172.21.15.92:0/1930220074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7805f230 con 0x7f1e780b29f0 2023-10-06T10:40:37.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.714+0000 7f1e667fc700 1 -- 172.21.15.92:0/1930220074 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1e6c00cd00 con 0x7f1e780b29f0 2023-10-06T10:40:37.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.714+0000 7f1e667fc700 1 --2- 172.21.15.92:0/1930220074 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1e58066f30 0x7f1e580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:37.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.714+0000 7f1e667fc700 1 -- 172.21.15.92:0/1930220074 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1e6c084540 con 0x7f1e780b29f0 2023-10-06T10:40:37.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.714+0000 7f1e7659c700 1 --2- 172.21.15.92:0/1930220074 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1e58066f30 0x7f1e580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:37.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.715+0000 7f1e7659c700 1 --2- 172.21.15.92:0/1930220074 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1e58066f30 0x7f1e580693e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1e7816ccc0 tx=0x7f1e68016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:37.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.718+0000 7f1e667fc700 1 -- 172.21.15.92:0/1930220074 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e6c054fa0 con 0x7f1e780b29f0 2023-10-06T10:40:37.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:37.942+0000 7f1e7759e700 1 -- 172.21.15.92:0/1930220074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1e780b3850 con 0x7f1e780b29f0 2023-10-06T10:40:38.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.580+0000 7f1e667fc700 1 -- 172.21.15.92:0/1930220074 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1e780b3850 con 0x7f1e780b29f0 2023-10-06T10:40:38.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.583+0000 7f1e7759e700 1 -- 172.21.15.92:0/1930220074 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1e58066f30 msgr2=0x7f1e580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:38.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.583+0000 7f1e7759e700 1 --2- 172.21.15.92:0/1930220074 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1e58066f30 0x7f1e580693e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1e7816ccc0 tx=0x7f1e68016000 comp rx=0 tx=0).stop 2023-10-06T10:40:38.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.583+0000 7f1e7759e700 1 -- 172.21.15.92:0/1930220074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e780b29f0 msgr2=0x7f1e78154c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:38.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.583+0000 7f1e7759e700 1 --2- 172.21.15.92:0/1930220074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e780b29f0 0x7f1e78154c90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00e8a0 tx=0x7f1e6c008040 comp rx=0 tx=0).stop 2023-10-06T10:40:38.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.583+0000 7f1e7759e700 1 -- 172.21.15.92:0/1930220074 shutdown_connections 2023-10-06T10:40:38.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.583+0000 7f1e7759e700 1 --2- 172.21.15.92:0/1930220074 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1e58066f30 0x7f1e580693e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:38.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.583+0000 7f1e7759e700 1 --2- 172.21.15.92:0/1930220074 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1e780ceaa0 0x7f1e78153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:38.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.583+0000 7f1e7759e700 1 --2- 172.21.15.92:0/1930220074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e780b2620 0x7f1e780c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:38.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.583+0000 7f1e7759e700 1 --2- 172.21.15.92:0/1930220074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e780b29f0 0x7f1e78154c90 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:38.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.583+0000 7f1e7759e700 1 -- 172.21.15.92:0/1930220074 >> 172.21.15.92:0/1930220074 conn(0x7f1e78099520 msgr2=0x7f1e780cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:38.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.583+0000 7f1e7759e700 1 -- 172.21.15.92:0/1930220074 shutdown_connections 2023-10-06T10:40:38.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.583+0000 7f1e7759e700 1 -- 172.21.15.92:0/1930220074 wait complete. 2023-10-06T10:40:38.599 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:40:38.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.714+0000 7fdc2746d700 1 Processor -- start 2023-10-06T10:40:38.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.714+0000 7fdc2746d700 1 -- start start 2023-10-06T10:40:38.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.715+0000 7fdc2746d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc20159560 0x7fdc200aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:38.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.715+0000 7fdc2746d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc200b0c60 con 0x7fdc200b49a0 2023-10-06T10:40:38.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.715+0000 7fdc2746d700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc200b0f40 con 0x7fdc200b0760 2023-10-06T10:40:38.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.715+0000 7fdc2746d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc200b1220 con 0x7fdc20159560 2023-10-06T10:40:38.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.715+0000 7fdc24a08700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdc200b49a0 0x7fdc200aaa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50314/0 (socket says 172.21.15.92:50314) 2023-10-06T10:40:38.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.715+0000 7fdc24a08700 1 -- 172.21.15.92:0/679278290 learned_addr learned my addr 172.21.15.92:0/679278290 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:38.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.715+0000 7fdc25209700 1 --2- 172.21.15.92:0/679278290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc20159560 0x7fdc200aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:38.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.715+0000 7fdc17fff700 1 -- 172.21.15.92:0/679278290 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224785075 0 0) 0x7fdc200b0c60 con 0x7fdc200b49a0 2023-10-06T10:40:38.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.715+0000 7fdc17fff700 1 -- 172.21.15.92:0/679278290 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc0c003570 con 0x7fdc200b49a0 2023-10-06T10:40:38.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.715+0000 7fdc17fff700 1 -- 172.21.15.92:0/679278290 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2732149396 0 0) 0x7fdc200b0f40 con 0x7fdc200b0760 2023-10-06T10:40:38.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.715+0000 7fdc17fff700 1 -- 172.21.15.92:0/679278290 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc200b0c60 con 0x7fdc200b0760 2023-10-06T10:40:38.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.715+0000 7fdc25209700 1 -- 172.21.15.92:0/679278290 >> v1:172.21.15.124:6789/0 conn(0x7fdc200b0760 legacy=0x7fdc200b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:38.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.715+0000 7fdc25209700 1 -- 172.21.15.92:0/679278290 >> v1:172.21.15.92:6789/0 conn(0x7fdc200b49a0 legacy=0x7fdc200aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:38.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.715+0000 7fdc25209700 1 -- 172.21.15.92:0/679278290 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc200b1500 con 0x7fdc20159560 2023-10-06T10:40:38.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.715+0000 7fdc25209700 1 --2- 172.21.15.92:0/679278290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc20159560 0x7fdc200aa530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdc18009ee0 tx=0x7fdc18009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41cf7a1bb176a88d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:38.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.716+0000 7fdc17fff700 1 -- 172.21.15.92:0/679278290 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1800e070 con 0x7fdc20159560 2023-10-06T10:40:38.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.716+0000 7fdc17fff700 1 -- 172.21.15.92:0/679278290 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1800bb80 con 0x7fdc20159560 2023-10-06T10:40:38.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.716+0000 7fdc17fff700 1 -- 172.21.15.92:0/679278290 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18012430 con 0x7fdc20159560 2023-10-06T10:40:38.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.716+0000 7fdc2746d700 1 -- 172.21.15.92:0/679278290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc20159560 msgr2=0x7fdc200aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:38.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.716+0000 7fdc2746d700 1 --2- 172.21.15.92:0/679278290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc20159560 0x7fdc200aa530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdc18009ee0 tx=0x7fdc18009780 comp rx=0 tx=0).stop 2023-10-06T10:40:38.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.716+0000 7fdc2746d700 1 -- 172.21.15.92:0/679278290 shutdown_connections 2023-10-06T10:40:38.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.716+0000 7fdc2746d700 1 --2- 172.21.15.92:0/679278290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc20159560 0x7fdc200aa530 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:38.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.716+0000 7fdc2746d700 1 -- 172.21.15.92:0/679278290 >> 172.21.15.92:0/679278290 conn(0x7fdc200997f0 msgr2=0x7fdc2006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:38.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.716+0000 7fdc2746d700 1 -- 172.21.15.92:0/679278290 shutdown_connections 2023-10-06T10:40:38.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.716+0000 7fdc2746d700 1 -- 172.21.15.92:0/679278290 wait complete. 2023-10-06T10:40:38.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.717+0000 7fdc2746d700 1 Processor -- start 2023-10-06T10:40:38.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.717+0000 7fdc2746d700 1 -- start start 2023-10-06T10:40:38.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.718+0000 7fdc2746d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc200b0760 0x7fdc20156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:38.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.718+0000 7fdc2746d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc200b49a0 0x7fdc200addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:38.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.718+0000 7fdc2746d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc20159560 0x7fdc200ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:38.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.718+0000 7fdc2746d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc200ab430 con 0x7fdc200b49a0 2023-10-06T10:40:38.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.718+0000 7fdc2746d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc200abc90 con 0x7fdc200b0760 2023-10-06T10:40:38.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.718+0000 7fdc2746d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2009cd50 con 0x7fdc20159560 2023-10-06T10:40:38.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.718+0000 7fdc24a08700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc200b49a0 0x7fdc200addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:38.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.718+0000 7fdc24a08700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc200b49a0 0x7fdc200addc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60612/0 (socket says 172.21.15.92:60612) 2023-10-06T10:40:38.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.718+0000 7fdc24a08700 1 -- 172.21.15.92:0/853656118 learned_addr learned my addr 172.21.15.92:0/853656118 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:38.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.719+0000 7fdc24a08700 1 -- 172.21.15.92:0/853656118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc20159560 msgr2=0x7fdc200ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:38.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.719+0000 7fdc24a08700 1 --2- 172.21.15.92:0/853656118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc20159560 0x7fdc200ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:38.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.719+0000 7fdc24a08700 1 -- 172.21.15.92:0/853656118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc200b0760 msgr2=0x7fdc20156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:38.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.719+0000 7fdc24a08700 1 --2- 172.21.15.92:0/853656118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc200b0760 0x7fdc20156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:38.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.719+0000 7fdc24a08700 1 -- 172.21.15.92:0/853656118 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc18009a20 con 0x7fdc200b49a0 2023-10-06T10:40:38.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.719+0000 7fdc24a08700 1 --2- 172.21.15.92:0/853656118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc200b49a0 0x7fdc200addc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000c790 tx=0x7fdc10007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:38.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.719+0000 7fdc157fa700 1 -- 172.21.15.92:0/853656118 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10005b30 con 0x7fdc200b49a0 2023-10-06T10:40:38.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.719+0000 7fdc157fa700 1 -- 172.21.15.92:0/853656118 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc10005c90 con 0x7fdc200b49a0 2023-10-06T10:40:38.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.719+0000 7fdc2746d700 1 -- 172.21.15.92:0/853656118 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc200ab640 con 0x7fdc200b49a0 2023-10-06T10:40:38.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.719+0000 7fdc157fa700 1 -- 172.21.15.92:0/853656118 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10007300 con 0x7fdc200b49a0 2023-10-06T10:40:38.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.719+0000 7fdc2746d700 1 -- 172.21.15.92:0/853656118 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc20156980 con 0x7fdc200b49a0 2023-10-06T10:40:38.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.720+0000 7fdc2746d700 1 -- 172.21.15.92:0/853656118 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc20064a90 con 0x7fdc200b49a0 2023-10-06T10:40:38.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.720+0000 7fdc157fa700 1 -- 172.21.15.92:0/853656118 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdc1001b400 con 0x7fdc200b49a0 2023-10-06T10:40:38.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.720+0000 7fdc157fa700 1 --2- 172.21.15.92:0/853656118 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdc04066f30 0x7fdc040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:38.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.721+0000 7fdc157fa700 1 -- 172.21.15.92:0/853656118 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fdc100842f0 con 0x7fdc200b49a0 2023-10-06T10:40:38.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.721+0000 7fdc25209700 1 --2- 172.21.15.92:0/853656118 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdc04066f30 0x7fdc040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:38.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.721+0000 7fdc25209700 1 --2- 172.21.15.92:0/853656118 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdc04066f30 0x7fdc040693e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdc200cd2c0 tx=0x7fdc18019000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:38.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.725+0000 7fdc157fa700 1 -- 172.21.15.92:0/853656118 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc10054e00 con 0x7fdc200b49a0 2023-10-06T10:40:38.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.949+0000 7fdc2746d700 1 -- 172.21.15.92:0/853656118 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc200abe00 con 0x7fdc200b49a0 2023-10-06T10:40:38.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.950+0000 7fdc157fa700 1 -- 172.21.15.92:0/853656118 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4600 (secure 0 0 0) 0x7fdc100076f0 con 0x7fdc200b49a0 2023-10-06T10:40:38.952 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:38.952 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":32,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3668157650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3668157650},{"type":"v1","addr":"172.21.15.124:6834","nonce":3668157650}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2986058955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2986058955},{"type":"v1","addr":"172.21.15.124:6835","nonce":2986058955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5733,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/831923836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":831923836},{"type":"v1","addr":"172.21.15.92:6836","nonce":831923836}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:40:34.932631+0000","modified":"2023-10-06T10:40:38.573961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5736},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5736":{"gid":5736,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2458340622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2458340622},{"type":"v1","addr":"172.21.15.92:6837","nonce":2458340622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T10:40:38.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.952+0000 7fdc2746d700 1 -- 172.21.15.92:0/853656118 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdc04066f30 msgr2=0x7fdc040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:38.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.952+0000 7fdc2746d700 1 --2- 172.21.15.92:0/853656118 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdc04066f30 0x7fdc040693e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdc200cd2c0 tx=0x7fdc18019000 comp rx=0 tx=0).stop 2023-10-06T10:40:38.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.952+0000 7fdc2746d700 1 -- 172.21.15.92:0/853656118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc200b49a0 msgr2=0x7fdc200addc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:38.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.952+0000 7fdc2746d700 1 --2- 172.21.15.92:0/853656118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc200b49a0 0x7fdc200addc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000c790 tx=0x7fdc10007e40 comp rx=0 tx=0).stop 2023-10-06T10:40:38.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.952+0000 7fdc2746d700 1 -- 172.21.15.92:0/853656118 shutdown_connections 2023-10-06T10:40:38.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.952+0000 7fdc2746d700 1 --2- 172.21.15.92:0/853656118 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdc04066f30 0x7fdc040693e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:38.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.952+0000 7fdc2746d700 1 --2- 172.21.15.92:0/853656118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc20159560 0x7fdc200ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:38.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.952+0000 7fdc2746d700 1 --2- 172.21.15.92:0/853656118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc200b0760 0x7fdc20156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:38.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.952+0000 7fdc2746d700 1 --2- 172.21.15.92:0/853656118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc200b49a0 0x7fdc200addc0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:38.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.952+0000 7fdc2746d700 1 -- 172.21.15.92:0/853656118 >> 172.21.15.92:0/853656118 conn(0x7fdc200997f0 msgr2=0x7fdc200b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:38.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.952+0000 7fdc2746d700 1 -- 172.21.15.92:0/853656118 shutdown_connections 2023-10-06T10:40:38.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:38.952+0000 7fdc2746d700 1 -- 172.21.15.92:0/853656118 wait complete. 2023-10-06T10:40:38.956 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 32 2023-10-06T10:40:38.966 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:40:39.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.078+0000 7fb2e888d700 1 Processor -- start 2023-10-06T10:40:39.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.078+0000 7fb2e888d700 1 -- start start 2023-10-06T10:40:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.079+0000 7fb2e888d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb2e40cbc30 0x7fb2e40cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.079+0000 7fb2e888d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb2e40b2100 0x7fb2e40b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.079+0000 7fb2e888d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2e40cb7a0 con 0x7fb2e4159440 2023-10-06T10:40:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.079+0000 7fb2e888d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e40b0860 con 0x7fb2e40cbc30 2023-10-06T10:40:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.079+0000 7fb2e888d700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e40cb150 con 0x7fb2e40b2100 2023-10-06T10:40:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.079+0000 7fb2e37fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb2e4159440 0x7fb2e40b3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50322/0 (socket says 172.21.15.92:50322) 2023-10-06T10:40:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.079+0000 7fb2e37fe700 1 -- 172.21.15.92:0/2893959786 learned_addr learned my addr 172.21.15.92:0/2893959786 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.079+0000 7fb2e2ffd700 1 --2- 172.21.15.92:0/2893959786 >> v2:172.21.15.92:3300/0 conn(0x7fb2e40cbc30 0x7fb2e40cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.079+0000 7fb2e3fff700 1 --2- 172.21.15.92:0/2893959786 >> v2:172.21.15.124:3300/0 conn(0x7fb2e40b2100 0x7fb2e40b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.079+0000 7fb2e2ffd700 1 -- 172.21.15.92:0/2893959786 >> v2:172.21.15.124:3300/0 conn(0x7fb2e40b2100 msgr2=0x7fb2e40b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.079+0000 7fb2e2ffd700 1 --2- 172.21.15.92:0/2893959786 >> v2:172.21.15.124:3300/0 conn(0x7fb2e40b2100 0x7fb2e40b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.079+0000 7fb2e2ffd700 1 -- 172.21.15.92:0/2893959786 >> v1:172.21.15.92:6789/0 conn(0x7fb2e4159440 legacy=0x7fb2e40b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.079+0000 7fb2e27fc700 1 -- 172.21.15.92:0/2893959786 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2785072876 0 0) 0x7fb2e40cb7a0 con 0x7fb2e4159440 2023-10-06T10:40:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.079+0000 7fb2e2ffd700 1 -- 172.21.15.92:0/2893959786 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e40cb430 con 0x7fb2e40cbc30 2023-10-06T10:40:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.079+0000 7fb2e3fff700 1 --2- 172.21.15.92:0/2893959786 >> v2:172.21.15.124:3300/0 conn(0x7fb2e40b2100 0x7fb2e40b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.080+0000 7fb2e2ffd700 1 --2- 172.21.15.92:0/2893959786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2e40cbc30 0x7fb2e40cc080 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb2e40ca7f0 tx=0x7fb2d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b77d995e4fd573f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.080+0000 7fb2e27fc700 1 -- 172.21.15.92:0/2893959786 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d800b6b0 con 0x7fb2e40cbc30 2023-10-06T10:40:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.080+0000 7fb2e27fc700 1 -- 172.21.15.92:0/2893959786 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d800b810 con 0x7fb2e40cbc30 2023-10-06T10:40:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.080+0000 7fb2e27fc700 1 -- 172.21.15.92:0/2893959786 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d800bbb0 con 0x7fb2e40cbc30 2023-10-06T10:40:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.080+0000 7fb2e888d700 1 -- 172.21.15.92:0/2893959786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2e40cbc30 msgr2=0x7fb2e40cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.080+0000 7fb2e888d700 1 --2- 172.21.15.92:0/2893959786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2e40cbc30 0x7fb2e40cc080 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb2e40ca7f0 tx=0x7fb2d80091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.080+0000 7fb2e888d700 1 -- 172.21.15.92:0/2893959786 shutdown_connections 2023-10-06T10:40:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.080+0000 7fb2e888d700 1 --2- 172.21.15.92:0/2893959786 >> v2:172.21.15.124:3300/0 conn(0x7fb2e40b2100 0x7fb2e40b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.080+0000 7fb2e888d700 1 --2- 172.21.15.92:0/2893959786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2e40cbc30 0x7fb2e40cc080 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.080+0000 7fb2e888d700 1 -- 172.21.15.92:0/2893959786 >> 172.21.15.92:0/2893959786 conn(0x7fb2e406ece0 msgr2=0x7fb2e406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.080+0000 7fb2e888d700 1 -- 172.21.15.92:0/2893959786 shutdown_connections 2023-10-06T10:40:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.080+0000 7fb2e888d700 1 -- 172.21.15.92:0/2893959786 wait complete. 2023-10-06T10:40:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.082+0000 7fb2e888d700 1 Processor -- start 2023-10-06T10:40:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.082+0000 7fb2e888d700 1 -- start start 2023-10-06T10:40:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.082+0000 7fb2e888d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2e40b2100 0x7fb2e40c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.082+0000 7fb2e888d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2e40cbc30 0x7fb2e414a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.082+0000 7fb2e888d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2e4159440 0x7fb2e415bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:39.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.082+0000 7fb2e888d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e4159a90 con 0x7fb2e40cbc30 2023-10-06T10:40:39.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.082+0000 7fb2e888d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e415a330 con 0x7fb2e4159440 2023-10-06T10:40:39.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.082+0000 7fb2e888d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40c8440 con 0x7fb2e40b2100 2023-10-06T10:40:39.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.082+0000 7fb2e2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2e40cbc30 0x7fb2e414a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:39.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.082+0000 7fb2e2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2e40cbc30 0x7fb2e414a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60638/0 (socket says 172.21.15.92:60638) 2023-10-06T10:40:39.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.082+0000 7fb2e2ffd700 1 -- 172.21.15.92:0/924092836 learned_addr learned my addr 172.21.15.92:0/924092836 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:39.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.083+0000 7fb2e3fff700 1 --2- 172.21.15.92:0/924092836 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2e4159440 0x7fb2e415bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:39.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.083+0000 7fb2e37fe700 1 --2- 172.21.15.92:0/924092836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2e40b2100 0x7fb2e40c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:39.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.083+0000 7fb2e2ffd700 1 -- 172.21.15.92:0/924092836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2e40b2100 msgr2=0x7fb2e40c7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:39.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.083+0000 7fb2e2ffd700 1 --2- 172.21.15.92:0/924092836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2e40b2100 0x7fb2e40c7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.083+0000 7fb2e2ffd700 1 -- 172.21.15.92:0/924092836 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2e4159440 msgr2=0x7fb2e415bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:39.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.083+0000 7fb2e2ffd700 1 --2- 172.21.15.92:0/924092836 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2e4159440 0x7fb2e415bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.083+0000 7fb2e2ffd700 1 -- 172.21.15.92:0/924092836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d8009420 con 0x7fb2e40cbc30 2023-10-06T10:40:39.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.083+0000 7fb2e2ffd700 1 --2- 172.21.15.92:0/924092836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2e40cbc30 0x7fb2e414a680 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800b0e0 tx=0x7fb2d8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:39.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.083+0000 7fb2cb7fe700 1 -- 172.21.15.92:0/924092836 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8011420 con 0x7fb2e40cbc30 2023-10-06T10:40:39.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.083+0000 7fb2cb7fe700 1 -- 172.21.15.92:0/924092836 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d80115a0 con 0x7fb2e40cbc30 2023-10-06T10:40:39.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.083+0000 7fb2e888d700 1 -- 172.21.15.92:0/924092836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e415c720 con 0x7fb2e40cbc30 2023-10-06T10:40:39.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.083+0000 7fb2cb7fe700 1 -- 172.21.15.92:0/924092836 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8011960 con 0x7fb2e40cbc30 2023-10-06T10:40:39.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.083+0000 7fb2e888d700 1 -- 172.21.15.92:0/924092836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e4159fc0 con 0x7fb2e40cbc30 2023-10-06T10:40:39.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.084+0000 7fb2e888d700 1 -- 172.21.15.92:0/924092836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e405f230 con 0x7fb2e40cbc30 2023-10-06T10:40:39.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.084+0000 7fb2cb7fe700 1 -- 172.21.15.92:0/924092836 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb2d8011d50 con 0x7fb2e40cbc30 2023-10-06T10:40:39.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.085+0000 7fb2cb7fe700 1 --2- 172.21.15.92:0/924092836 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb2d0066f30 0x7fb2d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:39.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.085+0000 7fb2e37fe700 1 --2- 172.21.15.92:0/924092836 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb2d0066f30 0x7fb2d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:39.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.085+0000 7fb2cb7fe700 1 -- 172.21.15.92:0/924092836 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb2d80862f0 con 0x7fb2e40cbc30 2023-10-06T10:40:39.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.085+0000 7fb2e37fe700 1 --2- 172.21.15.92:0/924092836 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb2d0066f30 0x7fb2d00693e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4002230 tx=0x7fb2d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:39.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.089+0000 7fb2cb7fe700 1 -- 172.21.15.92:0/924092836 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2d80532c0 con 0x7fb2e40cbc30 2023-10-06T10:40:39.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.272+0000 7fb2e888d700 1 -- 172.21.15.92:0/924092836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2e4159810 con 0x7fb2e40cbc30 2023-10-06T10:40:39.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.273+0000 7fb2cb7fe700 1 -- 172.21.15.92:0/924092836 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+17071 (secure 0 0 0) 0x7fb2d80568e0 con 0x7fb2e40cbc30 2023-10-06T10:40:39.275 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:39.275 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":52,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:40:36.557340+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:40:32.570535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-06T10:40:33.886794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:40:39.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.275+0000 7fb2e888d700 1 -- 172.21.15.92:0/924092836 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb2d0066f30 msgr2=0x7fb2d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:39.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.275+0000 7fb2e888d700 1 --2- 172.21.15.92:0/924092836 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb2d0066f30 0x7fb2d00693e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4002230 tx=0x7fb2d4009040 comp rx=0 tx=0).stop 2023-10-06T10:40:39.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.275+0000 7fb2e888d700 1 -- 172.21.15.92:0/924092836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2e40cbc30 msgr2=0x7fb2e414a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:39.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.275+0000 7fb2e888d700 1 --2- 172.21.15.92:0/924092836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2e40cbc30 0x7fb2e414a680 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800b0e0 tx=0x7fb2d8017000 comp rx=0 tx=0).stop 2023-10-06T10:40:39.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.275+0000 7fb2e888d700 1 -- 172.21.15.92:0/924092836 shutdown_connections 2023-10-06T10:40:39.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.275+0000 7fb2e888d700 1 --2- 172.21.15.92:0/924092836 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb2d0066f30 0x7fb2d00693e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.275+0000 7fb2e888d700 1 --2- 172.21.15.92:0/924092836 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2e4159440 0x7fb2e415bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.275+0000 7fb2e888d700 1 --2- 172.21.15.92:0/924092836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2e40b2100 0x7fb2e40c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.275+0000 7fb2e888d700 1 --2- 172.21.15.92:0/924092836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2e40cbc30 0x7fb2e414a680 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.275+0000 7fb2e888d700 1 -- 172.21.15.92:0/924092836 >> 172.21.15.92:0/924092836 conn(0x7fb2e406ece0 msgr2=0x7fb2e4155270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:39.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.275+0000 7fb2e888d700 1 -- 172.21.15.92:0/924092836 shutdown_connections 2023-10-06T10:40:39.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.275+0000 7fb2e888d700 1 -- 172.21.15.92:0/924092836 wait complete. 2023-10-06T10:40:39.289 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T10:40:39.290 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:40:39.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.403+0000 7f115f59e700 1 Processor -- start 2023-10-06T10:40:39.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.403+0000 7f115f59e700 1 -- start start 2023-10-06T10:40:39.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.404+0000 7f115f59e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1160159440 0x7f11600b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:39.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.404+0000 7f115f59e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11600b35d0 0x7f11600cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:39.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.404+0000 7f115f59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11600cc170 0x7f11600b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:39.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.404+0000 7f115f59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11600cb7a0 con 0x7f11600cc170 2023-10-06T10:40:39.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.404+0000 7f115f59e700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11600b0860 con 0x7f1160159440 2023-10-06T10:40:39.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.404+0000 7f115f59e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11600cb150 con 0x7f11600b35d0 2023-10-06T10:40:39.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.404+0000 7f115ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11600cc170 0x7f11600b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:39.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.404+0000 7f115ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11600cc170 0x7f11600b2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60642/0 (socket says 172.21.15.92:60642) 2023-10-06T10:40:39.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.404+0000 7f115ed9d700 1 -- 172.21.15.92:0/1868956896 learned_addr learned my addr 172.21.15.92:0/1868956896 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:39.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.404+0000 7f115dd9b700 1 --2- 172.21.15.92:0/1868956896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11600b35d0 0x7f11600cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:39.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.404+0000 7f115e59c700 1 --2- 172.21.15.92:0/1868956896 >> v2:172.21.15.124:3300/0 conn(0x7f1160159440 0x7f11600b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:39.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.404+0000 7f115ed9d700 1 -- 172.21.15.92:0/1868956896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11600b35d0 msgr2=0x7f11600cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:39.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.404+0000 7f115ed9d700 1 --2- 172.21.15.92:0/1868956896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11600b35d0 0x7f11600cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.404+0000 7f115ed9d700 1 -- 172.21.15.92:0/1868956896 >> v2:172.21.15.124:3300/0 conn(0x7f1160159440 msgr2=0x7f11600b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:39.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.404+0000 7f115ed9d700 1 --2- 172.21.15.92:0/1868956896 >> v2:172.21.15.124:3300/0 conn(0x7f1160159440 0x7f11600b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.404+0000 7f115ed9d700 1 -- 172.21.15.92:0/1868956896 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11600cb430 con 0x7f11600cc170 2023-10-06T10:40:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.405+0000 7f115ed9d700 1 --2- 172.21.15.92:0/1868956896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11600cc170 0x7f11600b2100 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f116006f9f0 tx=0x7f1150009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ac37d1e25945b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.405+0000 7f115d59a700 1 -- 172.21.15.92:0/1868956896 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115000e070 con 0x7f11600cc170 2023-10-06T10:40:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.405+0000 7f115d59a700 1 -- 172.21.15.92:0/1868956896 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115000b970 con 0x7f11600cc170 2023-10-06T10:40:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.405+0000 7f115d59a700 1 -- 172.21.15.92:0/1868956896 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11500092e0 con 0x7f11600cc170 2023-10-06T10:40:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.405+0000 7f115f59e700 1 -- 172.21.15.92:0/1868956896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11600cc170 msgr2=0x7f11600b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.405+0000 7f115f59e700 1 --2- 172.21.15.92:0/1868956896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11600cc170 0x7f11600b2100 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f116006f9f0 tx=0x7f1150009840 comp rx=0 tx=0).stop 2023-10-06T10:40:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.405+0000 7f115f59e700 1 -- 172.21.15.92:0/1868956896 shutdown_connections 2023-10-06T10:40:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.405+0000 7f115f59e700 1 --2- 172.21.15.92:0/1868956896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11600cc170 0x7f11600b2100 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.405+0000 7f115f59e700 1 --2- 172.21.15.92:0/1868956896 >> v2:172.21.15.124:3300/0 conn(0x7f1160159440 0x7f11600b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.405+0000 7f115f59e700 1 --2- 172.21.15.92:0/1868956896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11600b35d0 0x7f11600cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.405+0000 7f115f59e700 1 -- 172.21.15.92:0/1868956896 >> 172.21.15.92:0/1868956896 conn(0x7f116006ece0 msgr2=0x7f116006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:39.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.405+0000 7f115f59e700 1 -- 172.21.15.92:0/1868956896 shutdown_connections 2023-10-06T10:40:39.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.405+0000 7f115f59e700 1 -- 172.21.15.92:0/1868956896 wait complete. 2023-10-06T10:40:39.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.406+0000 7f115f59e700 1 Processor -- start 2023-10-06T10:40:39.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.406+0000 7f115f59e700 1 -- start start 2023-10-06T10:40:39.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.407+0000 7f115f59e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11600b35d0 0x7f11600ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:39.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.407+0000 7f115f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11600cc170 0x7f11600c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:39.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.407+0000 7f115f59e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1160159440 0x7f11600c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:39.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.407+0000 7f115f59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11600cf7a0 con 0x7f11600cc170 2023-10-06T10:40:39.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.407+0000 7f115f59e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1160159fc0 con 0x7f1160159440 2023-10-06T10:40:39.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.407+0000 7f115f59e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11600cf1e0 con 0x7f11600b35d0 2023-10-06T10:40:39.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.407+0000 7f115dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11600cc170 0x7f11600c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:39.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.407+0000 7f115dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11600cc170 0x7f11600c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60656/0 (socket says 172.21.15.92:60656) 2023-10-06T10:40:39.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.407+0000 7f115dd9b700 1 -- 172.21.15.92:0/2703790711 learned_addr learned my addr 172.21.15.92:0/2703790711 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:39.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.407+0000 7f115e59c700 1 --2- 172.21.15.92:0/2703790711 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11600b35d0 0x7f11600ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:39.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.407+0000 7f115ed9d700 1 --2- 172.21.15.92:0/2703790711 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1160159440 0x7f11600c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:39.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.408+0000 7f115dd9b700 1 -- 172.21.15.92:0/2703790711 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11600b35d0 msgr2=0x7f11600ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:39.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.408+0000 7f115dd9b700 1 --2- 172.21.15.92:0/2703790711 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11600b35d0 0x7f11600ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.408+0000 7f115dd9b700 1 -- 172.21.15.92:0/2703790711 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1160159440 msgr2=0x7f11600c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:39.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.408+0000 7f115dd9b700 1 --2- 172.21.15.92:0/2703790711 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1160159440 0x7f11600c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.408+0000 7f115dd9b700 1 -- 172.21.15.92:0/2703790711 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1150009ae0 con 0x7f11600cc170 2023-10-06T10:40:39.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.408+0000 7f115dd9b700 1 --2- 172.21.15.92:0/2703790711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11600cc170 0x7f11600c8580 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f114800ec80 tx=0x7f11480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:39.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.408+0000 7f11567fc700 1 -- 172.21.15.92:0/2703790711 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114800c850 con 0x7f11600cc170 2023-10-06T10:40:39.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.408+0000 7f11567fc700 1 -- 172.21.15.92:0/2703790711 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f114800c9b0 con 0x7f11600cc170 2023-10-06T10:40:39.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.408+0000 7f11567fc700 1 -- 172.21.15.92:0/2703790711 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114800cd50 con 0x7f11600cc170 2023-10-06T10:40:39.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.408+0000 7f115f59e700 1 -- 172.21.15.92:0/2703790711 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11600cf4c0 con 0x7f11600cc170 2023-10-06T10:40:39.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.408+0000 7f115f59e700 1 -- 172.21.15.92:0/2703790711 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11600d0a20 con 0x7f11600cc170 2023-10-06T10:40:39.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.409+0000 7f115f59e700 1 -- 172.21.15.92:0/2703790711 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1160159ab0 con 0x7f11600cc170 2023-10-06T10:40:39.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.409+0000 7f11567fc700 1 -- 172.21.15.92:0/2703790711 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1148013680 con 0x7f11600cc170 2023-10-06T10:40:39.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.409+0000 7f11567fc700 1 --2- 172.21.15.92:0/2703790711 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1140066f30 0x7f11400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:39.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.410+0000 7f11567fc700 1 -- 172.21.15.92:0/2703790711 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f11480849f0 con 0x7f11600cc170 2023-10-06T10:40:39.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.410+0000 7f115e59c700 1 --2- 172.21.15.92:0/2703790711 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1140066f30 0x7f11400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:39.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.410+0000 7f115e59c700 1 --2- 172.21.15.92:0/2703790711 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1140066f30 0x7f11400693e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1158009d10 tx=0x7f11580091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:39.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.414+0000 7f11567fc700 1 -- 172.21.15.92:0/2703790711 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1160159ab0 con 0x7f11600cc170 2023-10-06T10:40:39.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.595+0000 7f115f59e700 1 -- 172.21.15.92:0/2703790711 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1160159ab0 con 0x7f1140066f30 2023-10-06T10:40:39.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.603+0000 7f11567fc700 1 -- 172.21.15.92:0/2703790711 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300820 (secure 0 0 0) 0x7f1160159ab0 con 0x7f1140066f30 2023-10-06T10:40:39.608 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:39.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.606+0000 7f115f59e700 1 -- 172.21.15.92:0/2703790711 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1140066f30 msgr2=0x7f11400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:39.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.606+0000 7f115f59e700 1 --2- 172.21.15.92:0/2703790711 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1140066f30 0x7f11400693e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1158009d10 tx=0x7f11580091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.606+0000 7f115f59e700 1 -- 172.21.15.92:0/2703790711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11600cc170 msgr2=0x7f11600c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:39.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.606+0000 7f115f59e700 1 --2- 172.21.15.92:0/2703790711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11600cc170 0x7f11600c8580 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f114800ec80 tx=0x7f11480091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.606+0000 7f115f59e700 1 -- 172.21.15.92:0/2703790711 shutdown_connections 2023-10-06T10:40:39.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.606+0000 7f115f59e700 1 --2- 172.21.15.92:0/2703790711 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1140066f30 0x7f11400693e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.606+0000 7f115f59e700 1 --2- 172.21.15.92:0/2703790711 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1160159440 0x7f11600c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.606+0000 7f115f59e700 1 --2- 172.21.15.92:0/2703790711 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11600b35d0 0x7f11600ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.606+0000 7f115f59e700 1 --2- 172.21.15.92:0/2703790711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11600cc170 0x7f11600c8580 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.606+0000 7f115f59e700 1 -- 172.21.15.92:0/2703790711 >> 172.21.15.92:0/2703790711 conn(0x7f116006ece0 msgr2=0x7f1160154fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:39.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.606+0000 7f115f59e700 1 -- 172.21.15.92:0/2703790711 shutdown_connections 2023-10-06T10:40:39.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.606+0000 7f115f59e700 1 -- 172.21.15.92:0/2703790711 wait complete. 2023-10-06T10:40:39.614 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:40:39.620 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2023-10-06T10:40:38.112480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2384004,"kb_used_data":5700,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752590716,"statfs":{"total":773094113280,"available":770652893184,"internally_reserved":0,"allocated":5836800,"data_stored":3307687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.575902"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.539087+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:03.396798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.541292+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:38.780441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.538829+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:56.154679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.540171+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:39.967833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.539827+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:03.396798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.538853+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:13.047456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.542046+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:58:09.360521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.539879+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:43:52.519978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.543819+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:39.937778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.540329+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:03.396798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.544327+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:27:47.178791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.540337+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:14.711366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.540790+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:05.264641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.540853+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:28:50.958890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.541599+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:13:10.093056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.543711+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:28:50.958890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.543105+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:06:19.788485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.538379+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:58:09.360521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.540876+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:39.937778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.539641+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:21.420689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.542579+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:17:35.395865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.541053+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:05.264641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.542547+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:35.191932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.540467+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:43:52.519978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.538361+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:13:10.093056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":40,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T10:40:30.525308+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:40:30.525308+0000","last_peered":"2023-10-06T10:40:30.525308+0000","last_clean":"2023-10-06T10:40:30.525308+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:40:30.525308+0000","last_undegraded":"2023-10-06T10:40:30.525308+0000","last_fullsized":"2023-10-06T10:40:30.525308+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.541264+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:41.469721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T10:40:32.841471+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:40:32.841471+0000","last_peered":"2023-10-06T10:40:32.841471+0000","last_clean":"2023-10-06T10:40:32.841471+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:40:32.841471+0000","last_undegraded":"2023-10-06T10:40:32.841471+0000","last_fullsized":"2023-10-06T10:40:32.841471+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.541812+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:37:29.893321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":38,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T10:40:30.524317+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:40:30.524317+0000","last_peered":"2023-10-06T10:40:30.524317+0000","last_clean":"2023-10-06T10:40:30.524317+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:40:30.524317+0000","last_undegraded":"2023-10-06T10:40:30.524317+0000","last_fullsized":"2023-10-06T10:40:30.524317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":183,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T10:40:30.642375+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:40:30.642375+0000","last_peered":"2023-10-06T10:40:30.642375+0000","last_clean":"2023-10-06T10:40:30.642375+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:40:30.642375+0000","last_undegraded":"2023-10-06T10:40:30.642375+0000","last_fullsized":"2023-10-06T10:40:30.642375+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.540320+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:13.047456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":33,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T10:40:30.642376+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:40:30.642376+0000","last_peered":"2023-10-06T10:40:30.642376+0000","last_clean":"2023-10-06T10:40:30.642376+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:40:30.642376+0000","last_undegraded":"2023-10-06T10:40:30.642376+0000","last_fullsized":"2023-10-06T10:40:30.642376+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.540741+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:35:55.915864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":33,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T10:40:30.642361+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:40:30.642361+0000","last_peered":"2023-10-06T10:40:30.642361+0000","last_clean":"2023-10-06T10:40:30.642361+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:40:30.642361+0000","last_undegraded":"2023-10-06T10:40:30.642361+0000","last_fullsized":"2023-10-06T10:40:30.642361+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.540755+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:37:38.300612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T10:40:30.525296+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:40:30.525296+0000","last_peered":"2023-10-06T10:40:30.525296+0000","last_clean":"2023-10-06T10:40:30.525296+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:40:30.525296+0000","last_undegraded":"2023-10-06T10:40:30.525296+0000","last_fullsized":"2023-10-06T10:40:30.525296+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.540472+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:14.711366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T10:40:31.848862+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:40:31.848862+0000","last_peered":"2023-10-06T10:40:31.848862+0000","last_clean":"2023-10-06T10:40:31.848862+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:40:31.848862+0000","last_undegraded":"2023-10-06T10:40:31.848862+0000","last_fullsized":"2023-10-06T10:40:31.848862+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.538821+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:43:17.508745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":36,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T10:40:31.848848+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:40:31.848848+0000","last_peered":"2023-10-06T10:40:31.848848+0000","last_clean":"2023-10-06T10:40:31.848848+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:40:31.848848+0000","last_undegraded":"2023-10-06T10:40:31.848848+0000","last_fullsized":"2023-10-06T10:40:31.848848+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298436,"kb_used_data":1148,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073404,"statfs":{"total":96636764160,"available":96331165696,"internally_reserved":0,"allocated":1175552,"data_stored":856406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.48099999999999998,"15min":0.47799999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71699999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.40500000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78600000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60599999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.46600000000000003,"15min":0.45700000000000002},"min":{"1min":0.375,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65900000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.505,"15min":0.5},"min":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72399999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.752}]},{"osd":2,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.40600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.627},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.496,"15min":0.49199999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69199999999999995,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.70099999999999996}]},{"osd":3,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54800000000000004,"15min":0.54200000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.73799999999999999}]},{"osd":4,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.46800000000000003,"15min":0.47699999999999998},"min":{"1min":0.30499999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57699999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.501,"15min":0.50900000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.32700000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65900000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.34300000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66800000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.65700000000000003}]},{"osd":6,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.56799999999999995,"15min":0.57699999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.75700000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.747,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.76200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297852,"kb_used_data":564,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073988,"statfs":{"total":96636764160,"available":96331763712,"internally_reserved":0,"allocated":577536,"data_stored":264685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.49399999999999999,"15min":0.48899999999999999},"min":{"1min":0.36299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.435},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.44500000000000001,"15min":0.42899999999999999},"min":{"1min":0.38800000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.57299999999999995}]},{"osd":1,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.45100000000000001,"15min":0.437},"min":{"1min":0.33100000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.53100000000000003}]},{"osd":2,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.379,"5min":0.309,"15min":0.309},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.47699999999999998,"15min":0.46400000000000002},"min":{"1min":0.34999999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.45700000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.497,"15min":0.48799999999999999},"min":{"1min":0.38800000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.502},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.48799999999999999}]},{"osd":4,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47899999999999998,"15min":0.47799999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70899999999999996,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.40899999999999997}]},{"osd":5,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.48999999999999999,"15min":0.48599999999999999},"min":{"1min":0.28899999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69299999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33900000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.754,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.63}]}]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298436,"kb_used_data":1148,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073404,"statfs":{"total":96636764160,"available":96331165696,"internally_reserved":0,"allocated":1175552,"data_stored":856406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33000000000000002,"15min":0.32800000000000001},"min":{"1min":0.253,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.34999999999999998,"15min":0.34599999999999997},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.53900000000000003,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.58799999999999997}]},{"osd":2,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.32500000000000001,"15min":0.31900000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.36299999999999999},"min":{"1min":0.28999999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.60799999999999998}]},{"osd":3,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.375},"min":{"1min":0.30099999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.379,"15min":0.375},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.40300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.69499999999999995,"5min":0.746,"15min":0.746},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.80000000000000004}]},{"osd":5,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54800000000000004,"15min":0.54100000000000004},"min":{"1min":0.436,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.82899999999999996}]},{"osd":6,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.40100000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.47899999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.93500000000000005}]},{"osd":7,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58099999999999996,"15min":0.57399999999999995},"min":{"1min":0.48699999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.996},{"interface":"front","average":{"1min":0.622,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.439,"5min":0.436,"15min":0.436},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.88500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297856,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073984,"statfs":{"total":96636764160,"available":96331759616,"internally_reserved":0,"allocated":581632,"data_stored":266038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.32600000000000001,"15min":0.31900000000000001},"min":{"1min":0.23200000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.34899999999999998,"15min":0.34100000000000003},"min":{"1min":0.27600000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.35199999999999998}]},{"osd":2,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.312,"15min":0.30599999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.33300000000000002,"15min":0.32500000000000001},"min":{"1min":0.26500000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.52000000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.318}]},{"osd":3,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.35099999999999998,"15min":0.34300000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.35499999999999998,"15min":0.34899999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.34000000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.42599999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81200000000000006,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.54200000000000004,"15min":0.53300000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.501}]},{"osd":5,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.41799999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.91900000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.80900000000000005,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.51200000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58699999999999997,"15min":0.58799999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77300000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86399999999999999,"5min":1.075,"15min":1.075},"last":0.52800000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79700000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.441},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.61399999999999999,"15min":0.60999999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88800000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.54700000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297856,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073984,"statfs":{"total":96636764160,"available":96331759616,"internally_reserved":0,"allocated":581632,"data_stored":266038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.313,"15min":0.311},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.32500000000000001,"15min":0.317},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.29199999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.32900000000000001,"15min":0.32300000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.54200000000000004,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.247},{"interface":"front","average":{"1min":0.38,"5min":0.36699999999999999,"15min":0.36499999999999999},"min":{"1min":0.26400000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59299999999999997,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.26500000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.34899999999999998,"15min":0.34499999999999997},"min":{"1min":0.252,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.52700000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.35799999999999998,"15min":0.35499999999999998},"min":{"1min":0.27600000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.33200000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.35499999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.76100000000000001,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82699999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54100000000000004}]},{"osd":5,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.377,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81599999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.60699999999999998}]},{"osd":6,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.625,"15min":0.624},"min":{"1min":0.45800000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89000000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.624,"15min":0.623},"min":{"1min":0.433,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68000000000000005}]},{"osd":7,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.44500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79100000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.498,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.65800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297856,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073984,"statfs":{"total":96636764160,"available":96331759616,"internally_reserved":0,"allocated":581632,"data_stored":266038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.438,"15min":0.45600000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.50600000000000001,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.31},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.36799999999999999,"15min":0.36399999999999999},"min":{"1min":0.27100000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66800000000000004,"5min":1.218,"15min":1.218},"last":0.374}]},{"osd":1,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34100000000000003,"15min":0.34000000000000002},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.61899999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.48299999999999998,"15min":0.495},"min":{"1min":0.27800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60299999999999998,"5min":3.359,"15min":3.359},"last":0.39700000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38500000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.57699999999999996,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.63700000000000001,"15min":0.67300000000000004},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57699999999999996,"5min":7.069,"15min":7.069},"last":0.38400000000000001}]},{"osd":4,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89400000000000002,"5min":1.165,"15min":1.165},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59899999999999998,"15min":0.59699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79400000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.50800000000000001}]},{"osd":5,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.69799999999999995,"15min":0.71399999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92500000000000004,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60799999999999998,"15min":0.61099999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.96899999999999997,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.54300000000000004}]},{"osd":6,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.38,"5min":0.378,"15min":0.378},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.72699999999999998,"15min":0.74199999999999999},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84699999999999998,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.63100000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.61099999999999999},"min":{"1min":0.41299999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.99099999999999999,"5min":1.137,"15min":1.137},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.443,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91400000000000003,"5min":1.014,"15min":1.014},"last":0.55500000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297856,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073984,"statfs":{"total":96636764160,"available":96331759616,"internally_reserved":0,"allocated":581632,"data_stored":266038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47999999999999998,"15min":0.48099999999999998},"min":{"1min":0.33200000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.60799999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.496},"min":{"1min":0.33100000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.67000000000000004,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.52200000000000002}]},{"osd":1,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.495,"15min":0.496},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.59399999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.60299999999999998}]},{"osd":2,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50700000000000001,"15min":0.50800000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.63,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36099999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.68500000000000005,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.66900000000000004}]},{"osd":3,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51600000000000001,"15min":0.51200000000000001},"min":{"1min":0.42199999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.625,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.502,"5min":0.53000000000000003,"15min":0.53500000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.64800000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.57899999999999996}]},{"osd":5,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.495,"15min":0.50800000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.61599999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.377},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.49399999999999999,"15min":0.502},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.63300000000000001,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.47099999999999997}]},{"osd":6,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.53600000000000003,"15min":0.54600000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.68799999999999994,"5min":0.75,"15min":0.75},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.67700000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.49099999999999999}]},{"osd":7,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.56899999999999995,"15min":0.57999999999999996},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70499999999999996,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72499999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.44400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297856,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073984,"statfs":{"total":96636764160,"available":96331759616,"internally_reserved":0,"allocated":581632,"data_stored":266038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.47399999999999998,"15min":0.46300000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54000000000000004,"15min":0.53400000000000003},"min":{"1min":0.35899999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.748,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.57899999999999996}]},{"osd":1,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.496,"15min":0.48499999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.52500000000000002,"15min":0.51700000000000002},"min":{"1min":0.39200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.69299999999999995}]},{"osd":2,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.51000000000000001,"15min":0.499},"min":{"1min":0.32000000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.41499999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.754,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.66800000000000004}]},{"osd":3,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.57799999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77700000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78500000000000003,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.496}]},{"osd":4,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.32100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.69599999999999995,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.439},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.66100000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.51500000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.67500000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.746}]},{"osd":7,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68799999999999994,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76700000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.59099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:40:39.627 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:40:39.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.773+0000 7f6512e71700 1 Processor -- start 2023-10-06T10:40:39.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.773+0000 7f6512e71700 1 -- start start 2023-10-06T10:40:39.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.774+0000 7f6512e71700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f650c159560 0x7f650c0aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:39.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.774+0000 7f6512e71700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f650c0b0760 0x7f650c0b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:39.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.774+0000 7f6512e71700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f650c0b0c60 con 0x7f650c0b49a0 2023-10-06T10:40:39.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.774+0000 7f6512e71700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f650c0b0f40 con 0x7f650c0b0760 2023-10-06T10:40:39.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.774+0000 7f6512e71700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650c0b1220 con 0x7f650c159560 2023-10-06T10:40:39.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.774+0000 7f651140e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f650c0b0760 0x7f650c0b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:39.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.774+0000 7f651140e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f650c0b0760 0x7f650c0b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60670/0 (socket says 172.21.15.92:60670) 2023-10-06T10:40:39.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.774+0000 7f651140e700 1 -- 172.21.15.92:0/420636410 learned_addr learned my addr 172.21.15.92:0/420636410 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:39.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.774+0000 7f6510c0d700 1 --2- 172.21.15.92:0/420636410 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f650c159560 0x7f650c0aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:39.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.774+0000 7f651140e700 1 -- 172.21.15.92:0/420636410 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f650c159560 msgr2=0x7f650c0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:39.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.774+0000 7f651140e700 1 --2- 172.21.15.92:0/420636410 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f650c159560 0x7f650c0aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.774+0000 7f651140e700 1 -- 172.21.15.92:0/420636410 >> v1:172.21.15.124:6789/0 conn(0x7f650c0b49a0 legacy=0x7f650c0aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:39.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.774+0000 7f651140e700 1 -- 172.21.15.92:0/420636410 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650c0b1500 con 0x7f650c0b0760 2023-10-06T10:40:39.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.774+0000 7f651140e700 1 --2- 172.21.15.92:0/420636410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f650c0b0760 0x7f650c0b1bc0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6504009f20 tx=0x7f6504009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48290ad1dc67a5ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:39.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.774+0000 7f650b7fe700 1 -- 172.21.15.92:0/420636410 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650400e070 con 0x7f650c0b0760 2023-10-06T10:40:39.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.774+0000 7f650b7fe700 1 -- 172.21.15.92:0/420636410 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650400ba40 con 0x7f650c0b0760 2023-10-06T10:40:39.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.774+0000 7f650b7fe700 1 -- 172.21.15.92:0/420636410 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65040092e0 con 0x7f650c0b0760 2023-10-06T10:40:39.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.775+0000 7f6512e71700 1 -- 172.21.15.92:0/420636410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f650c0b0760 msgr2=0x7f650c0b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:39.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.775+0000 7f6512e71700 1 --2- 172.21.15.92:0/420636410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f650c0b0760 0x7f650c0b1bc0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6504009f20 tx=0x7f6504009840 comp rx=0 tx=0).stop 2023-10-06T10:40:39.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.775+0000 7f6512e71700 1 -- 172.21.15.92:0/420636410 shutdown_connections 2023-10-06T10:40:39.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.775+0000 7f6512e71700 1 --2- 172.21.15.92:0/420636410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f650c0b0760 0x7f650c0b1bc0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.775+0000 7f6512e71700 1 --2- 172.21.15.92:0/420636410 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f650c159560 0x7f650c0aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.775+0000 7f6512e71700 1 -- 172.21.15.92:0/420636410 >> 172.21.15.92:0/420636410 conn(0x7f650c0997f0 msgr2=0x7f650c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:39.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.775+0000 7f6512e71700 1 -- 172.21.15.92:0/420636410 shutdown_connections 2023-10-06T10:40:39.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.775+0000 7f6512e71700 1 -- 172.21.15.92:0/420636410 wait complete. 2023-10-06T10:40:39.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.776+0000 7f6512e71700 1 Processor -- start 2023-10-06T10:40:39.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.776+0000 7f6512e71700 1 -- start start 2023-10-06T10:40:39.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.777+0000 7f6512e71700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f650c0b0760 0x7f650c1584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:39.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.777+0000 7f6512e71700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f650c0b49a0 0x7f650c09bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:39.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.777+0000 7f6512e71700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f650c159560 0x7f650c09c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:39.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.777+0000 7f6512e71700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650c09ad70 con 0x7f650c0b0760 2023-10-06T10:40:39.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.777+0000 7f6512e71700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650c09a160 con 0x7f650c159560 2023-10-06T10:40:39.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.777+0000 7f6512e71700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650c09a440 con 0x7f650c0b49a0 2023-10-06T10:40:39.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.777+0000 7f6510c0d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f650c0b0760 0x7f650c1584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:39.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.777+0000 7f6510c0d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f650c0b0760 0x7f650c1584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60680/0 (socket says 172.21.15.92:60680) 2023-10-06T10:40:39.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.777+0000 7f6510c0d700 1 -- 172.21.15.92:0/2753380943 learned_addr learned my addr 172.21.15.92:0/2753380943 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:39.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.777+0000 7f6510c0d700 1 -- 172.21.15.92:0/2753380943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f650c0b49a0 msgr2=0x7f650c09bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:39.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.777+0000 7f6510c0d700 1 --2- 172.21.15.92:0/2753380943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f650c0b49a0 0x7f650c09bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.777+0000 7f6510c0d700 1 -- 172.21.15.92:0/2753380943 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f650c159560 msgr2=0x7f650c09c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:39.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.777+0000 7f6510c0d700 1 --2- 172.21.15.92:0/2753380943 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f650c159560 0x7f650c09c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.777+0000 7f6510c0d700 1 -- 172.21.15.92:0/2753380943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6504009ae0 con 0x7f650c0b0760 2023-10-06T10:40:39.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.777+0000 7f6510c0d700 1 --2- 172.21.15.92:0/2753380943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f650c0b0760 0x7f650c1584d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f650000d840 tx=0x7f6500008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:39.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.778+0000 7f6508ff9700 1 -- 172.21.15.92:0/2753380943 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6500014620 con 0x7f650c0b0760 2023-10-06T10:40:39.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.778+0000 7f6508ff9700 1 -- 172.21.15.92:0/2753380943 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6500014780 con 0x7f650c0b0760 2023-10-06T10:40:39.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.778+0000 7f6512e71700 1 -- 172.21.15.92:0/2753380943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f650c09a720 con 0x7f650c0b0760 2023-10-06T10:40:39.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.778+0000 7f6508ff9700 1 -- 172.21.15.92:0/2753380943 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6500014b40 con 0x7f650c0b0760 2023-10-06T10:40:39.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.778+0000 7f6512e71700 1 -- 172.21.15.92:0/2753380943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650c158a10 con 0x7f650c0b0760 2023-10-06T10:40:39.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.779+0000 7f6512e71700 1 -- 172.21.15.92:0/2753380943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f650c064a90 con 0x7f650c0b0760 2023-10-06T10:40:39.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.779+0000 7f6508ff9700 1 -- 172.21.15.92:0/2753380943 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f650001d400 con 0x7f650c0b0760 2023-10-06T10:40:39.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.779+0000 7f6508ff9700 1 --2- 172.21.15.92:0/2753380943 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f64f0066f30 0x7f64f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:39.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.779+0000 7f6508ff9700 1 -- 172.21.15.92:0/2753380943 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6500085020 con 0x7f650c0b0760 2023-10-06T10:40:39.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.780+0000 7f650bfff700 1 --2- 172.21.15.92:0/2753380943 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f64f0066f30 0x7f64f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:39.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.780+0000 7f650bfff700 1 --2- 172.21.15.92:0/2753380943 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f64f0066f30 0x7f64f00693e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f64f8007fd0 tx=0x7f64f8007b20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:39.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.784+0000 7f6508ff9700 1 -- 172.21.15.92:0/2753380943 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6500051ff0 con 0x7f650c0b0760 2023-10-06T10:40:39.839 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-10-06T10:40:39.837+0000 7fcf3b9ca700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T10:40:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.962+0000 7f6512e71700 1 -- 172.21.15.92:0/2753380943 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f650c06a2a0 con 0x7f64f0066f30 2023-10-06T10:40:39.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.970+0000 7f6508ff9700 1 -- 172.21.15.92:0/2753380943 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300820 (secure 0 0 0) 0x7f650c06a2a0 con 0x7f64f0066f30 2023-10-06T10:40:39.975 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:39.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.973+0000 7f6512e71700 1 -- 172.21.15.92:0/2753380943 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f64f0066f30 msgr2=0x7f64f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:39.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.973+0000 7f6512e71700 1 --2- 172.21.15.92:0/2753380943 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f64f0066f30 0x7f64f00693e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f64f8007fd0 tx=0x7f64f8007b20 comp rx=0 tx=0).stop 2023-10-06T10:40:39.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.973+0000 7f6512e71700 1 -- 172.21.15.92:0/2753380943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f650c0b0760 msgr2=0x7f650c1584d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:39.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.973+0000 7f6512e71700 1 --2- 172.21.15.92:0/2753380943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f650c0b0760 0x7f650c1584d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f650000d840 tx=0x7f6500008040 comp rx=0 tx=0).stop 2023-10-06T10:40:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.973+0000 7f6512e71700 1 -- 172.21.15.92:0/2753380943 shutdown_connections 2023-10-06T10:40:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.973+0000 7f6512e71700 1 --2- 172.21.15.92:0/2753380943 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f64f0066f30 0x7f64f00693e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.973+0000 7f6512e71700 1 --2- 172.21.15.92:0/2753380943 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f650c159560 0x7f650c09c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.973+0000 7f6512e71700 1 --2- 172.21.15.92:0/2753380943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f650c0b0760 0x7f650c1584d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.973+0000 7f6512e71700 1 --2- 172.21.15.92:0/2753380943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f650c0b49a0 0x7f650c09bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.973+0000 7f6512e71700 1 -- 172.21.15.92:0/2753380943 >> 172.21.15.92:0/2753380943 conn(0x7f650c0997f0 msgr2=0x7f650c0688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.973+0000 7f6512e71700 1 -- 172.21.15.92:0/2753380943 shutdown_connections 2023-10-06T10:40:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:39.973+0000 7f6512e71700 1 -- 172.21.15.92:0/2753380943 wait complete. 2023-10-06T10:40:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:40:39.987 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2023-10-06T10:40:38.112480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2384004,"kb_used_data":5700,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752590716,"statfs":{"total":773094113280,"available":770652893184,"internally_reserved":0,"allocated":5836800,"data_stored":3307687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.575902"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.539087+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:03.396798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.541292+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:38.780441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.538829+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:56.154679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.540171+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:39.967833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.539827+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:03.396798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.538853+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:13.047456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.542046+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:58:09.360521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.539879+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:43:52.519978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.543819+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:39.937778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.540329+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:03.396798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.544327+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:27:47.178791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.540337+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:14.711366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.540790+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:05.264641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.540853+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:28:50.958890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.541599+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:13:10.093056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.543711+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:28:50.958890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.543105+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:06:19.788485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.538379+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:58:09.360521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.540876+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:39.937778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.539641+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:21.420689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.542579+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:17:35.395865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.541053+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:05.264641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.542547+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:35.191932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.540467+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:43:52.519978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.538361+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:13:10.093056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":40,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T10:40:30.525308+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:40:30.525308+0000","last_peered":"2023-10-06T10:40:30.525308+0000","last_clean":"2023-10-06T10:40:30.525308+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:40:30.525308+0000","last_undegraded":"2023-10-06T10:40:30.525308+0000","last_fullsized":"2023-10-06T10:40:30.525308+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.541264+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:41.469721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T10:40:32.841471+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:40:32.841471+0000","last_peered":"2023-10-06T10:40:32.841471+0000","last_clean":"2023-10-06T10:40:32.841471+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:40:32.841471+0000","last_undegraded":"2023-10-06T10:40:32.841471+0000","last_fullsized":"2023-10-06T10:40:32.841471+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.541812+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:37:29.893321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":38,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T10:40:30.524317+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:40:30.524317+0000","last_peered":"2023-10-06T10:40:30.524317+0000","last_clean":"2023-10-06T10:40:30.524317+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:40:30.524317+0000","last_undegraded":"2023-10-06T10:40:30.524317+0000","last_fullsized":"2023-10-06T10:40:30.524317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":183,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T10:40:30.642375+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:40:30.642375+0000","last_peered":"2023-10-06T10:40:30.642375+0000","last_clean":"2023-10-06T10:40:30.642375+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:40:30.642375+0000","last_undegraded":"2023-10-06T10:40:30.642375+0000","last_fullsized":"2023-10-06T10:40:30.642375+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.540320+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:13.047456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":33,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T10:40:30.642376+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:40:30.642376+0000","last_peered":"2023-10-06T10:40:30.642376+0000","last_clean":"2023-10-06T10:40:30.642376+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:40:30.642376+0000","last_undegraded":"2023-10-06T10:40:30.642376+0000","last_fullsized":"2023-10-06T10:40:30.642376+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.540741+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:35:55.915864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":33,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T10:40:30.642361+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:40:30.642361+0000","last_peered":"2023-10-06T10:40:30.642361+0000","last_clean":"2023-10-06T10:40:30.642361+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:40:30.642361+0000","last_undegraded":"2023-10-06T10:40:30.642361+0000","last_fullsized":"2023-10-06T10:40:30.642361+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.540755+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:37:38.300612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T10:40:30.525296+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:40:30.525296+0000","last_peered":"2023-10-06T10:40:30.525296+0000","last_clean":"2023-10-06T10:40:30.525296+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:40:30.525296+0000","last_undegraded":"2023-10-06T10:40:30.525296+0000","last_fullsized":"2023-10-06T10:40:30.525296+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.540472+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:14.711366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T10:40:31.848862+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:40:31.848862+0000","last_peered":"2023-10-06T10:40:31.848862+0000","last_clean":"2023-10-06T10:40:31.848862+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:40:31.848862+0000","last_undegraded":"2023-10-06T10:40:31.848862+0000","last_fullsized":"2023-10-06T10:40:31.848862+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:40:34.536798+0000","last_change":"2023-10-06T10:40:34.536798+0000","last_active":"2023-10-06T10:40:34.536798+0000","last_peered":"2023-10-06T10:40:34.536798+0000","last_clean":"2023-10-06T10:40:34.536798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:34.536798+0000","last_undegraded":"2023-10-06T10:40:34.536798+0000","last_fullsized":"2023-10-06T10:40:34.536798+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T10:40:33.528022+0000","last_change":"2023-10-06T10:40:33.538821+0000","last_active":"2023-10-06T10:40:33.528022+0000","last_peered":"2023-10-06T10:40:33.528022+0000","last_clean":"2023-10-06T10:40:33.528022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:40:33.528022+0000","last_undegraded":"2023-10-06T10:40:33.528022+0000","last_fullsized":"2023-10-06T10:40:33.528022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:43:17.508745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":36,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T10:40:31.848848+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:40:31.848848+0000","last_peered":"2023-10-06T10:40:31.848848+0000","last_clean":"2023-10-06T10:40:31.848848+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:40:31.848848+0000","last_undegraded":"2023-10-06T10:40:31.848848+0000","last_fullsized":"2023-10-06T10:40:31.848848+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298436,"kb_used_data":1148,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073404,"statfs":{"total":96636764160,"available":96331165696,"internally_reserved":0,"allocated":1175552,"data_stored":856406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.48099999999999998,"15min":0.47799999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71699999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.40500000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78600000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60599999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.46600000000000003,"15min":0.45700000000000002},"min":{"1min":0.375,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65900000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.505,"15min":0.5},"min":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72399999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.752}]},{"osd":2,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.40600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.627},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.496,"15min":0.49199999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69199999999999995,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.70099999999999996}]},{"osd":3,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54800000000000004,"15min":0.54200000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.73799999999999999}]},{"osd":4,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.46800000000000003,"15min":0.47699999999999998},"min":{"1min":0.30499999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57699999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.501,"15min":0.50900000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.32700000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65900000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.34300000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66800000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.65700000000000003}]},{"osd":6,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.56799999999999995,"15min":0.57699999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.75700000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.747,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.76200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297852,"kb_used_data":564,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073988,"statfs":{"total":96636764160,"available":96331763712,"internally_reserved":0,"allocated":577536,"data_stored":264685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.49399999999999999,"15min":0.48899999999999999},"min":{"1min":0.36299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.435},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.44500000000000001,"15min":0.42899999999999999},"min":{"1min":0.38800000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.57299999999999995}]},{"osd":1,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.45100000000000001,"15min":0.437},"min":{"1min":0.33100000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.53100000000000003}]},{"osd":2,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.379,"5min":0.309,"15min":0.309},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.47699999999999998,"15min":0.46400000000000002},"min":{"1min":0.34999999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.45700000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.497,"15min":0.48799999999999999},"min":{"1min":0.38800000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.502},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.48799999999999999}]},{"osd":4,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47899999999999998,"15min":0.47799999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70899999999999996,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.40899999999999997}]},{"osd":5,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.48999999999999999,"15min":0.48599999999999999},"min":{"1min":0.28899999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69299999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33900000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.754,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.63}]}]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298436,"kb_used_data":1148,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073404,"statfs":{"total":96636764160,"available":96331165696,"internally_reserved":0,"allocated":1175552,"data_stored":856406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33000000000000002,"15min":0.32800000000000001},"min":{"1min":0.253,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.34999999999999998,"15min":0.34599999999999997},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.53900000000000003,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.58799999999999997}]},{"osd":2,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.32500000000000001,"15min":0.31900000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.36299999999999999},"min":{"1min":0.28999999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.60799999999999998}]},{"osd":3,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.375},"min":{"1min":0.30099999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.379,"15min":0.375},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.40300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.69499999999999995,"5min":0.746,"15min":0.746},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.80000000000000004}]},{"osd":5,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54800000000000004,"15min":0.54100000000000004},"min":{"1min":0.436,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.82899999999999996}]},{"osd":6,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.40100000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.47899999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.93500000000000005}]},{"osd":7,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58099999999999996,"15min":0.57399999999999995},"min":{"1min":0.48699999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.996},{"interface":"front","average":{"1min":0.622,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.439,"5min":0.436,"15min":0.436},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.88500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297856,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073984,"statfs":{"total":96636764160,"available":96331759616,"internally_reserved":0,"allocated":581632,"data_stored":266038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.32600000000000001,"15min":0.31900000000000001},"min":{"1min":0.23200000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.34899999999999998,"15min":0.34100000000000003},"min":{"1min":0.27600000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.35199999999999998}]},{"osd":2,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.312,"15min":0.30599999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.33300000000000002,"15min":0.32500000000000001},"min":{"1min":0.26500000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.52000000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.318}]},{"osd":3,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.35099999999999998,"15min":0.34300000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.35499999999999998,"15min":0.34899999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.34000000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.42599999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81200000000000006,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.54200000000000004,"15min":0.53300000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.501}]},{"osd":5,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.41799999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.91900000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.80900000000000005,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.51200000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58699999999999997,"15min":0.58799999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77300000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86399999999999999,"5min":1.075,"15min":1.075},"last":0.52800000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79700000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.441},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.61399999999999999,"15min":0.60999999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88800000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.54700000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297856,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073984,"statfs":{"total":96636764160,"available":96331759616,"internally_reserved":0,"allocated":581632,"data_stored":266038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.313,"15min":0.311},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.32500000000000001,"15min":0.317},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.29199999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.32900000000000001,"15min":0.32300000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.54200000000000004,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.247},{"interface":"front","average":{"1min":0.38,"5min":0.36699999999999999,"15min":0.36499999999999999},"min":{"1min":0.26400000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59299999999999997,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.26500000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.34899999999999998,"15min":0.34499999999999997},"min":{"1min":0.252,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.52700000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.35799999999999998,"15min":0.35499999999999998},"min":{"1min":0.27600000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.33200000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.35499999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.76100000000000001,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82699999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54100000000000004}]},{"osd":5,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.377,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81599999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.60699999999999998}]},{"osd":6,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.625,"15min":0.624},"min":{"1min":0.45800000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89000000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.624,"15min":0.623},"min":{"1min":0.433,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68000000000000005}]},{"osd":7,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.44500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79100000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.498,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.65800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297856,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073984,"statfs":{"total":96636764160,"available":96331759616,"internally_reserved":0,"allocated":581632,"data_stored":266038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.438,"15min":0.45600000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.50600000000000001,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.31},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.36799999999999999,"15min":0.36399999999999999},"min":{"1min":0.27100000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66800000000000004,"5min":1.218,"15min":1.218},"last":0.374}]},{"osd":1,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34100000000000003,"15min":0.34000000000000002},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.61899999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.48299999999999998,"15min":0.495},"min":{"1min":0.27800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60299999999999998,"5min":3.359,"15min":3.359},"last":0.39700000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38500000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.57699999999999996,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.63700000000000001,"15min":0.67300000000000004},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57699999999999996,"5min":7.069,"15min":7.069},"last":0.38400000000000001}]},{"osd":4,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89400000000000002,"5min":1.165,"15min":1.165},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59899999999999998,"15min":0.59699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79400000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.50800000000000001}]},{"osd":5,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.69799999999999995,"15min":0.71399999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92500000000000004,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60799999999999998,"15min":0.61099999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.96899999999999997,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.54300000000000004}]},{"osd":6,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.38,"5min":0.378,"15min":0.378},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.72699999999999998,"15min":0.74199999999999999},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84699999999999998,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.63100000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.61099999999999999},"min":{"1min":0.41299999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.99099999999999999,"5min":1.137,"15min":1.137},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.443,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91400000000000003,"5min":1.014,"15min":1.014},"last":0.55500000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297856,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073984,"statfs":{"total":96636764160,"available":96331759616,"internally_reserved":0,"allocated":581632,"data_stored":266038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47999999999999998,"15min":0.48099999999999998},"min":{"1min":0.33200000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.60799999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.496},"min":{"1min":0.33100000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.67000000000000004,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.52200000000000002}]},{"osd":1,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.495,"15min":0.496},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.59399999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.60299999999999998}]},{"osd":2,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50700000000000001,"15min":0.50800000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.63,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36099999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.68500000000000005,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.66900000000000004}]},{"osd":3,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51600000000000001,"15min":0.51200000000000001},"min":{"1min":0.42199999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.625,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.502,"5min":0.53000000000000003,"15min":0.53500000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.64800000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.57899999999999996}]},{"osd":5,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.495,"15min":0.50800000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.61599999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.377},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.49399999999999999,"15min":0.502},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.63300000000000001,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.47099999999999997}]},{"osd":6,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.53600000000000003,"15min":0.54600000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.68799999999999994,"5min":0.75,"15min":0.75},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.67700000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.49099999999999999}]},{"osd":7,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.56899999999999995,"15min":0.57999999999999996},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70499999999999996,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72499999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.44400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297856,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073984,"statfs":{"total":96636764160,"available":96331759616,"internally_reserved":0,"allocated":581632,"data_stored":266038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.47399999999999998,"15min":0.46300000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54000000000000004,"15min":0.53400000000000003},"min":{"1min":0.35899999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.748,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.57899999999999996}]},{"osd":1,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.496,"15min":0.48499999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.52500000000000002,"15min":0.51700000000000002},"min":{"1min":0.39200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.69299999999999995}]},{"osd":2,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.51000000000000001,"15min":0.499},"min":{"1min":0.32000000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.41499999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.754,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.66800000000000004}]},{"osd":3,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.57799999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77700000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78500000000000003,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.496}]},{"osd":4,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.32100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.69599999999999995,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.439},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.66100000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.51500000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.67500000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.746}]},{"osd":7,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68799999999999994,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76700000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.59099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:40:39.994 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:40:40.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.142+0000 7efc30473700 1 Processor -- start 2023-10-06T10:40:40.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.142+0000 7efc30473700 1 -- start start 2023-10-06T10:40:40.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.143+0000 7efc30473700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efc280b29b0 0x7efc28159120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:40.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.143+0000 7efc30473700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc28159660 0x7efc280cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.143+0000 7efc30473700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc280b0da0 con 0x7efc280b25e0 2023-10-06T10:40:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.143+0000 7efc30473700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc280cbfa0 con 0x7efc280b29b0 2023-10-06T10:40:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.143+0000 7efc30473700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc280cb390 con 0x7efc28159660 2023-10-06T10:40:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.143+0000 7efc2da0e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efc280b29b0 0x7efc28159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.143+0000 7efc2da0e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efc280b29b0 0x7efc28159120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60694/0 (socket says 172.21.15.92:60694) 2023-10-06T10:40:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.143+0000 7efc2da0e700 1 -- 172.21.15.92:0/1760384185 learned_addr learned my addr 172.21.15.92:0/1760384185 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.143+0000 7efc2ea10700 1 --2- 172.21.15.92:0/1760384185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc28159660 0x7efc280cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.144+0000 7efc2da0e700 1 -- 172.21.15.92:0/1760384185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc28159660 msgr2=0x7efc280cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.144+0000 7efc2da0e700 1 --2- 172.21.15.92:0/1760384185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc28159660 0x7efc280cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.144+0000 7efc2da0e700 1 -- 172.21.15.92:0/1760384185 >> v1:172.21.15.124:6789/0 conn(0x7efc280b25e0 legacy=0x7efc280b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.144+0000 7efc2da0e700 1 -- 172.21.15.92:0/1760384185 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc280cb670 con 0x7efc280b29b0 2023-10-06T10:40:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.144+0000 7efc2da0e700 1 --2- 172.21.15.92:0/1760384185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc280b29b0 0x7efc28159120 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7efc180099b0 tx=0x7efc18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0fd1d7bef6ff8f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.144+0000 7efc2d20d700 1 -- 172.21.15.92:0/1760384185 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18009ea0 con 0x7efc280b29b0 2023-10-06T10:40:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.144+0000 7efc2d20d700 1 -- 172.21.15.92:0/1760384185 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1800b860 con 0x7efc280b29b0 2023-10-06T10:40:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.144+0000 7efc2d20d700 1 -- 172.21.15.92:0/1760384185 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1800bc00 con 0x7efc280b29b0 2023-10-06T10:40:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.144+0000 7efc30473700 1 -- 172.21.15.92:0/1760384185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc280b29b0 msgr2=0x7efc28159120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.144+0000 7efc30473700 1 --2- 172.21.15.92:0/1760384185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc280b29b0 0x7efc28159120 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7efc180099b0 tx=0x7efc18009250 comp rx=0 tx=0).stop 2023-10-06T10:40:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.145+0000 7efc30473700 1 -- 172.21.15.92:0/1760384185 shutdown_connections 2023-10-06T10:40:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.145+0000 7efc30473700 1 --2- 172.21.15.92:0/1760384185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc28159660 0x7efc280cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.145+0000 7efc30473700 1 --2- 172.21.15.92:0/1760384185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc280b29b0 0x7efc28159120 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.145+0000 7efc30473700 1 -- 172.21.15.92:0/1760384185 >> 172.21.15.92:0/1760384185 conn(0x7efc2806ece0 msgr2=0x7efc2806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:40.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.145+0000 7efc30473700 1 -- 172.21.15.92:0/1760384185 shutdown_connections 2023-10-06T10:40:40.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.145+0000 7efc30473700 1 -- 172.21.15.92:0/1760384185 wait complete. 2023-10-06T10:40:40.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.146+0000 7efc30473700 1 Processor -- start 2023-10-06T10:40:40.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.146+0000 7efc30473700 1 -- start start 2023-10-06T10:40:40.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.146+0000 7efc30473700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc280b25e0 0x7efc2815b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.146+0000 7efc30473700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc280b29b0 0x7efc2815bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.146+0000 7efc30473700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc28159660 0x7efc280c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.146+0000 7efc30473700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc280c57b0 con 0x7efc280b29b0 2023-10-06T10:40:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.146+0000 7efc30473700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2815b1e0 con 0x7efc280b25e0 2023-10-06T10:40:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.146+0000 7efc30473700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc280c4b30 con 0x7efc28159660 2023-10-06T10:40:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.146+0000 7efc2da0e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc280b29b0 0x7efc2815bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.147+0000 7efc2da0e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc280b29b0 0x7efc2815bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60704/0 (socket says 172.21.15.92:60704) 2023-10-06T10:40:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.147+0000 7efc2da0e700 1 -- 172.21.15.92:0/1584655581 learned_addr learned my addr 172.21.15.92:0/1584655581 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.147+0000 7efc2ea10700 1 --2- 172.21.15.92:0/1584655581 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc28159660 0x7efc280c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.147+0000 7efc2e20f700 1 --2- 172.21.15.92:0/1584655581 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc280b25e0 0x7efc2815b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.147+0000 7efc2da0e700 1 -- 172.21.15.92:0/1584655581 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc28159660 msgr2=0x7efc280c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:40.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.147+0000 7efc2da0e700 1 --2- 172.21.15.92:0/1584655581 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc28159660 0x7efc280c3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.147+0000 7efc2da0e700 1 -- 172.21.15.92:0/1584655581 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc280b25e0 msgr2=0x7efc2815b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:40.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.147+0000 7efc2da0e700 1 --2- 172.21.15.92:0/1584655581 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc280b25e0 0x7efc2815b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.147+0000 7efc2da0e700 1 -- 172.21.15.92:0/1584655581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc180094f0 con 0x7efc280b29b0 2023-10-06T10:40:40.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.147+0000 7efc2da0e700 1 --2- 172.21.15.92:0/1584655581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc280b29b0 0x7efc2815bc70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7efc2815d1c0 tx=0x7efc18017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:40.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.147+0000 7efc1e7fc700 1 -- 172.21.15.92:0/1584655581 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18000d10 con 0x7efc280b29b0 2023-10-06T10:40:40.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.147+0000 7efc1e7fc700 1 -- 172.21.15.92:0/1584655581 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1800bbe0 con 0x7efc280b29b0 2023-10-06T10:40:40.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.147+0000 7efc30473700 1 -- 172.21.15.92:0/1584655581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc280c4e10 con 0x7efc280b29b0 2023-10-06T10:40:40.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.147+0000 7efc1e7fc700 1 -- 172.21.15.92:0/1584655581 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18011670 con 0x7efc280b29b0 2023-10-06T10:40:40.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.147+0000 7efc30473700 1 -- 172.21.15.92:0/1584655581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc280c53d0 con 0x7efc280b29b0 2023-10-06T10:40:40.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.148+0000 7efc30473700 1 -- 172.21.15.92:0/1584655581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc280c58f0 con 0x7efc280b29b0 2023-10-06T10:40:40.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.148+0000 7efc1e7fc700 1 -- 172.21.15.92:0/1584655581 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efc18011aa0 con 0x7efc280b29b0 2023-10-06T10:40:40.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.149+0000 7efc1e7fc700 1 --2- 172.21.15.92:0/1584655581 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efc0c066f30 0x7efc0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:40.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.149+0000 7efc1e7fc700 1 -- 172.21.15.92:0/1584655581 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7efc18086030 con 0x7efc280b29b0 2023-10-06T10:40:40.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.149+0000 7efc2e20f700 1 --2- 172.21.15.92:0/1584655581 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efc0c066f30 0x7efc0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:40.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.149+0000 7efc2e20f700 1 --2- 172.21.15.92:0/1584655581 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efc0c066f30 0x7efc0c0693e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efc2400c580 tx=0x7efc24001b20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:40.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.153+0000 7efc1e7fc700 1 -- 172.21.15.92:0/1584655581 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc18053000 con 0x7efc280b29b0 2023-10-06T10:40:40.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.330+0000 7efc30473700 1 -- 172.21.15.92:0/1584655581 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc28064a90 con 0x7efc0c066f30 2023-10-06T10:40:40.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.338+0000 7efc1e7fc700 1 -- 172.21.15.92:0/1584655581 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311702 (secure 0 0 0) 0x7efc28064a90 con 0x7efc0c066f30 2023-10-06T10:40:40.343 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:40.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.341+0000 7efc30473700 1 -- 172.21.15.92:0/1584655581 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efc0c066f30 msgr2=0x7efc0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:40.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.341+0000 7efc30473700 1 --2- 172.21.15.92:0/1584655581 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efc0c066f30 0x7efc0c0693e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efc2400c580 tx=0x7efc24001b20 comp rx=0 tx=0).stop 2023-10-06T10:40:40.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.341+0000 7efc30473700 1 -- 172.21.15.92:0/1584655581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc280b29b0 msgr2=0x7efc2815bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:40.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.341+0000 7efc30473700 1 --2- 172.21.15.92:0/1584655581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc280b29b0 0x7efc2815bc70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7efc2815d1c0 tx=0x7efc18017000 comp rx=0 tx=0).stop 2023-10-06T10:40:40.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.341+0000 7efc30473700 1 -- 172.21.15.92:0/1584655581 shutdown_connections 2023-10-06T10:40:40.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.341+0000 7efc30473700 1 --2- 172.21.15.92:0/1584655581 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efc0c066f30 0x7efc0c0693e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.341+0000 7efc30473700 1 --2- 172.21.15.92:0/1584655581 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc28159660 0x7efc280c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.341+0000 7efc30473700 1 --2- 172.21.15.92:0/1584655581 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc280b25e0 0x7efc2815b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.341+0000 7efc30473700 1 --2- 172.21.15.92:0/1584655581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc280b29b0 0x7efc2815bc70 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.341+0000 7efc30473700 1 -- 172.21.15.92:0/1584655581 >> 172.21.15.92:0/1584655581 conn(0x7efc2806ece0 msgr2=0x7efc28154850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:40.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.341+0000 7efc30473700 1 -- 172.21.15.92:0/1584655581 shutdown_connections 2023-10-06T10:40:40.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.341+0000 7efc30473700 1 -- 172.21.15.92:0/1584655581 wait complete. 2023-10-06T10:40:40.347 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:40:40.355 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2023-10-06T10:40:40.112966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":201,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2384356,"kb_used_data":6052,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752590364,"statfs":{"total":773094113280,"available":770652532736,"internally_reserved":0,"allocated":6197248,"data_stored":3482899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001152"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.575414+0000","last_change":"2023-10-06T10:40:35.572024+0000","last_active":"2023-10-06T10:40:35.575414+0000","last_peered":"2023-10-06T10:40:35.575414+0000","last_clean":"2023-10-06T10:40:35.575414+0000","last_became_active":"2023-10-06T10:40:35.571764+0000","last_became_peered":"2023-10-06T10:40:35.571764+0000","last_unstale":"2023-10-06T10:40:35.575414+0000","last_undegraded":"2023-10-06T10:40:35.575414+0000","last_fullsized":"2023-10-06T10:40:35.575414+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:41.911815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576306+0000","last_change":"2023-10-06T10:40:35.570742+0000","last_active":"2023-10-06T10:40:35.576306+0000","last_peered":"2023-10-06T10:40:35.576306+0000","last_clean":"2023-10-06T10:40:35.576306+0000","last_became_active":"2023-10-06T10:40:35.570475+0000","last_became_peered":"2023-10-06T10:40:35.570475+0000","last_unstale":"2023-10-06T10:40:35.576306+0000","last_undegraded":"2023-10-06T10:40:35.576306+0000","last_fullsized":"2023-10-06T10:40:35.576306+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:01:49.371198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569068+0000","last_change":"2023-10-06T10:40:35.563948+0000","last_active":"2023-10-06T10:40:35.569068+0000","last_peered":"2023-10-06T10:40:35.569068+0000","last_clean":"2023-10-06T10:40:35.569068+0000","last_became_active":"2023-10-06T10:40:35.563663+0000","last_became_peered":"2023-10-06T10:40:35.563663+0000","last_unstale":"2023-10-06T10:40:35.569068+0000","last_undegraded":"2023-10-06T10:40:35.569068+0000","last_fullsized":"2023-10-06T10:40:35.569068+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:16:56.359962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.564545+0000","last_change":"2023-10-06T10:40:35.560154+0000","last_active":"2023-10-06T10:40:35.564545+0000","last_peered":"2023-10-06T10:40:35.564545+0000","last_clean":"2023-10-06T10:40:35.564545+0000","last_became_active":"2023-10-06T10:40:35.559902+0000","last_became_peered":"2023-10-06T10:40:35.559902+0000","last_unstale":"2023-10-06T10:40:35.564545+0000","last_undegraded":"2023-10-06T10:40:35.564545+0000","last_fullsized":"2023-10-06T10:40:35.564545+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:10.010731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.558859+0000","last_change":"2023-10-06T10:40:35.557573+0000","last_active":"2023-10-06T10:40:35.558859+0000","last_peered":"2023-10-06T10:40:35.558859+0000","last_clean":"2023-10-06T10:40:35.558859+0000","last_became_active":"2023-10-06T10:40:35.557357+0000","last_became_peered":"2023-10-06T10:40:35.557357+0000","last_unstale":"2023-10-06T10:40:35.558859+0000","last_undegraded":"2023-10-06T10:40:35.558859+0000","last_fullsized":"2023-10-06T10:40:35.558859+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:13:29.740541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.573528+0000","last_change":"2023-10-06T10:40:35.567916+0000","last_active":"2023-10-06T10:40:35.573528+0000","last_peered":"2023-10-06T10:40:35.573528+0000","last_clean":"2023-10-06T10:40:35.573528+0000","last_became_active":"2023-10-06T10:40:35.567655+0000","last_became_peered":"2023-10-06T10:40:35.567655+0000","last_unstale":"2023-10-06T10:40:35.573528+0000","last_undegraded":"2023-10-06T10:40:35.573528+0000","last_fullsized":"2023-10-06T10:40:35.573528+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:08:59.916087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.564828+0000","last_change":"2023-10-06T10:40:35.556880+0000","last_active":"2023-10-06T10:40:35.564828+0000","last_peered":"2023-10-06T10:40:35.564828+0000","last_clean":"2023-10-06T10:40:35.564828+0000","last_became_active":"2023-10-06T10:40:35.556638+0000","last_became_peered":"2023-10-06T10:40:35.556638+0000","last_unstale":"2023-10-06T10:40:35.564828+0000","last_undegraded":"2023-10-06T10:40:35.564828+0000","last_fullsized":"2023-10-06T10:40:35.564828+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:15.757865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576825+0000","last_change":"2023-10-06T10:40:35.571655+0000","last_active":"2023-10-06T10:40:35.576825+0000","last_peered":"2023-10-06T10:40:35.576825+0000","last_clean":"2023-10-06T10:40:35.576825+0000","last_became_active":"2023-10-06T10:40:35.565829+0000","last_became_peered":"2023-10-06T10:40:35.565829+0000","last_unstale":"2023-10-06T10:40:35.576825+0000","last_undegraded":"2023-10-06T10:40:35.576825+0000","last_fullsized":"2023-10-06T10:40:35.576825+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:13.916559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569860+0000","last_change":"2023-10-06T10:40:35.566171+0000","last_active":"2023-10-06T10:40:35.569860+0000","last_peered":"2023-10-06T10:40:35.569860+0000","last_clean":"2023-10-06T10:40:35.569860+0000","last_became_active":"2023-10-06T10:40:35.565812+0000","last_became_peered":"2023-10-06T10:40:35.565812+0000","last_unstale":"2023-10-06T10:40:35.569860+0000","last_undegraded":"2023-10-06T10:40:35.569860+0000","last_fullsized":"2023-10-06T10:40:35.569860+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:10.010731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.574755+0000","last_change":"2023-10-06T10:40:35.566646+0000","last_active":"2023-10-06T10:40:35.574755+0000","last_peered":"2023-10-06T10:40:35.574755+0000","last_clean":"2023-10-06T10:40:35.574755+0000","last_became_active":"2023-10-06T10:40:35.566383+0000","last_became_peered":"2023-10-06T10:40:35.566383+0000","last_unstale":"2023-10-06T10:40:35.574755+0000","last_undegraded":"2023-10-06T10:40:35.574755+0000","last_fullsized":"2023-10-06T10:40:35.574755+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:27.389629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.577321+0000","last_change":"2023-10-06T10:40:35.576738+0000","last_active":"2023-10-06T10:40:35.577321+0000","last_peered":"2023-10-06T10:40:35.577321+0000","last_clean":"2023-10-06T10:40:35.577321+0000","last_became_active":"2023-10-06T10:40:35.576470+0000","last_became_peered":"2023-10-06T10:40:35.576470+0000","last_unstale":"2023-10-06T10:40:35.577321+0000","last_undegraded":"2023-10-06T10:40:35.577321+0000","last_fullsized":"2023-10-06T10:40:35.577321+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:34:51.555583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.564516+0000","last_change":"2023-10-06T10:40:35.558573+0000","last_active":"2023-10-06T10:40:35.564516+0000","last_peered":"2023-10-06T10:40:35.564516+0000","last_clean":"2023-10-06T10:40:35.564516+0000","last_became_active":"2023-10-06T10:40:35.558278+0000","last_became_peered":"2023-10-06T10:40:35.558278+0000","last_unstale":"2023-10-06T10:40:35.564516+0000","last_undegraded":"2023-10-06T10:40:35.564516+0000","last_fullsized":"2023-10-06T10:40:35.564516+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:22.905054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.558118+0000","last_change":"2023-10-06T10:40:35.555626+0000","last_active":"2023-10-06T10:40:35.558118+0000","last_peered":"2023-10-06T10:40:35.558118+0000","last_clean":"2023-10-06T10:40:35.558118+0000","last_became_active":"2023-10-06T10:40:35.555367+0000","last_became_peered":"2023-10-06T10:40:35.555367+0000","last_unstale":"2023-10-06T10:40:35.558118+0000","last_undegraded":"2023-10-06T10:40:35.558118+0000","last_fullsized":"2023-10-06T10:40:35.558118+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:22.429465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568058+0000","last_change":"2023-10-06T10:40:35.560779+0000","last_active":"2023-10-06T10:40:35.568058+0000","last_peered":"2023-10-06T10:40:35.568058+0000","last_clean":"2023-10-06T10:40:35.568058+0000","last_became_active":"2023-10-06T10:40:35.560517+0000","last_became_peered":"2023-10-06T10:40:35.560517+0000","last_unstale":"2023-10-06T10:40:35.568058+0000","last_undegraded":"2023-10-06T10:40:35.568058+0000","last_fullsized":"2023-10-06T10:40:35.568058+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:31.413269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568008+0000","last_change":"2023-10-06T10:40:35.559791+0000","last_active":"2023-10-06T10:40:35.568008+0000","last_peered":"2023-10-06T10:40:35.568008+0000","last_clean":"2023-10-06T10:40:35.568008+0000","last_became_active":"2023-10-06T10:40:35.559541+0000","last_became_peered":"2023-10-06T10:40:35.559541+0000","last_unstale":"2023-10-06T10:40:35.568008+0000","last_undegraded":"2023-10-06T10:40:35.568008+0000","last_fullsized":"2023-10-06T10:40:35.568008+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:48.719683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.572860+0000","last_change":"2023-10-06T10:40:35.562284+0000","last_active":"2023-10-06T10:40:35.572860+0000","last_peered":"2023-10-06T10:40:35.572860+0000","last_clean":"2023-10-06T10:40:35.572860+0000","last_became_active":"2023-10-06T10:40:35.561931+0000","last_became_peered":"2023-10-06T10:40:35.561931+0000","last_unstale":"2023-10-06T10:40:35.572860+0000","last_undegraded":"2023-10-06T10:40:35.572860+0000","last_fullsized":"2023-10-06T10:40:35.572860+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:55.843404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576252+0000","last_change":"2023-10-06T10:40:35.567770+0000","last_active":"2023-10-06T10:40:35.576252+0000","last_peered":"2023-10-06T10:40:35.576252+0000","last_clean":"2023-10-06T10:40:35.576252+0000","last_became_active":"2023-10-06T10:40:35.566223+0000","last_became_peered":"2023-10-06T10:40:35.566223+0000","last_unstale":"2023-10-06T10:40:35.576252+0000","last_undegraded":"2023-10-06T10:40:35.576252+0000","last_fullsized":"2023-10-06T10:40:35.576252+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:56.036704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.572361+0000","last_change":"2023-10-06T10:40:35.569168+0000","last_active":"2023-10-06T10:40:35.572361+0000","last_peered":"2023-10-06T10:40:35.572361+0000","last_clean":"2023-10-06T10:40:35.572361+0000","last_became_active":"2023-10-06T10:40:35.568859+0000","last_became_peered":"2023-10-06T10:40:35.568859+0000","last_unstale":"2023-10-06T10:40:35.572361+0000","last_undegraded":"2023-10-06T10:40:35.572361+0000","last_fullsized":"2023-10-06T10:40:35.572361+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:28:38.929115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569862+0000","last_change":"2023-10-06T10:40:35.566851+0000","last_active":"2023-10-06T10:40:35.569862+0000","last_peered":"2023-10-06T10:40:35.569862+0000","last_clean":"2023-10-06T10:40:35.569862+0000","last_became_active":"2023-10-06T10:40:35.566568+0000","last_became_peered":"2023-10-06T10:40:35.566568+0000","last_unstale":"2023-10-06T10:40:35.569862+0000","last_undegraded":"2023-10-06T10:40:35.569862+0000","last_fullsized":"2023-10-06T10:40:35.569862+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:50.111333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576908+0000","last_change":"2023-10-06T10:40:35.573233+0000","last_active":"2023-10-06T10:40:35.576908+0000","last_peered":"2023-10-06T10:40:35.576908+0000","last_clean":"2023-10-06T10:40:35.576908+0000","last_became_active":"2023-10-06T10:40:35.572994+0000","last_became_peered":"2023-10-06T10:40:35.572994+0000","last_unstale":"2023-10-06T10:40:35.576908+0000","last_undegraded":"2023-10-06T10:40:35.576908+0000","last_fullsized":"2023-10-06T10:40:35.576908+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:45:01.289103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.570019+0000","last_change":"2023-10-06T10:40:35.567541+0000","last_active":"2023-10-06T10:40:35.570019+0000","last_peered":"2023-10-06T10:40:35.570019+0000","last_clean":"2023-10-06T10:40:35.570019+0000","last_became_active":"2023-10-06T10:40:35.567165+0000","last_became_peered":"2023-10-06T10:40:35.567165+0000","last_unstale":"2023-10-06T10:40:35.570019+0000","last_undegraded":"2023-10-06T10:40:35.570019+0000","last_fullsized":"2023-10-06T10:40:35.570019+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:42.478496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576376+0000","last_change":"2023-10-06T10:40:35.568241+0000","last_active":"2023-10-06T10:40:35.576376+0000","last_peered":"2023-10-06T10:40:35.576376+0000","last_clean":"2023-10-06T10:40:35.576376+0000","last_became_active":"2023-10-06T10:40:35.568006+0000","last_became_peered":"2023-10-06T10:40:35.568006+0000","last_unstale":"2023-10-06T10:40:35.576376+0000","last_undegraded":"2023-10-06T10:40:35.576376+0000","last_fullsized":"2023-10-06T10:40:35.576376+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:16:58.559793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.577728+0000","last_change":"2023-10-06T10:40:35.577097+0000","last_active":"2023-10-06T10:40:35.577728+0000","last_peered":"2023-10-06T10:40:35.577728+0000","last_clean":"2023-10-06T10:40:35.577728+0000","last_became_active":"2023-10-06T10:40:35.576788+0000","last_became_peered":"2023-10-06T10:40:35.576788+0000","last_unstale":"2023-10-06T10:40:35.577728+0000","last_undegraded":"2023-10-06T10:40:35.577728+0000","last_fullsized":"2023-10-06T10:40:35.577728+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:47.164234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.563619+0000","last_change":"2023-10-06T10:40:35.562191+0000","last_active":"2023-10-06T10:40:35.563619+0000","last_peered":"2023-10-06T10:40:35.563619+0000","last_clean":"2023-10-06T10:40:35.563619+0000","last_became_active":"2023-10-06T10:40:35.561940+0000","last_became_peered":"2023-10-06T10:40:35.561940+0000","last_unstale":"2023-10-06T10:40:35.563619+0000","last_undegraded":"2023-10-06T10:40:35.563619+0000","last_fullsized":"2023-10-06T10:40:35.563619+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:40.960612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569894+0000","last_change":"2023-10-06T10:40:35.564934+0000","last_active":"2023-10-06T10:40:35.569894+0000","last_peered":"2023-10-06T10:40:35.569894+0000","last_clean":"2023-10-06T10:40:35.569894+0000","last_became_active":"2023-10-06T10:40:35.564686+0000","last_became_peered":"2023-10-06T10:40:35.564686+0000","last_unstale":"2023-10-06T10:40:35.569894+0000","last_undegraded":"2023-10-06T10:40:35.569894+0000","last_fullsized":"2023-10-06T10:40:35.569894+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:03.867863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.564523+0000","last_change":"2023-10-06T10:40:35.557812+0000","last_active":"2023-10-06T10:40:35.564523+0000","last_peered":"2023-10-06T10:40:35.564523+0000","last_clean":"2023-10-06T10:40:35.564523+0000","last_became_active":"2023-10-06T10:40:35.557534+0000","last_became_peered":"2023-10-06T10:40:35.557534+0000","last_unstale":"2023-10-06T10:40:35.564523+0000","last_undegraded":"2023-10-06T10:40:35.564523+0000","last_fullsized":"2023-10-06T10:40:35.564523+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:16:56.359962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569247+0000","last_change":"2023-10-06T10:40:35.562339+0000","last_active":"2023-10-06T10:40:35.569247+0000","last_peered":"2023-10-06T10:40:35.569247+0000","last_clean":"2023-10-06T10:40:35.569247+0000","last_became_active":"2023-10-06T10:40:35.562014+0000","last_became_peered":"2023-10-06T10:40:35.562014+0000","last_unstale":"2023-10-06T10:40:35.569247+0000","last_undegraded":"2023-10-06T10:40:35.569247+0000","last_fullsized":"2023-10-06T10:40:35.569247+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:55.143518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.558884+0000","last_change":"2023-10-06T10:40:35.555416+0000","last_active":"2023-10-06T10:40:35.558884+0000","last_peered":"2023-10-06T10:40:35.558884+0000","last_clean":"2023-10-06T10:40:35.558884+0000","last_became_active":"2023-10-06T10:40:35.555169+0000","last_became_peered":"2023-10-06T10:40:35.555169+0000","last_unstale":"2023-10-06T10:40:35.558884+0000","last_undegraded":"2023-10-06T10:40:35.558884+0000","last_fullsized":"2023-10-06T10:40:35.558884+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:11.988774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576530+0000","last_change":"2023-10-06T10:40:35.571179+0000","last_active":"2023-10-06T10:40:35.576530+0000","last_peered":"2023-10-06T10:40:35.576530+0000","last_clean":"2023-10-06T10:40:35.576530+0000","last_became_active":"2023-10-06T10:40:35.570784+0000","last_became_peered":"2023-10-06T10:40:35.570784+0000","last_unstale":"2023-10-06T10:40:35.576530+0000","last_undegraded":"2023-10-06T10:40:35.576530+0000","last_fullsized":"2023-10-06T10:40:35.576530+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:29:56.451363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569044+0000","last_change":"2023-10-06T10:40:35.564496+0000","last_active":"2023-10-06T10:40:35.569044+0000","last_peered":"2023-10-06T10:40:35.569044+0000","last_clean":"2023-10-06T10:40:35.569044+0000","last_became_active":"2023-10-06T10:40:35.564215+0000","last_became_peered":"2023-10-06T10:40:35.564215+0000","last_unstale":"2023-10-06T10:40:35.569044+0000","last_undegraded":"2023-10-06T10:40:35.569044+0000","last_fullsized":"2023-10-06T10:40:35.569044+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:06.273416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569438+0000","last_change":"2023-10-06T10:40:35.564889+0000","last_active":"2023-10-06T10:40:35.569438+0000","last_peered":"2023-10-06T10:40:35.569438+0000","last_clean":"2023-10-06T10:40:35.569438+0000","last_became_active":"2023-10-06T10:40:35.564630+0000","last_became_peered":"2023-10-06T10:40:35.564630+0000","last_unstale":"2023-10-06T10:40:35.569438+0000","last_undegraded":"2023-10-06T10:40:35.569438+0000","last_fullsized":"2023-10-06T10:40:35.569438+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:49.024348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.575282+0000","last_change":"2023-10-06T10:40:35.573554+0000","last_active":"2023-10-06T10:40:35.575282+0000","last_peered":"2023-10-06T10:40:35.575282+0000","last_clean":"2023-10-06T10:40:35.575282+0000","last_became_active":"2023-10-06T10:40:35.573307+0000","last_became_peered":"2023-10-06T10:40:35.573307+0000","last_unstale":"2023-10-06T10:40:35.575282+0000","last_undegraded":"2023-10-06T10:40:35.575282+0000","last_fullsized":"2023-10-06T10:40:35.575282+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:47.765804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568816+0000","last_change":"2023-10-06T10:40:35.564393+0000","last_active":"2023-10-06T10:40:35.568816+0000","last_peered":"2023-10-06T10:40:35.568816+0000","last_clean":"2023-10-06T10:40:35.568816+0000","last_became_active":"2023-10-06T10:40:35.564157+0000","last_became_peered":"2023-10-06T10:40:35.564157+0000","last_unstale":"2023-10-06T10:40:35.568816+0000","last_undegraded":"2023-10-06T10:40:35.568816+0000","last_fullsized":"2023-10-06T10:40:35.568816+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:06:14.418093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"51'4","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568646+0000","last_change":"2023-10-06T10:40:34.559819+0000","last_active":"2023-10-06T10:40:35.568646+0000","last_peered":"2023-10-06T10:40:35.568646+0000","last_clean":"2023-10-06T10:40:35.568646+0000","last_became_active":"2023-10-06T10:40:34.559433+0000","last_became_peered":"2023-10-06T10:40:34.559433+0000","last_unstale":"2023-10-06T10:40:35.568646+0000","last_undegraded":"2023-10-06T10:40:35.568646+0000","last_fullsized":"2023-10-06T10:40:35.568646+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:43:52.519978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.574934+0000","last_change":"2023-10-06T10:40:35.572023+0000","last_active":"2023-10-06T10:40:35.574934+0000","last_peered":"2023-10-06T10:40:35.574934+0000","last_clean":"2023-10-06T10:40:35.574934+0000","last_became_active":"2023-10-06T10:40:35.571788+0000","last_became_peered":"2023-10-06T10:40:35.571788+0000","last_unstale":"2023-10-06T10:40:35.574934+0000","last_undegraded":"2023-10-06T10:40:35.574934+0000","last_fullsized":"2023-10-06T10:40:35.574934+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:13.245750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"51'1","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.575594+0000","last_change":"2023-10-06T10:40:34.562460+0000","last_active":"2023-10-06T10:40:35.575594+0000","last_peered":"2023-10-06T10:40:35.575594+0000","last_clean":"2023-10-06T10:40:35.575594+0000","last_became_active":"2023-10-06T10:40:34.562241+0000","last_became_peered":"2023-10-06T10:40:34.562241+0000","last_unstale":"2023-10-06T10:40:35.575594+0000","last_undegraded":"2023-10-06T10:40:35.575594+0000","last_fullsized":"2023-10-06T10:40:35.575594+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:15.486524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.572595+0000","last_change":"2023-10-06T10:40:35.560246+0000","last_active":"2023-10-06T10:40:35.572595+0000","last_peered":"2023-10-06T10:40:35.572595+0000","last_clean":"2023-10-06T10:40:35.572595+0000","last_became_active":"2023-10-06T10:40:35.559981+0000","last_became_peered":"2023-10-06T10:40:35.559981+0000","last_unstale":"2023-10-06T10:40:35.572595+0000","last_undegraded":"2023-10-06T10:40:35.572595+0000","last_fullsized":"2023-10-06T10:40:35.572595+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:35.748566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"51'2","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.560216+0000","last_change":"2023-10-06T10:40:34.560326+0000","last_active":"2023-10-06T10:40:35.560216+0000","last_peered":"2023-10-06T10:40:35.560216+0000","last_clean":"2023-10-06T10:40:35.560216+0000","last_became_active":"2023-10-06T10:40:34.559983+0000","last_became_peered":"2023-10-06T10:40:34.559983+0000","last_unstale":"2023-10-06T10:40:35.560216+0000","last_undegraded":"2023-10-06T10:40:35.560216+0000","last_fullsized":"2023-10-06T10:40:35.560216+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:55:42.716617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.567779+0000","last_change":"2023-10-06T10:40:35.558007+0000","last_active":"2023-10-06T10:40:35.567779+0000","last_peered":"2023-10-06T10:40:35.567779+0000","last_clean":"2023-10-06T10:40:35.567779+0000","last_became_active":"2023-10-06T10:40:35.557606+0000","last_became_peered":"2023-10-06T10:40:35.557606+0000","last_unstale":"2023-10-06T10:40:35.567779+0000","last_undegraded":"2023-10-06T10:40:35.567779+0000","last_fullsized":"2023-10-06T10:40:35.567779+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:31.413269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"51'2","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.558037+0000","last_change":"2023-10-06T10:40:34.562219+0000","last_active":"2023-10-06T10:40:35.558037+0000","last_peered":"2023-10-06T10:40:35.558037+0000","last_clean":"2023-10-06T10:40:35.558037+0000","last_became_active":"2023-10-06T10:40:34.561960+0000","last_became_peered":"2023-10-06T10:40:34.561960+0000","last_unstale":"2023-10-06T10:40:35.558037+0000","last_undegraded":"2023-10-06T10:40:35.558037+0000","last_fullsized":"2023-10-06T10:40:35.558037+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:14.711366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576398+0000","last_change":"2023-10-06T10:40:35.572126+0000","last_active":"2023-10-06T10:40:35.576398+0000","last_peered":"2023-10-06T10:40:35.576398+0000","last_clean":"2023-10-06T10:40:35.576398+0000","last_became_active":"2023-10-06T10:40:35.571887+0000","last_became_peered":"2023-10-06T10:40:35.571887+0000","last_unstale":"2023-10-06T10:40:35.576398+0000","last_undegraded":"2023-10-06T10:40:35.576398+0000","last_fullsized":"2023-10-06T10:40:35.576398+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:47.765804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"51'1","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.560679+0000","last_change":"2023-10-06T10:40:34.557170+0000","last_active":"2023-10-06T10:40:35.560679+0000","last_peered":"2023-10-06T10:40:35.560679+0000","last_clean":"2023-10-06T10:40:35.560679+0000","last_became_active":"2023-10-06T10:40:34.556834+0000","last_became_peered":"2023-10-06T10:40:34.556834+0000","last_unstale":"2023-10-06T10:40:35.560679+0000","last_undegraded":"2023-10-06T10:40:35.560679+0000","last_fullsized":"2023-10-06T10:40:35.560679+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:38.780441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576498+0000","last_change":"2023-10-06T10:40:35.571018+0000","last_active":"2023-10-06T10:40:35.576498+0000","last_peered":"2023-10-06T10:40:35.576498+0000","last_clean":"2023-10-06T10:40:35.576498+0000","last_became_active":"2023-10-06T10:40:35.570523+0000","last_became_peered":"2023-10-06T10:40:35.570523+0000","last_unstale":"2023-10-06T10:40:35.576498+0000","last_undegraded":"2023-10-06T10:40:35.576498+0000","last_fullsized":"2023-10-06T10:40:35.576498+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:53.209242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"51'1","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568890+0000","last_change":"2023-10-06T10:40:34.555165+0000","last_active":"2023-10-06T10:40:35.568890+0000","last_peered":"2023-10-06T10:40:35.568890+0000","last_clean":"2023-10-06T10:40:35.568890+0000","last_became_active":"2023-10-06T10:40:34.554897+0000","last_became_peered":"2023-10-06T10:40:34.554897+0000","last_unstale":"2023-10-06T10:40:35.568890+0000","last_undegraded":"2023-10-06T10:40:35.568890+0000","last_fullsized":"2023-10-06T10:40:35.568890+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:05.264641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569444+0000","last_change":"2023-10-06T10:40:35.565210+0000","last_active":"2023-10-06T10:40:35.569444+0000","last_peered":"2023-10-06T10:40:35.569444+0000","last_clean":"2023-10-06T10:40:35.569444+0000","last_became_active":"2023-10-06T10:40:35.564970+0000","last_became_peered":"2023-10-06T10:40:35.564970+0000","last_unstale":"2023-10-06T10:40:35.569444+0000","last_undegraded":"2023-10-06T10:40:35.569444+0000","last_fullsized":"2023-10-06T10:40:35.569444+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:40.976608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.546745+0000","last_change":"2023-10-06T10:40:34.558169+0000","last_active":"2023-10-06T10:40:35.546745+0000","last_peered":"2023-10-06T10:40:35.546745+0000","last_clean":"2023-10-06T10:40:35.546745+0000","last_became_active":"2023-10-06T10:40:34.557947+0000","last_became_peered":"2023-10-06T10:40:34.557947+0000","last_unstale":"2023-10-06T10:40:35.546745+0000","last_undegraded":"2023-10-06T10:40:35.546745+0000","last_fullsized":"2023-10-06T10:40:35.546745+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:56.154679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576422+0000","last_change":"2023-10-06T10:40:35.572595+0000","last_active":"2023-10-06T10:40:35.576422+0000","last_peered":"2023-10-06T10:40:35.576422+0000","last_clean":"2023-10-06T10:40:35.576422+0000","last_became_active":"2023-10-06T10:40:35.572317+0000","last_became_peered":"2023-10-06T10:40:35.572317+0000","last_unstale":"2023-10-06T10:40:35.576422+0000","last_undegraded":"2023-10-06T10:40:35.576422+0000","last_fullsized":"2023-10-06T10:40:35.576422+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:17.634580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.547860+0000","last_change":"2023-10-06T10:40:34.556185+0000","last_active":"2023-10-06T10:40:35.547860+0000","last_peered":"2023-10-06T10:40:35.547860+0000","last_clean":"2023-10-06T10:40:35.547860+0000","last_became_active":"2023-10-06T10:40:34.555911+0000","last_became_peered":"2023-10-06T10:40:34.555911+0000","last_unstale":"2023-10-06T10:40:35.547860+0000","last_undegraded":"2023-10-06T10:40:35.547860+0000","last_fullsized":"2023-10-06T10:40:35.547860+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.285921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568612+0000","last_change":"2023-10-06T10:40:35.562668+0000","last_active":"2023-10-06T10:40:35.568612+0000","last_peered":"2023-10-06T10:40:35.568612+0000","last_clean":"2023-10-06T10:40:35.568612+0000","last_became_active":"2023-10-06T10:40:35.562428+0000","last_became_peered":"2023-10-06T10:40:35.562428+0000","last_unstale":"2023-10-06T10:40:35.568612+0000","last_undegraded":"2023-10-06T10:40:35.568612+0000","last_fullsized":"2023-10-06T10:40:35.568612+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:57:07.950757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.555289+0000","last_change":"2023-10-06T10:40:34.574923+0000","last_active":"2023-10-06T10:40:35.555289+0000","last_peered":"2023-10-06T10:40:35.555289+0000","last_clean":"2023-10-06T10:40:35.555289+0000","last_became_active":"2023-10-06T10:40:34.574684+0000","last_became_peered":"2023-10-06T10:40:34.574684+0000","last_unstale":"2023-10-06T10:40:35.555289+0000","last_undegraded":"2023-10-06T10:40:35.555289+0000","last_fullsized":"2023-10-06T10:40:35.555289+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:01:38.560878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.570091+0000","last_change":"2023-10-06T10:40:35.567735+0000","last_active":"2023-10-06T10:40:35.570091+0000","last_peered":"2023-10-06T10:40:35.570091+0000","last_clean":"2023-10-06T10:40:35.570091+0000","last_became_active":"2023-10-06T10:40:35.566258+0000","last_became_peered":"2023-10-06T10:40:35.566258+0000","last_unstale":"2023-10-06T10:40:35.570091+0000","last_undegraded":"2023-10-06T10:40:35.570091+0000","last_fullsized":"2023-10-06T10:40:35.570091+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:31.294697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.562660+0000","last_change":"2023-10-06T10:40:34.558613+0000","last_active":"2023-10-06T10:40:35.562660+0000","last_peered":"2023-10-06T10:40:35.562660+0000","last_clean":"2023-10-06T10:40:35.562660+0000","last_became_active":"2023-10-06T10:40:34.558128+0000","last_became_peered":"2023-10-06T10:40:34.558128+0000","last_unstale":"2023-10-06T10:40:35.562660+0000","last_undegraded":"2023-10-06T10:40:35.562660+0000","last_fullsized":"2023-10-06T10:40:35.562660+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:46.155459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.574833+0000","last_change":"2023-10-06T10:40:35.570712+0000","last_active":"2023-10-06T10:40:35.574833+0000","last_peered":"2023-10-06T10:40:35.574833+0000","last_clean":"2023-10-06T10:40:35.574833+0000","last_became_active":"2023-10-06T10:40:35.570450+0000","last_became_peered":"2023-10-06T10:40:35.570450+0000","last_unstale":"2023-10-06T10:40:35.574833+0000","last_undegraded":"2023-10-06T10:40:35.574833+0000","last_fullsized":"2023-10-06T10:40:35.574833+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:40.960612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"51'3","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.571980+0000","last_change":"2023-10-06T10:40:34.567975+0000","last_active":"2023-10-06T10:40:35.571980+0000","last_peered":"2023-10-06T10:40:35.571980+0000","last_clean":"2023-10-06T10:40:35.571980+0000","last_became_active":"2023-10-06T10:40:34.567763+0000","last_became_peered":"2023-10-06T10:40:34.567763+0000","last_unstale":"2023-10-06T10:40:35.571980+0000","last_undegraded":"2023-10-06T10:40:35.571980+0000","last_fullsized":"2023-10-06T10:40:35.571980+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:26.380853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569050+0000","last_change":"2023-10-06T10:40:35.563961+0000","last_active":"2023-10-06T10:40:35.569050+0000","last_peered":"2023-10-06T10:40:35.569050+0000","last_clean":"2023-10-06T10:40:35.569050+0000","last_became_active":"2023-10-06T10:40:35.563735+0000","last_became_peered":"2023-10-06T10:40:35.563735+0000","last_unstale":"2023-10-06T10:40:35.569050+0000","last_undegraded":"2023-10-06T10:40:35.569050+0000","last_fullsized":"2023-10-06T10:40:35.569050+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:40:08.170068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"51'6","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.579117+0000","last_change":"2023-10-06T10:40:34.555718+0000","last_active":"2023-10-06T10:40:35.579117+0000","last_peered":"2023-10-06T10:40:35.579117+0000","last_clean":"2023-10-06T10:40:35.579117+0000","last_became_active":"2023-10-06T10:40:34.555357+0000","last_became_peered":"2023-10-06T10:40:34.555357+0000","last_unstale":"2023-10-06T10:40:35.579117+0000","last_undegraded":"2023-10-06T10:40:35.579117+0000","last_fullsized":"2023-10-06T10:40:35.579117+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:46.155459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.564769+0000","last_change":"2023-10-06T10:40:35.562704+0000","last_active":"2023-10-06T10:40:35.564769+0000","last_peered":"2023-10-06T10:40:35.564769+0000","last_clean":"2023-10-06T10:40:35.564769+0000","last_became_active":"2023-10-06T10:40:35.562424+0000","last_became_peered":"2023-10-06T10:40:35.562424+0000","last_unstale":"2023-10-06T10:40:35.564769+0000","last_undegraded":"2023-10-06T10:40:35.564769+0000","last_fullsized":"2023-10-06T10:40:35.564769+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:14.394504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"51'2","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.575856+0000","last_change":"2023-10-06T10:40:34.558595+0000","last_active":"2023-10-06T10:40:35.575856+0000","last_peered":"2023-10-06T10:40:35.575856+0000","last_clean":"2023-10-06T10:40:35.575856+0000","last_became_active":"2023-10-06T10:40:34.558324+0000","last_became_peered":"2023-10-06T10:40:34.558324+0000","last_unstale":"2023-10-06T10:40:35.575856+0000","last_undegraded":"2023-10-06T10:40:35.575856+0000","last_fullsized":"2023-10-06T10:40:35.575856+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.285921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.567395+0000","last_change":"2023-10-06T10:40:35.560500+0000","last_active":"2023-10-06T10:40:35.567395+0000","last_peered":"2023-10-06T10:40:35.567395+0000","last_clean":"2023-10-06T10:40:35.567395+0000","last_became_active":"2023-10-06T10:40:35.559739+0000","last_became_peered":"2023-10-06T10:40:35.559739+0000","last_unstale":"2023-10-06T10:40:35.567395+0000","last_undegraded":"2023-10-06T10:40:35.567395+0000","last_fullsized":"2023-10-06T10:40:35.567395+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:39.789216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.550323+0000","last_change":"2023-10-06T10:40:34.563002+0000","last_active":"2023-10-06T10:40:35.550323+0000","last_peered":"2023-10-06T10:40:35.550323+0000","last_clean":"2023-10-06T10:40:35.550323+0000","last_became_active":"2023-10-06T10:40:34.562670+0000","last_became_peered":"2023-10-06T10:40:34.562670+0000","last_unstale":"2023-10-06T10:40:35.550323+0000","last_undegraded":"2023-10-06T10:40:35.550323+0000","last_fullsized":"2023-10-06T10:40:35.550323+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:26.380853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569802+0000","last_change":"2023-10-06T10:40:35.565320+0000","last_active":"2023-10-06T10:40:35.569802+0000","last_peered":"2023-10-06T10:40:35.569802+0000","last_clean":"2023-10-06T10:40:35.569802+0000","last_became_active":"2023-10-06T10:40:35.565079+0000","last_became_peered":"2023-10-06T10:40:35.565079+0000","last_unstale":"2023-10-06T10:40:35.569802+0000","last_undegraded":"2023-10-06T10:40:35.569802+0000","last_fullsized":"2023-10-06T10:40:35.569802+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:14.394504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.546791+0000","last_change":"2023-10-06T10:40:34.558954+0000","last_active":"2023-10-06T10:40:35.546791+0000","last_peered":"2023-10-06T10:40:35.546791+0000","last_clean":"2023-10-06T10:40:35.546791+0000","last_became_active":"2023-10-06T10:40:34.558311+0000","last_became_peered":"2023-10-06T10:40:34.558311+0000","last_unstale":"2023-10-06T10:40:35.546791+0000","last_undegraded":"2023-10-06T10:40:35.546791+0000","last_fullsized":"2023-10-06T10:40:35.546791+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:03.396798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.577709+0000","last_change":"2023-10-06T10:40:35.577044+0000","last_active":"2023-10-06T10:40:35.577709+0000","last_peered":"2023-10-06T10:40:35.577709+0000","last_clean":"2023-10-06T10:40:35.577709+0000","last_became_active":"2023-10-06T10:40:35.576791+0000","last_became_peered":"2023-10-06T10:40:35.576791+0000","last_unstale":"2023-10-06T10:40:35.577709+0000","last_undegraded":"2023-10-06T10:40:35.577709+0000","last_fullsized":"2023-10-06T10:40:35.577709+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:55:43.725392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"51'8","reported_seq":18,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568288+0000","last_change":"2023-10-06T10:40:34.562481+0000","last_active":"2023-10-06T10:40:35.568288+0000","last_peered":"2023-10-06T10:40:35.568288+0000","last_clean":"2023-10-06T10:40:35.568288+0000","last_became_active":"2023-10-06T10:40:34.562181+0000","last_became_peered":"2023-10-06T10:40:34.562181+0000","last_unstale":"2023-10-06T10:40:35.568288+0000","last_undegraded":"2023-10-06T10:40:35.568288+0000","last_fullsized":"2023-10-06T10:40:35.568288+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:04:34.719623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.566714+0000","last_change":"2023-10-06T10:40:35.562814+0000","last_active":"2023-10-06T10:40:35.566714+0000","last_peered":"2023-10-06T10:40:35.566714+0000","last_clean":"2023-10-06T10:40:35.566714+0000","last_became_active":"2023-10-06T10:40:35.562556+0000","last_became_peered":"2023-10-06T10:40:35.562556+0000","last_unstale":"2023-10-06T10:40:35.566714+0000","last_undegraded":"2023-10-06T10:40:35.566714+0000","last_fullsized":"2023-10-06T10:40:35.566714+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:48.719683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"51'2","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576493+0000","last_change":"2023-10-06T10:40:34.561496+0000","last_active":"2023-10-06T10:40:35.576493+0000","last_peered":"2023-10-06T10:40:35.576493+0000","last_clean":"2023-10-06T10:40:35.576493+0000","last_became_active":"2023-10-06T10:40:34.561276+0000","last_became_peered":"2023-10-06T10:40:34.561276+0000","last_unstale":"2023-10-06T10:40:35.576493+0000","last_undegraded":"2023-10-06T10:40:35.576493+0000","last_fullsized":"2023-10-06T10:40:35.576493+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:40.903040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.566308+0000","last_change":"2023-10-06T10:40:35.558844+0000","last_active":"2023-10-06T10:40:35.566308+0000","last_peered":"2023-10-06T10:40:35.566308+0000","last_clean":"2023-10-06T10:40:35.566308+0000","last_became_active":"2023-10-06T10:40:35.558577+0000","last_became_peered":"2023-10-06T10:40:35.558577+0000","last_unstale":"2023-10-06T10:40:35.566308+0000","last_undegraded":"2023-10-06T10:40:35.566308+0000","last_fullsized":"2023-10-06T10:40:35.566308+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:04:09.361668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.553797+0000","last_change":"2023-10-06T10:40:34.557813+0000","last_active":"2023-10-06T10:40:35.553797+0000","last_peered":"2023-10-06T10:40:35.553797+0000","last_clean":"2023-10-06T10:40:35.553797+0000","last_became_active":"2023-10-06T10:40:34.557512+0000","last_became_peered":"2023-10-06T10:40:34.557512+0000","last_unstale":"2023-10-06T10:40:35.553797+0000","last_undegraded":"2023-10-06T10:40:35.553797+0000","last_fullsized":"2023-10-06T10:40:35.553797+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:38.780441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.574854+0000","last_change":"2023-10-06T10:40:35.570944+0000","last_active":"2023-10-06T10:40:35.574854+0000","last_peered":"2023-10-06T10:40:35.574854+0000","last_clean":"2023-10-06T10:40:35.574854+0000","last_became_active":"2023-10-06T10:40:35.567192+0000","last_became_peered":"2023-10-06T10:40:35.567192+0000","last_unstale":"2023-10-06T10:40:35.574854+0000","last_undegraded":"2023-10-06T10:40:35.574854+0000","last_fullsized":"2023-10-06T10:40:35.574854+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:26.729670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.550419+0000","last_change":"2023-10-06T10:40:34.559497+0000","last_active":"2023-10-06T10:40:35.550419+0000","last_peered":"2023-10-06T10:40:35.550419+0000","last_clean":"2023-10-06T10:40:35.550419+0000","last_became_active":"2023-10-06T10:40:34.559159+0000","last_became_peered":"2023-10-06T10:40:34.559159+0000","last_unstale":"2023-10-06T10:40:35.550419+0000","last_undegraded":"2023-10-06T10:40:35.550419+0000","last_fullsized":"2023-10-06T10:40:35.550419+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:04:34.719623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.567761+0000","last_change":"2023-10-06T10:40:35.557851+0000","last_active":"2023-10-06T10:40:35.567761+0000","last_peered":"2023-10-06T10:40:35.567761+0000","last_clean":"2023-10-06T10:40:35.567761+0000","last_became_active":"2023-10-06T10:40:35.557289+0000","last_became_peered":"2023-10-06T10:40:35.557289+0000","last_unstale":"2023-10-06T10:40:35.567761+0000","last_undegraded":"2023-10-06T10:40:35.567761+0000","last_fullsized":"2023-10-06T10:40:35.567761+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:57:07.950757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.548294+0000","last_change":"2023-10-06T10:40:34.558179+0000","last_active":"2023-10-06T10:40:35.548294+0000","last_peered":"2023-10-06T10:40:35.548294+0000","last_clean":"2023-10-06T10:40:35.548294+0000","last_became_active":"2023-10-06T10:40:34.557924+0000","last_became_peered":"2023-10-06T10:40:34.557924+0000","last_unstale":"2023-10-06T10:40:35.548294+0000","last_undegraded":"2023-10-06T10:40:35.548294+0000","last_fullsized":"2023-10-06T10:40:35.548294+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:13.047456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576117+0000","last_change":"2023-10-06T10:40:35.565513+0000","last_active":"2023-10-06T10:40:35.576117+0000","last_peered":"2023-10-06T10:40:35.576117+0000","last_clean":"2023-10-06T10:40:35.576117+0000","last_became_active":"2023-10-06T10:40:35.565263+0000","last_became_peered":"2023-10-06T10:40:35.565263+0000","last_unstale":"2023-10-06T10:40:35.576117+0000","last_undegraded":"2023-10-06T10:40:35.576117+0000","last_fullsized":"2023-10-06T10:40:35.576117+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:12:37.044769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.559092+0000","last_change":"2023-10-06T10:40:34.559598+0000","last_active":"2023-10-06T10:40:35.559092+0000","last_peered":"2023-10-06T10:40:35.559092+0000","last_clean":"2023-10-06T10:40:35.559092+0000","last_became_active":"2023-10-06T10:40:34.559338+0000","last_became_peered":"2023-10-06T10:40:34.559338+0000","last_unstale":"2023-10-06T10:40:35.559092+0000","last_undegraded":"2023-10-06T10:40:35.559092+0000","last_fullsized":"2023-10-06T10:40:35.559092+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:10.979999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.577733+0000","last_change":"2023-10-06T10:40:35.577132+0000","last_active":"2023-10-06T10:40:35.577733+0000","last_peered":"2023-10-06T10:40:35.577733+0000","last_clean":"2023-10-06T10:40:35.577733+0000","last_became_active":"2023-10-06T10:40:35.576860+0000","last_became_peered":"2023-10-06T10:40:35.576860+0000","last_unstale":"2023-10-06T10:40:35.577733+0000","last_undegraded":"2023-10-06T10:40:35.577733+0000","last_fullsized":"2023-10-06T10:40:35.577733+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:16.495299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.548028+0000","last_change":"2023-10-06T10:40:34.555634+0000","last_active":"2023-10-06T10:40:35.548028+0000","last_peered":"2023-10-06T10:40:35.548028+0000","last_clean":"2023-10-06T10:40:35.548028+0000","last_became_active":"2023-10-06T10:40:34.555335+0000","last_became_peered":"2023-10-06T10:40:34.555335+0000","last_unstale":"2023-10-06T10:40:35.548028+0000","last_undegraded":"2023-10-06T10:40:35.548028+0000","last_fullsized":"2023-10-06T10:40:35.548028+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:55:42.716617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.574756+0000","last_change":"2023-10-06T10:40:35.566701+0000","last_active":"2023-10-06T10:40:35.574756+0000","last_peered":"2023-10-06T10:40:35.574756+0000","last_clean":"2023-10-06T10:40:35.574756+0000","last_became_active":"2023-10-06T10:40:35.566383+0000","last_became_peered":"2023-10-06T10:40:35.566383+0000","last_unstale":"2023-10-06T10:40:35.574756+0000","last_undegraded":"2023-10-06T10:40:35.574756+0000","last_fullsized":"2023-10-06T10:40:35.574756+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:01:39.569654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.559099+0000","last_change":"2023-10-06T10:40:34.558852+0000","last_active":"2023-10-06T10:40:35.559099+0000","last_peered":"2023-10-06T10:40:35.559099+0000","last_clean":"2023-10-06T10:40:35.559099+0000","last_became_active":"2023-10-06T10:40:34.558581+0000","last_became_peered":"2023-10-06T10:40:34.558581+0000","last_unstale":"2023-10-06T10:40:35.559099+0000","last_undegraded":"2023-10-06T10:40:35.559099+0000","last_fullsized":"2023-10-06T10:40:35.559099+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:49.102558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576292+0000","last_change":"2023-10-06T10:40:35.567776+0000","last_active":"2023-10-06T10:40:35.576292+0000","last_peered":"2023-10-06T10:40:35.576292+0000","last_clean":"2023-10-06T10:40:35.576292+0000","last_became_active":"2023-10-06T10:40:35.567440+0000","last_became_peered":"2023-10-06T10:40:35.567440+0000","last_unstale":"2023-10-06T10:40:35.576292+0000","last_undegraded":"2023-10-06T10:40:35.576292+0000","last_fullsized":"2023-10-06T10:40:35.576292+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:15.268020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.556015+0000","last_change":"2023-10-06T10:40:34.559435+0000","last_active":"2023-10-06T10:40:35.556015+0000","last_peered":"2023-10-06T10:40:35.556015+0000","last_clean":"2023-10-06T10:40:35.556015+0000","last_became_active":"2023-10-06T10:40:34.559184+0000","last_became_peered":"2023-10-06T10:40:34.559184+0000","last_unstale":"2023-10-06T10:40:35.556015+0000","last_undegraded":"2023-10-06T10:40:35.556015+0000","last_fullsized":"2023-10-06T10:40:35.556015+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:28:37.920340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.573009+0000","last_change":"2023-10-06T10:40:35.572131+0000","last_active":"2023-10-06T10:40:35.573009+0000","last_peered":"2023-10-06T10:40:35.573009+0000","last_clean":"2023-10-06T10:40:35.573009+0000","last_became_active":"2023-10-06T10:40:35.571859+0000","last_became_peered":"2023-10-06T10:40:35.571859+0000","last_unstale":"2023-10-06T10:40:35.573009+0000","last_undegraded":"2023-10-06T10:40:35.573009+0000","last_fullsized":"2023-10-06T10:40:35.573009+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:45.782226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.550794+0000","last_change":"2023-10-06T10:40:34.561843+0000","last_active":"2023-10-06T10:40:35.550794+0000","last_peered":"2023-10-06T10:40:35.550794+0000","last_clean":"2023-10-06T10:40:35.550794+0000","last_became_active":"2023-10-06T10:40:34.561550+0000","last_became_peered":"2023-10-06T10:40:34.561550+0000","last_unstale":"2023-10-06T10:40:35.550794+0000","last_undegraded":"2023-10-06T10:40:35.550794+0000","last_fullsized":"2023-10-06T10:40:35.550794+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:14.711366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":43,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.547102+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:40:35.547102+0000","last_peered":"2023-10-06T10:40:35.547102+0000","last_clean":"2023-10-06T10:40:35.547102+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:40:35.547102+0000","last_undegraded":"2023-10-06T10:40:35.547102+0000","last_fullsized":"2023-10-06T10:40:35.547102+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576189+0000","last_change":"2023-10-06T10:40:35.566096+0000","last_active":"2023-10-06T10:40:35.576189+0000","last_peered":"2023-10-06T10:40:35.576189+0000","last_clean":"2023-10-06T10:40:35.576189+0000","last_became_active":"2023-10-06T10:40:35.565831+0000","last_became_peered":"2023-10-06T10:40:35.565831+0000","last_unstale":"2023-10-06T10:40:35.576189+0000","last_undegraded":"2023-10-06T10:40:35.576189+0000","last_fullsized":"2023-10-06T10:40:35.576189+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:35.748566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"51'1","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.556108+0000","last_change":"2023-10-06T10:40:34.562301+0000","last_active":"2023-10-06T10:40:35.556108+0000","last_peered":"2023-10-06T10:40:35.556108+0000","last_clean":"2023-10-06T10:40:35.556108+0000","last_became_active":"2023-10-06T10:40:34.562030+0000","last_became_peered":"2023-10-06T10:40:34.562030+0000","last_unstale":"2023-10-06T10:40:35.556108+0000","last_undegraded":"2023-10-06T10:40:35.556108+0000","last_fullsized":"2023-10-06T10:40:35.556108+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:43:52.519978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.563486+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:40:35.563486+0000","last_peered":"2023-10-06T10:40:35.563486+0000","last_clean":"2023-10-06T10:40:35.563486+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:40:35.563486+0000","last_undegraded":"2023-10-06T10:40:35.563486+0000","last_fullsized":"2023-10-06T10:40:35.563486+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568740+0000","last_change":"2023-10-06T10:40:35.561940+0000","last_active":"2023-10-06T10:40:35.568740+0000","last_peered":"2023-10-06T10:40:35.568740+0000","last_clean":"2023-10-06T10:40:35.568740+0000","last_became_active":"2023-10-06T10:40:35.561665+0000","last_became_peered":"2023-10-06T10:40:35.561665+0000","last_unstale":"2023-10-06T10:40:35.568740+0000","last_undegraded":"2023-10-06T10:40:35.568740+0000","last_fullsized":"2023-10-06T10:40:35.568740+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:33:37.853079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.559769+0000","last_change":"2023-10-06T10:40:34.562242+0000","last_active":"2023-10-06T10:40:35.559769+0000","last_peered":"2023-10-06T10:40:35.559769+0000","last_clean":"2023-10-06T10:40:35.559769+0000","last_became_active":"2023-10-06T10:40:34.562001+0000","last_became_peered":"2023-10-06T10:40:34.562001+0000","last_unstale":"2023-10-06T10:40:35.559769+0000","last_undegraded":"2023-10-06T10:40:35.559769+0000","last_fullsized":"2023-10-06T10:40:35.559769+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:48.015572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":41,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.559170+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:40:35.559170+0000","last_peered":"2023-10-06T10:40:35.559170+0000","last_clean":"2023-10-06T10:40:35.559170+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:40:35.559170+0000","last_undegraded":"2023-10-06T10:40:35.559170+0000","last_fullsized":"2023-10-06T10:40:35.559170+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":186,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.553015+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:40:35.553015+0000","last_peered":"2023-10-06T10:40:35.553015+0000","last_clean":"2023-10-06T10:40:35.553015+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:40:35.553015+0000","last_undegraded":"2023-10-06T10:40:35.553015+0000","last_fullsized":"2023-10-06T10:40:35.553015+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.564750+0000","last_change":"2023-10-06T10:40:35.561025+0000","last_active":"2023-10-06T10:40:35.564750+0000","last_peered":"2023-10-06T10:40:35.564750+0000","last_clean":"2023-10-06T10:40:35.564750+0000","last_became_active":"2023-10-06T10:40:35.560733+0000","last_became_peered":"2023-10-06T10:40:35.560733+0000","last_unstale":"2023-10-06T10:40:35.564750+0000","last_undegraded":"2023-10-06T10:40:35.564750+0000","last_fullsized":"2023-10-06T10:40:35.564750+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:15.757865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"51'2","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.575749+0000","last_change":"2023-10-06T10:40:34.550122+0000","last_active":"2023-10-06T10:40:35.575749+0000","last_peered":"2023-10-06T10:40:35.575749+0000","last_clean":"2023-10-06T10:40:35.575749+0000","last_became_active":"2023-10-06T10:40:34.549859+0000","last_became_peered":"2023-10-06T10:40:34.549859+0000","last_unstale":"2023-10-06T10:40:35.575749+0000","last_undegraded":"2023-10-06T10:40:35.575749+0000","last_fullsized":"2023-10-06T10:40:35.575749+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:44.773451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":36,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.553171+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:40:35.553171+0000","last_peered":"2023-10-06T10:40:35.553171+0000","last_clean":"2023-10-06T10:40:35.553171+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:40:35.553171+0000","last_undegraded":"2023-10-06T10:40:35.553171+0000","last_fullsized":"2023-10-06T10:40:35.553171+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576107+0000","last_change":"2023-10-06T10:40:35.565493+0000","last_active":"2023-10-06T10:40:35.576107+0000","last_peered":"2023-10-06T10:40:35.576107+0000","last_clean":"2023-10-06T10:40:35.576107+0000","last_became_active":"2023-10-06T10:40:35.565234+0000","last_became_peered":"2023-10-06T10:40:35.565234+0000","last_unstale":"2023-10-06T10:40:35.576107+0000","last_undegraded":"2023-10-06T10:40:35.576107+0000","last_fullsized":"2023-10-06T10:40:35.576107+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:48.719683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.550113+0000","last_change":"2023-10-06T10:40:34.559498+0000","last_active":"2023-10-06T10:40:35.550113+0000","last_peered":"2023-10-06T10:40:35.550113+0000","last_clean":"2023-10-06T10:40:35.550113+0000","last_became_active":"2023-10-06T10:40:34.559262+0000","last_became_peered":"2023-10-06T10:40:34.559262+0000","last_unstale":"2023-10-06T10:40:35.550113+0000","last_undegraded":"2023-10-06T10:40:35.550113+0000","last_fullsized":"2023-10-06T10:40:35.550113+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:27:47.178791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":36,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.553009+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:40:35.553009+0000","last_peered":"2023-10-06T10:40:35.553009+0000","last_clean":"2023-10-06T10:40:35.553009+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:40:35.553009+0000","last_undegraded":"2023-10-06T10:40:35.553009+0000","last_fullsized":"2023-10-06T10:40:35.553009+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568628+0000","last_change":"2023-10-06T10:40:35.560232+0000","last_active":"2023-10-06T10:40:35.568628+0000","last_peered":"2023-10-06T10:40:35.568628+0000","last_clean":"2023-10-06T10:40:35.568628+0000","last_became_active":"2023-10-06T10:40:35.559942+0000","last_became_peered":"2023-10-06T10:40:35.559942+0000","last_unstale":"2023-10-06T10:40:35.568628+0000","last_undegraded":"2023-10-06T10:40:35.568628+0000","last_fullsized":"2023-10-06T10:40:35.568628+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:03.785102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.547498+0000","last_change":"2023-10-06T10:40:34.564639+0000","last_active":"2023-10-06T10:40:35.547498+0000","last_peered":"2023-10-06T10:40:35.547498+0000","last_clean":"2023-10-06T10:40:35.547498+0000","last_became_active":"2023-10-06T10:40:34.564425+0000","last_became_peered":"2023-10-06T10:40:34.564425+0000","last_unstale":"2023-10-06T10:40:35.547498+0000","last_undegraded":"2023-10-06T10:40:35.547498+0000","last_fullsized":"2023-10-06T10:40:35.547498+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:43:17.508745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":45,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.546832+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:40:35.546832+0000","last_peered":"2023-10-06T10:40:35.546832+0000","last_clean":"2023-10-06T10:40:35.546832+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:40:35.546832+0000","last_undegraded":"2023-10-06T10:40:35.546832+0000","last_fullsized":"2023-10-06T10:40:35.546832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569316+0000","last_change":"2023-10-06T10:40:35.565535+0000","last_active":"2023-10-06T10:40:35.569316+0000","last_peered":"2023-10-06T10:40:35.569316+0000","last_clean":"2023-10-06T10:40:35.569316+0000","last_became_active":"2023-10-06T10:40:35.565269+0000","last_became_peered":"2023-10-06T10:40:35.565269+0000","last_unstale":"2023-10-06T10:40:35.569316+0000","last_undegraded":"2023-10-06T10:40:35.569316+0000","last_fullsized":"2023-10-06T10:40:35.569316+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:15.720141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"51'1","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576121+0000","last_change":"2023-10-06T10:40:34.558460+0000","last_active":"2023-10-06T10:40:35.576121+0000","last_peered":"2023-10-06T10:40:35.576121+0000","last_clean":"2023-10-06T10:40:35.576121+0000","last_became_active":"2023-10-06T10:40:34.558193+0000","last_became_peered":"2023-10-06T10:40:34.558193+0000","last_unstale":"2023-10-06T10:40:35.576121+0000","last_undegraded":"2023-10-06T10:40:35.576121+0000","last_fullsized":"2023-10-06T10:40:35.576121+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:55:42.716617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.549822+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:40:35.549822+0000","last_peered":"2023-10-06T10:40:35.549822+0000","last_clean":"2023-10-06T10:40:35.549822+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:40:35.549822+0000","last_undegraded":"2023-10-06T10:40:35.549822+0000","last_fullsized":"2023-10-06T10:40:35.549822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569590+0000","last_change":"2023-10-06T10:40:35.566476+0000","last_active":"2023-10-06T10:40:35.569590+0000","last_peered":"2023-10-06T10:40:35.569590+0000","last_clean":"2023-10-06T10:40:35.569590+0000","last_became_active":"2023-10-06T10:40:35.566047+0000","last_became_peered":"2023-10-06T10:40:35.566047+0000","last_unstale":"2023-10-06T10:40:35.569590+0000","last_undegraded":"2023-10-06T10:40:35.569590+0000","last_fullsized":"2023-10-06T10:40:35.569590+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:57:07.950757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"51'2","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.564527+0000","last_change":"2023-10-06T10:40:34.557155+0000","last_active":"2023-10-06T10:40:35.564527+0000","last_peered":"2023-10-06T10:40:35.564527+0000","last_clean":"2023-10-06T10:40:35.564527+0000","last_became_active":"2023-10-06T10:40:34.556935+0000","last_became_peered":"2023-10-06T10:40:34.556935+0000","last_unstale":"2023-10-06T10:40:35.564527+0000","last_undegraded":"2023-10-06T10:40:35.564527+0000","last_fullsized":"2023-10-06T10:40:35.564527+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.285921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.550394+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:40:35.550394+0000","last_peered":"2023-10-06T10:40:35.550394+0000","last_clean":"2023-10-06T10:40:35.550394+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:40:35.550394+0000","last_undegraded":"2023-10-06T10:40:35.550394+0000","last_fullsized":"2023-10-06T10:40:35.550394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298480,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073360,"statfs":{"total":96636764160,"available":96331120640,"internally_reserved":0,"allocated":1220608,"data_stored":877891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.48099999999999998,"15min":0.47799999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71699999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.40500000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78600000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.54100000000000004}]},{"osd":1,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.46600000000000003,"15min":0.45700000000000002},"min":{"1min":0.375,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65900000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.5},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.505,"15min":0.5},"min":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72399999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.79100000000000004}]},{"osd":2,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.40600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":1.161},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.496,"15min":0.49199999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69199999999999995,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.747}]},{"osd":3,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54800000000000004,"15min":0.54200000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.66300000000000003}]},{"osd":4,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.46800000000000003,"15min":0.47699999999999998},"min":{"1min":0.30499999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57699999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.501,"15min":0.50900000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":1.0820000000000001}]},{"osd":5,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.32700000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65900000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.34300000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66800000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":1.052}]},{"osd":6,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.56799999999999995,"15min":0.57699999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.75700000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.747,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":1.129}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297888,"kb_used_data":600,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073952,"statfs":{"total":96636764160,"available":96331726848,"internally_reserved":0,"allocated":614400,"data_stored":286010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.49399999999999999,"15min":0.48899999999999999},"min":{"1min":0.36299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.44500000000000001,"15min":0.42899999999999999},"min":{"1min":0.38800000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.45100000000000001,"15min":0.437},"min":{"1min":0.33100000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.64300000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.379,"5min":0.309,"15min":0.309},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.47699999999999998,"15min":0.46400000000000002},"min":{"1min":0.34999999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.54400000000000004}]},{"osd":3,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.497,"15min":0.48799999999999999},"min":{"1min":0.38800000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.497},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.61399999999999999}]},{"osd":4,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47899999999999998,"15min":0.47799999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70899999999999996,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.65400000000000003}]},{"osd":5,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.48999999999999999,"15min":0.48599999999999999},"min":{"1min":0.28899999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69299999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33900000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.68899999999999995}]},{"osd":7,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.754,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.63}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298476,"kb_used_data":1188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073364,"statfs":{"total":96636764160,"available":96331124736,"internally_reserved":0,"allocated":1216512,"data_stored":877821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33000000000000002,"15min":0.32800000000000001},"min":{"1min":0.253,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.374,"5min":0.34999999999999998,"15min":0.34599999999999997},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.53900000000000003,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.29299999999999998}]},{"osd":2,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.32500000000000001,"15min":0.31900000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"last":0.31},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.36299999999999999},"min":{"1min":0.28999999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.36899999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.375},"min":{"1min":0.30099999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.379,"15min":0.375},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.34499999999999997}]},{"osd":4,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.40300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.69499999999999995,"5min":0.746,"15min":0.746},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.44700000000000001}]},{"osd":5,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54800000000000004,"15min":0.54100000000000004},"min":{"1min":0.436,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.39800000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.40100000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.433},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.47899999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.45600000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58099999999999996,"15min":0.57399999999999995},"min":{"1min":0.48699999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.439,"5min":0.436,"15min":0.436},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.38700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297908,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073932,"statfs":{"total":96636764160,"available":96331706368,"internally_reserved":0,"allocated":634880,"data_stored":289453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.32600000000000001,"15min":0.31900000000000001},"min":{"1min":0.23200000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.34899999999999998,"15min":0.34100000000000003},"min":{"1min":0.27600000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.45300000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.312,"15min":0.30599999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.371},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.33300000000000002,"15min":0.32500000000000001},"min":{"1min":0.26500000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.52000000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.41799999999999998}]},{"osd":3,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.35099999999999998,"15min":0.34300000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.35499999999999998,"15min":0.34899999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.436}]},{"osd":4,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.42599999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81200000000000006,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.54200000000000004,"15min":0.53300000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.48199999999999998}]},{"osd":5,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.41799999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.91900000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.497},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.80900000000000005,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.59199999999999997}]},{"osd":6,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58699999999999997,"15min":0.58799999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77300000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86399999999999999,"5min":1.075,"15min":1.075},"last":0.60299999999999998}]},{"osd":7,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79700000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.61399999999999999,"15min":0.60999999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88800000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297900,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073940,"statfs":{"total":96636764160,"available":96331714560,"internally_reserved":0,"allocated":626688,"data_stored":287419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.313,"15min":0.311},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.32500000000000001,"15min":0.317},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.33700000000000002}]},{"osd":1,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.32900000000000001,"15min":0.32300000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.54200000000000004,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.36699999999999999,"15min":0.36499999999999999},"min":{"1min":0.26400000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59299999999999997,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.30399999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.34899999999999998,"15min":0.34499999999999997},"min":{"1min":0.252,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.52700000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.35799999999999998,"15min":0.35499999999999998},"min":{"1min":0.27600000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.39900000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.35499999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.76100000000000001,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82699999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.42799999999999999}]},{"osd":5,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.436},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.377,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81599999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.46000000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.625,"15min":0.624},"min":{"1min":0.45800000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89000000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.624,"15min":0.623},"min":{"1min":0.433,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53300000000000003}]},{"osd":7,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.44500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79100000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.498,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.46999999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297900,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073940,"statfs":{"total":96636764160,"available":96331714560,"internally_reserved":0,"allocated":626688,"data_stored":287419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.438,"15min":0.45600000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.50600000000000001,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.36799999999999999,"15min":0.36399999999999999},"min":{"1min":0.27100000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66800000000000004,"5min":1.218,"15min":1.218},"last":0.34699999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34100000000000003,"15min":0.34000000000000002},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.61899999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.48299999999999998,"15min":0.495},"min":{"1min":0.27800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60299999999999998,"5min":3.359,"15min":3.359},"last":0.36099999999999999}]},{"osd":2,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38500000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.57699999999999996,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.315},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.63700000000000001,"15min":0.67300000000000004},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57699999999999996,"5min":7.069,"15min":7.069},"last":0.43099999999999999}]},{"osd":4,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89400000000000002,"5min":1.165,"15min":1.165},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59899999999999998,"15min":0.59699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79400000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.50700000000000001}]},{"osd":5,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.69799999999999995,"15min":0.71399999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92500000000000004,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60799999999999998,"15min":0.61099999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.96899999999999997,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.48299999999999998}]},{"osd":6,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.38,"5min":0.378,"15min":0.378},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.72699999999999998,"15min":0.74199999999999999},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84699999999999998,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.51600000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.61099999999999999},"min":{"1min":0.41299999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.99099999999999999,"5min":1.137,"15min":1.137},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.443,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91400000000000003,"5min":1.014,"15min":1.014},"last":0.54800000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297904,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073936,"statfs":{"total":96636764160,"available":96331710464,"internally_reserved":0,"allocated":630784,"data_stored":288899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47999999999999998,"15min":0.48099999999999998},"min":{"1min":0.33200000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.60799999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.496},"min":{"1min":0.33100000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.67000000000000004,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.60799999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.495,"15min":0.496},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.495,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.59399999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50700000000000001,"15min":0.50800000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.63,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36099999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.68500000000000005,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.56799999999999995}]},{"osd":3,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51600000000000001,"15min":0.51200000000000001},"min":{"1min":0.42199999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.625,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.53000000000000003,"15min":0.53500000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.64800000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64800000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.495,"15min":0.50800000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.61599999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.49399999999999999,"15min":0.502},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.63300000000000001,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.47899999999999998}]},{"osd":6,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.53600000000000003,"15min":0.54600000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.68799999999999994,"5min":0.75,"15min":0.75},"last":0.505},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.67700000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.46500000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.56899999999999995,"15min":0.57999999999999996},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70499999999999996,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72499999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.42899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297900,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073940,"statfs":{"total":96636764160,"available":96331714560,"internally_reserved":0,"allocated":626688,"data_stored":287987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.47399999999999998,"15min":0.46300000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54000000000000004,"15min":0.53400000000000003},"min":{"1min":0.35899999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.748,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.86499999999999999}]},{"osd":1,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.496,"15min":0.48499999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.52500000000000002,"15min":0.51700000000000002},"min":{"1min":0.39200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.80900000000000005}]},{"osd":2,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.51000000000000001,"15min":0.499},"min":{"1min":0.32000000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.41499999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.754,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.76300000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.57799999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77700000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78500000000000003,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.79100000000000004}]},{"osd":4,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.32100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.69599999999999995,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.501},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.66100000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.42299999999999999}]},{"osd":6,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.67500000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63400000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68799999999999994,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76700000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.55300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:40:40.363 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:40:40.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.508+0000 7faedc9f3700 1 Processor -- start 2023-10-06T10:40:40.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.508+0000 7faedc9f3700 1 -- start start 2023-10-06T10:40:40.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.509+0000 7faedc9f3700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faed80696e0 0x7faed8069ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:40.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.509+0000 7faedc9f3700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7faed80ada60 0x7faed806d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:40.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.509+0000 7faedc9f3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed806d890 con 0x7faed80aeeb0 2023-10-06T10:40:40.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.509+0000 7faedc9f3700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed80cd000 con 0x7faed80ada60 2023-10-06T10:40:40.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.509+0000 7faedc9f3700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed80aded0 con 0x7faed80696e0 2023-10-06T10:40:40.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.509+0000 7faed6d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7faed80aeeb0 0x7faed806eb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50332/0 (socket says 172.21.15.92:50332) 2023-10-06T10:40:40.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.509+0000 7faed6d9d700 1 -- 172.21.15.92:0/2220748686 learned_addr learned my addr 172.21.15.92:0/2220748686 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:40.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.509+0000 7faed659c700 1 --2- 172.21.15.92:0/2220748686 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faed80696e0 0x7faed8069ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:40.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.509+0000 7faed559a700 1 -- 172.21.15.92:0/2220748686 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266422320 0 0) 0x7faed806d890 con 0x7faed80aeeb0 2023-10-06T10:40:40.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.509+0000 7faed5d9b700 1 --2- 172.21.15.92:0/2220748686 >> v2:172.21.15.124:3300/0 conn(0x7faed80ada60 0x7faed806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:40.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.509+0000 7faed559a700 1 -- 172.21.15.92:0/2220748686 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faec0003570 con 0x7faed80aeeb0 2023-10-06T10:40:40.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.509+0000 7faed559a700 1 -- 172.21.15.92:0/2220748686 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 205281131 0 0) 0x7faec0003570 con 0x7faed80aeeb0 2023-10-06T10:40:40.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.509+0000 7faed559a700 1 -- 172.21.15.92:0/2220748686 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faed80696e0 msgr2=0x7faed8069ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:40.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.509+0000 7faed559a700 1 --2- 172.21.15.92:0/2220748686 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faed80696e0 0x7faed8069ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.509+0000 7faed559a700 1 -- 172.21.15.92:0/2220748686 >> v2:172.21.15.124:3300/0 conn(0x7faed80ada60 msgr2=0x7faed806d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:40.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.509+0000 7faed559a700 1 --2- 172.21.15.92:0/2220748686 >> v2:172.21.15.124:3300/0 conn(0x7faed80ada60 0x7faed806d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.509+0000 7faed559a700 1 -- 172.21.15.92:0/2220748686 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed806e6d0 con 0x7faed80aeeb0 2023-10-06T10:40:40.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.509+0000 7faed559a700 1 -- 172.21.15.92:0/2220748686 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7faed00017f0 con 0x7faed80aeeb0 2023-10-06T10:40:40.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.510+0000 7faed559a700 1 -- 172.21.15.92:0/2220748686 >> v1:172.21.15.92:6789/0 conn(0x7faed80aeeb0 legacy=0x7faed806eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:40.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.510+0000 7faed559a700 1 --2- 172.21.15.92:0/2220748686 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faec0003a60 0x7faec0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:40.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.510+0000 7faed559a700 1 --2- 172.21.15.92:0/2220748686 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faed80696e0 0x7faec0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:40.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.510+0000 7faed559a700 1 --2- 172.21.15.92:0/2220748686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faec0004d50 0x7faec00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:40.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.510+0000 7faed559a700 1 -- 172.21.15.92:0/2220748686 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed80cd000 con 0x7faec0004d50 2023-10-06T10:40:40.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.510+0000 7faed559a700 1 -- 172.21.15.92:0/2220748686 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec000ba60 con 0x7faec0003a60 2023-10-06T10:40:40.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.510+0000 7faed559a700 1 -- 172.21.15.92:0/2220748686 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec000bba0 con 0x7faed80696e0 2023-10-06T10:40:40.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.510+0000 7faed6d9d700 1 --2- 172.21.15.92:0/2220748686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faec0004d50 0x7faec00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:40.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.510+0000 7faed659c700 1 --2- 172.21.15.92:0/2220748686 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faec0003a60 0x7faec0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:40.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.510+0000 7faed5d9b700 1 --2- 172.21.15.92:0/2220748686 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faed80696e0 0x7faec0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:40.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.510+0000 7faed6d9d700 1 -- 172.21.15.92:0/2220748686 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faed80696e0 msgr2=0x7faec0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:40.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.510+0000 7faed6d9d700 1 --2- 172.21.15.92:0/2220748686 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faed80696e0 0x7faec0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.510+0000 7faed6d9d700 1 -- 172.21.15.92:0/2220748686 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faec0003a60 msgr2=0x7faec0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:40.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.510+0000 7faed6d9d700 1 --2- 172.21.15.92:0/2220748686 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faec0003a60 0x7faec0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.510+0000 7faed6d9d700 1 -- 172.21.15.92:0/2220748686 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec0003570 con 0x7faec0004d50 2023-10-06T10:40:40.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.510+0000 7faed6d9d700 1 --2- 172.21.15.92:0/2220748686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faec0004d50 0x7faec00051c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7faed000a970 tx=0x7faed0002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f40a8b2ff06bb3a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:40.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.510+0000 7faed559a700 1 -- 172.21.15.92:0/2220748686 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed0012440 con 0x7faec0004d50 2023-10-06T10:40:40.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.510+0000 7faed559a700 1 -- 172.21.15.92:0/2220748686 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed00125c0 con 0x7faec0004d50 2023-10-06T10:40:40.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.510+0000 7faed559a700 1 -- 172.21.15.92:0/2220748686 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed0012980 con 0x7faec0004d50 2023-10-06T10:40:40.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.511+0000 7faedc9f3700 1 -- 172.21.15.92:0/2220748686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faec0004d50 msgr2=0x7faec00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:40.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.511+0000 7faedc9f3700 1 --2- 172.21.15.92:0/2220748686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faec0004d50 0x7faec00051c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7faed000a970 tx=0x7faed0002210 comp rx=0 tx=0).stop 2023-10-06T10:40:40.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.511+0000 7faed6d9d700 1 -- 172.21.15.92:0/2220748686 reap_dead start 2023-10-06T10:40:40.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.511+0000 7faedc9f3700 1 -- 172.21.15.92:0/2220748686 shutdown_connections 2023-10-06T10:40:40.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.511+0000 7faedc9f3700 1 -- 172.21.15.92:0/2220748686 >> 172.21.15.92:0/2220748686 conn(0x7faed8068080 msgr2=0x7faed80cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:40.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.511+0000 7faedc9f3700 1 -- 172.21.15.92:0/2220748686 shutdown_connections 2023-10-06T10:40:40.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.511+0000 7faedc9f3700 1 -- 172.21.15.92:0/2220748686 wait complete. 2023-10-06T10:40:40.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.512+0000 7faedc9f3700 1 Processor -- start 2023-10-06T10:40:40.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.512+0000 7faedc9f3700 1 -- start start 2023-10-06T10:40:40.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.513+0000 7faedc9f3700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faec0004d50 0x7faed814f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:40.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.513+0000 7faedc9f3700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faed81584c0 0x7faed8158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:40.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.513+0000 7faedc9f3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faed80c7a10 0x7faed80c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:40.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.513+0000 7faedc9f3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed81514e0 con 0x7faed80c7a10 2023-10-06T10:40:40.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.513+0000 7faedc9f3700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed8158e70 con 0x7faed81584c0 2023-10-06T10:40:40.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.513+0000 7faedc9f3700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed80aa5f0 con 0x7faec0004d50 2023-10-06T10:40:40.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.513+0000 7faed6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faed80c7a10 0x7faed80c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:40.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.513+0000 7faed6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faed80c7a10 0x7faed80c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60728/0 (socket says 172.21.15.92:60728) 2023-10-06T10:40:40.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.513+0000 7faed6d9d700 1 -- 172.21.15.92:0/2923563554 learned_addr learned my addr 172.21.15.92:0/2923563554 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.513+0000 7faed659c700 1 --2- 172.21.15.92:0/2923563554 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faec0004d50 0x7faed814f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.513+0000 7faed5d9b700 1 --2- 172.21.15.92:0/2923563554 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faed81584c0 0x7faed8158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.513+0000 7faed6d9d700 1 -- 172.21.15.92:0/2923563554 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faec0004d50 msgr2=0x7faed814f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.513+0000 7faed6d9d700 1 --2- 172.21.15.92:0/2923563554 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faec0004d50 0x7faed814f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.513+0000 7faed6d9d700 1 -- 172.21.15.92:0/2923563554 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faed81584c0 msgr2=0x7faed8158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.513+0000 7faed6d9d700 1 --2- 172.21.15.92:0/2923563554 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faed81584c0 0x7faed8158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.513+0000 7faed6d9d700 1 -- 172.21.15.92:0/2923563554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed814fb90 con 0x7faed80c7a10 2023-10-06T10:40:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.513+0000 7faed6d9d700 1 --2- 172.21.15.92:0/2923563554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faed80c7a10 0x7faed80c7e80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7faec0004730 tx=0x7faed0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.513+0000 7faec67fc700 1 -- 172.21.15.92:0/2923563554 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed0012440 con 0x7faed80c7a10 2023-10-06T10:40:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.513+0000 7faec67fc700 1 -- 172.21.15.92:0/2923563554 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed00017b0 con 0x7faed80c7a10 2023-10-06T10:40:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.513+0000 7faedc9f3700 1 -- 172.21.15.92:0/2923563554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed80c83c0 con 0x7faed80c7a10 2023-10-06T10:40:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.513+0000 7faec67fc700 1 -- 172.21.15.92:0/2923563554 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed0001f30 con 0x7faed80c7a10 2023-10-06T10:40:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.514+0000 7faedc9f3700 1 -- 172.21.15.92:0/2923563554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed80a9cc0 con 0x7faed80c7a10 2023-10-06T10:40:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.514+0000 7faedc9f3700 1 -- 172.21.15.92:0/2923563554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed8068df0 con 0x7faed80c7a10 2023-10-06T10:40:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.515+0000 7faec67fc700 1 -- 172.21.15.92:0/2923563554 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faed0019070 con 0x7faed80c7a10 2023-10-06T10:40:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.515+0000 7faec67fc700 1 --2- 172.21.15.92:0/2923563554 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7faeb8066f30 0x7faeb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.515+0000 7faec67fc700 1 -- 172.21.15.92:0/2923563554 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7faed0086210 con 0x7faed80c7a10 2023-10-06T10:40:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.515+0000 7faed659c700 1 --2- 172.21.15.92:0/2923563554 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7faeb8066f30 0x7faeb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.516+0000 7faed659c700 1 --2- 172.21.15.92:0/2923563554 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7faeb8066f30 0x7faeb80693e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faec8006f00 tx=0x7faec80091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.519+0000 7faec67fc700 1 -- 172.21.15.92:0/2923563554 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faed00531e0 con 0x7faed80c7a10 2023-10-06T10:40:40.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.780+0000 7faedc9f3700 1 -- 172.21.15.92:0/2923563554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7faed8064a90 con 0x7faed80c7a10 2023-10-06T10:40:40.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.780+0000 7faec67fc700 1 -- 172.21.15.92:0/2923563554 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1017 (secure 0 0 0) 0x7faed0056800 con 0x7faed80c7a10 2023-10-06T10:40:40.782 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:40.782 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":162,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":52,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":23,"data_bytes":592674,"bytes_used":2441580544,"bytes_avail":770652532736,"bytes_total":773094113280,"write_bytes_sec":2218,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":32,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5736}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2023-10-06T10:40:24.110089+0000","services":{}},"progress_events":{}} 2023-10-06T10:40:40.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.782+0000 7faedc9f3700 1 -- 172.21.15.92:0/2923563554 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7faeb8066f30 msgr2=0x7faeb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:40.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.782+0000 7faedc9f3700 1 --2- 172.21.15.92:0/2923563554 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7faeb8066f30 0x7faeb80693e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faec8006f00 tx=0x7faec80091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.782+0000 7faedc9f3700 1 -- 172.21.15.92:0/2923563554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faed80c7a10 msgr2=0x7faed80c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:40.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.782+0000 7faedc9f3700 1 --2- 172.21.15.92:0/2923563554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faed80c7a10 0x7faed80c7e80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7faec0004730 tx=0x7faed0015000 comp rx=0 tx=0).stop 2023-10-06T10:40:40.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.782+0000 7faedc9f3700 1 -- 172.21.15.92:0/2923563554 shutdown_connections 2023-10-06T10:40:40.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.782+0000 7faedc9f3700 1 --2- 172.21.15.92:0/2923563554 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7faeb8066f30 0x7faeb80693e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.782+0000 7faedc9f3700 1 --2- 172.21.15.92:0/2923563554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faed80c7a10 0x7faed80c7e80 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.782+0000 7faedc9f3700 1 --2- 172.21.15.92:0/2923563554 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faec0004d50 0x7faed814f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.782+0000 7faedc9f3700 1 --2- 172.21.15.92:0/2923563554 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faed81584c0 0x7faed8158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.782+0000 7faedc9f3700 1 -- 172.21.15.92:0/2923563554 >> 172.21.15.92:0/2923563554 conn(0x7faed8068080 msgr2=0x7faed80b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:40.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.782+0000 7faedc9f3700 1 -- 172.21.15.92:0/2923563554 shutdown_connections 2023-10-06T10:40:40.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.782+0000 7faedc9f3700 1 -- 172.21.15.92:0/2923563554 wait complete. 2023-10-06T10:40:40.796 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T10:40:40.796 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:40:40.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.908+0000 7f5c493ae700 1 Processor -- start 2023-10-06T10:40:40.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.908+0000 7f5c493ae700 1 -- start start 2023-10-06T10:40:40.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.909+0000 7f5c493ae700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5c440b25e0 0x7f5c440b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:40.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.909+0000 7f5c493ae700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5c440b29b0 0x7f5c44159120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:40.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.909+0000 7f5c493ae700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c44159660 0x7f5c44068540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:40.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.909+0000 7f5c493ae700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c440cbfa0 con 0x7f5c440b25e0 2023-10-06T10:40:40.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.909+0000 7f5c493ae700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c440cb390 con 0x7f5c440b29b0 2023-10-06T10:40:40.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.909+0000 7f5c493ae700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c440cb670 con 0x7f5c44159660 2023-10-06T10:40:40.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.909+0000 7f5c42ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5c440b25e0 0x7f5c440b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:40.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.909+0000 7f5c42ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5c440b25e0 0x7f5c440b0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60730/0 (socket says 172.21.15.92:60730) 2023-10-06T10:40:40.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.909+0000 7f5c42ffd700 1 -- 172.21.15.92:0/2418988891 learned_addr learned my addr 172.21.15.92:0/2418988891 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:40.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.910+0000 7f5c437fe700 1 --2- 172.21.15.92:0/2418988891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c44159660 0x7f5c44068540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:40.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.910+0000 7f5c42ffd700 1 -- 172.21.15.92:0/2418988891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c44159660 msgr2=0x7f5c44068540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:40.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.910+0000 7f5c42ffd700 1 --2- 172.21.15.92:0/2418988891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c44159660 0x7f5c44068540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.910+0000 7f5c3a7fc700 1 --2- 172.21.15.92:0/2418988891 >> v2:172.21.15.124:3300/0 conn(0x7f5c440b29b0 0x7f5c44159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:40.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.910+0000 7f5c42ffd700 1 -- 172.21.15.92:0/2418988891 >> v2:172.21.15.124:3300/0 conn(0x7f5c440b29b0 msgr2=0x7f5c44159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:40.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.910+0000 7f5c42ffd700 1 --2- 172.21.15.92:0/2418988891 >> v2:172.21.15.124:3300/0 conn(0x7f5c440b29b0 0x7f5c44159120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.910+0000 7f5c42ffd700 1 -- 172.21.15.92:0/2418988891 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c440cb950 con 0x7f5c440b25e0 2023-10-06T10:40:40.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.910+0000 7f5c42ffd700 1 --2- 172.21.15.92:0/2418988891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c440b25e0 0x7f5c440b0690 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5c440cab10 tx=0x7f5c34009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6025bed59b921515 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:40.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.910+0000 7f5c427fc700 1 -- 172.21.15.92:0/2418988891 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34005c10 con 0x7f5c440b25e0 2023-10-06T10:40:40.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.910+0000 7f5c427fc700 1 -- 172.21.15.92:0/2418988891 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c34005d70 con 0x7f5c440b25e0 2023-10-06T10:40:40.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.910+0000 7f5c427fc700 1 -- 172.21.15.92:0/2418988891 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3400b880 con 0x7f5c440b25e0 2023-10-06T10:40:40.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.910+0000 7f5c493ae700 1 -- 172.21.15.92:0/2418988891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c440b25e0 msgr2=0x7f5c440b0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:40.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.910+0000 7f5c493ae700 1 --2- 172.21.15.92:0/2418988891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c440b25e0 0x7f5c440b0690 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5c440cab10 tx=0x7f5c34009220 comp rx=0 tx=0).stop 2023-10-06T10:40:40.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.910+0000 7f5c493ae700 1 -- 172.21.15.92:0/2418988891 shutdown_connections 2023-10-06T10:40:40.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.910+0000 7f5c493ae700 1 --2- 172.21.15.92:0/2418988891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c440b25e0 0x7f5c440b0690 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.910+0000 7f5c493ae700 1 --2- 172.21.15.92:0/2418988891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c44159660 0x7f5c44068540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.910+0000 7f5c493ae700 1 --2- 172.21.15.92:0/2418988891 >> v2:172.21.15.124:3300/0 conn(0x7f5c440b29b0 0x7f5c44159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.910+0000 7f5c493ae700 1 -- 172.21.15.92:0/2418988891 >> 172.21.15.92:0/2418988891 conn(0x7f5c4406ece0 msgr2=0x7f5c4406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:40.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.911+0000 7f5c493ae700 1 -- 172.21.15.92:0/2418988891 shutdown_connections 2023-10-06T10:40:40.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.911+0000 7f5c493ae700 1 -- 172.21.15.92:0/2418988891 wait complete. 2023-10-06T10:40:40.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.912+0000 7f5c493ae700 1 Processor -- start 2023-10-06T10:40:40.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.912+0000 7f5c493ae700 1 -- start start 2023-10-06T10:40:40.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.912+0000 7f5c493ae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c440b25e0 0x7f5c44157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:40.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.912+0000 7f5c493ae700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c440b29b0 0x7f5c440c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:40.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.912+0000 7f5c493ae700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c44159660 0x7f5c440b5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:40.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.912+0000 7f5c493ae700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c441575b0 con 0x7f5c440b25e0 2023-10-06T10:40:40.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.912+0000 7f5c493ae700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c440c7be0 con 0x7f5c44159660 2023-10-06T10:40:40.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.912+0000 7f5c493ae700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4416d6e0 con 0x7f5c440b29b0 2023-10-06T10:40:40.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.913+0000 7f5c42ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c440b25e0 0x7f5c44157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:40.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.913+0000 7f5c42ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c440b25e0 0x7f5c44157070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60742/0 (socket says 172.21.15.92:60742) 2023-10-06T10:40:40.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.913+0000 7f5c42ffd700 1 -- 172.21.15.92:0/3992280481 learned_addr learned my addr 172.21.15.92:0/3992280481 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:40.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.913+0000 7f5c437fe700 1 --2- 172.21.15.92:0/3992280481 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c44159660 0x7f5c440b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:40.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.913+0000 7f5c3a7fc700 1 --2- 172.21.15.92:0/3992280481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c440b29b0 0x7f5c440c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:40.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.913+0000 7f5c42ffd700 1 -- 172.21.15.92:0/3992280481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c440b29b0 msgr2=0x7f5c440c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:40.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.913+0000 7f5c42ffd700 1 --2- 172.21.15.92:0/3992280481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c440b29b0 0x7f5c440c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.913+0000 7f5c42ffd700 1 -- 172.21.15.92:0/3992280481 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c44159660 msgr2=0x7f5c440b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:40.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.913+0000 7f5c42ffd700 1 --2- 172.21.15.92:0/3992280481 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c44159660 0x7f5c440b5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:40.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.913+0000 7f5c42ffd700 1 -- 172.21.15.92:0/3992280481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c34009480 con 0x7f5c440b25e0 2023-10-06T10:40:40.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.913+0000 7f5c42ffd700 1 --2- 172.21.15.92:0/3992280481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c440b25e0 0x7f5c44157070 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400b3c0 tx=0x7f5c34016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:40.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.913+0000 7f5c3b7fe700 1 -- 172.21.15.92:0/3992280481 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34011740 con 0x7f5c440b25e0 2023-10-06T10:40:40.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.913+0000 7f5c3b7fe700 1 -- 172.21.15.92:0/3992280481 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c34020070 con 0x7f5c440b25e0 2023-10-06T10:40:40.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.913+0000 7f5c493ae700 1 -- 172.21.15.92:0/3992280481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4416d9c0 con 0x7f5c440b25e0 2023-10-06T10:40:40.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.913+0000 7f5c3b7fe700 1 -- 172.21.15.92:0/3992280481 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34011ad0 con 0x7f5c440b25e0 2023-10-06T10:40:40.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.913+0000 7f5c493ae700 1 -- 172.21.15.92:0/3992280481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4416df80 con 0x7f5c440b25e0 2023-10-06T10:40:40.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.914+0000 7f5c493ae700 1 -- 172.21.15.92:0/3992280481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c44064a90 con 0x7f5c440b25e0 2023-10-06T10:40:40.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.915+0000 7f5c3b7fe700 1 -- 172.21.15.92:0/3992280481 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5c34022030 con 0x7f5c440b25e0 2023-10-06T10:40:40.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.915+0000 7f5c3b7fe700 1 --2- 172.21.15.92:0/3992280481 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5c24066f30 0x7f5c240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:40.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.915+0000 7f5c3b7fe700 1 -- 172.21.15.92:0/3992280481 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5c34086320 con 0x7f5c440b25e0 2023-10-06T10:40:40.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.915+0000 7f5c3a7fc700 1 --2- 172.21.15.92:0/3992280481 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5c24066f30 0x7f5c240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:40.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.916+0000 7f5c3a7fc700 1 --2- 172.21.15.92:0/3992280481 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5c24066f30 0x7f5c240693e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5c440ce8a0 tx=0x7f5c2c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:40.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:40.919+0000 7f5c3b7fe700 1 -- 172.21.15.92:0/3992280481 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c340532f0 con 0x7f5c440b25e0 2023-10-06T10:40:41.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:41.099+0000 7f5c493ae700 1 -- 172.21.15.92:0/3992280481 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5c440cbc30 con 0x7f5c24066f30 2023-10-06T10:40:41.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:41.108+0000 7f5c3b7fe700 1 -- 172.21.15.92:0/3992280481 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311702 (secure 0 0 0) 0x7f5c440cbc30 con 0x7f5c24066f30 2023-10-06T10:40:41.113 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:41.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:41.111+0000 7f5c493ae700 1 -- 172.21.15.92:0/3992280481 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5c24066f30 msgr2=0x7f5c240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:41.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:41.111+0000 7f5c493ae700 1 --2- 172.21.15.92:0/3992280481 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5c24066f30 0x7f5c240693e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5c440ce8a0 tx=0x7f5c2c008040 comp rx=0 tx=0).stop 2023-10-06T10:40:41.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:41.111+0000 7f5c493ae700 1 -- 172.21.15.92:0/3992280481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c440b25e0 msgr2=0x7f5c44157070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:41.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:41.111+0000 7f5c493ae700 1 --2- 172.21.15.92:0/3992280481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c440b25e0 0x7f5c44157070 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400b3c0 tx=0x7f5c34016000 comp rx=0 tx=0).stop 2023-10-06T10:40:41.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:41.111+0000 7f5c493ae700 1 -- 172.21.15.92:0/3992280481 shutdown_connections 2023-10-06T10:40:41.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:41.111+0000 7f5c493ae700 1 --2- 172.21.15.92:0/3992280481 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5c24066f30 0x7f5c240693e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:41.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:41.111+0000 7f5c493ae700 1 --2- 172.21.15.92:0/3992280481 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c44159660 0x7f5c440b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:41.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:41.111+0000 7f5c493ae700 1 --2- 172.21.15.92:0/3992280481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c440b25e0 0x7f5c44157070 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:41.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:41.111+0000 7f5c493ae700 1 --2- 172.21.15.92:0/3992280481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c440b29b0 0x7f5c440c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:41.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:41.111+0000 7f5c493ae700 1 -- 172.21.15.92:0/3992280481 >> 172.21.15.92:0/3992280481 conn(0x7f5c4406ece0 msgr2=0x7f5c440b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:41.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:41.111+0000 7f5c493ae700 1 -- 172.21.15.92:0/3992280481 shutdown_connections 2023-10-06T10:40:41.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:41.111+0000 7f5c493ae700 1 -- 172.21.15.92:0/3992280481 wait complete. 2023-10-06T10:40:41.118 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:40:41.125 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2023-10-06T10:40:40.112966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":201,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2384356,"kb_used_data":6052,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752590364,"statfs":{"total":773094113280,"available":770652532736,"internally_reserved":0,"allocated":6197248,"data_stored":3482899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001152"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.575414+0000","last_change":"2023-10-06T10:40:35.572024+0000","last_active":"2023-10-06T10:40:35.575414+0000","last_peered":"2023-10-06T10:40:35.575414+0000","last_clean":"2023-10-06T10:40:35.575414+0000","last_became_active":"2023-10-06T10:40:35.571764+0000","last_became_peered":"2023-10-06T10:40:35.571764+0000","last_unstale":"2023-10-06T10:40:35.575414+0000","last_undegraded":"2023-10-06T10:40:35.575414+0000","last_fullsized":"2023-10-06T10:40:35.575414+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:41.911815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576306+0000","last_change":"2023-10-06T10:40:35.570742+0000","last_active":"2023-10-06T10:40:35.576306+0000","last_peered":"2023-10-06T10:40:35.576306+0000","last_clean":"2023-10-06T10:40:35.576306+0000","last_became_active":"2023-10-06T10:40:35.570475+0000","last_became_peered":"2023-10-06T10:40:35.570475+0000","last_unstale":"2023-10-06T10:40:35.576306+0000","last_undegraded":"2023-10-06T10:40:35.576306+0000","last_fullsized":"2023-10-06T10:40:35.576306+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:01:49.371198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569068+0000","last_change":"2023-10-06T10:40:35.563948+0000","last_active":"2023-10-06T10:40:35.569068+0000","last_peered":"2023-10-06T10:40:35.569068+0000","last_clean":"2023-10-06T10:40:35.569068+0000","last_became_active":"2023-10-06T10:40:35.563663+0000","last_became_peered":"2023-10-06T10:40:35.563663+0000","last_unstale":"2023-10-06T10:40:35.569068+0000","last_undegraded":"2023-10-06T10:40:35.569068+0000","last_fullsized":"2023-10-06T10:40:35.569068+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:16:56.359962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.564545+0000","last_change":"2023-10-06T10:40:35.560154+0000","last_active":"2023-10-06T10:40:35.564545+0000","last_peered":"2023-10-06T10:40:35.564545+0000","last_clean":"2023-10-06T10:40:35.564545+0000","last_became_active":"2023-10-06T10:40:35.559902+0000","last_became_peered":"2023-10-06T10:40:35.559902+0000","last_unstale":"2023-10-06T10:40:35.564545+0000","last_undegraded":"2023-10-06T10:40:35.564545+0000","last_fullsized":"2023-10-06T10:40:35.564545+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:10.010731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.558859+0000","last_change":"2023-10-06T10:40:35.557573+0000","last_active":"2023-10-06T10:40:35.558859+0000","last_peered":"2023-10-06T10:40:35.558859+0000","last_clean":"2023-10-06T10:40:35.558859+0000","last_became_active":"2023-10-06T10:40:35.557357+0000","last_became_peered":"2023-10-06T10:40:35.557357+0000","last_unstale":"2023-10-06T10:40:35.558859+0000","last_undegraded":"2023-10-06T10:40:35.558859+0000","last_fullsized":"2023-10-06T10:40:35.558859+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:13:29.740541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.573528+0000","last_change":"2023-10-06T10:40:35.567916+0000","last_active":"2023-10-06T10:40:35.573528+0000","last_peered":"2023-10-06T10:40:35.573528+0000","last_clean":"2023-10-06T10:40:35.573528+0000","last_became_active":"2023-10-06T10:40:35.567655+0000","last_became_peered":"2023-10-06T10:40:35.567655+0000","last_unstale":"2023-10-06T10:40:35.573528+0000","last_undegraded":"2023-10-06T10:40:35.573528+0000","last_fullsized":"2023-10-06T10:40:35.573528+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:08:59.916087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.564828+0000","last_change":"2023-10-06T10:40:35.556880+0000","last_active":"2023-10-06T10:40:35.564828+0000","last_peered":"2023-10-06T10:40:35.564828+0000","last_clean":"2023-10-06T10:40:35.564828+0000","last_became_active":"2023-10-06T10:40:35.556638+0000","last_became_peered":"2023-10-06T10:40:35.556638+0000","last_unstale":"2023-10-06T10:40:35.564828+0000","last_undegraded":"2023-10-06T10:40:35.564828+0000","last_fullsized":"2023-10-06T10:40:35.564828+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:15.757865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576825+0000","last_change":"2023-10-06T10:40:35.571655+0000","last_active":"2023-10-06T10:40:35.576825+0000","last_peered":"2023-10-06T10:40:35.576825+0000","last_clean":"2023-10-06T10:40:35.576825+0000","last_became_active":"2023-10-06T10:40:35.565829+0000","last_became_peered":"2023-10-06T10:40:35.565829+0000","last_unstale":"2023-10-06T10:40:35.576825+0000","last_undegraded":"2023-10-06T10:40:35.576825+0000","last_fullsized":"2023-10-06T10:40:35.576825+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:13.916559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569860+0000","last_change":"2023-10-06T10:40:35.566171+0000","last_active":"2023-10-06T10:40:35.569860+0000","last_peered":"2023-10-06T10:40:35.569860+0000","last_clean":"2023-10-06T10:40:35.569860+0000","last_became_active":"2023-10-06T10:40:35.565812+0000","last_became_peered":"2023-10-06T10:40:35.565812+0000","last_unstale":"2023-10-06T10:40:35.569860+0000","last_undegraded":"2023-10-06T10:40:35.569860+0000","last_fullsized":"2023-10-06T10:40:35.569860+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:10.010731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.574755+0000","last_change":"2023-10-06T10:40:35.566646+0000","last_active":"2023-10-06T10:40:35.574755+0000","last_peered":"2023-10-06T10:40:35.574755+0000","last_clean":"2023-10-06T10:40:35.574755+0000","last_became_active":"2023-10-06T10:40:35.566383+0000","last_became_peered":"2023-10-06T10:40:35.566383+0000","last_unstale":"2023-10-06T10:40:35.574755+0000","last_undegraded":"2023-10-06T10:40:35.574755+0000","last_fullsized":"2023-10-06T10:40:35.574755+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:27.389629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.577321+0000","last_change":"2023-10-06T10:40:35.576738+0000","last_active":"2023-10-06T10:40:35.577321+0000","last_peered":"2023-10-06T10:40:35.577321+0000","last_clean":"2023-10-06T10:40:35.577321+0000","last_became_active":"2023-10-06T10:40:35.576470+0000","last_became_peered":"2023-10-06T10:40:35.576470+0000","last_unstale":"2023-10-06T10:40:35.577321+0000","last_undegraded":"2023-10-06T10:40:35.577321+0000","last_fullsized":"2023-10-06T10:40:35.577321+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:34:51.555583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.564516+0000","last_change":"2023-10-06T10:40:35.558573+0000","last_active":"2023-10-06T10:40:35.564516+0000","last_peered":"2023-10-06T10:40:35.564516+0000","last_clean":"2023-10-06T10:40:35.564516+0000","last_became_active":"2023-10-06T10:40:35.558278+0000","last_became_peered":"2023-10-06T10:40:35.558278+0000","last_unstale":"2023-10-06T10:40:35.564516+0000","last_undegraded":"2023-10-06T10:40:35.564516+0000","last_fullsized":"2023-10-06T10:40:35.564516+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:22.905054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.558118+0000","last_change":"2023-10-06T10:40:35.555626+0000","last_active":"2023-10-06T10:40:35.558118+0000","last_peered":"2023-10-06T10:40:35.558118+0000","last_clean":"2023-10-06T10:40:35.558118+0000","last_became_active":"2023-10-06T10:40:35.555367+0000","last_became_peered":"2023-10-06T10:40:35.555367+0000","last_unstale":"2023-10-06T10:40:35.558118+0000","last_undegraded":"2023-10-06T10:40:35.558118+0000","last_fullsized":"2023-10-06T10:40:35.558118+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:22.429465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568058+0000","last_change":"2023-10-06T10:40:35.560779+0000","last_active":"2023-10-06T10:40:35.568058+0000","last_peered":"2023-10-06T10:40:35.568058+0000","last_clean":"2023-10-06T10:40:35.568058+0000","last_became_active":"2023-10-06T10:40:35.560517+0000","last_became_peered":"2023-10-06T10:40:35.560517+0000","last_unstale":"2023-10-06T10:40:35.568058+0000","last_undegraded":"2023-10-06T10:40:35.568058+0000","last_fullsized":"2023-10-06T10:40:35.568058+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:31.413269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568008+0000","last_change":"2023-10-06T10:40:35.559791+0000","last_active":"2023-10-06T10:40:35.568008+0000","last_peered":"2023-10-06T10:40:35.568008+0000","last_clean":"2023-10-06T10:40:35.568008+0000","last_became_active":"2023-10-06T10:40:35.559541+0000","last_became_peered":"2023-10-06T10:40:35.559541+0000","last_unstale":"2023-10-06T10:40:35.568008+0000","last_undegraded":"2023-10-06T10:40:35.568008+0000","last_fullsized":"2023-10-06T10:40:35.568008+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:48.719683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.572860+0000","last_change":"2023-10-06T10:40:35.562284+0000","last_active":"2023-10-06T10:40:35.572860+0000","last_peered":"2023-10-06T10:40:35.572860+0000","last_clean":"2023-10-06T10:40:35.572860+0000","last_became_active":"2023-10-06T10:40:35.561931+0000","last_became_peered":"2023-10-06T10:40:35.561931+0000","last_unstale":"2023-10-06T10:40:35.572860+0000","last_undegraded":"2023-10-06T10:40:35.572860+0000","last_fullsized":"2023-10-06T10:40:35.572860+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:55.843404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576252+0000","last_change":"2023-10-06T10:40:35.567770+0000","last_active":"2023-10-06T10:40:35.576252+0000","last_peered":"2023-10-06T10:40:35.576252+0000","last_clean":"2023-10-06T10:40:35.576252+0000","last_became_active":"2023-10-06T10:40:35.566223+0000","last_became_peered":"2023-10-06T10:40:35.566223+0000","last_unstale":"2023-10-06T10:40:35.576252+0000","last_undegraded":"2023-10-06T10:40:35.576252+0000","last_fullsized":"2023-10-06T10:40:35.576252+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:56.036704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.572361+0000","last_change":"2023-10-06T10:40:35.569168+0000","last_active":"2023-10-06T10:40:35.572361+0000","last_peered":"2023-10-06T10:40:35.572361+0000","last_clean":"2023-10-06T10:40:35.572361+0000","last_became_active":"2023-10-06T10:40:35.568859+0000","last_became_peered":"2023-10-06T10:40:35.568859+0000","last_unstale":"2023-10-06T10:40:35.572361+0000","last_undegraded":"2023-10-06T10:40:35.572361+0000","last_fullsized":"2023-10-06T10:40:35.572361+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:28:38.929115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569862+0000","last_change":"2023-10-06T10:40:35.566851+0000","last_active":"2023-10-06T10:40:35.569862+0000","last_peered":"2023-10-06T10:40:35.569862+0000","last_clean":"2023-10-06T10:40:35.569862+0000","last_became_active":"2023-10-06T10:40:35.566568+0000","last_became_peered":"2023-10-06T10:40:35.566568+0000","last_unstale":"2023-10-06T10:40:35.569862+0000","last_undegraded":"2023-10-06T10:40:35.569862+0000","last_fullsized":"2023-10-06T10:40:35.569862+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:50.111333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576908+0000","last_change":"2023-10-06T10:40:35.573233+0000","last_active":"2023-10-06T10:40:35.576908+0000","last_peered":"2023-10-06T10:40:35.576908+0000","last_clean":"2023-10-06T10:40:35.576908+0000","last_became_active":"2023-10-06T10:40:35.572994+0000","last_became_peered":"2023-10-06T10:40:35.572994+0000","last_unstale":"2023-10-06T10:40:35.576908+0000","last_undegraded":"2023-10-06T10:40:35.576908+0000","last_fullsized":"2023-10-06T10:40:35.576908+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:45:01.289103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.570019+0000","last_change":"2023-10-06T10:40:35.567541+0000","last_active":"2023-10-06T10:40:35.570019+0000","last_peered":"2023-10-06T10:40:35.570019+0000","last_clean":"2023-10-06T10:40:35.570019+0000","last_became_active":"2023-10-06T10:40:35.567165+0000","last_became_peered":"2023-10-06T10:40:35.567165+0000","last_unstale":"2023-10-06T10:40:35.570019+0000","last_undegraded":"2023-10-06T10:40:35.570019+0000","last_fullsized":"2023-10-06T10:40:35.570019+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:42.478496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576376+0000","last_change":"2023-10-06T10:40:35.568241+0000","last_active":"2023-10-06T10:40:35.576376+0000","last_peered":"2023-10-06T10:40:35.576376+0000","last_clean":"2023-10-06T10:40:35.576376+0000","last_became_active":"2023-10-06T10:40:35.568006+0000","last_became_peered":"2023-10-06T10:40:35.568006+0000","last_unstale":"2023-10-06T10:40:35.576376+0000","last_undegraded":"2023-10-06T10:40:35.576376+0000","last_fullsized":"2023-10-06T10:40:35.576376+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:16:58.559793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.577728+0000","last_change":"2023-10-06T10:40:35.577097+0000","last_active":"2023-10-06T10:40:35.577728+0000","last_peered":"2023-10-06T10:40:35.577728+0000","last_clean":"2023-10-06T10:40:35.577728+0000","last_became_active":"2023-10-06T10:40:35.576788+0000","last_became_peered":"2023-10-06T10:40:35.576788+0000","last_unstale":"2023-10-06T10:40:35.577728+0000","last_undegraded":"2023-10-06T10:40:35.577728+0000","last_fullsized":"2023-10-06T10:40:35.577728+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:47.164234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.563619+0000","last_change":"2023-10-06T10:40:35.562191+0000","last_active":"2023-10-06T10:40:35.563619+0000","last_peered":"2023-10-06T10:40:35.563619+0000","last_clean":"2023-10-06T10:40:35.563619+0000","last_became_active":"2023-10-06T10:40:35.561940+0000","last_became_peered":"2023-10-06T10:40:35.561940+0000","last_unstale":"2023-10-06T10:40:35.563619+0000","last_undegraded":"2023-10-06T10:40:35.563619+0000","last_fullsized":"2023-10-06T10:40:35.563619+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:40.960612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569894+0000","last_change":"2023-10-06T10:40:35.564934+0000","last_active":"2023-10-06T10:40:35.569894+0000","last_peered":"2023-10-06T10:40:35.569894+0000","last_clean":"2023-10-06T10:40:35.569894+0000","last_became_active":"2023-10-06T10:40:35.564686+0000","last_became_peered":"2023-10-06T10:40:35.564686+0000","last_unstale":"2023-10-06T10:40:35.569894+0000","last_undegraded":"2023-10-06T10:40:35.569894+0000","last_fullsized":"2023-10-06T10:40:35.569894+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:03.867863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.564523+0000","last_change":"2023-10-06T10:40:35.557812+0000","last_active":"2023-10-06T10:40:35.564523+0000","last_peered":"2023-10-06T10:40:35.564523+0000","last_clean":"2023-10-06T10:40:35.564523+0000","last_became_active":"2023-10-06T10:40:35.557534+0000","last_became_peered":"2023-10-06T10:40:35.557534+0000","last_unstale":"2023-10-06T10:40:35.564523+0000","last_undegraded":"2023-10-06T10:40:35.564523+0000","last_fullsized":"2023-10-06T10:40:35.564523+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:16:56.359962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569247+0000","last_change":"2023-10-06T10:40:35.562339+0000","last_active":"2023-10-06T10:40:35.569247+0000","last_peered":"2023-10-06T10:40:35.569247+0000","last_clean":"2023-10-06T10:40:35.569247+0000","last_became_active":"2023-10-06T10:40:35.562014+0000","last_became_peered":"2023-10-06T10:40:35.562014+0000","last_unstale":"2023-10-06T10:40:35.569247+0000","last_undegraded":"2023-10-06T10:40:35.569247+0000","last_fullsized":"2023-10-06T10:40:35.569247+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:55.143518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.558884+0000","last_change":"2023-10-06T10:40:35.555416+0000","last_active":"2023-10-06T10:40:35.558884+0000","last_peered":"2023-10-06T10:40:35.558884+0000","last_clean":"2023-10-06T10:40:35.558884+0000","last_became_active":"2023-10-06T10:40:35.555169+0000","last_became_peered":"2023-10-06T10:40:35.555169+0000","last_unstale":"2023-10-06T10:40:35.558884+0000","last_undegraded":"2023-10-06T10:40:35.558884+0000","last_fullsized":"2023-10-06T10:40:35.558884+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:11.988774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576530+0000","last_change":"2023-10-06T10:40:35.571179+0000","last_active":"2023-10-06T10:40:35.576530+0000","last_peered":"2023-10-06T10:40:35.576530+0000","last_clean":"2023-10-06T10:40:35.576530+0000","last_became_active":"2023-10-06T10:40:35.570784+0000","last_became_peered":"2023-10-06T10:40:35.570784+0000","last_unstale":"2023-10-06T10:40:35.576530+0000","last_undegraded":"2023-10-06T10:40:35.576530+0000","last_fullsized":"2023-10-06T10:40:35.576530+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:29:56.451363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569044+0000","last_change":"2023-10-06T10:40:35.564496+0000","last_active":"2023-10-06T10:40:35.569044+0000","last_peered":"2023-10-06T10:40:35.569044+0000","last_clean":"2023-10-06T10:40:35.569044+0000","last_became_active":"2023-10-06T10:40:35.564215+0000","last_became_peered":"2023-10-06T10:40:35.564215+0000","last_unstale":"2023-10-06T10:40:35.569044+0000","last_undegraded":"2023-10-06T10:40:35.569044+0000","last_fullsized":"2023-10-06T10:40:35.569044+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:06.273416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569438+0000","last_change":"2023-10-06T10:40:35.564889+0000","last_active":"2023-10-06T10:40:35.569438+0000","last_peered":"2023-10-06T10:40:35.569438+0000","last_clean":"2023-10-06T10:40:35.569438+0000","last_became_active":"2023-10-06T10:40:35.564630+0000","last_became_peered":"2023-10-06T10:40:35.564630+0000","last_unstale":"2023-10-06T10:40:35.569438+0000","last_undegraded":"2023-10-06T10:40:35.569438+0000","last_fullsized":"2023-10-06T10:40:35.569438+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:49.024348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.575282+0000","last_change":"2023-10-06T10:40:35.573554+0000","last_active":"2023-10-06T10:40:35.575282+0000","last_peered":"2023-10-06T10:40:35.575282+0000","last_clean":"2023-10-06T10:40:35.575282+0000","last_became_active":"2023-10-06T10:40:35.573307+0000","last_became_peered":"2023-10-06T10:40:35.573307+0000","last_unstale":"2023-10-06T10:40:35.575282+0000","last_undegraded":"2023-10-06T10:40:35.575282+0000","last_fullsized":"2023-10-06T10:40:35.575282+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:47.765804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568816+0000","last_change":"2023-10-06T10:40:35.564393+0000","last_active":"2023-10-06T10:40:35.568816+0000","last_peered":"2023-10-06T10:40:35.568816+0000","last_clean":"2023-10-06T10:40:35.568816+0000","last_became_active":"2023-10-06T10:40:35.564157+0000","last_became_peered":"2023-10-06T10:40:35.564157+0000","last_unstale":"2023-10-06T10:40:35.568816+0000","last_undegraded":"2023-10-06T10:40:35.568816+0000","last_fullsized":"2023-10-06T10:40:35.568816+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:06:14.418093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"51'4","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568646+0000","last_change":"2023-10-06T10:40:34.559819+0000","last_active":"2023-10-06T10:40:35.568646+0000","last_peered":"2023-10-06T10:40:35.568646+0000","last_clean":"2023-10-06T10:40:35.568646+0000","last_became_active":"2023-10-06T10:40:34.559433+0000","last_became_peered":"2023-10-06T10:40:34.559433+0000","last_unstale":"2023-10-06T10:40:35.568646+0000","last_undegraded":"2023-10-06T10:40:35.568646+0000","last_fullsized":"2023-10-06T10:40:35.568646+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:43:52.519978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.574934+0000","last_change":"2023-10-06T10:40:35.572023+0000","last_active":"2023-10-06T10:40:35.574934+0000","last_peered":"2023-10-06T10:40:35.574934+0000","last_clean":"2023-10-06T10:40:35.574934+0000","last_became_active":"2023-10-06T10:40:35.571788+0000","last_became_peered":"2023-10-06T10:40:35.571788+0000","last_unstale":"2023-10-06T10:40:35.574934+0000","last_undegraded":"2023-10-06T10:40:35.574934+0000","last_fullsized":"2023-10-06T10:40:35.574934+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:13.245750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"51'1","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.575594+0000","last_change":"2023-10-06T10:40:34.562460+0000","last_active":"2023-10-06T10:40:35.575594+0000","last_peered":"2023-10-06T10:40:35.575594+0000","last_clean":"2023-10-06T10:40:35.575594+0000","last_became_active":"2023-10-06T10:40:34.562241+0000","last_became_peered":"2023-10-06T10:40:34.562241+0000","last_unstale":"2023-10-06T10:40:35.575594+0000","last_undegraded":"2023-10-06T10:40:35.575594+0000","last_fullsized":"2023-10-06T10:40:35.575594+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:15.486524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.572595+0000","last_change":"2023-10-06T10:40:35.560246+0000","last_active":"2023-10-06T10:40:35.572595+0000","last_peered":"2023-10-06T10:40:35.572595+0000","last_clean":"2023-10-06T10:40:35.572595+0000","last_became_active":"2023-10-06T10:40:35.559981+0000","last_became_peered":"2023-10-06T10:40:35.559981+0000","last_unstale":"2023-10-06T10:40:35.572595+0000","last_undegraded":"2023-10-06T10:40:35.572595+0000","last_fullsized":"2023-10-06T10:40:35.572595+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:35.748566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"51'2","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.560216+0000","last_change":"2023-10-06T10:40:34.560326+0000","last_active":"2023-10-06T10:40:35.560216+0000","last_peered":"2023-10-06T10:40:35.560216+0000","last_clean":"2023-10-06T10:40:35.560216+0000","last_became_active":"2023-10-06T10:40:34.559983+0000","last_became_peered":"2023-10-06T10:40:34.559983+0000","last_unstale":"2023-10-06T10:40:35.560216+0000","last_undegraded":"2023-10-06T10:40:35.560216+0000","last_fullsized":"2023-10-06T10:40:35.560216+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:55:42.716617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.567779+0000","last_change":"2023-10-06T10:40:35.558007+0000","last_active":"2023-10-06T10:40:35.567779+0000","last_peered":"2023-10-06T10:40:35.567779+0000","last_clean":"2023-10-06T10:40:35.567779+0000","last_became_active":"2023-10-06T10:40:35.557606+0000","last_became_peered":"2023-10-06T10:40:35.557606+0000","last_unstale":"2023-10-06T10:40:35.567779+0000","last_undegraded":"2023-10-06T10:40:35.567779+0000","last_fullsized":"2023-10-06T10:40:35.567779+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:31.413269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"51'2","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.558037+0000","last_change":"2023-10-06T10:40:34.562219+0000","last_active":"2023-10-06T10:40:35.558037+0000","last_peered":"2023-10-06T10:40:35.558037+0000","last_clean":"2023-10-06T10:40:35.558037+0000","last_became_active":"2023-10-06T10:40:34.561960+0000","last_became_peered":"2023-10-06T10:40:34.561960+0000","last_unstale":"2023-10-06T10:40:35.558037+0000","last_undegraded":"2023-10-06T10:40:35.558037+0000","last_fullsized":"2023-10-06T10:40:35.558037+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:14.711366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576398+0000","last_change":"2023-10-06T10:40:35.572126+0000","last_active":"2023-10-06T10:40:35.576398+0000","last_peered":"2023-10-06T10:40:35.576398+0000","last_clean":"2023-10-06T10:40:35.576398+0000","last_became_active":"2023-10-06T10:40:35.571887+0000","last_became_peered":"2023-10-06T10:40:35.571887+0000","last_unstale":"2023-10-06T10:40:35.576398+0000","last_undegraded":"2023-10-06T10:40:35.576398+0000","last_fullsized":"2023-10-06T10:40:35.576398+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:47.765804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"51'1","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.560679+0000","last_change":"2023-10-06T10:40:34.557170+0000","last_active":"2023-10-06T10:40:35.560679+0000","last_peered":"2023-10-06T10:40:35.560679+0000","last_clean":"2023-10-06T10:40:35.560679+0000","last_became_active":"2023-10-06T10:40:34.556834+0000","last_became_peered":"2023-10-06T10:40:34.556834+0000","last_unstale":"2023-10-06T10:40:35.560679+0000","last_undegraded":"2023-10-06T10:40:35.560679+0000","last_fullsized":"2023-10-06T10:40:35.560679+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:38.780441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576498+0000","last_change":"2023-10-06T10:40:35.571018+0000","last_active":"2023-10-06T10:40:35.576498+0000","last_peered":"2023-10-06T10:40:35.576498+0000","last_clean":"2023-10-06T10:40:35.576498+0000","last_became_active":"2023-10-06T10:40:35.570523+0000","last_became_peered":"2023-10-06T10:40:35.570523+0000","last_unstale":"2023-10-06T10:40:35.576498+0000","last_undegraded":"2023-10-06T10:40:35.576498+0000","last_fullsized":"2023-10-06T10:40:35.576498+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:53.209242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"51'1","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568890+0000","last_change":"2023-10-06T10:40:34.555165+0000","last_active":"2023-10-06T10:40:35.568890+0000","last_peered":"2023-10-06T10:40:35.568890+0000","last_clean":"2023-10-06T10:40:35.568890+0000","last_became_active":"2023-10-06T10:40:34.554897+0000","last_became_peered":"2023-10-06T10:40:34.554897+0000","last_unstale":"2023-10-06T10:40:35.568890+0000","last_undegraded":"2023-10-06T10:40:35.568890+0000","last_fullsized":"2023-10-06T10:40:35.568890+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:05.264641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569444+0000","last_change":"2023-10-06T10:40:35.565210+0000","last_active":"2023-10-06T10:40:35.569444+0000","last_peered":"2023-10-06T10:40:35.569444+0000","last_clean":"2023-10-06T10:40:35.569444+0000","last_became_active":"2023-10-06T10:40:35.564970+0000","last_became_peered":"2023-10-06T10:40:35.564970+0000","last_unstale":"2023-10-06T10:40:35.569444+0000","last_undegraded":"2023-10-06T10:40:35.569444+0000","last_fullsized":"2023-10-06T10:40:35.569444+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:40.976608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.546745+0000","last_change":"2023-10-06T10:40:34.558169+0000","last_active":"2023-10-06T10:40:35.546745+0000","last_peered":"2023-10-06T10:40:35.546745+0000","last_clean":"2023-10-06T10:40:35.546745+0000","last_became_active":"2023-10-06T10:40:34.557947+0000","last_became_peered":"2023-10-06T10:40:34.557947+0000","last_unstale":"2023-10-06T10:40:35.546745+0000","last_undegraded":"2023-10-06T10:40:35.546745+0000","last_fullsized":"2023-10-06T10:40:35.546745+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:56.154679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576422+0000","last_change":"2023-10-06T10:40:35.572595+0000","last_active":"2023-10-06T10:40:35.576422+0000","last_peered":"2023-10-06T10:40:35.576422+0000","last_clean":"2023-10-06T10:40:35.576422+0000","last_became_active":"2023-10-06T10:40:35.572317+0000","last_became_peered":"2023-10-06T10:40:35.572317+0000","last_unstale":"2023-10-06T10:40:35.576422+0000","last_undegraded":"2023-10-06T10:40:35.576422+0000","last_fullsized":"2023-10-06T10:40:35.576422+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:17.634580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.547860+0000","last_change":"2023-10-06T10:40:34.556185+0000","last_active":"2023-10-06T10:40:35.547860+0000","last_peered":"2023-10-06T10:40:35.547860+0000","last_clean":"2023-10-06T10:40:35.547860+0000","last_became_active":"2023-10-06T10:40:34.555911+0000","last_became_peered":"2023-10-06T10:40:34.555911+0000","last_unstale":"2023-10-06T10:40:35.547860+0000","last_undegraded":"2023-10-06T10:40:35.547860+0000","last_fullsized":"2023-10-06T10:40:35.547860+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.285921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568612+0000","last_change":"2023-10-06T10:40:35.562668+0000","last_active":"2023-10-06T10:40:35.568612+0000","last_peered":"2023-10-06T10:40:35.568612+0000","last_clean":"2023-10-06T10:40:35.568612+0000","last_became_active":"2023-10-06T10:40:35.562428+0000","last_became_peered":"2023-10-06T10:40:35.562428+0000","last_unstale":"2023-10-06T10:40:35.568612+0000","last_undegraded":"2023-10-06T10:40:35.568612+0000","last_fullsized":"2023-10-06T10:40:35.568612+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:57:07.950757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.555289+0000","last_change":"2023-10-06T10:40:34.574923+0000","last_active":"2023-10-06T10:40:35.555289+0000","last_peered":"2023-10-06T10:40:35.555289+0000","last_clean":"2023-10-06T10:40:35.555289+0000","last_became_active":"2023-10-06T10:40:34.574684+0000","last_became_peered":"2023-10-06T10:40:34.574684+0000","last_unstale":"2023-10-06T10:40:35.555289+0000","last_undegraded":"2023-10-06T10:40:35.555289+0000","last_fullsized":"2023-10-06T10:40:35.555289+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:01:38.560878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.570091+0000","last_change":"2023-10-06T10:40:35.567735+0000","last_active":"2023-10-06T10:40:35.570091+0000","last_peered":"2023-10-06T10:40:35.570091+0000","last_clean":"2023-10-06T10:40:35.570091+0000","last_became_active":"2023-10-06T10:40:35.566258+0000","last_became_peered":"2023-10-06T10:40:35.566258+0000","last_unstale":"2023-10-06T10:40:35.570091+0000","last_undegraded":"2023-10-06T10:40:35.570091+0000","last_fullsized":"2023-10-06T10:40:35.570091+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:31.294697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.562660+0000","last_change":"2023-10-06T10:40:34.558613+0000","last_active":"2023-10-06T10:40:35.562660+0000","last_peered":"2023-10-06T10:40:35.562660+0000","last_clean":"2023-10-06T10:40:35.562660+0000","last_became_active":"2023-10-06T10:40:34.558128+0000","last_became_peered":"2023-10-06T10:40:34.558128+0000","last_unstale":"2023-10-06T10:40:35.562660+0000","last_undegraded":"2023-10-06T10:40:35.562660+0000","last_fullsized":"2023-10-06T10:40:35.562660+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:46.155459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.574833+0000","last_change":"2023-10-06T10:40:35.570712+0000","last_active":"2023-10-06T10:40:35.574833+0000","last_peered":"2023-10-06T10:40:35.574833+0000","last_clean":"2023-10-06T10:40:35.574833+0000","last_became_active":"2023-10-06T10:40:35.570450+0000","last_became_peered":"2023-10-06T10:40:35.570450+0000","last_unstale":"2023-10-06T10:40:35.574833+0000","last_undegraded":"2023-10-06T10:40:35.574833+0000","last_fullsized":"2023-10-06T10:40:35.574833+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:40.960612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"51'3","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.571980+0000","last_change":"2023-10-06T10:40:34.567975+0000","last_active":"2023-10-06T10:40:35.571980+0000","last_peered":"2023-10-06T10:40:35.571980+0000","last_clean":"2023-10-06T10:40:35.571980+0000","last_became_active":"2023-10-06T10:40:34.567763+0000","last_became_peered":"2023-10-06T10:40:34.567763+0000","last_unstale":"2023-10-06T10:40:35.571980+0000","last_undegraded":"2023-10-06T10:40:35.571980+0000","last_fullsized":"2023-10-06T10:40:35.571980+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:26.380853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569050+0000","last_change":"2023-10-06T10:40:35.563961+0000","last_active":"2023-10-06T10:40:35.569050+0000","last_peered":"2023-10-06T10:40:35.569050+0000","last_clean":"2023-10-06T10:40:35.569050+0000","last_became_active":"2023-10-06T10:40:35.563735+0000","last_became_peered":"2023-10-06T10:40:35.563735+0000","last_unstale":"2023-10-06T10:40:35.569050+0000","last_undegraded":"2023-10-06T10:40:35.569050+0000","last_fullsized":"2023-10-06T10:40:35.569050+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:40:08.170068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"51'6","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.579117+0000","last_change":"2023-10-06T10:40:34.555718+0000","last_active":"2023-10-06T10:40:35.579117+0000","last_peered":"2023-10-06T10:40:35.579117+0000","last_clean":"2023-10-06T10:40:35.579117+0000","last_became_active":"2023-10-06T10:40:34.555357+0000","last_became_peered":"2023-10-06T10:40:34.555357+0000","last_unstale":"2023-10-06T10:40:35.579117+0000","last_undegraded":"2023-10-06T10:40:35.579117+0000","last_fullsized":"2023-10-06T10:40:35.579117+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:46.155459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.564769+0000","last_change":"2023-10-06T10:40:35.562704+0000","last_active":"2023-10-06T10:40:35.564769+0000","last_peered":"2023-10-06T10:40:35.564769+0000","last_clean":"2023-10-06T10:40:35.564769+0000","last_became_active":"2023-10-06T10:40:35.562424+0000","last_became_peered":"2023-10-06T10:40:35.562424+0000","last_unstale":"2023-10-06T10:40:35.564769+0000","last_undegraded":"2023-10-06T10:40:35.564769+0000","last_fullsized":"2023-10-06T10:40:35.564769+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:14.394504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"51'2","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.575856+0000","last_change":"2023-10-06T10:40:34.558595+0000","last_active":"2023-10-06T10:40:35.575856+0000","last_peered":"2023-10-06T10:40:35.575856+0000","last_clean":"2023-10-06T10:40:35.575856+0000","last_became_active":"2023-10-06T10:40:34.558324+0000","last_became_peered":"2023-10-06T10:40:34.558324+0000","last_unstale":"2023-10-06T10:40:35.575856+0000","last_undegraded":"2023-10-06T10:40:35.575856+0000","last_fullsized":"2023-10-06T10:40:35.575856+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.285921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.567395+0000","last_change":"2023-10-06T10:40:35.560500+0000","last_active":"2023-10-06T10:40:35.567395+0000","last_peered":"2023-10-06T10:40:35.567395+0000","last_clean":"2023-10-06T10:40:35.567395+0000","last_became_active":"2023-10-06T10:40:35.559739+0000","last_became_peered":"2023-10-06T10:40:35.559739+0000","last_unstale":"2023-10-06T10:40:35.567395+0000","last_undegraded":"2023-10-06T10:40:35.567395+0000","last_fullsized":"2023-10-06T10:40:35.567395+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:39.789216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.550323+0000","last_change":"2023-10-06T10:40:34.563002+0000","last_active":"2023-10-06T10:40:35.550323+0000","last_peered":"2023-10-06T10:40:35.550323+0000","last_clean":"2023-10-06T10:40:35.550323+0000","last_became_active":"2023-10-06T10:40:34.562670+0000","last_became_peered":"2023-10-06T10:40:34.562670+0000","last_unstale":"2023-10-06T10:40:35.550323+0000","last_undegraded":"2023-10-06T10:40:35.550323+0000","last_fullsized":"2023-10-06T10:40:35.550323+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:26.380853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569802+0000","last_change":"2023-10-06T10:40:35.565320+0000","last_active":"2023-10-06T10:40:35.569802+0000","last_peered":"2023-10-06T10:40:35.569802+0000","last_clean":"2023-10-06T10:40:35.569802+0000","last_became_active":"2023-10-06T10:40:35.565079+0000","last_became_peered":"2023-10-06T10:40:35.565079+0000","last_unstale":"2023-10-06T10:40:35.569802+0000","last_undegraded":"2023-10-06T10:40:35.569802+0000","last_fullsized":"2023-10-06T10:40:35.569802+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:14.394504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.546791+0000","last_change":"2023-10-06T10:40:34.558954+0000","last_active":"2023-10-06T10:40:35.546791+0000","last_peered":"2023-10-06T10:40:35.546791+0000","last_clean":"2023-10-06T10:40:35.546791+0000","last_became_active":"2023-10-06T10:40:34.558311+0000","last_became_peered":"2023-10-06T10:40:34.558311+0000","last_unstale":"2023-10-06T10:40:35.546791+0000","last_undegraded":"2023-10-06T10:40:35.546791+0000","last_fullsized":"2023-10-06T10:40:35.546791+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:03.396798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.577709+0000","last_change":"2023-10-06T10:40:35.577044+0000","last_active":"2023-10-06T10:40:35.577709+0000","last_peered":"2023-10-06T10:40:35.577709+0000","last_clean":"2023-10-06T10:40:35.577709+0000","last_became_active":"2023-10-06T10:40:35.576791+0000","last_became_peered":"2023-10-06T10:40:35.576791+0000","last_unstale":"2023-10-06T10:40:35.577709+0000","last_undegraded":"2023-10-06T10:40:35.577709+0000","last_fullsized":"2023-10-06T10:40:35.577709+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:55:43.725392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"51'8","reported_seq":18,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568288+0000","last_change":"2023-10-06T10:40:34.562481+0000","last_active":"2023-10-06T10:40:35.568288+0000","last_peered":"2023-10-06T10:40:35.568288+0000","last_clean":"2023-10-06T10:40:35.568288+0000","last_became_active":"2023-10-06T10:40:34.562181+0000","last_became_peered":"2023-10-06T10:40:34.562181+0000","last_unstale":"2023-10-06T10:40:35.568288+0000","last_undegraded":"2023-10-06T10:40:35.568288+0000","last_fullsized":"2023-10-06T10:40:35.568288+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:04:34.719623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.566714+0000","last_change":"2023-10-06T10:40:35.562814+0000","last_active":"2023-10-06T10:40:35.566714+0000","last_peered":"2023-10-06T10:40:35.566714+0000","last_clean":"2023-10-06T10:40:35.566714+0000","last_became_active":"2023-10-06T10:40:35.562556+0000","last_became_peered":"2023-10-06T10:40:35.562556+0000","last_unstale":"2023-10-06T10:40:35.566714+0000","last_undegraded":"2023-10-06T10:40:35.566714+0000","last_fullsized":"2023-10-06T10:40:35.566714+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:48.719683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"51'2","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576493+0000","last_change":"2023-10-06T10:40:34.561496+0000","last_active":"2023-10-06T10:40:35.576493+0000","last_peered":"2023-10-06T10:40:35.576493+0000","last_clean":"2023-10-06T10:40:35.576493+0000","last_became_active":"2023-10-06T10:40:34.561276+0000","last_became_peered":"2023-10-06T10:40:34.561276+0000","last_unstale":"2023-10-06T10:40:35.576493+0000","last_undegraded":"2023-10-06T10:40:35.576493+0000","last_fullsized":"2023-10-06T10:40:35.576493+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:40.903040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.566308+0000","last_change":"2023-10-06T10:40:35.558844+0000","last_active":"2023-10-06T10:40:35.566308+0000","last_peered":"2023-10-06T10:40:35.566308+0000","last_clean":"2023-10-06T10:40:35.566308+0000","last_became_active":"2023-10-06T10:40:35.558577+0000","last_became_peered":"2023-10-06T10:40:35.558577+0000","last_unstale":"2023-10-06T10:40:35.566308+0000","last_undegraded":"2023-10-06T10:40:35.566308+0000","last_fullsized":"2023-10-06T10:40:35.566308+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:04:09.361668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.553797+0000","last_change":"2023-10-06T10:40:34.557813+0000","last_active":"2023-10-06T10:40:35.553797+0000","last_peered":"2023-10-06T10:40:35.553797+0000","last_clean":"2023-10-06T10:40:35.553797+0000","last_became_active":"2023-10-06T10:40:34.557512+0000","last_became_peered":"2023-10-06T10:40:34.557512+0000","last_unstale":"2023-10-06T10:40:35.553797+0000","last_undegraded":"2023-10-06T10:40:35.553797+0000","last_fullsized":"2023-10-06T10:40:35.553797+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:38.780441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.574854+0000","last_change":"2023-10-06T10:40:35.570944+0000","last_active":"2023-10-06T10:40:35.574854+0000","last_peered":"2023-10-06T10:40:35.574854+0000","last_clean":"2023-10-06T10:40:35.574854+0000","last_became_active":"2023-10-06T10:40:35.567192+0000","last_became_peered":"2023-10-06T10:40:35.567192+0000","last_unstale":"2023-10-06T10:40:35.574854+0000","last_undegraded":"2023-10-06T10:40:35.574854+0000","last_fullsized":"2023-10-06T10:40:35.574854+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:26.729670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.550419+0000","last_change":"2023-10-06T10:40:34.559497+0000","last_active":"2023-10-06T10:40:35.550419+0000","last_peered":"2023-10-06T10:40:35.550419+0000","last_clean":"2023-10-06T10:40:35.550419+0000","last_became_active":"2023-10-06T10:40:34.559159+0000","last_became_peered":"2023-10-06T10:40:34.559159+0000","last_unstale":"2023-10-06T10:40:35.550419+0000","last_undegraded":"2023-10-06T10:40:35.550419+0000","last_fullsized":"2023-10-06T10:40:35.550419+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:04:34.719623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.567761+0000","last_change":"2023-10-06T10:40:35.557851+0000","last_active":"2023-10-06T10:40:35.567761+0000","last_peered":"2023-10-06T10:40:35.567761+0000","last_clean":"2023-10-06T10:40:35.567761+0000","last_became_active":"2023-10-06T10:40:35.557289+0000","last_became_peered":"2023-10-06T10:40:35.557289+0000","last_unstale":"2023-10-06T10:40:35.567761+0000","last_undegraded":"2023-10-06T10:40:35.567761+0000","last_fullsized":"2023-10-06T10:40:35.567761+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:57:07.950757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.548294+0000","last_change":"2023-10-06T10:40:34.558179+0000","last_active":"2023-10-06T10:40:35.548294+0000","last_peered":"2023-10-06T10:40:35.548294+0000","last_clean":"2023-10-06T10:40:35.548294+0000","last_became_active":"2023-10-06T10:40:34.557924+0000","last_became_peered":"2023-10-06T10:40:34.557924+0000","last_unstale":"2023-10-06T10:40:35.548294+0000","last_undegraded":"2023-10-06T10:40:35.548294+0000","last_fullsized":"2023-10-06T10:40:35.548294+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:13.047456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576117+0000","last_change":"2023-10-06T10:40:35.565513+0000","last_active":"2023-10-06T10:40:35.576117+0000","last_peered":"2023-10-06T10:40:35.576117+0000","last_clean":"2023-10-06T10:40:35.576117+0000","last_became_active":"2023-10-06T10:40:35.565263+0000","last_became_peered":"2023-10-06T10:40:35.565263+0000","last_unstale":"2023-10-06T10:40:35.576117+0000","last_undegraded":"2023-10-06T10:40:35.576117+0000","last_fullsized":"2023-10-06T10:40:35.576117+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:12:37.044769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.559092+0000","last_change":"2023-10-06T10:40:34.559598+0000","last_active":"2023-10-06T10:40:35.559092+0000","last_peered":"2023-10-06T10:40:35.559092+0000","last_clean":"2023-10-06T10:40:35.559092+0000","last_became_active":"2023-10-06T10:40:34.559338+0000","last_became_peered":"2023-10-06T10:40:34.559338+0000","last_unstale":"2023-10-06T10:40:35.559092+0000","last_undegraded":"2023-10-06T10:40:35.559092+0000","last_fullsized":"2023-10-06T10:40:35.559092+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:10.979999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.577733+0000","last_change":"2023-10-06T10:40:35.577132+0000","last_active":"2023-10-06T10:40:35.577733+0000","last_peered":"2023-10-06T10:40:35.577733+0000","last_clean":"2023-10-06T10:40:35.577733+0000","last_became_active":"2023-10-06T10:40:35.576860+0000","last_became_peered":"2023-10-06T10:40:35.576860+0000","last_unstale":"2023-10-06T10:40:35.577733+0000","last_undegraded":"2023-10-06T10:40:35.577733+0000","last_fullsized":"2023-10-06T10:40:35.577733+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:16.495299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.548028+0000","last_change":"2023-10-06T10:40:34.555634+0000","last_active":"2023-10-06T10:40:35.548028+0000","last_peered":"2023-10-06T10:40:35.548028+0000","last_clean":"2023-10-06T10:40:35.548028+0000","last_became_active":"2023-10-06T10:40:34.555335+0000","last_became_peered":"2023-10-06T10:40:34.555335+0000","last_unstale":"2023-10-06T10:40:35.548028+0000","last_undegraded":"2023-10-06T10:40:35.548028+0000","last_fullsized":"2023-10-06T10:40:35.548028+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:55:42.716617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.574756+0000","last_change":"2023-10-06T10:40:35.566701+0000","last_active":"2023-10-06T10:40:35.574756+0000","last_peered":"2023-10-06T10:40:35.574756+0000","last_clean":"2023-10-06T10:40:35.574756+0000","last_became_active":"2023-10-06T10:40:35.566383+0000","last_became_peered":"2023-10-06T10:40:35.566383+0000","last_unstale":"2023-10-06T10:40:35.574756+0000","last_undegraded":"2023-10-06T10:40:35.574756+0000","last_fullsized":"2023-10-06T10:40:35.574756+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:01:39.569654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.559099+0000","last_change":"2023-10-06T10:40:34.558852+0000","last_active":"2023-10-06T10:40:35.559099+0000","last_peered":"2023-10-06T10:40:35.559099+0000","last_clean":"2023-10-06T10:40:35.559099+0000","last_became_active":"2023-10-06T10:40:34.558581+0000","last_became_peered":"2023-10-06T10:40:34.558581+0000","last_unstale":"2023-10-06T10:40:35.559099+0000","last_undegraded":"2023-10-06T10:40:35.559099+0000","last_fullsized":"2023-10-06T10:40:35.559099+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:49.102558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576292+0000","last_change":"2023-10-06T10:40:35.567776+0000","last_active":"2023-10-06T10:40:35.576292+0000","last_peered":"2023-10-06T10:40:35.576292+0000","last_clean":"2023-10-06T10:40:35.576292+0000","last_became_active":"2023-10-06T10:40:35.567440+0000","last_became_peered":"2023-10-06T10:40:35.567440+0000","last_unstale":"2023-10-06T10:40:35.576292+0000","last_undegraded":"2023-10-06T10:40:35.576292+0000","last_fullsized":"2023-10-06T10:40:35.576292+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:15.268020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.556015+0000","last_change":"2023-10-06T10:40:34.559435+0000","last_active":"2023-10-06T10:40:35.556015+0000","last_peered":"2023-10-06T10:40:35.556015+0000","last_clean":"2023-10-06T10:40:35.556015+0000","last_became_active":"2023-10-06T10:40:34.559184+0000","last_became_peered":"2023-10-06T10:40:34.559184+0000","last_unstale":"2023-10-06T10:40:35.556015+0000","last_undegraded":"2023-10-06T10:40:35.556015+0000","last_fullsized":"2023-10-06T10:40:35.556015+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:28:37.920340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.573009+0000","last_change":"2023-10-06T10:40:35.572131+0000","last_active":"2023-10-06T10:40:35.573009+0000","last_peered":"2023-10-06T10:40:35.573009+0000","last_clean":"2023-10-06T10:40:35.573009+0000","last_became_active":"2023-10-06T10:40:35.571859+0000","last_became_peered":"2023-10-06T10:40:35.571859+0000","last_unstale":"2023-10-06T10:40:35.573009+0000","last_undegraded":"2023-10-06T10:40:35.573009+0000","last_fullsized":"2023-10-06T10:40:35.573009+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:45.782226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.550794+0000","last_change":"2023-10-06T10:40:34.561843+0000","last_active":"2023-10-06T10:40:35.550794+0000","last_peered":"2023-10-06T10:40:35.550794+0000","last_clean":"2023-10-06T10:40:35.550794+0000","last_became_active":"2023-10-06T10:40:34.561550+0000","last_became_peered":"2023-10-06T10:40:34.561550+0000","last_unstale":"2023-10-06T10:40:35.550794+0000","last_undegraded":"2023-10-06T10:40:35.550794+0000","last_fullsized":"2023-10-06T10:40:35.550794+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:14.711366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":43,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.547102+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:40:35.547102+0000","last_peered":"2023-10-06T10:40:35.547102+0000","last_clean":"2023-10-06T10:40:35.547102+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:40:35.547102+0000","last_undegraded":"2023-10-06T10:40:35.547102+0000","last_fullsized":"2023-10-06T10:40:35.547102+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576189+0000","last_change":"2023-10-06T10:40:35.566096+0000","last_active":"2023-10-06T10:40:35.576189+0000","last_peered":"2023-10-06T10:40:35.576189+0000","last_clean":"2023-10-06T10:40:35.576189+0000","last_became_active":"2023-10-06T10:40:35.565831+0000","last_became_peered":"2023-10-06T10:40:35.565831+0000","last_unstale":"2023-10-06T10:40:35.576189+0000","last_undegraded":"2023-10-06T10:40:35.576189+0000","last_fullsized":"2023-10-06T10:40:35.576189+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:35.748566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"51'1","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.556108+0000","last_change":"2023-10-06T10:40:34.562301+0000","last_active":"2023-10-06T10:40:35.556108+0000","last_peered":"2023-10-06T10:40:35.556108+0000","last_clean":"2023-10-06T10:40:35.556108+0000","last_became_active":"2023-10-06T10:40:34.562030+0000","last_became_peered":"2023-10-06T10:40:34.562030+0000","last_unstale":"2023-10-06T10:40:35.556108+0000","last_undegraded":"2023-10-06T10:40:35.556108+0000","last_fullsized":"2023-10-06T10:40:35.556108+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:43:52.519978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.563486+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:40:35.563486+0000","last_peered":"2023-10-06T10:40:35.563486+0000","last_clean":"2023-10-06T10:40:35.563486+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:40:35.563486+0000","last_undegraded":"2023-10-06T10:40:35.563486+0000","last_fullsized":"2023-10-06T10:40:35.563486+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568740+0000","last_change":"2023-10-06T10:40:35.561940+0000","last_active":"2023-10-06T10:40:35.568740+0000","last_peered":"2023-10-06T10:40:35.568740+0000","last_clean":"2023-10-06T10:40:35.568740+0000","last_became_active":"2023-10-06T10:40:35.561665+0000","last_became_peered":"2023-10-06T10:40:35.561665+0000","last_unstale":"2023-10-06T10:40:35.568740+0000","last_undegraded":"2023-10-06T10:40:35.568740+0000","last_fullsized":"2023-10-06T10:40:35.568740+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:33:37.853079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.559769+0000","last_change":"2023-10-06T10:40:34.562242+0000","last_active":"2023-10-06T10:40:35.559769+0000","last_peered":"2023-10-06T10:40:35.559769+0000","last_clean":"2023-10-06T10:40:35.559769+0000","last_became_active":"2023-10-06T10:40:34.562001+0000","last_became_peered":"2023-10-06T10:40:34.562001+0000","last_unstale":"2023-10-06T10:40:35.559769+0000","last_undegraded":"2023-10-06T10:40:35.559769+0000","last_fullsized":"2023-10-06T10:40:35.559769+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:48.015572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":41,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.559170+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:40:35.559170+0000","last_peered":"2023-10-06T10:40:35.559170+0000","last_clean":"2023-10-06T10:40:35.559170+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:40:35.559170+0000","last_undegraded":"2023-10-06T10:40:35.559170+0000","last_fullsized":"2023-10-06T10:40:35.559170+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":186,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.553015+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:40:35.553015+0000","last_peered":"2023-10-06T10:40:35.553015+0000","last_clean":"2023-10-06T10:40:35.553015+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:40:35.553015+0000","last_undegraded":"2023-10-06T10:40:35.553015+0000","last_fullsized":"2023-10-06T10:40:35.553015+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.564750+0000","last_change":"2023-10-06T10:40:35.561025+0000","last_active":"2023-10-06T10:40:35.564750+0000","last_peered":"2023-10-06T10:40:35.564750+0000","last_clean":"2023-10-06T10:40:35.564750+0000","last_became_active":"2023-10-06T10:40:35.560733+0000","last_became_peered":"2023-10-06T10:40:35.560733+0000","last_unstale":"2023-10-06T10:40:35.564750+0000","last_undegraded":"2023-10-06T10:40:35.564750+0000","last_fullsized":"2023-10-06T10:40:35.564750+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:15.757865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"51'2","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.575749+0000","last_change":"2023-10-06T10:40:34.550122+0000","last_active":"2023-10-06T10:40:35.575749+0000","last_peered":"2023-10-06T10:40:35.575749+0000","last_clean":"2023-10-06T10:40:35.575749+0000","last_became_active":"2023-10-06T10:40:34.549859+0000","last_became_peered":"2023-10-06T10:40:34.549859+0000","last_unstale":"2023-10-06T10:40:35.575749+0000","last_undegraded":"2023-10-06T10:40:35.575749+0000","last_fullsized":"2023-10-06T10:40:35.575749+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:44.773451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":36,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.553171+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:40:35.553171+0000","last_peered":"2023-10-06T10:40:35.553171+0000","last_clean":"2023-10-06T10:40:35.553171+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:40:35.553171+0000","last_undegraded":"2023-10-06T10:40:35.553171+0000","last_fullsized":"2023-10-06T10:40:35.553171+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576107+0000","last_change":"2023-10-06T10:40:35.565493+0000","last_active":"2023-10-06T10:40:35.576107+0000","last_peered":"2023-10-06T10:40:35.576107+0000","last_clean":"2023-10-06T10:40:35.576107+0000","last_became_active":"2023-10-06T10:40:35.565234+0000","last_became_peered":"2023-10-06T10:40:35.565234+0000","last_unstale":"2023-10-06T10:40:35.576107+0000","last_undegraded":"2023-10-06T10:40:35.576107+0000","last_fullsized":"2023-10-06T10:40:35.576107+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:48.719683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.550113+0000","last_change":"2023-10-06T10:40:34.559498+0000","last_active":"2023-10-06T10:40:35.550113+0000","last_peered":"2023-10-06T10:40:35.550113+0000","last_clean":"2023-10-06T10:40:35.550113+0000","last_became_active":"2023-10-06T10:40:34.559262+0000","last_became_peered":"2023-10-06T10:40:34.559262+0000","last_unstale":"2023-10-06T10:40:35.550113+0000","last_undegraded":"2023-10-06T10:40:35.550113+0000","last_fullsized":"2023-10-06T10:40:35.550113+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:27:47.178791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":36,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.553009+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:40:35.553009+0000","last_peered":"2023-10-06T10:40:35.553009+0000","last_clean":"2023-10-06T10:40:35.553009+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:40:35.553009+0000","last_undegraded":"2023-10-06T10:40:35.553009+0000","last_fullsized":"2023-10-06T10:40:35.553009+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568628+0000","last_change":"2023-10-06T10:40:35.560232+0000","last_active":"2023-10-06T10:40:35.568628+0000","last_peered":"2023-10-06T10:40:35.568628+0000","last_clean":"2023-10-06T10:40:35.568628+0000","last_became_active":"2023-10-06T10:40:35.559942+0000","last_became_peered":"2023-10-06T10:40:35.559942+0000","last_unstale":"2023-10-06T10:40:35.568628+0000","last_undegraded":"2023-10-06T10:40:35.568628+0000","last_fullsized":"2023-10-06T10:40:35.568628+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:03.785102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.547498+0000","last_change":"2023-10-06T10:40:34.564639+0000","last_active":"2023-10-06T10:40:35.547498+0000","last_peered":"2023-10-06T10:40:35.547498+0000","last_clean":"2023-10-06T10:40:35.547498+0000","last_became_active":"2023-10-06T10:40:34.564425+0000","last_became_peered":"2023-10-06T10:40:34.564425+0000","last_unstale":"2023-10-06T10:40:35.547498+0000","last_undegraded":"2023-10-06T10:40:35.547498+0000","last_fullsized":"2023-10-06T10:40:35.547498+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:43:17.508745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":45,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.546832+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:40:35.546832+0000","last_peered":"2023-10-06T10:40:35.546832+0000","last_clean":"2023-10-06T10:40:35.546832+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:40:35.546832+0000","last_undegraded":"2023-10-06T10:40:35.546832+0000","last_fullsized":"2023-10-06T10:40:35.546832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569316+0000","last_change":"2023-10-06T10:40:35.565535+0000","last_active":"2023-10-06T10:40:35.569316+0000","last_peered":"2023-10-06T10:40:35.569316+0000","last_clean":"2023-10-06T10:40:35.569316+0000","last_became_active":"2023-10-06T10:40:35.565269+0000","last_became_peered":"2023-10-06T10:40:35.565269+0000","last_unstale":"2023-10-06T10:40:35.569316+0000","last_undegraded":"2023-10-06T10:40:35.569316+0000","last_fullsized":"2023-10-06T10:40:35.569316+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:15.720141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"51'1","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576121+0000","last_change":"2023-10-06T10:40:34.558460+0000","last_active":"2023-10-06T10:40:35.576121+0000","last_peered":"2023-10-06T10:40:35.576121+0000","last_clean":"2023-10-06T10:40:35.576121+0000","last_became_active":"2023-10-06T10:40:34.558193+0000","last_became_peered":"2023-10-06T10:40:34.558193+0000","last_unstale":"2023-10-06T10:40:35.576121+0000","last_undegraded":"2023-10-06T10:40:35.576121+0000","last_fullsized":"2023-10-06T10:40:35.576121+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:55:42.716617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.549822+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:40:35.549822+0000","last_peered":"2023-10-06T10:40:35.549822+0000","last_clean":"2023-10-06T10:40:35.549822+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:40:35.549822+0000","last_undegraded":"2023-10-06T10:40:35.549822+0000","last_fullsized":"2023-10-06T10:40:35.549822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569590+0000","last_change":"2023-10-06T10:40:35.566476+0000","last_active":"2023-10-06T10:40:35.569590+0000","last_peered":"2023-10-06T10:40:35.569590+0000","last_clean":"2023-10-06T10:40:35.569590+0000","last_became_active":"2023-10-06T10:40:35.566047+0000","last_became_peered":"2023-10-06T10:40:35.566047+0000","last_unstale":"2023-10-06T10:40:35.569590+0000","last_undegraded":"2023-10-06T10:40:35.569590+0000","last_fullsized":"2023-10-06T10:40:35.569590+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:57:07.950757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"51'2","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.564527+0000","last_change":"2023-10-06T10:40:34.557155+0000","last_active":"2023-10-06T10:40:35.564527+0000","last_peered":"2023-10-06T10:40:35.564527+0000","last_clean":"2023-10-06T10:40:35.564527+0000","last_became_active":"2023-10-06T10:40:34.556935+0000","last_became_peered":"2023-10-06T10:40:34.556935+0000","last_unstale":"2023-10-06T10:40:35.564527+0000","last_undegraded":"2023-10-06T10:40:35.564527+0000","last_fullsized":"2023-10-06T10:40:35.564527+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.285921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.550394+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:40:35.550394+0000","last_peered":"2023-10-06T10:40:35.550394+0000","last_clean":"2023-10-06T10:40:35.550394+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:40:35.550394+0000","last_undegraded":"2023-10-06T10:40:35.550394+0000","last_fullsized":"2023-10-06T10:40:35.550394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298480,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073360,"statfs":{"total":96636764160,"available":96331120640,"internally_reserved":0,"allocated":1220608,"data_stored":877891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.48099999999999998,"15min":0.47799999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71699999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.40500000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78600000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.54100000000000004}]},{"osd":1,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.46600000000000003,"15min":0.45700000000000002},"min":{"1min":0.375,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65900000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.5},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.505,"15min":0.5},"min":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72399999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.79100000000000004}]},{"osd":2,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.40600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":1.161},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.496,"15min":0.49199999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69199999999999995,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.747}]},{"osd":3,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54800000000000004,"15min":0.54200000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.66300000000000003}]},{"osd":4,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.46800000000000003,"15min":0.47699999999999998},"min":{"1min":0.30499999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57699999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.501,"15min":0.50900000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":1.0820000000000001}]},{"osd":5,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.32700000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65900000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.34300000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66800000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":1.052}]},{"osd":6,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.56799999999999995,"15min":0.57699999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.75700000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.747,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":1.129}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297888,"kb_used_data":600,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073952,"statfs":{"total":96636764160,"available":96331726848,"internally_reserved":0,"allocated":614400,"data_stored":286010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.49399999999999999,"15min":0.48899999999999999},"min":{"1min":0.36299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.44500000000000001,"15min":0.42899999999999999},"min":{"1min":0.38800000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.45100000000000001,"15min":0.437},"min":{"1min":0.33100000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.64300000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.379,"5min":0.309,"15min":0.309},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.47699999999999998,"15min":0.46400000000000002},"min":{"1min":0.34999999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.54400000000000004}]},{"osd":3,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.497,"15min":0.48799999999999999},"min":{"1min":0.38800000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.497},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.61399999999999999}]},{"osd":4,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47899999999999998,"15min":0.47799999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70899999999999996,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.65400000000000003}]},{"osd":5,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.48999999999999999,"15min":0.48599999999999999},"min":{"1min":0.28899999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69299999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33900000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.68899999999999995}]},{"osd":7,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.754,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.63}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298476,"kb_used_data":1188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073364,"statfs":{"total":96636764160,"available":96331124736,"internally_reserved":0,"allocated":1216512,"data_stored":877821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33000000000000002,"15min":0.32800000000000001},"min":{"1min":0.253,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.374,"5min":0.34999999999999998,"15min":0.34599999999999997},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.53900000000000003,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.29299999999999998}]},{"osd":2,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.32500000000000001,"15min":0.31900000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"last":0.31},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.36299999999999999},"min":{"1min":0.28999999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.36899999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.375},"min":{"1min":0.30099999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.379,"15min":0.375},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.34499999999999997}]},{"osd":4,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.40300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.69499999999999995,"5min":0.746,"15min":0.746},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.44700000000000001}]},{"osd":5,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54800000000000004,"15min":0.54100000000000004},"min":{"1min":0.436,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.39800000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.40100000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.433},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.47899999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.45600000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58099999999999996,"15min":0.57399999999999995},"min":{"1min":0.48699999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.439,"5min":0.436,"15min":0.436},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.38700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297908,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073932,"statfs":{"total":96636764160,"available":96331706368,"internally_reserved":0,"allocated":634880,"data_stored":289453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.32600000000000001,"15min":0.31900000000000001},"min":{"1min":0.23200000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.34899999999999998,"15min":0.34100000000000003},"min":{"1min":0.27600000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.45300000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.312,"15min":0.30599999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.371},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.33300000000000002,"15min":0.32500000000000001},"min":{"1min":0.26500000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.52000000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.41799999999999998}]},{"osd":3,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.35099999999999998,"15min":0.34300000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.35499999999999998,"15min":0.34899999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.436}]},{"osd":4,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.42599999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81200000000000006,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.54200000000000004,"15min":0.53300000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.48199999999999998}]},{"osd":5,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.41799999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.91900000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.497},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.80900000000000005,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.59199999999999997}]},{"osd":6,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58699999999999997,"15min":0.58799999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77300000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86399999999999999,"5min":1.075,"15min":1.075},"last":0.60299999999999998}]},{"osd":7,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79700000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.61399999999999999,"15min":0.60999999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88800000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297900,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073940,"statfs":{"total":96636764160,"available":96331714560,"internally_reserved":0,"allocated":626688,"data_stored":287419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.313,"15min":0.311},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.32500000000000001,"15min":0.317},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.33700000000000002}]},{"osd":1,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.32900000000000001,"15min":0.32300000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.54200000000000004,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.36699999999999999,"15min":0.36499999999999999},"min":{"1min":0.26400000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59299999999999997,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.30399999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.34899999999999998,"15min":0.34499999999999997},"min":{"1min":0.252,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.52700000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.35799999999999998,"15min":0.35499999999999998},"min":{"1min":0.27600000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.39900000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.35499999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.76100000000000001,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82699999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.42799999999999999}]},{"osd":5,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.436},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.377,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81599999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.46000000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.625,"15min":0.624},"min":{"1min":0.45800000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89000000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.624,"15min":0.623},"min":{"1min":0.433,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53300000000000003}]},{"osd":7,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.44500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79100000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.498,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.46999999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297900,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073940,"statfs":{"total":96636764160,"available":96331714560,"internally_reserved":0,"allocated":626688,"data_stored":287419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.438,"15min":0.45600000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.50600000000000001,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.36799999999999999,"15min":0.36399999999999999},"min":{"1min":0.27100000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66800000000000004,"5min":1.218,"15min":1.218},"last":0.34699999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34100000000000003,"15min":0.34000000000000002},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.61899999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.48299999999999998,"15min":0.495},"min":{"1min":0.27800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60299999999999998,"5min":3.359,"15min":3.359},"last":0.36099999999999999}]},{"osd":2,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38500000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.57699999999999996,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.315},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.63700000000000001,"15min":0.67300000000000004},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57699999999999996,"5min":7.069,"15min":7.069},"last":0.43099999999999999}]},{"osd":4,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89400000000000002,"5min":1.165,"15min":1.165},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59899999999999998,"15min":0.59699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79400000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.50700000000000001}]},{"osd":5,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.69799999999999995,"15min":0.71399999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92500000000000004,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60799999999999998,"15min":0.61099999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.96899999999999997,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.48299999999999998}]},{"osd":6,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.38,"5min":0.378,"15min":0.378},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.72699999999999998,"15min":0.74199999999999999},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84699999999999998,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.51600000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.61099999999999999},"min":{"1min":0.41299999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.99099999999999999,"5min":1.137,"15min":1.137},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.443,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91400000000000003,"5min":1.014,"15min":1.014},"last":0.54800000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297904,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073936,"statfs":{"total":96636764160,"available":96331710464,"internally_reserved":0,"allocated":630784,"data_stored":288899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47999999999999998,"15min":0.48099999999999998},"min":{"1min":0.33200000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.60799999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.496},"min":{"1min":0.33100000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.67000000000000004,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.60799999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.495,"15min":0.496},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.495,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.59399999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50700000000000001,"15min":0.50800000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.63,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36099999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.68500000000000005,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.56799999999999995}]},{"osd":3,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51600000000000001,"15min":0.51200000000000001},"min":{"1min":0.42199999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.625,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.53000000000000003,"15min":0.53500000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.64800000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64800000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.495,"15min":0.50800000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.61599999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.49399999999999999,"15min":0.502},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.63300000000000001,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.47899999999999998}]},{"osd":6,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.53600000000000003,"15min":0.54600000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.68799999999999994,"5min":0.75,"15min":0.75},"last":0.505},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.67700000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.46500000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.56899999999999995,"15min":0.57999999999999996},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70499999999999996,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72499999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.42899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297900,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073940,"statfs":{"total":96636764160,"available":96331714560,"internally_reserved":0,"allocated":626688,"data_stored":287987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.47399999999999998,"15min":0.46300000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54000000000000004,"15min":0.53400000000000003},"min":{"1min":0.35899999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.748,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.86499999999999999}]},{"osd":1,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.496,"15min":0.48499999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.52500000000000002,"15min":0.51700000000000002},"min":{"1min":0.39200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.80900000000000005}]},{"osd":2,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.51000000000000001,"15min":0.499},"min":{"1min":0.32000000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.41499999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.754,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.76300000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.57799999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77700000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78500000000000003,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.79100000000000004}]},{"osd":4,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.32100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.69599999999999995,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.501},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.66100000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.42299999999999999}]},{"osd":6,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.67500000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63400000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68799999999999994,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76700000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.55300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:40:44.134 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:40:44.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.253+0000 7fb50c6ff700 1 Processor -- start 2023-10-06T10:40:44.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.254+0000 7fb50c6ff700 1 -- start start 2023-10-06T10:40:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.254+0000 7fb50c6ff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb504159560 0x7fb5040aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.254+0000 7fb50c6ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb5040b49a0 0x7fb5040aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.254+0000 7fb50c6ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5040b0c60 con 0x7fb5040b0760 2023-10-06T10:40:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.254+0000 7fb50c6ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5040b0f40 con 0x7fb5040b49a0 2023-10-06T10:40:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.254+0000 7fb50c6ff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5040b1220 con 0x7fb504159560 2023-10-06T10:40:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.255+0000 7fb509c9a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb5040b49a0 0x7fb5040aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.255+0000 7fb50ac9c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb5040b0760 0x7fb5040b3890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36572/0 (socket says 172.21.15.92:36572) 2023-10-06T10:40:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.255+0000 7fb50ac9c700 1 -- 172.21.15.92:0/2410380467 learned_addr learned my addr 172.21.15.92:0/2410380467 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:44.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.255+0000 7fb509c9a700 1 -- 172.21.15.92:0/2410380467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb504159560 msgr2=0x7fb5040aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:44.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.255+0000 7fb50a49b700 1 --2- 172.21.15.92:0/2410380467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb504159560 0x7fb5040aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:44.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.255+0000 7fb509c9a700 1 --2- 172.21.15.92:0/2410380467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb504159560 0x7fb5040aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:44.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.255+0000 7fb509c9a700 1 -- 172.21.15.92:0/2410380467 >> v1:172.21.15.92:6789/0 conn(0x7fb5040b0760 legacy=0x7fb5040b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:44.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.255+0000 7fb509499700 1 -- 172.21.15.92:0/2410380467 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1686879792 0 0) 0x7fb5040b0c60 con 0x7fb5040b0760 2023-10-06T10:40:44.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.255+0000 7fb509c9a700 1 -- 172.21.15.92:0/2410380467 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5040b1500 con 0x7fb5040b49a0 2023-10-06T10:40:44.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.255+0000 7fb50a49b700 1 --2- 172.21.15.92:0/2410380467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb504159560 0x7fb5040aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:44.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.255+0000 7fb509c9a700 1 --2- 172.21.15.92:0/2410380467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5040b49a0 0x7fb5040aaa70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40098e0 tx=0x7fb4f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc215b48137ff9a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:44.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.255+0000 7fb509499700 1 -- 172.21.15.92:0/2410380467 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4009e80 con 0x7fb5040b49a0 2023-10-06T10:40:44.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.255+0000 7fb509499700 1 -- 172.21.15.92:0/2410380467 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f400b860 con 0x7fb5040b49a0 2023-10-06T10:40:44.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.255+0000 7fb509499700 1 -- 172.21.15.92:0/2410380467 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f400bbe0 con 0x7fb5040b49a0 2023-10-06T10:40:44.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.256+0000 7fb50c6ff700 1 -- 172.21.15.92:0/2410380467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5040b49a0 msgr2=0x7fb5040aaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:44.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.256+0000 7fb50c6ff700 1 --2- 172.21.15.92:0/2410380467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5040b49a0 0x7fb5040aaa70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40098e0 tx=0x7fb4f40091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:44.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.256+0000 7fb50c6ff700 1 -- 172.21.15.92:0/2410380467 shutdown_connections 2023-10-06T10:40:44.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.256+0000 7fb50c6ff700 1 --2- 172.21.15.92:0/2410380467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb504159560 0x7fb5040aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:44.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.256+0000 7fb50c6ff700 1 --2- 172.21.15.92:0/2410380467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5040b49a0 0x7fb5040aaa70 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:44.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.256+0000 7fb50c6ff700 1 -- 172.21.15.92:0/2410380467 >> 172.21.15.92:0/2410380467 conn(0x7fb5040997f0 msgr2=0x7fb50406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:44.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.256+0000 7fb50c6ff700 1 -- 172.21.15.92:0/2410380467 shutdown_connections 2023-10-06T10:40:44.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.256+0000 7fb50c6ff700 1 -- 172.21.15.92:0/2410380467 wait complete. 2023-10-06T10:40:44.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.257+0000 7fb50c6ff700 1 Processor -- start 2023-10-06T10:40:44.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.257+0000 7fb50c6ff700 1 -- start start 2023-10-06T10:40:44.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.258+0000 7fb50c6ff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5040b0760 0x7fb50415e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.258+0000 7fb50c6ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5040b49a0 0x7fb50409a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.258+0000 7fb50c6ff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb504159560 0x7fb50409b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.258+0000 7fb50c6ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50409a590 con 0x7fb5040b49a0 2023-10-06T10:40:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.258+0000 7fb50c6ff700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50409b3d0 con 0x7fb504159560 2023-10-06T10:40:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.258+0000 7fb50c6ff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50409a7c0 con 0x7fb5040b0760 2023-10-06T10:40:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.258+0000 7fb509c9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5040b49a0 0x7fb50409a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.258+0000 7fb509c9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5040b49a0 0x7fb50409a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34828/0 (socket says 172.21.15.92:34828) 2023-10-06T10:40:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.258+0000 7fb509c9a700 1 -- 172.21.15.92:0/3114516685 learned_addr learned my addr 172.21.15.92:0/3114516685 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.258+0000 7fb50a49b700 1 --2- 172.21.15.92:0/3114516685 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5040b0760 0x7fb50415e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.258+0000 7fb509c9a700 1 -- 172.21.15.92:0/3114516685 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5040b0760 msgr2=0x7fb50415e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:44.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.258+0000 7fb509c9a700 1 --2- 172.21.15.92:0/3114516685 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5040b0760 0x7fb50415e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:44.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.258+0000 7fb509c9a700 1 -- 172.21.15.92:0/3114516685 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb504159560 msgr2=0x7fb50409b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:44.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.258+0000 7fb509c9a700 1 --2- 172.21.15.92:0/3114516685 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb504159560 0x7fb50409b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:44.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.258+0000 7fb509c9a700 1 -- 172.21.15.92:0/3114516685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f4009420 con 0x7fb5040b49a0 2023-10-06T10:40:44.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.258+0000 7fb509c9a700 1 --2- 172.21.15.92:0/3114516685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5040b49a0 0x7fb50409a050 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4013000 tx=0x7fb4f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:44.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.258+0000 7fb4fa7fc700 1 -- 172.21.15.92:0/3114516685 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f400bbe0 con 0x7fb5040b49a0 2023-10-06T10:40:44.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.258+0000 7fb4fa7fc700 1 -- 172.21.15.92:0/3114516685 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f4011420 con 0x7fb5040b49a0 2023-10-06T10:40:44.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.258+0000 7fb50c6ff700 1 -- 172.21.15.92:0/3114516685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50409aaa0 con 0x7fb5040b49a0 2023-10-06T10:40:44.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.258+0000 7fb4fa7fc700 1 -- 172.21.15.92:0/3114516685 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f40117e0 con 0x7fb5040b49a0 2023-10-06T10:40:44.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.258+0000 7fb50c6ff700 1 -- 172.21.15.92:0/3114516685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50409b060 con 0x7fb5040b49a0 2023-10-06T10:40:44.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.259+0000 7fb50c6ff700 1 -- 172.21.15.92:0/3114516685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb50409b560 con 0x7fb5040b49a0 2023-10-06T10:40:44.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.260+0000 7fb4fa7fc700 1 -- 172.21.15.92:0/3114516685 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb4f4011bd0 con 0x7fb5040b49a0 2023-10-06T10:40:44.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.260+0000 7fb4fa7fc700 1 --2- 172.21.15.92:0/3114516685 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb4e8066f30 0x7fb4e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:44.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.260+0000 7fb4fa7fc700 1 -- 172.21.15.92:0/3114516685 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb4f4086320 con 0x7fb5040b49a0 2023-10-06T10:40:44.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.260+0000 7fb50a49b700 1 --2- 172.21.15.92:0/3114516685 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb4e8066f30 0x7fb4e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:44.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.260+0000 7fb50a49b700 1 --2- 172.21.15.92:0/3114516685 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb4e8066f30 0x7fb4e80693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb500007fb0 tx=0x7fb5000074c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:44.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.264+0000 7fb4fa7fc700 1 -- 172.21.15.92:0/3114516685 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4f40532f0 con 0x7fb5040b49a0 2023-10-06T10:40:44.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.443+0000 7fb50c6ff700 1 -- 172.21.15.92:0/3114516685 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb504064a90 con 0x7fb4e8066f30 2023-10-06T10:40:44.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.452+0000 7fb4fa7fc700 1 -- 172.21.15.92:0/3114516685 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311702 (secure 0 0 0) 0x7fb504064a90 con 0x7fb4e8066f30 2023-10-06T10:40:44.458 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:44.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.455+0000 7fb50c6ff700 1 -- 172.21.15.92:0/3114516685 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb4e8066f30 msgr2=0x7fb4e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:44.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.455+0000 7fb50c6ff700 1 --2- 172.21.15.92:0/3114516685 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb4e8066f30 0x7fb4e80693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb500007fb0 tx=0x7fb5000074c0 comp rx=0 tx=0).stop 2023-10-06T10:40:44.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.455+0000 7fb50c6ff700 1 -- 172.21.15.92:0/3114516685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5040b49a0 msgr2=0x7fb50409a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:44.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.455+0000 7fb50c6ff700 1 --2- 172.21.15.92:0/3114516685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5040b49a0 0x7fb50409a050 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4013000 tx=0x7fb4f4017000 comp rx=0 tx=0).stop 2023-10-06T10:40:44.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.455+0000 7fb50c6ff700 1 -- 172.21.15.92:0/3114516685 shutdown_connections 2023-10-06T10:40:44.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.455+0000 7fb50c6ff700 1 --2- 172.21.15.92:0/3114516685 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb4e8066f30 0x7fb4e80693e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:44.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.455+0000 7fb50c6ff700 1 --2- 172.21.15.92:0/3114516685 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb504159560 0x7fb50409b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:44.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.455+0000 7fb50c6ff700 1 --2- 172.21.15.92:0/3114516685 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5040b0760 0x7fb50415e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:44.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.455+0000 7fb50c6ff700 1 --2- 172.21.15.92:0/3114516685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5040b49a0 0x7fb50409a050 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:44.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.455+0000 7fb50c6ff700 1 -- 172.21.15.92:0/3114516685 >> 172.21.15.92:0/3114516685 conn(0x7fb5040997f0 msgr2=0x7fb504158770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:44.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.455+0000 7fb50c6ff700 1 -- 172.21.15.92:0/3114516685 shutdown_connections 2023-10-06T10:40:44.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.455+0000 7fb50c6ff700 1 -- 172.21.15.92:0/3114516685 wait complete. 2023-10-06T10:40:44.461 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:40:44.469 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2023-10-06T10:40:44.113775+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":201,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2384356,"kb_used_data":6052,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752590364,"statfs":{"total":773094113280,"available":770652532736,"internally_reserved":0,"allocated":6197248,"data_stored":3482899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.570233"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.575414+0000","last_change":"2023-10-06T10:40:35.572024+0000","last_active":"2023-10-06T10:40:35.575414+0000","last_peered":"2023-10-06T10:40:35.575414+0000","last_clean":"2023-10-06T10:40:35.575414+0000","last_became_active":"2023-10-06T10:40:35.571764+0000","last_became_peered":"2023-10-06T10:40:35.571764+0000","last_unstale":"2023-10-06T10:40:35.575414+0000","last_undegraded":"2023-10-06T10:40:35.575414+0000","last_fullsized":"2023-10-06T10:40:35.575414+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:41.911815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576306+0000","last_change":"2023-10-06T10:40:35.570742+0000","last_active":"2023-10-06T10:40:35.576306+0000","last_peered":"2023-10-06T10:40:35.576306+0000","last_clean":"2023-10-06T10:40:35.576306+0000","last_became_active":"2023-10-06T10:40:35.570475+0000","last_became_peered":"2023-10-06T10:40:35.570475+0000","last_unstale":"2023-10-06T10:40:35.576306+0000","last_undegraded":"2023-10-06T10:40:35.576306+0000","last_fullsized":"2023-10-06T10:40:35.576306+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:01:49.371198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569068+0000","last_change":"2023-10-06T10:40:35.563948+0000","last_active":"2023-10-06T10:40:35.569068+0000","last_peered":"2023-10-06T10:40:35.569068+0000","last_clean":"2023-10-06T10:40:35.569068+0000","last_became_active":"2023-10-06T10:40:35.563663+0000","last_became_peered":"2023-10-06T10:40:35.563663+0000","last_unstale":"2023-10-06T10:40:35.569068+0000","last_undegraded":"2023-10-06T10:40:35.569068+0000","last_fullsized":"2023-10-06T10:40:35.569068+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:16:56.359962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.564545+0000","last_change":"2023-10-06T10:40:35.560154+0000","last_active":"2023-10-06T10:40:35.564545+0000","last_peered":"2023-10-06T10:40:35.564545+0000","last_clean":"2023-10-06T10:40:35.564545+0000","last_became_active":"2023-10-06T10:40:35.559902+0000","last_became_peered":"2023-10-06T10:40:35.559902+0000","last_unstale":"2023-10-06T10:40:35.564545+0000","last_undegraded":"2023-10-06T10:40:35.564545+0000","last_fullsized":"2023-10-06T10:40:35.564545+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:10.010731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.558859+0000","last_change":"2023-10-06T10:40:35.557573+0000","last_active":"2023-10-06T10:40:35.558859+0000","last_peered":"2023-10-06T10:40:35.558859+0000","last_clean":"2023-10-06T10:40:35.558859+0000","last_became_active":"2023-10-06T10:40:35.557357+0000","last_became_peered":"2023-10-06T10:40:35.557357+0000","last_unstale":"2023-10-06T10:40:35.558859+0000","last_undegraded":"2023-10-06T10:40:35.558859+0000","last_fullsized":"2023-10-06T10:40:35.558859+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:13:29.740541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.573528+0000","last_change":"2023-10-06T10:40:35.567916+0000","last_active":"2023-10-06T10:40:35.573528+0000","last_peered":"2023-10-06T10:40:35.573528+0000","last_clean":"2023-10-06T10:40:35.573528+0000","last_became_active":"2023-10-06T10:40:35.567655+0000","last_became_peered":"2023-10-06T10:40:35.567655+0000","last_unstale":"2023-10-06T10:40:35.573528+0000","last_undegraded":"2023-10-06T10:40:35.573528+0000","last_fullsized":"2023-10-06T10:40:35.573528+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:08:59.916087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.564828+0000","last_change":"2023-10-06T10:40:35.556880+0000","last_active":"2023-10-06T10:40:35.564828+0000","last_peered":"2023-10-06T10:40:35.564828+0000","last_clean":"2023-10-06T10:40:35.564828+0000","last_became_active":"2023-10-06T10:40:35.556638+0000","last_became_peered":"2023-10-06T10:40:35.556638+0000","last_unstale":"2023-10-06T10:40:35.564828+0000","last_undegraded":"2023-10-06T10:40:35.564828+0000","last_fullsized":"2023-10-06T10:40:35.564828+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:15.757865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576825+0000","last_change":"2023-10-06T10:40:35.571655+0000","last_active":"2023-10-06T10:40:35.576825+0000","last_peered":"2023-10-06T10:40:35.576825+0000","last_clean":"2023-10-06T10:40:35.576825+0000","last_became_active":"2023-10-06T10:40:35.565829+0000","last_became_peered":"2023-10-06T10:40:35.565829+0000","last_unstale":"2023-10-06T10:40:35.576825+0000","last_undegraded":"2023-10-06T10:40:35.576825+0000","last_fullsized":"2023-10-06T10:40:35.576825+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:13.916559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569860+0000","last_change":"2023-10-06T10:40:35.566171+0000","last_active":"2023-10-06T10:40:35.569860+0000","last_peered":"2023-10-06T10:40:35.569860+0000","last_clean":"2023-10-06T10:40:35.569860+0000","last_became_active":"2023-10-06T10:40:35.565812+0000","last_became_peered":"2023-10-06T10:40:35.565812+0000","last_unstale":"2023-10-06T10:40:35.569860+0000","last_undegraded":"2023-10-06T10:40:35.569860+0000","last_fullsized":"2023-10-06T10:40:35.569860+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:10.010731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.574755+0000","last_change":"2023-10-06T10:40:35.566646+0000","last_active":"2023-10-06T10:40:35.574755+0000","last_peered":"2023-10-06T10:40:35.574755+0000","last_clean":"2023-10-06T10:40:35.574755+0000","last_became_active":"2023-10-06T10:40:35.566383+0000","last_became_peered":"2023-10-06T10:40:35.566383+0000","last_unstale":"2023-10-06T10:40:35.574755+0000","last_undegraded":"2023-10-06T10:40:35.574755+0000","last_fullsized":"2023-10-06T10:40:35.574755+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:27.389629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.577321+0000","last_change":"2023-10-06T10:40:35.576738+0000","last_active":"2023-10-06T10:40:35.577321+0000","last_peered":"2023-10-06T10:40:35.577321+0000","last_clean":"2023-10-06T10:40:35.577321+0000","last_became_active":"2023-10-06T10:40:35.576470+0000","last_became_peered":"2023-10-06T10:40:35.576470+0000","last_unstale":"2023-10-06T10:40:35.577321+0000","last_undegraded":"2023-10-06T10:40:35.577321+0000","last_fullsized":"2023-10-06T10:40:35.577321+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:34:51.555583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.564516+0000","last_change":"2023-10-06T10:40:35.558573+0000","last_active":"2023-10-06T10:40:35.564516+0000","last_peered":"2023-10-06T10:40:35.564516+0000","last_clean":"2023-10-06T10:40:35.564516+0000","last_became_active":"2023-10-06T10:40:35.558278+0000","last_became_peered":"2023-10-06T10:40:35.558278+0000","last_unstale":"2023-10-06T10:40:35.564516+0000","last_undegraded":"2023-10-06T10:40:35.564516+0000","last_fullsized":"2023-10-06T10:40:35.564516+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:22.905054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.558118+0000","last_change":"2023-10-06T10:40:35.555626+0000","last_active":"2023-10-06T10:40:35.558118+0000","last_peered":"2023-10-06T10:40:35.558118+0000","last_clean":"2023-10-06T10:40:35.558118+0000","last_became_active":"2023-10-06T10:40:35.555367+0000","last_became_peered":"2023-10-06T10:40:35.555367+0000","last_unstale":"2023-10-06T10:40:35.558118+0000","last_undegraded":"2023-10-06T10:40:35.558118+0000","last_fullsized":"2023-10-06T10:40:35.558118+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:22.429465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568058+0000","last_change":"2023-10-06T10:40:35.560779+0000","last_active":"2023-10-06T10:40:35.568058+0000","last_peered":"2023-10-06T10:40:35.568058+0000","last_clean":"2023-10-06T10:40:35.568058+0000","last_became_active":"2023-10-06T10:40:35.560517+0000","last_became_peered":"2023-10-06T10:40:35.560517+0000","last_unstale":"2023-10-06T10:40:35.568058+0000","last_undegraded":"2023-10-06T10:40:35.568058+0000","last_fullsized":"2023-10-06T10:40:35.568058+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:31.413269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568008+0000","last_change":"2023-10-06T10:40:35.559791+0000","last_active":"2023-10-06T10:40:35.568008+0000","last_peered":"2023-10-06T10:40:35.568008+0000","last_clean":"2023-10-06T10:40:35.568008+0000","last_became_active":"2023-10-06T10:40:35.559541+0000","last_became_peered":"2023-10-06T10:40:35.559541+0000","last_unstale":"2023-10-06T10:40:35.568008+0000","last_undegraded":"2023-10-06T10:40:35.568008+0000","last_fullsized":"2023-10-06T10:40:35.568008+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:48.719683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.572860+0000","last_change":"2023-10-06T10:40:35.562284+0000","last_active":"2023-10-06T10:40:35.572860+0000","last_peered":"2023-10-06T10:40:35.572860+0000","last_clean":"2023-10-06T10:40:35.572860+0000","last_became_active":"2023-10-06T10:40:35.561931+0000","last_became_peered":"2023-10-06T10:40:35.561931+0000","last_unstale":"2023-10-06T10:40:35.572860+0000","last_undegraded":"2023-10-06T10:40:35.572860+0000","last_fullsized":"2023-10-06T10:40:35.572860+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:55.843404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576252+0000","last_change":"2023-10-06T10:40:35.567770+0000","last_active":"2023-10-06T10:40:35.576252+0000","last_peered":"2023-10-06T10:40:35.576252+0000","last_clean":"2023-10-06T10:40:35.576252+0000","last_became_active":"2023-10-06T10:40:35.566223+0000","last_became_peered":"2023-10-06T10:40:35.566223+0000","last_unstale":"2023-10-06T10:40:35.576252+0000","last_undegraded":"2023-10-06T10:40:35.576252+0000","last_fullsized":"2023-10-06T10:40:35.576252+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:56.036704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.572361+0000","last_change":"2023-10-06T10:40:35.569168+0000","last_active":"2023-10-06T10:40:35.572361+0000","last_peered":"2023-10-06T10:40:35.572361+0000","last_clean":"2023-10-06T10:40:35.572361+0000","last_became_active":"2023-10-06T10:40:35.568859+0000","last_became_peered":"2023-10-06T10:40:35.568859+0000","last_unstale":"2023-10-06T10:40:35.572361+0000","last_undegraded":"2023-10-06T10:40:35.572361+0000","last_fullsized":"2023-10-06T10:40:35.572361+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:28:38.929115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569862+0000","last_change":"2023-10-06T10:40:35.566851+0000","last_active":"2023-10-06T10:40:35.569862+0000","last_peered":"2023-10-06T10:40:35.569862+0000","last_clean":"2023-10-06T10:40:35.569862+0000","last_became_active":"2023-10-06T10:40:35.566568+0000","last_became_peered":"2023-10-06T10:40:35.566568+0000","last_unstale":"2023-10-06T10:40:35.569862+0000","last_undegraded":"2023-10-06T10:40:35.569862+0000","last_fullsized":"2023-10-06T10:40:35.569862+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:50.111333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576908+0000","last_change":"2023-10-06T10:40:35.573233+0000","last_active":"2023-10-06T10:40:35.576908+0000","last_peered":"2023-10-06T10:40:35.576908+0000","last_clean":"2023-10-06T10:40:35.576908+0000","last_became_active":"2023-10-06T10:40:35.572994+0000","last_became_peered":"2023-10-06T10:40:35.572994+0000","last_unstale":"2023-10-06T10:40:35.576908+0000","last_undegraded":"2023-10-06T10:40:35.576908+0000","last_fullsized":"2023-10-06T10:40:35.576908+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:45:01.289103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.570019+0000","last_change":"2023-10-06T10:40:35.567541+0000","last_active":"2023-10-06T10:40:35.570019+0000","last_peered":"2023-10-06T10:40:35.570019+0000","last_clean":"2023-10-06T10:40:35.570019+0000","last_became_active":"2023-10-06T10:40:35.567165+0000","last_became_peered":"2023-10-06T10:40:35.567165+0000","last_unstale":"2023-10-06T10:40:35.570019+0000","last_undegraded":"2023-10-06T10:40:35.570019+0000","last_fullsized":"2023-10-06T10:40:35.570019+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:42.478496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576376+0000","last_change":"2023-10-06T10:40:35.568241+0000","last_active":"2023-10-06T10:40:35.576376+0000","last_peered":"2023-10-06T10:40:35.576376+0000","last_clean":"2023-10-06T10:40:35.576376+0000","last_became_active":"2023-10-06T10:40:35.568006+0000","last_became_peered":"2023-10-06T10:40:35.568006+0000","last_unstale":"2023-10-06T10:40:35.576376+0000","last_undegraded":"2023-10-06T10:40:35.576376+0000","last_fullsized":"2023-10-06T10:40:35.576376+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:16:58.559793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.577728+0000","last_change":"2023-10-06T10:40:35.577097+0000","last_active":"2023-10-06T10:40:35.577728+0000","last_peered":"2023-10-06T10:40:35.577728+0000","last_clean":"2023-10-06T10:40:35.577728+0000","last_became_active":"2023-10-06T10:40:35.576788+0000","last_became_peered":"2023-10-06T10:40:35.576788+0000","last_unstale":"2023-10-06T10:40:35.577728+0000","last_undegraded":"2023-10-06T10:40:35.577728+0000","last_fullsized":"2023-10-06T10:40:35.577728+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:47.164234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.563619+0000","last_change":"2023-10-06T10:40:35.562191+0000","last_active":"2023-10-06T10:40:35.563619+0000","last_peered":"2023-10-06T10:40:35.563619+0000","last_clean":"2023-10-06T10:40:35.563619+0000","last_became_active":"2023-10-06T10:40:35.561940+0000","last_became_peered":"2023-10-06T10:40:35.561940+0000","last_unstale":"2023-10-06T10:40:35.563619+0000","last_undegraded":"2023-10-06T10:40:35.563619+0000","last_fullsized":"2023-10-06T10:40:35.563619+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:40.960612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569894+0000","last_change":"2023-10-06T10:40:35.564934+0000","last_active":"2023-10-06T10:40:35.569894+0000","last_peered":"2023-10-06T10:40:35.569894+0000","last_clean":"2023-10-06T10:40:35.569894+0000","last_became_active":"2023-10-06T10:40:35.564686+0000","last_became_peered":"2023-10-06T10:40:35.564686+0000","last_unstale":"2023-10-06T10:40:35.569894+0000","last_undegraded":"2023-10-06T10:40:35.569894+0000","last_fullsized":"2023-10-06T10:40:35.569894+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:03.867863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.564523+0000","last_change":"2023-10-06T10:40:35.557812+0000","last_active":"2023-10-06T10:40:35.564523+0000","last_peered":"2023-10-06T10:40:35.564523+0000","last_clean":"2023-10-06T10:40:35.564523+0000","last_became_active":"2023-10-06T10:40:35.557534+0000","last_became_peered":"2023-10-06T10:40:35.557534+0000","last_unstale":"2023-10-06T10:40:35.564523+0000","last_undegraded":"2023-10-06T10:40:35.564523+0000","last_fullsized":"2023-10-06T10:40:35.564523+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:16:56.359962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569247+0000","last_change":"2023-10-06T10:40:35.562339+0000","last_active":"2023-10-06T10:40:35.569247+0000","last_peered":"2023-10-06T10:40:35.569247+0000","last_clean":"2023-10-06T10:40:35.569247+0000","last_became_active":"2023-10-06T10:40:35.562014+0000","last_became_peered":"2023-10-06T10:40:35.562014+0000","last_unstale":"2023-10-06T10:40:35.569247+0000","last_undegraded":"2023-10-06T10:40:35.569247+0000","last_fullsized":"2023-10-06T10:40:35.569247+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:55.143518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.558884+0000","last_change":"2023-10-06T10:40:35.555416+0000","last_active":"2023-10-06T10:40:35.558884+0000","last_peered":"2023-10-06T10:40:35.558884+0000","last_clean":"2023-10-06T10:40:35.558884+0000","last_became_active":"2023-10-06T10:40:35.555169+0000","last_became_peered":"2023-10-06T10:40:35.555169+0000","last_unstale":"2023-10-06T10:40:35.558884+0000","last_undegraded":"2023-10-06T10:40:35.558884+0000","last_fullsized":"2023-10-06T10:40:35.558884+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:11.988774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576530+0000","last_change":"2023-10-06T10:40:35.571179+0000","last_active":"2023-10-06T10:40:35.576530+0000","last_peered":"2023-10-06T10:40:35.576530+0000","last_clean":"2023-10-06T10:40:35.576530+0000","last_became_active":"2023-10-06T10:40:35.570784+0000","last_became_peered":"2023-10-06T10:40:35.570784+0000","last_unstale":"2023-10-06T10:40:35.576530+0000","last_undegraded":"2023-10-06T10:40:35.576530+0000","last_fullsized":"2023-10-06T10:40:35.576530+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:29:56.451363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569044+0000","last_change":"2023-10-06T10:40:35.564496+0000","last_active":"2023-10-06T10:40:35.569044+0000","last_peered":"2023-10-06T10:40:35.569044+0000","last_clean":"2023-10-06T10:40:35.569044+0000","last_became_active":"2023-10-06T10:40:35.564215+0000","last_became_peered":"2023-10-06T10:40:35.564215+0000","last_unstale":"2023-10-06T10:40:35.569044+0000","last_undegraded":"2023-10-06T10:40:35.569044+0000","last_fullsized":"2023-10-06T10:40:35.569044+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:06.273416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569438+0000","last_change":"2023-10-06T10:40:35.564889+0000","last_active":"2023-10-06T10:40:35.569438+0000","last_peered":"2023-10-06T10:40:35.569438+0000","last_clean":"2023-10-06T10:40:35.569438+0000","last_became_active":"2023-10-06T10:40:35.564630+0000","last_became_peered":"2023-10-06T10:40:35.564630+0000","last_unstale":"2023-10-06T10:40:35.569438+0000","last_undegraded":"2023-10-06T10:40:35.569438+0000","last_fullsized":"2023-10-06T10:40:35.569438+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:49.024348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.575282+0000","last_change":"2023-10-06T10:40:35.573554+0000","last_active":"2023-10-06T10:40:35.575282+0000","last_peered":"2023-10-06T10:40:35.575282+0000","last_clean":"2023-10-06T10:40:35.575282+0000","last_became_active":"2023-10-06T10:40:35.573307+0000","last_became_peered":"2023-10-06T10:40:35.573307+0000","last_unstale":"2023-10-06T10:40:35.575282+0000","last_undegraded":"2023-10-06T10:40:35.575282+0000","last_fullsized":"2023-10-06T10:40:35.575282+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:47.765804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568816+0000","last_change":"2023-10-06T10:40:35.564393+0000","last_active":"2023-10-06T10:40:35.568816+0000","last_peered":"2023-10-06T10:40:35.568816+0000","last_clean":"2023-10-06T10:40:35.568816+0000","last_became_active":"2023-10-06T10:40:35.564157+0000","last_became_peered":"2023-10-06T10:40:35.564157+0000","last_unstale":"2023-10-06T10:40:35.568816+0000","last_undegraded":"2023-10-06T10:40:35.568816+0000","last_fullsized":"2023-10-06T10:40:35.568816+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:06:14.418093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"51'4","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568646+0000","last_change":"2023-10-06T10:40:34.559819+0000","last_active":"2023-10-06T10:40:35.568646+0000","last_peered":"2023-10-06T10:40:35.568646+0000","last_clean":"2023-10-06T10:40:35.568646+0000","last_became_active":"2023-10-06T10:40:34.559433+0000","last_became_peered":"2023-10-06T10:40:34.559433+0000","last_unstale":"2023-10-06T10:40:35.568646+0000","last_undegraded":"2023-10-06T10:40:35.568646+0000","last_fullsized":"2023-10-06T10:40:35.568646+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:43:52.519978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.574934+0000","last_change":"2023-10-06T10:40:35.572023+0000","last_active":"2023-10-06T10:40:35.574934+0000","last_peered":"2023-10-06T10:40:35.574934+0000","last_clean":"2023-10-06T10:40:35.574934+0000","last_became_active":"2023-10-06T10:40:35.571788+0000","last_became_peered":"2023-10-06T10:40:35.571788+0000","last_unstale":"2023-10-06T10:40:35.574934+0000","last_undegraded":"2023-10-06T10:40:35.574934+0000","last_fullsized":"2023-10-06T10:40:35.574934+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:13.245750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"51'1","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.575594+0000","last_change":"2023-10-06T10:40:34.562460+0000","last_active":"2023-10-06T10:40:35.575594+0000","last_peered":"2023-10-06T10:40:35.575594+0000","last_clean":"2023-10-06T10:40:35.575594+0000","last_became_active":"2023-10-06T10:40:34.562241+0000","last_became_peered":"2023-10-06T10:40:34.562241+0000","last_unstale":"2023-10-06T10:40:35.575594+0000","last_undegraded":"2023-10-06T10:40:35.575594+0000","last_fullsized":"2023-10-06T10:40:35.575594+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:15.486524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.572595+0000","last_change":"2023-10-06T10:40:35.560246+0000","last_active":"2023-10-06T10:40:35.572595+0000","last_peered":"2023-10-06T10:40:35.572595+0000","last_clean":"2023-10-06T10:40:35.572595+0000","last_became_active":"2023-10-06T10:40:35.559981+0000","last_became_peered":"2023-10-06T10:40:35.559981+0000","last_unstale":"2023-10-06T10:40:35.572595+0000","last_undegraded":"2023-10-06T10:40:35.572595+0000","last_fullsized":"2023-10-06T10:40:35.572595+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:35.748566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"51'2","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.560216+0000","last_change":"2023-10-06T10:40:34.560326+0000","last_active":"2023-10-06T10:40:35.560216+0000","last_peered":"2023-10-06T10:40:35.560216+0000","last_clean":"2023-10-06T10:40:35.560216+0000","last_became_active":"2023-10-06T10:40:34.559983+0000","last_became_peered":"2023-10-06T10:40:34.559983+0000","last_unstale":"2023-10-06T10:40:35.560216+0000","last_undegraded":"2023-10-06T10:40:35.560216+0000","last_fullsized":"2023-10-06T10:40:35.560216+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:55:42.716617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.567779+0000","last_change":"2023-10-06T10:40:35.558007+0000","last_active":"2023-10-06T10:40:35.567779+0000","last_peered":"2023-10-06T10:40:35.567779+0000","last_clean":"2023-10-06T10:40:35.567779+0000","last_became_active":"2023-10-06T10:40:35.557606+0000","last_became_peered":"2023-10-06T10:40:35.557606+0000","last_unstale":"2023-10-06T10:40:35.567779+0000","last_undegraded":"2023-10-06T10:40:35.567779+0000","last_fullsized":"2023-10-06T10:40:35.567779+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:31.413269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"51'2","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.558037+0000","last_change":"2023-10-06T10:40:34.562219+0000","last_active":"2023-10-06T10:40:35.558037+0000","last_peered":"2023-10-06T10:40:35.558037+0000","last_clean":"2023-10-06T10:40:35.558037+0000","last_became_active":"2023-10-06T10:40:34.561960+0000","last_became_peered":"2023-10-06T10:40:34.561960+0000","last_unstale":"2023-10-06T10:40:35.558037+0000","last_undegraded":"2023-10-06T10:40:35.558037+0000","last_fullsized":"2023-10-06T10:40:35.558037+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:14.711366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576398+0000","last_change":"2023-10-06T10:40:35.572126+0000","last_active":"2023-10-06T10:40:35.576398+0000","last_peered":"2023-10-06T10:40:35.576398+0000","last_clean":"2023-10-06T10:40:35.576398+0000","last_became_active":"2023-10-06T10:40:35.571887+0000","last_became_peered":"2023-10-06T10:40:35.571887+0000","last_unstale":"2023-10-06T10:40:35.576398+0000","last_undegraded":"2023-10-06T10:40:35.576398+0000","last_fullsized":"2023-10-06T10:40:35.576398+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:47.765804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"51'1","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.560679+0000","last_change":"2023-10-06T10:40:34.557170+0000","last_active":"2023-10-06T10:40:35.560679+0000","last_peered":"2023-10-06T10:40:35.560679+0000","last_clean":"2023-10-06T10:40:35.560679+0000","last_became_active":"2023-10-06T10:40:34.556834+0000","last_became_peered":"2023-10-06T10:40:34.556834+0000","last_unstale":"2023-10-06T10:40:35.560679+0000","last_undegraded":"2023-10-06T10:40:35.560679+0000","last_fullsized":"2023-10-06T10:40:35.560679+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:38.780441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576498+0000","last_change":"2023-10-06T10:40:35.571018+0000","last_active":"2023-10-06T10:40:35.576498+0000","last_peered":"2023-10-06T10:40:35.576498+0000","last_clean":"2023-10-06T10:40:35.576498+0000","last_became_active":"2023-10-06T10:40:35.570523+0000","last_became_peered":"2023-10-06T10:40:35.570523+0000","last_unstale":"2023-10-06T10:40:35.576498+0000","last_undegraded":"2023-10-06T10:40:35.576498+0000","last_fullsized":"2023-10-06T10:40:35.576498+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:53.209242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"51'1","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568890+0000","last_change":"2023-10-06T10:40:34.555165+0000","last_active":"2023-10-06T10:40:35.568890+0000","last_peered":"2023-10-06T10:40:35.568890+0000","last_clean":"2023-10-06T10:40:35.568890+0000","last_became_active":"2023-10-06T10:40:34.554897+0000","last_became_peered":"2023-10-06T10:40:34.554897+0000","last_unstale":"2023-10-06T10:40:35.568890+0000","last_undegraded":"2023-10-06T10:40:35.568890+0000","last_fullsized":"2023-10-06T10:40:35.568890+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:05.264641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569444+0000","last_change":"2023-10-06T10:40:35.565210+0000","last_active":"2023-10-06T10:40:35.569444+0000","last_peered":"2023-10-06T10:40:35.569444+0000","last_clean":"2023-10-06T10:40:35.569444+0000","last_became_active":"2023-10-06T10:40:35.564970+0000","last_became_peered":"2023-10-06T10:40:35.564970+0000","last_unstale":"2023-10-06T10:40:35.569444+0000","last_undegraded":"2023-10-06T10:40:35.569444+0000","last_fullsized":"2023-10-06T10:40:35.569444+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:40.976608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.546745+0000","last_change":"2023-10-06T10:40:34.558169+0000","last_active":"2023-10-06T10:40:35.546745+0000","last_peered":"2023-10-06T10:40:35.546745+0000","last_clean":"2023-10-06T10:40:35.546745+0000","last_became_active":"2023-10-06T10:40:34.557947+0000","last_became_peered":"2023-10-06T10:40:34.557947+0000","last_unstale":"2023-10-06T10:40:35.546745+0000","last_undegraded":"2023-10-06T10:40:35.546745+0000","last_fullsized":"2023-10-06T10:40:35.546745+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:56.154679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576422+0000","last_change":"2023-10-06T10:40:35.572595+0000","last_active":"2023-10-06T10:40:35.576422+0000","last_peered":"2023-10-06T10:40:35.576422+0000","last_clean":"2023-10-06T10:40:35.576422+0000","last_became_active":"2023-10-06T10:40:35.572317+0000","last_became_peered":"2023-10-06T10:40:35.572317+0000","last_unstale":"2023-10-06T10:40:35.576422+0000","last_undegraded":"2023-10-06T10:40:35.576422+0000","last_fullsized":"2023-10-06T10:40:35.576422+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:17.634580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.547860+0000","last_change":"2023-10-06T10:40:34.556185+0000","last_active":"2023-10-06T10:40:35.547860+0000","last_peered":"2023-10-06T10:40:35.547860+0000","last_clean":"2023-10-06T10:40:35.547860+0000","last_became_active":"2023-10-06T10:40:34.555911+0000","last_became_peered":"2023-10-06T10:40:34.555911+0000","last_unstale":"2023-10-06T10:40:35.547860+0000","last_undegraded":"2023-10-06T10:40:35.547860+0000","last_fullsized":"2023-10-06T10:40:35.547860+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.285921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568612+0000","last_change":"2023-10-06T10:40:35.562668+0000","last_active":"2023-10-06T10:40:35.568612+0000","last_peered":"2023-10-06T10:40:35.568612+0000","last_clean":"2023-10-06T10:40:35.568612+0000","last_became_active":"2023-10-06T10:40:35.562428+0000","last_became_peered":"2023-10-06T10:40:35.562428+0000","last_unstale":"2023-10-06T10:40:35.568612+0000","last_undegraded":"2023-10-06T10:40:35.568612+0000","last_fullsized":"2023-10-06T10:40:35.568612+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:57:07.950757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.555289+0000","last_change":"2023-10-06T10:40:34.574923+0000","last_active":"2023-10-06T10:40:35.555289+0000","last_peered":"2023-10-06T10:40:35.555289+0000","last_clean":"2023-10-06T10:40:35.555289+0000","last_became_active":"2023-10-06T10:40:34.574684+0000","last_became_peered":"2023-10-06T10:40:34.574684+0000","last_unstale":"2023-10-06T10:40:35.555289+0000","last_undegraded":"2023-10-06T10:40:35.555289+0000","last_fullsized":"2023-10-06T10:40:35.555289+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:01:38.560878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.570091+0000","last_change":"2023-10-06T10:40:35.567735+0000","last_active":"2023-10-06T10:40:35.570091+0000","last_peered":"2023-10-06T10:40:35.570091+0000","last_clean":"2023-10-06T10:40:35.570091+0000","last_became_active":"2023-10-06T10:40:35.566258+0000","last_became_peered":"2023-10-06T10:40:35.566258+0000","last_unstale":"2023-10-06T10:40:35.570091+0000","last_undegraded":"2023-10-06T10:40:35.570091+0000","last_fullsized":"2023-10-06T10:40:35.570091+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:31.294697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.562660+0000","last_change":"2023-10-06T10:40:34.558613+0000","last_active":"2023-10-06T10:40:35.562660+0000","last_peered":"2023-10-06T10:40:35.562660+0000","last_clean":"2023-10-06T10:40:35.562660+0000","last_became_active":"2023-10-06T10:40:34.558128+0000","last_became_peered":"2023-10-06T10:40:34.558128+0000","last_unstale":"2023-10-06T10:40:35.562660+0000","last_undegraded":"2023-10-06T10:40:35.562660+0000","last_fullsized":"2023-10-06T10:40:35.562660+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:46.155459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.574833+0000","last_change":"2023-10-06T10:40:35.570712+0000","last_active":"2023-10-06T10:40:35.574833+0000","last_peered":"2023-10-06T10:40:35.574833+0000","last_clean":"2023-10-06T10:40:35.574833+0000","last_became_active":"2023-10-06T10:40:35.570450+0000","last_became_peered":"2023-10-06T10:40:35.570450+0000","last_unstale":"2023-10-06T10:40:35.574833+0000","last_undegraded":"2023-10-06T10:40:35.574833+0000","last_fullsized":"2023-10-06T10:40:35.574833+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:40.960612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"51'3","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.571980+0000","last_change":"2023-10-06T10:40:34.567975+0000","last_active":"2023-10-06T10:40:35.571980+0000","last_peered":"2023-10-06T10:40:35.571980+0000","last_clean":"2023-10-06T10:40:35.571980+0000","last_became_active":"2023-10-06T10:40:34.567763+0000","last_became_peered":"2023-10-06T10:40:34.567763+0000","last_unstale":"2023-10-06T10:40:35.571980+0000","last_undegraded":"2023-10-06T10:40:35.571980+0000","last_fullsized":"2023-10-06T10:40:35.571980+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:26.380853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569050+0000","last_change":"2023-10-06T10:40:35.563961+0000","last_active":"2023-10-06T10:40:35.569050+0000","last_peered":"2023-10-06T10:40:35.569050+0000","last_clean":"2023-10-06T10:40:35.569050+0000","last_became_active":"2023-10-06T10:40:35.563735+0000","last_became_peered":"2023-10-06T10:40:35.563735+0000","last_unstale":"2023-10-06T10:40:35.569050+0000","last_undegraded":"2023-10-06T10:40:35.569050+0000","last_fullsized":"2023-10-06T10:40:35.569050+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:40:08.170068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"51'6","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.579117+0000","last_change":"2023-10-06T10:40:34.555718+0000","last_active":"2023-10-06T10:40:35.579117+0000","last_peered":"2023-10-06T10:40:35.579117+0000","last_clean":"2023-10-06T10:40:35.579117+0000","last_became_active":"2023-10-06T10:40:34.555357+0000","last_became_peered":"2023-10-06T10:40:34.555357+0000","last_unstale":"2023-10-06T10:40:35.579117+0000","last_undegraded":"2023-10-06T10:40:35.579117+0000","last_fullsized":"2023-10-06T10:40:35.579117+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:46.155459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.564769+0000","last_change":"2023-10-06T10:40:35.562704+0000","last_active":"2023-10-06T10:40:35.564769+0000","last_peered":"2023-10-06T10:40:35.564769+0000","last_clean":"2023-10-06T10:40:35.564769+0000","last_became_active":"2023-10-06T10:40:35.562424+0000","last_became_peered":"2023-10-06T10:40:35.562424+0000","last_unstale":"2023-10-06T10:40:35.564769+0000","last_undegraded":"2023-10-06T10:40:35.564769+0000","last_fullsized":"2023-10-06T10:40:35.564769+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:14.394504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"51'2","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.575856+0000","last_change":"2023-10-06T10:40:34.558595+0000","last_active":"2023-10-06T10:40:35.575856+0000","last_peered":"2023-10-06T10:40:35.575856+0000","last_clean":"2023-10-06T10:40:35.575856+0000","last_became_active":"2023-10-06T10:40:34.558324+0000","last_became_peered":"2023-10-06T10:40:34.558324+0000","last_unstale":"2023-10-06T10:40:35.575856+0000","last_undegraded":"2023-10-06T10:40:35.575856+0000","last_fullsized":"2023-10-06T10:40:35.575856+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.285921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.567395+0000","last_change":"2023-10-06T10:40:35.560500+0000","last_active":"2023-10-06T10:40:35.567395+0000","last_peered":"2023-10-06T10:40:35.567395+0000","last_clean":"2023-10-06T10:40:35.567395+0000","last_became_active":"2023-10-06T10:40:35.559739+0000","last_became_peered":"2023-10-06T10:40:35.559739+0000","last_unstale":"2023-10-06T10:40:35.567395+0000","last_undegraded":"2023-10-06T10:40:35.567395+0000","last_fullsized":"2023-10-06T10:40:35.567395+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:39.789216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.550323+0000","last_change":"2023-10-06T10:40:34.563002+0000","last_active":"2023-10-06T10:40:35.550323+0000","last_peered":"2023-10-06T10:40:35.550323+0000","last_clean":"2023-10-06T10:40:35.550323+0000","last_became_active":"2023-10-06T10:40:34.562670+0000","last_became_peered":"2023-10-06T10:40:34.562670+0000","last_unstale":"2023-10-06T10:40:35.550323+0000","last_undegraded":"2023-10-06T10:40:35.550323+0000","last_fullsized":"2023-10-06T10:40:35.550323+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:26.380853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569802+0000","last_change":"2023-10-06T10:40:35.565320+0000","last_active":"2023-10-06T10:40:35.569802+0000","last_peered":"2023-10-06T10:40:35.569802+0000","last_clean":"2023-10-06T10:40:35.569802+0000","last_became_active":"2023-10-06T10:40:35.565079+0000","last_became_peered":"2023-10-06T10:40:35.565079+0000","last_unstale":"2023-10-06T10:40:35.569802+0000","last_undegraded":"2023-10-06T10:40:35.569802+0000","last_fullsized":"2023-10-06T10:40:35.569802+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:14.394504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.546791+0000","last_change":"2023-10-06T10:40:34.558954+0000","last_active":"2023-10-06T10:40:35.546791+0000","last_peered":"2023-10-06T10:40:35.546791+0000","last_clean":"2023-10-06T10:40:35.546791+0000","last_became_active":"2023-10-06T10:40:34.558311+0000","last_became_peered":"2023-10-06T10:40:34.558311+0000","last_unstale":"2023-10-06T10:40:35.546791+0000","last_undegraded":"2023-10-06T10:40:35.546791+0000","last_fullsized":"2023-10-06T10:40:35.546791+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:03.396798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.577709+0000","last_change":"2023-10-06T10:40:35.577044+0000","last_active":"2023-10-06T10:40:35.577709+0000","last_peered":"2023-10-06T10:40:35.577709+0000","last_clean":"2023-10-06T10:40:35.577709+0000","last_became_active":"2023-10-06T10:40:35.576791+0000","last_became_peered":"2023-10-06T10:40:35.576791+0000","last_unstale":"2023-10-06T10:40:35.577709+0000","last_undegraded":"2023-10-06T10:40:35.577709+0000","last_fullsized":"2023-10-06T10:40:35.577709+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:55:43.725392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"51'8","reported_seq":18,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568288+0000","last_change":"2023-10-06T10:40:34.562481+0000","last_active":"2023-10-06T10:40:35.568288+0000","last_peered":"2023-10-06T10:40:35.568288+0000","last_clean":"2023-10-06T10:40:35.568288+0000","last_became_active":"2023-10-06T10:40:34.562181+0000","last_became_peered":"2023-10-06T10:40:34.562181+0000","last_unstale":"2023-10-06T10:40:35.568288+0000","last_undegraded":"2023-10-06T10:40:35.568288+0000","last_fullsized":"2023-10-06T10:40:35.568288+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:04:34.719623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.566714+0000","last_change":"2023-10-06T10:40:35.562814+0000","last_active":"2023-10-06T10:40:35.566714+0000","last_peered":"2023-10-06T10:40:35.566714+0000","last_clean":"2023-10-06T10:40:35.566714+0000","last_became_active":"2023-10-06T10:40:35.562556+0000","last_became_peered":"2023-10-06T10:40:35.562556+0000","last_unstale":"2023-10-06T10:40:35.566714+0000","last_undegraded":"2023-10-06T10:40:35.566714+0000","last_fullsized":"2023-10-06T10:40:35.566714+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:48.719683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"51'2","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576493+0000","last_change":"2023-10-06T10:40:34.561496+0000","last_active":"2023-10-06T10:40:35.576493+0000","last_peered":"2023-10-06T10:40:35.576493+0000","last_clean":"2023-10-06T10:40:35.576493+0000","last_became_active":"2023-10-06T10:40:34.561276+0000","last_became_peered":"2023-10-06T10:40:34.561276+0000","last_unstale":"2023-10-06T10:40:35.576493+0000","last_undegraded":"2023-10-06T10:40:35.576493+0000","last_fullsized":"2023-10-06T10:40:35.576493+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:40.903040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.566308+0000","last_change":"2023-10-06T10:40:35.558844+0000","last_active":"2023-10-06T10:40:35.566308+0000","last_peered":"2023-10-06T10:40:35.566308+0000","last_clean":"2023-10-06T10:40:35.566308+0000","last_became_active":"2023-10-06T10:40:35.558577+0000","last_became_peered":"2023-10-06T10:40:35.558577+0000","last_unstale":"2023-10-06T10:40:35.566308+0000","last_undegraded":"2023-10-06T10:40:35.566308+0000","last_fullsized":"2023-10-06T10:40:35.566308+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:04:09.361668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.553797+0000","last_change":"2023-10-06T10:40:34.557813+0000","last_active":"2023-10-06T10:40:35.553797+0000","last_peered":"2023-10-06T10:40:35.553797+0000","last_clean":"2023-10-06T10:40:35.553797+0000","last_became_active":"2023-10-06T10:40:34.557512+0000","last_became_peered":"2023-10-06T10:40:34.557512+0000","last_unstale":"2023-10-06T10:40:35.553797+0000","last_undegraded":"2023-10-06T10:40:35.553797+0000","last_fullsized":"2023-10-06T10:40:35.553797+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:38.780441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.574854+0000","last_change":"2023-10-06T10:40:35.570944+0000","last_active":"2023-10-06T10:40:35.574854+0000","last_peered":"2023-10-06T10:40:35.574854+0000","last_clean":"2023-10-06T10:40:35.574854+0000","last_became_active":"2023-10-06T10:40:35.567192+0000","last_became_peered":"2023-10-06T10:40:35.567192+0000","last_unstale":"2023-10-06T10:40:35.574854+0000","last_undegraded":"2023-10-06T10:40:35.574854+0000","last_fullsized":"2023-10-06T10:40:35.574854+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:26.729670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.550419+0000","last_change":"2023-10-06T10:40:34.559497+0000","last_active":"2023-10-06T10:40:35.550419+0000","last_peered":"2023-10-06T10:40:35.550419+0000","last_clean":"2023-10-06T10:40:35.550419+0000","last_became_active":"2023-10-06T10:40:34.559159+0000","last_became_peered":"2023-10-06T10:40:34.559159+0000","last_unstale":"2023-10-06T10:40:35.550419+0000","last_undegraded":"2023-10-06T10:40:35.550419+0000","last_fullsized":"2023-10-06T10:40:35.550419+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:04:34.719623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.567761+0000","last_change":"2023-10-06T10:40:35.557851+0000","last_active":"2023-10-06T10:40:35.567761+0000","last_peered":"2023-10-06T10:40:35.567761+0000","last_clean":"2023-10-06T10:40:35.567761+0000","last_became_active":"2023-10-06T10:40:35.557289+0000","last_became_peered":"2023-10-06T10:40:35.557289+0000","last_unstale":"2023-10-06T10:40:35.567761+0000","last_undegraded":"2023-10-06T10:40:35.567761+0000","last_fullsized":"2023-10-06T10:40:35.567761+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:57:07.950757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.548294+0000","last_change":"2023-10-06T10:40:34.558179+0000","last_active":"2023-10-06T10:40:35.548294+0000","last_peered":"2023-10-06T10:40:35.548294+0000","last_clean":"2023-10-06T10:40:35.548294+0000","last_became_active":"2023-10-06T10:40:34.557924+0000","last_became_peered":"2023-10-06T10:40:34.557924+0000","last_unstale":"2023-10-06T10:40:35.548294+0000","last_undegraded":"2023-10-06T10:40:35.548294+0000","last_fullsized":"2023-10-06T10:40:35.548294+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:13.047456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576117+0000","last_change":"2023-10-06T10:40:35.565513+0000","last_active":"2023-10-06T10:40:35.576117+0000","last_peered":"2023-10-06T10:40:35.576117+0000","last_clean":"2023-10-06T10:40:35.576117+0000","last_became_active":"2023-10-06T10:40:35.565263+0000","last_became_peered":"2023-10-06T10:40:35.565263+0000","last_unstale":"2023-10-06T10:40:35.576117+0000","last_undegraded":"2023-10-06T10:40:35.576117+0000","last_fullsized":"2023-10-06T10:40:35.576117+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:12:37.044769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.559092+0000","last_change":"2023-10-06T10:40:34.559598+0000","last_active":"2023-10-06T10:40:35.559092+0000","last_peered":"2023-10-06T10:40:35.559092+0000","last_clean":"2023-10-06T10:40:35.559092+0000","last_became_active":"2023-10-06T10:40:34.559338+0000","last_became_peered":"2023-10-06T10:40:34.559338+0000","last_unstale":"2023-10-06T10:40:35.559092+0000","last_undegraded":"2023-10-06T10:40:35.559092+0000","last_fullsized":"2023-10-06T10:40:35.559092+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:10.979999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.577733+0000","last_change":"2023-10-06T10:40:35.577132+0000","last_active":"2023-10-06T10:40:35.577733+0000","last_peered":"2023-10-06T10:40:35.577733+0000","last_clean":"2023-10-06T10:40:35.577733+0000","last_became_active":"2023-10-06T10:40:35.576860+0000","last_became_peered":"2023-10-06T10:40:35.576860+0000","last_unstale":"2023-10-06T10:40:35.577733+0000","last_undegraded":"2023-10-06T10:40:35.577733+0000","last_fullsized":"2023-10-06T10:40:35.577733+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:16.495299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.548028+0000","last_change":"2023-10-06T10:40:34.555634+0000","last_active":"2023-10-06T10:40:35.548028+0000","last_peered":"2023-10-06T10:40:35.548028+0000","last_clean":"2023-10-06T10:40:35.548028+0000","last_became_active":"2023-10-06T10:40:34.555335+0000","last_became_peered":"2023-10-06T10:40:34.555335+0000","last_unstale":"2023-10-06T10:40:35.548028+0000","last_undegraded":"2023-10-06T10:40:35.548028+0000","last_fullsized":"2023-10-06T10:40:35.548028+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:55:42.716617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.574756+0000","last_change":"2023-10-06T10:40:35.566701+0000","last_active":"2023-10-06T10:40:35.574756+0000","last_peered":"2023-10-06T10:40:35.574756+0000","last_clean":"2023-10-06T10:40:35.574756+0000","last_became_active":"2023-10-06T10:40:35.566383+0000","last_became_peered":"2023-10-06T10:40:35.566383+0000","last_unstale":"2023-10-06T10:40:35.574756+0000","last_undegraded":"2023-10-06T10:40:35.574756+0000","last_fullsized":"2023-10-06T10:40:35.574756+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:01:39.569654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.559099+0000","last_change":"2023-10-06T10:40:34.558852+0000","last_active":"2023-10-06T10:40:35.559099+0000","last_peered":"2023-10-06T10:40:35.559099+0000","last_clean":"2023-10-06T10:40:35.559099+0000","last_became_active":"2023-10-06T10:40:34.558581+0000","last_became_peered":"2023-10-06T10:40:34.558581+0000","last_unstale":"2023-10-06T10:40:35.559099+0000","last_undegraded":"2023-10-06T10:40:35.559099+0000","last_fullsized":"2023-10-06T10:40:35.559099+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:49.102558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576292+0000","last_change":"2023-10-06T10:40:35.567776+0000","last_active":"2023-10-06T10:40:35.576292+0000","last_peered":"2023-10-06T10:40:35.576292+0000","last_clean":"2023-10-06T10:40:35.576292+0000","last_became_active":"2023-10-06T10:40:35.567440+0000","last_became_peered":"2023-10-06T10:40:35.567440+0000","last_unstale":"2023-10-06T10:40:35.576292+0000","last_undegraded":"2023-10-06T10:40:35.576292+0000","last_fullsized":"2023-10-06T10:40:35.576292+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:15.268020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.556015+0000","last_change":"2023-10-06T10:40:34.559435+0000","last_active":"2023-10-06T10:40:35.556015+0000","last_peered":"2023-10-06T10:40:35.556015+0000","last_clean":"2023-10-06T10:40:35.556015+0000","last_became_active":"2023-10-06T10:40:34.559184+0000","last_became_peered":"2023-10-06T10:40:34.559184+0000","last_unstale":"2023-10-06T10:40:35.556015+0000","last_undegraded":"2023-10-06T10:40:35.556015+0000","last_fullsized":"2023-10-06T10:40:35.556015+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:28:37.920340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.573009+0000","last_change":"2023-10-06T10:40:35.572131+0000","last_active":"2023-10-06T10:40:35.573009+0000","last_peered":"2023-10-06T10:40:35.573009+0000","last_clean":"2023-10-06T10:40:35.573009+0000","last_became_active":"2023-10-06T10:40:35.571859+0000","last_became_peered":"2023-10-06T10:40:35.571859+0000","last_unstale":"2023-10-06T10:40:35.573009+0000","last_undegraded":"2023-10-06T10:40:35.573009+0000","last_fullsized":"2023-10-06T10:40:35.573009+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:45.782226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.550794+0000","last_change":"2023-10-06T10:40:34.561843+0000","last_active":"2023-10-06T10:40:35.550794+0000","last_peered":"2023-10-06T10:40:35.550794+0000","last_clean":"2023-10-06T10:40:35.550794+0000","last_became_active":"2023-10-06T10:40:34.561550+0000","last_became_peered":"2023-10-06T10:40:34.561550+0000","last_unstale":"2023-10-06T10:40:35.550794+0000","last_undegraded":"2023-10-06T10:40:35.550794+0000","last_fullsized":"2023-10-06T10:40:35.550794+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:14.711366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":43,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.547102+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:40:35.547102+0000","last_peered":"2023-10-06T10:40:35.547102+0000","last_clean":"2023-10-06T10:40:35.547102+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:40:35.547102+0000","last_undegraded":"2023-10-06T10:40:35.547102+0000","last_fullsized":"2023-10-06T10:40:35.547102+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576189+0000","last_change":"2023-10-06T10:40:35.566096+0000","last_active":"2023-10-06T10:40:35.576189+0000","last_peered":"2023-10-06T10:40:35.576189+0000","last_clean":"2023-10-06T10:40:35.576189+0000","last_became_active":"2023-10-06T10:40:35.565831+0000","last_became_peered":"2023-10-06T10:40:35.565831+0000","last_unstale":"2023-10-06T10:40:35.576189+0000","last_undegraded":"2023-10-06T10:40:35.576189+0000","last_fullsized":"2023-10-06T10:40:35.576189+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:35.748566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"51'1","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.556108+0000","last_change":"2023-10-06T10:40:34.562301+0000","last_active":"2023-10-06T10:40:35.556108+0000","last_peered":"2023-10-06T10:40:35.556108+0000","last_clean":"2023-10-06T10:40:35.556108+0000","last_became_active":"2023-10-06T10:40:34.562030+0000","last_became_peered":"2023-10-06T10:40:34.562030+0000","last_unstale":"2023-10-06T10:40:35.556108+0000","last_undegraded":"2023-10-06T10:40:35.556108+0000","last_fullsized":"2023-10-06T10:40:35.556108+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:43:52.519978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.563486+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:40:35.563486+0000","last_peered":"2023-10-06T10:40:35.563486+0000","last_clean":"2023-10-06T10:40:35.563486+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:40:35.563486+0000","last_undegraded":"2023-10-06T10:40:35.563486+0000","last_fullsized":"2023-10-06T10:40:35.563486+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568740+0000","last_change":"2023-10-06T10:40:35.561940+0000","last_active":"2023-10-06T10:40:35.568740+0000","last_peered":"2023-10-06T10:40:35.568740+0000","last_clean":"2023-10-06T10:40:35.568740+0000","last_became_active":"2023-10-06T10:40:35.561665+0000","last_became_peered":"2023-10-06T10:40:35.561665+0000","last_unstale":"2023-10-06T10:40:35.568740+0000","last_undegraded":"2023-10-06T10:40:35.568740+0000","last_fullsized":"2023-10-06T10:40:35.568740+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:33:37.853079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.559769+0000","last_change":"2023-10-06T10:40:34.562242+0000","last_active":"2023-10-06T10:40:35.559769+0000","last_peered":"2023-10-06T10:40:35.559769+0000","last_clean":"2023-10-06T10:40:35.559769+0000","last_became_active":"2023-10-06T10:40:34.562001+0000","last_became_peered":"2023-10-06T10:40:34.562001+0000","last_unstale":"2023-10-06T10:40:35.559769+0000","last_undegraded":"2023-10-06T10:40:35.559769+0000","last_fullsized":"2023-10-06T10:40:35.559769+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:48.015572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":41,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.559170+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:40:35.559170+0000","last_peered":"2023-10-06T10:40:35.559170+0000","last_clean":"2023-10-06T10:40:35.559170+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:40:35.559170+0000","last_undegraded":"2023-10-06T10:40:35.559170+0000","last_fullsized":"2023-10-06T10:40:35.559170+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":186,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.553015+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:40:35.553015+0000","last_peered":"2023-10-06T10:40:35.553015+0000","last_clean":"2023-10-06T10:40:35.553015+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:40:35.553015+0000","last_undegraded":"2023-10-06T10:40:35.553015+0000","last_fullsized":"2023-10-06T10:40:35.553015+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.564750+0000","last_change":"2023-10-06T10:40:35.561025+0000","last_active":"2023-10-06T10:40:35.564750+0000","last_peered":"2023-10-06T10:40:35.564750+0000","last_clean":"2023-10-06T10:40:35.564750+0000","last_became_active":"2023-10-06T10:40:35.560733+0000","last_became_peered":"2023-10-06T10:40:35.560733+0000","last_unstale":"2023-10-06T10:40:35.564750+0000","last_undegraded":"2023-10-06T10:40:35.564750+0000","last_fullsized":"2023-10-06T10:40:35.564750+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:15.757865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"51'2","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.575749+0000","last_change":"2023-10-06T10:40:34.550122+0000","last_active":"2023-10-06T10:40:35.575749+0000","last_peered":"2023-10-06T10:40:35.575749+0000","last_clean":"2023-10-06T10:40:35.575749+0000","last_became_active":"2023-10-06T10:40:34.549859+0000","last_became_peered":"2023-10-06T10:40:34.549859+0000","last_unstale":"2023-10-06T10:40:35.575749+0000","last_undegraded":"2023-10-06T10:40:35.575749+0000","last_fullsized":"2023-10-06T10:40:35.575749+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:44.773451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":36,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.553171+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:40:35.553171+0000","last_peered":"2023-10-06T10:40:35.553171+0000","last_clean":"2023-10-06T10:40:35.553171+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:40:35.553171+0000","last_undegraded":"2023-10-06T10:40:35.553171+0000","last_fullsized":"2023-10-06T10:40:35.553171+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576107+0000","last_change":"2023-10-06T10:40:35.565493+0000","last_active":"2023-10-06T10:40:35.576107+0000","last_peered":"2023-10-06T10:40:35.576107+0000","last_clean":"2023-10-06T10:40:35.576107+0000","last_became_active":"2023-10-06T10:40:35.565234+0000","last_became_peered":"2023-10-06T10:40:35.565234+0000","last_unstale":"2023-10-06T10:40:35.576107+0000","last_undegraded":"2023-10-06T10:40:35.576107+0000","last_fullsized":"2023-10-06T10:40:35.576107+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:48.719683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.550113+0000","last_change":"2023-10-06T10:40:34.559498+0000","last_active":"2023-10-06T10:40:35.550113+0000","last_peered":"2023-10-06T10:40:35.550113+0000","last_clean":"2023-10-06T10:40:35.550113+0000","last_became_active":"2023-10-06T10:40:34.559262+0000","last_became_peered":"2023-10-06T10:40:34.559262+0000","last_unstale":"2023-10-06T10:40:35.550113+0000","last_undegraded":"2023-10-06T10:40:35.550113+0000","last_fullsized":"2023-10-06T10:40:35.550113+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:27:47.178791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":36,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.553009+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:40:35.553009+0000","last_peered":"2023-10-06T10:40:35.553009+0000","last_clean":"2023-10-06T10:40:35.553009+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:40:35.553009+0000","last_undegraded":"2023-10-06T10:40:35.553009+0000","last_fullsized":"2023-10-06T10:40:35.553009+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.568628+0000","last_change":"2023-10-06T10:40:35.560232+0000","last_active":"2023-10-06T10:40:35.568628+0000","last_peered":"2023-10-06T10:40:35.568628+0000","last_clean":"2023-10-06T10:40:35.568628+0000","last_became_active":"2023-10-06T10:40:35.559942+0000","last_became_peered":"2023-10-06T10:40:35.559942+0000","last_unstale":"2023-10-06T10:40:35.568628+0000","last_undegraded":"2023-10-06T10:40:35.568628+0000","last_fullsized":"2023-10-06T10:40:35.568628+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:03.785102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.547498+0000","last_change":"2023-10-06T10:40:34.564639+0000","last_active":"2023-10-06T10:40:35.547498+0000","last_peered":"2023-10-06T10:40:35.547498+0000","last_clean":"2023-10-06T10:40:35.547498+0000","last_became_active":"2023-10-06T10:40:34.564425+0000","last_became_peered":"2023-10-06T10:40:34.564425+0000","last_unstale":"2023-10-06T10:40:35.547498+0000","last_undegraded":"2023-10-06T10:40:35.547498+0000","last_fullsized":"2023-10-06T10:40:35.547498+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:43:17.508745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":45,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.546832+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:40:35.546832+0000","last_peered":"2023-10-06T10:40:35.546832+0000","last_clean":"2023-10-06T10:40:35.546832+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:40:35.546832+0000","last_undegraded":"2023-10-06T10:40:35.546832+0000","last_fullsized":"2023-10-06T10:40:35.546832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569316+0000","last_change":"2023-10-06T10:40:35.565535+0000","last_active":"2023-10-06T10:40:35.569316+0000","last_peered":"2023-10-06T10:40:35.569316+0000","last_clean":"2023-10-06T10:40:35.569316+0000","last_became_active":"2023-10-06T10:40:35.565269+0000","last_became_peered":"2023-10-06T10:40:35.565269+0000","last_unstale":"2023-10-06T10:40:35.569316+0000","last_undegraded":"2023-10-06T10:40:35.569316+0000","last_fullsized":"2023-10-06T10:40:35.569316+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:15.720141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"51'1","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.576121+0000","last_change":"2023-10-06T10:40:34.558460+0000","last_active":"2023-10-06T10:40:35.576121+0000","last_peered":"2023-10-06T10:40:35.576121+0000","last_clean":"2023-10-06T10:40:35.576121+0000","last_became_active":"2023-10-06T10:40:34.558193+0000","last_became_peered":"2023-10-06T10:40:34.558193+0000","last_unstale":"2023-10-06T10:40:35.576121+0000","last_undegraded":"2023-10-06T10:40:35.576121+0000","last_fullsized":"2023-10-06T10:40:35.576121+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:55:42.716617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.549822+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:40:35.549822+0000","last_peered":"2023-10-06T10:40:35.549822+0000","last_clean":"2023-10-06T10:40:35.549822+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:40:35.549822+0000","last_undegraded":"2023-10-06T10:40:35.549822+0000","last_fullsized":"2023-10-06T10:40:35.549822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.569590+0000","last_change":"2023-10-06T10:40:35.566476+0000","last_active":"2023-10-06T10:40:35.569590+0000","last_peered":"2023-10-06T10:40:35.569590+0000","last_clean":"2023-10-06T10:40:35.569590+0000","last_became_active":"2023-10-06T10:40:35.566047+0000","last_became_peered":"2023-10-06T10:40:35.566047+0000","last_unstale":"2023-10-06T10:40:35.569590+0000","last_undegraded":"2023-10-06T10:40:35.569590+0000","last_fullsized":"2023-10-06T10:40:35.569590+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:34.536798+0000","last_clean_scrub_stamp":"2023-10-06T10:40:34.536798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:57:07.950757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"51'2","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.564527+0000","last_change":"2023-10-06T10:40:34.557155+0000","last_active":"2023-10-06T10:40:35.564527+0000","last_peered":"2023-10-06T10:40:35.564527+0000","last_clean":"2023-10-06T10:40:35.564527+0000","last_became_active":"2023-10-06T10:40:34.556935+0000","last_became_peered":"2023-10-06T10:40:34.556935+0000","last_unstale":"2023-10-06T10:40:35.564527+0000","last_undegraded":"2023-10-06T10:40:35.564527+0000","last_fullsized":"2023-10-06T10:40:35.564527+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:40:33.528022+0000","last_clean_scrub_stamp":"2023-10-06T10:40:33.528022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.285921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T10:40:35.550394+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:40:35.550394+0000","last_peered":"2023-10-06T10:40:35.550394+0000","last_clean":"2023-10-06T10:40:35.550394+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:40:35.550394+0000","last_undegraded":"2023-10-06T10:40:35.550394+0000","last_fullsized":"2023-10-06T10:40:35.550394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298480,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073360,"statfs":{"total":96636764160,"available":96331120640,"internally_reserved":0,"allocated":1220608,"data_stored":877891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.48099999999999998,"15min":0.47799999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71699999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.40500000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78600000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.54100000000000004}]},{"osd":1,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.46600000000000003,"15min":0.45700000000000002},"min":{"1min":0.375,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65900000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.5},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.505,"15min":0.5},"min":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72399999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.79100000000000004}]},{"osd":2,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.40600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":1.161},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.496,"15min":0.49199999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69199999999999995,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.747}]},{"osd":3,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54800000000000004,"15min":0.54200000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.66300000000000003}]},{"osd":4,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.46800000000000003,"15min":0.47699999999999998},"min":{"1min":0.30499999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57699999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.501,"15min":0.50900000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":1.0820000000000001}]},{"osd":5,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.32700000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65900000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.34300000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66800000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":1.052}]},{"osd":6,"last update":"Fri Oct 6 10:40:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.56799999999999995,"15min":0.57699999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.75700000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.747,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":1.129}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297888,"kb_used_data":600,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073952,"statfs":{"total":96636764160,"available":96331726848,"internally_reserved":0,"allocated":614400,"data_stored":286010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.49399999999999999,"15min":0.48899999999999999},"min":{"1min":0.36299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.44500000000000001,"15min":0.42899999999999999},"min":{"1min":0.38800000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.45100000000000001,"15min":0.437},"min":{"1min":0.33100000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.64300000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.379,"5min":0.309,"15min":0.309},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.47699999999999998,"15min":0.46400000000000002},"min":{"1min":0.34999999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.54400000000000004}]},{"osd":3,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.497,"15min":0.48799999999999999},"min":{"1min":0.38800000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.497},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.61399999999999999}]},{"osd":4,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47899999999999998,"15min":0.47799999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70899999999999996,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.65400000000000003}]},{"osd":5,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.48999999999999999,"15min":0.48599999999999999},"min":{"1min":0.28899999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69299999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33900000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.68899999999999995}]},{"osd":7,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.754,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.63}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298476,"kb_used_data":1188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073364,"statfs":{"total":96636764160,"available":96331124736,"internally_reserved":0,"allocated":1216512,"data_stored":877821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33000000000000002,"15min":0.32800000000000001},"min":{"1min":0.253,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.374,"5min":0.34999999999999998,"15min":0.34599999999999997},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.53900000000000003,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.29299999999999998}]},{"osd":2,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.32500000000000001,"15min":0.31900000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"last":0.31},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.36299999999999999},"min":{"1min":0.28999999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.36899999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.375},"min":{"1min":0.30099999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.379,"15min":0.375},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.34499999999999997}]},{"osd":4,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.40300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.69499999999999995,"5min":0.746,"15min":0.746},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.44700000000000001}]},{"osd":5,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54800000000000004,"15min":0.54100000000000004},"min":{"1min":0.436,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.39800000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.40100000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.433},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.47899999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.45600000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58099999999999996,"15min":0.57399999999999995},"min":{"1min":0.48699999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.439,"5min":0.436,"15min":0.436},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.38700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297908,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073932,"statfs":{"total":96636764160,"available":96331706368,"internally_reserved":0,"allocated":634880,"data_stored":289453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.32600000000000001,"15min":0.31900000000000001},"min":{"1min":0.23200000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.34899999999999998,"15min":0.34100000000000003},"min":{"1min":0.27600000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.45300000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.312,"15min":0.30599999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.371},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.33300000000000002,"15min":0.32500000000000001},"min":{"1min":0.26500000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.52000000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.41799999999999998}]},{"osd":3,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.35099999999999998,"15min":0.34300000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.35499999999999998,"15min":0.34899999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.436}]},{"osd":4,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.42599999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81200000000000006,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.54200000000000004,"15min":0.53300000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.48199999999999998}]},{"osd":5,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.41799999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.91900000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.497},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.80900000000000005,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.59199999999999997}]},{"osd":6,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58699999999999997,"15min":0.58799999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77300000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86399999999999999,"5min":1.075,"15min":1.075},"last":0.60299999999999998}]},{"osd":7,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79700000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.61399999999999999,"15min":0.60999999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88800000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297900,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073940,"statfs":{"total":96636764160,"available":96331714560,"internally_reserved":0,"allocated":626688,"data_stored":287419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.313,"15min":0.311},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.32500000000000001,"15min":0.317},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.33700000000000002}]},{"osd":1,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.32900000000000001,"15min":0.32300000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.54200000000000004,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.36699999999999999,"15min":0.36499999999999999},"min":{"1min":0.26400000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59299999999999997,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.30399999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.34899999999999998,"15min":0.34499999999999997},"min":{"1min":0.252,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.52700000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.35799999999999998,"15min":0.35499999999999998},"min":{"1min":0.27600000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.39900000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.35499999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.76100000000000001,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82699999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.42799999999999999}]},{"osd":5,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.436},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.377,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81599999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.46000000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.625,"15min":0.624},"min":{"1min":0.45800000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89000000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.624,"15min":0.623},"min":{"1min":0.433,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53300000000000003}]},{"osd":7,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.44500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79100000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.498,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.46999999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297900,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073940,"statfs":{"total":96636764160,"available":96331714560,"internally_reserved":0,"allocated":626688,"data_stored":287419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.438,"15min":0.45600000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.50600000000000001,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.36799999999999999,"15min":0.36399999999999999},"min":{"1min":0.27100000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66800000000000004,"5min":1.218,"15min":1.218},"last":0.34699999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34100000000000003,"15min":0.34000000000000002},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.61899999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.48299999999999998,"15min":0.495},"min":{"1min":0.27800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60299999999999998,"5min":3.359,"15min":3.359},"last":0.36099999999999999}]},{"osd":2,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38500000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.57699999999999996,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.315},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.63700000000000001,"15min":0.67300000000000004},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57699999999999996,"5min":7.069,"15min":7.069},"last":0.43099999999999999}]},{"osd":4,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89400000000000002,"5min":1.165,"15min":1.165},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59899999999999998,"15min":0.59699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79400000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.50700000000000001}]},{"osd":5,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.69799999999999995,"15min":0.71399999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92500000000000004,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60799999999999998,"15min":0.61099999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.96899999999999997,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.48299999999999998}]},{"osd":6,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.38,"5min":0.378,"15min":0.378},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.72699999999999998,"15min":0.74199999999999999},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84699999999999998,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.51600000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.61099999999999999},"min":{"1min":0.41299999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.99099999999999999,"5min":1.137,"15min":1.137},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.443,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91400000000000003,"5min":1.014,"15min":1.014},"last":0.54800000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297904,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073936,"statfs":{"total":96636764160,"available":96331710464,"internally_reserved":0,"allocated":630784,"data_stored":288899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47999999999999998,"15min":0.48099999999999998},"min":{"1min":0.33200000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.60799999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.496},"min":{"1min":0.33100000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.67000000000000004,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.60799999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.495,"15min":0.496},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.495,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.59399999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50700000000000001,"15min":0.50800000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.63,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36099999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.68500000000000005,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.56799999999999995}]},{"osd":3,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51600000000000001,"15min":0.51200000000000001},"min":{"1min":0.42199999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.625,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.53000000000000003,"15min":0.53500000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.64800000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64800000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.495,"15min":0.50800000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.61599999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.49399999999999999,"15min":0.502},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.63300000000000001,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.47899999999999998}]},{"osd":6,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.53600000000000003,"15min":0.54600000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.68799999999999994,"5min":0.75,"15min":0.75},"last":0.505},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.67700000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.46500000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.56899999999999995,"15min":0.57999999999999996},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70499999999999996,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72499999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.42899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297900,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073940,"statfs":{"total":96636764160,"available":96331714560,"internally_reserved":0,"allocated":626688,"data_stored":287987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.47399999999999998,"15min":0.46300000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54000000000000004,"15min":0.53400000000000003},"min":{"1min":0.35899999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.748,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.86499999999999999}]},{"osd":1,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.496,"15min":0.48499999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.52500000000000002,"15min":0.51700000000000002},"min":{"1min":0.39200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.80900000000000005}]},{"osd":2,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.51000000000000001,"15min":0.499},"min":{"1min":0.32000000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.41499999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.754,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.76300000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.57799999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77700000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78500000000000003,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.79100000000000004}]},{"osd":4,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.32100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.69599999999999995,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.501},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.66100000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.42299999999999999}]},{"osd":6,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.67500000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63400000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68799999999999994,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76700000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.55300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:40:44.480 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T10:40:44.481 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T10:40:44.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.633+0000 7f5b0d311700 1 Processor -- start 2023-10-06T10:40:44.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.633+0000 7f5b0d311700 1 -- start start 2023-10-06T10:40:44.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b0d311700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5b081598a0 0x7f5b080ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b0d311700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b080cf5d0 con 0x7f5b080cbc70 2023-10-06T10:40:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b0d311700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b080cf8b0 con 0x7f5b081594d0 2023-10-06T10:40:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b0d311700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b080cfb90 con 0x7f5b081598a0 2023-10-06T10:40:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b077fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5b080cbc70 0x7f5b080ced20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36576/0 (socket says 172.21.15.92:36576) 2023-10-06T10:40:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b077fe700 1 -- 172.21.15.92:0/3914208905 learned_addr learned my addr 172.21.15.92:0/3914208905 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5afe7fc700 1 --2- 172.21.15.92:0/3914208905 >> v2:172.21.15.124:3300/0 conn(0x7f5b081598a0 0x7f5b080ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b067fc700 1 -- 172.21.15.92:0/3914208905 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200660586 0 0) 0x7f5b080cf5d0 con 0x7f5b080cbc70 2023-10-06T10:40:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b067fc700 1 -- 172.21.15.92:0/3914208905 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5af4003570 con 0x7f5b080cbc70 2023-10-06T10:40:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b067fc700 1 -- 172.21.15.92:0/3914208905 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1694714870 0 0) 0x7f5b080cf8b0 con 0x7f5b081594d0 2023-10-06T10:40:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b067fc700 1 -- 172.21.15.92:0/3914208905 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b080cf5d0 con 0x7f5b081594d0 2023-10-06T10:40:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b067fc700 1 -- 172.21.15.92:0/3914208905 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1494992245 0 0) 0x7f5af4003570 con 0x7f5b080cbc70 2023-10-06T10:40:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b067fc700 1 -- 172.21.15.92:0/3914208905 >> v2:172.21.15.124:3300/0 conn(0x7f5b081598a0 msgr2=0x7f5b080ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b067fc700 1 --2- 172.21.15.92:0/3914208905 >> v2:172.21.15.124:3300/0 conn(0x7f5b081598a0 0x7f5b080ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b067fc700 1 -- 172.21.15.92:0/3914208905 >> v1:172.21.15.124:6789/0 conn(0x7f5b081594d0 legacy=0x7f5b080cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b067fc700 1 -- 172.21.15.92:0/3914208905 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b080cfe70 con 0x7f5b080cbc70 2023-10-06T10:40:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b067fc700 1 -- 172.21.15.92:0/3914208905 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f5b000017f0 con 0x7f5b080cbc70 2023-10-06T10:40:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b067fc700 1 -- 172.21.15.92:0/3914208905 >> v1:172.21.15.92:6789/0 conn(0x7f5b080cbc70 legacy=0x7f5b080ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b067fc700 1 --2- 172.21.15.92:0/3914208905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5af4003c40 0x7f5af4004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b067fc700 1 --2- 172.21.15.92:0/3914208905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5af4004810 0x7f5af4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b067fc700 1 --2- 172.21.15.92:0/3914208905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5af4005210 0x7f5af400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b067fc700 1 -- 172.21.15.92:0/3914208905 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b080cfb90 con 0x7f5af4004810 2023-10-06T10:40:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b067fc700 1 -- 172.21.15.92:0/3914208905 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af400c130 con 0x7f5af4005210 2023-10-06T10:40:44.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.634+0000 7f5b067fc700 1 -- 172.21.15.92:0/3914208905 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af400c2c0 con 0x7f5af4003c40 2023-10-06T10:40:44.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.635+0000 7f5afe7fc700 1 --2- 172.21.15.92:0/3914208905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5af4004810 0x7f5af4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:44.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.635+0000 7f5b06ffd700 1 --2- 172.21.15.92:0/3914208905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5af4003c40 0x7f5af4004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:44.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.635+0000 7f5b077fe700 1 --2- 172.21.15.92:0/3914208905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5af4005210 0x7f5af400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:44.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.635+0000 7f5afe7fc700 1 -- 172.21.15.92:0/3914208905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5af4003c40 msgr2=0x7f5af4004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:44.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.635+0000 7f5afe7fc700 1 --2- 172.21.15.92:0/3914208905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5af4003c40 0x7f5af4004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:44.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.635+0000 7f5afe7fc700 1 -- 172.21.15.92:0/3914208905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5af4005210 msgr2=0x7f5af400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:44.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.635+0000 7f5afe7fc700 1 --2- 172.21.15.92:0/3914208905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5af4005210 0x7f5af400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:44.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.635+0000 7f5afe7fc700 1 -- 172.21.15.92:0/3914208905 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af4003570 con 0x7f5af4004810 2023-10-06T10:40:44.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.635+0000 7f5b077fe700 1 -- 172.21.15.92:0/3914208905 reap_dead start 2023-10-06T10:40:44.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.635+0000 7f5afe7fc700 1 --2- 172.21.15.92:0/3914208905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5af4004810 0x7f5af4004c80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f5b08067cf0 tx=0x7f5af00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7b6708776989d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:44.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.635+0000 7f5b067fc700 1 -- 172.21.15.92:0/3914208905 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af000e6e0 con 0x7f5af4004810 2023-10-06T10:40:44.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.635+0000 7f5b067fc700 1 -- 172.21.15.92:0/3914208905 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af000e840 con 0x7f5af4004810 2023-10-06T10:40:44.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.635+0000 7f5b067fc700 1 -- 172.21.15.92:0/3914208905 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af000ec00 con 0x7f5af4004810 2023-10-06T10:40:44.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.636+0000 7f5b0d311700 1 -- 172.21.15.92:0/3914208905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5af4004810 msgr2=0x7f5af4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:44.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.636+0000 7f5b0d311700 1 --2- 172.21.15.92:0/3914208905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5af4004810 0x7f5af4004c80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f5b08067cf0 tx=0x7f5af00091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:44.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.636+0000 7f5b0d311700 1 -- 172.21.15.92:0/3914208905 shutdown_connections 2023-10-06T10:40:44.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.636+0000 7f5b0d311700 1 --2- 172.21.15.92:0/3914208905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5af4004810 0x7f5af4004c80 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:44.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.636+0000 7f5b0d311700 1 -- 172.21.15.92:0/3914208905 >> 172.21.15.92:0/3914208905 conn(0x7f5b08089e50 msgr2=0x7f5b08099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:44.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.636+0000 7f5b0d311700 1 -- 172.21.15.92:0/3914208905 shutdown_connections 2023-10-06T10:40:44.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.636+0000 7f5b0d311700 1 -- 172.21.15.92:0/3914208905 wait complete. 2023-10-06T10:40:44.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.637+0000 7f5b0d311700 1 Processor -- start 2023-10-06T10:40:44.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.637+0000 7f5b0d311700 1 -- start start 2023-10-06T10:40:44.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.638+0000 7f5b0d311700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5af4004810 0x7f5b0815b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:44.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.638+0000 7f5b0d311700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b080b3990 0x7f5b0814c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:44.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.638+0000 7f5b0d311700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b0814dc90 0x7f5b0814e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:44.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.638+0000 7f5b0d311700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0814e640 con 0x7f5af4004810 2023-10-06T10:40:44.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.638+0000 7f5b0d311700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0815c650 con 0x7f5b0814dc90 2023-10-06T10:40:44.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.638+0000 7f5b0d311700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0815cae0 con 0x7f5b080b3990 2023-10-06T10:40:44.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.638+0000 7f5b06ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5af4004810 0x7f5b0815b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:44.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.638+0000 7f5b06ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5af4004810 0x7f5b0815b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34854/0 (socket says 172.21.15.92:34854) 2023-10-06T10:40:44.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.638+0000 7f5b06ffd700 1 -- 172.21.15.92:0/1051267680 learned_addr learned my addr 172.21.15.92:0/1051267680 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:44.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.638+0000 7f5b06ffd700 1 -- 172.21.15.92:0/1051267680 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b080b3990 msgr2=0x7f5b0814c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:44.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.638+0000 7f5b06ffd700 1 --2- 172.21.15.92:0/1051267680 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b080b3990 0x7f5b0814c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:44.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.638+0000 7f5b06ffd700 1 -- 172.21.15.92:0/1051267680 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b0814dc90 msgr2=0x7f5b0814e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:44.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.638+0000 7f5b06ffd700 1 --2- 172.21.15.92:0/1051267680 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b0814dc90 0x7f5b0814e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:44.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.638+0000 7f5b06ffd700 1 -- 172.21.15.92:0/1051267680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0815ba40 con 0x7f5af4004810 2023-10-06T10:40:44.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.638+0000 7f5b06ffd700 1 --2- 172.21.15.92:0/1051267680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5af4004810 0x7f5b0815b3b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5af8002b80 tx=0x7f5af8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:44.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.638+0000 7f5aff7fe700 1 -- 172.21.15.92:0/1051267680 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af800fa00 con 0x7f5af4004810 2023-10-06T10:40:44.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.638+0000 7f5aff7fe700 1 -- 172.21.15.92:0/1051267680 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af800fb80 con 0x7f5af4004810 2023-10-06T10:40:44.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.638+0000 7f5b0d311700 1 -- 172.21.15.92:0/1051267680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0815bd20 con 0x7f5af4004810 2023-10-06T10:40:44.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.638+0000 7f5aff7fe700 1 -- 172.21.15.92:0/1051267680 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af8006820 con 0x7f5af4004810 2023-10-06T10:40:44.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.638+0000 7f5b0d311700 1 -- 172.21.15.92:0/1051267680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0815c2e0 con 0x7f5af4004810 2023-10-06T10:40:44.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.639+0000 7f5b0d311700 1 -- 172.21.15.92:0/1051267680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b08064a90 con 0x7f5af4004810 2023-10-06T10:40:44.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.640+0000 7f5aff7fe700 1 -- 172.21.15.92:0/1051267680 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5af80133f0 con 0x7f5af4004810 2023-10-06T10:40:44.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.640+0000 7f5aff7fe700 1 --2- 172.21.15.92:0/1051267680 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5ae8066f30 0x7f5ae80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:44.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.640+0000 7f5aff7fe700 1 -- 172.21.15.92:0/1051267680 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5af8084b40 con 0x7f5af4004810 2023-10-06T10:40:44.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.640+0000 7f5afe7fc700 1 --2- 172.21.15.92:0/1051267680 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5ae8066f30 0x7f5ae80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:44.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.641+0000 7f5afe7fc700 1 --2- 172.21.15.92:0/1051267680 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5ae8066f30 0x7f5ae80693e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5b08099ba0 tx=0x7f5af001a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:44.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.644+0000 7f5aff7fe700 1 -- 172.21.15.92:0/1051267680 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5af8051b10 con 0x7f5af4004810 2023-10-06T10:40:44.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.896+0000 7f5b0d311700 1 -- 172.21.15.92:0/1051267680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5b080cb560 con 0x7f5af4004810 2023-10-06T10:40:44.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.900+0000 7f5aff7fe700 1 -- 172.21.15.92:0/1051267680 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7f5af8055130 con 0x7f5af4004810 2023-10-06T10:40:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.902+0000 7f5b0d311700 1 -- 172.21.15.92:0/1051267680 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5ae8066f30 msgr2=0x7f5ae80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:44.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.902+0000 7f5b0d311700 1 --2- 172.21.15.92:0/1051267680 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5ae8066f30 0x7f5ae80693e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5b08099ba0 tx=0x7f5af001a000 comp rx=0 tx=0).stop 2023-10-06T10:40:44.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.902+0000 7f5b0d311700 1 -- 172.21.15.92:0/1051267680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5af4004810 msgr2=0x7f5b0815b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:44.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.902+0000 7f5b0d311700 1 --2- 172.21.15.92:0/1051267680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5af4004810 0x7f5b0815b3b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5af8002b80 tx=0x7f5af8007d30 comp rx=0 tx=0).stop 2023-10-06T10:40:44.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.902+0000 7f5b0d311700 1 -- 172.21.15.92:0/1051267680 shutdown_connections 2023-10-06T10:40:44.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.902+0000 7f5b0d311700 1 --2- 172.21.15.92:0/1051267680 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5ae8066f30 0x7f5ae80693e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:44.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.902+0000 7f5b0d311700 1 --2- 172.21.15.92:0/1051267680 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b0814dc90 0x7f5b0814e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:44.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.902+0000 7f5b0d311700 1 --2- 172.21.15.92:0/1051267680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5af4004810 0x7f5b0815b3b0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:44.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.902+0000 7f5b0d311700 1 --2- 172.21.15.92:0/1051267680 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b080b3990 0x7f5b0814c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:44.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.902+0000 7f5b0d311700 1 -- 172.21.15.92:0/1051267680 >> 172.21.15.92:0/1051267680 conn(0x7f5b08089e50 msgr2=0x7f5b0814d500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:44.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.903+0000 7f5b0d311700 1 -- 172.21.15.92:0/1051267680 shutdown_connections 2023-10-06T10:40:44.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:44.903+0000 7f5b0d311700 1 -- 172.21.15.92:0/1051267680 wait complete. 2023-10-06T10:40:44.906 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-10-06T10:40:44.916 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:40:45.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.028+0000 7f224483c700 1 Processor -- start 2023-10-06T10:40:45.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.028+0000 7f224483c700 1 -- start start 2023-10-06T10:40:45.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.029+0000 7f224483c700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2240159440 0x7f22400b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.029+0000 7f224483c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22400b35d0 0x7f22400cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.029+0000 7f224483c700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22400cb7a0 con 0x7f22400cc170 2023-10-06T10:40:45.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.029+0000 7f224483c700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22400b0860 con 0x7f2240159440 2023-10-06T10:40:45.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.029+0000 7f224483c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22400cb150 con 0x7f22400b35d0 2023-10-06T10:40:45.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.029+0000 7f223ed9d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2240159440 0x7f22400b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.029+0000 7f223659c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22400b35d0 0x7f22400cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.029+0000 7f223f59e700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f22400cc170 0x7f22400b2100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:41016/0 (socket says 172.21.15.92:41016) 2023-10-06T10:40:45.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.029+0000 7f223ed9d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2240159440 0x7f22400b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:35936/0 (socket says 172.21.15.92:35936) 2023-10-06T10:40:45.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.029+0000 7f223659c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22400b35d0 0x7f22400cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:57892/0 (socket says 172.21.15.92:57892) 2023-10-06T10:40:45.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.029+0000 7f223f59e700 1 -- 172.21.15.92:0/3676825208 learned_addr learned my addr 172.21.15.92:0/3676825208 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:45.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.030+0000 7f223ed9d700 1 -- 172.21.15.92:0/3676825208 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22400b35d0 msgr2=0x7f22400cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:45.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.030+0000 7f223ed9d700 1 --2- 172.21.15.92:0/3676825208 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22400b35d0 0x7f22400cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.030+0000 7f223ed9d700 1 -- 172.21.15.92:0/3676825208 >> v1:172.21.15.124:6789/0 conn(0x7f22400cc170 legacy=0x7f22400b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:45.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.030+0000 7f223ed9d700 1 -- 172.21.15.92:0/3676825208 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22400cb430 con 0x7f2240159440 2023-10-06T10:40:45.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.030+0000 7f223659c700 1 --2- 172.21.15.92:0/3676825208 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22400b35d0 0x7f22400cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:40:45.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.030+0000 7f223ed9d700 1 --2- 172.21.15.92:0/3676825208 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2240159440 0x7f22400b3090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f22300098e0 tx=0x7f22300091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc8a0e9325618651 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:45.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.030+0000 7f223e59c700 1 -- 172.21.15.92:0/3676825208 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223000b640 con 0x7f2240159440 2023-10-06T10:40:45.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.030+0000 7f223e59c700 1 -- 172.21.15.92:0/3676825208 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223000b7a0 con 0x7f2240159440 2023-10-06T10:40:45.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.030+0000 7f223e59c700 1 -- 172.21.15.92:0/3676825208 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223000bb40 con 0x7f2240159440 2023-10-06T10:40:45.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.031+0000 7f224483c700 1 -- 172.21.15.92:0/3676825208 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2240159440 msgr2=0x7f22400b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:45.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.031+0000 7f224483c700 1 --2- 172.21.15.92:0/3676825208 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2240159440 0x7f22400b3090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f22300098e0 tx=0x7f22300091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.031+0000 7f224483c700 1 -- 172.21.15.92:0/3676825208 shutdown_connections 2023-10-06T10:40:45.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.031+0000 7f224483c700 1 --2- 172.21.15.92:0/3676825208 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2240159440 0x7f22400b3090 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.031+0000 7f224483c700 1 --2- 172.21.15.92:0/3676825208 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22400b35d0 0x7f22400cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.031+0000 7f224483c700 1 -- 172.21.15.92:0/3676825208 >> 172.21.15.92:0/3676825208 conn(0x7f224006ece0 msgr2=0x7f224006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:45.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.031+0000 7f224483c700 1 -- 172.21.15.92:0/3676825208 shutdown_connections 2023-10-06T10:40:45.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.031+0000 7f224483c700 1 -- 172.21.15.92:0/3676825208 wait complete. 2023-10-06T10:40:45.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.032+0000 7f224483c700 1 Processor -- start 2023-10-06T10:40:45.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.032+0000 7f224483c700 1 -- start start 2023-10-06T10:40:45.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.033+0000 7f224483c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22400b35d0 0x7f2240146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.033+0000 7f224483c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22400cc170 0x7f224015a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.033+0000 7f224483c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2240159440 0x7f22401552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.033+0000 7f224483c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22400c8c10 con 0x7f2240159440 2023-10-06T10:40:45.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.033+0000 7f224483c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22400c8000 con 0x7f22400cc170 2023-10-06T10:40:45.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.033+0000 7f224483c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22400c82e0 con 0x7f22400b35d0 2023-10-06T10:40:45.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.033+0000 7f223f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2240159440 0x7f22401552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.033+0000 7f223f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2240159440 0x7f22401552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34868/0 (socket says 172.21.15.92:34868) 2023-10-06T10:40:45.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.033+0000 7f223f59e700 1 -- 172.21.15.92:0/1886341469 learned_addr learned my addr 172.21.15.92:0/1886341469 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:45.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.033+0000 7f223ed9d700 1 --2- 172.21.15.92:0/1886341469 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22400b35d0 0x7f2240146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.033+0000 7f223659c700 1 --2- 172.21.15.92:0/1886341469 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22400cc170 0x7f224015a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.033+0000 7f223f59e700 1 -- 172.21.15.92:0/1886341469 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22400b35d0 msgr2=0x7f2240146ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:45.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.033+0000 7f223f59e700 1 --2- 172.21.15.92:0/1886341469 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22400b35d0 0x7f2240146ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.033+0000 7f223f59e700 1 -- 172.21.15.92:0/1886341469 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22400cc170 msgr2=0x7f224015a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:45.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.033+0000 7f223f59e700 1 --2- 172.21.15.92:0/1886341469 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22400cc170 0x7f224015a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.033+0000 7f223f59e700 1 -- 172.21.15.92:0/1886341469 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2228009420 con 0x7f2240159440 2023-10-06T10:40:45.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.033+0000 7f223f59e700 1 --2- 172.21.15.92:0/1886341469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2240159440 0x7f22401552e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f223800cc40 tx=0x7f2238001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:45.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.033+0000 7f22377fe700 1 -- 172.21.15.92:0/1886341469 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223800a8b0 con 0x7f2240159440 2023-10-06T10:40:45.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.033+0000 7f22377fe700 1 -- 172.21.15.92:0/1886341469 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223800aa10 con 0x7f2240159440 2023-10-06T10:40:45.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.033+0000 7f22377fe700 1 -- 172.21.15.92:0/1886341469 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223800add0 con 0x7f2240159440 2023-10-06T10:40:45.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.033+0000 7f224483c700 1 -- 172.21.15.92:0/1886341469 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2230009420 con 0x7f2240159440 2023-10-06T10:40:45.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.033+0000 7f224483c700 1 -- 172.21.15.92:0/1886341469 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22400c88a0 con 0x7f2240159440 2023-10-06T10:40:45.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.034+0000 7f224483c700 1 -- 172.21.15.92:0/1886341469 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22400c8dc0 con 0x7f2240159440 2023-10-06T10:40:45.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.035+0000 7f22377fe700 1 -- 172.21.15.92:0/1886341469 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2238007300 con 0x7f2240159440 2023-10-06T10:40:45.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.035+0000 7f22377fe700 1 --2- 172.21.15.92:0/1886341469 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2220066f30 0x7f22200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.035+0000 7f22377fe700 1 -- 172.21.15.92:0/1886341469 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2238084a40 con 0x7f2240159440 2023-10-06T10:40:45.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.035+0000 7f223ed9d700 1 --2- 172.21.15.92:0/1886341469 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2220066f30 0x7f22200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.035+0000 7f223ed9d700 1 --2- 172.21.15.92:0/1886341469 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2220066f30 0x7f22200693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f22300093f0 tx=0x7f2230017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:45.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.039+0000 7f22377fe700 1 -- 172.21.15.92:0/1886341469 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22400c8dc0 con 0x7f2240159440 2023-10-06T10:40:45.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.264+0000 7f224483c700 1 -- 172.21.15.92:0/1886341469 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22400c8d50 con 0x7f2240159440 2023-10-06T10:40:45.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.265+0000 7f22377fe700 1 -- 172.21.15.92:0/1886341469 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f2238086e60 con 0x7f2240159440 2023-10-06T10:40:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:45.267 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":32,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3668157650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3668157650},{"type":"v1","addr":"172.21.15.124:6834","nonce":3668157650}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2986058955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2986058955},{"type":"v1","addr":"172.21.15.124:6835","nonce":2986058955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5733,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/831923836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":831923836},{"type":"v1","addr":"172.21.15.92:6836","nonce":831923836}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:40:34.932631+0000","modified":"2023-10-06T10:40:38.573961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5736},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5736":{"gid":5736,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2458340622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2458340622},{"type":"v1","addr":"172.21.15.92:6837","nonce":2458340622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T10:40:45.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.267+0000 7f224483c700 1 -- 172.21.15.92:0/1886341469 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2220066f30 msgr2=0x7f22200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:45.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.267+0000 7f224483c700 1 --2- 172.21.15.92:0/1886341469 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2220066f30 0x7f22200693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f22300093f0 tx=0x7f2230017000 comp rx=0 tx=0).stop 2023-10-06T10:40:45.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.267+0000 7f224483c700 1 -- 172.21.15.92:0/1886341469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2240159440 msgr2=0x7f22401552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:45.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.267+0000 7f224483c700 1 --2- 172.21.15.92:0/1886341469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2240159440 0x7f22401552e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f223800cc40 tx=0x7f2238001a50 comp rx=0 tx=0).stop 2023-10-06T10:40:45.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.267+0000 7f224483c700 1 -- 172.21.15.92:0/1886341469 shutdown_connections 2023-10-06T10:40:45.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.267+0000 7f224483c700 1 --2- 172.21.15.92:0/1886341469 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2220066f30 0x7f22200693e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.267+0000 7f224483c700 1 --2- 172.21.15.92:0/1886341469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2240159440 0x7f22401552e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.267+0000 7f224483c700 1 --2- 172.21.15.92:0/1886341469 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22400b35d0 0x7f2240146ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.267+0000 7f224483c700 1 --2- 172.21.15.92:0/1886341469 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22400cc170 0x7f224015a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.267+0000 7f224483c700 1 -- 172.21.15.92:0/1886341469 >> 172.21.15.92:0/1886341469 conn(0x7f224006ece0 msgr2=0x7f22400b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:45.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.267+0000 7f224483c700 1 -- 172.21.15.92:0/1886341469 shutdown_connections 2023-10-06T10:40:45.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.267+0000 7f224483c700 1 -- 172.21.15.92:0/1886341469 wait complete. 2023-10-06T10:40:45.270 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 32 2023-10-06T10:40:45.683 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T10:40:34.932631+0000', 'modified': '2023-10-06T10:40:38.573961+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5736}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5736': {'gid': 5736, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6837/2458340622', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 2458340622}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 2458340622}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T10:40:45.683 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T10:40:45.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T10:40:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.797+0000 7f30ce18e700 1 Processor -- start 2023-10-06T10:40:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.797+0000 7f30ce18e700 1 -- start start 2023-10-06T10:40:45.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.798+0000 7f30ce18e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f30c80b49a0 0x7f30c80aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.798+0000 7f30ce18e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c80b0760 0x7f30c80b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.798+0000 7f30ce18e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c80b0c60 con 0x7f30c8159560 2023-10-06T10:40:45.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.798+0000 7f30ce18e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c80b0f40 con 0x7f30c80b49a0 2023-10-06T10:40:45.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.798+0000 7f30ce18e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c80b1220 con 0x7f30c80b0760 2023-10-06T10:40:45.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.798+0000 7f30c6ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f30c80b49a0 0x7f30c80aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.798+0000 7f30c77fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f30c8159560 0x7f30c80aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36578/0 (socket says 172.21.15.92:36578) 2023-10-06T10:40:45.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.798+0000 7f30c6ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f30c80b49a0 0x7f30c80aac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34882/0 (socket says 172.21.15.92:34882) 2023-10-06T10:40:45.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.798+0000 7f30c77fe700 1 -- 172.21.15.92:0/1609747025 learned_addr learned my addr 172.21.15.92:0/1609747025 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:45.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.798+0000 7f30c7fff700 1 --2- 172.21.15.92:0/1609747025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c80b0760 0x7f30c80b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.798+0000 7f30c6ffd700 1 -- 172.21.15.92:0/1609747025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c80b0760 msgr2=0x7f30c80b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:45.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.798+0000 7f30c6ffd700 1 --2- 172.21.15.92:0/1609747025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c80b0760 0x7f30c80b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.798+0000 7f30c6ffd700 1 -- 172.21.15.92:0/1609747025 >> v1:172.21.15.92:6789/0 conn(0x7f30c8159560 legacy=0x7f30c80aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:45.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.798+0000 7f30c67fc700 1 -- 172.21.15.92:0/1609747025 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265535635 0 0) 0x7f30c80b0c60 con 0x7f30c8159560 2023-10-06T10:40:45.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.798+0000 7f30c6ffd700 1 -- 172.21.15.92:0/1609747025 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c80b1500 con 0x7f30c80b49a0 2023-10-06T10:40:45.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.798+0000 7f30c7fff700 1 --2- 172.21.15.92:0/1609747025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c80b0760 0x7f30c80b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:45.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.798+0000 7f30c6ffd700 1 --2- 172.21.15.92:0/1609747025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c80b49a0 0x7f30c80aac40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009910 tx=0x7f30bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f770b3e832a7caa8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:45.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.799+0000 7f30c67fc700 1 -- 172.21.15.92:0/1609747025 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc00b820 con 0x7f30c80b49a0 2023-10-06T10:40:45.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.799+0000 7f30c67fc700 1 -- 172.21.15.92:0/1609747025 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30bc00b980 con 0x7f30c80b49a0 2023-10-06T10:40:45.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.799+0000 7f30c67fc700 1 -- 172.21.15.92:0/1609747025 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc00bd20 con 0x7f30c80b49a0 2023-10-06T10:40:45.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.799+0000 7f30ce18e700 1 -- 172.21.15.92:0/1609747025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c80b49a0 msgr2=0x7f30c80aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:45.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.799+0000 7f30ce18e700 1 --2- 172.21.15.92:0/1609747025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c80b49a0 0x7f30c80aac40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009910 tx=0x7f30bc0091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.799+0000 7f30ce18e700 1 -- 172.21.15.92:0/1609747025 shutdown_connections 2023-10-06T10:40:45.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.799+0000 7f30ce18e700 1 --2- 172.21.15.92:0/1609747025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c80b0760 0x7f30c80b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.799+0000 7f30ce18e700 1 --2- 172.21.15.92:0/1609747025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c80b49a0 0x7f30c80aac40 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.799+0000 7f30ce18e700 1 -- 172.21.15.92:0/1609747025 >> 172.21.15.92:0/1609747025 conn(0x7f30c80997f0 msgr2=0x7f30c806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:45.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.799+0000 7f30ce18e700 1 -- 172.21.15.92:0/1609747025 shutdown_connections 2023-10-06T10:40:45.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.799+0000 7f30ce18e700 1 -- 172.21.15.92:0/1609747025 wait complete. 2023-10-06T10:40:45.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.800+0000 7f30ce18e700 1 Processor -- start 2023-10-06T10:40:45.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.800+0000 7f30ce18e700 1 -- start start 2023-10-06T10:40:45.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.801+0000 7f30ce18e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c80b0760 0x7f30c809bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.801+0000 7f30ce18e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30c80b49a0 0x7f30c8156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.801+0000 7f30ce18e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c8159560 0x7f30c809d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.801+0000 7f30ce18e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c8157300 con 0x7f30c80b0760 2023-10-06T10:40:45.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.801+0000 7f30ce18e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c809bac0 con 0x7f30c80b49a0 2023-10-06T10:40:45.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.801+0000 7f30ce18e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c809aeb0 con 0x7f30c8159560 2023-10-06T10:40:45.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.801+0000 7f30c77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c80b0760 0x7f30c809bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.801+0000 7f30c77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c80b0760 0x7f30c809bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34898/0 (socket says 172.21.15.92:34898) 2023-10-06T10:40:45.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.801+0000 7f30c77fe700 1 -- 172.21.15.92:0/2455736404 learned_addr learned my addr 172.21.15.92:0/2455736404 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:45.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.801+0000 7f30c7fff700 1 --2- 172.21.15.92:0/2455736404 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c8159560 0x7f30c809d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.801+0000 7f30c6ffd700 1 --2- 172.21.15.92:0/2455736404 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30c80b49a0 0x7f30c8156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.802+0000 7f30c77fe700 1 -- 172.21.15.92:0/2455736404 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c8159560 msgr2=0x7f30c809d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:45.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.802+0000 7f30c77fe700 1 --2- 172.21.15.92:0/2455736404 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c8159560 0x7f30c809d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.802+0000 7f30c77fe700 1 -- 172.21.15.92:0/2455736404 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30c80b49a0 msgr2=0x7f30c8156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:45.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.802+0000 7f30c77fe700 1 --2- 172.21.15.92:0/2455736404 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30c80b49a0 0x7f30c8156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.802+0000 7f30c77fe700 1 -- 172.21.15.92:0/2455736404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30bc009420 con 0x7f30c80b0760 2023-10-06T10:40:45.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.802+0000 7f30c77fe700 1 --2- 172.21.15.92:0/2455736404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c80b0760 0x7f30c809bf50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f30b800c930 tx=0x7f30b80016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:45.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.802+0000 7f30af7fe700 1 -- 172.21.15.92:0/2455736404 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8005d80 con 0x7f30c80b0760 2023-10-06T10:40:45.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.802+0000 7f30af7fe700 1 -- 172.21.15.92:0/2455736404 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b800a5d0 con 0x7f30c80b0760 2023-10-06T10:40:45.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.802+0000 7f30af7fe700 1 -- 172.21.15.92:0/2455736404 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b800a990 con 0x7f30c80b0760 2023-10-06T10:40:45.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.802+0000 7f30ce18e700 1 -- 172.21.15.92:0/2455736404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c809b190 con 0x7f30c80b0760 2023-10-06T10:40:45.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.802+0000 7f30ce18e700 1 -- 172.21.15.92:0/2455736404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c809b750 con 0x7f30c80b0760 2023-10-06T10:40:45.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.803+0000 7f30af7fe700 1 -- 172.21.15.92:0/2455736404 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f30b80070f0 con 0x7f30c80b0760 2023-10-06T10:40:45.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.803+0000 7f30af7fe700 1 --2- 172.21.15.92:0/2455736404 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f30b4066f30 0x7f30b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.804+0000 7f30af7fe700 1 -- 172.21.15.92:0/2455736404 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f30b80845a0 con 0x7f30c80b0760 2023-10-06T10:40:45.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.804+0000 7f30c6ffd700 1 --2- 172.21.15.92:0/2455736404 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f30b4066f30 0x7f30b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.804+0000 7f30c6ffd700 1 --2- 172.21.15.92:0/2455736404 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f30b4066f30 0x7f30b40693e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009910 tx=0x7f30bc017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:45.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.805+0000 7f30d7486b80 1 Processor -- start 2023-10-06T10:40:45.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.806+0000 7f30d7486b80 1 -- start start 2023-10-06T10:40:45.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.806+0000 7f30d7486b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee074a90 0x5629ee074e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.806+0000 7f30d7486b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5629ee0309a0 0x5629ee02d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.806+0000 7f30d7486b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5629ee02ebd0 0x5629ee02dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.806+0000 7f30d7486b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629edfadc80 con 0x5629ee074a90 2023-10-06T10:40:45.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.806+0000 7f30d7486b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629edf84a70 con 0x5629ee02ebd0 2023-10-06T10:40:45.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.806+0000 7f30d7486b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629edf82270 con 0x5629ee0309a0 2023-10-06T10:40:45.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.806+0000 7f30c77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee074a90 0x5629ee074e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.807+0000 7f30c77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee074a90 0x5629ee074e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34902/0 (socket says 172.21.15.92:34902) 2023-10-06T10:40:45.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.807+0000 7f30c77fe700 1 -- 172.21.15.92:0/3455592255 learned_addr learned my addr 172.21.15.92:0/3455592255 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.807+0000 7f30c77fe700 1 -- 172.21.15.92:0/3455592255 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5629ee0309a0 msgr2=0x5629ee02d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.807+0000 7f30c6ffd700 1 --2- 172.21.15.92:0/3455592255 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5629ee0309a0 0x5629ee02d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.807+0000 7f30c7fff700 1 --2- 172.21.15.92:0/3455592255 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5629ee02ebd0 0x5629ee02dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.807+0000 7f30c77fe700 1 --2- 172.21.15.92:0/3455592255 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5629ee0309a0 0x5629ee02d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.807+0000 7f30c77fe700 1 -- 172.21.15.92:0/3455592255 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5629ee02ebd0 msgr2=0x5629ee02dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.807+0000 7f30c77fe700 1 --2- 172.21.15.92:0/3455592255 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5629ee02ebd0 0x5629ee02dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.807+0000 7f30c77fe700 1 -- 172.21.15.92:0/3455592255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629ede18270 con 0x5629ee074a90 2023-10-06T10:40:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.807+0000 7f30c77fe700 1 --2- 172.21.15.92:0/3455592255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee074a90 0x5629ee074e60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f30b8082820 tx=0x7f30b8082780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d8a814baa5c8ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.807+0000 7f30acff9700 1 -- 172.21.15.92:0/3455592255 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b80552b0 con 0x5629ee074a90 2023-10-06T10:40:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.807+0000 7f30acff9700 1 -- 172.21.15.92:0/3455592255 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b8058500 con 0x5629ee074a90 2023-10-06T10:40:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.807+0000 7f30acff9700 1 -- 172.21.15.92:0/3455592255 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b805c250 con 0x5629ee074a90 2023-10-06T10:40:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.808+0000 7f30d7486b80 1 -- 172.21.15.92:0/3455592255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee074a90 msgr2=0x5629ee074e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.808+0000 7f30d7486b80 1 --2- 172.21.15.92:0/3455592255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee074a90 0x5629ee074e60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f30b8082820 tx=0x7f30b8082780 comp rx=0 tx=0).stop 2023-10-06T10:40:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.808+0000 7f30d7486b80 1 -- 172.21.15.92:0/3455592255 shutdown_connections 2023-10-06T10:40:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.808+0000 7f30d7486b80 1 --2- 172.21.15.92:0/3455592255 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5629ee02ebd0 0x5629ee02dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.808+0000 7f30d7486b80 1 --2- 172.21.15.92:0/3455592255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee074a90 0x5629ee074e60 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.808+0000 7f30d7486b80 1 --2- 172.21.15.92:0/3455592255 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5629ee0309a0 0x5629ee02d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.808+0000 7f30d7486b80 1 -- 172.21.15.92:0/3455592255 >> 172.21.15.92:0/3455592255 conn(0x5629ee02f070 msgr2=0x5629ee02f440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.808+0000 7f30d7486b80 1 -- 172.21.15.92:0/3455592255 shutdown_connections 2023-10-06T10:40:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.808+0000 7f30d7486b80 1 -- 172.21.15.92:0/3455592255 wait complete. 2023-10-06T10:40:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.808+0000 7f30d7486b80 1 -- start start 2023-10-06T10:40:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.808+0000 7f30d7486b80 1 Processor -- start 2023-10-06T10:40:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.809+0000 7f30d7486b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5629ee02ebd0 0x5629ee05a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.809+0000 7f30d7486b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee0309a0 0x5629ee055b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.809+0000 7f30d7486b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5629ee074a90 0x5629ee056050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.809+0000 7f30d7486b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629edf7e060 con 0x5629ee0309a0 2023-10-06T10:40:45.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.809+0000 7f30d7486b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629edf29890 con 0x5629ee074a90 2023-10-06T10:40:45.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.809+0000 7f30d7486b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629edf44ea0 con 0x5629ee02ebd0 2023-10-06T10:40:45.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.809+0000 7f30d7486b80 10 client.0 _ll_register_callbacks cb 0x5629ede1af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:40:45.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.809+0000 7f30c6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee0309a0 0x5629ee055b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.809+0000 7f30c6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee0309a0 0x5629ee055b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34906/0 (socket says 172.21.15.92:34906) 2023-10-06T10:40:45.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.809+0000 7f30c6ffd700 1 -- 172.21.15.92:0/1985776347 learned_addr learned my addr 172.21.15.92:0/1985776347 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:45.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.809+0000 7f30c77fe700 1 --2- 172.21.15.92:0/1985776347 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5629ee02ebd0 0x5629ee05a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.809+0000 7f30c7fff700 1 --2- 172.21.15.92:0/1985776347 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5629ee074a90 0x5629ee056050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.810+0000 7f30c6ffd700 1 -- 172.21.15.92:0/1985776347 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5629ee02ebd0 msgr2=0x5629ee05a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:45.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.810+0000 7f30c6ffd700 1 --2- 172.21.15.92:0/1985776347 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5629ee02ebd0 0x5629ee05a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.810+0000 7f30c6ffd700 1 -- 172.21.15.92:0/1985776347 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5629ee074a90 msgr2=0x5629ee056050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.810+0000 7f30c6ffd700 1 --2- 172.21.15.92:0/1985776347 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5629ee074a90 0x5629ee056050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.810+0000 7f30c6ffd700 1 -- 172.21.15.92:0/1985776347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c809b750 con 0x5629ee0309a0 2023-10-06T10:40:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.810+0000 7f30c6ffd700 1 --2- 172.21.15.92:0/1985776347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee0309a0 0x5629ee055b10 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00b3c0 tx=0x7f30bc020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf07257de043218f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.810+0000 7f30a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:40:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.810+0000 7f30a5ffb700 1 -- 172.21.15.92:0/1985776347 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc02d070 con 0x5629ee0309a0 2023-10-06T10:40:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.810+0000 7f30a5ffb700 1 -- 172.21.15.92:0/1985776347 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f30bc011d20 con 0x5629ee0309a0 2023-10-06T10:40:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.810+0000 7f30d7486b80 1 -- 172.21.15.92:0/1985776347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629ee022690 con 0x5629ee0309a0 2023-10-06T10:40:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.810+0000 7f30a5ffb700 1 -- 172.21.15.92:0/1985776347 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30bc020070 con 0x5629ee0309a0 2023-10-06T10:40:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.810+0000 7f30d7486b80 10 client.5907 fetch_fsmap learned FSMap version 32 2023-10-06T10:40:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.810+0000 7f30d7486b80 1 -- 172.21.15.92:0/1985776347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5629eddbafe0 con 0x5629ee0309a0 2023-10-06T10:40:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.810+0000 7f30a5ffb700 1 -- 172.21.15.92:0/1985776347 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5629ee022690 con 0x5629ee0309a0 2023-10-06T10:40:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.810+0000 7f30a5ffb700 1 -- 172.21.15.92:0/1985776347 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 32) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f30bc01a420 con 0x5629ee0309a0 2023-10-06T10:40:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.810+0000 7f30d7486b80 10 client.5907 fetch_fsmap finished waiting for FSMap version 32 2023-10-06T10:40:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.810+0000 7f30d7486b80 10 client.5907 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T10:40:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.810+0000 7f30d7486b80 20 client.5907 populate_metadata read hostname 'smithi092' 2023-10-06T10:40:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.810+0000 7f30d7486b80 1 --2- 172.21.15.92:0/1985776347 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x5629ee058500 0x5629ee06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.810+0000 7f30d7486b80 4 client.5907 mds_command: new command op to 5736 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:40:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.810+0000 7f30d7486b80 1 -- 172.21.15.92:0/1985776347 --> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629edfff5c0 con 0x5629ee058500 2023-10-06T10:40:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.810+0000 7f30c77fe700 1 --2- 172.21.15.92:0/1985776347 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x5629ee058500 0x5629ee06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.811+0000 7f30a5ffb700 10 client.5907 ms_handle_connect on v2:172.21.15.92:6835/2458340622 2023-10-06T10:40:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.811+0000 7f30c77fe700 1 --2- 172.21.15.92:0/1985776347 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x5629ee058500 0x5629ee06eed0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=972250ba3f3d2bfb server_cookie=bc20c29da511e9d6 in_seq=0 out_seq=0 2023-10-06T10:40:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.811+0000 7f30a5ffb700 1 -- 172.21.15.92:0/1985776347 <== mds.0 v2:172.21.15.92:6835/2458340622 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5629edfff5c0 con 0x5629ee058500 2023-10-06T10:40:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.811+0000 7f30a5ffb700 10 client.5907 handle_command_reply: tid=0 2023-10-06T10:40:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.811+0000 7f30d7486b80 1 client.5907 shutdown 2023-10-06T10:40:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.811+0000 7f30d7486b80 1 -- 172.21.15.92:0/1985776347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee0309a0 msgr2=0x5629ee055b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.811+0000 7f30d7486b80 1 --2- 172.21.15.92:0/1985776347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee0309a0 0x5629ee055b10 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00b3c0 tx=0x7f30bc020000 comp rx=0 tx=0).stop 2023-10-06T10:40:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.811+0000 7f30d7486b80 1 -- 172.21.15.92:0/1985776347 shutdown_connections 2023-10-06T10:40:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.811+0000 7f30d7486b80 1 --2- 172.21.15.92:0/1985776347 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x5629ee058500 0x5629ee06eed0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.811+0000 7f30d7486b80 1 --2- 172.21.15.92:0/1985776347 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5629ee074a90 0x5629ee056050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.811+0000 7f30d7486b80 1 --2- 172.21.15.92:0/1985776347 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5629ee02ebd0 0x5629ee05a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.811+0000 7f30d7486b80 1 --2- 172.21.15.92:0/1985776347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee0309a0 0x5629ee055b10 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.811+0000 7f30a5ffb700 0 client.5907 ms_handle_reset on v2:172.21.15.92:6835/2458340622 2023-10-06T10:40:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.811+0000 7f30d7486b80 1 -- 172.21.15.92:0/1985776347 >> 172.21.15.92:0/1985776347 conn(0x5629ee02f070 msgr2=0x5629ee02a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.812+0000 7f30d7486b80 1 -- 172.21.15.92:0/1985776347 shutdown_connections 2023-10-06T10:40:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.812+0000 7f30d7486b80 1 -- 172.21.15.92:0/1985776347 wait complete. 2023-10-06T10:40:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.812+0000 7f30d7486b80 20 client.5907 trim_cache size 0 max 16384 2023-10-06T10:40:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.828+0000 7f30d7486b80 1 Processor -- start 2023-10-06T10:40:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.828+0000 7f30d7486b80 1 -- start start 2023-10-06T10:40:45.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.829+0000 7f30d7486b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee02ebd0 0x5629ee02fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.829+0000 7f30d7486b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5629ee0309a0 0x5629ee02ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.829+0000 7f30d7486b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5629ee074a90 0x5629ee0724a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.829+0000 7f30d7486b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629ee083a70 con 0x5629ee02ebd0 2023-10-06T10:40:45.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.829+0000 7f30d7486b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629ede00e30 con 0x5629ee0309a0 2023-10-06T10:40:45.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.829+0000 7f30d7486b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629ee0789c0 con 0x5629ee074a90 2023-10-06T10:40:45.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.829+0000 7f30c77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee02ebd0 0x5629ee02fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.829+0000 7f30c77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee02ebd0 0x5629ee02fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34908/0 (socket says 172.21.15.92:34908) 2023-10-06T10:40:45.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.829+0000 7f30c77fe700 1 -- 172.21.15.92:0/1940759560 learned_addr learned my addr 172.21.15.92:0/1940759560 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:45.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.829+0000 7f30c6ffd700 1 --2- 172.21.15.92:0/1940759560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5629ee0309a0 0x5629ee02ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.829+0000 7f30c7fff700 1 --2- 172.21.15.92:0/1940759560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5629ee074a90 0x5629ee0724a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.829+0000 7f30c77fe700 1 -- 172.21.15.92:0/1940759560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5629ee074a90 msgr2=0x5629ee0724a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:45.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.829+0000 7f30c77fe700 1 --2- 172.21.15.92:0/1940759560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5629ee074a90 0x5629ee0724a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.829+0000 7f30c77fe700 1 -- 172.21.15.92:0/1940759560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5629ee0309a0 msgr2=0x5629ee02ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:45.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.829+0000 7f30c77fe700 1 --2- 172.21.15.92:0/1940759560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5629ee0309a0 0x5629ee02ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.829+0000 7f30c77fe700 1 -- 172.21.15.92:0/1940759560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629edf4c780 con 0x5629ee02ebd0 2023-10-06T10:40:45.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.829+0000 7f30c77fe700 1 --2- 172.21.15.92:0/1940759560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee02ebd0 0x5629ee02fd00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f30b8082900 tx=0x7f30b807d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191c7a0754ad7814 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:45.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.829+0000 7f30ad7fa700 1 -- 172.21.15.92:0/1940759560 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8063070 con 0x5629ee02ebd0 2023-10-06T10:40:45.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.829+0000 7f30ad7fa700 1 -- 172.21.15.92:0/1940759560 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b8064040 con 0x5629ee02ebd0 2023-10-06T10:40:45.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.829+0000 7f30ad7fa700 1 -- 172.21.15.92:0/1940759560 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8067250 con 0x5629ee02ebd0 2023-10-06T10:40:45.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.830+0000 7f30d7486b80 1 -- 172.21.15.92:0/1940759560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee02ebd0 msgr2=0x5629ee02fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:45.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.830+0000 7f30d7486b80 1 --2- 172.21.15.92:0/1940759560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee02ebd0 0x5629ee02fd00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f30b8082900 tx=0x7f30b807d000 comp rx=0 tx=0).stop 2023-10-06T10:40:45.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.830+0000 7f30d7486b80 1 -- 172.21.15.92:0/1940759560 shutdown_connections 2023-10-06T10:40:45.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.830+0000 7f30d7486b80 1 --2- 172.21.15.92:0/1940759560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5629ee074a90 0x5629ee0724a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.830+0000 7f30d7486b80 1 --2- 172.21.15.92:0/1940759560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee02ebd0 0x5629ee02fd00 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.830+0000 7f30d7486b80 1 --2- 172.21.15.92:0/1940759560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5629ee0309a0 0x5629ee02ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.830+0000 7f30d7486b80 1 -- 172.21.15.92:0/1940759560 >> 172.21.15.92:0/1940759560 conn(0x5629ee02f070 msgr2=0x5629ee059ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:45.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.830+0000 7f30d7486b80 1 -- 172.21.15.92:0/1940759560 shutdown_connections 2023-10-06T10:40:45.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.830+0000 7f30d7486b80 1 -- 172.21.15.92:0/1940759560 wait complete. 2023-10-06T10:40:45.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.830+0000 7f30d7486b80 1 -- start start 2023-10-06T10:40:45.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.830+0000 7f30d7486b80 1 Processor -- start 2023-10-06T10:40:45.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.831+0000 7f30d7486b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee02ebd0 0x5629ee055af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.831+0000 7f30d7486b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5629ee0309a0 0x5629ee056030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.831+0000 7f30d7486b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5629ee074a90 0x5629ee0dce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.831+0000 7f30d7486b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629ee0d9b50 con 0x5629ee02ebd0 2023-10-06T10:40:45.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.831+0000 7f30d7486b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629ee055210 con 0x5629ee0309a0 2023-10-06T10:40:45.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.831+0000 7f30d7486b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629ee0dc9b0 con 0x5629ee074a90 2023-10-06T10:40:45.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.831+0000 7f30d7486b80 10 client.0 _ll_register_callbacks cb 0x5629ede1af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:40:45.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.831+0000 7f30c77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee02ebd0 0x5629ee055af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.831+0000 7f30c77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee02ebd0 0x5629ee055af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34912/0 (socket says 172.21.15.92:34912) 2023-10-06T10:40:45.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.831+0000 7f30c77fe700 1 -- 172.21.15.92:0/221746997 learned_addr learned my addr 172.21.15.92:0/221746997 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:45.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.831+0000 7f30c6ffd700 1 --2- 172.21.15.92:0/221746997 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5629ee0309a0 0x5629ee056030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.831+0000 7f30c7fff700 1 --2- 172.21.15.92:0/221746997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5629ee074a90 0x5629ee0dce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.832+0000 7f30c77fe700 1 -- 172.21.15.92:0/221746997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5629ee074a90 msgr2=0x5629ee0dce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:45.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.832+0000 7f30c77fe700 1 --2- 172.21.15.92:0/221746997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5629ee074a90 0x5629ee0dce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.832+0000 7f30c77fe700 1 -- 172.21.15.92:0/221746997 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5629ee0309a0 msgr2=0x5629ee056030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:45.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.832+0000 7f30c77fe700 1 --2- 172.21.15.92:0/221746997 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5629ee0309a0 0x5629ee056030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.832+0000 7f30c77fe700 1 -- 172.21.15.92:0/221746997 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629ede18270 con 0x5629ee02ebd0 2023-10-06T10:40:45.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.832+0000 7f30c77fe700 1 --2- 172.21.15.92:0/221746997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee02ebd0 0x5629ee055af0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f30b8055240 tx=0x7f30b8063000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd9b92e1eb4c274 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:45.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.832+0000 7f30a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:40:45.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.832+0000 7f30a5ffb700 1 -- 172.21.15.92:0/221746997 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b80552b0 con 0x5629ee02ebd0 2023-10-06T10:40:45.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.832+0000 7f30a5ffb700 1 -- 172.21.15.92:0/221746997 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f30b8053650 con 0x5629ee02ebd0 2023-10-06T10:40:45.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.832+0000 7f30d7486b80 1 -- 172.21.15.92:0/221746997 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30b8058500 con 0x5629ee02ebd0 2023-10-06T10:40:45.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.832+0000 7f30a5ffb700 1 -- 172.21.15.92:0/221746997 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30b8055510 con 0x5629ee02ebd0 2023-10-06T10:40:45.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.832+0000 7f30d7486b80 10 client.5913 fetch_fsmap learned FSMap version 32 2023-10-06T10:40:45.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.832+0000 7f30d7486b80 1 -- 172.21.15.92:0/221746997 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5629edf33470 con 0x5629ee02ebd0 2023-10-06T10:40:45.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.832+0000 7f30a5ffb700 1 -- 172.21.15.92:0/221746997 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b8058500 con 0x5629ee02ebd0 2023-10-06T10:40:45.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.832+0000 7f30a5ffb700 1 -- 172.21.15.92:0/221746997 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 32) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f30b80547a0 con 0x5629ee02ebd0 2023-10-06T10:40:45.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.832+0000 7f30d7486b80 10 client.5913 fetch_fsmap finished waiting for FSMap version 32 2023-10-06T10:40:45.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.832+0000 7f30d7486b80 10 client.5913 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T10:40:45.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.832+0000 7f30d7486b80 20 client.5913 populate_metadata read hostname 'smithi092' 2023-10-06T10:40:45.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.832+0000 7f30d7486b80 1 --2- 172.21.15.92:0/221746997 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x5629ee0729b0 0x5629ee0dd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.832+0000 7f30d7486b80 4 client.5913 mds_command: new command op to 5736 tid=0[{"prefix": "status"}] 2023-10-06T10:40:45.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.832+0000 7f30d7486b80 1 -- 172.21.15.92:0/221746997 --> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5629edf20e00 con 0x5629ee0729b0 2023-10-06T10:40:45.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.832+0000 7f30c6ffd700 1 --2- 172.21.15.92:0/221746997 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x5629ee0729b0 0x5629ee0dd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.833+0000 7f30a5ffb700 10 client.5913 ms_handle_connect on v2:172.21.15.92:6835/2458340622 2023-10-06T10:40:45.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.833+0000 7f30c6ffd700 1 --2- 172.21.15.92:0/221746997 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x5629ee0729b0 0x5629ee0dd380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7991bc89b79ce2d6 server_cookie=ea7e5a2670ab854c in_seq=0 out_seq=0 2023-10-06T10:40:45.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.833+0000 7f30a5ffb700 1 -- 172.21.15.92:0/221746997 <== mds.0 v2:172.21.15.92:6835/2458340622 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5629edf20e00 con 0x5629ee0729b0 2023-10-06T10:40:45.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.833+0000 7f30a5ffb700 10 client.5913 handle_command_reply: tid=0 2023-10-06T10:40:45.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.833+0000 7f30d7486b80 1 client.5913 shutdown 2023-10-06T10:40:45.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.833+0000 7f30d7486b80 1 -- 172.21.15.92:0/221746997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee02ebd0 msgr2=0x5629ee055af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:45.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.833+0000 7f30d7486b80 1 --2- 172.21.15.92:0/221746997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee02ebd0 0x5629ee055af0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f30b8055240 tx=0x7f30b8063000 comp rx=0 tx=0).stop 2023-10-06T10:40:45.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.833+0000 7f30d7486b80 1 -- 172.21.15.92:0/221746997 shutdown_connections 2023-10-06T10:40:45.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.833+0000 7f30d7486b80 1 --2- 172.21.15.92:0/221746997 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x5629ee0729b0 0x5629ee0dd380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.833+0000 7f30d7486b80 1 --2- 172.21.15.92:0/221746997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5629ee074a90 0x5629ee0dce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.833+0000 7f30a5ffb700 0 client.5913 ms_handle_reset on v2:172.21.15.92:6835/2458340622 2023-10-06T10:40:45.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.833+0000 7f30d7486b80 1 --2- 172.21.15.92:0/221746997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ee02ebd0 0x5629ee055af0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.833+0000 7f30d7486b80 1 --2- 172.21.15.92:0/221746997 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5629ee0309a0 0x5629ee056030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.833+0000 7f30d7486b80 1 -- 172.21.15.92:0/221746997 >> 172.21.15.92:0/221746997 conn(0x5629ee02f070 msgr2=0x5629ee06e490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:45.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.833+0000 7f30d7486b80 1 -- 172.21.15.92:0/221746997 shutdown_connections 2023-10-06T10:40:45.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.833+0000 7f30d7486b80 1 -- 172.21.15.92:0/221746997 wait complete. 2023-10-06T10:40:45.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.833+0000 7f30d7486b80 20 client.5913 trim_cache size 0 max 16384 2023-10-06T10:40:45.843 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:40:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:40:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T10:40:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 5736, 2023-10-06T10:40:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:40:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:40:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:40:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 10.291430196, 2023-10-06T10:40:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 32, 2023-10-06T10:40:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 52, 2023-10-06T10:40:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 52, 2023-10-06T10:40:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 16.416559590999999 2023-10-06T10:40:45.844 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:40:45.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.834+0000 7f30ce18e700 1 -- 172.21.15.92:0/2455736404 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f30b4066f30 msgr2=0x7f30b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:45.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.834+0000 7f30ce18e700 1 --2- 172.21.15.92:0/2455736404 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f30b4066f30 0x7f30b40693e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009910 tx=0x7f30bc017000 comp rx=0 tx=0).stop 2023-10-06T10:40:45.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.834+0000 7f30ce18e700 1 -- 172.21.15.92:0/2455736404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c80b0760 msgr2=0x7f30c809bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:45.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.834+0000 7f30ce18e700 1 --2- 172.21.15.92:0/2455736404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c80b0760 0x7f30c809bf50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f30b800c930 tx=0x7f30b80016f0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.834+0000 7f30ce18e700 1 -- 172.21.15.92:0/2455736404 shutdown_connections 2023-10-06T10:40:45.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.834+0000 7f30ce18e700 1 --2- 172.21.15.92:0/2455736404 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f30b4066f30 0x7f30b40693e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.834+0000 7f30ce18e700 1 --2- 172.21.15.92:0/2455736404 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c8159560 0x7f30c809d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.834+0000 7f30ce18e700 1 --2- 172.21.15.92:0/2455736404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c80b0760 0x7f30c809bf50 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.834+0000 7f30ce18e700 1 --2- 172.21.15.92:0/2455736404 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30c80b49a0 0x7f30c8156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.834+0000 7f30ce18e700 1 -- 172.21.15.92:0/2455736404 >> 172.21.15.92:0/2455736404 conn(0x7f30c80997f0 msgr2=0x7f30c8158b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:45.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.834+0000 7f30ce18e700 1 -- 172.21.15.92:0/2455736404 shutdown_connections 2023-10-06T10:40:45.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.834+0000 7f30ce18e700 1 -- 172.21.15.92:0/2455736404 wait complete. 2023-10-06T10:40:45.847 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T10:40:45.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.959+0000 7f48dd0fa700 1 Processor -- start 2023-10-06T10:40:45.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.959+0000 7f48dd0fa700 1 -- start start 2023-10-06T10:40:45.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.960+0000 7f48dd0fa700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48d8159440 0x7f48d80b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.960+0000 7f48dd0fa700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f48d80b35d0 0x7f48d80cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.960+0000 7f48dd0fa700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d80cb7a0 con 0x7f48d80cc170 2023-10-06T10:40:45.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.960+0000 7f48dd0fa700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d80b0860 con 0x7f48d80b35d0 2023-10-06T10:40:45.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.960+0000 7f48dd0fa700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d80cb150 con 0x7f48d8159440 2023-10-06T10:40:45.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.960+0000 7f48d77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f48d80b35d0 0x7f48d80cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.960+0000 7f48d77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f48d80b35d0 0x7f48d80cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34914/0 (socket says 172.21.15.92:34914) 2023-10-06T10:40:45.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.960+0000 7f48d77fe700 1 -- 172.21.15.92:0/1942105515 learned_addr learned my addr 172.21.15.92:0/1942105515 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:45.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.960+0000 7f48d7fff700 1 --2- 172.21.15.92:0/1942105515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48d8159440 0x7f48d80b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.960+0000 7f48d77fe700 1 -- 172.21.15.92:0/1942105515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48d8159440 msgr2=0x7f48d80b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:45.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.960+0000 7f48d77fe700 1 --2- 172.21.15.92:0/1942105515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48d8159440 0x7f48d80b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.960+0000 7f48d77fe700 1 -- 172.21.15.92:0/1942105515 >> v1:172.21.15.124:6789/0 conn(0x7f48d80cc170 legacy=0x7f48d80b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:45.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.961+0000 7f48d77fe700 1 -- 172.21.15.92:0/1942105515 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d80cb430 con 0x7f48d80b35d0 2023-10-06T10:40:45.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.961+0000 7f48d77fe700 1 --2- 172.21.15.92:0/1942105515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48d80b35d0 0x7f48d80cbc30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f48cc0099b0 tx=0x7f48cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ac8889a5be8a7f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:45.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.961+0000 7f48d6ffd700 1 -- 172.21.15.92:0/1942105515 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc00b860 con 0x7f48d80b35d0 2023-10-06T10:40:45.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.961+0000 7f48d6ffd700 1 -- 172.21.15.92:0/1942105515 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48cc00b9c0 con 0x7f48d80b35d0 2023-10-06T10:40:45.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.961+0000 7f48d6ffd700 1 -- 172.21.15.92:0/1942105515 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc00bd60 con 0x7f48d80b35d0 2023-10-06T10:40:45.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.961+0000 7f48dd0fa700 1 -- 172.21.15.92:0/1942105515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48d80b35d0 msgr2=0x7f48d80cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:45.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.961+0000 7f48dd0fa700 1 --2- 172.21.15.92:0/1942105515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48d80b35d0 0x7f48d80cbc30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f48cc0099b0 tx=0x7f48cc009250 comp rx=0 tx=0).stop 2023-10-06T10:40:45.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.961+0000 7f48dd0fa700 1 -- 172.21.15.92:0/1942105515 shutdown_connections 2023-10-06T10:40:45.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.961+0000 7f48dd0fa700 1 --2- 172.21.15.92:0/1942105515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48d8159440 0x7f48d80b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.961+0000 7f48dd0fa700 1 --2- 172.21.15.92:0/1942105515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48d80b35d0 0x7f48d80cbc30 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.961+0000 7f48dd0fa700 1 -- 172.21.15.92:0/1942105515 >> 172.21.15.92:0/1942105515 conn(0x7f48d806ece0 msgr2=0x7f48d806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:45.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.961+0000 7f48dd0fa700 1 -- 172.21.15.92:0/1942105515 shutdown_connections 2023-10-06T10:40:45.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.961+0000 7f48dd0fa700 1 -- 172.21.15.92:0/1942105515 wait complete. 2023-10-06T10:40:45.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.962+0000 7f48dd0fa700 1 Processor -- start 2023-10-06T10:40:45.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.963+0000 7f48dd0fa700 1 -- start start 2023-10-06T10:40:45.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.963+0000 7f48dd0fa700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f48d80b35d0 0x7f48d815b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.963+0000 7f48dd0fa700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48d80cc170 0x7f48d814a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.963+0000 7f48dd0fa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48d8159440 0x7f48d815a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.963+0000 7f48dd0fa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d80c72d0 con 0x7f48d8159440 2023-10-06T10:40:45.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.963+0000 7f48dd0fa700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d80c7e50 con 0x7f48d80b35d0 2023-10-06T10:40:45.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.963+0000 7f48dd0fa700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d814ad50 con 0x7f48d80cc170 2023-10-06T10:40:45.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.963+0000 7f48dc8f9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48d8159440 0x7f48d815a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.963+0000 7f48dc8f9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48d8159440 0x7f48d815a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34922/0 (socket says 172.21.15.92:34922) 2023-10-06T10:40:45.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.963+0000 7f48dc8f9700 1 -- 172.21.15.92:0/1808766887 learned_addr learned my addr 172.21.15.92:0/1808766887 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:45.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.964+0000 7f48d7fff700 1 --2- 172.21.15.92:0/1808766887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f48d80b35d0 0x7f48d815b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.964+0000 7f48d77fe700 1 --2- 172.21.15.92:0/1808766887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48d80cc170 0x7f48d814a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.964+0000 7f48dc8f9700 1 -- 172.21.15.92:0/1808766887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48d80cc170 msgr2=0x7f48d814a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.964+0000 7f48dc8f9700 1 --2- 172.21.15.92:0/1808766887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48d80cc170 0x7f48d814a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.964+0000 7f48dc8f9700 1 -- 172.21.15.92:0/1808766887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f48d80b35d0 msgr2=0x7f48d815b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.964+0000 7f48dc8f9700 1 --2- 172.21.15.92:0/1808766887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f48d80b35d0 0x7f48d815b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.964+0000 7f48dc8f9700 1 -- 172.21.15.92:0/1808766887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48cc0094f0 con 0x7f48d8159440 2023-10-06T10:40:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.964+0000 7f48dc8f9700 1 --2- 172.21.15.92:0/1808766887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48d8159440 0x7f48d815a440 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f48d000c8e0 tx=0x7f48d0001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.964+0000 7f48bffff700 1 -- 172.21.15.92:0/1808766887 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0012630 con 0x7f48d8159440 2023-10-06T10:40:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.964+0000 7f48bffff700 1 -- 172.21.15.92:0/1808766887 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d00127b0 con 0x7f48d8159440 2023-10-06T10:40:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.964+0000 7f48bffff700 1 -- 172.21.15.92:0/1808766887 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0012b70 con 0x7f48d8159440 2023-10-06T10:40:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.964+0000 7f48dd0fa700 1 -- 172.21.15.92:0/1808766887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d80c7520 con 0x7f48d8159440 2023-10-06T10:40:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.964+0000 7f48dd0fa700 1 -- 172.21.15.92:0/1808766887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d80c7ae0 con 0x7f48d8159440 2023-10-06T10:40:45.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.965+0000 7f48dd0fa700 1 -- 172.21.15.92:0/1808766887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d80c7fc0 con 0x7f48d8159440 2023-10-06T10:40:45.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.965+0000 7f48bffff700 1 -- 172.21.15.92:0/1808766887 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f48d0003030 con 0x7f48d8159440 2023-10-06T10:40:45.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.966+0000 7f48bffff700 1 --2- 172.21.15.92:0/1808766887 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f48c4066f30 0x7f48c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:45.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.966+0000 7f48bffff700 1 -- 172.21.15.92:0/1808766887 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f48d0084670 con 0x7f48d8159440 2023-10-06T10:40:45.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.966+0000 7f48d7fff700 1 --2- 172.21.15.92:0/1808766887 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f48c4066f30 0x7f48c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:45.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.966+0000 7f48d7fff700 1 --2- 172.21.15.92:0/1808766887 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f48c4066f30 0x7f48c40693e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f48c8006fd0 tx=0x7f48c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:45.970+0000 7f48bffff700 1 -- 172.21.15.92:0/1808766887 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48d0051640 con 0x7f48d8159440 2023-10-06T10:40:46.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.201+0000 7f48dd0fa700 1 -- 172.21.15.92:0/1808766887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f48d805f230 con 0x7f48d8159440 2023-10-06T10:40:46.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.201+0000 7f48bffff700 1 -- 172.21.15.92:0/1808766887 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7f48d0054c60 con 0x7f48d8159440 2023-10-06T10:40:46.203 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T10:40:46.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.203+0000 7f48dd0fa700 1 -- 172.21.15.92:0/1808766887 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f48c4066f30 msgr2=0x7f48c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:46.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.203+0000 7f48dd0fa700 1 --2- 172.21.15.92:0/1808766887 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f48c4066f30 0x7f48c40693e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f48c8006fd0 tx=0x7f48c8008040 comp rx=0 tx=0).stop 2023-10-06T10:40:46.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.203+0000 7f48dd0fa700 1 -- 172.21.15.92:0/1808766887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48d8159440 msgr2=0x7f48d815a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:46.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.203+0000 7f48dd0fa700 1 --2- 172.21.15.92:0/1808766887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48d8159440 0x7f48d815a440 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f48d000c8e0 tx=0x7f48d0001710 comp rx=0 tx=0).stop 2023-10-06T10:40:46.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.203+0000 7f48dd0fa700 1 -- 172.21.15.92:0/1808766887 shutdown_connections 2023-10-06T10:40:46.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.203+0000 7f48dd0fa700 1 --2- 172.21.15.92:0/1808766887 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f48c4066f30 0x7f48c40693e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:46.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.203+0000 7f48dd0fa700 1 --2- 172.21.15.92:0/1808766887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48d8159440 0x7f48d815a440 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:46.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.203+0000 7f48dd0fa700 1 --2- 172.21.15.92:0/1808766887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f48d80b35d0 0x7f48d815b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:46.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.203+0000 7f48dd0fa700 1 --2- 172.21.15.92:0/1808766887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48d80cc170 0x7f48d814a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:46.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.203+0000 7f48dd0fa700 1 -- 172.21.15.92:0/1808766887 >> 172.21.15.92:0/1808766887 conn(0x7f48d806ece0 msgr2=0x7f48d8157600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:46.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.203+0000 7f48dd0fa700 1 -- 172.21.15.92:0/1808766887 shutdown_connections 2023-10-06T10:40:46.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.203+0000 7f48dd0fa700 1 -- 172.21.15.92:0/1808766887 wait complete. 2023-10-06T10:40:46.217 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T10:40:46.217 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T10:40:46.217 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T10:40:46.217 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T10:40:46.217 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T10:40:46.217 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T10:40:46.217 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T10:40:46.217 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T10:40:46.217 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T10:40:46.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:40:46.218 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T10:40:46.232 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T10:40:46.233 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T10:40:46.233 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T10:40:46.233 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:40:46.233 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T10:40:46.233 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:40:46.233 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:40:46.233 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:40:46.233 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T10:40:46.233 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:40:46.233 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:40:46.233 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T10:40:46.234 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:40:46.234 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:40:46.234 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T10:40:46.234 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:40:46.234 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T10:40:46.234 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42363sec preferred_lft 42363sec 2023-10-06T10:40:46.234 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T10:40:46.234 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:40:46.234 INFO:teuthology.orchestra.run.smithi092.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T10:40:46.234 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 92:7f:88:24:4e:ca brd ff:ff:ff:ff:ff:ff 2023-10-06T10:40:46.234 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T10:40:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:40:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::9c9d:23ff:fee4:30ae/64 scope link 2023-10-06T10:40:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:40:46.235 INFO:teuthology.orchestra.run.smithi092.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T10:40:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 92:7f:88:24:4e:ca brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T10:40:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::907f:88ff:fe24:4eca/64 scope link 2023-10-06T10:40:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:40:46.235 INFO:teuthology.orchestra.run.smithi092.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T10:40:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether b6:32:f9:d9:96:1b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T10:40:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::b432:f9ff:fed9:961b/64 scope link 2023-10-06T10:40:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:40:46.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:40:46.236 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T10:40:46.295 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T10:40:46.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:40:46.297 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T10:40:46.354 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T10:40:46.354 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T10:40:46.355 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T10:40:46.355 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T10:40:46.355 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T10:40:46.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:40:46.356 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T10:40:46.356 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T10:40:46.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T10:40:46.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T10:40:46.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T10:40:46.356 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T10:40:46.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:40:46.455 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T10:40:46.469 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T10:40:46.470 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T10:40:46.470 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T10:40:46.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:40:46.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-10-06T10:40:46.534 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T10:40:46.534 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T10:40:46.590 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T10:40:46.590 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T10:40:46.590 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T10:40:46.644 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T10:40:46.644 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T10:40:46.644 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:40:46.724 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:40:46.724 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:40:46.743 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:40:46.743 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:40:46.743 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-06T10:40:46.743 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@8f307bfb-c4cb-4f66-aade-46fffe0c6e0c.cephfs=/ 2023-10-06T10:40:46.743 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-06T10:40:46.743 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T10:40:46.743 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T10:40:46.743 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T10:40:46.766 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:40:46.766 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:40:46.766 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-06T10:40:46.766 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@8f307bfb-c4cb-4f66-aade-46fffe0c6e0c.cephfs=/ 2023-10-06T10:40:46.766 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-06T10:40:46.766 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T10:40:46.766 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T10:40:46.766 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T10:40:46.766 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.124:6789,172.21.15.124:6790" 2023-10-06T10:40:46.767 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.124:6789,172.21.15.124:6790:/ 2023-10-06T10:40:46.767 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=8f307bfb-c4cb-4f66-aade-46fffe0c6e0c" will pass to kernel 2023-10-06T10:40:46.768 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T10:40:46.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:40:46.768 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T10:40:46.834 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-10-06T10:40:46.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.989+0000 7fef7d9db700 1 Processor -- start 2023-10-06T10:40:46.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.989+0000 7fef7d9db700 1 -- start start 2023-10-06T10:40:46.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.990+0000 7fef7d9db700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fef780cbc70 0x7fef780ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:46.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.990+0000 7fef7d9db700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef780cf5d0 con 0x7fef781598a0 2023-10-06T10:40:46.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.990+0000 7fef7d9db700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef780cf8b0 con 0x7fef781594d0 2023-10-06T10:40:46.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.990+0000 7fef7d9db700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef780cfb90 con 0x7fef780cbc70 2023-10-06T10:40:46.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.990+0000 7fef777fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fef780cbc70 0x7fef780ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:46.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.990+0000 7fef6ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fef781598a0 0x7fef780ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36594/0 (socket says 172.21.15.92:36594) 2023-10-06T10:40:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.990+0000 7fef777fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fef780cbc70 0x7fef780ceef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34924/0 (socket says 172.21.15.92:34924) 2023-10-06T10:40:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.990+0000 7fef6ffff700 1 -- 172.21.15.92:0/294268346 learned_addr learned my addr 172.21.15.92:0/294268346 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.990+0000 7fef777fe700 1 -- 172.21.15.92:0/294268346 >> v1:172.21.15.124:6789/0 conn(0x7fef781594d0 legacy=0x7fef780cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.990+0000 7fef777fe700 1 -- 172.21.15.92:0/294268346 >> v1:172.21.15.92:6789/0 conn(0x7fef781598a0 legacy=0x7fef780ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.990+0000 7fef777fe700 1 -- 172.21.15.92:0/294268346 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef780cfe70 con 0x7fef780cbc70 2023-10-06T10:40:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.990+0000 7fef767fc700 1 -- 172.21.15.92:0/294268346 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2921509752 0 0) 0x7fef780cf5d0 con 0x7fef781598a0 2023-10-06T10:40:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.990+0000 7fef777fe700 1 --2- 172.21.15.92:0/294268346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef780cbc70 0x7fef780ceef0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fef70009e60 tx=0x7fef70009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d297eb2f9d46b2bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.990+0000 7fef767fc700 1 -- 172.21.15.92:0/294268346 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7000e070 con 0x7fef780cbc70 2023-10-06T10:40:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.990+0000 7fef767fc700 1 -- 172.21.15.92:0/294268346 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7000bbf0 con 0x7fef780cbc70 2023-10-06T10:40:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.990+0000 7fef767fc700 1 -- 172.21.15.92:0/294268346 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70012430 con 0x7fef780cbc70 2023-10-06T10:40:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.991+0000 7fef7d9db700 1 -- 172.21.15.92:0/294268346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef780cbc70 msgr2=0x7fef780ceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.991+0000 7fef7d9db700 1 --2- 172.21.15.92:0/294268346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef780cbc70 0x7fef780ceef0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fef70009e60 tx=0x7fef70009780 comp rx=0 tx=0).stop 2023-10-06T10:40:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.991+0000 7fef7d9db700 1 -- 172.21.15.92:0/294268346 shutdown_connections 2023-10-06T10:40:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.991+0000 7fef7d9db700 1 --2- 172.21.15.92:0/294268346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef780cbc70 0x7fef780ceef0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.991+0000 7fef7d9db700 1 -- 172.21.15.92:0/294268346 >> 172.21.15.92:0/294268346 conn(0x7fef78089e50 msgr2=0x7fef78099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.991+0000 7fef7d9db700 1 -- 172.21.15.92:0/294268346 shutdown_connections 2023-10-06T10:40:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.991+0000 7fef7d9db700 1 -- 172.21.15.92:0/294268346 wait complete. 2023-10-06T10:40:46.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.992+0000 7fef7d9db700 1 Processor -- start 2023-10-06T10:40:46.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.992+0000 7fef7d9db700 1 -- start start 2023-10-06T10:40:46.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.993+0000 7fef7d9db700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef780cbc70 0x7fef78149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:46.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.993+0000 7fef7d9db700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef781594d0 0x7fef78154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:46.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.993+0000 7fef7d9db700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef781598a0 0x7fef78154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:46.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.993+0000 7fef7d9db700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef78157c10 con 0x7fef780cbc70 2023-10-06T10:40:46.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.993+0000 7fef7d9db700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef78157780 con 0x7fef781594d0 2023-10-06T10:40:46.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.993+0000 7fef7d9db700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef78156b70 con 0x7fef781598a0 2023-10-06T10:40:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.993+0000 7fef76ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef780cbc70 0x7fef78149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.993+0000 7fef76ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef780cbc70 0x7fef78149ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34936/0 (socket says 172.21.15.92:34936) 2023-10-06T10:40:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.993+0000 7fef76ffd700 1 -- 172.21.15.92:0/1793312694 learned_addr learned my addr 172.21.15.92:0/1793312694 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.993+0000 7fef777fe700 1 --2- 172.21.15.92:0/1793312694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef781598a0 0x7fef78154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.993+0000 7fef6ffff700 1 --2- 172.21.15.92:0/1793312694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef781594d0 0x7fef78154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.993+0000 7fef76ffd700 1 -- 172.21.15.92:0/1793312694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef781598a0 msgr2=0x7fef78154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.993+0000 7fef76ffd700 1 --2- 172.21.15.92:0/1793312694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef781598a0 0x7fef78154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.993+0000 7fef76ffd700 1 -- 172.21.15.92:0/1793312694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef781594d0 msgr2=0x7fef78154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.993+0000 7fef76ffd700 1 --2- 172.21.15.92:0/1793312694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef781594d0 0x7fef78154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.993+0000 7fef76ffd700 1 -- 172.21.15.92:0/1793312694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef70009a20 con 0x7fef780cbc70 2023-10-06T10:40:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.993+0000 7fef76ffd700 1 --2- 172.21.15.92:0/1793312694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef780cbc70 0x7fef78149ea0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fef6800c5d0 tx=0x7fef68007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.993+0000 7fef6effd700 1 -- 172.21.15.92:0/1793312694 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68011590 con 0x7fef780cbc70 2023-10-06T10:40:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.993+0000 7fef6effd700 1 -- 172.21.15.92:0/1793312694 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef68011710 con 0x7fef780cbc70 2023-10-06T10:40:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.994+0000 7fef7d9db700 1 -- 172.21.15.92:0/1793312694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef78156e50 con 0x7fef780cbc70 2023-10-06T10:40:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.994+0000 7fef6effd700 1 -- 172.21.15.92:0/1793312694 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68011ad0 con 0x7fef780cbc70 2023-10-06T10:40:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.994+0000 7fef7d9db700 1 -- 172.21.15.92:0/1793312694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef78157410 con 0x7fef780cbc70 2023-10-06T10:40:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.995+0000 7fef7d9db700 1 -- 172.21.15.92:0/1793312694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef78064a90 con 0x7fef780cbc70 2023-10-06T10:40:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.995+0000 7fef6effd700 1 -- 172.21.15.92:0/1793312694 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fef6801c460 con 0x7fef780cbc70 2023-10-06T10:40:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.995+0000 7fef6effd700 1 --2- 172.21.15.92:0/1793312694 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fef58066f30 0x7fef580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.995+0000 7fef6effd700 1 -- 172.21.15.92:0/1793312694 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fef68084570 con 0x7fef780cbc70 2023-10-06T10:40:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.995+0000 7fef6ffff700 1 --2- 172.21.15.92:0/1793312694 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fef58066f30 0x7fef580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.996+0000 7fef6ffff700 1 --2- 172.21.15.92:0/1793312694 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fef58066f30 0x7fef580693e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fef7809e330 tx=0x7fef60009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:46.999+0000 7fef6effd700 1 -- 172.21.15.92:0/1793312694 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef68051540 con 0x7fef780cbc70 2023-10-06T10:40:47.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.227+0000 7fef7d9db700 1 -- 172.21.15.92:0/1793312694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7fef78157a60 con 0x7fef780cbc70 2023-10-06T10:40:47.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.609+0000 7fef6effd700 1 -- 172.21.15.92:0/1793312694 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v33) v1 ==== 105+0+0 (secure 0 0 0) 0x7fef68054b60 con 0x7fef780cbc70 2023-10-06T10:40:47.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.611+0000 7fef7d9db700 1 -- 172.21.15.92:0/1793312694 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fef58066f30 msgr2=0x7fef580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:47.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.611+0000 7fef7d9db700 1 --2- 172.21.15.92:0/1793312694 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fef58066f30 0x7fef580693e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fef7809e330 tx=0x7fef60009040 comp rx=0 tx=0).stop 2023-10-06T10:40:47.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.611+0000 7fef7d9db700 1 -- 172.21.15.92:0/1793312694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef780cbc70 msgr2=0x7fef78149ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:47.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.611+0000 7fef7d9db700 1 --2- 172.21.15.92:0/1793312694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef780cbc70 0x7fef78149ea0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fef6800c5d0 tx=0x7fef68007f60 comp rx=0 tx=0).stop 2023-10-06T10:40:47.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.611+0000 7fef7d9db700 1 -- 172.21.15.92:0/1793312694 shutdown_connections 2023-10-06T10:40:47.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.611+0000 7fef7d9db700 1 --2- 172.21.15.92:0/1793312694 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fef58066f30 0x7fef580693e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:47.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.611+0000 7fef7d9db700 1 --2- 172.21.15.92:0/1793312694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef781598a0 0x7fef78154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:47.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.611+0000 7fef7d9db700 1 --2- 172.21.15.92:0/1793312694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef780cbc70 0x7fef78149ea0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:47.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.611+0000 7fef7d9db700 1 --2- 172.21.15.92:0/1793312694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef781594d0 0x7fef78154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:47.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.611+0000 7fef7d9db700 1 -- 172.21.15.92:0/1793312694 >> 172.21.15.92:0/1793312694 conn(0x7fef78089e50 msgr2=0x7fef780cccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:47.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.611+0000 7fef7d9db700 1 -- 172.21.15.92:0/1793312694 shutdown_connections 2023-10-06T10:40:47.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.611+0000 7fef7d9db700 1 -- 172.21.15.92:0/1793312694 wait complete. 2023-10-06T10:40:47.629 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:40:47.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.741+0000 7f3e6148f700 1 Processor -- start 2023-10-06T10:40:47.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.741+0000 7f3e6148f700 1 -- start start 2023-10-06T10:40:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.742+0000 7f3e6148f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e5c0cbc30 0x7f3e5c0cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.742+0000 7f3e6148f700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3e5c0b2100 0x7f3e5c0b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.742+0000 7f3e6148f700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e5c0cb7a0 con 0x7f3e5c159440 2023-10-06T10:40:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.742+0000 7f3e6148f700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e5c0b0860 con 0x7f3e5c0b2100 2023-10-06T10:40:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.742+0000 7f3e6148f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0cb150 con 0x7f3e5c0cbc30 2023-10-06T10:40:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.742+0000 7f3e5b7fe700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e5c0cbc30 0x7f3e5c0cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.742+0000 7f3e60c8e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3e5c0b2100 0x7f3e5c0b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.742+0000 7f3e5bfff700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f3e5c159440 0x7f3e5c0b3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:41036/0 (socket says 172.21.15.92:41036) 2023-10-06T10:40:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.742+0000 7f3e5bfff700 1 -- 172.21.15.92:0/3278463537 learned_addr learned my addr 172.21.15.92:0/3278463537 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.742+0000 7f3e60c8e700 1 -- 172.21.15.92:0/3278463537 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e5c0cbc30 msgr2=0x7f3e5c0cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.742+0000 7f3e60c8e700 1 --2- 172.21.15.92:0/3278463537 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e5c0cbc30 0x7f3e5c0cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.743+0000 7f3e60c8e700 1 -- 172.21.15.92:0/3278463537 >> v1:172.21.15.124:6789/0 conn(0x7f3e5c159440 legacy=0x7f3e5c0b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.743+0000 7f3e60c8e700 1 -- 172.21.15.92:0/3278463537 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5c0cb430 con 0x7f3e5c0b2100 2023-10-06T10:40:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.743+0000 7f3e5affd700 1 -- 172.21.15.92:0/3278463537 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122745264 0 0) 0x7f3e5c0cb7a0 con 0x7f3e5c159440 2023-10-06T10:40:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.743+0000 7f3e5b7fe700 1 --2- 172.21.15.92:0/3278463537 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e5c0cbc30 0x7f3e5c0cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:40:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.743+0000 7f3e60c8e700 1 --2- 172.21.15.92:0/3278463537 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e5c0b2100 0x7f3e5c0b2580 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009e60 tx=0x7f3e54009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe30ae7ae35f376d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.743+0000 7f3e5affd700 1 -- 172.21.15.92:0/3278463537 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5400e070 con 0x7f3e5c0b2100 2023-10-06T10:40:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.743+0000 7f3e5affd700 1 -- 172.21.15.92:0/3278463537 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5400bb60 con 0x7f3e5c0b2100 2023-10-06T10:40:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.743+0000 7f3e5affd700 1 -- 172.21.15.92:0/3278463537 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e54012430 con 0x7f3e5c0b2100 2023-10-06T10:40:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.743+0000 7f3e6148f700 1 -- 172.21.15.92:0/3278463537 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e5c0b2100 msgr2=0x7f3e5c0b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.743+0000 7f3e6148f700 1 --2- 172.21.15.92:0/3278463537 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e5c0b2100 0x7f3e5c0b2580 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009e60 tx=0x7f3e54009780 comp rx=0 tx=0).stop 2023-10-06T10:40:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.743+0000 7f3e6148f700 1 -- 172.21.15.92:0/3278463537 shutdown_connections 2023-10-06T10:40:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.743+0000 7f3e6148f700 1 --2- 172.21.15.92:0/3278463537 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e5c0b2100 0x7f3e5c0b2580 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.743+0000 7f3e6148f700 1 --2- 172.21.15.92:0/3278463537 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e5c0cbc30 0x7f3e5c0cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.743+0000 7f3e6148f700 1 -- 172.21.15.92:0/3278463537 >> 172.21.15.92:0/3278463537 conn(0x7f3e5c06ece0 msgr2=0x7f3e5c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.743+0000 7f3e6148f700 1 -- 172.21.15.92:0/3278463537 shutdown_connections 2023-10-06T10:40:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.744+0000 7f3e6148f700 1 -- 172.21.15.92:0/3278463537 wait complete. 2023-10-06T10:40:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.745+0000 7f3e6148f700 1 Processor -- start 2023-10-06T10:40:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.745+0000 7f3e6148f700 1 -- start start 2023-10-06T10:40:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.745+0000 7f3e6148f700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e5c0b2100 0x7f3e5c0c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.745+0000 7f3e6148f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e5c0cbc30 0x7f3e5c0c4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.745+0000 7f3e6148f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e5c159440 0x7f3e5c0c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.745+0000 7f3e6148f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0c9850 con 0x7f3e5c0cbc30 2023-10-06T10:40:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.745+0000 7f3e6148f700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0c9570 con 0x7f3e5c0b2100 2023-10-06T10:40:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.745+0000 7f3e6148f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0ce720 con 0x7f3e5c159440 2023-10-06T10:40:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.746+0000 7f3e5b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e5c0cbc30 0x7f3e5c0c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.746+0000 7f3e5b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e5c0cbc30 0x7f3e5c0c4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34952/0 (socket says 172.21.15.92:34952) 2023-10-06T10:40:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.746+0000 7f3e5b7fe700 1 -- 172.21.15.92:0/3486439750 learned_addr learned my addr 172.21.15.92:0/3486439750 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.746+0000 7f3e60c8e700 1 --2- 172.21.15.92:0/3486439750 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e5c159440 0x7f3e5c0c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.746+0000 7f3e5bfff700 1 --2- 172.21.15.92:0/3486439750 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e5c0b2100 0x7f3e5c0c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.746+0000 7f3e5b7fe700 1 -- 172.21.15.92:0/3486439750 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e5c159440 msgr2=0x7f3e5c0c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.746+0000 7f3e5b7fe700 1 --2- 172.21.15.92:0/3486439750 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e5c159440 0x7f3e5c0c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.746+0000 7f3e5b7fe700 1 -- 172.21.15.92:0/3486439750 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e5c0b2100 msgr2=0x7f3e5c0c3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.746+0000 7f3e5b7fe700 1 --2- 172.21.15.92:0/3486439750 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e5c0b2100 0x7f3e5c0c3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.746+0000 7f3e5b7fe700 1 -- 172.21.15.92:0/3486439750 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e50009420 con 0x7f3e5c0cbc30 2023-10-06T10:40:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.746+0000 7f3e5b7fe700 1 --2- 172.21.15.92:0/3486439750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e5c0cbc30 0x7f3e5c0c4140 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000ec80 tx=0x7f3e500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.746+0000 7f3e43fff700 1 -- 172.21.15.92:0/3486439750 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5000c9d0 con 0x7f3e5c0cbc30 2023-10-06T10:40:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.746+0000 7f3e43fff700 1 -- 172.21.15.92:0/3486439750 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5000cb30 con 0x7f3e5c0cbc30 2023-10-06T10:40:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.746+0000 7f3e6148f700 1 -- 172.21.15.92:0/3486439750 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e54009a20 con 0x7f3e5c0cbc30 2023-10-06T10:40:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.746+0000 7f3e43fff700 1 -- 172.21.15.92:0/3486439750 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5000ceb0 con 0x7f3e5c0cbc30 2023-10-06T10:40:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.746+0000 7f3e6148f700 1 -- 172.21.15.92:0/3486439750 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5c0cece0 con 0x7f3e5c0cbc30 2023-10-06T10:40:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.747+0000 7f3e6148f700 1 -- 172.21.15.92:0/3486439750 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e5c0cc080 con 0x7f3e5c0cbc30 2023-10-06T10:40:47.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.748+0000 7f3e43fff700 1 -- 172.21.15.92:0/3486439750 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3e50013680 con 0x7f3e5c0cbc30 2023-10-06T10:40:47.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.748+0000 7f3e43fff700 1 --2- 172.21.15.92:0/3486439750 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3e48066f30 0x7f3e480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:47.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.748+0000 7f3e43fff700 1 -- 172.21.15.92:0/3486439750 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3e50084c20 con 0x7f3e5c0cbc30 2023-10-06T10:40:47.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.748+0000 7f3e5bfff700 1 --2- 172.21.15.92:0/3486439750 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3e48066f30 0x7f3e480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:47.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.748+0000 7f3e5bfff700 1 --2- 172.21.15.92:0/3486439750 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3e48066f30 0x7f3e480693e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00c8f0 tx=0x7f3e4c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:47.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.752+0000 7f3e43fff700 1 -- 172.21.15.92:0/3486439750 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e50051bf0 con 0x7f3e5c0cbc30 2023-10-06T10:40:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.975+0000 7f3e6148f700 1 -- 172.21.15.92:0/3486439750 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e5c05f230 con 0x7f3e5c0cbc30 2023-10-06T10:40:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.975+0000 7f3e43fff700 1 -- 172.21.15.92:0/3486439750 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4616 (secure 0 0 0) 0x7f3e50055210 con 0x7f3e5c0cbc30 2023-10-06T10:40:47.977 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:47.978 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":34,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3668157650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3668157650},{"type":"v1","addr":"172.21.15.124:6834","nonce":3668157650}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2986058955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2986058955},{"type":"v1","addr":"172.21.15.124:6835","nonce":2986058955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:40:34.932631+0000","modified":"2023-10-06T10:40:47.610484+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":5736,"mds_1":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":1,"incarnation":34,"state":"up:creating","state_seq":1,"addr":"172.21.15.92:6836/831923836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":831923836},{"type":"v1","addr":"172.21.15.92:6836","nonce":831923836}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5736":{"gid":5736,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2458340622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2458340622},{"type":"v1","addr":"172.21.15.92:6837","nonce":2458340622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T10:40:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.977+0000 7f3e6148f700 1 -- 172.21.15.92:0/3486439750 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3e48066f30 msgr2=0x7f3e480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.978+0000 7f3e6148f700 1 --2- 172.21.15.92:0/3486439750 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3e48066f30 0x7f3e480693e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00c8f0 tx=0x7f3e4c009040 comp rx=0 tx=0).stop 2023-10-06T10:40:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.978+0000 7f3e6148f700 1 -- 172.21.15.92:0/3486439750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e5c0cbc30 msgr2=0x7f3e5c0c4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.978+0000 7f3e6148f700 1 --2- 172.21.15.92:0/3486439750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e5c0cbc30 0x7f3e5c0c4140 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000ec80 tx=0x7f3e500091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.978+0000 7f3e6148f700 1 -- 172.21.15.92:0/3486439750 shutdown_connections 2023-10-06T10:40:47.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.978+0000 7f3e6148f700 1 --2- 172.21.15.92:0/3486439750 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3e48066f30 0x7f3e480693e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:47.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.978+0000 7f3e6148f700 1 --2- 172.21.15.92:0/3486439750 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e5c159440 0x7f3e5c0c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:47.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.978+0000 7f3e6148f700 1 --2- 172.21.15.92:0/3486439750 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e5c0b2100 0x7f3e5c0c3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:47.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.978+0000 7f3e6148f700 1 --2- 172.21.15.92:0/3486439750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e5c0cbc30 0x7f3e5c0c4140 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:47.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.978+0000 7f3e6148f700 1 -- 172.21.15.92:0/3486439750 >> 172.21.15.92:0/3486439750 conn(0x7f3e5c06ece0 msgr2=0x7f3e5c158780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:47.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.978+0000 7f3e6148f700 1 -- 172.21.15.92:0/3486439750 shutdown_connections 2023-10-06T10:40:47.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:47.978+0000 7f3e6148f700 1 -- 172.21.15.92:0/3486439750 wait complete. 2023-10-06T10:40:47.982 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 34 2023-10-06T10:40:48.395 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T10:40:34.932631+0000', 'modified': '2023-10-06T10:40:47.610484+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 5736, 'mds_1': 5733}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5733': {'gid': 5733, 'name': 'c', 'rank': 1, 'incarnation': 34, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.92:6836/831923836', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 831923836}, {'type': 'v1', 'addr': '172.21.15.92:6836', 'nonce': 831923836}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_5736': {'gid': 5736, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6837/2458340622', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 2458340622}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 2458340622}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T10:40:48.395 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5733:up:creating 2023-10-06T10:40:49.396 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:40:49.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.520+0000 7f1cb19f5700 1 Processor -- start 2023-10-06T10:40:49.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.520+0000 7f1cb19f5700 1 -- start start 2023-10-06T10:40:49.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.521+0000 7f1cb19f5700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cac06eac0 0x7f1cac0ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:49.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.521+0000 7f1cb19f5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cac0acd20 con 0x7f1cac156500 2023-10-06T10:40:49.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.521+0000 7f1cb19f5700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cac1568d0 con 0x7f1cac06e650 2023-10-06T10:40:49.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.521+0000 7f1cb19f5700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac06f430 con 0x7f1cac06eac0 2023-10-06T10:40:49.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.521+0000 7f1caaffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1cac156500 0x7f1cac0c7030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36602/0 (socket says 172.21.15.92:36602) 2023-10-06T10:40:49.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.521+0000 7f1caaffd700 1 -- 172.21.15.92:0/1515113496 learned_addr learned my addr 172.21.15.92:0/1515113496 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:49.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.521+0000 7f1cab7fe700 1 --2- 172.21.15.92:0/1515113496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cac06eac0 0x7f1cac0ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:49.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.521+0000 7f1ca9ffb700 1 -- 172.21.15.92:0/1515113496 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661936258 0 0) 0x7f1cac0acd20 con 0x7f1cac156500 2023-10-06T10:40:49.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.521+0000 7f1ca9ffb700 1 -- 172.21.15.92:0/1515113496 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c94003570 con 0x7f1cac156500 2023-10-06T10:40:49.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.521+0000 7f1ca9ffb700 1 -- 172.21.15.92:0/1515113496 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970284776 0 0) 0x7f1cac1568d0 con 0x7f1cac06e650 2023-10-06T10:40:49.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.521+0000 7f1ca9ffb700 1 -- 172.21.15.92:0/1515113496 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cac0acd20 con 0x7f1cac06e650 2023-10-06T10:40:49.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.521+0000 7f1ca9ffb700 1 -- 172.21.15.92:0/1515113496 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3582458659 0 0) 0x7f1c94003570 con 0x7f1cac156500 2023-10-06T10:40:49.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.521+0000 7f1cab7fe700 1 -- 172.21.15.92:0/1515113496 >> v1:172.21.15.124:6789/0 conn(0x7f1cac06e650 legacy=0x7f1cac0ad1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:49.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.521+0000 7f1cab7fe700 1 -- 172.21.15.92:0/1515113496 >> v1:172.21.15.92:6789/0 conn(0x7f1cac156500 legacy=0x7f1cac0c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:49.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.521+0000 7f1cab7fe700 1 -- 172.21.15.92:0/1515113496 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac0c7b30 con 0x7f1cac06eac0 2023-10-06T10:40:49.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.522+0000 7f1cab7fe700 1 --2- 172.21.15.92:0/1515113496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cac06eac0 0x7f1cac0ad8c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009e90 tx=0x7f1ca4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cb61319f86a3fce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:49.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.522+0000 7f1ca9ffb700 1 -- 172.21.15.92:0/1515113496 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca400e070 con 0x7f1cac06eac0 2023-10-06T10:40:49.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.522+0000 7f1ca9ffb700 1 -- 172.21.15.92:0/1515113496 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca400b870 con 0x7f1cac06eac0 2023-10-06T10:40:49.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.522+0000 7f1ca9ffb700 1 -- 172.21.15.92:0/1515113496 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0001470 con 0x7f1cac06eac0 2023-10-06T10:40:49.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.522+0000 7f1cb19f5700 1 -- 172.21.15.92:0/1515113496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cac06eac0 msgr2=0x7f1cac0ad8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:49.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.522+0000 7f1cb19f5700 1 --2- 172.21.15.92:0/1515113496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cac06eac0 0x7f1cac0ad8c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009e90 tx=0x7f1ca4009780 comp rx=0 tx=0).stop 2023-10-06T10:40:49.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.522+0000 7f1cb19f5700 1 -- 172.21.15.92:0/1515113496 shutdown_connections 2023-10-06T10:40:49.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.522+0000 7f1cb19f5700 1 --2- 172.21.15.92:0/1515113496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cac06eac0 0x7f1cac0ad8c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:49.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.522+0000 7f1cb19f5700 1 -- 172.21.15.92:0/1515113496 >> 172.21.15.92:0/1515113496 conn(0x7f1cac068080 msgr2=0x7f1cac0cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:49.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.522+0000 7f1cb19f5700 1 -- 172.21.15.92:0/1515113496 shutdown_connections 2023-10-06T10:40:49.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.522+0000 7f1cb19f5700 1 -- 172.21.15.92:0/1515113496 wait complete. 2023-10-06T10:40:49.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.524+0000 7f1cb19f5700 1 Processor -- start 2023-10-06T10:40:49.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.524+0000 7f1cb19f5700 1 -- start start 2023-10-06T10:40:49.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.524+0000 7f1cb19f5700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1cac06e650 0x7f1cac14a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:49.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.524+0000 7f1cb19f5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cac06eac0 0x7f1cac152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:49.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.524+0000 7f1cb19f5700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cac156500 0x7f1cac1533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:49.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.524+0000 7f1cb19f5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac0c1010 con 0x7f1cac06eac0 2023-10-06T10:40:49.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.524+0000 7f1cb19f5700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac0afe80 con 0x7f1cac06e650 2023-10-06T10:40:49.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.524+0000 7f1cb19f5700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac0b0160 con 0x7f1cac156500 2023-10-06T10:40:49.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.524+0000 7f1caa7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cac06eac0 0x7f1cac152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:49.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.524+0000 7f1caa7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cac06eac0 0x7f1cac152e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34960/0 (socket says 172.21.15.92:34960) 2023-10-06T10:40:49.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.524+0000 7f1caa7fc700 1 -- 172.21.15.92:0/2215155133 learned_addr learned my addr 172.21.15.92:0/2215155133 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:49.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.525+0000 7f1cab7fe700 1 --2- 172.21.15.92:0/2215155133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cac156500 0x7f1cac1533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:49.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.525+0000 7f1caaffd700 1 --2- 172.21.15.92:0/2215155133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1cac06e650 0x7f1cac14a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:49.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.525+0000 7f1caa7fc700 1 -- 172.21.15.92:0/2215155133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cac156500 msgr2=0x7f1cac1533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:49.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.525+0000 7f1caa7fc700 1 --2- 172.21.15.92:0/2215155133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cac156500 0x7f1cac1533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:49.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.525+0000 7f1caa7fc700 1 -- 172.21.15.92:0/2215155133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1cac06e650 msgr2=0x7f1cac14a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:49.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.525+0000 7f1caa7fc700 1 --2- 172.21.15.92:0/2215155133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1cac06e650 0x7f1cac14a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:49.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.525+0000 7f1caa7fc700 1 -- 172.21.15.92:0/2215155133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca4009a20 con 0x7f1cac06eac0 2023-10-06T10:40:49.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.525+0000 7f1caa7fc700 1 --2- 172.21.15.92:0/2215155133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cac06eac0 0x7f1cac152e90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00e9d0 tx=0x7f1c9c007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:49.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.525+0000 7f1c92ffd700 1 -- 172.21.15.92:0/2215155133 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c00c740 con 0x7f1cac06eac0 2023-10-06T10:40:49.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.525+0000 7f1c92ffd700 1 -- 172.21.15.92:0/2215155133 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9c00c8e0 con 0x7f1cac06eac0 2023-10-06T10:40:49.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.525+0000 7f1cb19f5700 1 -- 172.21.15.92:0/2215155133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cac0b0440 con 0x7f1cac06eac0 2023-10-06T10:40:49.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.525+0000 7f1c92ffd700 1 -- 172.21.15.92:0/2215155133 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c00cca0 con 0x7f1cac06eac0 2023-10-06T10:40:49.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.525+0000 7f1cb19f5700 1 -- 172.21.15.92:0/2215155133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cac14ae50 con 0x7f1cac06eac0 2023-10-06T10:40:49.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.526+0000 7f1cb19f5700 1 -- 172.21.15.92:0/2215155133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cac064a90 con 0x7f1cac06eac0 2023-10-06T10:40:49.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.526+0000 7f1c92ffd700 1 -- 172.21.15.92:0/2215155133 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1c9c011680 con 0x7f1cac06eac0 2023-10-06T10:40:49.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.527+0000 7f1c92ffd700 1 --2- 172.21.15.92:0/2215155133 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1c98066f30 0x7f1c980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:49.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.527+0000 7f1c92ffd700 1 -- 172.21.15.92:0/2215155133 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1c9c015070 con 0x7f1cac06eac0 2023-10-06T10:40:49.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.527+0000 7f1caaffd700 1 --2- 172.21.15.92:0/2215155133 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1c98066f30 0x7f1c980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:49.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.527+0000 7f1caaffd700 1 --2- 172.21.15.92:0/2215155133 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1c98066f30 0x7f1c980693e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00011e0 tx=0x7f1ca0000bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:49.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.531+0000 7f1c92ffd700 1 -- 172.21.15.92:0/2215155133 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c9c054070 con 0x7f1cac06eac0 2023-10-06T10:40:49.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.757+0000 7f1cb19f5700 1 -- 172.21.15.92:0/2215155133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1cac0c1150 con 0x7f1cac06eac0 2023-10-06T10:40:49.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.757+0000 7f1c92ffd700 1 -- 172.21.15.92:0/2215155133 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4628 (secure 0 0 0) 0x7f1c9c057690 con 0x7f1cac06eac0 2023-10-06T10:40:49.759 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:49.759 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":37,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3668157650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3668157650},{"type":"v1","addr":"172.21.15.124:6834","nonce":3668157650}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:40:34.932631+0000","modified":"2023-10-06T10:40:49.624417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5736,"mds_1":5733,"mds_2":5036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5036":{"gid":5036,"name":"d","rank":2,"incarnation":36,"state":"up:active","state_seq":6,"addr":"172.21.15.124:6835/2986058955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2986058955},{"type":"v1","addr":"172.21.15.124:6835","nonce":2986058955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5733":{"gid":5733,"name":"c","rank":1,"incarnation":34,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6836/831923836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":831923836},{"type":"v1","addr":"172.21.15.92:6836","nonce":831923836}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5736":{"gid":5736,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2458340622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2458340622},{"type":"v1","addr":"172.21.15.92:6837","nonce":2458340622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T10:40:49.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.759+0000 7f1cb19f5700 1 -- 172.21.15.92:0/2215155133 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1c98066f30 msgr2=0x7f1c980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:49.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.759+0000 7f1cb19f5700 1 --2- 172.21.15.92:0/2215155133 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1c98066f30 0x7f1c980693e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00011e0 tx=0x7f1ca0000bc0 comp rx=0 tx=0).stop 2023-10-06T10:40:49.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.759+0000 7f1cb19f5700 1 -- 172.21.15.92:0/2215155133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cac06eac0 msgr2=0x7f1cac152e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:49.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.759+0000 7f1cb19f5700 1 --2- 172.21.15.92:0/2215155133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cac06eac0 0x7f1cac152e90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00e9d0 tx=0x7f1c9c007ea0 comp rx=0 tx=0).stop 2023-10-06T10:40:49.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.759+0000 7f1cb19f5700 1 -- 172.21.15.92:0/2215155133 shutdown_connections 2023-10-06T10:40:49.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.759+0000 7f1cb19f5700 1 --2- 172.21.15.92:0/2215155133 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1c98066f30 0x7f1c980693e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:49.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.759+0000 7f1cb19f5700 1 --2- 172.21.15.92:0/2215155133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cac156500 0x7f1cac1533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:49.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.759+0000 7f1cb19f5700 1 --2- 172.21.15.92:0/2215155133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1cac06e650 0x7f1cac14a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:49.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.759+0000 7f1cb19f5700 1 --2- 172.21.15.92:0/2215155133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cac06eac0 0x7f1cac152e90 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:49.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.759+0000 7f1cb19f5700 1 -- 172.21.15.92:0/2215155133 >> 172.21.15.92:0/2215155133 conn(0x7f1cac068080 msgr2=0x7f1cac155800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:49.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.759+0000 7f1cb19f5700 1 -- 172.21.15.92:0/2215155133 shutdown_connections 2023-10-06T10:40:49.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:49.759+0000 7f1cb19f5700 1 -- 172.21.15.92:0/2215155133 wait complete. 2023-10-06T10:40:49.764 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 37 2023-10-06T10:40:49.839 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-10-06T10:40:49.837+0000 7fcf3b9ca700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-06T10:40:49.839 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-10-06T10:40:49.837+0000 7fcf3b9ca700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-10-06T10:40:50.176 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T10:40:34.932631+0000', 'modified': '2023-10-06T10:40:49.624417+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 5736, 'mds_1': 5733, 'mds_2': 5036}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5036': {'gid': 5036, 'name': 'd', 'rank': 2, 'incarnation': 36, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.124:6835/2986058955', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6833', 'nonce': 2986058955}, {'type': 'v1', 'addr': '172.21.15.124:6835', 'nonce': 2986058955}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_5733': {'gid': 5733, 'name': 'c', 'rank': 1, 'incarnation': 34, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.92:6836/831923836', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 831923836}, {'type': 'v1', 'addr': '172.21.15.92:6836', 'nonce': 831923836}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_5736': {'gid': 5736, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6837/2458340622', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 2458340622}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 2458340622}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T10:40:50.176 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-10-06T10:40:50.176 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T10:40:50.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.291+0000 7f30e0993700 1 Processor -- start 2023-10-06T10:40:50.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.291+0000 7f30e0993700 1 -- start start 2023-10-06T10:40:50.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.292+0000 7f30e0993700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f30dc0ca370 0x7f30dc158270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.292+0000 7f30e0993700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30dc0ce8a0 0x7f30dc15aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.292+0000 7f30e0993700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30dc157de0 con 0x7f30dc0adb60 2023-10-06T10:40:50.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.292+0000 7f30e0993700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30dc0cbb90 con 0x7f30dc0ca370 2023-10-06T10:40:50.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.292+0000 7f30e0993700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc0cb5d0 con 0x7f30dc0ce8a0 2023-10-06T10:40:50.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.292+0000 7f30dad9d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30dc0ce8a0 0x7f30dc15aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.292+0000 7f30da59c700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f30dc0ca370 0x7f30dc158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.292+0000 7f30d1d9b700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f30dc0adb60 0x7f30dc0ce190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:41052/0 (socket says 172.21.15.92:41052) 2023-10-06T10:40:50.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.292+0000 7f30d1d9b700 1 -- 172.21.15.92:0/2518162243 learned_addr learned my addr 172.21.15.92:0/2518162243 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:50.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.292+0000 7f30dad9d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30dc0ce8a0 0x7f30dc15aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:58054/0 (socket says 172.21.15.92:58054) 2023-10-06T10:40:50.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.293+0000 7f30da59c700 1 -- 172.21.15.92:0/2518162243 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30dc0ce8a0 msgr2=0x7f30dc15aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.293+0000 7f30da59c700 1 --2- 172.21.15.92:0/2518162243 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30dc0ce8a0 0x7f30dc15aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.293+0000 7f30da59c700 1 -- 172.21.15.92:0/2518162243 >> v1:172.21.15.124:6789/0 conn(0x7f30dc0adb60 legacy=0x7f30dc0ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.293+0000 7f30da59c700 1 -- 172.21.15.92:0/2518162243 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30dc0cb8b0 con 0x7f30dc0ca370 2023-10-06T10:40:50.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.293+0000 7f30d9d9b700 1 -- 172.21.15.92:0/2518162243 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4256939811 0 0) 0x7f30dc157de0 con 0x7f30dc0adb60 2023-10-06T10:40:50.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.293+0000 7f30dad9d700 1 --2- 172.21.15.92:0/2518162243 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30dc0ce8a0 0x7f30dc15aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:40:50.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.293+0000 7f30da59c700 1 --2- 172.21.15.92:0/2518162243 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30dc0ca370 0x7f30dc158270 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0098e0 tx=0x7f30cc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d65e35de2cd9573 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.293+0000 7f30d9d9b700 1 -- 172.21.15.92:0/2518162243 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc00b640 con 0x7f30dc0ca370 2023-10-06T10:40:50.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.293+0000 7f30d9d9b700 1 -- 172.21.15.92:0/2518162243 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30cc00b7a0 con 0x7f30dc0ca370 2023-10-06T10:40:50.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.293+0000 7f30d9d9b700 1 -- 172.21.15.92:0/2518162243 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc00bb40 con 0x7f30dc0ca370 2023-10-06T10:40:50.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.294+0000 7f30e0993700 1 -- 172.21.15.92:0/2518162243 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30dc0ca370 msgr2=0x7f30dc158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.294+0000 7f30e0993700 1 --2- 172.21.15.92:0/2518162243 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30dc0ca370 0x7f30dc158270 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0098e0 tx=0x7f30cc0091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.294+0000 7f30e0993700 1 -- 172.21.15.92:0/2518162243 shutdown_connections 2023-10-06T10:40:50.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.294+0000 7f30e0993700 1 --2- 172.21.15.92:0/2518162243 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30dc0ce8a0 0x7f30dc15aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.294+0000 7f30e0993700 1 --2- 172.21.15.92:0/2518162243 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30dc0ca370 0x7f30dc158270 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.294+0000 7f30e0993700 1 -- 172.21.15.92:0/2518162243 >> 172.21.15.92:0/2518162243 conn(0x7f30dc0707d0 msgr2=0x7f30dc070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:50.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.294+0000 7f30e0993700 1 -- 172.21.15.92:0/2518162243 shutdown_connections 2023-10-06T10:40:50.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.294+0000 7f30e0993700 1 -- 172.21.15.92:0/2518162243 wait complete. 2023-10-06T10:40:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.295+0000 7f30e0993700 1 Processor -- start 2023-10-06T10:40:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.295+0000 7f30e0993700 1 -- start start 2023-10-06T10:40:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.296+0000 7f30e0993700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30dc0adb60 0x7f30dc0b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.296+0000 7f30e0993700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30dc0ca370 0x7f30dc0b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.296+0000 7f30e0993700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30dc0ce8a0 0x7f30dc0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.296+0000 7f30e0993700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc0c3320 con 0x7f30dc0ce8a0 2023-10-06T10:40:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.296+0000 7f30e0993700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc0c2e90 con 0x7f30dc0adb60 2023-10-06T10:40:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.296+0000 7f30e0993700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc0c2280 con 0x7f30dc0ca370 2023-10-06T10:40:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.296+0000 7f30dad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30dc0ce8a0 0x7f30dc0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.296+0000 7f30dad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30dc0ce8a0 0x7f30dc0b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34966/0 (socket says 172.21.15.92:34966) 2023-10-06T10:40:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.296+0000 7f30dad9d700 1 -- 172.21.15.92:0/3206637812 learned_addr learned my addr 172.21.15.92:0/3206637812 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.296+0000 7f30da59c700 1 --2- 172.21.15.92:0/3206637812 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30dc0adb60 0x7f30dc0b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.296+0000 7f30d1d9b700 1 --2- 172.21.15.92:0/3206637812 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30dc0ca370 0x7f30dc0b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.296+0000 7f30dad9d700 1 -- 172.21.15.92:0/3206637812 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30dc0ca370 msgr2=0x7f30dc0b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.296+0000 7f30dad9d700 1 --2- 172.21.15.92:0/3206637812 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30dc0ca370 0x7f30dc0b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.296+0000 7f30dad9d700 1 -- 172.21.15.92:0/3206637812 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30dc0adb60 msgr2=0x7f30dc0b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.296+0000 7f30dad9d700 1 --2- 172.21.15.92:0/3206637812 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30dc0adb60 0x7f30dc0b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.296+0000 7f30dad9d700 1 -- 172.21.15.92:0/3206637812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d4009a20 con 0x7f30dc0ce8a0 2023-10-06T10:40:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.296+0000 7f30dad9d700 1 --2- 172.21.15.92:0/3206637812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30dc0ce8a0 0x7f30dc0b15c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f30d400ef90 tx=0x7f30d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.296+0000 7f30d2ffd700 1 -- 172.21.15.92:0/3206637812 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d400cbd0 con 0x7f30dc0ce8a0 2023-10-06T10:40:50.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.297+0000 7f30e0993700 1 -- 172.21.15.92:0/3206637812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30cc009420 con 0x7f30dc0ce8a0 2023-10-06T10:40:50.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.297+0000 7f30d2ffd700 1 -- 172.21.15.92:0/3206637812 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d400f040 con 0x7f30dc0ce8a0 2023-10-06T10:40:50.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.297+0000 7f30d2ffd700 1 -- 172.21.15.92:0/3206637812 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d4014480 con 0x7f30dc0ce8a0 2023-10-06T10:40:50.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.297+0000 7f30e0993700 1 -- 172.21.15.92:0/3206637812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30dc0c2840 con 0x7f30dc0ce8a0 2023-10-06T10:40:50.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.298+0000 7f30d2ffd700 1 -- 172.21.15.92:0/3206637812 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f30d4014870 con 0x7f30dc0ce8a0 2023-10-06T10:40:50.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.298+0000 7f30d2ffd700 1 --2- 172.21.15.92:0/3206637812 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f30bc066f30 0x7f30bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.298+0000 7f30d2ffd700 1 -- 172.21.15.92:0/3206637812 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f30d40855c0 con 0x7f30dc0ce8a0 2023-10-06T10:40:50.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.298+0000 7f30da59c700 1 --2- 172.21.15.92:0/3206637812 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f30bc066f30 0x7f30bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.299+0000 7f30da59c700 1 --2- 172.21.15.92:0/3206637812 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f30bc066f30 0x7f30bc0693e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0093f0 tx=0x7f30cc017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.300+0000 7f30e9c8bb80 1 Processor -- start 2023-10-06T10:40:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.300+0000 7f30e9c8bb80 1 -- start start 2023-10-06T10:40:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.301+0000 7f30e9c8bb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b4ef13a90 0x557b4ef13e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.301+0000 7f30e9c8bb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b4eecf9a0 0x557b4eecc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.301+0000 7f30e9c8bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecdbd0 0x557b4eeccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.301+0000 7f30e9c8bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b4ee4cc80 con 0x557b4eecdbd0 2023-10-06T10:40:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.301+0000 7f30e9c8bb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b4ee23a70 con 0x557b4ef13a90 2023-10-06T10:40:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.301+0000 7f30e9c8bb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b4ee21270 con 0x557b4eecf9a0 2023-10-06T10:40:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.301+0000 7f30dad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecdbd0 0x557b4eeccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.301+0000 7f30dad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecdbd0 0x557b4eeccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34972/0 (socket says 172.21.15.92:34972) 2023-10-06T10:40:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.301+0000 7f30dad9d700 1 -- 172.21.15.92:0/1113647145 learned_addr learned my addr 172.21.15.92:0/1113647145 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.301+0000 7f30dad9d700 1 -- 172.21.15.92:0/1113647145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b4eecf9a0 msgr2=0x557b4eecc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.301+0000 7f30dad9d700 1 --2- 172.21.15.92:0/1113647145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b4eecf9a0 0x557b4eecc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.301+0000 7f30dad9d700 1 -- 172.21.15.92:0/1113647145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b4ef13a90 msgr2=0x557b4ef13e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.301+0000 7f30da59c700 1 --2- 172.21.15.92:0/1113647145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b4ef13a90 0x557b4ef13e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.301+0000 7f30dad9d700 1 --2- 172.21.15.92:0/1113647145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b4ef13a90 0x557b4ef13e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.301+0000 7f30dad9d700 1 -- 172.21.15.92:0/1113647145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b4ecb7270 con 0x557b4eecdbd0 2023-10-06T10:40:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.302+0000 7f30dad9d700 1 --2- 172.21.15.92:0/1113647145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecdbd0 0x557b4eeccac0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f30d40526b0 tx=0x7f30d4083700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3c175a3d53d964 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.302+0000 7f30bb7fe700 1 -- 172.21.15.92:0/1113647145 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d4059400 con 0x557b4eecdbd0 2023-10-06T10:40:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.302+0000 7f30bb7fe700 1 -- 172.21.15.92:0/1113647145 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d405b050 con 0x557b4eecdbd0 2023-10-06T10:40:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.302+0000 7f30bb7fe700 1 -- 172.21.15.92:0/1113647145 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d405d150 con 0x557b4eecdbd0 2023-10-06T10:40:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.302+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/1113647145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecdbd0 msgr2=0x557b4eeccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.302+0000 7f30e9c8bb80 1 --2- 172.21.15.92:0/1113647145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecdbd0 0x557b4eeccac0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f30d40526b0 tx=0x7f30d4083700 comp rx=0 tx=0).stop 2023-10-06T10:40:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.302+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/1113647145 shutdown_connections 2023-10-06T10:40:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.302+0000 7f30e9c8bb80 1 --2- 172.21.15.92:0/1113647145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecdbd0 0x557b4eeccac0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.302+0000 7f30e9c8bb80 1 --2- 172.21.15.92:0/1113647145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b4ef13a90 0x557b4ef13e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.302+0000 7f30e9c8bb80 1 --2- 172.21.15.92:0/1113647145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b4eecf9a0 0x557b4eecc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.302+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/1113647145 >> 172.21.15.92:0/1113647145 conn(0x557b4eece070 msgr2=0x557b4eece440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.302+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/1113647145 shutdown_connections 2023-10-06T10:40:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.302+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/1113647145 wait complete. 2023-10-06T10:40:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.303+0000 7f30e9c8bb80 1 -- start start 2023-10-06T10:40:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.303+0000 7f30e9c8bb80 1 Processor -- start 2023-10-06T10:40:50.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.304+0000 7f30e9c8bb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b4eecdbd0 0x557b4eeed8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.304+0000 7f30e9c8bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecf9a0 0x557b4ef108d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.304+0000 7f30e9c8bb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b4ef13a90 0x557b4ef0ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.304+0000 7f30e9c8bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b4ee1d060 con 0x557b4eecf9a0 2023-10-06T10:40:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.304+0000 7f30e9c8bb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b4edc8890 con 0x557b4eecdbd0 2023-10-06T10:40:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.304+0000 7f30e9c8bb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b4ede3ea0 con 0x557b4ef13a90 2023-10-06T10:40:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.304+0000 7f30e9c8bb80 10 client.0 _ll_register_callbacks cb 0x557b4ecb9f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:40:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.304+0000 7f30d1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecf9a0 0x557b4ef108d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.304+0000 7f30d1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecf9a0 0x557b4ef108d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34988/0 (socket says 172.21.15.92:34988) 2023-10-06T10:40:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.304+0000 7f30d1d9b700 1 -- 172.21.15.92:0/2739484667 learned_addr learned my addr 172.21.15.92:0/2739484667 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.304+0000 7f30dad9d700 1 --2- 172.21.15.92:0/2739484667 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b4ef13a90 0x557b4ef0ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.304+0000 7f30da59c700 1 --2- 172.21.15.92:0/2739484667 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b4eecdbd0 0x557b4eeed8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.304+0000 7f30d1d9b700 1 -- 172.21.15.92:0/2739484667 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b4ef13a90 msgr2=0x557b4ef0ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.304+0000 7f30d1d9b700 1 --2- 172.21.15.92:0/2739484667 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b4ef13a90 0x557b4ef0ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.304+0000 7f30d1d9b700 1 -- 172.21.15.92:0/2739484667 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b4eecdbd0 msgr2=0x557b4eeed8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.304+0000 7f30d1d9b700 1 --2- 172.21.15.92:0/2739484667 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b4eecdbd0 0x557b4eeed8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.304+0000 7f30d1d9b700 1 -- 172.21.15.92:0/2739484667 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30dc0c2840 con 0x557b4eecf9a0 2023-10-06T10:40:50.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.305+0000 7f30d1d9b700 1 --2- 172.21.15.92:0/2739484667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecf9a0 0x557b4ef108d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f30c40077f0 tx=0x7f30c4007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc50d5210c8888b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.305+0000 7f30b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:40:50.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.305+0000 7f30b8ff9700 1 -- 172.21.15.92:0/2739484667 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c4001d70 con 0x557b4eecf9a0 2023-10-06T10:40:50.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.305+0000 7f30b8ff9700 1 -- 172.21.15.92:0/2739484667 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f30c4015db0 con 0x557b4eecf9a0 2023-10-06T10:40:50.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.305+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/2739484667 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b4eec1690 con 0x557b4eecf9a0 2023-10-06T10:40:50.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.305+0000 7f30b8ff9700 1 -- 172.21.15.92:0/2739484667 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30c4005ab0 con 0x557b4eecf9a0 2023-10-06T10:40:50.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.305+0000 7f30e9c8bb80 10 client.5946 fetch_fsmap learned FSMap version 37 2023-10-06T10:40:50.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.305+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/2739484667 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x557b4ec59fe0 con 0x557b4eecf9a0 2023-10-06T10:40:50.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.305+0000 7f30b8ff9700 1 -- 172.21.15.92:0/2739484667 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557b4eec1690 con 0x557b4eecf9a0 2023-10-06T10:40:50.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.305+0000 7f30b8ff9700 1 -- 172.21.15.92:0/2739484667 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f30c4012070 con 0x557b4eecf9a0 2023-10-06T10:40:50.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.305+0000 7f30e9c8bb80 10 client.5946 fetch_fsmap finished waiting for FSMap version 37 2023-10-06T10:40:50.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.305+0000 7f30e9c8bb80 10 client.5946 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T10:40:50.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.305+0000 7f30e9c8bb80 20 client.5946 populate_metadata read hostname 'smithi092' 2023-10-06T10:40:50.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.305+0000 7f30e9c8bb80 1 --2- 172.21.15.92:0/2739484667 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x557b4ef16e50 0x557b4ef17220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.305+0000 7f30e9c8bb80 4 client.5946 mds_command: new command op to 5036 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:40:50.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.305+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/2739484667 --> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b4ee9e5c0 con 0x557b4ef16e50 2023-10-06T10:40:50.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.305+0000 7f30da59c700 1 --2- 172.21.15.92:0/2739484667 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x557b4ef16e50 0x557b4ef17220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.306+0000 7f30b8ff9700 10 client.5946 ms_handle_connect on v2:172.21.15.124:6833/2986058955 2023-10-06T10:40:50.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.306+0000 7f30da59c700 1 --2- 172.21.15.92:0/2739484667 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x557b4ef16e50 0x557b4ef17220 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=80e41ba8fc1987f5 server_cookie=6d3e375bea51cce5 in_seq=0 out_seq=0 2023-10-06T10:40:50.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.306+0000 7f30b8ff9700 1 -- 172.21.15.92:0/2739484667 <== mds.2 v2:172.21.15.124:6833/2986058955 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x557b4ee9e5c0 con 0x557b4ef16e50 2023-10-06T10:40:50.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.306+0000 7f30b8ff9700 10 client.5946 handle_command_reply: tid=0 2023-10-06T10:40:50.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.306+0000 7f30e9c8bb80 1 client.5946 shutdown 2023-10-06T10:40:50.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.306+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/2739484667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecf9a0 msgr2=0x557b4ef108d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.306+0000 7f30e9c8bb80 1 --2- 172.21.15.92:0/2739484667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecf9a0 0x557b4ef108d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f30c40077f0 tx=0x7f30c4007f50 comp rx=0 tx=0).stop 2023-10-06T10:40:50.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.307+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/2739484667 shutdown_connections 2023-10-06T10:40:50.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.307+0000 7f30e9c8bb80 1 --2- 172.21.15.92:0/2739484667 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x557b4ef16e50 0x557b4ef17220 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.307+0000 7f30e9c8bb80 1 --2- 172.21.15.92:0/2739484667 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b4ef13a90 0x557b4ef0ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.307+0000 7f30b8ff9700 0 client.5946 ms_handle_reset on v2:172.21.15.124:6833/2986058955 2023-10-06T10:40:50.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.307+0000 7f30e9c8bb80 1 --2- 172.21.15.92:0/2739484667 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b4eecdbd0 0x557b4eeed8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.307+0000 7f30e9c8bb80 1 --2- 172.21.15.92:0/2739484667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecf9a0 0x557b4ef108d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.307+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/2739484667 >> 172.21.15.92:0/2739484667 conn(0x557b4eece070 msgr2=0x557b4ef12e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:50.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.307+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/2739484667 shutdown_connections 2023-10-06T10:40:50.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.307+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/2739484667 wait complete. 2023-10-06T10:40:50.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.307+0000 7f30e9c8bb80 20 client.5946 trim_cache size 0 max 16384 2023-10-06T10:40:50.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.323+0000 7f30e9c8bb80 1 Processor -- start 2023-10-06T10:40:50.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.323+0000 7f30e9c8bb80 1 -- start start 2023-10-06T10:40:50.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.324+0000 7f30e9c8bb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b4eecdbd0 0x557b4ef25dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.324+0000 7f30e9c8bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecf9a0 0x557b4ef13e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.324+0000 7f30e9c8bb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b4ef13a90 0x557b4ef114a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.324+0000 7f30e9c8bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b4ec9fe30 con 0x557b4eecf9a0 2023-10-06T10:40:50.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.324+0000 7f30e9c8bb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b4eef38c0 con 0x557b4eecdbd0 2023-10-06T10:40:50.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.324+0000 7f30e9c8bb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b4ef26d50 con 0x557b4ef13a90 2023-10-06T10:40:50.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.324+0000 7f30d1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecf9a0 0x557b4ef13e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.324+0000 7f30d1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecf9a0 0x557b4ef13e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34990/0 (socket says 172.21.15.92:34990) 2023-10-06T10:40:50.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.324+0000 7f30d1d9b700 1 -- 172.21.15.92:0/884861484 learned_addr learned my addr 172.21.15.92:0/884861484 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:50.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.324+0000 7f30d1d9b700 1 -- 172.21.15.92:0/884861484 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b4ef13a90 msgr2=0x557b4ef114a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.324+0000 7f30d1d9b700 1 --2- 172.21.15.92:0/884861484 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b4ef13a90 0x557b4ef114a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.324+0000 7f30d1d9b700 1 -- 172.21.15.92:0/884861484 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b4eecdbd0 msgr2=0x557b4ef25dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.324+0000 7f30d1d9b700 1 --2- 172.21.15.92:0/884861484 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b4eecdbd0 0x557b4ef25dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.324+0000 7f30d1d9b700 1 -- 172.21.15.92:0/884861484 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b4eecb970 con 0x557b4eecf9a0 2023-10-06T10:40:50.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.325+0000 7f30d1d9b700 1 --2- 172.21.15.92:0/884861484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecf9a0 0x557b4ef13e60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f30c4006a80 tx=0x7f30c4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1168260b53cdad3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.325+0000 7f30bbfff700 1 -- 172.21.15.92:0/884861484 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c4001880 con 0x557b4eecf9a0 2023-10-06T10:40:50.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.325+0000 7f30bbfff700 1 -- 172.21.15.92:0/884861484 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c4005ab0 con 0x557b4eecf9a0 2023-10-06T10:40:50.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.325+0000 7f30bbfff700 1 -- 172.21.15.92:0/884861484 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c401fce0 con 0x557b4eecf9a0 2023-10-06T10:40:50.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.325+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/884861484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecf9a0 msgr2=0x557b4ef13e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.325+0000 7f30e9c8bb80 1 --2- 172.21.15.92:0/884861484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecf9a0 0x557b4ef13e60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f30c4006a80 tx=0x7f30c4018000 comp rx=0 tx=0).stop 2023-10-06T10:40:50.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.325+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/884861484 shutdown_connections 2023-10-06T10:40:50.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.325+0000 7f30e9c8bb80 1 --2- 172.21.15.92:0/884861484 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b4ef13a90 0x557b4ef114a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.325+0000 7f30e9c8bb80 1 --2- 172.21.15.92:0/884861484 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b4eecdbd0 0x557b4ef25dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.325+0000 7f30e9c8bb80 1 --2- 172.21.15.92:0/884861484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecf9a0 0x557b4ef13e60 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.325+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/884861484 >> 172.21.15.92:0/884861484 conn(0x557b4eece070 msgr2=0x557b4ef12dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:50.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.325+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/884861484 shutdown_connections 2023-10-06T10:40:50.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.325+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/884861484 wait complete. 2023-10-06T10:40:50.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.326+0000 7f30e9c8bb80 1 -- start start 2023-10-06T10:40:50.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.326+0000 7f30e9c8bb80 1 Processor -- start 2023-10-06T10:40:50.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30e9c8bb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b4eecdbd0 0x557b4ef78480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30e9c8bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecf9a0 0x557b4ef24af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30e9c8bb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b4ef13a90 0x557b4eec9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30e9c8bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b4ef253d0 con 0x557b4eecf9a0 2023-10-06T10:40:50.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30e9c8bb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b4ef125e0 con 0x557b4eecdbd0 2023-10-06T10:40:50.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30e9c8bb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b4eec9350 con 0x557b4ef13a90 2023-10-06T10:40:50.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30e9c8bb80 10 client.0 _ll_register_callbacks cb 0x557b4ecb9f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:40:50.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30d1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecf9a0 0x557b4ef24af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30d1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecf9a0 0x557b4ef24af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35006/0 (socket says 172.21.15.92:35006) 2023-10-06T10:40:50.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30d1d9b700 1 -- 172.21.15.92:0/4065027753 learned_addr learned my addr 172.21.15.92:0/4065027753 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:50.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30dad9d700 1 --2- 172.21.15.92:0/4065027753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b4ef13a90 0x557b4eec9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30da59c700 1 --2- 172.21.15.92:0/4065027753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b4eecdbd0 0x557b4ef78480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30d1d9b700 1 -- 172.21.15.92:0/4065027753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b4ef13a90 msgr2=0x557b4eec9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30d1d9b700 1 --2- 172.21.15.92:0/4065027753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b4ef13a90 0x557b4eec9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30d1d9b700 1 -- 172.21.15.92:0/4065027753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b4eecdbd0 msgr2=0x557b4ef78480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30d1d9b700 1 --2- 172.21.15.92:0/4065027753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b4eecdbd0 0x557b4ef78480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30d1d9b700 1 -- 172.21.15.92:0/4065027753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b4ec59fe0 con 0x557b4eecf9a0 2023-10-06T10:40:50.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30da59c700 1 --2- 172.21.15.92:0/4065027753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b4eecdbd0 0x557b4ef78480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:50.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30d1d9b700 1 --2- 172.21.15.92:0/4065027753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecf9a0 0x557b4ef24af0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f30c4018040 tx=0x7f30c4027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fce91548615fadb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:40:50.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30b8ff9700 1 -- 172.21.15.92:0/4065027753 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c4001880 con 0x557b4eecf9a0 2023-10-06T10:40:50.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30b8ff9700 1 -- 172.21.15.92:0/4065027753 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f30c401fd40 con 0x557b4eecf9a0 2023-10-06T10:40:50.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.327+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/4065027753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30d405b050 con 0x557b4eecf9a0 2023-10-06T10:40:50.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.328+0000 7f30b8ff9700 1 -- 172.21.15.92:0/4065027753 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30c4004a40 con 0x557b4eecf9a0 2023-10-06T10:40:50.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.328+0000 7f30e9c8bb80 10 client.5952 fetch_fsmap learned FSMap version 37 2023-10-06T10:40:50.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.328+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/4065027753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x557b4edeb780 con 0x557b4eecf9a0 2023-10-06T10:40:50.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.328+0000 7f30b8ff9700 1 -- 172.21.15.92:0/4065027753 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d405b050 con 0x557b4eecf9a0 2023-10-06T10:40:50.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.328+0000 7f30b8ff9700 1 -- 172.21.15.92:0/4065027753 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f30c4027070 con 0x557b4eecf9a0 2023-10-06T10:40:50.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.328+0000 7f30e9c8bb80 10 client.5952 fetch_fsmap finished waiting for FSMap version 37 2023-10-06T10:40:50.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.328+0000 7f30e9c8bb80 10 client.5952 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T10:40:50.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.328+0000 7f30e9c8bb80 20 client.5952 populate_metadata read hostname 'smithi092' 2023-10-06T10:40:50.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.328+0000 7f30e9c8bb80 1 --2- 172.21.15.92:0/4065027753 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x557b4eeca070 0x557b4eef3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.328+0000 7f30e9c8bb80 4 client.5952 mds_command: new command op to 5036 tid=0[{"prefix": "status"}] 2023-10-06T10:40:50.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.328+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/4065027753 --> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557b4edbfe00 con 0x557b4eeca070 2023-10-06T10:40:50.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.328+0000 7f30da59c700 1 --2- 172.21.15.92:0/4065027753 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x557b4eeca070 0x557b4eef3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.328+0000 7f30b8ff9700 10 client.5952 ms_handle_connect on v2:172.21.15.124:6833/2986058955 2023-10-06T10:40:50.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.328+0000 7f30da59c700 1 --2- 172.21.15.92:0/4065027753 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x557b4eeca070 0x557b4eef3be0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=84354022f738c646 server_cookie=fe095912374c0d90 in_seq=0 out_seq=0 2023-10-06T10:40:50.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.329+0000 7f30b8ff9700 1 -- 172.21.15.92:0/4065027753 <== mds.2 v2:172.21.15.124:6833/2986058955 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x557b4edbfe00 con 0x557b4eeca070 2023-10-06T10:40:50.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.329+0000 7f30b8ff9700 10 client.5952 handle_command_reply: tid=0 2023-10-06T10:40:50.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.329+0000 7f30e9c8bb80 1 client.5952 shutdown 2023-10-06T10:40:50.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.329+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/4065027753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecf9a0 msgr2=0x557b4ef24af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.329+0000 7f30e9c8bb80 1 --2- 172.21.15.92:0/4065027753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecf9a0 0x557b4ef24af0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f30c4018040 tx=0x7f30c4027000 comp rx=0 tx=0).stop 2023-10-06T10:40:50.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.329+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/4065027753 shutdown_connections 2023-10-06T10:40:50.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.329+0000 7f30e9c8bb80 1 --2- 172.21.15.92:0/4065027753 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x557b4eeca070 0x557b4eef3be0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.329+0000 7f30e9c8bb80 1 --2- 172.21.15.92:0/4065027753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b4ef13a90 0x557b4eec9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.329+0000 7f30e9c8bb80 1 --2- 172.21.15.92:0/4065027753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b4eecdbd0 0x557b4ef78480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.329+0000 7f30e9c8bb80 1 --2- 172.21.15.92:0/4065027753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b4eecf9a0 0x557b4ef24af0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.329+0000 7f30b8ff9700 0 client.5952 ms_handle_reset on v2:172.21.15.124:6833/2986058955 2023-10-06T10:40:50.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.329+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/4065027753 >> 172.21.15.92:0/4065027753 conn(0x557b4eece070 msgr2=0x557b4ef09260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:50.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.329+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/4065027753 shutdown_connections 2023-10-06T10:40:50.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.329+0000 7f30e9c8bb80 1 -- 172.21.15.92:0/4065027753 wait complete. 2023-10-06T10:40:50.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.329+0000 7f30e9c8bb80 20 client.5952 trim_cache size 0 max 16384 2023-10-06T10:40:50.338 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:40:50.338 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:40:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 2, 2023-10-06T10:40:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 5036, 2023-10-06T10:40:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:40:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:40:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:40:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 1.705252937, 2023-10-06T10:40:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 37, 2023-10-06T10:40:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 52, 2023-10-06T10:40:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 52, 2023-10-06T10:40:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 20.903774998999999 2023-10-06T10:40:50.339 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:40:50.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.329+0000 7f30e0993700 1 -- 172.21.15.92:0/3206637812 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f30bc066f30 msgr2=0x7f30bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.329+0000 7f30e0993700 1 --2- 172.21.15.92:0/3206637812 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f30bc066f30 0x7f30bc0693e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0093f0 tx=0x7f30cc017000 comp rx=0 tx=0).stop 2023-10-06T10:40:50.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.329+0000 7f30e0993700 1 -- 172.21.15.92:0/3206637812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30dc0ce8a0 msgr2=0x7f30dc0b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.329+0000 7f30e0993700 1 --2- 172.21.15.92:0/3206637812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30dc0ce8a0 0x7f30dc0b15c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f30d400ef90 tx=0x7f30d400b040 comp rx=0 tx=0).stop 2023-10-06T10:40:50.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.329+0000 7f30e0993700 1 -- 172.21.15.92:0/3206637812 shutdown_connections 2023-10-06T10:40:50.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.329+0000 7f30e0993700 1 --2- 172.21.15.92:0/3206637812 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f30bc066f30 0x7f30bc0693e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.330+0000 7f30e0993700 1 --2- 172.21.15.92:0/3206637812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30dc0ce8a0 0x7f30dc0b15c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.330+0000 7f30e0993700 1 --2- 172.21.15.92:0/3206637812 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30dc0adb60 0x7f30dc0b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.330+0000 7f30e0993700 1 --2- 172.21.15.92:0/3206637812 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30dc0ca370 0x7f30dc0b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.330+0000 7f30e0993700 1 -- 172.21.15.92:0/3206637812 >> 172.21.15.92:0/3206637812 conn(0x7f30dc0707d0 msgr2=0x7f30dc158840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:50.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.330+0000 7f30e0993700 1 -- 172.21.15.92:0/3206637812 shutdown_connections 2023-10-06T10:40:50.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.330+0000 7f30e0993700 1 -- 172.21.15.92:0/3206637812 wait complete. 2023-10-06T10:40:50.343 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T10:40:50.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.456+0000 7f782711c700 1 Processor -- start 2023-10-06T10:40:50.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.456+0000 7f782711c700 1 -- start start 2023-10-06T10:40:50.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.457+0000 7f782711c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78200cbc30 0x7f78200cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.457+0000 7f782711c700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f78200b2100 0x7f78200b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.457+0000 7f782711c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78200cb7a0 con 0x7f7820159440 2023-10-06T10:40:50.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.457+0000 7f782711c700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78200b0860 con 0x7f78200b2100 2023-10-06T10:40:50.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.457+0000 7f782711c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78200cb150 con 0x7f78200cbc30 2023-10-06T10:40:50.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.457+0000 7f782611a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7820159440 0x7f78200b3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36606/0 (socket says 172.21.15.92:36606) 2023-10-06T10:40:50.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.457+0000 7f782611a700 1 -- 172.21.15.92:0/1818752957 learned_addr learned my addr 172.21.15.92:0/1818752957 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:50.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.457+0000 7f7825919700 1 --2- 172.21.15.92:0/1818752957 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78200cbc30 0x7f78200cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.457+0000 7f782691b700 1 --2- 172.21.15.92:0/1818752957 >> v2:172.21.15.124:3300/0 conn(0x7f78200b2100 0x7f78200b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.457+0000 7f7825118700 1 -- 172.21.15.92:0/1818752957 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102040208 0 0) 0x7f78200cb7a0 con 0x7f7820159440 2023-10-06T10:40:50.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.457+0000 7f7825118700 1 -- 172.21.15.92:0/1818752957 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f780c003570 con 0x7f7820159440 2023-10-06T10:40:50.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f7825118700 1 -- 172.21.15.92:0/1818752957 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2827339777 0 0) 0x7f780c003570 con 0x7f7820159440 2023-10-06T10:40:50.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f7825118700 1 -- 172.21.15.92:0/1818752957 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78200cbc30 msgr2=0x7f78200cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f7825118700 1 --2- 172.21.15.92:0/1818752957 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78200cbc30 0x7f78200cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f7825118700 1 -- 172.21.15.92:0/1818752957 >> v2:172.21.15.124:3300/0 conn(0x7f78200b2100 msgr2=0x7f78200b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f7825118700 1 --2- 172.21.15.92:0/1818752957 >> v2:172.21.15.124:3300/0 conn(0x7f78200b2100 0x7f78200b2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f7825118700 1 -- 172.21.15.92:0/1818752957 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78200cb430 con 0x7f7820159440 2023-10-06T10:40:50.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f7825118700 1 -- 172.21.15.92:0/1818752957 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f781c001470 con 0x7f7820159440 2023-10-06T10:40:50.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f7825118700 1 -- 172.21.15.92:0/1818752957 >> v1:172.21.15.92:6789/0 conn(0x7f7820159440 legacy=0x7f78200b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f7825118700 1 --2- 172.21.15.92:0/1818752957 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f780c003a60 0x7f780c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f7825118700 1 --2- 172.21.15.92:0/1818752957 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78200cbc30 0x7f780c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f7825118700 1 --2- 172.21.15.92:0/1818752957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f780c004d50 0x7f780c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f7825118700 1 -- 172.21.15.92:0/1818752957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78200b0860 con 0x7f780c004d50 2023-10-06T10:40:50.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f7825118700 1 -- 172.21.15.92:0/1818752957 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78200cb150 con 0x7f780c003a60 2023-10-06T10:40:50.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f7825118700 1 -- 172.21.15.92:0/1818752957 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780c00ba60 con 0x7f78200cbc30 2023-10-06T10:40:50.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f782691b700 1 --2- 172.21.15.92:0/1818752957 >> v2:172.21.15.124:3300/0 conn(0x7f78200b2100 0x7f78200b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:40:50.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f782691b700 1 --2- 172.21.15.92:0/1818752957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f780c004d50 0x7f780c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f7825919700 1 --2- 172.21.15.92:0/1818752957 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78200cbc30 0x7f780c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f782611a700 1 --2- 172.21.15.92:0/1818752957 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f780c003a60 0x7f780c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f782691b700 1 -- 172.21.15.92:0/1818752957 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78200cbc30 msgr2=0x7f780c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f782691b700 1 --2- 172.21.15.92:0/1818752957 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78200cbc30 0x7f780c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f782691b700 1 -- 172.21.15.92:0/1818752957 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f780c003a60 msgr2=0x7f780c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f782691b700 1 --2- 172.21.15.92:0/1818752957 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f780c003a60 0x7f780c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f782691b700 1 -- 172.21.15.92:0/1818752957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780c003570 con 0x7f780c004d50 2023-10-06T10:40:50.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f782691b700 1 --2- 172.21.15.92:0/1818752957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f780c004d50 0x7f780c0051c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f781800e3c0 tx=0x7f781800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7c720dde577927 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f7825118700 1 -- 172.21.15.92:0/1818752957 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781800ec10 con 0x7f780c004d50 2023-10-06T10:40:50.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.458+0000 7f7825118700 1 -- 172.21.15.92:0/1818752957 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781800f040 con 0x7f780c004d50 2023-10-06T10:40:50.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.459+0000 7f7825118700 1 -- 172.21.15.92:0/1818752957 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7818014480 con 0x7f780c004d50 2023-10-06T10:40:50.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.459+0000 7f782711c700 1 -- 172.21.15.92:0/1818752957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f780c004d50 msgr2=0x7f780c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.459+0000 7f782711c700 1 --2- 172.21.15.92:0/1818752957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f780c004d50 0x7f780c0051c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f781800e3c0 tx=0x7f781800b040 comp rx=0 tx=0).stop 2023-10-06T10:40:50.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.459+0000 7f782691b700 1 -- 172.21.15.92:0/1818752957 reap_dead start 2023-10-06T10:40:50.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.459+0000 7f782711c700 1 -- 172.21.15.92:0/1818752957 shutdown_connections 2023-10-06T10:40:50.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.459+0000 7f782711c700 1 -- 172.21.15.92:0/1818752957 >> 172.21.15.92:0/1818752957 conn(0x7f782006ece0 msgr2=0x7f782006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:50.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.459+0000 7f782711c700 1 -- 172.21.15.92:0/1818752957 shutdown_connections 2023-10-06T10:40:50.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.459+0000 7f782711c700 1 -- 172.21.15.92:0/1818752957 wait complete. 2023-10-06T10:40:50.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.460+0000 7f782711c700 1 Processor -- start 2023-10-06T10:40:50.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.460+0000 7f782711c700 1 -- start start 2023-10-06T10:40:50.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.461+0000 7f782711c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f780c004d50 0x7f782016d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.461+0000 7f782711c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78200d0830 0x7f7820169d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.461+0000 7f782711c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7820157ff0 0x7f7820158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.461+0000 7f782711c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782016d040 con 0x7f780c004d50 2023-10-06T10:40:50.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.461+0000 7f782711c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78201589b0 con 0x7f78200d0830 2023-10-06T10:40:50.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.461+0000 7f782711c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78200c9fb0 con 0x7f7820157ff0 2023-10-06T10:40:50.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.461+0000 7f782611a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f780c004d50 0x7f782016d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.461+0000 7f782611a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f780c004d50 0x7f782016d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35016/0 (socket says 172.21.15.92:35016) 2023-10-06T10:40:50.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.461+0000 7f782611a700 1 -- 172.21.15.92:0/1718713740 learned_addr learned my addr 172.21.15.92:0/1718713740 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:50.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.461+0000 7f782691b700 1 --2- 172.21.15.92:0/1718713740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7820157ff0 0x7f7820158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.461+0000 7f7825919700 1 --2- 172.21.15.92:0/1718713740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78200d0830 0x7f7820169d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.461+0000 7f782611a700 1 -- 172.21.15.92:0/1718713740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7820157ff0 msgr2=0x7f7820158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.461+0000 7f782611a700 1 --2- 172.21.15.92:0/1718713740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7820157ff0 0x7f7820158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.461+0000 7f782611a700 1 -- 172.21.15.92:0/1718713740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78200d0830 msgr2=0x7f7820169d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.461+0000 7f782611a700 1 --2- 172.21.15.92:0/1718713740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78200d0830 0x7f7820169d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.461+0000 7f782611a700 1 -- 172.21.15.92:0/1718713740 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78200ca290 con 0x7f780c004d50 2023-10-06T10:40:50.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.461+0000 7f782611a700 1 --2- 172.21.15.92:0/1718713740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f780c004d50 0x7f782016d460 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f781c007390 tx=0x7f781c001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.462+0000 7f78167fc700 1 -- 172.21.15.92:0/1718713740 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c00fa70 con 0x7f780c004d50 2023-10-06T10:40:50.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.462+0000 7f78167fc700 1 -- 172.21.15.92:0/1718713740 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781c00fbf0 con 0x7f780c004d50 2023-10-06T10:40:50.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.462+0000 7f782711c700 1 -- 172.21.15.92:0/1718713740 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f782016c980 con 0x7f780c004d50 2023-10-06T10:40:50.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.462+0000 7f78167fc700 1 -- 172.21.15.92:0/1718713740 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c0133b0 con 0x7f780c004d50 2023-10-06T10:40:50.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.462+0000 7f782711c700 1 -- 172.21.15.92:0/1718713740 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7820152ea0 con 0x7f780c004d50 2023-10-06T10:40:50.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.463+0000 7f78167fc700 1 -- 172.21.15.92:0/1718713740 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f781c013760 con 0x7f780c004d50 2023-10-06T10:40:50.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.463+0000 7f78167fc700 1 --2- 172.21.15.92:0/1718713740 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7804066f30 0x7f78040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.463+0000 7f78167fc700 1 -- 172.21.15.92:0/1718713740 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f781c0847d0 con 0x7f780c004d50 2023-10-06T10:40:50.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.463+0000 7f7825919700 1 --2- 172.21.15.92:0/1718713740 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7804066f30 0x7f78040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.464+0000 7f7825919700 1 --2- 172.21.15.92:0/1718713740 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7804066f30 0x7f78040693e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7810006f00 tx=0x7f7810008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.465+0000 7f7831676b80 1 Processor -- start 2023-10-06T10:40:50.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.465+0000 7f7831676b80 1 -- start start 2023-10-06T10:40:50.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.466+0000 7f7831676b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5587fbf8e950 0x5587fbf8ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.466+0000 7f7831676b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5587fbfd8bc0 0x5587fbfb7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.466+0000 7f7831676b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbfb7e60 0x5587fbf8a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.466+0000 7f7831676b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fbf0ec80 con 0x5587fbfb7e60 2023-10-06T10:40:50.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.466+0000 7f7831676b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fbee5a70 con 0x5587fbf8e950 2023-10-06T10:40:50.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.466+0000 7f7831676b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5587fbee3270 con 0x5587fbfd8bc0 2023-10-06T10:40:50.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.466+0000 7f782691b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbfb7e60 0x5587fbf8a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.466+0000 7f782691b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbfb7e60 0x5587fbf8a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35020/0 (socket says 172.21.15.92:35020) 2023-10-06T10:40:50.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.466+0000 7f782691b700 1 -- 172.21.15.92:0/4238554040 learned_addr learned my addr 172.21.15.92:0/4238554040 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:50.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.466+0000 7f782691b700 1 -- 172.21.15.92:0/4238554040 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5587fbfd8bc0 msgr2=0x5587fbfb7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.466+0000 7f782691b700 1 --2- 172.21.15.92:0/4238554040 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5587fbfd8bc0 0x5587fbfb7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.466+0000 7f782691b700 1 -- 172.21.15.92:0/4238554040 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5587fbf8e950 msgr2=0x5587fbf8ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.466+0000 7f782691b700 1 --2- 172.21.15.92:0/4238554040 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5587fbf8e950 0x5587fbf8ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.466+0000 7f782691b700 1 -- 172.21.15.92:0/4238554040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587fbd79270 con 0x5587fbfb7e60 2023-10-06T10:40:50.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.467+0000 7f782691b700 1 --2- 172.21.15.92:0/4238554040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbfb7e60 0x5587fbf8a990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f780c004730 tx=0x7f7818019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d13c94307b7e308 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.467+0000 7f78037fe700 1 -- 172.21.15.92:0/4238554040 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7818014480 con 0x5587fbfb7e60 2023-10-06T10:40:50.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.467+0000 7f78037fe700 1 -- 172.21.15.92:0/4238554040 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7818009c50 con 0x5587fbfb7e60 2023-10-06T10:40:50.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.467+0000 7f78037fe700 1 -- 172.21.15.92:0/4238554040 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781800ee20 con 0x5587fbfb7e60 2023-10-06T10:40:50.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.467+0000 7f7831676b80 1 -- 172.21.15.92:0/4238554040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbfb7e60 msgr2=0x5587fbf8a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.467+0000 7f7831676b80 1 --2- 172.21.15.92:0/4238554040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbfb7e60 0x5587fbf8a990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f780c004730 tx=0x7f7818019000 comp rx=0 tx=0).stop 2023-10-06T10:40:50.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.467+0000 7f7831676b80 1 -- 172.21.15.92:0/4238554040 shutdown_connections 2023-10-06T10:40:50.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.467+0000 7f7831676b80 1 --2- 172.21.15.92:0/4238554040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbfb7e60 0x5587fbf8a990 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.467+0000 7f7831676b80 1 --2- 172.21.15.92:0/4238554040 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5587fbf8e950 0x5587fbf8ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.467+0000 7f7831676b80 1 --2- 172.21.15.92:0/4238554040 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5587fbfd8bc0 0x5587fbfb7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.467+0000 7f7831676b80 1 -- 172.21.15.92:0/4238554040 >> 172.21.15.92:0/4238554040 conn(0x5587fbf90270 msgr2=0x5587fbf90640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:50.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.467+0000 7f7831676b80 1 -- 172.21.15.92:0/4238554040 shutdown_connections 2023-10-06T10:40:50.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.467+0000 7f7831676b80 1 -- 172.21.15.92:0/4238554040 wait complete. 2023-10-06T10:40:50.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.468+0000 7f7831676b80 1 -- start start 2023-10-06T10:40:50.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.468+0000 7f7831676b80 1 Processor -- start 2023-10-06T10:40:50.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.469+0000 7f7831676b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbf8e950 0x5587fbfb4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.469+0000 7f7831676b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5587fbfb7e60 0x5587fbfb6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.469+0000 7f7831676b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5587fbfd8bc0 0x5587fbf851a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.469+0000 7f7831676b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fbedf060 con 0x5587fbf8e950 2023-10-06T10:40:50.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.469+0000 7f7831676b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fbe8a890 con 0x5587fbfb7e60 2023-10-06T10:40:50.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.469+0000 7f7831676b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5587fbea5ea0 con 0x5587fbfd8bc0 2023-10-06T10:40:50.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.469+0000 7f7831676b80 10 client.0 _ll_register_callbacks cb 0x5587fbd7bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:40:50.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.469+0000 7f782611a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbf8e950 0x5587fbfb4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.469+0000 7f782611a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbf8e950 0x5587fbfb4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35028/0 (socket says 172.21.15.92:35028) 2023-10-06T10:40:50.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.469+0000 7f782611a700 1 -- 172.21.15.92:0/3653256070 learned_addr learned my addr 172.21.15.92:0/3653256070 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:50.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.469+0000 7f7825919700 1 --2- 172.21.15.92:0/3653256070 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5587fbfb7e60 0x5587fbfb6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.469+0000 7f782691b700 1 --2- 172.21.15.92:0/3653256070 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5587fbfd8bc0 0x5587fbf851a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.469+0000 7f782611a700 1 -- 172.21.15.92:0/3653256070 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5587fbfd8bc0 msgr2=0x5587fbf851a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.469+0000 7f782611a700 1 --2- 172.21.15.92:0/3653256070 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5587fbfd8bc0 0x5587fbf851a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.469+0000 7f782611a700 1 -- 172.21.15.92:0/3653256070 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5587fbfb7e60 msgr2=0x5587fbfb6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.469+0000 7f782611a700 1 --2- 172.21.15.92:0/3653256070 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5587fbfb7e60 0x5587fbfb6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.469+0000 7f782611a700 1 -- 172.21.15.92:0/3653256070 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7818009a20 con 0x5587fbf8e950 2023-10-06T10:40:50.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.469+0000 7f782691b700 1 --2- 172.21.15.92:0/3653256070 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5587fbfd8bc0 0x5587fbf851a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:50.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.469+0000 7f7825919700 1 --2- 172.21.15.92:0/3653256070 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5587fbfb7e60 0x5587fbfb6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:50.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.470+0000 7f782611a700 1 --2- 172.21.15.92:0/3653256070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbf8e950 0x5587fbfb4230 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f781c051960 tx=0x7f781c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48c2040e5055c2d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.470+0000 7f7800ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:40:50.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.470+0000 7f7800ff9700 1 -- 172.21.15.92:0/3653256070 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c0586b0 con 0x5587fbf8e950 2023-10-06T10:40:50.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.470+0000 7f7800ff9700 1 -- 172.21.15.92:0/3653256070 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f781c05c400 con 0x5587fbf8e950 2023-10-06T10:40:50.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.470+0000 7f7831676b80 1 -- 172.21.15.92:0/3653256070 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587fbf83690 con 0x5587fbf8e950 2023-10-06T10:40:50.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.470+0000 7f7800ff9700 1 -- 172.21.15.92:0/3653256070 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f781c054b60 con 0x5587fbf8e950 2023-10-06T10:40:50.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.470+0000 7f7831676b80 10 client.5964 fetch_fsmap learned FSMap version 37 2023-10-06T10:40:50.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.470+0000 7f7831676b80 1 -- 172.21.15.92:0/3653256070 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x5587fbd1bfe0 con 0x5587fbf8e950 2023-10-06T10:40:50.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.470+0000 7f7800ff9700 1 -- 172.21.15.92:0/3653256070 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5587fbf83690 con 0x5587fbf8e950 2023-10-06T10:40:50.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.470+0000 7f7800ff9700 1 -- 172.21.15.92:0/3653256070 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f781c05bf70 con 0x5587fbf8e950 2023-10-06T10:40:50.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.470+0000 7f7831676b80 10 client.5964 fetch_fsmap finished waiting for FSMap version 37 2023-10-06T10:40:50.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.470+0000 7f7831676b80 10 client.5964 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:40:50.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.470+0000 7f7831676b80 20 client.5964 populate_metadata read hostname 'smithi092' 2023-10-06T10:40:50.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.470+0000 7f7831676b80 1 --2- 172.21.15.92:0/3653256070 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x5587fbfcea40 0x5587fbfcee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.470+0000 7f7831676b80 4 client.5964 mds_command: new command op to 5733 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:40:50.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.470+0000 7f7831676b80 1 -- 172.21.15.92:0/3653256070 --> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587fbf605c0 con 0x5587fbfcea40 2023-10-06T10:40:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.470+0000 7f7825919700 1 --2- 172.21.15.92:0/3653256070 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x5587fbfcea40 0x5587fbfcee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.470+0000 7f7825919700 1 --2- 172.21.15.92:0/3653256070 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x5587fbfcea40 0x5587fbfcee10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6468eb14b6a120aa server_cookie=18c1f0f4665d1f39 in_seq=0 out_seq=0 2023-10-06T10:40:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.470+0000 7f7800ff9700 10 client.5964 ms_handle_connect on v2:172.21.15.92:6834/831923836 2023-10-06T10:40:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.471+0000 7f7800ff9700 1 -- 172.21.15.92:0/3653256070 <== mds.1 v2:172.21.15.92:6834/831923836 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5587fbf605c0 con 0x5587fbfcea40 2023-10-06T10:40:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.471+0000 7f7800ff9700 10 client.5964 handle_command_reply: tid=0 2023-10-06T10:40:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.471+0000 7f7831676b80 1 client.5964 shutdown 2023-10-06T10:40:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.471+0000 7f7831676b80 1 -- 172.21.15.92:0/3653256070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbf8e950 msgr2=0x5587fbfb4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.471+0000 7f7831676b80 1 --2- 172.21.15.92:0/3653256070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbf8e950 0x5587fbfb4230 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f781c051960 tx=0x7f781c009040 comp rx=0 tx=0).stop 2023-10-06T10:40:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.471+0000 7f7831676b80 1 -- 172.21.15.92:0/3653256070 shutdown_connections 2023-10-06T10:40:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.471+0000 7f7831676b80 1 --2- 172.21.15.92:0/3653256070 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5587fbfd8bc0 0x5587fbf851a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.471+0000 7f7831676b80 1 --2- 172.21.15.92:0/3653256070 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x5587fbfcea40 0x5587fbfcee10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.471+0000 7f7831676b80 1 --2- 172.21.15.92:0/3653256070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbf8e950 0x5587fbfb4230 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.471+0000 7f7831676b80 1 --2- 172.21.15.92:0/3653256070 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5587fbfb7e60 0x5587fbfb6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.471+0000 7f7800ff9700 0 client.5964 ms_handle_reset on v2:172.21.15.92:6834/831923836 2023-10-06T10:40:50.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.471+0000 7f7831676b80 1 -- 172.21.15.92:0/3653256070 >> 172.21.15.92:0/3653256070 conn(0x5587fbf90270 msgr2=0x5587fbfd44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:50.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.471+0000 7f7831676b80 1 -- 172.21.15.92:0/3653256070 shutdown_connections 2023-10-06T10:40:50.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.471+0000 7f7831676b80 1 -- 172.21.15.92:0/3653256070 wait complete. 2023-10-06T10:40:50.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.471+0000 7f7831676b80 20 client.5964 trim_cache size 0 max 16384 2023-10-06T10:40:50.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.488+0000 7f7831676b80 1 Processor -- start 2023-10-06T10:40:50.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.488+0000 7f7831676b80 1 -- start start 2023-10-06T10:40:50.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.488+0000 7f7831676b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5587fbf90270 0x5587fbfee280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.488+0000 7f7831676b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5587fbf8e950 0x5587fbf8b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.489+0000 7f7831676b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbfb7e60 0x5587fc035e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.489+0000 7f7831676b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fbd61e30 con 0x5587fbfb7e60 2023-10-06T10:40:50.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.489+0000 7f7831676b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fbfeecb0 con 0x5587fbf8e950 2023-10-06T10:40:50.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.489+0000 7f7831676b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5587fbfcd570 con 0x5587fbf90270 2023-10-06T10:40:50.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.489+0000 7f782691b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbfb7e60 0x5587fc035e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.489+0000 7f782691b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbfb7e60 0x5587fc035e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35030/0 (socket says 172.21.15.92:35030) 2023-10-06T10:40:50.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.489+0000 7f782691b700 1 -- 172.21.15.92:0/1120239825 learned_addr learned my addr 172.21.15.92:0/1120239825 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:50.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.489+0000 7f782691b700 1 -- 172.21.15.92:0/1120239825 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5587fbf90270 msgr2=0x5587fbfee280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.489+0000 7f782691b700 1 --2- 172.21.15.92:0/1120239825 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5587fbf90270 0x5587fbfee280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.489+0000 7f782691b700 1 -- 172.21.15.92:0/1120239825 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5587fbf8e950 msgr2=0x5587fbf8b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.489+0000 7f782691b700 1 --2- 172.21.15.92:0/1120239825 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5587fbf8e950 0x5587fbf8b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.489+0000 7f782691b700 1 -- 172.21.15.92:0/1120239825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587fbead780 con 0x5587fbfb7e60 2023-10-06T10:40:50.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.489+0000 7f782691b700 1 --2- 172.21.15.92:0/1120239825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbfb7e60 0x5587fc035e20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7818009750 tx=0x7f7818020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1680b5305e18228 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.489+0000 7f7803fff700 1 -- 172.21.15.92:0/1120239825 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7818014480 con 0x5587fbfb7e60 2023-10-06T10:40:50.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.489+0000 7f7803fff700 1 -- 172.21.15.92:0/1120239825 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78180145e0 con 0x5587fbfb7e60 2023-10-06T10:40:50.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.489+0000 7f7803fff700 1 -- 172.21.15.92:0/1120239825 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781800ee20 con 0x5587fbfb7e60 2023-10-06T10:40:50.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.490+0000 7f7831676b80 1 -- 172.21.15.92:0/1120239825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbfb7e60 msgr2=0x5587fc035e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.490+0000 7f7831676b80 1 --2- 172.21.15.92:0/1120239825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbfb7e60 0x5587fc035e20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7818009750 tx=0x7f7818020040 comp rx=0 tx=0).stop 2023-10-06T10:40:50.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.490+0000 7f7831676b80 1 -- 172.21.15.92:0/1120239825 shutdown_connections 2023-10-06T10:40:50.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.490+0000 7f7831676b80 1 --2- 172.21.15.92:0/1120239825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbfb7e60 0x5587fc035e20 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.490+0000 7f7831676b80 1 --2- 172.21.15.92:0/1120239825 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5587fbf90270 0x5587fbfee280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.490+0000 7f7831676b80 1 --2- 172.21.15.92:0/1120239825 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5587fbf8e950 0x5587fbf8b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.490+0000 7f7831676b80 1 -- 172.21.15.92:0/1120239825 >> 172.21.15.92:0/1120239825 conn(0x5587fbf8d9b0 msgr2=0x5587fbf91cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:50.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.490+0000 7f7831676b80 1 -- 172.21.15.92:0/1120239825 shutdown_connections 2023-10-06T10:40:50.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.490+0000 7f7831676b80 1 -- 172.21.15.92:0/1120239825 wait complete. 2023-10-06T10:40:50.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.490+0000 7f7831676b80 1 -- start start 2023-10-06T10:40:50.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.490+0000 7f7831676b80 1 Processor -- start 2023-10-06T10:40:50.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.491+0000 7f7831676b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5587fbf8e950 0x5587fc035750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.491+0000 7f7831676b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbf90270 0x5587fbfee280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.491+0000 7f7831676b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5587fbfb7e60 0x5587fbfee7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.491+0000 7f7831676b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fbff57a0 con 0x5587fbf90270 2023-10-06T10:40:50.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.491+0000 7f7831676b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fbff2bb0 con 0x5587fbfb7e60 2023-10-06T10:40:50.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.491+0000 7f7831676b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5587fbff1fa0 con 0x5587fbf8e950 2023-10-06T10:40:50.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.491+0000 7f7831676b80 10 client.0 _ll_register_callbacks cb 0x5587fbd7bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:40:50.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.491+0000 7f7825919700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbf90270 0x5587fbfee280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.491+0000 7f7825919700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbf90270 0x5587fbfee280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35042/0 (socket says 172.21.15.92:35042) 2023-10-06T10:40:50.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.491+0000 7f7825919700 1 -- 172.21.15.92:0/2419797532 learned_addr learned my addr 172.21.15.92:0/2419797532 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:50.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.491+0000 7f782611a700 1 --2- 172.21.15.92:0/2419797532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5587fbf8e950 0x5587fc035750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.491+0000 7f782691b700 1 --2- 172.21.15.92:0/2419797532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5587fbfb7e60 0x5587fbfee7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f7825919700 1 -- 172.21.15.92:0/2419797532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5587fbf8e950 msgr2=0x5587fc035750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f7825919700 1 --2- 172.21.15.92:0/2419797532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5587fbf8e950 0x5587fc035750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f7825919700 1 -- 172.21.15.92:0/2419797532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5587fbfb7e60 msgr2=0x5587fbfee7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f7825919700 1 --2- 172.21.15.92:0/2419797532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5587fbfb7e60 0x5587fbfee7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f7825919700 1 -- 172.21.15.92:0/2419797532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587fbd79270 con 0x5587fbf90270 2023-10-06T10:40:50.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f782691b700 1 --2- 172.21.15.92:0/2419797532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5587fbfb7e60 0x5587fbfee7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:50.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f7825919700 1 --2- 172.21.15.92:0/2419797532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbf90270 0x5587fbfee280 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7810009760 tx=0x7f7810026040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66af93ec87747970 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f7800ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:40:50.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f7800ff9700 1 -- 172.21.15.92:0/2419797532 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7810030070 con 0x5587fbf90270 2023-10-06T10:40:50.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f7800ff9700 1 -- 172.21.15.92:0/2419797532 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f78100066b0 con 0x5587fbf90270 2023-10-06T10:40:50.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f7831676b80 1 -- 172.21.15.92:0/2419797532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7818009c50 con 0x5587fbf90270 2023-10-06T10:40:50.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f7800ff9700 1 -- 172.21.15.92:0/2419797532 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7810029040 con 0x5587fbf90270 2023-10-06T10:40:50.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f7831676b80 10 client.5970 fetch_fsmap learned FSMap version 37 2023-10-06T10:40:50.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f7831676b80 1 -- 172.21.15.92:0/2419797532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x5587fbe94470 con 0x5587fbf90270 2023-10-06T10:40:50.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f7800ff9700 1 -- 172.21.15.92:0/2419797532 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7818009c50 con 0x5587fbf90270 2023-10-06T10:40:50.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f7800ff9700 1 -- 172.21.15.92:0/2419797532 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f7810024430 con 0x5587fbf90270 2023-10-06T10:40:50.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f7831676b80 10 client.5970 fetch_fsmap finished waiting for FSMap version 37 2023-10-06T10:40:50.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f7831676b80 10 client.5970 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:40:50.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f7831676b80 20 client.5970 populate_metadata read hostname 'smithi092' 2023-10-06T10:40:50.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f7831676b80 1 --2- 172.21.15.92:0/2419797532 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x5587fbfd8bc0 0x5587fbff4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f7831676b80 4 client.5970 mds_command: new command op to 5733 tid=0[{"prefix": "status"}] 2023-10-06T10:40:50.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f7831676b80 1 -- 172.21.15.92:0/2419797532 --> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5587fbe81e00 con 0x5587fbfd8bc0 2023-10-06T10:40:50.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.492+0000 7f782611a700 1 --2- 172.21.15.92:0/2419797532 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x5587fbfd8bc0 0x5587fbff4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.493+0000 7f7800ff9700 10 client.5970 ms_handle_connect on v2:172.21.15.92:6834/831923836 2023-10-06T10:40:50.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.493+0000 7f782611a700 1 --2- 172.21.15.92:0/2419797532 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x5587fbfd8bc0 0x5587fbff4930 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6a34e2d3435d3f2d server_cookie=75d125259478ae6f in_seq=0 out_seq=0 2023-10-06T10:40:50.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.493+0000 7f7800ff9700 1 -- 172.21.15.92:0/2419797532 <== mds.1 v2:172.21.15.92:6834/831923836 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5587fbe81e00 con 0x5587fbfd8bc0 2023-10-06T10:40:50.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.493+0000 7f7800ff9700 10 client.5970 handle_command_reply: tid=0 2023-10-06T10:40:50.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.493+0000 7f7831676b80 1 client.5970 shutdown 2023-10-06T10:40:50.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.493+0000 7f7831676b80 1 -- 172.21.15.92:0/2419797532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbf90270 msgr2=0x5587fbfee280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.493+0000 7f7831676b80 1 --2- 172.21.15.92:0/2419797532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbf90270 0x5587fbfee280 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7810009760 tx=0x7f7810026040 comp rx=0 tx=0).stop 2023-10-06T10:40:50.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.493+0000 7f7831676b80 1 -- 172.21.15.92:0/2419797532 shutdown_connections 2023-10-06T10:40:50.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.493+0000 7f7831676b80 1 --2- 172.21.15.92:0/2419797532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5587fbfb7e60 0x5587fbfee7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.493+0000 7f7831676b80 1 --2- 172.21.15.92:0/2419797532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5587fbf8e950 0x5587fc035750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.493+0000 7f7831676b80 1 --2- 172.21.15.92:0/2419797532 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x5587fbfd8bc0 0x5587fbff4930 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.493+0000 7f7831676b80 1 --2- 172.21.15.92:0/2419797532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587fbf90270 0x5587fbfee280 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.493+0000 7f7800ff9700 0 client.5970 ms_handle_reset on v2:172.21.15.92:6834/831923836 2023-10-06T10:40:50.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.493+0000 7f7831676b80 1 -- 172.21.15.92:0/2419797532 >> 172.21.15.92:0/2419797532 conn(0x5587fbf8d9b0 msgr2=0x5587fbf90f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:50.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.493+0000 7f7831676b80 1 -- 172.21.15.92:0/2419797532 shutdown_connections 2023-10-06T10:40:50.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.493+0000 7f7831676b80 1 -- 172.21.15.92:0/2419797532 wait complete. 2023-10-06T10:40:50.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.493+0000 7f7831676b80 20 client.5970 trim_cache size 0 max 16384 2023-10-06T10:40:50.504 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:40:50.504 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:40:50.504 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 1, 2023-10-06T10:40:50.504 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 5733, 2023-10-06T10:40:50.504 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:40:50.505 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:40:50.505 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:40:50.505 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 2.8794613039999999, 2023-10-06T10:40:50.505 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 37, 2023-10-06T10:40:50.505 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 52, 2023-10-06T10:40:50.505 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 52, 2023-10-06T10:40:50.505 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 21.077066625 2023-10-06T10:40:50.505 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:40:50.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.494+0000 7f782711c700 1 -- 172.21.15.92:0/1718713740 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7804066f30 msgr2=0x7f78040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.494+0000 7f782711c700 1 --2- 172.21.15.92:0/1718713740 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7804066f30 0x7f78040693e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7810006f00 tx=0x7f7810008040 comp rx=0 tx=0).stop 2023-10-06T10:40:50.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.494+0000 7f782711c700 1 -- 172.21.15.92:0/1718713740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f780c004d50 msgr2=0x7f782016d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.494+0000 7f782711c700 1 --2- 172.21.15.92:0/1718713740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f780c004d50 0x7f782016d460 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f781c007390 tx=0x7f781c001e90 comp rx=0 tx=0).stop 2023-10-06T10:40:50.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.494+0000 7f782711c700 1 -- 172.21.15.92:0/1718713740 shutdown_connections 2023-10-06T10:40:50.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.494+0000 7f782711c700 1 --2- 172.21.15.92:0/1718713740 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7804066f30 0x7f78040693e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.494+0000 7f782711c700 1 --2- 172.21.15.92:0/1718713740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7820157ff0 0x7f7820158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.494+0000 7f782711c700 1 --2- 172.21.15.92:0/1718713740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f780c004d50 0x7f782016d460 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.494+0000 7f782711c700 1 --2- 172.21.15.92:0/1718713740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78200d0830 0x7f7820169d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.494+0000 7f782711c700 1 -- 172.21.15.92:0/1718713740 >> 172.21.15.92:0/1718713740 conn(0x7f782006ece0 msgr2=0x7f78200cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:50.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.494+0000 7f782711c700 1 -- 172.21.15.92:0/1718713740 shutdown_connections 2023-10-06T10:40:50.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.494+0000 7f782711c700 1 -- 172.21.15.92:0/1718713740 wait complete. 2023-10-06T10:40:50.508 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T10:40:50.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.623+0000 7f430d65a700 1 Processor -- start 2023-10-06T10:40:50.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.623+0000 7f430d65a700 1 -- start start 2023-10-06T10:40:50.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.624+0000 7f430d65a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f43080cbc30 0x7f43080cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.624+0000 7f430d65a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43080cb7a0 con 0x7f43080b2100 2023-10-06T10:40:50.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.624+0000 7f430d65a700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43080b0860 con 0x7f4308159440 2023-10-06T10:40:50.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.624+0000 7f430d65a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43080cb150 con 0x7f43080cbc30 2023-10-06T10:40:50.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.624+0000 7f43077fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f43080cbc30 0x7f43080cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.624+0000 7f430ce59700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f43080b2100 0x7f43080b2520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36616/0 (socket says 172.21.15.92:36616) 2023-10-06T10:40:50.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.624+0000 7f430ce59700 1 -- 172.21.15.92:0/4192668914 learned_addr learned my addr 172.21.15.92:0/4192668914 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:40:50.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.624+0000 7f43077fe700 1 -- 172.21.15.92:0/4192668914 >> v1:172.21.15.124:6789/0 conn(0x7f4308159440 legacy=0x7f43080b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.624+0000 7f43077fe700 1 -- 172.21.15.92:0/4192668914 >> v1:172.21.15.92:6789/0 conn(0x7f43080b2100 legacy=0x7f43080b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.624+0000 7f43077fe700 1 -- 172.21.15.92:0/4192668914 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43080cb430 con 0x7f43080cbc30 2023-10-06T10:40:50.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.624+0000 7f4306ffd700 1 -- 172.21.15.92:0/4192668914 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330346963 0 0) 0x7f43080cb7a0 con 0x7f43080b2100 2023-10-06T10:40:50.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.625+0000 7f43077fe700 1 --2- 172.21.15.92:0/4192668914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43080cbc30 0x7f43080cc080 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0098e0 tx=0x7f42fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4235baf7a235fc8e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.625+0000 7f4306ffd700 1 -- 172.21.15.92:0/4192668914 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc009e80 con 0x7f43080cbc30 2023-10-06T10:40:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.625+0000 7f4306ffd700 1 -- 172.21.15.92:0/4192668914 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42fc00b860 con 0x7f43080cbc30 2023-10-06T10:40:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.625+0000 7f4306ffd700 1 -- 172.21.15.92:0/4192668914 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc00bbe0 con 0x7f43080cbc30 2023-10-06T10:40:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.625+0000 7f430d65a700 1 -- 172.21.15.92:0/4192668914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43080cbc30 msgr2=0x7f43080cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.625+0000 7f430d65a700 1 --2- 172.21.15.92:0/4192668914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43080cbc30 0x7f43080cc080 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0098e0 tx=0x7f42fc0091a0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.626+0000 7f430d65a700 1 -- 172.21.15.92:0/4192668914 shutdown_connections 2023-10-06T10:40:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.626+0000 7f430d65a700 1 --2- 172.21.15.92:0/4192668914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43080cbc30 0x7f43080cc080 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.626+0000 7f430d65a700 1 -- 172.21.15.92:0/4192668914 >> 172.21.15.92:0/4192668914 conn(0x7f430806ece0 msgr2=0x7f430806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.626+0000 7f430d65a700 1 -- 172.21.15.92:0/4192668914 shutdown_connections 2023-10-06T10:40:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.626+0000 7f430d65a700 1 -- 172.21.15.92:0/4192668914 wait complete. 2023-10-06T10:40:50.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.627+0000 7f430d65a700 1 Processor -- start 2023-10-06T10:40:50.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.627+0000 7f430d65a700 1 -- start start 2023-10-06T10:40:50.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.628+0000 7f430d65a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43080b2100 0x7f43080d0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.628+0000 7f430d65a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43080cbc30 0x7f43081540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.628+0000 7f430d65a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4308159440 0x7f43080c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.628+0000 7f430d65a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43080d0d60 con 0x7f43080cbc30 2023-10-06T10:40:50.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.628+0000 7f430d65a700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43080c72d0 con 0x7f4308159440 2023-10-06T10:40:50.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.628+0000 7f430d65a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4308154ca0 con 0x7f43080b2100 2023-10-06T10:40:50.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.628+0000 7f43077fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43080cbc30 0x7f43081540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.628+0000 7f43077fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43080cbc30 0x7f43081540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35052/0 (socket says 172.21.15.92:35052) 2023-10-06T10:40:50.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.628+0000 7f43077fe700 1 -- 172.21.15.92:0/1187142257 learned_addr learned my addr 172.21.15.92:0/1187142257 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:50.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.628+0000 7f430ce59700 1 --2- 172.21.15.92:0/1187142257 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4308159440 0x7f43080c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.628+0000 7f4307fff700 1 --2- 172.21.15.92:0/1187142257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43080b2100 0x7f43080d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.629+0000 7f43077fe700 1 -- 172.21.15.92:0/1187142257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43080b2100 msgr2=0x7f43080d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.629+0000 7f43077fe700 1 --2- 172.21.15.92:0/1187142257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43080b2100 0x7f43080d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.629+0000 7f43077fe700 1 -- 172.21.15.92:0/1187142257 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4308159440 msgr2=0x7f43080c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.629+0000 7f43077fe700 1 --2- 172.21.15.92:0/1187142257 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4308159440 0x7f43080c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.629+0000 7f43077fe700 1 -- 172.21.15.92:0/1187142257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42fc009420 con 0x7f43080cbc30 2023-10-06T10:40:50.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.629+0000 7f43077fe700 1 --2- 172.21.15.92:0/1187142257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43080cbc30 0x7f43081540f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0091e0 tx=0x7f42fc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.631+0000 7f42effff700 1 -- 172.21.15.92:0/1187142257 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc00b9d0 con 0x7f43080cbc30 2023-10-06T10:40:50.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.631+0000 7f430d65a700 1 -- 172.21.15.92:0/1187142257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4308169f50 con 0x7f43080cbc30 2023-10-06T10:40:50.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.631+0000 7f42effff700 1 -- 172.21.15.92:0/1187142257 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42fc020070 con 0x7f43080cbc30 2023-10-06T10:40:50.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.631+0000 7f42effff700 1 -- 172.21.15.92:0/1187142257 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc01b6a0 con 0x7f43080cbc30 2023-10-06T10:40:50.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.631+0000 7f430d65a700 1 -- 172.21.15.92:0/1187142257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430816a590 con 0x7f43080cbc30 2023-10-06T10:40:50.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.632+0000 7f42effff700 1 -- 172.21.15.92:0/1187142257 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f42fc022030 con 0x7f43080cbc30 2023-10-06T10:40:50.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.632+0000 7f42effff700 1 --2- 172.21.15.92:0/1187142257 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f42f4066f30 0x7f42f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.632+0000 7f42effff700 1 -- 172.21.15.92:0/1187142257 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f42fc0866d0 con 0x7f43080cbc30 2023-10-06T10:40:50.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.633+0000 7f4307fff700 1 --2- 172.21.15.92:0/1187142257 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f42f4066f30 0x7f42f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.633+0000 7f4307fff700 1 --2- 172.21.15.92:0/1187142257 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f42f4066f30 0x7f42f40693e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f43080b4cd0 tx=0x7f42f8009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.634+0000 7f4317bb4b80 1 Processor -- start 2023-10-06T10:40:50.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.634+0000 7f4317bb4b80 1 -- start start 2023-10-06T10:40:50.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.635+0000 7f4317bb4b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bccb377950 0x55bccb377d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.635+0000 7f4317bb4b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bccb3c1bc0 0x55bccb3a0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.635+0000 7f4317bb4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb3a0e60 0x55bccb373990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.635+0000 7f4317bb4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bccb2f7c80 con 0x55bccb3a0e60 2023-10-06T10:40:50.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.635+0000 7f4317bb4b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bccb2cea70 con 0x55bccb377950 2023-10-06T10:40:50.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.635+0000 7f4317bb4b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bccb2cc270 con 0x55bccb3c1bc0 2023-10-06T10:40:50.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.635+0000 7f430ce59700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb3a0e60 0x55bccb373990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.635+0000 7f430ce59700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb3a0e60 0x55bccb373990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35054/0 (socket says 172.21.15.92:35054) 2023-10-06T10:40:50.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.635+0000 7f430ce59700 1 -- 172.21.15.92:0/2922580274 learned_addr learned my addr 172.21.15.92:0/2922580274 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:50.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.635+0000 7f43077fe700 1 --2- 172.21.15.92:0/2922580274 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bccb3c1bc0 0x55bccb3a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.635+0000 7f4307fff700 1 --2- 172.21.15.92:0/2922580274 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bccb377950 0x55bccb377d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.636+0000 7f430ce59700 1 -- 172.21.15.92:0/2922580274 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bccb3c1bc0 msgr2=0x55bccb3a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.636+0000 7f430ce59700 1 --2- 172.21.15.92:0/2922580274 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bccb3c1bc0 0x55bccb3a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.636+0000 7f430ce59700 1 -- 172.21.15.92:0/2922580274 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bccb377950 msgr2=0x55bccb377d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.636+0000 7f430ce59700 1 --2- 172.21.15.92:0/2922580274 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bccb377950 0x55bccb377d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.636+0000 7f430ce59700 1 -- 172.21.15.92:0/2922580274 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bccb162270 con 0x55bccb3a0e60 2023-10-06T10:40:50.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.636+0000 7f43077fe700 1 --2- 172.21.15.92:0/2922580274 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bccb3c1bc0 0x55bccb3a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:40:50.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.636+0000 7f430ce59700 1 --2- 172.21.15.92:0/2922580274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb3a0e60 0x55bccb373990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f430000f410 tx=0x7f4300009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb307524c7f74d6e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.636+0000 7f42ed7fa700 1 -- 172.21.15.92:0/2922580274 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43000070d0 con 0x55bccb3a0e60 2023-10-06T10:40:50.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.636+0000 7f42ed7fa700 1 -- 172.21.15.92:0/2922580274 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4300007230 con 0x55bccb3a0e60 2023-10-06T10:40:50.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.636+0000 7f42ed7fa700 1 -- 172.21.15.92:0/2922580274 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43000075f0 con 0x55bccb3a0e60 2023-10-06T10:40:50.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.636+0000 7f4317bb4b80 1 -- 172.21.15.92:0/2922580274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb3a0e60 msgr2=0x55bccb373990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.636+0000 7f4317bb4b80 1 --2- 172.21.15.92:0/2922580274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb3a0e60 0x55bccb373990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f430000f410 tx=0x7f4300009040 comp rx=0 tx=0).stop 2023-10-06T10:40:50.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.637+0000 7f4317bb4b80 1 -- 172.21.15.92:0/2922580274 shutdown_connections 2023-10-06T10:40:50.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.637+0000 7f4317bb4b80 1 --2- 172.21.15.92:0/2922580274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb3a0e60 0x55bccb373990 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.637+0000 7f4317bb4b80 1 --2- 172.21.15.92:0/2922580274 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bccb377950 0x55bccb377d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.637+0000 7f4317bb4b80 1 --2- 172.21.15.92:0/2922580274 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bccb3c1bc0 0x55bccb3a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.637+0000 7f4317bb4b80 1 -- 172.21.15.92:0/2922580274 >> 172.21.15.92:0/2922580274 conn(0x55bccb379270 msgr2=0x55bccb379640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:50.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.637+0000 7f4317bb4b80 1 -- 172.21.15.92:0/2922580274 shutdown_connections 2023-10-06T10:40:50.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.637+0000 7f4317bb4b80 1 -- 172.21.15.92:0/2922580274 wait complete. 2023-10-06T10:40:50.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.637+0000 7f4317bb4b80 1 -- start start 2023-10-06T10:40:50.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.637+0000 7f4317bb4b80 1 Processor -- start 2023-10-06T10:40:50.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.638+0000 7f4317bb4b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bccb377950 0x55bccb3b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.638+0000 7f4317bb4b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bccb3a0e60 0x55bccb3ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.638+0000 7f4317bb4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb3c1bc0 0x55bccb3b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.638+0000 7f4317bb4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bccb2c8060 con 0x55bccb3c1bc0 2023-10-06T10:40:50.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.638+0000 7f4317bb4b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bccb273890 con 0x55bccb3a0e60 2023-10-06T10:40:50.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.638+0000 7f4317bb4b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bccb28eea0 con 0x55bccb377950 2023-10-06T10:40:50.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.638+0000 7f4317bb4b80 10 client.0 _ll_register_callbacks cb 0x55bccb164f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:40:50.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.638+0000 7f430ce59700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb3c1bc0 0x55bccb3b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.638+0000 7f430ce59700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb3c1bc0 0x55bccb3b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35066/0 (socket says 172.21.15.92:35066) 2023-10-06T10:40:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.638+0000 7f430ce59700 1 -- 172.21.15.92:0/1350221975 learned_addr learned my addr 172.21.15.92:0/1350221975 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.638+0000 7f43077fe700 1 --2- 172.21.15.92:0/1350221975 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bccb3a0e60 0x55bccb3ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.638+0000 7f4307fff700 1 --2- 172.21.15.92:0/1350221975 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bccb377950 0x55bccb3b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.638+0000 7f430ce59700 1 -- 172.21.15.92:0/1350221975 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bccb377950 msgr2=0x55bccb3b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.638+0000 7f430ce59700 1 --2- 172.21.15.92:0/1350221975 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bccb377950 0x55bccb3b7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.638+0000 7f430ce59700 1 -- 172.21.15.92:0/1350221975 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bccb3a0e60 msgr2=0x55bccb3ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.638+0000 7f430ce59700 1 --2- 172.21.15.92:0/1350221975 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bccb3a0e60 0x55bccb3ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.638+0000 7f430ce59700 1 -- 172.21.15.92:0/1350221975 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4300002050 con 0x55bccb3c1bc0 2023-10-06T10:40:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.638+0000 7f430ce59700 1 --2- 172.21.15.92:0/1350221975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb3c1bc0 0x55bccb3b5ae0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4300001730 tx=0x7f4300018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1c7eb095134bc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.638+0000 7f42e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:40:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.639+0000 7f42e67fc700 1 -- 172.21.15.92:0/1350221975 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4300007570 con 0x55bccb3c1bc0 2023-10-06T10:40:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.639+0000 7f42e67fc700 1 -- 172.21.15.92:0/1350221975 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f430001f630 con 0x55bccb3c1bc0 2023-10-06T10:40:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.639+0000 7f4317bb4b80 1 -- 172.21.15.92:0/1350221975 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bccb36c690 con 0x55bccb3c1bc0 2023-10-06T10:40:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.639+0000 7f42e67fc700 1 -- 172.21.15.92:0/1350221975 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f430001fa40 con 0x55bccb3c1bc0 2023-10-06T10:40:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.639+0000 7f4317bb4b80 10 client.5985 fetch_fsmap learned FSMap version 37 2023-10-06T10:40:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.639+0000 7f4317bb4b80 1 -- 172.21.15.92:0/1350221975 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55bccb104fe0 con 0x55bccb3c1bc0 2023-10-06T10:40:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.639+0000 7f42e67fc700 1 -- 172.21.15.92:0/1350221975 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bccb36c690 con 0x55bccb3c1bc0 2023-10-06T10:40:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.639+0000 7f42e67fc700 1 -- 172.21.15.92:0/1350221975 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f4300012070 con 0x55bccb3c1bc0 2023-10-06T10:40:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.639+0000 7f4317bb4b80 10 client.5985 fetch_fsmap finished waiting for FSMap version 37 2023-10-06T10:40:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.639+0000 7f4317bb4b80 10 client.5985 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T10:40:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.639+0000 7f4317bb4b80 20 client.5985 populate_metadata read hostname 'smithi092' 2023-10-06T10:40:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.639+0000 7f4317bb4b80 1 --2- 172.21.15.92:0/1350221975 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x55bccb3d18a0 0x55bccb3d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.639+0000 7f4317bb4b80 4 client.5985 mds_command: new command op to 5736 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:40:50.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.639+0000 7f4317bb4b80 1 -- 172.21.15.92:0/1350221975 --> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bccb3495c0 con 0x55bccb3d18a0 2023-10-06T10:40:50.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.639+0000 7f4307fff700 1 --2- 172.21.15.92:0/1350221975 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x55bccb3d18a0 0x55bccb3d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.639+0000 7f42e67fc700 10 client.5985 ms_handle_connect on v2:172.21.15.92:6835/2458340622 2023-10-06T10:40:50.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.639+0000 7f4307fff700 1 --2- 172.21.15.92:0/1350221975 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x55bccb3d18a0 0x55bccb3d1c70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f7a753d236572c1 server_cookie=90534035d1ee9132 in_seq=0 out_seq=0 2023-10-06T10:40:50.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.640+0000 7f42e67fc700 1 -- 172.21.15.92:0/1350221975 <== mds.0 v2:172.21.15.92:6835/2458340622 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55bccb3495c0 con 0x55bccb3d18a0 2023-10-06T10:40:50.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.640+0000 7f42e67fc700 10 client.5985 handle_command_reply: tid=0 2023-10-06T10:40:50.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.640+0000 7f4317bb4b80 1 client.5985 shutdown 2023-10-06T10:40:50.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.640+0000 7f4317bb4b80 1 -- 172.21.15.92:0/1350221975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb3c1bc0 msgr2=0x55bccb3b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.640+0000 7f4317bb4b80 1 --2- 172.21.15.92:0/1350221975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb3c1bc0 0x55bccb3b5ae0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4300001730 tx=0x7f4300018000 comp rx=0 tx=0).stop 2023-10-06T10:40:50.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.640+0000 7f4317bb4b80 1 -- 172.21.15.92:0/1350221975 shutdown_connections 2023-10-06T10:40:50.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.640+0000 7f4317bb4b80 1 --2- 172.21.15.92:0/1350221975 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x55bccb3d18a0 0x55bccb3d1c70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.640+0000 7f4317bb4b80 1 --2- 172.21.15.92:0/1350221975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb3c1bc0 0x55bccb3b5ae0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.640+0000 7f4317bb4b80 1 --2- 172.21.15.92:0/1350221975 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bccb377950 0x55bccb3b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.640+0000 7f4317bb4b80 1 --2- 172.21.15.92:0/1350221975 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bccb3a0e60 0x55bccb3ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.640+0000 7f42e67fc700 0 client.5985 ms_handle_reset on v2:172.21.15.92:6835/2458340622 2023-10-06T10:40:50.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.640+0000 7f4317bb4b80 1 -- 172.21.15.92:0/1350221975 >> 172.21.15.92:0/1350221975 conn(0x55bccb379270 msgr2=0x55bccb36e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:50.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.640+0000 7f4317bb4b80 1 -- 172.21.15.92:0/1350221975 shutdown_connections 2023-10-06T10:40:50.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.640+0000 7f4317bb4b80 1 -- 172.21.15.92:0/1350221975 wait complete. 2023-10-06T10:40:50.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.640+0000 7f4317bb4b80 20 client.5985 trim_cache size 0 max 16384 2023-10-06T10:40:50.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.657+0000 7f4317bb4b80 1 Processor -- start 2023-10-06T10:40:50.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.657+0000 7f4317bb4b80 1 -- start start 2023-10-06T10:40:50.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.657+0000 7f4317bb4b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bccb377950 0x55bccb3a7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.657+0000 7f4317bb4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb3a0e60 0x55bccb3bb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.657+0000 7f4317bb4b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bccb3c1bc0 0x55bccb3bb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.657+0000 7f4317bb4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bccb14ae30 con 0x55bccb3a0e60 2023-10-06T10:40:50.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.657+0000 7f4317bb4b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bccb3bf3d0 con 0x55bccb3c1bc0 2023-10-06T10:40:50.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.657+0000 7f4317bb4b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bccb357030 con 0x55bccb377950 2023-10-06T10:40:50.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.658+0000 7f43077fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb3a0e60 0x55bccb3bb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.658+0000 7f43077fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb3a0e60 0x55bccb3bb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35076/0 (socket says 172.21.15.92:35076) 2023-10-06T10:40:50.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.658+0000 7f43077fe700 1 -- 172.21.15.92:0/4029140163 learned_addr learned my addr 172.21.15.92:0/4029140163 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:50.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.658+0000 7f4307fff700 1 --2- 172.21.15.92:0/4029140163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bccb377950 0x55bccb3a7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.658+0000 7f43077fe700 1 -- 172.21.15.92:0/4029140163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bccb377950 msgr2=0x55bccb3a7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.658+0000 7f43077fe700 1 --2- 172.21.15.92:0/4029140163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bccb377950 0x55bccb3a7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.658+0000 7f43077fe700 1 -- 172.21.15.92:0/4029140163 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bccb3c1bc0 msgr2=0x55bccb3bb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.658+0000 7f430ce59700 1 --2- 172.21.15.92:0/4029140163 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bccb3c1bc0 0x55bccb3bb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.658+0000 7f43077fe700 1 --2- 172.21.15.92:0/4029140163 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bccb3c1bc0 0x55bccb3bb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.658+0000 7f43077fe700 1 -- 172.21.15.92:0/4029140163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bccb296780 con 0x55bccb3a0e60 2023-10-06T10:40:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.658+0000 7f43077fe700 1 --2- 172.21.15.92:0/4029140163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb3a0e60 0x55bccb3bb460 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0849e0 tx=0x7f42fc086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ee7da60d401055 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.658+0000 7f42edffb700 1 -- 172.21.15.92:0/4029140163 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc057a60 con 0x55bccb3a0e60 2023-10-06T10:40:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.658+0000 7f42edffb700 1 -- 172.21.15.92:0/4029140163 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42fc057360 con 0x55bccb3a0e60 2023-10-06T10:40:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.658+0000 7f42edffb700 1 -- 172.21.15.92:0/4029140163 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc05a5b0 con 0x55bccb3a0e60 2023-10-06T10:40:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.659+0000 7f4317bb4b80 1 -- 172.21.15.92:0/4029140163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb3a0e60 msgr2=0x55bccb3bb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.659+0000 7f4317bb4b80 1 --2- 172.21.15.92:0/4029140163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb3a0e60 0x55bccb3bb460 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0849e0 tx=0x7f42fc086040 comp rx=0 tx=0).stop 2023-10-06T10:40:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.659+0000 7f4317bb4b80 1 -- 172.21.15.92:0/4029140163 shutdown_connections 2023-10-06T10:40:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.659+0000 7f4317bb4b80 1 --2- 172.21.15.92:0/4029140163 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bccb3c1bc0 0x55bccb3bb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.659+0000 7f4317bb4b80 1 --2- 172.21.15.92:0/4029140163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bccb377950 0x55bccb3a7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.659+0000 7f4317bb4b80 1 --2- 172.21.15.92:0/4029140163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb3a0e60 0x55bccb3bb460 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.659+0000 7f4317bb4b80 1 -- 172.21.15.92:0/4029140163 >> 172.21.15.92:0/4029140163 conn(0x55bccb379270 msgr2=0x55bccb3cd230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.659+0000 7f4317bb4b80 1 -- 172.21.15.92:0/4029140163 shutdown_connections 2023-10-06T10:40:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.659+0000 7f4317bb4b80 1 -- 172.21.15.92:0/4029140163 wait complete. 2023-10-06T10:40:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.659+0000 7f4317bb4b80 1 -- start start 2023-10-06T10:40:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.659+0000 7f4317bb4b80 1 Processor -- start 2023-10-06T10:40:50.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.660+0000 7f4317bb4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb377950 0x55bccb426a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.660+0000 7f4317bb4b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bccb3a0e60 0x55bccb3bb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.660+0000 7f4317bb4b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bccb3c1bc0 0x55bccb3bb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.660+0000 7f4317bb4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bccb3dedd0 con 0x55bccb377950 2023-10-06T10:40:50.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.660+0000 7f4317bb4b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bccb39dbb0 con 0x55bccb3a0e60 2023-10-06T10:40:50.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.660+0000 7f4317bb4b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bccb3dd600 con 0x55bccb3c1bc0 2023-10-06T10:40:50.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.660+0000 7f4317bb4b80 10 client.0 _ll_register_callbacks cb 0x55bccb164f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:40:50.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.660+0000 7f4307fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb377950 0x55bccb426a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.660+0000 7f4307fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb377950 0x55bccb426a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35078/0 (socket says 172.21.15.92:35078) 2023-10-06T10:40:50.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.660+0000 7f4307fff700 1 -- 172.21.15.92:0/2024816260 learned_addr learned my addr 172.21.15.92:0/2024816260 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:50.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.660+0000 7f43077fe700 1 --2- 172.21.15.92:0/2024816260 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bccb3a0e60 0x55bccb3bb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.660+0000 7f430ce59700 1 --2- 172.21.15.92:0/2024816260 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bccb3c1bc0 0x55bccb3bb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.660+0000 7f4307fff700 1 -- 172.21.15.92:0/2024816260 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bccb3c1bc0 msgr2=0x55bccb3bb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.661+0000 7f4307fff700 1 --2- 172.21.15.92:0/2024816260 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bccb3c1bc0 0x55bccb3bb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.661+0000 7f4307fff700 1 -- 172.21.15.92:0/2024816260 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bccb3a0e60 msgr2=0x55bccb3bb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.661+0000 7f4307fff700 1 --2- 172.21.15.92:0/2024816260 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bccb3a0e60 0x55bccb3bb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.661+0000 7f4307fff700 1 -- 172.21.15.92:0/2024816260 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430816a590 con 0x55bccb377950 2023-10-06T10:40:50.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.661+0000 7f43077fe700 1 --2- 172.21.15.92:0/2024816260 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bccb3a0e60 0x55bccb3bb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:40:50.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.661+0000 7f4307fff700 1 --2- 172.21.15.92:0/2024816260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb377950 0x55bccb426a20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x55bccb3d1420 tx=0x7f42f8024040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aef0991bafee7e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.661+0000 7f42e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:40:50.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.661+0000 7f42e67fc700 1 -- 172.21.15.92:0/2024816260 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f802e070 con 0x55bccb377950 2023-10-06T10:40:50.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.661+0000 7f42e67fc700 1 -- 172.21.15.92:0/2024816260 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f42f801b040 con 0x55bccb377950 2023-10-06T10:40:50.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.661+0000 7f4317bb4b80 1 -- 172.21.15.92:0/2024816260 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4300007230 con 0x55bccb377950 2023-10-06T10:40:50.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.661+0000 7f42e67fc700 1 -- 172.21.15.92:0/2024816260 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42f8027040 con 0x55bccb377950 2023-10-06T10:40:50.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.661+0000 7f4317bb4b80 10 client.5991 fetch_fsmap learned FSMap version 37 2023-10-06T10:40:50.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.661+0000 7f4317bb4b80 1 -- 172.21.15.92:0/2024816260 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55bccb27d470 con 0x55bccb377950 2023-10-06T10:40:50.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.661+0000 7f42e67fc700 1 -- 172.21.15.92:0/2024816260 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4300007230 con 0x55bccb377950 2023-10-06T10:40:50.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.661+0000 7f42e67fc700 1 -- 172.21.15.92:0/2024816260 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f42f8031020 con 0x55bccb377950 2023-10-06T10:40:50.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.661+0000 7f4317bb4b80 10 client.5991 fetch_fsmap finished waiting for FSMap version 37 2023-10-06T10:40:50.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.661+0000 7f4317bb4b80 10 client.5991 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T10:40:50.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.661+0000 7f4317bb4b80 20 client.5991 populate_metadata read hostname 'smithi092' 2023-10-06T10:40:50.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.661+0000 7f4317bb4b80 1 --2- 172.21.15.92:0/2024816260 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x55bccb3c2480 0x55bccb3c2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.661+0000 7f4317bb4b80 4 client.5991 mds_command: new command op to 5736 tid=0[{"prefix": "status"}] 2023-10-06T10:40:50.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.661+0000 7f4317bb4b80 1 -- 172.21.15.92:0/2024816260 --> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bccb26ae00 con 0x55bccb3c2480 2023-10-06T10:40:50.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.661+0000 7f43077fe700 1 --2- 172.21.15.92:0/2024816260 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x55bccb3c2480 0x55bccb3c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.662+0000 7f42e67fc700 10 client.5991 ms_handle_connect on v2:172.21.15.92:6835/2458340622 2023-10-06T10:40:50.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.662+0000 7f43077fe700 1 --2- 172.21.15.92:0/2024816260 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x55bccb3c2480 0x55bccb3c2850 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f8126fc3249e30d server_cookie=6be01cd05d88bcab in_seq=0 out_seq=0 2023-10-06T10:40:50.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.662+0000 7f42e67fc700 1 -- 172.21.15.92:0/2024816260 <== mds.0 v2:172.21.15.92:6835/2458340622 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x55bccb26ae00 con 0x55bccb3c2480 2023-10-06T10:40:50.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.662+0000 7f42e67fc700 10 client.5991 handle_command_reply: tid=0 2023-10-06T10:40:50.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.662+0000 7f4317bb4b80 1 client.5991 shutdown 2023-10-06T10:40:50.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.662+0000 7f4317bb4b80 1 -- 172.21.15.92:0/2024816260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb377950 msgr2=0x55bccb426a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.662+0000 7f4317bb4b80 1 --2- 172.21.15.92:0/2024816260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb377950 0x55bccb426a20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x55bccb3d1420 tx=0x7f42f8024040 comp rx=0 tx=0).stop 2023-10-06T10:40:50.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.662+0000 7f4317bb4b80 1 -- 172.21.15.92:0/2024816260 shutdown_connections 2023-10-06T10:40:50.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.662+0000 7f4317bb4b80 1 --2- 172.21.15.92:0/2024816260 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x55bccb3c2480 0x55bccb3c2850 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.662+0000 7f4317bb4b80 1 --2- 172.21.15.92:0/2024816260 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bccb3c1bc0 0x55bccb3bb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.662+0000 7f42e67fc700 0 client.5991 ms_handle_reset on v2:172.21.15.92:6835/2458340622 2023-10-06T10:40:50.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.662+0000 7f4317bb4b80 1 --2- 172.21.15.92:0/2024816260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bccb377950 0x55bccb426a20 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.662+0000 7f4317bb4b80 1 --2- 172.21.15.92:0/2024816260 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bccb3a0e60 0x55bccb3bb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.662+0000 7f4317bb4b80 1 -- 172.21.15.92:0/2024816260 >> 172.21.15.92:0/2024816260 conn(0x55bccb379270 msgr2=0x55bccb3d7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:50.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.662+0000 7f4317bb4b80 1 -- 172.21.15.92:0/2024816260 shutdown_connections 2023-10-06T10:40:50.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.662+0000 7f4317bb4b80 1 -- 172.21.15.92:0/2024816260 wait complete. 2023-10-06T10:40:50.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.662+0000 7f4317bb4b80 20 client.5991 trim_cache size 0 max 16384 2023-10-06T10:40:50.673 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:40:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:40:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T10:40:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 5736, 2023-10-06T10:40:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:40:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:40:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:40:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 15.120500584, 2023-10-06T10:40:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 37, 2023-10-06T10:40:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 52, 2023-10-06T10:40:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 52, 2023-10-06T10:40:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 21.245629543 2023-10-06T10:40:50.674 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:40:50.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.663+0000 7f430d65a700 1 -- 172.21.15.92:0/1187142257 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f42f4066f30 msgr2=0x7f42f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.663+0000 7f430d65a700 1 --2- 172.21.15.92:0/1187142257 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f42f4066f30 0x7f42f40693e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f43080b4cd0 tx=0x7f42f8009060 comp rx=0 tx=0).stop 2023-10-06T10:40:50.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.663+0000 7f430d65a700 1 -- 172.21.15.92:0/1187142257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43080cbc30 msgr2=0x7f43081540f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.663+0000 7f430d65a700 1 --2- 172.21.15.92:0/1187142257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43080cbc30 0x7f43081540f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0091e0 tx=0x7f42fc016000 comp rx=0 tx=0).stop 2023-10-06T10:40:50.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.663+0000 7f430d65a700 1 -- 172.21.15.92:0/1187142257 shutdown_connections 2023-10-06T10:40:50.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.663+0000 7f430d65a700 1 --2- 172.21.15.92:0/1187142257 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f42f4066f30 0x7f42f40693e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.663+0000 7f430d65a700 1 --2- 172.21.15.92:0/1187142257 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4308159440 0x7f43080c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.663+0000 7f430d65a700 1 --2- 172.21.15.92:0/1187142257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43080b2100 0x7f43080d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.663+0000 7f430d65a700 1 --2- 172.21.15.92:0/1187142257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43080cbc30 0x7f43081540f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.663+0000 7f430d65a700 1 -- 172.21.15.92:0/1187142257 >> 172.21.15.92:0/1187142257 conn(0x7f430806ece0 msgr2=0x7f4308155320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:50.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.663+0000 7f430d65a700 1 -- 172.21.15.92:0/1187142257 shutdown_connections 2023-10-06T10:40:50.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.663+0000 7f430d65a700 1 -- 172.21.15.92:0/1187142257 wait complete. 2023-10-06T10:40:50.677 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:40:50.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.789+0000 7f9a7f9b5700 1 Processor -- start 2023-10-06T10:40:50.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.789+0000 7f9a7f9b5700 1 -- start start 2023-10-06T10:40:50.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.790+0000 7f9a7f9b5700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a780b25e0 0x7f9a780b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.790+0000 7f9a7f9b5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9a78159830 0x7f9a780cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.790+0000 7f9a7f9b5700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a780cbfa0 con 0x7f9a780b29b0 2023-10-06T10:40:50.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.790+0000 7f9a7f9b5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a780cb390 con 0x7f9a78159830 2023-10-06T10:40:50.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.790+0000 7f9a7f9b5700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780cb670 con 0x7f9a780b25e0 2023-10-06T10:40:50.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.790+0000 7f9a7df52700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9a78159830 0x7f9a780cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.790+0000 7f9a7df52700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9a78159830 0x7f9a780cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35088/0 (socket says 172.21.15.92:35088) 2023-10-06T10:40:50.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.790+0000 7f9a7df52700 1 -- 172.21.15.92:0/1323907997 learned_addr learned my addr 172.21.15.92:0/1323907997 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:50.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.790+0000 7f9a7d751700 1 --2- 172.21.15.92:0/1323907997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a780b25e0 0x7f9a780b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.791+0000 7f9a6ffff700 1 -- 172.21.15.92:0/1323907997 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3005432561 0 0) 0x7f9a780cbfa0 con 0x7f9a780b29b0 2023-10-06T10:40:50.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.791+0000 7f9a7df52700 1 -- 172.21.15.92:0/1323907997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a780b25e0 msgr2=0x7f9a780b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:40:50.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.791+0000 7f9a7df52700 1 --2- 172.21.15.92:0/1323907997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a780b25e0 0x7f9a780b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.791+0000 7f9a7df52700 1 -- 172.21.15.92:0/1323907997 >> v1:172.21.15.124:6789/0 conn(0x7f9a780b29b0 legacy=0x7f9a78159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.791+0000 7f9a7df52700 1 -- 172.21.15.92:0/1323907997 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a780cb950 con 0x7f9a78159830 2023-10-06T10:40:50.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.791+0000 7f9a7df52700 1 --2- 172.21.15.92:0/1323907997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a78159830 0x7f9a780cc430 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f9a700099b0 tx=0x7f9a70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4886b98767e348a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.791+0000 7f9a6ffff700 1 -- 172.21.15.92:0/1323907997 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a70009ea0 con 0x7f9a78159830 2023-10-06T10:40:50.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.791+0000 7f9a6ffff700 1 -- 172.21.15.92:0/1323907997 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7000b860 con 0x7f9a78159830 2023-10-06T10:40:50.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.791+0000 7f9a6ffff700 1 -- 172.21.15.92:0/1323907997 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7000bc00 con 0x7f9a78159830 2023-10-06T10:40:50.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.792+0000 7f9a7f9b5700 1 -- 172.21.15.92:0/1323907997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a78159830 msgr2=0x7f9a780cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.792+0000 7f9a7f9b5700 1 --2- 172.21.15.92:0/1323907997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a78159830 0x7f9a780cc430 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f9a700099b0 tx=0x7f9a70009250 comp rx=0 tx=0).stop 2023-10-06T10:40:50.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.792+0000 7f9a7f9b5700 1 -- 172.21.15.92:0/1323907997 shutdown_connections 2023-10-06T10:40:50.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.792+0000 7f9a7f9b5700 1 --2- 172.21.15.92:0/1323907997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a78159830 0x7f9a780cc430 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.792+0000 7f9a7f9b5700 1 --2- 172.21.15.92:0/1323907997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a780b25e0 0x7f9a780b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.792+0000 7f9a7f9b5700 1 -- 172.21.15.92:0/1323907997 >> 172.21.15.92:0/1323907997 conn(0x7f9a7806ece0 msgr2=0x7f9a7806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:50.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.792+0000 7f9a7f9b5700 1 -- 172.21.15.92:0/1323907997 shutdown_connections 2023-10-06T10:40:50.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.792+0000 7f9a7f9b5700 1 -- 172.21.15.92:0/1323907997 wait complete. 2023-10-06T10:40:50.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.793+0000 7f9a7f9b5700 1 Processor -- start 2023-10-06T10:40:50.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.793+0000 7f9a7f9b5700 1 -- start start 2023-10-06T10:40:50.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a7f9b5700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a780b25e0 0x7f9a781488c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a7f9b5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a780b29b0 0x7f9a78156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a7f9b5700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a78159830 0x7f9a78154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a7f9b5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a78157ed0 con 0x7f9a780b29b0 2023-10-06T10:40:50.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a7f9b5700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a78148e00 con 0x7f9a780b25e0 2023-10-06T10:40:50.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a7f9b5700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a78158360 con 0x7f9a78159830 2023-10-06T10:40:50.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a7cf50700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a780b29b0 0x7f9a78156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a7cf50700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a780b29b0 0x7f9a78156940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35090/0 (socket says 172.21.15.92:35090) 2023-10-06T10:40:50.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a7cf50700 1 -- 172.21.15.92:0/683130740 learned_addr learned my addr 172.21.15.92:0/683130740 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:40:50.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a7d751700 1 --2- 172.21.15.92:0/683130740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a780b25e0 0x7f9a781488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a7df52700 1 --2- 172.21.15.92:0/683130740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a78159830 0x7f9a78154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a7cf50700 1 -- 172.21.15.92:0/683130740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a78159830 msgr2=0x7f9a78154650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a7cf50700 1 --2- 172.21.15.92:0/683130740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a78159830 0x7f9a78154650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a7cf50700 1 -- 172.21.15.92:0/683130740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a780b25e0 msgr2=0x7f9a781488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:50.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a7cf50700 1 --2- 172.21.15.92:0/683130740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a780b25e0 0x7f9a781488c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:50.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a7cf50700 1 -- 172.21.15.92:0/683130740 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a700094f0 con 0x7f9a780b29b0 2023-10-06T10:40:50.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a7d751700 1 --2- 172.21.15.92:0/683130740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a780b25e0 0x7f9a781488c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:40:50.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a7cf50700 1 --2- 172.21.15.92:0/683130740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a780b29b0 0x7f9a78156940 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800c7e0 tx=0x7f9a68007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a6d7fa700 1 -- 172.21.15.92:0/683130740 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a680105c0 con 0x7f9a780b29b0 2023-10-06T10:40:50.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a6d7fa700 1 -- 172.21.15.92:0/683130740 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a68010740 con 0x7f9a780b29b0 2023-10-06T10:40:50.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a7f9b5700 1 -- 172.21.15.92:0/683130740 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a781572c0 con 0x7f9a780b29b0 2023-10-06T10:40:50.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a6d7fa700 1 -- 172.21.15.92:0/683130740 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a68010b00 con 0x7f9a780b29b0 2023-10-06T10:40:50.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.794+0000 7f9a7f9b5700 1 -- 172.21.15.92:0/683130740 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a78157880 con 0x7f9a780b29b0 2023-10-06T10:40:50.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.795+0000 7f9a7f9b5700 1 -- 172.21.15.92:0/683130740 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a78158080 con 0x7f9a780b29b0 2023-10-06T10:40:50.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.796+0000 7f9a6d7fa700 1 -- 172.21.15.92:0/683130740 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9a6801b3b0 con 0x7f9a780b29b0 2023-10-06T10:40:50.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.796+0000 7f9a6d7fa700 1 --2- 172.21.15.92:0/683130740 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a5c066f30 0x7f9a5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:40:50.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.796+0000 7f9a6d7fa700 1 -- 172.21.15.92:0/683130740 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9a68083e00 con 0x7f9a780b29b0 2023-10-06T10:40:50.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.796+0000 7f9a7d751700 1 --2- 172.21.15.92:0/683130740 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a5c066f30 0x7f9a5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:40:50.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.796+0000 7f9a7d751700 1 --2- 172.21.15.92:0/683130740 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a5c066f30 0x7f9a5c0693e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400e9a0 tx=0x7f9a7400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:40:50.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:50.800+0000 7f9a6d7fa700 1 -- 172.21.15.92:0/683130740 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a78158080 con 0x7f9a780b29b0 2023-10-06T10:40:51.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:51.027+0000 7f9a7f9b5700 1 -- 172.21.15.92:0/683130740 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a78064a90 con 0x7f9a780b29b0 2023-10-06T10:40:51.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:51.027+0000 7f9a6d7fa700 1 -- 172.21.15.92:0/683130740 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4628 (secure 0 0 0) 0x7f9a68017070 con 0x7f9a780b29b0 2023-10-06T10:40:51.030 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:40:51.030 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":37,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3668157650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3668157650},{"type":"v1","addr":"172.21.15.124:6834","nonce":3668157650}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:40:34.932631+0000","modified":"2023-10-06T10:40:49.624417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5736,"mds_1":5733,"mds_2":5036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5036":{"gid":5036,"name":"d","rank":2,"incarnation":36,"state":"up:active","state_seq":6,"addr":"172.21.15.124:6835/2986058955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2986058955},{"type":"v1","addr":"172.21.15.124:6835","nonce":2986058955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5733":{"gid":5733,"name":"c","rank":1,"incarnation":34,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6836/831923836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":831923836},{"type":"v1","addr":"172.21.15.92:6836","nonce":831923836}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5736":{"gid":5736,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2458340622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2458340622},{"type":"v1","addr":"172.21.15.92:6837","nonce":2458340622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T10:40:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:51.029+0000 7f9a7f9b5700 1 -- 172.21.15.92:0/683130740 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a5c066f30 msgr2=0x7f9a5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:51.029+0000 7f9a7f9b5700 1 --2- 172.21.15.92:0/683130740 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a5c066f30 0x7f9a5c0693e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400e9a0 tx=0x7f9a7400b040 comp rx=0 tx=0).stop 2023-10-06T10:40:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:51.029+0000 7f9a7f9b5700 1 -- 172.21.15.92:0/683130740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a780b29b0 msgr2=0x7f9a78156940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:40:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:51.029+0000 7f9a7f9b5700 1 --2- 172.21.15.92:0/683130740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a780b29b0 0x7f9a78156940 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800c7e0 tx=0x7f9a68007df0 comp rx=0 tx=0).stop 2023-10-06T10:40:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:51.029+0000 7f9a7f9b5700 1 -- 172.21.15.92:0/683130740 shutdown_connections 2023-10-06T10:40:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:51.029+0000 7f9a7f9b5700 1 --2- 172.21.15.92:0/683130740 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a5c066f30 0x7f9a5c0693e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:51.029+0000 7f9a7f9b5700 1 --2- 172.21.15.92:0/683130740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a78159830 0x7f9a78154650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:51.029+0000 7f9a7f9b5700 1 --2- 172.21.15.92:0/683130740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a780b25e0 0x7f9a781488c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:51.029+0000 7f9a7f9b5700 1 --2- 172.21.15.92:0/683130740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a780b29b0 0x7f9a78156940 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:40:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:51.029+0000 7f9a7f9b5700 1 -- 172.21.15.92:0/683130740 >> 172.21.15.92:0/683130740 conn(0x7f9a7806ece0 msgr2=0x7f9a780b59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:40:51.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:51.029+0000 7f9a7f9b5700 1 -- 172.21.15.92:0/683130740 shutdown_connections 2023-10-06T10:40:51.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:40:51.029+0000 7f9a7f9b5700 1 -- 172.21.15.92:0/683130740 wait complete. 2023-10-06T10:40:51.033 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 37 2023-10-06T10:40:51.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:40:51.044 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2023-10-06T10:40:51.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:40:51.067 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2023-10-06T10:40:51.128 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:40:51.128 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2023-10-06T10:40:51.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:40:51.184 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2023-10-06T10:40:51.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:40:51.240 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2023-10-06T10:40:51.295 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-10-06T10:40:51.464 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:40:51.465 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:40:51.465 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:40:52.919 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:40:52.921 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:40:53.466 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-10-06T10:40:53.606 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:40:53.607 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:40:53.607 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:40:55.608 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-10-06T10:40:55.757 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:40:55.757 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:40:55.757 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:40:57.758 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-10-06T10:40:57.900 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:40:57.900 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:40:57.900 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:40:59.902 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-10-06T10:41:00.044 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2023-10-06T10:41:00.045 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:41:00.045 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:41:02.046 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-10-06T10:41:02.196 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2023-10-06T10:41:02.196 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2023-10-06T10:41:02.197 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:41:02.197 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:41:02.329 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-10-06T10:41:02.330 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:41:02.330 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:41:04.331 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:41:04.473 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-10-06T10:41:04.473 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:41:04.474 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:41:06.475 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:41:06.618 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2023-10-06T10:41:06.619 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2023-10-06T10:41:06.619 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:41:06.619 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2023-10-06T10:41:06.757 INFO:teuthology.orchestra.run.smithi124.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-10-06T10:41:06.757 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:41:06.758 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:41:06.758 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2023-10-06T10:41:06.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.872+0000 7f743e08f700 1 Processor -- start 2023-10-06T10:41:06.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.872+0000 7f743e08f700 1 -- start start 2023-10-06T10:41:06.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.873+0000 7f743e08f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74380cd3c0 0x7f74380ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:06.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.873+0000 7f743e08f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74380cc970 con 0x7f74380b2620 2023-10-06T10:41:06.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.873+0000 7f743e08f700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74380cbd60 con 0x7f74380b29f0 2023-10-06T10:41:06.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.873+0000 7f743e08f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74380cc040 con 0x7f74380cd3c0 2023-10-06T10:41:06.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.873+0000 7f743d08d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f74380b2620 0x7f74380cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59140/0 (socket says 172.21.15.92:59140) 2023-10-06T10:41:06.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.873+0000 7f743d08d700 1 -- 172.21.15.92:0/3891811442 learned_addr learned my addr 172.21.15.92:0/3891811442 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f743c88c700 1 --2- 172.21.15.92:0/3891811442 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74380cd3c0 0x7f74380ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f742ffff700 1 -- 172.21.15.92:0/3891811442 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2526572082 0 0) 0x7f74380cc970 con 0x7f74380b2620 2023-10-06T10:41:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f742ffff700 1 -- 172.21.15.92:0/3891811442 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7424003570 con 0x7f74380b2620 2023-10-06T10:41:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f742ffff700 1 -- 172.21.15.92:0/3891811442 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1635189708 0 0) 0x7f7424003570 con 0x7f74380b2620 2023-10-06T10:41:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f742ffff700 1 -- 172.21.15.92:0/3891811442 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74380cd3c0 msgr2=0x7f74380ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f742ffff700 1 --2- 172.21.15.92:0/3891811442 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74380cd3c0 0x7f74380ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f742ffff700 1 -- 172.21.15.92:0/3891811442 >> v1:172.21.15.124:6789/0 conn(0x7f74380b29f0 legacy=0x7f74380c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f742ffff700 1 -- 172.21.15.92:0/3891811442 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74380cc320 con 0x7f74380b2620 2023-10-06T10:41:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f742ffff700 1 -- 172.21.15.92:0/3891811442 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f7434001970 con 0x7f74380b2620 2023-10-06T10:41:06.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f742ffff700 1 -- 172.21.15.92:0/3891811442 >> v1:172.21.15.92:6789/0 conn(0x7f74380b2620 legacy=0x7f74380cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:06.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f742ffff700 1 --2- 172.21.15.92:0/3891811442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7424003a90 0x7f7424003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:06.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f742ffff700 1 --2- 172.21.15.92:0/3891811442 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74380cd3c0 0x7f7424004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:06.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f742ffff700 1 --2- 172.21.15.92:0/3891811442 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7424004d50 0x7f74240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:06.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f742ffff700 1 -- 172.21.15.92:0/3891811442 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74380cc040 con 0x7f7424003a90 2023-10-06T10:41:06.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f742ffff700 1 -- 172.21.15.92:0/3891811442 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742400bb10 con 0x7f7424004d50 2023-10-06T10:41:06.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f742ffff700 1 -- 172.21.15.92:0/3891811442 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742400bcd0 con 0x7f74380cd3c0 2023-10-06T10:41:06.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f743d08d700 1 --2- 172.21.15.92:0/3891811442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7424003a90 0x7f7424003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:06.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f743c88c700 1 --2- 172.21.15.92:0/3891811442 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74380cd3c0 0x7f7424004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:06.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f743d88e700 1 --2- 172.21.15.92:0/3891811442 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7424004d50 0x7f74240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:06.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f743d08d700 1 -- 172.21.15.92:0/3891811442 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74380cd3c0 msgr2=0x7f7424004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:06.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f743d08d700 1 --2- 172.21.15.92:0/3891811442 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74380cd3c0 0x7f7424004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f743d08d700 1 -- 172.21.15.92:0/3891811442 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7424004d50 msgr2=0x7f74240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:06.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f743d08d700 1 --2- 172.21.15.92:0/3891811442 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7424004d50 0x7f74240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.874+0000 7f743d08d700 1 -- 172.21.15.92:0/3891811442 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7424003570 con 0x7f7424003a90 2023-10-06T10:41:06.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.875+0000 7f743d08d700 1 --2- 172.21.15.92:0/3891811442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7424003a90 0x7f7424003e60 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f743400c5e0 tx=0x7f7434007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9040bd8753b57f2e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:06.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.875+0000 7f742ffff700 1 -- 172.21.15.92:0/3891811442 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7434007e30 con 0x7f7424003a90 2023-10-06T10:41:06.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.875+0000 7f742ffff700 1 -- 172.21.15.92:0/3891811442 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7434001e90 con 0x7f7424003a90 2023-10-06T10:41:06.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.875+0000 7f742ffff700 1 -- 172.21.15.92:0/3891811442 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7434005da0 con 0x7f7424003a90 2023-10-06T10:41:06.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.875+0000 7f743e08f700 1 -- 172.21.15.92:0/3891811442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7424003a90 msgr2=0x7f7424003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:06.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.875+0000 7f743e08f700 1 --2- 172.21.15.92:0/3891811442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7424003a90 0x7f7424003e60 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f743400c5e0 tx=0x7f7434007c20 comp rx=0 tx=0).stop 2023-10-06T10:41:06.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.875+0000 7f743d88e700 1 -- 172.21.15.92:0/3891811442 reap_dead start 2023-10-06T10:41:06.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.875+0000 7f743e08f700 1 -- 172.21.15.92:0/3891811442 shutdown_connections 2023-10-06T10:41:06.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.875+0000 7f743e08f700 1 -- 172.21.15.92:0/3891811442 >> 172.21.15.92:0/3891811442 conn(0x7f7438099520 msgr2=0x7f743806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:06.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.875+0000 7f743e08f700 1 -- 172.21.15.92:0/3891811442 shutdown_connections 2023-10-06T10:41:06.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.875+0000 7f743e08f700 1 -- 172.21.15.92:0/3891811442 wait complete. 2023-10-06T10:41:06.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.876+0000 7f743e08f700 1 Processor -- start 2023-10-06T10:41:06.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.876+0000 7f743e08f700 1 -- start start 2023-10-06T10:41:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.877+0000 7f743e08f700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7424003a90 0x7f7438146e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.877+0000 7f743e08f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74380c7560 0x7f74380c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.877+0000 7f743e08f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7438156c50 0x7f74381570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.877+0000 7f743e08f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74381473b0 con 0x7f7438156c50 2023-10-06T10:41:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.877+0000 7f743e08f700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7438157610 con 0x7f7424003a90 2023-10-06T10:41:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.877+0000 7f743e08f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74380c8600 con 0x7f74380c7560 2023-10-06T10:41:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.877+0000 7f743d88e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7438156c50 0x7f74381570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.877+0000 7f743d88e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7438156c50 0x7f74381570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59634/0 (socket says 172.21.15.92:59634) 2023-10-06T10:41:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.877+0000 7f743d88e700 1 -- 172.21.15.92:0/570134605 learned_addr learned my addr 172.21.15.92:0/570134605 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.877+0000 7f743d08d700 1 --2- 172.21.15.92:0/570134605 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7424003a90 0x7f7438146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.877+0000 7f743c88c700 1 --2- 172.21.15.92:0/570134605 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74380c7560 0x7f74380c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.878+0000 7f743d88e700 1 -- 172.21.15.92:0/570134605 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74380c7560 msgr2=0x7f74380c9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.878+0000 7f743d88e700 1 --2- 172.21.15.92:0/570134605 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74380c7560 0x7f74380c9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.878+0000 7f743d88e700 1 -- 172.21.15.92:0/570134605 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7424003a90 msgr2=0x7f7438146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.878+0000 7f743d88e700 1 --2- 172.21.15.92:0/570134605 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7424003a90 0x7f7438146e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.878+0000 7f743d88e700 1 -- 172.21.15.92:0/570134605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74380c79f0 con 0x7f7438156c50 2023-10-06T10:41:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.878+0000 7f743d88e700 1 --2- 172.21.15.92:0/570134605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7438156c50 0x7f74381570d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f74380b2e10 tx=0x7f7430007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.878+0000 7f742d7fa700 1 -- 172.21.15.92:0/570134605 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7430006a30 con 0x7f7438156c50 2023-10-06T10:41:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.878+0000 7f742d7fa700 1 -- 172.21.15.92:0/570134605 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7430006bd0 con 0x7f7438156c50 2023-10-06T10:41:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.878+0000 7f743e08f700 1 -- 172.21.15.92:0/570134605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74380c7cd0 con 0x7f7438156c50 2023-10-06T10:41:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.878+0000 7f742d7fa700 1 -- 172.21.15.92:0/570134605 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7430007030 con 0x7f7438156c50 2023-10-06T10:41:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.878+0000 7f743e08f700 1 -- 172.21.15.92:0/570134605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74380c8290 con 0x7f7438156c50 2023-10-06T10:41:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.879+0000 7f742d7fa700 1 -- 172.21.15.92:0/570134605 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f74300073e0 con 0x7f7438156c50 2023-10-06T10:41:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.880+0000 7f742d7fa700 1 --2- 172.21.15.92:0/570134605 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f741c066f30 0x7f741c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.880+0000 7f742d7fa700 1 -- 172.21.15.92:0/570134605 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7430083eb0 con 0x7f7438156c50 2023-10-06T10:41:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.880+0000 7f743d08d700 1 --2- 172.21.15.92:0/570134605 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f741c066f30 0x7f741c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.880+0000 7f743d08d700 1 --2- 172.21.15.92:0/570134605 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f741c066f30 0x7f741c0693e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7434001270 tx=0x7f7434015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.881+0000 7f74485e9b80 1 Processor -- start 2023-10-06T10:41:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.882+0000 7f74485e9b80 1 -- start start 2023-10-06T10:41:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.882+0000 7f74485e9b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56166570e9a0 0x56166570ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.882+0000 7f74485e9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561665758c10 0x561665737970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.882+0000 7f74485e9b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561665737eb0 0x56166570a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.882+0000 7f74485e9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166568ecd0 con 0x561665758c10 2023-10-06T10:41:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.882+0000 7f74485e9b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561665665ac0 con 0x56166570e9a0 2023-10-06T10:41:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.882+0000 7f74485e9b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616656632c0 con 0x561665737eb0 2023-10-06T10:41:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.882+0000 7f743c88c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561665758c10 0x561665737970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.882+0000 7f743c88c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561665758c10 0x561665737970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59636/0 (socket says 172.21.15.92:59636) 2023-10-06T10:41:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.882+0000 7f743c88c700 1 -- 172.21.15.92:0/4124813118 learned_addr learned my addr 172.21.15.92:0/4124813118 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.882+0000 7f743d08d700 1 --2- 172.21.15.92:0/4124813118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56166570e9a0 0x56166570ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.882+0000 7f743d88e700 1 --2- 172.21.15.92:0/4124813118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561665737eb0 0x56166570a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.883+0000 7f743c88c700 1 -- 172.21.15.92:0/4124813118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561665737eb0 msgr2=0x56166570a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.883+0000 7f743c88c700 1 --2- 172.21.15.92:0/4124813118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561665737eb0 0x56166570a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.883+0000 7f743c88c700 1 -- 172.21.15.92:0/4124813118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56166570e9a0 msgr2=0x56166570ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.883+0000 7f743c88c700 1 --2- 172.21.15.92:0/4124813118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56166570e9a0 0x56166570ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.883+0000 7f743c88c700 1 -- 172.21.15.92:0/4124813118 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616654f92c0 con 0x561665758c10 2023-10-06T10:41:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.883+0000 7f743d08d700 1 --2- 172.21.15.92:0/4124813118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56166570e9a0 0x56166570ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.883+0000 7f743d88e700 1 --2- 172.21.15.92:0/4124813118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561665737eb0 0x56166570a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.883+0000 7f743c88c700 1 --2- 172.21.15.92:0/4124813118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561665758c10 0x561665737970 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7428009d20 tx=0x7f74280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24956fa30b55643 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.883+0000 7f74227fc700 1 -- 172.21.15.92:0/4124813118 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7428014ab0 con 0x561665758c10 2023-10-06T10:41:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.883+0000 7f74227fc700 1 -- 172.21.15.92:0/4124813118 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7428014c10 con 0x561665758c10 2023-10-06T10:41:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.883+0000 7f74227fc700 1 -- 172.21.15.92:0/4124813118 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742801d440 con 0x561665758c10 2023-10-06T10:41:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.883+0000 7f74485e9b80 1 -- 172.21.15.92:0/4124813118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561665758c10 msgr2=0x561665737970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.883+0000 7f74485e9b80 1 --2- 172.21.15.92:0/4124813118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561665758c10 0x561665737970 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7428009d20 tx=0x7f74280091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.883+0000 7f74485e9b80 1 -- 172.21.15.92:0/4124813118 shutdown_connections 2023-10-06T10:41:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.883+0000 7f74485e9b80 1 --2- 172.21.15.92:0/4124813118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561665737eb0 0x56166570a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.883+0000 7f74485e9b80 1 --2- 172.21.15.92:0/4124813118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56166570e9a0 0x56166570ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.883+0000 7f74485e9b80 1 --2- 172.21.15.92:0/4124813118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561665758c10 0x561665737970 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.883+0000 7f74485e9b80 1 -- 172.21.15.92:0/4124813118 >> 172.21.15.92:0/4124813118 conn(0x5616657102c0 msgr2=0x561665710690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.884+0000 7f74485e9b80 1 -- 172.21.15.92:0/4124813118 shutdown_connections 2023-10-06T10:41:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.884+0000 7f74485e9b80 1 -- 172.21.15.92:0/4124813118 wait complete. 2023-10-06T10:41:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.884+0000 7f74485e9b80 1 -- start start 2023-10-06T10:41:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.884+0000 7f74485e9b80 1 Processor -- start 2023-10-06T10:41:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.885+0000 7f74485e9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56166570e9a0 0x56166574f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.885+0000 7f74485e9b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561665737eb0 0x561665768310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.885+0000 7f74485e9b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561665758c10 0x561665768850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.885+0000 7f74485e9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166565f0b0 con 0x56166570e9a0 2023-10-06T10:41:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.885+0000 7f74485e9b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166560a8e0 con 0x561665758c10 2023-10-06T10:41:06.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.885+0000 7f74485e9b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561665625ef0 con 0x561665737eb0 2023-10-06T10:41:06.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.885+0000 7f74485e9b80 10 client.0 _ll_register_callbacks cb 0x5616654fbf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:06.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.885+0000 7f743d08d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56166570e9a0 0x56166574f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:06.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.885+0000 7f743d08d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56166570e9a0 0x56166574f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59640/0 (socket says 172.21.15.92:59640) 2023-10-06T10:41:06.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.885+0000 7f743d08d700 1 -- 172.21.15.92:0/2143764901 learned_addr learned my addr 172.21.15.92:0/2143764901 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:06.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.885+0000 7f743d88e700 1 --2- 172.21.15.92:0/2143764901 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561665758c10 0x561665768850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:06.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.885+0000 7f743c88c700 1 --2- 172.21.15.92:0/2143764901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561665737eb0 0x561665768310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:06.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.885+0000 7f743d08d700 1 -- 172.21.15.92:0/2143764901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561665737eb0 msgr2=0x561665768310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:06.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.885+0000 7f743d08d700 1 --2- 172.21.15.92:0/2143764901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561665737eb0 0x561665768310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.885+0000 7f743d08d700 1 -- 172.21.15.92:0/2143764901 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561665758c10 msgr2=0x561665768850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.885+0000 7f743d08d700 1 --2- 172.21.15.92:0/2143764901 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561665758c10 0x561665768850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.885+0000 7f743d08d700 1 -- 172.21.15.92:0/2143764901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7428009a50 con 0x56166570e9a0 2023-10-06T10:41:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.885+0000 7f743c88c700 1 --2- 172.21.15.92:0/2143764901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561665737eb0 0x561665768310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.885+0000 7f743d88e700 1 --2- 172.21.15.92:0/2143764901 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561665758c10 0x561665768850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.886+0000 7f743d08d700 1 --2- 172.21.15.92:0/2143764901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56166570e9a0 0x56166574f440 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7434007d10 tx=0x7f7434023070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22acafe563edae0f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.886+0000 7f74177fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.886+0000 7f74177fe700 1 -- 172.21.15.92:0/2143764901 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7434007950 con 0x56166570e9a0 2023-10-06T10:41:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.886+0000 7f74177fe700 1 -- 172.21.15.92:0/2143764901 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7434021440 con 0x56166570e9a0 2023-10-06T10:41:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.886+0000 7f74485e9b80 1 -- 172.21.15.92:0/2143764901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616657036e0 con 0x56166570e9a0 2023-10-06T10:41:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.886+0000 7f74177fe700 1 -- 172.21.15.92:0/2143764901 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7434021640 con 0x56166570e9a0 2023-10-06T10:41:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.886+0000 7f74485e9b80 10 client.6009 fetch_fsmap learned FSMap version 39 2023-10-06T10:41:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.886+0000 7f74485e9b80 1 -- 172.21.15.92:0/2143764901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x56166549c030 con 0x56166570e9a0 2023-10-06T10:41:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.886+0000 7f74177fe700 1 -- 172.21.15.92:0/2143764901 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5616657036e0 con 0x56166570e9a0 2023-10-06T10:41:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.886+0000 7f74177fe700 1 -- 172.21.15.92:0/2143764901 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 39) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f74340217a0 con 0x56166570e9a0 2023-10-06T10:41:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.886+0000 7f74485e9b80 10 client.6009 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T10:41:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.886+0000 7f74485e9b80 10 client.6009 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-10-06T10:41:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.886+0000 7f74485e9b80 20 client.6009 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.886+0000 7f74485e9b80 1 --2- 172.21.15.92:0/2143764901 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x561665752f00 0x5616657340a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.886+0000 7f74485e9b80 4 client.6009 mds_command: new command op to 5736 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:41:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.886+0000 7f74485e9b80 1 -- 172.21.15.92:0/2143764901 --> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616656e0610 con 0x561665752f00 2023-10-06T10:41:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.886+0000 7f743c88c700 1 --2- 172.21.15.92:0/2143764901 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x561665752f00 0x5616657340a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.887+0000 7f743c88c700 1 --2- 172.21.15.92:0/2143764901 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x561665752f00 0x5616657340a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9566d17e4ceeaedc server_cookie=d0ae148866d72f80 in_seq=0 out_seq=0 2023-10-06T10:41:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.887+0000 7f74177fe700 10 client.6009 ms_handle_connect on v2:172.21.15.92:6835/2458340622 2023-10-06T10:41:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.887+0000 7f74177fe700 1 -- 172.21.15.92:0/2143764901 <== mds.0 v2:172.21.15.92:6835/2458340622 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5616656e0610 con 0x561665752f00 2023-10-06T10:41:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.887+0000 7f74177fe700 10 client.6009 handle_command_reply: tid=0 2023-10-06T10:41:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.887+0000 7f74485e9b80 1 client.6009 shutdown 2023-10-06T10:41:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.887+0000 7f74485e9b80 1 -- 172.21.15.92:0/2143764901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56166570e9a0 msgr2=0x56166574f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.887+0000 7f74485e9b80 1 --2- 172.21.15.92:0/2143764901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56166570e9a0 0x56166574f440 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7434007d10 tx=0x7f7434023070 comp rx=0 tx=0).stop 2023-10-06T10:41:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.887+0000 7f74485e9b80 1 -- 172.21.15.92:0/2143764901 shutdown_connections 2023-10-06T10:41:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.887+0000 7f74485e9b80 1 --2- 172.21.15.92:0/2143764901 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x561665752f00 0x5616657340a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.887+0000 7f74485e9b80 1 --2- 172.21.15.92:0/2143764901 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561665758c10 0x561665768850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.887+0000 7f74485e9b80 1 --2- 172.21.15.92:0/2143764901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56166570e9a0 0x56166574f440 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.887+0000 7f74485e9b80 1 --2- 172.21.15.92:0/2143764901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561665737eb0 0x561665768310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.887+0000 7f74177fe700 0 client.6009 ms_handle_reset on v2:172.21.15.92:6835/2458340622 2023-10-06T10:41:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.887+0000 7f74485e9b80 1 -- 172.21.15.92:0/2143764901 >> 172.21.15.92:0/2143764901 conn(0x5616657102c0 msgr2=0x5616657051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.887+0000 7f74485e9b80 1 -- 172.21.15.92:0/2143764901 shutdown_connections 2023-10-06T10:41:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.887+0000 7f74485e9b80 1 -- 172.21.15.92:0/2143764901 wait complete. 2023-10-06T10:41:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.888+0000 7f74485e9b80 20 client.6009 trim_cache size 0 max 16384 2023-10-06T10:41:06.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.901+0000 7f74485e9b80 1 Processor -- start 2023-10-06T10:41:06.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.901+0000 7f74485e9b80 1 -- start start 2023-10-06T10:41:06.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.902+0000 7f74485e9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56166570e9a0 0x561665734a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:06.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.902+0000 7f74485e9b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561665737eb0 0x561665764410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:06.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.902+0000 7f74485e9b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561665758c10 0x5616657051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:06.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.902+0000 7f74485e9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616654e1e80 con 0x56166570e9a0 2023-10-06T10:41:06.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.902+0000 7f74485e9b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166570bbc0 con 0x561665737eb0 2023-10-06T10:41:06.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.902+0000 7f74485e9b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561665705f50 con 0x561665758c10 2023-10-06T10:41:06.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.902+0000 7f743d08d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56166570e9a0 0x561665734a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:06.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.902+0000 7f743d08d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56166570e9a0 0x561665734a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59646/0 (socket says 172.21.15.92:59646) 2023-10-06T10:41:06.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.902+0000 7f743d08d700 1 -- 172.21.15.92:0/3801774099 learned_addr learned my addr 172.21.15.92:0/3801774099 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:06.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.902+0000 7f743d88e700 1 --2- 172.21.15.92:0/3801774099 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561665758c10 0x5616657051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:06.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.902+0000 7f743d08d700 1 -- 172.21.15.92:0/3801774099 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561665758c10 msgr2=0x5616657051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.902+0000 7f743d08d700 1 --2- 172.21.15.92:0/3801774099 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561665758c10 0x5616657051f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.902+0000 7f743c88c700 1 --2- 172.21.15.92:0/3801774099 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561665737eb0 0x561665764410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.902+0000 7f743d08d700 1 -- 172.21.15.92:0/3801774099 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561665737eb0 msgr2=0x561665764410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.902+0000 7f743d08d700 1 --2- 172.21.15.92:0/3801774099 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561665737eb0 0x561665764410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.902+0000 7f743d08d700 1 -- 172.21.15.92:0/3801774099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56166562d7d0 con 0x56166570e9a0 2023-10-06T10:41:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.902+0000 7f743c88c700 1 --2- 172.21.15.92:0/3801774099 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561665737eb0 0x561665764410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.903+0000 7f743d08d700 1 --2- 172.21.15.92:0/3801774099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56166570e9a0 0x561665734a70 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7434001620 tx=0x7f7434030070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcdae04fd839bea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.903+0000 7f7422ffd700 1 -- 172.21.15.92:0/3801774099 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74340075c0 con 0x56166570e9a0 2023-10-06T10:41:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.903+0000 7f7422ffd700 1 -- 172.21.15.92:0/3801774099 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7434021d00 con 0x56166570e9a0 2023-10-06T10:41:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.903+0000 7f7422ffd700 1 -- 172.21.15.92:0/3801774099 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7434007950 con 0x56166570e9a0 2023-10-06T10:41:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.903+0000 7f74485e9b80 1 -- 172.21.15.92:0/3801774099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56166570e9a0 msgr2=0x561665734a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.903+0000 7f74485e9b80 1 --2- 172.21.15.92:0/3801774099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56166570e9a0 0x561665734a70 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7434001620 tx=0x7f7434030070 comp rx=0 tx=0).stop 2023-10-06T10:41:06.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.903+0000 7f74485e9b80 1 -- 172.21.15.92:0/3801774099 shutdown_connections 2023-10-06T10:41:06.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.903+0000 7f74485e9b80 1 --2- 172.21.15.92:0/3801774099 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561665758c10 0x5616657051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.903+0000 7f74485e9b80 1 --2- 172.21.15.92:0/3801774099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56166570e9a0 0x561665734a70 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.903+0000 7f74485e9b80 1 --2- 172.21.15.92:0/3801774099 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561665737eb0 0x561665764410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.903+0000 7f74485e9b80 1 -- 172.21.15.92:0/3801774099 >> 172.21.15.92:0/3801774099 conn(0x5616657102c0 msgr2=0x561665710690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:06.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.903+0000 7f74485e9b80 1 -- 172.21.15.92:0/3801774099 shutdown_connections 2023-10-06T10:41:06.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.903+0000 7f74485e9b80 1 -- 172.21.15.92:0/3801774099 wait complete. 2023-10-06T10:41:06.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.904+0000 7f74485e9b80 1 -- start start 2023-10-06T10:41:06.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.904+0000 7f74485e9b80 1 Processor -- start 2023-10-06T10:41:06.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f74485e9b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56166570e9a0 0x5616657b7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:06.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f74485e9b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561665737eb0 0x561665754f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:06.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f74485e9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561665758c10 0x5616657051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:06.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f74485e9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616657564b0 con 0x561665758c10 2023-10-06T10:41:06.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f74485e9b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561665755590 con 0x56166570e9a0 2023-10-06T10:41:06.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f74485e9b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616657bb830 con 0x561665737eb0 2023-10-06T10:41:06.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f74485e9b80 10 client.0 _ll_register_callbacks cb 0x5616654fbf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:06.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f743d88e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561665758c10 0x5616657051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:06.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f743c88c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561665737eb0 0x561665754f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:06.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f743d08d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56166570e9a0 0x5616657b7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:06.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f743d88e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561665758c10 0x5616657051f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59648/0 (socket says 172.21.15.92:59648) 2023-10-06T10:41:06.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f743d88e700 1 -- 172.21.15.92:0/3170580315 learned_addr learned my addr 172.21.15.92:0/3170580315 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:06.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f743d88e700 1 -- 172.21.15.92:0/3170580315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561665737eb0 msgr2=0x561665754f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:06.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f743d88e700 1 --2- 172.21.15.92:0/3170580315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561665737eb0 0x561665754f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f743d88e700 1 -- 172.21.15.92:0/3170580315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56166570e9a0 msgr2=0x5616657b7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:06.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f743d88e700 1 --2- 172.21.15.92:0/3170580315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56166570e9a0 0x5616657b7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f743d88e700 1 -- 172.21.15.92:0/3170580315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56166549c030 con 0x561665758c10 2023-10-06T10:41:06.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f743d08d700 1 --2- 172.21.15.92:0/3170580315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56166570e9a0 0x5616657b7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:41:06.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f743d88e700 1 --2- 172.21.15.92:0/3170580315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561665758c10 0x5616657051f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f743000fe40 tx=0x7f7430009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd3c73b742edf2c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:06.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f74177fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:06.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f74177fe700 1 -- 172.21.15.92:0/3170580315 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743005bae0 con 0x561665758c10 2023-10-06T10:41:06.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f74177fe700 1 -- 172.21.15.92:0/3170580315 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7430007690 con 0x561665758c10 2023-10-06T10:41:06.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.905+0000 7f74485e9b80 1 -- 172.21.15.92:0/3170580315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56166570c1d0 con 0x561665758c10 2023-10-06T10:41:06.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.906+0000 7f74177fe700 1 -- 172.21.15.92:0/3170580315 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74300542f0 con 0x561665758c10 2023-10-06T10:41:06.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.906+0000 7f74485e9b80 10 client.6015 fetch_fsmap learned FSMap version 39 2023-10-06T10:41:06.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.906+0000 7f74485e9b80 1 -- 172.21.15.92:0/3170580315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x5616656144c0 con 0x561665758c10 2023-10-06T10:41:06.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.906+0000 7f74177fe700 1 -- 172.21.15.92:0/3170580315 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743005f020 con 0x561665758c10 2023-10-06T10:41:06.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.906+0000 7f74177fe700 1 -- 172.21.15.92:0/3170580315 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 39) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f743005b650 con 0x561665758c10 2023-10-06T10:41:06.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.906+0000 7f74485e9b80 10 client.6015 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T10:41:06.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.906+0000 7f74485e9b80 10 client.6015 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-10-06T10:41:06.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.906+0000 7f74485e9b80 20 client.6015 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:06.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.906+0000 7f74485e9b80 1 --2- 172.21.15.92:0/3170580315 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x5616657bb410 0x561665705730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:06.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.906+0000 7f74485e9b80 4 client.6015 mds_command: new command op to 5736 tid=0[{"prefix": "flush journal"}] 2023-10-06T10:41:06.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.906+0000 7f74485e9b80 1 -- 172.21.15.92:0/3170580315 --> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f7428014c10 con 0x5616657bb410 2023-10-06T10:41:06.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.906+0000 7f743d08d700 1 --2- 172.21.15.92:0/3170580315 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x5616657bb410 0x561665705730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:06.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.906+0000 7f74177fe700 10 client.6015 ms_handle_connect on v2:172.21.15.92:6835/2458340622 2023-10-06T10:41:06.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.906+0000 7f743d08d700 1 --2- 172.21.15.92:0/3170580315 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x5616657bb410 0x561665705730 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a527edcb2c3c5447 server_cookie=3833000857c318e7 in_seq=0 out_seq=0 2023-10-06T10:41:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.922+0000 7f74177fe700 1 -- 172.21.15.92:0/3170580315 <== mds.0 v2:172.21.15.92:6835/2458340622 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f7428014c10 con 0x5616657bb410 2023-10-06T10:41:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.922+0000 7f74177fe700 10 client.6015 handle_command_reply: tid=0 2023-10-06T10:41:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.922+0000 7f74485e9b80 1 client.6015 shutdown 2023-10-06T10:41:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.922+0000 7f74485e9b80 1 -- 172.21.15.92:0/3170580315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561665758c10 msgr2=0x5616657051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.922+0000 7f74485e9b80 1 --2- 172.21.15.92:0/3170580315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561665758c10 0x5616657051f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f743000fe40 tx=0x7f7430009040 comp rx=0 tx=0).stop 2023-10-06T10:41:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.922+0000 7f74485e9b80 1 -- 172.21.15.92:0/3170580315 shutdown_connections 2023-10-06T10:41:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.922+0000 7f74485e9b80 1 --2- 172.21.15.92:0/3170580315 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x5616657bb410 0x561665705730 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.922+0000 7f74485e9b80 1 --2- 172.21.15.92:0/3170580315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561665758c10 0x5616657051f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.922+0000 7f74485e9b80 1 --2- 172.21.15.92:0/3170580315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56166570e9a0 0x5616657b7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.922+0000 7f74177fe700 0 client.6015 ms_handle_reset on v2:172.21.15.92:6835/2458340622 2023-10-06T10:41:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.922+0000 7f74485e9b80 1 --2- 172.21.15.92:0/3170580315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561665737eb0 0x561665754f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.922+0000 7f74485e9b80 1 -- 172.21.15.92:0/3170580315 >> 172.21.15.92:0/3170580315 conn(0x5616657102c0 msgr2=0x561665734a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.922+0000 7f74485e9b80 1 -- 172.21.15.92:0/3170580315 shutdown_connections 2023-10-06T10:41:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.922+0000 7f74485e9b80 1 -- 172.21.15.92:0/3170580315 wait complete. 2023-10-06T10:41:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.923+0000 7f74485e9b80 20 client.6015 trim_cache size 0 max 16384 2023-10-06T10:41:06.927 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:41:06.927 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2023-10-06T10:41:06.927 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:41:06.927 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:41:06.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.923+0000 7f743e08f700 1 -- 172.21.15.92:0/570134605 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f741c066f30 msgr2=0x7f741c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:06.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.923+0000 7f743e08f700 1 --2- 172.21.15.92:0/570134605 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f741c066f30 0x7f741c0693e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7434001270 tx=0x7f7434015000 comp rx=0 tx=0).stop 2023-10-06T10:41:06.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.923+0000 7f743e08f700 1 -- 172.21.15.92:0/570134605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7438156c50 msgr2=0x7f74381570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:06.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.923+0000 7f743e08f700 1 --2- 172.21.15.92:0/570134605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7438156c50 0x7f74381570d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f74380b2e10 tx=0x7f7430007d30 comp rx=0 tx=0).stop 2023-10-06T10:41:06.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.923+0000 7f743e08f700 1 -- 172.21.15.92:0/570134605 shutdown_connections 2023-10-06T10:41:06.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.923+0000 7f743e08f700 1 --2- 172.21.15.92:0/570134605 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f741c066f30 0x7f741c0693e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.923+0000 7f743e08f700 1 --2- 172.21.15.92:0/570134605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7438156c50 0x7f74381570d0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.923+0000 7f743e08f700 1 --2- 172.21.15.92:0/570134605 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7424003a90 0x7f7438146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.923+0000 7f743e08f700 1 --2- 172.21.15.92:0/570134605 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74380c7560 0x7f74380c9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:06.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.923+0000 7f743e08f700 1 -- 172.21.15.92:0/570134605 >> 172.21.15.92:0/570134605 conn(0x7f7438099520 msgr2=0x7f743814d590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:06.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.923+0000 7f743e08f700 1 -- 172.21.15.92:0/570134605 shutdown_connections 2023-10-06T10:41:06.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:06.923+0000 7f743e08f700 1 -- 172.21.15.92:0/570134605 wait complete. 2023-10-06T10:41:06.937 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:1 flush journal 2023-10-06T10:41:07.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.050+0000 7effd36fe700 1 Processor -- start 2023-10-06T10:41:07.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.050+0000 7effd36fe700 1 -- start start 2023-10-06T10:41:07.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.051+0000 7effd36fe700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effcc067800 0x7effcc1584d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.051+0000 7effd36fe700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7effcc1405f0 0x7effcc140a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.051+0000 7effd36fe700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effcc157a60 con 0x7effcc067bd0 2023-10-06T10:41:07.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.051+0000 7effd36fe700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effcc0c9930 con 0x7effcc1405f0 2023-10-06T10:41:07.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.051+0000 7effd36fe700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effcc15ba80 con 0x7effcc067800 2023-10-06T10:41:07.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.051+0000 7effd149a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effcc067800 0x7effcc1584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.051+0000 7effd1c9b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7effcc1405f0 0x7effcc140a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.051+0000 7effd149a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effcc067800 0x7effcc1584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:54512/0 (socket says 172.21.15.92:54512) 2023-10-06T10:41:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.051+0000 7effd149a700 1 -- 172.21.15.92:0/1276280923 learned_addr learned my addr 172.21.15.92:0/1276280923 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.051+0000 7effd0c99700 1 --1- 172.21.15.92:0/1276280923 >> v1:172.21.15.124:6789/0 conn(0x7effcc067bd0 0x7effcc15af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:53702/0 (socket says 172.21.15.92:53702) 2023-10-06T10:41:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.052+0000 7effd1c9b700 1 -- 172.21.15.92:0/1276280923 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effcc067800 msgr2=0x7effcc1584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.052+0000 7effd1c9b700 1 --2- 172.21.15.92:0/1276280923 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effcc067800 0x7effcc1584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.052+0000 7effd1c9b700 1 -- 172.21.15.92:0/1276280923 >> v1:172.21.15.124:6789/0 conn(0x7effcc067bd0 legacy=0x7effcc15af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.052+0000 7effd1c9b700 1 -- 172.21.15.92:0/1276280923 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effcc157cd0 con 0x7effcc1405f0 2023-10-06T10:41:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.052+0000 7effd149a700 1 --2- 172.21.15.92:0/1276280923 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effcc067800 0x7effcc1584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:41:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.052+0000 7effd1c9b700 1 --2- 172.21.15.92:0/1276280923 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effcc1405f0 0x7effcc140a70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7effc4009910 tx=0x7effc40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2f719754d77934b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.052+0000 7effc3fff700 1 -- 172.21.15.92:0/1276280923 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc400b640 con 0x7effcc1405f0 2023-10-06T10:41:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.052+0000 7effc3fff700 1 -- 172.21.15.92:0/1276280923 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc400b7a0 con 0x7effcc1405f0 2023-10-06T10:41:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.052+0000 7effc3fff700 1 -- 172.21.15.92:0/1276280923 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc400bb60 con 0x7effcc1405f0 2023-10-06T10:41:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.052+0000 7effd36fe700 1 -- 172.21.15.92:0/1276280923 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effcc1405f0 msgr2=0x7effcc140a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.052+0000 7effd36fe700 1 --2- 172.21.15.92:0/1276280923 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effcc1405f0 0x7effcc140a70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7effc4009910 tx=0x7effc40091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.052+0000 7effd36fe700 1 -- 172.21.15.92:0/1276280923 shutdown_connections 2023-10-06T10:41:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.052+0000 7effd36fe700 1 --2- 172.21.15.92:0/1276280923 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effcc1405f0 0x7effcc140a70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.052+0000 7effd36fe700 1 --2- 172.21.15.92:0/1276280923 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effcc067800 0x7effcc1584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.052+0000 7effd36fe700 1 -- 172.21.15.92:0/1276280923 >> 172.21.15.92:0/1276280923 conn(0x7effcc0cd1a0 msgr2=0x7effcc0cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.053+0000 7effd36fe700 1 -- 172.21.15.92:0/1276280923 shutdown_connections 2023-10-06T10:41:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.053+0000 7effd36fe700 1 -- 172.21.15.92:0/1276280923 wait complete. 2023-10-06T10:41:07.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.054+0000 7effd36fe700 1 Processor -- start 2023-10-06T10:41:07.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.054+0000 7effd36fe700 1 -- start start 2023-10-06T10:41:07.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.054+0000 7effd36fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effcc067800 0x7effcc0c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.054+0000 7effd36fe700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effcc067bd0 0x7effcc13cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.054+0000 7effd36fe700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effcc1405f0 0x7effcc0cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.054+0000 7effd36fe700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc13d320 con 0x7effcc067800 2023-10-06T10:41:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.054+0000 7effd36fe700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc0c4250 con 0x7effcc067bd0 2023-10-06T10:41:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.054+0000 7effd36fe700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effcc0c4af0 con 0x7effcc1405f0 2023-10-06T10:41:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.055+0000 7effd149a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effcc067800 0x7effcc0c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.055+0000 7effd0c99700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effcc067bd0 0x7effcc13cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.055+0000 7effd1c9b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effcc1405f0 0x7effcc0cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.055+0000 7effd149a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effcc067800 0x7effcc0c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59656/0 (socket says 172.21.15.92:59656) 2023-10-06T10:41:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.055+0000 7effd149a700 1 -- 172.21.15.92:0/3175561749 learned_addr learned my addr 172.21.15.92:0/3175561749 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:07.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.055+0000 7effd0c99700 1 -- 172.21.15.92:0/3175561749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effcc1405f0 msgr2=0x7effcc0cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:07.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.055+0000 7effd0c99700 1 --2- 172.21.15.92:0/3175561749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effcc1405f0 0x7effcc0cb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.055+0000 7effd0c99700 1 -- 172.21.15.92:0/3175561749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effcc067800 msgr2=0x7effcc0c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:07.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.055+0000 7effd0c99700 1 --2- 172.21.15.92:0/3175561749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effcc067800 0x7effcc0c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.055+0000 7effd0c99700 1 -- 172.21.15.92:0/3175561749 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc8009ae0 con 0x7effcc067bd0 2023-10-06T10:41:07.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.055+0000 7effd149a700 1 --2- 172.21.15.92:0/3175561749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effcc067800 0x7effcc0c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:41:07.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.055+0000 7effd1c9b700 1 --2- 172.21.15.92:0/3175561749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effcc1405f0 0x7effcc0cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:41:07.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.055+0000 7effd0c99700 1 --2- 172.21.15.92:0/3175561749 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effcc067bd0 0x7effcc13cde0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7effbc00c7c0 tx=0x7effbc007eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:07.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.055+0000 7effc17fa700 1 -- 172.21.15.92:0/3175561749 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc005a70 con 0x7effcc067bd0 2023-10-06T10:41:07.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.055+0000 7effc17fa700 1 -- 172.21.15.92:0/3175561749 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effbc005bd0 con 0x7effcc067bd0 2023-10-06T10:41:07.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.055+0000 7effd36fe700 1 -- 172.21.15.92:0/3175561749 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc4009420 con 0x7effcc067bd0 2023-10-06T10:41:07.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.055+0000 7effc17fa700 1 -- 172.21.15.92:0/3175561749 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc0070d0 con 0x7effcc067bd0 2023-10-06T10:41:07.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.055+0000 7effd36fe700 1 -- 172.21.15.92:0/3175561749 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effcc0c44a0 con 0x7effcc067bd0 2023-10-06T10:41:07.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.057+0000 7effc17fa700 1 -- 172.21.15.92:0/3175561749 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7effbc007480 con 0x7effcc067bd0 2023-10-06T10:41:07.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.057+0000 7effc17fa700 1 --2- 172.21.15.92:0/3175561749 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7effb0066f30 0x7effb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.057+0000 7effc17fa700 1 -- 172.21.15.92:0/3175561749 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7effbc083bc0 con 0x7effcc067bd0 2023-10-06T10:41:07.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.057+0000 7effd149a700 1 --2- 172.21.15.92:0/3175561749 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7effb0066f30 0x7effb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.057+0000 7effd149a700 1 --2- 172.21.15.92:0/3175561749 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7effb0066f30 0x7effb00693e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7effc800eb00 tx=0x7effc800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:07.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.059+0000 7effdc9f6b80 1 Processor -- start 2023-10-06T10:41:07.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.059+0000 7effdc9f6b80 1 -- start start 2023-10-06T10:41:07.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.059+0000 7effdc9f6b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599c32c9ae0 0x5599c32c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.060+0000 7effdc9f6b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599c32859f0 0x5599c32825d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.060+0000 7effdc9f6b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c3283c20 0x5599c3282b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.060+0000 7effdc9f6b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c3202cd0 con 0x5599c3283c20 2023-10-06T10:41:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.060+0000 7effdc9f6b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c31d9ac0 con 0x5599c32c9ae0 2023-10-06T10:41:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.060+0000 7effdc9f6b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599c31d72c0 con 0x5599c32859f0 2023-10-06T10:41:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.060+0000 7effd1c9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c3283c20 0x5599c3282b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.060+0000 7effd149a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599c32c9ae0 0x5599c32c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.060+0000 7effd1c9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c3283c20 0x5599c3282b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59664/0 (socket says 172.21.15.92:59664) 2023-10-06T10:41:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.060+0000 7effd1c9b700 1 -- 172.21.15.92:0/3242120352 learned_addr learned my addr 172.21.15.92:0/3242120352 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.060+0000 7effd0c99700 1 --2- 172.21.15.92:0/3242120352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599c32859f0 0x5599c32825d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.060+0000 7effd1c9b700 1 -- 172.21.15.92:0/3242120352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599c32859f0 msgr2=0x5599c32825d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:07.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.060+0000 7effd1c9b700 1 --2- 172.21.15.92:0/3242120352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599c32859f0 0x5599c32825d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.060+0000 7effd1c9b700 1 -- 172.21.15.92:0/3242120352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599c32c9ae0 msgr2=0x5599c32c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:07.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.060+0000 7effd1c9b700 1 --2- 172.21.15.92:0/3242120352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599c32c9ae0 0x5599c32c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.060+0000 7effd1c9b700 1 -- 172.21.15.92:0/3242120352 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599c306d2c0 con 0x5599c3283c20 2023-10-06T10:41:07.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.060+0000 7effd1c9b700 1 --2- 172.21.15.92:0/3242120352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c3283c20 0x5599c3282b10 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7effcc06d340 tx=0x7effc4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3977f1dafda408e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:07.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.060+0000 7effb67fc700 1 -- 172.21.15.92:0/3242120352 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc400bed0 con 0x5599c3283c20 2023-10-06T10:41:07.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.060+0000 7effb67fc700 1 -- 172.21.15.92:0/3242120352 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc4020040 con 0x5599c3283c20 2023-10-06T10:41:07.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.060+0000 7effb67fc700 1 -- 172.21.15.92:0/3242120352 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc400bb20 con 0x5599c3283c20 2023-10-06T10:41:07.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.061+0000 7effdc9f6b80 1 -- 172.21.15.92:0/3242120352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c3283c20 msgr2=0x5599c3282b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:07.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.061+0000 7effdc9f6b80 1 --2- 172.21.15.92:0/3242120352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c3283c20 0x5599c3282b10 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7effcc06d340 tx=0x7effc4017000 comp rx=0 tx=0).stop 2023-10-06T10:41:07.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.061+0000 7effdc9f6b80 1 -- 172.21.15.92:0/3242120352 shutdown_connections 2023-10-06T10:41:07.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.061+0000 7effdc9f6b80 1 --2- 172.21.15.92:0/3242120352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c3283c20 0x5599c3282b10 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.061+0000 7effdc9f6b80 1 --2- 172.21.15.92:0/3242120352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599c32c9ae0 0x5599c32c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.061+0000 7effdc9f6b80 1 --2- 172.21.15.92:0/3242120352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599c32859f0 0x5599c32825d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.061+0000 7effdc9f6b80 1 -- 172.21.15.92:0/3242120352 >> 172.21.15.92:0/3242120352 conn(0x5599c32840c0 msgr2=0x5599c3284490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:07.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.061+0000 7effdc9f6b80 1 -- 172.21.15.92:0/3242120352 shutdown_connections 2023-10-06T10:41:07.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.061+0000 7effdc9f6b80 1 -- 172.21.15.92:0/3242120352 wait complete. 2023-10-06T10:41:07.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.061+0000 7effdc9f6b80 1 -- start start 2023-10-06T10:41:07.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.062+0000 7effdc9f6b80 1 Processor -- start 2023-10-06T10:41:07.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.062+0000 7effdc9f6b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599c3283c20 0x5599c32a93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.062+0000 7effdc9f6b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c32859f0 0x5599c32c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.062+0000 7effdc9f6b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599c32c9ae0 0x5599c32d8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.062+0000 7effdc9f6b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c31d30b0 con 0x5599c32859f0 2023-10-06T10:41:07.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.062+0000 7effdc9f6b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c317e8e0 con 0x5599c32c9ae0 2023-10-06T10:41:07.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.062+0000 7effdc9f6b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599c3199ef0 con 0x5599c3283c20 2023-10-06T10:41:07.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.062+0000 7effdc9f6b80 10 client.0 _ll_register_callbacks cb 0x5599c306ff80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:07.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.062+0000 7effd0c99700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c32859f0 0x5599c32c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.062+0000 7effd0c99700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c32859f0 0x5599c32c6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59672/0 (socket says 172.21.15.92:59672) 2023-10-06T10:41:07.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.062+0000 7effd0c99700 1 -- 172.21.15.92:0/1410518586 learned_addr learned my addr 172.21.15.92:0/1410518586 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:07.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.062+0000 7effd149a700 1 --2- 172.21.15.92:0/1410518586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599c3283c20 0x5599c32a93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.063+0000 7effd1c9b700 1 --2- 172.21.15.92:0/1410518586 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599c32c9ae0 0x5599c32d8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.063+0000 7effd0c99700 1 -- 172.21.15.92:0/1410518586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599c3283c20 msgr2=0x5599c32a93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:07.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.063+0000 7effd0c99700 1 --2- 172.21.15.92:0/1410518586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599c3283c20 0x5599c32a93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.063+0000 7effd0c99700 1 -- 172.21.15.92:0/1410518586 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599c32c9ae0 msgr2=0x5599c32d8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:07.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.063+0000 7effd0c99700 1 --2- 172.21.15.92:0/1410518586 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599c32c9ae0 0x5599c32d8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.063+0000 7effd0c99700 1 -- 172.21.15.92:0/1410518586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effcc157cd0 con 0x5599c32859f0 2023-10-06T10:41:07.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.063+0000 7effd0c99700 1 --2- 172.21.15.92:0/1410518586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c32859f0 0x5599c32c6b40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7effbc00cdb0 tx=0x7effbc081da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a39a382d3560be4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:07.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.063+0000 7effab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:07.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.063+0000 7effab7fe700 1 -- 172.21.15.92:0/1410518586 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc0548d0 con 0x5599c32859f0 2023-10-06T10:41:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.063+0000 7effab7fe700 1 -- 172.21.15.92:0/1410518586 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7effbc057aa0 con 0x5599c32859f0 2023-10-06T10:41:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.063+0000 7effdc9f6b80 1 -- 172.21.15.92:0/1410518586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599c32776e0 con 0x5599c32859f0 2023-10-06T10:41:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.063+0000 7effab7fe700 1 -- 172.21.15.92:0/1410518586 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7effbc05b7f0 con 0x5599c32859f0 2023-10-06T10:41:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.063+0000 7effdc9f6b80 10 client.6024 fetch_fsmap learned FSMap version 39 2023-10-06T10:41:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.063+0000 7effdc9f6b80 1 -- 172.21.15.92:0/1410518586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x5599c3010030 con 0x5599c32859f0 2023-10-06T10:41:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.063+0000 7effab7fe700 1 -- 172.21.15.92:0/1410518586 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5599c32776e0 con 0x5599c32859f0 2023-10-06T10:41:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.063+0000 7effab7fe700 1 -- 172.21.15.92:0/1410518586 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 39) v1 ==== 3136+0+0 (secure 0 0 0) 0x7effbc0702a0 con 0x5599c32859f0 2023-10-06T10:41:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.063+0000 7effdc9f6b80 10 client.6024 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T10:41:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.063+0000 7effdc9f6b80 10 client.6024 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2023-10-06T10:41:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.063+0000 7effdc9f6b80 20 client.6024 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.063+0000 7effdc9f6b80 1 --2- 172.21.15.92:0/1410518586 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x5599c32c50a0 0x5599c32aabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.063+0000 7effdc9f6b80 4 client.6024 mds_command: new command op to 5733 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:41:07.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.063+0000 7effdc9f6b80 1 -- 172.21.15.92:0/1410518586 --> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599c3254610 con 0x5599c32c50a0 2023-10-06T10:41:07.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.064+0000 7effd149a700 1 --2- 172.21.15.92:0/1410518586 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x5599c32c50a0 0x5599c32aabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.064+0000 7effd149a700 1 --2- 172.21.15.92:0/1410518586 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x5599c32c50a0 0x5599c32aabf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2802ce90008411f7 server_cookie=7dc5268b3db85014 in_seq=0 out_seq=0 2023-10-06T10:41:07.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.064+0000 7effab7fe700 10 client.6024 ms_handle_connect on v2:172.21.15.92:6834/831923836 2023-10-06T10:41:07.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.064+0000 7effab7fe700 1 -- 172.21.15.92:0/1410518586 <== mds.1 v2:172.21.15.92:6834/831923836 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5599c3254610 con 0x5599c32c50a0 2023-10-06T10:41:07.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.064+0000 7effab7fe700 10 client.6024 handle_command_reply: tid=0 2023-10-06T10:41:07.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.064+0000 7effdc9f6b80 1 client.6024 shutdown 2023-10-06T10:41:07.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.064+0000 7effdc9f6b80 1 -- 172.21.15.92:0/1410518586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c32859f0 msgr2=0x5599c32c6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:07.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.064+0000 7effdc9f6b80 1 --2- 172.21.15.92:0/1410518586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c32859f0 0x5599c32c6b40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7effbc00cdb0 tx=0x7effbc081da0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.065+0000 7effdc9f6b80 1 -- 172.21.15.92:0/1410518586 shutdown_connections 2023-10-06T10:41:07.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.065+0000 7effdc9f6b80 1 --2- 172.21.15.92:0/1410518586 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599c32c9ae0 0x5599c32d8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.065+0000 7effdc9f6b80 1 --2- 172.21.15.92:0/1410518586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599c3283c20 0x5599c32a93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.065+0000 7effdc9f6b80 1 --2- 172.21.15.92:0/1410518586 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x5599c32c50a0 0x5599c32aabf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.065+0000 7effdc9f6b80 1 --2- 172.21.15.92:0/1410518586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c32859f0 0x5599c32c6b40 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.065+0000 7effab7fe700 0 client.6024 ms_handle_reset on v2:172.21.15.92:6834/831923836 2023-10-06T10:41:07.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.065+0000 7effdc9f6b80 1 -- 172.21.15.92:0/1410518586 >> 172.21.15.92:0/1410518586 conn(0x5599c32840c0 msgr2=0x5599c327f870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:07.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.065+0000 7effdc9f6b80 1 -- 172.21.15.92:0/1410518586 shutdown_connections 2023-10-06T10:41:07.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.065+0000 7effdc9f6b80 1 -- 172.21.15.92:0/1410518586 wait complete. 2023-10-06T10:41:07.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.065+0000 7effdc9f6b80 20 client.6024 trim_cache size 0 max 16384 2023-10-06T10:41:07.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.078+0000 7effdc9f6b80 1 Processor -- start 2023-10-06T10:41:07.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.078+0000 7effdc9f6b80 1 -- start start 2023-10-06T10:41:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.079+0000 7effdc9f6b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599c3283c20 0x5599c32c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.079+0000 7effdc9f6b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599c32859f0 0x5599c32aabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.079+0000 7effdc9f6b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c32c9ae0 0x5599c32ab130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.079+0000 7effdc9f6b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c3055e80 con 0x5599c32c9ae0 2023-10-06T10:41:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.079+0000 7effdc9f6b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c32c6c30 con 0x5599c3283c20 2023-10-06T10:41:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.079+0000 7effdc9f6b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599c32c73d0 con 0x5599c32859f0 2023-10-06T10:41:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.079+0000 7effd1c9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c32c9ae0 0x5599c32ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.079+0000 7effd1c9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c32c9ae0 0x5599c32ab130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59674/0 (socket says 172.21.15.92:59674) 2023-10-06T10:41:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.079+0000 7effd1c9b700 1 -- 172.21.15.92:0/978748192 learned_addr learned my addr 172.21.15.92:0/978748192 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.079+0000 7effd1c9b700 1 -- 172.21.15.92:0/978748192 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599c32859f0 msgr2=0x5599c32aabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.079+0000 7effd1c9b700 1 --2- 172.21.15.92:0/978748192 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599c32859f0 0x5599c32aabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.079+0000 7effd149a700 1 --2- 172.21.15.92:0/978748192 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599c3283c20 0x5599c32c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.079+0000 7effd1c9b700 1 -- 172.21.15.92:0/978748192 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599c3283c20 msgr2=0x5599c32c7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.079+0000 7effd1c9b700 1 --2- 172.21.15.92:0/978748192 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599c3283c20 0x5599c32c7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.079+0000 7effd1c9b700 1 -- 172.21.15.92:0/978748192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599c31a17d0 con 0x5599c32c9ae0 2023-10-06T10:41:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.080+0000 7effd1c9b700 1 --2- 172.21.15.92:0/978748192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c32c9ae0 0x5599c32ab130 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7effc4024000 tx=0x7effc401c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8578e4aaf1f7eee2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.080+0000 7effb6ffd700 1 -- 172.21.15.92:0/978748192 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc401a9f0 con 0x5599c32c9ae0 2023-10-06T10:41:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.080+0000 7effb6ffd700 1 -- 172.21.15.92:0/978748192 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc4033030 con 0x5599c32c9ae0 2023-10-06T10:41:07.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.080+0000 7effb6ffd700 1 -- 172.21.15.92:0/978748192 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc401ad80 con 0x5599c32c9ae0 2023-10-06T10:41:07.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.080+0000 7effdc9f6b80 1 -- 172.21.15.92:0/978748192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c32c9ae0 msgr2=0x5599c32ab130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:07.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.080+0000 7effdc9f6b80 1 --2- 172.21.15.92:0/978748192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c32c9ae0 0x5599c32ab130 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7effc4024000 tx=0x7effc401c070 comp rx=0 tx=0).stop 2023-10-06T10:41:07.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.080+0000 7effdc9f6b80 1 -- 172.21.15.92:0/978748192 shutdown_connections 2023-10-06T10:41:07.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.080+0000 7effdc9f6b80 1 --2- 172.21.15.92:0/978748192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c32c9ae0 0x5599c32ab130 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.080+0000 7effdc9f6b80 1 --2- 172.21.15.92:0/978748192 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599c3283c20 0x5599c32c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.080+0000 7effdc9f6b80 1 --2- 172.21.15.92:0/978748192 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599c32859f0 0x5599c32aabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.080+0000 7effdc9f6b80 1 -- 172.21.15.92:0/978748192 >> 172.21.15.92:0/978748192 conn(0x5599c32840c0 msgr2=0x5599c32e2830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:07.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.080+0000 7effdc9f6b80 1 -- 172.21.15.92:0/978748192 shutdown_connections 2023-10-06T10:41:07.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.080+0000 7effdc9f6b80 1 -- 172.21.15.92:0/978748192 wait complete. 2023-10-06T10:41:07.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.081+0000 7effdc9f6b80 1 -- start start 2023-10-06T10:41:07.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.081+0000 7effdc9f6b80 1 Processor -- start 2023-10-06T10:41:07.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effdc9f6b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c3283c20 0x5599c33318c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effdc9f6b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599c32859f0 0x5599c32c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effdc9f6b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599c32c9ae0 0x5599c32c3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effdc9f6b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c32aeb30 con 0x5599c3283c20 2023-10-06T10:41:07.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effdc9f6b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c32ad080 con 0x5599c32c9ae0 2023-10-06T10:41:07.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effdc9f6b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599c32c8d10 con 0x5599c32859f0 2023-10-06T10:41:07.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effdc9f6b80 10 client.0 _ll_register_callbacks cb 0x5599c306ff80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:07.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effd149a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c3283c20 0x5599c33318c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effd149a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c3283c20 0x5599c33318c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59690/0 (socket says 172.21.15.92:59690) 2023-10-06T10:41:07.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effd149a700 1 -- 172.21.15.92:0/1497118831 learned_addr learned my addr 172.21.15.92:0/1497118831 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:07.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effd0c99700 1 --2- 172.21.15.92:0/1497118831 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599c32859f0 0x5599c32c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effd1c9b700 1 --2- 172.21.15.92:0/1497118831 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599c32c9ae0 0x5599c32c3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effd149a700 1 -- 172.21.15.92:0/1497118831 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599c32859f0 msgr2=0x5599c32c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:07.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effd149a700 1 --2- 172.21.15.92:0/1497118831 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599c32859f0 0x5599c32c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effd149a700 1 -- 172.21.15.92:0/1497118831 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599c32c9ae0 msgr2=0x5599c32c3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:07.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effd149a700 1 --2- 172.21.15.92:0/1497118831 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599c32c9ae0 0x5599c32c3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effd149a700 1 -- 172.21.15.92:0/1497118831 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599c306d2c0 con 0x5599c3283c20 2023-10-06T10:41:07.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effd0c99700 1 --2- 172.21.15.92:0/1497118831 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599c32859f0 0x5599c32c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:07.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effd1c9b700 1 --2- 172.21.15.92:0/1497118831 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599c32c9ae0 0x5599c32c3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:07.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effd149a700 1 --2- 172.21.15.92:0/1497118831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c3283c20 0x5599c33318c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7effc8002930 tx=0x7effc8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ea87cbaf490c7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:07.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:07.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effab7fe700 1 -- 172.21.15.92:0/1497118831 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc8029040 con 0x5599c3283c20 2023-10-06T10:41:07.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effab7fe700 1 -- 172.21.15.92:0/1497118831 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7effc8036d60 con 0x5599c3283c20 2023-10-06T10:41:07.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effdc9f6b80 1 -- 172.21.15.92:0/1497118831 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599c32c0820 con 0x5599c3283c20 2023-10-06T10:41:07.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effab7fe700 1 -- 172.21.15.92:0/1497118831 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7effc801c040 con 0x5599c3283c20 2023-10-06T10:41:07.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effdc9f6b80 10 client.6030 fetch_fsmap learned FSMap version 39 2023-10-06T10:41:07.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.082+0000 7effdc9f6b80 1 -- 172.21.15.92:0/1497118831 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x5599c31884c0 con 0x5599c3283c20 2023-10-06T10:41:07.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.083+0000 7effab7fe700 1 -- 172.21.15.92:0/1497118831 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5599c32c0820 con 0x5599c3283c20 2023-10-06T10:41:07.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.083+0000 7effab7fe700 1 -- 172.21.15.92:0/1497118831 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 39) v1 ==== 3136+0+0 (secure 0 0 0) 0x7effc8033020 con 0x5599c3283c20 2023-10-06T10:41:07.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.083+0000 7effdc9f6b80 10 client.6030 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T10:41:07.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.083+0000 7effdc9f6b80 10 client.6030 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2023-10-06T10:41:07.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.083+0000 7effdc9f6b80 20 client.6030 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:07.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.083+0000 7effdc9f6b80 1 --2- 172.21.15.92:0/1497118831 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x5599c32c39d0 0x5599c32ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.083+0000 7effdc9f6b80 4 client.6030 mds_command: new command op to 5733 tid=0[{"prefix": "flush journal"}] 2023-10-06T10:41:07.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.083+0000 7effdc9f6b80 1 -- 172.21.15.92:0/1497118831 --> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7effc4020040 con 0x5599c32c39d0 2023-10-06T10:41:07.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.083+0000 7effd0c99700 1 --2- 172.21.15.92:0/1497118831 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x5599c32c39d0 0x5599c32ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.083+0000 7effab7fe700 10 client.6030 ms_handle_connect on v2:172.21.15.92:6834/831923836 2023-10-06T10:41:07.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.083+0000 7effd0c99700 1 --2- 172.21.15.92:0/1497118831 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x5599c32c39d0 0x5599c32ccea0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6413128c3002af5d server_cookie=28dcac5f3bff1dac in_seq=0 out_seq=0 2023-10-06T10:41:07.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.099+0000 7effab7fe700 1 -- 172.21.15.92:0/1497118831 <== mds.1 v2:172.21.15.92:6834/831923836 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7effc4020040 con 0x5599c32c39d0 2023-10-06T10:41:07.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.099+0000 7effab7fe700 10 client.6030 handle_command_reply: tid=0 2023-10-06T10:41:07.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.099+0000 7effdc9f6b80 1 client.6030 shutdown 2023-10-06T10:41:07.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.099+0000 7effdc9f6b80 1 -- 172.21.15.92:0/1497118831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c3283c20 msgr2=0x5599c33318c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:07.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.099+0000 7effdc9f6b80 1 --2- 172.21.15.92:0/1497118831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c3283c20 0x5599c33318c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7effc8002930 tx=0x7effc8019000 comp rx=0 tx=0).stop 2023-10-06T10:41:07.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.099+0000 7effdc9f6b80 1 -- 172.21.15.92:0/1497118831 shutdown_connections 2023-10-06T10:41:07.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.099+0000 7effdc9f6b80 1 --2- 172.21.15.92:0/1497118831 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599c32c9ae0 0x5599c32c3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.099+0000 7effdc9f6b80 1 --2- 172.21.15.92:0/1497118831 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x5599c32c39d0 0x5599c32ccea0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.099+0000 7effdc9f6b80 1 --2- 172.21.15.92:0/1497118831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c3283c20 0x5599c33318c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.099+0000 7effdc9f6b80 1 --2- 172.21.15.92:0/1497118831 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599c32859f0 0x5599c32c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.099+0000 7effab7fe700 0 client.6030 ms_handle_reset on v2:172.21.15.92:6834/831923836 2023-10-06T10:41:07.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.099+0000 7effdc9f6b80 1 -- 172.21.15.92:0/1497118831 >> 172.21.15.92:0/1497118831 conn(0x5599c32840c0 msgr2=0x5599c32aab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:07.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.099+0000 7effdc9f6b80 1 -- 172.21.15.92:0/1497118831 shutdown_connections 2023-10-06T10:41:07.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.099+0000 7effdc9f6b80 1 -- 172.21.15.92:0/1497118831 wait complete. 2023-10-06T10:41:07.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.099+0000 7effdc9f6b80 20 client.6030 trim_cache size 0 max 16384 2023-10-06T10:41:07.103 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:41:07.103 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2023-10-06T10:41:07.103 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:41:07.103 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:41:07.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.100+0000 7effd36fe700 1 -- 172.21.15.92:0/3175561749 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7effb0066f30 msgr2=0x7effb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:07.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.100+0000 7effd36fe700 1 --2- 172.21.15.92:0/3175561749 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7effb0066f30 0x7effb00693e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7effc800eb00 tx=0x7effc800b040 comp rx=0 tx=0).stop 2023-10-06T10:41:07.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.100+0000 7effd36fe700 1 -- 172.21.15.92:0/3175561749 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effcc067bd0 msgr2=0x7effcc13cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:07.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.100+0000 7effd36fe700 1 --2- 172.21.15.92:0/3175561749 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effcc067bd0 0x7effcc13cde0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7effbc00c7c0 tx=0x7effbc007eb0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.100+0000 7effd36fe700 1 -- 172.21.15.92:0/3175561749 shutdown_connections 2023-10-06T10:41:07.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.100+0000 7effd36fe700 1 --2- 172.21.15.92:0/3175561749 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7effb0066f30 0x7effb00693e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.100+0000 7effd36fe700 1 --2- 172.21.15.92:0/3175561749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effcc1405f0 0x7effcc0cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.100+0000 7effd36fe700 1 --2- 172.21.15.92:0/3175561749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effcc067800 0x7effcc0c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.100+0000 7effd36fe700 1 --2- 172.21.15.92:0/3175561749 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effcc067bd0 0x7effcc13cde0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.100+0000 7effd36fe700 1 -- 172.21.15.92:0/3175561749 >> 172.21.15.92:0/3175561749 conn(0x7effcc0cd1a0 msgr2=0x7effcc0c3700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:07.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.100+0000 7effd36fe700 1 -- 172.21.15.92:0/3175561749 shutdown_connections 2023-10-06T10:41:07.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.100+0000 7effd36fe700 1 -- 172.21.15.92:0/3175561749 wait complete. 2023-10-06T10:41:07.113 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:2 flush journal 2023-10-06T10:41:07.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.227+0000 7f70ea9ac700 1 Processor -- start 2023-10-06T10:41:07.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.227+0000 7f70ea9ac700 1 -- start start 2023-10-06T10:41:07.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.228+0000 7f70ea9ac700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70e413fa80 0x7f70e406e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.228+0000 7f70ea9ac700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f70e40c8000 0x7f70e415a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.228+0000 7f70ea9ac700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e406e9e0 con 0x7f70e413fe50 2023-10-06T10:41:07.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.228+0000 7f70ea9ac700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e415af20 con 0x7f70e40c8000 2023-10-06T10:41:07.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.228+0000 7f70ea9ac700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e406ecc0 con 0x7f70e413fa80 2023-10-06T10:41:07.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.228+0000 7f70e8f49700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f70e40c8000 0x7f70e415a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.228+0000 7f70e37fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f70e413fe50 0x7f70e40c78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59144/0 (socket says 172.21.15.92:59144) 2023-10-06T10:41:07.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.228+0000 7f70e37fe700 1 -- 172.21.15.92:0/489222168 learned_addr learned my addr 172.21.15.92:0/489222168 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:07.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.229+0000 7f70e3fff700 1 --2- 172.21.15.92:0/489222168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70e413fa80 0x7f70e406e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.229+0000 7f70e8f49700 1 -- 172.21.15.92:0/489222168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70e413fa80 msgr2=0x7f70e406e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:07.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.229+0000 7f70e8f49700 1 --2- 172.21.15.92:0/489222168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70e413fa80 0x7f70e406e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.229+0000 7f70e8f49700 1 -- 172.21.15.92:0/489222168 >> v1:172.21.15.92:6789/0 conn(0x7f70e413fe50 legacy=0x7f70e40c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:07.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.229+0000 7f70e8f49700 1 -- 172.21.15.92:0/489222168 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e406efa0 con 0x7f70e40c8000 2023-10-06T10:41:07.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.229+0000 7f70e8f49700 1 --2- 172.21.15.92:0/489222168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e40c8000 0x7f70e415a9e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009e60 tx=0x7f70dc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a87f50a7dbb70e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:07.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.229+0000 7f70e2ffd700 1 -- 172.21.15.92:0/489222168 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc00e070 con 0x7f70e40c8000 2023-10-06T10:41:07.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.229+0000 7f70e2ffd700 1 -- 172.21.15.92:0/489222168 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70dc00bc30 con 0x7f70e40c8000 2023-10-06T10:41:07.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.229+0000 7f70e2ffd700 1 -- 172.21.15.92:0/489222168 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc012430 con 0x7f70e40c8000 2023-10-06T10:41:07.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.229+0000 7f70ea9ac700 1 -- 172.21.15.92:0/489222168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e40c8000 msgr2=0x7f70e415a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:07.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.230+0000 7f70ea9ac700 1 --2- 172.21.15.92:0/489222168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e40c8000 0x7f70e415a9e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009e60 tx=0x7f70dc009780 comp rx=0 tx=0).stop 2023-10-06T10:41:07.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.230+0000 7f70ea9ac700 1 -- 172.21.15.92:0/489222168 shutdown_connections 2023-10-06T10:41:07.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.230+0000 7f70ea9ac700 1 --2- 172.21.15.92:0/489222168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70e413fa80 0x7f70e406e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.230+0000 7f70ea9ac700 1 --2- 172.21.15.92:0/489222168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e40c8000 0x7f70e415a9e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.230+0000 7f70ea9ac700 1 -- 172.21.15.92:0/489222168 >> 172.21.15.92:0/489222168 conn(0x7f70e40cc090 msgr2=0x7f70e40cc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:07.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.230+0000 7f70ea9ac700 1 -- 172.21.15.92:0/489222168 shutdown_connections 2023-10-06T10:41:07.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.230+0000 7f70ea9ac700 1 -- 172.21.15.92:0/489222168 wait complete. 2023-10-06T10:41:07.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.231+0000 7f70ea9ac700 1 Processor -- start 2023-10-06T10:41:07.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.231+0000 7f70ea9ac700 1 -- start start 2023-10-06T10:41:07.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.231+0000 7f70ea9ac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e40c8000 0x7f70e40b0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.231+0000 7f70ea9ac700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70e413fa80 0x7f70e40b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.232+0000 7f70ea9ac700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70e413fe50 0x7f70e4156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.232+0000 7f70ea9ac700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e4169950 con 0x7f70e40c8000 2023-10-06T10:41:07.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.232+0000 7f70ea9ac700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e4169b70 con 0x7f70e413fa80 2023-10-06T10:41:07.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.232+0000 7f70ea9ac700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e4169e50 con 0x7f70e413fe50 2023-10-06T10:41:07.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.232+0000 7f70e3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e40c8000 0x7f70e40b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.232+0000 7f70e3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e40c8000 0x7f70e40b0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59704/0 (socket says 172.21.15.92:59704) 2023-10-06T10:41:07.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.232+0000 7f70e3fff700 1 -- 172.21.15.92:0/3456969163 learned_addr learned my addr 172.21.15.92:0/3456969163 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:07.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.232+0000 7f70e8f49700 1 --2- 172.21.15.92:0/3456969163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70e413fe50 0x7f70e4156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.232+0000 7f70e37fe700 1 --2- 172.21.15.92:0/3456969163 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70e413fa80 0x7f70e40b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.232+0000 7f70e3fff700 1 -- 172.21.15.92:0/3456969163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70e413fe50 msgr2=0x7f70e4156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:07.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.232+0000 7f70e3fff700 1 --2- 172.21.15.92:0/3456969163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70e413fe50 0x7f70e4156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.232+0000 7f70e3fff700 1 -- 172.21.15.92:0/3456969163 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70e413fa80 msgr2=0x7f70e40b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:07.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.232+0000 7f70e3fff700 1 --2- 172.21.15.92:0/3456969163 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70e413fa80 0x7f70e40b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.232+0000 7f70e3fff700 1 -- 172.21.15.92:0/3456969163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc009a20 con 0x7f70e40c8000 2023-10-06T10:41:07.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.232+0000 7f70e3fff700 1 --2- 172.21.15.92:0/3456969163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e40c8000 0x7f70e40b0180 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f70d400d890 tx=0x7f70d4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:07.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.232+0000 7f70cbfff700 1 -- 172.21.15.92:0/3456969163 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d40125f0 con 0x7f70e40c8000 2023-10-06T10:41:07.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.232+0000 7f70cbfff700 1 -- 172.21.15.92:0/3456969163 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d4012750 con 0x7f70e40c8000 2023-10-06T10:41:07.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.232+0000 7f70ea9ac700 1 -- 172.21.15.92:0/3456969163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e416a130 con 0x7f70e40c8000 2023-10-06T10:41:07.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.232+0000 7f70cbfff700 1 -- 172.21.15.92:0/3456969163 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4012af0 con 0x7f70e40c8000 2023-10-06T10:41:07.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.232+0000 7f70ea9ac700 1 -- 172.21.15.92:0/3456969163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e4157220 con 0x7f70e40c8000 2023-10-06T10:41:07.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.233+0000 7f70cbfff700 1 -- 172.21.15.92:0/3456969163 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f70d401e460 con 0x7f70e40c8000 2023-10-06T10:41:07.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.234+0000 7f70cbfff700 1 --2- 172.21.15.92:0/3456969163 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f70d0066f30 0x7f70d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.234+0000 7f70cbfff700 1 -- 172.21.15.92:0/3456969163 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f70d4084d00 con 0x7f70e40c8000 2023-10-06T10:41:07.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.234+0000 7f70e37fe700 1 --2- 172.21.15.92:0/3456969163 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f70d0066f30 0x7f70d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.234+0000 7f70e37fe700 1 --2- 172.21.15.92:0/3456969163 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f70d0066f30 0x7f70d00693e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f70e413ee00 tx=0x7f70d8007eb0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.236+0000 7f70f3ca4b80 1 Processor -- start 2023-10-06T10:41:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.236+0000 7f70f3ca4b80 1 -- start start 2023-10-06T10:41:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.237+0000 7f70f3ca4b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564e28809ae0 0x564e28809eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.237+0000 7f70f3ca4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e287c59f0 0x564e287c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.237+0000 7f70f3ca4b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564e287c3c20 0x564e287c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.237+0000 7f70f3ca4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e28742cd0 con 0x564e287c59f0 2023-10-06T10:41:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.237+0000 7f70f3ca4b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e28719ac0 con 0x564e287c3c20 2023-10-06T10:41:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.237+0000 7f70f3ca4b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e287172c0 con 0x564e28809ae0 2023-10-06T10:41:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.237+0000 7f70e37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e287c59f0 0x564e287c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.237+0000 7f70e37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e287c59f0 0x564e287c25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59718/0 (socket says 172.21.15.92:59718) 2023-10-06T10:41:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.237+0000 7f70e37fe700 1 -- 172.21.15.92:0/3373796117 learned_addr learned my addr 172.21.15.92:0/3373796117 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.237+0000 7f70e3fff700 1 --2- 172.21.15.92:0/3373796117 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564e28809ae0 0x564e28809eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.237+0000 7f70e8f49700 1 --2- 172.21.15.92:0/3373796117 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564e287c3c20 0x564e287c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.237+0000 7f70e37fe700 1 -- 172.21.15.92:0/3373796117 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564e28809ae0 msgr2=0x564e28809eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.237+0000 7f70e37fe700 1 --2- 172.21.15.92:0/3373796117 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564e28809ae0 0x564e28809eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.237+0000 7f70e37fe700 1 -- 172.21.15.92:0/3373796117 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564e287c3c20 msgr2=0x564e287c2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.237+0000 7f70e37fe700 1 --2- 172.21.15.92:0/3373796117 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564e287c3c20 0x564e287c2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.237+0000 7f70e37fe700 1 -- 172.21.15.92:0/3373796117 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e285ad2c0 con 0x564e287c59f0 2023-10-06T10:41:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.237+0000 7f70e8f49700 1 --2- 172.21.15.92:0/3373796117 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564e287c3c20 0x564e287c2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.237+0000 7f70e37fe700 1 --2- 172.21.15.92:0/3373796117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e287c59f0 0x564e287c25d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f70d8001550 tx=0x7f70d8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4673afa59053c534 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.237+0000 7f70c97fa700 1 -- 172.21.15.92:0/3373796117 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8004930 con 0x564e287c59f0 2023-10-06T10:41:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.237+0000 7f70c97fa700 1 -- 172.21.15.92:0/3373796117 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d8001cd0 con 0x564e287c59f0 2023-10-06T10:41:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.237+0000 7f70c97fa700 1 -- 172.21.15.92:0/3373796117 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8013600 con 0x564e287c59f0 2023-10-06T10:41:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.238+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/3373796117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e287c59f0 msgr2=0x564e287c25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.238+0000 7f70f3ca4b80 1 --2- 172.21.15.92:0/3373796117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e287c59f0 0x564e287c25d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f70d8001550 tx=0x7f70d8014000 comp rx=0 tx=0).stop 2023-10-06T10:41:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.238+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/3373796117 shutdown_connections 2023-10-06T10:41:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.238+0000 7f70f3ca4b80 1 --2- 172.21.15.92:0/3373796117 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564e287c3c20 0x564e287c2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.238+0000 7f70f3ca4b80 1 --2- 172.21.15.92:0/3373796117 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564e28809ae0 0x564e28809eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.238+0000 7f70f3ca4b80 1 --2- 172.21.15.92:0/3373796117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e287c59f0 0x564e287c25d0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.238+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/3373796117 >> 172.21.15.92:0/3373796117 conn(0x564e287c40c0 msgr2=0x564e287c4490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.238+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/3373796117 shutdown_connections 2023-10-06T10:41:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.238+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/3373796117 wait complete. 2023-10-06T10:41:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.239+0000 7f70f3ca4b80 1 -- start start 2023-10-06T10:41:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.239+0000 7f70f3ca4b80 1 Processor -- start 2023-10-06T10:41:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.239+0000 7f70f3ca4b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564e287c3c20 0x564e287efa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.239+0000 7f70f3ca4b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564e287c59f0 0x564e288067e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.239+0000 7f70f3ca4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e28809ae0 0x564e288049d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.239+0000 7f70f3ca4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e287130b0 con 0x564e28809ae0 2023-10-06T10:41:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.239+0000 7f70f3ca4b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e286be8e0 con 0x564e287c3c20 2023-10-06T10:41:07.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.239+0000 7f70f3ca4b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e286d9ef0 con 0x564e287c59f0 2023-10-06T10:41:07.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.239+0000 7f70f3ca4b80 10 client.0 _ll_register_callbacks cb 0x564e285aff80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:07.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.239+0000 7f70e8f49700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e28809ae0 0x564e288049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.239+0000 7f70e8f49700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e28809ae0 0x564e288049d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59724/0 (socket says 172.21.15.92:59724) 2023-10-06T10:41:07.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.239+0000 7f70e8f49700 1 -- 172.21.15.92:0/531914377 learned_addr learned my addr 172.21.15.92:0/531914377 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:07.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70e37fe700 1 --2- 172.21.15.92:0/531914377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564e287c59f0 0x564e288067e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70e3fff700 1 --2- 172.21.15.92:0/531914377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564e287c3c20 0x564e287efa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70e8f49700 1 -- 172.21.15.92:0/531914377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564e287c59f0 msgr2=0x564e288067e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:07.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70e8f49700 1 --2- 172.21.15.92:0/531914377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564e287c59f0 0x564e288067e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70e8f49700 1 -- 172.21.15.92:0/531914377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564e287c3c20 msgr2=0x564e287efa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70e8f49700 1 --2- 172.21.15.92:0/531914377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564e287c3c20 0x564e287efa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70e8f49700 1 -- 172.21.15.92:0/531914377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e406e9e0 con 0x564e28809ae0 2023-10-06T10:41:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70e8f49700 1 --2- 172.21.15.92:0/531914377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e28809ae0 0x564e288049d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f70e40650a0 tx=0x7f70dc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ed235151a045b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70c27fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70c27fc700 1 -- 172.21.15.92:0/531914377 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc023070 con 0x564e28809ae0 2023-10-06T10:41:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70c27fc700 1 -- 172.21.15.92:0/531914377 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f70dc020990 con 0x564e28809ae0 2023-10-06T10:41:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/531914377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e287b76e0 con 0x564e28809ae0 2023-10-06T10:41:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70c27fc700 1 -- 172.21.15.92:0/531914377 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70dc020d80 con 0x564e28809ae0 2023-10-06T10:41:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70f3ca4b80 10 client.6045 fetch_fsmap learned FSMap version 39 2023-10-06T10:41:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/531914377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x564e28550030 con 0x564e28809ae0 2023-10-06T10:41:07.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70c27fc700 1 -- 172.21.15.92:0/531914377 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564e287b76e0 con 0x564e28809ae0 2023-10-06T10:41:07.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70c27fc700 1 -- 172.21.15.92:0/531914377 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 39) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f70dc020080 con 0x564e28809ae0 2023-10-06T10:41:07.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70f3ca4b80 10 client.6045 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T10:41:07.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70f3ca4b80 10 client.6045 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.d 2023-10-06T10:41:07.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70f3ca4b80 20 client.6045 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:07.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70f3ca4b80 1 --2- 172.21.15.92:0/531914377 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x564e287ed420 0x564e287effd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70f3ca4b80 4 client.6045 mds_command: new command op to 5036 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:41:07.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.240+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/531914377 --> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e28794610 con 0x564e287ed420 2023-10-06T10:41:07.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.241+0000 7f70e3fff700 1 --2- 172.21.15.92:0/531914377 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x564e287ed420 0x564e287effd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.241+0000 7f70c27fc700 10 client.6045 ms_handle_connect on v2:172.21.15.124:6833/2986058955 2023-10-06T10:41:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.241+0000 7f70e3fff700 1 --2- 172.21.15.92:0/531914377 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x564e287ed420 0x564e287effd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1d499e45a345f1ee server_cookie=cda021bbe19057c1 in_seq=0 out_seq=0 2023-10-06T10:41:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.241+0000 7f70c27fc700 1 -- 172.21.15.92:0/531914377 <== mds.2 v2:172.21.15.124:6833/2986058955 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x564e28794610 con 0x564e287ed420 2023-10-06T10:41:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.241+0000 7f70c27fc700 10 client.6045 handle_command_reply: tid=0 2023-10-06T10:41:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.241+0000 7f70f3ca4b80 1 client.6045 shutdown 2023-10-06T10:41:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.242+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/531914377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e28809ae0 msgr2=0x564e288049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.242+0000 7f70f3ca4b80 1 --2- 172.21.15.92:0/531914377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e28809ae0 0x564e288049d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f70e40650a0 tx=0x7f70dc019000 comp rx=0 tx=0).stop 2023-10-06T10:41:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.242+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/531914377 shutdown_connections 2023-10-06T10:41:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.242+0000 7f70f3ca4b80 1 --2- 172.21.15.92:0/531914377 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x564e287ed420 0x564e287effd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.242+0000 7f70f3ca4b80 1 --2- 172.21.15.92:0/531914377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e28809ae0 0x564e288049d0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.242+0000 7f70c27fc700 0 client.6045 ms_handle_reset on v2:172.21.15.124:6833/2986058955 2023-10-06T10:41:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.242+0000 7f70f3ca4b80 1 --2- 172.21.15.92:0/531914377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564e287c3c20 0x564e287efa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.242+0000 7f70f3ca4b80 1 --2- 172.21.15.92:0/531914377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564e287c59f0 0x564e288067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.242+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/531914377 >> 172.21.15.92:0/531914377 conn(0x564e287c40c0 msgr2=0x564e288075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.242+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/531914377 shutdown_connections 2023-10-06T10:41:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.242+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/531914377 wait complete. 2023-10-06T10:41:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.242+0000 7f70f3ca4b80 20 client.6045 trim_cache size 0 max 16384 2023-10-06T10:41:07.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.256+0000 7f70f3ca4b80 1 Processor -- start 2023-10-06T10:41:07.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.256+0000 7f70f3ca4b80 1 -- start start 2023-10-06T10:41:07.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.256+0000 7f70f3ca4b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564e287c3c20 0x564e287ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.256+0000 7f70f3ca4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e287c59f0 0x564e287efa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.256+0000 7f70f3ca4b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564e28809ae0 0x564e287f1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.257+0000 7f70f3ca4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e28595e80 con 0x564e287c59f0 2023-10-06T10:41:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.257+0000 7f70f3ca4b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e287c16f0 con 0x564e287c3c20 2023-10-06T10:41:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.257+0000 7f70f3ca4b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e287effd0 con 0x564e28809ae0 2023-10-06T10:41:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.257+0000 7f70e37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e287c59f0 0x564e287efa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.257+0000 7f70e37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e287c59f0 0x564e287efa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59740/0 (socket says 172.21.15.92:59740) 2023-10-06T10:41:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.257+0000 7f70e37fe700 1 -- 172.21.15.92:0/3655113234 learned_addr learned my addr 172.21.15.92:0/3655113234 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.257+0000 7f70e3fff700 1 --2- 172.21.15.92:0/3655113234 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564e287c3c20 0x564e287ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.257+0000 7f70e8f49700 1 --2- 172.21.15.92:0/3655113234 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564e28809ae0 0x564e287f1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.257+0000 7f70e37fe700 1 -- 172.21.15.92:0/3655113234 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564e28809ae0 msgr2=0x564e287f1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.257+0000 7f70e37fe700 1 --2- 172.21.15.92:0/3655113234 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564e28809ae0 0x564e287f1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.257+0000 7f70e37fe700 1 -- 172.21.15.92:0/3655113234 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564e287c3c20 msgr2=0x564e287ef2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.257+0000 7f70e37fe700 1 --2- 172.21.15.92:0/3655113234 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564e287c3c20 0x564e287ef2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.257+0000 7f70e37fe700 1 -- 172.21.15.92:0/3655113234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e287c0870 con 0x564e287c59f0 2023-10-06T10:41:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.257+0000 7f70e37fe700 1 --2- 172.21.15.92:0/3655113234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e287c59f0 0x564e287efa90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f70d801f000 tx=0x7f70d8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf025e4d8cf44fbb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.257+0000 7f70c9ffb700 1 -- 172.21.15.92:0/3655113234 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d80133f0 con 0x564e287c59f0 2023-10-06T10:41:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.257+0000 7f70c9ffb700 1 -- 172.21.15.92:0/3655113234 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d800b070 con 0x564e287c59f0 2023-10-06T10:41:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.257+0000 7f70c9ffb700 1 -- 172.21.15.92:0/3655113234 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8013a30 con 0x564e287c59f0 2023-10-06T10:41:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.258+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/3655113234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e287c59f0 msgr2=0x564e287efa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.258+0000 7f70f3ca4b80 1 --2- 172.21.15.92:0/3655113234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e287c59f0 0x564e287efa90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f70d801f000 tx=0x7f70d8018000 comp rx=0 tx=0).stop 2023-10-06T10:41:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.258+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/3655113234 shutdown_connections 2023-10-06T10:41:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.258+0000 7f70f3ca4b80 1 --2- 172.21.15.92:0/3655113234 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564e28809ae0 0x564e287f1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.258+0000 7f70f3ca4b80 1 --2- 172.21.15.92:0/3655113234 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564e287c3c20 0x564e287ef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.258+0000 7f70f3ca4b80 1 --2- 172.21.15.92:0/3655113234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e287c59f0 0x564e287efa90 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.258+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/3655113234 >> 172.21.15.92:0/3655113234 conn(0x564e287c40c0 msgr2=0x564e287ec6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.258+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/3655113234 shutdown_connections 2023-10-06T10:41:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.258+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/3655113234 wait complete. 2023-10-06T10:41:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.258+0000 7f70f3ca4b80 1 -- start start 2023-10-06T10:41:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.259+0000 7f70f3ca4b80 1 Processor -- start 2023-10-06T10:41:07.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.259+0000 7f70f3ca4b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564e287c3c20 0x564e2886e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.259+0000 7f70f3ca4b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564e287c59f0 0x564e28803150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.259+0000 7f70f3ca4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e28809ae0 0x564e28803690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.259+0000 7f70f3ca4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e287bdb30 con 0x564e28809ae0 2023-10-06T10:41:07.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.259+0000 7f70f3ca4b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2886eb40 con 0x564e287c59f0 2023-10-06T10:41:07.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.259+0000 7f70f3ca4b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e287eb580 con 0x564e287c3c20 2023-10-06T10:41:07.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.259+0000 7f70f3ca4b80 10 client.0 _ll_register_callbacks cb 0x564e285aff80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:07.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.259+0000 7f70e8f49700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e28809ae0 0x564e28803690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.259+0000 7f70e8f49700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e28809ae0 0x564e28803690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59748/0 (socket says 172.21.15.92:59748) 2023-10-06T10:41:07.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.259+0000 7f70e8f49700 1 -- 172.21.15.92:0/1602483916 learned_addr learned my addr 172.21.15.92:0/1602483916 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:07.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.259+0000 7f70e3fff700 1 --2- 172.21.15.92:0/1602483916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564e287c3c20 0x564e2886e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.259+0000 7f70e37fe700 1 --2- 172.21.15.92:0/1602483916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564e287c59f0 0x564e28803150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.260+0000 7f70e8f49700 1 -- 172.21.15.92:0/1602483916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564e287c3c20 msgr2=0x564e2886e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:07.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.260+0000 7f70e8f49700 1 --2- 172.21.15.92:0/1602483916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564e287c3c20 0x564e2886e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.260+0000 7f70e8f49700 1 -- 172.21.15.92:0/1602483916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564e287c59f0 msgr2=0x564e28803150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:07.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.260+0000 7f70e8f49700 1 --2- 172.21.15.92:0/1602483916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564e287c59f0 0x564e28803150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.260+0000 7f70e8f49700 1 -- 172.21.15.92:0/1602483916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e285ad2c0 con 0x564e28809ae0 2023-10-06T10:41:07.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.260+0000 7f70e8f49700 1 --2- 172.21.15.92:0/1602483916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e28809ae0 0x564e28803690 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x564e28800650 tx=0x7f70dc023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae40f8b31e0e3cac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.260+0000 7f70c27fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.260+0000 7f70c27fc700 1 -- 172.21.15.92:0/1602483916 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc037070 con 0x564e28809ae0 2023-10-06T10:41:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.260+0000 7f70c27fc700 1 -- 172.21.15.92:0/1602483916 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f70dc0205e0 con 0x564e28809ae0 2023-10-06T10:41:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.260+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/1602483916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70d8001cd0 con 0x564e28809ae0 2023-10-06T10:41:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.260+0000 7f70c27fc700 1 -- 172.21.15.92:0/1602483916 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70dc020080 con 0x564e28809ae0 2023-10-06T10:41:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.260+0000 7f70f3ca4b80 10 client.6051 fetch_fsmap learned FSMap version 39 2023-10-06T10:41:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.260+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/1602483916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x564e286e17d0 con 0x564e28809ae0 2023-10-06T10:41:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.260+0000 7f70c27fc700 1 -- 172.21.15.92:0/1602483916 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d8001cd0 con 0x564e28809ae0 2023-10-06T10:41:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.260+0000 7f70c27fc700 1 -- 172.21.15.92:0/1602483916 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 39) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f70dc012430 con 0x564e28809ae0 2023-10-06T10:41:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.260+0000 7f70f3ca4b80 10 client.6051 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T10:41:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.260+0000 7f70f3ca4b80 10 client.6051 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.d 2023-10-06T10:41:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.260+0000 7f70f3ca4b80 20 client.6051 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:07.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.260+0000 7f70f3ca4b80 1 --2- 172.21.15.92:0/1602483916 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x564e287ef9a0 0x564e28817570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:07.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.260+0000 7f70f3ca4b80 4 client.6051 mds_command: new command op to 5036 tid=0[{"prefix": "flush journal"}] 2023-10-06T10:41:07.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.260+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/1602483916 --> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564e286b5e50 con 0x564e287ef9a0 2023-10-06T10:41:07.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.260+0000 7f70e3fff700 1 --2- 172.21.15.92:0/1602483916 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x564e287ef9a0 0x564e28817570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:07.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.261+0000 7f70c27fc700 10 client.6051 ms_handle_connect on v2:172.21.15.124:6833/2986058955 2023-10-06T10:41:07.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.261+0000 7f70e3fff700 1 --2- 172.21.15.92:0/1602483916 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x564e287ef9a0 0x564e28817570 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ba56465682f164d4 server_cookie=4a46f824c97fbad6 in_seq=0 out_seq=0 2023-10-06T10:41:07.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.274+0000 7f70c27fc700 1 -- 172.21.15.92:0/1602483916 <== mds.2 v2:172.21.15.124:6833/2986058955 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564e286b5e50 con 0x564e287ef9a0 2023-10-06T10:41:07.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.274+0000 7f70c27fc700 10 client.6051 handle_command_reply: tid=0 2023-10-06T10:41:07.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.274+0000 7f70f3ca4b80 1 client.6051 shutdown 2023-10-06T10:41:07.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.274+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/1602483916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e28809ae0 msgr2=0x564e28803690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:07.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.274+0000 7f70f3ca4b80 1 --2- 172.21.15.92:0/1602483916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e28809ae0 0x564e28803690 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x564e28800650 tx=0x7f70dc023000 comp rx=0 tx=0).stop 2023-10-06T10:41:07.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.274+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/1602483916 shutdown_connections 2023-10-06T10:41:07.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.274+0000 7f70f3ca4b80 1 --2- 172.21.15.92:0/1602483916 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x564e287ef9a0 0x564e28817570 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.274+0000 7f70f3ca4b80 1 --2- 172.21.15.92:0/1602483916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e28809ae0 0x564e28803690 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.274+0000 7f70c27fc700 0 client.6051 ms_handle_reset on v2:172.21.15.124:6833/2986058955 2023-10-06T10:41:07.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.274+0000 7f70f3ca4b80 1 --2- 172.21.15.92:0/1602483916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564e287c3c20 0x564e2886e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.274+0000 7f70f3ca4b80 1 --2- 172.21.15.92:0/1602483916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564e287c59f0 0x564e28803150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.274+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/1602483916 >> 172.21.15.92:0/1602483916 conn(0x564e287c40c0 msgr2=0x564e287f2340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:07.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.274+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/1602483916 shutdown_connections 2023-10-06T10:41:07.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.274+0000 7f70f3ca4b80 1 -- 172.21.15.92:0/1602483916 wait complete. 2023-10-06T10:41:07.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.274+0000 7f70f3ca4b80 20 client.6051 trim_cache size 0 max 16384 2023-10-06T10:41:07.278 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:41:07.278 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2023-10-06T10:41:07.278 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:41:07.278 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:41:07.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.274+0000 7f70ea9ac700 1 -- 172.21.15.92:0/3456969163 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f70d0066f30 msgr2=0x7f70d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:07.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.274+0000 7f70ea9ac700 1 --2- 172.21.15.92:0/3456969163 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f70d0066f30 0x7f70d00693e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f70e413ee00 tx=0x7f70d8007eb0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.274+0000 7f70ea9ac700 1 -- 172.21.15.92:0/3456969163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e40c8000 msgr2=0x7f70e40b0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:07.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.274+0000 7f70ea9ac700 1 --2- 172.21.15.92:0/3456969163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e40c8000 0x7f70e40b0180 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f70d400d890 tx=0x7f70d4008040 comp rx=0 tx=0).stop 2023-10-06T10:41:07.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.274+0000 7f70ea9ac700 1 -- 172.21.15.92:0/3456969163 shutdown_connections 2023-10-06T10:41:07.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.275+0000 7f70ea9ac700 1 --2- 172.21.15.92:0/3456969163 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f70d0066f30 0x7f70d00693e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.275+0000 7f70ea9ac700 1 --2- 172.21.15.92:0/3456969163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70e413fe50 0x7f70e4156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.275+0000 7f70ea9ac700 1 --2- 172.21.15.92:0/3456969163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e40c8000 0x7f70e40b0180 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.275+0000 7f70ea9ac700 1 --2- 172.21.15.92:0/3456969163 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70e413fa80 0x7f70e40b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:07.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.275+0000 7f70ea9ac700 1 -- 172.21.15.92:0/3456969163 >> 172.21.15.92:0/3456969163 conn(0x7f70e40cc090 msgr2=0x7f70e40c78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:07.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.275+0000 7f70ea9ac700 1 -- 172.21.15.92:0/3456969163 shutdown_connections 2023-10-06T10:41:07.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:07.275+0000 7f70ea9ac700 1 -- 172.21.15.92:0/3456969163 wait complete. 2023-10-06T10:41:07.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:41:07.288 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2023-10-06T10:41:07.309 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/ 2023-10-06T10:41:07.309 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4 2023-10-06T10:41:07.309 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4/d5 2023-10-06T10:41:07.310 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4/d5/d6 2023-10-06T10:41:07.310 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4/d5/d6/d7 2023-10-06T10:41:07.310 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2023-10-06T10:41:07.310 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:41:07.310 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:41:07.310 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:41:07.310 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:41:07.310 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2023-10-06T10:41:07.310 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:41:07.414 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627778 2023-10-06T10:41:07.603 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:41:07.603 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:41:07.603 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:41:07.603 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:41:07.603 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2023-10-06T10:41:07.603 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:41:07.706 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627779 2023-10-06T10:41:07.895 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:41:07.895 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:41:07.896 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:41:07.896 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:41:07.896 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2023-10-06T10:41:07.896 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:41:07.999 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627780 2023-10-06T10:41:08.189 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:41:08.189 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:41:08.189 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:41:08.189 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:41:08.189 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2023-10-06T10:41:08.189 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:41:08.292 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627781 2023-10-06T10:41:08.481 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:41:08.481 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:41:08.482 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:41:08.482 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:41:08.482 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2023-10-06T10:41:08.482 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:41:08.585 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627782 2023-10-06T10:41:08.775 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:41:08.775 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:41:08.775 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:41:08.775 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:41:08.775 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2023-10-06T10:41:08.775 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:41:08.878 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627783 2023-10-06T10:41:09.070 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2023-10-06T10:41:09.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.143+0000 7f8e6e3d7000 1 Processor -- start 2023-10-06T10:41:09.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.143+0000 7f8e6e3d7000 1 -- start start 2023-10-06T10:41:09.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.144+0000 7f8e6e3d7000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a6a933270 0x556a6aa03650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.144+0000 7f8e6e3d7000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a6a979600 0x556a6a95fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.144+0000 7f8e6e3d7000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a6a9799d0 0x556a6aa05d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.144+0000 7f8e6e3d7000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a6a87b980 con 0x556a6a9799d0 2023-10-06T10:41:09.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.144+0000 7f8e6e3d7000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a6a87b620 con 0x556a6a979600 2023-10-06T10:41:09.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.144+0000 7f8e6e3d7000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a6a872400 con 0x556a6a933270 2023-10-06T10:41:09.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.144+0000 7f8e67e13700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a6a9799d0 0x556a6aa05d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.144+0000 7f8e67e13700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a6a9799d0 0x556a6aa05d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59752/0 (socket says 172.21.15.92:59752) 2023-10-06T10:41:09.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.144+0000 7f8e67e13700 1 -- 172.21.15.92:0/3032383407 learned_addr learned my addr 172.21.15.92:0/3032383407 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:09.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.144+0000 7f8e66e11700 1 --2- 172.21.15.92:0/3032383407 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a6a979600 0x556a6a95fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.144+0000 7f8e67612700 1 --2- 172.21.15.92:0/3032383407 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a6a933270 0x556a6aa03650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.144+0000 7f8e67e13700 1 -- 172.21.15.92:0/3032383407 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a6a933270 msgr2=0x556a6aa03650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.144+0000 7f8e67e13700 1 --2- 172.21.15.92:0/3032383407 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a6a933270 0x556a6aa03650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.144+0000 7f8e67e13700 1 -- 172.21.15.92:0/3032383407 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a6a979600 msgr2=0x556a6a95fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.144+0000 7f8e67e13700 1 --2- 172.21.15.92:0/3032383407 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a6a979600 0x556a6a95fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.144+0000 7f8e67e13700 1 -- 172.21.15.92:0/3032383407 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6000a0e0 con 0x556a6a9799d0 2023-10-06T10:41:09.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.144+0000 7f8e67612700 1 --2- 172.21.15.92:0/3032383407 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a6a933270 0x556a6aa03650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:09.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.144+0000 7f8e67e13700 1 --2- 172.21.15.92:0/3032383407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a6a9799d0 0x556a6aa05d70 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000a900 tx=0x7f8e60007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96550556f68cb2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.144+0000 7f8e65e0f700 1 -- 172.21.15.92:0/3032383407 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6000d780 con 0x556a6a9799d0 2023-10-06T10:41:09.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.144+0000 7f8e65e0f700 1 -- 172.21.15.92:0/3032383407 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6000d920 con 0x556a6a9799d0 2023-10-06T10:41:09.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.144+0000 7f8e65e0f700 1 -- 172.21.15.92:0/3032383407 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6000dce0 con 0x556a6a9799d0 2023-10-06T10:41:09.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.145+0000 7f8e6e3d7000 1 -- 172.21.15.92:0/3032383407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a6a9799d0 msgr2=0x556a6aa05d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.145+0000 7f8e6e3d7000 1 --2- 172.21.15.92:0/3032383407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a6a9799d0 0x556a6aa05d70 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000a900 tx=0x7f8e60007040 comp rx=0 tx=0).stop 2023-10-06T10:41:09.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.145+0000 7f8e6e3d7000 1 -- 172.21.15.92:0/3032383407 shutdown_connections 2023-10-06T10:41:09.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.145+0000 7f8e6e3d7000 1 --2- 172.21.15.92:0/3032383407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a6a9799d0 0x556a6aa05d70 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.145+0000 7f8e6e3d7000 1 --2- 172.21.15.92:0/3032383407 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a6a933270 0x556a6aa03650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.145+0000 7f8e6e3d7000 1 --2- 172.21.15.92:0/3032383407 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a6a979600 0x556a6a95fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.145+0000 7f8e6e3d7000 1 -- 172.21.15.92:0/3032383407 >> 172.21.15.92:0/3032383407 conn(0x556a6a931750 msgr2=0x556a6a975dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.145+0000 7f8e6e3d7000 1 -- 172.21.15.92:0/3032383407 shutdown_connections 2023-10-06T10:41:09.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.145+0000 7f8e6e3d7000 1 -- 172.21.15.92:0/3032383407 wait complete. 2023-10-06T10:41:09.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.145+0000 7f8e6e3d7000 1 Processor -- start 2023-10-06T10:41:09.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.145+0000 7f8e6e3d7000 1 -- start start 2023-10-06T10:41:09.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.146+0000 7f8e6e3d7000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a6a933270 0x556a6aa10620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.146+0000 7f8e6e3d7000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a6a979600 0x556a6aa098e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.146+0000 7f8e6e3d7000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a6a9799d0 0x556a6aa029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.146+0000 7f8e6e3d7000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a6aa09e20 con 0x556a6a979600 2023-10-06T10:41:09.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.146+0000 7f8e6e3d7000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a6aa10c50 con 0x556a6a9799d0 2023-10-06T10:41:09.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.146+0000 7f8e6e3d7000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a6aa10ef0 con 0x556a6a933270 2023-10-06T10:41:09.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.146+0000 7f8e66e11700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a6a979600 0x556a6aa098e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.146+0000 7f8e66e11700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a6a979600 0x556a6aa098e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59760/0 (socket says 172.21.15.92:59760) 2023-10-06T10:41:09.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.146+0000 7f8e66e11700 1 -- 172.21.15.92:0/376716781 learned_addr learned my addr 172.21.15.92:0/376716781 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:09.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.146+0000 7f8e67612700 1 --2- 172.21.15.92:0/376716781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a6a933270 0x556a6aa10620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.146+0000 7f8e66e11700 1 -- 172.21.15.92:0/376716781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a6a933270 msgr2=0x556a6aa10620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.146+0000 7f8e66e11700 1 --2- 172.21.15.92:0/376716781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a6a933270 0x556a6aa10620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.146+0000 7f8e66e11700 1 -- 172.21.15.92:0/376716781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a6a9799d0 msgr2=0x556a6aa029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.146+0000 7f8e66e11700 1 --2- 172.21.15.92:0/376716781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a6a9799d0 0x556a6aa029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.146+0000 7f8e66e11700 1 -- 172.21.15.92:0/376716781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6000a5b0 con 0x556a6a979600 2023-10-06T10:41:09.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.147+0000 7f8e66e11700 1 --2- 172.21.15.92:0/376716781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a6a979600 0x556a6aa098e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c00c5d0 tx=0x7f8e5c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.147+0000 7f8e4ffff700 1 -- 172.21.15.92:0/376716781 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c010400 con 0x556a6a979600 2023-10-06T10:41:09.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.147+0000 7f8e4ffff700 1 -- 172.21.15.92:0/376716781 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e5c005070 con 0x556a6a979600 2023-10-06T10:41:09.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.147+0000 7f8e6e3d7000 1 -- 172.21.15.92:0/376716781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e58005db0 con 0x556a6a979600 2023-10-06T10:41:09.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.147+0000 7f8e4ffff700 1 -- 172.21.15.92:0/376716781 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c0107b0 con 0x556a6a979600 2023-10-06T10:41:09.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.147+0000 7f8e6e3d7000 1 -- 172.21.15.92:0/376716781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a6a88a3d0 con 0x556a6a979600 2023-10-06T10:41:09.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.148+0000 7f8e4ffff700 1 -- 172.21.15.92:0/376716781 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8e5c010ba0 con 0x556a6a979600 2023-10-06T10:41:09.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.148+0000 7f8e4ffff700 1 --2- 172.21.15.92:0/376716781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8e540658c0 0x7f8e54067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.148+0000 7f8e67612700 1 --2- 172.21.15.92:0/376716781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8e540658c0 0x7f8e54067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.149+0000 7f8e4ffff700 1 -- 172.21.15.92:0/376716781 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8e5c008070 con 0x556a6a979600 2023-10-06T10:41:09.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.149+0000 7f8e6e3d7000 1 --2- 172.21.15.92:0/376716781 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x556a6aa02ee0 0x556a6aa066b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.149+0000 7f8e6e3d7000 1 -- 172.21.15.92:0/376716781 --> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] -- osd_op(unknown.0.0:1 8.12 8:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52) v8 -- 0x556a6aa06bf0 con 0x556a6aa02ee0 2023-10-06T10:41:09.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.149+0000 7f8e67612700 1 --2- 172.21.15.92:0/376716781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8e540658c0 0x7f8e54067d70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800a5a0 tx=0x7f8e58016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.149+0000 7f8e67e13700 1 --2- 172.21.15.92:0/376716781 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x556a6aa02ee0 0x556a6aa066b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.149+0000 7f8e67e13700 1 --2- 172.21.15.92:0/376716781 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x556a6aa02ee0 0x556a6aa066b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.151+0000 7f8e67e13700 1 -- 172.21.15.92:0/376716781 <== osd.6 v2:172.21.15.124:6824/90914 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v52'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8e60019070 con 0x556a6aa02ee0 2023-10-06T10:41:09.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.151+0000 7f8e6e3d7000 1 -- 172.21.15.92:0/376716781 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x556a6aa02ee0 msgr2=0x556a6aa066b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.151+0000 7f8e6e3d7000 1 --2- 172.21.15.92:0/376716781 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x556a6aa02ee0 0x556a6aa066b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.151+0000 7f8e6e3d7000 1 -- 172.21.15.92:0/376716781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8e540658c0 msgr2=0x7f8e54067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.151+0000 7f8e6e3d7000 1 --2- 172.21.15.92:0/376716781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8e540658c0 0x7f8e54067d70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800a5a0 tx=0x7f8e58016000 comp rx=0 tx=0).stop 2023-10-06T10:41:09.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.151+0000 7f8e6e3d7000 1 -- 172.21.15.92:0/376716781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a6a979600 msgr2=0x556a6aa098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.151+0000 7f8e6e3d7000 1 --2- 172.21.15.92:0/376716781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a6a979600 0x556a6aa098e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c00c5d0 tx=0x7f8e5c000bc0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.151+0000 7f8e67e13700 1 -- 172.21.15.92:0/376716781 reap_dead start 2023-10-06T10:41:09.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.151+0000 7f8e6e3d7000 1 -- 172.21.15.92:0/376716781 shutdown_connections 2023-10-06T10:41:09.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.151+0000 7f8e6e3d7000 1 -- 172.21.15.92:0/376716781 >> 172.21.15.92:0/376716781 conn(0x556a6a931750 msgr2=0x556a6a976a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.151+0000 7f8e6e3d7000 1 -- 172.21.15.92:0/376716781 shutdown_connections 2023-10-06T10:41:09.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.151+0000 7f8e6e3d7000 1 -- 172.21.15.92:0/376716781 wait complete. 2023-10-06T10:41:09.158 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2023-10-06T10:41:09.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.234+0000 7fa074b2b000 1 Processor -- start 2023-10-06T10:41:09.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.234+0000 7fa074b2b000 1 -- start start 2023-10-06T10:41:09.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.234+0000 7fa074b2b000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f09496e9c0 0x55f094983010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.234+0000 7fa074b2b000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f094a54c10 0x55f0949aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.234+0000 7fa074b2b000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f094a55520 0x55f094a60ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.234+0000 7fa074b2b000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0948cb980 con 0x55f09496e9c0 2023-10-06T10:41:09.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.234+0000 7fa074b2b000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0948cb620 con 0x55f094a55520 2023-10-06T10:41:09.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.234+0000 7fa074b2b000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0948c2400 con 0x55f094a54c10 2023-10-06T10:41:09.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.234+0000 7fa06dd66700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f09496e9c0 0x55f094983010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.235+0000 7fa06dd66700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f09496e9c0 0x55f094983010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59766/0 (socket says 172.21.15.92:59766) 2023-10-06T10:41:09.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.235+0000 7fa06dd66700 1 -- 172.21.15.92:0/2381556608 learned_addr learned my addr 172.21.15.92:0/2381556608 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:09.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.235+0000 7fa06d565700 1 --2- 172.21.15.92:0/2381556608 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f094a54c10 0x55f0949aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.235+0000 7fa06e567700 1 --2- 172.21.15.92:0/2381556608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f094a55520 0x55f094a60ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.235+0000 7fa06dd66700 1 -- 172.21.15.92:0/2381556608 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f094a54c10 msgr2=0x55f0949aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.235+0000 7fa06dd66700 1 --2- 172.21.15.92:0/2381556608 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f094a54c10 0x55f0949aca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.235+0000 7fa06dd66700 1 -- 172.21.15.92:0/2381556608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f094a55520 msgr2=0x55f094a60ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.235+0000 7fa06dd66700 1 --2- 172.21.15.92:0/2381556608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f094a55520 0x55f094a60ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.235+0000 7fa06dd66700 1 -- 172.21.15.92:0/2381556608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa060005cf0 con 0x55f09496e9c0 2023-10-06T10:41:09.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.235+0000 7fa06e567700 1 --2- 172.21.15.92:0/2381556608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f094a55520 0x55f094a60ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:09.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.235+0000 7fa06dd66700 1 --2- 172.21.15.92:0/2381556608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f09496e9c0 0x55f094983010 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x55f09496f370 tx=0x7fa060016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e57fb020ae19979 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.235+0000 7fa067fff700 1 -- 172.21.15.92:0/2381556608 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060002f90 con 0x55f09496e9c0 2023-10-06T10:41:09.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.235+0000 7fa067fff700 1 -- 172.21.15.92:0/2381556608 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa060000bc0 con 0x55f09496e9c0 2023-10-06T10:41:09.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.235+0000 7fa067fff700 1 -- 172.21.15.92:0/2381556608 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060026460 con 0x55f09496e9c0 2023-10-06T10:41:09.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.236+0000 7fa074b2b000 1 -- 172.21.15.92:0/2381556608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f09496e9c0 msgr2=0x55f094983010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.236+0000 7fa074b2b000 1 --2- 172.21.15.92:0/2381556608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f09496e9c0 0x55f094983010 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x55f09496f370 tx=0x7fa060016000 comp rx=0 tx=0).stop 2023-10-06T10:41:09.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.236+0000 7fa074b2b000 1 -- 172.21.15.92:0/2381556608 shutdown_connections 2023-10-06T10:41:09.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.236+0000 7fa074b2b000 1 --2- 172.21.15.92:0/2381556608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f094a55520 0x55f094a60ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.236+0000 7fa074b2b000 1 --2- 172.21.15.92:0/2381556608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f09496e9c0 0x55f094983010 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.236+0000 7fa074b2b000 1 --2- 172.21.15.92:0/2381556608 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f094a54c10 0x55f0949aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.236+0000 7fa074b2b000 1 -- 172.21.15.92:0/2381556608 >> 172.21.15.92:0/2381556608 conn(0x55f094981750 msgr2=0x55f0949823b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.236+0000 7fa074b2b000 1 -- 172.21.15.92:0/2381556608 shutdown_connections 2023-10-06T10:41:09.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.236+0000 7fa074b2b000 1 -- 172.21.15.92:0/2381556608 wait complete. 2023-10-06T10:41:09.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.236+0000 7fa074b2b000 1 Processor -- start 2023-10-06T10:41:09.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.236+0000 7fa074b2b000 1 -- start start 2023-10-06T10:41:09.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.237+0000 7fa074b2b000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f09496e9c0 0x55f094a3bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.237+0000 7fa074b2b000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f094a54c10 0x55f094a681c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.237+0000 7fa074b2b000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f094a55520 0x55f094a5d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.237+0000 7fa074b2b000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f094981b20 con 0x55f094a54c10 2023-10-06T10:41:09.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.237+0000 7fa074b2b000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f094981d50 con 0x55f09496e9c0 2023-10-06T10:41:09.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.237+0000 7fa074b2b000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f094981ff0 con 0x55f094a55520 2023-10-06T10:41:09.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.237+0000 7fa06d565700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f094a54c10 0x55f094a681c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.237+0000 7fa06d565700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f094a54c10 0x55f094a681c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59770/0 (socket says 172.21.15.92:59770) 2023-10-06T10:41:09.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.237+0000 7fa06d565700 1 -- 172.21.15.92:0/1149066838 learned_addr learned my addr 172.21.15.92:0/1149066838 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:09.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.237+0000 7fa06e567700 1 --2- 172.21.15.92:0/1149066838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f094a55520 0x55f094a5d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.237+0000 7fa06dd66700 1 --2- 172.21.15.92:0/1149066838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f09496e9c0 0x55f094a3bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.237+0000 7fa06d565700 1 -- 172.21.15.92:0/1149066838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f094a55520 msgr2=0x55f094a5d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.237+0000 7fa06d565700 1 --2- 172.21.15.92:0/1149066838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f094a55520 0x55f094a5d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.237+0000 7fa06d565700 1 -- 172.21.15.92:0/1149066838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f09496e9c0 msgr2=0x55f094a3bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.237+0000 7fa06d565700 1 --2- 172.21.15.92:0/1149066838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f09496e9c0 0x55f094a3bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.237+0000 7fa06d565700 1 -- 172.21.15.92:0/1149066838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0948f41c0 con 0x55f094a54c10 2023-10-06T10:41:09.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.237+0000 7fa06d565700 1 --2- 172.21.15.92:0/1149066838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f094a54c10 0x55f094a681c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa058008b50 tx=0x7fa058005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.238+0000 7fa0667fc700 1 -- 172.21.15.92:0/1149066838 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05800f050 con 0x55f094a54c10 2023-10-06T10:41:09.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.238+0000 7fa0667fc700 1 -- 172.21.15.92:0/1149066838 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa058010040 con 0x55f094a54c10 2023-10-06T10:41:09.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.238+0000 7fa074b2b000 1 -- 172.21.15.92:0/1149066838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f0948da3d0 con 0x55f094a54c10 2023-10-06T10:41:09.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.238+0000 7fa0667fc700 1 -- 172.21.15.92:0/1149066838 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa058006cb0 con 0x55f094a54c10 2023-10-06T10:41:09.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.238+0000 7fa074b2b000 1 -- 172.21.15.92:0/1149066838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0948cdae0 con 0x55f094a54c10 2023-10-06T10:41:09.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.239+0000 7fa0667fc700 1 -- 172.21.15.92:0/1149066838 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa058015670 con 0x55f094a54c10 2023-10-06T10:41:09.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.239+0000 7fa0667fc700 1 --2- 172.21.15.92:0/1149066838 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa050065890 0x7fa050067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.239+0000 7fa0667fc700 1 -- 172.21.15.92:0/1149066838 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa058083c40 con 0x55f094a54c10 2023-10-06T10:41:09.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.239+0000 7fa06dd66700 1 --2- 172.21.15.92:0/1149066838 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa050065890 0x7fa050067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.239+0000 7fa074b2b000 1 --2- 172.21.15.92:0/1149066838 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x55f0949c7f20 0x55f0949c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.239+0000 7fa074b2b000 1 -- 172.21.15.92:0/1149066838 --> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] -- osd_op(unknown.0.0:1 8.12 8:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52) v8 -- 0x55f0949c8860 con 0x55f0949c7f20 2023-10-06T10:41:09.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.240+0000 7fa06e567700 1 --2- 172.21.15.92:0/1149066838 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x55f0949c7f20 0x55f0949c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.240+0000 7fa06dd66700 1 --2- 172.21.15.92:0/1149066838 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa050065890 0x7fa050067d40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa060008fd0 tx=0x7fa06001e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.240+0000 7fa06e567700 1 --2- 172.21.15.92:0/1149066838 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x55f0949c7f20 0x55f0949c8320 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.241+0000 7fa06e567700 1 -- 172.21.15.92:0/1149066838 <== osd.6 v2:172.21.15.124:6824/90914 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v52'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa068004630 con 0x55f0949c7f20 2023-10-06T10:41:09.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.241+0000 7fa074b2b000 1 -- 172.21.15.92:0/1149066838 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x55f0949c7f20 msgr2=0x55f0949c8320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.241+0000 7fa074b2b000 1 --2- 172.21.15.92:0/1149066838 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x55f0949c7f20 0x55f0949c8320 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.241+0000 7fa074b2b000 1 -- 172.21.15.92:0/1149066838 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa050065890 msgr2=0x7fa050067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.241+0000 7fa074b2b000 1 --2- 172.21.15.92:0/1149066838 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa050065890 0x7fa050067d40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa060008fd0 tx=0x7fa06001e000 comp rx=0 tx=0).stop 2023-10-06T10:41:09.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.241+0000 7fa074b2b000 1 -- 172.21.15.92:0/1149066838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f094a54c10 msgr2=0x55f094a681c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.241+0000 7fa074b2b000 1 --2- 172.21.15.92:0/1149066838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f094a54c10 0x55f094a681c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa058008b50 tx=0x7fa058005be0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.241+0000 7fa06e567700 1 -- 172.21.15.92:0/1149066838 reap_dead start 2023-10-06T10:41:09.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.241+0000 7fa074b2b000 1 -- 172.21.15.92:0/1149066838 shutdown_connections 2023-10-06T10:41:09.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.241+0000 7fa074b2b000 1 -- 172.21.15.92:0/1149066838 >> 172.21.15.92:0/1149066838 conn(0x55f094981750 msgr2=0x55f0949aca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.242+0000 7fa074b2b000 1 -- 172.21.15.92:0/1149066838 shutdown_connections 2023-10-06T10:41:09.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.242+0000 7fa074b2b000 1 -- 172.21.15.92:0/1149066838 wait complete. 2023-10-06T10:41:09.250 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2023-10-06T10:41:09.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.325+0000 7ff9136be000 1 Processor -- start 2023-10-06T10:41:09.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.325+0000 7ff9136be000 1 -- start start 2023-10-06T10:41:09.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.325+0000 7ff9136be000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b0e6fade0 0x557b0e79aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.325+0000 7ff9136be000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b0e6fe580 0x557b0e6ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.325+0000 7ff9136be000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b0e7a2ec0 0x557b0e6cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.325+0000 7ff9136be000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0e619980 con 0x557b0e6fe580 2023-10-06T10:41:09.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.326+0000 7ff9136be000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0e619620 con 0x557b0e6fade0 2023-10-06T10:41:09.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.326+0000 7ff9136be000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b0e610400 con 0x557b0e7a2ec0 2023-10-06T10:41:09.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.326+0000 7ff907fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b0e6fe580 0x557b0e6ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.326+0000 7ff907fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b0e6fe580 0x557b0e6ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59782/0 (socket says 172.21.15.92:59782) 2023-10-06T10:41:09.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.326+0000 7ff907fff700 1 -- 172.21.15.92:0/247821040 learned_addr learned my addr 172.21.15.92:0/247821040 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:09.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.326+0000 7ff90c8f9700 1 --2- 172.21.15.92:0/247821040 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b0e6fade0 0x557b0e79aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.326+0000 7ff90d0fa700 1 --2- 172.21.15.92:0/247821040 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b0e7a2ec0 0x557b0e6cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.326+0000 7ff907fff700 1 -- 172.21.15.92:0/247821040 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b0e7a2ec0 msgr2=0x557b0e6cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.326+0000 7ff907fff700 1 --2- 172.21.15.92:0/247821040 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b0e7a2ec0 0x557b0e6cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.326+0000 7ff907fff700 1 -- 172.21.15.92:0/247821040 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b0e6fade0 msgr2=0x557b0e79aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.326+0000 7ff907fff700 1 --2- 172.21.15.92:0/247821040 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b0e6fade0 0x557b0e79aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.326+0000 7ff907fff700 1 -- 172.21.15.92:0/247821040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f8005cf0 con 0x557b0e6fe580 2023-10-06T10:41:09.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.326+0000 7ff907fff700 1 --2- 172.21.15.92:0/247821040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b0e6fe580 0x557b0e6ccd20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800c010 tx=0x7ff8f8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12923b8732bb9eb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.326+0000 7ff906ffd700 1 -- 172.21.15.92:0/247821040 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f8000bc0 con 0x557b0e6fe580 2023-10-06T10:41:09.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.326+0000 7ff906ffd700 1 -- 172.21.15.92:0/247821040 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f8008970 con 0x557b0e6fe580 2023-10-06T10:41:09.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.326+0000 7ff906ffd700 1 -- 172.21.15.92:0/247821040 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f801c410 con 0x557b0e6fe580 2023-10-06T10:41:09.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.327+0000 7ff9136be000 1 -- 172.21.15.92:0/247821040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b0e6fe580 msgr2=0x557b0e6ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.327+0000 7ff9136be000 1 --2- 172.21.15.92:0/247821040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b0e6fe580 0x557b0e6ccd20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800c010 tx=0x7ff8f8016000 comp rx=0 tx=0).stop 2023-10-06T10:41:09.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.327+0000 7ff9136be000 1 -- 172.21.15.92:0/247821040 shutdown_connections 2023-10-06T10:41:09.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.327+0000 7ff9136be000 1 --2- 172.21.15.92:0/247821040 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b0e7a2ec0 0x557b0e6cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.327+0000 7ff9136be000 1 --2- 172.21.15.92:0/247821040 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b0e6fade0 0x557b0e79aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.327+0000 7ff9136be000 1 --2- 172.21.15.92:0/247821040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b0e6fe580 0x557b0e6ccd20 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.327+0000 7ff9136be000 1 -- 172.21.15.92:0/247821040 >> 172.21.15.92:0/247821040 conn(0x557b0e6cf520 msgr2=0x557b0e6d1900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.327+0000 7ff9136be000 1 -- 172.21.15.92:0/247821040 shutdown_connections 2023-10-06T10:41:09.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.327+0000 7ff9136be000 1 -- 172.21.15.92:0/247821040 wait complete. 2023-10-06T10:41:09.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.327+0000 7ff9136be000 1 Processor -- start 2023-10-06T10:41:09.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.327+0000 7ff9136be000 1 -- start start 2023-10-06T10:41:09.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.328+0000 7ff9136be000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b0e6fade0 0x557b0e6fc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.328+0000 7ff9136be000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b0e6fe580 0x557b0e797150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.328+0000 7ff9136be000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b0e7a2ec0 0x557b0e79c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.328+0000 7ff9136be000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0e6fc8b0 con 0x557b0e6fe580 2023-10-06T10:41:09.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.328+0000 7ff9136be000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0e7980c0 con 0x557b0e6fade0 2023-10-06T10:41:09.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.328+0000 7ff9136be000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b0e797690 con 0x557b0e7a2ec0 2023-10-06T10:41:09.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.328+0000 7ff907fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b0e6fe580 0x557b0e797150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.328+0000 7ff907fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b0e6fe580 0x557b0e797150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59796/0 (socket says 172.21.15.92:59796) 2023-10-06T10:41:09.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.328+0000 7ff907fff700 1 -- 172.21.15.92:0/688576562 learned_addr learned my addr 172.21.15.92:0/688576562 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:09.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.328+0000 7ff90c8f9700 1 --2- 172.21.15.92:0/688576562 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b0e6fade0 0x557b0e6fc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.328+0000 7ff90d0fa700 1 --2- 172.21.15.92:0/688576562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b0e7a2ec0 0x557b0e79c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.328+0000 7ff907fff700 1 -- 172.21.15.92:0/688576562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b0e7a2ec0 msgr2=0x557b0e79c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.328+0000 7ff907fff700 1 --2- 172.21.15.92:0/688576562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b0e7a2ec0 0x557b0e79c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.328+0000 7ff907fff700 1 -- 172.21.15.92:0/688576562 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b0e6fade0 msgr2=0x557b0e6fc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.328+0000 7ff907fff700 1 --2- 172.21.15.92:0/688576562 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b0e6fade0 0x557b0e6fc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.328+0000 7ff907fff700 1 -- 172.21.15.92:0/688576562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b0e6421c0 con 0x557b0e6fe580 2023-10-06T10:41:09.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.328+0000 7ff90d0fa700 1 --2- 172.21.15.92:0/688576562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557b0e7a2ec0 0x557b0e79c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:09.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.328+0000 7ff90c8f9700 1 --2- 172.21.15.92:0/688576562 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557b0e6fade0 0x557b0e6fc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:09.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.329+0000 7ff907fff700 1 --2- 172.21.15.92:0/688576562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b0e6fe580 0x557b0e797150 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x557b0e6cde00 tx=0x7ff8f8003050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.329+0000 7ff9057fa700 1 -- 172.21.15.92:0/688576562 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f802a070 con 0x557b0e6fe580 2023-10-06T10:41:09.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.329+0000 7ff9057fa700 1 -- 172.21.15.92:0/688576562 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f8008560 con 0x557b0e6fe580 2023-10-06T10:41:09.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.329+0000 7ff9136be000 1 -- 172.21.15.92:0/688576562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557b0e6283d0 con 0x557b0e6fe580 2023-10-06T10:41:09.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.329+0000 7ff9057fa700 1 -- 172.21.15.92:0/688576562 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f8025680 con 0x557b0e6fe580 2023-10-06T10:41:09.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.329+0000 7ff9136be000 1 -- 172.21.15.92:0/688576562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b0e61bae0 con 0x557b0e6fe580 2023-10-06T10:41:09.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.330+0000 7ff9057fa700 1 -- 172.21.15.92:0/688576562 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff8f802e070 con 0x557b0e6fe580 2023-10-06T10:41:09.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.330+0000 7ff9057fa700 1 --2- 172.21.15.92:0/688576562 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff8f0065890 0x7ff8f0067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.331+0000 7ff9057fa700 1 -- 172.21.15.92:0/688576562 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff8f8086350 con 0x557b0e6fe580 2023-10-06T10:41:09.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.331+0000 7ff90c8f9700 1 --2- 172.21.15.92:0/688576562 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff8f0065890 0x7ff8f0067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.331+0000 7ff9136be000 1 --2- 172.21.15.92:0/688576562 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x557b0e712bb0 0x557b0e7a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.331+0000 7ff9136be000 1 -- 172.21.15.92:0/688576562 --> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] -- osd_op(unknown.0.0:1 8.5 8:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52) v8 -- 0x557b0e6fb1b0 con 0x557b0e712bb0 2023-10-06T10:41:09.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.331+0000 7ff90d0fa700 1 --2- 172.21.15.92:0/688576562 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x557b0e712bb0 0x557b0e7a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.331+0000 7ff90c8f9700 1 --2- 172.21.15.92:0/688576562 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff8f0065890 0x7ff8f0067d40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff9000057a0 tx=0x7ff900005b40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.331+0000 7ff90d0fa700 1 --2- 172.21.15.92:0/688576562 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x557b0e712bb0 0x557b0e7a59d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.333+0000 7ff90d0fa700 1 -- 172.21.15.92:0/688576562 <== osd.6 v2:172.21.15.124:6824/90914 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v52'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff908001630 con 0x557b0e712bb0 2023-10-06T10:41:09.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.333+0000 7ff9136be000 1 -- 172.21.15.92:0/688576562 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x557b0e712bb0 msgr2=0x557b0e7a59d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.333+0000 7ff9136be000 1 --2- 172.21.15.92:0/688576562 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x557b0e712bb0 0x557b0e7a59d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.333+0000 7ff9136be000 1 -- 172.21.15.92:0/688576562 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff8f0065890 msgr2=0x7ff8f0067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.333+0000 7ff9136be000 1 --2- 172.21.15.92:0/688576562 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff8f0065890 0x7ff8f0067d40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff9000057a0 tx=0x7ff900005b40 comp rx=0 tx=0).stop 2023-10-06T10:41:09.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.333+0000 7ff9136be000 1 -- 172.21.15.92:0/688576562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b0e6fe580 msgr2=0x557b0e797150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.333+0000 7ff9136be000 1 --2- 172.21.15.92:0/688576562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b0e6fe580 0x557b0e797150 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x557b0e6cde00 tx=0x7ff8f8003050 comp rx=0 tx=0).stop 2023-10-06T10:41:09.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.333+0000 7ff90d0fa700 1 -- 172.21.15.92:0/688576562 reap_dead start 2023-10-06T10:41:09.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.333+0000 7ff9136be000 1 -- 172.21.15.92:0/688576562 shutdown_connections 2023-10-06T10:41:09.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.333+0000 7ff9136be000 1 -- 172.21.15.92:0/688576562 >> 172.21.15.92:0/688576562 conn(0x557b0e6cf520 msgr2=0x557b0e6d1900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.333+0000 7ff9136be000 1 -- 172.21.15.92:0/688576562 shutdown_connections 2023-10-06T10:41:09.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.333+0000 7ff9136be000 1 -- 172.21.15.92:0/688576562 wait complete. 2023-10-06T10:41:09.345 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2023-10-06T10:41:09.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.419+0000 7f39ee19b000 1 Processor -- start 2023-10-06T10:41:09.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.419+0000 7f39ee19b000 1 -- start start 2023-10-06T10:41:09.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.420+0000 7f39ee19b000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771f1a3270 0x55771f267140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.420+0000 7f39ee19b000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55771f273c80 0x55771f26fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.420+0000 7f39ee19b000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55771f274050 0x55771f1ca670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.420+0000 7f39ee19b000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55771f0eb980 con 0x55771f1a3270 2023-10-06T10:41:09.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.420+0000 7f39ee19b000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55771f0eb620 con 0x55771f274050 2023-10-06T10:41:09.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.420+0000 7f39ee19b000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55771f0e2400 con 0x55771f273c80 2023-10-06T10:41:09.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.420+0000 7f39e73d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771f1a3270 0x55771f267140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.420+0000 7f39e73d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771f1a3270 0x55771f267140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59814/0 (socket says 172.21.15.92:59814) 2023-10-06T10:41:09.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.420+0000 7f39e73d6700 1 -- 172.21.15.92:0/1456831549 learned_addr learned my addr 172.21.15.92:0/1456831549 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:09.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.420+0000 7f39e7bd7700 1 --2- 172.21.15.92:0/1456831549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55771f274050 0x55771f1ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.420+0000 7f39e6bd5700 1 --2- 172.21.15.92:0/1456831549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55771f273c80 0x55771f26fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.421+0000 7f39e73d6700 1 -- 172.21.15.92:0/1456831549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55771f273c80 msgr2=0x55771f26fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.421+0000 7f39e73d6700 1 --2- 172.21.15.92:0/1456831549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55771f273c80 0x55771f26fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.421+0000 7f39e73d6700 1 -- 172.21.15.92:0/1456831549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55771f274050 msgr2=0x55771f1ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.421+0000 7f39e73d6700 1 --2- 172.21.15.92:0/1456831549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55771f274050 0x55771f1ca670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.421+0000 7f39e73d6700 1 -- 172.21.15.92:0/1456831549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39dc005cf0 con 0x55771f1a3270 2023-10-06T10:41:09.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.421+0000 7f39e73d6700 1 --2- 172.21.15.92:0/1456831549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771f1a3270 0x55771f267140 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f39d8001810 tx=0x7f39d8006a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16223b706a501860 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.421+0000 7f39e5bd3700 1 -- 172.21.15.92:0/1456831549 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8006320 con 0x55771f1a3270 2023-10-06T10:41:09.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.421+0000 7f39e5bd3700 1 -- 172.21.15.92:0/1456831549 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d8004520 con 0x55771f1a3270 2023-10-06T10:41:09.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.421+0000 7f39e5bd3700 1 -- 172.21.15.92:0/1456831549 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d80048e0 con 0x55771f1a3270 2023-10-06T10:41:09.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.421+0000 7f39ee19b000 1 -- 172.21.15.92:0/1456831549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771f1a3270 msgr2=0x55771f267140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.421+0000 7f39ee19b000 1 --2- 172.21.15.92:0/1456831549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771f1a3270 0x55771f267140 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f39d8001810 tx=0x7f39d8006a10 comp rx=0 tx=0).stop 2023-10-06T10:41:09.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.421+0000 7f39ee19b000 1 -- 172.21.15.92:0/1456831549 shutdown_connections 2023-10-06T10:41:09.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.421+0000 7f39ee19b000 1 --2- 172.21.15.92:0/1456831549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55771f274050 0x55771f1ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.421+0000 7f39ee19b000 1 --2- 172.21.15.92:0/1456831549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771f1a3270 0x55771f267140 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.421+0000 7f39ee19b000 1 --2- 172.21.15.92:0/1456831549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55771f273c80 0x55771f26fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.421+0000 7f39ee19b000 1 -- 172.21.15.92:0/1456831549 >> 172.21.15.92:0/1456831549 conn(0x55771f1a1750 msgr2=0x55771f1cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.421+0000 7f39ee19b000 1 -- 172.21.15.92:0/1456831549 shutdown_connections 2023-10-06T10:41:09.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.421+0000 7f39ee19b000 1 -- 172.21.15.92:0/1456831549 wait complete. 2023-10-06T10:41:09.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.422+0000 7f39ee19b000 1 Processor -- start 2023-10-06T10:41:09.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.422+0000 7f39ee19b000 1 -- start start 2023-10-06T10:41:09.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.423+0000 7f39ee19b000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771f1a3270 0x55771f266450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.423+0000 7f39ee19b000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55771f273c80 0x55771f1a2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.423+0000 7f39ee19b000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55771f274050 0x55771f26cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.423+0000 7f39ee19b000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55771f2761b0 con 0x55771f1a3270 2023-10-06T10:41:09.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.423+0000 7f39ee19b000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55771f276450 con 0x55771f273c80 2023-10-06T10:41:09.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.423+0000 7f39ee19b000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55771f2766f0 con 0x55771f274050 2023-10-06T10:41:09.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.423+0000 7f39e73d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771f1a3270 0x55771f266450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.423+0000 7f39e73d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771f1a3270 0x55771f266450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59822/0 (socket says 172.21.15.92:59822) 2023-10-06T10:41:09.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.423+0000 7f39e73d6700 1 -- 172.21.15.92:0/1439000378 learned_addr learned my addr 172.21.15.92:0/1439000378 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:09.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.423+0000 7f39e6bd5700 1 --2- 172.21.15.92:0/1439000378 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55771f273c80 0x55771f1a2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.423+0000 7f39e7bd7700 1 --2- 172.21.15.92:0/1439000378 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55771f274050 0x55771f26cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.423+0000 7f39e73d6700 1 -- 172.21.15.92:0/1439000378 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55771f274050 msgr2=0x55771f26cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.423+0000 7f39e73d6700 1 --2- 172.21.15.92:0/1439000378 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55771f274050 0x55771f26cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.423+0000 7f39e73d6700 1 -- 172.21.15.92:0/1439000378 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55771f273c80 msgr2=0x55771f1a2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.423+0000 7f39e73d6700 1 --2- 172.21.15.92:0/1439000378 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55771f273c80 0x55771f1a2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.423+0000 7f39e73d6700 1 -- 172.21.15.92:0/1439000378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d8006cf0 con 0x55771f1a3270 2023-10-06T10:41:09.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.423+0000 7f39e73d6700 1 --2- 172.21.15.92:0/1439000378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771f1a3270 0x55771f266450 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f39d8001e00 tx=0x7f39d8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.423+0000 7f39cffff700 1 -- 172.21.15.92:0/1439000378 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8000f60 con 0x55771f1a3270 2023-10-06T10:41:09.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.423+0000 7f39cffff700 1 -- 172.21.15.92:0/1439000378 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d8004c50 con 0x55771f1a3270 2023-10-06T10:41:09.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.423+0000 7f39cffff700 1 -- 172.21.15.92:0/1439000378 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8004690 con 0x55771f1a3270 2023-10-06T10:41:09.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.423+0000 7f39ee19b000 1 -- 172.21.15.92:0/1439000378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55771f0fa3d0 con 0x55771f1a3270 2023-10-06T10:41:09.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.423+0000 7f39ee19b000 1 -- 172.21.15.92:0/1439000378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55771f0edae0 con 0x55771f1a3270 2023-10-06T10:41:09.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.425+0000 7f39cffff700 1 -- 172.21.15.92:0/1439000378 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f39d801e630 con 0x55771f1a3270 2023-10-06T10:41:09.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.425+0000 7f39cffff700 1 --2- 172.21.15.92:0/1439000378 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f39d4065890 0x7f39d4067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.425+0000 7f39cffff700 1 -- 172.21.15.92:0/1439000378 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f39d8085050 con 0x55771f1a3270 2023-10-06T10:41:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.425+0000 7f39e6bd5700 1 --2- 172.21.15.92:0/1439000378 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f39d4065890 0x7f39d4067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.425+0000 7f39ee19b000 1 --2- 172.21.15.92:0/1439000378 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x55771f26fbd0 0x55771f272df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.425+0000 7f39ee19b000 1 -- 172.21.15.92:0/1439000378 --> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] -- osd_op(unknown.0.0:1 8.9 8:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52) v8 -- 0x55771f1a2c40 con 0x55771f26fbd0 2023-10-06T10:41:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.425+0000 7f39e6bd5700 1 --2- 172.21.15.92:0/1439000378 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f39d4065890 0x7f39d4067d40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00c010 tx=0x7f39dc013000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.425+0000 7f39e7bd7700 1 --2- 172.21.15.92:0/1439000378 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x55771f26fbd0 0x55771f272df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.426+0000 7f39e7bd7700 1 --2- 172.21.15.92:0/1439000378 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x55771f26fbd0 0x55771f272df0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.427+0000 7f39e7bd7700 1 -- 172.21.15.92:0/1439000378 <== osd.6 v2:172.21.15.124:6824/90914 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v52'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f39e0009a70 con 0x55771f26fbd0 2023-10-06T10:41:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.427+0000 7f39ee19b000 1 -- 172.21.15.92:0/1439000378 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x55771f26fbd0 msgr2=0x55771f272df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.427+0000 7f39ee19b000 1 --2- 172.21.15.92:0/1439000378 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x55771f26fbd0 0x55771f272df0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.427+0000 7f39ee19b000 1 -- 172.21.15.92:0/1439000378 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f39d4065890 msgr2=0x7f39d4067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.427+0000 7f39ee19b000 1 --2- 172.21.15.92:0/1439000378 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f39d4065890 0x7f39d4067d40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00c010 tx=0x7f39dc013000 comp rx=0 tx=0).stop 2023-10-06T10:41:09.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.427+0000 7f39ee19b000 1 -- 172.21.15.92:0/1439000378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771f1a3270 msgr2=0x55771f266450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.427+0000 7f39ee19b000 1 --2- 172.21.15.92:0/1439000378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771f1a3270 0x55771f266450 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f39d8001e00 tx=0x7f39d8014000 comp rx=0 tx=0).stop 2023-10-06T10:41:09.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.427+0000 7f39e7bd7700 1 -- 172.21.15.92:0/1439000378 reap_dead start 2023-10-06T10:41:09.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.427+0000 7f39ee19b000 1 -- 172.21.15.92:0/1439000378 shutdown_connections 2023-10-06T10:41:09.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.427+0000 7f39ee19b000 1 -- 172.21.15.92:0/1439000378 >> 172.21.15.92:0/1439000378 conn(0x55771f1a1750 msgr2=0x55771f18e830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.427+0000 7f39ee19b000 1 -- 172.21.15.92:0/1439000378 shutdown_connections 2023-10-06T10:41:09.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.427+0000 7f39ee19b000 1 -- 172.21.15.92:0/1439000378 wait complete. 2023-10-06T10:41:09.434 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2023-10-06T10:41:09.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.509+0000 7f5cbf76d000 1 Processor -- start 2023-10-06T10:41:09.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.509+0000 7f5cbf76d000 1 -- start start 2023-10-06T10:41:09.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.509+0000 7f5cbf76d000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563ee99f1e70 0x563ee99ef040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.509+0000 7f5cbf76d000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563ee9a68990 0x563ee9a89290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.509+0000 7f5cbf76d000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ee9a80d20 0x563ee9a61c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.509+0000 7f5cbf76d000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ee98f3980 con 0x563ee9a80d20 2023-10-06T10:41:09.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.509+0000 7f5cbf76d000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ee98f3620 con 0x563ee99f1e70 2023-10-06T10:41:09.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.509+0000 7f5cbf76d000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ee98ea400 con 0x563ee9a68990 2023-10-06T10:41:09.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.509+0000 7f5cb91a9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ee9a80d20 0x563ee9a61c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.509+0000 7f5cb91a9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ee9a80d20 0x563ee9a61c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59838/0 (socket says 172.21.15.92:59838) 2023-10-06T10:41:09.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.509+0000 7f5cb91a9700 1 -- 172.21.15.92:0/2297077342 learned_addr learned my addr 172.21.15.92:0/2297077342 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:09.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.510+0000 7f5cb91a9700 1 -- 172.21.15.92:0/2297077342 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563ee9a68990 msgr2=0x563ee9a89290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.510+0000 7f5cb91a9700 1 --2- 172.21.15.92:0/2297077342 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563ee9a68990 0x563ee9a89290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.510+0000 7f5cb91a9700 1 -- 172.21.15.92:0/2297077342 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563ee99f1e70 msgr2=0x563ee99ef040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.510+0000 7f5cb91a9700 1 --2- 172.21.15.92:0/2297077342 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563ee99f1e70 0x563ee99ef040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.510+0000 7f5cb91a9700 1 -- 172.21.15.92:0/2297077342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac005cf0 con 0x563ee9a80d20 2023-10-06T10:41:09.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.510+0000 7f5cb91a9700 1 --2- 172.21.15.92:0/2297077342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ee9a80d20 0x563ee9a61c00 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4005d80 tx=0x7f5cb4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4bbde244108e2c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.510+0000 7f5cb2ffd700 1 -- 172.21.15.92:0/2297077342 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4003300 con 0x563ee9a80d20 2023-10-06T10:41:09.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.510+0000 7f5cb2ffd700 1 -- 172.21.15.92:0/2297077342 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb400f860 con 0x563ee9a80d20 2023-10-06T10:41:09.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.510+0000 7f5cb2ffd700 1 -- 172.21.15.92:0/2297077342 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40246a0 con 0x563ee9a80d20 2023-10-06T10:41:09.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.510+0000 7f5cbf76d000 1 -- 172.21.15.92:0/2297077342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ee9a80d20 msgr2=0x563ee9a61c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.510+0000 7f5cbf76d000 1 --2- 172.21.15.92:0/2297077342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ee9a80d20 0x563ee9a61c00 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4005d80 tx=0x7f5cb4015000 comp rx=0 tx=0).stop 2023-10-06T10:41:09.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.510+0000 7f5cbf76d000 1 -- 172.21.15.92:0/2297077342 shutdown_connections 2023-10-06T10:41:09.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.510+0000 7f5cbf76d000 1 --2- 172.21.15.92:0/2297077342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ee9a80d20 0x563ee9a61c00 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.510+0000 7f5cbf76d000 1 --2- 172.21.15.92:0/2297077342 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563ee99f1e70 0x563ee99ef040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.510+0000 7f5cbf76d000 1 --2- 172.21.15.92:0/2297077342 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563ee9a68990 0x563ee9a89290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.510+0000 7f5cbf76d000 1 -- 172.21.15.92:0/2297077342 >> 172.21.15.92:0/2297077342 conn(0x563ee99a9520 msgr2=0x563ee9a66330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.510+0000 7f5cbf76d000 1 -- 172.21.15.92:0/2297077342 shutdown_connections 2023-10-06T10:41:09.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.510+0000 7f5cbf76d000 1 -- 172.21.15.92:0/2297077342 wait complete. 2023-10-06T10:41:09.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.511+0000 7f5cbf76d000 1 Processor -- start 2023-10-06T10:41:09.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.511+0000 7f5cbf76d000 1 -- start start 2023-10-06T10:41:09.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cbf76d000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ee99f1e70 0x563ee9a8b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cbf76d000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563ee9a68990 0x563ee9a884b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cbf76d000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563ee9a80d20 0x563ee9a889f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cbf76d000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ee9a8a450 con 0x563ee99f1e70 2023-10-06T10:41:09.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cbf76d000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ee9a8a6b0 con 0x563ee9a80d20 2023-10-06T10:41:09.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cbf76d000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ee9a8a950 con 0x563ee9a68990 2023-10-06T10:41:09.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cb89a8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ee99f1e70 0x563ee9a8b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cb89a8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ee99f1e70 0x563ee9a8b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59846/0 (socket says 172.21.15.92:59846) 2023-10-06T10:41:09.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cb89a8700 1 -- 172.21.15.92:0/1808191043 learned_addr learned my addr 172.21.15.92:0/1808191043 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:09.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cb91a9700 1 --2- 172.21.15.92:0/1808191043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563ee9a80d20 0x563ee9a889f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cb3fff700 1 --2- 172.21.15.92:0/1808191043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563ee9a68990 0x563ee9a884b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cb89a8700 1 -- 172.21.15.92:0/1808191043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563ee9a68990 msgr2=0x563ee9a884b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cb89a8700 1 --2- 172.21.15.92:0/1808191043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563ee9a68990 0x563ee9a884b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cb89a8700 1 -- 172.21.15.92:0/1808191043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563ee9a80d20 msgr2=0x563ee9a889f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cb89a8700 1 --2- 172.21.15.92:0/1808191043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563ee9a80d20 0x563ee9a889f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cb89a8700 1 -- 172.21.15.92:0/1808191043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ee991c1c0 con 0x563ee99f1e70 2023-10-06T10:41:09.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cb91a9700 1 --2- 172.21.15.92:0/1808191043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563ee9a80d20 0x563ee9a889f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:09.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cb3fff700 1 --2- 172.21.15.92:0/1808191043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563ee9a68990 0x563ee9a884b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:09.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cb89a8700 1 --2- 172.21.15.92:0/1808191043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ee99f1e70 0x563ee9a8b330 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5cac008af0 tx=0x7f5cac00c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cb17fa700 1 -- 172.21.15.92:0/1808191043 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac00f040 con 0x563ee99f1e70 2023-10-06T10:41:09.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cb17fa700 1 -- 172.21.15.92:0/1808191043 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cac00d7c0 con 0x563ee99f1e70 2023-10-06T10:41:09.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cbf76d000 1 -- 172.21.15.92:0/1808191043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb400a0e0 con 0x563ee99f1e70 2023-10-06T10:41:09.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cb17fa700 1 -- 172.21.15.92:0/1808191043 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac00db80 con 0x563ee99f1e70 2023-10-06T10:41:09.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.512+0000 7f5cbf76d000 1 -- 172.21.15.92:0/1808191043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ee99023d0 con 0x563ee99f1e70 2023-10-06T10:41:09.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.514+0000 7f5cb17fa700 1 -- 172.21.15.92:0/1808191043 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5cac01c400 con 0x563ee99f1e70 2023-10-06T10:41:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.514+0000 7f5cb17fa700 1 --2- 172.21.15.92:0/1808191043 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5c9c0658c0 0x7f5c9c067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.514+0000 7f5cb17fa700 1 -- 172.21.15.92:0/1808191043 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5cac083c60 con 0x563ee99f1e70 2023-10-06T10:41:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.514+0000 7f5cb3fff700 1 --2- 172.21.15.92:0/1808191043 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5c9c0658c0 0x7f5c9c067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.514+0000 7f5cbf76d000 1 --2- 172.21.15.92:0/1808191043 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x563ee9a64720 0x563ee99a98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.514+0000 7f5cbf76d000 1 -- 172.21.15.92:0/1808191043 --> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] -- osd_op(unknown.0.0:1 8.5 8:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52) v8 -- 0x563ee99a9e30 con 0x563ee9a64720 2023-10-06T10:41:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.514+0000 7f5cb3fff700 1 --2- 172.21.15.92:0/1808191043 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5c9c0658c0 0x7f5c9c067d70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4004170 tx=0x7f5ca4000bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.514+0000 7f5cb91a9700 1 --2- 172.21.15.92:0/1808191043 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x563ee9a64720 0x563ee99a98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.515+0000 7f5cb91a9700 1 --2- 172.21.15.92:0/1808191043 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x563ee9a64720 0x563ee99a98f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.516+0000 7f5cb91a9700 1 -- 172.21.15.92:0/1808191043 <== osd.6 v2:172.21.15.124:6824/90914 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v52'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5cb4020030 con 0x563ee9a64720 2023-10-06T10:41:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.516+0000 7f5cbf76d000 1 -- 172.21.15.92:0/1808191043 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x563ee9a64720 msgr2=0x563ee99a98f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.516+0000 7f5cbf76d000 1 --2- 172.21.15.92:0/1808191043 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x563ee9a64720 0x563ee99a98f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.516+0000 7f5cbf76d000 1 -- 172.21.15.92:0/1808191043 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5c9c0658c0 msgr2=0x7f5c9c067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.516+0000 7f5cbf76d000 1 --2- 172.21.15.92:0/1808191043 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5c9c0658c0 0x7f5c9c067d70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4004170 tx=0x7f5ca4000bc0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.516+0000 7f5cbf76d000 1 -- 172.21.15.92:0/1808191043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ee99f1e70 msgr2=0x563ee9a8b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.516+0000 7f5cbf76d000 1 --2- 172.21.15.92:0/1808191043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ee99f1e70 0x563ee9a8b330 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5cac008af0 tx=0x7f5cac00c040 comp rx=0 tx=0).stop 2023-10-06T10:41:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.516+0000 7f5cb91a9700 1 -- 172.21.15.92:0/1808191043 reap_dead start 2023-10-06T10:41:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.516+0000 7f5cbf76d000 1 -- 172.21.15.92:0/1808191043 shutdown_connections 2023-10-06T10:41:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.516+0000 7f5cbf76d000 1 -- 172.21.15.92:0/1808191043 >> 172.21.15.92:0/1808191043 conn(0x563ee99a9520 msgr2=0x563ee99d4480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.516+0000 7f5cbf76d000 1 -- 172.21.15.92:0/1808191043 shutdown_connections 2023-10-06T10:41:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.516+0000 7f5cbf76d000 1 -- 172.21.15.92:0/1808191043 wait complete. 2023-10-06T10:41:09.524 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2023-10-06T10:41:09.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.597+0000 7f75a5e8f000 1 Processor -- start 2023-10-06T10:41:09.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.597+0000 7f75a5e8f000 1 -- start start 2023-10-06T10:41:09.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.598+0000 7f75a5e8f000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5555dcc0eb60 0x5555dcc3ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.598+0000 7f75a5e8f000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555dccdfc80 0x5555dcce1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.598+0000 7f75a5e8f000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5555dcce0050 0x5555dcce1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.598+0000 7f75a5e8f000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dcb57620 con 0x5555dccdfc80 2023-10-06T10:41:09.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.598+0000 7f75a5e8f000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dcb4e400 con 0x5555dcc0eb60 2023-10-06T10:41:09.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.598+0000 7f75a5e8f000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5555dcce0e00 con 0x5555dcce0050 2023-10-06T10:41:09.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.598+0000 7f75968c9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555dccdfc80 0x5555dcce1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.598+0000 7f75968c9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555dccdfc80 0x5555dcce1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59866/0 (socket says 172.21.15.92:59866) 2023-10-06T10:41:09.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.598+0000 7f75968c9700 1 -- 172.21.15.92:0/2434543976 learned_addr learned my addr 172.21.15.92:0/2434543976 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:09.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.598+0000 7f759f8cb700 1 --2- 172.21.15.92:0/2434543976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5555dcce0050 0x5555dcce1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.598+0000 7f759f0ca700 1 --2- 172.21.15.92:0/2434543976 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5555dcc0eb60 0x5555dcc3ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.598+0000 7f75968c9700 1 -- 172.21.15.92:0/2434543976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5555dcce0050 msgr2=0x5555dcce1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.598+0000 7f75968c9700 1 --2- 172.21.15.92:0/2434543976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5555dcce0050 0x5555dcce1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.598+0000 7f75968c9700 1 -- 172.21.15.92:0/2434543976 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5555dcc0eb60 msgr2=0x5555dcc3ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.598+0000 7f75968c9700 1 --2- 172.21.15.92:0/2434543976 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5555dcc0eb60 0x5555dcc3ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.598+0000 7f75968c9700 1 -- 172.21.15.92:0/2434543976 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7590005db0 con 0x5555dccdfc80 2023-10-06T10:41:09.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.598+0000 7f759f8cb700 1 --2- 172.21.15.92:0/2434543976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5555dcce0050 0x5555dcce1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:09.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.598+0000 7f75968c9700 1 --2- 172.21.15.92:0/2434543976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555dccdfc80 0x5555dcce1120 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f758800b880 tx=0x7f7588005b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64eadb47da1aa8ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.599+0000 7f759e0c8700 1 -- 172.21.15.92:0/2434543976 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588010070 con 0x5555dccdfc80 2023-10-06T10:41:09.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.599+0000 7f759e0c8700 1 -- 172.21.15.92:0/2434543976 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758800b2e0 con 0x5555dccdfc80 2023-10-06T10:41:09.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.599+0000 7f759e0c8700 1 -- 172.21.15.92:0/2434543976 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75880068f0 con 0x5555dccdfc80 2023-10-06T10:41:09.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.599+0000 7f75a5e8f000 1 -- 172.21.15.92:0/2434543976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555dccdfc80 msgr2=0x5555dcce1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.599+0000 7f75a5e8f000 1 --2- 172.21.15.92:0/2434543976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555dccdfc80 0x5555dcce1120 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f758800b880 tx=0x7f7588005b60 comp rx=0 tx=0).stop 2023-10-06T10:41:09.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.599+0000 7f75a5e8f000 1 -- 172.21.15.92:0/2434543976 shutdown_connections 2023-10-06T10:41:09.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.599+0000 7f75a5e8f000 1 --2- 172.21.15.92:0/2434543976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5555dcce0050 0x5555dcce1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.599+0000 7f75a5e8f000 1 --2- 172.21.15.92:0/2434543976 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5555dcc0eb60 0x5555dcc3ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.599+0000 7f75a5e8f000 1 --2- 172.21.15.92:0/2434543976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555dccdfc80 0x5555dcce1120 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.599+0000 7f75a5e8f000 1 -- 172.21.15.92:0/2434543976 >> 172.21.15.92:0/2434543976 conn(0x5555dcc0d750 msgr2=0x5555dcce2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.599+0000 7f75a5e8f000 1 -- 172.21.15.92:0/2434543976 shutdown_connections 2023-10-06T10:41:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.599+0000 7f75a5e8f000 1 -- 172.21.15.92:0/2434543976 wait complete. 2023-10-06T10:41:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.600+0000 7f75a5e8f000 1 Processor -- start 2023-10-06T10:41:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.600+0000 7f75a5e8f000 1 -- start start 2023-10-06T10:41:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.600+0000 7f75a5e8f000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555dcc0eb60 0x5555dcc4e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.600+0000 7f75a5e8f000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5555dccdfc80 0x5555dccda1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.600+0000 7f75a5e8f000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5555dcce0050 0x5555dcce10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.600+0000 7f75a5e8f000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dcc507c0 con 0x5555dcc0eb60 2023-10-06T10:41:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.600+0000 7f75a5e8f000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dcc4ecc0 con 0x5555dccdfc80 2023-10-06T10:41:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.600+0000 7f75a5e8f000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5555dcc4ef60 con 0x5555dcce0050 2023-10-06T10:41:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.600+0000 7f759f0ca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555dcc0eb60 0x5555dcc4e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.600+0000 7f759f0ca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555dcc0eb60 0x5555dcc4e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59874/0 (socket says 172.21.15.92:59874) 2023-10-06T10:41:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.600+0000 7f759f0ca700 1 -- 172.21.15.92:0/2253520833 learned_addr learned my addr 172.21.15.92:0/2253520833 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.601+0000 7f75968c9700 1 --2- 172.21.15.92:0/2253520833 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5555dccdfc80 0x5555dccda1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.601+0000 7f759f8cb700 1 --2- 172.21.15.92:0/2253520833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5555dcce0050 0x5555dcce10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.601+0000 7f759f0ca700 1 -- 172.21.15.92:0/2253520833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5555dcce0050 msgr2=0x5555dcce10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.601+0000 7f759f0ca700 1 --2- 172.21.15.92:0/2253520833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5555dcce0050 0x5555dcce10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.601+0000 7f759f0ca700 1 -- 172.21.15.92:0/2253520833 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5555dccdfc80 msgr2=0x5555dccda1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.601+0000 7f759f0ca700 1 --2- 172.21.15.92:0/2253520833 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5555dccdfc80 0x5555dccda1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.601+0000 7f759f0ca700 1 -- 172.21.15.92:0/2253520833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7588005e40 con 0x5555dcc0eb60 2023-10-06T10:41:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.601+0000 7f759f8cb700 1 --2- 172.21.15.92:0/2253520833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5555dcce0050 0x5555dcce10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.601+0000 7f75968c9700 1 --2- 172.21.15.92:0/2253520833 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5555dccdfc80 0x5555dccda1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.601+0000 7f759f0ca700 1 --2- 172.21.15.92:0/2253520833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555dcc0eb60 0x5555dcc4e5b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f7590012000 tx=0x7f7590016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.601+0000 7f759c8c5700 1 -- 172.21.15.92:0/2253520833 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7590008a30 con 0x5555dcc0eb60 2023-10-06T10:41:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.601+0000 7f759c8c5700 1 -- 172.21.15.92:0/2253520833 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7590008bb0 con 0x5555dcc0eb60 2023-10-06T10:41:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.601+0000 7f75a5e8f000 1 -- 172.21.15.92:0/2253520833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5555dcb801c0 con 0x5555dcc0eb60 2023-10-06T10:41:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.601+0000 7f759c8c5700 1 -- 172.21.15.92:0/2253520833 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75900266c0 con 0x5555dcc0eb60 2023-10-06T10:41:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.601+0000 7f75a5e8f000 1 -- 172.21.15.92:0/2253520833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5555dcb663d0 con 0x5555dcc0eb60 2023-10-06T10:41:09.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.602+0000 7f759c8c5700 1 -- 172.21.15.92:0/2253520833 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7590023070 con 0x5555dcc0eb60 2023-10-06T10:41:09.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.603+0000 7f759c8c5700 1 --2- 172.21.15.92:0/2253520833 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f75800658c0 0x7f7580067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.603+0000 7f759c8c5700 1 -- 172.21.15.92:0/2253520833 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7590085b80 con 0x5555dcc0eb60 2023-10-06T10:41:09.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.603+0000 7f75968c9700 1 --2- 172.21.15.92:0/2253520833 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f75800658c0 0x7f7580067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.603+0000 7f75a5e8f000 1 --2- 172.21.15.92:0/2253520833 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x5555dcc3bf20 0x5555dcc51360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.603+0000 7f75a5e8f000 1 -- 172.21.15.92:0/2253520833 --> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] -- osd_op(unknown.0.0:1 8.1d 8:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52) v8 -- 0x5555dcc518a0 con 0x5555dcc3bf20 2023-10-06T10:41:09.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.603+0000 7f759f8cb700 1 --2- 172.21.15.92:0/2253520833 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x5555dcc3bf20 0x5555dcc51360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.603+0000 7f75968c9700 1 --2- 172.21.15.92:0/2253520833 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f75800658c0 0x7f7580067d70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7588006010 tx=0x7f7588017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.603+0000 7f759f8cb700 1 --2- 172.21.15.92:0/2253520833 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x5555dcc3bf20 0x5555dcc51360 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.604+0000 7f759f8cb700 1 -- 172.21.15.92:0/2253520833 <== osd.0 v2:172.21.15.92:6800/90604 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v52'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f759800da50 con 0x5555dcc3bf20 2023-10-06T10:41:09.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.604+0000 7f75a5e8f000 1 -- 172.21.15.92:0/2253520833 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x5555dcc3bf20 msgr2=0x5555dcc51360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.604+0000 7f75a5e8f000 1 --2- 172.21.15.92:0/2253520833 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x5555dcc3bf20 0x5555dcc51360 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.605+0000 7f75a5e8f000 1 -- 172.21.15.92:0/2253520833 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f75800658c0 msgr2=0x7f7580067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.605+0000 7f75a5e8f000 1 --2- 172.21.15.92:0/2253520833 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f75800658c0 0x7f7580067d70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7588006010 tx=0x7f7588017000 comp rx=0 tx=0).stop 2023-10-06T10:41:09.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.605+0000 7f75a5e8f000 1 -- 172.21.15.92:0/2253520833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555dcc0eb60 msgr2=0x5555dcc4e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.605+0000 7f75a5e8f000 1 --2- 172.21.15.92:0/2253520833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555dcc0eb60 0x5555dcc4e5b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f7590012000 tx=0x7f7590016000 comp rx=0 tx=0).stop 2023-10-06T10:41:09.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.605+0000 7f759f8cb700 1 -- 172.21.15.92:0/2253520833 reap_dead start 2023-10-06T10:41:09.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.605+0000 7f75a5e8f000 1 -- 172.21.15.92:0/2253520833 shutdown_connections 2023-10-06T10:41:09.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.605+0000 7f75a5e8f000 1 -- 172.21.15.92:0/2253520833 >> 172.21.15.92:0/2253520833 conn(0x5555dcc0d750 msgr2=0x5555dcc3c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.605+0000 7f75a5e8f000 1 -- 172.21.15.92:0/2253520833 shutdown_connections 2023-10-06T10:41:09.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.605+0000 7f75a5e8f000 1 -- 172.21.15.92:0/2253520833 wait complete. 2023-10-06T10:41:09.613 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /d1/d2/d3 recursive,force 2023-10-06T10:41:09.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.768+0000 7f1d0d339700 1 Processor -- start 2023-10-06T10:41:09.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.768+0000 7f1d0d339700 1 -- start start 2023-10-06T10:41:09.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.769+0000 7f1d0d339700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1d080aefe0 0x7f1d080ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.769+0000 7f1d0d339700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d080c6e60 con 0x7f1d080ced90 2023-10-06T10:41:09.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.769+0000 7f1d0d339700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d080cf1d0 con 0x7f1d080aec10 2023-10-06T10:41:09.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.769+0000 7f1d0d339700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d080c6250 con 0x7f1d080aefe0 2023-10-06T10:41:09.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.769+0000 7f1d0cb38700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1d080ced90 0x7f1d080ad090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59212/0 (socket says 172.21.15.92:59212) 2023-10-06T10:41:09.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.769+0000 7f1d0cb38700 1 -- 172.21.15.92:0/3529853459 learned_addr learned my addr 172.21.15.92:0/3529853459 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:09.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.769+0000 7f1d077fe700 1 --2- 172.21.15.92:0/3529853459 >> v2:172.21.15.124:3300/0 conn(0x7f1d080aefe0 0x7f1d080ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.769+0000 7f1d06ffd700 1 -- 172.21.15.92:0/3529853459 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636719862 0 0) 0x7f1d080c6e60 con 0x7f1d080ced90 2023-10-06T10:41:09.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.769+0000 7f1d06ffd700 1 -- 172.21.15.92:0/3529853459 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cf0003570 con 0x7f1d080ced90 2023-10-06T10:41:09.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.769+0000 7f1d06ffd700 1 -- 172.21.15.92:0/3529853459 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1986012371 0 0) 0x7f1cf0003570 con 0x7f1d080ced90 2023-10-06T10:41:09.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.769+0000 7f1d06ffd700 1 -- 172.21.15.92:0/3529853459 >> v2:172.21.15.124:3300/0 conn(0x7f1d080aefe0 msgr2=0x7f1d080ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.769+0000 7f1d06ffd700 1 --2- 172.21.15.92:0/3529853459 >> v2:172.21.15.124:3300/0 conn(0x7f1d080aefe0 0x7f1d080ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.769+0000 7f1d06ffd700 1 -- 172.21.15.92:0/3529853459 >> v1:172.21.15.124:6789/0 conn(0x7f1d080aec10 legacy=0x7f1d080c5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.769+0000 7f1d06ffd700 1 -- 172.21.15.92:0/3529853459 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d080c6530 con 0x7f1d080ced90 2023-10-06T10:41:09.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.769+0000 7f1d06ffd700 1 -- 172.21.15.92:0/3529853459 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f1d000017f0 con 0x7f1d080ced90 2023-10-06T10:41:09.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.770+0000 7f1d06ffd700 1 -- 172.21.15.92:0/3529853459 >> v1:172.21.15.92:6789/0 conn(0x7f1d080ced90 legacy=0x7f1d080ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.770+0000 7f1d06ffd700 1 --2- 172.21.15.92:0/3529853459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cf0003a60 0x7f1cf0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.770+0000 7f1d06ffd700 1 --2- 172.21.15.92:0/3529853459 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cf0004810 0x7f1cf0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.770+0000 7f1d06ffd700 1 --2- 172.21.15.92:0/3529853459 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1cf00051c0 0x7f1cf000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.770+0000 7f1d06ffd700 1 -- 172.21.15.92:0/3529853459 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d080c6250 con 0x7f1cf0003a60 2023-10-06T10:41:09.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.770+0000 7f1d06ffd700 1 -- 172.21.15.92:0/3529853459 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf000c0d0 con 0x7f1cf00051c0 2023-10-06T10:41:09.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.770+0000 7f1d06ffd700 1 -- 172.21.15.92:0/3529853459 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf000c290 con 0x7f1cf0004810 2023-10-06T10:41:09.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.770+0000 7f1d07fff700 1 --2- 172.21.15.92:0/3529853459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cf0003a60 0x7f1cf0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.770+0000 7f1d0cb38700 1 --2- 172.21.15.92:0/3529853459 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1cf00051c0 0x7f1cf000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.770+0000 7f1d077fe700 1 --2- 172.21.15.92:0/3529853459 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cf0004810 0x7f1cf0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.770+0000 7f1d07fff700 1 -- 172.21.15.92:0/3529853459 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cf0004810 msgr2=0x7f1cf0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.770+0000 7f1d07fff700 1 --2- 172.21.15.92:0/3529853459 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cf0004810 0x7f1cf0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.770+0000 7f1d07fff700 1 -- 172.21.15.92:0/3529853459 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1cf00051c0 msgr2=0x7f1cf000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.770+0000 7f1d07fff700 1 --2- 172.21.15.92:0/3529853459 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1cf00051c0 0x7f1cf000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.770+0000 7f1d07fff700 1 -- 172.21.15.92:0/3529853459 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf0003570 con 0x7f1cf0003a60 2023-10-06T10:41:09.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.770+0000 7f1d0cb38700 1 -- 172.21.15.92:0/3529853459 reap_dead start 2023-10-06T10:41:09.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.770+0000 7f1d07fff700 1 --2- 172.21.15.92:0/3529853459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cf0003a60 0x7f1cf0003e70 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80018d0 tx=0x7f1cf8007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4bc25bfa5b74225 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.770+0000 7f1d06ffd700 1 -- 172.21.15.92:0/3529853459 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8001290 con 0x7f1cf0003a60 2023-10-06T10:41:09.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.770+0000 7f1d06ffd700 1 -- 172.21.15.92:0/3529853459 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf8012420 con 0x7f1cf0003a60 2023-10-06T10:41:09.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.770+0000 7f1d06ffd700 1 -- 172.21.15.92:0/3529853459 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf80127e0 con 0x7f1cf0003a60 2023-10-06T10:41:09.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.771+0000 7f1d0d339700 1 -- 172.21.15.92:0/3529853459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cf0003a60 msgr2=0x7f1cf0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.771+0000 7f1d0d339700 1 --2- 172.21.15.92:0/3529853459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cf0003a60 0x7f1cf0003e70 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80018d0 tx=0x7f1cf8007d80 comp rx=0 tx=0).stop 2023-10-06T10:41:09.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.771+0000 7f1d0d339700 1 -- 172.21.15.92:0/3529853459 shutdown_connections 2023-10-06T10:41:09.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.771+0000 7f1d0d339700 1 --2- 172.21.15.92:0/3529853459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cf0003a60 0x7f1cf0003e70 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.771+0000 7f1d0d339700 1 -- 172.21.15.92:0/3529853459 >> 172.21.15.92:0/3529853459 conn(0x7f1d08099540 msgr2=0x7f1d0806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.771+0000 7f1d0d339700 1 -- 172.21.15.92:0/3529853459 shutdown_connections 2023-10-06T10:41:09.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.771+0000 7f1d0d339700 1 -- 172.21.15.92:0/3529853459 wait complete. 2023-10-06T10:41:09.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.772+0000 7f1d0d339700 1 Processor -- start 2023-10-06T10:41:09.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.772+0000 7f1d0d339700 1 -- start start 2023-10-06T10:41:09.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.773+0000 7f1d0d339700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cf0003a60 0x7f1d0814bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.773+0000 7f1d0d339700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d081691b0 0x7f1d080afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.773+0000 7f1d0d339700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d08169620 0x7f1d080b1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.773+0000 7f1d0d339700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d080c7530 con 0x7f1cf0003a60 2023-10-06T10:41:09.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.773+0000 7f1d0d339700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d080c7810 con 0x7f1d081691b0 2023-10-06T10:41:09.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.773+0000 7f1d0d339700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d080c7af0 con 0x7f1d08169620 2023-10-06T10:41:09.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.773+0000 7f1d07fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cf0003a60 0x7f1d0814bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.773+0000 7f1d07fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cf0003a60 0x7f1d0814bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59890/0 (socket says 172.21.15.92:59890) 2023-10-06T10:41:09.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.773+0000 7f1d07fff700 1 -- 172.21.15.92:0/2063259603 learned_addr learned my addr 172.21.15.92:0/2063259603 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:09.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.773+0000 7f1d0cb38700 1 --2- 172.21.15.92:0/2063259603 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d08169620 0x7f1d080b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.773+0000 7f1d077fe700 1 --2- 172.21.15.92:0/2063259603 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d081691b0 0x7f1d080afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.773+0000 7f1d07fff700 1 -- 172.21.15.92:0/2063259603 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d08169620 msgr2=0x7f1d080b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.773+0000 7f1d07fff700 1 --2- 172.21.15.92:0/2063259603 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d08169620 0x7f1d080b1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.773+0000 7f1d07fff700 1 -- 172.21.15.92:0/2063259603 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d081691b0 msgr2=0x7f1d080afd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.773+0000 7f1d07fff700 1 --2- 172.21.15.92:0/2063259603 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d081691b0 0x7f1d080afd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.773+0000 7f1d07fff700 1 -- 172.21.15.92:0/2063259603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d080c7dd0 con 0x7f1cf0003a60 2023-10-06T10:41:09.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.773+0000 7f1d077fe700 1 --2- 172.21.15.92:0/2063259603 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d081691b0 0x7f1d080afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:09.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.773+0000 7f1d07fff700 1 --2- 172.21.15.92:0/2063259603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cf0003a60 0x7f1d0814bd40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8012fd0 tx=0x7f1cf8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.773+0000 7f1ceffff700 1 -- 172.21.15.92:0/2063259603 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8006a80 con 0x7f1cf0003a60 2023-10-06T10:41:09.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.773+0000 7f1ceffff700 1 -- 172.21.15.92:0/2063259603 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf8006c20 con 0x7f1cf0003a60 2023-10-06T10:41:09.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.773+0000 7f1d0d339700 1 -- 172.21.15.92:0/2063259603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d0814c280 con 0x7f1cf0003a60 2023-10-06T10:41:09.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.773+0000 7f1ceffff700 1 -- 172.21.15.92:0/2063259603 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8007030 con 0x7f1cf0003a60 2023-10-06T10:41:09.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.774+0000 7f1d0d339700 1 -- 172.21.15.92:0/2063259603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d080b02b0 con 0x7f1cf0003a60 2023-10-06T10:41:09.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.775+0000 7f1ceffff700 1 -- 172.21.15.92:0/2063259603 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1cf8019070 con 0x7f1cf0003a60 2023-10-06T10:41:09.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.775+0000 7f1ceffff700 1 --2- 172.21.15.92:0/2063259603 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1cf4066f30 0x7f1cf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.775+0000 7f1ceffff700 1 -- 172.21.15.92:0/2063259603 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1cf8085a30 con 0x7f1cf0003a60 2023-10-06T10:41:09.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.775+0000 7f1d077fe700 1 --2- 172.21.15.92:0/2063259603 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1cf4066f30 0x7f1cf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.776+0000 7f1d077fe700 1 --2- 172.21.15.92:0/2063259603 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1cf4066f30 0x7f1cf40693e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc006dc0 tx=0x7f1cfc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.777+0000 7f1d17893b80 1 Processor -- start 2023-10-06T10:41:09.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.777+0000 7f1d17893b80 1 -- start start 2023-10-06T10:41:09.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.778+0000 7f1d17893b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d81803a30 0x559d81803e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.778+0000 7f1d17893b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559d8184dca0 0x559d8182ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.778+0000 7f1d17893b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559d8182cf40 0x559d817ffa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.778+0000 7f1d17893b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d81783d60 con 0x559d81803a30 2023-10-06T10:41:09.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.778+0000 7f1d17893b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d8175ab50 con 0x559d8184dca0 2023-10-06T10:41:09.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.778+0000 7f1d17893b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d81758350 con 0x559d8182cf40 2023-10-06T10:41:09.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.778+0000 7f1d07fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d81803a30 0x559d81803e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.778+0000 7f1d07fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d81803a30 0x559d81803e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59900/0 (socket says 172.21.15.92:59900) 2023-10-06T10:41:09.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.778+0000 7f1d07fff700 1 -- 172.21.15.92:0/63994460 learned_addr learned my addr 172.21.15.92:0/63994460 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:09.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.778+0000 7f1d07fff700 1 -- 172.21.15.92:0/63994460 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559d8182cf40 msgr2=0x559d817ffa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.778+0000 7f1d07fff700 1 --2- 172.21.15.92:0/63994460 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559d8182cf40 0x559d817ffa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.778+0000 7f1d07fff700 1 -- 172.21.15.92:0/63994460 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559d8184dca0 msgr2=0x559d8182ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.778+0000 7f1d07fff700 1 --2- 172.21.15.92:0/63994460 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559d8184dca0 0x559d8182ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.778+0000 7f1d07fff700 1 -- 172.21.15.92:0/63994460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d815ee350 con 0x559d81803a30 2023-10-06T10:41:09.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.778+0000 7f1d07fff700 1 --2- 172.21.15.92:0/63994460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d81803a30 0x559d81803e00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80016e0 tx=0x7f1cf8083c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69d4f2aa8ff1ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.778+0000 7f1ced7fa700 1 -- 172.21.15.92:0/63994460 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8056780 con 0x559d81803a30 2023-10-06T10:41:09.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.778+0000 7f1ced7fa700 1 -- 172.21.15.92:0/63994460 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf80599d0 con 0x559d81803a30 2023-10-06T10:41:09.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.778+0000 7f1ced7fa700 1 -- 172.21.15.92:0/63994460 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf805d720 con 0x559d81803a30 2023-10-06T10:41:09.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.779+0000 7f1d17893b80 1 -- 172.21.15.92:0/63994460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d81803a30 msgr2=0x559d81803e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.779+0000 7f1d17893b80 1 --2- 172.21.15.92:0/63994460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d81803a30 0x559d81803e00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80016e0 tx=0x7f1cf8083c10 comp rx=0 tx=0).stop 2023-10-06T10:41:09.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.779+0000 7f1d17893b80 1 -- 172.21.15.92:0/63994460 shutdown_connections 2023-10-06T10:41:09.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.779+0000 7f1d17893b80 1 --2- 172.21.15.92:0/63994460 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559d8182cf40 0x559d817ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.779+0000 7f1d17893b80 1 --2- 172.21.15.92:0/63994460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d81803a30 0x559d81803e00 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.779+0000 7f1d17893b80 1 --2- 172.21.15.92:0/63994460 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559d8184dca0 0x559d8182ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.779+0000 7f1d17893b80 1 -- 172.21.15.92:0/63994460 >> 172.21.15.92:0/63994460 conn(0x559d81805350 msgr2=0x559d81805720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.779+0000 7f1d17893b80 1 -- 172.21.15.92:0/63994460 shutdown_connections 2023-10-06T10:41:09.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.779+0000 7f1d17893b80 1 -- 172.21.15.92:0/63994460 wait complete. 2023-10-06T10:41:09.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.780+0000 7f1d17893b80 1 -- start start 2023-10-06T10:41:09.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.780+0000 7f1d17893b80 1 Processor -- start 2023-10-06T10:41:09.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.781+0000 7f1d17893b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559d81803a30 0x559d81829770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.781+0000 7f1d17893b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559d8182cf40 0x559d81844600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.781+0000 7f1d17893b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d8184dca0 0x559d81841bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.781+0000 7f1d17893b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d81754140 con 0x559d8184dca0 2023-10-06T10:41:09.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.781+0000 7f1d17893b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d816ff970 con 0x559d81803a30 2023-10-06T10:41:09.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.781+0000 7f1d17893b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d8171af80 con 0x559d8182cf40 2023-10-06T10:41:09.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.781+0000 7f1d17893b80 10 client.0 _ll_register_callbacks cb 0x559d815f1010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:09.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.781+0000 7f1d0cb38700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d8184dca0 0x559d81841bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.781+0000 7f1d0cb38700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d8184dca0 0x559d81841bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59908/0 (socket says 172.21.15.92:59908) 2023-10-06T10:41:09.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.781+0000 7f1d0cb38700 1 -- 172.21.15.92:0/23501943 learned_addr learned my addr 172.21.15.92:0/23501943 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:09.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.781+0000 7f1d077fe700 1 --2- 172.21.15.92:0/23501943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559d8182cf40 0x559d81844600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.781+0000 7f1d07fff700 1 --2- 172.21.15.92:0/23501943 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559d81803a30 0x559d81829770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.781+0000 7f1d0cb38700 1 -- 172.21.15.92:0/23501943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559d8182cf40 msgr2=0x559d81844600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.781+0000 7f1d0cb38700 1 --2- 172.21.15.92:0/23501943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559d8182cf40 0x559d81844600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.781+0000 7f1d0cb38700 1 -- 172.21.15.92:0/23501943 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559d81803a30 msgr2=0x559d81829770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.781+0000 7f1d0cb38700 1 --2- 172.21.15.92:0/23501943 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559d81803a30 0x559d81829770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.781+0000 7f1d0cb38700 1 -- 172.21.15.92:0/23501943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d080b02b0 con 0x559d8184dca0 2023-10-06T10:41:09.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.781+0000 7f1d0cb38700 1 --2- 172.21.15.92:0/23501943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d8184dca0 0x559d81841bc0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1d0808a780 tx=0x7f1d00006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8bf1bb892898a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.781+0000 7f1ce67fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:09.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.781+0000 7f1ce67fc700 1 -- 172.21.15.92:0/23501943 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00012a90 con 0x559d8184dca0 2023-10-06T10:41:09.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.781+0000 7f1ce67fc700 1 -- 172.21.15.92:0/23501943 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1d00012bf0 con 0x559d8184dca0 2023-10-06T10:41:09.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.781+0000 7f1d17893b80 1 -- 172.21.15.92:0/23501943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d817f8770 con 0x559d8184dca0 2023-10-06T10:41:09.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.782+0000 7f1ce67fc700 1 -- 172.21.15.92:0/23501943 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d00015440 con 0x559d8184dca0 2023-10-06T10:41:09.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.782+0000 7f1d17893b80 10 client.6123 fetch_fsmap learned FSMap version 39 2023-10-06T10:41:09.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.782+0000 7f1d17893b80 1 -- 172.21.15.92:0/23501943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x559d815910c0 con 0x559d8184dca0 2023-10-06T10:41:09.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.782+0000 7f1ce67fc700 1 -- 172.21.15.92:0/23501943 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559d817f8770 con 0x559d8184dca0 2023-10-06T10:41:09.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.782+0000 7f1ce67fc700 1 -- 172.21.15.92:0/23501943 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 39) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f1d000155a0 con 0x559d8184dca0 2023-10-06T10:41:09.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.782+0000 7f1d17893b80 10 client.6123 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T10:41:09.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.782+0000 7f1d17893b80 10 client.6123 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-10-06T10:41:09.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.782+0000 7f1d17893b80 20 client.6123 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:09.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.782+0000 7f1d17893b80 1 --2- 172.21.15.92:0/23501943 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x559d81846280 0x559d8182b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.782+0000 7f1d17893b80 4 client.6123 mds_command: new command op to 5736 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:41:09.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.782+0000 7f1d17893b80 1 -- 172.21.15.92:0/23501943 --> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d817d56a0 con 0x559d81846280 2023-10-06T10:41:09.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.782+0000 7f1d07fff700 1 --2- 172.21.15.92:0/23501943 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x559d81846280 0x559d8182b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.782+0000 7f1ce67fc700 10 client.6123 ms_handle_connect on v2:172.21.15.92:6835/2458340622 2023-10-06T10:41:09.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.782+0000 7f1d07fff700 1 --2- 172.21.15.92:0/23501943 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x559d81846280 0x559d8182b0f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45071cf617e18e83 server_cookie=a882cedda5adde91 in_seq=0 out_seq=0 2023-10-06T10:41:09.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.783+0000 7f1ce67fc700 1 -- 172.21.15.92:0/23501943 <== mds.0 v2:172.21.15.92:6835/2458340622 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x559d817d56a0 con 0x559d81846280 2023-10-06T10:41:09.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.783+0000 7f1ce67fc700 10 client.6123 handle_command_reply: tid=0 2023-10-06T10:41:09.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.783+0000 7f1d17893b80 1 client.6123 shutdown 2023-10-06T10:41:09.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.783+0000 7f1d17893b80 1 -- 172.21.15.92:0/23501943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d8184dca0 msgr2=0x559d81841bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.783+0000 7f1d17893b80 1 --2- 172.21.15.92:0/23501943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d8184dca0 0x559d81841bc0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1d0808a780 tx=0x7f1d00006040 comp rx=0 tx=0).stop 2023-10-06T10:41:09.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.783+0000 7f1d17893b80 1 -- 172.21.15.92:0/23501943 shutdown_connections 2023-10-06T10:41:09.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.783+0000 7f1d17893b80 1 --2- 172.21.15.92:0/23501943 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x559d81846280 0x559d8182b0f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.783+0000 7f1d17893b80 1 --2- 172.21.15.92:0/23501943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d8184dca0 0x559d81841bc0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.783+0000 7f1ce67fc700 0 client.6123 ms_handle_reset on v2:172.21.15.92:6835/2458340622 2023-10-06T10:41:09.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.783+0000 7f1d17893b80 1 --2- 172.21.15.92:0/23501943 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559d81803a30 0x559d81829770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.783+0000 7f1d17893b80 1 --2- 172.21.15.92:0/23501943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559d8182cf40 0x559d81844600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.783+0000 7f1d17893b80 1 -- 172.21.15.92:0/23501943 >> 172.21.15.92:0/23501943 conn(0x559d81805350 msgr2=0x559d81849580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.783+0000 7f1d17893b80 1 -- 172.21.15.92:0/23501943 shutdown_connections 2023-10-06T10:41:09.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.783+0000 7f1d17893b80 1 -- 172.21.15.92:0/23501943 wait complete. 2023-10-06T10:41:09.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.783+0000 7f1d17893b80 20 client.6123 trim_cache size 0 max 16384 2023-10-06T10:41:09.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.799+0000 7f1d17893b80 1 Processor -- start 2023-10-06T10:41:09.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.799+0000 7f1d17893b80 1 -- start start 2023-10-06T10:41:09.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.800+0000 7f1d17893b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d81803a30 0x559d8182c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.800+0000 7f1d17893b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559d8182cf40 0x559d81804290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.800+0000 7f1d17893b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559d8184dca0 0x559d818047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.800+0000 7f1d17893b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d81845db0 con 0x559d81803a30 2023-10-06T10:41:09.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.800+0000 7f1d17893b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d815d6f10 con 0x559d8184dca0 2023-10-06T10:41:09.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.800+0000 7f1d17893b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d817e31b0 con 0x559d8182cf40 2023-10-06T10:41:09.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.800+0000 7f1d07fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d81803a30 0x559d8182c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.800+0000 7f1d07fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d81803a30 0x559d8182c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59916/0 (socket says 172.21.15.92:59916) 2023-10-06T10:41:09.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.800+0000 7f1d07fff700 1 -- 172.21.15.92:0/3653972767 learned_addr learned my addr 172.21.15.92:0/3653972767 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:09.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.800+0000 7f1d07fff700 1 -- 172.21.15.92:0/3653972767 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559d8182cf40 msgr2=0x559d81804290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.800+0000 7f1d07fff700 1 --2- 172.21.15.92:0/3653972767 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559d8182cf40 0x559d81804290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.800+0000 7f1d07fff700 1 -- 172.21.15.92:0/3653972767 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559d8184dca0 msgr2=0x559d818047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.800+0000 7f1d07fff700 1 --2- 172.21.15.92:0/3653972767 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559d8184dca0 0x559d818047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.800+0000 7f1d07fff700 1 -- 172.21.15.92:0/3653972767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d81722860 con 0x559d81803a30 2023-10-06T10:41:09.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.800+0000 7f1d07fff700 1 --2- 172.21.15.92:0/3653972767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d81803a30 0x559d8182c5d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80561c0 tx=0x7f1cf8080040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c7c1a123d50c2f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.800+0000 7f1cedffb700 1 -- 172.21.15.92:0/3653972767 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8056550 con 0x559d81803a30 2023-10-06T10:41:09.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.800+0000 7f1cedffb700 1 -- 172.21.15.92:0/3653972767 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf8056780 con 0x559d81803a30 2023-10-06T10:41:09.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.800+0000 7f1cedffb700 1 -- 172.21.15.92:0/3653972767 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf805d720 con 0x559d81803a30 2023-10-06T10:41:09.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.801+0000 7f1d17893b80 1 -- 172.21.15.92:0/3653972767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d81803a30 msgr2=0x559d8182c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.801+0000 7f1d17893b80 1 --2- 172.21.15.92:0/3653972767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d81803a30 0x559d8182c5d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80561c0 tx=0x7f1cf8080040 comp rx=0 tx=0).stop 2023-10-06T10:41:09.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.801+0000 7f1d17893b80 1 -- 172.21.15.92:0/3653972767 shutdown_connections 2023-10-06T10:41:09.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.801+0000 7f1d17893b80 1 --2- 172.21.15.92:0/3653972767 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559d8184dca0 0x559d818047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.801+0000 7f1d17893b80 1 --2- 172.21.15.92:0/3653972767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d81803a30 0x559d8182c5d0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.801+0000 7f1d17893b80 1 --2- 172.21.15.92:0/3653972767 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559d8182cf40 0x559d81804290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.801+0000 7f1d17893b80 1 -- 172.21.15.92:0/3653972767 >> 172.21.15.92:0/3653972767 conn(0x559d81805350 msgr2=0x559d81806d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.801+0000 7f1d17893b80 1 -- 172.21.15.92:0/3653972767 shutdown_connections 2023-10-06T10:41:09.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.801+0000 7f1d17893b80 1 -- 172.21.15.92:0/3653972767 wait complete. 2023-10-06T10:41:09.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.802+0000 7f1d17893b80 1 -- start start 2023-10-06T10:41:09.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.802+0000 7f1d17893b80 1 Processor -- start 2023-10-06T10:41:09.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.802+0000 7f1d17893b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d81803a30 0x559d818af520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.802+0000 7f1d17893b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559d8182cf40 0x559d8185c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.802+0000 7f1d17893b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559d8184dca0 0x559d8182ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.802+0000 7f1d17893b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d81842c40 con 0x559d81803a30 2023-10-06T10:41:09.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.802+0000 7f1d17893b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d818622a0 con 0x559d8184dca0 2023-10-06T10:41:09.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.802+0000 7f1d17893b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d81846650 con 0x559d8182cf40 2023-10-06T10:41:09.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.802+0000 7f1d17893b80 10 client.0 _ll_register_callbacks cb 0x559d815f1010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:09.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.802+0000 7f1d07fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d81803a30 0x559d818af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.802+0000 7f1d07fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d81803a30 0x559d818af520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59920/0 (socket says 172.21.15.92:59920) 2023-10-06T10:41:09.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.802+0000 7f1d07fff700 1 -- 172.21.15.92:0/4034028255 learned_addr learned my addr 172.21.15.92:0/4034028255 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:09.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1d077fe700 1 --2- 172.21.15.92:0/4034028255 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559d8182cf40 0x559d8185c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1d0cb38700 1 --2- 172.21.15.92:0/4034028255 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559d8184dca0 0x559d8182ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1d07fff700 1 -- 172.21.15.92:0/4034028255 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559d8182cf40 msgr2=0x559d8185c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1d07fff700 1 --2- 172.21.15.92:0/4034028255 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559d8182cf40 0x559d8185c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1d07fff700 1 -- 172.21.15.92:0/4034028255 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559d8184dca0 msgr2=0x559d8182ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1d07fff700 1 --2- 172.21.15.92:0/4034028255 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559d8184dca0 0x559d8182ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1d07fff700 1 -- 172.21.15.92:0/4034028255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d815ee350 con 0x559d81803a30 2023-10-06T10:41:09.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1d0cb38700 1 --2- 172.21.15.92:0/4034028255 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559d8184dca0 0x559d8182ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:09.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1d077fe700 1 --2- 172.21.15.92:0/4034028255 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559d8182cf40 0x559d8185c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:09.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1d07fff700 1 --2- 172.21.15.92:0/4034028255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d81803a30 0x559d818af520 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8056480 tx=0x7f1cf8074040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5ab52272eab9d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1ce67fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:09.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1ce67fc700 1 -- 172.21.15.92:0/4034028255 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf807e160 con 0x559d81803a30 2023-10-06T10:41:09.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1d17893b80 1 -- 172.21.15.92:0/4034028255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d81843f10 con 0x559d81803a30 2023-10-06T10:41:09.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1ce67fc700 1 -- 172.21.15.92:0/4034028255 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1cf80569e0 con 0x559d81803a30 2023-10-06T10:41:09.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1ce67fc700 1 -- 172.21.15.92:0/4034028255 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cf8055c70 con 0x559d81803a30 2023-10-06T10:41:09.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1d17893b80 10 client.6129 fetch_fsmap learned FSMap version 39 2023-10-06T10:41:09.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1d17893b80 1 -- 172.21.15.92:0/4034028255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x559d81709550 con 0x559d81803a30 2023-10-06T10:41:09.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1ce67fc700 1 -- 172.21.15.92:0/4034028255 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559d81843f10 con 0x559d81803a30 2023-10-06T10:41:09.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1ce67fc700 1 -- 172.21.15.92:0/4034028255 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 39) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f1cf805d290 con 0x559d81803a30 2023-10-06T10:41:09.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1d17893b80 10 client.6129 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T10:41:09.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1d17893b80 10 client.6129 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-10-06T10:41:09.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1d17893b80 20 client.6129 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:09.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1d17893b80 1 --2- 172.21.15.92:0/4034028255 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x559d8182b260 0x559d81861000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1d17893b80 4 client.6129 mds_command: new command op to 5736 tid=0[{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2023-10-06T10:41:09.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1d17893b80 1 -- 172.21.15.92:0/4034028255 --> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) v1 -- 0x7f1cf80599d0 con 0x559d8182b260 2023-10-06T10:41:09.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.803+0000 7f1d077fe700 1 --2- 172.21.15.92:0/4034028255 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x559d8182b260 0x559d81861000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.804+0000 7f1ce67fc700 10 client.6129 ms_handle_connect on v2:172.21.15.92:6835/2458340622 2023-10-06T10:41:09.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.804+0000 7f1d077fe700 1 --2- 172.21.15.92:0/4034028255 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x559d8182b260 0x559d81861000 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51c3ceb755630203 server_cookie=ac6748dbb0261ffb in_seq=0 out_seq=0 2023-10-06T10:41:09.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.804+0000 7f1ce67fc700 1 -- 172.21.15.92:0/4034028255 <== mds.0 v2:172.21.15.92:6835/2458340622 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f1cf80599d0 con 0x559d8182b260 2023-10-06T10:41:09.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.804+0000 7f1ce67fc700 10 client.6129 handle_command_reply: tid=0 2023-10-06T10:41:09.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.804+0000 7f1d17893b80 1 client.6129 shutdown 2023-10-06T10:41:09.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.804+0000 7f1d17893b80 1 -- 172.21.15.92:0/4034028255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d81803a30 msgr2=0x559d818af520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.804+0000 7f1d17893b80 1 --2- 172.21.15.92:0/4034028255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d81803a30 0x559d818af520 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8056480 tx=0x7f1cf8074040 comp rx=0 tx=0).stop 2023-10-06T10:41:09.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.804+0000 7f1d17893b80 1 -- 172.21.15.92:0/4034028255 shutdown_connections 2023-10-06T10:41:09.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.804+0000 7f1d17893b80 1 --2- 172.21.15.92:0/4034028255 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x559d8182b260 0x559d81861000 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.804+0000 7f1d17893b80 1 --2- 172.21.15.92:0/4034028255 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559d8184dca0 0x559d8182ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.804+0000 7f1ce67fc700 0 client.6129 ms_handle_reset on v2:172.21.15.92:6835/2458340622 2023-10-06T10:41:09.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.804+0000 7f1d17893b80 1 --2- 172.21.15.92:0/4034028255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d81803a30 0x559d818af520 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.804+0000 7f1d17893b80 1 --2- 172.21.15.92:0/4034028255 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559d8182cf40 0x559d8185c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.804+0000 7f1d17893b80 1 -- 172.21.15.92:0/4034028255 >> 172.21.15.92:0/4034028255 conn(0x559d81805350 msgr2=0x559d817fa280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.813 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-10-06T10:41:09.805+0000 7fcf381c3700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2023-10-06T10:40:51.061444+0000 1=0+1) n(v3 rc2023-10-06T10:40:51.291437+0000 6=0+6) (inest mix dirty) (ifile excl) (iversion lock) caps={5599=pAsLsXsFsx/pAsLsXsFsx@5},l=5599 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x56027bd32100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:40:51.061444+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:40:51.291437+0000 6=0+6)","memory_value.dirstat":"f(v0 m2023-10-06T10:40:51.061444+0000 1=0+1)","memory_value.rstat":"n(v3 rc2023-10-06T10:40:51.291437+0000 6=0+6)","error_str":""},"return_code":-61} 2023-10-06T10:41:09.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.805+0000 7f1d17893b80 1 -- 172.21.15.92:0/4034028255 shutdown_connections 2023-10-06T10:41:09.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.805+0000 7f1d17893b80 1 -- 172.21.15.92:0/4034028255 wait complete. 2023-10-06T10:41:09.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.805+0000 7f1d17893b80 20 client.6129 trim_cache size 0 max 16384 2023-10-06T10:41:09.814 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:41:09.815 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0, 2023-10-06T10:41:09.815 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_tag": "9514c1ce-0eea-44fe-937b-b21b60ec3c5b", 2023-10-06T10:41:09.815 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": "asynchronous" 2023-10-06T10:41:09.815 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:41:09.815 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-10-06T10:41:09.805+0000 7f3702072700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2023-10-06T10:40:51.062444+0000 1=0+1) n(v2 rc2023-10-06T10:40:51.291437+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={5599=pAsLsXsFsx/pAsLsXsFsx@4},l=5599 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x563031b62c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:40:51.062444+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:40:51.291437+0000 4=0+4)","memory_value.dirstat":"f(v0 m2023-10-06T10:40:51.062444+0000 1=0+1)","memory_value.rstat":"n(v2 rc2023-10-06T10:40:51.291437+0000 4=0+4)","error_str":""},"return_code":-61} 2023-10-06T10:41:09.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.805+0000 7f1d0d339700 1 -- 172.21.15.92:0/2063259603 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1cf4066f30 msgr2=0x7f1cf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.805+0000 7f1d0d339700 1 --2- 172.21.15.92:0/2063259603 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1cf4066f30 0x7f1cf40693e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc006dc0 tx=0x7f1cfc008040 comp rx=0 tx=0).stop 2023-10-06T10:41:09.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.805+0000 7f1d0d339700 1 -- 172.21.15.92:0/2063259603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cf0003a60 msgr2=0x7f1d0814bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.805+0000 7f1d0d339700 1 --2- 172.21.15.92:0/2063259603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cf0003a60 0x7f1d0814bd40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8012fd0 tx=0x7f1cf8015000 comp rx=0 tx=0).stop 2023-10-06T10:41:09.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.805+0000 7f1d0d339700 1 -- 172.21.15.92:0/2063259603 shutdown_connections 2023-10-06T10:41:09.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.805+0000 7f1d0d339700 1 --2- 172.21.15.92:0/2063259603 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1cf4066f30 0x7f1cf40693e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.805+0000 7f1d0d339700 1 --2- 172.21.15.92:0/2063259603 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d08169620 0x7f1d080b1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.805+0000 7f1d0d339700 1 --2- 172.21.15.92:0/2063259603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cf0003a60 0x7f1d0814bd40 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.805+0000 7f1d0d339700 1 --2- 172.21.15.92:0/2063259603 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d081691b0 0x7f1d080afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.805+0000 7f1d0d339700 1 -- 172.21.15.92:0/2063259603 >> 172.21.15.92:0/2063259603 conn(0x7f1d08099540 msgr2=0x7f1d080cb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.805+0000 7f1d0d339700 1 -- 172.21.15.92:0/2063259603 shutdown_connections 2023-10-06T10:41:09.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.806+0000 7f1d0d339700 1 -- 172.21.15.92:0/2063259603 wait complete. 2023-10-06T10:41:09.817 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-10-06T10:41:09.806+0000 7fcf3e1cf700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2023-10-06T10:40:51.061444+0000 1=0+1) n(v4 rc2023-10-06T10:40:51.291437+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={5599=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x56027bd33700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:40:51.061444+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:40:51.291437+0000 5=0+5)","memory_value.dirstat":"f(v0 m2023-10-06T10:40:51.061444+0000 1=0+1)","memory_value.rstat":"n(v4 rc2023-10-06T10:40:51.291437+0000 5=0+5)","error_str":""},"return_code":-61} 2023-10-06T10:41:09.817 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-10-06T10:41:09.807+0000 7f370807e700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2023-10-06T10:40:51.063444+0000 1=0+1) n(v1 rc2023-10-06T10:40:51.291437+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={5599=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x563031b63700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:40:51.063444+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:40:51.063444+0000 3=0+3)","memory_value.dirstat":"f(v0 m2023-10-06T10:40:51.063444+0000 1=0+1)","memory_value.rstat":"n(v1 rc2023-10-06T10:40:51.291437+0000 3=0+3)","error_str":""},"return_code":-61} 2023-10-06T10:41:09.818 INFO:tasks.ceph.mds.d.smithi124.stderr:2023-10-06T10:41:09.805+0000 7f8aee1e0700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2023-10-06T10:40:51.063444+0000 1=0+1) n(v0 rc2023-10-06T10:40:51.063444+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={5599=pAsLsXsFsx/pAsLsXsFsx@4},l=5599 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55aa0ec80580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:40:51.063444+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:40:51.063444+0000 2=0+2)","memory_value.dirstat":"f(v0 m2023-10-06T10:40:51.063444+0000 1=0+1)","memory_value.rstat":"n(v0 rc2023-10-06T10:40:51.063444+0000 2=0+2)","error_str":""},"return_code":-61} 2023-10-06T10:41:09.818 INFO:tasks.ceph.mds.d.smithi124.stderr:2023-10-06T10:41:09.806+0000 7f8aee1e0700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={5599=pAsLsXsFsx/AsLsXsFsx@2},l=5599 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55aa0ec80b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2023-10-06T10:41:09.819 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub abort 2023-10-06T10:41:09.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.934+0000 7fc5c1118700 1 Processor -- start 2023-10-06T10:41:09.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.934+0000 7fc5c1118700 1 -- start start 2023-10-06T10:41:09.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.935+0000 7fc5c1118700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5bc150940 0x7fc5bc159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.935+0000 7fc5c1118700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc5bc0cd4d0 0x7fc5bc0b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.935+0000 7fc5c1118700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5bc15ac30 con 0x7fc5bc15a160 2023-10-06T10:41:09.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.935+0000 7fc5c1118700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5bc0cb6a0 con 0x7fc5bc0cd4d0 2023-10-06T10:41:09.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.935+0000 7fc5c1118700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0cb090 con 0x7fc5bc150940 2023-10-06T10:41:09.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.935+0000 7fc5b259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc5bc15a160 0x7fc5bc0ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59214/0 (socket says 172.21.15.92:59214) 2023-10-06T10:41:09.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.935+0000 7fc5b259c700 1 -- 172.21.15.92:0/3072593320 learned_addr learned my addr 172.21.15.92:0/3072593320 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:09.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.935+0000 7fc5bad9d700 1 --2- 172.21.15.92:0/3072593320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5bc150940 0x7fc5bc159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.935+0000 7fc5bb59e700 1 --2- 172.21.15.92:0/3072593320 >> v2:172.21.15.124:3300/0 conn(0x7fc5bc0cd4d0 0x7fc5bc0b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.935+0000 7fc5ba59c700 1 -- 172.21.15.92:0/3072593320 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3582109857 0 0) 0x7fc5a4001110 con 0x7fc5bc15a160 2023-10-06T10:41:09.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.935+0000 7fc5ba59c700 1 -- 172.21.15.92:0/3072593320 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5a8003570 con 0x7fc5bc15a160 2023-10-06T10:41:09.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5ba59c700 1 -- 172.21.15.92:0/3072593320 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 852645288 0 0) 0x7fc5a8003570 con 0x7fc5bc15a160 2023-10-06T10:41:09.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5ba59c700 1 -- 172.21.15.92:0/3072593320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5bc150940 msgr2=0x7fc5bc159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5ba59c700 1 --2- 172.21.15.92:0/3072593320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5bc150940 0x7fc5bc159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5ba59c700 1 -- 172.21.15.92:0/3072593320 >> v2:172.21.15.124:3300/0 conn(0x7fc5bc0cd4d0 msgr2=0x7fc5bc0b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5ba59c700 1 --2- 172.21.15.92:0/3072593320 >> v2:172.21.15.124:3300/0 conn(0x7fc5bc0cd4d0 0x7fc5bc0b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5ba59c700 1 -- 172.21.15.92:0/3072593320 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc15a670 con 0x7fc5bc15a160 2023-10-06T10:41:09.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5ba59c700 1 -- 172.21.15.92:0/3072593320 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fc5a4001600 con 0x7fc5bc15a160 2023-10-06T10:41:09.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5ba59c700 1 -- 172.21.15.92:0/3072593320 >> v1:172.21.15.92:6789/0 conn(0x7fc5bc15a160 legacy=0x7fc5bc0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5ba59c700 1 --2- 172.21.15.92:0/3072593320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5a8003a60 0x7fc5a8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5ba59c700 1 --2- 172.21.15.92:0/3072593320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5a8004810 0x7fc5a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5ba59c700 1 --2- 172.21.15.92:0/3072593320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a80051c0 0x7fc5a800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5ba59c700 1 -- 172.21.15.92:0/3072593320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0cb6a0 con 0x7fc5a80051c0 2023-10-06T10:41:09.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5ba59c700 1 -- 172.21.15.92:0/3072593320 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0cb090 con 0x7fc5a8004810 2023-10-06T10:41:09.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5ba59c700 1 -- 172.21.15.92:0/3072593320 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a800bfa0 con 0x7fc5a8003a60 2023-10-06T10:41:09.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5bad9d700 1 --2- 172.21.15.92:0/3072593320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5bc150940 0x7fc5bc159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:41:09.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5bb59e700 1 --2- 172.21.15.92:0/3072593320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a80051c0 0x7fc5a800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5b259c700 1 --2- 172.21.15.92:0/3072593320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5a8004810 0x7fc5a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5bad9d700 1 --2- 172.21.15.92:0/3072593320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5a8003a60 0x7fc5a8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5bb59e700 1 -- 172.21.15.92:0/3072593320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5a8003a60 msgr2=0x7fc5a8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5bb59e700 1 --2- 172.21.15.92:0/3072593320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5a8003a60 0x7fc5a8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5bb59e700 1 -- 172.21.15.92:0/3072593320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5a8004810 msgr2=0x7fc5a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5bb59e700 1 --2- 172.21.15.92:0/3072593320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5a8004810 0x7fc5a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5bb59e700 1 -- 172.21.15.92:0/3072593320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a8003570 con 0x7fc5a80051c0 2023-10-06T10:41:09.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5bad9d700 1 --2- 172.21.15.92:0/3072593320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5a8003a60 0x7fc5a8003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:09.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5b259c700 1 --2- 172.21.15.92:0/3072593320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5a8004810 0x7fc5a8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:09.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5bb59e700 1 --2- 172.21.15.92:0/3072593320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a80051c0 0x7fc5a800ba60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0cd8d0 tx=0x7fc5b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430b7f4ac3773667 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5ba59c700 1 -- 172.21.15.92:0/3072593320 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b400ea60 con 0x7fc5a80051c0 2023-10-06T10:41:09.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5ba59c700 1 -- 172.21.15.92:0/3072593320 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b400f040 con 0x7fc5a80051c0 2023-10-06T10:41:09.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.936+0000 7fc5ba59c700 1 -- 172.21.15.92:0/3072593320 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b400edf0 con 0x7fc5a80051c0 2023-10-06T10:41:09.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.937+0000 7fc5c1118700 1 -- 172.21.15.92:0/3072593320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a80051c0 msgr2=0x7fc5a800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.937+0000 7fc5c1118700 1 --2- 172.21.15.92:0/3072593320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a80051c0 0x7fc5a800ba60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0cd8d0 tx=0x7fc5b400b040 comp rx=0 tx=0).stop 2023-10-06T10:41:09.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.937+0000 7fc5bb59e700 1 -- 172.21.15.92:0/3072593320 reap_dead start 2023-10-06T10:41:09.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.937+0000 7fc5c1118700 1 -- 172.21.15.92:0/3072593320 shutdown_connections 2023-10-06T10:41:09.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.937+0000 7fc5c1118700 1 -- 172.21.15.92:0/3072593320 >> 172.21.15.92:0/3072593320 conn(0x7fc5bc099700 msgr2=0x7fc5bc06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.937+0000 7fc5c1118700 1 -- 172.21.15.92:0/3072593320 shutdown_connections 2023-10-06T10:41:09.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.937+0000 7fc5c1118700 1 -- 172.21.15.92:0/3072593320 wait complete. 2023-10-06T10:41:09.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.938+0000 7fc5c1118700 1 Processor -- start 2023-10-06T10:41:09.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.938+0000 7fc5c1118700 1 -- start start 2023-10-06T10:41:09.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.939+0000 7fc5c1118700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5a80051c0 0x7fc5bc156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.939+0000 7fc5c1118700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5bc14f550 0x7fc5bc0b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.939+0000 7fc5c1118700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5bc0b4f70 0x7fc5bc141600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.939+0000 7fc5c1118700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0b3620 con 0x7fc5bc14f550 2023-10-06T10:41:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.939+0000 7fc5c1118700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc1542d0 con 0x7fc5bc0b4f70 2023-10-06T10:41:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.939+0000 7fc5c1118700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc154b70 con 0x7fc5a80051c0 2023-10-06T10:41:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.939+0000 7fc5b259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5bc14f550 0x7fc5bc0b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.939+0000 7fc5b259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5bc14f550 0x7fc5bc0b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59928/0 (socket says 172.21.15.92:59928) 2023-10-06T10:41:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.939+0000 7fc5b259c700 1 -- 172.21.15.92:0/2622610736 learned_addr learned my addr 172.21.15.92:0/2622610736 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.939+0000 7fc5bb59e700 1 --2- 172.21.15.92:0/2622610736 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5bc0b4f70 0x7fc5bc141600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.939+0000 7fc5bad9d700 1 --2- 172.21.15.92:0/2622610736 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5a80051c0 0x7fc5bc156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.939+0000 7fc5b259c700 1 -- 172.21.15.92:0/2622610736 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5a80051c0 msgr2=0x7fc5bc156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.939+0000 7fc5b259c700 1 --2- 172.21.15.92:0/2622610736 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5a80051c0 0x7fc5bc156a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.939+0000 7fc5b259c700 1 -- 172.21.15.92:0/2622610736 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5bc0b4f70 msgr2=0x7fc5bc141600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.939+0000 7fc5b259c700 1 --2- 172.21.15.92:0/2622610736 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5bc0b4f70 0x7fc5bc141600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.939+0000 7fc5b259c700 1 -- 172.21.15.92:0/2622610736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc152560 con 0x7fc5bc14f550 2023-10-06T10:41:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.939+0000 7fc5bb59e700 1 --2- 172.21.15.92:0/2622610736 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5bc0b4f70 0x7fc5bc141600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.939+0000 7fc5bad9d700 1 --2- 172.21.15.92:0/2622610736 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5a80051c0 0x7fc5bc156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.939+0000 7fc5b259c700 1 --2- 172.21.15.92:0/2622610736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5bc14f550 0x7fc5bc0b4a30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4007380 tx=0x7fc5a4001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.940+0000 7fc5b37fe700 1 -- 172.21.15.92:0/2622610736 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a400e7d0 con 0x7fc5bc14f550 2023-10-06T10:41:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.940+0000 7fc5b37fe700 1 -- 172.21.15.92:0/2622610736 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a4010040 con 0x7fc5bc14f550 2023-10-06T10:41:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.940+0000 7fc5c1118700 1 -- 172.21.15.92:0/2622610736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5bc154520 con 0x7fc5bc14f550 2023-10-06T10:41:09.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.940+0000 7fc5b37fe700 1 -- 172.21.15.92:0/2622610736 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a400eba0 con 0x7fc5bc14f550 2023-10-06T10:41:09.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.940+0000 7fc5c1118700 1 -- 172.21.15.92:0/2622610736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5bc0b53e0 con 0x7fc5bc14f550 2023-10-06T10:41:09.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.941+0000 7fc5b37fe700 1 -- 172.21.15.92:0/2622610736 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc5a401d400 con 0x7fc5bc14f550 2023-10-06T10:41:09.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.941+0000 7fc5b37fe700 1 --2- 172.21.15.92:0/2622610736 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc59c066fb0 0x7fc59c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.942+0000 7fc5b37fe700 1 -- 172.21.15.92:0/2622610736 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc5a4084230 con 0x7fc5bc14f550 2023-10-06T10:41:09.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.942+0000 7fc5bad9d700 1 --2- 172.21.15.92:0/2622610736 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc59c066fb0 0x7fc59c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.942+0000 7fc5bad9d700 1 --2- 172.21.15.92:0/2622610736 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc59c066fb0 0x7fc59c069460 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009390 tx=0x7fc5ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.943+0000 7fc5ca410b80 1 Processor -- start 2023-10-06T10:41:09.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.944+0000 7fc5ca410b80 1 -- start start 2023-10-06T10:41:09.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.944+0000 7fc5ca410b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55634c36fad0 0x55634c36fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.944+0000 7fc5ca410b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55634c32b9e0 0x55634c3285c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.944+0000 7fc5ca410b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c329c10 0x55634c328b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.944+0000 7fc5ca410b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634c2a8cc0 con 0x55634c329c10 2023-10-06T10:41:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.944+0000 7fc5ca410b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634c27fab0 con 0x55634c36fad0 2023-10-06T10:41:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.944+0000 7fc5ca410b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55634c27d2b0 con 0x55634c32b9e0 2023-10-06T10:41:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.945+0000 7fc5bb59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c329c10 0x55634c328b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.945+0000 7fc5bb59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c329c10 0x55634c328b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59934/0 (socket says 172.21.15.92:59934) 2023-10-06T10:41:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.945+0000 7fc5bb59e700 1 -- 172.21.15.92:0/1441943340 learned_addr learned my addr 172.21.15.92:0/1441943340 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.945+0000 7fc5b259c700 1 --2- 172.21.15.92:0/1441943340 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55634c32b9e0 0x55634c3285c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.945+0000 7fc5bad9d700 1 --2- 172.21.15.92:0/1441943340 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55634c36fad0 0x55634c36fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.945+0000 7fc5bb59e700 1 -- 172.21.15.92:0/1441943340 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55634c32b9e0 msgr2=0x55634c3285c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.945+0000 7fc5bb59e700 1 --2- 172.21.15.92:0/1441943340 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55634c32b9e0 0x55634c3285c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.945+0000 7fc5bb59e700 1 -- 172.21.15.92:0/1441943340 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55634c36fad0 msgr2=0x55634c36fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.945+0000 7fc5bb59e700 1 --2- 172.21.15.92:0/1441943340 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55634c36fad0 0x55634c36fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.945+0000 7fc5bb59e700 1 -- 172.21.15.92:0/1441943340 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55634c1132b0 con 0x55634c329c10 2023-10-06T10:41:09.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.945+0000 7fc5bb59e700 1 --2- 172.21.15.92:0/1441943340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c329c10 0x55634c328b00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8004650 tx=0x7fc5b401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39f155253e826c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.945+0000 7fc59bfff700 1 -- 172.21.15.92:0/1441943340 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4023070 con 0x55634c329c10 2023-10-06T10:41:09.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.945+0000 7fc59bfff700 1 -- 172.21.15.92:0/1441943340 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b4009ac0 con 0x55634c329c10 2023-10-06T10:41:09.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.945+0000 7fc59bfff700 1 -- 172.21.15.92:0/1441943340 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b400ec70 con 0x55634c329c10 2023-10-06T10:41:09.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.946+0000 7fc5ca410b80 1 -- 172.21.15.92:0/1441943340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c329c10 msgr2=0x55634c328b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.946+0000 7fc5ca410b80 1 --2- 172.21.15.92:0/1441943340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c329c10 0x55634c328b00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8004650 tx=0x7fc5b401a000 comp rx=0 tx=0).stop 2023-10-06T10:41:09.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.946+0000 7fc5ca410b80 1 -- 172.21.15.92:0/1441943340 shutdown_connections 2023-10-06T10:41:09.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.946+0000 7fc5ca410b80 1 --2- 172.21.15.92:0/1441943340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c329c10 0x55634c328b00 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.946+0000 7fc5ca410b80 1 --2- 172.21.15.92:0/1441943340 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55634c36fad0 0x55634c36fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.946+0000 7fc5ca410b80 1 --2- 172.21.15.92:0/1441943340 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55634c32b9e0 0x55634c3285c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.946+0000 7fc5ca410b80 1 -- 172.21.15.92:0/1441943340 >> 172.21.15.92:0/1441943340 conn(0x55634c32a0b0 msgr2=0x55634c32a480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.946+0000 7fc5ca410b80 1 -- 172.21.15.92:0/1441943340 shutdown_connections 2023-10-06T10:41:09.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.946+0000 7fc5ca410b80 1 -- 172.21.15.92:0/1441943340 wait complete. 2023-10-06T10:41:09.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.947+0000 7fc5ca410b80 1 -- start start 2023-10-06T10:41:09.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.947+0000 7fc5ca410b80 1 Processor -- start 2023-10-06T10:41:09.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.947+0000 7fc5ca410b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55634c329c10 0x55634c34f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.947+0000 7fc5ca410b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c32b9e0 0x55634c353640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.947+0000 7fc5ca410b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55634c36fad0 0x55634c350b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.947+0000 7fc5ca410b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634c2790a0 con 0x55634c32b9e0 2023-10-06T10:41:09.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.947+0000 7fc5ca410b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634c2248d0 con 0x55634c329c10 2023-10-06T10:41:09.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.947+0000 7fc5ca410b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55634c23fee0 con 0x55634c36fad0 2023-10-06T10:41:09.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5ca410b80 10 client.0 _ll_register_callbacks cb 0x55634c115f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:09.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5b259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c32b9e0 0x55634c353640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5b259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c32b9e0 0x55634c353640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59938/0 (socket says 172.21.15.92:59938) 2023-10-06T10:41:09.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5b259c700 1 -- 172.21.15.92:0/3642183376 learned_addr learned my addr 172.21.15.92:0/3642183376 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:09.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5bad9d700 1 --2- 172.21.15.92:0/3642183376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55634c329c10 0x55634c34f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5bb59e700 1 --2- 172.21.15.92:0/3642183376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55634c36fad0 0x55634c350b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5b259c700 1 -- 172.21.15.92:0/3642183376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55634c36fad0 msgr2=0x55634c350b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5b259c700 1 --2- 172.21.15.92:0/3642183376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55634c36fad0 0x55634c350b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5b259c700 1 -- 172.21.15.92:0/3642183376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55634c329c10 msgr2=0x55634c34f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5b259c700 1 --2- 172.21.15.92:0/3642183376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55634c329c10 0x55634c34f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5b259c700 1 -- 172.21.15.92:0/3642183376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a8003570 con 0x55634c32b9e0 2023-10-06T10:41:09.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5b259c700 1 --2- 172.21.15.92:0/3642183376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c32b9e0 0x55634c353640 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40513d0 tx=0x7fc5a4057040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5aca88dde30450 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5997fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:09.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5997fa700 1 -- 172.21.15.92:0/3642183376 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4058070 con 0x55634c32b9e0 2023-10-06T10:41:09.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5997fa700 1 -- 172.21.15.92:0/3642183376 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc5a405bdc0 con 0x55634c32b9e0 2023-10-06T10:41:09.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5ca410b80 1 -- 172.21.15.92:0/3642183376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55634c31d6d0 con 0x55634c32b9e0 2023-10-06T10:41:09.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5997fa700 1 -- 172.21.15.92:0/3642183376 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5a4082520 con 0x55634c32b9e0 2023-10-06T10:41:09.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5ca410b80 10 client.6141 fetch_fsmap learned FSMap version 39 2023-10-06T10:41:09.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5ca410b80 1 -- 172.21.15.92:0/3642183376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55634c0b6020 con 0x55634c32b9e0 2023-10-06T10:41:09.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5997fa700 1 -- 172.21.15.92:0/3642183376 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a4061080 con 0x55634c32b9e0 2023-10-06T10:41:09.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5997fa700 1 -- 172.21.15.92:0/3642183376 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 39) v1 ==== 3136+0+0 (secure 0 0 0) 0x7fc5a406b100 con 0x55634c32b9e0 2023-10-06T10:41:09.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5ca410b80 10 client.6141 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T10:41:09.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.948+0000 7fc5ca410b80 10 client.6141 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-10-06T10:41:09.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.949+0000 7fc5ca410b80 20 client.6141 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:09.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.949+0000 7fc5ca410b80 1 --2- 172.21.15.92:0/3642183376 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x55634c351040 0x55634c36c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.949+0000 7fc5ca410b80 4 client.6141 mds_command: new command op to 5736 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:41:09.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.949+0000 7fc5ca410b80 1 -- 172.21.15.92:0/3642183376 --> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55634c2fa600 con 0x55634c351040 2023-10-06T10:41:09.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.949+0000 7fc5bad9d700 1 --2- 172.21.15.92:0/3642183376 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x55634c351040 0x55634c36c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.949+0000 7fc5997fa700 10 client.6141 ms_handle_connect on v2:172.21.15.92:6835/2458340622 2023-10-06T10:41:09.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.949+0000 7fc5bad9d700 1 --2- 172.21.15.92:0/3642183376 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x55634c351040 0x55634c36c5a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b0619e1af03e7c4 server_cookie=f28c4a8de6b801fa in_seq=0 out_seq=0 2023-10-06T10:41:09.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.949+0000 7fc5997fa700 1 -- 172.21.15.92:0/3642183376 <== mds.0 v2:172.21.15.92:6835/2458340622 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55634c2fa600 con 0x55634c351040 2023-10-06T10:41:09.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.949+0000 7fc5997fa700 10 client.6141 handle_command_reply: tid=0 2023-10-06T10:41:09.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.949+0000 7fc5ca410b80 1 client.6141 shutdown 2023-10-06T10:41:09.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.949+0000 7fc5ca410b80 1 -- 172.21.15.92:0/3642183376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c32b9e0 msgr2=0x55634c353640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.949+0000 7fc5ca410b80 1 --2- 172.21.15.92:0/3642183376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c32b9e0 0x55634c353640 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40513d0 tx=0x7fc5a4057040 comp rx=0 tx=0).stop 2023-10-06T10:41:09.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.950+0000 7fc5ca410b80 1 -- 172.21.15.92:0/3642183376 shutdown_connections 2023-10-06T10:41:09.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.950+0000 7fc5ca410b80 1 --2- 172.21.15.92:0/3642183376 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x55634c351040 0x55634c36c5a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.950+0000 7fc5ca410b80 1 --2- 172.21.15.92:0/3642183376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55634c36fad0 0x55634c350b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.950+0000 7fc5997fa700 0 client.6141 ms_handle_reset on v2:172.21.15.92:6835/2458340622 2023-10-06T10:41:09.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.950+0000 7fc5ca410b80 1 --2- 172.21.15.92:0/3642183376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55634c329c10 0x55634c34f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.950+0000 7fc5ca410b80 1 --2- 172.21.15.92:0/3642183376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c32b9e0 0x55634c353640 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.950+0000 7fc5ca410b80 1 -- 172.21.15.92:0/3642183376 >> 172.21.15.92:0/3642183376 conn(0x55634c32a0b0 msgr2=0x55634c36ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.950+0000 7fc5ca410b80 1 -- 172.21.15.92:0/3642183376 shutdown_connections 2023-10-06T10:41:09.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.950+0000 7fc5ca410b80 1 -- 172.21.15.92:0/3642183376 wait complete. 2023-10-06T10:41:09.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.950+0000 7fc5ca410b80 20 client.6141 trim_cache size 0 max 16384 2023-10-06T10:41:09.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.965+0000 7fc5ca410b80 1 Processor -- start 2023-10-06T10:41:09.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.965+0000 7fc5ca410b80 1 -- start start 2023-10-06T10:41:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.966+0000 7fc5ca410b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55634c329c10 0x55634c36d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.966+0000 7fc5ca410b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55634c32b9e0 0x55634c36e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.966+0000 7fc5ca410b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c36fad0 0x55634c36e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.966+0000 7fc5ca410b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634c0fbe70 con 0x55634c36fad0 2023-10-06T10:41:09.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.966+0000 7fc5ca410b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634c36b500 con 0x55634c329c10 2023-10-06T10:41:09.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.966+0000 7fc5ca410b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55634c32ae80 con 0x55634c32b9e0 2023-10-06T10:41:09.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.966+0000 7fc5bb59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c36fad0 0x55634c36e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.966+0000 7fc5bb59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c36fad0 0x55634c36e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59944/0 (socket says 172.21.15.92:59944) 2023-10-06T10:41:09.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.966+0000 7fc5bb59e700 1 -- 172.21.15.92:0/105351015 learned_addr learned my addr 172.21.15.92:0/105351015 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:09.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.966+0000 7fc5bad9d700 1 --2- 172.21.15.92:0/105351015 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55634c329c10 0x55634c36d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.966+0000 7fc5b259c700 1 --2- 172.21.15.92:0/105351015 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55634c32b9e0 0x55634c36e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.966+0000 7fc5bb59e700 1 -- 172.21.15.92:0/105351015 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55634c32b9e0 msgr2=0x55634c36e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.966+0000 7fc5bb59e700 1 --2- 172.21.15.92:0/105351015 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55634c32b9e0 0x55634c36e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.966+0000 7fc5bb59e700 1 -- 172.21.15.92:0/105351015 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55634c329c10 msgr2=0x55634c36d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.966+0000 7fc5bb59e700 1 --2- 172.21.15.92:0/105351015 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55634c329c10 0x55634c36d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.966+0000 7fc5bb59e700 1 -- 172.21.15.92:0/105351015 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55634c2477c0 con 0x55634c36fad0 2023-10-06T10:41:09.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.966+0000 7fc5b259c700 1 --2- 172.21.15.92:0/105351015 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55634c32b9e0 0x55634c36e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:09.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.966+0000 7fc5bb59e700 1 --2- 172.21.15.92:0/105351015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c36fad0 0x55634c36e800 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009c30 tx=0x7fc5b4023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1a3014a42e6fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.966+0000 7fc5b0d99700 1 -- 172.21.15.92:0/105351015 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4030070 con 0x55634c36fad0 2023-10-06T10:41:09.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.967+0000 7fc5b0d99700 1 -- 172.21.15.92:0/105351015 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b400ea60 con 0x55634c36fad0 2023-10-06T10:41:09.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.967+0000 7fc5b0d99700 1 -- 172.21.15.92:0/105351015 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b400ee20 con 0x55634c36fad0 2023-10-06T10:41:09.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.967+0000 7fc5ca410b80 1 -- 172.21.15.92:0/105351015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c36fad0 msgr2=0x55634c36e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.967+0000 7fc5ca410b80 1 --2- 172.21.15.92:0/105351015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c36fad0 0x55634c36e800 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009c30 tx=0x7fc5b4023000 comp rx=0 tx=0).stop 2023-10-06T10:41:09.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.967+0000 7fc5ca410b80 1 -- 172.21.15.92:0/105351015 shutdown_connections 2023-10-06T10:41:09.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.967+0000 7fc5ca410b80 1 --2- 172.21.15.92:0/105351015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c36fad0 0x55634c36e800 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.967+0000 7fc5ca410b80 1 --2- 172.21.15.92:0/105351015 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55634c329c10 0x55634c36d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.967+0000 7fc5ca410b80 1 --2- 172.21.15.92:0/105351015 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55634c32b9e0 0x55634c36e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.967+0000 7fc5ca410b80 1 -- 172.21.15.92:0/105351015 >> 172.21.15.92:0/105351015 conn(0x55634c32a0b0 msgr2=0x55634c3696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.967+0000 7fc5ca410b80 1 -- 172.21.15.92:0/105351015 shutdown_connections 2023-10-06T10:41:09.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.967+0000 7fc5ca410b80 1 -- 172.21.15.92:0/105351015 wait complete. 2023-10-06T10:41:09.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.968+0000 7fc5ca410b80 1 -- start start 2023-10-06T10:41:09.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.968+0000 7fc5ca410b80 1 Processor -- start 2023-10-06T10:41:09.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.968+0000 7fc5ca410b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c329c10 0x55634c36fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.968+0000 7fc5ca410b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55634c32b9e0 0x55634c36ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.968+0000 7fc5ca410b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55634c36fad0 0x55634c325840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5ca410b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634c3d5350 con 0x55634c329c10 2023-10-06T10:41:09.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5ca410b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634c3704a0 con 0x55634c32b9e0 2023-10-06T10:41:09.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5ca410b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55634c36c5f0 con 0x55634c36fad0 2023-10-06T10:41:09.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5ca410b80 10 client.0 _ll_register_callbacks cb 0x55634c115f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:09.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5bad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c329c10 0x55634c36fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5bad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c329c10 0x55634c36fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59950/0 (socket says 172.21.15.92:59950) 2023-10-06T10:41:09.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5bad9d700 1 -- 172.21.15.92:0/1738988430 learned_addr learned my addr 172.21.15.92:0/1738988430 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:09.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5b259c700 1 --2- 172.21.15.92:0/1738988430 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55634c32b9e0 0x55634c36ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5bb59e700 1 --2- 172.21.15.92:0/1738988430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55634c36fad0 0x55634c325840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5bad9d700 1 -- 172.21.15.92:0/1738988430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55634c36fad0 msgr2=0x55634c325840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5bad9d700 1 --2- 172.21.15.92:0/1738988430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55634c36fad0 0x55634c325840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5bad9d700 1 -- 172.21.15.92:0/1738988430 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55634c32b9e0 msgr2=0x55634c36ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:09.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5bad9d700 1 --2- 172.21.15.92:0/1738988430 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55634c32b9e0 0x55634c36ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5bad9d700 1 -- 172.21.15.92:0/1738988430 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55634c1132b0 con 0x55634c329c10 2023-10-06T10:41:09.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5bb59e700 1 --2- 172.21.15.92:0/1738988430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55634c36fad0 0x55634c325840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:09.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5bad9d700 1 --2- 172.21.15.92:0/1738988430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c329c10 0x55634c36fea0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac013010 tx=0x7fc5ac025040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e1751fdc1c48e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:09.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5997fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:09.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5997fa700 1 -- 172.21.15.92:0/1738988430 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac02d620 con 0x55634c329c10 2023-10-06T10:41:09.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5997fa700 1 -- 172.21.15.92:0/1738988430 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc5ac01c040 con 0x55634c329c10 2023-10-06T10:41:09.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5ca410b80 1 -- 172.21.15.92:0/1738988430 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5b4009ac0 con 0x55634c329c10 2023-10-06T10:41:09.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5997fa700 1 -- 172.21.15.92:0/1738988430 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5ac02d990 con 0x55634c329c10 2023-10-06T10:41:09.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5ca410b80 10 client.6147 fetch_fsmap learned FSMap version 39 2023-10-06T10:41:09.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5ca410b80 1 -- 172.21.15.92:0/1738988430 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55634c22e4b0 con 0x55634c329c10 2023-10-06T10:41:09.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5997fa700 1 -- 172.21.15.92:0/1738988430 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b4009ac0 con 0x55634c329c10 2023-10-06T10:41:09.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.969+0000 7fc5997fa700 1 -- 172.21.15.92:0/1738988430 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 39) v1 ==== 3136+0+0 (secure 0 0 0) 0x7fc5ac02daf0 con 0x55634c329c10 2023-10-06T10:41:09.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5ca410b80 10 client.6147 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T10:41:09.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5ca410b80 10 client.6147 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-10-06T10:41:09.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5ca410b80 20 client.6147 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:09.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5ca410b80 1 --2- 172.21.15.92:0/1738988430 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x55634c36e2c0 0x55634c325d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:09.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5ca410b80 4 client.6147 mds_command: new command op to 5736 tid=0[{"prefix": "scrub abort"}] 2023-10-06T10:41:09.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5ca410b80 1 -- 172.21.15.92:0/1738988430 --> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x55634c21be40 con 0x55634c36e2c0 2023-10-06T10:41:09.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5b259c700 1 --2- 172.21.15.92:0/1738988430 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x55634c36e2c0 0x55634c325d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:09.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5997fa700 10 client.6147 ms_handle_connect on v2:172.21.15.92:6835/2458340622 2023-10-06T10:41:09.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5b259c700 1 --2- 172.21.15.92:0/1738988430 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x55634c36e2c0 0x55634c325d80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85c53c8e19027fce server_cookie=71bc150e9f263e99 in_seq=0 out_seq=0 2023-10-06T10:41:09.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5997fa700 1 -- 172.21.15.92:0/1738988430 <== mds.0 v2:172.21.15.92:6835/2458340622 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55634c21be40 con 0x55634c36e2c0 2023-10-06T10:41:09.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5997fa700 10 client.6147 handle_command_reply: tid=0 2023-10-06T10:41:09.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5ca410b80 1 client.6147 shutdown 2023-10-06T10:41:09.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5ca410b80 1 -- 172.21.15.92:0/1738988430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c329c10 msgr2=0x55634c36fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5ca410b80 1 --2- 172.21.15.92:0/1738988430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c329c10 0x55634c36fea0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac013010 tx=0x7fc5ac025040 comp rx=0 tx=0).stop 2023-10-06T10:41:09.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5ca410b80 1 -- 172.21.15.92:0/1738988430 shutdown_connections 2023-10-06T10:41:09.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5ca410b80 1 --2- 172.21.15.92:0/1738988430 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x55634c36e2c0 0x55634c325d80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5ca410b80 1 --2- 172.21.15.92:0/1738988430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55634c36fad0 0x55634c325840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5ca410b80 1 --2- 172.21.15.92:0/1738988430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55634c329c10 0x55634c36fea0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5997fa700 0 client.6147 ms_handle_reset on v2:172.21.15.92:6835/2458340622 2023-10-06T10:41:09.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5ca410b80 1 --2- 172.21.15.92:0/1738988430 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55634c32b9e0 0x55634c36ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5ca410b80 1 -- 172.21.15.92:0/1738988430 >> 172.21.15.92:0/1738988430 conn(0x55634c32a0b0 msgr2=0x55634c324fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5ca410b80 1 -- 172.21.15.92:0/1738988430 shutdown_connections 2023-10-06T10:41:09.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.970+0000 7fc5ca410b80 1 -- 172.21.15.92:0/1738988430 wait complete. 2023-10-06T10:41:09.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.971+0000 7fc5ca410b80 20 client.6147 trim_cache size 0 max 16384 2023-10-06T10:41:09.980 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:41:09.980 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:41:09.980 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:41:09.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.971+0000 7fc5c1118700 1 -- 172.21.15.92:0/2622610736 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc59c066fb0 msgr2=0x7fc59c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.971+0000 7fc5c1118700 1 --2- 172.21.15.92:0/2622610736 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc59c066fb0 0x7fc59c069460 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009390 tx=0x7fc5ac008040 comp rx=0 tx=0).stop 2023-10-06T10:41:09.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.971+0000 7fc5c1118700 1 -- 172.21.15.92:0/2622610736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5bc14f550 msgr2=0x7fc5bc0b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:09.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.971+0000 7fc5c1118700 1 --2- 172.21.15.92:0/2622610736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5bc14f550 0x7fc5bc0b4a30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4007380 tx=0x7fc5a4001230 comp rx=0 tx=0).stop 2023-10-06T10:41:09.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.971+0000 7fc5c1118700 1 -- 172.21.15.92:0/2622610736 shutdown_connections 2023-10-06T10:41:09.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.971+0000 7fc5c1118700 1 --2- 172.21.15.92:0/2622610736 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc59c066fb0 0x7fc59c069460 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.971+0000 7fc5c1118700 1 --2- 172.21.15.92:0/2622610736 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5bc0b4f70 0x7fc5bc141600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.971+0000 7fc5c1118700 1 --2- 172.21.15.92:0/2622610736 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5a80051c0 0x7fc5bc156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.971+0000 7fc5c1118700 1 --2- 172.21.15.92:0/2622610736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5bc14f550 0x7fc5bc0b4a30 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:09.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.971+0000 7fc5c1118700 1 -- 172.21.15.92:0/2622610736 >> 172.21.15.92:0/2622610736 conn(0x7fc5bc099700 msgr2=0x7fc5bc069400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:09.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.971+0000 7fc5c1118700 1 -- 172.21.15.92:0/2622610736 shutdown_connections 2023-10-06T10:41:09.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:09.971+0000 7fc5c1118700 1 -- 172.21.15.92:0/2622610736 wait complete. 2023-10-06T10:41:09.984 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:1 scrub status 2023-10-06T10:41:10.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.097+0000 7f768f59e700 1 Processor -- start 2023-10-06T10:41:10.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.097+0000 7f768f59e700 1 -- start start 2023-10-06T10:41:10.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.098+0000 7f768f59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f76900cd3c0 0x7f76900ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:10.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.098+0000 7f768f59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76900cc970 con 0x7f76900b29f0 2023-10-06T10:41:10.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.098+0000 7f768f59e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76900cbd60 con 0x7f76900b2620 2023-10-06T10:41:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.098+0000 7f768f59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76900cc040 con 0x7f76900cd3c0 2023-10-06T10:41:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.098+0000 7f7685d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f76900cd3c0 0x7f76900ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.098+0000 7f768ed9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f76900b29f0 0x7f76900c5350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59222/0 (socket says 172.21.15.92:59222) 2023-10-06T10:41:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.098+0000 7f768ed9d700 1 -- 172.21.15.92:0/3912570369 learned_addr learned my addr 172.21.15.92:0/3912570369 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.098+0000 7f7685d9b700 1 -- 172.21.15.92:0/3912570369 >> v1:172.21.15.124:6789/0 conn(0x7f76900b2620 legacy=0x7f76900cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.098+0000 7f7685d9b700 1 -- 172.21.15.92:0/3912570369 >> v1:172.21.15.92:6789/0 conn(0x7f76900b29f0 legacy=0x7f76900c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.098+0000 7f7685d9b700 1 -- 172.21.15.92:0/3912570369 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76900cc320 con 0x7f76900cd3c0 2023-10-06T10:41:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.098+0000 7f768dd9b700 1 -- 172.21.15.92:0/3912570369 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739917739 0 0) 0x7f76900cc970 con 0x7f76900b29f0 2023-10-06T10:41:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.099+0000 7f7685d9b700 1 --2- 172.21.15.92:0/3912570369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76900cd3c0 0x7f76900ceaa0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f76780098e0 tx=0x7f76780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a9b16a90d97b4d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.099+0000 7f768dd9b700 1 -- 172.21.15.92:0/3912570369 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7678009e80 con 0x7f76900cd3c0 2023-10-06T10:41:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.099+0000 7f768dd9b700 1 -- 172.21.15.92:0/3912570369 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767800b860 con 0x7f76900cd3c0 2023-10-06T10:41:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.099+0000 7f768dd9b700 1 -- 172.21.15.92:0/3912570369 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767800bbe0 con 0x7f76900cd3c0 2023-10-06T10:41:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.099+0000 7f768f59e700 1 -- 172.21.15.92:0/3912570369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76900cd3c0 msgr2=0x7f76900ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.099+0000 7f768f59e700 1 --2- 172.21.15.92:0/3912570369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76900cd3c0 0x7f76900ceaa0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f76780098e0 tx=0x7f76780091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.099+0000 7f768f59e700 1 -- 172.21.15.92:0/3912570369 shutdown_connections 2023-10-06T10:41:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.099+0000 7f768f59e700 1 --2- 172.21.15.92:0/3912570369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76900cd3c0 0x7f76900ceaa0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.099+0000 7f768f59e700 1 -- 172.21.15.92:0/3912570369 >> 172.21.15.92:0/3912570369 conn(0x7f7690099520 msgr2=0x7f769006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.099+0000 7f768f59e700 1 -- 172.21.15.92:0/3912570369 shutdown_connections 2023-10-06T10:41:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.099+0000 7f768f59e700 1 -- 172.21.15.92:0/3912570369 wait complete. 2023-10-06T10:41:10.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.101+0000 7f768f59e700 1 Processor -- start 2023-10-06T10:41:10.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.101+0000 7f768f59e700 1 -- start start 2023-10-06T10:41:10.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.101+0000 7f768f59e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76900b2620 0x7f769014fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:10.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.101+0000 7f768f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76900b29f0 0x7f7690159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:10.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.101+0000 7f768f59e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76900cd3c0 0x7f76900c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:10.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.101+0000 7f768f59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769016dc00 con 0x7f76900b29f0 2023-10-06T10:41:10.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.101+0000 7f768f59e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76900b3230 con 0x7f76900cd3c0 2023-10-06T10:41:10.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.101+0000 7f768f59e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76900b3510 con 0x7f76900b2620 2023-10-06T10:41:10.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.101+0000 7f7685d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76900b29f0 0x7f7690159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:10.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.101+0000 7f7685d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76900b29f0 0x7f7690159e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59962/0 (socket says 172.21.15.92:59962) 2023-10-06T10:41:10.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.101+0000 7f7685d9b700 1 -- 172.21.15.92:0/2554002069 learned_addr learned my addr 172.21.15.92:0/2554002069 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:10.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.102+0000 7f768ed9d700 1 --2- 172.21.15.92:0/2554002069 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76900cd3c0 0x7f76900c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:10.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.102+0000 7f768e59c700 1 --2- 172.21.15.92:0/2554002069 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76900b2620 0x7f769014fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:10.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.102+0000 7f7685d9b700 1 -- 172.21.15.92:0/2554002069 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76900b2620 msgr2=0x7f769014fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:10.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.102+0000 7f7685d9b700 1 --2- 172.21.15.92:0/2554002069 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76900b2620 0x7f769014fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.102+0000 7f7685d9b700 1 -- 172.21.15.92:0/2554002069 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76900cd3c0 msgr2=0x7f76900c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:10.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.102+0000 7f7685d9b700 1 --2- 172.21.15.92:0/2554002069 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76900cd3c0 0x7f76900c7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.102+0000 7f7685d9b700 1 -- 172.21.15.92:0/2554002069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7678009420 con 0x7f76900b29f0 2023-10-06T10:41:10.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.102+0000 7f7685d9b700 1 --2- 172.21.15.92:0/2554002069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76900b29f0 0x7f7690159e80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7678013000 tx=0x7f7678017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:10.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.102+0000 7f7686ffd700 1 -- 172.21.15.92:0/2554002069 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7678011420 con 0x7f76900b29f0 2023-10-06T10:41:10.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.102+0000 7f7686ffd700 1 -- 172.21.15.92:0/2554002069 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7678011580 con 0x7f76900b29f0 2023-10-06T10:41:10.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.102+0000 7f768f59e700 1 -- 172.21.15.92:0/2554002069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76900b37f0 con 0x7f76900b29f0 2023-10-06T10:41:10.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.102+0000 7f7686ffd700 1 -- 172.21.15.92:0/2554002069 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7678011940 con 0x7f76900b29f0 2023-10-06T10:41:10.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.102+0000 7f768f59e700 1 -- 172.21.15.92:0/2554002069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76900c8010 con 0x7f76900b29f0 2023-10-06T10:41:10.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.103+0000 7f7686ffd700 1 -- 172.21.15.92:0/2554002069 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f76780253f0 con 0x7f76900b29f0 2023-10-06T10:41:10.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.104+0000 7f7686ffd700 1 --2- 172.21.15.92:0/2554002069 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7670066f30 0x7f76700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:10.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.104+0000 7f768e59c700 1 --2- 172.21.15.92:0/2554002069 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7670066f30 0x7f76700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:10.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.104+0000 7f7686ffd700 1 -- 172.21.15.92:0/2554002069 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f76780863a0 con 0x7f76900b29f0 2023-10-06T10:41:10.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.104+0000 7f768e59c700 1 --2- 172.21.15.92:0/2554002069 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7670066f30 0x7f76700693e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7690158d40 tx=0x7f7680009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:10.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.106+0000 7f769e0dfb80 1 Processor -- start 2023-10-06T10:41:10.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.106+0000 7f769e0dfb80 1 -- start start 2023-10-06T10:41:10.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.107+0000 7f769e0dfb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ec99f0e9a0 0x55ec99f0ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:10.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.107+0000 7f769e0dfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f58c10 0x55ec99f37970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:10.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.107+0000 7f769e0dfb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ec99f37eb0 0x55ec99f0a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:10.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.107+0000 7f769e0dfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec99e8ecd0 con 0x55ec99f58c10 2023-10-06T10:41:10.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.107+0000 7f769e0dfb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec99e65ac0 con 0x55ec99f37eb0 2023-10-06T10:41:10.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.107+0000 7f769e0dfb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec99e632c0 con 0x55ec99f0e9a0 2023-10-06T10:41:10.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.107+0000 7f7685d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f58c10 0x55ec99f37970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:10.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.107+0000 7f7685d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f58c10 0x55ec99f37970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59978/0 (socket says 172.21.15.92:59978) 2023-10-06T10:41:10.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.107+0000 7f7685d9b700 1 -- 172.21.15.92:0/1106260001 learned_addr learned my addr 172.21.15.92:0/1106260001 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:10.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.107+0000 7f768e59c700 1 --2- 172.21.15.92:0/1106260001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ec99f0e9a0 0x55ec99f0ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:10.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.107+0000 7f768ed9d700 1 --2- 172.21.15.92:0/1106260001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ec99f37eb0 0x55ec99f0a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:10.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.107+0000 7f7685d9b700 1 -- 172.21.15.92:0/1106260001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ec99f0e9a0 msgr2=0x55ec99f0ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:10.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.107+0000 7f7685d9b700 1 --2- 172.21.15.92:0/1106260001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ec99f0e9a0 0x55ec99f0ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.107+0000 7f7685d9b700 1 -- 172.21.15.92:0/1106260001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ec99f37eb0 msgr2=0x55ec99f0a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:10.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.107+0000 7f7685d9b700 1 --2- 172.21.15.92:0/1106260001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ec99f37eb0 0x55ec99f0a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.107+0000 7f7685d9b700 1 -- 172.21.15.92:0/1106260001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec99cf92c0 con 0x55ec99f58c10 2023-10-06T10:41:10.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.107+0000 7f7685d9b700 1 --2- 172.21.15.92:0/1106260001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f58c10 0x55ec99f37970 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f76780845c0 tx=0x7f767805c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb160750da4823a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:10.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.107+0000 7f766f7fe700 1 -- 172.21.15.92:0/1106260001 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767805a030 con 0x55ec99f58c10 2023-10-06T10:41:10.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.108+0000 7f766f7fe700 1 -- 172.21.15.92:0/1106260001 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76780576d0 con 0x55ec99f58c10 2023-10-06T10:41:10.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.108+0000 7f766f7fe700 1 -- 172.21.15.92:0/1106260001 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7678056fd0 con 0x55ec99f58c10 2023-10-06T10:41:10.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.108+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1106260001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f58c10 msgr2=0x55ec99f37970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:10.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.108+0000 7f769e0dfb80 1 --2- 172.21.15.92:0/1106260001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f58c10 0x55ec99f37970 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f76780845c0 tx=0x7f767805c040 comp rx=0 tx=0).stop 2023-10-06T10:41:10.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.108+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1106260001 shutdown_connections 2023-10-06T10:41:10.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.108+0000 7f769e0dfb80 1 --2- 172.21.15.92:0/1106260001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ec99f37eb0 0x55ec99f0a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.108+0000 7f769e0dfb80 1 --2- 172.21.15.92:0/1106260001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ec99f0e9a0 0x55ec99f0ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.108+0000 7f769e0dfb80 1 --2- 172.21.15.92:0/1106260001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f58c10 0x55ec99f37970 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.108+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1106260001 >> 172.21.15.92:0/1106260001 conn(0x55ec99f102c0 msgr2=0x55ec99f10690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:10.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.108+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1106260001 shutdown_connections 2023-10-06T10:41:10.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.108+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1106260001 wait complete. 2023-10-06T10:41:10.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.109+0000 7f769e0dfb80 1 -- start start 2023-10-06T10:41:10.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.109+0000 7f769e0dfb80 1 Processor -- start 2023-10-06T10:41:10.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.110+0000 7f769e0dfb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ec99f0e9a0 0x55ec99f4e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:10.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.110+0000 7f769e0dfb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ec99f37eb0 0x55ec99f34160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:10.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.110+0000 7f769e0dfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f58c10 0x55ec99f051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:10.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.110+0000 7f769e0dfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec99e5f0b0 con 0x55ec99f58c10 2023-10-06T10:41:10.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.110+0000 7f769e0dfb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec99e0a8e0 con 0x55ec99f37eb0 2023-10-06T10:41:10.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.110+0000 7f769e0dfb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec99e25ef0 con 0x55ec99f0e9a0 2023-10-06T10:41:10.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.110+0000 7f769e0dfb80 10 client.0 _ll_register_callbacks cb 0x55ec99cfbf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:10.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.110+0000 7f768ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f58c10 0x55ec99f051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:10.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.110+0000 7f768ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f58c10 0x55ec99f051f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59992/0 (socket says 172.21.15.92:59992) 2023-10-06T10:41:10.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.110+0000 7f768ed9d700 1 -- 172.21.15.92:0/1522686535 learned_addr learned my addr 172.21.15.92:0/1522686535 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:10.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.110+0000 7f7685d9b700 1 --2- 172.21.15.92:0/1522686535 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ec99f37eb0 0x55ec99f34160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:10.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.110+0000 7f768e59c700 1 --2- 172.21.15.92:0/1522686535 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ec99f0e9a0 0x55ec99f4e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:10.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.110+0000 7f768ed9d700 1 -- 172.21.15.92:0/1522686535 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ec99f0e9a0 msgr2=0x55ec99f4e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:10.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.110+0000 7f768ed9d700 1 --2- 172.21.15.92:0/1522686535 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ec99f0e9a0 0x55ec99f4e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.110+0000 7f768ed9d700 1 -- 172.21.15.92:0/1522686535 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ec99f37eb0 msgr2=0x55ec99f34160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:10.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.110+0000 7f768ed9d700 1 --2- 172.21.15.92:0/1522686535 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ec99f37eb0 0x55ec99f34160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.110+0000 7f768ed9d700 1 -- 172.21.15.92:0/1522686535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76900c8010 con 0x55ec99f58c10 2023-10-06T10:41:10.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.110+0000 7f7685d9b700 1 --2- 172.21.15.92:0/1522686535 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ec99f37eb0 0x55ec99f34160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:10.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.110+0000 7f768ed9d700 1 --2- 172.21.15.92:0/1522686535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f58c10 0x55ec99f051f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f76880109c0 tx=0x7f7688009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db60037f8b8eb2de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:10.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.110+0000 7f766cff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:10.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.111+0000 7f766cff9700 1 -- 172.21.15.92:0/1522686535 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7688001c90 con 0x55ec99f58c10 2023-10-06T10:41:10.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.111+0000 7f766cff9700 1 -- 172.21.15.92:0/1522686535 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f768800e8c0 con 0x55ec99f58c10 2023-10-06T10:41:10.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.111+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1522686535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec99f036e0 con 0x55ec99f58c10 2023-10-06T10:41:10.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.111+0000 7f766cff9700 1 -- 172.21.15.92:0/1522686535 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f768800ec90 con 0x55ec99f58c10 2023-10-06T10:41:10.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.111+0000 7f769e0dfb80 10 client.6162 fetch_fsmap learned FSMap version 39 2023-10-06T10:41:10.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.111+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1522686535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55ec99c9c030 con 0x55ec99f58c10 2023-10-06T10:41:10.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.111+0000 7f766cff9700 1 -- 172.21.15.92:0/1522686535 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ec99f036e0 con 0x55ec99f58c10 2023-10-06T10:41:10.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.111+0000 7f766cff9700 1 -- 172.21.15.92:0/1522686535 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 39) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f7688012040 con 0x55ec99f58c10 2023-10-06T10:41:10.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.111+0000 7f769e0dfb80 10 client.6162 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T10:41:10.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.111+0000 7f769e0dfb80 10 client.6162 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2023-10-06T10:41:10.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.111+0000 7f769e0dfb80 20 client.6162 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:10.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.111+0000 7f769e0dfb80 1 --2- 172.21.15.92:0/1522686535 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x55ec99f05780 0x55ec99f36060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:10.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.111+0000 7f769e0dfb80 4 client.6162 mds_command: new command op to 5733 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:41:10.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.111+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1522686535 --> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec99ee0610 con 0x55ec99f05780 2023-10-06T10:41:10.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.111+0000 7f768e59c700 1 --2- 172.21.15.92:0/1522686535 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x55ec99f05780 0x55ec99f36060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:10.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.111+0000 7f766cff9700 10 client.6162 ms_handle_connect on v2:172.21.15.92:6834/831923836 2023-10-06T10:41:10.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.111+0000 7f768e59c700 1 --2- 172.21.15.92:0/1522686535 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x55ec99f05780 0x55ec99f36060 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28168f2021c36bab server_cookie=7e5ec15104521e48 in_seq=0 out_seq=0 2023-10-06T10:41:10.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.112+0000 7f766cff9700 1 -- 172.21.15.92:0/1522686535 <== mds.1 v2:172.21.15.92:6834/831923836 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55ec99ee0610 con 0x55ec99f05780 2023-10-06T10:41:10.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.112+0000 7f766cff9700 10 client.6162 handle_command_reply: tid=0 2023-10-06T10:41:10.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.112+0000 7f769e0dfb80 1 client.6162 shutdown 2023-10-06T10:41:10.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.112+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1522686535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f58c10 msgr2=0x55ec99f051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:10.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.112+0000 7f769e0dfb80 1 --2- 172.21.15.92:0/1522686535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f58c10 0x55ec99f051f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f76880109c0 tx=0x7f7688009040 comp rx=0 tx=0).stop 2023-10-06T10:41:10.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.112+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1522686535 shutdown_connections 2023-10-06T10:41:10.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.112+0000 7f769e0dfb80 1 --2- 172.21.15.92:0/1522686535 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x55ec99f05780 0x55ec99f36060 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.112+0000 7f769e0dfb80 1 --2- 172.21.15.92:0/1522686535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f58c10 0x55ec99f051f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.112+0000 7f766cff9700 0 client.6162 ms_handle_reset on v2:172.21.15.92:6834/831923836 2023-10-06T10:41:10.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.112+0000 7f769e0dfb80 1 --2- 172.21.15.92:0/1522686535 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ec99f0e9a0 0x55ec99f4e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.112+0000 7f769e0dfb80 1 --2- 172.21.15.92:0/1522686535 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ec99f37eb0 0x55ec99f34160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.112+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1522686535 >> 172.21.15.92:0/1522686535 conn(0x55ec99f102c0 msgr2=0x55ec99f348b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:10.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.112+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1522686535 shutdown_connections 2023-10-06T10:41:10.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.112+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1522686535 wait complete. 2023-10-06T10:41:10.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.112+0000 7f769e0dfb80 20 client.6162 trim_cache size 0 max 16384 2023-10-06T10:41:10.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.128+0000 7f769e0dfb80 1 Processor -- start 2023-10-06T10:41:10.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.129+0000 7f769e0dfb80 1 -- start start 2023-10-06T10:41:10.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.129+0000 7f769e0dfb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ec99f0e9a0 0x55ec99f4e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:10.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.129+0000 7f769e0dfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f37eb0 0x55ec99f53d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:10.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.129+0000 7f769e0dfb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ec99f58c10 0x55ec99f34160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:10.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.129+0000 7f769e0dfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec99ce1e80 con 0x55ec99f37eb0 2023-10-06T10:41:10.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.129+0000 7f769e0dfb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec99f53530 con 0x55ec99f0e9a0 2023-10-06T10:41:10.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.129+0000 7f769e0dfb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec99eee080 con 0x55ec99f58c10 2023-10-06T10:41:10.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.129+0000 7f7685d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f37eb0 0x55ec99f53d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:10.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.129+0000 7f7685d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f37eb0 0x55ec99f53d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60000/0 (socket says 172.21.15.92:60000) 2023-10-06T10:41:10.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.129+0000 7f7685d9b700 1 -- 172.21.15.92:0/1270186082 learned_addr learned my addr 172.21.15.92:0/1270186082 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:10.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.130+0000 7f7685d9b700 1 -- 172.21.15.92:0/1270186082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ec99f58c10 msgr2=0x55ec99f34160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.130+0000 7f7685d9b700 1 --2- 172.21.15.92:0/1270186082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ec99f58c10 0x55ec99f34160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.130+0000 7f7685d9b700 1 -- 172.21.15.92:0/1270186082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ec99f0e9a0 msgr2=0x55ec99f4e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.130+0000 7f7685d9b700 1 --2- 172.21.15.92:0/1270186082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ec99f0e9a0 0x55ec99f4e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.130+0000 7f7685d9b700 1 -- 172.21.15.92:0/1270186082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec99e2d7d0 con 0x55ec99f37eb0 2023-10-06T10:41:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.130+0000 7f7685d9b700 1 --2- 172.21.15.92:0/1270186082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f37eb0 0x55ec99f53d90 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7678056cd0 tx=0x7f767802f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e486e8991ac286 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.130+0000 7f766ffff700 1 -- 172.21.15.92:0/1270186082 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7678053310 con 0x55ec99f37eb0 2023-10-06T10:41:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.130+0000 7f766ffff700 1 -- 172.21.15.92:0/1270186082 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7678056b10 con 0x55ec99f37eb0 2023-10-06T10:41:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.130+0000 7f766ffff700 1 -- 172.21.15.92:0/1270186082 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767805a220 con 0x55ec99f37eb0 2023-10-06T10:41:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.130+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1270186082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f37eb0 msgr2=0x55ec99f53d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.130+0000 7f769e0dfb80 1 --2- 172.21.15.92:0/1270186082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f37eb0 0x55ec99f53d90 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7678056cd0 tx=0x7f767802f000 comp rx=0 tx=0).stop 2023-10-06T10:41:10.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.130+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1270186082 shutdown_connections 2023-10-06T10:41:10.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.130+0000 7f769e0dfb80 1 --2- 172.21.15.92:0/1270186082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ec99f58c10 0x55ec99f34160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.130+0000 7f769e0dfb80 1 --2- 172.21.15.92:0/1270186082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ec99f0e9a0 0x55ec99f4e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.130+0000 7f769e0dfb80 1 --2- 172.21.15.92:0/1270186082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f37eb0 0x55ec99f53d90 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.130+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1270186082 >> 172.21.15.92:0/1270186082 conn(0x55ec99f102c0 msgr2=0x55ec99f10690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:10.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.130+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1270186082 shutdown_connections 2023-10-06T10:41:10.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.130+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1270186082 wait complete. 2023-10-06T10:41:10.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.131+0000 7f769e0dfb80 1 -- start start 2023-10-06T10:41:10.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.131+0000 7f769e0dfb80 1 Processor -- start 2023-10-06T10:41:10.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.132+0000 7f769e0dfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f0e9a0 0x55ec99f722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:10.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.132+0000 7f769e0dfb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ec99f37eb0 0x55ec99fb40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:10.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.132+0000 7f769e0dfb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ec99f58c10 0x55ec99f3e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:10.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.132+0000 7f769e0dfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec99f35370 con 0x55ec99f0e9a0 2023-10-06T10:41:10.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.132+0000 7f769e0dfb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec99f3ef30 con 0x55ec99f58c10 2023-10-06T10:41:10.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.132+0000 7f769e0dfb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec99fbabc0 con 0x55ec99f37eb0 2023-10-06T10:41:10.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.132+0000 7f769e0dfb80 10 client.0 _ll_register_callbacks cb 0x55ec99cfbf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:10.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.132+0000 7f768e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f0e9a0 0x55ec99f722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:10.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.132+0000 7f768e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f0e9a0 0x55ec99f722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60008/0 (socket says 172.21.15.92:60008) 2023-10-06T10:41:10.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.132+0000 7f768e59c700 1 -- 172.21.15.92:0/1524549368 learned_addr learned my addr 172.21.15.92:0/1524549368 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:10.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.132+0000 7f768e59c700 1 -- 172.21.15.92:0/1524549368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ec99f37eb0 msgr2=0x55ec99fb40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:10.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.132+0000 7f768e59c700 1 --2- 172.21.15.92:0/1524549368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ec99f37eb0 0x55ec99fb40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.132+0000 7f768e59c700 1 -- 172.21.15.92:0/1524549368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ec99f58c10 msgr2=0x55ec99f3e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:10.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.132+0000 7f768e59c700 1 --2- 172.21.15.92:0/1524549368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ec99f58c10 0x55ec99f3e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.132+0000 7f768e59c700 1 -- 172.21.15.92:0/1524549368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec99cf92c0 con 0x55ec99f0e9a0 2023-10-06T10:41:10.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.132+0000 7f768e59c700 1 --2- 172.21.15.92:0/1524549368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f0e9a0 0x55ec99f722c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7680007fd0 tx=0x7f7680024040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb260f5f32e16f54 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:10.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.132+0000 7f766cff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:10.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.133+0000 7f766cff9700 1 -- 172.21.15.92:0/1524549368 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7680027040 con 0x55ec99f0e9a0 2023-10-06T10:41:10.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.133+0000 7f766cff9700 1 -- 172.21.15.92:0/1524549368 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7680022a80 con 0x55ec99f0e9a0 2023-10-06T10:41:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.133+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1524549368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76780576d0 con 0x55ec99f0e9a0 2023-10-06T10:41:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.133+0000 7f766cff9700 1 -- 172.21.15.92:0/1524549368 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f768002a400 con 0x55ec99f0e9a0 2023-10-06T10:41:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.133+0000 7f769e0dfb80 10 client.6168 fetch_fsmap learned FSMap version 39 2023-10-06T10:41:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.133+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1524549368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55ec99e144c0 con 0x55ec99f0e9a0 2023-10-06T10:41:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.133+0000 7f766cff9700 1 -- 172.21.15.92:0/1524549368 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76780576d0 con 0x55ec99f0e9a0 2023-10-06T10:41:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.133+0000 7f766cff9700 1 -- 172.21.15.92:0/1524549368 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 39) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f768002a560 con 0x55ec99f0e9a0 2023-10-06T10:41:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.133+0000 7f769e0dfb80 10 client.6168 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T10:41:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.133+0000 7f769e0dfb80 10 client.6168 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2023-10-06T10:41:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.133+0000 7f769e0dfb80 20 client.6168 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.133+0000 7f769e0dfb80 1 --2- 172.21.15.92:0/1524549368 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x55ec99f4cf50 0x55ec99f4b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:10.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.133+0000 7f769e0dfb80 4 client.6168 mds_command: new command op to 5733 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:41:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.133+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1524549368 --> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ec99e01e50 con 0x55ec99f4cf50 2023-10-06T10:41:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.133+0000 7f7685d9b700 1 --2- 172.21.15.92:0/1524549368 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x55ec99f4cf50 0x55ec99f4b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.133+0000 7f766cff9700 10 client.6168 ms_handle_connect on v2:172.21.15.92:6834/831923836 2023-10-06T10:41:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.133+0000 7f7685d9b700 1 --2- 172.21.15.92:0/1524549368 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x55ec99f4cf50 0x55ec99f4b470 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=59600ab3fb3e964e server_cookie=6a62e0e159aa1b0e in_seq=0 out_seq=0 2023-10-06T10:41:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.133+0000 7f766cff9700 1 -- 172.21.15.92:0/1524549368 <== mds.1 v2:172.21.15.92:6834/831923836 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55ec99e01e50 con 0x55ec99f4cf50 2023-10-06T10:41:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f766cff9700 10 client.6168 handle_command_reply: tid=0 2023-10-06T10:41:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f769e0dfb80 1 client.6168 shutdown 2023-10-06T10:41:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1524549368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f0e9a0 msgr2=0x55ec99f722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f769e0dfb80 1 --2- 172.21.15.92:0/1524549368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f0e9a0 0x55ec99f722c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7680007fd0 tx=0x7f7680024040 comp rx=0 tx=0).stop 2023-10-06T10:41:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1524549368 shutdown_connections 2023-10-06T10:41:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f769e0dfb80 1 --2- 172.21.15.92:0/1524549368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ec99f58c10 0x55ec99f3e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f769e0dfb80 1 --2- 172.21.15.92:0/1524549368 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x55ec99f4cf50 0x55ec99f4b470 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f769e0dfb80 1 --2- 172.21.15.92:0/1524549368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99f0e9a0 0x55ec99f722c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f766cff9700 0 client.6168 ms_handle_reset on v2:172.21.15.92:6834/831923836 2023-10-06T10:41:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f769e0dfb80 1 --2- 172.21.15.92:0/1524549368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ec99f37eb0 0x55ec99fb40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1524549368 >> 172.21.15.92:0/1524549368 conn(0x55ec99f102c0 msgr2=0x55ec99f0d680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1524549368 shutdown_connections 2023-10-06T10:41:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f769e0dfb80 1 -- 172.21.15.92:0/1524549368 wait complete. 2023-10-06T10:41:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f769e0dfb80 20 client.6168 trim_cache size 0 max 16384 2023-10-06T10:41:10.144 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:41:10.144 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T10:41:10.144 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:41:10.144 INFO:teuthology.orchestra.run.smithi092.stdout: "9514c1ce-0eea-44fe-937b-b21b60ec3c5b": { 2023-10-06T10:41:10.144 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:41:10.144 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "9514c1ce-0eea-44fe-937b-b21b60ec3c5b", 2023-10-06T10:41:10.144 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:41:10.144 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:10.144 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:10.145 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:41:10.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f768f59e700 1 -- 172.21.15.92:0/2554002069 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7670066f30 msgr2=0x7f76700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:10.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f768f59e700 1 --2- 172.21.15.92:0/2554002069 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7670066f30 0x7f76700693e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7690158d40 tx=0x7f7680009040 comp rx=0 tx=0).stop 2023-10-06T10:41:10.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f768f59e700 1 -- 172.21.15.92:0/2554002069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76900b29f0 msgr2=0x7f7690159e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:10.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f768f59e700 1 --2- 172.21.15.92:0/2554002069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76900b29f0 0x7f7690159e80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7678013000 tx=0x7f7678017000 comp rx=0 tx=0).stop 2023-10-06T10:41:10.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f768f59e700 1 -- 172.21.15.92:0/2554002069 shutdown_connections 2023-10-06T10:41:10.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f768f59e700 1 --2- 172.21.15.92:0/2554002069 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7670066f30 0x7f76700693e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f768f59e700 1 --2- 172.21.15.92:0/2554002069 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76900cd3c0 0x7f76900c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f768f59e700 1 --2- 172.21.15.92:0/2554002069 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76900b2620 0x7f769014fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f768f59e700 1 --2- 172.21.15.92:0/2554002069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76900b29f0 0x7f7690159e80 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:10.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f768f59e700 1 -- 172.21.15.92:0/2554002069 >> 172.21.15.92:0/2554002069 conn(0x7f7690099520 msgr2=0x7f7690150da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:10.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f768f59e700 1 -- 172.21.15.92:0/2554002069 shutdown_connections 2023-10-06T10:41:10.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:10.134+0000 7f768f59e700 1 -- 172.21.15.92:0/2554002069 wait complete. 2023-10-06T10:41:15.148 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:1 scrub status 2023-10-06T10:41:15.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.272+0000 7f629381e700 1 Processor -- start 2023-10-06T10:41:15.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.273+0000 7f629381e700 1 -- start start 2023-10-06T10:41:15.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.273+0000 7f629381e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628c0689f0 0x7f628c1646f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:15.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.273+0000 7f629381e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f628c0cdc60 con 0x7f628c1597f0 2023-10-06T10:41:15.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.273+0000 7f629381e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f628c0cd050 con 0x7f628c159420 2023-10-06T10:41:15.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.273+0000 7f629381e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0cd330 con 0x7f628c0689f0 2023-10-06T10:41:15.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.273+0000 7f6290db9700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f628c1597f0 0x7f628c0682e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39370/0 (socket says 172.21.15.92:39370) 2023-10-06T10:41:15.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.273+0000 7f6290db9700 1 -- 172.21.15.92:0/3188513629 learned_addr learned my addr 172.21.15.92:0/3188513629 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:15.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.274+0000 7f6291dbb700 1 --2- 172.21.15.92:0/3188513629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628c0689f0 0x7f628c1646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:15.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.274+0000 7f6283fff700 1 -- 172.21.15.92:0/3188513629 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2121159645 0 0) 0x7f628c0cdc60 con 0x7f628c1597f0 2023-10-06T10:41:15.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.274+0000 7f6283fff700 1 -- 172.21.15.92:0/3188513629 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6278003570 con 0x7f628c1597f0 2023-10-06T10:41:15.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.274+0000 7f6283fff700 1 -- 172.21.15.92:0/3188513629 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2470843617 0 0) 0x7f6278003570 con 0x7f628c1597f0 2023-10-06T10:41:15.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.274+0000 7f6283fff700 1 -- 172.21.15.92:0/3188513629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628c0689f0 msgr2=0x7f628c1646f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:15.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.274+0000 7f6283fff700 1 --2- 172.21.15.92:0/3188513629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628c0689f0 0x7f628c1646f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.274+0000 7f6283fff700 1 -- 172.21.15.92:0/3188513629 >> v1:172.21.15.124:6789/0 conn(0x7f628c159420 legacy=0x7f628c0cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:15.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.274+0000 7f6283fff700 1 -- 172.21.15.92:0/3188513629 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c0cd610 con 0x7f628c1597f0 2023-10-06T10:41:15.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.274+0000 7f6283fff700 1 -- 172.21.15.92:0/3188513629 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f627c001470 con 0x7f628c1597f0 2023-10-06T10:41:15.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.274+0000 7f6283fff700 1 -- 172.21.15.92:0/3188513629 >> v1:172.21.15.92:6789/0 conn(0x7f628c1597f0 legacy=0x7f628c0682e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:15.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.274+0000 7f6283fff700 1 --2- 172.21.15.92:0/3188513629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6278003a90 0x7f6278003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:15.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.274+0000 7f6283fff700 1 --2- 172.21.15.92:0/3188513629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6278004810 0x7f6278004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:15.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.274+0000 7f6283fff700 1 --2- 172.21.15.92:0/3188513629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628c0689f0 0x7f62780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:15.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.274+0000 7f6283fff700 1 -- 172.21.15.92:0/3188513629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0cd330 con 0x7f6278004810 2023-10-06T10:41:15.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.274+0000 7f6283fff700 1 -- 172.21.15.92:0/3188513629 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627800bb10 con 0x7f6278003a90 2023-10-06T10:41:15.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.274+0000 7f6283fff700 1 -- 172.21.15.92:0/3188513629 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627800bcd0 con 0x7f628c0689f0 2023-10-06T10:41:15.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.274+0000 7f6290db9700 1 --2- 172.21.15.92:0/3188513629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6278004810 0x7f6278004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:15.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.274+0000 7f62915ba700 1 --2- 172.21.15.92:0/3188513629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6278003a90 0x7f6278003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:15.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.274+0000 7f6291dbb700 1 --2- 172.21.15.92:0/3188513629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628c0689f0 0x7f62780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:15.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.275+0000 7f6290db9700 1 -- 172.21.15.92:0/3188513629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628c0689f0 msgr2=0x7f62780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:15.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.275+0000 7f6290db9700 1 --2- 172.21.15.92:0/3188513629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628c0689f0 0x7f62780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.275+0000 7f6290db9700 1 -- 172.21.15.92:0/3188513629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6278003a90 msgr2=0x7f6278003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:15.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.275+0000 7f6290db9700 1 --2- 172.21.15.92:0/3188513629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6278003a90 0x7f6278003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.275+0000 7f6290db9700 1 -- 172.21.15.92:0/3188513629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6278003570 con 0x7f6278004810 2023-10-06T10:41:15.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.275+0000 7f6290db9700 1 --2- 172.21.15.92:0/3188513629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6278004810 0x7f6278004c80 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f627c00c2d0 tx=0x7f627c001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99573c3e770dce66 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:15.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.275+0000 7f6283fff700 1 -- 172.21.15.92:0/3188513629 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c00cbc0 con 0x7f6278004810 2023-10-06T10:41:15.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.275+0000 7f6283fff700 1 -- 172.21.15.92:0/3188513629 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627c00cd20 con 0x7f6278004810 2023-10-06T10:41:15.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.275+0000 7f6283fff700 1 -- 172.21.15.92:0/3188513629 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c005b70 con 0x7f6278004810 2023-10-06T10:41:15.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.275+0000 7f629381e700 1 -- 172.21.15.92:0/3188513629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6278004810 msgr2=0x7f6278004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:15.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.275+0000 7f629381e700 1 --2- 172.21.15.92:0/3188513629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6278004810 0x7f6278004c80 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f627c00c2d0 tx=0x7f627c001df0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.275+0000 7f6291dbb700 1 -- 172.21.15.92:0/3188513629 reap_dead start 2023-10-06T10:41:15.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.275+0000 7f629381e700 1 -- 172.21.15.92:0/3188513629 shutdown_connections 2023-10-06T10:41:15.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.275+0000 7f629381e700 1 -- 172.21.15.92:0/3188513629 >> 172.21.15.92:0/3188513629 conn(0x7f628c06ece0 msgr2=0x7f628c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:15.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.275+0000 7f629381e700 1 -- 172.21.15.92:0/3188513629 shutdown_connections 2023-10-06T10:41:15.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.276+0000 7f629381e700 1 -- 172.21.15.92:0/3188513629 wait complete. 2023-10-06T10:41:15.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.277+0000 7f629381e700 1 Processor -- start 2023-10-06T10:41:15.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.277+0000 7f629381e700 1 -- start start 2023-10-06T10:41:15.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.277+0000 7f629381e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6278004810 0x7f628c14be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:15.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.277+0000 7f629381e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628c15e7f0 0x7f628c15ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:15.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.277+0000 7f629381e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628c15af20 0x7f628c15b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:15.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.277+0000 7f629381e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c15b900 con 0x7f628c15af20 2023-10-06T10:41:15.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.278+0000 7f629381e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c15f1a0 con 0x7f6278004810 2023-10-06T10:41:15.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.278+0000 7f629381e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0b37e0 con 0x7f628c15e7f0 2023-10-06T10:41:15.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.278+0000 7f6291dbb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628c15af20 0x7f628c15b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:15.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.278+0000 7f6291dbb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628c15af20 0x7f628c15b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40686/0 (socket says 172.21.15.92:40686) 2023-10-06T10:41:15.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.278+0000 7f6291dbb700 1 -- 172.21.15.92:0/3596095342 learned_addr learned my addr 172.21.15.92:0/3596095342 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:15.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.278+0000 7f62915ba700 1 --2- 172.21.15.92:0/3596095342 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6278004810 0x7f628c14be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:15.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.278+0000 7f6290db9700 1 --2- 172.21.15.92:0/3596095342 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628c15e7f0 0x7f628c15ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:15.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.278+0000 7f6291dbb700 1 -- 172.21.15.92:0/3596095342 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628c15e7f0 msgr2=0x7f628c15ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:15.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.278+0000 7f6291dbb700 1 --2- 172.21.15.92:0/3596095342 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628c15e7f0 0x7f628c15ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.278+0000 7f6291dbb700 1 -- 172.21.15.92:0/3596095342 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6278004810 msgr2=0x7f628c14be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:15.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.278+0000 7f6291dbb700 1 --2- 172.21.15.92:0/3596095342 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6278004810 0x7f628c14be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.278+0000 7f6291dbb700 1 -- 172.21.15.92:0/3596095342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c15c240 con 0x7f628c15af20 2023-10-06T10:41:15.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.278+0000 7f6291dbb700 1 --2- 172.21.15.92:0/3596095342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628c15af20 0x7f628c15b3c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f628c068df0 tx=0x7f628400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:15.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.278+0000 7f62817fa700 1 -- 172.21.15.92:0/3596095342 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628400e040 con 0x7f628c15af20 2023-10-06T10:41:15.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.278+0000 7f62817fa700 1 -- 172.21.15.92:0/3596095342 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628400f040 con 0x7f628c15af20 2023-10-06T10:41:15.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.278+0000 7f629381e700 1 -- 172.21.15.92:0/3596095342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628c0b3a30 con 0x7f628c15af20 2023-10-06T10:41:15.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.278+0000 7f62817fa700 1 -- 172.21.15.92:0/3596095342 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628400fd90 con 0x7f628c15af20 2023-10-06T10:41:15.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.278+0000 7f629381e700 1 -- 172.21.15.92:0/3596095342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628c0b3ff0 con 0x7f628c15af20 2023-10-06T10:41:15.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.279+0000 7f62817fa700 1 -- 172.21.15.92:0/3596095342 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6284015640 con 0x7f628c15af20 2023-10-06T10:41:15.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.280+0000 7f62817fa700 1 --2- 172.21.15.92:0/3596095342 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6270066f30 0x7f62700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:15.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.280+0000 7f62817fa700 1 -- 172.21.15.92:0/3596095342 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6284085080 con 0x7f628c15af20 2023-10-06T10:41:15.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.280+0000 7f62915ba700 1 --2- 172.21.15.92:0/3596095342 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6270066f30 0x7f62700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:15.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.280+0000 7f62915ba700 1 --2- 172.21.15.92:0/3596095342 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6270066f30 0x7f62700693e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6288003d10 tx=0x7f6288007d10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:15.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.282+0000 7f629cb16b80 1 Processor -- start 2023-10-06T10:41:15.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.282+0000 7f629cb16b80 1 -- start start 2023-10-06T10:41:15.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.283+0000 7f629cb16b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a2eb09ae0 0x556a2eb09eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:15.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.283+0000 7f629cb16b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a2eac59f0 0x556a2eac25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:15.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.283+0000 7f629cb16b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac3c20 0x556a2eac2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:15.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.283+0000 7f629cb16b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a2ea42cd0 con 0x556a2eac3c20 2023-10-06T10:41:15.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.283+0000 7f629cb16b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a2ea19ac0 con 0x556a2eb09ae0 2023-10-06T10:41:15.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.283+0000 7f629cb16b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a2ea172c0 con 0x556a2eac59f0 2023-10-06T10:41:15.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.283+0000 7f6291dbb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac3c20 0x556a2eac2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:15.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.283+0000 7f6291dbb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac3c20 0x556a2eac2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40692/0 (socket says 172.21.15.92:40692) 2023-10-06T10:41:15.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.283+0000 7f6291dbb700 1 -- 172.21.15.92:0/3989378011 learned_addr learned my addr 172.21.15.92:0/3989378011 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:15.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.283+0000 7f6290db9700 1 --2- 172.21.15.92:0/3989378011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a2eac59f0 0x556a2eac25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:15.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.283+0000 7f62915ba700 1 --2- 172.21.15.92:0/3989378011 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a2eb09ae0 0x556a2eb09eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:15.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.283+0000 7f6291dbb700 1 -- 172.21.15.92:0/3989378011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a2eac59f0 msgr2=0x556a2eac25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:15.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.283+0000 7f6291dbb700 1 --2- 172.21.15.92:0/3989378011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a2eac59f0 0x556a2eac25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.283+0000 7f6291dbb700 1 -- 172.21.15.92:0/3989378011 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a2eb09ae0 msgr2=0x556a2eb09eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:15.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.283+0000 7f6291dbb700 1 --2- 172.21.15.92:0/3989378011 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a2eb09ae0 0x556a2eb09eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.283+0000 7f6291dbb700 1 -- 172.21.15.92:0/3989378011 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a2e8ad2c0 con 0x556a2eac3c20 2023-10-06T10:41:15.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.283+0000 7f62915ba700 1 --2- 172.21.15.92:0/3989378011 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a2eb09ae0 0x556a2eb09eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:15.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.283+0000 7f6291dbb700 1 --2- 172.21.15.92:0/3989378011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac3c20 0x556a2eac2b10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6284002230 tx=0x7f6284083260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced692d6972d9a28 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:15.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.283+0000 7f62767fc700 1 -- 172.21.15.92:0/3989378011 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6284056410 con 0x556a2eac3c20 2023-10-06T10:41:15.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.283+0000 7f62767fc700 1 -- 172.21.15.92:0/3989378011 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6284055d10 con 0x556a2eac3c20 2023-10-06T10:41:15.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.283+0000 7f62767fc700 1 -- 172.21.15.92:0/3989378011 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6284058f60 con 0x556a2eac3c20 2023-10-06T10:41:15.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.284+0000 7f629cb16b80 1 -- 172.21.15.92:0/3989378011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac3c20 msgr2=0x556a2eac2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:15.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.284+0000 7f629cb16b80 1 --2- 172.21.15.92:0/3989378011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac3c20 0x556a2eac2b10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6284002230 tx=0x7f6284083260 comp rx=0 tx=0).stop 2023-10-06T10:41:15.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.284+0000 7f629cb16b80 1 -- 172.21.15.92:0/3989378011 shutdown_connections 2023-10-06T10:41:15.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.284+0000 7f629cb16b80 1 --2- 172.21.15.92:0/3989378011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac3c20 0x556a2eac2b10 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.284+0000 7f629cb16b80 1 --2- 172.21.15.92:0/3989378011 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a2eb09ae0 0x556a2eb09eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.284+0000 7f629cb16b80 1 --2- 172.21.15.92:0/3989378011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a2eac59f0 0x556a2eac25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.284+0000 7f629cb16b80 1 -- 172.21.15.92:0/3989378011 >> 172.21.15.92:0/3989378011 conn(0x556a2eac40c0 msgr2=0x556a2eac4490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:15.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.284+0000 7f629cb16b80 1 -- 172.21.15.92:0/3989378011 shutdown_connections 2023-10-06T10:41:15.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.284+0000 7f629cb16b80 1 -- 172.21.15.92:0/3989378011 wait complete. 2023-10-06T10:41:15.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.284+0000 7f629cb16b80 1 -- start start 2023-10-06T10:41:15.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.285+0000 7f629cb16b80 1 Processor -- start 2023-10-06T10:41:15.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.285+0000 7f629cb16b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac3c20 0x556a2eaf25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:15.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.285+0000 7f629cb16b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a2eac59f0 0x556a2eaeab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:15.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.285+0000 7f629cb16b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a2eb09ae0 0x556a2eaeb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:15.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.285+0000 7f629cb16b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a2ea130b0 con 0x556a2eac3c20 2023-10-06T10:41:15.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.285+0000 7f629cb16b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a2e9be8e0 con 0x556a2eac59f0 2023-10-06T10:41:15.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.285+0000 7f629cb16b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a2e9d9ef0 con 0x556a2eb09ae0 2023-10-06T10:41:15.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.286+0000 7f629cb16b80 10 client.0 _ll_register_callbacks cb 0x556a2e8aff80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:15.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.286+0000 7f62915ba700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac3c20 0x556a2eaf25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:15.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.286+0000 7f62915ba700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac3c20 0x556a2eaf25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40704/0 (socket says 172.21.15.92:40704) 2023-10-06T10:41:15.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.286+0000 7f62915ba700 1 -- 172.21.15.92:0/54451560 learned_addr learned my addr 172.21.15.92:0/54451560 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:15.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.286+0000 7f6290db9700 1 --2- 172.21.15.92:0/54451560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a2eac59f0 0x556a2eaeab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:15.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.286+0000 7f6291dbb700 1 --2- 172.21.15.92:0/54451560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a2eb09ae0 0x556a2eaeb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:15.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.286+0000 7f62915ba700 1 -- 172.21.15.92:0/54451560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a2eb09ae0 msgr2=0x556a2eaeb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:15.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.286+0000 7f62915ba700 1 --2- 172.21.15.92:0/54451560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a2eb09ae0 0x556a2eaeb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.286+0000 7f62915ba700 1 -- 172.21.15.92:0/54451560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a2eac59f0 msgr2=0x556a2eaeab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:15.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.286+0000 7f62915ba700 1 --2- 172.21.15.92:0/54451560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a2eac59f0 0x556a2eaeab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.286+0000 7f62915ba700 1 -- 172.21.15.92:0/54451560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628c0b3ff0 con 0x556a2eac3c20 2023-10-06T10:41:15.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.286+0000 7f62915ba700 1 --2- 172.21.15.92:0/54451560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac3c20 0x556a2eaf25e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6288015b50 tx=0x7f6288018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704dc84543f93008 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:15.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.286+0000 7f626b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:15.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.286+0000 7f626b7fe700 1 -- 172.21.15.92:0/54451560 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62880149c0 con 0x556a2eac3c20 2023-10-06T10:41:15.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.286+0000 7f626b7fe700 1 -- 172.21.15.92:0/54451560 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6288014b20 con 0x556a2eac3c20 2023-10-06T10:41:15.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.286+0000 7f629cb16b80 1 -- 172.21.15.92:0/54451560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a2eab76e0 con 0x556a2eac3c20 2023-10-06T10:41:15.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.286+0000 7f626b7fe700 1 -- 172.21.15.92:0/54451560 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62880223f0 con 0x556a2eac3c20 2023-10-06T10:41:15.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.286+0000 7f629cb16b80 10 client.6180 fetch_fsmap learned FSMap version 40 2023-10-06T10:41:15.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.286+0000 7f629cb16b80 1 -- 172.21.15.92:0/54451560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x556a2e850030 con 0x556a2eac3c20 2023-10-06T10:41:15.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.287+0000 7f626b7fe700 1 -- 172.21.15.92:0/54451560 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556a2eab76e0 con 0x556a2eac3c20 2023-10-06T10:41:15.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.287+0000 7f626b7fe700 1 -- 172.21.15.92:0/54451560 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f628802b430 con 0x556a2eac3c20 2023-10-06T10:41:15.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.287+0000 7f629cb16b80 10 client.6180 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T10:41:15.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.287+0000 7f629cb16b80 10 client.6180 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2023-10-06T10:41:15.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.287+0000 7f629cb16b80 20 client.6180 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:15.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.287+0000 7f629cb16b80 1 --2- 172.21.15.92:0/54451560 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x556a2eb065d0 0x556a2eaed600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:15.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.287+0000 7f629cb16b80 4 client.6180 mds_command: new command op to 5733 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:41:15.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.287+0000 7f629cb16b80 1 -- 172.21.15.92:0/54451560 --> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a2ea94610 con 0x556a2eb065d0 2023-10-06T10:41:15.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.287+0000 7f6290db9700 1 --2- 172.21.15.92:0/54451560 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x556a2eb065d0 0x556a2eaed600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.287+0000 7f626b7fe700 10 client.6180 ms_handle_connect on v2:172.21.15.92:6834/831923836 2023-10-06T10:41:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.287+0000 7f6290db9700 1 --2- 172.21.15.92:0/54451560 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x556a2eb065d0 0x556a2eaed600 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9fb449c0ebdd8426 server_cookie=b048a2f120632ead in_seq=0 out_seq=0 2023-10-06T10:41:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.287+0000 7f626b7fe700 1 -- 172.21.15.92:0/54451560 <== mds.1 v2:172.21.15.92:6834/831923836 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x556a2ea94610 con 0x556a2eb065d0 2023-10-06T10:41:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.288+0000 7f626b7fe700 10 client.6180 handle_command_reply: tid=0 2023-10-06T10:41:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.288+0000 7f629cb16b80 1 client.6180 shutdown 2023-10-06T10:41:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.288+0000 7f629cb16b80 1 -- 172.21.15.92:0/54451560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac3c20 msgr2=0x556a2eaf25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.288+0000 7f629cb16b80 1 --2- 172.21.15.92:0/54451560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac3c20 0x556a2eaf25e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6288015b50 tx=0x7f6288018000 comp rx=0 tx=0).stop 2023-10-06T10:41:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.288+0000 7f629cb16b80 1 -- 172.21.15.92:0/54451560 shutdown_connections 2023-10-06T10:41:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.288+0000 7f629cb16b80 1 --2- 172.21.15.92:0/54451560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a2eb09ae0 0x556a2eaeb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.288+0000 7f629cb16b80 1 --2- 172.21.15.92:0/54451560 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x556a2eb065d0 0x556a2eaed600 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.288+0000 7f629cb16b80 1 --2- 172.21.15.92:0/54451560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac3c20 0x556a2eaf25e0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.288+0000 7f629cb16b80 1 --2- 172.21.15.92:0/54451560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a2eac59f0 0x556a2eaeab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.288+0000 7f626b7fe700 0 client.6180 ms_handle_reset on v2:172.21.15.92:6834/831923836 2023-10-06T10:41:15.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.288+0000 7f629cb16b80 1 -- 172.21.15.92:0/54451560 >> 172.21.15.92:0/54451560 conn(0x556a2eac40c0 msgr2=0x556a2eabf840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:15.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.288+0000 7f629cb16b80 1 -- 172.21.15.92:0/54451560 shutdown_connections 2023-10-06T10:41:15.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.288+0000 7f629cb16b80 1 -- 172.21.15.92:0/54451560 wait complete. 2023-10-06T10:41:15.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.288+0000 7f629cb16b80 20 client.6180 trim_cache size 0 max 16384 2023-10-06T10:41:15.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.304+0000 7f629cb16b80 1 Processor -- start 2023-10-06T10:41:15.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.304+0000 7f629cb16b80 1 -- start start 2023-10-06T10:41:15.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.304+0000 7f629cb16b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac3c20 0x556a2eaef920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:15.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.304+0000 7f629cb16b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a2eac59f0 0x556a2eaefe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:15.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.304+0000 7f629cb16b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a2eb09ae0 0x556a2eaff2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:15.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.304+0000 7f629cb16b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a2e895e80 con 0x556a2eac3c20 2023-10-06T10:41:15.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.304+0000 7f629cb16b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a2eabb310 con 0x556a2eac59f0 2023-10-06T10:41:15.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.304+0000 7f629cb16b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a2eaf25c0 con 0x556a2eb09ae0 2023-10-06T10:41:15.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.304+0000 7f62915ba700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac3c20 0x556a2eaef920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:15.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.304+0000 7f62915ba700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac3c20 0x556a2eaef920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40710/0 (socket says 172.21.15.92:40710) 2023-10-06T10:41:15.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.304+0000 7f62915ba700 1 -- 172.21.15.92:0/2060183441 learned_addr learned my addr 172.21.15.92:0/2060183441 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:15.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.305+0000 7f62915ba700 1 -- 172.21.15.92:0/2060183441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a2eb09ae0 msgr2=0x556a2eaff2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:15.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.305+0000 7f62915ba700 1 --2- 172.21.15.92:0/2060183441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a2eb09ae0 0x556a2eaff2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.305+0000 7f62915ba700 1 -- 172.21.15.92:0/2060183441 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a2eac59f0 msgr2=0x556a2eaefe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:15.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.305+0000 7f62915ba700 1 --2- 172.21.15.92:0/2060183441 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a2eac59f0 0x556a2eaefe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.305+0000 7f62915ba700 1 -- 172.21.15.92:0/2060183441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a2e9e17d0 con 0x556a2eac3c20 2023-10-06T10:41:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.305+0000 7f62915ba700 1 --2- 172.21.15.92:0/2060183441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac3c20 0x556a2eaef920 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6288003560 tx=0x7f6288024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6398cfa5fc6fa664 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.305+0000 7f6276ffd700 1 -- 172.21.15.92:0/2060183441 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288039070 con 0x556a2eac3c20 2023-10-06T10:41:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.305+0000 7f6276ffd700 1 -- 172.21.15.92:0/2060183441 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6288015e60 con 0x556a2eac3c20 2023-10-06T10:41:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.305+0000 7f6276ffd700 1 -- 172.21.15.92:0/2060183441 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62880079d0 con 0x556a2eac3c20 2023-10-06T10:41:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.305+0000 7f629cb16b80 1 -- 172.21.15.92:0/2060183441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac3c20 msgr2=0x556a2eaef920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.305+0000 7f629cb16b80 1 --2- 172.21.15.92:0/2060183441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac3c20 0x556a2eaef920 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6288003560 tx=0x7f6288024000 comp rx=0 tx=0).stop 2023-10-06T10:41:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.306+0000 7f629cb16b80 1 -- 172.21.15.92:0/2060183441 shutdown_connections 2023-10-06T10:41:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.306+0000 7f629cb16b80 1 --2- 172.21.15.92:0/2060183441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a2eb09ae0 0x556a2eaff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.306+0000 7f629cb16b80 1 --2- 172.21.15.92:0/2060183441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac3c20 0x556a2eaef920 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.306+0000 7f629cb16b80 1 --2- 172.21.15.92:0/2060183441 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a2eac59f0 0x556a2eaefe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.306+0000 7f629cb16b80 1 -- 172.21.15.92:0/2060183441 >> 172.21.15.92:0/2060183441 conn(0x556a2eac40c0 msgr2=0x556a2eaeab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:15.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.306+0000 7f629cb16b80 1 -- 172.21.15.92:0/2060183441 shutdown_connections 2023-10-06T10:41:15.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.306+0000 7f629cb16b80 1 -- 172.21.15.92:0/2060183441 wait complete. 2023-10-06T10:41:15.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.306+0000 7f629cb16b80 1 -- start start 2023-10-06T10:41:15.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.306+0000 7f629cb16b80 1 Processor -- start 2023-10-06T10:41:15.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.307+0000 7f629cb16b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a2eac3c20 0x556a2eb6ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:15.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.307+0000 7f629cb16b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac59f0 0x556a2eaff7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:15.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.307+0000 7f629cb16b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a2eb09ae0 0x556a2eaffd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:15.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.307+0000 7f629cb16b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a2eac3520 con 0x556a2eac59f0 2023-10-06T10:41:15.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.307+0000 7f629cb16b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a2eb6f830 con 0x556a2eac3c20 2023-10-06T10:41:15.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.307+0000 7f629cb16b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a2eb6ef60 con 0x556a2eb09ae0 2023-10-06T10:41:15.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.307+0000 7f629cb16b80 10 client.0 _ll_register_callbacks cb 0x556a2e8aff80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:15.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.307+0000 7f6290db9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac59f0 0x556a2eaff7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:15.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.307+0000 7f6290db9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac59f0 0x556a2eaff7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40724/0 (socket says 172.21.15.92:40724) 2023-10-06T10:41:15.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.307+0000 7f6290db9700 1 -- 172.21.15.92:0/443383620 learned_addr learned my addr 172.21.15.92:0/443383620 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:15.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.307+0000 7f62915ba700 1 --2- 172.21.15.92:0/443383620 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a2eac3c20 0x556a2eb6ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:15.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.307+0000 7f6291dbb700 1 --2- 172.21.15.92:0/443383620 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a2eb09ae0 0x556a2eaffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:15.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.307+0000 7f6290db9700 1 -- 172.21.15.92:0/443383620 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a2eb09ae0 msgr2=0x556a2eaffd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:15.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.307+0000 7f6290db9700 1 --2- 172.21.15.92:0/443383620 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a2eb09ae0 0x556a2eaffd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.307+0000 7f6290db9700 1 -- 172.21.15.92:0/443383620 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a2eac3c20 msgr2=0x556a2eb6ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:15.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.307+0000 7f6290db9700 1 --2- 172.21.15.92:0/443383620 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a2eac3c20 0x556a2eb6ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.307+0000 7f6290db9700 1 -- 172.21.15.92:0/443383620 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a2e850030 con 0x556a2eac59f0 2023-10-06T10:41:15.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.308+0000 7f6290db9700 1 --2- 172.21.15.92:0/443383620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac59f0 0x556a2eaff7f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6278004770 tx=0x7f627c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77aa8007d6cedd4e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:15.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.308+0000 7f626b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:15.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.308+0000 7f626b7fe700 1 -- 172.21.15.92:0/443383620 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c023040 con 0x556a2eac59f0 2023-10-06T10:41:15.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.308+0000 7f626b7fe700 1 -- 172.21.15.92:0/443383620 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f627c001580 con 0x556a2eac59f0 2023-10-06T10:41:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.308+0000 7f629cb16b80 1 -- 172.21.15.92:0/443383620 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6284055d10 con 0x556a2eac59f0 2023-10-06T10:41:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.308+0000 7f626b7fe700 1 -- 172.21.15.92:0/443383620 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f627c001740 con 0x556a2eac59f0 2023-10-06T10:41:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.308+0000 7f629cb16b80 10 client.6186 fetch_fsmap learned FSMap version 40 2023-10-06T10:41:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.308+0000 7f629cb16b80 1 -- 172.21.15.92:0/443383620 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x556a2e9c84c0 con 0x556a2eac59f0 2023-10-06T10:41:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.308+0000 7f626b7fe700 1 -- 172.21.15.92:0/443383620 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6284055d10 con 0x556a2eac59f0 2023-10-06T10:41:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.308+0000 7f626b7fe700 1 -- 172.21.15.92:0/443383620 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f627c004a40 con 0x556a2eac59f0 2023-10-06T10:41:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.308+0000 7f629cb16b80 10 client.6186 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T10:41:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.308+0000 7f629cb16b80 10 client.6186 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2023-10-06T10:41:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.308+0000 7f629cb16b80 20 client.6186 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.308+0000 7f629cb16b80 1 --2- 172.21.15.92:0/443383620 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x556a2eb6fcc0 0x556a2eac25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.308+0000 7f629cb16b80 4 client.6186 mds_command: new command op to 5733 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:41:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.308+0000 7f629cb16b80 1 -- 172.21.15.92:0/443383620 --> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556a2e9b5e50 con 0x556a2eb6fcc0 2023-10-06T10:41:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.308+0000 7f62915ba700 1 --2- 172.21.15.92:0/443383620 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x556a2eb6fcc0 0x556a2eac25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.308+0000 7f626b7fe700 10 client.6186 ms_handle_connect on v2:172.21.15.92:6834/831923836 2023-10-06T10:41:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.308+0000 7f62915ba700 1 --2- 172.21.15.92:0/443383620 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x556a2eb6fcc0 0x556a2eac25d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11f97b75dc4b6633 server_cookie=1d652dca45ae3bd7 in_seq=0 out_seq=0 2023-10-06T10:41:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.309+0000 7f626b7fe700 1 -- 172.21.15.92:0/443383620 <== mds.1 v2:172.21.15.92:6834/831923836 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x556a2e9b5e50 con 0x556a2eb6fcc0 2023-10-06T10:41:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.309+0000 7f626b7fe700 10 client.6186 handle_command_reply: tid=0 2023-10-06T10:41:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.309+0000 7f629cb16b80 1 client.6186 shutdown 2023-10-06T10:41:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.309+0000 7f629cb16b80 1 -- 172.21.15.92:0/443383620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac59f0 msgr2=0x556a2eaff7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:15.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.309+0000 7f629cb16b80 1 --2- 172.21.15.92:0/443383620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac59f0 0x556a2eaff7f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6278004770 tx=0x7f627c012000 comp rx=0 tx=0).stop 2023-10-06T10:41:15.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.309+0000 7f629cb16b80 1 -- 172.21.15.92:0/443383620 shutdown_connections 2023-10-06T10:41:15.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.309+0000 7f629cb16b80 1 --2- 172.21.15.92:0/443383620 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a2eb09ae0 0x556a2eaffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.309+0000 7f629cb16b80 1 --2- 172.21.15.92:0/443383620 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a2eac3c20 0x556a2eb6ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.309+0000 7f629cb16b80 1 --2- 172.21.15.92:0/443383620 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x556a2eb6fcc0 0x556a2eac25d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.309+0000 7f629cb16b80 1 --2- 172.21.15.92:0/443383620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a2eac59f0 0x556a2eaff7f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.309+0000 7f626b7fe700 0 client.6186 ms_handle_reset on v2:172.21.15.92:6834/831923836 2023-10-06T10:41:15.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.309+0000 7f629cb16b80 1 -- 172.21.15.92:0/443383620 >> 172.21.15.92:0/443383620 conn(0x556a2eac40c0 msgr2=0x556a2eaef730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:15.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.309+0000 7f629cb16b80 1 -- 172.21.15.92:0/443383620 shutdown_connections 2023-10-06T10:41:15.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.309+0000 7f629cb16b80 1 -- 172.21.15.92:0/443383620 wait complete. 2023-10-06T10:41:15.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.309+0000 7f629cb16b80 20 client.6186 trim_cache size 0 max 16384 2023-10-06T10:41:15.321 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:41:15.321 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T10:41:15.321 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:41:15.321 INFO:teuthology.orchestra.run.smithi092.stdout: "9514c1ce-0eea-44fe-937b-b21b60ec3c5b": { 2023-10-06T10:41:15.321 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:41:15.322 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "9514c1ce-0eea-44fe-937b-b21b60ec3c5b", 2023-10-06T10:41:15.322 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:41:15.322 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:15.322 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:15.322 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:41:15.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.309+0000 7f629381e700 1 -- 172.21.15.92:0/3596095342 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6270066f30 msgr2=0x7f62700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:15.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.309+0000 7f629381e700 1 --2- 172.21.15.92:0/3596095342 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6270066f30 0x7f62700693e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6288003d10 tx=0x7f6288007d10 comp rx=0 tx=0).stop 2023-10-06T10:41:15.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.309+0000 7f629381e700 1 -- 172.21.15.92:0/3596095342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628c15af20 msgr2=0x7f628c15b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:15.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.309+0000 7f629381e700 1 --2- 172.21.15.92:0/3596095342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628c15af20 0x7f628c15b3c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f628c068df0 tx=0x7f628400b040 comp rx=0 tx=0).stop 2023-10-06T10:41:15.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.310+0000 7f629381e700 1 -- 172.21.15.92:0/3596095342 shutdown_connections 2023-10-06T10:41:15.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.310+0000 7f629381e700 1 --2- 172.21.15.92:0/3596095342 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6270066f30 0x7f62700693e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.310+0000 7f629381e700 1 --2- 172.21.15.92:0/3596095342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628c15af20 0x7f628c15b3c0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.310+0000 7f629381e700 1 --2- 172.21.15.92:0/3596095342 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6278004810 0x7f628c14be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.310+0000 7f629381e700 1 --2- 172.21.15.92:0/3596095342 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f628c15e7f0 0x7f628c15ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.310+0000 7f629381e700 1 -- 172.21.15.92:0/3596095342 >> 172.21.15.92:0/3596095342 conn(0x7f628c06ece0 msgr2=0x7f628c0b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.310+0000 7f629381e700 1 -- 172.21.15.92:0/3596095342 shutdown_connections 2023-10-06T10:41:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:15.310+0000 7f629381e700 1 -- 172.21.15.92:0/3596095342 wait complete. 2023-10-06T10:41:16.342 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T10:41:16.340+0000 7f9e81bcf700 -1 log_channel(cluster) log [ERR] : Health check failed: 2 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T10:41:20.326 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:1 scrub status 2023-10-06T10:41:20.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.458+0000 7f0d05057700 1 Processor -- start 2023-10-06T10:41:20.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.458+0000 7f0d05057700 1 -- start start 2023-10-06T10:41:20.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.459+0000 7f0d05057700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0d00159420 0x7f0d001597f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:20.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.459+0000 7f0d05057700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d000cdc60 con 0x7f0d000689f0 2023-10-06T10:41:20.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.459+0000 7f0d05057700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d000cd050 con 0x7f0d000cb6d0 2023-10-06T10:41:20.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.459+0000 7f0d05057700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d000cd330 con 0x7f0d00159420 2023-10-06T10:41:20.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.459+0000 7f0cfed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0d00159420 0x7f0d001597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:20.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.459+0000 7f0cff59e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0d000689f0 0x7f0d000ceb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39380/0 (socket says 172.21.15.92:39380) 2023-10-06T10:41:20.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.459+0000 7f0cfed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0d00159420 0x7f0d001597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40726/0 (socket says 172.21.15.92:40726) 2023-10-06T10:41:20.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.459+0000 7f0cff59e700 1 -- 172.21.15.92:0/4043856843 learned_addr learned my addr 172.21.15.92:0/4043856843 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:20.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.460+0000 7f0cfed9d700 1 -- 172.21.15.92:0/4043856843 >> v1:172.21.15.124:6789/0 conn(0x7f0d000cb6d0 legacy=0x7f0d000682e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:20.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.460+0000 7f0cfed9d700 1 -- 172.21.15.92:0/4043856843 >> v1:172.21.15.92:6789/0 conn(0x7f0d000689f0 legacy=0x7f0d000ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:20.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.460+0000 7f0cfed9d700 1 -- 172.21.15.92:0/4043856843 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d000cd610 con 0x7f0d00159420 2023-10-06T10:41:20.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.460+0000 7f0cfe59c700 1 -- 172.21.15.92:0/4043856843 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 831372276 0 0) 0x7f0d000cdc60 con 0x7f0d000689f0 2023-10-06T10:41:20.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.460+0000 7f0cfed9d700 1 --2- 172.21.15.92:0/4043856843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d00159420 0x7f0d001597f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00098e0 tx=0x7f0cf00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76cce9aab2cfce89 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:20.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.460+0000 7f0cfe59c700 1 -- 172.21.15.92:0/4043856843 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0009e80 con 0x7f0d00159420 2023-10-06T10:41:20.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.460+0000 7f0cfe59c700 1 -- 172.21.15.92:0/4043856843 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf000b820 con 0x7f0d00159420 2023-10-06T10:41:20.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.460+0000 7f0cfe59c700 1 -- 172.21.15.92:0/4043856843 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf000bba0 con 0x7f0d00159420 2023-10-06T10:41:20.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.460+0000 7f0d05057700 1 -- 172.21.15.92:0/4043856843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d00159420 msgr2=0x7f0d001597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:20.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.460+0000 7f0d05057700 1 --2- 172.21.15.92:0/4043856843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d00159420 0x7f0d001597f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00098e0 tx=0x7f0cf00091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.461+0000 7f0d05057700 1 -- 172.21.15.92:0/4043856843 shutdown_connections 2023-10-06T10:41:20.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.461+0000 7f0d05057700 1 --2- 172.21.15.92:0/4043856843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d00159420 0x7f0d001597f0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.461+0000 7f0d05057700 1 -- 172.21.15.92:0/4043856843 >> 172.21.15.92:0/4043856843 conn(0x7f0d0006ece0 msgr2=0x7f0d0006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:20.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.461+0000 7f0d05057700 1 -- 172.21.15.92:0/4043856843 shutdown_connections 2023-10-06T10:41:20.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.461+0000 7f0d05057700 1 -- 172.21.15.92:0/4043856843 wait complete. 2023-10-06T10:41:20.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.462+0000 7f0d05057700 1 Processor -- start 2023-10-06T10:41:20.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.462+0000 7f0d05057700 1 -- start start 2023-10-06T10:41:20.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0d05057700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d000689f0 0x7f0d000ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:20.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0d05057700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d000cb6d0 0x7f0d0016cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:20.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0d05057700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d00159420 0x7f0d0016d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:20.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0d05057700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0016de90 con 0x7f0d00159420 2023-10-06T10:41:20.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0d05057700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0016dbb0 con 0x7f0d000cb6d0 2023-10-06T10:41:20.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0d05057700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0016ac30 con 0x7f0d000689f0 2023-10-06T10:41:20.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0cff59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d00159420 0x7f0d0016d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:20.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0cff59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d00159420 0x7f0d0016d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40730/0 (socket says 172.21.15.92:40730) 2023-10-06T10:41:20.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0cff59e700 1 -- 172.21.15.92:0/1547825304 learned_addr learned my addr 172.21.15.92:0/1547825304 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:20.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0cfed9d700 1 --2- 172.21.15.92:0/1547825304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d000689f0 0x7f0d000ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:20.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0cf659c700 1 --2- 172.21.15.92:0/1547825304 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d000cb6d0 0x7f0d0016cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:20.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0cff59e700 1 -- 172.21.15.92:0/1547825304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d000689f0 msgr2=0x7f0d000ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:20.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0cff59e700 1 --2- 172.21.15.92:0/1547825304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d000689f0 0x7f0d000ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0cff59e700 1 -- 172.21.15.92:0/1547825304 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d000cb6d0 msgr2=0x7f0d0016cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:20.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0cff59e700 1 --2- 172.21.15.92:0/1547825304 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d000cb6d0 0x7f0d0016cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0cff59e700 1 -- 172.21.15.92:0/1547825304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf0009420 con 0x7f0d00159420 2023-10-06T10:41:20.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0cf659c700 1 --2- 172.21.15.92:0/1547825304 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d000cb6d0 0x7f0d0016cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:20.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0cff59e700 1 --2- 172.21.15.92:0/1547825304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d00159420 0x7f0d0016d510 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800ccc0 tx=0x7f0cf8001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:20.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0cf77fe700 1 -- 172.21.15.92:0/1547825304 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf800a930 con 0x7f0d00159420 2023-10-06T10:41:20.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0cf77fe700 1 -- 172.21.15.92:0/1547825304 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf800aa90 con 0x7f0d00159420 2023-10-06T10:41:20.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0d05057700 1 -- 172.21.15.92:0/1547825304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d0016af10 con 0x7f0d00159420 2023-10-06T10:41:20.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0cf77fe700 1 -- 172.21.15.92:0/1547825304 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf800ae30 con 0x7f0d00159420 2023-10-06T10:41:20.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.463+0000 7f0d05057700 1 -- 172.21.15.92:0/1547825304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d000aeba0 con 0x7f0d00159420 2023-10-06T10:41:20.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.465+0000 7f0cf77fe700 1 -- 172.21.15.92:0/1547825304 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0cf8007300 con 0x7f0d00159420 2023-10-06T10:41:20.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.465+0000 7f0cf77fe700 1 --2- 172.21.15.92:0/1547825304 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0ce0066f30 0x7f0ce00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:20.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.465+0000 7f0cf77fe700 1 -- 172.21.15.92:0/1547825304 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0cf8084c60 con 0x7f0d00159420 2023-10-06T10:41:20.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.465+0000 7f0cfed9d700 1 --2- 172.21.15.92:0/1547825304 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0ce0066f30 0x7f0ce00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:20.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.466+0000 7f0cfed9d700 1 --2- 172.21.15.92:0/1547825304 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0ce0066f30 0x7f0ce00693e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00030c0 tx=0x7f0cf0016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:20.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.467+0000 7f0d0e34fb80 1 Processor -- start 2023-10-06T10:41:20.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.467+0000 7f0d0e34fb80 1 -- start start 2023-10-06T10:41:20.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.468+0000 7f0d0e34fb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e0dd8baae0 0x55e0dd8baeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:20.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.468+0000 7f0d0e34fb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e0dd8769f0 0x55e0dd8735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:20.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.468+0000 7f0d0e34fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd874c20 0x55e0dd873b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:20.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.468+0000 7f0d0e34fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0dd7f3cd0 con 0x55e0dd874c20 2023-10-06T10:41:20.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.468+0000 7f0d0e34fb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0dd7caac0 con 0x55e0dd8baae0 2023-10-06T10:41:20.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.468+0000 7f0d0e34fb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0dd7c82c0 con 0x55e0dd8769f0 2023-10-06T10:41:20.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.468+0000 7f0cff59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd874c20 0x55e0dd873b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:20.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.468+0000 7f0cff59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd874c20 0x55e0dd873b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40742/0 (socket says 172.21.15.92:40742) 2023-10-06T10:41:20.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.468+0000 7f0cff59e700 1 -- 172.21.15.92:0/3499585313 learned_addr learned my addr 172.21.15.92:0/3499585313 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:20.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.468+0000 7f0cf659c700 1 --2- 172.21.15.92:0/3499585313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e0dd8769f0 0x55e0dd8735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:20.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.468+0000 7f0cfed9d700 1 --2- 172.21.15.92:0/3499585313 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e0dd8baae0 0x55e0dd8baeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:20.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.468+0000 7f0cff59e700 1 -- 172.21.15.92:0/3499585313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e0dd8769f0 msgr2=0x55e0dd8735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:20.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.468+0000 7f0cff59e700 1 --2- 172.21.15.92:0/3499585313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e0dd8769f0 0x55e0dd8735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.468+0000 7f0cff59e700 1 -- 172.21.15.92:0/3499585313 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e0dd8baae0 msgr2=0x55e0dd8baeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:20.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.468+0000 7f0cff59e700 1 --2- 172.21.15.92:0/3499585313 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e0dd8baae0 0x55e0dd8baeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.468+0000 7f0cff59e700 1 -- 172.21.15.92:0/3499585313 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0dd65e2c0 con 0x55e0dd874c20 2023-10-06T10:41:20.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.468+0000 7f0cfed9d700 1 --2- 172.21.15.92:0/3499585313 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e0dd8baae0 0x55e0dd8baeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:20.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.469+0000 7f0cff59e700 1 --2- 172.21.15.92:0/3499585313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd874c20 0x55e0dd873b10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8051dd0 tx=0x7f0cf8082da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11adb8448182a516 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:20.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.469+0000 7f0cdffff700 1 -- 172.21.15.92:0/3499585313 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8081070 con 0x55e0dd874c20 2023-10-06T10:41:20.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.469+0000 7f0cdffff700 1 -- 172.21.15.92:0/3499585313 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf805c870 con 0x55e0dd874c20 2023-10-06T10:41:20.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.469+0000 7f0cdffff700 1 -- 172.21.15.92:0/3499585313 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8053c70 con 0x55e0dd874c20 2023-10-06T10:41:20.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.469+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/3499585313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd874c20 msgr2=0x55e0dd873b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:20.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.469+0000 7f0d0e34fb80 1 --2- 172.21.15.92:0/3499585313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd874c20 0x55e0dd873b10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8051dd0 tx=0x7f0cf8082da0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.469+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/3499585313 shutdown_connections 2023-10-06T10:41:20.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.469+0000 7f0d0e34fb80 1 --2- 172.21.15.92:0/3499585313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd874c20 0x55e0dd873b10 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.469+0000 7f0d0e34fb80 1 --2- 172.21.15.92:0/3499585313 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e0dd8baae0 0x55e0dd8baeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.469+0000 7f0d0e34fb80 1 --2- 172.21.15.92:0/3499585313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e0dd8769f0 0x55e0dd8735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.469+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/3499585313 >> 172.21.15.92:0/3499585313 conn(0x55e0dd8750c0 msgr2=0x55e0dd875490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:20.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.470+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/3499585313 shutdown_connections 2023-10-06T10:41:20.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.470+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/3499585313 wait complete. 2023-10-06T10:41:20.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.470+0000 7f0d0e34fb80 1 -- start start 2023-10-06T10:41:20.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.470+0000 7f0d0e34fb80 1 Processor -- start 2023-10-06T10:41:20.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.471+0000 7f0d0e34fb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e0dd874c20 0x55e0dd8a32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:20.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.471+0000 7f0d0e34fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd8769f0 0x55e0dd89eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:20.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.471+0000 7f0d0e34fb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e0dd8baae0 0x55e0dd89f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:20.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.471+0000 7f0d0e34fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0dd7c40b0 con 0x55e0dd8769f0 2023-10-06T10:41:20.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.471+0000 7f0d0e34fb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0dd76f8e0 con 0x55e0dd874c20 2023-10-06T10:41:20.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.471+0000 7f0d0e34fb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0dd78aef0 con 0x55e0dd8baae0 2023-10-06T10:41:20.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.471+0000 7f0d0e34fb80 10 client.0 _ll_register_callbacks cb 0x55e0dd660f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:20.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.471+0000 7f0cf659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd8769f0 0x55e0dd89eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:20.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.471+0000 7f0cf659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd8769f0 0x55e0dd89eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40748/0 (socket says 172.21.15.92:40748) 2023-10-06T10:41:20.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.471+0000 7f0cf659c700 1 -- 172.21.15.92:0/3579238182 learned_addr learned my addr 172.21.15.92:0/3579238182 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:20.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.471+0000 7f0cff59e700 1 --2- 172.21.15.92:0/3579238182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e0dd8baae0 0x55e0dd89f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:20.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.471+0000 7f0cfed9d700 1 --2- 172.21.15.92:0/3579238182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e0dd874c20 0x55e0dd8a32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:20.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.472+0000 7f0cf659c700 1 -- 172.21.15.92:0/3579238182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e0dd8baae0 msgr2=0x55e0dd89f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:20.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.472+0000 7f0cf659c700 1 --2- 172.21.15.92:0/3579238182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e0dd8baae0 0x55e0dd89f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.472+0000 7f0cf659c700 1 -- 172.21.15.92:0/3579238182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e0dd874c20 msgr2=0x55e0dd8a32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:20.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.472+0000 7f0cf659c700 1 --2- 172.21.15.92:0/3579238182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e0dd874c20 0x55e0dd8a32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.472+0000 7f0cf659c700 1 -- 172.21.15.92:0/3579238182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d000aeba0 con 0x55e0dd8769f0 2023-10-06T10:41:20.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.472+0000 7f0cf659c700 1 --2- 172.21.15.92:0/3579238182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd8769f0 0x55e0dd89eaa0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8010340 tx=0x7f0ce8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ac50c6b128fca9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:20.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.472+0000 7f0cdd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:20.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.472+0000 7f0cdd7fa700 1 -- 172.21.15.92:0/3579238182 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8010da0 con 0x55e0dd8769f0 2023-10-06T10:41:20.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.472+0000 7f0cdd7fa700 1 -- 172.21.15.92:0/3579238182 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0ce8006820 con 0x55e0dd8769f0 2023-10-06T10:41:20.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.472+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/3579238182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0dd8686e0 con 0x55e0dd8769f0 2023-10-06T10:41:20.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.472+0000 7f0cdd7fa700 1 -- 172.21.15.92:0/3579238182 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ce8006bf0 con 0x55e0dd8769f0 2023-10-06T10:41:20.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.472+0000 7f0d0e34fb80 10 client.6201 fetch_fsmap learned FSMap version 42 2023-10-06T10:41:20.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.472+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/3579238182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x55e0dd601030 con 0x55e0dd8769f0 2023-10-06T10:41:20.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.472+0000 7f0cdd7fa700 1 -- 172.21.15.92:0/3579238182 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e0dd8686e0 con 0x55e0dd8769f0 2023-10-06T10:41:20.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.472+0000 7f0cdd7fa700 1 -- 172.21.15.92:0/3579238182 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 42) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f0ce8007020 con 0x55e0dd8769f0 2023-10-06T10:41:20.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.472+0000 7f0d0e34fb80 10 client.6201 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T10:41:20.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.472+0000 7f0d0e34fb80 10 client.6201 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2023-10-06T10:41:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.472+0000 7f0d0e34fb80 20 client.6201 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.472+0000 7f0d0e34fb80 1 --2- 172.21.15.92:0/3579238182 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x55e0dd89e320 0x55e0dd89bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.472+0000 7f0d0e34fb80 4 client.6201 mds_command: new command op to 5733 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:41:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.472+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/3579238182 --> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0dd845610 con 0x55e0dd89e320 2023-10-06T10:41:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.473+0000 7f0cfed9d700 1 --2- 172.21.15.92:0/3579238182 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x55e0dd89e320 0x55e0dd89bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.473+0000 7f0cdd7fa700 10 client.6201 ms_handle_connect on v2:172.21.15.92:6834/831923836 2023-10-06T10:41:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.473+0000 7f0cfed9d700 1 --2- 172.21.15.92:0/3579238182 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x55e0dd89e320 0x55e0dd89bbf0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be3d91797f4009f3 server_cookie=57db45c21a15d0cb in_seq=0 out_seq=0 2023-10-06T10:41:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.473+0000 7f0cdd7fa700 1 -- 172.21.15.92:0/3579238182 <== mds.1 v2:172.21.15.92:6834/831923836 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55e0dd845610 con 0x55e0dd89e320 2023-10-06T10:41:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.473+0000 7f0cdd7fa700 10 client.6201 handle_command_reply: tid=0 2023-10-06T10:41:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.473+0000 7f0d0e34fb80 1 client.6201 shutdown 2023-10-06T10:41:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.474+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/3579238182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd8769f0 msgr2=0x55e0dd89eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:20.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.474+0000 7f0d0e34fb80 1 --2- 172.21.15.92:0/3579238182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd8769f0 0x55e0dd89eaa0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8010340 tx=0x7f0ce8009040 comp rx=0 tx=0).stop 2023-10-06T10:41:20.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.474+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/3579238182 shutdown_connections 2023-10-06T10:41:20.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.474+0000 7f0d0e34fb80 1 --2- 172.21.15.92:0/3579238182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e0dd8baae0 0x55e0dd89f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.474+0000 7f0d0e34fb80 1 --2- 172.21.15.92:0/3579238182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e0dd874c20 0x55e0dd8a32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.474+0000 7f0d0e34fb80 1 --2- 172.21.15.92:0/3579238182 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x55e0dd89e320 0x55e0dd89bbf0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.474+0000 7f0d0e34fb80 1 --2- 172.21.15.92:0/3579238182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd8769f0 0x55e0dd89eaa0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.474+0000 7f0cdd7fa700 0 client.6201 ms_handle_reset on v2:172.21.15.92:6834/831923836 2023-10-06T10:41:20.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.474+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/3579238182 >> 172.21.15.92:0/3579238182 conn(0x55e0dd8750c0 msgr2=0x55e0dd8b9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:20.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.474+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/3579238182 shutdown_connections 2023-10-06T10:41:20.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.474+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/3579238182 wait complete. 2023-10-06T10:41:20.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.474+0000 7f0d0e34fb80 20 client.6201 trim_cache size 0 max 16384 2023-10-06T10:41:20.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.490+0000 7f0d0e34fb80 1 Processor -- start 2023-10-06T10:41:20.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.490+0000 7f0d0e34fb80 1 -- start start 2023-10-06T10:41:20.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.490+0000 7f0d0e34fb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e0dd874c20 0x55e0dd8b9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:20.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.490+0000 7f0d0e34fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd8769f0 0x55e0dd8cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:20.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.490+0000 7f0d0e34fb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e0dd8baae0 0x55e0dd8b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:20.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.490+0000 7f0d0e34fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0dd646e80 con 0x55e0dd8769f0 2023-10-06T10:41:20.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.490+0000 7f0d0e34fb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0dd8b9ec0 con 0x55e0dd8baae0 2023-10-06T10:41:20.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.490+0000 7f0d0e34fb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0dd8bb700 con 0x55e0dd874c20 2023-10-06T10:41:20.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.490+0000 7f0cf659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd8769f0 0x55e0dd8cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:20.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.491+0000 7f0cf659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd8769f0 0x55e0dd8cc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40760/0 (socket says 172.21.15.92:40760) 2023-10-06T10:41:20.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.491+0000 7f0cf659c700 1 -- 172.21.15.92:0/2602029347 learned_addr learned my addr 172.21.15.92:0/2602029347 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:20.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.491+0000 7f0cfed9d700 1 --2- 172.21.15.92:0/2602029347 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e0dd874c20 0x55e0dd8b9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:20.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.491+0000 7f0cff59e700 1 --2- 172.21.15.92:0/2602029347 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e0dd8baae0 0x55e0dd8b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:20.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.491+0000 7f0cf659c700 1 -- 172.21.15.92:0/2602029347 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e0dd874c20 msgr2=0x55e0dd8b9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:20.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.491+0000 7f0cf659c700 1 --2- 172.21.15.92:0/2602029347 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e0dd874c20 0x55e0dd8b9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.491+0000 7f0cf659c700 1 -- 172.21.15.92:0/2602029347 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e0dd8baae0 msgr2=0x55e0dd8b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:20.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.491+0000 7f0cf659c700 1 --2- 172.21.15.92:0/2602029347 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e0dd8baae0 0x55e0dd8b02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.491+0000 7f0cf659c700 1 -- 172.21.15.92:0/2602029347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0dd7927d0 con 0x55e0dd8769f0 2023-10-06T10:41:20.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.491+0000 7f0cfed9d700 1 --2- 172.21.15.92:0/2602029347 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e0dd874c20 0x55e0dd8b9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:20.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.491+0000 7f0cf659c700 1 --2- 172.21.15.92:0/2602029347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd8769f0 0x55e0dd8cc180 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8001ed0 tx=0x7f0ce8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30853b26b3c881b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:20.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.491+0000 7f0cf4d99700 1 -- 172.21.15.92:0/2602029347 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8006820 con 0x55e0dd8769f0 2023-10-06T10:41:20.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.491+0000 7f0cf4d99700 1 -- 172.21.15.92:0/2602029347 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce8007580 con 0x55e0dd8769f0 2023-10-06T10:41:20.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.491+0000 7f0cf4d99700 1 -- 172.21.15.92:0/2602029347 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8003370 con 0x55e0dd8769f0 2023-10-06T10:41:20.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.492+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/2602029347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd8769f0 msgr2=0x55e0dd8cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:20.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.492+0000 7f0d0e34fb80 1 --2- 172.21.15.92:0/2602029347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd8769f0 0x55e0dd8cc180 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8001ed0 tx=0x7f0ce8020000 comp rx=0 tx=0).stop 2023-10-06T10:41:20.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.492+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/2602029347 shutdown_connections 2023-10-06T10:41:20.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.492+0000 7f0d0e34fb80 1 --2- 172.21.15.92:0/2602029347 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e0dd8baae0 0x55e0dd8b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.492+0000 7f0d0e34fb80 1 --2- 172.21.15.92:0/2602029347 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e0dd874c20 0x55e0dd8b9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.492+0000 7f0d0e34fb80 1 --2- 172.21.15.92:0/2602029347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd8769f0 0x55e0dd8cc180 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.492+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/2602029347 >> 172.21.15.92:0/2602029347 conn(0x55e0dd8750c0 msgr2=0x55e0dd89bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:20.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.492+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/2602029347 shutdown_connections 2023-10-06T10:41:20.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.492+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/2602029347 wait complete. 2023-10-06T10:41:20.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.492+0000 7f0d0e34fb80 1 -- start start 2023-10-06T10:41:20.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.492+0000 7f0d0e34fb80 1 Processor -- start 2023-10-06T10:41:20.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.493+0000 7f0d0e34fb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e0dd874c20 0x55e0dd9169c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:20.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.493+0000 7f0d0e34fb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e0dd8769f0 0x55e0dd8735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:20.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.493+0000 7f0d0e34fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd8baae0 0x55e0dd8a3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:20.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.493+0000 7f0d0e34fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0dd9190b0 con 0x55e0dd8baae0 2023-10-06T10:41:20.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.493+0000 7f0d0e34fb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0dd9210b0 con 0x55e0dd8769f0 2023-10-06T10:41:20.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.493+0000 7f0d0e34fb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0dd919ed0 con 0x55e0dd874c20 2023-10-06T10:41:20.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.493+0000 7f0d0e34fb80 10 client.0 _ll_register_callbacks cb 0x55e0dd660f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:20.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.493+0000 7f0cff59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd8baae0 0x55e0dd8a3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:20.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.493+0000 7f0cff59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd8baae0 0x55e0dd8a3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40776/0 (socket says 172.21.15.92:40776) 2023-10-06T10:41:20.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.493+0000 7f0cff59e700 1 -- 172.21.15.92:0/3365611504 learned_addr learned my addr 172.21.15.92:0/3365611504 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:20.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.493+0000 7f0cf659c700 1 --2- 172.21.15.92:0/3365611504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e0dd8769f0 0x55e0dd8735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:20.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.493+0000 7f0cfed9d700 1 --2- 172.21.15.92:0/3365611504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e0dd874c20 0x55e0dd9169c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:20.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.494+0000 7f0cff59e700 1 -- 172.21.15.92:0/3365611504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e0dd874c20 msgr2=0x55e0dd9169c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:20.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.494+0000 7f0cff59e700 1 --2- 172.21.15.92:0/3365611504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e0dd874c20 0x55e0dd9169c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.494+0000 7f0cff59e700 1 -- 172.21.15.92:0/3365611504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e0dd8769f0 msgr2=0x55e0dd8735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:20.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.494+0000 7f0cff59e700 1 --2- 172.21.15.92:0/3365611504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e0dd8769f0 0x55e0dd8735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.494+0000 7f0cff59e700 1 -- 172.21.15.92:0/3365611504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0dd601030 con 0x55e0dd8baae0 2023-10-06T10:41:20.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.494+0000 7f0cff59e700 1 --2- 172.21.15.92:0/3365611504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd8baae0 0x55e0dd8a3250 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8055860 tx=0x7f0cf807f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b757154492f8d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:20.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.494+0000 7f0cdd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.494+0000 7f0cdd7fa700 1 -- 172.21.15.92:0/3365611504 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8081070 con 0x55e0dd8baae0 2023-10-06T10:41:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.494+0000 7f0cdd7fa700 1 -- 172.21.15.92:0/3365611504 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0cf8053a10 con 0x55e0dd8baae0 2023-10-06T10:41:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.494+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/3365611504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cf805c870 con 0x55e0dd8baae0 2023-10-06T10:41:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.494+0000 7f0cdd7fa700 1 -- 172.21.15.92:0/3365611504 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cf8062070 con 0x55e0dd8baae0 2023-10-06T10:41:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.494+0000 7f0d0e34fb80 10 client.6207 fetch_fsmap learned FSMap version 42 2023-10-06T10:41:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.494+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/3365611504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x55e0dd7794c0 con 0x55e0dd8baae0 2023-10-06T10:41:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.494+0000 7f0cdd7fa700 1 -- 172.21.15.92:0/3365611504 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf805c870 con 0x55e0dd8baae0 2023-10-06T10:41:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.494+0000 7f0cdd7fa700 1 -- 172.21.15.92:0/3365611504 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 42) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f0cf8065020 con 0x55e0dd8baae0 2023-10-06T10:41:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.494+0000 7f0d0e34fb80 10 client.6207 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T10:41:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.494+0000 7f0d0e34fb80 10 client.6207 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2023-10-06T10:41:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.494+0000 7f0d0e34fb80 20 client.6207 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:20.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.494+0000 7f0d0e34fb80 1 --2- 172.21.15.92:0/3365611504 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x55e0dd8a3790 0x55e0dd8a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:20.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.494+0000 7f0d0e34fb80 4 client.6207 mds_command: new command op to 5733 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:41:20.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.494+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/3365611504 --> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e0dd766e50 con 0x55e0dd8a3790 2023-10-06T10:41:20.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.494+0000 7f0cfed9d700 1 --2- 172.21.15.92:0/3365611504 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x55e0dd8a3790 0x55e0dd8a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:20.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.495+0000 7f0cdd7fa700 10 client.6207 ms_handle_connect on v2:172.21.15.92:6834/831923836 2023-10-06T10:41:20.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.495+0000 7f0cfed9d700 1 --2- 172.21.15.92:0/3365611504 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x55e0dd8a3790 0x55e0dd8a1cf0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da9730e60dbd4531 server_cookie=2d3c5997754b1dfe in_seq=0 out_seq=0 2023-10-06T10:41:20.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.495+0000 7f0cdd7fa700 1 -- 172.21.15.92:0/3365611504 <== mds.1 v2:172.21.15.92:6834/831923836 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55e0dd766e50 con 0x55e0dd8a3790 2023-10-06T10:41:20.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.495+0000 7f0cdd7fa700 10 client.6207 handle_command_reply: tid=0 2023-10-06T10:41:20.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.495+0000 7f0d0e34fb80 1 client.6207 shutdown 2023-10-06T10:41:20.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.495+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/3365611504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd8baae0 msgr2=0x55e0dd8a3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:20.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.495+0000 7f0d0e34fb80 1 --2- 172.21.15.92:0/3365611504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd8baae0 0x55e0dd8a3250 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8055860 tx=0x7f0cf807f000 comp rx=0 tx=0).stop 2023-10-06T10:41:20.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.495+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/3365611504 shutdown_connections 2023-10-06T10:41:20.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.495+0000 7f0d0e34fb80 1 --2- 172.21.15.92:0/3365611504 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x55e0dd8a3790 0x55e0dd8a1cf0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.495+0000 7f0d0e34fb80 1 --2- 172.21.15.92:0/3365611504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0dd8baae0 0x55e0dd8a3250 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.495+0000 7f0d0e34fb80 1 --2- 172.21.15.92:0/3365611504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e0dd874c20 0x55e0dd9169c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.495+0000 7f0d0e34fb80 1 --2- 172.21.15.92:0/3365611504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e0dd8769f0 0x55e0dd8735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.495+0000 7f0cdd7fa700 0 client.6207 ms_handle_reset on v2:172.21.15.92:6834/831923836 2023-10-06T10:41:20.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.495+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/3365611504 >> 172.21.15.92:0/3365611504 conn(0x55e0dd8750c0 msgr2=0x55e0dd8cc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:20.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.495+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/3365611504 shutdown_connections 2023-10-06T10:41:20.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.495+0000 7f0d0e34fb80 1 -- 172.21.15.92:0/3365611504 wait complete. 2023-10-06T10:41:20.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.495+0000 7f0d0e34fb80 20 client.6207 trim_cache size 0 max 16384 2023-10-06T10:41:20.506 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:41:20.506 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T10:41:20.506 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:41:20.506 INFO:teuthology.orchestra.run.smithi092.stdout: "9514c1ce-0eea-44fe-937b-b21b60ec3c5b": { 2023-10-06T10:41:20.506 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:41:20.506 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "9514c1ce-0eea-44fe-937b-b21b60ec3c5b", 2023-10-06T10:41:20.506 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:41:20.506 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:20.506 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:20.507 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:41:20.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.496+0000 7f0d05057700 1 -- 172.21.15.92:0/1547825304 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0ce0066f30 msgr2=0x7f0ce00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:20.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.496+0000 7f0d05057700 1 --2- 172.21.15.92:0/1547825304 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0ce0066f30 0x7f0ce00693e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00030c0 tx=0x7f0cf0016000 comp rx=0 tx=0).stop 2023-10-06T10:41:20.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.496+0000 7f0d05057700 1 -- 172.21.15.92:0/1547825304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d00159420 msgr2=0x7f0d0016d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:20.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.496+0000 7f0d05057700 1 --2- 172.21.15.92:0/1547825304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d00159420 0x7f0d0016d510 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800ccc0 tx=0x7f0cf8001a20 comp rx=0 tx=0).stop 2023-10-06T10:41:20.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.496+0000 7f0d05057700 1 -- 172.21.15.92:0/1547825304 shutdown_connections 2023-10-06T10:41:20.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.496+0000 7f0d05057700 1 --2- 172.21.15.92:0/1547825304 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0ce0066f30 0x7f0ce00693e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.496+0000 7f0d05057700 1 --2- 172.21.15.92:0/1547825304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d00159420 0x7f0d0016d510 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.496+0000 7f0d05057700 1 --2- 172.21.15.92:0/1547825304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d000689f0 0x7f0d000ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.496+0000 7f0d05057700 1 --2- 172.21.15.92:0/1547825304 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d000cb6d0 0x7f0d0016cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:20.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.496+0000 7f0d05057700 1 -- 172.21.15.92:0/1547825304 >> 172.21.15.92:0/1547825304 conn(0x7f0d0006ece0 msgr2=0x7f0d000af590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:20.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.496+0000 7f0d05057700 1 -- 172.21.15.92:0/1547825304 shutdown_connections 2023-10-06T10:41:20.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:20.496+0000 7f0d05057700 1 -- 172.21.15.92:0/1547825304 wait complete. 2023-10-06T10:41:23.017 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:41:23.020 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:41:23.307 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T10:41:23.306+0000 7f9e81bcf700 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T10:41:25.511 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:1 scrub status 2023-10-06T10:41:25.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.640+0000 7fa4797f8700 1 Processor -- start 2023-10-06T10:41:25.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.640+0000 7fa4797f8700 1 -- start start 2023-10-06T10:41:25.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.641+0000 7fa4797f8700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa474150940 0x7fa474159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.641+0000 7fa4797f8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47415ac30 con 0x7fa4740cd4d0 2023-10-06T10:41:25.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.641+0000 7fa4797f8700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4740cb6a0 con 0x7fa47415a160 2023-10-06T10:41:25.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.641+0000 7fa4797f8700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4740cb090 con 0x7fa474150940 2023-10-06T10:41:25.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.641+0000 7fa4737fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa4740cd4d0 0x7fa4740b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51062/0 (socket says 172.21.15.92:51062) 2023-10-06T10:41:25.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.641+0000 7fa4737fe700 1 -- 172.21.15.92:0/1944823422 learned_addr learned my addr 172.21.15.92:0/1944823422 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:25.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.641+0000 7fa472ffd700 1 --2- 172.21.15.92:0/1944823422 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa474150940 0x7fa474159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.641+0000 7fa471ffb700 1 -- 172.21.15.92:0/1944823422 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 340187621 0 0) 0x7fa464001490 con 0x7fa4740cd4d0 2023-10-06T10:41:25.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.641+0000 7fa471ffb700 1 -- 172.21.15.92:0/1944823422 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa460003570 con 0x7fa4740cd4d0 2023-10-06T10:41:25.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.641+0000 7fa471ffb700 1 -- 172.21.15.92:0/1944823422 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894905666 0 0) 0x7fa4740cb6a0 con 0x7fa47415a160 2023-10-06T10:41:25.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.641+0000 7fa471ffb700 1 -- 172.21.15.92:0/1944823422 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa464001490 con 0x7fa47415a160 2023-10-06T10:41:25.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.641+0000 7fa472ffd700 1 -- 172.21.15.92:0/1944823422 >> v1:172.21.15.124:6789/0 conn(0x7fa47415a160 legacy=0x7fa4740ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.641+0000 7fa472ffd700 1 -- 172.21.15.92:0/1944823422 >> v1:172.21.15.92:6789/0 conn(0x7fa4740cd4d0 legacy=0x7fa4740b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.641+0000 7fa472ffd700 1 -- 172.21.15.92:0/1944823422 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47415a670 con 0x7fa474150940 2023-10-06T10:41:25.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.642+0000 7fa472ffd700 1 --2- 172.21.15.92:0/1944823422 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa474150940 0x7fa474159c20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009900 tx=0x7fa46c009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb6d3c42736d3659 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:25.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.642+0000 7fa471ffb700 1 -- 172.21.15.92:0/1944823422 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c005c10 con 0x7fa474150940 2023-10-06T10:41:25.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.642+0000 7fa471ffb700 1 -- 172.21.15.92:0/1944823422 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c005d70 con 0x7fa474150940 2023-10-06T10:41:25.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.642+0000 7fa471ffb700 1 -- 172.21.15.92:0/1944823422 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c006260 con 0x7fa474150940 2023-10-06T10:41:25.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.642+0000 7fa4797f8700 1 -- 172.21.15.92:0/1944823422 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa474150940 msgr2=0x7fa474159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.642+0000 7fa4797f8700 1 --2- 172.21.15.92:0/1944823422 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa474150940 0x7fa474159c20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009900 tx=0x7fa46c009120 comp rx=0 tx=0).stop 2023-10-06T10:41:25.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.642+0000 7fa4797f8700 1 -- 172.21.15.92:0/1944823422 shutdown_connections 2023-10-06T10:41:25.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.642+0000 7fa4797f8700 1 --2- 172.21.15.92:0/1944823422 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa474150940 0x7fa474159c20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.642+0000 7fa4797f8700 1 -- 172.21.15.92:0/1944823422 >> 172.21.15.92:0/1944823422 conn(0x7fa474099700 msgr2=0x7fa47406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:25.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.642+0000 7fa4797f8700 1 -- 172.21.15.92:0/1944823422 shutdown_connections 2023-10-06T10:41:25.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.642+0000 7fa4797f8700 1 -- 172.21.15.92:0/1944823422 wait complete. 2023-10-06T10:41:25.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.644+0000 7fa4797f8700 1 Processor -- start 2023-10-06T10:41:25.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.644+0000 7fa4797f8700 1 -- start start 2023-10-06T10:41:25.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.644+0000 7fa4797f8700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4740cd4d0 0x7fa4741569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.644+0000 7fa4797f8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa47415a160 0x7fa47414fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.644+0000 7fa4797f8700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4740b58d0 0x7fa4740b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.644+0000 7fa4797f8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa474156510 con 0x7fa47415a160 2023-10-06T10:41:25.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.644+0000 7fa4797f8700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa474156ee0 con 0x7fa4740b58d0 2023-10-06T10:41:25.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.644+0000 7fa4797f8700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa474155900 con 0x7fa4740cd4d0 2023-10-06T10:41:25.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.645+0000 7fa4727fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa47415a160 0x7fa47414fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.645+0000 7fa4727fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa47415a160 0x7fa47414fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33144/0 (socket says 172.21.15.92:33144) 2023-10-06T10:41:25.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.645+0000 7fa4727fc700 1 -- 172.21.15.92:0/3944291495 learned_addr learned my addr 172.21.15.92:0/3944291495 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:25.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.645+0000 7fa472ffd700 1 --2- 172.21.15.92:0/3944291495 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4740cd4d0 0x7fa4741569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.645+0000 7fa4737fe700 1 --2- 172.21.15.92:0/3944291495 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4740b58d0 0x7fa4740b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.645+0000 7fa4727fc700 1 -- 172.21.15.92:0/3944291495 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4740cd4d0 msgr2=0x7fa4741569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.645+0000 7fa4727fc700 1 --2- 172.21.15.92:0/3944291495 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4740cd4d0 0x7fa4741569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.645+0000 7fa4727fc700 1 -- 172.21.15.92:0/3944291495 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4740b58d0 msgr2=0x7fa4740b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.645+0000 7fa4727fc700 1 --2- 172.21.15.92:0/3944291495 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4740b58d0 0x7fa4740b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.645+0000 7fa4727fc700 1 -- 172.21.15.92:0/3944291495 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46c0093c0 con 0x7fa47415a160 2023-10-06T10:41:25.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.645+0000 7fa472ffd700 1 --2- 172.21.15.92:0/3944291495 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4740cd4d0 0x7fa4741569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:41:25.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.645+0000 7fa4727fc700 1 --2- 172.21.15.92:0/3944291495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa47415a160 0x7fa47414fe20 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa45c00c590 tx=0x7fa45c007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:25.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.645+0000 7fa46affd700 1 -- 172.21.15.92:0/3944291495 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c011590 con 0x7fa47415a160 2023-10-06T10:41:25.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.645+0000 7fa46affd700 1 -- 172.21.15.92:0/3944291495 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45c011710 con 0x7fa47415a160 2023-10-06T10:41:25.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.645+0000 7fa4797f8700 1 -- 172.21.15.92:0/3944291495 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa474155be0 con 0x7fa47415a160 2023-10-06T10:41:25.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.645+0000 7fa46affd700 1 -- 172.21.15.92:0/3944291495 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c011ad0 con 0x7fa47415a160 2023-10-06T10:41:25.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.645+0000 7fa4797f8700 1 -- 172.21.15.92:0/3944291495 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4741561a0 con 0x7fa47415a160 2023-10-06T10:41:25.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.647+0000 7fa46affd700 1 -- 172.21.15.92:0/3944291495 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa45c012420 con 0x7fa47415a160 2023-10-06T10:41:25.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.647+0000 7fa46affd700 1 --2- 172.21.15.92:0/3944291495 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa454066f30 0x7fa4540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.647+0000 7fa46affd700 1 -- 172.21.15.92:0/3944291495 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa45c084120 con 0x7fa47415a160 2023-10-06T10:41:25.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.647+0000 7fa472ffd700 1 --2- 172.21.15.92:0/3944291495 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa454066f30 0x7fa4540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.647+0000 7fa472ffd700 1 --2- 172.21.15.92:0/3944291495 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa454066f30 0x7fa4540693e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009160 tx=0x7fa46c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:25.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.649+0000 7fa482af0b80 1 Processor -- start 2023-10-06T10:41:25.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.649+0000 7fa482af0b80 1 -- start start 2023-10-06T10:41:25.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.650+0000 7fa482af0b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ab6ab0ae0 0x558ab6ab0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.650+0000 7fa482af0b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ab6a6c9f0 0x558ab6a695d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.650+0000 7fa482af0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6a6ac20 0x558ab6a69b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.650+0000 7fa482af0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ab69e9cd0 con 0x558ab6a6ac20 2023-10-06T10:41:25.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.650+0000 7fa482af0b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ab69c0ac0 con 0x558ab6ab0ae0 2023-10-06T10:41:25.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.650+0000 7fa482af0b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ab69be2c0 con 0x558ab6a6c9f0 2023-10-06T10:41:25.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.650+0000 7fa4737fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6a6ac20 0x558ab6a69b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.650+0000 7fa4737fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6a6ac20 0x558ab6a69b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33154/0 (socket says 172.21.15.92:33154) 2023-10-06T10:41:25.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.650+0000 7fa4737fe700 1 -- 172.21.15.92:0/2282624665 learned_addr learned my addr 172.21.15.92:0/2282624665 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:25.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.650+0000 7fa4727fc700 1 --2- 172.21.15.92:0/2282624665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ab6a6c9f0 0x558ab6a695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.650+0000 7fa472ffd700 1 --2- 172.21.15.92:0/2282624665 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ab6ab0ae0 0x558ab6ab0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.650+0000 7fa4737fe700 1 -- 172.21.15.92:0/2282624665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ab6a6c9f0 msgr2=0x558ab6a695d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.650+0000 7fa4737fe700 1 --2- 172.21.15.92:0/2282624665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ab6a6c9f0 0x558ab6a695d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.650+0000 7fa4737fe700 1 -- 172.21.15.92:0/2282624665 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ab6ab0ae0 msgr2=0x558ab6ab0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.650+0000 7fa4737fe700 1 --2- 172.21.15.92:0/2282624665 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ab6ab0ae0 0x558ab6ab0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.650+0000 7fa4737fe700 1 -- 172.21.15.92:0/2282624665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ab68542c0 con 0x558ab6a6ac20 2023-10-06T10:41:25.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.650+0000 7fa472ffd700 1 --2- 172.21.15.92:0/2282624665 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ab6ab0ae0 0x558ab6ab0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:25.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.650+0000 7fa4737fe700 1 --2- 172.21.15.92:0/2282624665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6a6ac20 0x558ab6a69b10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa46400f300 tx=0x7fa464009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d08b03536af63ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:25.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.651+0000 7fa453fff700 1 -- 172.21.15.92:0/2282624665 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4640070d0 con 0x558ab6a6ac20 2023-10-06T10:41:25.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.651+0000 7fa453fff700 1 -- 172.21.15.92:0/2282624665 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa464007250 con 0x558ab6a6ac20 2023-10-06T10:41:25.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.651+0000 7fa453fff700 1 -- 172.21.15.92:0/2282624665 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa464007610 con 0x558ab6a6ac20 2023-10-06T10:41:25.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.651+0000 7fa482af0b80 1 -- 172.21.15.92:0/2282624665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6a6ac20 msgr2=0x558ab6a69b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.651+0000 7fa482af0b80 1 --2- 172.21.15.92:0/2282624665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6a6ac20 0x558ab6a69b10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa46400f300 tx=0x7fa464009040 comp rx=0 tx=0).stop 2023-10-06T10:41:25.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.651+0000 7fa482af0b80 1 -- 172.21.15.92:0/2282624665 shutdown_connections 2023-10-06T10:41:25.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.651+0000 7fa482af0b80 1 --2- 172.21.15.92:0/2282624665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6a6ac20 0x558ab6a69b10 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.651+0000 7fa482af0b80 1 --2- 172.21.15.92:0/2282624665 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ab6ab0ae0 0x558ab6ab0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.651+0000 7fa482af0b80 1 --2- 172.21.15.92:0/2282624665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ab6a6c9f0 0x558ab6a695d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.651+0000 7fa482af0b80 1 -- 172.21.15.92:0/2282624665 >> 172.21.15.92:0/2282624665 conn(0x558ab6a6b0c0 msgr2=0x558ab6a6b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:25.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.652+0000 7fa482af0b80 1 -- 172.21.15.92:0/2282624665 shutdown_connections 2023-10-06T10:41:25.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.652+0000 7fa482af0b80 1 -- 172.21.15.92:0/2282624665 wait complete. 2023-10-06T10:41:25.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.652+0000 7fa482af0b80 1 -- start start 2023-10-06T10:41:25.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.652+0000 7fa482af0b80 1 Processor -- start 2023-10-06T10:41:25.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.653+0000 7fa482af0b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ab6a6ac20 0x558ab6a90520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.653+0000 7fa482af0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6a6c9f0 0x558ab6aacdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.653+0000 7fa482af0b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ab6ab0ae0 0x558ab6a94170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.653+0000 7fa482af0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ab69ba0b0 con 0x558ab6a6c9f0 2023-10-06T10:41:25.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.653+0000 7fa482af0b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ab69658e0 con 0x558ab6ab0ae0 2023-10-06T10:41:25.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.653+0000 7fa482af0b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ab6980ef0 con 0x558ab6a6ac20 2023-10-06T10:41:25.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.653+0000 7fa482af0b80 10 client.0 _ll_register_callbacks cb 0x558ab6856f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:25.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.653+0000 7fa4727fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6a6c9f0 0x558ab6aacdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.653+0000 7fa4727fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6a6c9f0 0x558ab6aacdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33160/0 (socket says 172.21.15.92:33160) 2023-10-06T10:41:25.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.653+0000 7fa4727fc700 1 -- 172.21.15.92:0/292049712 learned_addr learned my addr 172.21.15.92:0/292049712 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:25.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.653+0000 7fa472ffd700 1 --2- 172.21.15.92:0/292049712 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ab6a6ac20 0x558ab6a90520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.653+0000 7fa4737fe700 1 --2- 172.21.15.92:0/292049712 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ab6ab0ae0 0x558ab6a94170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.654+0000 7fa4727fc700 1 -- 172.21.15.92:0/292049712 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ab6a6ac20 msgr2=0x558ab6a90520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.654+0000 7fa4727fc700 1 --2- 172.21.15.92:0/292049712 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ab6a6ac20 0x558ab6a90520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.654+0000 7fa4727fc700 1 -- 172.21.15.92:0/292049712 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ab6ab0ae0 msgr2=0x558ab6a94170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.654+0000 7fa4727fc700 1 --2- 172.21.15.92:0/292049712 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ab6ab0ae0 0x558ab6a94170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.654+0000 7fa4727fc700 1 -- 172.21.15.92:0/292049712 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa460003570 con 0x558ab6a6c9f0 2023-10-06T10:41:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.654+0000 7fa4727fc700 1 --2- 172.21.15.92:0/292049712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6a6c9f0 0x558ab6aacdf0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa45c001120 tx=0x7fa45c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452221c7f3286953 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.654+0000 7fa4517fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.654+0000 7fa4517fa700 1 -- 172.21.15.92:0/292049712 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c054b80 con 0x558ab6a6c9f0 2023-10-06T10:41:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.654+0000 7fa4517fa700 1 -- 172.21.15.92:0/292049712 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa45c05bd50 con 0x558ab6a6c9f0 2023-10-06T10:41:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.654+0000 7fa482af0b80 1 -- 172.21.15.92:0/292049712 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ab6a5e6e0 con 0x558ab6a6c9f0 2023-10-06T10:41:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.654+0000 7fa4517fa700 1 -- 172.21.15.92:0/292049712 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa45c0821a0 con 0x558ab6a6c9f0 2023-10-06T10:41:25.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.654+0000 7fa482af0b80 10 client.6219 fetch_fsmap learned FSMap version 42 2023-10-06T10:41:25.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.654+0000 7fa482af0b80 1 -- 172.21.15.92:0/292049712 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x558ab67f7030 con 0x558ab6a6c9f0 2023-10-06T10:41:25.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.654+0000 7fa4517fa700 1 -- 172.21.15.92:0/292049712 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558ab6a5e6e0 con 0x558ab6a6c9f0 2023-10-06T10:41:25.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.654+0000 7fa4517fa700 1 -- 172.21.15.92:0/292049712 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 42) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fa45c055010 con 0x558ab6a6c9f0 2023-10-06T10:41:25.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.654+0000 7fa482af0b80 10 client.6219 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T10:41:25.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.654+0000 7fa482af0b80 10 client.6219 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2023-10-06T10:41:25.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.654+0000 7fa482af0b80 20 client.6219 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:25.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.654+0000 7fa482af0b80 1 --2- 172.21.15.92:0/292049712 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x558ab6ac00a0 0x558ab6a91bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.654+0000 7fa482af0b80 4 client.6219 mds_command: new command op to 5733 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:41:25.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.654+0000 7fa482af0b80 1 -- 172.21.15.92:0/292049712 --> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ab6a3b610 con 0x558ab6ac00a0 2023-10-06T10:41:25.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.654+0000 7fa472ffd700 1 --2- 172.21.15.92:0/292049712 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x558ab6ac00a0 0x558ab6a91bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.655+0000 7fa4517fa700 10 client.6219 ms_handle_connect on v2:172.21.15.92:6834/831923836 2023-10-06T10:41:25.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.655+0000 7fa472ffd700 1 --2- 172.21.15.92:0/292049712 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x558ab6ac00a0 0x558ab6a91bf0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3ecaba01e404897 server_cookie=79b97ab219689434 in_seq=0 out_seq=0 2023-10-06T10:41:25.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.655+0000 7fa4517fa700 1 -- 172.21.15.92:0/292049712 <== mds.1 v2:172.21.15.92:6834/831923836 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x558ab6a3b610 con 0x558ab6ac00a0 2023-10-06T10:41:25.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.655+0000 7fa4517fa700 10 client.6219 handle_command_reply: tid=0 2023-10-06T10:41:25.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.655+0000 7fa482af0b80 1 client.6219 shutdown 2023-10-06T10:41:25.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.655+0000 7fa482af0b80 1 -- 172.21.15.92:0/292049712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6a6c9f0 msgr2=0x558ab6aacdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.655+0000 7fa482af0b80 1 --2- 172.21.15.92:0/292049712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6a6c9f0 0x558ab6aacdf0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa45c001120 tx=0x7fa45c014040 comp rx=0 tx=0).stop 2023-10-06T10:41:25.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.655+0000 7fa482af0b80 1 -- 172.21.15.92:0/292049712 shutdown_connections 2023-10-06T10:41:25.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.655+0000 7fa482af0b80 1 --2- 172.21.15.92:0/292049712 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ab6ab0ae0 0x558ab6a94170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.655+0000 7fa482af0b80 1 --2- 172.21.15.92:0/292049712 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ab6a6ac20 0x558ab6a90520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.655+0000 7fa482af0b80 1 --2- 172.21.15.92:0/292049712 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x558ab6ac00a0 0x558ab6a91bf0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.655+0000 7fa482af0b80 1 --2- 172.21.15.92:0/292049712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6a6c9f0 0x558ab6aacdf0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.655+0000 7fa4517fa700 0 client.6219 ms_handle_reset on v2:172.21.15.92:6834/831923836 2023-10-06T10:41:25.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.655+0000 7fa482af0b80 1 -- 172.21.15.92:0/292049712 >> 172.21.15.92:0/292049712 conn(0x558ab6a6b0c0 msgr2=0x558ab6a66870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:25.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.656+0000 7fa482af0b80 1 -- 172.21.15.92:0/292049712 shutdown_connections 2023-10-06T10:41:25.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.656+0000 7fa482af0b80 1 -- 172.21.15.92:0/292049712 wait complete. 2023-10-06T10:41:25.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.656+0000 7fa482af0b80 20 client.6219 trim_cache size 0 max 16384 2023-10-06T10:41:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.672+0000 7fa482af0b80 1 Processor -- start 2023-10-06T10:41:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.672+0000 7fa482af0b80 1 -- start start 2023-10-06T10:41:25.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.672+0000 7fa482af0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6a6ac20 0x558ab6aafbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.672+0000 7fa482af0b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ab6a6c9f0 0x558ab6ac9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.672+0000 7fa482af0b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ab6ab0ae0 0x558ab6abee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.672+0000 7fa482af0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ab683ce80 con 0x558ab6a6ac20 2023-10-06T10:41:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.672+0000 7fa482af0b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ab6a65c70 con 0x558ab6ab0ae0 2023-10-06T10:41:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.672+0000 7fa482af0b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ab6a95940 con 0x558ab6a6c9f0 2023-10-06T10:41:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.672+0000 7fa472ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6a6ac20 0x558ab6aafbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.672+0000 7fa472ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6a6ac20 0x558ab6aafbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33172/0 (socket says 172.21.15.92:33172) 2023-10-06T10:41:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.672+0000 7fa472ffd700 1 -- 172.21.15.92:0/3123202540 learned_addr learned my addr 172.21.15.92:0/3123202540 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.673+0000 7fa4727fc700 1 --2- 172.21.15.92:0/3123202540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ab6a6c9f0 0x558ab6ac9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.673+0000 7fa472ffd700 1 -- 172.21.15.92:0/3123202540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ab6a6c9f0 msgr2=0x558ab6ac9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.673+0000 7fa4737fe700 1 --2- 172.21.15.92:0/3123202540 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ab6ab0ae0 0x558ab6abee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.673+0000 7fa472ffd700 1 --2- 172.21.15.92:0/3123202540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ab6a6c9f0 0x558ab6ac9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.673+0000 7fa472ffd700 1 -- 172.21.15.92:0/3123202540 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ab6ab0ae0 msgr2=0x558ab6abee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.673+0000 7fa472ffd700 1 --2- 172.21.15.92:0/3123202540 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ab6ab0ae0 0x558ab6abee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.673+0000 7fa472ffd700 1 -- 172.21.15.92:0/3123202540 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ab69887d0 con 0x558ab6a6ac20 2023-10-06T10:41:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.673+0000 7fa472ffd700 1 --2- 172.21.15.92:0/3123202540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6a6ac20 0x558ab6aafbf0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa4740d14c0 tx=0x7fa46c01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07e1b7a6c861114 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.673+0000 7fa468ff9700 1 -- 172.21.15.92:0/3123202540 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c02e030 con 0x558ab6a6ac20 2023-10-06T10:41:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.673+0000 7fa468ff9700 1 -- 172.21.15.92:0/3123202540 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c02f040 con 0x558ab6a6ac20 2023-10-06T10:41:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.673+0000 7fa468ff9700 1 -- 172.21.15.92:0/3123202540 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c023070 con 0x558ab6a6ac20 2023-10-06T10:41:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.673+0000 7fa482af0b80 1 -- 172.21.15.92:0/3123202540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6a6ac20 msgr2=0x558ab6aafbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.673+0000 7fa482af0b80 1 --2- 172.21.15.92:0/3123202540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6a6ac20 0x558ab6aafbf0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa4740d14c0 tx=0x7fa46c01e040 comp rx=0 tx=0).stop 2023-10-06T10:41:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.673+0000 7fa482af0b80 1 -- 172.21.15.92:0/3123202540 shutdown_connections 2023-10-06T10:41:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.673+0000 7fa482af0b80 1 --2- 172.21.15.92:0/3123202540 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ab6ab0ae0 0x558ab6abee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.673+0000 7fa482af0b80 1 --2- 172.21.15.92:0/3123202540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6a6ac20 0x558ab6aafbf0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.673+0000 7fa482af0b80 1 --2- 172.21.15.92:0/3123202540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ab6a6c9f0 0x558ab6ac9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.674+0000 7fa482af0b80 1 -- 172.21.15.92:0/3123202540 >> 172.21.15.92:0/3123202540 conn(0x558ab6a6b0c0 msgr2=0x558ab6a66870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.674+0000 7fa482af0b80 1 -- 172.21.15.92:0/3123202540 shutdown_connections 2023-10-06T10:41:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.674+0000 7fa482af0b80 1 -- 172.21.15.92:0/3123202540 wait complete. 2023-10-06T10:41:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.674+0000 7fa482af0b80 1 -- start start 2023-10-06T10:41:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.674+0000 7fa482af0b80 1 Processor -- start 2023-10-06T10:41:25.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.675+0000 7fa482af0b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ab6a6ac20 0x558ab6a65180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.675+0000 7fa482af0b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ab6a6c9f0 0x558ab6aab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.675+0000 7fa482af0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6ab0ae0 0x558ab6a8d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.675+0000 7fa482af0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ab6a8de20 con 0x558ab6ab0ae0 2023-10-06T10:41:25.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.675+0000 7fa482af0b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ab6ac9bf0 con 0x558ab6a6ac20 2023-10-06T10:41:25.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.675+0000 7fa482af0b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ab6a6a110 con 0x558ab6a6c9f0 2023-10-06T10:41:25.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.675+0000 7fa482af0b80 10 client.0 _ll_register_callbacks cb 0x558ab6a6bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:25.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.675+0000 7fa4737fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6ab0ae0 0x558ab6a8d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.675+0000 7fa4737fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6ab0ae0 0x558ab6a8d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33188/0 (socket says 172.21.15.92:33188) 2023-10-06T10:41:25.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.675+0000 7fa4737fe700 1 -- 172.21.15.92:0/4163050906 learned_addr learned my addr 172.21.15.92:0/4163050906 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:25.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.675+0000 7fa472ffd700 1 --2- 172.21.15.92:0/4163050906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ab6a6ac20 0x558ab6a65180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.675+0000 7fa4727fc700 1 --2- 172.21.15.92:0/4163050906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ab6a6c9f0 0x558ab6aab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.675+0000 7fa4737fe700 1 -- 172.21.15.92:0/4163050906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ab6a6c9f0 msgr2=0x558ab6aab4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.675+0000 7fa4737fe700 1 --2- 172.21.15.92:0/4163050906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ab6a6c9f0 0x558ab6aab4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.675+0000 7fa4737fe700 1 -- 172.21.15.92:0/4163050906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ab6a6ac20 msgr2=0x558ab6a65180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.675+0000 7fa4737fe700 1 --2- 172.21.15.92:0/4163050906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ab6a6ac20 0x558ab6a65180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.675+0000 7fa4737fe700 1 -- 172.21.15.92:0/4163050906 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa47415a670 con 0x558ab6ab0ae0 2023-10-06T10:41:25.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.676+0000 7fa4737fe700 1 --2- 172.21.15.92:0/4163050906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6ab0ae0 0x558ab6a8d350 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa464001710 tx=0x7fa464018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342968b8c48fa948 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:25.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.676+0000 7fa4517fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:25.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.676+0000 7fa4517fa700 1 -- 172.21.15.92:0/4163050906 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa464023070 con 0x558ab6ab0ae0 2023-10-06T10:41:25.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.676+0000 7fa4517fa700 1 -- 172.21.15.92:0/4163050906 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa464001f80 con 0x558ab6ab0ae0 2023-10-06T10:41:25.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.676+0000 7fa482af0b80 1 -- 172.21.15.92:0/4163050906 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa464007250 con 0x558ab6ab0ae0 2023-10-06T10:41:25.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.676+0000 7fa4517fa700 1 -- 172.21.15.92:0/4163050906 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa46401c630 con 0x558ab6ab0ae0 2023-10-06T10:41:25.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.676+0000 7fa482af0b80 10 client.6225 fetch_fsmap learned FSMap version 42 2023-10-06T10:41:25.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.676+0000 7fa482af0b80 1 -- 172.21.15.92:0/4163050906 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x558ab696f4c0 con 0x558ab6ab0ae0 2023-10-06T10:41:25.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.676+0000 7fa4517fa700 1 -- 172.21.15.92:0/4163050906 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa464007250 con 0x558ab6ab0ae0 2023-10-06T10:41:25.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.676+0000 7fa4517fa700 1 -- 172.21.15.92:0/4163050906 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 42) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fa464012070 con 0x558ab6ab0ae0 2023-10-06T10:41:25.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.676+0000 7fa482af0b80 10 client.6225 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T10:41:25.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.676+0000 7fa482af0b80 10 client.6225 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2023-10-06T10:41:25.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.676+0000 7fa482af0b80 20 client.6225 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:25.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.676+0000 7fa482af0b80 1 --2- 172.21.15.92:0/4163050906 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x558ab6a8d890 0x558ab6aa62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.676+0000 7fa482af0b80 4 client.6225 mds_command: new command op to 5733 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:41:25.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.676+0000 7fa482af0b80 1 -- 172.21.15.92:0/4163050906 --> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558ab695ce50 con 0x558ab6a8d890 2023-10-06T10:41:25.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.676+0000 7fa472ffd700 1 --2- 172.21.15.92:0/4163050906 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x558ab6a8d890 0x558ab6aa62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.677+0000 7fa4517fa700 10 client.6225 ms_handle_connect on v2:172.21.15.92:6834/831923836 2023-10-06T10:41:25.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.677+0000 7fa472ffd700 1 --2- 172.21.15.92:0/4163050906 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x558ab6a8d890 0x558ab6aa62b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84c09d9d70577bbe server_cookie=400767004bda0e46 in_seq=0 out_seq=0 2023-10-06T10:41:25.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.677+0000 7fa4517fa700 1 -- 172.21.15.92:0/4163050906 <== mds.1 v2:172.21.15.92:6834/831923836 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558ab695ce50 con 0x558ab6a8d890 2023-10-06T10:41:25.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.677+0000 7fa4517fa700 10 client.6225 handle_command_reply: tid=0 2023-10-06T10:41:25.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.677+0000 7fa482af0b80 1 client.6225 shutdown 2023-10-06T10:41:25.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.677+0000 7fa482af0b80 1 -- 172.21.15.92:0/4163050906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6ab0ae0 msgr2=0x558ab6a8d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.677+0000 7fa482af0b80 1 --2- 172.21.15.92:0/4163050906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6ab0ae0 0x558ab6a8d350 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa464001710 tx=0x7fa464018000 comp rx=0 tx=0).stop 2023-10-06T10:41:25.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.677+0000 7fa482af0b80 1 -- 172.21.15.92:0/4163050906 shutdown_connections 2023-10-06T10:41:25.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.677+0000 7fa482af0b80 1 --2- 172.21.15.92:0/4163050906 >> [v2:172.21.15.92:6834/831923836,v1:172.21.15.92:6836/831923836] conn(0x558ab6a8d890 0x558ab6aa62b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.677+0000 7fa482af0b80 1 --2- 172.21.15.92:0/4163050906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ab6ab0ae0 0x558ab6a8d350 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.677+0000 7fa4517fa700 0 client.6225 ms_handle_reset on v2:172.21.15.92:6834/831923836 2023-10-06T10:41:25.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.677+0000 7fa482af0b80 1 --2- 172.21.15.92:0/4163050906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ab6a6ac20 0x558ab6a65180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.677+0000 7fa482af0b80 1 --2- 172.21.15.92:0/4163050906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ab6a6c9f0 0x558ab6aab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.677+0000 7fa482af0b80 1 -- 172.21.15.92:0/4163050906 >> 172.21.15.92:0/4163050906 conn(0x558ab6a6b0c0 msgr2=0x558ab6a90520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:25.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.677+0000 7fa482af0b80 1 -- 172.21.15.92:0/4163050906 shutdown_connections 2023-10-06T10:41:25.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.677+0000 7fa482af0b80 1 -- 172.21.15.92:0/4163050906 wait complete. 2023-10-06T10:41:25.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.677+0000 7fa482af0b80 20 client.6225 trim_cache size 0 max 16384 2023-10-06T10:41:25.686 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:41:25.686 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "no active scrubs running", 2023-10-06T10:41:25.686 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": {} 2023-10-06T10:41:25.686 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:41:25.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.678+0000 7fa4797f8700 1 -- 172.21.15.92:0/3944291495 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa454066f30 msgr2=0x7fa4540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.678+0000 7fa4797f8700 1 --2- 172.21.15.92:0/3944291495 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa454066f30 0x7fa4540693e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009160 tx=0x7fa46c016000 comp rx=0 tx=0).stop 2023-10-06T10:41:25.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.678+0000 7fa4797f8700 1 -- 172.21.15.92:0/3944291495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa47415a160 msgr2=0x7fa47414fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.678+0000 7fa4797f8700 1 --2- 172.21.15.92:0/3944291495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa47415a160 0x7fa47414fe20 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa45c00c590 tx=0x7fa45c007d60 comp rx=0 tx=0).stop 2023-10-06T10:41:25.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.678+0000 7fa4797f8700 1 -- 172.21.15.92:0/3944291495 shutdown_connections 2023-10-06T10:41:25.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.678+0000 7fa4797f8700 1 --2- 172.21.15.92:0/3944291495 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa454066f30 0x7fa4540693e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.678+0000 7fa4797f8700 1 --2- 172.21.15.92:0/3944291495 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4740b58d0 0x7fa4740b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.678+0000 7fa4797f8700 1 --2- 172.21.15.92:0/3944291495 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4740cd4d0 0x7fa4741569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.678+0000 7fa4797f8700 1 --2- 172.21.15.92:0/3944291495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa47415a160 0x7fa47414fe20 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.678+0000 7fa4797f8700 1 -- 172.21.15.92:0/3944291495 >> 172.21.15.92:0/3944291495 conn(0x7fa474099700 msgr2=0x7fa4740b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:25.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.678+0000 7fa4797f8700 1 -- 172.21.15.92:0/3944291495 shutdown_connections 2023-10-06T10:41:25.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.678+0000 7fa4797f8700 1 -- 172.21.15.92:0/3944291495 wait complete. 2023-10-06T10:41:25.691 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:41:25.691 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:2 scrub status 2023-10-06T10:41:25.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.805+0000 7f6df97d8700 1 Processor -- start 2023-10-06T10:41:25.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.805+0000 7f6df97d8700 1 -- start start 2023-10-06T10:41:25.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.805+0000 7f6df97d8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6df4157450 0x7f6df40cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.805+0000 7f6df97d8700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6df40d0310 0x7f6df40d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.805+0000 7f6df97d8700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df40ccb30 con 0x7f6df4159e20 2023-10-06T10:41:25.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.805+0000 7f6df97d8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df40cbde0 con 0x7f6df4157450 2023-10-06T10:41:25.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.805+0000 7f6df97d8700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df40cf040 con 0x7f6df40d0310 2023-10-06T10:41:25.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.806+0000 7f6df2ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6df4157450 0x7f6df40cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.806+0000 7f6df2ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6df4157450 0x7f6df40cb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33200/0 (socket says 172.21.15.92:33200) 2023-10-06T10:41:25.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.806+0000 7f6df2ffd700 1 -- 172.21.15.92:0/1542364611 learned_addr learned my addr 172.21.15.92:0/1542364611 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:25.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.806+0000 7f6df37fe700 1 --2- 172.21.15.92:0/1542364611 >> v2:172.21.15.124:3300/0 conn(0x7f6df40d0310 0x7f6df40d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.806+0000 7f6df2ffd700 1 -- 172.21.15.92:0/1542364611 >> v2:172.21.15.124:3300/0 conn(0x7f6df40d0310 msgr2=0x7f6df40d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.806+0000 7f6df2ffd700 1 --2- 172.21.15.92:0/1542364611 >> v2:172.21.15.124:3300/0 conn(0x7f6df40d0310 0x7f6df40d0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.806+0000 7f6df2ffd700 1 -- 172.21.15.92:0/1542364611 >> v1:172.21.15.124:6789/0 conn(0x7f6df4159e20 legacy=0x7f6df40ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.806+0000 7f6df2ffd700 1 -- 172.21.15.92:0/1542364611 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df415ae90 con 0x7f6df4157450 2023-10-06T10:41:25.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.806+0000 7f6df37fe700 1 --2- 172.21.15.92:0/1542364611 >> v2:172.21.15.124:3300/0 conn(0x7f6df40d0310 0x7f6df40d0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:25.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.806+0000 7f6df2ffd700 1 --2- 172.21.15.92:0/1542364611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6df4157450 0x7f6df40cb8a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6df40ccd70 tx=0x7f6de4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b7bfd6244aac02 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:25.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.806+0000 7f6df27fc700 1 -- 172.21.15.92:0/1542364611 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de400b820 con 0x7f6df4157450 2023-10-06T10:41:25.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.806+0000 7f6df27fc700 1 -- 172.21.15.92:0/1542364611 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de400b980 con 0x7f6df4157450 2023-10-06T10:41:25.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.806+0000 7f6df27fc700 1 -- 172.21.15.92:0/1542364611 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de400bd20 con 0x7f6df4157450 2023-10-06T10:41:25.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.807+0000 7f6df97d8700 1 -- 172.21.15.92:0/1542364611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6df4157450 msgr2=0x7f6df40cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.807+0000 7f6df97d8700 1 --2- 172.21.15.92:0/1542364611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6df4157450 0x7f6df40cb8a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6df40ccd70 tx=0x7f6de4009250 comp rx=0 tx=0).stop 2023-10-06T10:41:25.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.807+0000 7f6df97d8700 1 -- 172.21.15.92:0/1542364611 shutdown_connections 2023-10-06T10:41:25.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.807+0000 7f6df97d8700 1 --2- 172.21.15.92:0/1542364611 >> v2:172.21.15.124:3300/0 conn(0x7f6df40d0310 0x7f6df40d0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.807+0000 7f6df97d8700 1 --2- 172.21.15.92:0/1542364611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6df4157450 0x7f6df40cb8a0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.807+0000 7f6df97d8700 1 -- 172.21.15.92:0/1542364611 >> 172.21.15.92:0/1542364611 conn(0x7f6df406ece0 msgr2=0x7f6df406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:25.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.807+0000 7f6df97d8700 1 -- 172.21.15.92:0/1542364611 shutdown_connections 2023-10-06T10:41:25.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.807+0000 7f6df97d8700 1 -- 172.21.15.92:0/1542364611 wait complete. 2023-10-06T10:41:25.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.808+0000 7f6df97d8700 1 Processor -- start 2023-10-06T10:41:25.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.808+0000 7f6df97d8700 1 -- start start 2023-10-06T10:41:25.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.809+0000 7f6df97d8700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6df40d0310 0x7f6df4160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.809+0000 7f6df97d8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6df4157450 0x7f6df40c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.809+0000 7f6df97d8700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6df4159e20 0x7f6df4169cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.809+0000 7f6df97d8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df40c8a50 con 0x7f6df4157450 2023-10-06T10:41:25.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.809+0000 7f6df97d8700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df40c9010 con 0x7f6df4159e20 2023-10-06T10:41:25.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.809+0000 7f6df97d8700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df4160dd0 con 0x7f6df40d0310 2023-10-06T10:41:25.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.809+0000 7f6dea7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6df4157450 0x7f6df40c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.809+0000 7f6dea7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6df4157450 0x7f6df40c94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33206/0 (socket says 172.21.15.92:33206) 2023-10-06T10:41:25.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.809+0000 7f6dea7fc700 1 -- 172.21.15.92:0/1176036853 learned_addr learned my addr 172.21.15.92:0/1176036853 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:25.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.809+0000 7f6df2ffd700 1 --2- 172.21.15.92:0/1176036853 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6df40d0310 0x7f6df4160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.809+0000 7f6df37fe700 1 --2- 172.21.15.92:0/1176036853 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6df4159e20 0x7f6df4169cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.809+0000 7f6dea7fc700 1 -- 172.21.15.92:0/1176036853 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6df40d0310 msgr2=0x7f6df4160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.809+0000 7f6dea7fc700 1 --2- 172.21.15.92:0/1176036853 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6df40d0310 0x7f6df4160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.809+0000 7f6dea7fc700 1 -- 172.21.15.92:0/1176036853 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6df4159e20 msgr2=0x7f6df4169cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.809+0000 7f6dea7fc700 1 --2- 172.21.15.92:0/1176036853 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6df4159e20 0x7f6df4169cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.809+0000 7f6dea7fc700 1 -- 172.21.15.92:0/1176036853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de40094f0 con 0x7f6df4157450 2023-10-06T10:41:25.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.809+0000 7f6dea7fc700 1 --2- 172.21.15.92:0/1176036853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6df4157450 0x7f6df40c94a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00c510 tx=0x7f6ddc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:25.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.809+0000 7f6deb7fe700 1 -- 172.21.15.92:0/1176036853 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc011590 con 0x7f6df4157450 2023-10-06T10:41:25.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.809+0000 7f6deb7fe700 1 -- 172.21.15.92:0/1176036853 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc011710 con 0x7f6df4157450 2023-10-06T10:41:25.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.809+0000 7f6df97d8700 1 -- 172.21.15.92:0/1176036853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df40ca740 con 0x7f6df4157450 2023-10-06T10:41:25.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.809+0000 7f6deb7fe700 1 -- 172.21.15.92:0/1176036853 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc011ad0 con 0x7f6df4157450 2023-10-06T10:41:25.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.809+0000 7f6df97d8700 1 -- 172.21.15.92:0/1176036853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df416aaf0 con 0x7f6df4157450 2023-10-06T10:41:25.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.811+0000 7f6deb7fe700 1 -- 172.21.15.92:0/1176036853 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ddc01b3b0 con 0x7f6df4157450 2023-10-06T10:41:25.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.811+0000 7f6deb7fe700 1 --2- 172.21.15.92:0/1176036853 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6dd4066f30 0x7f6dd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.811+0000 7f6df2ffd700 1 --2- 172.21.15.92:0/1176036853 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6dd4066f30 0x7f6dd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.811+0000 7f6deb7fe700 1 -- 172.21.15.92:0/1176036853 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6ddc083ba0 con 0x7f6df4157450 2023-10-06T10:41:25.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.812+0000 7f6df2ffd700 1 --2- 172.21.15.92:0/1176036853 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6dd4066f30 0x7f6dd40693e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6df40ccd70 tx=0x7f6de4017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:25.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.813+0000 7f6e02ad0b80 1 Processor -- start 2023-10-06T10:41:25.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.813+0000 7f6e02ad0b80 1 -- start start 2023-10-06T10:41:25.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.814+0000 7f6e02ad0b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5ba0aae0 0x55cf5ba0aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.814+0000 7f6e02ad0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5b9c69f0 0x55cf5b9c35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.814+0000 7f6e02ad0b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55cf5b9c4c20 0x55cf5b9c3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.814+0000 7f6e02ad0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5b943cd0 con 0x55cf5b9c69f0 2023-10-06T10:41:25.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.814+0000 7f6e02ad0b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5b91aac0 con 0x55cf5ba0aae0 2023-10-06T10:41:25.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.814+0000 7f6e02ad0b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cf5b9182c0 con 0x55cf5b9c4c20 2023-10-06T10:41:25.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.814+0000 7f6dea7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5b9c69f0 0x55cf5b9c35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.814+0000 7f6dea7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5b9c69f0 0x55cf5b9c35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33218/0 (socket says 172.21.15.92:33218) 2023-10-06T10:41:25.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.814+0000 7f6dea7fc700 1 -- 172.21.15.92:0/410559813 learned_addr learned my addr 172.21.15.92:0/410559813 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:25.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.814+0000 7f6dea7fc700 1 -- 172.21.15.92:0/410559813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55cf5b9c4c20 msgr2=0x55cf5b9c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.814+0000 7f6dea7fc700 1 --2- 172.21.15.92:0/410559813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55cf5b9c4c20 0x55cf5b9c3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.814+0000 7f6dea7fc700 1 -- 172.21.15.92:0/410559813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5ba0aae0 msgr2=0x55cf5ba0aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.814+0000 7f6df2ffd700 1 --2- 172.21.15.92:0/410559813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5ba0aae0 0x55cf5ba0aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.814+0000 7f6dea7fc700 1 --2- 172.21.15.92:0/410559813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5ba0aae0 0x55cf5ba0aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.814+0000 7f6dea7fc700 1 -- 172.21.15.92:0/410559813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf5b7ae2c0 con 0x55cf5b9c69f0 2023-10-06T10:41:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.814+0000 7f6df2ffd700 1 --2- 172.21.15.92:0/410559813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5ba0aae0 0x55cf5ba0aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.815+0000 7f6dea7fc700 1 --2- 172.21.15.92:0/410559813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5b9c69f0 0x55cf5b9c35d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc001050 tx=0x7f6ddc081ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab431ae0d841bfe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.815+0000 7f6dd3fff700 1 -- 172.21.15.92:0/410559813 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc054790 con 0x55cf5b9c69f0 2023-10-06T10:41:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.815+0000 7f6dd3fff700 1 -- 172.21.15.92:0/410559813 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc0579e0 con 0x55cf5b9c69f0 2023-10-06T10:41:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.815+0000 7f6dd3fff700 1 -- 172.21.15.92:0/410559813 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc05b730 con 0x55cf5b9c69f0 2023-10-06T10:41:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.815+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/410559813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5b9c69f0 msgr2=0x55cf5b9c35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.815+0000 7f6e02ad0b80 1 --2- 172.21.15.92:0/410559813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5b9c69f0 0x55cf5b9c35d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc001050 tx=0x7f6ddc081ce0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.815+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/410559813 shutdown_connections 2023-10-06T10:41:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.815+0000 7f6e02ad0b80 1 --2- 172.21.15.92:0/410559813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55cf5b9c4c20 0x55cf5b9c3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.815+0000 7f6e02ad0b80 1 --2- 172.21.15.92:0/410559813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5ba0aae0 0x55cf5ba0aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.815+0000 7f6e02ad0b80 1 --2- 172.21.15.92:0/410559813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5b9c69f0 0x55cf5b9c35d0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.815+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/410559813 >> 172.21.15.92:0/410559813 conn(0x55cf5b9c50c0 msgr2=0x55cf5b9c5490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.815+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/410559813 shutdown_connections 2023-10-06T10:41:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.815+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/410559813 wait complete. 2023-10-06T10:41:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.816+0000 7f6e02ad0b80 1 -- start start 2023-10-06T10:41:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.816+0000 7f6e02ad0b80 1 Processor -- start 2023-10-06T10:41:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.817+0000 7f6e02ad0b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55cf5b9c4c20 0x55cf5ba04370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.817+0000 7f6e02ad0b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5b9c69f0 0x55cf5b9ee0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.817+0000 7f6e02ad0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5ba0aae0 0x55cf5b9ebbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.817+0000 7f6e02ad0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5b9140b0 con 0x55cf5ba0aae0 2023-10-06T10:41:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.817+0000 7f6e02ad0b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5b8bf8e0 con 0x55cf5b9c69f0 2023-10-06T10:41:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.817+0000 7f6e02ad0b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cf5b8daef0 con 0x55cf5b9c4c20 2023-10-06T10:41:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.817+0000 7f6e02ad0b80 10 client.0 _ll_register_callbacks cb 0x55cf5b7b0f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.817+0000 7f6df37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5ba0aae0 0x55cf5b9ebbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.817+0000 7f6df37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5ba0aae0 0x55cf5b9ebbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33224/0 (socket says 172.21.15.92:33224) 2023-10-06T10:41:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.817+0000 7f6df37fe700 1 -- 172.21.15.92:0/2060142063 learned_addr learned my addr 172.21.15.92:0/2060142063 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.817+0000 7f6df2ffd700 1 --2- 172.21.15.92:0/2060142063 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55cf5b9c4c20 0x55cf5ba04370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.817+0000 7f6dea7fc700 1 --2- 172.21.15.92:0/2060142063 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5b9c69f0 0x55cf5b9ee0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.817+0000 7f6df37fe700 1 -- 172.21.15.92:0/2060142063 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55cf5b9c4c20 msgr2=0x55cf5ba04370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.817+0000 7f6df37fe700 1 --2- 172.21.15.92:0/2060142063 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55cf5b9c4c20 0x55cf5ba04370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.817+0000 7f6df37fe700 1 -- 172.21.15.92:0/2060142063 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5b9c69f0 msgr2=0x55cf5b9ee0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.817+0000 7f6df37fe700 1 --2- 172.21.15.92:0/2060142063 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5b9c69f0 0x55cf5b9ee0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.817+0000 7f6df37fe700 1 -- 172.21.15.92:0/2060142063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df416aaf0 con 0x55cf5ba0aae0 2023-10-06T10:41:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.817+0000 7f6df37fe700 1 --2- 172.21.15.92:0/2060142063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5ba0aae0 0x55cf5b9ebbc0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6dec002dd0 tx=0x7f6dec009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20164b5baf023f29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.817+0000 7f6dd17fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.818+0000 7f6dd17fa700 1 -- 172.21.15.92:0/2060142063 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec007900 con 0x55cf5ba0aae0 2023-10-06T10:41:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.818+0000 7f6dd17fa700 1 -- 172.21.15.92:0/2060142063 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6dec007a60 con 0x55cf5ba0aae0 2023-10-06T10:41:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.818+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/2060142063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf5b9b86e0 con 0x55cf5ba0aae0 2023-10-06T10:41:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.818+0000 7f6dd17fa700 1 -- 172.21.15.92:0/2060142063 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dec007e70 con 0x55cf5ba0aae0 2023-10-06T10:41:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.818+0000 7f6e02ad0b80 10 client.6237 fetch_fsmap learned FSMap version 42 2023-10-06T10:41:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.818+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/2060142063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x55cf5b751030 con 0x55cf5ba0aae0 2023-10-06T10:41:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.818+0000 7f6dd17fa700 1 -- 172.21.15.92:0/2060142063 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cf5b9b86e0 con 0x55cf5ba0aae0 2023-10-06T10:41:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.818+0000 7f6dd17fa700 1 -- 172.21.15.92:0/2060142063 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 42) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f6dec01b2f0 con 0x55cf5ba0aae0 2023-10-06T10:41:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.818+0000 7f6e02ad0b80 10 client.6237 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T10:41:25.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.818+0000 7f6e02ad0b80 10 client.6237 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.d 2023-10-06T10:41:25.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.818+0000 7f6e02ad0b80 20 client.6237 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:25.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.818+0000 7f6e02ad0b80 1 --2- 172.21.15.92:0/2060142063 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x55cf5b9ee8a0 0x55cf5b9eeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.818+0000 7f6e02ad0b80 4 client.6237 mds_command: new command op to 5036 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:41:25.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.818+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/2060142063 --> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf5b995610 con 0x55cf5b9ee8a0 2023-10-06T10:41:25.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.818+0000 7f6df2ffd700 1 --2- 172.21.15.92:0/2060142063 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x55cf5b9ee8a0 0x55cf5b9eeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.819+0000 7f6dd17fa700 10 client.6237 ms_handle_connect on v2:172.21.15.124:6833/2986058955 2023-10-06T10:41:25.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.819+0000 7f6df2ffd700 1 --2- 172.21.15.92:0/2060142063 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x55cf5b9ee8a0 0x55cf5b9eeca0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2de336cb08b86a0c server_cookie=b087867e61025e55 in_seq=0 out_seq=0 2023-10-06T10:41:25.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.819+0000 7f6dd17fa700 1 -- 172.21.15.92:0/2060142063 <== mds.2 v2:172.21.15.124:6833/2986058955 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55cf5b995610 con 0x55cf5b9ee8a0 2023-10-06T10:41:25.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.819+0000 7f6dd17fa700 10 client.6237 handle_command_reply: tid=0 2023-10-06T10:41:25.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.819+0000 7f6e02ad0b80 1 client.6237 shutdown 2023-10-06T10:41:25.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.819+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/2060142063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5ba0aae0 msgr2=0x55cf5b9ebbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.819+0000 7f6e02ad0b80 1 --2- 172.21.15.92:0/2060142063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5ba0aae0 0x55cf5b9ebbc0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6dec002dd0 tx=0x7f6dec009a40 comp rx=0 tx=0).stop 2023-10-06T10:41:25.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.820+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/2060142063 shutdown_connections 2023-10-06T10:41:25.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.820+0000 7f6e02ad0b80 1 --2- 172.21.15.92:0/2060142063 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x55cf5b9ee8a0 0x55cf5b9eeca0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.820+0000 7f6e02ad0b80 1 --2- 172.21.15.92:0/2060142063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5ba0aae0 0x55cf5b9ebbc0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.820+0000 7f6dd17fa700 0 client.6237 ms_handle_reset on v2:172.21.15.124:6833/2986058955 2023-10-06T10:41:25.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.820+0000 7f6e02ad0b80 1 --2- 172.21.15.92:0/2060142063 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55cf5b9c4c20 0x55cf5ba04370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.820+0000 7f6e02ad0b80 1 --2- 172.21.15.92:0/2060142063 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5b9c69f0 0x55cf5b9ee0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.820+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/2060142063 >> 172.21.15.92:0/2060142063 conn(0x55cf5b9c50c0 msgr2=0x55cf5b9c0840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:25.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.820+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/2060142063 shutdown_connections 2023-10-06T10:41:25.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.820+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/2060142063 wait complete. 2023-10-06T10:41:25.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.820+0000 7f6e02ad0b80 20 client.6237 trim_cache size 0 max 16384 2023-10-06T10:41:25.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.835+0000 7f6e02ad0b80 1 Processor -- start 2023-10-06T10:41:25.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.836+0000 7f6e02ad0b80 1 -- start start 2023-10-06T10:41:25.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.836+0000 7f6e02ad0b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5b9c50c0 0x55cf5b9c0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.836+0000 7f6e02ad0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5b9c4c20 0x55cf5b9eec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.836+0000 7f6e02ad0b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55cf5b9c69f0 0x55cf5b9ef1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.836+0000 7f6e02ad0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5b796e80 con 0x55cf5b9c4c20 2023-10-06T10:41:25.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.836+0000 7f6e02ad0b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5ba07710 con 0x55cf5b9c50c0 2023-10-06T10:41:25.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.836+0000 7f6e02ad0b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cf5b9be110 con 0x55cf5b9c69f0 2023-10-06T10:41:25.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.836+0000 7f6dea7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5b9c4c20 0x55cf5b9eec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.836+0000 7f6dea7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5b9c4c20 0x55cf5b9eec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33240/0 (socket says 172.21.15.92:33240) 2023-10-06T10:41:25.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.836+0000 7f6dea7fc700 1 -- 172.21.15.92:0/2973220448 learned_addr learned my addr 172.21.15.92:0/2973220448 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:25.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.837+0000 7f6dea7fc700 1 -- 172.21.15.92:0/2973220448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55cf5b9c69f0 msgr2=0x55cf5b9ef1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.837+0000 7f6df2ffd700 1 --2- 172.21.15.92:0/2973220448 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5b9c50c0 0x55cf5b9c0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.837+0000 7f6df37fe700 1 --2- 172.21.15.92:0/2973220448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55cf5b9c69f0 0x55cf5b9ef1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.837+0000 7f6dea7fc700 1 --2- 172.21.15.92:0/2973220448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55cf5b9c69f0 0x55cf5b9ef1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.837+0000 7f6dea7fc700 1 -- 172.21.15.92:0/2973220448 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5b9c50c0 msgr2=0x55cf5b9c0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.837+0000 7f6dea7fc700 1 --2- 172.21.15.92:0/2973220448 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5b9c50c0 0x55cf5b9c0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.837+0000 7f6dea7fc700 1 -- 172.21.15.92:0/2973220448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf5b8e27d0 con 0x55cf5b9c4c20 2023-10-06T10:41:25.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.837+0000 7f6df2ffd700 1 --2- 172.21.15.92:0/2973220448 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5b9c50c0 0x55cf5b9c0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:25.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.837+0000 7f6dea7fc700 1 --2- 172.21.15.92:0/2973220448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5b9c4c20 0x55cf5b9eec70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc050c90 tx=0x7f6ddc002000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e974091bdc5ce6c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:25.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.837+0000 7f6de8ff9700 1 -- 172.21.15.92:0/2973220448 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc070070 con 0x55cf5b9c4c20 2023-10-06T10:41:25.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.837+0000 7f6de8ff9700 1 -- 172.21.15.92:0/2973220448 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc05b730 con 0x55cf5b9c4c20 2023-10-06T10:41:25.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.837+0000 7f6de8ff9700 1 -- 172.21.15.92:0/2973220448 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc052b30 con 0x55cf5b9c4c20 2023-10-06T10:41:25.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.837+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/2973220448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5b9c4c20 msgr2=0x55cf5b9eec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.837+0000 7f6e02ad0b80 1 --2- 172.21.15.92:0/2973220448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5b9c4c20 0x55cf5b9eec70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc050c90 tx=0x7f6ddc002000 comp rx=0 tx=0).stop 2023-10-06T10:41:25.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.837+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/2973220448 shutdown_connections 2023-10-06T10:41:25.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.837+0000 7f6e02ad0b80 1 --2- 172.21.15.92:0/2973220448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55cf5b9c69f0 0x55cf5b9ef1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.837+0000 7f6e02ad0b80 1 --2- 172.21.15.92:0/2973220448 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5b9c50c0 0x55cf5b9c0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.837+0000 7f6e02ad0b80 1 --2- 172.21.15.92:0/2973220448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5b9c4c20 0x55cf5b9eec70 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.838+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/2973220448 >> 172.21.15.92:0/2973220448 conn(0x55cf5ba03d90 msgr2=0x55cf5ba09570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:25.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.838+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/2973220448 shutdown_connections 2023-10-06T10:41:25.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.838+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/2973220448 wait complete. 2023-10-06T10:41:25.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.838+0000 7f6e02ad0b80 1 -- start start 2023-10-06T10:41:25.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.838+0000 7f6e02ad0b80 1 Processor -- start 2023-10-06T10:41:25.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.839+0000 7f6e02ad0b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5b9c4c20 0x55cf5ba6fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.839+0000 7f6e02ad0b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55cf5b9c50c0 0x55cf5b9f2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.839+0000 7f6e02ad0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5b9c69f0 0x55cf5b9f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.839+0000 7f6e02ad0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5ba703a0 con 0x55cf5b9c69f0 2023-10-06T10:41:25.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.839+0000 7f6e02ad0b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5b9ea1a0 con 0x55cf5b9c4c20 2023-10-06T10:41:25.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.839+0000 7f6e02ad0b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cf5b9f29f0 con 0x55cf5b9c50c0 2023-10-06T10:41:25.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.839+0000 7f6e02ad0b80 10 client.0 _ll_register_callbacks cb 0x55cf5b7b0f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:25.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.839+0000 7f6df37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5b9c69f0 0x55cf5b9f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.839+0000 7f6df37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5b9c69f0 0x55cf5b9f33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33246/0 (socket says 172.21.15.92:33246) 2023-10-06T10:41:25.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.839+0000 7f6df37fe700 1 -- 172.21.15.92:0/502127210 learned_addr learned my addr 172.21.15.92:0/502127210 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:25.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.839+0000 7f6dea7fc700 1 --2- 172.21.15.92:0/502127210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55cf5b9c50c0 0x55cf5b9f2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.839+0000 7f6df37fe700 1 -- 172.21.15.92:0/502127210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55cf5b9c50c0 msgr2=0x55cf5b9f2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.839+0000 7f6df2ffd700 1 --2- 172.21.15.92:0/502127210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5b9c4c20 0x55cf5ba6fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.839+0000 7f6df37fe700 1 --2- 172.21.15.92:0/502127210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55cf5b9c50c0 0x55cf5b9f2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.839+0000 7f6df37fe700 1 -- 172.21.15.92:0/502127210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5b9c4c20 msgr2=0x55cf5ba6fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.839+0000 7f6df37fe700 1 --2- 172.21.15.92:0/502127210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5b9c4c20 0x55cf5ba6fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.839+0000 7f6df37fe700 1 -- 172.21.15.92:0/502127210 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cf5b7ae2c0 con 0x55cf5b9c69f0 2023-10-06T10:41:25.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.839+0000 7f6dea7fc700 1 --2- 172.21.15.92:0/502127210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55cf5b9c50c0 0x55cf5b9f2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:25.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.839+0000 7f6df2ffd700 1 --2- 172.21.15.92:0/502127210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5b9c4c20 0x55cf5ba6fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:25.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.840+0000 7f6df37fe700 1 --2- 172.21.15.92:0/502127210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5b9c69f0 0x55cf5b9f33c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6dec01e040 tx=0x7f6dec018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c93da250b6b3f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:25.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.840+0000 7f6dd17fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:25.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.840+0000 7f6dd17fa700 1 -- 172.21.15.92:0/502127210 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec0074e0 con 0x55cf5b9c69f0 2023-10-06T10:41:25.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.840+0000 7f6dd17fa700 1 -- 172.21.15.92:0/502127210 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6dec01b850 con 0x55cf5b9c69f0 2023-10-06T10:41:25.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.840+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/502127210 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ddc0579e0 con 0x55cf5b9c69f0 2023-10-06T10:41:25.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.840+0000 7f6dd17fa700 1 -- 172.21.15.92:0/502127210 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dec0099b0 con 0x55cf5b9c69f0 2023-10-06T10:41:25.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.840+0000 7f6e02ad0b80 10 client.6243 fetch_fsmap learned FSMap version 42 2023-10-06T10:41:25.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.840+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/502127210 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x55cf5b8c94c0 con 0x55cf5b9c69f0 2023-10-06T10:41:25.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.840+0000 7f6dd17fa700 1 -- 172.21.15.92:0/502127210 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc0579e0 con 0x55cf5b9c69f0 2023-10-06T10:41:25.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.840+0000 7f6dd17fa700 1 -- 172.21.15.92:0/502127210 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 42) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f6dec02e480 con 0x55cf5b9c69f0 2023-10-06T10:41:25.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.840+0000 7f6e02ad0b80 10 client.6243 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T10:41:25.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.840+0000 7f6e02ad0b80 10 client.6243 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.d 2023-10-06T10:41:25.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.840+0000 7f6e02ad0b80 20 client.6243 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:25.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.840+0000 7f6e02ad0b80 1 --2- 172.21.15.92:0/502127210 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x55cf5b9ee8a0 0x55cf5b9eec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.840+0000 7f6e02ad0b80 4 client.6243 mds_command: new command op to 5036 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:41:25.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.840+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/502127210 --> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cf5b8b6e50 con 0x55cf5b9ee8a0 2023-10-06T10:41:25.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.840+0000 7f6df2ffd700 1 --2- 172.21.15.92:0/502127210 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x55cf5b9ee8a0 0x55cf5b9eec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.841+0000 7f6dd17fa700 10 client.6243 ms_handle_connect on v2:172.21.15.124:6833/2986058955 2023-10-06T10:41:25.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.841+0000 7f6df2ffd700 1 --2- 172.21.15.92:0/502127210 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x55cf5b9ee8a0 0x55cf5b9eec70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ceac4d3fe95c098 server_cookie=4ded27a261d6f301 in_seq=0 out_seq=0 2023-10-06T10:41:25.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.841+0000 7f6dd17fa700 1 -- 172.21.15.92:0/502127210 <== mds.2 v2:172.21.15.124:6833/2986058955 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55cf5b8b6e50 con 0x55cf5b9ee8a0 2023-10-06T10:41:25.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.841+0000 7f6dd17fa700 10 client.6243 handle_command_reply: tid=0 2023-10-06T10:41:25.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.841+0000 7f6e02ad0b80 1 client.6243 shutdown 2023-10-06T10:41:25.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.841+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/502127210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5b9c69f0 msgr2=0x55cf5b9f33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.841+0000 7f6e02ad0b80 1 --2- 172.21.15.92:0/502127210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5b9c69f0 0x55cf5b9f33c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6dec01e040 tx=0x7f6dec018000 comp rx=0 tx=0).stop 2023-10-06T10:41:25.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.841+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/502127210 shutdown_connections 2023-10-06T10:41:25.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.841+0000 7f6e02ad0b80 1 --2- 172.21.15.92:0/502127210 >> [v2:172.21.15.124:6833/2986058955,v1:172.21.15.124:6835/2986058955] conn(0x55cf5b9ee8a0 0x55cf5b9eec70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.841+0000 7f6e02ad0b80 1 --2- 172.21.15.92:0/502127210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf5b9c69f0 0x55cf5b9f33c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.841+0000 7f6e02ad0b80 1 --2- 172.21.15.92:0/502127210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55cf5b9c4c20 0x55cf5ba6fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.841+0000 7f6e02ad0b80 1 --2- 172.21.15.92:0/502127210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55cf5b9c50c0 0x55cf5b9f2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.841+0000 7f6dd17fa700 0 client.6243 ms_handle_reset on v2:172.21.15.124:6833/2986058955 2023-10-06T10:41:25.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.841+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/502127210 >> 172.21.15.92:0/502127210 conn(0x55cf5ba03d90 msgr2=0x55cf5ba09570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:25.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.841+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/502127210 shutdown_connections 2023-10-06T10:41:25.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.841+0000 7f6e02ad0b80 1 -- 172.21.15.92:0/502127210 wait complete. 2023-10-06T10:41:25.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.841+0000 7f6e02ad0b80 20 client.6243 trim_cache size 0 max 16384 2023-10-06T10:41:25.851 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:41:25.851 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "no active scrubs running", 2023-10-06T10:41:25.851 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": {} 2023-10-06T10:41:25.852 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:41:25.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.842+0000 7f6df97d8700 1 -- 172.21.15.92:0/1176036853 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6dd4066f30 msgr2=0x7f6dd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.842+0000 7f6df97d8700 1 --2- 172.21.15.92:0/1176036853 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6dd4066f30 0x7f6dd40693e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6df40ccd70 tx=0x7f6de4017000 comp rx=0 tx=0).stop 2023-10-06T10:41:25.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.842+0000 7f6df97d8700 1 -- 172.21.15.92:0/1176036853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6df4157450 msgr2=0x7f6df40c94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.842+0000 7f6df97d8700 1 --2- 172.21.15.92:0/1176036853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6df4157450 0x7f6df40c94a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00c510 tx=0x7f6ddc00c8d0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.842+0000 7f6df97d8700 1 -- 172.21.15.92:0/1176036853 shutdown_connections 2023-10-06T10:41:25.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.842+0000 7f6df97d8700 1 --2- 172.21.15.92:0/1176036853 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6dd4066f30 0x7f6dd40693e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.842+0000 7f6df97d8700 1 --2- 172.21.15.92:0/1176036853 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6df4159e20 0x7f6df4169cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.842+0000 7f6df97d8700 1 --2- 172.21.15.92:0/1176036853 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6df40d0310 0x7f6df4160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.842+0000 7f6df97d8700 1 --2- 172.21.15.92:0/1176036853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6df4157450 0x7f6df40c94a0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.842+0000 7f6df97d8700 1 -- 172.21.15.92:0/1176036853 >> 172.21.15.92:0/1176036853 conn(0x7f6df406ece0 msgr2=0x7f6df40b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:25.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.842+0000 7f6df97d8700 1 -- 172.21.15.92:0/1176036853 shutdown_connections 2023-10-06T10:41:25.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.842+0000 7f6df97d8700 1 -- 172.21.15.92:0/1176036853 wait complete. 2023-10-06T10:41:25.855 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:41:25.855 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-10-06T10:41:25.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.968+0000 7f7354010700 1 Processor -- start 2023-10-06T10:41:25.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.969+0000 7f7354010700 1 -- start start 2023-10-06T10:41:25.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.969+0000 7f7354010700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f734c06da40 0x7f734c0b1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.969+0000 7f7354010700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734c15ab50 con 0x7f734c06ce10 2023-10-06T10:41:25.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.969+0000 7f7354010700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734c15d650 con 0x7f734c06d670 2023-10-06T10:41:25.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.969+0000 7f7354010700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734c15a590 con 0x7f734c06da40 2023-10-06T10:41:25.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.969+0000 7f73515ab700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f734c06ce10 0x7f734c159af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51066/0 (socket says 172.21.15.92:51066) 2023-10-06T10:41:25.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f73515ab700 1 -- 172.21.15.92:0/2086568332 learned_addr learned my addr 172.21.15.92:0/2086568332 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:25.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f73525ad700 1 --2- 172.21.15.92:0/2086568332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f734c06da40 0x7f734c0b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7350daa700 1 -- 172.21.15.92:0/2086568332 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3928934393 0 0) 0x7f733c001110 con 0x7f734c06ce10 2023-10-06T10:41:25.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7350daa700 1 -- 172.21.15.92:0/2086568332 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7338003570 con 0x7f734c06ce10 2023-10-06T10:41:25.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7350daa700 1 -- 172.21.15.92:0/2086568332 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573697184 0 0) 0x7f734c15d650 con 0x7f734c06d670 2023-10-06T10:41:25.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7350daa700 1 -- 172.21.15.92:0/2086568332 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f733c001110 con 0x7f734c06d670 2023-10-06T10:41:25.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7350daa700 1 -- 172.21.15.92:0/2086568332 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3450667166 0 0) 0x7f7338003570 con 0x7f734c06ce10 2023-10-06T10:41:25.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7350daa700 1 -- 172.21.15.92:0/2086568332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f734c06da40 msgr2=0x7f734c0b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7350daa700 1 --2- 172.21.15.92:0/2086568332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f734c06da40 0x7f734c0b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7350daa700 1 -- 172.21.15.92:0/2086568332 >> v1:172.21.15.124:6789/0 conn(0x7f734c06d670 legacy=0x7f734c15dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7350daa700 1 -- 172.21.15.92:0/2086568332 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734c15a870 con 0x7f734c06ce10 2023-10-06T10:41:25.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7350daa700 1 -- 172.21.15.92:0/2086568332 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f733c001600 con 0x7f734c06ce10 2023-10-06T10:41:25.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7350daa700 1 -- 172.21.15.92:0/2086568332 >> v1:172.21.15.92:6789/0 conn(0x7f734c06ce10 legacy=0x7f734c159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7350daa700 1 --2- 172.21.15.92:0/2086568332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7338003c20 0x7f7338004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7350daa700 1 --2- 172.21.15.92:0/2086568332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7338004810 0x7f7338004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7350daa700 1 --2- 172.21.15.92:0/2086568332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f734c06da40 0x7f73380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7350daa700 1 -- 172.21.15.92:0/2086568332 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c15a590 con 0x7f7338003c20 2023-10-06T10:41:25.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7350daa700 1 -- 172.21.15.92:0/2086568332 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733800bb40 con 0x7f7338004810 2023-10-06T10:41:25.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7350daa700 1 -- 172.21.15.92:0/2086568332 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f733800bd00 con 0x7f734c06da40 2023-10-06T10:41:25.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7351dac700 1 --2- 172.21.15.92:0/2086568332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7338003c20 0x7f7338004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f73515ab700 1 --2- 172.21.15.92:0/2086568332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7338004810 0x7f7338004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f73525ad700 1 --2- 172.21.15.92:0/2086568332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f734c06da40 0x7f73380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7351dac700 1 -- 172.21.15.92:0/2086568332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f734c06da40 msgr2=0x7f73380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7351dac700 1 --2- 172.21.15.92:0/2086568332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f734c06da40 0x7f73380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7351dac700 1 -- 172.21.15.92:0/2086568332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7338004810 msgr2=0x7f7338004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7351dac700 1 --2- 172.21.15.92:0/2086568332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7338004810 0x7f7338004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.970+0000 7f7351dac700 1 -- 172.21.15.92:0/2086568332 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7338003570 con 0x7f7338003c20 2023-10-06T10:41:25.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.971+0000 7f7351dac700 1 --2- 172.21.15.92:0/2086568332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7338003c20 0x7f7338004010 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f734800c3c0 tx=0x7f7348001bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0cd8cb06a6c1121 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:25.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.971+0000 7f7350daa700 1 -- 172.21.15.92:0/2086568332 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7348001ee0 con 0x7f7338003c20 2023-10-06T10:41:25.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.971+0000 7f7350daa700 1 -- 172.21.15.92:0/2086568332 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734800ccb0 con 0x7f7338003c20 2023-10-06T10:41:25.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.971+0000 7f7350daa700 1 -- 172.21.15.92:0/2086568332 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7348012650 con 0x7f7338003c20 2023-10-06T10:41:25.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.971+0000 7f7354010700 1 -- 172.21.15.92:0/2086568332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7338003c20 msgr2=0x7f7338004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.971+0000 7f7354010700 1 --2- 172.21.15.92:0/2086568332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7338003c20 0x7f7338004010 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f734800c3c0 tx=0x7f7348001bd0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.971+0000 7f73525ad700 1 -- 172.21.15.92:0/2086568332 reap_dead start 2023-10-06T10:41:25.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.971+0000 7f7354010700 1 -- 172.21.15.92:0/2086568332 shutdown_connections 2023-10-06T10:41:25.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.971+0000 7f7354010700 1 -- 172.21.15.92:0/2086568332 >> 172.21.15.92:0/2086568332 conn(0x7f734c06b600 msgr2=0x7f734c06ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:25.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.971+0000 7f7354010700 1 -- 172.21.15.92:0/2086568332 shutdown_connections 2023-10-06T10:41:25.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.971+0000 7f7354010700 1 -- 172.21.15.92:0/2086568332 wait complete. 2023-10-06T10:41:25.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.972+0000 7f7354010700 1 Processor -- start 2023-10-06T10:41:25.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.972+0000 7f7354010700 1 -- start start 2023-10-06T10:41:25.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.973+0000 7f7354010700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7338003c20 0x7f734c156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.973+0000 7f7354010700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f734c1699a0 0x7f734c150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.973+0000 7f7354010700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f734c169e10 0x7f734c150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.973+0000 7f7354010700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c156090 con 0x7f734c169e10 2023-10-06T10:41:25.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.973+0000 7f7354010700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c0aee80 con 0x7f7338003c20 2023-10-06T10:41:25.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.973+0000 7f7354010700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734c0af120 con 0x7f734c1699a0 2023-10-06T10:41:25.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.973+0000 7f73525ad700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f734c169e10 0x7f734c150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.973+0000 7f73525ad700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f734c169e10 0x7f734c150650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33266/0 (socket says 172.21.15.92:33266) 2023-10-06T10:41:25.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.973+0000 7f73525ad700 1 -- 172.21.15.92:0/2770578691 learned_addr learned my addr 172.21.15.92:0/2770578691 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:25.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.973+0000 7f7351dac700 1 --2- 172.21.15.92:0/2770578691 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7338003c20 0x7f734c156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.973+0000 7f73515ab700 1 --2- 172.21.15.92:0/2770578691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f734c1699a0 0x7f734c150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.973+0000 7f73525ad700 1 -- 172.21.15.92:0/2770578691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f734c1699a0 msgr2=0x7f734c150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.974+0000 7f73525ad700 1 --2- 172.21.15.92:0/2770578691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f734c1699a0 0x7f734c150110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.974+0000 7f73525ad700 1 -- 172.21.15.92:0/2770578691 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7338003c20 msgr2=0x7f734c156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.974+0000 7f73525ad700 1 --2- 172.21.15.92:0/2770578691 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7338003c20 0x7f734c156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.974+0000 7f73525ad700 1 -- 172.21.15.92:0/2770578691 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734c0af400 con 0x7f734c169e10 2023-10-06T10:41:25.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.974+0000 7f73525ad700 1 --2- 172.21.15.92:0/2770578691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f734c169e10 0x7f734c150650 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f734c06ec90 tx=0x7f7344008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:25.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.974+0000 7f7341ffb700 1 -- 172.21.15.92:0/2770578691 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7344011ed0 con 0x7f734c169e10 2023-10-06T10:41:25.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.974+0000 7f7341ffb700 1 -- 172.21.15.92:0/2770578691 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7344015670 con 0x7f734c169e10 2023-10-06T10:41:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.974+0000 7f7354010700 1 -- 172.21.15.92:0/2770578691 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f734c0af6e0 con 0x7f734c169e10 2023-10-06T10:41:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.974+0000 7f7341ffb700 1 -- 172.21.15.92:0/2770578691 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7344015a30 con 0x7f734c169e10 2023-10-06T10:41:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.974+0000 7f7354010700 1 -- 172.21.15.92:0/2770578691 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734c157370 con 0x7f734c169e10 2023-10-06T10:41:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.975+0000 7f7341ffb700 1 -- 172.21.15.92:0/2770578691 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7344020460 con 0x7f734c169e10 2023-10-06T10:41:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.976+0000 7f7341ffb700 1 --2- 172.21.15.92:0/2770578691 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7330066f30 0x7f73300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.976+0000 7f7341ffb700 1 -- 172.21.15.92:0/2770578691 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7344084800 con 0x7f734c169e10 2023-10-06T10:41:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.976+0000 7f7351dac700 1 --2- 172.21.15.92:0/2770578691 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7330066f30 0x7f73300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.976+0000 7f7351dac700 1 --2- 172.21.15.92:0/2770578691 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7330066f30 0x7f73300693e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7348000f80 tx=0x7f7348016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.978+0000 7f735d308b80 1 Processor -- start 2023-10-06T10:41:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.978+0000 7f735d308b80 1 -- start start 2023-10-06T10:41:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.978+0000 7f735d308b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401334fae0 0x56401334feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.978+0000 7f735d308b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401330b9f0 0x5640133085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.978+0000 7f735d308b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564013309c20 0x564013308b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.978+0000 7f735d308b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564013288cd0 con 0x564013309c20 2023-10-06T10:41:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.978+0000 7f735d308b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56401325fac0 con 0x56401334fae0 2023-10-06T10:41:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.978+0000 7f735d308b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56401325d2c0 con 0x56401330b9f0 2023-10-06T10:41:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.978+0000 7f73525ad700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564013309c20 0x564013308b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.978+0000 7f73525ad700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564013309c20 0x564013308b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33274/0 (socket says 172.21.15.92:33274) 2023-10-06T10:41:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.979+0000 7f73525ad700 1 -- 172.21.15.92:0/2738009853 learned_addr learned my addr 172.21.15.92:0/2738009853 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.979+0000 7f7351dac700 1 --2- 172.21.15.92:0/2738009853 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401334fae0 0x56401334feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.979+0000 7f73515ab700 1 --2- 172.21.15.92:0/2738009853 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401330b9f0 0x5640133085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.979+0000 7f73525ad700 1 -- 172.21.15.92:0/2738009853 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401330b9f0 msgr2=0x5640133085d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.979+0000 7f73525ad700 1 --2- 172.21.15.92:0/2738009853 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401330b9f0 0x5640133085d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.979+0000 7f73525ad700 1 -- 172.21.15.92:0/2738009853 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401334fae0 msgr2=0x56401334feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.979+0000 7f73525ad700 1 --2- 172.21.15.92:0/2738009853 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401334fae0 0x56401334feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.979+0000 7f73525ad700 1 -- 172.21.15.92:0/2738009853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640130f32c0 con 0x564013309c20 2023-10-06T10:41:25.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.979+0000 7f7351dac700 1 --2- 172.21.15.92:0/2738009853 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401334fae0 0x56401334feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:25.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.979+0000 7f73525ad700 1 --2- 172.21.15.92:0/2738009853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564013309c20 0x564013308b10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7344009fd0 tx=0x7f73440829e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f244469f77e3d33f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:25.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.979+0000 7f7336ffd700 1 -- 172.21.15.92:0/2738009853 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7344082ea0 con 0x564013309c20 2023-10-06T10:41:25.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.979+0000 7f7336ffd700 1 -- 172.21.15.92:0/2738009853 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7344055260 con 0x564013309c20 2023-10-06T10:41:25.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.979+0000 7f7336ffd700 1 -- 172.21.15.92:0/2738009853 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73440586e0 con 0x564013309c20 2023-10-06T10:41:25.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.980+0000 7f735d308b80 1 -- 172.21.15.92:0/2738009853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564013309c20 msgr2=0x564013308b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.980+0000 7f735d308b80 1 --2- 172.21.15.92:0/2738009853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564013309c20 0x564013308b10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7344009fd0 tx=0x7f73440829e0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.980+0000 7f735d308b80 1 -- 172.21.15.92:0/2738009853 shutdown_connections 2023-10-06T10:41:25.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.980+0000 7f735d308b80 1 --2- 172.21.15.92:0/2738009853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564013309c20 0x564013308b10 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.980+0000 7f735d308b80 1 --2- 172.21.15.92:0/2738009853 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401334fae0 0x56401334feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.980+0000 7f735d308b80 1 --2- 172.21.15.92:0/2738009853 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401330b9f0 0x5640133085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.980+0000 7f735d308b80 1 -- 172.21.15.92:0/2738009853 >> 172.21.15.92:0/2738009853 conn(0x56401330a0c0 msgr2=0x56401330a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:25.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.980+0000 7f735d308b80 1 -- 172.21.15.92:0/2738009853 shutdown_connections 2023-10-06T10:41:25.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.980+0000 7f735d308b80 1 -- 172.21.15.92:0/2738009853 wait complete. 2023-10-06T10:41:25.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.980+0000 7f735d308b80 1 -- start start 2023-10-06T10:41:25.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.980+0000 7f735d308b80 1 Processor -- start 2023-10-06T10:41:25.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.981+0000 7f735d308b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564013309c20 0x564013333080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.981+0000 7f735d308b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401330b9f0 0x5640133335c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.981+0000 7f735d308b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401334fae0 0x5640133085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.981+0000 7f735d308b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640132590b0 con 0x56401330b9f0 2023-10-06T10:41:25.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.981+0000 7f735d308b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640132048e0 con 0x564013309c20 2023-10-06T10:41:25.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.981+0000 7f735d308b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56401321fef0 con 0x56401334fae0 2023-10-06T10:41:25.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.981+0000 7f73515ab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401330b9f0 0x5640133335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.981+0000 7f735d308b80 10 client.0 _ll_register_callbacks cb 0x5640130f5f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:25.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.981+0000 7f73515ab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401330b9f0 0x5640133335c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33276/0 (socket says 172.21.15.92:33276) 2023-10-06T10:41:25.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.981+0000 7f73515ab700 1 -- 172.21.15.92:0/1750544345 learned_addr learned my addr 172.21.15.92:0/1750544345 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:25.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.981+0000 7f73525ad700 1 --2- 172.21.15.92:0/1750544345 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401334fae0 0x5640133085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.981+0000 7f7351dac700 1 --2- 172.21.15.92:0/1750544345 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564013309c20 0x564013333080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f73515ab700 1 -- 172.21.15.92:0/1750544345 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401334fae0 msgr2=0x5640133085d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f73515ab700 1 --2- 172.21.15.92:0/1750544345 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401334fae0 0x5640133085d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f73515ab700 1 -- 172.21.15.92:0/1750544345 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564013309c20 msgr2=0x564013333080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:25.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f73515ab700 1 --2- 172.21.15.92:0/1750544345 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564013309c20 0x564013333080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f73515ab700 1 -- 172.21.15.92:0/1750544345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734c157370 con 0x56401330b9f0 2023-10-06T10:41:25.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f7351dac700 1 --2- 172.21.15.92:0/1750544345 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564013309c20 0x564013333080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:25.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f73515ab700 1 --2- 172.21.15.92:0/1750544345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401330b9f0 0x5640133335c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f733c004fd0 tx=0x7f733c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8841aaf6ff3714 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:25.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f732bfff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:25.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f732bfff700 1 -- 172.21.15.92:0/1750544345 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c016700 con 0x56401330b9f0 2023-10-06T10:41:25.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f732bfff700 1 -- 172.21.15.92:0/1750544345 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f733c00fb20 con 0x56401330b9f0 2023-10-06T10:41:25.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f735d308b80 1 -- 172.21.15.92:0/1750544345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640132fd6e0 con 0x56401330b9f0 2023-10-06T10:41:25.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f732bfff700 1 -- 172.21.15.92:0/1750544345 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f733c016860 con 0x56401330b9f0 2023-10-06T10:41:25.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f735d308b80 10 client.6255 fetch_fsmap learned FSMap version 42 2023-10-06T10:41:25.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f735d308b80 1 -- 172.21.15.92:0/1750544345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x564013096030 con 0x56401330b9f0 2023-10-06T10:41:25.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f732bfff700 1 -- 172.21.15.92:0/1750544345 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5640132fd6e0 con 0x56401330b9f0 2023-10-06T10:41:25.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f732bfff700 1 -- 172.21.15.92:0/1750544345 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 42) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f733c012070 con 0x56401330b9f0 2023-10-06T10:41:25.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f735d308b80 10 client.6255 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T10:41:25.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f735d308b80 10 client.6255 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-10-06T10:41:25.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f735d308b80 20 client.6255 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:25.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f735d308b80 1 --2- 172.21.15.92:0/1750544345 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x56401332f730 0x56401334ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:25.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f735d308b80 4 client.6255 mds_command: new command op to 5736 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:41:25.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f735d308b80 1 -- 172.21.15.92:0/1750544345 --> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640132da610 con 0x56401332f730 2023-10-06T10:41:25.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.982+0000 7f7351dac700 1 --2- 172.21.15.92:0/1750544345 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x56401332f730 0x56401334ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:25.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.983+0000 7f732bfff700 10 client.6255 ms_handle_connect on v2:172.21.15.92:6835/2458340622 2023-10-06T10:41:25.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.983+0000 7f7351dac700 1 --2- 172.21.15.92:0/1750544345 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x56401332f730 0x56401334ae80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cdd595cbcb6db8e server_cookie=cbac8babab5c837b in_seq=0 out_seq=0 2023-10-06T10:41:25.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.983+0000 7f732bfff700 1 -- 172.21.15.92:0/1750544345 <== mds.0 v2:172.21.15.92:6835/2458340622 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5640132da610 con 0x56401332f730 2023-10-06T10:41:25.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.983+0000 7f732bfff700 10 client.6255 handle_command_reply: tid=0 2023-10-06T10:41:25.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.983+0000 7f735d308b80 1 client.6255 shutdown 2023-10-06T10:41:25.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.983+0000 7f735d308b80 1 -- 172.21.15.92:0/1750544345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401330b9f0 msgr2=0x5640133335c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:25.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.983+0000 7f735d308b80 1 --2- 172.21.15.92:0/1750544345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401330b9f0 0x5640133335c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f733c004fd0 tx=0x7f733c009040 comp rx=0 tx=0).stop 2023-10-06T10:41:25.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.983+0000 7f735d308b80 1 -- 172.21.15.92:0/1750544345 shutdown_connections 2023-10-06T10:41:25.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.983+0000 7f735d308b80 1 --2- 172.21.15.92:0/1750544345 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x56401332f730 0x56401334ae80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.983+0000 7f735d308b80 1 --2- 172.21.15.92:0/1750544345 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401334fae0 0x5640133085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.983+0000 7f735d308b80 1 --2- 172.21.15.92:0/1750544345 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564013309c20 0x564013333080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.983+0000 7f732bfff700 0 client.6255 ms_handle_reset on v2:172.21.15.92:6835/2458340622 2023-10-06T10:41:25.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.983+0000 7f735d308b80 1 --2- 172.21.15.92:0/1750544345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401330b9f0 0x5640133335c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:25.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.983+0000 7f735d308b80 1 -- 172.21.15.92:0/1750544345 >> 172.21.15.92:0/1750544345 conn(0x56401330a0c0 msgr2=0x5640133057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:25.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.983+0000 7f735d308b80 1 -- 172.21.15.92:0/1750544345 shutdown_connections 2023-10-06T10:41:25.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.984+0000 7f735d308b80 1 -- 172.21.15.92:0/1750544345 wait complete. 2023-10-06T10:41:25.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.984+0000 7f735d308b80 20 client.6255 trim_cache size 0 max 16384 2023-10-06T10:41:26.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.999+0000 7f735d308b80 1 Processor -- start 2023-10-06T10:41:26.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:25.999+0000 7f735d308b80 1 -- start start 2023-10-06T10:41:26.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.000+0000 7f735d308b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564013309c20 0x564013304450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:26.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.000+0000 7f735d308b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401330b9f0 0x564013305670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:26.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.000+0000 7f735d308b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401334fae0 0x56401332e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:26.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.000+0000 7f735d308b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640130dbe80 con 0x56401330b9f0 2023-10-06T10:41:26.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.000+0000 7f735d308b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56401334acb0 con 0x564013309c20 2023-10-06T10:41:26.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.000+0000 7f735d308b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x564013304220 con 0x56401334fae0 2023-10-06T10:41:26.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.000+0000 7f73515ab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401330b9f0 0x564013305670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:26.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.000+0000 7f73515ab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401330b9f0 0x564013305670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33292/0 (socket says 172.21.15.92:33292) 2023-10-06T10:41:26.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.000+0000 7f73515ab700 1 -- 172.21.15.92:0/2763352904 learned_addr learned my addr 172.21.15.92:0/2763352904 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:26.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.000+0000 7f7351dac700 1 --2- 172.21.15.92:0/2763352904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564013309c20 0x564013304450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.000+0000 7f73525ad700 1 --2- 172.21.15.92:0/2763352904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401334fae0 0x56401332e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.000+0000 7f73515ab700 1 -- 172.21.15.92:0/2763352904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401334fae0 msgr2=0x56401332e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.000+0000 7f73515ab700 1 --2- 172.21.15.92:0/2763352904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401334fae0 0x56401332e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.000+0000 7f73515ab700 1 -- 172.21.15.92:0/2763352904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564013309c20 msgr2=0x564013304450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.000+0000 7f73515ab700 1 --2- 172.21.15.92:0/2763352904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564013309c20 0x564013304450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.000+0000 7f73515ab700 1 -- 172.21.15.92:0/2763352904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640132277d0 con 0x56401330b9f0 2023-10-06T10:41:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.001+0000 7f73515ab700 1 --2- 172.21.15.92:0/2763352904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401330b9f0 0x564013305670 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f733c000ea0 tx=0x7f733c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ddd5e900996cd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.001+0000 7f73377fe700 1 -- 172.21.15.92:0/2763352904 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c02e030 con 0x56401330b9f0 2023-10-06T10:41:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.001+0000 7f73377fe700 1 -- 172.21.15.92:0/2763352904 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733c007730 con 0x56401330b9f0 2023-10-06T10:41:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.001+0000 7f73377fe700 1 -- 172.21.15.92:0/2763352904 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c016700 con 0x56401330b9f0 2023-10-06T10:41:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.001+0000 7f735d308b80 1 -- 172.21.15.92:0/2763352904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401330b9f0 msgr2=0x564013305670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.001+0000 7f735d308b80 1 --2- 172.21.15.92:0/2763352904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401330b9f0 0x564013305670 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f733c000ea0 tx=0x7f733c019000 comp rx=0 tx=0).stop 2023-10-06T10:41:26.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.001+0000 7f735d308b80 1 -- 172.21.15.92:0/2763352904 shutdown_connections 2023-10-06T10:41:26.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.001+0000 7f735d308b80 1 --2- 172.21.15.92:0/2763352904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401334fae0 0x56401332e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.001+0000 7f735d308b80 1 --2- 172.21.15.92:0/2763352904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564013309c20 0x564013304450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.001+0000 7f735d308b80 1 --2- 172.21.15.92:0/2763352904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401330b9f0 0x564013305670 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.001+0000 7f735d308b80 1 -- 172.21.15.92:0/2763352904 >> 172.21.15.92:0/2763352904 conn(0x56401330a0c0 msgr2=0x564013304e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:26.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.001+0000 7f735d308b80 1 -- 172.21.15.92:0/2763352904 shutdown_connections 2023-10-06T10:41:26.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.001+0000 7f735d308b80 1 -- 172.21.15.92:0/2763352904 wait complete. 2023-10-06T10:41:26.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.002+0000 7f735d308b80 1 -- start start 2023-10-06T10:41:26.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.002+0000 7f735d308b80 1 Processor -- start 2023-10-06T10:41:26.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f735d308b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564013309c20 0x564013305860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:26.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f735d308b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401330b9f0 0x56401330a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:26.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f735d308b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401334fae0 0x56401330a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:26.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f735d308b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564013346600 con 0x564013309c20 2023-10-06T10:41:26.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f735d308b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x564013354f70 con 0x56401330b9f0 2023-10-06T10:41:26.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f735d308b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x564013345d80 con 0x56401334fae0 2023-10-06T10:41:26.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f735d308b80 10 client.0 _ll_register_callbacks cb 0x5640130f5f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:26.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f7351dac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564013309c20 0x564013305860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:26.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f7351dac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564013309c20 0x564013305860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33294/0 (socket says 172.21.15.92:33294) 2023-10-06T10:41:26.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f7351dac700 1 -- 172.21.15.92:0/2359407557 learned_addr learned my addr 172.21.15.92:0/2359407557 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:26.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f73515ab700 1 --2- 172.21.15.92:0/2359407557 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401330b9f0 0x56401330a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:26.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f73525ad700 1 --2- 172.21.15.92:0/2359407557 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401334fae0 0x56401330a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:26.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f7351dac700 1 -- 172.21.15.92:0/2359407557 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401334fae0 msgr2=0x56401330a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:26.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f7351dac700 1 --2- 172.21.15.92:0/2359407557 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401334fae0 0x56401330a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f7351dac700 1 -- 172.21.15.92:0/2359407557 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401330b9f0 msgr2=0x56401330a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:26.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f7351dac700 1 --2- 172.21.15.92:0/2359407557 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401330b9f0 0x56401330a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f7351dac700 1 -- 172.21.15.92:0/2359407557 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564013096030 con 0x564013309c20 2023-10-06T10:41:26.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f73515ab700 1 --2- 172.21.15.92:0/2359407557 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401330b9f0 0x56401330a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:26.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f7351dac700 1 --2- 172.21.15.92:0/2359407557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564013309c20 0x564013305860 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f734800d010 tx=0x7f7348022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753a3b3daf279a6a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:26.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f732bfff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:26.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f732bfff700 1 -- 172.21.15.92:0/2359407557 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7348032040 con 0x564013309c20 2023-10-06T10:41:26.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f732bfff700 1 -- 172.21.15.92:0/2359407557 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f734801e670 con 0x564013309c20 2023-10-06T10:41:26.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f735d308b80 1 -- 172.21.15.92:0/2359407557 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7344055260 con 0x564013309c20 2023-10-06T10:41:26.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f732bfff700 1 -- 172.21.15.92:0/2359407557 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7348036250 con 0x564013309c20 2023-10-06T10:41:26.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f735d308b80 10 client.6261 fetch_fsmap learned FSMap version 42 2023-10-06T10:41:26.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.003+0000 7f735d308b80 1 -- 172.21.15.92:0/2359407557 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x56401320e4c0 con 0x564013309c20 2023-10-06T10:41:26.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f732bfff700 1 -- 172.21.15.92:0/2359407557 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7344055260 con 0x564013309c20 2023-10-06T10:41:26.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f732bfff700 1 -- 172.21.15.92:0/2359407557 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 42) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f734801ea40 con 0x564013309c20 2023-10-06T10:41:26.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f735d308b80 10 client.6261 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T10:41:26.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f735d308b80 10 client.6261 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-10-06T10:41:26.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f735d308b80 20 client.6261 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:26.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f735d308b80 1 --2- 172.21.15.92:0/2359407557 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x564013345390 0x564013337ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:26.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f735d308b80 4 client.6261 mds_command: new command op to 5736 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:41:26.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f735d308b80 1 -- 172.21.15.92:0/2359407557 --> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5640131fbe50 con 0x564013345390 2023-10-06T10:41:26.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f73515ab700 1 --2- 172.21.15.92:0/2359407557 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x564013345390 0x564013337ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:26.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f732bfff700 10 client.6261 ms_handle_connect on v2:172.21.15.92:6835/2458340622 2023-10-06T10:41:26.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f73515ab700 1 --2- 172.21.15.92:0/2359407557 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x564013345390 0x564013337ad0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=212bfce399202996 server_cookie=90c5615fd9e4a5cb in_seq=0 out_seq=0 2023-10-06T10:41:26.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f732bfff700 1 -- 172.21.15.92:0/2359407557 <== mds.0 v2:172.21.15.92:6835/2458340622 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5640131fbe50 con 0x564013345390 2023-10-06T10:41:26.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f732bfff700 10 client.6261 handle_command_reply: tid=0 2023-10-06T10:41:26.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f735d308b80 1 client.6261 shutdown 2023-10-06T10:41:26.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f735d308b80 1 -- 172.21.15.92:0/2359407557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564013309c20 msgr2=0x564013305860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:26.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f735d308b80 1 --2- 172.21.15.92:0/2359407557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564013309c20 0x564013305860 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f734800d010 tx=0x7f7348022040 comp rx=0 tx=0).stop 2023-10-06T10:41:26.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f735d308b80 1 -- 172.21.15.92:0/2359407557 shutdown_connections 2023-10-06T10:41:26.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f735d308b80 1 --2- 172.21.15.92:0/2359407557 >> [v2:172.21.15.92:6835/2458340622,v1:172.21.15.92:6837/2458340622] conn(0x564013345390 0x564013337ad0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f735d308b80 1 --2- 172.21.15.92:0/2359407557 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401334fae0 0x56401330a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f732bfff700 0 client.6261 ms_handle_reset on v2:172.21.15.92:6835/2458340622 2023-10-06T10:41:26.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f735d308b80 1 --2- 172.21.15.92:0/2359407557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564013309c20 0x564013305860 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f735d308b80 1 --2- 172.21.15.92:0/2359407557 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401330b9f0 0x56401330a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.004+0000 7f735d308b80 1 -- 172.21.15.92:0/2359407557 >> 172.21.15.92:0/2359407557 conn(0x56401330a0c0 msgr2=0x5640133348f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:26.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.005+0000 7f735d308b80 1 -- 172.21.15.92:0/2359407557 shutdown_connections 2023-10-06T10:41:26.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.005+0000 7f735d308b80 1 -- 172.21.15.92:0/2359407557 wait complete. 2023-10-06T10:41:26.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.005+0000 7f735d308b80 20 client.6261 trim_cache size 0 max 16384 2023-10-06T10:41:26.015 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:41:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "no active scrubs running", 2023-10-06T10:41:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": {} 2023-10-06T10:41:26.015 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:41:26.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.005+0000 7f7354010700 1 -- 172.21.15.92:0/2770578691 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7330066f30 msgr2=0x7f73300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:26.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.005+0000 7f7354010700 1 --2- 172.21.15.92:0/2770578691 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7330066f30 0x7f73300693e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7348000f80 tx=0x7f7348016000 comp rx=0 tx=0).stop 2023-10-06T10:41:26.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.005+0000 7f7354010700 1 -- 172.21.15.92:0/2770578691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f734c169e10 msgr2=0x7f734c150650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:26.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.005+0000 7f7354010700 1 --2- 172.21.15.92:0/2770578691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f734c169e10 0x7f734c150650 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f734c06ec90 tx=0x7f7344008040 comp rx=0 tx=0).stop 2023-10-06T10:41:26.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.005+0000 7f7354010700 1 -- 172.21.15.92:0/2770578691 shutdown_connections 2023-10-06T10:41:26.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.005+0000 7f7354010700 1 --2- 172.21.15.92:0/2770578691 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7330066f30 0x7f73300693e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.005+0000 7f7354010700 1 --2- 172.21.15.92:0/2770578691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f734c169e10 0x7f734c150650 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.005+0000 7f7354010700 1 --2- 172.21.15.92:0/2770578691 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7338003c20 0x7f734c156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.005+0000 7f7354010700 1 --2- 172.21.15.92:0/2770578691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f734c1699a0 0x7f734c150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.005+0000 7f7354010700 1 -- 172.21.15.92:0/2770578691 >> 172.21.15.92:0/2770578691 conn(0x7f734c06b600 msgr2=0x7f734c157fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:26.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.005+0000 7f7354010700 1 -- 172.21.15.92:0/2770578691 shutdown_connections 2023-10-06T10:41:26.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.005+0000 7f7354010700 1 -- 172.21.15.92:0/2770578691 wait complete. 2023-10-06T10:41:26.018 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:41:26.019 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:41:26.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.132+0000 7f6823b09700 1 Processor -- start 2023-10-06T10:41:26.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.132+0000 7f6823b09700 1 -- start start 2023-10-06T10:41:26.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.133+0000 7f6823b09700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f681c157460 0x7f681c06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:26.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.133+0000 7f6823b09700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f681c067cc0 0x7f681c0ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:26.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.133+0000 7f6823b09700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f681c0aec90 con 0x7f681c0cdb00 2023-10-06T10:41:26.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.133+0000 7f6823b09700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f681c0c4030 con 0x7f681c067cc0 2023-10-06T10:41:26.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.133+0000 7f6823b09700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681c06f3b0 con 0x7f681c157460 2023-10-06T10:41:26.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.133+0000 7f68220a6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f681c067cc0 0x7f681c0ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:26.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.133+0000 7f68220a6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f681c067cc0 0x7f681c0ac5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33298/0 (socket says 172.21.15.92:33298) 2023-10-06T10:41:26.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.133+0000 7f68220a6700 1 -- 172.21.15.92:0/103971085 learned_addr learned my addr 172.21.15.92:0/103971085 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:26.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.133+0000 7f68210a4700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f681c0cdb00 0x7f681c0675b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51082/0 (socket says 172.21.15.92:51082) 2023-10-06T10:41:26.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.133+0000 7f68218a5700 1 --2- 172.21.15.92:0/103971085 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f681c157460 0x7f681c06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:26.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.133+0000 7f68220a6700 1 -- 172.21.15.92:0/103971085 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f681c157460 msgr2=0x7f681c06ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:26.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.133+0000 7f68220a6700 1 --2- 172.21.15.92:0/103971085 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f681c157460 0x7f681c06ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.133+0000 7f68220a6700 1 -- 172.21.15.92:0/103971085 >> v1:172.21.15.92:6789/0 conn(0x7f681c0cdb00 legacy=0x7f681c0675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:26.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.133+0000 7f68208a3700 1 -- 172.21.15.92:0/103971085 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1355638820 0 0) 0x7f681c0aec90 con 0x7f681c0cdb00 2023-10-06T10:41:26.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.133+0000 7f68220a6700 1 -- 172.21.15.92:0/103971085 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681c0aef70 con 0x7f681c067cc0 2023-10-06T10:41:26.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.133+0000 7f68220a6700 1 --2- 172.21.15.92:0/103971085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f681c067cc0 0x7f681c0ac5a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6814009f20 tx=0x7f6814009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fd9ce25899a558 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:26.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.133+0000 7f68208a3700 1 -- 172.21.15.92:0/103971085 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681400e070 con 0x7f681c067cc0 2023-10-06T10:41:26.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.133+0000 7f68208a3700 1 -- 172.21.15.92:0/103971085 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681400b9d0 con 0x7f681c067cc0 2023-10-06T10:41:26.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.133+0000 7f68208a3700 1 -- 172.21.15.92:0/103971085 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68140092e0 con 0x7f681c067cc0 2023-10-06T10:41:26.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.134+0000 7f6823b09700 1 -- 172.21.15.92:0/103971085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f681c067cc0 msgr2=0x7f681c0ac5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:26.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.134+0000 7f6823b09700 1 --2- 172.21.15.92:0/103971085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f681c067cc0 0x7f681c0ac5a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6814009f20 tx=0x7f6814009840 comp rx=0 tx=0).stop 2023-10-06T10:41:26.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.134+0000 7f6823b09700 1 -- 172.21.15.92:0/103971085 shutdown_connections 2023-10-06T10:41:26.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.134+0000 7f6823b09700 1 --2- 172.21.15.92:0/103971085 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f681c157460 0x7f681c06ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.134+0000 7f6823b09700 1 --2- 172.21.15.92:0/103971085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f681c067cc0 0x7f681c0ac5a0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.134+0000 7f6823b09700 1 -- 172.21.15.92:0/103971085 >> 172.21.15.92:0/103971085 conn(0x7f681c069140 msgr2=0x7f681c0cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:26.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.134+0000 7f6823b09700 1 -- 172.21.15.92:0/103971085 shutdown_connections 2023-10-06T10:41:26.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.134+0000 7f6823b09700 1 -- 172.21.15.92:0/103971085 wait complete. 2023-10-06T10:41:26.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.135+0000 7f6823b09700 1 Processor -- start 2023-10-06T10:41:26.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.135+0000 7f6823b09700 1 -- start start 2023-10-06T10:41:26.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.136+0000 7f6823b09700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f681c067cc0 0x7f681c0b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:26.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.136+0000 7f6823b09700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f681c0cdb00 0x7f681c154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:26.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.136+0000 7f6823b09700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f681c157460 0x7f681c0cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:26.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.136+0000 7f6823b09700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681c0b23a0 con 0x7f681c067cc0 2023-10-06T10:41:26.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.136+0000 7f6823b09700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681c0b12d0 con 0x7f681c157460 2023-10-06T10:41:26.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.136+0000 7f6823b09700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681c0b15b0 con 0x7f681c0cdb00 2023-10-06T10:41:26.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.136+0000 7f68218a5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f681c067cc0 0x7f681c0b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:26.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.136+0000 7f68218a5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f681c067cc0 0x7f681c0b2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33300/0 (socket says 172.21.15.92:33300) 2023-10-06T10:41:26.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.136+0000 7f68218a5700 1 -- 172.21.15.92:0/2562868381 learned_addr learned my addr 172.21.15.92:0/2562868381 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:26.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.136+0000 7f68220a6700 1 --2- 172.21.15.92:0/2562868381 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f681c157460 0x7f681c0cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:26.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.136+0000 7f68210a4700 1 --2- 172.21.15.92:0/2562868381 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f681c0cdb00 0x7f681c154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:26.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.136+0000 7f68218a5700 1 -- 172.21.15.92:0/2562868381 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f681c0cdb00 msgr2=0x7f681c154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:26.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.136+0000 7f68218a5700 1 --2- 172.21.15.92:0/2562868381 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f681c0cdb00 0x7f681c154670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.136+0000 7f68218a5700 1 -- 172.21.15.92:0/2562868381 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f681c157460 msgr2=0x7f681c0cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:26.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.136+0000 7f68218a5700 1 --2- 172.21.15.92:0/2562868381 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f681c157460 0x7f681c0cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.136+0000 7f68218a5700 1 -- 172.21.15.92:0/2562868381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6814009ae0 con 0x7f681c067cc0 2023-10-06T10:41:26.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.136+0000 7f68218a5700 1 --2- 172.21.15.92:0/2562868381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f681c067cc0 0x7f681c0b2860 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f681800e930 tx=0x7f68180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:26.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.136+0000 7f6811ffb700 1 -- 172.21.15.92:0/2562868381 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681800eed0 con 0x7f681c067cc0 2023-10-06T10:41:26.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.136+0000 7f6811ffb700 1 -- 172.21.15.92:0/2562868381 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681800c850 con 0x7f681c067cc0 2023-10-06T10:41:26.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.136+0000 7f6823b09700 1 -- 172.21.15.92:0/2562868381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681c0b1890 con 0x7f681c067cc0 2023-10-06T10:41:26.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.136+0000 7f6811ffb700 1 -- 172.21.15.92:0/2562868381 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681800cbf0 con 0x7f681c067cc0 2023-10-06T10:41:26.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.137+0000 7f6823b09700 1 -- 172.21.15.92:0/2562868381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681c0b1e50 con 0x7f681c067cc0 2023-10-06T10:41:26.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.138+0000 7f6823b09700 1 -- 172.21.15.92:0/2562868381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f681c15b030 con 0x7f681c067cc0 2023-10-06T10:41:26.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.138+0000 7f6811ffb700 1 -- 172.21.15.92:0/2562868381 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6818013470 con 0x7f681c067cc0 2023-10-06T10:41:26.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.138+0000 7f6811ffb700 1 --2- 172.21.15.92:0/2562868381 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6800066f30 0x7f68000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:26.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.138+0000 7f6811ffb700 1 -- 172.21.15.92:0/2562868381 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6818084a00 con 0x7f681c067cc0 2023-10-06T10:41:26.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.139+0000 7f68210a4700 1 --2- 172.21.15.92:0/2562868381 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6800066f30 0x7f68000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:26.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.139+0000 7f68210a4700 1 --2- 172.21.15.92:0/2562868381 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6800066f30 0x7f68000693e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f681c15b390 tx=0x7f680c007bf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:26.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.143+0000 7f6811ffb700 1 -- 172.21.15.92:0/2562868381 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68180519d0 con 0x7f681c067cc0 2023-10-06T10:41:26.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.366+0000 7f6823b09700 1 -- 172.21.15.92:0/2562868381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f681c064a90 con 0x7f681c067cc0 2023-10-06T10:41:26.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.366+0000 7f6811ffb700 1 -- 172.21.15.92:0/2562868381 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4637 (secure 0 0 0) 0x7f6818054ff0 con 0x7f681c067cc0 2023-10-06T10:41:26.368 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:26.368 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3668157650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3668157650},{"type":"v1","addr":"172.21.15.124:6834","nonce":3668157650}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:40:34.932631+0000","modified":"2023-10-06T10:41:17.396042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5736,"mds_1":5733,"mds_2":5036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5036":{"gid":5036,"name":"d","rank":2,"incarnation":36,"state":"up:active","state_seq":13,"addr":"172.21.15.124:6835/2986058955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2986058955},{"type":"v1","addr":"172.21.15.124:6835","nonce":2986058955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5733":{"gid":5733,"name":"c","rank":1,"incarnation":34,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6836/831923836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":831923836},{"type":"v1","addr":"172.21.15.92:6836","nonce":831923836}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5736":{"gid":5736,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6837/2458340622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2458340622},{"type":"v1","addr":"172.21.15.92:6837","nonce":2458340622}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T10:41:26.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.368+0000 7f6823b09700 1 -- 172.21.15.92:0/2562868381 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6800066f30 msgr2=0x7f68000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:26.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.368+0000 7f6823b09700 1 --2- 172.21.15.92:0/2562868381 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6800066f30 0x7f68000693e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f681c15b390 tx=0x7f680c007bf0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.368+0000 7f6823b09700 1 -- 172.21.15.92:0/2562868381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f681c067cc0 msgr2=0x7f681c0b2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:26.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.368+0000 7f6823b09700 1 --2- 172.21.15.92:0/2562868381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f681c067cc0 0x7f681c0b2860 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f681800e930 tx=0x7f68180091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.368+0000 7f6823b09700 1 -- 172.21.15.92:0/2562868381 shutdown_connections 2023-10-06T10:41:26.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.368+0000 7f6823b09700 1 --2- 172.21.15.92:0/2562868381 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6800066f30 0x7f68000693e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.368+0000 7f6823b09700 1 --2- 172.21.15.92:0/2562868381 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f681c157460 0x7f681c0cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.368+0000 7f6823b09700 1 --2- 172.21.15.92:0/2562868381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f681c067cc0 0x7f681c0b2860 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.368+0000 7f6823b09700 1 --2- 172.21.15.92:0/2562868381 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f681c0cdb00 0x7f681c154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.368+0000 7f6823b09700 1 -- 172.21.15.92:0/2562868381 >> 172.21.15.92:0/2562868381 conn(0x7f681c069140 msgr2=0x7f681c1565d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:26.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.368+0000 7f6823b09700 1 -- 172.21.15.92:0/2562868381 shutdown_connections 2023-10-06T10:41:26.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.368+0000 7f6823b09700 1 -- 172.21.15.92:0/2562868381 wait complete. 2023-10-06T10:41:26.372 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 42 2023-10-06T10:41:26.382 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'd'] 2023-10-06T10:41:26.382 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:41:26.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.495+0000 7efe1ad21700 1 Processor -- start 2023-10-06T10:41:26.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.495+0000 7efe1ad21700 1 -- start start 2023-10-06T10:41:26.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.496+0000 7efe1ad21700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe14159440 0x7efe140b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:26.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.496+0000 7efe1ad21700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe140cb7a0 con 0x7efe140b2100 2023-10-06T10:41:26.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.496+0000 7efe1ad21700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe140b0860 con 0x7efe140b35d0 2023-10-06T10:41:26.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.496+0000 7efe1ad21700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe140cb150 con 0x7efe14159440 2023-10-06T10:41:26.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.496+0000 7efe1a520700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7efe140b2100 0x7efe140b2580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51092/0 (socket says 172.21.15.92:51092) 2023-10-06T10:41:26.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.496+0000 7efe1a520700 1 -- 172.21.15.92:0/3032696194 learned_addr learned my addr 172.21.15.92:0/3032696194 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:26.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.496+0000 7efe19d1f700 1 --2- 172.21.15.92:0/3032696194 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe14159440 0x7efe140b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:26.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.496+0000 7efe18d1d700 1 -- 172.21.15.92:0/3032696194 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 322931288 0 0) 0x7efe140cb7a0 con 0x7efe140b2100 2023-10-06T10:41:26.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.496+0000 7efe18d1d700 1 -- 172.21.15.92:0/3032696194 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe00003570 con 0x7efe140b2100 2023-10-06T10:41:26.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe18d1d700 1 -- 172.21.15.92:0/3032696194 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248166847 0 0) 0x7efe140b0860 con 0x7efe140b35d0 2023-10-06T10:41:26.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe18d1d700 1 -- 172.21.15.92:0/3032696194 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe140cb7a0 con 0x7efe140b35d0 2023-10-06T10:41:26.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe18d1d700 1 -- 172.21.15.92:0/3032696194 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 950119916 0 0) 0x7efe00003570 con 0x7efe140b2100 2023-10-06T10:41:26.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe18d1d700 1 -- 172.21.15.92:0/3032696194 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe14159440 msgr2=0x7efe140b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:26.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe18d1d700 1 --2- 172.21.15.92:0/3032696194 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe14159440 0x7efe140b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe18d1d700 1 -- 172.21.15.92:0/3032696194 >> v1:172.21.15.124:6789/0 conn(0x7efe140b35d0 legacy=0x7efe140cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:26.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe18d1d700 1 -- 172.21.15.92:0/3032696194 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe140cb430 con 0x7efe140b2100 2023-10-06T10:41:26.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe18d1d700 1 -- 172.21.15.92:0/3032696194 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7efe0c0017f0 con 0x7efe140b2100 2023-10-06T10:41:26.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe19d1f700 1 --2- 172.21.15.92:0/3032696194 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe14159440 0x7efe140b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:41:26.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe18d1d700 1 -- 172.21.15.92:0/3032696194 >> v1:172.21.15.92:6789/0 conn(0x7efe140b2100 legacy=0x7efe140b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:26.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe18d1d700 1 --2- 172.21.15.92:0/3032696194 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe14159440 0x7efe00003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:26.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe18d1d700 1 --2- 172.21.15.92:0/3032696194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe00004160 0x7efe00004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:26.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe18d1d700 1 --2- 172.21.15.92:0/3032696194 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe00004d50 0x7efe000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:26.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe18d1d700 1 -- 172.21.15.92:0/3032696194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe140cb150 con 0x7efe00004160 2023-10-06T10:41:26.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe18d1d700 1 -- 172.21.15.92:0/3032696194 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0000bc20 con 0x7efe00004d50 2023-10-06T10:41:26.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe18d1d700 1 -- 172.21.15.92:0/3032696194 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0000bde0 con 0x7efe14159440 2023-10-06T10:41:26.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe1951e700 1 --2- 172.21.15.92:0/3032696194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe00004160 0x7efe00004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:26.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe19d1f700 1 --2- 172.21.15.92:0/3032696194 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe14159440 0x7efe00003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:26.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe1a520700 1 --2- 172.21.15.92:0/3032696194 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe00004d50 0x7efe000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:26.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe1951e700 1 -- 172.21.15.92:0/3032696194 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe14159440 msgr2=0x7efe00003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:26.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe1951e700 1 --2- 172.21.15.92:0/3032696194 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe14159440 0x7efe00003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe1951e700 1 -- 172.21.15.92:0/3032696194 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe00004d50 msgr2=0x7efe000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:26.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe1951e700 1 --2- 172.21.15.92:0/3032696194 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe00004d50 0x7efe000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe1951e700 1 -- 172.21.15.92:0/3032696194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe10009420 con 0x7efe00004160 2023-10-06T10:41:26.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe1951e700 1 --2- 172.21.15.92:0/3032696194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe00004160 0x7efe00004810 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7efe04001a10 tx=0x7efe04007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843f489f9c76efcd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:26.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe18d1d700 1 -- 172.21.15.92:0/3032696194 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0400c5d0 con 0x7efe00004160 2023-10-06T10:41:26.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe18d1d700 1 -- 172.21.15.92:0/3032696194 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0400c730 con 0x7efe00004160 2023-10-06T10:41:26.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.497+0000 7efe18d1d700 1 -- 172.21.15.92:0/3032696194 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0400cad0 con 0x7efe00004160 2023-10-06T10:41:26.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.498+0000 7efe1ad21700 1 -- 172.21.15.92:0/3032696194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe00004160 msgr2=0x7efe00004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:26.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.498+0000 7efe1ad21700 1 --2- 172.21.15.92:0/3032696194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe00004160 0x7efe00004810 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7efe04001a10 tx=0x7efe04007d30 comp rx=0 tx=0).stop 2023-10-06T10:41:26.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.498+0000 7efe1a520700 1 -- 172.21.15.92:0/3032696194 reap_dead start 2023-10-06T10:41:26.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.498+0000 7efe1ad21700 1 -- 172.21.15.92:0/3032696194 shutdown_connections 2023-10-06T10:41:26.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.498+0000 7efe1ad21700 1 -- 172.21.15.92:0/3032696194 >> 172.21.15.92:0/3032696194 conn(0x7efe1406ece0 msgr2=0x7efe1406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:26.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.498+0000 7efe1ad21700 1 -- 172.21.15.92:0/3032696194 shutdown_connections 2023-10-06T10:41:26.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.498+0000 7efe1ad21700 1 -- 172.21.15.92:0/3032696194 wait complete. 2023-10-06T10:41:26.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.499+0000 7efe1ad21700 1 Processor -- start 2023-10-06T10:41:26.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.499+0000 7efe1ad21700 1 -- start start 2023-10-06T10:41:26.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.500+0000 7efe1ad21700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe00004160 0x7efe1415aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:26.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.500+0000 7efe1ad21700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe14155800 0x7efe140c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:26.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.500+0000 7efe1ad21700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe140c9110 0x7efe14169650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:26.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.500+0000 7efe1ad21700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1416a640 con 0x7efe00004160 2023-10-06T10:41:26.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.500+0000 7efe1ad21700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe140c8740 con 0x7efe14155800 2023-10-06T10:41:26.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.500+0000 7efe1ad21700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe1416a080 con 0x7efe140c9110 2023-10-06T10:41:26.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.500+0000 7efe19d1f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe00004160 0x7efe1415aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:26.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.500+0000 7efe19d1f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe00004160 0x7efe1415aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33328/0 (socket says 172.21.15.92:33328) 2023-10-06T10:41:26.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.500+0000 7efe19d1f700 1 -- 172.21.15.92:0/421489730 learned_addr learned my addr 172.21.15.92:0/421489730 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:26.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.500+0000 7efe1951e700 1 --2- 172.21.15.92:0/421489730 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe14155800 0x7efe140c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:26.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.500+0000 7efe1a520700 1 --2- 172.21.15.92:0/421489730 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe140c9110 0x7efe14169650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:26.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.500+0000 7efe19d1f700 1 -- 172.21.15.92:0/421489730 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe140c9110 msgr2=0x7efe14169650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:26.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.500+0000 7efe19d1f700 1 --2- 172.21.15.92:0/421489730 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe140c9110 0x7efe14169650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.500+0000 7efe19d1f700 1 -- 172.21.15.92:0/421489730 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe14155800 msgr2=0x7efe140c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:26.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.500+0000 7efe19d1f700 1 --2- 172.21.15.92:0/421489730 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe14155800 0x7efe140c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.500+0000 7efe19d1f700 1 -- 172.21.15.92:0/421489730 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1416a360 con 0x7efe00004160 2023-10-06T10:41:26.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.500+0000 7efe19d1f700 1 --2- 172.21.15.92:0/421489730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe00004160 0x7efe1415aea0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7efe10011d70 tx=0x7efe100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:26.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.500+0000 7efe09ffb700 1 -- 172.21.15.92:0/421489730 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1000fcd0 con 0x7efe00004160 2023-10-06T10:41:26.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.501+0000 7efe09ffb700 1 -- 172.21.15.92:0/421489730 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1000fe30 con 0x7efe00004160 2023-10-06T10:41:26.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.501+0000 7efe1ad21700 1 -- 172.21.15.92:0/421489730 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe140c83d0 con 0x7efe00004160 2023-10-06T10:41:26.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.501+0000 7efe09ffb700 1 -- 172.21.15.92:0/421489730 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10014670 con 0x7efe00004160 2023-10-06T10:41:26.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.501+0000 7efe1ad21700 1 -- 172.21.15.92:0/421489730 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe14169dd0 con 0x7efe00004160 2023-10-06T10:41:26.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.502+0000 7efe1ad21700 1 -- 172.21.15.92:0/421489730 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe1405f230 con 0x7efe00004160 2023-10-06T10:41:26.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.502+0000 7efe09ffb700 1 -- 172.21.15.92:0/421489730 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efe10014aa0 con 0x7efe00004160 2023-10-06T10:41:26.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.502+0000 7efe09ffb700 1 --2- 172.21.15.92:0/421489730 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efdf8066f30 0x7efdf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:26.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.502+0000 7efe09ffb700 1 -- 172.21.15.92:0/421489730 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7efe10018030 con 0x7efe00004160 2023-10-06T10:41:26.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.502+0000 7efe1951e700 1 --2- 172.21.15.92:0/421489730 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efdf8066f30 0x7efdf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:26.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.503+0000 7efe1951e700 1 --2- 172.21.15.92:0/421489730 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efdf8066f30 0x7efdf80693e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efe04008fd0 tx=0x7efe04015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:26.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.507+0000 7efe09ffb700 1 -- 172.21.15.92:0/421489730 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe100526e0 con 0x7efe00004160 2023-10-06T10:41:26.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.766+0000 7efe1ad21700 1 -- 172.21.15.92:0/421489730 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7efe140c8a20 con 0x7efe00004160 2023-10-06T10:41:26.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.766+0000 7efe09ffb700 1 -- 172.21.15.92:0/421489730 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1921 (secure 0 0 0) 0x7efe10055d00 con 0x7efe00004160 2023-10-06T10:41:26.768 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:26.768 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":208,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":52,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":704416,"bytes_used":2441891840,"bytes_avail":770652221440,"bytes_total":773094113280,"write_bytes_sec":938,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":42,"id":3,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":3,"rank":2,"name":"d","status":"up:active","gid":5036},{"filesystem_id":3,"rank":1,"name":"c","status":"up:active","gid":5733},{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5736}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2023-10-06T10:41:26.124133+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T10:41:26.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.768+0000 7efe1ad21700 1 -- 172.21.15.92:0/421489730 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efdf8066f30 msgr2=0x7efdf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:26.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.768+0000 7efe1ad21700 1 --2- 172.21.15.92:0/421489730 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efdf8066f30 0x7efdf80693e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efe04008fd0 tx=0x7efe04015000 comp rx=0 tx=0).stop 2023-10-06T10:41:26.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.768+0000 7efe1ad21700 1 -- 172.21.15.92:0/421489730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe00004160 msgr2=0x7efe1415aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:26.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.768+0000 7efe1ad21700 1 --2- 172.21.15.92:0/421489730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe00004160 0x7efe1415aea0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7efe10011d70 tx=0x7efe100091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.768+0000 7efe1ad21700 1 -- 172.21.15.92:0/421489730 shutdown_connections 2023-10-06T10:41:26.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.768+0000 7efe1ad21700 1 --2- 172.21.15.92:0/421489730 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efdf8066f30 0x7efdf80693e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.768+0000 7efe1ad21700 1 --2- 172.21.15.92:0/421489730 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe140c9110 0x7efe14169650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.768+0000 7efe1ad21700 1 --2- 172.21.15.92:0/421489730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe00004160 0x7efe1415aea0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.768+0000 7efe1ad21700 1 --2- 172.21.15.92:0/421489730 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe14155800 0x7efe140c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:26.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.768+0000 7efe1ad21700 1 -- 172.21.15.92:0/421489730 >> 172.21.15.92:0/421489730 conn(0x7efe1406ece0 msgr2=0x7efe14158580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:26.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.768+0000 7efe1ad21700 1 -- 172.21.15.92:0/421489730 shutdown_connections 2023-10-06T10:41:26.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:26.768+0000 7efe1ad21700 1 -- 172.21.15.92:0/421489730 wait complete. 2023-10-06T10:41:26.782 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]'} 2023-10-06T10:41:27.782 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:41:27.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.904+0000 7f3d14b20700 1 Processor -- start 2023-10-06T10:41:27.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.904+0000 7f3d14b20700 1 -- start start 2023-10-06T10:41:27.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.905+0000 7f3d14b20700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d10159440 0x7f3d100b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:27.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.905+0000 7f3d14b20700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3d100b35d0 0x7f3d100cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:27.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.905+0000 7f3d14b20700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d100cb7a0 con 0x7f3d100cc170 2023-10-06T10:41:27.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.905+0000 7f3d14b20700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d100b0860 con 0x7f3d100b35d0 2023-10-06T10:41:27.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.905+0000 7f3d14b20700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d100cb150 con 0x7f3d10159440 2023-10-06T10:41:27.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.905+0000 7f3d0f7fe700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d10159440 0x7f3d100b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.905+0000 7f3d0effd700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3d100b35d0 0x7f3d100cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.905+0000 7f3d0ffff700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f3d100cc170 0x7f3d100b2100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:40962/0 (socket says 172.21.15.92:40962) 2023-10-06T10:41:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.905+0000 7f3d0effd700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3d100b35d0 0x7f3d100cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:47056/0 (socket says 172.21.15.92:47056) 2023-10-06T10:41:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.905+0000 7f3d0f7fe700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d10159440 0x7f3d100b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:37634/0 (socket says 172.21.15.92:37634) 2023-10-06T10:41:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.905+0000 7f3d0ffff700 1 -- 172.21.15.92:0/2450799696 learned_addr learned my addr 172.21.15.92:0/2450799696 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.906+0000 7f3d0e7fc700 1 -- 172.21.15.92:0/2450799696 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2057919611 0 0) 0x7f3d100cb7a0 con 0x7f3d100cc170 2023-10-06T10:41:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.906+0000 7f3d0e7fc700 1 -- 172.21.15.92:0/2450799696 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cf8003570 con 0x7f3d100cc170 2023-10-06T10:41:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.906+0000 7f3d0effd700 1 -- 172.21.15.92:0/2450799696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d10159440 msgr2=0x7f3d100b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.906+0000 7f3d0effd700 1 --2- 172.21.15.92:0/2450799696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d10159440 0x7f3d100b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.906+0000 7f3d0effd700 1 -- 172.21.15.92:0/2450799696 >> v1:172.21.15.124:6789/0 conn(0x7f3d100cc170 legacy=0x7f3d100b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.906+0000 7f3d0effd700 1 -- 172.21.15.92:0/2450799696 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d100cb430 con 0x7f3d100b35d0 2023-10-06T10:41:27.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.906+0000 7f3d0f7fe700 1 --2- 172.21.15.92:0/2450799696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d10159440 0x7f3d100b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:41:27.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.906+0000 7f3d0effd700 1 --2- 172.21.15.92:0/2450799696 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d100b35d0 0x7f3d100cbc30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3d040098e0 tx=0x7f3d040091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b8725ac1d5a9471 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:27.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.906+0000 7f3d0e7fc700 1 -- 172.21.15.92:0/2450799696 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d04011630 con 0x7f3d100b35d0 2023-10-06T10:41:27.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.906+0000 7f3d0e7fc700 1 -- 172.21.15.92:0/2450799696 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d04011790 con 0x7f3d100b35d0 2023-10-06T10:41:27.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.906+0000 7f3d0e7fc700 1 -- 172.21.15.92:0/2450799696 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d04011b30 con 0x7f3d100b35d0 2023-10-06T10:41:27.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.907+0000 7f3d14b20700 1 -- 172.21.15.92:0/2450799696 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d100b35d0 msgr2=0x7f3d100cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:27.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.907+0000 7f3d14b20700 1 --2- 172.21.15.92:0/2450799696 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d100b35d0 0x7f3d100cbc30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3d040098e0 tx=0x7f3d040091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:27.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.907+0000 7f3d14b20700 1 -- 172.21.15.92:0/2450799696 shutdown_connections 2023-10-06T10:41:27.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.907+0000 7f3d14b20700 1 --2- 172.21.15.92:0/2450799696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d10159440 0x7f3d100b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:27.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.907+0000 7f3d14b20700 1 --2- 172.21.15.92:0/2450799696 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d100b35d0 0x7f3d100cbc30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:27.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.907+0000 7f3d14b20700 1 -- 172.21.15.92:0/2450799696 >> 172.21.15.92:0/2450799696 conn(0x7f3d1006ece0 msgr2=0x7f3d1006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:27.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.907+0000 7f3d14b20700 1 -- 172.21.15.92:0/2450799696 shutdown_connections 2023-10-06T10:41:27.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.907+0000 7f3d14b20700 1 -- 172.21.15.92:0/2450799696 wait complete. 2023-10-06T10:41:27.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.908+0000 7f3d14b20700 1 Processor -- start 2023-10-06T10:41:27.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.908+0000 7f3d14b20700 1 -- start start 2023-10-06T10:41:27.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.909+0000 7f3d14b20700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d100b35d0 0x7f3d1015d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:27.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.909+0000 7f3d14b20700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d100cc170 0x7f3d1015b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:27.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.909+0000 7f3d14b20700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d10159440 0x7f3d1015bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:27.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.909+0000 7f3d14b20700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d100c7b70 con 0x7f3d100b35d0 2023-10-06T10:41:27.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.909+0000 7f3d14b20700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d100ce140 con 0x7f3d10159440 2023-10-06T10:41:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.909+0000 7f3d14b20700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d100ce420 con 0x7f3d100cc170 2023-10-06T10:41:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.909+0000 7f3d0f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d100b35d0 0x7f3d1015d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.909+0000 7f3d0f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d100b35d0 0x7f3d1015d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33334/0 (socket says 172.21.15.92:33334) 2023-10-06T10:41:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.909+0000 7f3d0f7fe700 1 -- 172.21.15.92:0/3526881618 learned_addr learned my addr 172.21.15.92:0/3526881618 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.909+0000 7f3d0effd700 1 --2- 172.21.15.92:0/3526881618 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d100cc170 0x7f3d1015b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.909+0000 7f3d0ffff700 1 --2- 172.21.15.92:0/3526881618 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d10159440 0x7f3d1015bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.909+0000 7f3d0f7fe700 1 -- 172.21.15.92:0/3526881618 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d100cc170 msgr2=0x7f3d1015b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:27.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.909+0000 7f3d0f7fe700 1 --2- 172.21.15.92:0/3526881618 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d100cc170 0x7f3d1015b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:27.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.909+0000 7f3d0f7fe700 1 -- 172.21.15.92:0/3526881618 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d10159440 msgr2=0x7f3d1015bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:27.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.909+0000 7f3d0f7fe700 1 --2- 172.21.15.92:0/3526881618 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d10159440 0x7f3d1015bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:27.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.909+0000 7f3d0f7fe700 1 -- 172.21.15.92:0/3526881618 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d00009a20 con 0x7f3d100b35d0 2023-10-06T10:41:27.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.909+0000 7f3d0f7fe700 1 --2- 172.21.15.92:0/3526881618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d100b35d0 0x7f3d1015d680 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000c0c0 tx=0x7f3d0000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:27.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.909+0000 7f3cf77fe700 1 -- 172.21.15.92:0/3526881618 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0000cd60 con 0x7f3d100b35d0 2023-10-06T10:41:27.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.910+0000 7f3cf77fe700 1 -- 172.21.15.92:0/3526881618 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0000f040 con 0x7f3d100b35d0 2023-10-06T10:41:27.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.910+0000 7f3d14b20700 1 -- 172.21.15.92:0/3526881618 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d04009420 con 0x7f3d100b35d0 2023-10-06T10:41:27.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.910+0000 7f3cf77fe700 1 -- 172.21.15.92:0/3526881618 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d00014690 con 0x7f3d100b35d0 2023-10-06T10:41:27.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.910+0000 7f3d14b20700 1 -- 172.21.15.92:0/3526881618 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d100ce9e0 con 0x7f3d100b35d0 2023-10-06T10:41:27.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.910+0000 7f3d14b20700 1 -- 172.21.15.92:0/3526881618 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d1005f230 con 0x7f3d100b35d0 2023-10-06T10:41:27.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.911+0000 7f3cf77fe700 1 -- 172.21.15.92:0/3526881618 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d00014a00 con 0x7f3d100b35d0 2023-10-06T10:41:27.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.911+0000 7f3cf77fe700 1 --2- 172.21.15.92:0/3526881618 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3cfc066f30 0x7f3cfc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:27.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.911+0000 7f3cf77fe700 1 -- 172.21.15.92:0/3526881618 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3d000856c0 con 0x7f3d100b35d0 2023-10-06T10:41:27.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.911+0000 7f3d0effd700 1 --2- 172.21.15.92:0/3526881618 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3cfc066f30 0x7f3cfc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:27.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.912+0000 7f3d0effd700 1 --2- 172.21.15.92:0/3526881618 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3cfc066f30 0x7f3cfc0693e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3d040091e0 tx=0x7f3d04017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:27.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:27.915+0000 7f3cf77fe700 1 -- 172.21.15.92:0/3526881618 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d00052690 con 0x7f3d100b35d0 2023-10-06T10:41:28.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.140+0000 7f3d14b20700 1 -- 172.21.15.92:0/3526881618 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d10159810 con 0x7f3d100b35d0 2023-10-06T10:41:28.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.140+0000 7f3cf77fe700 1 -- 172.21.15.92:0/3526881618 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4637 (secure 0 0 0) 0x7f3d00014cb0 con 0x7f3d100b35d0 2023-10-06T10:41:28.142 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:28.142 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3668157650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3668157650},{"type":"v1","addr":"172.21.15.124:6834","nonce":3668157650}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:40:34.932631+0000","modified":"2023-10-06T10:41:17.396042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5736,"mds_1":5733,"mds_2":5036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5036":{"gid":5036,"name":"d","rank":2,"incarnation":36,"state":"up:active","state_seq":13,"addr":"172.21.15.124:6835/2986058955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2986058955},{"type":"v1","addr":"172.21.15.124:6835","nonce":2986058955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5733":{"gid":5733,"name":"c","rank":1,"incarnation":34,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6836/831923836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":831923836},{"type":"v1","addr":"172.21.15.92:6836","nonce":831923836}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5736":{"gid":5736,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6837/2458340622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2458340622},{"type":"v1","addr":"172.21.15.92:6837","nonce":2458340622}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T10:41:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.142+0000 7f3d14b20700 1 -- 172.21.15.92:0/3526881618 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3cfc066f30 msgr2=0x7f3cfc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.142+0000 7f3d14b20700 1 --2- 172.21.15.92:0/3526881618 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3cfc066f30 0x7f3cfc0693e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3d040091e0 tx=0x7f3d04017000 comp rx=0 tx=0).stop 2023-10-06T10:41:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.142+0000 7f3d14b20700 1 -- 172.21.15.92:0/3526881618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d100b35d0 msgr2=0x7f3d1015d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.142+0000 7f3d14b20700 1 --2- 172.21.15.92:0/3526881618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d100b35d0 0x7f3d1015d680 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000c0c0 tx=0x7f3d0000b040 comp rx=0 tx=0).stop 2023-10-06T10:41:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.142+0000 7f3d14b20700 1 -- 172.21.15.92:0/3526881618 shutdown_connections 2023-10-06T10:41:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.142+0000 7f3d14b20700 1 --2- 172.21.15.92:0/3526881618 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3cfc066f30 0x7f3cfc0693e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.142+0000 7f3d14b20700 1 --2- 172.21.15.92:0/3526881618 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d10159440 0x7f3d1015bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.142+0000 7f3d14b20700 1 --2- 172.21.15.92:0/3526881618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d100b35d0 0x7f3d1015d680 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.142+0000 7f3d14b20700 1 --2- 172.21.15.92:0/3526881618 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d100cc170 0x7f3d1015b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.142+0000 7f3d14b20700 1 -- 172.21.15.92:0/3526881618 >> 172.21.15.92:0/3526881618 conn(0x7f3d1006ece0 msgr2=0x7f3d100b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.143+0000 7f3d14b20700 1 -- 172.21.15.92:0/3526881618 shutdown_connections 2023-10-06T10:41:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.143+0000 7f3d14b20700 1 -- 172.21.15.92:0/3526881618 wait complete. 2023-10-06T10:41:28.146 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 42 2023-10-06T10:41:28.156 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'd'] 2023-10-06T10:41:28.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:41:28.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.267+0000 7f72c38e5700 1 Processor -- start 2023-10-06T10:41:28.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.267+0000 7f72c38e5700 1 -- start start 2023-10-06T10:41:28.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.268+0000 7f72c38e5700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72bc159560 0x7f72bc0aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:28.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.268+0000 7f72c38e5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f72bc0b49a0 0x7f72bc0aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:28.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.268+0000 7f72c38e5700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f72bc0b0760 0x7f72bc0b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:28.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.268+0000 7f72c38e5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72bc0b0c60 con 0x7f72bc0b49a0 2023-10-06T10:41:28.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.268+0000 7f72c38e5700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72bc0b0f40 con 0x7f72bc0b0760 2023-10-06T10:41:28.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.268+0000 7f72c38e5700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc0b1220 con 0x7f72bc159560 2023-10-06T10:41:28.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.268+0000 7f72c0e80700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f72bc0b49a0 0x7f72bc0aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:28.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.268+0000 7f72c0e80700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f72bc0b49a0 0x7f72bc0aaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33338/0 (socket says 172.21.15.92:33338) 2023-10-06T10:41:28.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.268+0000 7f72c0e80700 1 -- 172.21.15.92:0/2391987312 learned_addr learned my addr 172.21.15.92:0/2391987312 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:28.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.269+0000 7f72c0e80700 1 -- 172.21.15.92:0/2391987312 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72bc159560 msgr2=0x7f72bc0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:28.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.269+0000 7f72c1681700 1 --2- 172.21.15.92:0/2391987312 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72bc159560 0x7f72bc0aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:28.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.269+0000 7f72c1e82700 1 --2- 172.21.15.92:0/2391987312 >> v2:172.21.15.124:3300/0 conn(0x7f72bc0b0760 0x7f72bc0b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:28.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.269+0000 7f72c0e80700 1 --2- 172.21.15.92:0/2391987312 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72bc159560 0x7f72bc0aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:28.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.269+0000 7f72c0e80700 1 -- 172.21.15.92:0/2391987312 >> v2:172.21.15.124:3300/0 conn(0x7f72bc0b0760 msgr2=0x7f72bc0b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:28.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.269+0000 7f72c0e80700 1 --2- 172.21.15.92:0/2391987312 >> v2:172.21.15.124:3300/0 conn(0x7f72bc0b0760 0x7f72bc0b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:28.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.269+0000 7f72c0e80700 1 -- 172.21.15.92:0/2391987312 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72bc0b1500 con 0x7f72bc0b49a0 2023-10-06T10:41:28.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.269+0000 7f72c1681700 1 --2- 172.21.15.92:0/2391987312 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72bc159560 0x7f72bc0aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:28.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.269+0000 7f72c1e82700 1 --2- 172.21.15.92:0/2391987312 >> v2:172.21.15.124:3300/0 conn(0x7f72bc0b0760 0x7f72bc0b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:28.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.269+0000 7f72c0e80700 1 --2- 172.21.15.92:0/2391987312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72bc0b49a0 0x7f72bc0aaa70 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0b2f10 tx=0x7f72ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54abf708f920610d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:28.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.269+0000 7f72b3fff700 1 -- 172.21.15.92:0/2391987312 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac00b860 con 0x7f72bc0b49a0 2023-10-06T10:41:28.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.269+0000 7f72b3fff700 1 -- 172.21.15.92:0/2391987312 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ac00b9c0 con 0x7f72bc0b49a0 2023-10-06T10:41:28.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.269+0000 7f72b3fff700 1 -- 172.21.15.92:0/2391987312 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac00bd60 con 0x7f72bc0b49a0 2023-10-06T10:41:28.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.269+0000 7f72c38e5700 1 -- 172.21.15.92:0/2391987312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72bc0b49a0 msgr2=0x7f72bc0aaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:28.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.269+0000 7f72c38e5700 1 --2- 172.21.15.92:0/2391987312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72bc0b49a0 0x7f72bc0aaa70 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0b2f10 tx=0x7f72ac009250 comp rx=0 tx=0).stop 2023-10-06T10:41:28.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.269+0000 7f72c38e5700 1 -- 172.21.15.92:0/2391987312 shutdown_connections 2023-10-06T10:41:28.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.269+0000 7f72c38e5700 1 --2- 172.21.15.92:0/2391987312 >> v2:172.21.15.124:3300/0 conn(0x7f72bc0b0760 0x7f72bc0b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:28.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.269+0000 7f72c38e5700 1 --2- 172.21.15.92:0/2391987312 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72bc159560 0x7f72bc0aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:28.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.269+0000 7f72c38e5700 1 --2- 172.21.15.92:0/2391987312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72bc0b49a0 0x7f72bc0aaa70 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:28.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.269+0000 7f72c38e5700 1 -- 172.21.15.92:0/2391987312 >> 172.21.15.92:0/2391987312 conn(0x7f72bc0997f0 msgr2=0x7f72bc06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:28.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.270+0000 7f72c38e5700 1 -- 172.21.15.92:0/2391987312 shutdown_connections 2023-10-06T10:41:28.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.270+0000 7f72c38e5700 1 -- 172.21.15.92:0/2391987312 wait complete. 2023-10-06T10:41:28.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.271+0000 7f72c38e5700 1 Processor -- start 2023-10-06T10:41:28.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.271+0000 7f72c38e5700 1 -- start start 2023-10-06T10:41:28.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.271+0000 7f72c38e5700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f72bc0b0760 0x7f72bc099f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:28.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.271+0000 7f72c38e5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72bc0b49a0 0x7f72bc0b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:28.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.271+0000 7f72c38e5700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72bc159560 0x7f72bc14a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:28.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.271+0000 7f72c38e5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc09b3d0 con 0x7f72bc0b49a0 2023-10-06T10:41:28.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.271+0000 7f72c38e5700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc09a850 con 0x7f72bc0b0760 2023-10-06T10:41:28.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.271+0000 7f72c38e5700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc09ab30 con 0x7f72bc159560 2023-10-06T10:41:28.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.272+0000 7f72c0e80700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72bc0b49a0 0x7f72bc0b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:28.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.272+0000 7f72c0e80700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72bc0b49a0 0x7f72bc0b3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33342/0 (socket says 172.21.15.92:33342) 2023-10-06T10:41:28.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.272+0000 7f72c1681700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f72bc0b0760 0x7f72bc099f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:28.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.272+0000 7f72c0e80700 1 -- 172.21.15.92:0/555195241 learned_addr learned my addr 172.21.15.92:0/555195241 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:28.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.272+0000 7f72c1e82700 1 --2- 172.21.15.92:0/555195241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72bc159560 0x7f72bc14a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:28.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.272+0000 7f72c0e80700 1 -- 172.21.15.92:0/555195241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72bc159560 msgr2=0x7f72bc14a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:28.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.272+0000 7f72c0e80700 1 --2- 172.21.15.92:0/555195241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72bc159560 0x7f72bc14a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:28.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.272+0000 7f72c0e80700 1 -- 172.21.15.92:0/555195241 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f72bc0b0760 msgr2=0x7f72bc099f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:28.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.272+0000 7f72c0e80700 1 --2- 172.21.15.92:0/555195241 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f72bc0b0760 0x7f72bc099f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:28.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.272+0000 7f72c0e80700 1 -- 172.21.15.92:0/555195241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ac0094f0 con 0x7f72bc0b49a0 2023-10-06T10:41:28.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.272+0000 7f72c0e80700 1 --2- 172.21.15.92:0/555195241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72bc0b49a0 0x7f72bc0b3720 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f72bc067c20 tx=0x7f72ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:28.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.272+0000 7f72b17fa700 1 -- 172.21.15.92:0/555195241 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac011dc0 con 0x7f72bc0b49a0 2023-10-06T10:41:28.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.272+0000 7f72b17fa700 1 -- 172.21.15.92:0/555195241 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ac024440 con 0x7f72bc0b49a0 2023-10-06T10:41:28.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.272+0000 7f72b17fa700 1 -- 172.21.15.92:0/555195241 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac024800 con 0x7f72bc0b49a0 2023-10-06T10:41:28.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.272+0000 7f72c38e5700 1 -- 172.21.15.92:0/555195241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72bc09ae10 con 0x7f72bc0b49a0 2023-10-06T10:41:28.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.272+0000 7f72c38e5700 1 -- 172.21.15.92:0/555195241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72bc09a490 con 0x7f72bc0b49a0 2023-10-06T10:41:28.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.273+0000 7f72c38e5700 1 -- 172.21.15.92:0/555195241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72bc064a90 con 0x7f72bc0b49a0 2023-10-06T10:41:28.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.274+0000 7f72b17fa700 1 -- 172.21.15.92:0/555195241 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f72ac01a420 con 0x7f72bc0b49a0 2023-10-06T10:41:28.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.274+0000 7f72b17fa700 1 --2- 172.21.15.92:0/555195241 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f72a0066f30 0x7f72a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:28.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.274+0000 7f72b17fa700 1 -- 172.21.15.92:0/555195241 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f72ac0865f0 con 0x7f72bc0b49a0 2023-10-06T10:41:28.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.274+0000 7f72c1681700 1 --2- 172.21.15.92:0/555195241 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f72a0066f30 0x7f72a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:28.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.274+0000 7f72c1681700 1 --2- 172.21.15.92:0/555195241 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f72a0066f30 0x7f72a00693e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f72b800aca0 tx=0x7f72b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:28.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.278+0000 7f72b17fa700 1 -- 172.21.15.92:0/555195241 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72ac0535c0 con 0x7f72bc0b49a0 2023-10-06T10:41:28.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.542+0000 7f72c38e5700 1 -- 172.21.15.92:0/555195241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f72bc0b17e0 con 0x7f72bc0b49a0 2023-10-06T10:41:28.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.542+0000 7f72b17fa700 1 -- 172.21.15.92:0/555195241 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1921 (secure 0 0 0) 0x7f72ac056be0 con 0x7f72bc0b49a0 2023-10-06T10:41:28.544 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:28.544 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":210,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":52,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":704416,"bytes_used":2441891840,"bytes_avail":770652221440,"bytes_total":773094113280,"write_bytes_sec":682,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":42,"id":3,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":3,"rank":2,"name":"d","status":"up:active","gid":5036},{"filesystem_id":3,"rank":1,"name":"c","status":"up:active","gid":5733},{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5736}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2023-10-06T10:41:26.124133+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T10:41:28.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.544+0000 7f72c38e5700 1 -- 172.21.15.92:0/555195241 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f72a0066f30 msgr2=0x7f72a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:28.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.544+0000 7f72c38e5700 1 --2- 172.21.15.92:0/555195241 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f72a0066f30 0x7f72a00693e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f72b800aca0 tx=0x7f72b8008040 comp rx=0 tx=0).stop 2023-10-06T10:41:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.544+0000 7f72c38e5700 1 -- 172.21.15.92:0/555195241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72bc0b49a0 msgr2=0x7f72bc0b3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.544+0000 7f72c38e5700 1 --2- 172.21.15.92:0/555195241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72bc0b49a0 0x7f72bc0b3720 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f72bc067c20 tx=0x7f72ac017000 comp rx=0 tx=0).stop 2023-10-06T10:41:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.544+0000 7f72c38e5700 1 -- 172.21.15.92:0/555195241 shutdown_connections 2023-10-06T10:41:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.544+0000 7f72c38e5700 1 --2- 172.21.15.92:0/555195241 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f72a0066f30 0x7f72a00693e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.544+0000 7f72c38e5700 1 --2- 172.21.15.92:0/555195241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72bc159560 0x7f72bc14a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.544+0000 7f72c38e5700 1 --2- 172.21.15.92:0/555195241 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f72bc0b0760 0x7f72bc099f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.544+0000 7f72c38e5700 1 --2- 172.21.15.92:0/555195241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72bc0b49a0 0x7f72bc0b3720 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.544+0000 7f72c38e5700 1 -- 172.21.15.92:0/555195241 >> 172.21.15.92:0/555195241 conn(0x7f72bc0997f0 msgr2=0x7f72bc069cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.545+0000 7f72c38e5700 1 -- 172.21.15.92:0/555195241 shutdown_connections 2023-10-06T10:41:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:28.545+0000 7f72c38e5700 1 -- 172.21.15.92:0/555195241 wait complete. 2023-10-06T10:41:28.558 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]'} 2023-10-06T10:41:29.559 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:41:29.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.688+0000 7fef87995700 1 Processor -- start 2023-10-06T10:41:29.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.689+0000 7fef87995700 1 -- start start 2023-10-06T10:41:29.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.689+0000 7fef87995700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fef800b25e0 0x7fef800b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:29.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.689+0000 7fef87995700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fef800b29b0 0x7fef80159120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:29.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.689+0000 7fef87995700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef80159660 0x7fef80068540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:29.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.689+0000 7fef87995700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef800cbfa0 con 0x7fef800b29b0 2023-10-06T10:41:29.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.689+0000 7fef87995700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef800cb390 con 0x7fef800b25e0 2023-10-06T10:41:29.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.689+0000 7fef87995700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef800cb670 con 0x7fef80159660 2023-10-06T10:41:29.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.689+0000 7fef84f30700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fef800b29b0 0x7fef80159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:29.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.690+0000 7fef84f30700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fef800b29b0 0x7fef80159120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33346/0 (socket says 172.21.15.92:33346) 2023-10-06T10:41:29.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.690+0000 7fef84f30700 1 -- 172.21.15.92:0/1037580591 learned_addr learned my addr 172.21.15.92:0/1037580591 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:29.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.690+0000 7fef85f32700 1 --2- 172.21.15.92:0/1037580591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef80159660 0x7fef80068540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:29.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.690+0000 7fef85731700 1 --2- 172.21.15.92:0/1037580591 >> v2:172.21.15.124:3300/0 conn(0x7fef800b25e0 0x7fef800b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:29.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.690+0000 7fef84f30700 1 -- 172.21.15.92:0/1037580591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef80159660 msgr2=0x7fef80068540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:29.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.690+0000 7fef84f30700 1 --2- 172.21.15.92:0/1037580591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef80159660 0x7fef80068540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:29.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.690+0000 7fef84f30700 1 -- 172.21.15.92:0/1037580591 >> v2:172.21.15.124:3300/0 conn(0x7fef800b25e0 msgr2=0x7fef800b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:29.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.690+0000 7fef84f30700 1 --2- 172.21.15.92:0/1037580591 >> v2:172.21.15.124:3300/0 conn(0x7fef800b25e0 0x7fef800b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:29.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.690+0000 7fef84f30700 1 -- 172.21.15.92:0/1037580591 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef800cb950 con 0x7fef800b29b0 2023-10-06T10:41:29.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.690+0000 7fef84f30700 1 --2- 172.21.15.92:0/1037580591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef800b29b0 0x7fef80159120 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fef800d0420 tx=0x7fef70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d1ceab2967eb99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:29.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.690+0000 7fef77fff700 1 -- 172.21.15.92:0/1037580591 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7000b640 con 0x7fef800b29b0 2023-10-06T10:41:29.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.690+0000 7fef77fff700 1 -- 172.21.15.92:0/1037580591 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7000b7a0 con 0x7fef800b29b0 2023-10-06T10:41:29.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.690+0000 7fef77fff700 1 -- 172.21.15.92:0/1037580591 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7000bb60 con 0x7fef800b29b0 2023-10-06T10:41:29.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.691+0000 7fef87995700 1 -- 172.21.15.92:0/1037580591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef800b29b0 msgr2=0x7fef80159120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:29.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.691+0000 7fef87995700 1 --2- 172.21.15.92:0/1037580591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef800b29b0 0x7fef80159120 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fef800d0420 tx=0x7fef70009250 comp rx=0 tx=0).stop 2023-10-06T10:41:29.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.691+0000 7fef87995700 1 -- 172.21.15.92:0/1037580591 shutdown_connections 2023-10-06T10:41:29.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.691+0000 7fef87995700 1 --2- 172.21.15.92:0/1037580591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef80159660 0x7fef80068540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:29.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.691+0000 7fef87995700 1 --2- 172.21.15.92:0/1037580591 >> v2:172.21.15.124:3300/0 conn(0x7fef800b25e0 0x7fef800b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:29.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.691+0000 7fef87995700 1 --2- 172.21.15.92:0/1037580591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef800b29b0 0x7fef80159120 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:29.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.691+0000 7fef87995700 1 -- 172.21.15.92:0/1037580591 >> 172.21.15.92:0/1037580591 conn(0x7fef8006ece0 msgr2=0x7fef8006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:29.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.691+0000 7fef87995700 1 -- 172.21.15.92:0/1037580591 shutdown_connections 2023-10-06T10:41:29.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.691+0000 7fef87995700 1 -- 172.21.15.92:0/1037580591 wait complete. 2023-10-06T10:41:29.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.692+0000 7fef87995700 1 Processor -- start 2023-10-06T10:41:29.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.692+0000 7fef87995700 1 -- start start 2023-10-06T10:41:29.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.693+0000 7fef87995700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef800b25e0 0x7fef8014e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:29.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.693+0000 7fef87995700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef800b29b0 0x7fef800c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.693+0000 7fef87995700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef80159660 0x7fef800c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.693+0000 7fef87995700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8015d660 con 0x7fef800b29b0 2023-10-06T10:41:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.693+0000 7fef87995700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef800c61f0 con 0x7fef800b25e0 2023-10-06T10:41:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.693+0000 7fef87995700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef8015c7d0 con 0x7fef80159660 2023-10-06T10:41:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.693+0000 7fef84f30700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef800b29b0 0x7fef800c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.693+0000 7fef84f30700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef800b29b0 0x7fef800c9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33350/0 (socket says 172.21.15.92:33350) 2023-10-06T10:41:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.693+0000 7fef84f30700 1 -- 172.21.15.92:0/546872507 learned_addr learned my addr 172.21.15.92:0/546872507 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.693+0000 7fef85f32700 1 --2- 172.21.15.92:0/546872507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef80159660 0x7fef800c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.693+0000 7fef85731700 1 --2- 172.21.15.92:0/546872507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef800b25e0 0x7fef8014e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.694+0000 7fef84f30700 1 -- 172.21.15.92:0/546872507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef80159660 msgr2=0x7fef800c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.694+0000 7fef84f30700 1 --2- 172.21.15.92:0/546872507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef80159660 0x7fef800c9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.694+0000 7fef84f30700 1 -- 172.21.15.92:0/546872507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef800b25e0 msgr2=0x7fef8014e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.694+0000 7fef84f30700 1 --2- 172.21.15.92:0/546872507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef800b25e0 0x7fef8014e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.694+0000 7fef84f30700 1 -- 172.21.15.92:0/546872507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef700094f0 con 0x7fef800b29b0 2023-10-06T10:41:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.694+0000 7fef85f32700 1 --2- 172.21.15.92:0/546872507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef80159660 0x7fef800c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:29.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.694+0000 7fef85731700 1 --2- 172.21.15.92:0/546872507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef800b25e0 0x7fef8014e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:29.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.694+0000 7fef84f30700 1 --2- 172.21.15.92:0/546872507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef800b29b0 0x7fef800c9440 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fef70009f70 tx=0x7fef70017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:29.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.694+0000 7fef757fa700 1 -- 172.21.15.92:0/546872507 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70000d10 con 0x7fef800b29b0 2023-10-06T10:41:29.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.694+0000 7fef757fa700 1 -- 172.21.15.92:0/546872507 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7000b910 con 0x7fef800b29b0 2023-10-06T10:41:29.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.694+0000 7fef87995700 1 -- 172.21.15.92:0/546872507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef8015ca50 con 0x7fef800b29b0 2023-10-06T10:41:29.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.694+0000 7fef757fa700 1 -- 172.21.15.92:0/546872507 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70026430 con 0x7fef800b29b0 2023-10-06T10:41:29.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.694+0000 7fef87995700 1 -- 172.21.15.92:0/546872507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8015d010 con 0x7fef800b29b0 2023-10-06T10:41:29.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.695+0000 7fef87995700 1 -- 172.21.15.92:0/546872507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef80064a90 con 0x7fef800b29b0 2023-10-06T10:41:29.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.695+0000 7fef757fa700 1 -- 172.21.15.92:0/546872507 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fef70023030 con 0x7fef800b29b0 2023-10-06T10:41:29.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.696+0000 7fef757fa700 1 --2- 172.21.15.92:0/546872507 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fef64066f30 0x7fef640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:29.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.696+0000 7fef757fa700 1 -- 172.21.15.92:0/546872507 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fef70085f80 con 0x7fef800b29b0 2023-10-06T10:41:29.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.696+0000 7fef85731700 1 --2- 172.21.15.92:0/546872507 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fef64066f30 0x7fef640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:29.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.696+0000 7fef85731700 1 --2- 172.21.15.92:0/546872507 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fef64066f30 0x7fef640693e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009d70 tx=0x7fef7c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:29.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.701+0000 7fef757fa700 1 -- 172.21.15.92:0/546872507 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef700b75a0 con 0x7fef800b29b0 2023-10-06T10:41:29.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.924+0000 7fef87995700 1 -- 172.21.15.92:0/546872507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef800cbc60 con 0x7fef800b29b0 2023-10-06T10:41:29.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.924+0000 7fef757fa700 1 -- 172.21.15.92:0/546872507 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4637 (secure 0 0 0) 0x7fef70052fd0 con 0x7fef800b29b0 2023-10-06T10:41:29.926 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:29.927 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3668157650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3668157650},{"type":"v1","addr":"172.21.15.124:6834","nonce":3668157650}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:40:34.932631+0000","modified":"2023-10-06T10:41:17.396042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5736,"mds_1":5733,"mds_2":5036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5036":{"gid":5036,"name":"d","rank":2,"incarnation":36,"state":"up:active","state_seq":13,"addr":"172.21.15.124:6835/2986058955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2986058955},{"type":"v1","addr":"172.21.15.124:6835","nonce":2986058955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5733":{"gid":5733,"name":"c","rank":1,"incarnation":34,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6836/831923836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":831923836},{"type":"v1","addr":"172.21.15.92:6836","nonce":831923836}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5736":{"gid":5736,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6837/2458340622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2458340622},{"type":"v1","addr":"172.21.15.92:6837","nonce":2458340622}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T10:41:29.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.926+0000 7fef87995700 1 -- 172.21.15.92:0/546872507 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fef64066f30 msgr2=0x7fef640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:29.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.926+0000 7fef87995700 1 --2- 172.21.15.92:0/546872507 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fef64066f30 0x7fef640693e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009d70 tx=0x7fef7c008040 comp rx=0 tx=0).stop 2023-10-06T10:41:29.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.926+0000 7fef87995700 1 -- 172.21.15.92:0/546872507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef800b29b0 msgr2=0x7fef800c9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:29.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.926+0000 7fef87995700 1 --2- 172.21.15.92:0/546872507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef800b29b0 0x7fef800c9440 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fef70009f70 tx=0x7fef70017000 comp rx=0 tx=0).stop 2023-10-06T10:41:29.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.926+0000 7fef87995700 1 -- 172.21.15.92:0/546872507 shutdown_connections 2023-10-06T10:41:29.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.926+0000 7fef87995700 1 --2- 172.21.15.92:0/546872507 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fef64066f30 0x7fef640693e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:29.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.926+0000 7fef87995700 1 --2- 172.21.15.92:0/546872507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef80159660 0x7fef800c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:29.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.926+0000 7fef87995700 1 --2- 172.21.15.92:0/546872507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef800b25e0 0x7fef8014e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:29.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.926+0000 7fef87995700 1 --2- 172.21.15.92:0/546872507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef800b29b0 0x7fef800c9440 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:29.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.926+0000 7fef87995700 1 -- 172.21.15.92:0/546872507 >> 172.21.15.92:0/546872507 conn(0x7fef8006ece0 msgr2=0x7fef800cccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:29.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.926+0000 7fef87995700 1 -- 172.21.15.92:0/546872507 shutdown_connections 2023-10-06T10:41:29.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:29.926+0000 7fef87995700 1 -- 172.21.15.92:0/546872507 wait complete. 2023-10-06T10:41:29.930 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 42 2023-10-06T10:41:29.940 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'd'] 2023-10-06T10:41:29.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:41:30.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.052+0000 7f04584e6700 1 Processor -- start 2023-10-06T10:41:30.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.052+0000 7f04584e6700 1 -- start start 2023-10-06T10:41:30.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.052+0000 7f04584e6700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04500ca370 0x7f0450158270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:30.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.052+0000 7f04584e6700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f04500adb60 0x7f04500ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:30.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.052+0000 7f04584e6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0450157de0 con 0x7f04500ce6d0 2023-10-06T10:41:30.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.052+0000 7f04584e6700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04500cbb90 con 0x7f04500adb60 2023-10-06T10:41:30.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.052+0000 7f04584e6700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04500ceaf0 con 0x7f04500ca370 2023-10-06T10:41:30.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.052+0000 7f0456a83700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f04500ce6d0 0x7f045015aed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51104/0 (socket says 172.21.15.92:51104) 2023-10-06T10:41:30.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.052+0000 7f0456a83700 1 -- 172.21.15.92:0/2538672864 learned_addr learned my addr 172.21.15.92:0/2538672864 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:30.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.052+0000 7f0456282700 1 --2- 172.21.15.92:0/2538672864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04500ca370 0x7f0450158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:30.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.052+0000 7f0455a81700 1 --2- 172.21.15.92:0/2538672864 >> v2:172.21.15.124:3300/0 conn(0x7f04500adb60 0x7f04500ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:30.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.052+0000 7f0455280700 1 -- 172.21.15.92:0/2538672864 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1668906336 0 0) 0x7f0450157de0 con 0x7f04500ce6d0 2023-10-06T10:41:30.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.052+0000 7f0455280700 1 -- 172.21.15.92:0/2538672864 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f043c003570 con 0x7f04500ce6d0 2023-10-06T10:41:30.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.052+0000 7f0455280700 1 -- 172.21.15.92:0/2538672864 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2781438886 0 0) 0x7f043c003570 con 0x7f04500ce6d0 2023-10-06T10:41:30.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.052+0000 7f0456282700 1 -- 172.21.15.92:0/2538672864 >> v2:172.21.15.124:3300/0 conn(0x7f04500adb60 msgr2=0x7f04500ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:30.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.052+0000 7f0456282700 1 --2- 172.21.15.92:0/2538672864 >> v2:172.21.15.124:3300/0 conn(0x7f04500adb60 0x7f04500ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:30.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.052+0000 7f0456282700 1 -- 172.21.15.92:0/2538672864 >> v1:172.21.15.92:6789/0 conn(0x7f04500ce6d0 legacy=0x7f045015aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:30.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.052+0000 7f0456282700 1 -- 172.21.15.92:0/2538672864 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04500cb5d0 con 0x7f04500ca370 2023-10-06T10:41:30.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.053+0000 7f0456282700 1 --2- 172.21.15.92:0/2538672864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04500ca370 0x7f0450158270 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f044c0098e0 tx=0x7f044c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41756c598264681d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:30.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.053+0000 7f0455280700 1 -- 172.21.15.92:0/2538672864 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04480017f0 con 0x7f04500ca370 2023-10-06T10:41:30.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.053+0000 7f0455280700 1 -- 172.21.15.92:0/2538672864 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044c011420 con 0x7f04500ca370 2023-10-06T10:41:30.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.053+0000 7f0455280700 1 -- 172.21.15.92:0/2538672864 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c0117c0 con 0x7f04500ca370 2023-10-06T10:41:30.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.053+0000 7f04584e6700 1 -- 172.21.15.92:0/2538672864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04500ca370 msgr2=0x7f0450158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:30.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.053+0000 7f04584e6700 1 --2- 172.21.15.92:0/2538672864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04500ca370 0x7f0450158270 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f044c0098e0 tx=0x7f044c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:30.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.053+0000 7f04584e6700 1 -- 172.21.15.92:0/2538672864 shutdown_connections 2023-10-06T10:41:30.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.053+0000 7f04584e6700 1 --2- 172.21.15.92:0/2538672864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04500ca370 0x7f0450158270 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:30.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.053+0000 7f04584e6700 1 --2- 172.21.15.92:0/2538672864 >> v2:172.21.15.124:3300/0 conn(0x7f04500adb60 0x7f04500ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:30.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.053+0000 7f04584e6700 1 -- 172.21.15.92:0/2538672864 >> 172.21.15.92:0/2538672864 conn(0x7f04500707d0 msgr2=0x7f0450070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:30.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.053+0000 7f04584e6700 1 -- 172.21.15.92:0/2538672864 shutdown_connections 2023-10-06T10:41:30.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.053+0000 7f04584e6700 1 -- 172.21.15.92:0/2538672864 wait complete. 2023-10-06T10:41:30.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.054+0000 7f04584e6700 1 Processor -- start 2023-10-06T10:41:30.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.054+0000 7f04584e6700 1 -- start start 2023-10-06T10:41:30.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.055+0000 7f04584e6700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04500adb60 0x7f045014d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:30.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.055+0000 7f04584e6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04500ca370 0x7f04500c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:30.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.055+0000 7f04584e6700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04500ce6d0 0x7f04500c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:30.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.055+0000 7f04584e6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0450154b80 con 0x7f04500ca370 2023-10-06T10:41:30.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.055+0000 7f04584e6700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0450069cb0 con 0x7f04500ce6d0 2023-10-06T10:41:30.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.055+0000 7f04584e6700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0450069f90 con 0x7f04500adb60 2023-10-06T10:41:30.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.055+0000 7f0455a81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04500ca370 0x7f04500c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:30.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.055+0000 7f0455a81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04500ca370 0x7f04500c96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33358/0 (socket says 172.21.15.92:33358) 2023-10-06T10:41:30.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.055+0000 7f0455a81700 1 -- 172.21.15.92:0/2279061077 learned_addr learned my addr 172.21.15.92:0/2279061077 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:30.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.055+0000 7f0456a83700 1 --2- 172.21.15.92:0/2279061077 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04500ce6d0 0x7f04500c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:30.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.055+0000 7f0456282700 1 --2- 172.21.15.92:0/2279061077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04500adb60 0x7f045014d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:30.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.056+0000 7f0455a81700 1 -- 172.21.15.92:0/2279061077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04500adb60 msgr2=0x7f045014d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:30.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.056+0000 7f0455a81700 1 --2- 172.21.15.92:0/2279061077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04500adb60 0x7f045014d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:30.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.056+0000 7f0455a81700 1 -- 172.21.15.92:0/2279061077 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04500ce6d0 msgr2=0x7f04500c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:30.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.056+0000 7f0455a81700 1 --2- 172.21.15.92:0/2279061077 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04500ce6d0 0x7f04500c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:30.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.056+0000 7f0455a81700 1 -- 172.21.15.92:0/2279061077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044c009420 con 0x7f04500ca370 2023-10-06T10:41:30.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.056+0000 7f0455a81700 1 --2- 172.21.15.92:0/2279061077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04500ca370 0x7f04500c96f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f044000e930 tx=0x7f04400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:30.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.056+0000 7f04467fc700 1 -- 172.21.15.92:0/2279061077 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044000eed0 con 0x7f04500ca370 2023-10-06T10:41:30.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.056+0000 7f04467fc700 1 -- 172.21.15.92:0/2279061077 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044000c850 con 0x7f04500ca370 2023-10-06T10:41:30.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.056+0000 7f04584e6700 1 -- 172.21.15.92:0/2279061077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f045006a270 con 0x7f04500ca370 2023-10-06T10:41:30.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.056+0000 7f04467fc700 1 -- 172.21.15.92:0/2279061077 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044000cbf0 con 0x7f04500ca370 2023-10-06T10:41:30.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.056+0000 7f04584e6700 1 -- 172.21.15.92:0/2279061077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04500b1360 con 0x7f04500ca370 2023-10-06T10:41:30.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.057+0000 7f04584e6700 1 -- 172.21.15.92:0/2279061077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04500b1610 con 0x7f04500ca370 2023-10-06T10:41:30.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.057+0000 7f04467fc700 1 -- 172.21.15.92:0/2279061077 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0440013470 con 0x7f04500ca370 2023-10-06T10:41:30.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.057+0000 7f04467fc700 1 --2- 172.21.15.92:0/2279061077 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0434066f30 0x7f04340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:30.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.058+0000 7f04467fc700 1 -- 172.21.15.92:0/2279061077 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0440084a40 con 0x7f04500ca370 2023-10-06T10:41:30.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.058+0000 7f0456282700 1 --2- 172.21.15.92:0/2279061077 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0434066f30 0x7f04340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:30.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.058+0000 7f0456282700 1 --2- 172.21.15.92:0/2279061077 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0434066f30 0x7f04340693e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f044c0030c0 tx=0x7f044c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:30.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.062+0000 7f04467fc700 1 -- 172.21.15.92:0/2279061077 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04500b1610 con 0x7f04500ca370 2023-10-06T10:41:30.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.319+0000 7f04584e6700 1 -- 172.21.15.92:0/2279061077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f04500b1610 con 0x7f04500ca370 2023-10-06T10:41:30.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.319+0000 7f04467fc700 1 -- 172.21.15.92:0/2279061077 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1921 (secure 0 0 0) 0x7f04500b1610 con 0x7f04500ca370 2023-10-06T10:41:30.321 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:30.321 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":211,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":52,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":704416,"bytes_used":2441891840,"bytes_avail":770652221440,"bytes_total":773094113280,"write_bytes_sec":682,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":42,"id":3,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":3,"rank":2,"name":"d","status":"up:active","gid":5036},{"filesystem_id":3,"rank":1,"name":"c","status":"up:active","gid":5733},{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5736}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2023-10-06T10:41:26.124133+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T10:41:30.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.321+0000 7f04584e6700 1 -- 172.21.15.92:0/2279061077 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0434066f30 msgr2=0x7f04340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:30.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.321+0000 7f04584e6700 1 --2- 172.21.15.92:0/2279061077 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0434066f30 0x7f04340693e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f044c0030c0 tx=0x7f044c016000 comp rx=0 tx=0).stop 2023-10-06T10:41:30.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.321+0000 7f04584e6700 1 -- 172.21.15.92:0/2279061077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04500ca370 msgr2=0x7f04500c96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:30.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.321+0000 7f04584e6700 1 --2- 172.21.15.92:0/2279061077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04500ca370 0x7f04500c96f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f044000e930 tx=0x7f04400091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:30.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.321+0000 7f04584e6700 1 -- 172.21.15.92:0/2279061077 shutdown_connections 2023-10-06T10:41:30.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.321+0000 7f04584e6700 1 --2- 172.21.15.92:0/2279061077 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0434066f30 0x7f04340693e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:30.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.321+0000 7f04584e6700 1 --2- 172.21.15.92:0/2279061077 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04500ce6d0 0x7f04500c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:30.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.321+0000 7f04584e6700 1 --2- 172.21.15.92:0/2279061077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04500adb60 0x7f045014d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:30.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.321+0000 7f04584e6700 1 --2- 172.21.15.92:0/2279061077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04500ca370 0x7f04500c96f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:30.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.321+0000 7f04584e6700 1 -- 172.21.15.92:0/2279061077 >> 172.21.15.92:0/2279061077 conn(0x7f04500707d0 msgr2=0x7f0450155df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:30.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.321+0000 7f04584e6700 1 -- 172.21.15.92:0/2279061077 shutdown_connections 2023-10-06T10:41:30.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:30.322+0000 7f04584e6700 1 -- 172.21.15.92:0/2279061077 wait complete. 2023-10-06T10:41:30.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]'} 2023-10-06T10:41:31.334 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:41:31.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.453+0000 7f634191c700 1 Processor -- start 2023-10-06T10:41:31.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.453+0000 7f634191c700 1 -- start start 2023-10-06T10:41:31.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.454+0000 7f634191c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f633c0ac000 0x7f633c069520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:31.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.454+0000 7f634191c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f633c069a60 0x7f633c0c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:31.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.454+0000 7f634191c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633c0c7330 con 0x7f633c0adf20 2023-10-06T10:41:31.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.454+0000 7f634191c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633c156110 con 0x7f633c0ac000 2023-10-06T10:41:31.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.454+0000 7f634191c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633c069ee0 con 0x7f633c069a60 2023-10-06T10:41:31.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.454+0000 7f63327fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f633c0ac000 0x7f633c069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:31.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.454+0000 7f633affd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f633c0adf20 0x7f633c06e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51108/0 (socket says 172.21.15.92:51108) 2023-10-06T10:41:31.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.454+0000 7f63327fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f633c0ac000 0x7f633c069520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33374/0 (socket says 172.21.15.92:33374) 2023-10-06T10:41:31.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.454+0000 7f633affd700 1 -- 172.21.15.92:0/4292876129 learned_addr learned my addr 172.21.15.92:0/4292876129 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:31.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.454+0000 7f633b7fe700 1 --2- 172.21.15.92:0/4292876129 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f633c069a60 0x7f633c0c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:31.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.455+0000 7f63327fc700 1 -- 172.21.15.92:0/4292876129 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f633c069a60 msgr2=0x7f633c0c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:31.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.455+0000 7f63327fc700 1 --2- 172.21.15.92:0/4292876129 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f633c069a60 0x7f633c0c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:31.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.455+0000 7f63327fc700 1 -- 172.21.15.92:0/4292876129 >> v1:172.21.15.92:6789/0 conn(0x7f633c0adf20 legacy=0x7f633c06e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:31.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.455+0000 7f63327fc700 1 -- 172.21.15.92:0/4292876129 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c0c6720 con 0x7f633c0ac000 2023-10-06T10:41:31.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.455+0000 7f633a7fc700 1 -- 172.21.15.92:0/4292876129 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 662767624 0 0) 0x7f633c0c7330 con 0x7f633c0adf20 2023-10-06T10:41:31.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.455+0000 7f633b7fe700 1 --2- 172.21.15.92:0/4292876129 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f633c069a60 0x7f633c0c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:31.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.455+0000 7f63327fc700 1 --2- 172.21.15.92:0/4292876129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f633c0ac000 0x7f633c069520 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f632400bbc0 tx=0x7f632400b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8039c3a79fd393fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:31.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.455+0000 7f633a7fc700 1 -- 172.21.15.92:0/4292876129 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632400da80 con 0x7f633c0ac000 2023-10-06T10:41:31.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.455+0000 7f633a7fc700 1 -- 172.21.15.92:0/4292876129 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632400dbe0 con 0x7f633c0ac000 2023-10-06T10:41:31.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.455+0000 7f633a7fc700 1 -- 172.21.15.92:0/4292876129 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324017410 con 0x7f633c0ac000 2023-10-06T10:41:31.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.455+0000 7f634191c700 1 -- 172.21.15.92:0/4292876129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f633c0ac000 msgr2=0x7f633c069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:31.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.455+0000 7f634191c700 1 --2- 172.21.15.92:0/4292876129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f633c0ac000 0x7f633c069520 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f632400bbc0 tx=0x7f632400b430 comp rx=0 tx=0).stop 2023-10-06T10:41:31.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.455+0000 7f634191c700 1 -- 172.21.15.92:0/4292876129 shutdown_connections 2023-10-06T10:41:31.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.455+0000 7f634191c700 1 --2- 172.21.15.92:0/4292876129 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f633c069a60 0x7f633c0c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:31.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.455+0000 7f634191c700 1 --2- 172.21.15.92:0/4292876129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f633c0ac000 0x7f633c069520 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:31.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.455+0000 7f634191c700 1 -- 172.21.15.92:0/4292876129 >> 172.21.15.92:0/4292876129 conn(0x7f633c068080 msgr2=0x7f633c0cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:31.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.455+0000 7f634191c700 1 -- 172.21.15.92:0/4292876129 shutdown_connections 2023-10-06T10:41:31.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.455+0000 7f634191c700 1 -- 172.21.15.92:0/4292876129 wait complete. 2023-10-06T10:41:31.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.457+0000 7f634191c700 1 Processor -- start 2023-10-06T10:41:31.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.457+0000 7f634191c700 1 -- start start 2023-10-06T10:41:31.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.458+0000 7f634191c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f633c069a60 0x7f633c0b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:31.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.458+0000 7f634191c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f633c0ac000 0x7f633c159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:31.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.458+0000 7f634191c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f633c0adf20 0x7f633c1689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:31.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.458+0000 7f634191c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c158f70 con 0x7f633c0adf20 2023-10-06T10:41:31.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.458+0000 7f634191c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c1694c0 con 0x7f633c0ac000 2023-10-06T10:41:31.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.458+0000 7f634191c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633c1697a0 con 0x7f633c069a60 2023-10-06T10:41:31.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.458+0000 7f633b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f633c0adf20 0x7f633c1689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:31.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.458+0000 7f633b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f633c0adf20 0x7f633c1689e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33378/0 (socket says 172.21.15.92:33378) 2023-10-06T10:41:31.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.458+0000 7f633b7fe700 1 -- 172.21.15.92:0/1736172717 learned_addr learned my addr 172.21.15.92:0/1736172717 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:31.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.458+0000 7f633affd700 1 --2- 172.21.15.92:0/1736172717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f633c069a60 0x7f633c0b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:31.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.458+0000 7f63327fc700 1 --2- 172.21.15.92:0/1736172717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f633c0ac000 0x7f633c159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:31.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.458+0000 7f633b7fe700 1 -- 172.21.15.92:0/1736172717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f633c069a60 msgr2=0x7f633c0b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:31.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.458+0000 7f633b7fe700 1 --2- 172.21.15.92:0/1736172717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f633c069a60 0x7f633c0b00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:31.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.458+0000 7f633b7fe700 1 -- 172.21.15.92:0/1736172717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f633c0ac000 msgr2=0x7f633c159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:31.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.458+0000 7f633b7fe700 1 --2- 172.21.15.92:0/1736172717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f633c0ac000 0x7f633c159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:31.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.458+0000 7f633b7fe700 1 -- 172.21.15.92:0/1736172717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632400b6d0 con 0x7f633c0adf20 2023-10-06T10:41:31.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.458+0000 7f633affd700 1 --2- 172.21.15.92:0/1736172717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f633c069a60 0x7f633c0b00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:31.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.458+0000 7f633b7fe700 1 --2- 172.21.15.92:0/1736172717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f633c0adf20 0x7f633c1689e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f633400d180 tx=0x7f6334009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:31.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.459+0000 7f63337fe700 1 -- 172.21.15.92:0/1736172717 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633400de20 con 0x7f633c0adf20 2023-10-06T10:41:31.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.459+0000 7f63337fe700 1 -- 172.21.15.92:0/1736172717 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633400f040 con 0x7f633c0adf20 2023-10-06T10:41:31.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.459+0000 7f63337fe700 1 -- 172.21.15.92:0/1736172717 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6334015690 con 0x7f633c0adf20 2023-10-06T10:41:31.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.459+0000 7f634191c700 1 -- 172.21.15.92:0/1736172717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633c158c00 con 0x7f633c0adf20 2023-10-06T10:41:31.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.459+0000 7f634191c700 1 -- 172.21.15.92:0/1736172717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633c158840 con 0x7f633c0adf20 2023-10-06T10:41:31.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.460+0000 7f634191c700 1 -- 172.21.15.92:0/1736172717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633c1590e0 con 0x7f633c0adf20 2023-10-06T10:41:31.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.460+0000 7f63337fe700 1 -- 172.21.15.92:0/1736172717 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f633401e3b0 con 0x7f633c0adf20 2023-10-06T10:41:31.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.460+0000 7f63337fe700 1 --2- 172.21.15.92:0/1736172717 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f631c066f00 0x7f631c0693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:31.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.460+0000 7f633affd700 1 --2- 172.21.15.92:0/1736172717 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f631c066f00 0x7f631c0693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:31.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.460+0000 7f63337fe700 1 -- 172.21.15.92:0/1736172717 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6334084de0 con 0x7f633c0adf20 2023-10-06T10:41:31.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.461+0000 7f633affd700 1 --2- 172.21.15.92:0/1736172717 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f631c066f00 0x7f631c0693b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f632c001790 tx=0x7f632c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:31.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.465+0000 7f63337fe700 1 -- 172.21.15.92:0/1736172717 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6334051db0 con 0x7f633c0adf20 2023-10-06T10:41:31.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.697+0000 7f634191c700 1 -- 172.21.15.92:0/1736172717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f633c064a90 con 0x7f633c0adf20 2023-10-06T10:41:31.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.698+0000 7f63337fe700 1 -- 172.21.15.92:0/1736172717 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4637 (secure 0 0 0) 0x7f633401a090 con 0x7f633c0adf20 2023-10-06T10:41:31.699 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:31.699 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3668157650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3668157650},{"type":"v1","addr":"172.21.15.124:6834","nonce":3668157650}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:40:34.932631+0000","modified":"2023-10-06T10:41:17.396042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5736,"mds_1":5733,"mds_2":5036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5036":{"gid":5036,"name":"d","rank":2,"incarnation":36,"state":"up:active","state_seq":13,"addr":"172.21.15.124:6835/2986058955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2986058955},{"type":"v1","addr":"172.21.15.124:6835","nonce":2986058955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5733":{"gid":5733,"name":"c","rank":1,"incarnation":34,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6836/831923836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":831923836},{"type":"v1","addr":"172.21.15.92:6836","nonce":831923836}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5736":{"gid":5736,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6837/2458340622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2458340622},{"type":"v1","addr":"172.21.15.92:6837","nonce":2458340622}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T10:41:31.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.700+0000 7f634191c700 1 -- 172.21.15.92:0/1736172717 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f631c066f00 msgr2=0x7f631c0693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:31.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.700+0000 7f634191c700 1 --2- 172.21.15.92:0/1736172717 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f631c066f00 0x7f631c0693b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f632c001790 tx=0x7f632c009040 comp rx=0 tx=0).stop 2023-10-06T10:41:31.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.700+0000 7f634191c700 1 -- 172.21.15.92:0/1736172717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f633c0adf20 msgr2=0x7f633c1689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:31.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.700+0000 7f634191c700 1 --2- 172.21.15.92:0/1736172717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f633c0adf20 0x7f633c1689e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f633400d180 tx=0x7f6334009a50 comp rx=0 tx=0).stop 2023-10-06T10:41:31.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.700+0000 7f634191c700 1 -- 172.21.15.92:0/1736172717 shutdown_connections 2023-10-06T10:41:31.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.700+0000 7f634191c700 1 --2- 172.21.15.92:0/1736172717 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f631c066f00 0x7f631c0693b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:31.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.700+0000 7f634191c700 1 --2- 172.21.15.92:0/1736172717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f633c0adf20 0x7f633c1689e0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:31.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.700+0000 7f634191c700 1 --2- 172.21.15.92:0/1736172717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f633c069a60 0x7f633c0b00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:31.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.700+0000 7f634191c700 1 --2- 172.21.15.92:0/1736172717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f633c0ac000 0x7f633c159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:31.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.700+0000 7f634191c700 1 -- 172.21.15.92:0/1736172717 >> 172.21.15.92:0/1736172717 conn(0x7f633c068080 msgr2=0x7f633c06d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:31.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.700+0000 7f634191c700 1 -- 172.21.15.92:0/1736172717 shutdown_connections 2023-10-06T10:41:31.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.700+0000 7f634191c700 1 -- 172.21.15.92:0/1736172717 wait complete. 2023-10-06T10:41:31.702 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 42 2023-10-06T10:41:31.713 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'd'] 2023-10-06T10:41:31.714 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:41:31.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.827+0000 7f2852211700 1 Processor -- start 2023-10-06T10:41:31.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.827+0000 7f2852211700 1 -- start start 2023-10-06T10:41:31.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.828+0000 7f2852211700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f284c159560 0x7f284c0aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:31.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.828+0000 7f2852211700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f284c0b0760 0x7f284c0b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:31.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.828+0000 7f2852211700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f284c0b0c60 con 0x7f284c0b49a0 2023-10-06T10:41:31.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.828+0000 7f2852211700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f284c0b0f40 con 0x7f284c159560 2023-10-06T10:41:31.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.828+0000 7f2852211700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284c0b1220 con 0x7f284c0b0760 2023-10-06T10:41:31.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.828+0000 7f284b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f284c159560 0x7f284c0aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:31.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.828+0000 7f284b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f284c159560 0x7f284c0aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33384/0 (socket says 172.21.15.92:33384) 2023-10-06T10:41:31.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.828+0000 7f284b7fe700 1 -- 172.21.15.92:0/2686222126 learned_addr learned my addr 172.21.15.92:0/2686222126 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:31.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.828+0000 7f284bfff700 1 --2- 172.21.15.92:0/2686222126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f284c0b0760 0x7f284c0b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:31.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.828+0000 7f284b7fe700 1 -- 172.21.15.92:0/2686222126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f284c0b0760 msgr2=0x7f284c0b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:31.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.828+0000 7f284b7fe700 1 --2- 172.21.15.92:0/2686222126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f284c0b0760 0x7f284c0b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:31.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.828+0000 7f284b7fe700 1 -- 172.21.15.92:0/2686222126 >> v1:172.21.15.124:6789/0 conn(0x7f284c0b49a0 legacy=0x7f284c0aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:31.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.828+0000 7f284b7fe700 1 -- 172.21.15.92:0/2686222126 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284c0b1500 con 0x7f284c159560 2023-10-06T10:41:31.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.828+0000 7f284bfff700 1 --2- 172.21.15.92:0/2686222126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f284c0b0760 0x7f284c0b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:31.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.829+0000 7f284b7fe700 1 --2- 172.21.15.92:0/2686222126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f284c159560 0x7f284c0aa530 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f283c0099c0 tx=0x7f283c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75fbfb680c7ea554 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:31.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.829+0000 7f284a7fc700 1 -- 172.21.15.92:0/2686222126 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c005c10 con 0x7f284c159560 2023-10-06T10:41:31.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.829+0000 7f284a7fc700 1 -- 172.21.15.92:0/2686222126 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283c005d70 con 0x7f284c159560 2023-10-06T10:41:31.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.829+0000 7f284a7fc700 1 -- 172.21.15.92:0/2686222126 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c00b840 con 0x7f284c159560 2023-10-06T10:41:31.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.829+0000 7f2852211700 1 -- 172.21.15.92:0/2686222126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f284c159560 msgr2=0x7f284c0aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:31.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.829+0000 7f2852211700 1 --2- 172.21.15.92:0/2686222126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f284c159560 0x7f284c0aa530 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f283c0099c0 tx=0x7f283c009220 comp rx=0 tx=0).stop 2023-10-06T10:41:31.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.829+0000 7f2852211700 1 -- 172.21.15.92:0/2686222126 shutdown_connections 2023-10-06T10:41:31.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.829+0000 7f2852211700 1 --2- 172.21.15.92:0/2686222126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f284c0b0760 0x7f284c0b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:31.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.829+0000 7f2852211700 1 --2- 172.21.15.92:0/2686222126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f284c159560 0x7f284c0aa530 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:31.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.829+0000 7f2852211700 1 -- 172.21.15.92:0/2686222126 >> 172.21.15.92:0/2686222126 conn(0x7f284c0997f0 msgr2=0x7f284c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:31.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.829+0000 7f2852211700 1 -- 172.21.15.92:0/2686222126 shutdown_connections 2023-10-06T10:41:31.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.829+0000 7f2852211700 1 -- 172.21.15.92:0/2686222126 wait complete. 2023-10-06T10:41:31.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.830+0000 7f2852211700 1 Processor -- start 2023-10-06T10:41:31.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.830+0000 7f2852211700 1 -- start start 2023-10-06T10:41:31.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.831+0000 7f2852211700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f284c0b0760 0x7f284c16a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:31.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.831+0000 7f2852211700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f284c0b49a0 0x7f284c0b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:31.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.831+0000 7f2852211700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f284c159560 0x7f284c14d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:31.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.831+0000 7f2852211700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c16aa00 con 0x7f284c0b49a0 2023-10-06T10:41:31.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.831+0000 7f2852211700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c0aef50 con 0x7f284c0b0760 2023-10-06T10:41:31.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.831+0000 7f2852211700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284c16ad40 con 0x7f284c159560 2023-10-06T10:41:31.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.831+0000 7f284affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f284c0b49a0 0x7f284c0b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:31.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.831+0000 7f284affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f284c0b49a0 0x7f284c0b1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33396/0 (socket says 172.21.15.92:33396) 2023-10-06T10:41:31.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.831+0000 7f284affd700 1 -- 172.21.15.92:0/3205004455 learned_addr learned my addr 172.21.15.92:0/3205004455 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:31.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.831+0000 7f284bfff700 1 --2- 172.21.15.92:0/3205004455 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f284c159560 0x7f284c14d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:31.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.831+0000 7f284b7fe700 1 --2- 172.21.15.92:0/3205004455 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f284c0b0760 0x7f284c16a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:31.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.832+0000 7f284affd700 1 -- 172.21.15.92:0/3205004455 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f284c159560 msgr2=0x7f284c14d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:31.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.832+0000 7f284affd700 1 --2- 172.21.15.92:0/3205004455 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f284c159560 0x7f284c14d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:31.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.832+0000 7f284affd700 1 -- 172.21.15.92:0/3205004455 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f284c0b0760 msgr2=0x7f284c16a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:31.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.832+0000 7f284affd700 1 --2- 172.21.15.92:0/3205004455 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f284c0b0760 0x7f284c16a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:31.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.832+0000 7f284affd700 1 -- 172.21.15.92:0/3205004455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283c009480 con 0x7f284c0b49a0 2023-10-06T10:41:31.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.832+0000 7f284b7fe700 1 --2- 172.21.15.92:0/3205004455 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f284c0b0760 0x7f284c16a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:31.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.832+0000 7f284affd700 1 --2- 172.21.15.92:0/3205004455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f284c0b49a0 0x7f284c0b1da0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f284000c790 tx=0x7f2840007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:31.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.832+0000 7f28337fe700 1 -- 172.21.15.92:0/3205004455 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840011590 con 0x7f284c0b49a0 2023-10-06T10:41:31.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.832+0000 7f28337fe700 1 -- 172.21.15.92:0/3205004455 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28400116f0 con 0x7f284c0b49a0 2023-10-06T10:41:31.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.832+0000 7f2852211700 1 -- 172.21.15.92:0/3205004455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f284c16b020 con 0x7f284c0b49a0 2023-10-06T10:41:31.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.832+0000 7f28337fe700 1 -- 172.21.15.92:0/3205004455 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840011ab0 con 0x7f284c0b49a0 2023-10-06T10:41:31.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.832+0000 7f2852211700 1 -- 172.21.15.92:0/3205004455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f284c157ca0 con 0x7f284c0b49a0 2023-10-06T10:41:31.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.833+0000 7f2852211700 1 -- 172.21.15.92:0/3205004455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f284c16aba0 con 0x7f284c0b49a0 2023-10-06T10:41:31.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.833+0000 7f28337fe700 1 -- 172.21.15.92:0/3205004455 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f284001b3b0 con 0x7f284c0b49a0 2023-10-06T10:41:31.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.833+0000 7f28337fe700 1 --2- 172.21.15.92:0/3205004455 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2838066f30 0x7f28380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:31.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.834+0000 7f28337fe700 1 -- 172.21.15.92:0/3205004455 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2840083c60 con 0x7f284c0b49a0 2023-10-06T10:41:31.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.834+0000 7f284b7fe700 1 --2- 172.21.15.92:0/3205004455 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2838066f30 0x7f28380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:31.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.834+0000 7f284b7fe700 1 --2- 172.21.15.92:0/3205004455 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2838066f30 0x7f28380693e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f283c009260 tx=0x7f283c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:31.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:31.838+0000 7f28337fe700 1 -- 172.21.15.92:0/3205004455 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f284c16aba0 con 0x7f284c0b49a0 2023-10-06T10:41:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.095+0000 7f2852211700 1 -- 172.21.15.92:0/3205004455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f284c16aba0 con 0x7f284c0b49a0 2023-10-06T10:41:32.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.095+0000 7f28337fe700 1 -- 172.21.15.92:0/3205004455 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1699 (secure 0 0 0) 0x7f284c16aba0 con 0x7f284c0b49a0 2023-10-06T10:41:32.096 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:32.096 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":213,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":52,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":710079,"bytes_used":2441900032,"bytes_avail":770652213248,"bytes_total":773094113280,"write_bytes_sec":1279,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":42,"id":3,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":3,"rank":2,"name":"d","status":"up:active","gid":5036},{"filesystem_id":3,"rank":1,"name":"c","status":"up:active","gid":5733},{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5736}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2023-10-06T10:41:30.124957+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T10:41:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.098+0000 7f2852211700 1 -- 172.21.15.92:0/3205004455 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2838066f30 msgr2=0x7f28380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.098+0000 7f2852211700 1 --2- 172.21.15.92:0/3205004455 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2838066f30 0x7f28380693e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f283c009260 tx=0x7f283c016000 comp rx=0 tx=0).stop 2023-10-06T10:41:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.098+0000 7f2852211700 1 -- 172.21.15.92:0/3205004455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f284c0b49a0 msgr2=0x7f284c0b1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.098+0000 7f2852211700 1 --2- 172.21.15.92:0/3205004455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f284c0b49a0 0x7f284c0b1da0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f284000c790 tx=0x7f2840007f20 comp rx=0 tx=0).stop 2023-10-06T10:41:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.098+0000 7f2852211700 1 -- 172.21.15.92:0/3205004455 shutdown_connections 2023-10-06T10:41:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.098+0000 7f2852211700 1 --2- 172.21.15.92:0/3205004455 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2838066f30 0x7f28380693e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.098+0000 7f2852211700 1 --2- 172.21.15.92:0/3205004455 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f284c159560 0x7f284c14d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.098+0000 7f2852211700 1 --2- 172.21.15.92:0/3205004455 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f284c0b0760 0x7f284c16a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.098+0000 7f2852211700 1 --2- 172.21.15.92:0/3205004455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f284c0b49a0 0x7f284c0b1da0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.098+0000 7f2852211700 1 -- 172.21.15.92:0/3205004455 >> 172.21.15.92:0/3205004455 conn(0x7f284c0997f0 msgr2=0x7f284c158960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.098+0000 7f2852211700 1 -- 172.21.15.92:0/3205004455 shutdown_connections 2023-10-06T10:41:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.098+0000 7f2852211700 1 -- 172.21.15.92:0/3205004455 wait complete. 2023-10-06T10:41:32.110 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2023-10-06T10:41:32.111 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T10:41:32.111 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:41:32.148 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:41:32.190 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T10:41:32.190 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T10:41:32.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:41:32.190 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T10:41:32.233 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T10:41:32.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:41:32.233 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T10:41:32.288 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:41:32.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.443+0000 7f2145a0a700 1 Processor -- start 2023-10-06T10:41:32.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.443+0000 7f2145a0a700 1 -- start start 2023-10-06T10:41:32.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.443+0000 7f2145a0a700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f21400ca370 0x7f2140158270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:32.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.444+0000 7f2145a0a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2140157de0 con 0x7f21400adb60 2023-10-06T10:41:32.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.444+0000 7f2145a0a700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21400cb5d0 con 0x7f21400ce8a0 2023-10-06T10:41:32.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.444+0000 7f2145a0a700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21400cbb90 con 0x7f21400ca370 2023-10-06T10:41:32.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.444+0000 7f213e7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f21400adb60 0x7f21400ce190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51118/0 (socket says 172.21.15.92:51118) 2023-10-06T10:41:32.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.444+0000 7f213e7fc700 1 -- 172.21.15.92:0/2832308524 learned_addr learned my addr 172.21.15.92:0/2832308524 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:32.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.444+0000 7f213effd700 1 --2- 172.21.15.92:0/2832308524 >> v2:172.21.15.124:3300/0 conn(0x7f21400ca370 0x7f2140158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:32.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.444+0000 7f213dffb700 1 -- 172.21.15.92:0/2832308524 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1076795904 0 0) 0x7f2140157de0 con 0x7f21400adb60 2023-10-06T10:41:32.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.444+0000 7f213dffb700 1 -- 172.21.15.92:0/2832308524 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2128003570 con 0x7f21400adb60 2023-10-06T10:41:32.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.444+0000 7f213effd700 1 -- 172.21.15.92:0/2832308524 >> v1:172.21.15.124:6789/0 conn(0x7f21400ce8a0 legacy=0x7f214015aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:32.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.444+0000 7f213effd700 1 -- 172.21.15.92:0/2832308524 >> v1:172.21.15.92:6789/0 conn(0x7f21400adb60 legacy=0x7f21400ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:32.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.444+0000 7f213dffb700 1 -- 172.21.15.92:0/2832308524 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3886171952 0 0) 0x7f2128003570 con 0x7f21400adb60 2023-10-06T10:41:32.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.444+0000 7f213effd700 1 -- 172.21.15.92:0/2832308524 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21400cb8b0 con 0x7f21400ca370 2023-10-06T10:41:32.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.444+0000 7f213effd700 1 --2- 172.21.15.92:0/2832308524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f21400ca370 0x7f2140158270 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f21300098e0 tx=0x7f21300091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=728ff65048aa2287 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:32.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.445+0000 7f213dffb700 1 -- 172.21.15.92:0/2832308524 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2134001470 con 0x7f21400ca370 2023-10-06T10:41:32.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.445+0000 7f213dffb700 1 -- 172.21.15.92:0/2832308524 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2130011420 con 0x7f21400ca370 2023-10-06T10:41:32.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.445+0000 7f213dffb700 1 -- 172.21.15.92:0/2832308524 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21300117c0 con 0x7f21400ca370 2023-10-06T10:41:32.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.446+0000 7f2145a0a700 1 -- 172.21.15.92:0/2832308524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f21400ca370 msgr2=0x7f2140158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:32.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.446+0000 7f2145a0a700 1 --2- 172.21.15.92:0/2832308524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f21400ca370 0x7f2140158270 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f21300098e0 tx=0x7f21300091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:32.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.446+0000 7f2145a0a700 1 -- 172.21.15.92:0/2832308524 shutdown_connections 2023-10-06T10:41:32.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.446+0000 7f2145a0a700 1 --2- 172.21.15.92:0/2832308524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f21400ca370 0x7f2140158270 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:32.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.446+0000 7f2145a0a700 1 -- 172.21.15.92:0/2832308524 >> 172.21.15.92:0/2832308524 conn(0x7f21400707d0 msgr2=0x7f2140070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:32.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.446+0000 7f2145a0a700 1 -- 172.21.15.92:0/2832308524 shutdown_connections 2023-10-06T10:41:32.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.446+0000 7f2145a0a700 1 -- 172.21.15.92:0/2832308524 wait complete. 2023-10-06T10:41:32.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.447+0000 7f2145a0a700 1 Processor -- start 2023-10-06T10:41:32.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.447+0000 7f2145a0a700 1 -- start start 2023-10-06T10:41:32.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f2145a0a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21400adb60 0x7f214014cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:32.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f2145a0a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f21400ca370 0x7f21400c24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:32.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f2145a0a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21400ce8a0 0x7f214014b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:32.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f2145a0a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214014d8c0 con 0x7f21400adb60 2023-10-06T10:41:32.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f2145a0a700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21400b1080 con 0x7f21400ca370 2023-10-06T10:41:32.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f2145a0a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21400c2a10 con 0x7f21400ce8a0 2023-10-06T10:41:32.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f213effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21400adb60 0x7f214014cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:32.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f213effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21400adb60 0x7f214014cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33412/0 (socket says 172.21.15.92:33412) 2023-10-06T10:41:32.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f213effd700 1 -- 172.21.15.92:0/1215353419 learned_addr learned my addr 172.21.15.92:0/1215353419 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:32.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f213e7fc700 1 --2- 172.21.15.92:0/1215353419 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f21400ca370 0x7f21400c24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:32.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f213f7fe700 1 --2- 172.21.15.92:0/1215353419 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21400ce8a0 0x7f214014b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:32.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f213effd700 1 -- 172.21.15.92:0/1215353419 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21400ce8a0 msgr2=0x7f214014b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:32.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f213effd700 1 --2- 172.21.15.92:0/1215353419 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21400ce8a0 0x7f214014b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:32.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f213effd700 1 -- 172.21.15.92:0/1215353419 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f21400ca370 msgr2=0x7f21400c24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:32.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f213effd700 1 --2- 172.21.15.92:0/1215353419 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f21400ca370 0x7f21400c24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:32.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f213effd700 1 -- 172.21.15.92:0/1215353419 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2130009420 con 0x7f21400adb60 2023-10-06T10:41:32.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f213e7fc700 1 --2- 172.21.15.92:0/1215353419 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f21400ca370 0x7f21400c24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:32.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f213effd700 1 --2- 172.21.15.92:0/1215353419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21400adb60 0x7f214014cfd0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f21300091e0 tx=0x7f2130016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:32.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f2126ffd700 1 -- 172.21.15.92:0/1215353419 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213001cc90 con 0x7f21400adb60 2023-10-06T10:41:32.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f2126ffd700 1 -- 172.21.15.92:0/1215353419 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213001cdf0 con 0x7f21400adb60 2023-10-06T10:41:32.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f2145a0a700 1 -- 172.21.15.92:0/1215353419 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f214014d5e0 con 0x7f21400adb60 2023-10-06T10:41:32.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f2126ffd700 1 -- 172.21.15.92:0/1215353419 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213001a6a0 con 0x7f21400adb60 2023-10-06T10:41:32.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.448+0000 7f2145a0a700 1 -- 172.21.15.92:0/1215353419 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21400b1570 con 0x7f21400adb60 2023-10-06T10:41:32.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.450+0000 7f2145a0a700 1 -- 172.21.15.92:0/1215353419 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2140064a90 con 0x7f21400adb60 2023-10-06T10:41:32.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.450+0000 7f2126ffd700 1 -- 172.21.15.92:0/1215353419 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f213001aa50 con 0x7f21400adb60 2023-10-06T10:41:32.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.450+0000 7f2126ffd700 1 --2- 172.21.15.92:0/1215353419 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f212c066f30 0x7f212c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:32.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.450+0000 7f213e7fc700 1 --2- 172.21.15.92:0/1215353419 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f212c066f30 0x7f212c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:32.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.450+0000 7f2126ffd700 1 -- 172.21.15.92:0/1215353419 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2130086250 con 0x7f21400adb60 2023-10-06T10:41:32.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.451+0000 7f213e7fc700 1 --2- 172.21.15.92:0/1215353419 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f212c066f30 0x7f212c0693e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f214014d510 tx=0x7f2134009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:32.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.454+0000 7f2126ffd700 1 -- 172.21.15.92:0/1215353419 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2130053220 con 0x7f21400adb60 2023-10-06T10:41:32.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.684+0000 7f2145a0a700 1 -- 172.21.15.92:0/1215353419 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2140157a70 con 0x7f21400adb60 2023-10-06T10:41:32.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.684+0000 7f2126ffd700 1 -- 172.21.15.92:0/1215353419 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4637 (secure 0 0 0) 0x7f2130056840 con 0x7f21400adb60 2023-10-06T10:41:32.685 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:32.686 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3668157650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3668157650},{"type":"v1","addr":"172.21.15.124:6834","nonce":3668157650}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:40:34.932631+0000","modified":"2023-10-06T10:41:17.396042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5736,"mds_1":5733,"mds_2":5036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5036":{"gid":5036,"name":"d","rank":2,"incarnation":36,"state":"up:active","state_seq":13,"addr":"172.21.15.124:6835/2986058955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2986058955},{"type":"v1","addr":"172.21.15.124:6835","nonce":2986058955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5733":{"gid":5733,"name":"c","rank":1,"incarnation":34,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6836/831923836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":831923836},{"type":"v1","addr":"172.21.15.92:6836","nonce":831923836}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5736":{"gid":5736,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6837/2458340622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2458340622},{"type":"v1","addr":"172.21.15.92:6837","nonce":2458340622}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T10:41:32.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.686+0000 7f2145a0a700 1 -- 172.21.15.92:0/1215353419 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f212c066f30 msgr2=0x7f212c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:32.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.686+0000 7f2145a0a700 1 --2- 172.21.15.92:0/1215353419 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f212c066f30 0x7f212c0693e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f214014d510 tx=0x7f2134009040 comp rx=0 tx=0).stop 2023-10-06T10:41:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.686+0000 7f2145a0a700 1 -- 172.21.15.92:0/1215353419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21400adb60 msgr2=0x7f214014cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.686+0000 7f2145a0a700 1 --2- 172.21.15.92:0/1215353419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21400adb60 0x7f214014cfd0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f21300091e0 tx=0x7f2130016000 comp rx=0 tx=0).stop 2023-10-06T10:41:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.686+0000 7f2145a0a700 1 -- 172.21.15.92:0/1215353419 shutdown_connections 2023-10-06T10:41:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.686+0000 7f2145a0a700 1 --2- 172.21.15.92:0/1215353419 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f212c066f30 0x7f212c0693e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.686+0000 7f2145a0a700 1 --2- 172.21.15.92:0/1215353419 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21400ce8a0 0x7f214014b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.686+0000 7f2145a0a700 1 --2- 172.21.15.92:0/1215353419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21400adb60 0x7f214014cfd0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.686+0000 7f2145a0a700 1 --2- 172.21.15.92:0/1215353419 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f21400ca370 0x7f21400c24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.686+0000 7f2145a0a700 1 -- 172.21.15.92:0/1215353419 >> 172.21.15.92:0/1215353419 conn(0x7f21400707d0 msgr2=0x7f21400ca740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.687+0000 7f2145a0a700 1 -- 172.21.15.92:0/1215353419 shutdown_connections 2023-10-06T10:41:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.687+0000 7f2145a0a700 1 -- 172.21.15.92:0/1215353419 wait complete. 2023-10-06T10:41:32.689 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 42 2023-10-06T10:41:32.699 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:41:32.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.812+0000 7f0962527700 1 Processor -- start 2023-10-06T10:41:32.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.812+0000 7f0962527700 1 -- start start 2023-10-06T10:41:32.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.813+0000 7f0962527700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f095c0b49a0 0x7f095c0aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:32.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.813+0000 7f0962527700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f095c0b0760 0x7f095c0b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:32.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.813+0000 7f0962527700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f095c0b0c60 con 0x7f095c159560 2023-10-06T10:41:32.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.813+0000 7f0962527700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095c0b0f40 con 0x7f095c0b49a0 2023-10-06T10:41:32.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.813+0000 7f0962527700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095c0b1220 con 0x7f095c0b0760 2023-10-06T10:41:32.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.813+0000 7f095bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f095c159560 0x7f095c0aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51128/0 (socket says 172.21.15.92:51128) 2023-10-06T10:41:32.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.813+0000 7f095b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f095c0b49a0 0x7f095c0aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:32.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.813+0000 7f095bfff700 1 -- 172.21.15.92:0/2007854486 learned_addr learned my addr 172.21.15.92:0/2007854486 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:32.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.813+0000 7f0960ac4700 1 --2- 172.21.15.92:0/2007854486 >> v2:172.21.15.124:3300/0 conn(0x7f095c0b0760 0x7f095c0b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:32.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.813+0000 7f095b7fe700 1 -- 172.21.15.92:0/2007854486 >> v2:172.21.15.124:3300/0 conn(0x7f095c0b0760 msgr2=0x7f095c0b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:32.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.813+0000 7f095b7fe700 1 --2- 172.21.15.92:0/2007854486 >> v2:172.21.15.124:3300/0 conn(0x7f095c0b0760 0x7f095c0b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:32.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.814+0000 7f095b7fe700 1 -- 172.21.15.92:0/2007854486 >> v1:172.21.15.92:6789/0 conn(0x7f095c159560 legacy=0x7f095c0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.814+0000 7f095b7fe700 1 -- 172.21.15.92:0/2007854486 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095c0b1500 con 0x7f095c0b49a0 2023-10-06T10:41:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.814+0000 7f095affd700 1 -- 172.21.15.92:0/2007854486 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3571767953 0 0) 0x7f095c0b0c60 con 0x7f095c159560 2023-10-06T10:41:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.814+0000 7f0960ac4700 1 --2- 172.21.15.92:0/2007854486 >> v2:172.21.15.124:3300/0 conn(0x7f095c0b0760 0x7f095c0b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.814+0000 7f095b7fe700 1 --2- 172.21.15.92:0/2007854486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095c0b49a0 0x7f095c0aac40 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f09500098e0 tx=0x7f09500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d424aa8215dd39ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.814+0000 7f095affd700 1 -- 172.21.15.92:0/2007854486 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950009e80 con 0x7f095c0b49a0 2023-10-06T10:41:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.814+0000 7f095affd700 1 -- 172.21.15.92:0/2007854486 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095000b820 con 0x7f095c0b49a0 2023-10-06T10:41:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.814+0000 7f095affd700 1 -- 172.21.15.92:0/2007854486 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095000bba0 con 0x7f095c0b49a0 2023-10-06T10:41:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.814+0000 7f0962527700 1 -- 172.21.15.92:0/2007854486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095c0b49a0 msgr2=0x7f095c0aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.814+0000 7f0962527700 1 --2- 172.21.15.92:0/2007854486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095c0b49a0 0x7f095c0aac40 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f09500098e0 tx=0x7f09500091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.814+0000 7f0962527700 1 -- 172.21.15.92:0/2007854486 shutdown_connections 2023-10-06T10:41:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.814+0000 7f0962527700 1 --2- 172.21.15.92:0/2007854486 >> v2:172.21.15.124:3300/0 conn(0x7f095c0b0760 0x7f095c0b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.814+0000 7f0962527700 1 --2- 172.21.15.92:0/2007854486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095c0b49a0 0x7f095c0aac40 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:32.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.814+0000 7f0962527700 1 -- 172.21.15.92:0/2007854486 >> 172.21.15.92:0/2007854486 conn(0x7f095c0997f0 msgr2=0x7f095c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:32.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.814+0000 7f0962527700 1 -- 172.21.15.92:0/2007854486 shutdown_connections 2023-10-06T10:41:32.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.814+0000 7f0962527700 1 -- 172.21.15.92:0/2007854486 wait complete. 2023-10-06T10:41:32.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.815+0000 7f0962527700 1 Processor -- start 2023-10-06T10:41:32.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.816+0000 7f0962527700 1 -- start start 2023-10-06T10:41:32.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.816+0000 7f0962527700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095c0b0760 0x7f095c169b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.816+0000 7f0962527700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f095c0b49a0 0x7f095c158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.816+0000 7f0962527700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f095c159560 0x7f095c09bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.816+0000 7f0962527700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c15d140 con 0x7f095c0b0760 2023-10-06T10:41:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.816+0000 7f0962527700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c15a020 con 0x7f095c159560 2023-10-06T10:41:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.816+0000 7f0962527700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095c09ae40 con 0x7f095c0b49a0 2023-10-06T10:41:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.816+0000 7f095bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095c0b0760 0x7f095c169b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.816+0000 7f095bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095c0b0760 0x7f095c169b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33428/0 (socket says 172.21.15.92:33428) 2023-10-06T10:41:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.816+0000 7f095bfff700 1 -- 172.21.15.92:0/2799378493 learned_addr learned my addr 172.21.15.92:0/2799378493 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.817+0000 7f0960ac4700 1 --2- 172.21.15.92:0/2799378493 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f095c159560 0x7f095c09bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.817+0000 7f095b7fe700 1 --2- 172.21.15.92:0/2799378493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f095c0b49a0 0x7f095c158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.817+0000 7f095bfff700 1 -- 172.21.15.92:0/2799378493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f095c0b49a0 msgr2=0x7f095c158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.817+0000 7f095bfff700 1 --2- 172.21.15.92:0/2799378493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f095c0b49a0 0x7f095c158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.817+0000 7f095bfff700 1 -- 172.21.15.92:0/2799378493 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f095c159560 msgr2=0x7f095c09bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.817+0000 7f095bfff700 1 --2- 172.21.15.92:0/2799378493 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f095c159560 0x7f095c09bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.817+0000 7f095bfff700 1 -- 172.21.15.92:0/2799378493 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0950009420 con 0x7f095c0b0760 2023-10-06T10:41:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.817+0000 7f095bfff700 1 --2- 172.21.15.92:0/2799378493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095c0b0760 0x7f095c169b20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f094c00c5e0 tx=0x7f094c007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.817+0000 7f0943fff700 1 -- 172.21.15.92:0/2799378493 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c0072e0 con 0x7f095c0b0760 2023-10-06T10:41:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.817+0000 7f0943fff700 1 -- 172.21.15.92:0/2799378493 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c007440 con 0x7f095c0b0760 2023-10-06T10:41:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.817+0000 7f0943fff700 1 -- 172.21.15.92:0/2799378493 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c0057f0 con 0x7f095c0b0760 2023-10-06T10:41:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.817+0000 7f0962527700 1 -- 172.21.15.92:0/2799378493 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f095c09b120 con 0x7f095c0b0760 2023-10-06T10:41:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.817+0000 7f0962527700 1 -- 172.21.15.92:0/2799378493 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095c159cb0 con 0x7f095c0b0760 2023-10-06T10:41:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.818+0000 7f0962527700 1 -- 172.21.15.92:0/2799378493 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f095c064a90 con 0x7f095c0b0760 2023-10-06T10:41:32.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.818+0000 7f0943fff700 1 -- 172.21.15.92:0/2799378493 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f094c005bc0 con 0x7f095c0b0760 2023-10-06T10:41:32.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.818+0000 7f0943fff700 1 --2- 172.21.15.92:0/2799378493 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0948066f30 0x7f09480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:32.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.819+0000 7f0943fff700 1 -- 172.21.15.92:0/2799378493 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f094c083d60 con 0x7f095c0b0760 2023-10-06T10:41:32.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.819+0000 7f095b7fe700 1 --2- 172.21.15.92:0/2799378493 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0948066f30 0x7f09480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:32.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.819+0000 7f095b7fe700 1 --2- 172.21.15.92:0/2799378493 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0948066f30 0x7f09480693e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f09500098e0 tx=0x7f0950017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:32.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:32.823+0000 7f0943fff700 1 -- 172.21.15.92:0/2799378493 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f094c050d30 con 0x7f095c0b0760 2023-10-06T10:41:33.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.007+0000 7f0962527700 1 -- 172.21.15.92:0/2799378493 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f095c0b17e0 con 0x7f095c0b0760 2023-10-06T10:41:33.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.008+0000 7f0943fff700 1 -- 172.21.15.92:0/2799378493 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+17071 (secure 0 0 0) 0x7f094c017070 con 0x7f095c0b0760 2023-10-06T10:41:33.009 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:33.009 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":52,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:40:36.557340+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:40:32.570535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-06T10:40:33.886794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:41:33.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.010+0000 7f0962527700 1 -- 172.21.15.92:0/2799378493 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0948066f30 msgr2=0x7f09480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:33.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.010+0000 7f0962527700 1 --2- 172.21.15.92:0/2799378493 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0948066f30 0x7f09480693e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f09500098e0 tx=0x7f0950017000 comp rx=0 tx=0).stop 2023-10-06T10:41:33.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.010+0000 7f0962527700 1 -- 172.21.15.92:0/2799378493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095c0b0760 msgr2=0x7f095c169b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:33.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.010+0000 7f0962527700 1 --2- 172.21.15.92:0/2799378493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095c0b0760 0x7f095c169b20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f094c00c5e0 tx=0x7f094c007e80 comp rx=0 tx=0).stop 2023-10-06T10:41:33.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.010+0000 7f0962527700 1 -- 172.21.15.92:0/2799378493 shutdown_connections 2023-10-06T10:41:33.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.010+0000 7f0962527700 1 --2- 172.21.15.92:0/2799378493 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0948066f30 0x7f09480693e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.010+0000 7f0962527700 1 --2- 172.21.15.92:0/2799378493 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f095c159560 0x7f095c09bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.010+0000 7f0962527700 1 --2- 172.21.15.92:0/2799378493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095c0b0760 0x7f095c169b20 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.010+0000 7f0962527700 1 --2- 172.21.15.92:0/2799378493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f095c0b49a0 0x7f095c158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.010+0000 7f0962527700 1 -- 172.21.15.92:0/2799378493 >> 172.21.15.92:0/2799378493 conn(0x7f095c0997f0 msgr2=0x7f095c0b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:33.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.010+0000 7f0962527700 1 -- 172.21.15.92:0/2799378493 shutdown_connections 2023-10-06T10:41:33.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.010+0000 7f0962527700 1 -- 172.21.15.92:0/2799378493 wait complete. 2023-10-06T10:41:33.023 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T10:38:09.680559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T10:38:10.120786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T10:40:32.570535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T10:40:33.886794+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T10:41:33.023 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T10:41:33.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.139+0000 7f1051e38700 1 Processor -- start 2023-10-06T10:41:33.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.139+0000 7f1051e38700 1 -- start start 2023-10-06T10:41:33.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.140+0000 7f1051e38700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f104c0b05d0 0x7f104c0cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.140+0000 7f1051e38700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f104c0c9580 0x7f104c0c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.140+0000 7f1051e38700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104c0cadd0 con 0x7f104c0ccc50 2023-10-06T10:41:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.140+0000 7f1051e38700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f104c0cd0f0 con 0x7f104c0c9580 2023-10-06T10:41:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.140+0000 7f1051e38700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f104c1577d0 con 0x7f104c0b05d0 2023-10-06T10:41:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.140+0000 7f1051637700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f104c0c9580 0x7f104c0c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.140+0000 7f104bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f104c0ccc50 0x7f104c0c8e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51140/0 (socket says 172.21.15.92:51140) 2023-10-06T10:41:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.140+0000 7f104bfff700 1 -- 172.21.15.92:0/1656830163 learned_addr learned my addr 172.21.15.92:0/1656830163 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.141+0000 7f1050e36700 1 --2- 172.21.15.92:0/1656830163 >> v2:172.21.15.124:3300/0 conn(0x7f104c0b05d0 0x7f104c0cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.141+0000 7f1051637700 1 -- 172.21.15.92:0/1656830163 >> v2:172.21.15.124:3300/0 conn(0x7f104c0b05d0 msgr2=0x7f104c0cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.141+0000 7f1051637700 1 --2- 172.21.15.92:0/1656830163 >> v2:172.21.15.124:3300/0 conn(0x7f104c0b05d0 0x7f104c0cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.141+0000 7f1051637700 1 -- 172.21.15.92:0/1656830163 >> v1:172.21.15.92:6789/0 conn(0x7f104c0ccc50 legacy=0x7f104c0c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.141+0000 7f1051637700 1 -- 172.21.15.92:0/1656830163 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104c0c9ae0 con 0x7f104c0c9580 2023-10-06T10:41:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.141+0000 7f1050e36700 1 --2- 172.21.15.92:0/1656830163 >> v2:172.21.15.124:3300/0 conn(0x7f104c0b05d0 0x7f104c0cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.141+0000 7f1051637700 1 --2- 172.21.15.92:0/1656830163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f104c0c9580 0x7f104c0c2f40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f104c06ad80 tx=0x7f10440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b91f16fa6bd7ee7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.141+0000 7f104b7fe700 1 -- 172.21.15.92:0/1656830163 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104400b6b0 con 0x7f104c0c9580 2023-10-06T10:41:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.141+0000 7f104b7fe700 1 -- 172.21.15.92:0/1656830163 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104400b810 con 0x7f104c0c9580 2023-10-06T10:41:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.141+0000 7f104b7fe700 1 -- 172.21.15.92:0/1656830163 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104400bbb0 con 0x7f104c0c9580 2023-10-06T10:41:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.141+0000 7f1051e38700 1 -- 172.21.15.92:0/1656830163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f104c0c9580 msgr2=0x7f104c0c2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.141+0000 7f1051e38700 1 --2- 172.21.15.92:0/1656830163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f104c0c9580 0x7f104c0c2f40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f104c06ad80 tx=0x7f10440091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.142+0000 7f1051e38700 1 -- 172.21.15.92:0/1656830163 shutdown_connections 2023-10-06T10:41:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.142+0000 7f1051e38700 1 --2- 172.21.15.92:0/1656830163 >> v2:172.21.15.124:3300/0 conn(0x7f104c0b05d0 0x7f104c0cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.142+0000 7f1051e38700 1 --2- 172.21.15.92:0/1656830163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f104c0c9580 0x7f104c0c2f40 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.142+0000 7f1051e38700 1 -- 172.21.15.92:0/1656830163 >> 172.21.15.92:0/1656830163 conn(0x7f104c096a30 msgr2=0x7f104c096e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:33.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.142+0000 7f1051e38700 1 -- 172.21.15.92:0/1656830163 shutdown_connections 2023-10-06T10:41:33.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.142+0000 7f1051e38700 1 -- 172.21.15.92:0/1656830163 wait complete. 2023-10-06T10:41:33.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.143+0000 7f1051e38700 1 Processor -- start 2023-10-06T10:41:33.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.143+0000 7f1051e38700 1 -- start start 2023-10-06T10:41:33.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.143+0000 7f1051e38700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f104c0b05d0 0x7f104c0b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.143+0000 7f1051e38700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f104c0c9580 0x7f104c0c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.143+0000 7f1051e38700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f104c0ccc50 0x7f104c154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.143+0000 7f1051e38700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c0b37a0 con 0x7f104c0c9580 2023-10-06T10:41:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.143+0000 7f1051e38700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c0b28f0 con 0x7f104c0ccc50 2023-10-06T10:41:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.143+0000 7f1051e38700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f104c0b2b90 con 0x7f104c0b05d0 2023-10-06T10:41:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.144+0000 7f104bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f104c0c9580 0x7f104c0c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.144+0000 7f104bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f104c0c9580 0x7f104c0c7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33438/0 (socket says 172.21.15.92:33438) 2023-10-06T10:41:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.144+0000 7f104bfff700 1 -- 172.21.15.92:0/1341863270 learned_addr learned my addr 172.21.15.92:0/1341863270 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.144+0000 7f1050e36700 1 --2- 172.21.15.92:0/1341863270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f104c0b05d0 0x7f104c0b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.144+0000 7f1051637700 1 --2- 172.21.15.92:0/1341863270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f104c0ccc50 0x7f104c154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.144+0000 7f104bfff700 1 -- 172.21.15.92:0/1341863270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f104c0b05d0 msgr2=0x7f104c0b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.144+0000 7f104bfff700 1 --2- 172.21.15.92:0/1341863270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f104c0b05d0 0x7f104c0b23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.144+0000 7f104bfff700 1 -- 172.21.15.92:0/1341863270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f104c0ccc50 msgr2=0x7f104c154600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.144+0000 7f104bfff700 1 --2- 172.21.15.92:0/1341863270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f104c0ccc50 0x7f104c154600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.144+0000 7f104bfff700 1 -- 172.21.15.92:0/1341863270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1044009420 con 0x7f104c0c9580 2023-10-06T10:41:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.144+0000 7f104bfff700 1 --2- 172.21.15.92:0/1341863270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f104c0c9580 0x7f104c0c7c20 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f104000c590 tx=0x7f1040007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.144+0000 7f1048ff9700 1 -- 172.21.15.92:0/1341863270 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1040001160 con 0x7f104c0c9580 2023-10-06T10:41:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.144+0000 7f1048ff9700 1 -- 172.21.15.92:0/1341863270 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10400072e0 con 0x7f104c0c9580 2023-10-06T10:41:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.144+0000 7f1051e38700 1 -- 172.21.15.92:0/1341863270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f104c0b2e70 con 0x7f104c0c9580 2023-10-06T10:41:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.144+0000 7f1048ff9700 1 -- 172.21.15.92:0/1341863270 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10400057f0 con 0x7f104c0c9580 2023-10-06T10:41:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.144+0000 7f1051e38700 1 -- 172.21.15.92:0/1341863270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f104c0b3430 con 0x7f104c0c9580 2023-10-06T10:41:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.145+0000 7f1051e38700 1 -- 172.21.15.92:0/1341863270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f104c0b3910 con 0x7f104c0c9580 2023-10-06T10:41:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.145+0000 7f1048ff9700 1 -- 172.21.15.92:0/1341863270 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1040005c20 con 0x7f104c0c9580 2023-10-06T10:41:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.146+0000 7f1048ff9700 1 --2- 172.21.15.92:0/1341863270 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1038066f30 0x7f10380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.146+0000 7f1050e36700 1 --2- 172.21.15.92:0/1341863270 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1038066f30 0x7f10380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.146+0000 7f1048ff9700 1 -- 172.21.15.92:0/1341863270 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1040083b60 con 0x7f104c0c9580 2023-10-06T10:41:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.146+0000 7f1050e36700 1 --2- 172.21.15.92:0/1341863270 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1038066f30 0x7f10380693e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f103c007740 tx=0x7f103c0099d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:33.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.150+0000 7f1048ff9700 1 -- 172.21.15.92:0/1341863270 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1040050b30 con 0x7f104c0c9580 2023-10-06T10:41:33.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.332+0000 7f1051e38700 1 -- 172.21.15.92:0/1341863270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f104c05f230 con 0x7f104c0c9580 2023-10-06T10:41:33.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.332+0000 7f1048ff9700 1 -- 172.21.15.92:0/1341863270 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v52) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1040017070 con 0x7f104c0c9580 2023-10-06T10:41:33.333 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T10:41:33.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.335+0000 7f1051e38700 1 -- 172.21.15.92:0/1341863270 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1038066f30 msgr2=0x7f10380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:33.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.335+0000 7f1051e38700 1 --2- 172.21.15.92:0/1341863270 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1038066f30 0x7f10380693e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f103c007740 tx=0x7f103c0099d0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.335+0000 7f1051e38700 1 -- 172.21.15.92:0/1341863270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f104c0c9580 msgr2=0x7f104c0c7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:33.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.335+0000 7f1051e38700 1 --2- 172.21.15.92:0/1341863270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f104c0c9580 0x7f104c0c7c20 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f104000c590 tx=0x7f1040007e40 comp rx=0 tx=0).stop 2023-10-06T10:41:33.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.335+0000 7f1051e38700 1 -- 172.21.15.92:0/1341863270 shutdown_connections 2023-10-06T10:41:33.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.335+0000 7f1051e38700 1 --2- 172.21.15.92:0/1341863270 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1038066f30 0x7f10380693e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.335+0000 7f1051e38700 1 --2- 172.21.15.92:0/1341863270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f104c0ccc50 0x7f104c154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.335+0000 7f1051e38700 1 --2- 172.21.15.92:0/1341863270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f104c0b05d0 0x7f104c0b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.335+0000 7f1051e38700 1 --2- 172.21.15.92:0/1341863270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f104c0c9580 0x7f104c0c7c20 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.335+0000 7f1051e38700 1 -- 172.21.15.92:0/1341863270 >> 172.21.15.92:0/1341863270 conn(0x7f104c096a30 msgr2=0x7f104c0cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:33.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.335+0000 7f1051e38700 1 -- 172.21.15.92:0/1341863270 shutdown_connections 2023-10-06T10:41:33.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.335+0000 7f1051e38700 1 -- 172.21.15.92:0/1341863270 wait complete. 2023-10-06T10:41:33.347 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T10:41:33.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.466+0000 7f3819188700 1 Processor -- start 2023-10-06T10:41:33.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.466+0000 7f3819188700 1 -- start start 2023-10-06T10:41:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.467+0000 7f3819188700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140b2620 0x7f38140cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.467+0000 7f3819188700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f38140cf1b0 0x7f38140cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.467+0000 7f3819188700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38140cc970 con 0x7f38140b29f0 2023-10-06T10:41:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.467+0000 7f3819188700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38140cbd60 con 0x7f38140cf1b0 2023-10-06T10:41:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.467+0000 7f3819188700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38140cc040 con 0x7f38140b2620 2023-10-06T10:41:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.467+0000 7f3818987700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f38140cf1b0 0x7f38140cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.467+0000 7f3818987700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f38140cf1b0 0x7f38140cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52798/0 (socket says 172.21.15.92:52798) 2023-10-06T10:41:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.467+0000 7f3818987700 1 -- 172.21.15.92:0/1251084289 learned_addr learned my addr 172.21.15.92:0/1251084289 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.467+0000 7f3813fff700 1 --2- 172.21.15.92:0/1251084289 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140b2620 0x7f38140cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.467+0000 7f3818987700 1 -- 172.21.15.92:0/1251084289 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140b2620 msgr2=0x7f38140cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.467+0000 7f3818987700 1 --2- 172.21.15.92:0/1251084289 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140b2620 0x7f38140cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.467+0000 7f3818987700 1 -- 172.21.15.92:0/1251084289 >> v1:172.21.15.124:6789/0 conn(0x7f38140b29f0 legacy=0x7f38140ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.467+0000 7f3818987700 1 -- 172.21.15.92:0/1251084289 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38140cc320 con 0x7f38140cf1b0 2023-10-06T10:41:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.467+0000 7f3818987700 1 --2- 172.21.15.92:0/1251084289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38140cf1b0 0x7f38140cd1f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f380c009f20 tx=0x7f380c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc6ec0aa64a9ed9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.468+0000 7f3812ffd700 1 -- 172.21.15.92:0/1251084289 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c00e070 con 0x7f38140cf1b0 2023-10-06T10:41:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.468+0000 7f3812ffd700 1 -- 172.21.15.92:0/1251084289 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380c00ba40 con 0x7f38140cf1b0 2023-10-06T10:41:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.468+0000 7f3812ffd700 1 -- 172.21.15.92:0/1251084289 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c0092e0 con 0x7f38140cf1b0 2023-10-06T10:41:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.468+0000 7f3819188700 1 -- 172.21.15.92:0/1251084289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38140cf1b0 msgr2=0x7f38140cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.468+0000 7f3819188700 1 --2- 172.21.15.92:0/1251084289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38140cf1b0 0x7f38140cd1f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f380c009f20 tx=0x7f380c009840 comp rx=0 tx=0).stop 2023-10-06T10:41:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.468+0000 7f3819188700 1 -- 172.21.15.92:0/1251084289 shutdown_connections 2023-10-06T10:41:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.468+0000 7f3819188700 1 --2- 172.21.15.92:0/1251084289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38140cf1b0 0x7f38140cd1f0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.468+0000 7f3819188700 1 --2- 172.21.15.92:0/1251084289 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140b2620 0x7f38140cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.468+0000 7f3819188700 1 -- 172.21.15.92:0/1251084289 >> 172.21.15.92:0/1251084289 conn(0x7f3814099520 msgr2=0x7f381406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.468+0000 7f3819188700 1 -- 172.21.15.92:0/1251084289 shutdown_connections 2023-10-06T10:41:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.468+0000 7f3819188700 1 -- 172.21.15.92:0/1251084289 wait complete. 2023-10-06T10:41:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.469+0000 7f3819188700 1 Processor -- start 2023-10-06T10:41:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.469+0000 7f3819188700 1 -- start start 2023-10-06T10:41:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.470+0000 7f3819188700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38140b2620 0x7f38141693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.470+0000 7f3819188700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140b29f0 0x7f381416c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.470+0000 7f3819188700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38140cf1b0 0x7f38140b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.470+0000 7f3819188700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381416c750 con 0x7f38140b2620 2023-10-06T10:41:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.470+0000 7f3819188700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38140b3090 con 0x7f38140cf1b0 2023-10-06T10:41:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.470+0000 7f3819188700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38140b3370 con 0x7f38140b29f0 2023-10-06T10:41:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.470+0000 7f3813fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38140b2620 0x7f38141693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.470+0000 7f3813fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38140b2620 0x7f38141693c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52808/0 (socket says 172.21.15.92:52808) 2023-10-06T10:41:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.470+0000 7f3813fff700 1 -- 172.21.15.92:0/1657381547 learned_addr learned my addr 172.21.15.92:0/1657381547 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.470+0000 7f3813fff700 1 -- 172.21.15.92:0/1657381547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140b29f0 msgr2=0x7f381416c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.470+0000 7f3813fff700 1 --2- 172.21.15.92:0/1657381547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140b29f0 0x7f381416c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.470+0000 7f3813fff700 1 -- 172.21.15.92:0/1657381547 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38140cf1b0 msgr2=0x7f38140b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.470+0000 7f3813fff700 1 --2- 172.21.15.92:0/1657381547 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38140cf1b0 0x7f38140b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.470+0000 7f3813fff700 1 -- 172.21.15.92:0/1657381547 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c009ae0 con 0x7f38140b2620 2023-10-06T10:41:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.471+0000 7f3813fff700 1 --2- 172.21.15.92:0/1657381547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38140b2620 0x7f38141693c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f380400d890 tx=0x7f3804008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.471+0000 7f37fbfff700 1 -- 172.21.15.92:0/1657381547 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3804018070 con 0x7f38140b2620 2023-10-06T10:41:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.471+0000 7f37fbfff700 1 -- 172.21.15.92:0/1657381547 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3804005a00 con 0x7f38140b2620 2023-10-06T10:41:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.471+0000 7f37fbfff700 1 -- 172.21.15.92:0/1657381547 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3804005dc0 con 0x7f38140b2620 2023-10-06T10:41:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.471+0000 7f3819188700 1 -- 172.21.15.92:0/1657381547 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38140b3650 con 0x7f38140b2620 2023-10-06T10:41:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.471+0000 7f3819188700 1 -- 172.21.15.92:0/1657381547 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38140b3c10 con 0x7f38140b2620 2023-10-06T10:41:33.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.472+0000 7f3819188700 1 -- 172.21.15.92:0/1657381547 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f381405f230 con 0x7f38140b2620 2023-10-06T10:41:33.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.472+0000 7f37fbfff700 1 -- 172.21.15.92:0/1657381547 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3804012800 con 0x7f38140b2620 2023-10-06T10:41:33.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.472+0000 7f37fbfff700 1 --2- 172.21.15.92:0/1657381547 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3800066f30 0x7f38000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:33.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.473+0000 7f37fbfff700 1 -- 172.21.15.92:0/1657381547 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3804084940 con 0x7f38140b2620 2023-10-06T10:41:33.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.473+0000 7f38137fe700 1 --2- 172.21.15.92:0/1657381547 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3800066f30 0x7f38000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:33.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.473+0000 7f38137fe700 1 --2- 172.21.15.92:0/1657381547 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3800066f30 0x7f38000693e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3808007fd0 tx=0x7f3808007b20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:33.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.477+0000 7f37fbfff700 1 -- 172.21.15.92:0/1657381547 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38040553a0 con 0x7f38140b2620 2023-10-06T10:41:33.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.660+0000 7f3819188700 1 -- 172.21.15.92:0/1657381547 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f38140cc600 con 0x7f38140b2620 2023-10-06T10:41:33.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.660+0000 7f37fbfff700 1 -- 172.21.15.92:0/1657381547 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3804054f30 con 0x7f38140b2620 2023-10-06T10:41:33.662 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T10:41:33.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.662+0000 7f3819188700 1 -- 172.21.15.92:0/1657381547 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3800066f30 msgr2=0x7f38000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:33.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.662+0000 7f3819188700 1 --2- 172.21.15.92:0/1657381547 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3800066f30 0x7f38000693e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3808007fd0 tx=0x7f3808007b20 comp rx=0 tx=0).stop 2023-10-06T10:41:33.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.662+0000 7f3819188700 1 -- 172.21.15.92:0/1657381547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38140b2620 msgr2=0x7f38141693c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:33.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.662+0000 7f3819188700 1 --2- 172.21.15.92:0/1657381547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38140b2620 0x7f38141693c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f380400d890 tx=0x7f3804008040 comp rx=0 tx=0).stop 2023-10-06T10:41:33.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.662+0000 7f3819188700 1 -- 172.21.15.92:0/1657381547 shutdown_connections 2023-10-06T10:41:33.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.662+0000 7f3819188700 1 --2- 172.21.15.92:0/1657381547 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3800066f30 0x7f38000693e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.662+0000 7f3819188700 1 --2- 172.21.15.92:0/1657381547 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38140cf1b0 0x7f38140b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.662+0000 7f3819188700 1 --2- 172.21.15.92:0/1657381547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38140b2620 0x7f38141693c0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.662+0000 7f3819188700 1 --2- 172.21.15.92:0/1657381547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140b29f0 0x7f381416c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.663+0000 7f3819188700 1 -- 172.21.15.92:0/1657381547 >> 172.21.15.92:0/1657381547 conn(0x7f3814099520 msgr2=0x7f3814068d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:33.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.663+0000 7f3819188700 1 -- 172.21.15.92:0/1657381547 shutdown_connections 2023-10-06T10:41:33.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.663+0000 7f3819188700 1 -- 172.21.15.92:0/1657381547 wait complete. 2023-10-06T10:41:33.675 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T10:41:33.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.791+0000 7f2d47943700 1 Processor -- start 2023-10-06T10:41:33.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.791+0000 7f2d47943700 1 -- start start 2023-10-06T10:41:33.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.792+0000 7f2d47943700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2d401598a0 0x7f2d400ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:33.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.792+0000 7f2d47943700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2d400cbc70 0x7f2d400ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:33.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.792+0000 7f2d47943700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d400cf5d0 con 0x7f2d401594d0 2023-10-06T10:41:33.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.792+0000 7f2d47943700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d400cf8b0 con 0x7f2d401598a0 2023-10-06T10:41:33.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.792+0000 7f2d47943700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d400cfb90 con 0x7f2d400cbc70 2023-10-06T10:41:33.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.792+0000 7f2d44ede700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2d401598a0 0x7f2d400ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:33.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.792+0000 7f2d44ede700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2d401598a0 0x7f2d400ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52818/0 (socket says 172.21.15.92:52818) 2023-10-06T10:41:33.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.792+0000 7f2d44ede700 1 -- 172.21.15.92:0/3793242236 learned_addr learned my addr 172.21.15.92:0/3793242236 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:33.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.792+0000 7f2d45ee0700 1 --2- 172.21.15.92:0/3793242236 >> v2:172.21.15.124:3300/0 conn(0x7f2d400cbc70 0x7f2d400ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:33.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.793+0000 7f2d44ede700 1 -- 172.21.15.92:0/3793242236 >> v2:172.21.15.124:3300/0 conn(0x7f2d400cbc70 msgr2=0x7f2d400ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:33.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.793+0000 7f2d44ede700 1 --2- 172.21.15.92:0/3793242236 >> v2:172.21.15.124:3300/0 conn(0x7f2d400cbc70 0x7f2d400ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.793+0000 7f2d44ede700 1 -- 172.21.15.92:0/3793242236 >> v1:172.21.15.124:6789/0 conn(0x7f2d401594d0 legacy=0x7f2d400cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:33.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.793+0000 7f2d44ede700 1 -- 172.21.15.92:0/3793242236 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d400cfe70 con 0x7f2d401598a0 2023-10-06T10:41:33.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.793+0000 7f2d45ee0700 1 --2- 172.21.15.92:0/3793242236 >> v2:172.21.15.124:3300/0 conn(0x7f2d400cbc70 0x7f2d400ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:33.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.793+0000 7f2d44ede700 1 --2- 172.21.15.92:0/3793242236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d401598a0 0x7f2d400ce7e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2d4006f950 tx=0x7f2d30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa53c73e1b48b2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:33.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.793+0000 7f2d37fff700 1 -- 172.21.15.92:0/3793242236 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3000b860 con 0x7f2d401598a0 2023-10-06T10:41:33.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.793+0000 7f2d37fff700 1 -- 172.21.15.92:0/3793242236 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3000b9c0 con 0x7f2d401598a0 2023-10-06T10:41:33.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.793+0000 7f2d37fff700 1 -- 172.21.15.92:0/3793242236 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3000bd60 con 0x7f2d401598a0 2023-10-06T10:41:33.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.793+0000 7f2d47943700 1 -- 172.21.15.92:0/3793242236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d401598a0 msgr2=0x7f2d400ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:33.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.793+0000 7f2d47943700 1 --2- 172.21.15.92:0/3793242236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d401598a0 0x7f2d400ce7e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2d4006f950 tx=0x7f2d30009250 comp rx=0 tx=0).stop 2023-10-06T10:41:33.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.793+0000 7f2d47943700 1 -- 172.21.15.92:0/3793242236 shutdown_connections 2023-10-06T10:41:33.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.793+0000 7f2d47943700 1 --2- 172.21.15.92:0/3793242236 >> v2:172.21.15.124:3300/0 conn(0x7f2d400cbc70 0x7f2d400ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.793+0000 7f2d47943700 1 --2- 172.21.15.92:0/3793242236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d401598a0 0x7f2d400ce7e0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.793+0000 7f2d47943700 1 -- 172.21.15.92:0/3793242236 >> 172.21.15.92:0/3793242236 conn(0x7f2d40089e50 msgr2=0x7f2d40099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:33.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.794+0000 7f2d47943700 1 -- 172.21.15.92:0/3793242236 shutdown_connections 2023-10-06T10:41:33.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.794+0000 7f2d47943700 1 -- 172.21.15.92:0/3793242236 wait complete. 2023-10-06T10:41:33.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.795+0000 7f2d47943700 1 Processor -- start 2023-10-06T10:41:33.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.795+0000 7f2d47943700 1 -- start start 2023-10-06T10:41:33.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.795+0000 7f2d47943700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d400cbc70 0x7f2d4014b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:33.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d47943700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d401594d0 0x7f2d40150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:33.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d47943700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d401598a0 0x7f2d40150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d47943700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d400b5360 con 0x7f2d401594d0 2023-10-06T10:41:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d47943700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4014daa0 con 0x7f2d401598a0 2023-10-06T10:41:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d47943700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4014d7c0 con 0x7f2d400cbc70 2023-10-06T10:41:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d44ede700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d401594d0 0x7f2d40150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d44ede700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d401594d0 0x7f2d40150020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52834/0 (socket says 172.21.15.92:52834) 2023-10-06T10:41:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d44ede700 1 -- 172.21.15.92:0/1829109950 learned_addr learned my addr 172.21.15.92:0/1829109950 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d456df700 1 --2- 172.21.15.92:0/1829109950 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d400cbc70 0x7f2d4014b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d45ee0700 1 --2- 172.21.15.92:0/1829109950 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d401598a0 0x7f2d40150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d44ede700 1 -- 172.21.15.92:0/1829109950 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d400cbc70 msgr2=0x7f2d4014b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d44ede700 1 --2- 172.21.15.92:0/1829109950 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d400cbc70 0x7f2d4014b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d44ede700 1 -- 172.21.15.92:0/1829109950 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d401598a0 msgr2=0x7f2d40150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d44ede700 1 --2- 172.21.15.92:0/1829109950 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d401598a0 0x7f2d40150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d44ede700 1 -- 172.21.15.92:0/1829109950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d300094f0 con 0x7f2d401594d0 2023-10-06T10:41:33.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d456df700 1 --2- 172.21.15.92:0/1829109950 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d400cbc70 0x7f2d4014b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:33.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d45ee0700 1 --2- 172.21.15.92:0/1829109950 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d401598a0 0x7f2d40150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:33.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d44ede700 1 --2- 172.21.15.92:0/1829109950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d401594d0 0x7f2d40150020 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009290 tx=0x7f2d30017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:33.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d357fa700 1 -- 172.21.15.92:0/1829109950 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d30011e60 con 0x7f2d401594d0 2023-10-06T10:41:33.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d357fa700 1 -- 172.21.15.92:0/1829109950 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3001f070 con 0x7f2d401594d0 2023-10-06T10:41:33.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d47943700 1 -- 172.21.15.92:0/1829109950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d400b4d10 con 0x7f2d401594d0 2023-10-06T10:41:33.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d357fa700 1 -- 172.21.15.92:0/1829109950 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3001a870 con 0x7f2d401594d0 2023-10-06T10:41:33.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.796+0000 7f2d47943700 1 -- 172.21.15.92:0/1829109950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4014b990 con 0x7f2d401594d0 2023-10-06T10:41:33.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.797+0000 7f2d47943700 1 -- 172.21.15.92:0/1829109950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d400b54d0 con 0x7f2d401594d0 2023-10-06T10:41:33.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.797+0000 7f2d357fa700 1 -- 172.21.15.92:0/1829109950 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2d3001abe0 con 0x7f2d401594d0 2023-10-06T10:41:33.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.798+0000 7f2d357fa700 1 --2- 172.21.15.92:0/1829109950 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2d24066f60 0x7f2d24069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:33.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.798+0000 7f2d357fa700 1 -- 172.21.15.92:0/1829109950 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2d30021070 con 0x7f2d401594d0 2023-10-06T10:41:33.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.798+0000 7f2d456df700 1 --2- 172.21.15.92:0/1829109950 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2d24066f60 0x7f2d24069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:33.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.798+0000 7f2d456df700 1 --2- 172.21.15.92:0/1829109950 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2d24066f60 0x7f2d24069410 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c007fd0 tx=0x7f2d3c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:33.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.802+0000 7f2d357fa700 1 -- 172.21.15.92:0/1829109950 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d30053700 con 0x7f2d401594d0 2023-10-06T10:41:33.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.982+0000 7f2d47943700 1 -- 172.21.15.92:0/1829109950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2d4014bbe0 con 0x7f2d401594d0 2023-10-06T10:41:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.983+0000 7f2d357fa700 1 -- 172.21.15.92:0/1829109950 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2d30056d20 con 0x7f2d401594d0 2023-10-06T10:41:33.984 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T10:41:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.985+0000 7f2d47943700 1 -- 172.21.15.92:0/1829109950 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2d24066f60 msgr2=0x7f2d24069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.985+0000 7f2d47943700 1 --2- 172.21.15.92:0/1829109950 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2d24066f60 0x7f2d24069410 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c007fd0 tx=0x7f2d3c009040 comp rx=0 tx=0).stop 2023-10-06T10:41:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.985+0000 7f2d47943700 1 -- 172.21.15.92:0/1829109950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d401594d0 msgr2=0x7f2d40150020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.985+0000 7f2d47943700 1 --2- 172.21.15.92:0/1829109950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d401594d0 0x7f2d40150020 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009290 tx=0x7f2d30017000 comp rx=0 tx=0).stop 2023-10-06T10:41:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.985+0000 7f2d47943700 1 -- 172.21.15.92:0/1829109950 shutdown_connections 2023-10-06T10:41:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.985+0000 7f2d47943700 1 --2- 172.21.15.92:0/1829109950 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2d24066f60 0x7f2d24069410 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.985+0000 7f2d47943700 1 --2- 172.21.15.92:0/1829109950 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d401598a0 0x7f2d40150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.985+0000 7f2d47943700 1 --2- 172.21.15.92:0/1829109950 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d400cbc70 0x7f2d4014b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.985+0000 7f2d47943700 1 --2- 172.21.15.92:0/1829109950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d401594d0 0x7f2d40150020 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.985+0000 7f2d47943700 1 -- 172.21.15.92:0/1829109950 >> 172.21.15.92:0/1829109950 conn(0x7f2d40089e50 msgr2=0x7f2d400ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.985+0000 7f2d47943700 1 -- 172.21.15.92:0/1829109950 shutdown_connections 2023-10-06T10:41:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:33.985+0000 7f2d47943700 1 -- 172.21.15.92:0/1829109950 wait complete. 2023-10-06T10:41:33.997 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T10:41:34.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.113+0000 7f423359e700 1 Processor -- start 2023-10-06T10:41:34.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.113+0000 7f423359e700 1 -- start start 2023-10-06T10:41:34.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.114+0000 7f423359e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f42341593c0 0x7f423406a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:34.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.114+0000 7f423359e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f42340cc240 0x7f42340c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:34.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.114+0000 7f423359e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4234067ee0 con 0x7f42340699d0 2023-10-06T10:41:34.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.114+0000 7f423359e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4234159790 con 0x7f42340cc240 2023-10-06T10:41:34.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.114+0000 7f423359e700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f423406ad50 con 0x7f42341593c0 2023-10-06T10:41:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.114+0000 7f4231d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f42340cc240 0x7f42340c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.114+0000 7f4232d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f42340699d0 0x7f4234069df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47430/0 (socket says 172.21.15.92:47430) 2023-10-06T10:41:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.114+0000 7f4231d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f42340cc240 0x7f42340c6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52844/0 (socket says 172.21.15.92:52844) 2023-10-06T10:41:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.114+0000 7f4232d9d700 1 -- 172.21.15.92:0/986127105 learned_addr learned my addr 172.21.15.92:0/986127105 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.114+0000 7f423259c700 1 --2- 172.21.15.92:0/986127105 >> v2:172.21.15.124:3300/0 conn(0x7f42341593c0 0x7f423406a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.114+0000 7f4231d9b700 1 -- 172.21.15.92:0/986127105 >> v2:172.21.15.124:3300/0 conn(0x7f42341593c0 msgr2=0x7f423406a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.114+0000 7f4231d9b700 1 --2- 172.21.15.92:0/986127105 >> v2:172.21.15.124:3300/0 conn(0x7f42341593c0 0x7f423406a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.114+0000 7f4231d9b700 1 -- 172.21.15.92:0/986127105 >> v1:172.21.15.92:6789/0 conn(0x7f42340699d0 legacy=0x7f4234069df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.114+0000 7f4231d9b700 1 -- 172.21.15.92:0/986127105 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42340675b0 con 0x7f42340cc240 2023-10-06T10:41:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.114+0000 7f423159a700 1 -- 172.21.15.92:0/986127105 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083072919 0 0) 0x7f4234067ee0 con 0x7f42340699d0 2023-10-06T10:41:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.114+0000 7f423259c700 1 --2- 172.21.15.92:0/986127105 >> v2:172.21.15.124:3300/0 conn(0x7f42341593c0 0x7f423406a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:34.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.114+0000 7f4231d9b700 1 --2- 172.21.15.92:0/986127105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42340cc240 0x7f42340c6da0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f42280098e0 tx=0x7f42280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e02dc7ea3ce8ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:34.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.114+0000 7f423159a700 1 -- 172.21.15.92:0/986127105 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228009e80 con 0x7f42340cc240 2023-10-06T10:41:34.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.114+0000 7f423159a700 1 -- 172.21.15.92:0/986127105 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422800b820 con 0x7f42340cc240 2023-10-06T10:41:34.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.114+0000 7f423159a700 1 -- 172.21.15.92:0/986127105 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422800bba0 con 0x7f42340cc240 2023-10-06T10:41:34.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.115+0000 7f423359e700 1 -- 172.21.15.92:0/986127105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42340cc240 msgr2=0x7f42340c6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:34.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.115+0000 7f423359e700 1 --2- 172.21.15.92:0/986127105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42340cc240 0x7f42340c6da0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f42280098e0 tx=0x7f42280091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.115+0000 7f423359e700 1 -- 172.21.15.92:0/986127105 shutdown_connections 2023-10-06T10:41:34.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.115+0000 7f423359e700 1 --2- 172.21.15.92:0/986127105 >> v2:172.21.15.124:3300/0 conn(0x7f42341593c0 0x7f423406a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.115+0000 7f423359e700 1 --2- 172.21.15.92:0/986127105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42340cc240 0x7f42340c6da0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.115+0000 7f423359e700 1 -- 172.21.15.92:0/986127105 >> 172.21.15.92:0/986127105 conn(0x7f42340cd1e0 msgr2=0x7f42340c91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:34.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.115+0000 7f423359e700 1 -- 172.21.15.92:0/986127105 shutdown_connections 2023-10-06T10:41:34.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.115+0000 7f423359e700 1 -- 172.21.15.92:0/986127105 wait complete. 2023-10-06T10:41:34.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.116+0000 7f423359e700 1 Processor -- start 2023-10-06T10:41:34.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.116+0000 7f423359e700 1 -- start start 2023-10-06T10:41:34.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.117+0000 7f423359e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42340699d0 0x7f423415b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:34.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.117+0000 7f423359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42340cc240 0x7f4234155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:34.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.117+0000 7f423359e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42341593c0 0x7f42340c24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:34.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.117+0000 7f423359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423415b340 con 0x7f42340cc240 2023-10-06T10:41:34.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.117+0000 7f423359e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42340b0520 con 0x7f42340699d0 2023-10-06T10:41:34.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.117+0000 7f423359e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423415acf0 con 0x7f42341593c0 2023-10-06T10:41:34.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.117+0000 7f4231d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42340cc240 0x7f4234155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:34.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.117+0000 7f4231d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42340cc240 0x7f4234155060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52846/0 (socket says 172.21.15.92:52846) 2023-10-06T10:41:34.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.117+0000 7f4231d9b700 1 -- 172.21.15.92:0/211167355 learned_addr learned my addr 172.21.15.92:0/211167355 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:34.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.117+0000 7f423259c700 1 --2- 172.21.15.92:0/211167355 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42340699d0 0x7f423415b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:34.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.117+0000 7f4232d9d700 1 --2- 172.21.15.92:0/211167355 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42341593c0 0x7f42340c24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:34.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.117+0000 7f4231d9b700 1 -- 172.21.15.92:0/211167355 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42341593c0 msgr2=0x7f42340c24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:34.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.117+0000 7f4231d9b700 1 --2- 172.21.15.92:0/211167355 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42341593c0 0x7f42340c24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.117+0000 7f4231d9b700 1 -- 172.21.15.92:0/211167355 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42340699d0 msgr2=0x7f423415b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:34.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.117+0000 7f4231d9b700 1 --2- 172.21.15.92:0/211167355 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42340699d0 0x7f423415b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.117+0000 7f4231d9b700 1 -- 172.21.15.92:0/211167355 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4228009420 con 0x7f42340cc240 2023-10-06T10:41:34.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.117+0000 7f4231d9b700 1 --2- 172.21.15.92:0/211167355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42340cc240 0x7f4234155060 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4234069550 tx=0x7f4228017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:34.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.118+0000 7f42227fc700 1 -- 172.21.15.92:0/211167355 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228011420 con 0x7f42340cc240 2023-10-06T10:41:34.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.118+0000 7f42227fc700 1 -- 172.21.15.92:0/211167355 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42280115a0 con 0x7f42340cc240 2023-10-06T10:41:34.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.118+0000 7f423359e700 1 -- 172.21.15.92:0/211167355 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423415afd0 con 0x7f42340cc240 2023-10-06T10:41:34.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.118+0000 7f42227fc700 1 -- 172.21.15.92:0/211167355 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228011960 con 0x7f42340cc240 2023-10-06T10:41:34.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.118+0000 7f423359e700 1 -- 172.21.15.92:0/211167355 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423415bd10 con 0x7f42340cc240 2023-10-06T10:41:34.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.119+0000 7f423359e700 1 -- 172.21.15.92:0/211167355 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f423415ab60 con 0x7f42340cc240 2023-10-06T10:41:34.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.119+0000 7f42227fc700 1 -- 172.21.15.92:0/211167355 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4228011cd0 con 0x7f42340cc240 2023-10-06T10:41:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.119+0000 7f42227fc700 1 --2- 172.21.15.92:0/211167355 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4214066f30 0x7f42140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.120+0000 7f423259c700 1 --2- 172.21.15.92:0/211167355 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4214066f30 0x7f42140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.120+0000 7f42227fc700 1 -- 172.21.15.92:0/211167355 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f4228086350 con 0x7f42340cc240 2023-10-06T10:41:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.120+0000 7f423259c700 1 --2- 172.21.15.92:0/211167355 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4214066f30 0x7f42140693e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4224006fd0 tx=0x7f4224008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.124+0000 7f42227fc700 1 -- 172.21.15.92:0/211167355 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4228053320 con 0x7f42340cc240 2023-10-06T10:41:34.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.305+0000 7f423359e700 1 -- 172.21.15.92:0/211167355 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f423405f230 con 0x7f42340cc240 2023-10-06T10:41:34.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.306+0000 7f42227fc700 1 -- 172.21.15.92:0/211167355 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4228056940 con 0x7f42340cc240 2023-10-06T10:41:34.307 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T10:41:34.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.308+0000 7f423359e700 1 -- 172.21.15.92:0/211167355 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4214066f30 msgr2=0x7f42140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:34.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.308+0000 7f423359e700 1 --2- 172.21.15.92:0/211167355 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4214066f30 0x7f42140693e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4224006fd0 tx=0x7f4224008040 comp rx=0 tx=0).stop 2023-10-06T10:41:34.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.308+0000 7f423359e700 1 -- 172.21.15.92:0/211167355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42340cc240 msgr2=0x7f4234155060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:34.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.308+0000 7f423359e700 1 --2- 172.21.15.92:0/211167355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42340cc240 0x7f4234155060 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4234069550 tx=0x7f4228017000 comp rx=0 tx=0).stop 2023-10-06T10:41:34.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.308+0000 7f423359e700 1 -- 172.21.15.92:0/211167355 shutdown_connections 2023-10-06T10:41:34.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.308+0000 7f423359e700 1 --2- 172.21.15.92:0/211167355 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4214066f30 0x7f42140693e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.308+0000 7f423359e700 1 --2- 172.21.15.92:0/211167355 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42341593c0 0x7f42340c24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.308+0000 7f423359e700 1 --2- 172.21.15.92:0/211167355 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42340699d0 0x7f423415b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.308+0000 7f423359e700 1 --2- 172.21.15.92:0/211167355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42340cc240 0x7f4234155060 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.308+0000 7f423359e700 1 -- 172.21.15.92:0/211167355 >> 172.21.15.92:0/211167355 conn(0x7f42340cd1e0 msgr2=0x7f4234156e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:34.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.308+0000 7f423359e700 1 -- 172.21.15.92:0/211167355 shutdown_connections 2023-10-06T10:41:34.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.308+0000 7f423359e700 1 -- 172.21.15.92:0/211167355 wait complete. 2023-10-06T10:41:34.328 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:41:34.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.442+0000 7fc69cda6700 1 Processor -- start 2023-10-06T10:41:34.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.442+0000 7fc69cda6700 1 -- start start 2023-10-06T10:41:34.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.443+0000 7fc69cda6700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc6980b29b0 0x7fc698159120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:34.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.443+0000 7fc69cda6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6980b0da0 con 0x7fc698159660 2023-10-06T10:41:34.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.443+0000 7fc69cda6700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6980cbfa0 con 0x7fc6980b25e0 2023-10-06T10:41:34.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.443+0000 7fc69cda6700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6980cb390 con 0x7fc6980b29b0 2023-10-06T10:41:34.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.443+0000 7fc696d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc698159660 0x7fc6980cc430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47440/0 (socket says 172.21.15.92:47440) 2023-10-06T10:41:34.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.443+0000 7fc696d9d700 1 -- 172.21.15.92:0/3489040626 learned_addr learned my addr 172.21.15.92:0/3489040626 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:34.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc68dd9b700 1 --2- 172.21.15.92:0/3489040626 >> v2:172.21.15.124:3300/0 conn(0x7fc6980b29b0 0x7fc698159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:34.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc695d9b700 1 -- 172.21.15.92:0/3489040626 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2235034203 0 0) 0x7fc690001490 con 0x7fc698159660 2023-10-06T10:41:34.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc695d9b700 1 -- 172.21.15.92:0/3489040626 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc684003570 con 0x7fc698159660 2023-10-06T10:41:34.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc695d9b700 1 -- 172.21.15.92:0/3489040626 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1530499895 0 0) 0x7fc684003570 con 0x7fc698159660 2023-10-06T10:41:34.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc695d9b700 1 -- 172.21.15.92:0/3489040626 >> v2:172.21.15.124:3300/0 conn(0x7fc6980b29b0 msgr2=0x7fc698159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:34.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc695d9b700 1 --2- 172.21.15.92:0/3489040626 >> v2:172.21.15.124:3300/0 conn(0x7fc6980b29b0 0x7fc698159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc695d9b700 1 -- 172.21.15.92:0/3489040626 >> v1:172.21.15.124:6789/0 conn(0x7fc6980b25e0 legacy=0x7fc6980b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:34.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc695d9b700 1 -- 172.21.15.92:0/3489040626 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6980cb670 con 0x7fc698159660 2023-10-06T10:41:34.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc695d9b700 1 -- 172.21.15.92:0/3489040626 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fc690001980 con 0x7fc698159660 2023-10-06T10:41:34.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc695d9b700 1 -- 172.21.15.92:0/3489040626 >> v1:172.21.15.92:6789/0 conn(0x7fc698159660 legacy=0x7fc6980cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:34.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc695d9b700 1 --2- 172.21.15.92:0/3489040626 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc684003a60 0x7fc684003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:34.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc695d9b700 1 --2- 172.21.15.92:0/3489040626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc684004810 0x7fc684004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:34.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc695d9b700 1 --2- 172.21.15.92:0/3489040626 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6840051c0 0x7fc68400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:34.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc695d9b700 1 -- 172.21.15.92:0/3489040626 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6980cb390 con 0x7fc684004810 2023-10-06T10:41:34.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc695d9b700 1 -- 172.21.15.92:0/3489040626 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68400c0d0 con 0x7fc684003a60 2023-10-06T10:41:34.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc695d9b700 1 -- 172.21.15.92:0/3489040626 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68400c290 con 0x7fc6840051c0 2023-10-06T10:41:34.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc68dd9b700 1 --2- 172.21.15.92:0/3489040626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc684004810 0x7fc684004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:34.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc69659c700 1 --2- 172.21.15.92:0/3489040626 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc684003a60 0x7fc684003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:34.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc696d9d700 1 --2- 172.21.15.92:0/3489040626 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6840051c0 0x7fc68400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:34.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc68dd9b700 1 -- 172.21.15.92:0/3489040626 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6840051c0 msgr2=0x7fc68400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:34.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc68dd9b700 1 --2- 172.21.15.92:0/3489040626 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6840051c0 0x7fc68400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc68dd9b700 1 -- 172.21.15.92:0/3489040626 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc684003a60 msgr2=0x7fc684003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:34.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc68dd9b700 1 --2- 172.21.15.92:0/3489040626 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc684003a60 0x7fc684003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc68dd9b700 1 -- 172.21.15.92:0/3489040626 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc684003570 con 0x7fc684004810 2023-10-06T10:41:34.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc696d9d700 1 --2- 172.21.15.92:0/3489040626 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6840051c0 0x7fc68400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:34.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.444+0000 7fc696d9d700 1 -- 172.21.15.92:0/3489040626 reap_dead start 2023-10-06T10:41:34.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.445+0000 7fc68dd9b700 1 --2- 172.21.15.92:0/3489040626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc684004810 0x7fc684004c80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc680009c00 tx=0x7fc6800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5548d8d8ee0717ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:34.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.445+0000 7fc695d9b700 1 -- 172.21.15.92:0/3489040626 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68000e710 con 0x7fc684004810 2023-10-06T10:41:34.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.445+0000 7fc695d9b700 1 -- 172.21.15.92:0/3489040626 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68000e870 con 0x7fc684004810 2023-10-06T10:41:34.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.445+0000 7fc695d9b700 1 -- 172.21.15.92:0/3489040626 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68000ec30 con 0x7fc684004810 2023-10-06T10:41:34.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.445+0000 7fc69cda6700 1 -- 172.21.15.92:0/3489040626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc684004810 msgr2=0x7fc684004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:34.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.445+0000 7fc69cda6700 1 --2- 172.21.15.92:0/3489040626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc684004810 0x7fc684004c80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc680009c00 tx=0x7fc6800091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.445+0000 7fc69cda6700 1 -- 172.21.15.92:0/3489040626 shutdown_connections 2023-10-06T10:41:34.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.445+0000 7fc69cda6700 1 --2- 172.21.15.92:0/3489040626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc684004810 0x7fc684004c80 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.445+0000 7fc69cda6700 1 -- 172.21.15.92:0/3489040626 >> 172.21.15.92:0/3489040626 conn(0x7fc69806ece0 msgr2=0x7fc69806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:34.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.445+0000 7fc69cda6700 1 -- 172.21.15.92:0/3489040626 shutdown_connections 2023-10-06T10:41:34.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.445+0000 7fc69cda6700 1 -- 172.21.15.92:0/3489040626 wait complete. 2023-10-06T10:41:34.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.446+0000 7fc69cda6700 1 Processor -- start 2023-10-06T10:41:34.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.446+0000 7fc69cda6700 1 -- start start 2023-10-06T10:41:34.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.447+0000 7fc69cda6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc684004810 0x7fc698155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:34.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.447+0000 7fc69cda6700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6980c7e90 0x7fc6980c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:34.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.447+0000 7fc69cda6700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc698151c80 0x7fc6980b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.447+0000 7fc69cda6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69815ce20 con 0x7fc684004810 2023-10-06T10:41:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.447+0000 7fc69cda6700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6980c8ff0 con 0x7fc698151c80 2023-10-06T10:41:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.447+0000 7fc69cda6700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6980c83e0 con 0x7fc6980c7e90 2023-10-06T10:41:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.447+0000 7fc69659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc684004810 0x7fc698155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.447+0000 7fc69659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc684004810 0x7fc698155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52876/0 (socket says 172.21.15.92:52876) 2023-10-06T10:41:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.447+0000 7fc69659c700 1 -- 172.21.15.92:0/152585525 learned_addr learned my addr 172.21.15.92:0/152585525 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.447+0000 7fc68dd9b700 1 --2- 172.21.15.92:0/152585525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6980c7e90 0x7fc6980c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.447+0000 7fc696d9d700 1 --2- 172.21.15.92:0/152585525 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc698151c80 0x7fc6980b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.447+0000 7fc69659c700 1 -- 172.21.15.92:0/152585525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6980c7e90 msgr2=0x7fc6980c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.447+0000 7fc69659c700 1 --2- 172.21.15.92:0/152585525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6980c7e90 0x7fc6980c6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.447+0000 7fc69659c700 1 -- 172.21.15.92:0/152585525 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc698151c80 msgr2=0x7fc6980b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.447+0000 7fc69659c700 1 --2- 172.21.15.92:0/152585525 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc698151c80 0x7fc6980b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.447+0000 7fc69659c700 1 -- 172.21.15.92:0/152585525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6980c86c0 con 0x7fc684004810 2023-10-06T10:41:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.448+0000 7fc69659c700 1 --2- 172.21.15.92:0/152585525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc684004810 0x7fc698155790 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fc688002920 tx=0x7fc688007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.448+0000 7fc68effd700 1 -- 172.21.15.92:0/152585525 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6880033e0 con 0x7fc684004810 2023-10-06T10:41:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.448+0000 7fc68effd700 1 -- 172.21.15.92:0/152585525 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68800f8d0 con 0x7fc684004810 2023-10-06T10:41:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.448+0000 7fc68effd700 1 -- 172.21.15.92:0/152585525 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68800fc70 con 0x7fc684004810 2023-10-06T10:41:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.448+0000 7fc69cda6700 1 -- 172.21.15.92:0/152585525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6980c89a0 con 0x7fc684004810 2023-10-06T10:41:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.448+0000 7fc69cda6700 1 -- 172.21.15.92:0/152585525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6980c6cc0 con 0x7fc684004810 2023-10-06T10:41:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.449+0000 7fc69cda6700 1 -- 172.21.15.92:0/152585525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc698064a90 con 0x7fc684004810 2023-10-06T10:41:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.449+0000 7fc68effd700 1 -- 172.21.15.92:0/152585525 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc688006870 con 0x7fc684004810 2023-10-06T10:41:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.449+0000 7fc68effd700 1 --2- 172.21.15.92:0/152585525 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc678066f60 0x7fc678069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.450+0000 7fc68effd700 1 -- 172.21.15.92:0/152585525 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc688084890 con 0x7fc684004810 2023-10-06T10:41:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.450+0000 7fc68dd9b700 1 --2- 172.21.15.92:0/152585525 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc678066f60 0x7fc678069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:34.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.450+0000 7fc68dd9b700 1 --2- 172.21.15.92:0/152585525 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc678066f60 0x7fc678069410 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc6800091e0 tx=0x7fc68001a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:34.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.455+0000 7fc68effd700 1 -- 172.21.15.92:0/152585525 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc688051860 con 0x7fc684004810 2023-10-06T10:41:34.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.683+0000 7fc69cda6700 1 -- 172.21.15.92:0/152585525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6980cb950 con 0x7fc684004810 2023-10-06T10:41:34.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.683+0000 7fc68effd700 1 -- 172.21.15.92:0/152585525 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4637 (secure 0 0 0) 0x7fc688054e80 con 0x7fc684004810 2023-10-06T10:41:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:34.684 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3668157650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3668157650},{"type":"v1","addr":"172.21.15.124:6834","nonce":3668157650}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:40:34.932631+0000","modified":"2023-10-06T10:41:17.396042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5736,"mds_1":5733,"mds_2":5036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5036":{"gid":5036,"name":"d","rank":2,"incarnation":36,"state":"up:active","state_seq":13,"addr":"172.21.15.124:6835/2986058955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2986058955},{"type":"v1","addr":"172.21.15.124:6835","nonce":2986058955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5733":{"gid":5733,"name":"c","rank":1,"incarnation":34,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6836/831923836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":831923836},{"type":"v1","addr":"172.21.15.92:6836","nonce":831923836}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5736":{"gid":5736,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6837/2458340622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2458340622},{"type":"v1","addr":"172.21.15.92:6837","nonce":2458340622}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T10:41:34.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.685+0000 7fc69cda6700 1 -- 172.21.15.92:0/152585525 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc678066f60 msgr2=0x7fc678069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:34.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.685+0000 7fc69cda6700 1 --2- 172.21.15.92:0/152585525 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc678066f60 0x7fc678069410 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc6800091e0 tx=0x7fc68001a000 comp rx=0 tx=0).stop 2023-10-06T10:41:34.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.686+0000 7fc69cda6700 1 -- 172.21.15.92:0/152585525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc684004810 msgr2=0x7fc698155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:34.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.686+0000 7fc69cda6700 1 --2- 172.21.15.92:0/152585525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc684004810 0x7fc698155790 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fc688002920 tx=0x7fc688007d80 comp rx=0 tx=0).stop 2023-10-06T10:41:34.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.686+0000 7fc69cda6700 1 -- 172.21.15.92:0/152585525 shutdown_connections 2023-10-06T10:41:34.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.686+0000 7fc69cda6700 1 --2- 172.21.15.92:0/152585525 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc678066f60 0x7fc678069410 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.686+0000 7fc69cda6700 1 --2- 172.21.15.92:0/152585525 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc698151c80 0x7fc6980b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.686+0000 7fc69cda6700 1 --2- 172.21.15.92:0/152585525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc684004810 0x7fc698155790 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.686+0000 7fc69cda6700 1 --2- 172.21.15.92:0/152585525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6980c7e90 0x7fc6980c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.686+0000 7fc69cda6700 1 -- 172.21.15.92:0/152585525 >> 172.21.15.92:0/152585525 conn(0x7fc69806ece0 msgr2=0x7fc698153ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:34.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.686+0000 7fc69cda6700 1 -- 172.21.15.92:0/152585525 shutdown_connections 2023-10-06T10:41:34.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.686+0000 7fc69cda6700 1 -- 172.21.15.92:0/152585525 wait complete. 2023-10-06T10:41:34.688 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 42 2023-10-06T10:41:34.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:41:34.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.811+0000 7fe72a6ae700 1 Processor -- start 2023-10-06T10:41:34.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.811+0000 7fe72a6ae700 1 -- start start 2023-10-06T10:41:34.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.812+0000 7fe72a6ae700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe724159440 0x7fe7240b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:34.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.812+0000 7fe72a6ae700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7240b35d0 0x7fe7240cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:34.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.812+0000 7fe72a6ae700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7240cb7a0 con 0x7fe7240cc170 2023-10-06T10:41:34.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.812+0000 7fe72a6ae700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7240b0860 con 0x7fe724159440 2023-10-06T10:41:34.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.812+0000 7fe72a6ae700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7240cb150 con 0x7fe7240b35d0 2023-10-06T10:41:34.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.812+0000 7fe7296ac700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe724159440 0x7fe7240b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:34.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.812+0000 7fe7296ac700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe724159440 0x7fe7240b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52880/0 (socket says 172.21.15.92:52880) 2023-10-06T10:41:34.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.812+0000 7fe7296ac700 1 -- 172.21.15.92:0/2666006056 learned_addr learned my addr 172.21.15.92:0/2666006056 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:34.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.812+0000 7fe728eab700 1 --2- 172.21.15.92:0/2666006056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7240b35d0 0x7fe7240cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:34.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.812+0000 7fe7296ac700 1 -- 172.21.15.92:0/2666006056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7240b35d0 msgr2=0x7fe7240cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:34.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.812+0000 7fe7296ac700 1 --2- 172.21.15.92:0/2666006056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7240b35d0 0x7fe7240cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.812+0000 7fe7296ac700 1 -- 172.21.15.92:0/2666006056 >> v1:172.21.15.124:6789/0 conn(0x7fe7240cc170 legacy=0x7fe7240b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:34.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.812+0000 7fe7296ac700 1 -- 172.21.15.92:0/2666006056 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7240cb430 con 0x7fe724159440 2023-10-06T10:41:34.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.813+0000 7fe7296ac700 1 --2- 172.21.15.92:0/2666006056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724159440 0x7fe7240b3090 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe720009f20 tx=0x7fe720009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de785159e06c06b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:34.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.813+0000 7fe71bfff700 1 -- 172.21.15.92:0/2666006056 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72000e070 con 0x7fe724159440 2023-10-06T10:41:34.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.813+0000 7fe71bfff700 1 -- 172.21.15.92:0/2666006056 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72000ba80 con 0x7fe724159440 2023-10-06T10:41:34.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.813+0000 7fe71bfff700 1 -- 172.21.15.92:0/2666006056 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7200092e0 con 0x7fe724159440 2023-10-06T10:41:34.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.813+0000 7fe72a6ae700 1 -- 172.21.15.92:0/2666006056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724159440 msgr2=0x7fe7240b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:34.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.813+0000 7fe72a6ae700 1 --2- 172.21.15.92:0/2666006056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724159440 0x7fe7240b3090 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe720009f20 tx=0x7fe720009840 comp rx=0 tx=0).stop 2023-10-06T10:41:34.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.813+0000 7fe72a6ae700 1 -- 172.21.15.92:0/2666006056 shutdown_connections 2023-10-06T10:41:34.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.813+0000 7fe72a6ae700 1 --2- 172.21.15.92:0/2666006056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724159440 0x7fe7240b3090 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.813+0000 7fe72a6ae700 1 --2- 172.21.15.92:0/2666006056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7240b35d0 0x7fe7240cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.813+0000 7fe72a6ae700 1 -- 172.21.15.92:0/2666006056 >> 172.21.15.92:0/2666006056 conn(0x7fe72406ece0 msgr2=0x7fe72406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:34.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.813+0000 7fe72a6ae700 1 -- 172.21.15.92:0/2666006056 shutdown_connections 2023-10-06T10:41:34.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.813+0000 7fe72a6ae700 1 -- 172.21.15.92:0/2666006056 wait complete. 2023-10-06T10:41:34.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.814+0000 7fe72a6ae700 1 Processor -- start 2023-10-06T10:41:34.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.814+0000 7fe72a6ae700 1 -- start start 2023-10-06T10:41:34.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.815+0000 7fe72a6ae700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7240b35d0 0x7fe7240c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:34.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.815+0000 7fe72a6ae700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7240cc170 0x7fe72415e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:34.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.815+0000 7fe72a6ae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724159440 0x7fe7241547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:34.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.815+0000 7fe72a6ae700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7240c77e0 con 0x7fe724159440 2023-10-06T10:41:34.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.815+0000 7fe72a6ae700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7240c7ac0 con 0x7fe7240b35d0 2023-10-06T10:41:34.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.815+0000 7fe72a6ae700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7240c7da0 con 0x7fe7240cc170 2023-10-06T10:41:34.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.815+0000 7fe729ead700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724159440 0x7fe7241547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:34.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.815+0000 7fe729ead700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724159440 0x7fe7241547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52884/0 (socket says 172.21.15.92:52884) 2023-10-06T10:41:34.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.815+0000 7fe729ead700 1 -- 172.21.15.92:0/728674034 learned_addr learned my addr 172.21.15.92:0/728674034 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:34.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.815+0000 7fe7296ac700 1 --2- 172.21.15.92:0/728674034 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7240b35d0 0x7fe7240c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:34.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.816+0000 7fe728eab700 1 --2- 172.21.15.92:0/728674034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7240cc170 0x7fe72415e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:34.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.816+0000 7fe729ead700 1 -- 172.21.15.92:0/728674034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7240cc170 msgr2=0x7fe72415e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:34.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.816+0000 7fe729ead700 1 --2- 172.21.15.92:0/728674034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7240cc170 0x7fe72415e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.816+0000 7fe729ead700 1 -- 172.21.15.92:0/728674034 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7240b35d0 msgr2=0x7fe7240c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:34.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.816+0000 7fe729ead700 1 --2- 172.21.15.92:0/728674034 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7240b35d0 0x7fe7240c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.816+0000 7fe729ead700 1 -- 172.21.15.92:0/728674034 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe720009ae0 con 0x7fe724159440 2023-10-06T10:41:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.816+0000 7fe7296ac700 1 --2- 172.21.15.92:0/728674034 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7240b35d0 0x7fe7240c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.816+0000 7fe728eab700 1 --2- 172.21.15.92:0/728674034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7240cc170 0x7fe72415e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.816+0000 7fe729ead700 1 --2- 172.21.15.92:0/728674034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724159440 0x7fe7241547f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe71c00c560 tx=0x7fe71c007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.816+0000 7fe7197fa700 1 -- 172.21.15.92:0/728674034 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c011590 con 0x7fe724159440 2023-10-06T10:41:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.816+0000 7fe7197fa700 1 -- 172.21.15.92:0/728674034 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71c0116f0 con 0x7fe724159440 2023-10-06T10:41:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.816+0000 7fe72a6ae700 1 -- 172.21.15.92:0/728674034 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7240c8080 con 0x7fe724159440 2023-10-06T10:41:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.816+0000 7fe7197fa700 1 -- 172.21.15.92:0/728674034 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c011ab0 con 0x7fe724159440 2023-10-06T10:41:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.816+0000 7fe72a6ae700 1 -- 172.21.15.92:0/728674034 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7240c9270 con 0x7fe724159440 2023-10-06T10:41:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.817+0000 7fe72a6ae700 1 -- 172.21.15.92:0/728674034 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7240b61a0 con 0x7fe724159440 2023-10-06T10:41:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.817+0000 7fe7197fa700 1 -- 172.21.15.92:0/728674034 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe71c01b3b0 con 0x7fe724159440 2023-10-06T10:41:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.818+0000 7fe7197fa700 1 --2- 172.21.15.92:0/728674034 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe708066f30 0x7fe7080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:34.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.818+0000 7fe7197fa700 1 -- 172.21.15.92:0/728674034 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe71c083cf0 con 0x7fe724159440 2023-10-06T10:41:34.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.818+0000 7fe7296ac700 1 --2- 172.21.15.92:0/728674034 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe708066f30 0x7fe7080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:34.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.818+0000 7fe7296ac700 1 --2- 172.21.15.92:0/728674034 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe708066f30 0x7fe7080693e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe720009880 tx=0x7fe720018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:34.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:34.822+0000 7fe7197fa700 1 -- 172.21.15.92:0/728674034 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe71c088070 con 0x7fe724159440 2023-10-06T10:41:35.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.004+0000 7fe72a6ae700 1 -- 172.21.15.92:0/728674034 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe72405f230 con 0x7fe724159440 2023-10-06T10:41:35.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.005+0000 7fe7197fa700 1 -- 172.21.15.92:0/728674034 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+17071 (secure 0 0 0) 0x7fe71c017070 con 0x7fe724159440 2023-10-06T10:41:35.006 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:35.007 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":52,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:40:36.557340+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:40:32.570535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-06T10:40:33.886794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:41:35.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.008+0000 7fe72a6ae700 1 -- 172.21.15.92:0/728674034 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe708066f30 msgr2=0x7fe7080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:35.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.008+0000 7fe72a6ae700 1 --2- 172.21.15.92:0/728674034 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe708066f30 0x7fe7080693e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe720009880 tx=0x7fe720018000 comp rx=0 tx=0).stop 2023-10-06T10:41:35.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.008+0000 7fe72a6ae700 1 -- 172.21.15.92:0/728674034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724159440 msgr2=0x7fe7241547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:35.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.008+0000 7fe72a6ae700 1 --2- 172.21.15.92:0/728674034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724159440 0x7fe7241547f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe71c00c560 tx=0x7fe71c007ef0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.008+0000 7fe72a6ae700 1 -- 172.21.15.92:0/728674034 shutdown_connections 2023-10-06T10:41:35.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.008+0000 7fe72a6ae700 1 --2- 172.21.15.92:0/728674034 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe708066f30 0x7fe7080693e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.008+0000 7fe72a6ae700 1 --2- 172.21.15.92:0/728674034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724159440 0x7fe7241547f0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.008+0000 7fe72a6ae700 1 --2- 172.21.15.92:0/728674034 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7240b35d0 0x7fe7240c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.008+0000 7fe72a6ae700 1 --2- 172.21.15.92:0/728674034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7240cc170 0x7fe72415e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.008+0000 7fe72a6ae700 1 -- 172.21.15.92:0/728674034 >> 172.21.15.92:0/728674034 conn(0x7fe72406ece0 msgr2=0x7fe7240af090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:35.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.008+0000 7fe72a6ae700 1 -- 172.21.15.92:0/728674034 shutdown_connections 2023-10-06T10:41:35.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.008+0000 7fe72a6ae700 1 -- 172.21.15.92:0/728674034 wait complete. 2023-10-06T10:41:35.020 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T10:41:35.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:41:35.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.134+0000 7f542eb01700 1 Processor -- start 2023-10-06T10:41:35.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.134+0000 7f542eb01700 1 -- start start 2023-10-06T10:41:35.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.135+0000 7f542eb01700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5428159440 0x7f54280b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:35.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.135+0000 7f542eb01700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f54280b35d0 0x7f54280cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:35.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.135+0000 7f542eb01700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f54280cc170 0x7f54280b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:35.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.135+0000 7f542eb01700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54280cb7a0 con 0x7f54280cc170 2023-10-06T10:41:35.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.135+0000 7f542eb01700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54280b0860 con 0x7f54280b35d0 2023-10-06T10:41:35.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.135+0000 7f542eb01700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54280cb150 con 0x7f5428159440 2023-10-06T10:41:35.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.135+0000 7f542e300700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f54280cc170 0x7f54280b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:35.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.135+0000 7f542e300700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f54280cc170 0x7f54280b2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52894/0 (socket says 172.21.15.92:52894) 2023-10-06T10:41:35.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.135+0000 7f542e300700 1 -- 172.21.15.92:0/903717361 learned_addr learned my addr 172.21.15.92:0/903717361 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:35.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.135+0000 7f542daff700 1 --2- 172.21.15.92:0/903717361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5428159440 0x7f54280b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:35.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.135+0000 7f542d2fe700 1 --2- 172.21.15.92:0/903717361 >> v2:172.21.15.124:3300/0 conn(0x7f54280b35d0 0x7f54280cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:35.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.135+0000 7f542e300700 1 -- 172.21.15.92:0/903717361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5428159440 msgr2=0x7f54280b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:35.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.135+0000 7f542e300700 1 --2- 172.21.15.92:0/903717361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5428159440 0x7f54280b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.135+0000 7f542e300700 1 -- 172.21.15.92:0/903717361 >> v2:172.21.15.124:3300/0 conn(0x7f54280b35d0 msgr2=0x7f54280cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:35.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.135+0000 7f542e300700 1 --2- 172.21.15.92:0/903717361 >> v2:172.21.15.124:3300/0 conn(0x7f54280b35d0 0x7f54280cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.135+0000 7f542e300700 1 -- 172.21.15.92:0/903717361 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54280cb430 con 0x7f54280cc170 2023-10-06T10:41:35.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.135+0000 7f542d2fe700 1 --2- 172.21.15.92:0/903717361 >> v2:172.21.15.124:3300/0 conn(0x7f54280b35d0 0x7f54280cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:35.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.136+0000 7f542e300700 1 --2- 172.21.15.92:0/903717361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54280cc170 0x7f54280b2100 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5428069120 tx=0x7f5420009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604112119f0fd889 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:35.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.136+0000 7f542cafd700 1 -- 172.21.15.92:0/903717361 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542000e070 con 0x7f54280cc170 2023-10-06T10:41:35.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.136+0000 7f542cafd700 1 -- 172.21.15.92:0/903717361 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542000ba20 con 0x7f54280cc170 2023-10-06T10:41:35.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.136+0000 7f542cafd700 1 -- 172.21.15.92:0/903717361 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54200092e0 con 0x7f54280cc170 2023-10-06T10:41:35.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.136+0000 7f542eb01700 1 -- 172.21.15.92:0/903717361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54280cc170 msgr2=0x7f54280b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:35.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.136+0000 7f542eb01700 1 --2- 172.21.15.92:0/903717361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54280cc170 0x7f54280b2100 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5428069120 tx=0x7f5420009840 comp rx=0 tx=0).stop 2023-10-06T10:41:35.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.136+0000 7f542eb01700 1 -- 172.21.15.92:0/903717361 shutdown_connections 2023-10-06T10:41:35.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.136+0000 7f542eb01700 1 --2- 172.21.15.92:0/903717361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54280cc170 0x7f54280b2100 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.136+0000 7f542eb01700 1 --2- 172.21.15.92:0/903717361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5428159440 0x7f54280b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.136+0000 7f542eb01700 1 --2- 172.21.15.92:0/903717361 >> v2:172.21.15.124:3300/0 conn(0x7f54280b35d0 0x7f54280cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.136+0000 7f542eb01700 1 -- 172.21.15.92:0/903717361 >> 172.21.15.92:0/903717361 conn(0x7f542806ece0 msgr2=0x7f542806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:35.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.136+0000 7f542eb01700 1 -- 172.21.15.92:0/903717361 shutdown_connections 2023-10-06T10:41:35.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.136+0000 7f542eb01700 1 -- 172.21.15.92:0/903717361 wait complete. 2023-10-06T10:41:35.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.137+0000 7f542eb01700 1 Processor -- start 2023-10-06T10:41:35.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.137+0000 7f542eb01700 1 -- start start 2023-10-06T10:41:35.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.138+0000 7f542eb01700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54280b35d0 0x7f54281545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:35.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.138+0000 7f542eb01700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54280cc170 0x7f54280b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:35.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.138+0000 7f542eb01700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5428159440 0x7f54280b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:35.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.138+0000 7f542eb01700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54280c52d0 con 0x7f5428159440 2023-10-06T10:41:35.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.138+0000 7f542eb01700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542816d440 con 0x7f54280b35d0 2023-10-06T10:41:35.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.138+0000 7f542eb01700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542816cb10 con 0x7f54280cc170 2023-10-06T10:41:35.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.138+0000 7f542e300700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5428159440 0x7f54280b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:35.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.138+0000 7f542e300700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5428159440 0x7f54280b4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52896/0 (socket says 172.21.15.92:52896) 2023-10-06T10:41:35.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.138+0000 7f542e300700 1 -- 172.21.15.92:0/328942205 learned_addr learned my addr 172.21.15.92:0/328942205 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:35.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.138+0000 7f542daff700 1 --2- 172.21.15.92:0/328942205 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54280b35d0 0x7f54281545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:35.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.138+0000 7f542d2fe700 1 --2- 172.21.15.92:0/328942205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54280cc170 0x7f54280b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:35.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.139+0000 7f542e300700 1 -- 172.21.15.92:0/328942205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54280cc170 msgr2=0x7f54280b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:35.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.139+0000 7f542e300700 1 --2- 172.21.15.92:0/328942205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54280cc170 0x7f54280b4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.139+0000 7f542e300700 1 -- 172.21.15.92:0/328942205 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54280b35d0 msgr2=0x7f54281545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:35.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.139+0000 7f542e300700 1 --2- 172.21.15.92:0/328942205 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54280b35d0 0x7f54281545e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.139+0000 7f542e300700 1 -- 172.21.15.92:0/328942205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5420009ae0 con 0x7f5428159440 2023-10-06T10:41:35.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.139+0000 7f542e300700 1 --2- 172.21.15.92:0/328942205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5428159440 0x7f54280b4a40 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f542000b6c0 tx=0x7f5420018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:35.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.139+0000 7f541dffb700 1 -- 172.21.15.92:0/328942205 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54200032f0 con 0x7f5428159440 2023-10-06T10:41:35.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.139+0000 7f541dffb700 1 -- 172.21.15.92:0/328942205 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54200092e0 con 0x7f5428159440 2023-10-06T10:41:35.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.139+0000 7f542eb01700 1 -- 172.21.15.92:0/328942205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542816cdf0 con 0x7f5428159440 2023-10-06T10:41:35.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.139+0000 7f541dffb700 1 -- 172.21.15.92:0/328942205 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5420026460 con 0x7f5428159440 2023-10-06T10:41:35.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.139+0000 7f542eb01700 1 -- 172.21.15.92:0/328942205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5428154b20 con 0x7f5428159440 2023-10-06T10:41:35.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.140+0000 7f542eb01700 1 -- 172.21.15.92:0/328942205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54280ca9b0 con 0x7f5428159440 2023-10-06T10:41:35.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.140+0000 7f541dffb700 1 -- 172.21.15.92:0/328942205 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5420003040 con 0x7f5428159440 2023-10-06T10:41:35.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.141+0000 7f541dffb700 1 --2- 172.21.15.92:0/328942205 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f540c066f30 0x7f540c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:35.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.141+0000 7f541dffb700 1 -- 172.21.15.92:0/328942205 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f54200867f0 con 0x7f5428159440 2023-10-06T10:41:35.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.141+0000 7f542daff700 1 --2- 172.21.15.92:0/328942205 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f540c066f30 0x7f540c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:35.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.141+0000 7f542daff700 1 --2- 172.21.15.92:0/328942205 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f540c066f30 0x7f540c0693e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5424009d10 tx=0x7f54240091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:35.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.145+0000 7f541dffb700 1 -- 172.21.15.92:0/328942205 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5420053870 con 0x7f5428159440 2023-10-06T10:41:35.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.369+0000 7f542eb01700 1 -- 172.21.15.92:0/328942205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f542805f230 con 0x7f5428159440 2023-10-06T10:41:35.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.369+0000 7f541dffb700 1 -- 172.21.15.92:0/328942205 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4637 (secure 0 0 0) 0x7f5420056e90 con 0x7f5428159440 2023-10-06T10:41:35.370 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:35.370 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3668157650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3668157650},{"type":"v1","addr":"172.21.15.124:6834","nonce":3668157650}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:40:34.932631+0000","modified":"2023-10-06T10:41:17.396042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5736,"mds_1":5733,"mds_2":5036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5036":{"gid":5036,"name":"d","rank":2,"incarnation":36,"state":"up:active","state_seq":13,"addr":"172.21.15.124:6835/2986058955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2986058955},{"type":"v1","addr":"172.21.15.124:6835","nonce":2986058955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5733":{"gid":5733,"name":"c","rank":1,"incarnation":34,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6836/831923836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":831923836},{"type":"v1","addr":"172.21.15.92:6836","nonce":831923836}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5736":{"gid":5736,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6837/2458340622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2458340622},{"type":"v1","addr":"172.21.15.92:6837","nonce":2458340622}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T10:41:35.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.371+0000 7f542eb01700 1 -- 172.21.15.92:0/328942205 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f540c066f30 msgr2=0x7f540c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:35.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.371+0000 7f542eb01700 1 --2- 172.21.15.92:0/328942205 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f540c066f30 0x7f540c0693e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5424009d10 tx=0x7f54240091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.371+0000 7f542eb01700 1 -- 172.21.15.92:0/328942205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5428159440 msgr2=0x7f54280b4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:35.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.371+0000 7f542eb01700 1 --2- 172.21.15.92:0/328942205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5428159440 0x7f54280b4a40 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f542000b6c0 tx=0x7f5420018000 comp rx=0 tx=0).stop 2023-10-06T10:41:35.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.371+0000 7f542eb01700 1 -- 172.21.15.92:0/328942205 shutdown_connections 2023-10-06T10:41:35.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.371+0000 7f542eb01700 1 --2- 172.21.15.92:0/328942205 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f540c066f30 0x7f540c0693e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.371+0000 7f542eb01700 1 --2- 172.21.15.92:0/328942205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5428159440 0x7f54280b4a40 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.371+0000 7f542eb01700 1 --2- 172.21.15.92:0/328942205 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54280b35d0 0x7f54281545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.371+0000 7f542eb01700 1 --2- 172.21.15.92:0/328942205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54280cc170 0x7f54280b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.371+0000 7f542eb01700 1 -- 172.21.15.92:0/328942205 >> 172.21.15.92:0/328942205 conn(0x7f542806ece0 msgr2=0x7f5428154fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:35.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.371+0000 7f542eb01700 1 -- 172.21.15.92:0/328942205 shutdown_connections 2023-10-06T10:41:35.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.372+0000 7f542eb01700 1 -- 172.21.15.92:0/328942205 wait complete. 2023-10-06T10:41:35.374 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 42 2023-10-06T10:41:35.384 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:41:35.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.498+0000 7fa867198700 1 Processor -- start 2023-10-06T10:41:35.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.498+0000 7fa867198700 1 -- start start 2023-10-06T10:41:35.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.499+0000 7fa867198700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa860159560 0x7fa8600aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:35.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.499+0000 7fa867198700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8600b49a0 0x7fa8600aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:35.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.499+0000 7fa867198700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8600b0c60 con 0x7fa8600b0760 2023-10-06T10:41:35.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.499+0000 7fa867198700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8600b0f40 con 0x7fa860159560 2023-10-06T10:41:35.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.499+0000 7fa867198700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8600b1220 con 0x7fa8600b49a0 2023-10-06T10:41:35.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.499+0000 7fa864f34700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa860159560 0x7fa8600aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:35.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.499+0000 7fa865735700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa8600b0760 0x7fa8600b3890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47442/0 (socket says 172.21.15.92:47442) 2023-10-06T10:41:35.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.499+0000 7fa865735700 1 -- 172.21.15.92:0/1069108290 learned_addr learned my addr 172.21.15.92:0/1069108290 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:35.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.499+0000 7fa85ffff700 1 --2- 172.21.15.92:0/1069108290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8600b49a0 0x7fa8600aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:35.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.499+0000 7fa864f34700 1 -- 172.21.15.92:0/1069108290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8600b49a0 msgr2=0x7fa8600aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:35.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.499+0000 7fa864f34700 1 --2- 172.21.15.92:0/1069108290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8600b49a0 0x7fa8600aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.499+0000 7fa864f34700 1 -- 172.21.15.92:0/1069108290 >> v1:172.21.15.92:6789/0 conn(0x7fa8600b0760 legacy=0x7fa8600b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:35.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.499+0000 7fa864f34700 1 -- 172.21.15.92:0/1069108290 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8600b1500 con 0x7fa860159560 2023-10-06T10:41:35.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.499+0000 7fa85f7fe700 1 -- 172.21.15.92:0/1069108290 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1060694826 0 0) 0x7fa8600b0c60 con 0x7fa8600b0760 2023-10-06T10:41:35.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.499+0000 7fa864f34700 1 --2- 172.21.15.92:0/1069108290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa860159560 0x7fa8600aa530 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa8540098d0 tx=0x7fa854009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6caa67fc1c79ba8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:35.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.499+0000 7fa85f7fe700 1 -- 172.21.15.92:0/1069108290 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854009e70 con 0x7fa860159560 2023-10-06T10:41:35.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.499+0000 7fa85f7fe700 1 -- 172.21.15.92:0/1069108290 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa854005c10 con 0x7fa860159560 2023-10-06T10:41:35.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.499+0000 7fa85f7fe700 1 -- 172.21.15.92:0/1069108290 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85400b650 con 0x7fa860159560 2023-10-06T10:41:35.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.500+0000 7fa867198700 1 -- 172.21.15.92:0/1069108290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa860159560 msgr2=0x7fa8600aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:35.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.500+0000 7fa867198700 1 --2- 172.21.15.92:0/1069108290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa860159560 0x7fa8600aa530 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa8540098d0 tx=0x7fa854009120 comp rx=0 tx=0).stop 2023-10-06T10:41:35.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.500+0000 7fa867198700 1 -- 172.21.15.92:0/1069108290 shutdown_connections 2023-10-06T10:41:35.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.500+0000 7fa867198700 1 --2- 172.21.15.92:0/1069108290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa860159560 0x7fa8600aa530 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.500+0000 7fa867198700 1 --2- 172.21.15.92:0/1069108290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8600b49a0 0x7fa8600aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.500+0000 7fa867198700 1 -- 172.21.15.92:0/1069108290 >> 172.21.15.92:0/1069108290 conn(0x7fa8600997f0 msgr2=0x7fa86006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:35.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.500+0000 7fa867198700 1 -- 172.21.15.92:0/1069108290 shutdown_connections 2023-10-06T10:41:35.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.500+0000 7fa867198700 1 -- 172.21.15.92:0/1069108290 wait complete. 2023-10-06T10:41:35.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.501+0000 7fa867198700 1 Processor -- start 2023-10-06T10:41:35.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.501+0000 7fa867198700 1 -- start start 2023-10-06T10:41:35.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.502+0000 7fa867198700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8600b0760 0x7fa86015c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:35.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.502+0000 7fa867198700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8600b49a0 0x7fa860158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:35.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.502+0000 7fa867198700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa860159560 0x7fa8601562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:35.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.502+0000 7fa867198700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86009df80 con 0x7fa8600b0760 2023-10-06T10:41:35.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.502+0000 7fa867198700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86016c920 con 0x7fa8600b49a0 2023-10-06T10:41:35.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.502+0000 7fa867198700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86016cc00 con 0x7fa860159560 2023-10-06T10:41:35.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.502+0000 7fa864f34700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8600b0760 0x7fa86015c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:35.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.502+0000 7fa864f34700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8600b0760 0x7fa86015c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52914/0 (socket says 172.21.15.92:52914) 2023-10-06T10:41:35.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.502+0000 7fa864f34700 1 -- 172.21.15.92:0/3564855192 learned_addr learned my addr 172.21.15.92:0/3564855192 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:35.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.502+0000 7fa865735700 1 --2- 172.21.15.92:0/3564855192 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa860159560 0x7fa8601562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:35.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.502+0000 7fa864f34700 1 -- 172.21.15.92:0/3564855192 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa860159560 msgr2=0x7fa8601562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:35.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.502+0000 7fa85ffff700 1 --2- 172.21.15.92:0/3564855192 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8600b49a0 0x7fa860158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:35.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.502+0000 7fa864f34700 1 --2- 172.21.15.92:0/3564855192 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa860159560 0x7fa8601562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.502+0000 7fa864f34700 1 -- 172.21.15.92:0/3564855192 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8600b49a0 msgr2=0x7fa860158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:35.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.502+0000 7fa864f34700 1 --2- 172.21.15.92:0/3564855192 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8600b49a0 0x7fa860158790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.502+0000 7fa864f34700 1 -- 172.21.15.92:0/3564855192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8540093c0 con 0x7fa8600b0760 2023-10-06T10:41:35.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.502+0000 7fa865735700 1 --2- 172.21.15.92:0/3564855192 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa860159560 0x7fa8601562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:35.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.502+0000 7fa85ffff700 1 --2- 172.21.15.92:0/3564855192 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8600b49a0 0x7fa860158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:35.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.502+0000 7fa864f34700 1 --2- 172.21.15.92:0/3564855192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8600b0760 0x7fa86015c250 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa85400b3c0 tx=0x7fa854016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:35.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.503+0000 7fa85cff9700 1 -- 172.21.15.92:0/3564855192 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854006310 con 0x7fa8600b0760 2023-10-06T10:41:35.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.503+0000 7fa85cff9700 1 -- 172.21.15.92:0/3564855192 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa854006490 con 0x7fa8600b0760 2023-10-06T10:41:35.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.503+0000 7fa867198700 1 -- 172.21.15.92:0/3564855192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86016cee0 con 0x7fa8600b0760 2023-10-06T10:41:35.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.503+0000 7fa85cff9700 1 -- 172.21.15.92:0/3564855192 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854006850 con 0x7fa8600b0760 2023-10-06T10:41:35.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.503+0000 7fa867198700 1 -- 172.21.15.92:0/3564855192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86016d4a0 con 0x7fa8600b0760 2023-10-06T10:41:35.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.503+0000 7fa867198700 1 -- 172.21.15.92:0/3564855192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86016d6c0 con 0x7fa8600b0760 2023-10-06T10:41:35.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.504+0000 7fa85cff9700 1 -- 172.21.15.92:0/3564855192 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa85401b690 con 0x7fa8600b0760 2023-10-06T10:41:35.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.504+0000 7fa85cff9700 1 --2- 172.21.15.92:0/3564855192 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa844066f30 0x7fa8440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:35.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.504+0000 7fa85cff9700 1 -- 172.21.15.92:0/3564855192 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa854086920 con 0x7fa8600b0760 2023-10-06T10:41:35.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.504+0000 7fa85ffff700 1 --2- 172.21.15.92:0/3564855192 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa844066f30 0x7fa8440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:35.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.505+0000 7fa85ffff700 1 --2- 172.21.15.92:0/3564855192 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa844066f30 0x7fa8440693e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa8600b42c0 tx=0x7fa84c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:35.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.508+0000 7fa85cff9700 1 -- 172.21.15.92:0/3564855192 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8540538f0 con 0x7fa8600b0760 2023-10-06T10:41:35.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.734+0000 7fa867198700 1 -- 172.21.15.92:0/3564855192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa860064a90 con 0x7fa8600b0760 2023-10-06T10:41:35.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.735+0000 7fa85cff9700 1 -- 172.21.15.92:0/3564855192 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4637 (secure 0 0 0) 0x7fa854056f10 con 0x7fa8600b0760 2023-10-06T10:41:35.736 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:35.736 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3668157650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3668157650},{"type":"v1","addr":"172.21.15.124:6834","nonce":3668157650}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:40:34.932631+0000","modified":"2023-10-06T10:41:17.396042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5736,"mds_1":5733,"mds_2":5036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5036":{"gid":5036,"name":"d","rank":2,"incarnation":36,"state":"up:active","state_seq":13,"addr":"172.21.15.124:6835/2986058955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2986058955},{"type":"v1","addr":"172.21.15.124:6835","nonce":2986058955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5733":{"gid":5733,"name":"c","rank":1,"incarnation":34,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6836/831923836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":831923836},{"type":"v1","addr":"172.21.15.92:6836","nonce":831923836}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5736":{"gid":5736,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6837/2458340622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2458340622},{"type":"v1","addr":"172.21.15.92:6837","nonce":2458340622}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T10:41:35.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.737+0000 7fa867198700 1 -- 172.21.15.92:0/3564855192 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa844066f30 msgr2=0x7fa8440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:35.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.737+0000 7fa867198700 1 --2- 172.21.15.92:0/3564855192 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa844066f30 0x7fa8440693e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa8600b42c0 tx=0x7fa84c008040 comp rx=0 tx=0).stop 2023-10-06T10:41:35.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.737+0000 7fa867198700 1 -- 172.21.15.92:0/3564855192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8600b0760 msgr2=0x7fa86015c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:35.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.737+0000 7fa867198700 1 --2- 172.21.15.92:0/3564855192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8600b0760 0x7fa86015c250 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa85400b3c0 tx=0x7fa854016000 comp rx=0 tx=0).stop 2023-10-06T10:41:35.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.737+0000 7fa867198700 1 -- 172.21.15.92:0/3564855192 shutdown_connections 2023-10-06T10:41:35.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.737+0000 7fa867198700 1 --2- 172.21.15.92:0/3564855192 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa844066f30 0x7fa8440693e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.737+0000 7fa867198700 1 --2- 172.21.15.92:0/3564855192 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa860159560 0x7fa8601562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.737+0000 7fa867198700 1 --2- 172.21.15.92:0/3564855192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8600b0760 0x7fa86015c250 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.737+0000 7fa867198700 1 --2- 172.21.15.92:0/3564855192 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8600b49a0 0x7fa860158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.737+0000 7fa867198700 1 -- 172.21.15.92:0/3564855192 >> 172.21.15.92:0/3564855192 conn(0x7fa8600997f0 msgr2=0x7fa860069630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:35.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.737+0000 7fa867198700 1 -- 172.21.15.92:0/3564855192 shutdown_connections 2023-10-06T10:41:35.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.737+0000 7fa867198700 1 -- 172.21.15.92:0/3564855192 wait complete. 2023-10-06T10:41:35.739 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 42 2023-10-06T10:41:35.750 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:41:35.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.863+0000 7f4ef141a700 1 Processor -- start 2023-10-06T10:41:35.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.863+0000 7f4ef141a700 1 -- start start 2023-10-06T10:41:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.864+0000 7f4ef141a700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4eec069a60 0x7f4eec06cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.864+0000 7f4ef141a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eec159f60 con 0x7f4eec06d120 2023-10-06T10:41:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.864+0000 7f4ef141a700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eec159980 con 0x7f4eec069690 2023-10-06T10:41:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.864+0000 7f4ef141a700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eec06d610 con 0x7f4eec069a60 2023-10-06T10:41:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.864+0000 7f4ef0c19700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4eec06d120 0x7f4eec0b11c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47456/0 (socket says 172.21.15.92:47456) 2023-10-06T10:41:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.864+0000 7f4ef0c19700 1 -- 172.21.15.92:0/2921659064 learned_addr learned my addr 172.21.15.92:0/2921659064 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.864+0000 7f4eeb7fe700 1 --2- 172.21.15.92:0/2921659064 >> v2:172.21.15.124:3300/0 conn(0x7f4eec069a60 0x7f4eec06cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.864+0000 7f4eeaffd700 1 -- 172.21.15.92:0/2921659064 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796093038 0 0) 0x7f4eec159f60 con 0x7f4eec06d120 2023-10-06T10:41:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.864+0000 7f4eeaffd700 1 -- 172.21.15.92:0/2921659064 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ed4003570 con 0x7f4eec06d120 2023-10-06T10:41:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.864+0000 7f4eeaffd700 1 -- 172.21.15.92:0/2921659064 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2714526666 0 0) 0x7f4ed4003570 con 0x7f4eec06d120 2023-10-06T10:41:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4eeaffd700 1 -- 172.21.15.92:0/2921659064 >> v2:172.21.15.124:3300/0 conn(0x7f4eec069a60 msgr2=0x7f4eec06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4eeaffd700 1 --2- 172.21.15.92:0/2921659064 >> v2:172.21.15.124:3300/0 conn(0x7f4eec069a60 0x7f4eec06cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4eeaffd700 1 -- 172.21.15.92:0/2921659064 >> v1:172.21.15.124:6789/0 conn(0x7f4eec069690 legacy=0x7f4eec15a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4eeaffd700 1 -- 172.21.15.92:0/2921659064 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eec06d8f0 con 0x7f4eec06d120 2023-10-06T10:41:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4eeaffd700 1 -- 172.21.15.92:0/2921659064 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f4ee40017f0 con 0x7f4eec06d120 2023-10-06T10:41:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4eeaffd700 1 -- 172.21.15.92:0/2921659064 >> v1:172.21.15.92:6789/0 conn(0x7f4eec06d120 legacy=0x7f4eec0b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:35.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4eeaffd700 1 --2- 172.21.15.92:0/2921659064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ed4003a60 0x7f4ed4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:35.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4eeaffd700 1 --2- 172.21.15.92:0/2921659064 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ed4004810 0x7f4ed4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:35.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4eeaffd700 1 --2- 172.21.15.92:0/2921659064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed40051c0 0x7f4ed400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:35.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4eeaffd700 1 -- 172.21.15.92:0/2921659064 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec06d610 con 0x7f4ed40051c0 2023-10-06T10:41:35.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4eeaffd700 1 -- 172.21.15.92:0/2921659064 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed400c0d0 con 0x7f4ed4004810 2023-10-06T10:41:35.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4eeaffd700 1 -- 172.21.15.92:0/2921659064 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed400c290 con 0x7f4ed4003a60 2023-10-06T10:41:35.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4ef0c19700 1 --2- 172.21.15.92:0/2921659064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed40051c0 0x7f4ed400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:35.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4eeb7fe700 1 --2- 172.21.15.92:0/2921659064 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ed4004810 0x7f4ed4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:35.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4eebfff700 1 --2- 172.21.15.92:0/2921659064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ed4003a60 0x7f4ed4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:35.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4ef0c19700 1 -- 172.21.15.92:0/2921659064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ed4003a60 msgr2=0x7f4ed4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:35.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4ef0c19700 1 --2- 172.21.15.92:0/2921659064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ed4003a60 0x7f4ed4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4ef0c19700 1 -- 172.21.15.92:0/2921659064 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ed4004810 msgr2=0x7f4ed4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:35.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4ef0c19700 1 --2- 172.21.15.92:0/2921659064 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ed4004810 0x7f4ed4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4ef0c19700 1 -- 172.21.15.92:0/2921659064 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed4003570 con 0x7f4ed40051c0 2023-10-06T10:41:35.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4ef0c19700 1 -- 172.21.15.92:0/2921659064 reap_dead start 2023-10-06T10:41:35.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4ef0c19700 1 --2- 172.21.15.92:0/2921659064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed40051c0 0x7f4ed400ba60 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4eec06d540 tx=0x7f4eec159ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30a309517d1a29f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:35.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4eeaffd700 1 -- 172.21.15.92:0/2921659064 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40070d0 con 0x7f4ed40051c0 2023-10-06T10:41:35.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4eeaffd700 1 -- 172.21.15.92:0/2921659064 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee4007230 con 0x7f4ed40051c0 2023-10-06T10:41:35.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.865+0000 7f4eeaffd700 1 -- 172.21.15.92:0/2921659064 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40075f0 con 0x7f4ed40051c0 2023-10-06T10:41:35.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.866+0000 7f4ef141a700 1 -- 172.21.15.92:0/2921659064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed40051c0 msgr2=0x7f4ed400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:35.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.866+0000 7f4ef141a700 1 --2- 172.21.15.92:0/2921659064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed40051c0 0x7f4ed400ba60 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4eec06d540 tx=0x7f4eec159ed0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.866+0000 7f4ef141a700 1 -- 172.21.15.92:0/2921659064 shutdown_connections 2023-10-06T10:41:35.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.866+0000 7f4ef141a700 1 --2- 172.21.15.92:0/2921659064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed40051c0 0x7f4ed400ba60 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.866+0000 7f4ef141a700 1 -- 172.21.15.92:0/2921659064 >> 172.21.15.92:0/2921659064 conn(0x7f4eec06b500 msgr2=0x7f4eec06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:35.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.866+0000 7f4ef141a700 1 -- 172.21.15.92:0/2921659064 shutdown_connections 2023-10-06T10:41:35.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.866+0000 7f4ef141a700 1 -- 172.21.15.92:0/2921659064 wait complete. 2023-10-06T10:41:35.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.867+0000 7f4ef141a700 1 Processor -- start 2023-10-06T10:41:35.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.867+0000 7f4ef141a700 1 -- start start 2023-10-06T10:41:35.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.868+0000 7f4ef141a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed40051c0 0x7f4eec16a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:35.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.868+0000 7f4ef141a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4eec1572e0 0x7f4eec0c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:35.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.868+0000 7f4ef141a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4eec156ae0 0x7f4eec0c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:35.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.868+0000 7f4ef141a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec0c94c0 con 0x7f4ed40051c0 2023-10-06T10:41:35.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.868+0000 7f4ef141a700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec0cb640 con 0x7f4eec1572e0 2023-10-06T10:41:35.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.868+0000 7f4ef141a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec0cac50 con 0x7f4eec156ae0 2023-10-06T10:41:35.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.868+0000 7f4eebfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed40051c0 0x7f4eec16a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:35.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.868+0000 7f4eebfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed40051c0 0x7f4eec16a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52930/0 (socket says 172.21.15.92:52930) 2023-10-06T10:41:35.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.868+0000 7f4eebfff700 1 -- 172.21.15.92:0/702365665 learned_addr learned my addr 172.21.15.92:0/702365665 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:35.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.868+0000 7f4ef0c19700 1 --2- 172.21.15.92:0/702365665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4eec156ae0 0x7f4eec0c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:35.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.868+0000 7f4eeb7fe700 1 --2- 172.21.15.92:0/702365665 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4eec1572e0 0x7f4eec0c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:35.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.868+0000 7f4eebfff700 1 -- 172.21.15.92:0/702365665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4eec156ae0 msgr2=0x7f4eec0c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:35.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.868+0000 7f4eebfff700 1 --2- 172.21.15.92:0/702365665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4eec156ae0 0x7f4eec0c9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.868+0000 7f4eebfff700 1 -- 172.21.15.92:0/702365665 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4eec1572e0 msgr2=0x7f4eec0c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:35.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.868+0000 7f4eebfff700 1 --2- 172.21.15.92:0/702365665 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4eec1572e0 0x7f4eec0c72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:35.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.868+0000 7f4eebfff700 1 -- 172.21.15.92:0/702365665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eec0c77e0 con 0x7f4ed40051c0 2023-10-06T10:41:35.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.868+0000 7f4eebfff700 1 --2- 172.21.15.92:0/702365665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed40051c0 0x7f4eec16a440 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f4edc002980 tx=0x7f4edc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:35.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.868+0000 7f4ed3fff700 1 -- 172.21.15.92:0/702365665 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc00f8d0 con 0x7f4ed40051c0 2023-10-06T10:41:35.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.868+0000 7f4ed3fff700 1 -- 172.21.15.92:0/702365665 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4edc00fa30 con 0x7f4ed40051c0 2023-10-06T10:41:35.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.868+0000 7f4ef141a700 1 -- 172.21.15.92:0/702365665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eec0c9710 con 0x7f4ed40051c0 2023-10-06T10:41:35.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.868+0000 7f4ed3fff700 1 -- 172.21.15.92:0/702365665 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc00fdf0 con 0x7f4ed40051c0 2023-10-06T10:41:35.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.868+0000 7f4ef141a700 1 -- 172.21.15.92:0/702365665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eec0cb890 con 0x7f4ed40051c0 2023-10-06T10:41:35.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.869+0000 7f4ef141a700 1 -- 172.21.15.92:0/702365665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eec0cbab0 con 0x7f4ed40051c0 2023-10-06T10:41:35.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.870+0000 7f4ed3fff700 1 -- 172.21.15.92:0/702365665 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4edc006a30 con 0x7f4ed40051c0 2023-10-06T10:41:35.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.870+0000 7f4ed3fff700 1 --2- 172.21.15.92:0/702365665 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4ed8066f30 0x7f4ed80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:35.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.870+0000 7f4ed3fff700 1 -- 172.21.15.92:0/702365665 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f4edc084930 con 0x7f4ed40051c0 2023-10-06T10:41:35.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.870+0000 7f4eeb7fe700 1 --2- 172.21.15.92:0/702365665 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4ed8066f30 0x7f4ed80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:35.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.871+0000 7f4eeb7fe700 1 --2- 172.21.15.92:0/702365665 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4ed8066f30 0x7f4ed80693e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0b14a0 tx=0x7f4ee0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:35.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:35.874+0000 7f4ed3fff700 1 -- 172.21.15.92:0/702365665 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4edc051900 con 0x7f4ed40051c0 2023-10-06T10:41:36.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.058+0000 7f4ef141a700 1 -- 172.21.15.92:0/702365665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4eec05f230 con 0x7f4ed40051c0 2023-10-06T10:41:36.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.059+0000 7f4ed3fff700 1 -- 172.21.15.92:0/702365665 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+17071 (secure 0 0 0) 0x7f4edc054f20 con 0x7f4ed40051c0 2023-10-06T10:41:36.060 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:36.060 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":52,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:40:36.557340+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:40:32.570535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-06T10:40:33.886794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:41:36.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.061+0000 7f4ef141a700 1 -- 172.21.15.92:0/702365665 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4ed8066f30 msgr2=0x7f4ed80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:36.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.061+0000 7f4ef141a700 1 --2- 172.21.15.92:0/702365665 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4ed8066f30 0x7f4ed80693e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0b14a0 tx=0x7f4ee0008040 comp rx=0 tx=0).stop 2023-10-06T10:41:36.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.061+0000 7f4ef141a700 1 -- 172.21.15.92:0/702365665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed40051c0 msgr2=0x7f4eec16a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:36.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.061+0000 7f4ef141a700 1 --2- 172.21.15.92:0/702365665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed40051c0 0x7f4eec16a440 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f4edc002980 tx=0x7f4edc007d30 comp rx=0 tx=0).stop 2023-10-06T10:41:36.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.061+0000 7f4ef141a700 1 -- 172.21.15.92:0/702365665 shutdown_connections 2023-10-06T10:41:36.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.061+0000 7f4ef141a700 1 --2- 172.21.15.92:0/702365665 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4ed8066f30 0x7f4ed80693e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:36.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.061+0000 7f4ef141a700 1 --2- 172.21.15.92:0/702365665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4eec156ae0 0x7f4eec0c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:36.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.061+0000 7f4ef141a700 1 --2- 172.21.15.92:0/702365665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed40051c0 0x7f4eec16a440 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:36.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.061+0000 7f4ef141a700 1 --2- 172.21.15.92:0/702365665 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4eec1572e0 0x7f4eec0c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:36.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.061+0000 7f4ef141a700 1 -- 172.21.15.92:0/702365665 >> 172.21.15.92:0/702365665 conn(0x7f4eec06b500 msgr2=0x7f4eec0ae240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:36.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.061+0000 7f4ef141a700 1 -- 172.21.15.92:0/702365665 shutdown_connections 2023-10-06T10:41:36.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.061+0000 7f4ef141a700 1 -- 172.21.15.92:0/702365665 wait complete. 2023-10-06T10:41:36.074 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T10:41:36.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.188+0000 7f970d1af700 1 Processor -- start 2023-10-06T10:41:36.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.188+0000 7f970d1af700 1 -- start start 2023-10-06T10:41:36.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.189+0000 7f970d1af700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9708159440 0x7f97080b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:36.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.189+0000 7f970d1af700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97080b2100 0x7f97080b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:36.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.189+0000 7f970d1af700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97080cb7a0 con 0x7f97080b35d0 2023-10-06T10:41:36.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.189+0000 7f970d1af700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97080b0860 con 0x7f9708159440 2023-10-06T10:41:36.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.189+0000 7f970d1af700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97080cb150 con 0x7f97080b2100 2023-10-06T10:41:36.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.189+0000 7f97077fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f97080b35d0 0x7f97080cbc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47472/0 (socket says 172.21.15.92:47472) 2023-10-06T10:41:36.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.189+0000 7f97077fe700 1 -- 172.21.15.92:0/604885178 learned_addr learned my addr 172.21.15.92:0/604885178 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:36.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.189+0000 7f9707fff700 1 --2- 172.21.15.92:0/604885178 >> v2:172.21.15.124:3300/0 conn(0x7f9708159440 0x7f97080b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:36.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.189+0000 7f970c9ae700 1 --2- 172.21.15.92:0/604885178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97080b2100 0x7f97080b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:36.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.189+0000 7f9706ffd700 1 -- 172.21.15.92:0/604885178 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890540493 0 0) 0x7f97080cb7a0 con 0x7f97080b35d0 2023-10-06T10:41:36.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.189+0000 7f9706ffd700 1 -- 172.21.15.92:0/604885178 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96f0003570 con 0x7f97080b35d0 2023-10-06T10:41:36.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.190+0000 7f9707fff700 1 -- 172.21.15.92:0/604885178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97080b2100 msgr2=0x7f97080b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:36.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.190+0000 7f9707fff700 1 --2- 172.21.15.92:0/604885178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97080b2100 0x7f97080b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.190+0000 7f9707fff700 1 -- 172.21.15.92:0/604885178 >> v1:172.21.15.92:6789/0 conn(0x7f97080b35d0 legacy=0x7f97080cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.190+0000 7f9706ffd700 1 -- 172.21.15.92:0/604885178 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2894161072 0 0) 0x7f96f0003570 con 0x7f97080b35d0 2023-10-06T10:41:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.190+0000 7f9707fff700 1 -- 172.21.15.92:0/604885178 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97080cb430 con 0x7f9708159440 2023-10-06T10:41:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.190+0000 7f970c9ae700 1 --2- 172.21.15.92:0/604885178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97080b2100 0x7f97080b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:41:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.190+0000 7f9707fff700 1 --2- 172.21.15.92:0/604885178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9708159440 0x7f97080b3090 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f96f80098e0 tx=0x7f96f80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d6bf848c1458d1f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.190+0000 7f9706ffd700 1 -- 172.21.15.92:0/604885178 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc001470 con 0x7f9708159440 2023-10-06T10:41:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.190+0000 7f9706ffd700 1 -- 172.21.15.92:0/604885178 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f8011420 con 0x7f9708159440 2023-10-06T10:41:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.190+0000 7f9706ffd700 1 -- 172.21.15.92:0/604885178 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f80117c0 con 0x7f9708159440 2023-10-06T10:41:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.190+0000 7f970d1af700 1 -- 172.21.15.92:0/604885178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9708159440 msgr2=0x7f97080b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.190+0000 7f970d1af700 1 --2- 172.21.15.92:0/604885178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9708159440 0x7f97080b3090 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f96f80098e0 tx=0x7f96f80091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.190+0000 7f970d1af700 1 -- 172.21.15.92:0/604885178 shutdown_connections 2023-10-06T10:41:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.191+0000 7f970d1af700 1 --2- 172.21.15.92:0/604885178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97080b2100 0x7f97080b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:36.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.191+0000 7f970d1af700 1 --2- 172.21.15.92:0/604885178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9708159440 0x7f97080b3090 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:36.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.191+0000 7f970d1af700 1 -- 172.21.15.92:0/604885178 >> 172.21.15.92:0/604885178 conn(0x7f970806ece0 msgr2=0x7f970806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:36.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.191+0000 7f970d1af700 1 -- 172.21.15.92:0/604885178 shutdown_connections 2023-10-06T10:41:36.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.191+0000 7f970d1af700 1 -- 172.21.15.92:0/604885178 wait complete. 2023-10-06T10:41:36.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.192+0000 7f970d1af700 1 Processor -- start 2023-10-06T10:41:36.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.192+0000 7f970d1af700 1 -- start start 2023-10-06T10:41:36.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.192+0000 7f970d1af700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97080b2100 0x7f970815d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:36.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.192+0000 7f970d1af700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97080b35d0 0x7f970815e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:36.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.192+0000 7f970d1af700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9708159440 0x7f9708155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.192+0000 7f970d1af700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9708155cd0 con 0x7f9708159440 2023-10-06T10:41:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.192+0000 7f970d1af700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9708169bf0 con 0x7f97080b2100 2023-10-06T10:41:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.192+0000 7f970d1af700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f970815f5e0 con 0x7f97080b35d0 2023-10-06T10:41:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.193+0000 7f970c9ae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9708159440 0x7f9708155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.193+0000 7f970c9ae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9708159440 0x7f9708155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52940/0 (socket says 172.21.15.92:52940) 2023-10-06T10:41:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.193+0000 7f970c9ae700 1 -- 172.21.15.92:0/505121474 learned_addr learned my addr 172.21.15.92:0/505121474 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.193+0000 7f97077fe700 1 --2- 172.21.15.92:0/505121474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97080b35d0 0x7f970815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.193+0000 7f9707fff700 1 --2- 172.21.15.92:0/505121474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97080b2100 0x7f970815d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.193+0000 7f970c9ae700 1 -- 172.21.15.92:0/505121474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97080b35d0 msgr2=0x7f970815e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.193+0000 7f970c9ae700 1 --2- 172.21.15.92:0/505121474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97080b35d0 0x7f970815e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.193+0000 7f970c9ae700 1 -- 172.21.15.92:0/505121474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97080b2100 msgr2=0x7f970815d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:36.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.193+0000 7f970c9ae700 1 --2- 172.21.15.92:0/505121474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97080b2100 0x7f970815d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:36.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.193+0000 7f970c9ae700 1 -- 172.21.15.92:0/505121474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9700009a20 con 0x7f9708159440 2023-10-06T10:41:36.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.193+0000 7f9707fff700 1 --2- 172.21.15.92:0/505121474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97080b2100 0x7f970815d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:36.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.193+0000 7f970c9ae700 1 --2- 172.21.15.92:0/505121474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9708159440 0x7f9708155790 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f970000c0f0 tx=0x7f970000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:36.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.193+0000 7f96effff700 1 -- 172.21.15.92:0/505121474 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970000cd90 con 0x7f9708159440 2023-10-06T10:41:36.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.193+0000 7f970d1af700 1 -- 172.21.15.92:0/505121474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f8009420 con 0x7f9708159440 2023-10-06T10:41:36.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.193+0000 7f96effff700 1 -- 172.21.15.92:0/505121474 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970000f040 con 0x7f9708159440 2023-10-06T10:41:36.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.193+0000 7f96effff700 1 -- 172.21.15.92:0/505121474 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97000146d0 con 0x7f9708159440 2023-10-06T10:41:36.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.193+0000 7f970d1af700 1 -- 172.21.15.92:0/505121474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9708169620 con 0x7f9708159440 2023-10-06T10:41:36.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.194+0000 7f970d1af700 1 -- 172.21.15.92:0/505121474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f970805f230 con 0x7f9708159440 2023-10-06T10:41:36.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.195+0000 7f96effff700 1 -- 172.21.15.92:0/505121474 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9700003020 con 0x7f9708159440 2023-10-06T10:41:36.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.195+0000 7f96effff700 1 --2- 172.21.15.92:0/505121474 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f96f4066f30 0x7f96f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:36.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.195+0000 7f9707fff700 1 --2- 172.21.15.92:0/505121474 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f96f4066f30 0x7f96f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:36.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.195+0000 7f96effff700 1 -- 172.21.15.92:0/505121474 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f97000852d0 con 0x7f9708159440 2023-10-06T10:41:36.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.195+0000 7f9707fff700 1 --2- 172.21.15.92:0/505121474 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f96f4066f30 0x7f96f40693e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f96f80093f0 tx=0x7f96f8016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:36.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.199+0000 7f96effff700 1 -- 172.21.15.92:0/505121474 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97000522a0 con 0x7f9708159440 2023-10-06T10:41:36.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.426+0000 7f970d1af700 1 -- 172.21.15.92:0/505121474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f97080b2aa0 con 0x7f9708159440 2023-10-06T10:41:36.427 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T10:41:36.427+0000 7f9e7f3ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T10:41:36.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.433+0000 7f96effff700 1 -- 172.21.15.92:0/505121474 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) v1 ==== 161+0+0 (secure 0 0 0) 0x7f97000558c0 con 0x7f9708159440 2023-10-06T10:41:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.435+0000 7f970d1af700 1 -- 172.21.15.92:0/505121474 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f96f4066f30 msgr2=0x7f96f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.435+0000 7f970d1af700 1 --2- 172.21.15.92:0/505121474 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f96f4066f30 0x7f96f40693e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f96f80093f0 tx=0x7f96f8016000 comp rx=0 tx=0).stop 2023-10-06T10:41:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.435+0000 7f970d1af700 1 -- 172.21.15.92:0/505121474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9708159440 msgr2=0x7f9708155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.435+0000 7f970d1af700 1 --2- 172.21.15.92:0/505121474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9708159440 0x7f9708155790 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f970000c0f0 tx=0x7f970000b040 comp rx=0 tx=0).stop 2023-10-06T10:41:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.435+0000 7f970d1af700 1 -- 172.21.15.92:0/505121474 shutdown_connections 2023-10-06T10:41:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.435+0000 7f970d1af700 1 --2- 172.21.15.92:0/505121474 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f96f4066f30 0x7f96f40693e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.435+0000 7f970d1af700 1 --2- 172.21.15.92:0/505121474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9708159440 0x7f9708155790 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.435+0000 7f970d1af700 1 --2- 172.21.15.92:0/505121474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97080b2100 0x7f970815d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:36.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.435+0000 7f970d1af700 1 --2- 172.21.15.92:0/505121474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97080b35d0 0x7f970815e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:36.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.435+0000 7f970d1af700 1 -- 172.21.15.92:0/505121474 >> 172.21.15.92:0/505121474 conn(0x7f970806ece0 msgr2=0x7f970806f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:36.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.435+0000 7f970d1af700 1 -- 172.21.15.92:0/505121474 shutdown_connections 2023-10-06T10:41:36.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.435+0000 7f970d1af700 1 -- 172.21.15.92:0/505121474 wait complete. 2023-10-06T10:41:36.438 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T10:41:36.438 INFO:tasks.ceph.mds.d.smithi124.stderr: -1014> 2023-10-06T10:41:09.805+0000 7f8aee1e0700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2023-10-06T10:40:51.063444+0000 1=0+1) n(v0 rc2023-10-06T10:40:51.063444+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={5599=pAsLsXsFsx/pAsLsXsFsx@4},l=5599 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55aa0ec80580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:40:51.063444+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:40:51.063444+0000 2=0+2)","memory_value.dirstat":"f(v0 m2023-10-06T10:40:51.063444+0000 1=0+1)","memory_value.rstat":"n(v0 rc2023-10-06T10:40:51.063444+0000 2=0+2)","error_str":""},"return_code":-61} 2023-10-06T10:41:36.438 INFO:tasks.ceph.mds.d.smithi124.stderr: -998> 2023-10-06T10:41:09.806+0000 7f8aee1e0700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={5599=pAsLsXsFsx/AsLsXsFsx@2},l=5599 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55aa0ec80b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2023-10-06T10:41:36.438 INFO:tasks.ceph.mds.c.smithi092.stderr: -1232> 2023-10-06T10:41:09.805+0000 7f3702072700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2023-10-06T10:40:51.062444+0000 1=0+1) n(v2 rc2023-10-06T10:40:51.291437+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={5599=pAsLsXsFsx/pAsLsXsFsx@4},l=5599 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x563031b62c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:40:51.062444+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:40:51.291437+0000 4=0+4)","memory_value.dirstat":"f(v0 m2023-10-06T10:40:51.062444+0000 1=0+1)","memory_value.rstat":"n(v2 rc2023-10-06T10:40:51.291437+0000 4=0+4)","error_str":""},"return_code":-61} 2023-10-06T10:41:36.438 INFO:tasks.ceph.mds.c.smithi092.stderr: -1112> 2023-10-06T10:41:09.807+0000 7f370807e700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2023-10-06T10:40:51.063444+0000 1=0+1) n(v1 rc2023-10-06T10:40:51.291437+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={5599=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x563031b63700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:40:51.063444+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:40:51.063444+0000 3=0+3)","memory_value.dirstat":"f(v0 m2023-10-06T10:40:51.063444+0000 1=0+1)","memory_value.rstat":"n(v1 rc2023-10-06T10:40:51.291437+0000 3=0+3)","error_str":""},"return_code":-61} 2023-10-06T10:41:36.443 INFO:tasks.ceph.mds.a.smithi092.stderr: -2643> 2023-10-06T10:41:09.805+0000 7fcf381c3700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2023-10-06T10:40:51.061444+0000 1=0+1) n(v3 rc2023-10-06T10:40:51.291437+0000 6=0+6) (inest mix dirty) (ifile excl) (iversion lock) caps={5599=pAsLsXsFsx/pAsLsXsFsx@5},l=5599 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x56027bd32100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:40:51.061444+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:40:51.291437+0000 6=0+6)","memory_value.dirstat":"f(v0 m2023-10-06T10:40:51.061444+0000 1=0+1)","memory_value.rstat":"n(v3 rc2023-10-06T10:40:51.291437+0000 6=0+6)","error_str":""},"return_code":-61} 2023-10-06T10:41:36.443 INFO:tasks.ceph.mds.a.smithi092.stderr: -2544> 2023-10-06T10:41:09.806+0000 7fcf3e1cf700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2023-10-06T10:40:51.061444+0000 1=0+1) n(v4 rc2023-10-06T10:40:51.291437+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={5599=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x56027bd33700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:40:51.061444+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:40:51.291437+0000 5=0+5)","memory_value.dirstat":"f(v0 m2023-10-06T10:40:51.061444+0000 1=0+1)","memory_value.rstat":"n(v4 rc2023-10-06T10:40:51.291437+0000 5=0+5)","error_str":""},"return_code":-61} 2023-10-06T10:41:36.450 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T10:41:36.463 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-06T10:41:36.463 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T10:41:36.467 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T10:41:36.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.567+0000 7fef0b256700 1 Processor -- start 2023-10-06T10:41:36.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.567+0000 7fef0b256700 1 -- start start 2023-10-06T10:41:36.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.568+0000 7fef0b256700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef040b2620 0x7fef040cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:36.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.568+0000 7fef0b256700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef040cc970 con 0x7fef040b29f0 2023-10-06T10:41:36.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.568+0000 7fef0b256700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef040cbd60 con 0x7fef040cf1b0 2023-10-06T10:41:36.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.568+0000 7fef0b256700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef040cc040 con 0x7fef040b2620 2023-10-06T10:41:36.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.568+0000 7fef09a53700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fef040b29f0 0x7fef040ceaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47484/0 (socket says 172.21.15.92:47484) 2023-10-06T10:41:36.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.568+0000 7fef09a53700 1 -- 172.21.15.92:0/374583750 learned_addr learned my addr 172.21.15.92:0/374583750 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:36.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.568+0000 7fef0a254700 1 --2- 172.21.15.92:0/374583750 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef040b2620 0x7fef040cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:36.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.568+0000 7fef09252700 1 -- 172.21.15.92:0/374583750 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715092712 0 0) 0x7fef040cc970 con 0x7fef040b29f0 2023-10-06T10:41:36.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.569+0000 7fef09252700 1 -- 172.21.15.92:0/374583750 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feef0003570 con 0x7fef040b29f0 2023-10-06T10:41:36.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.569+0000 7fef09252700 1 -- 172.21.15.92:0/374583750 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1539265224 0 0) 0x7fef040cbd60 con 0x7fef040cf1b0 2023-10-06T10:41:36.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.569+0000 7fef09252700 1 -- 172.21.15.92:0/374583750 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef040cc970 con 0x7fef040cf1b0 2023-10-06T10:41:36.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.569+0000 7fef0a254700 1 -- 172.21.15.92:0/374583750 >> v1:172.21.15.124:6789/0 conn(0x7fef040cf1b0 legacy=0x7fef040c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:36.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.569+0000 7fef0a254700 1 -- 172.21.15.92:0/374583750 >> v1:172.21.15.92:6789/0 conn(0x7fef040b29f0 legacy=0x7fef040ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:36.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.569+0000 7fef09252700 1 -- 172.21.15.92:0/374583750 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2845478080 0 0) 0x7feef0003570 con 0x7fef040b29f0 2023-10-06T10:41:36.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.569+0000 7fef0a254700 1 -- 172.21.15.92:0/374583750 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef040cc320 con 0x7fef040b2620 2023-10-06T10:41:36.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.569+0000 7fef0a254700 1 --2- 172.21.15.92:0/374583750 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef040b2620 0x7fef040cccb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fef00009900 tx=0x7fef00009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61261da816333675 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:36.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.569+0000 7fef09252700 1 -- 172.21.15.92:0/374583750 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef00005c10 con 0x7fef040b2620 2023-10-06T10:41:36.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.569+0000 7fef09252700 1 -- 172.21.15.92:0/374583750 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef00005d70 con 0x7fef040b2620 2023-10-06T10:41:36.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.569+0000 7fef09252700 1 -- 172.21.15.92:0/374583750 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef00006240 con 0x7fef040b2620 2023-10-06T10:41:36.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.570+0000 7fef0b256700 1 -- 172.21.15.92:0/374583750 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef040b2620 msgr2=0x7fef040cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:36.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.570+0000 7fef0b256700 1 --2- 172.21.15.92:0/374583750 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef040b2620 0x7fef040cccb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fef00009900 tx=0x7fef00009120 comp rx=0 tx=0).stop 2023-10-06T10:41:36.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.570+0000 7fef0b256700 1 -- 172.21.15.92:0/374583750 shutdown_connections 2023-10-06T10:41:36.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.570+0000 7fef0b256700 1 --2- 172.21.15.92:0/374583750 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef040b2620 0x7fef040cccb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:36.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.570+0000 7fef0b256700 1 -- 172.21.15.92:0/374583750 >> 172.21.15.92:0/374583750 conn(0x7fef04099520 msgr2=0x7fef0406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:36.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.570+0000 7fef0b256700 1 -- 172.21.15.92:0/374583750 shutdown_connections 2023-10-06T10:41:36.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.570+0000 7fef0b256700 1 -- 172.21.15.92:0/374583750 wait complete. 2023-10-06T10:41:36.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.571+0000 7fef0b256700 1 Processor -- start 2023-10-06T10:41:36.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.571+0000 7fef0b256700 1 -- start start 2023-10-06T10:41:36.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.571+0000 7fef0b256700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef040b2620 0x7fef040c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:36.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.571+0000 7fef0b256700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef040b29f0 0x7fef040ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:36.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.571+0000 7fef0b256700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef040cf1b0 0x7fef0415c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:36.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.571+0000 7fef0b256700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0415d3b0 con 0x7fef040b29f0 2023-10-06T10:41:36.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.571+0000 7fef0b256700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef04154370 con 0x7fef040b2620 2023-10-06T10:41:36.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.571+0000 7fef0b256700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0415d0d0 con 0x7fef040cf1b0 2023-10-06T10:41:36.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.572+0000 7fef09a53700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef040b29f0 0x7fef040ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:36.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.572+0000 7fef0a254700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef040b2620 0x7fef040c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:36.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.572+0000 7fef09a53700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef040b29f0 0x7fef040ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52976/0 (socket says 172.21.15.92:52976) 2023-10-06T10:41:36.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.572+0000 7fef09a53700 1 -- 172.21.15.92:0/3332294532 learned_addr learned my addr 172.21.15.92:0/3332294532 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:36.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.572+0000 7fef0aa55700 1 --2- 172.21.15.92:0/3332294532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef040cf1b0 0x7fef0415c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:36.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.572+0000 7fef09a53700 1 -- 172.21.15.92:0/3332294532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef040cf1b0 msgr2=0x7fef0415c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:36.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.572+0000 7fef09a53700 1 --2- 172.21.15.92:0/3332294532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef040cf1b0 0x7fef0415c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:36.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.572+0000 7fef09a53700 1 -- 172.21.15.92:0/3332294532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef040b2620 msgr2=0x7fef040c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:36.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.572+0000 7fef09a53700 1 --2- 172.21.15.92:0/3332294532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef040b2620 0x7fef040c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:36.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.572+0000 7fef09a53700 1 -- 172.21.15.92:0/3332294532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef000093c0 con 0x7fef040b29f0 2023-10-06T10:41:36.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.572+0000 7fef09a53700 1 --2- 172.21.15.92:0/3332294532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef040b29f0 0x7fef040ca1e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7feef400c5c0 tx=0x7feef4007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:36.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.572+0000 7feefa7fc700 1 -- 172.21.15.92:0/3332294532 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4007360 con 0x7fef040b29f0 2023-10-06T10:41:36.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.572+0000 7feefa7fc700 1 -- 172.21.15.92:0/3332294532 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef40074c0 con 0x7fef040b29f0 2023-10-06T10:41:36.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.572+0000 7fef0b256700 1 -- 172.21.15.92:0/3332294532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef040b4100 con 0x7fef040b29f0 2023-10-06T10:41:36.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.572+0000 7feefa7fc700 1 -- 172.21.15.92:0/3332294532 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4004830 con 0x7fef040b29f0 2023-10-06T10:41:36.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.572+0000 7fef0b256700 1 -- 172.21.15.92:0/3332294532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef040b46c0 con 0x7fef040b29f0 2023-10-06T10:41:36.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.573+0000 7fef0b256700 1 -- 172.21.15.92:0/3332294532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef040b48e0 con 0x7fef040b29f0 2023-10-06T10:41:36.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.574+0000 7feefa7fc700 1 -- 172.21.15.92:0/3332294532 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feef4004c20 con 0x7fef040b29f0 2023-10-06T10:41:36.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.574+0000 7feefa7fc700 1 --2- 172.21.15.92:0/3332294532 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7feee8066f30 0x7feee80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:36.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.574+0000 7feefa7fc700 1 -- 172.21.15.92:0/3332294532 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6636+0+0 (secure 0 0 0) 0x7feef4084570 con 0x7fef040b29f0 2023-10-06T10:41:36.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.574+0000 7fef0a254700 1 --2- 172.21.15.92:0/3332294532 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7feee8066f30 0x7feee80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:36.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.574+0000 7fef0a254700 1 --2- 172.21.15.92:0/3332294532 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7feee8066f30 0x7feee80693e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fef00009390 tx=0x7fef00016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:36.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.578+0000 7feefa7fc700 1 -- 172.21.15.92:0/3332294532 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feef40514c0 con 0x7fef040b29f0 2023-10-06T10:41:36.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:36.802+0000 7fef0b256700 1 -- 172.21.15.92:0/3332294532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fef0405f230 con 0x7fef040b29f0 2023-10-06T10:41:37.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.464+0000 7feefa7fc700 1 -- 172.21.15.92:0/3332294532 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) v1 ==== 104+0+0 (secure 0 0 0) 0x7feef4054ae0 con 0x7fef040b29f0 2023-10-06T10:41:37.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.467+0000 7fef0b256700 1 -- 172.21.15.92:0/3332294532 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7feee8066f30 msgr2=0x7feee80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:37.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.467+0000 7fef0b256700 1 --2- 172.21.15.92:0/3332294532 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7feee8066f30 0x7feee80693e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fef00009390 tx=0x7fef00016000 comp rx=0 tx=0).stop 2023-10-06T10:41:37.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.467+0000 7fef0b256700 1 -- 172.21.15.92:0/3332294532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef040b29f0 msgr2=0x7fef040ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:37.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.467+0000 7fef0b256700 1 --2- 172.21.15.92:0/3332294532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef040b29f0 0x7fef040ca1e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7feef400c5c0 tx=0x7feef4007b00 comp rx=0 tx=0).stop 2023-10-06T10:41:37.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.469+0000 7fef0b256700 1 -- 172.21.15.92:0/3332294532 shutdown_connections 2023-10-06T10:41:37.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.469+0000 7fef0b256700 1 --2- 172.21.15.92:0/3332294532 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7feee8066f30 0x7feee80693e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:37.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.469+0000 7fef0b256700 1 --2- 172.21.15.92:0/3332294532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef040cf1b0 0x7fef0415c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:37.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.469+0000 7fef0b256700 1 --2- 172.21.15.92:0/3332294532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef040b2620 0x7fef040c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:37.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.469+0000 7fef0b256700 1 --2- 172.21.15.92:0/3332294532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef040b29f0 0x7fef040ca1e0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:37.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.469+0000 7fef0b256700 1 -- 172.21.15.92:0/3332294532 >> 172.21.15.92:0/3332294532 conn(0x7fef04099520 msgr2=0x7fef0406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:37.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.469+0000 7fef0b256700 1 -- 172.21.15.92:0/3332294532 shutdown_connections 2023-10-06T10:41:37.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.469+0000 7fef0b256700 1 -- 172.21.15.92:0/3332294532 wait complete. 2023-10-06T10:41:37.482 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T10:41:37.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.642+0000 7fd549531700 1 Processor -- start 2023-10-06T10:41:37.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.642+0000 7fd549531700 1 -- start start 2023-10-06T10:41:37.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.643+0000 7fd549531700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5440a7be0 0x7fd54415a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:37.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.643+0000 7fd549531700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd5440ae570 0x7fd5440675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:37.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.643+0000 7fd549531700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5440c9a50 con 0x7fd544067af0 2023-10-06T10:41:37.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.643+0000 7fd549531700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd544067f70 con 0x7fd5440ae570 2023-10-06T10:41:37.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.643+0000 7fd549531700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5440c9440 con 0x7fd5440a7be0 2023-10-06T10:41:37.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.643+0000 7fd53a7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd5440ae570 0x7fd5440675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:37.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.643+0000 7fd5437fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd544067af0 0x7fd54406e440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47490/0 (socket says 172.21.15.92:47490) 2023-10-06T10:41:37.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.643+0000 7fd53a7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd5440ae570 0x7fd5440675b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52980/0 (socket says 172.21.15.92:52980) 2023-10-06T10:41:37.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.643+0000 7fd5437fe700 1 -- 172.21.15.92:0/2104705968 learned_addr learned my addr 172.21.15.92:0/2104705968 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:37.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.643+0000 7fd542ffd700 1 --2- 172.21.15.92:0/2104705968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5440a7be0 0x7fd54415a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:37.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.643+0000 7fd53a7fc700 1 -- 172.21.15.92:0/2104705968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5440a7be0 msgr2=0x7fd54415a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:37.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.643+0000 7fd53a7fc700 1 --2- 172.21.15.92:0/2104705968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5440a7be0 0x7fd54415a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:37.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.643+0000 7fd53a7fc700 1 -- 172.21.15.92:0/2104705968 >> v1:172.21.15.92:6789/0 conn(0x7fd544067af0 legacy=0x7fd54406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:37.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.643+0000 7fd53a7fc700 1 -- 172.21.15.92:0/2104705968 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5440ad0d0 con 0x7fd5440ae570 2023-10-06T10:41:37.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.643+0000 7fd5427fc700 1 -- 172.21.15.92:0/2104705968 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087171895 0 0) 0x7fd5440c9a50 con 0x7fd544067af0 2023-10-06T10:41:37.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.643+0000 7fd53a7fc700 1 --2- 172.21.15.92:0/2104705968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5440ae570 0x7fd5440675b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd52c0098e0 tx=0x7fd52c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b88a85c03e6bf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:37.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.644+0000 7fd5427fc700 1 -- 172.21.15.92:0/2104705968 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c009e80 con 0x7fd5440ae570 2023-10-06T10:41:37.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.644+0000 7fd5427fc700 1 -- 172.21.15.92:0/2104705968 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52c00b860 con 0x7fd5440ae570 2023-10-06T10:41:37.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.644+0000 7fd5427fc700 1 -- 172.21.15.92:0/2104705968 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c00bbe0 con 0x7fd5440ae570 2023-10-06T10:41:37.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.644+0000 7fd549531700 1 -- 172.21.15.92:0/2104705968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5440ae570 msgr2=0x7fd5440675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:37.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.644+0000 7fd549531700 1 --2- 172.21.15.92:0/2104705968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5440ae570 0x7fd5440675b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd52c0098e0 tx=0x7fd52c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:37.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.644+0000 7fd549531700 1 -- 172.21.15.92:0/2104705968 shutdown_connections 2023-10-06T10:41:37.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.644+0000 7fd549531700 1 --2- 172.21.15.92:0/2104705968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5440a7be0 0x7fd54415a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:37.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.644+0000 7fd549531700 1 --2- 172.21.15.92:0/2104705968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5440ae570 0x7fd5440675b0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:37.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.644+0000 7fd549531700 1 -- 172.21.15.92:0/2104705968 >> 172.21.15.92:0/2104705968 conn(0x7fd5440cd1e0 msgr2=0x7fd5440cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:37.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.644+0000 7fd549531700 1 -- 172.21.15.92:0/2104705968 shutdown_connections 2023-10-06T10:41:37.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.644+0000 7fd549531700 1 -- 172.21.15.92:0/2104705968 wait complete. 2023-10-06T10:41:37.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.645+0000 7fd549531700 1 Processor -- start 2023-10-06T10:41:37.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.645+0000 7fd549531700 1 -- start start 2023-10-06T10:41:37.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.646+0000 7fd549531700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd544067af0 0x7fd544156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:37.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.646+0000 7fd549531700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5440a7be0 0x7fd5440aff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:37.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.646+0000 7fd549531700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5440ae570 0x7fd544169f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:37.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.646+0000 7fd549531700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5441557e0 con 0x7fd5440ae570 2023-10-06T10:41:37.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.646+0000 7fd549531700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54416a470 con 0x7fd5440a7be0 2023-10-06T10:41:37.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.646+0000 7fd549531700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5441554b0 con 0x7fd544067af0 2023-10-06T10:41:37.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.646+0000 7fd5437fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5440ae570 0x7fd544169f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:37.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.646+0000 7fd5437fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5440ae570 0x7fd544169f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52982/0 (socket says 172.21.15.92:52982) 2023-10-06T10:41:37.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.646+0000 7fd5437fe700 1 -- 172.21.15.92:0/1069392708 learned_addr learned my addr 172.21.15.92:0/1069392708 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:37.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.646+0000 7fd53a7fc700 1 --2- 172.21.15.92:0/1069392708 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5440a7be0 0x7fd5440aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:37.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.646+0000 7fd542ffd700 1 --2- 172.21.15.92:0/1069392708 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd544067af0 0x7fd544156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:37.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.646+0000 7fd5437fe700 1 -- 172.21.15.92:0/1069392708 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd544067af0 msgr2=0x7fd544156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:37.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.646+0000 7fd5437fe700 1 --2- 172.21.15.92:0/1069392708 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd544067af0 0x7fd544156c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:37.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.646+0000 7fd5437fe700 1 -- 172.21.15.92:0/1069392708 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5440a7be0 msgr2=0x7fd5440aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:37.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.647+0000 7fd5437fe700 1 --2- 172.21.15.92:0/1069392708 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5440a7be0 0x7fd5440aff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:37.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.647+0000 7fd5437fe700 1 -- 172.21.15.92:0/1069392708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c009420 con 0x7fd5440ae570 2023-10-06T10:41:37.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.647+0000 7fd5437fe700 1 --2- 172.21.15.92:0/1069392708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5440ae570 0x7fd544169f30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00cbf0 tx=0x7fd53c0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:37.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.647+0000 7fd53b7fe700 1 -- 172.21.15.92:0/1069392708 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c00a9c0 con 0x7fd5440ae570 2023-10-06T10:41:37.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.647+0000 7fd53b7fe700 1 -- 172.21.15.92:0/1069392708 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53c00ab40 con 0x7fd5440ae570 2023-10-06T10:41:37.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.647+0000 7fd53b7fe700 1 -- 172.21.15.92:0/1069392708 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c0070f0 con 0x7fd5440ae570 2023-10-06T10:41:37.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.647+0000 7fd549531700 1 -- 172.21.15.92:0/1069392708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd54416af00 con 0x7fd5440ae570 2023-10-06T10:41:37.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.647+0000 7fd549531700 1 -- 172.21.15.92:0/1069392708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd54416a940 con 0x7fd5440ae570 2023-10-06T10:41:37.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.648+0000 7fd549531700 1 -- 172.21.15.92:0/1069392708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd544064a90 con 0x7fd5440ae570 2023-10-06T10:41:37.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.648+0000 7fd53b7fe700 1 -- 172.21.15.92:0/1069392708 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd53c007520 con 0x7fd5440ae570 2023-10-06T10:41:37.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.649+0000 7fd53b7fe700 1 --2- 172.21.15.92:0/1069392708 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd524066f30 0x7fd5240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:37.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.649+0000 7fd53b7fe700 1 -- 172.21.15.92:0/1069392708 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fd53c084c00 con 0x7fd5440ae570 2023-10-06T10:41:37.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.649+0000 7fd542ffd700 1 --2- 172.21.15.92:0/1069392708 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd524066f30 0x7fd5240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:37.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.649+0000 7fd542ffd700 1 --2- 172.21.15.92:0/1069392708 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd524066f30 0x7fd5240693e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd5340098e0 tx=0x7fd534008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:37.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.653+0000 7fd53b7fe700 1 -- 172.21.15.92:0/1069392708 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd53c08b080 con 0x7fd5440ae570 2023-10-06T10:41:37.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:37.835+0000 7fd549531700 1 -- 172.21.15.92:0/1069392708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd54409aed0 con 0x7fd5440ae570 2023-10-06T10:41:38.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.474+0000 7fd53b7fe700 1 -- 172.21.15.92:0/1069392708 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v54) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd53c051b50 con 0x7fd5440ae570 2023-10-06T10:41:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.476+0000 7fd549531700 1 -- 172.21.15.92:0/1069392708 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd524066f30 msgr2=0x7fd5240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.477+0000 7fd549531700 1 --2- 172.21.15.92:0/1069392708 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd524066f30 0x7fd5240693e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd5340098e0 tx=0x7fd534008040 comp rx=0 tx=0).stop 2023-10-06T10:41:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.477+0000 7fd549531700 1 -- 172.21.15.92:0/1069392708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5440ae570 msgr2=0x7fd544169f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.477+0000 7fd549531700 1 --2- 172.21.15.92:0/1069392708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5440ae570 0x7fd544169f30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00cbf0 tx=0x7fd53c0019b0 comp rx=0 tx=0).stop 2023-10-06T10:41:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.477+0000 7fd549531700 1 -- 172.21.15.92:0/1069392708 shutdown_connections 2023-10-06T10:41:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.477+0000 7fd549531700 1 --2- 172.21.15.92:0/1069392708 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd524066f30 0x7fd5240693e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.477+0000 7fd549531700 1 --2- 172.21.15.92:0/1069392708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5440ae570 0x7fd544169f30 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.477+0000 7fd549531700 1 --2- 172.21.15.92:0/1069392708 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd544067af0 0x7fd544156c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.477+0000 7fd549531700 1 --2- 172.21.15.92:0/1069392708 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5440a7be0 0x7fd5440aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.477+0000 7fd549531700 1 -- 172.21.15.92:0/1069392708 >> 172.21.15.92:0/1069392708 conn(0x7fd5440cd1e0 msgr2=0x7fd5440c99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.477+0000 7fd549531700 1 -- 172.21.15.92:0/1069392708 shutdown_connections 2023-10-06T10:41:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.477+0000 7fd549531700 1 -- 172.21.15.92:0/1069392708 wait complete. 2023-10-06T10:41:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-10-06T10:41:38.491 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T10:41:38.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.612+0000 7fa4ac56c700 1 Processor -- start 2023-10-06T10:41:38.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.612+0000 7fa4ac56c700 1 -- start start 2023-10-06T10:41:38.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.613+0000 7fa4ac56c700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa4a406ece0 0x7fa4a40b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:38.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.613+0000 7fa4ac56c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a40b2ce0 con 0x7fa4a40aede0 2023-10-06T10:41:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.613+0000 7fa4ac56c700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a40b0410 con 0x7fa4a406f0b0 2023-10-06T10:41:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.613+0000 7fa4ac56c700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4a406fb40 con 0x7fa4a406ece0 2023-10-06T10:41:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.613+0000 7fa4aab09700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa4a40aede0 0x7fa4a40af240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47506/0 (socket says 172.21.15.92:47506) 2023-10-06T10:41:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.613+0000 7fa4aab09700 1 -- 172.21.15.92:0/458090844 learned_addr learned my addr 172.21.15.92:0/458090844 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.613+0000 7fa4aa308700 1 --2- 172.21.15.92:0/458090844 >> v2:172.21.15.124:3300/0 conn(0x7fa4a406ece0 0x7fa4a40b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.613+0000 7fa4a9306700 1 -- 172.21.15.92:0/458090844 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2310464297 0 0) 0x7fa4a40b2ce0 con 0x7fa4a40aede0 2023-10-06T10:41:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4a9306700 1 -- 172.21.15.92:0/458090844 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa490003570 con 0x7fa4a40aede0 2023-10-06T10:41:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4a9306700 1 -- 172.21.15.92:0/458090844 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1387888661 0 0) 0x7fa490003570 con 0x7fa4a40aede0 2023-10-06T10:41:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4a9306700 1 -- 172.21.15.92:0/458090844 >> v2:172.21.15.124:3300/0 conn(0x7fa4a406ece0 msgr2=0x7fa4a40b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4a9306700 1 --2- 172.21.15.92:0/458090844 >> v2:172.21.15.124:3300/0 conn(0x7fa4a406ece0 0x7fa4a40b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4a9306700 1 -- 172.21.15.92:0/458090844 >> v1:172.21.15.124:6789/0 conn(0x7fa4a406f0b0 legacy=0x7fa4a40b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:38.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4a9306700 1 -- 172.21.15.92:0/458090844 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a40b2a00 con 0x7fa4a40aede0 2023-10-06T10:41:38.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4a9306700 1 -- 172.21.15.92:0/458090844 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fa49c001470 con 0x7fa4a40aede0 2023-10-06T10:41:38.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4a9306700 1 -- 172.21.15.92:0/458090844 >> v1:172.21.15.92:6789/0 conn(0x7fa4a40aede0 legacy=0x7fa4a40af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:38.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4a9306700 1 --2- 172.21.15.92:0/458090844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa490003a60 0x7fa490003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:38.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4a9306700 1 --2- 172.21.15.92:0/458090844 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa490004810 0x7fa490004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:38.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4a9306700 1 --2- 172.21.15.92:0/458090844 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4900051c0 0x7fa49000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:38.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4a9306700 1 -- 172.21.15.92:0/458090844 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49000c0a0 con 0x7fa490003a60 2023-10-06T10:41:38.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4a9306700 1 -- 172.21.15.92:0/458090844 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49000c210 con 0x7fa4900051c0 2023-10-06T10:41:38.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4a9306700 1 -- 172.21.15.92:0/458090844 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49000c3d0 con 0x7fa490004810 2023-10-06T10:41:38.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4aa308700 1 --2- 172.21.15.92:0/458090844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa490003a60 0x7fa490003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:38.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4aab09700 1 --2- 172.21.15.92:0/458090844 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4900051c0 0x7fa49000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:38.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4a9b07700 1 --2- 172.21.15.92:0/458090844 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa490004810 0x7fa490004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:38.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4aa308700 1 -- 172.21.15.92:0/458090844 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa490004810 msgr2=0x7fa490004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:38.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4aa308700 1 --2- 172.21.15.92:0/458090844 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa490004810 0x7fa490004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:38.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4aa308700 1 -- 172.21.15.92:0/458090844 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4900051c0 msgr2=0x7fa49000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:38.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4aa308700 1 --2- 172.21.15.92:0/458090844 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4900051c0 0x7fa49000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:38.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4aab09700 1 -- 172.21.15.92:0/458090844 reap_dead start 2023-10-06T10:41:38.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4aa308700 1 -- 172.21.15.92:0/458090844 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa490003570 con 0x7fa490003a60 2023-10-06T10:41:38.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4aa308700 1 --2- 172.21.15.92:0/458090844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa490003a60 0x7fa490003e70 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000e210 tx=0x7fa4a000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525732c06c2e8834 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:38.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4a9306700 1 -- 172.21.15.92:0/458090844 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a000eac0 con 0x7fa490003a60 2023-10-06T10:41:38.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4a9306700 1 -- 172.21.15.92:0/458090844 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a000f040 con 0x7fa490003a60 2023-10-06T10:41:38.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.614+0000 7fa4a9306700 1 -- 172.21.15.92:0/458090844 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a000ee70 con 0x7fa490003a60 2023-10-06T10:41:38.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.615+0000 7fa4ac56c700 1 -- 172.21.15.92:0/458090844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa490003a60 msgr2=0x7fa490003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:38.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.615+0000 7fa4ac56c700 1 --2- 172.21.15.92:0/458090844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa490003a60 0x7fa490003e70 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000e210 tx=0x7fa4a000b040 comp rx=0 tx=0).stop 2023-10-06T10:41:38.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.615+0000 7fa4ac56c700 1 -- 172.21.15.92:0/458090844 shutdown_connections 2023-10-06T10:41:38.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.615+0000 7fa4ac56c700 1 --2- 172.21.15.92:0/458090844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa490003a60 0x7fa490003e70 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:38.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.615+0000 7fa4ac56c700 1 -- 172.21.15.92:0/458090844 >> 172.21.15.92:0/458090844 conn(0x7fa4a40682f0 msgr2=0x7fa4a40686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:38.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.615+0000 7fa4ac56c700 1 -- 172.21.15.92:0/458090844 shutdown_connections 2023-10-06T10:41:38.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.615+0000 7fa4ac56c700 1 -- 172.21.15.92:0/458090844 wait complete. 2023-10-06T10:41:38.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.616+0000 7fa4ac56c700 1 Processor -- start 2023-10-06T10:41:38.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.616+0000 7fa4ac56c700 1 -- start start 2023-10-06T10:41:38.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.617+0000 7fa4ac56c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa490003a60 0x7fa4a40cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:38.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.617+0000 7fa4ac56c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a4153990 0x7fa4a40ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:38.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.617+0000 7fa4ac56c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4a40ca610 0x7fa4a415aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:38.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.617+0000 7fa4ac56c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a416d910 con 0x7fa4a4153990 2023-10-06T10:41:38.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.617+0000 7fa4ac56c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a416d630 con 0x7fa4a40ca610 2023-10-06T10:41:38.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.617+0000 7fa4ac56c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a416cd20 con 0x7fa490003a60 2023-10-06T10:41:38.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.617+0000 7fa4a9b07700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a4153990 0x7fa4a40ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:38.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.617+0000 7fa4a9b07700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a4153990 0x7fa4a40ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53000/0 (socket says 172.21.15.92:53000) 2023-10-06T10:41:38.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.617+0000 7fa4a9b07700 1 -- 172.21.15.92:0/807785650 learned_addr learned my addr 172.21.15.92:0/807785650 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:38.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.617+0000 7fa4aa308700 1 --2- 172.21.15.92:0/807785650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa490003a60 0x7fa4a40cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:38.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.617+0000 7fa4aab09700 1 --2- 172.21.15.92:0/807785650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4a40ca610 0x7fa4a415aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:38.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.617+0000 7fa4a9b07700 1 -- 172.21.15.92:0/807785650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa490003a60 msgr2=0x7fa4a40cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:38.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.617+0000 7fa4a9b07700 1 --2- 172.21.15.92:0/807785650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa490003a60 0x7fa4a40cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:38.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.617+0000 7fa4a9b07700 1 -- 172.21.15.92:0/807785650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4a40ca610 msgr2=0x7fa4a415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:38.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.617+0000 7fa4a9b07700 1 --2- 172.21.15.92:0/807785650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4a40ca610 0x7fa4a415aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:38.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.617+0000 7fa4a9b07700 1 -- 172.21.15.92:0/807785650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a416cfe0 con 0x7fa4a4153990 2023-10-06T10:41:38.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.617+0000 7fa4aa308700 1 --2- 172.21.15.92:0/807785650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa490003a60 0x7fa4a40cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:38.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.617+0000 7fa4aab09700 1 --2- 172.21.15.92:0/807785650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4a40ca610 0x7fa4a415aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:38.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.617+0000 7fa4a9b07700 1 --2- 172.21.15.92:0/807785650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a4153990 0x7fa4a40ca0d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa49400e5f0 tx=0x7fa494009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:38.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.618+0000 7fa49a7fc700 1 -- 172.21.15.92:0/807785650 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4940105f0 con 0x7fa4a4153990 2023-10-06T10:41:38.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.618+0000 7fa49a7fc700 1 -- 172.21.15.92:0/807785650 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa494010750 con 0x7fa4a4153990 2023-10-06T10:41:38.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.618+0000 7fa4ac56c700 1 -- 172.21.15.92:0/807785650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a416d2c0 con 0x7fa4a4153990 2023-10-06T10:41:38.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.618+0000 7fa49a7fc700 1 -- 172.21.15.92:0/807785650 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494010af0 con 0x7fa4a4153990 2023-10-06T10:41:38.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.618+0000 7fa4ac56c700 1 -- 172.21.15.92:0/807785650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a40cccf0 con 0x7fa4a4153990 2023-10-06T10:41:38.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.619+0000 7fa4ac56c700 1 -- 172.21.15.92:0/807785650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a4064a90 con 0x7fa4a4153990 2023-10-06T10:41:38.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.619+0000 7fa49a7fc700 1 -- 172.21.15.92:0/807785650 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa494014480 con 0x7fa4a4153990 2023-10-06T10:41:38.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.619+0000 7fa49a7fc700 1 --2- 172.21.15.92:0/807785650 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa488066f30 0x7fa4880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:38.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.619+0000 7fa49a7fc700 1 -- 172.21.15.92:0/807785650 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6198+0+0 (secure 0 0 0) 0x7fa494084470 con 0x7fa4a4153990 2023-10-06T10:41:38.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.619+0000 7fa4aa308700 1 --2- 172.21.15.92:0/807785650 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa488066f30 0x7fa4880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:38.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.620+0000 7fa4aa308700 1 --2- 172.21.15.92:0/807785650 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa488066f30 0x7fa4880693e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000afd0 tx=0x7fa4a001a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:38.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.624+0000 7fa49a7fc700 1 -- 172.21.15.92:0/807785650 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa49404d5a0 con 0x7fa4a4153990 2023-10-06T10:41:38.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:38.809+0000 7fa4ac56c700 1 -- 172.21.15.92:0/807785650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa4a40afdc0 con 0x7fa4a4153990 2023-10-06T10:41:39.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.478+0000 7fa49a7fc700 1 -- 172.21.15.92:0/807785650 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v55) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa494054da0 con 0x7fa4a4153990 2023-10-06T10:41:39.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.481+0000 7fa4ac56c700 1 -- 172.21.15.92:0/807785650 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa488066f30 msgr2=0x7fa4880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:39.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.481+0000 7fa4ac56c700 1 --2- 172.21.15.92:0/807785650 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa488066f30 0x7fa4880693e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000afd0 tx=0x7fa4a001a000 comp rx=0 tx=0).stop 2023-10-06T10:41:39.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.481+0000 7fa4ac56c700 1 -- 172.21.15.92:0/807785650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a4153990 msgr2=0x7fa4a40ca0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:39.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.481+0000 7fa4ac56c700 1 --2- 172.21.15.92:0/807785650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a4153990 0x7fa4a40ca0d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa49400e5f0 tx=0x7fa494009040 comp rx=0 tx=0).stop 2023-10-06T10:41:39.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.482+0000 7fa4ac56c700 1 -- 172.21.15.92:0/807785650 shutdown_connections 2023-10-06T10:41:39.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.482+0000 7fa4ac56c700 1 --2- 172.21.15.92:0/807785650 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa488066f30 0x7fa4880693e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:39.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.482+0000 7fa4ac56c700 1 --2- 172.21.15.92:0/807785650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4a40ca610 0x7fa4a415aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:39.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.482+0000 7fa4ac56c700 1 --2- 172.21.15.92:0/807785650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa490003a60 0x7fa4a40cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:39.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.482+0000 7fa4ac56c700 1 --2- 172.21.15.92:0/807785650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a4153990 0x7fa4a40ca0d0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:39.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.482+0000 7fa4ac56c700 1 -- 172.21.15.92:0/807785650 >> 172.21.15.92:0/807785650 conn(0x7fa4a40682f0 msgr2=0x7fa4a4156940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:39.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.482+0000 7fa4ac56c700 1 -- 172.21.15.92:0/807785650 shutdown_connections 2023-10-06T10:41:39.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.482+0000 7fa4ac56c700 1 -- 172.21.15.92:0/807785650 wait complete. 2023-10-06T10:41:39.484 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-10-06T10:41:39.496 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T10:41:39.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.615+0000 7f182ab90700 1 Processor -- start 2023-10-06T10:41:39.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.615+0000 7f182ab90700 1 -- start start 2023-10-06T10:41:39.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.616+0000 7f182ab90700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1824150940 0x7f1824159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.616+0000 7f182ab90700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f182415a160 0x7f18240ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.616+0000 7f182ab90700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182415ac30 con 0x7f18240c5250 2023-10-06T10:41:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.616+0000 7f182ab90700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18240cb6a0 con 0x7f182415a160 2023-10-06T10:41:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.616+0000 7f182ab90700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18240cb090 con 0x7f1824150940 2023-10-06T10:41:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.616+0000 7f181bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f182415a160 0x7f18240ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.616+0000 7f181bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f182415a160 0x7f18240ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53016/0 (socket says 172.21.15.92:53016) 2023-10-06T10:41:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.616+0000 7f181bfff700 1 -- 172.21.15.92:0/2560081717 learned_addr learned my addr 172.21.15.92:0/2560081717 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.616+0000 7f182892c700 1 --2- 172.21.15.92:0/2560081717 >> v2:172.21.15.124:3300/0 conn(0x7f1824150940 0x7f1824159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.616+0000 7f181bfff700 1 -- 172.21.15.92:0/2560081717 >> v2:172.21.15.124:3300/0 conn(0x7f1824150940 msgr2=0x7f1824159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:39.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.616+0000 7f181bfff700 1 --2- 172.21.15.92:0/2560081717 >> v2:172.21.15.124:3300/0 conn(0x7f1824150940 0x7f1824159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:39.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.616+0000 7f181bfff700 1 -- 172.21.15.92:0/2560081717 >> v1:172.21.15.124:6789/0 conn(0x7f18240c5250 legacy=0x7f18240b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:39.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.616+0000 7f181bfff700 1 -- 172.21.15.92:0/2560081717 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182415a670 con 0x7f182415a160 2023-10-06T10:41:39.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.617+0000 7f181bfff700 1 --2- 172.21.15.92:0/2560081717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f182415a160 0x7f18240ccdc0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f182408ab60 tx=0x7f1814009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e430791cc40053f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:39.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.617+0000 7f181b7fe700 1 -- 172.21.15.92:0/2560081717 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181400b860 con 0x7f182415a160 2023-10-06T10:41:39.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.617+0000 7f181b7fe700 1 -- 172.21.15.92:0/2560081717 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181400b9c0 con 0x7f182415a160 2023-10-06T10:41:39.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.617+0000 7f181b7fe700 1 -- 172.21.15.92:0/2560081717 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181400bd60 con 0x7f182415a160 2023-10-06T10:41:39.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.617+0000 7f182ab90700 1 -- 172.21.15.92:0/2560081717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f182415a160 msgr2=0x7f18240ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:39.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.617+0000 7f182ab90700 1 --2- 172.21.15.92:0/2560081717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f182415a160 0x7f18240ccdc0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f182408ab60 tx=0x7f1814009250 comp rx=0 tx=0).stop 2023-10-06T10:41:39.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.617+0000 7f182ab90700 1 -- 172.21.15.92:0/2560081717 shutdown_connections 2023-10-06T10:41:39.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.617+0000 7f182ab90700 1 --2- 172.21.15.92:0/2560081717 >> v2:172.21.15.124:3300/0 conn(0x7f1824150940 0x7f1824159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:39.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.617+0000 7f182ab90700 1 --2- 172.21.15.92:0/2560081717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f182415a160 0x7f18240ccdc0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:39.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.617+0000 7f182ab90700 1 -- 172.21.15.92:0/2560081717 >> 172.21.15.92:0/2560081717 conn(0x7f1824099700 msgr2=0x7f182406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:39.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.617+0000 7f182ab90700 1 -- 172.21.15.92:0/2560081717 shutdown_connections 2023-10-06T10:41:39.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.617+0000 7f182ab90700 1 -- 172.21.15.92:0/2560081717 wait complete. 2023-10-06T10:41:39.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.618+0000 7f182ab90700 1 Processor -- start 2023-10-06T10:41:39.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.619+0000 7f182ab90700 1 -- start start 2023-10-06T10:41:39.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.619+0000 7f182ab90700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18240c5250 0x7f182409db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:39.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.619+0000 7f182ab90700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f182415a160 0x7f18240c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:39.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.619+0000 7f182ab90700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1824169c40 0x7f182409a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:39.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.619+0000 7f182ab90700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18240c9ab0 con 0x7f1824169c40 2023-10-06T10:41:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.619+0000 7f182ab90700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18240c8ea0 con 0x7f182415a160 2023-10-06T10:41:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.619+0000 7f182ab90700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18240c9180 con 0x7f18240c5250 2023-10-06T10:41:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.619+0000 7f182912d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1824169c40 0x7f182409a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.619+0000 7f182912d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1824169c40 0x7f182409a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53032/0 (socket says 172.21.15.92:53032) 2023-10-06T10:41:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.619+0000 7f182912d700 1 -- 172.21.15.92:0/232332453 learned_addr learned my addr 172.21.15.92:0/232332453 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.619+0000 7f181bfff700 1 --2- 172.21.15.92:0/232332453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f182415a160 0x7f18240c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.619+0000 7f182892c700 1 --2- 172.21.15.92:0/232332453 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18240c5250 0x7f182409db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.620+0000 7f182912d700 1 -- 172.21.15.92:0/232332453 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18240c5250 msgr2=0x7f182409db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.620+0000 7f182912d700 1 --2- 172.21.15.92:0/232332453 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18240c5250 0x7f182409db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.620+0000 7f182912d700 1 -- 172.21.15.92:0/232332453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f182415a160 msgr2=0x7f18240c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.620+0000 7f182912d700 1 --2- 172.21.15.92:0/232332453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f182415a160 0x7f18240c8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.620+0000 7f182912d700 1 -- 172.21.15.92:0/232332453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18140094f0 con 0x7f1824169c40 2023-10-06T10:41:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.620+0000 7f181bfff700 1 --2- 172.21.15.92:0/232332453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f182415a160 0x7f18240c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.620+0000 7f182892c700 1 --2- 172.21.15.92:0/232332453 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18240c5250 0x7f182409db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.620+0000 7f182912d700 1 --2- 172.21.15.92:0/232332453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1824169c40 0x7f182409a070 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f181c00c8e0 tx=0x7f181c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.620+0000 7f1818ff9700 1 -- 172.21.15.92:0/232332453 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c00ce80 con 0x7f1824169c40 2023-10-06T10:41:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.620+0000 7f1818ff9700 1 -- 172.21.15.92:0/232332453 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c00a7e0 con 0x7f1824169c40 2023-10-06T10:41:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.620+0000 7f182ab90700 1 -- 172.21.15.92:0/232332453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18240c9460 con 0x7f1824169c40 2023-10-06T10:41:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.620+0000 7f1818ff9700 1 -- 172.21.15.92:0/232332453 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c00ab80 con 0x7f1824169c40 2023-10-06T10:41:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.620+0000 7f182ab90700 1 -- 172.21.15.92:0/232332453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182409e050 con 0x7f1824169c40 2023-10-06T10:41:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.621+0000 7f182ab90700 1 -- 172.21.15.92:0/232332453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1824064a90 con 0x7f1824169c40 2023-10-06T10:41:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.621+0000 7f1818ff9700 1 -- 172.21.15.92:0/232332453 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f181c012420 con 0x7f1824169c40 2023-10-06T10:41:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.621+0000 7f1818ff9700 1 --2- 172.21.15.92:0/232332453 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1810066f30 0x7f18100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.622+0000 7f1818ff9700 1 -- 172.21.15.92:0/232332453 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f181c084670 con 0x7f1824169c40 2023-10-06T10:41:39.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.622+0000 7f182892c700 1 --2- 172.21.15.92:0/232332453 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1810066f30 0x7f18100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:39.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.622+0000 7f182892c700 1 --2- 172.21.15.92:0/232332453 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1810066f30 0x7f18100693e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f18240cf650 tx=0x7f1820008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:39.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.626+0000 7f1818ff9700 1 -- 172.21.15.92:0/232332453 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f181c051890 con 0x7f1824169c40 2023-10-06T10:41:39.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.819+0000 7f182ab90700 1 -- 172.21.15.92:0/232332453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f182415ac30 con 0x7f1824169c40 2023-10-06T10:41:39.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.823+0000 7f1818ff9700 1 -- 172.21.15.92:0/232332453 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f182415ac30 con 0x7f1824169c40 2023-10-06T10:41:39.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.825+0000 7f182ab90700 1 -- 172.21.15.92:0/232332453 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1810066f30 msgr2=0x7f18100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:39.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.825+0000 7f182ab90700 1 --2- 172.21.15.92:0/232332453 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1810066f30 0x7f18100693e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f18240cf650 tx=0x7f1820008040 comp rx=0 tx=0).stop 2023-10-06T10:41:39.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.825+0000 7f182ab90700 1 -- 172.21.15.92:0/232332453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1824169c40 msgr2=0x7f182409a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:39.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.825+0000 7f182ab90700 1 --2- 172.21.15.92:0/232332453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1824169c40 0x7f182409a070 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f181c00c8e0 tx=0x7f181c001710 comp rx=0 tx=0).stop 2023-10-06T10:41:39.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.825+0000 7f182ab90700 1 -- 172.21.15.92:0/232332453 shutdown_connections 2023-10-06T10:41:39.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.825+0000 7f182ab90700 1 --2- 172.21.15.92:0/232332453 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1810066f30 0x7f18100693e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:39.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.825+0000 7f182ab90700 1 --2- 172.21.15.92:0/232332453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1824169c40 0x7f182409a070 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:39.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.825+0000 7f182ab90700 1 --2- 172.21.15.92:0/232332453 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18240c5250 0x7f182409db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:39.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.825+0000 7f182ab90700 1 --2- 172.21.15.92:0/232332453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f182415a160 0x7f18240c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:39.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.825+0000 7f182ab90700 1 -- 172.21.15.92:0/232332453 >> 172.21.15.92:0/232332453 conn(0x7f1824099700 msgr2=0x7f18240b5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:39.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.825+0000 7f182ab90700 1 -- 172.21.15.92:0/232332453 shutdown_connections 2023-10-06T10:41:39.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.825+0000 7f182ab90700 1 -- 172.21.15.92:0/232332453 wait complete. 2023-10-06T10:41:39.838 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0' 2023-10-06T10:41:39.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.952+0000 7fc6698b5700 1 Processor -- start 2023-10-06T10:41:39.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.952+0000 7fc6698b5700 1 -- start start 2023-10-06T10:41:39.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.952+0000 7fc6698b5700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc664159950 0x7fc6640675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:39.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.952+0000 7fc6698b5700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc664067af0 0x7fc6640b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:39.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.952+0000 7fc6698b5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc664067f00 con 0x7fc664159580 2023-10-06T10:41:39.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc6698b5700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6640cde30 con 0x7fc664159950 2023-10-06T10:41:39.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc6698b5700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66406ef20 con 0x7fc664067af0 2023-10-06T10:41:39.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc662ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc664159580 0x7fc66406e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47516/0 (socket says 172.21.15.92:47516) 2023-10-06T10:41:39.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc662ffd700 1 -- 172.21.15.92:0/2865816447 learned_addr learned my addr 172.21.15.92:0/2865816447 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:39.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc661ffb700 1 -- 172.21.15.92:0/2865816447 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3522957096 0 0) 0x7fc664067f00 con 0x7fc664159580 2023-10-06T10:41:39.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc6627fc700 1 --2- 172.21.15.92:0/2865816447 >> v2:172.21.15.124:3300/0 conn(0x7fc664159950 0x7fc6640675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:39.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc661ffb700 1 -- 172.21.15.92:0/2865816447 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc64c003570 con 0x7fc664159580 2023-10-06T10:41:39.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc6637fe700 1 --2- 172.21.15.92:0/2865816447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc664067af0 0x7fc6640b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:39.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc661ffb700 1 -- 172.21.15.92:0/2865816447 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3360967462 0 0) 0x7fc64c003570 con 0x7fc664159580 2023-10-06T10:41:39.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc661ffb700 1 -- 172.21.15.92:0/2865816447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc664067af0 msgr2=0x7fc6640b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:39.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc661ffb700 1 --2- 172.21.15.92:0/2865816447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc664067af0 0x7fc6640b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:39.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc661ffb700 1 -- 172.21.15.92:0/2865816447 >> v2:172.21.15.124:3300/0 conn(0x7fc664159950 msgr2=0x7fc6640675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:39.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc661ffb700 1 --2- 172.21.15.92:0/2865816447 >> v2:172.21.15.124:3300/0 conn(0x7fc664159950 0x7fc6640675b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:39.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc661ffb700 1 -- 172.21.15.92:0/2865816447 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66406f200 con 0x7fc664159580 2023-10-06T10:41:39.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc661ffb700 1 -- 172.21.15.92:0/2865816447 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fc654001470 con 0x7fc664159580 2023-10-06T10:41:39.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc661ffb700 1 -- 172.21.15.92:0/2865816447 >> v1:172.21.15.92:6789/0 conn(0x7fc664159580 legacy=0x7fc66406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:39.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc661ffb700 1 --2- 172.21.15.92:0/2865816447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc64c003a60 0x7fc64c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:39.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc661ffb700 1 --2- 172.21.15.92:0/2865816447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc664067af0 0x7fc64c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:39.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc661ffb700 1 --2- 172.21.15.92:0/2865816447 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc64c004d50 0x7fc64c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:39.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc661ffb700 1 -- 172.21.15.92:0/2865816447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6640cde30 con 0x7fc64c003a60 2023-10-06T10:41:39.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc661ffb700 1 -- 172.21.15.92:0/2865816447 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66406ef20 con 0x7fc64c004d50 2023-10-06T10:41:39.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc661ffb700 1 -- 172.21.15.92:0/2865816447 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c00ba60 con 0x7fc664067af0 2023-10-06T10:41:39.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc662ffd700 1 --2- 172.21.15.92:0/2865816447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc64c003a60 0x7fc64c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:39.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc6627fc700 1 --2- 172.21.15.92:0/2865816447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc664067af0 0x7fc64c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:39.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.953+0000 7fc6637fe700 1 --2- 172.21.15.92:0/2865816447 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc64c004d50 0x7fc64c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:39.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.954+0000 7fc662ffd700 1 -- 172.21.15.92:0/2865816447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc664067af0 msgr2=0x7fc64c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:39.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.954+0000 7fc662ffd700 1 --2- 172.21.15.92:0/2865816447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc664067af0 0x7fc64c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:39.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.954+0000 7fc662ffd700 1 -- 172.21.15.92:0/2865816447 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc64c004d50 msgr2=0x7fc64c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:39.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.954+0000 7fc662ffd700 1 --2- 172.21.15.92:0/2865816447 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc64c004d50 0x7fc64c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:39.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.954+0000 7fc662ffd700 1 -- 172.21.15.92:0/2865816447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c003570 con 0x7fc64c003a60 2023-10-06T10:41:39.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.954+0000 7fc6637fe700 1 --2- 172.21.15.92:0/2865816447 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc64c004d50 0x7fc64c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:39.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.954+0000 7fc662ffd700 1 --2- 172.21.15.92:0/2865816447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc64c003a60 0x7fc64c003e70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc65400c3c0 tx=0x7fc654001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837a03cc7138cf5f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:39.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.954+0000 7fc661ffb700 1 -- 172.21.15.92:0/2865816447 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65400cc30 con 0x7fc64c003a60 2023-10-06T10:41:39.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.954+0000 7fc661ffb700 1 -- 172.21.15.92:0/2865816447 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65400cdb0 con 0x7fc64c003a60 2023-10-06T10:41:39.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.954+0000 7fc661ffb700 1 -- 172.21.15.92:0/2865816447 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc654005b70 con 0x7fc64c003a60 2023-10-06T10:41:39.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.954+0000 7fc6698b5700 1 -- 172.21.15.92:0/2865816447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc64c003a60 msgr2=0x7fc64c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:39.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.954+0000 7fc6698b5700 1 --2- 172.21.15.92:0/2865816447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc64c003a60 0x7fc64c003e70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc65400c3c0 tx=0x7fc654001e90 comp rx=0 tx=0).stop 2023-10-06T10:41:39.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.954+0000 7fc6637fe700 1 -- 172.21.15.92:0/2865816447 reap_dead start 2023-10-06T10:41:39.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.954+0000 7fc6698b5700 1 -- 172.21.15.92:0/2865816447 shutdown_connections 2023-10-06T10:41:39.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.954+0000 7fc6698b5700 1 -- 172.21.15.92:0/2865816447 >> 172.21.15.92:0/2865816447 conn(0x7fc664069140 msgr2=0x7fc6640cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:39.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.955+0000 7fc6698b5700 1 -- 172.21.15.92:0/2865816447 shutdown_connections 2023-10-06T10:41:39.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.955+0000 7fc6698b5700 1 -- 172.21.15.92:0/2865816447 wait complete. 2023-10-06T10:41:39.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.956+0000 7fc6698b5700 1 Processor -- start 2023-10-06T10:41:39.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.956+0000 7fc6698b5700 1 -- start start 2023-10-06T10:41:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.957+0000 7fc6698b5700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc64c003a60 0x7fc6640bd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.957+0000 7fc6698b5700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6640c73c0 0x7fc6640bd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.957+0000 7fc6698b5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6640bcc20 0x7fc6640c5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.957+0000 7fc6698b5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6640c41f0 con 0x7fc6640bcc20 2023-10-06T10:41:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.957+0000 7fc6698b5700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6640c4d00 con 0x7fc64c003a60 2023-10-06T10:41:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.957+0000 7fc6698b5700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6640c43d0 con 0x7fc6640c73c0 2023-10-06T10:41:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.957+0000 7fc6637fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6640bcc20 0x7fc6640c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.957+0000 7fc6637fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6640bcc20 0x7fc6640c5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53060/0 (socket says 172.21.15.92:53060) 2023-10-06T10:41:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.957+0000 7fc6637fe700 1 -- 172.21.15.92:0/3168830794 learned_addr learned my addr 172.21.15.92:0/3168830794 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.957+0000 7fc662ffd700 1 --2- 172.21.15.92:0/3168830794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc64c003a60 0x7fc6640bd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.957+0000 7fc6627fc700 1 --2- 172.21.15.92:0/3168830794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6640c73c0 0x7fc6640bd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.957+0000 7fc6637fe700 1 -- 172.21.15.92:0/3168830794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6640c73c0 msgr2=0x7fc6640bd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.957+0000 7fc6637fe700 1 --2- 172.21.15.92:0/3168830794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6640c73c0 0x7fc6640bd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.957+0000 7fc6637fe700 1 -- 172.21.15.92:0/3168830794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc64c003a60 msgr2=0x7fc6640bd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.957+0000 7fc6637fe700 1 --2- 172.21.15.92:0/3168830794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc64c003a60 0x7fc6640bd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.957+0000 7fc6637fe700 1 -- 172.21.15.92:0/3168830794 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6640c46b0 con 0x7fc6640bcc20 2023-10-06T10:41:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.957+0000 7fc6637fe700 1 --2- 172.21.15.92:0/3168830794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6640bcc20 0x7fc6640c5190 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fc6640cacc0 tx=0x7fc65c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.957+0000 7fc64affd700 1 -- 172.21.15.92:0/3168830794 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c00e050 con 0x7fc6640bcc20 2023-10-06T10:41:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.957+0000 7fc64affd700 1 -- 172.21.15.92:0/3168830794 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65c00f040 con 0x7fc6640bcc20 2023-10-06T10:41:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.957+0000 7fc64affd700 1 -- 172.21.15.92:0/3168830794 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c010d70 con 0x7fc6640bcc20 2023-10-06T10:41:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.957+0000 7fc6698b5700 1 -- 172.21.15.92:0/3168830794 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6640c4990 con 0x7fc6640bcc20 2023-10-06T10:41:39.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.957+0000 7fc6698b5700 1 -- 172.21.15.92:0/3168830794 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6640c56d0 con 0x7fc6640bcc20 2023-10-06T10:41:39.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.958+0000 7fc6698b5700 1 -- 172.21.15.92:0/3168830794 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc664064a90 con 0x7fc6640bcc20 2023-10-06T10:41:39.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.959+0000 7fc64affd700 1 -- 172.21.15.92:0/3168830794 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc65c015650 con 0x7fc6640bcc20 2023-10-06T10:41:39.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.959+0000 7fc64affd700 1 --2- 172.21.15.92:0/3168830794 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc650066f30 0x7fc6500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:39.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.959+0000 7fc64affd700 1 -- 172.21.15.92:0/3168830794 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5784+0+0 (secure 0 0 0) 0x7fc65c084ef0 con 0x7fc6640bcc20 2023-10-06T10:41:39.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.959+0000 7fc662ffd700 1 --2- 172.21.15.92:0/3168830794 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc650066f30 0x7fc6500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:39.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.959+0000 7fc662ffd700 1 --2- 172.21.15.92:0/3168830794 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc650066f30 0x7fc6500693e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc654001ed0 tx=0x7fc654015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:39.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:39.963+0000 7fc64affd700 1 -- 172.21.15.92:0/3168830794 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc65c052110 con 0x7fc6640bcc20 2023-10-06T10:41:40.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.220+0000 7fc6698b5700 1 -- 172.21.15.92:0/3168830794 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]} v 0) v1 -- 0x7fc6640c4e40 con 0x7fc6640bcc20 2023-10-06T10:41:40.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.478+0000 7fc64affd700 1 -- 172.21.15.92:0/3168830794 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]}]=0 v183) v1 ==== 143+0+0 (secure 0 0 0) 0x7fc65c055730 con 0x7fc6640bcc20 2023-10-06T10:41:40.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.480+0000 7fc6698b5700 1 -- 172.21.15.92:0/3168830794 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc650066f30 msgr2=0x7fc6500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:40.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.480+0000 7fc6698b5700 1 --2- 172.21.15.92:0/3168830794 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc650066f30 0x7fc6500693e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc654001ed0 tx=0x7fc654015000 comp rx=0 tx=0).stop 2023-10-06T10:41:40.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.480+0000 7fc6698b5700 1 -- 172.21.15.92:0/3168830794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6640bcc20 msgr2=0x7fc6640c5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:40.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.480+0000 7fc6698b5700 1 --2- 172.21.15.92:0/3168830794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6640bcc20 0x7fc6640c5190 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fc6640cacc0 tx=0x7fc65c00b040 comp rx=0 tx=0).stop 2023-10-06T10:41:40.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.480+0000 7fc6698b5700 1 -- 172.21.15.92:0/3168830794 shutdown_connections 2023-10-06T10:41:40.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.480+0000 7fc6698b5700 1 --2- 172.21.15.92:0/3168830794 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc650066f30 0x7fc6500693e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:40.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.480+0000 7fc6698b5700 1 --2- 172.21.15.92:0/3168830794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6640bcc20 0x7fc6640c5190 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:40.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.480+0000 7fc6698b5700 1 --2- 172.21.15.92:0/3168830794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc64c003a60 0x7fc6640bd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:40.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.480+0000 7fc6698b5700 1 --2- 172.21.15.92:0/3168830794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6640c73c0 0x7fc6640bd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:40.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.480+0000 7fc6698b5700 1 -- 172.21.15.92:0/3168830794 >> 172.21.15.92:0/3168830794 conn(0x7fc664069140 msgr2=0x7fc6640b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:40.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.480+0000 7fc6698b5700 1 -- 172.21.15.92:0/3168830794 shutdown_connections 2023-10-06T10:41:40.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.480+0000 7fc6698b5700 1 -- 172.21.15.92:0/3168830794 wait complete. 2023-10-06T10:41:40.494 INFO:tasks.cephfs_test_runner:test_scrub_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2023-10-06T10:41:40.495 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace (tasks.cephfs.test_multimds_misc.TestScrub2) 2023-10-06T10:41:40.495 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace' 2023-10-06T10:41:40.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.607+0000 7f3f11242700 1 Processor -- start 2023-10-06T10:41:40.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.607+0000 7f3f11242700 1 -- start start 2023-10-06T10:41:40.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.608+0000 7f3f11242700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0c159560 0x7f3f0c0aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:40.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.608+0000 7f3f11242700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3f0c0b49a0 0x7f3f0c0aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:40.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.608+0000 7f3f11242700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0c0b0c60 con 0x7f3f0c0b0760 2023-10-06T10:41:40.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.608+0000 7f3f11242700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0c0b0f40 con 0x7f3f0c0b49a0 2023-10-06T10:41:40.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.608+0000 7f3f11242700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c0b1220 con 0x7f3f0c159560 2023-10-06T10:41:40.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.608+0000 7f3f0259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3f0c0b49a0 0x7f3f0c0aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:40.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.608+0000 7f3f0b59e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3f0c0b0760 0x7f3f0c0b3890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47518/0 (socket says 172.21.15.92:47518) 2023-10-06T10:41:40.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.608+0000 7f3f0b59e700 1 -- 172.21.15.92:0/664199341 learned_addr learned my addr 172.21.15.92:0/664199341 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:40.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.608+0000 7f3f0ad9d700 1 --2- 172.21.15.92:0/664199341 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0c159560 0x7f3f0c0aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:40.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.609+0000 7f3f0259c700 1 -- 172.21.15.92:0/664199341 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0c159560 msgr2=0x7f3f0c0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.609+0000 7f3f0259c700 1 --2- 172.21.15.92:0/664199341 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0c159560 0x7f3f0c0aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.609+0000 7f3f0259c700 1 -- 172.21.15.92:0/664199341 >> v1:172.21.15.92:6789/0 conn(0x7f3f0c0b0760 legacy=0x7f3f0c0b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.609+0000 7f3f0259c700 1 -- 172.21.15.92:0/664199341 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c0b1500 con 0x7f3f0c0b49a0 2023-10-06T10:41:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.609+0000 7f3f0a59c700 1 -- 172.21.15.92:0/664199341 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22705150 0 0) 0x7f3f0c0b0c60 con 0x7f3f0c0b0760 2023-10-06T10:41:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.609+0000 7f3f0259c700 1 --2- 172.21.15.92:0/664199341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f0c0b49a0 0x7f3f0c0aaa70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40098e0 tx=0x7f3ef40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce480f9a80d3f63e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.609+0000 7f3f0a59c700 1 -- 172.21.15.92:0/664199341 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef400b640 con 0x7f3f0c0b49a0 2023-10-06T10:41:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.609+0000 7f3f0a59c700 1 -- 172.21.15.92:0/664199341 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef400b7a0 con 0x7f3f0c0b49a0 2023-10-06T10:41:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.609+0000 7f3f0a59c700 1 -- 172.21.15.92:0/664199341 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef400bb40 con 0x7f3f0c0b49a0 2023-10-06T10:41:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.609+0000 7f3f11242700 1 -- 172.21.15.92:0/664199341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f0c0b49a0 msgr2=0x7f3f0c0aaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.609+0000 7f3f11242700 1 --2- 172.21.15.92:0/664199341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f0c0b49a0 0x7f3f0c0aaa70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40098e0 tx=0x7f3ef40091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.609+0000 7f3f11242700 1 -- 172.21.15.92:0/664199341 shutdown_connections 2023-10-06T10:41:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.609+0000 7f3f11242700 1 --2- 172.21.15.92:0/664199341 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0c159560 0x7f3f0c0aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:40.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.609+0000 7f3f11242700 1 --2- 172.21.15.92:0/664199341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f0c0b49a0 0x7f3f0c0aaa70 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:40.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.609+0000 7f3f11242700 1 -- 172.21.15.92:0/664199341 >> 172.21.15.92:0/664199341 conn(0x7f3f0c0997f0 msgr2=0x7f3f0c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:40.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.609+0000 7f3f11242700 1 -- 172.21.15.92:0/664199341 shutdown_connections 2023-10-06T10:41:40.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.609+0000 7f3f11242700 1 -- 172.21.15.92:0/664199341 wait complete. 2023-10-06T10:41:40.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.611+0000 7f3f11242700 1 Processor -- start 2023-10-06T10:41:40.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.611+0000 7f3f11242700 1 -- start start 2023-10-06T10:41:40.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.611+0000 7f3f11242700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f0c0b0760 0x7f3f0c147dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:40.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.611+0000 7f3f11242700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0c0b49a0 0x7f3f0c146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:40.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.611+0000 7f3f11242700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f0c159560 0x7f3f0c1474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:40.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.611+0000 7f3f11242700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c146ae0 con 0x7f3f0c0b0760 2023-10-06T10:41:40.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.611+0000 7f3f11242700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c15d290 con 0x7f3f0c159560 2023-10-06T10:41:40.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.611+0000 7f3f11242700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c145ed0 con 0x7f3f0c0b49a0 2023-10-06T10:41:40.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.611+0000 7f3f0ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f0c0b0760 0x7f3f0c147dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:40.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.611+0000 7f3f0ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f0c0b0760 0x7f3f0c147dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53090/0 (socket says 172.21.15.92:53090) 2023-10-06T10:41:40.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.611+0000 7f3f0ad9d700 1 -- 172.21.15.92:0/3702162841 learned_addr learned my addr 172.21.15.92:0/3702162841 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:40.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.612+0000 7f3f0259c700 1 --2- 172.21.15.92:0/3702162841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0c0b49a0 0x7f3f0c146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:40.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.612+0000 7f3f0ad9d700 1 -- 172.21.15.92:0/3702162841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0c0b49a0 msgr2=0x7f3f0c146f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:40.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.612+0000 7f3f0ad9d700 1 --2- 172.21.15.92:0/3702162841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0c0b49a0 0x7f3f0c146f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:40.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.612+0000 7f3f0ad9d700 1 -- 172.21.15.92:0/3702162841 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f0c159560 msgr2=0x7f3f0c1474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:40.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.612+0000 7f3f0ad9d700 1 --2- 172.21.15.92:0/3702162841 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f0c159560 0x7f3f0c1474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:40.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.612+0000 7f3f0ad9d700 1 -- 172.21.15.92:0/3702162841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef4009420 con 0x7f3f0c0b0760 2023-10-06T10:41:40.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.612+0000 7f3f0ad9d700 1 --2- 172.21.15.92:0/3702162841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f0c0b0760 0x7f3f0c147dd0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00ea00 tx=0x7f3efc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:40.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.612+0000 7f3f037fe700 1 -- 172.21.15.92:0/3702162841 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc00c850 con 0x7f3f0c0b0760 2023-10-06T10:41:40.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.612+0000 7f3f037fe700 1 -- 172.21.15.92:0/3702162841 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3efc00c9b0 con 0x7f3f0c0b0760 2023-10-06T10:41:40.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.612+0000 7f3f11242700 1 -- 172.21.15.92:0/3702162841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0c1461b0 con 0x7f3f0c0b0760 2023-10-06T10:41:40.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.612+0000 7f3f037fe700 1 -- 172.21.15.92:0/3702162841 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc00cd30 con 0x7f3f0c0b0760 2023-10-06T10:41:40.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.612+0000 7f3f11242700 1 -- 172.21.15.92:0/3702162841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0c146770 con 0x7f3f0c0b0760 2023-10-06T10:41:40.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.613+0000 7f3f11242700 1 -- 172.21.15.92:0/3702162841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0c064a90 con 0x7f3f0c0b0760 2023-10-06T10:41:40.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.613+0000 7f3f037fe700 1 -- 172.21.15.92:0/3702162841 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3efc013680 con 0x7f3f0c0b0760 2023-10-06T10:41:40.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.614+0000 7f3f037fe700 1 --2- 172.21.15.92:0/3702162841 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3eec066f30 0x7f3eec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:40.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.614+0000 7f3f0259c700 1 --2- 172.21.15.92:0/3702162841 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3eec066f30 0x7f3eec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:40.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.614+0000 7f3f037fe700 1 -- 172.21.15.92:0/3702162841 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f3efc084c20 con 0x7f3f0c0b0760 2023-10-06T10:41:40.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.614+0000 7f3f0259c700 1 --2- 172.21.15.92:0/3702162841 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3eec066f30 0x7f3eec0693e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0b4db0 tx=0x7f3ef4017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:40.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.618+0000 7f3f037fe700 1 -- 172.21.15.92:0/3702162841 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3efc051e40 con 0x7f3f0c0b0760 2023-10-06T10:41:40.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:40.877+0000 7f3f11242700 1 -- 172.21.15.92:0/3702162841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]} v 0) v1 -- 0x7f3f0c0b17e0 con 0x7f3f0c0b0760 2023-10-06T10:41:41.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.479+0000 7f3f037fe700 1 -- 172.21.15.92:0/3702162841 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]}]=0 v184) v1 ==== 145+0+0 (secure 0 0 0) 0x7f3efc0210d0 con 0x7f3f0c0b0760 2023-10-06T10:41:41.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.481+0000 7f3f11242700 1 -- 172.21.15.92:0/3702162841 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3eec066f30 msgr2=0x7f3eec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:41.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.481+0000 7f3f11242700 1 --2- 172.21.15.92:0/3702162841 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3eec066f30 0x7f3eec0693e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0b4db0 tx=0x7f3ef4017000 comp rx=0 tx=0).stop 2023-10-06T10:41:41.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.481+0000 7f3f11242700 1 -- 172.21.15.92:0/3702162841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f0c0b0760 msgr2=0x7f3f0c147dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:41.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.481+0000 7f3f11242700 1 --2- 172.21.15.92:0/3702162841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f0c0b0760 0x7f3f0c147dd0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00ea00 tx=0x7f3efc009120 comp rx=0 tx=0).stop 2023-10-06T10:41:41.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.481+0000 7f3f11242700 1 -- 172.21.15.92:0/3702162841 shutdown_connections 2023-10-06T10:41:41.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.481+0000 7f3f11242700 1 --2- 172.21.15.92:0/3702162841 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3eec066f30 0x7f3eec0693e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:41.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.481+0000 7f3f11242700 1 --2- 172.21.15.92:0/3702162841 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f0c159560 0x7f3f0c1474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:41.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.481+0000 7f3f11242700 1 --2- 172.21.15.92:0/3702162841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f0c0b0760 0x7f3f0c147dd0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:41.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.481+0000 7f3f11242700 1 --2- 172.21.15.92:0/3702162841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0c0b49a0 0x7f3f0c146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:41.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.481+0000 7f3f11242700 1 -- 172.21.15.92:0/3702162841 >> 172.21.15.92:0/3702162841 conn(0x7f3f0c0997f0 msgr2=0x7f3f0c0d0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:41.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.481+0000 7f3f11242700 1 -- 172.21.15.92:0/3702162841 shutdown_connections 2023-10-06T10:41:41.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.481+0000 7f3f11242700 1 -- 172.21.15.92:0/3702162841 wait complete. 2023-10-06T10:41:41.495 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T10:41:41.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.611+0000 7fca43fff700 1 Processor -- start 2023-10-06T10:41:41.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.611+0000 7fca43fff700 1 -- start start 2023-10-06T10:41:41.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.612+0000 7fca43fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fca440b29f0 0x7fca44164320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:41.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.612+0000 7fca43fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca440cc970 con 0x7fca440b2620 2023-10-06T10:41:41.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.612+0000 7fca43fff700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca440cbd60 con 0x7fca440cd3c0 2023-10-06T10:41:41.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.612+0000 7fca43fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca440cc040 con 0x7fca440b29f0 2023-10-06T10:41:41.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.612+0000 7fca437fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fca440b29f0 0x7fca44164320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:41.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.612+0000 7fca437fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fca440b29f0 0x7fca44164320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53102/0 (socket says 172.21.15.92:53102) 2023-10-06T10:41:41.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.612+0000 7fca42ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fca440b2620 0x7fca440cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47528/0 (socket says 172.21.15.92:47528) 2023-10-06T10:41:41.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.612+0000 7fca437fe700 1 -- 172.21.15.92:0/1934877970 learned_addr learned my addr 172.21.15.92:0/1934877970 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:41.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.613+0000 7fca437fe700 1 -- 172.21.15.92:0/1934877970 >> v1:172.21.15.124:6789/0 conn(0x7fca440cd3c0 legacy=0x7fca440ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:41.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.613+0000 7fca437fe700 1 -- 172.21.15.92:0/1934877970 >> v1:172.21.15.92:6789/0 conn(0x7fca440b2620 legacy=0x7fca440cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:41.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.613+0000 7fca437fe700 1 -- 172.21.15.92:0/1934877970 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca440cc320 con 0x7fca440b29f0 2023-10-06T10:41:41.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.613+0000 7fca427fc700 1 -- 172.21.15.92:0/1934877970 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 60892335 0 0) 0x7fca440cc970 con 0x7fca440b2620 2023-10-06T10:41:41.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.613+0000 7fca437fe700 1 --2- 172.21.15.92:0/1934877970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca440b29f0 0x7fca44164320 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009f20 tx=0x7fca3c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5ea4504b7413eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:41.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.613+0000 7fca427fc700 1 -- 172.21.15.92:0/1934877970 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c00e070 con 0x7fca440b29f0 2023-10-06T10:41:41.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.613+0000 7fca427fc700 1 -- 172.21.15.92:0/1934877970 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3c00ba40 con 0x7fca440b29f0 2023-10-06T10:41:41.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.613+0000 7fca427fc700 1 -- 172.21.15.92:0/1934877970 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c0092e0 con 0x7fca440b29f0 2023-10-06T10:41:41.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.613+0000 7fca43fff700 1 -- 172.21.15.92:0/1934877970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca440b29f0 msgr2=0x7fca44164320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:41.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.613+0000 7fca43fff700 1 --2- 172.21.15.92:0/1934877970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca440b29f0 0x7fca44164320 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009f20 tx=0x7fca3c009840 comp rx=0 tx=0).stop 2023-10-06T10:41:41.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.613+0000 7fca43fff700 1 -- 172.21.15.92:0/1934877970 shutdown_connections 2023-10-06T10:41:41.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.613+0000 7fca43fff700 1 --2- 172.21.15.92:0/1934877970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca440b29f0 0x7fca44164320 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:41.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.613+0000 7fca43fff700 1 -- 172.21.15.92:0/1934877970 >> 172.21.15.92:0/1934877970 conn(0x7fca44099520 msgr2=0x7fca4406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:41.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.614+0000 7fca43fff700 1 -- 172.21.15.92:0/1934877970 shutdown_connections 2023-10-06T10:41:41.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.614+0000 7fca43fff700 1 -- 172.21.15.92:0/1934877970 wait complete. 2023-10-06T10:41:41.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.615+0000 7fca43fff700 1 Processor -- start 2023-10-06T10:41:41.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.615+0000 7fca43fff700 1 -- start start 2023-10-06T10:41:41.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.615+0000 7fca43fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca440b2620 0x7fca44154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:41.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.615+0000 7fca43fff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca440b29f0 0x7fca441545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:41.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.615+0000 7fca43fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca440cd3c0 0x7fca44156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:41.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.615+0000 7fca43fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4414bd30 con 0x7fca440b2620 2023-10-06T10:41:41.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.615+0000 7fca43fff700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4415a210 con 0x7fca440b29f0 2023-10-06T10:41:41.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.615+0000 7fca43fff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4416d8d0 con 0x7fca440cd3c0 2023-10-06T10:41:41.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.615+0000 7fca42ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca440b2620 0x7fca44154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:41.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.615+0000 7fca42ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca440b2620 0x7fca44154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53108/0 (socket says 172.21.15.92:53108) 2023-10-06T10:41:41.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.615+0000 7fca42ffd700 1 -- 172.21.15.92:0/3571492658 learned_addr learned my addr 172.21.15.92:0/3571492658 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:41.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.616+0000 7fca3bfff700 1 --2- 172.21.15.92:0/3571492658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca440b29f0 0x7fca441545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:41.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.616+0000 7fca437fe700 1 --2- 172.21.15.92:0/3571492658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca440cd3c0 0x7fca44156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:41.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.616+0000 7fca42ffd700 1 -- 172.21.15.92:0/3571492658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca440cd3c0 msgr2=0x7fca44156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:41.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.616+0000 7fca42ffd700 1 --2- 172.21.15.92:0/3571492658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca440cd3c0 0x7fca44156da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:41.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.616+0000 7fca42ffd700 1 -- 172.21.15.92:0/3571492658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca440b29f0 msgr2=0x7fca441545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:41.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.616+0000 7fca42ffd700 1 --2- 172.21.15.92:0/3571492658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca440b29f0 0x7fca441545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:41.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.616+0000 7fca42ffd700 1 -- 172.21.15.92:0/3571492658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c009ae0 con 0x7fca440b2620 2023-10-06T10:41:41.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.616+0000 7fca3bfff700 1 --2- 172.21.15.92:0/3571492658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca440b29f0 0x7fca441545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:41.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.616+0000 7fca437fe700 1 --2- 172.21.15.92:0/3571492658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca440cd3c0 0x7fca44156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:41.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.616+0000 7fca42ffd700 1 --2- 172.21.15.92:0/3571492658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca440b2620 0x7fca44154080 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fca3400c5b0 tx=0x7fca34007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:41.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.616+0000 7fca3affd700 1 -- 172.21.15.92:0/3571492658 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca340072e0 con 0x7fca440b2620 2023-10-06T10:41:41.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.616+0000 7fca3affd700 1 -- 172.21.15.92:0/3571492658 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca34007460 con 0x7fca440b2620 2023-10-06T10:41:41.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.616+0000 7fca43fff700 1 -- 172.21.15.92:0/3571492658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4416dbb0 con 0x7fca440b2620 2023-10-06T10:41:41.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.616+0000 7fca3affd700 1 -- 172.21.15.92:0/3571492658 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca340057f0 con 0x7fca440b2620 2023-10-06T10:41:41.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.616+0000 7fca43fff700 1 -- 172.21.15.92:0/3571492658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca44159ea0 con 0x7fca440b2620 2023-10-06T10:41:41.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.617+0000 7fca43fff700 1 -- 172.21.15.92:0/3571492658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4405f230 con 0x7fca440b2620 2023-10-06T10:41:41.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.617+0000 7fca3affd700 1 -- 172.21.15.92:0/3571492658 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fca34005bc0 con 0x7fca440b2620 2023-10-06T10:41:41.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.618+0000 7fca3affd700 1 --2- 172.21.15.92:0/3571492658 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fca24066f30 0x7fca240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:41.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.618+0000 7fca3affd700 1 -- 172.21.15.92:0/3571492658 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5784+0+0 (secure 0 0 0) 0x7fca34083ab0 con 0x7fca440b2620 2023-10-06T10:41:41.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.618+0000 7fca3bfff700 1 --2- 172.21.15.92:0/3571492658 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fca24066f30 0x7fca240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:41.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.618+0000 7fca3bfff700 1 --2- 172.21.15.92:0/3571492658 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fca24066f30 0x7fca240693e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fca440ca530 tx=0x7fca2c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:41.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.622+0000 7fca3affd700 1 -- 172.21.15.92:0/3571492658 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca34050cd0 con 0x7fca440b2620 2023-10-06T10:41:41.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.805+0000 7fca43fff700 1 -- 172.21.15.92:0/3571492658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fca440cc600 con 0x7fca440b2620 2023-10-06T10:41:41.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.808+0000 7fca3affd700 1 -- 172.21.15.92:0/3571492658 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fca34017020 con 0x7fca440b2620 2023-10-06T10:41:41.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.810+0000 7fca43fff700 1 -- 172.21.15.92:0/3571492658 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fca24066f30 msgr2=0x7fca240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:41.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.810+0000 7fca43fff700 1 --2- 172.21.15.92:0/3571492658 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fca24066f30 0x7fca240693e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fca440ca530 tx=0x7fca2c009040 comp rx=0 tx=0).stop 2023-10-06T10:41:41.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.810+0000 7fca43fff700 1 -- 172.21.15.92:0/3571492658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca440b2620 msgr2=0x7fca44154080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:41.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.810+0000 7fca43fff700 1 --2- 172.21.15.92:0/3571492658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca440b2620 0x7fca44154080 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fca3400c5b0 tx=0x7fca34007f40 comp rx=0 tx=0).stop 2023-10-06T10:41:41.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.810+0000 7fca43fff700 1 -- 172.21.15.92:0/3571492658 shutdown_connections 2023-10-06T10:41:41.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.810+0000 7fca43fff700 1 --2- 172.21.15.92:0/3571492658 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fca24066f30 0x7fca240693e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:41.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.810+0000 7fca43fff700 1 --2- 172.21.15.92:0/3571492658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca440cd3c0 0x7fca44156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:41.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.810+0000 7fca43fff700 1 --2- 172.21.15.92:0/3571492658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca440b2620 0x7fca44154080 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:41.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.810+0000 7fca43fff700 1 --2- 172.21.15.92:0/3571492658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca440b29f0 0x7fca441545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:41.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.810+0000 7fca43fff700 1 -- 172.21.15.92:0/3571492658 >> 172.21.15.92:0/3571492658 conn(0x7fca44099520 msgr2=0x7fca44068c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:41.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.810+0000 7fca43fff700 1 -- 172.21.15.92:0/3571492658 shutdown_connections 2023-10-06T10:41:41.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:41.810+0000 7fca43fff700 1 -- 172.21.15.92:0/3571492658 wait complete. 2023-10-06T10:41:41.824 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T10:41:41.824 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:41:41.870 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:41:41.917 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T10:41:41.918 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T10:41:41.918 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:41:42.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.030+0000 7f0108172700 1 Processor -- start 2023-10-06T10:41:42.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.031+0000 7f0108172700 1 -- start start 2023-10-06T10:41:42.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.031+0000 7f0108172700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f01000ad210 0x7f01000cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:42.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.031+0000 7f0108172700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01000ad5e0 0x7f010015d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:42.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.032+0000 7f0108172700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01000cecf0 con 0x7f010015d940 2023-10-06T10:41:42.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.032+0000 7f0108172700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f010015dde0 con 0x7f01000ad210 2023-10-06T10:41:42.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.032+0000 7f0108172700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01000cefd0 con 0x7f01000ad5e0 2023-10-06T10:41:42.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.032+0000 7f0105f0e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f01000ad210 0x7f01000cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:42.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.032+0000 7f0105f0e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f01000ad210 0x7f01000cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53120/0 (socket says 172.21.15.92:53120) 2023-10-06T10:41:42.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.032+0000 7f0105f0e700 1 -- 172.21.15.92:0/2224936036 learned_addr learned my addr 172.21.15.92:0/2224936036 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:42.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.032+0000 7f010570d700 1 --2- 172.21.15.92:0/2224936036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01000ad5e0 0x7f010015d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:42.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.032+0000 7f0105f0e700 1 -- 172.21.15.92:0/2224936036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01000ad5e0 msgr2=0x7f010015d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:42.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.032+0000 7f0105f0e700 1 --2- 172.21.15.92:0/2224936036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01000ad5e0 0x7f010015d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:42.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.032+0000 7f0105f0e700 1 -- 172.21.15.92:0/2224936036 >> v1:172.21.15.124:6789/0 conn(0x7f010015d940 legacy=0x7f01000686c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:42.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.032+0000 7f0105f0e700 1 -- 172.21.15.92:0/2224936036 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01000cca90 con 0x7f01000ad210 2023-10-06T10:41:42.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.032+0000 7f0105f0e700 1 --2- 172.21.15.92:0/2224936036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01000ad210 0x7f01000cd290 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009990 tx=0x7f00fc009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7160cf7d54c49 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:42.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.032+0000 7f0104f0c700 1 -- 172.21.15.92:0/2224936036 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc005c10 con 0x7f01000ad210 2023-10-06T10:41:42.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.032+0000 7f0104f0c700 1 -- 172.21.15.92:0/2224936036 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00fc005d70 con 0x7f01000ad210 2023-10-06T10:41:42.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.032+0000 7f0104f0c700 1 -- 172.21.15.92:0/2224936036 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc00b880 con 0x7f01000ad210 2023-10-06T10:41:42.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.033+0000 7f0108172700 1 -- 172.21.15.92:0/2224936036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01000ad210 msgr2=0x7f01000cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:42.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.033+0000 7f0108172700 1 --2- 172.21.15.92:0/2224936036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01000ad210 0x7f01000cd290 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009990 tx=0x7f00fc009220 comp rx=0 tx=0).stop 2023-10-06T10:41:42.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.033+0000 7f0108172700 1 -- 172.21.15.92:0/2224936036 shutdown_connections 2023-10-06T10:41:42.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.033+0000 7f0108172700 1 --2- 172.21.15.92:0/2224936036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01000ad210 0x7f01000cd290 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:42.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.033+0000 7f0108172700 1 --2- 172.21.15.92:0/2224936036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01000ad5e0 0x7f010015d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:42.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.033+0000 7f0108172700 1 -- 172.21.15.92:0/2224936036 >> 172.21.15.92:0/2224936036 conn(0x7f0100067d90 msgr2=0x7f01000ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:42.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.033+0000 7f0108172700 1 -- 172.21.15.92:0/2224936036 shutdown_connections 2023-10-06T10:41:42.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.033+0000 7f0108172700 1 -- 172.21.15.92:0/2224936036 wait complete. 2023-10-06T10:41:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.034+0000 7f0108172700 1 Processor -- start 2023-10-06T10:41:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.034+0000 7f0108172700 1 -- start start 2023-10-06T10:41:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.035+0000 7f0108172700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01000ad210 0x7f01000af480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.035+0000 7f0108172700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01000ad5e0 0x7f010016aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.035+0000 7f0108172700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f010015d940 0x7f010006c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.035+0000 7f0108172700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010015a350 con 0x7f010015d940 2023-10-06T10:41:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.035+0000 7f0108172700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01000c4720 con 0x7f01000ad210 2023-10-06T10:41:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.035+0000 7f0108172700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010016b3f0 con 0x7f01000ad5e0 2023-10-06T10:41:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.035+0000 7f010670f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f010015d940 0x7f010006c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.035+0000 7f010670f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f010015d940 0x7f010006c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53128/0 (socket says 172.21.15.92:53128) 2023-10-06T10:41:42.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.035+0000 7f010670f700 1 -- 172.21.15.92:0/854503746 learned_addr learned my addr 172.21.15.92:0/854503746 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:42.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.035+0000 7f0105f0e700 1 --2- 172.21.15.92:0/854503746 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01000ad210 0x7f01000af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:42.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.035+0000 7f010570d700 1 --2- 172.21.15.92:0/854503746 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01000ad5e0 0x7f010016aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:42.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.035+0000 7f010670f700 1 -- 172.21.15.92:0/854503746 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01000ad5e0 msgr2=0x7f010016aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:42.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.035+0000 7f010670f700 1 --2- 172.21.15.92:0/854503746 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01000ad5e0 0x7f010016aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:42.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.035+0000 7f010670f700 1 -- 172.21.15.92:0/854503746 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01000ad210 msgr2=0x7f01000af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:42.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.035+0000 7f010670f700 1 --2- 172.21.15.92:0/854503746 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01000ad210 0x7f01000af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:42.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.035+0000 7f010670f700 1 -- 172.21.15.92:0/854503746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc009480 con 0x7f010015d940 2023-10-06T10:41:42.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.035+0000 7f0105f0e700 1 --2- 172.21.15.92:0/854503746 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01000ad210 0x7f01000af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:42.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.035+0000 7f010670f700 1 --2- 172.21.15.92:0/854503746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f010015d940 0x7f010006c620 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f00f800c870 tx=0x7f00f8001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:42.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.035+0000 7f00f5ffb700 1 -- 172.21.15.92:0/854503746 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8012630 con 0x7f010015d940 2023-10-06T10:41:42.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.035+0000 7f00f5ffb700 1 -- 172.21.15.92:0/854503746 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f80127b0 con 0x7f010015d940 2023-10-06T10:41:42.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.035+0000 7f0108172700 1 -- 172.21.15.92:0/854503746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01000c1f30 con 0x7f010015d940 2023-10-06T10:41:42.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.035+0000 7f00f5ffb700 1 -- 172.21.15.92:0/854503746 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8012b70 con 0x7f010015d940 2023-10-06T10:41:42.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.036+0000 7f0108172700 1 -- 172.21.15.92:0/854503746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01000c3df0 con 0x7f010015d940 2023-10-06T10:41:42.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.037+0000 7f0108172700 1 -- 172.21.15.92:0/854503746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01000c48b0 con 0x7f010015d940 2023-10-06T10:41:42.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.037+0000 7f00f5ffb700 1 -- 172.21.15.92:0/854503746 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f00f8003030 con 0x7f010015d940 2023-10-06T10:41:42.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.037+0000 7f00f5ffb700 1 --2- 172.21.15.92:0/854503746 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f00e4066f30 0x7f00e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:42.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.038+0000 7f00f5ffb700 1 -- 172.21.15.92:0/854503746 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f00f8084500 con 0x7f010015d940 2023-10-06T10:41:42.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.038+0000 7f0105f0e700 1 --2- 172.21.15.92:0/854503746 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f00e4066f30 0x7f00e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:42.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.038+0000 7f0105f0e700 1 --2- 172.21.15.92:0/854503746 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f00e4066f30 0x7f00e40693e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f01000b0210 tx=0x7f00fc016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:42.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.042+0000 7f00f5ffb700 1 -- 172.21.15.92:0/854503746 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00f8051720 con 0x7f010015d940 2023-10-06T10:41:42.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.267+0000 7f0108172700 1 -- 172.21.15.92:0/854503746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0100064a90 con 0x7f010015d940 2023-10-06T10:41:42.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.267+0000 7f00f5ffb700 1 -- 172.21.15.92:0/854503746 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f00f8054d40 con 0x7f010015d940 2023-10-06T10:41:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:42.270 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":44,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3668157650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3668157650},{"type":"v1","addr":"172.21.15.124:6834","nonce":3668157650}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5956,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/636108550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":636108550},{"type":"v1","addr":"172.21.15.124:6835","nonce":636108550}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6408,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/634164946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":634164946},{"type":"v1","addr":"172.21.15.92:6835","nonce":634164946}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6414,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/3734325571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3734325571},{"type":"v1","addr":"172.21.15.92:6837","nonce":3734325571}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[]} 2023-10-06T10:41:42.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.269+0000 7f0108172700 1 -- 172.21.15.92:0/854503746 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f00e4066f30 msgr2=0x7f00e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:42.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.269+0000 7f0108172700 1 --2- 172.21.15.92:0/854503746 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f00e4066f30 0x7f00e40693e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f01000b0210 tx=0x7f00fc016000 comp rx=0 tx=0).stop 2023-10-06T10:41:42.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.269+0000 7f0108172700 1 -- 172.21.15.92:0/854503746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f010015d940 msgr2=0x7f010006c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:42.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.269+0000 7f0108172700 1 --2- 172.21.15.92:0/854503746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f010015d940 0x7f010006c620 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f00f800c870 tx=0x7f00f8001670 comp rx=0 tx=0).stop 2023-10-06T10:41:42.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.269+0000 7f0108172700 1 -- 172.21.15.92:0/854503746 shutdown_connections 2023-10-06T10:41:42.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.269+0000 7f0108172700 1 --2- 172.21.15.92:0/854503746 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f00e4066f30 0x7f00e40693e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:42.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.269+0000 7f0108172700 1 --2- 172.21.15.92:0/854503746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f010015d940 0x7f010006c620 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:42.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.269+0000 7f0108172700 1 --2- 172.21.15.92:0/854503746 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01000ad210 0x7f01000af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:42.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.269+0000 7f0108172700 1 --2- 172.21.15.92:0/854503746 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01000ad5e0 0x7f010016aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:42.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.269+0000 7f0108172700 1 -- 172.21.15.92:0/854503746 >> 172.21.15.92:0/854503746 conn(0x7f0100067d90 msgr2=0x7f0100069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:42.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.270+0000 7f0108172700 1 -- 172.21.15.92:0/854503746 shutdown_connections 2023-10-06T10:41:42.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.270+0000 7f0108172700 1 -- 172.21.15.92:0/854503746 wait complete. 2023-10-06T10:41:42.273 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 44 2023-10-06T10:41:42.283 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T10:41:42.284 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T10:41:42.284 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T10:41:42.284 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:41:42.284 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T10:41:42.284 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T10:41:42.284 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T10:41:42.285 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:41:42.285 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T10:41:42.285 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T10:41:42.285 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T10:41:42.285 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:41:42.285 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T10:41:42.285 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T10:41:42.286 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T10:41:42.286 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:41:42.293 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T10:41:42.293 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T10:41:42.294 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T10:41:42.294 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T10:41:42.296 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T10:41:42.296 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T10:41:42.297 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T10:41:42.297 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T10:41:42.335 INFO:tasks.ceph.mds.a:Started 2023-10-06T10:41:42.336 DEBUG:teuthology.parallel:result is None 2023-10-06T10:41:42.337 INFO:tasks.ceph.mds.c:Started 2023-10-06T10:41:42.337 DEBUG:teuthology.parallel:result is None 2023-10-06T10:41:42.339 INFO:tasks.ceph.mds.b:Started 2023-10-06T10:41:42.340 DEBUG:teuthology.parallel:result is None 2023-10-06T10:41:42.341 INFO:tasks.ceph.mds.d:Started 2023-10-06T10:41:42.341 DEBUG:teuthology.parallel:result is None 2023-10-06T10:41:42.342 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T10:41:42.409 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T10:41:42.409 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T10:41:42.420 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-06T10:41:42.421 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-06T10:41:42.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.462+0000 7f6bb504e700 1 Processor -- start 2023-10-06T10:41:42.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.462+0000 7f6bb504e700 1 -- start start 2023-10-06T10:41:42.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.463+0000 7f6bb504e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6bb0159440 0x7f6bb00b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:42.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.463+0000 7f6bb504e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb00b2100 0x7f6bb00b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:42.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.463+0000 7f6bb504e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb00cb7a0 con 0x7f6bb00b35d0 2023-10-06T10:41:42.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.463+0000 7f6bb504e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bb00b0860 con 0x7f6bb0159440 2023-10-06T10:41:42.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.463+0000 7f6bb504e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb00cb150 con 0x7f6bb00b2100 2023-10-06T10:41:42.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.463+0000 7f6baffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6bb0159440 0x7f6bb00b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:42.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.463+0000 7f6baffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6bb0159440 0x7f6bb00b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53180/0 (socket says 172.21.15.92:53180) 2023-10-06T10:41:42.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.463+0000 7f6baffff700 1 -- 172.21.15.92:0/4148989361 learned_addr learned my addr 172.21.15.92:0/4148989361 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:42.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.463+0000 7f6bb484d700 1 --2- 172.21.15.92:0/4148989361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb00b2100 0x7f6bb00b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:42.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.463+0000 7f6baffff700 1 -- 172.21.15.92:0/4148989361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb00b2100 msgr2=0x7f6bb00b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:42.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.463+0000 7f6baffff700 1 --2- 172.21.15.92:0/4148989361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb00b2100 0x7f6bb00b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:42.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.463+0000 7f6baffff700 1 -- 172.21.15.92:0/4148989361 >> v1:172.21.15.124:6789/0 conn(0x7f6bb00b35d0 legacy=0x7f6bb00cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:42.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.463+0000 7f6baffff700 1 -- 172.21.15.92:0/4148989361 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb00cb430 con 0x7f6bb0159440 2023-10-06T10:41:42.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.464+0000 7f6baffff700 1 --2- 172.21.15.92:0/4148989361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bb0159440 0x7f6bb00b3090 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00099b0 tx=0x7f6ba0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3793f276c6143cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:42.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.464+0000 7f6baeffd700 1 -- 172.21.15.92:0/4148989361 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba000b820 con 0x7f6bb0159440 2023-10-06T10:41:42.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.464+0000 7f6baeffd700 1 -- 172.21.15.92:0/4148989361 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba000b980 con 0x7f6bb0159440 2023-10-06T10:41:42.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.464+0000 7f6baeffd700 1 -- 172.21.15.92:0/4148989361 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba000bd20 con 0x7f6bb0159440 2023-10-06T10:41:42.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.464+0000 7f6bb504e700 1 -- 172.21.15.92:0/4148989361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bb0159440 msgr2=0x7f6bb00b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:42.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.464+0000 7f6bb504e700 1 --2- 172.21.15.92:0/4148989361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bb0159440 0x7f6bb00b3090 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00099b0 tx=0x7f6ba0009250 comp rx=0 tx=0).stop 2023-10-06T10:41:42.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.464+0000 7f6bb504e700 1 -- 172.21.15.92:0/4148989361 shutdown_connections 2023-10-06T10:41:42.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.464+0000 7f6bb504e700 1 --2- 172.21.15.92:0/4148989361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb00b2100 0x7f6bb00b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:42.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.464+0000 7f6bb504e700 1 --2- 172.21.15.92:0/4148989361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bb0159440 0x7f6bb00b3090 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:42.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.464+0000 7f6bb504e700 1 -- 172.21.15.92:0/4148989361 >> 172.21.15.92:0/4148989361 conn(0x7f6bb006ece0 msgr2=0x7f6bb006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:42.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.464+0000 7f6bb504e700 1 -- 172.21.15.92:0/4148989361 shutdown_connections 2023-10-06T10:41:42.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.464+0000 7f6bb504e700 1 -- 172.21.15.92:0/4148989361 wait complete. 2023-10-06T10:41:42.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.465+0000 7f6bb504e700 1 Processor -- start 2023-10-06T10:41:42.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.466+0000 7f6bb504e700 1 -- start start 2023-10-06T10:41:42.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.466+0000 7f6bb504e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb00b2100 0x7f6bb0153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:42.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.466+0000 7f6bb504e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bb00b35d0 0x7f6bb0150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:42.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.466+0000 7f6bb504e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bb0159440 0x7f6bb00c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:42.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.466+0000 7f6bb504e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb00c8b30 con 0x7f6bb0159440 2023-10-06T10:41:42.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.466+0000 7f6bb504e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb00c8e10 con 0x7f6bb00b35d0 2023-10-06T10:41:42.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.466+0000 7f6bb504e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb00c90f0 con 0x7f6bb00b2100 2023-10-06T10:41:42.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.466+0000 7f6bb484d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bb0159440 0x7f6bb00c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:42.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.466+0000 7f6baffff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb00b2100 0x7f6bb0153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:42.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.466+0000 7f6bb484d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bb0159440 0x7f6bb00c81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53188/0 (socket says 172.21.15.92:53188) 2023-10-06T10:41:42.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.466+0000 7f6baffff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb00b2100 0x7f6bb0153bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:56800/0 (socket says 172.21.15.92:56800) 2023-10-06T10:41:42.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.466+0000 7f6bb484d700 1 -- 172.21.15.92:0/1364412464 learned_addr learned my addr 172.21.15.92:0/1364412464 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:42.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.466+0000 7f6baf7fe700 1 --2- 172.21.15.92:0/1364412464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bb00b35d0 0x7f6bb0150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:42.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.467+0000 7f6bb484d700 1 -- 172.21.15.92:0/1364412464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb00b2100 msgr2=0x7f6bb0153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:42.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.467+0000 7f6bb484d700 1 --2- 172.21.15.92:0/1364412464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb00b2100 0x7f6bb0153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:42.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.467+0000 7f6bb484d700 1 -- 172.21.15.92:0/1364412464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bb00b35d0 msgr2=0x7f6bb0150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:42.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.467+0000 7f6bb484d700 1 --2- 172.21.15.92:0/1364412464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bb00b35d0 0x7f6bb0150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:42.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.467+0000 7f6bb484d700 1 -- 172.21.15.92:0/1364412464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba00094f0 con 0x7f6bb0159440 2023-10-06T10:41:42.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.467+0000 7f6bb484d700 1 --2- 172.21.15.92:0/1364412464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bb0159440 0x7f6bb00c81e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009fa0 tx=0x7f6ba8009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:42.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.467+0000 7f6b97fff700 1 -- 172.21.15.92:0/1364412464 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba800ebc0 con 0x7f6bb0159440 2023-10-06T10:41:42.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.467+0000 7f6b97fff700 1 -- 172.21.15.92:0/1364412464 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba800ed20 con 0x7f6bb0159440 2023-10-06T10:41:42.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.467+0000 7f6bb504e700 1 -- 172.21.15.92:0/1364412464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb00c93d0 con 0x7f6bb0159440 2023-10-06T10:41:42.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.467+0000 7f6b97fff700 1 -- 172.21.15.92:0/1364412464 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8014670 con 0x7f6bb0159440 2023-10-06T10:41:42.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.467+0000 7f6bb504e700 1 -- 172.21.15.92:0/1364412464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb00c8720 con 0x7f6bb0159440 2023-10-06T10:41:42.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.468+0000 7f6bb504e700 1 -- 172.21.15.92:0/1364412464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb00b3070 con 0x7f6bb0159440 2023-10-06T10:41:42.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.468+0000 7f6b97fff700 1 -- 172.21.15.92:0/1364412464 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ba80149e0 con 0x7f6bb0159440 2023-10-06T10:41:42.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.469+0000 7f6b97fff700 1 --2- 172.21.15.92:0/1364412464 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6b9c066f30 0x7f6b9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:42.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.469+0000 7f6b97fff700 1 -- 172.21.15.92:0/1364412464 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f6ba8084c20 con 0x7f6bb0159440 2023-10-06T10:41:42.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.469+0000 7f6baffff700 1 --2- 172.21.15.92:0/1364412464 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6b9c066f30 0x7f6b9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:42.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.469+0000 7f6baffff700 1 --2- 172.21.15.92:0/1364412464 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6b9c066f30 0x7f6b9c0693e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00099b0 tx=0x7f6ba0016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:42.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.473+0000 7f6b97fff700 1 -- 172.21.15.92:0/1364412464 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ba8051e40 con 0x7f6bb0159440 2023-10-06T10:41:42.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:42.674+0000 7f6bb504e700 1 -- 172.21.15.92:0/1364412464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6bb005f230 con 0x7f6bb0159440 2023-10-06T10:41:43.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.496+0000 7f6b97fff700 1 -- 172.21.15.92:0/1364412464 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v60) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6ba8055460 con 0x7f6bb0159440 2023-10-06T10:41:43.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.498+0000 7f6bb504e700 1 -- 172.21.15.92:0/1364412464 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6b9c066f30 msgr2=0x7f6b9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:43.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.498+0000 7f6bb504e700 1 --2- 172.21.15.92:0/1364412464 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6b9c066f30 0x7f6b9c0693e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00099b0 tx=0x7f6ba0016000 comp rx=0 tx=0).stop 2023-10-06T10:41:43.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.498+0000 7f6bb504e700 1 -- 172.21.15.92:0/1364412464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bb0159440 msgr2=0x7f6bb00c81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:43.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.498+0000 7f6bb504e700 1 --2- 172.21.15.92:0/1364412464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bb0159440 0x7f6bb00c81e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009fa0 tx=0x7f6ba8009900 comp rx=0 tx=0).stop 2023-10-06T10:41:43.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.499+0000 7f6bb504e700 1 -- 172.21.15.92:0/1364412464 shutdown_connections 2023-10-06T10:41:43.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.499+0000 7f6bb504e700 1 --2- 172.21.15.92:0/1364412464 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6b9c066f30 0x7f6b9c0693e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:43.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.499+0000 7f6bb504e700 1 --2- 172.21.15.92:0/1364412464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bb0159440 0x7f6bb00c81e0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:43.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.499+0000 7f6bb504e700 1 --2- 172.21.15.92:0/1364412464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb00b2100 0x7f6bb0153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:43.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.499+0000 7f6bb504e700 1 --2- 172.21.15.92:0/1364412464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bb00b35d0 0x7f6bb0150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:43.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.499+0000 7f6bb504e700 1 -- 172.21.15.92:0/1364412464 >> 172.21.15.92:0/1364412464 conn(0x7f6bb006ece0 msgr2=0x7f6bb0152630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:43.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.499+0000 7f6bb504e700 1 -- 172.21.15.92:0/1364412464 shutdown_connections 2023-10-06T10:41:43.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.499+0000 7f6bb504e700 1 -- 172.21.15.92:0/1364412464 wait complete. 2023-10-06T10:41:43.502 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-10-06T10:41:43.513 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T10:41:43.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.627+0000 7ff9e6c2f700 1 Processor -- start 2023-10-06T10:41:43.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.627+0000 7ff9e6c2f700 1 -- start start 2023-10-06T10:41:43.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9e6c2f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9e0069c30 0x7ff9e00c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:43.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9e6c2f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e00c7330 con 0x7ff9e00ac000 2023-10-06T10:41:43.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9e6c2f700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e00c6720 con 0x7ff9e00adf20 2023-10-06T10:41:43.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9e6c2f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e0156110 con 0x7ff9e0069c30 2023-10-06T10:41:43.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9d7fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff9e00ac000 0x7ff9e0069520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33152/0 (socket says 172.21.15.92:33152) 2023-10-06T10:41:43.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9d7fff700 1 -- 172.21.15.92:0/4265720258 learned_addr learned my addr 172.21.15.92:0/4265720258 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:43.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9d77fe700 1 -- 172.21.15.92:0/4265720258 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2700931420 0 0) 0x7ff9e00c7330 con 0x7ff9e00ac000 2023-10-06T10:41:43.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9d77fe700 1 -- 172.21.15.92:0/4265720258 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9c8003570 con 0x7ff9e00ac000 2023-10-06T10:41:43.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9e51cc700 1 --2- 172.21.15.92:0/4265720258 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9e0069c30 0x7ff9e00c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:43.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9d77fe700 1 -- 172.21.15.92:0/4265720258 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2506888841 0 0) 0x7ff9c8003570 con 0x7ff9e00ac000 2023-10-06T10:41:43.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9d77fe700 1 -- 172.21.15.92:0/4265720258 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9e0069c30 msgr2=0x7ff9e00c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:43.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9d77fe700 1 --2- 172.21.15.92:0/4265720258 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9e0069c30 0x7ff9e00c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:43.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9d77fe700 1 -- 172.21.15.92:0/4265720258 >> v1:172.21.15.124:6789/0 conn(0x7ff9e00adf20 legacy=0x7ff9e006e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:43.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9d77fe700 1 -- 172.21.15.92:0/4265720258 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e00c6a00 con 0x7ff9e00ac000 2023-10-06T10:41:43.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9d77fe700 1 -- 172.21.15.92:0/4265720258 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7ff9d0001470 con 0x7ff9e00ac000 2023-10-06T10:41:43.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9d77fe700 1 -- 172.21.15.92:0/4265720258 >> v1:172.21.15.92:6789/0 conn(0x7ff9e00ac000 legacy=0x7ff9e0069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:43.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9d77fe700 1 --2- 172.21.15.92:0/4265720258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c8003a90 0x7ff9c8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:43.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9d77fe700 1 --2- 172.21.15.92:0/4265720258 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9e0069c30 0x7ff9c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:43.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9d77fe700 1 --2- 172.21.15.92:0/4265720258 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9c8004d50 0x7ff9c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:43.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9d77fe700 1 -- 172.21.15.92:0/4265720258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e0156110 con 0x7ff9c8003a90 2023-10-06T10:41:43.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9d77fe700 1 -- 172.21.15.92:0/4265720258 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c800cc80 con 0x7ff9c8004d50 2023-10-06T10:41:43.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9d77fe700 1 -- 172.21.15.92:0/4265720258 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c800ce40 con 0x7ff9e0069c30 2023-10-06T10:41:43.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9e49cb700 1 --2- 172.21.15.92:0/4265720258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c8003a90 0x7ff9c8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:43.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.628+0000 7ff9d7fff700 1 --2- 172.21.15.92:0/4265720258 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9e0069c30 0x7ff9c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:43.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.629+0000 7ff9e51cc700 1 --2- 172.21.15.92:0/4265720258 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9c8004d50 0x7ff9c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:43.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.629+0000 7ff9e49cb700 1 -- 172.21.15.92:0/4265720258 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9e0069c30 msgr2=0x7ff9c8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:43.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.629+0000 7ff9e49cb700 1 --2- 172.21.15.92:0/4265720258 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9e0069c30 0x7ff9c8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:43.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.629+0000 7ff9e49cb700 1 -- 172.21.15.92:0/4265720258 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9c8004d50 msgr2=0x7ff9c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:43.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.629+0000 7ff9e49cb700 1 --2- 172.21.15.92:0/4265720258 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9c8004d50 0x7ff9c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:43.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.629+0000 7ff9e49cb700 1 -- 172.21.15.92:0/4265720258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c8003570 con 0x7ff9c8003a90 2023-10-06T10:41:43.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.629+0000 7ff9e49cb700 1 --2- 172.21.15.92:0/4265720258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c8003a90 0x7ff9c8003ea0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc001aa0 tx=0x7ff9dc001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602619da4a3fb1e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:43.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.629+0000 7ff9d77fe700 1 -- 172.21.15.92:0/4265720258 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc00c8b0 con 0x7ff9c8003a90 2023-10-06T10:41:43.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.629+0000 7ff9d77fe700 1 -- 172.21.15.92:0/4265720258 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9dc00ca10 con 0x7ff9c8003a90 2023-10-06T10:41:43.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.629+0000 7ff9d77fe700 1 -- 172.21.15.92:0/4265720258 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc00cdb0 con 0x7ff9c8003a90 2023-10-06T10:41:43.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.629+0000 7ff9e6c2f700 1 -- 172.21.15.92:0/4265720258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c8003a90 msgr2=0x7ff9c8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:43.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.629+0000 7ff9e6c2f700 1 --2- 172.21.15.92:0/4265720258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c8003a90 0x7ff9c8003ea0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc001aa0 tx=0x7ff9dc001670 comp rx=0 tx=0).stop 2023-10-06T10:41:43.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.629+0000 7ff9e51cc700 1 -- 172.21.15.92:0/4265720258 reap_dead start 2023-10-06T10:41:43.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.630+0000 7ff9e6c2f700 1 -- 172.21.15.92:0/4265720258 shutdown_connections 2023-10-06T10:41:43.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.630+0000 7ff9e6c2f700 1 -- 172.21.15.92:0/4265720258 >> 172.21.15.92:0/4265720258 conn(0x7ff9e0068080 msgr2=0x7ff9e00cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:43.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.630+0000 7ff9e6c2f700 1 -- 172.21.15.92:0/4265720258 shutdown_connections 2023-10-06T10:41:43.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.630+0000 7ff9e6c2f700 1 -- 172.21.15.92:0/4265720258 wait complete. 2023-10-06T10:41:43.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.631+0000 7ff9e6c2f700 1 Processor -- start 2023-10-06T10:41:43.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.631+0000 7ff9e6c2f700 1 -- start start 2023-10-06T10:41:43.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9e6c2f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9c8003a90 0x7ff9e0155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:43.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9e6c2f700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9e00c1de0 0x7ff9e0159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:43.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9e6c2f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e0157040 0x7ff9e01574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:43.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9e6c2f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e0155c70 con 0x7ff9e0157040 2023-10-06T10:41:43.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9e6c2f700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e015a4d0 con 0x7ff9e00c1de0 2023-10-06T10:41:43.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9e6c2f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e01579f0 con 0x7ff9c8003a90 2023-10-06T10:41:43.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9e51cc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e0157040 0x7ff9e01574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:43.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9e51cc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e0157040 0x7ff9e01574b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57024/0 (socket says 172.21.15.92:57024) 2023-10-06T10:41:43.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9e51cc700 1 -- 172.21.15.92:0/1825973370 learned_addr learned my addr 172.21.15.92:0/1825973370 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:43.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9e49cb700 1 --2- 172.21.15.92:0/1825973370 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9c8003a90 0x7ff9e0155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:43.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9d7fff700 1 --2- 172.21.15.92:0/1825973370 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9e00c1de0 0x7ff9e0159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:43.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9e51cc700 1 -- 172.21.15.92:0/1825973370 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9c8003a90 msgr2=0x7ff9e0155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:43.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9e51cc700 1 --2- 172.21.15.92:0/1825973370 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9c8003a90 0x7ff9e0155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:43.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9e51cc700 1 -- 172.21.15.92:0/1825973370 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9e00c1de0 msgr2=0x7ff9e0159050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:43.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9e51cc700 1 --2- 172.21.15.92:0/1825973370 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9e00c1de0 0x7ff9e0159050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:43.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9e51cc700 1 -- 172.21.15.92:0/1825973370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e01598c0 con 0x7ff9e0157040 2023-10-06T10:41:43.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9e49cb700 1 --2- 172.21.15.92:0/1825973370 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9c8003a90 0x7ff9e0155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:43.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9e51cc700 1 --2- 172.21.15.92:0/1825973370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e0157040 0x7ff9e01574b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00abdf0 tx=0x7ff9d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:43.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9d4ff9700 1 -- 172.21.15.92:0/1825973370 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d800f8e0 con 0x7ff9e0157040 2023-10-06T10:41:43.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9d4ff9700 1 -- 172.21.15.92:0/1825973370 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d800fa60 con 0x7ff9e0157040 2023-10-06T10:41:43.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9e6c2f700 1 -- 172.21.15.92:0/1825973370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e0159ba0 con 0x7ff9e0157040 2023-10-06T10:41:43.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9d4ff9700 1 -- 172.21.15.92:0/1825973370 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d800fe20 con 0x7ff9e0157040 2023-10-06T10:41:43.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.632+0000 7ff9e6c2f700 1 -- 172.21.15.92:0/1825973370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e015a160 con 0x7ff9e0157040 2023-10-06T10:41:43.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.633+0000 7ff9e6c2f700 1 -- 172.21.15.92:0/1825973370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e0064a90 con 0x7ff9e0157040 2023-10-06T10:41:43.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.634+0000 7ff9d4ff9700 1 -- 172.21.15.92:0/1825973370 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff9d8014650 con 0x7ff9e0157040 2023-10-06T10:41:43.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.634+0000 7ff9d4ff9700 1 --2- 172.21.15.92:0/1825973370 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff9cc066f30 0x7ff9cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:43.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.634+0000 7ff9d4ff9700 1 -- 172.21.15.92:0/1825973370 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5526+0+0 (secure 0 0 0) 0x7ff9d8084950 con 0x7ff9e0157040 2023-10-06T10:41:43.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.634+0000 7ff9e49cb700 1 --2- 172.21.15.92:0/1825973370 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff9cc066f30 0x7ff9cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:43.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.635+0000 7ff9e49cb700 1 --2- 172.21.15.92:0/1825973370 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff9cc066f30 0x7ff9cc0693e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc001a70 tx=0x7ff9dc017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:43.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.639+0000 7ff9d4ff9700 1 -- 172.21.15.92:0/1825973370 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9d8051cf0 con 0x7ff9e0157040 2023-10-06T10:41:43.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.893+0000 7ff9e6c2f700 1 -- 172.21.15.92:0/1825973370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff9e00c6ce0 con 0x7ff9e0157040 2023-10-06T10:41:43.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.894+0000 7ff9d4ff9700 1 -- 172.21.15.92:0/1825973370 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff9d8022090 con 0x7ff9e0157040 2023-10-06T10:41:43.896 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:43.896 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:41:43.896 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-10-06T10:41:43.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:41:43.896 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-10-06T10:41:43.897 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lc28gCBAAVrjhXIawHuZEox0PE3/Rsg==", 2023-10-06T10:41:43.897 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:41:43.897 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:41:43.897 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:41:43.897 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:41:43.897 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:41:43.897 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:43.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:41:43.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:41:43.897 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-10-06T10:41:43.898 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lQpdOExAAm44s0n4Jekhy8AJk/o6Waw==", 2023-10-06T10:41:43.898 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:41:43.898 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:41:43.898 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:41:43.898 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:41:43.898 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:41:43.898 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:43.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:41:43.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:41:43.898 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-10-06T10:41:43.899 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lrCrsDhAAvp++wvvnSpZRGzvRTrGWEw==", 2023-10-06T10:41:43.899 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:41:43.899 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:41:43.899 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:41:43.899 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:41:43.899 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:41:43.899 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:43.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:41:43.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:41:43.899 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-10-06T10:41:43.900 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9l/DgWGhAAy3JMA5ntn/UaRkuTCyPdkg==", 2023-10-06T10:41:43.900 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:41:43.900 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:41:43.900 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:41:43.900 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:41:43.900 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:41:43.900 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:43.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:41:43.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:41:43.900 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-10-06T10:41:43.901 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBo4x9lDEIFMBAAc3DY1fYpY2UiY7MX7qSP5A==", 2023-10-06T10:41:43.901 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:41:43.901 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:41:43.901 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:41:43.901 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:41:43.901 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:43.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:41:43.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:41:43.901 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-10-06T10:41:43.901 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBr4x9lKzX8LRAApryLtFGkK6urkE8W70l5Cg==", 2023-10-06T10:41:43.902 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:41:43.902 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:41:43.902 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:41:43.902 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:41:43.902 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:43.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:41:43.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:41:43.902 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-10-06T10:41:43.902 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBu4x9lCMZpLhAAEBPa40Wnb820p6j+lv+PsA==", 2023-10-06T10:41:43.902 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:41:43.903 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:41:43.903 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:41:43.903 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:41:43.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:43.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:41:43.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:41:43.903 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-10-06T10:41:43.903 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBx4x9l4AHrLRAAFn6SnNkHnOiVXmmiU8qGyA==", 2023-10-06T10:41:43.903 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:41:43.903 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:41:43.904 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:41:43.904 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:41:43.904 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:43.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:41:43.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:41:43.904 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-10-06T10:41:43.904 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB34x9l7V7WMBAAvs5kvBP5n0Th478l/idjeQ==", 2023-10-06T10:41:43.904 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:41:43.904 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:41:43.904 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:41:43.905 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:41:43.905 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:43.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:41:43.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:41:43.905 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-10-06T10:41:43.905 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB64x9lfHIlLhAADCuqt+O/zPEETgoj//hwVg==", 2023-10-06T10:41:43.905 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:41:43.905 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:41:43.905 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:41:43.905 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:41:43.905 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:43.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:41:43.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:41:43.906 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-10-06T10:41:43.906 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB94x9lUmU7LhAAvxmtGmJxrI8TlOOD7taHOQ==", 2023-10-06T10:41:43.906 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:41:43.906 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:41:43.906 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:41:43.906 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:41:43.906 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:43.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:41:43.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:41:43.907 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-10-06T10:41:43.907 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQCA4x9lJE3nLBAAnREk9yNK+FhX4fnG+Za5Ug==", 2023-10-06T10:41:43.907 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:41:43.907 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:41:43.907 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:41:43.907 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:41:43.907 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:43.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:41:43.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:41:43.908 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-10-06T10:41:43.908 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lMePcHRAAMyftXWJWk7rwmWFkq18Rqg==", 2023-10-06T10:41:43.908 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:41:43.908 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:41:43.908 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-10-06T10:41:43.908 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T10:41:43.908 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:43.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:41:43.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:41:43.908 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-10-06T10:41:43.909 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lp1s0JhAAKvZ23MYjObxOK83q3UqJBQ==", 2023-10-06T10:41:43.909 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:41:43.909 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:41:43.909 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T10:41:43.909 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T10:41:43.909 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T10:41:43.909 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:43.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:41:43.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:41:43.909 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-10-06T10:41:43.910 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lc6o9IxAAaOY7S5rYn+OdeMNWK1UYMA==", 2023-10-06T10:41:43.910 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:41:43.910 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:41:43.910 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T10:41:43.910 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T10:41:43.910 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T10:41:43.910 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:43.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:41:43.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:41:43.911 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-10-06T10:41:43.911 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lwoyOKxAA811lehuQQ8MQQQzuTFAApg==", 2023-10-06T10:41:43.911 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:41:43.911 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:41:43.911 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T10:41:43.911 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T10:41:43.911 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T10:41:43.911 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:43.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:41:43.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:41:43.912 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-10-06T10:41:43.912 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBk4x9lONCoJhAAepF/sou/5M2GBvvfNAs5wQ==", 2023-10-06T10:41:43.912 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:41:43.912 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T10:41:43.912 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:41:43.912 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:41:43.912 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:41:43.912 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:43.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:41:43.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:41:43.913 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-10-06T10:41:43.913 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9ll8wgBRAAQUrHERQ3Igv1tosa+wryIg==", 2023-10-06T10:41:43.913 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:41:43.913 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T10:41:43.913 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T10:41:43.913 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:41:43.913 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:43.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:41:43.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:41:43.914 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-10-06T10:41:43.914 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBk4x9lgYa/OhAAyoI717Ga+Wv72srZ4FK+lQ==", 2023-10-06T10:41:43.914 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:41:43.914 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T10:41:43.914 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T10:41:43.914 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:41:43.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:43.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:43.914 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T10:41:43.914 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:41:43.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.896+0000 7ff9e6c2f700 1 -- 172.21.15.92:0/1825973370 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff9cc066f30 msgr2=0x7ff9cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:43.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.896+0000 7ff9e6c2f700 1 --2- 172.21.15.92:0/1825973370 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff9cc066f30 0x7ff9cc0693e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc001a70 tx=0x7ff9dc017000 comp rx=0 tx=0).stop 2023-10-06T10:41:43.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.896+0000 7ff9e6c2f700 1 -- 172.21.15.92:0/1825973370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e0157040 msgr2=0x7ff9e01574b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:43.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.896+0000 7ff9e6c2f700 1 --2- 172.21.15.92:0/1825973370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e0157040 0x7ff9e01574b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00abdf0 tx=0x7ff9d80091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:43.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.896+0000 7ff9e6c2f700 1 -- 172.21.15.92:0/1825973370 shutdown_connections 2023-10-06T10:41:43.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.896+0000 7ff9e6c2f700 1 --2- 172.21.15.92:0/1825973370 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff9cc066f30 0x7ff9cc0693e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:43.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.896+0000 7ff9e6c2f700 1 --2- 172.21.15.92:0/1825973370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e0157040 0x7ff9e01574b0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:43.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.896+0000 7ff9e6c2f700 1 --2- 172.21.15.92:0/1825973370 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9c8003a90 0x7ff9e0155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:43.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.896+0000 7ff9e6c2f700 1 --2- 172.21.15.92:0/1825973370 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9e00c1de0 0x7ff9e0159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:43.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.896+0000 7ff9e6c2f700 1 -- 172.21.15.92:0/1825973370 >> 172.21.15.92:0/1825973370 conn(0x7ff9e0068080 msgr2=0x7ff9e006d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:43.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.896+0000 7ff9e6c2f700 1 -- 172.21.15.92:0/1825973370 shutdown_connections 2023-10-06T10:41:43.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:43.896+0000 7ff9e6c2f700 1 -- 172.21.15.92:0/1825973370 wait complete. 2023-10-06T10:41:43.918 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:41:44.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.030+0000 7f6481b02700 1 Processor -- start 2023-10-06T10:41:44.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.030+0000 7f6481b02700 1 -- start start 2023-10-06T10:41:44.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.031+0000 7f6481b02700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f647c159560 0x7f647c0aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:44.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.031+0000 7f6481b02700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f647c0b49a0 0x7f647c0aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:44.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.031+0000 7f6481b02700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f647c0b0c60 con 0x7f647c0b0760 2023-10-06T10:41:44.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.031+0000 7f6481b02700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f647c0b0f40 con 0x7f647c159560 2023-10-06T10:41:44.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.031+0000 7f6481b02700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f647c0b1220 con 0x7f647c0b49a0 2023-10-06T10:41:44.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.031+0000 7f647bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f647c0b0760 0x7f647c0b3890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33162/0 (socket says 172.21.15.92:33162) 2023-10-06T10:41:44.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.031+0000 7f647bfff700 1 -- 172.21.15.92:0/1641653642 learned_addr learned my addr 172.21.15.92:0/1641653642 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:44.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647a7fc700 1 -- 172.21.15.92:0/1641653642 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2813796802 0 0) 0x7f647c0b0c60 con 0x7f647c0b0760 2023-10-06T10:41:44.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647affd700 1 --2- 172.21.15.92:0/1641653642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f647c0b49a0 0x7f647c0aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:44.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647a7fc700 1 -- 172.21.15.92:0/1641653642 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6464003570 con 0x7f647c0b0760 2023-10-06T10:41:44.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647b7fe700 1 --2- 172.21.15.92:0/1641653642 >> v2:172.21.15.124:3300/0 conn(0x7f647c159560 0x7f647c0aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:44.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647a7fc700 1 -- 172.21.15.92:0/1641653642 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1765301255 0 0) 0x7f6464003570 con 0x7f647c0b0760 2023-10-06T10:41:44.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647a7fc700 1 -- 172.21.15.92:0/1641653642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f647c0b49a0 msgr2=0x7f647c0aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:44.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647a7fc700 1 --2- 172.21.15.92:0/1641653642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f647c0b49a0 0x7f647c0aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647a7fc700 1 -- 172.21.15.92:0/1641653642 >> v2:172.21.15.124:3300/0 conn(0x7f647c159560 msgr2=0x7f647c0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:44.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647a7fc700 1 --2- 172.21.15.92:0/1641653642 >> v2:172.21.15.124:3300/0 conn(0x7f647c159560 0x7f647c0aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647a7fc700 1 -- 172.21.15.92:0/1641653642 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647c0b1500 con 0x7f647c0b0760 2023-10-06T10:41:44.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647a7fc700 1 -- 172.21.15.92:0/1641653642 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f64740017f0 con 0x7f647c0b0760 2023-10-06T10:41:44.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647a7fc700 1 -- 172.21.15.92:0/1641653642 >> v1:172.21.15.92:6789/0 conn(0x7f647c0b0760 legacy=0x7f647c0b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:44.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647a7fc700 1 --2- 172.21.15.92:0/1641653642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6464003a60 0x7f6464003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:44.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647a7fc700 1 --2- 172.21.15.92:0/1641653642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6464004810 0x7f6464004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:44.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647a7fc700 1 --2- 172.21.15.92:0/1641653642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f647c0b49a0 0x7f64640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:44.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647a7fc700 1 -- 172.21.15.92:0/1641653642 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c0b0f40 con 0x7f6464004810 2023-10-06T10:41:44.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647a7fc700 1 -- 172.21.15.92:0/1641653642 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c0b1220 con 0x7f6464003a60 2023-10-06T10:41:44.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647a7fc700 1 -- 172.21.15.92:0/1641653642 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646400ba60 con 0x7f647c0b49a0 2023-10-06T10:41:44.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647affd700 1 --2- 172.21.15.92:0/1641653642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6464004810 0x7f6464004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:44.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647b7fe700 1 --2- 172.21.15.92:0/1641653642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6464003a60 0x7f6464003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:44.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647bfff700 1 --2- 172.21.15.92:0/1641653642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f647c0b49a0 0x7f64640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:44.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647affd700 1 -- 172.21.15.92:0/1641653642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f647c0b49a0 msgr2=0x7f64640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:44.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647affd700 1 --2- 172.21.15.92:0/1641653642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f647c0b49a0 0x7f64640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647affd700 1 -- 172.21.15.92:0/1641653642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6464003a60 msgr2=0x7f6464003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:44.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647affd700 1 --2- 172.21.15.92:0/1641653642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6464003a60 0x7f6464003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647affd700 1 -- 172.21.15.92:0/1641653642 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6464003570 con 0x7f6464004810 2023-10-06T10:41:44.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647bfff700 1 --2- 172.21.15.92:0/1641653642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f647c0b49a0 0x7f64640051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:44.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.032+0000 7f647affd700 1 --2- 172.21.15.92:0/1641653642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6464004810 0x7f6464004c80 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f6470009c30 tx=0x7f64700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e366146af7f86352 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:44.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.033+0000 7f647a7fc700 1 -- 172.21.15.92:0/1641653642 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647000e710 con 0x7f6464004810 2023-10-06T10:41:44.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.033+0000 7f647a7fc700 1 -- 172.21.15.92:0/1641653642 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647000e870 con 0x7f6464004810 2023-10-06T10:41:44.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.033+0000 7f647a7fc700 1 -- 172.21.15.92:0/1641653642 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647000ec30 con 0x7f6464004810 2023-10-06T10:41:44.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.033+0000 7f6481b02700 1 -- 172.21.15.92:0/1641653642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6464004810 msgr2=0x7f6464004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:44.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.033+0000 7f6481b02700 1 --2- 172.21.15.92:0/1641653642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6464004810 0x7f6464004c80 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f6470009c30 tx=0x7f64700091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.033+0000 7f647bfff700 1 -- 172.21.15.92:0/1641653642 reap_dead start 2023-10-06T10:41:44.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.033+0000 7f6481b02700 1 -- 172.21.15.92:0/1641653642 shutdown_connections 2023-10-06T10:41:44.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.033+0000 7f6481b02700 1 -- 172.21.15.92:0/1641653642 >> 172.21.15.92:0/1641653642 conn(0x7f647c0997f0 msgr2=0x7f647c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:44.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.033+0000 7f6481b02700 1 -- 172.21.15.92:0/1641653642 shutdown_connections 2023-10-06T10:41:44.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.033+0000 7f6481b02700 1 -- 172.21.15.92:0/1641653642 wait complete. 2023-10-06T10:41:44.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.034+0000 7f6481b02700 1 Processor -- start 2023-10-06T10:41:44.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.034+0000 7f6481b02700 1 -- start start 2023-10-06T10:41:44.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.035+0000 7f6481b02700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6464004810 0x7f647c14c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:44.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.035+0000 7f6481b02700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f647c16a5a0 0x7f647c099f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:44.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.035+0000 7f6481b02700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f647c09a490 0x7f647c09d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:44.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.035+0000 7f6481b02700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c09d2b0 con 0x7f647c09a490 2023-10-06T10:41:44.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.035+0000 7f6481b02700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c09cc40 con 0x7f647c16a5a0 2023-10-06T10:41:44.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.035+0000 7f6481b02700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f647c156ec0 con 0x7f6464004810 2023-10-06T10:41:44.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.035+0000 7f647bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f647c09a490 0x7f647c09d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:44.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.035+0000 7f647bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f647c09a490 0x7f647c09d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57038/0 (socket says 172.21.15.92:57038) 2023-10-06T10:41:44.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.035+0000 7f647bfff700 1 -- 172.21.15.92:0/2878603926 learned_addr learned my addr 172.21.15.92:0/2878603926 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:44.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.035+0000 7f647b7fe700 1 --2- 172.21.15.92:0/2878603926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6464004810 0x7f647c14c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:44.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.035+0000 7f647affd700 1 --2- 172.21.15.92:0/2878603926 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f647c16a5a0 0x7f647c099f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:44.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.036+0000 7f647bfff700 1 -- 172.21.15.92:0/2878603926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6464004810 msgr2=0x7f647c14c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:44.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.036+0000 7f647bfff700 1 --2- 172.21.15.92:0/2878603926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6464004810 0x7f647c14c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.036+0000 7f647bfff700 1 -- 172.21.15.92:0/2878603926 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f647c16a5a0 msgr2=0x7f647c099f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:44.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.036+0000 7f647bfff700 1 --2- 172.21.15.92:0/2878603926 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f647c16a5a0 0x7f647c099f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.036+0000 7f647bfff700 1 -- 172.21.15.92:0/2878603926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647c09cf80 con 0x7f647c09a490 2023-10-06T10:41:44.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.036+0000 7f647bfff700 1 --2- 172.21.15.92:0/2878603926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f647c09a490 0x7f647c09d740 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f647c15d290 tx=0x7f6474002150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:44.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.036+0000 7f64637fe700 1 -- 172.21.15.92:0/2878603926 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647400eb10 con 0x7f647c09a490 2023-10-06T10:41:44.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.036+0000 7f64637fe700 1 -- 172.21.15.92:0/2878603926 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647400ec70 con 0x7f647c09a490 2023-10-06T10:41:44.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.036+0000 7f6481b02700 1 -- 172.21.15.92:0/2878603926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f647c09c680 con 0x7f647c09a490 2023-10-06T10:41:44.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.036+0000 7f64637fe700 1 -- 172.21.15.92:0/2878603926 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6474013420 con 0x7f647c09a490 2023-10-06T10:41:44.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.036+0000 7f6481b02700 1 -- 172.21.15.92:0/2878603926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647c14cea0 con 0x7f647c09a490 2023-10-06T10:41:44.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.037+0000 7f6481b02700 1 -- 172.21.15.92:0/2878603926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f647c09d420 con 0x7f647c09a490 2023-10-06T10:41:44.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.037+0000 7f64637fe700 1 -- 172.21.15.92:0/2878603926 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6474013830 con 0x7f647c09a490 2023-10-06T10:41:44.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.038+0000 7f64637fe700 1 --2- 172.21.15.92:0/2878603926 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6468066f30 0x7f64680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:44.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.038+0000 7f64637fe700 1 -- 172.21.15.92:0/2878603926 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f6474084950 con 0x7f647c09a490 2023-10-06T10:41:44.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.038+0000 7f647b7fe700 1 --2- 172.21.15.92:0/2878603926 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6468066f30 0x7f64680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:44.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.038+0000 7f647b7fe700 1 --2- 172.21.15.92:0/2878603926 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6468066f30 0x7f64680693e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f646c006ea0 tx=0x7f646c009120 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:44.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.042+0000 7f64637fe700 1 -- 172.21.15.92:0/2878603926 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6474051c70 con 0x7f647c09a490 2023-10-06T10:41:44.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.227+0000 7f6481b02700 1 -- 172.21.15.92:0/2878603926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f647c064a90 con 0x7f647c09a490 2023-10-06T10:41:44.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.227+0000 7f64637fe700 1 -- 172.21.15.92:0/2878603926 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+13855 (secure 0 0 0) 0x7f6474055290 con 0x7f647c09a490 2023-10-06T10:41:44.229 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:44.230 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":60,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:41:43.494506+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:41:44.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.230+0000 7f6481b02700 1 -- 172.21.15.92:0/2878603926 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6468066f30 msgr2=0x7f64680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:44.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.230+0000 7f6481b02700 1 --2- 172.21.15.92:0/2878603926 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6468066f30 0x7f64680693e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f646c006ea0 tx=0x7f646c009120 comp rx=0 tx=0).stop 2023-10-06T10:41:44.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.230+0000 7f6481b02700 1 -- 172.21.15.92:0/2878603926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f647c09a490 msgr2=0x7f647c09d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:44.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.230+0000 7f6481b02700 1 --2- 172.21.15.92:0/2878603926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f647c09a490 0x7f647c09d740 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f647c15d290 tx=0x7f6474002150 comp rx=0 tx=0).stop 2023-10-06T10:41:44.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.230+0000 7f6481b02700 1 -- 172.21.15.92:0/2878603926 shutdown_connections 2023-10-06T10:41:44.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.230+0000 7f6481b02700 1 --2- 172.21.15.92:0/2878603926 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6468066f30 0x7f64680693e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.230+0000 7f6481b02700 1 --2- 172.21.15.92:0/2878603926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f647c09a490 0x7f647c09d740 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.230+0000 7f6481b02700 1 --2- 172.21.15.92:0/2878603926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6464004810 0x7f647c14c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.230+0000 7f6481b02700 1 --2- 172.21.15.92:0/2878603926 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f647c16a5a0 0x7f647c099f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.230+0000 7f6481b02700 1 -- 172.21.15.92:0/2878603926 >> 172.21.15.92:0/2878603926 conn(0x7f647c0997f0 msgr2=0x7f647c1571f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:44.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.230+0000 7f6481b02700 1 -- 172.21.15.92:0/2878603926 shutdown_connections 2023-10-06T10:41:44.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.230+0000 7f6481b02700 1 -- 172.21.15.92:0/2878603926 wait complete. 2023-10-06T10:41:44.244 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T10:38:09.680559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T10:38:10.120786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T10:41:44.244 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T10:41:44.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.360+0000 7f6fc4dc6700 1 Processor -- start 2023-10-06T10:41:44.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.360+0000 7f6fc4dc6700 1 -- start start 2023-10-06T10:41:44.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.361+0000 7f6fc4dc6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6fc01574f0 0x7f6fc006e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:44.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.361+0000 7f6fc4dc6700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fc006e9c0 0x7f6fc00ab120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:44.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.361+0000 7f6fc4dc6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc00c7cd0 con 0x7f6fc01507e0 2023-10-06T10:41:44.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.361+0000 7f6fc4dc6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc006bb80 con 0x7f6fc01574f0 2023-10-06T10:41:44.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.361+0000 7f6fc4dc6700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc006ee40 con 0x7f6fc006e9c0 2023-10-06T10:41:44.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.361+0000 7f6fb5d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6fc01574f0 0x7f6fc006e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:44.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.361+0000 7f6fbe59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6fc01507e0 0x7f6fc006b470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33176/0 (socket says 172.21.15.92:33176) 2023-10-06T10:41:44.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.361+0000 7f6fbe59c700 1 -- 172.21.15.92:0/1369787751 learned_addr learned my addr 172.21.15.92:0/1369787751 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:44.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.361+0000 7f6fb5d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6fc01574f0 0x7f6fc006e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57050/0 (socket says 172.21.15.92:57050) 2023-10-06T10:41:44.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.361+0000 7f6fb5d9b700 1 -- 172.21.15.92:0/1369787751 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fc006e9c0 msgr2=0x7f6fc00ab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:44.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.361+0000 7f6fb5d9b700 1 --2- 172.21.15.92:0/1369787751 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fc006e9c0 0x7f6fc00ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.361+0000 7f6fb5d9b700 1 -- 172.21.15.92:0/1369787751 >> v1:172.21.15.92:6789/0 conn(0x7f6fc01507e0 legacy=0x7f6fc006b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:44.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.361+0000 7f6fb5d9b700 1 -- 172.21.15.92:0/1369787751 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc00c9170 con 0x7f6fc01574f0 2023-10-06T10:41:44.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.361+0000 7f6fbdd9b700 1 -- 172.21.15.92:0/1369787751 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2492619450 0 0) 0x7f6fc00c7cd0 con 0x7f6fc01507e0 2023-10-06T10:41:44.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.362+0000 7f6fb5d9b700 1 --2- 172.21.15.92:0/1369787751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fc01574f0 0x7f6fc006e480 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009910 tx=0x7f6fa80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4106dd174f5bf8b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:44.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.362+0000 7f6fbdd9b700 1 -- 172.21.15.92:0/1369787751 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa800b820 con 0x7f6fc01574f0 2023-10-06T10:41:44.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.362+0000 7f6fbdd9b700 1 -- 172.21.15.92:0/1369787751 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa800b980 con 0x7f6fc01574f0 2023-10-06T10:41:44.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.362+0000 7f6fbdd9b700 1 -- 172.21.15.92:0/1369787751 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa800bd20 con 0x7f6fc01574f0 2023-10-06T10:41:44.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.362+0000 7f6fc4dc6700 1 -- 172.21.15.92:0/1369787751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fc01574f0 msgr2=0x7f6fc006e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:44.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.362+0000 7f6fc4dc6700 1 --2- 172.21.15.92:0/1369787751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fc01574f0 0x7f6fc006e480 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009910 tx=0x7f6fa80091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.362+0000 7f6fc4dc6700 1 -- 172.21.15.92:0/1369787751 shutdown_connections 2023-10-06T10:41:44.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.362+0000 7f6fc4dc6700 1 --2- 172.21.15.92:0/1369787751 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fc006e9c0 0x7f6fc00ab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.362+0000 7f6fc4dc6700 1 --2- 172.21.15.92:0/1369787751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fc01574f0 0x7f6fc006e480 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.362+0000 7f6fc4dc6700 1 -- 172.21.15.92:0/1369787751 >> 172.21.15.92:0/1369787751 conn(0x7f6fc0069190 msgr2=0x7f6fc00cd180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:44.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.362+0000 7f6fc4dc6700 1 -- 172.21.15.92:0/1369787751 shutdown_connections 2023-10-06T10:41:44.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.362+0000 7f6fc4dc6700 1 -- 172.21.15.92:0/1369787751 wait complete. 2023-10-06T10:41:44.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.363+0000 7f6fc4dc6700 1 Processor -- start 2023-10-06T10:41:44.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.364+0000 7f6fc4dc6700 1 -- start start 2023-10-06T10:41:44.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.364+0000 7f6fc4dc6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fc006e9c0 0x7f6fc01699e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:44.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.364+0000 7f6fc4dc6700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fc01507e0 0x7f6fc00675b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:44.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.364+0000 7f6fc4dc6700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fc01574f0 0x7f6fc0067af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:44.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.364+0000 7f6fc4dc6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc00acbc0 con 0x7f6fc006e9c0 2023-10-06T10:41:44.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.364+0000 7f6fc4dc6700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc00ac600 con 0x7f6fc01574f0 2023-10-06T10:41:44.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.364+0000 7f6fc4dc6700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc00ac8e0 con 0x7f6fc01507e0 2023-10-06T10:41:44.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.364+0000 7f6fbe59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fc006e9c0 0x7f6fc01699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:44.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.364+0000 7f6fbe59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fc006e9c0 0x7f6fc01699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57066/0 (socket says 172.21.15.92:57066) 2023-10-06T10:41:44.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.364+0000 7f6fbe59c700 1 -- 172.21.15.92:0/32908579 learned_addr learned my addr 172.21.15.92:0/32908579 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:44.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.365+0000 7f6fbe59c700 1 -- 172.21.15.92:0/32908579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fc01507e0 msgr2=0x7f6fc00675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:44.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.365+0000 7f6fbe59c700 1 --2- 172.21.15.92:0/32908579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fc01507e0 0x7f6fc00675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.365+0000 7f6fbe59c700 1 -- 172.21.15.92:0/32908579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fc01574f0 msgr2=0x7f6fc0067af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:44.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.365+0000 7f6fbe59c700 1 --2- 172.21.15.92:0/32908579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fc01574f0 0x7f6fc0067af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.365+0000 7f6fbe59c700 1 -- 172.21.15.92:0/32908579 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa8009420 con 0x7f6fc006e9c0 2023-10-06T10:41:44.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.365+0000 7f6fbe59c700 1 --2- 172.21.15.92:0/32908579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fc006e9c0 0x7f6fc01699e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000c7e0 tx=0x7f6fb0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:44.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.365+0000 7f6fb6ffd700 1 -- 172.21.15.92:0/32908579 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0005b30 con 0x7f6fc006e9c0 2023-10-06T10:41:44.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.365+0000 7f6fb6ffd700 1 -- 172.21.15.92:0/32908579 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb0005c90 con 0x7f6fc006e9c0 2023-10-06T10:41:44.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.365+0000 7f6fc4dc6700 1 -- 172.21.15.92:0/32908579 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc00aeec0 con 0x7f6fc006e9c0 2023-10-06T10:41:44.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.365+0000 7f6fb6ffd700 1 -- 172.21.15.92:0/32908579 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0007300 con 0x7f6fc006e9c0 2023-10-06T10:41:44.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.365+0000 7f6fc4dc6700 1 -- 172.21.15.92:0/32908579 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc0169f20 con 0x7f6fc006e9c0 2023-10-06T10:41:44.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.366+0000 7f6fc4dc6700 1 -- 172.21.15.92:0/32908579 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc00aed10 con 0x7f6fc006e9c0 2023-10-06T10:41:44.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.366+0000 7f6fb6ffd700 1 -- 172.21.15.92:0/32908579 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6fb001b400 con 0x7f6fc006e9c0 2023-10-06T10:41:44.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.367+0000 7f6fb6ffd700 1 --2- 172.21.15.92:0/32908579 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6fa0066f30 0x7f6fa00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:44.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.367+0000 7f6fb6ffd700 1 -- 172.21.15.92:0/32908579 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f6fb0083fd0 con 0x7f6fc006e9c0 2023-10-06T10:41:44.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.367+0000 7f6fb5d9b700 1 --2- 172.21.15.92:0/32908579 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6fa0066f30 0x7f6fa00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:44.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.367+0000 7f6fb5d9b700 1 --2- 172.21.15.92:0/32908579 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6fa0066f30 0x7f6fa00693e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0068690 tx=0x7f6fa8017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:44.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.371+0000 7f6fb6ffd700 1 -- 172.21.15.92:0/32908579 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6fb0051370 con 0x7f6fc006e9c0 2023-10-06T10:41:44.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.552+0000 7f6fc4dc6700 1 -- 172.21.15.92:0/32908579 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6fc0064a90 con 0x7f6fc006e9c0 2023-10-06T10:41:44.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.553+0000 7f6fb6ffd700 1 -- 172.21.15.92:0/32908579 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v60) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6fb0054990 con 0x7f6fc006e9c0 2023-10-06T10:41:44.555 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T10:41:44.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.555+0000 7f6fc4dc6700 1 -- 172.21.15.92:0/32908579 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6fa0066f30 msgr2=0x7f6fa00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:44.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.555+0000 7f6fc4dc6700 1 --2- 172.21.15.92:0/32908579 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6fa0066f30 0x7f6fa00693e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0068690 tx=0x7f6fa8017000 comp rx=0 tx=0).stop 2023-10-06T10:41:44.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.555+0000 7f6fc4dc6700 1 -- 172.21.15.92:0/32908579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fc006e9c0 msgr2=0x7f6fc01699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:44.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.555+0000 7f6fc4dc6700 1 --2- 172.21.15.92:0/32908579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fc006e9c0 0x7f6fc01699e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000c7e0 tx=0x7f6fb0007df0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.555+0000 7f6fc4dc6700 1 -- 172.21.15.92:0/32908579 shutdown_connections 2023-10-06T10:41:44.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.555+0000 7f6fc4dc6700 1 --2- 172.21.15.92:0/32908579 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6fa0066f30 0x7f6fa00693e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.555+0000 7f6fc4dc6700 1 --2- 172.21.15.92:0/32908579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fc01574f0 0x7f6fc0067af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.555+0000 7f6fc4dc6700 1 --2- 172.21.15.92:0/32908579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fc006e9c0 0x7f6fc01699e0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.555+0000 7f6fc4dc6700 1 --2- 172.21.15.92:0/32908579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fc01507e0 0x7f6fc00675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.555+0000 7f6fc4dc6700 1 -- 172.21.15.92:0/32908579 >> 172.21.15.92:0/32908579 conn(0x7f6fc0069190 msgr2=0x7f6fc00abd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:44.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.555+0000 7f6fc4dc6700 1 -- 172.21.15.92:0/32908579 shutdown_connections 2023-10-06T10:41:44.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.555+0000 7f6fc4dc6700 1 -- 172.21.15.92:0/32908579 wait complete. 2023-10-06T10:41:44.569 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T10:41:44.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.684+0000 7f117759e700 1 Processor -- start 2023-10-06T10:41:44.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.684+0000 7f117759e700 1 -- start start 2023-10-06T10:41:44.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.685+0000 7f117759e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11780cd3c0 0x7f11780ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:44.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.685+0000 7f117759e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11780cc970 con 0x7f11780b29f0 2023-10-06T10:41:44.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.685+0000 7f117759e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11780cbd60 con 0x7f11780b2620 2023-10-06T10:41:44.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.685+0000 7f117759e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11780cc040 con 0x7f11780cd3c0 2023-10-06T10:41:44.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.685+0000 7f1176d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f11780b29f0 0x7f11780c5350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33190/0 (socket says 172.21.15.92:33190) 2023-10-06T10:41:44.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.685+0000 7f1176d9d700 1 -- 172.21.15.92:0/3576931945 learned_addr learned my addr 172.21.15.92:0/3576931945 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:44.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.685+0000 7f1175d9b700 1 --2- 172.21.15.92:0/3576931945 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11780cd3c0 0x7f11780ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:44.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.685+0000 7f117559a700 1 -- 172.21.15.92:0/3576931945 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1714896767 0 0) 0x7f11780cc970 con 0x7f11780b29f0 2023-10-06T10:41:44.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.685+0000 7f117559a700 1 -- 172.21.15.92:0/3576931945 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1160003570 con 0x7f11780b29f0 2023-10-06T10:41:44.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.686+0000 7f117559a700 1 -- 172.21.15.92:0/3576931945 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 318488591 0 0) 0x7f11780cbd60 con 0x7f11780b2620 2023-10-06T10:41:44.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.686+0000 7f117559a700 1 -- 172.21.15.92:0/3576931945 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11780cc970 con 0x7f11780b2620 2023-10-06T10:41:44.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.686+0000 7f117559a700 1 -- 172.21.15.92:0/3576931945 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3026403616 0 0) 0x7f1160003570 con 0x7f11780b29f0 2023-10-06T10:41:44.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.686+0000 7f1175d9b700 1 -- 172.21.15.92:0/3576931945 >> v1:172.21.15.124:6789/0 conn(0x7f11780b2620 legacy=0x7f11780cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:44.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.686+0000 7f1175d9b700 1 -- 172.21.15.92:0/3576931945 >> v1:172.21.15.92:6789/0 conn(0x7f11780b29f0 legacy=0x7f11780c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:44.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.686+0000 7f1175d9b700 1 -- 172.21.15.92:0/3576931945 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11780cc320 con 0x7f11780cd3c0 2023-10-06T10:41:44.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.686+0000 7f1175d9b700 1 --2- 172.21.15.92:0/3576931945 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11780cd3c0 0x7f11780ceaa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f116c009910 tx=0x7f116c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3149535a44da1d84 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:44.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.686+0000 7f117559a700 1 -- 172.21.15.92:0/3576931945 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11700017f0 con 0x7f11780cd3c0 2023-10-06T10:41:44.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.686+0000 7f117559a700 1 -- 172.21.15.92:0/3576931945 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116c011420 con 0x7f11780cd3c0 2023-10-06T10:41:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.686+0000 7f117559a700 1 -- 172.21.15.92:0/3576931945 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c0117c0 con 0x7f11780cd3c0 2023-10-06T10:41:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.686+0000 7f117759e700 1 -- 172.21.15.92:0/3576931945 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11780cd3c0 msgr2=0x7f11780ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.686+0000 7f117759e700 1 --2- 172.21.15.92:0/3576931945 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11780cd3c0 0x7f11780ceaa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f116c009910 tx=0x7f116c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.687+0000 7f117759e700 1 -- 172.21.15.92:0/3576931945 shutdown_connections 2023-10-06T10:41:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.687+0000 7f117759e700 1 --2- 172.21.15.92:0/3576931945 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11780cd3c0 0x7f11780ceaa0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.687+0000 7f117759e700 1 -- 172.21.15.92:0/3576931945 >> 172.21.15.92:0/3576931945 conn(0x7f1178099520 msgr2=0x7f117806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:44.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.687+0000 7f117759e700 1 -- 172.21.15.92:0/3576931945 shutdown_connections 2023-10-06T10:41:44.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.687+0000 7f117759e700 1 -- 172.21.15.92:0/3576931945 wait complete. 2023-10-06T10:41:44.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.688+0000 7f117759e700 1 Processor -- start 2023-10-06T10:41:44.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.688+0000 7f117759e700 1 -- start start 2023-10-06T10:41:44.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.688+0000 7f117759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11780b2620 0x7f11780ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:44.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.688+0000 7f117759e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11780b29f0 0x7f11780b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:44.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.688+0000 7f117759e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11780cd3c0 0x7f1178152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:44.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.688+0000 7f117759e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1178156750 con 0x7f11780b2620 2023-10-06T10:41:44.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.688+0000 7f117759e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117815a900 con 0x7f11780cd3c0 2023-10-06T10:41:44.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.688+0000 7f117759e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1178159cf0 con 0x7f11780b29f0 2023-10-06T10:41:44.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.688+0000 7f117659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11780b2620 0x7f11780ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:44.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.689+0000 7f117659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11780b2620 0x7f11780ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57082/0 (socket says 172.21.15.92:57082) 2023-10-06T10:41:44.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.689+0000 7f117659c700 1 -- 172.21.15.92:0/1560338627 learned_addr learned my addr 172.21.15.92:0/1560338627 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:44.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.689+0000 7f1176d9d700 1 --2- 172.21.15.92:0/1560338627 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11780cd3c0 0x7f1178152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:44.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.689+0000 7f1175d9b700 1 --2- 172.21.15.92:0/1560338627 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11780b29f0 0x7f11780b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:44.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.689+0000 7f117659c700 1 -- 172.21.15.92:0/1560338627 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11780b29f0 msgr2=0x7f11780b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:44.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.689+0000 7f117659c700 1 --2- 172.21.15.92:0/1560338627 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11780b29f0 0x7f11780b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.689+0000 7f117659c700 1 -- 172.21.15.92:0/1560338627 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11780cd3c0 msgr2=0x7f1178152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:44.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.689+0000 7f117659c700 1 --2- 172.21.15.92:0/1560338627 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11780cd3c0 0x7f1178152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.689+0000 7f117659c700 1 -- 172.21.15.92:0/1560338627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116c009420 con 0x7f11780b2620 2023-10-06T10:41:44.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.689+0000 7f1176d9d700 1 --2- 172.21.15.92:0/1560338627 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11780cd3c0 0x7f1178152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:44.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.689+0000 7f117659c700 1 --2- 172.21.15.92:0/1560338627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11780b2620 0x7f11780ce3a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f116800c560 tx=0x7f1168007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:44.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.689+0000 7f11667fc700 1 -- 172.21.15.92:0/1560338627 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168010510 con 0x7f11780b2620 2023-10-06T10:41:44.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.689+0000 7f11667fc700 1 -- 172.21.15.92:0/1560338627 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1168010670 con 0x7f11780b2620 2023-10-06T10:41:44.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.689+0000 7f117759e700 1 -- 172.21.15.92:0/1560338627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1178159fd0 con 0x7f11780b2620 2023-10-06T10:41:44.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.689+0000 7f11667fc700 1 -- 172.21.15.92:0/1560338627 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168010a30 con 0x7f11780b2620 2023-10-06T10:41:44.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.689+0000 7f117759e700 1 -- 172.21.15.92:0/1560338627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117815a590 con 0x7f11780b2620 2023-10-06T10:41:44.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.690+0000 7f117759e700 1 -- 172.21.15.92:0/1560338627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f117805f230 con 0x7f11780b2620 2023-10-06T10:41:44.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.690+0000 7f11667fc700 1 -- 172.21.15.92:0/1560338627 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1168002050 con 0x7f11780b2620 2023-10-06T10:41:44.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.691+0000 7f11667fc700 1 --2- 172.21.15.92:0/1560338627 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1158066f30 0x7f11580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:44.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.691+0000 7f11667fc700 1 -- 172.21.15.92:0/1560338627 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f1168083e90 con 0x7f11780b2620 2023-10-06T10:41:44.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.691+0000 7f1175d9b700 1 --2- 172.21.15.92:0/1560338627 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1158066f30 0x7f11580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:44.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.691+0000 7f1175d9b700 1 --2- 172.21.15.92:0/1560338627 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1158066f30 0x7f11580693e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f116c0030c0 tx=0x7f116c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:44.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.695+0000 7f11667fc700 1 -- 172.21.15.92:0/1560338627 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11680511b0 con 0x7f11780b2620 2023-10-06T10:41:44.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.876+0000 7f117759e700 1 -- 172.21.15.92:0/1560338627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11780cc600 con 0x7f11780b2620 2023-10-06T10:41:44.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.877+0000 7f11667fc700 1 -- 172.21.15.92:0/1560338627 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11680547d0 con 0x7f11780b2620 2023-10-06T10:41:44.879 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T10:41:44.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.879+0000 7f117759e700 1 -- 172.21.15.92:0/1560338627 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1158066f30 msgr2=0x7f11580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:44.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.879+0000 7f117759e700 1 --2- 172.21.15.92:0/1560338627 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1158066f30 0x7f11580693e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f116c0030c0 tx=0x7f116c016000 comp rx=0 tx=0).stop 2023-10-06T10:41:44.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.879+0000 7f117759e700 1 -- 172.21.15.92:0/1560338627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11780b2620 msgr2=0x7f11780ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:44.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.879+0000 7f117759e700 1 --2- 172.21.15.92:0/1560338627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11780b2620 0x7f11780ce3a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f116800c560 tx=0x7f1168007e00 comp rx=0 tx=0).stop 2023-10-06T10:41:44.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.879+0000 7f117759e700 1 -- 172.21.15.92:0/1560338627 shutdown_connections 2023-10-06T10:41:44.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.879+0000 7f117759e700 1 --2- 172.21.15.92:0/1560338627 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1158066f30 0x7f11580693e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.879+0000 7f117759e700 1 --2- 172.21.15.92:0/1560338627 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11780cd3c0 0x7f1178152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.879+0000 7f117759e700 1 --2- 172.21.15.92:0/1560338627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11780b2620 0x7f11780ce3a0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.879+0000 7f117759e700 1 --2- 172.21.15.92:0/1560338627 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11780b29f0 0x7f11780b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:44.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.879+0000 7f117759e700 1 -- 172.21.15.92:0/1560338627 >> 172.21.15.92:0/1560338627 conn(0x7f1178099520 msgr2=0x7f11780c6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:44.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.879+0000 7f117759e700 1 -- 172.21.15.92:0/1560338627 shutdown_connections 2023-10-06T10:41:44.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:44.879+0000 7f117759e700 1 -- 172.21.15.92:0/1560338627 wait complete. 2023-10-06T10:41:44.893 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T10:41:45.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.005+0000 7f2ae7fff700 1 Processor -- start 2023-10-06T10:41:45.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.005+0000 7f2ae7fff700 1 -- start start 2023-10-06T10:41:45.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.006+0000 7f2ae7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2ae8157410 0x7f2ae80cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:45.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.006+0000 7f2ae7fff700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2ae815b660 0x7f2ae80c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:45.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.006+0000 7f2ae7fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ae815b1d0 con 0x7f2ae8157040 2023-10-06T10:41:45.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.006+0000 7f2ae7fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ae80ce4b0 con 0x7f2ae8157410 2023-10-06T10:41:45.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.006+0000 7f2ae7fff700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ae80ce790 con 0x7f2ae815b660 2023-10-06T10:41:45.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.006+0000 7f2ade7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2ae8157410 0x7f2ae80cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:45.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.006+0000 7f2ade7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2ae8157410 0x7f2ae80cde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57086/0 (socket says 172.21.15.92:57086) 2023-10-06T10:41:45.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.006+0000 7f2ade7fc700 1 -- 172.21.15.92:0/692507820 learned_addr learned my addr 172.21.15.92:0/692507820 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:45.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.006+0000 7f2ae6ffd700 1 --1- 172.21.15.92:0/692507820 >> v1:172.21.15.92:6789/0 conn(0x7f2ae8157040 0x7f2ae806d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33202/0 (socket says 172.21.15.92:33202) 2023-10-06T10:41:45.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.006+0000 7f2ae77fe700 1 --2- 172.21.15.92:0/692507820 >> v2:172.21.15.124:3300/0 conn(0x7f2ae815b660 0x7f2ae80c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:45.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.006+0000 7f2ade7fc700 1 -- 172.21.15.92:0/692507820 >> v2:172.21.15.124:3300/0 conn(0x7f2ae815b660 msgr2=0x7f2ae80c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:45.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.006+0000 7f2ae67fc700 1 -- 172.21.15.92:0/692507820 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2513217521 0 0) 0x7f2ae815b1d0 con 0x7f2ae8157040 2023-10-06T10:41:45.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.007+0000 7f2ade7fc700 1 --2- 172.21.15.92:0/692507820 >> v2:172.21.15.124:3300/0 conn(0x7f2ae815b660 0x7f2ae80c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:45.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.007+0000 7f2ade7fc700 1 -- 172.21.15.92:0/692507820 >> v1:172.21.15.92:6789/0 conn(0x7f2ae8157040 legacy=0x7f2ae806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:45.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.007+0000 7f2ade7fc700 1 -- 172.21.15.92:0/692507820 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae815ae60 con 0x7f2ae8157410 2023-10-06T10:41:45.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.007+0000 7f2ae77fe700 1 --2- 172.21.15.92:0/692507820 >> v2:172.21.15.124:3300/0 conn(0x7f2ae815b660 0x7f2ae80c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:45.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.007+0000 7f2ade7fc700 1 --2- 172.21.15.92:0/692507820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ae8157410 0x7f2ae80cde60 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2ae815b140 tx=0x7f2ad0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2046dcd5977e7fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:45.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.007+0000 7f2ae67fc700 1 -- 172.21.15.92:0/692507820 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad000b860 con 0x7f2ae8157410 2023-10-06T10:41:45.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.007+0000 7f2ae67fc700 1 -- 172.21.15.92:0/692507820 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad000b9c0 con 0x7f2ae8157410 2023-10-06T10:41:45.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.007+0000 7f2ae67fc700 1 -- 172.21.15.92:0/692507820 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad000bd60 con 0x7f2ae8157410 2023-10-06T10:41:45.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.007+0000 7f2ae7fff700 1 -- 172.21.15.92:0/692507820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ae8157410 msgr2=0x7f2ae80cde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:45.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.007+0000 7f2ae7fff700 1 --2- 172.21.15.92:0/692507820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ae8157410 0x7f2ae80cde60 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2ae815b140 tx=0x7f2ad0009250 comp rx=0 tx=0).stop 2023-10-06T10:41:45.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.007+0000 7f2ae7fff700 1 -- 172.21.15.92:0/692507820 shutdown_connections 2023-10-06T10:41:45.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.007+0000 7f2ae7fff700 1 --2- 172.21.15.92:0/692507820 >> v2:172.21.15.124:3300/0 conn(0x7f2ae815b660 0x7f2ae80c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:45.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.007+0000 7f2ae7fff700 1 --2- 172.21.15.92:0/692507820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ae8157410 0x7f2ae80cde60 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:45.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.007+0000 7f2ae7fff700 1 -- 172.21.15.92:0/692507820 >> 172.21.15.92:0/692507820 conn(0x7f2ae806b500 msgr2=0x7f2ae806b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:45.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.007+0000 7f2ae7fff700 1 -- 172.21.15.92:0/692507820 shutdown_connections 2023-10-06T10:41:45.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.007+0000 7f2ae7fff700 1 -- 172.21.15.92:0/692507820 wait complete. 2023-10-06T10:41:45.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.009+0000 7f2ae7fff700 1 Processor -- start 2023-10-06T10:41:45.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.009+0000 7f2ae7fff700 1 -- start start 2023-10-06T10:41:45.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.009+0000 7f2ae7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ae8157040 0x7f2ae814ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:45.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.009+0000 7f2ae7fff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ae8157410 0x7f2ae8168f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:45.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.009+0000 7f2ae7fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ae815b660 0x7f2ae8159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:45.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.009+0000 7f2ae7fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae8168d70 con 0x7f2ae8157040 2023-10-06T10:41:45.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.009+0000 7f2ae7fff700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae8158380 con 0x7f2ae8157410 2023-10-06T10:41:45.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.009+0000 7f2ae7fff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae8157a50 con 0x7f2ae815b660 2023-10-06T10:41:45.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.009+0000 7f2ae6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ae8157040 0x7f2ae814ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:45.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.009+0000 7f2ae6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ae8157040 0x7f2ae814ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57098/0 (socket says 172.21.15.92:57098) 2023-10-06T10:41:45.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.010+0000 7f2ae6ffd700 1 -- 172.21.15.92:0/2023937705 learned_addr learned my addr 172.21.15.92:0/2023937705 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:45.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.010+0000 7f2ade7fc700 1 --2- 172.21.15.92:0/2023937705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ae8157410 0x7f2ae8168f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:45.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.010+0000 7f2ae77fe700 1 --2- 172.21.15.92:0/2023937705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ae815b660 0x7f2ae8159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:45.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.010+0000 7f2ae6ffd700 1 -- 172.21.15.92:0/2023937705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ae815b660 msgr2=0x7f2ae8159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:45.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.010+0000 7f2ae6ffd700 1 --2- 172.21.15.92:0/2023937705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ae815b660 0x7f2ae8159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:45.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.010+0000 7f2ae6ffd700 1 -- 172.21.15.92:0/2023937705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ae8157410 msgr2=0x7f2ae8168f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:45.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.010+0000 7f2ae6ffd700 1 --2- 172.21.15.92:0/2023937705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ae8157410 0x7f2ae8168f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:45.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.010+0000 7f2ae6ffd700 1 -- 172.21.15.92:0/2023937705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad00094f0 con 0x7f2ae8157040 2023-10-06T10:41:45.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.010+0000 7f2ae77fe700 1 --2- 172.21.15.92:0/2023937705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ae815b660 0x7f2ae8159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:45.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.010+0000 7f2ae6ffd700 1 --2- 172.21.15.92:0/2023937705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ae8157040 0x7f2ae814ca00 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80011e0 tx=0x7f2ad8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:45.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.010+0000 7f2adf7fe700 1 -- 172.21.15.92:0/2023937705 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8007300 con 0x7f2ae8157040 2023-10-06T10:41:45.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.010+0000 7f2adf7fe700 1 -- 172.21.15.92:0/2023937705 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad8007460 con 0x7f2ae8157040 2023-10-06T10:41:45.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.010+0000 7f2ae7fff700 1 -- 172.21.15.92:0/2023937705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae8157d30 con 0x7f2ae8157040 2023-10-06T10:41:45.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.010+0000 7f2adf7fe700 1 -- 172.21.15.92:0/2023937705 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8004830 con 0x7f2ae8157040 2023-10-06T10:41:45.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.010+0000 7f2ae7fff700 1 -- 172.21.15.92:0/2023937705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae814cf40 con 0x7f2ae8157040 2023-10-06T10:41:45.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.011+0000 7f2ae7fff700 1 -- 172.21.15.92:0/2023937705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae805f230 con 0x7f2ae8157040 2023-10-06T10:41:45.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.011+0000 7f2adf7fe700 1 -- 172.21.15.92:0/2023937705 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ad8004be0 con 0x7f2ae8157040 2023-10-06T10:41:45.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.012+0000 7f2adf7fe700 1 --2- 172.21.15.92:0/2023937705 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2ac8066f30 0x7f2ac80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:45.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.012+0000 7f2adf7fe700 1 -- 172.21.15.92:0/2023937705 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f2ad80839c0 con 0x7f2ae8157040 2023-10-06T10:41:45.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.012+0000 7f2ade7fc700 1 --2- 172.21.15.92:0/2023937705 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2ac8066f30 0x7f2ac80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:45.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.012+0000 7f2ade7fc700 1 --2- 172.21.15.92:0/2023937705 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2ac8066f30 0x7f2ac80693e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8157880 tx=0x7f2ad0017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:45.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.016+0000 7f2adf7fe700 1 -- 172.21.15.92:0/2023937705 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ad8050d60 con 0x7f2ae8157040 2023-10-06T10:41:45.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.201+0000 7f2ae7fff700 1 -- 172.21.15.92:0/2023937705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2ae80cea70 con 0x7f2ae8157040 2023-10-06T10:41:45.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.201+0000 7f2adf7fe700 1 -- 172.21.15.92:0/2023937705 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v60) v1 ==== 84+0+126 (secure 0 0 0) 0x7f2ad8018020 con 0x7f2ae8157040 2023-10-06T10:41:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:45.203 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T10:41:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:41:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-10-06T10:41:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-10-06T10:41:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:41:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:41:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-10-06T10:41:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-10-06T10:41:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:41:45.205 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T10:41:45.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.203+0000 7f2ae7fff700 1 -- 172.21.15.92:0/2023937705 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2ac8066f30 msgr2=0x7f2ac80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:45.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.203+0000 7f2ae7fff700 1 --2- 172.21.15.92:0/2023937705 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2ac8066f30 0x7f2ac80693e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8157880 tx=0x7f2ad0017000 comp rx=0 tx=0).stop 2023-10-06T10:41:45.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.203+0000 7f2ae7fff700 1 -- 172.21.15.92:0/2023937705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ae8157040 msgr2=0x7f2ae814ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:45.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.203+0000 7f2ae7fff700 1 --2- 172.21.15.92:0/2023937705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ae8157040 0x7f2ae814ca00 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80011e0 tx=0x7f2ad8007e10 comp rx=0 tx=0).stop 2023-10-06T10:41:45.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.203+0000 7f2ae7fff700 1 -- 172.21.15.92:0/2023937705 shutdown_connections 2023-10-06T10:41:45.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.203+0000 7f2ae7fff700 1 --2- 172.21.15.92:0/2023937705 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2ac8066f30 0x7f2ac80693e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:45.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.203+0000 7f2ae7fff700 1 --2- 172.21.15.92:0/2023937705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ae815b660 0x7f2ae8159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:45.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.203+0000 7f2ae7fff700 1 --2- 172.21.15.92:0/2023937705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ae8157040 0x7f2ae814ca00 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:45.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.203+0000 7f2ae7fff700 1 --2- 172.21.15.92:0/2023937705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ae8157410 0x7f2ae8168f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:45.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.203+0000 7f2ae7fff700 1 -- 172.21.15.92:0/2023937705 >> 172.21.15.92:0/2023937705 conn(0x7f2ae806b500 msgr2=0x7f2ae80cd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:45.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.203+0000 7f2ae7fff700 1 -- 172.21.15.92:0/2023937705 shutdown_connections 2023-10-06T10:41:45.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.203+0000 7f2ae7fff700 1 -- 172.21.15.92:0/2023937705 wait complete. 2023-10-06T10:41:45.217 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T10:41:45.217 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T10:41:45.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.332+0000 7f18ed86c700 1 Processor -- start 2023-10-06T10:41:45.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.332+0000 7f18ed86c700 1 -- start start 2023-10-06T10:41:45.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.333+0000 7f18ed86c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18e806ece0 0x7f18e80b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:45.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.333+0000 7f18ed86c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f18e80aede0 0x7f18e80af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:45.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.333+0000 7f18ed86c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e80b2ce0 con 0x7f18e806f0b0 2023-10-06T10:41:45.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.333+0000 7f18ed86c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e806fb40 con 0x7f18e80aede0 2023-10-06T10:41:45.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.333+0000 7f18ed86c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e80b0410 con 0x7f18e806ece0 2023-10-06T10:41:45.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.333+0000 7f18e77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f18e80aede0 0x7f18e80af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:45.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.333+0000 7f18de7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f18e806f0b0 0x7f18e80b08a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33206/0 (socket says 172.21.15.92:33206) 2023-10-06T10:41:45.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.333+0000 7f18de7fc700 1 -- 172.21.15.92:0/1204607878 learned_addr learned my addr 172.21.15.92:0/1204607878 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:45.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.333+0000 7f18e77fe700 1 --2- 172.21.15.92:0/1204607878 >> v2:172.21.15.92:3300/0 conn(0x7f18e80aede0 0x7f18e80af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57112/0 (socket says 172.21.15.92:57112) 2023-10-06T10:41:45.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.333+0000 7f18e6ffd700 1 --2- 172.21.15.92:0/1204607878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18e806ece0 0x7f18e80b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:45.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.333+0000 7f18e77fe700 1 -- 172.21.15.92:0/1204607878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18e806ece0 msgr2=0x7f18e80b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:45.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.333+0000 7f18e77fe700 1 --2- 172.21.15.92:0/1204607878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18e806ece0 0x7f18e80b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:45.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.333+0000 7f18e77fe700 1 -- 172.21.15.92:0/1204607878 >> v1:172.21.15.92:6789/0 conn(0x7f18e806f0b0 legacy=0x7f18e80b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:45.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.333+0000 7f18e77fe700 1 -- 172.21.15.92:0/1204607878 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e80af780 con 0x7f18e80aede0 2023-10-06T10:41:45.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.334+0000 7f18e77fe700 1 --2- 172.21.15.92:0/1204607878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18e80aede0 0x7f18e80af240 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009e60 tx=0x7f18e0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba56695314e2b751 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:45.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.334+0000 7f18e67fc700 1 -- 172.21.15.92:0/1204607878 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e000e070 con 0x7f18e80aede0 2023-10-06T10:41:45.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.334+0000 7f18e67fc700 1 -- 172.21.15.92:0/1204607878 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e000bb30 con 0x7f18e80aede0 2023-10-06T10:41:45.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.334+0000 7f18e67fc700 1 -- 172.21.15.92:0/1204607878 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e0012430 con 0x7f18e80aede0 2023-10-06T10:41:45.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.334+0000 7f18ed86c700 1 -- 172.21.15.92:0/1204607878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18e80aede0 msgr2=0x7f18e80af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:45.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.334+0000 7f18ed86c700 1 --2- 172.21.15.92:0/1204607878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18e80aede0 0x7f18e80af240 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009e60 tx=0x7f18e0009780 comp rx=0 tx=0).stop 2023-10-06T10:41:45.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.334+0000 7f18ed86c700 1 -- 172.21.15.92:0/1204607878 shutdown_connections 2023-10-06T10:41:45.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.334+0000 7f18ed86c700 1 --2- 172.21.15.92:0/1204607878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18e806ece0 0x7f18e80b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:45.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.334+0000 7f18ed86c700 1 --2- 172.21.15.92:0/1204607878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18e80aede0 0x7f18e80af240 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:45.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.334+0000 7f18ed86c700 1 -- 172.21.15.92:0/1204607878 >> 172.21.15.92:0/1204607878 conn(0x7f18e80682f0 msgr2=0x7f18e80686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:45.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.334+0000 7f18ed86c700 1 -- 172.21.15.92:0/1204607878 shutdown_connections 2023-10-06T10:41:45.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.334+0000 7f18ed86c700 1 -- 172.21.15.92:0/1204607878 wait complete. 2023-10-06T10:41:45.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.335+0000 7f18ed86c700 1 Processor -- start 2023-10-06T10:41:45.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.336+0000 7f18ed86c700 1 -- start start 2023-10-06T10:41:45.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.336+0000 7f18ed86c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18e806ece0 0x7f18e81594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:45.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.336+0000 7f18ed86c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18e806f0b0 0x7f18e80ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:45.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.336+0000 7f18ed86c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18e80aede0 0x7f18e814b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:45.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.336+0000 7f18ed86c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e8159d90 con 0x7f18e806f0b0 2023-10-06T10:41:45.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.336+0000 7f18ed86c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e815a070 con 0x7f18e806ece0 2023-10-06T10:41:45.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.336+0000 7f18ed86c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e815a350 con 0x7f18e80aede0 2023-10-06T10:41:45.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.336+0000 7f18de7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18e806f0b0 0x7f18e80ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:45.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.336+0000 7f18de7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18e806f0b0 0x7f18e80ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57124/0 (socket says 172.21.15.92:57124) 2023-10-06T10:41:45.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.336+0000 7f18de7fc700 1 -- 172.21.15.92:0/4240029044 learned_addr learned my addr 172.21.15.92:0/4240029044 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:45.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.336+0000 7f18e6ffd700 1 --2- 172.21.15.92:0/4240029044 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18e806ece0 0x7f18e81594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:45.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.337+0000 7f18e77fe700 1 --2- 172.21.15.92:0/4240029044 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18e80aede0 0x7f18e814b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:45.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.337+0000 7f18de7fc700 1 -- 172.21.15.92:0/4240029044 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18e80aede0 msgr2=0x7f18e814b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:45.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.337+0000 7f18de7fc700 1 --2- 172.21.15.92:0/4240029044 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18e80aede0 0x7f18e814b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:45.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.337+0000 7f18de7fc700 1 -- 172.21.15.92:0/4240029044 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18e806ece0 msgr2=0x7f18e81594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:45.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.337+0000 7f18de7fc700 1 --2- 172.21.15.92:0/4240029044 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18e806ece0 0x7f18e81594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:45.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.337+0000 7f18de7fc700 1 -- 172.21.15.92:0/4240029044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e0009a20 con 0x7f18e806f0b0 2023-10-06T10:41:45.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.337+0000 7f18de7fc700 1 --2- 172.21.15.92:0/4240029044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18e806f0b0 0x7f18e80ce210 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f18d000c760 tx=0x7f18d0007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:45.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.337+0000 7f18df7fe700 1 -- 172.21.15.92:0/4240029044 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d0005b30 con 0x7f18e806f0b0 2023-10-06T10:41:45.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.337+0000 7f18df7fe700 1 -- 172.21.15.92:0/4240029044 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d0005cb0 con 0x7f18e806f0b0 2023-10-06T10:41:45.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.337+0000 7f18ed86c700 1 -- 172.21.15.92:0/4240029044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e815a630 con 0x7f18e806f0b0 2023-10-06T10:41:45.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.337+0000 7f18df7fe700 1 -- 172.21.15.92:0/4240029044 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d0007320 con 0x7f18e806f0b0 2023-10-06T10:41:45.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.337+0000 7f18ed86c700 1 -- 172.21.15.92:0/4240029044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e80ce750 con 0x7f18e806f0b0 2023-10-06T10:41:45.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.338+0000 7f18ed86c700 1 -- 172.21.15.92:0/4240029044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e80ce970 con 0x7f18e806f0b0 2023-10-06T10:41:45.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.338+0000 7f18df7fe700 1 -- 172.21.15.92:0/4240029044 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f18d001b3b0 con 0x7f18e806f0b0 2023-10-06T10:41:45.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.339+0000 7f18df7fe700 1 --2- 172.21.15.92:0/4240029044 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f18c8066f30 0x7f18c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:45.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.339+0000 7f18df7fe700 1 -- 172.21.15.92:0/4240029044 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f18d00839b0 con 0x7f18e806f0b0 2023-10-06T10:41:45.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.339+0000 7f18e6ffd700 1 --2- 172.21.15.92:0/4240029044 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f18c8066f30 0x7f18c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:45.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.339+0000 7f18e6ffd700 1 --2- 172.21.15.92:0/4240029044 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f18c8066f30 0x7f18c80693e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f18d8006fd0 tx=0x7f18d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:45.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.343+0000 7f18df7fe700 1 -- 172.21.15.92:0/4240029044 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18d0050cd0 con 0x7f18e806f0b0 2023-10-06T10:41:45.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:45.527+0000 7f18ed86c700 1 -- 172.21.15.92:0/4240029044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f18e8064a90 con 0x7f18e806f0b0 2023-10-06T10:41:46.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.511+0000 7f18df7fe700 1 -- 172.21.15.92:0/4240029044 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v61) v1 ==== 138+0+0 (secure 0 0 0) 0x7f18d0017020 con 0x7f18e806f0b0 2023-10-06T10:41:46.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.514+0000 7f18ed86c700 1 -- 172.21.15.92:0/4240029044 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f18c8066f30 msgr2=0x7f18c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:46.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.514+0000 7f18ed86c700 1 --2- 172.21.15.92:0/4240029044 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f18c8066f30 0x7f18c80693e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f18d8006fd0 tx=0x7f18d8008040 comp rx=0 tx=0).stop 2023-10-06T10:41:46.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.514+0000 7f18ed86c700 1 -- 172.21.15.92:0/4240029044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18e806f0b0 msgr2=0x7f18e80ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:46.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.514+0000 7f18ed86c700 1 --2- 172.21.15.92:0/4240029044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18e806f0b0 0x7f18e80ce210 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f18d000c760 tx=0x7f18d0007ef0 comp rx=0 tx=0).stop 2023-10-06T10:41:46.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.514+0000 7f18ed86c700 1 -- 172.21.15.92:0/4240029044 shutdown_connections 2023-10-06T10:41:46.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.514+0000 7f18ed86c700 1 --2- 172.21.15.92:0/4240029044 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f18c8066f30 0x7f18c80693e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:46.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.514+0000 7f18ed86c700 1 --2- 172.21.15.92:0/4240029044 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18e80aede0 0x7f18e814b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:46.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.514+0000 7f18ed86c700 1 --2- 172.21.15.92:0/4240029044 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18e806ece0 0x7f18e81594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:46.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.514+0000 7f18ed86c700 1 --2- 172.21.15.92:0/4240029044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18e806f0b0 0x7f18e80ce210 secure :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f18d000c760 tx=0x7f18d0007ef0 comp rx=0 tx=0).stop 2023-10-06T10:41:46.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.514+0000 7f18ed86c700 1 -- 172.21.15.92:0/4240029044 >> 172.21.15.92:0/4240029044 conn(0x7f18e80682f0 msgr2=0x7f18e8156a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:46.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.514+0000 7f18ed86c700 1 -- 172.21.15.92:0/4240029044 shutdown_connections 2023-10-06T10:41:46.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.514+0000 7f18ed86c700 1 -- 172.21.15.92:0/4240029044 wait complete. 2023-10-06T10:41:46.518 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-10-06T10:41:46.530 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T10:41:46.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.651+0000 7ff7336f0700 1 Processor -- start 2023-10-06T10:41:46.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.651+0000 7ff7336f0700 1 -- start start 2023-10-06T10:41:46.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.652+0000 7ff7336f0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff72c1594d0 0x7ff72c0cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:46.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.652+0000 7ff7336f0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff72c1598a0 0x7ff72c0cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:46.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.652+0000 7ff7336f0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff72c0cf310 con 0x7ff72c0ce7e0 2023-10-06T10:41:46.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.652+0000 7ff7336f0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff72c0cf5d0 con 0x7ff72c1598a0 2023-10-06T10:41:46.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.652+0000 7ff7336f0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c0cf8b0 con 0x7ff72c1594d0 2023-10-06T10:41:46.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.652+0000 7ff730c8b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff72c1598a0 0x7ff72c0cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:46.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.652+0000 7ff730c8b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff72c1598a0 0x7ff72c0cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57132/0 (socket says 172.21.15.92:57132) 2023-10-06T10:41:46.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.652+0000 7ff731c8d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff72c0ce7e0 0x7ff72c0cec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33208/0 (socket says 172.21.15.92:33208) 2023-10-06T10:41:46.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.652+0000 7ff730c8b700 1 -- 172.21.15.92:0/1869175073 learned_addr learned my addr 172.21.15.92:0/1869175073 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:46.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.652+0000 7ff73148c700 1 --2- 172.21.15.92:0/1869175073 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff72c1594d0 0x7ff72c0cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:46.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.652+0000 7ff730c8b700 1 -- 172.21.15.92:0/1869175073 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff72c1594d0 msgr2=0x7ff72c0cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:46.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.652+0000 7ff730c8b700 1 --2- 172.21.15.92:0/1869175073 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff72c1594d0 0x7ff72c0cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:46.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.652+0000 7ff730c8b700 1 -- 172.21.15.92:0/1869175073 >> v1:172.21.15.92:6789/0 conn(0x7ff72c0ce7e0 legacy=0x7ff72c0cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:46.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.652+0000 7ff72bfff700 1 -- 172.21.15.92:0/1869175073 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 924747108 0 0) 0x7ff72c0cf310 con 0x7ff72c0ce7e0 2023-10-06T10:41:46.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.652+0000 7ff730c8b700 1 -- 172.21.15.92:0/1869175073 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72c0cfb90 con 0x7ff72c1598a0 2023-10-06T10:41:46.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.652+0000 7ff730c8b700 1 --2- 172.21.15.92:0/1869175073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff72c1598a0 0x7ff72c0cbaa0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff7200099b0 tx=0x7ff720009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe2b32e193bbb06 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:46.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.652+0000 7ff72bfff700 1 -- 172.21.15.92:0/1869175073 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720009ea0 con 0x7ff72c1598a0 2023-10-06T10:41:46.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.652+0000 7ff72bfff700 1 -- 172.21.15.92:0/1869175073 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72000b860 con 0x7ff72c1598a0 2023-10-06T10:41:46.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.652+0000 7ff72bfff700 1 -- 172.21.15.92:0/1869175073 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72000bc00 con 0x7ff72c1598a0 2023-10-06T10:41:46.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.653+0000 7ff7336f0700 1 -- 172.21.15.92:0/1869175073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff72c1598a0 msgr2=0x7ff72c0cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:46.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.653+0000 7ff7336f0700 1 --2- 172.21.15.92:0/1869175073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff72c1598a0 0x7ff72c0cbaa0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff7200099b0 tx=0x7ff720009250 comp rx=0 tx=0).stop 2023-10-06T10:41:46.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.653+0000 7ff7336f0700 1 -- 172.21.15.92:0/1869175073 shutdown_connections 2023-10-06T10:41:46.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.653+0000 7ff7336f0700 1 --2- 172.21.15.92:0/1869175073 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff72c1594d0 0x7ff72c0cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:46.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.653+0000 7ff7336f0700 1 --2- 172.21.15.92:0/1869175073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff72c1598a0 0x7ff72c0cbaa0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:46.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.653+0000 7ff7336f0700 1 -- 172.21.15.92:0/1869175073 >> 172.21.15.92:0/1869175073 conn(0x7ff72c089e50 msgr2=0x7ff72c099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:46.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.653+0000 7ff7336f0700 1 -- 172.21.15.92:0/1869175073 shutdown_connections 2023-10-06T10:41:46.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.653+0000 7ff7336f0700 1 -- 172.21.15.92:0/1869175073 wait complete. 2023-10-06T10:41:46.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.654+0000 7ff7336f0700 1 Processor -- start 2023-10-06T10:41:46.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.654+0000 7ff7336f0700 1 -- start start 2023-10-06T10:41:46.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.655+0000 7ff7336f0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff72c0ce7e0 0x7ff72c14b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:46.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.655+0000 7ff7336f0700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff72c1594d0 0x7ff72c157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:46.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.655+0000 7ff7336f0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff72c1598a0 0x7ff72c157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:46.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.655+0000 7ff7336f0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c0cbb90 con 0x7ff72c0ce7e0 2023-10-06T10:41:46.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.655+0000 7ff7336f0700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c15c0a0 con 0x7ff72c1594d0 2023-10-06T10:41:46.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.655+0000 7ff7336f0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c15c530 con 0x7ff72c1598a0 2023-10-06T10:41:46.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.655+0000 7ff73148c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff72c0ce7e0 0x7ff72c14b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:46.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.655+0000 7ff73148c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff72c0ce7e0 0x7ff72c14b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57142/0 (socket says 172.21.15.92:57142) 2023-10-06T10:41:46.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.655+0000 7ff73148c700 1 -- 172.21.15.92:0/13472862 learned_addr learned my addr 172.21.15.92:0/13472862 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:46.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.655+0000 7ff731c8d700 1 --2- 172.21.15.92:0/13472862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff72c1598a0 0x7ff72c157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:46.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.655+0000 7ff73148c700 1 -- 172.21.15.92:0/13472862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff72c1598a0 msgr2=0x7ff72c157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:46.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.655+0000 7ff73148c700 1 --2- 172.21.15.92:0/13472862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff72c1598a0 0x7ff72c157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:46.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.655+0000 7ff73148c700 1 -- 172.21.15.92:0/13472862 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff72c1594d0 msgr2=0x7ff72c157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:46.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.655+0000 7ff730c8b700 1 --2- 172.21.15.92:0/13472862 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff72c1594d0 0x7ff72c157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:46.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.655+0000 7ff73148c700 1 --2- 172.21.15.92:0/13472862 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff72c1594d0 0x7ff72c157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:46.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.655+0000 7ff73148c700 1 -- 172.21.15.92:0/13472862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7200094f0 con 0x7ff72c0ce7e0 2023-10-06T10:41:46.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.655+0000 7ff731c8d700 1 --2- 172.21.15.92:0/13472862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff72c1598a0 0x7ff72c157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:46.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.655+0000 7ff730c8b700 1 --2- 172.21.15.92:0/13472862 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff72c1594d0 0x7ff72c157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:46.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.656+0000 7ff73148c700 1 --2- 172.21.15.92:0/13472862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff72c0ce7e0 0x7ff72c14b4a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff72400ec80 tx=0x7ff7240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:46.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.656+0000 7ff7297fa700 1 -- 172.21.15.92:0/13472862 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72400c9d0 con 0x7ff72c0ce7e0 2023-10-06T10:41:46.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.656+0000 7ff7297fa700 1 -- 172.21.15.92:0/13472862 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72400cb30 con 0x7ff72c0ce7e0 2023-10-06T10:41:46.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.656+0000 7ff7336f0700 1 -- 172.21.15.92:0/13472862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff72c15b1b0 con 0x7ff72c0ce7e0 2023-10-06T10:41:46.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.656+0000 7ff7297fa700 1 -- 172.21.15.92:0/13472862 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff724013470 con 0x7ff72c0ce7e0 2023-10-06T10:41:46.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.656+0000 7ff7336f0700 1 -- 172.21.15.92:0/13472862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff72c15b770 con 0x7ff72c0ce7e0 2023-10-06T10:41:46.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.657+0000 7ff7336f0700 1 -- 172.21.15.92:0/13472862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72c15c380 con 0x7ff72c0ce7e0 2023-10-06T10:41:46.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.657+0000 7ff7297fa700 1 -- 172.21.15.92:0/13472862 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff724013820 con 0x7ff72c0ce7e0 2023-10-06T10:41:46.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.658+0000 7ff7297fa700 1 --2- 172.21.15.92:0/13472862 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff718066f60 0x7ff718069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:46.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.658+0000 7ff7297fa700 1 -- 172.21.15.92:0/13472862 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5896+0+0 (secure 0 0 0) 0x7ff724084960 con 0x7ff72c0ce7e0 2023-10-06T10:41:46.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.658+0000 7ff730c8b700 1 --2- 172.21.15.92:0/13472862 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff718066f60 0x7ff718069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:46.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.658+0000 7ff730c8b700 1 --2- 172.21.15.92:0/13472862 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff718066f60 0x7ff718069410 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff7200099b0 tx=0x7ff720017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:46.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.662+0000 7ff7297fa700 1 -- 172.21.15.92:0/13472862 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff724051b10 con 0x7ff72c0ce7e0 2023-10-06T10:41:46.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:46.844+0000 7ff7336f0700 1 -- 172.21.15.92:0/13472862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff72c064a90 con 0x7ff72c0ce7e0 2023-10-06T10:41:47.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.514+0000 7ff7297fa700 1 -- 172.21.15.92:0/13472862 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v62) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff724055130 con 0x7ff72c0ce7e0 2023-10-06T10:41:47.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.519+0000 7ff7336f0700 1 -- 172.21.15.92:0/13472862 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff718066f60 msgr2=0x7ff718069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:47.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.519+0000 7ff7336f0700 1 --2- 172.21.15.92:0/13472862 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff718066f60 0x7ff718069410 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff7200099b0 tx=0x7ff720017000 comp rx=0 tx=0).stop 2023-10-06T10:41:47.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.519+0000 7ff7336f0700 1 -- 172.21.15.92:0/13472862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff72c0ce7e0 msgr2=0x7ff72c14b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:47.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.519+0000 7ff7336f0700 1 --2- 172.21.15.92:0/13472862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff72c0ce7e0 0x7ff72c14b4a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff72400ec80 tx=0x7ff7240091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:47.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.519+0000 7ff7336f0700 1 -- 172.21.15.92:0/13472862 shutdown_connections 2023-10-06T10:41:47.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.519+0000 7ff7336f0700 1 --2- 172.21.15.92:0/13472862 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff718066f60 0x7ff718069410 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:47.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.519+0000 7ff7336f0700 1 --2- 172.21.15.92:0/13472862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff72c1598a0 0x7ff72c157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:47.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.519+0000 7ff7336f0700 1 --2- 172.21.15.92:0/13472862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff72c0ce7e0 0x7ff72c14b4a0 secure :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff72400ec80 tx=0x7ff7240091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:47.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.519+0000 7ff7336f0700 1 --2- 172.21.15.92:0/13472862 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff72c1594d0 0x7ff72c157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:47.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.519+0000 7ff7336f0700 1 -- 172.21.15.92:0/13472862 >> 172.21.15.92:0/13472862 conn(0x7ff72c089e50 msgr2=0x7ff72c155830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:47.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.520+0000 7ff7336f0700 1 -- 172.21.15.92:0/13472862 shutdown_connections 2023-10-06T10:41:47.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.522+0000 7ff7336f0700 1 -- 172.21.15.92:0/13472862 wait complete. 2023-10-06T10:41:47.529 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-10-06T10:41:47.545 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T10:41:47.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.663+0000 7f233ec39700 1 Processor -- start 2023-10-06T10:41:47.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.663+0000 7f233ec39700 1 -- start start 2023-10-06T10:41:47.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.663+0000 7f233ec39700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f23381598a0 0x7f23380ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:47.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.663+0000 7f233ec39700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23380cbc70 0x7f23380ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:47.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.663+0000 7f233ec39700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23380cf5d0 con 0x7f23381594d0 2023-10-06T10:41:47.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.664+0000 7f233ec39700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23380cf8b0 con 0x7f23381598a0 2023-10-06T10:41:47.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.664+0000 7f233ec39700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23380cfb90 con 0x7f23380cbc70 2023-10-06T10:41:47.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.664+0000 7f232ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f23381598a0 0x7f23380ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:47.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.664+0000 7f232ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f23381598a0 0x7f23380ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57156/0 (socket says 172.21.15.92:57156) 2023-10-06T10:41:47.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.664+0000 7f232ffff700 1 -- 172.21.15.92:0/635310022 learned_addr learned my addr 172.21.15.92:0/635310022 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:47.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.664+0000 7f233d1d6700 1 --2- 172.21.15.92:0/635310022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23380cbc70 0x7f23380ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:47.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.664+0000 7f232ffff700 1 -- 172.21.15.92:0/635310022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23380cbc70 msgr2=0x7f23380ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:47.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.664+0000 7f232ffff700 1 --2- 172.21.15.92:0/635310022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23380cbc70 0x7f23380ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:47.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.664+0000 7f232ffff700 1 -- 172.21.15.92:0/635310022 >> v1:172.21.15.124:6789/0 conn(0x7f23381594d0 legacy=0x7f23380cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:47.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.664+0000 7f232ffff700 1 -- 172.21.15.92:0/635310022 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23380cfe70 con 0x7f23381598a0 2023-10-06T10:41:47.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.664+0000 7f232ffff700 1 --2- 172.21.15.92:0/635310022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23381598a0 0x7f23380ce7e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f23280099b0 tx=0x7f2328009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83de3250d1fe1cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:47.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.664+0000 7f232f7fe700 1 -- 172.21.15.92:0/635310022 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328009ea0 con 0x7f23381598a0 2023-10-06T10:41:47.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.664+0000 7f232f7fe700 1 -- 172.21.15.92:0/635310022 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232800b860 con 0x7f23381598a0 2023-10-06T10:41:47.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.664+0000 7f232f7fe700 1 -- 172.21.15.92:0/635310022 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232800bc00 con 0x7f23381598a0 2023-10-06T10:41:47.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.665+0000 7f233ec39700 1 -- 172.21.15.92:0/635310022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23381598a0 msgr2=0x7f23380ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:47.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.665+0000 7f233ec39700 1 --2- 172.21.15.92:0/635310022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23381598a0 0x7f23380ce7e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f23280099b0 tx=0x7f2328009250 comp rx=0 tx=0).stop 2023-10-06T10:41:47.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.665+0000 7f233ec39700 1 -- 172.21.15.92:0/635310022 shutdown_connections 2023-10-06T10:41:47.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.665+0000 7f233ec39700 1 --2- 172.21.15.92:0/635310022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23380cbc70 0x7f23380ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:47.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.665+0000 7f233ec39700 1 --2- 172.21.15.92:0/635310022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23381598a0 0x7f23380ce7e0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:47.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.665+0000 7f233ec39700 1 -- 172.21.15.92:0/635310022 >> 172.21.15.92:0/635310022 conn(0x7f2338089e50 msgr2=0x7f2338099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:47.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.665+0000 7f233ec39700 1 -- 172.21.15.92:0/635310022 shutdown_connections 2023-10-06T10:41:47.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.665+0000 7f233ec39700 1 -- 172.21.15.92:0/635310022 wait complete. 2023-10-06T10:41:47.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.666+0000 7f233ec39700 1 Processor -- start 2023-10-06T10:41:47.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.666+0000 7f233ec39700 1 -- start start 2023-10-06T10:41:47.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.667+0000 7f233ec39700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23380cbc70 0x7f233815b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:47.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.667+0000 7f233ec39700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23381594d0 0x7f2338169a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:47.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.667+0000 7f233ec39700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23381598a0 0x7f2338153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:47.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.667+0000 7f233ec39700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233815b670 con 0x7f23381598a0 2023-10-06T10:41:47.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.667+0000 7f233ec39700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233814ffb0 con 0x7f23381594d0 2023-10-06T10:41:47.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.667+0000 7f233ec39700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233815aca0 con 0x7f23380cbc70 2023-10-06T10:41:47.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.667+0000 7f233d1d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23381598a0 0x7f2338153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:47.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.667+0000 7f233d1d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23381598a0 0x7f2338153540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57166/0 (socket says 172.21.15.92:57166) 2023-10-06T10:41:47.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.667+0000 7f233d1d6700 1 -- 172.21.15.92:0/94863935 learned_addr learned my addr 172.21.15.92:0/94863935 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:47.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.667+0000 7f233d1d6700 1 -- 172.21.15.92:0/94863935 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23380cbc70 msgr2=0x7f233815b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:47.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.667+0000 7f232ffff700 1 --2- 172.21.15.92:0/94863935 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23381594d0 0x7f2338169a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:47.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.667+0000 7f233d1d6700 1 --2- 172.21.15.92:0/94863935 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23380cbc70 0x7f233815b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:47.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.667+0000 7f233d1d6700 1 -- 172.21.15.92:0/94863935 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23381594d0 msgr2=0x7f2338169a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:47.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.667+0000 7f233d1d6700 1 --2- 172.21.15.92:0/94863935 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23381594d0 0x7f2338169a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:47.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.667+0000 7f233d1d6700 1 -- 172.21.15.92:0/94863935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23280094f0 con 0x7f23381598a0 2023-10-06T10:41:47.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.667+0000 7f233d1d6700 1 --2- 172.21.15.92:0/94863935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23381598a0 0x7f2338153540 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f233000ea80 tx=0x7f23300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:47.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.668+0000 7f232cff9700 1 -- 172.21.15.92:0/94863935 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233000c920 con 0x7f23381598a0 2023-10-06T10:41:47.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.668+0000 7f232cff9700 1 -- 172.21.15.92:0/94863935 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233000ca80 con 0x7f23381598a0 2023-10-06T10:41:47.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.668+0000 7f233ec39700 1 -- 172.21.15.92:0/94863935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f233815a970 con 0x7f23381598a0 2023-10-06T10:41:47.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.668+0000 7f232cff9700 1 -- 172.21.15.92:0/94863935 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233000ce40 con 0x7f23381598a0 2023-10-06T10:41:47.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.668+0000 7f233ec39700 1 -- 172.21.15.92:0/94863935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f233815c500 con 0x7f23381598a0 2023-10-06T10:41:47.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.669+0000 7f233ec39700 1 -- 172.21.15.92:0/94863935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2338064a90 con 0x7f23381598a0 2023-10-06T10:41:47.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.669+0000 7f232cff9700 1 -- 172.21.15.92:0/94863935 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2330013680 con 0x7f23381598a0 2023-10-06T10:41:47.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.669+0000 7f232cff9700 1 --2- 172.21.15.92:0/94863935 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2324066f30 0x7f23240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:47.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.669+0000 7f232cff9700 1 -- 172.21.15.92:0/94863935 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6278+0+0 (secure 0 0 0) 0x7f2330084e70 con 0x7f23381598a0 2023-10-06T10:41:47.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.669+0000 7f233c9d5700 1 --2- 172.21.15.92:0/94863935 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2324066f30 0x7f23240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:47.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.670+0000 7f233c9d5700 1 --2- 172.21.15.92:0/94863935 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2324066f30 0x7f23240693e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2334001d90 tx=0x7f2334007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:47.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.674+0000 7f232cff9700 1 -- 172.21.15.92:0/94863935 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2330051ea0 con 0x7f23381598a0 2023-10-06T10:41:47.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:47.896+0000 7f233ec39700 1 -- 172.21.15.92:0/94863935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f23380cf260 con 0x7f23381598a0 2023-10-06T10:41:48.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.532+0000 7f232cff9700 1 -- 172.21.15.92:0/94863935 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v46) v1 ==== 174+0+0 (secure 0 0 0) 0x7f23300170c0 con 0x7f23381598a0 2023-10-06T10:41:48.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.536+0000 7f233ec39700 1 -- 172.21.15.92:0/94863935 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2324066f30 msgr2=0x7f23240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:48.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.536+0000 7f233ec39700 1 --2- 172.21.15.92:0/94863935 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2324066f30 0x7f23240693e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2334001d90 tx=0x7f2334007ec0 comp rx=0 tx=0).stop 2023-10-06T10:41:48.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.536+0000 7f233ec39700 1 -- 172.21.15.92:0/94863935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23381598a0 msgr2=0x7f2338153540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:48.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.536+0000 7f233ec39700 1 --2- 172.21.15.92:0/94863935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23381598a0 0x7f2338153540 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f233000ea80 tx=0x7f23300091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:48.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.536+0000 7f233ec39700 1 -- 172.21.15.92:0/94863935 shutdown_connections 2023-10-06T10:41:48.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.536+0000 7f233ec39700 1 --2- 172.21.15.92:0/94863935 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2324066f30 0x7f23240693e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:48.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.536+0000 7f233ec39700 1 --2- 172.21.15.92:0/94863935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23381598a0 0x7f2338153540 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:48.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.536+0000 7f233ec39700 1 --2- 172.21.15.92:0/94863935 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23380cbc70 0x7f233815b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:48.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.536+0000 7f233ec39700 1 --2- 172.21.15.92:0/94863935 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23381594d0 0x7f2338169a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:48.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.536+0000 7f233ec39700 1 -- 172.21.15.92:0/94863935 >> 172.21.15.92:0/94863935 conn(0x7f2338089e50 msgr2=0x7f2338099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:48.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.536+0000 7f233ec39700 1 -- 172.21.15.92:0/94863935 shutdown_connections 2023-10-06T10:41:48.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.536+0000 7f233ec39700 1 -- 172.21.15.92:0/94863935 wait complete. 2023-10-06T10:41:48.540 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 10 and data pool 11 2023-10-06T10:41:48.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:41:48.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.665+0000 7f4323b19700 1 Processor -- start 2023-10-06T10:41:48.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.665+0000 7f4323b19700 1 -- start start 2023-10-06T10:41:48.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.666+0000 7f4323b19700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f431c0ad320 0x7f431c0ac230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:48.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.666+0000 7f4323b19700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f431c0ad6f0 0x7f431c0685c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:48.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.666+0000 7f4323b19700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f431c068b00 0x7f431c15d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:48.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.666+0000 7f4323b19700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f431c068f80 con 0x7f431c068b00 2023-10-06T10:41:48.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.666+0000 7f4323b19700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f431c159f40 con 0x7f431c0ad320 2023-10-06T10:41:48.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.666+0000 7f4323b19700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431c15a220 con 0x7f431c0ad6f0 2023-10-06T10:41:48.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.666+0000 7f43220b6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f431c068b00 0x7f431c15d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:48.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.667+0000 7f43220b6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f431c068b00 0x7f431c15d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57180/0 (socket says 172.21.15.92:57180) 2023-10-06T10:41:48.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.667+0000 7f43220b6700 1 -- 172.21.15.92:0/480312627 learned_addr learned my addr 172.21.15.92:0/480312627 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:48.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.667+0000 7f43210b4700 1 --2- 172.21.15.92:0/480312627 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f431c0ad6f0 0x7f431c0685c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:48.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.667+0000 7f43218b5700 1 --2- 172.21.15.92:0/480312627 >> v2:172.21.15.124:3300/0 conn(0x7f431c0ad320 0x7f431c0ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:48.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.667+0000 7f43220b6700 1 -- 172.21.15.92:0/480312627 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f431c0ad6f0 msgr2=0x7f431c0685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:48.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.667+0000 7f43220b6700 1 --2- 172.21.15.92:0/480312627 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f431c0ad6f0 0x7f431c0685c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:48.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.667+0000 7f43220b6700 1 -- 172.21.15.92:0/480312627 >> v2:172.21.15.124:3300/0 conn(0x7f431c0ad320 msgr2=0x7f431c0ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:48.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.667+0000 7f43220b6700 1 --2- 172.21.15.92:0/480312627 >> v2:172.21.15.124:3300/0 conn(0x7f431c0ad320 0x7f431c0ac230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:48.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.667+0000 7f43220b6700 1 -- 172.21.15.92:0/480312627 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c15a500 con 0x7f431c068b00 2023-10-06T10:41:48.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.667+0000 7f43220b6700 1 --2- 172.21.15.92:0/480312627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f431c068b00 0x7f431c15d110 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f431c063db0 tx=0x7f4314009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283c624307bd5cf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:48.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.667+0000 7f43208b3700 1 -- 172.21.15.92:0/480312627 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431400e070 con 0x7f431c068b00 2023-10-06T10:41:48.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.667+0000 7f43208b3700 1 -- 172.21.15.92:0/480312627 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431400ba50 con 0x7f431c068b00 2023-10-06T10:41:48.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.667+0000 7f43208b3700 1 -- 172.21.15.92:0/480312627 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43140092e0 con 0x7f431c068b00 2023-10-06T10:41:48.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.668+0000 7f4323b19700 1 -- 172.21.15.92:0/480312627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f431c068b00 msgr2=0x7f431c15d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:48.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.668+0000 7f4323b19700 1 --2- 172.21.15.92:0/480312627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f431c068b00 0x7f431c15d110 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f431c063db0 tx=0x7f4314009840 comp rx=0 tx=0).stop 2023-10-06T10:41:48.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.668+0000 7f4323b19700 1 -- 172.21.15.92:0/480312627 shutdown_connections 2023-10-06T10:41:48.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.668+0000 7f4323b19700 1 --2- 172.21.15.92:0/480312627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f431c068b00 0x7f431c15d110 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:48.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.668+0000 7f4323b19700 1 --2- 172.21.15.92:0/480312627 >> v2:172.21.15.124:3300/0 conn(0x7f431c0ad320 0x7f431c0ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:48.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.668+0000 7f4323b19700 1 --2- 172.21.15.92:0/480312627 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f431c0ad6f0 0x7f431c0685c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:48.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.668+0000 7f4323b19700 1 -- 172.21.15.92:0/480312627 >> 172.21.15.92:0/480312627 conn(0x7f431c06dcd0 msgr2=0x7f431c06e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:48.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.668+0000 7f4323b19700 1 -- 172.21.15.92:0/480312627 shutdown_connections 2023-10-06T10:41:48.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.668+0000 7f4323b19700 1 -- 172.21.15.92:0/480312627 wait complete. 2023-10-06T10:41:48.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.669+0000 7f4323b19700 1 Processor -- start 2023-10-06T10:41:48.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.669+0000 7f4323b19700 1 -- start start 2023-10-06T10:41:48.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.670+0000 7f4323b19700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f431c068b00 0x7f431c0b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:48.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.670+0000 7f4323b19700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f431c0ad320 0x7f431c1502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:48.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.670+0000 7f4323b19700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f431c0ad6f0 0x7f431c1582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:48.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.670+0000 7f4323b19700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c0b2150 con 0x7f431c0ad320 2023-10-06T10:41:48.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.670+0000 7f4323b19700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c0b00f0 con 0x7f431c068b00 2023-10-06T10:41:48.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.670+0000 7f4323b19700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1507f0 con 0x7f431c0ad6f0 2023-10-06T10:41:48.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.670+0000 7f43210b4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f431c0ad320 0x7f431c1502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:48.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.670+0000 7f43210b4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f431c0ad320 0x7f431c1502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57194/0 (socket says 172.21.15.92:57194) 2023-10-06T10:41:48.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.670+0000 7f43210b4700 1 -- 172.21.15.92:0/3968988823 learned_addr learned my addr 172.21.15.92:0/3968988823 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:48.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.670+0000 7f43218b5700 1 --2- 172.21.15.92:0/3968988823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f431c068b00 0x7f431c0b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:48.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.670+0000 7f43220b6700 1 --2- 172.21.15.92:0/3968988823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f431c0ad6f0 0x7f431c1582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:48.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.670+0000 7f43210b4700 1 -- 172.21.15.92:0/3968988823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f431c0ad6f0 msgr2=0x7f431c1582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:48.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.670+0000 7f43210b4700 1 --2- 172.21.15.92:0/3968988823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f431c0ad6f0 0x7f431c1582d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:48.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.670+0000 7f43210b4700 1 -- 172.21.15.92:0/3968988823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f431c068b00 msgr2=0x7f431c0b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:48.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.670+0000 7f43210b4700 1 --2- 172.21.15.92:0/3968988823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f431c068b00 0x7f431c0b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:48.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.670+0000 7f43210b4700 1 -- 172.21.15.92:0/3968988823 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4314009ae0 con 0x7f431c0ad320 2023-10-06T10:41:48.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.670+0000 7f43218b5700 1 --2- 172.21.15.92:0/3968988823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f431c068b00 0x7f431c0b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:48.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.670+0000 7f43210b4700 1 --2- 172.21.15.92:0/3968988823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f431c0ad320 0x7f431c1502b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f430c00d860 tx=0x7f430c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:48.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.670+0000 7f4311ffb700 1 -- 172.21.15.92:0/3968988823 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c0125f0 con 0x7f431c0ad320 2023-10-06T10:41:48.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.670+0000 7f4311ffb700 1 -- 172.21.15.92:0/3968988823 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430c012750 con 0x7f431c0ad320 2023-10-06T10:41:48.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.670+0000 7f4323b19700 1 -- 172.21.15.92:0/3968988823 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f431c0b0350 con 0x7f431c0ad320 2023-10-06T10:41:48.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.670+0000 7f4311ffb700 1 -- 172.21.15.92:0/3968988823 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c012b10 con 0x7f431c0ad320 2023-10-06T10:41:48.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.671+0000 7f4323b19700 1 -- 172.21.15.92:0/3968988823 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431c157ad0 con 0x7f431c0ad320 2023-10-06T10:41:48.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.671+0000 7f4323b19700 1 -- 172.21.15.92:0/3968988823 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431c157650 con 0x7f431c0ad320 2023-10-06T10:41:48.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.672+0000 7f4311ffb700 1 -- 172.21.15.92:0/3968988823 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f430c01e460 con 0x7f431c0ad320 2023-10-06T10:41:48.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.672+0000 7f4311ffb700 1 --2- 172.21.15.92:0/3968988823 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4300066f30 0x7f43000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:48.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.672+0000 7f4311ffb700 1 -- 172.21.15.92:0/3968988823 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f430c084e90 con 0x7f431c0ad320 2023-10-06T10:41:48.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.672+0000 7f43218b5700 1 --2- 172.21.15.92:0/3968988823 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4300066f30 0x7f43000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:48.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.672+0000 7f43218b5700 1 --2- 172.21.15.92:0/3968988823 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4300066f30 0x7f43000693e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4318006fd0 tx=0x7f4318008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:48.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.676+0000 7f4311ffb700 1 -- 172.21.15.92:0/3968988823 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f431c157650 con 0x7f431c0ad320 2023-10-06T10:41:48.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.862+0000 7f4323b19700 1 -- 172.21.15.92:0/3968988823 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f431c157650 con 0x7f431c0ad320 2023-10-06T10:41:48.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.862+0000 7f4311ffb700 1 -- 172.21.15.92:0/3968988823 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+17087 (secure 0 0 0) 0x7f431c157650 con 0x7f431c0ad320 2023-10-06T10:41:48.865 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:48.865 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":63,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:41:47.898400+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:41:45.546409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-06T10:41:46.862955+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:41:48.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.865+0000 7f4323b19700 1 -- 172.21.15.92:0/3968988823 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4300066f30 msgr2=0x7f43000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:48.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.865+0000 7f4323b19700 1 --2- 172.21.15.92:0/3968988823 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4300066f30 0x7f43000693e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4318006fd0 tx=0x7f4318008040 comp rx=0 tx=0).stop 2023-10-06T10:41:48.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.865+0000 7f4323b19700 1 -- 172.21.15.92:0/3968988823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f431c0ad320 msgr2=0x7f431c1502b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:48.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.865+0000 7f4323b19700 1 --2- 172.21.15.92:0/3968988823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f431c0ad320 0x7f431c1502b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f430c00d860 tx=0x7f430c008040 comp rx=0 tx=0).stop 2023-10-06T10:41:48.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.865+0000 7f4323b19700 1 -- 172.21.15.92:0/3968988823 shutdown_connections 2023-10-06T10:41:48.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.865+0000 7f4323b19700 1 --2- 172.21.15.92:0/3968988823 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4300066f30 0x7f43000693e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:48.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.865+0000 7f4323b19700 1 --2- 172.21.15.92:0/3968988823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f431c0ad6f0 0x7f431c1582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:48.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.865+0000 7f4323b19700 1 --2- 172.21.15.92:0/3968988823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f431c068b00 0x7f431c0b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:48.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.865+0000 7f4323b19700 1 --2- 172.21.15.92:0/3968988823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f431c0ad320 0x7f431c1502b0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:48.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.865+0000 7f4323b19700 1 -- 172.21.15.92:0/3968988823 >> 172.21.15.92:0/3968988823 conn(0x7f431c06dcd0 msgr2=0x7f431c152d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:48.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.865+0000 7f4323b19700 1 -- 172.21.15.92:0/3968988823 shutdown_connections 2023-10-06T10:41:48.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.865+0000 7f4323b19700 1 -- 172.21.15.92:0/3968988823 wait complete. 2023-10-06T10:41:48.879 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:41:48.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.991+0000 7f9ad8e28700 1 Processor -- start 2023-10-06T10:41:48.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.991+0000 7f9ad8e28700 1 -- start start 2023-10-06T10:41:48.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.992+0000 7f9ad8e28700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9ad40cbc30 0x7f9ad40cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.992+0000 7f9ad8e28700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9ad40b2100 0x7f9ad40b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.992+0000 7f9ad8e28700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad40cb7a0 con 0x7f9ad4159440 2023-10-06T10:41:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.992+0000 7f9ad8e28700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ad40b0860 con 0x7f9ad40b2100 2023-10-06T10:41:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.992+0000 7f9ad8e28700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ad40cb150 con 0x7f9ad40cbc30 2023-10-06T10:41:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.992+0000 7f9ad3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9ad40b2100 0x7f9ad40b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.992+0000 7f9ad37fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9ad4159440 0x7f9ad40b3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33222/0 (socket says 172.21.15.92:33222) 2023-10-06T10:41:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.992+0000 7f9ad37fe700 1 -- 172.21.15.92:0/3484666388 learned_addr learned my addr 172.21.15.92:0/3484666388 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.992+0000 7f9ad2ffd700 1 --2- 172.21.15.92:0/3484666388 >> v2:172.21.15.124:3300/0 conn(0x7f9ad40cbc30 0x7f9ad40cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.992+0000 7f9ad27fc700 1 -- 172.21.15.92:0/3484666388 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1037540873 0 0) 0x7f9ad40cb7a0 con 0x7f9ad4159440 2023-10-06T10:41:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.992+0000 7f9ad3fff700 1 -- 172.21.15.92:0/3484666388 >> v2:172.21.15.124:3300/0 conn(0x7f9ad40cbc30 msgr2=0x7f9ad40cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.992+0000 7f9ad3fff700 1 --2- 172.21.15.92:0/3484666388 >> v2:172.21.15.124:3300/0 conn(0x7f9ad40cbc30 0x7f9ad40cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:49.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.992+0000 7f9ad3fff700 1 -- 172.21.15.92:0/3484666388 >> v1:172.21.15.92:6789/0 conn(0x7f9ad4159440 legacy=0x7f9ad40b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:49.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.992+0000 7f9ad3fff700 1 -- 172.21.15.92:0/3484666388 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad40cb430 con 0x7f9ad40b2100 2023-10-06T10:41:49.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.993+0000 7f9ad3fff700 1 --2- 172.21.15.92:0/3484666388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ad40b2100 0x7f9ad40b2520 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40d0220 tx=0x7f9acc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20f34e10077d3ff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:49.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.993+0000 7f9ad27fc700 1 -- 172.21.15.92:0/3484666388 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc00b6b0 con 0x7f9ad40b2100 2023-10-06T10:41:49.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.993+0000 7f9ad27fc700 1 -- 172.21.15.92:0/3484666388 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9acc00b810 con 0x7f9ad40b2100 2023-10-06T10:41:49.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.993+0000 7f9ad27fc700 1 -- 172.21.15.92:0/3484666388 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc00bbb0 con 0x7f9ad40b2100 2023-10-06T10:41:49.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.993+0000 7f9ad8e28700 1 -- 172.21.15.92:0/3484666388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ad40b2100 msgr2=0x7f9ad40b2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:49.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.993+0000 7f9ad8e28700 1 --2- 172.21.15.92:0/3484666388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ad40b2100 0x7f9ad40b2520 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40d0220 tx=0x7f9acc0091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:49.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.993+0000 7f9ad8e28700 1 -- 172.21.15.92:0/3484666388 shutdown_connections 2023-10-06T10:41:49.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.993+0000 7f9ad8e28700 1 --2- 172.21.15.92:0/3484666388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ad40b2100 0x7f9ad40b2520 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:49.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.993+0000 7f9ad8e28700 1 --2- 172.21.15.92:0/3484666388 >> v2:172.21.15.124:3300/0 conn(0x7f9ad40cbc30 0x7f9ad40cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:49.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.993+0000 7f9ad8e28700 1 -- 172.21.15.92:0/3484666388 >> 172.21.15.92:0/3484666388 conn(0x7f9ad406ece0 msgr2=0x7f9ad406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:49.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.993+0000 7f9ad8e28700 1 -- 172.21.15.92:0/3484666388 shutdown_connections 2023-10-06T10:41:49.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.993+0000 7f9ad8e28700 1 -- 172.21.15.92:0/3484666388 wait complete. 2023-10-06T10:41:49.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.995+0000 7f9ad8e28700 1 Processor -- start 2023-10-06T10:41:49.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.995+0000 7f9ad8e28700 1 -- start start 2023-10-06T10:41:49.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.995+0000 7f9ad8e28700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ad40b2100 0x7f9ad40c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:49.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.995+0000 7f9ad8e28700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ad40cbc30 0x7f9ad4154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:49.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.995+0000 7f9ad8e28700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ad4159440 0x7f9ad41698f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:49.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.995+0000 7f9ad8e28700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad40c5950 con 0x7f9ad40b2100 2023-10-06T10:41:49.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.995+0000 7f9ad8e28700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad40c5020 con 0x7f9ad4159440 2023-10-06T10:41:49.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.995+0000 7f9ad8e28700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad40c5300 con 0x7f9ad40cbc30 2023-10-06T10:41:49.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.995+0000 7f9ad37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ad40b2100 0x7f9ad40c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:49.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.995+0000 7f9ad37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ad40b2100 0x7f9ad40c5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57224/0 (socket says 172.21.15.92:57224) 2023-10-06T10:41:49.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.995+0000 7f9ad37fe700 1 -- 172.21.15.92:0/3821831138 learned_addr learned my addr 172.21.15.92:0/3821831138 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:49.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.996+0000 7f9ad3fff700 1 --2- 172.21.15.92:0/3821831138 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ad4159440 0x7f9ad41698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:49.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.996+0000 7f9ad2ffd700 1 --2- 172.21.15.92:0/3821831138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ad40cbc30 0x7f9ad4154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:49.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.996+0000 7f9ad37fe700 1 -- 172.21.15.92:0/3821831138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ad40cbc30 msgr2=0x7f9ad4154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:49.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.996+0000 7f9ad37fe700 1 --2- 172.21.15.92:0/3821831138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ad40cbc30 0x7f9ad4154940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:49.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.996+0000 7f9ad37fe700 1 -- 172.21.15.92:0/3821831138 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ad4159440 msgr2=0x7f9ad41698f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:49.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.996+0000 7f9ad37fe700 1 --2- 172.21.15.92:0/3821831138 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ad4159440 0x7f9ad41698f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:49.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.996+0000 7f9ad37fe700 1 -- 172.21.15.92:0/3821831138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc009420 con 0x7f9ad40b2100 2023-10-06T10:41:49.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.996+0000 7f9ad3fff700 1 --2- 172.21.15.92:0/3821831138 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ad4159440 0x7f9ad41698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:49.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.996+0000 7f9ad37fe700 1 --2- 172.21.15.92:0/3821831138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ad40b2100 0x7f9ad40c5de0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400cc70 tx=0x7f9ac4001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:49.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.996+0000 7f9abb7fe700 1 -- 172.21.15.92:0/3821831138 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400ce80 con 0x7f9ad40b2100 2023-10-06T10:41:49.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.996+0000 7f9abb7fe700 1 -- 172.21.15.92:0/3821831138 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac400ab70 con 0x7f9ad40b2100 2023-10-06T10:41:49.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.996+0000 7f9abb7fe700 1 -- 172.21.15.92:0/3821831138 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac40070f0 con 0x7f9ad40b2100 2023-10-06T10:41:49.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.996+0000 7f9ad8e28700 1 -- 172.21.15.92:0/3821831138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad40c55e0 con 0x7f9ad40b2100 2023-10-06T10:41:49.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.996+0000 7f9ad8e28700 1 -- 172.21.15.92:0/3821831138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad415f330 con 0x7f9ad40b2100 2023-10-06T10:41:49.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.997+0000 7f9ad8e28700 1 -- 172.21.15.92:0/3821831138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad415f970 con 0x7f9ad40b2100 2023-10-06T10:41:49.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.997+0000 7f9abb7fe700 1 -- 172.21.15.92:0/3821831138 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ac40074e0 con 0x7f9ad40b2100 2023-10-06T10:41:49.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.998+0000 7f9abb7fe700 1 --2- 172.21.15.92:0/3821831138 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9ac0066f30 0x7f9ac00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:49.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.998+0000 7f9abb7fe700 1 -- 172.21.15.92:0/3821831138 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9ac4084bb0 con 0x7f9ad40b2100 2023-10-06T10:41:49.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.998+0000 7f9ad2ffd700 1 --2- 172.21.15.92:0/3821831138 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9ac0066f30 0x7f9ac00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:49.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:48.998+0000 7f9ad2ffd700 1 --2- 172.21.15.92:0/3821831138 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9ac0066f30 0x7f9ac00693e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40b5ec0 tx=0x7f9ac8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:49.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.002+0000 7f9abb7fe700 1 -- 172.21.15.92:0/3821831138 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ac4051c00 con 0x7f9ad40b2100 2023-10-06T10:41:49.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.184+0000 7f9ad8e28700 1 -- 172.21.15.92:0/3821831138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ad40c5c30 con 0x7f9ad40b2100 2023-10-06T10:41:49.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.185+0000 7f9abb7fe700 1 -- 172.21.15.92:0/3821831138 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+17087 (secure 0 0 0) 0x7f9ac4055220 con 0x7f9ad40b2100 2023-10-06T10:41:49.187 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:49.188 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":63,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:41:47.898400+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:41:45.546409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-06T10:41:46.862955+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:41:49.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.187+0000 7f9ad8e28700 1 -- 172.21.15.92:0/3821831138 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9ac0066f30 msgr2=0x7f9ac00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:49.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.187+0000 7f9ad8e28700 1 --2- 172.21.15.92:0/3821831138 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9ac0066f30 0x7f9ac00693e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40b5ec0 tx=0x7f9ac8008040 comp rx=0 tx=0).stop 2023-10-06T10:41:49.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.187+0000 7f9ad8e28700 1 -- 172.21.15.92:0/3821831138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ad40b2100 msgr2=0x7f9ad40c5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:49.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.187+0000 7f9ad8e28700 1 --2- 172.21.15.92:0/3821831138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ad40b2100 0x7f9ad40c5de0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400cc70 tx=0x7f9ac4001a30 comp rx=0 tx=0).stop 2023-10-06T10:41:49.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.187+0000 7f9ad8e28700 1 -- 172.21.15.92:0/3821831138 shutdown_connections 2023-10-06T10:41:49.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.187+0000 7f9ad8e28700 1 --2- 172.21.15.92:0/3821831138 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9ac0066f30 0x7f9ac00693e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:49.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.187+0000 7f9ad8e28700 1 --2- 172.21.15.92:0/3821831138 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ad4159440 0x7f9ad41698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:49.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.187+0000 7f9ad8e28700 1 --2- 172.21.15.92:0/3821831138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ad40b2100 0x7f9ad40c5de0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:49.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.187+0000 7f9ad8e28700 1 --2- 172.21.15.92:0/3821831138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ad40cbc30 0x7f9ad4154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:49.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.187+0000 7f9ad8e28700 1 -- 172.21.15.92:0/3821831138 >> 172.21.15.92:0/3821831138 conn(0x7f9ad406ece0 msgr2=0x7f9ad406f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:49.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.187+0000 7f9ad8e28700 1 -- 172.21.15.92:0/3821831138 shutdown_connections 2023-10-06T10:41:49.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.187+0000 7f9ad8e28700 1 -- 172.21.15.92:0/3821831138 wait complete. 2023-10-06T10:41:49.201 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T10:41:49.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.319+0000 7f7c9709c700 1 Processor -- start 2023-10-06T10:41:49.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.319+0000 7f7c9709c700 1 -- start start 2023-10-06T10:41:49.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.320+0000 7f7c9709c700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7c900b0a10 0x7f7c900cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:49.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.320+0000 7f7c9709c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c900cc420 0x7f7c900c9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:49.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.320+0000 7f7c9709c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c900c53c0 con 0x7f7c900b0640 2023-10-06T10:41:49.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.320+0000 7f7c9709c700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c900ca790 con 0x7f7c900b0a10 2023-10-06T10:41:49.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.320+0000 7f7c9709c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c900ccd00 con 0x7f7c900cc420 2023-10-06T10:41:49.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.320+0000 7f7c94e38700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7c900b0640 0x7f7c900cecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33238/0 (socket says 172.21.15.92:33238) 2023-10-06T10:41:49.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.320+0000 7f7c94e38700 1 -- 172.21.15.92:0/549943569 learned_addr learned my addr 172.21.15.92:0/549943569 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:49.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.320+0000 7f7c95639700 1 --2- 172.21.15.92:0/549943569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c900cc420 0x7f7c900c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:49.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c8ffff700 1 --2- 172.21.15.92:0/549943569 >> v2:172.21.15.124:3300/0 conn(0x7f7c900b0a10 0x7f7c900cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:49.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c8f7fe700 1 -- 172.21.15.92:0/549943569 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1567962814 0 0) 0x7f7c900c53c0 con 0x7f7c900b0640 2023-10-06T10:41:49.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c8f7fe700 1 -- 172.21.15.92:0/549943569 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c80003570 con 0x7f7c900b0640 2023-10-06T10:41:49.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c8f7fe700 1 -- 172.21.15.92:0/549943569 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3870085293 0 0) 0x7f7c80003570 con 0x7f7c900b0640 2023-10-06T10:41:49.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c8f7fe700 1 -- 172.21.15.92:0/549943569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c900cc420 msgr2=0x7f7c900c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:49.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c8f7fe700 1 --2- 172.21.15.92:0/549943569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c900cc420 0x7f7c900c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:49.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c8f7fe700 1 -- 172.21.15.92:0/549943569 >> v2:172.21.15.124:3300/0 conn(0x7f7c900b0a10 msgr2=0x7f7c900cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:49.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c8f7fe700 1 --2- 172.21.15.92:0/549943569 >> v2:172.21.15.124:3300/0 conn(0x7f7c900b0a10 0x7f7c900cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:49.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c8f7fe700 1 -- 172.21.15.92:0/549943569 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c900ccfe0 con 0x7f7c900b0640 2023-10-06T10:41:49.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c8f7fe700 1 -- 172.21.15.92:0/549943569 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f7c840015f0 con 0x7f7c900b0640 2023-10-06T10:41:49.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c8f7fe700 1 -- 172.21.15.92:0/549943569 >> v1:172.21.15.92:6789/0 conn(0x7f7c900b0640 legacy=0x7f7c900cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:49.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c8f7fe700 1 --2- 172.21.15.92:0/549943569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c80003a60 0x7f7c80003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:49.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c8f7fe700 1 --2- 172.21.15.92:0/549943569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c900cc420 0x7f7c80004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:49.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c8f7fe700 1 --2- 172.21.15.92:0/549943569 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c80004d50 0x7f7c800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:49.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c8f7fe700 1 -- 172.21.15.92:0/549943569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c900ca790 con 0x7f7c80003a60 2023-10-06T10:41:49.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c8f7fe700 1 -- 172.21.15.92:0/549943569 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c900ccd00 con 0x7f7c80004d50 2023-10-06T10:41:49.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c8f7fe700 1 -- 172.21.15.92:0/549943569 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8000ba60 con 0x7f7c900cc420 2023-10-06T10:41:49.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c94e38700 1 --2- 172.21.15.92:0/549943569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c80003a60 0x7f7c80003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:49.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c8ffff700 1 --2- 172.21.15.92:0/549943569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c900cc420 0x7f7c80004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:49.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c95639700 1 --2- 172.21.15.92:0/549943569 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c80004d50 0x7f7c800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:49.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c94e38700 1 -- 172.21.15.92:0/549943569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c900cc420 msgr2=0x7f7c80004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:49.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c94e38700 1 --2- 172.21.15.92:0/549943569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c900cc420 0x7f7c80004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:49.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c94e38700 1 -- 172.21.15.92:0/549943569 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c80004d50 msgr2=0x7f7c800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:49.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c94e38700 1 --2- 172.21.15.92:0/549943569 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c80004d50 0x7f7c800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:49.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c94e38700 1 -- 172.21.15.92:0/549943569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c80003570 con 0x7f7c80003a60 2023-10-06T10:41:49.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.321+0000 7f7c94e38700 1 --2- 172.21.15.92:0/549943569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c80003a60 0x7f7c80003e70 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f7c8400c3c0 tx=0x7f7c84007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb224ae94449716 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:49.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.322+0000 7f7c8f7fe700 1 -- 172.21.15.92:0/549943569 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8400cc30 con 0x7f7c80003a60 2023-10-06T10:41:49.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.322+0000 7f7c8f7fe700 1 -- 172.21.15.92:0/549943569 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c8400cd90 con 0x7f7c80003a60 2023-10-06T10:41:49.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.322+0000 7f7c8f7fe700 1 -- 172.21.15.92:0/549943569 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c84007320 con 0x7f7c80003a60 2023-10-06T10:41:49.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.322+0000 7f7c9709c700 1 -- 172.21.15.92:0/549943569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c80003a60 msgr2=0x7f7c80003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:49.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.322+0000 7f7c9709c700 1 --2- 172.21.15.92:0/549943569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c80003a60 0x7f7c80003e70 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f7c8400c3c0 tx=0x7f7c84007920 comp rx=0 tx=0).stop 2023-10-06T10:41:49.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.322+0000 7f7c95639700 1 -- 172.21.15.92:0/549943569 reap_dead start 2023-10-06T10:41:49.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.322+0000 7f7c9709c700 1 -- 172.21.15.92:0/549943569 shutdown_connections 2023-10-06T10:41:49.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.322+0000 7f7c9709c700 1 -- 172.21.15.92:0/549943569 >> 172.21.15.92:0/549943569 conn(0x7f7c90087fc0 msgr2=0x7f7c900ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:49.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.322+0000 7f7c9709c700 1 -- 172.21.15.92:0/549943569 shutdown_connections 2023-10-06T10:41:49.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.322+0000 7f7c9709c700 1 -- 172.21.15.92:0/549943569 wait complete. 2023-10-06T10:41:49.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.323+0000 7f7c9709c700 1 Processor -- start 2023-10-06T10:41:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.324+0000 7f7c9709c700 1 -- start start 2023-10-06T10:41:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.324+0000 7f7c9709c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c80003a60 0x7f7c9016ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.324+0000 7f7c9709c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c901549e0 0x7f7c901541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.324+0000 7f7c9709c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c900b1820 0x7f7c900b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.324+0000 7f7c9709c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c900b2200 con 0x7f7c901549e0 2023-10-06T10:41:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.324+0000 7f7c9709c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c90157dd0 con 0x7f7c900b1820 2023-10-06T10:41:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.324+0000 7f7c9709c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9016b200 con 0x7f7c80003a60 2023-10-06T10:41:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.324+0000 7f7c8ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c901549e0 0x7f7c901541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:49.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.324+0000 7f7c8ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c901549e0 0x7f7c901541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57234/0 (socket says 172.21.15.92:57234) 2023-10-06T10:41:49.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.324+0000 7f7c8ffff700 1 -- 172.21.15.92:0/2431211006 learned_addr learned my addr 172.21.15.92:0/2431211006 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:49.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.324+0000 7f7c94e38700 1 --2- 172.21.15.92:0/2431211006 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c80003a60 0x7f7c9016ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:49.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.324+0000 7f7c95639700 1 --2- 172.21.15.92:0/2431211006 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c900b1820 0x7f7c900b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:49.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.325+0000 7f7c8ffff700 1 -- 172.21.15.92:0/2431211006 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c80003a60 msgr2=0x7f7c9016ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:49.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.325+0000 7f7c8ffff700 1 --2- 172.21.15.92:0/2431211006 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c80003a60 0x7f7c9016ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:49.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.325+0000 7f7c8ffff700 1 -- 172.21.15.92:0/2431211006 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c900b1820 msgr2=0x7f7c900b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:49.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.325+0000 7f7c8ffff700 1 --2- 172.21.15.92:0/2431211006 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c900b1820 0x7f7c900b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:49.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.325+0000 7f7c8ffff700 1 -- 172.21.15.92:0/2431211006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9016b4e0 con 0x7f7c901549e0 2023-10-06T10:41:49.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.325+0000 7f7c8ffff700 1 --2- 172.21.15.92:0/2431211006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c901549e0 0x7f7c901541b0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c011d70 tx=0x7f7c7c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:49.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.325+0000 7f7c8cff9700 1 -- 172.21.15.92:0/2431211006 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c00f930 con 0x7f7c901549e0 2023-10-06T10:41:49.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.325+0000 7f7c8cff9700 1 -- 172.21.15.92:0/2431211006 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7c00fa90 con 0x7f7c901549e0 2023-10-06T10:41:49.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.325+0000 7f7c9709c700 1 -- 172.21.15.92:0/2431211006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c9016b7c0 con 0x7f7c901549e0 2023-10-06T10:41:49.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.325+0000 7f7c8cff9700 1 -- 172.21.15.92:0/2431211006 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c00fe10 con 0x7f7c901549e0 2023-10-06T10:41:49.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.325+0000 7f7c9709c700 1 -- 172.21.15.92:0/2431211006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c901546f0 con 0x7f7c901549e0 2023-10-06T10:41:49.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.326+0000 7f7c9709c700 1 -- 172.21.15.92:0/2431211006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c90064a90 con 0x7f7c901549e0 2023-10-06T10:41:49.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.326+0000 7f7c8cff9700 1 -- 172.21.15.92:0/2431211006 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7c7c014650 con 0x7f7c901549e0 2023-10-06T10:41:49.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.327+0000 7f7c8cff9700 1 --2- 172.21.15.92:0/2431211006 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7c74066f30 0x7f7c740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:49.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.327+0000 7f7c94e38700 1 --2- 172.21.15.92:0/2431211006 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7c74066f30 0x7f7c740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:49.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.327+0000 7f7c8cff9700 1 -- 172.21.15.92:0/2431211006 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7c7c018030 con 0x7f7c901549e0 2023-10-06T10:41:49.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.327+0000 7f7c94e38700 1 --2- 172.21.15.92:0/2431211006 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7c74066f30 0x7f7c740693e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7c84008fd0 tx=0x7f7c84015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:49.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.331+0000 7f7c8cff9700 1 -- 172.21.15.92:0/2431211006 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7c7c052000 con 0x7f7c901549e0 2023-10-06T10:41:49.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:49.560+0000 7f7c9709c700 1 -- 172.21.15.92:0/2431211006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7c90157f10 con 0x7f7c901549e0 2023-10-06T10:41:50.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.550+0000 7f7c8cff9700 1 -- 172.21.15.92:0/2431211006 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v49) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7c7c0220b0 con 0x7f7c901549e0 2023-10-06T10:41:50.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.553+0000 7f7c9709c700 1 -- 172.21.15.92:0/2431211006 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7c74066f30 msgr2=0x7f7c740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:50.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.553+0000 7f7c9709c700 1 --2- 172.21.15.92:0/2431211006 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7c74066f30 0x7f7c740693e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7c84008fd0 tx=0x7f7c84015000 comp rx=0 tx=0).stop 2023-10-06T10:41:50.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.553+0000 7f7c9709c700 1 -- 172.21.15.92:0/2431211006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c901549e0 msgr2=0x7f7c901541b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:50.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.553+0000 7f7c9709c700 1 --2- 172.21.15.92:0/2431211006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c901549e0 0x7f7c901541b0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c011d70 tx=0x7f7c7c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:50.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.553+0000 7f7c9709c700 1 -- 172.21.15.92:0/2431211006 shutdown_connections 2023-10-06T10:41:50.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.553+0000 7f7c9709c700 1 --2- 172.21.15.92:0/2431211006 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7c74066f30 0x7f7c740693e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:50.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.553+0000 7f7c9709c700 1 --2- 172.21.15.92:0/2431211006 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c900b1820 0x7f7c900b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:50.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.553+0000 7f7c9709c700 1 --2- 172.21.15.92:0/2431211006 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c80003a60 0x7f7c9016ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:50.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.553+0000 7f7c9709c700 1 --2- 172.21.15.92:0/2431211006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c901549e0 0x7f7c901541b0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:50.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.553+0000 7f7c9709c700 1 -- 172.21.15.92:0/2431211006 >> 172.21.15.92:0/2431211006 conn(0x7f7c90087fc0 msgr2=0x7f7c90156e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:50.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.553+0000 7f7c9709c700 1 -- 172.21.15.92:0/2431211006 shutdown_connections 2023-10-06T10:41:50.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.553+0000 7f7c9709c700 1 -- 172.21.15.92:0/2431211006 wait complete. 2023-10-06T10:41:50.567 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T10:41:50.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.684+0000 7f68311ea700 1 Processor -- start 2023-10-06T10:41:50.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.684+0000 7f68311ea700 1 -- start start 2023-10-06T10:41:50.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.684+0000 7f68311ea700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f682c0cbc70 0x7f682c0ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:50.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.684+0000 7f68311ea700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f682c0cf5d0 con 0x7f682c1594d0 2023-10-06T10:41:50.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.685+0000 7f68311ea700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f682c0cf8b0 con 0x7f682c1598a0 2023-10-06T10:41:50.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.685+0000 7f68311ea700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f682c0cfb90 con 0x7f682c0cbc70 2023-10-06T10:41:50.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.685+0000 7f682b59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f682c0cbc70 0x7f682c0ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:50.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.685+0000 7f682b59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f682c0cbc70 0x7f682c0ceef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57246/0 (socket says 172.21.15.92:57246) 2023-10-06T10:41:50.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.685+0000 7f682ad9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f682c1594d0 0x7f682c0cb560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33240/0 (socket says 172.21.15.92:33240) 2023-10-06T10:41:50.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.685+0000 7f682b59e700 1 -- 172.21.15.92:0/3411118324 learned_addr learned my addr 172.21.15.92:0/3411118324 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:50.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.685+0000 7f682b59e700 1 -- 172.21.15.92:0/3411118324 >> v1:172.21.15.124:6789/0 conn(0x7f682c1598a0 legacy=0x7f682c0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:50.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.685+0000 7f682b59e700 1 -- 172.21.15.92:0/3411118324 >> v1:172.21.15.92:6789/0 conn(0x7f682c1594d0 legacy=0x7f682c0cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:50.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.685+0000 7f682a59c700 1 -- 172.21.15.92:0/3411118324 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4026421731 0 0) 0x7f682c0cf5d0 con 0x7f682c1594d0 2023-10-06T10:41:50.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.685+0000 7f682b59e700 1 -- 172.21.15.92:0/3411118324 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682c0cfe70 con 0x7f682c0cbc70 2023-10-06T10:41:50.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.685+0000 7f682b59e700 1 --2- 172.21.15.92:0/3411118324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f682c0cbc70 0x7f682c0ceef0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6824009f20 tx=0x7f6824009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13bc213af98c6b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:50.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.685+0000 7f682a59c700 1 -- 172.21.15.92:0/3411118324 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682400e070 con 0x7f682c0cbc70 2023-10-06T10:41:50.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.685+0000 7f682a59c700 1 -- 172.21.15.92:0/3411118324 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682400ba80 con 0x7f682c0cbc70 2023-10-06T10:41:50.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.685+0000 7f682a59c700 1 -- 172.21.15.92:0/3411118324 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68240092e0 con 0x7f682c0cbc70 2023-10-06T10:41:50.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.686+0000 7f68311ea700 1 -- 172.21.15.92:0/3411118324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f682c0cbc70 msgr2=0x7f682c0ceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:50.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.686+0000 7f68311ea700 1 --2- 172.21.15.92:0/3411118324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f682c0cbc70 0x7f682c0ceef0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6824009f20 tx=0x7f6824009840 comp rx=0 tx=0).stop 2023-10-06T10:41:50.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.686+0000 7f68311ea700 1 -- 172.21.15.92:0/3411118324 shutdown_connections 2023-10-06T10:41:50.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.686+0000 7f68311ea700 1 --2- 172.21.15.92:0/3411118324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f682c0cbc70 0x7f682c0ceef0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:50.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.686+0000 7f68311ea700 1 -- 172.21.15.92:0/3411118324 >> 172.21.15.92:0/3411118324 conn(0x7f682c089e50 msgr2=0x7f682c099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:50.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.686+0000 7f68311ea700 1 -- 172.21.15.92:0/3411118324 shutdown_connections 2023-10-06T10:41:50.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.686+0000 7f68311ea700 1 -- 172.21.15.92:0/3411118324 wait complete. 2023-10-06T10:41:50.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.687+0000 7f68311ea700 1 Processor -- start 2023-10-06T10:41:50.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.687+0000 7f68311ea700 1 -- start start 2023-10-06T10:41:50.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.688+0000 7f68311ea700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f682c0cbc70 0x7f682c159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:50.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.688+0000 7f68311ea700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f682c1594d0 0x7f682c152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:50.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.688+0000 7f68311ea700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f682c1598a0 0x7f682c1529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:50.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.688+0000 7f68311ea700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682c15ab60 con 0x7f682c1594d0 2023-10-06T10:41:50.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.688+0000 7f68311ea700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682c152f00 con 0x7f682c0cbc70 2023-10-06T10:41:50.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.688+0000 7f68311ea700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f682c150a60 con 0x7f682c1598a0 2023-10-06T10:41:50.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.688+0000 7f6823fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f682c1594d0 0x7f682c152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:50.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.688+0000 7f6823fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f682c1594d0 0x7f682c152480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57250/0 (socket says 172.21.15.92:57250) 2023-10-06T10:41:50.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.688+0000 7f6823fff700 1 -- 172.21.15.92:0/2650291418 learned_addr learned my addr 172.21.15.92:0/2650291418 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:50.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.688+0000 7f682ad9d700 1 --2- 172.21.15.92:0/2650291418 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f682c0cbc70 0x7f682c159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:50.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.688+0000 7f682b59e700 1 --2- 172.21.15.92:0/2650291418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f682c1598a0 0x7f682c1529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:50.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.688+0000 7f6823fff700 1 -- 172.21.15.92:0/2650291418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f682c1598a0 msgr2=0x7f682c1529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:50.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.688+0000 7f6823fff700 1 --2- 172.21.15.92:0/2650291418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f682c1598a0 0x7f682c1529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:50.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.688+0000 7f6823fff700 1 -- 172.21.15.92:0/2650291418 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f682c0cbc70 msgr2=0x7f682c159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:50.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.688+0000 7f6823fff700 1 --2- 172.21.15.92:0/2650291418 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f682c0cbc70 0x7f682c159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:50.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.688+0000 7f6823fff700 1 -- 172.21.15.92:0/2650291418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6824009ae0 con 0x7f682c1594d0 2023-10-06T10:41:50.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.688+0000 7f6823fff700 1 --2- 172.21.15.92:0/2650291418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f682c1594d0 0x7f682c152480 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f681400c4c0 tx=0x7f6814007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:50.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.688+0000 7f6822ffd700 1 -- 172.21.15.92:0/2650291418 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681400ce70 con 0x7f682c1594d0 2023-10-06T10:41:50.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.688+0000 7f6822ffd700 1 -- 172.21.15.92:0/2650291418 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6814011590 con 0x7f682c1594d0 2023-10-06T10:41:50.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.688+0000 7f68311ea700 1 -- 172.21.15.92:0/2650291418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f682c150d00 con 0x7f682c1594d0 2023-10-06T10:41:50.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.688+0000 7f6822ffd700 1 -- 172.21.15.92:0/2650291418 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814011970 con 0x7f682c1594d0 2023-10-06T10:41:50.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.688+0000 7f68311ea700 1 -- 172.21.15.92:0/2650291418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f682c1515a0 con 0x7f682c1594d0 2023-10-06T10:41:50.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.689+0000 7f68311ea700 1 -- 172.21.15.92:0/2650291418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f682c150ef0 con 0x7f682c1594d0 2023-10-06T10:41:50.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.690+0000 7f6822ffd700 1 -- 172.21.15.92:0/2650291418 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6814011d20 con 0x7f682c1594d0 2023-10-06T10:41:50.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.690+0000 7f6822ffd700 1 --2- 172.21.15.92:0/2650291418 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f680c066f30 0x7f680c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:50.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.690+0000 7f6822ffd700 1 -- 172.21.15.92:0/2650291418 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6814083ab0 con 0x7f682c1594d0 2023-10-06T10:41:50.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.690+0000 7f682ad9d700 1 --2- 172.21.15.92:0/2650291418 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f680c066f30 0x7f680c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:50.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.690+0000 7f682ad9d700 1 --2- 172.21.15.92:0/2650291418 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f680c066f30 0x7f680c0693e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f682c16d340 tx=0x7f681c007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:50.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.694+0000 7f6822ffd700 1 -- 172.21.15.92:0/2650291418 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6814050a80 con 0x7f682c1594d0 2023-10-06T10:41:50.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:50.921+0000 7f68311ea700 1 -- 172.21.15.92:0/2650291418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f682c1510f0 con 0x7f682c1594d0 2023-10-06T10:41:51.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.551+0000 7f6822ffd700 1 -- 172.21.15.92:0/2650291418 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v50) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6814017070 con 0x7f682c1594d0 2023-10-06T10:41:51.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.553+0000 7f68311ea700 1 -- 172.21.15.92:0/2650291418 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f680c066f30 msgr2=0x7f680c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:51.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.553+0000 7f68311ea700 1 --2- 172.21.15.92:0/2650291418 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f680c066f30 0x7f680c0693e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f682c16d340 tx=0x7f681c007d30 comp rx=0 tx=0).stop 2023-10-06T10:41:51.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.553+0000 7f68311ea700 1 -- 172.21.15.92:0/2650291418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f682c1594d0 msgr2=0x7f682c152480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:51.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.553+0000 7f68311ea700 1 --2- 172.21.15.92:0/2650291418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f682c1594d0 0x7f682c152480 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f681400c4c0 tx=0x7f6814007ef0 comp rx=0 tx=0).stop 2023-10-06T10:41:51.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.554+0000 7f68311ea700 1 -- 172.21.15.92:0/2650291418 shutdown_connections 2023-10-06T10:41:51.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.554+0000 7f68311ea700 1 --2- 172.21.15.92:0/2650291418 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f680c066f30 0x7f680c0693e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:51.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.554+0000 7f68311ea700 1 --2- 172.21.15.92:0/2650291418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f682c1598a0 0x7f682c1529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:51.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.554+0000 7f68311ea700 1 --2- 172.21.15.92:0/2650291418 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f682c0cbc70 0x7f682c159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:51.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.554+0000 7f68311ea700 1 --2- 172.21.15.92:0/2650291418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f682c1594d0 0x7f682c152480 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:51.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.554+0000 7f68311ea700 1 -- 172.21.15.92:0/2650291418 >> 172.21.15.92:0/2650291418 conn(0x7f682c089e50 msgr2=0x7f682c0d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:51.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.554+0000 7f68311ea700 1 -- 172.21.15.92:0/2650291418 shutdown_connections 2023-10-06T10:41:51.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.554+0000 7f68311ea700 1 -- 172.21.15.92:0/2650291418 wait complete. 2023-10-06T10:41:51.567 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:41:51.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.680+0000 7fa456e61700 1 Processor -- start 2023-10-06T10:41:51.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.680+0000 7fa456e61700 1 -- start start 2023-10-06T10:41:51.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.680+0000 7fa456e61700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa4500ad210 0x7fa4500cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:51.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.680+0000 7fa456e61700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa45015db10 0x7fa4500ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:51.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.680+0000 7fa456e61700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4500ca600 con 0x7fa4500ad5e0 2023-10-06T10:41:51.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.680+0000 7fa456e61700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4500cecf0 con 0x7fa4500ad210 2023-10-06T10:41:51.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.680+0000 7fa456e61700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4500cefd0 con 0x7fa45015db10 2023-10-06T10:41:51.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.681+0000 7fa454bfd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa4500ad210 0x7fa4500cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:51.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.681+0000 7fa447fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa4500ad5e0 0x7fa45015d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33246/0 (socket says 172.21.15.92:33246) 2023-10-06T10:41:51.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.681+0000 7fa447fff700 1 -- 172.21.15.92:0/691119913 learned_addr learned my addr 172.21.15.92:0/691119913 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:51.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.681+0000 7fa4553fe700 1 --2- 172.21.15.92:0/691119913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa45015db10 0x7fa4500ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:51.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.681+0000 7fa454bfd700 1 -- 172.21.15.92:0/691119913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa45015db10 msgr2=0x7fa4500ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:51.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.681+0000 7fa454bfd700 1 --2- 172.21.15.92:0/691119913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa45015db10 0x7fa4500ca0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:51.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.681+0000 7fa454bfd700 1 -- 172.21.15.92:0/691119913 >> v1:172.21.15.92:6789/0 conn(0x7fa4500ad5e0 legacy=0x7fa45015d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:51.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.681+0000 7fa454bfd700 1 -- 172.21.15.92:0/691119913 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4500cca90 con 0x7fa4500ad210 2023-10-06T10:41:51.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.681+0000 7fa4477fe700 1 -- 172.21.15.92:0/691119913 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1161533719 0 0) 0x7fa4500ca600 con 0x7fa4500ad5e0 2023-10-06T10:41:51.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.681+0000 7fa454bfd700 1 --2- 172.21.15.92:0/691119913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4500ad210 0x7fa4500cd290 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0098d0 tx=0x7fa44c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce449522b83cf3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:51.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.681+0000 7fa4477fe700 1 -- 172.21.15.92:0/691119913 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c009e70 con 0x7fa4500ad210 2023-10-06T10:41:51.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.681+0000 7fa4477fe700 1 -- 172.21.15.92:0/691119913 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44c005c10 con 0x7fa4500ad210 2023-10-06T10:41:51.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.681+0000 7fa4477fe700 1 -- 172.21.15.92:0/691119913 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c00b650 con 0x7fa4500ad210 2023-10-06T10:41:51.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.682+0000 7fa456e61700 1 -- 172.21.15.92:0/691119913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4500ad210 msgr2=0x7fa4500cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:51.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.682+0000 7fa456e61700 1 --2- 172.21.15.92:0/691119913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4500ad210 0x7fa4500cd290 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0098d0 tx=0x7fa44c009120 comp rx=0 tx=0).stop 2023-10-06T10:41:51.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.682+0000 7fa456e61700 1 -- 172.21.15.92:0/691119913 shutdown_connections 2023-10-06T10:41:51.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.682+0000 7fa456e61700 1 --2- 172.21.15.92:0/691119913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa45015db10 0x7fa4500ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:51.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.682+0000 7fa456e61700 1 --2- 172.21.15.92:0/691119913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4500ad210 0x7fa4500cd290 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:51.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.682+0000 7fa456e61700 1 -- 172.21.15.92:0/691119913 >> 172.21.15.92:0/691119913 conn(0x7fa450067d90 msgr2=0x7fa4500ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:51.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.682+0000 7fa456e61700 1 -- 172.21.15.92:0/691119913 shutdown_connections 2023-10-06T10:41:51.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.682+0000 7fa456e61700 1 -- 172.21.15.92:0/691119913 wait complete. 2023-10-06T10:41:51.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.683+0000 7fa456e61700 1 Processor -- start 2023-10-06T10:41:51.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.683+0000 7fa456e61700 1 -- start start 2023-10-06T10:41:51.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.684+0000 7fa456e61700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4500ad210 0x7fa4500b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:51.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.684+0000 7fa456e61700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4500ad5e0 0x7fa4500c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:51.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.684+0000 7fa456e61700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa45015db10 0x7fa45016df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:51.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.684+0000 7fa456e61700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4500af260 con 0x7fa45015db10 2023-10-06T10:41:51.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.684+0000 7fa456e61700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4500af500 con 0x7fa4500ad5e0 2023-10-06T10:41:51.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.684+0000 7fa456e61700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4500af7e0 con 0x7fa4500ad210 2023-10-06T10:41:51.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.684+0000 7fa4553fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa45015db10 0x7fa45016df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:51.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.684+0000 7fa4553fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa45015db10 0x7fa45016df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57258/0 (socket says 172.21.15.92:57258) 2023-10-06T10:41:51.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.684+0000 7fa4553fe700 1 -- 172.21.15.92:0/2156366505 learned_addr learned my addr 172.21.15.92:0/2156366505 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:51.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.684+0000 7fa454bfd700 1 --2- 172.21.15.92:0/2156366505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4500ad210 0x7fa4500b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:51.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.684+0000 7fa4553fe700 1 -- 172.21.15.92:0/2156366505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4500ad210 msgr2=0x7fa4500b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:51.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.684+0000 7fa447fff700 1 --2- 172.21.15.92:0/2156366505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4500ad5e0 0x7fa4500c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:51.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.684+0000 7fa4553fe700 1 --2- 172.21.15.92:0/2156366505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4500ad210 0x7fa4500b05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:51.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.684+0000 7fa4553fe700 1 -- 172.21.15.92:0/2156366505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4500ad5e0 msgr2=0x7fa4500c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:51.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.684+0000 7fa4553fe700 1 --2- 172.21.15.92:0/2156366505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4500ad5e0 0x7fa4500c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:51.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.684+0000 7fa4553fe700 1 -- 172.21.15.92:0/2156366505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c0093c0 con 0x7fa45015db10 2023-10-06T10:41:51.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.684+0000 7fa447fff700 1 --2- 172.21.15.92:0/2156366505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4500ad5e0 0x7fa4500c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:51.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.684+0000 7fa4553fe700 1 --2- 172.21.15.92:0/2156366505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa45015db10 0x7fa45016df90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa44800cd10 tx=0x7fa448007770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:51.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.684+0000 7fa444ff9700 1 -- 172.21.15.92:0/2156366505 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44800aac0 con 0x7fa45015db10 2023-10-06T10:41:51.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.684+0000 7fa444ff9700 1 -- 172.21.15.92:0/2156366505 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44800ac20 con 0x7fa45015db10 2023-10-06T10:41:51.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.684+0000 7fa456e61700 1 -- 172.21.15.92:0/2156366505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4500afac0 con 0x7fa45015db10 2023-10-06T10:41:51.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.685+0000 7fa444ff9700 1 -- 172.21.15.92:0/2156366505 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448017430 con 0x7fa45015db10 2023-10-06T10:41:51.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.685+0000 7fa456e61700 1 -- 172.21.15.92:0/2156366505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4500b0080 con 0x7fa45015db10 2023-10-06T10:41:51.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.686+0000 7fa456e61700 1 -- 172.21.15.92:0/2156366505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4500b02a0 con 0x7fa45015db10 2023-10-06T10:41:51.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.686+0000 7fa444ff9700 1 -- 172.21.15.92:0/2156366505 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa448003020 con 0x7fa45015db10 2023-10-06T10:41:51.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.686+0000 7fa444ff9700 1 --2- 172.21.15.92:0/2156366505 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa43c066f30 0x7fa43c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:51.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.686+0000 7fa444ff9700 1 -- 172.21.15.92:0/2156366505 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa448013070 con 0x7fa45015db10 2023-10-06T10:41:51.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.686+0000 7fa454bfd700 1 --2- 172.21.15.92:0/2156366505 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa43c066f30 0x7fa43c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:51.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.687+0000 7fa454bfd700 1 --2- 172.21.15.92:0/2156366505 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa43c066f30 0x7fa43c0693e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009160 tx=0x7fa44c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:51.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.690+0000 7fa444ff9700 1 -- 172.21.15.92:0/2156366505 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa4480518d0 con 0x7fa45015db10 2023-10-06T10:41:51.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.914+0000 7fa456e61700 1 -- 172.21.15.92:0/2156366505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa450064a90 con 0x7fa45015db10 2023-10-06T10:41:51.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.915+0000 7fa444ff9700 1 -- 172.21.15.92:0/2156366505 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fa448054ef0 con 0x7fa45015db10 2023-10-06T10:41:51.917 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:51.917 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":50,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/1409495728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":1409495728},{"type":"v1","addr":"172.21.15.124:6834","nonce":1409495728}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6019,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2297612118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2297612118},{"type":"v1","addr":"172.21.15.124:6835","nonce":2297612118}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/171578176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":171578176},{"type":"v1","addr":"172.21.15.92:6836","nonce":171578176}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:41:47.898774+0000","modified":"2023-10-06T10:41:51.550457+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/818031030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":818031030},{"type":"v1","addr":"172.21.15.92:6837","nonce":818031030}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T10:41:51.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.917+0000 7fa456e61700 1 -- 172.21.15.92:0/2156366505 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa43c066f30 msgr2=0x7fa43c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:51.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.917+0000 7fa456e61700 1 --2- 172.21.15.92:0/2156366505 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa43c066f30 0x7fa43c0693e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009160 tx=0x7fa44c016000 comp rx=0 tx=0).stop 2023-10-06T10:41:51.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.917+0000 7fa456e61700 1 -- 172.21.15.92:0/2156366505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa45015db10 msgr2=0x7fa45016df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:51.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.917+0000 7fa456e61700 1 --2- 172.21.15.92:0/2156366505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa45015db10 0x7fa45016df90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa44800cd10 tx=0x7fa448007770 comp rx=0 tx=0).stop 2023-10-06T10:41:51.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.917+0000 7fa456e61700 1 -- 172.21.15.92:0/2156366505 shutdown_connections 2023-10-06T10:41:51.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.917+0000 7fa456e61700 1 --2- 172.21.15.92:0/2156366505 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa43c066f30 0x7fa43c0693e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:51.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.917+0000 7fa456e61700 1 --2- 172.21.15.92:0/2156366505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa45015db10 0x7fa45016df90 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:51.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.917+0000 7fa456e61700 1 --2- 172.21.15.92:0/2156366505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4500ad210 0x7fa4500b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:51.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.917+0000 7fa456e61700 1 --2- 172.21.15.92:0/2156366505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4500ad5e0 0x7fa4500c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:51.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.917+0000 7fa456e61700 1 -- 172.21.15.92:0/2156366505 >> 172.21.15.92:0/2156366505 conn(0x7fa450067d90 msgr2=0x7fa450069290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:51.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.917+0000 7fa456e61700 1 -- 172.21.15.92:0/2156366505 shutdown_connections 2023-10-06T10:41:51.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:51.917+0000 7fa456e61700 1 -- 172.21.15.92:0/2156366505 wait complete. 2023-10-06T10:41:51.920 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 50 2023-10-06T10:41:51.931 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:41:52.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.044+0000 7f45be34b700 1 Processor -- start 2023-10-06T10:41:52.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.044+0000 7f45be34b700 1 -- start start 2023-10-06T10:41:52.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.045+0000 7f45be34b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f45b8159470 0x7f45b80ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:52.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.045+0000 7f45be34b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45b80cd160 0x7f45b80ce980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:52.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.045+0000 7f45be34b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f45b80ceec0 0x7f45b80c5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:52.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.045+0000 7f45be34b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b80cd5d0 con 0x7f45b8159470 2023-10-06T10:41:52.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.045+0000 7f45be34b700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b80cc800 con 0x7f45b80ceec0 2023-10-06T10:41:52.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.045+0000 7f45be34b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b80cbc30 con 0x7f45b80cd160 2023-10-06T10:41:52.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.045+0000 7f45bd349700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f45b8159470 0x7f45b80ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:52.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.045+0000 7f45bd349700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f45b8159470 0x7f45b80ccc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57260/0 (socket says 172.21.15.92:57260) 2023-10-06T10:41:52.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.045+0000 7f45bd349700 1 -- 172.21.15.92:0/1454841001 learned_addr learned my addr 172.21.15.92:0/1454841001 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:52.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.045+0000 7f45bcb48700 1 --2- 172.21.15.92:0/1454841001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45b80cd160 0x7f45b80ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:52.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.045+0000 7f45bd349700 1 -- 172.21.15.92:0/1454841001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45b80cd160 msgr2=0x7f45b80ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:52.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.045+0000 7f45bd349700 1 --2- 172.21.15.92:0/1454841001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45b80cd160 0x7f45b80ce980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.045+0000 7f45bd349700 1 -- 172.21.15.92:0/1454841001 >> v2:172.21.15.124:3300/0 conn(0x7f45b80ceec0 msgr2=0x7f45b80c5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:52.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.045+0000 7f45bdb4a700 1 --2- 172.21.15.92:0/1454841001 >> v2:172.21.15.124:3300/0 conn(0x7f45b80ceec0 0x7f45b80c5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:52.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.045+0000 7f45bd349700 1 --2- 172.21.15.92:0/1454841001 >> v2:172.21.15.124:3300/0 conn(0x7f45b80ceec0 0x7f45b80c5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.045+0000 7f45bd349700 1 -- 172.21.15.92:0/1454841001 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b80cbed0 con 0x7f45b8159470 2023-10-06T10:41:52.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.046+0000 7f45bdb4a700 1 --2- 172.21.15.92:0/1454841001 >> v2:172.21.15.124:3300/0 conn(0x7f45b80ceec0 0x7f45b80c5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:52.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.046+0000 7f45bd349700 1 --2- 172.21.15.92:0/1454841001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45b8159470 0x7f45b80ccc20 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f45b80ca8d0 tx=0x7f45b4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6096841ef34c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:52.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.046+0000 7f45affff700 1 -- 172.21.15.92:0/1454841001 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b4005c10 con 0x7f45b8159470 2023-10-06T10:41:52.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.046+0000 7f45affff700 1 -- 172.21.15.92:0/1454841001 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b4005d70 con 0x7f45b8159470 2023-10-06T10:41:52.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.046+0000 7f45affff700 1 -- 172.21.15.92:0/1454841001 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b400b8a0 con 0x7f45b8159470 2023-10-06T10:41:52.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.046+0000 7f45be34b700 1 -- 172.21.15.92:0/1454841001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45b8159470 msgr2=0x7f45b80ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:52.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.046+0000 7f45be34b700 1 --2- 172.21.15.92:0/1454841001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45b8159470 0x7f45b80ccc20 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f45b80ca8d0 tx=0x7f45b4009220 comp rx=0 tx=0).stop 2023-10-06T10:41:52.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.046+0000 7f45be34b700 1 -- 172.21.15.92:0/1454841001 shutdown_connections 2023-10-06T10:41:52.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.046+0000 7f45be34b700 1 --2- 172.21.15.92:0/1454841001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45b8159470 0x7f45b80ccc20 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.046+0000 7f45be34b700 1 --2- 172.21.15.92:0/1454841001 >> v2:172.21.15.124:3300/0 conn(0x7f45b80ceec0 0x7f45b80c5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.046+0000 7f45be34b700 1 --2- 172.21.15.92:0/1454841001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45b80cd160 0x7f45b80ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.046+0000 7f45be34b700 1 -- 172.21.15.92:0/1454841001 >> 172.21.15.92:0/1454841001 conn(0x7f45b80995c0 msgr2=0x7f45b806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:52.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.046+0000 7f45be34b700 1 -- 172.21.15.92:0/1454841001 shutdown_connections 2023-10-06T10:41:52.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.046+0000 7f45be34b700 1 -- 172.21.15.92:0/1454841001 wait complete. 2023-10-06T10:41:52.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.047+0000 7f45be34b700 1 Processor -- start 2023-10-06T10:41:52.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.048+0000 7f45be34b700 1 -- start start 2023-10-06T10:41:52.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.048+0000 7f45be34b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45b80cd160 0x7f45b8159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:52.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.048+0000 7f45be34b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45b80ceec0 0x7f45b80c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:52.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.048+0000 7f45be34b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f45b8159470 0x7f45b814c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:52.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.048+0000 7f45be34b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b80c8eb0 con 0x7f45b80cd160 2023-10-06T10:41:52.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.048+0000 7f45be34b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b8159ed0 con 0x7f45b8159470 2023-10-06T10:41:52.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.048+0000 7f45be34b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b815a1b0 con 0x7f45b80ceec0 2023-10-06T10:41:52.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.048+0000 7f45bd349700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45b80cd160 0x7f45b8159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:52.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.048+0000 7f45bd349700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45b80cd160 0x7f45b8159840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57264/0 (socket says 172.21.15.92:57264) 2023-10-06T10:41:52.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.048+0000 7f45bd349700 1 -- 172.21.15.92:0/3800010382 learned_addr learned my addr 172.21.15.92:0/3800010382 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:52.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.049+0000 7f45bcb48700 1 --2- 172.21.15.92:0/3800010382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45b80ceec0 0x7f45b80c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:52.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.049+0000 7f45bdb4a700 1 --2- 172.21.15.92:0/3800010382 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f45b8159470 0x7f45b814c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:52.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.049+0000 7f45bd349700 1 -- 172.21.15.92:0/3800010382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45b80ceec0 msgr2=0x7f45b80c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:52.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.049+0000 7f45bd349700 1 --2- 172.21.15.92:0/3800010382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45b80ceec0 0x7f45b80c99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.049+0000 7f45bd349700 1 -- 172.21.15.92:0/3800010382 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f45b8159470 msgr2=0x7f45b814c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:52.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.049+0000 7f45bd349700 1 --2- 172.21.15.92:0/3800010382 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f45b8159470 0x7f45b814c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.049+0000 7f45bd349700 1 -- 172.21.15.92:0/3800010382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b4009480 con 0x7f45b80cd160 2023-10-06T10:41:52.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.049+0000 7f45bdb4a700 1 --2- 172.21.15.92:0/3800010382 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f45b8159470 0x7f45b814c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:52.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.049+0000 7f45bd349700 1 --2- 172.21.15.92:0/3800010382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45b80cd160 0x7f45b8159840 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f45b806fc20 tx=0x7f45b4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:52.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.049+0000 7f45ad7fa700 1 -- 172.21.15.92:0/3800010382 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b401a420 con 0x7f45b80cd160 2023-10-06T10:41:52.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.049+0000 7f45ad7fa700 1 -- 172.21.15.92:0/3800010382 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b401a5e0 con 0x7f45b80cd160 2023-10-06T10:41:52.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.049+0000 7f45ad7fa700 1 -- 172.21.15.92:0/3800010382 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b401a9a0 con 0x7f45b80cd160 2023-10-06T10:41:52.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.049+0000 7f45be34b700 1 -- 172.21.15.92:0/3800010382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b815a490 con 0x7f45b80cd160 2023-10-06T10:41:52.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.049+0000 7f45be34b700 1 -- 172.21.15.92:0/3800010382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b815aa50 con 0x7f45b80cd160 2023-10-06T10:41:52.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.050+0000 7f45be34b700 1 -- 172.21.15.92:0/3800010382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b815ac70 con 0x7f45b80cd160 2023-10-06T10:41:52.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.050+0000 7f45ad7fa700 1 -- 172.21.15.92:0/3800010382 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f45b401ad10 con 0x7f45b80cd160 2023-10-06T10:41:52.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.051+0000 7f45ad7fa700 1 --2- 172.21.15.92:0/3800010382 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f459c066f30 0x7f459c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:52.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.051+0000 7f45ad7fa700 1 -- 172.21.15.92:0/3800010382 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f45b4085ba0 con 0x7f45b80cd160 2023-10-06T10:41:52.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.051+0000 7f45bcb48700 1 --2- 172.21.15.92:0/3800010382 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f459c066f30 0x7f459c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:52.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.051+0000 7f45bcb48700 1 --2- 172.21.15.92:0/3800010382 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f459c066f30 0x7f459c0693e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f45a800aa40 tx=0x7f45a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:52.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.055+0000 7f45ad7fa700 1 -- 172.21.15.92:0/3800010382 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f45b4052c20 con 0x7f45b80cd160 2023-10-06T10:41:52.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.238+0000 7f45be34b700 1 -- 172.21.15.92:0/3800010382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45b805f230 con 0x7f45b80cd160 2023-10-06T10:41:52.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.239+0000 7f45ad7fa700 1 -- 172.21.15.92:0/3800010382 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+17074 (secure 0 0 0) 0x7f45b4029020 con 0x7f45b80cd160 2023-10-06T10:41:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:52.241 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":64,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:41:49.312921+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:41:45.546409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-06T10:41:46.862955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:41:52.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.241+0000 7f45be34b700 1 -- 172.21.15.92:0/3800010382 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f459c066f30 msgr2=0x7f459c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:52.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.241+0000 7f45be34b700 1 --2- 172.21.15.92:0/3800010382 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f459c066f30 0x7f459c0693e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f45a800aa40 tx=0x7f45a8008040 comp rx=0 tx=0).stop 2023-10-06T10:41:52.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.241+0000 7f45be34b700 1 -- 172.21.15.92:0/3800010382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45b80cd160 msgr2=0x7f45b8159840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:52.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.241+0000 7f45be34b700 1 --2- 172.21.15.92:0/3800010382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45b80cd160 0x7f45b8159840 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f45b806fc20 tx=0x7f45b4016000 comp rx=0 tx=0).stop 2023-10-06T10:41:52.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.241+0000 7f45be34b700 1 -- 172.21.15.92:0/3800010382 shutdown_connections 2023-10-06T10:41:52.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.241+0000 7f45be34b700 1 --2- 172.21.15.92:0/3800010382 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f459c066f30 0x7f459c0693e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.241+0000 7f45be34b700 1 --2- 172.21.15.92:0/3800010382 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f45b8159470 0x7f45b814c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.241+0000 7f45be34b700 1 --2- 172.21.15.92:0/3800010382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45b80cd160 0x7f45b8159840 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.241+0000 7f45be34b700 1 --2- 172.21.15.92:0/3800010382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45b80ceec0 0x7f45b80c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.241+0000 7f45be34b700 1 -- 172.21.15.92:0/3800010382 >> 172.21.15.92:0/3800010382 conn(0x7f45b80995c0 msgr2=0x7f45b8068ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:52.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.241+0000 7f45be34b700 1 -- 172.21.15.92:0/3800010382 shutdown_connections 2023-10-06T10:41:52.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.241+0000 7f45be34b700 1 -- 172.21.15.92:0/3800010382 wait complete. 2023-10-06T10:41:52.255 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T10:41:52.255 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:41:52.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.368+0000 7fd15e899700 1 Processor -- start 2023-10-06T10:41:52.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.368+0000 7fd15e899700 1 -- start start 2023-10-06T10:41:52.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.369+0000 7fd15e899700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd1580b0760 0x7fd1580ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:52.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.369+0000 7fd15e899700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1580b0c60 con 0x7fd1580b49a0 2023-10-06T10:41:52.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.369+0000 7fd15e899700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1580b0f40 con 0x7fd158159560 2023-10-06T10:41:52.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.369+0000 7fd15e899700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1580ab350 con 0x7fd1580b0760 2023-10-06T10:41:52.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.369+0000 7fd1577fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd1580b49a0 0x7fd1580aac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33254/0 (socket says 172.21.15.92:33254) 2023-10-06T10:41:52.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.369+0000 7fd1577fe700 1 -- 172.21.15.92:0/654107385 learned_addr learned my addr 172.21.15.92:0/654107385 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:52.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.369+0000 7fd15ce36700 1 --2- 172.21.15.92:0/654107385 >> v2:172.21.15.124:3300/0 conn(0x7fd1580b0760 0x7fd1580ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:52.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.369+0000 7fd156ffd700 1 -- 172.21.15.92:0/654107385 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3625899785 0 0) 0x7fd1580b0c60 con 0x7fd1580b49a0 2023-10-06T10:41:52.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.369+0000 7fd156ffd700 1 -- 172.21.15.92:0/654107385 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd140003570 con 0x7fd1580b49a0 2023-10-06T10:41:52.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.369+0000 7fd156ffd700 1 -- 172.21.15.92:0/654107385 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 903482502 0 0) 0x7fd1580b0f40 con 0x7fd158159560 2023-10-06T10:41:52.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.369+0000 7fd156ffd700 1 -- 172.21.15.92:0/654107385 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1580b0c60 con 0x7fd158159560 2023-10-06T10:41:52.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.369+0000 7fd15ce36700 1 -- 172.21.15.92:0/654107385 >> v1:172.21.15.124:6789/0 conn(0x7fd158159560 legacy=0x7fd1580aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:52.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.369+0000 7fd15ce36700 1 -- 172.21.15.92:0/654107385 >> v1:172.21.15.92:6789/0 conn(0x7fd1580b49a0 legacy=0x7fd1580aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:52.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.369+0000 7fd156ffd700 1 -- 172.21.15.92:0/654107385 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 939230143 0 0) 0x7fd140003570 con 0x7fd1580b49a0 2023-10-06T10:41:52.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.369+0000 7fd15ce36700 1 -- 172.21.15.92:0/654107385 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1580b1220 con 0x7fd1580b0760 2023-10-06T10:41:52.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.370+0000 7fd15ce36700 1 --2- 172.21.15.92:0/654107385 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1580b0760 0x7fd1580ccf90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd150009e90 tx=0x7fd150009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa104f8d21703a0b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:52.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.370+0000 7fd156ffd700 1 -- 172.21.15.92:0/654107385 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15000e070 con 0x7fd1580b0760 2023-10-06T10:41:52.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.370+0000 7fd156ffd700 1 -- 172.21.15.92:0/654107385 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15000b870 con 0x7fd1580b0760 2023-10-06T10:41:52.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.370+0000 7fd156ffd700 1 -- 172.21.15.92:0/654107385 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c001470 con 0x7fd1580b0760 2023-10-06T10:41:52.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.370+0000 7fd15e899700 1 -- 172.21.15.92:0/654107385 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1580b0760 msgr2=0x7fd1580ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:52.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.370+0000 7fd15e899700 1 --2- 172.21.15.92:0/654107385 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1580b0760 0x7fd1580ccf90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd150009e90 tx=0x7fd150009780 comp rx=0 tx=0).stop 2023-10-06T10:41:52.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.370+0000 7fd15e899700 1 -- 172.21.15.92:0/654107385 shutdown_connections 2023-10-06T10:41:52.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.370+0000 7fd15e899700 1 --2- 172.21.15.92:0/654107385 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1580b0760 0x7fd1580ccf90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.370+0000 7fd15e899700 1 -- 172.21.15.92:0/654107385 >> 172.21.15.92:0/654107385 conn(0x7fd1580997f0 msgr2=0x7fd15806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:52.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.370+0000 7fd15e899700 1 -- 172.21.15.92:0/654107385 shutdown_connections 2023-10-06T10:41:52.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.370+0000 7fd15e899700 1 -- 172.21.15.92:0/654107385 wait complete. 2023-10-06T10:41:52.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.371+0000 7fd15e899700 1 Processor -- start 2023-10-06T10:41:52.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.372+0000 7fd15e899700 1 -- start start 2023-10-06T10:41:52.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.372+0000 7fd15e899700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1580b0760 0x7fd158158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:52.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.372+0000 7fd15e899700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1580b49a0 0x7fd15809aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.372+0000 7fd15e899700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd158159560 0x7fd15815ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.372+0000 7fd15e899700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd158159000 con 0x7fd158159560 2023-10-06T10:41:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.372+0000 7fd15e899700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15815a990 con 0x7fd1580b49a0 2023-10-06T10:41:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.372+0000 7fd15e899700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd158159d80 con 0x7fd1580b0760 2023-10-06T10:41:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.372+0000 7fd15ce36700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd158159560 0x7fd15815ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.373+0000 7fd15ce36700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd158159560 0x7fd15815ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57274/0 (socket says 172.21.15.92:57274) 2023-10-06T10:41:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.373+0000 7fd15ce36700 1 -- 172.21.15.92:0/321426350 learned_addr learned my addr 172.21.15.92:0/321426350 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.373+0000 7fd1577fe700 1 --2- 172.21.15.92:0/321426350 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1580b49a0 0x7fd15809aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.373+0000 7fd157fff700 1 --2- 172.21.15.92:0/321426350 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1580b0760 0x7fd158158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.373+0000 7fd15ce36700 1 -- 172.21.15.92:0/321426350 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1580b0760 msgr2=0x7fd158158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.373+0000 7fd15ce36700 1 --2- 172.21.15.92:0/321426350 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1580b0760 0x7fd158158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.373+0000 7fd15ce36700 1 -- 172.21.15.92:0/321426350 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1580b49a0 msgr2=0x7fd15809aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.373+0000 7fd15ce36700 1 --2- 172.21.15.92:0/321426350 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1580b49a0 0x7fd15809aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.373+0000 7fd15ce36700 1 -- 172.21.15.92:0/321426350 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd150009a20 con 0x7fd158159560 2023-10-06T10:41:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.373+0000 7fd15ce36700 1 --2- 172.21.15.92:0/321426350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd158159560 0x7fd15815ae20 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd1500099f0 tx=0x7fd150017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.373+0000 7fd13ffff700 1 -- 172.21.15.92:0/321426350 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd150012b70 con 0x7fd158159560 2023-10-06T10:41:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.373+0000 7fd13ffff700 1 -- 172.21.15.92:0/321426350 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd150012d10 con 0x7fd158159560 2023-10-06T10:41:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.373+0000 7fd15e899700 1 -- 172.21.15.92:0/321426350 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15815a060 con 0x7fd158159560 2023-10-06T10:41:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.373+0000 7fd13ffff700 1 -- 172.21.15.92:0/321426350 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15001e6a0 con 0x7fd158159560 2023-10-06T10:41:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.373+0000 7fd15e899700 1 -- 172.21.15.92:0/321426350 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd15815a620 con 0x7fd158159560 2023-10-06T10:41:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.374+0000 7fd15e899700 1 -- 172.21.15.92:0/321426350 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd158064a90 con 0x7fd158159560 2023-10-06T10:41:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.375+0000 7fd13ffff700 1 -- 172.21.15.92:0/321426350 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd15001ea90 con 0x7fd158159560 2023-10-06T10:41:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.375+0000 7fd13ffff700 1 --2- 172.21.15.92:0/321426350 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd144066f30 0x7fd1440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.375+0000 7fd13ffff700 1 -- 172.21.15.92:0/321426350 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fd1500868f0 con 0x7fd158159560 2023-10-06T10:41:52.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.375+0000 7fd157fff700 1 --2- 172.21.15.92:0/321426350 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd144066f30 0x7fd1440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:52.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.375+0000 7fd157fff700 1 --2- 172.21.15.92:0/321426350 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd144066f30 0x7fd1440693e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd148003a40 tx=0x7fd148007d40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:52.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.379+0000 7fd13ffff700 1 -- 172.21.15.92:0/321426350 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd150057350 con 0x7fd158159560 2023-10-06T10:41:52.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.558+0000 7fd15e899700 1 -- 172.21.15.92:0/321426350 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd1580b1500 con 0x7fd144066f30 2023-10-06T10:41:52.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.567+0000 7fd13ffff700 1 -- 172.21.15.92:0/321426350 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311733 (secure 0 0 0) 0x7fd1580b1500 con 0x7fd144066f30 2023-10-06T10:41:52.572 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.570+0000 7fd15e899700 1 -- 172.21.15.92:0/321426350 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd144066f30 msgr2=0x7fd1440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.570+0000 7fd15e899700 1 --2- 172.21.15.92:0/321426350 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd144066f30 0x7fd1440693e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd148003a40 tx=0x7fd148007d40 comp rx=0 tx=0).stop 2023-10-06T10:41:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.570+0000 7fd15e899700 1 -- 172.21.15.92:0/321426350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd158159560 msgr2=0x7fd15815ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.570+0000 7fd15e899700 1 --2- 172.21.15.92:0/321426350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd158159560 0x7fd15815ae20 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd1500099f0 tx=0x7fd150017000 comp rx=0 tx=0).stop 2023-10-06T10:41:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.570+0000 7fd15e899700 1 -- 172.21.15.92:0/321426350 shutdown_connections 2023-10-06T10:41:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.570+0000 7fd15e899700 1 --2- 172.21.15.92:0/321426350 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd144066f30 0x7fd1440693e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.570+0000 7fd15e899700 1 --2- 172.21.15.92:0/321426350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd158159560 0x7fd15815ae20 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.570+0000 7fd15e899700 1 --2- 172.21.15.92:0/321426350 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1580b0760 0x7fd158158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.570+0000 7fd15e899700 1 --2- 172.21.15.92:0/321426350 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1580b49a0 0x7fd15809aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.570+0000 7fd15e899700 1 -- 172.21.15.92:0/321426350 >> 172.21.15.92:0/321426350 conn(0x7fd1580997f0 msgr2=0x7fd1580d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:52.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.570+0000 7fd15e899700 1 -- 172.21.15.92:0/321426350 shutdown_connections 2023-10-06T10:41:52.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.570+0000 7fd15e899700 1 -- 172.21.15.92:0/321426350 wait complete. 2023-10-06T10:41:52.576 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:41:52.584 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2023-10-06T10:41:52.129492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385464,"kb_used_data":7160,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752589256,"statfs":{"total":773094113280,"available":770651398144,"internally_reserved":0,"allocated":7331840,"data_stored":4049379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.618633"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.334353+0000","last_change":"2023-10-06T10:41:48.334353+0000","last_active":"2023-10-06T10:41:48.334353+0000","last_peered":"2023-10-06T10:41:48.334353+0000","last_clean":"2023-10-06T10:41:48.334353+0000","last_became_active":"2023-10-06T10:41:48.333910+0000","last_became_peered":"2023-10-06T10:41:48.333910+0000","last_unstale":"2023-10-06T10:41:48.334353+0000","last_undegraded":"2023-10-06T10:41:48.334353+0000","last_fullsized":"2023-10-06T10:41:48.334353+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:33.349366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.336176+0000","last_change":"2023-10-06T10:41:48.336176+0000","last_active":"2023-10-06T10:41:48.336176+0000","last_peered":"2023-10-06T10:41:48.336176+0000","last_clean":"2023-10-06T10:41:48.336176+0000","last_became_active":"2023-10-06T10:41:48.335922+0000","last_became_peered":"2023-10-06T10:41:48.335922+0000","last_unstale":"2023-10-06T10:41:48.336176+0000","last_undegraded":"2023-10-06T10:41:48.336176+0000","last_fullsized":"2023-10-06T10:41:48.336176+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:34:40.831811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.327517+0000","last_change":"2023-10-06T10:41:48.325103+0000","last_active":"2023-10-06T10:41:48.327517+0000","last_peered":"2023-10-06T10:41:48.327517+0000","last_clean":"2023-10-06T10:41:48.327517+0000","last_became_active":"2023-10-06T10:41:48.324843+0000","last_became_peered":"2023-10-06T10:41:48.324843+0000","last_unstale":"2023-10-06T10:41:48.327517+0000","last_undegraded":"2023-10-06T10:41:48.327517+0000","last_fullsized":"2023-10-06T10:41:48.327517+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:59.082862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.328110+0000","last_change":"2023-10-06T10:41:48.328110+0000","last_active":"2023-10-06T10:41:48.328110+0000","last_peered":"2023-10-06T10:41:48.328110+0000","last_clean":"2023-10-06T10:41:48.328110+0000","last_became_active":"2023-10-06T10:41:48.327860+0000","last_became_peered":"2023-10-06T10:41:48.327860+0000","last_unstale":"2023-10-06T10:41:48.328110+0000","last_undegraded":"2023-10-06T10:41:48.328110+0000","last_fullsized":"2023-10-06T10:41:48.328110+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:40:18.488714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.333623+0000","last_change":"2023-10-06T10:41:48.333623+0000","last_active":"2023-10-06T10:41:48.333623+0000","last_peered":"2023-10-06T10:41:48.333623+0000","last_clean":"2023-10-06T10:41:48.333623+0000","last_became_active":"2023-10-06T10:41:48.333368+0000","last_became_peered":"2023-10-06T10:41:48.333368+0000","last_unstale":"2023-10-06T10:41:48.333623+0000","last_undegraded":"2023-10-06T10:41:48.333623+0000","last_fullsized":"2023-10-06T10:41:48.333623+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:08.220000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.337025+0000","last_change":"2023-10-06T10:41:48.337025+0000","last_active":"2023-10-06T10:41:48.337025+0000","last_peered":"2023-10-06T10:41:48.337025+0000","last_clean":"2023-10-06T10:41:48.337025+0000","last_became_active":"2023-10-06T10:41:48.336750+0000","last_became_peered":"2023-10-06T10:41:48.336750+0000","last_unstale":"2023-10-06T10:41:48.337025+0000","last_undegraded":"2023-10-06T10:41:48.337025+0000","last_fullsized":"2023-10-06T10:41:48.337025+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:03:36.216614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.329303+0000","last_change":"2023-10-06T10:41:48.329303+0000","last_active":"2023-10-06T10:41:48.329303+0000","last_peered":"2023-10-06T10:41:48.329303+0000","last_clean":"2023-10-06T10:41:48.329303+0000","last_became_active":"2023-10-06T10:41:48.329041+0000","last_became_peered":"2023-10-06T10:41:48.329041+0000","last_unstale":"2023-10-06T10:41:48.329303+0000","last_undegraded":"2023-10-06T10:41:48.329303+0000","last_fullsized":"2023-10-06T10:41:48.329303+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:30:27.462161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.323257+0000","last_change":"2023-10-06T10:41:48.323257+0000","last_active":"2023-10-06T10:41:48.323257+0000","last_peered":"2023-10-06T10:41:48.323257+0000","last_clean":"2023-10-06T10:41:48.323257+0000","last_became_active":"2023-10-06T10:41:48.323046+0000","last_became_peered":"2023-10-06T10:41:48.323046+0000","last_unstale":"2023-10-06T10:41:48.323257+0000","last_undegraded":"2023-10-06T10:41:48.323257+0000","last_fullsized":"2023-10-06T10:41:48.323257+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:02:12.095392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.329622+0000","last_change":"2023-10-06T10:41:48.329622+0000","last_active":"2023-10-06T10:41:48.329622+0000","last_peered":"2023-10-06T10:41:48.329622+0000","last_clean":"2023-10-06T10:41:48.329622+0000","last_became_active":"2023-10-06T10:41:48.329262+0000","last_became_peered":"2023-10-06T10:41:48.329262+0000","last_unstale":"2023-10-06T10:41:48.329622+0000","last_undegraded":"2023-10-06T10:41:48.329622+0000","last_fullsized":"2023-10-06T10:41:48.329622+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:44:16.796433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.337987+0000","last_change":"2023-10-06T10:41:48.337987+0000","last_active":"2023-10-06T10:41:48.337987+0000","last_peered":"2023-10-06T10:41:48.337987+0000","last_clean":"2023-10-06T10:41:48.337987+0000","last_became_active":"2023-10-06T10:41:48.336173+0000","last_became_peered":"2023-10-06T10:41:48.336173+0000","last_unstale":"2023-10-06T10:41:48.337987+0000","last_undegraded":"2023-10-06T10:41:48.337987+0000","last_fullsized":"2023-10-06T10:41:48.337987+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:07.543490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.337897+0000","last_change":"2023-10-06T10:41:48.337897+0000","last_active":"2023-10-06T10:41:48.337897+0000","last_peered":"2023-10-06T10:41:48.337897+0000","last_clean":"2023-10-06T10:41:48.337897+0000","last_became_active":"2023-10-06T10:41:48.337607+0000","last_became_peered":"2023-10-06T10:41:48.337607+0000","last_unstale":"2023-10-06T10:41:48.337897+0000","last_undegraded":"2023-10-06T10:41:48.337897+0000","last_fullsized":"2023-10-06T10:41:48.337897+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:30:27.462161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.331384+0000","last_change":"2023-10-06T10:41:48.331384+0000","last_active":"2023-10-06T10:41:48.331384+0000","last_peered":"2023-10-06T10:41:48.331384+0000","last_clean":"2023-10-06T10:41:48.331384+0000","last_became_active":"2023-10-06T10:41:48.329149+0000","last_became_peered":"2023-10-06T10:41:48.329149+0000","last_unstale":"2023-10-06T10:41:48.331384+0000","last_undegraded":"2023-10-06T10:41:48.331384+0000","last_fullsized":"2023-10-06T10:41:48.331384+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:44:16.796433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.331741+0000","last_change":"2023-10-06T10:41:48.331741+0000","last_active":"2023-10-06T10:41:48.331741+0000","last_peered":"2023-10-06T10:41:48.331741+0000","last_clean":"2023-10-06T10:41:48.331741+0000","last_became_active":"2023-10-06T10:41:48.331498+0000","last_became_peered":"2023-10-06T10:41:48.331498+0000","last_unstale":"2023-10-06T10:41:48.331741+0000","last_undegraded":"2023-10-06T10:41:48.331741+0000","last_fullsized":"2023-10-06T10:41:48.331741+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:31.491560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.327499+0000","last_change":"2023-10-06T10:41:48.327499+0000","last_active":"2023-10-06T10:41:48.327499+0000","last_peered":"2023-10-06T10:41:48.327499+0000","last_clean":"2023-10-06T10:41:48.327499+0000","last_became_active":"2023-10-06T10:41:48.327214+0000","last_became_peered":"2023-10-06T10:41:48.327214+0000","last_unstale":"2023-10-06T10:41:48.327499+0000","last_undegraded":"2023-10-06T10:41:48.327499+0000","last_fullsized":"2023-10-06T10:41:48.327499+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:50.041063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.322768+0000","last_change":"2023-10-06T10:41:48.322768+0000","last_active":"2023-10-06T10:41:48.322768+0000","last_peered":"2023-10-06T10:41:48.322768+0000","last_clean":"2023-10-06T10:41:48.322768+0000","last_became_active":"2023-10-06T10:41:48.322542+0000","last_became_peered":"2023-10-06T10:41:48.322542+0000","last_unstale":"2023-10-06T10:41:48.322768+0000","last_undegraded":"2023-10-06T10:41:48.322768+0000","last_fullsized":"2023-10-06T10:41:48.322768+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:53:24.847598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.340276+0000","last_change":"2023-10-06T10:41:48.340276+0000","last_active":"2023-10-06T10:41:48.340276+0000","last_peered":"2023-10-06T10:41:48.340276+0000","last_clean":"2023-10-06T10:41:48.340276+0000","last_became_active":"2023-10-06T10:41:48.340010+0000","last_became_peered":"2023-10-06T10:41:48.340010+0000","last_unstale":"2023-10-06T10:41:48.340276+0000","last_undegraded":"2023-10-06T10:41:48.340276+0000","last_fullsized":"2023-10-06T10:41:48.340276+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:37:55.371863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.329511+0000","last_change":"2023-10-06T10:41:48.329511+0000","last_active":"2023-10-06T10:41:48.329511+0000","last_peered":"2023-10-06T10:41:48.329511+0000","last_clean":"2023-10-06T10:41:48.329511+0000","last_became_active":"2023-10-06T10:41:48.328909+0000","last_became_peered":"2023-10-06T10:41:48.328909+0000","last_unstale":"2023-10-06T10:41:48.329511+0000","last_undegraded":"2023-10-06T10:41:48.329511+0000","last_fullsized":"2023-10-06T10:41:48.329511+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:39.629579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.329240+0000","last_change":"2023-10-06T10:41:48.329240+0000","last_active":"2023-10-06T10:41:48.329240+0000","last_peered":"2023-10-06T10:41:48.329240+0000","last_clean":"2023-10-06T10:41:48.329240+0000","last_became_active":"2023-10-06T10:41:48.328443+0000","last_became_peered":"2023-10-06T10:41:48.328443+0000","last_unstale":"2023-10-06T10:41:48.329240+0000","last_undegraded":"2023-10-06T10:41:48.329240+0000","last_fullsized":"2023-10-06T10:41:48.329240+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:58.300560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.324185+0000","last_change":"2023-10-06T10:41:48.324185+0000","last_active":"2023-10-06T10:41:48.324185+0000","last_peered":"2023-10-06T10:41:48.324185+0000","last_clean":"2023-10-06T10:41:48.324185+0000","last_became_active":"2023-10-06T10:41:48.323887+0000","last_became_peered":"2023-10-06T10:41:48.323887+0000","last_unstale":"2023-10-06T10:41:48.324185+0000","last_undegraded":"2023-10-06T10:41:48.324185+0000","last_fullsized":"2023-10-06T10:41:48.324185+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:04:13.357982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.329022+0000","last_change":"2023-10-06T10:41:48.329022+0000","last_active":"2023-10-06T10:41:48.329022+0000","last_peered":"2023-10-06T10:41:48.329022+0000","last_clean":"2023-10-06T10:41:48.329022+0000","last_became_active":"2023-10-06T10:41:48.328775+0000","last_became_peered":"2023-10-06T10:41:48.328775+0000","last_unstale":"2023-10-06T10:41:48.329022+0000","last_undegraded":"2023-10-06T10:41:48.329022+0000","last_fullsized":"2023-10-06T10:41:48.329022+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:41:58.036031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.335405+0000","last_change":"2023-10-06T10:41:48.335405+0000","last_active":"2023-10-06T10:41:48.335405+0000","last_peered":"2023-10-06T10:41:48.335405+0000","last_clean":"2023-10-06T10:41:48.335405+0000","last_became_active":"2023-10-06T10:41:48.335101+0000","last_became_peered":"2023-10-06T10:41:48.335101+0000","last_unstale":"2023-10-06T10:41:48.335405+0000","last_undegraded":"2023-10-06T10:41:48.335405+0000","last_fullsized":"2023-10-06T10:41:48.335405+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:38:45.566420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.337218+0000","last_change":"2023-10-06T10:41:48.337218+0000","last_active":"2023-10-06T10:41:48.337218+0000","last_peered":"2023-10-06T10:41:48.337218+0000","last_clean":"2023-10-06T10:41:48.337218+0000","last_became_active":"2023-10-06T10:41:48.336794+0000","last_became_peered":"2023-10-06T10:41:48.336794+0000","last_unstale":"2023-10-06T10:41:48.337218+0000","last_undegraded":"2023-10-06T10:41:48.337218+0000","last_fullsized":"2023-10-06T10:41:48.337218+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:26:44.645107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.333748+0000","last_change":"2023-10-06T10:41:48.333748+0000","last_active":"2023-10-06T10:41:48.333748+0000","last_peered":"2023-10-06T10:41:48.333748+0000","last_clean":"2023-10-06T10:41:48.333748+0000","last_became_active":"2023-10-06T10:41:48.333347+0000","last_became_peered":"2023-10-06T10:41:48.333347+0000","last_unstale":"2023-10-06T10:41:48.333748+0000","last_undegraded":"2023-10-06T10:41:48.333748+0000","last_fullsized":"2023-10-06T10:41:48.333748+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:56:52.523361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.335423+0000","last_change":"2023-10-06T10:41:48.335423+0000","last_active":"2023-10-06T10:41:48.335423+0000","last_peered":"2023-10-06T10:41:48.335423+0000","last_clean":"2023-10-06T10:41:48.335423+0000","last_became_active":"2023-10-06T10:41:48.335144+0000","last_became_peered":"2023-10-06T10:41:48.335144+0000","last_unstale":"2023-10-06T10:41:48.335423+0000","last_undegraded":"2023-10-06T10:41:48.335423+0000","last_fullsized":"2023-10-06T10:41:48.335423+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:28:14.345962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.334747+0000","last_change":"2023-10-06T10:41:48.334747+0000","last_active":"2023-10-06T10:41:48.334747+0000","last_peered":"2023-10-06T10:41:48.334747+0000","last_clean":"2023-10-06T10:41:48.334747+0000","last_became_active":"2023-10-06T10:41:48.334442+0000","last_became_peered":"2023-10-06T10:41:48.334442+0000","last_unstale":"2023-10-06T10:41:48.334747+0000","last_undegraded":"2023-10-06T10:41:48.334747+0000","last_fullsized":"2023-10-06T10:41:48.334747+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:12.905330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.332027+0000","last_change":"2023-10-06T10:41:48.332027+0000","last_active":"2023-10-06T10:41:48.332027+0000","last_peered":"2023-10-06T10:41:48.332027+0000","last_clean":"2023-10-06T10:41:48.332027+0000","last_became_active":"2023-10-06T10:41:48.331757+0000","last_became_peered":"2023-10-06T10:41:48.331757+0000","last_unstale":"2023-10-06T10:41:48.332027+0000","last_undegraded":"2023-10-06T10:41:48.332027+0000","last_fullsized":"2023-10-06T10:41:48.332027+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:28:14.345962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.335393+0000","last_change":"2023-10-06T10:41:48.333717+0000","last_active":"2023-10-06T10:41:48.335393+0000","last_peered":"2023-10-06T10:41:48.335393+0000","last_clean":"2023-10-06T10:41:48.335393+0000","last_became_active":"2023-10-06T10:41:48.333448+0000","last_became_peered":"2023-10-06T10:41:48.333448+0000","last_unstale":"2023-10-06T10:41:48.335393+0000","last_undegraded":"2023-10-06T10:41:48.335393+0000","last_fullsized":"2023-10-06T10:41:48.335393+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:39.085051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.336543+0000","last_change":"2023-10-06T10:41:48.336543+0000","last_active":"2023-10-06T10:41:48.336543+0000","last_peered":"2023-10-06T10:41:48.336543+0000","last_clean":"2023-10-06T10:41:48.336543+0000","last_became_active":"2023-10-06T10:41:48.336061+0000","last_became_peered":"2023-10-06T10:41:48.336061+0000","last_unstale":"2023-10-06T10:41:48.336543+0000","last_undegraded":"2023-10-06T10:41:48.336543+0000","last_fullsized":"2023-10-06T10:41:48.336543+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:00:35.112716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.329114+0000","last_change":"2023-10-06T10:41:48.326927+0000","last_active":"2023-10-06T10:41:48.329114+0000","last_peered":"2023-10-06T10:41:48.329114+0000","last_clean":"2023-10-06T10:41:48.329114+0000","last_became_active":"2023-10-06T10:41:48.326680+0000","last_became_peered":"2023-10-06T10:41:48.326680+0000","last_unstale":"2023-10-06T10:41:48.329114+0000","last_undegraded":"2023-10-06T10:41:48.329114+0000","last_fullsized":"2023-10-06T10:41:48.329114+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:50.376270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.331738+0000","last_change":"2023-10-06T10:41:48.331738+0000","last_active":"2023-10-06T10:41:48.331738+0000","last_peered":"2023-10-06T10:41:48.331738+0000","last_clean":"2023-10-06T10:41:48.331738+0000","last_became_active":"2023-10-06T10:41:48.331553+0000","last_became_peered":"2023-10-06T10:41:48.331553+0000","last_unstale":"2023-10-06T10:41:48.331738+0000","last_undegraded":"2023-10-06T10:41:48.331738+0000","last_fullsized":"2023-10-06T10:41:48.331738+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:38.254889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.328881+0000","last_change":"2023-10-06T10:41:48.328881+0000","last_active":"2023-10-06T10:41:48.328881+0000","last_peered":"2023-10-06T10:41:48.328881+0000","last_clean":"2023-10-06T10:41:48.328881+0000","last_became_active":"2023-10-06T10:41:48.328606+0000","last_became_peered":"2023-10-06T10:41:48.328606+0000","last_unstale":"2023-10-06T10:41:48.328881+0000","last_undegraded":"2023-10-06T10:41:48.328881+0000","last_fullsized":"2023-10-06T10:41:48.328881+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:43.365081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.328986+0000","last_change":"2023-10-06T10:41:48.328986+0000","last_active":"2023-10-06T10:41:48.328986+0000","last_peered":"2023-10-06T10:41:48.328986+0000","last_clean":"2023-10-06T10:41:48.328986+0000","last_became_active":"2023-10-06T10:41:48.328731+0000","last_became_peered":"2023-10-06T10:41:48.328731+0000","last_unstale":"2023-10-06T10:41:48.328986+0000","last_undegraded":"2023-10-06T10:41:48.328986+0000","last_fullsized":"2023-10-06T10:41:48.328986+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:27.366430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.335582+0000","last_change":"2023-10-06T10:41:48.335582+0000","last_active":"2023-10-06T10:41:48.335582+0000","last_peered":"2023-10-06T10:41:48.335582+0000","last_clean":"2023-10-06T10:41:48.335582+0000","last_became_active":"2023-10-06T10:41:48.334133+0000","last_became_peered":"2023-10-06T10:41:48.334133+0000","last_unstale":"2023-10-06T10:41:48.335582+0000","last_undegraded":"2023-10-06T10:41:48.335582+0000","last_fullsized":"2023-10-06T10:41:48.335582+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:11:17.706840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"63'4","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.545571+0000","last_change":"2023-10-06T10:41:47.528286+0000","last_active":"2023-10-06T10:41:48.545571+0000","last_peered":"2023-10-06T10:41:48.545571+0000","last_clean":"2023-10-06T10:41:48.545571+0000","last_became_active":"2023-10-06T10:41:47.528034+0000","last_became_peered":"2023-10-06T10:41:47.528034+0000","last_unstale":"2023-10-06T10:41:48.545571+0000","last_undegraded":"2023-10-06T10:41:48.545571+0000","last_fullsized":"2023-10-06T10:41:48.545571+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:58.970822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.322155+0000","last_change":"2023-10-06T10:41:48.322155+0000","last_active":"2023-10-06T10:41:48.322155+0000","last_peered":"2023-10-06T10:41:48.322155+0000","last_clean":"2023-10-06T10:41:48.322155+0000","last_became_active":"2023-10-06T10:41:48.321890+0000","last_became_peered":"2023-10-06T10:41:48.321890+0000","last_unstale":"2023-10-06T10:41:48.322155+0000","last_undegraded":"2023-10-06T10:41:48.322155+0000","last_fullsized":"2023-10-06T10:41:48.322155+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:01:21.300442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"63'1","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.317883+0000","last_change":"2023-10-06T10:41:47.540634+0000","last_active":"2023-10-06T10:41:48.317883+0000","last_peered":"2023-10-06T10:41:48.317883+0000","last_clean":"2023-10-06T10:41:48.317883+0000","last_became_active":"2023-10-06T10:41:47.540380+0000","last_became_peered":"2023-10-06T10:41:47.540380+0000","last_unstale":"2023-10-06T10:41:48.317883+0000","last_undegraded":"2023-10-06T10:41:48.317883+0000","last_fullsized":"2023-10-06T10:41:48.317883+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:56:51.516760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.336024+0000","last_change":"2023-10-06T10:41:48.336024+0000","last_active":"2023-10-06T10:41:48.336024+0000","last_peered":"2023-10-06T10:41:48.336024+0000","last_clean":"2023-10-06T10:41:48.336024+0000","last_became_active":"2023-10-06T10:41:48.332793+0000","last_became_peered":"2023-10-06T10:41:48.332793+0000","last_unstale":"2023-10-06T10:41:48.336024+0000","last_undegraded":"2023-10-06T10:41:48.336024+0000","last_fullsized":"2023-10-06T10:41:48.336024+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:38:06.749974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"63'2","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.538496+0000","last_change":"2023-10-06T10:41:47.524576+0000","last_active":"2023-10-06T10:41:48.538496+0000","last_peered":"2023-10-06T10:41:48.538496+0000","last_clean":"2023-10-06T10:41:48.538496+0000","last_became_active":"2023-10-06T10:41:47.524273+0000","last_became_peered":"2023-10-06T10:41:47.524273+0000","last_unstale":"2023-10-06T10:41:48.538496+0000","last_undegraded":"2023-10-06T10:41:48.538496+0000","last_fullsized":"2023-10-06T10:41:48.538496+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:02:11.088791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.338070+0000","last_change":"2023-10-06T10:41:48.338070+0000","last_active":"2023-10-06T10:41:48.338070+0000","last_peered":"2023-10-06T10:41:48.338070+0000","last_clean":"2023-10-06T10:41:48.338070+0000","last_became_active":"2023-10-06T10:41:48.337652+0000","last_became_peered":"2023-10-06T10:41:48.337652+0000","last_unstale":"2023-10-06T10:41:48.338070+0000","last_undegraded":"2023-10-06T10:41:48.338070+0000","last_fullsized":"2023-10-06T10:41:48.338070+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:34:40.831811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"63'2","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.545658+0000","last_change":"2023-10-06T10:41:47.528178+0000","last_active":"2023-10-06T10:41:48.545658+0000","last_peered":"2023-10-06T10:41:48.545658+0000","last_clean":"2023-10-06T10:41:48.545658+0000","last_became_active":"2023-10-06T10:41:47.527686+0000","last_became_peered":"2023-10-06T10:41:47.527686+0000","last_unstale":"2023-10-06T10:41:48.545658+0000","last_undegraded":"2023-10-06T10:41:48.545658+0000","last_fullsized":"2023-10-06T10:41:48.545658+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:06.760378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.327244+0000","last_change":"2023-10-06T10:41:48.327244+0000","last_active":"2023-10-06T10:41:48.327244+0000","last_peered":"2023-10-06T10:41:48.327244+0000","last_clean":"2023-10-06T10:41:48.327244+0000","last_became_active":"2023-10-06T10:41:48.326934+0000","last_became_peered":"2023-10-06T10:41:48.326934+0000","last_unstale":"2023-10-06T10:41:48.327244+0000","last_undegraded":"2023-10-06T10:41:48.327244+0000","last_fullsized":"2023-10-06T10:41:48.327244+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:26:44.645107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.542465+0000","last_change":"2023-10-06T10:41:47.535047+0000","last_active":"2023-10-06T10:41:48.542465+0000","last_peered":"2023-10-06T10:41:48.542465+0000","last_clean":"2023-10-06T10:41:48.542465+0000","last_became_active":"2023-10-06T10:41:47.534834+0000","last_became_peered":"2023-10-06T10:41:47.534834+0000","last_unstale":"2023-10-06T10:41:48.542465+0000","last_undegraded":"2023-10-06T10:41:48.542465+0000","last_fullsized":"2023-10-06T10:41:48.542465+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:38:44.559819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.335371+0000","last_change":"2023-10-06T10:41:48.335371+0000","last_active":"2023-10-06T10:41:48.335371+0000","last_peered":"2023-10-06T10:41:48.335371+0000","last_clean":"2023-10-06T10:41:48.335371+0000","last_became_active":"2023-10-06T10:41:48.335062+0000","last_became_peered":"2023-10-06T10:41:48.335062+0000","last_unstale":"2023-10-06T10:41:48.335371+0000","last_undegraded":"2023-10-06T10:41:48.335371+0000","last_fullsized":"2023-10-06T10:41:48.335371+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:58.647093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"63'1","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.319341+0000","last_change":"2023-10-06T10:41:47.529246+0000","last_active":"2023-10-06T10:41:48.319341+0000","last_peered":"2023-10-06T10:41:48.319341+0000","last_clean":"2023-10-06T10:41:48.319341+0000","last_became_active":"2023-10-06T10:41:47.528977+0000","last_became_peered":"2023-10-06T10:41:47.528977+0000","last_unstale":"2023-10-06T10:41:48.319341+0000","last_undegraded":"2023-10-06T10:41:48.319341+0000","last_fullsized":"2023-10-06T10:41:48.319341+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:14:07.267985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.336874+0000","last_change":"2023-10-06T10:41:48.336874+0000","last_active":"2023-10-06T10:41:48.336874+0000","last_peered":"2023-10-06T10:41:48.336874+0000","last_clean":"2023-10-06T10:41:48.336874+0000","last_became_active":"2023-10-06T10:41:48.336604+0000","last_became_peered":"2023-10-06T10:41:48.336604+0000","last_unstale":"2023-10-06T10:41:48.336874+0000","last_undegraded":"2023-10-06T10:41:48.336874+0000","last_fullsized":"2023-10-06T10:41:48.336874+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:00.973193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.531377+0000","last_change":"2023-10-06T10:41:47.525088+0000","last_active":"2023-10-06T10:41:47.531377+0000","last_peered":"2023-10-06T10:41:47.531377+0000","last_clean":"2023-10-06T10:41:47.531377+0000","last_became_active":"2023-10-06T10:41:47.524813+0000","last_became_peered":"2023-10-06T10:41:47.524813+0000","last_unstale":"2023-10-06T10:41:47.531377+0000","last_undegraded":"2023-10-06T10:41:47.531377+0000","last_fullsized":"2023-10-06T10:41:47.531377+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:29.932628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.331020+0000","last_change":"2023-10-06T10:41:48.331020+0000","last_active":"2023-10-06T10:41:48.331020+0000","last_peered":"2023-10-06T10:41:48.331020+0000","last_clean":"2023-10-06T10:41:48.331020+0000","last_became_active":"2023-10-06T10:41:48.330803+0000","last_became_peered":"2023-10-06T10:41:48.330803+0000","last_unstale":"2023-10-06T10:41:48.331020+0000","last_undegraded":"2023-10-06T10:41:48.331020+0000","last_fullsized":"2023-10-06T10:41:48.331020+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:22:06.050969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.540328+0000","last_change":"2023-10-06T10:41:47.527923+0000","last_active":"2023-10-06T10:41:47.540328+0000","last_peered":"2023-10-06T10:41:47.540328+0000","last_clean":"2023-10-06T10:41:47.540328+0000","last_became_active":"2023-10-06T10:41:47.527668+0000","last_became_peered":"2023-10-06T10:41:47.527668+0000","last_unstale":"2023-10-06T10:41:47.540328+0000","last_undegraded":"2023-10-06T10:41:47.540328+0000","last_fullsized":"2023-10-06T10:41:47.540328+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:50:00.843187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.331182+0000","last_change":"2023-10-06T10:41:48.331182+0000","last_active":"2023-10-06T10:41:48.331182+0000","last_peered":"2023-10-06T10:41:48.331182+0000","last_clean":"2023-10-06T10:41:48.331182+0000","last_became_active":"2023-10-06T10:41:48.330878+0000","last_became_peered":"2023-10-06T10:41:48.330878+0000","last_unstale":"2023-10-06T10:41:48.331182+0000","last_undegraded":"2023-10-06T10:41:48.331182+0000","last_fullsized":"2023-10-06T10:41:48.331182+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:07.543490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.534448+0000","last_change":"2023-10-06T10:41:47.526247+0000","last_active":"2023-10-06T10:41:47.534448+0000","last_peered":"2023-10-06T10:41:47.534448+0000","last_clean":"2023-10-06T10:41:47.534448+0000","last_became_active":"2023-10-06T10:41:47.525980+0000","last_became_peered":"2023-10-06T10:41:47.525980+0000","last_unstale":"2023-10-06T10:41:47.534448+0000","last_undegraded":"2023-10-06T10:41:47.534448+0000","last_fullsized":"2023-10-06T10:41:47.534448+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:29:08.881968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.334515+0000","last_change":"2023-10-06T10:41:48.334515+0000","last_active":"2023-10-06T10:41:48.334515+0000","last_peered":"2023-10-06T10:41:48.334515+0000","last_clean":"2023-10-06T10:41:48.334515+0000","last_became_active":"2023-10-06T10:41:48.334078+0000","last_became_peered":"2023-10-06T10:41:48.334078+0000","last_unstale":"2023-10-06T10:41:48.334515+0000","last_undegraded":"2023-10-06T10:41:48.334515+0000","last_fullsized":"2023-10-06T10:41:48.334515+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:38.254889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.539924+0000","last_change":"2023-10-06T10:41:47.529220+0000","last_active":"2023-10-06T10:41:47.539924+0000","last_peered":"2023-10-06T10:41:47.539924+0000","last_clean":"2023-10-06T10:41:47.539924+0000","last_became_active":"2023-10-06T10:41:47.528931+0000","last_became_peered":"2023-10-06T10:41:47.528931+0000","last_unstale":"2023-10-06T10:41:47.539924+0000","last_undegraded":"2023-10-06T10:41:47.539924+0000","last_fullsized":"2023-10-06T10:41:47.539924+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:41:31.882577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.330952+0000","last_change":"2023-10-06T10:41:48.330952+0000","last_active":"2023-10-06T10:41:48.330952+0000","last_peered":"2023-10-06T10:41:48.330952+0000","last_clean":"2023-10-06T10:41:48.330952+0000","last_became_active":"2023-10-06T10:41:48.330657+0000","last_became_peered":"2023-10-06T10:41:48.330657+0000","last_unstale":"2023-10-06T10:41:48.330952+0000","last_undegraded":"2023-10-06T10:41:48.330952+0000","last_fullsized":"2023-10-06T10:41:48.330952+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:29:09.888570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"63'3","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.545343+0000","last_change":"2023-10-06T10:41:47.527983+0000","last_active":"2023-10-06T10:41:48.545343+0000","last_peered":"2023-10-06T10:41:48.545343+0000","last_clean":"2023-10-06T10:41:48.545343+0000","last_became_active":"2023-10-06T10:41:47.527703+0000","last_became_peered":"2023-10-06T10:41:47.527703+0000","last_unstale":"2023-10-06T10:41:48.545343+0000","last_undegraded":"2023-10-06T10:41:48.545343+0000","last_fullsized":"2023-10-06T10:41:48.545343+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:37:54.365261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.322040+0000","last_change":"2023-10-06T10:41:48.322040+0000","last_active":"2023-10-06T10:41:48.322040+0000","last_peered":"2023-10-06T10:41:48.322040+0000","last_clean":"2023-10-06T10:41:48.322040+0000","last_became_active":"2023-10-06T10:41:48.321689+0000","last_became_peered":"2023-10-06T10:41:48.321689+0000","last_unstale":"2023-10-06T10:41:48.322040+0000","last_undegraded":"2023-10-06T10:41:48.322040+0000","last_fullsized":"2023-10-06T10:41:48.322040+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:17:42.162415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"63'6","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.545070+0000","last_change":"2023-10-06T10:41:47.531271+0000","last_active":"2023-10-06T10:41:48.545070+0000","last_peered":"2023-10-06T10:41:48.545070+0000","last_clean":"2023-10-06T10:41:48.545070+0000","last_became_active":"2023-10-06T10:41:47.531031+0000","last_became_peered":"2023-10-06T10:41:47.531031+0000","last_unstale":"2023-10-06T10:41:48.545070+0000","last_undegraded":"2023-10-06T10:41:48.545070+0000","last_fullsized":"2023-10-06T10:41:48.545070+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:23:54.319502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.333553+0000","last_change":"2023-10-06T10:41:48.333553+0000","last_active":"2023-10-06T10:41:48.333553+0000","last_peered":"2023-10-06T10:41:48.333553+0000","last_clean":"2023-10-06T10:41:48.333553+0000","last_became_active":"2023-10-06T10:41:48.333242+0000","last_became_peered":"2023-10-06T10:41:48.333242+0000","last_unstale":"2023-10-06T10:41:48.333553+0000","last_undegraded":"2023-10-06T10:41:48.333553+0000","last_fullsized":"2023-10-06T10:41:48.333553+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:03:36.216614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"63'2","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.543243+0000","last_change":"2023-10-06T10:41:47.530477+0000","last_active":"2023-10-06T10:41:48.543243+0000","last_peered":"2023-10-06T10:41:48.543243+0000","last_clean":"2023-10-06T10:41:48.543243+0000","last_became_active":"2023-10-06T10:41:47.529579+0000","last_became_peered":"2023-10-06T10:41:47.529579+0000","last_unstale":"2023-10-06T10:41:48.543243+0000","last_undegraded":"2023-10-06T10:41:48.543243+0000","last_fullsized":"2023-10-06T10:41:48.543243+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:36.835370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.337721+0000","last_change":"2023-10-06T10:41:48.337721+0000","last_active":"2023-10-06T10:41:48.337721+0000","last_peered":"2023-10-06T10:41:48.337721+0000","last_clean":"2023-10-06T10:41:48.337721+0000","last_became_active":"2023-10-06T10:41:48.337460+0000","last_became_peered":"2023-10-06T10:41:48.337460+0000","last_unstale":"2023-10-06T10:41:48.337721+0000","last_undegraded":"2023-10-06T10:41:48.337721+0000","last_fullsized":"2023-10-06T10:41:48.337721+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:56:52.523361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.539984+0000","last_change":"2023-10-06T10:41:47.531430+0000","last_active":"2023-10-06T10:41:47.539984+0000","last_peered":"2023-10-06T10:41:47.539984+0000","last_clean":"2023-10-06T10:41:47.539984+0000","last_became_active":"2023-10-06T10:41:47.531123+0000","last_became_peered":"2023-10-06T10:41:47.531123+0000","last_unstale":"2023-10-06T10:41:47.539984+0000","last_undegraded":"2023-10-06T10:41:47.539984+0000","last_fullsized":"2023-10-06T10:41:47.539984+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:44:15.789832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.330145+0000","last_change":"2023-10-06T10:41:48.330145+0000","last_active":"2023-10-06T10:41:48.330145+0000","last_peered":"2023-10-06T10:41:48.330145+0000","last_clean":"2023-10-06T10:41:48.330145+0000","last_became_active":"2023-10-06T10:41:48.329914+0000","last_became_peered":"2023-10-06T10:41:48.329914+0000","last_unstale":"2023-10-06T10:41:48.330145+0000","last_undegraded":"2023-10-06T10:41:48.330145+0000","last_fullsized":"2023-10-06T10:41:48.330145+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:23:52.001648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.539844+0000","last_change":"2023-10-06T10:41:47.529634+0000","last_active":"2023-10-06T10:41:47.539844+0000","last_peered":"2023-10-06T10:41:47.539844+0000","last_clean":"2023-10-06T10:41:47.539844+0000","last_became_active":"2023-10-06T10:41:47.528952+0000","last_became_peered":"2023-10-06T10:41:47.528952+0000","last_unstale":"2023-10-06T10:41:47.539844+0000","last_undegraded":"2023-10-06T10:41:47.539844+0000","last_fullsized":"2023-10-06T10:41:47.539844+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:47:44.875853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.322014+0000","last_change":"2023-10-06T10:41:48.322014+0000","last_active":"2023-10-06T10:41:48.322014+0000","last_peered":"2023-10-06T10:41:48.322014+0000","last_clean":"2023-10-06T10:41:48.322014+0000","last_became_active":"2023-10-06T10:41:48.321625+0000","last_became_peered":"2023-10-06T10:41:48.321625+0000","last_unstale":"2023-10-06T10:41:48.322014+0000","last_undegraded":"2023-10-06T10:41:48.322014+0000","last_fullsized":"2023-10-06T10:41:48.322014+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:58.300560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"63'8","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.546236+0000","last_change":"2023-10-06T10:41:47.528391+0000","last_active":"2023-10-06T10:41:48.546236+0000","last_peered":"2023-10-06T10:41:48.546236+0000","last_clean":"2023-10-06T10:41:48.546236+0000","last_became_active":"2023-10-06T10:41:47.528090+0000","last_became_peered":"2023-10-06T10:41:47.528090+0000","last_unstale":"2023-10-06T10:41:48.546236+0000","last_undegraded":"2023-10-06T10:41:48.546236+0000","last_fullsized":"2023-10-06T10:41:48.546236+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:27.657374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.323299+0000","last_change":"2023-10-06T10:41:48.323299+0000","last_active":"2023-10-06T10:41:48.323299+0000","last_peered":"2023-10-06T10:41:48.323299+0000","last_clean":"2023-10-06T10:41:48.323299+0000","last_became_active":"2023-10-06T10:41:48.323069+0000","last_became_peered":"2023-10-06T10:41:48.323069+0000","last_unstale":"2023-10-06T10:41:48.323299+0000","last_undegraded":"2023-10-06T10:41:48.323299+0000","last_fullsized":"2023-10-06T10:41:48.323299+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:45.653458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"63'2","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.545657+0000","last_change":"2023-10-06T10:41:47.530883+0000","last_active":"2023-10-06T10:41:48.545657+0000","last_peered":"2023-10-06T10:41:48.545657+0000","last_clean":"2023-10-06T10:41:48.545657+0000","last_became_active":"2023-10-06T10:41:47.530598+0000","last_became_peered":"2023-10-06T10:41:47.530598+0000","last_unstale":"2023-10-06T10:41:48.545657+0000","last_undegraded":"2023-10-06T10:41:48.545657+0000","last_fullsized":"2023-10-06T10:41:48.545657+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:49.034461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.328913+0000","last_change":"2023-10-06T10:41:48.328913+0000","last_active":"2023-10-06T10:41:48.328913+0000","last_peered":"2023-10-06T10:41:48.328913+0000","last_clean":"2023-10-06T10:41:48.328913+0000","last_became_active":"2023-10-06T10:41:48.328627+0000","last_became_peered":"2023-10-06T10:41:48.328627+0000","last_unstale":"2023-10-06T10:41:48.328913+0000","last_undegraded":"2023-10-06T10:41:48.328913+0000","last_fullsized":"2023-10-06T10:41:48.328913+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:18.730732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.539766+0000","last_change":"2023-10-06T10:41:47.529290+0000","last_active":"2023-10-06T10:41:47.539766+0000","last_peered":"2023-10-06T10:41:47.539766+0000","last_clean":"2023-10-06T10:41:47.539766+0000","last_became_active":"2023-10-06T10:41:47.529007+0000","last_became_peered":"2023-10-06T10:41:47.529007+0000","last_unstale":"2023-10-06T10:41:47.539766+0000","last_undegraded":"2023-10-06T10:41:47.539766+0000","last_fullsized":"2023-10-06T10:41:47.539766+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:58:28.156731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.333979+0000","last_change":"2023-10-06T10:41:48.333979+0000","last_active":"2023-10-06T10:41:48.333979+0000","last_peered":"2023-10-06T10:41:48.333979+0000","last_clean":"2023-10-06T10:41:48.333979+0000","last_became_active":"2023-10-06T10:41:48.333451+0000","last_became_peered":"2023-10-06T10:41:48.333451+0000","last_unstale":"2023-10-06T10:41:48.333979+0000","last_undegraded":"2023-10-06T10:41:48.333979+0000","last_fullsized":"2023-10-06T10:41:48.333979+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:43.365081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.540431+0000","last_change":"2023-10-06T10:41:47.532259+0000","last_active":"2023-10-06T10:41:47.540431+0000","last_peered":"2023-10-06T10:41:47.540431+0000","last_clean":"2023-10-06T10:41:47.540431+0000","last_became_active":"2023-10-06T10:41:47.531994+0000","last_became_peered":"2023-10-06T10:41:47.531994+0000","last_unstale":"2023-10-06T10:41:47.540431+0000","last_undegraded":"2023-10-06T10:41:47.540431+0000","last_fullsized":"2023-10-06T10:41:47.540431+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:36.921171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.338250+0000","last_change":"2023-10-06T10:41:48.338250+0000","last_active":"2023-10-06T10:41:48.338250+0000","last_peered":"2023-10-06T10:41:48.338250+0000","last_clean":"2023-10-06T10:41:48.338250+0000","last_became_active":"2023-10-06T10:41:48.337971+0000","last_became_peered":"2023-10-06T10:41:48.337971+0000","last_unstale":"2023-10-06T10:41:48.338250+0000","last_undegraded":"2023-10-06T10:41:48.338250+0000","last_fullsized":"2023-10-06T10:41:48.338250+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:18.730732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.534055+0000","last_change":"2023-10-06T10:41:47.530446+0000","last_active":"2023-10-06T10:41:47.534055+0000","last_peered":"2023-10-06T10:41:47.534055+0000","last_clean":"2023-10-06T10:41:47.534055+0000","last_became_active":"2023-10-06T10:41:47.530155+0000","last_became_peered":"2023-10-06T10:41:47.530155+0000","last_unstale":"2023-10-06T10:41:47.534055+0000","last_undegraded":"2023-10-06T10:41:47.534055+0000","last_fullsized":"2023-10-06T10:41:47.534055+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:28:13.339361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.322286+0000","last_change":"2023-10-06T10:41:48.322286+0000","last_active":"2023-10-06T10:41:48.322286+0000","last_peered":"2023-10-06T10:41:48.322286+0000","last_clean":"2023-10-06T10:41:48.322286+0000","last_became_active":"2023-10-06T10:41:48.322042+0000","last_became_peered":"2023-10-06T10:41:48.322042+0000","last_unstale":"2023-10-06T10:41:48.322286+0000","last_undegraded":"2023-10-06T10:41:48.322286+0000","last_fullsized":"2023-10-06T10:41:48.322286+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:40:18.488714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.539571+0000","last_change":"2023-10-06T10:41:47.526231+0000","last_active":"2023-10-06T10:41:47.539571+0000","last_peered":"2023-10-06T10:41:47.539571+0000","last_clean":"2023-10-06T10:41:47.539571+0000","last_became_active":"2023-10-06T10:41:47.525260+0000","last_became_peered":"2023-10-06T10:41:47.525260+0000","last_unstale":"2023-10-06T10:41:47.539571+0000","last_undegraded":"2023-10-06T10:41:47.539571+0000","last_fullsized":"2023-10-06T10:41:47.539571+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:54:04.991122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.335071+0000","last_change":"2023-10-06T10:41:48.335071+0000","last_active":"2023-10-06T10:41:48.335071+0000","last_peered":"2023-10-06T10:41:48.335071+0000","last_clean":"2023-10-06T10:41:48.335071+0000","last_became_active":"2023-10-06T10:41:48.334778+0000","last_became_peered":"2023-10-06T10:41:48.334778+0000","last_unstale":"2023-10-06T10:41:48.335071+0000","last_undegraded":"2023-10-06T10:41:48.335071+0000","last_fullsized":"2023-10-06T10:41:48.335071+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:33.349366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.530973+0000","last_change":"2023-10-06T10:41:47.525929+0000","last_active":"2023-10-06T10:41:47.530973+0000","last_peered":"2023-10-06T10:41:47.530973+0000","last_clean":"2023-10-06T10:41:47.530973+0000","last_became_active":"2023-10-06T10:41:47.525660+0000","last_became_peered":"2023-10-06T10:41:47.525660+0000","last_unstale":"2023-10-06T10:41:47.530973+0000","last_undegraded":"2023-10-06T10:41:47.530973+0000","last_fullsized":"2023-10-06T10:41:47.530973+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:25:37.651048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.326241+0000","last_change":"2023-10-06T10:41:48.326241+0000","last_active":"2023-10-06T10:41:48.326241+0000","last_peered":"2023-10-06T10:41:48.326241+0000","last_clean":"2023-10-06T10:41:48.326241+0000","last_became_active":"2023-10-06T10:41:48.326021+0000","last_became_peered":"2023-10-06T10:41:48.326021+0000","last_unstale":"2023-10-06T10:41:48.326241+0000","last_undegraded":"2023-10-06T10:41:48.326241+0000","last_fullsized":"2023-10-06T10:41:48.326241+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:50.041063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.534537+0000","last_change":"2023-10-06T10:41:47.528732+0000","last_active":"2023-10-06T10:41:47.534537+0000","last_peered":"2023-10-06T10:41:47.534537+0000","last_clean":"2023-10-06T10:41:47.534537+0000","last_became_active":"2023-10-06T10:41:47.528481+0000","last_became_peered":"2023-10-06T10:41:47.528481+0000","last_unstale":"2023-10-06T10:41:47.534537+0000","last_undegraded":"2023-10-06T10:41:47.534537+0000","last_fullsized":"2023-10-06T10:41:47.534537+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:03:35.210012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.330679+0000","last_change":"2023-10-06T10:41:48.330679+0000","last_active":"2023-10-06T10:41:48.330679+0000","last_peered":"2023-10-06T10:41:48.330679+0000","last_clean":"2023-10-06T10:41:48.330679+0000","last_became_active":"2023-10-06T10:41:48.330394+0000","last_became_peered":"2023-10-06T10:41:48.330394+0000","last_unstale":"2023-10-06T10:41:48.330679+0000","last_undegraded":"2023-10-06T10:41:48.330679+0000","last_fullsized":"2023-10-06T10:41:48.330679+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:11:17.706840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.534404+0000","last_change":"2023-10-06T10:41:47.526181+0000","last_active":"2023-10-06T10:41:47.534404+0000","last_peered":"2023-10-06T10:41:47.534404+0000","last_clean":"2023-10-06T10:41:47.534404+0000","last_became_active":"2023-10-06T10:41:47.525930+0000","last_became_peered":"2023-10-06T10:41:47.525930+0000","last_unstale":"2023-10-06T10:41:47.534404+0000","last_undegraded":"2023-10-06T10:41:47.534404+0000","last_fullsized":"2023-10-06T10:41:47.534404+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:12:00.503217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.334169+0000","last_change":"2023-10-06T10:41:48.334169+0000","last_active":"2023-10-06T10:41:48.334169+0000","last_peered":"2023-10-06T10:41:48.334169+0000","last_clean":"2023-10-06T10:41:48.334169+0000","last_became_active":"2023-10-06T10:41:48.333660+0000","last_became_peered":"2023-10-06T10:41:48.333660+0000","last_unstale":"2023-10-06T10:41:48.334169+0000","last_undegraded":"2023-10-06T10:41:48.334169+0000","last_fullsized":"2023-10-06T10:41:48.334169+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:02:12.095392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"63'2","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.538484+0000","last_change":"2023-10-06T10:41:47.531157+0000","last_active":"2023-10-06T10:41:48.538484+0000","last_peered":"2023-10-06T10:41:48.538484+0000","last_clean":"2023-10-06T10:41:48.538484+0000","last_became_active":"2023-10-06T10:41:47.530899+0000","last_became_peered":"2023-10-06T10:41:47.530899+0000","last_unstale":"2023-10-06T10:41:48.538484+0000","last_undegraded":"2023-10-06T10:41:48.538484+0000","last_fullsized":"2023-10-06T10:41:48.538484+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:11:16.700239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":53,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.520894+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:41:47.520894+0000","last_peered":"2023-10-06T10:41:47.520894+0000","last_clean":"2023-10-06T10:41:47.520894+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:41:47.520894+0000","last_undegraded":"2023-10-06T10:41:47.520894+0000","last_fullsized":"2023-10-06T10:41:47.520894+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.338317+0000","last_change":"2023-10-06T10:41:48.338317+0000","last_active":"2023-10-06T10:41:48.338317+0000","last_peered":"2023-10-06T10:41:48.338317+0000","last_clean":"2023-10-06T10:41:48.338317+0000","last_became_active":"2023-10-06T10:41:48.337659+0000","last_became_peered":"2023-10-06T10:41:48.337659+0000","last_unstale":"2023-10-06T10:41:48.338317+0000","last_undegraded":"2023-10-06T10:41:48.338317+0000","last_fullsized":"2023-10-06T10:41:48.338317+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:28:14.345962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"63'1","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.319500+0000","last_change":"2023-10-06T10:41:47.530910+0000","last_active":"2023-10-06T10:41:48.319500+0000","last_peered":"2023-10-06T10:41:48.319500+0000","last_clean":"2023-10-06T10:41:48.319500+0000","last_became_active":"2023-10-06T10:41:47.530622+0000","last_became_peered":"2023-10-06T10:41:47.530622+0000","last_unstale":"2023-10-06T10:41:48.319500+0000","last_undegraded":"2023-10-06T10:41:48.319500+0000","last_fullsized":"2023-10-06T10:41:48.319500+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:01:20.293841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":48,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.319582+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:41:48.319582+0000","last_peered":"2023-10-06T10:41:48.319582+0000","last_clean":"2023-10-06T10:41:48.319582+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:41:48.319582+0000","last_undegraded":"2023-10-06T10:41:48.319582+0000","last_fullsized":"2023-10-06T10:41:48.319582+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.328008+0000","last_change":"2023-10-06T10:41:48.325930+0000","last_active":"2023-10-06T10:41:48.328008+0000","last_peered":"2023-10-06T10:41:48.328008+0000","last_clean":"2023-10-06T10:41:48.328008+0000","last_became_active":"2023-10-06T10:41:48.325650+0000","last_became_peered":"2023-10-06T10:41:48.325650+0000","last_unstale":"2023-10-06T10:41:48.328008+0000","last_undegraded":"2023-10-06T10:41:48.328008+0000","last_fullsized":"2023-10-06T10:41:48.328008+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:28.883834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.545786+0000","last_change":"2023-10-06T10:41:47.540263+0000","last_active":"2023-10-06T10:41:47.545786+0000","last_peered":"2023-10-06T10:41:47.545786+0000","last_clean":"2023-10-06T10:41:47.545786+0000","last_became_active":"2023-10-06T10:41:47.539939+0000","last_became_peered":"2023-10-06T10:41:47.539939+0000","last_unstale":"2023-10-06T10:41:47.545786+0000","last_undegraded":"2023-10-06T10:41:47.545786+0000","last_fullsized":"2023-10-06T10:41:47.545786+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:37.248288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.513961+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:41:47.513961+0000","last_peered":"2023-10-06T10:41:47.513961+0000","last_clean":"2023-10-06T10:41:47.513961+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:41:47.513961+0000","last_undegraded":"2023-10-06T10:41:47.513961+0000","last_fullsized":"2023-10-06T10:41:47.513961+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":194,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.518263+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:41:47.518263+0000","last_peered":"2023-10-06T10:41:47.518263+0000","last_clean":"2023-10-06T10:41:47.518263+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:41:47.518263+0000","last_undegraded":"2023-10-06T10:41:47.518263+0000","last_fullsized":"2023-10-06T10:41:47.518263+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.332796+0000","last_change":"2023-10-06T10:41:48.330936+0000","last_active":"2023-10-06T10:41:48.332796+0000","last_peered":"2023-10-06T10:41:48.332796+0000","last_clean":"2023-10-06T10:41:48.332796+0000","last_became_active":"2023-10-06T10:41:48.330681+0000","last_became_peered":"2023-10-06T10:41:48.330681+0000","last_unstale":"2023-10-06T10:41:48.332796+0000","last_undegraded":"2023-10-06T10:41:48.332796+0000","last_fullsized":"2023-10-06T10:41:48.332796+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:54:03.085432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.534187+0000","last_change":"2023-10-06T10:41:47.528684+0000","last_active":"2023-10-06T10:41:47.534187+0000","last_peered":"2023-10-06T10:41:47.534187+0000","last_clean":"2023-10-06T10:41:47.534187+0000","last_became_active":"2023-10-06T10:41:47.528423+0000","last_became_peered":"2023-10-06T10:41:47.528423+0000","last_unstale":"2023-10-06T10:41:47.534187+0000","last_undegraded":"2023-10-06T10:41:47.534187+0000","last_fullsized":"2023-10-06T10:41:47.534187+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:04:12.351380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":44,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.520537+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:41:47.520537+0000","last_peered":"2023-10-06T10:41:47.520537+0000","last_clean":"2023-10-06T10:41:47.520537+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:41:47.520537+0000","last_undegraded":"2023-10-06T10:41:47.520537+0000","last_fullsized":"2023-10-06T10:41:47.520537+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.327973+0000","last_change":"2023-10-06T10:41:48.325694+0000","last_active":"2023-10-06T10:41:48.327973+0000","last_peered":"2023-10-06T10:41:48.327973+0000","last_clean":"2023-10-06T10:41:48.327973+0000","last_became_active":"2023-10-06T10:41:48.325486+0000","last_became_peered":"2023-10-06T10:41:48.325486+0000","last_unstale":"2023-10-06T10:41:48.327973+0000","last_undegraded":"2023-10-06T10:41:48.327973+0000","last_fullsized":"2023-10-06T10:41:48.327973+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:33:38.497872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"63'2","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.545729+0000","last_change":"2023-10-06T10:41:47.526416+0000","last_active":"2023-10-06T10:41:48.545729+0000","last_peered":"2023-10-06T10:41:48.545729+0000","last_clean":"2023-10-06T10:41:48.545729+0000","last_became_active":"2023-10-06T10:41:47.526071+0000","last_became_peered":"2023-10-06T10:41:47.526071+0000","last_unstale":"2023-10-06T10:41:48.545729+0000","last_undegraded":"2023-10-06T10:41:48.545729+0000","last_fullsized":"2023-10-06T10:41:48.545729+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:24.524122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":44,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.518145+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:41:47.518145+0000","last_peered":"2023-10-06T10:41:47.518145+0000","last_clean":"2023-10-06T10:41:47.518145+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:41:47.518145+0000","last_undegraded":"2023-10-06T10:41:47.518145+0000","last_fullsized":"2023-10-06T10:41:47.518145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.326164+0000","last_change":"2023-10-06T10:41:48.323837+0000","last_active":"2023-10-06T10:41:48.326164+0000","last_peered":"2023-10-06T10:41:48.326164+0000","last_clean":"2023-10-06T10:41:48.326164+0000","last_became_active":"2023-10-06T10:41:48.323293+0000","last_became_peered":"2023-10-06T10:41:48.323293+0000","last_unstale":"2023-10-06T10:41:48.326164+0000","last_undegraded":"2023-10-06T10:41:48.326164+0000","last_fullsized":"2023-10-06T10:41:48.326164+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:53:57.341709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.539841+0000","last_change":"2023-10-06T10:41:47.529077+0000","last_active":"2023-10-06T10:41:47.539841+0000","last_peered":"2023-10-06T10:41:47.539841+0000","last_clean":"2023-10-06T10:41:47.539841+0000","last_became_active":"2023-10-06T10:41:47.528659+0000","last_became_peered":"2023-10-06T10:41:47.528659+0000","last_unstale":"2023-10-06T10:41:47.539841+0000","last_undegraded":"2023-10-06T10:41:47.539841+0000","last_fullsized":"2023-10-06T10:41:47.539841+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:38.622977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.515837+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:41:47.515837+0000","last_peered":"2023-10-06T10:41:47.515837+0000","last_clean":"2023-10-06T10:41:47.515837+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:41:47.515837+0000","last_undegraded":"2023-10-06T10:41:47.515837+0000","last_fullsized":"2023-10-06T10:41:47.515837+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.322054+0000","last_change":"2023-10-06T10:41:48.322054+0000","last_active":"2023-10-06T10:41:48.322054+0000","last_peered":"2023-10-06T10:41:48.322054+0000","last_clean":"2023-10-06T10:41:48.322054+0000","last_became_active":"2023-10-06T10:41:48.321825+0000","last_became_peered":"2023-10-06T10:41:48.321825+0000","last_unstale":"2023-10-06T10:41:48.322054+0000","last_undegraded":"2023-10-06T10:41:48.322054+0000","last_fullsized":"2023-10-06T10:41:48.322054+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:39.629579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"63'1","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.319265+0000","last_change":"2023-10-06T10:41:47.527450+0000","last_active":"2023-10-06T10:41:48.319265+0000","last_peered":"2023-10-06T10:41:48.319265+0000","last_clean":"2023-10-06T10:41:48.319265+0000","last_became_active":"2023-10-06T10:41:47.526356+0000","last_became_peered":"2023-10-06T10:41:47.526356+0000","last_unstale":"2023-10-06T10:41:48.319265+0000","last_undegraded":"2023-10-06T10:41:48.319265+0000","last_fullsized":"2023-10-06T10:41:48.319265+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:57.293959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":48,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.522694+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:41:47.522694+0000","last_peered":"2023-10-06T10:41:47.522694+0000","last_clean":"2023-10-06T10:41:47.522694+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:41:47.522694+0000","last_undegraded":"2023-10-06T10:41:47.522694+0000","last_fullsized":"2023-10-06T10:41:47.522694+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.336900+0000","last_change":"2023-10-06T10:41:48.336900+0000","last_active":"2023-10-06T10:41:48.336900+0000","last_peered":"2023-10-06T10:41:48.336900+0000","last_clean":"2023-10-06T10:41:48.336900+0000","last_became_active":"2023-10-06T10:41:48.336683+0000","last_became_peered":"2023-10-06T10:41:48.336683+0000","last_unstale":"2023-10-06T10:41:48.336900+0000","last_undegraded":"2023-10-06T10:41:48.336900+0000","last_fullsized":"2023-10-06T10:41:48.336900+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:41:32.889178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.531772+0000","last_change":"2023-10-06T10:41:47.528008+0000","last_active":"2023-10-06T10:41:47.531772+0000","last_peered":"2023-10-06T10:41:47.531772+0000","last_clean":"2023-10-06T10:41:47.531772+0000","last_became_active":"2023-10-06T10:41:47.527743+0000","last_became_peered":"2023-10-06T10:41:47.527743+0000","last_unstale":"2023-10-06T10:41:47.531772+0000","last_undegraded":"2023-10-06T10:41:47.531772+0000","last_fullsized":"2023-10-06T10:41:47.531772+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:26:00.759449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":48,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.522511+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:41:47.522511+0000","last_peered":"2023-10-06T10:41:47.522511+0000","last_clean":"2023-10-06T10:41:47.522511+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:41:47.522511+0000","last_undegraded":"2023-10-06T10:41:47.522511+0000","last_fullsized":"2023-10-06T10:41:47.522511+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298620,"kb_used_data":1332,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073220,"statfs":{"total":96636764160,"available":96330977280,"internally_reserved":0,"allocated":1363968,"data_stored":949321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.49399999999999999,"15min":0.48199999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.50900000000000001,"15min":0.51100000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76100000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.41399999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47099999999999997,"15min":0.45900000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.63300000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.44},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.51000000000000001,"15min":0.502},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.33800000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.42499999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.747,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.36699999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.52200000000000002,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55300000000000005,"15min":0.54400000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.55400000000000005}]},{"osd":4,"last update":"Fri Oct 6 10:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.46000000000000002,"15min":0.47399999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.503,"15min":0.50900000000000001},"min":{"1min":0.29599999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.51200000000000001}]},{"osd":5,"last update":"Fri Oct 6 10:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66500000000000004,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.502},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54000000000000004,"15min":0.55100000000000005},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.54100000000000004}]},{"osd":6,"last update":"Fri Oct 6 10:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56000000000000005,"15min":0.57399999999999995},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.77700000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.56399999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298036,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073804,"statfs":{"total":96636764160,"available":96331575296,"internally_reserved":0,"allocated":765952,"data_stored":357104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.36199999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.68400000000000005,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.499},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.45900000000000002,"15min":0.434},"min":{"1min":0.41999999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.60599999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.57599999999999996}]},{"osd":1,"last update":"Fri Oct 6 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.46400000000000002,"15min":0.441},"min":{"1min":0.32200000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.69499999999999995,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.49399999999999999,"15min":0.47199999999999998},"min":{"1min":0.42499999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.66200000000000003,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.434}]},{"osd":2,"last update":"Fri Oct 6 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48699999999999999,"15min":0.47099999999999997},"min":{"1min":0.35899999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.64200000000000002,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.48999999999999999,"15min":0.46800000000000003},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.63200000000000001,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.65300000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.50600000000000001,"15min":0.49099999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.66500000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52200000000000002,"15min":0.50800000000000001},"min":{"1min":0.40100000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65300000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.55600000000000005}]},{"osd":4,"last update":"Fri Oct 6 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.69799999999999995,"5min":0.745,"15min":0.745},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.48099999999999998,"15min":0.48299999999999998},"min":{"1min":0.31900000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.72099999999999997,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.48499999999999999,"15min":0.48399999999999999},"min":{"1min":0.33700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.67400000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.36299999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.73699999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.63900000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.373,"5min":0.313,"15min":0.313},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.39100000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70799999999999996,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.627}]}]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298632,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073208,"statfs":{"total":96636764160,"available":96330964992,"internally_reserved":0,"allocated":1376256,"data_stored":950857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33100000000000002,"15min":0.32900000000000001},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.35399999999999998,"15min":0.34799999999999998},"min":{"1min":0.26400000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.28000000000000003}]},{"osd":2,"last update":"Fri Oct 6 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.33200000000000002,"15min":0.32100000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.374,"15min":0.36399999999999999},"min":{"1min":0.255,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.35899999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.39000000000000001,"15min":0.378},"min":{"1min":0.27600000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.379,"15min":0.375},"min":{"1min":0.254,"5min":0.247,"15min":0.247},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.33400000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.52100000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57399999999999995,"15min":0.57099999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55400000000000005,"15min":0.54300000000000004},"min":{"1min":0.37,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.45600000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.39600000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61199999999999999,"15min":0.60699999999999998},"min":{"1min":0.41599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93500000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.51200000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.56899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298032,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073808,"statfs":{"total":96636764160,"available":96331579392,"internally_reserved":0,"allocated":761856,"data_stored":358365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.32400000000000001,"15min":0.318},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.53200000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.255},{"interface":"front","average":{"1min":0.374,"5min":0.35599999999999998,"15min":0.34300000000000003},"min":{"1min":0.28000000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57599999999999996,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.29599999999999999}]},{"osd":2,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.311,"15min":0.30499999999999999},"min":{"1min":0.22600000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.443,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.33700000000000002,"15min":0.32600000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.51600000000000001,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.33700000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34899999999999998,"15min":0.34300000000000003},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35599999999999998,"15min":0.34999999999999998},"min":{"1min":0.27400000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.315}]},{"osd":4,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55800000000000005,"15min":0.56000000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69099999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.67800000000000005,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.38}]},{"osd":5,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.39700000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68300000000000005,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72899999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.52400000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.436,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.68600000000000005,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.503},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.69299999999999995,"5min":1.075,"15min":1.075},"last":0.39500000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.441,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72199999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.41899999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.73499999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.54400000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298032,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073808,"statfs":{"total":96636764160,"available":96331579392,"internally_reserved":0,"allocated":761856,"data_stored":358365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.315,"15min":0.311},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.496,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.221},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.33400000000000002,"15min":0.32000000000000001},"min":{"1min":0.23499999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.51900000000000002,"5min":0.622,"15min":0.622},"last":0.29899999999999999}]},{"osd":1,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32900000000000001,"15min":0.32300000000000001},"min":{"1min":0.22500000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.441,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36699999999999999,"15min":0.36499999999999999},"min":{"1min":0.26500000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.47699999999999998,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.23599999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.35499999999999998,"15min":0.34699999999999998},"min":{"1min":0.28499999999999998,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.50900000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.36399999999999999,"15min":0.35699999999999998},"min":{"1min":0.28899999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.52800000000000002,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.28899999999999998}]},{"osd":4,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.33400000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.70299999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.67800000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.49099999999999999}]},{"osd":5,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70699999999999996,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.36599999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.748,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.499}]},{"osd":6,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61599999999999999,"15min":0.621},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72299999999999998,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.61499999999999999,"15min":0.62},"min":{"1min":0.45700000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71699999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.58999999999999997}]},{"osd":7,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.40600000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76600000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.45500000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.78700000000000003,"5min":0.874,"15min":0.874},"last":0.56899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298040,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073800,"statfs":{"total":96636764160,"available":96331571200,"internally_reserved":0,"allocated":770048,"data_stored":358477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.40899999999999997,"15min":0.44700000000000001},"min":{"1min":0.224,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.44500000000000001,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.35999999999999999,"15min":0.36199999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.41499999999999998,"5min":1.218,"15min":1.218},"last":0.249}]},{"osd":1,"last update":"Fri Oct 6 10:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.26900000000000002,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.432,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.45200000000000001,"15min":0.48499999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.442,"5min":3.359,"15min":3.359},"last":0.32600000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.378,"15min":0.38200000000000001},"min":{"1min":0.26000000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.47599999999999998,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.312},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.57199999999999995,"15min":0.65200000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.43099999999999999,"5min":7.069,"15min":7.069},"last":0.30499999999999999}]},{"osd":4,"last update":"Fri Oct 6 10:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.58299999999999996,"15min":0.58799999999999997},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.68899999999999995,"5min":1.165,"15min":1.165},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.378,"5min":0.375,"15min":0.375},"max":{"1min":0.747,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.73799999999999999}]},{"osd":5,"last update":"Fri Oct 6 10:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.66900000000000004,"15min":0.70399999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.75,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39700000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68899999999999995,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.51600000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.45200000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.74199999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.69999999999999996,"15min":0.73299999999999998},"min":{"1min":0.45800000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79200000000000004,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.71099999999999997}]},{"osd":7,"last update":"Fri Oct 6 10:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.46000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.72899999999999998,"5min":1.137,"15min":1.137},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.44400000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.71799999999999997,"5min":1.014,"15min":1.014},"last":0.71699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298036,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073804,"statfs":{"total":96636764160,"available":96331575296,"internally_reserved":0,"allocated":765952,"data_stored":358435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.48999999999999999,"15min":0.48399999999999999},"min":{"1min":0.38800000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66800000000000004,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.41299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66100000000000003,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.503,"15min":0.499},"min":{"1min":0.372,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53200000000000003,"15min":0.53900000000000003},"min":{"1min":0.34699999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66900000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.39000000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.51300000000000001,"15min":0.51000000000000001},"min":{"1min":0.41599999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65800000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.442,"5min":0.315,"15min":0.315},"max":{"1min":0.66900000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48099999999999998}]},{"osd":3,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.42099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.65900000000000003,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.53600000000000003},"min":{"1min":0.39200000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69499999999999995,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.48299999999999998,"15min":0.504},"min":{"1min":0.33800000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67500000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.436},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.495,"15min":0.502},"min":{"1min":0.36799999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.40400000000000003}]},{"osd":6,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.52400000000000002,"15min":0.54200000000000004},"min":{"1min":0.32400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.71799999999999997,"5min":0.75,"15min":0.75},"last":0.377},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56200000000000006,"15min":0.57699999999999996},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.68200000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.53900000000000003}]},{"osd":7,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55900000000000005,"15min":0.57699999999999996},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57999999999999996,"15min":0.59799999999999998},"min":{"1min":0.39400000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81299999999999994,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.51800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574895,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298036,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073804,"statfs":{"total":96636764160,"available":96331575296,"internally_reserved":0,"allocated":765952,"data_stored":358455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48099999999999998,"15min":0.46500000000000002},"min":{"1min":0.315,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72299999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.35399999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.86499999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.39000000000000001}]},{"osd":1,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.496,"15min":0.48499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70699999999999996,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52900000000000003,"15min":0.51800000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.52100000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51900000000000002,"15min":0.502},"min":{"1min":0.34799999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76300000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.56699999999999995}]},{"osd":3,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56799999999999995,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.67900000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.62},{"interface":"front","average":{"1min":0.5,"5min":0.55000000000000004,"15min":0.55900000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79100000000000004,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.55400000000000005}]},{"osd":4,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47999999999999998,"15min":0.48699999999999999},"min":{"1min":0.32600000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.64900000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.505,"15min":0.50800000000000001},"min":{"1min":0.35099999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.70399999999999996,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.505}]},{"osd":6,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.50900000000000001,"15min":0.52000000000000002},"min":{"1min":0.32100000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77200000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.746,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.42999999999999999}]},{"osd":7,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57299999999999995,"15min":0.58299999999999996},"min":{"1min":0.42999999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.745,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.39200000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.63700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:41:52.592 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:41:52.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.737+0000 7fbdfd9dc700 1 Processor -- start 2023-10-06T10:41:52.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.737+0000 7fbdfd9dc700 1 -- start start 2023-10-06T10:41:52.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.738+0000 7fbdfd9dc700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdf80cb5f0 0x7fbdf80691a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:52.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.738+0000 7fbdfd9dc700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbdf80696e0 0x7fbdf8159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:52.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.738+0000 7fbdfd9dc700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf8068d10 con 0x7fbdf815a010 2023-10-06T10:41:52.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.738+0000 7fbdfd9dc700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdf8154020 con 0x7fbdf80696e0 2023-10-06T10:41:52.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.738+0000 7fbdfd9dc700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf815a450 con 0x7fbdf80cb5f0 2023-10-06T10:41:52.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.739+0000 7fbdee7fc700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbdf80696e0 0x7fbdf8159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:52.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.739+0000 7fbdf6ffd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdf80cb5f0 0x7fbdf80691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:52.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.739+0000 7fbdee7fc700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbdf80696e0 0x7fbdf8159ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:45020/0 (socket says 172.21.15.92:45020) 2023-10-06T10:41:52.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.739+0000 7fbdf6ffd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdf80cb5f0 0x7fbdf80691a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:39334/0 (socket says 172.21.15.92:39334) 2023-10-06T10:41:52.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.739+0000 7fbdee7fc700 1 -- 172.21.15.92:0/2971060395 learned_addr learned my addr 172.21.15.92:0/2971060395 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:52.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.739+0000 7fbdf77fe700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fbdf815a010 0x7fbdf80ca670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:41760/0 (socket says 172.21.15.92:41760) 2023-10-06T10:41:52.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.739+0000 7fbdf6ffd700 1 -- 172.21.15.92:0/2971060395 >> v2:172.21.15.124:3300/0 conn(0x7fbdf80696e0 msgr2=0x7fbdf8159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:52.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.739+0000 7fbdf6ffd700 1 --2- 172.21.15.92:0/2971060395 >> v2:172.21.15.124:3300/0 conn(0x7fbdf80696e0 0x7fbdf8159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.739+0000 7fbdf6ffd700 1 -- 172.21.15.92:0/2971060395 >> v1:172.21.15.124:6789/0 conn(0x7fbdf815a010 legacy=0x7fbdf80ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:52.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.739+0000 7fbdf67fc700 1 -- 172.21.15.92:0/2971060395 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 472342383 0 0) 0x7fbdf8068d10 con 0x7fbdf815a010 2023-10-06T10:41:52.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.739+0000 7fbdf6ffd700 1 -- 172.21.15.92:0/2971060395 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf815a670 con 0x7fbdf80cb5f0 2023-10-06T10:41:52.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.739+0000 7fbdee7fc700 1 --2- 172.21.15.92:0/2971060395 >> v2:172.21.15.124:3300/0 conn(0x7fbdf80696e0 0x7fbdf8159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:41:52.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.739+0000 7fbdf6ffd700 1 --2- 172.21.15.92:0/2971060395 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdf80cb5f0 0x7fbdf80691a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbde80098e0 tx=0x7fbde80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bbfcaac0a28f49c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:52.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.739+0000 7fbdf67fc700 1 -- 172.21.15.92:0/2971060395 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde800b640 con 0x7fbdf80cb5f0 2023-10-06T10:41:52.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.739+0000 7fbdf67fc700 1 -- 172.21.15.92:0/2971060395 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde800b7a0 con 0x7fbdf80cb5f0 2023-10-06T10:41:52.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.739+0000 7fbdf67fc700 1 -- 172.21.15.92:0/2971060395 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde800bb40 con 0x7fbdf80cb5f0 2023-10-06T10:41:52.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.740+0000 7fbdfd9dc700 1 -- 172.21.15.92:0/2971060395 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdf80cb5f0 msgr2=0x7fbdf80691a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:52.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.740+0000 7fbdfd9dc700 1 --2- 172.21.15.92:0/2971060395 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdf80cb5f0 0x7fbdf80691a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbde80098e0 tx=0x7fbde80091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.740+0000 7fbdfd9dc700 1 -- 172.21.15.92:0/2971060395 shutdown_connections 2023-10-06T10:41:52.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.740+0000 7fbdfd9dc700 1 --2- 172.21.15.92:0/2971060395 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdf80cb5f0 0x7fbdf80691a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.740+0000 7fbdfd9dc700 1 --2- 172.21.15.92:0/2971060395 >> v2:172.21.15.124:3300/0 conn(0x7fbdf80696e0 0x7fbdf8159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.740+0000 7fbdfd9dc700 1 -- 172.21.15.92:0/2971060395 >> 172.21.15.92:0/2971060395 conn(0x7fbdf80707d0 msgr2=0x7fbdf8070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:52.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.740+0000 7fbdfd9dc700 1 -- 172.21.15.92:0/2971060395 shutdown_connections 2023-10-06T10:41:52.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.740+0000 7fbdfd9dc700 1 -- 172.21.15.92:0/2971060395 wait complete. 2023-10-06T10:41:52.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.741+0000 7fbdfd9dc700 1 Processor -- start 2023-10-06T10:41:52.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.741+0000 7fbdfd9dc700 1 -- start start 2023-10-06T10:41:52.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.742+0000 7fbdfd9dc700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdf80696e0 0x7fbdf8152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:52.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.742+0000 7fbdfd9dc700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdf815a010 0x7fbdf816b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:52.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.742+0000 7fbdfd9dc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdf80b3380 0x7fbdf80b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:52.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.742+0000 7fbdfd9dc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf816b1c0 con 0x7fbdf80b3380 2023-10-06T10:41:52.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.742+0000 7fbdfd9dc700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf8155770 con 0x7fbdf80696e0 2023-10-06T10:41:52.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.742+0000 7fbdfd9dc700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf8155a50 con 0x7fbdf815a010 2023-10-06T10:41:52.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.742+0000 7fbdf77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdf80b3380 0x7fbdf80b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:52.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.742+0000 7fbdf77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdf80b3380 0x7fbdf80b37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57278/0 (socket says 172.21.15.92:57278) 2023-10-06T10:41:52.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.742+0000 7fbdf77fe700 1 -- 172.21.15.92:0/3154276530 learned_addr learned my addr 172.21.15.92:0/3154276530 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:52.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.742+0000 7fbdf6ffd700 1 --2- 172.21.15.92:0/3154276530 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdf80696e0 0x7fbdf8152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:52.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.742+0000 7fbdee7fc700 1 --2- 172.21.15.92:0/3154276530 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdf815a010 0x7fbdf816b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:52.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.742+0000 7fbdf77fe700 1 -- 172.21.15.92:0/3154276530 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdf815a010 msgr2=0x7fbdf816b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:52.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.742+0000 7fbdf77fe700 1 --2- 172.21.15.92:0/3154276530 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdf815a010 0x7fbdf816b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.742+0000 7fbdf77fe700 1 -- 172.21.15.92:0/3154276530 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdf80696e0 msgr2=0x7fbdf8152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:52.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.742+0000 7fbdf77fe700 1 --2- 172.21.15.92:0/3154276530 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdf80696e0 0x7fbdf8152ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.742+0000 7fbdf77fe700 1 -- 172.21.15.92:0/3154276530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde00094f0 con 0x7fbdf80b3380 2023-10-06T10:41:52.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.742+0000 7fbdf77fe700 1 --2- 172.21.15.92:0/3154276530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdf80b3380 0x7fbdf80b37f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000ccc0 tx=0x7fbdf0001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:52.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.743+0000 7fbdef7fe700 1 -- 172.21.15.92:0/3154276530 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf000ce00 con 0x7fbdf80b3380 2023-10-06T10:41:52.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.743+0000 7fbdef7fe700 1 -- 172.21.15.92:0/3154276530 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf0005df0 con 0x7fbdf80b3380 2023-10-06T10:41:52.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.743+0000 7fbdfd9dc700 1 -- 172.21.15.92:0/3154276530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde8009420 con 0x7fbdf80b3380 2023-10-06T10:41:52.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.743+0000 7fbdef7fe700 1 -- 172.21.15.92:0/3154276530 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf000abb0 con 0x7fbdf80b3380 2023-10-06T10:41:52.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.743+0000 7fbdfd9dc700 1 -- 172.21.15.92:0/3154276530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf8156010 con 0x7fbdf80b3380 2023-10-06T10:41:52.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.743+0000 7fbdfd9dc700 1 -- 172.21.15.92:0/3154276530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf80cdca0 con 0x7fbdf80b3380 2023-10-06T10:41:52.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.744+0000 7fbdef7fe700 1 -- 172.21.15.92:0/3154276530 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbdf00070f0 con 0x7fbdf80b3380 2023-10-06T10:41:52.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.744+0000 7fbdef7fe700 1 --2- 172.21.15.92:0/3154276530 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbdd8066f30 0x7fbdd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:52.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.744+0000 7fbdef7fe700 1 -- 172.21.15.92:0/3154276530 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fbdf0084910 con 0x7fbdf80b3380 2023-10-06T10:41:52.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.744+0000 7fbdf6ffd700 1 --2- 172.21.15.92:0/3154276530 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbdd8066f30 0x7fbdd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:52.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.745+0000 7fbdf6ffd700 1 --2- 172.21.15.92:0/3154276530 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbdd8066f30 0x7fbdd80693e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbdf80705d0 tx=0x7fbde8017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:52.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.749+0000 7fbdef7fe700 1 -- 172.21.15.92:0/3154276530 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbdf0051960 con 0x7fbdf80b3380 2023-10-06T10:41:52.818 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-10-06T10:41:52.816+0000 7fdec7cae700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T10:41:52.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.927+0000 7fbdfd9dc700 1 -- 172.21.15.92:0/3154276530 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbdf8064a90 con 0x7fbdd8066f30 2023-10-06T10:41:52.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.936+0000 7fbdef7fe700 1 -- 172.21.15.92:0/3154276530 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311733 (secure 0 0 0) 0x7fbdf8064a90 con 0x7fbdd8066f30 2023-10-06T10:41:52.941 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:52.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.939+0000 7fbdfd9dc700 1 -- 172.21.15.92:0/3154276530 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbdd8066f30 msgr2=0x7fbdd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:52.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.939+0000 7fbdfd9dc700 1 --2- 172.21.15.92:0/3154276530 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbdd8066f30 0x7fbdd80693e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbdf80705d0 tx=0x7fbde8017000 comp rx=0 tx=0).stop 2023-10-06T10:41:52.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.939+0000 7fbdfd9dc700 1 -- 172.21.15.92:0/3154276530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdf80b3380 msgr2=0x7fbdf80b37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:52.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.939+0000 7fbdfd9dc700 1 --2- 172.21.15.92:0/3154276530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdf80b3380 0x7fbdf80b37f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000ccc0 tx=0x7fbdf0001a70 comp rx=0 tx=0).stop 2023-10-06T10:41:52.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.939+0000 7fbdfd9dc700 1 -- 172.21.15.92:0/3154276530 shutdown_connections 2023-10-06T10:41:52.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.939+0000 7fbdfd9dc700 1 --2- 172.21.15.92:0/3154276530 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbdd8066f30 0x7fbdd80693e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.939+0000 7fbdfd9dc700 1 --2- 172.21.15.92:0/3154276530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdf80b3380 0x7fbdf80b37f0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.939+0000 7fbdfd9dc700 1 --2- 172.21.15.92:0/3154276530 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdf80696e0 0x7fbdf8152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.939+0000 7fbdfd9dc700 1 --2- 172.21.15.92:0/3154276530 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdf815a010 0x7fbdf816b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:52.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.939+0000 7fbdfd9dc700 1 -- 172.21.15.92:0/3154276530 >> 172.21.15.92:0/3154276530 conn(0x7fbdf80707d0 msgr2=0x7fbdf80ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:52.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.940+0000 7fbdfd9dc700 1 -- 172.21.15.92:0/3154276530 shutdown_connections 2023-10-06T10:41:52.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:52.940+0000 7fbdfd9dc700 1 -- 172.21.15.92:0/3154276530 wait complete. 2023-10-06T10:41:52.945 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:41:52.953 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2023-10-06T10:41:52.129492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385464,"kb_used_data":7160,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752589256,"statfs":{"total":773094113280,"available":770651398144,"internally_reserved":0,"allocated":7331840,"data_stored":4049379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.618633"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.334353+0000","last_change":"2023-10-06T10:41:48.334353+0000","last_active":"2023-10-06T10:41:48.334353+0000","last_peered":"2023-10-06T10:41:48.334353+0000","last_clean":"2023-10-06T10:41:48.334353+0000","last_became_active":"2023-10-06T10:41:48.333910+0000","last_became_peered":"2023-10-06T10:41:48.333910+0000","last_unstale":"2023-10-06T10:41:48.334353+0000","last_undegraded":"2023-10-06T10:41:48.334353+0000","last_fullsized":"2023-10-06T10:41:48.334353+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:33.349366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.336176+0000","last_change":"2023-10-06T10:41:48.336176+0000","last_active":"2023-10-06T10:41:48.336176+0000","last_peered":"2023-10-06T10:41:48.336176+0000","last_clean":"2023-10-06T10:41:48.336176+0000","last_became_active":"2023-10-06T10:41:48.335922+0000","last_became_peered":"2023-10-06T10:41:48.335922+0000","last_unstale":"2023-10-06T10:41:48.336176+0000","last_undegraded":"2023-10-06T10:41:48.336176+0000","last_fullsized":"2023-10-06T10:41:48.336176+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:34:40.831811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.327517+0000","last_change":"2023-10-06T10:41:48.325103+0000","last_active":"2023-10-06T10:41:48.327517+0000","last_peered":"2023-10-06T10:41:48.327517+0000","last_clean":"2023-10-06T10:41:48.327517+0000","last_became_active":"2023-10-06T10:41:48.324843+0000","last_became_peered":"2023-10-06T10:41:48.324843+0000","last_unstale":"2023-10-06T10:41:48.327517+0000","last_undegraded":"2023-10-06T10:41:48.327517+0000","last_fullsized":"2023-10-06T10:41:48.327517+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:59.082862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.328110+0000","last_change":"2023-10-06T10:41:48.328110+0000","last_active":"2023-10-06T10:41:48.328110+0000","last_peered":"2023-10-06T10:41:48.328110+0000","last_clean":"2023-10-06T10:41:48.328110+0000","last_became_active":"2023-10-06T10:41:48.327860+0000","last_became_peered":"2023-10-06T10:41:48.327860+0000","last_unstale":"2023-10-06T10:41:48.328110+0000","last_undegraded":"2023-10-06T10:41:48.328110+0000","last_fullsized":"2023-10-06T10:41:48.328110+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:40:18.488714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.333623+0000","last_change":"2023-10-06T10:41:48.333623+0000","last_active":"2023-10-06T10:41:48.333623+0000","last_peered":"2023-10-06T10:41:48.333623+0000","last_clean":"2023-10-06T10:41:48.333623+0000","last_became_active":"2023-10-06T10:41:48.333368+0000","last_became_peered":"2023-10-06T10:41:48.333368+0000","last_unstale":"2023-10-06T10:41:48.333623+0000","last_undegraded":"2023-10-06T10:41:48.333623+0000","last_fullsized":"2023-10-06T10:41:48.333623+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:08.220000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.337025+0000","last_change":"2023-10-06T10:41:48.337025+0000","last_active":"2023-10-06T10:41:48.337025+0000","last_peered":"2023-10-06T10:41:48.337025+0000","last_clean":"2023-10-06T10:41:48.337025+0000","last_became_active":"2023-10-06T10:41:48.336750+0000","last_became_peered":"2023-10-06T10:41:48.336750+0000","last_unstale":"2023-10-06T10:41:48.337025+0000","last_undegraded":"2023-10-06T10:41:48.337025+0000","last_fullsized":"2023-10-06T10:41:48.337025+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:03:36.216614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.329303+0000","last_change":"2023-10-06T10:41:48.329303+0000","last_active":"2023-10-06T10:41:48.329303+0000","last_peered":"2023-10-06T10:41:48.329303+0000","last_clean":"2023-10-06T10:41:48.329303+0000","last_became_active":"2023-10-06T10:41:48.329041+0000","last_became_peered":"2023-10-06T10:41:48.329041+0000","last_unstale":"2023-10-06T10:41:48.329303+0000","last_undegraded":"2023-10-06T10:41:48.329303+0000","last_fullsized":"2023-10-06T10:41:48.329303+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:30:27.462161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.323257+0000","last_change":"2023-10-06T10:41:48.323257+0000","last_active":"2023-10-06T10:41:48.323257+0000","last_peered":"2023-10-06T10:41:48.323257+0000","last_clean":"2023-10-06T10:41:48.323257+0000","last_became_active":"2023-10-06T10:41:48.323046+0000","last_became_peered":"2023-10-06T10:41:48.323046+0000","last_unstale":"2023-10-06T10:41:48.323257+0000","last_undegraded":"2023-10-06T10:41:48.323257+0000","last_fullsized":"2023-10-06T10:41:48.323257+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:02:12.095392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.329622+0000","last_change":"2023-10-06T10:41:48.329622+0000","last_active":"2023-10-06T10:41:48.329622+0000","last_peered":"2023-10-06T10:41:48.329622+0000","last_clean":"2023-10-06T10:41:48.329622+0000","last_became_active":"2023-10-06T10:41:48.329262+0000","last_became_peered":"2023-10-06T10:41:48.329262+0000","last_unstale":"2023-10-06T10:41:48.329622+0000","last_undegraded":"2023-10-06T10:41:48.329622+0000","last_fullsized":"2023-10-06T10:41:48.329622+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:44:16.796433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.337987+0000","last_change":"2023-10-06T10:41:48.337987+0000","last_active":"2023-10-06T10:41:48.337987+0000","last_peered":"2023-10-06T10:41:48.337987+0000","last_clean":"2023-10-06T10:41:48.337987+0000","last_became_active":"2023-10-06T10:41:48.336173+0000","last_became_peered":"2023-10-06T10:41:48.336173+0000","last_unstale":"2023-10-06T10:41:48.337987+0000","last_undegraded":"2023-10-06T10:41:48.337987+0000","last_fullsized":"2023-10-06T10:41:48.337987+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:07.543490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.337897+0000","last_change":"2023-10-06T10:41:48.337897+0000","last_active":"2023-10-06T10:41:48.337897+0000","last_peered":"2023-10-06T10:41:48.337897+0000","last_clean":"2023-10-06T10:41:48.337897+0000","last_became_active":"2023-10-06T10:41:48.337607+0000","last_became_peered":"2023-10-06T10:41:48.337607+0000","last_unstale":"2023-10-06T10:41:48.337897+0000","last_undegraded":"2023-10-06T10:41:48.337897+0000","last_fullsized":"2023-10-06T10:41:48.337897+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:30:27.462161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.331384+0000","last_change":"2023-10-06T10:41:48.331384+0000","last_active":"2023-10-06T10:41:48.331384+0000","last_peered":"2023-10-06T10:41:48.331384+0000","last_clean":"2023-10-06T10:41:48.331384+0000","last_became_active":"2023-10-06T10:41:48.329149+0000","last_became_peered":"2023-10-06T10:41:48.329149+0000","last_unstale":"2023-10-06T10:41:48.331384+0000","last_undegraded":"2023-10-06T10:41:48.331384+0000","last_fullsized":"2023-10-06T10:41:48.331384+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:44:16.796433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.331741+0000","last_change":"2023-10-06T10:41:48.331741+0000","last_active":"2023-10-06T10:41:48.331741+0000","last_peered":"2023-10-06T10:41:48.331741+0000","last_clean":"2023-10-06T10:41:48.331741+0000","last_became_active":"2023-10-06T10:41:48.331498+0000","last_became_peered":"2023-10-06T10:41:48.331498+0000","last_unstale":"2023-10-06T10:41:48.331741+0000","last_undegraded":"2023-10-06T10:41:48.331741+0000","last_fullsized":"2023-10-06T10:41:48.331741+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:31.491560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.327499+0000","last_change":"2023-10-06T10:41:48.327499+0000","last_active":"2023-10-06T10:41:48.327499+0000","last_peered":"2023-10-06T10:41:48.327499+0000","last_clean":"2023-10-06T10:41:48.327499+0000","last_became_active":"2023-10-06T10:41:48.327214+0000","last_became_peered":"2023-10-06T10:41:48.327214+0000","last_unstale":"2023-10-06T10:41:48.327499+0000","last_undegraded":"2023-10-06T10:41:48.327499+0000","last_fullsized":"2023-10-06T10:41:48.327499+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:50.041063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.322768+0000","last_change":"2023-10-06T10:41:48.322768+0000","last_active":"2023-10-06T10:41:48.322768+0000","last_peered":"2023-10-06T10:41:48.322768+0000","last_clean":"2023-10-06T10:41:48.322768+0000","last_became_active":"2023-10-06T10:41:48.322542+0000","last_became_peered":"2023-10-06T10:41:48.322542+0000","last_unstale":"2023-10-06T10:41:48.322768+0000","last_undegraded":"2023-10-06T10:41:48.322768+0000","last_fullsized":"2023-10-06T10:41:48.322768+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:53:24.847598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.340276+0000","last_change":"2023-10-06T10:41:48.340276+0000","last_active":"2023-10-06T10:41:48.340276+0000","last_peered":"2023-10-06T10:41:48.340276+0000","last_clean":"2023-10-06T10:41:48.340276+0000","last_became_active":"2023-10-06T10:41:48.340010+0000","last_became_peered":"2023-10-06T10:41:48.340010+0000","last_unstale":"2023-10-06T10:41:48.340276+0000","last_undegraded":"2023-10-06T10:41:48.340276+0000","last_fullsized":"2023-10-06T10:41:48.340276+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:37:55.371863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.329511+0000","last_change":"2023-10-06T10:41:48.329511+0000","last_active":"2023-10-06T10:41:48.329511+0000","last_peered":"2023-10-06T10:41:48.329511+0000","last_clean":"2023-10-06T10:41:48.329511+0000","last_became_active":"2023-10-06T10:41:48.328909+0000","last_became_peered":"2023-10-06T10:41:48.328909+0000","last_unstale":"2023-10-06T10:41:48.329511+0000","last_undegraded":"2023-10-06T10:41:48.329511+0000","last_fullsized":"2023-10-06T10:41:48.329511+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:39.629579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.329240+0000","last_change":"2023-10-06T10:41:48.329240+0000","last_active":"2023-10-06T10:41:48.329240+0000","last_peered":"2023-10-06T10:41:48.329240+0000","last_clean":"2023-10-06T10:41:48.329240+0000","last_became_active":"2023-10-06T10:41:48.328443+0000","last_became_peered":"2023-10-06T10:41:48.328443+0000","last_unstale":"2023-10-06T10:41:48.329240+0000","last_undegraded":"2023-10-06T10:41:48.329240+0000","last_fullsized":"2023-10-06T10:41:48.329240+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:58.300560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.324185+0000","last_change":"2023-10-06T10:41:48.324185+0000","last_active":"2023-10-06T10:41:48.324185+0000","last_peered":"2023-10-06T10:41:48.324185+0000","last_clean":"2023-10-06T10:41:48.324185+0000","last_became_active":"2023-10-06T10:41:48.323887+0000","last_became_peered":"2023-10-06T10:41:48.323887+0000","last_unstale":"2023-10-06T10:41:48.324185+0000","last_undegraded":"2023-10-06T10:41:48.324185+0000","last_fullsized":"2023-10-06T10:41:48.324185+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:04:13.357982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.329022+0000","last_change":"2023-10-06T10:41:48.329022+0000","last_active":"2023-10-06T10:41:48.329022+0000","last_peered":"2023-10-06T10:41:48.329022+0000","last_clean":"2023-10-06T10:41:48.329022+0000","last_became_active":"2023-10-06T10:41:48.328775+0000","last_became_peered":"2023-10-06T10:41:48.328775+0000","last_unstale":"2023-10-06T10:41:48.329022+0000","last_undegraded":"2023-10-06T10:41:48.329022+0000","last_fullsized":"2023-10-06T10:41:48.329022+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:41:58.036031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.335405+0000","last_change":"2023-10-06T10:41:48.335405+0000","last_active":"2023-10-06T10:41:48.335405+0000","last_peered":"2023-10-06T10:41:48.335405+0000","last_clean":"2023-10-06T10:41:48.335405+0000","last_became_active":"2023-10-06T10:41:48.335101+0000","last_became_peered":"2023-10-06T10:41:48.335101+0000","last_unstale":"2023-10-06T10:41:48.335405+0000","last_undegraded":"2023-10-06T10:41:48.335405+0000","last_fullsized":"2023-10-06T10:41:48.335405+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:38:45.566420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.337218+0000","last_change":"2023-10-06T10:41:48.337218+0000","last_active":"2023-10-06T10:41:48.337218+0000","last_peered":"2023-10-06T10:41:48.337218+0000","last_clean":"2023-10-06T10:41:48.337218+0000","last_became_active":"2023-10-06T10:41:48.336794+0000","last_became_peered":"2023-10-06T10:41:48.336794+0000","last_unstale":"2023-10-06T10:41:48.337218+0000","last_undegraded":"2023-10-06T10:41:48.337218+0000","last_fullsized":"2023-10-06T10:41:48.337218+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:26:44.645107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.333748+0000","last_change":"2023-10-06T10:41:48.333748+0000","last_active":"2023-10-06T10:41:48.333748+0000","last_peered":"2023-10-06T10:41:48.333748+0000","last_clean":"2023-10-06T10:41:48.333748+0000","last_became_active":"2023-10-06T10:41:48.333347+0000","last_became_peered":"2023-10-06T10:41:48.333347+0000","last_unstale":"2023-10-06T10:41:48.333748+0000","last_undegraded":"2023-10-06T10:41:48.333748+0000","last_fullsized":"2023-10-06T10:41:48.333748+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:56:52.523361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.335423+0000","last_change":"2023-10-06T10:41:48.335423+0000","last_active":"2023-10-06T10:41:48.335423+0000","last_peered":"2023-10-06T10:41:48.335423+0000","last_clean":"2023-10-06T10:41:48.335423+0000","last_became_active":"2023-10-06T10:41:48.335144+0000","last_became_peered":"2023-10-06T10:41:48.335144+0000","last_unstale":"2023-10-06T10:41:48.335423+0000","last_undegraded":"2023-10-06T10:41:48.335423+0000","last_fullsized":"2023-10-06T10:41:48.335423+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:28:14.345962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.334747+0000","last_change":"2023-10-06T10:41:48.334747+0000","last_active":"2023-10-06T10:41:48.334747+0000","last_peered":"2023-10-06T10:41:48.334747+0000","last_clean":"2023-10-06T10:41:48.334747+0000","last_became_active":"2023-10-06T10:41:48.334442+0000","last_became_peered":"2023-10-06T10:41:48.334442+0000","last_unstale":"2023-10-06T10:41:48.334747+0000","last_undegraded":"2023-10-06T10:41:48.334747+0000","last_fullsized":"2023-10-06T10:41:48.334747+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:12.905330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.332027+0000","last_change":"2023-10-06T10:41:48.332027+0000","last_active":"2023-10-06T10:41:48.332027+0000","last_peered":"2023-10-06T10:41:48.332027+0000","last_clean":"2023-10-06T10:41:48.332027+0000","last_became_active":"2023-10-06T10:41:48.331757+0000","last_became_peered":"2023-10-06T10:41:48.331757+0000","last_unstale":"2023-10-06T10:41:48.332027+0000","last_undegraded":"2023-10-06T10:41:48.332027+0000","last_fullsized":"2023-10-06T10:41:48.332027+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:28:14.345962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.335393+0000","last_change":"2023-10-06T10:41:48.333717+0000","last_active":"2023-10-06T10:41:48.335393+0000","last_peered":"2023-10-06T10:41:48.335393+0000","last_clean":"2023-10-06T10:41:48.335393+0000","last_became_active":"2023-10-06T10:41:48.333448+0000","last_became_peered":"2023-10-06T10:41:48.333448+0000","last_unstale":"2023-10-06T10:41:48.335393+0000","last_undegraded":"2023-10-06T10:41:48.335393+0000","last_fullsized":"2023-10-06T10:41:48.335393+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:39.085051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.336543+0000","last_change":"2023-10-06T10:41:48.336543+0000","last_active":"2023-10-06T10:41:48.336543+0000","last_peered":"2023-10-06T10:41:48.336543+0000","last_clean":"2023-10-06T10:41:48.336543+0000","last_became_active":"2023-10-06T10:41:48.336061+0000","last_became_peered":"2023-10-06T10:41:48.336061+0000","last_unstale":"2023-10-06T10:41:48.336543+0000","last_undegraded":"2023-10-06T10:41:48.336543+0000","last_fullsized":"2023-10-06T10:41:48.336543+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:00:35.112716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.329114+0000","last_change":"2023-10-06T10:41:48.326927+0000","last_active":"2023-10-06T10:41:48.329114+0000","last_peered":"2023-10-06T10:41:48.329114+0000","last_clean":"2023-10-06T10:41:48.329114+0000","last_became_active":"2023-10-06T10:41:48.326680+0000","last_became_peered":"2023-10-06T10:41:48.326680+0000","last_unstale":"2023-10-06T10:41:48.329114+0000","last_undegraded":"2023-10-06T10:41:48.329114+0000","last_fullsized":"2023-10-06T10:41:48.329114+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:50.376270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.331738+0000","last_change":"2023-10-06T10:41:48.331738+0000","last_active":"2023-10-06T10:41:48.331738+0000","last_peered":"2023-10-06T10:41:48.331738+0000","last_clean":"2023-10-06T10:41:48.331738+0000","last_became_active":"2023-10-06T10:41:48.331553+0000","last_became_peered":"2023-10-06T10:41:48.331553+0000","last_unstale":"2023-10-06T10:41:48.331738+0000","last_undegraded":"2023-10-06T10:41:48.331738+0000","last_fullsized":"2023-10-06T10:41:48.331738+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:38.254889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.328881+0000","last_change":"2023-10-06T10:41:48.328881+0000","last_active":"2023-10-06T10:41:48.328881+0000","last_peered":"2023-10-06T10:41:48.328881+0000","last_clean":"2023-10-06T10:41:48.328881+0000","last_became_active":"2023-10-06T10:41:48.328606+0000","last_became_peered":"2023-10-06T10:41:48.328606+0000","last_unstale":"2023-10-06T10:41:48.328881+0000","last_undegraded":"2023-10-06T10:41:48.328881+0000","last_fullsized":"2023-10-06T10:41:48.328881+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:43.365081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.328986+0000","last_change":"2023-10-06T10:41:48.328986+0000","last_active":"2023-10-06T10:41:48.328986+0000","last_peered":"2023-10-06T10:41:48.328986+0000","last_clean":"2023-10-06T10:41:48.328986+0000","last_became_active":"2023-10-06T10:41:48.328731+0000","last_became_peered":"2023-10-06T10:41:48.328731+0000","last_unstale":"2023-10-06T10:41:48.328986+0000","last_undegraded":"2023-10-06T10:41:48.328986+0000","last_fullsized":"2023-10-06T10:41:48.328986+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:27.366430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.335582+0000","last_change":"2023-10-06T10:41:48.335582+0000","last_active":"2023-10-06T10:41:48.335582+0000","last_peered":"2023-10-06T10:41:48.335582+0000","last_clean":"2023-10-06T10:41:48.335582+0000","last_became_active":"2023-10-06T10:41:48.334133+0000","last_became_peered":"2023-10-06T10:41:48.334133+0000","last_unstale":"2023-10-06T10:41:48.335582+0000","last_undegraded":"2023-10-06T10:41:48.335582+0000","last_fullsized":"2023-10-06T10:41:48.335582+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:11:17.706840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"63'4","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.545571+0000","last_change":"2023-10-06T10:41:47.528286+0000","last_active":"2023-10-06T10:41:48.545571+0000","last_peered":"2023-10-06T10:41:48.545571+0000","last_clean":"2023-10-06T10:41:48.545571+0000","last_became_active":"2023-10-06T10:41:47.528034+0000","last_became_peered":"2023-10-06T10:41:47.528034+0000","last_unstale":"2023-10-06T10:41:48.545571+0000","last_undegraded":"2023-10-06T10:41:48.545571+0000","last_fullsized":"2023-10-06T10:41:48.545571+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:58.970822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.322155+0000","last_change":"2023-10-06T10:41:48.322155+0000","last_active":"2023-10-06T10:41:48.322155+0000","last_peered":"2023-10-06T10:41:48.322155+0000","last_clean":"2023-10-06T10:41:48.322155+0000","last_became_active":"2023-10-06T10:41:48.321890+0000","last_became_peered":"2023-10-06T10:41:48.321890+0000","last_unstale":"2023-10-06T10:41:48.322155+0000","last_undegraded":"2023-10-06T10:41:48.322155+0000","last_fullsized":"2023-10-06T10:41:48.322155+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:01:21.300442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"63'1","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.317883+0000","last_change":"2023-10-06T10:41:47.540634+0000","last_active":"2023-10-06T10:41:48.317883+0000","last_peered":"2023-10-06T10:41:48.317883+0000","last_clean":"2023-10-06T10:41:48.317883+0000","last_became_active":"2023-10-06T10:41:47.540380+0000","last_became_peered":"2023-10-06T10:41:47.540380+0000","last_unstale":"2023-10-06T10:41:48.317883+0000","last_undegraded":"2023-10-06T10:41:48.317883+0000","last_fullsized":"2023-10-06T10:41:48.317883+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:56:51.516760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.336024+0000","last_change":"2023-10-06T10:41:48.336024+0000","last_active":"2023-10-06T10:41:48.336024+0000","last_peered":"2023-10-06T10:41:48.336024+0000","last_clean":"2023-10-06T10:41:48.336024+0000","last_became_active":"2023-10-06T10:41:48.332793+0000","last_became_peered":"2023-10-06T10:41:48.332793+0000","last_unstale":"2023-10-06T10:41:48.336024+0000","last_undegraded":"2023-10-06T10:41:48.336024+0000","last_fullsized":"2023-10-06T10:41:48.336024+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:38:06.749974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"63'2","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.538496+0000","last_change":"2023-10-06T10:41:47.524576+0000","last_active":"2023-10-06T10:41:48.538496+0000","last_peered":"2023-10-06T10:41:48.538496+0000","last_clean":"2023-10-06T10:41:48.538496+0000","last_became_active":"2023-10-06T10:41:47.524273+0000","last_became_peered":"2023-10-06T10:41:47.524273+0000","last_unstale":"2023-10-06T10:41:48.538496+0000","last_undegraded":"2023-10-06T10:41:48.538496+0000","last_fullsized":"2023-10-06T10:41:48.538496+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:02:11.088791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.338070+0000","last_change":"2023-10-06T10:41:48.338070+0000","last_active":"2023-10-06T10:41:48.338070+0000","last_peered":"2023-10-06T10:41:48.338070+0000","last_clean":"2023-10-06T10:41:48.338070+0000","last_became_active":"2023-10-06T10:41:48.337652+0000","last_became_peered":"2023-10-06T10:41:48.337652+0000","last_unstale":"2023-10-06T10:41:48.338070+0000","last_undegraded":"2023-10-06T10:41:48.338070+0000","last_fullsized":"2023-10-06T10:41:48.338070+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:34:40.831811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"63'2","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.545658+0000","last_change":"2023-10-06T10:41:47.528178+0000","last_active":"2023-10-06T10:41:48.545658+0000","last_peered":"2023-10-06T10:41:48.545658+0000","last_clean":"2023-10-06T10:41:48.545658+0000","last_became_active":"2023-10-06T10:41:47.527686+0000","last_became_peered":"2023-10-06T10:41:47.527686+0000","last_unstale":"2023-10-06T10:41:48.545658+0000","last_undegraded":"2023-10-06T10:41:48.545658+0000","last_fullsized":"2023-10-06T10:41:48.545658+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:06.760378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.327244+0000","last_change":"2023-10-06T10:41:48.327244+0000","last_active":"2023-10-06T10:41:48.327244+0000","last_peered":"2023-10-06T10:41:48.327244+0000","last_clean":"2023-10-06T10:41:48.327244+0000","last_became_active":"2023-10-06T10:41:48.326934+0000","last_became_peered":"2023-10-06T10:41:48.326934+0000","last_unstale":"2023-10-06T10:41:48.327244+0000","last_undegraded":"2023-10-06T10:41:48.327244+0000","last_fullsized":"2023-10-06T10:41:48.327244+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:26:44.645107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.542465+0000","last_change":"2023-10-06T10:41:47.535047+0000","last_active":"2023-10-06T10:41:48.542465+0000","last_peered":"2023-10-06T10:41:48.542465+0000","last_clean":"2023-10-06T10:41:48.542465+0000","last_became_active":"2023-10-06T10:41:47.534834+0000","last_became_peered":"2023-10-06T10:41:47.534834+0000","last_unstale":"2023-10-06T10:41:48.542465+0000","last_undegraded":"2023-10-06T10:41:48.542465+0000","last_fullsized":"2023-10-06T10:41:48.542465+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:38:44.559819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.335371+0000","last_change":"2023-10-06T10:41:48.335371+0000","last_active":"2023-10-06T10:41:48.335371+0000","last_peered":"2023-10-06T10:41:48.335371+0000","last_clean":"2023-10-06T10:41:48.335371+0000","last_became_active":"2023-10-06T10:41:48.335062+0000","last_became_peered":"2023-10-06T10:41:48.335062+0000","last_unstale":"2023-10-06T10:41:48.335371+0000","last_undegraded":"2023-10-06T10:41:48.335371+0000","last_fullsized":"2023-10-06T10:41:48.335371+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:58.647093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"63'1","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.319341+0000","last_change":"2023-10-06T10:41:47.529246+0000","last_active":"2023-10-06T10:41:48.319341+0000","last_peered":"2023-10-06T10:41:48.319341+0000","last_clean":"2023-10-06T10:41:48.319341+0000","last_became_active":"2023-10-06T10:41:47.528977+0000","last_became_peered":"2023-10-06T10:41:47.528977+0000","last_unstale":"2023-10-06T10:41:48.319341+0000","last_undegraded":"2023-10-06T10:41:48.319341+0000","last_fullsized":"2023-10-06T10:41:48.319341+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:14:07.267985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.336874+0000","last_change":"2023-10-06T10:41:48.336874+0000","last_active":"2023-10-06T10:41:48.336874+0000","last_peered":"2023-10-06T10:41:48.336874+0000","last_clean":"2023-10-06T10:41:48.336874+0000","last_became_active":"2023-10-06T10:41:48.336604+0000","last_became_peered":"2023-10-06T10:41:48.336604+0000","last_unstale":"2023-10-06T10:41:48.336874+0000","last_undegraded":"2023-10-06T10:41:48.336874+0000","last_fullsized":"2023-10-06T10:41:48.336874+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:00.973193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.531377+0000","last_change":"2023-10-06T10:41:47.525088+0000","last_active":"2023-10-06T10:41:47.531377+0000","last_peered":"2023-10-06T10:41:47.531377+0000","last_clean":"2023-10-06T10:41:47.531377+0000","last_became_active":"2023-10-06T10:41:47.524813+0000","last_became_peered":"2023-10-06T10:41:47.524813+0000","last_unstale":"2023-10-06T10:41:47.531377+0000","last_undegraded":"2023-10-06T10:41:47.531377+0000","last_fullsized":"2023-10-06T10:41:47.531377+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:29.932628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.331020+0000","last_change":"2023-10-06T10:41:48.331020+0000","last_active":"2023-10-06T10:41:48.331020+0000","last_peered":"2023-10-06T10:41:48.331020+0000","last_clean":"2023-10-06T10:41:48.331020+0000","last_became_active":"2023-10-06T10:41:48.330803+0000","last_became_peered":"2023-10-06T10:41:48.330803+0000","last_unstale":"2023-10-06T10:41:48.331020+0000","last_undegraded":"2023-10-06T10:41:48.331020+0000","last_fullsized":"2023-10-06T10:41:48.331020+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:22:06.050969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.540328+0000","last_change":"2023-10-06T10:41:47.527923+0000","last_active":"2023-10-06T10:41:47.540328+0000","last_peered":"2023-10-06T10:41:47.540328+0000","last_clean":"2023-10-06T10:41:47.540328+0000","last_became_active":"2023-10-06T10:41:47.527668+0000","last_became_peered":"2023-10-06T10:41:47.527668+0000","last_unstale":"2023-10-06T10:41:47.540328+0000","last_undegraded":"2023-10-06T10:41:47.540328+0000","last_fullsized":"2023-10-06T10:41:47.540328+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:50:00.843187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.331182+0000","last_change":"2023-10-06T10:41:48.331182+0000","last_active":"2023-10-06T10:41:48.331182+0000","last_peered":"2023-10-06T10:41:48.331182+0000","last_clean":"2023-10-06T10:41:48.331182+0000","last_became_active":"2023-10-06T10:41:48.330878+0000","last_became_peered":"2023-10-06T10:41:48.330878+0000","last_unstale":"2023-10-06T10:41:48.331182+0000","last_undegraded":"2023-10-06T10:41:48.331182+0000","last_fullsized":"2023-10-06T10:41:48.331182+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:07.543490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.534448+0000","last_change":"2023-10-06T10:41:47.526247+0000","last_active":"2023-10-06T10:41:47.534448+0000","last_peered":"2023-10-06T10:41:47.534448+0000","last_clean":"2023-10-06T10:41:47.534448+0000","last_became_active":"2023-10-06T10:41:47.525980+0000","last_became_peered":"2023-10-06T10:41:47.525980+0000","last_unstale":"2023-10-06T10:41:47.534448+0000","last_undegraded":"2023-10-06T10:41:47.534448+0000","last_fullsized":"2023-10-06T10:41:47.534448+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:29:08.881968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.334515+0000","last_change":"2023-10-06T10:41:48.334515+0000","last_active":"2023-10-06T10:41:48.334515+0000","last_peered":"2023-10-06T10:41:48.334515+0000","last_clean":"2023-10-06T10:41:48.334515+0000","last_became_active":"2023-10-06T10:41:48.334078+0000","last_became_peered":"2023-10-06T10:41:48.334078+0000","last_unstale":"2023-10-06T10:41:48.334515+0000","last_undegraded":"2023-10-06T10:41:48.334515+0000","last_fullsized":"2023-10-06T10:41:48.334515+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:38.254889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.539924+0000","last_change":"2023-10-06T10:41:47.529220+0000","last_active":"2023-10-06T10:41:47.539924+0000","last_peered":"2023-10-06T10:41:47.539924+0000","last_clean":"2023-10-06T10:41:47.539924+0000","last_became_active":"2023-10-06T10:41:47.528931+0000","last_became_peered":"2023-10-06T10:41:47.528931+0000","last_unstale":"2023-10-06T10:41:47.539924+0000","last_undegraded":"2023-10-06T10:41:47.539924+0000","last_fullsized":"2023-10-06T10:41:47.539924+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:41:31.882577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.330952+0000","last_change":"2023-10-06T10:41:48.330952+0000","last_active":"2023-10-06T10:41:48.330952+0000","last_peered":"2023-10-06T10:41:48.330952+0000","last_clean":"2023-10-06T10:41:48.330952+0000","last_became_active":"2023-10-06T10:41:48.330657+0000","last_became_peered":"2023-10-06T10:41:48.330657+0000","last_unstale":"2023-10-06T10:41:48.330952+0000","last_undegraded":"2023-10-06T10:41:48.330952+0000","last_fullsized":"2023-10-06T10:41:48.330952+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:29:09.888570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"63'3","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.545343+0000","last_change":"2023-10-06T10:41:47.527983+0000","last_active":"2023-10-06T10:41:48.545343+0000","last_peered":"2023-10-06T10:41:48.545343+0000","last_clean":"2023-10-06T10:41:48.545343+0000","last_became_active":"2023-10-06T10:41:47.527703+0000","last_became_peered":"2023-10-06T10:41:47.527703+0000","last_unstale":"2023-10-06T10:41:48.545343+0000","last_undegraded":"2023-10-06T10:41:48.545343+0000","last_fullsized":"2023-10-06T10:41:48.545343+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:37:54.365261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.322040+0000","last_change":"2023-10-06T10:41:48.322040+0000","last_active":"2023-10-06T10:41:48.322040+0000","last_peered":"2023-10-06T10:41:48.322040+0000","last_clean":"2023-10-06T10:41:48.322040+0000","last_became_active":"2023-10-06T10:41:48.321689+0000","last_became_peered":"2023-10-06T10:41:48.321689+0000","last_unstale":"2023-10-06T10:41:48.322040+0000","last_undegraded":"2023-10-06T10:41:48.322040+0000","last_fullsized":"2023-10-06T10:41:48.322040+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:17:42.162415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"63'6","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.545070+0000","last_change":"2023-10-06T10:41:47.531271+0000","last_active":"2023-10-06T10:41:48.545070+0000","last_peered":"2023-10-06T10:41:48.545070+0000","last_clean":"2023-10-06T10:41:48.545070+0000","last_became_active":"2023-10-06T10:41:47.531031+0000","last_became_peered":"2023-10-06T10:41:47.531031+0000","last_unstale":"2023-10-06T10:41:48.545070+0000","last_undegraded":"2023-10-06T10:41:48.545070+0000","last_fullsized":"2023-10-06T10:41:48.545070+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:23:54.319502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.333553+0000","last_change":"2023-10-06T10:41:48.333553+0000","last_active":"2023-10-06T10:41:48.333553+0000","last_peered":"2023-10-06T10:41:48.333553+0000","last_clean":"2023-10-06T10:41:48.333553+0000","last_became_active":"2023-10-06T10:41:48.333242+0000","last_became_peered":"2023-10-06T10:41:48.333242+0000","last_unstale":"2023-10-06T10:41:48.333553+0000","last_undegraded":"2023-10-06T10:41:48.333553+0000","last_fullsized":"2023-10-06T10:41:48.333553+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:03:36.216614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"63'2","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.543243+0000","last_change":"2023-10-06T10:41:47.530477+0000","last_active":"2023-10-06T10:41:48.543243+0000","last_peered":"2023-10-06T10:41:48.543243+0000","last_clean":"2023-10-06T10:41:48.543243+0000","last_became_active":"2023-10-06T10:41:47.529579+0000","last_became_peered":"2023-10-06T10:41:47.529579+0000","last_unstale":"2023-10-06T10:41:48.543243+0000","last_undegraded":"2023-10-06T10:41:48.543243+0000","last_fullsized":"2023-10-06T10:41:48.543243+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:36.835370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.337721+0000","last_change":"2023-10-06T10:41:48.337721+0000","last_active":"2023-10-06T10:41:48.337721+0000","last_peered":"2023-10-06T10:41:48.337721+0000","last_clean":"2023-10-06T10:41:48.337721+0000","last_became_active":"2023-10-06T10:41:48.337460+0000","last_became_peered":"2023-10-06T10:41:48.337460+0000","last_unstale":"2023-10-06T10:41:48.337721+0000","last_undegraded":"2023-10-06T10:41:48.337721+0000","last_fullsized":"2023-10-06T10:41:48.337721+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:56:52.523361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.539984+0000","last_change":"2023-10-06T10:41:47.531430+0000","last_active":"2023-10-06T10:41:47.539984+0000","last_peered":"2023-10-06T10:41:47.539984+0000","last_clean":"2023-10-06T10:41:47.539984+0000","last_became_active":"2023-10-06T10:41:47.531123+0000","last_became_peered":"2023-10-06T10:41:47.531123+0000","last_unstale":"2023-10-06T10:41:47.539984+0000","last_undegraded":"2023-10-06T10:41:47.539984+0000","last_fullsized":"2023-10-06T10:41:47.539984+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:44:15.789832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.330145+0000","last_change":"2023-10-06T10:41:48.330145+0000","last_active":"2023-10-06T10:41:48.330145+0000","last_peered":"2023-10-06T10:41:48.330145+0000","last_clean":"2023-10-06T10:41:48.330145+0000","last_became_active":"2023-10-06T10:41:48.329914+0000","last_became_peered":"2023-10-06T10:41:48.329914+0000","last_unstale":"2023-10-06T10:41:48.330145+0000","last_undegraded":"2023-10-06T10:41:48.330145+0000","last_fullsized":"2023-10-06T10:41:48.330145+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:23:52.001648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.539844+0000","last_change":"2023-10-06T10:41:47.529634+0000","last_active":"2023-10-06T10:41:47.539844+0000","last_peered":"2023-10-06T10:41:47.539844+0000","last_clean":"2023-10-06T10:41:47.539844+0000","last_became_active":"2023-10-06T10:41:47.528952+0000","last_became_peered":"2023-10-06T10:41:47.528952+0000","last_unstale":"2023-10-06T10:41:47.539844+0000","last_undegraded":"2023-10-06T10:41:47.539844+0000","last_fullsized":"2023-10-06T10:41:47.539844+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:47:44.875853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.322014+0000","last_change":"2023-10-06T10:41:48.322014+0000","last_active":"2023-10-06T10:41:48.322014+0000","last_peered":"2023-10-06T10:41:48.322014+0000","last_clean":"2023-10-06T10:41:48.322014+0000","last_became_active":"2023-10-06T10:41:48.321625+0000","last_became_peered":"2023-10-06T10:41:48.321625+0000","last_unstale":"2023-10-06T10:41:48.322014+0000","last_undegraded":"2023-10-06T10:41:48.322014+0000","last_fullsized":"2023-10-06T10:41:48.322014+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:58.300560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"63'8","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.546236+0000","last_change":"2023-10-06T10:41:47.528391+0000","last_active":"2023-10-06T10:41:48.546236+0000","last_peered":"2023-10-06T10:41:48.546236+0000","last_clean":"2023-10-06T10:41:48.546236+0000","last_became_active":"2023-10-06T10:41:47.528090+0000","last_became_peered":"2023-10-06T10:41:47.528090+0000","last_unstale":"2023-10-06T10:41:48.546236+0000","last_undegraded":"2023-10-06T10:41:48.546236+0000","last_fullsized":"2023-10-06T10:41:48.546236+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:27.657374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.323299+0000","last_change":"2023-10-06T10:41:48.323299+0000","last_active":"2023-10-06T10:41:48.323299+0000","last_peered":"2023-10-06T10:41:48.323299+0000","last_clean":"2023-10-06T10:41:48.323299+0000","last_became_active":"2023-10-06T10:41:48.323069+0000","last_became_peered":"2023-10-06T10:41:48.323069+0000","last_unstale":"2023-10-06T10:41:48.323299+0000","last_undegraded":"2023-10-06T10:41:48.323299+0000","last_fullsized":"2023-10-06T10:41:48.323299+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:45.653458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"63'2","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.545657+0000","last_change":"2023-10-06T10:41:47.530883+0000","last_active":"2023-10-06T10:41:48.545657+0000","last_peered":"2023-10-06T10:41:48.545657+0000","last_clean":"2023-10-06T10:41:48.545657+0000","last_became_active":"2023-10-06T10:41:47.530598+0000","last_became_peered":"2023-10-06T10:41:47.530598+0000","last_unstale":"2023-10-06T10:41:48.545657+0000","last_undegraded":"2023-10-06T10:41:48.545657+0000","last_fullsized":"2023-10-06T10:41:48.545657+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:49.034461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.328913+0000","last_change":"2023-10-06T10:41:48.328913+0000","last_active":"2023-10-06T10:41:48.328913+0000","last_peered":"2023-10-06T10:41:48.328913+0000","last_clean":"2023-10-06T10:41:48.328913+0000","last_became_active":"2023-10-06T10:41:48.328627+0000","last_became_peered":"2023-10-06T10:41:48.328627+0000","last_unstale":"2023-10-06T10:41:48.328913+0000","last_undegraded":"2023-10-06T10:41:48.328913+0000","last_fullsized":"2023-10-06T10:41:48.328913+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:18.730732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.539766+0000","last_change":"2023-10-06T10:41:47.529290+0000","last_active":"2023-10-06T10:41:47.539766+0000","last_peered":"2023-10-06T10:41:47.539766+0000","last_clean":"2023-10-06T10:41:47.539766+0000","last_became_active":"2023-10-06T10:41:47.529007+0000","last_became_peered":"2023-10-06T10:41:47.529007+0000","last_unstale":"2023-10-06T10:41:47.539766+0000","last_undegraded":"2023-10-06T10:41:47.539766+0000","last_fullsized":"2023-10-06T10:41:47.539766+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:58:28.156731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.333979+0000","last_change":"2023-10-06T10:41:48.333979+0000","last_active":"2023-10-06T10:41:48.333979+0000","last_peered":"2023-10-06T10:41:48.333979+0000","last_clean":"2023-10-06T10:41:48.333979+0000","last_became_active":"2023-10-06T10:41:48.333451+0000","last_became_peered":"2023-10-06T10:41:48.333451+0000","last_unstale":"2023-10-06T10:41:48.333979+0000","last_undegraded":"2023-10-06T10:41:48.333979+0000","last_fullsized":"2023-10-06T10:41:48.333979+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:43.365081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.540431+0000","last_change":"2023-10-06T10:41:47.532259+0000","last_active":"2023-10-06T10:41:47.540431+0000","last_peered":"2023-10-06T10:41:47.540431+0000","last_clean":"2023-10-06T10:41:47.540431+0000","last_became_active":"2023-10-06T10:41:47.531994+0000","last_became_peered":"2023-10-06T10:41:47.531994+0000","last_unstale":"2023-10-06T10:41:47.540431+0000","last_undegraded":"2023-10-06T10:41:47.540431+0000","last_fullsized":"2023-10-06T10:41:47.540431+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:36.921171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.338250+0000","last_change":"2023-10-06T10:41:48.338250+0000","last_active":"2023-10-06T10:41:48.338250+0000","last_peered":"2023-10-06T10:41:48.338250+0000","last_clean":"2023-10-06T10:41:48.338250+0000","last_became_active":"2023-10-06T10:41:48.337971+0000","last_became_peered":"2023-10-06T10:41:48.337971+0000","last_unstale":"2023-10-06T10:41:48.338250+0000","last_undegraded":"2023-10-06T10:41:48.338250+0000","last_fullsized":"2023-10-06T10:41:48.338250+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:18.730732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.534055+0000","last_change":"2023-10-06T10:41:47.530446+0000","last_active":"2023-10-06T10:41:47.534055+0000","last_peered":"2023-10-06T10:41:47.534055+0000","last_clean":"2023-10-06T10:41:47.534055+0000","last_became_active":"2023-10-06T10:41:47.530155+0000","last_became_peered":"2023-10-06T10:41:47.530155+0000","last_unstale":"2023-10-06T10:41:47.534055+0000","last_undegraded":"2023-10-06T10:41:47.534055+0000","last_fullsized":"2023-10-06T10:41:47.534055+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:28:13.339361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.322286+0000","last_change":"2023-10-06T10:41:48.322286+0000","last_active":"2023-10-06T10:41:48.322286+0000","last_peered":"2023-10-06T10:41:48.322286+0000","last_clean":"2023-10-06T10:41:48.322286+0000","last_became_active":"2023-10-06T10:41:48.322042+0000","last_became_peered":"2023-10-06T10:41:48.322042+0000","last_unstale":"2023-10-06T10:41:48.322286+0000","last_undegraded":"2023-10-06T10:41:48.322286+0000","last_fullsized":"2023-10-06T10:41:48.322286+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:40:18.488714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.539571+0000","last_change":"2023-10-06T10:41:47.526231+0000","last_active":"2023-10-06T10:41:47.539571+0000","last_peered":"2023-10-06T10:41:47.539571+0000","last_clean":"2023-10-06T10:41:47.539571+0000","last_became_active":"2023-10-06T10:41:47.525260+0000","last_became_peered":"2023-10-06T10:41:47.525260+0000","last_unstale":"2023-10-06T10:41:47.539571+0000","last_undegraded":"2023-10-06T10:41:47.539571+0000","last_fullsized":"2023-10-06T10:41:47.539571+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:54:04.991122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.335071+0000","last_change":"2023-10-06T10:41:48.335071+0000","last_active":"2023-10-06T10:41:48.335071+0000","last_peered":"2023-10-06T10:41:48.335071+0000","last_clean":"2023-10-06T10:41:48.335071+0000","last_became_active":"2023-10-06T10:41:48.334778+0000","last_became_peered":"2023-10-06T10:41:48.334778+0000","last_unstale":"2023-10-06T10:41:48.335071+0000","last_undegraded":"2023-10-06T10:41:48.335071+0000","last_fullsized":"2023-10-06T10:41:48.335071+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:33.349366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.530973+0000","last_change":"2023-10-06T10:41:47.525929+0000","last_active":"2023-10-06T10:41:47.530973+0000","last_peered":"2023-10-06T10:41:47.530973+0000","last_clean":"2023-10-06T10:41:47.530973+0000","last_became_active":"2023-10-06T10:41:47.525660+0000","last_became_peered":"2023-10-06T10:41:47.525660+0000","last_unstale":"2023-10-06T10:41:47.530973+0000","last_undegraded":"2023-10-06T10:41:47.530973+0000","last_fullsized":"2023-10-06T10:41:47.530973+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:25:37.651048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.326241+0000","last_change":"2023-10-06T10:41:48.326241+0000","last_active":"2023-10-06T10:41:48.326241+0000","last_peered":"2023-10-06T10:41:48.326241+0000","last_clean":"2023-10-06T10:41:48.326241+0000","last_became_active":"2023-10-06T10:41:48.326021+0000","last_became_peered":"2023-10-06T10:41:48.326021+0000","last_unstale":"2023-10-06T10:41:48.326241+0000","last_undegraded":"2023-10-06T10:41:48.326241+0000","last_fullsized":"2023-10-06T10:41:48.326241+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:50.041063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.534537+0000","last_change":"2023-10-06T10:41:47.528732+0000","last_active":"2023-10-06T10:41:47.534537+0000","last_peered":"2023-10-06T10:41:47.534537+0000","last_clean":"2023-10-06T10:41:47.534537+0000","last_became_active":"2023-10-06T10:41:47.528481+0000","last_became_peered":"2023-10-06T10:41:47.528481+0000","last_unstale":"2023-10-06T10:41:47.534537+0000","last_undegraded":"2023-10-06T10:41:47.534537+0000","last_fullsized":"2023-10-06T10:41:47.534537+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:03:35.210012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.330679+0000","last_change":"2023-10-06T10:41:48.330679+0000","last_active":"2023-10-06T10:41:48.330679+0000","last_peered":"2023-10-06T10:41:48.330679+0000","last_clean":"2023-10-06T10:41:48.330679+0000","last_became_active":"2023-10-06T10:41:48.330394+0000","last_became_peered":"2023-10-06T10:41:48.330394+0000","last_unstale":"2023-10-06T10:41:48.330679+0000","last_undegraded":"2023-10-06T10:41:48.330679+0000","last_fullsized":"2023-10-06T10:41:48.330679+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:11:17.706840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.534404+0000","last_change":"2023-10-06T10:41:47.526181+0000","last_active":"2023-10-06T10:41:47.534404+0000","last_peered":"2023-10-06T10:41:47.534404+0000","last_clean":"2023-10-06T10:41:47.534404+0000","last_became_active":"2023-10-06T10:41:47.525930+0000","last_became_peered":"2023-10-06T10:41:47.525930+0000","last_unstale":"2023-10-06T10:41:47.534404+0000","last_undegraded":"2023-10-06T10:41:47.534404+0000","last_fullsized":"2023-10-06T10:41:47.534404+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:12:00.503217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.334169+0000","last_change":"2023-10-06T10:41:48.334169+0000","last_active":"2023-10-06T10:41:48.334169+0000","last_peered":"2023-10-06T10:41:48.334169+0000","last_clean":"2023-10-06T10:41:48.334169+0000","last_became_active":"2023-10-06T10:41:48.333660+0000","last_became_peered":"2023-10-06T10:41:48.333660+0000","last_unstale":"2023-10-06T10:41:48.334169+0000","last_undegraded":"2023-10-06T10:41:48.334169+0000","last_fullsized":"2023-10-06T10:41:48.334169+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:02:12.095392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"63'2","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.538484+0000","last_change":"2023-10-06T10:41:47.531157+0000","last_active":"2023-10-06T10:41:48.538484+0000","last_peered":"2023-10-06T10:41:48.538484+0000","last_clean":"2023-10-06T10:41:48.538484+0000","last_became_active":"2023-10-06T10:41:47.530899+0000","last_became_peered":"2023-10-06T10:41:47.530899+0000","last_unstale":"2023-10-06T10:41:48.538484+0000","last_undegraded":"2023-10-06T10:41:48.538484+0000","last_fullsized":"2023-10-06T10:41:48.538484+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:11:16.700239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":53,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.520894+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:41:47.520894+0000","last_peered":"2023-10-06T10:41:47.520894+0000","last_clean":"2023-10-06T10:41:47.520894+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:41:47.520894+0000","last_undegraded":"2023-10-06T10:41:47.520894+0000","last_fullsized":"2023-10-06T10:41:47.520894+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.338317+0000","last_change":"2023-10-06T10:41:48.338317+0000","last_active":"2023-10-06T10:41:48.338317+0000","last_peered":"2023-10-06T10:41:48.338317+0000","last_clean":"2023-10-06T10:41:48.338317+0000","last_became_active":"2023-10-06T10:41:48.337659+0000","last_became_peered":"2023-10-06T10:41:48.337659+0000","last_unstale":"2023-10-06T10:41:48.338317+0000","last_undegraded":"2023-10-06T10:41:48.338317+0000","last_fullsized":"2023-10-06T10:41:48.338317+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:28:14.345962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"63'1","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.319500+0000","last_change":"2023-10-06T10:41:47.530910+0000","last_active":"2023-10-06T10:41:48.319500+0000","last_peered":"2023-10-06T10:41:48.319500+0000","last_clean":"2023-10-06T10:41:48.319500+0000","last_became_active":"2023-10-06T10:41:47.530622+0000","last_became_peered":"2023-10-06T10:41:47.530622+0000","last_unstale":"2023-10-06T10:41:48.319500+0000","last_undegraded":"2023-10-06T10:41:48.319500+0000","last_fullsized":"2023-10-06T10:41:48.319500+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:01:20.293841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":48,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.319582+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:41:48.319582+0000","last_peered":"2023-10-06T10:41:48.319582+0000","last_clean":"2023-10-06T10:41:48.319582+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:41:48.319582+0000","last_undegraded":"2023-10-06T10:41:48.319582+0000","last_fullsized":"2023-10-06T10:41:48.319582+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.328008+0000","last_change":"2023-10-06T10:41:48.325930+0000","last_active":"2023-10-06T10:41:48.328008+0000","last_peered":"2023-10-06T10:41:48.328008+0000","last_clean":"2023-10-06T10:41:48.328008+0000","last_became_active":"2023-10-06T10:41:48.325650+0000","last_became_peered":"2023-10-06T10:41:48.325650+0000","last_unstale":"2023-10-06T10:41:48.328008+0000","last_undegraded":"2023-10-06T10:41:48.328008+0000","last_fullsized":"2023-10-06T10:41:48.328008+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:28.883834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.545786+0000","last_change":"2023-10-06T10:41:47.540263+0000","last_active":"2023-10-06T10:41:47.545786+0000","last_peered":"2023-10-06T10:41:47.545786+0000","last_clean":"2023-10-06T10:41:47.545786+0000","last_became_active":"2023-10-06T10:41:47.539939+0000","last_became_peered":"2023-10-06T10:41:47.539939+0000","last_unstale":"2023-10-06T10:41:47.545786+0000","last_undegraded":"2023-10-06T10:41:47.545786+0000","last_fullsized":"2023-10-06T10:41:47.545786+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:37.248288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.513961+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:41:47.513961+0000","last_peered":"2023-10-06T10:41:47.513961+0000","last_clean":"2023-10-06T10:41:47.513961+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:41:47.513961+0000","last_undegraded":"2023-10-06T10:41:47.513961+0000","last_fullsized":"2023-10-06T10:41:47.513961+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":194,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.518263+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:41:47.518263+0000","last_peered":"2023-10-06T10:41:47.518263+0000","last_clean":"2023-10-06T10:41:47.518263+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:41:47.518263+0000","last_undegraded":"2023-10-06T10:41:47.518263+0000","last_fullsized":"2023-10-06T10:41:47.518263+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.332796+0000","last_change":"2023-10-06T10:41:48.330936+0000","last_active":"2023-10-06T10:41:48.332796+0000","last_peered":"2023-10-06T10:41:48.332796+0000","last_clean":"2023-10-06T10:41:48.332796+0000","last_became_active":"2023-10-06T10:41:48.330681+0000","last_became_peered":"2023-10-06T10:41:48.330681+0000","last_unstale":"2023-10-06T10:41:48.332796+0000","last_undegraded":"2023-10-06T10:41:48.332796+0000","last_fullsized":"2023-10-06T10:41:48.332796+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:54:03.085432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.534187+0000","last_change":"2023-10-06T10:41:47.528684+0000","last_active":"2023-10-06T10:41:47.534187+0000","last_peered":"2023-10-06T10:41:47.534187+0000","last_clean":"2023-10-06T10:41:47.534187+0000","last_became_active":"2023-10-06T10:41:47.528423+0000","last_became_peered":"2023-10-06T10:41:47.528423+0000","last_unstale":"2023-10-06T10:41:47.534187+0000","last_undegraded":"2023-10-06T10:41:47.534187+0000","last_fullsized":"2023-10-06T10:41:47.534187+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:04:12.351380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":44,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.520537+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:41:47.520537+0000","last_peered":"2023-10-06T10:41:47.520537+0000","last_clean":"2023-10-06T10:41:47.520537+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:41:47.520537+0000","last_undegraded":"2023-10-06T10:41:47.520537+0000","last_fullsized":"2023-10-06T10:41:47.520537+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.327973+0000","last_change":"2023-10-06T10:41:48.325694+0000","last_active":"2023-10-06T10:41:48.327973+0000","last_peered":"2023-10-06T10:41:48.327973+0000","last_clean":"2023-10-06T10:41:48.327973+0000","last_became_active":"2023-10-06T10:41:48.325486+0000","last_became_peered":"2023-10-06T10:41:48.325486+0000","last_unstale":"2023-10-06T10:41:48.327973+0000","last_undegraded":"2023-10-06T10:41:48.327973+0000","last_fullsized":"2023-10-06T10:41:48.327973+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:33:38.497872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"63'2","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.545729+0000","last_change":"2023-10-06T10:41:47.526416+0000","last_active":"2023-10-06T10:41:48.545729+0000","last_peered":"2023-10-06T10:41:48.545729+0000","last_clean":"2023-10-06T10:41:48.545729+0000","last_became_active":"2023-10-06T10:41:47.526071+0000","last_became_peered":"2023-10-06T10:41:47.526071+0000","last_unstale":"2023-10-06T10:41:48.545729+0000","last_undegraded":"2023-10-06T10:41:48.545729+0000","last_fullsized":"2023-10-06T10:41:48.545729+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:24.524122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":44,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.518145+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:41:47.518145+0000","last_peered":"2023-10-06T10:41:47.518145+0000","last_clean":"2023-10-06T10:41:47.518145+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:41:47.518145+0000","last_undegraded":"2023-10-06T10:41:47.518145+0000","last_fullsized":"2023-10-06T10:41:47.518145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.326164+0000","last_change":"2023-10-06T10:41:48.323837+0000","last_active":"2023-10-06T10:41:48.326164+0000","last_peered":"2023-10-06T10:41:48.326164+0000","last_clean":"2023-10-06T10:41:48.326164+0000","last_became_active":"2023-10-06T10:41:48.323293+0000","last_became_peered":"2023-10-06T10:41:48.323293+0000","last_unstale":"2023-10-06T10:41:48.326164+0000","last_undegraded":"2023-10-06T10:41:48.326164+0000","last_fullsized":"2023-10-06T10:41:48.326164+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:53:57.341709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.539841+0000","last_change":"2023-10-06T10:41:47.529077+0000","last_active":"2023-10-06T10:41:47.539841+0000","last_peered":"2023-10-06T10:41:47.539841+0000","last_clean":"2023-10-06T10:41:47.539841+0000","last_became_active":"2023-10-06T10:41:47.528659+0000","last_became_peered":"2023-10-06T10:41:47.528659+0000","last_unstale":"2023-10-06T10:41:47.539841+0000","last_undegraded":"2023-10-06T10:41:47.539841+0000","last_fullsized":"2023-10-06T10:41:47.539841+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:38.622977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.515837+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:41:47.515837+0000","last_peered":"2023-10-06T10:41:47.515837+0000","last_clean":"2023-10-06T10:41:47.515837+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:41:47.515837+0000","last_undegraded":"2023-10-06T10:41:47.515837+0000","last_fullsized":"2023-10-06T10:41:47.515837+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.322054+0000","last_change":"2023-10-06T10:41:48.322054+0000","last_active":"2023-10-06T10:41:48.322054+0000","last_peered":"2023-10-06T10:41:48.322054+0000","last_clean":"2023-10-06T10:41:48.322054+0000","last_became_active":"2023-10-06T10:41:48.321825+0000","last_became_peered":"2023-10-06T10:41:48.321825+0000","last_unstale":"2023-10-06T10:41:48.322054+0000","last_undegraded":"2023-10-06T10:41:48.322054+0000","last_fullsized":"2023-10-06T10:41:48.322054+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:39.629579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"63'1","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.319265+0000","last_change":"2023-10-06T10:41:47.527450+0000","last_active":"2023-10-06T10:41:48.319265+0000","last_peered":"2023-10-06T10:41:48.319265+0000","last_clean":"2023-10-06T10:41:48.319265+0000","last_became_active":"2023-10-06T10:41:47.526356+0000","last_became_peered":"2023-10-06T10:41:47.526356+0000","last_unstale":"2023-10-06T10:41:48.319265+0000","last_undegraded":"2023-10-06T10:41:48.319265+0000","last_fullsized":"2023-10-06T10:41:48.319265+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:57.293959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":48,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.522694+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:41:47.522694+0000","last_peered":"2023-10-06T10:41:47.522694+0000","last_clean":"2023-10-06T10:41:47.522694+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:41:47.522694+0000","last_undegraded":"2023-10-06T10:41:47.522694+0000","last_fullsized":"2023-10-06T10:41:47.522694+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T10:41:48.336900+0000","last_change":"2023-10-06T10:41:48.336900+0000","last_active":"2023-10-06T10:41:48.336900+0000","last_peered":"2023-10-06T10:41:48.336900+0000","last_clean":"2023-10-06T10:41:48.336900+0000","last_became_active":"2023-10-06T10:41:48.336683+0000","last_became_peered":"2023-10-06T10:41:48.336683+0000","last_unstale":"2023-10-06T10:41:48.336900+0000","last_undegraded":"2023-10-06T10:41:48.336900+0000","last_fullsized":"2023-10-06T10:41:48.336900+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:47.507813+0000","last_clean_scrub_stamp":"2023-10-06T10:41:47.507813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:41:32.889178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.531772+0000","last_change":"2023-10-06T10:41:47.528008+0000","last_active":"2023-10-06T10:41:47.531772+0000","last_peered":"2023-10-06T10:41:47.531772+0000","last_clean":"2023-10-06T10:41:47.531772+0000","last_became_active":"2023-10-06T10:41:47.527743+0000","last_became_peered":"2023-10-06T10:41:47.527743+0000","last_unstale":"2023-10-06T10:41:47.531772+0000","last_undegraded":"2023-10-06T10:41:47.531772+0000","last_fullsized":"2023-10-06T10:41:47.531772+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:41:46.501211+0000","last_clean_scrub_stamp":"2023-10-06T10:41:46.501211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:26:00.759449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":48,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-06T10:41:47.522511+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:41:47.522511+0000","last_peered":"2023-10-06T10:41:47.522511+0000","last_clean":"2023-10-06T10:41:47.522511+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:41:47.522511+0000","last_undegraded":"2023-10-06T10:41:47.522511+0000","last_fullsized":"2023-10-06T10:41:47.522511+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298620,"kb_used_data":1332,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073220,"statfs":{"total":96636764160,"available":96330977280,"internally_reserved":0,"allocated":1363968,"data_stored":949321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.49399999999999999,"15min":0.48199999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.50900000000000001,"15min":0.51100000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76100000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.41399999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47099999999999997,"15min":0.45900000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.63300000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.44},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.51000000000000001,"15min":0.502},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.33800000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.42499999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.747,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.36699999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.52200000000000002,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55300000000000005,"15min":0.54400000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.55400000000000005}]},{"osd":4,"last update":"Fri Oct 6 10:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.46000000000000002,"15min":0.47399999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.503,"15min":0.50900000000000001},"min":{"1min":0.29599999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.51200000000000001}]},{"osd":5,"last update":"Fri Oct 6 10:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66500000000000004,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.502},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54000000000000004,"15min":0.55100000000000005},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.54100000000000004}]},{"osd":6,"last update":"Fri Oct 6 10:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56000000000000005,"15min":0.57399999999999995},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.77700000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.56399999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298036,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073804,"statfs":{"total":96636764160,"available":96331575296,"internally_reserved":0,"allocated":765952,"data_stored":357104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.36199999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.68400000000000005,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.499},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.45900000000000002,"15min":0.434},"min":{"1min":0.41999999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.60599999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.57599999999999996}]},{"osd":1,"last update":"Fri Oct 6 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.46400000000000002,"15min":0.441},"min":{"1min":0.32200000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.69499999999999995,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.49399999999999999,"15min":0.47199999999999998},"min":{"1min":0.42499999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.66200000000000003,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.434}]},{"osd":2,"last update":"Fri Oct 6 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48699999999999999,"15min":0.47099999999999997},"min":{"1min":0.35899999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.64200000000000002,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.48999999999999999,"15min":0.46800000000000003},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.63200000000000001,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.65300000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.50600000000000001,"15min":0.49099999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.66500000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52200000000000002,"15min":0.50800000000000001},"min":{"1min":0.40100000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65300000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.55600000000000005}]},{"osd":4,"last update":"Fri Oct 6 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.69799999999999995,"5min":0.745,"15min":0.745},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.48099999999999998,"15min":0.48299999999999998},"min":{"1min":0.31900000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.72099999999999997,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.48499999999999999,"15min":0.48399999999999999},"min":{"1min":0.33700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.67400000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.36299999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.73699999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.63900000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.373,"5min":0.313,"15min":0.313},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.39100000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70799999999999996,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.627}]}]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298632,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073208,"statfs":{"total":96636764160,"available":96330964992,"internally_reserved":0,"allocated":1376256,"data_stored":950857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33100000000000002,"15min":0.32900000000000001},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.35399999999999998,"15min":0.34799999999999998},"min":{"1min":0.26400000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.28000000000000003}]},{"osd":2,"last update":"Fri Oct 6 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.33200000000000002,"15min":0.32100000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.374,"15min":0.36399999999999999},"min":{"1min":0.255,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.35899999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.39000000000000001,"15min":0.378},"min":{"1min":0.27600000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.379,"15min":0.375},"min":{"1min":0.254,"5min":0.247,"15min":0.247},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.33400000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.52100000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57399999999999995,"15min":0.57099999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55400000000000005,"15min":0.54300000000000004},"min":{"1min":0.37,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.45600000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.39600000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61199999999999999,"15min":0.60699999999999998},"min":{"1min":0.41599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93500000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.51200000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.56899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298032,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073808,"statfs":{"total":96636764160,"available":96331579392,"internally_reserved":0,"allocated":761856,"data_stored":358365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.32400000000000001,"15min":0.318},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.53200000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.255},{"interface":"front","average":{"1min":0.374,"5min":0.35599999999999998,"15min":0.34300000000000003},"min":{"1min":0.28000000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57599999999999996,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.29599999999999999}]},{"osd":2,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.311,"15min":0.30499999999999999},"min":{"1min":0.22600000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.443,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.33700000000000002,"15min":0.32600000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.51600000000000001,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.33700000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34899999999999998,"15min":0.34300000000000003},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35599999999999998,"15min":0.34999999999999998},"min":{"1min":0.27400000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.315}]},{"osd":4,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55800000000000005,"15min":0.56000000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69099999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.67800000000000005,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.38}]},{"osd":5,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.39700000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68300000000000005,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72899999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.52400000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.436,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.68600000000000005,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.503},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.69299999999999995,"5min":1.075,"15min":1.075},"last":0.39500000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.441,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72199999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.41899999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.73499999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.54400000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298032,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073808,"statfs":{"total":96636764160,"available":96331579392,"internally_reserved":0,"allocated":761856,"data_stored":358365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.315,"15min":0.311},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.496,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.221},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.33400000000000002,"15min":0.32000000000000001},"min":{"1min":0.23499999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.51900000000000002,"5min":0.622,"15min":0.622},"last":0.29899999999999999}]},{"osd":1,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32900000000000001,"15min":0.32300000000000001},"min":{"1min":0.22500000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.441,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36699999999999999,"15min":0.36499999999999999},"min":{"1min":0.26500000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.47699999999999998,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.23599999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.35499999999999998,"15min":0.34699999999999998},"min":{"1min":0.28499999999999998,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.50900000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.36399999999999999,"15min":0.35699999999999998},"min":{"1min":0.28899999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.52800000000000002,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.28899999999999998}]},{"osd":4,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.33400000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.70299999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.67800000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.49099999999999999}]},{"osd":5,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70699999999999996,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.36599999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.748,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.499}]},{"osd":6,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61599999999999999,"15min":0.621},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72299999999999998,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.61499999999999999,"15min":0.62},"min":{"1min":0.45700000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71699999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.58999999999999997}]},{"osd":7,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.40600000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76600000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.45500000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.78700000000000003,"5min":0.874,"15min":0.874},"last":0.56899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298040,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073800,"statfs":{"total":96636764160,"available":96331571200,"internally_reserved":0,"allocated":770048,"data_stored":358477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.40899999999999997,"15min":0.44700000000000001},"min":{"1min":0.224,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.44500000000000001,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.35999999999999999,"15min":0.36199999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.41499999999999998,"5min":1.218,"15min":1.218},"last":0.249}]},{"osd":1,"last update":"Fri Oct 6 10:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.26900000000000002,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.432,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.45200000000000001,"15min":0.48499999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.442,"5min":3.359,"15min":3.359},"last":0.32600000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.378,"15min":0.38200000000000001},"min":{"1min":0.26000000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.47599999999999998,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.312},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.57199999999999995,"15min":0.65200000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.43099999999999999,"5min":7.069,"15min":7.069},"last":0.30499999999999999}]},{"osd":4,"last update":"Fri Oct 6 10:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.58299999999999996,"15min":0.58799999999999997},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.68899999999999995,"5min":1.165,"15min":1.165},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.378,"5min":0.375,"15min":0.375},"max":{"1min":0.747,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.73799999999999999}]},{"osd":5,"last update":"Fri Oct 6 10:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.66900000000000004,"15min":0.70399999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.75,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39700000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68899999999999995,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.51600000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.45200000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.74199999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.69999999999999996,"15min":0.73299999999999998},"min":{"1min":0.45800000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79200000000000004,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.71099999999999997}]},{"osd":7,"last update":"Fri Oct 6 10:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.46000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.72899999999999998,"5min":1.137,"15min":1.137},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.44400000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.71799999999999997,"5min":1.014,"15min":1.014},"last":0.71699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298036,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073804,"statfs":{"total":96636764160,"available":96331575296,"internally_reserved":0,"allocated":765952,"data_stored":358435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.48999999999999999,"15min":0.48399999999999999},"min":{"1min":0.38800000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66800000000000004,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.41299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66100000000000003,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.503,"15min":0.499},"min":{"1min":0.372,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53200000000000003,"15min":0.53900000000000003},"min":{"1min":0.34699999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66900000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.39000000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.51300000000000001,"15min":0.51000000000000001},"min":{"1min":0.41599999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65800000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.442,"5min":0.315,"15min":0.315},"max":{"1min":0.66900000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48099999999999998}]},{"osd":3,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.42099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.65900000000000003,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.53600000000000003},"min":{"1min":0.39200000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69499999999999995,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.48299999999999998,"15min":0.504},"min":{"1min":0.33800000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67500000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.436},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.495,"15min":0.502},"min":{"1min":0.36799999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.40400000000000003}]},{"osd":6,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.52400000000000002,"15min":0.54200000000000004},"min":{"1min":0.32400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.71799999999999997,"5min":0.75,"15min":0.75},"last":0.377},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56200000000000006,"15min":0.57699999999999996},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.68200000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.53900000000000003}]},{"osd":7,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55900000000000005,"15min":0.57699999999999996},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57999999999999996,"15min":0.59799999999999998},"min":{"1min":0.39400000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81299999999999994,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.51800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574895,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298036,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073804,"statfs":{"total":96636764160,"available":96331575296,"internally_reserved":0,"allocated":765952,"data_stored":358455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48099999999999998,"15min":0.46500000000000002},"min":{"1min":0.315,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72299999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.35399999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.86499999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.39000000000000001}]},{"osd":1,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.496,"15min":0.48499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70699999999999996,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52900000000000003,"15min":0.51800000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.52100000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51900000000000002,"15min":0.502},"min":{"1min":0.34799999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76300000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.56699999999999995}]},{"osd":3,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56799999999999995,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.67900000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.62},{"interface":"front","average":{"1min":0.5,"5min":0.55000000000000004,"15min":0.55900000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79100000000000004,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.55400000000000005}]},{"osd":4,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47999999999999998,"15min":0.48699999999999999},"min":{"1min":0.32600000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.64900000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.505,"15min":0.50800000000000001},"min":{"1min":0.35099999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.70399999999999996,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.505}]},{"osd":6,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.50900000000000001,"15min":0.52000000000000002},"min":{"1min":0.32100000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77200000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.746,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.42999999999999999}]},{"osd":7,"last update":"Fri Oct 6 10:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57299999999999995,"15min":0.58299999999999996},"min":{"1min":0.42999999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.745,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.39200000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.63700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:41:52.963 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T10:41:52.964 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T10:41:53.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:41:53.078 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:41:53.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.115+0000 7f2ebf181700 1 Processor -- start 2023-10-06T10:41:53.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.115+0000 7f2ebf181700 1 -- start start 2023-10-06T10:41:53.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.116+0000 7f2ebf181700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2eb80ccbf0 0x7f2eb806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:53.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.116+0000 7f2ebf181700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2eb80675b0 0x7f2eb80679f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:53.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.116+0000 7f2ebf181700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb8067f30 con 0x7f2eb80cc820 2023-10-06T10:41:53.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.116+0000 7f2ebf181700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb8159bc0 con 0x7f2eb80ccbf0 2023-10-06T10:41:53.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.116+0000 7f2ebf181700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb815ab70 con 0x7f2eb80675b0 2023-10-06T10:41:53.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.116+0000 7f2eaffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2eb80ccbf0 0x7f2eb806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:53.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.116+0000 7f2ebcf1d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2eb80cc820 0x7f2eb80c9220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33266/0 (socket says 172.21.15.92:33266) 2023-10-06T10:41:53.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.116+0000 7f2eaffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2eb80ccbf0 0x7f2eb806efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57280/0 (socket says 172.21.15.92:57280) 2023-10-06T10:41:53.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.116+0000 7f2ebcf1d700 1 -- 172.21.15.92:0/671207601 learned_addr learned my addr 172.21.15.92:0/671207601 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:53.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.117+0000 7f2ebd71e700 1 --2- 172.21.15.92:0/671207601 >> v2:172.21.15.124:3300/0 conn(0x7f2eb80675b0 0x7f2eb80679f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:53.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.117+0000 7f2eaffff700 1 -- 172.21.15.92:0/671207601 >> v2:172.21.15.124:3300/0 conn(0x7f2eb80675b0 msgr2=0x7f2eb80679f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:53.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.117+0000 7f2eaffff700 1 --2- 172.21.15.92:0/671207601 >> v2:172.21.15.124:3300/0 conn(0x7f2eb80675b0 0x7f2eb80679f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:53.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.117+0000 7f2eaffff700 1 -- 172.21.15.92:0/671207601 >> v1:172.21.15.92:6789/0 conn(0x7f2eb80cc820 legacy=0x7f2eb80c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:53.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.117+0000 7f2eaffff700 1 -- 172.21.15.92:0/671207601 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb80cbf70 con 0x7f2eb80ccbf0 2023-10-06T10:41:53.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.117+0000 7f2eaf7fe700 1 -- 172.21.15.92:0/671207601 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334998842 0 0) 0x7f2eb8067f30 con 0x7f2eb80cc820 2023-10-06T10:41:53.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.117+0000 7f2ebd71e700 1 --2- 172.21.15.92:0/671207601 >> v2:172.21.15.124:3300/0 conn(0x7f2eb80675b0 0x7f2eb80679f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:53.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.117+0000 7f2eaffff700 1 --2- 172.21.15.92:0/671207601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2eb80ccbf0 0x7f2eb806efb0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8071270 tx=0x7f2ea80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c966fe69ec0a149 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:53.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.117+0000 7f2eaf7fe700 1 -- 172.21.15.92:0/671207601 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea800b6b0 con 0x7f2eb80ccbf0 2023-10-06T10:41:53.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.117+0000 7f2eaf7fe700 1 -- 172.21.15.92:0/671207601 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea800b810 con 0x7f2eb80ccbf0 2023-10-06T10:41:53.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.117+0000 7f2eaf7fe700 1 -- 172.21.15.92:0/671207601 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea800bbb0 con 0x7f2eb80ccbf0 2023-10-06T10:41:53.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.117+0000 7f2ebf181700 1 -- 172.21.15.92:0/671207601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2eb80ccbf0 msgr2=0x7f2eb806efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:53.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.117+0000 7f2ebf181700 1 --2- 172.21.15.92:0/671207601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2eb80ccbf0 0x7f2eb806efb0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8071270 tx=0x7f2ea80091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:53.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.118+0000 7f2ebf181700 1 -- 172.21.15.92:0/671207601 shutdown_connections 2023-10-06T10:41:53.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.118+0000 7f2ebf181700 1 --2- 172.21.15.92:0/671207601 >> v2:172.21.15.124:3300/0 conn(0x7f2eb80675b0 0x7f2eb80679f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:53.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.118+0000 7f2ebf181700 1 --2- 172.21.15.92:0/671207601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2eb80ccbf0 0x7f2eb806efb0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:53.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.118+0000 7f2ebf181700 1 -- 172.21.15.92:0/671207601 >> 172.21.15.92:0/671207601 conn(0x7f2eb8069190 msgr2=0x7f2eb80cd180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:53.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.118+0000 7f2ebf181700 1 -- 172.21.15.92:0/671207601 shutdown_connections 2023-10-06T10:41:53.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.118+0000 7f2ebf181700 1 -- 172.21.15.92:0/671207601 wait complete. 2023-10-06T10:41:53.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.119+0000 7f2ebf181700 1 Processor -- start 2023-10-06T10:41:53.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.119+0000 7f2ebf181700 1 -- start start 2023-10-06T10:41:53.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.119+0000 7f2ebf181700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2eb80675b0 0x7f2eb8169250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:53.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.119+0000 7f2ebf181700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2eb80cc820 0x7f2eb8159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:53.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.119+0000 7f2ebf181700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2eb80ccbf0 0x7f2eb80acc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:53.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.119+0000 7f2ebf181700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb814f4e0 con 0x7f2eb80cc820 2023-10-06T10:41:53.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.119+0000 7f2ebf181700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb814f050 con 0x7f2eb80ccbf0 2023-10-06T10:41:53.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.119+0000 7f2ebf181700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb814e440 con 0x7f2eb80675b0 2023-10-06T10:41:53.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.120+0000 7f2eaffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2eb80cc820 0x7f2eb8159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:53.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.120+0000 7f2eaffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2eb80cc820 0x7f2eb8159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57290/0 (socket says 172.21.15.92:57290) 2023-10-06T10:41:53.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.120+0000 7f2eaffff700 1 -- 172.21.15.92:0/3941349102 learned_addr learned my addr 172.21.15.92:0/3941349102 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:53.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.120+0000 7f2ebd71e700 1 --2- 172.21.15.92:0/3941349102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2eb80ccbf0 0x7f2eb80acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:53.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.120+0000 7f2ebcf1d700 1 --2- 172.21.15.92:0/3941349102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2eb80675b0 0x7f2eb8169250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:53.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.120+0000 7f2eaffff700 1 -- 172.21.15.92:0/3941349102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2eb80675b0 msgr2=0x7f2eb8169250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:53.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.120+0000 7f2eaffff700 1 --2- 172.21.15.92:0/3941349102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2eb80675b0 0x7f2eb8169250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:53.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.120+0000 7f2eaffff700 1 -- 172.21.15.92:0/3941349102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2eb80ccbf0 msgr2=0x7f2eb80acc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:53.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.120+0000 7f2eaffff700 1 --2- 172.21.15.92:0/3941349102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2eb80ccbf0 0x7f2eb80acc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:53.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.120+0000 7f2eaffff700 1 -- 172.21.15.92:0/3941349102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea8009420 con 0x7f2eb80cc820 2023-10-06T10:41:53.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.120+0000 7f2ebcf1d700 1 --2- 172.21.15.92:0/3941349102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2eb80675b0 0x7f2eb8169250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:53.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.120+0000 7f2eaffff700 1 --2- 172.21.15.92:0/3941349102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2eb80cc820 0x7f2eb8159310 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f2eb806dfb0 tx=0x7f2ea8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:53.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.120+0000 7f2eacff9700 1 -- 172.21.15.92:0/3941349102 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea800bb90 con 0x7f2eb80cc820 2023-10-06T10:41:53.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.120+0000 7f2eacff9700 1 -- 172.21.15.92:0/3941349102 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea8011420 con 0x7f2eb80cc820 2023-10-06T10:41:53.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.120+0000 7f2ebf181700 1 -- 172.21.15.92:0/3941349102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb814e720 con 0x7f2eb80cc820 2023-10-06T10:41:53.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.120+0000 7f2eacff9700 1 -- 172.21.15.92:0/3941349102 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea80117a0 con 0x7f2eb80cc820 2023-10-06T10:41:53.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.120+0000 7f2ebf181700 1 -- 172.21.15.92:0/3941349102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb814ece0 con 0x7f2eb80cc820 2023-10-06T10:41:53.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.121+0000 7f2ebf181700 1 -- 172.21.15.92:0/3941349102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb80b10a0 con 0x7f2eb80cc820 2023-10-06T10:41:53.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.122+0000 7f2eacff9700 1 -- 172.21.15.92:0/3941349102 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ea8011b90 con 0x7f2eb80cc820 2023-10-06T10:41:53.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.122+0000 7f2eacff9700 1 --2- 172.21.15.92:0/3941349102 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2ea4066f30 0x7f2ea40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:53.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.122+0000 7f2eacff9700 1 -- 172.21.15.92:0/3941349102 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2ea8086140 con 0x7f2eb80cc820 2023-10-06T10:41:53.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.122+0000 7f2ebcf1d700 1 --2- 172.21.15.92:0/3941349102 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2ea4066f30 0x7f2ea40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:53.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.122+0000 7f2ebcf1d700 1 --2- 172.21.15.92:0/3941349102 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2ea4066f30 0x7f2ea40693e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2eb816ce80 tx=0x7f2eb4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:53.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.126+0000 7f2eacff9700 1 -- 172.21.15.92:0/3941349102 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ea8053110 con 0x7f2eb80cc820 2023-10-06T10:41:53.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.383+0000 7f2ebf181700 1 -- 172.21.15.92:0/3941349102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2eb814f1e0 con 0x7f2eb80cc820 2023-10-06T10:41:53.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.386+0000 7f2eacff9700 1 -- 172.21.15.92:0/3941349102 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f2ea8056730 con 0x7f2eb80cc820 2023-10-06T10:41:53.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.389+0000 7f2ebf181700 1 -- 172.21.15.92:0/3941349102 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2ea4066f30 msgr2=0x7f2ea40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:53.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.389+0000 7f2ebf181700 1 --2- 172.21.15.92:0/3941349102 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2ea4066f30 0x7f2ea40693e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2eb816ce80 tx=0x7f2eb4009040 comp rx=0 tx=0).stop 2023-10-06T10:41:53.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.389+0000 7f2ebf181700 1 -- 172.21.15.92:0/3941349102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2eb80cc820 msgr2=0x7f2eb8159310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:53.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.389+0000 7f2ebf181700 1 --2- 172.21.15.92:0/3941349102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2eb80cc820 0x7f2eb8159310 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f2eb806dfb0 tx=0x7f2ea8017000 comp rx=0 tx=0).stop 2023-10-06T10:41:53.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.389+0000 7f2ebf181700 1 -- 172.21.15.92:0/3941349102 shutdown_connections 2023-10-06T10:41:53.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.389+0000 7f2ebf181700 1 --2- 172.21.15.92:0/3941349102 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2ea4066f30 0x7f2ea40693e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:53.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.389+0000 7f2ebf181700 1 --2- 172.21.15.92:0/3941349102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2eb80ccbf0 0x7f2eb80acc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:53.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.389+0000 7f2ebf181700 1 --2- 172.21.15.92:0/3941349102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2eb80675b0 0x7f2eb8169250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:53.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.389+0000 7f2ebf181700 1 --2- 172.21.15.92:0/3941349102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2eb80cc820 0x7f2eb8159310 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:53.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.389+0000 7f2ebf181700 1 -- 172.21.15.92:0/3941349102 >> 172.21.15.92:0/3941349102 conn(0x7f2eb8069190 msgr2=0x7f2eb80cb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:53.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.389+0000 7f2ebf181700 1 -- 172.21.15.92:0/3941349102 shutdown_connections 2023-10-06T10:41:53.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.389+0000 7f2ebf181700 1 -- 172.21.15.92:0/3941349102 wait complete. 2023-10-06T10:41:53.393 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-10-06T10:41:53.403 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:41:53.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.516+0000 7fcc2d413700 1 Processor -- start 2023-10-06T10:41:53.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.516+0000 7fcc2d413700 1 -- start start 2023-10-06T10:41:53.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.517+0000 7fcc2d413700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcc280b0760 0x7fcc280ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:53.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.517+0000 7fcc2d413700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc280b0c60 con 0x7fcc28159560 2023-10-06T10:41:53.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.517+0000 7fcc2d413700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc280b0f40 con 0x7fcc280b49a0 2023-10-06T10:41:53.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.517+0000 7fcc2d413700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc280ab350 con 0x7fcc280b0760 2023-10-06T10:41:53.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.517+0000 7fcc26ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcc28159560 0x7fcc280aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54762/0 (socket says 172.21.15.92:54762) 2023-10-06T10:41:53.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.517+0000 7fcc26ffd700 1 -- 172.21.15.92:0/1084464360 learned_addr learned my addr 172.21.15.92:0/1084464360 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:53.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.517+0000 7fcc277fe700 1 --2- 172.21.15.92:0/1084464360 >> v2:172.21.15.124:3300/0 conn(0x7fcc280b0760 0x7fcc280ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:53.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.517+0000 7fcc267fc700 1 -- 172.21.15.92:0/1084464360 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3008215923 0 0) 0x7fcc280b0c60 con 0x7fcc28159560 2023-10-06T10:41:53.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.517+0000 7fcc267fc700 1 -- 172.21.15.92:0/1084464360 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc14003570 con 0x7fcc28159560 2023-10-06T10:41:53.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.517+0000 7fcc277fe700 1 -- 172.21.15.92:0/1084464360 >> v1:172.21.15.124:6789/0 conn(0x7fcc280b49a0 legacy=0x7fcc280aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:53.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.517+0000 7fcc267fc700 1 -- 172.21.15.92:0/1084464360 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 470727115 0 0) 0x7fcc14003570 con 0x7fcc28159560 2023-10-06T10:41:53.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.517+0000 7fcc277fe700 1 -- 172.21.15.92:0/1084464360 >> v1:172.21.15.92:6789/0 conn(0x7fcc28159560 legacy=0x7fcc280aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:53.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.517+0000 7fcc277fe700 1 -- 172.21.15.92:0/1084464360 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc280b1220 con 0x7fcc280b0760 2023-10-06T10:41:53.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.518+0000 7fcc277fe700 1 --2- 172.21.15.92:0/1084464360 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc280b0760 0x7fcc280ccf90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009e90 tx=0x7fcc20009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9acbcc34886a4f00 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:53.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.518+0000 7fcc267fc700 1 -- 172.21.15.92:0/1084464360 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2000e070 con 0x7fcc280b0760 2023-10-06T10:41:53.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.518+0000 7fcc267fc700 1 -- 172.21.15.92:0/1084464360 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2000b920 con 0x7fcc280b0760 2023-10-06T10:41:53.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.518+0000 7fcc267fc700 1 -- 172.21.15.92:0/1084464360 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc180015f0 con 0x7fcc280b0760 2023-10-06T10:41:53.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.518+0000 7fcc2d413700 1 -- 172.21.15.92:0/1084464360 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc280b0760 msgr2=0x7fcc280ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:53.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.518+0000 7fcc2d413700 1 --2- 172.21.15.92:0/1084464360 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc280b0760 0x7fcc280ccf90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009e90 tx=0x7fcc20009780 comp rx=0 tx=0).stop 2023-10-06T10:41:53.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.518+0000 7fcc2d413700 1 -- 172.21.15.92:0/1084464360 shutdown_connections 2023-10-06T10:41:53.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.518+0000 7fcc2d413700 1 --2- 172.21.15.92:0/1084464360 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc280b0760 0x7fcc280ccf90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:53.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.518+0000 7fcc2d413700 1 -- 172.21.15.92:0/1084464360 >> 172.21.15.92:0/1084464360 conn(0x7fcc280997f0 msgr2=0x7fcc2806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:53.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.518+0000 7fcc2d413700 1 -- 172.21.15.92:0/1084464360 shutdown_connections 2023-10-06T10:41:53.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.518+0000 7fcc2d413700 1 -- 172.21.15.92:0/1084464360 wait complete. 2023-10-06T10:41:53.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.519+0000 7fcc2d413700 1 Processor -- start 2023-10-06T10:41:53.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.519+0000 7fcc2d413700 1 -- start start 2023-10-06T10:41:53.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.520+0000 7fcc2d413700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc280b0760 0x7fcc28169b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:53.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.520+0000 7fcc2d413700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc280b49a0 0x7fcc280ab490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:53.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.520+0000 7fcc2d413700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc28159560 0x7fcc280ab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:53.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.520+0000 7fcc2d413700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2816a0d0 con 0x7fcc28159560 2023-10-06T10:41:53.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.520+0000 7fcc2d413700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280abf10 con 0x7fcc280b0760 2023-10-06T10:41:53.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.520+0000 7fcc2d413700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280acd60 con 0x7fcc280b49a0 2023-10-06T10:41:53.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.520+0000 7fcc277fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc28159560 0x7fcc280ab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:53.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.520+0000 7fcc277fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc28159560 0x7fcc280ab9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58440/0 (socket says 172.21.15.92:58440) 2023-10-06T10:41:53.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.520+0000 7fcc277fe700 1 -- 172.21.15.92:0/2939483979 learned_addr learned my addr 172.21.15.92:0/2939483979 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:53.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.521+0000 7fcc26ffd700 1 --2- 172.21.15.92:0/2939483979 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc280b0760 0x7fcc28169b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:53.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.521+0000 7fcc277fe700 1 -- 172.21.15.92:0/2939483979 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc280b49a0 msgr2=0x7fcc280ab490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:53.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.521+0000 7fcc1ffff700 1 --2- 172.21.15.92:0/2939483979 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc280b49a0 0x7fcc280ab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:53.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.521+0000 7fcc277fe700 1 --2- 172.21.15.92:0/2939483979 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc280b49a0 0x7fcc280ab490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:53.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.521+0000 7fcc277fe700 1 -- 172.21.15.92:0/2939483979 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc280b0760 msgr2=0x7fcc28169b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:53.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.521+0000 7fcc277fe700 1 --2- 172.21.15.92:0/2939483979 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc280b0760 0x7fcc28169b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:53.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.521+0000 7fcc277fe700 1 -- 172.21.15.92:0/2939483979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc20009a20 con 0x7fcc28159560 2023-10-06T10:41:53.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.521+0000 7fcc26ffd700 1 --2- 172.21.15.92:0/2939483979 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc280b0760 0x7fcc28169b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:53.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.521+0000 7fcc1ffff700 1 --2- 172.21.15.92:0/2939483979 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc280b49a0 0x7fcc280ab490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:53.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.521+0000 7fcc277fe700 1 --2- 172.21.15.92:0/2939483979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc28159560 0x7fcc280ab9d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009e60 tx=0x7fcc20017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:53.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.521+0000 7fcc1effd700 1 -- 172.21.15.92:0/2939483979 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc20012d50 con 0x7fcc28159560 2023-10-06T10:41:53.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.521+0000 7fcc1effd700 1 -- 172.21.15.92:0/2939483979 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2001e450 con 0x7fcc28159560 2023-10-06T10:41:53.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.521+0000 7fcc2d413700 1 -- 172.21.15.92:0/2939483979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc2816cf30 con 0x7fcc28159560 2023-10-06T10:41:53.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.521+0000 7fcc1effd700 1 -- 172.21.15.92:0/2939483979 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2001e810 con 0x7fcc28159560 2023-10-06T10:41:53.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.521+0000 7fcc2d413700 1 -- 172.21.15.92:0/2939483979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc280ac430 con 0x7fcc28159560 2023-10-06T10:41:53.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.522+0000 7fcc2d413700 1 -- 172.21.15.92:0/2939483979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc28064a90 con 0x7fcc28159560 2023-10-06T10:41:53.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.522+0000 7fcc1effd700 1 -- 172.21.15.92:0/2939483979 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcc2001ec00 con 0x7fcc28159560 2023-10-06T10:41:53.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.522+0000 7fcc1effd700 1 --2- 172.21.15.92:0/2939483979 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcc08066f30 0x7fcc080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:53.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.523+0000 7fcc1effd700 1 -- 172.21.15.92:0/2939483979 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fcc20086d90 con 0x7fcc28159560 2023-10-06T10:41:53.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.523+0000 7fcc26ffd700 1 --2- 172.21.15.92:0/2939483979 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcc08066f30 0x7fcc080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:53.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.523+0000 7fcc26ffd700 1 --2- 172.21.15.92:0/2939483979 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcc08066f30 0x7fcc080693e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcc18000c00 tx=0x7fcc18009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:53.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.527+0000 7fcc1effd700 1 -- 172.21.15.92:0/2939483979 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc20053d60 con 0x7fcc28159560 2023-10-06T10:41:53.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.751+0000 7fcc2d413700 1 -- 172.21.15.92:0/2939483979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc280acea0 con 0x7fcc28159560 2023-10-06T10:41:53.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.752+0000 7fcc1effd700 1 -- 172.21.15.92:0/2939483979 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fcc20016090 con 0x7fcc28159560 2023-10-06T10:41:53.754 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:53.754 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":50,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/1409495728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":1409495728},{"type":"v1","addr":"172.21.15.124:6834","nonce":1409495728}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6019,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2297612118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2297612118},{"type":"v1","addr":"172.21.15.124:6835","nonce":2297612118}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/171578176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":171578176},{"type":"v1","addr":"172.21.15.92:6836","nonce":171578176}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:41:47.898774+0000","modified":"2023-10-06T10:41:51.550457+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/818031030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":818031030},{"type":"v1","addr":"172.21.15.92:6837","nonce":818031030}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T10:41:53.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.754+0000 7fcc2d413700 1 -- 172.21.15.92:0/2939483979 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcc08066f30 msgr2=0x7fcc080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:53.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.754+0000 7fcc2d413700 1 --2- 172.21.15.92:0/2939483979 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcc08066f30 0x7fcc080693e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcc18000c00 tx=0x7fcc18009040 comp rx=0 tx=0).stop 2023-10-06T10:41:53.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.754+0000 7fcc2d413700 1 -- 172.21.15.92:0/2939483979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc28159560 msgr2=0x7fcc280ab9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:53.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.754+0000 7fcc2d413700 1 --2- 172.21.15.92:0/2939483979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc28159560 0x7fcc280ab9d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009e60 tx=0x7fcc20017000 comp rx=0 tx=0).stop 2023-10-06T10:41:53.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.754+0000 7fcc2d413700 1 -- 172.21.15.92:0/2939483979 shutdown_connections 2023-10-06T10:41:53.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.754+0000 7fcc2d413700 1 --2- 172.21.15.92:0/2939483979 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcc08066f30 0x7fcc080693e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:53.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.754+0000 7fcc2d413700 1 --2- 172.21.15.92:0/2939483979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc28159560 0x7fcc280ab9d0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:53.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.754+0000 7fcc2d413700 1 --2- 172.21.15.92:0/2939483979 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc280b0760 0x7fcc28169b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:53.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.754+0000 7fcc2d413700 1 --2- 172.21.15.92:0/2939483979 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc280b49a0 0x7fcc280ab490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:53.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.754+0000 7fcc2d413700 1 -- 172.21.15.92:0/2939483979 >> 172.21.15.92:0/2939483979 conn(0x7fcc280997f0 msgr2=0x7fcc280695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:53.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.754+0000 7fcc2d413700 1 -- 172.21.15.92:0/2939483979 shutdown_connections 2023-10-06T10:41:53.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:53.754+0000 7fcc2d413700 1 -- 172.21.15.92:0/2939483979 wait complete. 2023-10-06T10:41:53.758 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 50 2023-10-06T10:41:54.171 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T10:41:47.898774+0000', 'modified': '2023-10-06T10:41:51.550457+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6486}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6486': {'gid': 6486, 'name': 'c', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6837/818031030', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 818031030}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 818031030}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T10:41:54.172 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T10:41:54.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T10:41:54.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.287+0000 7fc163ecf700 1 Processor -- start 2023-10-06T10:41:54.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.287+0000 7fc163ecf700 1 -- start start 2023-10-06T10:41:54.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.288+0000 7fc163ecf700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc15c0b0760 0x7fc15c0ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.288+0000 7fc163ecf700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc15c0b0c60 con 0x7fc15c159560 2023-10-06T10:41:54.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.288+0000 7fc163ecf700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc15c0b0f40 con 0x7fc15c0b49a0 2023-10-06T10:41:54.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.288+0000 7fc163ecf700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c0ab350 con 0x7fc15c0b0760 2023-10-06T10:41:54.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.288+0000 7fc161c6b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc15c159560 0x7fc15c0aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54768/0 (socket says 172.21.15.92:54768) 2023-10-06T10:41:54.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.288+0000 7fc161c6b700 1 -- 172.21.15.92:0/96143817 learned_addr learned my addr 172.21.15.92:0/96143817 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:54.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc16246c700 1 --2- 172.21.15.92:0/96143817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc15c0b0760 0x7fc15c0ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc160c69700 1 -- 172.21.15.92:0/96143817 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2279201436 0 0) 0x7fc15c0b0c60 con 0x7fc15c159560 2023-10-06T10:41:54.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc160c69700 1 -- 172.21.15.92:0/96143817 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc148003570 con 0x7fc15c159560 2023-10-06T10:41:54.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc160c69700 1 -- 172.21.15.92:0/96143817 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 332572008 0 0) 0x7fc148003570 con 0x7fc15c159560 2023-10-06T10:41:54.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc160c69700 1 -- 172.21.15.92:0/96143817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc15c0b0760 msgr2=0x7fc15c0ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:54.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc160c69700 1 --2- 172.21.15.92:0/96143817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc15c0b0760 0x7fc15c0ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc160c69700 1 -- 172.21.15.92:0/96143817 >> v1:172.21.15.124:6789/0 conn(0x7fc15c0b49a0 legacy=0x7fc15c0aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:54.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc160c69700 1 -- 172.21.15.92:0/96143817 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c0b1220 con 0x7fc15c159560 2023-10-06T10:41:54.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc160c69700 1 -- 172.21.15.92:0/96143817 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fc158001970 con 0x7fc15c159560 2023-10-06T10:41:54.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc160c69700 1 -- 172.21.15.92:0/96143817 >> v1:172.21.15.92:6789/0 conn(0x7fc15c159560 legacy=0x7fc15c0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:54.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc160c69700 1 --2- 172.21.15.92:0/96143817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc15c0b0760 0x7fc148003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc160c69700 1 --2- 172.21.15.92:0/96143817 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc148004010 0x7fc148004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc160c69700 1 --2- 172.21.15.92:0/96143817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc148004d50 0x7fc1480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc160c69700 1 -- 172.21.15.92:0/96143817 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c0ab350 con 0x7fc148004d50 2023-10-06T10:41:54.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc160c69700 1 -- 172.21.15.92:0/96143817 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14800bbf0 con 0x7fc148004010 2023-10-06T10:41:54.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc160c69700 1 -- 172.21.15.92:0/96143817 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14800bdb0 con 0x7fc15c0b0760 2023-10-06T10:41:54.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc16246c700 1 --2- 172.21.15.92:0/96143817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc148004d50 0x7fc1480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc16146a700 1 --2- 172.21.15.92:0/96143817 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc148004010 0x7fc148004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc161c6b700 1 --2- 172.21.15.92:0/96143817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc15c0b0760 0x7fc148003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc16246c700 1 -- 172.21.15.92:0/96143817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc15c0b0760 msgr2=0x7fc148003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:54.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc16246c700 1 --2- 172.21.15.92:0/96143817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc15c0b0760 0x7fc148003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc16246c700 1 -- 172.21.15.92:0/96143817 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc148004010 msgr2=0x7fc148004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:54.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc16246c700 1 --2- 172.21.15.92:0/96143817 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc148004010 0x7fc148004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.289+0000 7fc16246c700 1 -- 172.21.15.92:0/96143817 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc148003570 con 0x7fc148004d50 2023-10-06T10:41:54.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.290+0000 7fc16246c700 1 --2- 172.21.15.92:0/96143817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc148004d50 0x7fc1480051c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc154006f00 tx=0x7fc1540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c074678932a45e80 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:54.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.290+0000 7fc160c69700 1 -- 172.21.15.92:0/96143817 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15400e6e0 con 0x7fc148004d50 2023-10-06T10:41:54.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.290+0000 7fc160c69700 1 -- 172.21.15.92:0/96143817 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc15400e840 con 0x7fc148004d50 2023-10-06T10:41:54.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.290+0000 7fc160c69700 1 -- 172.21.15.92:0/96143817 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15400ec00 con 0x7fc148004d50 2023-10-06T10:41:54.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.290+0000 7fc163ecf700 1 -- 172.21.15.92:0/96143817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc148004d50 msgr2=0x7fc1480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:54.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.290+0000 7fc163ecf700 1 --2- 172.21.15.92:0/96143817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc148004d50 0x7fc1480051c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc154006f00 tx=0x7fc1540091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.290+0000 7fc16246c700 1 -- 172.21.15.92:0/96143817 reap_dead start 2023-10-06T10:41:54.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.290+0000 7fc163ecf700 1 -- 172.21.15.92:0/96143817 shutdown_connections 2023-10-06T10:41:54.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.290+0000 7fc163ecf700 1 -- 172.21.15.92:0/96143817 >> 172.21.15.92:0/96143817 conn(0x7fc15c0997f0 msgr2=0x7fc15c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:54.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.290+0000 7fc163ecf700 1 -- 172.21.15.92:0/96143817 shutdown_connections 2023-10-06T10:41:54.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.290+0000 7fc163ecf700 1 -- 172.21.15.92:0/96143817 wait complete. 2023-10-06T10:41:54.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.291+0000 7fc163ecf700 1 Processor -- start 2023-10-06T10:41:54.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.291+0000 7fc163ecf700 1 -- start start 2023-10-06T10:41:54.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.292+0000 7fc163ecf700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc15c14b250 0x7fc15c0ada60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.292+0000 7fc163ecf700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc15c14b620 0x7fc15c0c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.292+0000 7fc163ecf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc15c09d030 0x7fc15c09d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.292+0000 7fc163ecf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c09da10 con 0x7fc15c09d030 2023-10-06T10:41:54.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.292+0000 7fc163ecf700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c169730 con 0x7fc15c14b250 2023-10-06T10:41:54.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.292+0000 7fc163ecf700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c0aed10 con 0x7fc15c14b620 2023-10-06T10:41:54.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.292+0000 7fc16246c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc15c09d030 0x7fc15c09d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.292+0000 7fc16246c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc15c09d030 0x7fc15c09d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58460/0 (socket says 172.21.15.92:58460) 2023-10-06T10:41:54.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.292+0000 7fc16246c700 1 -- 172.21.15.92:0/2765343432 learned_addr learned my addr 172.21.15.92:0/2765343432 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:54.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.292+0000 7fc161c6b700 1 --2- 172.21.15.92:0/2765343432 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc15c14b250 0x7fc15c0ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.292+0000 7fc16146a700 1 --2- 172.21.15.92:0/2765343432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc15c14b620 0x7fc15c0c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.292+0000 7fc16246c700 1 -- 172.21.15.92:0/2765343432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc15c14b620 msgr2=0x7fc15c0c5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:54.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.292+0000 7fc16246c700 1 --2- 172.21.15.92:0/2765343432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc15c14b620 0x7fc15c0c5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.292+0000 7fc16246c700 1 -- 172.21.15.92:0/2765343432 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc15c14b250 msgr2=0x7fc15c0ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:54.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.292+0000 7fc16246c700 1 --2- 172.21.15.92:0/2765343432 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc15c14b250 0x7fc15c0ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.292+0000 7fc16246c700 1 -- 172.21.15.92:0/2765343432 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c0c5770 con 0x7fc15c09d030 2023-10-06T10:41:54.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.293+0000 7fc16246c700 1 --2- 172.21.15.92:0/2765343432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc15c09d030 0x7fc15c09d4d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc1480057d0 tx=0x7fc15401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:54.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.293+0000 7fc151ffb700 1 -- 172.21.15.92:0/2765343432 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc154022070 con 0x7fc15c09d030 2023-10-06T10:41:54.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.293+0000 7fc151ffb700 1 -- 172.21.15.92:0/2765343432 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc154013e60 con 0x7fc15c09d030 2023-10-06T10:41:54.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.293+0000 7fc163ecf700 1 -- 172.21.15.92:0/2765343432 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc15c0ae100 con 0x7fc15c09d030 2023-10-06T10:41:54.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.293+0000 7fc151ffb700 1 -- 172.21.15.92:0/2765343432 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15400e6e0 con 0x7fc15c09d030 2023-10-06T10:41:54.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.293+0000 7fc163ecf700 1 -- 172.21.15.92:0/2765343432 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15c0ae6c0 con 0x7fc15c09d030 2023-10-06T10:41:54.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.294+0000 7fc151ffb700 1 -- 172.21.15.92:0/2765343432 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc15401c970 con 0x7fc15c09d030 2023-10-06T10:41:54.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.295+0000 7fc151ffb700 1 --2- 172.21.15.92:0/2765343432 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc140066f30 0x7fc1400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.295+0000 7fc151ffb700 1 -- 172.21.15.92:0/2765343432 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc154000e40 con 0x7fc15c09d030 2023-10-06T10:41:54.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.295+0000 7fc161c6b700 1 --2- 172.21.15.92:0/2765343432 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc140066f30 0x7fc1400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.295+0000 7fc161c6b700 1 --2- 172.21.15.92:0/2765343432 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc140066f30 0x7fc1400693e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc1580031f0 tx=0x7fc158009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:54.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.296+0000 7fc16d1c7b80 1 Processor -- start 2023-10-06T10:41:54.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.297+0000 7fc16d1c7b80 1 -- start start 2023-10-06T10:41:54.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.297+0000 7fc16d1c7b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566889a3a90 0x5566889a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.297+0000 7fc16d1c7b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55668895f9a0 0x55668895c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.297+0000 7fc16d1c7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668895dbd0 0x55668895cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.297+0000 7fc16d1c7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566888dcc80 con 0x55668895dbd0 2023-10-06T10:41:54.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.297+0000 7fc16d1c7b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566888b3a70 con 0x5566889a3a90 2023-10-06T10:41:54.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.297+0000 7fc16d1c7b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566888b1270 con 0x55668895f9a0 2023-10-06T10:41:54.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.297+0000 7fc16246c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668895dbd0 0x55668895cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.297+0000 7fc161c6b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566889a3a90 0x5566889a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.297+0000 7fc16246c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668895dbd0 0x55668895cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58472/0 (socket says 172.21.15.92:58472) 2023-10-06T10:41:54.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.297+0000 7fc16246c700 1 -- 172.21.15.92:0/2967175722 learned_addr learned my addr 172.21.15.92:0/2967175722 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:54.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.297+0000 7fc16146a700 1 --2- 172.21.15.92:0/2967175722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55668895f9a0 0x55668895c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.298+0000 7fc16246c700 1 -- 172.21.15.92:0/2967175722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55668895f9a0 msgr2=0x55668895c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:54.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.298+0000 7fc16246c700 1 --2- 172.21.15.92:0/2967175722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55668895f9a0 0x55668895c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.298+0000 7fc16246c700 1 -- 172.21.15.92:0/2967175722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566889a3a90 msgr2=0x5566889a3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:54.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.298+0000 7fc16246c700 1 --2- 172.21.15.92:0/2967175722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566889a3a90 0x5566889a3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.298+0000 7fc16246c700 1 -- 172.21.15.92:0/2967175722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556688747270 con 0x55668895dbd0 2023-10-06T10:41:54.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.298+0000 7fc16246c700 1 --2- 172.21.15.92:0/2967175722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668895dbd0 0x55668895cac0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fc1540844f0 tx=0x7fc154084420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800005c602f84c13 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:54.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.298+0000 7fc146ffd700 1 -- 172.21.15.92:0/2967175722 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1540860b0 con 0x55668895dbd0 2023-10-06T10:41:54.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.298+0000 7fc146ffd700 1 -- 172.21.15.92:0/2967175722 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc154057690 con 0x55668895dbd0 2023-10-06T10:41:54.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.298+0000 7fc146ffd700 1 -- 172.21.15.92:0/2967175722 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc154056f90 con 0x55668895dbd0 2023-10-06T10:41:54.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.298+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/2967175722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668895dbd0 msgr2=0x55668895cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:54.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.298+0000 7fc16d1c7b80 1 --2- 172.21.15.92:0/2967175722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668895dbd0 0x55668895cac0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fc1540844f0 tx=0x7fc154084420 comp rx=0 tx=0).stop 2023-10-06T10:41:54.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.298+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/2967175722 shutdown_connections 2023-10-06T10:41:54.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.298+0000 7fc16d1c7b80 1 --2- 172.21.15.92:0/2967175722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668895dbd0 0x55668895cac0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.298+0000 7fc16d1c7b80 1 --2- 172.21.15.92:0/2967175722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566889a3a90 0x5566889a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.298+0000 7fc16d1c7b80 1 --2- 172.21.15.92:0/2967175722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55668895f9a0 0x55668895c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.298+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/2967175722 >> 172.21.15.92:0/2967175722 conn(0x55668895e070 msgr2=0x55668895e440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:54.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.299+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/2967175722 shutdown_connections 2023-10-06T10:41:54.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.299+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/2967175722 wait complete. 2023-10-06T10:41:54.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.299+0000 7fc16d1c7b80 1 -- start start 2023-10-06T10:41:54.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.299+0000 7fc16d1c7b80 1 Processor -- start 2023-10-06T10:41:54.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.300+0000 7fc16d1c7b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55668895dbd0 0x556688989080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.300+0000 7fc16d1c7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668895f9a0 0x556688959550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.300+0000 7fc16d1c7b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566889a3a90 0x556688959a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.300+0000 7fc16d1c7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566888ad060 con 0x55668895f9a0 2023-10-06T10:41:54.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.300+0000 7fc16d1c7b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x556688858890 con 0x5566889a3a90 2023-10-06T10:41:54.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.300+0000 7fc16d1c7b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x556688873ea0 con 0x55668895dbd0 2023-10-06T10:41:54.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.300+0000 7fc16d1c7b80 10 client.0 _ll_register_callbacks cb 0x556688749f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:54.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.300+0000 7fc16146a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668895f9a0 0x556688959550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.300+0000 7fc16146a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668895f9a0 0x556688959550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58488/0 (socket says 172.21.15.92:58488) 2023-10-06T10:41:54.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.300+0000 7fc16146a700 1 -- 172.21.15.92:0/208328481 learned_addr learned my addr 172.21.15.92:0/208328481 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:54.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.300+0000 7fc161c6b700 1 --2- 172.21.15.92:0/208328481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55668895dbd0 0x556688989080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.300+0000 7fc16246c700 1 --2- 172.21.15.92:0/208328481 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566889a3a90 0x556688959a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.300+0000 7fc16146a700 1 -- 172.21.15.92:0/208328481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55668895dbd0 msgr2=0x556688989080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:54.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.300+0000 7fc16146a700 1 --2- 172.21.15.92:0/208328481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55668895dbd0 0x556688989080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.300+0000 7fc16146a700 1 -- 172.21.15.92:0/208328481 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566889a3a90 msgr2=0x556688959a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:54.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.300+0000 7fc16146a700 1 --2- 172.21.15.92:0/208328481 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566889a3a90 0x556688959a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.300+0000 7fc16146a700 1 -- 172.21.15.92:0/208328481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15c0ae6c0 con 0x55668895f9a0 2023-10-06T10:41:54.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.300+0000 7fc16246c700 1 --2- 172.21.15.92:0/208328481 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566889a3a90 0x556688959a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:54.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.300+0000 7fc161c6b700 1 --2- 172.21.15.92:0/208328481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55668895dbd0 0x556688989080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:54.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.301+0000 7fc16146a700 1 --2- 172.21.15.92:0/208328481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668895f9a0 0x556688959550 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc15c099f20 tx=0x7fc14c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27c98d19a3ac974 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:54.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.301+0000 7fc13bfff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:54.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.301+0000 7fc13bfff700 1 -- 172.21.15.92:0/208328481 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c015e70 con 0x55668895f9a0 2023-10-06T10:41:54.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.301+0000 7fc13bfff700 1 -- 172.21.15.92:0/208328481 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc14c013a00 con 0x55668895f9a0 2023-10-06T10:41:54.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.301+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/208328481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556688951690 con 0x55668895f9a0 2023-10-06T10:41:54.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.301+0000 7fc13bfff700 1 -- 172.21.15.92:0/208328481 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc14c013bc0 con 0x55668895f9a0 2023-10-06T10:41:54.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.301+0000 7fc16d1c7b80 10 client.6645 fetch_fsmap learned FSMap version 50 2023-10-06T10:41:54.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.301+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/208328481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x5566886e9fe0 con 0x55668895f9a0 2023-10-06T10:41:54.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.301+0000 7fc13bfff700 1 -- 172.21.15.92:0/208328481 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556688951690 con 0x55668895f9a0 2023-10-06T10:41:54.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.301+0000 7fc13bfff700 1 -- 172.21.15.92:0/208328481 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 50) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc14c012050 con 0x55668895f9a0 2023-10-06T10:41:54.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.301+0000 7fc16d1c7b80 10 client.6645 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T10:41:54.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.302+0000 7fc16d1c7b80 10 client.6645 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:41:54.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.302+0000 7fc16d1c7b80 20 client.6645 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:54.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.302+0000 7fc16d1c7b80 1 --2- 172.21.15.92:0/208328481 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x55668898a490 0x55668899ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.302+0000 7fc16d1c7b80 4 client.6645 mds_command: new command op to 6486 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:41:54.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.302+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/208328481 --> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55668892e5c0 con 0x55668898a490 2023-10-06T10:41:54.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.302+0000 7fc161c6b700 1 --2- 172.21.15.92:0/208328481 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x55668898a490 0x55668899ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.302+0000 7fc13bfff700 10 client.6645 ms_handle_connect on v2:172.21.15.92:6835/818031030 2023-10-06T10:41:54.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.302+0000 7fc161c6b700 1 --2- 172.21.15.92:0/208328481 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x55668898a490 0x55668899ded0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7e4e7878aa8b19c server_cookie=233031f3d57eb247 in_seq=0 out_seq=0 2023-10-06T10:41:54.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.303+0000 7fc13bfff700 1 -- 172.21.15.92:0/208328481 <== mds.0 v2:172.21.15.92:6835/818031030 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55668892e5c0 con 0x55668898a490 2023-10-06T10:41:54.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.303+0000 7fc13bfff700 10 client.6645 handle_command_reply: tid=0 2023-10-06T10:41:54.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.303+0000 7fc16d1c7b80 1 client.6645 shutdown 2023-10-06T10:41:54.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.303+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/208328481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668895f9a0 msgr2=0x556688959550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:54.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.303+0000 7fc16d1c7b80 1 --2- 172.21.15.92:0/208328481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668895f9a0 0x556688959550 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc15c099f20 tx=0x7fc14c007d30 comp rx=0 tx=0).stop 2023-10-06T10:41:54.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.303+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/208328481 shutdown_connections 2023-10-06T10:41:54.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.303+0000 7fc16d1c7b80 1 --2- 172.21.15.92:0/208328481 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566889a3a90 0x556688959a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.303+0000 7fc16d1c7b80 1 --2- 172.21.15.92:0/208328481 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x55668898a490 0x55668899ded0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.303+0000 7fc16d1c7b80 1 --2- 172.21.15.92:0/208328481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55668895dbd0 0x556688989080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.303+0000 7fc13bfff700 0 client.6645 ms_handle_reset on v2:172.21.15.92:6835/818031030 2023-10-06T10:41:54.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.303+0000 7fc16d1c7b80 1 --2- 172.21.15.92:0/208328481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668895f9a0 0x556688959550 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.303+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/208328481 >> 172.21.15.92:0/208328481 conn(0x55668895e070 msgr2=0x5566889a2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:54.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.303+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/208328481 shutdown_connections 2023-10-06T10:41:54.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.303+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/208328481 wait complete. 2023-10-06T10:41:54.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.303+0000 7fc16d1c7b80 20 client.6645 trim_cache size 0 max 16384 2023-10-06T10:41:54.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.320+0000 7fc16d1c7b80 1 Processor -- start 2023-10-06T10:41:54.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.320+0000 7fc16d1c7b80 1 -- start start 2023-10-06T10:41:54.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.321+0000 7fc16d1c7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668895dbd0 0x5566889578b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.321+0000 7fc16d1c7b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55668895f9a0 0x5566889a2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.321+0000 7fc16d1c7b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566889a3a90 0x5566889a27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.321+0000 7fc16d1c7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55668872fe30 con 0x55668895dbd0 2023-10-06T10:41:54.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.321+0000 7fc16d1c7b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566889a0b60 con 0x5566889a3a90 2023-10-06T10:41:54.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.321+0000 7fc16d1c7b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566889877b0 con 0x55668895f9a0 2023-10-06T10:41:54.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.321+0000 7fc161c6b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668895dbd0 0x5566889578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.321+0000 7fc161c6b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668895dbd0 0x5566889578b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58492/0 (socket says 172.21.15.92:58492) 2023-10-06T10:41:54.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.321+0000 7fc161c6b700 1 -- 172.21.15.92:0/3215204393 learned_addr learned my addr 172.21.15.92:0/3215204393 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:54.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.321+0000 7fc16146a700 1 --2- 172.21.15.92:0/3215204393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55668895f9a0 0x5566889a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.321+0000 7fc161c6b700 1 -- 172.21.15.92:0/3215204393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55668895f9a0 msgr2=0x5566889a2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:54.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.321+0000 7fc161c6b700 1 --2- 172.21.15.92:0/3215204393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55668895f9a0 0x5566889a2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.321+0000 7fc161c6b700 1 -- 172.21.15.92:0/3215204393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566889a3a90 msgr2=0x5566889a27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:54.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.321+0000 7fc161c6b700 1 --2- 172.21.15.92:0/3215204393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566889a3a90 0x5566889a27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.321+0000 7fc161c6b700 1 -- 172.21.15.92:0/3215204393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566889a0240 con 0x55668895dbd0 2023-10-06T10:41:54.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.321+0000 7fc161c6b700 1 --2- 172.21.15.92:0/3215204393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668895dbd0 0x5566889578b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc158007fd0 tx=0x7fc158018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92932d24aea71bf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:54.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.321+0000 7fc1477fe700 1 -- 172.21.15.92:0/3215204393 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158006b80 con 0x55668895dbd0 2023-10-06T10:41:54.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.321+0000 7fc1477fe700 1 -- 172.21.15.92:0/3215204393 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc158029040 con 0x55668895dbd0 2023-10-06T10:41:54.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.321+0000 7fc1477fe700 1 -- 172.21.15.92:0/3215204393 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158007050 con 0x55668895dbd0 2023-10-06T10:41:54.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.322+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/3215204393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668895dbd0 msgr2=0x5566889578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:54.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.322+0000 7fc16d1c7b80 1 --2- 172.21.15.92:0/3215204393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668895dbd0 0x5566889578b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc158007fd0 tx=0x7fc158018000 comp rx=0 tx=0).stop 2023-10-06T10:41:54.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.322+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/3215204393 shutdown_connections 2023-10-06T10:41:54.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.322+0000 7fc16d1c7b80 1 --2- 172.21.15.92:0/3215204393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566889a3a90 0x5566889a27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.322+0000 7fc16d1c7b80 1 --2- 172.21.15.92:0/3215204393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668895dbd0 0x5566889578b0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.322+0000 7fc16d1c7b80 1 --2- 172.21.15.92:0/3215204393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55668895f9a0 0x5566889a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.322+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/3215204393 >> 172.21.15.92:0/3215204393 conn(0x55668895e070 msgr2=0x55668895e440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:54.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.322+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/3215204393 shutdown_connections 2023-10-06T10:41:54.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.322+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/3215204393 wait complete. 2023-10-06T10:41:54.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.322+0000 7fc16d1c7b80 1 -- start start 2023-10-06T10:41:54.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.323+0000 7fc16d1c7b80 1 Processor -- start 2023-10-06T10:41:54.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.323+0000 7fc16d1c7b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55668895dbd0 0x5566889825f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.323+0000 7fc16d1c7b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55668895f9a0 0x5566889bc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.323+0000 7fc16d1c7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566889a3a90 0x5566889a2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.323+0000 7fc16d1c7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556688989420 con 0x5566889a3a90 2023-10-06T10:41:54.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.323+0000 7fc16d1c7b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x556688982e80 con 0x55668895f9a0 2023-10-06T10:41:54.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.323+0000 7fc16d1c7b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566889891d0 con 0x55668895dbd0 2023-10-06T10:41:54.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.324+0000 7fc16d1c7b80 10 client.0 _ll_register_callbacks cb 0x556688749f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:54.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.324+0000 7fc16246c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566889a3a90 0x5566889a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.324+0000 7fc16246c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566889a3a90 0x5566889a2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58504/0 (socket says 172.21.15.92:58504) 2023-10-06T10:41:54.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.324+0000 7fc16246c700 1 -- 172.21.15.92:0/2686091148 learned_addr learned my addr 172.21.15.92:0/2686091148 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:54.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.324+0000 7fc16146a700 1 --2- 172.21.15.92:0/2686091148 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55668895f9a0 0x5566889bc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.324+0000 7fc161c6b700 1 --2- 172.21.15.92:0/2686091148 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55668895dbd0 0x5566889825f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.324+0000 7fc16246c700 1 -- 172.21.15.92:0/2686091148 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55668895dbd0 msgr2=0x5566889825f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:54.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.324+0000 7fc16246c700 1 --2- 172.21.15.92:0/2686091148 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55668895dbd0 0x5566889825f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.324+0000 7fc16246c700 1 -- 172.21.15.92:0/2686091148 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55668895f9a0 msgr2=0x5566889bc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:54.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.324+0000 7fc16246c700 1 --2- 172.21.15.92:0/2686091148 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55668895f9a0 0x5566889bc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.324+0000 7fc16246c700 1 -- 172.21.15.92:0/2686091148 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15c0b1220 con 0x5566889a3a90 2023-10-06T10:41:54.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.324+0000 7fc16246c700 1 --2- 172.21.15.92:0/2686091148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566889a3a90 0x5566889a2280 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc15400edb0 tx=0x7fc15406e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb1f87207978a6f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:54.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.324+0000 7fc13bfff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:54.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.324+0000 7fc13bfff700 1 -- 172.21.15.92:0/2686091148 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15405a1e0 con 0x5566889a3a90 2023-10-06T10:41:54.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.324+0000 7fc13bfff700 1 -- 172.21.15.92:0/2686091148 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc15405df30 con 0x5566889a3a90 2023-10-06T10:41:54.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.324+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/2686091148 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc154057690 con 0x5566889a3a90 2023-10-06T10:41:54.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.324+0000 7fc13bfff700 1 -- 172.21.15.92:0/2686091148 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc154056f90 con 0x5566889a3a90 2023-10-06T10:41:54.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.324+0000 7fc16d1c7b80 10 client.6651 fetch_fsmap learned FSMap version 50 2023-10-06T10:41:54.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.324+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/2686091148 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x55668887b780 con 0x5566889a3a90 2023-10-06T10:41:54.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.324+0000 7fc13bfff700 1 -- 172.21.15.92:0/2686091148 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc154057690 con 0x5566889a3a90 2023-10-06T10:41:54.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.324+0000 7fc13bfff700 1 -- 172.21.15.92:0/2686091148 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 50) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc15401cc20 con 0x5566889a3a90 2023-10-06T10:41:54.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc16d1c7b80 10 client.6651 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T10:41:54.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc16d1c7b80 10 client.6651 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:41:54.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc16d1c7b80 20 client.6651 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:54.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc16d1c7b80 1 --2- 172.21.15.92:0/2686091148 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x556688a07d60 0x5566889a27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc16d1c7b80 4 client.6651 mds_command: new command op to 6486 tid=0[{"prefix": "status"}] 2023-10-06T10:41:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/2686091148 --> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55668884fe00 con 0x556688a07d60 2023-10-06T10:41:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc161c6b700 1 --2- 172.21.15.92:0/2686091148 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x556688a07d60 0x5566889a27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc13bfff700 10 client.6651 ms_handle_connect on v2:172.21.15.92:6835/818031030 2023-10-06T10:41:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc161c6b700 1 --2- 172.21.15.92:0/2686091148 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x556688a07d60 0x5566889a27c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2e65ec0d9178f9c server_cookie=b5c8c21e47607ff9 in_seq=0 out_seq=0 2023-10-06T10:41:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc13bfff700 1 -- 172.21.15.92:0/2686091148 <== mds.0 v2:172.21.15.92:6835/818031030 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55668884fe00 con 0x556688a07d60 2023-10-06T10:41:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc13bfff700 10 client.6651 handle_command_reply: tid=0 2023-10-06T10:41:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc16d1c7b80 1 client.6651 shutdown 2023-10-06T10:41:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/2686091148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566889a3a90 msgr2=0x5566889a2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc16d1c7b80 1 --2- 172.21.15.92:0/2686091148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566889a3a90 0x5566889a2280 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc15400edb0 tx=0x7fc15406e000 comp rx=0 tx=0).stop 2023-10-06T10:41:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/2686091148 shutdown_connections 2023-10-06T10:41:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc16d1c7b80 1 --2- 172.21.15.92:0/2686091148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566889a3a90 0x5566889a2280 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc16d1c7b80 1 --2- 172.21.15.92:0/2686091148 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x556688a07d60 0x5566889a27c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc16d1c7b80 1 --2- 172.21.15.92:0/2686091148 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55668895dbd0 0x5566889825f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc13bfff700 0 client.6651 ms_handle_reset on v2:172.21.15.92:6835/818031030 2023-10-06T10:41:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc16d1c7b80 1 --2- 172.21.15.92:0/2686091148 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55668895f9a0 0x5566889bc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/2686091148 >> 172.21.15.92:0/2686091148 conn(0x55668895e070 msgr2=0x55668895d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/2686091148 shutdown_connections 2023-10-06T10:41:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.325+0000 7fc16d1c7b80 1 -- 172.21.15.92:0/2686091148 wait complete. 2023-10-06T10:41:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.326+0000 7fc16d1c7b80 20 client.6651 trim_cache size 0 max 16384 2023-10-06T10:41:54.334 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:41:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:41:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T10:41:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 6486, 2023-10-06T10:41:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:41:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:41:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:41:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 5.7945472679999996, 2023-10-06T10:41:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 50, 2023-10-06T10:41:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 64, 2023-10-06T10:41:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 64, 2023-10-06T10:41:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 11.916641838 2023-10-06T10:41:54.335 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:41:54.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.326+0000 7fc163ecf700 1 -- 172.21.15.92:0/2765343432 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc140066f30 msgr2=0x7fc1400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:54.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.326+0000 7fc163ecf700 1 --2- 172.21.15.92:0/2765343432 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc140066f30 0x7fc1400693e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc1580031f0 tx=0x7fc158009040 comp rx=0 tx=0).stop 2023-10-06T10:41:54.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.326+0000 7fc163ecf700 1 -- 172.21.15.92:0/2765343432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc15c09d030 msgr2=0x7fc15c09d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:54.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.326+0000 7fc163ecf700 1 --2- 172.21.15.92:0/2765343432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc15c09d030 0x7fc15c09d4d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc1480057d0 tx=0x7fc15401a000 comp rx=0 tx=0).stop 2023-10-06T10:41:54.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.326+0000 7fc163ecf700 1 -- 172.21.15.92:0/2765343432 shutdown_connections 2023-10-06T10:41:54.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.326+0000 7fc163ecf700 1 --2- 172.21.15.92:0/2765343432 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc140066f30 0x7fc1400693e0 secure :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc1580031f0 tx=0x7fc158009040 comp rx=0 tx=0).stop 2023-10-06T10:41:54.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.326+0000 7fc163ecf700 1 --2- 172.21.15.92:0/2765343432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc15c09d030 0x7fc15c09d4d0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.326+0000 7fc163ecf700 1 --2- 172.21.15.92:0/2765343432 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc15c14b250 0x7fc15c0ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.326+0000 7fc163ecf700 1 --2- 172.21.15.92:0/2765343432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc15c14b620 0x7fc15c0c5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.326+0000 7fc163ecf700 1 -- 172.21.15.92:0/2765343432 >> 172.21.15.92:0/2765343432 conn(0x7fc15c0997f0 msgr2=0x7fc15c068840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:54.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.326+0000 7fc163ecf700 1 -- 172.21.15.92:0/2765343432 shutdown_connections 2023-10-06T10:41:54.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.326+0000 7fc163ecf700 1 -- 172.21.15.92:0/2765343432 wait complete. 2023-10-06T10:41:54.340 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T10:41:54.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.452+0000 7f5ba5062700 1 Processor -- start 2023-10-06T10:41:54.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.452+0000 7f5ba5062700 1 -- start start 2023-10-06T10:41:54.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.453+0000 7f5ba5062700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5ba0159440 0x7f5ba00b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.453+0000 7f5ba5062700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba00cb7a0 con 0x7f5ba00b35d0 2023-10-06T10:41:54.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.453+0000 7f5ba5062700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba00b0860 con 0x7f5ba00b2100 2023-10-06T10:41:54.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.453+0000 7f5ba5062700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba00cb150 con 0x7f5ba0159440 2023-10-06T10:41:54.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.453+0000 7f5b9ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5ba0159440 0x7f5ba00b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.453+0000 7f5b9ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5ba0159440 0x7f5ba00b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58518/0 (socket says 172.21.15.92:58518) 2023-10-06T10:41:54.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.453+0000 7f5b9ffff700 1 -- 172.21.15.92:0/3294489525 learned_addr learned my addr 172.21.15.92:0/3294489525 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:54.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.453+0000 7f5b9f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5ba00b35d0 0x7f5ba00cbc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54780/0 (socket says 172.21.15.92:54780) 2023-10-06T10:41:54.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.453+0000 7f5b9ffff700 1 -- 172.21.15.92:0/3294489525 >> v1:172.21.15.124:6789/0 conn(0x7f5ba00b2100 legacy=0x7f5ba00b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:54.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.453+0000 7f5b9ffff700 1 -- 172.21.15.92:0/3294489525 >> v1:172.21.15.92:6789/0 conn(0x7f5ba00b35d0 legacy=0x7f5ba00cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:54.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.454+0000 7f5b9ffff700 1 -- 172.21.15.92:0/3294489525 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba00cb430 con 0x7f5ba0159440 2023-10-06T10:41:54.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.453+0000 7f5b9effd700 1 -- 172.21.15.92:0/3294489525 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 691938699 0 0) 0x7f5ba00cb7a0 con 0x7f5ba00b35d0 2023-10-06T10:41:54.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.454+0000 7f5b9ffff700 1 --2- 172.21.15.92:0/3294489525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ba0159440 0x7f5ba00b3090 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f5b900099b0 tx=0x7f5b90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f36542d12ef1ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:54.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.454+0000 7f5b9effd700 1 -- 172.21.15.92:0/3294489525 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9000b860 con 0x7f5ba0159440 2023-10-06T10:41:54.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.454+0000 7f5b9effd700 1 -- 172.21.15.92:0/3294489525 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9000b9c0 con 0x7f5ba0159440 2023-10-06T10:41:54.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.454+0000 7f5b9effd700 1 -- 172.21.15.92:0/3294489525 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9000bd60 con 0x7f5ba0159440 2023-10-06T10:41:54.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.454+0000 7f5ba5062700 1 -- 172.21.15.92:0/3294489525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ba0159440 msgr2=0x7f5ba00b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:54.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.454+0000 7f5ba5062700 1 --2- 172.21.15.92:0/3294489525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ba0159440 0x7f5ba00b3090 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f5b900099b0 tx=0x7f5b90009250 comp rx=0 tx=0).stop 2023-10-06T10:41:54.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.454+0000 7f5ba5062700 1 -- 172.21.15.92:0/3294489525 shutdown_connections 2023-10-06T10:41:54.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.454+0000 7f5ba5062700 1 --2- 172.21.15.92:0/3294489525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ba0159440 0x7f5ba00b3090 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.454+0000 7f5ba5062700 1 -- 172.21.15.92:0/3294489525 >> 172.21.15.92:0/3294489525 conn(0x7f5ba006ece0 msgr2=0x7f5ba006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:54.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.454+0000 7f5ba5062700 1 -- 172.21.15.92:0/3294489525 shutdown_connections 2023-10-06T10:41:54.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.454+0000 7f5ba5062700 1 -- 172.21.15.92:0/3294489525 wait complete. 2023-10-06T10:41:54.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.455+0000 7f5ba5062700 1 Processor -- start 2023-10-06T10:41:54.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.456+0000 7f5ba5062700 1 -- start start 2023-10-06T10:41:54.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.456+0000 7f5ba5062700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ba00b2100 0x7f5ba00c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.456+0000 7f5ba5062700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ba00b35d0 0x7f5ba016d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.456+0000 7f5ba5062700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ba0159440 0x7f5ba0152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.456+0000 7f5ba5062700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba00c7110 con 0x7f5ba00b35d0 2023-10-06T10:41:54.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.456+0000 7f5ba5062700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba00c6740 con 0x7f5ba0159440 2023-10-06T10:41:54.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.456+0000 7f5ba5062700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba016da00 con 0x7f5ba00b2100 2023-10-06T10:41:54.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.456+0000 7f5b9f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ba00b35d0 0x7f5ba016d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.456+0000 7f5b9f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ba00b35d0 0x7f5ba016d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58522/0 (socket says 172.21.15.92:58522) 2023-10-06T10:41:54.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.456+0000 7f5b9f7fe700 1 -- 172.21.15.92:0/917646536 learned_addr learned my addr 172.21.15.92:0/917646536 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:54.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.456+0000 7f5ba4861700 1 --2- 172.21.15.92:0/917646536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ba0159440 0x7f5ba0152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.457+0000 7f5b9ffff700 1 --2- 172.21.15.92:0/917646536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ba00b2100 0x7f5ba00c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.457+0000 7f5b9f7fe700 1 -- 172.21.15.92:0/917646536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ba00b2100 msgr2=0x7f5ba00c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:54.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.457+0000 7f5b9f7fe700 1 --2- 172.21.15.92:0/917646536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ba00b2100 0x7f5ba00c6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.457+0000 7f5b9f7fe700 1 -- 172.21.15.92:0/917646536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ba0159440 msgr2=0x7f5ba0152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:54.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.457+0000 7f5b9f7fe700 1 --2- 172.21.15.92:0/917646536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ba0159440 0x7f5ba0152900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.457+0000 7f5b9f7fe700 1 -- 172.21.15.92:0/917646536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b900094f0 con 0x7f5ba00b35d0 2023-10-06T10:41:54.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.457+0000 7f5b9f7fe700 1 --2- 172.21.15.92:0/917646536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ba00b35d0 0x7f5ba016d4c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400c840 tx=0x7f5b94007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:54.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.457+0000 7f5b87fff700 1 -- 172.21.15.92:0/917646536 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b94005b40 con 0x7f5ba00b35d0 2023-10-06T10:41:54.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.457+0000 7f5b87fff700 1 -- 172.21.15.92:0/917646536 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b94005cc0 con 0x7f5ba00b35d0 2023-10-06T10:41:54.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.457+0000 7f5b87fff700 1 -- 172.21.15.92:0/917646536 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b94007320 con 0x7f5ba00b35d0 2023-10-06T10:41:54.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.457+0000 7f5ba5062700 1 -- 172.21.15.92:0/917646536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba00c5b30 con 0x7f5ba00b35d0 2023-10-06T10:41:54.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.457+0000 7f5ba5062700 1 -- 172.21.15.92:0/917646536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba00c60f0 con 0x7f5ba00b35d0 2023-10-06T10:41:54.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.458+0000 7f5ba5062700 1 -- 172.21.15.92:0/917646536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba00c68f0 con 0x7f5ba00b35d0 2023-10-06T10:41:54.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.458+0000 7f5b87fff700 1 -- 172.21.15.92:0/917646536 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5b9401b3b0 con 0x7f5ba00b35d0 2023-10-06T10:41:54.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.459+0000 7f5b87fff700 1 --2- 172.21.15.92:0/917646536 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5b8c066f30 0x7f5b8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:54.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.459+0000 7f5b9ffff700 1 --2- 172.21.15.92:0/917646536 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5b8c066f30 0x7f5b8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:54.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.459+0000 7f5b87fff700 1 -- 172.21.15.92:0/917646536 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5b94083da0 con 0x7f5ba00b35d0 2023-10-06T10:41:54.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.459+0000 7f5b9ffff700 1 --2- 172.21.15.92:0/917646536 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5b8c066f30 0x7f5b8c0693e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5b900099b0 tx=0x7f5b90017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:54.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.463+0000 7f5b87fff700 1 -- 172.21.15.92:0/917646536 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ba00c68f0 con 0x7f5ba00b35d0 2023-10-06T10:41:54.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.689+0000 7f5ba5062700 1 -- 172.21.15.92:0/917646536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5ba00c68f0 con 0x7f5ba00b35d0 2023-10-06T10:41:54.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.689+0000 7f5b87fff700 1 -- 172.21.15.92:0/917646536 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5ba00c68f0 con 0x7f5ba00b35d0 2023-10-06T10:41:54.691 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T10:41:54.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.691+0000 7f5ba5062700 1 -- 172.21.15.92:0/917646536 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5b8c066f30 msgr2=0x7f5b8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:54.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.691+0000 7f5ba5062700 1 --2- 172.21.15.92:0/917646536 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5b8c066f30 0x7f5b8c0693e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5b900099b0 tx=0x7f5b90017000 comp rx=0 tx=0).stop 2023-10-06T10:41:54.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.691+0000 7f5ba5062700 1 -- 172.21.15.92:0/917646536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ba00b35d0 msgr2=0x7f5ba016d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:54.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.691+0000 7f5ba5062700 1 --2- 172.21.15.92:0/917646536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ba00b35d0 0x7f5ba016d4c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400c840 tx=0x7f5b94007d90 comp rx=0 tx=0).stop 2023-10-06T10:41:54.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.691+0000 7f5ba5062700 1 -- 172.21.15.92:0/917646536 shutdown_connections 2023-10-06T10:41:54.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.691+0000 7f5ba5062700 1 --2- 172.21.15.92:0/917646536 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5b8c066f30 0x7f5b8c0693e0 secure :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5b900099b0 tx=0x7f5b90017000 comp rx=0 tx=0).stop 2023-10-06T10:41:54.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.691+0000 7f5ba5062700 1 --2- 172.21.15.92:0/917646536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ba0159440 0x7f5ba0152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.691+0000 7f5ba5062700 1 --2- 172.21.15.92:0/917646536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ba00b2100 0x7f5ba00c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.691+0000 7f5ba5062700 1 --2- 172.21.15.92:0/917646536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ba00b35d0 0x7f5ba016d4c0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:54.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.691+0000 7f5ba5062700 1 -- 172.21.15.92:0/917646536 >> 172.21.15.92:0/917646536 conn(0x7f5ba006ece0 msgr2=0x7f5ba0158690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:54.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.692+0000 7f5ba5062700 1 -- 172.21.15.92:0/917646536 shutdown_connections 2023-10-06T10:41:54.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:54.692+0000 7f5ba5062700 1 -- 172.21.15.92:0/917646536 wait complete. 2023-10-06T10:41:54.705 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T10:41:54.705 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T10:41:54.705 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T10:41:54.705 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T10:41:54.706 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T10:41:54.706 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T10:41:54.706 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T10:41:54.706 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T10:41:54.706 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T10:41:54.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:41:54.706 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T10:41:54.721 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T10:41:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T10:41:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T10:41:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:41:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T10:41:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:41:54.722 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:41:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:41:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T10:41:54.722 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:41:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:41:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T10:41:54.722 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:41:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:41:54.723 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T10:41:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:41:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T10:41:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42295sec preferred_lft 42295sec 2023-10-06T10:41:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T10:41:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:41:54.723 INFO:teuthology.orchestra.run.smithi092.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T10:41:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 92:7f:88:24:4e:ca brd ff:ff:ff:ff:ff:ff 2023-10-06T10:41:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T10:41:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:41:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::9c9d:23ff:fee4:30ae/64 scope link 2023-10-06T10:41:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:41:54.724 INFO:teuthology.orchestra.run.smithi092.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T10:41:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 92:7f:88:24:4e:ca brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T10:41:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::907f:88ff:fe24:4eca/64 scope link 2023-10-06T10:41:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:41:54.724 INFO:teuthology.orchestra.run.smithi092.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T10:41:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether b6:32:f9:d9:96:1b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T10:41:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::b432:f9ff:fed9:961b/64 scope link 2023-10-06T10:41:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:41:54.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:41:54.725 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T10:41:54.785 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T10:41:54.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:41:54.786 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T10:41:54.843 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T10:41:54.843 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T10:41:54.843 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T10:41:54.843 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T10:41:54.843 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T10:41:54.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:41:54.843 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T10:41:54.843 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T10:41:54.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T10:41:54.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T10:41:54.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T10:41:54.844 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T10:41:54.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:41:54.942 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T10:41:54.957 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T10:41:54.957 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T10:41:54.958 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T10:41:54.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:41:54.958 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-10-06T10:41:55.022 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T10:41:55.022 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T10:41:55.077 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T10:41:55.077 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T10:41:55.077 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T10:41:55.132 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T10:41:55.132 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T10:41:55.132 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:41:55.211 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:41:55.211 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:41:55.230 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:41:55.230 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:41:55.230 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-06T10:41:55.230 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@8f307bfb-c4cb-4f66-aade-46fffe0c6e0c.cephfs=/ 2023-10-06T10:41:55.230 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-06T10:41:55.230 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T10:41:55.230 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T10:41:55.230 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T10:41:55.251 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:41:55.252 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:41:55.252 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-06T10:41:55.252 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@8f307bfb-c4cb-4f66-aade-46fffe0c6e0c.cephfs=/ 2023-10-06T10:41:55.252 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-06T10:41:55.252 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T10:41:55.252 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T10:41:55.252 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T10:41:55.252 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.124:6789,172.21.15.124:6790" 2023-10-06T10:41:55.252 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.124:6789,172.21.15.124:6790:/ 2023-10-06T10:41:55.252 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=8f307bfb-c4cb-4f66-aade-46fffe0c6e0c" will pass to kernel 2023-10-06T10:41:55.254 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T10:41:55.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:41:55.254 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T10:41:55.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-10-06T10:41:55.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.475+0000 7f3c8f713700 1 Processor -- start 2023-10-06T10:41:55.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.475+0000 7f3c8f713700 1 -- start start 2023-10-06T10:41:55.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.476+0000 7f3c8f713700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3c880cbc70 0x7f3c880ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:55.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.476+0000 7f3c8f713700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c880cf5d0 con 0x7f3c881594d0 2023-10-06T10:41:55.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.476+0000 7f3c8f713700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c880cf8b0 con 0x7f3c881598a0 2023-10-06T10:41:55.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.476+0000 7f3c8f713700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c880cfb90 con 0x7f3c880cbc70 2023-10-06T10:41:55.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.476+0000 7f3c8d4af700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3c881594d0 0x7f3c880cb560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54786/0 (socket says 172.21.15.92:54786) 2023-10-06T10:41:55.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.476+0000 7f3c8d4af700 1 -- 172.21.15.92:0/751007057 learned_addr learned my addr 172.21.15.92:0/751007057 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:55.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.476+0000 7f3c8dcb0700 1 --2- 172.21.15.92:0/751007057 >> v2:172.21.15.124:3300/0 conn(0x7f3c880cbc70 0x7f3c880ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:55.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.477+0000 7f3c7ffff700 1 -- 172.21.15.92:0/751007057 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1201615284 0 0) 0x7f3c880cf5d0 con 0x7f3c881594d0 2023-10-06T10:41:55.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.477+0000 7f3c7ffff700 1 -- 172.21.15.92:0/751007057 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c74003570 con 0x7f3c881594d0 2023-10-06T10:41:55.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.477+0000 7f3c7ffff700 1 -- 172.21.15.92:0/751007057 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1085772713 0 0) 0x7f3c74003570 con 0x7f3c881594d0 2023-10-06T10:41:55.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.477+0000 7f3c8dcb0700 1 -- 172.21.15.92:0/751007057 >> v1:172.21.15.124:6789/0 conn(0x7f3c881598a0 legacy=0x7f3c880ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:55.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.477+0000 7f3c8dcb0700 1 -- 172.21.15.92:0/751007057 >> v1:172.21.15.92:6789/0 conn(0x7f3c881594d0 legacy=0x7f3c880cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:55.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.477+0000 7f3c8dcb0700 1 -- 172.21.15.92:0/751007057 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c880cfe70 con 0x7f3c880cbc70 2023-10-06T10:41:55.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.477+0000 7f3c8dcb0700 1 --2- 172.21.15.92:0/751007057 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c880cbc70 0x7f3c880ceef0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3c800098e0 tx=0x7f3c800091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3bf1c8440d8d15e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:55.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.477+0000 7f3c7ffff700 1 -- 172.21.15.92:0/751007057 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c840017f0 con 0x7f3c880cbc70 2023-10-06T10:41:55.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.477+0000 7f3c7ffff700 1 -- 172.21.15.92:0/751007057 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c80011420 con 0x7f3c880cbc70 2023-10-06T10:41:55.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.477+0000 7f3c7ffff700 1 -- 172.21.15.92:0/751007057 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c800117c0 con 0x7f3c880cbc70 2023-10-06T10:41:55.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.478+0000 7f3c8f713700 1 -- 172.21.15.92:0/751007057 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c880cbc70 msgr2=0x7f3c880ceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:55.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.478+0000 7f3c8f713700 1 --2- 172.21.15.92:0/751007057 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c880cbc70 0x7f3c880ceef0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3c800098e0 tx=0x7f3c800091a0 comp rx=0 tx=0).stop 2023-10-06T10:41:55.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.478+0000 7f3c8f713700 1 -- 172.21.15.92:0/751007057 shutdown_connections 2023-10-06T10:41:55.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.478+0000 7f3c8f713700 1 --2- 172.21.15.92:0/751007057 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c880cbc70 0x7f3c880ceef0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:55.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.478+0000 7f3c8f713700 1 -- 172.21.15.92:0/751007057 >> 172.21.15.92:0/751007057 conn(0x7f3c88089e50 msgr2=0x7f3c88099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:55.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.478+0000 7f3c8f713700 1 -- 172.21.15.92:0/751007057 shutdown_connections 2023-10-06T10:41:55.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.478+0000 7f3c8f713700 1 -- 172.21.15.92:0/751007057 wait complete. 2023-10-06T10:41:55.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.479+0000 7f3c8f713700 1 Processor -- start 2023-10-06T10:41:55.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.479+0000 7f3c8f713700 1 -- start start 2023-10-06T10:41:55.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.480+0000 7f3c8f713700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c880cbc70 0x7f3c8814d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:55.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.480+0000 7f3c8f713700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c881594d0 0x7f3c88147f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:55.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.480+0000 7f3c8f713700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c881598a0 0x7f3c8814ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:55.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.480+0000 7f3c8f713700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8814a980 con 0x7f3c881598a0 2023-10-06T10:41:55.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.480+0000 7f3c8f713700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c880b4730 con 0x7f3c881594d0 2023-10-06T10:41:55.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.480+0000 7f3c8f713700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c880b4a10 con 0x7f3c880cbc70 2023-10-06T10:41:55.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.480+0000 7f3c8dcb0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c881598a0 0x7f3c8814ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:55.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.480+0000 7f3c8dcb0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c881598a0 0x7f3c8814ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58528/0 (socket says 172.21.15.92:58528) 2023-10-06T10:41:55.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.480+0000 7f3c8dcb0700 1 -- 172.21.15.92:0/1640716248 learned_addr learned my addr 172.21.15.92:0/1640716248 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:55.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.480+0000 7f3c8d4af700 1 --2- 172.21.15.92:0/1640716248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c880cbc70 0x7f3c8814d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:55.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.480+0000 7f3c8ccae700 1 --2- 172.21.15.92:0/1640716248 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c881594d0 0x7f3c88147f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:55.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.480+0000 7f3c8dcb0700 1 -- 172.21.15.92:0/1640716248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c880cbc70 msgr2=0x7f3c8814d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:55.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.480+0000 7f3c8dcb0700 1 --2- 172.21.15.92:0/1640716248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c880cbc70 0x7f3c8814d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:55.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.480+0000 7f3c8dcb0700 1 -- 172.21.15.92:0/1640716248 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c881594d0 msgr2=0x7f3c88147f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:55.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.480+0000 7f3c8dcb0700 1 --2- 172.21.15.92:0/1640716248 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c881594d0 0x7f3c88147f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:55.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.480+0000 7f3c8dcb0700 1 -- 172.21.15.92:0/1640716248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c80009420 con 0x7f3c881598a0 2023-10-06T10:41:55.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.480+0000 7f3c8dcb0700 1 --2- 172.21.15.92:0/1640716248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c881598a0 0x7f3c8814ffb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3c88067c20 tx=0x7f3c80016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:55.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.480+0000 7f3c7d7fa700 1 -- 172.21.15.92:0/1640716248 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8001cc40 con 0x7f3c881598a0 2023-10-06T10:41:55.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.480+0000 7f3c7d7fa700 1 -- 172.21.15.92:0/1640716248 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c8001cda0 con 0x7f3c881598a0 2023-10-06T10:41:55.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.480+0000 7f3c8f713700 1 -- 172.21.15.92:0/1640716248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c880b4cf0 con 0x7f3c881598a0 2023-10-06T10:41:55.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.480+0000 7f3c7d7fa700 1 -- 172.21.15.92:0/1640716248 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8001a680 con 0x7f3c881598a0 2023-10-06T10:41:55.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.481+0000 7f3c8f713700 1 -- 172.21.15.92:0/1640716248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c880b52b0 con 0x7f3c881598a0 2023-10-06T10:41:55.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.481+0000 7f3c8f713700 1 -- 172.21.15.92:0/1640716248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c880b54a0 con 0x7f3c881598a0 2023-10-06T10:41:55.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.482+0000 7f3c7d7fa700 1 -- 172.21.15.92:0/1640716248 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c8001aa30 con 0x7f3c881598a0 2023-10-06T10:41:55.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.482+0000 7f3c7d7fa700 1 --2- 172.21.15.92:0/1640716248 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c6c066f30 0x7f3c6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:55.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.482+0000 7f3c7d7fa700 1 -- 172.21.15.92:0/1640716248 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3c80086220 con 0x7f3c881598a0 2023-10-06T10:41:55.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.482+0000 7f3c8d4af700 1 --2- 172.21.15.92:0/1640716248 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c6c066f30 0x7f3c6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:55.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.483+0000 7f3c8d4af700 1 --2- 172.21.15.92:0/1640716248 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c6c066f30 0x7f3c6c0693e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3c84007fd0 tx=0x7f3c84007c70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:55.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.486+0000 7f3c7d7fa700 1 -- 172.21.15.92:0/1640716248 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c800531f0 con 0x7f3c881598a0 2023-10-06T10:41:55.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:55.717+0000 7f3c8f713700 1 -- 172.21.15.92:0/1640716248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f3c88064a90 con 0x7f3c881598a0 2023-10-06T10:41:56.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.566+0000 7f3c7d7fa700 1 -- 172.21.15.92:0/1640716248 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v51) v1 ==== 105+0+0 (secure 0 0 0) 0x7f3c80056810 con 0x7f3c881598a0 2023-10-06T10:41:56.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.569+0000 7f3c8f713700 1 -- 172.21.15.92:0/1640716248 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c6c066f30 msgr2=0x7f3c6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:56.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.569+0000 7f3c8f713700 1 --2- 172.21.15.92:0/1640716248 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c6c066f30 0x7f3c6c0693e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3c84007fd0 tx=0x7f3c84007c70 comp rx=0 tx=0).stop 2023-10-06T10:41:56.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.569+0000 7f3c8f713700 1 -- 172.21.15.92:0/1640716248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c881598a0 msgr2=0x7f3c8814ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:56.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.569+0000 7f3c8f713700 1 --2- 172.21.15.92:0/1640716248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c881598a0 0x7f3c8814ffb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3c88067c20 tx=0x7f3c80016000 comp rx=0 tx=0).stop 2023-10-06T10:41:56.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.569+0000 7f3c8f713700 1 -- 172.21.15.92:0/1640716248 shutdown_connections 2023-10-06T10:41:56.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.569+0000 7f3c8f713700 1 --2- 172.21.15.92:0/1640716248 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c6c066f30 0x7f3c6c0693e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:56.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.569+0000 7f3c8f713700 1 --2- 172.21.15.92:0/1640716248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c881598a0 0x7f3c8814ffb0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:56.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.569+0000 7f3c8f713700 1 --2- 172.21.15.92:0/1640716248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c880cbc70 0x7f3c8814d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:56.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.569+0000 7f3c8f713700 1 --2- 172.21.15.92:0/1640716248 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c881594d0 0x7f3c88147f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:56.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.569+0000 7f3c8f713700 1 -- 172.21.15.92:0/1640716248 >> 172.21.15.92:0/1640716248 conn(0x7f3c88089e50 msgr2=0x7f3c880d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:56.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.569+0000 7f3c8f713700 1 -- 172.21.15.92:0/1640716248 shutdown_connections 2023-10-06T10:41:56.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.569+0000 7f3c8f713700 1 -- 172.21.15.92:0/1640716248 wait complete. 2023-10-06T10:41:56.587 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:41:56.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.701+0000 7f941f619700 1 Processor -- start 2023-10-06T10:41:56.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.701+0000 7f941f619700 1 -- start start 2023-10-06T10:41:56.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f941f619700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f94180b49a0 0x7f94180aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:56.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f941f619700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94180b0760 0x7f94180b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:56.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f941f619700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94180b0c60 con 0x7f9418159560 2023-10-06T10:41:56.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f941f619700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94180b0f40 con 0x7f94180b49a0 2023-10-06T10:41:56.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f941f619700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94180b1220 con 0x7f94180b0760 2023-10-06T10:41:56.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f941d3b5700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9418159560 0x7f94180aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54790/0 (socket says 172.21.15.92:54790) 2023-10-06T10:41:56.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f941d3b5700 1 -- 172.21.15.92:0/1981442813 learned_addr learned my addr 172.21.15.92:0/1981442813 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:56.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f941cbb4700 1 --2- 172.21.15.92:0/1981442813 >> v2:172.21.15.124:3300/0 conn(0x7f94180b49a0 0x7f94180aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:56.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f941dbb6700 1 --2- 172.21.15.92:0/1981442813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94180b0760 0x7f94180b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:56.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f9417fff700 1 -- 172.21.15.92:0/1981442813 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925112044 0 0) 0x7f94180b0c60 con 0x7f9418159560 2023-10-06T10:41:56.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f9417fff700 1 -- 172.21.15.92:0/1981442813 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9400003570 con 0x7f9418159560 2023-10-06T10:41:56.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f9417fff700 1 -- 172.21.15.92:0/1981442813 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1885779168 0 0) 0x7f9400003570 con 0x7f9418159560 2023-10-06T10:41:56.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f9417fff700 1 -- 172.21.15.92:0/1981442813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94180b0760 msgr2=0x7f94180b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:56.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f9417fff700 1 --2- 172.21.15.92:0/1981442813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94180b0760 0x7f94180b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:56.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f9417fff700 1 -- 172.21.15.92:0/1981442813 >> v2:172.21.15.124:3300/0 conn(0x7f94180b49a0 msgr2=0x7f94180aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:56.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f9417fff700 1 --2- 172.21.15.92:0/1981442813 >> v2:172.21.15.124:3300/0 conn(0x7f94180b49a0 0x7f94180aac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:56.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f9417fff700 1 -- 172.21.15.92:0/1981442813 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94180b1500 con 0x7f9418159560 2023-10-06T10:41:56.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f9417fff700 1 -- 172.21.15.92:0/1981442813 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f94100015f0 con 0x7f9418159560 2023-10-06T10:41:56.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f9417fff700 1 -- 172.21.15.92:0/1981442813 >> v1:172.21.15.92:6789/0 conn(0x7f9418159560 legacy=0x7f94180aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:56.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f9417fff700 1 --2- 172.21.15.92:0/1981442813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9400003a60 0x7f9400003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:56.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f9417fff700 1 --2- 172.21.15.92:0/1981442813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94180b0760 0x7f9400004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:56.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f9417fff700 1 --2- 172.21.15.92:0/1981442813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9400004d50 0x7f94000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:56.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f9417fff700 1 -- 172.21.15.92:0/1981442813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94180b0f40 con 0x7f9400003a60 2023-10-06T10:41:56.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f9417fff700 1 -- 172.21.15.92:0/1981442813 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94180b1220 con 0x7f9400004d50 2023-10-06T10:41:56.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.702+0000 7f9417fff700 1 -- 172.21.15.92:0/1981442813 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f940000ba60 con 0x7f94180b0760 2023-10-06T10:41:56.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.703+0000 7f941d3b5700 1 --2- 172.21.15.92:0/1981442813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9400003a60 0x7f9400003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:56.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.703+0000 7f941cbb4700 1 --2- 172.21.15.92:0/1981442813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94180b0760 0x7f9400004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:56.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.703+0000 7f941dbb6700 1 --2- 172.21.15.92:0/1981442813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9400004d50 0x7f94000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:56.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.703+0000 7f941d3b5700 1 -- 172.21.15.92:0/1981442813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94180b0760 msgr2=0x7f9400004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:56.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.703+0000 7f941d3b5700 1 --2- 172.21.15.92:0/1981442813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94180b0760 0x7f9400004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:56.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.703+0000 7f941d3b5700 1 -- 172.21.15.92:0/1981442813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9400004d50 msgr2=0x7f94000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:56.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.703+0000 7f941d3b5700 1 --2- 172.21.15.92:0/1981442813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9400004d50 0x7f94000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:56.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.703+0000 7f941d3b5700 1 -- 172.21.15.92:0/1981442813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9400003570 con 0x7f9400003a60 2023-10-06T10:41:56.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.703+0000 7f941cbb4700 1 --2- 172.21.15.92:0/1981442813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94180b0760 0x7f9400004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:56.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.703+0000 7f941dbb6700 1 --2- 172.21.15.92:0/1981442813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9400004d50 0x7f94000051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:56.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.703+0000 7f941d3b5700 1 --2- 172.21.15.92:0/1981442813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9400003a60 0x7f9400003e70 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f941000c3c0 tx=0x7f9410007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf8e6ddbe16b58b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:56.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.703+0000 7f9417fff700 1 -- 172.21.15.92:0/1981442813 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941000cc30 con 0x7f9400003a60 2023-10-06T10:41:56.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.703+0000 7f9417fff700 1 -- 172.21.15.92:0/1981442813 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941000cdb0 con 0x7f9400003a60 2023-10-06T10:41:56.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.703+0000 7f9417fff700 1 -- 172.21.15.92:0/1981442813 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9410007320 con 0x7f9400003a60 2023-10-06T10:41:56.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.703+0000 7f941f619700 1 -- 172.21.15.92:0/1981442813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9400003a60 msgr2=0x7f9400003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:56.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.703+0000 7f941f619700 1 --2- 172.21.15.92:0/1981442813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9400003a60 0x7f9400003e70 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f941000c3c0 tx=0x7f9410007920 comp rx=0 tx=0).stop 2023-10-06T10:41:56.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.704+0000 7f941dbb6700 1 -- 172.21.15.92:0/1981442813 reap_dead start 2023-10-06T10:41:56.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.704+0000 7f941f619700 1 -- 172.21.15.92:0/1981442813 shutdown_connections 2023-10-06T10:41:56.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.704+0000 7f941f619700 1 -- 172.21.15.92:0/1981442813 >> 172.21.15.92:0/1981442813 conn(0x7f94180997f0 msgr2=0x7f941806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:56.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.704+0000 7f941f619700 1 -- 172.21.15.92:0/1981442813 shutdown_connections 2023-10-06T10:41:56.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.704+0000 7f941f619700 1 -- 172.21.15.92:0/1981442813 wait complete. 2023-10-06T10:41:56.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.705+0000 7f941f619700 1 Processor -- start 2023-10-06T10:41:56.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.705+0000 7f941f619700 1 -- start start 2023-10-06T10:41:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.706+0000 7f941f619700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9400003a60 0x7f94181579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.706+0000 7f941f619700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94180aebf0 0x7f941809aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.706+0000 7f941f619700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94180cc460 0x7f94180cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.706+0000 7f941f619700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94180cce40 con 0x7f94180cc460 2023-10-06T10:41:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.706+0000 7f941f619700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94180ae760 con 0x7f9400003a60 2023-10-06T10:41:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.706+0000 7f941f619700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94180ab800 con 0x7f94180aebf0 2023-10-06T10:41:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.706+0000 7f941dbb6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94180cc460 0x7f94180cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.706+0000 7f941dbb6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94180cc460 0x7f94180cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58552/0 (socket says 172.21.15.92:58552) 2023-10-06T10:41:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.706+0000 7f941dbb6700 1 -- 172.21.15.92:0/3957461808 learned_addr learned my addr 172.21.15.92:0/3957461808 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:56.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.706+0000 7f941cbb4700 1 --2- 172.21.15.92:0/3957461808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94180aebf0 0x7f941809aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:56.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.706+0000 7f941d3b5700 1 --2- 172.21.15.92:0/3957461808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9400003a60 0x7f94181579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:56.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.706+0000 7f941dbb6700 1 -- 172.21.15.92:0/3957461808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94180aebf0 msgr2=0x7f941809aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:56.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.706+0000 7f941dbb6700 1 --2- 172.21.15.92:0/3957461808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94180aebf0 0x7f941809aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:56.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.706+0000 7f941dbb6700 1 -- 172.21.15.92:0/3957461808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9400003a60 msgr2=0x7f94181579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:56.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.706+0000 7f941dbb6700 1 --2- 172.21.15.92:0/3957461808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9400003a60 0x7f94181579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:56.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.706+0000 7f941dbb6700 1 -- 172.21.15.92:0/3957461808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94180adb50 con 0x7f94180cc460 2023-10-06T10:41:56.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.706+0000 7f941dbb6700 1 --2- 172.21.15.92:0/3957461808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94180cc460 0x7f94180cc900 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f94180b2f10 tx=0x7f9408008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:56.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.706+0000 7f94157fa700 1 -- 172.21.15.92:0/3957461808 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9408015670 con 0x7f94180cc460 2023-10-06T10:41:56.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.706+0000 7f94157fa700 1 -- 172.21.15.92:0/3957461808 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94080157d0 con 0x7f94180cc460 2023-10-06T10:41:56.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.706+0000 7f94157fa700 1 -- 172.21.15.92:0/3957461808 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9408015b90 con 0x7f94180cc460 2023-10-06T10:41:56.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.707+0000 7f941f619700 1 -- 172.21.15.92:0/3957461808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94180ade30 con 0x7f94180cc460 2023-10-06T10:41:56.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.707+0000 7f941f619700 1 -- 172.21.15.92:0/3957461808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94180ae3f0 con 0x7f94180cc460 2023-10-06T10:41:56.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.707+0000 7f941f619700 1 -- 172.21.15.92:0/3957461808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94180ae8f0 con 0x7f94180cc460 2023-10-06T10:41:56.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.708+0000 7f94157fa700 1 -- 172.21.15.92:0/3957461808 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f940801f410 con 0x7f94180cc460 2023-10-06T10:41:56.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.708+0000 7f94157fa700 1 --2- 172.21.15.92:0/3957461808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9404066f30 0x7f94040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:56.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.708+0000 7f94157fa700 1 -- 172.21.15.92:0/3957461808 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9408084650 con 0x7f94180cc460 2023-10-06T10:41:56.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.708+0000 7f941d3b5700 1 --2- 172.21.15.92:0/3957461808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9404066f30 0x7f94040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:56.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.709+0000 7f941d3b5700 1 --2- 172.21.15.92:0/3957461808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9404066f30 0x7f94040693e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9410008fd0 tx=0x7f9410015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:56.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.712+0000 7f94157fa700 1 -- 172.21.15.92:0/3957461808 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9408051580 con 0x7f94180cc460 2023-10-06T10:41:56.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.937+0000 7f941f619700 1 -- 172.21.15.92:0/3957461808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9418064a90 con 0x7f94180cc460 2023-10-06T10:41:56.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.938+0000 7f94157fa700 1 -- 172.21.15.92:0/3957461808 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4615 (secure 0 0 0) 0x7f9408054ba0 con 0x7f94180cc460 2023-10-06T10:41:56.940 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:56.940 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":52,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/1409495728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":1409495728},{"type":"v1","addr":"172.21.15.124:6834","nonce":1409495728}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6019,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2297612118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2297612118},{"type":"v1","addr":"172.21.15.124:6835","nonce":2297612118}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:41:47.898774+0000","modified":"2023-10-06T10:41:56.568199+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":6486,"mds_1":6483},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6483":{"gid":6483,"name":"a","rank":1,"incarnation":52,"state":"up:creating","state_seq":1,"addr":"172.21.15.92:6836/171578176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":171578176},{"type":"v1","addr":"172.21.15.92:6836","nonce":171578176}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6486":{"gid":6486,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/818031030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":818031030},{"type":"v1","addr":"172.21.15.92:6837","nonce":818031030}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T10:41:56.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.940+0000 7f941f619700 1 -- 172.21.15.92:0/3957461808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9404066f30 msgr2=0x7f94040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:56.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.940+0000 7f941f619700 1 --2- 172.21.15.92:0/3957461808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9404066f30 0x7f94040693e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9410008fd0 tx=0x7f9410015000 comp rx=0 tx=0).stop 2023-10-06T10:41:56.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.940+0000 7f941f619700 1 -- 172.21.15.92:0/3957461808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94180cc460 msgr2=0x7f94180cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:56.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.940+0000 7f941f619700 1 --2- 172.21.15.92:0/3957461808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94180cc460 0x7f94180cc900 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f94180b2f10 tx=0x7f9408008040 comp rx=0 tx=0).stop 2023-10-06T10:41:56.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.940+0000 7f941f619700 1 -- 172.21.15.92:0/3957461808 shutdown_connections 2023-10-06T10:41:56.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.940+0000 7f941f619700 1 --2- 172.21.15.92:0/3957461808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9404066f30 0x7f94040693e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:56.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.940+0000 7f941f619700 1 --2- 172.21.15.92:0/3957461808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94180cc460 0x7f94180cc900 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:56.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.940+0000 7f941f619700 1 --2- 172.21.15.92:0/3957461808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9400003a60 0x7f94181579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:56.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.940+0000 7f941f619700 1 --2- 172.21.15.92:0/3957461808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94180aebf0 0x7f941809aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:56.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.940+0000 7f941f619700 1 -- 172.21.15.92:0/3957461808 >> 172.21.15.92:0/3957461808 conn(0x7f94180997f0 msgr2=0x7f94181588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:56.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.940+0000 7f941f619700 1 -- 172.21.15.92:0/3957461808 shutdown_connections 2023-10-06T10:41:56.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:56.940+0000 7f941f619700 1 -- 172.21.15.92:0/3957461808 wait complete. 2023-10-06T10:41:56.943 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 52 2023-10-06T10:41:57.356 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T10:41:47.898774+0000', 'modified': '2023-10-06T10:41:56.568199+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 6486, 'mds_1': 6483}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6483': {'gid': 6483, 'name': 'a', 'rank': 1, 'incarnation': 52, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.92:6836/171578176', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 171578176}, {'type': 'v1', 'addr': '172.21.15.92:6836', 'nonce': 171578176}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_6486': {'gid': 6486, 'name': 'c', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6837/818031030', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 818031030}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 818031030}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T10:41:57.356 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6483:up:creating 2023-10-06T10:41:57.818 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-10-06T10:41:57.817+0000 7fdec7cae700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-06T10:41:58.357 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:41:58.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.471+0000 7f89b4c88700 1 Processor -- start 2023-10-06T10:41:58.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.471+0000 7f89b4c88700 1 -- start start 2023-10-06T10:41:58.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.471+0000 7f89b4c88700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f89b00b0760 0x7f89b00ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:58.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.471+0000 7f89b4c88700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89b00b0c60 con 0x7f89b0159560 2023-10-06T10:41:58.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.471+0000 7f89b4c88700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89b00b0f40 con 0x7f89b00b49a0 2023-10-06T10:41:58.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.471+0000 7f89b4c88700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b00ab350 con 0x7f89b00b0760 2023-10-06T10:41:58.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.472+0000 7f89ae59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f89b0159560 0x7f89b00aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54800/0 (socket says 172.21.15.92:54800) 2023-10-06T10:41:58.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.472+0000 7f89ae59c700 1 -- 172.21.15.92:0/3304683958 learned_addr learned my addr 172.21.15.92:0/3304683958 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:58.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.472+0000 7f89aed9d700 1 --2- 172.21.15.92:0/3304683958 >> v2:172.21.15.92:3300/0 conn(0x7f89b00b0760 0x7f89b00ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:58.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.472+0000 7f89add9b700 1 -- 172.21.15.92:0/3304683958 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995690266 0 0) 0x7f89b00b0c60 con 0x7f89b0159560 2023-10-06T10:41:58.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.472+0000 7f89add9b700 1 -- 172.21.15.92:0/3304683958 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f899c003570 con 0x7f89b0159560 2023-10-06T10:41:58.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.472+0000 7f89aed9d700 1 -- 172.21.15.92:0/3304683958 >> v1:172.21.15.124:6789/0 conn(0x7f89b00b49a0 legacy=0x7f89b00aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:58.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.472+0000 7f89aed9d700 1 -- 172.21.15.92:0/3304683958 >> v1:172.21.15.92:6789/0 conn(0x7f89b0159560 legacy=0x7f89b00aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:58.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.472+0000 7f89aed9d700 1 -- 172.21.15.92:0/3304683958 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b00b1220 con 0x7f89b00b0760 2023-10-06T10:41:58.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.472+0000 7f89aed9d700 1 --2- 172.21.15.92:0/3304683958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89b00b0760 0x7f89b00ccf90 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009e90 tx=0x7f89a8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95c9dfc59a059c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:58.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.472+0000 7f89add9b700 1 -- 172.21.15.92:0/3304683958 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a800e070 con 0x7f89b00b0760 2023-10-06T10:41:58.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.472+0000 7f89add9b700 1 -- 172.21.15.92:0/3304683958 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a800bc40 con 0x7f89b00b0760 2023-10-06T10:41:58.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.472+0000 7f89add9b700 1 -- 172.21.15.92:0/3304683958 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8012430 con 0x7f89b00b0760 2023-10-06T10:41:58.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.473+0000 7f89b4c88700 1 -- 172.21.15.92:0/3304683958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89b00b0760 msgr2=0x7f89b00ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:58.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.473+0000 7f89b4c88700 1 --2- 172.21.15.92:0/3304683958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89b00b0760 0x7f89b00ccf90 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009e90 tx=0x7f89a8009780 comp rx=0 tx=0).stop 2023-10-06T10:41:58.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.473+0000 7f89b4c88700 1 -- 172.21.15.92:0/3304683958 shutdown_connections 2023-10-06T10:41:58.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.473+0000 7f89b4c88700 1 --2- 172.21.15.92:0/3304683958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89b00b0760 0x7f89b00ccf90 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:58.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.473+0000 7f89b4c88700 1 -- 172.21.15.92:0/3304683958 >> 172.21.15.92:0/3304683958 conn(0x7f89b00997f0 msgr2=0x7f89b006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:58.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.473+0000 7f89b4c88700 1 -- 172.21.15.92:0/3304683958 shutdown_connections 2023-10-06T10:41:58.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.473+0000 7f89b4c88700 1 -- 172.21.15.92:0/3304683958 wait complete. 2023-10-06T10:41:58.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.474+0000 7f89b4c88700 1 Processor -- start 2023-10-06T10:41:58.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.474+0000 7f89b4c88700 1 -- start start 2023-10-06T10:41:58.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89b4c88700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89b00b0760 0x7f89b015bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:58.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89b4c88700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89b00b49a0 0x7f89b009dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:58.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89b4c88700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89b0159560 0x7f89b015e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:58.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89b4c88700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b009e0f0 con 0x7f89b00b0760 2023-10-06T10:41:58.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89b4c88700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b016de10 con 0x7f89b0159560 2023-10-06T10:41:58.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89b4c88700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b016d290 con 0x7f89b00b49a0 2023-10-06T10:41:58.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89ae59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89b00b0760 0x7f89b015bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:58.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89ae59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89b00b0760 0x7f89b015bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58576/0 (socket says 172.21.15.92:58576) 2023-10-06T10:41:58.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89ae59c700 1 -- 172.21.15.92:0/784329189 learned_addr learned my addr 172.21.15.92:0/784329189 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:58.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89a5d9b700 1 --2- 172.21.15.92:0/784329189 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89b00b49a0 0x7f89b009dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:58.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89aed9d700 1 --2- 172.21.15.92:0/784329189 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89b0159560 0x7f89b015e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:58.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89ae59c700 1 -- 172.21.15.92:0/784329189 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89b00b49a0 msgr2=0x7f89b009dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:58.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89ae59c700 1 --2- 172.21.15.92:0/784329189 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89b00b49a0 0x7f89b009dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:58.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89ae59c700 1 -- 172.21.15.92:0/784329189 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89b0159560 msgr2=0x7f89b015e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:58.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89ae59c700 1 --2- 172.21.15.92:0/784329189 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89b0159560 0x7f89b015e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:58.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89ae59c700 1 -- 172.21.15.92:0/784329189 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a8009a20 con 0x7f89b00b0760 2023-10-06T10:41:58.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89a5d9b700 1 --2- 172.21.15.92:0/784329189 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89b00b49a0 0x7f89b009dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:58.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89aed9d700 1 --2- 172.21.15.92:0/784329189 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89b0159560 0x7f89b015e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:58.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89ae59c700 1 --2- 172.21.15.92:0/784329189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89b00b0760 0x7f89b015bce0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f89a000c790 tx=0x7f89a0007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:58.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89a6ffd700 1 -- 172.21.15.92:0/784329189 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a00072e0 con 0x7f89b00b0760 2023-10-06T10:41:58.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89a6ffd700 1 -- 172.21.15.92:0/784329189 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a0007440 con 0x7f89b00b0760 2023-10-06T10:41:58.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89b4c88700 1 -- 172.21.15.92:0/784329189 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b016d570 con 0x7f89b00b0760 2023-10-06T10:41:58.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89a6ffd700 1 -- 172.21.15.92:0/784329189 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a00057f0 con 0x7f89b00b0760 2023-10-06T10:41:58.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.475+0000 7f89b4c88700 1 -- 172.21.15.92:0/784329189 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b016db30 con 0x7f89b00b0760 2023-10-06T10:41:58.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.476+0000 7f89b4c88700 1 -- 172.21.15.92:0/784329189 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b015a990 con 0x7f89b00b0760 2023-10-06T10:41:58.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.477+0000 7f89a6ffd700 1 -- 172.21.15.92:0/784329189 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f89a0005c20 con 0x7f89b00b0760 2023-10-06T10:41:58.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.477+0000 7f89a6ffd700 1 --2- 172.21.15.92:0/784329189 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8990066f30 0x7f89900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:58.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.477+0000 7f89a5d9b700 1 --2- 172.21.15.92:0/784329189 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8990066f30 0x7f89900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:58.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.477+0000 7f89a6ffd700 1 -- 172.21.15.92:0/784329189 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f89a0083b10 con 0x7f89b00b0760 2023-10-06T10:41:58.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.478+0000 7f89a5d9b700 1 --2- 172.21.15.92:0/784329189 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8990066f30 0x7f89900693e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f89b015f020 tx=0x7f8998009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:58.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.481+0000 7f89a6ffd700 1 -- 172.21.15.92:0/784329189 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89a004b5a0 con 0x7f89b00b0760 2023-10-06T10:41:58.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.711+0000 7f89b4c88700 1 -- 172.21.15.92:0/784329189 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89b0064a90 con 0x7f89b00b0760 2023-10-06T10:41:58.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.712+0000 7f89a6ffd700 1 -- 172.21.15.92:0/784329189 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4627 (secure 0 0 0) 0x7f89a0017020 con 0x7f89b00b0760 2023-10-06T10:41:58.714 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:58.714 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":55,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/1409495728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":1409495728},{"type":"v1","addr":"172.21.15.124:6834","nonce":1409495728}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:41:47.898774+0000","modified":"2023-10-06T10:41:58.576160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6486,"mds_1":6483,"mds_2":6019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6019":{"gid":6019,"name":"b","rank":2,"incarnation":54,"state":"up:active","state_seq":5,"addr":"172.21.15.124:6835/2297612118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2297612118},{"type":"v1","addr":"172.21.15.124:6835","nonce":2297612118}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6483":{"gid":6483,"name":"a","rank":1,"incarnation":52,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6836/171578176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":171578176},{"type":"v1","addr":"172.21.15.92:6836","nonce":171578176}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6486":{"gid":6486,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/818031030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":818031030},{"type":"v1","addr":"172.21.15.92:6837","nonce":818031030}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T10:41:58.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.713+0000 7f89b4c88700 1 -- 172.21.15.92:0/784329189 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8990066f30 msgr2=0x7f89900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:58.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.713+0000 7f89b4c88700 1 --2- 172.21.15.92:0/784329189 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8990066f30 0x7f89900693e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f89b015f020 tx=0x7f8998009040 comp rx=0 tx=0).stop 2023-10-06T10:41:58.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.713+0000 7f89b4c88700 1 -- 172.21.15.92:0/784329189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89b00b0760 msgr2=0x7f89b015bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:58.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.713+0000 7f89b4c88700 1 --2- 172.21.15.92:0/784329189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89b00b0760 0x7f89b015bce0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f89a000c790 tx=0x7f89a0007c80 comp rx=0 tx=0).stop 2023-10-06T10:41:58.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.714+0000 7f89b4c88700 1 -- 172.21.15.92:0/784329189 shutdown_connections 2023-10-06T10:41:58.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.714+0000 7f89b4c88700 1 --2- 172.21.15.92:0/784329189 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8990066f30 0x7f89900693e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:58.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.714+0000 7f89b4c88700 1 --2- 172.21.15.92:0/784329189 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89b0159560 0x7f89b015e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:58.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.714+0000 7f89b4c88700 1 --2- 172.21.15.92:0/784329189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89b00b0760 0x7f89b015bce0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:58.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.714+0000 7f89b4c88700 1 --2- 172.21.15.92:0/784329189 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89b00b49a0 0x7f89b009dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:58.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.714+0000 7f89b4c88700 1 -- 172.21.15.92:0/784329189 >> 172.21.15.92:0/784329189 conn(0x7f89b00997f0 msgr2=0x7f89b0158860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:58.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.714+0000 7f89b4c88700 1 -- 172.21.15.92:0/784329189 shutdown_connections 2023-10-06T10:41:58.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:58.714+0000 7f89b4c88700 1 -- 172.21.15.92:0/784329189 wait complete. 2023-10-06T10:41:58.717 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 55 2023-10-06T10:41:59.130 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T10:41:47.898774+0000', 'modified': '2023-10-06T10:41:58.576160+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 6486, 'mds_1': 6483, 'mds_2': 6019}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6019': {'gid': 6019, 'name': 'b', 'rank': 2, 'incarnation': 54, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.124:6835/2297612118', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6833', 'nonce': 2297612118}, {'type': 'v1', 'addr': '172.21.15.124:6835', 'nonce': 2297612118}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_6483': {'gid': 6483, 'name': 'a', 'rank': 1, 'incarnation': 52, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.92:6836/171578176', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 171578176}, {'type': 'v1', 'addr': '172.21.15.92:6836', 'nonce': 171578176}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_6486': {'gid': 6486, 'name': 'c', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6837/818031030', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 818031030}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 818031030}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T10:41:59.131 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-10-06T10:41:59.131 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T10:41:59.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.248+0000 7f389a289700 1 Processor -- start 2023-10-06T10:41:59.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.248+0000 7f389a289700 1 -- start start 2023-10-06T10:41:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.249+0000 7f389a289700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3894159560 0x7f38940aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.249+0000 7f389a289700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38940b49a0 0x7f38940aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.249+0000 7f389a289700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f38940b0760 0x7f38940b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.249+0000 7f389a289700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38940b0c60 con 0x7f38940b0760 2023-10-06T10:41:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.249+0000 7f389a289700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38940b0f40 con 0x7f3894159560 2023-10-06T10:41:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.249+0000 7f389a289700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38940b1220 con 0x7f38940b49a0 2023-10-06T10:41:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.249+0000 7f3898826700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f38940b0760 0x7f38940b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.249+0000 7f3898826700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f38940b0760 0x7f38940b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58592/0 (socket says 172.21.15.92:58592) 2023-10-06T10:41:59.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.249+0000 7f3898826700 1 -- 172.21.15.92:0/2921575247 learned_addr learned my addr 172.21.15.92:0/2921575247 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:59.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.249+0000 7f3893fff700 1 --2- 172.21.15.92:0/2921575247 >> v2:172.21.15.124:3300/0 conn(0x7f3894159560 0x7f38940aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.249+0000 7f38937fe700 1 --2- 172.21.15.92:0/2921575247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38940b49a0 0x7f38940aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.250+0000 7f3898826700 1 -- 172.21.15.92:0/2921575247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38940b49a0 msgr2=0x7f38940aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.250+0000 7f3898826700 1 --2- 172.21.15.92:0/2921575247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38940b49a0 0x7f38940aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.250+0000 7f3898826700 1 -- 172.21.15.92:0/2921575247 >> v2:172.21.15.124:3300/0 conn(0x7f3894159560 msgr2=0x7f38940aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.250+0000 7f3898826700 1 --2- 172.21.15.92:0/2921575247 >> v2:172.21.15.124:3300/0 conn(0x7f3894159560 0x7f38940aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.250+0000 7f3898826700 1 -- 172.21.15.92:0/2921575247 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38940b1500 con 0x7f38940b0760 2023-10-06T10:41:59.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.250+0000 7f3893fff700 1 --2- 172.21.15.92:0/2921575247 >> v2:172.21.15.124:3300/0 conn(0x7f3894159560 0x7f38940aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:59.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.250+0000 7f3898826700 1 --2- 172.21.15.92:0/2921575247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38940b0760 0x7f38940b3890 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3894088c80 tx=0x7f388c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf92d3b3a2c971cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.250+0000 7f3892ffd700 1 -- 172.21.15.92:0/2921575247 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c00e070 con 0x7f38940b0760 2023-10-06T10:41:59.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.250+0000 7f3892ffd700 1 -- 172.21.15.92:0/2921575247 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388c00ba50 con 0x7f38940b0760 2023-10-06T10:41:59.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.250+0000 7f3892ffd700 1 -- 172.21.15.92:0/2921575247 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c0092e0 con 0x7f38940b0760 2023-10-06T10:41:59.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.250+0000 7f389a289700 1 -- 172.21.15.92:0/2921575247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38940b0760 msgr2=0x7f38940b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.250+0000 7f389a289700 1 --2- 172.21.15.92:0/2921575247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38940b0760 0x7f38940b3890 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3894088c80 tx=0x7f388c009840 comp rx=0 tx=0).stop 2023-10-06T10:41:59.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.250+0000 7f389a289700 1 -- 172.21.15.92:0/2921575247 shutdown_connections 2023-10-06T10:41:59.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.250+0000 7f389a289700 1 --2- 172.21.15.92:0/2921575247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38940b0760 0x7f38940b3890 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.250+0000 7f389a289700 1 --2- 172.21.15.92:0/2921575247 >> v2:172.21.15.124:3300/0 conn(0x7f3894159560 0x7f38940aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.250+0000 7f389a289700 1 --2- 172.21.15.92:0/2921575247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38940b49a0 0x7f38940aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.250+0000 7f389a289700 1 -- 172.21.15.92:0/2921575247 >> 172.21.15.92:0/2921575247 conn(0x7f38940997f0 msgr2=0x7f389406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:59.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.250+0000 7f389a289700 1 -- 172.21.15.92:0/2921575247 shutdown_connections 2023-10-06T10:41:59.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.250+0000 7f389a289700 1 -- 172.21.15.92:0/2921575247 wait complete. 2023-10-06T10:41:59.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.251+0000 7f389a289700 1 Processor -- start 2023-10-06T10:41:59.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.252+0000 7f389a289700 1 -- start start 2023-10-06T10:41:59.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.252+0000 7f389a289700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38940b0760 0x7f389415dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.252+0000 7f389a289700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38940b49a0 0x7f38941566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.252+0000 7f389a289700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3894159560 0x7f389409a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.252+0000 7f389a289700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389409d1b0 con 0x7f38940b0760 2023-10-06T10:41:59.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.252+0000 7f389a289700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389415e1b0 con 0x7f38940b49a0 2023-10-06T10:41:59.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.252+0000 7f389a289700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389416a040 con 0x7f3894159560 2023-10-06T10:41:59.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.252+0000 7f3893fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38940b0760 0x7f389415dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.252+0000 7f3893fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38940b0760 0x7f389415dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58600/0 (socket says 172.21.15.92:58600) 2023-10-06T10:41:59.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.252+0000 7f3893fff700 1 -- 172.21.15.92:0/2776345995 learned_addr learned my addr 172.21.15.92:0/2776345995 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:59.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.253+0000 7f38937fe700 1 --2- 172.21.15.92:0/2776345995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38940b49a0 0x7f38941566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.253+0000 7f3898826700 1 --2- 172.21.15.92:0/2776345995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3894159560 0x7f389409a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.253+0000 7f3893fff700 1 -- 172.21.15.92:0/2776345995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3894159560 msgr2=0x7f389409a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.253+0000 7f3893fff700 1 --2- 172.21.15.92:0/2776345995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3894159560 0x7f389409a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.253+0000 7f3893fff700 1 -- 172.21.15.92:0/2776345995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38940b49a0 msgr2=0x7f38941566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.253+0000 7f3893fff700 1 --2- 172.21.15.92:0/2776345995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38940b49a0 0x7f38941566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.253+0000 7f3893fff700 1 -- 172.21.15.92:0/2776345995 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c009ae0 con 0x7f38940b0760 2023-10-06T10:41:59.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.253+0000 7f3898826700 1 --2- 172.21.15.92:0/2776345995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3894159560 0x7f389409a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:59.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.253+0000 7f3893fff700 1 --2- 172.21.15.92:0/2776345995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38940b0760 0x7f389415dc70 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f388400d890 tx=0x7f3884008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.253+0000 7f387bfff700 1 -- 172.21.15.92:0/2776345995 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3884005a00 con 0x7f38940b0760 2023-10-06T10:41:59.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.253+0000 7f387bfff700 1 -- 172.21.15.92:0/2776345995 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3884005b60 con 0x7f38940b0760 2023-10-06T10:41:59.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.253+0000 7f389a289700 1 -- 172.21.15.92:0/2776345995 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f389416a320 con 0x7f38940b0760 2023-10-06T10:41:59.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.253+0000 7f387bfff700 1 -- 172.21.15.92:0/2776345995 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38840125f0 con 0x7f38940b0760 2023-10-06T10:41:59.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.253+0000 7f389a289700 1 -- 172.21.15.92:0/2776345995 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3894156c10 con 0x7f38940b0760 2023-10-06T10:41:59.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.254+0000 7f387bfff700 1 -- 172.21.15.92:0/2776345995 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f38840129a0 con 0x7f38940b0760 2023-10-06T10:41:59.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.255+0000 7f387bfff700 1 --2- 172.21.15.92:0/2776345995 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3880066f30 0x7f38800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.255+0000 7f387bfff700 1 -- 172.21.15.92:0/2776345995 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3884084a10 con 0x7f38940b0760 2023-10-06T10:41:59.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.255+0000 7f38937fe700 1 --2- 172.21.15.92:0/2776345995 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3880066f30 0x7f38800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.255+0000 7f38937fe700 1 --2- 172.21.15.92:0/2776345995 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3880066f30 0x7f38800693e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f38940cc1b0 tx=0x7f3888008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.257+0000 7f38a3581b80 1 Processor -- start 2023-10-06T10:41:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.257+0000 7f38a3581b80 1 -- start start 2023-10-06T10:41:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.258+0000 7f38a3581b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a42ccb7a90 0x55a42ccb7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.258+0000 7f38a3581b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a42cc739a0 0x55a42cc70580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.258+0000 7f38a3581b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42cc71bd0 0x55a42cc70ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.258+0000 7f38a3581b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42cbf0c80 con 0x55a42cc71bd0 2023-10-06T10:41:59.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.258+0000 7f38a3581b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42cbc7a70 con 0x55a42ccb7a90 2023-10-06T10:41:59.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.258+0000 7f38a3581b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a42cbc5270 con 0x55a42cc739a0 2023-10-06T10:41:59.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.258+0000 7f3898826700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42cc71bd0 0x55a42cc70ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.258+0000 7f3898826700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42cc71bd0 0x55a42cc70ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58604/0 (socket says 172.21.15.92:58604) 2023-10-06T10:41:59.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.258+0000 7f3898826700 1 -- 172.21.15.92:0/3190253780 learned_addr learned my addr 172.21.15.92:0/3190253780 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:59.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.258+0000 7f38937fe700 1 --2- 172.21.15.92:0/3190253780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a42cc739a0 0x55a42cc70580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.258+0000 7f3893fff700 1 --2- 172.21.15.92:0/3190253780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a42ccb7a90 0x55a42ccb7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.258+0000 7f3898826700 1 -- 172.21.15.92:0/3190253780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a42cc739a0 msgr2=0x55a42cc70580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.258+0000 7f3898826700 1 --2- 172.21.15.92:0/3190253780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a42cc739a0 0x55a42cc70580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.258+0000 7f3898826700 1 -- 172.21.15.92:0/3190253780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a42ccb7a90 msgr2=0x55a42ccb7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.258+0000 7f3898826700 1 --2- 172.21.15.92:0/3190253780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a42ccb7a90 0x55a42ccb7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.258+0000 7f3898826700 1 -- 172.21.15.92:0/3190253780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a42ca5b270 con 0x55a42cc71bd0 2023-10-06T10:41:59.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.258+0000 7f3898826700 1 --2- 172.21.15.92:0/3190253780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42cc71bd0 0x55a42cc70ac0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f388c009fd0 tx=0x7f388c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c6c11e9647bb76 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.258+0000 7f38797fa700 1 -- 172.21.15.92:0/3190253780 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c0032f0 con 0x55a42cc71bd0 2023-10-06T10:41:59.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.258+0000 7f38797fa700 1 -- 172.21.15.92:0/3190253780 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388c007200 con 0x55a42cc71bd0 2023-10-06T10:41:59.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.258+0000 7f38797fa700 1 -- 172.21.15.92:0/3190253780 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c00bdd0 con 0x55a42cc71bd0 2023-10-06T10:41:59.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.259+0000 7f38a3581b80 1 -- 172.21.15.92:0/3190253780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42cc71bd0 msgr2=0x55a42cc70ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.259+0000 7f38a3581b80 1 --2- 172.21.15.92:0/3190253780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42cc71bd0 0x55a42cc70ac0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f388c009fd0 tx=0x7f388c018000 comp rx=0 tx=0).stop 2023-10-06T10:41:59.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.259+0000 7f38a3581b80 1 -- 172.21.15.92:0/3190253780 shutdown_connections 2023-10-06T10:41:59.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.259+0000 7f38a3581b80 1 --2- 172.21.15.92:0/3190253780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42cc71bd0 0x55a42cc70ac0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.259+0000 7f38a3581b80 1 --2- 172.21.15.92:0/3190253780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a42ccb7a90 0x55a42ccb7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.259+0000 7f38a3581b80 1 --2- 172.21.15.92:0/3190253780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a42cc739a0 0x55a42cc70580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.259+0000 7f38a3581b80 1 -- 172.21.15.92:0/3190253780 >> 172.21.15.92:0/3190253780 conn(0x55a42cc72070 msgr2=0x55a42cc72440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:59.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.259+0000 7f38a3581b80 1 -- 172.21.15.92:0/3190253780 shutdown_connections 2023-10-06T10:41:59.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.259+0000 7f38a3581b80 1 -- 172.21.15.92:0/3190253780 wait complete. 2023-10-06T10:41:59.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.260+0000 7f38a3581b80 1 -- start start 2023-10-06T10:41:59.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.260+0000 7f38a3581b80 1 Processor -- start 2023-10-06T10:41:59.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.260+0000 7f38a3581b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42cc71bd0 0x55a42ccb28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.260+0000 7f38a3581b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a42cc739a0 0x55a42cc9b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f38a3581b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a42ccb7a90 0x55a42cc990b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f38a3581b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42cbc1060 con 0x55a42cc71bd0 2023-10-06T10:41:59.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f38a3581b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42cb6c890 con 0x55a42cc739a0 2023-10-06T10:41:59.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f38a3581b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a42cb87ea0 con 0x55a42ccb7a90 2023-10-06T10:41:59.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f38a3581b80 10 client.0 _ll_register_callbacks cb 0x55a42ca5df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:59.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f3893fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42cc71bd0 0x55a42ccb28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f3893fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42cc71bd0 0x55a42ccb28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58616/0 (socket says 172.21.15.92:58616) 2023-10-06T10:41:59.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f3893fff700 1 -- 172.21.15.92:0/2691997004 learned_addr learned my addr 172.21.15.92:0/2691997004 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:59.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f38937fe700 1 --2- 172.21.15.92:0/2691997004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a42cc739a0 0x55a42cc9b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f3898826700 1 --2- 172.21.15.92:0/2691997004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a42ccb7a90 0x55a42cc990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f3893fff700 1 -- 172.21.15.92:0/2691997004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a42ccb7a90 msgr2=0x55a42cc990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f3893fff700 1 --2- 172.21.15.92:0/2691997004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a42ccb7a90 0x55a42cc990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f3893fff700 1 -- 172.21.15.92:0/2691997004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a42cc739a0 msgr2=0x55a42cc9b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f3893fff700 1 --2- 172.21.15.92:0/2691997004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a42cc739a0 0x55a42cc9b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f3893fff700 1 -- 172.21.15.92:0/2691997004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38940b1500 con 0x55a42cc71bd0 2023-10-06T10:41:59.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f38937fe700 1 --2- 172.21.15.92:0/2691997004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a42cc739a0 0x55a42cc9b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:59.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f3893fff700 1 --2- 172.21.15.92:0/2691997004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42cc71bd0 0x55a42ccb28e0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f388400d860 tx=0x7f3884058040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367fd3efaa67ce94 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f38727fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:59.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f38727fc700 1 -- 172.21.15.92:0/2691997004 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3884071270 con 0x55a42cc71bd0 2023-10-06T10:41:59.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f38727fc700 1 -- 172.21.15.92:0/2691997004 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3884012c50 con 0x55a42cc71bd0 2023-10-06T10:41:59.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f38a3581b80 1 -- 172.21.15.92:0/2691997004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a42cc65690 con 0x55a42cc71bd0 2023-10-06T10:41:59.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f38727fc700 1 -- 172.21.15.92:0/2691997004 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3884055860 con 0x55a42cc71bd0 2023-10-06T10:41:59.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f38a3581b80 10 client.6693 fetch_fsmap learned FSMap version 55 2023-10-06T10:41:59.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f38a3581b80 1 -- 172.21.15.92:0/2691997004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x55a42c9fdfe0 con 0x55a42cc71bd0 2023-10-06T10:41:59.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f38727fc700 1 -- 172.21.15.92:0/2691997004 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a42cc65690 con 0x55a42cc71bd0 2023-10-06T10:41:59.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f38727fc700 1 -- 172.21.15.92:0/2691997004 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 55) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f388407fd20 con 0x55a42cc71bd0 2023-10-06T10:41:59.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.261+0000 7f38a3581b80 10 client.6693 fetch_fsmap finished waiting for FSMap version 55 2023-10-06T10:41:59.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.262+0000 7f38a3581b80 10 client.6693 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T10:41:59.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.262+0000 7f38a3581b80 20 client.6693 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:59.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.262+0000 7f38a3581b80 1 --2- 172.21.15.92:0/2691997004 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x55a42cc9d100 0x55a42ccb3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.262+0000 7f38a3581b80 4 client.6693 mds_command: new command op to 6019 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:41:59.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.262+0000 7f38a3581b80 1 -- 172.21.15.92:0/2691997004 --> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a42cc425c0 con 0x55a42cc9d100 2023-10-06T10:41:59.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.262+0000 7f38937fe700 1 --2- 172.21.15.92:0/2691997004 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x55a42cc9d100 0x55a42ccb3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.262+0000 7f38937fe700 1 --2- 172.21.15.92:0/2691997004 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x55a42cc9d100 0x55a42ccb3450 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49e6d58b7c016f3a server_cookie=e1fc5ae7f57a775b in_seq=0 out_seq=0 2023-10-06T10:41:59.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.262+0000 7f38727fc700 10 client.6693 ms_handle_connect on v2:172.21.15.124:6833/2297612118 2023-10-06T10:41:59.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.263+0000 7f38727fc700 1 -- 172.21.15.92:0/2691997004 <== mds.2 v2:172.21.15.124:6833/2297612118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a42cc425c0 con 0x55a42cc9d100 2023-10-06T10:41:59.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.263+0000 7f38727fc700 10 client.6693 handle_command_reply: tid=0 2023-10-06T10:41:59.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.263+0000 7f38a3581b80 1 client.6693 shutdown 2023-10-06T10:41:59.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.263+0000 7f38a3581b80 1 -- 172.21.15.92:0/2691997004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42cc71bd0 msgr2=0x55a42ccb28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.263+0000 7f38a3581b80 1 --2- 172.21.15.92:0/2691997004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42cc71bd0 0x55a42ccb28e0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f388400d860 tx=0x7f3884058040 comp rx=0 tx=0).stop 2023-10-06T10:41:59.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.263+0000 7f38a3581b80 1 -- 172.21.15.92:0/2691997004 shutdown_connections 2023-10-06T10:41:59.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.263+0000 7f38a3581b80 1 --2- 172.21.15.92:0/2691997004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a42ccb7a90 0x55a42cc990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.263+0000 7f38a3581b80 1 --2- 172.21.15.92:0/2691997004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42cc71bd0 0x55a42ccb28e0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.263+0000 7f38a3581b80 1 --2- 172.21.15.92:0/2691997004 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x55a42cc9d100 0x55a42ccb3450 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.263+0000 7f38a3581b80 1 --2- 172.21.15.92:0/2691997004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a42cc739a0 0x55a42cc9b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.263+0000 7f38a3581b80 1 -- 172.21.15.92:0/2691997004 >> 172.21.15.92:0/2691997004 conn(0x55a42cc72070 msgr2=0x55a42ccb5560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:59.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.263+0000 7f38727fc700 0 client.6693 ms_handle_reset on v2:172.21.15.124:6833/2297612118 2023-10-06T10:41:59.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.263+0000 7f38a3581b80 1 -- 172.21.15.92:0/2691997004 shutdown_connections 2023-10-06T10:41:59.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.263+0000 7f38a3581b80 1 -- 172.21.15.92:0/2691997004 wait complete. 2023-10-06T10:41:59.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.263+0000 7f38a3581b80 20 client.6693 trim_cache size 0 max 16384 2023-10-06T10:41:59.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.280+0000 7f38a3581b80 1 Processor -- start 2023-10-06T10:41:59.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.280+0000 7f38a3581b80 1 -- start start 2023-10-06T10:41:59.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.281+0000 7f38a3581b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a42cc71bd0 0x55a42cc996f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.281+0000 7f38a3581b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a42cc739a0 0x55a42cd165a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.281+0000 7f38a3581b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42ccb7a90 0x55a42cc6d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.281+0000 7f38a3581b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42ca43e30 con 0x55a42ccb7a90 2023-10-06T10:41:59.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.281+0000 7f38a3581b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42ccb5a80 con 0x55a42cc739a0 2023-10-06T10:41:59.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.281+0000 7f38a3581b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a42ccb5100 con 0x55a42cc71bd0 2023-10-06T10:41:59.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.281+0000 7f3898826700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42ccb7a90 0x55a42cc6d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.281+0000 7f3898826700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42ccb7a90 0x55a42cc6d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58618/0 (socket says 172.21.15.92:58618) 2023-10-06T10:41:59.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.281+0000 7f3898826700 1 -- 172.21.15.92:0/475462155 learned_addr learned my addr 172.21.15.92:0/475462155 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:59.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.281+0000 7f3893fff700 1 --2- 172.21.15.92:0/475462155 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a42cc71bd0 0x55a42cc996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.281+0000 7f38937fe700 1 --2- 172.21.15.92:0/475462155 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a42cc739a0 0x55a42cd165a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.281+0000 7f3898826700 1 -- 172.21.15.92:0/475462155 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a42cc71bd0 msgr2=0x55a42cc996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.281+0000 7f3898826700 1 --2- 172.21.15.92:0/475462155 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a42cc71bd0 0x55a42cc996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.281+0000 7f3898826700 1 -- 172.21.15.92:0/475462155 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a42cc739a0 msgr2=0x55a42cd165a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.281+0000 7f3898826700 1 --2- 172.21.15.92:0/475462155 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a42cc739a0 0x55a42cd165a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.281+0000 7f3898826700 1 -- 172.21.15.92:0/475462155 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a42cc9a260 con 0x55a42ccb7a90 2023-10-06T10:41:59.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.281+0000 7f3893fff700 1 --2- 172.21.15.92:0/475462155 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a42cc71bd0 0x55a42cc996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:59.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.281+0000 7f3898826700 1 --2- 172.21.15.92:0/475462155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42ccb7a90 0x55a42cc6d9d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f388c009450 tx=0x7f388c020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c0dd727b12f928 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.281+0000 7f3879ffb700 1 -- 172.21.15.92:0/475462155 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c02c070 con 0x55a42ccb7a90 2023-10-06T10:41:59.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.281+0000 7f3879ffb700 1 -- 172.21.15.92:0/475462155 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388c0032f0 con 0x55a42ccb7a90 2023-10-06T10:41:59.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.281+0000 7f3879ffb700 1 -- 172.21.15.92:0/475462155 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c00bbc0 con 0x55a42ccb7a90 2023-10-06T10:41:59.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.282+0000 7f38a3581b80 1 -- 172.21.15.92:0/475462155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42ccb7a90 msgr2=0x55a42cc6d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.282+0000 7f38a3581b80 1 --2- 172.21.15.92:0/475462155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42ccb7a90 0x55a42cc6d9d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f388c009450 tx=0x7f388c020070 comp rx=0 tx=0).stop 2023-10-06T10:41:59.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.282+0000 7f38a3581b80 1 -- 172.21.15.92:0/475462155 shutdown_connections 2023-10-06T10:41:59.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.282+0000 7f38a3581b80 1 --2- 172.21.15.92:0/475462155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42ccb7a90 0x55a42cc6d9d0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.282+0000 7f38a3581b80 1 --2- 172.21.15.92:0/475462155 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a42cc71bd0 0x55a42cc996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.282+0000 7f38a3581b80 1 --2- 172.21.15.92:0/475462155 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a42cc739a0 0x55a42cd165a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.282+0000 7f38a3581b80 1 -- 172.21.15.92:0/475462155 >> 172.21.15.92:0/475462155 conn(0x55a42cc72070 msgr2=0x55a42cc72440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:59.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.282+0000 7f38a3581b80 1 -- 172.21.15.92:0/475462155 shutdown_connections 2023-10-06T10:41:59.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.282+0000 7f38a3581b80 1 -- 172.21.15.92:0/475462155 wait complete. 2023-10-06T10:41:59.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.282+0000 7f38a3581b80 1 -- start start 2023-10-06T10:41:59.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.283+0000 7f38a3581b80 1 Processor -- start 2023-10-06T10:41:59.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.283+0000 7f38a3581b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a42cc71bd0 0x55a42cd15a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.283+0000 7f38a3581b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42cc739a0 0x55a42ccb44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.283+0000 7f38a3581b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a42ccb7a90 0x55a42ccb49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.283+0000 7f38a3581b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42cd1de20 con 0x55a42cc739a0 2023-10-06T10:41:59.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.283+0000 7f38a3581b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42ccd00c0 con 0x55a42cc71bd0 2023-10-06T10:41:59.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.283+0000 7f38a3581b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a42ccd0300 con 0x55a42ccb7a90 2023-10-06T10:41:59.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.283+0000 7f38a3581b80 10 client.0 _ll_register_callbacks cb 0x55a42ca5df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:59.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.283+0000 7f38937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42cc739a0 0x55a42ccb44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42cc739a0 0x55a42ccb44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58634/0 (socket says 172.21.15.92:58634) 2023-10-06T10:41:59.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38937fe700 1 -- 172.21.15.92:0/663806919 learned_addr learned my addr 172.21.15.92:0/663806919 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:59.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f3898826700 1 --2- 172.21.15.92:0/663806919 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a42ccb7a90 0x55a42ccb49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f3893fff700 1 --2- 172.21.15.92:0/663806919 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a42cc71bd0 0x55a42cd15a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38937fe700 1 -- 172.21.15.92:0/663806919 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a42ccb7a90 msgr2=0x55a42ccb49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38937fe700 1 --2- 172.21.15.92:0/663806919 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a42ccb7a90 0x55a42ccb49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38937fe700 1 -- 172.21.15.92:0/663806919 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a42cc71bd0 msgr2=0x55a42cd15a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38937fe700 1 --2- 172.21.15.92:0/663806919 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a42cc71bd0 0x55a42cd15a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38937fe700 1 -- 172.21.15.92:0/663806919 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a42ca5b270 con 0x55a42cc739a0 2023-10-06T10:41:59.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f3893fff700 1 --2- 172.21.15.92:0/663806919 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a42cc71bd0 0x55a42cd15a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:59.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38937fe700 1 --2- 172.21.15.92:0/663806919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42cc739a0 0x55a42ccb44a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f38940af710 tx=0x7f3888019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b0c95ee429423f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38727fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:59.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38727fc700 1 -- 172.21.15.92:0/663806919 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3888016ea0 con 0x55a42cc739a0 2023-10-06T10:41:59.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38727fc700 1 -- 172.21.15.92:0/663806919 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f388802d680 con 0x55a42cc739a0 2023-10-06T10:41:59.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38a3581b80 1 -- 172.21.15.92:0/663806919 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a42cb63e00 con 0x55a42cc739a0 2023-10-06T10:41:59.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38727fc700 1 -- 172.21.15.92:0/663806919 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f388802da50 con 0x55a42cc739a0 2023-10-06T10:41:59.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38a3581b80 10 client.6699 fetch_fsmap learned FSMap version 55 2023-10-06T10:41:59.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38a3581b80 1 -- 172.21.15.92:0/663806919 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x55a42cb8f780 con 0x55a42cc739a0 2023-10-06T10:41:59.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38727fc700 1 -- 172.21.15.92:0/663806919 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a42cb63e00 con 0x55a42cc739a0 2023-10-06T10:41:59.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38727fc700 1 -- 172.21.15.92:0/663806919 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 55) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f388802dbb0 con 0x55a42cc739a0 2023-10-06T10:41:59.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38a3581b80 10 client.6699 fetch_fsmap finished waiting for FSMap version 55 2023-10-06T10:41:59.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38a3581b80 10 client.6699 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T10:41:59.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38a3581b80 20 client.6699 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:59.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38a3581b80 1 --2- 172.21.15.92:0/663806919 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x55a42cc6ba10 0x55a42cc6bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38a3581b80 4 client.6699 mds_command: new command op to 6019 tid=0[{"prefix": "status"}] 2023-10-06T10:41:59.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.284+0000 7f38a3581b80 1 -- 172.21.15.92:0/663806919 --> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a42cb63b20 con 0x55a42cc6ba10 2023-10-06T10:41:59.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.285+0000 7f3893fff700 1 --2- 172.21.15.92:0/663806919 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x55a42cc6ba10 0x55a42cc6bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.285+0000 7f3893fff700 1 --2- 172.21.15.92:0/663806919 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x55a42cc6ba10 0x55a42cc6bde0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f35a69679c18cab server_cookie=a62cdb06fd0e9cab in_seq=0 out_seq=0 2023-10-06T10:41:59.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.285+0000 7f38727fc700 10 client.6699 ms_handle_connect on v2:172.21.15.124:6833/2297612118 2023-10-06T10:41:59.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.285+0000 7f38727fc700 1 -- 172.21.15.92:0/663806919 <== mds.2 v2:172.21.15.124:6833/2297612118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55a42cb63b20 con 0x55a42cc6ba10 2023-10-06T10:41:59.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.285+0000 7f38727fc700 10 client.6699 handle_command_reply: tid=0 2023-10-06T10:41:59.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.285+0000 7f38a3581b80 1 client.6699 shutdown 2023-10-06T10:41:59.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.285+0000 7f38a3581b80 1 -- 172.21.15.92:0/663806919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42cc739a0 msgr2=0x55a42ccb44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.285+0000 7f38a3581b80 1 --2- 172.21.15.92:0/663806919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42cc739a0 0x55a42ccb44a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f38940af710 tx=0x7f3888019040 comp rx=0 tx=0).stop 2023-10-06T10:41:59.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.285+0000 7f38a3581b80 1 -- 172.21.15.92:0/663806919 shutdown_connections 2023-10-06T10:41:59.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.285+0000 7f38a3581b80 1 --2- 172.21.15.92:0/663806919 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a42ccb7a90 0x55a42ccb49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.285+0000 7f38a3581b80 1 --2- 172.21.15.92:0/663806919 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x55a42cc6ba10 0x55a42cc6bde0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.285+0000 7f38a3581b80 1 --2- 172.21.15.92:0/663806919 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a42cc71bd0 0x55a42cd15a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.285+0000 7f38727fc700 0 client.6699 ms_handle_reset on v2:172.21.15.124:6833/2297612118 2023-10-06T10:41:59.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.285+0000 7f38a3581b80 1 --2- 172.21.15.92:0/663806919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a42cc739a0 0x55a42ccb44a0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.285+0000 7f38a3581b80 1 -- 172.21.15.92:0/663806919 >> 172.21.15.92:0/663806919 conn(0x55a42cc72070 msgr2=0x55a42ccb6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:59.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.285+0000 7f38a3581b80 1 -- 172.21.15.92:0/663806919 shutdown_connections 2023-10-06T10:41:59.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.285+0000 7f38a3581b80 1 -- 172.21.15.92:0/663806919 wait complete. 2023-10-06T10:41:59.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.286+0000 7f38a3581b80 20 client.6699 trim_cache size 0 max 16384 2023-10-06T10:41:59.298 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:41:59.298 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:41:59.299 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 2, 2023-10-06T10:41:59.299 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 6019, 2023-10-06T10:41:59.299 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:41:59.299 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:41:59.299 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:41:59.299 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 1.710004614, 2023-10-06T10:41:59.299 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 55, 2023-10-06T10:41:59.299 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 64, 2023-10-06T10:41:59.299 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 64, 2023-10-06T10:41:59.299 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 16.865190046999999 2023-10-06T10:41:59.300 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:41:59.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.286+0000 7f389a289700 1 -- 172.21.15.92:0/2776345995 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3880066f30 msgr2=0x7f38800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.286+0000 7f389a289700 1 --2- 172.21.15.92:0/2776345995 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3880066f30 0x7f38800693e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f38940cc1b0 tx=0x7f3888008040 comp rx=0 tx=0).stop 2023-10-06T10:41:59.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.286+0000 7f389a289700 1 -- 172.21.15.92:0/2776345995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38940b0760 msgr2=0x7f389415dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.286+0000 7f389a289700 1 --2- 172.21.15.92:0/2776345995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38940b0760 0x7f389415dc70 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f388400d890 tx=0x7f3884008040 comp rx=0 tx=0).stop 2023-10-06T10:41:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.286+0000 7f389a289700 1 -- 172.21.15.92:0/2776345995 shutdown_connections 2023-10-06T10:41:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.286+0000 7f389a289700 1 --2- 172.21.15.92:0/2776345995 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3880066f30 0x7f38800693e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.286+0000 7f389a289700 1 --2- 172.21.15.92:0/2776345995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3894159560 0x7f389409a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.286+0000 7f389a289700 1 --2- 172.21.15.92:0/2776345995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38940b0760 0x7f389415dc70 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.286+0000 7f389a289700 1 --2- 172.21.15.92:0/2776345995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38940b49a0 0x7f38941566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.286+0000 7f389a289700 1 -- 172.21.15.92:0/2776345995 >> 172.21.15.92:0/2776345995 conn(0x7f38940997f0 msgr2=0x7f38940b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.286+0000 7f389a289700 1 -- 172.21.15.92:0/2776345995 shutdown_connections 2023-10-06T10:41:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.286+0000 7f389a289700 1 -- 172.21.15.92:0/2776345995 wait complete. 2023-10-06T10:41:59.303 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T10:41:59.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.416+0000 7f1fba8c0700 1 Processor -- start 2023-10-06T10:41:59.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.416+0000 7f1fba8c0700 1 -- start start 2023-10-06T10:41:59.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.417+0000 7f1fba8c0700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1fb40ca370 0x7f1fb4158270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.417+0000 7f1fba8c0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb4157de0 con 0x7f1fb40adb60 2023-10-06T10:41:59.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.417+0000 7f1fba8c0700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb40cb5d0 con 0x7f1fb40ce8a0 2023-10-06T10:41:59.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.417+0000 7f1fba8c0700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb40cbb90 con 0x7f1fb40ca370 2023-10-06T10:41:59.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.417+0000 7f1fb37fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1fb40adb60 0x7f1fb40ce190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54804/0 (socket says 172.21.15.92:54804) 2023-10-06T10:41:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.417+0000 7f1fb37fe700 1 -- 172.21.15.92:0/3054617335 learned_addr learned my addr 172.21.15.92:0/3054617335 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb2ffd700 1 -- 172.21.15.92:0/3054617335 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 590838953 0 0) 0x7f1fb4157de0 con 0x7f1fb40adb60 2023-10-06T10:41:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb3fff700 1 --2- 172.21.15.92:0/3054617335 >> v2:172.21.15.124:3300/0 conn(0x7f1fb40ca370 0x7f1fb4158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb2ffd700 1 -- 172.21.15.92:0/3054617335 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f9c003570 con 0x7f1fb40adb60 2023-10-06T10:41:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb2ffd700 1 -- 172.21.15.92:0/3054617335 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3704713282 0 0) 0x7f1f9c003570 con 0x7f1fb40adb60 2023-10-06T10:41:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb2ffd700 1 -- 172.21.15.92:0/3054617335 >> v2:172.21.15.124:3300/0 conn(0x7f1fb40ca370 msgr2=0x7f1fb4158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb2ffd700 1 --2- 172.21.15.92:0/3054617335 >> v2:172.21.15.124:3300/0 conn(0x7f1fb40ca370 0x7f1fb4158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb2ffd700 1 -- 172.21.15.92:0/3054617335 >> v1:172.21.15.124:6789/0 conn(0x7f1fb40ce8a0 legacy=0x7f1fb415aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb2ffd700 1 -- 172.21.15.92:0/3054617335 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb40cb8b0 con 0x7f1fb40adb60 2023-10-06T10:41:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb2ffd700 1 -- 172.21.15.92:0/3054617335 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f1fa8001470 con 0x7f1fb40adb60 2023-10-06T10:41:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb2ffd700 1 -- 172.21.15.92:0/3054617335 >> v1:172.21.15.92:6789/0 conn(0x7f1fb40adb60 legacy=0x7f1fb40ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb2ffd700 1 --2- 172.21.15.92:0/3054617335 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f9c003a60 0x7f1f9c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb2ffd700 1 --2- 172.21.15.92:0/3054617335 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f9c004810 0x7f1f9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb2ffd700 1 --2- 172.21.15.92:0/3054617335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f9c0051c0 0x7f1f9c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb2ffd700 1 -- 172.21.15.92:0/3054617335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb40cbb90 con 0x7f1f9c0051c0 2023-10-06T10:41:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb2ffd700 1 -- 172.21.15.92:0/3054617335 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c00c0d0 con 0x7f1f9c003a60 2023-10-06T10:41:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb2ffd700 1 -- 172.21.15.92:0/3054617335 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c00c290 con 0x7f1f9c004810 2023-10-06T10:41:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb8e5d700 1 --2- 172.21.15.92:0/3054617335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f9c0051c0 0x7f1f9c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb3fff700 1 --2- 172.21.15.92:0/3054617335 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f9c003a60 0x7f1f9c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb37fe700 1 --2- 172.21.15.92:0/3054617335 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f9c004810 0x7f1f9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb8e5d700 1 -- 172.21.15.92:0/3054617335 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f9c004810 msgr2=0x7f1f9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb8e5d700 1 --2- 172.21.15.92:0/3054617335 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f9c004810 0x7f1f9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb8e5d700 1 -- 172.21.15.92:0/3054617335 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f9c003a60 msgr2=0x7f1f9c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb8e5d700 1 --2- 172.21.15.92:0/3054617335 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f9c003a60 0x7f1f9c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb8e5d700 1 -- 172.21.15.92:0/3054617335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c003570 con 0x7f1f9c0051c0 2023-10-06T10:41:59.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb8e5d700 1 -- 172.21.15.92:0/3054617335 reap_dead start 2023-10-06T10:41:59.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.418+0000 7f1fb8e5d700 1 --2- 172.21.15.92:0/3054617335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f9c0051c0 0x7f1f9c00ba60 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f1fac001f90 tx=0x7f1fac001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14222a994dad332d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.419+0000 7f1fb2ffd700 1 -- 172.21.15.92:0/3054617335 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac001950 con 0x7f1f9c0051c0 2023-10-06T10:41:59.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.419+0000 7f1fb2ffd700 1 -- 172.21.15.92:0/3054617335 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fac00c8b0 con 0x7f1f9c0051c0 2023-10-06T10:41:59.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.419+0000 7f1fb2ffd700 1 -- 172.21.15.92:0/3054617335 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac00cc50 con 0x7f1f9c0051c0 2023-10-06T10:41:59.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.419+0000 7f1fba8c0700 1 -- 172.21.15.92:0/3054617335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f9c0051c0 msgr2=0x7f1f9c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.419+0000 7f1fba8c0700 1 --2- 172.21.15.92:0/3054617335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f9c0051c0 0x7f1f9c00ba60 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f1fac001f90 tx=0x7f1fac001640 comp rx=0 tx=0).stop 2023-10-06T10:41:59.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.419+0000 7f1fba8c0700 1 -- 172.21.15.92:0/3054617335 shutdown_connections 2023-10-06T10:41:59.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.419+0000 7f1fba8c0700 1 --2- 172.21.15.92:0/3054617335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f9c0051c0 0x7f1f9c00ba60 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.419+0000 7f1fba8c0700 1 -- 172.21.15.92:0/3054617335 >> 172.21.15.92:0/3054617335 conn(0x7f1fb40707d0 msgr2=0x7f1fb4070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:59.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.419+0000 7f1fba8c0700 1 -- 172.21.15.92:0/3054617335 shutdown_connections 2023-10-06T10:41:59.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.419+0000 7f1fba8c0700 1 -- 172.21.15.92:0/3054617335 wait complete. 2023-10-06T10:41:59.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.420+0000 7f1fba8c0700 1 Processor -- start 2023-10-06T10:41:59.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.420+0000 7f1fba8c0700 1 -- start start 2023-10-06T10:41:59.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.421+0000 7f1fba8c0700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f9c0051c0 0x7f1fb4159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.421+0000 7f1fba8c0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fb4151ad0 0x7f1fb40c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.421+0000 7f1fba8c0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fb415ced0 0x7f1fb415d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.421+0000 7f1fba8c0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb415d880 con 0x7f1fb415ced0 2023-10-06T10:41:59.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.421+0000 7f1fba8c0700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb40686c0 con 0x7f1f9c0051c0 2023-10-06T10:41:59.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.421+0000 7f1fba8c0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb4068c80 con 0x7f1fb4151ad0 2023-10-06T10:41:59.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.421+0000 7f1fb8e5d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fb415ced0 0x7f1fb415d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.421+0000 7f1fb8e5d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fb415ced0 0x7f1fb415d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58652/0 (socket says 172.21.15.92:58652) 2023-10-06T10:41:59.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.421+0000 7f1fb8e5d700 1 -- 172.21.15.92:0/1230051714 learned_addr learned my addr 172.21.15.92:0/1230051714 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:59.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.421+0000 7f1fb3fff700 1 --2- 172.21.15.92:0/1230051714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f9c0051c0 0x7f1fb4159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.421+0000 7f1fb37fe700 1 --2- 172.21.15.92:0/1230051714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fb4151ad0 0x7f1fb40c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.421+0000 7f1fb8e5d700 1 -- 172.21.15.92:0/1230051714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fb4151ad0 msgr2=0x7f1fb40c3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.421+0000 7f1fb8e5d700 1 --2- 172.21.15.92:0/1230051714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fb4151ad0 0x7f1fb40c3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.421+0000 7f1fb8e5d700 1 -- 172.21.15.92:0/1230051714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f9c0051c0 msgr2=0x7f1fb4159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.421+0000 7f1fb8e5d700 1 --2- 172.21.15.92:0/1230051714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f9c0051c0 0x7f1fb4159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.421+0000 7f1fb8e5d700 1 -- 172.21.15.92:0/1230051714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb4151f40 con 0x7f1fb415ced0 2023-10-06T10:41:59.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.421+0000 7f1fb3fff700 1 --2- 172.21.15.92:0/1230051714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f9c0051c0 0x7f1fb4159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:59.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.422+0000 7f1fb8e5d700 1 --2- 172.21.15.92:0/1230051714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fb415ced0 0x7f1fb415d340 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00c6a0 tx=0x7f1fac015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.422+0000 7f1f9bfff700 1 -- 172.21.15.92:0/1230051714 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac00c6d0 con 0x7f1fb415ced0 2023-10-06T10:41:59.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.422+0000 7f1f9bfff700 1 -- 172.21.15.92:0/1230051714 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fac00cc30 con 0x7f1fb415ced0 2023-10-06T10:41:59.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.422+0000 7f1fba8c0700 1 -- 172.21.15.92:0/1230051714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb40c4690 con 0x7f1fb415ced0 2023-10-06T10:41:59.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.422+0000 7f1f9bfff700 1 -- 172.21.15.92:0/1230051714 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac006db0 con 0x7f1fb415ced0 2023-10-06T10:41:59.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.422+0000 7f1fba8c0700 1 -- 172.21.15.92:0/1230051714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb4068910 con 0x7f1fb415ced0 2023-10-06T10:41:59.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.423+0000 7f1f9bfff700 1 -- 172.21.15.92:0/1230051714 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1fac0263b0 con 0x7f1fb415ced0 2023-10-06T10:41:59.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.423+0000 7f1f9bfff700 1 --2- 172.21.15.92:0/1230051714 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1fa0066f30 0x7f1fa00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.424+0000 7f1f9bfff700 1 -- 172.21.15.92:0/1230051714 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1fac085410 con 0x7f1fb415ced0 2023-10-06T10:41:59.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.424+0000 7f1fb3fff700 1 --2- 172.21.15.92:0/1230051714 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1fa0066f30 0x7f1fa00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.424+0000 7f1fb3fff700 1 --2- 172.21.15.92:0/1230051714 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1fa0066f30 0x7f1fa00693e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4006f00 tx=0x7f1fa4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.425+0000 7f1fc3bb8b80 1 Processor -- start 2023-10-06T10:41:59.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.425+0000 7f1fc3bb8b80 1 -- start start 2023-10-06T10:41:59.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.426+0000 7f1fc3bb8b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558f0efc6a90 0x558f0efc6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.426+0000 7f1fc3bb8b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558f0ef829a0 0x558f0ef7f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.426+0000 7f1fc3bb8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0ef80bd0 0x558f0ef7fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.426+0000 7f1fc3bb8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0eeffc80 con 0x558f0ef80bd0 2023-10-06T10:41:59.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.426+0000 7f1fc3bb8b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0eed6a70 con 0x558f0efc6a90 2023-10-06T10:41:59.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.426+0000 7f1fc3bb8b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f0eed4270 con 0x558f0ef829a0 2023-10-06T10:41:59.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.426+0000 7f1fb8e5d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0ef80bd0 0x558f0ef7fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.426+0000 7f1fb8e5d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0ef80bd0 0x558f0ef7fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58662/0 (socket says 172.21.15.92:58662) 2023-10-06T10:41:59.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.426+0000 7f1fb8e5d700 1 -- 172.21.15.92:0/2808088275 learned_addr learned my addr 172.21.15.92:0/2808088275 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:59.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.427+0000 7f1fb37fe700 1 --2- 172.21.15.92:0/2808088275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558f0ef829a0 0x558f0ef7f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.427+0000 7f1fb3fff700 1 --2- 172.21.15.92:0/2808088275 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558f0efc6a90 0x558f0efc6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.427+0000 7f1fb8e5d700 1 -- 172.21.15.92:0/2808088275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558f0ef829a0 msgr2=0x558f0ef7f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.427+0000 7f1fb8e5d700 1 --2- 172.21.15.92:0/2808088275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558f0ef829a0 0x558f0ef7f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.427+0000 7f1fb8e5d700 1 -- 172.21.15.92:0/2808088275 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558f0efc6a90 msgr2=0x558f0efc6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.427+0000 7f1fb8e5d700 1 --2- 172.21.15.92:0/2808088275 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558f0efc6a90 0x558f0efc6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.427+0000 7f1fb8e5d700 1 -- 172.21.15.92:0/2808088275 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f0ed6a270 con 0x558f0ef80bd0 2023-10-06T10:41:59.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.427+0000 7f1fb37fe700 1 --2- 172.21.15.92:0/2808088275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558f0ef829a0 0x558f0ef7f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:59.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.427+0000 7f1fb3fff700 1 --2- 172.21.15.92:0/2808088275 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558f0efc6a90 0x558f0efc6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:59.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.427+0000 7f1fb8e5d700 1 --2- 172.21.15.92:0/2808088275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0ef80bd0 0x558f0ef7fac0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0836c0 tx=0x7f1fac0835f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449ea137025b0be2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.427+0000 7f1f997fa700 1 -- 172.21.15.92:0/2808088275 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac0560a0 con 0x558f0ef80bd0 2023-10-06T10:41:59.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.427+0000 7f1f997fa700 1 -- 172.21.15.92:0/2808088275 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fac0592f0 con 0x558f0ef80bd0 2023-10-06T10:41:59.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.427+0000 7f1f997fa700 1 -- 172.21.15.92:0/2808088275 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac05d040 con 0x558f0ef80bd0 2023-10-06T10:41:59.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.428+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/2808088275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0ef80bd0 msgr2=0x558f0ef7fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.428+0000 7f1fc3bb8b80 1 --2- 172.21.15.92:0/2808088275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0ef80bd0 0x558f0ef7fac0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0836c0 tx=0x7f1fac0835f0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.428+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/2808088275 shutdown_connections 2023-10-06T10:41:59.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.428+0000 7f1fc3bb8b80 1 --2- 172.21.15.92:0/2808088275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0ef80bd0 0x558f0ef7fac0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.428+0000 7f1fc3bb8b80 1 --2- 172.21.15.92:0/2808088275 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558f0efc6a90 0x558f0efc6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.428+0000 7f1fc3bb8b80 1 --2- 172.21.15.92:0/2808088275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558f0ef829a0 0x558f0ef7f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.428+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/2808088275 >> 172.21.15.92:0/2808088275 conn(0x558f0ef81070 msgr2=0x558f0ef81440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:59.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.428+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/2808088275 shutdown_connections 2023-10-06T10:41:59.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.428+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/2808088275 wait complete. 2023-10-06T10:41:59.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.428+0000 7f1fc3bb8b80 1 -- start start 2023-10-06T10:41:59.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.428+0000 7f1fc3bb8b80 1 Processor -- start 2023-10-06T10:41:59.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.429+0000 7f1fc3bb8b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558f0ef80bd0 0x558f0efc2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.429+0000 7f1fc3bb8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0ef829a0 0x558f0ef7f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.429+0000 7f1fc3bb8b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558f0efc6a90 0x558f0ef7fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.429+0000 7f1fc3bb8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0eed0060 con 0x558f0ef829a0 2023-10-06T10:41:59.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.429+0000 7f1fc3bb8b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0ee7b890 con 0x558f0efc6a90 2023-10-06T10:41:59.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.429+0000 7f1fc3bb8b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f0ee96ea0 con 0x558f0ef80bd0 2023-10-06T10:41:59.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.429+0000 7f1fc3bb8b80 10 client.0 _ll_register_callbacks cb 0x558f0ed6cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:59.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.429+0000 7f1fb37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0ef829a0 0x558f0ef7f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.429+0000 7f1fb37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0ef829a0 0x558f0ef7f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58672/0 (socket says 172.21.15.92:58672) 2023-10-06T10:41:59.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.429+0000 7f1fb37fe700 1 -- 172.21.15.92:0/1061356887 learned_addr learned my addr 172.21.15.92:0/1061356887 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:59.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.429+0000 7f1fb8e5d700 1 --2- 172.21.15.92:0/1061356887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558f0efc6a90 0x558f0ef7fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.429+0000 7f1fb3fff700 1 --2- 172.21.15.92:0/1061356887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558f0ef80bd0 0x558f0efc2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.429+0000 7f1fb37fe700 1 -- 172.21.15.92:0/1061356887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558f0ef80bd0 msgr2=0x558f0efc2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.429+0000 7f1fb37fe700 1 --2- 172.21.15.92:0/1061356887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558f0ef80bd0 0x558f0efc2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.429+0000 7f1fb37fe700 1 -- 172.21.15.92:0/1061356887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558f0efc6a90 msgr2=0x558f0ef7fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.429+0000 7f1fb37fe700 1 --2- 172.21.15.92:0/1061356887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558f0efc6a90 0x558f0ef7fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.429+0000 7f1fb37fe700 1 -- 172.21.15.92:0/1061356887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb4068910 con 0x558f0ef829a0 2023-10-06T10:41:59.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.429+0000 7f1fb8e5d700 1 --2- 172.21.15.92:0/1061356887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558f0efc6a90 0x558f0ef7fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:59.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.430+0000 7f1fb37fe700 1 --2- 172.21.15.92:0/1061356887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0ef829a0 0x558f0ef7f580 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8000c00 tx=0x7f1fa8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0700dabd765b9e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.430+0000 7f1f927fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:59.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.430+0000 7f1f927fc700 1 -- 172.21.15.92:0/1061356887 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8015690 con 0x558f0ef829a0 2023-10-06T10:41:59.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.430+0000 7f1f927fc700 1 -- 172.21.15.92:0/1061356887 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1fa8017db0 con 0x558f0ef829a0 2023-10-06T10:41:59.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.430+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/1061356887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f0ef74690 con 0x558f0ef829a0 2023-10-06T10:41:59.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.430+0000 7f1f927fc700 1 -- 172.21.15.92:0/1061356887 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fa8015a00 con 0x558f0ef829a0 2023-10-06T10:41:59.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.430+0000 7f1fc3bb8b80 10 client.6711 fetch_fsmap learned FSMap version 55 2023-10-06T10:41:59.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.430+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/1061356887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x558f0ed0cfe0 con 0x558f0ef829a0 2023-10-06T10:41:59.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.430+0000 7f1f927fc700 1 -- 172.21.15.92:0/1061356887 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558f0ef74690 con 0x558f0ef829a0 2023-10-06T10:41:59.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.430+0000 7f1f927fc700 1 -- 172.21.15.92:0/1061356887 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 55) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f1fa801f070 con 0x558f0ef829a0 2023-10-06T10:41:59.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.430+0000 7f1fc3bb8b80 10 client.6711 fetch_fsmap finished waiting for FSMap version 55 2023-10-06T10:41:59.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.430+0000 7f1fc3bb8b80 10 client.6711 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T10:41:59.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.430+0000 7f1fc3bb8b80 20 client.6711 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:59.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.430+0000 7f1fc3bb8b80 1 --2- 172.21.15.92:0/1061356887 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x558f0efabe30 0x558f0efac200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.430+0000 7f1fc3bb8b80 4 client.6711 mds_command: new command op to 6483 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:41:59.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.430+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/1061356887 --> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f0ef515c0 con 0x558f0efabe30 2023-10-06T10:41:59.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.430+0000 7f1fb3fff700 1 --2- 172.21.15.92:0/1061356887 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x558f0efabe30 0x558f0efac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.430+0000 7f1f927fc700 10 client.6711 ms_handle_connect on v2:172.21.15.92:6834/171578176 2023-10-06T10:41:59.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.430+0000 7f1fb3fff700 1 --2- 172.21.15.92:0/1061356887 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x558f0efabe30 0x558f0efac200 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b26b2c1b6a8f96a7 server_cookie=7721fa88d30a97f in_seq=0 out_seq=0 2023-10-06T10:41:59.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.431+0000 7f1f927fc700 1 -- 172.21.15.92:0/1061356887 <== mds.1 v2:172.21.15.92:6834/171578176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x558f0ef515c0 con 0x558f0efabe30 2023-10-06T10:41:59.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.431+0000 7f1f927fc700 10 client.6711 handle_command_reply: tid=0 2023-10-06T10:41:59.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.431+0000 7f1fc3bb8b80 1 client.6711 shutdown 2023-10-06T10:41:59.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.431+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/1061356887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0ef829a0 msgr2=0x558f0ef7f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.431+0000 7f1fc3bb8b80 1 --2- 172.21.15.92:0/1061356887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0ef829a0 0x558f0ef7f580 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8000c00 tx=0x7f1fa8009040 comp rx=0 tx=0).stop 2023-10-06T10:41:59.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.431+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/1061356887 shutdown_connections 2023-10-06T10:41:59.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.431+0000 7f1fc3bb8b80 1 --2- 172.21.15.92:0/1061356887 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x558f0efabe30 0x558f0efac200 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.431+0000 7f1fc3bb8b80 1 --2- 172.21.15.92:0/1061356887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558f0efc6a90 0x558f0ef7fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.431+0000 7f1f927fc700 0 client.6711 ms_handle_reset on v2:172.21.15.92:6834/171578176 2023-10-06T10:41:59.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.431+0000 7f1fc3bb8b80 1 --2- 172.21.15.92:0/1061356887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558f0ef80bd0 0x558f0efc2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.431+0000 7f1fc3bb8b80 1 --2- 172.21.15.92:0/1061356887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0ef829a0 0x558f0ef7f580 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.431+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/1061356887 >> 172.21.15.92:0/1061356887 conn(0x558f0ef81070 msgr2=0x558f0efc5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:59.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.431+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/1061356887 shutdown_connections 2023-10-06T10:41:59.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.431+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/1061356887 wait complete. 2023-10-06T10:41:59.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.431+0000 7f1fc3bb8b80 20 client.6711 trim_cache size 0 max 16384 2023-10-06T10:41:59.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.448+0000 7f1fc3bb8b80 1 Processor -- start 2023-10-06T10:41:59.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.448+0000 7f1fc3bb8b80 1 -- start start 2023-10-06T10:41:59.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.449+0000 7f1fc3bb8b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558f0ef80bd0 0x558f0efaf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.449+0000 7f1fc3bb8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0ef829a0 0x558f0ef7b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.449+0000 7f1fc3bb8b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558f0efc6a90 0x558f0ef7f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.449+0000 7f1fc3bb8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0ed52e30 con 0x558f0ef829a0 2023-10-06T10:41:59.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.449+0000 7f1fc3bb8b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0efc44f0 con 0x558f0efc6a90 2023-10-06T10:41:59.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.449+0000 7f1fc3bb8b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f0efadf50 con 0x558f0ef80bd0 2023-10-06T10:41:59.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.449+0000 7f1fb37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0ef829a0 0x558f0ef7b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.449+0000 7f1fb37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0ef829a0 0x558f0ef7b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58686/0 (socket says 172.21.15.92:58686) 2023-10-06T10:41:59.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.449+0000 7f1fb37fe700 1 -- 172.21.15.92:0/2141187553 learned_addr learned my addr 172.21.15.92:0/2141187553 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:59.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.449+0000 7f1fb8e5d700 1 --2- 172.21.15.92:0/2141187553 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558f0efc6a90 0x558f0ef7f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.449+0000 7f1fb3fff700 1 --2- 172.21.15.92:0/2141187553 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558f0ef80bd0 0x558f0efaf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.449+0000 7f1fb37fe700 1 -- 172.21.15.92:0/2141187553 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558f0ef80bd0 msgr2=0x558f0efaf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.449+0000 7f1fb37fe700 1 --2- 172.21.15.92:0/2141187553 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558f0ef80bd0 0x558f0efaf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.449+0000 7f1fb37fe700 1 -- 172.21.15.92:0/2141187553 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558f0efc6a90 msgr2=0x558f0ef7f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.449+0000 7f1fb37fe700 1 --2- 172.21.15.92:0/2141187553 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558f0efc6a90 0x558f0ef7f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.449+0000 7f1fb37fe700 1 -- 172.21.15.92:0/2141187553 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f0ee9e780 con 0x558f0ef829a0 2023-10-06T10:41:59.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.449+0000 7f1fb3fff700 1 --2- 172.21.15.92:0/2141187553 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558f0ef80bd0 0x558f0efaf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:41:59.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.449+0000 7f1fb37fe700 1 --2- 172.21.15.92:0/2141187553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0ef829a0 0x558f0ef7b7d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8000ec0 tx=0x7f1fa800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf4874a75ee0f9d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.450+0000 7f1f99ffb700 1 -- 172.21.15.92:0/2141187553 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8015690 con 0x558f0ef829a0 2023-10-06T10:41:59.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.450+0000 7f1f99ffb700 1 -- 172.21.15.92:0/2141187553 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa80077e0 con 0x558f0ef829a0 2023-10-06T10:41:59.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.450+0000 7f1f99ffb700 1 -- 172.21.15.92:0/2141187553 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa80203a0 con 0x558f0ef829a0 2023-10-06T10:41:59.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.450+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/2141187553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0ef829a0 msgr2=0x558f0ef7b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.450+0000 7f1fc3bb8b80 1 --2- 172.21.15.92:0/2141187553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0ef829a0 0x558f0ef7b7d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8000ec0 tx=0x7f1fa800a000 comp rx=0 tx=0).stop 2023-10-06T10:41:59.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.450+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/2141187553 shutdown_connections 2023-10-06T10:41:59.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.450+0000 7f1fc3bb8b80 1 --2- 172.21.15.92:0/2141187553 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558f0efc6a90 0x558f0ef7f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.450+0000 7f1fc3bb8b80 1 --2- 172.21.15.92:0/2141187553 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558f0ef80bd0 0x558f0efaf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.450+0000 7f1fc3bb8b80 1 --2- 172.21.15.92:0/2141187553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0ef829a0 0x558f0ef7b7d0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.450+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/2141187553 >> 172.21.15.92:0/2141187553 conn(0x558f0ef81070 msgr2=0x558f0ef81440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:59.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.450+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/2141187553 shutdown_connections 2023-10-06T10:41:59.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.450+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/2141187553 wait complete. 2023-10-06T10:41:59.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.451+0000 7f1fc3bb8b80 1 -- start start 2023-10-06T10:41:59.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.451+0000 7f1fc3bb8b80 1 Processor -- start 2023-10-06T10:41:59.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.452+0000 7f1fc3bb8b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558f0ef80bd0 0x558f0f014750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.452+0000 7f1fc3bb8b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558f0ef829a0 0x558f0ef7b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.452+0000 7f1fc3bb8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0efc6a90 0x558f0ef7bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.452+0000 7f1fc3bb8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0efc19f0 con 0x558f0efc6a90 2023-10-06T10:41:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.452+0000 7f1fc3bb8b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0efc0e70 con 0x558f0ef80bd0 2023-10-06T10:41:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.452+0000 7f1fc3bb8b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f0efc10b0 con 0x558f0ef829a0 2023-10-06T10:41:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.452+0000 7f1fc3bb8b80 10 client.0 _ll_register_callbacks cb 0x558f0ed6cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.452+0000 7f1fb8e5d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0efc6a90 0x558f0ef7bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.452+0000 7f1fb8e5d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0efc6a90 0x558f0ef7bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58698/0 (socket says 172.21.15.92:58698) 2023-10-06T10:41:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.452+0000 7f1fb8e5d700 1 -- 172.21.15.92:0/3334830035 learned_addr learned my addr 172.21.15.92:0/3334830035 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.452+0000 7f1fb37fe700 1 --2- 172.21.15.92:0/3334830035 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558f0ef829a0 0x558f0ef7b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.452+0000 7f1fb3fff700 1 --2- 172.21.15.92:0/3334830035 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558f0ef80bd0 0x558f0f014750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.452+0000 7f1fb8e5d700 1 -- 172.21.15.92:0/3334830035 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558f0ef829a0 msgr2=0x558f0ef7b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.452+0000 7f1fb8e5d700 1 --2- 172.21.15.92:0/3334830035 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558f0ef829a0 0x558f0ef7b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.452+0000 7f1fb8e5d700 1 -- 172.21.15.92:0/3334830035 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558f0ef80bd0 msgr2=0x558f0f014750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.452+0000 7f1fb8e5d700 1 --2- 172.21.15.92:0/3334830035 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558f0ef80bd0 0x558f0f014750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.452+0000 7f1fb8e5d700 1 -- 172.21.15.92:0/3334830035 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f0ed0cfe0 con 0x558f0efc6a90 2023-10-06T10:41:59.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.452+0000 7f1fb8e5d700 1 --2- 172.21.15.92:0/3334830035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0efc6a90 0x558f0ef7bd10 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00f040 tx=0x7f1fac069000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8394de8136e6a6cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.452+0000 7f1f927fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:59.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.452+0000 7f1f927fc700 1 -- 172.21.15.92:0/3334830035 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac0523e0 con 0x558f0efc6a90 2023-10-06T10:41:59.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.452+0000 7f1f927fc700 1 -- 172.21.15.92:0/3334830035 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1fac0560a0 con 0x558f0efc6a90 2023-10-06T10:41:59.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.452+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/3334830035 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f0ee72e00 con 0x558f0efc6a90 2023-10-06T10:41:59.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.453+0000 7f1f927fc700 1 -- 172.21.15.92:0/3334830035 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fac05d040 con 0x558f0efc6a90 2023-10-06T10:41:59.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.453+0000 7f1fc3bb8b80 10 client.6717 fetch_fsmap learned FSMap version 55 2023-10-06T10:41:59.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.453+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/3334830035 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x558f0ee85470 con 0x558f0efc6a90 2023-10-06T10:41:59.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.453+0000 7f1f927fc700 1 -- 172.21.15.92:0/3334830035 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558f0ee72e00 con 0x558f0efc6a90 2023-10-06T10:41:59.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.453+0000 7f1f927fc700 1 -- 172.21.15.92:0/3334830035 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 55) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f1fac05cbb0 con 0x558f0efc6a90 2023-10-06T10:41:59.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.453+0000 7f1fc3bb8b80 10 client.6717 fetch_fsmap finished waiting for FSMap version 55 2023-10-06T10:41:59.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.453+0000 7f1fc3bb8b80 10 client.6717 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T10:41:59.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.453+0000 7f1fc3bb8b80 20 client.6717 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:59.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.453+0000 7f1fc3bb8b80 1 --2- 172.21.15.92:0/3334830035 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x558f0efa70b0 0x558f0efa7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.453+0000 7f1fc3bb8b80 4 client.6717 mds_command: new command op to 6483 tid=0[{"prefix": "status"}] 2023-10-06T10:41:59.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.453+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/3334830035 --> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558f0ee72b20 con 0x558f0efa70b0 2023-10-06T10:41:59.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.453+0000 7f1fb3fff700 1 --2- 172.21.15.92:0/3334830035 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x558f0efa70b0 0x558f0efa7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.453+0000 7f1f927fc700 10 client.6717 ms_handle_connect on v2:172.21.15.92:6834/171578176 2023-10-06T10:41:59.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.453+0000 7f1fb3fff700 1 --2- 172.21.15.92:0/3334830035 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x558f0efa70b0 0x558f0efa7480 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=891b4b3a1964c4f2 server_cookie=8471a9327272137f in_seq=0 out_seq=0 2023-10-06T10:41:59.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.453+0000 7f1f927fc700 1 -- 172.21.15.92:0/3334830035 <== mds.1 v2:172.21.15.92:6834/171578176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+323 (crc 0 0 0) 0x558f0ee72b20 con 0x558f0efa70b0 2023-10-06T10:41:59.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.453+0000 7f1f927fc700 10 client.6717 handle_command_reply: tid=0 2023-10-06T10:41:59.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.453+0000 7f1fc3bb8b80 1 client.6717 shutdown 2023-10-06T10:41:59.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.453+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/3334830035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0efc6a90 msgr2=0x558f0ef7bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.453+0000 7f1fc3bb8b80 1 --2- 172.21.15.92:0/3334830035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0efc6a90 0x558f0ef7bd10 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00f040 tx=0x7f1fac069000 comp rx=0 tx=0).stop 2023-10-06T10:41:59.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.454+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/3334830035 shutdown_connections 2023-10-06T10:41:59.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.454+0000 7f1fc3bb8b80 1 --2- 172.21.15.92:0/3334830035 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x558f0efa70b0 0x558f0efa7480 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.454+0000 7f1fc3bb8b80 1 --2- 172.21.15.92:0/3334830035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0efc6a90 0x558f0ef7bd10 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.454+0000 7f1f927fc700 0 client.6717 ms_handle_reset on v2:172.21.15.92:6834/171578176 2023-10-06T10:41:59.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.454+0000 7f1fc3bb8b80 1 --2- 172.21.15.92:0/3334830035 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558f0ef80bd0 0x558f0f014750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.454+0000 7f1fc3bb8b80 1 --2- 172.21.15.92:0/3334830035 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558f0ef829a0 0x558f0ef7b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.454+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/3334830035 >> 172.21.15.92:0/3334830035 conn(0x558f0ef81070 msgr2=0x558f0efaf440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:59.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.454+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/3334830035 shutdown_connections 2023-10-06T10:41:59.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.454+0000 7f1fc3bb8b80 1 -- 172.21.15.92:0/3334830035 wait complete. 2023-10-06T10:41:59.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.454+0000 7f1fc3bb8b80 20 client.6717 trim_cache size 0 max 16384 2023-10-06T10:41:59.463 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:41:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:41:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 1, 2023-10-06T10:41:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 6483, 2023-10-06T10:41:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:41:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:41:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:41:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 2.88531702, 2023-10-06T10:41:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 55, 2023-10-06T10:41:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 64, 2023-10-06T10:41:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 64, 2023-10-06T10:41:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 17.045036719999999 2023-10-06T10:41:59.464 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:41:59.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.454+0000 7f1fba8c0700 1 -- 172.21.15.92:0/1230051714 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1fa0066f30 msgr2=0x7f1fa00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.454+0000 7f1fba8c0700 1 --2- 172.21.15.92:0/1230051714 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1fa0066f30 0x7f1fa00693e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4006f00 tx=0x7f1fa4008040 comp rx=0 tx=0).stop 2023-10-06T10:41:59.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.454+0000 7f1fba8c0700 1 -- 172.21.15.92:0/1230051714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fb415ced0 msgr2=0x7f1fb415d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.454+0000 7f1fba8c0700 1 --2- 172.21.15.92:0/1230051714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fb415ced0 0x7f1fb415d340 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00c6a0 tx=0x7f1fac015000 comp rx=0 tx=0).stop 2023-10-06T10:41:59.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.454+0000 7f1fba8c0700 1 -- 172.21.15.92:0/1230051714 shutdown_connections 2023-10-06T10:41:59.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.454+0000 7f1fba8c0700 1 --2- 172.21.15.92:0/1230051714 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1fa0066f30 0x7f1fa00693e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.454+0000 7f1fba8c0700 1 --2- 172.21.15.92:0/1230051714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fb415ced0 0x7f1fb415d340 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.454+0000 7f1fba8c0700 1 --2- 172.21.15.92:0/1230051714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f9c0051c0 0x7f1fb4159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.454+0000 7f1fba8c0700 1 --2- 172.21.15.92:0/1230051714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fb4151ad0 0x7f1fb40c3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.454+0000 7f1fba8c0700 1 -- 172.21.15.92:0/1230051714 >> 172.21.15.92:0/1230051714 conn(0x7f1fb40707d0 msgr2=0x7f1fb4156150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:59.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.454+0000 7f1fba8c0700 1 -- 172.21.15.92:0/1230051714 shutdown_connections 2023-10-06T10:41:59.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.454+0000 7f1fba8c0700 1 -- 172.21.15.92:0/1230051714 wait complete. 2023-10-06T10:41:59.468 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T10:41:59.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.583+0000 7f24e48a0700 1 Processor -- start 2023-10-06T10:41:59.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.583+0000 7f24e48a0700 1 -- start start 2023-10-06T10:41:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.584+0000 7f24e48a0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f24e00cbc30 0x7f24e00cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.584+0000 7f24e48a0700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f24e00b2100 0x7f24e00b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.584+0000 7f24e48a0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24e00cb7a0 con 0x7f24e0159440 2023-10-06T10:41:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.584+0000 7f24e48a0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e00b0860 con 0x7f24e00cbc30 2023-10-06T10:41:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.584+0000 7f24e48a0700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e00cb150 con 0x7f24e00b2100 2023-10-06T10:41:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.584+0000 7f24de59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f24e00cbc30 0x7f24e00cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.584+0000 7f24de59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f24e00cbc30 0x7f24e00cc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58714/0 (socket says 172.21.15.92:58714) 2023-10-06T10:41:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.584+0000 7f24ded9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f24e0159440 0x7f24e00b3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54814/0 (socket says 172.21.15.92:54814) 2023-10-06T10:41:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.584+0000 7f24de59c700 1 -- 172.21.15.92:0/266571920 learned_addr learned my addr 172.21.15.92:0/266571920 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.584+0000 7f24df59e700 1 --2- 172.21.15.92:0/266571920 >> v2:172.21.15.124:3300/0 conn(0x7f24e00b2100 0x7f24e00b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.584+0000 7f24de59c700 1 -- 172.21.15.92:0/266571920 >> v2:172.21.15.124:3300/0 conn(0x7f24e00b2100 msgr2=0x7f24e00b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.584+0000 7f24de59c700 1 --2- 172.21.15.92:0/266571920 >> v2:172.21.15.124:3300/0 conn(0x7f24e00b2100 0x7f24e00b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.584+0000 7f24de59c700 1 -- 172.21.15.92:0/266571920 >> v1:172.21.15.92:6789/0 conn(0x7f24e0159440 legacy=0x7f24e00b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.584+0000 7f24de59c700 1 -- 172.21.15.92:0/266571920 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e00cb430 con 0x7f24e00cbc30 2023-10-06T10:41:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.584+0000 7f24ddd9b700 1 -- 172.21.15.92:0/266571920 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4099965747 0 0) 0x7f24e00cb7a0 con 0x7f24e0159440 2023-10-06T10:41:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.584+0000 7f24de59c700 1 --2- 172.21.15.92:0/266571920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e00cbc30 0x7f24e00cc080 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f24e00ca7f0 tx=0x7f24d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b72a49919608bde server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.584+0000 7f24ddd9b700 1 -- 172.21.15.92:0/266571920 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d000b860 con 0x7f24e00cbc30 2023-10-06T10:41:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.584+0000 7f24ddd9b700 1 -- 172.21.15.92:0/266571920 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d000b9c0 con 0x7f24e00cbc30 2023-10-06T10:41:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.584+0000 7f24ddd9b700 1 -- 172.21.15.92:0/266571920 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d000bd60 con 0x7f24e00cbc30 2023-10-06T10:41:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.585+0000 7f24e48a0700 1 -- 172.21.15.92:0/266571920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e00cbc30 msgr2=0x7f24e00cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.585+0000 7f24e48a0700 1 --2- 172.21.15.92:0/266571920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e00cbc30 0x7f24e00cc080 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f24e00ca7f0 tx=0x7f24d0009250 comp rx=0 tx=0).stop 2023-10-06T10:41:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.585+0000 7f24e48a0700 1 -- 172.21.15.92:0/266571920 shutdown_connections 2023-10-06T10:41:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.585+0000 7f24e48a0700 1 --2- 172.21.15.92:0/266571920 >> v2:172.21.15.124:3300/0 conn(0x7f24e00b2100 0x7f24e00b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.585+0000 7f24e48a0700 1 --2- 172.21.15.92:0/266571920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e00cbc30 0x7f24e00cc080 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.585+0000 7f24e48a0700 1 -- 172.21.15.92:0/266571920 >> 172.21.15.92:0/266571920 conn(0x7f24e006ece0 msgr2=0x7f24e006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.585+0000 7f24e48a0700 1 -- 172.21.15.92:0/266571920 shutdown_connections 2023-10-06T10:41:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.585+0000 7f24e48a0700 1 -- 172.21.15.92:0/266571920 wait complete. 2023-10-06T10:41:59.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.586+0000 7f24e48a0700 1 Processor -- start 2023-10-06T10:41:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.586+0000 7f24e48a0700 1 -- start start 2023-10-06T10:41:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.587+0000 7f24e48a0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e00b2100 0x7f24e0153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.587+0000 7f24e48a0700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24e00cbc30 0x7f24e00c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.587+0000 7f24e48a0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24e0159440 0x7f24e00c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.587+0000 7f24e48a0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e01541c0 con 0x7f24e00b2100 2023-10-06T10:41:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.587+0000 7f24e48a0700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e00ce440 con 0x7f24e00cbc30 2023-10-06T10:41:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.587+0000 7f24e48a0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e00c9730 con 0x7f24e0159440 2023-10-06T10:41:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.587+0000 7f24ded9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e00b2100 0x7f24e0153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.587+0000 7f24ded9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e00b2100 0x7f24e0153c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58728/0 (socket says 172.21.15.92:58728) 2023-10-06T10:41:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.587+0000 7f24ded9d700 1 -- 172.21.15.92:0/3697369866 learned_addr learned my addr 172.21.15.92:0/3697369866 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.587+0000 7f24ded9d700 1 -- 172.21.15.92:0/3697369866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24e0159440 msgr2=0x7f24e00c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.587+0000 7f24ded9d700 1 --2- 172.21.15.92:0/3697369866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24e0159440 0x7f24e00c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.587+0000 7f24ded9d700 1 -- 172.21.15.92:0/3697369866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24e00cbc30 msgr2=0x7f24e00c72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.587+0000 7f24ded9d700 1 --2- 172.21.15.92:0/3697369866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24e00cbc30 0x7f24e00c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.587+0000 7f24ded9d700 1 -- 172.21.15.92:0/3697369866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d00094f0 con 0x7f24e00b2100 2023-10-06T10:41:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.587+0000 7f24ded9d700 1 --2- 172.21.15.92:0/3697369866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e00b2100 0x7f24e0153c80 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f24d400c5b0 tx=0x7f24d4001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.587+0000 7f24c6ffd700 1 -- 172.21.15.92:0/3697369866 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d4005a70 con 0x7f24e00b2100 2023-10-06T10:41:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.587+0000 7f24c6ffd700 1 -- 172.21.15.92:0/3697369866 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d4005bd0 con 0x7f24e00b2100 2023-10-06T10:41:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.587+0000 7f24e48a0700 1 -- 172.21.15.92:0/3697369866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e00c9a10 con 0x7f24e00b2100 2023-10-06T10:41:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.587+0000 7f24c6ffd700 1 -- 172.21.15.92:0/3697369866 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d40070d0 con 0x7f24e00b2100 2023-10-06T10:41:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.587+0000 7f24e48a0700 1 -- 172.21.15.92:0/3697369866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e00ce0d0 con 0x7f24e00b2100 2023-10-06T10:41:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.589+0000 7f24c6ffd700 1 -- 172.21.15.92:0/3697369866 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f24d4007480 con 0x7f24e00b2100 2023-10-06T10:41:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.589+0000 7f24c6ffd700 1 --2- 172.21.15.92:0/3697369866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f24cc066f30 0x7f24cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.590+0000 7f24c6ffd700 1 -- 172.21.15.92:0/3697369866 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f24d4083cd0 con 0x7f24e00b2100 2023-10-06T10:41:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.590+0000 7f24de59c700 1 --2- 172.21.15.92:0/3697369866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f24cc066f30 0x7f24cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.590+0000 7f24de59c700 1 --2- 172.21.15.92:0/3697369866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f24cc066f30 0x7f24cc0693e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f24e00ca7f0 tx=0x7f24d0017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.591+0000 7f24ee399b80 1 Processor -- start 2023-10-06T10:41:59.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.592+0000 7f24ee399b80 1 -- start start 2023-10-06T10:41:59.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.592+0000 7f24ee399b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca78ef950 0x558ca78efd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.592+0000 7f24ee399b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ca7939bc0 0x558ca7918920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.592+0000 7f24ee399b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ca7918e60 0x558ca78eb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.592+0000 7f24ee399b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca786fc80 con 0x558ca78ef950 2023-10-06T10:41:59.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.592+0000 7f24ee399b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca7846a70 con 0x558ca7939bc0 2023-10-06T10:41:59.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.592+0000 7f24ee399b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ca7844270 con 0x558ca7918e60 2023-10-06T10:41:59.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.592+0000 7f24ded9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca78ef950 0x558ca78efd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.592+0000 7f24ded9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca78ef950 0x558ca78efd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58734/0 (socket says 172.21.15.92:58734) 2023-10-06T10:41:59.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.592+0000 7f24ded9d700 1 -- 172.21.15.92:0/2037576263 learned_addr learned my addr 172.21.15.92:0/2037576263 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:59.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.593+0000 7f24ded9d700 1 -- 172.21.15.92:0/2037576263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ca7918e60 msgr2=0x558ca78eb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.593+0000 7f24ded9d700 1 --2- 172.21.15.92:0/2037576263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ca7918e60 0x558ca78eb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.593+0000 7f24ded9d700 1 -- 172.21.15.92:0/2037576263 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ca7939bc0 msgr2=0x558ca7918920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.593+0000 7f24ded9d700 1 --2- 172.21.15.92:0/2037576263 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ca7939bc0 0x558ca7918920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.593+0000 7f24ded9d700 1 -- 172.21.15.92:0/2037576263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ca76da270 con 0x558ca78ef950 2023-10-06T10:41:59.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.593+0000 7f24ded9d700 1 --2- 172.21.15.92:0/2037576263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca78ef950 0x558ca78efd20 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f24d4001010 tx=0x7f24d4081eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870d0eca3859fa76 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.593+0000 7f24bffff700 1 -- 172.21.15.92:0/2037576263 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d4057bb0 con 0x558ca78ef950 2023-10-06T10:41:59.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.593+0000 7f24bffff700 1 -- 172.21.15.92:0/2037576263 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d405b900 con 0x558ca78ef950 2023-10-06T10:41:59.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.593+0000 7f24bffff700 1 -- 172.21.15.92:0/2037576263 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d4052d00 con 0x558ca78ef950 2023-10-06T10:41:59.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.593+0000 7f24ee399b80 1 -- 172.21.15.92:0/2037576263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca78ef950 msgr2=0x558ca78efd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.593+0000 7f24ee399b80 1 --2- 172.21.15.92:0/2037576263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca78ef950 0x558ca78efd20 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f24d4001010 tx=0x7f24d4081eb0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.594+0000 7f24ee399b80 1 -- 172.21.15.92:0/2037576263 shutdown_connections 2023-10-06T10:41:59.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.594+0000 7f24ee399b80 1 --2- 172.21.15.92:0/2037576263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ca7918e60 0x558ca78eb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.594+0000 7f24ee399b80 1 --2- 172.21.15.92:0/2037576263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca78ef950 0x558ca78efd20 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.594+0000 7f24ee399b80 1 --2- 172.21.15.92:0/2037576263 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ca7939bc0 0x558ca7918920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.594+0000 7f24ee399b80 1 -- 172.21.15.92:0/2037576263 >> 172.21.15.92:0/2037576263 conn(0x558ca78f1270 msgr2=0x558ca78f1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:59.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.594+0000 7f24ee399b80 1 -- 172.21.15.92:0/2037576263 shutdown_connections 2023-10-06T10:41:59.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.594+0000 7f24ee399b80 1 -- 172.21.15.92:0/2037576263 wait complete. 2023-10-06T10:41:59.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.594+0000 7f24ee399b80 1 -- start start 2023-10-06T10:41:59.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.594+0000 7f24ee399b80 1 Processor -- start 2023-10-06T10:41:59.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.595+0000 7f24ee399b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca78ef950 0x558ca792e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.595+0000 7f24ee399b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ca7918e60 0x558ca78e6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.595+0000 7f24ee399b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ca7939bc0 0x558ca78e6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.595+0000 7f24ee399b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca7840060 con 0x558ca78ef950 2023-10-06T10:41:59.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.595+0000 7f24ee399b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca77eb890 con 0x558ca7918e60 2023-10-06T10:41:59.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.595+0000 7f24ee399b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ca7806ea0 con 0x558ca7939bc0 2023-10-06T10:41:59.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.595+0000 7f24ee399b80 10 client.0 _ll_register_callbacks cb 0x558ca76dcf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:59.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.595+0000 7f24ded9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca78ef950 0x558ca792e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.595+0000 7f24ded9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca78ef950 0x558ca792e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58742/0 (socket says 172.21.15.92:58742) 2023-10-06T10:41:59.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.595+0000 7f24ded9d700 1 -- 172.21.15.92:0/788098436 learned_addr learned my addr 172.21.15.92:0/788098436 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:59.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.595+0000 7f24df59e700 1 --2- 172.21.15.92:0/788098436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ca7939bc0 0x558ca78e6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.595+0000 7f24de59c700 1 --2- 172.21.15.92:0/788098436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ca7918e60 0x558ca78e6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24ded9d700 1 -- 172.21.15.92:0/788098436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ca7939bc0 msgr2=0x558ca78e6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24ded9d700 1 --2- 172.21.15.92:0/788098436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ca7939bc0 0x558ca78e6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24ded9d700 1 -- 172.21.15.92:0/788098436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ca7918e60 msgr2=0x558ca78e6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24ded9d700 1 --2- 172.21.15.92:0/788098436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ca7918e60 0x558ca78e6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24ded9d700 1 -- 172.21.15.92:0/788098436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e00ce0d0 con 0x558ca78ef950 2023-10-06T10:41:59.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24de59c700 1 --2- 172.21.15.92:0/788098436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ca7918e60 0x558ca78e6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:59.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24ded9d700 1 --2- 172.21.15.92:0/788098436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca78ef950 0x558ca792e150 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f24d40070a0 tx=0x7f24d4069000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9131a636606a4fbb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24bd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:59.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24bd7fa700 1 -- 172.21.15.92:0/788098436 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d4007230 con 0x558ca78ef950 2023-10-06T10:41:59.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24bd7fa700 1 -- 172.21.15.92:0/788098436 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f24d4052d00 con 0x558ca78ef950 2023-10-06T10:41:59.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24ee399b80 1 -- 172.21.15.92:0/788098436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ca78e4690 con 0x558ca78ef950 2023-10-06T10:41:59.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24bd7fa700 1 -- 172.21.15.92:0/788098436 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24d405bd60 con 0x558ca78ef950 2023-10-06T10:41:59.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24ee399b80 10 client.6732 fetch_fsmap learned FSMap version 55 2023-10-06T10:41:59.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24ee399b80 1 -- 172.21.15.92:0/788098436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x558ca767cfe0 con 0x558ca78ef950 2023-10-06T10:41:59.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24bd7fa700 1 -- 172.21.15.92:0/788098436 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558ca78e4690 con 0x558ca78ef950 2023-10-06T10:41:59.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24bd7fa700 1 -- 172.21.15.92:0/788098436 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 55) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f24d406a020 con 0x558ca78ef950 2023-10-06T10:41:59.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24ee399b80 10 client.6732 fetch_fsmap finished waiting for FSMap version 55 2023-10-06T10:41:59.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24ee399b80 10 client.6732 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:41:59.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24ee399b80 20 client.6732 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:59.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24ee399b80 1 --2- 172.21.15.92:0/788098436 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x558ca7917320 0x558ca79176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24ee399b80 4 client.6732 mds_command: new command op to 6486 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:41:59.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24ee399b80 1 -- 172.21.15.92:0/788098436 --> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ca78c15c0 con 0x558ca7917320 2023-10-06T10:41:59.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.596+0000 7f24de59c700 1 --2- 172.21.15.92:0/788098436 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x558ca7917320 0x558ca79176f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.597+0000 7f24bd7fa700 10 client.6732 ms_handle_connect on v2:172.21.15.92:6835/818031030 2023-10-06T10:41:59.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.597+0000 7f24de59c700 1 --2- 172.21.15.92:0/788098436 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x558ca7917320 0x558ca79176f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c77105a07ca1d0c server_cookie=bd2b1d4cbe77ae7d in_seq=0 out_seq=0 2023-10-06T10:41:59.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.597+0000 7f24bd7fa700 1 -- 172.21.15.92:0/788098436 <== mds.0 v2:172.21.15.92:6835/818031030 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x558ca78c15c0 con 0x558ca7917320 2023-10-06T10:41:59.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.597+0000 7f24bd7fa700 10 client.6732 handle_command_reply: tid=0 2023-10-06T10:41:59.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.597+0000 7f24ee399b80 1 client.6732 shutdown 2023-10-06T10:41:59.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.597+0000 7f24ee399b80 1 -- 172.21.15.92:0/788098436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca78ef950 msgr2=0x558ca792e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.597+0000 7f24ee399b80 1 --2- 172.21.15.92:0/788098436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca78ef950 0x558ca792e150 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f24d40070a0 tx=0x7f24d4069000 comp rx=0 tx=0).stop 2023-10-06T10:41:59.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.597+0000 7f24ee399b80 1 -- 172.21.15.92:0/788098436 shutdown_connections 2023-10-06T10:41:59.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.597+0000 7f24ee399b80 1 --2- 172.21.15.92:0/788098436 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x558ca7917320 0x558ca79176f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.597+0000 7f24ee399b80 1 --2- 172.21.15.92:0/788098436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ca7939bc0 0x558ca78e6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.597+0000 7f24ee399b80 1 --2- 172.21.15.92:0/788098436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca78ef950 0x558ca792e150 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.597+0000 7f24ee399b80 1 --2- 172.21.15.92:0/788098436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ca7918e60 0x558ca78e6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.597+0000 7f24bd7fa700 0 client.6732 ms_handle_reset on v2:172.21.15.92:6835/818031030 2023-10-06T10:41:59.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.597+0000 7f24ee399b80 1 -- 172.21.15.92:0/788098436 >> 172.21.15.92:0/788098436 conn(0x558ca78f1270 msgr2=0x558ca79354a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:59.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.597+0000 7f24ee399b80 1 -- 172.21.15.92:0/788098436 shutdown_connections 2023-10-06T10:41:59.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.598+0000 7f24ee399b80 1 -- 172.21.15.92:0/788098436 wait complete. 2023-10-06T10:41:59.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.598+0000 7f24ee399b80 20 client.6732 trim_cache size 0 max 16384 2023-10-06T10:41:59.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.614+0000 7f24ee399b80 1 Processor -- start 2023-10-06T10:41:59.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.614+0000 7f24ee399b80 1 -- start start 2023-10-06T10:41:59.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.615+0000 7f24ee399b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ca78ef950 0x558ca78f0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.615+0000 7f24ee399b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca7918e60 0x558ca79184a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.615+0000 7f24ee399b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ca7939bc0 0x558ca7914e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.615+0000 7f24ee399b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca78eec40 con 0x558ca7918e60 2023-10-06T10:41:59.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.615+0000 7f24ee399b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca76c2e30 con 0x558ca7939bc0 2023-10-06T10:41:59.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.615+0000 7f24ee399b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ca792dcd0 con 0x558ca78ef950 2023-10-06T10:41:59.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.615+0000 7f24de59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca7918e60 0x558ca79184a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.615+0000 7f24de59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca7918e60 0x558ca79184a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58750/0 (socket says 172.21.15.92:58750) 2023-10-06T10:41:59.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.615+0000 7f24de59c700 1 -- 172.21.15.92:0/2885194175 learned_addr learned my addr 172.21.15.92:0/2885194175 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:59.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.615+0000 7f24ded9d700 1 --2- 172.21.15.92:0/2885194175 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ca78ef950 0x558ca78f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.615+0000 7f24df59e700 1 --2- 172.21.15.92:0/2885194175 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ca7939bc0 0x558ca7914e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.615+0000 7f24de59c700 1 -- 172.21.15.92:0/2885194175 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ca78ef950 msgr2=0x558ca78f0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.615+0000 7f24de59c700 1 --2- 172.21.15.92:0/2885194175 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ca78ef950 0x558ca78f0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.615+0000 7f24de59c700 1 -- 172.21.15.92:0/2885194175 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ca7939bc0 msgr2=0x558ca7914e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.615+0000 7f24de59c700 1 --2- 172.21.15.92:0/2885194175 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ca7939bc0 0x558ca7914e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.615+0000 7f24de59c700 1 -- 172.21.15.92:0/2885194175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ca780e780 con 0x558ca7918e60 2023-10-06T10:41:59.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.615+0000 7f24de59c700 1 --2- 172.21.15.92:0/2885194175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca7918e60 0x558ca79184a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f24d000bc70 tx=0x7f24d002e250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d943b40a7669e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.615+0000 7f24c4ff9700 1 -- 172.21.15.92:0/2885194175 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d002d070 con 0x558ca7918e60 2023-10-06T10:41:59.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.616+0000 7f24c4ff9700 1 -- 172.21.15.92:0/2885194175 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d0011c40 con 0x558ca7918e60 2023-10-06T10:41:59.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.616+0000 7f24c4ff9700 1 -- 172.21.15.92:0/2885194175 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d0011db0 con 0x558ca7918e60 2023-10-06T10:41:59.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.616+0000 7f24ee399b80 1 -- 172.21.15.92:0/2885194175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca7918e60 msgr2=0x558ca79184a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.616+0000 7f24ee399b80 1 --2- 172.21.15.92:0/2885194175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca7918e60 0x558ca79184a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f24d000bc70 tx=0x7f24d002e250 comp rx=0 tx=0).stop 2023-10-06T10:41:59.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.616+0000 7f24ee399b80 1 -- 172.21.15.92:0/2885194175 shutdown_connections 2023-10-06T10:41:59.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.616+0000 7f24ee399b80 1 --2- 172.21.15.92:0/2885194175 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ca7939bc0 0x558ca7914e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.616+0000 7f24ee399b80 1 --2- 172.21.15.92:0/2885194175 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ca78ef950 0x558ca78f0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.616+0000 7f24ee399b80 1 --2- 172.21.15.92:0/2885194175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca7918e60 0x558ca79184a0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.616+0000 7f24ee399b80 1 -- 172.21.15.92:0/2885194175 >> 172.21.15.92:0/2885194175 conn(0x558ca78f1270 msgr2=0x558ca7948300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:59.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.616+0000 7f24ee399b80 1 -- 172.21.15.92:0/2885194175 shutdown_connections 2023-10-06T10:41:59.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.616+0000 7f24ee399b80 1 -- 172.21.15.92:0/2885194175 wait complete. 2023-10-06T10:41:59.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.617+0000 7f24ee399b80 1 -- start start 2023-10-06T10:41:59.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.617+0000 7f24ee399b80 1 Processor -- start 2023-10-06T10:41:59.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24ee399b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ca78ef950 0x558ca7916140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24ee399b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca7918e60 0x558ca799f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24ee399b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ca7939bc0 0x558ca7939f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24ee399b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca7914de0 con 0x558ca7918e60 2023-10-06T10:41:59.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24ee399b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca7916710 con 0x558ca7939bc0 2023-10-06T10:41:59.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24ee399b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ca7914c40 con 0x558ca78ef950 2023-10-06T10:41:59.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24ee399b80 10 client.0 _ll_register_callbacks cb 0x558ca76dcf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:41:59.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24de59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca7918e60 0x558ca799f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24de59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca7918e60 0x558ca799f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58758/0 (socket says 172.21.15.92:58758) 2023-10-06T10:41:59.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24de59c700 1 -- 172.21.15.92:0/1899340932 learned_addr learned my addr 172.21.15.92:0/1899340932 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:59.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24df59e700 1 --2- 172.21.15.92:0/1899340932 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ca7939bc0 0x558ca7939f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24ded9d700 1 --2- 172.21.15.92:0/1899340932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ca78ef950 0x558ca7916140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24de59c700 1 -- 172.21.15.92:0/1899340932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ca78ef950 msgr2=0x558ca7916140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24de59c700 1 --2- 172.21.15.92:0/1899340932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ca78ef950 0x558ca7916140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24de59c700 1 -- 172.21.15.92:0/1899340932 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ca7939bc0 msgr2=0x558ca7939f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24de59c700 1 --2- 172.21.15.92:0/1899340932 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ca7939bc0 0x558ca7939f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24de59c700 1 -- 172.21.15.92:0/1899340932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d0021050 con 0x558ca7918e60 2023-10-06T10:41:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24ded9d700 1 --2- 172.21.15.92:0/1899340932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ca78ef950 0x558ca7916140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24de59c700 1 --2- 172.21.15.92:0/1899340932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca7918e60 0x558ca799f070 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f24d0008000 tx=0x7f24d002d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e00b197a718679 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24bd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:41:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24bd7fa700 1 -- 172.21.15.92:0/1899340932 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d0031900 con 0x558ca7918e60 2023-10-06T10:41:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24bd7fa700 1 -- 172.21.15.92:0/1899340932 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f24d0031a60 con 0x558ca7918e60 2023-10-06T10:41:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24ee399b80 1 -- 172.21.15.92:0/1899340932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24d405b900 con 0x558ca7918e60 2023-10-06T10:41:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24bd7fa700 1 -- 172.21.15.92:0/1899340932 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24d0031e70 con 0x558ca7918e60 2023-10-06T10:41:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24ee399b80 10 client.6738 fetch_fsmap learned FSMap version 55 2023-10-06T10:41:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24ee399b80 1 -- 172.21.15.92:0/1899340932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x558ca77f5470 con 0x558ca7918e60 2023-10-06T10:41:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.618+0000 7f24bd7fa700 1 -- 172.21.15.92:0/1899340932 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d405b900 con 0x558ca7918e60 2023-10-06T10:41:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24bd7fa700 1 -- 172.21.15.92:0/1899340932 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 55) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f24d001b430 con 0x558ca7918e60 2023-10-06T10:41:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24ee399b80 10 client.6738 fetch_fsmap finished waiting for FSMap version 55 2023-10-06T10:41:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24ee399b80 10 client.6738 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:41:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24ee399b80 20 client.6738 populate_metadata read hostname 'smithi092' 2023-10-06T10:41:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24ee399b80 1 --2- 172.21.15.92:0/1899340932 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x558ca793a610 0x558ca799a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24ee399b80 4 client.6738 mds_command: new command op to 6486 tid=0[{"prefix": "status"}] 2023-10-06T10:41:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24ee399b80 1 -- 172.21.15.92:0/1899340932 --> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558ca77e2e00 con 0x558ca793a610 2023-10-06T10:41:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24ded9d700 1 --2- 172.21.15.92:0/1899340932 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x558ca793a610 0x558ca799a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24bd7fa700 10 client.6738 ms_handle_connect on v2:172.21.15.92:6835/818031030 2023-10-06T10:41:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24ded9d700 1 --2- 172.21.15.92:0/1899340932 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x558ca793a610 0x558ca799a9e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ff074df6751e22c server_cookie=35150ebc1ea13017 in_seq=0 out_seq=0 2023-10-06T10:41:59.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24bd7fa700 1 -- 172.21.15.92:0/1899340932 <== mds.0 v2:172.21.15.92:6835/818031030 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x558ca77e2e00 con 0x558ca793a610 2023-10-06T10:41:59.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24bd7fa700 10 client.6738 handle_command_reply: tid=0 2023-10-06T10:41:59.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24ee399b80 1 client.6738 shutdown 2023-10-06T10:41:59.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24ee399b80 1 -- 172.21.15.92:0/1899340932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca7918e60 msgr2=0x558ca799f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24ee399b80 1 --2- 172.21.15.92:0/1899340932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca7918e60 0x558ca799f070 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f24d0008000 tx=0x7f24d002d000 comp rx=0 tx=0).stop 2023-10-06T10:41:59.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24ee399b80 1 -- 172.21.15.92:0/1899340932 shutdown_connections 2023-10-06T10:41:59.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24ee399b80 1 --2- 172.21.15.92:0/1899340932 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x558ca7939bc0 0x558ca7939f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24ee399b80 1 --2- 172.21.15.92:0/1899340932 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x558ca793a610 0x558ca799a9e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24ee399b80 1 --2- 172.21.15.92:0/1899340932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x558ca78ef950 0x558ca7916140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24ee399b80 1 --2- 172.21.15.92:0/1899340932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ca7918e60 0x558ca799f070 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24bd7fa700 0 client.6738 ms_handle_reset on v2:172.21.15.92:6835/818031030 2023-10-06T10:41:59.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24ee399b80 1 -- 172.21.15.92:0/1899340932 >> 172.21.15.92:0/1899340932 conn(0x558ca78f1270 msgr2=0x558ca79184a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:59.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24ee399b80 1 -- 172.21.15.92:0/1899340932 shutdown_connections 2023-10-06T10:41:59.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.619+0000 7f24ee399b80 1 -- 172.21.15.92:0/1899340932 wait complete. 2023-10-06T10:41:59.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.620+0000 7f24ee399b80 20 client.6738 trim_cache size 0 max 16384 2023-10-06T10:41:59.629 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:41:59.629 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:41:59.629 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T10:41:59.629 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 6486, 2023-10-06T10:41:59.629 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:41:59.629 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:41:59.630 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:41:59.630 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 11.088527235999999, 2023-10-06T10:41:59.630 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 55, 2023-10-06T10:41:59.630 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 64, 2023-10-06T10:41:59.630 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 64, 2023-10-06T10:41:59.630 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 17.210622378 2023-10-06T10:41:59.630 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:41:59.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.620+0000 7f24e48a0700 1 -- 172.21.15.92:0/3697369866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f24cc066f30 msgr2=0x7f24cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.620+0000 7f24e48a0700 1 --2- 172.21.15.92:0/3697369866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f24cc066f30 0x7f24cc0693e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f24e00ca7f0 tx=0x7f24d0017000 comp rx=0 tx=0).stop 2023-10-06T10:41:59.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.620+0000 7f24e48a0700 1 -- 172.21.15.92:0/3697369866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e00b2100 msgr2=0x7f24e0153c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.620+0000 7f24e48a0700 1 --2- 172.21.15.92:0/3697369866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e00b2100 0x7f24e0153c80 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f24d400c5b0 tx=0x7f24d4001040 comp rx=0 tx=0).stop 2023-10-06T10:41:59.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.620+0000 7f24e48a0700 1 -- 172.21.15.92:0/3697369866 shutdown_connections 2023-10-06T10:41:59.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.620+0000 7f24e48a0700 1 --2- 172.21.15.92:0/3697369866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f24cc066f30 0x7f24cc0693e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.620+0000 7f24e48a0700 1 --2- 172.21.15.92:0/3697369866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24e0159440 0x7f24e00c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.620+0000 7f24e48a0700 1 --2- 172.21.15.92:0/3697369866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e00b2100 0x7f24e0153c80 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.620+0000 7f24e48a0700 1 --2- 172.21.15.92:0/3697369866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24e00cbc30 0x7f24e00c72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.620+0000 7f24e48a0700 1 -- 172.21.15.92:0/3697369866 >> 172.21.15.92:0/3697369866 conn(0x7f24e006ece0 msgr2=0x7f24e0158570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:59.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.620+0000 7f24e48a0700 1 -- 172.21.15.92:0/3697369866 shutdown_connections 2023-10-06T10:41:59.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.620+0000 7f24e48a0700 1 -- 172.21.15.92:0/3697369866 wait complete. 2023-10-06T10:41:59.633 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:41:59.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.746+0000 7fdaf9987700 1 Processor -- start 2023-10-06T10:41:59.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.746+0000 7fdaf9987700 1 -- start start 2023-10-06T10:41:59.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.747+0000 7fdaf9987700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdaf40b49a0 0x7fdaf40aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.747+0000 7fdaf9987700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf40b0c60 con 0x7fdaf4159560 2023-10-06T10:41:59.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.747+0000 7fdaf9987700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf40b0f40 con 0x7fdaf40b0760 2023-10-06T10:41:59.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.747+0000 7fdaf9987700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf40b1220 con 0x7fdaf40b49a0 2023-10-06T10:41:59.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.747+0000 7fdaf2ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdaf4159560 0x7fdaf40aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54824/0 (socket says 172.21.15.92:54824) 2023-10-06T10:41:59.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.747+0000 7fdaf2ffd700 1 -- 172.21.15.92:0/2745309894 learned_addr learned my addr 172.21.15.92:0/2745309894 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:41:59.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.747+0000 7fdaf27fc700 1 --2- 172.21.15.92:0/2745309894 >> v2:172.21.15.124:3300/0 conn(0x7fdaf40b49a0 0x7fdaf40aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.747+0000 7fdaf1ffb700 1 -- 172.21.15.92:0/2745309894 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1416894070 0 0) 0x7fdaf40b0c60 con 0x7fdaf4159560 2023-10-06T10:41:59.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.747+0000 7fdaf1ffb700 1 -- 172.21.15.92:0/2745309894 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdae0003570 con 0x7fdaf4159560 2023-10-06T10:41:59.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.747+0000 7fdaf1ffb700 1 -- 172.21.15.92:0/2745309894 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 918762999 0 0) 0x7fdae0003570 con 0x7fdaf4159560 2023-10-06T10:41:59.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.747+0000 7fdaf1ffb700 1 -- 172.21.15.92:0/2745309894 >> v2:172.21.15.124:3300/0 conn(0x7fdaf40b49a0 msgr2=0x7fdaf40aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.747+0000 7fdaf1ffb700 1 --2- 172.21.15.92:0/2745309894 >> v2:172.21.15.124:3300/0 conn(0x7fdaf40b49a0 0x7fdaf40aac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.747+0000 7fdaf1ffb700 1 -- 172.21.15.92:0/2745309894 >> v1:172.21.15.124:6789/0 conn(0x7fdaf40b0760 legacy=0x7fdaf40b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.747+0000 7fdaf1ffb700 1 -- 172.21.15.92:0/2745309894 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf40b1500 con 0x7fdaf4159560 2023-10-06T10:41:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.748+0000 7fdaf1ffb700 1 -- 172.21.15.92:0/2745309894 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fdaec001970 con 0x7fdaf4159560 2023-10-06T10:41:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.748+0000 7fdaf1ffb700 1 -- 172.21.15.92:0/2745309894 >> v1:172.21.15.92:6789/0 conn(0x7fdaf4159560 legacy=0x7fdaf40aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.748+0000 7fdaf1ffb700 1 --2- 172.21.15.92:0/2745309894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdae0003a90 0x7fdae0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.748+0000 7fdaf1ffb700 1 --2- 172.21.15.92:0/2745309894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdae0004810 0x7fdae0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.748+0000 7fdaf1ffb700 1 --2- 172.21.15.92:0/2745309894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdae00052e0 0x7fdae000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.748+0000 7fdaf1ffb700 1 -- 172.21.15.92:0/2745309894 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf40b1220 con 0x7fdae0004810 2023-10-06T10:41:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.748+0000 7fdaf1ffb700 1 -- 172.21.15.92:0/2745309894 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae000c1b0 con 0x7fdae0003a90 2023-10-06T10:41:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.748+0000 7fdaf1ffb700 1 -- 172.21.15.92:0/2745309894 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae000c370 con 0x7fdae00052e0 2023-10-06T10:41:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.748+0000 7fdaf27fc700 1 --2- 172.21.15.92:0/2745309894 >> v2:172.21.15.124:3300/0 conn(0x7fdaf40b49a0 0x7fdaf40aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.748+0000 7fdaf2ffd700 1 --2- 172.21.15.92:0/2745309894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdae0003a90 0x7fdae0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.748+0000 7fdaf27fc700 1 --2- 172.21.15.92:0/2745309894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdae0004810 0x7fdae0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.748+0000 7fdaf37fe700 1 --2- 172.21.15.92:0/2745309894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdae00052e0 0x7fdae000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.748+0000 7fdaf27fc700 1 -- 172.21.15.92:0/2745309894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdae00052e0 msgr2=0x7fdae000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.748+0000 7fdaf27fc700 1 --2- 172.21.15.92:0/2745309894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdae00052e0 0x7fdae000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.748+0000 7fdaf27fc700 1 -- 172.21.15.92:0/2745309894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdae0003a90 msgr2=0x7fdae0003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:41:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.748+0000 7fdaf27fc700 1 --2- 172.21.15.92:0/2745309894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdae0003a90 0x7fdae0003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.748+0000 7fdaf37fe700 1 -- 172.21.15.92:0/2745309894 reap_dead start 2023-10-06T10:41:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.748+0000 7fdaf27fc700 1 -- 172.21.15.92:0/2745309894 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae0003570 con 0x7fdae0004810 2023-10-06T10:41:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.748+0000 7fdaf27fc700 1 --2- 172.21.15.92:0/2745309894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdae0004810 0x7fdae0004c80 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4099f20 tx=0x7fdadc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6597a8eaf948c562 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.748+0000 7fdaf1ffb700 1 -- 172.21.15.92:0/2745309894 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc00e5b0 con 0x7fdae0004810 2023-10-06T10:41:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.748+0000 7fdaf1ffb700 1 -- 172.21.15.92:0/2745309894 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdadc00e710 con 0x7fdae0004810 2023-10-06T10:41:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.748+0000 7fdaf1ffb700 1 -- 172.21.15.92:0/2745309894 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc00e8c0 con 0x7fdae0004810 2023-10-06T10:41:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.749+0000 7fdaf9987700 1 -- 172.21.15.92:0/2745309894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdae0004810 msgr2=0x7fdae0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.749+0000 7fdaf9987700 1 --2- 172.21.15.92:0/2745309894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdae0004810 0x7fdae0004c80 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4099f20 tx=0x7fdadc008040 comp rx=0 tx=0).stop 2023-10-06T10:41:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.749+0000 7fdaf9987700 1 -- 172.21.15.92:0/2745309894 shutdown_connections 2023-10-06T10:41:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.749+0000 7fdaf9987700 1 --2- 172.21.15.92:0/2745309894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdae0004810 0x7fdae0004c80 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.749+0000 7fdaf9987700 1 -- 172.21.15.92:0/2745309894 >> 172.21.15.92:0/2745309894 conn(0x7fdaf40997f0 msgr2=0x7fdaf406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:59.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.749+0000 7fdaf9987700 1 -- 172.21.15.92:0/2745309894 shutdown_connections 2023-10-06T10:41:59.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.749+0000 7fdaf9987700 1 -- 172.21.15.92:0/2745309894 wait complete. 2023-10-06T10:41:59.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.750+0000 7fdaf9987700 1 Processor -- start 2023-10-06T10:41:59.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.750+0000 7fdaf9987700 1 -- start start 2023-10-06T10:41:59.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.751+0000 7fdaf9987700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdae0004810 0x7fdaf4155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.751+0000 7fdaf9987700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdaf4155a90 0x7fdaf4169ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.751+0000 7fdaf9987700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf416acb0 0x7fdaf40691d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.751+0000 7fdaf9987700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf416ab10 con 0x7fdaf416acb0 2023-10-06T10:41:59.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.751+0000 7fdaf9987700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf415a130 con 0x7fdae0004810 2023-10-06T10:41:59.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.751+0000 7fdaf9987700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf4159e00 con 0x7fdaf4155a90 2023-10-06T10:41:59.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.751+0000 7fdaf37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf416acb0 0x7fdaf40691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.751+0000 7fdaf37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf416acb0 0x7fdaf40691d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58766/0 (socket says 172.21.15.92:58766) 2023-10-06T10:41:59.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.751+0000 7fdaf37fe700 1 -- 172.21.15.92:0/1389427684 learned_addr learned my addr 172.21.15.92:0/1389427684 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:41:59.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.751+0000 7fdaf2ffd700 1 --2- 172.21.15.92:0/1389427684 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdae0004810 0x7fdaf4155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.751+0000 7fdaf27fc700 1 --2- 172.21.15.92:0/1389427684 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdaf4155a90 0x7fdaf4169ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.751+0000 7fdaf37fe700 1 -- 172.21.15.92:0/1389427684 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdaf4155a90 msgr2=0x7fdaf4169ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.751+0000 7fdaf37fe700 1 --2- 172.21.15.92:0/1389427684 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdaf4155a90 0x7fdaf4169ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.751+0000 7fdaf37fe700 1 -- 172.21.15.92:0/1389427684 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdae0004810 msgr2=0x7fdaf4155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.751+0000 7fdaf37fe700 1 --2- 172.21.15.92:0/1389427684 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdae0004810 0x7fdaf4155170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.751+0000 7fdaf37fe700 1 -- 172.21.15.92:0/1389427684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf416a550 con 0x7fdaf416acb0 2023-10-06T10:41:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.751+0000 7fdaf27fc700 1 --2- 172.21.15.92:0/1389427684 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdaf4155a90 0x7fdaf4169ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:41:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.752+0000 7fdaf37fe700 1 --2- 172.21.15.92:0/1389427684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf416acb0 0x7fdaf40691d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40b2f10 tx=0x7fdae4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.752+0000 7fdaeaffd700 1 -- 172.21.15.92:0/1389427684 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae4006a30 con 0x7fdaf416acb0 2023-10-06T10:41:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.752+0000 7fdaeaffd700 1 -- 172.21.15.92:0/1389427684 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae4006b90 con 0x7fdaf416acb0 2023-10-06T10:41:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.752+0000 7fdaeaffd700 1 -- 172.21.15.92:0/1389427684 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae4007030 con 0x7fdaf416acb0 2023-10-06T10:41:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.752+0000 7fdaf9987700 1 -- 172.21.15.92:0/1389427684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf416a830 con 0x7fdaf416acb0 2023-10-06T10:41:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.752+0000 7fdaf9987700 1 -- 172.21.15.92:0/1389427684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf4169ff0 con 0x7fdaf416acb0 2023-10-06T10:41:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.753+0000 7fdaf9987700 1 -- 172.21.15.92:0/1389427684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf4159c60 con 0x7fdaf416acb0 2023-10-06T10:41:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.753+0000 7fdaeaffd700 1 -- 172.21.15.92:0/1389427684 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdae40073a0 con 0x7fdaf416acb0 2023-10-06T10:41:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.753+0000 7fdaeaffd700 1 --2- 172.21.15.92:0/1389427684 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdad4066f30 0x7fdad40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:41:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.754+0000 7fdaeaffd700 1 -- 172.21.15.92:0/1389427684 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fdae4083f30 con 0x7fdaf416acb0 2023-10-06T10:41:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.754+0000 7fdaf2ffd700 1 --2- 172.21.15.92:0/1389427684 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdad4066f30 0x7fdad40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:41:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.754+0000 7fdaf2ffd700 1 --2- 172.21.15.92:0/1389427684 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdad4066f30 0x7fdad40693e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdaf415ebc0 tx=0x7fdaec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:41:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.758+0000 7fdaeaffd700 1 -- 172.21.15.92:0/1389427684 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdaf4159c60 con 0x7fdaf416acb0 2023-10-06T10:41:59.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.984+0000 7fdaf9987700 1 -- 172.21.15.92:0/1389427684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdaf4159c60 con 0x7fdaf416acb0 2023-10-06T10:41:59.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.984+0000 7fdaeaffd700 1 -- 172.21.15.92:0/1389427684 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4627 (secure 0 0 0) 0x7fdaf4159c60 con 0x7fdaf416acb0 2023-10-06T10:41:59.986 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:41:59.986 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":55,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/1409495728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":1409495728},{"type":"v1","addr":"172.21.15.124:6834","nonce":1409495728}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:41:47.898774+0000","modified":"2023-10-06T10:41:58.576160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6486,"mds_1":6483,"mds_2":6019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6019":{"gid":6019,"name":"b","rank":2,"incarnation":54,"state":"up:active","state_seq":5,"addr":"172.21.15.124:6835/2297612118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2297612118},{"type":"v1","addr":"172.21.15.124:6835","nonce":2297612118}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6483":{"gid":6483,"name":"a","rank":1,"incarnation":52,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6836/171578176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":171578176},{"type":"v1","addr":"172.21.15.92:6836","nonce":171578176}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6486":{"gid":6486,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/818031030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":818031030},{"type":"v1","addr":"172.21.15.92:6837","nonce":818031030}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T10:41:59.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.986+0000 7fdaf9987700 1 -- 172.21.15.92:0/1389427684 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdad4066f30 msgr2=0x7fdad40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.986+0000 7fdaf9987700 1 --2- 172.21.15.92:0/1389427684 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdad4066f30 0x7fdad40693e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdaf415ebc0 tx=0x7fdaec009040 comp rx=0 tx=0).stop 2023-10-06T10:41:59.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.986+0000 7fdaf9987700 1 -- 172.21.15.92:0/1389427684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf416acb0 msgr2=0x7fdaf40691d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:41:59.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.986+0000 7fdaf9987700 1 --2- 172.21.15.92:0/1389427684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf416acb0 0x7fdaf40691d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40b2f10 tx=0x7fdae4007d30 comp rx=0 tx=0).stop 2023-10-06T10:41:59.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.986+0000 7fdaf9987700 1 -- 172.21.15.92:0/1389427684 shutdown_connections 2023-10-06T10:41:59.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.986+0000 7fdaf9987700 1 --2- 172.21.15.92:0/1389427684 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdad4066f30 0x7fdad40693e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.986+0000 7fdaf9987700 1 --2- 172.21.15.92:0/1389427684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf416acb0 0x7fdaf40691d0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.987+0000 7fdaf9987700 1 --2- 172.21.15.92:0/1389427684 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdae0004810 0x7fdaf4155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.987+0000 7fdaf9987700 1 --2- 172.21.15.92:0/1389427684 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdaf4155a90 0x7fdaf4169ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:41:59.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.987+0000 7fdaf9987700 1 -- 172.21.15.92:0/1389427684 >> 172.21.15.92:0/1389427684 conn(0x7fdaf40997f0 msgr2=0x7fdaf40afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:41:59.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.987+0000 7fdaf9987700 1 -- 172.21.15.92:0/1389427684 shutdown_connections 2023-10-06T10:41:59.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:41:59.987+0000 7fdaf9987700 1 -- 172.21.15.92:0/1389427684 wait complete. 2023-10-06T10:41:59.990 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 55 2023-10-06T10:42:00.000 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:42:00.001 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2023-10-06T10:42:00.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:42:00.023 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2023-10-06T10:42:00.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:42:00.084 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2023-10-06T10:42:00.139 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:42:00.139 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2023-10-06T10:42:00.195 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:42:00.195 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2023-10-06T10:42:00.251 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:42:00.421 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:42:00.422 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:42:00.422 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:42:02.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:42:02.567 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:42:02.568 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:42:02.568 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:42:04.569 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:42:04.710 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:42:04.710 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:42:04.711 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:42:06.712 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:42:06.851 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:42:06.852 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:42:06.852 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:42:08.854 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:42:08.997 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2023-10-06T10:42:08.998 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2023-10-06T10:42:08.999 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:42:08.999 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-10-06T10:42:09.128 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2023-10-06T10:42:09.129 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2023-10-06T10:42:09.129 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:42:11.131 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-10-06T10:42:11.271 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2023-10-06T10:42:11.271 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2023-10-06T10:42:11.272 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:42:13.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-10-06T10:42:13.413 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2023-10-06T10:42:13.414 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2023-10-06T10:42:13.414 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:42:13.414 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-10-06T10:42:13.556 INFO:teuthology.orchestra.run.smithi124.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-10-06T10:42:13.557 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:42:13.557 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:42:13.557 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2023-10-06T10:42:13.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.673+0000 7f0638d26700 1 Processor -- start 2023-10-06T10:42:13.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.673+0000 7f0638d26700 1 -- start start 2023-10-06T10:42:13.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.674+0000 7f0638d26700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0634150940 0x7f0634159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.674+0000 7f0638d26700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f06340cd4d0 0x7f06340b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.674+0000 7f0638d26700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063415ac30 con 0x7f063415a160 2023-10-06T10:42:13.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.674+0000 7f0638d26700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06340cb6a0 con 0x7f06340cd4d0 2023-10-06T10:42:13.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.674+0000 7f0638d26700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06340cb090 con 0x7f0634150940 2023-10-06T10:42:13.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.674+0000 7f0632d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f06340cd4d0 0x7f06340b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.674+0000 7f0632d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f06340cd4d0 0x7f06340b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52954/0 (socket says 172.21.15.92:52954) 2023-10-06T10:42:13.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.674+0000 7f0632d9d700 1 -- 172.21.15.92:0/3228248108 learned_addr learned my addr 172.21.15.92:0/3228248108 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:13.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.674+0000 7f063259c700 1 --2- 172.21.15.92:0/3228248108 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0634150940 0x7f0634159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.674+0000 7f0632d9d700 1 -- 172.21.15.92:0/3228248108 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0634150940 msgr2=0x7f0634159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:13.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.674+0000 7f0632d9d700 1 --2- 172.21.15.92:0/3228248108 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0634150940 0x7f0634159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.674+0000 7f0632d9d700 1 -- 172.21.15.92:0/3228248108 >> v1:172.21.15.92:6789/0 conn(0x7f063415a160 legacy=0x7f06340ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:13.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.674+0000 7f063159a700 1 -- 172.21.15.92:0/3228248108 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276758426 0 0) 0x7f0628001110 con 0x7f063415a160 2023-10-06T10:42:13.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.674+0000 7f0632d9d700 1 -- 172.21.15.92:0/3228248108 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063415a670 con 0x7f06340cd4d0 2023-10-06T10:42:13.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.674+0000 7f0632d9d700 1 --2- 172.21.15.92:0/3228248108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06340cd4d0 0x7f06340b2050 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f062c009f20 tx=0x7f062c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4e84331a5aae48 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:13.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.675+0000 7f063159a700 1 -- 172.21.15.92:0/3228248108 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c00e070 con 0x7f06340cd4d0 2023-10-06T10:42:13.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.675+0000 7f063159a700 1 -- 172.21.15.92:0/3228248108 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c00ba80 con 0x7f06340cd4d0 2023-10-06T10:42:13.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.675+0000 7f063159a700 1 -- 172.21.15.92:0/3228248108 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c0092e0 con 0x7f06340cd4d0 2023-10-06T10:42:13.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.675+0000 7f0638d26700 1 -- 172.21.15.92:0/3228248108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06340cd4d0 msgr2=0x7f06340b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:13.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.675+0000 7f0638d26700 1 --2- 172.21.15.92:0/3228248108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06340cd4d0 0x7f06340b2050 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f062c009f20 tx=0x7f062c009840 comp rx=0 tx=0).stop 2023-10-06T10:42:13.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.675+0000 7f0638d26700 1 -- 172.21.15.92:0/3228248108 shutdown_connections 2023-10-06T10:42:13.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.675+0000 7f0638d26700 1 --2- 172.21.15.92:0/3228248108 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0634150940 0x7f0634159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.675+0000 7f0638d26700 1 --2- 172.21.15.92:0/3228248108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06340cd4d0 0x7f06340b2050 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.675+0000 7f0638d26700 1 -- 172.21.15.92:0/3228248108 >> 172.21.15.92:0/3228248108 conn(0x7f0634099700 msgr2=0x7f063406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:13.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.675+0000 7f0638d26700 1 -- 172.21.15.92:0/3228248108 shutdown_connections 2023-10-06T10:42:13.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.675+0000 7f0638d26700 1 -- 172.21.15.92:0/3228248108 wait complete. 2023-10-06T10:42:13.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.676+0000 7f0638d26700 1 Processor -- start 2023-10-06T10:42:13.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.676+0000 7f0638d26700 1 -- start start 2023-10-06T10:42:13.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.677+0000 7f0638d26700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06340cd4d0 0x7f0634152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.677+0000 7f0638d26700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f063415a160 0x7f0634154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.677+0000 7f0638d26700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0634159400 0x7f063416c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.677+0000 7f0638d26700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06341551d0 con 0x7f0634159400 2023-10-06T10:42:13.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.677+0000 7f0638d26700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0634155410 con 0x7f063415a160 2023-10-06T10:42:13.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.677+0000 7f0638d26700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06341556f0 con 0x7f06340cd4d0 2023-10-06T10:42:13.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.677+0000 7f0632d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0634159400 0x7f063416c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.677+0000 7f0632d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0634159400 0x7f063416c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52960/0 (socket says 172.21.15.92:52960) 2023-10-06T10:42:13.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.677+0000 7f0632d9d700 1 -- 172.21.15.92:0/3085432760 learned_addr learned my addr 172.21.15.92:0/3085432760 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:13.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.677+0000 7f0632d9d700 1 -- 172.21.15.92:0/3085432760 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06340cd4d0 msgr2=0x7f0634152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:13.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.677+0000 7f0632d9d700 1 --2- 172.21.15.92:0/3085432760 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06340cd4d0 0x7f0634152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.677+0000 7f0632d9d700 1 -- 172.21.15.92:0/3085432760 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f063415a160 msgr2=0x7f0634154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:13.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.677+0000 7f0632d9d700 1 --2- 172.21.15.92:0/3085432760 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f063415a160 0x7f0634154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.677+0000 7f0632d9d700 1 -- 172.21.15.92:0/3085432760 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062c009ae0 con 0x7f0634159400 2023-10-06T10:42:13.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.677+0000 7f0632d9d700 1 --2- 172.21.15.92:0/3085432760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0634159400 0x7f063416c160 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f062c012000 tx=0x7f062c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:13.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.678+0000 7f06227fc700 1 -- 172.21.15.92:0/3085432760 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c0032f0 con 0x7f0634159400 2023-10-06T10:42:13.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.678+0000 7f06227fc700 1 -- 172.21.15.92:0/3085432760 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c026460 con 0x7f0634159400 2023-10-06T10:42:13.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.678+0000 7f0638d26700 1 -- 172.21.15.92:0/3085432760 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06341559d0 con 0x7f0634159400 2023-10-06T10:42:13.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.678+0000 7f06227fc700 1 -- 172.21.15.92:0/3085432760 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c026800 con 0x7f0634159400 2023-10-06T10:42:13.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.678+0000 7f0638d26700 1 -- 172.21.15.92:0/3085432760 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0634155f90 con 0x7f0634159400 2023-10-06T10:42:13.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.679+0000 7f06227fc700 1 -- 172.21.15.92:0/3085432760 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f062c003040 con 0x7f0634159400 2023-10-06T10:42:13.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.679+0000 7f06227fc700 1 --2- 172.21.15.92:0/3085432760 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0614066f30 0x7f06140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.679+0000 7f06227fc700 1 -- 172.21.15.92:0/3085432760 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f062c0869b0 con 0x7f0634159400 2023-10-06T10:42:13.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.679+0000 7f063259c700 1 --2- 172.21.15.92:0/3085432760 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0614066f30 0x7f06140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.680+0000 7f063259c700 1 --2- 172.21.15.92:0/3085432760 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0614066f30 0x7f06140693e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0624009a40 tx=0x7f0624008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:13.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.681+0000 7f064201eb80 1 Processor -- start 2023-10-06T10:42:13.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.681+0000 7f064201eb80 1 -- start start 2023-10-06T10:42:13.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.682+0000 7f064201eb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567ed74dae0 0x5567ed74deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.682+0000 7f064201eb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567ed7099f0 0x5567ed7065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.682+0000 7f064201eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed707c20 0x5567ed706b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.682+0000 7f064201eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ed686cd0 con 0x5567ed707c20 2023-10-06T10:42:13.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.682+0000 7f064201eb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ed65dac0 con 0x5567ed74dae0 2023-10-06T10:42:13.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.682+0000 7f064201eb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567ed65b2c0 con 0x5567ed7099f0 2023-10-06T10:42:13.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.682+0000 7f0632d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed707c20 0x5567ed706b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.682+0000 7f0632d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed707c20 0x5567ed706b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52978/0 (socket says 172.21.15.92:52978) 2023-10-06T10:42:13.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.682+0000 7f0632d9d700 1 -- 172.21.15.92:0/2925188003 learned_addr learned my addr 172.21.15.92:0/2925188003 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:13.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.682+0000 7f0632d9d700 1 -- 172.21.15.92:0/2925188003 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567ed7099f0 msgr2=0x5567ed7065d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:13.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.682+0000 7f0632d9d700 1 --2- 172.21.15.92:0/2925188003 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567ed7099f0 0x5567ed7065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.682+0000 7f063259c700 1 --2- 172.21.15.92:0/2925188003 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567ed74dae0 0x5567ed74deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.682+0000 7f0632d9d700 1 -- 172.21.15.92:0/2925188003 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567ed74dae0 msgr2=0x5567ed74deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:13.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.682+0000 7f0632d9d700 1 --2- 172.21.15.92:0/2925188003 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567ed74dae0 0x5567ed74deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.682+0000 7f0632d9d700 1 -- 172.21.15.92:0/2925188003 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567ed4f12c0 con 0x5567ed707c20 2023-10-06T10:42:13.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.683+0000 7f0632d9d700 1 --2- 172.21.15.92:0/2925188003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed707c20 0x5567ed706b10 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f062c009a60 tx=0x7f062c084b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7433749debba290 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:13.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.683+0000 7f06137fe700 1 -- 172.21.15.92:0/2925188003 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c00e030 con 0x5567ed707c20 2023-10-06T10:42:13.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.683+0000 7f06137fe700 1 -- 172.21.15.92:0/2925188003 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c0557f0 con 0x5567ed707c20 2023-10-06T10:42:13.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.683+0000 7f06137fe700 1 -- 172.21.15.92:0/2925188003 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c055a50 con 0x5567ed707c20 2023-10-06T10:42:13.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.683+0000 7f064201eb80 1 -- 172.21.15.92:0/2925188003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed707c20 msgr2=0x5567ed706b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:13.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.683+0000 7f064201eb80 1 --2- 172.21.15.92:0/2925188003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed707c20 0x5567ed706b10 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f062c009a60 tx=0x7f062c084b50 comp rx=0 tx=0).stop 2023-10-06T10:42:13.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.683+0000 7f064201eb80 1 -- 172.21.15.92:0/2925188003 shutdown_connections 2023-10-06T10:42:13.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.683+0000 7f064201eb80 1 --2- 172.21.15.92:0/2925188003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed707c20 0x5567ed706b10 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.683+0000 7f064201eb80 1 --2- 172.21.15.92:0/2925188003 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567ed74dae0 0x5567ed74deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.683+0000 7f064201eb80 1 --2- 172.21.15.92:0/2925188003 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567ed7099f0 0x5567ed7065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.683+0000 7f064201eb80 1 -- 172.21.15.92:0/2925188003 >> 172.21.15.92:0/2925188003 conn(0x5567ed7080c0 msgr2=0x5567ed708490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:13.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.683+0000 7f064201eb80 1 -- 172.21.15.92:0/2925188003 shutdown_connections 2023-10-06T10:42:13.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.683+0000 7f064201eb80 1 -- 172.21.15.92:0/2925188003 wait complete. 2023-10-06T10:42:13.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.684+0000 7f064201eb80 1 -- start start 2023-10-06T10:42:13.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.684+0000 7f064201eb80 1 Processor -- start 2023-10-06T10:42:13.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.685+0000 7f064201eb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567ed707c20 0x5567ed7488d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.685+0000 7f064201eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed7099f0 0x5567ed74a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.685+0000 7f064201eb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567ed74dae0 0x5567ed7306c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.685+0000 7f064201eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ed6570b0 con 0x5567ed7099f0 2023-10-06T10:42:13.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.685+0000 7f064201eb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ed6028e0 con 0x5567ed74dae0 2023-10-06T10:42:13.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.685+0000 7f064201eb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567ed61def0 con 0x5567ed707c20 2023-10-06T10:42:13.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.685+0000 7f0631d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed7099f0 0x5567ed74a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.685+0000 7f064201eb80 10 client.0 _ll_register_callbacks cb 0x5567ed4f3f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:42:13.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.685+0000 7f0631d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed7099f0 0x5567ed74a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52988/0 (socket says 172.21.15.92:52988) 2023-10-06T10:42:13.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.685+0000 7f0631d9b700 1 -- 172.21.15.92:0/4196674593 learned_addr learned my addr 172.21.15.92:0/4196674593 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:13.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.685+0000 7f0632d9d700 1 --2- 172.21.15.92:0/4196674593 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567ed74dae0 0x5567ed7306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.685+0000 7f063259c700 1 --2- 172.21.15.92:0/4196674593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567ed707c20 0x5567ed7488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.685+0000 7f0631d9b700 1 -- 172.21.15.92:0/4196674593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567ed707c20 msgr2=0x5567ed7488d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:13.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.685+0000 7f0631d9b700 1 --2- 172.21.15.92:0/4196674593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567ed707c20 0x5567ed7488d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.685+0000 7f0631d9b700 1 -- 172.21.15.92:0/4196674593 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567ed74dae0 msgr2=0x5567ed7306c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:13.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.685+0000 7f0631d9b700 1 --2- 172.21.15.92:0/4196674593 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567ed74dae0 0x5567ed7306c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.685+0000 7f0631d9b700 1 -- 172.21.15.92:0/4196674593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0634155f90 con 0x5567ed7099f0 2023-10-06T10:42:13.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.686+0000 7f0631d9b700 1 --2- 172.21.15.92:0/4196674593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed7099f0 0x5567ed74a1b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f0628001e90 tx=0x7f0628007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca62080caabe1c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:13.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.686+0000 7f0610ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:42:13.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.686+0000 7f0610ff9700 1 -- 172.21.15.92:0/4196674593 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062800cca0 con 0x5567ed7099f0 2023-10-06T10:42:13.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.686+0000 7f0610ff9700 1 -- 172.21.15.92:0/4196674593 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f062800ce00 con 0x5567ed7099f0 2023-10-06T10:42:13.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.686+0000 7f064201eb80 1 -- 172.21.15.92:0/4196674593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567ed6fb6e0 con 0x5567ed7099f0 2023-10-06T10:42:13.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.686+0000 7f0610ff9700 1 -- 172.21.15.92:0/4196674593 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0628012610 con 0x5567ed7099f0 2023-10-06T10:42:13.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.686+0000 7f064201eb80 10 client.6759 fetch_fsmap learned FSMap version 57 2023-10-06T10:42:13.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.686+0000 7f064201eb80 1 -- 172.21.15.92:0/4196674593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5567ed494030 con 0x5567ed7099f0 2023-10-06T10:42:13.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.686+0000 7f0610ff9700 1 -- 172.21.15.92:0/4196674593 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5567ed6fb6e0 con 0x5567ed7099f0 2023-10-06T10:42:13.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.686+0000 7f0610ff9700 1 -- 172.21.15.92:0/4196674593 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 57) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f0628012770 con 0x5567ed7099f0 2023-10-06T10:42:13.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.686+0000 7f064201eb80 10 client.6759 fetch_fsmap finished waiting for FSMap version 57 2023-10-06T10:42:13.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.686+0000 7f064201eb80 10 client.6759 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-10-06T10:42:13.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.686+0000 7f064201eb80 20 client.6759 populate_metadata read hostname 'smithi092' 2023-10-06T10:42:13.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.686+0000 7f064201eb80 1 --2- 172.21.15.92:0/4196674593 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x5567ed733150 0x5567ed72eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.686+0000 7f064201eb80 4 client.6759 mds_command: new command op to 6486 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:42:13.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.686+0000 7f064201eb80 1 -- 172.21.15.92:0/4196674593 --> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567ed6d8610 con 0x5567ed733150 2023-10-06T10:42:13.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.686+0000 7f063259c700 1 --2- 172.21.15.92:0/4196674593 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x5567ed733150 0x5567ed72eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.686+0000 7f0610ff9700 10 client.6759 ms_handle_connect on v2:172.21.15.92:6835/818031030 2023-10-06T10:42:13.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.687+0000 7f063259c700 1 --2- 172.21.15.92:0/4196674593 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x5567ed733150 0x5567ed72eaf0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86974ffc55ad6871 server_cookie=f5f77104b86008e2 in_seq=0 out_seq=0 2023-10-06T10:42:13.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.687+0000 7f0610ff9700 1 -- 172.21.15.92:0/4196674593 <== mds.0 v2:172.21.15.92:6835/818031030 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5567ed6d8610 con 0x5567ed733150 2023-10-06T10:42:13.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.687+0000 7f0610ff9700 10 client.6759 handle_command_reply: tid=0 2023-10-06T10:42:13.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.687+0000 7f064201eb80 1 client.6759 shutdown 2023-10-06T10:42:13.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.687+0000 7f064201eb80 1 -- 172.21.15.92:0/4196674593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed7099f0 msgr2=0x5567ed74a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:13.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.687+0000 7f064201eb80 1 --2- 172.21.15.92:0/4196674593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed7099f0 0x5567ed74a1b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f0628001e90 tx=0x7f0628007e80 comp rx=0 tx=0).stop 2023-10-06T10:42:13.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.687+0000 7f064201eb80 1 -- 172.21.15.92:0/4196674593 shutdown_connections 2023-10-06T10:42:13.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.687+0000 7f064201eb80 1 --2- 172.21.15.92:0/4196674593 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567ed74dae0 0x5567ed7306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.687+0000 7f064201eb80 1 --2- 172.21.15.92:0/4196674593 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x5567ed733150 0x5567ed72eaf0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.687+0000 7f064201eb80 1 --2- 172.21.15.92:0/4196674593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567ed707c20 0x5567ed7488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.687+0000 7f064201eb80 1 --2- 172.21.15.92:0/4196674593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed7099f0 0x5567ed74a1b0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.687+0000 7f064201eb80 1 -- 172.21.15.92:0/4196674593 >> 172.21.15.92:0/4196674593 conn(0x5567ed7080c0 msgr2=0x5567ed74cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:13.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.687+0000 7f0610ff9700 0 client.6759 ms_handle_reset on v2:172.21.15.92:6835/818031030 2023-10-06T10:42:13.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.687+0000 7f064201eb80 1 -- 172.21.15.92:0/4196674593 shutdown_connections 2023-10-06T10:42:13.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.687+0000 7f064201eb80 1 -- 172.21.15.92:0/4196674593 wait complete. 2023-10-06T10:42:13.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.688+0000 7f064201eb80 20 client.6759 trim_cache size 0 max 16384 2023-10-06T10:42:13.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.702+0000 7f064201eb80 1 Processor -- start 2023-10-06T10:42:13.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.702+0000 7f064201eb80 1 -- start start 2023-10-06T10:42:13.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.703+0000 7f064201eb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567ed707c20 0x5567ed708660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.703+0000 7f064201eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed7099f0 0x5567ed74a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.703+0000 7f064201eb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567ed74dae0 0x5567ed74a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.703+0000 7f064201eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ed4d9e80 con 0x5567ed7099f0 2023-10-06T10:42:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.703+0000 7f064201eb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ed736c40 con 0x5567ed707c20 2023-10-06T10:42:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.703+0000 7f064201eb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567ed733400 con 0x5567ed74dae0 2023-10-06T10:42:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.703+0000 7f0631d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed7099f0 0x5567ed74a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.703+0000 7f0631d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed7099f0 0x5567ed74a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53002/0 (socket says 172.21.15.92:53002) 2023-10-06T10:42:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.703+0000 7f0631d9b700 1 -- 172.21.15.92:0/1459894592 learned_addr learned my addr 172.21.15.92:0/1459894592 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.703+0000 7f0632d9d700 1 --2- 172.21.15.92:0/1459894592 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567ed74dae0 0x5567ed74a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.703+0000 7f063259c700 1 --2- 172.21.15.92:0/1459894592 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567ed707c20 0x5567ed708660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.703+0000 7f0631d9b700 1 -- 172.21.15.92:0/1459894592 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567ed74dae0 msgr2=0x5567ed74a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:13.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.703+0000 7f0631d9b700 1 --2- 172.21.15.92:0/1459894592 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567ed74dae0 0x5567ed74a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.703+0000 7f0631d9b700 1 -- 172.21.15.92:0/1459894592 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567ed707c20 msgr2=0x5567ed708660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:13.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.703+0000 7f0631d9b700 1 --2- 172.21.15.92:0/1459894592 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567ed707c20 0x5567ed708660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.703+0000 7f0631d9b700 1 -- 172.21.15.92:0/1459894592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567ed6257d0 con 0x5567ed7099f0 2023-10-06T10:42:13.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.703+0000 7f063259c700 1 --2- 172.21.15.92:0/1459894592 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567ed707c20 0x5567ed708660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:13.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.703+0000 7f0631d9b700 1 --2- 172.21.15.92:0/1459894592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed7099f0 0x5567ed74a190 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f0628010380 tx=0x7f0628015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2236b7f2e3f2e9fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:13.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.703+0000 7f0613fff700 1 -- 172.21.15.92:0/1459894592 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062800c750 con 0x5567ed7099f0 2023-10-06T10:42:13.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.703+0000 7f0613fff700 1 -- 172.21.15.92:0/1459894592 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0628012c80 con 0x5567ed7099f0 2023-10-06T10:42:13.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.703+0000 7f0613fff700 1 -- 172.21.15.92:0/1459894592 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0628000c30 con 0x5567ed7099f0 2023-10-06T10:42:13.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.704+0000 7f064201eb80 1 -- 172.21.15.92:0/1459894592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed7099f0 msgr2=0x5567ed74a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:13.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.704+0000 7f064201eb80 1 --2- 172.21.15.92:0/1459894592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed7099f0 0x5567ed74a190 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f0628010380 tx=0x7f0628015000 comp rx=0 tx=0).stop 2023-10-06T10:42:13.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.704+0000 7f064201eb80 1 -- 172.21.15.92:0/1459894592 shutdown_connections 2023-10-06T10:42:13.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.704+0000 7f064201eb80 1 --2- 172.21.15.92:0/1459894592 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567ed74dae0 0x5567ed74a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.704+0000 7f064201eb80 1 --2- 172.21.15.92:0/1459894592 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567ed707c20 0x5567ed708660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.704+0000 7f064201eb80 1 --2- 172.21.15.92:0/1459894592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed7099f0 0x5567ed74a190 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.704+0000 7f064201eb80 1 -- 172.21.15.92:0/1459894592 >> 172.21.15.92:0/1459894592 conn(0x5567ed7080c0 msgr2=0x5567ed7045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:13.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.704+0000 7f064201eb80 1 -- 172.21.15.92:0/1459894592 shutdown_connections 2023-10-06T10:42:13.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.704+0000 7f064201eb80 1 -- 172.21.15.92:0/1459894592 wait complete. 2023-10-06T10:42:13.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.705+0000 7f064201eb80 1 -- start start 2023-10-06T10:42:13.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.705+0000 7f064201eb80 1 Processor -- start 2023-10-06T10:42:13.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f064201eb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567ed707c20 0x5567ed731280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f064201eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed7099f0 0x5567ed7ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f064201eb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567ed74dae0 0x5567ed7038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f064201eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ed7b2d00 con 0x5567ed7099f0 2023-10-06T10:42:13.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f064201eb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ed7b3a00 con 0x5567ed74dae0 2023-10-06T10:42:13.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f064201eb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567ed7b2ea0 con 0x5567ed707c20 2023-10-06T10:42:13.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f064201eb80 10 client.0 _ll_register_callbacks cb 0x5567ed4f3f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:42:13.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f0631d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed7099f0 0x5567ed7ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f0631d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed7099f0 0x5567ed7ac3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53018/0 (socket says 172.21.15.92:53018) 2023-10-06T10:42:13.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f0631d9b700 1 -- 172.21.15.92:0/1628863517 learned_addr learned my addr 172.21.15.92:0/1628863517 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:13.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f063259c700 1 --2- 172.21.15.92:0/1628863517 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567ed707c20 0x5567ed731280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f0632d9d700 1 --2- 172.21.15.92:0/1628863517 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567ed74dae0 0x5567ed7038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f0631d9b700 1 -- 172.21.15.92:0/1628863517 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567ed707c20 msgr2=0x5567ed731280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:13.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f0631d9b700 1 --2- 172.21.15.92:0/1628863517 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567ed707c20 0x5567ed731280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f0631d9b700 1 -- 172.21.15.92:0/1628863517 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567ed74dae0 msgr2=0x5567ed7038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:13.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f0631d9b700 1 --2- 172.21.15.92:0/1628863517 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567ed74dae0 0x5567ed7038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f0631d9b700 1 -- 172.21.15.92:0/1628863517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567ed494030 con 0x5567ed7099f0 2023-10-06T10:42:13.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f0631d9b700 1 --2- 172.21.15.92:0/1628863517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed7099f0 0x5567ed7ac3c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f0628001d80 tx=0x7f0628023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc97de7ddac42cc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:13.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f0610ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:42:13.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f0610ff9700 1 -- 172.21.15.92:0/1628863517 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0628031070 con 0x5567ed7099f0 2023-10-06T10:42:13.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f0610ff9700 1 -- 172.21.15.92:0/1628863517 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f0628000c30 con 0x5567ed7099f0 2023-10-06T10:42:13.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f064201eb80 1 -- 172.21.15.92:0/1628863517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f062c0557f0 con 0x5567ed7099f0 2023-10-06T10:42:13.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f0610ff9700 1 -- 172.21.15.92:0/1628863517 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f062800ceb0 con 0x5567ed7099f0 2023-10-06T10:42:13.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f064201eb80 10 client.6765 fetch_fsmap learned FSMap version 57 2023-10-06T10:42:13.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f064201eb80 1 -- 172.21.15.92:0/1628863517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5567ed60c4c0 con 0x5567ed7099f0 2023-10-06T10:42:13.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.706+0000 7f0610ff9700 1 -- 172.21.15.92:0/1628863517 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c0557f0 con 0x5567ed7099f0 2023-10-06T10:42:13.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.707+0000 7f0610ff9700 1 -- 172.21.15.92:0/1628863517 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 57) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f0628006a10 con 0x5567ed7099f0 2023-10-06T10:42:13.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.707+0000 7f064201eb80 10 client.6765 fetch_fsmap finished waiting for FSMap version 57 2023-10-06T10:42:13.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.707+0000 7f064201eb80 10 client.6765 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-10-06T10:42:13.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.707+0000 7f064201eb80 20 client.6765 populate_metadata read hostname 'smithi092' 2023-10-06T10:42:13.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.707+0000 7f064201eb80 1 --2- 172.21.15.92:0/1628863517 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x5567ed703e20 0x5567ed751210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.707+0000 7f064201eb80 4 client.6765 mds_command: new command op to 6486 tid=0[{"prefix": "flush journal"}] 2023-10-06T10:42:13.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.707+0000 7f064201eb80 1 -- 172.21.15.92:0/1628863517 --> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5567ed5f9e50 con 0x5567ed703e20 2023-10-06T10:42:13.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.707+0000 7f063259c700 1 --2- 172.21.15.92:0/1628863517 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x5567ed703e20 0x5567ed751210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.707+0000 7f0610ff9700 10 client.6765 ms_handle_connect on v2:172.21.15.92:6835/818031030 2023-10-06T10:42:13.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.707+0000 7f063259c700 1 --2- 172.21.15.92:0/1628863517 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x5567ed703e20 0x5567ed751210 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bdd59f55306064b server_cookie=b70de893880b7e18 in_seq=0 out_seq=0 2023-10-06T10:42:13.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.721+0000 7f0610ff9700 1 -- 172.21.15.92:0/1628863517 <== mds.0 v2:172.21.15.92:6835/818031030 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5567ed5f9e50 con 0x5567ed703e20 2023-10-06T10:42:13.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.721+0000 7f0610ff9700 10 client.6765 handle_command_reply: tid=0 2023-10-06T10:42:13.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.721+0000 7f064201eb80 1 client.6765 shutdown 2023-10-06T10:42:13.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.721+0000 7f064201eb80 1 -- 172.21.15.92:0/1628863517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed7099f0 msgr2=0x5567ed7ac3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:13.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.721+0000 7f064201eb80 1 --2- 172.21.15.92:0/1628863517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed7099f0 0x5567ed7ac3c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f0628001d80 tx=0x7f0628023040 comp rx=0 tx=0).stop 2023-10-06T10:42:13.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.721+0000 7f064201eb80 1 -- 172.21.15.92:0/1628863517 shutdown_connections 2023-10-06T10:42:13.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.721+0000 7f064201eb80 1 --2- 172.21.15.92:0/1628863517 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567ed74dae0 0x5567ed7038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.721+0000 7f064201eb80 1 --2- 172.21.15.92:0/1628863517 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x5567ed703e20 0x5567ed751210 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.721+0000 7f064201eb80 1 --2- 172.21.15.92:0/1628863517 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567ed707c20 0x5567ed731280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.721+0000 7f0610ff9700 0 client.6765 ms_handle_reset on v2:172.21.15.92:6835/818031030 2023-10-06T10:42:13.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.721+0000 7f064201eb80 1 --2- 172.21.15.92:0/1628863517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567ed7099f0 0x5567ed7ac3c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.721+0000 7f064201eb80 1 -- 172.21.15.92:0/1628863517 >> 172.21.15.92:0/1628863517 conn(0x5567ed7080c0 msgr2=0x5567ed72efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:13.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.721+0000 7f064201eb80 1 -- 172.21.15.92:0/1628863517 shutdown_connections 2023-10-06T10:42:13.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.721+0000 7f064201eb80 1 -- 172.21.15.92:0/1628863517 wait complete. 2023-10-06T10:42:13.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.721+0000 7f064201eb80 20 client.6765 trim_cache size 0 max 16384 2023-10-06T10:42:13.724 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:42:13.724 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2023-10-06T10:42:13.724 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:42:13.724 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:42:13.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.722+0000 7f0638d26700 1 -- 172.21.15.92:0/3085432760 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0614066f30 msgr2=0x7f06140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:13.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.722+0000 7f0638d26700 1 --2- 172.21.15.92:0/3085432760 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0614066f30 0x7f06140693e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0624009a40 tx=0x7f0624008040 comp rx=0 tx=0).stop 2023-10-06T10:42:13.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.722+0000 7f0638d26700 1 -- 172.21.15.92:0/3085432760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0634159400 msgr2=0x7f063416c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:13.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.722+0000 7f0638d26700 1 --2- 172.21.15.92:0/3085432760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0634159400 0x7f063416c160 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f062c012000 tx=0x7f062c018000 comp rx=0 tx=0).stop 2023-10-06T10:42:13.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.722+0000 7f0638d26700 1 -- 172.21.15.92:0/3085432760 shutdown_connections 2023-10-06T10:42:13.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.722+0000 7f0638d26700 1 --2- 172.21.15.92:0/3085432760 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0614066f30 0x7f06140693e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.722+0000 7f0638d26700 1 --2- 172.21.15.92:0/3085432760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0634159400 0x7f063416c160 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.722+0000 7f0638d26700 1 --2- 172.21.15.92:0/3085432760 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06340cd4d0 0x7f0634152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.722+0000 7f0638d26700 1 --2- 172.21.15.92:0/3085432760 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f063415a160 0x7f0634154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.722+0000 7f0638d26700 1 -- 172.21.15.92:0/3085432760 >> 172.21.15.92:0/3085432760 conn(0x7f0634099700 msgr2=0x7f06340b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:13.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.722+0000 7f0638d26700 1 -- 172.21.15.92:0/3085432760 shutdown_connections 2023-10-06T10:42:13.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.722+0000 7f0638d26700 1 -- 172.21.15.92:0/3085432760 wait complete. 2023-10-06T10:42:13.734 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:1 flush journal 2023-10-06T10:42:13.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.849+0000 7ff763114700 1 Processor -- start 2023-10-06T10:42:13.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.849+0000 7ff763114700 1 -- start start 2023-10-06T10:42:13.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.850+0000 7ff763114700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff75c159420 0x7ff75c1597f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.850+0000 7ff763114700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff75c0689f0 0x7ff75c0cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.850+0000 7ff763114700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75c0cdc60 con 0x7ff75c0cb6d0 2023-10-06T10:42:13.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.850+0000 7ff763114700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75c0cd050 con 0x7ff75c0689f0 2023-10-06T10:42:13.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.850+0000 7ff763114700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75c0cd330 con 0x7ff75c159420 2023-10-06T10:42:13.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.850+0000 7ff7616b1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff75c0689f0 0x7ff75c0cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.851+0000 7ff7616b1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff75c0689f0 0x7ff75c0cbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53024/0 (socket says 172.21.15.92:53024) 2023-10-06T10:42:13.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.851+0000 7ff7616b1700 1 -- 172.21.15.92:0/3892549800 learned_addr learned my addr 172.21.15.92:0/3892549800 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:13.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.851+0000 7ff760eb0700 1 --2- 172.21.15.92:0/3892549800 >> v2:172.21.15.124:3300/0 conn(0x7ff75c159420 0x7ff75c1597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.851+0000 7ff7616b1700 1 -- 172.21.15.92:0/3892549800 >> v2:172.21.15.124:3300/0 conn(0x7ff75c159420 msgr2=0x7ff75c1597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:13.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.851+0000 7ff7616b1700 1 --2- 172.21.15.92:0/3892549800 >> v2:172.21.15.124:3300/0 conn(0x7ff75c159420 0x7ff75c1597f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.851+0000 7ff7616b1700 1 -- 172.21.15.92:0/3892549800 >> v1:172.21.15.124:6789/0 conn(0x7ff75c0cb6d0 legacy=0x7ff75c0682e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:13.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.851+0000 7ff7616b1700 1 -- 172.21.15.92:0/3892549800 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c0cd610 con 0x7ff75c0689f0 2023-10-06T10:42:13.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.851+0000 7ff7616b1700 1 --2- 172.21.15.92:0/3892549800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff75c0689f0 0x7ff75c0cbb70 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff75c068020 tx=0x7ff754009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=767740cc44a32260 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:13.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.851+0000 7ff7537fe700 1 -- 172.21.15.92:0/3892549800 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754009e70 con 0x7ff75c0689f0 2023-10-06T10:42:13.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.851+0000 7ff7537fe700 1 -- 172.21.15.92:0/3892549800 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75400b860 con 0x7ff75c0689f0 2023-10-06T10:42:13.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.851+0000 7ff7537fe700 1 -- 172.21.15.92:0/3892549800 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75400bc00 con 0x7ff75c0689f0 2023-10-06T10:42:13.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.852+0000 7ff763114700 1 -- 172.21.15.92:0/3892549800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff75c0689f0 msgr2=0x7ff75c0cbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:13.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.852+0000 7ff763114700 1 --2- 172.21.15.92:0/3892549800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff75c0689f0 0x7ff75c0cbb70 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff75c068020 tx=0x7ff754009250 comp rx=0 tx=0).stop 2023-10-06T10:42:13.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.852+0000 7ff763114700 1 -- 172.21.15.92:0/3892549800 shutdown_connections 2023-10-06T10:42:13.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.852+0000 7ff763114700 1 --2- 172.21.15.92:0/3892549800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff75c0689f0 0x7ff75c0cbb70 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.852+0000 7ff763114700 1 --2- 172.21.15.92:0/3892549800 >> v2:172.21.15.124:3300/0 conn(0x7ff75c159420 0x7ff75c1597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.852+0000 7ff763114700 1 -- 172.21.15.92:0/3892549800 >> 172.21.15.92:0/3892549800 conn(0x7ff75c06ece0 msgr2=0x7ff75c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:13.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.852+0000 7ff763114700 1 -- 172.21.15.92:0/3892549800 shutdown_connections 2023-10-06T10:42:13.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.852+0000 7ff763114700 1 -- 172.21.15.92:0/3892549800 wait complete. 2023-10-06T10:42:13.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.853+0000 7ff763114700 1 Processor -- start 2023-10-06T10:42:13.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.853+0000 7ff763114700 1 -- start start 2023-10-06T10:42:13.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.854+0000 7ff763114700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff75c0689f0 0x7ff75c15c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.854+0000 7ff763114700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff75c0cb6d0 0x7ff75c0c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.854+0000 7ff763114700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff75c159420 0x7ff75c16cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.854+0000 7ff763114700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c0b3b70 con 0x7ff75c0689f0 2023-10-06T10:42:13.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.854+0000 7ff763114700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c0c8b50 con 0x7ff75c159420 2023-10-06T10:42:13.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.854+0000 7ff763114700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c0c9670 con 0x7ff75c0cb6d0 2023-10-06T10:42:13.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.854+0000 7ff760eb0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff75c0689f0 0x7ff75c15c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.854+0000 7ff760eb0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff75c0689f0 0x7ff75c15c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53028/0 (socket says 172.21.15.92:53028) 2023-10-06T10:42:13.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.854+0000 7ff760eb0700 1 -- 172.21.15.92:0/3381969366 learned_addr learned my addr 172.21.15.92:0/3381969366 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:13.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.854+0000 7ff7616b1700 1 --2- 172.21.15.92:0/3381969366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff75c159420 0x7ff75c16cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.854+0000 7ff753fff700 1 --2- 172.21.15.92:0/3381969366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff75c0cb6d0 0x7ff75c0c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.854+0000 7ff760eb0700 1 -- 172.21.15.92:0/3381969366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff75c0cb6d0 msgr2=0x7ff75c0c8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.854+0000 7ff760eb0700 1 --2- 172.21.15.92:0/3381969366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff75c0cb6d0 0x7ff75c0c8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.854+0000 7ff760eb0700 1 -- 172.21.15.92:0/3381969366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff75c159420 msgr2=0x7ff75c16cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.854+0000 7ff760eb0700 1 --2- 172.21.15.92:0/3381969366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff75c159420 0x7ff75c16cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.854+0000 7ff760eb0700 1 -- 172.21.15.92:0/3381969366 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7540094f0 con 0x7ff75c0689f0 2023-10-06T10:42:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.854+0000 7ff760eb0700 1 --2- 172.21.15.92:0/3381969366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff75c0689f0 0x7ff75c15c8e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff758009ef0 tx=0x7ff75800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.854+0000 7ff750ff9700 1 -- 172.21.15.92:0/3381969366 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75800cb70 con 0x7ff75c0689f0 2023-10-06T10:42:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.854+0000 7ff750ff9700 1 -- 172.21.15.92:0/3381969366 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75800f040 con 0x7ff75c0689f0 2023-10-06T10:42:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.854+0000 7ff763114700 1 -- 172.21.15.92:0/3381969366 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75c16d440 con 0x7ff75c0689f0 2023-10-06T10:42:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.854+0000 7ff750ff9700 1 -- 172.21.15.92:0/3381969366 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758014480 con 0x7ff75c0689f0 2023-10-06T10:42:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.854+0000 7ff763114700 1 -- 172.21.15.92:0/3381969366 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75c0c90b0 con 0x7ff75c0689f0 2023-10-06T10:42:13.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.856+0000 7ff750ff9700 1 -- 172.21.15.92:0/3381969366 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff7580147f0 con 0x7ff75c0689f0 2023-10-06T10:42:13.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.856+0000 7ff750ff9700 1 --2- 172.21.15.92:0/3381969366 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff748066f30 0x7ff7480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.856+0000 7ff750ff9700 1 -- 172.21.15.92:0/3381969366 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7ff758085140 con 0x7ff75c0689f0 2023-10-06T10:42:13.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.856+0000 7ff753fff700 1 --2- 172.21.15.92:0/3381969366 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff748066f30 0x7ff7480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.856+0000 7ff753fff700 1 --2- 172.21.15.92:0/3381969366 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff748066f30 0x7ff7480693e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0c6940 tx=0x7ff74c007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.858+0000 7ff76c40cb80 1 Processor -- start 2023-10-06T10:42:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.858+0000 7ff76c40cb80 1 -- start start 2023-10-06T10:42:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.859+0000 7ff76c40cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324ef2ae0 0x565324ef2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.859+0000 7ff76c40cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x565324eae9f0 0x565324eab5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.859+0000 7ff76c40cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x565324eacc20 0x565324eabb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.859+0000 7ff76c40cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x565324e2bcd0 con 0x565324ef2ae0 2023-10-06T10:42:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.859+0000 7ff76c40cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x565324e02ac0 con 0x565324eacc20 2023-10-06T10:42:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.859+0000 7ff76c40cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x565324e002c0 con 0x565324eae9f0 2023-10-06T10:42:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.859+0000 7ff760eb0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324ef2ae0 0x565324ef2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.859+0000 7ff760eb0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324ef2ae0 0x565324ef2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53040/0 (socket says 172.21.15.92:53040) 2023-10-06T10:42:13.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.859+0000 7ff760eb0700 1 -- 172.21.15.92:0/2030724507 learned_addr learned my addr 172.21.15.92:0/2030724507 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:13.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.859+0000 7ff753fff700 1 --2- 172.21.15.92:0/2030724507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x565324eae9f0 0x565324eab5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.859+0000 7ff7616b1700 1 --2- 172.21.15.92:0/2030724507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x565324eacc20 0x565324eabb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.859+0000 7ff760eb0700 1 -- 172.21.15.92:0/2030724507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x565324eae9f0 msgr2=0x565324eab5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:13.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.859+0000 7ff760eb0700 1 --2- 172.21.15.92:0/2030724507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x565324eae9f0 0x565324eab5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.859+0000 7ff760eb0700 1 -- 172.21.15.92:0/2030724507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x565324eacc20 msgr2=0x565324eabb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:13.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.859+0000 7ff760eb0700 1 --2- 172.21.15.92:0/2030724507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x565324eacc20 0x565324eabb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.859+0000 7ff760eb0700 1 -- 172.21.15.92:0/2030724507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565324c962c0 con 0x565324ef2ae0 2023-10-06T10:42:13.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.859+0000 7ff7616b1700 1 --2- 172.21.15.92:0/2030724507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x565324eacc20 0x565324eabb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:13.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.859+0000 7ff753fff700 1 --2- 172.21.15.92:0/2030724507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x565324eae9f0 0x565324eab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:13.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.859+0000 7ff760eb0700 1 --2- 172.21.15.92:0/2030724507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324ef2ae0 0x565324ef2eb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff758083380 tx=0x7ff7580832e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1009d7c7ae26ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:13.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.859+0000 7ff741ffb700 1 -- 172.21.15.92:0/2030724507 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75805b0c0 con 0x565324ef2ae0 2023-10-06T10:42:13.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.859+0000 7ff741ffb700 1 -- 172.21.15.92:0/2030724507 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff758058fe0 con 0x565324ef2ae0 2023-10-06T10:42:13.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.859+0000 7ff741ffb700 1 -- 172.21.15.92:0/2030724507 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75805cd30 con 0x565324ef2ae0 2023-10-06T10:42:13.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.860+0000 7ff76c40cb80 1 -- 172.21.15.92:0/2030724507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324ef2ae0 msgr2=0x565324ef2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:13.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.860+0000 7ff76c40cb80 1 --2- 172.21.15.92:0/2030724507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324ef2ae0 0x565324ef2eb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff758083380 tx=0x7ff7580832e0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.860+0000 7ff76c40cb80 1 -- 172.21.15.92:0/2030724507 shutdown_connections 2023-10-06T10:42:13.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.860+0000 7ff76c40cb80 1 --2- 172.21.15.92:0/2030724507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x565324eacc20 0x565324eabb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.860+0000 7ff76c40cb80 1 --2- 172.21.15.92:0/2030724507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324ef2ae0 0x565324ef2eb0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.860+0000 7ff76c40cb80 1 --2- 172.21.15.92:0/2030724507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x565324eae9f0 0x565324eab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.860+0000 7ff76c40cb80 1 -- 172.21.15.92:0/2030724507 >> 172.21.15.92:0/2030724507 conn(0x565324ead0c0 msgr2=0x565324ead490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:13.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.860+0000 7ff76c40cb80 1 -- 172.21.15.92:0/2030724507 shutdown_connections 2023-10-06T10:42:13.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.860+0000 7ff76c40cb80 1 -- 172.21.15.92:0/2030724507 wait complete. 2023-10-06T10:42:13.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.860+0000 7ff76c40cb80 1 -- start start 2023-10-06T10:42:13.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.861+0000 7ff76c40cb80 1 Processor -- start 2023-10-06T10:42:13.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.861+0000 7ff76c40cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324eacc20 0x565324ed4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.861+0000 7ff76c40cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x565324eae9f0 0x565324ed58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.861+0000 7ff76c40cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x565324ef2ae0 0x565324ed20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.861+0000 7ff76c40cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x565324dfc0b0 con 0x565324eacc20 2023-10-06T10:42:13.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.861+0000 7ff76c40cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x565324da78e0 con 0x565324eae9f0 2023-10-06T10:42:13.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.861+0000 7ff76c40cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x565324dc2ef0 con 0x565324ef2ae0 2023-10-06T10:42:13.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.861+0000 7ff76c40cb80 10 client.0 _ll_register_callbacks cb 0x565324c98f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:42:13.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.861+0000 7ff760eb0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324eacc20 0x565324ed4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.861+0000 7ff760eb0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324eacc20 0x565324ed4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53054/0 (socket says 172.21.15.92:53054) 2023-10-06T10:42:13.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.861+0000 7ff760eb0700 1 -- 172.21.15.92:0/1814691948 learned_addr learned my addr 172.21.15.92:0/1814691948 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:13.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff7616b1700 1 --2- 172.21.15.92:0/1814691948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x565324ef2ae0 0x565324ed20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff753fff700 1 --2- 172.21.15.92:0/1814691948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x565324eae9f0 0x565324ed58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff760eb0700 1 -- 172.21.15.92:0/1814691948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x565324ef2ae0 msgr2=0x565324ed20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:13.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff760eb0700 1 --2- 172.21.15.92:0/1814691948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x565324ef2ae0 0x565324ed20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff760eb0700 1 -- 172.21.15.92:0/1814691948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x565324eae9f0 msgr2=0x565324ed58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:13.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff760eb0700 1 --2- 172.21.15.92:0/1814691948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x565324eae9f0 0x565324ed58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff760eb0700 1 -- 172.21.15.92:0/1814691948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75c0c90b0 con 0x565324eacc20 2023-10-06T10:42:13.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff760eb0700 1 --2- 172.21.15.92:0/1814691948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324eacc20 0x565324ed4180 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff758055d20 tx=0x7ff75806a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88fcdcef7e28c3ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:13.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff73affd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:42:13.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff73affd700 1 -- 172.21.15.92:0/1814691948 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758060070 con 0x565324eacc20 2023-10-06T10:42:13.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff73affd700 1 -- 172.21.15.92:0/1814691948 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7ff75805b0c0 con 0x565324eacc20 2023-10-06T10:42:13.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff76c40cb80 1 -- 172.21.15.92:0/1814691948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565324ea06e0 con 0x565324eacc20 2023-10-06T10:42:13.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff73affd700 1 -- 172.21.15.92:0/1814691948 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff75805cd30 con 0x565324eacc20 2023-10-06T10:42:13.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff76c40cb80 10 client.6777 fetch_fsmap learned FSMap version 57 2023-10-06T10:42:13.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff76c40cb80 1 -- 172.21.15.92:0/1814691948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x565324c39030 con 0x565324eacc20 2023-10-06T10:42:13.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff73affd700 1 -- 172.21.15.92:0/1814691948 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565324ea06e0 con 0x565324eacc20 2023-10-06T10:42:13.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff73affd700 1 -- 172.21.15.92:0/1814691948 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 57) v1 ==== 3136+0+0 (secure 0 0 0) 0x7ff758068240 con 0x565324eacc20 2023-10-06T10:42:13.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff76c40cb80 10 client.6777 fetch_fsmap finished waiting for FSMap version 57 2023-10-06T10:42:13.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff76c40cb80 10 client.6777 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.a 2023-10-06T10:42:13.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff76c40cb80 20 client.6777 populate_metadata read hostname 'smithi092' 2023-10-06T10:42:13.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff76c40cb80 1 --2- 172.21.15.92:0/1814691948 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x565324eecf20 0x565324eed320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff76c40cb80 4 client.6777 mds_command: new command op to 6483 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:42:13.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.862+0000 7ff76c40cb80 1 -- 172.21.15.92:0/1814691948 --> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565324e7d610 con 0x565324eecf20 2023-10-06T10:42:13.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.863+0000 7ff753fff700 1 --2- 172.21.15.92:0/1814691948 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x565324eecf20 0x565324eed320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.863+0000 7ff753fff700 1 --2- 172.21.15.92:0/1814691948 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x565324eecf20 0x565324eed320 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=893e925ab0fdfca server_cookie=89866fc48c1034e9 in_seq=0 out_seq=0 2023-10-06T10:42:13.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.863+0000 7ff73affd700 10 client.6777 ms_handle_connect on v2:172.21.15.92:6834/171578176 2023-10-06T10:42:13.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.863+0000 7ff73affd700 1 -- 172.21.15.92:0/1814691948 <== mds.1 v2:172.21.15.92:6834/171578176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x565324e7d610 con 0x565324eecf20 2023-10-06T10:42:13.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.863+0000 7ff73affd700 10 client.6777 handle_command_reply: tid=0 2023-10-06T10:42:13.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.863+0000 7ff76c40cb80 1 client.6777 shutdown 2023-10-06T10:42:13.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.864+0000 7ff76c40cb80 1 -- 172.21.15.92:0/1814691948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324eacc20 msgr2=0x565324ed4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:13.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.864+0000 7ff76c40cb80 1 --2- 172.21.15.92:0/1814691948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324eacc20 0x565324ed4180 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff758055d20 tx=0x7ff75806a000 comp rx=0 tx=0).stop 2023-10-06T10:42:13.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.864+0000 7ff76c40cb80 1 -- 172.21.15.92:0/1814691948 shutdown_connections 2023-10-06T10:42:13.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.864+0000 7ff76c40cb80 1 --2- 172.21.15.92:0/1814691948 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x565324eecf20 0x565324eed320 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.864+0000 7ff76c40cb80 1 --2- 172.21.15.92:0/1814691948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x565324ef2ae0 0x565324ed20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.864+0000 7ff73affd700 0 client.6777 ms_handle_reset on v2:172.21.15.92:6834/171578176 2023-10-06T10:42:13.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.864+0000 7ff76c40cb80 1 --2- 172.21.15.92:0/1814691948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324eacc20 0x565324ed4180 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.864+0000 7ff76c40cb80 1 --2- 172.21.15.92:0/1814691948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x565324eae9f0 0x565324ed58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.864+0000 7ff76c40cb80 1 -- 172.21.15.92:0/1814691948 >> 172.21.15.92:0/1814691948 conn(0x565324ead0c0 msgr2=0x565324ef0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:13.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.864+0000 7ff76c40cb80 1 -- 172.21.15.92:0/1814691948 shutdown_connections 2023-10-06T10:42:13.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.864+0000 7ff76c40cb80 1 -- 172.21.15.92:0/1814691948 wait complete. 2023-10-06T10:42:13.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.864+0000 7ff76c40cb80 20 client.6777 trim_cache size 0 max 16384 2023-10-06T10:42:13.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.877+0000 7ff76c40cb80 1 Processor -- start 2023-10-06T10:42:13.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.878+0000 7ff76c40cb80 1 -- start start 2023-10-06T10:42:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.878+0000 7ff76c40cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x565324eacc20 0x565324ef1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.878+0000 7ff76c40cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x565324eae9f0 0x565324ee8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.878+0000 7ff76c40cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324ef2ae0 0x565324ef5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.878+0000 7ff76c40cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x565324c7ee80 con 0x565324ef2ae0 2023-10-06T10:42:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.878+0000 7ff76c40cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x565324ea4270 con 0x565324eacc20 2023-10-06T10:42:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.878+0000 7ff76c40cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x565324ed7c00 con 0x565324eae9f0 2023-10-06T10:42:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.878+0000 7ff7616b1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324ef2ae0 0x565324ef5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.878+0000 7ff7616b1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324ef2ae0 0x565324ef5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53066/0 (socket says 172.21.15.92:53066) 2023-10-06T10:42:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.878+0000 7ff7616b1700 1 -- 172.21.15.92:0/3263403245 learned_addr learned my addr 172.21.15.92:0/3263403245 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.878+0000 7ff753fff700 1 --2- 172.21.15.92:0/3263403245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x565324eae9f0 0x565324ee8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.878+0000 7ff760eb0700 1 --2- 172.21.15.92:0/3263403245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x565324eacc20 0x565324ef1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.879+0000 7ff7616b1700 1 -- 172.21.15.92:0/3263403245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x565324eae9f0 msgr2=0x565324ee8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.879+0000 7ff7616b1700 1 --2- 172.21.15.92:0/3263403245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x565324eae9f0 0x565324ee8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.879+0000 7ff7616b1700 1 -- 172.21.15.92:0/3263403245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x565324eacc20 msgr2=0x565324ef1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.879+0000 7ff7616b1700 1 --2- 172.21.15.92:0/3263403245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x565324eacc20 0x565324ef1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.879+0000 7ff7616b1700 1 -- 172.21.15.92:0/3263403245 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565324dca7d0 con 0x565324ef2ae0 2023-10-06T10:42:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.879+0000 7ff7616b1700 1 --2- 172.21.15.92:0/3263403245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324ef2ae0 0x565324ef5090 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff75400b3c0 tx=0x7ff754017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b82cc784272dd55 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.879+0000 7ff7427fc700 1 -- 172.21.15.92:0/3263403245 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75400bb20 con 0x565324ef2ae0 2023-10-06T10:42:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.879+0000 7ff7427fc700 1 -- 172.21.15.92:0/3263403245 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff754021040 con 0x565324ef2ae0 2023-10-06T10:42:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.879+0000 7ff7427fc700 1 -- 172.21.15.92:0/3263403245 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7540276d0 con 0x565324ef2ae0 2023-10-06T10:42:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.879+0000 7ff76c40cb80 1 -- 172.21.15.92:0/3263403245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324ef2ae0 msgr2=0x565324ef5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.879+0000 7ff76c40cb80 1 --2- 172.21.15.92:0/3263403245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324ef2ae0 0x565324ef5090 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff75400b3c0 tx=0x7ff754017000 comp rx=0 tx=0).stop 2023-10-06T10:42:13.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.879+0000 7ff76c40cb80 1 -- 172.21.15.92:0/3263403245 shutdown_connections 2023-10-06T10:42:13.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.879+0000 7ff76c40cb80 1 --2- 172.21.15.92:0/3263403245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324ef2ae0 0x565324ef5090 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.879+0000 7ff76c40cb80 1 --2- 172.21.15.92:0/3263403245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x565324eacc20 0x565324ef1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.879+0000 7ff76c40cb80 1 --2- 172.21.15.92:0/3263403245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x565324eae9f0 0x565324ee8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.879+0000 7ff76c40cb80 1 -- 172.21.15.92:0/3263403245 >> 172.21.15.92:0/3263403245 conn(0x565324ead0c0 msgr2=0x565324f01440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:13.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.879+0000 7ff76c40cb80 1 -- 172.21.15.92:0/3263403245 shutdown_connections 2023-10-06T10:42:13.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.879+0000 7ff76c40cb80 1 -- 172.21.15.92:0/3263403245 wait complete. 2023-10-06T10:42:13.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.880+0000 7ff76c40cb80 1 -- start start 2023-10-06T10:42:13.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.880+0000 7ff76c40cb80 1 Processor -- start 2023-10-06T10:42:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.881+0000 7ff76c40cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x565324eacc20 0x565324eee520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.881+0000 7ff76c40cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324eae9f0 0x565324eebff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.881+0000 7ff76c40cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x565324ef2ae0 0x565324ed2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.881+0000 7ff76c40cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x565324eeedb0 con 0x565324eae9f0 2023-10-06T10:42:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.881+0000 7ff76c40cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x565324ef52b0 con 0x565324ef2ae0 2023-10-06T10:42:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.881+0000 7ff76c40cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x565324eebb60 con 0x565324eacc20 2023-10-06T10:42:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.881+0000 7ff76c40cb80 10 client.0 _ll_register_callbacks cb 0x565324c98f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:42:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.881+0000 7ff753fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324eae9f0 0x565324eebff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.881+0000 7ff753fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324eae9f0 0x565324eebff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53082/0 (socket says 172.21.15.92:53082) 2023-10-06T10:42:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.881+0000 7ff753fff700 1 -- 172.21.15.92:0/121251488 learned_addr learned my addr 172.21.15.92:0/121251488 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.881+0000 7ff760eb0700 1 --2- 172.21.15.92:0/121251488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x565324eacc20 0x565324eee520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.881+0000 7ff7616b1700 1 --2- 172.21.15.92:0/121251488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x565324ef2ae0 0x565324ed2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.881+0000 7ff753fff700 1 -- 172.21.15.92:0/121251488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x565324eacc20 msgr2=0x565324eee520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.881+0000 7ff753fff700 1 --2- 172.21.15.92:0/121251488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x565324eacc20 0x565324eee520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.881+0000 7ff753fff700 1 -- 172.21.15.92:0/121251488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x565324ef2ae0 msgr2=0x565324ed2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.881+0000 7ff753fff700 1 --2- 172.21.15.92:0/121251488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x565324ef2ae0 0x565324ed2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.881+0000 7ff753fff700 1 -- 172.21.15.92:0/121251488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75c0cd610 con 0x565324eae9f0 2023-10-06T10:42:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.881+0000 7ff753fff700 1 --2- 172.21.15.92:0/121251488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324eae9f0 0x565324eebff0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x565324ed8a60 tx=0x7ff74c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48dd52465114eb92 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.881+0000 7ff73affd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:42:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.882+0000 7ff73affd700 1 -- 172.21.15.92:0/121251488 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c014890 con 0x565324eae9f0 2023-10-06T10:42:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.882+0000 7ff73affd700 1 -- 172.21.15.92:0/121251488 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7ff74c0149f0 con 0x565324eae9f0 2023-10-06T10:42:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.882+0000 7ff76c40cb80 1 -- 172.21.15.92:0/121251488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff758058fe0 con 0x565324eae9f0 2023-10-06T10:42:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.882+0000 7ff73affd700 1 -- 172.21.15.92:0/121251488 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff74c014dc0 con 0x565324eae9f0 2023-10-06T10:42:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.882+0000 7ff76c40cb80 10 client.6783 fetch_fsmap learned FSMap version 57 2023-10-06T10:42:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.882+0000 7ff76c40cb80 1 -- 172.21.15.92:0/121251488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x565324db14c0 con 0x565324eae9f0 2023-10-06T10:42:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.882+0000 7ff73affd700 1 -- 172.21.15.92:0/121251488 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff758058fe0 con 0x565324eae9f0 2023-10-06T10:42:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.882+0000 7ff73affd700 1 -- 172.21.15.92:0/121251488 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 57) v1 ==== 3136+0+0 (secure 0 0 0) 0x7ff74c032060 con 0x565324eae9f0 2023-10-06T10:42:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.882+0000 7ff76c40cb80 10 client.6783 fetch_fsmap finished waiting for FSMap version 57 2023-10-06T10:42:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.882+0000 7ff76c40cb80 10 client.6783 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.a 2023-10-06T10:42:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.882+0000 7ff76c40cb80 20 client.6783 populate_metadata read hostname 'smithi092' 2023-10-06T10:42:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.882+0000 7ff76c40cb80 1 --2- 172.21.15.92:0/121251488 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x565324ef18c0 0x565324ed3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.882+0000 7ff76c40cb80 4 client.6783 mds_command: new command op to 6483 tid=0[{"prefix": "flush journal"}] 2023-10-06T10:42:13.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.882+0000 7ff76c40cb80 1 -- 172.21.15.92:0/121251488 --> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x565324d9ee50 con 0x565324ef18c0 2023-10-06T10:42:13.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.882+0000 7ff760eb0700 1 --2- 172.21.15.92:0/121251488 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x565324ef18c0 0x565324ed3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:13.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.882+0000 7ff73affd700 10 client.6783 ms_handle_connect on v2:172.21.15.92:6834/171578176 2023-10-06T10:42:13.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.882+0000 7ff760eb0700 1 --2- 172.21.15.92:0/121251488 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x565324ef18c0 0x565324ed3140 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b305c79d34d410da server_cookie=f5f14c4111c91c33 in_seq=0 out_seq=0 2023-10-06T10:42:13.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.894+0000 7ff73affd700 1 -- 172.21.15.92:0/121251488 <== mds.1 v2:172.21.15.92:6834/171578176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x565324d9ee50 con 0x565324ef18c0 2023-10-06T10:42:13.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.894+0000 7ff73affd700 10 client.6783 handle_command_reply: tid=0 2023-10-06T10:42:13.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.894+0000 7ff76c40cb80 1 client.6783 shutdown 2023-10-06T10:42:13.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.894+0000 7ff76c40cb80 1 -- 172.21.15.92:0/121251488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324eae9f0 msgr2=0x565324eebff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:13.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.894+0000 7ff76c40cb80 1 --2- 172.21.15.92:0/121251488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324eae9f0 0x565324eebff0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x565324ed8a60 tx=0x7ff74c014000 comp rx=0 tx=0).stop 2023-10-06T10:42:13.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.894+0000 7ff76c40cb80 1 -- 172.21.15.92:0/121251488 shutdown_connections 2023-10-06T10:42:13.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.894+0000 7ff76c40cb80 1 --2- 172.21.15.92:0/121251488 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x565324ef18c0 0x565324ed3140 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.894+0000 7ff76c40cb80 1 --2- 172.21.15.92:0/121251488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x565324ef2ae0 0x565324ed2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.894+0000 7ff73affd700 0 client.6783 ms_handle_reset on v2:172.21.15.92:6834/171578176 2023-10-06T10:42:13.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.894+0000 7ff76c40cb80 1 --2- 172.21.15.92:0/121251488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x565324eacc20 0x565324eee520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.894+0000 7ff76c40cb80 1 --2- 172.21.15.92:0/121251488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565324eae9f0 0x565324eebff0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.894+0000 7ff76c40cb80 1 -- 172.21.15.92:0/121251488 >> 172.21.15.92:0/121251488 conn(0x565324ead0c0 msgr2=0x565324f0b270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:13.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.894+0000 7ff76c40cb80 1 -- 172.21.15.92:0/121251488 shutdown_connections 2023-10-06T10:42:13.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.894+0000 7ff76c40cb80 1 -- 172.21.15.92:0/121251488 wait complete. 2023-10-06T10:42:13.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.894+0000 7ff76c40cb80 20 client.6783 trim_cache size 0 max 16384 2023-10-06T10:42:13.897 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:42:13.897 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2023-10-06T10:42:13.897 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:42:13.897 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:42:13.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.895+0000 7ff763114700 1 -- 172.21.15.92:0/3381969366 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff748066f30 msgr2=0x7ff7480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:13.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.895+0000 7ff763114700 1 --2- 172.21.15.92:0/3381969366 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff748066f30 0x7ff7480693e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0c6940 tx=0x7ff74c007ec0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.895+0000 7ff763114700 1 -- 172.21.15.92:0/3381969366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff75c0689f0 msgr2=0x7ff75c15c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:13.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.895+0000 7ff763114700 1 --2- 172.21.15.92:0/3381969366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff75c0689f0 0x7ff75c15c8e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff758009ef0 tx=0x7ff75800b040 comp rx=0 tx=0).stop 2023-10-06T10:42:13.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.895+0000 7ff763114700 1 -- 172.21.15.92:0/3381969366 shutdown_connections 2023-10-06T10:42:13.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.895+0000 7ff763114700 1 --2- 172.21.15.92:0/3381969366 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff748066f30 0x7ff7480693e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.895+0000 7ff763114700 1 --2- 172.21.15.92:0/3381969366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff75c159420 0x7ff75c16cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.895+0000 7ff763114700 1 --2- 172.21.15.92:0/3381969366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff75c0689f0 0x7ff75c15c8e0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.895+0000 7ff763114700 1 --2- 172.21.15.92:0/3381969366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff75c0cb6d0 0x7ff75c0c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:13.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.895+0000 7ff763114700 1 -- 172.21.15.92:0/3381969366 >> 172.21.15.92:0/3381969366 conn(0x7ff75c06ece0 msgr2=0x7ff75c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:13.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.895+0000 7ff763114700 1 -- 172.21.15.92:0/3381969366 shutdown_connections 2023-10-06T10:42:13.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:13.895+0000 7ff763114700 1 -- 172.21.15.92:0/3381969366 wait complete. 2023-10-06T10:42:13.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:2 flush journal 2023-10-06T10:42:14.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.022+0000 7fe65daec700 1 Processor -- start 2023-10-06T10:42:14.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.022+0000 7fe65daec700 1 -- start start 2023-10-06T10:42:14.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.022+0000 7fe65daec700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe658150940 0x7fe658159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:14.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.023+0000 7fe65daec700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe65815a160 0x7fe6580ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:14.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.023+0000 7fe65daec700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe65815ac30 con 0x7fe6580c5250 2023-10-06T10:42:14.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.023+0000 7fe65daec700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6580cb6a0 con 0x7fe658150940 2023-10-06T10:42:14.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.023+0000 7fe65daec700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6580cb090 con 0x7fe65815a160 2023-10-06T10:42:14.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.023+0000 7fe6577fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe658150940 0x7fe658159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:14.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.023+0000 7fe657fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe6580c5250 0x7fe6580b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56458/0 (socket says 172.21.15.92:56458) 2023-10-06T10:42:14.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.023+0000 7fe6577fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe658150940 0x7fe658159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53088/0 (socket says 172.21.15.92:53088) 2023-10-06T10:42:14.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.023+0000 7fe657fff700 1 -- 172.21.15.92:0/3854028861 learned_addr learned my addr 172.21.15.92:0/3854028861 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:42:14.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.023+0000 7fe656ffd700 1 --2- 172.21.15.92:0/3854028861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe65815a160 0x7fe6580ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:14.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.023+0000 7fe6577fe700 1 -- 172.21.15.92:0/3854028861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe65815a160 msgr2=0x7fe6580ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:14.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.023+0000 7fe6577fe700 1 --2- 172.21.15.92:0/3854028861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe65815a160 0x7fe6580ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.023+0000 7fe6577fe700 1 -- 172.21.15.92:0/3854028861 >> v1:172.21.15.92:6789/0 conn(0x7fe6580c5250 legacy=0x7fe6580b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:14.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.023+0000 7fe6577fe700 1 -- 172.21.15.92:0/3854028861 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65815a670 con 0x7fe658150940 2023-10-06T10:42:14.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.023+0000 7fe6567fc700 1 -- 172.21.15.92:0/3854028861 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24912864 0 0) 0x7fe650001490 con 0x7fe6580c5250 2023-10-06T10:42:14.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.023+0000 7fe656ffd700 1 --2- 172.21.15.92:0/3854028861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe65815a160 0x7fe6580ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:14.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.023+0000 7fe6577fe700 1 --2- 172.21.15.92:0/3854028861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe658150940 0x7fe658159c20 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe6400098d0 tx=0x7fe640009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72960dcce871a20a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:14.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.023+0000 7fe6567fc700 1 -- 172.21.15.92:0/3854028861 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe640009e70 con 0x7fe658150940 2023-10-06T10:42:14.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.023+0000 7fe6567fc700 1 -- 172.21.15.92:0/3854028861 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe640005c10 con 0x7fe658150940 2023-10-06T10:42:14.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.023+0000 7fe6567fc700 1 -- 172.21.15.92:0/3854028861 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64000b650 con 0x7fe658150940 2023-10-06T10:42:14.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.024+0000 7fe65daec700 1 -- 172.21.15.92:0/3854028861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe658150940 msgr2=0x7fe658159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:14.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.024+0000 7fe65daec700 1 --2- 172.21.15.92:0/3854028861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe658150940 0x7fe658159c20 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe6400098d0 tx=0x7fe640009120 comp rx=0 tx=0).stop 2023-10-06T10:42:14.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.024+0000 7fe65daec700 1 -- 172.21.15.92:0/3854028861 shutdown_connections 2023-10-06T10:42:14.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.024+0000 7fe65daec700 1 --2- 172.21.15.92:0/3854028861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe658150940 0x7fe658159c20 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.024+0000 7fe65daec700 1 --2- 172.21.15.92:0/3854028861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe65815a160 0x7fe6580ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.024+0000 7fe65daec700 1 -- 172.21.15.92:0/3854028861 >> 172.21.15.92:0/3854028861 conn(0x7fe658099700 msgr2=0x7fe65806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:14.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.024+0000 7fe65daec700 1 -- 172.21.15.92:0/3854028861 shutdown_connections 2023-10-06T10:42:14.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.024+0000 7fe65daec700 1 -- 172.21.15.92:0/3854028861 wait complete. 2023-10-06T10:42:14.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.025+0000 7fe65daec700 1 Processor -- start 2023-10-06T10:42:14.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.025+0000 7fe65daec700 1 -- start start 2023-10-06T10:42:14.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.026+0000 7fe65daec700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe6580c5250 0x7fe65814ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:14.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.026+0000 7fe65daec700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe65815a160 0x7fe6580cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:14.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.026+0000 7fe65daec700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe658156eb0 0x7fe6580cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:14.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.026+0000 7fe65daec700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65816a280 con 0x7fe65815a160 2023-10-06T10:42:14.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.026+0000 7fe65daec700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe658169970 con 0x7fe658156eb0 2023-10-06T10:42:14.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.026+0000 7fe65daec700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe658157420 con 0x7fe6580c5250 2023-10-06T10:42:14.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.026+0000 7fe656ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe65815a160 0x7fe6580cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:14.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.026+0000 7fe656ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe65815a160 0x7fe6580cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53092/0 (socket says 172.21.15.92:53092) 2023-10-06T10:42:14.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.026+0000 7fe656ffd700 1 -- 172.21.15.92:0/2619971611 learned_addr learned my addr 172.21.15.92:0/2619971611 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:14.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.026+0000 7fe6577fe700 1 --2- 172.21.15.92:0/2619971611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe6580c5250 0x7fe65814ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:14.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.026+0000 7fe657fff700 1 --2- 172.21.15.92:0/2619971611 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe658156eb0 0x7fe6580cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:14.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.026+0000 7fe656ffd700 1 -- 172.21.15.92:0/2619971611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe6580c5250 msgr2=0x7fe65814ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:14.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.026+0000 7fe656ffd700 1 --2- 172.21.15.92:0/2619971611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe6580c5250 0x7fe65814ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.026+0000 7fe656ffd700 1 -- 172.21.15.92:0/2619971611 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe658156eb0 msgr2=0x7fe6580cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:14.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.026+0000 7fe656ffd700 1 --2- 172.21.15.92:0/2619971611 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe658156eb0 0x7fe6580cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.026+0000 7fe656ffd700 1 -- 172.21.15.92:0/2619971611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6400093c0 con 0x7fe65815a160 2023-10-06T10:42:14.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.026+0000 7fe657fff700 1 --2- 172.21.15.92:0/2619971611 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe658156eb0 0x7fe6580cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:14.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.026+0000 7fe656ffd700 1 --2- 172.21.15.92:0/2619971611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe65815a160 0x7fe6580cbb30 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fe64800c800 tx=0x7fe648008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.027+0000 7fe64f7fe700 1 -- 172.21.15.92:0/2619971611 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe648011600 con 0x7fe65815a160 2023-10-06T10:42:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.027+0000 7fe64f7fe700 1 -- 172.21.15.92:0/2619971611 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe648011760 con 0x7fe65815a160 2023-10-06T10:42:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.027+0000 7fe65daec700 1 -- 172.21.15.92:0/2619971611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe658169c30 con 0x7fe65815a160 2023-10-06T10:42:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.027+0000 7fe64f7fe700 1 -- 172.21.15.92:0/2619971611 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe648011ae0 con 0x7fe65815a160 2023-10-06T10:42:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.027+0000 7fe65daec700 1 -- 172.21.15.92:0/2619971611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe658150e70 con 0x7fe65815a160 2023-10-06T10:42:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.028+0000 7fe64f7fe700 1 -- 172.21.15.92:0/2619971611 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe648021350 con 0x7fe65815a160 2023-10-06T10:42:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.029+0000 7fe64f7fe700 1 --2- 172.21.15.92:0/2619971611 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe638066f30 0x7fe6380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.029+0000 7fe64f7fe700 1 -- 172.21.15.92:0/2619971611 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fe648084760 con 0x7fe65815a160 2023-10-06T10:42:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.029+0000 7fe6577fe700 1 --2- 172.21.15.92:0/2619971611 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe638066f30 0x7fe6380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.029+0000 7fe6577fe700 1 --2- 172.21.15.92:0/2619971611 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe638066f30 0x7fe6380693e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe640009160 tx=0x7fe640016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:14.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.030+0000 7fe666de4b80 1 Processor -- start 2023-10-06T10:42:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.031+0000 7fe666de4b80 1 -- start start 2023-10-06T10:42:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.031+0000 7fe666de4b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561947be1ae0 0x561947be1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.031+0000 7fe666de4b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561947b9d9f0 0x561947b9a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.031+0000 7fe666de4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947b9bc20 0x561947b9ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.031+0000 7fe666de4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561947b1acd0 con 0x561947b9bc20 2023-10-06T10:42:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.031+0000 7fe666de4b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561947af1ac0 con 0x561947b9d9f0 2023-10-06T10:42:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.031+0000 7fe666de4b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561947aef2c0 con 0x561947be1ae0 2023-10-06T10:42:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.032+0000 7fe657fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947b9bc20 0x561947b9ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.032+0000 7fe657fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947b9bc20 0x561947b9ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53104/0 (socket says 172.21.15.92:53104) 2023-10-06T10:42:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.032+0000 7fe657fff700 1 -- 172.21.15.92:0/145043693 learned_addr learned my addr 172.21.15.92:0/145043693 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.032+0000 7fe656ffd700 1 --2- 172.21.15.92:0/145043693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561947b9d9f0 0x561947b9a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.032+0000 7fe6577fe700 1 --2- 172.21.15.92:0/145043693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561947be1ae0 0x561947be1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.032+0000 7fe657fff700 1 -- 172.21.15.92:0/145043693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561947be1ae0 msgr2=0x561947be1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.032+0000 7fe657fff700 1 --2- 172.21.15.92:0/145043693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561947be1ae0 0x561947be1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.032+0000 7fe657fff700 1 -- 172.21.15.92:0/145043693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561947b9d9f0 msgr2=0x561947b9a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.032+0000 7fe657fff700 1 --2- 172.21.15.92:0/145043693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561947b9d9f0 0x561947b9a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.032+0000 7fe657fff700 1 -- 172.21.15.92:0/145043693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619479852c0 con 0x561947b9bc20 2023-10-06T10:42:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.032+0000 7fe656ffd700 1 --2- 172.21.15.92:0/145043693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561947b9d9f0 0x561947b9a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.032+0000 7fe657fff700 1 --2- 172.21.15.92:0/145043693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947b9bc20 0x561947b9ab10 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fe6500021e0 tx=0x7fe650009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42862faae52f519e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.032+0000 7fe64cff9700 1 -- 172.21.15.92:0/145043693 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65000edd0 con 0x561947b9bc20 2023-10-06T10:42:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.032+0000 7fe64cff9700 1 -- 172.21.15.92:0/145043693 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe650014480 con 0x561947b9bc20 2023-10-06T10:42:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.032+0000 7fe64cff9700 1 -- 172.21.15.92:0/145043693 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650014820 con 0x561947b9bc20 2023-10-06T10:42:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.033+0000 7fe666de4b80 1 -- 172.21.15.92:0/145043693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947b9bc20 msgr2=0x561947b9ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.033+0000 7fe666de4b80 1 --2- 172.21.15.92:0/145043693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947b9bc20 0x561947b9ab10 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fe6500021e0 tx=0x7fe650009040 comp rx=0 tx=0).stop 2023-10-06T10:42:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.033+0000 7fe666de4b80 1 -- 172.21.15.92:0/145043693 shutdown_connections 2023-10-06T10:42:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.033+0000 7fe666de4b80 1 --2- 172.21.15.92:0/145043693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947b9bc20 0x561947b9ab10 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.033+0000 7fe666de4b80 1 --2- 172.21.15.92:0/145043693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561947be1ae0 0x561947be1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.033+0000 7fe666de4b80 1 --2- 172.21.15.92:0/145043693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561947b9d9f0 0x561947b9a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.033+0000 7fe666de4b80 1 -- 172.21.15.92:0/145043693 >> 172.21.15.92:0/145043693 conn(0x561947b9c0c0 msgr2=0x561947b9c490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.033+0000 7fe666de4b80 1 -- 172.21.15.92:0/145043693 shutdown_connections 2023-10-06T10:42:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.033+0000 7fe666de4b80 1 -- 172.21.15.92:0/145043693 wait complete. 2023-10-06T10:42:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.033+0000 7fe666de4b80 1 -- start start 2023-10-06T10:42:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.034+0000 7fe666de4b80 1 Processor -- start 2023-10-06T10:42:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.034+0000 7fe666de4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947b9bc20 0x561947bc1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.034+0000 7fe666de4b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561947b9d9f0 0x561947bc6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:14.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.034+0000 7fe666de4b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561947be1ae0 0x561947b9a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:14.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.034+0000 7fe666de4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561947aeb0b0 con 0x561947b9bc20 2023-10-06T10:42:14.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.034+0000 7fe666de4b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561947a968e0 con 0x561947be1ae0 2023-10-06T10:42:14.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.034+0000 7fe666de4b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561947ab1ef0 con 0x561947b9d9f0 2023-10-06T10:42:14.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.034+0000 7fe666de4b80 10 client.0 _ll_register_callbacks cb 0x561947987f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:42:14.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.034+0000 7fe6577fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947b9bc20 0x561947bc1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:14.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.034+0000 7fe6577fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947b9bc20 0x561947bc1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53112/0 (socket says 172.21.15.92:53112) 2023-10-06T10:42:14.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.034+0000 7fe6577fe700 1 -- 172.21.15.92:0/1198584310 learned_addr learned my addr 172.21.15.92:0/1198584310 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:14.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.034+0000 7fe657fff700 1 --2- 172.21.15.92:0/1198584310 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561947be1ae0 0x561947b9a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:14.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.034+0000 7fe656ffd700 1 --2- 172.21.15.92:0/1198584310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561947b9d9f0 0x561947bc6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:14.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.035+0000 7fe6577fe700 1 -- 172.21.15.92:0/1198584310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561947b9d9f0 msgr2=0x561947bc6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:14.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.035+0000 7fe6577fe700 1 --2- 172.21.15.92:0/1198584310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561947b9d9f0 0x561947bc6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.035+0000 7fe6577fe700 1 -- 172.21.15.92:0/1198584310 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561947be1ae0 msgr2=0x561947b9a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:14.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.035+0000 7fe6577fe700 1 --2- 172.21.15.92:0/1198584310 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561947be1ae0 0x561947b9a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.035+0000 7fe6577fe700 1 -- 172.21.15.92:0/1198584310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65000f040 con 0x561947b9bc20 2023-10-06T10:42:14.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.035+0000 7fe6577fe700 1 --2- 172.21.15.92:0/1198584310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947b9bc20 0x561947bc1440 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe64000ba90 tx=0x7fe640023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349212e0f1e35143 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:14.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.035+0000 7fe635ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:42:14.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.035+0000 7fe635ffb700 1 -- 172.21.15.92:0/1198584310 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe640031070 con 0x561947b9bc20 2023-10-06T10:42:14.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.035+0000 7fe635ffb700 1 -- 172.21.15.92:0/1198584310 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fe64000be70 con 0x561947b9bc20 2023-10-06T10:42:14.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.035+0000 7fe666de4b80 1 -- 172.21.15.92:0/1198584310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561947b8f6e0 con 0x561947b9bc20 2023-10-06T10:42:14.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.035+0000 7fe635ffb700 1 -- 172.21.15.92:0/1198584310 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe640023070 con 0x561947b9bc20 2023-10-06T10:42:14.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.035+0000 7fe666de4b80 10 client.6798 fetch_fsmap learned FSMap version 57 2023-10-06T10:42:14.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.035+0000 7fe666de4b80 1 -- 172.21.15.92:0/1198584310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x561947928030 con 0x561947b9bc20 2023-10-06T10:42:14.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.035+0000 7fe635ffb700 1 -- 172.21.15.92:0/1198584310 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561947b8f6e0 con 0x561947b9bc20 2023-10-06T10:42:14.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.035+0000 7fe635ffb700 1 -- 172.21.15.92:0/1198584310 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 57) v1 ==== 3136+0+0 (secure 0 0 0) 0x7fe64002b420 con 0x561947b9bc20 2023-10-06T10:42:14.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.035+0000 7fe666de4b80 10 client.6798 fetch_fsmap finished waiting for FSMap version 57 2023-10-06T10:42:14.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.035+0000 7fe666de4b80 10 client.6798 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.b 2023-10-06T10:42:14.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.035+0000 7fe666de4b80 20 client.6798 populate_metadata read hostname 'smithi092' 2023-10-06T10:42:14.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.035+0000 7fe666de4b80 1 --2- 172.21.15.92:0/1198584310 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x561947b9ab10 0x561947be4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:14.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.035+0000 7fe666de4b80 4 client.6798 mds_command: new command op to 6019 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:42:14.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.035+0000 7fe666de4b80 1 -- 172.21.15.92:0/1198584310 --> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561947b6c610 con 0x561947b9ab10 2023-10-06T10:42:14.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.036+0000 7fe656ffd700 1 --2- 172.21.15.92:0/1198584310 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x561947b9ab10 0x561947be4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:14.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.036+0000 7fe635ffb700 10 client.6798 ms_handle_connect on v2:172.21.15.124:6833/2297612118 2023-10-06T10:42:14.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.036+0000 7fe656ffd700 1 --2- 172.21.15.92:0/1198584310 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x561947b9ab10 0x561947be4ea0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e21a9e53faba7a58 server_cookie=24ff785bec7087d8 in_seq=0 out_seq=0 2023-10-06T10:42:14.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.036+0000 7fe635ffb700 1 -- 172.21.15.92:0/1198584310 <== mds.2 v2:172.21.15.124:6833/2297612118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x561947b6c610 con 0x561947b9ab10 2023-10-06T10:42:14.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.036+0000 7fe635ffb700 10 client.6798 handle_command_reply: tid=0 2023-10-06T10:42:14.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.036+0000 7fe666de4b80 1 client.6798 shutdown 2023-10-06T10:42:14.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.037+0000 7fe666de4b80 1 -- 172.21.15.92:0/1198584310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947b9bc20 msgr2=0x561947bc1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:14.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.037+0000 7fe666de4b80 1 --2- 172.21.15.92:0/1198584310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947b9bc20 0x561947bc1440 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe64000ba90 tx=0x7fe640023000 comp rx=0 tx=0).stop 2023-10-06T10:42:14.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.037+0000 7fe666de4b80 1 -- 172.21.15.92:0/1198584310 shutdown_connections 2023-10-06T10:42:14.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.037+0000 7fe666de4b80 1 --2- 172.21.15.92:0/1198584310 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x561947b9ab10 0x561947be4ea0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.037+0000 7fe666de4b80 1 --2- 172.21.15.92:0/1198584310 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561947be1ae0 0x561947b9a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.037+0000 7fe666de4b80 1 --2- 172.21.15.92:0/1198584310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947b9bc20 0x561947bc1440 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.037+0000 7fe666de4b80 1 --2- 172.21.15.92:0/1198584310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561947b9d9f0 0x561947bc6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.037+0000 7fe635ffb700 0 client.6798 ms_handle_reset on v2:172.21.15.124:6833/2297612118 2023-10-06T10:42:14.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.037+0000 7fe666de4b80 1 -- 172.21.15.92:0/1198584310 >> 172.21.15.92:0/1198584310 conn(0x561947b9c0c0 msgr2=0x561947b97680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:14.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.037+0000 7fe666de4b80 1 -- 172.21.15.92:0/1198584310 shutdown_connections 2023-10-06T10:42:14.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.037+0000 7fe666de4b80 1 -- 172.21.15.92:0/1198584310 wait complete. 2023-10-06T10:42:14.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.037+0000 7fe666de4b80 20 client.6798 trim_cache size 0 max 16384 2023-10-06T10:42:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.050+0000 7fe666de4b80 1 Processor -- start 2023-10-06T10:42:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.050+0000 7fe666de4b80 1 -- start start 2023-10-06T10:42:14.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.051+0000 7fe666de4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947b9bc20 0x561947b969d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.051+0000 7fe666de4b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561947b9d9f0 0x561947bc05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.051+0000 7fe666de4b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561947be1ae0 0x561947bc46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.051+0000 7fe666de4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56194796de80 con 0x561947b9bc20 2023-10-06T10:42:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.051+0000 7fe666de4b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561947b7a120 con 0x561947be1ae0 2023-10-06T10:42:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.051+0000 7fe666de4b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561947be2200 con 0x561947b9d9f0 2023-10-06T10:42:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.051+0000 7fe6577fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947b9bc20 0x561947b969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.051+0000 7fe6577fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947b9bc20 0x561947b969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53130/0 (socket says 172.21.15.92:53130) 2023-10-06T10:42:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.051+0000 7fe6577fe700 1 -- 172.21.15.92:0/1242827304 learned_addr learned my addr 172.21.15.92:0/1242827304 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.051+0000 7fe656ffd700 1 --2- 172.21.15.92:0/1242827304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561947b9d9f0 0x561947bc05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.051+0000 7fe657fff700 1 --2- 172.21.15.92:0/1242827304 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561947be1ae0 0x561947bc46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.051+0000 7fe6577fe700 1 -- 172.21.15.92:0/1242827304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561947b9d9f0 msgr2=0x561947bc05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.051+0000 7fe6577fe700 1 --2- 172.21.15.92:0/1242827304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561947b9d9f0 0x561947bc05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.051+0000 7fe6577fe700 1 -- 172.21.15.92:0/1242827304 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561947be1ae0 msgr2=0x561947bc46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.051+0000 7fe6577fe700 1 --2- 172.21.15.92:0/1242827304 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561947be1ae0 0x561947bc46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.051+0000 7fe6577fe700 1 -- 172.21.15.92:0/1242827304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561947ab97d0 con 0x561947b9bc20 2023-10-06T10:42:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.051+0000 7fe657fff700 1 --2- 172.21.15.92:0/1242827304 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561947be1ae0 0x561947bc46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.051+0000 7fe656ffd700 1 --2- 172.21.15.92:0/1242827304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561947b9d9f0 0x561947bc05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.052+0000 7fe6577fe700 1 --2- 172.21.15.92:0/1242827304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947b9bc20 0x561947b969d0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fe640009f30 tx=0x7fe640031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab01f95ff130f25a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.052+0000 7fe64d7fa700 1 -- 172.21.15.92:0/1242827304 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe640023040 con 0x561947b9bc20 2023-10-06T10:42:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.052+0000 7fe64d7fa700 1 -- 172.21.15.92:0/1242827304 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe640006930 con 0x561947b9bc20 2023-10-06T10:42:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.052+0000 7fe64d7fa700 1 -- 172.21.15.92:0/1242827304 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64002b420 con 0x561947b9bc20 2023-10-06T10:42:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.052+0000 7fe666de4b80 1 -- 172.21.15.92:0/1242827304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947b9bc20 msgr2=0x561947b969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.052+0000 7fe666de4b80 1 --2- 172.21.15.92:0/1242827304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947b9bc20 0x561947b969d0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fe640009f30 tx=0x7fe640031000 comp rx=0 tx=0).stop 2023-10-06T10:42:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.052+0000 7fe666de4b80 1 -- 172.21.15.92:0/1242827304 shutdown_connections 2023-10-06T10:42:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.052+0000 7fe666de4b80 1 --2- 172.21.15.92:0/1242827304 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561947be1ae0 0x561947bc46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.052+0000 7fe666de4b80 1 --2- 172.21.15.92:0/1242827304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947b9bc20 0x561947b969d0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.052+0000 7fe666de4b80 1 --2- 172.21.15.92:0/1242827304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561947b9d9f0 0x561947bc05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.052+0000 7fe666de4b80 1 -- 172.21.15.92:0/1242827304 >> 172.21.15.92:0/1242827304 conn(0x561947b9c0c0 msgr2=0x561947bc6990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.052+0000 7fe666de4b80 1 -- 172.21.15.92:0/1242827304 shutdown_connections 2023-10-06T10:42:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.052+0000 7fe666de4b80 1 -- 172.21.15.92:0/1242827304 wait complete. 2023-10-06T10:42:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.053+0000 7fe666de4b80 1 -- start start 2023-10-06T10:42:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.053+0000 7fe666de4b80 1 Processor -- start 2023-10-06T10:42:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe666de4b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561947b9bc20 0x561947bca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe666de4b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561947b9d9f0 0x561947b97680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe666de4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947be1ae0 0x561947b97bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:14.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe666de4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561947bdda10 con 0x561947be1ae0 2023-10-06T10:42:14.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe666de4b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561947bddc50 con 0x561947b9bc20 2023-10-06T10:42:14.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe666de4b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561947c46930 con 0x561947b9d9f0 2023-10-06T10:42:14.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe666de4b80 10 client.0 _ll_register_callbacks cb 0x561947987f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:42:14.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe657fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947be1ae0 0x561947b97bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:14.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe657fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947be1ae0 0x561947b97bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53144/0 (socket says 172.21.15.92:53144) 2023-10-06T10:42:14.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe657fff700 1 -- 172.21.15.92:0/1918196826 learned_addr learned my addr 172.21.15.92:0/1918196826 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:14.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe6577fe700 1 --2- 172.21.15.92:0/1918196826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561947b9bc20 0x561947bca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:14.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe656ffd700 1 --2- 172.21.15.92:0/1918196826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561947b9d9f0 0x561947b97680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:14.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe657fff700 1 -- 172.21.15.92:0/1918196826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561947b9d9f0 msgr2=0x561947b97680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:14.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe657fff700 1 --2- 172.21.15.92:0/1918196826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561947b9d9f0 0x561947b97680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe657fff700 1 -- 172.21.15.92:0/1918196826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561947b9bc20 msgr2=0x561947bca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:14.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe657fff700 1 --2- 172.21.15.92:0/1918196826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561947b9bc20 0x561947bca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe657fff700 1 -- 172.21.15.92:0/1918196826 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561947928030 con 0x561947be1ae0 2023-10-06T10:42:14.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe657fff700 1 --2- 172.21.15.92:0/1918196826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947be1ae0 0x561947b97bc0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fe650013000 tx=0x7fe650019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d0d707ee68ac45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:14.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe635ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:42:14.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe635ffb700 1 -- 172.21.15.92:0/1918196826 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65000ee50 con 0x561947be1ae0 2023-10-06T10:42:14.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe635ffb700 1 -- 172.21.15.92:0/1918196826 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fe650014800 con 0x561947be1ae0 2023-10-06T10:42:14.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe666de4b80 1 -- 172.21.15.92:0/1918196826 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe650014480 con 0x561947be1ae0 2023-10-06T10:42:14.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe635ffb700 1 -- 172.21.15.92:0/1918196826 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe650004dc0 con 0x561947be1ae0 2023-10-06T10:42:14.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe666de4b80 10 client.6804 fetch_fsmap learned FSMap version 57 2023-10-06T10:42:14.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.054+0000 7fe666de4b80 1 -- 172.21.15.92:0/1918196826 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x561947aa04c0 con 0x561947be1ae0 2023-10-06T10:42:14.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.055+0000 7fe635ffb700 1 -- 172.21.15.92:0/1918196826 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe650014480 con 0x561947be1ae0 2023-10-06T10:42:14.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.055+0000 7fe635ffb700 1 -- 172.21.15.92:0/1918196826 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 57) v1 ==== 3136+0+0 (secure 0 0 0) 0x7fe650005020 con 0x561947be1ae0 2023-10-06T10:42:14.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.055+0000 7fe666de4b80 10 client.6804 fetch_fsmap finished waiting for FSMap version 57 2023-10-06T10:42:14.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.055+0000 7fe666de4b80 10 client.6804 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.b 2023-10-06T10:42:14.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.055+0000 7fe666de4b80 20 client.6804 populate_metadata read hostname 'smithi092' 2023-10-06T10:42:14.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.055+0000 7fe666de4b80 1 --2- 172.21.15.92:0/1918196826 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x561947c46dc0 0x561947c496d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:14.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.055+0000 7fe666de4b80 4 client.6804 mds_command: new command op to 6019 tid=0[{"prefix": "flush journal"}] 2023-10-06T10:42:14.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.055+0000 7fe666de4b80 1 -- 172.21.15.92:0/1918196826 --> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561947a8de50 con 0x561947c46dc0 2023-10-06T10:42:14.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.055+0000 7fe6577fe700 1 --2- 172.21.15.92:0/1918196826 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x561947c46dc0 0x561947c496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:14.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.055+0000 7fe635ffb700 10 client.6804 ms_handle_connect on v2:172.21.15.124:6833/2297612118 2023-10-06T10:42:14.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.055+0000 7fe6577fe700 1 --2- 172.21.15.92:0/1918196826 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x561947c46dc0 0x561947c496d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a04547a4c7b294a3 server_cookie=c7db3448712193f3 in_seq=0 out_seq=0 2023-10-06T10:42:14.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.066+0000 7fe635ffb700 1 -- 172.21.15.92:0/1918196826 <== mds.2 v2:172.21.15.124:6833/2297612118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561947a8de50 con 0x561947c46dc0 2023-10-06T10:42:14.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.066+0000 7fe635ffb700 10 client.6804 handle_command_reply: tid=0 2023-10-06T10:42:14.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.066+0000 7fe666de4b80 1 client.6804 shutdown 2023-10-06T10:42:14.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.066+0000 7fe666de4b80 1 -- 172.21.15.92:0/1918196826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947be1ae0 msgr2=0x561947b97bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:14.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.066+0000 7fe666de4b80 1 --2- 172.21.15.92:0/1918196826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947be1ae0 0x561947b97bc0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fe650013000 tx=0x7fe650019000 comp rx=0 tx=0).stop 2023-10-06T10:42:14.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.066+0000 7fe666de4b80 1 -- 172.21.15.92:0/1918196826 shutdown_connections 2023-10-06T10:42:14.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.066+0000 7fe666de4b80 1 --2- 172.21.15.92:0/1918196826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561947be1ae0 0x561947b97bc0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.066+0000 7fe666de4b80 1 --2- 172.21.15.92:0/1918196826 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x561947c46dc0 0x561947c496d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.066+0000 7fe666de4b80 1 --2- 172.21.15.92:0/1918196826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561947b9bc20 0x561947bca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.066+0000 7fe635ffb700 0 client.6804 ms_handle_reset on v2:172.21.15.124:6833/2297612118 2023-10-06T10:42:14.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.066+0000 7fe666de4b80 1 --2- 172.21.15.92:0/1918196826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561947b9d9f0 0x561947b97680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.066+0000 7fe666de4b80 1 -- 172.21.15.92:0/1918196826 >> 172.21.15.92:0/1918196826 conn(0x561947b9c0c0 msgr2=0x561947be02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:14.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.066+0000 7fe666de4b80 1 -- 172.21.15.92:0/1918196826 shutdown_connections 2023-10-06T10:42:14.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.066+0000 7fe666de4b80 1 -- 172.21.15.92:0/1918196826 wait complete. 2023-10-06T10:42:14.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.066+0000 7fe666de4b80 20 client.6804 trim_cache size 0 max 16384 2023-10-06T10:42:14.069 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:42:14.069 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2023-10-06T10:42:14.069 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:42:14.069 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:42:14.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.067+0000 7fe65daec700 1 -- 172.21.15.92:0/2619971611 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe638066f30 msgr2=0x7fe6380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:14.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.067+0000 7fe65daec700 1 --2- 172.21.15.92:0/2619971611 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe638066f30 0x7fe6380693e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe640009160 tx=0x7fe640016000 comp rx=0 tx=0).stop 2023-10-06T10:42:14.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.067+0000 7fe65daec700 1 -- 172.21.15.92:0/2619971611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe65815a160 msgr2=0x7fe6580cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:14.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.067+0000 7fe65daec700 1 --2- 172.21.15.92:0/2619971611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe65815a160 0x7fe6580cbb30 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fe64800c800 tx=0x7fe648008040 comp rx=0 tx=0).stop 2023-10-06T10:42:14.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.067+0000 7fe65daec700 1 -- 172.21.15.92:0/2619971611 shutdown_connections 2023-10-06T10:42:14.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.067+0000 7fe65daec700 1 --2- 172.21.15.92:0/2619971611 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe638066f30 0x7fe6380693e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.067+0000 7fe65daec700 1 --2- 172.21.15.92:0/2619971611 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe658156eb0 0x7fe6580cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.067+0000 7fe65daec700 1 --2- 172.21.15.92:0/2619971611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe6580c5250 0x7fe65814ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.067+0000 7fe65daec700 1 --2- 172.21.15.92:0/2619971611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe65815a160 0x7fe6580cbb30 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:14.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.067+0000 7fe65daec700 1 -- 172.21.15.92:0/2619971611 >> 172.21.15.92:0/2619971611 conn(0x7fe658099700 msgr2=0x7fe658159850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:14.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.067+0000 7fe65daec700 1 -- 172.21.15.92:0/2619971611 shutdown_connections 2023-10-06T10:42:14.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:14.067+0000 7fe65daec700 1 -- 172.21.15.92:0/2619971611 wait complete. 2023-10-06T10:42:14.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:42:14.079 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2023-10-06T10:42:14.102 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/ 2023-10-06T10:42:14.102 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4 2023-10-06T10:42:14.102 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4/d5 2023-10-06T10:42:14.102 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4/d5/d6 2023-10-06T10:42:14.102 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4/d5/d6/d7 2023-10-06T10:42:14.102 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2023-10-06T10:42:14.103 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:42:14.103 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:42:14.103 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:42:14.103 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:42:14.103 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2023-10-06T10:42:14.104 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:42:14.207 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627778 2023-10-06T10:42:14.396 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:42:14.396 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:42:14.396 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:42:14.396 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:42:14.397 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2023-10-06T10:42:14.397 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:42:14.500 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627779 2023-10-06T10:42:14.689 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:42:14.690 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:42:14.690 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:42:14.690 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:42:14.690 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2023-10-06T10:42:14.690 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:42:14.793 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627780 2023-10-06T10:42:14.982 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:42:14.982 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:42:14.983 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:42:14.983 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:42:14.983 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2023-10-06T10:42:14.983 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:42:15.087 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627781 2023-10-06T10:42:15.276 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:42:15.276 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:42:15.276 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:42:15.276 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:42:15.276 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2023-10-06T10:42:15.277 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:42:15.380 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627782 2023-10-06T10:42:15.569 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:42:15.569 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:42:15.569 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:42:15.569 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:42:15.569 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2023-10-06T10:42:15.570 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:42:15.673 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627783 2023-10-06T10:42:15.862 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2023-10-06T10:42:15.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.939+0000 7f68bb4f1000 1 Processor -- start 2023-10-06T10:42:15.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.939+0000 7f68bb4f1000 1 -- start start 2023-10-06T10:42:15.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.940+0000 7f68bb4f1000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563230e1a9c0 0x563230e58ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:15.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.940+0000 7f68bb4f1000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563230e1ad90 0x563230e580a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:15.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.940+0000 7f68bb4f1000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563230f00c10 0x563230e72710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:15.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.940+0000 7f68bb4f1000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563230d77980 con 0x563230e1a9c0 2023-10-06T10:42:15.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.940+0000 7f68bb4f1000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563230d77620 con 0x563230f00c10 2023-10-06T10:42:15.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.940+0000 7f68bb4f1000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563230d6e400 con 0x563230e1ad90 2023-10-06T10:42:15.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.940+0000 7f68b472c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563230e1a9c0 0x563230e58ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:15.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.940+0000 7f68b472c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563230e1a9c0 0x563230e58ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53156/0 (socket says 172.21.15.92:53156) 2023-10-06T10:42:15.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.940+0000 7f68b472c700 1 -- 172.21.15.92:0/1149214715 learned_addr learned my addr 172.21.15.92:0/1149214715 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:15.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.940+0000 7f68b4f2d700 1 --2- 172.21.15.92:0/1149214715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563230f00c10 0x563230e72710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:15.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.940+0000 7f68b472c700 1 -- 172.21.15.92:0/1149214715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563230e1ad90 msgr2=0x563230e580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:15.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.940+0000 7f68abfff700 1 --2- 172.21.15.92:0/1149214715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563230e1ad90 0x563230e580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:15.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.941+0000 7f68b472c700 1 --2- 172.21.15.92:0/1149214715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563230e1ad90 0x563230e580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:15.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.941+0000 7f68b472c700 1 -- 172.21.15.92:0/1149214715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563230f00c10 msgr2=0x563230e72710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:15.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.941+0000 7f68b472c700 1 --2- 172.21.15.92:0/1149214715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563230f00c10 0x563230e72710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:15.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.941+0000 7f68b472c700 1 -- 172.21.15.92:0/1149214715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac00a180 con 0x563230e1a9c0 2023-10-06T10:42:15.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.941+0000 7f68b4f2d700 1 --2- 172.21.15.92:0/1149214715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563230f00c10 0x563230e72710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:15.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.941+0000 7f68abfff700 1 --2- 172.21.15.92:0/1149214715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563230e1ad90 0x563230e580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:15.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.941+0000 7f68b472c700 1 --2- 172.21.15.92:0/1149214715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563230e1a9c0 0x563230e58ad0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009670 tx=0x7f68a4005b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6200792495c332 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:15.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.941+0000 7f68b372a700 1 -- 172.21.15.92:0/1149214715 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a400e070 con 0x563230e1a9c0 2023-10-06T10:42:15.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.941+0000 7f68b372a700 1 -- 172.21.15.92:0/1149214715 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a40064d0 con 0x563230e1a9c0 2023-10-06T10:42:15.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.941+0000 7f68b372a700 1 -- 172.21.15.92:0/1149214715 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4006850 con 0x563230e1a9c0 2023-10-06T10:42:15.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.941+0000 7f68bb4f1000 1 -- 172.21.15.92:0/1149214715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563230e1a9c0 msgr2=0x563230e58ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:15.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.941+0000 7f68bb4f1000 1 --2- 172.21.15.92:0/1149214715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563230e1a9c0 0x563230e58ad0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009670 tx=0x7f68a4005b60 comp rx=0 tx=0).stop 2023-10-06T10:42:15.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.941+0000 7f68bb4f1000 1 -- 172.21.15.92:0/1149214715 shutdown_connections 2023-10-06T10:42:15.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.941+0000 7f68bb4f1000 1 --2- 172.21.15.92:0/1149214715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563230f00c10 0x563230e72710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:15.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.941+0000 7f68bb4f1000 1 --2- 172.21.15.92:0/1149214715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563230e1a9c0 0x563230e58ad0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:15.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.941+0000 7f68bb4f1000 1 --2- 172.21.15.92:0/1149214715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563230e1ad90 0x563230e580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:15.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.941+0000 7f68bb4f1000 1 -- 172.21.15.92:0/1149214715 >> 172.21.15.92:0/1149214715 conn(0x563230e2d750 msgr2=0x563230e56fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:15.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.941+0000 7f68bb4f1000 1 -- 172.21.15.92:0/1149214715 shutdown_connections 2023-10-06T10:42:15.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.941+0000 7f68bb4f1000 1 -- 172.21.15.92:0/1149214715 wait complete. 2023-10-06T10:42:15.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.942+0000 7f68bb4f1000 1 Processor -- start 2023-10-06T10:42:15.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.942+0000 7f68bb4f1000 1 -- start start 2023-10-06T10:42:15.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.943+0000 7f68bb4f1000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563230e1ad90 0x563230eff500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:15.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.943+0000 7f68bb4f1000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563230f00c10 0x563230e71de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:15.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.943+0000 7f68bb4f1000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563230ef75e0 0x563230e56bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:15.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.943+0000 7f68bb4f1000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563230efe8f0 con 0x563230f00c10 2023-10-06T10:42:15.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.943+0000 7f68bb4f1000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563230f02b30 con 0x563230ef75e0 2023-10-06T10:42:15.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.943+0000 7f68bb4f1000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563230e72930 con 0x563230e1ad90 2023-10-06T10:42:15.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.943+0000 7f68abfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563230f00c10 0x563230e71de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:15.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.943+0000 7f68abfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563230f00c10 0x563230e71de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53172/0 (socket says 172.21.15.92:53172) 2023-10-06T10:42:15.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.943+0000 7f68abfff700 1 -- 172.21.15.92:0/474210934 learned_addr learned my addr 172.21.15.92:0/474210934 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:15.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.943+0000 7f68b472c700 1 --2- 172.21.15.92:0/474210934 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563230e1ad90 0x563230eff500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:15.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.943+0000 7f68b4f2d700 1 --2- 172.21.15.92:0/474210934 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563230ef75e0 0x563230e56bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:15.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.943+0000 7f68abfff700 1 -- 172.21.15.92:0/474210934 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563230e1ad90 msgr2=0x563230eff500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:15.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.943+0000 7f68abfff700 1 --2- 172.21.15.92:0/474210934 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563230e1ad90 0x563230eff500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:15.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.943+0000 7f68abfff700 1 -- 172.21.15.92:0/474210934 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563230ef75e0 msgr2=0x563230e56bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:15.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.943+0000 7f68abfff700 1 --2- 172.21.15.92:0/474210934 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563230ef75e0 0x563230e56bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:15.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.943+0000 7f68abfff700 1 -- 172.21.15.92:0/474210934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a4005e40 con 0x563230f00c10 2023-10-06T10:42:15.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.943+0000 7f68abfff700 1 --2- 172.21.15.92:0/474210934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563230f00c10 0x563230e71de0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f689c00a2a0 tx=0x7f689c0018d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:15.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.943+0000 7f68b1f27700 1 -- 172.21.15.92:0/474210934 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c00ada0 con 0x563230f00c10 2023-10-06T10:42:15.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.943+0000 7f68b1f27700 1 -- 172.21.15.92:0/474210934 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689c00f420 con 0x563230f00c10 2023-10-06T10:42:15.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.943+0000 7f68bb4f1000 1 -- 172.21.15.92:0/474210934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563230d863d0 con 0x563230f00c10 2023-10-06T10:42:15.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.943+0000 7f68b1f27700 1 -- 172.21.15.92:0/474210934 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c00f7c0 con 0x563230f00c10 2023-10-06T10:42:15.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.944+0000 7f68bb4f1000 1 -- 172.21.15.92:0/474210934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563230d79ae0 con 0x563230f00c10 2023-10-06T10:42:15.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.945+0000 7f68b1f27700 1 -- 172.21.15.92:0/474210934 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f689c00fb30 con 0x563230f00c10 2023-10-06T10:42:15.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.945+0000 7f68b1f27700 1 --2- 172.21.15.92:0/474210934 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f68940658c0 0x7f6894067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:15.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.945+0000 7f68b1f27700 1 -- 172.21.15.92:0/474210934 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f689c008040 con 0x563230f00c10 2023-10-06T10:42:15.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.945+0000 7f68b472c700 1 --2- 172.21.15.92:0/474210934 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f68940658c0 0x7f6894067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:15.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.945+0000 7f68bb4f1000 1 --2- 172.21.15.92:0/474210934 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x563230ef79b0 0x563230f11dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:15.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.945+0000 7f68bb4f1000 1 -- 172.21.15.92:0/474210934 --> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] -- osd_op(unknown.0.0:1 10.12 10:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x563230f12310 con 0x563230ef79b0 2023-10-06T10:42:15.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.946+0000 7f68b472c700 1 --2- 172.21.15.92:0/474210934 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f68940658c0 0x7f6894067d70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x563230f00790 tx=0x7f68a4015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:15.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.946+0000 7f68b4f2d700 1 --2- 172.21.15.92:0/474210934 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x563230ef79b0 0x563230f11dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:15.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.946+0000 7f68b4f2d700 1 --2- 172.21.15.92:0/474210934 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x563230ef79b0 0x563230f11dd0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:15.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.947+0000 7f68b4f2d700 1 -- 172.21.15.92:0/474210934 <== osd.6 v2:172.21.15.124:6824/90914 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v66'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f68ac016020 con 0x563230ef79b0 2023-10-06T10:42:15.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.947+0000 7f68bb4f1000 1 -- 172.21.15.92:0/474210934 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x563230ef79b0 msgr2=0x563230f11dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:15.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.947+0000 7f68bb4f1000 1 --2- 172.21.15.92:0/474210934 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x563230ef79b0 0x563230f11dd0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:15.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.947+0000 7f68bb4f1000 1 -- 172.21.15.92:0/474210934 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f68940658c0 msgr2=0x7f6894067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:15.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.947+0000 7f68bb4f1000 1 --2- 172.21.15.92:0/474210934 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f68940658c0 0x7f6894067d70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x563230f00790 tx=0x7f68a4015000 comp rx=0 tx=0).stop 2023-10-06T10:42:15.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.947+0000 7f68bb4f1000 1 -- 172.21.15.92:0/474210934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563230f00c10 msgr2=0x563230e71de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:15.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.947+0000 7f68bb4f1000 1 --2- 172.21.15.92:0/474210934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563230f00c10 0x563230e71de0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f689c00a2a0 tx=0x7f689c0018d0 comp rx=0 tx=0).stop 2023-10-06T10:42:15.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.947+0000 7f68b4f2d700 1 -- 172.21.15.92:0/474210934 reap_dead start 2023-10-06T10:42:15.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.947+0000 7f68bb4f1000 1 -- 172.21.15.92:0/474210934 shutdown_connections 2023-10-06T10:42:15.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.947+0000 7f68bb4f1000 1 -- 172.21.15.92:0/474210934 >> 172.21.15.92:0/474210934 conn(0x563230e2d750 msgr2=0x563230e57e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:15.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.948+0000 7f68bb4f1000 1 -- 172.21.15.92:0/474210934 shutdown_connections 2023-10-06T10:42:15.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:15.948+0000 7f68bb4f1000 1 -- 172.21.15.92:0/474210934 wait complete. 2023-10-06T10:42:15.955 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2023-10-06T10:42:16.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.028+0000 7f2554e23000 1 Processor -- start 2023-10-06T10:42:16.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.028+0000 7f2554e23000 1 -- start start 2023-10-06T10:42:16.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.029+0000 7f2554e23000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f253c002330 0x55ef3116f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.029+0000 7f2554e23000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ef310f9fa0 0x55ef3116fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.029+0000 7f2554e23000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef31187690 0x55ef310f31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.029+0000 7f2554e23000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef30ffe980 con 0x55ef31187690 2023-10-06T10:42:16.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.029+0000 7f2554e23000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef30ffe620 con 0x7f253c002330 2023-10-06T10:42:16.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.029+0000 7f2554e23000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ef30ff5400 con 0x55ef310f9fa0 2023-10-06T10:42:16.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.029+0000 7f254e85f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef31187690 0x55ef310f31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.029+0000 7f254e05e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f253c002330 0x55ef3116f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.029+0000 7f254e85f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef31187690 0x55ef310f31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53182/0 (socket says 172.21.15.92:53182) 2023-10-06T10:42:16.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.029+0000 7f254e85f700 1 -- 172.21.15.92:0/655737417 learned_addr learned my addr 172.21.15.92:0/655737417 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:16.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.029+0000 7f2547fff700 1 --2- 172.21.15.92:0/655737417 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ef310f9fa0 0x55ef3116fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.029+0000 7f254e85f700 1 -- 172.21.15.92:0/655737417 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ef310f9fa0 msgr2=0x55ef3116fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.029+0000 7f254e85f700 1 --2- 172.21.15.92:0/655737417 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ef310f9fa0 0x55ef3116fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.029+0000 7f254e85f700 1 -- 172.21.15.92:0/655737417 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f253c002330 msgr2=0x55ef3116f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.029+0000 7f254e85f700 1 --2- 172.21.15.92:0/655737417 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f253c002330 0x55ef3116f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.029+0000 7f254e85f700 1 -- 172.21.15.92:0/655737417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef3100d3d0 con 0x55ef31187690 2023-10-06T10:42:16.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.029+0000 7f254e85f700 1 --2- 172.21.15.92:0/655737417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef31187690 0x55ef310f31f0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x55ef310fdfe0 tx=0x7f2548008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a785d2f867a6de5c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.029+0000 7f254d05c700 1 -- 172.21.15.92:0/655737417 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2548016030 con 0x55ef31187690 2023-10-06T10:42:16.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.029+0000 7f254d05c700 1 -- 172.21.15.92:0/655737417 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25480062f0 con 0x55ef31187690 2023-10-06T10:42:16.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.029+0000 7f254d05c700 1 -- 172.21.15.92:0/655737417 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2548006c80 con 0x55ef31187690 2023-10-06T10:42:16.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.030+0000 7f2554e23000 1 -- 172.21.15.92:0/655737417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef31187690 msgr2=0x55ef310f31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.030+0000 7f2554e23000 1 --2- 172.21.15.92:0/655737417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef31187690 0x55ef310f31f0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x55ef310fdfe0 tx=0x7f2548008040 comp rx=0 tx=0).stop 2023-10-06T10:42:16.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.030+0000 7f2554e23000 1 -- 172.21.15.92:0/655737417 shutdown_connections 2023-10-06T10:42:16.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.030+0000 7f2554e23000 1 --2- 172.21.15.92:0/655737417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef31187690 0x55ef310f31f0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.030+0000 7f2554e23000 1 --2- 172.21.15.92:0/655737417 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f253c002330 0x55ef3116f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.030+0000 7f2554e23000 1 --2- 172.21.15.92:0/655737417 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ef310f9fa0 0x55ef3116fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.030+0000 7f2554e23000 1 -- 172.21.15.92:0/655737417 >> 172.21.15.92:0/655737417 conn(0x55ef310b4520 msgr2=0x55ef310defc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:16.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.030+0000 7f2554e23000 1 -- 172.21.15.92:0/655737417 shutdown_connections 2023-10-06T10:42:16.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.030+0000 7f2554e23000 1 -- 172.21.15.92:0/655737417 wait complete. 2023-10-06T10:42:16.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.030+0000 7f2554e23000 1 Processor -- start 2023-10-06T10:42:16.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.031+0000 7f2554e23000 1 -- start start 2023-10-06T10:42:16.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.031+0000 7f2554e23000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ef310f9fa0 0x55ef310faab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.031+0000 7f2554e23000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef31187690 0x55ef3119aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.031+0000 7f2554e23000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f253c002330 0x55ef3119afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.031+0000 7f2554e23000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3119c290 con 0x55ef31187690 2023-10-06T10:42:16.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.031+0000 7f2554e23000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3119c530 con 0x7f253c002330 2023-10-06T10:42:16.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.031+0000 7f2554e23000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ef3119c7d0 con 0x55ef310f9fa0 2023-10-06T10:42:16.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.031+0000 7f2547fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef31187690 0x55ef3119aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.031+0000 7f2547fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef31187690 0x55ef3119aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53184/0 (socket says 172.21.15.92:53184) 2023-10-06T10:42:16.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.031+0000 7f2547fff700 1 -- 172.21.15.92:0/3414744621 learned_addr learned my addr 172.21.15.92:0/3414744621 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:16.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.031+0000 7f254e85f700 1 --2- 172.21.15.92:0/3414744621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f253c002330 0x55ef3119afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.031+0000 7f254e05e700 1 --2- 172.21.15.92:0/3414744621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ef310f9fa0 0x55ef310faab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.032+0000 7f2547fff700 1 -- 172.21.15.92:0/3414744621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ef310f9fa0 msgr2=0x55ef310faab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.032+0000 7f2547fff700 1 --2- 172.21.15.92:0/3414744621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ef310f9fa0 0x55ef310faab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.032+0000 7f2547fff700 1 -- 172.21.15.92:0/3414744621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f253c002330 msgr2=0x55ef3119afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.032+0000 7f2547fff700 1 --2- 172.21.15.92:0/3414744621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f253c002330 0x55ef3119afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.032+0000 7f2547fff700 1 -- 172.21.15.92:0/3414744621 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef310271c0 con 0x55ef31187690 2023-10-06T10:42:16.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.032+0000 7f2547fff700 1 --2- 172.21.15.92:0/3414744621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef31187690 0x55ef3119aaa0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2538007b40 tx=0x7f253800c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.032+0000 7f2546ffd700 1 -- 172.21.15.92:0/3414744621 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253800f070 con 0x55ef31187690 2023-10-06T10:42:16.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.032+0000 7f2546ffd700 1 -- 172.21.15.92:0/3414744621 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2538010040 con 0x55ef31187690 2023-10-06T10:42:16.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.032+0000 7f2546ffd700 1 -- 172.21.15.92:0/3414744621 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538006cf0 con 0x55ef31187690 2023-10-06T10:42:16.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.032+0000 7f2554e23000 1 -- 172.21.15.92:0/3414744621 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ef31000ae0 con 0x55ef31187690 2023-10-06T10:42:16.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.032+0000 7f2554e23000 1 -- 172.21.15.92:0/3414744621 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef31000540 con 0x55ef31187690 2023-10-06T10:42:16.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.033+0000 7f2546ffd700 1 -- 172.21.15.92:0/3414744621 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2538016660 con 0x55ef31187690 2023-10-06T10:42:16.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.034+0000 7f2546ffd700 1 --2- 172.21.15.92:0/3414744621 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f25300658c0 0x7f2530067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.034+0000 7f2546ffd700 1 -- 172.21.15.92:0/3414744621 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f2538084310 con 0x55ef31187690 2023-10-06T10:42:16.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.034+0000 7f254e05e700 1 --2- 172.21.15.92:0/3414744621 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f25300658c0 0x7f2530067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.034+0000 7f2554e23000 1 --2- 172.21.15.92:0/3414744621 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x55ef311a45e0 0x55ef311a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.034+0000 7f2554e23000 1 -- 172.21.15.92:0/3414744621 --> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] -- osd_op(unknown.0.0:1 10.12 10:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ef311a6f00 con 0x55ef311a45e0 2023-10-06T10:42:16.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.034+0000 7f254e05e700 1 --2- 172.21.15.92:0/3414744621 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f25300658c0 0x7f2530067d70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f25400096e0 tx=0x7f2540018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.034+0000 7f254e85f700 1 --2- 172.21.15.92:0/3414744621 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x55ef311a45e0 0x55ef311a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.034+0000 7f254e85f700 1 --2- 172.21.15.92:0/3414744621 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x55ef311a45e0 0x55ef311a69c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.036+0000 7f254e85f700 1 -- 172.21.15.92:0/3414744621 <== osd.6 v2:172.21.15.124:6824/90914 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v66'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2548003660 con 0x55ef311a45e0 2023-10-06T10:42:16.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.036+0000 7f2554e23000 1 -- 172.21.15.92:0/3414744621 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x55ef311a45e0 msgr2=0x55ef311a69c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.036+0000 7f2554e23000 1 --2- 172.21.15.92:0/3414744621 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x55ef311a45e0 0x55ef311a69c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.036+0000 7f2554e23000 1 -- 172.21.15.92:0/3414744621 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f25300658c0 msgr2=0x7f2530067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.036+0000 7f2554e23000 1 --2- 172.21.15.92:0/3414744621 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f25300658c0 0x7f2530067d70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f25400096e0 tx=0x7f2540018000 comp rx=0 tx=0).stop 2023-10-06T10:42:16.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.036+0000 7f2554e23000 1 -- 172.21.15.92:0/3414744621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef31187690 msgr2=0x55ef3119aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.036+0000 7f2554e23000 1 --2- 172.21.15.92:0/3414744621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef31187690 0x55ef3119aaa0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2538007b40 tx=0x7f253800c040 comp rx=0 tx=0).stop 2023-10-06T10:42:16.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.036+0000 7f254e85f700 1 -- 172.21.15.92:0/3414744621 reap_dead start 2023-10-06T10:42:16.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.036+0000 7f2554e23000 1 -- 172.21.15.92:0/3414744621 shutdown_connections 2023-10-06T10:42:16.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.036+0000 7f2554e23000 1 -- 172.21.15.92:0/3414744621 >> 172.21.15.92:0/3414744621 conn(0x55ef310b4520 msgr2=0x55ef31187a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:16.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.036+0000 7f2554e23000 1 -- 172.21.15.92:0/3414744621 shutdown_connections 2023-10-06T10:42:16.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.036+0000 7f2554e23000 1 -- 172.21.15.92:0/3414744621 wait complete. 2023-10-06T10:42:16.045 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2023-10-06T10:42:16.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.119+0000 7ff0d530e000 1 Processor -- start 2023-10-06T10:42:16.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.119+0000 7ff0d530e000 1 -- start start 2023-10-06T10:42:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.119+0000 7ff0d530e000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5564645cd1e0 0x55646460ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.120+0000 7ff0d530e000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556464686ff0 0x556464685ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.120+0000 7ff0d530e000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564646873c0 0x5564645f7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.120+0000 7ff0d530e000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556464571320 con 0x5564646873c0 2023-10-06T10:42:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.120+0000 7ff0d530e000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x556464515980 con 0x556464686ff0 2023-10-06T10:42:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.120+0000 7ff0d530e000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x556464515620 con 0x5564645cd1e0 2023-10-06T10:42:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.120+0000 7ff0ced4a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564646873c0 0x5564645f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.120+0000 7ff0ced4a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564646873c0 0x5564645f7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53206/0 (socket says 172.21.15.92:53206) 2023-10-06T10:42:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.120+0000 7ff0ced4a700 1 -- 172.21.15.92:0/2987962153 learned_addr learned my addr 172.21.15.92:0/2987962153 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.120+0000 7ff0cdd48700 1 --2- 172.21.15.92:0/2987962153 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556464686ff0 0x556464685ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.120+0000 7ff0ce549700 1 --2- 172.21.15.92:0/2987962153 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5564645cd1e0 0x55646460ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.120+0000 7ff0ced4a700 1 -- 172.21.15.92:0/2987962153 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5564645cd1e0 msgr2=0x55646460ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.120+0000 7ff0ced4a700 1 --2- 172.21.15.92:0/2987962153 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5564645cd1e0 0x55646460ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.120+0000 7ff0ced4a700 1 -- 172.21.15.92:0/2987962153 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556464686ff0 msgr2=0x556464685ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.120+0000 7ff0ced4a700 1 --2- 172.21.15.92:0/2987962153 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556464686ff0 0x556464685ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.120+0000 7ff0ced4a700 1 -- 172.21.15.92:0/2987962153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c0005db0 con 0x5564646873c0 2023-10-06T10:42:16.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.120+0000 7ff0ced4a700 1 --2- 172.21.15.92:0/2987962153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564646873c0 0x5564645f7950 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80026e0 tx=0x7ff0c8006b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5528e2ae8a084bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.120+0000 7ff0ccd46700 1 -- 172.21.15.92:0/2987962153 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8004120 con 0x5564646873c0 2023-10-06T10:42:16.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.120+0000 7ff0ccd46700 1 -- 172.21.15.92:0/2987962153 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c800e470 con 0x5564646873c0 2023-10-06T10:42:16.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.120+0000 7ff0ccd46700 1 -- 172.21.15.92:0/2987962153 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c800e810 con 0x5564646873c0 2023-10-06T10:42:16.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.121+0000 7ff0d530e000 1 -- 172.21.15.92:0/2987962153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564646873c0 msgr2=0x5564645f7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.121+0000 7ff0d530e000 1 --2- 172.21.15.92:0/2987962153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564646873c0 0x5564645f7950 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80026e0 tx=0x7ff0c8006b80 comp rx=0 tx=0).stop 2023-10-06T10:42:16.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.121+0000 7ff0d530e000 1 -- 172.21.15.92:0/2987962153 shutdown_connections 2023-10-06T10:42:16.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.121+0000 7ff0d530e000 1 --2- 172.21.15.92:0/2987962153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564646873c0 0x5564645f7950 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.121+0000 7ff0d530e000 1 --2- 172.21.15.92:0/2987962153 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5564645cd1e0 0x55646460ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.121+0000 7ff0d530e000 1 --2- 172.21.15.92:0/2987962153 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556464686ff0 0x556464685ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.121+0000 7ff0d530e000 1 -- 172.21.15.92:0/2987962153 >> 172.21.15.92:0/2987962153 conn(0x5564645cb6c0 msgr2=0x5564646098d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:16.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.121+0000 7ff0d530e000 1 -- 172.21.15.92:0/2987962153 shutdown_connections 2023-10-06T10:42:16.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.121+0000 7ff0d530e000 1 -- 172.21.15.92:0/2987962153 wait complete. 2023-10-06T10:42:16.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.121+0000 7ff0d530e000 1 Processor -- start 2023-10-06T10:42:16.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.121+0000 7ff0d530e000 1 -- start start 2023-10-06T10:42:16.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.122+0000 7ff0d530e000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5564645cd1e0 0x5564645f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.122+0000 7ff0d530e000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556464686ff0 0x556464682920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.122+0000 7ff0d530e000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564646873c0 0x5564646b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.122+0000 7ff0d530e000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646450c400 con 0x5564646873c0 2023-10-06T10:42:16.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.122+0000 7ff0d530e000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564646b2440 con 0x556464686ff0 2023-10-06T10:42:16.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.122+0000 7ff0d530e000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55646460bfd0 con 0x5564645cd1e0 2023-10-06T10:42:16.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.122+0000 7ff0ced4a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564646873c0 0x5564646b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.122+0000 7ff0ced4a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564646873c0 0x5564646b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53208/0 (socket says 172.21.15.92:53208) 2023-10-06T10:42:16.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.122+0000 7ff0ced4a700 1 -- 172.21.15.92:0/1697582045 learned_addr learned my addr 172.21.15.92:0/1697582045 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:16.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.122+0000 7ff0cdd48700 1 --2- 172.21.15.92:0/1697582045 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556464686ff0 0x556464682920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.122+0000 7ff0ce549700 1 --2- 172.21.15.92:0/1697582045 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5564645cd1e0 0x5564645f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.122+0000 7ff0ced4a700 1 -- 172.21.15.92:0/1697582045 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5564645cd1e0 msgr2=0x5564645f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.122+0000 7ff0ced4a700 1 --2- 172.21.15.92:0/1697582045 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5564645cd1e0 0x5564645f2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.122+0000 7ff0ced4a700 1 -- 172.21.15.92:0/1697582045 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556464686ff0 msgr2=0x556464682920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.122+0000 7ff0ced4a700 1 --2- 172.21.15.92:0/1697582045 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556464686ff0 0x556464682920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.122+0000 7ff0ced4a700 1 -- 172.21.15.92:0/1697582045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c8006e40 con 0x5564646873c0 2023-10-06T10:42:16.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.122+0000 7ff0cdd48700 1 --2- 172.21.15.92:0/1697582045 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556464686ff0 0x556464682920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:16.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.122+0000 7ff0ce549700 1 --2- 172.21.15.92:0/1697582045 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5564645cd1e0 0x5564645f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:16.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.123+0000 7ff0ced4a700 1 --2- 172.21.15.92:0/1697582045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564646873c0 0x5564646b1f00 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800e8f0 tx=0x7ff0c8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.123+0000 7ff0beffd700 1 -- 172.21.15.92:0/1697582045 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8011790 con 0x5564646873c0 2023-10-06T10:42:16.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.123+0000 7ff0beffd700 1 -- 172.21.15.92:0/1697582045 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c8011910 con 0x5564646873c0 2023-10-06T10:42:16.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.123+0000 7ff0d530e000 1 -- 172.21.15.92:0/1697582045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556464517ae0 con 0x5564646873c0 2023-10-06T10:42:16.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.123+0000 7ff0beffd700 1 -- 172.21.15.92:0/1697582045 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8011cd0 con 0x5564646873c0 2023-10-06T10:42:16.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.123+0000 7ff0d530e000 1 -- 172.21.15.92:0/1697582045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556464517540 con 0x5564646873c0 2023-10-06T10:42:16.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.124+0000 7ff0beffd700 1 -- 172.21.15.92:0/1697582045 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff0c8018030 con 0x5564646873c0 2023-10-06T10:42:16.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.124+0000 7ff0beffd700 1 --2- 172.21.15.92:0/1697582045 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff0b00658c0 0x7ff0b0067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.124+0000 7ff0beffd700 1 -- 172.21.15.92:0/1697582045 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7ff0c8084c10 con 0x5564646873c0 2023-10-06T10:42:16.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.124+0000 7ff0ce549700 1 --2- 172.21.15.92:0/1697582045 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff0b00658c0 0x7ff0b0067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.125+0000 7ff0d530e000 1 --2- 172.21.15.92:0/1697582045 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x5564645f7eb0 0x55646460cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.125+0000 7ff0d530e000 1 -- 172.21.15.92:0/1697582045 --> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] -- osd_op(unknown.0.0:1 10.5 10:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55646460d430 con 0x5564645f7eb0 2023-10-06T10:42:16.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.125+0000 7ff0cdd48700 1 --2- 172.21.15.92:0/1697582045 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x5564645f7eb0 0x55646460cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.125+0000 7ff0ce549700 1 --2- 172.21.15.92:0/1697582045 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff0b00658c0 0x7ff0b0067d70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00030a0 tx=0x7ff0c0018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.125+0000 7ff0cdd48700 1 --2- 172.21.15.92:0/1697582045 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x5564645f7eb0 0x55646460cef0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.126+0000 7ff0cdd48700 1 -- 172.21.15.92:0/1697582045 <== osd.2 v2:172.21.15.92:6824/90615 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v66'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff0c400b040 con 0x5564645f7eb0 2023-10-06T10:42:16.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.126+0000 7ff0d530e000 1 -- 172.21.15.92:0/1697582045 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x5564645f7eb0 msgr2=0x55646460cef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.126+0000 7ff0d530e000 1 --2- 172.21.15.92:0/1697582045 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x5564645f7eb0 0x55646460cef0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.126+0000 7ff0d530e000 1 -- 172.21.15.92:0/1697582045 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff0b00658c0 msgr2=0x7ff0b0067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.126+0000 7ff0d530e000 1 --2- 172.21.15.92:0/1697582045 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff0b00658c0 0x7ff0b0067d70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00030a0 tx=0x7ff0c0018000 comp rx=0 tx=0).stop 2023-10-06T10:42:16.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.126+0000 7ff0d530e000 1 -- 172.21.15.92:0/1697582045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564646873c0 msgr2=0x5564646b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.126+0000 7ff0d530e000 1 --2- 172.21.15.92:0/1697582045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564646873c0 0x5564646b1f00 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800e8f0 tx=0x7ff0c8014000 comp rx=0 tx=0).stop 2023-10-06T10:42:16.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.126+0000 7ff0ced4a700 1 -- 172.21.15.92:0/1697582045 reap_dead start 2023-10-06T10:42:16.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.126+0000 7ff0d530e000 1 -- 172.21.15.92:0/1697582045 shutdown_connections 2023-10-06T10:42:16.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.126+0000 7ff0d530e000 1 -- 172.21.15.92:0/1697582045 >> 172.21.15.92:0/1697582045 conn(0x5564645cb6c0 msgr2=0x5564646098d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:16.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.126+0000 7ff0d530e000 1 -- 172.21.15.92:0/1697582045 shutdown_connections 2023-10-06T10:42:16.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.126+0000 7ff0d530e000 1 -- 172.21.15.92:0/1697582045 wait complete. 2023-10-06T10:42:16.134 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2023-10-06T10:42:16.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.208+0000 7fb1eb965000 1 Processor -- start 2023-10-06T10:42:16.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.208+0000 7fb1eb965000 1 -- start start 2023-10-06T10:42:16.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.209+0000 7fb1eb965000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cde9dab60 0x562cdeaa9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.209+0000 7fb1eb965000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cdea21600 0x562cdeaaa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.209+0000 7fb1eb965000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cdea219d0 0x562cdeab0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.209+0000 7fb1eb965000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cde923980 con 0x562cdea219d0 2023-10-06T10:42:16.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.209+0000 7fb1eb965000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cde923620 con 0x562cde9dab60 2023-10-06T10:42:16.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.209+0000 7fb1eb965000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562cde91a400 con 0x562cdea21600 2023-10-06T10:42:16.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.209+0000 7fb1e53a1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cdea219d0 0x562cdeab0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.209+0000 7fb1e53a1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cdea219d0 0x562cdeab0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53222/0 (socket says 172.21.15.92:53222) 2023-10-06T10:42:16.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.209+0000 7fb1e53a1700 1 -- 172.21.15.92:0/1316074065 learned_addr learned my addr 172.21.15.92:0/1316074065 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:16.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.209+0000 7fb1e53a1700 1 -- 172.21.15.92:0/1316074065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cdea21600 msgr2=0x562cdeaaa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.209+0000 7fb1e53a1700 1 --2- 172.21.15.92:0/1316074065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cdea21600 0x562cdeaaa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.209+0000 7fb1e53a1700 1 -- 172.21.15.92:0/1316074065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cde9dab60 msgr2=0x562cdeaa9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.209+0000 7fb1e53a1700 1 --2- 172.21.15.92:0/1316074065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cde9dab60 0x562cdeaa9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.209+0000 7fb1e53a1700 1 -- 172.21.15.92:0/1316074065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d8005cf0 con 0x562cdea219d0 2023-10-06T10:42:16.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.209+0000 7fb1e53a1700 1 --2- 172.21.15.92:0/1316074065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cdea219d0 0x562cdeab0560 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc002950 tx=0x7fb1dc005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6a02f9162d0505 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.209+0000 7fb1e339d700 1 -- 172.21.15.92:0/1316074065 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc00d960 con 0x562cdea219d0 2023-10-06T10:42:16.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.209+0000 7fb1e339d700 1 -- 172.21.15.92:0/1316074065 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1dc00dac0 con 0x562cdea219d0 2023-10-06T10:42:16.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.209+0000 7fb1e339d700 1 -- 172.21.15.92:0/1316074065 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc00de80 con 0x562cdea219d0 2023-10-06T10:42:16.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.210+0000 7fb1eb965000 1 -- 172.21.15.92:0/1316074065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cdea219d0 msgr2=0x562cdeab0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.210+0000 7fb1eb965000 1 --2- 172.21.15.92:0/1316074065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cdea219d0 0x562cdeab0560 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc002950 tx=0x7fb1dc005d40 comp rx=0 tx=0).stop 2023-10-06T10:42:16.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.210+0000 7fb1eb965000 1 -- 172.21.15.92:0/1316074065 shutdown_connections 2023-10-06T10:42:16.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.210+0000 7fb1eb965000 1 --2- 172.21.15.92:0/1316074065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cdea219d0 0x562cdeab0560 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.210+0000 7fb1eb965000 1 --2- 172.21.15.92:0/1316074065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cde9dab60 0x562cdeaa9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.210+0000 7fb1eb965000 1 --2- 172.21.15.92:0/1316074065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cdea21600 0x562cdeaaa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.210+0000 7fb1eb965000 1 -- 172.21.15.92:0/1316074065 >> 172.21.15.92:0/1316074065 conn(0x562cde9d9750 msgr2=0x562cdea08260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:16.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.210+0000 7fb1eb965000 1 -- 172.21.15.92:0/1316074065 shutdown_connections 2023-10-06T10:42:16.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.210+0000 7fb1eb965000 1 -- 172.21.15.92:0/1316074065 wait complete. 2023-10-06T10:42:16.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.211+0000 7fb1eb965000 1 Processor -- start 2023-10-06T10:42:16.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.211+0000 7fb1eb965000 1 -- start start 2023-10-06T10:42:16.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.211+0000 7fb1eb965000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cde9dab60 0x562cdeab87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.211+0000 7fb1eb965000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cdea21600 0x562cdeaa9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.211+0000 7fb1eb965000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cdea219d0 0x562cdeab2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.211+0000 7fb1eb965000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdeaaa1f0 con 0x562cde9dab60 2023-10-06T10:42:16.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.211+0000 7fb1eb965000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdeab7860 con 0x562cdea219d0 2023-10-06T10:42:16.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.211+0000 7fb1eb965000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562cdea90d30 con 0x562cdea21600 2023-10-06T10:42:16.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.211+0000 7fb1e4ba0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cde9dab60 0x562cdeab87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.211+0000 7fb1e4ba0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cde9dab60 0x562cdeab87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53224/0 (socket says 172.21.15.92:53224) 2023-10-06T10:42:16.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.211+0000 7fb1e4ba0700 1 -- 172.21.15.92:0/249976123 learned_addr learned my addr 172.21.15.92:0/249976123 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:16.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.211+0000 7fb1e439f700 1 --2- 172.21.15.92:0/249976123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cdea21600 0x562cdeaa9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.212+0000 7fb1e53a1700 1 --2- 172.21.15.92:0/249976123 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cdea219d0 0x562cdeab2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.212+0000 7fb1e4ba0700 1 -- 172.21.15.92:0/249976123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cdea21600 msgr2=0x562cdeaa9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.212+0000 7fb1e4ba0700 1 --2- 172.21.15.92:0/249976123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cdea21600 0x562cdeaa9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.212+0000 7fb1e4ba0700 1 -- 172.21.15.92:0/249976123 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cdea219d0 msgr2=0x562cdeab2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.212+0000 7fb1e4ba0700 1 --2- 172.21.15.92:0/249976123 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cdea219d0 0x562cdeab2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.212+0000 7fb1e4ba0700 1 -- 172.21.15.92:0/249976123 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1dc002600 con 0x562cde9dab60 2023-10-06T10:42:16.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.212+0000 7fb1e53a1700 1 --2- 172.21.15.92:0/249976123 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cdea219d0 0x562cdeab2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:16.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.212+0000 7fb1e4ba0700 1 --2- 172.21.15.92:0/249976123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cde9dab60 0x562cdeab87c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400bd70 tx=0x7fb1d40017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.212+0000 7fb1e1b9a700 1 -- 172.21.15.92:0/249976123 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4009d80 con 0x562cde9dab60 2023-10-06T10:42:16.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.212+0000 7fb1e1b9a700 1 -- 172.21.15.92:0/249976123 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d400e470 con 0x562cde9dab60 2023-10-06T10:42:16.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.212+0000 7fb1eb965000 1 -- 172.21.15.92:0/249976123 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562cde9323d0 con 0x562cde9dab60 2023-10-06T10:42:16.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.212+0000 7fb1e1b9a700 1 -- 172.21.15.92:0/249976123 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d400e870 con 0x562cde9dab60 2023-10-06T10:42:16.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.212+0000 7fb1eb965000 1 -- 172.21.15.92:0/249976123 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562cde925ae0 con 0x562cde9dab60 2023-10-06T10:42:16.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.213+0000 7fb1e1b9a700 1 -- 172.21.15.92:0/249976123 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb1d400eca0 con 0x562cde9dab60 2023-10-06T10:42:16.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.213+0000 7fb1e1b9a700 1 --2- 172.21.15.92:0/249976123 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb1d0065890 0x7fb1d0067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.214+0000 7fb1e1b9a700 1 -- 172.21.15.92:0/249976123 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fb1d4083620 con 0x562cde9dab60 2023-10-06T10:42:16.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.214+0000 7fb1e439f700 1 --2- 172.21.15.92:0/249976123 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb1d0065890 0x7fb1d0067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.214+0000 7fb1eb965000 1 --2- 172.21.15.92:0/249976123 >> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] conn(0x562cdeab5b10 0x562cdeab5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.214+0000 7fb1eb965000 1 -- 172.21.15.92:0/249976123 --> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] -- osd_op(unknown.0.0:1 10.9 10:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x562cdeab6450 con 0x562cdeab5b10 2023-10-06T10:42:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.214+0000 7fb1e439f700 1 --2- 172.21.15.92:0/249976123 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb1d0065890 0x7fb1d0067d40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800a650 tx=0x7fb1d8016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.214+0000 7fb1e53a1700 1 --2- 172.21.15.92:0/249976123 >> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] conn(0x562cdeab5b10 0x562cdeab5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.214+0000 7fb1e53a1700 1 --2- 172.21.15.92:0/249976123 >> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] conn(0x562cdeab5b10 0x562cdeab5f10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.216+0000 7fb1e53a1700 1 -- 172.21.15.92:0/249976123 <== osd.5 v2:172.21.15.124:6816/90907 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v66'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fb1dc018070 con 0x562cdeab5b10 2023-10-06T10:42:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.216+0000 7fb1eb965000 1 -- 172.21.15.92:0/249976123 >> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] conn(0x562cdeab5b10 msgr2=0x562cdeab5f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.216+0000 7fb1eb965000 1 --2- 172.21.15.92:0/249976123 >> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] conn(0x562cdeab5b10 0x562cdeab5f10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.216+0000 7fb1eb965000 1 -- 172.21.15.92:0/249976123 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb1d0065890 msgr2=0x7fb1d0067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.216+0000 7fb1eb965000 1 --2- 172.21.15.92:0/249976123 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb1d0065890 0x7fb1d0067d40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800a650 tx=0x7fb1d8016000 comp rx=0 tx=0).stop 2023-10-06T10:42:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.216+0000 7fb1eb965000 1 -- 172.21.15.92:0/249976123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cde9dab60 msgr2=0x562cdeab87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.216+0000 7fb1eb965000 1 --2- 172.21.15.92:0/249976123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cde9dab60 0x562cdeab87c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400bd70 tx=0x7fb1d40017b0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.216+0000 7fb1e53a1700 1 -- 172.21.15.92:0/249976123 reap_dead start 2023-10-06T10:42:16.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.216+0000 7fb1eb965000 1 -- 172.21.15.92:0/249976123 shutdown_connections 2023-10-06T10:42:16.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.216+0000 7fb1eb965000 1 -- 172.21.15.92:0/249976123 >> 172.21.15.92:0/249976123 conn(0x562cde9d9750 msgr2=0x562cdea918d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:16.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.216+0000 7fb1eb965000 1 -- 172.21.15.92:0/249976123 shutdown_connections 2023-10-06T10:42:16.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.216+0000 7fb1eb965000 1 -- 172.21.15.92:0/249976123 wait complete. 2023-10-06T10:42:16.223 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2023-10-06T10:42:16.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.298+0000 7fc7d2fcc000 1 Processor -- start 2023-10-06T10:42:16.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.299+0000 7fc7d2fcc000 1 -- start start 2023-10-06T10:42:16.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.299+0000 7fc7d2fcc000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afef84ab60 0x55afef876560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.299+0000 7fc7d2fcc000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55afef891600 0x55afef876aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.299+0000 7fc7d2fcc000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55afef8919d0 0x55afef876fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.299+0000 7fc7d2fcc000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afef793980 con 0x55afef84ab60 2023-10-06T10:42:16.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.299+0000 7fc7d2fcc000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afef793620 con 0x55afef891600 2023-10-06T10:42:16.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.299+0000 7fc7d2fcc000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55afef78a400 con 0x55afef8919d0 2023-10-06T10:42:16.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.299+0000 7fc7cc207700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afef84ab60 0x55afef876560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.299+0000 7fc7cc207700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afef84ab60 0x55afef876560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53230/0 (socket says 172.21.15.92:53230) 2023-10-06T10:42:16.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.299+0000 7fc7cc207700 1 -- 172.21.15.92:0/1777274245 learned_addr learned my addr 172.21.15.92:0/1777274245 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:16.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.300+0000 7fc7cc207700 1 -- 172.21.15.92:0/1777274245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55afef8919d0 msgr2=0x55afef876fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.300+0000 7fc7cc207700 1 --2- 172.21.15.92:0/1777274245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55afef8919d0 0x55afef876fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.300+0000 7fc7cc207700 1 -- 172.21.15.92:0/1777274245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55afef891600 msgr2=0x55afef876aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.300+0000 7fc7cc207700 1 --2- 172.21.15.92:0/1777274245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55afef891600 0x55afef876aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.300+0000 7fc7cc207700 1 -- 172.21.15.92:0/1777274245 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c4005cf0 con 0x55afef84ab60 2023-10-06T10:42:16.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.300+0000 7fc7cc207700 1 --2- 172.21.15.92:0/1777274245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afef84ab60 0x55afef876560 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400ada0 tx=0x7fc7c4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184967da955d2f39 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.300+0000 7fc7caa04700 1 -- 172.21.15.92:0/1777274245 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4000bc0 con 0x55afef84ab60 2023-10-06T10:42:16.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.300+0000 7fc7caa04700 1 -- 172.21.15.92:0/1777274245 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c40089d0 con 0x55afef84ab60 2023-10-06T10:42:16.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.300+0000 7fc7caa04700 1 -- 172.21.15.92:0/1777274245 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4000d20 con 0x55afef84ab60 2023-10-06T10:42:16.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.300+0000 7fc7d2fcc000 1 -- 172.21.15.92:0/1777274245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afef84ab60 msgr2=0x55afef876560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.300+0000 7fc7d2fcc000 1 --2- 172.21.15.92:0/1777274245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afef84ab60 0x55afef876560 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400ada0 tx=0x7fc7c4016000 comp rx=0 tx=0).stop 2023-10-06T10:42:16.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.300+0000 7fc7d2fcc000 1 -- 172.21.15.92:0/1777274245 shutdown_connections 2023-10-06T10:42:16.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.300+0000 7fc7d2fcc000 1 --2- 172.21.15.92:0/1777274245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55afef8919d0 0x55afef876fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.300+0000 7fc7d2fcc000 1 --2- 172.21.15.92:0/1777274245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afef84ab60 0x55afef876560 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.300+0000 7fc7d2fcc000 1 --2- 172.21.15.92:0/1777274245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55afef891600 0x55afef876aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.300+0000 7fc7d2fcc000 1 -- 172.21.15.92:0/1777274245 >> 172.21.15.92:0/1777274245 conn(0x55afef849750 msgr2=0x55afef88f110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:16.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.300+0000 7fc7d2fcc000 1 -- 172.21.15.92:0/1777274245 shutdown_connections 2023-10-06T10:42:16.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.300+0000 7fc7d2fcc000 1 -- 172.21.15.92:0/1777274245 wait complete. 2023-10-06T10:42:16.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.301+0000 7fc7d2fcc000 1 Processor -- start 2023-10-06T10:42:16.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.301+0000 7fc7d2fcc000 1 -- start start 2023-10-06T10:42:16.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7d2fcc000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55afef84ab60 0x55afef91c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7d2fcc000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55afef891600 0x55afef9217e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7d2fcc000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afef8919d0 0x55afef92f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7d2fcc000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afef86a0c0 con 0x55afef8919d0 2023-10-06T10:42:16.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7d2fcc000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afef86a360 con 0x55afef84ab60 2023-10-06T10:42:16.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7d2fcc000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55afef86a600 con 0x55afef891600 2023-10-06T10:42:16.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7cca08700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afef8919d0 0x55afef92f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7cca08700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afef8919d0 0x55afef92f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53244/0 (socket says 172.21.15.92:53244) 2023-10-06T10:42:16.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7cca08700 1 -- 172.21.15.92:0/2695829548 learned_addr learned my addr 172.21.15.92:0/2695829548 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:16.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7cc207700 1 --2- 172.21.15.92:0/2695829548 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55afef84ab60 0x55afef91c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7cba06700 1 --2- 172.21.15.92:0/2695829548 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55afef891600 0x55afef9217e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7cca08700 1 -- 172.21.15.92:0/2695829548 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55afef891600 msgr2=0x55afef9217e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7cca08700 1 --2- 172.21.15.92:0/2695829548 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55afef891600 0x55afef9217e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7cca08700 1 -- 172.21.15.92:0/2695829548 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55afef84ab60 msgr2=0x55afef91c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7cca08700 1 --2- 172.21.15.92:0/2695829548 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55afef84ab60 0x55afef91c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7cca08700 1 -- 172.21.15.92:0/2695829548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afef7bc1c0 con 0x55afef8919d0 2023-10-06T10:42:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7cc207700 1 --2- 172.21.15.92:0/2695829548 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55afef84ab60 0x55afef91c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7cca08700 1 --2- 172.21.15.92:0/2695829548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afef8919d0 0x55afef92f820 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00b160 tx=0x7fc7bc0068d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7c9201700 1 -- 172.21.15.92:0/2695829548 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc00be80 con 0x55afef8919d0 2023-10-06T10:42:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7c9201700 1 -- 172.21.15.92:0/2695829548 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7bc0060d0 con 0x55afef8919d0 2023-10-06T10:42:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7d2fcc000 1 -- 172.21.15.92:0/2695829548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55afef7a23d0 con 0x55afef8919d0 2023-10-06T10:42:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7c9201700 1 -- 172.21.15.92:0/2695829548 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc004520 con 0x55afef8919d0 2023-10-06T10:42:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.302+0000 7fc7d2fcc000 1 -- 172.21.15.92:0/2695829548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55afef795ae0 con 0x55afef8919d0 2023-10-06T10:42:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.304+0000 7fc7c9201700 1 -- 172.21.15.92:0/2695829548 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc7bc0048d0 con 0x55afef8919d0 2023-10-06T10:42:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.304+0000 7fc7c9201700 1 --2- 172.21.15.92:0/2695829548 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc7b8065890 0x7fc7b8067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.304+0000 7fc7c9201700 1 -- 172.21.15.92:0/2695829548 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fc7bc082940 con 0x55afef8919d0 2023-10-06T10:42:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.304+0000 7fc7cc207700 1 --2- 172.21.15.92:0/2695829548 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc7b8065890 0x7fc7b8067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.304+0000 7fc7d2fcc000 1 --2- 172.21.15.92:0/2695829548 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x55afef921d20 0x55afef86afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.304+0000 7fc7d2fcc000 1 -- 172.21.15.92:0/2695829548 --> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] -- osd_op(unknown.0.0:1 10.5 10:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55afef92fd60 con 0x55afef921d20 2023-10-06T10:42:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.304+0000 7fc7cba06700 1 --2- 172.21.15.92:0/2695829548 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x55afef921d20 0x55afef86afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.304+0000 7fc7cc207700 1 --2- 172.21.15.92:0/2695829548 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc7b8065890 0x7fc7b8067d40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400ada0 tx=0x7fc7c401e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.305+0000 7fc7cba06700 1 --2- 172.21.15.92:0/2695829548 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x55afef921d20 0x55afef86afe0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.306+0000 7fc7cba06700 1 -- 172.21.15.92:0/2695829548 <== osd.2 v2:172.21.15.92:6824/90615 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v66'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fc7c0011040 con 0x55afef921d20 2023-10-06T10:42:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.306+0000 7fc7d2fcc000 1 -- 172.21.15.92:0/2695829548 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x55afef921d20 msgr2=0x55afef86afe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.306+0000 7fc7d2fcc000 1 --2- 172.21.15.92:0/2695829548 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x55afef921d20 0x55afef86afe0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.306+0000 7fc7d2fcc000 1 -- 172.21.15.92:0/2695829548 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc7b8065890 msgr2=0x7fc7b8067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.306+0000 7fc7d2fcc000 1 --2- 172.21.15.92:0/2695829548 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc7b8065890 0x7fc7b8067d40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400ada0 tx=0x7fc7c401e000 comp rx=0 tx=0).stop 2023-10-06T10:42:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.306+0000 7fc7d2fcc000 1 -- 172.21.15.92:0/2695829548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afef8919d0 msgr2=0x55afef92f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.306+0000 7fc7d2fcc000 1 --2- 172.21.15.92:0/2695829548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afef8919d0 0x55afef92f820 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00b160 tx=0x7fc7bc0068d0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.306+0000 7fc7cca08700 1 -- 172.21.15.92:0/2695829548 reap_dead start 2023-10-06T10:42:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.306+0000 7fc7d2fcc000 1 -- 172.21.15.92:0/2695829548 shutdown_connections 2023-10-06T10:42:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.306+0000 7fc7d2fcc000 1 -- 172.21.15.92:0/2695829548 >> 172.21.15.92:0/2695829548 conn(0x55afef849750 msgr2=0x55afef875a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.306+0000 7fc7d2fcc000 1 -- 172.21.15.92:0/2695829548 shutdown_connections 2023-10-06T10:42:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.306+0000 7fc7d2fcc000 1 -- 172.21.15.92:0/2695829548 wait complete. 2023-10-06T10:42:16.313 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2023-10-06T10:42:16.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.390+0000 7fef76834000 1 Processor -- start 2023-10-06T10:42:16.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.390+0000 7fef76834000 1 -- start start 2023-10-06T10:42:16.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.391+0000 7fef76834000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56121ae23270 0x56121aefd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.391+0000 7fef76834000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56121aedd040 0x56121ae662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.391+0000 7fef76834000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56121aedd410 0x56121ae4e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.391+0000 7fef76834000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56121ad6b980 con 0x56121aedd410 2023-10-06T10:42:16.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.391+0000 7fef76834000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56121ad6b620 con 0x56121aedd040 2023-10-06T10:42:16.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.391+0000 7fef76834000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56121ad62400 con 0x56121ae23270 2023-10-06T10:42:16.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.391+0000 7fef70270700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56121aedd410 0x56121ae4e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.391+0000 7fef70270700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56121aedd410 0x56121ae4e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53276/0 (socket says 172.21.15.92:53276) 2023-10-06T10:42:16.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.391+0000 7fef70270700 1 -- 172.21.15.92:0/2992972970 learned_addr learned my addr 172.21.15.92:0/2992972970 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:16.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.391+0000 7fef6726e700 1 --2- 172.21.15.92:0/2992972970 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56121aedd040 0x56121ae662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.391+0000 7fef70270700 1 -- 172.21.15.92:0/2992972970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56121ae23270 msgr2=0x56121aefd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.391+0000 7fef6fa6f700 1 --2- 172.21.15.92:0/2992972970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56121ae23270 0x56121aefd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.391+0000 7fef70270700 1 --2- 172.21.15.92:0/2992972970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56121ae23270 0x56121aefd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.391+0000 7fef70270700 1 -- 172.21.15.92:0/2992972970 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56121aedd040 msgr2=0x56121ae662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.391+0000 7fef70270700 1 --2- 172.21.15.92:0/2992972970 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56121aedd040 0x56121ae662c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.391+0000 7fef70270700 1 -- 172.21.15.92:0/2992972970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef58005db0 con 0x56121aedd410 2023-10-06T10:42:16.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.391+0000 7fef6726e700 1 --2- 172.21.15.92:0/2992972970 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56121aedd040 0x56121ae662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:16.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.391+0000 7fef6fa6f700 1 --2- 172.21.15.92:0/2992972970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56121ae23270 0x56121aefd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:16.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.392+0000 7fef70270700 1 --2- 172.21.15.92:0/2992972970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56121aedd410 0x56121ae4e590 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fef6800a7c0 tx=0x7fef68005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9504cc6307fc2204 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.392+0000 7fef6ea6d700 1 -- 172.21.15.92:0/2992972970 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6800f070 con 0x56121aedd410 2023-10-06T10:42:16.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.392+0000 7fef6ea6d700 1 -- 172.21.15.92:0/2992972970 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef68008550 con 0x56121aedd410 2023-10-06T10:42:16.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.392+0000 7fef6ea6d700 1 -- 172.21.15.92:0/2992972970 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef680088d0 con 0x56121aedd410 2023-10-06T10:42:16.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.392+0000 7fef76834000 1 -- 172.21.15.92:0/2992972970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56121aedd410 msgr2=0x56121ae4e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.392+0000 7fef76834000 1 --2- 172.21.15.92:0/2992972970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56121aedd410 0x56121ae4e590 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fef6800a7c0 tx=0x7fef68005bc0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.392+0000 7fef76834000 1 -- 172.21.15.92:0/2992972970 shutdown_connections 2023-10-06T10:42:16.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.392+0000 7fef76834000 1 --2- 172.21.15.92:0/2992972970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56121aedd410 0x56121ae4e590 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.392+0000 7fef76834000 1 --2- 172.21.15.92:0/2992972970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56121ae23270 0x56121aefd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.392+0000 7fef76834000 1 --2- 172.21.15.92:0/2992972970 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56121aedd040 0x56121ae662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.392+0000 7fef76834000 1 -- 172.21.15.92:0/2992972970 >> 172.21.15.92:0/2992972970 conn(0x56121ae21750 msgr2=0x56121aef3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:16.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.392+0000 7fef76834000 1 -- 172.21.15.92:0/2992972970 shutdown_connections 2023-10-06T10:42:16.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.392+0000 7fef76834000 1 -- 172.21.15.92:0/2992972970 wait complete. 2023-10-06T10:42:16.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.393+0000 7fef76834000 1 Processor -- start 2023-10-06T10:42:16.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.393+0000 7fef76834000 1 -- start start 2023-10-06T10:42:16.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.393+0000 7fef76834000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56121ae23270 0x56121aed9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.393+0000 7fef76834000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56121aedd040 0x56121aef48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.393+0000 7fef76834000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56121aedd410 0x56121aef4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.393+0000 7fef76834000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56121af03520 con 0x56121aedd040 2023-10-06T10:42:16.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.394+0000 7fef76834000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56121aeda4c0 con 0x56121ae23270 2023-10-06T10:42:16.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.394+0000 7fef76834000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56121ae66360 con 0x56121aedd410 2023-10-06T10:42:16.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.394+0000 7fef6726e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56121aedd040 0x56121aef48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.394+0000 7fef6726e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56121aedd040 0x56121aef48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53288/0 (socket says 172.21.15.92:53288) 2023-10-06T10:42:16.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.394+0000 7fef6726e700 1 -- 172.21.15.92:0/13350857 learned_addr learned my addr 172.21.15.92:0/13350857 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:16.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.394+0000 7fef6fa6f700 1 --2- 172.21.15.92:0/13350857 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56121ae23270 0x56121aed9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.394+0000 7fef70270700 1 --2- 172.21.15.92:0/13350857 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56121aedd410 0x56121aef4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.394+0000 7fef6726e700 1 -- 172.21.15.92:0/13350857 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56121aedd410 msgr2=0x56121aef4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.394+0000 7fef6726e700 1 --2- 172.21.15.92:0/13350857 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56121aedd410 0x56121aef4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.394+0000 7fef6726e700 1 -- 172.21.15.92:0/13350857 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56121ae23270 msgr2=0x56121aed9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.394+0000 7fef6726e700 1 --2- 172.21.15.92:0/13350857 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56121ae23270 0x56121aed9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.394+0000 7fef6726e700 1 -- 172.21.15.92:0/13350857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6800a440 con 0x56121aedd040 2023-10-06T10:42:16.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.394+0000 7fef70270700 1 --2- 172.21.15.92:0/13350857 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56121aedd410 0x56121aef4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:16.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.394+0000 7fef6726e700 1 --2- 172.21.15.92:0/13350857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56121aedd040 0x56121aef48f0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fef58005b50 tx=0x7fef58019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.394+0000 7fef6d26a700 1 -- 172.21.15.92:0/13350857 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef58008d80 con 0x56121aedd040 2023-10-06T10:42:16.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.394+0000 7fef6d26a700 1 -- 172.21.15.92:0/13350857 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5801e420 con 0x56121aedd040 2023-10-06T10:42:16.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.394+0000 7fef76834000 1 -- 172.21.15.92:0/13350857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56121ad7a3d0 con 0x56121aedd040 2023-10-06T10:42:16.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.394+0000 7fef6d26a700 1 -- 172.21.15.92:0/13350857 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5801e7e0 con 0x56121aedd040 2023-10-06T10:42:16.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.394+0000 7fef76834000 1 -- 172.21.15.92:0/13350857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56121ad6dae0 con 0x56121aedd040 2023-10-06T10:42:16.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.395+0000 7fef6d26a700 1 -- 172.21.15.92:0/13350857 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fef5801ec10 con 0x56121aedd040 2023-10-06T10:42:16.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.396+0000 7fef6d26a700 1 --2- 172.21.15.92:0/13350857 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fef50065890 0x7fef50067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.396+0000 7fef6d26a700 1 -- 172.21.15.92:0/13350857 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fef58086e00 con 0x56121aedd040 2023-10-06T10:42:16.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.396+0000 7fef6fa6f700 1 --2- 172.21.15.92:0/13350857 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fef50065890 0x7fef50067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.396+0000 7fef76834000 1 --2- 172.21.15.92:0/13350857 >> [v2:172.21.15.124:6800/90883,v1:172.21.15.124:6801/90883] conn(0x56121ae4c4a0 0x56121aef8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.396+0000 7fef76834000 1 -- 172.21.15.92:0/13350857 --> [v2:172.21.15.124:6800/90883,v1:172.21.15.124:6801/90883] -- osd_op(unknown.0.0:1 10.1d 10:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x56121aef9080 con 0x56121ae4c4a0 2023-10-06T10:42:16.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.396+0000 7fef70270700 1 --2- 172.21.15.92:0/13350857 >> [v2:172.21.15.124:6800/90883,v1:172.21.15.124:6801/90883] conn(0x56121ae4c4a0 0x56121aef8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.396+0000 7fef6fa6f700 1 --2- 172.21.15.92:0/13350857 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fef50065890 0x7fef50067d40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fef60006fd0 tx=0x7fef60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.397+0000 7fef70270700 1 --2- 172.21.15.92:0/13350857 >> [v2:172.21.15.124:6800/90883,v1:172.21.15.124:6801/90883] conn(0x56121ae4c4a0 0x56121aef8b40 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.398+0000 7fef70270700 1 -- 172.21.15.92:0/13350857 <== osd.4 v2:172.21.15.124:6800/90883 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v66'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef68008d50 con 0x56121ae4c4a0 2023-10-06T10:42:16.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.398+0000 7fef76834000 1 -- 172.21.15.92:0/13350857 >> [v2:172.21.15.124:6800/90883,v1:172.21.15.124:6801/90883] conn(0x56121ae4c4a0 msgr2=0x56121aef8b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.398+0000 7fef76834000 1 --2- 172.21.15.92:0/13350857 >> [v2:172.21.15.124:6800/90883,v1:172.21.15.124:6801/90883] conn(0x56121ae4c4a0 0x56121aef8b40 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.398+0000 7fef76834000 1 -- 172.21.15.92:0/13350857 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fef50065890 msgr2=0x7fef50067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.398+0000 7fef76834000 1 --2- 172.21.15.92:0/13350857 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fef50065890 0x7fef50067d40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fef60006fd0 tx=0x7fef60008040 comp rx=0 tx=0).stop 2023-10-06T10:42:16.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.398+0000 7fef76834000 1 -- 172.21.15.92:0/13350857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56121aedd040 msgr2=0x56121aef48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.398+0000 7fef76834000 1 --2- 172.21.15.92:0/13350857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56121aedd040 0x56121aef48f0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fef58005b50 tx=0x7fef58019000 comp rx=0 tx=0).stop 2023-10-06T10:42:16.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.398+0000 7fef70270700 1 -- 172.21.15.92:0/13350857 reap_dead start 2023-10-06T10:42:16.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.398+0000 7fef76834000 1 -- 172.21.15.92:0/13350857 shutdown_connections 2023-10-06T10:42:16.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.398+0000 7fef76834000 1 -- 172.21.15.92:0/13350857 >> 172.21.15.92:0/13350857 conn(0x56121ae21750 msgr2=0x56121aef3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:16.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.398+0000 7fef76834000 1 -- 172.21.15.92:0/13350857 shutdown_connections 2023-10-06T10:42:16.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.398+0000 7fef76834000 1 -- 172.21.15.92:0/13350857 wait complete. 2023-10-06T10:42:16.407 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start /d1/d2/d3 recursive,force 2023-10-06T10:42:16.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.561+0000 7f9aa359e700 1 Processor -- start 2023-10-06T10:42:16.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.561+0000 7f9aa359e700 1 -- start start 2023-10-06T10:42:16.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.562+0000 7f9aa359e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9aa40d09c0 0x7f9aa41642f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.562+0000 7f9aa359e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aa40cdbf0 con 0x7f9aa40cca70 2023-10-06T10:42:16.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.562+0000 7f9aa359e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aa40cb4c0 con 0x7f9aa4153990 2023-10-06T10:42:16.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.562+0000 7f9aa359e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa40ccfe0 con 0x7f9aa40d09c0 2023-10-06T10:42:16.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.562+0000 7f9a99d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9aa40cca70 0x7f9aa40d02b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56518/0 (socket says 172.21.15.92:56518) 2023-10-06T10:42:16.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.562+0000 7f9a99d9b700 1 -- 172.21.15.92:0/4002506225 learned_addr learned my addr 172.21.15.92:0/4002506225 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:42:16.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.562+0000 7f9aa2d9d700 1 --2- 172.21.15.92:0/4002506225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9aa40d09c0 0x7f9aa41642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.562+0000 7f9aa1d9b700 1 -- 172.21.15.92:0/4002506225 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 855428404 0 0) 0x7f9aa40cdbf0 con 0x7f9aa40cca70 2023-10-06T10:42:16.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.562+0000 7f9aa1d9b700 1 -- 172.21.15.92:0/4002506225 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a90003570 con 0x7f9aa40cca70 2023-10-06T10:42:16.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.562+0000 7f9aa1d9b700 1 -- 172.21.15.92:0/4002506225 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1775334324 0 0) 0x7f9a90003570 con 0x7f9aa40cca70 2023-10-06T10:42:16.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.562+0000 7f9aa1d9b700 1 -- 172.21.15.92:0/4002506225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9aa40d09c0 msgr2=0x7f9aa41642f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.562+0000 7f9aa1d9b700 1 --2- 172.21.15.92:0/4002506225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9aa40d09c0 0x7f9aa41642f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.562+0000 7f9aa1d9b700 1 -- 172.21.15.92:0/4002506225 >> v1:172.21.15.124:6789/0 conn(0x7f9aa4153990 legacy=0x7f9aa415a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.562+0000 7f9aa1d9b700 1 -- 172.21.15.92:0/4002506225 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa40cd2c0 con 0x7f9aa40cca70 2023-10-06T10:42:16.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.562+0000 7f9aa1d9b700 1 -- 172.21.15.92:0/4002506225 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f9a8c001470 con 0x7f9aa40cca70 2023-10-06T10:42:16.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.562+0000 7f9aa1d9b700 1 -- 172.21.15.92:0/4002506225 >> v1:172.21.15.92:6789/0 conn(0x7f9aa40cca70 legacy=0x7f9aa40d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.562+0000 7f9aa1d9b700 1 --2- 172.21.15.92:0/4002506225 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a90003a90 0x7f9a90003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.562+0000 7f9aa1d9b700 1 --2- 172.21.15.92:0/4002506225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a90004810 0x7f9a90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.562+0000 7f9aa1d9b700 1 --2- 172.21.15.92:0/4002506225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9aa40d09c0 0x7f9a900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.562+0000 7f9aa1d9b700 1 -- 172.21.15.92:0/4002506225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa40ccfe0 con 0x7f9a90004810 2023-10-06T10:42:16.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.562+0000 7f9aa1d9b700 1 -- 172.21.15.92:0/4002506225 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9000bb10 con 0x7f9a90003a90 2023-10-06T10:42:16.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.562+0000 7f9aa1d9b700 1 -- 172.21.15.92:0/4002506225 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9000bcd0 con 0x7f9aa40d09c0 2023-10-06T10:42:16.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.563+0000 7f9a99d9b700 1 --2- 172.21.15.92:0/4002506225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a90004810 0x7f9a90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.563+0000 7f9aa259c700 1 --2- 172.21.15.92:0/4002506225 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a90003a90 0x7f9a90003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.563+0000 7f9aa2d9d700 1 --2- 172.21.15.92:0/4002506225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9aa40d09c0 0x7f9a900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.563+0000 7f9a99d9b700 1 -- 172.21.15.92:0/4002506225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9aa40d09c0 msgr2=0x7f9a900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.563+0000 7f9a99d9b700 1 --2- 172.21.15.92:0/4002506225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9aa40d09c0 0x7f9a900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.563+0000 7f9a99d9b700 1 -- 172.21.15.92:0/4002506225 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a90003a90 msgr2=0x7f9a90003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.563+0000 7f9a99d9b700 1 --2- 172.21.15.92:0/4002506225 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a90003a90 0x7f9a90003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.563+0000 7f9a99d9b700 1 -- 172.21.15.92:0/4002506225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a90003570 con 0x7f9a90004810 2023-10-06T10:42:16.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.563+0000 7f9a99d9b700 1 --2- 172.21.15.92:0/4002506225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a90004810 0x7f9a90004c80 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00c3c0 tx=0x7f9a8c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b5035e690e64c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.563+0000 7f9aa1d9b700 1 -- 172.21.15.92:0/4002506225 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c00cc90 con 0x7f9a90004810 2023-10-06T10:42:16.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.563+0000 7f9aa1d9b700 1 -- 172.21.15.92:0/4002506225 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8c00cdf0 con 0x7f9a90004810 2023-10-06T10:42:16.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.563+0000 7f9aa1d9b700 1 -- 172.21.15.92:0/4002506225 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c005b70 con 0x7f9a90004810 2023-10-06T10:42:16.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.564+0000 7f9aa359e700 1 -- 172.21.15.92:0/4002506225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a90004810 msgr2=0x7f9a90004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.564+0000 7f9aa359e700 1 --2- 172.21.15.92:0/4002506225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a90004810 0x7f9a90004c80 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00c3c0 tx=0x7f9a8c001e70 comp rx=0 tx=0).stop 2023-10-06T10:42:16.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.564+0000 7f9aa2d9d700 1 -- 172.21.15.92:0/4002506225 reap_dead start 2023-10-06T10:42:16.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.564+0000 7f9aa359e700 1 -- 172.21.15.92:0/4002506225 shutdown_connections 2023-10-06T10:42:16.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.564+0000 7f9aa359e700 1 -- 172.21.15.92:0/4002506225 >> 172.21.15.92:0/4002506225 conn(0x7f9aa40680c0 msgr2=0x7f9aa4068490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:16.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.564+0000 7f9aa359e700 1 -- 172.21.15.92:0/4002506225 shutdown_connections 2023-10-06T10:42:16.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.564+0000 7f9aa359e700 1 -- 172.21.15.92:0/4002506225 wait complete. 2023-10-06T10:42:16.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.565+0000 7f9aa359e700 1 Processor -- start 2023-10-06T10:42:16.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.565+0000 7f9aa359e700 1 -- start start 2023-10-06T10:42:16.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.566+0000 7f9aa359e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a90004810 0x7f9aa414f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.566+0000 7f9aa359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9aa40c5ad0 0x7f9aa40b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.566+0000 7f9aa359e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9aa4150aa0 0x7f9aa4150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.566+0000 7f9aa359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa4151460 con 0x7f9aa40c5ad0 2023-10-06T10:42:16.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.566+0000 7f9aa359e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa40c7ec0 con 0x7f9aa4150aa0 2023-10-06T10:42:16.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.566+0000 7f9aa359e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa40c5f60 con 0x7f9a90004810 2023-10-06T10:42:16.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.566+0000 7f9a99d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9aa40c5ad0 0x7f9aa40b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.566+0000 7f9a99d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9aa40c5ad0 0x7f9aa40b13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53304/0 (socket says 172.21.15.92:53304) 2023-10-06T10:42:16.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.566+0000 7f9a99d9b700 1 -- 172.21.15.92:0/3139481915 learned_addr learned my addr 172.21.15.92:0/3139481915 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:16.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.566+0000 7f9aa2d9d700 1 --2- 172.21.15.92:0/3139481915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9aa4150aa0 0x7f9aa4150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.566+0000 7f9aa259c700 1 --2- 172.21.15.92:0/3139481915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a90004810 0x7f9aa414f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.566+0000 7f9a99d9b700 1 -- 172.21.15.92:0/3139481915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a90004810 msgr2=0x7f9aa414f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.566+0000 7f9a99d9b700 1 --2- 172.21.15.92:0/3139481915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a90004810 0x7f9aa414f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.566+0000 7f9a99d9b700 1 -- 172.21.15.92:0/3139481915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9aa4150aa0 msgr2=0x7f9aa4150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.566+0000 7f9a99d9b700 1 --2- 172.21.15.92:0/3139481915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9aa4150aa0 0x7f9aa4150f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.566+0000 7f9a99d9b700 1 -- 172.21.15.92:0/3139481915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa40c6240 con 0x7f9aa40c5ad0 2023-10-06T10:42:16.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.566+0000 7f9a99d9b700 1 --2- 172.21.15.92:0/3139481915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9aa40c5ad0 0x7f9aa40b13c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9a90004770 tx=0x7f9a8c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.566+0000 7f9a9affd700 1 -- 172.21.15.92:0/3139481915 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c00cc90 con 0x7f9aa40c5ad0 2023-10-06T10:42:16.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.566+0000 7f9a9affd700 1 -- 172.21.15.92:0/3139481915 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8c000fe0 con 0x7f9aa40c5ad0 2023-10-06T10:42:16.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.566+0000 7f9aa359e700 1 -- 172.21.15.92:0/3139481915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa40c6520 con 0x7f9aa40c5ad0 2023-10-06T10:42:16.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.566+0000 7f9a9affd700 1 -- 172.21.15.92:0/3139481915 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c007530 con 0x7f9aa40c5ad0 2023-10-06T10:42:16.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.566+0000 7f9aa359e700 1 -- 172.21.15.92:0/3139481915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa414fcd0 con 0x7f9aa40c5ad0 2023-10-06T10:42:16.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.568+0000 7f9a9affd700 1 -- 172.21.15.92:0/3139481915 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9a8c019030 con 0x7f9aa40c5ad0 2023-10-06T10:42:16.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.568+0000 7f9a9affd700 1 --2- 172.21.15.92:0/3139481915 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a84066f30 0x7f9a840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.568+0000 7f9a9affd700 1 -- 172.21.15.92:0/3139481915 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f9a8c0862e0 con 0x7f9aa40c5ad0 2023-10-06T10:42:16.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.568+0000 7f9aa259c700 1 --2- 172.21.15.92:0/3139481915 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a84066f30 0x7f9a840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.568+0000 7f9aa259c700 1 --2- 172.21.15.92:0/3139481915 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a84066f30 0x7f9a840693e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9a94003bf0 tx=0x7f9a94009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.570+0000 7f9ab2055b80 1 Processor -- start 2023-10-06T10:42:16.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.570+0000 7f9ab2055b80 1 -- start start 2023-10-06T10:42:16.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.571+0000 7f9ab2055b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559de2486a30 0x559de2486e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.571+0000 7f9ab2055b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de24d0ca0 0x559de24afa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.571+0000 7f9ab2055b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559de24aff40 0x559de2482a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.571+0000 7f9ab2055b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de2406d60 con 0x559de24d0ca0 2023-10-06T10:42:16.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.571+0000 7f9ab2055b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de23ddb50 con 0x559de24aff40 2023-10-06T10:42:16.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.571+0000 7f9ab2055b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559de23db350 con 0x559de2486a30 2023-10-06T10:42:16.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.571+0000 7f9a99d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de24d0ca0 0x559de24afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.571+0000 7f9a99d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de24d0ca0 0x559de24afa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53314/0 (socket says 172.21.15.92:53314) 2023-10-06T10:42:16.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.571+0000 7f9a99d9b700 1 -- 172.21.15.92:0/1866532693 learned_addr learned my addr 172.21.15.92:0/1866532693 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:16.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.571+0000 7f9aa2d9d700 1 --2- 172.21.15.92:0/1866532693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559de24aff40 0x559de2482a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.571+0000 7f9aa259c700 1 --2- 172.21.15.92:0/1866532693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559de2486a30 0x559de2486e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.571+0000 7f9a99d9b700 1 -- 172.21.15.92:0/1866532693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559de2486a30 msgr2=0x559de2486e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.571+0000 7f9a99d9b700 1 --2- 172.21.15.92:0/1866532693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559de2486a30 0x559de2486e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.571+0000 7f9a99d9b700 1 -- 172.21.15.92:0/1866532693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559de24aff40 msgr2=0x559de2482a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.571+0000 7f9a99d9b700 1 --2- 172.21.15.92:0/1866532693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559de24aff40 0x559de2482a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.571+0000 7f9a99d9b700 1 -- 172.21.15.92:0/1866532693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559de2271350 con 0x559de24d0ca0 2023-10-06T10:42:16.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.571+0000 7f9aa2d9d700 1 --2- 172.21.15.92:0/1866532693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559de24aff40 0x559de2482a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:16.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.571+0000 7f9aa259c700 1 --2- 172.21.15.92:0/1866532693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559de2486a30 0x559de2486e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:16.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.571+0000 7f9a99d9b700 1 --2- 172.21.15.92:0/1866532693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de24d0ca0 0x559de24afa00 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c084520 tx=0x7f9a8c084480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0553b2bb5edb2d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.571+0000 7f9a837fe700 1 -- 172.21.15.92:0/1866532693 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c05a180 con 0x559de24d0ca0 2023-10-06T10:42:16.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.571+0000 7f9a837fe700 1 -- 172.21.15.92:0/1866532693 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8c05ded0 con 0x559de24d0ca0 2023-10-06T10:42:16.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.571+0000 7f9a837fe700 1 -- 172.21.15.92:0/1866532693 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c055070 con 0x559de24d0ca0 2023-10-06T10:42:16.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.572+0000 7f9ab2055b80 1 -- 172.21.15.92:0/1866532693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de24d0ca0 msgr2=0x559de24afa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.572+0000 7f9ab2055b80 1 --2- 172.21.15.92:0/1866532693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de24d0ca0 0x559de24afa00 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c084520 tx=0x7f9a8c084480 comp rx=0 tx=0).stop 2023-10-06T10:42:16.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.572+0000 7f9ab2055b80 1 -- 172.21.15.92:0/1866532693 shutdown_connections 2023-10-06T10:42:16.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.572+0000 7f9ab2055b80 1 --2- 172.21.15.92:0/1866532693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559de24aff40 0x559de2482a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.572+0000 7f9ab2055b80 1 --2- 172.21.15.92:0/1866532693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559de2486a30 0x559de2486e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.572+0000 7f9ab2055b80 1 --2- 172.21.15.92:0/1866532693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de24d0ca0 0x559de24afa00 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.572+0000 7f9ab2055b80 1 -- 172.21.15.92:0/1866532693 >> 172.21.15.92:0/1866532693 conn(0x559de2488350 msgr2=0x559de2488720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:16.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.572+0000 7f9ab2055b80 1 -- 172.21.15.92:0/1866532693 shutdown_connections 2023-10-06T10:42:16.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.572+0000 7f9ab2055b80 1 -- 172.21.15.92:0/1866532693 wait complete. 2023-10-06T10:42:16.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.573+0000 7f9ab2055b80 1 -- start start 2023-10-06T10:42:16.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.573+0000 7f9ab2055b80 1 Processor -- start 2023-10-06T10:42:16.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.574+0000 7f9ab2055b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559de2486a30 0x559de24c7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.574+0000 7f9ab2055b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559de24aff40 0x559de247d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.574+0000 7f9ab2055b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de24d0ca0 0x559de247da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.574+0000 7f9ab2055b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de23d7140 con 0x559de24d0ca0 2023-10-06T10:42:16.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.574+0000 7f9ab2055b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de2382970 con 0x559de2486a30 2023-10-06T10:42:16.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.574+0000 7f9ab2055b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559de239df80 con 0x559de24aff40 2023-10-06T10:42:16.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.574+0000 7f9ab2055b80 10 client.0 _ll_register_callbacks cb 0x559de2274010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:42:16.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.574+0000 7f9aa2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de24d0ca0 0x559de247da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.574+0000 7f9aa2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de24d0ca0 0x559de247da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53324/0 (socket says 172.21.15.92:53324) 2023-10-06T10:42:16.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.574+0000 7f9aa2d9d700 1 -- 172.21.15.92:0/2842349433 learned_addr learned my addr 172.21.15.92:0/2842349433 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:16.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.574+0000 7f9a99d9b700 1 --2- 172.21.15.92:0/2842349433 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559de24aff40 0x559de247d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.574+0000 7f9aa259c700 1 --2- 172.21.15.92:0/2842349433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559de2486a30 0x559de24c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.574+0000 7f9aa2d9d700 1 -- 172.21.15.92:0/2842349433 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559de24aff40 msgr2=0x559de247d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.574+0000 7f9aa2d9d700 1 --2- 172.21.15.92:0/2842349433 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559de24aff40 0x559de247d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.574+0000 7f9aa2d9d700 1 -- 172.21.15.92:0/2842349433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559de2486a30 msgr2=0x559de24c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.574+0000 7f9aa2d9d700 1 --2- 172.21.15.92:0/2842349433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559de2486a30 0x559de24c7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.574+0000 7f9aa2d9d700 1 -- 172.21.15.92:0/2842349433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa414fcd0 con 0x559de24d0ca0 2023-10-06T10:42:16.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.574+0000 7f9aa2d9d700 1 --2- 172.21.15.92:0/2842349433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de24d0ca0 0x559de247da70 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4069110 tx=0x7f9a9c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb52aae1f376d2f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.574+0000 7f9a80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:42:16.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.574+0000 7f9a80ff9700 1 -- 172.21.15.92:0/2842349433 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c00e040 con 0x559de24d0ca0 2023-10-06T10:42:16.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.574+0000 7f9a80ff9700 1 -- 172.21.15.92:0/2842349433 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f9a9c007e20 con 0x559de24d0ca0 2023-10-06T10:42:16.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.574+0000 7f9ab2055b80 1 -- 172.21.15.92:0/2842349433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559de247b770 con 0x559de24d0ca0 2023-10-06T10:42:16.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.575+0000 7f9a80ff9700 1 -- 172.21.15.92:0/2842349433 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a9c017470 con 0x559de24d0ca0 2023-10-06T10:42:16.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.575+0000 7f9ab2055b80 10 client.6882 fetch_fsmap learned FSMap version 57 2023-10-06T10:42:16.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.575+0000 7f9ab2055b80 1 -- 172.21.15.92:0/2842349433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x559de22140c0 con 0x559de24d0ca0 2023-10-06T10:42:16.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.575+0000 7f9a80ff9700 1 -- 172.21.15.92:0/2842349433 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559de247b770 con 0x559de24d0ca0 2023-10-06T10:42:16.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.575+0000 7f9a80ff9700 1 -- 172.21.15.92:0/2842349433 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 57) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f9a9c0203f0 con 0x559de24d0ca0 2023-10-06T10:42:16.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.575+0000 7f9ab2055b80 10 client.6882 fetch_fsmap finished waiting for FSMap version 57 2023-10-06T10:42:16.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.575+0000 7f9ab2055b80 10 client.6882 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-10-06T10:42:16.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.575+0000 7f9ab2055b80 20 client.6882 populate_metadata read hostname 'smithi092' 2023-10-06T10:42:16.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.575+0000 7f9ab2055b80 1 --2- 172.21.15.92:0/2842349433 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x559de24ad120 0x559de24ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.575+0000 7f9ab2055b80 4 client.6882 mds_command: new command op to 6486 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:42:16.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.575+0000 7f9ab2055b80 1 -- 172.21.15.92:0/2842349433 --> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559de24586a0 con 0x559de24ad120 2023-10-06T10:42:16.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.575+0000 7f9aa259c700 1 --2- 172.21.15.92:0/2842349433 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x559de24ad120 0x559de24ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.575+0000 7f9a80ff9700 10 client.6882 ms_handle_connect on v2:172.21.15.92:6835/818031030 2023-10-06T10:42:16.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.575+0000 7f9aa259c700 1 --2- 172.21.15.92:0/2842349433 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x559de24ad120 0x559de24ad520 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6c4bf2382b23a05 server_cookie=ada66c265b78882 in_seq=0 out_seq=0 2023-10-06T10:42:16.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.576+0000 7f9a80ff9700 1 -- 172.21.15.92:0/2842349433 <== mds.0 v2:172.21.15.92:6835/818031030 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x559de24586a0 con 0x559de24ad120 2023-10-06T10:42:16.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.576+0000 7f9a80ff9700 10 client.6882 handle_command_reply: tid=0 2023-10-06T10:42:16.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.576+0000 7f9ab2055b80 1 client.6882 shutdown 2023-10-06T10:42:16.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.576+0000 7f9ab2055b80 1 -- 172.21.15.92:0/2842349433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de24d0ca0 msgr2=0x559de247da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.576+0000 7f9ab2055b80 1 --2- 172.21.15.92:0/2842349433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de24d0ca0 0x559de247da70 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4069110 tx=0x7f9a9c00b040 comp rx=0 tx=0).stop 2023-10-06T10:42:16.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.576+0000 7f9ab2055b80 1 -- 172.21.15.92:0/2842349433 shutdown_connections 2023-10-06T10:42:16.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.576+0000 7f9ab2055b80 1 --2- 172.21.15.92:0/2842349433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de24d0ca0 0x559de247da70 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.576+0000 7f9ab2055b80 1 --2- 172.21.15.92:0/2842349433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559de2486a30 0x559de24c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.576+0000 7f9ab2055b80 1 --2- 172.21.15.92:0/2842349433 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x559de24ad120 0x559de24ad520 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.576+0000 7f9ab2055b80 1 --2- 172.21.15.92:0/2842349433 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559de24aff40 0x559de247d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.576+0000 7f9a80ff9700 0 client.6882 ms_handle_reset on v2:172.21.15.92:6835/818031030 2023-10-06T10:42:16.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.576+0000 7f9ab2055b80 1 -- 172.21.15.92:0/2842349433 >> 172.21.15.92:0/2842349433 conn(0x559de2488350 msgr2=0x559de24ac7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:16.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.576+0000 7f9ab2055b80 1 -- 172.21.15.92:0/2842349433 shutdown_connections 2023-10-06T10:42:16.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.576+0000 7f9ab2055b80 1 -- 172.21.15.92:0/2842349433 wait complete. 2023-10-06T10:42:16.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.576+0000 7f9ab2055b80 20 client.6882 trim_cache size 0 max 16384 2023-10-06T10:42:16.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.592+0000 7f9ab2055b80 1 Processor -- start 2023-10-06T10:42:16.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.592+0000 7f9ab2055b80 1 -- start start 2023-10-06T10:42:16.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.593+0000 7f9ab2055b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559de2488350 0x559de2484b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.593+0000 7f9ab2055b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559de2486a30 0x559de247d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.593+0000 7f9ab2055b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de24aff40 0x559de24ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.593+0000 7f9ab2055b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de2259f10 con 0x559de24aff40 2023-10-06T10:42:16.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.593+0000 7f9ab2055b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de24850a0 con 0x559de2488350 2023-10-06T10:42:16.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.593+0000 7f9ab2055b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559de24af410 con 0x559de2486a30 2023-10-06T10:42:16.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.593+0000 7f9aa2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de24aff40 0x559de24ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.593+0000 7f9aa2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de24aff40 0x559de24ad520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53330/0 (socket says 172.21.15.92:53330) 2023-10-06T10:42:16.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.593+0000 7f9aa2d9d700 1 -- 172.21.15.92:0/1720513358 learned_addr learned my addr 172.21.15.92:0/1720513358 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:16.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.593+0000 7f9aa259c700 1 --2- 172.21.15.92:0/1720513358 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559de2488350 0x559de2484b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.593+0000 7f9a99d9b700 1 --2- 172.21.15.92:0/1720513358 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559de2486a30 0x559de247d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.593+0000 7f9aa2d9d700 1 -- 172.21.15.92:0/1720513358 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559de2486a30 msgr2=0x559de247d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.593+0000 7f9aa2d9d700 1 --2- 172.21.15.92:0/1720513358 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559de2486a30 0x559de247d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.593+0000 7f9aa2d9d700 1 -- 172.21.15.92:0/1720513358 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559de2488350 msgr2=0x559de2484b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.593+0000 7f9aa2d9d700 1 --2- 172.21.15.92:0/1720513358 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559de2488350 0x559de2484b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.593+0000 7f9aa2d9d700 1 -- 172.21.15.92:0/1720513358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559de24889f0 con 0x559de24aff40 2023-10-06T10:42:16.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.594+0000 7f9aa2d9d700 1 --2- 172.21.15.92:0/1720513358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de24aff40 0x559de24ad520 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c000f80 tx=0x7f9a9c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44008c6270004853 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.594+0000 7f9a83fff700 1 -- 172.21.15.92:0/1720513358 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c00e040 con 0x559de24aff40 2023-10-06T10:42:16.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.594+0000 7f9a83fff700 1 -- 172.21.15.92:0/1720513358 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a9c00f040 con 0x559de24aff40 2023-10-06T10:42:16.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.594+0000 7f9a83fff700 1 -- 172.21.15.92:0/1720513358 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c020b40 con 0x559de24aff40 2023-10-06T10:42:16.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.594+0000 7f9ab2055b80 1 -- 172.21.15.92:0/1720513358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de24aff40 msgr2=0x559de24ad520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.594+0000 7f9ab2055b80 1 --2- 172.21.15.92:0/1720513358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de24aff40 0x559de24ad520 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c000f80 tx=0x7f9a9c01d000 comp rx=0 tx=0).stop 2023-10-06T10:42:16.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.594+0000 7f9ab2055b80 1 -- 172.21.15.92:0/1720513358 shutdown_connections 2023-10-06T10:42:16.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.594+0000 7f9ab2055b80 1 --2- 172.21.15.92:0/1720513358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de24aff40 0x559de24ad520 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.594+0000 7f9ab2055b80 1 --2- 172.21.15.92:0/1720513358 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559de2488350 0x559de2484b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.594+0000 7f9ab2055b80 1 --2- 172.21.15.92:0/1720513358 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559de2486a30 0x559de247d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.594+0000 7f9ab2055b80 1 -- 172.21.15.92:0/1720513358 >> 172.21.15.92:0/1720513358 conn(0x559de2484000 msgr2=0x559de24ae610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:16.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.594+0000 7f9ab2055b80 1 -- 172.21.15.92:0/1720513358 shutdown_connections 2023-10-06T10:42:16.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.594+0000 7f9ab2055b80 1 -- 172.21.15.92:0/1720513358 wait complete. 2023-10-06T10:42:16.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.595+0000 7f9ab2055b80 1 -- start start 2023-10-06T10:42:16.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.595+0000 7f9ab2055b80 1 Processor -- start 2023-10-06T10:42:16.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.596+0000 7f9ab2055b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de2486a30 0x559de252a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.596+0000 7f9ab2055b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559de2488350 0x559de24dede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.596+0000 7f9ab2055b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559de24aff40 0x559de24db3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.596+0000 7f9ab2055b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de24db230 con 0x559de2486a30 2023-10-06T10:42:16.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.596+0000 7f9ab2055b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de24dc440 con 0x559de24aff40 2023-10-06T10:42:16.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.596+0000 7f9ab2055b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559de24aa910 con 0x559de2488350 2023-10-06T10:42:16.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.596+0000 7f9ab2055b80 10 client.0 _ll_register_callbacks cb 0x559de2274010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:42:16.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.596+0000 7f9aa259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de2486a30 0x559de252a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.596+0000 7f9aa259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de2486a30 0x559de252a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53342/0 (socket says 172.21.15.92:53342) 2023-10-06T10:42:16.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.596+0000 7f9aa259c700 1 -- 172.21.15.92:0/3653380595 learned_addr learned my addr 172.21.15.92:0/3653380595 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:16.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.596+0000 7f9aa2d9d700 1 --2- 172.21.15.92:0/3653380595 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559de24aff40 0x559de24db3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.596+0000 7f9a99d9b700 1 --2- 172.21.15.92:0/3653380595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559de2488350 0x559de24dede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.596+0000 7f9aa259c700 1 -- 172.21.15.92:0/3653380595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559de2488350 msgr2=0x559de24dede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.596+0000 7f9aa259c700 1 --2- 172.21.15.92:0/3653380595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559de2488350 0x559de24dede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.596+0000 7f9aa259c700 1 -- 172.21.15.92:0/3653380595 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559de24aff40 msgr2=0x559de24db3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.596+0000 7f9aa259c700 1 --2- 172.21.15.92:0/3653380595 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559de24aff40 0x559de24db3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.596+0000 7f9aa259c700 1 -- 172.21.15.92:0/3653380595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559de22140c0 con 0x559de2486a30 2023-10-06T10:42:16.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.596+0000 7f9aa259c700 1 --2- 172.21.15.92:0/3653380595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de2486a30 0x559de252a1a0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x559de24c82d0 tx=0x7f9a94028040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62293c597a033ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.596+0000 7f9a80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:42:16.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.596+0000 7f9a80ff9700 1 -- 172.21.15.92:0/3653380595 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a94026b50 con 0x559de2486a30 2023-10-06T10:42:16.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.596+0000 7f9a80ff9700 1 -- 172.21.15.92:0/3653380595 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f9a94030db0 con 0x559de2486a30 2023-10-06T10:42:16.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.596+0000 7f9ab2055b80 1 -- 172.21.15.92:0/3653380595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a8c05ded0 con 0x559de2486a30 2023-10-06T10:42:16.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.597+0000 7f9a80ff9700 1 -- 172.21.15.92:0/3653380595 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a94015460 con 0x559de2486a30 2023-10-06T10:42:16.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.597+0000 7f9ab2055b80 10 client.6888 fetch_fsmap learned FSMap version 57 2023-10-06T10:42:16.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.597+0000 7f9ab2055b80 1 -- 172.21.15.92:0/3653380595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x559de23a5860 con 0x559de2486a30 2023-10-06T10:42:16.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.597+0000 7f9a80ff9700 1 -- 172.21.15.92:0/3653380595 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8c05ded0 con 0x559de2486a30 2023-10-06T10:42:16.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.597+0000 7f9a80ff9700 1 -- 172.21.15.92:0/3653380595 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 57) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f9a940155c0 con 0x559de2486a30 2023-10-06T10:42:16.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.597+0000 7f9ab2055b80 10 client.6888 fetch_fsmap finished waiting for FSMap version 57 2023-10-06T10:42:16.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.597+0000 7f9ab2055b80 10 client.6888 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-10-06T10:42:16.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.597+0000 7f9ab2055b80 20 client.6888 populate_metadata read hostname 'smithi092' 2023-10-06T10:42:16.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.597+0000 7f9ab2055b80 1 --2- 172.21.15.92:0/3653380595 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x559de24d0ca0 0x559de24db8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.597+0000 7f9ab2055b80 4 client.6888 mds_command: new command op to 6486 tid=0[{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2023-10-06T10:42:16.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.597+0000 7f9ab2055b80 1 -- 172.21.15.92:0/3653380595 --> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) v1 -- 0x559de2379ee0 con 0x559de24d0ca0 2023-10-06T10:42:16.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.597+0000 7f9a99d9b700 1 --2- 172.21.15.92:0/3653380595 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x559de24d0ca0 0x559de24db8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.597+0000 7f9a80ff9700 10 client.6888 ms_handle_connect on v2:172.21.15.92:6835/818031030 2023-10-06T10:42:16.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.597+0000 7f9a99d9b700 1 --2- 172.21.15.92:0/3653380595 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x559de24d0ca0 0x559de24db8e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2bdb0a29c3bf130 server_cookie=dc47f4387eca3c9 in_seq=0 out_seq=0 2023-10-06T10:42:16.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.598+0000 7f9a80ff9700 1 -- 172.21.15.92:0/3653380595 <== mds.0 v2:172.21.15.92:6835/818031030 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559de2379ee0 con 0x559de24d0ca0 2023-10-06T10:42:16.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.598+0000 7f9a80ff9700 10 client.6888 handle_command_reply: tid=0 2023-10-06T10:42:16.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.598+0000 7f9ab2055b80 1 client.6888 shutdown 2023-10-06T10:42:16.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.598+0000 7f9ab2055b80 1 -- 172.21.15.92:0/3653380595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de2486a30 msgr2=0x559de252a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.598+0000 7f9ab2055b80 1 --2- 172.21.15.92:0/3653380595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de2486a30 0x559de252a1a0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x559de24c82d0 tx=0x7f9a94028040 comp rx=0 tx=0).stop 2023-10-06T10:42:16.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.598+0000 7f9ab2055b80 1 -- 172.21.15.92:0/3653380595 shutdown_connections 2023-10-06T10:42:16.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.598+0000 7f9ab2055b80 1 --2- 172.21.15.92:0/3653380595 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559de24aff40 0x559de24db3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.598+0000 7f9ab2055b80 1 --2- 172.21.15.92:0/3653380595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559de2486a30 0x559de252a1a0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.598+0000 7f9ab2055b80 1 --2- 172.21.15.92:0/3653380595 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x559de24d0ca0 0x559de24db8e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.598+0000 7f9ab2055b80 1 --2- 172.21.15.92:0/3653380595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559de2488350 0x559de24dede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.598+0000 7f9a80ff9700 0 client.6888 ms_handle_reset on v2:172.21.15.92:6835/818031030 2023-10-06T10:42:16.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.598+0000 7f9ab2055b80 1 -- 172.21.15.92:0/3653380595 >> 172.21.15.92:0/3653380595 conn(0x559de2484000 msgr2=0x559de24ad520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:16.607 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-10-06T10:42:16.598+0000 7fdec44a7700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2023-10-06T10:42:00.018428+0000 1=0+1) n(v3 rc2023-10-06T10:42:00.248421+0000 6=0+6) (inest mix dirty) (ifile excl) (iversion lock) caps={6112=pAsLsXsFsx/pAsLsXsFsx@5},l=6112 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x5573e59ba100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:42:00.018428+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:42:00.248421+0000 6=0+6)","memory_value.dirstat":"f(v0 m2023-10-06T10:42:00.018428+0000 1=0+1)","memory_value.rstat":"n(v3 rc2023-10-06T10:42:00.248421+0000 6=0+6)","error_str":""},"return_code":-61} 2023-10-06T10:42:16.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.598+0000 7f9ab2055b80 1 -- 172.21.15.92:0/3653380595 shutdown_connections 2023-10-06T10:42:16.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.598+0000 7f9ab2055b80 1 -- 172.21.15.92:0/3653380595 wait complete. 2023-10-06T10:42:16.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.598+0000 7f9ab2055b80 20 client.6888 trim_cache size 0 max 16384 2023-10-06T10:42:16.608 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:42:16.609 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0, 2023-10-06T10:42:16.609 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_tag": "3d9baef9-cff7-482f-9129-a793337c06e6", 2023-10-06T10:42:16.609 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": "asynchronous" 2023-10-06T10:42:16.609 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:42:16.609 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-10-06T10:42:16.599+0000 7faa59df7700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2023-10-06T10:42:00.019428+0000 1=0+1) n(v2 rc2023-10-06T10:42:00.248421+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={6112=pAsLsXsFsx/pAsLsXsFsx@4},l=6112 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x56421d0c2000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:42:00.019428+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:42:00.248421+0000 4=0+4)","memory_value.dirstat":"f(v0 m2023-10-06T10:42:00.019428+0000 1=0+1)","memory_value.rstat":"n(v2 rc2023-10-06T10:42:00.248421+0000 4=0+4)","error_str":""},"return_code":-61} 2023-10-06T10:42:16.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.599+0000 7f9aa359e700 1 -- 172.21.15.92:0/3139481915 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a84066f30 msgr2=0x7f9a840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.599+0000 7f9aa359e700 1 --2- 172.21.15.92:0/3139481915 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a84066f30 0x7f9a840693e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9a94003bf0 tx=0x7f9a94009040 comp rx=0 tx=0).stop 2023-10-06T10:42:16.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.599+0000 7f9aa359e700 1 -- 172.21.15.92:0/3139481915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9aa40c5ad0 msgr2=0x7f9aa40b13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.599+0000 7f9aa359e700 1 --2- 172.21.15.92:0/3139481915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9aa40c5ad0 0x7f9aa40b13c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9a90004770 tx=0x7f9a8c015000 comp rx=0 tx=0).stop 2023-10-06T10:42:16.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.599+0000 7f9aa359e700 1 -- 172.21.15.92:0/3139481915 shutdown_connections 2023-10-06T10:42:16.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.599+0000 7f9aa359e700 1 --2- 172.21.15.92:0/3139481915 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a84066f30 0x7f9a840693e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.599+0000 7f9aa359e700 1 --2- 172.21.15.92:0/3139481915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9aa4150aa0 0x7f9aa4150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.599+0000 7f9aa359e700 1 --2- 172.21.15.92:0/3139481915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a90004810 0x7f9aa414f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.599+0000 7f9aa359e700 1 --2- 172.21.15.92:0/3139481915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9aa40c5ad0 0x7f9aa40b13c0 secure :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9a90004770 tx=0x7f9a8c015000 comp rx=0 tx=0).stop 2023-10-06T10:42:16.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.599+0000 7f9aa359e700 1 -- 172.21.15.92:0/3139481915 >> 172.21.15.92:0/3139481915 conn(0x7f9aa40680c0 msgr2=0x7f9aa415c410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:16.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.599+0000 7f9aa359e700 1 -- 172.21.15.92:0/3139481915 shutdown_connections 2023-10-06T10:42:16.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.599+0000 7f9aa359e700 1 -- 172.21.15.92:0/3139481915 wait complete. 2023-10-06T10:42:16.611 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-10-06T10:42:16.600+0000 7fdeca4b3700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2023-10-06T10:42:00.018428+0000 1=0+1) n(v4 rc2023-10-06T10:42:00.248421+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={6112=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x5573e5a28000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:42:00.018428+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:42:00.248421+0000 5=0+5)","memory_value.dirstat":"f(v0 m2023-10-06T10:42:00.018428+0000 1=0+1)","memory_value.rstat":"n(v4 rc2023-10-06T10:42:00.248421+0000 5=0+5)","error_str":""},"return_code":-61} 2023-10-06T10:42:16.612 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-10-06T10:42:16.600+0000 7faa5fe03700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2023-10-06T10:42:00.019428+0000 1=0+1) n(v1 rc2023-10-06T10:42:00.248421+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={6112=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x56421d0c2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:42:00.019428+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:42:00.020428+0000 3=0+3)","memory_value.dirstat":"f(v0 m2023-10-06T10:42:00.019428+0000 1=0+1)","memory_value.rstat":"n(v1 rc2023-10-06T10:42:00.248421+0000 3=0+3)","error_str":""},"return_code":-61} 2023-10-06T10:42:16.612 INFO:tasks.ceph.mds.b.smithi124.stderr:2023-10-06T10:42:16.599+0000 7f75e6ba2700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2023-10-06T10:42:00.020428+0000 1=0+1) n(v0 rc2023-10-06T10:42:00.020428+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={6112=pAsLsXsFsx/pAsLsXsFsx@4},l=6112 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55f480e4ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:42:00.020428+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:42:00.020428+0000 2=0+2)","memory_value.dirstat":"f(v0 m2023-10-06T10:42:00.020428+0000 1=0+1)","memory_value.rstat":"n(v0 rc2023-10-06T10:42:00.020428+0000 2=0+2)","error_str":""},"return_code":-61} 2023-10-06T10:42:16.612 INFO:tasks.ceph.mds.b.smithi124.stderr:2023-10-06T10:42:16.599+0000 7f75e6ba2700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={6112=pAsLsXsFsx/AsLsXsFsx@2},l=6112 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55f480e4b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2023-10-06T10:42:16.613 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-10-06T10:42:16.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.727+0000 7fa99ad83700 1 Processor -- start 2023-10-06T10:42:16.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.727+0000 7fa99ad83700 1 -- start start 2023-10-06T10:42:16.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.728+0000 7fa99ad83700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9940b29f0 0x7fa994164320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.728+0000 7fa99ad83700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9940cc970 con 0x7fa9940cd3c0 2023-10-06T10:42:16.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.728+0000 7fa99ad83700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9940cbd60 con 0x7fa9940b2620 2023-10-06T10:42:16.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.728+0000 7fa99ad83700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940cc040 con 0x7fa9940b29f0 2023-10-06T10:42:16.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.728+0000 7fa999580700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa9940cd3c0 0x7fa9940ceaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56534/0 (socket says 172.21.15.92:56534) 2023-10-06T10:42:16.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.728+0000 7fa999580700 1 -- 172.21.15.92:0/1640734147 learned_addr learned my addr 172.21.15.92:0/1640734147 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:42:16.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.728+0000 7fa998d7f700 1 -- 172.21.15.92:0/1640734147 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1464389067 0 0) 0x7fa9940cc970 con 0x7fa9940cd3c0 2023-10-06T10:42:16.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.728+0000 7fa99a582700 1 --2- 172.21.15.92:0/1640734147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9940b29f0 0x7fa994164320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.728+0000 7fa998d7f700 1 -- 172.21.15.92:0/1640734147 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa984003570 con 0x7fa9940cd3c0 2023-10-06T10:42:16.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa998d7f700 1 -- 172.21.15.92:0/1640734147 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 564121256 0 0) 0x7fa984003570 con 0x7fa9940cd3c0 2023-10-06T10:42:16.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa998d7f700 1 -- 172.21.15.92:0/1640734147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9940b29f0 msgr2=0x7fa994164320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa998d7f700 1 --2- 172.21.15.92:0/1640734147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9940b29f0 0x7fa994164320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa998d7f700 1 -- 172.21.15.92:0/1640734147 >> v1:172.21.15.124:6789/0 conn(0x7fa9940b2620 legacy=0x7fa9940cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa998d7f700 1 -- 172.21.15.92:0/1640734147 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9940cc320 con 0x7fa9940cd3c0 2023-10-06T10:42:16.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa998d7f700 1 -- 172.21.15.92:0/1640734147 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fa980001470 con 0x7fa9940cd3c0 2023-10-06T10:42:16.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa998d7f700 1 -- 172.21.15.92:0/1640734147 >> v1:172.21.15.92:6789/0 conn(0x7fa9940cd3c0 legacy=0x7fa9940ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa998d7f700 1 --2- 172.21.15.92:0/1640734147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9940b29f0 0x7fa984003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa998d7f700 1 --2- 172.21.15.92:0/1640734147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa984004060 0x7fa984004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa998d7f700 1 --2- 172.21.15.92:0/1640734147 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa984004d50 0x7fa9840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa998d7f700 1 -- 172.21.15.92:0/1640734147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940cc040 con 0x7fa984004060 2023-10-06T10:42:16.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa998d7f700 1 -- 172.21.15.92:0/1640734147 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98400bc20 con 0x7fa984004d50 2023-10-06T10:42:16.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa998d7f700 1 -- 172.21.15.92:0/1640734147 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98400bde0 con 0x7fa9940b29f0 2023-10-06T10:42:16.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa999580700 1 --2- 172.21.15.92:0/1640734147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa984004060 0x7fa984004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa999d81700 1 --2- 172.21.15.92:0/1640734147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9940b29f0 0x7fa984003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa99a582700 1 --2- 172.21.15.92:0/1640734147 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa984004d50 0x7fa9840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa999580700 1 -- 172.21.15.92:0/1640734147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9940b29f0 msgr2=0x7fa984003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa999580700 1 --2- 172.21.15.92:0/1640734147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9940b29f0 0x7fa984003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa999580700 1 -- 172.21.15.92:0/1640734147 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa984004d50 msgr2=0x7fa9840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa999580700 1 --2- 172.21.15.92:0/1640734147 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa984004d50 0x7fa9840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa999580700 1 -- 172.21.15.92:0/1640734147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa984003570 con 0x7fa984004060 2023-10-06T10:42:16.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa999580700 1 --2- 172.21.15.92:0/1640734147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa984004060 0x7fa984004810 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fa98000c2a0 tx=0x7fa980001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c39fba8f4af130 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.729+0000 7fa998d7f700 1 -- 172.21.15.92:0/1640734147 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98000cb70 con 0x7fa984004060 2023-10-06T10:42:16.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.730+0000 7fa998d7f700 1 -- 172.21.15.92:0/1640734147 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98000ccf0 con 0x7fa984004060 2023-10-06T10:42:16.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.730+0000 7fa998d7f700 1 -- 172.21.15.92:0/1640734147 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa980005b70 con 0x7fa984004060 2023-10-06T10:42:16.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.730+0000 7fa99ad83700 1 -- 172.21.15.92:0/1640734147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa984004060 msgr2=0x7fa984004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.730+0000 7fa99ad83700 1 --2- 172.21.15.92:0/1640734147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa984004060 0x7fa984004810 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fa98000c2a0 tx=0x7fa980001d80 comp rx=0 tx=0).stop 2023-10-06T10:42:16.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.730+0000 7fa99a582700 1 -- 172.21.15.92:0/1640734147 reap_dead start 2023-10-06T10:42:16.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.730+0000 7fa99ad83700 1 -- 172.21.15.92:0/1640734147 shutdown_connections 2023-10-06T10:42:16.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.730+0000 7fa99ad83700 1 -- 172.21.15.92:0/1640734147 >> 172.21.15.92:0/1640734147 conn(0x7fa994099520 msgr2=0x7fa99406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:16.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.730+0000 7fa99ad83700 1 -- 172.21.15.92:0/1640734147 shutdown_connections 2023-10-06T10:42:16.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.730+0000 7fa99ad83700 1 -- 172.21.15.92:0/1640734147 wait complete. 2023-10-06T10:42:16.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.731+0000 7fa99ad83700 1 Processor -- start 2023-10-06T10:42:16.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.731+0000 7fa99ad83700 1 -- start start 2023-10-06T10:42:16.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.732+0000 7fa99ad83700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa984004060 0x7fa994159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.732+0000 7fa99ad83700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa99415bf70 0x7fa994150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.732+0000 7fa99ad83700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9940c98a0 0x7fa9940c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.732+0000 7fa99ad83700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99415c3c0 con 0x7fa9940c98a0 2023-10-06T10:42:16.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.732+0000 7fa99ad83700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940ca260 con 0x7fa984004060 2023-10-06T10:42:16.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.732+0000 7fa99ad83700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940c8ec0 con 0x7fa99415bf70 2023-10-06T10:42:16.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.732+0000 7fa99a582700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9940c98a0 0x7fa9940c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.732+0000 7fa99a582700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9940c98a0 0x7fa9940c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53360/0 (socket says 172.21.15.92:53360) 2023-10-06T10:42:16.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.732+0000 7fa99a582700 1 -- 172.21.15.92:0/481626448 learned_addr learned my addr 172.21.15.92:0/481626448 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:16.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.732+0000 7fa999d81700 1 --2- 172.21.15.92:0/481626448 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa984004060 0x7fa994159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.732+0000 7fa99a582700 1 -- 172.21.15.92:0/481626448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa99415bf70 msgr2=0x7fa994150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.732+0000 7fa99a582700 1 --2- 172.21.15.92:0/481626448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa99415bf70 0x7fa994150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.732+0000 7fa99a582700 1 -- 172.21.15.92:0/481626448 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa984004060 msgr2=0x7fa994159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.732+0000 7fa99a582700 1 --2- 172.21.15.92:0/481626448 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa984004060 0x7fa994159bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.732+0000 7fa99a582700 1 -- 172.21.15.92:0/481626448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9940b3460 con 0x7fa9940c98a0 2023-10-06T10:42:16.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.732+0000 7fa99a582700 1 --2- 172.21.15.92:0/481626448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9940c98a0 0x7fa9940c9d20 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fa9940b2e10 tx=0x7fa98c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.733+0000 7fa991ffb700 1 -- 172.21.15.92:0/481626448 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c00f940 con 0x7fa9940c98a0 2023-10-06T10:42:16.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.733+0000 7fa991ffb700 1 -- 172.21.15.92:0/481626448 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c00faa0 con 0x7fa9940c98a0 2023-10-06T10:42:16.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.733+0000 7fa991ffb700 1 -- 172.21.15.92:0/481626448 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c00fe40 con 0x7fa9940c98a0 2023-10-06T10:42:16.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.733+0000 7fa99ad83700 1 -- 172.21.15.92:0/481626448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9940b3130 con 0x7fa9940c98a0 2023-10-06T10:42:16.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.733+0000 7fa99ad83700 1 -- 172.21.15.92:0/481626448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99415c880 con 0x7fa9940c98a0 2023-10-06T10:42:16.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.734+0000 7fa991ffb700 1 -- 172.21.15.92:0/481626448 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa98c014650 con 0x7fa9940c98a0 2023-10-06T10:42:16.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.735+0000 7fa991ffb700 1 --2- 172.21.15.92:0/481626448 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa978066f30 0x7fa9780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.735+0000 7fa991ffb700 1 -- 172.21.15.92:0/481626448 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fa98c084f30 con 0x7fa9940c98a0 2023-10-06T10:42:16.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.735+0000 7fa999d81700 1 --2- 172.21.15.92:0/481626448 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa978066f30 0x7fa9780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.735+0000 7fa999d81700 1 --2- 172.21.15.92:0/481626448 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa978066f30 0x7fa9780693e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa9880042e0 tx=0x7fa988001aa0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.736+0000 7fa9a52ddb80 1 Processor -- start 2023-10-06T10:42:16.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.737+0000 7fa9a52ddb80 1 -- start start 2023-10-06T10:42:16.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.737+0000 7fa9a52ddb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b009a0 0x5572e4b00d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.737+0000 7fa9a52ddb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5572e4b4ac10 0x5572e4b29970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.737+0000 7fa9a52ddb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5572e4b29eb0 0x5572e4afc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.737+0000 7fa9a52ddb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e4a80cd0 con 0x5572e4b009a0 2023-10-06T10:42:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.737+0000 7fa9a52ddb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e4a57ac0 con 0x5572e4b4ac10 2023-10-06T10:42:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.737+0000 7fa9a52ddb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5572e4a552c0 con 0x5572e4b29eb0 2023-10-06T10:42:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.737+0000 7fa999d81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b009a0 0x5572e4b00d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.737+0000 7fa999d81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b009a0 0x5572e4b00d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53368/0 (socket says 172.21.15.92:53368) 2023-10-06T10:42:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.737+0000 7fa999d81700 1 -- 172.21.15.92:0/1876589763 learned_addr learned my addr 172.21.15.92:0/1876589763 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.737+0000 7fa999580700 1 --2- 172.21.15.92:0/1876589763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5572e4b4ac10 0x5572e4b29970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.738+0000 7fa99a582700 1 --2- 172.21.15.92:0/1876589763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5572e4b29eb0 0x5572e4afc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.738+0000 7fa999d81700 1 -- 172.21.15.92:0/1876589763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5572e4b29eb0 msgr2=0x5572e4afc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.738+0000 7fa999d81700 1 --2- 172.21.15.92:0/1876589763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5572e4b29eb0 0x5572e4afc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.738+0000 7fa999d81700 1 -- 172.21.15.92:0/1876589763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5572e4b4ac10 msgr2=0x5572e4b29970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.738+0000 7fa999d81700 1 --2- 172.21.15.92:0/1876589763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5572e4b4ac10 0x5572e4b29970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.738+0000 7fa999d81700 1 -- 172.21.15.92:0/1876589763 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572e48eb2c0 con 0x5572e4b009a0 2023-10-06T10:42:16.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.738+0000 7fa99a582700 1 --2- 172.21.15.92:0/1876589763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5572e4b29eb0 0x5572e4afc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:16.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.738+0000 7fa999d81700 1 --2- 172.21.15.92:0/1876589763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b009a0 0x5572e4b00d70 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fa988004200 tx=0x7fa988016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568a55d7cdebb7ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.738+0000 7fa976ffd700 1 -- 172.21.15.92:0/1876589763 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988020440 con 0x5572e4b009a0 2023-10-06T10:42:16.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.738+0000 7fa976ffd700 1 -- 172.21.15.92:0/1876589763 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa988019070 con 0x5572e4b009a0 2023-10-06T10:42:16.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.738+0000 7fa976ffd700 1 -- 172.21.15.92:0/1876589763 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9880207b0 con 0x5572e4b009a0 2023-10-06T10:42:16.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.738+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/1876589763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b009a0 msgr2=0x5572e4b00d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.738+0000 7fa9a52ddb80 1 --2- 172.21.15.92:0/1876589763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b009a0 0x5572e4b00d70 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fa988004200 tx=0x7fa988016000 comp rx=0 tx=0).stop 2023-10-06T10:42:16.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.738+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/1876589763 shutdown_connections 2023-10-06T10:42:16.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.739+0000 7fa9a52ddb80 1 --2- 172.21.15.92:0/1876589763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5572e4b29eb0 0x5572e4afc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.739+0000 7fa9a52ddb80 1 --2- 172.21.15.92:0/1876589763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b009a0 0x5572e4b00d70 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.739+0000 7fa9a52ddb80 1 --2- 172.21.15.92:0/1876589763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5572e4b4ac10 0x5572e4b29970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.739+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/1876589763 >> 172.21.15.92:0/1876589763 conn(0x5572e4b022c0 msgr2=0x5572e4b02690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:16.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.739+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/1876589763 shutdown_connections 2023-10-06T10:42:16.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.739+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/1876589763 wait complete. 2023-10-06T10:42:16.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.739+0000 7fa9a52ddb80 1 -- start start 2023-10-06T10:42:16.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.739+0000 7fa9a52ddb80 1 Processor -- start 2023-10-06T10:42:16.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.740+0000 7fa9a52ddb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b009a0 0x5572e4b57670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.740+0000 7fa9a52ddb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5572e4b29eb0 0x5572e4af71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.740+0000 7fa9a52ddb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5572e4b4ac10 0x5572e4af7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.740+0000 7fa9a52ddb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e4a510b0 con 0x5572e4b009a0 2023-10-06T10:42:16.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.740+0000 7fa9a52ddb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e49fc8e0 con 0x5572e4b4ac10 2023-10-06T10:42:16.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.740+0000 7fa9a52ddb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5572e4a17ef0 con 0x5572e4b29eb0 2023-10-06T10:42:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.740+0000 7fa999d81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b009a0 0x5572e4b57670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.740+0000 7fa9a52ddb80 10 client.0 _ll_register_callbacks cb 0x5572e48edf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:42:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.740+0000 7fa999d81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b009a0 0x5572e4b57670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53380/0 (socket says 172.21.15.92:53380) 2023-10-06T10:42:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.740+0000 7fa999d81700 1 -- 172.21.15.92:0/2074220343 learned_addr learned my addr 172.21.15.92:0/2074220343 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.740+0000 7fa99a582700 1 --2- 172.21.15.92:0/2074220343 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5572e4b4ac10 0x5572e4af7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.740+0000 7fa999580700 1 --2- 172.21.15.92:0/2074220343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5572e4b29eb0 0x5572e4af71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa999d81700 1 -- 172.21.15.92:0/2074220343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5572e4b29eb0 msgr2=0x5572e4af71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa999d81700 1 --2- 172.21.15.92:0/2074220343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5572e4b29eb0 0x5572e4af71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa999d81700 1 -- 172.21.15.92:0/2074220343 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5572e4b4ac10 msgr2=0x5572e4af7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa999d81700 1 --2- 172.21.15.92:0/2074220343 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5572e4b4ac10 0x5572e4af7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa999d81700 1 -- 172.21.15.92:0/2074220343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9940cbd60 con 0x5572e4b009a0 2023-10-06T10:42:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa99a582700 1 --2- 172.21.15.92:0/2074220343 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5572e4b4ac10 0x5572e4af7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa999d81700 1 --2- 172.21.15.92:0/2074220343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b009a0 0x5572e4b57670 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fa988022040 tx=0x7fa988019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d07a4360d3f23c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa96ffff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:42:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa96ffff700 1 -- 172.21.15.92:0/2074220343 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988015610 con 0x5572e4b009a0 2023-10-06T10:42:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa96ffff700 1 -- 172.21.15.92:0/2074220343 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fa988015770 con 0x5572e4b009a0 2023-10-06T10:42:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/2074220343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572e4af56e0 con 0x5572e4b009a0 2023-10-06T10:42:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa96ffff700 1 -- 172.21.15.92:0/2074220343 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa988015b40 con 0x5572e4b009a0 2023-10-06T10:42:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa9a52ddb80 10 client.6900 fetch_fsmap learned FSMap version 57 2023-10-06T10:42:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/2074220343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5572e488e030 con 0x5572e4b009a0 2023-10-06T10:42:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa96ffff700 1 -- 172.21.15.92:0/2074220343 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5572e4af56e0 con 0x5572e4b009a0 2023-10-06T10:42:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa96ffff700 1 -- 172.21.15.92:0/2074220343 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 57) v1 ==== 3136+0+0 (secure 0 0 0) 0x7fa988015ca0 con 0x5572e4b009a0 2023-10-06T10:42:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa9a52ddb80 10 client.6900 fetch_fsmap finished waiting for FSMap version 57 2023-10-06T10:42:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa9a52ddb80 10 client.6900 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-10-06T10:42:16.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa9a52ddb80 20 client.6900 populate_metadata read hostname 'smithi092' 2023-10-06T10:42:16.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa9a52ddb80 1 --2- 172.21.15.92:0/2074220343 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x5572e4b41920 0x5572e4b5aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa9a52ddb80 4 client.6900 mds_command: new command op to 6486 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:42:16.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/2074220343 --> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572e4ad2610 con 0x5572e4b41920 2023-10-06T10:42:16.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.741+0000 7fa999580700 1 --2- 172.21.15.92:0/2074220343 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x5572e4b41920 0x5572e4b5aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.742+0000 7fa96ffff700 10 client.6900 ms_handle_connect on v2:172.21.15.92:6835/818031030 2023-10-06T10:42:16.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.742+0000 7fa999580700 1 --2- 172.21.15.92:0/2074220343 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x5572e4b41920 0x5572e4b5aaf0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1cc29807e4417ec server_cookie=b514b196eede6195 in_seq=0 out_seq=0 2023-10-06T10:42:16.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.742+0000 7fa96ffff700 1 -- 172.21.15.92:0/2074220343 <== mds.0 v2:172.21.15.92:6835/818031030 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5572e4ad2610 con 0x5572e4b41920 2023-10-06T10:42:16.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.742+0000 7fa96ffff700 10 client.6900 handle_command_reply: tid=0 2023-10-06T10:42:16.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.742+0000 7fa9a52ddb80 1 client.6900 shutdown 2023-10-06T10:42:16.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.742+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/2074220343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b009a0 msgr2=0x5572e4b57670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.742+0000 7fa9a52ddb80 1 --2- 172.21.15.92:0/2074220343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b009a0 0x5572e4b57670 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fa988022040 tx=0x7fa988019000 comp rx=0 tx=0).stop 2023-10-06T10:42:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.742+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/2074220343 shutdown_connections 2023-10-06T10:42:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.742+0000 7fa9a52ddb80 1 --2- 172.21.15.92:0/2074220343 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5572e4b4ac10 0x5572e4af7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.742+0000 7fa9a52ddb80 1 --2- 172.21.15.92:0/2074220343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b009a0 0x5572e4b57670 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.742+0000 7fa9a52ddb80 1 --2- 172.21.15.92:0/2074220343 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x5572e4b41920 0x5572e4b5aaf0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.742+0000 7fa9a52ddb80 1 --2- 172.21.15.92:0/2074220343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5572e4b29eb0 0x5572e4af71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.742+0000 7fa96ffff700 0 client.6900 ms_handle_reset on v2:172.21.15.92:6835/818031030 2023-10-06T10:42:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.742+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/2074220343 >> 172.21.15.92:0/2074220343 conn(0x5572e4b022c0 msgr2=0x5572e4b464f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.743+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/2074220343 shutdown_connections 2023-10-06T10:42:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.743+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/2074220343 wait complete. 2023-10-06T10:42:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.743+0000 7fa9a52ddb80 20 client.6900 trim_cache size 0 max 16384 2023-10-06T10:42:16.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.758+0000 7fa9a52ddb80 1 Processor -- start 2023-10-06T10:42:16.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.758+0000 7fa9a52ddb80 1 -- start start 2023-10-06T10:42:16.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.759+0000 7fa9a52ddb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b009a0 0x5572e4af71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.759+0000 7fa9a52ddb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5572e4b29eb0 0x5572e4af7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.759+0000 7fa9a52ddb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5572e4b4ac10 0x5572e4b56d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.759+0000 7fa9a52ddb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e48d3e80 con 0x5572e4b009a0 2023-10-06T10:42:16.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.759+0000 7fa9a52ddb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e4b462a0 con 0x5572e4b29eb0 2023-10-06T10:42:16.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.759+0000 7fa9a52ddb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5572e4afce20 con 0x5572e4b4ac10 2023-10-06T10:42:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.759+0000 7fa999d81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b009a0 0x5572e4af71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.759+0000 7fa999d81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b009a0 0x5572e4af71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53386/0 (socket says 172.21.15.92:53386) 2023-10-06T10:42:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.759+0000 7fa999d81700 1 -- 172.21.15.92:0/136993273 learned_addr learned my addr 172.21.15.92:0/136993273 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.759+0000 7fa999580700 1 --2- 172.21.15.92:0/136993273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5572e4b29eb0 0x5572e4af7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.759+0000 7fa99a582700 1 --2- 172.21.15.92:0/136993273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5572e4b4ac10 0x5572e4b56d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.759+0000 7fa999d81700 1 -- 172.21.15.92:0/136993273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5572e4b4ac10 msgr2=0x5572e4b56d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.759+0000 7fa999d81700 1 --2- 172.21.15.92:0/136993273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5572e4b4ac10 0x5572e4b56d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.759+0000 7fa999d81700 1 -- 172.21.15.92:0/136993273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5572e4b29eb0 msgr2=0x5572e4af7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.759+0000 7fa999d81700 1 --2- 172.21.15.92:0/136993273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5572e4b29eb0 0x5572e4af7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.759+0000 7fa999d81700 1 -- 172.21.15.92:0/136993273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572e4a1f7d0 con 0x5572e4b009a0 2023-10-06T10:42:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.759+0000 7fa999580700 1 --2- 172.21.15.92:0/136993273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5572e4b29eb0 0x5572e4af7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:16.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.759+0000 7fa99a582700 1 --2- 172.21.15.92:0/136993273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5572e4b4ac10 0x5572e4b56d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:16.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.760+0000 7fa999d81700 1 --2- 172.21.15.92:0/136993273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b009a0 0x5572e4af71f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fa988004200 tx=0x7fa988004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2087348396f51d2d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.760+0000 7fa9777fe700 1 -- 172.21.15.92:0/136993273 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988015610 con 0x5572e4b009a0 2023-10-06T10:42:16.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.760+0000 7fa9777fe700 1 -- 172.21.15.92:0/136993273 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa988015770 con 0x5572e4b009a0 2023-10-06T10:42:16.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.760+0000 7fa9777fe700 1 -- 172.21.15.92:0/136993273 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988014d50 con 0x5572e4b009a0 2023-10-06T10:42:16.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.760+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/136993273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b009a0 msgr2=0x5572e4af71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.760+0000 7fa9a52ddb80 1 --2- 172.21.15.92:0/136993273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b009a0 0x5572e4af71f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fa988004200 tx=0x7fa988004050 comp rx=0 tx=0).stop 2023-10-06T10:42:16.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.760+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/136993273 shutdown_connections 2023-10-06T10:42:16.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.760+0000 7fa9a52ddb80 1 --2- 172.21.15.92:0/136993273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5572e4b4ac10 0x5572e4b56d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.760+0000 7fa9a52ddb80 1 --2- 172.21.15.92:0/136993273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b009a0 0x5572e4af71f0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.760+0000 7fa9a52ddb80 1 --2- 172.21.15.92:0/136993273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5572e4b29eb0 0x5572e4af7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.760+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/136993273 >> 172.21.15.92:0/136993273 conn(0x5572e4b022c0 msgr2=0x5572e4b45a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:16.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.760+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/136993273 shutdown_connections 2023-10-06T10:42:16.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.760+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/136993273 wait complete. 2023-10-06T10:42:16.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.761+0000 7fa9a52ddb80 1 -- start start 2023-10-06T10:42:16.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.761+0000 7fa9a52ddb80 1 Processor -- start 2023-10-06T10:42:16.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.762+0000 7fa9a52ddb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5572e4b009a0 0x5572e4bad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.762+0000 7fa9a52ddb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b29eb0 0x5572e4b253c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.762+0000 7fa9a52ddb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5572e4b4ac10 0x5572e4b3d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.762+0000 7fa9a52ddb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e4b57a00 con 0x5572e4b29eb0 2023-10-06T10:42:16.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.762+0000 7fa9a52ddb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e4af7bc0 con 0x5572e4b4ac10 2023-10-06T10:42:16.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.762+0000 7fa9a52ddb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5572e4b57570 con 0x5572e4b009a0 2023-10-06T10:42:16.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.762+0000 7fa9a52ddb80 10 client.0 _ll_register_callbacks cb 0x5572e48edf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:42:16.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.762+0000 7fa999580700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b29eb0 0x5572e4b253c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.762+0000 7fa999580700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b29eb0 0x5572e4b253c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53388/0 (socket says 172.21.15.92:53388) 2023-10-06T10:42:16.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.762+0000 7fa999580700 1 -- 172.21.15.92:0/4158440866 learned_addr learned my addr 172.21.15.92:0/4158440866 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:16.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.762+0000 7fa99a582700 1 --2- 172.21.15.92:0/4158440866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5572e4b4ac10 0x5572e4b3d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.762+0000 7fa999d81700 1 --2- 172.21.15.92:0/4158440866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5572e4b009a0 0x5572e4bad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.762+0000 7fa999580700 1 -- 172.21.15.92:0/4158440866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5572e4b009a0 msgr2=0x5572e4bad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.762+0000 7fa999580700 1 --2- 172.21.15.92:0/4158440866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5572e4b009a0 0x5572e4bad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.762+0000 7fa999580700 1 -- 172.21.15.92:0/4158440866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5572e4b4ac10 msgr2=0x5572e4b3d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:16.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.762+0000 7fa999580700 1 --2- 172.21.15.92:0/4158440866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5572e4b4ac10 0x5572e4b3d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.762+0000 7fa999580700 1 -- 172.21.15.92:0/4158440866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572e488e030 con 0x5572e4b29eb0 2023-10-06T10:42:16.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.763+0000 7fa999580700 1 --2- 172.21.15.92:0/4158440866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b29eb0 0x5572e4b253c0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fa980011000 tx=0x7fa980026250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06c292bf8587ef0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:16.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.763+0000 7fa96ffff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:42:16.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.763+0000 7fa96ffff700 1 -- 172.21.15.92:0/4158440866 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa980024070 con 0x5572e4b29eb0 2023-10-06T10:42:16.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.763+0000 7fa96ffff700 1 -- 172.21.15.92:0/4158440866 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fa98000ce60 con 0x5572e4b29eb0 2023-10-06T10:42:16.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.763+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/4158440866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa988019070 con 0x5572e4b29eb0 2023-10-06T10:42:16.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.763+0000 7fa96ffff700 1 -- 172.21.15.92:0/4158440866 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa980007a30 con 0x5572e4b29eb0 2023-10-06T10:42:16.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.763+0000 7fa9a52ddb80 10 client.6906 fetch_fsmap learned FSMap version 57 2023-10-06T10:42:16.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.763+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/4158440866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5572e4a064c0 con 0x5572e4b29eb0 2023-10-06T10:42:16.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.763+0000 7fa96ffff700 1 -- 172.21.15.92:0/4158440866 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa988019070 con 0x5572e4b29eb0 2023-10-06T10:42:16.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.763+0000 7fa96ffff700 1 -- 172.21.15.92:0/4158440866 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 57) v1 ==== 3136+0+0 (secure 0 0 0) 0x7fa980001370 con 0x5572e4b29eb0 2023-10-06T10:42:16.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.763+0000 7fa9a52ddb80 10 client.6906 fetch_fsmap finished waiting for FSMap version 57 2023-10-06T10:42:16.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.763+0000 7fa9a52ddb80 10 client.6906 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-10-06T10:42:16.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.763+0000 7fa9a52ddb80 20 client.6906 populate_metadata read hostname 'smithi092' 2023-10-06T10:42:16.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.763+0000 7fa9a52ddb80 1 --2- 172.21.15.92:0/4158440866 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x5572e4b23c80 0x5572e4b24080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:16.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.763+0000 7fa9a52ddb80 4 client.6906 mds_command: new command op to 6486 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:42:16.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.763+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/4158440866 --> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5572e49f3e50 con 0x5572e4b23c80 2023-10-06T10:42:16.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.763+0000 7fa999d81700 1 --2- 172.21.15.92:0/4158440866 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x5572e4b23c80 0x5572e4b24080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:16.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.763+0000 7fa96ffff700 10 client.6906 ms_handle_connect on v2:172.21.15.92:6835/818031030 2023-10-06T10:42:16.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.764+0000 7fa999d81700 1 --2- 172.21.15.92:0/4158440866 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x5572e4b23c80 0x5572e4b24080 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a9a4fab27612d9c server_cookie=c47f0daed0bd99fc in_seq=0 out_seq=0 2023-10-06T10:42:16.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.764+0000 7fa96ffff700 1 -- 172.21.15.92:0/4158440866 <== mds.0 v2:172.21.15.92:6835/818031030 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5572e49f3e50 con 0x5572e4b23c80 2023-10-06T10:42:16.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.764+0000 7fa96ffff700 10 client.6906 handle_command_reply: tid=0 2023-10-06T10:42:16.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.764+0000 7fa9a52ddb80 1 client.6906 shutdown 2023-10-06T10:42:16.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.764+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/4158440866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b29eb0 msgr2=0x5572e4b253c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.764+0000 7fa9a52ddb80 1 --2- 172.21.15.92:0/4158440866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b29eb0 0x5572e4b253c0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fa980011000 tx=0x7fa980026250 comp rx=0 tx=0).stop 2023-10-06T10:42:16.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.764+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/4158440866 shutdown_connections 2023-10-06T10:42:16.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.764+0000 7fa9a52ddb80 1 --2- 172.21.15.92:0/4158440866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5572e4b4ac10 0x5572e4b3d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.764+0000 7fa9a52ddb80 1 --2- 172.21.15.92:0/4158440866 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x5572e4b23c80 0x5572e4b24080 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.764+0000 7fa9a52ddb80 1 --2- 172.21.15.92:0/4158440866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5572e4b009a0 0x5572e4bad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.764+0000 7fa96ffff700 0 client.6906 ms_handle_reset on v2:172.21.15.92:6835/818031030 2023-10-06T10:42:16.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.764+0000 7fa9a52ddb80 1 --2- 172.21.15.92:0/4158440866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5572e4b29eb0 0x5572e4b253c0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.764+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/4158440866 >> 172.21.15.92:0/4158440866 conn(0x5572e4b022c0 msgr2=0x5572e4b45a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:16.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.764+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/4158440866 shutdown_connections 2023-10-06T10:42:16.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.764+0000 7fa9a52ddb80 1 -- 172.21.15.92:0/4158440866 wait complete. 2023-10-06T10:42:16.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.764+0000 7fa9a52ddb80 20 client.6906 trim_cache size 0 max 16384 2023-10-06T10:42:16.774 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:42:16.774 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T10:42:16.775 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:42:16.775 INFO:teuthology.orchestra.run.smithi092.stdout: "3d9baef9-cff7-482f-9129-a793337c06e6": { 2023-10-06T10:42:16.775 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:42:16.775 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "3d9baef9-cff7-482f-9129-a793337c06e6", 2023-10-06T10:42:16.775 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:42:16.775 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:16.775 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:16.775 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:42:16.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.765+0000 7fa99ad83700 1 -- 172.21.15.92:0/481626448 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa978066f30 msgr2=0x7fa9780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.765+0000 7fa99ad83700 1 --2- 172.21.15.92:0/481626448 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa978066f30 0x7fa9780693e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa9880042e0 tx=0x7fa988001aa0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.765+0000 7fa99ad83700 1 -- 172.21.15.92:0/481626448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9940c98a0 msgr2=0x7fa9940c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:16.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.765+0000 7fa99ad83700 1 --2- 172.21.15.92:0/481626448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9940c98a0 0x7fa9940c9d20 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fa9940b2e10 tx=0x7fa98c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.765+0000 7fa99ad83700 1 -- 172.21.15.92:0/481626448 shutdown_connections 2023-10-06T10:42:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.765+0000 7fa99ad83700 1 --2- 172.21.15.92:0/481626448 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa978066f30 0x7fa9780693e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.765+0000 7fa99ad83700 1 --2- 172.21.15.92:0/481626448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9940c98a0 0x7fa9940c9d20 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.765+0000 7fa99ad83700 1 --2- 172.21.15.92:0/481626448 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa984004060 0x7fa994159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.765+0000 7fa99ad83700 1 --2- 172.21.15.92:0/481626448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa99415bf70 0x7fa994150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.765+0000 7fa99ad83700 1 -- 172.21.15.92:0/481626448 >> 172.21.15.92:0/481626448 conn(0x7fa994099520 msgr2=0x7fa9940b56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.765+0000 7fa99ad83700 1 -- 172.21.15.92:0/481626448 shutdown_connections 2023-10-06T10:42:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:16.765+0000 7fa99ad83700 1 -- 172.21.15.92:0/481626448 wait complete. 2023-10-06T10:42:16.778 INFO:tasks.cephfs.filesystem:scrub status for tag:3d9baef9-cff7-482f-9129-a793337c06e6 - {'path': '/d1/d2/d3', 'tag': '3d9baef9-cff7-482f-9129-a793337c06e6', 'options': 'recursive,force'} 2023-10-06T10:42:18.323 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T10:42:18.322+0000 7f9e81bcf700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T10:42:23.138 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:42:23.141 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:42:28.318 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T10:42:28.317+0000 7f9e81bcf700 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T10:42:46.779 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-10-06T10:42:46.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.904+0000 7fc26eaa4700 1 Processor -- start 2023-10-06T10:42:46.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.904+0000 7fc26eaa4700 1 -- start start 2023-10-06T10:42:46.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.905+0000 7fc26eaa4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc268151980 0x7fc26806b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:46.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.905+0000 7fc26eaa4700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc26815b900 0x7fc268158b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:46.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.905+0000 7fc26eaa4700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc26806c250 con 0x7fc26806bde0 2023-10-06T10:42:46.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.905+0000 7fc26eaa4700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2680b0500 con 0x7fc26815b900 2023-10-06T10:42:46.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.905+0000 7fc26eaa4700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc26815bd20 con 0x7fc268151980 2023-10-06T10:42:46.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.905+0000 7fc26d041700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc26815b900 0x7fc268158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:46.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.905+0000 7fc26c840700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc268151980 0x7fc26806b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:46.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.906+0000 7fc267fff700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fc26806bde0 0x7fc26815b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:51068/0 (socket says 172.21.15.92:51068) 2023-10-06T10:42:46.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.906+0000 7fc26d041700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc26815b900 0x7fc268158b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:37766/0 (socket says 172.21.15.92:37766) 2023-10-06T10:42:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.906+0000 7fc267fff700 1 -- 172.21.15.92:0/81902532 learned_addr learned my addr 172.21.15.92:0/81902532 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:42:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.906+0000 7fc26c840700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc268151980 0x7fc26806b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:33182/0 (socket says 172.21.15.92:33182) 2023-10-06T10:42:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.906+0000 7fc2677fe700 1 -- 172.21.15.92:0/81902532 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4221235427 0 0) 0x7fc26806c250 con 0x7fc26806bde0 2023-10-06T10:42:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.906+0000 7fc2677fe700 1 -- 172.21.15.92:0/81902532 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc250003570 con 0x7fc26806bde0 2023-10-06T10:42:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.906+0000 7fc26c840700 1 -- 172.21.15.92:0/81902532 >> v2:172.21.15.124:3300/0 conn(0x7fc26815b900 msgr2=0x7fc268158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.906+0000 7fc26c840700 1 --2- 172.21.15.92:0/81902532 >> v2:172.21.15.124:3300/0 conn(0x7fc26815b900 0x7fc268158b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.906+0000 7fc26c840700 1 -- 172.21.15.92:0/81902532 >> v1:172.21.15.124:6789/0 conn(0x7fc26806bde0 legacy=0x7fc26815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.906+0000 7fc26c840700 1 -- 172.21.15.92:0/81902532 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2680afbd0 con 0x7fc268151980 2023-10-06T10:42:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.906+0000 7fc26d041700 1 --2- 172.21.15.92:0/81902532 >> v2:172.21.15.124:3300/0 conn(0x7fc26815b900 0x7fc268158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:42:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.906+0000 7fc26c840700 1 --2- 172.21.15.92:0/81902532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc268151980 0x7fc26806b8a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc258009e60 tx=0x7fc258009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0884061a0583922 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.906+0000 7fc2677fe700 1 -- 172.21.15.92:0/81902532 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25800e070 con 0x7fc268151980 2023-10-06T10:42:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.906+0000 7fc2677fe700 1 -- 172.21.15.92:0/81902532 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25800bab0 con 0x7fc268151980 2023-10-06T10:42:46.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.906+0000 7fc2677fe700 1 -- 172.21.15.92:0/81902532 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25800be70 con 0x7fc268151980 2023-10-06T10:42:46.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.907+0000 7fc26eaa4700 1 -- 172.21.15.92:0/81902532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc268151980 msgr2=0x7fc26806b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:46.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.907+0000 7fc26eaa4700 1 --2- 172.21.15.92:0/81902532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc268151980 0x7fc26806b8a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc258009e60 tx=0x7fc258009780 comp rx=0 tx=0).stop 2023-10-06T10:42:46.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.907+0000 7fc26eaa4700 1 -- 172.21.15.92:0/81902532 shutdown_connections 2023-10-06T10:42:46.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.907+0000 7fc26eaa4700 1 --2- 172.21.15.92:0/81902532 >> v2:172.21.15.124:3300/0 conn(0x7fc26815b900 0x7fc268158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.907+0000 7fc26eaa4700 1 --2- 172.21.15.92:0/81902532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc268151980 0x7fc26806b8a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.907+0000 7fc26eaa4700 1 -- 172.21.15.92:0/81902532 >> 172.21.15.92:0/81902532 conn(0x7fc2680ce2f0 msgr2=0x7fc2680ce6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:46.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.907+0000 7fc26eaa4700 1 -- 172.21.15.92:0/81902532 shutdown_connections 2023-10-06T10:42:46.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.907+0000 7fc26eaa4700 1 -- 172.21.15.92:0/81902532 wait complete. 2023-10-06T10:42:46.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.908+0000 7fc26eaa4700 1 Processor -- start 2023-10-06T10:42:46.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.908+0000 7fc26eaa4700 1 -- start start 2023-10-06T10:42:46.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.909+0000 7fc26eaa4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc26806bde0 0x7fc2681684a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:46.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.909+0000 7fc26eaa4700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc268151980 0x7fc2680c0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:46.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.909+0000 7fc26eaa4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc26815b900 0x7fc2680c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:46.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.909+0000 7fc26eaa4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2681689e0 con 0x7fc26815b900 2023-10-06T10:42:46.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.909+0000 7fc26eaa4700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2680ce080 con 0x7fc268151980 2023-10-06T10:42:46.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.909+0000 7fc26eaa4700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2680c8150 con 0x7fc26806bde0 2023-10-06T10:42:46.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.909+0000 7fc26d041700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc26815b900 0x7fc2680c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:46.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.909+0000 7fc26d041700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc26815b900 0x7fc2680c5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48386/0 (socket says 172.21.15.92:48386) 2023-10-06T10:42:46.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.909+0000 7fc26d041700 1 -- 172.21.15.92:0/3321764400 learned_addr learned my addr 172.21.15.92:0/3321764400 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:46.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.909+0000 7fc26c840700 1 --2- 172.21.15.92:0/3321764400 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc26806bde0 0x7fc2681684a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:46.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.909+0000 7fc267fff700 1 --2- 172.21.15.92:0/3321764400 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc268151980 0x7fc2680c0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:46.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.909+0000 7fc26d041700 1 -- 172.21.15.92:0/3321764400 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc26806bde0 msgr2=0x7fc2681684a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:46.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.909+0000 7fc26d041700 1 --2- 172.21.15.92:0/3321764400 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc26806bde0 0x7fc2681684a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.909+0000 7fc26d041700 1 -- 172.21.15.92:0/3321764400 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc268151980 msgr2=0x7fc2680c0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:46.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.909+0000 7fc26d041700 1 --2- 172.21.15.92:0/3321764400 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc268151980 0x7fc2680c0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.909+0000 7fc26d041700 1 -- 172.21.15.92:0/3321764400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc260009420 con 0x7fc26815b900 2023-10-06T10:42:46.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.909+0000 7fc26c840700 1 --2- 172.21.15.92:0/3321764400 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc26806bde0 0x7fc2681684a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:42:46.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.910+0000 7fc26d041700 1 --2- 172.21.15.92:0/3321764400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc26815b900 0x7fc2680c5d30 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc26000ea20 tx=0x7fc2600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:46.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.910+0000 7fc264ff9700 1 -- 172.21.15.92:0/3321764400 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26000ee60 con 0x7fc26815b900 2023-10-06T10:42:46.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.910+0000 7fc264ff9700 1 -- 172.21.15.92:0/3321764400 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26000c850 con 0x7fc26815b900 2023-10-06T10:42:46.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.910+0000 7fc26eaa4700 1 -- 172.21.15.92:0/3321764400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc258009a20 con 0x7fc26815b900 2023-10-06T10:42:46.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.910+0000 7fc264ff9700 1 -- 172.21.15.92:0/3321764400 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26000cc10 con 0x7fc26815b900 2023-10-06T10:42:46.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.910+0000 7fc26eaa4700 1 -- 172.21.15.92:0/3321764400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc268168c90 con 0x7fc26815b900 2023-10-06T10:42:46.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.911+0000 7fc264ff9700 1 -- 172.21.15.92:0/3321764400 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc260013470 con 0x7fc26815b900 2023-10-06T10:42:46.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.911+0000 7fc264ff9700 1 --2- 172.21.15.92:0/3321764400 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc254066f60 0x7fc254069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:46.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.912+0000 7fc264ff9700 1 -- 172.21.15.92:0/3321764400 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fc260084810 con 0x7fc26815b900 2023-10-06T10:42:46.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.912+0000 7fc26c840700 1 --2- 172.21.15.92:0/3321764400 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc254066f60 0x7fc254069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:46.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.912+0000 7fc26c840700 1 --2- 172.21.15.92:0/3321764400 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc254066f60 0x7fc254069410 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc2580097c0 tx=0x7fc258019000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:46.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.913+0000 7fc277d9cb80 1 Processor -- start 2023-10-06T10:42:46.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.913+0000 7fc277d9cb80 1 -- start start 2023-10-06T10:42:46.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.914+0000 7fc277d9cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401c9a1ae0 0x56401c9a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:46.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.914+0000 7fc277d9cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401c95d9f0 0x56401c95a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:46.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.914+0000 7fc277d9cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c95bc20 0x56401c95ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:46.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.914+0000 7fc277d9cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56401c8dacd0 con 0x56401c95bc20 2023-10-06T10:42:46.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.914+0000 7fc277d9cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56401c8b1ac0 con 0x56401c9a1ae0 2023-10-06T10:42:46.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.914+0000 7fc277d9cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56401c8af2c0 con 0x56401c95d9f0 2023-10-06T10:42:46.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.914+0000 7fc26d041700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c95bc20 0x56401c95ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:46.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.914+0000 7fc26d041700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c95bc20 0x56401c95ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48398/0 (socket says 172.21.15.92:48398) 2023-10-06T10:42:46.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.914+0000 7fc26d041700 1 -- 172.21.15.92:0/3134613813 learned_addr learned my addr 172.21.15.92:0/3134613813 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:46.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.914+0000 7fc26c840700 1 --2- 172.21.15.92:0/3134613813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401c9a1ae0 0x56401c9a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:46.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.914+0000 7fc267fff700 1 --2- 172.21.15.92:0/3134613813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401c95d9f0 0x56401c95a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:46.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.914+0000 7fc26d041700 1 -- 172.21.15.92:0/3134613813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401c95d9f0 msgr2=0x56401c95a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.914+0000 7fc26d041700 1 --2- 172.21.15.92:0/3134613813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401c95d9f0 0x56401c95a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.914+0000 7fc26d041700 1 -- 172.21.15.92:0/3134613813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401c9a1ae0 msgr2=0x56401c9a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.914+0000 7fc26d041700 1 --2- 172.21.15.92:0/3134613813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401c9a1ae0 0x56401c9a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.914+0000 7fc26d041700 1 -- 172.21.15.92:0/3134613813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56401c7452c0 con 0x56401c95bc20 2023-10-06T10:42:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.915+0000 7fc26d041700 1 --2- 172.21.15.92:0/3134613813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c95bc20 0x56401c95ab10 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fc260055bc0 tx=0x7fc260082910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917adb51ebc0bc79 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.915+0000 7fc24dffb700 1 -- 172.21.15.92:0/3134613813 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc260058610 con 0x56401c95bc20 2023-10-06T10:42:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.915+0000 7fc24dffb700 1 -- 172.21.15.92:0/3134613813 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26005c360 con 0x56401c95bc20 2023-10-06T10:42:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.915+0000 7fc24dffb700 1 -- 172.21.15.92:0/3134613813 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc260053760 con 0x56401c95bc20 2023-10-06T10:42:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.915+0000 7fc277d9cb80 1 -- 172.21.15.92:0/3134613813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c95bc20 msgr2=0x56401c95ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.915+0000 7fc277d9cb80 1 --2- 172.21.15.92:0/3134613813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c95bc20 0x56401c95ab10 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fc260055bc0 tx=0x7fc260082910 comp rx=0 tx=0).stop 2023-10-06T10:42:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.915+0000 7fc277d9cb80 1 -- 172.21.15.92:0/3134613813 shutdown_connections 2023-10-06T10:42:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.915+0000 7fc277d9cb80 1 --2- 172.21.15.92:0/3134613813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c95bc20 0x56401c95ab10 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.915+0000 7fc277d9cb80 1 --2- 172.21.15.92:0/3134613813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401c9a1ae0 0x56401c9a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.915+0000 7fc277d9cb80 1 --2- 172.21.15.92:0/3134613813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401c95d9f0 0x56401c95a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.915+0000 7fc277d9cb80 1 -- 172.21.15.92:0/3134613813 >> 172.21.15.92:0/3134613813 conn(0x56401c95c0c0 msgr2=0x56401c95c490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.915+0000 7fc277d9cb80 1 -- 172.21.15.92:0/3134613813 shutdown_connections 2023-10-06T10:42:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.915+0000 7fc277d9cb80 1 -- 172.21.15.92:0/3134613813 wait complete. 2023-10-06T10:42:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.916+0000 7fc277d9cb80 1 -- start start 2023-10-06T10:42:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.916+0000 7fc277d9cb80 1 Processor -- start 2023-10-06T10:42:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.917+0000 7fc277d9cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401c95bc20 0x56401c97dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.917+0000 7fc277d9cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401c95d9f0 0x56401c99d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.917+0000 7fc277d9cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c9a1ae0 0x56401c95a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.917+0000 7fc277d9cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56401c8ab0b0 con 0x56401c9a1ae0 2023-10-06T10:42:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.917+0000 7fc277d9cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56401c8568e0 con 0x56401c95bc20 2023-10-06T10:42:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.917+0000 7fc277d9cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56401c871ef0 con 0x56401c95d9f0 2023-10-06T10:42:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.917+0000 7fc277d9cb80 10 client.0 _ll_register_callbacks cb 0x56401c747f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:42:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.917+0000 7fc26d041700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c9a1ae0 0x56401c95a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.917+0000 7fc26d041700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c9a1ae0 0x56401c95a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48410/0 (socket says 172.21.15.92:48410) 2023-10-06T10:42:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.917+0000 7fc26d041700 1 -- 172.21.15.92:0/1443276813 learned_addr learned my addr 172.21.15.92:0/1443276813 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.917+0000 7fc26c840700 1 --2- 172.21.15.92:0/1443276813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401c95bc20 0x56401c97dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.917+0000 7fc267fff700 1 --2- 172.21.15.92:0/1443276813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401c95d9f0 0x56401c99d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.917+0000 7fc26d041700 1 -- 172.21.15.92:0/1443276813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401c95d9f0 msgr2=0x56401c99d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.917+0000 7fc26d041700 1 --2- 172.21.15.92:0/1443276813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401c95d9f0 0x56401c99d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.917+0000 7fc26d041700 1 -- 172.21.15.92:0/1443276813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401c95bc20 msgr2=0x56401c97dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.917+0000 7fc26d041700 1 --2- 172.21.15.92:0/1443276813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401c95bc20 0x56401c97dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.917+0000 7fc26d041700 1 -- 172.21.15.92:0/1443276813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc268168c90 con 0x56401c9a1ae0 2023-10-06T10:42:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.917+0000 7fc26d041700 1 --2- 172.21.15.92:0/1443276813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c9a1ae0 0x56401c95a5d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fc260055580 tx=0x7fc26006b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49dd42ad934d06c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.917+0000 7fc246ffd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:42:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.918+0000 7fc246ffd700 1 -- 172.21.15.92:0/1443276813 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc260053760 con 0x56401c9a1ae0 2023-10-06T10:42:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.918+0000 7fc246ffd700 1 -- 172.21.15.92:0/1443276813 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fc26007f070 con 0x56401c9a1ae0 2023-10-06T10:42:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.918+0000 7fc277d9cb80 1 -- 172.21.15.92:0/1443276813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56401c94f6e0 con 0x56401c9a1ae0 2023-10-06T10:42:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.918+0000 7fc246ffd700 1 -- 172.21.15.92:0/1443276813 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc26005c0e0 con 0x56401c9a1ae0 2023-10-06T10:42:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.918+0000 7fc277d9cb80 10 client.6915 fetch_fsmap learned FSMap version 59 2023-10-06T10:42:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.918+0000 7fc277d9cb80 1 -- 172.21.15.92:0/1443276813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x56401c6e8030 con 0x56401c9a1ae0 2023-10-06T10:42:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.918+0000 7fc246ffd700 1 -- 172.21.15.92:0/1443276813 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56401c94f6e0 con 0x56401c9a1ae0 2023-10-06T10:42:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.918+0000 7fc246ffd700 1 -- 172.21.15.92:0/1443276813 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 59) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fc26005a3f0 con 0x56401c9a1ae0 2023-10-06T10:42:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.918+0000 7fc277d9cb80 10 client.6915 fetch_fsmap finished waiting for FSMap version 59 2023-10-06T10:42:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.918+0000 7fc277d9cb80 10 client.6915 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-10-06T10:42:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.918+0000 7fc277d9cb80 20 client.6915 populate_metadata read hostname 'smithi092' 2023-10-06T10:42:46.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.918+0000 7fc277d9cb80 1 --2- 172.21.15.92:0/1443276813 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x56401c9b18f0 0x56401c99d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:46.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.918+0000 7fc277d9cb80 4 client.6915 mds_command: new command op to 6486 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:42:46.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.918+0000 7fc277d9cb80 1 -- 172.21.15.92:0/1443276813 --> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56401c92c610 con 0x56401c9b18f0 2023-10-06T10:42:46.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.918+0000 7fc26c840700 1 --2- 172.21.15.92:0/1443276813 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x56401c9b18f0 0x56401c99d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:46.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.918+0000 7fc246ffd700 10 client.6915 ms_handle_connect on v2:172.21.15.92:6835/818031030 2023-10-06T10:42:46.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.918+0000 7fc26c840700 1 --2- 172.21.15.92:0/1443276813 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x56401c9b18f0 0x56401c99d940 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70d9b53906ac497a server_cookie=1cb0f86bda47724d in_seq=0 out_seq=0 2023-10-06T10:42:46.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.919+0000 7fc246ffd700 1 -- 172.21.15.92:0/1443276813 <== mds.0 v2:172.21.15.92:6835/818031030 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x56401c92c610 con 0x56401c9b18f0 2023-10-06T10:42:46.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.919+0000 7fc246ffd700 10 client.6915 handle_command_reply: tid=0 2023-10-06T10:42:46.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.919+0000 7fc277d9cb80 1 client.6915 shutdown 2023-10-06T10:42:46.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.919+0000 7fc277d9cb80 1 -- 172.21.15.92:0/1443276813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c9a1ae0 msgr2=0x56401c95a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:46.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.919+0000 7fc277d9cb80 1 --2- 172.21.15.92:0/1443276813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c9a1ae0 0x56401c95a5d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fc260055580 tx=0x7fc26006b000 comp rx=0 tx=0).stop 2023-10-06T10:42:46.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.919+0000 7fc277d9cb80 1 -- 172.21.15.92:0/1443276813 shutdown_connections 2023-10-06T10:42:46.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.919+0000 7fc277d9cb80 1 --2- 172.21.15.92:0/1443276813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c9a1ae0 0x56401c95a5d0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.919+0000 7fc277d9cb80 1 --2- 172.21.15.92:0/1443276813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401c95bc20 0x56401c97dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.919+0000 7fc277d9cb80 1 --2- 172.21.15.92:0/1443276813 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x56401c9b18f0 0x56401c99d940 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.919+0000 7fc277d9cb80 1 --2- 172.21.15.92:0/1443276813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401c95d9f0 0x56401c99d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.919+0000 7fc246ffd700 0 client.6915 ms_handle_reset on v2:172.21.15.92:6835/818031030 2023-10-06T10:42:46.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.919+0000 7fc277d9cb80 1 -- 172.21.15.92:0/1443276813 >> 172.21.15.92:0/1443276813 conn(0x56401c95c0c0 msgr2=0x56401c99f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:46.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.919+0000 7fc277d9cb80 1 -- 172.21.15.92:0/1443276813 shutdown_connections 2023-10-06T10:42:46.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.919+0000 7fc277d9cb80 1 -- 172.21.15.92:0/1443276813 wait complete. 2023-10-06T10:42:46.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.919+0000 7fc277d9cb80 20 client.6915 trim_cache size 0 max 16384 2023-10-06T10:42:46.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.935+0000 7fc277d9cb80 1 Processor -- start 2023-10-06T10:42:46.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.935+0000 7fc277d9cb80 1 -- start start 2023-10-06T10:42:46.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.936+0000 7fc277d9cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401c95bc20 0x56401c95c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:46.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.936+0000 7fc277d9cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401c95d9f0 0x56401c95ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:46.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.936+0000 7fc277d9cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c9a1ae0 0x56401c957b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:46.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.936+0000 7fc277d9cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56401c72de80 con 0x56401c9a1ae0 2023-10-06T10:42:46.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.936+0000 7fc277d9cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56401c95bad0 con 0x56401c95bc20 2023-10-06T10:42:46.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.936+0000 7fc277d9cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56401c99d120 con 0x56401c95d9f0 2023-10-06T10:42:46.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.936+0000 7fc26d041700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c9a1ae0 0x56401c957b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:46.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.936+0000 7fc26d041700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c9a1ae0 0x56401c957b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48424/0 (socket says 172.21.15.92:48424) 2023-10-06T10:42:46.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.936+0000 7fc26d041700 1 -- 172.21.15.92:0/3704433367 learned_addr learned my addr 172.21.15.92:0/3704433367 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:46.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.936+0000 7fc26d041700 1 -- 172.21.15.92:0/3704433367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401c95d9f0 msgr2=0x56401c95ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:46.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.936+0000 7fc26d041700 1 --2- 172.21.15.92:0/3704433367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401c95d9f0 0x56401c95ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.936+0000 7fc26d041700 1 -- 172.21.15.92:0/3704433367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401c95bc20 msgr2=0x56401c95c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:46.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.936+0000 7fc26d041700 1 --2- 172.21.15.92:0/3704433367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401c95bc20 0x56401c95c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.936+0000 7fc26d041700 1 -- 172.21.15.92:0/3704433367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56401c8797d0 con 0x56401c9a1ae0 2023-10-06T10:42:46.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.936+0000 7fc26d041700 1 --2- 172.21.15.92:0/3704433367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c9a1ae0 0x56401c957b20 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fc2600539b0 tx=0x7fc26007e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6401bee34d310571 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:46.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.936+0000 7fc24e7fc700 1 -- 172.21.15.92:0/3704433367 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc260053760 con 0x56401c9a1ae0 2023-10-06T10:42:46.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.936+0000 7fc24e7fc700 1 -- 172.21.15.92:0/3704433367 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc260071050 con 0x56401c9a1ae0 2023-10-06T10:42:46.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.936+0000 7fc24e7fc700 1 -- 172.21.15.92:0/3704433367 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc260058610 con 0x56401c9a1ae0 2023-10-06T10:42:46.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.937+0000 7fc277d9cb80 1 -- 172.21.15.92:0/3704433367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c9a1ae0 msgr2=0x56401c957b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:46.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.937+0000 7fc277d9cb80 1 --2- 172.21.15.92:0/3704433367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c9a1ae0 0x56401c957b20 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fc2600539b0 tx=0x7fc26007e000 comp rx=0 tx=0).stop 2023-10-06T10:42:46.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.937+0000 7fc277d9cb80 1 -- 172.21.15.92:0/3704433367 shutdown_connections 2023-10-06T10:42:46.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.937+0000 7fc277d9cb80 1 --2- 172.21.15.92:0/3704433367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c9a1ae0 0x56401c957b20 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.937+0000 7fc277d9cb80 1 --2- 172.21.15.92:0/3704433367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401c95bc20 0x56401c95c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.937+0000 7fc277d9cb80 1 --2- 172.21.15.92:0/3704433367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401c95d9f0 0x56401c95ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.937+0000 7fc277d9cb80 1 -- 172.21.15.92:0/3704433367 >> 172.21.15.92:0/3704433367 conn(0x56401c95c0c0 msgr2=0x56401c9815d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:46.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.937+0000 7fc277d9cb80 1 -- 172.21.15.92:0/3704433367 shutdown_connections 2023-10-06T10:42:46.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.937+0000 7fc277d9cb80 1 -- 172.21.15.92:0/3704433367 wait complete. 2023-10-06T10:42:46.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.937+0000 7fc277d9cb80 1 -- start start 2023-10-06T10:42:46.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.938+0000 7fc277d9cb80 1 Processor -- start 2023-10-06T10:42:46.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.938+0000 7fc277d9cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401c95bc20 0x56401ca00840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:46.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.938+0000 7fc277d9cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c95d9f0 0x56401ca06330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:46.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.938+0000 7fc277d9cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401c9a1ae0 0x56401ca06870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:46.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.938+0000 7fc277d9cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56401c981d90 con 0x56401c95d9f0 2023-10-06T10:42:46.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.938+0000 7fc277d9cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56401ca07360 con 0x56401c95bc20 2023-10-06T10:42:46.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.938+0000 7fc277d9cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56401c981f00 con 0x56401c9a1ae0 2023-10-06T10:42:46.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.938+0000 7fc277d9cb80 10 client.0 _ll_register_callbacks cb 0x56401c747f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:42:46.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.938+0000 7fc267fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c95d9f0 0x56401ca06330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:46.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.938+0000 7fc267fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c95d9f0 0x56401ca06330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48438/0 (socket says 172.21.15.92:48438) 2023-10-06T10:42:46.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.939+0000 7fc267fff700 1 -- 172.21.15.92:0/3339875674 learned_addr learned my addr 172.21.15.92:0/3339875674 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:46.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.939+0000 7fc267fff700 1 -- 172.21.15.92:0/3339875674 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401c9a1ae0 msgr2=0x56401ca06870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:46.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.939+0000 7fc267fff700 1 --2- 172.21.15.92:0/3339875674 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401c9a1ae0 0x56401ca06870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.939+0000 7fc267fff700 1 -- 172.21.15.92:0/3339875674 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401c95bc20 msgr2=0x56401ca00840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:46.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.939+0000 7fc267fff700 1 --2- 172.21.15.92:0/3339875674 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401c95bc20 0x56401ca00840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.939+0000 7fc267fff700 1 -- 172.21.15.92:0/3339875674 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56401c6e8030 con 0x56401c95d9f0 2023-10-06T10:42:46.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.939+0000 7fc267fff700 1 --2- 172.21.15.92:0/3339875674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c95d9f0 0x56401ca06330 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc25c007fd0 tx=0x7fc25c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678c0ad298068ace server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:46.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.939+0000 7fc246ffd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:42:46.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.939+0000 7fc246ffd700 1 -- 172.21.15.92:0/3339875674 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c005b10 con 0x56401c95d9f0 2023-10-06T10:42:46.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.939+0000 7fc246ffd700 1 -- 172.21.15.92:0/3339875674 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fc25c005c70 con 0x56401c95d9f0 2023-10-06T10:42:46.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.939+0000 7fc277d9cb80 1 -- 172.21.15.92:0/3339875674 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc26005c360 con 0x56401c95d9f0 2023-10-06T10:42:46.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.939+0000 7fc246ffd700 1 -- 172.21.15.92:0/3339875674 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc25c001090 con 0x56401c95d9f0 2023-10-06T10:42:46.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.939+0000 7fc277d9cb80 10 client.6921 fetch_fsmap learned FSMap version 59 2023-10-06T10:42:46.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.939+0000 7fc277d9cb80 1 -- 172.21.15.92:0/3339875674 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x56401c8604c0 con 0x56401c95d9f0 2023-10-06T10:42:46.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.939+0000 7fc246ffd700 1 -- 172.21.15.92:0/3339875674 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26005c360 con 0x56401c95d9f0 2023-10-06T10:42:46.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.939+0000 7fc246ffd700 1 -- 172.21.15.92:0/3339875674 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 59) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fc25c012050 con 0x56401c95d9f0 2023-10-06T10:42:46.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.939+0000 7fc277d9cb80 10 client.6921 fetch_fsmap finished waiting for FSMap version 59 2023-10-06T10:42:46.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.939+0000 7fc277d9cb80 10 client.6921 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-10-06T10:42:46.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.939+0000 7fc277d9cb80 20 client.6921 populate_metadata read hostname 'smithi092' 2023-10-06T10:42:46.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.939+0000 7fc277d9cb80 1 --2- 172.21.15.92:0/3339875674 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x56401c957e50 0x56401c989aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:46.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.939+0000 7fc277d9cb80 4 client.6921 mds_command: new command op to 6486 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:42:46.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.939+0000 7fc277d9cb80 1 -- 172.21.15.92:0/3339875674 --> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56401c84de50 con 0x56401c957e50 2023-10-06T10:42:46.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.940+0000 7fc26c840700 1 --2- 172.21.15.92:0/3339875674 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x56401c957e50 0x56401c989aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:46.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.940+0000 7fc246ffd700 10 client.6921 ms_handle_connect on v2:172.21.15.92:6835/818031030 2023-10-06T10:42:46.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.940+0000 7fc26c840700 1 --2- 172.21.15.92:0/3339875674 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x56401c957e50 0x56401c989aa0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=611c50fe8fa350ec server_cookie=5428e008be76ba87 in_seq=0 out_seq=0 2023-10-06T10:42:46.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.940+0000 7fc246ffd700 1 -- 172.21.15.92:0/3339875674 <== mds.0 v2:172.21.15.92:6835/818031030 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56401c84de50 con 0x56401c957e50 2023-10-06T10:42:46.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.940+0000 7fc246ffd700 10 client.6921 handle_command_reply: tid=0 2023-10-06T10:42:46.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.940+0000 7fc277d9cb80 1 client.6921 shutdown 2023-10-06T10:42:46.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.940+0000 7fc277d9cb80 1 -- 172.21.15.92:0/3339875674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c95d9f0 msgr2=0x56401ca06330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:46.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.940+0000 7fc277d9cb80 1 --2- 172.21.15.92:0/3339875674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c95d9f0 0x56401ca06330 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc25c007fd0 tx=0x7fc25c009040 comp rx=0 tx=0).stop 2023-10-06T10:42:46.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.940+0000 7fc277d9cb80 1 -- 172.21.15.92:0/3339875674 shutdown_connections 2023-10-06T10:42:46.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.940+0000 7fc277d9cb80 1 --2- 172.21.15.92:0/3339875674 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x56401c957e50 0x56401c989aa0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.940+0000 7fc277d9cb80 1 --2- 172.21.15.92:0/3339875674 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56401c9a1ae0 0x56401ca06870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.940+0000 7fc246ffd700 0 client.6921 ms_handle_reset on v2:172.21.15.92:6835/818031030 2023-10-06T10:42:46.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.940+0000 7fc277d9cb80 1 --2- 172.21.15.92:0/3339875674 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56401c95bc20 0x56401ca00840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.940+0000 7fc277d9cb80 1 --2- 172.21.15.92:0/3339875674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56401c95d9f0 0x56401ca06330 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.940+0000 7fc277d9cb80 1 -- 172.21.15.92:0/3339875674 >> 172.21.15.92:0/3339875674 conn(0x56401c95c0c0 msgr2=0x56401c95c490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:46.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.940+0000 7fc277d9cb80 1 -- 172.21.15.92:0/3339875674 shutdown_connections 2023-10-06T10:42:46.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.940+0000 7fc277d9cb80 1 -- 172.21.15.92:0/3339875674 wait complete. 2023-10-06T10:42:46.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.940+0000 7fc277d9cb80 20 client.6921 trim_cache size 0 max 16384 2023-10-06T10:42:46.948 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:42:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "no active scrubs running", 2023-10-06T10:42:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": {} 2023-10-06T10:42:46.948 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:42:46.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.941+0000 7fc26eaa4700 1 -- 172.21.15.92:0/3321764400 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc254066f60 msgr2=0x7fc254069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:46.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.941+0000 7fc26eaa4700 1 --2- 172.21.15.92:0/3321764400 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc254066f60 0x7fc254069410 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc2580097c0 tx=0x7fc258019000 comp rx=0 tx=0).stop 2023-10-06T10:42:46.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.941+0000 7fc26eaa4700 1 -- 172.21.15.92:0/3321764400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc26815b900 msgr2=0x7fc2680c5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:46.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.941+0000 7fc26eaa4700 1 --2- 172.21.15.92:0/3321764400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc26815b900 0x7fc2680c5d30 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc26000ea20 tx=0x7fc2600091a0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.941+0000 7fc26eaa4700 1 -- 172.21.15.92:0/3321764400 shutdown_connections 2023-10-06T10:42:46.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.941+0000 7fc26eaa4700 1 --2- 172.21.15.92:0/3321764400 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc254066f60 0x7fc254069410 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.941+0000 7fc26eaa4700 1 --2- 172.21.15.92:0/3321764400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc26815b900 0x7fc2680c5d30 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.941+0000 7fc26eaa4700 1 --2- 172.21.15.92:0/3321764400 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc26806bde0 0x7fc2681684a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.941+0000 7fc26eaa4700 1 --2- 172.21.15.92:0/3321764400 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc268151980 0x7fc2680c0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:46.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.941+0000 7fc26eaa4700 1 -- 172.21.15.92:0/3321764400 >> 172.21.15.92:0/3321764400 conn(0x7fc2680ce2f0 msgr2=0x7fc26814d300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:46.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.941+0000 7fc26eaa4700 1 -- 172.21.15.92:0/3321764400 shutdown_connections 2023-10-06T10:42:46.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:46.941+0000 7fc26eaa4700 1 -- 172.21.15.92:0/3321764400 wait complete. 2023-10-06T10:42:46.953 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:42:46.953 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 damage ls 2023-10-06T10:42:47.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.067+0000 7f365429d700 1 Processor -- start 2023-10-06T10:42:47.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.067+0000 7f365429d700 1 -- start start 2023-10-06T10:42:47.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.068+0000 7f365429d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f364c150940 0x7f364c159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.068+0000 7f365429d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f364c0cd4d0 0x7f364c0b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.068+0000 7f365429d700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f364c15ac30 con 0x7f364c15a160 2023-10-06T10:42:47.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.068+0000 7f365429d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f364c0cb6a0 con 0x7f364c150940 2023-10-06T10:42:47.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.068+0000 7f365429d700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f364c0cb090 con 0x7f364c0cd4d0 2023-10-06T10:42:47.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.069+0000 7f3652039700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f364c150940 0x7f364c159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.069+0000 7f3652039700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f364c150940 0x7f364c159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48446/0 (socket says 172.21.15.92:48446) 2023-10-06T10:42:47.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.069+0000 7f3652039700 1 -- 172.21.15.92:0/3406142860 learned_addr learned my addr 172.21.15.92:0/3406142860 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.069+0000 7f365283a700 1 --2- 172.21.15.92:0/3406142860 >> v2:172.21.15.124:3300/0 conn(0x7f364c0cd4d0 0x7f364c0b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.069+0000 7f3652039700 1 -- 172.21.15.92:0/3406142860 >> v2:172.21.15.124:3300/0 conn(0x7f364c0cd4d0 msgr2=0x7f364c0b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.069+0000 7f3652039700 1 --2- 172.21.15.92:0/3406142860 >> v2:172.21.15.124:3300/0 conn(0x7f364c0cd4d0 0x7f364c0b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.069+0000 7f3652039700 1 -- 172.21.15.92:0/3406142860 >> v1:172.21.15.124:6789/0 conn(0x7f364c15a160 legacy=0x7f364c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.069+0000 7f3652039700 1 -- 172.21.15.92:0/3406142860 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c15a670 con 0x7f364c150940 2023-10-06T10:42:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.069+0000 7f365283a700 1 --2- 172.21.15.92:0/3406142860 >> v2:172.21.15.124:3300/0 conn(0x7f364c0cd4d0 0x7f364c0b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.069+0000 7f3652039700 1 --2- 172.21.15.92:0/3406142860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364c150940 0x7f364c159c20 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f364c0d0460 tx=0x7f363c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49de01eb752962f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.069+0000 7f3651037700 1 -- 172.21.15.92:0/3406142860 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c005c10 con 0x7f364c150940 2023-10-06T10:42:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.069+0000 7f3651037700 1 -- 172.21.15.92:0/3406142860 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363c005d70 con 0x7f364c150940 2023-10-06T10:42:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.069+0000 7f3651037700 1 -- 172.21.15.92:0/3406142860 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c00b880 con 0x7f364c150940 2023-10-06T10:42:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.070+0000 7f365429d700 1 -- 172.21.15.92:0/3406142860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364c150940 msgr2=0x7f364c159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.070+0000 7f365429d700 1 --2- 172.21.15.92:0/3406142860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364c150940 0x7f364c159c20 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f364c0d0460 tx=0x7f363c009220 comp rx=0 tx=0).stop 2023-10-06T10:42:47.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.070+0000 7f365429d700 1 -- 172.21.15.92:0/3406142860 shutdown_connections 2023-10-06T10:42:47.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.070+0000 7f365429d700 1 --2- 172.21.15.92:0/3406142860 >> v2:172.21.15.124:3300/0 conn(0x7f364c0cd4d0 0x7f364c0b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.070+0000 7f365429d700 1 --2- 172.21.15.92:0/3406142860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364c150940 0x7f364c159c20 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.070+0000 7f365429d700 1 -- 172.21.15.92:0/3406142860 >> 172.21.15.92:0/3406142860 conn(0x7f364c099700 msgr2=0x7f364c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:47.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.070+0000 7f365429d700 1 -- 172.21.15.92:0/3406142860 shutdown_connections 2023-10-06T10:42:47.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.070+0000 7f365429d700 1 -- 172.21.15.92:0/3406142860 wait complete. 2023-10-06T10:42:47.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.071+0000 7f365429d700 1 Processor -- start 2023-10-06T10:42:47.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.071+0000 7f365429d700 1 -- start start 2023-10-06T10:42:47.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.072+0000 7f365429d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364c0cd4d0 0x7f364c155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.072+0000 7f365429d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f364c15a160 0x7f364c16a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.072+0000 7f365429d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f364c156020 0x7f364c152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.072+0000 7f365429d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c16e060 con 0x7f364c0cd4d0 2023-10-06T10:42:47.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.072+0000 7f365429d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c16d450 con 0x7f364c15a160 2023-10-06T10:42:47.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.072+0000 7f365429d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364c16d730 con 0x7f364c156020 2023-10-06T10:42:47.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.072+0000 7f3652039700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364c0cd4d0 0x7f364c155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.072+0000 7f3652039700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364c0cd4d0 0x7f364c155ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48454/0 (socket says 172.21.15.92:48454) 2023-10-06T10:42:47.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.072+0000 7f3652039700 1 -- 172.21.15.92:0/1897032726 learned_addr learned my addr 172.21.15.92:0/1897032726 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:47.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.072+0000 7f3652039700 1 -- 172.21.15.92:0/1897032726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f364c156020 msgr2=0x7f364c152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.072+0000 7f3652039700 1 --2- 172.21.15.92:0/1897032726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f364c156020 0x7f364c152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.072+0000 7f3652039700 1 -- 172.21.15.92:0/1897032726 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f364c15a160 msgr2=0x7f364c16a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.072+0000 7f3652039700 1 --2- 172.21.15.92:0/1897032726 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f364c15a160 0x7f364c16a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.072+0000 7f3652039700 1 -- 172.21.15.92:0/1897032726 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c009480 con 0x7f364c0cd4d0 2023-10-06T10:42:47.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.072+0000 7f3652039700 1 --2- 172.21.15.92:0/1897032726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364c0cd4d0 0x7f364c155ae0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f364c0d14c0 tx=0x7f363c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.072+0000 7f364a7fc700 1 -- 172.21.15.92:0/1897032726 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c0118e0 con 0x7f364c0cd4d0 2023-10-06T10:42:47.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.072+0000 7f364a7fc700 1 -- 172.21.15.92:0/1897032726 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363c020070 con 0x7f364c0cd4d0 2023-10-06T10:42:47.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.072+0000 7f365429d700 1 -- 172.21.15.92:0/1897032726 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364c16da10 con 0x7f364c0cd4d0 2023-10-06T10:42:47.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.072+0000 7f364a7fc700 1 -- 172.21.15.92:0/1897032726 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c011cb0 con 0x7f364c0cd4d0 2023-10-06T10:42:47.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.072+0000 7f365429d700 1 -- 172.21.15.92:0/1897032726 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364c16a930 con 0x7f364c0cd4d0 2023-10-06T10:42:47.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.074+0000 7f364a7fc700 1 -- 172.21.15.92:0/1897032726 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f363c022030 con 0x7f364c0cd4d0 2023-10-06T10:42:47.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.074+0000 7f364a7fc700 1 --2- 172.21.15.92:0/1897032726 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3638066f30 0x7f36380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.074+0000 7f364a7fc700 1 -- 172.21.15.92:0/1897032726 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f363c0864b0 con 0x7f364c0cd4d0 2023-10-06T10:42:47.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.074+0000 7f3651838700 1 --2- 172.21.15.92:0/1897032726 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3638066f30 0x7f36380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.074+0000 7f3651838700 1 --2- 172.21.15.92:0/1897032726 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3638066f30 0x7f36380693e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f364c1559c0 tx=0x7f3640007b40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.076+0000 7f365d595b80 1 Processor -- start 2023-10-06T10:42:47.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.076+0000 7f365d595b80 1 -- start start 2023-10-06T10:42:47.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.077+0000 7f365d595b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bf5af0bae0 0x55bf5af0beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.077+0000 7f365d595b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5aec79f0 0x55bf5aec45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.077+0000 7f365d595b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bf5aec5c20 0x55bf5aec4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.077+0000 7f365d595b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf5ae44cd0 con 0x55bf5aec79f0 2023-10-06T10:42:47.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.077+0000 7f365d595b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf5ae1bac0 con 0x55bf5aec5c20 2023-10-06T10:42:47.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.077+0000 7f365d595b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf5ae192c0 con 0x55bf5af0bae0 2023-10-06T10:42:47.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.077+0000 7f3651838700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5aec79f0 0x55bf5aec45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.077+0000 7f3651838700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5aec79f0 0x55bf5aec45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48462/0 (socket says 172.21.15.92:48462) 2023-10-06T10:42:47.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.077+0000 7f3651838700 1 -- 172.21.15.92:0/199757012 learned_addr learned my addr 172.21.15.92:0/199757012 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:47.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.077+0000 7f365283a700 1 --2- 172.21.15.92:0/199757012 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bf5aec5c20 0x55bf5aec4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.077+0000 7f3651838700 1 -- 172.21.15.92:0/199757012 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bf5af0bae0 msgr2=0x55bf5af0beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.077+0000 7f3651838700 1 --2- 172.21.15.92:0/199757012 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bf5af0bae0 0x55bf5af0beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.077+0000 7f3651838700 1 -- 172.21.15.92:0/199757012 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bf5aec5c20 msgr2=0x55bf5aec4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.077+0000 7f3651838700 1 --2- 172.21.15.92:0/199757012 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bf5aec5c20 0x55bf5aec4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.077+0000 7f3651838700 1 -- 172.21.15.92:0/199757012 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf5acaf2c0 con 0x55bf5aec79f0 2023-10-06T10:42:47.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.078+0000 7f3651838700 1 --2- 172.21.15.92:0/199757012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5aec79f0 0x55bf5aec45d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f36400188d0 tx=0x7f3640010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12dae011f3ee5032 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.078+0000 7f36337fe700 1 -- 172.21.15.92:0/199757012 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640018330 con 0x55bf5aec79f0 2023-10-06T10:42:47.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.078+0000 7f36337fe700 1 -- 172.21.15.92:0/199757012 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3640016880 con 0x55bf5aec79f0 2023-10-06T10:42:47.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.078+0000 7f36337fe700 1 -- 172.21.15.92:0/199757012 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640016c40 con 0x55bf5aec79f0 2023-10-06T10:42:47.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.078+0000 7f365d595b80 1 -- 172.21.15.92:0/199757012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5aec79f0 msgr2=0x55bf5aec45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.078+0000 7f365d595b80 1 --2- 172.21.15.92:0/199757012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5aec79f0 0x55bf5aec45d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f36400188d0 tx=0x7f3640010000 comp rx=0 tx=0).stop 2023-10-06T10:42:47.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.078+0000 7f365d595b80 1 -- 172.21.15.92:0/199757012 shutdown_connections 2023-10-06T10:42:47.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.078+0000 7f365d595b80 1 --2- 172.21.15.92:0/199757012 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bf5aec5c20 0x55bf5aec4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.078+0000 7f365d595b80 1 --2- 172.21.15.92:0/199757012 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bf5af0bae0 0x55bf5af0beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.078+0000 7f365d595b80 1 --2- 172.21.15.92:0/199757012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5aec79f0 0x55bf5aec45d0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.078+0000 7f365d595b80 1 -- 172.21.15.92:0/199757012 >> 172.21.15.92:0/199757012 conn(0x55bf5aec60c0 msgr2=0x55bf5aec6490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:47.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.078+0000 7f365d595b80 1 -- 172.21.15.92:0/199757012 shutdown_connections 2023-10-06T10:42:47.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.078+0000 7f365d595b80 1 -- 172.21.15.92:0/199757012 wait complete. 2023-10-06T10:42:47.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.079+0000 7f365d595b80 1 -- start start 2023-10-06T10:42:47.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.079+0000 7f365d595b80 1 Processor -- start 2023-10-06T10:42:47.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.080+0000 7f365d595b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5aec5c20 0x55bf5aeeeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.080+0000 7f365d595b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bf5aec79f0 0x55bf5aeeb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.080+0000 7f365d595b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bf5af0bae0 0x55bf5af02a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.080+0000 7f365d595b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf5ae150b0 con 0x55bf5aec5c20 2023-10-06T10:42:47.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.080+0000 7f365d595b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf5adc08e0 con 0x55bf5af0bae0 2023-10-06T10:42:47.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.080+0000 7f365d595b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf5addbef0 con 0x55bf5aec79f0 2023-10-06T10:42:47.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.080+0000 7f365d595b80 10 client.0 _ll_register_callbacks cb 0x55bf5acb1f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:42:47.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.080+0000 7f3652039700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5aec5c20 0x55bf5aeeeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.080+0000 7f3652039700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5aec5c20 0x55bf5aeeeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48464/0 (socket says 172.21.15.92:48464) 2023-10-06T10:42:47.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.080+0000 7f3652039700 1 -- 172.21.15.92:0/1840230057 learned_addr learned my addr 172.21.15.92:0/1840230057 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:47.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.080+0000 7f365283a700 1 --2- 172.21.15.92:0/1840230057 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bf5af0bae0 0x55bf5af02a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.080+0000 7f3651838700 1 --2- 172.21.15.92:0/1840230057 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bf5aec79f0 0x55bf5aeeb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.080+0000 7f3652039700 1 -- 172.21.15.92:0/1840230057 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bf5aec79f0 msgr2=0x55bf5aeeb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.080+0000 7f3652039700 1 --2- 172.21.15.92:0/1840230057 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bf5aec79f0 0x55bf5aeeb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.080+0000 7f3652039700 1 -- 172.21.15.92:0/1840230057 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bf5af0bae0 msgr2=0x55bf5af02a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.080+0000 7f3652039700 1 --2- 172.21.15.92:0/1840230057 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bf5af0bae0 0x55bf5af02a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.080+0000 7f3652039700 1 -- 172.21.15.92:0/1840230057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3640018550 con 0x55bf5aec5c20 2023-10-06T10:42:47.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.080+0000 7f3652039700 1 --2- 172.21.15.92:0/1840230057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5aec5c20 0x55bf5aeeeff0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f363c00b3c0 tx=0x7f363c01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67131baf12b23f8f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.080+0000 7f3630ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:42:47.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.081+0000 7f3630ff9700 1 -- 172.21.15.92:0/1840230057 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c011e10 con 0x55bf5aec5c20 2023-10-06T10:42:47.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.081+0000 7f3630ff9700 1 -- 172.21.15.92:0/1840230057 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f363c05a050 con 0x55bf5aec5c20 2023-10-06T10:42:47.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.081+0000 7f365d595b80 1 -- 172.21.15.92:0/1840230057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf5aeb96e0 con 0x55bf5aec5c20 2023-10-06T10:42:47.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.081+0000 7f3630ff9700 1 -- 172.21.15.92:0/1840230057 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f363c056ed0 con 0x55bf5aec5c20 2023-10-06T10:42:47.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.081+0000 7f365d595b80 10 client.6933 fetch_fsmap learned FSMap version 59 2023-10-06T10:42:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.081+0000 7f365d595b80 1 -- 172.21.15.92:0/1840230057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x55bf5ac52030 con 0x55bf5aec5c20 2023-10-06T10:42:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.081+0000 7f3630ff9700 1 -- 172.21.15.92:0/1840230057 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bf5aeb96e0 con 0x55bf5aec5c20 2023-10-06T10:42:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.081+0000 7f3630ff9700 1 -- 172.21.15.92:0/1840230057 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 59) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f363c0860e0 con 0x55bf5aec5c20 2023-10-06T10:42:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.081+0000 7f365d595b80 10 client.6933 fetch_fsmap finished waiting for FSMap version 59 2023-10-06T10:42:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.081+0000 7f365d595b80 10 client.6933 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-10-06T10:42:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.081+0000 7f365d595b80 20 client.6933 populate_metadata read hostname 'smithi092' 2023-10-06T10:42:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.081+0000 7f365d595b80 1 --2- 172.21.15.92:0/1840230057 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x55bf5aeee730 0x55bf5aeecaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.081+0000 7f365d595b80 4 client.6933 mds_command: new command op to 6486 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:42:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.081+0000 7f365d595b80 1 -- 172.21.15.92:0/1840230057 --> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf5ae96610 con 0x55bf5aeee730 2023-10-06T10:42:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.081+0000 7f3651838700 1 --2- 172.21.15.92:0/1840230057 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x55bf5aeee730 0x55bf5aeecaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.081+0000 7f3630ff9700 10 client.6933 ms_handle_connect on v2:172.21.15.92:6835/818031030 2023-10-06T10:42:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.081+0000 7f3651838700 1 --2- 172.21.15.92:0/1840230057 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x55bf5aeee730 0x55bf5aeecaf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=758d141b2c13a346 server_cookie=14a1b970b8a4c47b in_seq=0 out_seq=0 2023-10-06T10:42:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.082+0000 7f3630ff9700 1 -- 172.21.15.92:0/1840230057 <== mds.0 v2:172.21.15.92:6835/818031030 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55bf5ae96610 con 0x55bf5aeee730 2023-10-06T10:42:47.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.082+0000 7f3630ff9700 10 client.6933 handle_command_reply: tid=0 2023-10-06T10:42:47.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.082+0000 7f365d595b80 1 client.6933 shutdown 2023-10-06T10:42:47.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.082+0000 7f365d595b80 1 -- 172.21.15.92:0/1840230057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5aec5c20 msgr2=0x55bf5aeeeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.082+0000 7f365d595b80 1 --2- 172.21.15.92:0/1840230057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5aec5c20 0x55bf5aeeeff0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f363c00b3c0 tx=0x7f363c01e040 comp rx=0 tx=0).stop 2023-10-06T10:42:47.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.082+0000 7f365d595b80 1 -- 172.21.15.92:0/1840230057 shutdown_connections 2023-10-06T10:42:47.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.082+0000 7f365d595b80 1 --2- 172.21.15.92:0/1840230057 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bf5af0bae0 0x55bf5af02a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.082+0000 7f365d595b80 1 --2- 172.21.15.92:0/1840230057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5aec5c20 0x55bf5aeeeff0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.082+0000 7f365d595b80 1 --2- 172.21.15.92:0/1840230057 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x55bf5aeee730 0x55bf5aeecaf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.082+0000 7f365d595b80 1 --2- 172.21.15.92:0/1840230057 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bf5aec79f0 0x55bf5aeeb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.082+0000 7f3630ff9700 0 client.6933 ms_handle_reset on v2:172.21.15.92:6835/818031030 2023-10-06T10:42:47.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.082+0000 7f365d595b80 1 -- 172.21.15.92:0/1840230057 >> 172.21.15.92:0/1840230057 conn(0x55bf5aec60c0 msgr2=0x55bf5af09340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:47.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.082+0000 7f365d595b80 1 -- 172.21.15.92:0/1840230057 shutdown_connections 2023-10-06T10:42:47.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.082+0000 7f365d595b80 1 -- 172.21.15.92:0/1840230057 wait complete. 2023-10-06T10:42:47.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.082+0000 7f365d595b80 20 client.6933 trim_cache size 0 max 16384 2023-10-06T10:42:47.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.095+0000 7f365d595b80 1 Processor -- start 2023-10-06T10:42:47.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.095+0000 7f365d595b80 1 -- start start 2023-10-06T10:42:47.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.095+0000 7f365d595b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bf5aec5c20 0x55bf5af0a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.095+0000 7f365d595b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5aec79f0 0x55bf5aeed1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.095+0000 7f365d595b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bf5af0bae0 0x55bf5aeecaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.095+0000 7f365d595b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf5ac97e80 con 0x55bf5aec79f0 2023-10-06T10:42:47.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.095+0000 7f365d595b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf5aec7300 con 0x55bf5aec5c20 2023-10-06T10:42:47.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.095+0000 7f365d595b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf5aebf3a0 con 0x55bf5af0bae0 2023-10-06T10:42:47.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.095+0000 7f3651838700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5aec79f0 0x55bf5aeed1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.095+0000 7f3651838700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5aec79f0 0x55bf5aeed1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48476/0 (socket says 172.21.15.92:48476) 2023-10-06T10:42:47.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.095+0000 7f3651838700 1 -- 172.21.15.92:0/3481626592 learned_addr learned my addr 172.21.15.92:0/3481626592 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:47.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.096+0000 7f365283a700 1 --2- 172.21.15.92:0/3481626592 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bf5af0bae0 0x55bf5aeecaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.096+0000 7f3652039700 1 --2- 172.21.15.92:0/3481626592 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bf5aec5c20 0x55bf5af0a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.096+0000 7f3651838700 1 -- 172.21.15.92:0/3481626592 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bf5af0bae0 msgr2=0x55bf5aeecaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.096+0000 7f3651838700 1 --2- 172.21.15.92:0/3481626592 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bf5af0bae0 0x55bf5aeecaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.096+0000 7f3651838700 1 -- 172.21.15.92:0/3481626592 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bf5aec5c20 msgr2=0x55bf5af0a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.096+0000 7f3651838700 1 --2- 172.21.15.92:0/3481626592 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bf5aec5c20 0x55bf5af0a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.096+0000 7f3651838700 1 -- 172.21.15.92:0/3481626592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf5ade37d0 con 0x55bf5aec79f0 2023-10-06T10:42:47.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.096+0000 7f3651838700 1 --2- 172.21.15.92:0/3481626592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5aec79f0 0x55bf5aeed1e0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f36400187c0 tx=0x7f3640019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ca6b5acd74fc72 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.096+0000 7f3633fff700 1 -- 172.21.15.92:0/3481626592 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640035030 con 0x55bf5aec79f0 2023-10-06T10:42:47.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.096+0000 7f3633fff700 1 -- 172.21.15.92:0/3481626592 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3640016ac0 con 0x55bf5aec79f0 2023-10-06T10:42:47.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.096+0000 7f3633fff700 1 -- 172.21.15.92:0/3481626592 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640016c30 con 0x55bf5aec79f0 2023-10-06T10:42:47.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.096+0000 7f365d595b80 1 -- 172.21.15.92:0/3481626592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5aec79f0 msgr2=0x55bf5aeed1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.096+0000 7f365d595b80 1 --2- 172.21.15.92:0/3481626592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5aec79f0 0x55bf5aeed1e0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f36400187c0 tx=0x7f3640019000 comp rx=0 tx=0).stop 2023-10-06T10:42:47.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.096+0000 7f365d595b80 1 -- 172.21.15.92:0/3481626592 shutdown_connections 2023-10-06T10:42:47.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.096+0000 7f365d595b80 1 --2- 172.21.15.92:0/3481626592 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bf5af0bae0 0x55bf5aeecaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.096+0000 7f365d595b80 1 --2- 172.21.15.92:0/3481626592 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bf5aec5c20 0x55bf5af0a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.096+0000 7f365d595b80 1 --2- 172.21.15.92:0/3481626592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5aec79f0 0x55bf5aeed1e0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.096+0000 7f365d595b80 1 -- 172.21.15.92:0/3481626592 >> 172.21.15.92:0/3481626592 conn(0x55bf5aec60c0 msgr2=0x55bf5af0beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:47.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.097+0000 7f365d595b80 1 -- 172.21.15.92:0/3481626592 shutdown_connections 2023-10-06T10:42:47.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.097+0000 7f365d595b80 1 -- 172.21.15.92:0/3481626592 wait complete. 2023-10-06T10:42:47.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.097+0000 7f365d595b80 1 -- start start 2023-10-06T10:42:47.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.097+0000 7f365d595b80 1 Processor -- start 2023-10-06T10:42:47.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.098+0000 7f365d595b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bf5aec5c20 0x55bf5aeeb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.098+0000 7f365d595b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bf5aec79f0 0x55bf5aeebe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.098+0000 7f365d595b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5af0bae0 0x55bf5aebf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.098+0000 7f365d595b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf5aec47e0 con 0x55bf5af0bae0 2023-10-06T10:42:47.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.098+0000 7f365d595b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf5af71000 con 0x55bf5aec5c20 2023-10-06T10:42:47.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.098+0000 7f365d595b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf5af247a0 con 0x55bf5aec79f0 2023-10-06T10:42:47.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.098+0000 7f365d595b80 10 client.0 _ll_register_callbacks cb 0x55bf5acb1f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:42:47.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.098+0000 7f365283a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5af0bae0 0x55bf5aebf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.098+0000 7f365283a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5af0bae0 0x55bf5aebf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48484/0 (socket says 172.21.15.92:48484) 2023-10-06T10:42:47.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.098+0000 7f365283a700 1 -- 172.21.15.92:0/3219715696 learned_addr learned my addr 172.21.15.92:0/3219715696 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:47.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.098+0000 7f3652039700 1 --2- 172.21.15.92:0/3219715696 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bf5aec5c20 0x55bf5aeeb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.098+0000 7f3651838700 1 --2- 172.21.15.92:0/3219715696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bf5aec79f0 0x55bf5aeebe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.098+0000 7f365283a700 1 -- 172.21.15.92:0/3219715696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bf5aec79f0 msgr2=0x55bf5aeebe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.098+0000 7f365283a700 1 --2- 172.21.15.92:0/3219715696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bf5aec79f0 0x55bf5aeebe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.098+0000 7f365283a700 1 -- 172.21.15.92:0/3219715696 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bf5aec5c20 msgr2=0x55bf5aeeb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.098+0000 7f365283a700 1 --2- 172.21.15.92:0/3219715696 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bf5aec5c20 0x55bf5aeeb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.098+0000 7f365283a700 1 -- 172.21.15.92:0/3219715696 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364002b040 con 0x55bf5af0bae0 2023-10-06T10:42:47.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.099+0000 7f365283a700 1 --2- 172.21.15.92:0/3219715696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5af0bae0 0x55bf5aebf790 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f364400ce60 tx=0x7f3644007af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb3b5615fc72b1b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.099+0000 7f3630ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:42:47.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.099+0000 7f3630ff9700 1 -- 172.21.15.92:0/3219715696 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644007e60 con 0x55bf5af0bae0 2023-10-06T10:42:47.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.099+0000 7f3630ff9700 1 -- 172.21.15.92:0/3219715696 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f36440173f0 con 0x55bf5af0bae0 2023-10-06T10:42:47.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.099+0000 7f365d595b80 1 -- 172.21.15.92:0/3219715696 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3640016880 con 0x55bf5af0bae0 2023-10-06T10:42:47.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.099+0000 7f3630ff9700 1 -- 172.21.15.92:0/3219715696 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3644007c30 con 0x55bf5af0bae0 2023-10-06T10:42:47.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.099+0000 7f365d595b80 10 client.6939 fetch_fsmap learned FSMap version 59 2023-10-06T10:42:47.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.099+0000 7f365d595b80 1 -- 172.21.15.92:0/3219715696 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x55bf5adca4c0 con 0x55bf5af0bae0 2023-10-06T10:42:47.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.099+0000 7f3630ff9700 1 -- 172.21.15.92:0/3219715696 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3640016880 con 0x55bf5af0bae0 2023-10-06T10:42:47.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.099+0000 7f3630ff9700 1 -- 172.21.15.92:0/3219715696 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 59) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f3644016070 con 0x55bf5af0bae0 2023-10-06T10:42:47.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.099+0000 7f365d595b80 10 client.6939 fetch_fsmap finished waiting for FSMap version 59 2023-10-06T10:42:47.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.099+0000 7f365d595b80 10 client.6939 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-10-06T10:42:47.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.099+0000 7f365d595b80 20 client.6939 populate_metadata read hostname 'smithi092' 2023-10-06T10:42:47.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.099+0000 7f365d595b80 1 --2- 172.21.15.92:0/3219715696 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x55bf5aeec3a0 0x55bf5aebfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.099+0000 7f365d595b80 4 client.6939 mds_command: new command op to 6486 tid=0[{"prefix": "damage ls"}] 2023-10-06T10:42:47.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.099+0000 7f365d595b80 1 -- 172.21.15.92:0/3219715696 --> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bf5adb7e50 con 0x55bf5aeec3a0 2023-10-06T10:42:47.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.099+0000 7f3652039700 1 --2- 172.21.15.92:0/3219715696 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x55bf5aeec3a0 0x55bf5aebfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.099+0000 7f3630ff9700 10 client.6939 ms_handle_connect on v2:172.21.15.92:6835/818031030 2023-10-06T10:42:47.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.099+0000 7f3652039700 1 --2- 172.21.15.92:0/3219715696 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x55bf5aeec3a0 0x55bf5aebfcd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df9278b6141fbe13 server_cookie=d567d9575440052f in_seq=0 out_seq=0 2023-10-06T10:42:47.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f3630ff9700 1 -- 172.21.15.92:0/3219715696 <== mds.0 v2:172.21.15.92:6835/818031030 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+272 (crc 0 0 0) 0x55bf5adb7e50 con 0x55bf5aeec3a0 2023-10-06T10:42:47.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f3630ff9700 10 client.6939 handle_command_reply: tid=0 2023-10-06T10:42:47.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f365d595b80 1 client.6939 shutdown 2023-10-06T10:42:47.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f365d595b80 1 -- 172.21.15.92:0/3219715696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5af0bae0 msgr2=0x55bf5aebf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f365d595b80 1 --2- 172.21.15.92:0/3219715696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5af0bae0 0x55bf5aebf790 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f364400ce60 tx=0x7f3644007af0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f365d595b80 1 -- 172.21.15.92:0/3219715696 shutdown_connections 2023-10-06T10:42:47.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f365d595b80 1 --2- 172.21.15.92:0/3219715696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5af0bae0 0x55bf5aebf790 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f365d595b80 1 --2- 172.21.15.92:0/3219715696 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bf5aec5c20 0x55bf5aeeb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f365d595b80 1 --2- 172.21.15.92:0/3219715696 >> [v2:172.21.15.92:6835/818031030,v1:172.21.15.92:6837/818031030] conn(0x55bf5aeec3a0 0x55bf5aebfcd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f365d595b80 1 --2- 172.21.15.92:0/3219715696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bf5aec79f0 0x55bf5aeebe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f3630ff9700 0 client.6939 ms_handle_reset on v2:172.21.15.92:6835/818031030 2023-10-06T10:42:47.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f365d595b80 1 -- 172.21.15.92:0/3219715696 >> 172.21.15.92:0/3219715696 conn(0x55bf5aec60c0 msgr2=0x55bf5aeea220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:47.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f365d595b80 1 -- 172.21.15.92:0/3219715696 shutdown_connections 2023-10-06T10:42:47.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f365d595b80 1 -- 172.21.15.92:0/3219715696 wait complete. 2023-10-06T10:42:47.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f365d595b80 20 client.6939 trim_cache size 0 max 16384 2023-10-06T10:42:47.109 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T10:42:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "damage_type": "backtrace", 2023-10-06T10:42:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 1062741865, 2023-10-06T10:42:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 1099511627779, 2023-10-06T10:42:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3/d4" 2023-10-06T10:42:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:42:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "damage_type": "backtrace", 2023-10-06T10:42:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 3273467894, 2023-10-06T10:42:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 1099511627778, 2023-10-06T10:42:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3" 2023-10-06T10:42:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:47.110 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T10:42:47.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f365429d700 1 -- 172.21.15.92:0/1897032726 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3638066f30 msgr2=0x7f36380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f365429d700 1 --2- 172.21.15.92:0/1897032726 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3638066f30 0x7f36380693e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f364c1559c0 tx=0x7f3640007b40 comp rx=0 tx=0).stop 2023-10-06T10:42:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f365429d700 1 -- 172.21.15.92:0/1897032726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364c0cd4d0 msgr2=0x7f364c155ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f365429d700 1 --2- 172.21.15.92:0/1897032726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364c0cd4d0 0x7f364c155ae0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f364c0d14c0 tx=0x7f363c016000 comp rx=0 tx=0).stop 2023-10-06T10:42:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f365429d700 1 -- 172.21.15.92:0/1897032726 shutdown_connections 2023-10-06T10:42:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f365429d700 1 --2- 172.21.15.92:0/1897032726 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3638066f30 0x7f36380693e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f365429d700 1 --2- 172.21.15.92:0/1897032726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f364c156020 0x7f364c152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f365429d700 1 --2- 172.21.15.92:0/1897032726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364c0cd4d0 0x7f364c155ae0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f365429d700 1 --2- 172.21.15.92:0/1897032726 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f364c15a160 0x7f364c16a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.100+0000 7f365429d700 1 -- 172.21.15.92:0/1897032726 >> 172.21.15.92:0/1897032726 conn(0x7f364c099700 msgr2=0x7f364c0cbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.101+0000 7f365429d700 1 -- 172.21.15.92:0/1897032726 shutdown_connections 2023-10-06T10:42:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.101+0000 7f365429d700 1 -- 172.21.15.92:0/1897032726 wait complete. 2023-10-06T10:42:47.113 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:1 damage ls 2023-10-06T10:42:47.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.227+0000 7ff0b98a7700 1 Processor -- start 2023-10-06T10:42:47.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.227+0000 7ff0b98a7700 1 -- start start 2023-10-06T10:42:47.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.228+0000 7ff0b98a7700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0b40c5250 0x7ff0b40ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.228+0000 7ff0b98a7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff0b40cd300 0x7ff0b40b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.228+0000 7ff0b98a7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b415ac30 con 0x7ff0b4150940 2023-10-06T10:42:47.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.228+0000 7ff0b98a7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b40cd780 con 0x7ff0b40cd300 2023-10-06T10:42:47.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.228+0000 7ff0b98a7700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b40cb6a0 con 0x7ff0b40c5250 2023-10-06T10:42:47.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.228+0000 7ff0b2ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff0b4150940 0x7ff0b4159c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56916/0 (socket says 172.21.15.92:56916) 2023-10-06T10:42:47.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.228+0000 7ff0b2ffd700 1 -- 172.21.15.92:0/250083486 learned_addr learned my addr 172.21.15.92:0/250083486 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:42:47.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.228+0000 7ff0b37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff0b40cd300 0x7ff0b40b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.228+0000 7ff0b27fc700 1 -- 172.21.15.92:0/250083486 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 455493907 0 0) 0x7ff0a4001290 con 0x7ff0b4150940 2023-10-06T10:42:47.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.228+0000 7ff0b27fc700 1 -- 172.21.15.92:0/250083486 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0a0003570 con 0x7ff0b4150940 2023-10-06T10:42:47.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.228+0000 7ff0b37fe700 1 -- 172.21.15.92:0/250083486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0b40c5250 msgr2=0x7ff0b40ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.228+0000 7ff0aa7fc700 1 --2- 172.21.15.92:0/250083486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0b40c5250 0x7ff0b40ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.228+0000 7ff0b37fe700 1 --2- 172.21.15.92:0/250083486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0b40c5250 0x7ff0b40ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.228+0000 7ff0b37fe700 1 -- 172.21.15.92:0/250083486 >> v1:172.21.15.92:6789/0 conn(0x7ff0b4150940 legacy=0x7ff0b4159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.228+0000 7ff0b37fe700 1 -- 172.21.15.92:0/250083486 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b40cb090 con 0x7ff0b40cd300 2023-10-06T10:42:47.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.228+0000 7ff0aa7fc700 1 --2- 172.21.15.92:0/250083486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0b40c5250 0x7ff0b40ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:47.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.228+0000 7ff0b37fe700 1 --2- 172.21.15.92:0/250083486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0b40cd300 0x7ff0b40b2050 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009e60 tx=0x7ff0ac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30369bd1e1ba028 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.229+0000 7ff0b27fc700 1 -- 172.21.15.92:0/250083486 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac00e070 con 0x7ff0b40cd300 2023-10-06T10:42:47.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.229+0000 7ff0b27fc700 1 -- 172.21.15.92:0/250083486 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ac00bb60 con 0x7ff0b40cd300 2023-10-06T10:42:47.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.229+0000 7ff0b27fc700 1 -- 172.21.15.92:0/250083486 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac012430 con 0x7ff0b40cd300 2023-10-06T10:42:47.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.229+0000 7ff0b98a7700 1 -- 172.21.15.92:0/250083486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0b40cd300 msgr2=0x7ff0b40b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.229+0000 7ff0b98a7700 1 --2- 172.21.15.92:0/250083486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0b40cd300 0x7ff0b40b2050 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009e60 tx=0x7ff0ac009780 comp rx=0 tx=0).stop 2023-10-06T10:42:47.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.229+0000 7ff0b98a7700 1 -- 172.21.15.92:0/250083486 shutdown_connections 2023-10-06T10:42:47.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.229+0000 7ff0b98a7700 1 --2- 172.21.15.92:0/250083486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0b40cd300 0x7ff0b40b2050 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.229+0000 7ff0b98a7700 1 --2- 172.21.15.92:0/250083486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0b40c5250 0x7ff0b40ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.229+0000 7ff0b98a7700 1 -- 172.21.15.92:0/250083486 >> 172.21.15.92:0/250083486 conn(0x7ff0b4099700 msgr2=0x7ff0b406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:47.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.229+0000 7ff0b98a7700 1 -- 172.21.15.92:0/250083486 shutdown_connections 2023-10-06T10:42:47.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.229+0000 7ff0b98a7700 1 -- 172.21.15.92:0/250083486 wait complete. 2023-10-06T10:42:47.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.230+0000 7ff0b98a7700 1 Processor -- start 2023-10-06T10:42:47.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.230+0000 7ff0b98a7700 1 -- start start 2023-10-06T10:42:47.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.231+0000 7ff0b98a7700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0b40c5250 0x7ff0b4154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.231+0000 7ff0b98a7700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0b40cd300 0x7ff0b40c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.231+0000 7ff0b98a7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0b414ffc0 0x7ff0b41560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.231+0000 7ff0b98a7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b4156b80 con 0x7ff0b414ffc0 2023-10-06T10:42:47.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.231+0000 7ff0b98a7700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b40c8340 con 0x7ff0b40c5250 2023-10-06T10:42:47.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.231+0000 7ff0b98a7700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b40b39d0 con 0x7ff0b40cd300 2023-10-06T10:42:47.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.231+0000 7ff0b37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0b414ffc0 0x7ff0b41560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.231+0000 7ff0b37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0b414ffc0 0x7ff0b41560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48504/0 (socket says 172.21.15.92:48504) 2023-10-06T10:42:47.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.231+0000 7ff0b37fe700 1 -- 172.21.15.92:0/3084243020 learned_addr learned my addr 172.21.15.92:0/3084243020 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:47.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.231+0000 7ff0aa7fc700 1 --2- 172.21.15.92:0/3084243020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0b40cd300 0x7ff0b40c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.231+0000 7ff0b2ffd700 1 --2- 172.21.15.92:0/3084243020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0b40c5250 0x7ff0b4154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.232+0000 7ff0b37fe700 1 -- 172.21.15.92:0/3084243020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0b40cd300 msgr2=0x7ff0b40c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.232+0000 7ff0b37fe700 1 --2- 172.21.15.92:0/3084243020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0b40cd300 0x7ff0b40c7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.232+0000 7ff0b37fe700 1 -- 172.21.15.92:0/3084243020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0b40c5250 msgr2=0x7ff0b4154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.232+0000 7ff0b37fe700 1 --2- 172.21.15.92:0/3084243020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0b40c5250 0x7ff0b4154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.232+0000 7ff0b37fe700 1 -- 172.21.15.92:0/3084243020 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ac009a20 con 0x7ff0b414ffc0 2023-10-06T10:42:47.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.232+0000 7ff0b2ffd700 1 --2- 172.21.15.92:0/3084243020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0b40c5250 0x7ff0b4154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:47.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.232+0000 7ff0aa7fc700 1 --2- 172.21.15.92:0/3084243020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0b40cd300 0x7ff0b40c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:47.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.232+0000 7ff0b37fe700 1 --2- 172.21.15.92:0/3084243020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0b414ffc0 0x7ff0b41560e0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40d14c0 tx=0x7ff0ac019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.232+0000 7ff0ab7fe700 1 -- 172.21.15.92:0/3084243020 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac01d630 con 0x7ff0b414ffc0 2023-10-06T10:42:47.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.232+0000 7ff0ab7fe700 1 -- 172.21.15.92:0/3084243020 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ac01d7b0 con 0x7ff0b414ffc0 2023-10-06T10:42:47.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.232+0000 7ff0b98a7700 1 -- 172.21.15.92:0/3084243020 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b4156620 con 0x7ff0b414ffc0 2023-10-06T10:42:47.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.232+0000 7ff0ab7fe700 1 -- 172.21.15.92:0/3084243020 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac01db70 con 0x7ff0b414ffc0 2023-10-06T10:42:47.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.232+0000 7ff0b98a7700 1 -- 172.21.15.92:0/3084243020 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b40b3380 con 0x7ff0b414ffc0 2023-10-06T10:42:47.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.233+0000 7ff0ab7fe700 1 -- 172.21.15.92:0/3084243020 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff0ac024070 con 0x7ff0b414ffc0 2023-10-06T10:42:47.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.234+0000 7ff0ab7fe700 1 --2- 172.21.15.92:0/3084243020 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff094066f30 0x7ff0940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.234+0000 7ff0ab7fe700 1 -- 172.21.15.92:0/3084243020 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7ff0ac086f70 con 0x7ff0b414ffc0 2023-10-06T10:42:47.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.234+0000 7ff0b2ffd700 1 --2- 172.21.15.92:0/3084243020 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff094066f30 0x7ff0940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.234+0000 7ff0b2ffd700 1 --2- 172.21.15.92:0/3084243020 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff094066f30 0x7ff0940693e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40013f0 tx=0x7ff0a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.235+0000 7ff0c2b9fb80 1 Processor -- start 2023-10-06T10:42:47.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.236+0000 7ff0c2b9fb80 1 -- start start 2023-10-06T10:42:47.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.236+0000 7ff0c2b9fb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c818109ae0 0x55c818109eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.236+0000 7ff0c2b9fb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c8180c59f0 0x55c8180c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.236+0000 7ff0c2b9fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c3c20 0x55c8180c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.236+0000 7ff0c2b9fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c818042cd0 con 0x55c8180c3c20 2023-10-06T10:42:47.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.236+0000 7ff0c2b9fb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c818019ac0 con 0x55c818109ae0 2023-10-06T10:42:47.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.236+0000 7ff0c2b9fb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8180172c0 con 0x55c8180c59f0 2023-10-06T10:42:47.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.237+0000 7ff0b37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c3c20 0x55c8180c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.237+0000 7ff0b37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c3c20 0x55c8180c2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48512/0 (socket says 172.21.15.92:48512) 2023-10-06T10:42:47.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.237+0000 7ff0b37fe700 1 -- 172.21.15.92:0/813036629 learned_addr learned my addr 172.21.15.92:0/813036629 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:47.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.237+0000 7ff0aa7fc700 1 --2- 172.21.15.92:0/813036629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c8180c59f0 0x55c8180c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.237+0000 7ff0b2ffd700 1 --2- 172.21.15.92:0/813036629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c818109ae0 0x55c818109eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.237+0000 7ff0b37fe700 1 -- 172.21.15.92:0/813036629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c8180c59f0 msgr2=0x55c8180c25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.237+0000 7ff0b37fe700 1 --2- 172.21.15.92:0/813036629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c8180c59f0 0x55c8180c25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.237+0000 7ff0b37fe700 1 -- 172.21.15.92:0/813036629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c818109ae0 msgr2=0x55c818109eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.237+0000 7ff0b37fe700 1 --2- 172.21.15.92:0/813036629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c818109ae0 0x55c818109eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.237+0000 7ff0b37fe700 1 -- 172.21.15.92:0/813036629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c817ead2c0 con 0x55c8180c3c20 2023-10-06T10:42:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.237+0000 7ff0b2ffd700 1 --2- 172.21.15.92:0/813036629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c818109ae0 0x55c818109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.237+0000 7ff0b37fe700 1 --2- 172.21.15.92:0/813036629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c3c20 0x55c8180c2b10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00c010 tx=0x7ff0ac020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1752976481588a20 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.237+0000 7ff093fff700 1 -- 172.21.15.92:0/813036629 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac00e040 con 0x55c8180c3c20 2023-10-06T10:42:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.237+0000 7ff093fff700 1 -- 172.21.15.92:0/813036629 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ac057520 con 0x55c8180c3c20 2023-10-06T10:42:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.237+0000 7ff093fff700 1 -- 172.21.15.92:0/813036629 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac0582c0 con 0x55c8180c3c20 2023-10-06T10:42:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.238+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/813036629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c3c20 msgr2=0x55c8180c2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.238+0000 7ff0c2b9fb80 1 --2- 172.21.15.92:0/813036629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c3c20 0x55c8180c2b10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00c010 tx=0x7ff0ac020040 comp rx=0 tx=0).stop 2023-10-06T10:42:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.238+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/813036629 shutdown_connections 2023-10-06T10:42:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.238+0000 7ff0c2b9fb80 1 --2- 172.21.15.92:0/813036629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c3c20 0x55c8180c2b10 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.238+0000 7ff0c2b9fb80 1 --2- 172.21.15.92:0/813036629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c818109ae0 0x55c818109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.238+0000 7ff0c2b9fb80 1 --2- 172.21.15.92:0/813036629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c8180c59f0 0x55c8180c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.238+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/813036629 >> 172.21.15.92:0/813036629 conn(0x55c8180c40c0 msgr2=0x55c8180c4490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.238+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/813036629 shutdown_connections 2023-10-06T10:42:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.238+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/813036629 wait complete. 2023-10-06T10:42:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.238+0000 7ff0c2b9fb80 1 -- start start 2023-10-06T10:42:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.238+0000 7ff0c2b9fb80 1 Processor -- start 2023-10-06T10:42:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.239+0000 7ff0c2b9fb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c8180c3c20 0x55c818104930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.239+0000 7ff0c2b9fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c59f0 0x55c8180e70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.239+0000 7ff0c2b9fb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c818109ae0 0x55c8180eaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.239+0000 7ff0c2b9fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8180130b0 con 0x55c8180c59f0 2023-10-06T10:42:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.239+0000 7ff0c2b9fb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c817fbe8e0 con 0x55c8180c3c20 2023-10-06T10:42:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.239+0000 7ff0c2b9fb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c817fd9ef0 con 0x55c818109ae0 2023-10-06T10:42:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.239+0000 7ff0c2b9fb80 10 client.0 _ll_register_callbacks cb 0x55c817eaff80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:42:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.239+0000 7ff0aa7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c59f0 0x55c8180e70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.239+0000 7ff0aa7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c59f0 0x55c8180e70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48516/0 (socket says 172.21.15.92:48516) 2023-10-06T10:42:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.240+0000 7ff0aa7fc700 1 -- 172.21.15.92:0/930792838 learned_addr learned my addr 172.21.15.92:0/930792838 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.240+0000 7ff0b2ffd700 1 --2- 172.21.15.92:0/930792838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c8180c3c20 0x55c818104930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.240+0000 7ff0b37fe700 1 --2- 172.21.15.92:0/930792838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c818109ae0 0x55c8180eaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.240+0000 7ff0aa7fc700 1 -- 172.21.15.92:0/930792838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c818109ae0 msgr2=0x55c8180eaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.240+0000 7ff0aa7fc700 1 --2- 172.21.15.92:0/930792838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c818109ae0 0x55c8180eaa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.240+0000 7ff0aa7fc700 1 -- 172.21.15.92:0/930792838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c8180c3c20 msgr2=0x55c818104930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.240+0000 7ff0aa7fc700 1 --2- 172.21.15.92:0/930792838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c8180c3c20 0x55c818104930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.240+0000 7ff0aa7fc700 1 -- 172.21.15.92:0/930792838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b40b3380 con 0x55c8180c59f0 2023-10-06T10:42:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.240+0000 7ff0aa7fc700 1 --2- 172.21.15.92:0/930792838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c59f0 0x55c8180e70e0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ff09c011640 tx=0x7ff09c006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4527a2bd45bd0f2a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.240+0000 7ff0917fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:42:47.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.240+0000 7ff0917fa700 1 -- 172.21.15.92:0/930792838 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c011e40 con 0x55c8180c59f0 2023-10-06T10:42:47.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.240+0000 7ff0917fa700 1 -- 172.21.15.92:0/930792838 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7ff09c015650 con 0x55c8180c59f0 2023-10-06T10:42:47.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.240+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/930792838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8180b76e0 con 0x55c8180c59f0 2023-10-06T10:42:47.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.240+0000 7ff0917fa700 1 -- 172.21.15.92:0/930792838 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff09c015a20 con 0x55c8180c59f0 2023-10-06T10:42:47.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.240+0000 7ff0c2b9fb80 10 client.6954 fetch_fsmap learned FSMap version 59 2023-10-06T10:42:47.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.240+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/930792838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x55c817e50030 con 0x55c8180c59f0 2023-10-06T10:42:47.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.240+0000 7ff0917fa700 1 -- 172.21.15.92:0/930792838 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c8180b76e0 con 0x55c8180c59f0 2023-10-06T10:42:47.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.240+0000 7ff0917fa700 1 -- 172.21.15.92:0/930792838 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 59) v1 ==== 3140+0+0 (secure 0 0 0) 0x7ff09c015b80 con 0x55c8180c59f0 2023-10-06T10:42:47.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.240+0000 7ff0c2b9fb80 10 client.6954 fetch_fsmap finished waiting for FSMap version 59 2023-10-06T10:42:47.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.240+0000 7ff0c2b9fb80 10 client.6954 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.a 2023-10-06T10:42:47.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.241+0000 7ff0c2b9fb80 20 client.6954 populate_metadata read hostname 'smithi092' 2023-10-06T10:42:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.241+0000 7ff0c2b9fb80 1 --2- 172.21.15.92:0/930792838 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x55c8180e8f90 0x55c8180ecfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.241+0000 7ff0c2b9fb80 4 client.6954 mds_command: new command op to 6483 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:42:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.241+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/930792838 --> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c818094610 con 0x55c8180e8f90 2023-10-06T10:42:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.241+0000 7ff0b2ffd700 1 --2- 172.21.15.92:0/930792838 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x55c8180e8f90 0x55c8180ecfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.241+0000 7ff0917fa700 10 client.6954 ms_handle_connect on v2:172.21.15.92:6834/171578176 2023-10-06T10:42:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.241+0000 7ff0b2ffd700 1 --2- 172.21.15.92:0/930792838 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x55c8180e8f90 0x55c8180ecfb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15e4df4cf26d4b5e server_cookie=389558c9c79bc5bb in_seq=0 out_seq=0 2023-10-06T10:42:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.241+0000 7ff0917fa700 1 -- 172.21.15.92:0/930792838 <== mds.1 v2:172.21.15.92:6834/171578176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55c818094610 con 0x55c8180e8f90 2023-10-06T10:42:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.241+0000 7ff0917fa700 10 client.6954 handle_command_reply: tid=0 2023-10-06T10:42:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.241+0000 7ff0c2b9fb80 1 client.6954 shutdown 2023-10-06T10:42:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.242+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/930792838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c59f0 msgr2=0x55c8180e70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.242+0000 7ff0c2b9fb80 1 --2- 172.21.15.92:0/930792838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c59f0 0x55c8180e70e0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ff09c011640 tx=0x7ff09c006f90 comp rx=0 tx=0).stop 2023-10-06T10:42:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.242+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/930792838 shutdown_connections 2023-10-06T10:42:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.242+0000 7ff0c2b9fb80 1 --2- 172.21.15.92:0/930792838 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x55c8180e8f90 0x55c8180ecfb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.242+0000 7ff0c2b9fb80 1 --2- 172.21.15.92:0/930792838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c818109ae0 0x55c8180eaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.242+0000 7ff0917fa700 0 client.6954 ms_handle_reset on v2:172.21.15.92:6834/171578176 2023-10-06T10:42:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.242+0000 7ff0c2b9fb80 1 --2- 172.21.15.92:0/930792838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c8180c3c20 0x55c818104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.242+0000 7ff0c2b9fb80 1 --2- 172.21.15.92:0/930792838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c59f0 0x55c8180e70e0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.242+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/930792838 >> 172.21.15.92:0/930792838 conn(0x55c8180c40c0 msgr2=0x55c818108ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.242+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/930792838 shutdown_connections 2023-10-06T10:42:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.242+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/930792838 wait complete. 2023-10-06T10:42:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.242+0000 7ff0c2b9fb80 20 client.6954 trim_cache size 0 max 16384 2023-10-06T10:42:47.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.254+0000 7ff0c2b9fb80 1 Processor -- start 2023-10-06T10:42:47.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.254+0000 7ff0c2b9fb80 1 -- start start 2023-10-06T10:42:47.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.255+0000 7ff0c2b9fb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c8180c3c20 0x55c818107530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.255+0000 7ff0c2b9fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c59f0 0x55c8181082d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.255+0000 7ff0c2b9fb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c818109ae0 0x55c818108810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.255+0000 7ff0c2b9fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c817e95e80 con 0x55c8180c59f0 2023-10-06T10:42:47.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.255+0000 7ff0c2b9fb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c818108090 con 0x55c818109ae0 2023-10-06T10:42:47.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.255+0000 7ff0c2b9fb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8180c4ef0 con 0x55c8180c3c20 2023-10-06T10:42:47.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.255+0000 7ff0aa7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c59f0 0x55c8181082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.255+0000 7ff0aa7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c59f0 0x55c8181082d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48526/0 (socket says 172.21.15.92:48526) 2023-10-06T10:42:47.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.255+0000 7ff0aa7fc700 1 -- 172.21.15.92:0/3597775685 learned_addr learned my addr 172.21.15.92:0/3597775685 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:47.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.255+0000 7ff0b2ffd700 1 --2- 172.21.15.92:0/3597775685 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c8180c3c20 0x55c818107530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.255+0000 7ff0b37fe700 1 --2- 172.21.15.92:0/3597775685 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c818109ae0 0x55c818108810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.255+0000 7ff0aa7fc700 1 -- 172.21.15.92:0/3597775685 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c8180c3c20 msgr2=0x55c818107530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.255+0000 7ff0aa7fc700 1 --2- 172.21.15.92:0/3597775685 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c8180c3c20 0x55c818107530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.255+0000 7ff0aa7fc700 1 -- 172.21.15.92:0/3597775685 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c818109ae0 msgr2=0x55c818108810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.255+0000 7ff0aa7fc700 1 --2- 172.21.15.92:0/3597775685 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c818109ae0 0x55c818108810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.255+0000 7ff0aa7fc700 1 -- 172.21.15.92:0/3597775685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c817fe17d0 con 0x55c8180c59f0 2023-10-06T10:42:47.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.255+0000 7ff0aa7fc700 1 --2- 172.21.15.92:0/3597775685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c59f0 0x55c8181082d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7ff09c011ce0 tx=0x7ff09c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd99028a9a91996 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.255+0000 7ff0a8ff9700 1 -- 172.21.15.92:0/3597775685 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c00a5e0 con 0x55c8180c59f0 2023-10-06T10:42:47.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.255+0000 7ff0a8ff9700 1 -- 172.21.15.92:0/3597775685 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09c00c040 con 0x55c8180c59f0 2023-10-06T10:42:47.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.255+0000 7ff0a8ff9700 1 -- 172.21.15.92:0/3597775685 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c00a740 con 0x55c8180c59f0 2023-10-06T10:42:47.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.256+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/3597775685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c59f0 msgr2=0x55c8181082d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.256+0000 7ff0c2b9fb80 1 --2- 172.21.15.92:0/3597775685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c59f0 0x55c8181082d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7ff09c011ce0 tx=0x7ff09c01c000 comp rx=0 tx=0).stop 2023-10-06T10:42:47.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.256+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/3597775685 shutdown_connections 2023-10-06T10:42:47.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.256+0000 7ff0c2b9fb80 1 --2- 172.21.15.92:0/3597775685 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c818109ae0 0x55c818108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.256+0000 7ff0c2b9fb80 1 --2- 172.21.15.92:0/3597775685 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c8180c3c20 0x55c818107530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.256+0000 7ff0c2b9fb80 1 --2- 172.21.15.92:0/3597775685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c59f0 0x55c8181082d0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.256+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/3597775685 >> 172.21.15.92:0/3597775685 conn(0x55c8180c40c0 msgr2=0x55c8180e62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:47.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.256+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/3597775685 shutdown_connections 2023-10-06T10:42:47.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.256+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/3597775685 wait complete. 2023-10-06T10:42:47.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.256+0000 7ff0c2b9fb80 1 -- start start 2023-10-06T10:42:47.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.257+0000 7ff0c2b9fb80 1 Processor -- start 2023-10-06T10:42:47.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.257+0000 7ff0c2b9fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c3c20 0x55c81816e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.257+0000 7ff0c2b9fb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c8180c59f0 0x55c8180f19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.257+0000 7ff0c2b9fb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c818109ae0 0x55c818104620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.257+0000 7ff0c2b9fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8180c45c0 con 0x55c8180c3c20 2023-10-06T10:42:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.257+0000 7ff0c2b9fb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c81816ecb0 con 0x55c8180c59f0 2023-10-06T10:42:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.257+0000 7ff0c2b9fb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c818103880 con 0x55c818109ae0 2023-10-06T10:42:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.257+0000 7ff0c2b9fb80 10 client.0 _ll_register_callbacks cb 0x55c817eaff80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:42:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.257+0000 7ff0b2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c3c20 0x55c81816e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.257+0000 7ff0b2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c3c20 0x55c81816e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48538/0 (socket says 172.21.15.92:48538) 2023-10-06T10:42:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.257+0000 7ff0b2ffd700 1 -- 172.21.15.92:0/822626705 learned_addr learned my addr 172.21.15.92:0/822626705 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0b37fe700 1 --2- 172.21.15.92:0/822626705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c818109ae0 0x55c818104620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0aa7fc700 1 --2- 172.21.15.92:0/822626705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c8180c59f0 0x55c8180f19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0b2ffd700 1 -- 172.21.15.92:0/822626705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c818109ae0 msgr2=0x55c818104620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0b2ffd700 1 --2- 172.21.15.92:0/822626705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c818109ae0 0x55c818104620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0b2ffd700 1 -- 172.21.15.92:0/822626705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c8180c59f0 msgr2=0x55c8180f19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0b2ffd700 1 --2- 172.21.15.92:0/822626705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c8180c59f0 0x55c8180f19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0b2ffd700 1 -- 172.21.15.92:0/822626705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c817e50030 con 0x55c8180c3c20 2023-10-06T10:42:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0aa7fc700 1 --2- 172.21.15.92:0/822626705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c8180c59f0 0x55c8180f19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0b2ffd700 1 --2- 172.21.15.92:0/822626705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c3c20 0x55c81816e770 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4002ee0 tx=0x7ff0a4015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1fbb95282c266e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0917fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:42:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0917fa700 1 -- 172.21.15.92:0/822626705 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4029030 con 0x55c8180c3c20 2023-10-06T10:42:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0917fa700 1 -- 172.21.15.92:0/822626705 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7ff0a402a070 con 0x55c8180c3c20 2023-10-06T10:42:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/822626705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0ac057520 con 0x55c8180c3c20 2023-10-06T10:42:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0917fa700 1 -- 172.21.15.92:0/822626705 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0a4018040 con 0x55c8180c3c20 2023-10-06T10:42:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0c2b9fb80 10 client.6960 fetch_fsmap learned FSMap version 59 2023-10-06T10:42:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/822626705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x55c817fc84c0 con 0x55c8180c3c20 2023-10-06T10:42:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0917fa700 1 -- 172.21.15.92:0/822626705 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ac057520 con 0x55c8180c3c20 2023-10-06T10:42:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0917fa700 1 -- 172.21.15.92:0/822626705 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 59) v1 ==== 3140+0+0 (secure 0 0 0) 0x7ff0a400ea50 con 0x55c8180c3c20 2023-10-06T10:42:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0c2b9fb80 10 client.6960 fetch_fsmap finished waiting for FSMap version 59 2023-10-06T10:42:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0c2b9fb80 10 client.6960 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.a 2023-10-06T10:42:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0c2b9fb80 20 client.6960 populate_metadata read hostname 'smithi092' 2023-10-06T10:42:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0c2b9fb80 1 --2- 172.21.15.92:0/822626705 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x55c818104b60 0x55c818102a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0c2b9fb80 4 client.6960 mds_command: new command op to 6483 tid=0[{"prefix": "damage ls"}] 2023-10-06T10:42:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.258+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/822626705 --> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c817fb5e50 con 0x55c818104b60 2023-10-06T10:42:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.259+0000 7ff0aa7fc700 1 --2- 172.21.15.92:0/822626705 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x55c818104b60 0x55c818102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.259+0000 7ff0aa7fc700 1 --2- 172.21.15.92:0/822626705 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x55c818104b60 0x55c818102a30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=99986ea6e6b1da5e server_cookie=121edc9887ce21d2 in_seq=0 out_seq=0 2023-10-06T10:42:47.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.259+0000 7ff0917fa700 10 client.6960 ms_handle_connect on v2:172.21.15.92:6834/171578176 2023-10-06T10:42:47.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.259+0000 7ff0917fa700 1 -- 172.21.15.92:0/822626705 <== mds.1 v2:172.21.15.92:6834/171578176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+283 (crc 0 0 0) 0x55c817fb5e50 con 0x55c818104b60 2023-10-06T10:42:47.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.259+0000 7ff0917fa700 10 client.6960 handle_command_reply: tid=0 2023-10-06T10:42:47.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.259+0000 7ff0c2b9fb80 1 client.6960 shutdown 2023-10-06T10:42:47.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.259+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/822626705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c3c20 msgr2=0x55c81816e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.259+0000 7ff0c2b9fb80 1 --2- 172.21.15.92:0/822626705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c3c20 0x55c81816e770 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4002ee0 tx=0x7ff0a4015040 comp rx=0 tx=0).stop 2023-10-06T10:42:47.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.259+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/822626705 shutdown_connections 2023-10-06T10:42:47.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.259+0000 7ff0c2b9fb80 1 --2- 172.21.15.92:0/822626705 >> [v2:172.21.15.92:6834/171578176,v1:172.21.15.92:6836/171578176] conn(0x55c818104b60 0x55c818102a30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.259+0000 7ff0c2b9fb80 1 --2- 172.21.15.92:0/822626705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c818109ae0 0x55c818104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.259+0000 7ff0917fa700 0 client.6960 ms_handle_reset on v2:172.21.15.92:6834/171578176 2023-10-06T10:42:47.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.259+0000 7ff0c2b9fb80 1 --2- 172.21.15.92:0/822626705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8180c3c20 0x55c81816e770 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.259+0000 7ff0c2b9fb80 1 --2- 172.21.15.92:0/822626705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c8180c59f0 0x55c8180f19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.259+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/822626705 >> 172.21.15.92:0/822626705 conn(0x55c8180c40c0 msgr2=0x55c8180bf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:47.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.259+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/822626705 shutdown_connections 2023-10-06T10:42:47.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.259+0000 7ff0c2b9fb80 1 -- 172.21.15.92:0/822626705 wait complete. 2023-10-06T10:42:47.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.259+0000 7ff0c2b9fb80 20 client.6960 trim_cache size 0 max 16384 2023-10-06T10:42:47.268 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T10:42:47.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:47.268 INFO:teuthology.orchestra.run.smithi092.stdout: "damage_type": "backtrace", 2023-10-06T10:42:47.268 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 293383644, 2023-10-06T10:42:47.268 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 1099511627780, 2023-10-06T10:42:47.269 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3/d4/d5" 2023-10-06T10:42:47.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:42:47.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:47.269 INFO:teuthology.orchestra.run.smithi092.stdout: "damage_type": "backtrace", 2023-10-06T10:42:47.269 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 1894422809, 2023-10-06T10:42:47.269 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 1099511627781, 2023-10-06T10:42:47.269 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3/d4/d5/d6" 2023-10-06T10:42:47.269 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:47.269 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T10:42:47.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.260+0000 7ff0b98a7700 1 -- 172.21.15.92:0/3084243020 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff094066f30 msgr2=0x7ff0940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.260+0000 7ff0b98a7700 1 --2- 172.21.15.92:0/3084243020 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff094066f30 0x7ff0940693e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40013f0 tx=0x7ff0a4009040 comp rx=0 tx=0).stop 2023-10-06T10:42:47.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.260+0000 7ff0b98a7700 1 -- 172.21.15.92:0/3084243020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0b414ffc0 msgr2=0x7ff0b41560e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.260+0000 7ff0b98a7700 1 --2- 172.21.15.92:0/3084243020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0b414ffc0 0x7ff0b41560e0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40d14c0 tx=0x7ff0ac019000 comp rx=0 tx=0).stop 2023-10-06T10:42:47.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.260+0000 7ff0b98a7700 1 -- 172.21.15.92:0/3084243020 shutdown_connections 2023-10-06T10:42:47.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.260+0000 7ff0b98a7700 1 --2- 172.21.15.92:0/3084243020 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff094066f30 0x7ff0940693e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.260+0000 7ff0b98a7700 1 --2- 172.21.15.92:0/3084243020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0b414ffc0 0x7ff0b41560e0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.260+0000 7ff0b98a7700 1 --2- 172.21.15.92:0/3084243020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0b40c5250 0x7ff0b4154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.260+0000 7ff0b98a7700 1 --2- 172.21.15.92:0/3084243020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0b40cd300 0x7ff0b40c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.260+0000 7ff0b98a7700 1 -- 172.21.15.92:0/3084243020 >> 172.21.15.92:0/3084243020 conn(0x7ff0b4099700 msgr2=0x7ff0b40694c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:47.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.260+0000 7ff0b98a7700 1 -- 172.21.15.92:0/3084243020 shutdown_connections 2023-10-06T10:42:47.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.260+0000 7ff0b98a7700 1 -- 172.21.15.92:0/3084243020 wait complete. 2023-10-06T10:42:47.272 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:2 damage ls 2023-10-06T10:42:47.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.387+0000 7fddf1edd700 1 Processor -- start 2023-10-06T10:42:47.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.387+0000 7fddf1edd700 1 -- start start 2023-10-06T10:42:47.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.388+0000 7fddf1edd700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fddec140a70 0x7fddec0ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.388+0000 7fddf1edd700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddec0cc640 con 0x7fddec140e40 2023-10-06T10:42:47.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.388+0000 7fddf1edd700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddec0ccbe0 con 0x7fddec0caf40 2023-10-06T10:42:47.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.388+0000 7fddf1edd700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddec0c9df0 con 0x7fddec140a70 2023-10-06T10:42:47.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.388+0000 7fddebfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fddec140e40 0x7fddec0ca830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56920/0 (socket says 172.21.15.92:56920) 2023-10-06T10:42:47.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.388+0000 7fddebfff700 1 -- 172.21.15.92:0/3978555270 learned_addr learned my addr 172.21.15.92:0/3978555270 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:42:47.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.388+0000 7fddf0edb700 1 --2- 172.21.15.92:0/3978555270 >> v2:172.21.15.124:3300/0 conn(0x7fddec140a70 0x7fddec0ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.388+0000 7fddeb7fe700 1 -- 172.21.15.92:0/3978555270 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3109407626 0 0) 0x7fddec0cc640 con 0x7fddec140e40 2023-10-06T10:42:47.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.388+0000 7fddeb7fe700 1 -- 172.21.15.92:0/3978555270 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdddc003570 con 0x7fddec140e40 2023-10-06T10:42:47.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.389+0000 7fddf0edb700 1 -- 172.21.15.92:0/3978555270 >> v1:172.21.15.124:6789/0 conn(0x7fddec0caf40 legacy=0x7fddec0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.389+0000 7fddf0edb700 1 -- 172.21.15.92:0/3978555270 >> v1:172.21.15.92:6789/0 conn(0x7fddec140e40 legacy=0x7fddec0ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.389+0000 7fddf0edb700 1 -- 172.21.15.92:0/3978555270 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddec0cd070 con 0x7fddec140a70 2023-10-06T10:42:47.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.389+0000 7fddf0edb700 1 --2- 172.21.15.92:0/3978555270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fddec140a70 0x7fddec0ca2f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdde00098e0 tx=0x7fdde00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7ab8c6c00753885 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.389+0000 7fddeb7fe700 1 -- 172.21.15.92:0/3978555270 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde000b640 con 0x7fddec140a70 2023-10-06T10:42:47.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.390+0000 7fddeb7fe700 1 -- 172.21.15.92:0/3978555270 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde000b7a0 con 0x7fddec140a70 2023-10-06T10:42:47.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.390+0000 7fddeb7fe700 1 -- 172.21.15.92:0/3978555270 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde000bb40 con 0x7fddec140a70 2023-10-06T10:42:47.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.390+0000 7fddf1edd700 1 -- 172.21.15.92:0/3978555270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fddec140a70 msgr2=0x7fddec0ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.390+0000 7fddf1edd700 1 --2- 172.21.15.92:0/3978555270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fddec140a70 0x7fddec0ca2f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdde00098e0 tx=0x7fdde00091a0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.390+0000 7fddf1edd700 1 -- 172.21.15.92:0/3978555270 shutdown_connections 2023-10-06T10:42:47.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.390+0000 7fddf1edd700 1 --2- 172.21.15.92:0/3978555270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fddec140a70 0x7fddec0ca2f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.390+0000 7fddf1edd700 1 -- 172.21.15.92:0/3978555270 >> 172.21.15.92:0/3978555270 conn(0x7fddec06b330 msgr2=0x7fddec06b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:47.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.390+0000 7fddf1edd700 1 -- 172.21.15.92:0/3978555270 shutdown_connections 2023-10-06T10:42:47.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.390+0000 7fddf1edd700 1 -- 172.21.15.92:0/3978555270 wait complete. 2023-10-06T10:42:47.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.391+0000 7fddf1edd700 1 Processor -- start 2023-10-06T10:42:47.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.391+0000 7fddf1edd700 1 -- start start 2023-10-06T10:42:47.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.392+0000 7fddf1edd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddec0caf40 0x7fddec158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.392+0000 7fddf1edd700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fddec140a70 0x7fddec0c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.392+0000 7fddf1edd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fddec140e40 0x7fddec0c5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.392+0000 7fddf1edd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddec0c66e0 con 0x7fddec0caf40 2023-10-06T10:42:47.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.392+0000 7fddf1edd700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddec0c5ad0 con 0x7fddec140a70 2023-10-06T10:42:47.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.392+0000 7fddf1edd700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddec0c5db0 con 0x7fddec140e40 2023-10-06T10:42:47.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.392+0000 7fddf0edb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddec0caf40 0x7fddec158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.392+0000 7fddf0edb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddec0caf40 0x7fddec158c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48554/0 (socket says 172.21.15.92:48554) 2023-10-06T10:42:47.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.392+0000 7fddf0edb700 1 -- 172.21.15.92:0/3370287173 learned_addr learned my addr 172.21.15.92:0/3370287173 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:47.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.393+0000 7fddebfff700 1 --2- 172.21.15.92:0/3370287173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fddec140a70 0x7fddec0c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.393+0000 7fddf0edb700 1 -- 172.21.15.92:0/3370287173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fddec140e40 msgr2=0x7fddec0c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.393+0000 7fddf16dc700 1 --2- 172.21.15.92:0/3370287173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fddec140e40 0x7fddec0c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.393+0000 7fddf0edb700 1 --2- 172.21.15.92:0/3370287173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fddec140e40 0x7fddec0c5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.393+0000 7fddf0edb700 1 -- 172.21.15.92:0/3370287173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fddec140a70 msgr2=0x7fddec0c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.393+0000 7fddf0edb700 1 --2- 172.21.15.92:0/3370287173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fddec140a70 0x7fddec0c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.393+0000 7fddf0edb700 1 -- 172.21.15.92:0/3370287173 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde0009420 con 0x7fddec0caf40 2023-10-06T10:42:47.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.393+0000 7fddebfff700 1 --2- 172.21.15.92:0/3370287173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fddec140a70 0x7fddec0c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:47.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.393+0000 7fddf16dc700 1 --2- 172.21.15.92:0/3370287173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fddec140e40 0x7fddec0c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:47.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.393+0000 7fddf0edb700 1 --2- 172.21.15.92:0/3370287173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddec0caf40 0x7fddec158c10 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fddec0712e0 tx=0x7fdde0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.393+0000 7fdde8ff9700 1 -- 172.21.15.92:0/3370287173 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde000bb20 con 0x7fddec0caf40 2023-10-06T10:42:47.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.393+0000 7fdde8ff9700 1 -- 172.21.15.92:0/3370287173 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde0011420 con 0x7fddec0caf40 2023-10-06T10:42:47.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.393+0000 7fdde8ff9700 1 -- 172.21.15.92:0/3370287173 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde00117a0 con 0x7fddec0caf40 2023-10-06T10:42:47.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.393+0000 7fddf1edd700 1 -- 172.21.15.92:0/3370287173 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddec0c6090 con 0x7fddec0caf40 2023-10-06T10:42:47.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.393+0000 7fddf1edd700 1 -- 172.21.15.92:0/3370287173 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddec159150 con 0x7fddec0caf40 2023-10-06T10:42:47.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.394+0000 7fdde8ff9700 1 -- 172.21.15.92:0/3370287173 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdde0011b10 con 0x7fddec0caf40 2023-10-06T10:42:47.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.395+0000 7fdde8ff9700 1 --2- 172.21.15.92:0/3370287173 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fddd0066f30 0x7fddd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.395+0000 7fdde8ff9700 1 -- 172.21.15.92:0/3370287173 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fdde0085c60 con 0x7fddec0caf40 2023-10-06T10:42:47.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.395+0000 7fddebfff700 1 --2- 172.21.15.92:0/3370287173 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fddd0066f30 0x7fddd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.395+0000 7fddebfff700 1 --2- 172.21.15.92:0/3370287173 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fddd0066f30 0x7fddd00693e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fddd8007fd0 tx=0x7fddd8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.397+0000 7fddfc437b80 1 Processor -- start 2023-10-06T10:42:47.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.397+0000 7fddfc437b80 1 -- start start 2023-10-06T10:42:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.397+0000 7fddfc437b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bd5021c9a0 0x55bd5021cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.397+0000 7fddfc437b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd50266c10 0x55bd50245970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.397+0000 7fddfc437b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bd50245eb0 0x55bd502189e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.397+0000 7fddfc437b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd5019ccd0 con 0x55bd50266c10 2023-10-06T10:42:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.397+0000 7fddfc437b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd50173ac0 con 0x55bd50245eb0 2023-10-06T10:42:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.397+0000 7fddfc437b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd501712c0 con 0x55bd5021c9a0 2023-10-06T10:42:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.397+0000 7fddebfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd50266c10 0x55bd50245970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.397+0000 7fddebfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd50266c10 0x55bd50245970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48558/0 (socket says 172.21.15.92:48558) 2023-10-06T10:42:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.397+0000 7fddebfff700 1 -- 172.21.15.92:0/1913777630 learned_addr learned my addr 172.21.15.92:0/1913777630 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.398+0000 7fddf16dc700 1 --2- 172.21.15.92:0/1913777630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bd50245eb0 0x55bd502189e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.398+0000 7fddebfff700 1 -- 172.21.15.92:0/1913777630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bd5021c9a0 msgr2=0x55bd5021cd70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T10:42:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.398+0000 7fddebfff700 1 --2- 172.21.15.92:0/1913777630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bd5021c9a0 0x55bd5021cd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.398+0000 7fddebfff700 1 -- 172.21.15.92:0/1913777630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bd50245eb0 msgr2=0x55bd502189e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.398+0000 7fddebfff700 1 --2- 172.21.15.92:0/1913777630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bd50245eb0 0x55bd502189e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.398+0000 7fddebfff700 1 -- 172.21.15.92:0/1913777630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd500072c0 con 0x55bd50266c10 2023-10-06T10:42:47.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.398+0000 7fddf16dc700 1 --2- 172.21.15.92:0/1913777630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bd50245eb0 0x55bd502189e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:47.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.398+0000 7fddebfff700 1 --2- 172.21.15.92:0/1913777630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd50266c10 0x55bd50245970 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fddd8005b10 tx=0x7fddd8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b84064ba386048 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.398+0000 7fddd5ffb700 1 -- 172.21.15.92:0/1913777630 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd80076f0 con 0x55bd50266c10 2023-10-06T10:42:47.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.398+0000 7fddd5ffb700 1 -- 172.21.15.92:0/1913777630 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd800d660 con 0x55bd50266c10 2023-10-06T10:42:47.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.398+0000 7fddd5ffb700 1 -- 172.21.15.92:0/1913777630 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd800da20 con 0x55bd50266c10 2023-10-06T10:42:47.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.399+0000 7fddfc437b80 1 -- 172.21.15.92:0/1913777630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd50266c10 msgr2=0x55bd50245970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.399+0000 7fddfc437b80 1 --2- 172.21.15.92:0/1913777630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd50266c10 0x55bd50245970 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fddd8005b10 tx=0x7fddd8012000 comp rx=0 tx=0).stop 2023-10-06T10:42:47.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.399+0000 7fddfc437b80 1 -- 172.21.15.92:0/1913777630 shutdown_connections 2023-10-06T10:42:47.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.399+0000 7fddfc437b80 1 --2- 172.21.15.92:0/1913777630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bd50245eb0 0x55bd502189e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.399+0000 7fddfc437b80 1 --2- 172.21.15.92:0/1913777630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bd5021c9a0 0x55bd5021cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.399+0000 7fddfc437b80 1 --2- 172.21.15.92:0/1913777630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd50266c10 0x55bd50245970 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.399+0000 7fddfc437b80 1 -- 172.21.15.92:0/1913777630 >> 172.21.15.92:0/1913777630 conn(0x55bd5021e2c0 msgr2=0x55bd5021e690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:47.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.399+0000 7fddfc437b80 1 -- 172.21.15.92:0/1913777630 shutdown_connections 2023-10-06T10:42:47.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.399+0000 7fddfc437b80 1 -- 172.21.15.92:0/1913777630 wait complete. 2023-10-06T10:42:47.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.399+0000 7fddfc437b80 1 -- start start 2023-10-06T10:42:47.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.399+0000 7fddfc437b80 1 Processor -- start 2023-10-06T10:42:47.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.400+0000 7fddfc437b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd5021c9a0 0x55bd50273ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.400+0000 7fddfc437b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bd50245eb0 0x55bd5025cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.400+0000 7fddfc437b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bd50266c10 0x55bd5025ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.400+0000 7fddfc437b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd5016d0b0 con 0x55bd5021c9a0 2023-10-06T10:42:47.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.400+0000 7fddfc437b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd501188e0 con 0x55bd50266c10 2023-10-06T10:42:47.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.400+0000 7fddfc437b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd50133ef0 con 0x55bd50245eb0 2023-10-06T10:42:47.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.400+0000 7fddfc437b80 10 client.0 _ll_register_callbacks cb 0x55bd50009f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:42:47.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.400+0000 7fddf0edb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd5021c9a0 0x55bd50273ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.400+0000 7fddf0edb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd5021c9a0 0x55bd50273ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48574/0 (socket says 172.21.15.92:48574) 2023-10-06T10:42:47.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.400+0000 7fddf0edb700 1 -- 172.21.15.92:0/3216001118 learned_addr learned my addr 172.21.15.92:0/3216001118 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:47.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.400+0000 7fddebfff700 1 --2- 172.21.15.92:0/3216001118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bd50245eb0 0x55bd5025cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.400+0000 7fddf16dc700 1 --2- 172.21.15.92:0/3216001118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bd50266c10 0x55bd5025ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.401+0000 7fddf0edb700 1 -- 172.21.15.92:0/3216001118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bd50245eb0 msgr2=0x55bd5025cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.401+0000 7fddf0edb700 1 --2- 172.21.15.92:0/3216001118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bd50245eb0 0x55bd5025cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.401+0000 7fddf0edb700 1 -- 172.21.15.92:0/3216001118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bd50266c10 msgr2=0x55bd5025ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.401+0000 7fddf0edb700 1 --2- 172.21.15.92:0/3216001118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bd50266c10 0x55bd5025ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.401+0000 7fddf0edb700 1 -- 172.21.15.92:0/3216001118 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdddc003570 con 0x55bd5021c9a0 2023-10-06T10:42:47.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.401+0000 7fddf0edb700 1 --2- 172.21.15.92:0/3216001118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd5021c9a0 0x55bd50273ac0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fdde0052e30 tx=0x7fdde0083e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d941c55b75040cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.401+0000 7fddcaffd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:42:47.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.401+0000 7fddcaffd700 1 -- 172.21.15.92:0/3216001118 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde0056930 con 0x55bd5021c9a0 2023-10-06T10:42:47.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.401+0000 7fddcaffd700 1 -- 172.21.15.92:0/3216001118 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fdde0059b80 con 0x55bd5021c9a0 2023-10-06T10:42:47.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.401+0000 7fddfc437b80 1 -- 172.21.15.92:0/3216001118 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bd502116e0 con 0x55bd5021c9a0 2023-10-06T10:42:47.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.401+0000 7fddcaffd700 1 -- 172.21.15.92:0/3216001118 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdde005d8d0 con 0x55bd5021c9a0 2023-10-06T10:42:47.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.401+0000 7fddfc437b80 10 client.6972 fetch_fsmap learned FSMap version 59 2023-10-06T10:42:47.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.401+0000 7fddfc437b80 1 -- 172.21.15.92:0/3216001118 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x55bd4ffaa030 con 0x55bd5021c9a0 2023-10-06T10:42:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.401+0000 7fddcaffd700 1 -- 172.21.15.92:0/3216001118 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bd502116e0 con 0x55bd5021c9a0 2023-10-06T10:42:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.401+0000 7fddcaffd700 1 -- 172.21.15.92:0/3216001118 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 59) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fdde0055e20 con 0x55bd5021c9a0 2023-10-06T10:42:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.401+0000 7fddfc437b80 10 client.6972 fetch_fsmap finished waiting for FSMap version 59 2023-10-06T10:42:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.401+0000 7fddfc437b80 10 client.6972 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.b 2023-10-06T10:42:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.401+0000 7fddfc437b80 20 client.6972 populate_metadata read hostname 'smithi092' 2023-10-06T10:42:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.401+0000 7fddfc437b80 1 --2- 172.21.15.92:0/3216001118 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x55bd5025d240 0x55bd50243a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.401+0000 7fddfc437b80 4 client.6972 mds_command: new command op to 6019 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:42:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.401+0000 7fddfc437b80 1 -- 172.21.15.92:0/3216001118 --> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd501ee610 con 0x55bd5025d240 2023-10-06T10:42:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.401+0000 7fddebfff700 1 --2- 172.21.15.92:0/3216001118 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x55bd5025d240 0x55bd50243a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.402+0000 7fddebfff700 1 --2- 172.21.15.92:0/3216001118 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x55bd5025d240 0x55bd50243a10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b4942b1e04fc1b0 server_cookie=d7e192695523bced in_seq=0 out_seq=0 2023-10-06T10:42:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.402+0000 7fddcaffd700 10 client.6972 ms_handle_connect on v2:172.21.15.124:6833/2297612118 2023-10-06T10:42:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.402+0000 7fddcaffd700 1 -- 172.21.15.92:0/3216001118 <== mds.2 v2:172.21.15.124:6833/2297612118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55bd501ee610 con 0x55bd5025d240 2023-10-06T10:42:47.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.402+0000 7fddcaffd700 10 client.6972 handle_command_reply: tid=0 2023-10-06T10:42:47.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.402+0000 7fddfc437b80 1 client.6972 shutdown 2023-10-06T10:42:47.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.403+0000 7fddfc437b80 1 -- 172.21.15.92:0/3216001118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd5021c9a0 msgr2=0x55bd50273ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.403+0000 7fddfc437b80 1 --2- 172.21.15.92:0/3216001118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd5021c9a0 0x55bd50273ac0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fdde0052e30 tx=0x7fdde0083e00 comp rx=0 tx=0).stop 2023-10-06T10:42:47.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.403+0000 7fddfc437b80 1 -- 172.21.15.92:0/3216001118 shutdown_connections 2023-10-06T10:42:47.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.403+0000 7fddfc437b80 1 --2- 172.21.15.92:0/3216001118 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x55bd5025d240 0x55bd50243a10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.403+0000 7fddfc437b80 1 --2- 172.21.15.92:0/3216001118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bd50266c10 0x55bd5025ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.403+0000 7fddcaffd700 0 client.6972 ms_handle_reset on v2:172.21.15.124:6833/2297612118 2023-10-06T10:42:47.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.403+0000 7fddfc437b80 1 --2- 172.21.15.92:0/3216001118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd5021c9a0 0x55bd50273ac0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.403+0000 7fddfc437b80 1 --2- 172.21.15.92:0/3216001118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bd50245eb0 0x55bd5025cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.403+0000 7fddfc437b80 1 -- 172.21.15.92:0/3216001118 >> 172.21.15.92:0/3216001118 conn(0x55bd5021e2c0 msgr2=0x55bd5021d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:47.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.403+0000 7fddfc437b80 1 -- 172.21.15.92:0/3216001118 shutdown_connections 2023-10-06T10:42:47.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.403+0000 7fddfc437b80 1 -- 172.21.15.92:0/3216001118 wait complete. 2023-10-06T10:42:47.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.403+0000 7fddfc437b80 20 client.6972 trim_cache size 0 max 16384 2023-10-06T10:42:47.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.415+0000 7fddfc437b80 1 Processor -- start 2023-10-06T10:42:47.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.415+0000 7fddfc437b80 1 -- start start 2023-10-06T10:42:47.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.416+0000 7fddfc437b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd5021c9a0 0x55bd502840f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.416+0000 7fddfc437b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bd50245eb0 0x55bd502131f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.416+0000 7fddfc437b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bd50266c10 0x55bd50213730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.416+0000 7fddfc437b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd4ffefe80 con 0x55bd5021c9a0 2023-10-06T10:42:47.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.416+0000 7fddfc437b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd502437d0 con 0x55bd50266c10 2023-10-06T10:42:47.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.416+0000 7fddfc437b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd502708f0 con 0x55bd50245eb0 2023-10-06T10:42:47.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.416+0000 7fddf0edb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd5021c9a0 0x55bd502840f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.416+0000 7fddf0edb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd5021c9a0 0x55bd502840f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48590/0 (socket says 172.21.15.92:48590) 2023-10-06T10:42:47.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.416+0000 7fddf0edb700 1 -- 172.21.15.92:0/933653805 learned_addr learned my addr 172.21.15.92:0/933653805 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:47.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.416+0000 7fddf0edb700 1 -- 172.21.15.92:0/933653805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bd50245eb0 msgr2=0x55bd502131f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.416+0000 7fddf0edb700 1 --2- 172.21.15.92:0/933653805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bd50245eb0 0x55bd502131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.416+0000 7fddf0edb700 1 -- 172.21.15.92:0/933653805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bd50266c10 msgr2=0x55bd50213730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.416+0000 7fddf16dc700 1 --2- 172.21.15.92:0/933653805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bd50266c10 0x55bd50213730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.416+0000 7fddf0edb700 1 --2- 172.21.15.92:0/933653805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bd50266c10 0x55bd50213730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.416+0000 7fddf0edb700 1 -- 172.21.15.92:0/933653805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd5013b7d0 con 0x55bd5021c9a0 2023-10-06T10:42:47.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.416+0000 7fddf0edb700 1 --2- 172.21.15.92:0/933653805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd5021c9a0 0x55bd502840f0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fdde0006d60 tx=0x7fdde002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d47f871ab04b88 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.416+0000 7fddd67fc700 1 -- 172.21.15.92:0/933653805 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde0059b80 con 0x55bd5021c9a0 2023-10-06T10:42:47.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.417+0000 7fddd67fc700 1 -- 172.21.15.92:0/933653805 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde0056930 con 0x55bd5021c9a0 2023-10-06T10:42:47.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.417+0000 7fddd67fc700 1 -- 172.21.15.92:0/933653805 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde005d8d0 con 0x55bd5021c9a0 2023-10-06T10:42:47.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.417+0000 7fddfc437b80 1 -- 172.21.15.92:0/933653805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd5021c9a0 msgr2=0x55bd502840f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.417+0000 7fddfc437b80 1 --2- 172.21.15.92:0/933653805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd5021c9a0 0x55bd502840f0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fdde0006d60 tx=0x7fdde002e000 comp rx=0 tx=0).stop 2023-10-06T10:42:47.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.417+0000 7fddfc437b80 1 -- 172.21.15.92:0/933653805 shutdown_connections 2023-10-06T10:42:47.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.417+0000 7fddfc437b80 1 --2- 172.21.15.92:0/933653805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bd50266c10 0x55bd50213730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.417+0000 7fddfc437b80 1 --2- 172.21.15.92:0/933653805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd5021c9a0 0x55bd502840f0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.417+0000 7fddfc437b80 1 --2- 172.21.15.92:0/933653805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bd50245eb0 0x55bd502131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.417+0000 7fddfc437b80 1 -- 172.21.15.92:0/933653805 >> 172.21.15.92:0/933653805 conn(0x55bd5021e2c0 msgr2=0x55bd5021fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:47.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.417+0000 7fddfc437b80 1 -- 172.21.15.92:0/933653805 shutdown_connections 2023-10-06T10:42:47.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.417+0000 7fddfc437b80 1 -- 172.21.15.92:0/933653805 wait complete. 2023-10-06T10:42:47.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.418+0000 7fddfc437b80 1 -- start start 2023-10-06T10:42:47.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.418+0000 7fddfc437b80 1 Processor -- start 2023-10-06T10:42:47.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddfc437b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bd5021c9a0 0x55bd502131f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddfc437b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd50245eb0 0x55bd50213730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddfc437b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bd50266c10 0x55bd50259470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddfc437b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd502ce4a0 con 0x55bd50245eb0 2023-10-06T10:42:47.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddfc437b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd502ce640 con 0x55bd5021c9a0 2023-10-06T10:42:47.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddfc437b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd502ce0e0 con 0x55bd50266c10 2023-10-06T10:42:47.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddfc437b80 10 client.0 _ll_register_callbacks cb 0x55bd50009f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:42:47.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddebfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd50245eb0 0x55bd50213730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddebfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd50245eb0 0x55bd50213730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48596/0 (socket says 172.21.15.92:48596) 2023-10-06T10:42:47.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddebfff700 1 -- 172.21.15.92:0/3413619618 learned_addr learned my addr 172.21.15.92:0/3413619618 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:47.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddf0edb700 1 --2- 172.21.15.92:0/3413619618 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bd5021c9a0 0x55bd502131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddf16dc700 1 --2- 172.21.15.92:0/3413619618 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bd50266c10 0x55bd50259470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddebfff700 1 -- 172.21.15.92:0/3413619618 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bd50266c10 msgr2=0x55bd50259470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddebfff700 1 --2- 172.21.15.92:0/3413619618 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bd50266c10 0x55bd50259470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddebfff700 1 -- 172.21.15.92:0/3413619618 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bd5021c9a0 msgr2=0x55bd502131f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddebfff700 1 --2- 172.21.15.92:0/3413619618 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bd5021c9a0 0x55bd502131f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddebfff700 1 -- 172.21.15.92:0/3413619618 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd4ffaa030 con 0x55bd50245eb0 2023-10-06T10:42:47.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddebfff700 1 --2- 172.21.15.92:0/3413619618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd50245eb0 0x55bd50213730 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fddd801d000 tx=0x7fddd8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7051394a080030f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddcaffd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:42:47.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddcaffd700 1 -- 172.21.15.92:0/3413619618 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd802f0e0 con 0x55bd50245eb0 2023-10-06T10:42:47.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddcaffd700 1 -- 172.21.15.92:0/3413619618 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fddd800de10 con 0x55bd50245eb0 2023-10-06T10:42:47.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddfc437b80 1 -- 172.21.15.92:0/3413619618 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fddd800d660 con 0x55bd50245eb0 2023-10-06T10:42:47.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddcaffd700 1 -- 172.21.15.92:0/3413619618 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddd800d7d0 con 0x55bd50245eb0 2023-10-06T10:42:47.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddfc437b80 10 client.6978 fetch_fsmap learned FSMap version 59 2023-10-06T10:42:47.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddfc437b80 1 -- 172.21.15.92:0/3413619618 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x55bd501224c0 con 0x55bd50245eb0 2023-10-06T10:42:47.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.419+0000 7fddcaffd700 1 -- 172.21.15.92:0/3413619618 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd800d660 con 0x55bd50245eb0 2023-10-06T10:42:47.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.420+0000 7fddcaffd700 1 -- 172.21.15.92:0/3413619618 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 59) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fddd802f5c0 con 0x55bd50245eb0 2023-10-06T10:42:47.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.420+0000 7fddfc437b80 10 client.6978 fetch_fsmap finished waiting for FSMap version 59 2023-10-06T10:42:47.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.420+0000 7fddfc437b80 10 client.6978 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.b 2023-10-06T10:42:47.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.420+0000 7fddfc437b80 20 client.6978 populate_metadata read hostname 'smithi092' 2023-10-06T10:42:47.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.420+0000 7fddfc437b80 1 --2- 172.21.15.92:0/3413619618 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x55bd5025ab30 0x55bd502599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.420+0000 7fddfc437b80 4 client.6978 mds_command: new command op to 6019 tid=0[{"prefix": "damage ls"}] 2023-10-06T10:42:47.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.420+0000 7fddfc437b80 1 -- 172.21.15.92:0/3413619618 --> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bd5010fe50 con 0x55bd5025ab30 2023-10-06T10:42:47.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.420+0000 7fddf0edb700 1 --2- 172.21.15.92:0/3413619618 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x55bd5025ab30 0x55bd502599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.420+0000 7fddcaffd700 10 client.6978 ms_handle_connect on v2:172.21.15.124:6833/2297612118 2023-10-06T10:42:47.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.420+0000 7fddf0edb700 1 --2- 172.21.15.92:0/3413619618 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x55bd5025ab30 0x55bd502599b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3dbec31876fab137 server_cookie=cb3d382b13739e7 in_seq=0 out_seq=0 2023-10-06T10:42:47.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.420+0000 7fddcaffd700 1 -- 172.21.15.92:0/3413619618 <== mds.2 v2:172.21.15.124:6833/2297612118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+295 (crc 0 0 0) 0x55bd5010fe50 con 0x55bd5025ab30 2023-10-06T10:42:47.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.420+0000 7fddcaffd700 10 client.6978 handle_command_reply: tid=0 2023-10-06T10:42:47.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.420+0000 7fddfc437b80 1 client.6978 shutdown 2023-10-06T10:42:47.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.420+0000 7fddfc437b80 1 -- 172.21.15.92:0/3413619618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd50245eb0 msgr2=0x55bd50213730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.420+0000 7fddfc437b80 1 --2- 172.21.15.92:0/3413619618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd50245eb0 0x55bd50213730 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fddd801d000 tx=0x7fddd8016000 comp rx=0 tx=0).stop 2023-10-06T10:42:47.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.421+0000 7fddfc437b80 1 -- 172.21.15.92:0/3413619618 shutdown_connections 2023-10-06T10:42:47.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.421+0000 7fddfc437b80 1 --2- 172.21.15.92:0/3413619618 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bd50266c10 0x55bd50259470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.421+0000 7fddfc437b80 1 --2- 172.21.15.92:0/3413619618 >> [v2:172.21.15.124:6833/2297612118,v1:172.21.15.124:6835/2297612118] conn(0x55bd5025ab30 0x55bd502599b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.421+0000 7fddfc437b80 1 --2- 172.21.15.92:0/3413619618 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bd5021c9a0 0x55bd502131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.421+0000 7fddfc437b80 1 --2- 172.21.15.92:0/3413619618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd50245eb0 0x55bd50213730 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.421+0000 7fddcaffd700 0 client.6978 ms_handle_reset on v2:172.21.15.124:6833/2297612118 2023-10-06T10:42:47.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.421+0000 7fddfc437b80 1 -- 172.21.15.92:0/3413619618 >> 172.21.15.92:0/3413619618 conn(0x55bd5021e2c0 msgr2=0x55bd5025de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:47.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.421+0000 7fddfc437b80 1 -- 172.21.15.92:0/3413619618 shutdown_connections 2023-10-06T10:42:47.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.421+0000 7fddfc437b80 1 -- 172.21.15.92:0/3413619618 wait complete. 2023-10-06T10:42:47.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.421+0000 7fddfc437b80 20 client.6978 trim_cache size 0 max 16384 2023-10-06T10:42:47.429 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T10:42:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: "damage_type": "backtrace", 2023-10-06T10:42:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 773293003, 2023-10-06T10:42:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 1099511627783, 2023-10-06T10:42:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3/d4/d5/d6/d7/d8" 2023-10-06T10:42:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:42:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: "damage_type": "backtrace", 2023-10-06T10:42:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 1367203446, 2023-10-06T10:42:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 1099511627782, 2023-10-06T10:42:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3/d4/d5/d6/d7" 2023-10-06T10:42:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:47.430 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T10:42:47.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.421+0000 7fddf1edd700 1 -- 172.21.15.92:0/3370287173 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fddd0066f30 msgr2=0x7fddd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.421+0000 7fddf1edd700 1 --2- 172.21.15.92:0/3370287173 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fddd0066f30 0x7fddd00693e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fddd8007fd0 tx=0x7fddd8009040 comp rx=0 tx=0).stop 2023-10-06T10:42:47.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.421+0000 7fddf1edd700 1 -- 172.21.15.92:0/3370287173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddec0caf40 msgr2=0x7fddec158c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.421+0000 7fddf1edd700 1 --2- 172.21.15.92:0/3370287173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddec0caf40 0x7fddec158c10 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fddec0712e0 tx=0x7fdde0017000 comp rx=0 tx=0).stop 2023-10-06T10:42:47.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.421+0000 7fddf1edd700 1 -- 172.21.15.92:0/3370287173 shutdown_connections 2023-10-06T10:42:47.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.421+0000 7fddf1edd700 1 --2- 172.21.15.92:0/3370287173 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fddd0066f30 0x7fddd00693e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.421+0000 7fddf1edd700 1 --2- 172.21.15.92:0/3370287173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fddec140e40 0x7fddec0c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.421+0000 7fddf1edd700 1 --2- 172.21.15.92:0/3370287173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddec0caf40 0x7fddec158c10 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.421+0000 7fddf1edd700 1 --2- 172.21.15.92:0/3370287173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fddec140a70 0x7fddec0c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.421+0000 7fddf1edd700 1 -- 172.21.15.92:0/3370287173 >> 172.21.15.92:0/3370287173 conn(0x7fddec06b330 msgr2=0x7fddec0c3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:47.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.421+0000 7fddf1edd700 1 -- 172.21.15.92:0/3370287173 shutdown_connections 2023-10-06T10:42:47.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.421+0000 7fddf1edd700 1 -- 172.21.15.92:0/3370287173 wait complete. 2023-10-06T10:42:47.433 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T10:42:47.434 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:42:47.473 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:42:47.516 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T10:42:47.516 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T10:42:47.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:42:47.516 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T10:42:47.555 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T10:42:47.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:42:47.555 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T10:42:47.610 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:42:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.763+0000 7f20ee79e700 1 Processor -- start 2023-10-06T10:42:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.763+0000 7f20ee79e700 1 -- start start 2023-10-06T10:42:47.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.764+0000 7f20ee79e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f20e8159560 0x7f20e80aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.764+0000 7f20ee79e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e80b0c60 con 0x7f20e80b49a0 2023-10-06T10:42:47.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.764+0000 7f20ee79e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e80b0f40 con 0x7f20e80b0760 2023-10-06T10:42:47.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.764+0000 7f20ee79e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e80b1220 con 0x7f20e8159560 2023-10-06T10:42:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.764+0000 7f20e7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f20e8159560 0x7f20e80aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.764+0000 7f20e7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f20e8159560 0x7f20e80aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48608/0 (socket says 172.21.15.92:48608) 2023-10-06T10:42:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.764+0000 7f20e7fff700 1 -- 172.21.15.92:0/2532321213 learned_addr learned my addr 172.21.15.92:0/2532321213 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.764+0000 7f20e77fe700 1 --1- 172.21.15.92:0/2532321213 >> v1:172.21.15.92:6789/0 conn(0x7f20e80b49a0 0x7f20e80aaa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56936/0 (socket says 172.21.15.92:56936) 2023-10-06T10:42:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.764+0000 7f20e7fff700 1 -- 172.21.15.92:0/2532321213 >> v1:172.21.15.124:6789/0 conn(0x7f20e80b0760 legacy=0x7f20e80b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.764+0000 7f20e7fff700 1 -- 172.21.15.92:0/2532321213 >> v1:172.21.15.92:6789/0 conn(0x7f20e80b49a0 legacy=0x7f20e80aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.764+0000 7f20e7fff700 1 -- 172.21.15.92:0/2532321213 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e80b1500 con 0x7f20e8159560 2023-10-06T10:42:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.764+0000 7f20e7fff700 1 --2- 172.21.15.92:0/2532321213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20e8159560 0x7f20e80aa530 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f20d80099c0 tx=0x7f20d8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce9256c498ccd2d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.764+0000 7f20e6ffd700 1 -- 172.21.15.92:0/2532321213 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8005c10 con 0x7f20e8159560 2023-10-06T10:42:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.764+0000 7f20e6ffd700 1 -- 172.21.15.92:0/2532321213 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d8005d70 con 0x7f20e8159560 2023-10-06T10:42:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.764+0000 7f20e6ffd700 1 -- 172.21.15.92:0/2532321213 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d800b800 con 0x7f20e8159560 2023-10-06T10:42:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.765+0000 7f20ee79e700 1 -- 172.21.15.92:0/2532321213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20e8159560 msgr2=0x7f20e80aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.765+0000 7f20ee79e700 1 --2- 172.21.15.92:0/2532321213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20e8159560 0x7f20e80aa530 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f20d80099c0 tx=0x7f20d8009220 comp rx=0 tx=0).stop 2023-10-06T10:42:47.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.765+0000 7f20ee79e700 1 -- 172.21.15.92:0/2532321213 shutdown_connections 2023-10-06T10:42:47.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.765+0000 7f20ee79e700 1 --2- 172.21.15.92:0/2532321213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20e8159560 0x7f20e80aa530 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.765+0000 7f20ee79e700 1 -- 172.21.15.92:0/2532321213 >> 172.21.15.92:0/2532321213 conn(0x7f20e80997f0 msgr2=0x7f20e806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:47.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.765+0000 7f20ee79e700 1 -- 172.21.15.92:0/2532321213 shutdown_connections 2023-10-06T10:42:47.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.765+0000 7f20ee79e700 1 -- 172.21.15.92:0/2532321213 wait complete. 2023-10-06T10:42:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.766+0000 7f20ee79e700 1 Processor -- start 2023-10-06T10:42:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.766+0000 7f20ee79e700 1 -- start start 2023-10-06T10:42:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.767+0000 7f20ee79e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20e80b0760 0x7f20e80ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.767+0000 7f20ee79e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20e80b49a0 0x7f20e816a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.767+0000 7f20ee79e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20e8158120 0x7f20e80abc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.767+0000 7f20ee79e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e81585a0 con 0x7f20e80b0760 2023-10-06T10:42:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.767+0000 7f20ee79e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e80ac1c0 con 0x7f20e8158120 2023-10-06T10:42:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.767+0000 7f20ee79e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e80ae430 con 0x7f20e80b49a0 2023-10-06T10:42:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.767+0000 7f20e7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20e80b0760 0x7f20e80ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.767+0000 7f20e7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20e80b0760 0x7f20e80ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48620/0 (socket says 172.21.15.92:48620) 2023-10-06T10:42:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.767+0000 7f20e7fff700 1 -- 172.21.15.92:0/1537818483 learned_addr learned my addr 172.21.15.92:0/1537818483 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.767+0000 7f20e7fff700 1 -- 172.21.15.92:0/1537818483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20e80b49a0 msgr2=0x7f20e816a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.767+0000 7f20e7fff700 1 --2- 172.21.15.92:0/1537818483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20e80b49a0 0x7f20e816a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.767+0000 7f20e7fff700 1 -- 172.21.15.92:0/1537818483 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20e8158120 msgr2=0x7f20e80abc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.767+0000 7f20e7fff700 1 --2- 172.21.15.92:0/1537818483 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20e8158120 0x7f20e80abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.767+0000 7f20e7fff700 1 -- 172.21.15.92:0/1537818483 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d8009480 con 0x7f20e80b0760 2023-10-06T10:42:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.767+0000 7f20e7fff700 1 --2- 172.21.15.92:0/1537818483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20e80b0760 0x7f20e80ae8c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f20d8003080 tx=0x7f20d8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.768+0000 7f20cffff700 1 -- 172.21.15.92:0/1537818483 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8000d10 con 0x7f20e80b0760 2023-10-06T10:42:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.768+0000 7f20cffff700 1 -- 172.21.15.92:0/1537818483 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d8011cc0 con 0x7f20e80b0760 2023-10-06T10:42:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.768+0000 7f20ee79e700 1 -- 172.21.15.92:0/1537818483 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e80ad860 con 0x7f20e80b0760 2023-10-06T10:42:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.768+0000 7f20cffff700 1 -- 172.21.15.92:0/1537818483 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d801d680 con 0x7f20e80b0760 2023-10-06T10:42:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.768+0000 7f20ee79e700 1 -- 172.21.15.92:0/1537818483 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e80adde0 con 0x7f20e80b0760 2023-10-06T10:42:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.769+0000 7f20ee79e700 1 -- 172.21.15.92:0/1537818483 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e8064a90 con 0x7f20e80b0760 2023-10-06T10:42:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.769+0000 7f20cffff700 1 -- 172.21.15.92:0/1537818483 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f20d801da70 con 0x7f20e80b0760 2023-10-06T10:42:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.769+0000 7f20cffff700 1 --2- 172.21.15.92:0/1537818483 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f20d4066f30 0x7f20d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:47.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.769+0000 7f20cffff700 1 -- 172.21.15.92:0/1537818483 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f20d80866e0 con 0x7f20e80b0760 2023-10-06T10:42:47.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.769+0000 7f20e77fe700 1 --2- 172.21.15.92:0/1537818483 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f20d4066f30 0x7f20d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:47.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.770+0000 7f20e77fe700 1 --2- 172.21.15.92:0/1537818483 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f20d4066f30 0x7f20d40693e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f20e8099f20 tx=0x7f20dc007b30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:47.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.774+0000 7f20cffff700 1 -- 172.21.15.92:0/1537818483 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f20d8053670 con 0x7f20e80b0760 2023-10-06T10:42:47.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.997+0000 7f20ee79e700 1 -- 172.21.15.92:0/1537818483 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20e80b17e0 con 0x7f20e80b0760 2023-10-06T10:42:47.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.997+0000 7f20cffff700 1 -- 172.21.15.92:0/1537818483 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4636 (secure 0 0 0) 0x7f20d8056c90 con 0x7f20e80b0760 2023-10-06T10:42:47.998 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:42:47.998 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":59,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/1409495728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":1409495728},{"type":"v1","addr":"172.21.15.124:6834","nonce":1409495728}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:41:47.898774+0000","modified":"2023-10-06T10:42:21.332932+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6486,"mds_1":6483,"mds_2":6019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6019":{"gid":6019,"name":"b","rank":2,"incarnation":54,"state":"up:active","state_seq":10,"addr":"172.21.15.124:6835/2297612118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2297612118},{"type":"v1","addr":"172.21.15.124:6835","nonce":2297612118}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6483":{"gid":6483,"name":"a","rank":1,"incarnation":52,"state":"up:active","state_seq":11,"addr":"172.21.15.92:6836/171578176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":171578176},{"type":"v1","addr":"172.21.15.92:6836","nonce":171578176}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6486":{"gid":6486,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":11,"addr":"172.21.15.92:6837/818031030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":818031030},{"type":"v1","addr":"172.21.15.92:6837","nonce":818031030}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T10:42:48.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.999+0000 7f20ee79e700 1 -- 172.21.15.92:0/1537818483 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f20d4066f30 msgr2=0x7f20d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:48.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.999+0000 7f20ee79e700 1 --2- 172.21.15.92:0/1537818483 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f20d4066f30 0x7f20d40693e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f20e8099f20 tx=0x7f20dc007b30 comp rx=0 tx=0).stop 2023-10-06T10:42:48.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.999+0000 7f20ee79e700 1 -- 172.21.15.92:0/1537818483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20e80b0760 msgr2=0x7f20e80ae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:48.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.999+0000 7f20ee79e700 1 --2- 172.21.15.92:0/1537818483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20e80b0760 0x7f20e80ae8c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f20d8003080 tx=0x7f20d8015000 comp rx=0 tx=0).stop 2023-10-06T10:42:48.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.999+0000 7f20ee79e700 1 -- 172.21.15.92:0/1537818483 shutdown_connections 2023-10-06T10:42:48.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.999+0000 7f20ee79e700 1 --2- 172.21.15.92:0/1537818483 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f20d4066f30 0x7f20d40693e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.999+0000 7f20ee79e700 1 --2- 172.21.15.92:0/1537818483 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20e8158120 0x7f20e80abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.999+0000 7f20ee79e700 1 --2- 172.21.15.92:0/1537818483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20e80b0760 0x7f20e80ae8c0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.999+0000 7f20ee79e700 1 --2- 172.21.15.92:0/1537818483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20e80b49a0 0x7f20e816a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:47.999+0000 7f20ee79e700 1 -- 172.21.15.92:0/1537818483 >> 172.21.15.92:0/1537818483 conn(0x7f20e80997f0 msgr2=0x7f20e80683c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:48.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.000+0000 7f20ee79e700 1 -- 172.21.15.92:0/1537818483 shutdown_connections 2023-10-06T10:42:48.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.000+0000 7f20ee79e700 1 -- 172.21.15.92:0/1537818483 wait complete. 2023-10-06T10:42:48.002 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 59 2023-10-06T10:42:48.012 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:42:48.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.125+0000 7f3686a17700 1 Processor -- start 2023-10-06T10:42:48.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.125+0000 7f3686a17700 1 -- start start 2023-10-06T10:42:48.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.126+0000 7f3686a17700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3680069690 0x7f3680069a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.126+0000 7f3686a17700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36800b11c0 0x7f36800b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.126+0000 7f3686a17700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3680159f60 con 0x7f368015a3f0 2023-10-06T10:42:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.126+0000 7f3686a17700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36800b1b40 con 0x7f3680069690 2023-10-06T10:42:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.126+0000 7f3686a17700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3680159980 con 0x7f36800b11c0 2023-10-06T10:42:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.126+0000 7f3685a15700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3680069690 0x7f3680069a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.126+0000 7f3685214700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f368015a3f0 0x7f368006cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56950/0 (socket says 172.21.15.92:56950) 2023-10-06T10:42:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.126+0000 7f3685214700 1 -- 172.21.15.92:0/514694310 learned_addr learned my addr 172.21.15.92:0/514694310 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:42:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.127+0000 7f3686216700 1 --2- 172.21.15.92:0/514694310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36800b11c0 0x7f36800b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.127+0000 7f3685a15700 1 -- 172.21.15.92:0/514694310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36800b11c0 msgr2=0x7f36800b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.127+0000 7f3685a15700 1 --2- 172.21.15.92:0/514694310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36800b11c0 0x7f36800b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.127+0000 7f3684a13700 1 -- 172.21.15.92:0/514694310 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3120117584 0 0) 0x7f3680159f60 con 0x7f368015a3f0 2023-10-06T10:42:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.127+0000 7f3685a15700 1 -- 172.21.15.92:0/514694310 >> v1:172.21.15.92:6789/0 conn(0x7f368015a3f0 legacy=0x7f368006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.127+0000 7f3685a15700 1 -- 172.21.15.92:0/514694310 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368006d610 con 0x7f3680069690 2023-10-06T10:42:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.127+0000 7f3686216700 1 --2- 172.21.15.92:0/514694310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36800b11c0 0x7f36800b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.127+0000 7f3685a15700 1 --2- 172.21.15.92:0/514694310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3680069690 0x7f3680069a60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f367c0098e0 tx=0x7f367c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33eb92510df876f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.127+0000 7f3684a13700 1 -- 172.21.15.92:0/514694310 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c009e80 con 0x7f3680069690 2023-10-06T10:42:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.127+0000 7f3684a13700 1 -- 172.21.15.92:0/514694310 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367c00b860 con 0x7f3680069690 2023-10-06T10:42:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.127+0000 7f3684a13700 1 -- 172.21.15.92:0/514694310 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c00bbe0 con 0x7f3680069690 2023-10-06T10:42:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.127+0000 7f3686a17700 1 -- 172.21.15.92:0/514694310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3680069690 msgr2=0x7f3680069a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.127+0000 7f3686a17700 1 --2- 172.21.15.92:0/514694310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3680069690 0x7f3680069a60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f367c0098e0 tx=0x7f367c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.128+0000 7f3686a17700 1 -- 172.21.15.92:0/514694310 shutdown_connections 2023-10-06T10:42:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.128+0000 7f3686a17700 1 --2- 172.21.15.92:0/514694310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36800b11c0 0x7f36800b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.128+0000 7f3686a17700 1 --2- 172.21.15.92:0/514694310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3680069690 0x7f3680069a60 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.128+0000 7f3686a17700 1 -- 172.21.15.92:0/514694310 >> 172.21.15.92:0/514694310 conn(0x7f368006b500 msgr2=0x7f368006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:48.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.128+0000 7f3686a17700 1 -- 172.21.15.92:0/514694310 shutdown_connections 2023-10-06T10:42:48.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.128+0000 7f3686a17700 1 -- 172.21.15.92:0/514694310 wait complete. 2023-10-06T10:42:48.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.129+0000 7f3686a17700 1 Processor -- start 2023-10-06T10:42:48.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.129+0000 7f3686a17700 1 -- start start 2023-10-06T10:42:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.129+0000 7f3686a17700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3680069690 0x7f3680152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.129+0000 7f3686a17700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36800b11c0 0x7f368015eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.129+0000 7f3686a17700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f368015a3f0 0x7f36800a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.129+0000 7f3686a17700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36800aec70 con 0x7f368015a3f0 2023-10-06T10:42:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.129+0000 7f3686a17700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368015fab0 con 0x7f36800b11c0 2023-10-06T10:42:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.129+0000 7f3686a17700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36800ae940 con 0x7f3680069690 2023-10-06T10:42:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.130+0000 7f3686216700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f368015a3f0 0x7f36800a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.130+0000 7f3686216700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f368015a3f0 0x7f36800a5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48636/0 (socket says 172.21.15.92:48636) 2023-10-06T10:42:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.130+0000 7f3686216700 1 -- 172.21.15.92:0/1300130585 learned_addr learned my addr 172.21.15.92:0/1300130585 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.130+0000 7f3685a15700 1 --2- 172.21.15.92:0/1300130585 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3680069690 0x7f3680152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.130+0000 7f3685214700 1 --2- 172.21.15.92:0/1300130585 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36800b11c0 0x7f368015eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.130+0000 7f3686216700 1 -- 172.21.15.92:0/1300130585 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3680069690 msgr2=0x7f3680152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.130+0000 7f3686216700 1 --2- 172.21.15.92:0/1300130585 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3680069690 0x7f3680152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.130+0000 7f3686216700 1 -- 172.21.15.92:0/1300130585 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36800b11c0 msgr2=0x7f368015eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.130+0000 7f3686216700 1 --2- 172.21.15.92:0/1300130585 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36800b11c0 0x7f368015eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.130+0000 7f3686216700 1 -- 172.21.15.92:0/1300130585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367c009420 con 0x7f368015a3f0 2023-10-06T10:42:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.130+0000 7f3686216700 1 --2- 172.21.15.92:0/1300130585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f368015a3f0 0x7f36800a5e70 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f367800a380 tx=0x7f3678007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.130+0000 7f3675ffb700 1 -- 172.21.15.92:0/1300130585 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367800ae10 con 0x7f368015a3f0 2023-10-06T10:42:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.130+0000 7f3675ffb700 1 -- 172.21.15.92:0/1300130585 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367800d650 con 0x7f368015a3f0 2023-10-06T10:42:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.130+0000 7f3686a17700 1 -- 172.21.15.92:0/1300130585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368015f4f0 con 0x7f368015a3f0 2023-10-06T10:42:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.130+0000 7f3675ffb700 1 -- 172.21.15.92:0/1300130585 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367800da10 con 0x7f368015a3f0 2023-10-06T10:42:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.130+0000 7f3686a17700 1 -- 172.21.15.92:0/1300130585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3680153350 con 0x7f368015a3f0 2023-10-06T10:42:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.131+0000 7f3686a17700 1 -- 172.21.15.92:0/1300130585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36800aee20 con 0x7f368015a3f0 2023-10-06T10:42:48.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.132+0000 7f3675ffb700 1 -- 172.21.15.92:0/1300130585 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f367801e3b0 con 0x7f368015a3f0 2023-10-06T10:42:48.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.132+0000 7f3675ffb700 1 --2- 172.21.15.92:0/1300130585 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3664066f30 0x7f36640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:48.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.132+0000 7f3675ffb700 1 -- 172.21.15.92:0/1300130585 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f3678085210 con 0x7f368015a3f0 2023-10-06T10:42:48.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.132+0000 7f3685a15700 1 --2- 172.21.15.92:0/1300130585 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3664066f30 0x7f36640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:48.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.132+0000 7f3685a15700 1 --2- 172.21.15.92:0/1300130585 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3664066f30 0x7f36640693e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f367c0091e0 tx=0x7f367c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:48.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.136+0000 7f3675ffb700 1 -- 172.21.15.92:0/1300130585 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36800aee20 con 0x7f368015a3f0 2023-10-06T10:42:48.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.320+0000 7f3686a17700 1 -- 172.21.15.92:0/1300130585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f368005f230 con 0x7f368015a3f0 2023-10-06T10:42:48.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.320+0000 7f3675ffb700 1 -- 172.21.15.92:0/1300130585 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+17171 (secure 0 0 0) 0x7f36780521a0 con 0x7f368015a3f0 2023-10-06T10:42:48.321 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:42:48.322 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":66,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:42:06.597181+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:41:45.546409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-06T10:41:46.862955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.8","mappings":[{"from":6,"to":0}]},{"pgid":"11.30","mappings":[{"from":6,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:42:48.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.323+0000 7f3686a17700 1 -- 172.21.15.92:0/1300130585 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3664066f30 msgr2=0x7f36640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:48.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.323+0000 7f3686a17700 1 --2- 172.21.15.92:0/1300130585 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3664066f30 0x7f36640693e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f367c0091e0 tx=0x7f367c017000 comp rx=0 tx=0).stop 2023-10-06T10:42:48.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.323+0000 7f3686a17700 1 -- 172.21.15.92:0/1300130585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f368015a3f0 msgr2=0x7f36800a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:48.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.323+0000 7f3686a17700 1 --2- 172.21.15.92:0/1300130585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f368015a3f0 0x7f36800a5e70 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f367800a380 tx=0x7f3678007900 comp rx=0 tx=0).stop 2023-10-06T10:42:48.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.323+0000 7f3686a17700 1 -- 172.21.15.92:0/1300130585 shutdown_connections 2023-10-06T10:42:48.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.323+0000 7f3686a17700 1 --2- 172.21.15.92:0/1300130585 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3664066f30 0x7f36640693e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.323+0000 7f3686a17700 1 --2- 172.21.15.92:0/1300130585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f368015a3f0 0x7f36800a5e70 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.323+0000 7f3686a17700 1 --2- 172.21.15.92:0/1300130585 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3680069690 0x7f3680152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.323+0000 7f3686a17700 1 --2- 172.21.15.92:0/1300130585 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36800b11c0 0x7f368015eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.323+0000 7f3686a17700 1 -- 172.21.15.92:0/1300130585 >> 172.21.15.92:0/1300130585 conn(0x7f368006b500 msgr2=0x7f36800b3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:48.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.323+0000 7f3686a17700 1 -- 172.21.15.92:0/1300130585 shutdown_connections 2023-10-06T10:42:48.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.323+0000 7f3686a17700 1 -- 172.21.15.92:0/1300130585 wait complete. 2023-10-06T10:42:48.336 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T10:38:09.680559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T10:38:10.120786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T10:41:45.546409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T10:41:46.862955+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T10:42:48.337 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T10:42:48.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.453+0000 7fc54bdbf700 1 Processor -- start 2023-10-06T10:42:48.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.453+0000 7fc54bdbf700 1 -- start start 2023-10-06T10:42:48.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.454+0000 7fc54bdbf700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5441598a0 0x7fc5440ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:48.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.454+0000 7fc54bdbf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc5440cbc70 0x7fc5440ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:48.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.454+0000 7fc54bdbf700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5440cf5d0 con 0x7fc5441594d0 2023-10-06T10:42:48.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.454+0000 7fc54bdbf700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5440cf8b0 con 0x7fc5440cbc70 2023-10-06T10:42:48.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.454+0000 7fc54bdbf700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5440cfb90 con 0x7fc5441598a0 2023-10-06T10:42:48.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.454+0000 7fc54a35c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc5440cbc70 0x7fc5440ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:48.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.454+0000 7fc549b5b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc5441594d0 0x7fc5440cb560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56962/0 (socket says 172.21.15.92:56962) 2023-10-06T10:42:48.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.454+0000 7fc549b5b700 1 -- 172.21.15.92:0/2975506987 learned_addr learned my addr 172.21.15.92:0/2975506987 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:42:48.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.455+0000 7fc54935a700 1 --2- 172.21.15.92:0/2975506987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5441598a0 0x7fc5440ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:48.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.455+0000 7fc54a35c700 1 -- 172.21.15.92:0/2975506987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5441598a0 msgr2=0x7fc5440ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:48.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.455+0000 7fc54a35c700 1 --2- 172.21.15.92:0/2975506987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5441598a0 0x7fc5440ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.455+0000 7fc54a35c700 1 -- 172.21.15.92:0/2975506987 >> v1:172.21.15.92:6789/0 conn(0x7fc5441594d0 legacy=0x7fc5440cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:48.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.455+0000 7fc548b59700 1 -- 172.21.15.92:0/2975506987 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3621208639 0 0) 0x7fc5440cf5d0 con 0x7fc5441594d0 2023-10-06T10:42:48.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.455+0000 7fc54a35c700 1 -- 172.21.15.92:0/2975506987 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5440cfe70 con 0x7fc5440cbc70 2023-10-06T10:42:48.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.455+0000 7fc54935a700 1 --2- 172.21.15.92:0/2975506987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5441598a0 0x7fc5440ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:48.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.455+0000 7fc54a35c700 1 --2- 172.21.15.92:0/2975506987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5440cbc70 0x7fc5440ced20 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fc53c0098e0 tx=0x7fc53c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1644f34c386ed21c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:48.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.455+0000 7fc548b59700 1 -- 172.21.15.92:0/2975506987 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c009e80 con 0x7fc5440cbc70 2023-10-06T10:42:48.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.455+0000 7fc548b59700 1 -- 172.21.15.92:0/2975506987 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53c00b860 con 0x7fc5440cbc70 2023-10-06T10:42:48.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.455+0000 7fc548b59700 1 -- 172.21.15.92:0/2975506987 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c00bbe0 con 0x7fc5440cbc70 2023-10-06T10:42:48.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.455+0000 7fc54bdbf700 1 -- 172.21.15.92:0/2975506987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5440cbc70 msgr2=0x7fc5440ced20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:48.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.455+0000 7fc54bdbf700 1 --2- 172.21.15.92:0/2975506987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5440cbc70 0x7fc5440ced20 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fc53c0098e0 tx=0x7fc53c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.455+0000 7fc54bdbf700 1 -- 172.21.15.92:0/2975506987 shutdown_connections 2023-10-06T10:42:48.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.455+0000 7fc54bdbf700 1 --2- 172.21.15.92:0/2975506987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5440cbc70 0x7fc5440ced20 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.455+0000 7fc54bdbf700 1 --2- 172.21.15.92:0/2975506987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5441598a0 0x7fc5440ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.455+0000 7fc54bdbf700 1 -- 172.21.15.92:0/2975506987 >> 172.21.15.92:0/2975506987 conn(0x7fc544089e50 msgr2=0x7fc544099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:48.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.456+0000 7fc54bdbf700 1 -- 172.21.15.92:0/2975506987 shutdown_connections 2023-10-06T10:42:48.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.456+0000 7fc54bdbf700 1 -- 172.21.15.92:0/2975506987 wait complete. 2023-10-06T10:42:48.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.457+0000 7fc54bdbf700 1 Processor -- start 2023-10-06T10:42:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.457+0000 7fc54bdbf700 1 -- start start 2023-10-06T10:42:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.457+0000 7fc54bdbf700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5440cbc70 0x7fc54416cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.457+0000 7fc54bdbf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5441594d0 0x7fc54416a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.457+0000 7fc54bdbf700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5441598a0 0x7fc5440ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.457+0000 7fc54bdbf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5441699f0 con 0x7fc5441594d0 2023-10-06T10:42:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.457+0000 7fc54bdbf700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54416d990 con 0x7fc5441598a0 2023-10-06T10:42:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.457+0000 7fc54bdbf700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54416dc70 con 0x7fc5440cbc70 2023-10-06T10:42:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.458+0000 7fc54935a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5441594d0 0x7fc54416a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.458+0000 7fc54935a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5441594d0 0x7fc54416a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48646/0 (socket says 172.21.15.92:48646) 2023-10-06T10:42:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.458+0000 7fc54935a700 1 -- 172.21.15.92:0/2205966152 learned_addr learned my addr 172.21.15.92:0/2205966152 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.458+0000 7fc549b5b700 1 --2- 172.21.15.92:0/2205966152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5440cbc70 0x7fc54416cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:48.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.458+0000 7fc54a35c700 1 --2- 172.21.15.92:0/2205966152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5441598a0 0x7fc5440ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:48.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.458+0000 7fc54935a700 1 -- 172.21.15.92:0/2205966152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5440cbc70 msgr2=0x7fc54416cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:48.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.458+0000 7fc54935a700 1 --2- 172.21.15.92:0/2205966152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5440cbc70 0x7fc54416cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.458+0000 7fc54935a700 1 -- 172.21.15.92:0/2205966152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5441598a0 msgr2=0x7fc5440ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:48.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.458+0000 7fc54935a700 1 --2- 172.21.15.92:0/2205966152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5441598a0 0x7fc5440ccd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.458+0000 7fc54935a700 1 -- 172.21.15.92:0/2205966152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53c009420 con 0x7fc5441594d0 2023-10-06T10:42:48.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.458+0000 7fc54a35c700 1 --2- 172.21.15.92:0/2205966152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5441598a0 0x7fc5440ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:48.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.458+0000 7fc54935a700 1 --2- 172.21.15.92:0/2205966152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5441594d0 0x7fc54416a400 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c800 tx=0x7fc534008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:48.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.458+0000 7fc539ffb700 1 -- 172.21.15.92:0/2205966152 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc534011600 con 0x7fc5441594d0 2023-10-06T10:42:48.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.458+0000 7fc539ffb700 1 -- 172.21.15.92:0/2205966152 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc534011760 con 0x7fc5441594d0 2023-10-06T10:42:48.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.458+0000 7fc54bdbf700 1 -- 172.21.15.92:0/2205966152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc544169c00 con 0x7fc5441594d0 2023-10-06T10:42:48.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.458+0000 7fc539ffb700 1 -- 172.21.15.92:0/2205966152 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc534011ae0 con 0x7fc5441594d0 2023-10-06T10:42:48.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.458+0000 7fc54bdbf700 1 -- 172.21.15.92:0/2205966152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc544169f90 con 0x7fc5441594d0 2023-10-06T10:42:48.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.459+0000 7fc54bdbf700 1 -- 172.21.15.92:0/2205966152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54416a120 con 0x7fc5441594d0 2023-10-06T10:42:48.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.459+0000 7fc539ffb700 1 -- 172.21.15.92:0/2205966152 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc534010350 con 0x7fc5441594d0 2023-10-06T10:42:48.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.460+0000 7fc539ffb700 1 --2- 172.21.15.92:0/2205966152 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc528066f30 0x7fc5280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:48.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.460+0000 7fc539ffb700 1 -- 172.21.15.92:0/2205966152 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fc534084760 con 0x7fc5441594d0 2023-10-06T10:42:48.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.460+0000 7fc549b5b700 1 --2- 172.21.15.92:0/2205966152 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc528066f30 0x7fc5280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:48.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.460+0000 7fc549b5b700 1 --2- 172.21.15.92:0/2205966152 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc528066f30 0x7fc5280693e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc540001560 tx=0x7fc540009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:48.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.464+0000 7fc539ffb700 1 -- 172.21.15.92:0/2205966152 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc54416a120 con 0x7fc5441594d0 2023-10-06T10:42:48.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.645+0000 7fc54bdbf700 1 -- 172.21.15.92:0/2205966152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc54416a120 con 0x7fc5441594d0 2023-10-06T10:42:48.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.646+0000 7fc539ffb700 1 -- 172.21.15.92:0/2205966152 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v66) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc54416a120 con 0x7fc5441594d0 2023-10-06T10:42:48.647 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T10:42:48.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.648+0000 7fc54bdbf700 1 -- 172.21.15.92:0/2205966152 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc528066f30 msgr2=0x7fc5280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:48.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.648+0000 7fc54bdbf700 1 --2- 172.21.15.92:0/2205966152 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc528066f30 0x7fc5280693e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc540001560 tx=0x7fc540009040 comp rx=0 tx=0).stop 2023-10-06T10:42:48.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.648+0000 7fc54bdbf700 1 -- 172.21.15.92:0/2205966152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5441594d0 msgr2=0x7fc54416a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:48.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.648+0000 7fc54bdbf700 1 --2- 172.21.15.92:0/2205966152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5441594d0 0x7fc54416a400 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c800 tx=0x7fc534008040 comp rx=0 tx=0).stop 2023-10-06T10:42:48.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.648+0000 7fc54bdbf700 1 -- 172.21.15.92:0/2205966152 shutdown_connections 2023-10-06T10:42:48.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.648+0000 7fc54bdbf700 1 --2- 172.21.15.92:0/2205966152 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc528066f30 0x7fc5280693e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.648+0000 7fc54bdbf700 1 --2- 172.21.15.92:0/2205966152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5441598a0 0x7fc5440ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.648+0000 7fc54bdbf700 1 --2- 172.21.15.92:0/2205966152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5440cbc70 0x7fc54416cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.648+0000 7fc54bdbf700 1 --2- 172.21.15.92:0/2205966152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5441594d0 0x7fc54416a400 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.648+0000 7fc54bdbf700 1 -- 172.21.15.92:0/2205966152 >> 172.21.15.92:0/2205966152 conn(0x7fc544089e50 msgr2=0x7fc54414b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:48.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.648+0000 7fc54bdbf700 1 -- 172.21.15.92:0/2205966152 shutdown_connections 2023-10-06T10:42:48.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.648+0000 7fc54bdbf700 1 -- 172.21.15.92:0/2205966152 wait complete. 2023-10-06T10:42:48.660 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T10:42:48.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.775+0000 7f7dc307f700 1 Processor -- start 2023-10-06T10:42:48.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.775+0000 7f7dc307f700 1 -- start start 2023-10-06T10:42:48.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.776+0000 7f7dc307f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7dbc0c4450 0x7f7dbc0686c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:48.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.776+0000 7f7dc307f700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7dbc0cc740 0x7f7dbc0ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:48.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.776+0000 7f7dc307f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dbc0c9670 con 0x7f7dbc0b0820 2023-10-06T10:42:48.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.776+0000 7f7dc307f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dbc159c00 con 0x7f7dbc0c4450 2023-10-06T10:42:48.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.776+0000 7f7dc307f700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dbc0c8a60 con 0x7f7dbc0cc740 2023-10-06T10:42:48.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.776+0000 7f7dbbfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7dbc0c4450 0x7f7dbc0686c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:48.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.776+0000 7f7dbbfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7dbc0c4450 0x7f7dbc0686c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48648/0 (socket says 172.21.15.92:48648) 2023-10-06T10:42:48.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.776+0000 7f7dbbfff700 1 -- 172.21.15.92:0/2500734512 learned_addr learned my addr 172.21.15.92:0/2500734512 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:48.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.777+0000 7f7dc161c700 1 --2- 172.21.15.92:0/2500734512 >> v2:172.21.15.124:3300/0 conn(0x7f7dbc0cc740 0x7f7dbc0ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:48.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.777+0000 7f7dbbfff700 1 -- 172.21.15.92:0/2500734512 >> v2:172.21.15.124:3300/0 conn(0x7f7dbc0cc740 msgr2=0x7f7dbc0ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:48.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.777+0000 7f7dbb7fe700 1 -- 172.21.15.92:0/2500734512 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3033941694 0 0) 0x7f7dbc0c9670 con 0x7f7dbc0b0820 2023-10-06T10:42:48.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.777+0000 7f7dbbfff700 1 --2- 172.21.15.92:0/2500734512 >> v2:172.21.15.124:3300/0 conn(0x7f7dbc0cc740 0x7f7dbc0ccb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.777+0000 7f7dbbfff700 1 -- 172.21.15.92:0/2500734512 >> v1:172.21.15.92:6789/0 conn(0x7f7dbc0b0820 legacy=0x7f7dbc0cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:48.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.777+0000 7f7dbbfff700 1 -- 172.21.15.92:0/2500734512 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc0c8d40 con 0x7f7dbc0c4450 2023-10-06T10:42:48.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.777+0000 7f7dbbfff700 1 --2- 172.21.15.92:0/2500734512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dbc0c4450 0x7f7dbc0686c0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc0cddd0 tx=0x7f7db0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebab4710c51c2ad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:48.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.777+0000 7f7dbb7fe700 1 -- 172.21.15.92:0/2500734512 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db000b860 con 0x7f7dbc0c4450 2023-10-06T10:42:48.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.777+0000 7f7dbb7fe700 1 -- 172.21.15.92:0/2500734512 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db000b9c0 con 0x7f7dbc0c4450 2023-10-06T10:42:48.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.777+0000 7f7dbb7fe700 1 -- 172.21.15.92:0/2500734512 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db000bd60 con 0x7f7dbc0c4450 2023-10-06T10:42:48.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.777+0000 7f7dc307f700 1 -- 172.21.15.92:0/2500734512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dbc0c4450 msgr2=0x7f7dbc0686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:48.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.777+0000 7f7dc307f700 1 --2- 172.21.15.92:0/2500734512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dbc0c4450 0x7f7dbc0686c0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc0cddd0 tx=0x7f7db0009250 comp rx=0 tx=0).stop 2023-10-06T10:42:48.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.777+0000 7f7dc307f700 1 -- 172.21.15.92:0/2500734512 shutdown_connections 2023-10-06T10:42:48.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.777+0000 7f7dc307f700 1 --2- 172.21.15.92:0/2500734512 >> v2:172.21.15.124:3300/0 conn(0x7f7dbc0cc740 0x7f7dbc0ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.777+0000 7f7dc307f700 1 --2- 172.21.15.92:0/2500734512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dbc0c4450 0x7f7dbc0686c0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.777+0000 7f7dc307f700 1 -- 172.21.15.92:0/2500734512 >> 172.21.15.92:0/2500734512 conn(0x7f7dbc087fc0 msgr2=0x7f7dbc0ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:48.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.778+0000 7f7dc307f700 1 -- 172.21.15.92:0/2500734512 shutdown_connections 2023-10-06T10:42:48.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.778+0000 7f7dc307f700 1 -- 172.21.15.92:0/2500734512 wait complete. 2023-10-06T10:42:48.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.779+0000 7f7dc307f700 1 Processor -- start 2023-10-06T10:42:48.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.779+0000 7f7dc307f700 1 -- start start 2023-10-06T10:42:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.779+0000 7f7dc307f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dbc0b0820 0x7f7dbc0c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.779+0000 7f7dc307f700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7dbc0c4450 0x7f7dbc14e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.779+0000 7f7dc307f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7dbc0cc740 0x7f7dbc0c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.780+0000 7f7dc307f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc15b210 con 0x7f7dbc0b0820 2023-10-06T10:42:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.780+0000 7f7dc307f700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc15a670 con 0x7f7dbc0c4450 2023-10-06T10:42:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.780+0000 7f7dc307f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc0c72e0 con 0x7f7dbc0cc740 2023-10-06T10:42:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.780+0000 7f7dc0e1b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dbc0b0820 0x7f7dbc0c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.780+0000 7f7dc0e1b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dbc0b0820 0x7f7dbc0c7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48658/0 (socket says 172.21.15.92:48658) 2023-10-06T10:42:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.780+0000 7f7dc0e1b700 1 -- 172.21.15.92:0/2952278776 learned_addr learned my addr 172.21.15.92:0/2952278776 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.780+0000 7f7dbbfff700 1 --2- 172.21.15.92:0/2952278776 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7dbc0c4450 0x7f7dbc14e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.780+0000 7f7dc161c700 1 --2- 172.21.15.92:0/2952278776 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7dbc0cc740 0x7f7dbc0c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.780+0000 7f7dc0e1b700 1 -- 172.21.15.92:0/2952278776 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7dbc0cc740 msgr2=0x7f7dbc0c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.780+0000 7f7dc0e1b700 1 --2- 172.21.15.92:0/2952278776 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7dbc0cc740 0x7f7dbc0c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.780+0000 7f7dc0e1b700 1 -- 172.21.15.92:0/2952278776 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7dbc0c4450 msgr2=0x7f7dbc14e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:48.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.780+0000 7f7dc0e1b700 1 --2- 172.21.15.92:0/2952278776 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7dbc0c4450 0x7f7dbc14e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.780+0000 7f7dc0e1b700 1 -- 172.21.15.92:0/2952278776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db00094f0 con 0x7f7dbc0b0820 2023-10-06T10:42:48.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.780+0000 7f7dbbfff700 1 --2- 172.21.15.92:0/2952278776 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7dbc0c4450 0x7f7dbc14e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:48.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.780+0000 7f7dc0e1b700 1 --2- 172.21.15.92:0/2952278776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dbc0b0820 0x7f7dbc0c7a20 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00cbc0 tx=0x7f7dac0017e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:48.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.780+0000 7f7db8ff9700 1 -- 172.21.15.92:0/2952278776 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac00a7f0 con 0x7f7dbc0b0820 2023-10-06T10:42:48.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.780+0000 7f7db8ff9700 1 -- 172.21.15.92:0/2952278776 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dac00a970 con 0x7f7dbc0b0820 2023-10-06T10:42:48.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.780+0000 7f7dc307f700 1 -- 172.21.15.92:0/2952278776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dbc15a8e0 con 0x7f7dbc0b0820 2023-10-06T10:42:48.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.780+0000 7f7db8ff9700 1 -- 172.21.15.92:0/2952278776 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac00ad30 con 0x7f7dbc0b0820 2023-10-06T10:42:48.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.780+0000 7f7dc307f700 1 -- 172.21.15.92:0/2952278776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dbc15aea0 con 0x7f7dbc0b0820 2023-10-06T10:42:48.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.781+0000 7f7dc307f700 1 -- 172.21.15.92:0/2952278776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dbc064a90 con 0x7f7dbc0b0820 2023-10-06T10:42:48.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.782+0000 7f7db8ff9700 1 -- 172.21.15.92:0/2952278776 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7dac003050 con 0x7f7dbc0b0820 2023-10-06T10:42:48.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.782+0000 7f7db8ff9700 1 --2- 172.21.15.92:0/2952278776 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7da8066f30 0x7f7da80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:48.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.782+0000 7f7dbbfff700 1 --2- 172.21.15.92:0/2952278776 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7da8066f30 0x7f7da80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:48.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.782+0000 7f7db8ff9700 1 -- 172.21.15.92:0/2952278776 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f7dac084b00 con 0x7f7dbc0b0820 2023-10-06T10:42:48.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.783+0000 7f7dbbfff700 1 --2- 172.21.15.92:0/2952278776 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7da8066f30 0x7f7da80693e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc0cddd0 tx=0x7f7db0017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:48.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.786+0000 7f7db8ff9700 1 -- 172.21.15.92:0/2952278776 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7dac051a90 con 0x7f7dbc0b0820 2023-10-06T10:42:48.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.968+0000 7f7dc307f700 1 -- 172.21.15.92:0/2952278776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7dbc04bb40 con 0x7f7dbc0b0820 2023-10-06T10:42:48.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.968+0000 7f7db8ff9700 1 -- 172.21.15.92:0/2952278776 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7dac0550b0 con 0x7f7dbc0b0820 2023-10-06T10:42:48.971 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T10:42:48.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.970+0000 7f7dc307f700 1 -- 172.21.15.92:0/2952278776 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7da8066f30 msgr2=0x7f7da80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:48.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.970+0000 7f7dc307f700 1 --2- 172.21.15.92:0/2952278776 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7da8066f30 0x7f7da80693e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc0cddd0 tx=0x7f7db0017000 comp rx=0 tx=0).stop 2023-10-06T10:42:48.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.970+0000 7f7dc307f700 1 -- 172.21.15.92:0/2952278776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dbc0b0820 msgr2=0x7f7dbc0c7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:48.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.970+0000 7f7dc307f700 1 --2- 172.21.15.92:0/2952278776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dbc0b0820 0x7f7dbc0c7a20 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00cbc0 tx=0x7f7dac0017e0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.970+0000 7f7dc307f700 1 -- 172.21.15.92:0/2952278776 shutdown_connections 2023-10-06T10:42:48.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.970+0000 7f7dc307f700 1 --2- 172.21.15.92:0/2952278776 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7da8066f30 0x7f7da80693e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.970+0000 7f7dc307f700 1 --2- 172.21.15.92:0/2952278776 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7dbc0cc740 0x7f7dbc0c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.970+0000 7f7dc307f700 1 --2- 172.21.15.92:0/2952278776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dbc0b0820 0x7f7dbc0c7a20 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.970+0000 7f7dc307f700 1 --2- 172.21.15.92:0/2952278776 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7dbc0c4450 0x7f7dbc14e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:48.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.970+0000 7f7dc307f700 1 -- 172.21.15.92:0/2952278776 >> 172.21.15.92:0/2952278776 conn(0x7f7dbc087fc0 msgr2=0x7f7dbc14ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:48.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.970+0000 7f7dc307f700 1 -- 172.21.15.92:0/2952278776 shutdown_connections 2023-10-06T10:42:48.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:48.970+0000 7f7dc307f700 1 -- 172.21.15.92:0/2952278776 wait complete. 2023-10-06T10:42:48.984 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T10:42:49.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.099+0000 7f569576a700 1 Processor -- start 2023-10-06T10:42:49.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.099+0000 7f569576a700 1 -- start start 2023-10-06T10:42:49.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.100+0000 7f569576a700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5690140cc0 0x7f56900c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:49.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.100+0000 7f569576a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f569006dab0 0x7f56900695b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:49.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.100+0000 7f569576a700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5690141090 con 0x7f56900c6c20 2023-10-06T10:42:49.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.100+0000 7f569576a700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56900c7e60 con 0x7f5690140cc0 2023-10-06T10:42:49.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.100+0000 7f569576a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569006f6e0 con 0x7f569006dab0 2023-10-06T10:42:49.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.100+0000 7f568effd700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5690140cc0 0x7f56900c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:49.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.100+0000 7f568effd700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5690140cc0 0x7f56900c66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:38044/0 (socket says 172.21.15.92:38044) 2023-10-06T10:42:49.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.100+0000 7f568effd700 1 -- 172.21.15.92:0/1348338049 learned_addr learned my addr 172.21.15.92:0/1348338049 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:49.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.100+0000 7f56867fc700 1 --1- 172.21.15.92:0/1348338049 >> v1:172.21.15.124:6789/0 conn(0x7f56900c6c20 0x7f569006d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:51104/0 (socket says 172.21.15.92:51104) 2023-10-06T10:42:49.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.100+0000 7f568f7fe700 1 --2- 172.21.15.92:0/1348338049 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f569006dab0 0x7f56900695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:49.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.101+0000 7f568effd700 1 -- 172.21.15.92:0/1348338049 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f569006dab0 msgr2=0x7f56900695b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.101+0000 7f568effd700 1 --2- 172.21.15.92:0/1348338049 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f569006dab0 0x7f56900695b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.101+0000 7f568effd700 1 -- 172.21.15.92:0/1348338049 >> v1:172.21.15.124:6789/0 conn(0x7f56900c6c20 legacy=0x7f569006d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.101+0000 7f568effd700 1 -- 172.21.15.92:0/1348338049 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56900c7250 con 0x7f5690140cc0 2023-10-06T10:42:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.101+0000 7f568e7fc700 1 -- 172.21.15.92:0/1348338049 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 951420764 0 0) 0x7f5690141090 con 0x7f56900c6c20 2023-10-06T10:42:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.101+0000 7f568effd700 1 --2- 172.21.15.92:0/1348338049 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5690140cc0 0x7f56900c66e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f56800099b0 tx=0x7f5680009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8caff715b64fc891 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.101+0000 7f568e7fc700 1 -- 172.21.15.92:0/1348338049 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5680009ea0 con 0x7f5690140cc0 2023-10-06T10:42:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.101+0000 7f568e7fc700 1 -- 172.21.15.92:0/1348338049 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568000b860 con 0x7f5690140cc0 2023-10-06T10:42:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.101+0000 7f568e7fc700 1 -- 172.21.15.92:0/1348338049 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568000bc00 con 0x7f5690140cc0 2023-10-06T10:42:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.102+0000 7f569576a700 1 -- 172.21.15.92:0/1348338049 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5690140cc0 msgr2=0x7f56900c66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.102+0000 7f569576a700 1 --2- 172.21.15.92:0/1348338049 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5690140cc0 0x7f56900c66e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f56800099b0 tx=0x7f5680009250 comp rx=0 tx=0).stop 2023-10-06T10:42:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.102+0000 7f569576a700 1 -- 172.21.15.92:0/1348338049 shutdown_connections 2023-10-06T10:42:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.102+0000 7f569576a700 1 --2- 172.21.15.92:0/1348338049 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f569006dab0 0x7f56900695b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.102+0000 7f569576a700 1 --2- 172.21.15.92:0/1348338049 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5690140cc0 0x7f56900c66e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.102+0000 7f569576a700 1 -- 172.21.15.92:0/1348338049 >> 172.21.15.92:0/1348338049 conn(0x7f56900cc0a0 msgr2=0x7f56900cc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:49.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.102+0000 7f569576a700 1 -- 172.21.15.92:0/1348338049 shutdown_connections 2023-10-06T10:42:49.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.102+0000 7f569576a700 1 -- 172.21.15.92:0/1348338049 wait complete. 2023-10-06T10:42:49.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.103+0000 7f569576a700 1 Processor -- start 2023-10-06T10:42:49.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.103+0000 7f569576a700 1 -- start start 2023-10-06T10:42:49.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.104+0000 7f569576a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f569006dab0 0x7f56900a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:49.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.104+0000 7f569576a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56900c6c20 0x7f56900adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:49.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.104+0000 7f569576a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5690140cc0 0x7f569013e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:49.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.104+0000 7f569576a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56900ae170 con 0x7f56900c6c20 2023-10-06T10:42:49.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.104+0000 7f569576a700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56900c64b0 con 0x7f569006dab0 2023-10-06T10:42:49.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.104+0000 7f569576a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5690168240 con 0x7f5690140cc0 2023-10-06T10:42:49.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.104+0000 7f56867fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56900c6c20 0x7f56900adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:49.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.104+0000 7f56867fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56900c6c20 0x7f56900adc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48666/0 (socket says 172.21.15.92:48666) 2023-10-06T10:42:49.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.104+0000 7f56867fc700 1 -- 172.21.15.92:0/2548767127 learned_addr learned my addr 172.21.15.92:0/2548767127 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:49.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.104+0000 7f568effd700 1 --2- 172.21.15.92:0/2548767127 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f569006dab0 0x7f56900a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:49.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.104+0000 7f568f7fe700 1 --2- 172.21.15.92:0/2548767127 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5690140cc0 0x7f569013e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:49.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.104+0000 7f56867fc700 1 -- 172.21.15.92:0/2548767127 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5690140cc0 msgr2=0x7f569013e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:49.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.104+0000 7f56867fc700 1 --2- 172.21.15.92:0/2548767127 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5690140cc0 0x7f569013e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.104+0000 7f56867fc700 1 -- 172.21.15.92:0/2548767127 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f569006dab0 msgr2=0x7f56900a5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:49.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.104+0000 7f56867fc700 1 --2- 172.21.15.92:0/2548767127 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f569006dab0 0x7f56900a5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.104+0000 7f56867fc700 1 -- 172.21.15.92:0/2548767127 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56800094f0 con 0x7f56900c6c20 2023-10-06T10:42:49.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.104+0000 7f56867fc700 1 --2- 172.21.15.92:0/2548767127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56900c6c20 0x7f56900adc30 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f56780011e0 tx=0x7f5678007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:49.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.105+0000 7f56877fe700 1 -- 172.21.15.92:0/2548767127 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678005a70 con 0x7f56900c6c20 2023-10-06T10:42:49.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.105+0000 7f56877fe700 1 -- 172.21.15.92:0/2548767127 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5678005bd0 con 0x7f56900c6c20 2023-10-06T10:42:49.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.105+0000 7f569576a700 1 -- 172.21.15.92:0/2548767127 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56900c5930 con 0x7f56900c6c20 2023-10-06T10:42:49.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.105+0000 7f56877fe700 1 -- 172.21.15.92:0/2548767127 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56780070d0 con 0x7f56900c6c20 2023-10-06T10:42:49.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.105+0000 7f569576a700 1 -- 172.21.15.92:0/2548767127 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56900c5ef0 con 0x7f56900c6c20 2023-10-06T10:42:49.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.106+0000 7f569576a700 1 -- 172.21.15.92:0/2548767127 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5690064a90 con 0x7f56900c6c20 2023-10-06T10:42:49.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.106+0000 7f56877fe700 1 -- 172.21.15.92:0/2548767127 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5678007480 con 0x7f56900c6c20 2023-10-06T10:42:49.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.106+0000 7f56877fe700 1 --2- 172.21.15.92:0/2548767127 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5670066f30 0x7f56700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:49.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.106+0000 7f56877fe700 1 -- 172.21.15.92:0/2548767127 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f5678083cc0 con 0x7f56900c6c20 2023-10-06T10:42:49.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.106+0000 7f568effd700 1 --2- 172.21.15.92:0/2548767127 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5670066f30 0x7f56700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:49.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.107+0000 7f568effd700 1 --2- 172.21.15.92:0/2548767127 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5670066f30 0x7f56700693e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f56800099b0 tx=0x7f5680017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:49.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.110+0000 7f56877fe700 1 -- 172.21.15.92:0/2548767127 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5678050c50 con 0x7f56900c6c20 2023-10-06T10:42:49.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.293+0000 7f569576a700 1 -- 172.21.15.92:0/2548767127 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f569004bb40 con 0x7f56900c6c20 2023-10-06T10:42:49.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.293+0000 7f56877fe700 1 -- 172.21.15.92:0/2548767127 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5678017020 con 0x7f56900c6c20 2023-10-06T10:42:49.295 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T10:42:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.295+0000 7f569576a700 1 -- 172.21.15.92:0/2548767127 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5670066f30 msgr2=0x7f56700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.295+0000 7f569576a700 1 --2- 172.21.15.92:0/2548767127 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5670066f30 0x7f56700693e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f56800099b0 tx=0x7f5680017000 comp rx=0 tx=0).stop 2023-10-06T10:42:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.295+0000 7f569576a700 1 -- 172.21.15.92:0/2548767127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56900c6c20 msgr2=0x7f56900adc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.295+0000 7f569576a700 1 --2- 172.21.15.92:0/2548767127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56900c6c20 0x7f56900adc30 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f56780011e0 tx=0x7f5678007e10 comp rx=0 tx=0).stop 2023-10-06T10:42:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.295+0000 7f569576a700 1 -- 172.21.15.92:0/2548767127 shutdown_connections 2023-10-06T10:42:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.295+0000 7f569576a700 1 --2- 172.21.15.92:0/2548767127 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5670066f30 0x7f56700693e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.295+0000 7f569576a700 1 --2- 172.21.15.92:0/2548767127 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5690140cc0 0x7f569013e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.295+0000 7f569576a700 1 --2- 172.21.15.92:0/2548767127 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f569006dab0 0x7f56900a5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.295+0000 7f569576a700 1 --2- 172.21.15.92:0/2548767127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56900c6c20 0x7f56900adc30 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.295+0000 7f569576a700 1 -- 172.21.15.92:0/2548767127 >> 172.21.15.92:0/2548767127 conn(0x7f56900cc0a0 msgr2=0x7f569013f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.295+0000 7f569576a700 1 -- 172.21.15.92:0/2548767127 shutdown_connections 2023-10-06T10:42:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.295+0000 7f569576a700 1 -- 172.21.15.92:0/2548767127 wait complete. 2023-10-06T10:42:49.309 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T10:42:49.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.425+0000 7fc699d0c700 1 Processor -- start 2023-10-06T10:42:49.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.425+0000 7fc699d0c700 1 -- start start 2023-10-06T10:42:49.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.426+0000 7fc699d0c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc694068430 0x7fc69415a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:49.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.426+0000 7fc699d0c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc694067ee0 0x7fc69415a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:49.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.426+0000 7fc699d0c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc694068f90 con 0x7fc6940b1da0 2023-10-06T10:42:49.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.426+0000 7fc699d0c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc694067a50 con 0x7fc694067ee0 2023-10-06T10:42:49.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.426+0000 7fc699d0c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6940689d0 con 0x7fc694068430 2023-10-06T10:42:49.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.426+0000 7fc693fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc694067ee0 0x7fc69415a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:49.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.426+0000 7fc693fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc694067ee0 0x7fc69415a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48674/0 (socket says 172.21.15.92:48674) 2023-10-06T10:42:49.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.426+0000 7fc693fff700 1 -- 172.21.15.92:0/2614828721 learned_addr learned my addr 172.21.15.92:0/2614828721 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:49.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.426+0000 7fc6937fe700 1 --1- 172.21.15.92:0/2614828721 >> v1:172.21.15.92:6789/0 conn(0x7fc6940b1da0 0x7fc6940aedf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56984/0 (socket says 172.21.15.92:56984) 2023-10-06T10:42:49.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.426+0000 7fc692ffd700 1 --2- 172.21.15.92:0/2614828721 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc694068430 0x7fc69415a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:49.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.427+0000 7fc693fff700 1 -- 172.21.15.92:0/2614828721 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc694068430 msgr2=0x7fc69415a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:49.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.427+0000 7fc693fff700 1 --2- 172.21.15.92:0/2614828721 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc694068430 0x7fc69415a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.427+0000 7fc693fff700 1 -- 172.21.15.92:0/2614828721 >> v1:172.21.15.92:6789/0 conn(0x7fc6940b1da0 legacy=0x7fc6940aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:49.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.427+0000 7fc693fff700 1 -- 172.21.15.92:0/2614828721 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc694068cb0 con 0x7fc694067ee0 2023-10-06T10:42:49.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.427+0000 7fc6927fc700 1 -- 172.21.15.92:0/2614828721 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4139869185 0 0) 0x7fc694068f90 con 0x7fc6940b1da0 2023-10-06T10:42:49.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.427+0000 7fc693fff700 1 --2- 172.21.15.92:0/2614828721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc694067ee0 0x7fc69415a9b0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009f20 tx=0x7fc68c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b33be3eb75153e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:49.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.427+0000 7fc6927fc700 1 -- 172.21.15.92:0/2614828721 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c00e070 con 0x7fc694067ee0 2023-10-06T10:42:49.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.427+0000 7fc6927fc700 1 -- 172.21.15.92:0/2614828721 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c00ba80 con 0x7fc694067ee0 2023-10-06T10:42:49.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.427+0000 7fc6927fc700 1 -- 172.21.15.92:0/2614828721 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0092e0 con 0x7fc694067ee0 2023-10-06T10:42:49.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.427+0000 7fc699d0c700 1 -- 172.21.15.92:0/2614828721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc694067ee0 msgr2=0x7fc69415a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:49.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.427+0000 7fc699d0c700 1 --2- 172.21.15.92:0/2614828721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc694067ee0 0x7fc69415a9b0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009f20 tx=0x7fc68c009840 comp rx=0 tx=0).stop 2023-10-06T10:42:49.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.427+0000 7fc699d0c700 1 -- 172.21.15.92:0/2614828721 shutdown_connections 2023-10-06T10:42:49.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.427+0000 7fc699d0c700 1 --2- 172.21.15.92:0/2614828721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc694067ee0 0x7fc69415a9b0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.427+0000 7fc699d0c700 1 --2- 172.21.15.92:0/2614828721 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc694068430 0x7fc69415a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.427+0000 7fc699d0c700 1 -- 172.21.15.92:0/2614828721 >> 172.21.15.92:0/2614828721 conn(0x7fc6940cd390 msgr2=0x7fc6940ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:49.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.427+0000 7fc699d0c700 1 -- 172.21.15.92:0/2614828721 shutdown_connections 2023-10-06T10:42:49.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.427+0000 7fc699d0c700 1 -- 172.21.15.92:0/2614828721 wait complete. 2023-10-06T10:42:49.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.429+0000 7fc699d0c700 1 Processor -- start 2023-10-06T10:42:49.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.429+0000 7fc699d0c700 1 -- start start 2023-10-06T10:42:49.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.429+0000 7fc699d0c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc694067ee0 0x7fc694155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:49.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.429+0000 7fc699d0c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc694068430 0x7fc6940c3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:49.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.429+0000 7fc699d0c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6940b1da0 0x7fc6940c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:49.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.429+0000 7fc699d0c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6940c2fc0 con 0x7fc694068430 2023-10-06T10:42:49.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.429+0000 7fc699d0c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6940c8320 con 0x7fc694067ee0 2023-10-06T10:42:49.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.429+0000 7fc699d0c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6940c23b0 con 0x7fc6940b1da0 2023-10-06T10:42:49.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.429+0000 7fc692ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc694068430 0x7fc6940c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:49.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.430+0000 7fc692ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc694068430 0x7fc6940c3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48690/0 (socket says 172.21.15.92:48690) 2023-10-06T10:42:49.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.430+0000 7fc692ffd700 1 -- 172.21.15.92:0/1990275808 learned_addr learned my addr 172.21.15.92:0/1990275808 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:49.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.430+0000 7fc6937fe700 1 --2- 172.21.15.92:0/1990275808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc694067ee0 0x7fc694155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:49.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.430+0000 7fc693fff700 1 --2- 172.21.15.92:0/1990275808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6940b1da0 0x7fc6940c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:49.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.430+0000 7fc692ffd700 1 -- 172.21.15.92:0/1990275808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6940b1da0 msgr2=0x7fc6940c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:49.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.430+0000 7fc692ffd700 1 --2- 172.21.15.92:0/1990275808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6940b1da0 0x7fc6940c7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.430+0000 7fc692ffd700 1 -- 172.21.15.92:0/1990275808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc694067ee0 msgr2=0x7fc694155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:49.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.430+0000 7fc692ffd700 1 --2- 172.21.15.92:0/1990275808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc694067ee0 0x7fc694155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.430+0000 7fc692ffd700 1 -- 172.21.15.92:0/1990275808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68c009ae0 con 0x7fc694068430 2023-10-06T10:42:49.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.430+0000 7fc692ffd700 1 --2- 172.21.15.92:0/1990275808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc694068430 0x7fc6940c3450 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fc68800d890 tx=0x7fc688008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:49.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.430+0000 7fc67b7fe700 1 -- 172.21.15.92:0/1990275808 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6880125f0 con 0x7fc694068430 2023-10-06T10:42:49.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.430+0000 7fc67b7fe700 1 -- 172.21.15.92:0/1990275808 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc688012750 con 0x7fc694068430 2023-10-06T10:42:49.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.430+0000 7fc699d0c700 1 -- 172.21.15.92:0/1990275808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6940c2690 con 0x7fc694068430 2023-10-06T10:42:49.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.430+0000 7fc67b7fe700 1 -- 172.21.15.92:0/1990275808 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc688012af0 con 0x7fc694068430 2023-10-06T10:42:49.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.430+0000 7fc699d0c700 1 -- 172.21.15.92:0/1990275808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6940c2c50 con 0x7fc694068430 2023-10-06T10:42:49.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.431+0000 7fc699d0c700 1 -- 172.21.15.92:0/1990275808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6940c3130 con 0x7fc694068430 2023-10-06T10:42:49.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.431+0000 7fc67b7fe700 1 -- 172.21.15.92:0/1990275808 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc68801e460 con 0x7fc694068430 2023-10-06T10:42:49.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.432+0000 7fc67b7fe700 1 --2- 172.21.15.92:0/1990275808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc680066f30 0x7fc6800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:49.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.432+0000 7fc67b7fe700 1 -- 172.21.15.92:0/1990275808 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fc688084d80 con 0x7fc694068430 2023-10-06T10:42:49.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.432+0000 7fc6937fe700 1 --2- 172.21.15.92:0/1990275808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc680066f30 0x7fc6800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:49.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.432+0000 7fc6937fe700 1 --2- 172.21.15.92:0/1990275808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc680066f30 0x7fc6800693e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc684007fd0 tx=0x7fc684007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:49.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.436+0000 7fc67b7fe700 1 -- 172.21.15.92:0/1990275808 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc688051d10 con 0x7fc694068430 2023-10-06T10:42:49.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.619+0000 7fc699d0c700 1 -- 172.21.15.92:0/1990275808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc694064a90 con 0x7fc694068430 2023-10-06T10:42:49.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.620+0000 7fc67b7fe700 1 -- 172.21.15.92:0/1990275808 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc688055330 con 0x7fc694068430 2023-10-06T10:42:49.622 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T10:42:49.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.622+0000 7fc699d0c700 1 -- 172.21.15.92:0/1990275808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc680066f30 msgr2=0x7fc6800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:49.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.622+0000 7fc699d0c700 1 --2- 172.21.15.92:0/1990275808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc680066f30 0x7fc6800693e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc684007fd0 tx=0x7fc684007d50 comp rx=0 tx=0).stop 2023-10-06T10:42:49.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.622+0000 7fc699d0c700 1 -- 172.21.15.92:0/1990275808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc694068430 msgr2=0x7fc6940c3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:49.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.622+0000 7fc699d0c700 1 --2- 172.21.15.92:0/1990275808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc694068430 0x7fc6940c3450 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fc68800d890 tx=0x7fc688008040 comp rx=0 tx=0).stop 2023-10-06T10:42:49.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.622+0000 7fc699d0c700 1 -- 172.21.15.92:0/1990275808 shutdown_connections 2023-10-06T10:42:49.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.622+0000 7fc699d0c700 1 --2- 172.21.15.92:0/1990275808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc680066f30 0x7fc6800693e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.622+0000 7fc699d0c700 1 --2- 172.21.15.92:0/1990275808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6940b1da0 0x7fc6940c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.622+0000 7fc699d0c700 1 --2- 172.21.15.92:0/1990275808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc694067ee0 0x7fc694155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.622+0000 7fc699d0c700 1 --2- 172.21.15.92:0/1990275808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc694068430 0x7fc6940c3450 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.622+0000 7fc699d0c700 1 -- 172.21.15.92:0/1990275808 >> 172.21.15.92:0/1990275808 conn(0x7fc6940cd390 msgr2=0x7fc6940ca900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:49.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.622+0000 7fc699d0c700 1 -- 172.21.15.92:0/1990275808 shutdown_connections 2023-10-06T10:42:49.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.622+0000 7fc699d0c700 1 -- 172.21.15.92:0/1990275808 wait complete. 2023-10-06T10:42:49.636 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:42:49.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.747+0000 7f2ef7efb700 1 Processor -- start 2023-10-06T10:42:49.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.747+0000 7f2ef7efb700 1 -- start start 2023-10-06T10:42:49.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.748+0000 7f2ef7efb700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ef0159560 0x7f2ef00aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:49.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.748+0000 7f2ef7efb700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2ef00b0760 0x7f2ef00b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:49.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.748+0000 7f2ef7efb700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef00b0c60 con 0x7f2ef00b49a0 2023-10-06T10:42:49.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.748+0000 7f2ef7efb700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef00b0f40 con 0x7f2ef00b0760 2023-10-06T10:42:49.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.748+0000 7f2ef7efb700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00b1220 con 0x7f2ef0159560 2023-10-06T10:42:49.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.748+0000 7f2ef6498700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2ef00b0760 0x7f2ef00b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:49.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.748+0000 7f2ef5c97700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ef0159560 0x7f2ef00aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:49.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.748+0000 7f2ef5c97700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ef0159560 0x7f2ef00aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:33462/0 (socket says 172.21.15.92:33462) 2023-10-06T10:42:49.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.748+0000 7f2ef6498700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2ef00b0760 0x7f2ef00b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:38064/0 (socket says 172.21.15.92:38064) 2023-10-06T10:42:49.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.748+0000 7f2ef5496700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f2ef00b49a0 0x7f2ef00aaa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:51110/0 (socket says 172.21.15.92:51110) 2023-10-06T10:42:49.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.748+0000 7f2ef5c97700 1 -- 172.21.15.92:0/3305710654 learned_addr learned my addr 172.21.15.92:0/3305710654 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:49.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.749+0000 7f2ef6498700 1 -- 172.21.15.92:0/3305710654 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ef0159560 msgr2=0x7f2ef00aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:49.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.749+0000 7f2ef6498700 1 --2- 172.21.15.92:0/3305710654 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ef0159560 0x7f2ef00aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.749+0000 7f2ef6498700 1 -- 172.21.15.92:0/3305710654 >> v1:172.21.15.124:6789/0 conn(0x7f2ef00b49a0 legacy=0x7f2ef00aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:49.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.749+0000 7f2ef4c95700 1 -- 172.21.15.92:0/3305710654 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3149461109 0 0) 0x7f2ef00b0c60 con 0x7f2ef00b49a0 2023-10-06T10:42:49.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.749+0000 7f2ef6498700 1 -- 172.21.15.92:0/3305710654 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef00b1500 con 0x7f2ef00b0760 2023-10-06T10:42:49.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.749+0000 7f2ef6498700 1 --2- 172.21.15.92:0/3305710654 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ef00b0760 0x7f2ef00b1bc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009e60 tx=0x7f2ee8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16a028827f06d615 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:49.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.749+0000 7f2ef4c95700 1 -- 172.21.15.92:0/3305710654 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee800e070 con 0x7f2ef00b0760 2023-10-06T10:42:49.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.749+0000 7f2ef4c95700 1 -- 172.21.15.92:0/3305710654 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee800bb60 con 0x7f2ef00b0760 2023-10-06T10:42:49.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.749+0000 7f2ef4c95700 1 -- 172.21.15.92:0/3305710654 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8012430 con 0x7f2ef00b0760 2023-10-06T10:42:49.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.750+0000 7f2ef7efb700 1 -- 172.21.15.92:0/3305710654 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ef00b0760 msgr2=0x7f2ef00b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:49.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.750+0000 7f2ef7efb700 1 --2- 172.21.15.92:0/3305710654 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ef00b0760 0x7f2ef00b1bc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009e60 tx=0x7f2ee8009780 comp rx=0 tx=0).stop 2023-10-06T10:42:49.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.750+0000 7f2ef7efb700 1 -- 172.21.15.92:0/3305710654 shutdown_connections 2023-10-06T10:42:49.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.750+0000 7f2ef7efb700 1 --2- 172.21.15.92:0/3305710654 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ef00b0760 0x7f2ef00b1bc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.750+0000 7f2ef7efb700 1 --2- 172.21.15.92:0/3305710654 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ef0159560 0x7f2ef00aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.750+0000 7f2ef7efb700 1 -- 172.21.15.92:0/3305710654 >> 172.21.15.92:0/3305710654 conn(0x7f2ef00997f0 msgr2=0x7f2ef006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:49.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.750+0000 7f2ef7efb700 1 -- 172.21.15.92:0/3305710654 shutdown_connections 2023-10-06T10:42:49.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.750+0000 7f2ef7efb700 1 -- 172.21.15.92:0/3305710654 wait complete. 2023-10-06T10:42:49.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.751+0000 7f2ef7efb700 1 Processor -- start 2023-10-06T10:42:49.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.751+0000 7f2ef7efb700 1 -- start start 2023-10-06T10:42:49.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.752+0000 7f2ef7efb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ef00b0760 0x7f2ef0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:49.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.752+0000 7f2ef7efb700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ef00b49a0 0x7f2ef01694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:49.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.752+0000 7f2ef7efb700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ef0159560 0x7f2ef0169a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:49.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.752+0000 7f2ef7efb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00aed30 con 0x7f2ef00b0760 2023-10-06T10:42:49.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.752+0000 7f2ef7efb700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef0169f60 con 0x7f2ef0159560 2023-10-06T10:42:49.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.752+0000 7f2ef7efb700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef016a460 con 0x7f2ef00b49a0 2023-10-06T10:42:49.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.752+0000 7f2ef5c97700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ef00b0760 0x7f2ef0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:49.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.752+0000 7f2ef5c97700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ef00b0760 0x7f2ef0069980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48696/0 (socket says 172.21.15.92:48696) 2023-10-06T10:42:49.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.752+0000 7f2ef5c97700 1 -- 172.21.15.92:0/3242671864 learned_addr learned my addr 172.21.15.92:0/3242671864 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:49.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.752+0000 7f2ef6498700 1 --2- 172.21.15.92:0/3242671864 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ef0159560 0x7f2ef0169a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:49.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.752+0000 7f2ef5496700 1 --2- 172.21.15.92:0/3242671864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ef00b49a0 0x7f2ef01694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:49.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.752+0000 7f2ef5c97700 1 -- 172.21.15.92:0/3242671864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ef00b49a0 msgr2=0x7f2ef01694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:49.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.752+0000 7f2ef5c97700 1 --2- 172.21.15.92:0/3242671864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ef00b49a0 0x7f2ef01694e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.752+0000 7f2ef5c97700 1 -- 172.21.15.92:0/3242671864 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ef0159560 msgr2=0x7f2ef0169a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:49.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.752+0000 7f2ef5c97700 1 --2- 172.21.15.92:0/3242671864 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ef0159560 0x7f2ef0169a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.752+0000 7f2ef5c97700 1 -- 172.21.15.92:0/3242671864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee8009a20 con 0x7f2ef00b0760 2023-10-06T10:42:49.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.752+0000 7f2ef5c97700 1 --2- 172.21.15.92:0/3242671864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ef00b0760 0x7f2ef0069980 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00e9a0 tx=0x7f2eec009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:49.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.753+0000 7f2ee5ffb700 1 -- 172.21.15.92:0/3242671864 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec00c850 con 0x7f2ef00b0760 2023-10-06T10:42:49.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.753+0000 7f2ee5ffb700 1 -- 172.21.15.92:0/3242671864 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eec00c9b0 con 0x7f2ef00b0760 2023-10-06T10:42:49.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.753+0000 7f2ef7efb700 1 -- 172.21.15.92:0/3242671864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef016a180 con 0x7f2ef00b0760 2023-10-06T10:42:49.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.753+0000 7f2ee5ffb700 1 -- 172.21.15.92:0/3242671864 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec00cd70 con 0x7f2ef00b0760 2023-10-06T10:42:49.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.753+0000 7f2ef7efb700 1 -- 172.21.15.92:0/3242671864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef016cf50 con 0x7f2ef00b0760 2023-10-06T10:42:49.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.754+0000 7f2ef7efb700 1 -- 172.21.15.92:0/3242671864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef0064a90 con 0x7f2ef00b0760 2023-10-06T10:42:49.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.754+0000 7f2ee5ffb700 1 -- 172.21.15.92:0/3242671864 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2eec013680 con 0x7f2ef00b0760 2023-10-06T10:42:49.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.755+0000 7f2ee5ffb700 1 --2- 172.21.15.92:0/3242671864 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2ed4066f30 0x7f2ed40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:49.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.755+0000 7f2ef5496700 1 --2- 172.21.15.92:0/3242671864 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2ed4066f30 0x7f2ed40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:49.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.755+0000 7f2ee5ffb700 1 -- 172.21.15.92:0/3242671864 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f2eec084d80 con 0x7f2ef00b0760 2023-10-06T10:42:49.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.755+0000 7f2ef5496700 1 --2- 172.21.15.92:0/3242671864 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2ed4066f30 0x7f2ed40693e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00b0620 tx=0x7f2ee0007bf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:49.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.759+0000 7f2ee5ffb700 1 -- 172.21.15.92:0/3242671864 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2eec051d10 con 0x7f2ef00b0760 2023-10-06T10:42:49.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.984+0000 7f2ef7efb700 1 -- 172.21.15.92:0/3242671864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ef00b17e0 con 0x7f2ef00b0760 2023-10-06T10:42:49.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.984+0000 7f2ee5ffb700 1 -- 172.21.15.92:0/3242671864 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4636 (secure 0 0 0) 0x7f2eec021090 con 0x7f2ef00b0760 2023-10-06T10:42:49.986 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:42:49.986 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":59,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/1409495728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":1409495728},{"type":"v1","addr":"172.21.15.124:6834","nonce":1409495728}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:41:47.898774+0000","modified":"2023-10-06T10:42:21.332932+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6486,"mds_1":6483,"mds_2":6019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6019":{"gid":6019,"name":"b","rank":2,"incarnation":54,"state":"up:active","state_seq":10,"addr":"172.21.15.124:6835/2297612118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2297612118},{"type":"v1","addr":"172.21.15.124:6835","nonce":2297612118}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6483":{"gid":6483,"name":"a","rank":1,"incarnation":52,"state":"up:active","state_seq":11,"addr":"172.21.15.92:6836/171578176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":171578176},{"type":"v1","addr":"172.21.15.92:6836","nonce":171578176}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6486":{"gid":6486,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":11,"addr":"172.21.15.92:6837/818031030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":818031030},{"type":"v1","addr":"172.21.15.92:6837","nonce":818031030}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T10:42:49.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.986+0000 7f2ef7efb700 1 -- 172.21.15.92:0/3242671864 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2ed4066f30 msgr2=0x7f2ed40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:49.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.986+0000 7f2ef7efb700 1 --2- 172.21.15.92:0/3242671864 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2ed4066f30 0x7f2ed40693e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00b0620 tx=0x7f2ee0007bf0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.986+0000 7f2ef7efb700 1 -- 172.21.15.92:0/3242671864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ef00b0760 msgr2=0x7f2ef0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:49.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.986+0000 7f2ef7efb700 1 --2- 172.21.15.92:0/3242671864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ef00b0760 0x7f2ef0069980 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00e9a0 tx=0x7f2eec009220 comp rx=0 tx=0).stop 2023-10-06T10:42:49.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.986+0000 7f2ef7efb700 1 -- 172.21.15.92:0/3242671864 shutdown_connections 2023-10-06T10:42:49.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.986+0000 7f2ef7efb700 1 --2- 172.21.15.92:0/3242671864 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2ed4066f30 0x7f2ed40693e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.986+0000 7f2ef7efb700 1 --2- 172.21.15.92:0/3242671864 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ef0159560 0x7f2ef0169a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.986+0000 7f2ef7efb700 1 --2- 172.21.15.92:0/3242671864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ef00b0760 0x7f2ef0069980 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.986+0000 7f2ef7efb700 1 --2- 172.21.15.92:0/3242671864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ef00b49a0 0x7f2ef01694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:49.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.986+0000 7f2ef7efb700 1 -- 172.21.15.92:0/3242671864 >> 172.21.15.92:0/3242671864 conn(0x7f2ef00997f0 msgr2=0x7f2ef00cf1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:49.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.986+0000 7f2ef7efb700 1 -- 172.21.15.92:0/3242671864 shutdown_connections 2023-10-06T10:42:49.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:49.986+0000 7f2ef7efb700 1 -- 172.21.15.92:0/3242671864 wait complete. 2023-10-06T10:42:49.990 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 59 2023-10-06T10:42:50.000 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:42:50.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.111+0000 7fdf00481700 1 Processor -- start 2023-10-06T10:42:50.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.111+0000 7fdf00481700 1 -- start start 2023-10-06T10:42:50.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.112+0000 7fdf00481700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdef80b49a0 0x7fdef80aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:50.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.112+0000 7fdf00481700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdef80b0760 0x7fdef80b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:50.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.112+0000 7fdf00481700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef80b0c60 con 0x7fdef8159560 2023-10-06T10:42:50.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.112+0000 7fdf00481700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef80b0f40 con 0x7fdef80b0760 2023-10-06T10:42:50.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.112+0000 7fdf00481700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef80b1220 con 0x7fdef80b49a0 2023-10-06T10:42:50.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.112+0000 7fdefda1c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdef80b49a0 0x7fdef80aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:50.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.112+0000 7fdefea1e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdef80b0760 0x7fdef80b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:50.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.112+0000 7fdefe21d700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fdef8159560 0x7fdef80aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:51122/0 (socket says 172.21.15.92:51122) 2023-10-06T10:42:50.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.112+0000 7fdefe21d700 1 -- 172.21.15.92:0/1167312725 learned_addr learned my addr 172.21.15.92:0/1167312725 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:42:50.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.112+0000 7fdefda1c700 1 --2- 172.21.15.92:0/1167312725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdef80b49a0 0x7fdef80aac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:33490/0 (socket says 172.21.15.92:33490) 2023-10-06T10:42:50.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.113+0000 7fdefda1c700 1 -- 172.21.15.92:0/1167312725 >> v2:172.21.15.124:3300/0 conn(0x7fdef80b0760 msgr2=0x7fdef80b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:50.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.113+0000 7fdefda1c700 1 --2- 172.21.15.92:0/1167312725 >> v2:172.21.15.124:3300/0 conn(0x7fdef80b0760 0x7fdef80b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.113+0000 7fdefda1c700 1 -- 172.21.15.92:0/1167312725 >> v1:172.21.15.124:6789/0 conn(0x7fdef8159560 legacy=0x7fdef80aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:50.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.113+0000 7fdefda1c700 1 -- 172.21.15.92:0/1167312725 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef80b1500 con 0x7fdef80b49a0 2023-10-06T10:42:50.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.113+0000 7fdefda1c700 1 --2- 172.21.15.92:0/1167312725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdef80b49a0 0x7fdef80aac40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdee80098e0 tx=0x7fdee80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65bb397cb68b0415 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:50.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.113+0000 7fdefd21b700 1 -- 172.21.15.92:0/1167312725 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee800b640 con 0x7fdef80b49a0 2023-10-06T10:42:50.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.113+0000 7fdefd21b700 1 -- 172.21.15.92:0/1167312725 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee800b7a0 con 0x7fdef80b49a0 2023-10-06T10:42:50.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.113+0000 7fdefd21b700 1 -- 172.21.15.92:0/1167312725 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee800bb40 con 0x7fdef80b49a0 2023-10-06T10:42:50.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.113+0000 7fdf00481700 1 -- 172.21.15.92:0/1167312725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdef80b49a0 msgr2=0x7fdef80aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:50.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.113+0000 7fdf00481700 1 --2- 172.21.15.92:0/1167312725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdef80b49a0 0x7fdef80aac40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdee80098e0 tx=0x7fdee80091a0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.114+0000 7fdf00481700 1 -- 172.21.15.92:0/1167312725 shutdown_connections 2023-10-06T10:42:50.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.114+0000 7fdf00481700 1 --2- 172.21.15.92:0/1167312725 >> v2:172.21.15.124:3300/0 conn(0x7fdef80b0760 0x7fdef80b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.114+0000 7fdf00481700 1 --2- 172.21.15.92:0/1167312725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdef80b49a0 0x7fdef80aac40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.114+0000 7fdf00481700 1 -- 172.21.15.92:0/1167312725 >> 172.21.15.92:0/1167312725 conn(0x7fdef80997f0 msgr2=0x7fdef806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:50.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.114+0000 7fdf00481700 1 -- 172.21.15.92:0/1167312725 shutdown_connections 2023-10-06T10:42:50.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.114+0000 7fdf00481700 1 -- 172.21.15.92:0/1167312725 wait complete. 2023-10-06T10:42:50.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.115+0000 7fdf00481700 1 Processor -- start 2023-10-06T10:42:50.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.115+0000 7fdf00481700 1 -- start start 2023-10-06T10:42:50.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.115+0000 7fdf00481700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdef80b0760 0x7fdef8157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:50.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.115+0000 7fdf00481700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdef80b49a0 0x7fdef816a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:50.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.115+0000 7fdf00481700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdef8159560 0x7fdef80ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:50.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.115+0000 7fdf00481700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef80ac260 con 0x7fdef80b49a0 2023-10-06T10:42:50.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.115+0000 7fdf00481700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef809acb0 con 0x7fdef80b0760 2023-10-06T10:42:50.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.115+0000 7fdf00481700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef8157e50 con 0x7fdef8159560 2023-10-06T10:42:50.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.115+0000 7fdefda1c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdef80b49a0 0x7fdef816a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:50.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.115+0000 7fdefda1c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdef80b49a0 0x7fdef816a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48712/0 (socket says 172.21.15.92:48712) 2023-10-06T10:42:50.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.116+0000 7fdefda1c700 1 -- 172.21.15.92:0/257556308 learned_addr learned my addr 172.21.15.92:0/257556308 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:50.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.116+0000 7fdefe21d700 1 --2- 172.21.15.92:0/257556308 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdef80b0760 0x7fdef8157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:50.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.116+0000 7fdefea1e700 1 --2- 172.21.15.92:0/257556308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdef8159560 0x7fdef80ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:50.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.116+0000 7fdefda1c700 1 -- 172.21.15.92:0/257556308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdef8159560 msgr2=0x7fdef80ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:50.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.116+0000 7fdefda1c700 1 --2- 172.21.15.92:0/257556308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdef8159560 0x7fdef80ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.116+0000 7fdefda1c700 1 -- 172.21.15.92:0/257556308 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdef80b0760 msgr2=0x7fdef8157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:50.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.116+0000 7fdefda1c700 1 --2- 172.21.15.92:0/257556308 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdef80b0760 0x7fdef8157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.116+0000 7fdefda1c700 1 -- 172.21.15.92:0/257556308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee8009420 con 0x7fdef80b49a0 2023-10-06T10:42:50.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.116+0000 7fdefda1c700 1 --2- 172.21.15.92:0/257556308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdef80b49a0 0x7fdef816a290 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fdef80b1360 tx=0x7fdee8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:50.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.116+0000 7fdeee7fc700 1 -- 172.21.15.92:0/257556308 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee801bc40 con 0x7fdef80b49a0 2023-10-06T10:42:50.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.116+0000 7fdeee7fc700 1 -- 172.21.15.92:0/257556308 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee801bde0 con 0x7fdef80b49a0 2023-10-06T10:42:50.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.116+0000 7fdeee7fc700 1 -- 172.21.15.92:0/257556308 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8011670 con 0x7fdef80b49a0 2023-10-06T10:42:50.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.116+0000 7fdf00481700 1 -- 172.21.15.92:0/257556308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef809c530 con 0x7fdef80b49a0 2023-10-06T10:42:50.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.116+0000 7fdf00481700 1 -- 172.21.15.92:0/257556308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef80aba40 con 0x7fdef80b49a0 2023-10-06T10:42:50.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.117+0000 7fdf00481700 1 -- 172.21.15.92:0/257556308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef8064a90 con 0x7fdef80b49a0 2023-10-06T10:42:50.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.117+0000 7fdeee7fc700 1 -- 172.21.15.92:0/257556308 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdee80119e0 con 0x7fdef80b49a0 2023-10-06T10:42:50.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.118+0000 7fdeee7fc700 1 --2- 172.21.15.92:0/257556308 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdedc066f30 0x7fdedc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:50.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.118+0000 7fdeee7fc700 1 -- 172.21.15.92:0/257556308 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fdee8085e50 con 0x7fdef80b49a0 2023-10-06T10:42:50.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.118+0000 7fdefe21d700 1 --2- 172.21.15.92:0/257556308 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdedc066f30 0x7fdedc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:50.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.118+0000 7fdefe21d700 1 --2- 172.21.15.92:0/257556308 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdedc066f30 0x7fdedc0693e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdef81587d0 tx=0x7fdef4001960 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:50.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.122+0000 7fdeee7fc700 1 -- 172.21.15.92:0/257556308 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdee8052de0 con 0x7fdef80b49a0 2023-10-06T10:42:50.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.306+0000 7fdf00481700 1 -- 172.21.15.92:0/257556308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdef809c6c0 con 0x7fdef80b49a0 2023-10-06T10:42:50.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.307+0000 7fdeee7fc700 1 -- 172.21.15.92:0/257556308 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+17171 (secure 0 0 0) 0x7fdee8056400 con 0x7fdef80b49a0 2023-10-06T10:42:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:42:50.309 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":66,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:42:06.597181+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:41:45.546409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-06T10:41:46.862955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.8","mappings":[{"from":6,"to":0}]},{"pgid":"11.30","mappings":[{"from":6,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:42:50.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.309+0000 7fdf00481700 1 -- 172.21.15.92:0/257556308 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdedc066f30 msgr2=0x7fdedc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:50.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.309+0000 7fdf00481700 1 --2- 172.21.15.92:0/257556308 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdedc066f30 0x7fdedc0693e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdef81587d0 tx=0x7fdef4001960 comp rx=0 tx=0).stop 2023-10-06T10:42:50.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.309+0000 7fdf00481700 1 -- 172.21.15.92:0/257556308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdef80b49a0 msgr2=0x7fdef816a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:50.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.309+0000 7fdf00481700 1 --2- 172.21.15.92:0/257556308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdef80b49a0 0x7fdef816a290 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fdef80b1360 tx=0x7fdee8017000 comp rx=0 tx=0).stop 2023-10-06T10:42:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.309+0000 7fdf00481700 1 -- 172.21.15.92:0/257556308 shutdown_connections 2023-10-06T10:42:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.309+0000 7fdf00481700 1 --2- 172.21.15.92:0/257556308 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdedc066f30 0x7fdedc0693e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.309+0000 7fdf00481700 1 --2- 172.21.15.92:0/257556308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdef8159560 0x7fdef80ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.309+0000 7fdf00481700 1 --2- 172.21.15.92:0/257556308 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdef80b0760 0x7fdef8157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.309+0000 7fdf00481700 1 --2- 172.21.15.92:0/257556308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdef80b49a0 0x7fdef816a290 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.309+0000 7fdf00481700 1 -- 172.21.15.92:0/257556308 >> 172.21.15.92:0/257556308 conn(0x7fdef80997f0 msgr2=0x7fdef80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:50.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.309+0000 7fdf00481700 1 -- 172.21.15.92:0/257556308 shutdown_connections 2023-10-06T10:42:50.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.309+0000 7fdf00481700 1 -- 172.21.15.92:0/257556308 wait complete. 2023-10-06T10:42:50.323 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T10:42:50.323 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:42:50.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.436+0000 7f417fd35700 1 Processor -- start 2023-10-06T10:42:50.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.437+0000 7f417fd35700 1 -- start start 2023-10-06T10:42:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.437+0000 7f417fd35700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4178156350 0x7f41780cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.437+0000 7f417fd35700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f41780cc710 0x7f417806bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.437+0000 7f417fd35700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41780ccb80 con 0x7f417806c160 2023-10-06T10:42:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.437+0000 7f417fd35700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417806c9a0 con 0x7f41780cc710 2023-10-06T10:42:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.437+0000 7f417fd35700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4178156b20 con 0x7f4178156350 2023-10-06T10:42:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.438+0000 7f417d2d0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f41780cc710 0x7f417806bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.438+0000 7f417d2d0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f41780cc710 0x7f417806bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48724/0 (socket says 172.21.15.92:48724) 2023-10-06T10:42:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.438+0000 7f417d2d0700 1 -- 172.21.15.92:0/4179575413 learned_addr learned my addr 172.21.15.92:0/4179575413 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.438+0000 7f417dad1700 1 --2- 172.21.15.92:0/4179575413 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4178156350 0x7f41780cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.438+0000 7f417d2d0700 1 -- 172.21.15.92:0/4179575413 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4178156350 msgr2=0x7f41780cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:50.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.438+0000 7f417d2d0700 1 --2- 172.21.15.92:0/4179575413 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4178156350 0x7f41780cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.438+0000 7f417d2d0700 1 -- 172.21.15.92:0/4179575413 >> v1:172.21.15.124:6789/0 conn(0x7f417806c160 legacy=0x7f41780cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:50.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.438+0000 7f417d2d0700 1 -- 172.21.15.92:0/4179575413 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4178156e00 con 0x7f41780cc710 2023-10-06T10:42:50.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.438+0000 7f417d2d0700 1 --2- 172.21.15.92:0/4179575413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41780cc710 0x7f417806bc20 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f41680099b0 tx=0x7f4168009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418204f0f81dc8cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:50.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.438+0000 7f417cacf700 1 -- 172.21.15.92:0/4179575413 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4168009ea0 con 0x7f41780cc710 2023-10-06T10:42:50.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.438+0000 7f417cacf700 1 -- 172.21.15.92:0/4179575413 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416800b860 con 0x7f41780cc710 2023-10-06T10:42:50.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.438+0000 7f417cacf700 1 -- 172.21.15.92:0/4179575413 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416800bc00 con 0x7f41780cc710 2023-10-06T10:42:50.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.439+0000 7f417fd35700 1 -- 172.21.15.92:0/4179575413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41780cc710 msgr2=0x7f417806bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:50.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.439+0000 7f417fd35700 1 --2- 172.21.15.92:0/4179575413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41780cc710 0x7f417806bc20 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f41680099b0 tx=0x7f4168009250 comp rx=0 tx=0).stop 2023-10-06T10:42:50.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.439+0000 7f417fd35700 1 -- 172.21.15.92:0/4179575413 shutdown_connections 2023-10-06T10:42:50.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.439+0000 7f417fd35700 1 --2- 172.21.15.92:0/4179575413 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4178156350 0x7f41780cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.439+0000 7f417fd35700 1 --2- 172.21.15.92:0/4179575413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41780cc710 0x7f417806bc20 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.439+0000 7f417fd35700 1 -- 172.21.15.92:0/4179575413 >> 172.21.15.92:0/4179575413 conn(0x7f41780cdeb0 msgr2=0x7f41780c9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:50.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.439+0000 7f417fd35700 1 -- 172.21.15.92:0/4179575413 shutdown_connections 2023-10-06T10:42:50.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.439+0000 7f417fd35700 1 -- 172.21.15.92:0/4179575413 wait complete. 2023-10-06T10:42:50.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.440+0000 7f417fd35700 1 Processor -- start 2023-10-06T10:42:50.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.440+0000 7f417fd35700 1 -- start start 2023-10-06T10:42:50.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f417fd35700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f417806c160 0x7f417814ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:50.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f417fd35700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41780cc710 0x7f4178150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:50.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f417fd35700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4178156350 0x7f41780b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:50.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f417fd35700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417814b3b0 con 0x7f4178156350 2023-10-06T10:42:50.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f417fd35700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41781521c0 con 0x7f417806c160 2023-10-06T10:42:50.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f417fd35700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41781515b0 con 0x7f41780cc710 2023-10-06T10:42:50.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f417e2d2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4178156350 0x7f41780b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:50.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f417e2d2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4178156350 0x7f41780b2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48740/0 (socket says 172.21.15.92:48740) 2023-10-06T10:42:50.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f417e2d2700 1 -- 172.21.15.92:0/620766253 learned_addr learned my addr 172.21.15.92:0/620766253 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:50.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f417d2d0700 1 --2- 172.21.15.92:0/620766253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41780cc710 0x7f4178150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:50.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f417dad1700 1 --2- 172.21.15.92:0/620766253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f417806c160 0x7f417814ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:50.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f417e2d2700 1 -- 172.21.15.92:0/620766253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41780cc710 msgr2=0x7f4178150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:50.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f417e2d2700 1 --2- 172.21.15.92:0/620766253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41780cc710 0x7f4178150d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f417e2d2700 1 -- 172.21.15.92:0/620766253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f417806c160 msgr2=0x7f417814ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:50.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f417e2d2700 1 --2- 172.21.15.92:0/620766253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f417806c160 0x7f417814ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f417e2d2700 1 -- 172.21.15.92:0/620766253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41680094f0 con 0x7f4178156350 2023-10-06T10:42:50.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f417d2d0700 1 --2- 172.21.15.92:0/620766253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41780cc710 0x7f4178150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:50.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f417dad1700 1 --2- 172.21.15.92:0/620766253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f417806c160 0x7f417814ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:50.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f417e2d2700 1 --2- 172.21.15.92:0/620766253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4178156350 0x7f41780b2420 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f417000c8e0 tx=0x7f4170001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:50.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f416dffb700 1 -- 172.21.15.92:0/620766253 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417000ce80 con 0x7f4178156350 2023-10-06T10:42:50.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f416dffb700 1 -- 172.21.15.92:0/620766253 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417000a7e0 con 0x7f4178156350 2023-10-06T10:42:50.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f417fd35700 1 -- 172.21.15.92:0/620766253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4178151890 con 0x7f4178156350 2023-10-06T10:42:50.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f416dffb700 1 -- 172.21.15.92:0/620766253 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417000ab80 con 0x7f4178156350 2023-10-06T10:42:50.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.441+0000 7f417fd35700 1 -- 172.21.15.92:0/620766253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4178151e50 con 0x7f4178156350 2023-10-06T10:42:50.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.442+0000 7f417fd35700 1 -- 172.21.15.92:0/620766253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4178152330 con 0x7f4178156350 2023-10-06T10:42:50.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.443+0000 7f416dffb700 1 -- 172.21.15.92:0/620766253 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4170012420 con 0x7f4178156350 2023-10-06T10:42:50.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.443+0000 7f416dffb700 1 --2- 172.21.15.92:0/620766253 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f415c066f30 0x7f415c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:50.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.443+0000 7f417dad1700 1 --2- 172.21.15.92:0/620766253 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f415c066f30 0x7f415c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:50.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.443+0000 7f416dffb700 1 -- 172.21.15.92:0/620766253 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f4170084b00 con 0x7f4178156350 2023-10-06T10:42:50.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.444+0000 7f417dad1700 1 --2- 172.21.15.92:0/620766253 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f415c066f30 0x7f415c0693e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4178152900 tx=0x7f4174008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:50.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.447+0000 7f416dffb700 1 -- 172.21.15.92:0/620766253 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4170051a90 con 0x7f4178156350 2023-10-06T10:42:50.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.674+0000 7f417fd35700 1 -- 172.21.15.92:0/620766253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4178064a90 con 0x7f4178156350 2023-10-06T10:42:50.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.674+0000 7f416dffb700 1 -- 172.21.15.92:0/620766253 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4636 (secure 0 0 0) 0x7f41700550b0 con 0x7f4178156350 2023-10-06T10:42:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:42:50.676 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":59,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/1409495728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":1409495728},{"type":"v1","addr":"172.21.15.124:6834","nonce":1409495728}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:41:47.898774+0000","modified":"2023-10-06T10:42:21.332932+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6486,"mds_1":6483,"mds_2":6019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6019":{"gid":6019,"name":"b","rank":2,"incarnation":54,"state":"up:active","state_seq":10,"addr":"172.21.15.124:6835/2297612118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2297612118},{"type":"v1","addr":"172.21.15.124:6835","nonce":2297612118}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6483":{"gid":6483,"name":"a","rank":1,"incarnation":52,"state":"up:active","state_seq":11,"addr":"172.21.15.92:6836/171578176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":171578176},{"type":"v1","addr":"172.21.15.92:6836","nonce":171578176}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6486":{"gid":6486,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":11,"addr":"172.21.15.92:6837/818031030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":818031030},{"type":"v1","addr":"172.21.15.92:6837","nonce":818031030}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T10:42:50.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.676+0000 7f417fd35700 1 -- 172.21.15.92:0/620766253 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f415c066f30 msgr2=0x7f415c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:50.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.676+0000 7f417fd35700 1 --2- 172.21.15.92:0/620766253 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f415c066f30 0x7f415c0693e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4178152900 tx=0x7f4174008040 comp rx=0 tx=0).stop 2023-10-06T10:42:50.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.676+0000 7f417fd35700 1 -- 172.21.15.92:0/620766253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4178156350 msgr2=0x7f41780b2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:50.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.676+0000 7f417fd35700 1 --2- 172.21.15.92:0/620766253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4178156350 0x7f41780b2420 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f417000c8e0 tx=0x7f4170001710 comp rx=0 tx=0).stop 2023-10-06T10:42:50.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.677+0000 7f417fd35700 1 -- 172.21.15.92:0/620766253 shutdown_connections 2023-10-06T10:42:50.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.677+0000 7f417fd35700 1 --2- 172.21.15.92:0/620766253 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f415c066f30 0x7f415c0693e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.677+0000 7f417fd35700 1 --2- 172.21.15.92:0/620766253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4178156350 0x7f41780b2420 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.677+0000 7f417fd35700 1 --2- 172.21.15.92:0/620766253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f417806c160 0x7f417814ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.677+0000 7f417fd35700 1 --2- 172.21.15.92:0/620766253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41780cc710 0x7f4178150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.677+0000 7f417fd35700 1 -- 172.21.15.92:0/620766253 >> 172.21.15.92:0/620766253 conn(0x7f41780cdeb0 msgr2=0x7f41780c2180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:50.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.677+0000 7f417fd35700 1 -- 172.21.15.92:0/620766253 shutdown_connections 2023-10-06T10:42:50.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.677+0000 7f417fd35700 1 -- 172.21.15.92:0/620766253 wait complete. 2023-10-06T10:42:50.680 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 59 2023-10-06T10:42:50.690 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:42:50.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.802+0000 7fded577c700 1 Processor -- start 2023-10-06T10:42:50.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.802+0000 7fded577c700 1 -- start start 2023-10-06T10:42:50.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.803+0000 7fded577c700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fded00b2810 0x7fded00cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:50.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.803+0000 7fded577c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded00cb7a0 con 0x7fded0159440 2023-10-06T10:42:50.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.803+0000 7fded577c700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded00b0860 con 0x7fded00cbc30 2023-10-06T10:42:50.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.803+0000 7fded577c700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded00cb150 con 0x7fded00b2810 2023-10-06T10:42:50.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.803+0000 7fdecffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fded0159440 0x7fded00b3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56998/0 (socket says 172.21.15.92:56998) 2023-10-06T10:42:50.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.803+0000 7fdecffff700 1 -- 172.21.15.92:0/2245009598 learned_addr learned my addr 172.21.15.92:0/2245009598 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:42:50.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.803+0000 7fded4f7b700 1 --2- 172.21.15.92:0/2245009598 >> v2:172.21.15.124:3300/0 conn(0x7fded00b2810 0x7fded00cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:50.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.803+0000 7fdeceffd700 1 -- 172.21.15.92:0/2245009598 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137073193 0 0) 0x7fded00cb7a0 con 0x7fded0159440 2023-10-06T10:42:50.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.803+0000 7fdeceffd700 1 -- 172.21.15.92:0/2245009598 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdeb8003570 con 0x7fded0159440 2023-10-06T10:42:50.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.803+0000 7fdeceffd700 1 -- 172.21.15.92:0/2245009598 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 335748851 0 0) 0x7fdeb8003570 con 0x7fded0159440 2023-10-06T10:42:50.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdeceffd700 1 -- 172.21.15.92:0/2245009598 >> v2:172.21.15.124:3300/0 conn(0x7fded00b2810 msgr2=0x7fded00cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:50.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdeceffd700 1 --2- 172.21.15.92:0/2245009598 >> v2:172.21.15.124:3300/0 conn(0x7fded00b2810 0x7fded00cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdeceffd700 1 -- 172.21.15.92:0/2245009598 >> v1:172.21.15.124:6789/0 conn(0x7fded00cbc30 legacy=0x7fded00b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:50.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdeceffd700 1 -- 172.21.15.92:0/2245009598 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded00cb430 con 0x7fded0159440 2023-10-06T10:42:50.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdeceffd700 1 -- 172.21.15.92:0/2245009598 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fdec4001470 con 0x7fded0159440 2023-10-06T10:42:50.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fded4f7b700 1 --2- 172.21.15.92:0/2245009598 >> v2:172.21.15.124:3300/0 conn(0x7fded00b2810 0x7fded00cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:42:50.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdeceffd700 1 -- 172.21.15.92:0/2245009598 >> v1:172.21.15.92:6789/0 conn(0x7fded0159440 legacy=0x7fded00b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:50.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdeceffd700 1 --2- 172.21.15.92:0/2245009598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdeb8003a60 0x7fdeb8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:50.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdeceffd700 1 --2- 172.21.15.92:0/2245009598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdeb8004810 0x7fdeb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:50.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdeceffd700 1 --2- 172.21.15.92:0/2245009598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdeb80051c0 0x7fdeb800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:50.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdeceffd700 1 -- 172.21.15.92:0/2245009598 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded00cb150 con 0x7fdeb8003a60 2023-10-06T10:42:50.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdeceffd700 1 -- 172.21.15.92:0/2245009598 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb800c0d0 con 0x7fdeb8004810 2023-10-06T10:42:50.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdeceffd700 1 -- 172.21.15.92:0/2245009598 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb800c290 con 0x7fdeb80051c0 2023-10-06T10:42:50.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdecffff700 1 --2- 172.21.15.92:0/2245009598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdeb8003a60 0x7fdeb8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:50.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdecf7fe700 1 --2- 172.21.15.92:0/2245009598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdeb8004810 0x7fdeb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:50.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fded4f7b700 1 --2- 172.21.15.92:0/2245009598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdeb80051c0 0x7fdeb800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdecffff700 1 -- 172.21.15.92:0/2245009598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdeb80051c0 msgr2=0x7fdeb800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdecffff700 1 --2- 172.21.15.92:0/2245009598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdeb80051c0 0x7fdeb800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdecffff700 1 -- 172.21.15.92:0/2245009598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdeb8004810 msgr2=0x7fdeb8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdecffff700 1 --2- 172.21.15.92:0/2245009598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdeb8004810 0x7fdeb8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdecffff700 1 -- 172.21.15.92:0/2245009598 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb8003570 con 0x7fdeb8003a60 2023-10-06T10:42:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdecf7fe700 1 --2- 172.21.15.92:0/2245009598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdeb8004810 0x7fdeb8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fded4f7b700 1 --2- 172.21.15.92:0/2245009598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdeb80051c0 0x7fdeb800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fded4f7b700 1 -- 172.21.15.92:0/2245009598 reap_dead start 2023-10-06T10:42:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdecffff700 1 --2- 172.21.15.92:0/2245009598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdeb8003a60 0x7fdeb8003e70 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fdec400c3c0 tx=0x7fdec4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c9d55d7c8a57e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdeceffd700 1 -- 172.21.15.92:0/2245009598 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec400cc90 con 0x7fdeb8003a60 2023-10-06T10:42:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdeceffd700 1 -- 172.21.15.92:0/2245009598 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec400cdf0 con 0x7fdeb8003a60 2023-10-06T10:42:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.804+0000 7fdeceffd700 1 -- 172.21.15.92:0/2245009598 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec4005b50 con 0x7fdeb8003a60 2023-10-06T10:42:50.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.805+0000 7fded577c700 1 -- 172.21.15.92:0/2245009598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdeb8003a60 msgr2=0x7fdeb8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:50.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.805+0000 7fded577c700 1 --2- 172.21.15.92:0/2245009598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdeb8003a60 0x7fdeb8003e70 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fdec400c3c0 tx=0x7fdec4001e50 comp rx=0 tx=0).stop 2023-10-06T10:42:50.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.805+0000 7fded577c700 1 -- 172.21.15.92:0/2245009598 shutdown_connections 2023-10-06T10:42:50.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.805+0000 7fded577c700 1 --2- 172.21.15.92:0/2245009598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdeb8003a60 0x7fdeb8003e70 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.805+0000 7fded577c700 1 -- 172.21.15.92:0/2245009598 >> 172.21.15.92:0/2245009598 conn(0x7fded006ece0 msgr2=0x7fded006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:50.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.805+0000 7fded577c700 1 -- 172.21.15.92:0/2245009598 shutdown_connections 2023-10-06T10:42:50.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.805+0000 7fded577c700 1 -- 172.21.15.92:0/2245009598 wait complete. 2023-10-06T10:42:50.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.806+0000 7fded577c700 1 Processor -- start 2023-10-06T10:42:50.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.806+0000 7fded577c700 1 -- start start 2023-10-06T10:42:50.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.807+0000 7fded577c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdeb8003a60 0x7fded00cf750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:50.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.807+0000 7fded577c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fded00cee30 0x7fded0154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:50.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.807+0000 7fded577c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fded00b4830 0x7fded00b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:50.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.807+0000 7fded577c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded00d0ed0 con 0x7fded00b4830 2023-10-06T10:42:50.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.807+0000 7fded577c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded00d0a40 con 0x7fded00cee30 2023-10-06T10:42:50.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.807+0000 7fded577c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded00b51c0 con 0x7fdeb8003a60 2023-10-06T10:42:50.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.807+0000 7fded4f7b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fded00b4830 0x7fded00b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:50.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.807+0000 7fded4f7b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fded00b4830 0x7fded00b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48754/0 (socket says 172.21.15.92:48754) 2023-10-06T10:42:50.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.807+0000 7fded4f7b700 1 -- 172.21.15.92:0/2349562066 learned_addr learned my addr 172.21.15.92:0/2349562066 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:50.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.807+0000 7fdecffff700 1 --2- 172.21.15.92:0/2349562066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdeb8003a60 0x7fded00cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:50.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.807+0000 7fdecf7fe700 1 --2- 172.21.15.92:0/2349562066 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fded00cee30 0x7fded0154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:50.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.807+0000 7fded4f7b700 1 -- 172.21.15.92:0/2349562066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdeb8003a60 msgr2=0x7fded00cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:50.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.807+0000 7fded4f7b700 1 --2- 172.21.15.92:0/2349562066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdeb8003a60 0x7fded00cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.807+0000 7fded4f7b700 1 -- 172.21.15.92:0/2349562066 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fded00cee30 msgr2=0x7fded0154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:50.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.807+0000 7fded4f7b700 1 --2- 172.21.15.92:0/2349562066 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fded00cee30 0x7fded0154690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:50.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.807+0000 7fded4f7b700 1 -- 172.21.15.92:0/2349562066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded00cf2a0 con 0x7fded00b4830 2023-10-06T10:42:50.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.807+0000 7fdecf7fe700 1 --2- 172.21.15.92:0/2349562066 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fded00cee30 0x7fded0154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:50.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.807+0000 7fded4f7b700 1 --2- 172.21.15.92:0/2349562066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fded00b4830 0x7fded00b4c80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fded00c3a90 tx=0x7fdec800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:50.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.807+0000 7fdeb7fff700 1 -- 172.21.15.92:0/2349562066 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec800e070 con 0x7fded00b4830 2023-10-06T10:42:50.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.807+0000 7fdeb7fff700 1 -- 172.21.15.92:0/2349562066 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec8011db0 con 0x7fded00b4830 2023-10-06T10:42:50.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.807+0000 7fded577c700 1 -- 172.21.15.92:0/2349562066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded00cfe30 con 0x7fded00b4830 2023-10-06T10:42:50.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.807+0000 7fdeb7fff700 1 -- 172.21.15.92:0/2349562066 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec80166c0 con 0x7fded00b4830 2023-10-06T10:42:50.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.808+0000 7fded577c700 1 -- 172.21.15.92:0/2349562066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded00d03f0 con 0x7fded00b4830 2023-10-06T10:42:50.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.809+0000 7fded577c700 1 -- 172.21.15.92:0/2349562066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded00d0bb0 con 0x7fded00b4830 2023-10-06T10:42:50.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.809+0000 7fdeb7fff700 1 -- 172.21.15.92:0/2349562066 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdec8016a70 con 0x7fded00b4830 2023-10-06T10:42:50.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.809+0000 7fdeb7fff700 1 --2- 172.21.15.92:0/2349562066 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdebc066f30 0x7fdebc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:50.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.809+0000 7fdeb7fff700 1 -- 172.21.15.92:0/2349562066 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fdec80855d0 con 0x7fded00b4830 2023-10-06T10:42:50.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.809+0000 7fdecffff700 1 --2- 172.21.15.92:0/2349562066 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdebc066f30 0x7fdebc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:50.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.810+0000 7fdecffff700 1 --2- 172.21.15.92:0/2349562066 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdebc066f30 0x7fdebc0693e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdec400c1a0 tx=0x7fdec4015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:50.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:50.813+0000 7fdeb7fff700 1 -- 172.21.15.92:0/2349562066 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdec8052560 con 0x7fded00b4830 2023-10-06T10:42:51.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.036+0000 7fded577c700 1 -- 172.21.15.92:0/2349562066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fded005f230 con 0x7fded00b4830 2023-10-06T10:42:51.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.036+0000 7fdeb7fff700 1 -- 172.21.15.92:0/2349562066 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4636 (secure 0 0 0) 0x7fdec8055b80 con 0x7fded00b4830 2023-10-06T10:42:51.038 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:42:51.038 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":59,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/1409495728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":1409495728},{"type":"v1","addr":"172.21.15.124:6834","nonce":1409495728}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:41:47.898774+0000","modified":"2023-10-06T10:42:21.332932+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6486,"mds_1":6483,"mds_2":6019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6019":{"gid":6019,"name":"b","rank":2,"incarnation":54,"state":"up:active","state_seq":10,"addr":"172.21.15.124:6835/2297612118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2297612118},{"type":"v1","addr":"172.21.15.124:6835","nonce":2297612118}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6483":{"gid":6483,"name":"a","rank":1,"incarnation":52,"state":"up:active","state_seq":11,"addr":"172.21.15.92:6836/171578176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":171578176},{"type":"v1","addr":"172.21.15.92:6836","nonce":171578176}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6486":{"gid":6486,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":11,"addr":"172.21.15.92:6837/818031030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":818031030},{"type":"v1","addr":"172.21.15.92:6837","nonce":818031030}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T10:42:51.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.038+0000 7fded577c700 1 -- 172.21.15.92:0/2349562066 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdebc066f30 msgr2=0x7fdebc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:51.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.038+0000 7fded577c700 1 --2- 172.21.15.92:0/2349562066 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdebc066f30 0x7fdebc0693e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdec400c1a0 tx=0x7fdec4015000 comp rx=0 tx=0).stop 2023-10-06T10:42:51.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.038+0000 7fded577c700 1 -- 172.21.15.92:0/2349562066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fded00b4830 msgr2=0x7fded00b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:51.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.038+0000 7fded577c700 1 --2- 172.21.15.92:0/2349562066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fded00b4830 0x7fded00b4c80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fded00c3a90 tx=0x7fdec800b040 comp rx=0 tx=0).stop 2023-10-06T10:42:51.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.038+0000 7fded577c700 1 -- 172.21.15.92:0/2349562066 shutdown_connections 2023-10-06T10:42:51.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.038+0000 7fded577c700 1 --2- 172.21.15.92:0/2349562066 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdebc066f30 0x7fdebc0693e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.038+0000 7fded577c700 1 --2- 172.21.15.92:0/2349562066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fded00b4830 0x7fded00b4c80 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.038+0000 7fded577c700 1 --2- 172.21.15.92:0/2349562066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdeb8003a60 0x7fded00cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.038+0000 7fded577c700 1 --2- 172.21.15.92:0/2349562066 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fded00cee30 0x7fded0154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.038+0000 7fded577c700 1 -- 172.21.15.92:0/2349562066 >> 172.21.15.92:0/2349562066 conn(0x7fded006ece0 msgr2=0x7fded00c5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:51.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.038+0000 7fded577c700 1 -- 172.21.15.92:0/2349562066 shutdown_connections 2023-10-06T10:42:51.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.038+0000 7fded577c700 1 -- 172.21.15.92:0/2349562066 wait complete. 2023-10-06T10:42:51.042 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 59 2023-10-06T10:42:51.052 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:42:51.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.164+0000 7f8d208a0700 1 Processor -- start 2023-10-06T10:42:51.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.164+0000 7f8d208a0700 1 -- start start 2023-10-06T10:42:51.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.165+0000 7f8d208a0700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8d1c0b49a0 0x7f8d1c0aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.165+0000 7f8d208a0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d1c0b0760 0x7f8d1c0b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.165+0000 7f8d208a0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d1c0b0c60 con 0x7f8d1c159560 2023-10-06T10:42:51.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.165+0000 7f8d208a0700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d1c0b0f40 con 0x7f8d1c0b49a0 2023-10-06T10:42:51.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.165+0000 7f8d208a0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c0b1220 con 0x7f8d1c0b0760 2023-10-06T10:42:51.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.165+0000 7f8d1a59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8d1c159560 0x7f8d1c0aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57008/0 (socket says 172.21.15.92:57008) 2023-10-06T10:42:51.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.165+0000 7f8d1a59c700 1 -- 172.21.15.92:0/1043354011 learned_addr learned my addr 172.21.15.92:0/1043354011 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:42:51.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d19d9b700 1 --2- 172.21.15.92:0/1043354011 >> v2:172.21.15.124:3300/0 conn(0x7f8d1c0b49a0 0x7f8d1c0aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1ad9d700 1 --2- 172.21.15.92:0/1043354011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d1c0b0760 0x7f8d1c0b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1959a700 1 -- 172.21.15.92:0/1043354011 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3926802709 0 0) 0x7f8d1c0b0c60 con 0x7f8d1c159560 2023-10-06T10:42:51.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1959a700 1 -- 172.21.15.92:0/1043354011 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d08003570 con 0x7f8d1c159560 2023-10-06T10:42:51.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1959a700 1 -- 172.21.15.92:0/1043354011 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2791918383 0 0) 0x7f8d08003570 con 0x7f8d1c159560 2023-10-06T10:42:51.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1959a700 1 -- 172.21.15.92:0/1043354011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d1c0b0760 msgr2=0x7f8d1c0b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:51.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1959a700 1 --2- 172.21.15.92:0/1043354011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d1c0b0760 0x7f8d1c0b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1959a700 1 -- 172.21.15.92:0/1043354011 >> v2:172.21.15.124:3300/0 conn(0x7f8d1c0b49a0 msgr2=0x7f8d1c0aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:51.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1959a700 1 --2- 172.21.15.92:0/1043354011 >> v2:172.21.15.124:3300/0 conn(0x7f8d1c0b49a0 0x7f8d1c0aac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1959a700 1 -- 172.21.15.92:0/1043354011 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c0b1500 con 0x7f8d1c159560 2023-10-06T10:42:51.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1959a700 1 -- 172.21.15.92:0/1043354011 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f8d0c001970 con 0x7f8d1c159560 2023-10-06T10:42:51.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1959a700 1 -- 172.21.15.92:0/1043354011 >> v1:172.21.15.92:6789/0 conn(0x7f8d1c159560 legacy=0x7f8d1c0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:51.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1959a700 1 --2- 172.21.15.92:0/1043354011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d08003a60 0x7f8d08003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1959a700 1 --2- 172.21.15.92:0/1043354011 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d08004810 0x7f8d08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1959a700 1 --2- 172.21.15.92:0/1043354011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d1c0b0760 0x7f8d080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1959a700 1 -- 172.21.15.92:0/1043354011 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c0b0f40 con 0x7f8d08003a60 2023-10-06T10:42:51.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1959a700 1 -- 172.21.15.92:0/1043354011 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c0b1220 con 0x7f8d08004810 2023-10-06T10:42:51.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1959a700 1 -- 172.21.15.92:0/1043354011 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0800ba60 con 0x7f8d1c0b0760 2023-10-06T10:42:51.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1a59c700 1 --2- 172.21.15.92:0/1043354011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d08003a60 0x7f8d08003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d19d9b700 1 --2- 172.21.15.92:0/1043354011 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d08004810 0x7f8d08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1ad9d700 1 --2- 172.21.15.92:0/1043354011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d1c0b0760 0x7f8d080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1a59c700 1 -- 172.21.15.92:0/1043354011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d1c0b0760 msgr2=0x7f8d080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:51.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1a59c700 1 --2- 172.21.15.92:0/1043354011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d1c0b0760 0x7f8d080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1a59c700 1 -- 172.21.15.92:0/1043354011 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d08004810 msgr2=0x7f8d08004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:51.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1a59c700 1 --2- 172.21.15.92:0/1043354011 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d08004810 0x7f8d08004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.166+0000 7f8d1a59c700 1 -- 172.21.15.92:0/1043354011 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d08003570 con 0x7f8d08003a60 2023-10-06T10:42:51.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.167+0000 7f8d1a59c700 1 --2- 172.21.15.92:0/1043354011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d08003a60 0x7f8d08003e70 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00c6a0 tx=0x7f8d0c007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbca9a51358bc200 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:51.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.167+0000 7f8d1959a700 1 -- 172.21.15.92:0/1043354011 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c005b70 con 0x7f8d08003a60 2023-10-06T10:42:51.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.167+0000 7f8d1959a700 1 -- 172.21.15.92:0/1043354011 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0c005cf0 con 0x7f8d08003a60 2023-10-06T10:42:51.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.167+0000 7f8d1959a700 1 -- 172.21.15.92:0/1043354011 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c007360 con 0x7f8d08003a60 2023-10-06T10:42:51.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.167+0000 7f8d208a0700 1 -- 172.21.15.92:0/1043354011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d08003a60 msgr2=0x7f8d08003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:51.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.167+0000 7f8d208a0700 1 --2- 172.21.15.92:0/1043354011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d08003a60 0x7f8d08003e70 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00c6a0 tx=0x7f8d0c007ca0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.167+0000 7f8d1ad9d700 1 -- 172.21.15.92:0/1043354011 reap_dead start 2023-10-06T10:42:51.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.167+0000 7f8d208a0700 1 -- 172.21.15.92:0/1043354011 shutdown_connections 2023-10-06T10:42:51.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.167+0000 7f8d208a0700 1 -- 172.21.15.92:0/1043354011 >> 172.21.15.92:0/1043354011 conn(0x7f8d1c0997f0 msgr2=0x7f8d1c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:51.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.167+0000 7f8d208a0700 1 -- 172.21.15.92:0/1043354011 shutdown_connections 2023-10-06T10:42:51.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.167+0000 7f8d208a0700 1 -- 172.21.15.92:0/1043354011 wait complete. 2023-10-06T10:42:51.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.168+0000 7f8d208a0700 1 Processor -- start 2023-10-06T10:42:51.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.168+0000 7f8d208a0700 1 -- start start 2023-10-06T10:42:51.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.169+0000 7f8d208a0700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d08003a60 0x7f8d1c0ac540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.169+0000 7f8d208a0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d1c0ade00 0x7f8d1c169ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.169+0000 7f8d208a0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d1c0ae270 0x7f8d1c16a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.169+0000 7f8d208a0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c0af300 con 0x7f8d1c0ade00 2023-10-06T10:42:51.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.169+0000 7f8d208a0700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c0ae6f0 con 0x7f8d08003a60 2023-10-06T10:42:51.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.169+0000 7f8d208a0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c0ae9d0 con 0x7f8d1c0ae270 2023-10-06T10:42:51.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.169+0000 7f8d19d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d1c0ade00 0x7f8d1c169ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.169+0000 7f8d19d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d1c0ade00 0x7f8d1c169ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48778/0 (socket says 172.21.15.92:48778) 2023-10-06T10:42:51.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.169+0000 7f8d19d9b700 1 -- 172.21.15.92:0/2815470065 learned_addr learned my addr 172.21.15.92:0/2815470065 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:51.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.169+0000 7f8d1ad9d700 1 --2- 172.21.15.92:0/2815470065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d1c0ae270 0x7f8d1c16a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.169+0000 7f8d19d9b700 1 -- 172.21.15.92:0/2815470065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d1c0ae270 msgr2=0x7f8d1c16a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:51.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.169+0000 7f8d19d9b700 1 --2- 172.21.15.92:0/2815470065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d1c0ae270 0x7f8d1c16a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.169+0000 7f8d19d9b700 1 -- 172.21.15.92:0/2815470065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d08003a60 msgr2=0x7f8d1c0ac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:51.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.169+0000 7f8d1a59c700 1 --2- 172.21.15.92:0/2815470065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d08003a60 0x7f8d1c0ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.169+0000 7f8d19d9b700 1 --2- 172.21.15.92:0/2815470065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d08003a60 0x7f8d1c0ac540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.169+0000 7f8d19d9b700 1 -- 172.21.15.92:0/2815470065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c0aecb0 con 0x7f8d1c0ade00 2023-10-06T10:42:51.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.170+0000 7f8d19d9b700 1 --2- 172.21.15.92:0/2815470065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d1c0ade00 0x7f8d1c169ed0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f8d04011af0 tx=0x7f8d040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:51.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.170+0000 7f8d127fc700 1 -- 172.21.15.92:0/2815470065 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0400fa10 con 0x7f8d1c0ade00 2023-10-06T10:42:51.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.170+0000 7f8d127fc700 1 -- 172.21.15.92:0/2815470065 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0400fb70 con 0x7f8d1c0ade00 2023-10-06T10:42:51.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.170+0000 7f8d208a0700 1 -- 172.21.15.92:0/2815470065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d1c0aef90 con 0x7f8d1c0ade00 2023-10-06T10:42:51.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.170+0000 7f8d127fc700 1 -- 172.21.15.92:0/2815470065 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d04014440 con 0x7f8d1c0ade00 2023-10-06T10:42:51.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.170+0000 7f8d208a0700 1 -- 172.21.15.92:0/2815470065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1c157a30 con 0x7f8d1c0ade00 2023-10-06T10:42:51.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.171+0000 7f8d208a0700 1 -- 172.21.15.92:0/2815470065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d1c0af470 con 0x7f8d1c0ade00 2023-10-06T10:42:51.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.171+0000 7f8d127fc700 1 -- 172.21.15.92:0/2815470065 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8d04014870 con 0x7f8d1c0ade00 2023-10-06T10:42:51.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.171+0000 7f8d127fc700 1 --2- 172.21.15.92:0/2815470065 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8cfc066f30 0x7f8cfc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.171+0000 7f8d127fc700 1 -- 172.21.15.92:0/2815470065 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f8d04018030 con 0x7f8d1c0ade00 2023-10-06T10:42:51.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.171+0000 7f8d1a59c700 1 --2- 172.21.15.92:0/2815470065 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8cfc066f30 0x7f8cfc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.172+0000 7f8d1a59c700 1 --2- 172.21.15.92:0/2815470065 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8cfc066f30 0x7f8cfc0693e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00c6a0 tx=0x7f8d0c015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:51.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.175+0000 7f8d127fc700 1 -- 172.21.15.92:0/2815470065 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d04052540 con 0x7f8d1c0ade00 2023-10-06T10:42:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.358+0000 7f8d208a0700 1 -- 172.21.15.92:0/2815470065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d1c064a90 con 0x7f8d1c0ade00 2023-10-06T10:42:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.359+0000 7f8d127fc700 1 -- 172.21.15.92:0/2815470065 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+17171 (secure 0 0 0) 0x7f8d04055b60 con 0x7f8d1c0ade00 2023-10-06T10:42:51.361 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:42:51.361 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":66,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:42:06.597181+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:41:45.546409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-06T10:41:46.862955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.8","mappings":[{"from":6,"to":0}]},{"pgid":"11.30","mappings":[{"from":6,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:42:51.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.361+0000 7f8d208a0700 1 -- 172.21.15.92:0/2815470065 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8cfc066f30 msgr2=0x7f8cfc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:51.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.361+0000 7f8d208a0700 1 --2- 172.21.15.92:0/2815470065 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8cfc066f30 0x7f8cfc0693e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00c6a0 tx=0x7f8d0c015000 comp rx=0 tx=0).stop 2023-10-06T10:42:51.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.361+0000 7f8d208a0700 1 -- 172.21.15.92:0/2815470065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d1c0ade00 msgr2=0x7f8d1c169ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:51.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.361+0000 7f8d208a0700 1 --2- 172.21.15.92:0/2815470065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d1c0ade00 0x7f8d1c169ed0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f8d04011af0 tx=0x7f8d040091a0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.361+0000 7f8d208a0700 1 -- 172.21.15.92:0/2815470065 shutdown_connections 2023-10-06T10:42:51.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.361+0000 7f8d208a0700 1 --2- 172.21.15.92:0/2815470065 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8cfc066f30 0x7f8cfc0693e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.361+0000 7f8d208a0700 1 --2- 172.21.15.92:0/2815470065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d1c0ae270 0x7f8d1c16a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.361+0000 7f8d208a0700 1 --2- 172.21.15.92:0/2815470065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d08003a60 0x7f8d1c0ac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.361+0000 7f8d208a0700 1 --2- 172.21.15.92:0/2815470065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d1c0ade00 0x7f8d1c169ed0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.361+0000 7f8d208a0700 1 -- 172.21.15.92:0/2815470065 >> 172.21.15.92:0/2815470065 conn(0x7f8d1c0997f0 msgr2=0x7f8d1c069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:51.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.361+0000 7f8d208a0700 1 -- 172.21.15.92:0/2815470065 shutdown_connections 2023-10-06T10:42:51.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.361+0000 7f8d208a0700 1 -- 172.21.15.92:0/2815470065 wait complete. 2023-10-06T10:42:51.375 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T10:42:51.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.488+0000 7f31ac95c700 1 Processor -- start 2023-10-06T10:42:51.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.488+0000 7f31ac95c700 1 -- start start 2023-10-06T10:42:51.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.488+0000 7f31ac95c700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f31a8159440 0x7f31a80b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31ac95c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31a80b2100 0x7f31a80b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31ac95c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a80cb7a0 con 0x7f31a80b35d0 2023-10-06T10:42:51.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31ac95c700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a80b0860 con 0x7f31a8159440 2023-10-06T10:42:51.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31ac95c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80cb150 con 0x7f31a80b2100 2023-10-06T10:42:51.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31a659c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f31a80b35d0 0x7f31a80cbc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57014/0 (socket says 172.21.15.92:57014) 2023-10-06T10:42:51.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31a659c700 1 -- 172.21.15.92:0/667577201 learned_addr learned my addr 172.21.15.92:0/667577201 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:42:51.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31a759e700 1 --2- 172.21.15.92:0/667577201 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31a80b2100 0x7f31a80b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31a6d9d700 1 --2- 172.21.15.92:0/667577201 >> v2:172.21.15.124:3300/0 conn(0x7f31a8159440 0x7f31a80b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31a5d9b700 1 -- 172.21.15.92:0/667577201 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2154131088 0 0) 0x7f31a80cb7a0 con 0x7f31a80b35d0 2023-10-06T10:42:51.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31a5d9b700 1 -- 172.21.15.92:0/667577201 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3194003570 con 0x7f31a80b35d0 2023-10-06T10:42:51.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31a5d9b700 1 -- 172.21.15.92:0/667577201 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3661349521 0 0) 0x7f3194003570 con 0x7f31a80b35d0 2023-10-06T10:42:51.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31a5d9b700 1 -- 172.21.15.92:0/667577201 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31a80b2100 msgr2=0x7f31a80b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:51.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31a5d9b700 1 --2- 172.21.15.92:0/667577201 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31a80b2100 0x7f31a80b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31a5d9b700 1 -- 172.21.15.92:0/667577201 >> v2:172.21.15.124:3300/0 conn(0x7f31a8159440 msgr2=0x7f31a80b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:51.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31a5d9b700 1 --2- 172.21.15.92:0/667577201 >> v2:172.21.15.124:3300/0 conn(0x7f31a8159440 0x7f31a80b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31a5d9b700 1 -- 172.21.15.92:0/667577201 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a80cb430 con 0x7f31a80b35d0 2023-10-06T10:42:51.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31a5d9b700 1 -- 172.21.15.92:0/667577201 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f3190001470 con 0x7f31a80b35d0 2023-10-06T10:42:51.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31a5d9b700 1 -- 172.21.15.92:0/667577201 >> v1:172.21.15.92:6789/0 conn(0x7f31a80b35d0 legacy=0x7f31a80cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:51.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31a5d9b700 1 --2- 172.21.15.92:0/667577201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3194003a60 0x7f3194003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31a5d9b700 1 --2- 172.21.15.92:0/667577201 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31a80b2100 0x7f3194004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31a5d9b700 1 --2- 172.21.15.92:0/667577201 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3194004d50 0x7f31940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31a5d9b700 1 -- 172.21.15.92:0/667577201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80b0860 con 0x7f3194003a60 2023-10-06T10:42:51.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31a5d9b700 1 -- 172.21.15.92:0/667577201 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80cb150 con 0x7f3194004d50 2023-10-06T10:42:51.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31a5d9b700 1 -- 172.21.15.92:0/667577201 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319400ba60 con 0x7f31a80b2100 2023-10-06T10:42:51.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.489+0000 7f31a6d9d700 1 --2- 172.21.15.92:0/667577201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3194003a60 0x7f3194003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.490+0000 7f31a659c700 1 --2- 172.21.15.92:0/667577201 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31a80b2100 0x7f3194004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.490+0000 7f31a759e700 1 --2- 172.21.15.92:0/667577201 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3194004d50 0x7f31940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.490+0000 7f31a6d9d700 1 -- 172.21.15.92:0/667577201 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31a80b2100 msgr2=0x7f3194004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:51.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.490+0000 7f31a6d9d700 1 --2- 172.21.15.92:0/667577201 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31a80b2100 0x7f3194004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.490+0000 7f31a6d9d700 1 -- 172.21.15.92:0/667577201 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3194004d50 msgr2=0x7f31940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:51.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.490+0000 7f31a6d9d700 1 --2- 172.21.15.92:0/667577201 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3194004d50 0x7f31940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.490+0000 7f31a6d9d700 1 -- 172.21.15.92:0/667577201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3194003570 con 0x7f3194003a60 2023-10-06T10:42:51.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.490+0000 7f31a6d9d700 1 --2- 172.21.15.92:0/667577201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3194003a60 0x7f3194003e70 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f31a000e210 tx=0x7f31a000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe384240463f6f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:51.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.490+0000 7f31a5d9b700 1 -- 172.21.15.92:0/667577201 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a000ea60 con 0x7f3194003a60 2023-10-06T10:42:51.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.490+0000 7f31a5d9b700 1 -- 172.21.15.92:0/667577201 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a000f040 con 0x7f3194003a60 2023-10-06T10:42:51.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.490+0000 7f31a5d9b700 1 -- 172.21.15.92:0/667577201 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a000ee10 con 0x7f3194003a60 2023-10-06T10:42:51.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.490+0000 7f31ac95c700 1 -- 172.21.15.92:0/667577201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3194003a60 msgr2=0x7f3194003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:51.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.490+0000 7f31ac95c700 1 --2- 172.21.15.92:0/667577201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3194003a60 0x7f3194003e70 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f31a000e210 tx=0x7f31a000b040 comp rx=0 tx=0).stop 2023-10-06T10:42:51.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.490+0000 7f31a759e700 1 -- 172.21.15.92:0/667577201 reap_dead start 2023-10-06T10:42:51.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.491+0000 7f31ac95c700 1 -- 172.21.15.92:0/667577201 shutdown_connections 2023-10-06T10:42:51.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.491+0000 7f31ac95c700 1 -- 172.21.15.92:0/667577201 >> 172.21.15.92:0/667577201 conn(0x7f31a806ece0 msgr2=0x7f31a806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:51.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.491+0000 7f31ac95c700 1 -- 172.21.15.92:0/667577201 shutdown_connections 2023-10-06T10:42:51.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.491+0000 7f31ac95c700 1 -- 172.21.15.92:0/667577201 wait complete. 2023-10-06T10:42:51.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.492+0000 7f31ac95c700 1 Processor -- start 2023-10-06T10:42:51.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.492+0000 7f31ac95c700 1 -- start start 2023-10-06T10:42:51.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.493+0000 7f31ac95c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3194003a60 0x7f31a8154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.493+0000 7f31ac95c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31a815bfe0 0x7f31a815c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.493+0000 7f31ac95c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31a815deb0 0x7f31a815e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.493+0000 7f31ac95c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a815cdb0 con 0x7f31a815bfe0 2023-10-06T10:42:51.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.493+0000 7f31ac95c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a815da20 con 0x7f3194003a60 2023-10-06T10:42:51.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.493+0000 7f31ac95c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a815d0f0 con 0x7f31a815deb0 2023-10-06T10:42:51.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.493+0000 7f31a659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31a815bfe0 0x7f31a815c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.493+0000 7f31a659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31a815bfe0 0x7f31a815c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48790/0 (socket says 172.21.15.92:48790) 2023-10-06T10:42:51.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.493+0000 7f31a659c700 1 -- 172.21.15.92:0/2880233582 learned_addr learned my addr 172.21.15.92:0/2880233582 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:51.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.493+0000 7f31a6d9d700 1 --2- 172.21.15.92:0/2880233582 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3194003a60 0x7f31a8154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.493+0000 7f31a759e700 1 --2- 172.21.15.92:0/2880233582 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31a815deb0 0x7f31a815e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.493+0000 7f31a659c700 1 -- 172.21.15.92:0/2880233582 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31a815deb0 msgr2=0x7f31a815e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:51.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.493+0000 7f31a659c700 1 --2- 172.21.15.92:0/2880233582 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31a815deb0 0x7f31a815e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.493+0000 7f31a659c700 1 -- 172.21.15.92:0/2880233582 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3194003a60 msgr2=0x7f31a8154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:51.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.493+0000 7f31a659c700 1 --2- 172.21.15.92:0/2880233582 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3194003a60 0x7f31a8154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.493+0000 7f31a659c700 1 -- 172.21.15.92:0/2880233582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a815d3d0 con 0x7f31a815bfe0 2023-10-06T10:42:51.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.493+0000 7f31a659c700 1 --2- 172.21.15.92:0/2880233582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31a815bfe0 0x7f31a815c450 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f3190001120 tx=0x7f3190001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:51.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.493+0000 7f319effd700 1 -- 172.21.15.92:0/2880233582 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3190018030 con 0x7f31a815bfe0 2023-10-06T10:42:51.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.493+0000 7f319effd700 1 -- 172.21.15.92:0/2880233582 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319000f690 con 0x7f31a815bfe0 2023-10-06T10:42:51.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.493+0000 7f31ac95c700 1 -- 172.21.15.92:0/2880233582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a815d6b0 con 0x7f31a815bfe0 2023-10-06T10:42:51.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.493+0000 7f319effd700 1 -- 172.21.15.92:0/2880233582 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319000fa30 con 0x7f31a815bfe0 2023-10-06T10:42:51.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.493+0000 7f31ac95c700 1 -- 172.21.15.92:0/2880233582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a815c990 con 0x7f31a815bfe0 2023-10-06T10:42:51.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.494+0000 7f31ac95c700 1 -- 172.21.15.92:0/2880233582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a815dbd0 con 0x7f31a815bfe0 2023-10-06T10:42:51.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.494+0000 7f319effd700 1 -- 172.21.15.92:0/2880233582 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f319001b070 con 0x7f31a815bfe0 2023-10-06T10:42:51.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.495+0000 7f319effd700 1 --2- 172.21.15.92:0/2880233582 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3188066f30 0x7f31880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.495+0000 7f319effd700 1 -- 172.21.15.92:0/2880233582 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f3190019070 con 0x7f31a815bfe0 2023-10-06T10:42:51.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.495+0000 7f31a6d9d700 1 --2- 172.21.15.92:0/2880233582 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3188066f30 0x7f31880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.495+0000 7f31a6d9d700 1 --2- 172.21.15.92:0/2880233582 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3188066f30 0x7f31880693e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f31a000afd0 tx=0x7f31a001a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:51.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.499+0000 7f319effd700 1 -- 172.21.15.92:0/2880233582 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31a815dbd0 con 0x7f31a815bfe0 2023-10-06T10:42:51.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.728+0000 7f31ac95c700 1 -- 172.21.15.92:0/2880233582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f31a815db90 con 0x7f31a815bfe0 2023-10-06T10:42:51.730 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T10:42:51.729+0000 7f9e7f3ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T10:42:51.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.735+0000 7f319effd700 1 -- 172.21.15.92:0/2880233582 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v60) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3190055200 con 0x7f31a815bfe0 2023-10-06T10:42:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.738+0000 7f31ac95c700 1 -- 172.21.15.92:0/2880233582 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3188066f30 msgr2=0x7f31880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.738+0000 7f31ac95c700 1 --2- 172.21.15.92:0/2880233582 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3188066f30 0x7f31880693e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f31a000afd0 tx=0x7f31a001a000 comp rx=0 tx=0).stop 2023-10-06T10:42:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.738+0000 7f31ac95c700 1 -- 172.21.15.92:0/2880233582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31a815bfe0 msgr2=0x7f31a815c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.738+0000 7f31ac95c700 1 --2- 172.21.15.92:0/2880233582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31a815bfe0 0x7f31a815c450 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f3190001120 tx=0x7f3190001e90 comp rx=0 tx=0).stop 2023-10-06T10:42:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.738+0000 7f31ac95c700 1 -- 172.21.15.92:0/2880233582 shutdown_connections 2023-10-06T10:42:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.738+0000 7f31ac95c700 1 --2- 172.21.15.92:0/2880233582 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3188066f30 0x7f31880693e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.738+0000 7f31ac95c700 1 --2- 172.21.15.92:0/2880233582 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31a815deb0 0x7f31a815e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.738+0000 7f31ac95c700 1 --2- 172.21.15.92:0/2880233582 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3194003a60 0x7f31a8154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.738+0000 7f31ac95c700 1 --2- 172.21.15.92:0/2880233582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31a815bfe0 0x7f31a815c450 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.738+0000 7f31ac95c700 1 -- 172.21.15.92:0/2880233582 >> 172.21.15.92:0/2880233582 conn(0x7f31a806ece0 msgr2=0x7f31a80b49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.738+0000 7f31ac95c700 1 -- 172.21.15.92:0/2880233582 shutdown_connections 2023-10-06T10:42:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.738+0000 7f31ac95c700 1 -- 172.21.15.92:0/2880233582 wait complete. 2023-10-06T10:42:51.742 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T10:42:51.742 INFO:tasks.ceph.mds.b.smithi124.stderr: -1214> 2023-10-06T10:42:16.599+0000 7f75e6ba2700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2023-10-06T10:42:00.020428+0000 1=0+1) n(v0 rc2023-10-06T10:42:00.020428+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={6112=pAsLsXsFsx/pAsLsXsFsx@4},l=6112 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55f480e4ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:42:00.020428+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:42:00.020428+0000 2=0+2)","memory_value.dirstat":"f(v0 m2023-10-06T10:42:00.020428+0000 1=0+1)","memory_value.rstat":"n(v0 rc2023-10-06T10:42:00.020428+0000 2=0+2)","error_str":""},"return_code":-61} 2023-10-06T10:42:51.742 INFO:tasks.ceph.mds.b.smithi124.stderr: -1194> 2023-10-06T10:42:16.599+0000 7f75e6ba2700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={6112=pAsLsXsFsx/AsLsXsFsx@2},l=6112 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55f480e4b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2023-10-06T10:42:51.743 INFO:tasks.ceph.mds.a.smithi092.stderr: -1300> 2023-10-06T10:42:16.599+0000 7faa59df7700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2023-10-06T10:42:00.019428+0000 1=0+1) n(v2 rc2023-10-06T10:42:00.248421+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={6112=pAsLsXsFsx/pAsLsXsFsx@4},l=6112 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x56421d0c2000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:42:00.019428+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:42:00.248421+0000 4=0+4)","memory_value.dirstat":"f(v0 m2023-10-06T10:42:00.019428+0000 1=0+1)","memory_value.rstat":"n(v2 rc2023-10-06T10:42:00.248421+0000 4=0+4)","error_str":""},"return_code":-61} 2023-10-06T10:42:51.743 INFO:tasks.ceph.mds.a.smithi092.stderr: -1180> 2023-10-06T10:42:16.600+0000 7faa5fe03700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2023-10-06T10:42:00.019428+0000 1=0+1) n(v1 rc2023-10-06T10:42:00.248421+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={6112=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x56421d0c2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:42:00.019428+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:42:00.020428+0000 3=0+3)","memory_value.dirstat":"f(v0 m2023-10-06T10:42:00.019428+0000 1=0+1)","memory_value.rstat":"n(v1 rc2023-10-06T10:42:00.248421+0000 3=0+3)","error_str":""},"return_code":-61} 2023-10-06T10:42:51.753 INFO:tasks.ceph.mds.c.smithi092.stderr: -3057> 2023-10-06T10:42:16.598+0000 7fdec44a7700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2023-10-06T10:42:00.018428+0000 1=0+1) n(v3 rc2023-10-06T10:42:00.248421+0000 6=0+6) (inest mix dirty) (ifile excl) (iversion lock) caps={6112=pAsLsXsFsx/pAsLsXsFsx@5},l=6112 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x5573e59ba100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:42:00.018428+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:42:00.248421+0000 6=0+6)","memory_value.dirstat":"f(v0 m2023-10-06T10:42:00.018428+0000 1=0+1)","memory_value.rstat":"n(v3 rc2023-10-06T10:42:00.248421+0000 6=0+6)","error_str":""},"return_code":-61} 2023-10-06T10:42:51.753 INFO:tasks.ceph.mds.c.smithi092.stderr: -2958> 2023-10-06T10:42:16.600+0000 7fdeca4b3700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2023-10-06T10:42:00.018428+0000 1=0+1) n(v4 rc2023-10-06T10:42:00.248421+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={6112=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x5573e5a28000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:42:00.018428+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:42:00.248421+0000 5=0+5)","memory_value.dirstat":"f(v0 m2023-10-06T10:42:00.018428+0000 1=0+1)","memory_value.rstat":"n(v4 rc2023-10-06T10:42:00.248421+0000 5=0+5)","error_str":""},"return_code":-61} 2023-10-06T10:42:51.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T10:42:51.764 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-06T10:42:51.765 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T10:42:51.778 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T10:42:51.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.874+0000 7f11b63f8700 1 Processor -- start 2023-10-06T10:42:51.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.874+0000 7f11b63f8700 1 -- start start 2023-10-06T10:42:51.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.875+0000 7f11b63f8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11b0150940 0x7f11b0159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.875+0000 7f11b63f8700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11b015a160 0x7f11b00ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.875+0000 7f11b63f8700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f11b00c5250 0x7f11b00cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.875+0000 7f11b63f8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11b015ac30 con 0x7f11b0150940 2023-10-06T10:42:51.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.875+0000 7f11b63f8700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11b00cb6a0 con 0x7f11b00c5250 2023-10-06T10:42:51.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.875+0000 7f11b63f8700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b00cb090 con 0x7f11b015a160 2023-10-06T10:42:51.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.875+0000 7f11affff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11b0150940 0x7f11b0159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.875+0000 7f11affff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11b0150940 0x7f11b0159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48810/0 (socket says 172.21.15.92:48810) 2023-10-06T10:42:51.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.875+0000 7f11affff700 1 -- 172.21.15.92:0/1113834861 learned_addr learned my addr 172.21.15.92:0/1113834861 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:51.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.875+0000 7f11b4995700 1 --2- 172.21.15.92:0/1113834861 >> v2:172.21.15.124:3300/0 conn(0x7f11b00c5250 0x7f11b00cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.875+0000 7f11af7fe700 1 --2- 172.21.15.92:0/1113834861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11b015a160 0x7f11b00ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.875+0000 7f11affff700 1 -- 172.21.15.92:0/1113834861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11b015a160 msgr2=0x7f11b00ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:51.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.875+0000 7f11affff700 1 --2- 172.21.15.92:0/1113834861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11b015a160 0x7f11b00ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.875+0000 7f11affff700 1 -- 172.21.15.92:0/1113834861 >> v2:172.21.15.124:3300/0 conn(0x7f11b00c5250 msgr2=0x7f11b00cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:51.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.875+0000 7f11affff700 1 --2- 172.21.15.92:0/1113834861 >> v2:172.21.15.124:3300/0 conn(0x7f11b00c5250 0x7f11b00cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.875+0000 7f11affff700 1 -- 172.21.15.92:0/1113834861 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b015a670 con 0x7f11b0150940 2023-10-06T10:42:51.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.875+0000 7f11b4995700 1 --2- 172.21.15.92:0/1113834861 >> v2:172.21.15.124:3300/0 conn(0x7f11b00c5250 0x7f11b00cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:51.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.875+0000 7f11af7fe700 1 --2- 172.21.15.92:0/1113834861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11b015a160 0x7f11b00ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:51.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.875+0000 7f11affff700 1 --2- 172.21.15.92:0/1113834861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11b0150940 0x7f11b0159c20 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f11b0071190 tx=0x7f11a0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e550a05a8cd9e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:51.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.876+0000 7f11aeffd700 1 -- 172.21.15.92:0/1113834861 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a0005c10 con 0x7f11b0150940 2023-10-06T10:42:51.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.876+0000 7f11aeffd700 1 -- 172.21.15.92:0/1113834861 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a0005d70 con 0x7f11b0150940 2023-10-06T10:42:51.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.876+0000 7f11aeffd700 1 -- 172.21.15.92:0/1113834861 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a000b880 con 0x7f11b0150940 2023-10-06T10:42:51.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.876+0000 7f11b63f8700 1 -- 172.21.15.92:0/1113834861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11b0150940 msgr2=0x7f11b0159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:51.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.876+0000 7f11b63f8700 1 --2- 172.21.15.92:0/1113834861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11b0150940 0x7f11b0159c20 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f11b0071190 tx=0x7f11a0009220 comp rx=0 tx=0).stop 2023-10-06T10:42:51.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.876+0000 7f11b63f8700 1 -- 172.21.15.92:0/1113834861 shutdown_connections 2023-10-06T10:42:51.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.876+0000 7f11b63f8700 1 --2- 172.21.15.92:0/1113834861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11b0150940 0x7f11b0159c20 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.876+0000 7f11b63f8700 1 --2- 172.21.15.92:0/1113834861 >> v2:172.21.15.124:3300/0 conn(0x7f11b00c5250 0x7f11b00cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.876+0000 7f11b63f8700 1 --2- 172.21.15.92:0/1113834861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11b015a160 0x7f11b00ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.876+0000 7f11b63f8700 1 -- 172.21.15.92:0/1113834861 >> 172.21.15.92:0/1113834861 conn(0x7f11b0099700 msgr2=0x7f11b006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:51.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.876+0000 7f11b63f8700 1 -- 172.21.15.92:0/1113834861 shutdown_connections 2023-10-06T10:42:51.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.876+0000 7f11b63f8700 1 -- 172.21.15.92:0/1113834861 wait complete. 2023-10-06T10:42:51.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.877+0000 7f11b63f8700 1 Processor -- start 2023-10-06T10:42:51.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.877+0000 7f11b63f8700 1 -- start start 2023-10-06T10:42:51.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.878+0000 7f11b63f8700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11b00c5250 0x7f11b014eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.878+0000 7f11b63f8700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11b015a160 0x7f11b00c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.878+0000 7f11b63f8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11b00c5d20 0x7f11b014def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.878+0000 7f11b63f8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b016ce50 con 0x7f11b00c5d20 2023-10-06T10:42:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.878+0000 7f11b63f8700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b00c9d70 con 0x7f11b015a160 2023-10-06T10:42:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.878+0000 7f11b63f8700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b00c97b0 con 0x7f11b00c5250 2023-10-06T10:42:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.878+0000 7f11b4995700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11b00c5d20 0x7f11b014def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.878+0000 7f11affff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11b00c5250 0x7f11b014eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.878+0000 7f11af7fe700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11b015a160 0x7f11b00c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.878+0000 7f11af7fe700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11b015a160 0x7f11b00c8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:38234/0 (socket says 172.21.15.92:38234) 2023-10-06T10:42:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.878+0000 7f11affff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11b00c5250 0x7f11b014eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:33638/0 (socket says 172.21.15.92:33638) 2023-10-06T10:42:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.878+0000 7f11b4995700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11b00c5d20 0x7f11b014def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48822/0 (socket says 172.21.15.92:48822) 2023-10-06T10:42:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.878+0000 7f11af7fe700 1 -- 172.21.15.92:0/1040097456 learned_addr learned my addr 172.21.15.92:0/1040097456 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.879+0000 7f11b4995700 1 -- 172.21.15.92:0/1040097456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11b00c5250 msgr2=0x7f11b014eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.879+0000 7f11b4995700 1 --2- 172.21.15.92:0/1040097456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11b00c5250 0x7f11b014eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.879+0000 7f11b4995700 1 -- 172.21.15.92:0/1040097456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11b015a160 msgr2=0x7f11b00c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:51.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.879+0000 7f11b4995700 1 --2- 172.21.15.92:0/1040097456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11b015a160 0x7f11b00c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:51.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.879+0000 7f11b4995700 1 -- 172.21.15.92:0/1040097456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a0009480 con 0x7f11b00c5d20 2023-10-06T10:42:51.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.879+0000 7f11af7fe700 1 --2- 172.21.15.92:0/1040097456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11b015a160 0x7f11b00c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:42:51.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.879+0000 7f11b4995700 1 --2- 172.21.15.92:0/1040097456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11b00c5d20 0x7f11b014def0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f11a8009fa0 tx=0x7f11a8009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:51.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.879+0000 7f1197fff700 1 -- 172.21.15.92:0/1040097456 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a800dbb0 con 0x7f11b00c5d20 2023-10-06T10:42:51.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.879+0000 7f1197fff700 1 -- 172.21.15.92:0/1040097456 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a800f040 con 0x7f11b00c5d20 2023-10-06T10:42:51.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.879+0000 7f11b63f8700 1 -- 172.21.15.92:0/1040097456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b00c9a90 con 0x7f11b00c5d20 2023-10-06T10:42:51.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.879+0000 7f1197fff700 1 -- 172.21.15.92:0/1040097456 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a8015440 con 0x7f11b00c5d20 2023-10-06T10:42:51.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.879+0000 7f11b63f8700 1 -- 172.21.15.92:0/1040097456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b00c94c0 con 0x7f11b00c5d20 2023-10-06T10:42:51.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.880+0000 7f11b63f8700 1 -- 172.21.15.92:0/1040097456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b016ccb0 con 0x7f11b00c5d20 2023-10-06T10:42:51.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.880+0000 7f1197fff700 1 -- 172.21.15.92:0/1040097456 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f11a801e3b0 con 0x7f11b00c5d20 2023-10-06T10:42:51.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.881+0000 7f1197fff700 1 --2- 172.21.15.92:0/1040097456 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f119c066f30 0x7f119c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:51.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.881+0000 7f1197fff700 1 -- 172.21.15.92:0/1040097456 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f11a8084ce0 con 0x7f11b00c5d20 2023-10-06T10:42:51.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.881+0000 7f11affff700 1 --2- 172.21.15.92:0/1040097456 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f119c066f30 0x7f119c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:51.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.881+0000 7f11affff700 1 --2- 172.21.15.92:0/1040097456 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f119c066f30 0x7f119c0693e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009260 tx=0x7f11a0016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:51.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:51.885+0000 7f1197fff700 1 -- 172.21.15.92:0/1040097456 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11b016ccb0 con 0x7f11b00c5d20 2023-10-06T10:42:52.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.109+0000 7f11b63f8700 1 -- 172.21.15.92:0/1040097456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f11b016d330 con 0x7f11b00c5d20 2023-10-06T10:42:52.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.739+0000 7f1197fff700 1 -- 172.21.15.92:0/1040097456 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) v1 ==== 104+0+0 (secure 0 0 0) 0x7f11b016d330 con 0x7f11b00c5d20 2023-10-06T10:42:52.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.742+0000 7f11b63f8700 1 -- 172.21.15.92:0/1040097456 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f119c066f30 msgr2=0x7f119c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:52.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.742+0000 7f11b63f8700 1 --2- 172.21.15.92:0/1040097456 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f119c066f30 0x7f119c0693e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009260 tx=0x7f11a0016000 comp rx=0 tx=0).stop 2023-10-06T10:42:52.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.742+0000 7f11b63f8700 1 -- 172.21.15.92:0/1040097456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11b00c5d20 msgr2=0x7f11b014def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:52.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.742+0000 7f11b63f8700 1 --2- 172.21.15.92:0/1040097456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11b00c5d20 0x7f11b014def0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f11a8009fa0 tx=0x7f11a8009990 comp rx=0 tx=0).stop 2023-10-06T10:42:52.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.742+0000 7f11b63f8700 1 -- 172.21.15.92:0/1040097456 shutdown_connections 2023-10-06T10:42:52.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.742+0000 7f11b63f8700 1 --2- 172.21.15.92:0/1040097456 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f119c066f30 0x7f119c0693e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:52.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.742+0000 7f11b63f8700 1 --2- 172.21.15.92:0/1040097456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11b00c5d20 0x7f11b014def0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:52.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.742+0000 7f11b63f8700 1 --2- 172.21.15.92:0/1040097456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11b00c5250 0x7f11b014eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:52.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.742+0000 7f11b63f8700 1 --2- 172.21.15.92:0/1040097456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11b015a160 0x7f11b00c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:52.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.742+0000 7f11b63f8700 1 -- 172.21.15.92:0/1040097456 >> 172.21.15.92:0/1040097456 conn(0x7f11b0099700 msgr2=0x7f11b00ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:52.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.743+0000 7f11b63f8700 1 -- 172.21.15.92:0/1040097456 shutdown_connections 2023-10-06T10:42:52.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.743+0000 7f11b63f8700 1 -- 172.21.15.92:0/1040097456 wait complete. 2023-10-06T10:42:52.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T10:42:52.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.916+0000 7fd019baf700 1 Processor -- start 2023-10-06T10:42:52.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.916+0000 7fd019baf700 1 -- start start 2023-10-06T10:42:52.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.916+0000 7fd019baf700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd01406ece0 0x7fd0140b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:52.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.916+0000 7fd019baf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd0140aede0 0x7fd0140af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:52.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.916+0000 7fd019baf700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0140b2ce0 con 0x7fd01406f0b0 2023-10-06T10:42:52.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.916+0000 7fd019baf700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd01406fb40 con 0x7fd0140aede0 2023-10-06T10:42:52.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.916+0000 7fd019baf700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0140b0410 con 0x7fd01406ece0 2023-10-06T10:42:52.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.917+0000 7fd013fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd0140aede0 0x7fd0140af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:52.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.917+0000 7fd013fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd0140aede0 0x7fd0140af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48838/0 (socket says 172.21.15.92:48838) 2023-10-06T10:42:52.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.917+0000 7fd012ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd01406f0b0 0x7fd0140b08a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57032/0 (socket says 172.21.15.92:57032) 2023-10-06T10:42:52.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.917+0000 7fd013fff700 1 -- 172.21.15.92:0/3274446577 learned_addr learned my addr 172.21.15.92:0/3274446577 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:52.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.917+0000 7fd0137fe700 1 --2- 172.21.15.92:0/3274446577 >> v2:172.21.15.124:3300/0 conn(0x7fd01406ece0 0x7fd0140b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:52.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.917+0000 7fd013fff700 1 -- 172.21.15.92:0/3274446577 >> v2:172.21.15.124:3300/0 conn(0x7fd01406ece0 msgr2=0x7fd0140b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:52.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.917+0000 7fd013fff700 1 --2- 172.21.15.92:0/3274446577 >> v2:172.21.15.124:3300/0 conn(0x7fd01406ece0 0x7fd0140b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:52.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.917+0000 7fd013fff700 1 -- 172.21.15.92:0/3274446577 >> v1:172.21.15.92:6789/0 conn(0x7fd01406f0b0 legacy=0x7fd0140b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:52.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.917+0000 7fd013fff700 1 -- 172.21.15.92:0/3274446577 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0140af780 con 0x7fd0140aede0 2023-10-06T10:42:52.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.917+0000 7fd0127fc700 1 -- 172.21.15.92:0/3274446577 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4216134483 0 0) 0x7fd0140b2ce0 con 0x7fd01406f0b0 2023-10-06T10:42:52.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.917+0000 7fd013fff700 1 --2- 172.21.15.92:0/3274446577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0140aede0 0x7fd0140af240 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd0140cdcf0 tx=0x7fd00c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4ccc740ed62bb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:52.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.917+0000 7fd0127fc700 1 -- 172.21.15.92:0/3274446577 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00c00e070 con 0x7fd0140aede0 2023-10-06T10:42:52.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.917+0000 7fd0127fc700 1 -- 172.21.15.92:0/3274446577 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00c00ba20 con 0x7fd0140aede0 2023-10-06T10:42:52.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.917+0000 7fd0127fc700 1 -- 172.21.15.92:0/3274446577 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00c0092e0 con 0x7fd0140aede0 2023-10-06T10:42:52.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.918+0000 7fd019baf700 1 -- 172.21.15.92:0/3274446577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0140aede0 msgr2=0x7fd0140af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:52.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.918+0000 7fd019baf700 1 --2- 172.21.15.92:0/3274446577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0140aede0 0x7fd0140af240 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd0140cdcf0 tx=0x7fd00c009840 comp rx=0 tx=0).stop 2023-10-06T10:42:52.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.918+0000 7fd019baf700 1 -- 172.21.15.92:0/3274446577 shutdown_connections 2023-10-06T10:42:52.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.918+0000 7fd019baf700 1 --2- 172.21.15.92:0/3274446577 >> v2:172.21.15.124:3300/0 conn(0x7fd01406ece0 0x7fd0140b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:52.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.918+0000 7fd019baf700 1 --2- 172.21.15.92:0/3274446577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0140aede0 0x7fd0140af240 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:52.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.918+0000 7fd019baf700 1 -- 172.21.15.92:0/3274446577 >> 172.21.15.92:0/3274446577 conn(0x7fd0140682f0 msgr2=0x7fd0140686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:52.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.918+0000 7fd019baf700 1 -- 172.21.15.92:0/3274446577 shutdown_connections 2023-10-06T10:42:52.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.918+0000 7fd019baf700 1 -- 172.21.15.92:0/3274446577 wait complete. 2023-10-06T10:42:52.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.919+0000 7fd019baf700 1 Processor -- start 2023-10-06T10:42:52.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.919+0000 7fd019baf700 1 -- start start 2023-10-06T10:42:52.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.920+0000 7fd019baf700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd01406ece0 0x7fd014154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:52.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.920+0000 7fd019baf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd01406f0b0 0x7fd0140c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:52.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.920+0000 7fd019baf700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0140aede0 0x7fd0140c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:52.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.920+0000 7fd019baf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0140c99f0 con 0x7fd01406f0b0 2023-10-06T10:42:52.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.920+0000 7fd019baf700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd014157750 con 0x7fd01406ece0 2023-10-06T10:42:52.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.920+0000 7fd019baf700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd014157a30 con 0x7fd0140aede0 2023-10-06T10:42:52.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.920+0000 7fd012ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd01406f0b0 0x7fd0140c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:52.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.920+0000 7fd012ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd01406f0b0 0x7fd0140c8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48846/0 (socket says 172.21.15.92:48846) 2023-10-06T10:42:52.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.920+0000 7fd012ffd700 1 -- 172.21.15.92:0/3200671578 learned_addr learned my addr 172.21.15.92:0/3200671578 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:52.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.920+0000 7fd012ffd700 1 -- 172.21.15.92:0/3200671578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0140aede0 msgr2=0x7fd0140c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:52.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.920+0000 7fd012ffd700 1 --2- 172.21.15.92:0/3200671578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0140aede0 0x7fd0140c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:52.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.920+0000 7fd012ffd700 1 -- 172.21.15.92:0/3200671578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd01406ece0 msgr2=0x7fd014154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:52.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.920+0000 7fd012ffd700 1 --2- 172.21.15.92:0/3200671578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd01406ece0 0x7fd014154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:52.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.920+0000 7fd012ffd700 1 -- 172.21.15.92:0/3200671578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00c009ae0 con 0x7fd01406f0b0 2023-10-06T10:42:52.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.920+0000 7fd012ffd700 1 --2- 172.21.15.92:0/3200671578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd01406f0b0 0x7fd0140c8e80 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd00800c5b0 tx=0x7fd008001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:52.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.920+0000 7fcffb7fe700 1 -- 172.21.15.92:0/3200671578 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd008005a70 con 0x7fd01406f0b0 2023-10-06T10:42:52.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.920+0000 7fcffb7fe700 1 -- 172.21.15.92:0/3200671578 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd008005bd0 con 0x7fd01406f0b0 2023-10-06T10:42:52.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.920+0000 7fd019baf700 1 -- 172.21.15.92:0/3200671578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd014157d10 con 0x7fd01406f0b0 2023-10-06T10:42:52.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.920+0000 7fcffb7fe700 1 -- 172.21.15.92:0/3200671578 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0080070d0 con 0x7fd01406f0b0 2023-10-06T10:42:52.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.920+0000 7fd019baf700 1 -- 172.21.15.92:0/3200671578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd014154a50 con 0x7fd01406f0b0 2023-10-06T10:42:52.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.921+0000 7fd019baf700 1 -- 172.21.15.92:0/3200671578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd014064a90 con 0x7fd01406f0b0 2023-10-06T10:42:52.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.922+0000 7fcffb7fe700 1 -- 172.21.15.92:0/3200671578 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd008007480 con 0x7fd01406f0b0 2023-10-06T10:42:52.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.922+0000 7fcffb7fe700 1 --2- 172.21.15.92:0/3200671578 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd000066f30 0x7fd0000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:52.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.922+0000 7fcffb7fe700 1 -- 172.21.15.92:0/3200671578 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6694+0+0 (secure 0 0 0) 0x7fd008083e30 con 0x7fd01406f0b0 2023-10-06T10:42:52.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.922+0000 7fd0137fe700 1 --2- 172.21.15.92:0/3200671578 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd000066f30 0x7fd0000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:52.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.923+0000 7fd0137fe700 1 --2- 172.21.15.92:0/3200671578 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd000066f30 0x7fd0000693e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd014070220 tx=0x7fd0040091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:52.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:52.926+0000 7fcffb7fe700 1 -- 172.21.15.92:0/3200671578 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd008050cc0 con 0x7fd01406f0b0 2023-10-06T10:42:53.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.108+0000 7fd019baf700 1 -- 172.21.15.92:0/3200671578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd0140cd2e0 con 0x7fd01406f0b0 2023-10-06T10:42:53.227 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:42:53.230 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:42:53.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.741+0000 7fcffb7fe700 1 -- 172.21.15.92:0/3200671578 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v68) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd008017020 con 0x7fd01406f0b0 2023-10-06T10:42:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.745+0000 7fd019baf700 1 -- 172.21.15.92:0/3200671578 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd000066f30 msgr2=0x7fd0000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.745+0000 7fd019baf700 1 --2- 172.21.15.92:0/3200671578 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd000066f30 0x7fd0000693e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd014070220 tx=0x7fd0040091a0 comp rx=0 tx=0).stop 2023-10-06T10:42:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.745+0000 7fd019baf700 1 -- 172.21.15.92:0/3200671578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd01406f0b0 msgr2=0x7fd0140c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.745+0000 7fd019baf700 1 --2- 172.21.15.92:0/3200671578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd01406f0b0 0x7fd0140c8e80 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd00800c5b0 tx=0x7fd008001040 comp rx=0 tx=0).stop 2023-10-06T10:42:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.745+0000 7fd019baf700 1 -- 172.21.15.92:0/3200671578 shutdown_connections 2023-10-06T10:42:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.745+0000 7fd019baf700 1 --2- 172.21.15.92:0/3200671578 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd000066f30 0x7fd0000693e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.745+0000 7fd019baf700 1 --2- 172.21.15.92:0/3200671578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0140aede0 0x7fd0140c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.745+0000 7fd019baf700 1 --2- 172.21.15.92:0/3200671578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd01406ece0 0x7fd014154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.745+0000 7fd019baf700 1 --2- 172.21.15.92:0/3200671578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd01406f0b0 0x7fd0140c8e80 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.745+0000 7fd019baf700 1 -- 172.21.15.92:0/3200671578 >> 172.21.15.92:0/3200671578 conn(0x7fd0140682f0 msgr2=0x7fd014156840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.745+0000 7fd019baf700 1 -- 172.21.15.92:0/3200671578 shutdown_connections 2023-10-06T10:42:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.745+0000 7fd019baf700 1 -- 172.21.15.92:0/3200671578 wait complete. 2023-10-06T10:42:53.749 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-10-06T10:42:53.760 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T10:42:53.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.877+0000 7f9bd232b700 1 Processor -- start 2023-10-06T10:42:53.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.877+0000 7f9bd232b700 1 -- start start 2023-10-06T10:42:53.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.878+0000 7f9bd232b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc06ece0 0x7f9bcc0b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:53.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.878+0000 7f9bd232b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9bcc06f0b0 0x7f9bcc0b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:53.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.878+0000 7f9bd232b700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bcc0b2ce0 con 0x7f9bcc0aede0 2023-10-06T10:42:53.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.878+0000 7f9bd232b700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bcc06fb40 con 0x7f9bcc06f0b0 2023-10-06T10:42:53.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.878+0000 7f9bd232b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc0b0410 con 0x7f9bcc06ece0 2023-10-06T10:42:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.879+0000 7f9bcbfff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc06ece0 0x7f9bcc0b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.879+0000 7f9bcbfff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc06ece0 0x7f9bcc0b2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:48462/0 (socket says 172.21.15.92:48462) 2023-10-06T10:42:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.879+0000 7f9bcb7fe700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9bcc06f0b0 0x7f9bcc0b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.879+0000 7f9bcbfff700 1 -- 172.21.15.92:0/3209719725 learned_addr learned my addr 172.21.15.92:0/3209719725 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.879+0000 7f9bcbfff700 1 -- 172.21.15.92:0/3209719725 >> v2:172.21.15.124:3300/0 conn(0x7f9bcc06f0b0 msgr2=0x7f9bcc0b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.879+0000 7f9bcbfff700 1 --2- 172.21.15.92:0/3209719725 >> v2:172.21.15.124:3300/0 conn(0x7f9bcc06f0b0 0x7f9bcc0b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.879+0000 7f9bcbfff700 1 -- 172.21.15.92:0/3209719725 >> v1:172.21.15.124:6789/0 conn(0x7f9bcc0aede0 legacy=0x7f9bcc0af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.879+0000 7f9bcbfff700 1 -- 172.21.15.92:0/3209719725 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bcc0b2a00 con 0x7f9bcc06ece0 2023-10-06T10:42:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.879+0000 7f9bcb7fe700 1 --2- 172.21.15.92:0/3209719725 >> v2:172.21.15.124:3300/0 conn(0x7f9bcc06f0b0 0x7f9bcc0b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:42:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.879+0000 7f9bcbfff700 1 --2- 172.21.15.92:0/3209719725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc06ece0 0x7f9bcc0b2360 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0099e0 tx=0x7f9bbc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=260749eb38ef493c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:53.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.879+0000 7f9bcaffd700 1 -- 172.21.15.92:0/3209719725 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc00b820 con 0x7f9bcc06ece0 2023-10-06T10:42:53.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.879+0000 7f9bcaffd700 1 -- 172.21.15.92:0/3209719725 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bbc00b980 con 0x7f9bcc06ece0 2023-10-06T10:42:53.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.879+0000 7f9bcaffd700 1 -- 172.21.15.92:0/3209719725 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc00bd20 con 0x7f9bcc06ece0 2023-10-06T10:42:53.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.880+0000 7f9bd232b700 1 -- 172.21.15.92:0/3209719725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc06ece0 msgr2=0x7f9bcc0b2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:53.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.880+0000 7f9bd232b700 1 --2- 172.21.15.92:0/3209719725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc06ece0 0x7f9bcc0b2360 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0099e0 tx=0x7f9bbc009250 comp rx=0 tx=0).stop 2023-10-06T10:42:53.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.880+0000 7f9bd232b700 1 -- 172.21.15.92:0/3209719725 shutdown_connections 2023-10-06T10:42:53.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.880+0000 7f9bd232b700 1 --2- 172.21.15.92:0/3209719725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc06ece0 0x7f9bcc0b2360 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:53.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.880+0000 7f9bd232b700 1 --2- 172.21.15.92:0/3209719725 >> v2:172.21.15.124:3300/0 conn(0x7f9bcc06f0b0 0x7f9bcc0b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:53.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.880+0000 7f9bd232b700 1 -- 172.21.15.92:0/3209719725 >> 172.21.15.92:0/3209719725 conn(0x7f9bcc0682f0 msgr2=0x7f9bcc0686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:53.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.880+0000 7f9bd232b700 1 -- 172.21.15.92:0/3209719725 shutdown_connections 2023-10-06T10:42:53.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.880+0000 7f9bd232b700 1 -- 172.21.15.92:0/3209719725 wait complete. 2023-10-06T10:42:53.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.881+0000 7f9bd232b700 1 Processor -- start 2023-10-06T10:42:53.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.881+0000 7f9bd232b700 1 -- start start 2023-10-06T10:42:53.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.882+0000 7f9bd232b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc06ece0 0x7f9bcc15a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:53.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.882+0000 7f9bd232b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bcc06f0b0 0x7f9bcc0c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:53.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.882+0000 7f9bd232b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bcc0aede0 0x7f9bcc0c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:53.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.882+0000 7f9bd232b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc157850 con 0x7f9bcc06f0b0 2023-10-06T10:42:53.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.882+0000 7f9bd232b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc159ff0 con 0x7f9bcc0aede0 2023-10-06T10:42:53.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.882+0000 7f9bd232b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc156ca0 con 0x7f9bcc06ece0 2023-10-06T10:42:53.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.882+0000 7f9bcb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bcc06f0b0 0x7f9bcc0c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:53.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.882+0000 7f9bcb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bcc06f0b0 0x7f9bcc0c68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37452/0 (socket says 172.21.15.92:37452) 2023-10-06T10:42:53.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.882+0000 7f9bcb7fe700 1 -- 172.21.15.92:0/828722393 learned_addr learned my addr 172.21.15.92:0/828722393 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:53.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.882+0000 7f9bcbfff700 1 --2- 172.21.15.92:0/828722393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc06ece0 0x7f9bcc15a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:53.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.882+0000 7f9bd08c8700 1 --2- 172.21.15.92:0/828722393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bcc0aede0 0x7f9bcc0c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:53.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.883+0000 7f9bcb7fe700 1 -- 172.21.15.92:0/828722393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc06ece0 msgr2=0x7f9bcc15a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:53.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.883+0000 7f9bcb7fe700 1 --2- 172.21.15.92:0/828722393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc06ece0 0x7f9bcc15a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:53.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.883+0000 7f9bcb7fe700 1 -- 172.21.15.92:0/828722393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bcc0aede0 msgr2=0x7f9bcc0c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:53.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.883+0000 7f9bcb7fe700 1 --2- 172.21.15.92:0/828722393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bcc0aede0 0x7f9bcc0c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:53.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.883+0000 7f9bcb7fe700 1 -- 172.21.15.92:0/828722393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc0009420 con 0x7f9bcc06f0b0 2023-10-06T10:42:53.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.883+0000 7f9bcb7fe700 1 --2- 172.21.15.92:0/828722393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bcc06f0b0 0x7f9bcc0c68d0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000ec80 tx=0x7f9bc00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:53.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.883+0000 7f9bb3fff700 1 -- 172.21.15.92:0/828722393 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc000c9b0 con 0x7f9bcc06f0b0 2023-10-06T10:42:53.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.883+0000 7f9bb3fff700 1 -- 172.21.15.92:0/828722393 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc000cb10 con 0x7f9bcc06f0b0 2023-10-06T10:42:53.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.883+0000 7f9bd232b700 1 -- 172.21.15.92:0/828722393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bbc0094f0 con 0x7f9bcc06f0b0 2023-10-06T10:42:53.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.883+0000 7f9bb3fff700 1 -- 172.21.15.92:0/828722393 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc000ce90 con 0x7f9bcc06f0b0 2023-10-06T10:42:53.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.883+0000 7f9bd232b700 1 -- 172.21.15.92:0/828722393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bcc0c9090 con 0x7f9bcc06f0b0 2023-10-06T10:42:53.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.884+0000 7f9bd232b700 1 -- 172.21.15.92:0/828722393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bcc157990 con 0x7f9bcc06f0b0 2023-10-06T10:42:53.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.884+0000 7f9bb3fff700 1 -- 172.21.15.92:0/828722393 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9bc0013680 con 0x7f9bcc06f0b0 2023-10-06T10:42:53.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.885+0000 7f9bb3fff700 1 --2- 172.21.15.92:0/828722393 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9bb8066f30 0x7f9bb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:53.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.885+0000 7f9bb3fff700 1 -- 172.21.15.92:0/828722393 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6256+0+0 (secure 0 0 0) 0x7f9bc0084f90 con 0x7f9bcc06f0b0 2023-10-06T10:42:53.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.885+0000 7f9bcbfff700 1 --2- 172.21.15.92:0/828722393 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9bb8066f30 0x7f9bb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:53.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.885+0000 7f9bcbfff700 1 --2- 172.21.15.92:0/828722393 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9bb8066f30 0x7f9bb80693e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0099e0 tx=0x7f9bbc017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:53.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:53.889+0000 7f9bb3fff700 1 -- 172.21.15.92:0/828722393 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bcc157990 con 0x7f9bcc06f0b0 2023-10-06T10:42:54.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.078+0000 7f9bd232b700 1 -- 172.21.15.92:0/828722393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9bcc157990 con 0x7f9bcc06f0b0 2023-10-06T10:42:54.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.750+0000 7f9bb3fff700 1 -- 172.21.15.92:0/828722393 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v69) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9bcc157990 con 0x7f9bcc06f0b0 2023-10-06T10:42:54.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.753+0000 7f9bd232b700 1 -- 172.21.15.92:0/828722393 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9bb8066f30 msgr2=0x7f9bb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:54.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.753+0000 7f9bd232b700 1 --2- 172.21.15.92:0/828722393 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9bb8066f30 0x7f9bb80693e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0099e0 tx=0x7f9bbc017000 comp rx=0 tx=0).stop 2023-10-06T10:42:54.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.753+0000 7f9bd232b700 1 -- 172.21.15.92:0/828722393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bcc06f0b0 msgr2=0x7f9bcc0c68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:54.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.753+0000 7f9bd232b700 1 --2- 172.21.15.92:0/828722393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bcc06f0b0 0x7f9bcc0c68d0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000ec80 tx=0x7f9bc00091a0 comp rx=0 tx=0).stop 2023-10-06T10:42:54.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.753+0000 7f9bd232b700 1 -- 172.21.15.92:0/828722393 shutdown_connections 2023-10-06T10:42:54.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.753+0000 7f9bd232b700 1 --2- 172.21.15.92:0/828722393 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9bb8066f30 0x7f9bb80693e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:54.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.753+0000 7f9bd232b700 1 --2- 172.21.15.92:0/828722393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bcc0aede0 0x7f9bcc0c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:54.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.753+0000 7f9bd232b700 1 --2- 172.21.15.92:0/828722393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc06ece0 0x7f9bcc15a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:54.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.753+0000 7f9bd232b700 1 --2- 172.21.15.92:0/828722393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bcc06f0b0 0x7f9bcc0c68d0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:54.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.753+0000 7f9bd232b700 1 -- 172.21.15.92:0/828722393 >> 172.21.15.92:0/828722393 conn(0x7f9bcc0682f0 msgr2=0x7f9bcc0cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:54.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.753+0000 7f9bd232b700 1 -- 172.21.15.92:0/828722393 shutdown_connections 2023-10-06T10:42:54.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.753+0000 7f9bd232b700 1 -- 172.21.15.92:0/828722393 wait complete. 2023-10-06T10:42:54.757 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-10-06T10:42:54.769 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T10:42:54.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.886+0000 7fb91603a700 1 Processor -- start 2023-10-06T10:42:54.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.887+0000 7fb91603a700 1 -- start start 2023-10-06T10:42:54.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.887+0000 7fb91603a700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb910159420 0x7fb9101597f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:54.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.887+0000 7fb91603a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb9100cb6d0 0x7fb9100cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:54.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.887+0000 7fb91603a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9100cdc60 con 0x7fb9100682e0 2023-10-06T10:42:54.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.887+0000 7fb91603a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9100cd050 con 0x7fb9100cb6d0 2023-10-06T10:42:54.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.887+0000 7fb91603a700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9100cd330 con 0x7fb910159420 2023-10-06T10:42:54.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.887+0000 7fb906ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb9100cb6d0 0x7fb9100cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:54.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.888+0000 7fb90ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb9100682e0 0x7fb9100686e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53076/0 (socket says 172.21.15.92:53076) 2023-10-06T10:42:54.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.888+0000 7fb90ffff700 1 -- 172.21.15.92:0/145190768 learned_addr learned my addr 172.21.15.92:0/145190768 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:42:54.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.888+0000 7fb90f7fe700 1 --2- 172.21.15.92:0/145190768 >> v2:172.21.15.124:3300/0 conn(0x7fb910159420 0x7fb9101597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:54.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.888+0000 7fb906ffd700 1 -- 172.21.15.92:0/145190768 >> v2:172.21.15.124:3300/0 conn(0x7fb910159420 msgr2=0x7fb9101597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:54.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.888+0000 7fb906ffd700 1 --2- 172.21.15.92:0/145190768 >> v2:172.21.15.124:3300/0 conn(0x7fb910159420 0x7fb9101597f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:54.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.888+0000 7fb906ffd700 1 -- 172.21.15.92:0/145190768 >> v1:172.21.15.92:6789/0 conn(0x7fb9100682e0 legacy=0x7fb9100686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:54.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.888+0000 7fb90effd700 1 -- 172.21.15.92:0/145190768 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318465231 0 0) 0x7fb9100cdc60 con 0x7fb9100682e0 2023-10-06T10:42:54.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.888+0000 7fb906ffd700 1 -- 172.21.15.92:0/145190768 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9100cd610 con 0x7fb9100cb6d0 2023-10-06T10:42:54.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.888+0000 7fb906ffd700 1 --2- 172.21.15.92:0/145190768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9100cb6d0 0x7fb9100cbb70 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fb910068020 tx=0x7fb8f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16446c41f32d6992 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:54.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.888+0000 7fb90effd700 1 -- 172.21.15.92:0/145190768 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f800b6b0 con 0x7fb9100cb6d0 2023-10-06T10:42:54.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.888+0000 7fb90effd700 1 -- 172.21.15.92:0/145190768 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f800b810 con 0x7fb9100cb6d0 2023-10-06T10:42:54.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.888+0000 7fb90effd700 1 -- 172.21.15.92:0/145190768 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f800bbb0 con 0x7fb9100cb6d0 2023-10-06T10:42:54.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.889+0000 7fb91603a700 1 -- 172.21.15.92:0/145190768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9100cb6d0 msgr2=0x7fb9100cbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:54.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.889+0000 7fb91603a700 1 --2- 172.21.15.92:0/145190768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9100cb6d0 0x7fb9100cbb70 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fb910068020 tx=0x7fb8f80091a0 comp rx=0 tx=0).stop 2023-10-06T10:42:54.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.889+0000 7fb91603a700 1 -- 172.21.15.92:0/145190768 shutdown_connections 2023-10-06T10:42:54.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.889+0000 7fb91603a700 1 --2- 172.21.15.92:0/145190768 >> v2:172.21.15.124:3300/0 conn(0x7fb910159420 0x7fb9101597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:54.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.889+0000 7fb91603a700 1 --2- 172.21.15.92:0/145190768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9100cb6d0 0x7fb9100cbb70 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:54.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.889+0000 7fb91603a700 1 -- 172.21.15.92:0/145190768 >> 172.21.15.92:0/145190768 conn(0x7fb91006ece0 msgr2=0x7fb91006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:54.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.889+0000 7fb91603a700 1 -- 172.21.15.92:0/145190768 shutdown_connections 2023-10-06T10:42:54.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.889+0000 7fb91603a700 1 -- 172.21.15.92:0/145190768 wait complete. 2023-10-06T10:42:54.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.890+0000 7fb91603a700 1 Processor -- start 2023-10-06T10:42:54.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.890+0000 7fb91603a700 1 -- start start 2023-10-06T10:42:54.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb91603a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9100682e0 0x7fb91015c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:54.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb91603a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9100cb6d0 0x7fb910153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:54.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb91603a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb910159420 0x7fb9101544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:54.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb91603a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91015c6a0 con 0x7fb9100cb6d0 2023-10-06T10:42:54.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb91603a700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9100b3270 con 0x7fb910159420 2023-10-06T10:42:54.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb91603a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9100b3830 con 0x7fb9100682e0 2023-10-06T10:42:54.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb906ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9100cb6d0 0x7fb910153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:54.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb906ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9100cb6d0 0x7fb910153f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37470/0 (socket says 172.21.15.92:37470) 2023-10-06T10:42:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb906ffd700 1 -- 172.21.15.92:0/2746156996 learned_addr learned my addr 172.21.15.92:0/2746156996 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb90f7fe700 1 --2- 172.21.15.92:0/2746156996 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9100682e0 0x7fb91015c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb90ffff700 1 --2- 172.21.15.92:0/2746156996 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb910159420 0x7fb9101544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb906ffd700 1 -- 172.21.15.92:0/2746156996 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9100682e0 msgr2=0x7fb91015c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb906ffd700 1 --2- 172.21.15.92:0/2746156996 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9100682e0 0x7fb91015c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb906ffd700 1 -- 172.21.15.92:0/2746156996 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb910159420 msgr2=0x7fb9101544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb906ffd700 1 --2- 172.21.15.92:0/2746156996 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb910159420 0x7fb9101544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb906ffd700 1 -- 172.21.15.92:0/2746156996 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f8009420 con 0x7fb9100cb6d0 2023-10-06T10:42:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb90ffff700 1 --2- 172.21.15.92:0/2746156996 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb910159420 0x7fb9101544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb906ffd700 1 --2- 172.21.15.92:0/2746156996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9100cb6d0 0x7fb910153f80 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800b0e0 tx=0x7fb8f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb907fff700 1 -- 172.21.15.92:0/2746156996 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8011420 con 0x7fb9100cb6d0 2023-10-06T10:42:54.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb907fff700 1 -- 172.21.15.92:0/2746156996 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f8011580 con 0x7fb9100cb6d0 2023-10-06T10:42:54.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb91603a700 1 -- 172.21.15.92:0/2746156996 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb91015c8c0 con 0x7fb9100cb6d0 2023-10-06T10:42:54.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb907fff700 1 -- 172.21.15.92:0/2746156996 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8011940 con 0x7fb9100cb6d0 2023-10-06T10:42:54.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.891+0000 7fb91603a700 1 -- 172.21.15.92:0/2746156996 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9100b34c0 con 0x7fb9100cb6d0 2023-10-06T10:42:54.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.892+0000 7fb91603a700 1 -- 172.21.15.92:0/2746156996 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb91015cdc0 con 0x7fb9100cb6d0 2023-10-06T10:42:54.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.893+0000 7fb907fff700 1 -- 172.21.15.92:0/2746156996 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb8f8011cf0 con 0x7fb9100cb6d0 2023-10-06T10:42:54.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.893+0000 7fb907fff700 1 --2- 172.21.15.92:0/2746156996 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb8f0066f30 0x7fb8f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:54.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.893+0000 7fb90f7fe700 1 --2- 172.21.15.92:0/2746156996 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb8f0066f30 0x7fb8f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:54.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.893+0000 7fb907fff700 1 -- 172.21.15.92:0/2746156996 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5784+0+0 (secure 0 0 0) 0x7fb8f8085f10 con 0x7fb9100cb6d0 2023-10-06T10:42:54.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.894+0000 7fb90f7fe700 1 --2- 172.21.15.92:0/2746156996 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb8f0066f30 0x7fb8f00693e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb900006fd0 tx=0x7fb900008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:54.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:54.898+0000 7fb907fff700 1 -- 172.21.15.92:0/2746156996 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8f8053130 con 0x7fb9100cb6d0 2023-10-06T10:42:55.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.089+0000 7fb91603a700 1 -- 172.21.15.92:0/2746156996 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb9100b3970 con 0x7fb9100cb6d0 2023-10-06T10:42:55.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.093+0000 7fb907fff700 1 -- 172.21.15.92:0/2746156996 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb8f8056750 con 0x7fb9100cb6d0 2023-10-06T10:42:55.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.095+0000 7fb91603a700 1 -- 172.21.15.92:0/2746156996 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb8f0066f30 msgr2=0x7fb8f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:55.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.095+0000 7fb91603a700 1 --2- 172.21.15.92:0/2746156996 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb8f0066f30 0x7fb8f00693e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb900006fd0 tx=0x7fb900008040 comp rx=0 tx=0).stop 2023-10-06T10:42:55.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.095+0000 7fb91603a700 1 -- 172.21.15.92:0/2746156996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9100cb6d0 msgr2=0x7fb910153f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:55.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.095+0000 7fb91603a700 1 --2- 172.21.15.92:0/2746156996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9100cb6d0 0x7fb910153f80 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800b0e0 tx=0x7fb8f8017000 comp rx=0 tx=0).stop 2023-10-06T10:42:55.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.095+0000 7fb91603a700 1 -- 172.21.15.92:0/2746156996 shutdown_connections 2023-10-06T10:42:55.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.095+0000 7fb91603a700 1 --2- 172.21.15.92:0/2746156996 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb8f0066f30 0x7fb8f00693e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:55.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.095+0000 7fb91603a700 1 --2- 172.21.15.92:0/2746156996 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb910159420 0x7fb9101544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:55.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.095+0000 7fb91603a700 1 --2- 172.21.15.92:0/2746156996 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9100682e0 0x7fb91015c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:55.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.095+0000 7fb91603a700 1 --2- 172.21.15.92:0/2746156996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9100cb6d0 0x7fb910153f80 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:55.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.096+0000 7fb91603a700 1 -- 172.21.15.92:0/2746156996 >> 172.21.15.92:0/2746156996 conn(0x7fb91006ece0 msgr2=0x7fb9100c6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:55.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.096+0000 7fb91603a700 1 -- 172.21.15.92:0/2746156996 shutdown_connections 2023-10-06T10:42:55.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.096+0000 7fb91603a700 1 -- 172.21.15.92:0/2746156996 wait complete. 2023-10-06T10:42:55.109 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace' 2023-10-06T10:42:55.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.222+0000 7fbb9c6f0700 1 Processor -- start 2023-10-06T10:42:55.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.222+0000 7fbb9c6f0700 1 -- start start 2023-10-06T10:42:55.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.223+0000 7fbb9c6f0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb94159560 0x7fbb940aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:55.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.223+0000 7fbb9c6f0700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbb940b49a0 0x7fbb940aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:55.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.223+0000 7fbb9c6f0700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb940b0c60 con 0x7fbb940b0760 2023-10-06T10:42:55.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.223+0000 7fbb9c6f0700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb940b0f40 con 0x7fbb940b49a0 2023-10-06T10:42:55.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.223+0000 7fbb9c6f0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb940b1220 con 0x7fbb94159560 2023-10-06T10:42:55.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.223+0000 7fbb9a48c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb94159560 0x7fbb940aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:55.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.223+0000 7fbb99c8b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbb940b49a0 0x7fbb940aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:55.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.223+0000 7fbb9ac8d700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fbb940b0760 0x7fbb940b3890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:57328/0 (socket says 172.21.15.92:57328) 2023-10-06T10:42:55.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.223+0000 7fbb9ac8d700 1 -- 172.21.15.92:0/2250385520 learned_addr learned my addr 172.21.15.92:0/2250385520 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:42:55.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.223+0000 7fbb99c8b700 1 -- 172.21.15.92:0/2250385520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb94159560 msgr2=0x7fbb940aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:55.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.223+0000 7fbb99c8b700 1 --2- 172.21.15.92:0/2250385520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb94159560 0x7fbb940aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:55.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.223+0000 7fbb99c8b700 1 -- 172.21.15.92:0/2250385520 >> v1:172.21.15.124:6789/0 conn(0x7fbb940b0760 legacy=0x7fbb940b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:55.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.223+0000 7fbb99c8b700 1 -- 172.21.15.92:0/2250385520 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb940b1500 con 0x7fbb940b49a0 2023-10-06T10:42:55.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.223+0000 7fbb9a48c700 1 --2- 172.21.15.92:0/2250385520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb94159560 0x7fbb940aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:42:55.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.224+0000 7fbb99c8b700 1 --2- 172.21.15.92:0/2250385520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb940b49a0 0x7fbb940aaa70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbb840098e0 tx=0x7fbb840091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0424b231fa0073a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:55.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.224+0000 7fbb9948a700 1 -- 172.21.15.92:0/2250385520 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8400b640 con 0x7fbb940b49a0 2023-10-06T10:42:55.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.224+0000 7fbb9948a700 1 -- 172.21.15.92:0/2250385520 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8400b7a0 con 0x7fbb940b49a0 2023-10-06T10:42:55.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.224+0000 7fbb9948a700 1 -- 172.21.15.92:0/2250385520 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8400bb40 con 0x7fbb940b49a0 2023-10-06T10:42:55.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.224+0000 7fbb9c6f0700 1 -- 172.21.15.92:0/2250385520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb940b49a0 msgr2=0x7fbb940aaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:55.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.224+0000 7fbb9c6f0700 1 --2- 172.21.15.92:0/2250385520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb940b49a0 0x7fbb940aaa70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbb840098e0 tx=0x7fbb840091a0 comp rx=0 tx=0).stop 2023-10-06T10:42:55.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.224+0000 7fbb9c6f0700 1 -- 172.21.15.92:0/2250385520 shutdown_connections 2023-10-06T10:42:55.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.224+0000 7fbb9c6f0700 1 --2- 172.21.15.92:0/2250385520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb94159560 0x7fbb940aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:55.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.224+0000 7fbb9c6f0700 1 --2- 172.21.15.92:0/2250385520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb940b49a0 0x7fbb940aaa70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:55.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.224+0000 7fbb9c6f0700 1 -- 172.21.15.92:0/2250385520 >> 172.21.15.92:0/2250385520 conn(0x7fbb940997f0 msgr2=0x7fbb9406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:55.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.224+0000 7fbb9c6f0700 1 -- 172.21.15.92:0/2250385520 shutdown_connections 2023-10-06T10:42:55.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.225+0000 7fbb9c6f0700 1 -- 172.21.15.92:0/2250385520 wait complete. 2023-10-06T10:42:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.226+0000 7fbb9c6f0700 1 Processor -- start 2023-10-06T10:42:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.226+0000 7fbb9c6f0700 1 -- start start 2023-10-06T10:42:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.226+0000 7fbb9c6f0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb940b0760 0x7fbb94157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.226+0000 7fbb9c6f0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb940b49a0 0x7fbb9409c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.226+0000 7fbb9c6f0700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb94159560 0x7fbb9409ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.226+0000 7fbb9c6f0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9415bd40 con 0x7fbb940b49a0 2023-10-06T10:42:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.226+0000 7fbb9c6f0700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9415c020 con 0x7fbb94159560 2023-10-06T10:42:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.226+0000 7fbb9c6f0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9415c300 con 0x7fbb940b0760 2023-10-06T10:42:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.226+0000 7fbb99c8b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb940b49a0 0x7fbb9409c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.226+0000 7fbb99c8b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb940b49a0 0x7fbb9409c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37474/0 (socket says 172.21.15.92:37474) 2023-10-06T10:42:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.226+0000 7fbb99c8b700 1 -- 172.21.15.92:0/3368451715 learned_addr learned my addr 172.21.15.92:0/3368451715 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.226+0000 7fbb9ac8d700 1 --2- 172.21.15.92:0/3368451715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb94159560 0x7fbb9409ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.226+0000 7fbb9a48c700 1 --2- 172.21.15.92:0/3368451715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb940b0760 0x7fbb94157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.227+0000 7fbb99c8b700 1 -- 172.21.15.92:0/3368451715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb940b0760 msgr2=0x7fbb94157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.227+0000 7fbb99c8b700 1 --2- 172.21.15.92:0/3368451715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb940b0760 0x7fbb94157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.227+0000 7fbb99c8b700 1 -- 172.21.15.92:0/3368451715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb94159560 msgr2=0x7fbb9409ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.227+0000 7fbb99c8b700 1 --2- 172.21.15.92:0/3368451715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb94159560 0x7fbb9409ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.227+0000 7fbb99c8b700 1 -- 172.21.15.92:0/3368451715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb900093c0 con 0x7fbb940b49a0 2023-10-06T10:42:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.227+0000 7fbb99c8b700 1 --2- 172.21.15.92:0/3368451715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb940b49a0 0x7fbb9409c8f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fbb94067c20 tx=0x7fbb84017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.227+0000 7fbb8a7fc700 1 -- 172.21.15.92:0/3368451715 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8400b910 con 0x7fbb940b49a0 2023-10-06T10:42:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.227+0000 7fbb8a7fc700 1 -- 172.21.15.92:0/3368451715 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8400ba70 con 0x7fbb940b49a0 2023-10-06T10:42:55.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.227+0000 7fbb9c6f0700 1 -- 172.21.15.92:0/3368451715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb84009420 con 0x7fbb940b49a0 2023-10-06T10:42:55.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.227+0000 7fbb8a7fc700 1 -- 172.21.15.92:0/3368451715 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84024670 con 0x7fbb940b49a0 2023-10-06T10:42:55.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.227+0000 7fbb9c6f0700 1 -- 172.21.15.92:0/3368451715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb941696c0 con 0x7fbb940b49a0 2023-10-06T10:42:55.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.228+0000 7fbb9c6f0700 1 -- 172.21.15.92:0/3368451715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb94064a90 con 0x7fbb940b49a0 2023-10-06T10:42:55.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.228+0000 7fbb8a7fc700 1 -- 172.21.15.92:0/3368451715 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbb8402d440 con 0x7fbb940b49a0 2023-10-06T10:42:55.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.229+0000 7fbb8a7fc700 1 --2- 172.21.15.92:0/3368451715 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbb78066f30 0x7fbb780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:55.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.229+0000 7fbb8a7fc700 1 -- 172.21.15.92:0/3368451715 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5784+0+0 (secure 0 0 0) 0x7fbb84085e40 con 0x7fbb940b49a0 2023-10-06T10:42:55.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.229+0000 7fbb9a48c700 1 --2- 172.21.15.92:0/3368451715 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbb78066f30 0x7fbb780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:55.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.229+0000 7fbb9a48c700 1 --2- 172.21.15.92:0/3368451715 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbb78066f30 0x7fbb780693e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009e90 tx=0x7fbb90009120 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:55.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.233+0000 7fbb8a7fc700 1 -- 172.21.15.92:0/3368451715 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb84053060 con 0x7fbb940b49a0 2023-10-06T10:42:55.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.489+0000 7fbb9c6f0700 1 -- 172.21.15.92:0/3368451715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]} v 0) v1 -- 0x7fbb940b17e0 con 0x7fbb940b49a0 2023-10-06T10:42:55.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.750+0000 7fbb8a7fc700 1 -- 172.21.15.92:0/3368451715 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]}]=0 v240) v1 ==== 142+0+0 (secure 0 0 0) 0x7fbb84056680 con 0x7fbb940b49a0 2023-10-06T10:42:55.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.752+0000 7fbb9c6f0700 1 -- 172.21.15.92:0/3368451715 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbb78066f30 msgr2=0x7fbb780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:55.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.752+0000 7fbb9c6f0700 1 --2- 172.21.15.92:0/3368451715 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbb78066f30 0x7fbb780693e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009e90 tx=0x7fbb90009120 comp rx=0 tx=0).stop 2023-10-06T10:42:55.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.752+0000 7fbb9c6f0700 1 -- 172.21.15.92:0/3368451715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb940b49a0 msgr2=0x7fbb9409c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:55.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.752+0000 7fbb9c6f0700 1 --2- 172.21.15.92:0/3368451715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb940b49a0 0x7fbb9409c8f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fbb94067c20 tx=0x7fbb84017000 comp rx=0 tx=0).stop 2023-10-06T10:42:55.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.752+0000 7fbb9c6f0700 1 -- 172.21.15.92:0/3368451715 shutdown_connections 2023-10-06T10:42:55.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.752+0000 7fbb9c6f0700 1 --2- 172.21.15.92:0/3368451715 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbb78066f30 0x7fbb780693e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:55.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.752+0000 7fbb9c6f0700 1 --2- 172.21.15.92:0/3368451715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb94159560 0x7fbb9409ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:55.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.752+0000 7fbb9c6f0700 1 --2- 172.21.15.92:0/3368451715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb940b0760 0x7fbb94157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:55.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.752+0000 7fbb9c6f0700 1 --2- 172.21.15.92:0/3368451715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb940b49a0 0x7fbb9409c8f0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:55.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.752+0000 7fbb9c6f0700 1 -- 172.21.15.92:0/3368451715 >> 172.21.15.92:0/3368451715 conn(0x7fbb940997f0 msgr2=0x7fbb94069300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:55.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.752+0000 7fbb9c6f0700 1 -- 172.21.15.92:0/3368451715 shutdown_connections 2023-10-06T10:42:55.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.752+0000 7fbb9c6f0700 1 -- 172.21.15.92:0/3368451715 wait complete. 2023-10-06T10:42:55.766 INFO:tasks.cephfs_test_runner:test_scrub_backtrace (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2023-10-06T10:42:55.766 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_non_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2023-10-06T10:42:55.766 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0' 2023-10-06T10:42:55.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.880+0000 7efd9a7ec700 1 Processor -- start 2023-10-06T10:42:55.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.880+0000 7efd9a7ec700 1 -- start start 2023-10-06T10:42:55.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.881+0000 7efd9a7ec700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd941570f0 0x7efd94069ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:55.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.881+0000 7efd9a7ec700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efd9406e8b0 0x7efd940685c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:55.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.881+0000 7efd9a7ec700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd94068f40 con 0x7efd94068b00 2023-10-06T10:42:55.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.881+0000 7efd9a7ec700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd940cad20 con 0x7efd9406e8b0 2023-10-06T10:42:55.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.881+0000 7efd9a7ec700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd940cc4f0 con 0x7efd941570f0 2023-10-06T10:42:55.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.881+0000 7efd99feb700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7efd94068b00 0x7efd9415ada0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53086/0 (socket says 172.21.15.92:53086) 2023-10-06T10:42:55.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.881+0000 7efd99feb700 1 -- 172.21.15.92:0/2833452187 learned_addr learned my addr 172.21.15.92:0/2833452187 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:42:55.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.881+0000 7efd997ea700 1 --2- 172.21.15.92:0/2833452187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd941570f0 0x7efd94069ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:55.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd98fe9700 1 --2- 172.21.15.92:0/2833452187 >> v2:172.21.15.124:3300/0 conn(0x7efd9406e8b0 0x7efd940685c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:55.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd8bfff700 1 -- 172.21.15.92:0/2833452187 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2887411268 0 0) 0x7efd94068f40 con 0x7efd94068b00 2023-10-06T10:42:55.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd8bfff700 1 -- 172.21.15.92:0/2833452187 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd80003570 con 0x7efd94068b00 2023-10-06T10:42:55.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd8bfff700 1 -- 172.21.15.92:0/2833452187 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2549548153 0 0) 0x7efd80003570 con 0x7efd94068b00 2023-10-06T10:42:55.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd8bfff700 1 -- 172.21.15.92:0/2833452187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd941570f0 msgr2=0x7efd94069ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:55.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd8bfff700 1 --2- 172.21.15.92:0/2833452187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd941570f0 0x7efd94069ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:55.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd8bfff700 1 -- 172.21.15.92:0/2833452187 >> v2:172.21.15.124:3300/0 conn(0x7efd9406e8b0 msgr2=0x7efd940685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:55.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd8bfff700 1 --2- 172.21.15.92:0/2833452187 >> v2:172.21.15.124:3300/0 conn(0x7efd9406e8b0 0x7efd940685c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:55.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd8bfff700 1 -- 172.21.15.92:0/2833452187 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9406a560 con 0x7efd94068b00 2023-10-06T10:42:55.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd8bfff700 1 -- 172.21.15.92:0/2833452187 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7efd8c0017f0 con 0x7efd94068b00 2023-10-06T10:42:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd8bfff700 1 -- 172.21.15.92:0/2833452187 >> v1:172.21.15.92:6789/0 conn(0x7efd94068b00 legacy=0x7efd9415ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd8bfff700 1 --2- 172.21.15.92:0/2833452187 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd80003a60 0x7efd80003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd8bfff700 1 --2- 172.21.15.92:0/2833452187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd80004810 0x7efd80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd8bfff700 1 --2- 172.21.15.92:0/2833452187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd941570f0 0x7efd800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd8bfff700 1 -- 172.21.15.92:0/2833452187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd940cad20 con 0x7efd80004810 2023-10-06T10:42:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd8bfff700 1 -- 172.21.15.92:0/2833452187 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd940cc4f0 con 0x7efd80003a60 2023-10-06T10:42:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd8bfff700 1 -- 172.21.15.92:0/2833452187 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8000bae0 con 0x7efd941570f0 2023-10-06T10:42:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd98fe9700 1 --2- 172.21.15.92:0/2833452187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd80004810 0x7efd80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd997ea700 1 --2- 172.21.15.92:0/2833452187 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd80003a60 0x7efd80003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd99feb700 1 --2- 172.21.15.92:0/2833452187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd941570f0 0x7efd800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd98fe9700 1 -- 172.21.15.92:0/2833452187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd941570f0 msgr2=0x7efd800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd98fe9700 1 --2- 172.21.15.92:0/2833452187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd941570f0 0x7efd800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd98fe9700 1 -- 172.21.15.92:0/2833452187 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd80003a60 msgr2=0x7efd80003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd98fe9700 1 --2- 172.21.15.92:0/2833452187 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd80003a60 0x7efd80003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd98fe9700 1 -- 172.21.15.92:0/2833452187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd80003570 con 0x7efd80004810 2023-10-06T10:42:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd99feb700 1 --2- 172.21.15.92:0/2833452187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd941570f0 0x7efd800051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.882+0000 7efd98fe9700 1 --2- 172.21.15.92:0/2833452187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd80004810 0x7efd80004c80 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7efd84009c00 tx=0x7efd840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13dda50829852257 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.883+0000 7efd8bfff700 1 -- 172.21.15.92:0/2833452187 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8400e710 con 0x7efd80004810 2023-10-06T10:42:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.883+0000 7efd8bfff700 1 -- 172.21.15.92:0/2833452187 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8400e870 con 0x7efd80004810 2023-10-06T10:42:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.883+0000 7efd8bfff700 1 -- 172.21.15.92:0/2833452187 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8400ec30 con 0x7efd80004810 2023-10-06T10:42:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.883+0000 7efd9a7ec700 1 -- 172.21.15.92:0/2833452187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd80004810 msgr2=0x7efd80004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.883+0000 7efd9a7ec700 1 --2- 172.21.15.92:0/2833452187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd80004810 0x7efd80004c80 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7efd84009c00 tx=0x7efd840091a0 comp rx=0 tx=0).stop 2023-10-06T10:42:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.883+0000 7efd99feb700 1 -- 172.21.15.92:0/2833452187 reap_dead start 2023-10-06T10:42:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.883+0000 7efd9a7ec700 1 -- 172.21.15.92:0/2833452187 shutdown_connections 2023-10-06T10:42:55.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.883+0000 7efd9a7ec700 1 -- 172.21.15.92:0/2833452187 >> 172.21.15.92:0/2833452187 conn(0x7efd940c9c10 msgr2=0x7efd9406dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:55.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.883+0000 7efd9a7ec700 1 -- 172.21.15.92:0/2833452187 shutdown_connections 2023-10-06T10:42:55.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.883+0000 7efd9a7ec700 1 -- 172.21.15.92:0/2833452187 wait complete. 2023-10-06T10:42:55.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.884+0000 7efd9a7ec700 1 Processor -- start 2023-10-06T10:42:55.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.884+0000 7efd9a7ec700 1 -- start start 2023-10-06T10:42:55.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.885+0000 7efd9a7ec700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd80004810 0x7efd94151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:55.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.885+0000 7efd9a7ec700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd94154550 0x7efd941552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:55.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.885+0000 7efd9a7ec700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd94155d20 0x7efd9416a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:55.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.885+0000 7efd9a7ec700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd941613e0 con 0x7efd94155d20 2023-10-06T10:42:55.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.885+0000 7efd9a7ec700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd941535e0 con 0x7efd80004810 2023-10-06T10:42:55.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.885+0000 7efd9a7ec700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd941559e0 con 0x7efd94154550 2023-10-06T10:42:55.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.885+0000 7efd99feb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd94155d20 0x7efd9416a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:55.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.885+0000 7efd99feb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd94155d20 0x7efd9416a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37502/0 (socket says 172.21.15.92:37502) 2023-10-06T10:42:55.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.885+0000 7efd99feb700 1 -- 172.21.15.92:0/3039850239 learned_addr learned my addr 172.21.15.92:0/3039850239 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:55.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.885+0000 7efd997ea700 1 --2- 172.21.15.92:0/3039850239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd80004810 0x7efd94151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:55.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.886+0000 7efd98fe9700 1 --2- 172.21.15.92:0/3039850239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd94154550 0x7efd941552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:55.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.886+0000 7efd99feb700 1 -- 172.21.15.92:0/3039850239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd94154550 msgr2=0x7efd941552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:55.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.886+0000 7efd99feb700 1 --2- 172.21.15.92:0/3039850239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd94154550 0x7efd941552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:55.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.886+0000 7efd99feb700 1 -- 172.21.15.92:0/3039850239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd80004810 msgr2=0x7efd94151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:55.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.886+0000 7efd99feb700 1 --2- 172.21.15.92:0/3039850239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd80004810 0x7efd94151ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:55.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.886+0000 7efd99feb700 1 -- 172.21.15.92:0/3039850239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd941540c0 con 0x7efd94155d20 2023-10-06T10:42:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.886+0000 7efd99feb700 1 --2- 172.21.15.92:0/3039850239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd94155d20 0x7efd9416a480 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7efd940c9010 tx=0x7efd8c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.886+0000 7efd897fa700 1 -- 172.21.15.92:0/3039850239 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c00eb10 con 0x7efd94155d20 2023-10-06T10:42:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.886+0000 7efd897fa700 1 -- 172.21.15.92:0/3039850239 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8c00ec90 con 0x7efd94155d20 2023-10-06T10:42:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.886+0000 7efd9a7ec700 1 -- 172.21.15.92:0/3039850239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd941537d0 con 0x7efd94155d20 2023-10-06T10:42:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.886+0000 7efd897fa700 1 -- 172.21.15.92:0/3039850239 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c013420 con 0x7efd94155d20 2023-10-06T10:42:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.886+0000 7efd9a7ec700 1 -- 172.21.15.92:0/3039850239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd94153a70 con 0x7efd94155d20 2023-10-06T10:42:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.887+0000 7efd9a7ec700 1 -- 172.21.15.92:0/3039850239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd94154250 con 0x7efd94155d20 2023-10-06T10:42:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.887+0000 7efd897fa700 1 -- 172.21.15.92:0/3039850239 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efd8c013810 con 0x7efd94155d20 2023-10-06T10:42:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.888+0000 7efd897fa700 1 --2- 172.21.15.92:0/3039850239 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efd78066f30 0x7efd780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.888+0000 7efd897fa700 1 -- 172.21.15.92:0/3039850239 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5784+0+0 (secure 0 0 0) 0x7efd8c084a00 con 0x7efd94155d20 2023-10-06T10:42:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.888+0000 7efd997ea700 1 --2- 172.21.15.92:0/3039850239 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efd78066f30 0x7efd780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.888+0000 7efd997ea700 1 --2- 172.21.15.92:0/3039850239 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efd78066f30 0x7efd780693e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efd940c5410 tx=0x7efd900091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:55.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:55.892+0000 7efd897fa700 1 -- 172.21.15.92:0/3039850239 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd8c051c20 con 0x7efd94155d20 2023-10-06T10:42:56.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.150+0000 7efd9a7ec700 1 -- 172.21.15.92:0/3039850239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]} v 0) v1 -- 0x7efd9405f230 con 0x7efd94155d20 2023-10-06T10:42:56.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.751+0000 7efd897fa700 1 -- 172.21.15.92:0/3039850239 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]}]=0 v241) v1 ==== 144+0+0 (secure 0 0 0) 0x7efd8c055240 con 0x7efd94155d20 2023-10-06T10:42:56.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.753+0000 7efd9a7ec700 1 -- 172.21.15.92:0/3039850239 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efd78066f30 msgr2=0x7efd780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:56.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.753+0000 7efd9a7ec700 1 --2- 172.21.15.92:0/3039850239 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efd78066f30 0x7efd780693e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efd940c5410 tx=0x7efd900091a0 comp rx=0 tx=0).stop 2023-10-06T10:42:56.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.753+0000 7efd9a7ec700 1 -- 172.21.15.92:0/3039850239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd94155d20 msgr2=0x7efd9416a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:56.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.753+0000 7efd9a7ec700 1 --2- 172.21.15.92:0/3039850239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd94155d20 0x7efd9416a480 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7efd940c9010 tx=0x7efd8c001250 comp rx=0 tx=0).stop 2023-10-06T10:42:56.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.753+0000 7efd9a7ec700 1 -- 172.21.15.92:0/3039850239 shutdown_connections 2023-10-06T10:42:56.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.753+0000 7efd9a7ec700 1 --2- 172.21.15.92:0/3039850239 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efd78066f30 0x7efd780693e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:56.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.753+0000 7efd9a7ec700 1 --2- 172.21.15.92:0/3039850239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd94155d20 0x7efd9416a480 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:56.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.753+0000 7efd9a7ec700 1 --2- 172.21.15.92:0/3039850239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd80004810 0x7efd94151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:56.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.753+0000 7efd9a7ec700 1 --2- 172.21.15.92:0/3039850239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd94154550 0x7efd941552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:56.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.753+0000 7efd9a7ec700 1 -- 172.21.15.92:0/3039850239 >> 172.21.15.92:0/3039850239 conn(0x7efd940c9c10 msgr2=0x7efd9406dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:56.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.753+0000 7efd9a7ec700 1 -- 172.21.15.92:0/3039850239 shutdown_connections 2023-10-06T10:42:56.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.753+0000 7efd9a7ec700 1 -- 172.21.15.92:0/3039850239 wait complete. 2023-10-06T10:42:56.768 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T10:42:56.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.885+0000 7f074ca88700 1 Processor -- start 2023-10-06T10:42:56.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.885+0000 7f074ca88700 1 -- start start 2023-10-06T10:42:56.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.886+0000 7f074ca88700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f07481594d0 0x7f07480cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:56.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.886+0000 7f074ca88700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07480cbee0 con 0x7f07481598a0 2023-10-06T10:42:56.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.886+0000 7f074ca88700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07480cf5d0 con 0x7f07480cbaa0 2023-10-06T10:42:56.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.886+0000 7f074ca88700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07480cf8b0 con 0x7f07481594d0 2023-10-06T10:42:56.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.886+0000 7f074659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f07481594d0 0x7f07480cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:56.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.886+0000 7f073dd9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f07481598a0 0x7f07480ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53090/0 (socket says 172.21.15.92:53090) 2023-10-06T10:42:56.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.886+0000 7f073dd9b700 1 -- 172.21.15.92:0/72240864 learned_addr learned my addr 172.21.15.92:0/72240864 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:42:56.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.886+0000 7f074659c700 1 --2- 172.21.15.92:0/72240864 >> v2:172.21.15.92:3300/0 conn(0x7f07481594d0 0x7f07480cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37508/0 (socket says 172.21.15.92:37508) 2023-10-06T10:42:56.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.887+0000 7f074659c700 1 -- 172.21.15.92:0/72240864 >> v1:172.21.15.124:6789/0 conn(0x7f07480cbaa0 legacy=0x7f07480c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:56.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.887+0000 7f074659c700 1 -- 172.21.15.92:0/72240864 >> v1:172.21.15.92:6789/0 conn(0x7f07481598a0 legacy=0x7f07480ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:56.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.887+0000 7f074659c700 1 -- 172.21.15.92:0/72240864 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07480cfb90 con 0x7f07481594d0 2023-10-06T10:42:56.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.887+0000 7f0745d9b700 1 -- 172.21.15.92:0/72240864 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 77148124 0 0) 0x7f07480cbee0 con 0x7f07481598a0 2023-10-06T10:42:56.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.887+0000 7f074659c700 1 --2- 172.21.15.92:0/72240864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07481594d0 0x7f07480cb560 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f07380098e0 tx=0x7f07380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2723e098ac78f7a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:56.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.887+0000 7f0745d9b700 1 -- 172.21.15.92:0/72240864 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0738009e80 con 0x7f07481594d0 2023-10-06T10:42:56.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.887+0000 7f0745d9b700 1 -- 172.21.15.92:0/72240864 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f073800b820 con 0x7f07481594d0 2023-10-06T10:42:56.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.887+0000 7f0745d9b700 1 -- 172.21.15.92:0/72240864 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073800bbe0 con 0x7f07481594d0 2023-10-06T10:42:56.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.887+0000 7f074ca88700 1 -- 172.21.15.92:0/72240864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07481594d0 msgr2=0x7f07480cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:56.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.887+0000 7f074ca88700 1 --2- 172.21.15.92:0/72240864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07481594d0 0x7f07480cb560 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f07380098e0 tx=0x7f07380091a0 comp rx=0 tx=0).stop 2023-10-06T10:42:56.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.887+0000 7f074ca88700 1 -- 172.21.15.92:0/72240864 shutdown_connections 2023-10-06T10:42:56.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.887+0000 7f074ca88700 1 --2- 172.21.15.92:0/72240864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07481594d0 0x7f07480cb560 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:56.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.887+0000 7f074ca88700 1 -- 172.21.15.92:0/72240864 >> 172.21.15.92:0/72240864 conn(0x7f0748089e50 msgr2=0x7f0748099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:56.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.888+0000 7f074ca88700 1 -- 172.21.15.92:0/72240864 shutdown_connections 2023-10-06T10:42:56.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.888+0000 7f074ca88700 1 -- 172.21.15.92:0/72240864 wait complete. 2023-10-06T10:42:56.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.889+0000 7f074ca88700 1 Processor -- start 2023-10-06T10:42:56.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.889+0000 7f074ca88700 1 -- start start 2023-10-06T10:42:56.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.889+0000 7f074ca88700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07480cbaa0 0x7f074814f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:56.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.890+0000 7f074ca88700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07481594d0 0x7f0748158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:56.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.890+0000 7f074ca88700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07481598a0 0x7f0748068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:56.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.890+0000 7f074ca88700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074809ddd0 con 0x7f07481598a0 2023-10-06T10:42:56.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.890+0000 7f074ca88700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074814f880 con 0x7f07480cbaa0 2023-10-06T10:42:56.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.890+0000 7f074ca88700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f074809d1c0 con 0x7f07481594d0 2023-10-06T10:42:56.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.890+0000 7f0746d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07481598a0 0x7f0748068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:56.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.890+0000 7f0746d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07481598a0 0x7f0748068370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37510/0 (socket says 172.21.15.92:37510) 2023-10-06T10:42:56.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.890+0000 7f0746d9d700 1 -- 172.21.15.92:0/1160373391 learned_addr learned my addr 172.21.15.92:0/1160373391 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:56.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.890+0000 7f074659c700 1 --2- 172.21.15.92:0/1160373391 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07480cbaa0 0x7f074814f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:56.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.890+0000 7f073dd9b700 1 --2- 172.21.15.92:0/1160373391 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07481594d0 0x7f0748158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:56.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.890+0000 7f0746d9d700 1 -- 172.21.15.92:0/1160373391 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07481594d0 msgr2=0x7f0748158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:56.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.890+0000 7f0746d9d700 1 --2- 172.21.15.92:0/1160373391 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07481594d0 0x7f0748158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:56.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.890+0000 7f0746d9d700 1 -- 172.21.15.92:0/1160373391 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07480cbaa0 msgr2=0x7f074814f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:56.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.890+0000 7f0746d9d700 1 --2- 172.21.15.92:0/1160373391 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07480cbaa0 0x7f074814f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:56.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.890+0000 7f0746d9d700 1 -- 172.21.15.92:0/1160373391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0738009420 con 0x7f07481598a0 2023-10-06T10:42:56.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.890+0000 7f0746d9d700 1 --2- 172.21.15.92:0/1160373391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07481598a0 0x7f0748068370 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f074000c930 tx=0x7f07400026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:56.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.890+0000 7f073effd700 1 -- 172.21.15.92:0/1160373391 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0740012630 con 0x7f07481598a0 2023-10-06T10:42:56.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.890+0000 7f073effd700 1 -- 172.21.15.92:0/1160373391 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07400127b0 con 0x7f07481598a0 2023-10-06T10:42:56.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.890+0000 7f074ca88700 1 -- 172.21.15.92:0/1160373391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f074809d4a0 con 0x7f07481598a0 2023-10-06T10:42:56.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.890+0000 7f073effd700 1 -- 172.21.15.92:0/1160373391 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0740012b70 con 0x7f07481598a0 2023-10-06T10:42:56.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.890+0000 7f074ca88700 1 -- 172.21.15.92:0/1160373391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f074809da60 con 0x7f07481598a0 2023-10-06T10:42:56.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.891+0000 7f074ca88700 1 -- 172.21.15.92:0/1160373391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0748064a90 con 0x7f07481598a0 2023-10-06T10:42:56.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.892+0000 7f073effd700 1 -- 172.21.15.92:0/1160373391 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f074000a3b0 con 0x7f07481598a0 2023-10-06T10:42:56.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.892+0000 7f073effd700 1 --2- 172.21.15.92:0/1160373391 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0728066f30 0x7f07280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:56.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.892+0000 7f074659c700 1 --2- 172.21.15.92:0/1160373391 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0728066f30 0x7f07280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:56.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.892+0000 7f073effd700 1 -- 172.21.15.92:0/1160373391 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f07400841e0 con 0x7f07481598a0 2023-10-06T10:42:56.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.892+0000 7f074659c700 1 --2- 172.21.15.92:0/1160373391 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0728066f30 0x7f07280693e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f07380098e0 tx=0x7f0738016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:56.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:56.896+0000 7f073effd700 1 -- 172.21.15.92:0/1160373391 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0740051400 con 0x7f07481598a0 2023-10-06T10:42:57.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.079+0000 7f074ca88700 1 -- 172.21.15.92:0/1160373391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f07480cfe70 con 0x7f07481598a0 2023-10-06T10:42:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.085+0000 7f073effd700 1 -- 172.21.15.92:0/1160373391 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0740054a20 con 0x7f07481598a0 2023-10-06T10:42:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.087+0000 7f074ca88700 1 -- 172.21.15.92:0/1160373391 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0728066f30 msgr2=0x7f07280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.087+0000 7f074ca88700 1 --2- 172.21.15.92:0/1160373391 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0728066f30 0x7f07280693e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f07380098e0 tx=0x7f0738016000 comp rx=0 tx=0).stop 2023-10-06T10:42:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.087+0000 7f074ca88700 1 -- 172.21.15.92:0/1160373391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07481598a0 msgr2=0x7f0748068370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.087+0000 7f074ca88700 1 --2- 172.21.15.92:0/1160373391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07481598a0 0x7f0748068370 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f074000c930 tx=0x7f07400026e0 comp rx=0 tx=0).stop 2023-10-06T10:42:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.087+0000 7f074ca88700 1 -- 172.21.15.92:0/1160373391 shutdown_connections 2023-10-06T10:42:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.087+0000 7f074ca88700 1 --2- 172.21.15.92:0/1160373391 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0728066f30 0x7f07280693e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.087+0000 7f074ca88700 1 --2- 172.21.15.92:0/1160373391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07481598a0 0x7f0748068370 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.087+0000 7f074ca88700 1 --2- 172.21.15.92:0/1160373391 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07480cbaa0 0x7f074814f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.087+0000 7f074ca88700 1 --2- 172.21.15.92:0/1160373391 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07481594d0 0x7f0748158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.087+0000 7f074ca88700 1 -- 172.21.15.92:0/1160373391 >> 172.21.15.92:0/1160373391 conn(0x7f0748089e50 msgr2=0x7f07480d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.087+0000 7f074ca88700 1 -- 172.21.15.92:0/1160373391 shutdown_connections 2023-10-06T10:42:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.087+0000 7f074ca88700 1 -- 172.21.15.92:0/1160373391 wait complete. 2023-10-06T10:42:57.101 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T10:42:57.101 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:42:57.147 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:42:57.191 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T10:42:57.191 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T10:42:57.192 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:42:57.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.304+0000 7f808714d700 1 Processor -- start 2023-10-06T10:42:57.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.304+0000 7f808714d700 1 -- start start 2023-10-06T10:42:57.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.305+0000 7f808714d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8080159560 0x7f80800aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:57.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.305+0000 7f808714d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80800b0c60 con 0x7f80800b49a0 2023-10-06T10:42:57.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.305+0000 7f808714d700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80800b0f40 con 0x7f80800b0760 2023-10-06T10:42:57.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.305+0000 7f808714d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80800b1220 con 0x7f8080159560 2023-10-06T10:42:57.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.305+0000 7f8084ee9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8080159560 0x7f80800aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:57.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.305+0000 7f8084ee9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8080159560 0x7f80800aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37520/0 (socket says 172.21.15.92:37520) 2023-10-06T10:42:57.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.305+0000 7f8077fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f80800b49a0 0x7f80800aaa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53102/0 (socket says 172.21.15.92:53102) 2023-10-06T10:42:57.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.305+0000 7f8084ee9700 1 -- 172.21.15.92:0/1441852857 learned_addr learned my addr 172.21.15.92:0/1441852857 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:57.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.306+0000 7f8084ee9700 1 -- 172.21.15.92:0/1441852857 >> v1:172.21.15.124:6789/0 conn(0x7f80800b0760 legacy=0x7f80800b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:57.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.306+0000 7f8084ee9700 1 -- 172.21.15.92:0/1441852857 >> v1:172.21.15.92:6789/0 conn(0x7f80800b49a0 legacy=0x7f80800aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:57.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.306+0000 7f8084ee9700 1 -- 172.21.15.92:0/1441852857 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80800b1500 con 0x7f8080159560 2023-10-06T10:42:57.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.306+0000 7f80777fe700 1 -- 172.21.15.92:0/1441852857 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3933394106 0 0) 0x7f80800b0c60 con 0x7f80800b49a0 2023-10-06T10:42:57.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.306+0000 7f8084ee9700 1 --2- 172.21.15.92:0/1441852857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8080159560 0x7f80800aa530 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f807c0099c0 tx=0x7f807c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6dc03d373c18 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:57.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.306+0000 7f80777fe700 1 -- 172.21.15.92:0/1441852857 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c005c10 con 0x7f8080159560 2023-10-06T10:42:57.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.306+0000 7f80777fe700 1 -- 172.21.15.92:0/1441852857 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807c005d70 con 0x7f8080159560 2023-10-06T10:42:57.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.306+0000 7f80777fe700 1 -- 172.21.15.92:0/1441852857 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c00b840 con 0x7f8080159560 2023-10-06T10:42:57.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.306+0000 7f808714d700 1 -- 172.21.15.92:0/1441852857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8080159560 msgr2=0x7f80800aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:57.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.306+0000 7f808714d700 1 --2- 172.21.15.92:0/1441852857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8080159560 0x7f80800aa530 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f807c0099c0 tx=0x7f807c009220 comp rx=0 tx=0).stop 2023-10-06T10:42:57.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.307+0000 7f808714d700 1 -- 172.21.15.92:0/1441852857 shutdown_connections 2023-10-06T10:42:57.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.307+0000 7f808714d700 1 --2- 172.21.15.92:0/1441852857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8080159560 0x7f80800aa530 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:57.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.307+0000 7f808714d700 1 -- 172.21.15.92:0/1441852857 >> 172.21.15.92:0/1441852857 conn(0x7f80800997f0 msgr2=0x7f808006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:57.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.307+0000 7f808714d700 1 -- 172.21.15.92:0/1441852857 shutdown_connections 2023-10-06T10:42:57.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.307+0000 7f808714d700 1 -- 172.21.15.92:0/1441852857 wait complete. 2023-10-06T10:42:57.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.308+0000 7f808714d700 1 Processor -- start 2023-10-06T10:42:57.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.308+0000 7f808714d700 1 -- start start 2023-10-06T10:42:57.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.308+0000 7f808714d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f80800b0760 0x7f8080147180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:57.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.309+0000 7f808714d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80800b49a0 0x7f808015bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:57.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.309+0000 7f808714d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8080159560 0x7f808015c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:57.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.309+0000 7f808714d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808009ab00 con 0x7f80800b49a0 2023-10-06T10:42:57.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.309+0000 7f808714d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808009a1d0 con 0x7f8080159560 2023-10-06T10:42:57.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.309+0000 7f808714d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808009a4b0 con 0x7f80800b0760 2023-10-06T10:42:57.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.309+0000 7f8077fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80800b49a0 0x7f808015bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:57.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.309+0000 7f8077fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80800b49a0 0x7f808015bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37522/0 (socket says 172.21.15.92:37522) 2023-10-06T10:42:57.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.309+0000 7f8077fff700 1 -- 172.21.15.92:0/3179085630 learned_addr learned my addr 172.21.15.92:0/3179085630 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:57.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.309+0000 7f80856ea700 1 --2- 172.21.15.92:0/3179085630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8080159560 0x7f808015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:57.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.309+0000 7f8084ee9700 1 --2- 172.21.15.92:0/3179085630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f80800b0760 0x7f8080147180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:57.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.309+0000 7f8077fff700 1 -- 172.21.15.92:0/3179085630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f80800b0760 msgr2=0x7f8080147180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:57.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.309+0000 7f8077fff700 1 --2- 172.21.15.92:0/3179085630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f80800b0760 0x7f8080147180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:57.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.309+0000 7f8077fff700 1 -- 172.21.15.92:0/3179085630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8080159560 msgr2=0x7f808015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:57.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.309+0000 7f8077fff700 1 --2- 172.21.15.92:0/3179085630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8080159560 0x7f808015c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:57.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.309+0000 7f8077fff700 1 -- 172.21.15.92:0/3179085630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c009480 con 0x7f80800b49a0 2023-10-06T10:42:57.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.309+0000 7f8084ee9700 1 --2- 172.21.15.92:0/3179085630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f80800b0760 0x7f8080147180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:57.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.309+0000 7f8077fff700 1 --2- 172.21.15.92:0/3179085630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80800b49a0 0x7f808015bce0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f807000c5b0 tx=0x7f8070007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:57.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.309+0000 7f8074ff9700 1 -- 172.21.15.92:0/3179085630 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070005a70 con 0x7f80800b49a0 2023-10-06T10:42:57.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.309+0000 7f8074ff9700 1 -- 172.21.15.92:0/3179085630 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8070005bf0 con 0x7f80800b49a0 2023-10-06T10:42:57.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.309+0000 7f808714d700 1 -- 172.21.15.92:0/3179085630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808009a790 con 0x7f80800b49a0 2023-10-06T10:42:57.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.309+0000 7f8074ff9700 1 -- 172.21.15.92:0/3179085630 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80700070d0 con 0x7f80800b49a0 2023-10-06T10:42:57.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.309+0000 7f808714d700 1 -- 172.21.15.92:0/3179085630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80801476c0 con 0x7f80800b49a0 2023-10-06T10:42:57.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.310+0000 7f808714d700 1 -- 172.21.15.92:0/3179085630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8080147880 con 0x7f80800b49a0 2023-10-06T10:42:57.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.311+0000 7f8074ff9700 1 -- 172.21.15.92:0/3179085630 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8070007440 con 0x7f80800b49a0 2023-10-06T10:42:57.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.311+0000 7f8074ff9700 1 --2- 172.21.15.92:0/3179085630 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f806c066f30 0x7f806c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:57.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.311+0000 7f8074ff9700 1 -- 172.21.15.92:0/3179085630 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f8070083a30 con 0x7f80800b49a0 2023-10-06T10:42:57.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.311+0000 7f8084ee9700 1 --2- 172.21.15.92:0/3179085630 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f806c066f30 0x7f806c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:57.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.311+0000 7f8084ee9700 1 --2- 172.21.15.92:0/3179085630 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f806c066f30 0x7f806c0693e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f807c009260 tx=0x7f807c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:57.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.315+0000 7f8074ff9700 1 -- 172.21.15.92:0/3179085630 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8070050c50 con 0x7f80800b49a0 2023-10-06T10:42:57.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.540+0000 7f808714d700 1 -- 172.21.15.92:0/3179085630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8080064a90 con 0x7f80800b49a0 2023-10-06T10:42:57.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.541+0000 7f8074ff9700 1 -- 172.21.15.92:0/3179085630 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f8070017020 con 0x7f80800b49a0 2023-10-06T10:42:57.543 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:42:57.543 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":61,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/1409495728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":1409495728},{"type":"v1","addr":"172.21.15.124:6834","nonce":1409495728}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6394,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1668742194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":1668742194},{"type":"v1","addr":"172.21.15.124:6835","nonce":1668742194}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":7071,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1434107144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1434107144},{"type":"v1","addr":"172.21.15.92:6835","nonce":1434107144}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":7080,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2171063595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2171063595},{"type":"v1","addr":"172.21.15.92:6837","nonce":2171063595}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[]} 2023-10-06T10:42:57.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.543+0000 7f808714d700 1 -- 172.21.15.92:0/3179085630 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f806c066f30 msgr2=0x7f806c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:57.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.543+0000 7f808714d700 1 --2- 172.21.15.92:0/3179085630 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f806c066f30 0x7f806c0693e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f807c009260 tx=0x7f807c016000 comp rx=0 tx=0).stop 2023-10-06T10:42:57.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.543+0000 7f808714d700 1 -- 172.21.15.92:0/3179085630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80800b49a0 msgr2=0x7f808015bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:57.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.543+0000 7f808714d700 1 --2- 172.21.15.92:0/3179085630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80800b49a0 0x7f808015bce0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f807000c5b0 tx=0x7f8070007d30 comp rx=0 tx=0).stop 2023-10-06T10:42:57.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.543+0000 7f808714d700 1 -- 172.21.15.92:0/3179085630 shutdown_connections 2023-10-06T10:42:57.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.543+0000 7f808714d700 1 --2- 172.21.15.92:0/3179085630 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f806c066f30 0x7f806c0693e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:57.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.543+0000 7f808714d700 1 --2- 172.21.15.92:0/3179085630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8080159560 0x7f808015c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:57.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.543+0000 7f808714d700 1 --2- 172.21.15.92:0/3179085630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f80800b0760 0x7f8080147180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:57.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.543+0000 7f808714d700 1 --2- 172.21.15.92:0/3179085630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80800b49a0 0x7f808015bce0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:57.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.543+0000 7f808714d700 1 -- 172.21.15.92:0/3179085630 >> 172.21.15.92:0/3179085630 conn(0x7f80800997f0 msgr2=0x7f80800683c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:57.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.543+0000 7f808714d700 1 -- 172.21.15.92:0/3179085630 shutdown_connections 2023-10-06T10:42:57.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.543+0000 7f808714d700 1 -- 172.21.15.92:0/3179085630 wait complete. 2023-10-06T10:42:57.547 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 61 2023-10-06T10:42:57.557 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T10:42:57.558 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T10:42:57.558 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T10:42:57.558 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:42:57.558 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T10:42:57.558 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T10:42:57.559 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T10:42:57.559 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:42:57.559 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T10:42:57.559 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T10:42:57.559 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T10:42:57.559 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:42:57.559 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T10:42:57.560 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T10:42:57.560 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T10:42:57.560 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:42:57.566 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T10:42:57.567 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T10:42:57.567 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T10:42:57.568 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T10:42:57.571 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T10:42:57.572 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T10:42:57.573 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T10:42:57.573 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T10:42:57.609 INFO:tasks.ceph.mds.a:Started 2023-10-06T10:42:57.609 DEBUG:teuthology.parallel:result is None 2023-10-06T10:42:57.610 INFO:tasks.ceph.mds.c:Started 2023-10-06T10:42:57.610 DEBUG:teuthology.parallel:result is None 2023-10-06T10:42:57.613 INFO:tasks.ceph.mds.b:Started 2023-10-06T10:42:57.614 DEBUG:teuthology.parallel:result is None 2023-10-06T10:42:57.614 INFO:tasks.ceph.mds.d:Started 2023-10-06T10:42:57.614 DEBUG:teuthology.parallel:result is None 2023-10-06T10:42:57.615 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T10:42:57.682 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T10:42:57.683 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T10:42:57.694 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-06T10:42:57.694 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-06T10:42:57.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.733+0000 7face56d6700 1 Processor -- start 2023-10-06T10:42:57.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.733+0000 7face56d6700 1 -- start start 2023-10-06T10:42:57.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.734+0000 7face56d6700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7face0067cc0 0x7face006e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:57.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.734+0000 7face56d6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7face00ca940 0x7face00b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:57.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.734+0000 7face56d6700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face006bcc0 con 0x7face0157600 2023-10-06T10:42:57.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.734+0000 7face56d6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face00b06e0 con 0x7face00ca940 2023-10-06T10:42:57.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.734+0000 7face56d6700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face00ca6f0 con 0x7face0067cc0 2023-10-06T10:42:57.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.734+0000 7facdf7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7face00ca940 0x7face00b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:57.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.734+0000 7facdf7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7face00ca940 0x7face00b01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37572/0 (socket says 172.21.15.92:37572) 2023-10-06T10:42:57.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.734+0000 7facdf7fe700 1 -- 172.21.15.92:0/1517087206 learned_addr learned my addr 172.21.15.92:0/1517087206 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:57.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.735+0000 7facde7fc700 1 --2- 172.21.15.92:0/1517087206 >> v2:172.21.15.124:3300/0 conn(0x7face0067cc0 0x7face006e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:57.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.735+0000 7facdf7fe700 1 -- 172.21.15.92:0/1517087206 >> v2:172.21.15.124:3300/0 conn(0x7face0067cc0 msgr2=0x7face006e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:57.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.735+0000 7facdf7fe700 1 --2- 172.21.15.92:0/1517087206 >> v2:172.21.15.124:3300/0 conn(0x7face0067cc0 0x7face006e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:57.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.735+0000 7facdf7fe700 1 -- 172.21.15.92:0/1517087206 >> v1:172.21.15.124:6789/0 conn(0x7face0157600 legacy=0x7face00675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:57.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.735+0000 7facdf7fe700 1 -- 172.21.15.92:0/1517087206 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face006ecd0 con 0x7face00ca940 2023-10-06T10:42:57.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.735+0000 7facdf7fe700 1 --2- 172.21.15.92:0/1517087206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7face00ca940 0x7face00b01a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7face00ac970 tx=0x7facd8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9140a3ae1077105 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:57.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.735+0000 7facddffb700 1 -- 172.21.15.92:0/1517087206 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd800e070 con 0x7face00ca940 2023-10-06T10:42:57.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.735+0000 7facddffb700 1 -- 172.21.15.92:0/1517087206 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd800ba10 con 0x7face00ca940 2023-10-06T10:42:57.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.735+0000 7facddffb700 1 -- 172.21.15.92:0/1517087206 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd80092e0 con 0x7face00ca940 2023-10-06T10:42:57.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.735+0000 7face56d6700 1 -- 172.21.15.92:0/1517087206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7face00ca940 msgr2=0x7face00b01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:57.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.735+0000 7face56d6700 1 --2- 172.21.15.92:0/1517087206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7face00ca940 0x7face00b01a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7face00ac970 tx=0x7facd8009840 comp rx=0 tx=0).stop 2023-10-06T10:42:57.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.735+0000 7face56d6700 1 -- 172.21.15.92:0/1517087206 shutdown_connections 2023-10-06T10:42:57.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.735+0000 7face56d6700 1 --2- 172.21.15.92:0/1517087206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7face00ca940 0x7face00b01a0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:57.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.735+0000 7face56d6700 1 --2- 172.21.15.92:0/1517087206 >> v2:172.21.15.124:3300/0 conn(0x7face0067cc0 0x7face006e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:57.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.735+0000 7face56d6700 1 -- 172.21.15.92:0/1517087206 >> 172.21.15.92:0/1517087206 conn(0x7face00cdeb0 msgr2=0x7face00c9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:57.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.735+0000 7face56d6700 1 -- 172.21.15.92:0/1517087206 shutdown_connections 2023-10-06T10:42:57.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.735+0000 7face56d6700 1 -- 172.21.15.92:0/1517087206 wait complete. 2023-10-06T10:42:57.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.737+0000 7face56d6700 1 Processor -- start 2023-10-06T10:42:57.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.737+0000 7face56d6700 1 -- start start 2023-10-06T10:42:57.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.737+0000 7face56d6700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7face0067cc0 0x7face00c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.737+0000 7face56d6700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7face00ca940 0x7face00c4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.737+0000 7face56d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7face0157600 0x7face00c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.737+0000 7face56d6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face0159b70 con 0x7face0157600 2023-10-06T10:42:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.737+0000 7face56d6700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face0158f60 con 0x7face0067cc0 2023-10-06T10:42:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.737+0000 7face56d6700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face0159240 con 0x7face00ca940 2023-10-06T10:42:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.737+0000 7facdf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7face0157600 0x7face00c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.738+0000 7facdf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7face0157600 0x7face00c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37574/0 (socket says 172.21.15.92:37574) 2023-10-06T10:42:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.738+0000 7facdf7fe700 1 -- 172.21.15.92:0/1237771151 learned_addr learned my addr 172.21.15.92:0/1237771151 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.738+0000 7facde7fc700 1 --2- 172.21.15.92:0/1237771151 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7face00ca940 0x7face00c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.738+0000 7facdeffd700 1 --2- 172.21.15.92:0/1237771151 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7face0067cc0 0x7face00c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.738+0000 7facdf7fe700 1 -- 172.21.15.92:0/1237771151 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7face00ca940 msgr2=0x7face00c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:57.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.738+0000 7facdf7fe700 1 --2- 172.21.15.92:0/1237771151 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7face00ca940 0x7face00c4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:57.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.738+0000 7facdf7fe700 1 -- 172.21.15.92:0/1237771151 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7face0067cc0 msgr2=0x7face00c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:57.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.738+0000 7facdf7fe700 1 --2- 172.21.15.92:0/1237771151 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7face0067cc0 0x7face00c7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:57.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.738+0000 7facdf7fe700 1 -- 172.21.15.92:0/1237771151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd8009ae0 con 0x7face0157600 2023-10-06T10:42:57.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.738+0000 7facdf7fe700 1 --2- 172.21.15.92:0/1237771151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7face0157600 0x7face00c49d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7face006dce0 tx=0x7facd8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:57.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.738+0000 7facc6ffd700 1 -- 172.21.15.92:0/1237771151 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd80092e0 con 0x7face0157600 2023-10-06T10:42:57.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.738+0000 7facc6ffd700 1 -- 172.21.15.92:0/1237771151 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd8012d00 con 0x7face0157600 2023-10-06T10:42:57.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.738+0000 7face56d6700 1 -- 172.21.15.92:0/1237771151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face0159520 con 0x7face0157600 2023-10-06T10:42:57.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.738+0000 7facc6ffd700 1 -- 172.21.15.92:0/1237771151 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd801d650 con 0x7face0157600 2023-10-06T10:42:57.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.738+0000 7face56d6700 1 -- 172.21.15.92:0/1237771151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face00c7880 con 0x7face0157600 2023-10-06T10:42:57.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.739+0000 7face56d6700 1 -- 172.21.15.92:0/1237771151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face0159d20 con 0x7face0157600 2023-10-06T10:42:57.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.740+0000 7facc6ffd700 1 -- 172.21.15.92:0/1237771151 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7facd801da00 con 0x7face0157600 2023-10-06T10:42:57.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.740+0000 7facc6ffd700 1 --2- 172.21.15.92:0/1237771151 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7faccc066f30 0x7faccc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:57.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.740+0000 7facc6ffd700 1 -- 172.21.15.92:0/1237771151 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5784+0+0 (secure 0 0 0) 0x7facd8085d30 con 0x7face0157600 2023-10-06T10:42:57.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.740+0000 7facdeffd700 1 --2- 172.21.15.92:0/1237771151 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7faccc066f30 0x7faccc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:57.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.740+0000 7facdeffd700 1 --2- 172.21.15.92:0/1237771151 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7faccc066f30 0x7faccc0693e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7facd0003a30 tx=0x7facd0007ea0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:57.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.744+0000 7facc6ffd700 1 -- 172.21.15.92:0/1237771151 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7face0159d20 con 0x7face0157600 2023-10-06T10:42:57.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:57.944+0000 7face56d6700 1 -- 172.21.15.92:0/1237771151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7face00c7aa0 con 0x7face0157600 2023-10-06T10:42:58.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.768+0000 7facc6ffd700 1 -- 172.21.15.92:0/1237771151 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v74) v1 ==== 97+0+0 (secure 0 0 0) 0x7facd8052f50 con 0x7face0157600 2023-10-06T10:42:58.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.771+0000 7face56d6700 1 -- 172.21.15.92:0/1237771151 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7faccc066f30 msgr2=0x7faccc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:58.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.771+0000 7face56d6700 1 --2- 172.21.15.92:0/1237771151 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7faccc066f30 0x7faccc0693e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7facd0003a30 tx=0x7facd0007ea0 comp rx=0 tx=0).stop 2023-10-06T10:42:58.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.771+0000 7face56d6700 1 -- 172.21.15.92:0/1237771151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7face0157600 msgr2=0x7face00c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:58.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.771+0000 7face56d6700 1 --2- 172.21.15.92:0/1237771151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7face0157600 0x7face00c49d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7face006dce0 tx=0x7facd8017000 comp rx=0 tx=0).stop 2023-10-06T10:42:58.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.771+0000 7face56d6700 1 -- 172.21.15.92:0/1237771151 shutdown_connections 2023-10-06T10:42:58.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.771+0000 7face56d6700 1 --2- 172.21.15.92:0/1237771151 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7faccc066f30 0x7faccc0693e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:58.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.771+0000 7face56d6700 1 --2- 172.21.15.92:0/1237771151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7face0157600 0x7face00c49d0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:58.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.771+0000 7face56d6700 1 --2- 172.21.15.92:0/1237771151 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7face0067cc0 0x7face00c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:58.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.771+0000 7face56d6700 1 --2- 172.21.15.92:0/1237771151 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7face00ca940 0x7face00c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:58.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.771+0000 7face56d6700 1 -- 172.21.15.92:0/1237771151 >> 172.21.15.92:0/1237771151 conn(0x7face00cdeb0 msgr2=0x7face0151ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:58.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.771+0000 7face56d6700 1 -- 172.21.15.92:0/1237771151 shutdown_connections 2023-10-06T10:42:58.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.771+0000 7face56d6700 1 -- 172.21.15.92:0/1237771151 wait complete. 2023-10-06T10:42:58.774 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-10-06T10:42:58.786 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T10:42:58.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.899+0000 7f06a39a1700 1 Processor -- start 2023-10-06T10:42:58.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.899+0000 7f06a39a1700 1 -- start start 2023-10-06T10:42:58.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.900+0000 7f06a39a1700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f069c0ca370 0x7f069c158270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:58.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.900+0000 7f06a39a1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f069c0adb60 0x7f069c0ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:58.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.900+0000 7f06a39a1700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069c157de0 con 0x7f069c0ce6d0 2023-10-06T10:42:58.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.900+0000 7f06a39a1700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f069c0cbb90 con 0x7f069c0adb60 2023-10-06T10:42:58.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.900+0000 7f06a39a1700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f069c0ceb10 con 0x7f069c0ca370 2023-10-06T10:42:58.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.900+0000 7f06a0f3c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f069c0adb60 0x7f069c0ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:58.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.900+0000 7f06a0f3c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f069c0adb60 0x7f069c0ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37584/0 (socket says 172.21.15.92:37584) 2023-10-06T10:42:58.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.900+0000 7f06a0f3c700 1 -- 172.21.15.92:0/1689700790 learned_addr learned my addr 172.21.15.92:0/1689700790 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:58.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.900+0000 7f06a173d700 1 --2- 172.21.15.92:0/1689700790 >> v2:172.21.15.124:3300/0 conn(0x7f069c0ca370 0x7f069c158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:58.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.901+0000 7f06a0f3c700 1 -- 172.21.15.92:0/1689700790 >> v2:172.21.15.124:3300/0 conn(0x7f069c0ca370 msgr2=0x7f069c158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:58.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.901+0000 7f06a0f3c700 1 --2- 172.21.15.92:0/1689700790 >> v2:172.21.15.124:3300/0 conn(0x7f069c0ca370 0x7f069c158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:58.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.901+0000 7f06a0f3c700 1 -- 172.21.15.92:0/1689700790 >> v1:172.21.15.124:6789/0 conn(0x7f069c0ce6d0 legacy=0x7f069c15aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:58.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.901+0000 7f06a0f3c700 1 -- 172.21.15.92:0/1689700790 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c0cb5d0 con 0x7f069c0adb60 2023-10-06T10:42:58.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.901+0000 7f06a0f3c700 1 --2- 172.21.15.92:0/1689700790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f069c0adb60 0x7f069c0ce190 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f069c06f2c0 tx=0x7f068c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5ece590d00e71b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:58.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.901+0000 7f0693fff700 1 -- 172.21.15.92:0/1689700790 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c009e70 con 0x7f069c0adb60 2023-10-06T10:42:58.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.901+0000 7f0693fff700 1 -- 172.21.15.92:0/1689700790 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068c00b860 con 0x7f069c0adb60 2023-10-06T10:42:58.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.901+0000 7f0693fff700 1 -- 172.21.15.92:0/1689700790 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c00bc00 con 0x7f069c0adb60 2023-10-06T10:42:58.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.901+0000 7f06a39a1700 1 -- 172.21.15.92:0/1689700790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f069c0adb60 msgr2=0x7f069c0ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:58.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.901+0000 7f06a39a1700 1 --2- 172.21.15.92:0/1689700790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f069c0adb60 0x7f069c0ce190 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f069c06f2c0 tx=0x7f068c009250 comp rx=0 tx=0).stop 2023-10-06T10:42:58.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.901+0000 7f06a39a1700 1 -- 172.21.15.92:0/1689700790 shutdown_connections 2023-10-06T10:42:58.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.901+0000 7f06a39a1700 1 --2- 172.21.15.92:0/1689700790 >> v2:172.21.15.124:3300/0 conn(0x7f069c0ca370 0x7f069c158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:58.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.901+0000 7f06a39a1700 1 --2- 172.21.15.92:0/1689700790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f069c0adb60 0x7f069c0ce190 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:58.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.901+0000 7f06a39a1700 1 -- 172.21.15.92:0/1689700790 >> 172.21.15.92:0/1689700790 conn(0x7f069c0707d0 msgr2=0x7f069c070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:58.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.902+0000 7f06a39a1700 1 -- 172.21.15.92:0/1689700790 shutdown_connections 2023-10-06T10:42:58.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.902+0000 7f06a39a1700 1 -- 172.21.15.92:0/1689700790 wait complete. 2023-10-06T10:42:58.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.903+0000 7f06a39a1700 1 Processor -- start 2023-10-06T10:42:58.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.903+0000 7f06a39a1700 1 -- start start 2023-10-06T10:42:58.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.903+0000 7f06a39a1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f069c0adb60 0x7f069c0c3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:58.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.903+0000 7f06a39a1700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f069c0ca370 0x7f069c0c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:58.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.903+0000 7f06a39a1700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f069c0ce6d0 0x7f069c0b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:58.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.904+0000 7f06a39a1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c0c4550 con 0x7f069c0adb60 2023-10-06T10:42:58.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.904+0000 7f06a39a1700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c0c4830 con 0x7f069c0ca370 2023-10-06T10:42:58.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.904+0000 7f06a39a1700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069c0c4b10 con 0x7f069c0ce6d0 2023-10-06T10:42:58.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.904+0000 7f06a173d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f069c0adb60 0x7f069c0c3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:58.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.904+0000 7f06a173d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f069c0adb60 0x7f069c0c3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37600/0 (socket says 172.21.15.92:37600) 2023-10-06T10:42:58.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.904+0000 7f06a173d700 1 -- 172.21.15.92:0/2807878346 learned_addr learned my addr 172.21.15.92:0/2807878346 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:58.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.904+0000 7f06a0f3c700 1 --2- 172.21.15.92:0/2807878346 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f069c0ca370 0x7f069c0c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:58.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.904+0000 7f06a1f3e700 1 --2- 172.21.15.92:0/2807878346 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f069c0ce6d0 0x7f069c0b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:58.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.904+0000 7f06a173d700 1 -- 172.21.15.92:0/2807878346 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f069c0ce6d0 msgr2=0x7f069c0b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:58.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.904+0000 7f06a173d700 1 --2- 172.21.15.92:0/2807878346 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f069c0ce6d0 0x7f069c0b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:58.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.904+0000 7f06a173d700 1 -- 172.21.15.92:0/2807878346 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f069c0ca370 msgr2=0x7f069c0c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:58.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.904+0000 7f06a173d700 1 --2- 172.21.15.92:0/2807878346 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f069c0ca370 0x7f069c0c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:58.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.904+0000 7f06a173d700 1 -- 172.21.15.92:0/2807878346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068c0094f0 con 0x7f069c0adb60 2023-10-06T10:42:58.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.904+0000 7f06a0f3c700 1 --2- 172.21.15.92:0/2807878346 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f069c0ca370 0x7f069c0c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:58.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.904+0000 7f06a173d700 1 --2- 172.21.15.92:0/2807878346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f069c0adb60 0x7f069c0c3bb0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f069800e930 tx=0x7f06980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:58.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.904+0000 7f06917fa700 1 -- 172.21.15.92:0/2807878346 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069800eed0 con 0x7f069c0adb60 2023-10-06T10:42:58.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.904+0000 7f06917fa700 1 -- 172.21.15.92:0/2807878346 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069800c850 con 0x7f069c0adb60 2023-10-06T10:42:58.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.904+0000 7f06a39a1700 1 -- 172.21.15.92:0/2807878346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069c0c4df0 con 0x7f069c0adb60 2023-10-06T10:42:58.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.904+0000 7f06917fa700 1 -- 172.21.15.92:0/2807878346 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069800cbf0 con 0x7f069c0adb60 2023-10-06T10:42:58.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.904+0000 7f06a39a1700 1 -- 172.21.15.92:0/2807878346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069c0c40f0 con 0x7f069c0adb60 2023-10-06T10:42:58.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.905+0000 7f06a39a1700 1 -- 172.21.15.92:0/2807878346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069c0c4f80 con 0x7f069c0adb60 2023-10-06T10:42:58.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.905+0000 7f06917fa700 1 -- 172.21.15.92:0/2807878346 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0698013470 con 0x7f069c0adb60 2023-10-06T10:42:58.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.906+0000 7f06917fa700 1 --2- 172.21.15.92:0/2807878346 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0680066f30 0x7f06800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:58.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.906+0000 7f06917fa700 1 -- 172.21.15.92:0/2807878346 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f06980846f0 con 0x7f069c0adb60 2023-10-06T10:42:58.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.906+0000 7f06a0f3c700 1 --2- 172.21.15.92:0/2807878346 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0680066f30 0x7f06800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:58.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.906+0000 7f06a0f3c700 1 --2- 172.21.15.92:0/2807878346 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0680066f30 0x7f06800693e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f069c06f2c0 tx=0x7f068c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:58.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:58.910+0000 7f06917fa700 1 -- 172.21.15.92:0/2807878346 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0698051a10 con 0x7f069c0adb60 2023-10-06T10:42:59.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.164+0000 7f06a39a1700 1 -- 172.21.15.92:0/2807878346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f069c064a90 con 0x7f069c0adb60 2023-10-06T10:42:59.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.165+0000 7f06917fa700 1 -- 172.21.15.92:0/2807878346 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0698055030 con 0x7f069c0adb60 2023-10-06T10:42:59.167 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:42:59.167 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:42:59.167 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-10-06T10:42:59.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-10-06T10:42:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lc28gCBAAVrjhXIawHuZEox0PE3/Rsg==", 2023-10-06T10:42:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:42:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:42:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:42:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:42:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:42:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:42:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-10-06T10:42:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lQpdOExAAm44s0n4Jekhy8AJk/o6Waw==", 2023-10-06T10:42:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:42:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:42:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:42:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:42:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:42:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:42:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-10-06T10:42:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lrCrsDhAAvp++wvvnSpZRGzvRTrGWEw==", 2023-10-06T10:42:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:42:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:42:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:42:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:42:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:42:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:42:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-10-06T10:42:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9l/DgWGhAAy3JMA5ntn/UaRkuTCyPdkg==", 2023-10-06T10:42:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:42:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:42:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:42:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:42:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:42:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:42:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-10-06T10:42:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBo4x9lDEIFMBAAc3DY1fYpY2UiY7MX7qSP5A==", 2023-10-06T10:42:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:42:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:42:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:42:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:42:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:42:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-10-06T10:42:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBr4x9lKzX8LRAApryLtFGkK6urkE8W70l5Cg==", 2023-10-06T10:42:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:42:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:42:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:42:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:42:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:42:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-10-06T10:42:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBu4x9lCMZpLhAAEBPa40Wnb820p6j+lv+PsA==", 2023-10-06T10:42:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:42:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:42:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:42:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:42:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:42:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-10-06T10:42:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBx4x9l4AHrLRAAFn6SnNkHnOiVXmmiU8qGyA==", 2023-10-06T10:42:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:42:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:42:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:42:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:42:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:42:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-10-06T10:42:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB34x9l7V7WMBAAvs5kvBP5n0Th478l/idjeQ==", 2023-10-06T10:42:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:42:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:42:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:42:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:42:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:42:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-10-06T10:42:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB64x9lfHIlLhAADCuqt+O/zPEETgoj//hwVg==", 2023-10-06T10:42:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:42:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:42:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:42:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:42:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:42:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-10-06T10:42:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB94x9lUmU7LhAAvxmtGmJxrI8TlOOD7taHOQ==", 2023-10-06T10:42:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:42:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:42:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:42:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:42:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:42:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-10-06T10:42:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQCA4x9lJE3nLBAAnREk9yNK+FhX4fnG+Za5Ug==", 2023-10-06T10:42:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:42:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:42:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:42:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:42:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:42:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-10-06T10:42:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lMePcHRAAMyftXWJWk7rwmWFkq18Rqg==", 2023-10-06T10:42:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:42:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:42:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-10-06T10:42:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T10:42:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:42:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-10-06T10:42:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lp1s0JhAAKvZ23MYjObxOK83q3UqJBQ==", 2023-10-06T10:42:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:42:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:42:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T10:42:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T10:42:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T10:42:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:42:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-10-06T10:42:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lc6o9IxAAaOY7S5rYn+OdeMNWK1UYMA==", 2023-10-06T10:42:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:42:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:42:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T10:42:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T10:42:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T10:42:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:42:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-10-06T10:42:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lwoyOKxAA811lehuQQ8MQQQzuTFAApg==", 2023-10-06T10:42:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:42:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:42:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T10:42:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T10:42:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T10:42:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:42:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-10-06T10:42:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBk4x9lONCoJhAAepF/sou/5M2GBvvfNAs5wQ==", 2023-10-06T10:42:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:42:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T10:42:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:42:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:42:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:42:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:42:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-10-06T10:42:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9ll8wgBRAAQUrHERQ3Igv1tosa+wryIg==", 2023-10-06T10:42:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:42:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T10:42:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T10:42:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:42:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:42:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:42:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-10-06T10:42:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBk4x9lgYa/OhAAyoI717Ga+Wv72srZ4FK+lQ==", 2023-10-06T10:42:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:42:59.182 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T10:42:59.182 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T10:42:59.182 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:42:59.182 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:59.182 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:42:59.182 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T10:42:59.182 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:42:59.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.167+0000 7f06a39a1700 1 -- 172.21.15.92:0/2807878346 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0680066f30 msgr2=0x7f06800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:59.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.167+0000 7f06a39a1700 1 --2- 172.21.15.92:0/2807878346 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0680066f30 0x7f06800693e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f069c06f2c0 tx=0x7f068c017000 comp rx=0 tx=0).stop 2023-10-06T10:42:59.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.167+0000 7f06a39a1700 1 -- 172.21.15.92:0/2807878346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f069c0adb60 msgr2=0x7f069c0c3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:59.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.167+0000 7f06a39a1700 1 --2- 172.21.15.92:0/2807878346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f069c0adb60 0x7f069c0c3bb0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f069800e930 tx=0x7f06980091a0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.167+0000 7f06a39a1700 1 -- 172.21.15.92:0/2807878346 shutdown_connections 2023-10-06T10:42:59.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.167+0000 7f06a39a1700 1 --2- 172.21.15.92:0/2807878346 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0680066f30 0x7f06800693e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.167+0000 7f06a39a1700 1 --2- 172.21.15.92:0/2807878346 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f069c0ce6d0 0x7f069c0b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.167+0000 7f06a39a1700 1 --2- 172.21.15.92:0/2807878346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f069c0adb60 0x7f069c0c3bb0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.167+0000 7f06a39a1700 1 --2- 172.21.15.92:0/2807878346 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f069c0ca370 0x7f069c0c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.167+0000 7f06a39a1700 1 -- 172.21.15.92:0/2807878346 >> 172.21.15.92:0/2807878346 conn(0x7f069c0707d0 msgr2=0x7f069c15a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:59.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.167+0000 7f06a39a1700 1 -- 172.21.15.92:0/2807878346 shutdown_connections 2023-10-06T10:42:59.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.167+0000 7f06a39a1700 1 -- 172.21.15.92:0/2807878346 wait complete. 2023-10-06T10:42:59.185 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:42:59.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.296+0000 7fc7b752c700 1 Processor -- start 2023-10-06T10:42:59.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.297+0000 7fc7b752c700 1 -- start start 2023-10-06T10:42:59.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.297+0000 7fc7b752c700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc7b0156500 0x7fc7b00c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:59.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.297+0000 7fc7b752c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc7b006ed60 0x7fc7b00ad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:59.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.297+0000 7fc7b752c700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b00acd20 con 0x7fc7b00c7570 2023-10-06T10:42:59.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.297+0000 7fc7b752c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7b006f430 con 0x7fc7b006ed60 2023-10-06T10:42:59.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.297+0000 7fc7b752c700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7b01568d0 con 0x7fc7b0156500 2023-10-06T10:42:59.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.297+0000 7fc7b5ac9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc7b006ed60 0x7fc7b00ad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:59.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.297+0000 7fc7b5ac9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc7b006ed60 0x7fc7b00ad1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37604/0 (socket says 172.21.15.92:37604) 2023-10-06T10:42:59.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.298+0000 7fc7b5ac9700 1 -- 172.21.15.92:0/4158891844 learned_addr learned my addr 172.21.15.92:0/4158891844 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.298+0000 7fc7b52c8700 1 --2- 172.21.15.92:0/4158891844 >> v2:172.21.15.124:3300/0 conn(0x7fc7b0156500 0x7fc7b00c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.298+0000 7fc7b5ac9700 1 -- 172.21.15.92:0/4158891844 >> v2:172.21.15.124:3300/0 conn(0x7fc7b0156500 msgr2=0x7fc7b00c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.298+0000 7fc7b5ac9700 1 --2- 172.21.15.92:0/4158891844 >> v2:172.21.15.124:3300/0 conn(0x7fc7b0156500 0x7fc7b00c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.298+0000 7fc7b5ac9700 1 -- 172.21.15.92:0/4158891844 >> v1:172.21.15.124:6789/0 conn(0x7fc7b00c7570 legacy=0x7fc7b006e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.298+0000 7fc7b5ac9700 1 -- 172.21.15.92:0/4158891844 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b00c7b30 con 0x7fc7b006ed60 2023-10-06T10:42:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.298+0000 7fc7b5ac9700 1 --2- 172.21.15.92:0/4158891844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7b006ed60 0x7fc7b00ad1b0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fc7b006fd40 tx=0x7fc7a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b0b39a6eff5882 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.298+0000 7fc7a7fff700 1 -- 172.21.15.92:0/4158891844 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a800e070 con 0x7fc7b006ed60 2023-10-06T10:42:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.298+0000 7fc7a7fff700 1 -- 172.21.15.92:0/4158891844 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a800ba20 con 0x7fc7b006ed60 2023-10-06T10:42:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.298+0000 7fc7a7fff700 1 -- 172.21.15.92:0/4158891844 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a80092e0 con 0x7fc7b006ed60 2023-10-06T10:42:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.298+0000 7fc7b752c700 1 -- 172.21.15.92:0/4158891844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7b006ed60 msgr2=0x7fc7b00ad1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.299+0000 7fc7b752c700 1 --2- 172.21.15.92:0/4158891844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7b006ed60 0x7fc7b00ad1b0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fc7b006fd40 tx=0x7fc7a8009840 comp rx=0 tx=0).stop 2023-10-06T10:42:59.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.299+0000 7fc7b752c700 1 -- 172.21.15.92:0/4158891844 shutdown_connections 2023-10-06T10:42:59.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.299+0000 7fc7b752c700 1 --2- 172.21.15.92:0/4158891844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7b006ed60 0x7fc7b00ad1b0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.299+0000 7fc7b752c700 1 --2- 172.21.15.92:0/4158891844 >> v2:172.21.15.124:3300/0 conn(0x7fc7b0156500 0x7fc7b00c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.299+0000 7fc7b752c700 1 -- 172.21.15.92:0/4158891844 >> 172.21.15.92:0/4158891844 conn(0x7fc7b0068080 msgr2=0x7fc7b00cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:59.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.299+0000 7fc7b752c700 1 -- 172.21.15.92:0/4158891844 shutdown_connections 2023-10-06T10:42:59.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.299+0000 7fc7b752c700 1 -- 172.21.15.92:0/4158891844 wait complete. 2023-10-06T10:42:59.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.300+0000 7fc7b752c700 1 Processor -- start 2023-10-06T10:42:59.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.300+0000 7fc7b752c700 1 -- start start 2023-10-06T10:42:59.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7b752c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7b006ed60 0x7fc7b00b1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:59.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7b752c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7b00c7570 0x7fc7b00c1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:59.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7b752c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7b0156500 0x7fc7b00c2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:59.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7b752c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b01574a0 con 0x7fc7b0156500 2023-10-06T10:42:59.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7b752c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b00afb80 con 0x7fc7b006ed60 2023-10-06T10:42:59.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7b752c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b0156b70 con 0x7fc7b00c7570 2023-10-06T10:42:59.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7b5ac9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7b0156500 0x7fc7b00c2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:59.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7b5ac9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7b0156500 0x7fc7b00c2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37610/0 (socket says 172.21.15.92:37610) 2023-10-06T10:42:59.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7b5ac9700 1 -- 172.21.15.92:0/3634942503 learned_addr learned my addr 172.21.15.92:0/3634942503 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:59.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7b4ac7700 1 --2- 172.21.15.92:0/3634942503 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7b00c7570 0x7fc7b00c1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:59.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7b52c8700 1 --2- 172.21.15.92:0/3634942503 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7b006ed60 0x7fc7b00b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:59.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7b5ac9700 1 -- 172.21.15.92:0/3634942503 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7b00c7570 msgr2=0x7fc7b00c1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:59.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7b5ac9700 1 --2- 172.21.15.92:0/3634942503 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7b00c7570 0x7fc7b00c1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7b5ac9700 1 -- 172.21.15.92:0/3634942503 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7b006ed60 msgr2=0x7fc7b00b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:59.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7b5ac9700 1 --2- 172.21.15.92:0/3634942503 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7b006ed60 0x7fc7b00b1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7b5ac9700 1 -- 172.21.15.92:0/3634942503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a8009ae0 con 0x7fc7b0156500 2023-10-06T10:42:59.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7b4ac7700 1 --2- 172.21.15.92:0/3634942503 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7b00c7570 0x7fc7b00c1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:59.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7b52c8700 1 --2- 172.21.15.92:0/3634942503 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7b006ed60 0x7fc7b00b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:59.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7b5ac9700 1 --2- 172.21.15.92:0/3634942503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7b0156500 0x7fc7b00c2140 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009880 tx=0x7fc7a8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:59.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7a57fa700 1 -- 172.21.15.92:0/3634942503 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8008be0 con 0x7fc7b0156500 2023-10-06T10:42:59.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7a57fa700 1 -- 172.21.15.92:0/3634942503 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a8008d60 con 0x7fc7b0156500 2023-10-06T10:42:59.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7b752c700 1 -- 172.21.15.92:0/3634942503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b0156e50 con 0x7fc7b0156500 2023-10-06T10:42:59.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7a57fa700 1 -- 172.21.15.92:0/3634942503 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a80266b0 con 0x7fc7b0156500 2023-10-06T10:42:59.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.301+0000 7fc7b752c700 1 -- 172.21.15.92:0/3634942503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b0159940 con 0x7fc7b0156500 2023-10-06T10:42:59.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.302+0000 7fc7b752c700 1 -- 172.21.15.92:0/3634942503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b0157610 con 0x7fc7b0156500 2023-10-06T10:42:59.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.302+0000 7fc7a57fa700 1 -- 172.21.15.92:0/3634942503 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc7a8003040 con 0x7fc7b0156500 2023-10-06T10:42:59.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.303+0000 7fc7a57fa700 1 --2- 172.21.15.92:0/3634942503 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc794066f30 0x7fc7940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:59.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.303+0000 7fc7a57fa700 1 -- 172.21.15.92:0/3634942503 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fc7a80864f0 con 0x7fc7b0156500 2023-10-06T10:42:59.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.303+0000 7fc7b52c8700 1 --2- 172.21.15.92:0/3634942503 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc794066f30 0x7fc7940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:59.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.303+0000 7fc7b52c8700 1 --2- 172.21.15.92:0/3634942503 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc794066f30 0x7fc7940693e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac009ce0 tx=0x7fc7ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:59.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.307+0000 7fc7a57fa700 1 -- 172.21.15.92:0/3634942503 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7a8053810 con 0x7fc7b0156500 2023-10-06T10:42:59.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.493+0000 7fc7b752c700 1 -- 172.21.15.92:0/3634942503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7b0064a90 con 0x7fc7b0156500 2023-10-06T10:42:59.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.494+0000 7fc7a57fa700 1 -- 172.21.15.92:0/3634942503 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+13856 (secure 0 0 0) 0x7fc7a8056e30 con 0x7fc7b0156500 2023-10-06T10:42:59.495 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:42:59.496 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":74,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:42:58.767134+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:42:59.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.496+0000 7fc7b752c700 1 -- 172.21.15.92:0/3634942503 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc794066f30 msgr2=0x7fc7940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:59.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.496+0000 7fc7b752c700 1 --2- 172.21.15.92:0/3634942503 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc794066f30 0x7fc7940693e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac009ce0 tx=0x7fc7ac008040 comp rx=0 tx=0).stop 2023-10-06T10:42:59.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.496+0000 7fc7b752c700 1 -- 172.21.15.92:0/3634942503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7b0156500 msgr2=0x7fc7b00c2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:59.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.496+0000 7fc7b752c700 1 --2- 172.21.15.92:0/3634942503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7b0156500 0x7fc7b00c2140 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009880 tx=0x7fc7a8018000 comp rx=0 tx=0).stop 2023-10-06T10:42:59.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.496+0000 7fc7b752c700 1 -- 172.21.15.92:0/3634942503 shutdown_connections 2023-10-06T10:42:59.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.496+0000 7fc7b752c700 1 --2- 172.21.15.92:0/3634942503 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc794066f30 0x7fc7940693e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.496+0000 7fc7b752c700 1 --2- 172.21.15.92:0/3634942503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7b0156500 0x7fc7b00c2140 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.496+0000 7fc7b752c700 1 --2- 172.21.15.92:0/3634942503 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7b006ed60 0x7fc7b00b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.496+0000 7fc7b752c700 1 --2- 172.21.15.92:0/3634942503 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7b00c7570 0x7fc7b00c1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.496+0000 7fc7b752c700 1 -- 172.21.15.92:0/3634942503 >> 172.21.15.92:0/3634942503 conn(0x7fc7b0068080 msgr2=0x7fc7b00ca4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:59.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.496+0000 7fc7b752c700 1 -- 172.21.15.92:0/3634942503 shutdown_connections 2023-10-06T10:42:59.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.496+0000 7fc7b752c700 1 -- 172.21.15.92:0/3634942503 wait complete. 2023-10-06T10:42:59.510 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T10:38:09.680559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T10:38:10.120786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T10:42:59.510 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T10:42:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.625+0000 7f9868466700 1 Processor -- start 2023-10-06T10:42:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.625+0000 7f9868466700 1 -- start start 2023-10-06T10:42:59.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.626+0000 7f9868466700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98601594d0 0x7f98600cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:59.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.626+0000 7f9868466700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98600cbec0 con 0x7f98600cbaa0 2023-10-06T10:42:59.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.626+0000 7f9868466700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98600cf5d0 con 0x7f98601598a0 2023-10-06T10:42:59.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.626+0000 7f9868466700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98600cf8b0 con 0x7f98601594d0 2023-10-06T10:42:59.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.627+0000 7f9866a03700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f98600cbaa0 0x7f98600c5690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53144/0 (socket says 172.21.15.92:53144) 2023-10-06T10:42:59.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.627+0000 7f9866a03700 1 -- 172.21.15.92:0/513553524 learned_addr learned my addr 172.21.15.92:0/513553524 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:42:59.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.627+0000 7f9866202700 1 --2- 172.21.15.92:0/513553524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98601594d0 0x7f98600cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:59.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.627+0000 7f9865200700 1 -- 172.21.15.92:0/513553524 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347638918 0 0) 0x7f98600cbec0 con 0x7f98600cbaa0 2023-10-06T10:42:59.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.627+0000 7f9865200700 1 -- 172.21.15.92:0/513553524 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f984c003570 con 0x7f98600cbaa0 2023-10-06T10:42:59.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.627+0000 7f9865200700 1 -- 172.21.15.92:0/513553524 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3028227607 0 0) 0x7f984c003570 con 0x7f98600cbaa0 2023-10-06T10:42:59.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.627+0000 7f9865200700 1 -- 172.21.15.92:0/513553524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98601594d0 msgr2=0x7f98600cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:59.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.627+0000 7f9865200700 1 --2- 172.21.15.92:0/513553524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98601594d0 0x7f98600cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.627+0000 7f9865200700 1 -- 172.21.15.92:0/513553524 >> v1:172.21.15.124:6789/0 conn(0x7f98601598a0 legacy=0x7f98600ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:59.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.627+0000 7f9865200700 1 -- 172.21.15.92:0/513553524 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98600cfb90 con 0x7f98600cbaa0 2023-10-06T10:42:59.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.627+0000 7f9865200700 1 -- 172.21.15.92:0/513553524 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f98580017f0 con 0x7f98600cbaa0 2023-10-06T10:42:59.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.627+0000 7f9865200700 1 -- 172.21.15.92:0/513553524 >> v1:172.21.15.92:6789/0 conn(0x7f98600cbaa0 legacy=0x7f98600c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:59.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.627+0000 7f9865200700 1 --2- 172.21.15.92:0/513553524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98601594d0 0x7f984c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:59.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.627+0000 7f9865200700 1 --2- 172.21.15.92:0/513553524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f984c003fd0 0x7f984c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:59.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.627+0000 7f9865200700 1 --2- 172.21.15.92:0/513553524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f984c004d50 0x7f984c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:59.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.627+0000 7f9865200700 1 -- 172.21.15.92:0/513553524 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98600cf8b0 con 0x7f984c003fd0 2023-10-06T10:42:59.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.627+0000 7f9865200700 1 -- 172.21.15.92:0/513553524 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c00bb40 con 0x7f984c004d50 2023-10-06T10:42:59.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.627+0000 7f9865200700 1 -- 172.21.15.92:0/513553524 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984c00bd00 con 0x7f98601594d0 2023-10-06T10:42:59.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.627+0000 7f9865a01700 1 --2- 172.21.15.92:0/513553524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f984c003fd0 0x7f984c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:59.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.628+0000 7f9866202700 1 --2- 172.21.15.92:0/513553524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98601594d0 0x7f984c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:59.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.628+0000 7f9866a03700 1 --2- 172.21.15.92:0/513553524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f984c004d50 0x7f984c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:59.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.628+0000 7f9865a01700 1 -- 172.21.15.92:0/513553524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98601594d0 msgr2=0x7f984c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:59.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.628+0000 7f9865a01700 1 --2- 172.21.15.92:0/513553524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98601594d0 0x7f984c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.628+0000 7f9865a01700 1 -- 172.21.15.92:0/513553524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f984c004d50 msgr2=0x7f984c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:59.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.628+0000 7f9865a01700 1 --2- 172.21.15.92:0/513553524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f984c004d50 0x7f984c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.628+0000 7f9865a01700 1 -- 172.21.15.92:0/513553524 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984c003570 con 0x7f984c003fd0 2023-10-06T10:42:59.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.628+0000 7f9866a03700 1 --2- 172.21.15.92:0/513553524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f984c004d50 0x7f984c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:42:59.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.628+0000 7f9865a01700 1 --2- 172.21.15.92:0/513553524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f984c003fd0 0x7f984c004810 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f9860159c90 tx=0x7f9850007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11d9588b616910 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:59.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.628+0000 7f9865200700 1 -- 172.21.15.92:0/513553524 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985000c590 con 0x7f984c003fd0 2023-10-06T10:42:59.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.628+0000 7f9865200700 1 -- 172.21.15.92:0/513553524 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985000c710 con 0x7f984c003fd0 2023-10-06T10:42:59.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.628+0000 7f9865200700 1 -- 172.21.15.92:0/513553524 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985000cad0 con 0x7f984c003fd0 2023-10-06T10:42:59.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.629+0000 7f9868466700 1 -- 172.21.15.92:0/513553524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f984c003fd0 msgr2=0x7f984c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:59.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.629+0000 7f9868466700 1 --2- 172.21.15.92:0/513553524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f984c003fd0 0x7f984c004810 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f9860159c90 tx=0x7f9850007d30 comp rx=0 tx=0).stop 2023-10-06T10:42:59.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.629+0000 7f9866a03700 1 -- 172.21.15.92:0/513553524 reap_dead start 2023-10-06T10:42:59.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.629+0000 7f9868466700 1 -- 172.21.15.92:0/513553524 shutdown_connections 2023-10-06T10:42:59.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.629+0000 7f9868466700 1 -- 172.21.15.92:0/513553524 >> 172.21.15.92:0/513553524 conn(0x7f9860089e50 msgr2=0x7f9860099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:59.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.629+0000 7f9868466700 1 -- 172.21.15.92:0/513553524 shutdown_connections 2023-10-06T10:42:59.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.629+0000 7f9868466700 1 -- 172.21.15.92:0/513553524 wait complete. 2023-10-06T10:42:59.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.630+0000 7f9868466700 1 Processor -- start 2023-10-06T10:42:59.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.630+0000 7f9868466700 1 -- start start 2023-10-06T10:42:59.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.631+0000 7f9868466700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f984c003fd0 0x7f98600b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:59.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.631+0000 7f9868466700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f986009a330 0x7f98600b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:59.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.631+0000 7f9868466700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98600b5f30 0x7f9860151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:59.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.631+0000 7f9868466700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98600b5260 con 0x7f98600b5f30 2023-10-06T10:42:59.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.631+0000 7f9868466700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98600b3a30 con 0x7f984c003fd0 2023-10-06T10:42:59.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.631+0000 7f9868466700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98600b4890 con 0x7f986009a330 2023-10-06T10:42:59.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.631+0000 7f9866a03700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98600b5f30 0x7f9860151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:59.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.631+0000 7f9866a03700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98600b5f30 0x7f9860151760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37632/0 (socket says 172.21.15.92:37632) 2023-10-06T10:42:59.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.631+0000 7f9866a03700 1 -- 172.21.15.92:0/308261707 learned_addr learned my addr 172.21.15.92:0/308261707 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:59.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.631+0000 7f9866202700 1 --2- 172.21.15.92:0/308261707 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f984c003fd0 0x7f98600b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:59.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.631+0000 7f9865a01700 1 --2- 172.21.15.92:0/308261707 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f986009a330 0x7f98600b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:59.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.631+0000 7f9866a03700 1 -- 172.21.15.92:0/308261707 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f986009a330 msgr2=0x7f98600b59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:59.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.631+0000 7f9866a03700 1 --2- 172.21.15.92:0/308261707 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f986009a330 0x7f98600b59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.631+0000 7f9866a03700 1 -- 172.21.15.92:0/308261707 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f984c003fd0 msgr2=0x7f98600b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:59.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.631+0000 7f9866a03700 1 --2- 172.21.15.92:0/308261707 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f984c003fd0 0x7f98600b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.631+0000 7f9866a03700 1 -- 172.21.15.92:0/308261707 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98600b3c80 con 0x7f98600b5f30 2023-10-06T10:42:59.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.631+0000 7f9866a03700 1 --2- 172.21.15.92:0/308261707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98600b5f30 0x7f9860151760 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f9858002000 tx=0x7f9858001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:59.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.631+0000 7f98567fc700 1 -- 172.21.15.92:0/308261707 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985800eb10 con 0x7f98600b5f30 2023-10-06T10:42:59.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.631+0000 7f98567fc700 1 -- 172.21.15.92:0/308261707 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985800ec70 con 0x7f98600b5f30 2023-10-06T10:42:59.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.631+0000 7f98567fc700 1 -- 172.21.15.92:0/308261707 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9858013420 con 0x7f98600b5f30 2023-10-06T10:42:59.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.631+0000 7f9868466700 1 -- 172.21.15.92:0/308261707 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98600b3f60 con 0x7f98600b5f30 2023-10-06T10:42:59.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.632+0000 7f9868466700 1 -- 172.21.15.92:0/308261707 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98600b4520 con 0x7f98600b5f30 2023-10-06T10:42:59.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.632+0000 7f9868466700 1 -- 172.21.15.92:0/308261707 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98600b4a40 con 0x7f98600b5f30 2023-10-06T10:42:59.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.633+0000 7f98567fc700 1 -- 172.21.15.92:0/308261707 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9858013810 con 0x7f98600b5f30 2023-10-06T10:42:59.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.633+0000 7f98567fc700 1 --2- 172.21.15.92:0/308261707 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9844066f60 0x7f9844069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:59.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.633+0000 7f98567fc700 1 -- 172.21.15.92:0/308261707 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f9858084860 con 0x7f98600b5f30 2023-10-06T10:42:59.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.633+0000 7f9866202700 1 --2- 172.21.15.92:0/308261707 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9844066f60 0x7f9844069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:59.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.633+0000 7f9866202700 1 --2- 172.21.15.92:0/308261707 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9844066f60 0x7f9844069410 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f985c006f00 tx=0x7f985c0091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:59.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.637+0000 7f98567fc700 1 -- 172.21.15.92:0/308261707 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98600b4a40 con 0x7f98600b5f30 2023-10-06T10:42:59.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.820+0000 7f9868466700 1 -- 172.21.15.92:0/308261707 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f98600b4a00 con 0x7f98600b5f30 2023-10-06T10:42:59.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.820+0000 7f98567fc700 1 -- 172.21.15.92:0/308261707 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v74) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9858055610 con 0x7f98600b5f30 2023-10-06T10:42:59.822 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T10:42:59.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.822+0000 7f9868466700 1 -- 172.21.15.92:0/308261707 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9844066f60 msgr2=0x7f9844069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:59.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.822+0000 7f9868466700 1 --2- 172.21.15.92:0/308261707 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9844066f60 0x7f9844069410 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f985c006f00 tx=0x7f985c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.822+0000 7f9868466700 1 -- 172.21.15.92:0/308261707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98600b5f30 msgr2=0x7f9860151760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:59.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.822+0000 7f9868466700 1 --2- 172.21.15.92:0/308261707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98600b5f30 0x7f9860151760 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f9858002000 tx=0x7f9858001250 comp rx=0 tx=0).stop 2023-10-06T10:42:59.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.822+0000 7f9868466700 1 -- 172.21.15.92:0/308261707 shutdown_connections 2023-10-06T10:42:59.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.822+0000 7f9868466700 1 --2- 172.21.15.92:0/308261707 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9844066f60 0x7f9844069410 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.822+0000 7f9868466700 1 --2- 172.21.15.92:0/308261707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98600b5f30 0x7f9860151760 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.822+0000 7f9868466700 1 --2- 172.21.15.92:0/308261707 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f984c003fd0 0x7f98600b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.822+0000 7f9868466700 1 --2- 172.21.15.92:0/308261707 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f986009a330 0x7f98600b59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.822+0000 7f9868466700 1 -- 172.21.15.92:0/308261707 >> 172.21.15.92:0/308261707 conn(0x7f9860089e50 msgr2=0x7f9860154150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:59.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.822+0000 7f9868466700 1 -- 172.21.15.92:0/308261707 shutdown_connections 2023-10-06T10:42:59.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.823+0000 7f9868466700 1 -- 172.21.15.92:0/308261707 wait complete. 2023-10-06T10:42:59.836 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T10:42:59.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.952+0000 7fdb1e644700 1 Processor -- start 2023-10-06T10:42:59.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.952+0000 7fdb1e644700 1 -- start start 2023-10-06T10:42:59.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.953+0000 7fdb1e644700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdb181594d0 0x7fdb180cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:59.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.953+0000 7fdb1e644700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb180cbee0 con 0x7fdb181598a0 2023-10-06T10:42:59.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.953+0000 7fdb1e644700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb180cf5d0 con 0x7fdb180cbaa0 2023-10-06T10:42:59.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.953+0000 7fdb1e644700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb180cf8b0 con 0x7fdb181594d0 2023-10-06T10:42:59.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.953+0000 7fdb17fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdb181594d0 0x7fdb180cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:59.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.953+0000 7fdb177fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdb181598a0 0x7fdb180ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53156/0 (socket says 172.21.15.92:53156) 2023-10-06T10:42:59.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.953+0000 7fdb177fe700 1 -- 172.21.15.92:0/659727635 learned_addr learned my addr 172.21.15.92:0/659727635 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:42:59.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.953+0000 7fdb17fff700 1 -- 172.21.15.92:0/659727635 >> v1:172.21.15.124:6789/0 conn(0x7fdb180cbaa0 legacy=0x7fdb180c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:42:59.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.953+0000 7fdb17fff700 1 -- 172.21.15.92:0/659727635 >> v1:172.21.15.92:6789/0 conn(0x7fdb181598a0 legacy=0x7fdb180ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:59.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.954+0000 7fdb16ffd700 1 -- 172.21.15.92:0/659727635 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1869809443 0 0) 0x7fdb180cbee0 con 0x7fdb181598a0 2023-10-06T10:42:59.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.954+0000 7fdb17fff700 1 -- 172.21.15.92:0/659727635 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb180cfb90 con 0x7fdb181594d0 2023-10-06T10:42:59.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.954+0000 7fdb17fff700 1 --2- 172.21.15.92:0/659727635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb181594d0 0x7fdb180cb560 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fdb080098e0 tx=0x7fdb080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79abaf2386418fcf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:59.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.954+0000 7fdb16ffd700 1 -- 172.21.15.92:0/659727635 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08009e80 con 0x7fdb181594d0 2023-10-06T10:42:59.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.954+0000 7fdb16ffd700 1 -- 172.21.15.92:0/659727635 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0800b820 con 0x7fdb181594d0 2023-10-06T10:42:59.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.954+0000 7fdb16ffd700 1 -- 172.21.15.92:0/659727635 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0800bbe0 con 0x7fdb181594d0 2023-10-06T10:42:59.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.954+0000 7fdb1e644700 1 -- 172.21.15.92:0/659727635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb181594d0 msgr2=0x7fdb180cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:59.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.954+0000 7fdb1e644700 1 --2- 172.21.15.92:0/659727635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb181594d0 0x7fdb180cb560 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fdb080098e0 tx=0x7fdb080091a0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.954+0000 7fdb1e644700 1 -- 172.21.15.92:0/659727635 shutdown_connections 2023-10-06T10:42:59.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.954+0000 7fdb1e644700 1 --2- 172.21.15.92:0/659727635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb181594d0 0x7fdb180cb560 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.954+0000 7fdb1e644700 1 -- 172.21.15.92:0/659727635 >> 172.21.15.92:0/659727635 conn(0x7fdb18089e50 msgr2=0x7fdb18099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:42:59.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.954+0000 7fdb1e644700 1 -- 172.21.15.92:0/659727635 shutdown_connections 2023-10-06T10:42:59.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.954+0000 7fdb1e644700 1 -- 172.21.15.92:0/659727635 wait complete. 2023-10-06T10:42:59.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.956+0000 7fdb1e644700 1 Processor -- start 2023-10-06T10:42:59.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.956+0000 7fdb1e644700 1 -- start start 2023-10-06T10:42:59.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.956+0000 7fdb1e644700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb180cbaa0 0x7fdb1814ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:59.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.956+0000 7fdb1e644700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb181594d0 0x7fdb1809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:59.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.956+0000 7fdb1e644700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb181598a0 0x7fdb180c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:59.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.956+0000 7fdb1e644700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb180b32b0 con 0x7fdb180cbaa0 2023-10-06T10:42:59.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.956+0000 7fdb1e644700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1809a5e0 con 0x7fdb181594d0 2023-10-06T10:42:59.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.956+0000 7fdb1e644700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1809a8c0 con 0x7fdb181598a0 2023-10-06T10:42:59.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.957+0000 7fdb17fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb180cbaa0 0x7fdb1814ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:59.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.957+0000 7fdb17fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb180cbaa0 0x7fdb1814ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37644/0 (socket says 172.21.15.92:37644) 2023-10-06T10:42:59.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.957+0000 7fdb17fff700 1 -- 172.21.15.92:0/3016558224 learned_addr learned my addr 172.21.15.92:0/3016558224 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:42:59.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.957+0000 7fdb177fe700 1 --2- 172.21.15.92:0/3016558224 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb181594d0 0x7fdb1809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:59.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.957+0000 7fdb17fff700 1 -- 172.21.15.92:0/3016558224 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb181598a0 msgr2=0x7fdb180c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:59.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.957+0000 7fdb17fff700 1 --2- 172.21.15.92:0/3016558224 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb181598a0 0x7fdb180c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.957+0000 7fdb17fff700 1 -- 172.21.15.92:0/3016558224 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb181594d0 msgr2=0x7fdb1809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:42:59.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.957+0000 7fdb17fff700 1 --2- 172.21.15.92:0/3016558224 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb181594d0 0x7fdb1809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:42:59.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.957+0000 7fdb17fff700 1 -- 172.21.15.92:0/3016558224 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb08009420 con 0x7fdb180cbaa0 2023-10-06T10:42:59.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.957+0000 7fdb177fe700 1 --2- 172.21.15.92:0/3016558224 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb181594d0 0x7fdb1809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:42:59.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.957+0000 7fdb17fff700 1 --2- 172.21.15.92:0/3016558224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb180cbaa0 0x7fdb1814ef80 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fdb08006eb0 tx=0x7fdb08016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:59.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.957+0000 7fdafffff700 1 -- 172.21.15.92:0/3016558224 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0800bae0 con 0x7fdb180cbaa0 2023-10-06T10:42:59.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.957+0000 7fdafffff700 1 -- 172.21.15.92:0/3016558224 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb08020070 con 0x7fdb180cbaa0 2023-10-06T10:42:59.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.957+0000 7fdb1e644700 1 -- 172.21.15.92:0/3016558224 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb1809aba0 con 0x7fdb180cbaa0 2023-10-06T10:42:59.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.957+0000 7fdafffff700 1 -- 172.21.15.92:0/3016558224 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0801b6a0 con 0x7fdb180cbaa0 2023-10-06T10:42:59.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.957+0000 7fdb1e644700 1 -- 172.21.15.92:0/3016558224 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb1814f4c0 con 0x7fdb180cbaa0 2023-10-06T10:42:59.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.958+0000 7fdb1e644700 1 -- 172.21.15.92:0/3016558224 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb18064a90 con 0x7fdb180cbaa0 2023-10-06T10:42:59.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.958+0000 7fdafffff700 1 -- 172.21.15.92:0/3016558224 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb08022030 con 0x7fdb180cbaa0 2023-10-06T10:42:59.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.959+0000 7fdafffff700 1 --2- 172.21.15.92:0/3016558224 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdb04066f30 0x7fdb040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:42:59.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.959+0000 7fdafffff700 1 -- 172.21.15.92:0/3016558224 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fdb080864d0 con 0x7fdb180cbaa0 2023-10-06T10:42:59.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.959+0000 7fdb177fe700 1 --2- 172.21.15.92:0/3016558224 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdb04066f30 0x7fdb040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:42:59.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.959+0000 7fdb177fe700 1 --2- 172.21.15.92:0/3016558224 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdb04066f30 0x7fdb040693e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdb1814fc10 tx=0x7fdb0c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:42:59.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:42:59.963+0000 7fdafffff700 1 -- 172.21.15.92:0/3016558224 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb080537f0 con 0x7fdb180cbaa0 2023-10-06T10:43:00.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.144+0000 7fdb1e644700 1 -- 172.21.15.92:0/3016558224 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdb18064a90 con 0x7fdb180cbaa0 2023-10-06T10:43:00.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.145+0000 7fdafffff700 1 -- 172.21.15.92:0/3016558224 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdb08056e10 con 0x7fdb180cbaa0 2023-10-06T10:43:00.147 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T10:43:00.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.147+0000 7fdb1e644700 1 -- 172.21.15.92:0/3016558224 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdb04066f30 msgr2=0x7fdb040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:00.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.147+0000 7fdb1e644700 1 --2- 172.21.15.92:0/3016558224 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdb04066f30 0x7fdb040693e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdb1814fc10 tx=0x7fdb0c009040 comp rx=0 tx=0).stop 2023-10-06T10:43:00.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.147+0000 7fdb1e644700 1 -- 172.21.15.92:0/3016558224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb180cbaa0 msgr2=0x7fdb1814ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:00.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.147+0000 7fdb1e644700 1 --2- 172.21.15.92:0/3016558224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb180cbaa0 0x7fdb1814ef80 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fdb08006eb0 tx=0x7fdb08016000 comp rx=0 tx=0).stop 2023-10-06T10:43:00.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.147+0000 7fdb1e644700 1 -- 172.21.15.92:0/3016558224 shutdown_connections 2023-10-06T10:43:00.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.147+0000 7fdb1e644700 1 --2- 172.21.15.92:0/3016558224 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdb04066f30 0x7fdb040693e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:00.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.147+0000 7fdb1e644700 1 --2- 172.21.15.92:0/3016558224 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb181598a0 0x7fdb180c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:00.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.147+0000 7fdb1e644700 1 --2- 172.21.15.92:0/3016558224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb180cbaa0 0x7fdb1814ef80 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:00.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.147+0000 7fdb1e644700 1 --2- 172.21.15.92:0/3016558224 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb181594d0 0x7fdb1809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:00.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.147+0000 7fdb1e644700 1 -- 172.21.15.92:0/3016558224 >> 172.21.15.92:0/3016558224 conn(0x7fdb18089e50 msgr2=0x7fdb180d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:00.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.147+0000 7fdb1e644700 1 -- 172.21.15.92:0/3016558224 shutdown_connections 2023-10-06T10:43:00.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.147+0000 7fdb1e644700 1 -- 172.21.15.92:0/3016558224 wait complete. 2023-10-06T10:43:00.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T10:43:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.274+0000 7f279af33700 1 Processor -- start 2023-10-06T10:43:00.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.274+0000 7f279af33700 1 -- start start 2023-10-06T10:43:00.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f279af33700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2794159df0 0x7f279415a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:00.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f279af33700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27940cd850 0x7f27940cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:00.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f279af33700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f279415e020 con 0x7f279415a700 2023-10-06T10:43:00.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f279af33700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f279415d4a0 con 0x7f2794159df0 2023-10-06T10:43:00.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f279af33700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279415d780 con 0x7f27940cd850 2023-10-06T10:43:00.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f278bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f279415a700 0x7f27940cd140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53164/0 (socket says 172.21.15.92:53164) 2023-10-06T10:43:00.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f278bfff700 1 -- 172.21.15.92:0/3289626927 learned_addr learned my addr 172.21.15.92:0/3289626927 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:00.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f2798ccf700 1 --2- 172.21.15.92:0/3289626927 >> v2:172.21.15.124:3300/0 conn(0x7f2794159df0 0x7f279415a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:00.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f27994d0700 1 --2- 172.21.15.92:0/3289626927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27940cd850 0x7f27940cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:00.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f278b7fe700 1 -- 172.21.15.92:0/3289626927 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1674180308 0 0) 0x7f279415e020 con 0x7f279415a700 2023-10-06T10:43:00.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f278b7fe700 1 -- 172.21.15.92:0/3289626927 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f277c003570 con 0x7f279415a700 2023-10-06T10:43:00.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f278b7fe700 1 -- 172.21.15.92:0/3289626927 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1621356003 0 0) 0x7f277c003570 con 0x7f279415a700 2023-10-06T10:43:00.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f278b7fe700 1 -- 172.21.15.92:0/3289626927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27940cd850 msgr2=0x7f27940cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:00.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f278b7fe700 1 --2- 172.21.15.92:0/3289626927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27940cd850 0x7f27940cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:00.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f278b7fe700 1 -- 172.21.15.92:0/3289626927 >> v2:172.21.15.124:3300/0 conn(0x7f2794159df0 msgr2=0x7f279415a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:00.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f278b7fe700 1 --2- 172.21.15.92:0/3289626927 >> v2:172.21.15.124:3300/0 conn(0x7f2794159df0 0x7f279415a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:00.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f278b7fe700 1 -- 172.21.15.92:0/3289626927 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279415da60 con 0x7f279415a700 2023-10-06T10:43:00.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f278b7fe700 1 -- 172.21.15.92:0/3289626927 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f2784001470 con 0x7f279415a700 2023-10-06T10:43:00.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f278b7fe700 1 -- 172.21.15.92:0/3289626927 >> v1:172.21.15.92:6789/0 conn(0x7f279415a700 legacy=0x7f27940cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:00.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f278b7fe700 1 --2- 172.21.15.92:0/3289626927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f277c003a60 0x7f277c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:00.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f278b7fe700 1 --2- 172.21.15.92:0/3289626927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27940cd850 0x7f277c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:00.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f278b7fe700 1 --2- 172.21.15.92:0/3289626927 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f277c004d50 0x7f277c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:00.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f278b7fe700 1 -- 172.21.15.92:0/3289626927 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279415d4a0 con 0x7f277c003a60 2023-10-06T10:43:00.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f278b7fe700 1 -- 172.21.15.92:0/3289626927 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279415d780 con 0x7f277c004d50 2023-10-06T10:43:00.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f278b7fe700 1 -- 172.21.15.92:0/3289626927 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277c00bae0 con 0x7f27940cd850 2023-10-06T10:43:00.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.275+0000 7f2798ccf700 1 --2- 172.21.15.92:0/3289626927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f277c003a60 0x7f277c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:00.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.276+0000 7f27994d0700 1 --2- 172.21.15.92:0/3289626927 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f277c004d50 0x7f277c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:00.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.276+0000 7f278bfff700 1 --2- 172.21.15.92:0/3289626927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27940cd850 0x7f277c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:00.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.276+0000 7f2798ccf700 1 -- 172.21.15.92:0/3289626927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27940cd850 msgr2=0x7f277c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:00.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.276+0000 7f2798ccf700 1 --2- 172.21.15.92:0/3289626927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27940cd850 0x7f277c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:00.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.276+0000 7f2798ccf700 1 -- 172.21.15.92:0/3289626927 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f277c004d50 msgr2=0x7f277c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:00.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.276+0000 7f2798ccf700 1 --2- 172.21.15.92:0/3289626927 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f277c004d50 0x7f277c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:00.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.276+0000 7f2798ccf700 1 -- 172.21.15.92:0/3289626927 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277c003570 con 0x7f277c003a60 2023-10-06T10:43:00.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.276+0000 7f27994d0700 1 --2- 172.21.15.92:0/3289626927 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f277c004d50 0x7f277c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:00.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.276+0000 7f2798ccf700 1 --2- 172.21.15.92:0/3289626927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f277c003a60 0x7f277c003e70 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f2790009c00 tx=0x7f27900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe87892bee6301f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:00.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.276+0000 7f278b7fe700 1 -- 172.21.15.92:0/3289626927 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279000e710 con 0x7f277c003a60 2023-10-06T10:43:00.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.276+0000 7f278b7fe700 1 -- 172.21.15.92:0/3289626927 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279000e870 con 0x7f277c003a60 2023-10-06T10:43:00.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.276+0000 7f278b7fe700 1 -- 172.21.15.92:0/3289626927 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279000ec30 con 0x7f277c003a60 2023-10-06T10:43:00.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.276+0000 7f279af33700 1 -- 172.21.15.92:0/3289626927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f277c003a60 msgr2=0x7f277c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:00.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.276+0000 7f279af33700 1 --2- 172.21.15.92:0/3289626927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f277c003a60 0x7f277c003e70 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f2790009c00 tx=0x7f27900091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:00.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.276+0000 7f27994d0700 1 -- 172.21.15.92:0/3289626927 reap_dead start 2023-10-06T10:43:00.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.276+0000 7f279af33700 1 -- 172.21.15.92:0/3289626927 shutdown_connections 2023-10-06T10:43:00.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.276+0000 7f279af33700 1 -- 172.21.15.92:0/3289626927 >> 172.21.15.92:0/3289626927 conn(0x7f27940707d0 msgr2=0x7f2794070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:00.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.276+0000 7f279af33700 1 -- 172.21.15.92:0/3289626927 shutdown_connections 2023-10-06T10:43:00.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.277+0000 7f279af33700 1 -- 172.21.15.92:0/3289626927 wait complete. 2023-10-06T10:43:00.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.278+0000 7f279af33700 1 Processor -- start 2023-10-06T10:43:00.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.278+0000 7f279af33700 1 -- start start 2023-10-06T10:43:00.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.278+0000 7f279af33700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f277c003a60 0x7f27940c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:00.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.278+0000 7f279af33700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f279416d640 0x7f27940c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:00.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.278+0000 7f279af33700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27940cbed0 0x7f27940cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:00.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.278+0000 7f279af33700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27940ca390 con 0x7f27940cbed0 2023-10-06T10:43:00.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.278+0000 7f279af33700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27940cc880 con 0x7f279416d640 2023-10-06T10:43:00.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.278+0000 7f279af33700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27940c7e90 con 0x7f277c003a60 2023-10-06T10:43:00.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.279+0000 7f27994d0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27940cbed0 0x7f27940cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:00.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.279+0000 7f27994d0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27940cbed0 0x7f27940cc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37670/0 (socket says 172.21.15.92:37670) 2023-10-06T10:43:00.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.279+0000 7f27994d0700 1 -- 172.21.15.92:0/1692334795 learned_addr learned my addr 172.21.15.92:0/1692334795 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:00.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.279+0000 7f278bfff700 1 --2- 172.21.15.92:0/1692334795 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f279416d640 0x7f27940c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:00.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.279+0000 7f2798ccf700 1 --2- 172.21.15.92:0/1692334795 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f277c003a60 0x7f27940c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:00.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.279+0000 7f27994d0700 1 -- 172.21.15.92:0/1692334795 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f277c003a60 msgr2=0x7f27940c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:00.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.279+0000 7f27994d0700 1 --2- 172.21.15.92:0/1692334795 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f277c003a60 0x7f27940c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:00.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.279+0000 7f27994d0700 1 -- 172.21.15.92:0/1692334795 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f279416d640 msgr2=0x7f27940c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:00.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.279+0000 7f27994d0700 1 --2- 172.21.15.92:0/1692334795 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f279416d640 0x7f27940c9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:00.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.279+0000 7f27994d0700 1 -- 172.21.15.92:0/1692334795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27940c7560 con 0x7f27940cbed0 2023-10-06T10:43:00.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.279+0000 7f27994d0700 1 --2- 172.21.15.92:0/1692334795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27940cbed0 0x7f27940cc340 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f278c009f80 tx=0x7f278c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:00.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.279+0000 7f2788ff9700 1 -- 172.21.15.92:0/1692334795 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c01b070 con 0x7f27940cbed0 2023-10-06T10:43:00.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.279+0000 7f2788ff9700 1 -- 172.21.15.92:0/1692334795 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278c00f040 con 0x7f27940cbed0 2023-10-06T10:43:00.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.279+0000 7f2788ff9700 1 -- 172.21.15.92:0/1692334795 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c00e050 con 0x7f27940cbed0 2023-10-06T10:43:00.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.279+0000 7f279af33700 1 -- 172.21.15.92:0/1692334795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27940c7840 con 0x7f27940cbed0 2023-10-06T10:43:00.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.279+0000 7f279af33700 1 -- 172.21.15.92:0/1692334795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27941551f0 con 0x7f27940cbed0 2023-10-06T10:43:00.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.280+0000 7f279af33700 1 -- 172.21.15.92:0/1692334795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27940c8020 con 0x7f27940cbed0 2023-10-06T10:43:00.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.280+0000 7f2788ff9700 1 -- 172.21.15.92:0/1692334795 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f278c003020 con 0x7f27940cbed0 2023-10-06T10:43:00.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.281+0000 7f2788ff9700 1 --2- 172.21.15.92:0/1692334795 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2780066f60 0x7f2780069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:00.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.281+0000 7f2788ff9700 1 -- 172.21.15.92:0/1692334795 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f278c084860 con 0x7f27940cbed0 2023-10-06T10:43:00.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.281+0000 7f2798ccf700 1 --2- 172.21.15.92:0/1692334795 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2780066f60 0x7f2780069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:00.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.281+0000 7f2798ccf700 1 --2- 172.21.15.92:0/1692334795 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2780066f60 0x7f2780069410 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f27900091e0 tx=0x7f279001a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:00.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.285+0000 7f2788ff9700 1 -- 172.21.15.92:0/1692334795 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f278c051b80 con 0x7f27940cbed0 2023-10-06T10:43:00.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.471+0000 7f279af33700 1 -- 172.21.15.92:0/1692334795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2794064a90 con 0x7f27940cbed0 2023-10-06T10:43:00.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.472+0000 7f2788ff9700 1 -- 172.21.15.92:0/1692334795 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) v1 ==== 84+0+126 (secure 0 0 0) 0x7f278c0551a0 con 0x7f27940cbed0 2023-10-06T10:43:00.474 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:00.474 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T10:43:00.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:00.474 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-10-06T10:43:00.474 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-10-06T10:43:00.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:43:00.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:00.474 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-10-06T10:43:00.474 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-10-06T10:43:00.474 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:43:00.474 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T10:43:00.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.473+0000 7f279af33700 1 -- 172.21.15.92:0/1692334795 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2780066f60 msgr2=0x7f2780069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:00.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.473+0000 7f279af33700 1 --2- 172.21.15.92:0/1692334795 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2780066f60 0x7f2780069410 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f27900091e0 tx=0x7f279001a000 comp rx=0 tx=0).stop 2023-10-06T10:43:00.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.473+0000 7f279af33700 1 -- 172.21.15.92:0/1692334795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27940cbed0 msgr2=0x7f27940cc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:00.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.473+0000 7f279af33700 1 --2- 172.21.15.92:0/1692334795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27940cbed0 0x7f27940cc340 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f278c009f80 tx=0x7f278c00b040 comp rx=0 tx=0).stop 2023-10-06T10:43:00.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.473+0000 7f279af33700 1 -- 172.21.15.92:0/1692334795 shutdown_connections 2023-10-06T10:43:00.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.473+0000 7f279af33700 1 --2- 172.21.15.92:0/1692334795 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2780066f60 0x7f2780069410 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:00.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.473+0000 7f279af33700 1 --2- 172.21.15.92:0/1692334795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27940cbed0 0x7f27940cc340 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:00.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.473+0000 7f279af33700 1 --2- 172.21.15.92:0/1692334795 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f277c003a60 0x7f27940c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:00.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.473+0000 7f279af33700 1 --2- 172.21.15.92:0/1692334795 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f279416d640 0x7f27940c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:00.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.473+0000 7f279af33700 1 -- 172.21.15.92:0/1692334795 >> 172.21.15.92:0/1692334795 conn(0x7f27940707d0 msgr2=0x7f2794063670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:00.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.474+0000 7f279af33700 1 -- 172.21.15.92:0/1692334795 shutdown_connections 2023-10-06T10:43:00.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.474+0000 7f279af33700 1 -- 172.21.15.92:0/1692334795 wait complete. 2023-10-06T10:43:00.487 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T10:43:00.487 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T10:43:00.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.603+0000 7f10b2685700 1 Processor -- start 2023-10-06T10:43:00.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.603+0000 7f10b2685700 1 -- start start 2023-10-06T10:43:00.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.604+0000 7f10b2685700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f10ac06ece0 0x7f10ac0b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:00.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.604+0000 7f10b2685700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10ac0aede0 0x7f10ac0af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:00.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.604+0000 7f10b2685700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10ac0b2ce0 con 0x7f10ac06f0b0 2023-10-06T10:43:00.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.604+0000 7f10b2685700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10ac06fb40 con 0x7f10ac06ece0 2023-10-06T10:43:00.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.604+0000 7f10b2685700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac0b0410 con 0x7f10ac0aede0 2023-10-06T10:43:00.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.604+0000 7f10abfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f10ac06ece0 0x7f10ac0b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:00.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.604+0000 7f10ab7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f10ac06f0b0 0x7f10ac0b08a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53166/0 (socket says 172.21.15.92:53166) 2023-10-06T10:43:00.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.604+0000 7f10ab7fe700 1 -- 172.21.15.92:0/709912809 learned_addr learned my addr 172.21.15.92:0/709912809 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:00.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.605+0000 7f10b0c22700 1 --2- 172.21.15.92:0/709912809 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10ac0aede0 0x7f10ac0af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:00.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.605+0000 7f10abfff700 1 -- 172.21.15.92:0/709912809 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10ac0aede0 msgr2=0x7f10ac0af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:00.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.605+0000 7f10abfff700 1 --2- 172.21.15.92:0/709912809 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10ac0aede0 0x7f10ac0af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:00.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.605+0000 7f10abfff700 1 -- 172.21.15.92:0/709912809 >> v1:172.21.15.92:6789/0 conn(0x7f10ac06f0b0 legacy=0x7f10ac0b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:00.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.605+0000 7f10abfff700 1 -- 172.21.15.92:0/709912809 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac0af780 con 0x7f10ac06ece0 2023-10-06T10:43:00.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.605+0000 7f10aaffd700 1 -- 172.21.15.92:0/709912809 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3583451897 0 0) 0x7f10ac0b2ce0 con 0x7f10ac06f0b0 2023-10-06T10:43:00.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.605+0000 7f10abfff700 1 --2- 172.21.15.92:0/709912809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ac06ece0 0x7f10ac0b2360 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f109c0098e0 tx=0x7f109c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59fea0f5cff9cc05 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:00.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.605+0000 7f10aaffd700 1 -- 172.21.15.92:0/709912809 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c009e80 con 0x7f10ac06ece0 2023-10-06T10:43:00.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.605+0000 7f10aaffd700 1 -- 172.21.15.92:0/709912809 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109c00b860 con 0x7f10ac06ece0 2023-10-06T10:43:00.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.605+0000 7f10aaffd700 1 -- 172.21.15.92:0/709912809 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c00bbe0 con 0x7f10ac06ece0 2023-10-06T10:43:00.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.605+0000 7f10b2685700 1 -- 172.21.15.92:0/709912809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ac06ece0 msgr2=0x7f10ac0b2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:00.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.605+0000 7f10b2685700 1 --2- 172.21.15.92:0/709912809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ac06ece0 0x7f10ac0b2360 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f109c0098e0 tx=0x7f109c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:00.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.606+0000 7f10b2685700 1 -- 172.21.15.92:0/709912809 shutdown_connections 2023-10-06T10:43:00.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.606+0000 7f10b2685700 1 --2- 172.21.15.92:0/709912809 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10ac0aede0 0x7f10ac0af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:00.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.606+0000 7f10b2685700 1 --2- 172.21.15.92:0/709912809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ac06ece0 0x7f10ac0b2360 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:00.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.606+0000 7f10b2685700 1 -- 172.21.15.92:0/709912809 >> 172.21.15.92:0/709912809 conn(0x7f10ac0682f0 msgr2=0x7f10ac0686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:00.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.606+0000 7f10b2685700 1 -- 172.21.15.92:0/709912809 shutdown_connections 2023-10-06T10:43:00.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.606+0000 7f10b2685700 1 -- 172.21.15.92:0/709912809 wait complete. 2023-10-06T10:43:00.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.607+0000 7f10b2685700 1 Processor -- start 2023-10-06T10:43:00.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.607+0000 7f10b2685700 1 -- start start 2023-10-06T10:43:00.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.607+0000 7f10b2685700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10ac06ece0 0x7f10ac16d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:00.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.607+0000 7f10b2685700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ac06f0b0 0x7f10ac1560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:00.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.608+0000 7f10b2685700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10ac0aede0 0x7f10ac1565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:00.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.608+0000 7f10b2685700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac16d7a0 con 0x7f10ac06f0b0 2023-10-06T10:43:00.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.608+0000 7f10b2685700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac0aaa60 con 0x7f10ac06ece0 2023-10-06T10:43:00.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.608+0000 7f10b2685700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac0a9ee0 con 0x7f10ac0aede0 2023-10-06T10:43:00.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.608+0000 7f10ab7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ac06f0b0 0x7f10ac1560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:00.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.608+0000 7f10ab7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ac06f0b0 0x7f10ac1560b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37690/0 (socket says 172.21.15.92:37690) 2023-10-06T10:43:00.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.608+0000 7f10ab7fe700 1 -- 172.21.15.92:0/994644728 learned_addr learned my addr 172.21.15.92:0/994644728 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:00.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.608+0000 7f10abfff700 1 --2- 172.21.15.92:0/994644728 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10ac06ece0 0x7f10ac16d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:00.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.608+0000 7f10b0c22700 1 --2- 172.21.15.92:0/994644728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10ac0aede0 0x7f10ac1565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:00.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.608+0000 7f10ab7fe700 1 -- 172.21.15.92:0/994644728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10ac0aede0 msgr2=0x7f10ac1565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:00.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.608+0000 7f10ab7fe700 1 --2- 172.21.15.92:0/994644728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10ac0aede0 0x7f10ac1565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:00.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.608+0000 7f10ab7fe700 1 -- 172.21.15.92:0/994644728 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10ac06ece0 msgr2=0x7f10ac16d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:00.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.608+0000 7f10ab7fe700 1 --2- 172.21.15.92:0/994644728 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10ac06ece0 0x7f10ac16d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:00.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.608+0000 7f10ab7fe700 1 -- 172.21.15.92:0/994644728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c009420 con 0x7f10ac06f0b0 2023-10-06T10:43:00.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.608+0000 7f10ab7fe700 1 --2- 172.21.15.92:0/994644728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ac06f0b0 0x7f10ac1560b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f10a000c7e0 tx=0x7f10a0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:00.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.608+0000 7f1093fff700 1 -- 172.21.15.92:0/994644728 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a0005b30 con 0x7f10ac06f0b0 2023-10-06T10:43:00.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.608+0000 7f1093fff700 1 -- 172.21.15.92:0/994644728 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a0005c90 con 0x7f10ac06f0b0 2023-10-06T10:43:00.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.608+0000 7f1093fff700 1 -- 172.21.15.92:0/994644728 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a0007300 con 0x7f10ac06f0b0 2023-10-06T10:43:00.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.608+0000 7f10b2685700 1 -- 172.21.15.92:0/994644728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10ac0aa1c0 con 0x7f10ac06f0b0 2023-10-06T10:43:00.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.608+0000 7f10b2685700 1 -- 172.21.15.92:0/994644728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ac0aa780 con 0x7f10ac06f0b0 2023-10-06T10:43:00.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.609+0000 7f10b2685700 1 -- 172.21.15.92:0/994644728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10ac0aae90 con 0x7f10ac06f0b0 2023-10-06T10:43:00.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.610+0000 7f1093fff700 1 -- 172.21.15.92:0/994644728 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f10a001b3b0 con 0x7f10ac06f0b0 2023-10-06T10:43:00.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.610+0000 7f1093fff700 1 --2- 172.21.15.92:0/994644728 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1098066f30 0x7f10980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:00.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.610+0000 7f1093fff700 1 -- 172.21.15.92:0/994644728 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f10a00839a0 con 0x7f10ac06f0b0 2023-10-06T10:43:00.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.610+0000 7f10abfff700 1 --2- 172.21.15.92:0/994644728 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1098066f30 0x7f10980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:00.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.610+0000 7f10abfff700 1 --2- 172.21.15.92:0/994644728 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1098066f30 0x7f10980693e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f109c0091e0 tx=0x7f109c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:00.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.614+0000 7f1093fff700 1 -- 172.21.15.92:0/994644728 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f10ac0aae90 con 0x7f10ac06f0b0 2023-10-06T10:43:00.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:00.798+0000 7f10b2685700 1 -- 172.21.15.92:0/994644728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f10ac0aae90 con 0x7f10ac06f0b0 2023-10-06T10:43:01.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.778+0000 7f1093fff700 1 -- 172.21.15.92:0/994644728 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) v1 ==== 138+0+0 (secure 0 0 0) 0x7f10ac0aae90 con 0x7f10ac06f0b0 2023-10-06T10:43:01.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.781+0000 7f10b2685700 1 -- 172.21.15.92:0/994644728 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1098066f30 msgr2=0x7f10980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:01.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.781+0000 7f10b2685700 1 --2- 172.21.15.92:0/994644728 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1098066f30 0x7f10980693e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f109c0091e0 tx=0x7f109c017000 comp rx=0 tx=0).stop 2023-10-06T10:43:01.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.781+0000 7f10b2685700 1 -- 172.21.15.92:0/994644728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ac06f0b0 msgr2=0x7f10ac1560b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:01.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.781+0000 7f10b2685700 1 --2- 172.21.15.92:0/994644728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ac06f0b0 0x7f10ac1560b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f10a000c7e0 tx=0x7f10a0007df0 comp rx=0 tx=0).stop 2023-10-06T10:43:01.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.781+0000 7f10b2685700 1 -- 172.21.15.92:0/994644728 shutdown_connections 2023-10-06T10:43:01.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.781+0000 7f10b2685700 1 --2- 172.21.15.92:0/994644728 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1098066f30 0x7f10980693e0 secure :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f109c0091e0 tx=0x7f109c017000 comp rx=0 tx=0).stop 2023-10-06T10:43:01.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.781+0000 7f10b2685700 1 --2- 172.21.15.92:0/994644728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10ac0aede0 0x7f10ac1565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:01.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.781+0000 7f10b2685700 1 --2- 172.21.15.92:0/994644728 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10ac06ece0 0x7f10ac16d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:01.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.781+0000 7f10b2685700 1 --2- 172.21.15.92:0/994644728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ac06f0b0 0x7f10ac1560b0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:01.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.781+0000 7f10b2685700 1 -- 172.21.15.92:0/994644728 >> 172.21.15.92:0/994644728 conn(0x7f10ac0682f0 msgr2=0x7f10ac0cd240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:01.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.782+0000 7f10b2685700 1 -- 172.21.15.92:0/994644728 shutdown_connections 2023-10-06T10:43:01.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.782+0000 7f10b2685700 1 -- 172.21.15.92:0/994644728 wait complete. 2023-10-06T10:43:01.786 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-10-06T10:43:01.798 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T10:43:01.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.917+0000 7f4b6e936700 1 Processor -- start 2023-10-06T10:43:01.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.917+0000 7f4b6e936700 1 -- start start 2023-10-06T10:43:01.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.918+0000 7f4b6e936700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b681594d0 0x7f4b680cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:01.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.918+0000 7f4b6e936700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4b681598a0 0x7f4b680cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:01.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.918+0000 7f4b6e936700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b680cf310 con 0x7f4b680ce7e0 2023-10-06T10:43:01.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.918+0000 7f4b6e936700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b680cf5d0 con 0x7f4b681598a0 2023-10-06T10:43:01.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.918+0000 7f4b6e936700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b680cf8b0 con 0x7f4b681594d0 2023-10-06T10:43:01.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.918+0000 7f4b6ced3700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4b680ce7e0 0x7f4b680cec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53176/0 (socket says 172.21.15.92:53176) 2023-10-06T10:43:01.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.918+0000 7f4b6ced3700 1 -- 172.21.15.92:0/282228279 learned_addr learned my addr 172.21.15.92:0/282228279 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:01.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.918+0000 7f4b67fff700 1 --2- 172.21.15.92:0/282228279 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b681594d0 0x7f4b680cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:01.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.918+0000 7f4b677fe700 1 --2- 172.21.15.92:0/282228279 >> v2:172.21.15.124:3300/0 conn(0x7f4b681598a0 0x7f4b680cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:01.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b66ffd700 1 -- 172.21.15.92:0/282228279 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 464090884 0 0) 0x7f4b680cf310 con 0x7f4b680ce7e0 2023-10-06T10:43:01.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b66ffd700 1 -- 172.21.15.92:0/282228279 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b50003570 con 0x7f4b680ce7e0 2023-10-06T10:43:01.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b66ffd700 1 -- 172.21.15.92:0/282228279 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1671699551 0 0) 0x7f4b50003570 con 0x7f4b680ce7e0 2023-10-06T10:43:01.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b66ffd700 1 -- 172.21.15.92:0/282228279 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b681594d0 msgr2=0x7f4b680cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:01.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b66ffd700 1 --2- 172.21.15.92:0/282228279 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b681594d0 0x7f4b680cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:01.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b66ffd700 1 -- 172.21.15.92:0/282228279 >> v2:172.21.15.124:3300/0 conn(0x7f4b681598a0 msgr2=0x7f4b680cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:01.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b66ffd700 1 --2- 172.21.15.92:0/282228279 >> v2:172.21.15.124:3300/0 conn(0x7f4b681598a0 0x7f4b680cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:01.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b66ffd700 1 -- 172.21.15.92:0/282228279 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b680cfb90 con 0x7f4b680ce7e0 2023-10-06T10:43:01.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b66ffd700 1 -- 172.21.15.92:0/282228279 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f4b600017f0 con 0x7f4b680ce7e0 2023-10-06T10:43:01.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b66ffd700 1 -- 172.21.15.92:0/282228279 >> v1:172.21.15.92:6789/0 conn(0x7f4b680ce7e0 legacy=0x7f4b680cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:01.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b66ffd700 1 --2- 172.21.15.92:0/282228279 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b681594d0 0x7f4b50003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:01.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b66ffd700 1 --2- 172.21.15.92:0/282228279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b50003fe0 0x7f4b50004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:01.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b66ffd700 1 --2- 172.21.15.92:0/282228279 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b50004d50 0x7f4b500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:01.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b66ffd700 1 -- 172.21.15.92:0/282228279 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b680cf5d0 con 0x7f4b50003fe0 2023-10-06T10:43:01.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b66ffd700 1 -- 172.21.15.92:0/282228279 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b680cf8b0 con 0x7f4b50004d50 2023-10-06T10:43:01.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b66ffd700 1 -- 172.21.15.92:0/282228279 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5000ba90 con 0x7f4b681594d0 2023-10-06T10:43:01.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b677fe700 1 --2- 172.21.15.92:0/282228279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b50003fe0 0x7f4b50004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:01.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b67fff700 1 --2- 172.21.15.92:0/282228279 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b681594d0 0x7f4b50003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:01.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b6ced3700 1 --2- 172.21.15.92:0/282228279 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b50004d50 0x7f4b500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:01.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b677fe700 1 -- 172.21.15.92:0/282228279 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b681594d0 msgr2=0x7f4b50003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:01.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b677fe700 1 --2- 172.21.15.92:0/282228279 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b681594d0 0x7f4b50003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:01.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b677fe700 1 -- 172.21.15.92:0/282228279 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b50004d50 msgr2=0x7f4b500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:01.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b677fe700 1 --2- 172.21.15.92:0/282228279 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b50004d50 0x7f4b500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:01.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.919+0000 7f4b677fe700 1 -- 172.21.15.92:0/282228279 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b50003570 con 0x7f4b50003fe0 2023-10-06T10:43:01.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.920+0000 7f4b677fe700 1 --2- 172.21.15.92:0/282228279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b50003fe0 0x7f4b50004810 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f4b68159c90 tx=0x7f4b5c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c0fe3cebe402c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:01.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.920+0000 7f4b66ffd700 1 -- 172.21.15.92:0/282228279 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c00e6b0 con 0x7f4b50003fe0 2023-10-06T10:43:01.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.920+0000 7f4b66ffd700 1 -- 172.21.15.92:0/282228279 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5c00e810 con 0x7f4b50003fe0 2023-10-06T10:43:01.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.920+0000 7f4b66ffd700 1 -- 172.21.15.92:0/282228279 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c00ebd0 con 0x7f4b50003fe0 2023-10-06T10:43:01.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.920+0000 7f4b6e936700 1 -- 172.21.15.92:0/282228279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b50003fe0 msgr2=0x7f4b50004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:01.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.920+0000 7f4b6e936700 1 --2- 172.21.15.92:0/282228279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b50003fe0 0x7f4b50004810 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f4b68159c90 tx=0x7f4b5c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:01.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.920+0000 7f4b6ced3700 1 -- 172.21.15.92:0/282228279 reap_dead start 2023-10-06T10:43:01.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.920+0000 7f4b6e936700 1 -- 172.21.15.92:0/282228279 shutdown_connections 2023-10-06T10:43:01.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.920+0000 7f4b6e936700 1 -- 172.21.15.92:0/282228279 >> 172.21.15.92:0/282228279 conn(0x7f4b68089e50 msgr2=0x7f4b68099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:01.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.920+0000 7f4b6e936700 1 -- 172.21.15.92:0/282228279 shutdown_connections 2023-10-06T10:43:01.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.920+0000 7f4b6e936700 1 -- 172.21.15.92:0/282228279 wait complete. 2023-10-06T10:43:01.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.921+0000 7f4b6e936700 1 Processor -- start 2023-10-06T10:43:01.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.921+0000 7f4b6e936700 1 -- start start 2023-10-06T10:43:01.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.922+0000 7f4b6e936700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b681481b0 0x7f4b6816adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:01.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.922+0000 7f4b6e936700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b680d04a0 0x7f4b680d0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:01.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.922+0000 7f4b6e936700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b680c9b60 0x7f4b680ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:01.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.922+0000 7f4b6e936700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b680d0e50 con 0x7f4b680c9b60 2023-10-06T10:43:01.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.922+0000 7f4b6e936700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b680c8370 con 0x7f4b680d04a0 2023-10-06T10:43:01.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.922+0000 7f4b6e936700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b68148580 con 0x7f4b681481b0 2023-10-06T10:43:01.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.922+0000 7f4b6ced3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b680c9b60 0x7f4b680ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:01.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.922+0000 7f4b6ced3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b680c9b60 0x7f4b680ca000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37708/0 (socket says 172.21.15.92:37708) 2023-10-06T10:43:01.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.922+0000 7f4b6ced3700 1 -- 172.21.15.92:0/1950598321 learned_addr learned my addr 172.21.15.92:0/1950598321 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:01.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.922+0000 7f4b67fff700 1 --2- 172.21.15.92:0/1950598321 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b681481b0 0x7f4b6816adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:01.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.922+0000 7f4b677fe700 1 --2- 172.21.15.92:0/1950598321 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b680d04a0 0x7f4b680d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:01.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.923+0000 7f4b6ced3700 1 -- 172.21.15.92:0/1950598321 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b681481b0 msgr2=0x7f4b6816adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:01.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.923+0000 7f4b6ced3700 1 --2- 172.21.15.92:0/1950598321 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b681481b0 0x7f4b6816adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:01.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.923+0000 7f4b6ced3700 1 -- 172.21.15.92:0/1950598321 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b680d04a0 msgr2=0x7f4b680d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:01.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.923+0000 7f4b6ced3700 1 --2- 172.21.15.92:0/1950598321 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b680d04a0 0x7f4b680d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:01.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.923+0000 7f4b6ced3700 1 -- 172.21.15.92:0/1950598321 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b680ca540 con 0x7f4b680c9b60 2023-10-06T10:43:01.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.923+0000 7f4b6ced3700 1 --2- 172.21.15.92:0/1950598321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b680c9b60 0x7f4b680ca000 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4b68088cb0 tx=0x7f4b60002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:01.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.923+0000 7f4b4ffff700 1 -- 172.21.15.92:0/1950598321 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b600133f0 con 0x7f4b680c9b60 2023-10-06T10:43:01.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.923+0000 7f4b4ffff700 1 -- 172.21.15.92:0/1950598321 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b60013570 con 0x7f4b680c9b60 2023-10-06T10:43:01.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.923+0000 7f4b6e936700 1 -- 172.21.15.92:0/1950598321 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6816b300 con 0x7f4b680c9b60 2023-10-06T10:43:01.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.923+0000 7f4b4ffff700 1 -- 172.21.15.92:0/1950598321 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b60013930 con 0x7f4b680c9b60 2023-10-06T10:43:01.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.923+0000 7f4b6e936700 1 -- 172.21.15.92:0/1950598321 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b680c7a40 con 0x7f4b680c9b60 2023-10-06T10:43:01.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.924+0000 7f4b6e936700 1 -- 172.21.15.92:0/1950598321 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b68064a90 con 0x7f4b680c9b60 2023-10-06T10:43:01.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.924+0000 7f4b4ffff700 1 -- 172.21.15.92:0/1950598321 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4b60013ca0 con 0x7f4b680c9b60 2023-10-06T10:43:01.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.925+0000 7f4b4ffff700 1 --2- 172.21.15.92:0/1950598321 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4b54066f30 0x7f4b540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:01.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.925+0000 7f4b4ffff700 1 -- 172.21.15.92:0/1950598321 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5896+0+0 (secure 0 0 0) 0x7f4b600851f0 con 0x7f4b680c9b60 2023-10-06T10:43:01.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.925+0000 7f4b67fff700 1 --2- 172.21.15.92:0/1950598321 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4b54066f30 0x7f4b540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:01.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.925+0000 7f4b67fff700 1 --2- 172.21.15.92:0/1950598321 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4b54066f30 0x7f4b540693e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4b680ca7e0 tx=0x7f4b580091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:01.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:01.929+0000 7f4b4ffff700 1 -- 172.21.15.92:0/1950598321 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b600523a0 con 0x7f4b680c9b60 2023-10-06T10:43:02.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.110+0000 7f4b6e936700 1 -- 172.21.15.92:0/1950598321 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4b680cfe70 con 0x7f4b680c9b60 2023-10-06T10:43:02.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.785+0000 7f4b4ffff700 1 -- 172.21.15.92:0/1950598321 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v76) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4b600559c0 con 0x7f4b680c9b60 2023-10-06T10:43:02.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.789+0000 7f4b6e936700 1 -- 172.21.15.92:0/1950598321 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4b54066f30 msgr2=0x7f4b540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:02.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.789+0000 7f4b6e936700 1 --2- 172.21.15.92:0/1950598321 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4b54066f30 0x7f4b540693e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4b680ca7e0 tx=0x7f4b580091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:02.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.789+0000 7f4b6e936700 1 -- 172.21.15.92:0/1950598321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b680c9b60 msgr2=0x7f4b680ca000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:02.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.789+0000 7f4b6e936700 1 --2- 172.21.15.92:0/1950598321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b680c9b60 0x7f4b680ca000 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4b68088cb0 tx=0x7f4b60002210 comp rx=0 tx=0).stop 2023-10-06T10:43:02.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.790+0000 7f4b6e936700 1 -- 172.21.15.92:0/1950598321 shutdown_connections 2023-10-06T10:43:02.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.790+0000 7f4b6e936700 1 --2- 172.21.15.92:0/1950598321 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4b54066f30 0x7f4b540693e0 secure :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4b680ca7e0 tx=0x7f4b580091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:02.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.790+0000 7f4b6e936700 1 --2- 172.21.15.92:0/1950598321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b680c9b60 0x7f4b680ca000 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:02.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.790+0000 7f4b6e936700 1 --2- 172.21.15.92:0/1950598321 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b681481b0 0x7f4b6816adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:02.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.790+0000 7f4b6e936700 1 --2- 172.21.15.92:0/1950598321 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b680d04a0 0x7f4b680d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:02.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.790+0000 7f4b6e936700 1 -- 172.21.15.92:0/1950598321 >> 172.21.15.92:0/1950598321 conn(0x7f4b68089e50 msgr2=0x7f4b681544c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:02.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.795+0000 7f4b6e936700 1 -- 172.21.15.92:0/1950598321 shutdown_connections 2023-10-06T10:43:02.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.796+0000 7f4b6e936700 1 -- 172.21.15.92:0/1950598321 wait complete. 2023-10-06T10:43:02.806 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-10-06T10:43:02.817 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T10:43:02.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.933+0000 7fa9f68ac700 1 Processor -- start 2023-10-06T10:43:02.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.933+0000 7fa9f68ac700 1 -- start start 2023-10-06T10:43:02.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.934+0000 7fa9f68ac700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9f00b2620 0x7fa9f00cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:02.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.934+0000 7fa9f68ac700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa9f00b29f0 0x7fa9f00cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:02.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.934+0000 7fa9f68ac700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa9f00ceaa0 0x7fa9f00cef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:02.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.934+0000 7fa9f68ac700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f00cc970 con 0x7fa9f00ceaa0 2023-10-06T10:43:02.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.934+0000 7fa9f68ac700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f00cf480 con 0x7fa9f00b29f0 2023-10-06T10:43:02.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.934+0000 7fa9f68ac700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00cbd60 con 0x7fa9f00b2620 2023-10-06T10:43:02.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.934+0000 7fa9f60ab700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa9f00ceaa0 0x7fa9f00cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:02.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.934+0000 7fa9f60ab700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa9f00ceaa0 0x7fa9f00cef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37724/0 (socket says 172.21.15.92:37724) 2023-10-06T10:43:02.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.934+0000 7fa9f60ab700 1 -- 172.21.15.92:0/4164708741 learned_addr learned my addr 172.21.15.92:0/4164708741 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:02.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.934+0000 7fa9f58aa700 1 --2- 172.21.15.92:0/4164708741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9f00b2620 0x7fa9f00cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:02.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.934+0000 7fa9f50a9700 1 --2- 172.21.15.92:0/4164708741 >> v2:172.21.15.124:3300/0 conn(0x7fa9f00b29f0 0x7fa9f00cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:02.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.934+0000 7fa9f60ab700 1 -- 172.21.15.92:0/4164708741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9f00b2620 msgr2=0x7fa9f00cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:02.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.934+0000 7fa9f60ab700 1 --2- 172.21.15.92:0/4164708741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9f00b2620 0x7fa9f00cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:02.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.934+0000 7fa9f60ab700 1 -- 172.21.15.92:0/4164708741 >> v2:172.21.15.124:3300/0 conn(0x7fa9f00b29f0 msgr2=0x7fa9f00cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:02.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.934+0000 7fa9f60ab700 1 --2- 172.21.15.92:0/4164708741 >> v2:172.21.15.124:3300/0 conn(0x7fa9f00b29f0 0x7fa9f00cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:02.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.934+0000 7fa9f60ab700 1 -- 172.21.15.92:0/4164708741 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f00cc040 con 0x7fa9f00ceaa0 2023-10-06T10:43:02.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.934+0000 7fa9f60ab700 1 --2- 172.21.15.92:0/4164708741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9f00ceaa0 0x7fa9f00cef40 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fa9f00893b0 tx=0x7fa9e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ebf1350ea1fd2ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:02.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.935+0000 7fa9f48a8700 1 -- 172.21.15.92:0/4164708741 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e000b640 con 0x7fa9f00ceaa0 2023-10-06T10:43:02.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.935+0000 7fa9f48a8700 1 -- 172.21.15.92:0/4164708741 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e000b7a0 con 0x7fa9f00ceaa0 2023-10-06T10:43:02.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.935+0000 7fa9f48a8700 1 -- 172.21.15.92:0/4164708741 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e000bb60 con 0x7fa9f00ceaa0 2023-10-06T10:43:02.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.935+0000 7fa9f68ac700 1 -- 172.21.15.92:0/4164708741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9f00ceaa0 msgr2=0x7fa9f00cef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:02.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.935+0000 7fa9f68ac700 1 --2- 172.21.15.92:0/4164708741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9f00ceaa0 0x7fa9f00cef40 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fa9f00893b0 tx=0x7fa9e0009250 comp rx=0 tx=0).stop 2023-10-06T10:43:02.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.935+0000 7fa9f68ac700 1 -- 172.21.15.92:0/4164708741 shutdown_connections 2023-10-06T10:43:02.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.935+0000 7fa9f68ac700 1 --2- 172.21.15.92:0/4164708741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9f00ceaa0 0x7fa9f00cef40 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:02.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.935+0000 7fa9f68ac700 1 --2- 172.21.15.92:0/4164708741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9f00b2620 0x7fa9f00cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:02.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.935+0000 7fa9f68ac700 1 --2- 172.21.15.92:0/4164708741 >> v2:172.21.15.124:3300/0 conn(0x7fa9f00b29f0 0x7fa9f00cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:02.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.935+0000 7fa9f68ac700 1 -- 172.21.15.92:0/4164708741 >> 172.21.15.92:0/4164708741 conn(0x7fa9f0099520 msgr2=0x7fa9f006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:02.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.935+0000 7fa9f68ac700 1 -- 172.21.15.92:0/4164708741 shutdown_connections 2023-10-06T10:43:02.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.935+0000 7fa9f68ac700 1 -- 172.21.15.92:0/4164708741 wait complete. 2023-10-06T10:43:02.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.936+0000 7fa9f68ac700 1 Processor -- start 2023-10-06T10:43:02.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.936+0000 7fa9f68ac700 1 -- start start 2023-10-06T10:43:02.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.937+0000 7fa9f68ac700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9f00b2620 0x7fa9f00b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:02.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.937+0000 7fa9f68ac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9f00b29f0 0x7fa9f0154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:02.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.937+0000 7fa9f68ac700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9f00ceaa0 0x7fa9f00c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:02.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.937+0000 7fa9f68ac700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f01415e0 con 0x7fa9f00b29f0 2023-10-06T10:43:02.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.937+0000 7fa9f68ac700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00c91d0 con 0x7fa9f00b2620 2023-10-06T10:43:02.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.937+0000 7fa9f68ac700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00c85c0 con 0x7fa9f00ceaa0 2023-10-06T10:43:02.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.937+0000 7fa9f50a9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9f00b29f0 0x7fa9f0154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:02.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.937+0000 7fa9f50a9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9f00b29f0 0x7fa9f0154980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37734/0 (socket says 172.21.15.92:37734) 2023-10-06T10:43:02.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.937+0000 7fa9f50a9700 1 -- 172.21.15.92:0/1675289898 learned_addr learned my addr 172.21.15.92:0/1675289898 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:02.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.937+0000 7fa9f58aa700 1 --2- 172.21.15.92:0/1675289898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9f00b2620 0x7fa9f00b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:02.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.937+0000 7fa9f60ab700 1 --2- 172.21.15.92:0/1675289898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9f00ceaa0 0x7fa9f00c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:02.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.937+0000 7fa9f50a9700 1 -- 172.21.15.92:0/1675289898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9f00ceaa0 msgr2=0x7fa9f00c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:02.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.937+0000 7fa9f50a9700 1 --2- 172.21.15.92:0/1675289898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9f00ceaa0 0x7fa9f00c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:02.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.937+0000 7fa9f50a9700 1 -- 172.21.15.92:0/1675289898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9f00b2620 msgr2=0x7fa9f00b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:02.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.937+0000 7fa9f50a9700 1 --2- 172.21.15.92:0/1675289898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9f00b2620 0x7fa9f00b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:02.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.938+0000 7fa9f50a9700 1 -- 172.21.15.92:0/1675289898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e00094f0 con 0x7fa9f00b29f0 2023-10-06T10:43:02.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.938+0000 7fa9f60ab700 1 --2- 172.21.15.92:0/1675289898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9f00ceaa0 0x7fa9f00c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:02.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.938+0000 7fa9f50a9700 1 --2- 172.21.15.92:0/1675289898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9f00b29f0 0x7fa9f0154980 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400ecb0 tx=0x7fa9e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:02.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.938+0000 7fa9edffb700 1 -- 172.21.15.92:0/1675289898 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e400c8c0 con 0x7fa9f00b29f0 2023-10-06T10:43:02.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.938+0000 7fa9edffb700 1 -- 172.21.15.92:0/1675289898 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e400ca20 con 0x7fa9f00b29f0 2023-10-06T10:43:02.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.938+0000 7fa9f68ac700 1 -- 172.21.15.92:0/1675289898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f00c88a0 con 0x7fa9f00b29f0 2023-10-06T10:43:02.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.938+0000 7fa9edffb700 1 -- 172.21.15.92:0/1675289898 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e400cdc0 con 0x7fa9f00b29f0 2023-10-06T10:43:02.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.938+0000 7fa9f68ac700 1 -- 172.21.15.92:0/1675289898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f00c8e60 con 0x7fa9f00b29f0 2023-10-06T10:43:02.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.939+0000 7fa9f68ac700 1 -- 172.21.15.92:0/1675289898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f005f230 con 0x7fa9f00b29f0 2023-10-06T10:43:02.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.939+0000 7fa9edffb700 1 -- 172.21.15.92:0/1675289898 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa9e4013680 con 0x7fa9f00b29f0 2023-10-06T10:43:02.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.940+0000 7fa9edffb700 1 --2- 172.21.15.92:0/1675289898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa9dc066f30 0x7fa9dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:02.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.940+0000 7fa9edffb700 1 -- 172.21.15.92:0/1675289898 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6278+0+0 (secure 0 0 0) 0x7fa9e4084a40 con 0x7fa9f00b29f0 2023-10-06T10:43:02.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.940+0000 7fa9f58aa700 1 --2- 172.21.15.92:0/1675289898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa9dc066f30 0x7fa9dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:02.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.940+0000 7fa9f58aa700 1 --2- 172.21.15.92:0/1675289898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa9dc066f30 0x7fa9dc0693e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800f510 tx=0x7fa9e800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:02.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:02.944+0000 7fa9edffb700 1 -- 172.21.15.92:0/1675289898 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9e4086e60 con 0x7fa9f00b29f0 2023-10-06T10:43:03.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.171+0000 7fa9f68ac700 1 -- 172.21.15.92:0/1675289898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa9f00c9310 con 0x7fa9f00b29f0 2023-10-06T10:43:03.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.786+0000 7fa9edffb700 1 -- 172.21.15.92:0/1675289898 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v63) v1 ==== 174+0+0 (secure 0 0 0) 0x7fa9e4051a70 con 0x7fa9f00b29f0 2023-10-06T10:43:03.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.790+0000 7fa9f68ac700 1 -- 172.21.15.92:0/1675289898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa9dc066f30 msgr2=0x7fa9dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:03.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.790+0000 7fa9f68ac700 1 --2- 172.21.15.92:0/1675289898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa9dc066f30 0x7fa9dc0693e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800f510 tx=0x7fa9e800b040 comp rx=0 tx=0).stop 2023-10-06T10:43:03.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.790+0000 7fa9f68ac700 1 -- 172.21.15.92:0/1675289898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9f00b29f0 msgr2=0x7fa9f0154980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:03.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.790+0000 7fa9f68ac700 1 --2- 172.21.15.92:0/1675289898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9f00b29f0 0x7fa9f0154980 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400ecb0 tx=0x7fa9e40091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:03.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.791+0000 7fa9f68ac700 1 -- 172.21.15.92:0/1675289898 shutdown_connections 2023-10-06T10:43:03.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.791+0000 7fa9f68ac700 1 --2- 172.21.15.92:0/1675289898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa9dc066f30 0x7fa9dc0693e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:03.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.791+0000 7fa9f68ac700 1 --2- 172.21.15.92:0/1675289898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9f00ceaa0 0x7fa9f00c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:03.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.791+0000 7fa9f68ac700 1 --2- 172.21.15.92:0/1675289898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9f00b2620 0x7fa9f00b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:03.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.791+0000 7fa9f68ac700 1 --2- 172.21.15.92:0/1675289898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9f00b29f0 0x7fa9f0154980 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:03.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.791+0000 7fa9f68ac700 1 -- 172.21.15.92:0/1675289898 >> 172.21.15.92:0/1675289898 conn(0x7fa9f0099520 msgr2=0x7fa9f0068b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:03.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.793+0000 7fa9f68ac700 1 -- 172.21.15.92:0/1675289898 shutdown_connections 2023-10-06T10:43:03.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.793+0000 7fa9f68ac700 1 -- 172.21.15.92:0/1675289898 wait complete. 2023-10-06T10:43:03.797 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 12 and data pool 13 2023-10-06T10:43:03.810 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:43:03.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.923+0000 7ff4c3fff700 1 Processor -- start 2023-10-06T10:43:03.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.923+0000 7ff4c3fff700 1 -- start start 2023-10-06T10:43:03.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.924+0000 7ff4c3fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4c40b2810 0x7ff4c40cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:03.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.924+0000 7ff4c3fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c40cb7a0 con 0x7ff4c4159440 2023-10-06T10:43:03.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.924+0000 7ff4c3fff700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c40b0860 con 0x7ff4c40cbc30 2023-10-06T10:43:03.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.924+0000 7ff4c3fff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c40cb150 con 0x7ff4c40b2810 2023-10-06T10:43:03.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.924+0000 7ff4c2ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff4c4159440 0x7ff4c40b3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51848/0 (socket says 172.21.15.92:51848) 2023-10-06T10:43:03.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.924+0000 7ff4c2ffd700 1 -- 172.21.15.92:0/3419952484 learned_addr learned my addr 172.21.15.92:0/3419952484 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:03.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.924+0000 7ff4c37fe700 1 --2- 172.21.15.92:0/3419952484 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4c40b2810 0x7ff4c40cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:03.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.924+0000 7ff4c27fc700 1 -- 172.21.15.92:0/3419952484 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1646010591 0 0) 0x7ff4c40cb7a0 con 0x7ff4c4159440 2023-10-06T10:43:03.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.924+0000 7ff4c27fc700 1 -- 172.21.15.92:0/3419952484 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4b0003570 con 0x7ff4c4159440 2023-10-06T10:43:03.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.924+0000 7ff4c27fc700 1 -- 172.21.15.92:0/3419952484 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722303251 0 0) 0x7ff4c40b0860 con 0x7ff4c40cbc30 2023-10-06T10:43:03.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.925+0000 7ff4c27fc700 1 -- 172.21.15.92:0/3419952484 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4c40cb7a0 con 0x7ff4c40cbc30 2023-10-06T10:43:03.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.925+0000 7ff4c27fc700 1 -- 172.21.15.92:0/3419952484 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3266960924 0 0) 0x7ff4b0003570 con 0x7ff4c4159440 2023-10-06T10:43:03.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.925+0000 7ff4c37fe700 1 -- 172.21.15.92:0/3419952484 >> v1:172.21.15.124:6789/0 conn(0x7ff4c40cbc30 legacy=0x7ff4c40b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:03.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.925+0000 7ff4c37fe700 1 -- 172.21.15.92:0/3419952484 >> v1:172.21.15.92:6789/0 conn(0x7ff4c4159440 legacy=0x7ff4c40b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:03.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.925+0000 7ff4c37fe700 1 -- 172.21.15.92:0/3419952484 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c40cb430 con 0x7ff4c40b2810 2023-10-06T10:43:03.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.925+0000 7ff4c37fe700 1 --2- 172.21.15.92:0/3419952484 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4c40b2810 0x7ff4c40cc080 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009e90 tx=0x7ff4bc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7977dd49f2e49a3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:03.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.925+0000 7ff4c27fc700 1 -- 172.21.15.92:0/3419952484 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc00e070 con 0x7ff4c40b2810 2023-10-06T10:43:03.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.925+0000 7ff4c27fc700 1 -- 172.21.15.92:0/3419952484 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc00b870 con 0x7ff4c40b2810 2023-10-06T10:43:03.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.925+0000 7ff4c27fc700 1 -- 172.21.15.92:0/3419952484 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4001470 con 0x7ff4c40b2810 2023-10-06T10:43:03.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.925+0000 7ff4c3fff700 1 -- 172.21.15.92:0/3419952484 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4c40b2810 msgr2=0x7ff4c40cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:03.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.925+0000 7ff4c3fff700 1 --2- 172.21.15.92:0/3419952484 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4c40b2810 0x7ff4c40cc080 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009e90 tx=0x7ff4bc009780 comp rx=0 tx=0).stop 2023-10-06T10:43:03.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.925+0000 7ff4c3fff700 1 -- 172.21.15.92:0/3419952484 shutdown_connections 2023-10-06T10:43:03.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.925+0000 7ff4c3fff700 1 --2- 172.21.15.92:0/3419952484 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4c40b2810 0x7ff4c40cc080 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:03.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.925+0000 7ff4c3fff700 1 -- 172.21.15.92:0/3419952484 >> 172.21.15.92:0/3419952484 conn(0x7ff4c406ece0 msgr2=0x7ff4c406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:03.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.925+0000 7ff4c3fff700 1 -- 172.21.15.92:0/3419952484 shutdown_connections 2023-10-06T10:43:03.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.925+0000 7ff4c3fff700 1 -- 172.21.15.92:0/3419952484 wait complete. 2023-10-06T10:43:03.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.927+0000 7ff4c3fff700 1 Processor -- start 2023-10-06T10:43:03.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.927+0000 7ff4c3fff700 1 -- start start 2023-10-06T10:43:03.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.927+0000 7ff4c3fff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4c40b2810 0x7ff4c40d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:03.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.927+0000 7ff4c3fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4c40cbc30 0x7ff4c40d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:03.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.927+0000 7ff4c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c4159440 0x7ff4c41487e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:03.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.927+0000 7ff4c3fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c40d0140 con 0x7ff4c4159440 2023-10-06T10:43:03.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.927+0000 7ff4c3fff700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c415d8b0 con 0x7ff4c40b2810 2023-10-06T10:43:03.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.927+0000 7ff4c3fff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c40cfaf0 con 0x7ff4c40cbc30 2023-10-06T10:43:03.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.928+0000 7ff4c37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c4159440 0x7ff4c41487e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:03.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.928+0000 7ff4c37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c4159440 0x7ff4c41487e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33812/0 (socket says 172.21.15.92:33812) 2023-10-06T10:43:03.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.928+0000 7ff4c37fe700 1 -- 172.21.15.92:0/1615436394 learned_addr learned my addr 172.21.15.92:0/1615436394 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:03.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.928+0000 7ff4c2ffd700 1 --2- 172.21.15.92:0/1615436394 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4c40b2810 0x7ff4c40d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:03.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.928+0000 7ff4ba7fc700 1 --2- 172.21.15.92:0/1615436394 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4c40cbc30 0x7ff4c40d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:03.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.928+0000 7ff4c37fe700 1 -- 172.21.15.92:0/1615436394 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4c40cbc30 msgr2=0x7ff4c40d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:03.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.928+0000 7ff4c37fe700 1 --2- 172.21.15.92:0/1615436394 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4c40cbc30 0x7ff4c40d05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:03.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.928+0000 7ff4c37fe700 1 -- 172.21.15.92:0/1615436394 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4c40b2810 msgr2=0x7ff4c40d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:03.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.928+0000 7ff4c37fe700 1 --2- 172.21.15.92:0/1615436394 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4c40b2810 0x7ff4c40d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:03.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.928+0000 7ff4c37fe700 1 -- 172.21.15.92:0/1615436394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4bc009a20 con 0x7ff4c4159440 2023-10-06T10:43:03.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.928+0000 7ff4c37fe700 1 --2- 172.21.15.92:0/1615436394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c4159440 0x7ff4c41487e0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4069150 tx=0x7ff4bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:03.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.928+0000 7ff4bb7fe700 1 -- 172.21.15.92:0/1615436394 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc012bb0 con 0x7ff4c4159440 2023-10-06T10:43:03.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.928+0000 7ff4bb7fe700 1 -- 172.21.15.92:0/1615436394 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc012d30 con 0x7ff4c4159440 2023-10-06T10:43:03.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.928+0000 7ff4c3fff700 1 -- 172.21.15.92:0/1615436394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c40cfdd0 con 0x7ff4c4159440 2023-10-06T10:43:03.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.928+0000 7ff4bb7fe700 1 -- 172.21.15.92:0/1615436394 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc012ea0 con 0x7ff4c4159440 2023-10-06T10:43:03.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.928+0000 7ff4c3fff700 1 -- 172.21.15.92:0/1615436394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c41559b0 con 0x7ff4c4159440 2023-10-06T10:43:03.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.929+0000 7ff4c3fff700 1 -- 172.21.15.92:0/1615436394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c405f230 con 0x7ff4c4159440 2023-10-06T10:43:03.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.929+0000 7ff4bb7fe700 1 -- 172.21.15.92:0/1615436394 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff4bc01e8b0 con 0x7ff4c4159440 2023-10-06T10:43:03.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.930+0000 7ff4bb7fe700 1 --2- 172.21.15.92:0/1615436394 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff4a4066f30 0x7ff4a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:03.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.930+0000 7ff4bb7fe700 1 -- 172.21.15.92:0/1615436394 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff4bc086cb0 con 0x7ff4c4159440 2023-10-06T10:43:03.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.930+0000 7ff4c2ffd700 1 --2- 172.21.15.92:0/1615436394 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff4a4066f30 0x7ff4a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:03.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.930+0000 7ff4c2ffd700 1 --2- 172.21.15.92:0/1615436394 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff4a4066f30 0x7ff4a40693e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4157970 tx=0x7ff4b4000bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:03.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:03.934+0000 7ff4bb7fe700 1 -- 172.21.15.92:0/1615436394 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4bc053c80 con 0x7ff4c4159440 2023-10-06T10:43:04.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.118+0000 7ff4c3fff700 1 -- 172.21.15.92:0/1615436394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4c40d0280 con 0x7ff4c4159440 2023-10-06T10:43:04.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.119+0000 7ff4bb7fe700 1 -- 172.21.15.92:0/1615436394 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+17087 (secure 0 0 0) 0x7ff4bc0572a0 con 0x7ff4c4159440 2023-10-06T10:43:04.121 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:04.121 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":77,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:43:03.172745+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:43:00.816800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-06T10:43:02.128579+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:43:04.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.121+0000 7ff4c3fff700 1 -- 172.21.15.92:0/1615436394 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff4a4066f30 msgr2=0x7ff4a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:04.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.121+0000 7ff4c3fff700 1 --2- 172.21.15.92:0/1615436394 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff4a4066f30 0x7ff4a40693e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4157970 tx=0x7ff4b4000bc0 comp rx=0 tx=0).stop 2023-10-06T10:43:04.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.121+0000 7ff4c3fff700 1 -- 172.21.15.92:0/1615436394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c4159440 msgr2=0x7ff4c41487e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:04.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.121+0000 7ff4c3fff700 1 --2- 172.21.15.92:0/1615436394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c4159440 0x7ff4c41487e0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4069150 tx=0x7ff4bc017000 comp rx=0 tx=0).stop 2023-10-06T10:43:04.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.121+0000 7ff4c3fff700 1 -- 172.21.15.92:0/1615436394 shutdown_connections 2023-10-06T10:43:04.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.121+0000 7ff4c3fff700 1 --2- 172.21.15.92:0/1615436394 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff4a4066f30 0x7ff4a40693e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:04.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.121+0000 7ff4c3fff700 1 --2- 172.21.15.92:0/1615436394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c4159440 0x7ff4c41487e0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:04.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.121+0000 7ff4c3fff700 1 --2- 172.21.15.92:0/1615436394 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4c40b2810 0x7ff4c40d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:04.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.121+0000 7ff4c3fff700 1 --2- 172.21.15.92:0/1615436394 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4c40cbc30 0x7ff4c40d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:04.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.121+0000 7ff4c3fff700 1 -- 172.21.15.92:0/1615436394 >> 172.21.15.92:0/1615436394 conn(0x7ff4c406ece0 msgr2=0x7ff4c40b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:04.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.121+0000 7ff4c3fff700 1 -- 172.21.15.92:0/1615436394 shutdown_connections 2023-10-06T10:43:04.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.121+0000 7ff4c3fff700 1 -- 172.21.15.92:0/1615436394 wait complete. 2023-10-06T10:43:04.136 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:43:04.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.249+0000 7f94866e4700 1 Processor -- start 2023-10-06T10:43:04.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.249+0000 7f94866e4700 1 -- start start 2023-10-06T10:43:04.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.249+0000 7f94866e4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94800cb5f0 0x7f94800691a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:04.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.249+0000 7f94866e4700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f948015a1e0 0x7f94800ca670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:04.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.249+0000 7f94866e4700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9480068d10 con 0x7f94800696e0 2023-10-06T10:43:04.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.249+0000 7f94866e4700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9480154020 con 0x7f948015a1e0 2023-10-06T10:43:04.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.249+0000 7f94866e4700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948015a670 con 0x7f94800cb5f0 2023-10-06T10:43:04.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.250+0000 7f947ffff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94800cb5f0 0x7f94800691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:04.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.250+0000 7f9484c81700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f948015a1e0 0x7f94800ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:04.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.250+0000 7f947ffff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94800cb5f0 0x7f94800691a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:44770/0 (socket says 172.21.15.92:44770) 2023-10-06T10:43:04.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.250+0000 7f947ffff700 1 -- 172.21.15.92:0/2578217620 learned_addr learned my addr 172.21.15.92:0/2578217620 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:04.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.250+0000 7f947ffff700 1 -- 172.21.15.92:0/2578217620 >> v2:172.21.15.124:3300/0 conn(0x7f948015a1e0 msgr2=0x7f94800ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:04.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.250+0000 7f947ffff700 1 --2- 172.21.15.92:0/2578217620 >> v2:172.21.15.124:3300/0 conn(0x7f948015a1e0 0x7f94800ca670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:04.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.250+0000 7f947ffff700 1 -- 172.21.15.92:0/2578217620 >> v1:172.21.15.124:6789/0 conn(0x7f94800696e0 legacy=0x7f9480159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:04.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.250+0000 7f947ffff700 1 -- 172.21.15.92:0/2578217620 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948015a950 con 0x7f94800cb5f0 2023-10-06T10:43:04.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.250+0000 7f947ffff700 1 --2- 172.21.15.92:0/2578217620 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94800cb5f0 0x7f94800691a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f94700099b0 tx=0x7f9470009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a98d7c13e966d4e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:04.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.251+0000 7f947effd700 1 -- 172.21.15.92:0/2578217620 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9470009ea0 con 0x7f94800cb5f0 2023-10-06T10:43:04.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.251+0000 7f947effd700 1 -- 172.21.15.92:0/2578217620 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947000b860 con 0x7f94800cb5f0 2023-10-06T10:43:04.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.251+0000 7f947effd700 1 -- 172.21.15.92:0/2578217620 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947000bc00 con 0x7f94800cb5f0 2023-10-06T10:43:04.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.251+0000 7f94866e4700 1 -- 172.21.15.92:0/2578217620 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94800cb5f0 msgr2=0x7f94800691a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:04.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.251+0000 7f94866e4700 1 --2- 172.21.15.92:0/2578217620 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94800cb5f0 0x7f94800691a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f94700099b0 tx=0x7f9470009250 comp rx=0 tx=0).stop 2023-10-06T10:43:04.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.251+0000 7f94866e4700 1 -- 172.21.15.92:0/2578217620 shutdown_connections 2023-10-06T10:43:04.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.251+0000 7f94866e4700 1 --2- 172.21.15.92:0/2578217620 >> v2:172.21.15.124:3300/0 conn(0x7f948015a1e0 0x7f94800ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:04.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.251+0000 7f94866e4700 1 --2- 172.21.15.92:0/2578217620 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94800cb5f0 0x7f94800691a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:04.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.251+0000 7f94866e4700 1 -- 172.21.15.92:0/2578217620 >> 172.21.15.92:0/2578217620 conn(0x7f94800707d0 msgr2=0x7f9480070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:04.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.251+0000 7f94866e4700 1 -- 172.21.15.92:0/2578217620 shutdown_connections 2023-10-06T10:43:04.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.251+0000 7f94866e4700 1 -- 172.21.15.92:0/2578217620 wait complete. 2023-10-06T10:43:04.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.252+0000 7f94866e4700 1 Processor -- start 2023-10-06T10:43:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.252+0000 7f94866e4700 1 -- start start 2023-10-06T10:43:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.253+0000 7f94866e4700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94800696e0 0x7f94800b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.253+0000 7f94866e4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94800cb5f0 0x7f948016a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.253+0000 7f94866e4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f948015a1e0 0x7f94800c4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.253+0000 7f94866e4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94800b3a40 con 0x7f948015a1e0 2023-10-06T10:43:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.253+0000 7f94866e4700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948016a7b0 con 0x7f94800696e0 2023-10-06T10:43:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.253+0000 7f94866e4700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94800c6ae0 con 0x7f94800cb5f0 2023-10-06T10:43:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.253+0000 7f9484c81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f948015a1e0 0x7f94800c4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.253+0000 7f9484c81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f948015a1e0 0x7f94800c4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33814/0 (socket says 172.21.15.92:33814) 2023-10-06T10:43:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.253+0000 7f9484c81700 1 -- 172.21.15.92:0/4016948133 learned_addr learned my addr 172.21.15.92:0/4016948133 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.253+0000 7f947ffff700 1 --2- 172.21.15.92:0/4016948133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94800696e0 0x7f94800b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.253+0000 7f947f7fe700 1 --2- 172.21.15.92:0/4016948133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94800cb5f0 0x7f948016a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.253+0000 7f9484c81700 1 -- 172.21.15.92:0/4016948133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94800cb5f0 msgr2=0x7f948016a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.253+0000 7f9484c81700 1 --2- 172.21.15.92:0/4016948133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94800cb5f0 0x7f948016a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.254+0000 7f9484c81700 1 -- 172.21.15.92:0/4016948133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94800696e0 msgr2=0x7f94800b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.254+0000 7f9484c81700 1 --2- 172.21.15.92:0/4016948133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94800696e0 0x7f94800b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.254+0000 7f9484c81700 1 -- 172.21.15.92:0/4016948133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94700094f0 con 0x7f948015a1e0 2023-10-06T10:43:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.254+0000 7f9484c81700 1 --2- 172.21.15.92:0/4016948133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f948015a1e0 0x7f94800c4220 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f9478009e90 tx=0x7f947800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.254+0000 7f9467fff700 1 -- 172.21.15.92:0/4016948133 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947800caa0 con 0x7f948015a1e0 2023-10-06T10:43:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.254+0000 7f9467fff700 1 -- 172.21.15.92:0/4016948133 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947800f040 con 0x7f948015a1e0 2023-10-06T10:43:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.254+0000 7f94866e4700 1 -- 172.21.15.92:0/4016948133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94800c6dc0 con 0x7f948015a1e0 2023-10-06T10:43:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.254+0000 7f9467fff700 1 -- 172.21.15.92:0/4016948133 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947800ce50 con 0x7f948015a1e0 2023-10-06T10:43:04.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.254+0000 7f94866e4700 1 -- 172.21.15.92:0/4016948133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94800c7380 con 0x7f948015a1e0 2023-10-06T10:43:04.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.255+0000 7f94866e4700 1 -- 172.21.15.92:0/4016948133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9480064a90 con 0x7f948015a1e0 2023-10-06T10:43:04.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.255+0000 7f9467fff700 1 -- 172.21.15.92:0/4016948133 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9478014690 con 0x7f948015a1e0 2023-10-06T10:43:04.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.256+0000 7f9467fff700 1 --2- 172.21.15.92:0/4016948133 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f946c066fb0 0x7f946c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:04.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.256+0000 7f9467fff700 1 -- 172.21.15.92:0/4016948133 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9478085500 con 0x7f948015a1e0 2023-10-06T10:43:04.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.256+0000 7f947ffff700 1 --2- 172.21.15.92:0/4016948133 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f946c066fb0 0x7f946c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:04.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.256+0000 7f947ffff700 1 --2- 172.21.15.92:0/4016948133 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f946c066fb0 0x7f946c069460 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f94700099b0 tx=0x7f9470017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:04.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.260+0000 7f9467fff700 1 -- 172.21.15.92:0/4016948133 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9478055ec0 con 0x7f948015a1e0 2023-10-06T10:43:04.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.447+0000 7f94866e4700 1 -- 172.21.15.92:0/4016948133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f948004bb40 con 0x7f948015a1e0 2023-10-06T10:43:04.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.448+0000 7f9467fff700 1 -- 172.21.15.92:0/4016948133 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+17074 (secure 0 0 0) 0x7f9478055a50 con 0x7f948015a1e0 2023-10-06T10:43:04.450 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:04.450 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":78,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:43:04.326791+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:43:00.816800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-06T10:43:02.128579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:43:04.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.450+0000 7f94866e4700 1 -- 172.21.15.92:0/4016948133 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f946c066fb0 msgr2=0x7f946c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:04.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.450+0000 7f94866e4700 1 --2- 172.21.15.92:0/4016948133 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f946c066fb0 0x7f946c069460 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f94700099b0 tx=0x7f9470017000 comp rx=0 tx=0).stop 2023-10-06T10:43:04.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.450+0000 7f94866e4700 1 -- 172.21.15.92:0/4016948133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f948015a1e0 msgr2=0x7f94800c4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:04.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.450+0000 7f94866e4700 1 --2- 172.21.15.92:0/4016948133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f948015a1e0 0x7f94800c4220 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f9478009e90 tx=0x7f947800b040 comp rx=0 tx=0).stop 2023-10-06T10:43:04.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.450+0000 7f94866e4700 1 -- 172.21.15.92:0/4016948133 shutdown_connections 2023-10-06T10:43:04.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.450+0000 7f94866e4700 1 --2- 172.21.15.92:0/4016948133 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f946c066fb0 0x7f946c069460 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:04.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.450+0000 7f94866e4700 1 --2- 172.21.15.92:0/4016948133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f948015a1e0 0x7f94800c4220 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:04.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.450+0000 7f94866e4700 1 --2- 172.21.15.92:0/4016948133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94800696e0 0x7f94800b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:04.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.450+0000 7f94866e4700 1 --2- 172.21.15.92:0/4016948133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94800cb5f0 0x7f948016a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:04.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.450+0000 7f94866e4700 1 -- 172.21.15.92:0/4016948133 >> 172.21.15.92:0/4016948133 conn(0x7f94800707d0 msgr2=0x7f94800cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:04.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.450+0000 7f94866e4700 1 -- 172.21.15.92:0/4016948133 shutdown_connections 2023-10-06T10:43:04.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.450+0000 7f94866e4700 1 -- 172.21.15.92:0/4016948133 wait complete. 2023-10-06T10:43:04.464 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T10:43:04.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.579+0000 7fb914b40700 1 Processor -- start 2023-10-06T10:43:04.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.579+0000 7fb914b40700 1 -- start start 2023-10-06T10:43:04.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.580+0000 7fb914b40700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb9100b2620 0x7fb9100cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:04.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.580+0000 7fb914b40700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb9100b29f0 0x7fb9100cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:04.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.580+0000 7fb914b40700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9100ceaa0 0x7fb9100cef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:04.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.580+0000 7fb914b40700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9100cc970 con 0x7fb9100b2620 2023-10-06T10:43:04.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.580+0000 7fb914b40700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9100cf480 con 0x7fb9100b29f0 2023-10-06T10:43:04.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.580+0000 7fb914b40700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9100cbd60 con 0x7fb9100ceaa0 2023-10-06T10:43:04.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.580+0000 7fb90f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb9100b2620 0x7fb9100cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:04.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.580+0000 7fb90f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb9100b2620 0x7fb9100cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33818/0 (socket says 172.21.15.92:33818) 2023-10-06T10:43:04.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.580+0000 7fb90f7fe700 1 -- 172.21.15.92:0/1519700457 learned_addr learned my addr 172.21.15.92:0/1519700457 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:04.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.580+0000 7fb90ffff700 1 --2- 172.21.15.92:0/1519700457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9100ceaa0 0x7fb9100cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:04.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.580+0000 7fb90effd700 1 --2- 172.21.15.92:0/1519700457 >> v2:172.21.15.124:3300/0 conn(0x7fb9100b29f0 0x7fb9100cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:04.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.580+0000 7fb90f7fe700 1 -- 172.21.15.92:0/1519700457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9100ceaa0 msgr2=0x7fb9100cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:04.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.580+0000 7fb90f7fe700 1 --2- 172.21.15.92:0/1519700457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9100ceaa0 0x7fb9100cef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:04.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.580+0000 7fb90f7fe700 1 -- 172.21.15.92:0/1519700457 >> v2:172.21.15.124:3300/0 conn(0x7fb9100b29f0 msgr2=0x7fb9100cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:04.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.580+0000 7fb90f7fe700 1 --2- 172.21.15.92:0/1519700457 >> v2:172.21.15.124:3300/0 conn(0x7fb9100b29f0 0x7fb9100cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:04.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.580+0000 7fb90f7fe700 1 -- 172.21.15.92:0/1519700457 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9100cc040 con 0x7fb9100b2620 2023-10-06T10:43:04.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.580+0000 7fb90effd700 1 --2- 172.21.15.92:0/1519700457 >> v2:172.21.15.124:3300/0 conn(0x7fb9100b29f0 0x7fb9100cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:04.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.581+0000 7fb90f7fe700 1 --2- 172.21.15.92:0/1519700457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9100b2620 0x7fb9100cccb0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fb9100b62d0 tx=0x7fb904009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc53bc9649b95bf7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:04.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.581+0000 7fb90e7fc700 1 -- 172.21.15.92:0/1519700457 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb904005c10 con 0x7fb9100b2620 2023-10-06T10:43:04.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.581+0000 7fb90e7fc700 1 -- 172.21.15.92:0/1519700457 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb904005d70 con 0x7fb9100b2620 2023-10-06T10:43:04.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.581+0000 7fb90e7fc700 1 -- 172.21.15.92:0/1519700457 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90400b880 con 0x7fb9100b2620 2023-10-06T10:43:04.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.581+0000 7fb914b40700 1 -- 172.21.15.92:0/1519700457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9100b2620 msgr2=0x7fb9100cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:04.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.581+0000 7fb914b40700 1 --2- 172.21.15.92:0/1519700457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9100b2620 0x7fb9100cccb0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fb9100b62d0 tx=0x7fb904009220 comp rx=0 tx=0).stop 2023-10-06T10:43:04.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.581+0000 7fb914b40700 1 -- 172.21.15.92:0/1519700457 shutdown_connections 2023-10-06T10:43:04.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.581+0000 7fb914b40700 1 --2- 172.21.15.92:0/1519700457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9100b2620 0x7fb9100cccb0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:04.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.581+0000 7fb914b40700 1 --2- 172.21.15.92:0/1519700457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9100ceaa0 0x7fb9100cef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:04.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.581+0000 7fb914b40700 1 --2- 172.21.15.92:0/1519700457 >> v2:172.21.15.124:3300/0 conn(0x7fb9100b29f0 0x7fb9100cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:04.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.581+0000 7fb914b40700 1 -- 172.21.15.92:0/1519700457 >> 172.21.15.92:0/1519700457 conn(0x7fb910099520 msgr2=0x7fb91006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:04.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.581+0000 7fb914b40700 1 -- 172.21.15.92:0/1519700457 shutdown_connections 2023-10-06T10:43:04.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.581+0000 7fb914b40700 1 -- 172.21.15.92:0/1519700457 wait complete. 2023-10-06T10:43:04.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.582+0000 7fb914b40700 1 Processor -- start 2023-10-06T10:43:04.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.583+0000 7fb914b40700 1 -- start start 2023-10-06T10:43:04.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.583+0000 7fb914b40700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9100b2620 0x7fb9100b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:04.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.583+0000 7fb914b40700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9100b29f0 0x7fb91009cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:04.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.583+0000 7fb914b40700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9100ceaa0 0x7fb91009d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:04.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.583+0000 7fb914b40700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9100c8af0 con 0x7fb9100b2620 2023-10-06T10:43:04.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.583+0000 7fb914b40700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9101477b0 con 0x7fb9100b29f0 2023-10-06T10:43:04.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.583+0000 7fb914b40700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb91009d9d0 con 0x7fb9100ceaa0 2023-10-06T10:43:04.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.583+0000 7fb90f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9100b2620 0x7fb9100b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:04.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.583+0000 7fb90f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9100b2620 0x7fb9100b4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33828/0 (socket says 172.21.15.92:33828) 2023-10-06T10:43:04.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.583+0000 7fb90f7fe700 1 -- 172.21.15.92:0/2019093321 learned_addr learned my addr 172.21.15.92:0/2019093321 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:04.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.584+0000 7fb90effd700 1 --2- 172.21.15.92:0/2019093321 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9100b29f0 0x7fb91009cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:04.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.584+0000 7fb90ffff700 1 --2- 172.21.15.92:0/2019093321 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9100ceaa0 0x7fb91009d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:04.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.584+0000 7fb90f7fe700 1 -- 172.21.15.92:0/2019093321 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9100ceaa0 msgr2=0x7fb91009d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:04.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.584+0000 7fb90f7fe700 1 --2- 172.21.15.92:0/2019093321 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9100ceaa0 0x7fb91009d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:04.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.584+0000 7fb90f7fe700 1 -- 172.21.15.92:0/2019093321 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9100b29f0 msgr2=0x7fb91009cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:04.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.584+0000 7fb90f7fe700 1 --2- 172.21.15.92:0/2019093321 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9100b29f0 0x7fb91009cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:04.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.584+0000 7fb90f7fe700 1 -- 172.21.15.92:0/2019093321 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb904009480 con 0x7fb9100b2620 2023-10-06T10:43:04.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.584+0000 7fb90effd700 1 --2- 172.21.15.92:0/2019093321 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9100b29f0 0x7fb91009cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:04.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.584+0000 7fb90ffff700 1 --2- 172.21.15.92:0/2019093321 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9100ceaa0 0x7fb91009d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:04.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.584+0000 7fb90f7fe700 1 --2- 172.21.15.92:0/2019093321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9100b2620 0x7fb9100b4da0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fb90400b3c0 tx=0x7fb904016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:04.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.584+0000 7fb8f77fe700 1 -- 172.21.15.92:0/2019093321 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb904011920 con 0x7fb9100b2620 2023-10-06T10:43:04.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.584+0000 7fb8f77fe700 1 -- 172.21.15.92:0/2019093321 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb904020070 con 0x7fb9100b2620 2023-10-06T10:43:04.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.584+0000 7fb914b40700 1 -- 172.21.15.92:0/2019093321 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb91009dcb0 con 0x7fb9100b2620 2023-10-06T10:43:04.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.584+0000 7fb8f77fe700 1 -- 172.21.15.92:0/2019093321 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb904011cb0 con 0x7fb9100b2620 2023-10-06T10:43:04.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.584+0000 7fb914b40700 1 -- 172.21.15.92:0/2019093321 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9100c9060 con 0x7fb9100b2620 2023-10-06T10:43:04.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.585+0000 7fb914b40700 1 -- 172.21.15.92:0/2019093321 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb91009dea0 con 0x7fb9100b2620 2023-10-06T10:43:04.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.585+0000 7fb8f77fe700 1 -- 172.21.15.92:0/2019093321 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb904022030 con 0x7fb9100b2620 2023-10-06T10:43:04.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.586+0000 7fb8f77fe700 1 --2- 172.21.15.92:0/2019093321 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb8fc066f30 0x7fb8fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:04.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.586+0000 7fb8f77fe700 1 -- 172.21.15.92:0/2019093321 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb904086490 con 0x7fb9100b2620 2023-10-06T10:43:04.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.586+0000 7fb90effd700 1 --2- 172.21.15.92:0/2019093321 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb8fc066f30 0x7fb8fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:04.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.586+0000 7fb90effd700 1 --2- 172.21.15.92:0/2019093321 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb8fc066f30 0x7fb8fc0693e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb9100b5b90 tx=0x7fb900008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:04.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.590+0000 7fb8f77fe700 1 -- 172.21.15.92:0/2019093321 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb91009dea0 con 0x7fb9100b2620 2023-10-06T10:43:04.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:04.818+0000 7fb914b40700 1 -- 172.21.15.92:0/2019093321 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb91009dea0 con 0x7fb9100b2620 2023-10-06T10:43:05.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.807+0000 7fb8f77fe700 1 -- 172.21.15.92:0/2019093321 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v66) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb91009dea0 con 0x7fb9100b2620 2023-10-06T10:43:05.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.809+0000 7fb914b40700 1 -- 172.21.15.92:0/2019093321 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb8fc066f30 msgr2=0x7fb8fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:05.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.809+0000 7fb914b40700 1 --2- 172.21.15.92:0/2019093321 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb8fc066f30 0x7fb8fc0693e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb9100b5b90 tx=0x7fb900008040 comp rx=0 tx=0).stop 2023-10-06T10:43:05.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.809+0000 7fb914b40700 1 -- 172.21.15.92:0/2019093321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9100b2620 msgr2=0x7fb9100b4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:05.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.809+0000 7fb914b40700 1 --2- 172.21.15.92:0/2019093321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9100b2620 0x7fb9100b4da0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fb90400b3c0 tx=0x7fb904016000 comp rx=0 tx=0).stop 2023-10-06T10:43:05.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.809+0000 7fb914b40700 1 -- 172.21.15.92:0/2019093321 shutdown_connections 2023-10-06T10:43:05.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.809+0000 7fb914b40700 1 --2- 172.21.15.92:0/2019093321 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb8fc066f30 0x7fb8fc0693e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:05.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.809+0000 7fb914b40700 1 --2- 172.21.15.92:0/2019093321 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9100ceaa0 0x7fb91009d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:05.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.809+0000 7fb914b40700 1 --2- 172.21.15.92:0/2019093321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9100b2620 0x7fb9100b4da0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:05.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.809+0000 7fb914b40700 1 --2- 172.21.15.92:0/2019093321 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9100b29f0 0x7fb91009cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:05.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.809+0000 7fb914b40700 1 -- 172.21.15.92:0/2019093321 >> 172.21.15.92:0/2019093321 conn(0x7fb910099520 msgr2=0x7fb9100cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:05.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.810+0000 7fb914b40700 1 -- 172.21.15.92:0/2019093321 shutdown_connections 2023-10-06T10:43:05.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.810+0000 7fb914b40700 1 -- 172.21.15.92:0/2019093321 wait complete. 2023-10-06T10:43:05.826 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T10:43:05.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.945+0000 7f9e9d598700 1 Processor -- start 2023-10-06T10:43:05.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.945+0000 7f9e9d598700 1 -- start start 2023-10-06T10:43:05.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.946+0000 7f9e9d598700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9e9815da80 0x7f9e980c8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:05.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.946+0000 7f9e9d598700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e980c8860 con 0x7f9e981593b0 2023-10-06T10:43:05.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.946+0000 7f9e9d598700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e980c5f40 con 0x7f9e980685c0 2023-10-06T10:43:05.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.946+0000 7f9e9d598700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e980c9050 con 0x7f9e9815da80 2023-10-06T10:43:05.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.946+0000 7f9e96ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9e981593b0 0x7f9e980c63d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51860/0 (socket says 172.21.15.92:51860) 2023-10-06T10:43:05.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.946+0000 7f9e96ffd700 1 -- 172.21.15.92:0/2434352791 learned_addr learned my addr 172.21.15.92:0/2434352791 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:05.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.946+0000 7f9e8e7fc700 1 --2- 172.21.15.92:0/2434352791 >> v2:172.21.15.124:3300/0 conn(0x7f9e9815da80 0x7f9e980c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:05.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.946+0000 7f9e967fc700 1 -- 172.21.15.92:0/2434352791 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1555205337 0 0) 0x7f9e980c8860 con 0x7f9e981593b0 2023-10-06T10:43:05.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.946+0000 7f9e967fc700 1 -- 172.21.15.92:0/2434352791 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e84003570 con 0x7f9e981593b0 2023-10-06T10:43:05.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e967fc700 1 -- 172.21.15.92:0/2434352791 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 193173002 0 0) 0x7f9e84003570 con 0x7f9e981593b0 2023-10-06T10:43:05.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e967fc700 1 -- 172.21.15.92:0/2434352791 >> v2:172.21.15.124:3300/0 conn(0x7f9e9815da80 msgr2=0x7f9e980c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:05.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e967fc700 1 --2- 172.21.15.92:0/2434352791 >> v2:172.21.15.124:3300/0 conn(0x7f9e9815da80 0x7f9e980c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:05.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e967fc700 1 -- 172.21.15.92:0/2434352791 >> v1:172.21.15.124:6789/0 conn(0x7f9e980685c0 legacy=0x7f9e9806add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:05.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e967fc700 1 -- 172.21.15.92:0/2434352791 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e980c8a90 con 0x7f9e981593b0 2023-10-06T10:43:05.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e967fc700 1 -- 172.21.15.92:0/2434352791 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f9e88001470 con 0x7f9e981593b0 2023-10-06T10:43:05.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e8e7fc700 1 --2- 172.21.15.92:0/2434352791 >> v2:172.21.15.124:3300/0 conn(0x7f9e9815da80 0x7f9e980c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:43:05.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e967fc700 1 -- 172.21.15.92:0/2434352791 >> v1:172.21.15.92:6789/0 conn(0x7f9e981593b0 legacy=0x7f9e980c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:05.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e967fc700 1 --2- 172.21.15.92:0/2434352791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e84003a60 0x7f9e84003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:05.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e967fc700 1 --2- 172.21.15.92:0/2434352791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e84004810 0x7f9e84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:05.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e967fc700 1 --2- 172.21.15.92:0/2434352791 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e840051c0 0x7f9e8400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:05.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e967fc700 1 -- 172.21.15.92:0/2434352791 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e980c9050 con 0x7f9e84004810 2023-10-06T10:43:05.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e967fc700 1 -- 172.21.15.92:0/2434352791 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8400c0d0 con 0x7f9e840051c0 2023-10-06T10:43:05.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e967fc700 1 -- 172.21.15.92:0/2434352791 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8400c290 con 0x7f9e84003a60 2023-10-06T10:43:05.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e8e7fc700 1 --2- 172.21.15.92:0/2434352791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e84004810 0x7f9e84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:05.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e96ffd700 1 --2- 172.21.15.92:0/2434352791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e84003a60 0x7f9e84003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:05.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e977fe700 1 --2- 172.21.15.92:0/2434352791 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e840051c0 0x7f9e8400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:05.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e8e7fc700 1 -- 172.21.15.92:0/2434352791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e84003a60 msgr2=0x7f9e84003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:05.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e8e7fc700 1 --2- 172.21.15.92:0/2434352791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e84003a60 0x7f9e84003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:05.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e8e7fc700 1 -- 172.21.15.92:0/2434352791 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e840051c0 msgr2=0x7f9e8400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:05.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e8e7fc700 1 --2- 172.21.15.92:0/2434352791 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e840051c0 0x7f9e8400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:05.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e977fe700 1 -- 172.21.15.92:0/2434352791 reap_dead start 2023-10-06T10:43:05.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e8e7fc700 1 -- 172.21.15.92:0/2434352791 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e84003570 con 0x7f9e84004810 2023-10-06T10:43:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e96ffd700 1 --2- 172.21.15.92:0/2434352791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e84003a60 0x7f9e84003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e8e7fc700 1 --2- 172.21.15.92:0/2434352791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e84004810 0x7f9e84004c80 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009df0 tx=0x7f9e800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c14dce6ab1322c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e967fc700 1 -- 172.21.15.92:0/2434352791 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e80009740 con 0x7f9e84004810 2023-10-06T10:43:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e967fc700 1 -- 172.21.15.92:0/2434352791 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8000e920 con 0x7f9e84004810 2023-10-06T10:43:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.947+0000 7f9e967fc700 1 -- 172.21.15.92:0/2434352791 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8000ece0 con 0x7f9e84004810 2023-10-06T10:43:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.948+0000 7f9e9d598700 1 -- 172.21.15.92:0/2434352791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e84004810 msgr2=0x7f9e84004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.948+0000 7f9e9d598700 1 --2- 172.21.15.92:0/2434352791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e84004810 0x7f9e84004c80 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009df0 tx=0x7f9e800091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.948+0000 7f9e9d598700 1 -- 172.21.15.92:0/2434352791 shutdown_connections 2023-10-06T10:43:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.948+0000 7f9e9d598700 1 --2- 172.21.15.92:0/2434352791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e84004810 0x7f9e84004c80 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.948+0000 7f9e9d598700 1 -- 172.21.15.92:0/2434352791 >> 172.21.15.92:0/2434352791 conn(0x7f9e980c9c90 msgr2=0x7f9e9806da00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.948+0000 7f9e9d598700 1 -- 172.21.15.92:0/2434352791 shutdown_connections 2023-10-06T10:43:05.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.948+0000 7f9e9d598700 1 -- 172.21.15.92:0/2434352791 wait complete. 2023-10-06T10:43:05.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.949+0000 7f9e9d598700 1 Processor -- start 2023-10-06T10:43:05.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.949+0000 7f9e9d598700 1 -- start start 2023-10-06T10:43:05.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.950+0000 7f9e9d598700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e84004810 0x7f9e98150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:05.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.950+0000 7f9e9d598700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e9814cae0 0x7f9e980b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:05.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.950+0000 7f9e9d598700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e9814cf50 0x7f9e980b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:05.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.950+0000 7f9e9d598700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e98150f90 con 0x7f9e84004810 2023-10-06T10:43:05.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.950+0000 7f9e9d598700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e98169c50 con 0x7f9e9814cae0 2023-10-06T10:43:05.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.950+0000 7f9e9d598700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e98169f30 con 0x7f9e9814cf50 2023-10-06T10:43:05.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.950+0000 7f9e96ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e84004810 0x7f9e98150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:05.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.950+0000 7f9e96ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e84004810 0x7f9e98150a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33846/0 (socket says 172.21.15.92:33846) 2023-10-06T10:43:05.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.950+0000 7f9e96ffd700 1 -- 172.21.15.92:0/1386959943 learned_addr learned my addr 172.21.15.92:0/1386959943 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:05.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.950+0000 7f9e8e7fc700 1 --2- 172.21.15.92:0/1386959943 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e9814cae0 0x7f9e980b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:05.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.950+0000 7f9e977fe700 1 --2- 172.21.15.92:0/1386959943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e9814cf50 0x7f9e980b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:05.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.950+0000 7f9e96ffd700 1 -- 172.21.15.92:0/1386959943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e9814cf50 msgr2=0x7f9e980b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:05.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.950+0000 7f9e96ffd700 1 --2- 172.21.15.92:0/1386959943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e9814cf50 0x7f9e980b0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:05.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.950+0000 7f9e96ffd700 1 -- 172.21.15.92:0/1386959943 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e9814cae0 msgr2=0x7f9e980b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:05.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.950+0000 7f9e96ffd700 1 --2- 172.21.15.92:0/1386959943 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e9814cae0 0x7f9e980b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:05.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.950+0000 7f9e96ffd700 1 -- 172.21.15.92:0/1386959943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9816a210 con 0x7f9e84004810 2023-10-06T10:43:05.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.950+0000 7f9e96ffd700 1 --2- 172.21.15.92:0/1386959943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e84004810 0x7f9e98150a50 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f9e88000fd0 tx=0x7f9e88001e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:05.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.951+0000 7f9e8f7fe700 1 -- 172.21.15.92:0/1386959943 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8800c7e0 con 0x7f9e84004810 2023-10-06T10:43:05.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.951+0000 7f9e8f7fe700 1 -- 172.21.15.92:0/1386959943 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8800c940 con 0x7f9e84004810 2023-10-06T10:43:05.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.951+0000 7f9e9d598700 1 -- 172.21.15.92:0/1386959943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e9816a4f0 con 0x7f9e84004810 2023-10-06T10:43:05.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.951+0000 7f9e8f7fe700 1 -- 172.21.15.92:0/1386959943 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8800cd00 con 0x7f9e84004810 2023-10-06T10:43:05.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.951+0000 7f9e9d598700 1 -- 172.21.15.92:0/1386959943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e98169950 con 0x7f9e84004810 2023-10-06T10:43:05.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.951+0000 7f9e9d598700 1 -- 172.21.15.92:0/1386959943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e9805f230 con 0x7f9e84004810 2023-10-06T10:43:05.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.952+0000 7f9e8f7fe700 1 -- 172.21.15.92:0/1386959943 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9e88013630 con 0x7f9e84004810 2023-10-06T10:43:05.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.952+0000 7f9e8f7fe700 1 --2- 172.21.15.92:0/1386959943 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9e78066f60 0x7f9e78069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:05.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.952+0000 7f9e8f7fe700 1 -- 172.21.15.92:0/1386959943 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9e88084860 con 0x7f9e84004810 2023-10-06T10:43:05.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.952+0000 7f9e8e7fc700 1 --2- 172.21.15.92:0/1386959943 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9e78066f60 0x7f9e78069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:05.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.953+0000 7f9e8e7fc700 1 --2- 172.21.15.92:0/1386959943 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9e78066f60 0x7f9e78069410 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9e800091e0 tx=0x7f9e8001a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:05.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:05.956+0000 7f9e8f7fe700 1 -- 172.21.15.92:0/1386959943 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e88051830 con 0x7f9e84004810 2023-10-06T10:43:06.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.184+0000 7f9e9d598700 1 -- 172.21.15.92:0/1386959943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9e980ca100 con 0x7f9e84004810 2023-10-06T10:43:06.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.814+0000 7f9e8f7fe700 1 -- 172.21.15.92:0/1386959943 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v67) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9e8800ce60 con 0x7f9e84004810 2023-10-06T10:43:06.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.817+0000 7f9e9d598700 1 -- 172.21.15.92:0/1386959943 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9e78066f60 msgr2=0x7f9e78069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:06.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.817+0000 7f9e9d598700 1 --2- 172.21.15.92:0/1386959943 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9e78066f60 0x7f9e78069410 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9e800091e0 tx=0x7f9e8001a000 comp rx=0 tx=0).stop 2023-10-06T10:43:06.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.817+0000 7f9e9d598700 1 -- 172.21.15.92:0/1386959943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e84004810 msgr2=0x7f9e98150a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:06.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.817+0000 7f9e9d598700 1 --2- 172.21.15.92:0/1386959943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e84004810 0x7f9e98150a50 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f9e88000fd0 tx=0x7f9e88001e20 comp rx=0 tx=0).stop 2023-10-06T10:43:06.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.817+0000 7f9e9d598700 1 -- 172.21.15.92:0/1386959943 shutdown_connections 2023-10-06T10:43:06.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.817+0000 7f9e9d598700 1 --2- 172.21.15.92:0/1386959943 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9e78066f60 0x7f9e78069410 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:06.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.817+0000 7f9e9d598700 1 --2- 172.21.15.92:0/1386959943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e9814cf50 0x7f9e980b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:06.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.817+0000 7f9e9d598700 1 --2- 172.21.15.92:0/1386959943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e84004810 0x7f9e98150a50 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:06.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.817+0000 7f9e9d598700 1 --2- 172.21.15.92:0/1386959943 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e9814cae0 0x7f9e980b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:06.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.817+0000 7f9e9d598700 1 -- 172.21.15.92:0/1386959943 >> 172.21.15.92:0/1386959943 conn(0x7f9e980c9c90 msgr2=0x7f9e98157520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:06.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.817+0000 7f9e9d598700 1 -- 172.21.15.92:0/1386959943 shutdown_connections 2023-10-06T10:43:06.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.817+0000 7f9e9d598700 1 -- 172.21.15.92:0/1386959943 wait complete. 2023-10-06T10:43:06.832 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:43:06.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.946+0000 7fe11ad51700 1 Processor -- start 2023-10-06T10:43:06.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.946+0000 7fe11ad51700 1 -- start start 2023-10-06T10:43:06.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.947+0000 7fe11ad51700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe114159560 0x7fe1140aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:06.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.947+0000 7fe11ad51700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe1140b49a0 0x7fe1140aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:06.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.947+0000 7fe11ad51700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1140b0c60 con 0x7fe1140b0760 2023-10-06T10:43:06.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.947+0000 7fe11ad51700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1140b0f40 con 0x7fe1140b49a0 2023-10-06T10:43:06.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.947+0000 7fe11ad51700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1140b1220 con 0x7fe114159560 2023-10-06T10:43:06.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.947+0000 7fe10bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe1140b49a0 0x7fe1140aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:06.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.947+0000 7fe10bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe1140b49a0 0x7fe1140aaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33854/0 (socket says 172.21.15.92:33854) 2023-10-06T10:43:06.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.947+0000 7fe10bfff700 1 -- 172.21.15.92:0/92418174 learned_addr learned my addr 172.21.15.92:0/92418174 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:06.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.948+0000 7fe118aed700 1 --2- 172.21.15.92:0/92418174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe114159560 0x7fe1140aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:06.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.948+0000 7fe10bfff700 1 -- 172.21.15.92:0/92418174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe114159560 msgr2=0x7fe1140aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:06.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.948+0000 7fe10bfff700 1 --2- 172.21.15.92:0/92418174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe114159560 0x7fe1140aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:06.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.948+0000 7fe10bfff700 1 -- 172.21.15.92:0/92418174 >> v1:172.21.15.124:6789/0 conn(0x7fe1140b0760 legacy=0x7fe1140b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:06.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.948+0000 7fe10bfff700 1 -- 172.21.15.92:0/92418174 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1140b1500 con 0x7fe1140b49a0 2023-10-06T10:43:06.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.948+0000 7fe118aed700 1 --2- 172.21.15.92:0/92418174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe114159560 0x7fe1140aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:06.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.948+0000 7fe10bfff700 1 --2- 172.21.15.92:0/92418174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1140b49a0 0x7fe1140aaa70 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe1040099b0 tx=0x7fe104009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9432ce78d659fd89 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:06.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.948+0000 7fe10b7fe700 1 -- 172.21.15.92:0/92418174 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10400b820 con 0x7fe1140b49a0 2023-10-06T10:43:06.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.948+0000 7fe10b7fe700 1 -- 172.21.15.92:0/92418174 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10400b980 con 0x7fe1140b49a0 2023-10-06T10:43:06.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.948+0000 7fe10b7fe700 1 -- 172.21.15.92:0/92418174 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10400bd20 con 0x7fe1140b49a0 2023-10-06T10:43:06.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.948+0000 7fe11ad51700 1 -- 172.21.15.92:0/92418174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1140b49a0 msgr2=0x7fe1140aaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:06.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.948+0000 7fe11ad51700 1 --2- 172.21.15.92:0/92418174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1140b49a0 0x7fe1140aaa70 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe1040099b0 tx=0x7fe104009250 comp rx=0 tx=0).stop 2023-10-06T10:43:06.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.948+0000 7fe11ad51700 1 -- 172.21.15.92:0/92418174 shutdown_connections 2023-10-06T10:43:06.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.948+0000 7fe11ad51700 1 --2- 172.21.15.92:0/92418174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe114159560 0x7fe1140aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:06.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.948+0000 7fe11ad51700 1 --2- 172.21.15.92:0/92418174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1140b49a0 0x7fe1140aaa70 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:06.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.948+0000 7fe11ad51700 1 -- 172.21.15.92:0/92418174 >> 172.21.15.92:0/92418174 conn(0x7fe1140997f0 msgr2=0x7fe11406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:06.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.948+0000 7fe11ad51700 1 -- 172.21.15.92:0/92418174 shutdown_connections 2023-10-06T10:43:06.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.948+0000 7fe11ad51700 1 -- 172.21.15.92:0/92418174 wait complete. 2023-10-06T10:43:06.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.949+0000 7fe11ad51700 1 Processor -- start 2023-10-06T10:43:06.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.950+0000 7fe11ad51700 1 -- start start 2023-10-06T10:43:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.950+0000 7fe11ad51700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe1140b0760 0x7fe1140aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.950+0000 7fe11ad51700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1140b49a0 0x7fe1140c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.950+0000 7fe11ad51700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe114159560 0x7fe114169860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.950+0000 7fe11ad51700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1140c7f70 con 0x7fe114159560 2023-10-06T10:43:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.950+0000 7fe11ad51700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1140c8250 con 0x7fe1140b0760 2023-10-06T10:43:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.950+0000 7fe11ad51700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1140af390 con 0x7fe1140b49a0 2023-10-06T10:43:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.950+0000 7fe1192ee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe114159560 0x7fe114169860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.951+0000 7fe1192ee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe114159560 0x7fe114169860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33868/0 (socket says 172.21.15.92:33868) 2023-10-06T10:43:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.951+0000 7fe1192ee700 1 -- 172.21.15.92:0/1258196554 learned_addr learned my addr 172.21.15.92:0/1258196554 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.951+0000 7fe10bfff700 1 --2- 172.21.15.92:0/1258196554 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1140b49a0 0x7fe1140c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.951+0000 7fe118aed700 1 --2- 172.21.15.92:0/1258196554 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe1140b0760 0x7fe1140aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:06.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.951+0000 7fe1192ee700 1 -- 172.21.15.92:0/1258196554 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1140b49a0 msgr2=0x7fe1140c5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:06.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.951+0000 7fe1192ee700 1 --2- 172.21.15.92:0/1258196554 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1140b49a0 0x7fe1140c5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:06.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.951+0000 7fe1192ee700 1 -- 172.21.15.92:0/1258196554 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe1140b0760 msgr2=0x7fe1140aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:06.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.951+0000 7fe1192ee700 1 --2- 172.21.15.92:0/1258196554 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe1140b0760 0x7fe1140aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:06.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.951+0000 7fe1192ee700 1 -- 172.21.15.92:0/1258196554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1040094f0 con 0x7fe114159560 2023-10-06T10:43:06.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.951+0000 7fe1192ee700 1 --2- 172.21.15.92:0/1258196554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe114159560 0x7fe114169860 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00c960 tx=0x7fe10c0026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:06.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.951+0000 7fe108ff9700 1 -- 172.21.15.92:0/1258196554 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c012630 con 0x7fe114159560 2023-10-06T10:43:06.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.951+0000 7fe108ff9700 1 -- 172.21.15.92:0/1258196554 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10c012790 con 0x7fe114159560 2023-10-06T10:43:06.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.951+0000 7fe108ff9700 1 -- 172.21.15.92:0/1258196554 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c012b50 con 0x7fe114159560 2023-10-06T10:43:06.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.951+0000 7fe11ad51700 1 -- 172.21.15.92:0/1258196554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11416a0b0 con 0x7fe114159560 2023-10-06T10:43:06.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.951+0000 7fe11ad51700 1 -- 172.21.15.92:0/1258196554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11416a670 con 0x7fe114159560 2023-10-06T10:43:06.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.952+0000 7fe11ad51700 1 -- 172.21.15.92:0/1258196554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1140c83c0 con 0x7fe114159560 2023-10-06T10:43:06.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.952+0000 7fe108ff9700 1 -- 172.21.15.92:0/1258196554 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe10c01b400 con 0x7fe114159560 2023-10-06T10:43:06.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.953+0000 7fe108ff9700 1 --2- 172.21.15.92:0/1258196554 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe100066f30 0x7fe1000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:06.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.953+0000 7fe108ff9700 1 -- 172.21.15.92:0/1258196554 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe10c084a40 con 0x7fe114159560 2023-10-06T10:43:06.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.953+0000 7fe118aed700 1 --2- 172.21.15.92:0/1258196554 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe100066f30 0x7fe1000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:06.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.953+0000 7fe118aed700 1 --2- 172.21.15.92:0/1258196554 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe100066f30 0x7fe1000693e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe1140c9660 tx=0x7fe110008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:06.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:06.957+0000 7fe108ff9700 1 -- 172.21.15.92:0/1258196554 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe10c086e60 con 0x7fe114159560 2023-10-06T10:43:07.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.185+0000 7fe11ad51700 1 -- 172.21.15.92:0/1258196554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe114064a90 con 0x7fe114159560 2023-10-06T10:43:07.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.185+0000 7fe108ff9700 1 -- 172.21.15.92:0/1258196554 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fe10c0554a0 con 0x7fe114159560 2023-10-06T10:43:07.187 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:07.188 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":67,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3469511807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3469511807},{"type":"v1","addr":"172.21.15.124:6834","nonce":3469511807}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7146,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/4003376156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4003376156},{"type":"v1","addr":"172.21.15.92:6837","nonce":4003376156}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7149,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3268908749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3268908749},{"type":"v1","addr":"172.21.15.92:6836","nonce":3268908749}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:03.173076+0000","modified":"2023-10-06T10:43:06.807414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7164},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7164":{"gid":7164,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/2971348516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2971348516},{"type":"v1","addr":"172.21.15.124:6835","nonce":2971348516}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T10:43:07.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.187+0000 7fe11ad51700 1 -- 172.21.15.92:0/1258196554 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe100066f30 msgr2=0x7fe1000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:07.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.187+0000 7fe11ad51700 1 --2- 172.21.15.92:0/1258196554 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe100066f30 0x7fe1000693e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe1140c9660 tx=0x7fe110008040 comp rx=0 tx=0).stop 2023-10-06T10:43:07.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.187+0000 7fe11ad51700 1 -- 172.21.15.92:0/1258196554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe114159560 msgr2=0x7fe114169860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:07.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.187+0000 7fe11ad51700 1 --2- 172.21.15.92:0/1258196554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe114159560 0x7fe114169860 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00c960 tx=0x7fe10c0026e0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.187+0000 7fe11ad51700 1 -- 172.21.15.92:0/1258196554 shutdown_connections 2023-10-06T10:43:07.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.187+0000 7fe11ad51700 1 --2- 172.21.15.92:0/1258196554 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe100066f30 0x7fe1000693e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.187+0000 7fe11ad51700 1 --2- 172.21.15.92:0/1258196554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe114159560 0x7fe114169860 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.187+0000 7fe11ad51700 1 --2- 172.21.15.92:0/1258196554 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe1140b0760 0x7fe1140aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.187+0000 7fe11ad51700 1 --2- 172.21.15.92:0/1258196554 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1140b49a0 0x7fe1140c5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.187+0000 7fe11ad51700 1 -- 172.21.15.92:0/1258196554 >> 172.21.15.92:0/1258196554 conn(0x7fe1140997f0 msgr2=0x7fe1140ccf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:07.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.188+0000 7fe11ad51700 1 -- 172.21.15.92:0/1258196554 shutdown_connections 2023-10-06T10:43:07.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.188+0000 7fe11ad51700 1 -- 172.21.15.92:0/1258196554 wait complete. 2023-10-06T10:43:07.191 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 67 2023-10-06T10:43:07.201 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:43:07.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.313+0000 7f953359e700 1 Processor -- start 2023-10-06T10:43:07.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.313+0000 7f953359e700 1 -- start start 2023-10-06T10:43:07.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.314+0000 7f953359e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9534159440 0x7f95340b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:07.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.314+0000 7f953359e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95340cb7a0 con 0x7f95340b2100 2023-10-06T10:43:07.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.314+0000 7f953359e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95340b0860 con 0x7f95340b35d0 2023-10-06T10:43:07.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.314+0000 7f953359e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95340cb150 con 0x7f9534159440 2023-10-06T10:43:07.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.314+0000 7f953259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9534159440 0x7f95340b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:07.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.314+0000 7f9532d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f95340b2100 0x7f95340b2520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51872/0 (socket says 172.21.15.92:51872) 2023-10-06T10:43:07.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.314+0000 7f953259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9534159440 0x7f95340b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33884/0 (socket says 172.21.15.92:33884) 2023-10-06T10:43:07.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.314+0000 7f9532d9d700 1 -- 172.21.15.92:0/2829449739 learned_addr learned my addr 172.21.15.92:0/2829449739 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:07.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.314+0000 7f953259c700 1 -- 172.21.15.92:0/2829449739 >> v1:172.21.15.124:6789/0 conn(0x7f95340b35d0 legacy=0x7f95340cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:07.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.314+0000 7f953259c700 1 -- 172.21.15.92:0/2829449739 >> v1:172.21.15.92:6789/0 conn(0x7f95340b2100 legacy=0x7f95340b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:07.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.314+0000 7f953259c700 1 -- 172.21.15.92:0/2829449739 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95340cb430 con 0x7f9534159440 2023-10-06T10:43:07.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.315+0000 7f953259c700 1 --2- 172.21.15.92:0/2829449739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9534159440 0x7f95340b3090 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f952c0098e0 tx=0x7f952c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2625b85cd43f4aaa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:07.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.315+0000 7f953159a700 1 -- 172.21.15.92:0/2829449739 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c009e80 con 0x7f9534159440 2023-10-06T10:43:07.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.315+0000 7f953159a700 1 -- 172.21.15.92:0/2829449739 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952c00b860 con 0x7f9534159440 2023-10-06T10:43:07.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.315+0000 7f953159a700 1 -- 172.21.15.92:0/2829449739 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c00bbe0 con 0x7f9534159440 2023-10-06T10:43:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.315+0000 7f953359e700 1 -- 172.21.15.92:0/2829449739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9534159440 msgr2=0x7f95340b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.315+0000 7f953359e700 1 --2- 172.21.15.92:0/2829449739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9534159440 0x7f95340b3090 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f952c0098e0 tx=0x7f952c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.315+0000 7f953359e700 1 -- 172.21.15.92:0/2829449739 shutdown_connections 2023-10-06T10:43:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.315+0000 7f953359e700 1 --2- 172.21.15.92:0/2829449739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9534159440 0x7f95340b3090 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.315+0000 7f953359e700 1 -- 172.21.15.92:0/2829449739 >> 172.21.15.92:0/2829449739 conn(0x7f953406ece0 msgr2=0x7f953406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.315+0000 7f953359e700 1 -- 172.21.15.92:0/2829449739 shutdown_connections 2023-10-06T10:43:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.316+0000 7f953359e700 1 -- 172.21.15.92:0/2829449739 wait complete. 2023-10-06T10:43:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.317+0000 7f953359e700 1 Processor -- start 2023-10-06T10:43:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.317+0000 7f953359e700 1 -- start start 2023-10-06T10:43:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.317+0000 7f953359e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95340b2100 0x7f9534148540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.317+0000 7f953359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95340b35d0 0x7f95340cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.317+0000 7f953359e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9534159440 0x7f95340ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.317+0000 7f953359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95340c5b30 con 0x7f95340b35d0 2023-10-06T10:43:07.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.317+0000 7f953359e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95340c4a90 con 0x7f95340b2100 2023-10-06T10:43:07.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.317+0000 7f953359e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95340c4d70 con 0x7f9534159440 2023-10-06T10:43:07.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.317+0000 7f9531d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95340b35d0 0x7f95340cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:07.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.317+0000 7f9531d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95340b35d0 0x7f95340cc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33892/0 (socket says 172.21.15.92:33892) 2023-10-06T10:43:07.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.317+0000 7f9531d9b700 1 -- 172.21.15.92:0/725136167 learned_addr learned my addr 172.21.15.92:0/725136167 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:07.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.318+0000 7f953259c700 1 --2- 172.21.15.92:0/725136167 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95340b2100 0x7f9534148540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:07.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.318+0000 7f9532d9d700 1 --2- 172.21.15.92:0/725136167 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9534159440 0x7f95340ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:07.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.318+0000 7f9531d9b700 1 -- 172.21.15.92:0/725136167 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9534159440 msgr2=0x7f95340ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:07.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.318+0000 7f9531d9b700 1 --2- 172.21.15.92:0/725136167 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9534159440 0x7f95340ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.318+0000 7f9531d9b700 1 -- 172.21.15.92:0/725136167 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95340b2100 msgr2=0x7f9534148540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:07.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.318+0000 7f9531d9b700 1 --2- 172.21.15.92:0/725136167 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95340b2100 0x7f9534148540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.318+0000 7f9531d9b700 1 -- 172.21.15.92:0/725136167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c009420 con 0x7f95340b35d0 2023-10-06T10:43:07.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.318+0000 7f9531d9b700 1 --2- 172.21.15.92:0/725136167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95340b35d0 0x7f95340cc6f0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f952800c790 tx=0x7f9528007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:07.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.318+0000 7f95227fc700 1 -- 172.21.15.92:0/725136167 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9528012630 con 0x7f95340b35d0 2023-10-06T10:43:07.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.318+0000 7f95227fc700 1 -- 172.21.15.92:0/725136167 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95280127b0 con 0x7f95340b35d0 2023-10-06T10:43:07.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.318+0000 7f953359e700 1 -- 172.21.15.92:0/725136167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95340c5050 con 0x7f95340b35d0 2023-10-06T10:43:07.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.318+0000 7f95227fc700 1 -- 172.21.15.92:0/725136167 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9528012b70 con 0x7f95340b35d0 2023-10-06T10:43:07.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.318+0000 7f953359e700 1 -- 172.21.15.92:0/725136167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95340c5610 con 0x7f95340b35d0 2023-10-06T10:43:07.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.319+0000 7f953359e700 1 -- 172.21.15.92:0/725136167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95340c5830 con 0x7f95340b35d0 2023-10-06T10:43:07.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.319+0000 7f95227fc700 1 -- 172.21.15.92:0/725136167 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f952801b400 con 0x7f95340b35d0 2023-10-06T10:43:07.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.320+0000 7f95227fc700 1 --2- 172.21.15.92:0/725136167 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9514066f30 0x7f95140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:07.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.320+0000 7f95227fc700 1 -- 172.21.15.92:0/725136167 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9528084360 con 0x7f95340b35d0 2023-10-06T10:43:07.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.320+0000 7f953259c700 1 --2- 172.21.15.92:0/725136167 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9514066f30 0x7f95140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:07.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.320+0000 7f953259c700 1 --2- 172.21.15.92:0/725136167 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9514066f30 0x7f95140693e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f952c0093f0 tx=0x7f952c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:07.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.324+0000 7f95227fc700 1 -- 172.21.15.92:0/725136167 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9528051330 con 0x7f95340b35d0 2023-10-06T10:43:07.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.508+0000 7f953359e700 1 -- 172.21.15.92:0/725136167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f953405f230 con 0x7f95340b35d0 2023-10-06T10:43:07.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.509+0000 7f95227fc700 1 -- 172.21.15.92:0/725136167 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+17074 (secure 0 0 0) 0x7f9528054950 con 0x7f95340b35d0 2023-10-06T10:43:07.511 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:07.511 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":78,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:43:04.326791+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:43:00.816800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-06T10:43:02.128579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:43:07.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.511+0000 7f953359e700 1 -- 172.21.15.92:0/725136167 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9514066f30 msgr2=0x7f95140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:07.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.511+0000 7f953359e700 1 --2- 172.21.15.92:0/725136167 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9514066f30 0x7f95140693e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f952c0093f0 tx=0x7f952c017000 comp rx=0 tx=0).stop 2023-10-06T10:43:07.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.511+0000 7f953359e700 1 -- 172.21.15.92:0/725136167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95340b35d0 msgr2=0x7f95340cc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:07.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.511+0000 7f953359e700 1 --2- 172.21.15.92:0/725136167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95340b35d0 0x7f95340cc6f0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f952800c790 tx=0x7f9528007f60 comp rx=0 tx=0).stop 2023-10-06T10:43:07.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.511+0000 7f953359e700 1 -- 172.21.15.92:0/725136167 shutdown_connections 2023-10-06T10:43:07.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.511+0000 7f953359e700 1 --2- 172.21.15.92:0/725136167 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9514066f30 0x7f95140693e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.511+0000 7f953359e700 1 --2- 172.21.15.92:0/725136167 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9534159440 0x7f95340ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.511+0000 7f953359e700 1 --2- 172.21.15.92:0/725136167 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95340b2100 0x7f9534148540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.511+0000 7f953359e700 1 --2- 172.21.15.92:0/725136167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95340b35d0 0x7f95340cc6f0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.511+0000 7f953359e700 1 -- 172.21.15.92:0/725136167 >> 172.21.15.92:0/725136167 conn(0x7f953406ece0 msgr2=0x7f95340b4770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:07.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.511+0000 7f953359e700 1 -- 172.21.15.92:0/725136167 shutdown_connections 2023-10-06T10:43:07.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.511+0000 7f953359e700 1 -- 172.21.15.92:0/725136167 wait complete. 2023-10-06T10:43:07.525 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T10:43:07.525 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:43:07.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.637+0000 7f010ee75700 1 Processor -- start 2023-10-06T10:43:07.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.637+0000 7f010ee75700 1 -- start start 2023-10-06T10:43:07.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.638+0000 7f010ee75700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01080b49a0 0x7f01080aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:07.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.638+0000 7f010ee75700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f01080b0760 0x7f01080b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:07.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.638+0000 7f010ee75700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01080b0c60 con 0x7f0108159560 2023-10-06T10:43:07.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.638+0000 7f010ee75700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01080b0f40 con 0x7f01080b0760 2023-10-06T10:43:07.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.638+0000 7f010ee75700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01080b1220 con 0x7f01080b49a0 2023-10-06T10:43:07.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.638+0000 7f010cc11700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0108159560 0x7f01080aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51874/0 (socket says 172.21.15.92:51874) 2023-10-06T10:43:07.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.638+0000 7f010cc11700 1 -- 172.21.15.92:0/3317143887 learned_addr learned my addr 172.21.15.92:0/3317143887 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:07.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.638+0000 7f010d412700 1 --2- 172.21.15.92:0/3317143887 >> v2:172.21.15.124:3300/0 conn(0x7f01080b0760 0x7f01080b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:07.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.638+0000 7f0107fff700 1 --2- 172.21.15.92:0/3317143887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01080b49a0 0x7f01080aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:07.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.638+0000 7f01077fe700 1 -- 172.21.15.92:0/3317143887 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042014820 0 0) 0x7f01080b0c60 con 0x7f0108159560 2023-10-06T10:43:07.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.638+0000 7f01077fe700 1 -- 172.21.15.92:0/3317143887 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00f8003570 con 0x7f0108159560 2023-10-06T10:43:07.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.639+0000 7f010d412700 1 -- 172.21.15.92:0/3317143887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01080b49a0 msgr2=0x7f01080aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:07.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.639+0000 7f010d412700 1 --2- 172.21.15.92:0/3317143887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01080b49a0 0x7f01080aac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.639+0000 7f01077fe700 1 -- 172.21.15.92:0/3317143887 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3299485610 0 0) 0x7f00f8003570 con 0x7f0108159560 2023-10-06T10:43:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.639+0000 7f010d412700 1 -- 172.21.15.92:0/3317143887 >> v1:172.21.15.92:6789/0 conn(0x7f0108159560 legacy=0x7f01080aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.639+0000 7f010d412700 1 -- 172.21.15.92:0/3317143887 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01080b1500 con 0x7f01080b0760 2023-10-06T10:43:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.639+0000 7f010d412700 1 --2- 172.21.15.92:0/3317143887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01080b0760 0x7f01080b1bc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0100009e60 tx=0x7f0100009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=184a529a8dc14aba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.639+0000 7f01077fe700 1 -- 172.21.15.92:0/3317143887 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010000e070 con 0x7f01080b0760 2023-10-06T10:43:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.639+0000 7f01077fe700 1 -- 172.21.15.92:0/3317143887 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010000b7a0 con 0x7f01080b0760 2023-10-06T10:43:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.639+0000 7f01077fe700 1 -- 172.21.15.92:0/3317143887 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc0015f0 con 0x7f01080b0760 2023-10-06T10:43:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.640+0000 7f010ee75700 1 -- 172.21.15.92:0/3317143887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01080b0760 msgr2=0x7f01080b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.640+0000 7f010ee75700 1 --2- 172.21.15.92:0/3317143887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01080b0760 0x7f01080b1bc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0100009e60 tx=0x7f0100009780 comp rx=0 tx=0).stop 2023-10-06T10:43:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.640+0000 7f010ee75700 1 -- 172.21.15.92:0/3317143887 shutdown_connections 2023-10-06T10:43:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.640+0000 7f010ee75700 1 --2- 172.21.15.92:0/3317143887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01080b0760 0x7f01080b1bc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.640+0000 7f010ee75700 1 --2- 172.21.15.92:0/3317143887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01080b49a0 0x7f01080aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.640+0000 7f010ee75700 1 -- 172.21.15.92:0/3317143887 >> 172.21.15.92:0/3317143887 conn(0x7f01080997f0 msgr2=0x7f010806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:07.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.640+0000 7f010ee75700 1 -- 172.21.15.92:0/3317143887 shutdown_connections 2023-10-06T10:43:07.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.640+0000 7f010ee75700 1 -- 172.21.15.92:0/3317143887 wait complete. 2023-10-06T10:43:07.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.641+0000 7f010ee75700 1 Processor -- start 2023-10-06T10:43:07.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.641+0000 7f010ee75700 1 -- start start 2023-10-06T10:43:07.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.642+0000 7f010ee75700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01080b0760 0x7f010809d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:07.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.642+0000 7f010ee75700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01080b49a0 0x7f0108159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:07.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.642+0000 7f010ee75700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0108159560 0x7f0108159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:07.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.642+0000 7f010ee75700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010809aa90 con 0x7f01080b0760 2023-10-06T10:43:07.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.642+0000 7f010ee75700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010815c1f0 con 0x7f01080b49a0 2023-10-06T10:43:07.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.642+0000 7f010ee75700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010809dbc0 con 0x7f0108159560 2023-10-06T10:43:07.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.642+0000 7f010cc11700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01080b0760 0x7f010809d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:07.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.642+0000 7f010cc11700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01080b0760 0x7f010809d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33894/0 (socket says 172.21.15.92:33894) 2023-10-06T10:43:07.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.642+0000 7f010cc11700 1 -- 172.21.15.92:0/2560361757 learned_addr learned my addr 172.21.15.92:0/2560361757 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:07.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.642+0000 7f0107fff700 1 --2- 172.21.15.92:0/2560361757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01080b49a0 0x7f0108159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:07.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.642+0000 7f010d412700 1 --2- 172.21.15.92:0/2560361757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0108159560 0x7f0108159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:07.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.642+0000 7f010cc11700 1 -- 172.21.15.92:0/2560361757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0108159560 msgr2=0x7f0108159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:07.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.642+0000 7f010cc11700 1 --2- 172.21.15.92:0/2560361757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0108159560 0x7f0108159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.642+0000 7f010cc11700 1 -- 172.21.15.92:0/2560361757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01080b49a0 msgr2=0x7f0108159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:07.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.642+0000 7f010cc11700 1 --2- 172.21.15.92:0/2560361757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01080b49a0 0x7f0108159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.642+0000 7f010cc11700 1 -- 172.21.15.92:0/2560361757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0100009a20 con 0x7f01080b0760 2023-10-06T10:43:07.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.642+0000 7f010cc11700 1 --2- 172.21.15.92:0/2560361757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01080b0760 0x7f010809d680 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00cb60 tx=0x7f00fc0013b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:07.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.642+0000 7f0104ff9700 1 -- 172.21.15.92:0/2560361757 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc0070f0 con 0x7f01080b0760 2023-10-06T10:43:07.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.642+0000 7f0104ff9700 1 -- 172.21.15.92:0/2560361757 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00fc007270 con 0x7f01080b0760 2023-10-06T10:43:07.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.642+0000 7f010ee75700 1 -- 172.21.15.92:0/2560361757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01080af000 con 0x7f01080b0760 2023-10-06T10:43:07.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.642+0000 7f0104ff9700 1 -- 172.21.15.92:0/2560361757 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc007630 con 0x7f01080b0760 2023-10-06T10:43:07.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.642+0000 7f010ee75700 1 -- 172.21.15.92:0/2560361757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010809a0f0 con 0x7f01080b0760 2023-10-06T10:43:07.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.643+0000 7f010ee75700 1 -- 172.21.15.92:0/2560361757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0108064a90 con 0x7f01080b0760 2023-10-06T10:43:07.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.644+0000 7f0104ff9700 1 -- 172.21.15.92:0/2560361757 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f00fc005a00 con 0x7f01080b0760 2023-10-06T10:43:07.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.644+0000 7f0104ff9700 1 --2- 172.21.15.92:0/2560361757 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f00ec066f30 0x7f00ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:07.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.644+0000 7f0104ff9700 1 -- 172.21.15.92:0/2560361757 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f00fc0843a0 con 0x7f01080b0760 2023-10-06T10:43:07.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.644+0000 7f0107fff700 1 --2- 172.21.15.92:0/2560361757 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f00ec066f30 0x7f00ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:07.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.645+0000 7f0107fff700 1 --2- 172.21.15.92:0/2560361757 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f00ec066f30 0x7f00ec0693e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f010815c8e0 tx=0x7f00f40091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:07.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.648+0000 7f0104ff9700 1 -- 172.21.15.92:0/2560361757 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00fc0513f0 con 0x7f01080b0760 2023-10-06T10:43:07.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.830+0000 7f010ee75700 1 -- 172.21.15.92:0/2560361757 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f01080b17e0 con 0x7f00ec066f30 2023-10-06T10:43:07.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.839+0000 7f0104ff9700 1 -- 172.21.15.92:0/2560361757 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311653 (secure 0 0 0) 0x7f01080b17e0 con 0x7f00ec066f30 2023-10-06T10:43:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:07.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.842+0000 7f010ee75700 1 -- 172.21.15.92:0/2560361757 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f00ec066f30 msgr2=0x7f00ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:07.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.842+0000 7f010ee75700 1 --2- 172.21.15.92:0/2560361757 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f00ec066f30 0x7f00ec0693e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f010815c8e0 tx=0x7f00f40091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.842+0000 7f010ee75700 1 -- 172.21.15.92:0/2560361757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01080b0760 msgr2=0x7f010809d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:07.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.842+0000 7f010ee75700 1 --2- 172.21.15.92:0/2560361757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01080b0760 0x7f010809d680 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00cb60 tx=0x7f00fc0013b0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.842+0000 7f010ee75700 1 -- 172.21.15.92:0/2560361757 shutdown_connections 2023-10-06T10:43:07.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.842+0000 7f010ee75700 1 --2- 172.21.15.92:0/2560361757 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f00ec066f30 0x7f00ec0693e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.842+0000 7f010ee75700 1 --2- 172.21.15.92:0/2560361757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0108159560 0x7f0108159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.842+0000 7f010ee75700 1 --2- 172.21.15.92:0/2560361757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01080b0760 0x7f010809d680 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.842+0000 7f010ee75700 1 --2- 172.21.15.92:0/2560361757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01080b49a0 0x7f0108159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:07.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.842+0000 7f010ee75700 1 -- 172.21.15.92:0/2560361757 >> 172.21.15.92:0/2560361757 conn(0x7f01080997f0 msgr2=0x7f0108069350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:07.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.842+0000 7f010ee75700 1 -- 172.21.15.92:0/2560361757 shutdown_connections 2023-10-06T10:43:07.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:07.842+0000 7f010ee75700 1 -- 172.21.15.92:0/2560361757 wait complete. 2023-10-06T10:43:07.847 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:43:07.856 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2023-10-06T10:43:06.145645+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":125,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2386868,"kb_used_data":8564,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752587852,"statfs":{"total":773094113280,"available":770649960448,"internally_reserved":0,"allocated":8769536,"data_stored":4781759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.367447"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.348825+0000","last_change":"2023-10-06T10:43:03.348825+0000","last_active":"2023-10-06T10:43:03.348825+0000","last_peered":"2023-10-06T10:43:03.348825+0000","last_clean":"2023-10-06T10:43:03.348825+0000","last_became_active":"2023-10-06T10:43:03.348377+0000","last_became_peered":"2023-10-06T10:43:03.348377+0000","last_unstale":"2023-10-06T10:43:03.348825+0000","last_undegraded":"2023-10-06T10:43:03.348825+0000","last_fullsized":"2023-10-06T10:43:03.348825+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:12:07.366824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.348970+0000","last_change":"2023-10-06T10:43:03.348970+0000","last_active":"2023-10-06T10:43:03.348970+0000","last_peered":"2023-10-06T10:43:03.348970+0000","last_clean":"2023-10-06T10:43:03.348970+0000","last_became_active":"2023-10-06T10:43:03.347340+0000","last_became_peered":"2023-10-06T10:43:03.347340+0000","last_unstale":"2023-10-06T10:43:03.348970+0000","last_undegraded":"2023-10-06T10:43:03.348970+0000","last_fullsized":"2023-10-06T10:43:03.348970+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:07.715335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.350831+0000","last_change":"2023-10-06T10:43:03.350831+0000","last_active":"2023-10-06T10:43:03.350831+0000","last_peered":"2023-10-06T10:43:03.350831+0000","last_clean":"2023-10-06T10:43:03.350831+0000","last_became_active":"2023-10-06T10:43:03.350571+0000","last_became_peered":"2023-10-06T10:43:03.350571+0000","last_unstale":"2023-10-06T10:43:03.350831+0000","last_undegraded":"2023-10-06T10:43:03.350831+0000","last_fullsized":"2023-10-06T10:43:03.350831+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:51.848204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.342981+0000","last_change":"2023-10-06T10:43:03.342981+0000","last_active":"2023-10-06T10:43:03.342981+0000","last_peered":"2023-10-06T10:43:03.342981+0000","last_clean":"2023-10-06T10:43:03.342981+0000","last_became_active":"2023-10-06T10:43:03.342730+0000","last_became_peered":"2023-10-06T10:43:03.342730+0000","last_unstale":"2023-10-06T10:43:03.342981+0000","last_undegraded":"2023-10-06T10:43:03.342981+0000","last_fullsized":"2023-10-06T10:43:03.342981+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:35:50.641107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.340918+0000","last_change":"2023-10-06T10:43:03.340918+0000","last_active":"2023-10-06T10:43:03.340918+0000","last_peered":"2023-10-06T10:43:03.340918+0000","last_clean":"2023-10-06T10:43:03.340918+0000","last_became_active":"2023-10-06T10:43:03.340633+0000","last_became_peered":"2023-10-06T10:43:03.340633+0000","last_unstale":"2023-10-06T10:43:03.340918+0000","last_undegraded":"2023-10-06T10:43:03.340918+0000","last_fullsized":"2023-10-06T10:43:03.340918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:25:13.175569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.341651+0000","last_change":"2023-10-06T10:43:03.341651+0000","last_active":"2023-10-06T10:43:03.341651+0000","last_peered":"2023-10-06T10:43:03.341651+0000","last_clean":"2023-10-06T10:43:03.341651+0000","last_became_active":"2023-10-06T10:43:03.341409+0000","last_became_peered":"2023-10-06T10:43:03.341409+0000","last_unstale":"2023-10-06T10:43:03.341651+0000","last_undegraded":"2023-10-06T10:43:03.341651+0000","last_fullsized":"2023-10-06T10:43:03.341651+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:54:25.531456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.347365+0000","last_change":"2023-10-06T10:43:03.347365+0000","last_active":"2023-10-06T10:43:03.347365+0000","last_peered":"2023-10-06T10:43:03.347365+0000","last_clean":"2023-10-06T10:43:03.347365+0000","last_became_active":"2023-10-06T10:43:03.347119+0000","last_became_peered":"2023-10-06T10:43:03.347119+0000","last_unstale":"2023-10-06T10:43:03.347365+0000","last_undegraded":"2023-10-06T10:43:03.347365+0000","last_fullsized":"2023-10-06T10:43:03.347365+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:54.517332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.344615+0000","last_change":"2023-10-06T10:43:03.344615+0000","last_active":"2023-10-06T10:43:03.344615+0000","last_peered":"2023-10-06T10:43:03.344615+0000","last_clean":"2023-10-06T10:43:03.344615+0000","last_became_active":"2023-10-06T10:43:03.344361+0000","last_became_peered":"2023-10-06T10:43:03.344361+0000","last_unstale":"2023-10-06T10:43:03.344615+0000","last_undegraded":"2023-10-06T10:43:03.344615+0000","last_fullsized":"2023-10-06T10:43:03.344615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:05.694257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.346369+0000","last_change":"2023-10-06T10:43:03.346369+0000","last_active":"2023-10-06T10:43:03.346369+0000","last_peered":"2023-10-06T10:43:03.346369+0000","last_clean":"2023-10-06T10:43:03.346369+0000","last_became_active":"2023-10-06T10:43:03.346096+0000","last_became_peered":"2023-10-06T10:43:03.346096+0000","last_unstale":"2023-10-06T10:43:03.346369+0000","last_undegraded":"2023-10-06T10:43:03.346369+0000","last_fullsized":"2023-10-06T10:43:03.346369+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:51.848204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.344088+0000","last_change":"2023-10-06T10:43:03.344088+0000","last_active":"2023-10-06T10:43:03.344088+0000","last_peered":"2023-10-06T10:43:03.344088+0000","last_clean":"2023-10-06T10:43:03.344088+0000","last_became_active":"2023-10-06T10:43:03.343824+0000","last_became_peered":"2023-10-06T10:43:03.343824+0000","last_unstale":"2023-10-06T10:43:03.344088+0000","last_undegraded":"2023-10-06T10:43:03.344088+0000","last_fullsized":"2023-10-06T10:43:03.344088+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:05.694257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.348177+0000","last_change":"2023-10-06T10:43:03.348177+0000","last_active":"2023-10-06T10:43:03.348177+0000","last_peered":"2023-10-06T10:43:03.348177+0000","last_clean":"2023-10-06T10:43:03.348177+0000","last_became_active":"2023-10-06T10:43:03.347898+0000","last_became_peered":"2023-10-06T10:43:03.347898+0000","last_unstale":"2023-10-06T10:43:03.348177+0000","last_undegraded":"2023-10-06T10:43:03.348177+0000","last_fullsized":"2023-10-06T10:43:03.348177+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:13.782049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.345566+0000","last_change":"2023-10-06T10:43:03.345566+0000","last_active":"2023-10-06T10:43:03.345566+0000","last_peered":"2023-10-06T10:43:03.345566+0000","last_clean":"2023-10-06T10:43:03.345566+0000","last_became_active":"2023-10-06T10:43:03.345291+0000","last_became_peered":"2023-10-06T10:43:03.345291+0000","last_unstale":"2023-10-06T10:43:03.345566+0000","last_undegraded":"2023-10-06T10:43:03.345566+0000","last_fullsized":"2023-10-06T10:43:03.345566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:42:16.964813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.352521+0000","last_change":"2023-10-06T10:43:03.352521+0000","last_active":"2023-10-06T10:43:03.352521+0000","last_peered":"2023-10-06T10:43:03.352521+0000","last_clean":"2023-10-06T10:43:03.352521+0000","last_became_active":"2023-10-06T10:43:03.352214+0000","last_became_peered":"2023-10-06T10:43:03.352214+0000","last_unstale":"2023-10-06T10:43:03.352521+0000","last_undegraded":"2023-10-06T10:43:03.352521+0000","last_fullsized":"2023-10-06T10:43:03.352521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:19:17.831613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.344001+0000","last_change":"2023-10-06T10:43:03.344001+0000","last_active":"2023-10-06T10:43:03.344001+0000","last_peered":"2023-10-06T10:43:03.344001+0000","last_clean":"2023-10-06T10:43:03.344001+0000","last_became_active":"2023-10-06T10:43:03.343756+0000","last_became_peered":"2023-10-06T10:43:03.343756+0000","last_unstale":"2023-10-06T10:43:03.344001+0000","last_undegraded":"2023-10-06T10:43:03.344001+0000","last_fullsized":"2023-10-06T10:43:03.344001+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:16.354467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.347968+0000","last_change":"2023-10-06T10:43:03.347968+0000","last_active":"2023-10-06T10:43:03.347968+0000","last_peered":"2023-10-06T10:43:03.347968+0000","last_clean":"2023-10-06T10:43:03.347968+0000","last_became_active":"2023-10-06T10:43:03.347720+0000","last_became_peered":"2023-10-06T10:43:03.347720+0000","last_unstale":"2023-10-06T10:43:03.347968+0000","last_undegraded":"2023-10-06T10:43:03.347968+0000","last_fullsized":"2023-10-06T10:43:03.347968+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:33.284454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.340506+0000","last_change":"2023-10-06T10:43:03.340506+0000","last_active":"2023-10-06T10:43:03.340506+0000","last_peered":"2023-10-06T10:43:03.340506+0000","last_clean":"2023-10-06T10:43:03.340506+0000","last_became_active":"2023-10-06T10:43:03.340213+0000","last_became_peered":"2023-10-06T10:43:03.340213+0000","last_unstale":"2023-10-06T10:43:03.340506+0000","last_undegraded":"2023-10-06T10:43:03.340506+0000","last_fullsized":"2023-10-06T10:43:03.340506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:51.840777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.341514+0000","last_change":"2023-10-06T10:43:03.341514+0000","last_active":"2023-10-06T10:43:03.341514+0000","last_peered":"2023-10-06T10:43:03.341514+0000","last_clean":"2023-10-06T10:43:03.341514+0000","last_became_active":"2023-10-06T10:43:03.340789+0000","last_became_peered":"2023-10-06T10:43:03.340789+0000","last_unstale":"2023-10-06T10:43:03.341514+0000","last_undegraded":"2023-10-06T10:43:03.341514+0000","last_fullsized":"2023-10-06T10:43:03.341514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:39.097044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.348045+0000","last_change":"2023-10-06T10:43:03.348045+0000","last_active":"2023-10-06T10:43:03.348045+0000","last_peered":"2023-10-06T10:43:03.348045+0000","last_clean":"2023-10-06T10:43:03.348045+0000","last_became_active":"2023-10-06T10:43:03.347825+0000","last_became_peered":"2023-10-06T10:43:03.347825+0000","last_unstale":"2023-10-06T10:43:03.348045+0000","last_undegraded":"2023-10-06T10:43:03.348045+0000","last_fullsized":"2023-10-06T10:43:03.348045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:13.782049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.346596+0000","last_change":"2023-10-06T10:43:03.346596+0000","last_active":"2023-10-06T10:43:03.346596+0000","last_peered":"2023-10-06T10:43:03.346596+0000","last_clean":"2023-10-06T10:43:03.346596+0000","last_became_active":"2023-10-06T10:43:03.346290+0000","last_became_peered":"2023-10-06T10:43:03.346290+0000","last_unstale":"2023-10-06T10:43:03.346596+0000","last_undegraded":"2023-10-06T10:43:03.346596+0000","last_fullsized":"2023-10-06T10:43:03.346596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:39.097044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.346373+0000","last_change":"2023-10-06T10:43:03.346373+0000","last_active":"2023-10-06T10:43:03.346373+0000","last_peered":"2023-10-06T10:43:03.346373+0000","last_clean":"2023-10-06T10:43:03.346373+0000","last_became_active":"2023-10-06T10:43:03.345998+0000","last_became_peered":"2023-10-06T10:43:03.345998+0000","last_unstale":"2023-10-06T10:43:03.346373+0000","last_undegraded":"2023-10-06T10:43:03.346373+0000","last_fullsized":"2023-10-06T10:43:03.346373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:01.155194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.346220+0000","last_change":"2023-10-06T10:43:03.346220+0000","last_active":"2023-10-06T10:43:03.346220+0000","last_peered":"2023-10-06T10:43:03.346220+0000","last_clean":"2023-10-06T10:43:03.346220+0000","last_became_active":"2023-10-06T10:43:03.345984+0000","last_became_peered":"2023-10-06T10:43:03.345984+0000","last_unstale":"2023-10-06T10:43:03.346220+0000","last_undegraded":"2023-10-06T10:43:03.346220+0000","last_fullsized":"2023-10-06T10:43:03.346220+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:36.448540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.345790+0000","last_change":"2023-10-06T10:43:03.345790+0000","last_active":"2023-10-06T10:43:03.345790+0000","last_peered":"2023-10-06T10:43:03.345790+0000","last_clean":"2023-10-06T10:43:03.345790+0000","last_became_active":"2023-10-06T10:43:03.345549+0000","last_became_peered":"2023-10-06T10:43:03.345549+0000","last_unstale":"2023-10-06T10:43:03.345790+0000","last_undegraded":"2023-10-06T10:43:03.345790+0000","last_fullsized":"2023-10-06T10:43:03.345790+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:48:25.036868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.346212+0000","last_change":"2023-10-06T10:43:03.346212+0000","last_active":"2023-10-06T10:43:03.346212+0000","last_peered":"2023-10-06T10:43:03.346212+0000","last_clean":"2023-10-06T10:43:03.346212+0000","last_became_active":"2023-10-06T10:43:03.345908+0000","last_became_peered":"2023-10-06T10:43:03.345908+0000","last_unstale":"2023-10-06T10:43:03.346212+0000","last_undegraded":"2023-10-06T10:43:03.346212+0000","last_fullsized":"2023-10-06T10:43:03.346212+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:51.848204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.345658+0000","last_change":"2023-10-06T10:43:03.345658+0000","last_active":"2023-10-06T10:43:03.345658+0000","last_peered":"2023-10-06T10:43:03.345658+0000","last_clean":"2023-10-06T10:43:03.345658+0000","last_became_active":"2023-10-06T10:43:03.345386+0000","last_became_peered":"2023-10-06T10:43:03.345386+0000","last_unstale":"2023-10-06T10:43:03.345658+0000","last_undegraded":"2023-10-06T10:43:03.345658+0000","last_fullsized":"2023-10-06T10:43:03.345658+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:55.127339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.349245+0000","last_change":"2023-10-06T10:43:03.349245+0000","last_active":"2023-10-06T10:43:03.349245+0000","last_peered":"2023-10-06T10:43:03.349245+0000","last_clean":"2023-10-06T10:43:03.349245+0000","last_became_active":"2023-10-06T10:43:03.348976+0000","last_became_peered":"2023-10-06T10:43:03.348976+0000","last_unstale":"2023-10-06T10:43:03.349245+0000","last_undegraded":"2023-10-06T10:43:03.349245+0000","last_fullsized":"2023-10-06T10:43:03.349245+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:18:33.527316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.351925+0000","last_change":"2023-10-06T10:43:03.351925+0000","last_active":"2023-10-06T10:43:03.351925+0000","last_peered":"2023-10-06T10:43:03.351925+0000","last_clean":"2023-10-06T10:43:03.351925+0000","last_became_active":"2023-10-06T10:43:03.351677+0000","last_became_peered":"2023-10-06T10:43:03.351677+0000","last_unstale":"2023-10-06T10:43:03.351925+0000","last_undegraded":"2023-10-06T10:43:03.351925+0000","last_fullsized":"2023-10-06T10:43:03.351925+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:05:18.178107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.347216+0000","last_change":"2023-10-06T10:43:03.347216+0000","last_active":"2023-10-06T10:43:03.347216+0000","last_peered":"2023-10-06T10:43:03.347216+0000","last_clean":"2023-10-06T10:43:03.347216+0000","last_became_active":"2023-10-06T10:43:03.346584+0000","last_became_peered":"2023-10-06T10:43:03.346584+0000","last_unstale":"2023-10-06T10:43:03.347216+0000","last_undegraded":"2023-10-06T10:43:03.347216+0000","last_fullsized":"2023-10-06T10:43:03.347216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:08.238984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.350148+0000","last_change":"2023-10-06T10:43:03.350148+0000","last_active":"2023-10-06T10:43:03.350148+0000","last_peered":"2023-10-06T10:43:03.350148+0000","last_clean":"2023-10-06T10:43:03.350148+0000","last_became_active":"2023-10-06T10:43:03.349687+0000","last_became_peered":"2023-10-06T10:43:03.349687+0000","last_unstale":"2023-10-06T10:43:03.350148+0000","last_undegraded":"2023-10-06T10:43:03.350148+0000","last_fullsized":"2023-10-06T10:43:03.350148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:39.049097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.343448+0000","last_change":"2023-10-06T10:43:03.343448+0000","last_active":"2023-10-06T10:43:03.343448+0000","last_peered":"2023-10-06T10:43:03.343448+0000","last_clean":"2023-10-06T10:43:03.343448+0000","last_became_active":"2023-10-06T10:43:03.343195+0000","last_became_peered":"2023-10-06T10:43:03.343195+0000","last_unstale":"2023-10-06T10:43:03.343448+0000","last_undegraded":"2023-10-06T10:43:03.343448+0000","last_fullsized":"2023-10-06T10:43:03.343448+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:27.895051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.340843+0000","last_change":"2023-10-06T10:43:03.340843+0000","last_active":"2023-10-06T10:43:03.340843+0000","last_peered":"2023-10-06T10:43:03.340843+0000","last_clean":"2023-10-06T10:43:03.340843+0000","last_became_active":"2023-10-06T10:43:03.340655+0000","last_became_peered":"2023-10-06T10:43:03.340655+0000","last_unstale":"2023-10-06T10:43:03.340843+0000","last_undegraded":"2023-10-06T10:43:03.340843+0000","last_fullsized":"2023-10-06T10:43:03.340843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:08.779362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.347744+0000","last_change":"2023-10-06T10:43:03.347744+0000","last_active":"2023-10-06T10:43:03.347744+0000","last_peered":"2023-10-06T10:43:03.347744+0000","last_clean":"2023-10-06T10:43:03.347744+0000","last_became_active":"2023-10-06T10:43:03.346519+0000","last_became_peered":"2023-10-06T10:43:03.346519+0000","last_unstale":"2023-10-06T10:43:03.347744+0000","last_undegraded":"2023-10-06T10:43:03.347744+0000","last_fullsized":"2023-10-06T10:43:03.347744+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:33.284454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.345365+0000","last_change":"2023-10-06T10:43:03.345365+0000","last_active":"2023-10-06T10:43:03.345365+0000","last_peered":"2023-10-06T10:43:03.345365+0000","last_clean":"2023-10-06T10:43:03.345365+0000","last_became_active":"2023-10-06T10:43:03.345107+0000","last_became_peered":"2023-10-06T10:43:03.345107+0000","last_unstale":"2023-10-06T10:43:03.345365+0000","last_undegraded":"2023-10-06T10:43:03.345365+0000","last_fullsized":"2023-10-06T10:43:03.345365+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:02.436557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.347589+0000","last_change":"2023-10-06T10:43:03.347589+0000","last_active":"2023-10-06T10:43:03.347589+0000","last_peered":"2023-10-06T10:43:03.347589+0000","last_clean":"2023-10-06T10:43:03.347589+0000","last_became_active":"2023-10-06T10:43:03.347334+0000","last_became_peered":"2023-10-06T10:43:03.347334+0000","last_unstale":"2023-10-06T10:43:03.347589+0000","last_undegraded":"2023-10-06T10:43:03.347589+0000","last_fullsized":"2023-10-06T10:43:03.347589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:17:36.881748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"77'4","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.796402+0000","last_change":"2023-10-06T10:43:02.804016+0000","last_active":"2023-10-06T10:43:03.796402+0000","last_peered":"2023-10-06T10:43:03.796402+0000","last_clean":"2023-10-06T10:43:03.796402+0000","last_became_active":"2023-10-06T10:43:02.803783+0000","last_became_peered":"2023-10-06T10:43:02.803783+0000","last_unstale":"2023-10-06T10:43:03.796402+0000","last_undegraded":"2023-10-06T10:43:03.796402+0000","last_fullsized":"2023-10-06T10:43:03.796402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:55:52.806135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.348623+0000","last_change":"2023-10-06T10:43:03.348623+0000","last_active":"2023-10-06T10:43:03.348623+0000","last_peered":"2023-10-06T10:43:03.348623+0000","last_clean":"2023-10-06T10:43:03.348623+0000","last_became_active":"2023-10-06T10:43:03.348015+0000","last_became_peered":"2023-10-06T10:43:03.348015+0000","last_unstale":"2023-10-06T10:43:03.348623+0000","last_undegraded":"2023-10-06T10:43:03.348623+0000","last_fullsized":"2023-10-06T10:43:03.348623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:17:21.634775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"77'1","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.334466+0000","last_change":"2023-10-06T10:43:02.802669+0000","last_active":"2023-10-06T10:43:03.334466+0000","last_peered":"2023-10-06T10:43:03.334466+0000","last_clean":"2023-10-06T10:43:03.334466+0000","last_became_active":"2023-10-06T10:43:02.802413+0000","last_became_peered":"2023-10-06T10:43:02.802413+0000","last_unstale":"2023-10-06T10:43:03.334466+0000","last_undegraded":"2023-10-06T10:43:03.334466+0000","last_fullsized":"2023-10-06T10:43:03.334466+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:50.843346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.345762+0000","last_change":"2023-10-06T10:43:03.345762+0000","last_active":"2023-10-06T10:43:03.345762+0000","last_peered":"2023-10-06T10:43:03.345762+0000","last_clean":"2023-10-06T10:43:03.345762+0000","last_became_active":"2023-10-06T10:43:03.344991+0000","last_became_peered":"2023-10-06T10:43:03.344991+0000","last_unstale":"2023-10-06T10:43:03.345762+0000","last_undegraded":"2023-10-06T10:43:03.345762+0000","last_fullsized":"2023-10-06T10:43:03.345762+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:29:39.107500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"77'2","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.794933+0000","last_change":"2023-10-06T10:43:02.796112+0000","last_active":"2023-10-06T10:43:03.794933+0000","last_peered":"2023-10-06T10:43:03.794933+0000","last_clean":"2023-10-06T10:43:03.794933+0000","last_became_active":"2023-10-06T10:43:02.795857+0000","last_became_peered":"2023-10-06T10:43:02.795857+0000","last_unstale":"2023-10-06T10:43:03.794933+0000","last_undegraded":"2023-10-06T10:43:03.794933+0000","last_fullsized":"2023-10-06T10:43:03.794933+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:38.044240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.352342+0000","last_change":"2023-10-06T10:43:03.352342+0000","last_active":"2023-10-06T10:43:03.352342+0000","last_peered":"2023-10-06T10:43:03.352342+0000","last_clean":"2023-10-06T10:43:03.352342+0000","last_became_active":"2023-10-06T10:43:03.352042+0000","last_became_peered":"2023-10-06T10:43:03.352042+0000","last_unstale":"2023-10-06T10:43:03.352342+0000","last_undegraded":"2023-10-06T10:43:03.352342+0000","last_fullsized":"2023-10-06T10:43:03.352342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:21.180392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"77'2","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.795169+0000","last_change":"2023-10-06T10:43:02.796566+0000","last_active":"2023-10-06T10:43:03.795169+0000","last_peered":"2023-10-06T10:43:03.795169+0000","last_clean":"2023-10-06T10:43:03.795169+0000","last_became_active":"2023-10-06T10:43:02.796337+0000","last_became_peered":"2023-10-06T10:43:02.796337+0000","last_unstale":"2023-10-06T10:43:03.795169+0000","last_undegraded":"2023-10-06T10:43:03.795169+0000","last_fullsized":"2023-10-06T10:43:03.795169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:01.431700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.340735+0000","last_change":"2023-10-06T10:43:03.340735+0000","last_active":"2023-10-06T10:43:03.340735+0000","last_peered":"2023-10-06T10:43:03.340735+0000","last_clean":"2023-10-06T10:43:03.340735+0000","last_became_active":"2023-10-06T10:43:03.340528+0000","last_became_peered":"2023-10-06T10:43:03.340528+0000","last_unstale":"2023-10-06T10:43:03.340735+0000","last_undegraded":"2023-10-06T10:43:03.340735+0000","last_fullsized":"2023-10-06T10:43:03.340735+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:17:24.862748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"77'1","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.332883+0000","last_change":"2023-10-06T10:43:02.795595+0000","last_active":"2023-10-06T10:43:03.332883+0000","last_peered":"2023-10-06T10:43:03.332883+0000","last_clean":"2023-10-06T10:43:03.332883+0000","last_became_active":"2023-10-06T10:43:02.795301+0000","last_became_peered":"2023-10-06T10:43:02.795301+0000","last_unstale":"2023-10-06T10:43:03.332883+0000","last_undegraded":"2023-10-06T10:43:03.332883+0000","last_fullsized":"2023-10-06T10:43:03.332883+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:06:32.177702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.341265+0000","last_change":"2023-10-06T10:43:03.341265+0000","last_active":"2023-10-06T10:43:03.341265+0000","last_peered":"2023-10-06T10:43:03.341265+0000","last_clean":"2023-10-06T10:43:03.341265+0000","last_became_active":"2023-10-06T10:43:03.340997+0000","last_became_peered":"2023-10-06T10:43:03.340997+0000","last_unstale":"2023-10-06T10:43:03.341265+0000","last_undegraded":"2023-10-06T10:43:03.341265+0000","last_fullsized":"2023-10-06T10:43:03.341265+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:32:40.836119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"77'1","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.332883+0000","last_change":"2023-10-06T10:43:02.798764+0000","last_active":"2023-10-06T10:43:03.332883+0000","last_peered":"2023-10-06T10:43:03.332883+0000","last_clean":"2023-10-06T10:43:03.332883+0000","last_became_active":"2023-10-06T10:43:02.798517+0000","last_became_peered":"2023-10-06T10:43:02.798517+0000","last_unstale":"2023-10-06T10:43:03.332883+0000","last_undegraded":"2023-10-06T10:43:03.332883+0000","last_fullsized":"2023-10-06T10:43:03.332883+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:54:24.526598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.350778+0000","last_change":"2023-10-06T10:43:03.350778+0000","last_active":"2023-10-06T10:43:03.350778+0000","last_peered":"2023-10-06T10:43:03.350778+0000","last_clean":"2023-10-06T10:43:03.350778+0000","last_became_active":"2023-10-06T10:43:03.350495+0000","last_became_peered":"2023-10-06T10:43:03.350495+0000","last_unstale":"2023-10-06T10:43:03.350778+0000","last_undegraded":"2023-10-06T10:43:03.350778+0000","last_fullsized":"2023-10-06T10:43:03.350778+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:54:25.531456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.800185+0000","last_change":"2023-10-06T10:43:02.797585+0000","last_active":"2023-10-06T10:43:02.800185+0000","last_peered":"2023-10-06T10:43:02.800185+0000","last_clean":"2023-10-06T10:43:02.800185+0000","last_became_active":"2023-10-06T10:43:02.797370+0000","last_became_peered":"2023-10-06T10:43:02.797370+0000","last_unstale":"2023-10-06T10:43:02.800185+0000","last_undegraded":"2023-10-06T10:43:02.800185+0000","last_fullsized":"2023-10-06T10:43:02.800185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:29:38.102642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.347684+0000","last_change":"2023-10-06T10:43:03.347684+0000","last_active":"2023-10-06T10:43:03.347684+0000","last_peered":"2023-10-06T10:43:03.347684+0000","last_clean":"2023-10-06T10:43:03.347684+0000","last_became_active":"2023-10-06T10:43:03.347398+0000","last_became_peered":"2023-10-06T10:43:03.347398+0000","last_unstale":"2023-10-06T10:43:03.347684+0000","last_undegraded":"2023-10-06T10:43:03.347684+0000","last_fullsized":"2023-10-06T10:43:03.347684+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:50.399077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.805748+0000","last_change":"2023-10-06T10:43:02.803051+0000","last_active":"2023-10-06T10:43:02.805748+0000","last_peered":"2023-10-06T10:43:02.805748+0000","last_clean":"2023-10-06T10:43:02.805748+0000","last_became_active":"2023-10-06T10:43:02.802783+0000","last_became_peered":"2023-10-06T10:43:02.802783+0000","last_unstale":"2023-10-06T10:43:02.805748+0000","last_undegraded":"2023-10-06T10:43:02.805748+0000","last_fullsized":"2023-10-06T10:43:02.805748+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:10.643656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.342893+0000","last_change":"2023-10-06T10:43:03.342893+0000","last_active":"2023-10-06T10:43:03.342893+0000","last_peered":"2023-10-06T10:43:03.342893+0000","last_clean":"2023-10-06T10:43:03.342893+0000","last_became_active":"2023-10-06T10:43:03.342659+0000","last_became_peered":"2023-10-06T10:43:03.342659+0000","last_unstale":"2023-10-06T10:43:03.342893+0000","last_undegraded":"2023-10-06T10:43:03.342893+0000","last_fullsized":"2023-10-06T10:43:03.342893+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:18.479800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.810153+0000","last_change":"2023-10-06T10:43:02.807224+0000","last_active":"2023-10-06T10:43:02.810153+0000","last_peered":"2023-10-06T10:43:02.810153+0000","last_clean":"2023-10-06T10:43:02.810153+0000","last_became_active":"2023-10-06T10:43:02.806999+0000","last_became_peered":"2023-10-06T10:43:02.806999+0000","last_unstale":"2023-10-06T10:43:02.810153+0000","last_undegraded":"2023-10-06T10:43:02.810153+0000","last_fullsized":"2023-10-06T10:43:02.810153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:18:32.522458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.341600+0000","last_change":"2023-10-06T10:43:03.341600+0000","last_active":"2023-10-06T10:43:03.341600+0000","last_peered":"2023-10-06T10:43:03.341600+0000","last_clean":"2023-10-06T10:43:03.341600+0000","last_became_active":"2023-10-06T10:43:03.341329+0000","last_became_peered":"2023-10-06T10:43:03.341329+0000","last_unstale":"2023-10-06T10:43:03.341600+0000","last_undegraded":"2023-10-06T10:43:03.341600+0000","last_fullsized":"2023-10-06T10:43:03.341600+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:19:17.831613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.806855+0000","last_change":"2023-10-06T10:43:02.801375+0000","last_active":"2023-10-06T10:43:02.806855+0000","last_peered":"2023-10-06T10:43:02.806855+0000","last_clean":"2023-10-06T10:43:02.806855+0000","last_became_active":"2023-10-06T10:43:02.800825+0000","last_became_peered":"2023-10-06T10:43:02.800825+0000","last_unstale":"2023-10-06T10:43:02.806855+0000","last_undegraded":"2023-10-06T10:43:02.806855+0000","last_fullsized":"2023-10-06T10:43:02.806855+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:32:16.160390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.350255+0000","last_change":"2023-10-06T10:43:03.350255+0000","last_active":"2023-10-06T10:43:03.350255+0000","last_peered":"2023-10-06T10:43:03.350255+0000","last_clean":"2023-10-06T10:43:03.350255+0000","last_became_active":"2023-10-06T10:43:03.350080+0000","last_became_peered":"2023-10-06T10:43:03.350080+0000","last_unstale":"2023-10-06T10:43:03.350255+0000","last_undegraded":"2023-10-06T10:43:03.350255+0000","last_fullsized":"2023-10-06T10:43:03.350255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:50.399077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"77'3","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.796173+0000","last_change":"2023-10-06T10:43:02.802962+0000","last_active":"2023-10-06T10:43:03.796173+0000","last_peered":"2023-10-06T10:43:03.796173+0000","last_clean":"2023-10-06T10:43:03.796173+0000","last_became_active":"2023-10-06T10:43:02.801745+0000","last_became_peered":"2023-10-06T10:43:02.801745+0000","last_unstale":"2023-10-06T10:43:03.796173+0000","last_undegraded":"2023-10-06T10:43:03.796173+0000","last_fullsized":"2023-10-06T10:43:03.796173+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:49:24.573246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.347913+0000","last_change":"2023-10-06T10:43:03.347913+0000","last_active":"2023-10-06T10:43:03.347913+0000","last_peered":"2023-10-06T10:43:03.347913+0000","last_clean":"2023-10-06T10:43:03.347913+0000","last_became_active":"2023-10-06T10:43:03.347497+0000","last_became_peered":"2023-10-06T10:43:03.347497+0000","last_unstale":"2023-10-06T10:43:03.347913+0000","last_undegraded":"2023-10-06T10:43:03.347913+0000","last_fullsized":"2023-10-06T10:43:03.347913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:21:55.222924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"77'6","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.797539+0000","last_change":"2023-10-06T10:43:02.799454+0000","last_active":"2023-10-06T10:43:03.797539+0000","last_peered":"2023-10-06T10:43:03.797539+0000","last_clean":"2023-10-06T10:43:03.797539+0000","last_became_active":"2023-10-06T10:43:02.799143+0000","last_became_peered":"2023-10-06T10:43:02.799143+0000","last_unstale":"2023-10-06T10:43:03.797539+0000","last_undegraded":"2023-10-06T10:43:03.797539+0000","last_fullsized":"2023-10-06T10:43:03.797539+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:49:24.573246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.340576+0000","last_change":"2023-10-06T10:43:03.340576+0000","last_active":"2023-10-06T10:43:03.340576+0000","last_peered":"2023-10-06T10:43:03.340576+0000","last_clean":"2023-10-06T10:43:03.340576+0000","last_became_active":"2023-10-06T10:43:03.340299+0000","last_became_peered":"2023-10-06T10:43:03.340299+0000","last_unstale":"2023-10-06T10:43:03.340576+0000","last_undegraded":"2023-10-06T10:43:03.340576+0000","last_fullsized":"2023-10-06T10:43:03.340576+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:08.238984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"77'2","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.795759+0000","last_change":"2023-10-06T10:43:02.803801+0000","last_active":"2023-10-06T10:43:03.795759+0000","last_peered":"2023-10-06T10:43:03.795759+0000","last_clean":"2023-10-06T10:43:03.795759+0000","last_became_active":"2023-10-06T10:43:02.803595+0000","last_became_peered":"2023-10-06T10:43:02.803595+0000","last_unstale":"2023-10-06T10:43:03.795759+0000","last_undegraded":"2023-10-06T10:43:03.795759+0000","last_fullsized":"2023-10-06T10:43:03.795759+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:00.410985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.350784+0000","last_change":"2023-10-06T10:43:03.350784+0000","last_active":"2023-10-06T10:43:03.350784+0000","last_peered":"2023-10-06T10:43:03.350784+0000","last_clean":"2023-10-06T10:43:03.350784+0000","last_became_active":"2023-10-06T10:43:03.350314+0000","last_became_peered":"2023-10-06T10:43:03.350314+0000","last_unstale":"2023-10-06T10:43:03.350784+0000","last_undegraded":"2023-10-06T10:43:03.350784+0000","last_fullsized":"2023-10-06T10:43:03.350784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:05.694257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.802071+0000","last_change":"2023-10-06T10:43:02.801545+0000","last_active":"2023-10-06T10:43:02.802071+0000","last_peered":"2023-10-06T10:43:02.802071+0000","last_clean":"2023-10-06T10:43:02.802071+0000","last_became_active":"2023-10-06T10:43:02.797698+0000","last_became_peered":"2023-10-06T10:43:02.797698+0000","last_unstale":"2023-10-06T10:43:02.802071+0000","last_undegraded":"2023-10-06T10:43:02.802071+0000","last_fullsized":"2023-10-06T10:43:02.802071+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:07:44.434485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.339839+0000","last_change":"2023-10-06T10:43:03.339839+0000","last_active":"2023-10-06T10:43:03.339839+0000","last_peered":"2023-10-06T10:43:03.339839+0000","last_clean":"2023-10-06T10:43:03.339839+0000","last_became_active":"2023-10-06T10:43:03.339604+0000","last_became_peered":"2023-10-06T10:43:03.339604+0000","last_unstale":"2023-10-06T10:43:03.339839+0000","last_undegraded":"2023-10-06T10:43:03.339839+0000","last_fullsized":"2023-10-06T10:43:03.339839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:36.448540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.806675+0000","last_change":"2023-10-06T10:43:02.799814+0000","last_active":"2023-10-06T10:43:02.806675+0000","last_peered":"2023-10-06T10:43:02.806675+0000","last_clean":"2023-10-06T10:43:02.806675+0000","last_became_active":"2023-10-06T10:43:02.799531+0000","last_became_peered":"2023-10-06T10:43:02.799531+0000","last_unstale":"2023-10-06T10:43:02.806675+0000","last_undegraded":"2023-10-06T10:43:02.806675+0000","last_fullsized":"2023-10-06T10:43:02.806675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:32:16.160390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.347466+0000","last_change":"2023-10-06T10:43:03.347466+0000","last_active":"2023-10-06T10:43:03.347466+0000","last_peered":"2023-10-06T10:43:03.347466+0000","last_clean":"2023-10-06T10:43:03.347466+0000","last_became_active":"2023-10-06T10:43:03.347222+0000","last_became_peered":"2023-10-06T10:43:03.347222+0000","last_unstale":"2023-10-06T10:43:03.347466+0000","last_undegraded":"2023-10-06T10:43:03.347466+0000","last_fullsized":"2023-10-06T10:43:03.347466+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:45:47.387287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"77'8","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.796790+0000","last_change":"2023-10-06T10:43:02.796068+0000","last_active":"2023-10-06T10:43:03.796790+0000","last_peered":"2023-10-06T10:43:03.796790+0000","last_clean":"2023-10-06T10:43:03.796790+0000","last_became_active":"2023-10-06T10:43:02.795811+0000","last_became_peered":"2023-10-06T10:43:02.795811+0000","last_unstale":"2023-10-06T10:43:03.796790+0000","last_undegraded":"2023-10-06T10:43:03.796790+0000","last_fullsized":"2023-10-06T10:43:03.796790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:17:20.629917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.343249+0000","last_change":"2023-10-06T10:43:03.343249+0000","last_active":"2023-10-06T10:43:03.343249+0000","last_peered":"2023-10-06T10:43:03.343249+0000","last_clean":"2023-10-06T10:43:03.343249+0000","last_became_active":"2023-10-06T10:43:03.342470+0000","last_became_peered":"2023-10-06T10:43:03.342470+0000","last_unstale":"2023-10-06T10:43:03.343249+0000","last_undegraded":"2023-10-06T10:43:03.343249+0000","last_fullsized":"2023-10-06T10:43:03.343249+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:50.399077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"77'2","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.796130+0000","last_change":"2023-10-06T10:43:02.799931+0000","last_active":"2023-10-06T10:43:03.796130+0000","last_peered":"2023-10-06T10:43:03.796130+0000","last_clean":"2023-10-06T10:43:03.796130+0000","last_became_active":"2023-10-06T10:43:02.799668+0000","last_became_peered":"2023-10-06T10:43:02.799668+0000","last_unstale":"2023-10-06T10:43:03.796130+0000","last_undegraded":"2023-10-06T10:43:03.796130+0000","last_fullsized":"2023-10-06T10:43:03.796130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:18:15.358712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.351186+0000","last_change":"2023-10-06T10:43:03.351186+0000","last_active":"2023-10-06T10:43:03.351186+0000","last_peered":"2023-10-06T10:43:03.351186+0000","last_clean":"2023-10-06T10:43:03.351186+0000","last_became_active":"2023-10-06T10:43:03.351008+0000","last_became_peered":"2023-10-06T10:43:03.351008+0000","last_unstale":"2023-10-06T10:43:03.351186+0000","last_undegraded":"2023-10-06T10:43:03.351186+0000","last_fullsized":"2023-10-06T10:43:03.351186+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:42:16.964813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.809167+0000","last_change":"2023-10-06T10:43:02.804748+0000","last_active":"2023-10-06T10:43:02.809167+0000","last_peered":"2023-10-06T10:43:02.809167+0000","last_clean":"2023-10-06T10:43:02.809167+0000","last_became_active":"2023-10-06T10:43:02.804502+0000","last_became_peered":"2023-10-06T10:43:02.804502+0000","last_unstale":"2023-10-06T10:43:02.809167+0000","last_undegraded":"2023-10-06T10:43:02.809167+0000","last_fullsized":"2023-10-06T10:43:02.809167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:21:04.019415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.342222+0000","last_change":"2023-10-06T10:43:03.342222+0000","last_active":"2023-10-06T10:43:03.342222+0000","last_peered":"2023-10-06T10:43:03.342222+0000","last_clean":"2023-10-06T10:43:03.342222+0000","last_became_active":"2023-10-06T10:43:03.341949+0000","last_became_peered":"2023-10-06T10:43:03.341949+0000","last_unstale":"2023-10-06T10:43:03.342222+0000","last_undegraded":"2023-10-06T10:43:03.342222+0000","last_fullsized":"2023-10-06T10:43:03.342222+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:35:01.365904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.807039+0000","last_change":"2023-10-06T10:43:02.803575+0000","last_active":"2023-10-06T10:43:02.807039+0000","last_peered":"2023-10-06T10:43:02.807039+0000","last_clean":"2023-10-06T10:43:02.807039+0000","last_became_active":"2023-10-06T10:43:02.803253+0000","last_became_peered":"2023-10-06T10:43:02.803253+0000","last_unstale":"2023-10-06T10:43:02.807039+0000","last_undegraded":"2023-10-06T10:43:02.807039+0000","last_fullsized":"2023-10-06T10:43:02.807039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:14.438218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.347269+0000","last_change":"2023-10-06T10:43:03.347269+0000","last_active":"2023-10-06T10:43:03.347269+0000","last_peered":"2023-10-06T10:43:03.347269+0000","last_clean":"2023-10-06T10:43:03.347269+0000","last_became_active":"2023-10-06T10:43:03.346944+0000","last_became_peered":"2023-10-06T10:43:03.346944+0000","last_unstale":"2023-10-06T10:43:03.347269+0000","last_undegraded":"2023-10-06T10:43:03.347269+0000","last_fullsized":"2023-10-06T10:43:03.347269+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:18.479800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.807064+0000","last_change":"2023-10-06T10:43:02.801869+0000","last_active":"2023-10-06T10:43:02.807064+0000","last_peered":"2023-10-06T10:43:02.807064+0000","last_clean":"2023-10-06T10:43:02.807064+0000","last_became_active":"2023-10-06T10:43:02.801599+0000","last_became_peered":"2023-10-06T10:43:02.801599+0000","last_unstale":"2023-10-06T10:43:02.807064+0000","last_undegraded":"2023-10-06T10:43:02.807064+0000","last_fullsized":"2023-10-06T10:43:02.807064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:37:25.769419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.344505+0000","last_change":"2023-10-06T10:43:03.344505+0000","last_active":"2023-10-06T10:43:03.344505+0000","last_peered":"2023-10-06T10:43:03.344505+0000","last_clean":"2023-10-06T10:43:03.344505+0000","last_became_active":"2023-10-06T10:43:03.344273+0000","last_became_peered":"2023-10-06T10:43:03.344273+0000","last_unstale":"2023-10-06T10:43:03.344505+0000","last_undegraded":"2023-10-06T10:43:03.344505+0000","last_fullsized":"2023-10-06T10:43:03.344505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:06:33.182560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.800971+0000","last_change":"2023-10-06T10:43:02.799176+0000","last_active":"2023-10-06T10:43:02.800971+0000","last_peered":"2023-10-06T10:43:02.800971+0000","last_clean":"2023-10-06T10:43:02.800971+0000","last_became_active":"2023-10-06T10:43:02.798691+0000","last_became_peered":"2023-10-06T10:43:02.798691+0000","last_unstale":"2023-10-06T10:43:02.800971+0000","last_undegraded":"2023-10-06T10:43:02.800971+0000","last_fullsized":"2023-10-06T10:43:02.800971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:35:00.361047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.345413+0000","last_change":"2023-10-06T10:43:03.345413+0000","last_active":"2023-10-06T10:43:03.345413+0000","last_peered":"2023-10-06T10:43:03.345413+0000","last_clean":"2023-10-06T10:43:03.345413+0000","last_became_active":"2023-10-06T10:43:03.345109+0000","last_became_peered":"2023-10-06T10:43:03.345109+0000","last_unstale":"2023-10-06T10:43:03.345413+0000","last_undegraded":"2023-10-06T10:43:03.345413+0000","last_fullsized":"2023-10-06T10:43:03.345413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:21:55.222924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.803859+0000","last_change":"2023-10-06T10:43:02.798298+0000","last_active":"2023-10-06T10:43:02.803859+0000","last_peered":"2023-10-06T10:43:02.803859+0000","last_clean":"2023-10-06T10:43:02.803859+0000","last_became_active":"2023-10-06T10:43:02.798024+0000","last_became_peered":"2023-10-06T10:43:02.798024+0000","last_unstale":"2023-10-06T10:43:02.803859+0000","last_undegraded":"2023-10-06T10:43:02.803859+0000","last_fullsized":"2023-10-06T10:43:02.803859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:19:16.826755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.352352+0000","last_change":"2023-10-06T10:43:03.352352+0000","last_active":"2023-10-06T10:43:03.352352+0000","last_peered":"2023-10-06T10:43:03.352352+0000","last_clean":"2023-10-06T10:43:03.352352+0000","last_became_active":"2023-10-06T10:43:03.352060+0000","last_became_peered":"2023-10-06T10:43:03.352060+0000","last_unstale":"2023-10-06T10:43:03.352352+0000","last_undegraded":"2023-10-06T10:43:03.352352+0000","last_fullsized":"2023-10-06T10:43:03.352352+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:06:33.182560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.806220+0000","last_change":"2023-10-06T10:43:02.800357+0000","last_active":"2023-10-06T10:43:02.806220+0000","last_peered":"2023-10-06T10:43:02.806220+0000","last_clean":"2023-10-06T10:43:02.806220+0000","last_became_active":"2023-10-06T10:43:02.798261+0000","last_became_peered":"2023-10-06T10:43:02.798261+0000","last_unstale":"2023-10-06T10:43:02.806220+0000","last_undegraded":"2023-10-06T10:43:02.806220+0000","last_fullsized":"2023-10-06T10:43:02.806220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:06.710477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.341047+0000","last_change":"2023-10-06T10:43:03.341047+0000","last_active":"2023-10-06T10:43:03.341047+0000","last_peered":"2023-10-06T10:43:03.341047+0000","last_clean":"2023-10-06T10:43:03.341047+0000","last_became_active":"2023-10-06T10:43:03.340788+0000","last_became_peered":"2023-10-06T10:43:03.340788+0000","last_unstale":"2023-10-06T10:43:03.341047+0000","last_undegraded":"2023-10-06T10:43:03.341047+0000","last_fullsized":"2023-10-06T10:43:03.341047+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:59:16.697883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.806709+0000","last_change":"2023-10-06T10:43:02.802081+0000","last_active":"2023-10-06T10:43:02.806709+0000","last_peered":"2023-10-06T10:43:02.806709+0000","last_clean":"2023-10-06T10:43:02.806709+0000","last_became_active":"2023-10-06T10:43:02.801790+0000","last_became_peered":"2023-10-06T10:43:02.801790+0000","last_unstale":"2023-10-06T10:43:02.806709+0000","last_undegraded":"2023-10-06T10:43:02.806709+0000","last_fullsized":"2023-10-06T10:43:02.806709+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:04.689399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.346663+0000","last_change":"2023-10-06T10:43:03.346663+0000","last_active":"2023-10-06T10:43:03.346663+0000","last_peered":"2023-10-06T10:43:03.346663+0000","last_clean":"2023-10-06T10:43:03.346663+0000","last_became_active":"2023-10-06T10:43:03.346401+0000","last_became_peered":"2023-10-06T10:43:03.346401+0000","last_unstale":"2023-10-06T10:43:03.346663+0000","last_undegraded":"2023-10-06T10:43:03.346663+0000","last_fullsized":"2023-10-06T10:43:03.346663+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:42:16.964813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.800971+0000","last_change":"2023-10-06T10:43:02.797198+0000","last_active":"2023-10-06T10:43:02.800971+0000","last_peered":"2023-10-06T10:43:02.800971+0000","last_clean":"2023-10-06T10:43:02.800971+0000","last_became_active":"2023-10-06T10:43:02.796953+0000","last_became_peered":"2023-10-06T10:43:02.796953+0000","last_unstale":"2023-10-06T10:43:02.800971+0000","last_undegraded":"2023-10-06T10:43:02.800971+0000","last_fullsized":"2023-10-06T10:43:02.800971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:18:39.029604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.794375+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:43:02.794375+0000","last_peered":"2023-10-06T10:43:02.794375+0000","last_clean":"2023-10-06T10:43:02.794375+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:43:02.794375+0000","last_undegraded":"2023-10-06T10:43:02.794375+0000","last_fullsized":"2023-10-06T10:43:02.794375+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.341614+0000","last_change":"2023-10-06T10:43:03.341614+0000","last_active":"2023-10-06T10:43:03.341614+0000","last_peered":"2023-10-06T10:43:03.341614+0000","last_clean":"2023-10-06T10:43:03.341614+0000","last_became_active":"2023-10-06T10:43:03.341387+0000","last_became_peered":"2023-10-06T10:43:03.341387+0000","last_unstale":"2023-10-06T10:43:03.341614+0000","last_undegraded":"2023-10-06T10:43:03.341614+0000","last_fullsized":"2023-10-06T10:43:03.341614+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:59.776158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.809006+0000","last_change":"2023-10-06T10:43:02.807812+0000","last_active":"2023-10-06T10:43:02.809006+0000","last_peered":"2023-10-06T10:43:02.809006+0000","last_clean":"2023-10-06T10:43:02.809006+0000","last_became_active":"2023-10-06T10:43:02.807600+0000","last_became_peered":"2023-10-06T10:43:02.807600+0000","last_unstale":"2023-10-06T10:43:02.809006+0000","last_undegraded":"2023-10-06T10:43:02.809006+0000","last_fullsized":"2023-10-06T10:43:02.809006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:50:29.907645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":58,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.788550+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:43:02.788550+0000","last_peered":"2023-10-06T10:43:02.788550+0000","last_clean":"2023-10-06T10:43:02.788550+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:43:02.788550+0000","last_undegraded":"2023-10-06T10:43:02.788550+0000","last_fullsized":"2023-10-06T10:43:02.788550+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.348702+0000","last_change":"2023-10-06T10:43:03.348702+0000","last_active":"2023-10-06T10:43:03.348702+0000","last_peered":"2023-10-06T10:43:03.348702+0000","last_clean":"2023-10-06T10:43:03.348702+0000","last_became_active":"2023-10-06T10:43:03.348301+0000","last_became_peered":"2023-10-06T10:43:03.348301+0000","last_unstale":"2023-10-06T10:43:03.348702+0000","last_undegraded":"2023-10-06T10:43:03.348702+0000","last_fullsized":"2023-10-06T10:43:03.348702+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:23.657941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"77'2","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.796269+0000","last_change":"2023-10-06T10:43:02.797219+0000","last_active":"2023-10-06T10:43:03.796269+0000","last_peered":"2023-10-06T10:43:03.796269+0000","last_clean":"2023-10-06T10:43:03.796269+0000","last_became_active":"2023-10-06T10:43:02.795540+0000","last_became_peered":"2023-10-06T10:43:02.795540+0000","last_unstale":"2023-10-06T10:43:03.796269+0000","last_undegraded":"2023-10-06T10:43:03.796269+0000","last_fullsized":"2023-10-06T10:43:03.796269+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:12:06.361966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":64,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.802523+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:43:02.802523+0000","last_peered":"2023-10-06T10:43:02.802523+0000","last_clean":"2023-10-06T10:43:02.802523+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:43:02.802523+0000","last_undegraded":"2023-10-06T10:43:02.802523+0000","last_fullsized":"2023-10-06T10:43:02.802523+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":205,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.795727+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:43:02.795727+0000","last_peered":"2023-10-06T10:43:02.795727+0000","last_clean":"2023-10-06T10:43:02.795727+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:43:02.795727+0000","last_undegraded":"2023-10-06T10:43:02.795727+0000","last_fullsized":"2023-10-06T10:43:02.795727+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.350663+0000","last_change":"2023-10-06T10:43:03.350663+0000","last_active":"2023-10-06T10:43:03.350663+0000","last_peered":"2023-10-06T10:43:03.350663+0000","last_clean":"2023-10-06T10:43:03.350663+0000","last_became_active":"2023-10-06T10:43:03.350397+0000","last_became_peered":"2023-10-06T10:43:03.350397+0000","last_unstale":"2023-10-06T10:43:03.350663+0000","last_undegraded":"2023-10-06T10:43:03.350663+0000","last_fullsized":"2023-10-06T10:43:03.350663+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:19.454862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"77'1","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.338350+0000","last_change":"2023-10-06T10:43:02.807119+0000","last_active":"2023-10-06T10:43:03.338350+0000","last_peered":"2023-10-06T10:43:03.338350+0000","last_clean":"2023-10-06T10:43:03.338350+0000","last_became_active":"2023-10-06T10:43:02.806656+0000","last_became_peered":"2023-10-06T10:43:02.806656+0000","last_unstale":"2023-10-06T10:43:03.338350+0000","last_undegraded":"2023-10-06T10:43:03.338350+0000","last_fullsized":"2023-10-06T10:43:03.338350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:34.441224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.795651+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:43:02.795651+0000","last_peered":"2023-10-06T10:43:02.795651+0000","last_clean":"2023-10-06T10:43:02.795651+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:43:02.795651+0000","last_undegraded":"2023-10-06T10:43:02.795651+0000","last_fullsized":"2023-10-06T10:43:02.795651+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.341684+0000","last_change":"2023-10-06T10:43:03.341684+0000","last_active":"2023-10-06T10:43:03.341684+0000","last_peered":"2023-10-06T10:43:03.341684+0000","last_clean":"2023-10-06T10:43:03.341684+0000","last_became_active":"2023-10-06T10:43:03.341430+0000","last_became_peered":"2023-10-06T10:43:03.341430+0000","last_unstale":"2023-10-06T10:43:03.341684+0000","last_undegraded":"2023-10-06T10:43:03.341684+0000","last_fullsized":"2023-10-06T10:43:03.341684+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:43.580246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"77'1","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.343264+0000","last_change":"2023-10-06T10:43:02.805773+0000","last_active":"2023-10-06T10:43:03.343264+0000","last_peered":"2023-10-06T10:43:03.343264+0000","last_clean":"2023-10-06T10:43:03.343264+0000","last_became_active":"2023-10-06T10:43:02.805528+0000","last_became_peered":"2023-10-06T10:43:02.805528+0000","last_unstale":"2023-10-06T10:43:03.343264+0000","last_undegraded":"2023-10-06T10:43:03.343264+0000","last_fullsized":"2023-10-06T10:43:03.343264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:55:52.806135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.795638+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:43:02.795638+0000","last_peered":"2023-10-06T10:43:02.795638+0000","last_clean":"2023-10-06T10:43:02.795638+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:43:02.795638+0000","last_undegraded":"2023-10-06T10:43:02.795638+0000","last_fullsized":"2023-10-06T10:43:02.795638+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.340724+0000","last_change":"2023-10-06T10:43:03.340724+0000","last_active":"2023-10-06T10:43:03.340724+0000","last_peered":"2023-10-06T10:43:03.340724+0000","last_clean":"2023-10-06T10:43:03.340724+0000","last_became_active":"2023-10-06T10:43:03.340443+0000","last_became_peered":"2023-10-06T10:43:03.340443+0000","last_unstale":"2023-10-06T10:43:03.340724+0000","last_undegraded":"2023-10-06T10:43:03.340724+0000","last_fullsized":"2023-10-06T10:43:03.340724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:21:12.430514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.800213+0000","last_change":"2023-10-06T10:43:02.797717+0000","last_active":"2023-10-06T10:43:02.800213+0000","last_peered":"2023-10-06T10:43:02.800213+0000","last_clean":"2023-10-06T10:43:02.800213+0000","last_became_active":"2023-10-06T10:43:02.796852+0000","last_became_peered":"2023-10-06T10:43:02.796852+0000","last_unstale":"2023-10-06T10:43:02.800213+0000","last_undegraded":"2023-10-06T10:43:02.800213+0000","last_fullsized":"2023-10-06T10:43:02.800213+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:22.653083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.791624+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:43:02.791624+0000","last_peered":"2023-10-06T10:43:02.791624+0000","last_clean":"2023-10-06T10:43:02.791624+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:43:02.791624+0000","last_undegraded":"2023-10-06T10:43:02.791624+0000","last_fullsized":"2023-10-06T10:43:02.791624+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.344137+0000","last_change":"2023-10-06T10:43:03.344137+0000","last_active":"2023-10-06T10:43:03.344137+0000","last_peered":"2023-10-06T10:43:03.344137+0000","last_clean":"2023-10-06T10:43:03.344137+0000","last_became_active":"2023-10-06T10:43:03.343862+0000","last_became_peered":"2023-10-06T10:43:03.343862+0000","last_unstale":"2023-10-06T10:43:03.344137+0000","last_undegraded":"2023-10-06T10:43:03.344137+0000","last_fullsized":"2023-10-06T10:43:03.344137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:27.895051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"77'2","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.796664+0000","last_change":"2023-10-06T10:43:02.796564+0000","last_active":"2023-10-06T10:43:03.796664+0000","last_peered":"2023-10-06T10:43:03.796664+0000","last_clean":"2023-10-06T10:43:03.796664+0000","last_became_active":"2023-10-06T10:43:02.796316+0000","last_became_peered":"2023-10-06T10:43:02.796316+0000","last_unstale":"2023-10-06T10:43:03.796664+0000","last_undegraded":"2023-10-06T10:43:03.796664+0000","last_fullsized":"2023-10-06T10:43:03.796664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:52:25.427534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.791418+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:43:02.791418+0000","last_peered":"2023-10-06T10:43:02.791418+0000","last_clean":"2023-10-06T10:43:02.791418+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:43:02.791418+0000","last_undegraded":"2023-10-06T10:43:02.791418+0000","last_fullsized":"2023-10-06T10:43:02.791418+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.344483+0000","last_change":"2023-10-06T10:43:03.344483+0000","last_active":"2023-10-06T10:43:03.344483+0000","last_peered":"2023-10-06T10:43:03.344483+0000","last_clean":"2023-10-06T10:43:03.344483+0000","last_became_active":"2023-10-06T10:43:03.344193+0000","last_became_peered":"2023-10-06T10:43:03.344193+0000","last_unstale":"2023-10-06T10:43:03.344483+0000","last_undegraded":"2023-10-06T10:43:03.344483+0000","last_fullsized":"2023-10-06T10:43:03.344483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:19.454862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.799554+0000","last_change":"2023-10-06T10:43:02.795287+0000","last_active":"2023-10-06T10:43:02.799554+0000","last_peered":"2023-10-06T10:43:02.799554+0000","last_clean":"2023-10-06T10:43:02.799554+0000","last_became_active":"2023-10-06T10:43:02.794762+0000","last_became_peered":"2023-10-06T10:43:02.794762+0000","last_unstale":"2023-10-06T10:43:02.799554+0000","last_undegraded":"2023-10-06T10:43:02.799554+0000","last_fullsized":"2023-10-06T10:43:02.799554+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:20:22.057282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.798319+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:43:02.798319+0000","last_peered":"2023-10-06T10:43:02.798319+0000","last_clean":"2023-10-06T10:43:02.798319+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:43:02.798319+0000","last_undegraded":"2023-10-06T10:43:02.798319+0000","last_fullsized":"2023-10-06T10:43:02.798319+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298792,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073048,"statfs":{"total":96636764160,"available":96330801152,"internally_reserved":0,"allocated":1540096,"data_stored":1040246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.5,"15min":0.48399999999999999},"min":{"1min":0.29799999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.173,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.498},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.503,"15min":0.50900000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.65400000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.70899999999999996}]},{"osd":1,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.47299999999999998,"15min":0.45900000000000002},"min":{"1min":0.34300000000000003,"5min":0.252,"15min":0.252},"max":{"1min":0.64100000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.61699999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.66500000000000004}]},{"osd":2,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52700000000000002,"15min":0.50900000000000001},"min":{"1min":0.38700000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.503,"15min":0.49399999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.63800000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.68300000000000005}]},{"osd":3,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52900000000000003,"15min":0.50900000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.68400000000000005,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55100000000000005,"15min":0.54300000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.67100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.443,"15min":0.46899999999999997},"min":{"1min":0.27300000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.499,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.38},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.48999999999999999,"15min":0.505},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64100000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.311}]},{"osd":5,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.47099999999999997,"15min":0.48299999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.59299999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.52400000000000002,"15min":0.54600000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.61699999999999999,"5min":1.052,"15min":1.052},"last":0.33800000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54900000000000004,"15min":0.57099999999999995},"min":{"1min":0.35399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.63300000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.40600000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.42999999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298204,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073636,"statfs":{"total":96636764160,"available":96331403264,"internally_reserved":0,"allocated":937984,"data_stored":448455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.29299999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66100000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.48199999999999998,"15min":0.442},"min":{"1min":0.32500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.50900000000000001}]},{"osd":1,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.47899999999999998,"15min":0.44600000000000001},"min":{"1min":0.35699999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.67400000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.51400000000000001,"15min":0.47899999999999998},"min":{"1min":0.40100000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.58599999999999997}]},{"osd":2,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.497,"15min":0.47399999999999998},"min":{"1min":0.32500000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.72999999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.50900000000000001,"15min":0.47499999999999998},"min":{"1min":0.39200000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65300000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.54200000000000004}]},{"osd":3,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.51900000000000002,"15min":0.495},"min":{"1min":0.41199999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70599999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52800000000000002,"15min":0.51000000000000001},"min":{"1min":0.41099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64900000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.52700000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.47499999999999998,"15min":0.47699999999999998},"min":{"1min":0.35199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67400000000000004,"5min":0.745,"15min":0.745},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.48799999999999999,"15min":0.48599999999999999},"min":{"1min":0.36499999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.40000000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.48899999999999999,"15min":0.48499999999999999},"min":{"1min":0.34399999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53000000000000003,"15min":0.53600000000000003},"min":{"1min":0.38500000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.71399999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.495}]},{"osd":7,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.36299999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.64900000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54900000000000004,"15min":0.53900000000000003},"min":{"1min":0.40400000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.56999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298800,"kb_used_data":1512,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073040,"statfs":{"total":96636764160,"available":96330792960,"internally_reserved":0,"allocated":1548288,"data_stored":1040370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:42:20 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33200000000000002,"15min":0.32900000000000001},"min":{"1min":0.21099999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.49399999999999999,"5min":0.505,"15min":0.505},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.35199999999999998,"15min":0.34699999999999998},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.502,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.34599999999999997}]},{"osd":2,"last update":"Fri Oct 6 10:42:20 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.33200000000000002,"15min":0.32100000000000001},"min":{"1min":0.23200000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.47799999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.254},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.375,"15min":0.36499999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.52100000000000002,"5min":0.62,"15min":0.62},"last":0.33600000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:42:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.39200000000000002,"15min":0.379},"min":{"1min":0.29499999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58899999999999997,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.317},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.378,"15min":0.374},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.54300000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.27400000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:42:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.40799999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70399999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.441,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.754,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.45500000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:42:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.57199999999999995},"min":{"1min":0.41999999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77900000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56399999999999995,"15min":0.54600000000000004},"min":{"1min":0.42999999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79700000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.433}]},{"osd":6,"last update":"Fri Oct 6 10:42:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.44,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81200000000000006,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.40000000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77000000000000002,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.46500000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:42:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.57799999999999996},"min":{"1min":0.41099999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.77300000000000002,"5min":0.996,"15min":0.996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.47099999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84999999999999998,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.50600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298208,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073632,"statfs":{"total":96636764160,"available":96331399168,"internally_reserved":0,"allocated":942080,"data_stored":449808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.32300000000000001,"15min":0.318},"min":{"1min":0.218,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.39900000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35899999999999999,"15min":0.34399999999999997},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.44600000000000001,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.26700000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.311,"15min":0.30499999999999999},"min":{"1min":0.23200000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.46000000000000002,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.219},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34200000000000003,"15min":0.32800000000000001},"min":{"1min":0.253,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.52800000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.30399999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.34799999999999998,"15min":0.34200000000000003},"min":{"1min":0.27300000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.42399999999999999,"5min":0.622,"15min":0.622},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35699999999999998,"15min":0.34999999999999998},"min":{"1min":0.27200000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.52700000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.27900000000000003}]},{"osd":4,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.42399999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76200000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.53500000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.70799999999999996,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.78100000000000003}]},{"osd":5,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.42199999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.748,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70999999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.64800000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57499999999999996,"15min":0.58399999999999996},"min":{"1min":0.45400000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.71699999999999997,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.68999999999999995,"5min":1.075,"15min":1.075},"last":0.58899999999999997}]},{"osd":7,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.434,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72199999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.45900000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72199999999999998,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.751}]}]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298212,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073628,"statfs":{"total":96636764160,"available":96331395072,"internally_reserved":0,"allocated":946176,"data_stored":450362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.311,"15min":0.31},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.38900000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.33800000000000002,"15min":0.32100000000000001},"min":{"1min":0.23699999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.441,"5min":0.622,"15min":0.622},"last":0.35599999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.33000000000000002,"15min":0.32400000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.46700000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.36499999999999999,"15min":0.36399999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47899999999999998,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.38700000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.35999999999999999,"15min":0.34799999999999998},"min":{"1min":0.25700000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.49299999999999999,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.311},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36599999999999999,"15min":0.35799999999999998},"min":{"1min":0.27700000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.48399999999999999,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.33800000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55200000000000005,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.77300000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.432,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76300000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.5}]},{"osd":5,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.42099999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.79000000000000004,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71699999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.44700000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.746,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.45600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80400000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.434}]},{"osd":7,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59599999999999997,"15min":0.60999999999999999},"min":{"1min":0.39600000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72899999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61199999999999999,"15min":0.623},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80000000000000004,"5min":0.874,"15min":0.874},"last":0.49099999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574910,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298220,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073620,"statfs":{"total":96636764160,"available":96331386880,"internally_reserved":0,"allocated":954368,"data_stored":451344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.29099999999999998,"5min":0.374,"15min":0.435},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.42799999999999999,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.35199999999999998,"15min":0.35899999999999999},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.53200000000000003,"5min":1.218,"15min":1.218},"last":0.39100000000000001}]},{"osd":1,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33600000000000002,"15min":0.33800000000000002},"min":{"1min":0.23100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.44700000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.41799999999999998,"15min":0.47399999999999998},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.51300000000000001,"5min":3.359,"15min":3.359},"last":0.28000000000000003}]},{"osd":2,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.37,"15min":0.379},"min":{"1min":0.255,"5min":0.215,"15min":0.215},"max":{"1min":0.45900000000000002,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.50700000000000001,"15min":0.63},"min":{"1min":0.27200000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.51200000000000001,"5min":7.069,"15min":7.069},"last":0.376}]},{"osd":4,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57299999999999995,"15min":0.58499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78200000000000003,"5min":1.165,"15min":1.165},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.40100000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.79100000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.70399999999999996}]},{"osd":5,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.64000000000000001,"15min":0.69399999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84499999999999997,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58699999999999997,"15min":0.60399999999999998},"min":{"1min":0.39500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83299999999999996,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.80000000000000004}]},{"osd":6,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60599999999999998,"15min":0.61399999999999999},"min":{"1min":0.41699999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.77600000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.67300000000000004,"15min":0.72399999999999998},"min":{"1min":0.442,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80100000000000005,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.81799999999999995}]},{"osd":7,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.40899999999999997,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86699999999999999,"5min":1.137,"15min":1.137},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60699999999999998,"15min":0.61099999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82499999999999996,"5min":1.014,"15min":1.014},"last":0.73499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298208,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073632,"statfs":{"total":96636764160,"available":96331399168,"internally_reserved":0,"allocated":942080,"data_stored":449808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.496,"15min":0.48599999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.74299999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.504},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51600000000000001,"15min":0.502},"min":{"1min":0.40600000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71399999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.35299999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50600000000000001,"15min":0.5},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68799999999999994,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.52300000000000002,"15min":0.53600000000000003},"min":{"1min":0.34499999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70299999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.56100000000000005}]},{"osd":2,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68999999999999995,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.52000000000000002},"min":{"1min":0.40300000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.46100000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53800000000000003,"15min":0.52000000000000002},"min":{"1min":0.36599999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.38200000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72699999999999998,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.53500000000000003}]},{"osd":5,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47199999999999998,"15min":0.5},"min":{"1min":0.28899999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77800000000000002,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49199999999999999,"15min":0.501},"min":{"1min":0.32000000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.69599999999999995,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.52100000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51900000000000002,"15min":0.54100000000000004},"min":{"1min":0.35199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55800000000000005,"15min":0.57599999999999996},"min":{"1min":0.36199999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.88,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.59699999999999998}]},{"osd":7,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.57499999999999996},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57199999999999995,"15min":0.59599999999999997},"min":{"1min":0.38600000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.749,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.61099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073616,"statfs":{"total":96636764160,"available":96331382784,"internally_reserved":0,"allocated":958464,"data_stored":451366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.49399999999999999,"15min":0.46999999999999997},"min":{"1min":0.28999999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68999999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.52600000000000002,"15min":0.53000000000000003},"min":{"1min":0.34499999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.73399999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.47199999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.317,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70399999999999996,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.622},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52900000000000003,"15min":0.51800000000000002},"min":{"1min":0.34699999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.79900000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.504}]},{"osd":2,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52500000000000002,"15min":0.504},"min":{"1min":0.308,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74299999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56100000000000005,"15min":0.56499999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52700000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.42099999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.77800000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53600000000000003,"15min":0.55400000000000005},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69299999999999995,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.58299999999999996}]},{"osd":4,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47499999999999998,"15min":0.48499999999999999},"min":{"1min":0.32600000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.75600000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.498,"15min":0.50600000000000001},"min":{"1min":0.31900000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.62,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.63800000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.501,"15min":0.51700000000000002},"min":{"1min":0.32600000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.67900000000000005,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.65400000000000003}]},{"osd":7,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56100000000000005,"15min":0.57899999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70199999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55400000000000005,"15min":0.55700000000000005},"min":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.747,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.69999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:43:07.864 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:43:08.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.010+0000 7f69b5914700 1 Processor -- start 2023-10-06T10:43:08.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.011+0000 7f69b5914700 1 -- start start 2023-10-06T10:43:08.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.011+0000 7f69b5914700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f69b015db10 0x7f69b00cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:08.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.011+0000 7f69b5914700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f69b00ad5e0 0x7f69b00ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:08.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.011+0000 7f69b5914700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b00ca600 con 0x7f69b00ad210 2023-10-06T10:43:08.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.011+0000 7f69b5914700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b00cecf0 con 0x7f69b00ad5e0 2023-10-06T10:43:08.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.011+0000 7f69b5914700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b00cefd0 con 0x7f69b015db10 2023-10-06T10:43:08.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.011+0000 7f69af7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f69b00ad5e0 0x7f69b00ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:08.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.011+0000 7f69af7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f69b00ad5e0 0x7f69b00ca0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33910/0 (socket says 172.21.15.92:33910) 2023-10-06T10:43:08.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.012+0000 7f69af7fe700 1 -- 172.21.15.92:0/3189737197 learned_addr learned my addr 172.21.15.92:0/3189737197 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:08.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.012+0000 7f69a67fc700 1 --2- 172.21.15.92:0/3189737197 >> v2:172.21.15.124:3300/0 conn(0x7f69b015db10 0x7f69b00cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:08.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.012+0000 7f69af7fe700 1 -- 172.21.15.92:0/3189737197 >> v2:172.21.15.124:3300/0 conn(0x7f69b015db10 msgr2=0x7f69b00cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:08.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.012+0000 7f69af7fe700 1 --2- 172.21.15.92:0/3189737197 >> v2:172.21.15.124:3300/0 conn(0x7f69b015db10 0x7f69b00cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:08.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.012+0000 7f69af7fe700 1 -- 172.21.15.92:0/3189737197 >> v1:172.21.15.124:6789/0 conn(0x7f69b00ad210 legacy=0x7f69b015d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:08.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.012+0000 7f69af7fe700 1 -- 172.21.15.92:0/3189737197 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b00cca90 con 0x7f69b00ad5e0 2023-10-06T10:43:08.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.012+0000 7f69af7fe700 1 --2- 172.21.15.92:0/3189737197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b00ad5e0 0x7f69b00ca0c0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f69b006ed40 tx=0x7f69a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6cde958d64eb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:08.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.012+0000 7f69ae7fc700 1 -- 172.21.15.92:0/3189737197 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a800e070 con 0x7f69b00ad5e0 2023-10-06T10:43:08.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.012+0000 7f69ae7fc700 1 -- 172.21.15.92:0/3189737197 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a800ba10 con 0x7f69b00ad5e0 2023-10-06T10:43:08.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.012+0000 7f69ae7fc700 1 -- 172.21.15.92:0/3189737197 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a80092e0 con 0x7f69b00ad5e0 2023-10-06T10:43:08.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.013+0000 7f69b5914700 1 -- 172.21.15.92:0/3189737197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b00ad5e0 msgr2=0x7f69b00ca0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:08.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.013+0000 7f69b5914700 1 --2- 172.21.15.92:0/3189737197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b00ad5e0 0x7f69b00ca0c0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f69b006ed40 tx=0x7f69a8009840 comp rx=0 tx=0).stop 2023-10-06T10:43:08.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.013+0000 7f69b5914700 1 -- 172.21.15.92:0/3189737197 shutdown_connections 2023-10-06T10:43:08.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.013+0000 7f69b5914700 1 --2- 172.21.15.92:0/3189737197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b00ad5e0 0x7f69b00ca0c0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:08.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.013+0000 7f69b5914700 1 --2- 172.21.15.92:0/3189737197 >> v2:172.21.15.124:3300/0 conn(0x7f69b015db10 0x7f69b00cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:08.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.013+0000 7f69b5914700 1 -- 172.21.15.92:0/3189737197 >> 172.21.15.92:0/3189737197 conn(0x7f69b0067d90 msgr2=0x7f69b00ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:08.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.013+0000 7f69b5914700 1 -- 172.21.15.92:0/3189737197 shutdown_connections 2023-10-06T10:43:08.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.013+0000 7f69b5914700 1 -- 172.21.15.92:0/3189737197 wait complete. 2023-10-06T10:43:08.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.014+0000 7f69b5914700 1 Processor -- start 2023-10-06T10:43:08.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.014+0000 7f69b5914700 1 -- start start 2023-10-06T10:43:08.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.015+0000 7f69b5914700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69b00ad210 0x7f69b00c2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:08.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.015+0000 7f69b5914700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b00ad5e0 0x7f69b015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:08.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.015+0000 7f69b5914700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69b015db10 0x7f69b006c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:08.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.015+0000 7f69b5914700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b00b19b0 con 0x7f69b00ad5e0 2023-10-06T10:43:08.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.015+0000 7f69b5914700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b00b40b0 con 0x7f69b015db10 2023-10-06T10:43:08.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.015+0000 7f69b5914700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b00b10c0 con 0x7f69b00ad210 2023-10-06T10:43:08.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.015+0000 7f69a67fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b00ad5e0 0x7f69b015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:08.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.015+0000 7f69a67fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b00ad5e0 0x7f69b015ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33926/0 (socket says 172.21.15.92:33926) 2023-10-06T10:43:08.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.015+0000 7f69a67fc700 1 -- 172.21.15.92:0/4177836090 learned_addr learned my addr 172.21.15.92:0/4177836090 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:08.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.015+0000 7f69aeffd700 1 --2- 172.21.15.92:0/4177836090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69b00ad210 0x7f69b00c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:08.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.015+0000 7f69af7fe700 1 --2- 172.21.15.92:0/4177836090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69b015db10 0x7f69b006c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:08.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.015+0000 7f69a67fc700 1 -- 172.21.15.92:0/4177836090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69b00ad210 msgr2=0x7f69b00c2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:08.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.015+0000 7f69a67fc700 1 --2- 172.21.15.92:0/4177836090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69b00ad210 0x7f69b00c2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:08.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.015+0000 7f69a67fc700 1 -- 172.21.15.92:0/4177836090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69b015db10 msgr2=0x7f69b006c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:08.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.015+0000 7f69a67fc700 1 --2- 172.21.15.92:0/4177836090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69b015db10 0x7f69b006c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:08.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.015+0000 7f69a67fc700 1 -- 172.21.15.92:0/4177836090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a8009ae0 con 0x7f69b00ad5e0 2023-10-06T10:43:08.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.015+0000 7f69af7fe700 1 --2- 172.21.15.92:0/4177836090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69b015db10 0x7f69b006c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:08.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.015+0000 7f69a67fc700 1 --2- 172.21.15.92:0/4177836090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b00ad5e0 0x7f69b015ac30 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f699800d860 tx=0x7f6998008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:08.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.015+0000 7f69a77fe700 1 -- 172.21.15.92:0/4177836090 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6998012630 con 0x7f69b00ad5e0 2023-10-06T10:43:08.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.015+0000 7f69a77fe700 1 -- 172.21.15.92:0/4177836090 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6998012790 con 0x7f69b00ad5e0 2023-10-06T10:43:08.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.015+0000 7f69b5914700 1 -- 172.21.15.92:0/4177836090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b00b1360 con 0x7f69b00ad5e0 2023-10-06T10:43:08.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.015+0000 7f69a77fe700 1 -- 172.21.15.92:0/4177836090 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6998012b50 con 0x7f69b00ad5e0 2023-10-06T10:43:08.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.016+0000 7f69b5914700 1 -- 172.21.15.92:0/4177836090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b015b170 con 0x7f69b00ad5e0 2023-10-06T10:43:08.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.016+0000 7f69b5914700 1 -- 172.21.15.92:0/4177836090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b00b1b60 con 0x7f69b00ad5e0 2023-10-06T10:43:08.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.017+0000 7f69a77fe700 1 -- 172.21.15.92:0/4177836090 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f699801d400 con 0x7f69b00ad5e0 2023-10-06T10:43:08.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.017+0000 7f69a77fe700 1 --2- 172.21.15.92:0/4177836090 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6990066f30 0x7f69900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:08.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.017+0000 7f69a77fe700 1 -- 172.21.15.92:0/4177836090 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6998084bf0 con 0x7f69b00ad5e0 2023-10-06T10:43:08.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.017+0000 7f69aeffd700 1 --2- 172.21.15.92:0/4177836090 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6990066f30 0x7f69900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:08.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.018+0000 7f69aeffd700 1 --2- 172.21.15.92:0/4177836090 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6990066f30 0x7f69900693e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f69a00038a0 tx=0x7f69a0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:08.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.021+0000 7f69a77fe700 1 -- 172.21.15.92:0/4177836090 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69b00b1b60 con 0x7f69b00ad5e0 2023-10-06T10:43:08.092 INFO:tasks.ceph.mds.d.smithi124.stderr:2023-10-06T10:43:08.091+0000 7f984e348700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T10:43:08.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.204+0000 7f69b5914700 1 -- 172.21.15.92:0/4177836090 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69b00c2bc0 con 0x7f6990066f30 2023-10-06T10:43:08.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.212+0000 7f69a77fe700 1 -- 172.21.15.92:0/4177836090 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311653 (secure 0 0 0) 0x7f69b00c2bc0 con 0x7f6990066f30 2023-10-06T10:43:08.217 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:08.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.215+0000 7f69b5914700 1 -- 172.21.15.92:0/4177836090 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6990066f30 msgr2=0x7f69900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:08.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.215+0000 7f69b5914700 1 --2- 172.21.15.92:0/4177836090 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6990066f30 0x7f69900693e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f69a00038a0 tx=0x7f69a0009040 comp rx=0 tx=0).stop 2023-10-06T10:43:08.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.215+0000 7f69b5914700 1 -- 172.21.15.92:0/4177836090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b00ad5e0 msgr2=0x7f69b015ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:08.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.215+0000 7f69b5914700 1 --2- 172.21.15.92:0/4177836090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b00ad5e0 0x7f69b015ac30 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f699800d860 tx=0x7f6998008040 comp rx=0 tx=0).stop 2023-10-06T10:43:08.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.215+0000 7f69b5914700 1 -- 172.21.15.92:0/4177836090 shutdown_connections 2023-10-06T10:43:08.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.215+0000 7f69b5914700 1 --2- 172.21.15.92:0/4177836090 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6990066f30 0x7f69900693e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:08.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.215+0000 7f69b5914700 1 --2- 172.21.15.92:0/4177836090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69b015db10 0x7f69b006c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:08.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.215+0000 7f69b5914700 1 --2- 172.21.15.92:0/4177836090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69b00ad210 0x7f69b00c2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:08.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.215+0000 7f69b5914700 1 --2- 172.21.15.92:0/4177836090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b00ad5e0 0x7f69b015ac30 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:08.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.215+0000 7f69b5914700 1 -- 172.21.15.92:0/4177836090 >> 172.21.15.92:0/4177836090 conn(0x7f69b0067d90 msgr2=0x7f69b015ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:08.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.215+0000 7f69b5914700 1 -- 172.21.15.92:0/4177836090 shutdown_connections 2023-10-06T10:43:08.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.215+0000 7f69b5914700 1 -- 172.21.15.92:0/4177836090 wait complete. 2023-10-06T10:43:08.221 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:43:08.229 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2023-10-06T10:43:08.146011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":125,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2386868,"kb_used_data":8564,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752587852,"statfs":{"total":773094113280,"available":770649960448,"internally_reserved":0,"allocated":8769536,"data_stored":4781759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000949"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.348825+0000","last_change":"2023-10-06T10:43:03.348825+0000","last_active":"2023-10-06T10:43:03.348825+0000","last_peered":"2023-10-06T10:43:03.348825+0000","last_clean":"2023-10-06T10:43:03.348825+0000","last_became_active":"2023-10-06T10:43:03.348377+0000","last_became_peered":"2023-10-06T10:43:03.348377+0000","last_unstale":"2023-10-06T10:43:03.348825+0000","last_undegraded":"2023-10-06T10:43:03.348825+0000","last_fullsized":"2023-10-06T10:43:03.348825+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:12:07.366824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.348970+0000","last_change":"2023-10-06T10:43:03.348970+0000","last_active":"2023-10-06T10:43:03.348970+0000","last_peered":"2023-10-06T10:43:03.348970+0000","last_clean":"2023-10-06T10:43:03.348970+0000","last_became_active":"2023-10-06T10:43:03.347340+0000","last_became_peered":"2023-10-06T10:43:03.347340+0000","last_unstale":"2023-10-06T10:43:03.348970+0000","last_undegraded":"2023-10-06T10:43:03.348970+0000","last_fullsized":"2023-10-06T10:43:03.348970+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:07.715335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.350831+0000","last_change":"2023-10-06T10:43:03.350831+0000","last_active":"2023-10-06T10:43:03.350831+0000","last_peered":"2023-10-06T10:43:03.350831+0000","last_clean":"2023-10-06T10:43:03.350831+0000","last_became_active":"2023-10-06T10:43:03.350571+0000","last_became_peered":"2023-10-06T10:43:03.350571+0000","last_unstale":"2023-10-06T10:43:03.350831+0000","last_undegraded":"2023-10-06T10:43:03.350831+0000","last_fullsized":"2023-10-06T10:43:03.350831+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:51.848204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.342981+0000","last_change":"2023-10-06T10:43:03.342981+0000","last_active":"2023-10-06T10:43:03.342981+0000","last_peered":"2023-10-06T10:43:03.342981+0000","last_clean":"2023-10-06T10:43:03.342981+0000","last_became_active":"2023-10-06T10:43:03.342730+0000","last_became_peered":"2023-10-06T10:43:03.342730+0000","last_unstale":"2023-10-06T10:43:03.342981+0000","last_undegraded":"2023-10-06T10:43:03.342981+0000","last_fullsized":"2023-10-06T10:43:03.342981+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:35:50.641107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.340918+0000","last_change":"2023-10-06T10:43:03.340918+0000","last_active":"2023-10-06T10:43:03.340918+0000","last_peered":"2023-10-06T10:43:03.340918+0000","last_clean":"2023-10-06T10:43:03.340918+0000","last_became_active":"2023-10-06T10:43:03.340633+0000","last_became_peered":"2023-10-06T10:43:03.340633+0000","last_unstale":"2023-10-06T10:43:03.340918+0000","last_undegraded":"2023-10-06T10:43:03.340918+0000","last_fullsized":"2023-10-06T10:43:03.340918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:25:13.175569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.341651+0000","last_change":"2023-10-06T10:43:03.341651+0000","last_active":"2023-10-06T10:43:03.341651+0000","last_peered":"2023-10-06T10:43:03.341651+0000","last_clean":"2023-10-06T10:43:03.341651+0000","last_became_active":"2023-10-06T10:43:03.341409+0000","last_became_peered":"2023-10-06T10:43:03.341409+0000","last_unstale":"2023-10-06T10:43:03.341651+0000","last_undegraded":"2023-10-06T10:43:03.341651+0000","last_fullsized":"2023-10-06T10:43:03.341651+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:54:25.531456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.347365+0000","last_change":"2023-10-06T10:43:03.347365+0000","last_active":"2023-10-06T10:43:03.347365+0000","last_peered":"2023-10-06T10:43:03.347365+0000","last_clean":"2023-10-06T10:43:03.347365+0000","last_became_active":"2023-10-06T10:43:03.347119+0000","last_became_peered":"2023-10-06T10:43:03.347119+0000","last_unstale":"2023-10-06T10:43:03.347365+0000","last_undegraded":"2023-10-06T10:43:03.347365+0000","last_fullsized":"2023-10-06T10:43:03.347365+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:54.517332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.344615+0000","last_change":"2023-10-06T10:43:03.344615+0000","last_active":"2023-10-06T10:43:03.344615+0000","last_peered":"2023-10-06T10:43:03.344615+0000","last_clean":"2023-10-06T10:43:03.344615+0000","last_became_active":"2023-10-06T10:43:03.344361+0000","last_became_peered":"2023-10-06T10:43:03.344361+0000","last_unstale":"2023-10-06T10:43:03.344615+0000","last_undegraded":"2023-10-06T10:43:03.344615+0000","last_fullsized":"2023-10-06T10:43:03.344615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:05.694257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.346369+0000","last_change":"2023-10-06T10:43:03.346369+0000","last_active":"2023-10-06T10:43:03.346369+0000","last_peered":"2023-10-06T10:43:03.346369+0000","last_clean":"2023-10-06T10:43:03.346369+0000","last_became_active":"2023-10-06T10:43:03.346096+0000","last_became_peered":"2023-10-06T10:43:03.346096+0000","last_unstale":"2023-10-06T10:43:03.346369+0000","last_undegraded":"2023-10-06T10:43:03.346369+0000","last_fullsized":"2023-10-06T10:43:03.346369+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:51.848204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.344088+0000","last_change":"2023-10-06T10:43:03.344088+0000","last_active":"2023-10-06T10:43:03.344088+0000","last_peered":"2023-10-06T10:43:03.344088+0000","last_clean":"2023-10-06T10:43:03.344088+0000","last_became_active":"2023-10-06T10:43:03.343824+0000","last_became_peered":"2023-10-06T10:43:03.343824+0000","last_unstale":"2023-10-06T10:43:03.344088+0000","last_undegraded":"2023-10-06T10:43:03.344088+0000","last_fullsized":"2023-10-06T10:43:03.344088+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:05.694257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.348177+0000","last_change":"2023-10-06T10:43:03.348177+0000","last_active":"2023-10-06T10:43:03.348177+0000","last_peered":"2023-10-06T10:43:03.348177+0000","last_clean":"2023-10-06T10:43:03.348177+0000","last_became_active":"2023-10-06T10:43:03.347898+0000","last_became_peered":"2023-10-06T10:43:03.347898+0000","last_unstale":"2023-10-06T10:43:03.348177+0000","last_undegraded":"2023-10-06T10:43:03.348177+0000","last_fullsized":"2023-10-06T10:43:03.348177+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:13.782049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.345566+0000","last_change":"2023-10-06T10:43:03.345566+0000","last_active":"2023-10-06T10:43:03.345566+0000","last_peered":"2023-10-06T10:43:03.345566+0000","last_clean":"2023-10-06T10:43:03.345566+0000","last_became_active":"2023-10-06T10:43:03.345291+0000","last_became_peered":"2023-10-06T10:43:03.345291+0000","last_unstale":"2023-10-06T10:43:03.345566+0000","last_undegraded":"2023-10-06T10:43:03.345566+0000","last_fullsized":"2023-10-06T10:43:03.345566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:42:16.964813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.352521+0000","last_change":"2023-10-06T10:43:03.352521+0000","last_active":"2023-10-06T10:43:03.352521+0000","last_peered":"2023-10-06T10:43:03.352521+0000","last_clean":"2023-10-06T10:43:03.352521+0000","last_became_active":"2023-10-06T10:43:03.352214+0000","last_became_peered":"2023-10-06T10:43:03.352214+0000","last_unstale":"2023-10-06T10:43:03.352521+0000","last_undegraded":"2023-10-06T10:43:03.352521+0000","last_fullsized":"2023-10-06T10:43:03.352521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:19:17.831613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.344001+0000","last_change":"2023-10-06T10:43:03.344001+0000","last_active":"2023-10-06T10:43:03.344001+0000","last_peered":"2023-10-06T10:43:03.344001+0000","last_clean":"2023-10-06T10:43:03.344001+0000","last_became_active":"2023-10-06T10:43:03.343756+0000","last_became_peered":"2023-10-06T10:43:03.343756+0000","last_unstale":"2023-10-06T10:43:03.344001+0000","last_undegraded":"2023-10-06T10:43:03.344001+0000","last_fullsized":"2023-10-06T10:43:03.344001+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:16.354467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.347968+0000","last_change":"2023-10-06T10:43:03.347968+0000","last_active":"2023-10-06T10:43:03.347968+0000","last_peered":"2023-10-06T10:43:03.347968+0000","last_clean":"2023-10-06T10:43:03.347968+0000","last_became_active":"2023-10-06T10:43:03.347720+0000","last_became_peered":"2023-10-06T10:43:03.347720+0000","last_unstale":"2023-10-06T10:43:03.347968+0000","last_undegraded":"2023-10-06T10:43:03.347968+0000","last_fullsized":"2023-10-06T10:43:03.347968+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:33.284454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.340506+0000","last_change":"2023-10-06T10:43:03.340506+0000","last_active":"2023-10-06T10:43:03.340506+0000","last_peered":"2023-10-06T10:43:03.340506+0000","last_clean":"2023-10-06T10:43:03.340506+0000","last_became_active":"2023-10-06T10:43:03.340213+0000","last_became_peered":"2023-10-06T10:43:03.340213+0000","last_unstale":"2023-10-06T10:43:03.340506+0000","last_undegraded":"2023-10-06T10:43:03.340506+0000","last_fullsized":"2023-10-06T10:43:03.340506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:51.840777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.341514+0000","last_change":"2023-10-06T10:43:03.341514+0000","last_active":"2023-10-06T10:43:03.341514+0000","last_peered":"2023-10-06T10:43:03.341514+0000","last_clean":"2023-10-06T10:43:03.341514+0000","last_became_active":"2023-10-06T10:43:03.340789+0000","last_became_peered":"2023-10-06T10:43:03.340789+0000","last_unstale":"2023-10-06T10:43:03.341514+0000","last_undegraded":"2023-10-06T10:43:03.341514+0000","last_fullsized":"2023-10-06T10:43:03.341514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:39.097044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.348045+0000","last_change":"2023-10-06T10:43:03.348045+0000","last_active":"2023-10-06T10:43:03.348045+0000","last_peered":"2023-10-06T10:43:03.348045+0000","last_clean":"2023-10-06T10:43:03.348045+0000","last_became_active":"2023-10-06T10:43:03.347825+0000","last_became_peered":"2023-10-06T10:43:03.347825+0000","last_unstale":"2023-10-06T10:43:03.348045+0000","last_undegraded":"2023-10-06T10:43:03.348045+0000","last_fullsized":"2023-10-06T10:43:03.348045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:13.782049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.346596+0000","last_change":"2023-10-06T10:43:03.346596+0000","last_active":"2023-10-06T10:43:03.346596+0000","last_peered":"2023-10-06T10:43:03.346596+0000","last_clean":"2023-10-06T10:43:03.346596+0000","last_became_active":"2023-10-06T10:43:03.346290+0000","last_became_peered":"2023-10-06T10:43:03.346290+0000","last_unstale":"2023-10-06T10:43:03.346596+0000","last_undegraded":"2023-10-06T10:43:03.346596+0000","last_fullsized":"2023-10-06T10:43:03.346596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:39.097044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.346373+0000","last_change":"2023-10-06T10:43:03.346373+0000","last_active":"2023-10-06T10:43:03.346373+0000","last_peered":"2023-10-06T10:43:03.346373+0000","last_clean":"2023-10-06T10:43:03.346373+0000","last_became_active":"2023-10-06T10:43:03.345998+0000","last_became_peered":"2023-10-06T10:43:03.345998+0000","last_unstale":"2023-10-06T10:43:03.346373+0000","last_undegraded":"2023-10-06T10:43:03.346373+0000","last_fullsized":"2023-10-06T10:43:03.346373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:01.155194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.346220+0000","last_change":"2023-10-06T10:43:03.346220+0000","last_active":"2023-10-06T10:43:03.346220+0000","last_peered":"2023-10-06T10:43:03.346220+0000","last_clean":"2023-10-06T10:43:03.346220+0000","last_became_active":"2023-10-06T10:43:03.345984+0000","last_became_peered":"2023-10-06T10:43:03.345984+0000","last_unstale":"2023-10-06T10:43:03.346220+0000","last_undegraded":"2023-10-06T10:43:03.346220+0000","last_fullsized":"2023-10-06T10:43:03.346220+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:36.448540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.345790+0000","last_change":"2023-10-06T10:43:03.345790+0000","last_active":"2023-10-06T10:43:03.345790+0000","last_peered":"2023-10-06T10:43:03.345790+0000","last_clean":"2023-10-06T10:43:03.345790+0000","last_became_active":"2023-10-06T10:43:03.345549+0000","last_became_peered":"2023-10-06T10:43:03.345549+0000","last_unstale":"2023-10-06T10:43:03.345790+0000","last_undegraded":"2023-10-06T10:43:03.345790+0000","last_fullsized":"2023-10-06T10:43:03.345790+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:48:25.036868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.346212+0000","last_change":"2023-10-06T10:43:03.346212+0000","last_active":"2023-10-06T10:43:03.346212+0000","last_peered":"2023-10-06T10:43:03.346212+0000","last_clean":"2023-10-06T10:43:03.346212+0000","last_became_active":"2023-10-06T10:43:03.345908+0000","last_became_peered":"2023-10-06T10:43:03.345908+0000","last_unstale":"2023-10-06T10:43:03.346212+0000","last_undegraded":"2023-10-06T10:43:03.346212+0000","last_fullsized":"2023-10-06T10:43:03.346212+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:51.848204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.345658+0000","last_change":"2023-10-06T10:43:03.345658+0000","last_active":"2023-10-06T10:43:03.345658+0000","last_peered":"2023-10-06T10:43:03.345658+0000","last_clean":"2023-10-06T10:43:03.345658+0000","last_became_active":"2023-10-06T10:43:03.345386+0000","last_became_peered":"2023-10-06T10:43:03.345386+0000","last_unstale":"2023-10-06T10:43:03.345658+0000","last_undegraded":"2023-10-06T10:43:03.345658+0000","last_fullsized":"2023-10-06T10:43:03.345658+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:55.127339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.349245+0000","last_change":"2023-10-06T10:43:03.349245+0000","last_active":"2023-10-06T10:43:03.349245+0000","last_peered":"2023-10-06T10:43:03.349245+0000","last_clean":"2023-10-06T10:43:03.349245+0000","last_became_active":"2023-10-06T10:43:03.348976+0000","last_became_peered":"2023-10-06T10:43:03.348976+0000","last_unstale":"2023-10-06T10:43:03.349245+0000","last_undegraded":"2023-10-06T10:43:03.349245+0000","last_fullsized":"2023-10-06T10:43:03.349245+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:18:33.527316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.351925+0000","last_change":"2023-10-06T10:43:03.351925+0000","last_active":"2023-10-06T10:43:03.351925+0000","last_peered":"2023-10-06T10:43:03.351925+0000","last_clean":"2023-10-06T10:43:03.351925+0000","last_became_active":"2023-10-06T10:43:03.351677+0000","last_became_peered":"2023-10-06T10:43:03.351677+0000","last_unstale":"2023-10-06T10:43:03.351925+0000","last_undegraded":"2023-10-06T10:43:03.351925+0000","last_fullsized":"2023-10-06T10:43:03.351925+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:05:18.178107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.347216+0000","last_change":"2023-10-06T10:43:03.347216+0000","last_active":"2023-10-06T10:43:03.347216+0000","last_peered":"2023-10-06T10:43:03.347216+0000","last_clean":"2023-10-06T10:43:03.347216+0000","last_became_active":"2023-10-06T10:43:03.346584+0000","last_became_peered":"2023-10-06T10:43:03.346584+0000","last_unstale":"2023-10-06T10:43:03.347216+0000","last_undegraded":"2023-10-06T10:43:03.347216+0000","last_fullsized":"2023-10-06T10:43:03.347216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:08.238984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.350148+0000","last_change":"2023-10-06T10:43:03.350148+0000","last_active":"2023-10-06T10:43:03.350148+0000","last_peered":"2023-10-06T10:43:03.350148+0000","last_clean":"2023-10-06T10:43:03.350148+0000","last_became_active":"2023-10-06T10:43:03.349687+0000","last_became_peered":"2023-10-06T10:43:03.349687+0000","last_unstale":"2023-10-06T10:43:03.350148+0000","last_undegraded":"2023-10-06T10:43:03.350148+0000","last_fullsized":"2023-10-06T10:43:03.350148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:39.049097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.343448+0000","last_change":"2023-10-06T10:43:03.343448+0000","last_active":"2023-10-06T10:43:03.343448+0000","last_peered":"2023-10-06T10:43:03.343448+0000","last_clean":"2023-10-06T10:43:03.343448+0000","last_became_active":"2023-10-06T10:43:03.343195+0000","last_became_peered":"2023-10-06T10:43:03.343195+0000","last_unstale":"2023-10-06T10:43:03.343448+0000","last_undegraded":"2023-10-06T10:43:03.343448+0000","last_fullsized":"2023-10-06T10:43:03.343448+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:27.895051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.340843+0000","last_change":"2023-10-06T10:43:03.340843+0000","last_active":"2023-10-06T10:43:03.340843+0000","last_peered":"2023-10-06T10:43:03.340843+0000","last_clean":"2023-10-06T10:43:03.340843+0000","last_became_active":"2023-10-06T10:43:03.340655+0000","last_became_peered":"2023-10-06T10:43:03.340655+0000","last_unstale":"2023-10-06T10:43:03.340843+0000","last_undegraded":"2023-10-06T10:43:03.340843+0000","last_fullsized":"2023-10-06T10:43:03.340843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:08.779362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.347744+0000","last_change":"2023-10-06T10:43:03.347744+0000","last_active":"2023-10-06T10:43:03.347744+0000","last_peered":"2023-10-06T10:43:03.347744+0000","last_clean":"2023-10-06T10:43:03.347744+0000","last_became_active":"2023-10-06T10:43:03.346519+0000","last_became_peered":"2023-10-06T10:43:03.346519+0000","last_unstale":"2023-10-06T10:43:03.347744+0000","last_undegraded":"2023-10-06T10:43:03.347744+0000","last_fullsized":"2023-10-06T10:43:03.347744+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:33.284454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.345365+0000","last_change":"2023-10-06T10:43:03.345365+0000","last_active":"2023-10-06T10:43:03.345365+0000","last_peered":"2023-10-06T10:43:03.345365+0000","last_clean":"2023-10-06T10:43:03.345365+0000","last_became_active":"2023-10-06T10:43:03.345107+0000","last_became_peered":"2023-10-06T10:43:03.345107+0000","last_unstale":"2023-10-06T10:43:03.345365+0000","last_undegraded":"2023-10-06T10:43:03.345365+0000","last_fullsized":"2023-10-06T10:43:03.345365+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:02.436557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.347589+0000","last_change":"2023-10-06T10:43:03.347589+0000","last_active":"2023-10-06T10:43:03.347589+0000","last_peered":"2023-10-06T10:43:03.347589+0000","last_clean":"2023-10-06T10:43:03.347589+0000","last_became_active":"2023-10-06T10:43:03.347334+0000","last_became_peered":"2023-10-06T10:43:03.347334+0000","last_unstale":"2023-10-06T10:43:03.347589+0000","last_undegraded":"2023-10-06T10:43:03.347589+0000","last_fullsized":"2023-10-06T10:43:03.347589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:17:36.881748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"77'4","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.796402+0000","last_change":"2023-10-06T10:43:02.804016+0000","last_active":"2023-10-06T10:43:03.796402+0000","last_peered":"2023-10-06T10:43:03.796402+0000","last_clean":"2023-10-06T10:43:03.796402+0000","last_became_active":"2023-10-06T10:43:02.803783+0000","last_became_peered":"2023-10-06T10:43:02.803783+0000","last_unstale":"2023-10-06T10:43:03.796402+0000","last_undegraded":"2023-10-06T10:43:03.796402+0000","last_fullsized":"2023-10-06T10:43:03.796402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:55:52.806135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.348623+0000","last_change":"2023-10-06T10:43:03.348623+0000","last_active":"2023-10-06T10:43:03.348623+0000","last_peered":"2023-10-06T10:43:03.348623+0000","last_clean":"2023-10-06T10:43:03.348623+0000","last_became_active":"2023-10-06T10:43:03.348015+0000","last_became_peered":"2023-10-06T10:43:03.348015+0000","last_unstale":"2023-10-06T10:43:03.348623+0000","last_undegraded":"2023-10-06T10:43:03.348623+0000","last_fullsized":"2023-10-06T10:43:03.348623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:17:21.634775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"77'1","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.334466+0000","last_change":"2023-10-06T10:43:02.802669+0000","last_active":"2023-10-06T10:43:03.334466+0000","last_peered":"2023-10-06T10:43:03.334466+0000","last_clean":"2023-10-06T10:43:03.334466+0000","last_became_active":"2023-10-06T10:43:02.802413+0000","last_became_peered":"2023-10-06T10:43:02.802413+0000","last_unstale":"2023-10-06T10:43:03.334466+0000","last_undegraded":"2023-10-06T10:43:03.334466+0000","last_fullsized":"2023-10-06T10:43:03.334466+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:50.843346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.345762+0000","last_change":"2023-10-06T10:43:03.345762+0000","last_active":"2023-10-06T10:43:03.345762+0000","last_peered":"2023-10-06T10:43:03.345762+0000","last_clean":"2023-10-06T10:43:03.345762+0000","last_became_active":"2023-10-06T10:43:03.344991+0000","last_became_peered":"2023-10-06T10:43:03.344991+0000","last_unstale":"2023-10-06T10:43:03.345762+0000","last_undegraded":"2023-10-06T10:43:03.345762+0000","last_fullsized":"2023-10-06T10:43:03.345762+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:29:39.107500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"77'2","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.794933+0000","last_change":"2023-10-06T10:43:02.796112+0000","last_active":"2023-10-06T10:43:03.794933+0000","last_peered":"2023-10-06T10:43:03.794933+0000","last_clean":"2023-10-06T10:43:03.794933+0000","last_became_active":"2023-10-06T10:43:02.795857+0000","last_became_peered":"2023-10-06T10:43:02.795857+0000","last_unstale":"2023-10-06T10:43:03.794933+0000","last_undegraded":"2023-10-06T10:43:03.794933+0000","last_fullsized":"2023-10-06T10:43:03.794933+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:38.044240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.352342+0000","last_change":"2023-10-06T10:43:03.352342+0000","last_active":"2023-10-06T10:43:03.352342+0000","last_peered":"2023-10-06T10:43:03.352342+0000","last_clean":"2023-10-06T10:43:03.352342+0000","last_became_active":"2023-10-06T10:43:03.352042+0000","last_became_peered":"2023-10-06T10:43:03.352042+0000","last_unstale":"2023-10-06T10:43:03.352342+0000","last_undegraded":"2023-10-06T10:43:03.352342+0000","last_fullsized":"2023-10-06T10:43:03.352342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:21.180392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"77'2","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.795169+0000","last_change":"2023-10-06T10:43:02.796566+0000","last_active":"2023-10-06T10:43:03.795169+0000","last_peered":"2023-10-06T10:43:03.795169+0000","last_clean":"2023-10-06T10:43:03.795169+0000","last_became_active":"2023-10-06T10:43:02.796337+0000","last_became_peered":"2023-10-06T10:43:02.796337+0000","last_unstale":"2023-10-06T10:43:03.795169+0000","last_undegraded":"2023-10-06T10:43:03.795169+0000","last_fullsized":"2023-10-06T10:43:03.795169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:01.431700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.340735+0000","last_change":"2023-10-06T10:43:03.340735+0000","last_active":"2023-10-06T10:43:03.340735+0000","last_peered":"2023-10-06T10:43:03.340735+0000","last_clean":"2023-10-06T10:43:03.340735+0000","last_became_active":"2023-10-06T10:43:03.340528+0000","last_became_peered":"2023-10-06T10:43:03.340528+0000","last_unstale":"2023-10-06T10:43:03.340735+0000","last_undegraded":"2023-10-06T10:43:03.340735+0000","last_fullsized":"2023-10-06T10:43:03.340735+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:17:24.862748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"77'1","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.332883+0000","last_change":"2023-10-06T10:43:02.795595+0000","last_active":"2023-10-06T10:43:03.332883+0000","last_peered":"2023-10-06T10:43:03.332883+0000","last_clean":"2023-10-06T10:43:03.332883+0000","last_became_active":"2023-10-06T10:43:02.795301+0000","last_became_peered":"2023-10-06T10:43:02.795301+0000","last_unstale":"2023-10-06T10:43:03.332883+0000","last_undegraded":"2023-10-06T10:43:03.332883+0000","last_fullsized":"2023-10-06T10:43:03.332883+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:06:32.177702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.341265+0000","last_change":"2023-10-06T10:43:03.341265+0000","last_active":"2023-10-06T10:43:03.341265+0000","last_peered":"2023-10-06T10:43:03.341265+0000","last_clean":"2023-10-06T10:43:03.341265+0000","last_became_active":"2023-10-06T10:43:03.340997+0000","last_became_peered":"2023-10-06T10:43:03.340997+0000","last_unstale":"2023-10-06T10:43:03.341265+0000","last_undegraded":"2023-10-06T10:43:03.341265+0000","last_fullsized":"2023-10-06T10:43:03.341265+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:32:40.836119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"77'1","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.332883+0000","last_change":"2023-10-06T10:43:02.798764+0000","last_active":"2023-10-06T10:43:03.332883+0000","last_peered":"2023-10-06T10:43:03.332883+0000","last_clean":"2023-10-06T10:43:03.332883+0000","last_became_active":"2023-10-06T10:43:02.798517+0000","last_became_peered":"2023-10-06T10:43:02.798517+0000","last_unstale":"2023-10-06T10:43:03.332883+0000","last_undegraded":"2023-10-06T10:43:03.332883+0000","last_fullsized":"2023-10-06T10:43:03.332883+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:54:24.526598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.350778+0000","last_change":"2023-10-06T10:43:03.350778+0000","last_active":"2023-10-06T10:43:03.350778+0000","last_peered":"2023-10-06T10:43:03.350778+0000","last_clean":"2023-10-06T10:43:03.350778+0000","last_became_active":"2023-10-06T10:43:03.350495+0000","last_became_peered":"2023-10-06T10:43:03.350495+0000","last_unstale":"2023-10-06T10:43:03.350778+0000","last_undegraded":"2023-10-06T10:43:03.350778+0000","last_fullsized":"2023-10-06T10:43:03.350778+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:54:25.531456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.800185+0000","last_change":"2023-10-06T10:43:02.797585+0000","last_active":"2023-10-06T10:43:02.800185+0000","last_peered":"2023-10-06T10:43:02.800185+0000","last_clean":"2023-10-06T10:43:02.800185+0000","last_became_active":"2023-10-06T10:43:02.797370+0000","last_became_peered":"2023-10-06T10:43:02.797370+0000","last_unstale":"2023-10-06T10:43:02.800185+0000","last_undegraded":"2023-10-06T10:43:02.800185+0000","last_fullsized":"2023-10-06T10:43:02.800185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:29:38.102642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.347684+0000","last_change":"2023-10-06T10:43:03.347684+0000","last_active":"2023-10-06T10:43:03.347684+0000","last_peered":"2023-10-06T10:43:03.347684+0000","last_clean":"2023-10-06T10:43:03.347684+0000","last_became_active":"2023-10-06T10:43:03.347398+0000","last_became_peered":"2023-10-06T10:43:03.347398+0000","last_unstale":"2023-10-06T10:43:03.347684+0000","last_undegraded":"2023-10-06T10:43:03.347684+0000","last_fullsized":"2023-10-06T10:43:03.347684+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:50.399077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.805748+0000","last_change":"2023-10-06T10:43:02.803051+0000","last_active":"2023-10-06T10:43:02.805748+0000","last_peered":"2023-10-06T10:43:02.805748+0000","last_clean":"2023-10-06T10:43:02.805748+0000","last_became_active":"2023-10-06T10:43:02.802783+0000","last_became_peered":"2023-10-06T10:43:02.802783+0000","last_unstale":"2023-10-06T10:43:02.805748+0000","last_undegraded":"2023-10-06T10:43:02.805748+0000","last_fullsized":"2023-10-06T10:43:02.805748+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:10.643656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.342893+0000","last_change":"2023-10-06T10:43:03.342893+0000","last_active":"2023-10-06T10:43:03.342893+0000","last_peered":"2023-10-06T10:43:03.342893+0000","last_clean":"2023-10-06T10:43:03.342893+0000","last_became_active":"2023-10-06T10:43:03.342659+0000","last_became_peered":"2023-10-06T10:43:03.342659+0000","last_unstale":"2023-10-06T10:43:03.342893+0000","last_undegraded":"2023-10-06T10:43:03.342893+0000","last_fullsized":"2023-10-06T10:43:03.342893+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:18.479800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.810153+0000","last_change":"2023-10-06T10:43:02.807224+0000","last_active":"2023-10-06T10:43:02.810153+0000","last_peered":"2023-10-06T10:43:02.810153+0000","last_clean":"2023-10-06T10:43:02.810153+0000","last_became_active":"2023-10-06T10:43:02.806999+0000","last_became_peered":"2023-10-06T10:43:02.806999+0000","last_unstale":"2023-10-06T10:43:02.810153+0000","last_undegraded":"2023-10-06T10:43:02.810153+0000","last_fullsized":"2023-10-06T10:43:02.810153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:18:32.522458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.341600+0000","last_change":"2023-10-06T10:43:03.341600+0000","last_active":"2023-10-06T10:43:03.341600+0000","last_peered":"2023-10-06T10:43:03.341600+0000","last_clean":"2023-10-06T10:43:03.341600+0000","last_became_active":"2023-10-06T10:43:03.341329+0000","last_became_peered":"2023-10-06T10:43:03.341329+0000","last_unstale":"2023-10-06T10:43:03.341600+0000","last_undegraded":"2023-10-06T10:43:03.341600+0000","last_fullsized":"2023-10-06T10:43:03.341600+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:19:17.831613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.806855+0000","last_change":"2023-10-06T10:43:02.801375+0000","last_active":"2023-10-06T10:43:02.806855+0000","last_peered":"2023-10-06T10:43:02.806855+0000","last_clean":"2023-10-06T10:43:02.806855+0000","last_became_active":"2023-10-06T10:43:02.800825+0000","last_became_peered":"2023-10-06T10:43:02.800825+0000","last_unstale":"2023-10-06T10:43:02.806855+0000","last_undegraded":"2023-10-06T10:43:02.806855+0000","last_fullsized":"2023-10-06T10:43:02.806855+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:32:16.160390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.350255+0000","last_change":"2023-10-06T10:43:03.350255+0000","last_active":"2023-10-06T10:43:03.350255+0000","last_peered":"2023-10-06T10:43:03.350255+0000","last_clean":"2023-10-06T10:43:03.350255+0000","last_became_active":"2023-10-06T10:43:03.350080+0000","last_became_peered":"2023-10-06T10:43:03.350080+0000","last_unstale":"2023-10-06T10:43:03.350255+0000","last_undegraded":"2023-10-06T10:43:03.350255+0000","last_fullsized":"2023-10-06T10:43:03.350255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:50.399077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"77'3","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.796173+0000","last_change":"2023-10-06T10:43:02.802962+0000","last_active":"2023-10-06T10:43:03.796173+0000","last_peered":"2023-10-06T10:43:03.796173+0000","last_clean":"2023-10-06T10:43:03.796173+0000","last_became_active":"2023-10-06T10:43:02.801745+0000","last_became_peered":"2023-10-06T10:43:02.801745+0000","last_unstale":"2023-10-06T10:43:03.796173+0000","last_undegraded":"2023-10-06T10:43:03.796173+0000","last_fullsized":"2023-10-06T10:43:03.796173+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:49:24.573246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.347913+0000","last_change":"2023-10-06T10:43:03.347913+0000","last_active":"2023-10-06T10:43:03.347913+0000","last_peered":"2023-10-06T10:43:03.347913+0000","last_clean":"2023-10-06T10:43:03.347913+0000","last_became_active":"2023-10-06T10:43:03.347497+0000","last_became_peered":"2023-10-06T10:43:03.347497+0000","last_unstale":"2023-10-06T10:43:03.347913+0000","last_undegraded":"2023-10-06T10:43:03.347913+0000","last_fullsized":"2023-10-06T10:43:03.347913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:21:55.222924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"77'6","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.797539+0000","last_change":"2023-10-06T10:43:02.799454+0000","last_active":"2023-10-06T10:43:03.797539+0000","last_peered":"2023-10-06T10:43:03.797539+0000","last_clean":"2023-10-06T10:43:03.797539+0000","last_became_active":"2023-10-06T10:43:02.799143+0000","last_became_peered":"2023-10-06T10:43:02.799143+0000","last_unstale":"2023-10-06T10:43:03.797539+0000","last_undegraded":"2023-10-06T10:43:03.797539+0000","last_fullsized":"2023-10-06T10:43:03.797539+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:49:24.573246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.340576+0000","last_change":"2023-10-06T10:43:03.340576+0000","last_active":"2023-10-06T10:43:03.340576+0000","last_peered":"2023-10-06T10:43:03.340576+0000","last_clean":"2023-10-06T10:43:03.340576+0000","last_became_active":"2023-10-06T10:43:03.340299+0000","last_became_peered":"2023-10-06T10:43:03.340299+0000","last_unstale":"2023-10-06T10:43:03.340576+0000","last_undegraded":"2023-10-06T10:43:03.340576+0000","last_fullsized":"2023-10-06T10:43:03.340576+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:08.238984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"77'2","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.795759+0000","last_change":"2023-10-06T10:43:02.803801+0000","last_active":"2023-10-06T10:43:03.795759+0000","last_peered":"2023-10-06T10:43:03.795759+0000","last_clean":"2023-10-06T10:43:03.795759+0000","last_became_active":"2023-10-06T10:43:02.803595+0000","last_became_peered":"2023-10-06T10:43:02.803595+0000","last_unstale":"2023-10-06T10:43:03.795759+0000","last_undegraded":"2023-10-06T10:43:03.795759+0000","last_fullsized":"2023-10-06T10:43:03.795759+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:00.410985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.350784+0000","last_change":"2023-10-06T10:43:03.350784+0000","last_active":"2023-10-06T10:43:03.350784+0000","last_peered":"2023-10-06T10:43:03.350784+0000","last_clean":"2023-10-06T10:43:03.350784+0000","last_became_active":"2023-10-06T10:43:03.350314+0000","last_became_peered":"2023-10-06T10:43:03.350314+0000","last_unstale":"2023-10-06T10:43:03.350784+0000","last_undegraded":"2023-10-06T10:43:03.350784+0000","last_fullsized":"2023-10-06T10:43:03.350784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:05.694257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.802071+0000","last_change":"2023-10-06T10:43:02.801545+0000","last_active":"2023-10-06T10:43:02.802071+0000","last_peered":"2023-10-06T10:43:02.802071+0000","last_clean":"2023-10-06T10:43:02.802071+0000","last_became_active":"2023-10-06T10:43:02.797698+0000","last_became_peered":"2023-10-06T10:43:02.797698+0000","last_unstale":"2023-10-06T10:43:02.802071+0000","last_undegraded":"2023-10-06T10:43:02.802071+0000","last_fullsized":"2023-10-06T10:43:02.802071+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:07:44.434485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.339839+0000","last_change":"2023-10-06T10:43:03.339839+0000","last_active":"2023-10-06T10:43:03.339839+0000","last_peered":"2023-10-06T10:43:03.339839+0000","last_clean":"2023-10-06T10:43:03.339839+0000","last_became_active":"2023-10-06T10:43:03.339604+0000","last_became_peered":"2023-10-06T10:43:03.339604+0000","last_unstale":"2023-10-06T10:43:03.339839+0000","last_undegraded":"2023-10-06T10:43:03.339839+0000","last_fullsized":"2023-10-06T10:43:03.339839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:36.448540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.806675+0000","last_change":"2023-10-06T10:43:02.799814+0000","last_active":"2023-10-06T10:43:02.806675+0000","last_peered":"2023-10-06T10:43:02.806675+0000","last_clean":"2023-10-06T10:43:02.806675+0000","last_became_active":"2023-10-06T10:43:02.799531+0000","last_became_peered":"2023-10-06T10:43:02.799531+0000","last_unstale":"2023-10-06T10:43:02.806675+0000","last_undegraded":"2023-10-06T10:43:02.806675+0000","last_fullsized":"2023-10-06T10:43:02.806675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:32:16.160390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.347466+0000","last_change":"2023-10-06T10:43:03.347466+0000","last_active":"2023-10-06T10:43:03.347466+0000","last_peered":"2023-10-06T10:43:03.347466+0000","last_clean":"2023-10-06T10:43:03.347466+0000","last_became_active":"2023-10-06T10:43:03.347222+0000","last_became_peered":"2023-10-06T10:43:03.347222+0000","last_unstale":"2023-10-06T10:43:03.347466+0000","last_undegraded":"2023-10-06T10:43:03.347466+0000","last_fullsized":"2023-10-06T10:43:03.347466+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:45:47.387287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"77'8","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.796790+0000","last_change":"2023-10-06T10:43:02.796068+0000","last_active":"2023-10-06T10:43:03.796790+0000","last_peered":"2023-10-06T10:43:03.796790+0000","last_clean":"2023-10-06T10:43:03.796790+0000","last_became_active":"2023-10-06T10:43:02.795811+0000","last_became_peered":"2023-10-06T10:43:02.795811+0000","last_unstale":"2023-10-06T10:43:03.796790+0000","last_undegraded":"2023-10-06T10:43:03.796790+0000","last_fullsized":"2023-10-06T10:43:03.796790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:17:20.629917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.343249+0000","last_change":"2023-10-06T10:43:03.343249+0000","last_active":"2023-10-06T10:43:03.343249+0000","last_peered":"2023-10-06T10:43:03.343249+0000","last_clean":"2023-10-06T10:43:03.343249+0000","last_became_active":"2023-10-06T10:43:03.342470+0000","last_became_peered":"2023-10-06T10:43:03.342470+0000","last_unstale":"2023-10-06T10:43:03.343249+0000","last_undegraded":"2023-10-06T10:43:03.343249+0000","last_fullsized":"2023-10-06T10:43:03.343249+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:50.399077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"77'2","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.796130+0000","last_change":"2023-10-06T10:43:02.799931+0000","last_active":"2023-10-06T10:43:03.796130+0000","last_peered":"2023-10-06T10:43:03.796130+0000","last_clean":"2023-10-06T10:43:03.796130+0000","last_became_active":"2023-10-06T10:43:02.799668+0000","last_became_peered":"2023-10-06T10:43:02.799668+0000","last_unstale":"2023-10-06T10:43:03.796130+0000","last_undegraded":"2023-10-06T10:43:03.796130+0000","last_fullsized":"2023-10-06T10:43:03.796130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:18:15.358712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.351186+0000","last_change":"2023-10-06T10:43:03.351186+0000","last_active":"2023-10-06T10:43:03.351186+0000","last_peered":"2023-10-06T10:43:03.351186+0000","last_clean":"2023-10-06T10:43:03.351186+0000","last_became_active":"2023-10-06T10:43:03.351008+0000","last_became_peered":"2023-10-06T10:43:03.351008+0000","last_unstale":"2023-10-06T10:43:03.351186+0000","last_undegraded":"2023-10-06T10:43:03.351186+0000","last_fullsized":"2023-10-06T10:43:03.351186+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:42:16.964813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.809167+0000","last_change":"2023-10-06T10:43:02.804748+0000","last_active":"2023-10-06T10:43:02.809167+0000","last_peered":"2023-10-06T10:43:02.809167+0000","last_clean":"2023-10-06T10:43:02.809167+0000","last_became_active":"2023-10-06T10:43:02.804502+0000","last_became_peered":"2023-10-06T10:43:02.804502+0000","last_unstale":"2023-10-06T10:43:02.809167+0000","last_undegraded":"2023-10-06T10:43:02.809167+0000","last_fullsized":"2023-10-06T10:43:02.809167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:21:04.019415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.342222+0000","last_change":"2023-10-06T10:43:03.342222+0000","last_active":"2023-10-06T10:43:03.342222+0000","last_peered":"2023-10-06T10:43:03.342222+0000","last_clean":"2023-10-06T10:43:03.342222+0000","last_became_active":"2023-10-06T10:43:03.341949+0000","last_became_peered":"2023-10-06T10:43:03.341949+0000","last_unstale":"2023-10-06T10:43:03.342222+0000","last_undegraded":"2023-10-06T10:43:03.342222+0000","last_fullsized":"2023-10-06T10:43:03.342222+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:35:01.365904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.807039+0000","last_change":"2023-10-06T10:43:02.803575+0000","last_active":"2023-10-06T10:43:02.807039+0000","last_peered":"2023-10-06T10:43:02.807039+0000","last_clean":"2023-10-06T10:43:02.807039+0000","last_became_active":"2023-10-06T10:43:02.803253+0000","last_became_peered":"2023-10-06T10:43:02.803253+0000","last_unstale":"2023-10-06T10:43:02.807039+0000","last_undegraded":"2023-10-06T10:43:02.807039+0000","last_fullsized":"2023-10-06T10:43:02.807039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:14.438218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.347269+0000","last_change":"2023-10-06T10:43:03.347269+0000","last_active":"2023-10-06T10:43:03.347269+0000","last_peered":"2023-10-06T10:43:03.347269+0000","last_clean":"2023-10-06T10:43:03.347269+0000","last_became_active":"2023-10-06T10:43:03.346944+0000","last_became_peered":"2023-10-06T10:43:03.346944+0000","last_unstale":"2023-10-06T10:43:03.347269+0000","last_undegraded":"2023-10-06T10:43:03.347269+0000","last_fullsized":"2023-10-06T10:43:03.347269+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:18.479800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.807064+0000","last_change":"2023-10-06T10:43:02.801869+0000","last_active":"2023-10-06T10:43:02.807064+0000","last_peered":"2023-10-06T10:43:02.807064+0000","last_clean":"2023-10-06T10:43:02.807064+0000","last_became_active":"2023-10-06T10:43:02.801599+0000","last_became_peered":"2023-10-06T10:43:02.801599+0000","last_unstale":"2023-10-06T10:43:02.807064+0000","last_undegraded":"2023-10-06T10:43:02.807064+0000","last_fullsized":"2023-10-06T10:43:02.807064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:37:25.769419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.344505+0000","last_change":"2023-10-06T10:43:03.344505+0000","last_active":"2023-10-06T10:43:03.344505+0000","last_peered":"2023-10-06T10:43:03.344505+0000","last_clean":"2023-10-06T10:43:03.344505+0000","last_became_active":"2023-10-06T10:43:03.344273+0000","last_became_peered":"2023-10-06T10:43:03.344273+0000","last_unstale":"2023-10-06T10:43:03.344505+0000","last_undegraded":"2023-10-06T10:43:03.344505+0000","last_fullsized":"2023-10-06T10:43:03.344505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:06:33.182560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.800971+0000","last_change":"2023-10-06T10:43:02.799176+0000","last_active":"2023-10-06T10:43:02.800971+0000","last_peered":"2023-10-06T10:43:02.800971+0000","last_clean":"2023-10-06T10:43:02.800971+0000","last_became_active":"2023-10-06T10:43:02.798691+0000","last_became_peered":"2023-10-06T10:43:02.798691+0000","last_unstale":"2023-10-06T10:43:02.800971+0000","last_undegraded":"2023-10-06T10:43:02.800971+0000","last_fullsized":"2023-10-06T10:43:02.800971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:35:00.361047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.345413+0000","last_change":"2023-10-06T10:43:03.345413+0000","last_active":"2023-10-06T10:43:03.345413+0000","last_peered":"2023-10-06T10:43:03.345413+0000","last_clean":"2023-10-06T10:43:03.345413+0000","last_became_active":"2023-10-06T10:43:03.345109+0000","last_became_peered":"2023-10-06T10:43:03.345109+0000","last_unstale":"2023-10-06T10:43:03.345413+0000","last_undegraded":"2023-10-06T10:43:03.345413+0000","last_fullsized":"2023-10-06T10:43:03.345413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:21:55.222924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.803859+0000","last_change":"2023-10-06T10:43:02.798298+0000","last_active":"2023-10-06T10:43:02.803859+0000","last_peered":"2023-10-06T10:43:02.803859+0000","last_clean":"2023-10-06T10:43:02.803859+0000","last_became_active":"2023-10-06T10:43:02.798024+0000","last_became_peered":"2023-10-06T10:43:02.798024+0000","last_unstale":"2023-10-06T10:43:02.803859+0000","last_undegraded":"2023-10-06T10:43:02.803859+0000","last_fullsized":"2023-10-06T10:43:02.803859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:19:16.826755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.352352+0000","last_change":"2023-10-06T10:43:03.352352+0000","last_active":"2023-10-06T10:43:03.352352+0000","last_peered":"2023-10-06T10:43:03.352352+0000","last_clean":"2023-10-06T10:43:03.352352+0000","last_became_active":"2023-10-06T10:43:03.352060+0000","last_became_peered":"2023-10-06T10:43:03.352060+0000","last_unstale":"2023-10-06T10:43:03.352352+0000","last_undegraded":"2023-10-06T10:43:03.352352+0000","last_fullsized":"2023-10-06T10:43:03.352352+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:06:33.182560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.806220+0000","last_change":"2023-10-06T10:43:02.800357+0000","last_active":"2023-10-06T10:43:02.806220+0000","last_peered":"2023-10-06T10:43:02.806220+0000","last_clean":"2023-10-06T10:43:02.806220+0000","last_became_active":"2023-10-06T10:43:02.798261+0000","last_became_peered":"2023-10-06T10:43:02.798261+0000","last_unstale":"2023-10-06T10:43:02.806220+0000","last_undegraded":"2023-10-06T10:43:02.806220+0000","last_fullsized":"2023-10-06T10:43:02.806220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:06.710477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.341047+0000","last_change":"2023-10-06T10:43:03.341047+0000","last_active":"2023-10-06T10:43:03.341047+0000","last_peered":"2023-10-06T10:43:03.341047+0000","last_clean":"2023-10-06T10:43:03.341047+0000","last_became_active":"2023-10-06T10:43:03.340788+0000","last_became_peered":"2023-10-06T10:43:03.340788+0000","last_unstale":"2023-10-06T10:43:03.341047+0000","last_undegraded":"2023-10-06T10:43:03.341047+0000","last_fullsized":"2023-10-06T10:43:03.341047+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:59:16.697883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.806709+0000","last_change":"2023-10-06T10:43:02.802081+0000","last_active":"2023-10-06T10:43:02.806709+0000","last_peered":"2023-10-06T10:43:02.806709+0000","last_clean":"2023-10-06T10:43:02.806709+0000","last_became_active":"2023-10-06T10:43:02.801790+0000","last_became_peered":"2023-10-06T10:43:02.801790+0000","last_unstale":"2023-10-06T10:43:02.806709+0000","last_undegraded":"2023-10-06T10:43:02.806709+0000","last_fullsized":"2023-10-06T10:43:02.806709+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:04.689399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.346663+0000","last_change":"2023-10-06T10:43:03.346663+0000","last_active":"2023-10-06T10:43:03.346663+0000","last_peered":"2023-10-06T10:43:03.346663+0000","last_clean":"2023-10-06T10:43:03.346663+0000","last_became_active":"2023-10-06T10:43:03.346401+0000","last_became_peered":"2023-10-06T10:43:03.346401+0000","last_unstale":"2023-10-06T10:43:03.346663+0000","last_undegraded":"2023-10-06T10:43:03.346663+0000","last_fullsized":"2023-10-06T10:43:03.346663+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:42:16.964813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.800971+0000","last_change":"2023-10-06T10:43:02.797198+0000","last_active":"2023-10-06T10:43:02.800971+0000","last_peered":"2023-10-06T10:43:02.800971+0000","last_clean":"2023-10-06T10:43:02.800971+0000","last_became_active":"2023-10-06T10:43:02.796953+0000","last_became_peered":"2023-10-06T10:43:02.796953+0000","last_unstale":"2023-10-06T10:43:02.800971+0000","last_undegraded":"2023-10-06T10:43:02.800971+0000","last_fullsized":"2023-10-06T10:43:02.800971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:18:39.029604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.794375+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:43:02.794375+0000","last_peered":"2023-10-06T10:43:02.794375+0000","last_clean":"2023-10-06T10:43:02.794375+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:43:02.794375+0000","last_undegraded":"2023-10-06T10:43:02.794375+0000","last_fullsized":"2023-10-06T10:43:02.794375+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.341614+0000","last_change":"2023-10-06T10:43:03.341614+0000","last_active":"2023-10-06T10:43:03.341614+0000","last_peered":"2023-10-06T10:43:03.341614+0000","last_clean":"2023-10-06T10:43:03.341614+0000","last_became_active":"2023-10-06T10:43:03.341387+0000","last_became_peered":"2023-10-06T10:43:03.341387+0000","last_unstale":"2023-10-06T10:43:03.341614+0000","last_undegraded":"2023-10-06T10:43:03.341614+0000","last_fullsized":"2023-10-06T10:43:03.341614+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:59.776158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.809006+0000","last_change":"2023-10-06T10:43:02.807812+0000","last_active":"2023-10-06T10:43:02.809006+0000","last_peered":"2023-10-06T10:43:02.809006+0000","last_clean":"2023-10-06T10:43:02.809006+0000","last_became_active":"2023-10-06T10:43:02.807600+0000","last_became_peered":"2023-10-06T10:43:02.807600+0000","last_unstale":"2023-10-06T10:43:02.809006+0000","last_undegraded":"2023-10-06T10:43:02.809006+0000","last_fullsized":"2023-10-06T10:43:02.809006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:50:29.907645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":58,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.788550+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:43:02.788550+0000","last_peered":"2023-10-06T10:43:02.788550+0000","last_clean":"2023-10-06T10:43:02.788550+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:43:02.788550+0000","last_undegraded":"2023-10-06T10:43:02.788550+0000","last_fullsized":"2023-10-06T10:43:02.788550+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.348702+0000","last_change":"2023-10-06T10:43:03.348702+0000","last_active":"2023-10-06T10:43:03.348702+0000","last_peered":"2023-10-06T10:43:03.348702+0000","last_clean":"2023-10-06T10:43:03.348702+0000","last_became_active":"2023-10-06T10:43:03.348301+0000","last_became_peered":"2023-10-06T10:43:03.348301+0000","last_unstale":"2023-10-06T10:43:03.348702+0000","last_undegraded":"2023-10-06T10:43:03.348702+0000","last_fullsized":"2023-10-06T10:43:03.348702+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:23.657941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"77'2","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.796269+0000","last_change":"2023-10-06T10:43:02.797219+0000","last_active":"2023-10-06T10:43:03.796269+0000","last_peered":"2023-10-06T10:43:03.796269+0000","last_clean":"2023-10-06T10:43:03.796269+0000","last_became_active":"2023-10-06T10:43:02.795540+0000","last_became_peered":"2023-10-06T10:43:02.795540+0000","last_unstale":"2023-10-06T10:43:03.796269+0000","last_undegraded":"2023-10-06T10:43:03.796269+0000","last_fullsized":"2023-10-06T10:43:03.796269+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:12:06.361966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":64,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.802523+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:43:02.802523+0000","last_peered":"2023-10-06T10:43:02.802523+0000","last_clean":"2023-10-06T10:43:02.802523+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:43:02.802523+0000","last_undegraded":"2023-10-06T10:43:02.802523+0000","last_fullsized":"2023-10-06T10:43:02.802523+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":205,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.795727+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:43:02.795727+0000","last_peered":"2023-10-06T10:43:02.795727+0000","last_clean":"2023-10-06T10:43:02.795727+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:43:02.795727+0000","last_undegraded":"2023-10-06T10:43:02.795727+0000","last_fullsized":"2023-10-06T10:43:02.795727+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.350663+0000","last_change":"2023-10-06T10:43:03.350663+0000","last_active":"2023-10-06T10:43:03.350663+0000","last_peered":"2023-10-06T10:43:03.350663+0000","last_clean":"2023-10-06T10:43:03.350663+0000","last_became_active":"2023-10-06T10:43:03.350397+0000","last_became_peered":"2023-10-06T10:43:03.350397+0000","last_unstale":"2023-10-06T10:43:03.350663+0000","last_undegraded":"2023-10-06T10:43:03.350663+0000","last_fullsized":"2023-10-06T10:43:03.350663+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:19.454862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"77'1","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.338350+0000","last_change":"2023-10-06T10:43:02.807119+0000","last_active":"2023-10-06T10:43:03.338350+0000","last_peered":"2023-10-06T10:43:03.338350+0000","last_clean":"2023-10-06T10:43:03.338350+0000","last_became_active":"2023-10-06T10:43:02.806656+0000","last_became_peered":"2023-10-06T10:43:02.806656+0000","last_unstale":"2023-10-06T10:43:03.338350+0000","last_undegraded":"2023-10-06T10:43:03.338350+0000","last_fullsized":"2023-10-06T10:43:03.338350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:34.441224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.795651+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:43:02.795651+0000","last_peered":"2023-10-06T10:43:02.795651+0000","last_clean":"2023-10-06T10:43:02.795651+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:43:02.795651+0000","last_undegraded":"2023-10-06T10:43:02.795651+0000","last_fullsized":"2023-10-06T10:43:02.795651+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.341684+0000","last_change":"2023-10-06T10:43:03.341684+0000","last_active":"2023-10-06T10:43:03.341684+0000","last_peered":"2023-10-06T10:43:03.341684+0000","last_clean":"2023-10-06T10:43:03.341684+0000","last_became_active":"2023-10-06T10:43:03.341430+0000","last_became_peered":"2023-10-06T10:43:03.341430+0000","last_unstale":"2023-10-06T10:43:03.341684+0000","last_undegraded":"2023-10-06T10:43:03.341684+0000","last_fullsized":"2023-10-06T10:43:03.341684+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:43.580246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"77'1","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.343264+0000","last_change":"2023-10-06T10:43:02.805773+0000","last_active":"2023-10-06T10:43:03.343264+0000","last_peered":"2023-10-06T10:43:03.343264+0000","last_clean":"2023-10-06T10:43:03.343264+0000","last_became_active":"2023-10-06T10:43:02.805528+0000","last_became_peered":"2023-10-06T10:43:02.805528+0000","last_unstale":"2023-10-06T10:43:03.343264+0000","last_undegraded":"2023-10-06T10:43:03.343264+0000","last_fullsized":"2023-10-06T10:43:03.343264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:55:52.806135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.795638+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:43:02.795638+0000","last_peered":"2023-10-06T10:43:02.795638+0000","last_clean":"2023-10-06T10:43:02.795638+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:43:02.795638+0000","last_undegraded":"2023-10-06T10:43:02.795638+0000","last_fullsized":"2023-10-06T10:43:02.795638+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.340724+0000","last_change":"2023-10-06T10:43:03.340724+0000","last_active":"2023-10-06T10:43:03.340724+0000","last_peered":"2023-10-06T10:43:03.340724+0000","last_clean":"2023-10-06T10:43:03.340724+0000","last_became_active":"2023-10-06T10:43:03.340443+0000","last_became_peered":"2023-10-06T10:43:03.340443+0000","last_unstale":"2023-10-06T10:43:03.340724+0000","last_undegraded":"2023-10-06T10:43:03.340724+0000","last_fullsized":"2023-10-06T10:43:03.340724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:21:12.430514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.800213+0000","last_change":"2023-10-06T10:43:02.797717+0000","last_active":"2023-10-06T10:43:02.800213+0000","last_peered":"2023-10-06T10:43:02.800213+0000","last_clean":"2023-10-06T10:43:02.800213+0000","last_became_active":"2023-10-06T10:43:02.796852+0000","last_became_peered":"2023-10-06T10:43:02.796852+0000","last_unstale":"2023-10-06T10:43:02.800213+0000","last_undegraded":"2023-10-06T10:43:02.800213+0000","last_fullsized":"2023-10-06T10:43:02.800213+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:22.653083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.791624+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:43:02.791624+0000","last_peered":"2023-10-06T10:43:02.791624+0000","last_clean":"2023-10-06T10:43:02.791624+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:43:02.791624+0000","last_undegraded":"2023-10-06T10:43:02.791624+0000","last_fullsized":"2023-10-06T10:43:02.791624+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.344137+0000","last_change":"2023-10-06T10:43:03.344137+0000","last_active":"2023-10-06T10:43:03.344137+0000","last_peered":"2023-10-06T10:43:03.344137+0000","last_clean":"2023-10-06T10:43:03.344137+0000","last_became_active":"2023-10-06T10:43:03.343862+0000","last_became_peered":"2023-10-06T10:43:03.343862+0000","last_unstale":"2023-10-06T10:43:03.344137+0000","last_undegraded":"2023-10-06T10:43:03.344137+0000","last_fullsized":"2023-10-06T10:43:03.344137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:27.895051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"77'2","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.796664+0000","last_change":"2023-10-06T10:43:02.796564+0000","last_active":"2023-10-06T10:43:03.796664+0000","last_peered":"2023-10-06T10:43:03.796664+0000","last_clean":"2023-10-06T10:43:03.796664+0000","last_became_active":"2023-10-06T10:43:02.796316+0000","last_became_peered":"2023-10-06T10:43:02.796316+0000","last_unstale":"2023-10-06T10:43:03.796664+0000","last_undegraded":"2023-10-06T10:43:03.796664+0000","last_fullsized":"2023-10-06T10:43:03.796664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:52:25.427534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.791418+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:43:02.791418+0000","last_peered":"2023-10-06T10:43:02.791418+0000","last_clean":"2023-10-06T10:43:02.791418+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:43:02.791418+0000","last_undegraded":"2023-10-06T10:43:02.791418+0000","last_fullsized":"2023-10-06T10:43:02.791418+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":8,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T10:43:03.344483+0000","last_change":"2023-10-06T10:43:03.344483+0000","last_active":"2023-10-06T10:43:03.344483+0000","last_peered":"2023-10-06T10:43:03.344483+0000","last_clean":"2023-10-06T10:43:03.344483+0000","last_became_active":"2023-10-06T10:43:03.344193+0000","last_became_peered":"2023-10-06T10:43:03.344193+0000","last_unstale":"2023-10-06T10:43:03.344483+0000","last_undegraded":"2023-10-06T10:43:03.344483+0000","last_fullsized":"2023-10-06T10:43:03.344483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:02.778443+0000","last_clean_scrub_stamp":"2023-10-06T10:43:02.778443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:19.454862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.799554+0000","last_change":"2023-10-06T10:43:02.795287+0000","last_active":"2023-10-06T10:43:02.799554+0000","last_peered":"2023-10-06T10:43:02.799554+0000","last_clean":"2023-10-06T10:43:02.799554+0000","last_became_active":"2023-10-06T10:43:02.794762+0000","last_became_peered":"2023-10-06T10:43:02.794762+0000","last_unstale":"2023-10-06T10:43:02.799554+0000","last_undegraded":"2023-10-06T10:43:02.799554+0000","last_fullsized":"2023-10-06T10:43:02.799554+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:01.773585+0000","last_clean_scrub_stamp":"2023-10-06T10:43:01.773585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:20:22.057282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T10:43:02.798319+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:43:02.798319+0000","last_peered":"2023-10-06T10:43:02.798319+0000","last_clean":"2023-10-06T10:43:02.798319+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:43:02.798319+0000","last_undegraded":"2023-10-06T10:43:02.798319+0000","last_fullsized":"2023-10-06T10:43:02.798319+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298792,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073048,"statfs":{"total":96636764160,"available":96330801152,"internally_reserved":0,"allocated":1540096,"data_stored":1040246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.5,"15min":0.48399999999999999},"min":{"1min":0.29799999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.173,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.498},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.503,"15min":0.50900000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.65400000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.70899999999999996}]},{"osd":1,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.47299999999999998,"15min":0.45900000000000002},"min":{"1min":0.34300000000000003,"5min":0.252,"15min":0.252},"max":{"1min":0.64100000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.61699999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.66500000000000004}]},{"osd":2,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52700000000000002,"15min":0.50900000000000001},"min":{"1min":0.38700000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.503,"15min":0.49399999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.63800000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.68300000000000005}]},{"osd":3,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52900000000000003,"15min":0.50900000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.68400000000000005,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55100000000000005,"15min":0.54300000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.67100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.443,"15min":0.46899999999999997},"min":{"1min":0.27300000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.499,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.38},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.48999999999999999,"15min":0.505},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64100000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.311}]},{"osd":5,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.47099999999999997,"15min":0.48299999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.59299999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.52400000000000002,"15min":0.54600000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.61699999999999999,"5min":1.052,"15min":1.052},"last":0.33800000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54900000000000004,"15min":0.57099999999999995},"min":{"1min":0.35399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.63300000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.40600000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.42999999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298204,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073636,"statfs":{"total":96636764160,"available":96331403264,"internally_reserved":0,"allocated":937984,"data_stored":448455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.29299999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66100000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.48199999999999998,"15min":0.442},"min":{"1min":0.32500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.50900000000000001}]},{"osd":1,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.47899999999999998,"15min":0.44600000000000001},"min":{"1min":0.35699999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.67400000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.51400000000000001,"15min":0.47899999999999998},"min":{"1min":0.40100000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.58599999999999997}]},{"osd":2,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.497,"15min":0.47399999999999998},"min":{"1min":0.32500000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.72999999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.50900000000000001,"15min":0.47499999999999998},"min":{"1min":0.39200000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65300000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.54200000000000004}]},{"osd":3,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.51900000000000002,"15min":0.495},"min":{"1min":0.41199999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70599999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52800000000000002,"15min":0.51000000000000001},"min":{"1min":0.41099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64900000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.52700000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.47499999999999998,"15min":0.47699999999999998},"min":{"1min":0.35199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67400000000000004,"5min":0.745,"15min":0.745},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.48799999999999999,"15min":0.48599999999999999},"min":{"1min":0.36499999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.40000000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.48899999999999999,"15min":0.48499999999999999},"min":{"1min":0.34399999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53000000000000003,"15min":0.53600000000000003},"min":{"1min":0.38500000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.71399999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.495}]},{"osd":7,"last update":"Fri Oct 6 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.36299999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.64900000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54900000000000004,"15min":0.53900000000000003},"min":{"1min":0.40400000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.56999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298800,"kb_used_data":1512,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073040,"statfs":{"total":96636764160,"available":96330792960,"internally_reserved":0,"allocated":1548288,"data_stored":1040370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:42:20 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33200000000000002,"15min":0.32900000000000001},"min":{"1min":0.21099999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.49399999999999999,"5min":0.505,"15min":0.505},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.35199999999999998,"15min":0.34699999999999998},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.502,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.34599999999999997}]},{"osd":2,"last update":"Fri Oct 6 10:42:20 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.33200000000000002,"15min":0.32100000000000001},"min":{"1min":0.23200000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.47799999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.254},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.375,"15min":0.36499999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.52100000000000002,"5min":0.62,"15min":0.62},"last":0.33600000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:42:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.39200000000000002,"15min":0.379},"min":{"1min":0.29499999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58899999999999997,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.317},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.378,"15min":0.374},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.54300000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.27400000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:42:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.40799999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70399999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.441,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.754,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.45500000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:42:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.57199999999999995},"min":{"1min":0.41999999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77900000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56399999999999995,"15min":0.54600000000000004},"min":{"1min":0.42999999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79700000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.433}]},{"osd":6,"last update":"Fri Oct 6 10:42:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.44,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81200000000000006,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.40000000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77000000000000002,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.46500000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:42:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.57799999999999996},"min":{"1min":0.41099999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.77300000000000002,"5min":0.996,"15min":0.996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.47099999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84999999999999998,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.50600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298208,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073632,"statfs":{"total":96636764160,"available":96331399168,"internally_reserved":0,"allocated":942080,"data_stored":449808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.32300000000000001,"15min":0.318},"min":{"1min":0.218,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.39900000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35899999999999999,"15min":0.34399999999999997},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.44600000000000001,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.26700000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.311,"15min":0.30499999999999999},"min":{"1min":0.23200000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.46000000000000002,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.219},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34200000000000003,"15min":0.32800000000000001},"min":{"1min":0.253,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.52800000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.30399999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.34799999999999998,"15min":0.34200000000000003},"min":{"1min":0.27300000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.42399999999999999,"5min":0.622,"15min":0.622},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35699999999999998,"15min":0.34999999999999998},"min":{"1min":0.27200000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.52700000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.27900000000000003}]},{"osd":4,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.42399999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76200000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.53500000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.70799999999999996,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.78100000000000003}]},{"osd":5,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.42199999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.748,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70999999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.64800000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57499999999999996,"15min":0.58399999999999996},"min":{"1min":0.45400000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.71699999999999997,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.68999999999999995,"5min":1.075,"15min":1.075},"last":0.58899999999999997}]},{"osd":7,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.434,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72199999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.45900000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72199999999999998,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.751}]}]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298212,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073628,"statfs":{"total":96636764160,"available":96331395072,"internally_reserved":0,"allocated":946176,"data_stored":450362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.311,"15min":0.31},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.38900000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.33800000000000002,"15min":0.32100000000000001},"min":{"1min":0.23699999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.441,"5min":0.622,"15min":0.622},"last":0.35599999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.33000000000000002,"15min":0.32400000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.46700000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.36499999999999999,"15min":0.36399999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47899999999999998,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.38700000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.35999999999999999,"15min":0.34799999999999998},"min":{"1min":0.25700000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.49299999999999999,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.311},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36599999999999999,"15min":0.35799999999999998},"min":{"1min":0.27700000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.48399999999999999,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.33800000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55200000000000005,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.77300000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.432,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76300000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.5}]},{"osd":5,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.42099999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.79000000000000004,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71699999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.44700000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.746,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.45600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80400000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.434}]},{"osd":7,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59599999999999997,"15min":0.60999999999999999},"min":{"1min":0.39600000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72899999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61199999999999999,"15min":0.623},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80000000000000004,"5min":0.874,"15min":0.874},"last":0.49099999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574910,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298220,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073620,"statfs":{"total":96636764160,"available":96331386880,"internally_reserved":0,"allocated":954368,"data_stored":451344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.29099999999999998,"5min":0.374,"15min":0.435},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.42799999999999999,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.35199999999999998,"15min":0.35899999999999999},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.53200000000000003,"5min":1.218,"15min":1.218},"last":0.39100000000000001}]},{"osd":1,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33600000000000002,"15min":0.33800000000000002},"min":{"1min":0.23100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.44700000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.41799999999999998,"15min":0.47399999999999998},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.51300000000000001,"5min":3.359,"15min":3.359},"last":0.28000000000000003}]},{"osd":2,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.37,"15min":0.379},"min":{"1min":0.255,"5min":0.215,"15min":0.215},"max":{"1min":0.45900000000000002,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.50700000000000001,"15min":0.63},"min":{"1min":0.27200000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.51200000000000001,"5min":7.069,"15min":7.069},"last":0.376}]},{"osd":4,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57299999999999995,"15min":0.58499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78200000000000003,"5min":1.165,"15min":1.165},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.40100000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.79100000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.70399999999999996}]},{"osd":5,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.64000000000000001,"15min":0.69399999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84499999999999997,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58699999999999997,"15min":0.60399999999999998},"min":{"1min":0.39500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83299999999999996,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.80000000000000004}]},{"osd":6,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60599999999999998,"15min":0.61399999999999999},"min":{"1min":0.41699999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.77600000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.67300000000000004,"15min":0.72399999999999998},"min":{"1min":0.442,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80100000000000005,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.81799999999999995}]},{"osd":7,"last update":"Fri Oct 6 10:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.40899999999999997,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86699999999999999,"5min":1.137,"15min":1.137},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60699999999999998,"15min":0.61099999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82499999999999996,"5min":1.014,"15min":1.014},"last":0.73499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298208,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073632,"statfs":{"total":96636764160,"available":96331399168,"internally_reserved":0,"allocated":942080,"data_stored":449808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.496,"15min":0.48599999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.74299999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.504},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51600000000000001,"15min":0.502},"min":{"1min":0.40600000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71399999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.35299999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50600000000000001,"15min":0.5},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68799999999999994,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.52300000000000002,"15min":0.53600000000000003},"min":{"1min":0.34499999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70299999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.56100000000000005}]},{"osd":2,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68999999999999995,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.52000000000000002},"min":{"1min":0.40300000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.46100000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53800000000000003,"15min":0.52000000000000002},"min":{"1min":0.36599999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.38200000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72699999999999998,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.53500000000000003}]},{"osd":5,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47199999999999998,"15min":0.5},"min":{"1min":0.28899999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77800000000000002,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49199999999999999,"15min":0.501},"min":{"1min":0.32000000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.69599999999999995,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.52100000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51900000000000002,"15min":0.54100000000000004},"min":{"1min":0.35199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55800000000000005,"15min":0.57599999999999996},"min":{"1min":0.36199999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.88,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.59699999999999998}]},{"osd":7,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.57499999999999996},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57199999999999995,"15min":0.59599999999999997},"min":{"1min":0.38600000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.749,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.61099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073616,"statfs":{"total":96636764160,"available":96331382784,"internally_reserved":0,"allocated":958464,"data_stored":451366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.49399999999999999,"15min":0.46999999999999997},"min":{"1min":0.28999999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68999999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.52600000000000002,"15min":0.53000000000000003},"min":{"1min":0.34499999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.73399999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.47199999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.317,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70399999999999996,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.622},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52900000000000003,"15min":0.51800000000000002},"min":{"1min":0.34699999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.79900000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.504}]},{"osd":2,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52500000000000002,"15min":0.504},"min":{"1min":0.308,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74299999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56100000000000005,"15min":0.56499999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52700000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.42099999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.77800000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53600000000000003,"15min":0.55400000000000005},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69299999999999995,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.58299999999999996}]},{"osd":4,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47499999999999998,"15min":0.48499999999999999},"min":{"1min":0.32600000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.75600000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.498,"15min":0.50600000000000001},"min":{"1min":0.31900000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.62,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.63800000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.501,"15min":0.51700000000000002},"min":{"1min":0.32600000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.67900000000000005,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.65400000000000003}]},{"osd":7,"last update":"Fri Oct 6 10:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56100000000000005,"15min":0.57899999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70199999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55400000000000005,"15min":0.55700000000000005},"min":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.747,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.69999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:43:08.236 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T10:43:08.236 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T10:43:08.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.390+0000 7f9f5f9d3700 1 Processor -- start 2023-10-06T10:43:08.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.390+0000 7f9f5f9d3700 1 -- start start 2023-10-06T10:43:08.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.391+0000 7f9f5f9d3700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9f581594d0 0x7f9f580cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:08.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.391+0000 7f9f5f9d3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f580cbaa0 0x7f9f580ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:08.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.391+0000 7f9f5f9d3700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f580cbee0 con 0x7f9f581598a0 2023-10-06T10:43:08.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.391+0000 7f9f5f9d3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f580cf5d0 con 0x7f9f580cbaa0 2023-10-06T10:43:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.391+0000 7f9f5f9d3700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f580cf8b0 con 0x7f9f581594d0 2023-10-06T10:43:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.391+0000 7f9f5df70700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f580cbaa0 0x7f9f580ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.391+0000 7f9f5df70700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f580cbaa0 0x7f9f580ceef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33940/0 (socket says 172.21.15.92:33940) 2023-10-06T10:43:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.391+0000 7f9f5df70700 1 -- 172.21.15.92:0/2510618868 learned_addr learned my addr 172.21.15.92:0/2510618868 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.391+0000 7f9f5d76f700 1 --2- 172.21.15.92:0/2510618868 >> v2:172.21.15.124:3300/0 conn(0x7f9f581594d0 0x7f9f580cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.391+0000 7f9f5df70700 1 -- 172.21.15.92:0/2510618868 >> v2:172.21.15.124:3300/0 conn(0x7f9f581594d0 msgr2=0x7f9f580cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.391+0000 7f9f5df70700 1 --2- 172.21.15.92:0/2510618868 >> v2:172.21.15.124:3300/0 conn(0x7f9f581594d0 0x7f9f580cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.391+0000 7f9f5df70700 1 -- 172.21.15.92:0/2510618868 >> v1:172.21.15.124:6789/0 conn(0x7f9f581598a0 legacy=0x7f9f580ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.392+0000 7f9f5df70700 1 -- 172.21.15.92:0/2510618868 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f580cfb90 con 0x7f9f580cbaa0 2023-10-06T10:43:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.392+0000 7f9f5df70700 1 --2- 172.21.15.92:0/2510618868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f580cbaa0 0x7f9f580ceef0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f9f58088f70 tx=0x7f9f50009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7100a28bffc27c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.392+0000 7f9f4ffff700 1 -- 172.21.15.92:0/2510618868 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5000e070 con 0x7f9f580cbaa0 2023-10-06T10:43:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.392+0000 7f9f4ffff700 1 -- 172.21.15.92:0/2510618868 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5000ba50 con 0x7f9f580cbaa0 2023-10-06T10:43:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.392+0000 7f9f4ffff700 1 -- 172.21.15.92:0/2510618868 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f500092e0 con 0x7f9f580cbaa0 2023-10-06T10:43:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.392+0000 7f9f5f9d3700 1 -- 172.21.15.92:0/2510618868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f580cbaa0 msgr2=0x7f9f580ceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.392+0000 7f9f5f9d3700 1 --2- 172.21.15.92:0/2510618868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f580cbaa0 0x7f9f580ceef0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f9f58088f70 tx=0x7f9f50009840 comp rx=0 tx=0).stop 2023-10-06T10:43:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.392+0000 7f9f5f9d3700 1 -- 172.21.15.92:0/2510618868 shutdown_connections 2023-10-06T10:43:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.392+0000 7f9f5f9d3700 1 --2- 172.21.15.92:0/2510618868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f580cbaa0 0x7f9f580ceef0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.392+0000 7f9f5f9d3700 1 --2- 172.21.15.92:0/2510618868 >> v2:172.21.15.124:3300/0 conn(0x7f9f581594d0 0x7f9f580cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.392+0000 7f9f5f9d3700 1 -- 172.21.15.92:0/2510618868 >> 172.21.15.92:0/2510618868 conn(0x7f9f58089e50 msgr2=0x7f9f58099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.392+0000 7f9f5f9d3700 1 -- 172.21.15.92:0/2510618868 shutdown_connections 2023-10-06T10:43:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.392+0000 7f9f5f9d3700 1 -- 172.21.15.92:0/2510618868 wait complete. 2023-10-06T10:43:08.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.393+0000 7f9f5f9d3700 1 Processor -- start 2023-10-06T10:43:08.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.394+0000 7f9f5f9d3700 1 -- start start 2023-10-06T10:43:08.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.394+0000 7f9f5f9d3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f580cbaa0 0x7f9f58149f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:08.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.394+0000 7f9f5f9d3700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f581594d0 0x7f9f5815bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:08.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.394+0000 7f9f5f9d3700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f581598a0 0x7f9f5815c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:08.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.394+0000 7f9f5f9d3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5815cbb0 con 0x7f9f580cbaa0 2023-10-06T10:43:08.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.394+0000 7f9f5f9d3700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5815b370 con 0x7f9f581598a0 2023-10-06T10:43:08.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.394+0000 7f9f5f9d3700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5815aa40 con 0x7f9f581594d0 2023-10-06T10:43:08.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.394+0000 7f9f5d76f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f580cbaa0 0x7f9f58149f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:08.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.395+0000 7f9f5d76f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f580cbaa0 0x7f9f58149f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33948/0 (socket says 172.21.15.92:33948) 2023-10-06T10:43:08.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.395+0000 7f9f5d76f700 1 -- 172.21.15.92:0/2952429976 learned_addr learned my addr 172.21.15.92:0/2952429976 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:08.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.395+0000 7f9f5cf6e700 1 --2- 172.21.15.92:0/2952429976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f581594d0 0x7f9f5815bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:08.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.395+0000 7f9f5df70700 1 --2- 172.21.15.92:0/2952429976 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f581598a0 0x7f9f5815c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:08.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.395+0000 7f9f5d76f700 1 -- 172.21.15.92:0/2952429976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f581594d0 msgr2=0x7f9f5815bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:08.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.395+0000 7f9f5d76f700 1 --2- 172.21.15.92:0/2952429976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f581594d0 0x7f9f5815bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:08.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.395+0000 7f9f5d76f700 1 -- 172.21.15.92:0/2952429976 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f581598a0 msgr2=0x7f9f5815c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:08.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.395+0000 7f9f5d76f700 1 --2- 172.21.15.92:0/2952429976 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f581598a0 0x7f9f5815c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:08.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.395+0000 7f9f5d76f700 1 -- 172.21.15.92:0/2952429976 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f50009ae0 con 0x7f9f580cbaa0 2023-10-06T10:43:08.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.395+0000 7f9f5df70700 1 --2- 172.21.15.92:0/2952429976 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f581598a0 0x7f9f5815c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:08.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.395+0000 7f9f5d76f700 1 --2- 172.21.15.92:0/2952429976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f580cbaa0 0x7f9f58149f30 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400d890 tx=0x7f9f54008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:08.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.395+0000 7f9f4d7fa700 1 -- 172.21.15.92:0/2952429976 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f540125f0 con 0x7f9f580cbaa0 2023-10-06T10:43:08.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.395+0000 7f9f4d7fa700 1 -- 172.21.15.92:0/2952429976 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f54012750 con 0x7f9f580cbaa0 2023-10-06T10:43:08.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.395+0000 7f9f5f9d3700 1 -- 172.21.15.92:0/2952429976 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5815ad20 con 0x7f9f580cbaa0 2023-10-06T10:43:08.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.395+0000 7f9f4d7fa700 1 -- 172.21.15.92:0/2952429976 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54012ad0 con 0x7f9f580cbaa0 2023-10-06T10:43:08.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.395+0000 7f9f5f9d3700 1 -- 172.21.15.92:0/2952429976 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5815a700 con 0x7f9f580cbaa0 2023-10-06T10:43:08.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.396+0000 7f9f5f9d3700 1 -- 172.21.15.92:0/2952429976 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f58064a90 con 0x7f9f580cbaa0 2023-10-06T10:43:08.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.396+0000 7f9f4d7fa700 1 -- 172.21.15.92:0/2952429976 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f5401e460 con 0x7f9f580cbaa0 2023-10-06T10:43:08.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.397+0000 7f9f4d7fa700 1 --2- 172.21.15.92:0/2952429976 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f3c066f30 0x7f9f3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:08.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.397+0000 7f9f4d7fa700 1 -- 172.21.15.92:0/2952429976 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9f54084df0 con 0x7f9f580cbaa0 2023-10-06T10:43:08.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.397+0000 7f9f5cf6e700 1 --2- 172.21.15.92:0/2952429976 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f3c066f30 0x7f9f3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:08.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.397+0000 7f9f5cf6e700 1 --2- 172.21.15.92:0/2952429976 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f3c066f30 0x7f9f3c0693e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9f580cad00 tx=0x7f9f48007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:08.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.401+0000 7f9f4d7fa700 1 -- 172.21.15.92:0/2952429976 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f54051dc0 con 0x7f9f580cbaa0 2023-10-06T10:43:08.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.658+0000 7f9f5f9d3700 1 -- 172.21.15.92:0/2952429976 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9f580cfe70 con 0x7f9f580cbaa0 2023-10-06T10:43:08.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.664+0000 7f9f4d7fa700 1 -- 172.21.15.92:0/2952429976 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7f9f540553e0 con 0x7f9f580cbaa0 2023-10-06T10:43:08.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.666+0000 7f9f5f9d3700 1 -- 172.21.15.92:0/2952429976 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f3c066f30 msgr2=0x7f9f3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:08.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.666+0000 7f9f5f9d3700 1 --2- 172.21.15.92:0/2952429976 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f3c066f30 0x7f9f3c0693e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9f580cad00 tx=0x7f9f48007ec0 comp rx=0 tx=0).stop 2023-10-06T10:43:08.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.666+0000 7f9f5f9d3700 1 -- 172.21.15.92:0/2952429976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f580cbaa0 msgr2=0x7f9f58149f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:08.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.666+0000 7f9f5f9d3700 1 --2- 172.21.15.92:0/2952429976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f580cbaa0 0x7f9f58149f30 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400d890 tx=0x7f9f54008040 comp rx=0 tx=0).stop 2023-10-06T10:43:08.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.666+0000 7f9f5f9d3700 1 -- 172.21.15.92:0/2952429976 shutdown_connections 2023-10-06T10:43:08.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.666+0000 7f9f5f9d3700 1 --2- 172.21.15.92:0/2952429976 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f3c066f30 0x7f9f3c0693e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:08.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.666+0000 7f9f5f9d3700 1 --2- 172.21.15.92:0/2952429976 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f581598a0 0x7f9f5815c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:08.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.666+0000 7f9f5f9d3700 1 --2- 172.21.15.92:0/2952429976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f580cbaa0 0x7f9f58149f30 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:08.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.666+0000 7f9f5f9d3700 1 --2- 172.21.15.92:0/2952429976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f581594d0 0x7f9f5815bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:08.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.666+0000 7f9f5f9d3700 1 -- 172.21.15.92:0/2952429976 >> 172.21.15.92:0/2952429976 conn(0x7f9f58089e50 msgr2=0x7f9f580cad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:08.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.666+0000 7f9f5f9d3700 1 -- 172.21.15.92:0/2952429976 shutdown_connections 2023-10-06T10:43:08.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.666+0000 7f9f5f9d3700 1 -- 172.21.15.92:0/2952429976 wait complete. 2023-10-06T10:43:08.669 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-10-06T10:43:08.680 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:43:08.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.792+0000 7fb934fdc700 1 Processor -- start 2023-10-06T10:43:08.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.792+0000 7fb934fdc700 1 -- start start 2023-10-06T10:43:08.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.793+0000 7fb934fdc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb9300b0760 0x7fb9300ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:08.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.793+0000 7fb934fdc700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9300b0c60 con 0x7fb930159560 2023-10-06T10:43:08.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.793+0000 7fb934fdc700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9300b0f40 con 0x7fb9300b49a0 2023-10-06T10:43:08.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.793+0000 7fb934fdc700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9300ab350 con 0x7fb9300b0760 2023-10-06T10:43:08.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.793+0000 7fb92ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb9300b0760 0x7fb9300ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:08.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.793+0000 7fb92e59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb930159560 0x7fb9300aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51878/0 (socket says 172.21.15.92:51878) 2023-10-06T10:43:08.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.793+0000 7fb92ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb9300b0760 0x7fb9300ccf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33962/0 (socket says 172.21.15.92:33962) 2023-10-06T10:43:08.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.793+0000 7fb92e59c700 1 -- 172.21.15.92:0/2490913207 learned_addr learned my addr 172.21.15.92:0/2490913207 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:08.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.793+0000 7fb92ed9d700 1 -- 172.21.15.92:0/2490913207 >> v1:172.21.15.124:6789/0 conn(0x7fb9300b49a0 legacy=0x7fb9300aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:08.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.793+0000 7fb92ed9d700 1 -- 172.21.15.92:0/2490913207 >> v1:172.21.15.92:6789/0 conn(0x7fb930159560 legacy=0x7fb9300aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:08.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.793+0000 7fb92dd9b700 1 -- 172.21.15.92:0/2490913207 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 450892063 0 0) 0x7fb9300b0c60 con 0x7fb930159560 2023-10-06T10:43:08.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.794+0000 7fb92ed9d700 1 -- 172.21.15.92:0/2490913207 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9300b1220 con 0x7fb9300b0760 2023-10-06T10:43:08.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.794+0000 7fb92ed9d700 1 --2- 172.21.15.92:0/2490913207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9300b0760 0x7fb9300ccf90 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fb928009e90 tx=0x7fb928009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aaa2327452bbcec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:08.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.794+0000 7fb92dd9b700 1 -- 172.21.15.92:0/2490913207 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92800e070 con 0x7fb9300b0760 2023-10-06T10:43:08.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.794+0000 7fb92dd9b700 1 -- 172.21.15.92:0/2490913207 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92800bbb0 con 0x7fb9300b0760 2023-10-06T10:43:08.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.794+0000 7fb92dd9b700 1 -- 172.21.15.92:0/2490913207 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928012430 con 0x7fb9300b0760 2023-10-06T10:43:08.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.794+0000 7fb934fdc700 1 -- 172.21.15.92:0/2490913207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9300b0760 msgr2=0x7fb9300ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:08.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.794+0000 7fb934fdc700 1 --2- 172.21.15.92:0/2490913207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9300b0760 0x7fb9300ccf90 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fb928009e90 tx=0x7fb928009780 comp rx=0 tx=0).stop 2023-10-06T10:43:08.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.794+0000 7fb934fdc700 1 -- 172.21.15.92:0/2490913207 shutdown_connections 2023-10-06T10:43:08.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.794+0000 7fb934fdc700 1 --2- 172.21.15.92:0/2490913207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9300b0760 0x7fb9300ccf90 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:08.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.794+0000 7fb934fdc700 1 -- 172.21.15.92:0/2490913207 >> 172.21.15.92:0/2490913207 conn(0x7fb9300997f0 msgr2=0x7fb93006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:08.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.794+0000 7fb934fdc700 1 -- 172.21.15.92:0/2490913207 shutdown_connections 2023-10-06T10:43:08.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.794+0000 7fb934fdc700 1 -- 172.21.15.92:0/2490913207 wait complete. 2023-10-06T10:43:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.796+0000 7fb934fdc700 1 Processor -- start 2023-10-06T10:43:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.796+0000 7fb934fdc700 1 -- start start 2023-10-06T10:43:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.796+0000 7fb934fdc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9300b0760 0x7fb930158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.796+0000 7fb934fdc700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9300b49a0 0x7fb93009a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.796+0000 7fb934fdc700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb930159560 0x7fb93015bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.796+0000 7fb934fdc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9301599d0 con 0x7fb9300b0760 2023-10-06T10:43:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.796+0000 7fb934fdc700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb930159cb0 con 0x7fb9300b49a0 2023-10-06T10:43:08.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.796+0000 7fb934fdc700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb930159f90 con 0x7fb930159560 2023-10-06T10:43:08.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.796+0000 7fb92e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9300b0760 0x7fb930158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:08.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.796+0000 7fb92e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9300b0760 0x7fb930158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33970/0 (socket says 172.21.15.92:33970) 2023-10-06T10:43:08.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.797+0000 7fb92e59c700 1 -- 172.21.15.92:0/2294119271 learned_addr learned my addr 172.21.15.92:0/2294119271 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:08.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.797+0000 7fb92ed9d700 1 --2- 172.21.15.92:0/2294119271 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb930159560 0x7fb93015bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:08.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.797+0000 7fb925d9b700 1 --2- 172.21.15.92:0/2294119271 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9300b49a0 0x7fb93009a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:08.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.797+0000 7fb92e59c700 1 -- 172.21.15.92:0/2294119271 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb930159560 msgr2=0x7fb93015bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:08.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.797+0000 7fb92e59c700 1 --2- 172.21.15.92:0/2294119271 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb930159560 0x7fb93015bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:08.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.797+0000 7fb92e59c700 1 -- 172.21.15.92:0/2294119271 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9300b49a0 msgr2=0x7fb93009a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:08.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.797+0000 7fb92e59c700 1 --2- 172.21.15.92:0/2294119271 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9300b49a0 0x7fb93009a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:08.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.797+0000 7fb92e59c700 1 -- 172.21.15.92:0/2294119271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb928009a20 con 0x7fb9300b0760 2023-10-06T10:43:08.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.797+0000 7fb92e59c700 1 --2- 172.21.15.92:0/2294119271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9300b0760 0x7fb930158920 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fb92000c5e0 tx=0x7fb920007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.797+0000 7fb926ffd700 1 -- 172.21.15.92:0/2294119271 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9200072e0 con 0x7fb9300b0760 2023-10-06T10:43:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.797+0000 7fb926ffd700 1 -- 172.21.15.92:0/2294119271 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb920007460 con 0x7fb9300b0760 2023-10-06T10:43:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.797+0000 7fb934fdc700 1 -- 172.21.15.92:0/2294119271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb93015a270 con 0x7fb9300b0760 2023-10-06T10:43:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.797+0000 7fb926ffd700 1 -- 172.21.15.92:0/2294119271 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9200057f0 con 0x7fb9300b0760 2023-10-06T10:43:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.797+0000 7fb934fdc700 1 -- 172.21.15.92:0/2294119271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb930158e60 con 0x7fb9300b0760 2023-10-06T10:43:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.798+0000 7fb934fdc700 1 -- 172.21.15.92:0/2294119271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb930064a90 con 0x7fb9300b0760 2023-10-06T10:43:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.798+0000 7fb926ffd700 1 -- 172.21.15.92:0/2294119271 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb920005c00 con 0x7fb9300b0760 2023-10-06T10:43:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.799+0000 7fb926ffd700 1 --2- 172.21.15.92:0/2294119271 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb910066f30 0x7fb9100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.799+0000 7fb926ffd700 1 -- 172.21.15.92:0/2294119271 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb920083c50 con 0x7fb9300b0760 2023-10-06T10:43:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.799+0000 7fb925d9b700 1 --2- 172.21.15.92:0/2294119271 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb910066f30 0x7fb9100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.799+0000 7fb925d9b700 1 --2- 172.21.15.92:0/2294119271 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb910066f30 0x7fb9100693e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb9300b6280 tx=0x7fb918007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:08.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:08.803+0000 7fb926ffd700 1 -- 172.21.15.92:0/2294119271 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb920089020 con 0x7fb9300b0760 2023-10-06T10:43:09.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.029+0000 7fb934fdc700 1 -- 172.21.15.92:0/2294119271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9300b1500 con 0x7fb9300b0760 2023-10-06T10:43:09.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.029+0000 7fb926ffd700 1 -- 172.21.15.92:0/2294119271 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fb920017070 con 0x7fb9300b0760 2023-10-06T10:43:09.031 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:09.031 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":67,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3469511807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3469511807},{"type":"v1","addr":"172.21.15.124:6834","nonce":3469511807}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7146,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/4003376156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4003376156},{"type":"v1","addr":"172.21.15.92:6837","nonce":4003376156}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7149,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3268908749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3268908749},{"type":"v1","addr":"172.21.15.92:6836","nonce":3268908749}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:03.173076+0000","modified":"2023-10-06T10:43:06.807414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7164},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7164":{"gid":7164,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/2971348516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2971348516},{"type":"v1","addr":"172.21.15.124:6835","nonce":2971348516}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T10:43:09.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.032+0000 7fb934fdc700 1 -- 172.21.15.92:0/2294119271 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb910066f30 msgr2=0x7fb9100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:09.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.032+0000 7fb934fdc700 1 --2- 172.21.15.92:0/2294119271 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb910066f30 0x7fb9100693e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb9300b6280 tx=0x7fb918007ec0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.032+0000 7fb934fdc700 1 -- 172.21.15.92:0/2294119271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9300b0760 msgr2=0x7fb930158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:09.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.032+0000 7fb934fdc700 1 --2- 172.21.15.92:0/2294119271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9300b0760 0x7fb930158920 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fb92000c5e0 tx=0x7fb920007e80 comp rx=0 tx=0).stop 2023-10-06T10:43:09.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.032+0000 7fb934fdc700 1 -- 172.21.15.92:0/2294119271 shutdown_connections 2023-10-06T10:43:09.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.032+0000 7fb934fdc700 1 --2- 172.21.15.92:0/2294119271 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb910066f30 0x7fb9100693e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.032+0000 7fb934fdc700 1 --2- 172.21.15.92:0/2294119271 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb930159560 0x7fb93015bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.032+0000 7fb934fdc700 1 --2- 172.21.15.92:0/2294119271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9300b0760 0x7fb930158920 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.032+0000 7fb934fdc700 1 --2- 172.21.15.92:0/2294119271 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9300b49a0 0x7fb93009a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.032+0000 7fb934fdc700 1 -- 172.21.15.92:0/2294119271 >> 172.21.15.92:0/2294119271 conn(0x7fb9300997f0 msgr2=0x7fb930069ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:09.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.032+0000 7fb934fdc700 1 -- 172.21.15.92:0/2294119271 shutdown_connections 2023-10-06T10:43:09.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.032+0000 7fb934fdc700 1 -- 172.21.15.92:0/2294119271 wait complete. 2023-10-06T10:43:09.035 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 67 2023-10-06T10:43:09.448 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T10:43:03.173076+0000', 'modified': '2023-10-06T10:43:06.807414+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7164}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7164': {'gid': 7164, 'name': 'd', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.124:6835/2971348516', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6833', 'nonce': 2971348516}, {'type': 'v1', 'addr': '172.21.15.124:6835', 'nonce': 2971348516}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T10:43:09.448 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T10:43:09.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T10:43:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.562+0000 7f7c8eef5700 1 Processor -- start 2023-10-06T10:43:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.562+0000 7f7c8eef5700 1 -- start start 2023-10-06T10:43:09.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.563+0000 7f7c8eef5700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7c88159440 0x7f7c880b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.563+0000 7f7c8eef5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c880cb7a0 con 0x7f7c880b35d0 2023-10-06T10:43:09.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.563+0000 7f7c8eef5700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c880b0860 con 0x7f7c880b2100 2023-10-06T10:43:09.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.563+0000 7f7c8eef5700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c880cb150 con 0x7f7c88159440 2023-10-06T10:43:09.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.563+0000 7f7c8d6f2700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7c880b35d0 0x7f7c880cbc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51882/0 (socket says 172.21.15.92:51882) 2023-10-06T10:43:09.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.563+0000 7f7c8d6f2700 1 -- 172.21.15.92:0/753130146 learned_addr learned my addr 172.21.15.92:0/753130146 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:09.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.563+0000 7f7c8def3700 1 --2- 172.21.15.92:0/753130146 >> v2:172.21.15.124:3300/0 conn(0x7f7c88159440 0x7f7c880b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.563+0000 7f7c8cef1700 1 -- 172.21.15.92:0/753130146 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2317254636 0 0) 0x7f7c880cb7a0 con 0x7f7c880b35d0 2023-10-06T10:43:09.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.563+0000 7f7c8cef1700 1 -- 172.21.15.92:0/753130146 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c74003570 con 0x7f7c880b35d0 2023-10-06T10:43:09.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8cef1700 1 -- 172.21.15.92:0/753130146 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3479696754 0 0) 0x7f7c74003570 con 0x7f7c880b35d0 2023-10-06T10:43:09.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8cef1700 1 -- 172.21.15.92:0/753130146 >> v2:172.21.15.124:3300/0 conn(0x7f7c88159440 msgr2=0x7f7c880b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:09.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8cef1700 1 --2- 172.21.15.92:0/753130146 >> v2:172.21.15.124:3300/0 conn(0x7f7c88159440 0x7f7c880b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8cef1700 1 -- 172.21.15.92:0/753130146 >> v1:172.21.15.124:6789/0 conn(0x7f7c880b2100 legacy=0x7f7c880b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:09.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8cef1700 1 -- 172.21.15.92:0/753130146 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c880cb430 con 0x7f7c880b35d0 2023-10-06T10:43:09.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8cef1700 1 -- 172.21.15.92:0/753130146 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f7c78001470 con 0x7f7c880b35d0 2023-10-06T10:43:09.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8def3700 1 --2- 172.21.15.92:0/753130146 >> v2:172.21.15.124:3300/0 conn(0x7f7c88159440 0x7f7c880b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:43:09.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8cef1700 1 -- 172.21.15.92:0/753130146 >> v1:172.21.15.92:6789/0 conn(0x7f7c880b35d0 legacy=0x7f7c880cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:09.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8cef1700 1 --2- 172.21.15.92:0/753130146 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c74003a60 0x7f7c74003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8cef1700 1 --2- 172.21.15.92:0/753130146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c74004810 0x7f7c74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8cef1700 1 --2- 172.21.15.92:0/753130146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c740051c0 0x7f7c7400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8cef1700 1 -- 172.21.15.92:0/753130146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c880cb150 con 0x7f7c740051c0 2023-10-06T10:43:09.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8cef1700 1 -- 172.21.15.92:0/753130146 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7400c160 con 0x7f7c74003a60 2023-10-06T10:43:09.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8cef1700 1 -- 172.21.15.92:0/753130146 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7400c320 con 0x7f7c74004810 2023-10-06T10:43:09.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8e6f4700 1 --2- 172.21.15.92:0/753130146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c740051c0 0x7f7c7400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8def3700 1 --2- 172.21.15.92:0/753130146 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c74003a60 0x7f7c74003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8d6f2700 1 --2- 172.21.15.92:0/753130146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c74004810 0x7f7c74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8e6f4700 1 -- 172.21.15.92:0/753130146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c74004810 msgr2=0x7f7c74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:09.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8e6f4700 1 --2- 172.21.15.92:0/753130146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c74004810 0x7f7c74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8e6f4700 1 -- 172.21.15.92:0/753130146 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c74003a60 msgr2=0x7f7c74003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:09.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8e6f4700 1 --2- 172.21.15.92:0/753130146 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c74003a60 0x7f7c74003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8e6f4700 1 -- 172.21.15.92:0/753130146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c74003570 con 0x7f7c740051c0 2023-10-06T10:43:09.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8e6f4700 1 -- 172.21.15.92:0/753130146 reap_dead start 2023-10-06T10:43:09.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8e6f4700 1 --2- 172.21.15.92:0/753130146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c740051c0 0x7f7c7400ba60 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f7c80001ff0 tx=0x7f7c80001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e95e30beaac5f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:09.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.564+0000 7f7c8cef1700 1 -- 172.21.15.92:0/753130146 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c80001980 con 0x7f7c740051c0 2023-10-06T10:43:09.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.565+0000 7f7c8cef1700 1 -- 172.21.15.92:0/753130146 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c8000c8b0 con 0x7f7c740051c0 2023-10-06T10:43:09.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.565+0000 7f7c8cef1700 1 -- 172.21.15.92:0/753130146 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8000cc90 con 0x7f7c740051c0 2023-10-06T10:43:09.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.565+0000 7f7c8eef5700 1 -- 172.21.15.92:0/753130146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c740051c0 msgr2=0x7f7c7400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:09.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.565+0000 7f7c8eef5700 1 --2- 172.21.15.92:0/753130146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c740051c0 0x7f7c7400ba60 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f7c80001ff0 tx=0x7f7c80001670 comp rx=0 tx=0).stop 2023-10-06T10:43:09.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.565+0000 7f7c8eef5700 1 -- 172.21.15.92:0/753130146 shutdown_connections 2023-10-06T10:43:09.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.565+0000 7f7c8eef5700 1 --2- 172.21.15.92:0/753130146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c740051c0 0x7f7c7400ba60 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.565+0000 7f7c8eef5700 1 -- 172.21.15.92:0/753130146 >> 172.21.15.92:0/753130146 conn(0x7f7c8806ece0 msgr2=0x7f7c8806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:09.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.565+0000 7f7c8eef5700 1 -- 172.21.15.92:0/753130146 shutdown_connections 2023-10-06T10:43:09.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.565+0000 7f7c8eef5700 1 -- 172.21.15.92:0/753130146 wait complete. 2023-10-06T10:43:09.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.566+0000 7f7c8eef5700 1 Processor -- start 2023-10-06T10:43:09.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.566+0000 7f7c8eef5700 1 -- start start 2023-10-06T10:43:09.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.567+0000 7f7c8eef5700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c740051c0 0x7f7c880c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.567+0000 7f7c8eef5700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c880c4570 0x7f7c880af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.567+0000 7f7c8eef5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c880cddf0 0x7f7c880ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.567+0000 7f7c8eef5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c880ce7d0 con 0x7f7c880cddf0 2023-10-06T10:43:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.567+0000 7f7c8eef5700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c88154f50 con 0x7f7c880c4570 2023-10-06T10:43:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.567+0000 7f7c8eef5700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c880c4b40 con 0x7f7c740051c0 2023-10-06T10:43:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.567+0000 7f7c8e6f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c880cddf0 0x7f7c880ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.567+0000 7f7c8e6f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c880cddf0 0x7f7c880ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33984/0 (socket says 172.21.15.92:33984) 2023-10-06T10:43:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.567+0000 7f7c8e6f4700 1 -- 172.21.15.92:0/3914163718 learned_addr learned my addr 172.21.15.92:0/3914163718 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.567+0000 7f7c8d6f2700 1 --2- 172.21.15.92:0/3914163718 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c880c4570 0x7f7c880af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.567+0000 7f7c8def3700 1 --2- 172.21.15.92:0/3914163718 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c740051c0 0x7f7c880c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.567+0000 7f7c8e6f4700 1 -- 172.21.15.92:0/3914163718 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c740051c0 msgr2=0x7f7c880c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:09.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.567+0000 7f7c8e6f4700 1 --2- 172.21.15.92:0/3914163718 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c740051c0 0x7f7c880c63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.567+0000 7f7c8e6f4700 1 -- 172.21.15.92:0/3914163718 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c880c4570 msgr2=0x7f7c880af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:09.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.567+0000 7f7c8e6f4700 1 --2- 172.21.15.92:0/3914163718 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c880c4570 0x7f7c880af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.567+0000 7f7c8e6f4700 1 -- 172.21.15.92:0/3914163718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c880c4e20 con 0x7f7c880cddf0 2023-10-06T10:43:09.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.567+0000 7f7c8d6f2700 1 --2- 172.21.15.92:0/3914163718 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c880c4570 0x7f7c880af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:09.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.567+0000 7f7c8e6f4700 1 --2- 172.21.15.92:0/3914163718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c880cddf0 0x7f7c880ce290 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000c6a0 tx=0x7f7c80015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:09.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.568+0000 7f7c7dffb700 1 -- 172.21.15.92:0/3914163718 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c80001980 con 0x7f7c880cddf0 2023-10-06T10:43:09.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.568+0000 7f7c7dffb700 1 -- 172.21.15.92:0/3914163718 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c8000c6d0 con 0x7f7c880cddf0 2023-10-06T10:43:09.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.568+0000 7f7c8eef5700 1 -- 172.21.15.92:0/3914163718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c880c5100 con 0x7f7c880cddf0 2023-10-06T10:43:09.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.568+0000 7f7c7dffb700 1 -- 172.21.15.92:0/3914163718 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8001e870 con 0x7f7c880cddf0 2023-10-06T10:43:09.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.568+0000 7f7c8eef5700 1 -- 172.21.15.92:0/3914163718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c880c6910 con 0x7f7c880cddf0 2023-10-06T10:43:09.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.569+0000 7f7c7dffb700 1 -- 172.21.15.92:0/3914163718 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7c8001ebe0 con 0x7f7c880cddf0 2023-10-06T10:43:09.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.570+0000 7f7c7dffb700 1 --2- 172.21.15.92:0/3914163718 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7c6c066f30 0x7f7c6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.570+0000 7f7c7dffb700 1 -- 172.21.15.92:0/3914163718 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7c80086420 con 0x7f7c880cddf0 2023-10-06T10:43:09.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.570+0000 7f7c8def3700 1 --2- 172.21.15.92:0/3914163718 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7c6c066f30 0x7f7c6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.570+0000 7f7c8def3700 1 --2- 172.21.15.92:0/3914163718 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7c6c066f30 0x7f7c6c0693e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7c84006f00 tx=0x7f7c84008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:09.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.571+0000 7f7c9944fb80 1 Processor -- start 2023-10-06T10:43:09.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.572+0000 7f7c9944fb80 1 -- start start 2023-10-06T10:43:09.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.572+0000 7f7c9944fb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561d3e1e2950 0x561d3e1e2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.572+0000 7f7c9944fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e22cbc0 0x561d3e20b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.572+0000 7f7c9944fb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561d3e20be60 0x561d3e1de990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.572+0000 7f7c9944fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d3e162c80 con 0x561d3e22cbc0 2023-10-06T10:43:09.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.572+0000 7f7c9944fb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d3e139a70 con 0x561d3e20be60 2023-10-06T10:43:09.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.572+0000 7f7c9944fb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d3e137270 con 0x561d3e1e2950 2023-10-06T10:43:09.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.572+0000 7f7c8d6f2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e22cbc0 0x561d3e20b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.572+0000 7f7c8d6f2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e22cbc0 0x561d3e20b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33988/0 (socket says 172.21.15.92:33988) 2023-10-06T10:43:09.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.572+0000 7f7c8d6f2700 1 -- 172.21.15.92:0/3375233504 learned_addr learned my addr 172.21.15.92:0/3375233504 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:09.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.572+0000 7f7c8e6f4700 1 --2- 172.21.15.92:0/3375233504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561d3e20be60 0x561d3e1de990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.572+0000 7f7c8def3700 1 --2- 172.21.15.92:0/3375233504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561d3e1e2950 0x561d3e1e2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.573+0000 7f7c8d6f2700 1 -- 172.21.15.92:0/3375233504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561d3e1e2950 msgr2=0x561d3e1e2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:09.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.573+0000 7f7c8d6f2700 1 --2- 172.21.15.92:0/3375233504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561d3e1e2950 0x561d3e1e2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.573+0000 7f7c8d6f2700 1 -- 172.21.15.92:0/3375233504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561d3e20be60 msgr2=0x561d3e1de990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:09.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.573+0000 7f7c8d6f2700 1 --2- 172.21.15.92:0/3375233504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561d3e20be60 0x561d3e1de990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.573+0000 7f7c8d6f2700 1 -- 172.21.15.92:0/3375233504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d3dfcd270 con 0x561d3e22cbc0 2023-10-06T10:43:09.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.573+0000 7f7c8e6f4700 1 --2- 172.21.15.92:0/3375233504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561d3e20be60 0x561d3e1de990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:09.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.573+0000 7f7c8def3700 1 --2- 172.21.15.92:0/3375233504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561d3e1e2950 0x561d3e1e2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.573+0000 7f7c8d6f2700 1 --2- 172.21.15.92:0/3375233504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e22cbc0 0x561d3e20b920 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f7c78007cf0 tx=0x7f7c78009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91108ddb6762b44b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.573+0000 7f7c72ffd700 1 -- 172.21.15.92:0/3375233504 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7801a070 con 0x561d3e22cbc0 2023-10-06T10:43:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.573+0000 7f7c72ffd700 1 -- 172.21.15.92:0/3375233504 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c78007460 con 0x561d3e22cbc0 2023-10-06T10:43:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.573+0000 7f7c72ffd700 1 -- 172.21.15.92:0/3375233504 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c780077e0 con 0x561d3e22cbc0 2023-10-06T10:43:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.573+0000 7f7c9944fb80 1 -- 172.21.15.92:0/3375233504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e22cbc0 msgr2=0x561d3e20b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.573+0000 7f7c9944fb80 1 --2- 172.21.15.92:0/3375233504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e22cbc0 0x561d3e20b920 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f7c78007cf0 tx=0x7f7c78009040 comp rx=0 tx=0).stop 2023-10-06T10:43:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.574+0000 7f7c9944fb80 1 -- 172.21.15.92:0/3375233504 shutdown_connections 2023-10-06T10:43:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.574+0000 7f7c9944fb80 1 --2- 172.21.15.92:0/3375233504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561d3e20be60 0x561d3e1de990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.574+0000 7f7c9944fb80 1 --2- 172.21.15.92:0/3375233504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561d3e1e2950 0x561d3e1e2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.574+0000 7f7c9944fb80 1 --2- 172.21.15.92:0/3375233504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e22cbc0 0x561d3e20b920 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.574+0000 7f7c9944fb80 1 -- 172.21.15.92:0/3375233504 >> 172.21.15.92:0/3375233504 conn(0x561d3e1e4270 msgr2=0x561d3e1e4640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.574+0000 7f7c9944fb80 1 -- 172.21.15.92:0/3375233504 shutdown_connections 2023-10-06T10:43:09.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.574+0000 7f7c9944fb80 1 -- 172.21.15.92:0/3375233504 wait complete. 2023-10-06T10:43:09.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.574+0000 7f7c9944fb80 1 -- start start 2023-10-06T10:43:09.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.574+0000 7f7c9944fb80 1 Processor -- start 2023-10-06T10:43:09.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.575+0000 7f7c9944fb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561d3e1e2950 0x561d3e221420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.575+0000 7f7c9944fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e20be60 0x561d3e23a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.575+0000 7f7c9944fb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561d3e22cbc0 0x561d3e223770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.575+0000 7f7c9944fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d3e133060 con 0x561d3e20be60 2023-10-06T10:43:09.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.575+0000 7f7c9944fb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d3e0de890 con 0x561d3e22cbc0 2023-10-06T10:43:09.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.575+0000 7f7c9944fb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d3e0f9ea0 con 0x561d3e1e2950 2023-10-06T10:43:09.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.575+0000 7f7c9944fb80 10 client.0 _ll_register_callbacks cb 0x561d3dfcff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:09.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.575+0000 7f7c8d6f2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e20be60 0x561d3e23a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.575+0000 7f7c8d6f2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e20be60 0x561d3e23a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34000/0 (socket says 172.21.15.92:34000) 2023-10-06T10:43:09.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.575+0000 7f7c8d6f2700 1 -- 172.21.15.92:0/3784253450 learned_addr learned my addr 172.21.15.92:0/3784253450 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:09.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.575+0000 7f7c8def3700 1 --2- 172.21.15.92:0/3784253450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561d3e1e2950 0x561d3e221420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.576+0000 7f7c8e6f4700 1 --2- 172.21.15.92:0/3784253450 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561d3e22cbc0 0x561d3e223770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.576+0000 7f7c8d6f2700 1 -- 172.21.15.92:0/3784253450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561d3e1e2950 msgr2=0x561d3e221420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:09.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.576+0000 7f7c8d6f2700 1 --2- 172.21.15.92:0/3784253450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561d3e1e2950 0x561d3e221420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.576+0000 7f7c8d6f2700 1 -- 172.21.15.92:0/3784253450 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561d3e22cbc0 msgr2=0x561d3e223770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:09.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.576+0000 7f7c8d6f2700 1 --2- 172.21.15.92:0/3784253450 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561d3e22cbc0 0x561d3e223770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.576+0000 7f7c8d6f2700 1 -- 172.21.15.92:0/3784253450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c880cb430 con 0x561d3e20be60 2023-10-06T10:43:09.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.576+0000 7f7c8d6f2700 1 --2- 172.21.15.92:0/3784253450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e20be60 0x561d3e23a230 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f7c78004ea0 tx=0x7f7c78016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4bc1ff422be72d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:09.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.576+0000 7f7c67fff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:09.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.576+0000 7f7c67fff700 1 -- 172.21.15.92:0/3784253450 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7801a030 con 0x561d3e20be60 2023-10-06T10:43:09.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.576+0000 7f7c67fff700 1 -- 172.21.15.92:0/3784253450 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7c78015aa0 con 0x561d3e20be60 2023-10-06T10:43:09.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.576+0000 7f7c9944fb80 1 -- 172.21.15.92:0/3784253450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d3e1d7690 con 0x561d3e20be60 2023-10-06T10:43:09.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.576+0000 7f7c67fff700 1 -- 172.21.15.92:0/3784253450 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c78015e70 con 0x561d3e20be60 2023-10-06T10:43:09.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.576+0000 7f7c9944fb80 10 client.7302 fetch_fsmap learned FSMap version 67 2023-10-06T10:43:09.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.576+0000 7f7c9944fb80 1 -- 172.21.15.92:0/3784253450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x561d3df6ffe0 con 0x561d3e20be60 2023-10-06T10:43:09.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.576+0000 7f7c67fff700 1 -- 172.21.15.92:0/3784253450 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561d3e1d7690 con 0x561d3e20be60 2023-10-06T10:43:09.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.576+0000 7f7c67fff700 1 -- 172.21.15.92:0/3784253450 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 67) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f7c78021430 con 0x561d3e20be60 2023-10-06T10:43:09.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.576+0000 7f7c9944fb80 10 client.7302 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T10:43:09.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.576+0000 7f7c9944fb80 10 client.7302 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T10:43:09.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.576+0000 7f7c9944fb80 20 client.7302 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:09.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.576+0000 7f7c9944fb80 1 --2- 172.21.15.92:0/3784253450 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x561d3e229aa0 0x561d3e229e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.576+0000 7f7c9944fb80 4 client.7302 mds_command: new command op to 7164 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:43:09.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.576+0000 7f7c9944fb80 1 -- 172.21.15.92:0/3784253450 --> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d3e1b45c0 con 0x561d3e229aa0 2023-10-06T10:43:09.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.577+0000 7f7c8def3700 1 --2- 172.21.15.92:0/3784253450 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x561d3e229aa0 0x561d3e229e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.577+0000 7f7c67fff700 10 client.7302 ms_handle_connect on v2:172.21.15.124:6833/2971348516 2023-10-06T10:43:09.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.577+0000 7f7c8def3700 1 --2- 172.21.15.92:0/3784253450 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x561d3e229aa0 0x561d3e229e70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42c648534454ac15 server_cookie=513b6269de880a90 in_seq=0 out_seq=0 2023-10-06T10:43:09.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.578+0000 7f7c67fff700 1 -- 172.21.15.92:0/3784253450 <== mds.0 v2:172.21.15.124:6833/2971348516 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x561d3e1b45c0 con 0x561d3e229aa0 2023-10-06T10:43:09.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.578+0000 7f7c67fff700 10 client.7302 handle_command_reply: tid=0 2023-10-06T10:43:09.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.578+0000 7f7c9944fb80 1 client.7302 shutdown 2023-10-06T10:43:09.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.578+0000 7f7c9944fb80 1 -- 172.21.15.92:0/3784253450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e20be60 msgr2=0x561d3e23a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.578+0000 7f7c9944fb80 1 --2- 172.21.15.92:0/3784253450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e20be60 0x561d3e23a230 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f7c78004ea0 tx=0x7f7c78016000 comp rx=0 tx=0).stop 2023-10-06T10:43:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.578+0000 7f7c9944fb80 1 -- 172.21.15.92:0/3784253450 shutdown_connections 2023-10-06T10:43:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.578+0000 7f7c9944fb80 1 --2- 172.21.15.92:0/3784253450 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561d3e22cbc0 0x561d3e223770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.578+0000 7f7c9944fb80 1 --2- 172.21.15.92:0/3784253450 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x561d3e229aa0 0x561d3e229e70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.578+0000 7f7c9944fb80 1 --2- 172.21.15.92:0/3784253450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561d3e1e2950 0x561d3e221420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.578+0000 7f7c9944fb80 1 --2- 172.21.15.92:0/3784253450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e20be60 0x561d3e23a230 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.578+0000 7f7c67fff700 0 client.7302 ms_handle_reset on v2:172.21.15.124:6833/2971348516 2023-10-06T10:43:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.578+0000 7f7c9944fb80 1 -- 172.21.15.92:0/3784253450 >> 172.21.15.92:0/3784253450 conn(0x561d3e1e4270 msgr2=0x561d3e1d91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.578+0000 7f7c9944fb80 1 -- 172.21.15.92:0/3784253450 shutdown_connections 2023-10-06T10:43:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.578+0000 7f7c9944fb80 1 -- 172.21.15.92:0/3784253450 wait complete. 2023-10-06T10:43:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.578+0000 7f7c9944fb80 20 client.7302 trim_cache size 0 max 16384 2023-10-06T10:43:09.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.595+0000 7f7c9944fb80 1 Processor -- start 2023-10-06T10:43:09.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.595+0000 7f7c9944fb80 1 -- start start 2023-10-06T10:43:09.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.595+0000 7f7c9944fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e1e2950 0x561d3e20b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.595+0000 7f7c9944fb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561d3e20be60 0x561d3e1d9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.595+0000 7f7c9944fb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561d3e22cbc0 0x561d3e1d98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.595+0000 7f7c9944fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d3dfb5e30 con 0x561d3e1e2950 2023-10-06T10:43:09.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.595+0000 7f7c9944fb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d3e2083f0 con 0x561d3e22cbc0 2023-10-06T10:43:09.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.595+0000 7f7c9944fb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d3e22a6a0 con 0x561d3e20be60 2023-10-06T10:43:09.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.595+0000 7f7c8def3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e1e2950 0x561d3e20b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.595+0000 7f7c8def3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e1e2950 0x561d3e20b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34002/0 (socket says 172.21.15.92:34002) 2023-10-06T10:43:09.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.595+0000 7f7c8def3700 1 -- 172.21.15.92:0/1502582825 learned_addr learned my addr 172.21.15.92:0/1502582825 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:09.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.596+0000 7f7c8d6f2700 1 --2- 172.21.15.92:0/1502582825 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561d3e20be60 0x561d3e1d9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.596+0000 7f7c8e6f4700 1 --2- 172.21.15.92:0/1502582825 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561d3e22cbc0 0x561d3e1d98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.596+0000 7f7c8def3700 1 -- 172.21.15.92:0/1502582825 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561d3e20be60 msgr2=0x561d3e1d9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:09.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.596+0000 7f7c8def3700 1 --2- 172.21.15.92:0/1502582825 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561d3e20be60 0x561d3e1d9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.596+0000 7f7c8def3700 1 -- 172.21.15.92:0/1502582825 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561d3e22cbc0 msgr2=0x561d3e1d98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:09.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.596+0000 7f7c8def3700 1 --2- 172.21.15.92:0/1502582825 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561d3e22cbc0 0x561d3e1d98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.596+0000 7f7c8def3700 1 -- 172.21.15.92:0/1502582825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d3e101780 con 0x561d3e1e2950 2023-10-06T10:43:09.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.596+0000 7f7c8e6f4700 1 --2- 172.21.15.92:0/1502582825 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561d3e22cbc0 0x561d3e1d98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:09.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.596+0000 7f7c8def3700 1 --2- 172.21.15.92:0/1502582825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e1e2950 0x561d3e20b250 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009d00 tx=0x7f7c84018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2fd1b52a24343 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:09.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.596+0000 7f7c737fe700 1 -- 172.21.15.92:0/1502582825 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8401c040 con 0x561d3e1e2950 2023-10-06T10:43:09.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.596+0000 7f7c737fe700 1 -- 172.21.15.92:0/1502582825 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c840280d0 con 0x561d3e1e2950 2023-10-06T10:43:09.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.596+0000 7f7c737fe700 1 -- 172.21.15.92:0/1502582825 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8402f6c0 con 0x561d3e1e2950 2023-10-06T10:43:09.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.596+0000 7f7c9944fb80 1 -- 172.21.15.92:0/1502582825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e1e2950 msgr2=0x561d3e20b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:09.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.596+0000 7f7c9944fb80 1 --2- 172.21.15.92:0/1502582825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e1e2950 0x561d3e20b250 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009d00 tx=0x7f7c84018000 comp rx=0 tx=0).stop 2023-10-06T10:43:09.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.597+0000 7f7c9944fb80 1 -- 172.21.15.92:0/1502582825 shutdown_connections 2023-10-06T10:43:09.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.597+0000 7f7c9944fb80 1 --2- 172.21.15.92:0/1502582825 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561d3e22cbc0 0x561d3e1d98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.597+0000 7f7c9944fb80 1 --2- 172.21.15.92:0/1502582825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e1e2950 0x561d3e20b250 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.597+0000 7f7c9944fb80 1 --2- 172.21.15.92:0/1502582825 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561d3e20be60 0x561d3e1d9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.597+0000 7f7c9944fb80 1 -- 172.21.15.92:0/1502582825 >> 172.21.15.92:0/1502582825 conn(0x561d3e1e4270 msgr2=0x561d3e221420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:09.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.597+0000 7f7c9944fb80 1 -- 172.21.15.92:0/1502582825 shutdown_connections 2023-10-06T10:43:09.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.597+0000 7f7c9944fb80 1 -- 172.21.15.92:0/1502582825 wait complete. 2023-10-06T10:43:09.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.597+0000 7f7c9944fb80 1 -- start start 2023-10-06T10:43:09.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.597+0000 7f7c9944fb80 1 Processor -- start 2023-10-06T10:43:09.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.598+0000 7f7c9944fb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561d3e1e2950 0x561d3e287580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.598+0000 7f7c9944fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e20be60 0x561d3e287ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.598+0000 7f7c9944fb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561d3e22cbc0 0x561d3e226460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.598+0000 7f7c9944fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d3e288000 con 0x561d3e20be60 2023-10-06T10:43:09.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.598+0000 7f7c9944fb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d3e1def20 con 0x561d3e22cbc0 2023-10-06T10:43:09.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.598+0000 7f7c9944fb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d3e285030 con 0x561d3e1e2950 2023-10-06T10:43:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.598+0000 7f7c9944fb80 10 client.0 _ll_register_callbacks cb 0x561d3dfcff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.598+0000 7f7c8d6f2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e20be60 0x561d3e287ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.598+0000 7f7c8d6f2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e20be60 0x561d3e287ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34004/0 (socket says 172.21.15.92:34004) 2023-10-06T10:43:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.598+0000 7f7c8d6f2700 1 -- 172.21.15.92:0/3594841986 learned_addr learned my addr 172.21.15.92:0/3594841986 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.598+0000 7f7c8def3700 1 --2- 172.21.15.92:0/3594841986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561d3e1e2950 0x561d3e287580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.598+0000 7f7c8d6f2700 1 -- 172.21.15.92:0/3594841986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561d3e1e2950 msgr2=0x561d3e287580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.598+0000 7f7c8d6f2700 1 --2- 172.21.15.92:0/3594841986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561d3e1e2950 0x561d3e287580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.598+0000 7f7c8e6f4700 1 --2- 172.21.15.92:0/3594841986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561d3e22cbc0 0x561d3e226460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.598+0000 7f7c8d6f2700 1 -- 172.21.15.92:0/3594841986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561d3e22cbc0 msgr2=0x561d3e226460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.598+0000 7f7c8d6f2700 1 --2- 172.21.15.92:0/3594841986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561d3e22cbc0 0x561d3e226460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.598+0000 7f7c8d6f2700 1 -- 172.21.15.92:0/3594841986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c84028440 con 0x561d3e20be60 2023-10-06T10:43:09.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.599+0000 7f7c8d6f2700 1 --2- 172.21.15.92:0/3594841986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e20be60 0x561d3e287ac0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f7c78004ea0 tx=0x7f7c78022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c7f44af1d28d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.599+0000 7f7c67fff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.599+0000 7f7c67fff700 1 -- 172.21.15.92:0/3594841986 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7801a030 con 0x561d3e20be60 2023-10-06T10:43:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.599+0000 7f7c67fff700 1 -- 172.21.15.92:0/3594841986 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7c78015aa0 con 0x561d3e20be60 2023-10-06T10:43:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.599+0000 7f7c9944fb80 1 -- 172.21.15.92:0/3594841986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c78007460 con 0x561d3e20be60 2023-10-06T10:43:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.599+0000 7f7c67fff700 1 -- 172.21.15.92:0/3594841986 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c78021b50 con 0x561d3e20be60 2023-10-06T10:43:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.599+0000 7f7c9944fb80 10 client.7308 fetch_fsmap learned FSMap version 67 2023-10-06T10:43:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.599+0000 7f7c9944fb80 1 -- 172.21.15.92:0/3594841986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x561d3e0e8470 con 0x561d3e20be60 2023-10-06T10:43:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.599+0000 7f7c67fff700 1 -- 172.21.15.92:0/3594841986 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c78007460 con 0x561d3e20be60 2023-10-06T10:43:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.599+0000 7f7c67fff700 1 -- 172.21.15.92:0/3594841986 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 67) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f7c78004bc0 con 0x561d3e20be60 2023-10-06T10:43:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.599+0000 7f7c9944fb80 10 client.7308 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T10:43:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.599+0000 7f7c9944fb80 10 client.7308 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T10:43:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.599+0000 7f7c9944fb80 20 client.7308 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.599+0000 7f7c9944fb80 1 --2- 172.21.15.92:0/3594841986 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x561d3e2422f0 0x561d3e2426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.599+0000 7f7c9944fb80 4 client.7308 mds_command: new command op to 7164 tid=0[{"prefix": "status"}] 2023-10-06T10:43:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.599+0000 7f7c9944fb80 1 -- 172.21.15.92:0/3594841986 --> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561d3e0d5e00 con 0x561d3e2422f0 2023-10-06T10:43:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.599+0000 7f7c8def3700 1 --2- 172.21.15.92:0/3594841986 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x561d3e2422f0 0x561d3e2426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.600+0000 7f7c67fff700 10 client.7308 ms_handle_connect on v2:172.21.15.124:6833/2971348516 2023-10-06T10:43:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.600+0000 7f7c8def3700 1 --2- 172.21.15.92:0/3594841986 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x561d3e2422f0 0x561d3e2426c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e57fdacf43f0ea61 server_cookie=6576484fdb3345cc in_seq=0 out_seq=0 2023-10-06T10:43:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.600+0000 7f7c67fff700 1 -- 172.21.15.92:0/3594841986 <== mds.0 v2:172.21.15.124:6833/2971348516 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x561d3e0d5e00 con 0x561d3e2422f0 2023-10-06T10:43:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.600+0000 7f7c67fff700 10 client.7308 handle_command_reply: tid=0 2023-10-06T10:43:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.600+0000 7f7c9944fb80 1 client.7308 shutdown 2023-10-06T10:43:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.600+0000 7f7c9944fb80 1 -- 172.21.15.92:0/3594841986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e20be60 msgr2=0x561d3e287ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.600+0000 7f7c9944fb80 1 --2- 172.21.15.92:0/3594841986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e20be60 0x561d3e287ac0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f7c78004ea0 tx=0x7f7c78022000 comp rx=0 tx=0).stop 2023-10-06T10:43:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.600+0000 7f7c9944fb80 1 -- 172.21.15.92:0/3594841986 shutdown_connections 2023-10-06T10:43:09.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.600+0000 7f7c9944fb80 1 --2- 172.21.15.92:0/3594841986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561d3e22cbc0 0x561d3e226460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.600+0000 7f7c9944fb80 1 --2- 172.21.15.92:0/3594841986 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x561d3e2422f0 0x561d3e2426c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.600+0000 7f7c9944fb80 1 --2- 172.21.15.92:0/3594841986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561d3e1e2950 0x561d3e287580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.600+0000 7f7c9944fb80 1 --2- 172.21.15.92:0/3594841986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d3e20be60 0x561d3e287ac0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.600+0000 7f7c67fff700 0 client.7308 ms_handle_reset on v2:172.21.15.124:6833/2971348516 2023-10-06T10:43:09.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.600+0000 7f7c9944fb80 1 -- 172.21.15.92:0/3594841986 >> 172.21.15.92:0/3594841986 conn(0x561d3e1e4270 msgr2=0x561d3e1d9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:09.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.600+0000 7f7c9944fb80 1 -- 172.21.15.92:0/3594841986 shutdown_connections 2023-10-06T10:43:09.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.600+0000 7f7c9944fb80 1 -- 172.21.15.92:0/3594841986 wait complete. 2023-10-06T10:43:09.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.600+0000 7f7c9944fb80 20 client.7308 trim_cache size 0 max 16384 2023-10-06T10:43:09.609 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:43:09.610 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:43:09.610 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T10:43:09.610 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 7164, 2023-10-06T10:43:09.610 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:43:09.610 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:43:09.610 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:43:09.610 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 5.8147842479999996, 2023-10-06T10:43:09.610 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 67, 2023-10-06T10:43:09.610 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 78, 2023-10-06T10:43:09.610 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 78, 2023-10-06T10:43:09.610 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 11.906652494999999 2023-10-06T10:43:09.611 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:43:09.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.601+0000 7f7c8eef5700 1 -- 172.21.15.92:0/3914163718 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7c6c066f30 msgr2=0x7f7c6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:09.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.601+0000 7f7c8eef5700 1 --2- 172.21.15.92:0/3914163718 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7c6c066f30 0x7f7c6c0693e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7c84006f00 tx=0x7f7c84008040 comp rx=0 tx=0).stop 2023-10-06T10:43:09.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.601+0000 7f7c8eef5700 1 -- 172.21.15.92:0/3914163718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c880cddf0 msgr2=0x7f7c880ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:09.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.601+0000 7f7c8eef5700 1 --2- 172.21.15.92:0/3914163718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c880cddf0 0x7f7c880ce290 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000c6a0 tx=0x7f7c80015000 comp rx=0 tx=0).stop 2023-10-06T10:43:09.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.601+0000 7f7c8eef5700 1 -- 172.21.15.92:0/3914163718 shutdown_connections 2023-10-06T10:43:09.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.601+0000 7f7c8eef5700 1 --2- 172.21.15.92:0/3914163718 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7c6c066f30 0x7f7c6c0693e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.601+0000 7f7c8eef5700 1 --2- 172.21.15.92:0/3914163718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c880cddf0 0x7f7c880ce290 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.601+0000 7f7c8eef5700 1 --2- 172.21.15.92:0/3914163718 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c740051c0 0x7f7c880c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.601+0000 7f7c8eef5700 1 --2- 172.21.15.92:0/3914163718 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c880c4570 0x7f7c880af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.601+0000 7f7c8eef5700 1 -- 172.21.15.92:0/3914163718 >> 172.21.15.92:0/3914163718 conn(0x7f7c8806ece0 msgr2=0x7f7c880b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:09.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.601+0000 7f7c8eef5700 1 -- 172.21.15.92:0/3914163718 shutdown_connections 2023-10-06T10:43:09.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.601+0000 7f7c8eef5700 1 -- 172.21.15.92:0/3914163718 wait complete. 2023-10-06T10:43:09.614 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T10:43:09.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.726+0000 7f1b1ffef700 1 Processor -- start 2023-10-06T10:43:09.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.726+0000 7f1b1ffef700 1 -- start start 2023-10-06T10:43:09.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.727+0000 7f1b1ffef700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1b180b0760 0x7f1b180ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.727+0000 7f1b1ffef700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b180b0c60 con 0x7f1b18159560 2023-10-06T10:43:09.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.727+0000 7f1b1ffef700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b180b0f40 con 0x7f1b180b49a0 2023-10-06T10:43:09.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.727+0000 7f1b1ffef700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b180ab350 con 0x7f1b180b0760 2023-10-06T10:43:09.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.727+0000 7f1b1e58c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1b180b0760 0x7f1b180ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.727+0000 7f1b1dd8b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1b18159560 0x7f1b180aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51896/0 (socket says 172.21.15.92:51896) 2023-10-06T10:43:09.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.727+0000 7f1b1dd8b700 1 -- 172.21.15.92:0/236132543 learned_addr learned my addr 172.21.15.92:0/236132543 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:09.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.727+0000 7f1b1cd89700 1 -- 172.21.15.92:0/236132543 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3916048932 0 0) 0x7f1b180b0c60 con 0x7f1b18159560 2023-10-06T10:43:09.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.727+0000 7f1b1cd89700 1 -- 172.21.15.92:0/236132543 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b04003570 con 0x7f1b18159560 2023-10-06T10:43:09.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.727+0000 7f1b1e58c700 1 -- 172.21.15.92:0/236132543 >> v1:172.21.15.124:6789/0 conn(0x7f1b180b49a0 legacy=0x7f1b180aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:09.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.727+0000 7f1b1e58c700 1 -- 172.21.15.92:0/236132543 >> v1:172.21.15.92:6789/0 conn(0x7f1b18159560 legacy=0x7f1b180aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:09.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.727+0000 7f1b1e58c700 1 -- 172.21.15.92:0/236132543 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b180b1220 con 0x7f1b180b0760 2023-10-06T10:43:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.728+0000 7f1b1e58c700 1 --2- 172.21.15.92:0/236132543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b180b0760 0x7f1b180ccf90 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f1b10009e90 tx=0x7f1b10009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d78a423a0ebc54 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.728+0000 7f1b1cd89700 1 -- 172.21.15.92:0/236132543 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1000e070 con 0x7f1b180b0760 2023-10-06T10:43:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.728+0000 7f1b1cd89700 1 -- 172.21.15.92:0/236132543 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b1000bc40 con 0x7f1b180b0760 2023-10-06T10:43:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.728+0000 7f1b1cd89700 1 -- 172.21.15.92:0/236132543 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b10012430 con 0x7f1b180b0760 2023-10-06T10:43:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.728+0000 7f1b1ffef700 1 -- 172.21.15.92:0/236132543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b180b0760 msgr2=0x7f1b180ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.728+0000 7f1b1ffef700 1 --2- 172.21.15.92:0/236132543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b180b0760 0x7f1b180ccf90 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f1b10009e90 tx=0x7f1b10009780 comp rx=0 tx=0).stop 2023-10-06T10:43:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.728+0000 7f1b1ffef700 1 -- 172.21.15.92:0/236132543 shutdown_connections 2023-10-06T10:43:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.728+0000 7f1b1ffef700 1 --2- 172.21.15.92:0/236132543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b180b0760 0x7f1b180ccf90 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.728+0000 7f1b1ffef700 1 -- 172.21.15.92:0/236132543 >> 172.21.15.92:0/236132543 conn(0x7f1b180997f0 msgr2=0x7f1b1806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.728+0000 7f1b1ffef700 1 -- 172.21.15.92:0/236132543 shutdown_connections 2023-10-06T10:43:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.728+0000 7f1b1ffef700 1 -- 172.21.15.92:0/236132543 wait complete. 2023-10-06T10:43:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.729+0000 7f1b1ffef700 1 Processor -- start 2023-10-06T10:43:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.729+0000 7f1b1ffef700 1 -- start start 2023-10-06T10:43:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.730+0000 7f1b1ffef700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b180b0760 0x7f1b1815bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.730+0000 7f1b1ffef700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b180b49a0 0x7f1b1809dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.730+0000 7f1b1ffef700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b18159560 0x7f1b1815e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.730+0000 7f1b1ffef700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1809e0f0 con 0x7f1b180b49a0 2023-10-06T10:43:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.730+0000 7f1b1ffef700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1816de10 con 0x7f1b18159560 2023-10-06T10:43:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.730+0000 7f1b1ffef700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1816d290 con 0x7f1b180b0760 2023-10-06T10:43:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.730+0000 7f1b1d58a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b180b49a0 0x7f1b1809dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.730+0000 7f1b1d58a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b180b49a0 0x7f1b1809dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34028/0 (socket says 172.21.15.92:34028) 2023-10-06T10:43:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.730+0000 7f1b1d58a700 1 -- 172.21.15.92:0/3175359986 learned_addr learned my addr 172.21.15.92:0/3175359986 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.730+0000 7f1b1e58c700 1 --2- 172.21.15.92:0/3175359986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b18159560 0x7f1b1815e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.730+0000 7f1b1dd8b700 1 --2- 172.21.15.92:0/3175359986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b180b0760 0x7f1b1815bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.731+0000 7f1b1d58a700 1 -- 172.21.15.92:0/3175359986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b180b0760 msgr2=0x7f1b1815bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.731+0000 7f1b1d58a700 1 --2- 172.21.15.92:0/3175359986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b180b0760 0x7f1b1815bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.731+0000 7f1b1d58a700 1 -- 172.21.15.92:0/3175359986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b18159560 msgr2=0x7f1b1815e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.731+0000 7f1b1d58a700 1 --2- 172.21.15.92:0/3175359986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b18159560 0x7f1b1815e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.731+0000 7f1b1d58a700 1 -- 172.21.15.92:0/3175359986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b10009a20 con 0x7f1b180b49a0 2023-10-06T10:43:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.731+0000 7f1b1dd8b700 1 --2- 172.21.15.92:0/3175359986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b180b0760 0x7f1b1815bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:09.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.731+0000 7f1b1d58a700 1 --2- 172.21.15.92:0/3175359986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b180b49a0 0x7f1b1809dbb0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800c790 tx=0x7f1b08007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:09.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.731+0000 7f1b0dffb700 1 -- 172.21.15.92:0/3175359986 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08011590 con 0x7f1b180b49a0 2023-10-06T10:43:09.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.731+0000 7f1b0dffb700 1 -- 172.21.15.92:0/3175359986 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b08011710 con 0x7f1b180b49a0 2023-10-06T10:43:09.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.731+0000 7f1b0dffb700 1 -- 172.21.15.92:0/3175359986 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08011ad0 con 0x7f1b180b49a0 2023-10-06T10:43:09.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.731+0000 7f1b1ffef700 1 -- 172.21.15.92:0/3175359986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b1816d570 con 0x7f1b180b49a0 2023-10-06T10:43:09.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.731+0000 7f1b1ffef700 1 -- 172.21.15.92:0/3175359986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b1816db30 con 0x7f1b180b49a0 2023-10-06T10:43:09.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.732+0000 7f1b1ffef700 1 -- 172.21.15.92:0/3175359986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b1815a990 con 0x7f1b180b49a0 2023-10-06T10:43:09.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.732+0000 7f1b0dffb700 1 -- 172.21.15.92:0/3175359986 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1b0801b3b0 con 0x7f1b180b49a0 2023-10-06T10:43:09.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.733+0000 7f1b0dffb700 1 --2- 172.21.15.92:0/3175359986 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1afc066f30 0x7f1afc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:09.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.733+0000 7f1b0dffb700 1 -- 172.21.15.92:0/3175359986 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1b08083bb0 con 0x7f1b180b49a0 2023-10-06T10:43:09.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.733+0000 7f1b1dd8b700 1 --2- 172.21.15.92:0/3175359986 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1afc066f30 0x7f1afc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:09.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.733+0000 7f1b1dd8b700 1 --2- 172.21.15.92:0/3175359986 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1afc066f30 0x7f1afc0693e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1b14001360 tx=0x7f1b14009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:09.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.737+0000 7f1b0dffb700 1 -- 172.21.15.92:0/3175359986 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b1815a990 con 0x7f1b180b49a0 2023-10-06T10:43:09.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.964+0000 7f1b1ffef700 1 -- 172.21.15.92:0/3175359986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1b1815a990 con 0x7f1b180b49a0 2023-10-06T10:43:09.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.964+0000 7f1b0dffb700 1 -- 172.21.15.92:0/3175359986 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1b1815a990 con 0x7f1b180b49a0 2023-10-06T10:43:09.966 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T10:43:09.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.966+0000 7f1b1ffef700 1 -- 172.21.15.92:0/3175359986 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1afc066f30 msgr2=0x7f1afc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:09.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.966+0000 7f1b1ffef700 1 --2- 172.21.15.92:0/3175359986 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1afc066f30 0x7f1afc0693e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1b14001360 tx=0x7f1b14009040 comp rx=0 tx=0).stop 2023-10-06T10:43:09.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.966+0000 7f1b1ffef700 1 -- 172.21.15.92:0/3175359986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b180b49a0 msgr2=0x7f1b1809dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:09.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.966+0000 7f1b1ffef700 1 --2- 172.21.15.92:0/3175359986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b180b49a0 0x7f1b1809dbb0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800c790 tx=0x7f1b08007f20 comp rx=0 tx=0).stop 2023-10-06T10:43:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.966+0000 7f1b1ffef700 1 -- 172.21.15.92:0/3175359986 shutdown_connections 2023-10-06T10:43:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.966+0000 7f1b1ffef700 1 --2- 172.21.15.92:0/3175359986 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1afc066f30 0x7f1afc0693e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.966+0000 7f1b1ffef700 1 --2- 172.21.15.92:0/3175359986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b18159560 0x7f1b1815e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.966+0000 7f1b1ffef700 1 --2- 172.21.15.92:0/3175359986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b180b0760 0x7f1b1815bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.966+0000 7f1b1ffef700 1 --2- 172.21.15.92:0/3175359986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b180b49a0 0x7f1b1809dbb0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.966+0000 7f1b1ffef700 1 -- 172.21.15.92:0/3175359986 >> 172.21.15.92:0/3175359986 conn(0x7f1b180997f0 msgr2=0x7f1b18158860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.966+0000 7f1b1ffef700 1 -- 172.21.15.92:0/3175359986 shutdown_connections 2023-10-06T10:43:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:09.966+0000 7f1b1ffef700 1 -- 172.21.15.92:0/3175359986 wait complete. 2023-10-06T10:43:09.980 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T10:43:09.980 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T10:43:09.980 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T10:43:09.980 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T10:43:09.980 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T10:43:09.980 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T10:43:09.980 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T10:43:09.981 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T10:43:09.981 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T10:43:09.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:43:09.981 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T10:43:09.996 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T10:43:09.996 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T10:43:09.997 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T10:43:09.997 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:43:09.997 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T10:43:09.997 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:43:09.997 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:43:09.997 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:43:09.997 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T10:43:09.998 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:43:09.998 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:43:09.998 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T10:43:09.998 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:43:09.998 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:43:09.998 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T10:43:09.998 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:43:09.998 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T10:43:09.998 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42219sec preferred_lft 42219sec 2023-10-06T10:43:09.998 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T10:43:09.998 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:43:09.998 INFO:teuthology.orchestra.run.smithi092.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T10:43:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 92:7f:88:24:4e:ca brd ff:ff:ff:ff:ff:ff 2023-10-06T10:43:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T10:43:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:43:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::9c9d:23ff:fee4:30ae/64 scope link 2023-10-06T10:43:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:43:09.999 INFO:teuthology.orchestra.run.smithi092.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T10:43:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 92:7f:88:24:4e:ca brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T10:43:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::907f:88ff:fe24:4eca/64 scope link 2023-10-06T10:43:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:43:09.999 INFO:teuthology.orchestra.run.smithi092.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T10:43:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether b6:32:f9:d9:96:1b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T10:43:09.999 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::b432:f9ff:fed9:961b/64 scope link 2023-10-06T10:43:10.000 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:43:10.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:43:10.000 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T10:43:10.060 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T10:43:10.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:43:10.061 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T10:43:10.117 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T10:43:10.117 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T10:43:10.118 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T10:43:10.118 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T10:43:10.118 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T10:43:10.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:43:10.118 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T10:43:10.118 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T10:43:10.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T10:43:10.119 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T10:43:10.119 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T10:43:10.119 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T10:43:10.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:43:10.217 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T10:43:10.231 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T10:43:10.231 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T10:43:10.232 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T10:43:10.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:43:10.232 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-10-06T10:43:10.297 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T10:43:10.297 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T10:43:10.352 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T10:43:10.352 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T10:43:10.352 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T10:43:10.406 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T10:43:10.407 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T10:43:10.407 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:43:10.486 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:43:10.486 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:43:10.505 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:43:10.505 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:43:10.505 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-06T10:43:10.505 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@8f307bfb-c4cb-4f66-aade-46fffe0c6e0c.cephfs=/ 2023-10-06T10:43:10.506 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-06T10:43:10.506 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T10:43:10.506 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T10:43:10.506 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T10:43:10.528 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:43:10.529 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:43:10.529 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-06T10:43:10.529 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@8f307bfb-c4cb-4f66-aade-46fffe0c6e0c.cephfs=/ 2023-10-06T10:43:10.529 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-06T10:43:10.529 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T10:43:10.529 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T10:43:10.529 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T10:43:10.529 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.124:6789,172.21.15.124:6790" 2023-10-06T10:43:10.529 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.124:6789,172.21.15.124:6790:/ 2023-10-06T10:43:10.529 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=8f307bfb-c4cb-4f66-aade-46fffe0c6e0c" will pass to kernel 2023-10-06T10:43:10.531 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T10:43:10.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:43:10.531 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T10:43:10.596 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-10-06T10:43:10.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.751+0000 7ff8ff427700 1 Processor -- start 2023-10-06T10:43:10.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.751+0000 7ff8ff427700 1 -- start start 2023-10-06T10:43:10.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.752+0000 7ff8ff427700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff8f80cbc70 0x7ff8f80ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:10.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.752+0000 7ff8ff427700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f80cf5d0 con 0x7ff8f81594d0 2023-10-06T10:43:10.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.752+0000 7ff8ff427700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f80cf8b0 con 0x7ff8f81598a0 2023-10-06T10:43:10.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.752+0000 7ff8ff427700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f80cfb90 con 0x7ff8f80cbc70 2023-10-06T10:43:10.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.752+0000 7ff8fd9c4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff8f80cbc70 0x7ff8f80ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:10.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.752+0000 7ff8fd9c4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff8f80cbc70 0x7ff8f80ceef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34036/0 (socket says 172.21.15.92:34036) 2023-10-06T10:43:10.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.752+0000 7ff8fd9c4700 1 -- 172.21.15.92:0/1181898744 learned_addr learned my addr 172.21.15.92:0/1181898744 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:10.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.752+0000 7ff8fd1c3700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff8f81594d0 0x7ff8f80cb560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51906/0 (socket says 172.21.15.92:51906) 2023-10-06T10:43:10.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.752+0000 7ff8fd9c4700 1 -- 172.21.15.92:0/1181898744 >> v1:172.21.15.124:6789/0 conn(0x7ff8f81598a0 legacy=0x7ff8f80ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:10.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.752+0000 7ff8fd9c4700 1 -- 172.21.15.92:0/1181898744 >> v1:172.21.15.92:6789/0 conn(0x7ff8f81594d0 legacy=0x7ff8f80cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:10.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.752+0000 7ff8fd9c4700 1 -- 172.21.15.92:0/1181898744 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f80cfe70 con 0x7ff8f80cbc70 2023-10-06T10:43:10.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.752+0000 7ff8effff700 1 -- 172.21.15.92:0/1181898744 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2020837039 0 0) 0x7ff8f80cf5d0 con 0x7ff8f81594d0 2023-10-06T10:43:10.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.752+0000 7ff8fd9c4700 1 --2- 172.21.15.92:0/1181898744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f80cbc70 0x7ff8f80ceef0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00099b0 tx=0x7ff8f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77068ab2c111f7e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:10.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.752+0000 7ff8effff700 1 -- 172.21.15.92:0/1181898744 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f000b820 con 0x7ff8f80cbc70 2023-10-06T10:43:10.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.752+0000 7ff8effff700 1 -- 172.21.15.92:0/1181898744 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f000b980 con 0x7ff8f80cbc70 2023-10-06T10:43:10.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.752+0000 7ff8effff700 1 -- 172.21.15.92:0/1181898744 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f000bd20 con 0x7ff8f80cbc70 2023-10-06T10:43:10.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.753+0000 7ff8ff427700 1 -- 172.21.15.92:0/1181898744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f80cbc70 msgr2=0x7ff8f80ceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:10.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.753+0000 7ff8ff427700 1 --2- 172.21.15.92:0/1181898744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f80cbc70 0x7ff8f80ceef0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00099b0 tx=0x7ff8f0009250 comp rx=0 tx=0).stop 2023-10-06T10:43:10.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.753+0000 7ff8ff427700 1 -- 172.21.15.92:0/1181898744 shutdown_connections 2023-10-06T10:43:10.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.753+0000 7ff8ff427700 1 --2- 172.21.15.92:0/1181898744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f80cbc70 0x7ff8f80ceef0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:10.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.753+0000 7ff8ff427700 1 -- 172.21.15.92:0/1181898744 >> 172.21.15.92:0/1181898744 conn(0x7ff8f8089e50 msgr2=0x7ff8f8099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:10.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.753+0000 7ff8ff427700 1 -- 172.21.15.92:0/1181898744 shutdown_connections 2023-10-06T10:43:10.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.753+0000 7ff8ff427700 1 -- 172.21.15.92:0/1181898744 wait complete. 2023-10-06T10:43:10.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.754+0000 7ff8ff427700 1 Processor -- start 2023-10-06T10:43:10.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.754+0000 7ff8ff427700 1 -- start start 2023-10-06T10:43:10.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.755+0000 7ff8ff427700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8f80cbc70 0x7ff8f8149270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:10.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.755+0000 7ff8ff427700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8f81594d0 0x7ff8f80b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:10.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.755+0000 7ff8ff427700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f81598a0 0x7ff8f80b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:10.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.755+0000 7ff8ff427700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f80c86b0 con 0x7ff8f81598a0 2023-10-06T10:43:10.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.755+0000 7ff8ff427700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f80b3e90 con 0x7ff8f81594d0 2023-10-06T10:43:10.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.755+0000 7ff8ff427700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f80c8aa0 con 0x7ff8f80cbc70 2023-10-06T10:43:10.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.755+0000 7ff8fd9c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f81598a0 0x7ff8f80b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:10.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.755+0000 7ff8fd9c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f81598a0 0x7ff8f80b37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34038/0 (socket says 172.21.15.92:34038) 2023-10-06T10:43:10.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.755+0000 7ff8fd9c4700 1 -- 172.21.15.92:0/3235011832 learned_addr learned my addr 172.21.15.92:0/3235011832 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:10.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.755+0000 7ff8fd1c3700 1 --2- 172.21.15.92:0/3235011832 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8f80cbc70 0x7ff8f8149270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:10.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.755+0000 7ff8fc9c2700 1 --2- 172.21.15.92:0/3235011832 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8f81594d0 0x7ff8f80b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:10.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.755+0000 7ff8fd9c4700 1 -- 172.21.15.92:0/3235011832 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8f80cbc70 msgr2=0x7ff8f8149270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:10.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.755+0000 7ff8fd9c4700 1 --2- 172.21.15.92:0/3235011832 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8f80cbc70 0x7ff8f8149270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:10.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.755+0000 7ff8fd9c4700 1 -- 172.21.15.92:0/3235011832 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8f81594d0 msgr2=0x7ff8f80b32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:10.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.755+0000 7ff8fd9c4700 1 --2- 172.21.15.92:0/3235011832 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8f81594d0 0x7ff8f80b32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:10.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.755+0000 7ff8fd9c4700 1 -- 172.21.15.92:0/3235011832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f00094f0 con 0x7ff8f81598a0 2023-10-06T10:43:10.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.755+0000 7ff8fd9c4700 1 --2- 172.21.15.92:0/3235011832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f81598a0 0x7ff8f80b37f0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009290 tx=0x7ff8f0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:10.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.756+0000 7ff8ed7fa700 1 -- 172.21.15.92:0/3235011832 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f001b450 con 0x7ff8f81598a0 2023-10-06T10:43:10.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.756+0000 7ff8ed7fa700 1 -- 172.21.15.92:0/3235011832 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f0020070 con 0x7ff8f81598a0 2023-10-06T10:43:10.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.756+0000 7ff8ff427700 1 -- 172.21.15.92:0/3235011832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f80c8d80 con 0x7ff8f81598a0 2023-10-06T10:43:10.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.756+0000 7ff8ed7fa700 1 -- 172.21.15.92:0/3235011832 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f001b820 con 0x7ff8f81598a0 2023-10-06T10:43:10.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.756+0000 7ff8ff427700 1 -- 172.21.15.92:0/3235011832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f80b4170 con 0x7ff8f81598a0 2023-10-06T10:43:10.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.757+0000 7ff8ff427700 1 -- 172.21.15.92:0/3235011832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f80c88f0 con 0x7ff8f81598a0 2023-10-06T10:43:10.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.757+0000 7ff8ed7fa700 1 -- 172.21.15.92:0/3235011832 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff8f0022030 con 0x7ff8f81598a0 2023-10-06T10:43:10.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.757+0000 7ff8ed7fa700 1 --2- 172.21.15.92:0/3235011832 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff8dc066f30 0x7ff8dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:10.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.757+0000 7ff8ed7fa700 1 -- 172.21.15.92:0/3235011832 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff8f0086960 con 0x7ff8f81598a0 2023-10-06T10:43:10.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.757+0000 7ff8fd1c3700 1 --2- 172.21.15.92:0/3235011832 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff8dc066f30 0x7ff8dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:10.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.758+0000 7ff8fd1c3700 1 --2- 172.21.15.92:0/3235011832 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff8dc066f30 0x7ff8dc0693e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4001560 tx=0x7ff8f4005d10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:10.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.762+0000 7ff8ed7fa700 1 -- 172.21.15.92:0/3235011832 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff8f0053930 con 0x7ff8f81598a0 2023-10-06T10:43:10.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:10.992+0000 7ff8ff427700 1 -- 172.21.15.92:0/3235011832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7ff8f80c88f0 con 0x7ff8f81598a0 2023-10-06T10:43:11.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.340+0000 7ff8ed7fa700 1 -- 172.21.15.92:0/3235011832 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v68) v1 ==== 105+0+0 (secure 0 0 0) 0x7ff8f0056f50 con 0x7ff8f81598a0 2023-10-06T10:43:11.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.342+0000 7ff8ff427700 1 -- 172.21.15.92:0/3235011832 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff8dc066f30 msgr2=0x7ff8dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:11.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.343+0000 7ff8ff427700 1 --2- 172.21.15.92:0/3235011832 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff8dc066f30 0x7ff8dc0693e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4001560 tx=0x7ff8f4005d10 comp rx=0 tx=0).stop 2023-10-06T10:43:11.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.343+0000 7ff8ff427700 1 -- 172.21.15.92:0/3235011832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f81598a0 msgr2=0x7ff8f80b37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:11.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.343+0000 7ff8ff427700 1 --2- 172.21.15.92:0/3235011832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f81598a0 0x7ff8f80b37f0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009290 tx=0x7ff8f0016000 comp rx=0 tx=0).stop 2023-10-06T10:43:11.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.343+0000 7ff8ff427700 1 -- 172.21.15.92:0/3235011832 shutdown_connections 2023-10-06T10:43:11.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.343+0000 7ff8ff427700 1 --2- 172.21.15.92:0/3235011832 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff8dc066f30 0x7ff8dc0693e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:11.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.343+0000 7ff8ff427700 1 --2- 172.21.15.92:0/3235011832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f81598a0 0x7ff8f80b37f0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:11.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.343+0000 7ff8ff427700 1 --2- 172.21.15.92:0/3235011832 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8f80cbc70 0x7ff8f8149270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:11.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.343+0000 7ff8ff427700 1 --2- 172.21.15.92:0/3235011832 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8f81594d0 0x7ff8f80b32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:11.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.343+0000 7ff8ff427700 1 -- 172.21.15.92:0/3235011832 >> 172.21.15.92:0/3235011832 conn(0x7ff8f8089e50 msgr2=0x7ff8f80ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:11.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.343+0000 7ff8ff427700 1 -- 172.21.15.92:0/3235011832 shutdown_connections 2023-10-06T10:43:11.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.343+0000 7ff8ff427700 1 -- 172.21.15.92:0/3235011832 wait complete. 2023-10-06T10:43:11.362 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:43:11.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.474+0000 7fcb851f1700 1 Processor -- start 2023-10-06T10:43:11.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.474+0000 7fcb851f1700 1 -- start start 2023-10-06T10:43:11.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.475+0000 7fcb851f1700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb80157420 0x7fcb800ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:11.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.475+0000 7fcb851f1700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcb800ac440 0x7fcb800685c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:11.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.475+0000 7fcb851f1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb800cae20 con 0x7fcb801577f0 2023-10-06T10:43:11.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.475+0000 7fcb851f1700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb800aee90 con 0x7fcb800ac440 2023-10-06T10:43:11.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.475+0000 7fcb851f1700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb800af170 con 0x7fcb80157420 2023-10-06T10:43:11.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.475+0000 7fcb7e59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcb801577f0 0x7fcb800cc260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51908/0 (socket says 172.21.15.92:51908) 2023-10-06T10:43:11.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.475+0000 7fcb7e59c700 1 -- 172.21.15.92:0/1375759977 learned_addr learned my addr 172.21.15.92:0/1375759977 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:11.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.475+0000 7fcb7f59e700 1 --2- 172.21.15.92:0/1375759977 >> v2:172.21.15.124:3300/0 conn(0x7fcb800ac440 0x7fcb800685c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:11.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.475+0000 7fcb7ed9d700 1 --2- 172.21.15.92:0/1375759977 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb80157420 0x7fcb800ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:11.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.475+0000 7fcb7dd9b700 1 -- 172.21.15.92:0/1375759977 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2687295504 0 0) 0x7fcb800cae20 con 0x7fcb801577f0 2023-10-06T10:43:11.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.475+0000 7fcb7dd9b700 1 -- 172.21.15.92:0/1375759977 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb68003570 con 0x7fcb801577f0 2023-10-06T10:43:11.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.475+0000 7fcb7ed9d700 1 -- 172.21.15.92:0/1375759977 >> v2:172.21.15.124:3300/0 conn(0x7fcb800ac440 msgr2=0x7fcb800685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:11.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.475+0000 7fcb7ed9d700 1 --2- 172.21.15.92:0/1375759977 >> v2:172.21.15.124:3300/0 conn(0x7fcb800ac440 0x7fcb800685c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:11.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.475+0000 7fcb7ed9d700 1 -- 172.21.15.92:0/1375759977 >> v1:172.21.15.92:6789/0 conn(0x7fcb801577f0 legacy=0x7fcb800cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:11.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.475+0000 7fcb7dd9b700 1 -- 172.21.15.92:0/1375759977 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3384132585 0 0) 0x7fcb68003570 con 0x7fcb801577f0 2023-10-06T10:43:11.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.475+0000 7fcb7ed9d700 1 -- 172.21.15.92:0/1375759977 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb800af450 con 0x7fcb80157420 2023-10-06T10:43:11.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.475+0000 7fcb7f59e700 1 --2- 172.21.15.92:0/1375759977 >> v2:172.21.15.124:3300/0 conn(0x7fcb800ac440 0x7fcb800685c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:43:11.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.476+0000 7fcb7ed9d700 1 --2- 172.21.15.92:0/1375759977 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb80157420 0x7fcb800ae8b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009eb0 tx=0x7fcb70009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27421f052efd3203 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:11.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.476+0000 7fcb7dd9b700 1 -- 172.21.15.92:0/1375759977 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7000e070 con 0x7fcb80157420 2023-10-06T10:43:11.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.476+0000 7fcb7dd9b700 1 -- 172.21.15.92:0/1375759977 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7000bb80 con 0x7fcb80157420 2023-10-06T10:43:11.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.476+0000 7fcb7dd9b700 1 -- 172.21.15.92:0/1375759977 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb70012430 con 0x7fcb80157420 2023-10-06T10:43:11.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.476+0000 7fcb851f1700 1 -- 172.21.15.92:0/1375759977 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb80157420 msgr2=0x7fcb800ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:11.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.476+0000 7fcb851f1700 1 --2- 172.21.15.92:0/1375759977 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb80157420 0x7fcb800ae8b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009eb0 tx=0x7fcb70009780 comp rx=0 tx=0).stop 2023-10-06T10:43:11.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.476+0000 7fcb851f1700 1 -- 172.21.15.92:0/1375759977 shutdown_connections 2023-10-06T10:43:11.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.476+0000 7fcb851f1700 1 --2- 172.21.15.92:0/1375759977 >> v2:172.21.15.124:3300/0 conn(0x7fcb800ac440 0x7fcb800685c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:11.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.476+0000 7fcb851f1700 1 --2- 172.21.15.92:0/1375759977 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb80157420 0x7fcb800ae8b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:11.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.476+0000 7fcb851f1700 1 -- 172.21.15.92:0/1375759977 >> 172.21.15.92:0/1375759977 conn(0x7fcb8006de40 msgr2=0x7fcb8006e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:11.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.476+0000 7fcb851f1700 1 -- 172.21.15.92:0/1375759977 shutdown_connections 2023-10-06T10:43:11.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.476+0000 7fcb851f1700 1 -- 172.21.15.92:0/1375759977 wait complete. 2023-10-06T10:43:11.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.478+0000 7fcb851f1700 1 Processor -- start 2023-10-06T10:43:11.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.478+0000 7fcb851f1700 1 -- start start 2023-10-06T10:43:11.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.478+0000 7fcb851f1700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb800ac440 0x7fcb80153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:11.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.478+0000 7fcb851f1700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb80157420 0x7fcb8015abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:11.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.478+0000 7fcb851f1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb801577f0 0x7fcb8015b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:11.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.478+0000 7fcb851f1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb800a9d90 con 0x7fcb801577f0 2023-10-06T10:43:11.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.478+0000 7fcb851f1700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb800b1260 con 0x7fcb800ac440 2023-10-06T10:43:11.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.478+0000 7fcb851f1700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb800b1540 con 0x7fcb80157420 2023-10-06T10:43:11.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.479+0000 7fcb7f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb801577f0 0x7fcb8015b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:11.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.479+0000 7fcb7f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb801577f0 0x7fcb8015b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34046/0 (socket says 172.21.15.92:34046) 2023-10-06T10:43:11.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.479+0000 7fcb7f59e700 1 -- 172.21.15.92:0/2899181678 learned_addr learned my addr 172.21.15.92:0/2899181678 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:11.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.479+0000 7fcb7e59c700 1 --2- 172.21.15.92:0/2899181678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb80157420 0x7fcb8015abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:11.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.479+0000 7fcb7ed9d700 1 --2- 172.21.15.92:0/2899181678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb800ac440 0x7fcb80153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:11.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.479+0000 7fcb7f59e700 1 -- 172.21.15.92:0/2899181678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb80157420 msgr2=0x7fcb8015abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:11.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.479+0000 7fcb7f59e700 1 --2- 172.21.15.92:0/2899181678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb80157420 0x7fcb8015abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:11.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.479+0000 7fcb7f59e700 1 -- 172.21.15.92:0/2899181678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb800ac440 msgr2=0x7fcb80153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:11.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.479+0000 7fcb7f59e700 1 --2- 172.21.15.92:0/2899181678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb800ac440 0x7fcb80153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:11.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.479+0000 7fcb7f59e700 1 -- 172.21.15.92:0/2899181678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb78009420 con 0x7fcb801577f0 2023-10-06T10:43:11.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.479+0000 7fcb7ed9d700 1 --2- 172.21.15.92:0/2899181678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb800ac440 0x7fcb80153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:11.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.479+0000 7fcb7f59e700 1 --2- 172.21.15.92:0/2899181678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb801577f0 0x7fcb8015b100 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800ec80 tx=0x7fcb780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:11.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.479+0000 7fcb66ffd700 1 -- 172.21.15.92:0/2899181678 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7800c850 con 0x7fcb801577f0 2023-10-06T10:43:11.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.479+0000 7fcb66ffd700 1 -- 172.21.15.92:0/2899181678 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7800c9b0 con 0x7fcb801577f0 2023-10-06T10:43:11.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.479+0000 7fcb851f1700 1 -- 172.21.15.92:0/2899181678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb70009a20 con 0x7fcb801577f0 2023-10-06T10:43:11.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.479+0000 7fcb66ffd700 1 -- 172.21.15.92:0/2899181678 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7800cd70 con 0x7fcb801577f0 2023-10-06T10:43:11.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.479+0000 7fcb851f1700 1 -- 172.21.15.92:0/2899181678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb80153f80 con 0x7fcb801577f0 2023-10-06T10:43:11.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.480+0000 7fcb851f1700 1 -- 172.21.15.92:0/2899181678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb800a9f40 con 0x7fcb801577f0 2023-10-06T10:43:11.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.481+0000 7fcb66ffd700 1 -- 172.21.15.92:0/2899181678 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcb78013680 con 0x7fcb801577f0 2023-10-06T10:43:11.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.481+0000 7fcb66ffd700 1 --2- 172.21.15.92:0/2899181678 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcb6c066f30 0x7fcb6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:11.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.481+0000 7fcb66ffd700 1 -- 172.21.15.92:0/2899181678 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fcb78084890 con 0x7fcb801577f0 2023-10-06T10:43:11.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.481+0000 7fcb7ed9d700 1 --2- 172.21.15.92:0/2899181678 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcb6c066f30 0x7fcb6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:11.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.481+0000 7fcb7ed9d700 1 --2- 172.21.15.92:0/2899181678 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcb6c066f30 0x7fcb6c0693e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcb700097c0 tx=0x7fcb70019000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:11.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.485+0000 7fcb66ffd700 1 -- 172.21.15.92:0/2899181678 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb78051860 con 0x7fcb801577f0 2023-10-06T10:43:11.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.707+0000 7fcb851f1700 1 -- 172.21.15.92:0/2899181678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb80064a90 con 0x7fcb801577f0 2023-10-06T10:43:11.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.708+0000 7fcb66ffd700 1 -- 172.21.15.92:0/2899181678 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4621 (secure 0 0 0) 0x7fcb78054e80 con 0x7fcb801577f0 2023-10-06T10:43:11.710 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:11.710 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":69,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3469511807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3469511807},{"type":"v1","addr":"172.21.15.124:6834","nonce":3469511807}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7146,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/4003376156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4003376156},{"type":"v1","addr":"172.21.15.92:6837","nonce":4003376156}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:03.173076+0000","modified":"2023-10-06T10:43:11.342044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":7164,"mds_1":7149},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7149":{"gid":7149,"name":"c","rank":1,"incarnation":69,"state":"up:creating","state_seq":1,"addr":"172.21.15.92:6836/3268908749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3268908749},{"type":"v1","addr":"172.21.15.92:6836","nonce":3268908749}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7164":{"gid":7164,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/2971348516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2971348516},{"type":"v1","addr":"172.21.15.124:6835","nonce":2971348516}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T10:43:11.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.710+0000 7fcb851f1700 1 -- 172.21.15.92:0/2899181678 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcb6c066f30 msgr2=0x7fcb6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:11.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.710+0000 7fcb851f1700 1 --2- 172.21.15.92:0/2899181678 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcb6c066f30 0x7fcb6c0693e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcb700097c0 tx=0x7fcb70019000 comp rx=0 tx=0).stop 2023-10-06T10:43:11.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.710+0000 7fcb851f1700 1 -- 172.21.15.92:0/2899181678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb801577f0 msgr2=0x7fcb8015b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:11.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.710+0000 7fcb851f1700 1 --2- 172.21.15.92:0/2899181678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb801577f0 0x7fcb8015b100 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800ec80 tx=0x7fcb780091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:11.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.710+0000 7fcb851f1700 1 -- 172.21.15.92:0/2899181678 shutdown_connections 2023-10-06T10:43:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.710+0000 7fcb851f1700 1 --2- 172.21.15.92:0/2899181678 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcb6c066f30 0x7fcb6c0693e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.710+0000 7fcb851f1700 1 --2- 172.21.15.92:0/2899181678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb801577f0 0x7fcb8015b100 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.710+0000 7fcb851f1700 1 --2- 172.21.15.92:0/2899181678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb800ac440 0x7fcb80153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.710+0000 7fcb851f1700 1 --2- 172.21.15.92:0/2899181678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb80157420 0x7fcb8015abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.710+0000 7fcb851f1700 1 -- 172.21.15.92:0/2899181678 >> 172.21.15.92:0/2899181678 conn(0x7fcb8006de40 msgr2=0x7fcb80155bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.710+0000 7fcb851f1700 1 -- 172.21.15.92:0/2899181678 shutdown_connections 2023-10-06T10:43:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:11.710+0000 7fcb851f1700 1 -- 172.21.15.92:0/2899181678 wait complete. 2023-10-06T10:43:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 69 2023-10-06T10:43:12.126 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T10:43:03.173076+0000', 'modified': '2023-10-06T10:43:11.342044+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 7164, 'mds_1': 7149}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7149': {'gid': 7149, 'name': 'c', 'rank': 1, 'incarnation': 69, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.92:6836/3268908749', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 3268908749}, {'type': 'v1', 'addr': '172.21.15.92:6836', 'nonce': 3268908749}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_7164': {'gid': 7164, 'name': 'd', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.124:6835/2971348516', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6833', 'nonce': 2971348516}, {'type': 'v1', 'addr': '172.21.15.124:6835', 'nonce': 2971348516}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T10:43:12.126 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7149:up:creating 2023-10-06T10:43:13.093 INFO:tasks.ceph.mds.d.smithi124.stderr:2023-10-06T10:43:13.091+0000 7f984e348700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-06T10:43:13.127 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:43:13.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.242+0000 7f75c791b700 1 Processor -- start 2023-10-06T10:43:13.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.242+0000 7f75c791b700 1 -- start start 2023-10-06T10:43:13.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.242+0000 7f75c791b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f75c0159560 0x7f75c00aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:13.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.243+0000 7f75c791b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f75c00b49a0 0x7f75c00aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:13.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.243+0000 7f75c791b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c00b0c60 con 0x7f75c00b0760 2023-10-06T10:43:13.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.243+0000 7f75c791b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75c00b0f40 con 0x7f75c00b49a0 2023-10-06T10:43:13.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.243+0000 7f75c791b700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75c00b1220 con 0x7f75c0159560 2023-10-06T10:43:13.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.243+0000 7f75c4eb6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f75c00b49a0 0x7f75c00aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:13.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.243+0000 7f75c4eb6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f75c00b49a0 0x7f75c00aaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34062/0 (socket says 172.21.15.92:34062) 2023-10-06T10:43:13.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.243+0000 7f75c5eb8700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f75c00b0760 0x7f75c00b3890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51916/0 (socket says 172.21.15.92:51916) 2023-10-06T10:43:13.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.243+0000 7f75c4eb6700 1 -- 172.21.15.92:0/2091615390 learned_addr learned my addr 172.21.15.92:0/2091615390 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:13.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.243+0000 7f75c56b7700 1 --2- 172.21.15.92:0/2091615390 >> v2:172.21.15.124:3300/0 conn(0x7f75c0159560 0x7f75c00aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:13.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.243+0000 7f75c4eb6700 1 -- 172.21.15.92:0/2091615390 >> v2:172.21.15.124:3300/0 conn(0x7f75c0159560 msgr2=0x7f75c00aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:13.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.243+0000 7f75c4eb6700 1 --2- 172.21.15.92:0/2091615390 >> v2:172.21.15.124:3300/0 conn(0x7f75c0159560 0x7f75c00aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:13.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.243+0000 7f75c4eb6700 1 -- 172.21.15.92:0/2091615390 >> v1:172.21.15.92:6789/0 conn(0x7f75c00b0760 legacy=0x7f75c00b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:13.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.243+0000 7f75c4eb6700 1 -- 172.21.15.92:0/2091615390 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c00b1500 con 0x7f75c00b49a0 2023-10-06T10:43:13.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.243+0000 7f75c4eb6700 1 --2- 172.21.15.92:0/2091615390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75c00b49a0 0x7f75c00aaa70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f75b00099b0 tx=0x7f75b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3238a8e18300fb3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:13.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.243+0000 7f75b7fff700 1 -- 172.21.15.92:0/2091615390 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b000b760 con 0x7f75c00b49a0 2023-10-06T10:43:13.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.243+0000 7f75b7fff700 1 -- 172.21.15.92:0/2091615390 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b000b8c0 con 0x7f75c00b49a0 2023-10-06T10:43:13.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.243+0000 7f75b7fff700 1 -- 172.21.15.92:0/2091615390 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b000bc40 con 0x7f75c00b49a0 2023-10-06T10:43:13.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.244+0000 7f75c791b700 1 -- 172.21.15.92:0/2091615390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75c00b49a0 msgr2=0x7f75c00aaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:13.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.244+0000 7f75c791b700 1 --2- 172.21.15.92:0/2091615390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75c00b49a0 0x7f75c00aaa70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f75b00099b0 tx=0x7f75b0009250 comp rx=0 tx=0).stop 2023-10-06T10:43:13.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.244+0000 7f75c791b700 1 -- 172.21.15.92:0/2091615390 shutdown_connections 2023-10-06T10:43:13.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.244+0000 7f75c791b700 1 --2- 172.21.15.92:0/2091615390 >> v2:172.21.15.124:3300/0 conn(0x7f75c0159560 0x7f75c00aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:13.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.244+0000 7f75c791b700 1 --2- 172.21.15.92:0/2091615390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75c00b49a0 0x7f75c00aaa70 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:13.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.244+0000 7f75c791b700 1 -- 172.21.15.92:0/2091615390 >> 172.21.15.92:0/2091615390 conn(0x7f75c00997f0 msgr2=0x7f75c006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:13.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.244+0000 7f75c791b700 1 -- 172.21.15.92:0/2091615390 shutdown_connections 2023-10-06T10:43:13.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.244+0000 7f75c791b700 1 -- 172.21.15.92:0/2091615390 wait complete. 2023-10-06T10:43:13.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.245+0000 7f75c791b700 1 Processor -- start 2023-10-06T10:43:13.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.245+0000 7f75c791b700 1 -- start start 2023-10-06T10:43:13.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.246+0000 7f75c791b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75c00b0760 0x7f75c015c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:13.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.246+0000 7f75c791b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75c00b49a0 0x7f75c016e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:13.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.246+0000 7f75c791b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75c0159560 0x7f75c00aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:13.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.246+0000 7f75c791b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c016e820 con 0x7f75c00b0760 2023-10-06T10:43:13.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.246+0000 7f75c791b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c00ce770 con 0x7f75c0159560 2023-10-06T10:43:13.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.246+0000 7f75c791b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c00cdb60 con 0x7f75c00b49a0 2023-10-06T10:43:13.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.246+0000 7f75c56b7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75c00b0760 0x7f75c015c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:13.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.246+0000 7f75c56b7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75c00b0760 0x7f75c015c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34074/0 (socket says 172.21.15.92:34074) 2023-10-06T10:43:13.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.246+0000 7f75c56b7700 1 -- 172.21.15.92:0/989884821 learned_addr learned my addr 172.21.15.92:0/989884821 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:13.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.246+0000 7f75c5eb8700 1 --2- 172.21.15.92:0/989884821 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75c0159560 0x7f75c00aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:13.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.246+0000 7f75c4eb6700 1 --2- 172.21.15.92:0/989884821 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75c00b49a0 0x7f75c016e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:13.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.246+0000 7f75c56b7700 1 -- 172.21.15.92:0/989884821 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75c00b49a0 msgr2=0x7f75c016e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:13.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.246+0000 7f75c56b7700 1 --2- 172.21.15.92:0/989884821 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75c00b49a0 0x7f75c016e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:13.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.246+0000 7f75c56b7700 1 -- 172.21.15.92:0/989884821 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75c0159560 msgr2=0x7f75c00aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:13.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.246+0000 7f75c56b7700 1 --2- 172.21.15.92:0/989884821 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75c0159560 0x7f75c00aef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:13.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.246+0000 7f75c56b7700 1 -- 172.21.15.92:0/989884821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b00094f0 con 0x7f75c00b0760 2023-10-06T10:43:13.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.247+0000 7f75c56b7700 1 --2- 172.21.15.92:0/989884821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75c00b0760 0x7f75c015c5b0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00e0e0 tx=0x7f75bc0098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:13.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.247+0000 7f75b57fa700 1 -- 172.21.15.92:0/989884821 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc00ec20 con 0x7f75c00b0760 2023-10-06T10:43:13.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.247+0000 7f75b57fa700 1 -- 172.21.15.92:0/989884821 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75bc00ed80 con 0x7f75c00b0760 2023-10-06T10:43:13.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.247+0000 7f75c791b700 1 -- 172.21.15.92:0/989884821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c00cde40 con 0x7f75c00b0760 2023-10-06T10:43:13.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.247+0000 7f75b57fa700 1 -- 172.21.15.92:0/989884821 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc014670 con 0x7f75c00b0760 2023-10-06T10:43:13.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.247+0000 7f75c791b700 1 -- 172.21.15.92:0/989884821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c00ce400 con 0x7f75c00b0760 2023-10-06T10:43:13.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.248+0000 7f75c791b700 1 -- 172.21.15.92:0/989884821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75c0064a90 con 0x7f75c00b0760 2023-10-06T10:43:13.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.248+0000 7f75b57fa700 1 -- 172.21.15.92:0/989884821 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f75bc014a20 con 0x7f75c00b0760 2023-10-06T10:43:13.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.248+0000 7f75b57fa700 1 --2- 172.21.15.92:0/989884821 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f75a4066f30 0x7f75a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:13.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.249+0000 7f75b57fa700 1 -- 172.21.15.92:0/989884821 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f75bc084d70 con 0x7f75c00b0760 2023-10-06T10:43:13.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.249+0000 7f75c4eb6700 1 --2- 172.21.15.92:0/989884821 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f75a4066f30 0x7f75a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:13.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.249+0000 7f75c4eb6700 1 --2- 172.21.15.92:0/989884821 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f75a4066f30 0x7f75a40693e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f75c0088de0 tx=0x7f75b0016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:13.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.253+0000 7f75b57fa700 1 -- 172.21.15.92:0/989884821 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75bc051d40 con 0x7f75c00b0760 2023-10-06T10:43:13.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.481+0000 7f75c791b700 1 -- 172.21.15.92:0/989884821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75c00b17e0 con 0x7f75c00b0760 2023-10-06T10:43:13.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.482+0000 7f75b57fa700 1 -- 172.21.15.92:0/989884821 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4633 (secure 0 0 0) 0x7f75bc055360 con 0x7f75c00b0760 2023-10-06T10:43:13.483 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:13.483 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":72,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3469511807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3469511807},{"type":"v1","addr":"172.21.15.124:6834","nonce":3469511807}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:03.173076+0000","modified":"2023-10-06T10:43:13.353809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7164,"mds_1":7149,"mds_2":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"a","rank":2,"incarnation":71,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/4003376156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4003376156},{"type":"v1","addr":"172.21.15.92:6837","nonce":4003376156}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7149":{"gid":7149,"name":"c","rank":1,"incarnation":69,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6836/3268908749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3268908749},{"type":"v1","addr":"172.21.15.92:6836","nonce":3268908749}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7164":{"gid":7164,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/2971348516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2971348516},{"type":"v1","addr":"172.21.15.124:6835","nonce":2971348516}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T10:43:13.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.484+0000 7f75c791b700 1 -- 172.21.15.92:0/989884821 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f75a4066f30 msgr2=0x7f75a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:13.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.484+0000 7f75c791b700 1 --2- 172.21.15.92:0/989884821 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f75a4066f30 0x7f75a40693e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f75c0088de0 tx=0x7f75b0016000 comp rx=0 tx=0).stop 2023-10-06T10:43:13.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.484+0000 7f75c791b700 1 -- 172.21.15.92:0/989884821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75c00b0760 msgr2=0x7f75c015c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:13.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.484+0000 7f75c791b700 1 --2- 172.21.15.92:0/989884821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75c00b0760 0x7f75c015c5b0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00e0e0 tx=0x7f75bc0098d0 comp rx=0 tx=0).stop 2023-10-06T10:43:13.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.484+0000 7f75c791b700 1 -- 172.21.15.92:0/989884821 shutdown_connections 2023-10-06T10:43:13.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.484+0000 7f75c791b700 1 --2- 172.21.15.92:0/989884821 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f75a4066f30 0x7f75a40693e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:13.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.484+0000 7f75c791b700 1 --2- 172.21.15.92:0/989884821 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75c0159560 0x7f75c00aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:13.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.484+0000 7f75c791b700 1 --2- 172.21.15.92:0/989884821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75c00b0760 0x7f75c015c5b0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:13.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.484+0000 7f75c791b700 1 --2- 172.21.15.92:0/989884821 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75c00b49a0 0x7f75c016e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:13.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.484+0000 7f75c791b700 1 -- 172.21.15.92:0/989884821 >> 172.21.15.92:0/989884821 conn(0x7f75c00997f0 msgr2=0x7f75c00b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:13.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.484+0000 7f75c791b700 1 -- 172.21.15.92:0/989884821 shutdown_connections 2023-10-06T10:43:13.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:13.484+0000 7f75c791b700 1 -- 172.21.15.92:0/989884821 wait complete. 2023-10-06T10:43:13.486 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 72 2023-10-06T10:43:13.899 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T10:43:03.173076+0000', 'modified': '2023-10-06T10:43:13.353809+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 7164, 'mds_1': 7149, 'mds_2': 7146}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7146': {'gid': 7146, 'name': 'a', 'rank': 2, 'incarnation': 71, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.92:6837/4003376156', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 4003376156}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 4003376156}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_7149': {'gid': 7149, 'name': 'c', 'rank': 1, 'incarnation': 69, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.92:6836/3268908749', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 3268908749}, {'type': 'v1', 'addr': '172.21.15.92:6836', 'nonce': 3268908749}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_7164': {'gid': 7164, 'name': 'd', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.124:6835/2971348516', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6833', 'nonce': 2971348516}, {'type': 'v1', 'addr': '172.21.15.124:6835', 'nonce': 2971348516}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T10:43:13.899 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-10-06T10:43:13.899 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T10:43:14.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.014+0000 7f6103656700 1 Processor -- start 2023-10-06T10:43:14.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.014+0000 7f6103656700 1 -- start start 2023-10-06T10:43:14.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.015+0000 7f6103656700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f60fc0ad320 0x7f60fc0ac230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.015+0000 7f6103656700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60fc159f40 con 0x7f60fc0ad6f0 2023-10-06T10:43:14.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.015+0000 7f6103656700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60fc15a220 con 0x7f60fc068cd0 2023-10-06T10:43:14.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.015+0000 7f6103656700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60fc15a500 con 0x7f60fc0ad320 2023-10-06T10:43:14.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.015+0000 7f6100bf1700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f60fc0ad6f0 0x7f60fc0685c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44910/0 (socket says 172.21.15.92:44910) 2023-10-06T10:43:14.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.015+0000 7f6100bf1700 1 -- 172.21.15.92:0/2789202635 learned_addr learned my addr 172.21.15.92:0/2789202635 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:14.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f61013f2700 1 --2- 172.21.15.92:0/2789202635 >> v2:172.21.15.124:3300/0 conn(0x7f60fc0ad320 0x7f60fc0ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f60f3fff700 1 -- 172.21.15.92:0/2789202635 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2110905859 0 0) 0x7f60fc159f40 con 0x7f60fc0ad6f0 2023-10-06T10:43:14.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f60f3fff700 1 -- 172.21.15.92:0/2789202635 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60e8003570 con 0x7f60fc0ad6f0 2023-10-06T10:43:14.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f60f3fff700 1 -- 172.21.15.92:0/2789202635 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3012233477 0 0) 0x7f60e8003570 con 0x7f60fc0ad6f0 2023-10-06T10:43:14.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f60f3fff700 1 -- 172.21.15.92:0/2789202635 >> v2:172.21.15.124:3300/0 conn(0x7f60fc0ad320 msgr2=0x7f60fc0ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f60f3fff700 1 --2- 172.21.15.92:0/2789202635 >> v2:172.21.15.124:3300/0 conn(0x7f60fc0ad320 0x7f60fc0ac230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f60f3fff700 1 -- 172.21.15.92:0/2789202635 >> v1:172.21.15.124:6789/0 conn(0x7f60fc068cd0 legacy=0x7f60fc15b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f60f3fff700 1 -- 172.21.15.92:0/2789202635 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60fc15a7e0 con 0x7f60fc0ad6f0 2023-10-06T10:43:14.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f60f3fff700 1 -- 172.21.15.92:0/2789202635 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f60ec001470 con 0x7f60fc0ad6f0 2023-10-06T10:43:14.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f60f3fff700 1 -- 172.21.15.92:0/2789202635 >> v1:172.21.15.92:6789/0 conn(0x7f60fc0ad6f0 legacy=0x7f60fc0685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f60f3fff700 1 --2- 172.21.15.92:0/2789202635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60e8003a60 0x7f60e8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f60f3fff700 1 --2- 172.21.15.92:0/2789202635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60e8004810 0x7f60e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f60f3fff700 1 --2- 172.21.15.92:0/2789202635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60e80051c0 0x7f60e800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f60f3fff700 1 -- 172.21.15.92:0/2789202635 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc15a500 con 0x7f60e8004810 2023-10-06T10:43:14.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f60f3fff700 1 -- 172.21.15.92:0/2789202635 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e800c0e0 con 0x7f60e8003a60 2023-10-06T10:43:14.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f60f3fff700 1 -- 172.21.15.92:0/2789202635 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60e800c2a0 con 0x7f60e80051c0 2023-10-06T10:43:14.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f6100bf1700 1 --2- 172.21.15.92:0/2789202635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60e8004810 0x7f60e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f61013f2700 1 --2- 172.21.15.92:0/2789202635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60e8003a60 0x7f60e8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f6101bf3700 1 --2- 172.21.15.92:0/2789202635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60e80051c0 0x7f60e800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f6100bf1700 1 -- 172.21.15.92:0/2789202635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60e80051c0 msgr2=0x7f60e800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f6100bf1700 1 --2- 172.21.15.92:0/2789202635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60e80051c0 0x7f60e800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f6100bf1700 1 -- 172.21.15.92:0/2789202635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60e8003a60 msgr2=0x7f60e8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f6100bf1700 1 --2- 172.21.15.92:0/2789202635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60e8003a60 0x7f60e8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f6100bf1700 1 -- 172.21.15.92:0/2789202635 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e8003570 con 0x7f60e8004810 2023-10-06T10:43:14.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f6101bf3700 1 -- 172.21.15.92:0/2789202635 reap_dead start 2023-10-06T10:43:14.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f61013f2700 1 --2- 172.21.15.92:0/2789202635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60e8003a60 0x7f60e8003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:14.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.016+0000 7f6100bf1700 1 --2- 172.21.15.92:0/2789202635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60e8004810 0x7f60e8004c80 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00c3c0 tx=0x7f60ec001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aefe1dbc9eb9d59 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.017+0000 7f60f3fff700 1 -- 172.21.15.92:0/2789202635 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec00cc90 con 0x7f60e8004810 2023-10-06T10:43:14.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.017+0000 7f60f3fff700 1 -- 172.21.15.92:0/2789202635 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ec00cdf0 con 0x7f60e8004810 2023-10-06T10:43:14.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.017+0000 7f60f3fff700 1 -- 172.21.15.92:0/2789202635 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec005b70 con 0x7f60e8004810 2023-10-06T10:43:14.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.017+0000 7f6103656700 1 -- 172.21.15.92:0/2789202635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60e8004810 msgr2=0x7f60e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.017+0000 7f6103656700 1 --2- 172.21.15.92:0/2789202635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60e8004810 0x7f60e8004c80 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00c3c0 tx=0x7f60ec001e70 comp rx=0 tx=0).stop 2023-10-06T10:43:14.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.017+0000 7f6103656700 1 -- 172.21.15.92:0/2789202635 shutdown_connections 2023-10-06T10:43:14.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.017+0000 7f6103656700 1 --2- 172.21.15.92:0/2789202635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60e8004810 0x7f60e8004c80 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.017+0000 7f6103656700 1 -- 172.21.15.92:0/2789202635 >> 172.21.15.92:0/2789202635 conn(0x7f60fc06dcd0 msgr2=0x7f60fc06e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:14.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.017+0000 7f6103656700 1 -- 172.21.15.92:0/2789202635 shutdown_connections 2023-10-06T10:43:14.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.017+0000 7f6103656700 1 -- 172.21.15.92:0/2789202635 wait complete. 2023-10-06T10:43:14.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.018+0000 7f6103656700 1 Processor -- start 2023-10-06T10:43:14.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.018+0000 7f6103656700 1 -- start start 2023-10-06T10:43:14.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.019+0000 7f6103656700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60e8004810 0x7f60fc0b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.019+0000 7f6103656700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60fc16da20 0x7f60fc155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.019+0000 7f6103656700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60fc0af730 0x7f60fc0afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.019+0000 7f6103656700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc0b2730 con 0x7f60fc16da20 2023-10-06T10:43:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.019+0000 7f6103656700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc157970 con 0x7f60e8004810 2023-10-06T10:43:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.019+0000 7f6103656700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc1584d0 con 0x7f60fc0af730 2023-10-06T10:43:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.019+0000 7f6100bf1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60fc16da20 0x7f60fc155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.019+0000 7f6100bf1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60fc16da20 0x7f60fc155ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48280/0 (socket says 172.21.15.92:48280) 2023-10-06T10:43:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.019+0000 7f6100bf1700 1 -- 172.21.15.92:0/484774536 learned_addr learned my addr 172.21.15.92:0/484774536 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.019+0000 7f61013f2700 1 --2- 172.21.15.92:0/484774536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60e8004810 0x7f60fc0b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.019+0000 7f6101bf3700 1 --2- 172.21.15.92:0/484774536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60fc0af730 0x7f60fc0afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.020+0000 7f6100bf1700 1 -- 172.21.15.92:0/484774536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60fc0af730 msgr2=0x7f60fc0afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.020+0000 7f6100bf1700 1 --2- 172.21.15.92:0/484774536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60fc0af730 0x7f60fc0afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.020+0000 7f6100bf1700 1 -- 172.21.15.92:0/484774536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60e8004810 msgr2=0x7f60fc0b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.020+0000 7f6100bf1700 1 --2- 172.21.15.92:0/484774536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60e8004810 0x7f60fc0b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.020+0000 7f6100bf1700 1 -- 172.21.15.92:0/484774536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60fc157ba0 con 0x7f60fc16da20 2023-10-06T10:43:14.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.020+0000 7f6100bf1700 1 --2- 172.21.15.92:0/484774536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60fc16da20 0x7f60fc155ac0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f60e800c0b0 tx=0x7f60ec015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.020+0000 7f60f17fa700 1 -- 172.21.15.92:0/484774536 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec022070 con 0x7f60fc16da20 2023-10-06T10:43:14.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.020+0000 7f60f17fa700 1 -- 172.21.15.92:0/484774536 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ec001470 con 0x7f60fc16da20 2023-10-06T10:43:14.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.020+0000 7f6103656700 1 -- 172.21.15.92:0/484774536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60fc157e80 con 0x7f60fc16da20 2023-10-06T10:43:14.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.020+0000 7f60f17fa700 1 -- 172.21.15.92:0/484774536 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec007300 con 0x7f60fc16da20 2023-10-06T10:43:14.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.020+0000 7f6103656700 1 -- 172.21.15.92:0/484774536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60fc0b23e0 con 0x7f60fc16da20 2023-10-06T10:43:14.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.021+0000 7f60f17fa700 1 -- 172.21.15.92:0/484774536 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f60ec007670 con 0x7f60fc16da20 2023-10-06T10:43:14.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.022+0000 7f60f17fa700 1 --2- 172.21.15.92:0/484774536 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f60e0066f30 0x7f60e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.022+0000 7f60f17fa700 1 -- 172.21.15.92:0/484774536 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f60ec085840 con 0x7f60fc16da20 2023-10-06T10:43:14.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.022+0000 7f61013f2700 1 --2- 172.21.15.92:0/484774536 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f60e0066f30 0x7f60e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.022+0000 7f61013f2700 1 --2- 172.21.15.92:0/484774536 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f60e0066f30 0x7f60e00693e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f60f8009fd0 tx=0x7f60f800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.024+0000 7f610c94eb80 1 Processor -- start 2023-10-06T10:43:14.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.024+0000 7f610c94eb80 1 -- start start 2023-10-06T10:43:14.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.025+0000 7f610c94eb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5560845e2a90 0x5560845e2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.025+0000 7f610c94eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459e9a0 0x55608459b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.025+0000 7f610c94eb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55608459cbd0 0x55608459bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.025+0000 7f610c94eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608451bc80 con 0x55608459e9a0 2023-10-06T10:43:14.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.025+0000 7f610c94eb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560844f2a70 con 0x55608459cbd0 2023-10-06T10:43:14.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.025+0000 7f610c94eb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5560844f0270 con 0x5560845e2a90 2023-10-06T10:43:14.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.025+0000 7f6100bf1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459e9a0 0x55608459b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.025+0000 7f6100bf1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459e9a0 0x55608459b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48290/0 (socket says 172.21.15.92:48290) 2023-10-06T10:43:14.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.025+0000 7f6100bf1700 1 -- 172.21.15.92:0/787901136 learned_addr learned my addr 172.21.15.92:0/787901136 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:14.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.025+0000 7f61013f2700 1 --2- 172.21.15.92:0/787901136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5560845e2a90 0x5560845e2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.025+0000 7f6101bf3700 1 --2- 172.21.15.92:0/787901136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55608459cbd0 0x55608459bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.025+0000 7f6100bf1700 1 -- 172.21.15.92:0/787901136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5560845e2a90 msgr2=0x5560845e2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.025+0000 7f6100bf1700 1 --2- 172.21.15.92:0/787901136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5560845e2a90 0x5560845e2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.025+0000 7f6100bf1700 1 -- 172.21.15.92:0/787901136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55608459cbd0 msgr2=0x55608459bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.025+0000 7f6100bf1700 1 --2- 172.21.15.92:0/787901136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55608459cbd0 0x55608459bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.025+0000 7f6100bf1700 1 -- 172.21.15.92:0/787901136 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556084386270 con 0x55608459e9a0 2023-10-06T10:43:14.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.025+0000 7f6101bf3700 1 --2- 172.21.15.92:0/787901136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55608459cbd0 0x55608459bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:14.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.025+0000 7f6100bf1700 1 --2- 172.21.15.92:0/787901136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459e9a0 0x55608459b580 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f60ec083a90 tx=0x7f60ec083a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9218f26aff7215d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.025+0000 7f60e67fc700 1 -- 172.21.15.92:0/787901136 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec056bd0 con 0x55608459e9a0 2023-10-06T10:43:14.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.025+0000 7f60e67fc700 1 -- 172.21.15.92:0/787901136 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ec0564d0 con 0x55608459e9a0 2023-10-06T10:43:14.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.025+0000 7f60e67fc700 1 -- 172.21.15.92:0/787901136 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec059720 con 0x55608459e9a0 2023-10-06T10:43:14.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.026+0000 7f610c94eb80 1 -- 172.21.15.92:0/787901136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459e9a0 msgr2=0x55608459b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.026+0000 7f610c94eb80 1 --2- 172.21.15.92:0/787901136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459e9a0 0x55608459b580 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f60ec083a90 tx=0x7f60ec083a20 comp rx=0 tx=0).stop 2023-10-06T10:43:14.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.026+0000 7f610c94eb80 1 -- 172.21.15.92:0/787901136 shutdown_connections 2023-10-06T10:43:14.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.026+0000 7f610c94eb80 1 --2- 172.21.15.92:0/787901136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55608459cbd0 0x55608459bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.026+0000 7f610c94eb80 1 --2- 172.21.15.92:0/787901136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5560845e2a90 0x5560845e2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.026+0000 7f610c94eb80 1 --2- 172.21.15.92:0/787901136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459e9a0 0x55608459b580 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.026+0000 7f610c94eb80 1 -- 172.21.15.92:0/787901136 >> 172.21.15.92:0/787901136 conn(0x55608459d070 msgr2=0x55608459d440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:14.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.026+0000 7f610c94eb80 1 -- 172.21.15.92:0/787901136 shutdown_connections 2023-10-06T10:43:14.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.026+0000 7f610c94eb80 1 -- 172.21.15.92:0/787901136 wait complete. 2023-10-06T10:43:14.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.026+0000 7f610c94eb80 1 -- start start 2023-10-06T10:43:14.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.027+0000 7f610c94eb80 1 Processor -- start 2023-10-06T10:43:14.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.027+0000 7f610c94eb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55608459cbd0 0x5560845bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.027+0000 7f610c94eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459e9a0 0x5560845dced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.027+0000 7f610c94eb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5560845e2a90 0x5560845c5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.027+0000 7f610c94eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560844ec060 con 0x55608459e9a0 2023-10-06T10:43:14.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.027+0000 7f610c94eb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x556084497890 con 0x55608459cbd0 2023-10-06T10:43:14.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.027+0000 7f610c94eb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5560844b2ea0 con 0x5560845e2a90 2023-10-06T10:43:14.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.027+0000 7f610c94eb80 10 client.0 _ll_register_callbacks cb 0x556084388f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:14.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.027+0000 7f6100bf1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459e9a0 0x5560845dced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.027+0000 7f6100bf1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459e9a0 0x5560845dced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48300/0 (socket says 172.21.15.92:48300) 2023-10-06T10:43:14.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.027+0000 7f6100bf1700 1 -- 172.21.15.92:0/459351157 learned_addr learned my addr 172.21.15.92:0/459351157 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:14.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.027+0000 7f6101bf3700 1 --2- 172.21.15.92:0/459351157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5560845e2a90 0x5560845c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f61013f2700 1 --2- 172.21.15.92:0/459351157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55608459cbd0 0x5560845bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f6100bf1700 1 -- 172.21.15.92:0/459351157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5560845e2a90 msgr2=0x5560845c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f6100bf1700 1 --2- 172.21.15.92:0/459351157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5560845e2a90 0x5560845c5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f6100bf1700 1 -- 172.21.15.92:0/459351157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55608459cbd0 msgr2=0x5560845bfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f6100bf1700 1 --2- 172.21.15.92:0/459351157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55608459cbd0 0x5560845bfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f6100bf1700 1 -- 172.21.15.92:0/459351157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60fc0b23e0 con 0x55608459e9a0 2023-10-06T10:43:14.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f61013f2700 1 --2- 172.21.15.92:0/459351157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55608459cbd0 0x5560845bfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:14.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f6100bf1700 1 --2- 172.21.15.92:0/459351157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459e9a0 0x5560845dced0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f60ec056460 tx=0x7f60ec009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de796052eb63c5af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f60db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f60db7fe700 1 -- 172.21.15.92:0/459351157 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec059720 con 0x55608459e9a0 2023-10-06T10:43:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f60db7fe700 1 -- 172.21.15.92:0/459351157 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f60ec028d70 con 0x55608459e9a0 2023-10-06T10:43:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f610c94eb80 1 -- 172.21.15.92:0/459351157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556084590690 con 0x55608459e9a0 2023-10-06T10:43:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f60db7fe700 1 -- 172.21.15.92:0/459351157 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60ec056940 con 0x55608459e9a0 2023-10-06T10:43:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f610c94eb80 10 client.7353 fetch_fsmap learned FSMap version 72 2023-10-06T10:43:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f610c94eb80 1 -- 172.21.15.92:0/459351157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x556084328fe0 con 0x55608459e9a0 2023-10-06T10:43:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f60db7fe700 1 -- 172.21.15.92:0/459351157 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556084590690 con 0x55608459e9a0 2023-10-06T10:43:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f60db7fe700 1 -- 172.21.15.92:0/459351157 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 72) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f60ec081070 con 0x55608459e9a0 2023-10-06T10:43:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f610c94eb80 10 client.7353 fetch_fsmap finished waiting for FSMap version 72 2023-10-06T10:43:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f610c94eb80 10 client.7353 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T10:43:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f610c94eb80 20 client.7353 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f610c94eb80 1 --2- 172.21.15.92:0/459351157 >> [v2:172.21.15.92:6835/4003376156,v1:172.21.15.92:6837/4003376156] conn(0x5560845c2940 0x5560845c2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f610c94eb80 4 client.7353 mds_command: new command op to 7146 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:43:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.028+0000 7f610c94eb80 1 -- 172.21.15.92:0/459351157 --> [v2:172.21.15.92:6835/4003376156,v1:172.21.15.92:6837/4003376156] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55608456d5c0 con 0x5560845c2940 2023-10-06T10:43:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.029+0000 7f61013f2700 1 --2- 172.21.15.92:0/459351157 >> [v2:172.21.15.92:6835/4003376156,v1:172.21.15.92:6837/4003376156] conn(0x5560845c2940 0x5560845c2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.029+0000 7f60db7fe700 10 client.7353 ms_handle_connect on v2:172.21.15.92:6835/4003376156 2023-10-06T10:43:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.029+0000 7f61013f2700 1 --2- 172.21.15.92:0/459351157 >> [v2:172.21.15.92:6835/4003376156,v1:172.21.15.92:6837/4003376156] conn(0x5560845c2940 0x5560845c2d10 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=edd021ea3f8851b6 server_cookie=5a999fdd4d37f20f in_seq=0 out_seq=0 2023-10-06T10:43:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.029+0000 7f60db7fe700 1 -- 172.21.15.92:0/459351157 <== mds.2 v2:172.21.15.92:6835/4003376156 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55608456d5c0 con 0x5560845c2940 2023-10-06T10:43:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.029+0000 7f60db7fe700 10 client.7353 handle_command_reply: tid=0 2023-10-06T10:43:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.029+0000 7f610c94eb80 1 client.7353 shutdown 2023-10-06T10:43:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.029+0000 7f610c94eb80 1 -- 172.21.15.92:0/459351157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459e9a0 msgr2=0x5560845dced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.029+0000 7f610c94eb80 1 --2- 172.21.15.92:0/459351157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459e9a0 0x5560845dced0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f60ec056460 tx=0x7f60ec009030 comp rx=0 tx=0).stop 2023-10-06T10:43:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.030+0000 7f610c94eb80 1 -- 172.21.15.92:0/459351157 shutdown_connections 2023-10-06T10:43:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.030+0000 7f610c94eb80 1 --2- 172.21.15.92:0/459351157 >> [v2:172.21.15.92:6835/4003376156,v1:172.21.15.92:6837/4003376156] conn(0x5560845c2940 0x5560845c2d10 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.030+0000 7f610c94eb80 1 --2- 172.21.15.92:0/459351157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5560845e2a90 0x5560845c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.030+0000 7f60db7fe700 0 client.7353 ms_handle_reset on v2:172.21.15.92:6835/4003376156 2023-10-06T10:43:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.030+0000 7f610c94eb80 1 --2- 172.21.15.92:0/459351157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55608459cbd0 0x5560845bfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.030+0000 7f610c94eb80 1 --2- 172.21.15.92:0/459351157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459e9a0 0x5560845dced0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.030+0000 7f610c94eb80 1 -- 172.21.15.92:0/459351157 >> 172.21.15.92:0/459351157 conn(0x55608459d070 msgr2=0x556084598c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.030+0000 7f610c94eb80 1 -- 172.21.15.92:0/459351157 shutdown_connections 2023-10-06T10:43:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.030+0000 7f610c94eb80 1 -- 172.21.15.92:0/459351157 wait complete. 2023-10-06T10:43:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.030+0000 7f610c94eb80 20 client.7353 trim_cache size 0 max 16384 2023-10-06T10:43:14.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.046+0000 7f610c94eb80 1 Processor -- start 2023-10-06T10:43:14.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.047+0000 7f610c94eb80 1 -- start start 2023-10-06T10:43:14.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.047+0000 7f610c94eb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55608459cbd0 0x556084598b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.047+0000 7f610c94eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459e9a0 0x5560845fb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.047+0000 7f610c94eb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5560845e2a90 0x5560845fb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.047+0000 7f610c94eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608436ee30 con 0x55608459e9a0 2023-10-06T10:43:14.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.047+0000 7f610c94eb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608459c810 con 0x55608459cbd0 2023-10-06T10:43:14.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.047+0000 7f610c94eb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5560845bf580 con 0x5560845e2a90 2023-10-06T10:43:14.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.047+0000 7f6100bf1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459e9a0 0x5560845fb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.047+0000 7f6100bf1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459e9a0 0x5560845fb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48312/0 (socket says 172.21.15.92:48312) 2023-10-06T10:43:14.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.047+0000 7f6100bf1700 1 -- 172.21.15.92:0/2691092150 learned_addr learned my addr 172.21.15.92:0/2691092150 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:14.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.048+0000 7f6100bf1700 1 -- 172.21.15.92:0/2691092150 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5560845e2a90 msgr2=0x5560845fb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.048+0000 7f6100bf1700 1 --2- 172.21.15.92:0/2691092150 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5560845e2a90 0x5560845fb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.048+0000 7f6100bf1700 1 -- 172.21.15.92:0/2691092150 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55608459cbd0 msgr2=0x556084598b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.048+0000 7f6100bf1700 1 --2- 172.21.15.92:0/2691092150 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55608459cbd0 0x556084598b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.048+0000 7f6100bf1700 1 -- 172.21.15.92:0/2691092150 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560844ba780 con 0x55608459e9a0 2023-10-06T10:43:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.048+0000 7f6100bf1700 1 --2- 172.21.15.92:0/2691092150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459e9a0 0x5560845fb0c0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f60ec052480 tx=0x7f60ec06f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da4abc49580387a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.048+0000 7f60e6ffd700 1 -- 172.21.15.92:0/2691092150 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec056bd0 con 0x55608459e9a0 2023-10-06T10:43:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.048+0000 7f60e6ffd700 1 -- 172.21.15.92:0/2691092150 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ec059720 con 0x55608459e9a0 2023-10-06T10:43:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.048+0000 7f60e6ffd700 1 -- 172.21.15.92:0/2691092150 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec05d470 con 0x55608459e9a0 2023-10-06T10:43:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.048+0000 7f610c94eb80 1 -- 172.21.15.92:0/2691092150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459e9a0 msgr2=0x5560845fb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.048+0000 7f610c94eb80 1 --2- 172.21.15.92:0/2691092150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459e9a0 0x5560845fb0c0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f60ec052480 tx=0x7f60ec06f040 comp rx=0 tx=0).stop 2023-10-06T10:43:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.048+0000 7f610c94eb80 1 -- 172.21.15.92:0/2691092150 shutdown_connections 2023-10-06T10:43:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.048+0000 7f610c94eb80 1 --2- 172.21.15.92:0/2691092150 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5560845e2a90 0x5560845fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.048+0000 7f610c94eb80 1 --2- 172.21.15.92:0/2691092150 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55608459cbd0 0x556084598b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.048+0000 7f610c94eb80 1 --2- 172.21.15.92:0/2691092150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459e9a0 0x5560845fb0c0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.048+0000 7f610c94eb80 1 -- 172.21.15.92:0/2691092150 >> 172.21.15.92:0/2691092150 conn(0x55608459d070 msgr2=0x5560845c6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:14.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.048+0000 7f610c94eb80 1 -- 172.21.15.92:0/2691092150 shutdown_connections 2023-10-06T10:43:14.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.048+0000 7f610c94eb80 1 -- 172.21.15.92:0/2691092150 wait complete. 2023-10-06T10:43:14.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.049+0000 7f610c94eb80 1 -- start start 2023-10-06T10:43:14.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.049+0000 7f610c94eb80 1 Processor -- start 2023-10-06T10:43:14.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.050+0000 7f610c94eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459cbd0 0x5560845e58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.050+0000 7f610c94eb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55608459e9a0 0x5560845ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.050+0000 7f610c94eb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5560845e2a90 0x5560845cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.050+0000 7f610c94eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560845cb630 con 0x55608459cbd0 2023-10-06T10:43:14.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.050+0000 7f610c94eb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560845ddd30 con 0x55608459e9a0 2023-10-06T10:43:14.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.050+0000 7f610c94eb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5560845df1c0 con 0x5560845e2a90 2023-10-06T10:43:14.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.050+0000 7f610c94eb80 10 client.0 _ll_register_callbacks cb 0x556084388f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:14.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.050+0000 7f61013f2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459cbd0 0x5560845e58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.050+0000 7f61013f2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459cbd0 0x5560845e58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48324/0 (socket says 172.21.15.92:48324) 2023-10-06T10:43:14.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.050+0000 7f61013f2700 1 -- 172.21.15.92:0/3256949496 learned_addr learned my addr 172.21.15.92:0/3256949496 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.050+0000 7f6100bf1700 1 --2- 172.21.15.92:0/3256949496 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55608459e9a0 0x5560845ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.050+0000 7f6101bf3700 1 --2- 172.21.15.92:0/3256949496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5560845e2a90 0x5560845cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.050+0000 7f61013f2700 1 -- 172.21.15.92:0/3256949496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5560845e2a90 msgr2=0x5560845cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.050+0000 7f61013f2700 1 --2- 172.21.15.92:0/3256949496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5560845e2a90 0x5560845cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.050+0000 7f61013f2700 1 -- 172.21.15.92:0/3256949496 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55608459e9a0 msgr2=0x5560845ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.050+0000 7f61013f2700 1 --2- 172.21.15.92:0/3256949496 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55608459e9a0 0x5560845ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.050+0000 7f61013f2700 1 -- 172.21.15.92:0/3256949496 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60fc0b23e0 con 0x55608459cbd0 2023-10-06T10:43:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.050+0000 7f6101bf3700 1 --2- 172.21.15.92:0/3256949496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5560845e2a90 0x5560845cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.050+0000 7f6100bf1700 1 --2- 172.21.15.92:0/3256949496 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55608459e9a0 0x5560845ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.050+0000 7f61013f2700 1 --2- 172.21.15.92:0/3256949496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459cbd0 0x5560845e58a0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f60f8008010 tx=0x7f60f8027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=454595b821bbbcd7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.050+0000 7f60db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:14.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.050+0000 7f60db7fe700 1 -- 172.21.15.92:0/3256949496 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f802a040 con 0x55608459cbd0 2023-10-06T10:43:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.051+0000 7f60db7fe700 1 -- 172.21.15.92:0/3256949496 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f60f8025a70 con 0x55608459cbd0 2023-10-06T10:43:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.051+0000 7f610c94eb80 1 -- 172.21.15.92:0/3256949496 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60ec0564d0 con 0x55608459cbd0 2023-10-06T10:43:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.051+0000 7f60db7fe700 1 -- 172.21.15.92:0/3256949496 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60f800e050 con 0x55608459cbd0 2023-10-06T10:43:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.051+0000 7f610c94eb80 10 client.7359 fetch_fsmap learned FSMap version 72 2023-10-06T10:43:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.051+0000 7f610c94eb80 1 -- 172.21.15.92:0/3256949496 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x5560844a1470 con 0x55608459cbd0 2023-10-06T10:43:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.051+0000 7f60db7fe700 1 -- 172.21.15.92:0/3256949496 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ec0564d0 con 0x55608459cbd0 2023-10-06T10:43:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.051+0000 7f60db7fe700 1 -- 172.21.15.92:0/3256949496 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 72) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f60f802e460 con 0x55608459cbd0 2023-10-06T10:43:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.051+0000 7f610c94eb80 10 client.7359 fetch_fsmap finished waiting for FSMap version 72 2023-10-06T10:43:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.051+0000 7f610c94eb80 10 client.7359 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T10:43:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.051+0000 7f610c94eb80 20 client.7359 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.051+0000 7f610c94eb80 1 --2- 172.21.15.92:0/3256949496 >> [v2:172.21.15.92:6835/4003376156,v1:172.21.15.92:6837/4003376156] conn(0x5560845c11c0 0x556084646720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.051+0000 7f610c94eb80 4 client.7359 mds_command: new command op to 7146 tid=0[{"prefix": "status"}] 2023-10-06T10:43:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.051+0000 7f610c94eb80 1 -- 172.21.15.92:0/3256949496 --> [v2:172.21.15.92:6835/4003376156,v1:172.21.15.92:6837/4003376156] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55608448ee00 con 0x5560845c11c0 2023-10-06T10:43:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.051+0000 7f6100bf1700 1 --2- 172.21.15.92:0/3256949496 >> [v2:172.21.15.92:6835/4003376156,v1:172.21.15.92:6837/4003376156] conn(0x5560845c11c0 0x556084646720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.051+0000 7f60db7fe700 10 client.7359 ms_handle_connect on v2:172.21.15.92:6835/4003376156 2023-10-06T10:43:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.051+0000 7f6100bf1700 1 --2- 172.21.15.92:0/3256949496 >> [v2:172.21.15.92:6835/4003376156,v1:172.21.15.92:6837/4003376156] conn(0x5560845c11c0 0x556084646720 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d1c98530c3828bf server_cookie=20dc6855491a4c93 in_seq=0 out_seq=0 2023-10-06T10:43:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.051+0000 7f60db7fe700 1 -- 172.21.15.92:0/3256949496 <== mds.2 v2:172.21.15.92:6835/4003376156 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55608448ee00 con 0x5560845c11c0 2023-10-06T10:43:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.051+0000 7f60db7fe700 10 client.7359 handle_command_reply: tid=0 2023-10-06T10:43:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.051+0000 7f610c94eb80 1 client.7359 shutdown 2023-10-06T10:43:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f610c94eb80 1 -- 172.21.15.92:0/3256949496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459cbd0 msgr2=0x5560845e58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f610c94eb80 1 --2- 172.21.15.92:0/3256949496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459cbd0 0x5560845e58a0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f60f8008010 tx=0x7f60f8027040 comp rx=0 tx=0).stop 2023-10-06T10:43:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f610c94eb80 1 -- 172.21.15.92:0/3256949496 shutdown_connections 2023-10-06T10:43:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f610c94eb80 1 --2- 172.21.15.92:0/3256949496 >> [v2:172.21.15.92:6835/4003376156,v1:172.21.15.92:6837/4003376156] conn(0x5560845c11c0 0x556084646720 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f610c94eb80 1 --2- 172.21.15.92:0/3256949496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5560845e2a90 0x5560845cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f610c94eb80 1 --2- 172.21.15.92:0/3256949496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55608459cbd0 0x5560845e58a0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f60db7fe700 0 client.7359 ms_handle_reset on v2:172.21.15.92:6835/4003376156 2023-10-06T10:43:14.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f610c94eb80 1 --2- 172.21.15.92:0/3256949496 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55608459e9a0 0x5560845ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f610c94eb80 1 -- 172.21.15.92:0/3256949496 >> 172.21.15.92:0/3256949496 conn(0x55608459d070 msgr2=0x556084598ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:14.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f610c94eb80 1 -- 172.21.15.92:0/3256949496 shutdown_connections 2023-10-06T10:43:14.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f610c94eb80 1 -- 172.21.15.92:0/3256949496 wait complete. 2023-10-06T10:43:14.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f610c94eb80 20 client.7359 trim_cache size 0 max 16384 2023-10-06T10:43:14.060 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:43:14.060 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:43:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 2, 2023-10-06T10:43:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 7146, 2023-10-06T10:43:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:43:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:43:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:43:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 1.6980600130000001, 2023-10-06T10:43:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 72, 2023-10-06T10:43:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 78, 2023-10-06T10:43:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 78, 2023-10-06T10:43:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 16.368443147000001 2023-10-06T10:43:14.061 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:43:14.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f6103656700 1 -- 172.21.15.92:0/484774536 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f60e0066f30 msgr2=0x7f60e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f6103656700 1 --2- 172.21.15.92:0/484774536 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f60e0066f30 0x7f60e00693e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f60f8009fd0 tx=0x7f60f800b040 comp rx=0 tx=0).stop 2023-10-06T10:43:14.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f6103656700 1 -- 172.21.15.92:0/484774536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60fc16da20 msgr2=0x7f60fc155ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f6103656700 1 --2- 172.21.15.92:0/484774536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60fc16da20 0x7f60fc155ac0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f60e800c0b0 tx=0x7f60ec015000 comp rx=0 tx=0).stop 2023-10-06T10:43:14.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f6103656700 1 -- 172.21.15.92:0/484774536 shutdown_connections 2023-10-06T10:43:14.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f6103656700 1 --2- 172.21.15.92:0/484774536 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f60e0066f30 0x7f60e00693e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f6103656700 1 --2- 172.21.15.92:0/484774536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60fc0af730 0x7f60fc0afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f6103656700 1 --2- 172.21.15.92:0/484774536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60e8004810 0x7f60fc0b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f6103656700 1 --2- 172.21.15.92:0/484774536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60fc16da20 0x7f60fc155ac0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f6103656700 1 -- 172.21.15.92:0/484774536 >> 172.21.15.92:0/484774536 conn(0x7f60fc06dcd0 msgr2=0x7f60fc15b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:14.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f6103656700 1 -- 172.21.15.92:0/484774536 shutdown_connections 2023-10-06T10:43:14.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.052+0000 7f6103656700 1 -- 172.21.15.92:0/484774536 wait complete. 2023-10-06T10:43:14.065 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T10:43:14.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.179+0000 7fbdce130700 1 Processor -- start 2023-10-06T10:43:14.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.179+0000 7fbdce130700 1 -- start start 2023-10-06T10:43:14.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.180+0000 7fbdce130700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbdc8159440 0x7fbdc80b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.180+0000 7fbdce130700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbdc80b35d0 0x7fbdc80cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.180+0000 7fbdce130700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdc80cc170 0x7fbdc80b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.180+0000 7fbdce130700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc80cb7a0 con 0x7fbdc8159440 2023-10-06T10:43:14.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.180+0000 7fbdce130700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc80b0860 con 0x7fbdc80b35d0 2023-10-06T10:43:14.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.180+0000 7fbdce130700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc80cb150 con 0x7fbdc80cc170 2023-10-06T10:43:14.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.180+0000 7fbdcd12e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbdc8159440 0x7fbdc80b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.180+0000 7fbdcd12e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbdc8159440 0x7fbdc80b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48328/0 (socket says 172.21.15.92:48328) 2023-10-06T10:43:14.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.180+0000 7fbdcd12e700 1 -- 172.21.15.92:0/3331981702 learned_addr learned my addr 172.21.15.92:0/3331981702 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:14.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.180+0000 7fbdcd92f700 1 --2- 172.21.15.92:0/3331981702 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdc80cc170 0x7fbdc80b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.180+0000 7fbdcc92d700 1 --2- 172.21.15.92:0/3331981702 >> v2:172.21.15.124:3300/0 conn(0x7fbdc80b35d0 0x7fbdc80cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.180+0000 7fbdcd12e700 1 -- 172.21.15.92:0/3331981702 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdc80cc170 msgr2=0x7fbdc80b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.180+0000 7fbdcd12e700 1 --2- 172.21.15.92:0/3331981702 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdc80cc170 0x7fbdc80b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.180+0000 7fbdcd12e700 1 -- 172.21.15.92:0/3331981702 >> v2:172.21.15.124:3300/0 conn(0x7fbdc80b35d0 msgr2=0x7fbdc80cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.180+0000 7fbdcd12e700 1 --2- 172.21.15.92:0/3331981702 >> v2:172.21.15.124:3300/0 conn(0x7fbdc80b35d0 0x7fbdc80cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.180+0000 7fbdcd12e700 1 -- 172.21.15.92:0/3331981702 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc80cb430 con 0x7fbdc8159440 2023-10-06T10:43:14.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.181+0000 7fbdcd12e700 1 --2- 172.21.15.92:0/3331981702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc8159440 0x7fbdc80b3090 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80d0220 tx=0x7fbdc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb38df98b91f19 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.181+0000 7fbdbffff700 1 -- 172.21.15.92:0/3331981702 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4009e70 con 0x7fbdc8159440 2023-10-06T10:43:14.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.181+0000 7fbdbffff700 1 -- 172.21.15.92:0/3331981702 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc400b860 con 0x7fbdc8159440 2023-10-06T10:43:14.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.181+0000 7fbdbffff700 1 -- 172.21.15.92:0/3331981702 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc400bc00 con 0x7fbdc8159440 2023-10-06T10:43:14.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.181+0000 7fbdce130700 1 -- 172.21.15.92:0/3331981702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc8159440 msgr2=0x7fbdc80b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.181+0000 7fbdce130700 1 --2- 172.21.15.92:0/3331981702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc8159440 0x7fbdc80b3090 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80d0220 tx=0x7fbdc4009250 comp rx=0 tx=0).stop 2023-10-06T10:43:14.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.181+0000 7fbdce130700 1 -- 172.21.15.92:0/3331981702 shutdown_connections 2023-10-06T10:43:14.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.181+0000 7fbdce130700 1 --2- 172.21.15.92:0/3331981702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc8159440 0x7fbdc80b3090 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.181+0000 7fbdce130700 1 --2- 172.21.15.92:0/3331981702 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdc80cc170 0x7fbdc80b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.181+0000 7fbdce130700 1 --2- 172.21.15.92:0/3331981702 >> v2:172.21.15.124:3300/0 conn(0x7fbdc80b35d0 0x7fbdc80cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.181+0000 7fbdce130700 1 -- 172.21.15.92:0/3331981702 >> 172.21.15.92:0/3331981702 conn(0x7fbdc806ece0 msgr2=0x7fbdc806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:14.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.181+0000 7fbdce130700 1 -- 172.21.15.92:0/3331981702 shutdown_connections 2023-10-06T10:43:14.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.181+0000 7fbdce130700 1 -- 172.21.15.92:0/3331981702 wait complete. 2023-10-06T10:43:14.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.182+0000 7fbdce130700 1 Processor -- start 2023-10-06T10:43:14.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.182+0000 7fbdce130700 1 -- start start 2023-10-06T10:43:14.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.183+0000 7fbdce130700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdc80b35d0 0x7fbdc80c4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.183+0000 7fbdce130700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc80cc170 0x7fbdc8154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.183+0000 7fbdce130700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdc8159440 0x7fbdc816ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.183+0000 7fbdce130700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc80c3f50 con 0x7fbdc80cc170 2023-10-06T10:43:14.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.183+0000 7fbdce130700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc816d5f0 con 0x7fbdc80b35d0 2023-10-06T10:43:14.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.183+0000 7fbdce130700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc816d8d0 con 0x7fbdc8159440 2023-10-06T10:43:14.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.183+0000 7fbdcc92d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc80cc170 0x7fbdc8154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.183+0000 7fbdcc92d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc80cc170 0x7fbdc8154730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48340/0 (socket says 172.21.15.92:48340) 2023-10-06T10:43:14.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.183+0000 7fbdcc92d700 1 -- 172.21.15.92:0/2048770816 learned_addr learned my addr 172.21.15.92:0/2048770816 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:14.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.183+0000 7fbdcd92f700 1 --2- 172.21.15.92:0/2048770816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdc8159440 0x7fbdc816ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.183+0000 7fbdcd12e700 1 --2- 172.21.15.92:0/2048770816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdc80b35d0 0x7fbdc80c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.184+0000 7fbdcc92d700 1 -- 172.21.15.92:0/2048770816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdc8159440 msgr2=0x7fbdc816ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.184+0000 7fbdcc92d700 1 --2- 172.21.15.92:0/2048770816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdc8159440 0x7fbdc816ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.184+0000 7fbdcc92d700 1 -- 172.21.15.92:0/2048770816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdc80b35d0 msgr2=0x7fbdc80c4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.184+0000 7fbdcc92d700 1 --2- 172.21.15.92:0/2048770816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdc80b35d0 0x7fbdc80c4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.184+0000 7fbdcc92d700 1 -- 172.21.15.92:0/2048770816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc40094f0 con 0x7fbdc80cc170 2023-10-06T10:43:14.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.184+0000 7fbdcc92d700 1 --2- 172.21.15.92:0/2048770816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc80cc170 0x7fbdc8154730 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800d890 tx=0x7fbdb8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.184+0000 7fbdbd7fa700 1 -- 172.21.15.92:0/2048770816 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb80125f0 con 0x7fbdc80cc170 2023-10-06T10:43:14.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.184+0000 7fbdbd7fa700 1 -- 172.21.15.92:0/2048770816 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb8012750 con 0x7fbdc80cc170 2023-10-06T10:43:14.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.184+0000 7fbdbd7fa700 1 -- 172.21.15.92:0/2048770816 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8012af0 con 0x7fbdc80cc170 2023-10-06T10:43:14.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.184+0000 7fbdce130700 1 -- 172.21.15.92:0/2048770816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc815b1e0 con 0x7fbdc80cc170 2023-10-06T10:43:14.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.184+0000 7fbdce130700 1 -- 172.21.15.92:0/2048770816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc815b7a0 con 0x7fbdc80cc170 2023-10-06T10:43:14.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.186+0000 7fbdbd7fa700 1 -- 172.21.15.92:0/2048770816 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbdb801e460 con 0x7fbdc80cc170 2023-10-06T10:43:14.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.186+0000 7fbdbd7fa700 1 --2- 172.21.15.92:0/2048770816 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbdac066f30 0x7fbdac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.186+0000 7fbdbd7fa700 1 -- 172.21.15.92:0/2048770816 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fbdb8084d40 con 0x7fbdc80cc170 2023-10-06T10:43:14.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.186+0000 7fbdcd12e700 1 --2- 172.21.15.92:0/2048770816 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbdac066f30 0x7fbdac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.187+0000 7fbdcd12e700 1 --2- 172.21.15.92:0/2048770816 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbdac066f30 0x7fbdac0693e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80d0220 tx=0x7fbdc4017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.188+0000 7fbdd868ab80 1 Processor -- start 2023-10-06T10:43:14.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.188+0000 7fbdd868ab80 1 -- start start 2023-10-06T10:43:14.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.189+0000 7fbdd868ab80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e01e073950 0x55e01e073d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.189+0000 7fbdd868ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e0bdbc0 0x55e01e09c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.189+0000 7fbdd868ab80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e01e09ce60 0x55e01e06f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.189+0000 7fbdd868ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01dff3c80 con 0x55e01e0bdbc0 2023-10-06T10:43:14.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.189+0000 7fbdd868ab80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01dfcaa70 con 0x55e01e09ce60 2023-10-06T10:43:14.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.189+0000 7fbdd868ab80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e01dfc8270 con 0x55e01e073950 2023-10-06T10:43:14.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.189+0000 7fbdcc92d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e0bdbc0 0x55e01e09c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.189+0000 7fbdcc92d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e0bdbc0 0x55e01e09c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48350/0 (socket says 172.21.15.92:48350) 2023-10-06T10:43:14.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.189+0000 7fbdcc92d700 1 -- 172.21.15.92:0/1130893404 learned_addr learned my addr 172.21.15.92:0/1130893404 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:14.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.189+0000 7fbdcd12e700 1 --2- 172.21.15.92:0/1130893404 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e01e073950 0x55e01e073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.189+0000 7fbdcd92f700 1 --2- 172.21.15.92:0/1130893404 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e01e09ce60 0x55e01e06f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.189+0000 7fbdcc92d700 1 -- 172.21.15.92:0/1130893404 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e01e073950 msgr2=0x55e01e073d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.189+0000 7fbdcc92d700 1 --2- 172.21.15.92:0/1130893404 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e01e073950 0x55e01e073d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.189+0000 7fbdcc92d700 1 -- 172.21.15.92:0/1130893404 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e01e09ce60 msgr2=0x55e01e06f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.189+0000 7fbdcc92d700 1 --2- 172.21.15.92:0/1130893404 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e01e09ce60 0x55e01e06f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.189+0000 7fbdcc92d700 1 -- 172.21.15.92:0/1130893404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e01de5e270 con 0x55e01e0bdbc0 2023-10-06T10:43:14.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.189+0000 7fbdcd92f700 1 --2- 172.21.15.92:0/1130893404 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e01e09ce60 0x55e01e06f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:14.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.189+0000 7fbdcd12e700 1 --2- 172.21.15.92:0/1130893404 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e01e073950 0x55e01e073d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:14.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.189+0000 7fbdcc92d700 1 --2- 172.21.15.92:0/1130893404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e0bdbc0 0x55e01e09c920 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8082ff0 tx=0x7fbdb8082f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b2911447ddcf72 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.189+0000 7fbdb27fc700 1 -- 172.21.15.92:0/1130893404 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8058ca0 con 0x55e01e0bdbc0 2023-10-06T10:43:14.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.189+0000 7fbdb27fc700 1 -- 172.21.15.92:0/1130893404 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb8070020 con 0x55e01e0bdbc0 2023-10-06T10:43:14.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.189+0000 7fbdb27fc700 1 -- 172.21.15.92:0/1130893404 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb805c970 con 0x55e01e0bdbc0 2023-10-06T10:43:14.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.190+0000 7fbdd868ab80 1 -- 172.21.15.92:0/1130893404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e0bdbc0 msgr2=0x55e01e09c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.190+0000 7fbdd868ab80 1 --2- 172.21.15.92:0/1130893404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e0bdbc0 0x55e01e09c920 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8082ff0 tx=0x7fbdb8082f20 comp rx=0 tx=0).stop 2023-10-06T10:43:14.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.190+0000 7fbdd868ab80 1 -- 172.21.15.92:0/1130893404 shutdown_connections 2023-10-06T10:43:14.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.190+0000 7fbdd868ab80 1 --2- 172.21.15.92:0/1130893404 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e01e09ce60 0x55e01e06f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.190+0000 7fbdd868ab80 1 --2- 172.21.15.92:0/1130893404 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e01e073950 0x55e01e073d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.190+0000 7fbdd868ab80 1 --2- 172.21.15.92:0/1130893404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e0bdbc0 0x55e01e09c920 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.190+0000 7fbdd868ab80 1 -- 172.21.15.92:0/1130893404 >> 172.21.15.92:0/1130893404 conn(0x55e01e075270 msgr2=0x55e01e075640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:14.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.190+0000 7fbdd868ab80 1 -- 172.21.15.92:0/1130893404 shutdown_connections 2023-10-06T10:43:14.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.190+0000 7fbdd868ab80 1 -- 172.21.15.92:0/1130893404 wait complete. 2023-10-06T10:43:14.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.190+0000 7fbdd868ab80 1 -- start start 2023-10-06T10:43:14.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.191+0000 7fbdd868ab80 1 Processor -- start 2023-10-06T10:43:14.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.191+0000 7fbdd868ab80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e01e073950 0x55e01e099320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.191+0000 7fbdd868ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e09ce60 0x55e01e0b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.191+0000 7fbdd868ab80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e01e0bdbc0 0x55e01e06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.191+0000 7fbdd868ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01dfc4060 con 0x55e01e09ce60 2023-10-06T10:43:14.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.191+0000 7fbdd868ab80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01df6f890 con 0x55e01e073950 2023-10-06T10:43:14.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.191+0000 7fbdd868ab80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e01df8aea0 con 0x55e01e0bdbc0 2023-10-06T10:43:14.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.192+0000 7fbdd868ab80 10 client.0 _ll_register_callbacks cb 0x55e01de60f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:14.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.192+0000 7fbdcc92d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e09ce60 0x55e01e0b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.192+0000 7fbdcc92d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e09ce60 0x55e01e0b35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48364/0 (socket says 172.21.15.92:48364) 2023-10-06T10:43:14.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.192+0000 7fbdcc92d700 1 -- 172.21.15.92:0/208756840 learned_addr learned my addr 172.21.15.92:0/208756840 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:14.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.192+0000 7fbdcd12e700 1 --2- 172.21.15.92:0/208756840 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e01e073950 0x55e01e099320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.192+0000 7fbdcd92f700 1 --2- 172.21.15.92:0/208756840 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e01e0bdbc0 0x55e01e06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.192+0000 7fbdcc92d700 1 -- 172.21.15.92:0/208756840 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e01e0bdbc0 msgr2=0x55e01e06a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.192+0000 7fbdcc92d700 1 --2- 172.21.15.92:0/208756840 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e01e0bdbc0 0x55e01e06a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.192+0000 7fbdcc92d700 1 -- 172.21.15.92:0/208756840 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e01e073950 msgr2=0x55e01e099320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.192+0000 7fbdcc92d700 1 --2- 172.21.15.92:0/208756840 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e01e073950 0x55e01e099320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.192+0000 7fbdcc92d700 1 -- 172.21.15.92:0/208756840 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc815b7a0 con 0x55e01e09ce60 2023-10-06T10:43:14.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.192+0000 7fbdcc92d700 1 --2- 172.21.15.92:0/208756840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e09ce60 0x55e01e0b35f0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800f6b0 tx=0x7fbdb806e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51beb597380e1d18 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.192+0000 7fbda77fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:14.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.192+0000 7fbda77fe700 1 -- 172.21.15.92:0/208756840 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8012e60 con 0x55e01e09ce60 2023-10-06T10:43:14.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.192+0000 7fbdd868ab80 1 -- 172.21.15.92:0/208756840 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e01e068690 con 0x55e01e09ce60 2023-10-06T10:43:14.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.192+0000 7fbda77fe700 1 -- 172.21.15.92:0/208756840 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fbdb8071070 con 0x55e01e09ce60 2023-10-06T10:43:14.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.192+0000 7fbda77fe700 1 -- 172.21.15.92:0/208756840 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdb8055e40 con 0x55e01e09ce60 2023-10-06T10:43:14.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.192+0000 7fbdd868ab80 10 client.7371 fetch_fsmap learned FSMap version 72 2023-10-06T10:43:14.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.192+0000 7fbdd868ab80 1 -- 172.21.15.92:0/208756840 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55e01de00fe0 con 0x55e01e09ce60 2023-10-06T10:43:14.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.192+0000 7fbda77fe700 1 -- 172.21.15.92:0/208756840 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e01e068690 con 0x55e01e09ce60 2023-10-06T10:43:14.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.193+0000 7fbda77fe700 1 -- 172.21.15.92:0/208756840 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 72) v1 ==== 3124+0+0 (secure 0 0 0) 0x7fbdb8054ec0 con 0x55e01e09ce60 2023-10-06T10:43:14.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.193+0000 7fbdd868ab80 10 client.7371 fetch_fsmap finished waiting for FSMap version 72 2023-10-06T10:43:14.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.193+0000 7fbdd868ab80 10 client.7371 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:43:14.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.193+0000 7fbdd868ab80 20 client.7371 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:14.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.193+0000 7fbdd868ab80 1 --2- 172.21.15.92:0/208756840 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55e01e09b320 0x55e01e09b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.193+0000 7fbdd868ab80 4 client.7371 mds_command: new command op to 7149 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:43:14.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.193+0000 7fbdd868ab80 1 -- 172.21.15.92:0/208756840 --> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e01e0455c0 con 0x55e01e09b320 2023-10-06T10:43:14.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.193+0000 7fbdcd12e700 1 --2- 172.21.15.92:0/208756840 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55e01e09b320 0x55e01e09b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.193+0000 7fbda77fe700 10 client.7371 ms_handle_connect on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:14.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.193+0000 7fbdcd12e700 1 --2- 172.21.15.92:0/208756840 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55e01e09b320 0x55e01e09b6f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e505ce9c2c8b584 server_cookie=58294cde9e47334 in_seq=0 out_seq=0 2023-10-06T10:43:14.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.193+0000 7fbda77fe700 1 -- 172.21.15.92:0/208756840 <== mds.1 v2:172.21.15.92:6834/3268908749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55e01e0455c0 con 0x55e01e09b320 2023-10-06T10:43:14.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.194+0000 7fbda77fe700 10 client.7371 handle_command_reply: tid=0 2023-10-06T10:43:14.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.194+0000 7fbdd868ab80 1 client.7371 shutdown 2023-10-06T10:43:14.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.194+0000 7fbdd868ab80 1 -- 172.21.15.92:0/208756840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e09ce60 msgr2=0x55e01e0b35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.194+0000 7fbdd868ab80 1 --2- 172.21.15.92:0/208756840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e09ce60 0x55e01e0b35f0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800f6b0 tx=0x7fbdb806e030 comp rx=0 tx=0).stop 2023-10-06T10:43:14.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.194+0000 7fbdd868ab80 1 -- 172.21.15.92:0/208756840 shutdown_connections 2023-10-06T10:43:14.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.194+0000 7fbdd868ab80 1 --2- 172.21.15.92:0/208756840 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55e01e09b320 0x55e01e09b6f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.194+0000 7fbdd868ab80 1 --2- 172.21.15.92:0/208756840 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e01e0bdbc0 0x55e01e06a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.194+0000 7fbdd868ab80 1 --2- 172.21.15.92:0/208756840 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e01e073950 0x55e01e099320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.194+0000 7fbdd868ab80 1 --2- 172.21.15.92:0/208756840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e09ce60 0x55e01e0b35f0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.194+0000 7fbda77fe700 0 client.7371 ms_handle_reset on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:14.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.194+0000 7fbdd868ab80 1 -- 172.21.15.92:0/208756840 >> 172.21.15.92:0/208756840 conn(0x55e01e075270 msgr2=0x55e01e0b94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:14.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.194+0000 7fbdd868ab80 1 -- 172.21.15.92:0/208756840 shutdown_connections 2023-10-06T10:43:14.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.194+0000 7fbdd868ab80 1 -- 172.21.15.92:0/208756840 wait complete. 2023-10-06T10:43:14.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.194+0000 7fbdd868ab80 20 client.7371 trim_cache size 0 max 16384 2023-10-06T10:43:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.210+0000 7fbdd868ab80 1 Processor -- start 2023-10-06T10:43:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.210+0000 7fbdd868ab80 1 -- start start 2023-10-06T10:43:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.211+0000 7fbdd868ab80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e01e073950 0x55e01e071710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.211+0000 7fbdd868ab80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e01e09ce60 0x55e01e09b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.211+0000 7fbdd868ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e0bdbc0 0x55e01e09bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.211+0000 7fbdd868ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01de46e30 con 0x55e01e0bdbc0 2023-10-06T10:43:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.211+0000 7fbdd868ab80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01e06f8c0 con 0x55e01e073950 2023-10-06T10:43:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.211+0000 7fbdd868ab80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e01e0989e0 con 0x55e01e09ce60 2023-10-06T10:43:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.211+0000 7fbdcd92f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e0bdbc0 0x55e01e09bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.211+0000 7fbdcd92f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e0bdbc0 0x55e01e09bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48370/0 (socket says 172.21.15.92:48370) 2023-10-06T10:43:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.211+0000 7fbdcd92f700 1 -- 172.21.15.92:0/3420966543 learned_addr learned my addr 172.21.15.92:0/3420966543 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.211+0000 7fbdcd92f700 1 -- 172.21.15.92:0/3420966543 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e01e09ce60 msgr2=0x55e01e09b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.211+0000 7fbdcd92f700 1 --2- 172.21.15.92:0/3420966543 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e01e09ce60 0x55e01e09b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.211+0000 7fbdcd92f700 1 -- 172.21.15.92:0/3420966543 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e01e073950 msgr2=0x55e01e071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.211+0000 7fbdcd92f700 1 --2- 172.21.15.92:0/3420966543 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e01e073950 0x55e01e071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.211+0000 7fbdcd92f700 1 -- 172.21.15.92:0/3420966543 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e01df92780 con 0x55e01e0bdbc0 2023-10-06T10:43:14.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.212+0000 7fbdcd92f700 1 --2- 172.21.15.92:0/3420966543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e0bdbc0 0x55e01e09bc30 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0007830 tx=0x7fbdc000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b0dbe0f9a771d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.212+0000 7fbdb2ffd700 1 -- 172.21.15.92:0/3420966543 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc000e040 con 0x55e01e0bdbc0 2023-10-06T10:43:14.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.212+0000 7fbdb2ffd700 1 -- 172.21.15.92:0/3420966543 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc000f040 con 0x55e01e0bdbc0 2023-10-06T10:43:14.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.212+0000 7fbdb2ffd700 1 -- 172.21.15.92:0/3420966543 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0017830 con 0x55e01e0bdbc0 2023-10-06T10:43:14.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.212+0000 7fbdd868ab80 1 -- 172.21.15.92:0/3420966543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e0bdbc0 msgr2=0x55e01e09bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.212+0000 7fbdd868ab80 1 --2- 172.21.15.92:0/3420966543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e0bdbc0 0x55e01e09bc30 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0007830 tx=0x7fbdc000b040 comp rx=0 tx=0).stop 2023-10-06T10:43:14.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.212+0000 7fbdd868ab80 1 -- 172.21.15.92:0/3420966543 shutdown_connections 2023-10-06T10:43:14.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.212+0000 7fbdd868ab80 1 --2- 172.21.15.92:0/3420966543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e0bdbc0 0x55e01e09bc30 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.212+0000 7fbdd868ab80 1 --2- 172.21.15.92:0/3420966543 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e01e073950 0x55e01e071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.212+0000 7fbdd868ab80 1 --2- 172.21.15.92:0/3420966543 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e01e09ce60 0x55e01e09b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.212+0000 7fbdd868ab80 1 -- 172.21.15.92:0/3420966543 >> 172.21.15.92:0/3420966543 conn(0x55e01e075270 msgr2=0x55e01e0a3830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:14.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.212+0000 7fbdd868ab80 1 -- 172.21.15.92:0/3420966543 shutdown_connections 2023-10-06T10:43:14.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.212+0000 7fbdd868ab80 1 -- 172.21.15.92:0/3420966543 wait complete. 2023-10-06T10:43:14.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.213+0000 7fbdd868ab80 1 -- start start 2023-10-06T10:43:14.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.213+0000 7fbdd868ab80 1 Processor -- start 2023-10-06T10:43:14.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.214+0000 7fbdd868ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e073950 0x55e01e11f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.214+0000 7fbdd868ab80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e01e09ce60 0x55e01e11f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.214+0000 7fbdd868ab80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e01e0bdbc0 0x55e01e074050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.214+0000 7fbdd868ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01e0b5d80 con 0x55e01e073950 2023-10-06T10:43:14.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.214+0000 7fbdd868ab80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01e0d4020 con 0x55e01e0bdbc0 2023-10-06T10:43:14.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.214+0000 7fbdd868ab80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e01e0b5f70 con 0x55e01e09ce60 2023-10-06T10:43:14.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.214+0000 7fbdd868ab80 10 client.0 _ll_register_callbacks cb 0x55e01de60f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:14.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.214+0000 7fbdcd12e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e073950 0x55e01e11f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.214+0000 7fbdcd12e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e073950 0x55e01e11f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48374/0 (socket says 172.21.15.92:48374) 2023-10-06T10:43:14.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.214+0000 7fbdcd12e700 1 -- 172.21.15.92:0/1305378344 learned_addr learned my addr 172.21.15.92:0/1305378344 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:14.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.214+0000 7fbdcd12e700 1 -- 172.21.15.92:0/1305378344 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e01e09ce60 msgr2=0x55e01e11f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.214+0000 7fbdcd12e700 1 --2- 172.21.15.92:0/1305378344 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e01e09ce60 0x55e01e11f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.214+0000 7fbdcd12e700 1 -- 172.21.15.92:0/1305378344 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e01e0bdbc0 msgr2=0x55e01e074050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.214+0000 7fbdcd12e700 1 --2- 172.21.15.92:0/1305378344 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e01e0bdbc0 0x55e01e074050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.214+0000 7fbdcd12e700 1 -- 172.21.15.92:0/1305378344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc0007560 con 0x55e01e073950 2023-10-06T10:43:14.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.214+0000 7fbdcd12e700 1 --2- 172.21.15.92:0/1305378344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e073950 0x55e01e11f2f0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4006fd0 tx=0x7fbdc402c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259e2d7750fdb348 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.214+0000 7fbda77fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:14.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.214+0000 7fbda77fe700 1 -- 172.21.15.92:0/1305378344 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc402f070 con 0x55e01e073950 2023-10-06T10:43:14.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.214+0000 7fbda77fe700 1 -- 172.21.15.92:0/1305378344 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fbdc401ace0 con 0x55e01e073950 2023-10-06T10:43:14.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.214+0000 7fbdd868ab80 1 -- 172.21.15.92:0/1305378344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbdb8070020 con 0x55e01e073950 2023-10-06T10:43:14.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.215+0000 7fbda77fe700 1 -- 172.21.15.92:0/1305378344 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdc4021070 con 0x55e01e073950 2023-10-06T10:43:14.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.215+0000 7fbdd868ab80 10 client.7377 fetch_fsmap learned FSMap version 72 2023-10-06T10:43:14.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.215+0000 7fbdd868ab80 1 -- 172.21.15.92:0/1305378344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55e01df79470 con 0x55e01e073950 2023-10-06T10:43:14.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.215+0000 7fbda77fe700 1 -- 172.21.15.92:0/1305378344 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb8070020 con 0x55e01e073950 2023-10-06T10:43:14.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.215+0000 7fbda77fe700 1 -- 172.21.15.92:0/1305378344 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 72) v1 ==== 3124+0+0 (secure 0 0 0) 0x7fbdc40366b0 con 0x55e01e073950 2023-10-06T10:43:14.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.215+0000 7fbdd868ab80 10 client.7377 fetch_fsmap finished waiting for FSMap version 72 2023-10-06T10:43:14.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.215+0000 7fbdd868ab80 10 client.7377 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:43:14.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.215+0000 7fbdd868ab80 20 client.7377 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:14.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.215+0000 7fbdd868ab80 1 --2- 172.21.15.92:0/1305378344 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55e01e097d30 0x55e01e098100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.215+0000 7fbdd868ab80 4 client.7377 mds_command: new command op to 7149 tid=0[{"prefix": "status"}] 2023-10-06T10:43:14.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.215+0000 7fbdd868ab80 1 -- 172.21.15.92:0/1305378344 --> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e01df66e00 con 0x55e01e097d30 2023-10-06T10:43:14.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.215+0000 7fbdcc92d700 1 --2- 172.21.15.92:0/1305378344 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55e01e097d30 0x55e01e098100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.215+0000 7fbda77fe700 10 client.7377 ms_handle_connect on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:14.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.215+0000 7fbdcc92d700 1 --2- 172.21.15.92:0/1305378344 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55e01e097d30 0x55e01e098100 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ace0febb1b7c1f6 server_cookie=b166259a8cc8765 in_seq=0 out_seq=0 2023-10-06T10:43:14.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.215+0000 7fbda77fe700 1 -- 172.21.15.92:0/1305378344 <== mds.1 v2:172.21.15.92:6834/3268908749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55e01df66e00 con 0x55e01e097d30 2023-10-06T10:43:14.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.215+0000 7fbda77fe700 10 client.7377 handle_command_reply: tid=0 2023-10-06T10:43:14.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.215+0000 7fbdd868ab80 1 client.7377 shutdown 2023-10-06T10:43:14.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdd868ab80 1 -- 172.21.15.92:0/1305378344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e073950 msgr2=0x55e01e11f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdd868ab80 1 --2- 172.21.15.92:0/1305378344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e073950 0x55e01e11f2f0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4006fd0 tx=0x7fbdc402c040 comp rx=0 tx=0).stop 2023-10-06T10:43:14.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdd868ab80 1 -- 172.21.15.92:0/1305378344 shutdown_connections 2023-10-06T10:43:14.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdd868ab80 1 --2- 172.21.15.92:0/1305378344 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55e01e097d30 0x55e01e098100 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdd868ab80 1 --2- 172.21.15.92:0/1305378344 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e01e0bdbc0 0x55e01e074050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbda77fe700 0 client.7377 ms_handle_reset on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:14.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdd868ab80 1 --2- 172.21.15.92:0/1305378344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e01e073950 0x55e01e11f2f0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdd868ab80 1 --2- 172.21.15.92:0/1305378344 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e01e09ce60 0x55e01e11f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdd868ab80 1 -- 172.21.15.92:0/1305378344 >> 172.21.15.92:0/1305378344 conn(0x55e01e075270 msgr2=0x55e01e071710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:14.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdd868ab80 1 -- 172.21.15.92:0/1305378344 shutdown_connections 2023-10-06T10:43:14.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdd868ab80 1 -- 172.21.15.92:0/1305378344 wait complete. 2023-10-06T10:43:14.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdd868ab80 20 client.7377 trim_cache size 0 max 16384 2023-10-06T10:43:14.224 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:43:14.224 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:43:14.224 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 1, 2023-10-06T10:43:14.224 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 7149, 2023-10-06T10:43:14.224 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:43:14.224 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:43:14.224 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:43:14.224 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 2.8706574630000001, 2023-10-06T10:43:14.224 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 72, 2023-10-06T10:43:14.224 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 78, 2023-10-06T10:43:14.224 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 78, 2023-10-06T10:43:14.225 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 16.532448178999999 2023-10-06T10:43:14.225 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:43:14.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdce130700 1 -- 172.21.15.92:0/2048770816 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbdac066f30 msgr2=0x7fbdac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdce130700 1 --2- 172.21.15.92:0/2048770816 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbdac066f30 0x7fbdac0693e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80d0220 tx=0x7fbdc4017000 comp rx=0 tx=0).stop 2023-10-06T10:43:14.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdce130700 1 -- 172.21.15.92:0/2048770816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc80cc170 msgr2=0x7fbdc8154730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdce130700 1 --2- 172.21.15.92:0/2048770816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc80cc170 0x7fbdc8154730 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800d890 tx=0x7fbdb8008040 comp rx=0 tx=0).stop 2023-10-06T10:43:14.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdce130700 1 -- 172.21.15.92:0/2048770816 shutdown_connections 2023-10-06T10:43:14.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdce130700 1 --2- 172.21.15.92:0/2048770816 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbdac066f30 0x7fbdac0693e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdce130700 1 --2- 172.21.15.92:0/2048770816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdc8159440 0x7fbdc816ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdce130700 1 --2- 172.21.15.92:0/2048770816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdc80b35d0 0x7fbdc80c4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdce130700 1 --2- 172.21.15.92:0/2048770816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc80cc170 0x7fbdc8154730 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdce130700 1 -- 172.21.15.92:0/2048770816 >> 172.21.15.92:0/2048770816 conn(0x7fbdc806ece0 msgr2=0x7fbdc8156900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:14.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdce130700 1 -- 172.21.15.92:0/2048770816 shutdown_connections 2023-10-06T10:43:14.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.216+0000 7fbdce130700 1 -- 172.21.15.92:0/2048770816 wait complete. 2023-10-06T10:43:14.229 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T10:43:14.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.343+0000 7fbeebfff700 1 Processor -- start 2023-10-06T10:43:14.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.343+0000 7fbeebfff700 1 -- start start 2023-10-06T10:43:14.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.344+0000 7fbeebfff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbeec159440 0x7fbeec0b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.344+0000 7fbeebfff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeec0cb7a0 con 0x7fbeec0b35d0 2023-10-06T10:43:14.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.344+0000 7fbeebfff700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeec0b0860 con 0x7fbeec0b2100 2023-10-06T10:43:14.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.344+0000 7fbeebfff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeec0cb150 con 0x7fbeec159440 2023-10-06T10:43:14.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.344+0000 7fbee27fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbeec0b35d0 0x7fbeec0cbc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44916/0 (socket says 172.21.15.92:44916) 2023-10-06T10:43:14.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.344+0000 7fbee27fc700 1 -- 172.21.15.92:0/1795913927 learned_addr learned my addr 172.21.15.92:0/1795913927 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:14.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.344+0000 7fbeeaffd700 1 --2- 172.21.15.92:0/1795913927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbeec159440 0x7fbeec0b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.344+0000 7fbeea7fc700 1 -- 172.21.15.92:0/1795913927 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 478397659 0 0) 0x7fbeec0cb7a0 con 0x7fbeec0b35d0 2023-10-06T10:43:14.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.344+0000 7fbeea7fc700 1 -- 172.21.15.92:0/1795913927 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbed8003570 con 0x7fbeec0b35d0 2023-10-06T10:43:14.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.345+0000 7fbeea7fc700 1 -- 172.21.15.92:0/1795913927 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 246953634 0 0) 0x7fbed8003570 con 0x7fbeec0b35d0 2023-10-06T10:43:14.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.345+0000 7fbeeaffd700 1 -- 172.21.15.92:0/1795913927 >> v1:172.21.15.124:6789/0 conn(0x7fbeec0b2100 legacy=0x7fbeec0b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.345+0000 7fbeeaffd700 1 -- 172.21.15.92:0/1795913927 >> v1:172.21.15.92:6789/0 conn(0x7fbeec0b35d0 legacy=0x7fbeec0cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.345+0000 7fbeeaffd700 1 -- 172.21.15.92:0/1795913927 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeec0cb430 con 0x7fbeec159440 2023-10-06T10:43:14.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.345+0000 7fbeeaffd700 1 --2- 172.21.15.92:0/1795913927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbeec159440 0x7fbeec0b3090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009910 tx=0x7fbedc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a279037ae3c629c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.345+0000 7fbeea7fc700 1 -- 172.21.15.92:0/1795913927 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed4001470 con 0x7fbeec159440 2023-10-06T10:43:14.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.345+0000 7fbeea7fc700 1 -- 172.21.15.92:0/1795913927 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbedc011420 con 0x7fbeec159440 2023-10-06T10:43:14.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.345+0000 7fbeea7fc700 1 -- 172.21.15.92:0/1795913927 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc0117c0 con 0x7fbeec159440 2023-10-06T10:43:14.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.345+0000 7fbeebfff700 1 -- 172.21.15.92:0/1795913927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbeec159440 msgr2=0x7fbeec0b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.345+0000 7fbeebfff700 1 --2- 172.21.15.92:0/1795913927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbeec159440 0x7fbeec0b3090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009910 tx=0x7fbedc0091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.346+0000 7fbeebfff700 1 -- 172.21.15.92:0/1795913927 shutdown_connections 2023-10-06T10:43:14.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.346+0000 7fbeebfff700 1 --2- 172.21.15.92:0/1795913927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbeec159440 0x7fbeec0b3090 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.346+0000 7fbeebfff700 1 -- 172.21.15.92:0/1795913927 >> 172.21.15.92:0/1795913927 conn(0x7fbeec06ece0 msgr2=0x7fbeec06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:14.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.346+0000 7fbeebfff700 1 -- 172.21.15.92:0/1795913927 shutdown_connections 2023-10-06T10:43:14.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.346+0000 7fbeebfff700 1 -- 172.21.15.92:0/1795913927 wait complete. 2023-10-06T10:43:14.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.347+0000 7fbeebfff700 1 Processor -- start 2023-10-06T10:43:14.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.347+0000 7fbeebfff700 1 -- start start 2023-10-06T10:43:14.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbeebfff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbeec0b2100 0x7fbeec153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbeebfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbeec0b35d0 0x7fbeec0c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbeebfff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbeec159440 0x7fbeec0d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbeebfff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeec169940 con 0x7fbeec0b35d0 2023-10-06T10:43:14.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbeebfff700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeec0ceb00 con 0x7fbeec0b2100 2023-10-06T10:43:14.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbeebfff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeec0cede0 con 0x7fbeec159440 2023-10-06T10:43:14.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbee27fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbeec0b35d0 0x7fbeec0c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbee27fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbeec0b35d0 0x7fbeec0c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48378/0 (socket says 172.21.15.92:48378) 2023-10-06T10:43:14.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbee27fc700 1 -- 172.21.15.92:0/1861670493 learned_addr learned my addr 172.21.15.92:0/1861670493 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:14.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbeeb7fe700 1 --2- 172.21.15.92:0/1861670493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbeec159440 0x7fbeec0d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbeeaffd700 1 --2- 172.21.15.92:0/1861670493 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbeec0b2100 0x7fbeec153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbee27fc700 1 -- 172.21.15.92:0/1861670493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbeec159440 msgr2=0x7fbeec0d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbee27fc700 1 --2- 172.21.15.92:0/1861670493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbeec159440 0x7fbeec0d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbee27fc700 1 -- 172.21.15.92:0/1861670493 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbeec0b2100 msgr2=0x7fbeec153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbee27fc700 1 --2- 172.21.15.92:0/1861670493 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbeec0b2100 0x7fbeec153ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbee27fc700 1 -- 172.21.15.92:0/1861670493 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbedc009420 con 0x7fbeec0b35d0 2023-10-06T10:43:14.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbeeaffd700 1 --2- 172.21.15.92:0/1861670493 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbeec0b2100 0x7fbeec153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:14.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbee27fc700 1 --2- 172.21.15.92:0/1861670493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbeec0b35d0 0x7fbeec0c8400 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fbed400cb60 tx=0x7fbed4001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbee37fe700 1 -- 172.21.15.92:0/1861670493 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed4001d60 con 0x7fbeec0b35d0 2023-10-06T10:43:14.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbee37fe700 1 -- 172.21.15.92:0/1861670493 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed4007360 con 0x7fbeec0b35d0 2023-10-06T10:43:14.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbeebfff700 1 -- 172.21.15.92:0/1861670493 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeec0cf0c0 con 0x7fbeec0b35d0 2023-10-06T10:43:14.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbee37fe700 1 -- 172.21.15.92:0/1861670493 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed4004830 con 0x7fbeec0b35d0 2023-10-06T10:43:14.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.348+0000 7fbeebfff700 1 -- 172.21.15.92:0/1861670493 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeec154000 con 0x7fbeec0b35d0 2023-10-06T10:43:14.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.350+0000 7fbee37fe700 1 -- 172.21.15.92:0/1861670493 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbed4004ba0 con 0x7fbeec0b35d0 2023-10-06T10:43:14.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.350+0000 7fbee37fe700 1 --2- 172.21.15.92:0/1861670493 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbecc066f30 0x7fbecc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.350+0000 7fbee37fe700 1 -- 172.21.15.92:0/1861670493 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fbed40840e0 con 0x7fbeec0b35d0 2023-10-06T10:43:14.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.350+0000 7fbeeaffd700 1 --2- 172.21.15.92:0/1861670493 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbecc066f30 0x7fbecc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.351+0000 7fbeeaffd700 1 --2- 172.21.15.92:0/1861670493 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbecc066f30 0x7fbecc0693e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009860 tx=0x7fbedc017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.352+0000 7fbefabcfb80 1 Processor -- start 2023-10-06T10:43:14.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.352+0000 7fbefabcfb80 1 -- start start 2023-10-06T10:43:14.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.353+0000 7fbefabcfb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55705dfa4950 0x55705dfa4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.353+0000 7fbefabcfb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55705dfeebc0 0x55705dfcd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.353+0000 7fbefabcfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfcde60 0x55705dfa0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.353+0000 7fbefabcfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55705df24c80 con 0x55705dfcde60 2023-10-06T10:43:14.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.353+0000 7fbefabcfb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55705defba70 con 0x55705dfa4950 2023-10-06T10:43:14.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.353+0000 7fbefabcfb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55705def9270 con 0x55705dfeebc0 2023-10-06T10:43:14.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.353+0000 7fbeeb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfcde60 0x55705dfa0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.353+0000 7fbeeb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfcde60 0x55705dfa0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48394/0 (socket says 172.21.15.92:48394) 2023-10-06T10:43:14.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.353+0000 7fbeeb7fe700 1 -- 172.21.15.92:0/2306508472 learned_addr learned my addr 172.21.15.92:0/2306508472 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:14.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.353+0000 7fbeeb7fe700 1 -- 172.21.15.92:0/2306508472 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55705dfeebc0 msgr2=0x55705dfcd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.353+0000 7fbeeb7fe700 1 --2- 172.21.15.92:0/2306508472 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55705dfeebc0 0x55705dfcd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.353+0000 7fbeeb7fe700 1 -- 172.21.15.92:0/2306508472 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55705dfa4950 msgr2=0x55705dfa4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.353+0000 7fbeeaffd700 1 --2- 172.21.15.92:0/2306508472 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55705dfa4950 0x55705dfa4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.353+0000 7fbeeb7fe700 1 --2- 172.21.15.92:0/2306508472 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55705dfa4950 0x55705dfa4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.353+0000 7fbeeb7fe700 1 -- 172.21.15.92:0/2306508472 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55705dd8f270 con 0x55705dfcde60 2023-10-06T10:43:14.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.353+0000 7fbeeb7fe700 1 --2- 172.21.15.92:0/2306508472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfcde60 0x55705dfa0990 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fbee4001ea0 tx=0x7fbee4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43cb6e1e32287acf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.353+0000 7fbecbfff700 1 -- 172.21.15.92:0/2306508472 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee400ea40 con 0x55705dfcde60 2023-10-06T10:43:14.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.354+0000 7fbecbfff700 1 -- 172.21.15.92:0/2306508472 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee400eba0 con 0x55705dfcde60 2023-10-06T10:43:14.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.354+0000 7fbecbfff700 1 -- 172.21.15.92:0/2306508472 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee4013420 con 0x55705dfcde60 2023-10-06T10:43:14.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.354+0000 7fbefabcfb80 1 -- 172.21.15.92:0/2306508472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfcde60 msgr2=0x55705dfa0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.354+0000 7fbefabcfb80 1 --2- 172.21.15.92:0/2306508472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfcde60 0x55705dfa0990 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fbee4001ea0 tx=0x7fbee4009040 comp rx=0 tx=0).stop 2023-10-06T10:43:14.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.354+0000 7fbefabcfb80 1 -- 172.21.15.92:0/2306508472 shutdown_connections 2023-10-06T10:43:14.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.354+0000 7fbefabcfb80 1 --2- 172.21.15.92:0/2306508472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfcde60 0x55705dfa0990 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.354+0000 7fbefabcfb80 1 --2- 172.21.15.92:0/2306508472 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55705dfa4950 0x55705dfa4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.354+0000 7fbefabcfb80 1 --2- 172.21.15.92:0/2306508472 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55705dfeebc0 0x55705dfcd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.354+0000 7fbefabcfb80 1 -- 172.21.15.92:0/2306508472 >> 172.21.15.92:0/2306508472 conn(0x55705dfa6270 msgr2=0x55705dfa6640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:14.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.354+0000 7fbefabcfb80 1 -- 172.21.15.92:0/2306508472 shutdown_connections 2023-10-06T10:43:14.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.354+0000 7fbefabcfb80 1 -- 172.21.15.92:0/2306508472 wait complete. 2023-10-06T10:43:14.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.355+0000 7fbefabcfb80 1 -- start start 2023-10-06T10:43:14.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.355+0000 7fbefabcfb80 1 Processor -- start 2023-10-06T10:43:14.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.356+0000 7fbefabcfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfa4950 0x55705dfcc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.356+0000 7fbefabcfb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55705dfcde60 0x55705dfe6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.356+0000 7fbefabcfb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55705dfeebc0 0x55705dfe87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.356+0000 7fbefabcfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55705def5060 con 0x55705dfa4950 2023-10-06T10:43:14.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.356+0000 7fbefabcfb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55705dea0890 con 0x55705dfcde60 2023-10-06T10:43:14.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.356+0000 7fbefabcfb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55705debbea0 con 0x55705dfeebc0 2023-10-06T10:43:14.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.356+0000 7fbefabcfb80 10 client.0 _ll_register_callbacks cb 0x55705dd91f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:14.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.356+0000 7fbeeaffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfa4950 0x55705dfcc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.356+0000 7fbeeaffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfa4950 0x55705dfcc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48406/0 (socket says 172.21.15.92:48406) 2023-10-06T10:43:14.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.356+0000 7fbeeaffd700 1 -- 172.21.15.92:0/1234976090 learned_addr learned my addr 172.21.15.92:0/1234976090 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.356+0000 7fbee27fc700 1 --2- 172.21.15.92:0/1234976090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55705dfcde60 0x55705dfe6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.356+0000 7fbeeb7fe700 1 --2- 172.21.15.92:0/1234976090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55705dfeebc0 0x55705dfe87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.357+0000 7fbeeaffd700 1 -- 172.21.15.92:0/1234976090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55705dfeebc0 msgr2=0x55705dfe87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.357+0000 7fbeeaffd700 1 --2- 172.21.15.92:0/1234976090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55705dfeebc0 0x55705dfe87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.357+0000 7fbeeaffd700 1 -- 172.21.15.92:0/1234976090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55705dfcde60 msgr2=0x55705dfe6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.357+0000 7fbeeaffd700 1 --2- 172.21.15.92:0/1234976090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55705dfcde60 0x55705dfe6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.357+0000 7fbeeaffd700 1 -- 172.21.15.92:0/1234976090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee400f040 con 0x55705dfa4950 2023-10-06T10:43:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.357+0000 7fbeeaffd700 1 --2- 172.21.15.92:0/1234976090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfa4950 0x55705dfcc4a0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fbeec069150 tx=0x7fbedc01f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3418e7b3cdfdbcc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.357+0000 7fbec97fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.357+0000 7fbec97fa700 1 -- 172.21.15.92:0/1234976090 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc01d890 con 0x55705dfa4950 2023-10-06T10:43:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.357+0000 7fbefabcfb80 1 -- 172.21.15.92:0/1234976090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55705df99690 con 0x55705dfa4950 2023-10-06T10:43:14.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.357+0000 7fbec97fa700 1 -- 172.21.15.92:0/1234976090 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fbedc023070 con 0x55705dfa4950 2023-10-06T10:43:14.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.357+0000 7fbec97fa700 1 -- 172.21.15.92:0/1234976090 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbedc033d90 con 0x55705dfa4950 2023-10-06T10:43:14.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.357+0000 7fbefabcfb80 10 client.7389 fetch_fsmap learned FSMap version 72 2023-10-06T10:43:14.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.357+0000 7fbefabcfb80 1 -- 172.21.15.92:0/1234976090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55705dd31fe0 con 0x55705dfa4950 2023-10-06T10:43:14.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.358+0000 7fbec97fa700 1 -- 172.21.15.92:0/1234976090 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55705df99690 con 0x55705dfa4950 2023-10-06T10:43:14.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.358+0000 7fbec97fa700 1 -- 172.21.15.92:0/1234976090 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 72) v1 ==== 3124+0+0 (secure 0 0 0) 0x7fbedc01dc00 con 0x55705dfa4950 2023-10-06T10:43:14.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.358+0000 7fbefabcfb80 10 client.7389 fetch_fsmap finished waiting for FSMap version 72 2023-10-06T10:43:14.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.358+0000 7fbefabcfb80 10 client.7389 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T10:43:14.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.358+0000 7fbefabcfb80 20 client.7389 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:14.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.358+0000 7fbefabcfb80 1 --2- 172.21.15.92:0/1234976090 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x55705dfe4a20 0x55705dfe4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.358+0000 7fbefabcfb80 4 client.7389 mds_command: new command op to 7164 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:43:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.358+0000 7fbefabcfb80 1 -- 172.21.15.92:0/1234976090 --> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55705df765c0 con 0x55705dfe4a20 2023-10-06T10:43:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.358+0000 7fbee27fc700 1 --2- 172.21.15.92:0/1234976090 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x55705dfe4a20 0x55705dfe4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.358+0000 7fbec97fa700 10 client.7389 ms_handle_connect on v2:172.21.15.124:6833/2971348516 2023-10-06T10:43:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.358+0000 7fbee27fc700 1 --2- 172.21.15.92:0/1234976090 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x55705dfe4a20 0x55705dfe4df0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d29bdba8a29100a server_cookie=571f243c9e8ccef3 in_seq=0 out_seq=0 2023-10-06T10:43:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.359+0000 7fbec97fa700 1 -- 172.21.15.92:0/1234976090 <== mds.0 v2:172.21.15.124:6833/2971348516 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55705df765c0 con 0x55705dfe4a20 2023-10-06T10:43:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.359+0000 7fbec97fa700 10 client.7389 handle_command_reply: tid=0 2023-10-06T10:43:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.359+0000 7fbefabcfb80 1 client.7389 shutdown 2023-10-06T10:43:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.359+0000 7fbefabcfb80 1 -- 172.21.15.92:0/1234976090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfa4950 msgr2=0x55705dfcc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.359+0000 7fbefabcfb80 1 --2- 172.21.15.92:0/1234976090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfa4950 0x55705dfcc4a0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fbeec069150 tx=0x7fbedc01f070 comp rx=0 tx=0).stop 2023-10-06T10:43:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.359+0000 7fbefabcfb80 1 -- 172.21.15.92:0/1234976090 shutdown_connections 2023-10-06T10:43:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.359+0000 7fbefabcfb80 1 --2- 172.21.15.92:0/1234976090 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x55705dfe4a20 0x55705dfe4df0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.359+0000 7fbefabcfb80 1 --2- 172.21.15.92:0/1234976090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55705dfeebc0 0x55705dfe87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.359+0000 7fbefabcfb80 1 --2- 172.21.15.92:0/1234976090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfa4950 0x55705dfcc4a0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.359+0000 7fbec97fa700 0 client.7389 ms_handle_reset on v2:172.21.15.124:6833/2971348516 2023-10-06T10:43:14.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.359+0000 7fbefabcfb80 1 --2- 172.21.15.92:0/1234976090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55705dfcde60 0x55705dfe6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.359+0000 7fbefabcfb80 1 -- 172.21.15.92:0/1234976090 >> 172.21.15.92:0/1234976090 conn(0x55705dfa6270 msgr2=0x55705df9b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:14.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.359+0000 7fbefabcfb80 1 -- 172.21.15.92:0/1234976090 shutdown_connections 2023-10-06T10:43:14.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.359+0000 7fbefabcfb80 1 -- 172.21.15.92:0/1234976090 wait complete. 2023-10-06T10:43:14.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.359+0000 7fbefabcfb80 20 client.7389 trim_cache size 0 max 16384 2023-10-06T10:43:14.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.376+0000 7fbefabcfb80 1 Processor -- start 2023-10-06T10:43:14.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.376+0000 7fbefabcfb80 1 -- start start 2023-10-06T10:43:14.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.376+0000 7fbefabcfb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55705dfa4950 0x55705dfcc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.376+0000 7fbefabcfb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55705dfcde60 0x55705dfe7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.376+0000 7fbefabcfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfeebc0 0x55705df9b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.377+0000 7fbefabcfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55705dd77e30 con 0x55705dfeebc0 2023-10-06T10:43:14.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.377+0000 7fbefabcfb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55705dfca740 con 0x55705dfcde60 2023-10-06T10:43:14.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.377+0000 7fbefabcfb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55705e00c7d0 con 0x55705dfa4950 2023-10-06T10:43:14.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.377+0000 7fbeeb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfeebc0 0x55705df9b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.377+0000 7fbeeb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfeebc0 0x55705df9b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48418/0 (socket says 172.21.15.92:48418) 2023-10-06T10:43:14.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.377+0000 7fbeeb7fe700 1 -- 172.21.15.92:0/2358742565 learned_addr learned my addr 172.21.15.92:0/2358742565 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:14.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.377+0000 7fbeeaffd700 1 --2- 172.21.15.92:0/2358742565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55705dfa4950 0x55705dfcc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.377+0000 7fbee27fc700 1 --2- 172.21.15.92:0/2358742565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55705dfcde60 0x55705dfe7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.377+0000 7fbeeb7fe700 1 -- 172.21.15.92:0/2358742565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55705dfa4950 msgr2=0x55705dfcc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.377+0000 7fbeeb7fe700 1 --2- 172.21.15.92:0/2358742565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55705dfa4950 0x55705dfcc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.377+0000 7fbeeb7fe700 1 -- 172.21.15.92:0/2358742565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55705dfcde60 msgr2=0x55705dfe7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.377+0000 7fbeeb7fe700 1 --2- 172.21.15.92:0/2358742565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55705dfcde60 0x55705dfe7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.377+0000 7fbeeb7fe700 1 -- 172.21.15.92:0/2358742565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55705dfcbfd0 con 0x55705dfeebc0 2023-10-06T10:43:14.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.377+0000 7fbeeaffd700 1 --2- 172.21.15.92:0/2358742565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55705dfa4950 0x55705dfcc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:14.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.377+0000 7fbee27fc700 1 --2- 172.21.15.92:0/2358742565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55705dfcde60 0x55705dfe7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:14.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.377+0000 7fbeeb7fe700 1 --2- 172.21.15.92:0/2358742565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfeebc0 0x55705df9b1a0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fbee400e4f0 tx=0x7fbee401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de32d94c4008782 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.377+0000 7fbee0ff9700 1 -- 172.21.15.92:0/2358742565 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee4025070 con 0x55705dfeebc0 2023-10-06T10:43:14.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.377+0000 7fbee0ff9700 1 -- 172.21.15.92:0/2358742565 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee4001990 con 0x55705dfeebc0 2023-10-06T10:43:14.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.377+0000 7fbee0ff9700 1 -- 172.21.15.92:0/2358742565 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee400ed10 con 0x55705dfeebc0 2023-10-06T10:43:14.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.378+0000 7fbefabcfb80 1 -- 172.21.15.92:0/2358742565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfeebc0 msgr2=0x55705df9b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.378+0000 7fbefabcfb80 1 --2- 172.21.15.92:0/2358742565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfeebc0 0x55705df9b1a0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fbee400e4f0 tx=0x7fbee401a000 comp rx=0 tx=0).stop 2023-10-06T10:43:14.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.378+0000 7fbefabcfb80 1 -- 172.21.15.92:0/2358742565 shutdown_connections 2023-10-06T10:43:14.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.378+0000 7fbefabcfb80 1 --2- 172.21.15.92:0/2358742565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfeebc0 0x55705df9b1a0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.378+0000 7fbefabcfb80 1 --2- 172.21.15.92:0/2358742565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55705dfa4950 0x55705dfcc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.378+0000 7fbefabcfb80 1 --2- 172.21.15.92:0/2358742565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55705dfcde60 0x55705dfe7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.378+0000 7fbefabcfb80 1 -- 172.21.15.92:0/2358742565 >> 172.21.15.92:0/2358742565 conn(0x55705dfa6270 msgr2=0x55705dfa7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:14.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.378+0000 7fbefabcfb80 1 -- 172.21.15.92:0/2358742565 shutdown_connections 2023-10-06T10:43:14.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.378+0000 7fbefabcfb80 1 -- 172.21.15.92:0/2358742565 wait complete. 2023-10-06T10:43:14.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.378+0000 7fbefabcfb80 1 -- start start 2023-10-06T10:43:14.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.378+0000 7fbefabcfb80 1 Processor -- start 2023-10-06T10:43:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.379+0000 7fbefabcfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfa4950 0x55705e050530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.379+0000 7fbefabcfb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55705dfcde60 0x55705dfcc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.379+0000 7fbefabcfb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55705dfeebc0 0x55705dfcc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.379+0000 7fbefabcfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55705dfa19d0 con 0x55705dfa4950 2023-10-06T10:43:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.379+0000 7fbefabcfb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55705e050a70 con 0x55705dfcde60 2023-10-06T10:43:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.379+0000 7fbefabcfb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55705dfc8790 con 0x55705dfeebc0 2023-10-06T10:43:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.379+0000 7fbefabcfb80 10 client.0 _ll_register_callbacks cb 0x55705dd91f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.379+0000 7fbeeaffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfa4950 0x55705e050530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbeeaffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfa4950 0x55705e050530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48434/0 (socket says 172.21.15.92:48434) 2023-10-06T10:43:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbeeaffd700 1 -- 172.21.15.92:0/4133778396 learned_addr learned my addr 172.21.15.92:0/4133778396 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:14.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbee27fc700 1 --2- 172.21.15.92:0/4133778396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55705dfcde60 0x55705dfcc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbeeb7fe700 1 --2- 172.21.15.92:0/4133778396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55705dfeebc0 0x55705dfcc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbeeaffd700 1 -- 172.21.15.92:0/4133778396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55705dfeebc0 msgr2=0x55705dfcc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbeeaffd700 1 --2- 172.21.15.92:0/4133778396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55705dfeebc0 0x55705dfcc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbeeaffd700 1 -- 172.21.15.92:0/4133778396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55705dfcde60 msgr2=0x55705dfcc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbeeaffd700 1 --2- 172.21.15.92:0/4133778396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55705dfcde60 0x55705dfcc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbeeaffd700 1 -- 172.21.15.92:0/4133778396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55705dd8f270 con 0x55705dfa4950 2023-10-06T10:43:14.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbeeaffd700 1 --2- 172.21.15.92:0/4133778396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfa4950 0x55705e050530 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fbedc01deb0 tx=0x7fbedc02b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b556440c8fc4d84f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbec97fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:14.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbec97fa700 1 -- 172.21.15.92:0/4133778396 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc011a50 con 0x55705dfa4950 2023-10-06T10:43:14.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbec97fa700 1 -- 172.21.15.92:0/4133778396 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fbedc023040 con 0x55705dfa4950 2023-10-06T10:43:14.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbefabcfb80 1 -- 172.21.15.92:0/4133778396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbee400eba0 con 0x55705dfa4950 2023-10-06T10:43:14.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbec97fa700 1 -- 172.21.15.92:0/4133778396 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbedc011bb0 con 0x55705dfa4950 2023-10-06T10:43:14.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbefabcfb80 10 client.7395 fetch_fsmap learned FSMap version 72 2023-10-06T10:43:14.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbefabcfb80 1 -- 172.21.15.92:0/4133778396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55705dec3780 con 0x55705dfa4950 2023-10-06T10:43:14.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbec97fa700 1 -- 172.21.15.92:0/4133778396 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee400eba0 con 0x55705dfa4950 2023-10-06T10:43:14.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbec97fa700 1 -- 172.21.15.92:0/4133778396 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 72) v1 ==== 3124+0+0 (secure 0 0 0) 0x7fbedc032640 con 0x55705dfa4950 2023-10-06T10:43:14.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbefabcfb80 10 client.7395 fetch_fsmap finished waiting for FSMap version 72 2023-10-06T10:43:14.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbefabcfb80 10 client.7395 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T10:43:14.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbefabcfb80 20 client.7395 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:14.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbefabcfb80 1 --2- 172.21.15.92:0/4133778396 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x55705e054510 0x55705e0548e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbefabcfb80 4 client.7395 mds_command: new command op to 7164 tid=0[{"prefix": "status"}] 2023-10-06T10:43:14.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.380+0000 7fbefabcfb80 1 -- 172.21.15.92:0/4133778396 --> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55705de97e00 con 0x55705e054510 2023-10-06T10:43:14.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.381+0000 7fbee27fc700 1 --2- 172.21.15.92:0/4133778396 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x55705e054510 0x55705e0548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.381+0000 7fbec97fa700 10 client.7395 ms_handle_connect on v2:172.21.15.124:6833/2971348516 2023-10-06T10:43:14.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.381+0000 7fbee27fc700 1 --2- 172.21.15.92:0/4133778396 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x55705e054510 0x55705e0548e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce0503069e029731 server_cookie=6041965262e314ae in_seq=0 out_seq=0 2023-10-06T10:43:14.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.381+0000 7fbec97fa700 1 -- 172.21.15.92:0/4133778396 <== mds.0 v2:172.21.15.124:6833/2971348516 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55705de97e00 con 0x55705e054510 2023-10-06T10:43:14.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.381+0000 7fbec97fa700 10 client.7395 handle_command_reply: tid=0 2023-10-06T10:43:14.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.381+0000 7fbefabcfb80 1 client.7395 shutdown 2023-10-06T10:43:14.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.381+0000 7fbefabcfb80 1 -- 172.21.15.92:0/4133778396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfa4950 msgr2=0x55705e050530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.381+0000 7fbefabcfb80 1 --2- 172.21.15.92:0/4133778396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfa4950 0x55705e050530 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fbedc01deb0 tx=0x7fbedc02b070 comp rx=0 tx=0).stop 2023-10-06T10:43:14.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.381+0000 7fbefabcfb80 1 -- 172.21.15.92:0/4133778396 shutdown_connections 2023-10-06T10:43:14.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.381+0000 7fbefabcfb80 1 --2- 172.21.15.92:0/4133778396 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x55705e054510 0x55705e0548e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.381+0000 7fbefabcfb80 1 --2- 172.21.15.92:0/4133778396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55705dfeebc0 0x55705dfcc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.381+0000 7fbefabcfb80 1 --2- 172.21.15.92:0/4133778396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55705dfa4950 0x55705e050530 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.381+0000 7fbefabcfb80 1 --2- 172.21.15.92:0/4133778396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55705dfcde60 0x55705dfcc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.381+0000 7fbec97fa700 0 client.7395 ms_handle_reset on v2:172.21.15.124:6833/2971348516 2023-10-06T10:43:14.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.381+0000 7fbefabcfb80 1 -- 172.21.15.92:0/4133778396 >> 172.21.15.92:0/4133778396 conn(0x55705dfa6270 msgr2=0x55705df9b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:14.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.381+0000 7fbefabcfb80 1 -- 172.21.15.92:0/4133778396 shutdown_connections 2023-10-06T10:43:14.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.381+0000 7fbefabcfb80 1 -- 172.21.15.92:0/4133778396 wait complete. 2023-10-06T10:43:14.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.382+0000 7fbefabcfb80 20 client.7395 trim_cache size 0 max 16384 2023-10-06T10:43:14.391 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:43:14.391 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:43:14.391 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T10:43:14.391 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 7164, 2023-10-06T10:43:14.391 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:43:14.391 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:43:14.392 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:43:14.392 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 10.595073946999999, 2023-10-06T10:43:14.392 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 72, 2023-10-06T10:43:14.392 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 78, 2023-10-06T10:43:14.392 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 78, 2023-10-06T10:43:14.392 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 16.686943908 2023-10-06T10:43:14.392 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:43:14.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.382+0000 7fbeebfff700 1 -- 172.21.15.92:0/1861670493 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbecc066f30 msgr2=0x7fbecc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.382+0000 7fbeebfff700 1 --2- 172.21.15.92:0/1861670493 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbecc066f30 0x7fbecc0693e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009860 tx=0x7fbedc017000 comp rx=0 tx=0).stop 2023-10-06T10:43:14.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.382+0000 7fbeebfff700 1 -- 172.21.15.92:0/1861670493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbeec0b35d0 msgr2=0x7fbeec0c8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.382+0000 7fbeebfff700 1 --2- 172.21.15.92:0/1861670493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbeec0b35d0 0x7fbeec0c8400 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fbed400cb60 tx=0x7fbed4001ad0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.382+0000 7fbeebfff700 1 -- 172.21.15.92:0/1861670493 shutdown_connections 2023-10-06T10:43:14.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.382+0000 7fbeebfff700 1 --2- 172.21.15.92:0/1861670493 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbecc066f30 0x7fbecc0693e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.382+0000 7fbeebfff700 1 --2- 172.21.15.92:0/1861670493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbeec159440 0x7fbeec0d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.382+0000 7fbeebfff700 1 --2- 172.21.15.92:0/1861670493 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbeec0b2100 0x7fbeec153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.382+0000 7fbeebfff700 1 --2- 172.21.15.92:0/1861670493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbeec0b35d0 0x7fbeec0c8400 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.382+0000 7fbeebfff700 1 -- 172.21.15.92:0/1861670493 >> 172.21.15.92:0/1861670493 conn(0x7fbeec06ece0 msgr2=0x7fbeec154980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:14.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.382+0000 7fbeebfff700 1 -- 172.21.15.92:0/1861670493 shutdown_connections 2023-10-06T10:43:14.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.382+0000 7fbeebfff700 1 -- 172.21.15.92:0/1861670493 wait complete. 2023-10-06T10:43:14.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:43:14.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.508+0000 7fa3bbfff700 1 Processor -- start 2023-10-06T10:43:14.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.508+0000 7fa3bbfff700 1 -- start start 2023-10-06T10:43:14.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.509+0000 7fa3bbfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa3bc159440 0x7fa3bc0b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.509+0000 7fa3bbfff700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa3bc0b2100 0x7fa3bc0b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.509+0000 7fa3bbfff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3bc0cb7a0 con 0x7fa3bc0b35d0 2023-10-06T10:43:14.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.509+0000 7fa3bbfff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3bc0b0860 con 0x7fa3bc159440 2023-10-06T10:43:14.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.509+0000 7fa3bbfff700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3bc0cb150 con 0x7fa3bc0b2100 2023-10-06T10:43:14.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.509+0000 7fa3baffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa3bc159440 0x7fa3bc0b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.509+0000 7fa3ba7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa3bc0b35d0 0x7fa3bc0cbc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44928/0 (socket says 172.21.15.92:44928) 2023-10-06T10:43:14.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.509+0000 7fa3ba7fc700 1 -- 172.21.15.92:0/2929394177 learned_addr learned my addr 172.21.15.92:0/2929394177 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:14.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.509+0000 7fa3baffd700 1 -- 172.21.15.92:0/2929394177 >> v2:172.21.15.124:3300/0 conn(0x7fa3bc0b2100 msgr2=0x7fa3bc0b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:14.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.509+0000 7fa3bb7fe700 1 --2- 172.21.15.92:0/2929394177 >> v2:172.21.15.124:3300/0 conn(0x7fa3bc0b2100 0x7fa3bc0b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.509+0000 7fa3baffd700 1 --2- 172.21.15.92:0/2929394177 >> v2:172.21.15.124:3300/0 conn(0x7fa3bc0b2100 0x7fa3bc0b2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.509+0000 7fa3baffd700 1 -- 172.21.15.92:0/2929394177 >> v1:172.21.15.92:6789/0 conn(0x7fa3bc0b35d0 legacy=0x7fa3bc0cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.509+0000 7fa3b9ffb700 1 -- 172.21.15.92:0/2929394177 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573570722 0 0) 0x7fa3bc0cb7a0 con 0x7fa3bc0b35d0 2023-10-06T10:43:14.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.509+0000 7fa3baffd700 1 -- 172.21.15.92:0/2929394177 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3bc0cb430 con 0x7fa3bc159440 2023-10-06T10:43:14.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.510+0000 7fa3baffd700 1 --2- 172.21.15.92:0/2929394177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3bc159440 0x7fa3bc0b3090 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0ca7f0 tx=0x7fa3ac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a7edd4d13f26d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.510+0000 7fa3b9ffb700 1 -- 172.21.15.92:0/2929394177 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac00e070 con 0x7fa3bc159440 2023-10-06T10:43:14.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.510+0000 7fa3b9ffb700 1 -- 172.21.15.92:0/2929394177 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ac00bc30 con 0x7fa3bc159440 2023-10-06T10:43:14.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.510+0000 7fa3b9ffb700 1 -- 172.21.15.92:0/2929394177 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac012430 con 0x7fa3bc159440 2023-10-06T10:43:14.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.510+0000 7fa3bbfff700 1 -- 172.21.15.92:0/2929394177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3bc159440 msgr2=0x7fa3bc0b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.510+0000 7fa3bbfff700 1 --2- 172.21.15.92:0/2929394177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3bc159440 0x7fa3bc0b3090 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0ca7f0 tx=0x7fa3ac009780 comp rx=0 tx=0).stop 2023-10-06T10:43:14.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.510+0000 7fa3bbfff700 1 -- 172.21.15.92:0/2929394177 shutdown_connections 2023-10-06T10:43:14.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.510+0000 7fa3bbfff700 1 --2- 172.21.15.92:0/2929394177 >> v2:172.21.15.124:3300/0 conn(0x7fa3bc0b2100 0x7fa3bc0b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.510+0000 7fa3bbfff700 1 --2- 172.21.15.92:0/2929394177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3bc159440 0x7fa3bc0b3090 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.510+0000 7fa3bbfff700 1 -- 172.21.15.92:0/2929394177 >> 172.21.15.92:0/2929394177 conn(0x7fa3bc06ece0 msgr2=0x7fa3bc06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:14.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.510+0000 7fa3bbfff700 1 -- 172.21.15.92:0/2929394177 shutdown_connections 2023-10-06T10:43:14.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.510+0000 7fa3bbfff700 1 -- 172.21.15.92:0/2929394177 wait complete. 2023-10-06T10:43:14.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.511+0000 7fa3bbfff700 1 Processor -- start 2023-10-06T10:43:14.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.512+0000 7fa3bbfff700 1 -- start start 2023-10-06T10:43:14.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.512+0000 7fa3bbfff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3bc0b2100 0x7fa3bc0c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.512+0000 7fa3bbfff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3bc0b35d0 0x7fa3bc15cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.512+0000 7fa3bbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3bc159440 0x7fa3bc154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.512+0000 7fa3bbfff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc15d450 con 0x7fa3bc159440 2023-10-06T10:43:14.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.512+0000 7fa3bbfff700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc15d640 con 0x7fa3bc0b2100 2023-10-06T10:43:14.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.512+0000 7fa3bbfff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc15d920 con 0x7fa3bc0b35d0 2023-10-06T10:43:14.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.512+0000 7fa3bb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3bc159440 0x7fa3bc154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.512+0000 7fa3bb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3bc159440 0x7fa3bc154940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48452/0 (socket says 172.21.15.92:48452) 2023-10-06T10:43:14.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.512+0000 7fa3bb7fe700 1 -- 172.21.15.92:0/3038060715 learned_addr learned my addr 172.21.15.92:0/3038060715 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:14.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.513+0000 7fa3ba7fc700 1 --2- 172.21.15.92:0/3038060715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3bc0b35d0 0x7fa3bc15cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.513+0000 7fa3baffd700 1 --2- 172.21.15.92:0/3038060715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3bc0b2100 0x7fa3bc0c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.513+0000 7fa3bb7fe700 1 -- 172.21.15.92:0/3038060715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3bc0b35d0 msgr2=0x7fa3bc15cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.513+0000 7fa3bb7fe700 1 --2- 172.21.15.92:0/3038060715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3bc0b35d0 0x7fa3bc15cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.513+0000 7fa3bb7fe700 1 -- 172.21.15.92:0/3038060715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3bc0b2100 msgr2=0x7fa3bc0c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.513+0000 7fa3bb7fe700 1 --2- 172.21.15.92:0/3038060715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3bc0b2100 0x7fa3bc0c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.513+0000 7fa3bb7fe700 1 -- 172.21.15.92:0/3038060715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac009a20 con 0x7fa3bc159440 2023-10-06T10:43:14.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.513+0000 7fa3baffd700 1 --2- 172.21.15.92:0/3038060715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3bc0b2100 0x7fa3bc0c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:14.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.513+0000 7fa3bb7fe700 1 --2- 172.21.15.92:0/3038060715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3bc159440 0x7fa3bc154940 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400c880 tx=0x7fa3b4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.513+0000 7fa3a2ffd700 1 -- 172.21.15.92:0/3038060715 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4011600 con 0x7fa3bc159440 2023-10-06T10:43:14.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.513+0000 7fa3a2ffd700 1 -- 172.21.15.92:0/3038060715 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b4011760 con 0x7fa3bc159440 2023-10-06T10:43:14.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.513+0000 7fa3bbfff700 1 -- 172.21.15.92:0/3038060715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3bc15dc00 con 0x7fa3bc159440 2023-10-06T10:43:14.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.513+0000 7fa3a2ffd700 1 -- 172.21.15.92:0/3038060715 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4011ae0 con 0x7fa3bc159440 2023-10-06T10:43:14.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.513+0000 7fa3bbfff700 1 -- 172.21.15.92:0/3038060715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3bc15e1c0 con 0x7fa3bc159440 2023-10-06T10:43:14.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.514+0000 7fa3bbfff700 1 -- 172.21.15.92:0/3038060715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3bc15e3e0 con 0x7fa3bc159440 2023-10-06T10:43:14.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.514+0000 7fa3a2ffd700 1 -- 172.21.15.92:0/3038060715 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa3b40193f0 con 0x7fa3bc159440 2023-10-06T10:43:14.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.515+0000 7fa3a2ffd700 1 --2- 172.21.15.92:0/3038060715 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa3a8066f30 0x7fa3a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:14.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.515+0000 7fa3baffd700 1 --2- 172.21.15.92:0/3038060715 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa3a8066f30 0x7fa3a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:14.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.515+0000 7fa3a2ffd700 1 -- 172.21.15.92:0/3038060715 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa3b4084bc0 con 0x7fa3bc159440 2023-10-06T10:43:14.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.515+0000 7fa3baffd700 1 --2- 172.21.15.92:0/3038060715 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa3a8066f30 0x7fa3a80693e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0ca7f0 tx=0x7fa3ac019000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:14.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.519+0000 7fa3a2ffd700 1 -- 172.21.15.92:0/3038060715 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3b4051af0 con 0x7fa3bc159440 2023-10-06T10:43:14.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.746+0000 7fa3bbfff700 1 -- 172.21.15.92:0/3038060715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3bc05f230 con 0x7fa3bc159440 2023-10-06T10:43:14.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.746+0000 7fa3a2ffd700 1 -- 172.21.15.92:0/3038060715 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4633 (secure 0 0 0) 0x7fa3b4055110 con 0x7fa3bc159440 2023-10-06T10:43:14.748 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:14.748 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":72,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3469511807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3469511807},{"type":"v1","addr":"172.21.15.124:6834","nonce":3469511807}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:03.173076+0000","modified":"2023-10-06T10:43:13.353809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7164,"mds_1":7149,"mds_2":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"a","rank":2,"incarnation":71,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/4003376156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4003376156},{"type":"v1","addr":"172.21.15.92:6837","nonce":4003376156}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7149":{"gid":7149,"name":"c","rank":1,"incarnation":69,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6836/3268908749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3268908749},{"type":"v1","addr":"172.21.15.92:6836","nonce":3268908749}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7164":{"gid":7164,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/2971348516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2971348516},{"type":"v1","addr":"172.21.15.124:6835","nonce":2971348516}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T10:43:14.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.748+0000 7fa3bbfff700 1 -- 172.21.15.92:0/3038060715 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa3a8066f30 msgr2=0x7fa3a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.748+0000 7fa3bbfff700 1 --2- 172.21.15.92:0/3038060715 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa3a8066f30 0x7fa3a80693e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0ca7f0 tx=0x7fa3ac019000 comp rx=0 tx=0).stop 2023-10-06T10:43:14.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.748+0000 7fa3bbfff700 1 -- 172.21.15.92:0/3038060715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3bc159440 msgr2=0x7fa3bc154940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:14.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.748+0000 7fa3bbfff700 1 --2- 172.21.15.92:0/3038060715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3bc159440 0x7fa3bc154940 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400c880 tx=0x7fa3b4008040 comp rx=0 tx=0).stop 2023-10-06T10:43:14.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.748+0000 7fa3bbfff700 1 -- 172.21.15.92:0/3038060715 shutdown_connections 2023-10-06T10:43:14.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.748+0000 7fa3bbfff700 1 --2- 172.21.15.92:0/3038060715 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa3a8066f30 0x7fa3a80693e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.748+0000 7fa3bbfff700 1 --2- 172.21.15.92:0/3038060715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3bc159440 0x7fa3bc154940 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.748+0000 7fa3bbfff700 1 --2- 172.21.15.92:0/3038060715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3bc0b2100 0x7fa3bc0c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.748+0000 7fa3bbfff700 1 --2- 172.21.15.92:0/3038060715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3bc0b35d0 0x7fa3bc15cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:14.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.748+0000 7fa3bbfff700 1 -- 172.21.15.92:0/3038060715 >> 172.21.15.92:0/3038060715 conn(0x7fa3bc06ece0 msgr2=0x7fa3bc0cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:14.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.748+0000 7fa3bbfff700 1 -- 172.21.15.92:0/3038060715 shutdown_connections 2023-10-06T10:43:14.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:14.749+0000 7fa3bbfff700 1 -- 172.21.15.92:0/3038060715 wait complete. 2023-10-06T10:43:14.751 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 72 2023-10-06T10:43:14.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:43:14.761 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2023-10-06T10:43:14.786 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:43:14.786 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2023-10-06T10:43:14.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:43:14.849 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2023-10-06T10:43:14.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:43:14.905 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2023-10-06T10:43:14.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:43:14.960 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2023-10-06T10:43:15.015 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2023-10-06T10:43:15.154 INFO:teuthology.orchestra.run.smithi124.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:43:15.154 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:43:15.155 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:43:17.156 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2023-10-06T10:43:17.295 INFO:teuthology.orchestra.run.smithi124.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:43:17.296 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:43:17.296 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:43:19.297 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2023-10-06T10:43:19.443 INFO:teuthology.orchestra.run.smithi124.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:43:19.444 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:43:19.444 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:43:21.445 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2023-10-06T10:43:21.585 INFO:teuthology.orchestra.run.smithi124.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:43:21.586 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:43:21.586 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:43:23.292 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:43:23.294 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:43:23.587 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2023-10-06T10:43:23.727 INFO:teuthology.orchestra.run.smithi124.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2023-10-06T10:43:23.728 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2023-10-06T10:43:23.728 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:43:23.728 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:43:23.875 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-10-06T10:43:23.876 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:43:23.876 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:43:25.877 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:43:26.019 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-10-06T10:43:26.020 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:43:26.020 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:43:28.022 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:43:28.163 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-10-06T10:43:28.164 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:43:28.164 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:43:30.165 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:43:30.305 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2023-10-06T10:43:30.306 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2023-10-06T10:43:30.306 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:43:30.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-10-06T10:43:30.437 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-10-06T10:43:30.438 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:43:30.438 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:43:30.438 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2023-10-06T10:43:30.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.553+0000 7efd8eaf4700 1 Processor -- start 2023-10-06T10:43:30.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.553+0000 7efd8eaf4700 1 -- start start 2023-10-06T10:43:30.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.554+0000 7efd8eaf4700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efd880b2620 0x7efd880cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.554+0000 7efd8eaf4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd880b29f0 0x7efd880cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.554+0000 7efd8eaf4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd880cc970 con 0x7efd880ceaa0 2023-10-06T10:43:30.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.554+0000 7efd8eaf4700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd880cbd60 con 0x7efd880b2620 2023-10-06T10:43:30.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.554+0000 7efd8eaf4700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd880cc040 con 0x7efd880b29f0 2023-10-06T10:43:30.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.554+0000 7efd8e2f3700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7efd880ceaa0 0x7efd880cef40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53938/0 (socket says 172.21.15.92:53938) 2023-10-06T10:43:30.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.554+0000 7efd8e2f3700 1 -- 172.21.15.92:0/1192821046 learned_addr learned my addr 172.21.15.92:0/1192821046 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:30.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.554+0000 7efd8daf2700 1 --2- 172.21.15.92:0/1192821046 >> v2:172.21.15.124:3300/0 conn(0x7efd880b2620 0x7efd880cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.554+0000 7efd8d2f1700 1 --2- 172.21.15.92:0/1192821046 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd880b29f0 0x7efd880cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.554+0000 7efd8caf0700 1 -- 172.21.15.92:0/1192821046 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2527902048 0 0) 0x7efd880cc970 con 0x7efd880ceaa0 2023-10-06T10:43:30.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.554+0000 7efd8caf0700 1 -- 172.21.15.92:0/1192821046 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd74003570 con 0x7efd880ceaa0 2023-10-06T10:43:30.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.554+0000 7efd8caf0700 1 -- 172.21.15.92:0/1192821046 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 754591581 0 0) 0x7efd74003570 con 0x7efd880ceaa0 2023-10-06T10:43:30.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.554+0000 7efd8caf0700 1 -- 172.21.15.92:0/1192821046 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd880b29f0 msgr2=0x7efd880cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.554+0000 7efd8caf0700 1 --2- 172.21.15.92:0/1192821046 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd880b29f0 0x7efd880cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.554+0000 7efd8caf0700 1 -- 172.21.15.92:0/1192821046 >> v2:172.21.15.124:3300/0 conn(0x7efd880b2620 msgr2=0x7efd880cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.554+0000 7efd8caf0700 1 --2- 172.21.15.92:0/1192821046 >> v2:172.21.15.124:3300/0 conn(0x7efd880b2620 0x7efd880cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.555+0000 7efd8caf0700 1 -- 172.21.15.92:0/1192821046 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd880cc320 con 0x7efd880ceaa0 2023-10-06T10:43:30.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.555+0000 7efd8caf0700 1 -- 172.21.15.92:0/1192821046 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7efd800017f0 con 0x7efd880ceaa0 2023-10-06T10:43:30.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.555+0000 7efd8caf0700 1 -- 172.21.15.92:0/1192821046 >> v1:172.21.15.92:6789/0 conn(0x7efd880ceaa0 legacy=0x7efd880cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.555+0000 7efd8caf0700 1 --2- 172.21.15.92:0/1192821046 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd74003a60 0x7efd74003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.555+0000 7efd8caf0700 1 --2- 172.21.15.92:0/1192821046 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd880b29f0 0x7efd74004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.555+0000 7efd8caf0700 1 --2- 172.21.15.92:0/1192821046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd74004d50 0x7efd740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.555+0000 7efd8caf0700 1 -- 172.21.15.92:0/1192821046 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd880cbd60 con 0x7efd74004d50 2023-10-06T10:43:30.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.555+0000 7efd8caf0700 1 -- 172.21.15.92:0/1192821046 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd880cc040 con 0x7efd74003a60 2023-10-06T10:43:30.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.555+0000 7efd8caf0700 1 -- 172.21.15.92:0/1192821046 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd7400ba60 con 0x7efd880b29f0 2023-10-06T10:43:30.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.555+0000 7efd8e2f3700 1 --2- 172.21.15.92:0/1192821046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd74004d50 0x7efd740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.555+0000 7efd8daf2700 1 --2- 172.21.15.92:0/1192821046 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd74003a60 0x7efd74003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.555+0000 7efd8d2f1700 1 --2- 172.21.15.92:0/1192821046 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd880b29f0 0x7efd74004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.555+0000 7efd8e2f3700 1 -- 172.21.15.92:0/1192821046 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd880b29f0 msgr2=0x7efd74004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.555+0000 7efd8e2f3700 1 --2- 172.21.15.92:0/1192821046 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd880b29f0 0x7efd74004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.555+0000 7efd8e2f3700 1 -- 172.21.15.92:0/1192821046 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd74003a60 msgr2=0x7efd74003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.555+0000 7efd8e2f3700 1 --2- 172.21.15.92:0/1192821046 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd74003a60 0x7efd74003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.555+0000 7efd8e2f3700 1 -- 172.21.15.92:0/1192821046 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd74003570 con 0x7efd74004d50 2023-10-06T10:43:30.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.555+0000 7efd8e2f3700 1 --2- 172.21.15.92:0/1192821046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd74004d50 0x7efd740051c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7efd8000a970 tx=0x7efd80002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b911a6a5889cc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:30.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.555+0000 7efd8caf0700 1 -- 172.21.15.92:0/1192821046 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd80012440 con 0x7efd74004d50 2023-10-06T10:43:30.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.555+0000 7efd8caf0700 1 -- 172.21.15.92:0/1192821046 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd800125c0 con 0x7efd74004d50 2023-10-06T10:43:30.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.555+0000 7efd8caf0700 1 -- 172.21.15.92:0/1192821046 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd80012980 con 0x7efd74004d50 2023-10-06T10:43:30.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.556+0000 7efd8eaf4700 1 -- 172.21.15.92:0/1192821046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd74004d50 msgr2=0x7efd740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.556+0000 7efd8eaf4700 1 --2- 172.21.15.92:0/1192821046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd74004d50 0x7efd740051c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7efd8000a970 tx=0x7efd80002210 comp rx=0 tx=0).stop 2023-10-06T10:43:30.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.556+0000 7efd8e2f3700 1 -- 172.21.15.92:0/1192821046 reap_dead start 2023-10-06T10:43:30.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.556+0000 7efd8eaf4700 1 -- 172.21.15.92:0/1192821046 shutdown_connections 2023-10-06T10:43:30.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.556+0000 7efd8eaf4700 1 -- 172.21.15.92:0/1192821046 >> 172.21.15.92:0/1192821046 conn(0x7efd88099520 msgr2=0x7efd8806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:30.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.556+0000 7efd8eaf4700 1 -- 172.21.15.92:0/1192821046 shutdown_connections 2023-10-06T10:43:30.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.556+0000 7efd8eaf4700 1 -- 172.21.15.92:0/1192821046 wait complete. 2023-10-06T10:43:30.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.557+0000 7efd8eaf4700 1 Processor -- start 2023-10-06T10:43:30.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.557+0000 7efd8eaf4700 1 -- start start 2023-10-06T10:43:30.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.558+0000 7efd8eaf4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd8814c9d0 0x7efd881537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.558+0000 7efd8eaf4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd880c7510 0x7efd88152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.558+0000 7efd8eaf4700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd88152e90 0x7efd880b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.558+0000 7efd8eaf4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd880c7a10 con 0x7efd880c7510 2023-10-06T10:43:30.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.558+0000 7efd8eaf4700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd880c7cf0 con 0x7efd88152e90 2023-10-06T10:43:30.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.558+0000 7efd8eaf4700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd880c7fd0 con 0x7efd8814c9d0 2023-10-06T10:43:30.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.558+0000 7efd8d2f1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd880c7510 0x7efd88152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.558+0000 7efd8d2f1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd880c7510 0x7efd88152950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60664/0 (socket says 172.21.15.92:60664) 2023-10-06T10:43:30.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.558+0000 7efd8d2f1700 1 -- 172.21.15.92:0/2159981886 learned_addr learned my addr 172.21.15.92:0/2159981886 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:30.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.558+0000 7efd8daf2700 1 --2- 172.21.15.92:0/2159981886 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd8814c9d0 0x7efd881537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.558+0000 7efd8e2f3700 1 --2- 172.21.15.92:0/2159981886 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd88152e90 0x7efd880b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.558+0000 7efd8d2f1700 1 -- 172.21.15.92:0/2159981886 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd8814c9d0 msgr2=0x7efd881537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.558+0000 7efd8d2f1700 1 --2- 172.21.15.92:0/2159981886 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd8814c9d0 0x7efd881537b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.558+0000 7efd8d2f1700 1 -- 172.21.15.92:0/2159981886 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd88152e90 msgr2=0x7efd880b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.558+0000 7efd8d2f1700 1 --2- 172.21.15.92:0/2159981886 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd88152e90 0x7efd880b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.558+0000 7efd8d2f1700 1 -- 172.21.15.92:0/2159981886 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd880c82b0 con 0x7efd880c7510 2023-10-06T10:43:30.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.558+0000 7efd8e2f3700 1 --2- 172.21.15.92:0/2159981886 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd88152e90 0x7efd880b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:30.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.558+0000 7efd8d2f1700 1 --2- 172.21.15.92:0/2159981886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd880c7510 0x7efd88152950 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7efd78006ac0 tx=0x7efd780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:30.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.559+0000 7efd7dffb700 1 -- 172.21.15.92:0/2159981886 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7800f940 con 0x7efd880c7510 2023-10-06T10:43:30.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.559+0000 7efd7dffb700 1 -- 172.21.15.92:0/2159981886 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7800faa0 con 0x7efd880c7510 2023-10-06T10:43:30.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.559+0000 7efd8eaf4700 1 -- 172.21.15.92:0/2159981886 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd880c8590 con 0x7efd880c7510 2023-10-06T10:43:30.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.559+0000 7efd7dffb700 1 -- 172.21.15.92:0/2159981886 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7800fe20 con 0x7efd880c7510 2023-10-06T10:43:30.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.559+0000 7efd8eaf4700 1 -- 172.21.15.92:0/2159981886 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd88153cf0 con 0x7efd880c7510 2023-10-06T10:43:30.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.560+0000 7efd7dffb700 1 -- 172.21.15.92:0/2159981886 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efd78014650 con 0x7efd880c7510 2023-10-06T10:43:30.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.560+0000 7efd7dffb700 1 --2- 172.21.15.92:0/2159981886 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efd6c066f30 0x7efd6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.561+0000 7efd7dffb700 1 -- 172.21.15.92:0/2159981886 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7efd78018030 con 0x7efd880c7510 2023-10-06T10:43:30.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.561+0000 7efd8daf2700 1 --2- 172.21.15.92:0/2159981886 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efd6c066f30 0x7efd6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.561+0000 7efd8daf2700 1 --2- 172.21.15.92:0/2159981886 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efd6c066f30 0x7efd6c0693e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7efd880c6b40 tx=0x7efd84009120 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:30.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.562+0000 7efd9904eb80 1 Processor -- start 2023-10-06T10:43:30.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.562+0000 7efd9904eb80 1 -- start start 2023-10-06T10:43:30.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.563+0000 7efd9904eb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a16c5b9a0 0x556a16c5bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.563+0000 7efd9904eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16ca5c10 0x556a16c84970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.563+0000 7efd9904eb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a16c84eb0 0x556a16c579e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.563+0000 7efd9904eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a16bdbcd0 con 0x556a16ca5c10 2023-10-06T10:43:30.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.563+0000 7efd9904eb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a16bb2ac0 con 0x556a16c5b9a0 2023-10-06T10:43:30.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.563+0000 7efd9904eb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a16bb02c0 con 0x556a16c84eb0 2023-10-06T10:43:30.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.563+0000 7efd8d2f1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16ca5c10 0x556a16c84970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.563+0000 7efd8d2f1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16ca5c10 0x556a16c84970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60678/0 (socket says 172.21.15.92:60678) 2023-10-06T10:43:30.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.563+0000 7efd8daf2700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a16c5b9a0 0x556a16c5bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.563+0000 7efd8d2f1700 1 -- 172.21.15.92:0/1196918375 learned_addr learned my addr 172.21.15.92:0/1196918375 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:30.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.563+0000 7efd8e2f3700 1 --2- 172.21.15.92:0/1196918375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a16c84eb0 0x556a16c579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.563+0000 7efd8d2f1700 1 -- 172.21.15.92:0/1196918375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a16c84eb0 msgr2=0x556a16c579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.563+0000 7efd8d2f1700 1 --2- 172.21.15.92:0/1196918375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a16c84eb0 0x556a16c579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.563+0000 7efd8d2f1700 1 -- 172.21.15.92:0/1196918375 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a16c5b9a0 msgr2=0x556a16c5bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.563+0000 7efd8d2f1700 1 --2- 172.21.15.92:0/1196918375 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a16c5b9a0 0x556a16c5bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.563+0000 7efd8d2f1700 1 -- 172.21.15.92:0/1196918375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a16a462c0 con 0x556a16ca5c10 2023-10-06T10:43:30.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.563+0000 7efd8d2f1700 1 --2- 172.21.15.92:0/1196918375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16ca5c10 0x556a16c84970 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7efd78083440 tx=0x7efd780833a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4377a0d4b7e286ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:30.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.564+0000 7efd72ffd700 1 -- 172.21.15.92:0/1196918375 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd78055ed0 con 0x556a16ca5c10 2023-10-06T10:43:30.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.564+0000 7efd72ffd700 1 -- 172.21.15.92:0/1196918375 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd78059120 con 0x556a16ca5c10 2023-10-06T10:43:30.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.564+0000 7efd72ffd700 1 -- 172.21.15.92:0/1196918375 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7805ce70 con 0x556a16ca5c10 2023-10-06T10:43:30.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.564+0000 7efd9904eb80 1 -- 172.21.15.92:0/1196918375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16ca5c10 msgr2=0x556a16c84970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.564+0000 7efd9904eb80 1 --2- 172.21.15.92:0/1196918375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16ca5c10 0x556a16c84970 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7efd78083440 tx=0x7efd780833a0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.564+0000 7efd9904eb80 1 -- 172.21.15.92:0/1196918375 shutdown_connections 2023-10-06T10:43:30.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.564+0000 7efd9904eb80 1 --2- 172.21.15.92:0/1196918375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a16c84eb0 0x556a16c579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.564+0000 7efd9904eb80 1 --2- 172.21.15.92:0/1196918375 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a16c5b9a0 0x556a16c5bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.564+0000 7efd9904eb80 1 --2- 172.21.15.92:0/1196918375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16ca5c10 0x556a16c84970 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.564+0000 7efd9904eb80 1 -- 172.21.15.92:0/1196918375 >> 172.21.15.92:0/1196918375 conn(0x556a16c5d2c0 msgr2=0x556a16c5d690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:30.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.564+0000 7efd9904eb80 1 -- 172.21.15.92:0/1196918375 shutdown_connections 2023-10-06T10:43:30.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.564+0000 7efd9904eb80 1 -- 172.21.15.92:0/1196918375 wait complete. 2023-10-06T10:43:30.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.565+0000 7efd9904eb80 1 -- start start 2023-10-06T10:43:30.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.565+0000 7efd9904eb80 1 Processor -- start 2023-10-06T10:43:30.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd9904eb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a16c5b9a0 0x556a16c80d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd9904eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16c84eb0 0x556a16c83990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd9904eb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a16ca5c10 0x556a16c9d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd9904eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a16bac0b0 con 0x556a16c84eb0 2023-10-06T10:43:30.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd9904eb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a16b578e0 con 0x556a16ca5c10 2023-10-06T10:43:30.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd9904eb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a16b72ef0 con 0x556a16c5b9a0 2023-10-06T10:43:30.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd9904eb80 10 client.0 _ll_register_callbacks cb 0x556a16a48f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:30.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd8d2f1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16c84eb0 0x556a16c83990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd8d2f1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16c84eb0 0x556a16c83990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60694/0 (socket says 172.21.15.92:60694) 2023-10-06T10:43:30.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd8d2f1700 1 -- 172.21.15.92:0/3923404066 learned_addr learned my addr 172.21.15.92:0/3923404066 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:30.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd8daf2700 1 --2- 172.21.15.92:0/3923404066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a16c5b9a0 0x556a16c80d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd8e2f3700 1 --2- 172.21.15.92:0/3923404066 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a16ca5c10 0x556a16c9d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd8d2f1700 1 -- 172.21.15.92:0/3923404066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a16c5b9a0 msgr2=0x556a16c80d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd8d2f1700 1 --2- 172.21.15.92:0/3923404066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a16c5b9a0 0x556a16c80d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd8d2f1700 1 -- 172.21.15.92:0/3923404066 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a16ca5c10 msgr2=0x556a16c9d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd8d2f1700 1 --2- 172.21.15.92:0/3923404066 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a16ca5c10 0x556a16c9d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd8d2f1700 1 -- 172.21.15.92:0/3923404066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd88153cf0 con 0x556a16c84eb0 2023-10-06T10:43:30.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd8daf2700 1 --2- 172.21.15.92:0/3923404066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a16c5b9a0 0x556a16c80d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:30.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd8d2f1700 1 --2- 172.21.15.92:0/3923404066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16c84eb0 0x556a16c83990 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7efd780544c0 tx=0x7efd78008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3918a8f4231049 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:30.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd67fff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:30.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd67fff700 1 -- 172.21.15.92:0/3923404066 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7806c170 con 0x556a16c84eb0 2023-10-06T10:43:30.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd67fff700 1 -- 172.21.15.92:0/3923404066 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7efd7806c2d0 con 0x556a16c84eb0 2023-10-06T10:43:30.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.566+0000 7efd9904eb80 1 -- 172.21.15.92:0/3923404066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a16c506e0 con 0x556a16c84eb0 2023-10-06T10:43:30.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.567+0000 7efd67fff700 1 -- 172.21.15.92:0/3923404066 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd780553c0 con 0x556a16c84eb0 2023-10-06T10:43:30.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.567+0000 7efd9904eb80 10 client.7416 fetch_fsmap learned FSMap version 74 2023-10-06T10:43:30.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.567+0000 7efd9904eb80 1 -- 172.21.15.92:0/3923404066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x556a169e9030 con 0x556a16c84eb0 2023-10-06T10:43:30.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.567+0000 7efd67fff700 1 -- 172.21.15.92:0/3923404066 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556a16c506e0 con 0x556a16c84eb0 2023-10-06T10:43:30.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.567+0000 7efd67fff700 1 -- 172.21.15.92:0/3923404066 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 74) v1 ==== 3136+0+0 (secure 0 0 0) 0x7efd7805c9e0 con 0x556a16c84eb0 2023-10-06T10:43:30.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.567+0000 7efd9904eb80 10 client.7416 fetch_fsmap finished waiting for FSMap version 74 2023-10-06T10:43:30.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.567+0000 7efd9904eb80 10 client.7416 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2023-10-06T10:43:30.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.567+0000 7efd9904eb80 20 client.7416 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:30.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.567+0000 7efd9904eb80 1 --2- 172.21.15.92:0/3923404066 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x556a16c526e0 0x556a16cb1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.567+0000 7efd9904eb80 4 client.7416 mds_command: new command op to 7164 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:43:30.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.567+0000 7efd9904eb80 1 -- 172.21.15.92:0/3923404066 --> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a16c2d610 con 0x556a16c526e0 2023-10-06T10:43:30.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.567+0000 7efd8daf2700 1 --2- 172.21.15.92:0/3923404066 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x556a16c526e0 0x556a16cb1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.567+0000 7efd67fff700 10 client.7416 ms_handle_connect on v2:172.21.15.124:6833/2971348516 2023-10-06T10:43:30.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.567+0000 7efd8daf2700 1 --2- 172.21.15.92:0/3923404066 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x556a16c526e0 0x556a16cb1e00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc55b0628a0ad678 server_cookie=87e7774ebaf7ff23 in_seq=0 out_seq=0 2023-10-06T10:43:30.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.568+0000 7efd67fff700 1 -- 172.21.15.92:0/3923404066 <== mds.0 v2:172.21.15.124:6833/2971348516 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x556a16c2d610 con 0x556a16c526e0 2023-10-06T10:43:30.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.568+0000 7efd67fff700 10 client.7416 handle_command_reply: tid=0 2023-10-06T10:43:30.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.568+0000 7efd9904eb80 1 client.7416 shutdown 2023-10-06T10:43:30.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.568+0000 7efd9904eb80 1 -- 172.21.15.92:0/3923404066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16c84eb0 msgr2=0x556a16c83990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.568+0000 7efd9904eb80 1 --2- 172.21.15.92:0/3923404066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16c84eb0 0x556a16c83990 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7efd780544c0 tx=0x7efd78008030 comp rx=0 tx=0).stop 2023-10-06T10:43:30.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.568+0000 7efd9904eb80 1 -- 172.21.15.92:0/3923404066 shutdown_connections 2023-10-06T10:43:30.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.568+0000 7efd9904eb80 1 --2- 172.21.15.92:0/3923404066 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a16ca5c10 0x556a16c9d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.568+0000 7efd9904eb80 1 --2- 172.21.15.92:0/3923404066 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x556a16c526e0 0x556a16cb1e00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.568+0000 7efd9904eb80 1 --2- 172.21.15.92:0/3923404066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a16c5b9a0 0x556a16c80d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.568+0000 7efd9904eb80 1 --2- 172.21.15.92:0/3923404066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16c84eb0 0x556a16c83990 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.568+0000 7efd67fff700 0 client.7416 ms_handle_reset on v2:172.21.15.124:6833/2971348516 2023-10-06T10:43:30.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.568+0000 7efd9904eb80 1 -- 172.21.15.92:0/3923404066 >> 172.21.15.92:0/3923404066 conn(0x556a16c5d2c0 msgr2=0x556a16c5bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:30.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.568+0000 7efd9904eb80 1 -- 172.21.15.92:0/3923404066 shutdown_connections 2023-10-06T10:43:30.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.568+0000 7efd9904eb80 1 -- 172.21.15.92:0/3923404066 wait complete. 2023-10-06T10:43:30.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.569+0000 7efd9904eb80 20 client.7416 trim_cache size 0 max 16384 2023-10-06T10:43:30.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.582+0000 7efd9904eb80 1 Processor -- start 2023-10-06T10:43:30.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.582+0000 7efd9904eb80 1 -- start start 2023-10-06T10:43:30.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.583+0000 7efd9904eb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a16c5b9a0 0x556a16c80090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.583+0000 7efd9904eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16c84eb0 0x556a16c99b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.583+0000 7efd9904eb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a16ca5c10 0x556a16c9a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.583+0000 7efd9904eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a16c59120 con 0x556a16c84eb0 2023-10-06T10:43:30.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.583+0000 7efd9904eb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a16a2ee80 con 0x556a16ca5c10 2023-10-06T10:43:30.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.583+0000 7efd9904eb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a16c8b430 con 0x556a16c5b9a0 2023-10-06T10:43:30.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.583+0000 7efd8d2f1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16c84eb0 0x556a16c99b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.583+0000 7efd8d2f1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16c84eb0 0x556a16c99b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60708/0 (socket says 172.21.15.92:60708) 2023-10-06T10:43:30.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.583+0000 7efd8d2f1700 1 -- 172.21.15.92:0/647569945 learned_addr learned my addr 172.21.15.92:0/647569945 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:30.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.583+0000 7efd8d2f1700 1 -- 172.21.15.92:0/647569945 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a16c5b9a0 msgr2=0x556a16c80090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.583+0000 7efd8d2f1700 1 --2- 172.21.15.92:0/647569945 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a16c5b9a0 0x556a16c80090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.583+0000 7efd8d2f1700 1 -- 172.21.15.92:0/647569945 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a16ca5c10 msgr2=0x556a16c9a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.583+0000 7efd8d2f1700 1 --2- 172.21.15.92:0/647569945 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a16ca5c10 0x556a16c9a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.583+0000 7efd8d2f1700 1 -- 172.21.15.92:0/647569945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a16c9b860 con 0x556a16c84eb0 2023-10-06T10:43:30.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.583+0000 7efd8d2f1700 1 --2- 172.21.15.92:0/647569945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16c84eb0 0x556a16c99b30 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7efd78058010 tx=0x7efd7807f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485bd7d569fff509 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:30.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.583+0000 7efd737fe700 1 -- 172.21.15.92:0/647569945 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd78055a10 con 0x556a16c84eb0 2023-10-06T10:43:30.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.584+0000 7efd737fe700 1 -- 172.21.15.92:0/647569945 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7805ce70 con 0x556a16c84eb0 2023-10-06T10:43:30.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.584+0000 7efd737fe700 1 -- 172.21.15.92:0/647569945 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd78051cc0 con 0x556a16c84eb0 2023-10-06T10:43:30.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.584+0000 7efd9904eb80 1 -- 172.21.15.92:0/647569945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16c84eb0 msgr2=0x556a16c99b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.584+0000 7efd9904eb80 1 --2- 172.21.15.92:0/647569945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16c84eb0 0x556a16c99b30 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7efd78058010 tx=0x7efd7807f040 comp rx=0 tx=0).stop 2023-10-06T10:43:30.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.584+0000 7efd9904eb80 1 -- 172.21.15.92:0/647569945 shutdown_connections 2023-10-06T10:43:30.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.584+0000 7efd9904eb80 1 --2- 172.21.15.92:0/647569945 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a16ca5c10 0x556a16c9a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.584+0000 7efd9904eb80 1 --2- 172.21.15.92:0/647569945 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a16c5b9a0 0x556a16c80090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.584+0000 7efd9904eb80 1 --2- 172.21.15.92:0/647569945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16c84eb0 0x556a16c99b30 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.584+0000 7efd9904eb80 1 -- 172.21.15.92:0/647569945 >> 172.21.15.92:0/647569945 conn(0x556a16c5d2c0 msgr2=0x556a16c9cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:30.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.584+0000 7efd9904eb80 1 -- 172.21.15.92:0/647569945 shutdown_connections 2023-10-06T10:43:30.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.584+0000 7efd9904eb80 1 -- 172.21.15.92:0/647569945 wait complete. 2023-10-06T10:43:30.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.585+0000 7efd9904eb80 1 -- start start 2023-10-06T10:43:30.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.585+0000 7efd9904eb80 1 Processor -- start 2023-10-06T10:43:30.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd9904eb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a16c5b9a0 0x556a16d0a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd9904eb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a16c84eb0 0x556a16c99a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd9904eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16ca5c10 0x556a16c99fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd9904eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a16c80640 con 0x556a16ca5c10 2023-10-06T10:43:30.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd9904eb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a16c9edf0 con 0x556a16c5b9a0 2023-10-06T10:43:30.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd9904eb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a16c81fe0 con 0x556a16c84eb0 2023-10-06T10:43:30.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd9904eb80 10 client.0 _ll_register_callbacks cb 0x556a16a48f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:30.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd8e2f3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16ca5c10 0x556a16c99fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd8e2f3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16ca5c10 0x556a16c99fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60714/0 (socket says 172.21.15.92:60714) 2023-10-06T10:43:30.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd8e2f3700 1 -- 172.21.15.92:0/911250988 learned_addr learned my addr 172.21.15.92:0/911250988 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:30.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd8daf2700 1 --2- 172.21.15.92:0/911250988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a16c5b9a0 0x556a16d0a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd8d2f1700 1 --2- 172.21.15.92:0/911250988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a16c84eb0 0x556a16c99a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd8e2f3700 1 -- 172.21.15.92:0/911250988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a16c84eb0 msgr2=0x556a16c99a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd8e2f3700 1 --2- 172.21.15.92:0/911250988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a16c84eb0 0x556a16c99a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd8e2f3700 1 -- 172.21.15.92:0/911250988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a16c5b9a0 msgr2=0x556a16d0a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd8e2f3700 1 --2- 172.21.15.92:0/911250988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a16c5b9a0 0x556a16d0a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd8e2f3700 1 -- 172.21.15.92:0/911250988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a169e9030 con 0x556a16ca5c10 2023-10-06T10:43:30.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd8d2f1700 1 --2- 172.21.15.92:0/911250988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a16c84eb0 0x556a16c99a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:30.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd8e2f3700 1 --2- 172.21.15.92:0/911250988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16ca5c10 0x556a16c99fc0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7efd80002250 tx=0x7efd80015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4078dceb7bfd65a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:30.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd67fff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:30.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd67fff700 1 -- 172.21.15.92:0/911250988 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd80025070 con 0x556a16ca5c10 2023-10-06T10:43:30.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd67fff700 1 -- 172.21.15.92:0/911250988 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7efd8001f410 con 0x556a16ca5c10 2023-10-06T10:43:30.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd9904eb80 1 -- 172.21.15.92:0/911250988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd78059120 con 0x556a16ca5c10 2023-10-06T10:43:30.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd67fff700 1 -- 172.21.15.92:0/911250988 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd80012440 con 0x556a16ca5c10 2023-10-06T10:43:30.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd9904eb80 10 client.7422 fetch_fsmap learned FSMap version 74 2023-10-06T10:43:30.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd9904eb80 1 -- 172.21.15.92:0/911250988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x556a16b7a7d0 con 0x556a16ca5c10 2023-10-06T10:43:30.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd67fff700 1 -- 172.21.15.92:0/911250988 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd78059120 con 0x556a16ca5c10 2023-10-06T10:43:30.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.586+0000 7efd67fff700 1 -- 172.21.15.92:0/911250988 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 74) v1 ==== 3136+0+0 (secure 0 0 0) 0x7efd8001f7e0 con 0x556a16ca5c10 2023-10-06T10:43:30.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.587+0000 7efd9904eb80 10 client.7422 fetch_fsmap finished waiting for FSMap version 74 2023-10-06T10:43:30.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.587+0000 7efd9904eb80 10 client.7422 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2023-10-06T10:43:30.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.587+0000 7efd9904eb80 20 client.7422 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:30.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.587+0000 7efd9904eb80 1 --2- 172.21.15.92:0/911250988 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x556a16c9d7d0 0x556a16cbaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.587+0000 7efd9904eb80 4 client.7422 mds_command: new command op to 7164 tid=0[{"prefix": "flush journal"}] 2023-10-06T10:43:30.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.587+0000 7efd9904eb80 1 -- 172.21.15.92:0/911250988 --> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556a16b4ee50 con 0x556a16c9d7d0 2023-10-06T10:43:30.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.587+0000 7efd8daf2700 1 --2- 172.21.15.92:0/911250988 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x556a16c9d7d0 0x556a16cbaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.587+0000 7efd67fff700 10 client.7422 ms_handle_connect on v2:172.21.15.124:6833/2971348516 2023-10-06T10:43:30.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.587+0000 7efd8daf2700 1 --2- 172.21.15.92:0/911250988 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x556a16c9d7d0 0x556a16cbaf50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfd57f0bb3bcaf6f server_cookie=69adda6f0309a5bb in_seq=0 out_seq=0 2023-10-06T10:43:30.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.604+0000 7efd67fff700 1 -- 172.21.15.92:0/911250988 <== mds.0 v2:172.21.15.124:6833/2971348516 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556a16b4ee50 con 0x556a16c9d7d0 2023-10-06T10:43:30.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.604+0000 7efd67fff700 10 client.7422 handle_command_reply: tid=0 2023-10-06T10:43:30.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.604+0000 7efd9904eb80 1 client.7422 shutdown 2023-10-06T10:43:30.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.604+0000 7efd9904eb80 1 -- 172.21.15.92:0/911250988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16ca5c10 msgr2=0x556a16c99fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.604+0000 7efd9904eb80 1 --2- 172.21.15.92:0/911250988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16ca5c10 0x556a16c99fc0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7efd80002250 tx=0x7efd80015000 comp rx=0 tx=0).stop 2023-10-06T10:43:30.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.604+0000 7efd9904eb80 1 -- 172.21.15.92:0/911250988 shutdown_connections 2023-10-06T10:43:30.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.604+0000 7efd9904eb80 1 --2- 172.21.15.92:0/911250988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a16ca5c10 0x556a16c99fc0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.604+0000 7efd9904eb80 1 --2- 172.21.15.92:0/911250988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556a16c5b9a0 0x556a16d0a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.604+0000 7efd9904eb80 1 --2- 172.21.15.92:0/911250988 >> [v2:172.21.15.124:6833/2971348516,v1:172.21.15.124:6835/2971348516] conn(0x556a16c9d7d0 0x556a16cbaf50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.604+0000 7efd9904eb80 1 --2- 172.21.15.92:0/911250988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556a16c84eb0 0x556a16c99a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.604+0000 7efd67fff700 0 client.7422 ms_handle_reset on v2:172.21.15.124:6833/2971348516 2023-10-06T10:43:30.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.604+0000 7efd9904eb80 1 -- 172.21.15.92:0/911250988 >> 172.21.15.92:0/911250988 conn(0x556a16c5d2c0 msgr2=0x556a16c9cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:30.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.605+0000 7efd9904eb80 1 -- 172.21.15.92:0/911250988 shutdown_connections 2023-10-06T10:43:30.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.605+0000 7efd9904eb80 1 -- 172.21.15.92:0/911250988 wait complete. 2023-10-06T10:43:30.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.605+0000 7efd9904eb80 20 client.7422 trim_cache size 0 max 16384 2023-10-06T10:43:30.608 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:43:30.608 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2023-10-06T10:43:30.609 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:43:30.609 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:43:30.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.605+0000 7efd8eaf4700 1 -- 172.21.15.92:0/2159981886 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efd6c066f30 msgr2=0x7efd6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.605+0000 7efd8eaf4700 1 --2- 172.21.15.92:0/2159981886 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efd6c066f30 0x7efd6c0693e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7efd880c6b40 tx=0x7efd84009120 comp rx=0 tx=0).stop 2023-10-06T10:43:30.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.605+0000 7efd8eaf4700 1 -- 172.21.15.92:0/2159981886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd880c7510 msgr2=0x7efd88152950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.605+0000 7efd8eaf4700 1 --2- 172.21.15.92:0/2159981886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd880c7510 0x7efd88152950 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7efd78006ac0 tx=0x7efd780091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.605+0000 7efd8eaf4700 1 -- 172.21.15.92:0/2159981886 shutdown_connections 2023-10-06T10:43:30.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.605+0000 7efd8eaf4700 1 --2- 172.21.15.92:0/2159981886 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efd6c066f30 0x7efd6c0693e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.605+0000 7efd8eaf4700 1 --2- 172.21.15.92:0/2159981886 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd88152e90 0x7efd880b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.605+0000 7efd8eaf4700 1 --2- 172.21.15.92:0/2159981886 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd8814c9d0 0x7efd881537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.605+0000 7efd8eaf4700 1 --2- 172.21.15.92:0/2159981886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd880c7510 0x7efd88152950 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.605+0000 7efd8eaf4700 1 -- 172.21.15.92:0/2159981886 >> 172.21.15.92:0/2159981886 conn(0x7efd88099520 msgr2=0x7efd88157e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:30.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.605+0000 7efd8eaf4700 1 -- 172.21.15.92:0/2159981886 shutdown_connections 2023-10-06T10:43:30.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.605+0000 7efd8eaf4700 1 -- 172.21.15.92:0/2159981886 wait complete. 2023-10-06T10:43:30.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:1 flush journal 2023-10-06T10:43:30.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.733+0000 7f7574754700 1 Processor -- start 2023-10-06T10:43:30.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.733+0000 7f7574754700 1 -- start start 2023-10-06T10:43:30.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.734+0000 7f7574754700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f756c151980 0x7f756c06b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.734+0000 7f7574754700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f756c06bde0 0x7f756c15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.734+0000 7f7574754700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f756c0b0500 con 0x7f756c15b730 2023-10-06T10:43:30.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.734+0000 7f7574754700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f756c06c260 con 0x7f756c151980 2023-10-06T10:43:30.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.734+0000 7f7574754700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f756c0afbd0 con 0x7f756c06bde0 2023-10-06T10:43:30.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.735+0000 7f75724f0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f756c151980 0x7f756c06b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.735+0000 7f7572cf1700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f756c15b730 0x7f756c06cdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53948/0 (socket says 172.21.15.92:53948) 2023-10-06T10:43:30.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.735+0000 7f7572cf1700 1 -- 172.21.15.92:0/2126168239 learned_addr learned my addr 172.21.15.92:0/2126168239 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:30.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.735+0000 7f75724f0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f756c151980 0x7f756c06b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60720/0 (socket says 172.21.15.92:60720) 2023-10-06T10:43:30.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.735+0000 7f75724f0700 1 -- 172.21.15.92:0/2126168239 >> v2:172.21.15.124:3300/0 conn(0x7f756c06bde0 msgr2=0x7f756c15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.735+0000 7f75724f0700 1 --2- 172.21.15.92:0/2126168239 >> v2:172.21.15.124:3300/0 conn(0x7f756c06bde0 0x7f756c15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.735+0000 7f75724f0700 1 -- 172.21.15.92:0/2126168239 >> v1:172.21.15.92:6789/0 conn(0x7f756c15b730 legacy=0x7f756c06cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.735+0000 7f75724f0700 1 -- 172.21.15.92:0/2126168239 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c0afeb0 con 0x7f756c151980 2023-10-06T10:43:30.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.735+0000 7f75714ee700 1 -- 172.21.15.92:0/2126168239 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4041130468 0 0) 0x7f756c0b0500 con 0x7f756c15b730 2023-10-06T10:43:30.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.735+0000 7f75724f0700 1 --2- 172.21.15.92:0/2126168239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f756c151980 0x7f756c06b8a0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f756c065060 tx=0x7f75680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d422d13f2609af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:30.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.735+0000 7f75714ee700 1 -- 172.21.15.92:0/2126168239 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756800b6b0 con 0x7f756c151980 2023-10-06T10:43:30.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.735+0000 7f75714ee700 1 -- 172.21.15.92:0/2126168239 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756800b810 con 0x7f756c151980 2023-10-06T10:43:30.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.735+0000 7f75714ee700 1 -- 172.21.15.92:0/2126168239 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756800bbb0 con 0x7f756c151980 2023-10-06T10:43:30.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.736+0000 7f7574754700 1 -- 172.21.15.92:0/2126168239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f756c151980 msgr2=0x7f756c06b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.736+0000 7f7574754700 1 --2- 172.21.15.92:0/2126168239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f756c151980 0x7f756c06b8a0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f756c065060 tx=0x7f75680091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.736+0000 7f7574754700 1 -- 172.21.15.92:0/2126168239 shutdown_connections 2023-10-06T10:43:30.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.736+0000 7f7574754700 1 --2- 172.21.15.92:0/2126168239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f756c151980 0x7f756c06b8a0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.736+0000 7f7574754700 1 --2- 172.21.15.92:0/2126168239 >> v2:172.21.15.124:3300/0 conn(0x7f756c06bde0 0x7f756c15b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.736+0000 7f7574754700 1 -- 172.21.15.92:0/2126168239 >> 172.21.15.92:0/2126168239 conn(0x7f756c0ce2f0 msgr2=0x7f756c0ce6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:30.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.736+0000 7f7574754700 1 -- 172.21.15.92:0/2126168239 shutdown_connections 2023-10-06T10:43:30.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.736+0000 7f7574754700 1 -- 172.21.15.92:0/2126168239 wait complete. 2023-10-06T10:43:30.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.737+0000 7f7574754700 1 Processor -- start 2023-10-06T10:43:30.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.737+0000 7f7574754700 1 -- start start 2023-10-06T10:43:30.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.737+0000 7f7574754700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f756c06bde0 0x7f756c16dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.737+0000 7f7574754700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f756c151980 0x7f756c14fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.737+0000 7f7574754700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f756c15b730 0x7f756c169df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.737+0000 7f7574754700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c16ade0 con 0x7f756c151980 2023-10-06T10:43:30.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.738+0000 7f7574754700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c16d8e0 con 0x7f756c06bde0 2023-10-06T10:43:30.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.738+0000 7f7574754700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756c16a820 con 0x7f756c15b730 2023-10-06T10:43:30.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.738+0000 7f7571cef700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f756c151980 0x7f756c14fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.738+0000 7f7571cef700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f756c151980 0x7f756c14fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60724/0 (socket says 172.21.15.92:60724) 2023-10-06T10:43:30.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.738+0000 7f7571cef700 1 -- 172.21.15.92:0/3777919318 learned_addr learned my addr 172.21.15.92:0/3777919318 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:30.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.738+0000 7f75724f0700 1 --2- 172.21.15.92:0/3777919318 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f756c06bde0 0x7f756c16dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.738+0000 7f7572cf1700 1 --2- 172.21.15.92:0/3777919318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f756c15b730 0x7f756c169df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.738+0000 7f7571cef700 1 -- 172.21.15.92:0/3777919318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f756c15b730 msgr2=0x7f756c169df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.738+0000 7f7571cef700 1 --2- 172.21.15.92:0/3777919318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f756c15b730 0x7f756c169df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.738+0000 7f7571cef700 1 -- 172.21.15.92:0/3777919318 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f756c06bde0 msgr2=0x7f756c16dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.738+0000 7f7571cef700 1 --2- 172.21.15.92:0/3777919318 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f756c06bde0 0x7f756c16dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.738+0000 7f7571cef700 1 -- 172.21.15.92:0/3777919318 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7568009420 con 0x7f756c151980 2023-10-06T10:43:30.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.738+0000 7f7571cef700 1 --2- 172.21.15.92:0/3777919318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f756c151980 0x7f756c14fd30 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f755c00b840 tx=0x7f755c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:30.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.738+0000 7f75627fc700 1 -- 172.21.15.92:0/3777919318 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c00f510 con 0x7f756c151980 2023-10-06T10:43:30.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.738+0000 7f75627fc700 1 -- 172.21.15.92:0/3777919318 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755c00f690 con 0x7f756c151980 2023-10-06T10:43:30.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.738+0000 7f75627fc700 1 -- 172.21.15.92:0/3777919318 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c00fa50 con 0x7f756c151980 2023-10-06T10:43:30.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.738+0000 7f7574754700 1 -- 172.21.15.92:0/3777919318 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756c16ab00 con 0x7f756c151980 2023-10-06T10:43:30.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.738+0000 7f7574754700 1 -- 172.21.15.92:0/3777919318 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756c16e2b0 con 0x7f756c151980 2023-10-06T10:43:30.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.740+0000 7f75627fc700 1 -- 172.21.15.92:0/3777919318 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f755c01a400 con 0x7f756c151980 2023-10-06T10:43:30.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.740+0000 7f75627fc700 1 --2- 172.21.15.92:0/3777919318 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7550066f30 0x7f75500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.740+0000 7f75627fc700 1 -- 172.21.15.92:0/3777919318 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f755c084350 con 0x7f756c151980 2023-10-06T10:43:30.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.740+0000 7f75724f0700 1 --2- 172.21.15.92:0/3777919318 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7550066f30 0x7f75500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.741+0000 7f75724f0700 1 --2- 172.21.15.92:0/3777919318 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7550066f30 0x7f75500693e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f75680093f0 tx=0x7f7568017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:30.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.742+0000 7f757da4cb80 1 Processor -- start 2023-10-06T10:43:30.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.742+0000 7f757da4cb80 1 -- start start 2023-10-06T10:43:30.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.743+0000 7f757da4cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c9752aae0 0x563c9752aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.743+0000 7f757da4cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563c974e69f0 0x563c974e35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.743+0000 7f757da4cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563c974e4c20 0x563c974e3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.743+0000 7f757da4cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c97463cd0 con 0x563c9752aae0 2023-10-06T10:43:30.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.743+0000 7f757da4cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c9743aac0 con 0x563c974e4c20 2023-10-06T10:43:30.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.743+0000 7f757da4cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563c974382c0 con 0x563c974e69f0 2023-10-06T10:43:30.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.743+0000 7f75724f0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c9752aae0 0x563c9752aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.743+0000 7f75724f0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c9752aae0 0x563c9752aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60728/0 (socket says 172.21.15.92:60728) 2023-10-06T10:43:30.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.743+0000 7f75724f0700 1 -- 172.21.15.92:0/852200959 learned_addr learned my addr 172.21.15.92:0/852200959 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:30.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.743+0000 7f7572cf1700 1 --2- 172.21.15.92:0/852200959 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563c974e4c20 0x563c974e3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.743+0000 7f7571cef700 1 --2- 172.21.15.92:0/852200959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563c974e69f0 0x563c974e35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.743+0000 7f75724f0700 1 -- 172.21.15.92:0/852200959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563c974e69f0 msgr2=0x563c974e35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.743+0000 7f75724f0700 1 --2- 172.21.15.92:0/852200959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563c974e69f0 0x563c974e35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.743+0000 7f75724f0700 1 -- 172.21.15.92:0/852200959 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563c974e4c20 msgr2=0x563c974e3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.743+0000 7f75724f0700 1 --2- 172.21.15.92:0/852200959 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563c974e4c20 0x563c974e3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.743+0000 7f75724f0700 1 -- 172.21.15.92:0/852200959 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c972ce2c0 con 0x563c9752aae0 2023-10-06T10:43:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.743+0000 7f75724f0700 1 --2- 172.21.15.92:0/852200959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c9752aae0 0x563c9752aeb0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f756800b0e0 tx=0x7f7568020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f16163789c1d833 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.743+0000 7f754f7fe700 1 -- 172.21.15.92:0/852200959 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7568020070 con 0x563c9752aae0 2023-10-06T10:43:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.743+0000 7f754f7fe700 1 -- 172.21.15.92:0/852200959 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756800bb60 con 0x563c9752aae0 2023-10-06T10:43:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.743+0000 7f754f7fe700 1 -- 172.21.15.92:0/852200959 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7568032480 con 0x563c9752aae0 2023-10-06T10:43:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.744+0000 7f757da4cb80 1 -- 172.21.15.92:0/852200959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c9752aae0 msgr2=0x563c9752aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.744+0000 7f757da4cb80 1 --2- 172.21.15.92:0/852200959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c9752aae0 0x563c9752aeb0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f756800b0e0 tx=0x7f7568020000 comp rx=0 tx=0).stop 2023-10-06T10:43:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.744+0000 7f757da4cb80 1 -- 172.21.15.92:0/852200959 shutdown_connections 2023-10-06T10:43:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.744+0000 7f757da4cb80 1 --2- 172.21.15.92:0/852200959 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563c974e4c20 0x563c974e3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.744+0000 7f757da4cb80 1 --2- 172.21.15.92:0/852200959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c9752aae0 0x563c9752aeb0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.744+0000 7f757da4cb80 1 --2- 172.21.15.92:0/852200959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563c974e69f0 0x563c974e35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.744+0000 7f757da4cb80 1 -- 172.21.15.92:0/852200959 >> 172.21.15.92:0/852200959 conn(0x563c974e50c0 msgr2=0x563c974e5490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.744+0000 7f757da4cb80 1 -- 172.21.15.92:0/852200959 shutdown_connections 2023-10-06T10:43:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.744+0000 7f757da4cb80 1 -- 172.21.15.92:0/852200959 wait complete. 2023-10-06T10:43:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.745+0000 7f757da4cb80 1 -- start start 2023-10-06T10:43:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.745+0000 7f757da4cb80 1 Processor -- start 2023-10-06T10:43:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f757da4cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c974e4c20 0x563c97525a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f757da4cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563c974e69f0 0x563c9750db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f757da4cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563c9752aae0 0x563c97521a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f757da4cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c974340b0 con 0x563c974e4c20 2023-10-06T10:43:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f757da4cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c973df8e0 con 0x563c9752aae0 2023-10-06T10:43:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f757da4cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563c973faef0 con 0x563c974e69f0 2023-10-06T10:43:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f757da4cb80 10 client.0 _ll_register_callbacks cb 0x563c972d0f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f75724f0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c974e4c20 0x563c97525a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f75724f0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c974e4c20 0x563c97525a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60740/0 (socket says 172.21.15.92:60740) 2023-10-06T10:43:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f75724f0700 1 -- 172.21.15.92:0/3113481634 learned_addr learned my addr 172.21.15.92:0/3113481634 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f7571cef700 1 --2- 172.21.15.92:0/3113481634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563c974e69f0 0x563c9750db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f7572cf1700 1 --2- 172.21.15.92:0/3113481634 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563c9752aae0 0x563c97521a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f75724f0700 1 -- 172.21.15.92:0/3113481634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563c974e69f0 msgr2=0x563c9750db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f75724f0700 1 --2- 172.21.15.92:0/3113481634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563c974e69f0 0x563c9750db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f75724f0700 1 -- 172.21.15.92:0/3113481634 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563c9752aae0 msgr2=0x563c97521a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f75724f0700 1 --2- 172.21.15.92:0/3113481634 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563c9752aae0 0x563c97521a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f75724f0700 1 -- 172.21.15.92:0/3113481634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756c0afeb0 con 0x563c974e4c20 2023-10-06T10:43:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f7572cf1700 1 --2- 172.21.15.92:0/3113481634 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563c9752aae0 0x563c97521a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f7571cef700 1 --2- 172.21.15.92:0/3113481634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563c974e69f0 0x563c9750db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f75724f0700 1 --2- 172.21.15.92:0/3113481634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c974e4c20 0x563c97525a80 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f75680093c0 tx=0x7f756802b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7541c3c68c5d7d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:30.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f754cff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:30.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f754cff9700 1 -- 172.21.15.92:0/3113481634 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7568020050 con 0x563c974e4c20 2023-10-06T10:43:30.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f754cff9700 1 -- 172.21.15.92:0/3113481634 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7568032480 con 0x563c974e4c20 2023-10-06T10:43:30.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f757da4cb80 1 -- 172.21.15.92:0/3113481634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c974d86e0 con 0x563c974e4c20 2023-10-06T10:43:30.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.746+0000 7f754cff9700 1 -- 172.21.15.92:0/3113481634 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f756801a840 con 0x563c974e4c20 2023-10-06T10:43:30.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.747+0000 7f757da4cb80 10 client.7437 fetch_fsmap learned FSMap version 74 2023-10-06T10:43:30.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.747+0000 7f757da4cb80 1 -- 172.21.15.92:0/3113481634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x563c97271030 con 0x563c974e4c20 2023-10-06T10:43:30.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.747+0000 7f754cff9700 1 -- 172.21.15.92:0/3113481634 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563c974d86e0 con 0x563c974e4c20 2023-10-06T10:43:30.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.747+0000 7f754cff9700 1 -- 172.21.15.92:0/3113481634 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 74) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f756801a9a0 con 0x563c974e4c20 2023-10-06T10:43:30.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.747+0000 7f757da4cb80 10 client.7437 fetch_fsmap finished waiting for FSMap version 74 2023-10-06T10:43:30.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.747+0000 7f757da4cb80 10 client.7437 resolve_mds: resolved 5:1 to role '5:1' aka daemon mds.c 2023-10-06T10:43:30.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.747+0000 7f757da4cb80 20 client.7437 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:30.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.747+0000 7f757da4cb80 1 --2- 172.21.15.92:0/3113481634 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x563c9750e330 0x563c975271d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.747+0000 7f757da4cb80 4 client.7437 mds_command: new command op to 7149 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:43:30.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.747+0000 7f757da4cb80 1 -- 172.21.15.92:0/3113481634 --> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c974b5610 con 0x563c9750e330 2023-10-06T10:43:30.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.747+0000 7f7571cef700 1 --2- 172.21.15.92:0/3113481634 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x563c9750e330 0x563c975271d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.747+0000 7f754cff9700 10 client.7437 ms_handle_connect on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:30.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.747+0000 7f7571cef700 1 --2- 172.21.15.92:0/3113481634 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x563c9750e330 0x563c975271d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21f151db4ca5606b server_cookie=a1c6f7ce94550053 in_seq=0 out_seq=0 2023-10-06T10:43:30.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.748+0000 7f754cff9700 1 -- 172.21.15.92:0/3113481634 <== mds.1 v2:172.21.15.92:6834/3268908749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x563c974b5610 con 0x563c9750e330 2023-10-06T10:43:30.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.748+0000 7f754cff9700 10 client.7437 handle_command_reply: tid=0 2023-10-06T10:43:30.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.748+0000 7f757da4cb80 1 client.7437 shutdown 2023-10-06T10:43:30.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.748+0000 7f757da4cb80 1 -- 172.21.15.92:0/3113481634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c974e4c20 msgr2=0x563c97525a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.748+0000 7f757da4cb80 1 --2- 172.21.15.92:0/3113481634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c974e4c20 0x563c97525a80 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f75680093c0 tx=0x7f756802b040 comp rx=0 tx=0).stop 2023-10-06T10:43:30.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.748+0000 7f757da4cb80 1 -- 172.21.15.92:0/3113481634 shutdown_connections 2023-10-06T10:43:30.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.748+0000 7f757da4cb80 1 --2- 172.21.15.92:0/3113481634 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x563c9750e330 0x563c975271d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.748+0000 7f757da4cb80 1 --2- 172.21.15.92:0/3113481634 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563c9752aae0 0x563c97521a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.748+0000 7f754cff9700 0 client.7437 ms_handle_reset on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:30.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.748+0000 7f757da4cb80 1 --2- 172.21.15.92:0/3113481634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c974e4c20 0x563c97525a80 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.748+0000 7f757da4cb80 1 --2- 172.21.15.92:0/3113481634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563c974e69f0 0x563c9750db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.748+0000 7f757da4cb80 1 -- 172.21.15.92:0/3113481634 >> 172.21.15.92:0/3113481634 conn(0x563c974e50c0 msgr2=0x563c975286a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:30.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.748+0000 7f757da4cb80 1 -- 172.21.15.92:0/3113481634 shutdown_connections 2023-10-06T10:43:30.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.748+0000 7f757da4cb80 1 -- 172.21.15.92:0/3113481634 wait complete. 2023-10-06T10:43:30.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.748+0000 7f757da4cb80 20 client.7437 trim_cache size 0 max 16384 2023-10-06T10:43:30.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.762+0000 7f757da4cb80 1 Processor -- start 2023-10-06T10:43:30.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.762+0000 7f757da4cb80 1 -- start start 2023-10-06T10:43:30.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.763+0000 7f757da4cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563c974e4c20 0x563c97512930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.763+0000 7f757da4cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563c974e69f0 0x563c97512e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.763+0000 7f757da4cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c9752aae0 0x563c975133b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.763+0000 7f757da4cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c972b6e80 con 0x563c9752aae0 2023-10-06T10:43:30.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.763+0000 7f757da4cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c974de3a0 con 0x563c974e69f0 2023-10-06T10:43:30.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.763+0000 7f757da4cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563c97522110 con 0x563c974e4c20 2023-10-06T10:43:30.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.763+0000 7f7572cf1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c9752aae0 0x563c975133b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.763+0000 7f7572cf1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c9752aae0 0x563c975133b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60744/0 (socket says 172.21.15.92:60744) 2023-10-06T10:43:30.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.763+0000 7f7572cf1700 1 -- 172.21.15.92:0/3593291162 learned_addr learned my addr 172.21.15.92:0/3593291162 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:30.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.763+0000 7f7571cef700 1 --2- 172.21.15.92:0/3593291162 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563c974e69f0 0x563c97512e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.763+0000 7f75724f0700 1 --2- 172.21.15.92:0/3593291162 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563c974e4c20 0x563c97512930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.763+0000 7f7572cf1700 1 -- 172.21.15.92:0/3593291162 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563c974e4c20 msgr2=0x563c97512930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.763+0000 7f7572cf1700 1 --2- 172.21.15.92:0/3593291162 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563c974e4c20 0x563c97512930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.763+0000 7f7572cf1700 1 -- 172.21.15.92:0/3593291162 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563c974e69f0 msgr2=0x563c97512e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.763+0000 7f7572cf1700 1 --2- 172.21.15.92:0/3593291162 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563c974e69f0 0x563c97512e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.763+0000 7f7572cf1700 1 -- 172.21.15.92:0/3593291162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c974027d0 con 0x563c9752aae0 2023-10-06T10:43:30.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.763+0000 7f7572cf1700 1 --2- 172.21.15.92:0/3593291162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c9752aae0 0x563c975133b0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f7564001560 tx=0x7f7564009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f20671d1422489c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:30.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.763+0000 7f754ffff700 1 -- 172.21.15.92:0/3593291162 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564007640 con 0x563c9752aae0 2023-10-06T10:43:30.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.763+0000 7f754ffff700 1 -- 172.21.15.92:0/3593291162 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75640077c0 con 0x563c9752aae0 2023-10-06T10:43:30.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.763+0000 7f754ffff700 1 -- 172.21.15.92:0/3593291162 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564015410 con 0x563c9752aae0 2023-10-06T10:43:30.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.764+0000 7f757da4cb80 1 -- 172.21.15.92:0/3593291162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c9752aae0 msgr2=0x563c975133b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.764+0000 7f757da4cb80 1 --2- 172.21.15.92:0/3593291162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c9752aae0 0x563c975133b0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f7564001560 tx=0x7f7564009040 comp rx=0 tx=0).stop 2023-10-06T10:43:30.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.764+0000 7f757da4cb80 1 -- 172.21.15.92:0/3593291162 shutdown_connections 2023-10-06T10:43:30.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.764+0000 7f757da4cb80 1 --2- 172.21.15.92:0/3593291162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c9752aae0 0x563c975133b0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.764+0000 7f757da4cb80 1 --2- 172.21.15.92:0/3593291162 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563c974e4c20 0x563c97512930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.764+0000 7f757da4cb80 1 --2- 172.21.15.92:0/3593291162 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563c974e69f0 0x563c97512e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.764+0000 7f757da4cb80 1 -- 172.21.15.92:0/3593291162 >> 172.21.15.92:0/3593291162 conn(0x563c974e50c0 msgr2=0x563c97524310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:30.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.764+0000 7f757da4cb80 1 -- 172.21.15.92:0/3593291162 shutdown_connections 2023-10-06T10:43:30.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.764+0000 7f757da4cb80 1 -- 172.21.15.92:0/3593291162 wait complete. 2023-10-06T10:43:30.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.764+0000 7f757da4cb80 1 -- start start 2023-10-06T10:43:30.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.764+0000 7f757da4cb80 1 Processor -- start 2023-10-06T10:43:30.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.765+0000 7f757da4cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563c974e4c20 0x563c97524310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.765+0000 7f757da4cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c974e69f0 0x563c974e04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.765+0000 7f757da4cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563c9752aae0 0x563c9753c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.765+0000 7f757da4cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c97524a40 con 0x563c974e69f0 2023-10-06T10:43:30.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.765+0000 7f757da4cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c9750cce0 con 0x563c9752aae0 2023-10-06T10:43:30.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.765+0000 7f757da4cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563c974e0fc0 con 0x563c974e4c20 2023-10-06T10:43:30.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.765+0000 7f757da4cb80 10 client.0 _ll_register_callbacks cb 0x563c972d0f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:30.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.765+0000 7f7571cef700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c974e69f0 0x563c974e04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.765+0000 7f7571cef700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c974e69f0 0x563c974e04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60748/0 (socket says 172.21.15.92:60748) 2023-10-06T10:43:30.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.765+0000 7f7571cef700 1 -- 172.21.15.92:0/4111009734 learned_addr learned my addr 172.21.15.92:0/4111009734 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:30.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.765+0000 7f7572cf1700 1 --2- 172.21.15.92:0/4111009734 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563c9752aae0 0x563c9753c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.765+0000 7f75724f0700 1 --2- 172.21.15.92:0/4111009734 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563c974e4c20 0x563c97524310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.766+0000 7f7571cef700 1 -- 172.21.15.92:0/4111009734 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563c974e4c20 msgr2=0x563c97524310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.766+0000 7f7571cef700 1 --2- 172.21.15.92:0/4111009734 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563c974e4c20 0x563c97524310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.766+0000 7f7571cef700 1 -- 172.21.15.92:0/4111009734 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563c9752aae0 msgr2=0x563c9753c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.766+0000 7f7571cef700 1 --2- 172.21.15.92:0/4111009734 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563c9752aae0 0x563c9753c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.766+0000 7f7571cef700 1 -- 172.21.15.92:0/4111009734 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7564001bb0 con 0x563c974e69f0 2023-10-06T10:43:30.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.766+0000 7f7571cef700 1 --2- 172.21.15.92:0/4111009734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c974e69f0 0x563c974e04d0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f755c00bf90 tx=0x7f755c01c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d3e71568842927 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:30.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.766+0000 7f754cff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:30.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.766+0000 7f754cff9700 1 -- 172.21.15.92:0/4111009734 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c0822c0 con 0x563c974e69f0 2023-10-06T10:43:30.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.766+0000 7f754cff9700 1 -- 172.21.15.92:0/4111009734 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f755c053400 con 0x563c974e69f0 2023-10-06T10:43:30.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.766+0000 7f757da4cb80 1 -- 172.21.15.92:0/4111009734 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f756800bb60 con 0x563c974e69f0 2023-10-06T10:43:30.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.766+0000 7f754cff9700 1 -- 172.21.15.92:0/4111009734 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f755c00fdc0 con 0x563c974e69f0 2023-10-06T10:43:30.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.766+0000 7f757da4cb80 10 client.7443 fetch_fsmap learned FSMap version 74 2023-10-06T10:43:30.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.766+0000 7f757da4cb80 1 -- 172.21.15.92:0/4111009734 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x563c973e94c0 con 0x563c974e69f0 2023-10-06T10:43:30.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.766+0000 7f754cff9700 1 -- 172.21.15.92:0/4111009734 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756800bb60 con 0x563c974e69f0 2023-10-06T10:43:30.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.766+0000 7f754cff9700 1 -- 172.21.15.92:0/4111009734 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 74) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f755c00d290 con 0x563c974e69f0 2023-10-06T10:43:30.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.766+0000 7f757da4cb80 10 client.7443 fetch_fsmap finished waiting for FSMap version 74 2023-10-06T10:43:30.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.766+0000 7f757da4cb80 10 client.7443 resolve_mds: resolved 5:1 to role '5:1' aka daemon mds.c 2023-10-06T10:43:30.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.766+0000 7f757da4cb80 20 client.7443 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:30.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.766+0000 7f757da4cb80 1 --2- 172.21.15.92:0/4111009734 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x563c975892c0 0x563c97543680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.766+0000 7f757da4cb80 4 client.7443 mds_command: new command op to 7149 tid=0[{"prefix": "flush journal"}] 2023-10-06T10:43:30.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.766+0000 7f757da4cb80 1 -- 172.21.15.92:0/4111009734 --> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563c973d6e50 con 0x563c975892c0 2023-10-06T10:43:30.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.766+0000 7f75724f0700 1 --2- 172.21.15.92:0/4111009734 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x563c975892c0 0x563c97543680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.767+0000 7f754cff9700 10 client.7443 ms_handle_connect on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:30.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.767+0000 7f75724f0700 1 --2- 172.21.15.92:0/4111009734 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x563c975892c0 0x563c97543680 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dcbcb8fee13b699f server_cookie=21e05e381b128a3c in_seq=0 out_seq=0 2023-10-06T10:43:30.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.774+0000 7f754cff9700 1 -- 172.21.15.92:0/4111009734 <== mds.1 v2:172.21.15.92:6834/3268908749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563c973d6e50 con 0x563c975892c0 2023-10-06T10:43:30.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.774+0000 7f754cff9700 10 client.7443 handle_command_reply: tid=0 2023-10-06T10:43:30.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.774+0000 7f757da4cb80 1 client.7443 shutdown 2023-10-06T10:43:30.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.774+0000 7f757da4cb80 1 -- 172.21.15.92:0/4111009734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c974e69f0 msgr2=0x563c974e04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.774+0000 7f757da4cb80 1 --2- 172.21.15.92:0/4111009734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c974e69f0 0x563c974e04d0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f755c00bf90 tx=0x7f755c01c040 comp rx=0 tx=0).stop 2023-10-06T10:43:30.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.774+0000 7f757da4cb80 1 -- 172.21.15.92:0/4111009734 shutdown_connections 2023-10-06T10:43:30.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.774+0000 7f757da4cb80 1 --2- 172.21.15.92:0/4111009734 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x563c975892c0 0x563c97543680 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.774+0000 7f757da4cb80 1 --2- 172.21.15.92:0/4111009734 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563c9752aae0 0x563c9753c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.774+0000 7f757da4cb80 1 --2- 172.21.15.92:0/4111009734 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563c974e4c20 0x563c97524310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.774+0000 7f754cff9700 0 client.7443 ms_handle_reset on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:30.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.774+0000 7f757da4cb80 1 --2- 172.21.15.92:0/4111009734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c974e69f0 0x563c974e04d0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.774+0000 7f757da4cb80 1 -- 172.21.15.92:0/4111009734 >> 172.21.15.92:0/4111009734 conn(0x563c974e50c0 msgr2=0x563c975293d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:30.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.774+0000 7f757da4cb80 1 -- 172.21.15.92:0/4111009734 shutdown_connections 2023-10-06T10:43:30.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.774+0000 7f757da4cb80 1 -- 172.21.15.92:0/4111009734 wait complete. 2023-10-06T10:43:30.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.775+0000 7f757da4cb80 20 client.7443 trim_cache size 0 max 16384 2023-10-06T10:43:30.779 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:43:30.779 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2023-10-06T10:43:30.779 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:43:30.780 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:43:30.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.775+0000 7f7574754700 1 -- 172.21.15.92:0/3777919318 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7550066f30 msgr2=0x7f75500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.775+0000 7f7574754700 1 --2- 172.21.15.92:0/3777919318 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7550066f30 0x7f75500693e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f75680093f0 tx=0x7f7568017000 comp rx=0 tx=0).stop 2023-10-06T10:43:30.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.775+0000 7f7574754700 1 -- 172.21.15.92:0/3777919318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f756c151980 msgr2=0x7f756c14fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.775+0000 7f7574754700 1 --2- 172.21.15.92:0/3777919318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f756c151980 0x7f756c14fd30 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f755c00b840 tx=0x7f755c0091d0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.775+0000 7f7574754700 1 -- 172.21.15.92:0/3777919318 shutdown_connections 2023-10-06T10:43:30.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.775+0000 7f7574754700 1 --2- 172.21.15.92:0/3777919318 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7550066f30 0x7f75500693e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.775+0000 7f7574754700 1 --2- 172.21.15.92:0/3777919318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f756c15b730 0x7f756c169df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.775+0000 7f7574754700 1 --2- 172.21.15.92:0/3777919318 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f756c06bde0 0x7f756c16dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.775+0000 7f7574754700 1 --2- 172.21.15.92:0/3777919318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f756c151980 0x7f756c14fd30 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.775+0000 7f7574754700 1 -- 172.21.15.92:0/3777919318 >> 172.21.15.92:0/3777919318 conn(0x7f756c0ce2f0 msgr2=0x7f756c1590f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:30.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.775+0000 7f7574754700 1 -- 172.21.15.92:0/3777919318 shutdown_connections 2023-10-06T10:43:30.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.775+0000 7f7574754700 1 -- 172.21.15.92:0/3777919318 wait complete. 2023-10-06T10:43:30.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:2 flush journal 2023-10-06T10:43:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.902+0000 7fe258a0b700 1 Processor -- start 2023-10-06T10:43:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.902+0000 7fe258a0b700 1 -- start start 2023-10-06T10:43:30.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.903+0000 7fe258a0b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2540b2620 0x7fe2540cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.903+0000 7fe258a0b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe2540cf1b0 0x7fe2540cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.903+0000 7fe258a0b700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2540cc970 con 0x7fe2540b29f0 2023-10-06T10:43:30.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.903+0000 7fe258a0b700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2540cbd60 con 0x7fe2540cf1b0 2023-10-06T10:43:30.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.903+0000 7fe258a0b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2540cc040 con 0x7fe2540b2620 2023-10-06T10:43:30.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.904+0000 7fe253fff700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe2540cf1b0 0x7fe2540cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.904+0000 7fe2537fe700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2540b2620 0x7fe2540cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.904+0000 7fe252ffd700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fe2540b29f0 0x7fe2540ceaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:35248/0 (socket says 172.21.15.92:35248) 2023-10-06T10:43:30.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.904+0000 7fe252ffd700 1 -- 172.21.15.92:0/3553562286 learned_addr learned my addr 172.21.15.92:0/3553562286 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:30.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.904+0000 7fe2527fc700 1 -- 172.21.15.92:0/3553562286 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2513369270 0 0) 0x7fe2540cc970 con 0x7fe2540b29f0 2023-10-06T10:43:30.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.904+0000 7fe2527fc700 1 -- 172.21.15.92:0/3553562286 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe23c003570 con 0x7fe2540b29f0 2023-10-06T10:43:30.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.904+0000 7fe2537fe700 1 -- 172.21.15.92:0/3553562286 >> v2:172.21.15.124:3300/0 conn(0x7fe2540cf1b0 msgr2=0x7fe2540cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.904+0000 7fe2537fe700 1 --2- 172.21.15.92:0/3553562286 >> v2:172.21.15.124:3300/0 conn(0x7fe2540cf1b0 0x7fe2540cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.904+0000 7fe2537fe700 1 -- 172.21.15.92:0/3553562286 >> v1:172.21.15.124:6789/0 conn(0x7fe2540b29f0 legacy=0x7fe2540ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.904+0000 7fe2537fe700 1 -- 172.21.15.92:0/3553562286 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2540cc320 con 0x7fe2540b2620 2023-10-06T10:43:30.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.904+0000 7fe253fff700 1 --2- 172.21.15.92:0/3553562286 >> v2:172.21.15.124:3300/0 conn(0x7fe2540cf1b0 0x7fe2540cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:43:30.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.904+0000 7fe2537fe700 1 --2- 172.21.15.92:0/3553562286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2540b2620 0x7fe2540cccb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe244009eb0 tx=0x7fe244009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff75e14aaa6058db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:30.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.904+0000 7fe2527fc700 1 -- 172.21.15.92:0/3553562286 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24400e070 con 0x7fe2540b2620 2023-10-06T10:43:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.904+0000 7fe2527fc700 1 -- 172.21.15.92:0/3553562286 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24400bb80 con 0x7fe2540b2620 2023-10-06T10:43:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.904+0000 7fe2527fc700 1 -- 172.21.15.92:0/3553562286 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe244012430 con 0x7fe2540b2620 2023-10-06T10:43:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.905+0000 7fe258a0b700 1 -- 172.21.15.92:0/3553562286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2540b2620 msgr2=0x7fe2540cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.905+0000 7fe258a0b700 1 --2- 172.21.15.92:0/3553562286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2540b2620 0x7fe2540cccb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe244009eb0 tx=0x7fe244009780 comp rx=0 tx=0).stop 2023-10-06T10:43:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.905+0000 7fe258a0b700 1 -- 172.21.15.92:0/3553562286 shutdown_connections 2023-10-06T10:43:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.905+0000 7fe258a0b700 1 --2- 172.21.15.92:0/3553562286 >> v2:172.21.15.124:3300/0 conn(0x7fe2540cf1b0 0x7fe2540cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.905+0000 7fe258a0b700 1 --2- 172.21.15.92:0/3553562286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2540b2620 0x7fe2540cccb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.905+0000 7fe258a0b700 1 -- 172.21.15.92:0/3553562286 >> 172.21.15.92:0/3553562286 conn(0x7fe254099520 msgr2=0x7fe25406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.905+0000 7fe258a0b700 1 -- 172.21.15.92:0/3553562286 shutdown_connections 2023-10-06T10:43:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.905+0000 7fe258a0b700 1 -- 172.21.15.92:0/3553562286 wait complete. 2023-10-06T10:43:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.906+0000 7fe258a0b700 1 Processor -- start 2023-10-06T10:43:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.906+0000 7fe258a0b700 1 -- start start 2023-10-06T10:43:30.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.907+0000 7fe258a0b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2540b2620 0x7fe25415f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.907+0000 7fe258a0b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2540b29f0 0x7fe2540b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.907+0000 7fe258a0b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2540cf1b0 0x7fe2540b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.907+0000 7fe258a0b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2540c75c0 con 0x7fe2540cf1b0 2023-10-06T10:43:30.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.907+0000 7fe258a0b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2540c7b80 con 0x7fe2540b2620 2023-10-06T10:43:30.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.907+0000 7fe258a0b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2540b5c70 con 0x7fe2540b29f0 2023-10-06T10:43:30.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.907+0000 7fe253fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2540cf1b0 0x7fe2540b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.907+0000 7fe253fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2540cf1b0 0x7fe2540b4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60758/0 (socket says 172.21.15.92:60758) 2023-10-06T10:43:30.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.907+0000 7fe253fff700 1 -- 172.21.15.92:0/1019355308 learned_addr learned my addr 172.21.15.92:0/1019355308 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:30.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.907+0000 7fe252ffd700 1 --2- 172.21.15.92:0/1019355308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2540b29f0 0x7fe2540b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.907+0000 7fe2537fe700 1 --2- 172.21.15.92:0/1019355308 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2540b2620 0x7fe25415f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.907+0000 7fe253fff700 1 -- 172.21.15.92:0/1019355308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2540b29f0 msgr2=0x7fe2540b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.907+0000 7fe253fff700 1 --2- 172.21.15.92:0/1019355308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2540b29f0 0x7fe2540b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.907+0000 7fe253fff700 1 -- 172.21.15.92:0/1019355308 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2540b2620 msgr2=0x7fe25415f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.907+0000 7fe253fff700 1 --2- 172.21.15.92:0/1019355308 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2540b2620 0x7fe25415f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.907+0000 7fe253fff700 1 -- 172.21.15.92:0/1019355308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24c009420 con 0x7fe2540cf1b0 2023-10-06T10:43:30.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.908+0000 7fe253fff700 1 --2- 172.21.15.92:0/1019355308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2540cf1b0 0x7fe2540b4d00 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00ec80 tx=0x7fe24c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:30.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.908+0000 7fe23b7fe700 1 -- 172.21.15.92:0/1019355308 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c00c9d0 con 0x7fe2540cf1b0 2023-10-06T10:43:30.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.908+0000 7fe23b7fe700 1 -- 172.21.15.92:0/1019355308 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24c00cb30 con 0x7fe2540cf1b0 2023-10-06T10:43:30.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.908+0000 7fe258a0b700 1 -- 172.21.15.92:0/1019355308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe244009a20 con 0x7fe2540cf1b0 2023-10-06T10:43:30.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.908+0000 7fe23b7fe700 1 -- 172.21.15.92:0/1019355308 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c013470 con 0x7fe2540cf1b0 2023-10-06T10:43:30.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.908+0000 7fe258a0b700 1 -- 172.21.15.92:0/1019355308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2540c7810 con 0x7fe2540cf1b0 2023-10-06T10:43:30.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.909+0000 7fe23b7fe700 1 -- 172.21.15.92:0/1019355308 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe24c013860 con 0x7fe2540cf1b0 2023-10-06T10:43:30.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.909+0000 7fe23b7fe700 1 --2- 172.21.15.92:0/1019355308 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe240066f30 0x7fe2400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.910+0000 7fe23b7fe700 1 -- 172.21.15.92:0/1019355308 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe24c084900 con 0x7fe2540cf1b0 2023-10-06T10:43:30.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.910+0000 7fe2537fe700 1 --2- 172.21.15.92:0/1019355308 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe240066f30 0x7fe2400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.910+0000 7fe2537fe700 1 --2- 172.21.15.92:0/1019355308 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe240066f30 0x7fe2400693e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe2440097c0 tx=0x7fe244019000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:30.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.911+0000 7fe262f65b80 1 Processor -- start 2023-10-06T10:43:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.912+0000 7fe262f65b80 1 -- start start 2023-10-06T10:43:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.912+0000 7fe262f65b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7582159a0 0x55c758215d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.912+0000 7fe262f65b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75825fc10 0x55c75823e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.912+0000 7fe262f65b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c75823eeb0 0x55c7582119e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.912+0000 7fe262f65b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c758195cd0 con 0x55c75825fc10 2023-10-06T10:43:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.912+0000 7fe262f65b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c75816cac0 con 0x55c75823eeb0 2023-10-06T10:43:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.912+0000 7fe262f65b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c75816a2c0 con 0x55c7582159a0 2023-10-06T10:43:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.912+0000 7fe252ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75825fc10 0x55c75823e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.912+0000 7fe252ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75825fc10 0x55c75823e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60768/0 (socket says 172.21.15.92:60768) 2023-10-06T10:43:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.912+0000 7fe252ffd700 1 -- 172.21.15.92:0/2622070223 learned_addr learned my addr 172.21.15.92:0/2622070223 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.913+0000 7fe2537fe700 1 --2- 172.21.15.92:0/2622070223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7582159a0 0x55c758215d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.913+0000 7fe253fff700 1 --2- 172.21.15.92:0/2622070223 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c75823eeb0 0x55c7582119e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.913+0000 7fe252ffd700 1 -- 172.21.15.92:0/2622070223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7582159a0 msgr2=0x55c758215d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.913+0000 7fe252ffd700 1 --2- 172.21.15.92:0/2622070223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7582159a0 0x55c758215d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.913+0000 7fe252ffd700 1 -- 172.21.15.92:0/2622070223 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c75823eeb0 msgr2=0x55c7582119e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.913+0000 7fe252ffd700 1 --2- 172.21.15.92:0/2622070223 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c75823eeb0 0x55c7582119e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.913+0000 7fe252ffd700 1 -- 172.21.15.92:0/2622070223 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7580002c0 con 0x55c75825fc10 2023-10-06T10:43:30.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.913+0000 7fe252ffd700 1 --2- 172.21.15.92:0/2622070223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75825fc10 0x55c75823e970 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fe248001640 tx=0x7fe248009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c754e9c3122db852 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:30.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.913+0000 7fe238ff9700 1 -- 172.21.15.92:0/2622070223 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24800b070 con 0x55c75825fc10 2023-10-06T10:43:30.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.913+0000 7fe238ff9700 1 -- 172.21.15.92:0/2622070223 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe248005b10 con 0x55c75825fc10 2023-10-06T10:43:30.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.913+0000 7fe238ff9700 1 -- 172.21.15.92:0/2622070223 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2480070d0 con 0x55c75825fc10 2023-10-06T10:43:30.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.914+0000 7fe262f65b80 1 -- 172.21.15.92:0/2622070223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75825fc10 msgr2=0x55c75823e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.914+0000 7fe262f65b80 1 --2- 172.21.15.92:0/2622070223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75825fc10 0x55c75823e970 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fe248001640 tx=0x7fe248009040 comp rx=0 tx=0).stop 2023-10-06T10:43:30.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.914+0000 7fe262f65b80 1 -- 172.21.15.92:0/2622070223 shutdown_connections 2023-10-06T10:43:30.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.914+0000 7fe262f65b80 1 --2- 172.21.15.92:0/2622070223 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c75823eeb0 0x55c7582119e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.914+0000 7fe262f65b80 1 --2- 172.21.15.92:0/2622070223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7582159a0 0x55c758215d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.914+0000 7fe262f65b80 1 --2- 172.21.15.92:0/2622070223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75825fc10 0x55c75823e970 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.914+0000 7fe262f65b80 1 -- 172.21.15.92:0/2622070223 >> 172.21.15.92:0/2622070223 conn(0x55c7582172c0 msgr2=0x55c758217690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:30.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.914+0000 7fe262f65b80 1 -- 172.21.15.92:0/2622070223 shutdown_connections 2023-10-06T10:43:30.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.914+0000 7fe262f65b80 1 -- 172.21.15.92:0/2622070223 wait complete. 2023-10-06T10:43:30.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.914+0000 7fe262f65b80 1 -- start start 2023-10-06T10:43:30.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.914+0000 7fe262f65b80 1 Processor -- start 2023-10-06T10:43:30.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.915+0000 7fe262f65b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7582159a0 0x55c758253f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.915+0000 7fe262f65b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c75823eeb0 0x55c758255de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.915+0000 7fe262f65b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75825fc10 0x55c75826e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.915+0000 7fe262f65b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7581660b0 con 0x55c75825fc10 2023-10-06T10:43:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.915+0000 7fe262f65b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7581118e0 con 0x55c75823eeb0 2023-10-06T10:43:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.915+0000 7fe262f65b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c75812cef0 con 0x55c7582159a0 2023-10-06T10:43:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.915+0000 7fe262f65b80 10 client.0 _ll_register_callbacks cb 0x55c758002f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.915+0000 7fe253fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75825fc10 0x55c75826e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.915+0000 7fe253fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75825fc10 0x55c75826e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60772/0 (socket says 172.21.15.92:60772) 2023-10-06T10:43:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.915+0000 7fe253fff700 1 -- 172.21.15.92:0/2435859974 learned_addr learned my addr 172.21.15.92:0/2435859974 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.915+0000 7fe2537fe700 1 --2- 172.21.15.92:0/2435859974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7582159a0 0x55c758253f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.915+0000 7fe252ffd700 1 --2- 172.21.15.92:0/2435859974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c75823eeb0 0x55c758255de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.916+0000 7fe253fff700 1 -- 172.21.15.92:0/2435859974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7582159a0 msgr2=0x55c758253f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.916+0000 7fe253fff700 1 --2- 172.21.15.92:0/2435859974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7582159a0 0x55c758253f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.916+0000 7fe253fff700 1 -- 172.21.15.92:0/2435859974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c75823eeb0 msgr2=0x55c758255de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.916+0000 7fe253fff700 1 --2- 172.21.15.92:0/2435859974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c75823eeb0 0x55c758255de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.916+0000 7fe253fff700 1 -- 172.21.15.92:0/2435859974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe23c003570 con 0x55c75825fc10 2023-10-06T10:43:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.916+0000 7fe253fff700 1 --2- 172.21.15.92:0/2435859974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75825fc10 0x55c75826e770 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fe24c051a90 tx=0x7fe24c082ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e703c8fd36f54f3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.916+0000 7fe231ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.916+0000 7fe231ffb700 1 -- 172.21.15.92:0/2435859974 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c05c530 con 0x55c75825fc10 2023-10-06T10:43:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.916+0000 7fe231ffb700 1 -- 172.21.15.92:0/2435859974 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe24c054a80 con 0x55c75825fc10 2023-10-06T10:43:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.916+0000 7fe262f65b80 1 -- 172.21.15.92:0/2435859974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c75820a6e0 con 0x55c75825fc10 2023-10-06T10:43:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.916+0000 7fe231ffb700 1 -- 172.21.15.92:0/2435859974 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe24c05c0a0 con 0x55c75825fc10 2023-10-06T10:43:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.916+0000 7fe262f65b80 10 client.7452 fetch_fsmap learned FSMap version 74 2023-10-06T10:43:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.916+0000 7fe262f65b80 1 -- 172.21.15.92:0/2435859974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x55c757fa3030 con 0x55c75825fc10 2023-10-06T10:43:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.916+0000 7fe231ffb700 1 -- 172.21.15.92:0/2435859974 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c75820a6e0 con 0x55c75825fc10 2023-10-06T10:43:30.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.916+0000 7fe231ffb700 1 -- 172.21.15.92:0/2435859974 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 74) v1 ==== 3136+0+0 (secure 0 0 0) 0x7fe24c06bbb0 con 0x55c75825fc10 2023-10-06T10:43:30.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.916+0000 7fe262f65b80 10 client.7452 fetch_fsmap finished waiting for FSMap version 74 2023-10-06T10:43:30.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.916+0000 7fe262f65b80 10 client.7452 resolve_mds: resolved 5:2 to role '5:2' aka daemon mds.a 2023-10-06T10:43:30.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.916+0000 7fe262f65b80 20 client.7452 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:30.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.916+0000 7fe262f65b80 1 --2- 172.21.15.92:0/2435859974 >> [v2:172.21.15.92:6835/4003376156,v1:172.21.15.92:6837/4003376156] conn(0x55c75826ecb0 0x55c75823cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.916+0000 7fe262f65b80 4 client.7452 mds_command: new command op to 7146 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:43:30.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.916+0000 7fe262f65b80 1 -- 172.21.15.92:0/2435859974 --> [v2:172.21.15.92:6835/4003376156,v1:172.21.15.92:6837/4003376156] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7581e7610 con 0x55c75826ecb0 2023-10-06T10:43:30.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.916+0000 7fe2537fe700 1 --2- 172.21.15.92:0/2435859974 >> [v2:172.21.15.92:6835/4003376156,v1:172.21.15.92:6837/4003376156] conn(0x55c75826ecb0 0x55c75823cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.917+0000 7fe231ffb700 10 client.7452 ms_handle_connect on v2:172.21.15.92:6835/4003376156 2023-10-06T10:43:30.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.917+0000 7fe2537fe700 1 --2- 172.21.15.92:0/2435859974 >> [v2:172.21.15.92:6835/4003376156,v1:172.21.15.92:6837/4003376156] conn(0x55c75826ecb0 0x55c75823cc30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21503297ab12cbc3 server_cookie=56190ceab4db5bd8 in_seq=0 out_seq=0 2023-10-06T10:43:30.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.917+0000 7fe231ffb700 1 -- 172.21.15.92:0/2435859974 <== mds.2 v2:172.21.15.92:6835/4003376156 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55c7581e7610 con 0x55c75826ecb0 2023-10-06T10:43:30.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.917+0000 7fe231ffb700 10 client.7452 handle_command_reply: tid=0 2023-10-06T10:43:30.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.917+0000 7fe262f65b80 1 client.7452 shutdown 2023-10-06T10:43:30.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.917+0000 7fe262f65b80 1 -- 172.21.15.92:0/2435859974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75825fc10 msgr2=0x55c75826e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.917+0000 7fe262f65b80 1 --2- 172.21.15.92:0/2435859974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75825fc10 0x55c75826e770 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fe24c051a90 tx=0x7fe24c082ae0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.917+0000 7fe262f65b80 1 -- 172.21.15.92:0/2435859974 shutdown_connections 2023-10-06T10:43:30.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.917+0000 7fe262f65b80 1 --2- 172.21.15.92:0/2435859974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75825fc10 0x55c75826e770 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.917+0000 7fe262f65b80 1 --2- 172.21.15.92:0/2435859974 >> [v2:172.21.15.92:6835/4003376156,v1:172.21.15.92:6837/4003376156] conn(0x55c75826ecb0 0x55c75823cc30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.917+0000 7fe262f65b80 1 --2- 172.21.15.92:0/2435859974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7582159a0 0x55c758253f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.917+0000 7fe231ffb700 0 client.7452 ms_handle_reset on v2:172.21.15.92:6835/4003376156 2023-10-06T10:43:30.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.917+0000 7fe262f65b80 1 --2- 172.21.15.92:0/2435859974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c75823eeb0 0x55c758255de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.917+0000 7fe262f65b80 1 -- 172.21.15.92:0/2435859974 >> 172.21.15.92:0/2435859974 conn(0x55c7582172c0 msgr2=0x55c758216260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:30.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.917+0000 7fe262f65b80 1 -- 172.21.15.92:0/2435859974 shutdown_connections 2023-10-06T10:43:30.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.917+0000 7fe262f65b80 1 -- 172.21.15.92:0/2435859974 wait complete. 2023-10-06T10:43:30.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.918+0000 7fe262f65b80 20 client.7452 trim_cache size 0 max 16384 2023-10-06T10:43:30.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.931+0000 7fe262f65b80 1 Processor -- start 2023-10-06T10:43:30.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.931+0000 7fe262f65b80 1 -- start start 2023-10-06T10:43:30.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.932+0000 7fe262f65b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7582159a0 0x55c75826b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.932+0000 7fe262f65b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75823eeb0 0x55c7582455b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.932+0000 7fe262f65b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c75825fc10 0x55c75826d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.932+0000 7fe262f65b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c757fe8e80 con 0x55c75823eeb0 2023-10-06T10:43:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.932+0000 7fe262f65b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c75826f100 con 0x55c7582159a0 2023-10-06T10:43:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.932+0000 7fe262f65b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c75825fa90 con 0x55c75825fc10 2023-10-06T10:43:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.932+0000 7fe252ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75823eeb0 0x55c7582455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.932+0000 7fe252ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75823eeb0 0x55c7582455b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60780/0 (socket says 172.21.15.92:60780) 2023-10-06T10:43:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.932+0000 7fe252ffd700 1 -- 172.21.15.92:0/449767470 learned_addr learned my addr 172.21.15.92:0/449767470 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.932+0000 7fe252ffd700 1 -- 172.21.15.92:0/449767470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c75825fc10 msgr2=0x55c75826d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.932+0000 7fe252ffd700 1 --2- 172.21.15.92:0/449767470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c75825fc10 0x55c75826d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.932+0000 7fe2537fe700 1 --2- 172.21.15.92:0/449767470 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7582159a0 0x55c75826b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.932+0000 7fe252ffd700 1 -- 172.21.15.92:0/449767470 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7582159a0 msgr2=0x55c75826b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.932+0000 7fe252ffd700 1 --2- 172.21.15.92:0/449767470 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7582159a0 0x55c75826b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.932+0000 7fe252ffd700 1 -- 172.21.15.92:0/449767470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c75825b0f0 con 0x55c75823eeb0 2023-10-06T10:43:30.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.932+0000 7fe252ffd700 1 --2- 172.21.15.92:0/449767470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75823eeb0 0x55c7582455b0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fe248001d90 tx=0x7fe248018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2b2b8dcfc4a1a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:30.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.932+0000 7fe2397fa700 1 -- 172.21.15.92:0/449767470 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2480070d0 con 0x55c75823eeb0 2023-10-06T10:43:30.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.933+0000 7fe2397fa700 1 -- 172.21.15.92:0/449767470 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2480079d0 con 0x55c75823eeb0 2023-10-06T10:43:30.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.933+0000 7fe2397fa700 1 -- 172.21.15.92:0/449767470 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe248007480 con 0x55c75823eeb0 2023-10-06T10:43:30.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.933+0000 7fe262f65b80 1 -- 172.21.15.92:0/449767470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75823eeb0 msgr2=0x55c7582455b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.933+0000 7fe262f65b80 1 --2- 172.21.15.92:0/449767470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75823eeb0 0x55c7582455b0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fe248001d90 tx=0x7fe248018000 comp rx=0 tx=0).stop 2023-10-06T10:43:30.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.933+0000 7fe262f65b80 1 -- 172.21.15.92:0/449767470 shutdown_connections 2023-10-06T10:43:30.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.933+0000 7fe262f65b80 1 --2- 172.21.15.92:0/449767470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c75825fc10 0x55c75826d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.933+0000 7fe262f65b80 1 --2- 172.21.15.92:0/449767470 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7582159a0 0x55c75826b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.933+0000 7fe262f65b80 1 --2- 172.21.15.92:0/449767470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75823eeb0 0x55c7582455b0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.933+0000 7fe262f65b80 1 -- 172.21.15.92:0/449767470 >> 172.21.15.92:0/449767470 conn(0x55c7582172c0 msgr2=0x55c75823e060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:30.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.933+0000 7fe262f65b80 1 -- 172.21.15.92:0/449767470 shutdown_connections 2023-10-06T10:43:30.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.933+0000 7fe262f65b80 1 -- 172.21.15.92:0/449767470 wait complete. 2023-10-06T10:43:30.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.934+0000 7fe262f65b80 1 -- start start 2023-10-06T10:43:30.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.934+0000 7fe262f65b80 1 Processor -- start 2023-10-06T10:43:30.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.934+0000 7fe262f65b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7582159a0 0x55c758253f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.934+0000 7fe262f65b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c75823eeb0 0x55c75825f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.934+0000 7fe262f65b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75825fc10 0x55c7582c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe262f65b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c758216780 con 0x55c75825fc10 2023-10-06T10:43:30.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe262f65b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c75826a0d0 con 0x55c7582159a0 2023-10-06T10:43:30.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe262f65b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c75826a3b0 con 0x55c75823eeb0 2023-10-06T10:43:30.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe262f65b80 10 client.0 _ll_register_callbacks cb 0x55c758002f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:30.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe253fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75825fc10 0x55c7582c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe253fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75825fc10 0x55c7582c12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60790/0 (socket says 172.21.15.92:60790) 2023-10-06T10:43:30.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe253fff700 1 -- 172.21.15.92:0/2704381466 learned_addr learned my addr 172.21.15.92:0/2704381466 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:30.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe2537fe700 1 --2- 172.21.15.92:0/2704381466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7582159a0 0x55c758253f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe252ffd700 1 --2- 172.21.15.92:0/2704381466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c75823eeb0 0x55c75825f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe253fff700 1 -- 172.21.15.92:0/2704381466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c75823eeb0 msgr2=0x55c75825f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe253fff700 1 --2- 172.21.15.92:0/2704381466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c75823eeb0 0x55c75825f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe253fff700 1 -- 172.21.15.92:0/2704381466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7582159a0 msgr2=0x55c758253f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:30.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe253fff700 1 --2- 172.21.15.92:0/2704381466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7582159a0 0x55c758253f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe253fff700 1 -- 172.21.15.92:0/2704381466 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7580002c0 con 0x55c75825fc10 2023-10-06T10:43:30.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe253fff700 1 --2- 172.21.15.92:0/2704381466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75825fc10 0x55c7582c12b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fe24c084be0 tx=0x7fe24c069000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c55df08c21ed2c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:30.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe231ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:30.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe231ffb700 1 -- 172.21.15.92:0/2704381466 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c00e040 con 0x55c75825fc10 2023-10-06T10:43:30.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe231ffb700 1 -- 172.21.15.92:0/2704381466 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe24c054a80 con 0x55c75825fc10 2023-10-06T10:43:30.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe262f65b80 1 -- 172.21.15.92:0/2704381466 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe248005b10 con 0x55c75825fc10 2023-10-06T10:43:30.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe231ffb700 1 -- 172.21.15.92:0/2704381466 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe24c0550d0 con 0x55c75825fc10 2023-10-06T10:43:30.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe262f65b80 10 client.7458 fetch_fsmap learned FSMap version 74 2023-10-06T10:43:30.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe262f65b80 1 -- 172.21.15.92:0/2704381466 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x55c7581347d0 con 0x55c75825fc10 2023-10-06T10:43:30.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe231ffb700 1 -- 172.21.15.92:0/2704381466 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe248005b10 con 0x55c75825fc10 2023-10-06T10:43:30.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.935+0000 7fe231ffb700 1 -- 172.21.15.92:0/2704381466 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 74) v1 ==== 3136+0+0 (secure 0 0 0) 0x7fe24c013b10 con 0x55c75825fc10 2023-10-06T10:43:30.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.936+0000 7fe262f65b80 10 client.7458 fetch_fsmap finished waiting for FSMap version 74 2023-10-06T10:43:30.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.936+0000 7fe262f65b80 10 client.7458 resolve_mds: resolved 5:2 to role '5:2' aka daemon mds.a 2023-10-06T10:43:30.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.936+0000 7fe262f65b80 20 client.7458 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:30.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.936+0000 7fe262f65b80 1 --2- 172.21.15.92:0/2704381466 >> [v2:172.21.15.92:6835/4003376156,v1:172.21.15.92:6837/4003376156] conn(0x55c75825d9a0 0x55c75825dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:30.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.936+0000 7fe262f65b80 4 client.7458 mds_command: new command op to 7146 tid=0[{"prefix": "flush journal"}] 2023-10-06T10:43:30.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.936+0000 7fe262f65b80 1 -- 172.21.15.92:0/2704381466 --> [v2:172.21.15.92:6835/4003376156,v1:172.21.15.92:6837/4003376156] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c758108e50 con 0x55c75825d9a0 2023-10-06T10:43:30.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.936+0000 7fe2537fe700 1 --2- 172.21.15.92:0/2704381466 >> [v2:172.21.15.92:6835/4003376156,v1:172.21.15.92:6837/4003376156] conn(0x55c75825d9a0 0x55c75825dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:30.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.936+0000 7fe231ffb700 10 client.7458 ms_handle_connect on v2:172.21.15.92:6835/4003376156 2023-10-06T10:43:30.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.936+0000 7fe2537fe700 1 --2- 172.21.15.92:0/2704381466 >> [v2:172.21.15.92:6835/4003376156,v1:172.21.15.92:6837/4003376156] conn(0x55c75825d9a0 0x55c75825dd70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=369bb7e6f81f1916 server_cookie=ae6486a5d9f89e0f in_seq=0 out_seq=0 2023-10-06T10:43:30.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.943+0000 7fe231ffb700 1 -- 172.21.15.92:0/2704381466 <== mds.2 v2:172.21.15.92:6835/4003376156 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c758108e50 con 0x55c75825d9a0 2023-10-06T10:43:30.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.943+0000 7fe231ffb700 10 client.7458 handle_command_reply: tid=0 2023-10-06T10:43:30.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.943+0000 7fe262f65b80 1 client.7458 shutdown 2023-10-06T10:43:30.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.943+0000 7fe262f65b80 1 -- 172.21.15.92:0/2704381466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75825fc10 msgr2=0x55c7582c12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.943+0000 7fe262f65b80 1 --2- 172.21.15.92:0/2704381466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75825fc10 0x55c7582c12b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fe24c084be0 tx=0x7fe24c069000 comp rx=0 tx=0).stop 2023-10-06T10:43:30.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.943+0000 7fe262f65b80 1 -- 172.21.15.92:0/2704381466 shutdown_connections 2023-10-06T10:43:30.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.943+0000 7fe262f65b80 1 --2- 172.21.15.92:0/2704381466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c75825fc10 0x55c7582c12b0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.943+0000 7fe262f65b80 1 --2- 172.21.15.92:0/2704381466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7582159a0 0x55c758253f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.943+0000 7fe262f65b80 1 --2- 172.21.15.92:0/2704381466 >> [v2:172.21.15.92:6835/4003376156,v1:172.21.15.92:6837/4003376156] conn(0x55c75825d9a0 0x55c75825dd70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.943+0000 7fe262f65b80 1 --2- 172.21.15.92:0/2704381466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c75823eeb0 0x55c75825f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.943+0000 7fe231ffb700 0 client.7458 ms_handle_reset on v2:172.21.15.92:6835/4003376156 2023-10-06T10:43:30.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.943+0000 7fe262f65b80 1 -- 172.21.15.92:0/2704381466 >> 172.21.15.92:0/2704381466 conn(0x55c7582172c0 msgr2=0x55c758239560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:30.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.943+0000 7fe262f65b80 1 -- 172.21.15.92:0/2704381466 shutdown_connections 2023-10-06T10:43:30.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.943+0000 7fe262f65b80 1 -- 172.21.15.92:0/2704381466 wait complete. 2023-10-06T10:43:30.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.943+0000 7fe262f65b80 20 client.7458 trim_cache size 0 max 16384 2023-10-06T10:43:30.948 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:43:30.948 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2023-10-06T10:43:30.948 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:43:30.948 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:43:30.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.944+0000 7fe258a0b700 1 -- 172.21.15.92:0/1019355308 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe240066f30 msgr2=0x7fe2400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.944+0000 7fe258a0b700 1 --2- 172.21.15.92:0/1019355308 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe240066f30 0x7fe2400693e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe2440097c0 tx=0x7fe244019000 comp rx=0 tx=0).stop 2023-10-06T10:43:30.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.944+0000 7fe258a0b700 1 -- 172.21.15.92:0/1019355308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2540cf1b0 msgr2=0x7fe2540b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:30.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.944+0000 7fe258a0b700 1 --2- 172.21.15.92:0/1019355308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2540cf1b0 0x7fe2540b4d00 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00ec80 tx=0x7fe24c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.944+0000 7fe258a0b700 1 -- 172.21.15.92:0/1019355308 shutdown_connections 2023-10-06T10:43:30.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.944+0000 7fe258a0b700 1 --2- 172.21.15.92:0/1019355308 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe240066f30 0x7fe2400693e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.944+0000 7fe258a0b700 1 --2- 172.21.15.92:0/1019355308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2540cf1b0 0x7fe2540b4d00 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.944+0000 7fe258a0b700 1 --2- 172.21.15.92:0/1019355308 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2540b2620 0x7fe25415f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.944+0000 7fe258a0b700 1 --2- 172.21.15.92:0/1019355308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2540b29f0 0x7fe2540b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:30.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.944+0000 7fe258a0b700 1 -- 172.21.15.92:0/1019355308 >> 172.21.15.92:0/1019355308 conn(0x7fe254099520 msgr2=0x7fe2540cde80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:30.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.944+0000 7fe258a0b700 1 -- 172.21.15.92:0/1019355308 shutdown_connections 2023-10-06T10:43:30.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:30.944+0000 7fe258a0b700 1 -- 172.21.15.92:0/1019355308 wait complete. 2023-10-06T10:43:30.957 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:43:31.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.069+0000 7f9fa4bd5700 1 Processor -- start 2023-10-06T10:43:31.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.069+0000 7f9fa4bd5700 1 -- start start 2023-10-06T10:43:31.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.070+0000 7f9fa4bd5700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9fa0159560 0x7f9fa00aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.070+0000 7f9fa4bd5700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa00b49a0 0x7f9fa00aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.070+0000 7f9fa4bd5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa00b0c60 con 0x7f9fa00b0760 2023-10-06T10:43:31.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.070+0000 7f9fa4bd5700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa00b0f40 con 0x7f9fa0159560 2023-10-06T10:43:31.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.070+0000 7f9fa4bd5700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa00b1220 con 0x7f9fa00b49a0 2023-10-06T10:43:31.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.070+0000 7f9f9ed9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9fa00b0760 0x7f9fa00b3890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53956/0 (socket says 172.21.15.92:53956) 2023-10-06T10:43:31.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.070+0000 7f9f9ed9d700 1 -- 172.21.15.92:0/2555505970 learned_addr learned my addr 172.21.15.92:0/2555505970 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:31.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.070+0000 7f9f9e59c700 1 --2- 172.21.15.92:0/2555505970 >> v2:172.21.15.124:3300/0 conn(0x7f9fa0159560 0x7f9fa00aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.070+0000 7f9f95d9b700 1 --2- 172.21.15.92:0/2555505970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa00b49a0 0x7f9fa00aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.070+0000 7f9f9dd9b700 1 -- 172.21.15.92:0/2555505970 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3970573443 0 0) 0x7f9fa00b0c60 con 0x7f9fa00b0760 2023-10-06T10:43:31.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9dd9b700 1 -- 172.21.15.92:0/2555505970 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f8c003570 con 0x7f9fa00b0760 2023-10-06T10:43:31.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9dd9b700 1 -- 172.21.15.92:0/2555505970 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3671955078 0 0) 0x7f9f8c003570 con 0x7f9fa00b0760 2023-10-06T10:43:31.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9dd9b700 1 -- 172.21.15.92:0/2555505970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa00b49a0 msgr2=0x7f9fa00aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9dd9b700 1 --2- 172.21.15.92:0/2555505970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa00b49a0 0x7f9fa00aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9dd9b700 1 -- 172.21.15.92:0/2555505970 >> v2:172.21.15.124:3300/0 conn(0x7f9fa0159560 msgr2=0x7f9fa00aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9dd9b700 1 --2- 172.21.15.92:0/2555505970 >> v2:172.21.15.124:3300/0 conn(0x7f9fa0159560 0x7f9fa00aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9dd9b700 1 -- 172.21.15.92:0/2555505970 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa00b1500 con 0x7f9fa00b0760 2023-10-06T10:43:31.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9dd9b700 1 -- 172.21.15.92:0/2555505970 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f9f980017f0 con 0x7f9fa00b0760 2023-10-06T10:43:31.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9dd9b700 1 -- 172.21.15.92:0/2555505970 >> v1:172.21.15.92:6789/0 conn(0x7f9fa00b0760 legacy=0x7f9fa00b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9dd9b700 1 --2- 172.21.15.92:0/2555505970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa00b49a0 0x7f9f8c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9dd9b700 1 --2- 172.21.15.92:0/2555505970 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f8c003fe0 0x7f9f8c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9dd9b700 1 --2- 172.21.15.92:0/2555505970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f8c004d50 0x7f9f8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9dd9b700 1 -- 172.21.15.92:0/2555505970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa00b0f40 con 0x7f9f8c004d50 2023-10-06T10:43:31.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9dd9b700 1 -- 172.21.15.92:0/2555505970 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa00b1220 con 0x7f9f8c003fe0 2023-10-06T10:43:31.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9dd9b700 1 -- 172.21.15.92:0/2555505970 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c00ba90 con 0x7f9fa00b49a0 2023-10-06T10:43:31.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9ed9d700 1 --2- 172.21.15.92:0/2555505970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f8c004d50 0x7f9f8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f95d9b700 1 --2- 172.21.15.92:0/2555505970 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f8c003fe0 0x7f9f8c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9e59c700 1 --2- 172.21.15.92:0/2555505970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa00b49a0 0x7f9f8c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9ed9d700 1 -- 172.21.15.92:0/2555505970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa00b49a0 msgr2=0x7f9f8c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9ed9d700 1 --2- 172.21.15.92:0/2555505970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa00b49a0 0x7f9f8c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9ed9d700 1 -- 172.21.15.92:0/2555505970 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f8c003fe0 msgr2=0x7f9f8c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9ed9d700 1 --2- 172.21.15.92:0/2555505970 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f8c003fe0 0x7f9f8c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9ed9d700 1 -- 172.21.15.92:0/2555505970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c003570 con 0x7f9f8c004d50 2023-10-06T10:43:31.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.071+0000 7f9f9ed9d700 1 --2- 172.21.15.92:0/2555505970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f8c004d50 0x7f9f8c0051c0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f9f9800a970 tx=0x7f9f98002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df195b60f71a825f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.072+0000 7f9f9dd9b700 1 -- 172.21.15.92:0/2555505970 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f98012440 con 0x7f9f8c004d50 2023-10-06T10:43:31.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.072+0000 7f9f9dd9b700 1 -- 172.21.15.92:0/2555505970 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f980125c0 con 0x7f9f8c004d50 2023-10-06T10:43:31.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.072+0000 7f9f9dd9b700 1 -- 172.21.15.92:0/2555505970 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f98012980 con 0x7f9f8c004d50 2023-10-06T10:43:31.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.072+0000 7f9fa4bd5700 1 -- 172.21.15.92:0/2555505970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f8c004d50 msgr2=0x7f9f8c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.072+0000 7f9fa4bd5700 1 --2- 172.21.15.92:0/2555505970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f8c004d50 0x7f9f8c0051c0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f9f9800a970 tx=0x7f9f98002210 comp rx=0 tx=0).stop 2023-10-06T10:43:31.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.072+0000 7f9f9ed9d700 1 -- 172.21.15.92:0/2555505970 reap_dead start 2023-10-06T10:43:31.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.072+0000 7f9fa4bd5700 1 -- 172.21.15.92:0/2555505970 shutdown_connections 2023-10-06T10:43:31.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.072+0000 7f9fa4bd5700 1 -- 172.21.15.92:0/2555505970 >> 172.21.15.92:0/2555505970 conn(0x7f9fa00997f0 msgr2=0x7f9fa006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:31.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.072+0000 7f9fa4bd5700 1 -- 172.21.15.92:0/2555505970 shutdown_connections 2023-10-06T10:43:31.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.072+0000 7f9fa4bd5700 1 -- 172.21.15.92:0/2555505970 wait complete. 2023-10-06T10:43:31.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.073+0000 7f9fa4bd5700 1 Processor -- start 2023-10-06T10:43:31.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.073+0000 7f9fa4bd5700 1 -- start start 2023-10-06T10:43:31.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.074+0000 7f9fa4bd5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fa0155b00 0x7f9fa009cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.074+0000 7f9fa4bd5700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fa009d160 0x7f9fa0157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.074+0000 7f9fa4bd5700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa0157cd0 0x7f9fa00ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.074+0000 7f9fa4bd5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa009ada0 con 0x7f9fa0155b00 2023-10-06T10:43:31.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.074+0000 7f9fa4bd5700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa0155ed0 con 0x7f9fa009d160 2023-10-06T10:43:31.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.074+0000 7f9fa4bd5700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa0158140 con 0x7f9fa0157cd0 2023-10-06T10:43:31.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.074+0000 7f9f9e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fa0155b00 0x7f9fa009cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.074+0000 7f9f9e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fa0155b00 0x7f9fa009cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60804/0 (socket says 172.21.15.92:60804) 2023-10-06T10:43:31.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.074+0000 7f9f9e59c700 1 -- 172.21.15.92:0/1072739853 learned_addr learned my addr 172.21.15.92:0/1072739853 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.074+0000 7f9f95d9b700 1 --2- 172.21.15.92:0/1072739853 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fa009d160 0x7f9fa0157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.074+0000 7f9f9ed9d700 1 --2- 172.21.15.92:0/1072739853 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa0157cd0 0x7f9fa00ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.074+0000 7f9f9e59c700 1 -- 172.21.15.92:0/1072739853 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa0157cd0 msgr2=0x7f9fa00ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.074+0000 7f9f9e59c700 1 --2- 172.21.15.92:0/1072739853 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa0157cd0 0x7f9fa00ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.074+0000 7f9f9e59c700 1 -- 172.21.15.92:0/1072739853 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fa009d160 msgr2=0x7f9fa0157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.074+0000 7f9f9e59c700 1 --2- 172.21.15.92:0/1072739853 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fa009d160 0x7f9fa0157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.074+0000 7f9f9e59c700 1 -- 172.21.15.92:0/1072739853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa015d1b0 con 0x7f9fa0155b00 2023-10-06T10:43:31.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.074+0000 7f9f95d9b700 1 --2- 172.21.15.92:0/1072739853 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fa009d160 0x7f9fa0157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:31.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.075+0000 7f9f9e59c700 1 --2- 172.21.15.92:0/1072739853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fa0155b00 0x7f9fa009cc20 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f9f90011a40 tx=0x7f9f90009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.075+0000 7f9f96ffd700 1 -- 172.21.15.92:0/1072739853 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f90005c10 con 0x7f9fa0155b00 2023-10-06T10:43:31.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.075+0000 7f9f96ffd700 1 -- 172.21.15.92:0/1072739853 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f90005d70 con 0x7f9fa0155b00 2023-10-06T10:43:31.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.075+0000 7f9fa4bd5700 1 -- 172.21.15.92:0/1072739853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa009d6c0 con 0x7f9fa0155b00 2023-10-06T10:43:31.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.075+0000 7f9f96ffd700 1 -- 172.21.15.92:0/1072739853 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9000f960 con 0x7f9fa0155b00 2023-10-06T10:43:31.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.075+0000 7f9fa4bd5700 1 -- 172.21.15.92:0/1072739853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa015cb60 con 0x7f9fa0155b00 2023-10-06T10:43:31.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.076+0000 7f9fa4bd5700 1 -- 172.21.15.92:0/1072739853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa0064a90 con 0x7f9fa0155b00 2023-10-06T10:43:31.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.076+0000 7f9f96ffd700 1 -- 172.21.15.92:0/1072739853 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f9000fd90 con 0x7f9fa0155b00 2023-10-06T10:43:31.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.077+0000 7f9f96ffd700 1 --2- 172.21.15.92:0/1072739853 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f80066f30 0x7f9f800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.077+0000 7f9f95d9b700 1 --2- 172.21.15.92:0/1072739853 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f80066f30 0x7f9f800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.077+0000 7f9f96ffd700 1 -- 172.21.15.92:0/1072739853 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9f90085160 con 0x7f9fa0155b00 2023-10-06T10:43:31.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.077+0000 7f9f95d9b700 1 --2- 172.21.15.92:0/1072739853 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f80066f30 0x7f9f800693e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9fa016d180 tx=0x7f9f88008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.081+0000 7f9f96ffd700 1 -- 172.21.15.92:0/1072739853 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f90052130 con 0x7f9fa0155b00 2023-10-06T10:43:31.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.304+0000 7f9fa4bd5700 1 -- 172.21.15.92:0/1072739853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fa00b17e0 con 0x7f9fa0155b00 2023-10-06T10:43:31.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.304+0000 7f9f96ffd700 1 -- 172.21.15.92:0/1072739853 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4637 (secure 0 0 0) 0x7f9f90055750 con 0x7f9fa0155b00 2023-10-06T10:43:31.306 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:31.307 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":74,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3469511807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3469511807},{"type":"v1","addr":"172.21.15.124:6834","nonce":3469511807}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:03.173076+0000","modified":"2023-10-06T10:43:28.388741+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7164,"mds_1":7149,"mds_2":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"a","rank":2,"incarnation":71,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/4003376156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4003376156},{"type":"v1","addr":"172.21.15.92:6837","nonce":4003376156}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7149":{"gid":7149,"name":"c","rank":1,"incarnation":69,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6836/3268908749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3268908749},{"type":"v1","addr":"172.21.15.92:6836","nonce":3268908749}]},"join_fscid":-1,"export_targets":[2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7164":{"gid":7164,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/2971348516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2971348516},{"type":"v1","addr":"172.21.15.124:6835","nonce":2971348516}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T10:43:31.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.307+0000 7f9fa4bd5700 1 -- 172.21.15.92:0/1072739853 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f80066f30 msgr2=0x7f9f800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.307+0000 7f9fa4bd5700 1 --2- 172.21.15.92:0/1072739853 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f80066f30 0x7f9f800693e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9fa016d180 tx=0x7f9f88008040 comp rx=0 tx=0).stop 2023-10-06T10:43:31.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.307+0000 7f9fa4bd5700 1 -- 172.21.15.92:0/1072739853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fa0155b00 msgr2=0x7f9fa009cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.307+0000 7f9fa4bd5700 1 --2- 172.21.15.92:0/1072739853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fa0155b00 0x7f9fa009cc20 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f9f90011a40 tx=0x7f9f90009120 comp rx=0 tx=0).stop 2023-10-06T10:43:31.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.307+0000 7f9fa4bd5700 1 -- 172.21.15.92:0/1072739853 shutdown_connections 2023-10-06T10:43:31.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.307+0000 7f9fa4bd5700 1 --2- 172.21.15.92:0/1072739853 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f80066f30 0x7f9f800693e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.307+0000 7f9fa4bd5700 1 --2- 172.21.15.92:0/1072739853 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa0157cd0 0x7f9fa00ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.307+0000 7f9fa4bd5700 1 --2- 172.21.15.92:0/1072739853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fa0155b00 0x7f9fa009cc20 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.307+0000 7f9fa4bd5700 1 --2- 172.21.15.92:0/1072739853 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fa009d160 0x7f9fa0157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.307+0000 7f9fa4bd5700 1 -- 172.21.15.92:0/1072739853 >> 172.21.15.92:0/1072739853 conn(0x7f9fa00997f0 msgr2=0x7f9fa00cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:31.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.307+0000 7f9fa4bd5700 1 -- 172.21.15.92:0/1072739853 shutdown_connections 2023-10-06T10:43:31.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.307+0000 7f9fa4bd5700 1 -- 172.21.15.92:0/1072739853 wait complete. 2023-10-06T10:43:31.310 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 74 2023-10-06T10:43:31.320 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub start /d1/d2/d3 2023-10-06T10:43:31.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.437+0000 7fdadb5f5700 1 Processor -- start 2023-10-06T10:43:31.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.437+0000 7fdadb5f5700 1 -- start start 2023-10-06T10:43:31.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdadb5f5700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdad4159840 0x7fdad40c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdadb5f5700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdad40cf090 0x7fdad40c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdadb5f5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad40cbb80 con 0x7fdad4159470 2023-10-06T10:43:31.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdadb5f5700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad40c6f00 con 0x7fdad4159840 2023-10-06T10:43:31.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdadb5f5700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad40cb530 con 0x7fdad40cf090 2023-10-06T10:43:31.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdad9391700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdad4159470 0x7fdad40ce980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53970/0 (socket says 172.21.15.92:53970) 2023-10-06T10:43:31.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdad9391700 1 -- 172.21.15.92:0/1471665749 learned_addr learned my addr 172.21.15.92:0/1471665749 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:31.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdad9b92700 1 --2- 172.21.15.92:0/1471665749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdad40cf090 0x7fdad40c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdad8b90700 1 --2- 172.21.15.92:0/1471665749 >> v2:172.21.15.124:3300/0 conn(0x7fdad4159840 0x7fdad40c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdacbfff700 1 -- 172.21.15.92:0/1471665749 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199221824 0 0) 0x7fdad40cbb80 con 0x7fdad4159470 2023-10-06T10:43:31.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdacbfff700 1 -- 172.21.15.92:0/1471665749 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdac0003570 con 0x7fdad4159470 2023-10-06T10:43:31.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdacbfff700 1 -- 172.21.15.92:0/1471665749 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3557594379 0 0) 0x7fdac0003570 con 0x7fdad4159470 2023-10-06T10:43:31.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdacbfff700 1 -- 172.21.15.92:0/1471665749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdad40cf090 msgr2=0x7fdad40c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdacbfff700 1 --2- 172.21.15.92:0/1471665749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdad40cf090 0x7fdad40c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdacbfff700 1 -- 172.21.15.92:0/1471665749 >> v2:172.21.15.124:3300/0 conn(0x7fdad4159840 msgr2=0x7fdad40c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdacbfff700 1 --2- 172.21.15.92:0/1471665749 >> v2:172.21.15.124:3300/0 conn(0x7fdad4159840 0x7fdad40c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdacbfff700 1 -- 172.21.15.92:0/1471665749 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad40cb810 con 0x7fdad4159470 2023-10-06T10:43:31.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdacbfff700 1 -- 172.21.15.92:0/1471665749 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fdad00015f0 con 0x7fdad4159470 2023-10-06T10:43:31.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdacbfff700 1 -- 172.21.15.92:0/1471665749 >> v1:172.21.15.92:6789/0 conn(0x7fdad4159470 legacy=0x7fdad40ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdacbfff700 1 --2- 172.21.15.92:0/1471665749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdad40cf090 0x7fdac0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdacbfff700 1 --2- 172.21.15.92:0/1471665749 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdac0003fe0 0x7fdac0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdacbfff700 1 --2- 172.21.15.92:0/1471665749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdac0004d50 0x7fdac00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.438+0000 7fdacbfff700 1 -- 172.21.15.92:0/1471665749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad40c6f00 con 0x7fdac0004d50 2023-10-06T10:43:31.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.439+0000 7fdacbfff700 1 -- 172.21.15.92:0/1471665749 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad40cb530 con 0x7fdac0003fe0 2023-10-06T10:43:31.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.439+0000 7fdacbfff700 1 -- 172.21.15.92:0/1471665749 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac000ba90 con 0x7fdad40cf090 2023-10-06T10:43:31.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.439+0000 7fdad9b92700 1 --2- 172.21.15.92:0/1471665749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdac0004d50 0x7fdac00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.439+0000 7fdad9391700 1 --2- 172.21.15.92:0/1471665749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdad40cf090 0x7fdac0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.439+0000 7fdad8b90700 1 --2- 172.21.15.92:0/1471665749 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdac0003fe0 0x7fdac0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.439+0000 7fdad9b92700 1 -- 172.21.15.92:0/1471665749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdad40cf090 msgr2=0x7fdac0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.439+0000 7fdad9b92700 1 --2- 172.21.15.92:0/1471665749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdad40cf090 0x7fdac0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.439+0000 7fdad9b92700 1 -- 172.21.15.92:0/1471665749 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdac0003fe0 msgr2=0x7fdac0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.439+0000 7fdad9b92700 1 --2- 172.21.15.92:0/1471665749 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdac0003fe0 0x7fdac0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.439+0000 7fdad9b92700 1 -- 172.21.15.92:0/1471665749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac0003570 con 0x7fdac0004d50 2023-10-06T10:43:31.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.439+0000 7fdad9b92700 1 --2- 172.21.15.92:0/1471665749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdac0004d50 0x7fdac00051c0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fdad40cf490 tx=0x7fdacc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d710d21fc67ea55 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.439+0000 7fdacbfff700 1 -- 172.21.15.92:0/1471665749 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc00ea30 con 0x7fdac0004d50 2023-10-06T10:43:31.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.439+0000 7fdacbfff700 1 -- 172.21.15.92:0/1471665749 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdacc00f040 con 0x7fdac0004d50 2023-10-06T10:43:31.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.439+0000 7fdacbfff700 1 -- 172.21.15.92:0/1471665749 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc00ede0 con 0x7fdac0004d50 2023-10-06T10:43:31.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.440+0000 7fdadb5f5700 1 -- 172.21.15.92:0/1471665749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdac0004d50 msgr2=0x7fdac00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.440+0000 7fdadb5f5700 1 --2- 172.21.15.92:0/1471665749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdac0004d50 0x7fdac00051c0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fdad40cf490 tx=0x7fdacc00b040 comp rx=0 tx=0).stop 2023-10-06T10:43:31.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.440+0000 7fdad9b92700 1 -- 172.21.15.92:0/1471665749 reap_dead start 2023-10-06T10:43:31.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.440+0000 7fdadb5f5700 1 -- 172.21.15.92:0/1471665749 shutdown_connections 2023-10-06T10:43:31.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.440+0000 7fdadb5f5700 1 -- 172.21.15.92:0/1471665749 >> 172.21.15.92:0/1471665749 conn(0x7fdad40996e0 msgr2=0x7fdad406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:31.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.440+0000 7fdadb5f5700 1 -- 172.21.15.92:0/1471665749 shutdown_connections 2023-10-06T10:43:31.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.440+0000 7fdadb5f5700 1 -- 172.21.15.92:0/1471665749 wait complete. 2023-10-06T10:43:31.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.441+0000 7fdadb5f5700 1 Processor -- start 2023-10-06T10:43:31.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.441+0000 7fdadb5f5700 1 -- start start 2023-10-06T10:43:31.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.442+0000 7fdadb5f5700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdad4151ea0 0x7fdad415a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.442+0000 7fdadb5f5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad415ad10 0x7fdad415b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.442+0000 7fdadb5f5700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdad415b9d0 0x7fdad40b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.442+0000 7fdadb5f5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad4152270 con 0x7fdad415ad10 2023-10-06T10:43:31.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.442+0000 7fdadb5f5700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad415a340 con 0x7fdad415b9d0 2023-10-06T10:43:31.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.442+0000 7fdadb5f5700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad416a5f0 con 0x7fdad4151ea0 2023-10-06T10:43:31.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.442+0000 7fdad8b90700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad415ad10 0x7fdad415b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.442+0000 7fdad8b90700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad415ad10 0x7fdad415b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60828/0 (socket says 172.21.15.92:60828) 2023-10-06T10:43:31.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.442+0000 7fdad8b90700 1 -- 172.21.15.92:0/2511963022 learned_addr learned my addr 172.21.15.92:0/2511963022 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.442+0000 7fdad9391700 1 --2- 172.21.15.92:0/2511963022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdad4151ea0 0x7fdad415a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.442+0000 7fdad9b92700 1 --2- 172.21.15.92:0/2511963022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdad415b9d0 0x7fdad40b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.442+0000 7fdad8b90700 1 -- 172.21.15.92:0/2511963022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdad4151ea0 msgr2=0x7fdad415a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.442+0000 7fdad8b90700 1 --2- 172.21.15.92:0/2511963022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdad4151ea0 0x7fdad415a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.442+0000 7fdad8b90700 1 -- 172.21.15.92:0/2511963022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdad415b9d0 msgr2=0x7fdad40b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.442+0000 7fdad8b90700 1 --2- 172.21.15.92:0/2511963022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdad415b9d0 0x7fdad40b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.442+0000 7fdad8b90700 1 -- 172.21.15.92:0/2511963022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad4159cf0 con 0x7fdad415ad10 2023-10-06T10:43:31.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.442+0000 7fdad8b90700 1 --2- 172.21.15.92:0/2511963022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad415ad10 0x7fdad415b490 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdac4006ac0 tx=0x7fdac40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.443+0000 7fdac97fa700 1 -- 172.21.15.92:0/2511963022 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac400fa10 con 0x7fdad415ad10 2023-10-06T10:43:31.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.443+0000 7fdac97fa700 1 -- 172.21.15.92:0/2511963022 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac400fb70 con 0x7fdad415ad10 2023-10-06T10:43:31.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.443+0000 7fdadb5f5700 1 -- 172.21.15.92:0/2511963022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad4159fd0 con 0x7fdad415ad10 2023-10-06T10:43:31.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.443+0000 7fdac97fa700 1 -- 172.21.15.92:0/2511963022 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac4014440 con 0x7fdad415ad10 2023-10-06T10:43:31.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.443+0000 7fdadb5f5700 1 -- 172.21.15.92:0/2511963022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad416a8d0 con 0x7fdad415ad10 2023-10-06T10:43:31.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.444+0000 7fdac97fa700 1 -- 172.21.15.92:0/2511963022 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdac40147f0 con 0x7fdad415ad10 2023-10-06T10:43:31.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.444+0000 7fdac97fa700 1 --2- 172.21.15.92:0/2511963022 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdab8066f30 0x7fdab80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.444+0000 7fdac97fa700 1 -- 172.21.15.92:0/2511963022 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fdac4018030 con 0x7fdad415ad10 2023-10-06T10:43:31.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.444+0000 7fdad9391700 1 --2- 172.21.15.92:0/2511963022 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdab8066f30 0x7fdab80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.445+0000 7fdad9391700 1 --2- 172.21.15.92:0/2511963022 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fdab8066f30 0x7fdab80693e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdad0000e10 tx=0x7fdad0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.446+0000 7fdae48edb80 1 Processor -- start 2023-10-06T10:43:31.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.446+0000 7fdae48edb80 1 -- start start 2023-10-06T10:43:31.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.447+0000 7fdae48edb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563a25841af0 0x563a25841ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.447+0000 7fdae48edb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fda00 0x563a257fa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.447+0000 7fdae48edb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563a257fbc30 0x563a257fab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.447+0000 7fdae48edb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a2577ace0 con 0x563a257fda00 2023-10-06T10:43:31.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.447+0000 7fdae48edb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a25751ad0 con 0x563a25841af0 2023-10-06T10:43:31.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.447+0000 7fdae48edb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a2574f2d0 con 0x563a257fbc30 2023-10-06T10:43:31.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.447+0000 7fdad8b90700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fda00 0x563a257fa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.447+0000 7fdad8b90700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fda00 0x563a257fa5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60842/0 (socket says 172.21.15.92:60842) 2023-10-06T10:43:31.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.447+0000 7fdad8b90700 1 -- 172.21.15.92:0/815534178 learned_addr learned my addr 172.21.15.92:0/815534178 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.447+0000 7fdad9391700 1 --2- 172.21.15.92:0/815534178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563a25841af0 0x563a25841ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.447+0000 7fdad9b92700 1 --2- 172.21.15.92:0/815534178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563a257fbc30 0x563a257fab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.447+0000 7fdad8b90700 1 -- 172.21.15.92:0/815534178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563a257fbc30 msgr2=0x563a257fab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.447+0000 7fdad8b90700 1 --2- 172.21.15.92:0/815534178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563a257fbc30 0x563a257fab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.447+0000 7fdad8b90700 1 -- 172.21.15.92:0/815534178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563a25841af0 msgr2=0x563a25841ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.447+0000 7fdad8b90700 1 --2- 172.21.15.92:0/815534178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563a25841af0 0x563a25841ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.447+0000 7fdad8b90700 1 -- 172.21.15.92:0/815534178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a255e52d0 con 0x563a257fda00 2023-10-06T10:43:31.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.447+0000 7fdad8b90700 1 --2- 172.21.15.92:0/815534178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fda00 0x563a257fa5e0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fdac4083410 tx=0x7fdac4083370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c711b89f9d4244 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.447+0000 7fdabe7fc700 1 -- 172.21.15.92:0/815534178 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac4055ea0 con 0x563a257fda00 2023-10-06T10:43:31.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.447+0000 7fdabe7fc700 1 -- 172.21.15.92:0/815534178 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac40590f0 con 0x563a257fda00 2023-10-06T10:43:31.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.447+0000 7fdabe7fc700 1 -- 172.21.15.92:0/815534178 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac405ce40 con 0x563a257fda00 2023-10-06T10:43:31.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.448+0000 7fdae48edb80 1 -- 172.21.15.92:0/815534178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fda00 msgr2=0x563a257fa5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.448+0000 7fdae48edb80 1 --2- 172.21.15.92:0/815534178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fda00 0x563a257fa5e0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fdac4083410 tx=0x7fdac4083370 comp rx=0 tx=0).stop 2023-10-06T10:43:31.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.448+0000 7fdae48edb80 1 -- 172.21.15.92:0/815534178 shutdown_connections 2023-10-06T10:43:31.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.448+0000 7fdae48edb80 1 --2- 172.21.15.92:0/815534178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563a257fbc30 0x563a257fab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.448+0000 7fdae48edb80 1 --2- 172.21.15.92:0/815534178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563a25841af0 0x563a25841ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.448+0000 7fdae48edb80 1 --2- 172.21.15.92:0/815534178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fda00 0x563a257fa5e0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.448+0000 7fdae48edb80 1 -- 172.21.15.92:0/815534178 >> 172.21.15.92:0/815534178 conn(0x563a257fc0d0 msgr2=0x563a257fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:31.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.448+0000 7fdae48edb80 1 -- 172.21.15.92:0/815534178 shutdown_connections 2023-10-06T10:43:31.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.448+0000 7fdae48edb80 1 -- 172.21.15.92:0/815534178 wait complete. 2023-10-06T10:43:31.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.449+0000 7fdae48edb80 1 -- start start 2023-10-06T10:43:31.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.449+0000 7fdae48edb80 1 Processor -- start 2023-10-06T10:43:31.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdae48edb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fbc30 0x563a2583c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdae48edb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563a257fda00 0x563a25822b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdae48edb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563a25841af0 0x563a25823060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdae48edb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a2574b0c0 con 0x563a257fbc30 2023-10-06T10:43:31.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdae48edb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a256f68f0 con 0x563a257fda00 2023-10-06T10:43:31.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdae48edb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a25711f00 con 0x563a25841af0 2023-10-06T10:43:31.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdae48edb80 10 client.0 _ll_register_callbacks cb 0x563a255e7f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:31.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdad9391700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fbc30 0x563a2583c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdad9391700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fbc30 0x563a2583c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60850/0 (socket says 172.21.15.92:60850) 2023-10-06T10:43:31.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdad9391700 1 -- 172.21.15.92:0/1064071790 learned_addr learned my addr 172.21.15.92:0/1064071790 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdad9391700 1 -- 172.21.15.92:0/1064071790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563a25841af0 msgr2=0x563a25823060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdad9391700 1 --2- 172.21.15.92:0/1064071790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563a25841af0 0x563a25823060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdad9391700 1 -- 172.21.15.92:0/1064071790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563a257fda00 msgr2=0x563a25822b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdad9391700 1 --2- 172.21.15.92:0/1064071790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563a257fda00 0x563a25822b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdad9391700 1 -- 172.21.15.92:0/1064071790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad416a8d0 con 0x563a257fbc30 2023-10-06T10:43:31.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdad9391700 1 --2- 172.21.15.92:0/1064071790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fbc30 0x563a2583c860 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fdad0014460 tx=0x7fdad0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d62c7faab0cf93 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdab37fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:31.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdab37fe700 1 -- 172.21.15.92:0/1064071790 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad0014e30 con 0x563a257fbc30 2023-10-06T10:43:31.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdab37fe700 1 -- 172.21.15.92:0/1064071790 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fdad001c070 con 0x563a257fbc30 2023-10-06T10:43:31.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdae48edb80 1 -- 172.21.15.92:0/1064071790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a257ef6f0 con 0x563a257fbc30 2023-10-06T10:43:31.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdab37fe700 1 -- 172.21.15.92:0/1064071790 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdad0013570 con 0x563a257fbc30 2023-10-06T10:43:31.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdae48edb80 10 client.7476 fetch_fsmap learned FSMap version 74 2023-10-06T10:43:31.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.450+0000 7fdae48edb80 1 -- 172.21.15.92:0/1064071790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x563a25588040 con 0x563a257fbc30 2023-10-06T10:43:31.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.451+0000 7fdab37fe700 1 -- 172.21.15.92:0/1064071790 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563a257ef6f0 con 0x563a257fbc30 2023-10-06T10:43:31.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.451+0000 7fdab37fe700 1 -- 172.21.15.92:0/1064071790 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 74) v1 ==== 3136+0+0 (secure 0 0 0) 0x7fdad00136d0 con 0x563a257fbc30 2023-10-06T10:43:31.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.451+0000 7fdae48edb80 10 client.7476 fetch_fsmap finished waiting for FSMap version 74 2023-10-06T10:43:31.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.451+0000 7fdae48edb80 10 client.7476 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:43:31.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.451+0000 7fdae48edb80 20 client.7476 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:31.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.451+0000 7fdae48edb80 1 --2- 172.21.15.92:0/1064071790 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x563a25824990 0x563a25824d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.451+0000 7fdae48edb80 4 client.7476 mds_command: new command op to 7149 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:43:31.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.451+0000 7fdae48edb80 1 -- 172.21.15.92:0/1064071790 --> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a257cc620 con 0x563a25824990 2023-10-06T10:43:31.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.451+0000 7fdad8b90700 1 --2- 172.21.15.92:0/1064071790 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x563a25824990 0x563a25824d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.451+0000 7fdab37fe700 10 client.7476 ms_handle_connect on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:31.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.451+0000 7fdad8b90700 1 --2- 172.21.15.92:0/1064071790 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x563a25824990 0x563a25824d80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a1625bf77b80eec6 server_cookie=4ad60d7f2ac93712 in_seq=0 out_seq=0 2023-10-06T10:43:31.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.452+0000 7fdab37fe700 1 -- 172.21.15.92:0/1064071790 <== mds.1 v2:172.21.15.92:6834/3268908749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x563a257cc620 con 0x563a25824990 2023-10-06T10:43:31.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.452+0000 7fdab37fe700 10 client.7476 handle_command_reply: tid=0 2023-10-06T10:43:31.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.452+0000 7fdae48edb80 1 client.7476 shutdown 2023-10-06T10:43:31.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.452+0000 7fdae48edb80 1 -- 172.21.15.92:0/1064071790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fbc30 msgr2=0x563a2583c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.452+0000 7fdae48edb80 1 --2- 172.21.15.92:0/1064071790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fbc30 0x563a2583c860 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fdad0014460 tx=0x7fdad0015000 comp rx=0 tx=0).stop 2023-10-06T10:43:31.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.452+0000 7fdae48edb80 1 -- 172.21.15.92:0/1064071790 shutdown_connections 2023-10-06T10:43:31.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.452+0000 7fdae48edb80 1 --2- 172.21.15.92:0/1064071790 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x563a25824990 0x563a25824d80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.452+0000 7fdae48edb80 1 --2- 172.21.15.92:0/1064071790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563a25841af0 0x563a25823060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.452+0000 7fdae48edb80 1 --2- 172.21.15.92:0/1064071790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fbc30 0x563a2583c860 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.452+0000 7fdab37fe700 0 client.7476 ms_handle_reset on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:31.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.452+0000 7fdae48edb80 1 --2- 172.21.15.92:0/1064071790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563a257fda00 0x563a25822b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.452+0000 7fdae48edb80 1 -- 172.21.15.92:0/1064071790 >> 172.21.15.92:0/1064071790 conn(0x563a257fc0d0 msgr2=0x563a25841040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:31.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.452+0000 7fdae48edb80 1 -- 172.21.15.92:0/1064071790 shutdown_connections 2023-10-06T10:43:31.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.452+0000 7fdae48edb80 1 -- 172.21.15.92:0/1064071790 wait complete. 2023-10-06T10:43:31.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.452+0000 7fdae48edb80 20 client.7476 trim_cache size 0 max 16384 2023-10-06T10:43:31.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.468+0000 7fdae48edb80 1 Processor -- start 2023-10-06T10:43:31.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.468+0000 7fdae48edb80 1 -- start start 2023-10-06T10:43:31.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.469+0000 7fdae48edb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fbc30 0x563a25822b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.469+0000 7fdae48edb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563a257fda00 0x563a25823060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.469+0000 7fdae48edb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563a25841af0 0x563a257f68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.469+0000 7fdae48edb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a255cde90 con 0x563a257fbc30 2023-10-06T10:43:31.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.469+0000 7fdae48edb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a257da130 con 0x563a25841af0 2023-10-06T10:43:31.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.469+0000 7fdae48edb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a2583d020 con 0x563a257fda00 2023-10-06T10:43:31.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.469+0000 7fdad9391700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fbc30 0x563a25822b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.469+0000 7fdad9391700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fbc30 0x563a25822b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60862/0 (socket says 172.21.15.92:60862) 2023-10-06T10:43:31.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.469+0000 7fdad9391700 1 -- 172.21.15.92:0/2502983502 learned_addr learned my addr 172.21.15.92:0/2502983502 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.469+0000 7fdad8b90700 1 --2- 172.21.15.92:0/2502983502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563a257fda00 0x563a25823060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.469+0000 7fdad9b92700 1 --2- 172.21.15.92:0/2502983502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563a25841af0 0x563a257f68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.469+0000 7fdad9391700 1 -- 172.21.15.92:0/2502983502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563a257fda00 msgr2=0x563a25823060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.469+0000 7fdad9391700 1 --2- 172.21.15.92:0/2502983502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563a257fda00 0x563a25823060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.469+0000 7fdad9391700 1 -- 172.21.15.92:0/2502983502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563a25841af0 msgr2=0x563a257f68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.469+0000 7fdad9391700 1 --2- 172.21.15.92:0/2502983502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563a25841af0 0x563a257f68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.469+0000 7fdad9391700 1 -- 172.21.15.92:0/2502983502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a257f3300 con 0x563a257fbc30 2023-10-06T10:43:31.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.469+0000 7fdad9391700 1 --2- 172.21.15.92:0/2502983502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fbc30 0x563a25822b20 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fdad0000ef0 tx=0x7fdad0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f753d33164e8ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.470+0000 7fdabeffd700 1 -- 172.21.15.92:0/2502983502 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad0013e70 con 0x563a257fbc30 2023-10-06T10:43:31.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.470+0000 7fdabeffd700 1 -- 172.21.15.92:0/2502983502 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad0013c30 con 0x563a257fbc30 2023-10-06T10:43:31.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.470+0000 7fdabeffd700 1 -- 172.21.15.92:0/2502983502 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad0013570 con 0x563a257fbc30 2023-10-06T10:43:31.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.470+0000 7fdae48edb80 1 -- 172.21.15.92:0/2502983502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fbc30 msgr2=0x563a25822b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.470+0000 7fdae48edb80 1 --2- 172.21.15.92:0/2502983502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fbc30 0x563a25822b20 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fdad0000ef0 tx=0x7fdad0019000 comp rx=0 tx=0).stop 2023-10-06T10:43:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.470+0000 7fdae48edb80 1 -- 172.21.15.92:0/2502983502 shutdown_connections 2023-10-06T10:43:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.470+0000 7fdae48edb80 1 --2- 172.21.15.92:0/2502983502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563a25841af0 0x563a257f68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.470+0000 7fdae48edb80 1 --2- 172.21.15.92:0/2502983502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fbc30 0x563a25822b20 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.470+0000 7fdae48edb80 1 --2- 172.21.15.92:0/2502983502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563a257fda00 0x563a25823060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.470+0000 7fdae48edb80 1 -- 172.21.15.92:0/2502983502 >> 172.21.15.92:0/2502983502 conn(0x563a257fc0d0 msgr2=0x563a2583c860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.470+0000 7fdae48edb80 1 -- 172.21.15.92:0/2502983502 shutdown_connections 2023-10-06T10:43:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.470+0000 7fdae48edb80 1 -- 172.21.15.92:0/2502983502 wait complete. 2023-10-06T10:43:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.471+0000 7fdae48edb80 1 -- start start 2023-10-06T10:43:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.471+0000 7fdae48edb80 1 Processor -- start 2023-10-06T10:43:31.479 DEBUG:teuthology.orchestra.run:got remote process result: 18 2023-10-06T10:43:31.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.471+0000 7fdae48edb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fbc30 0x563a2583f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdae48edb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563a257fda00 0x563a2583f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdae48edb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563a25841af0 0x563a2583aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdae48edb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a25837600 con 0x563a257fbc30 2023-10-06T10:43:31.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdae48edb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a258a7640 con 0x563a25841af0 2023-10-06T10:43:31.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdae48edb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a25837790 con 0x563a257fda00 2023-10-06T10:43:31.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdae48edb80 10 client.0 _ll_register_callbacks cb 0x563a255e7f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:31.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdad9391700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fbc30 0x563a2583f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdad9391700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fbc30 0x563a2583f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60876/0 (socket says 172.21.15.92:60876) 2023-10-06T10:43:31.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdad9391700 1 -- 172.21.15.92:0/1411472653 learned_addr learned my addr 172.21.15.92:0/1411472653 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdad8b90700 1 --2- 172.21.15.92:0/1411472653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563a257fda00 0x563a2583f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdad9b92700 1 --2- 172.21.15.92:0/1411472653 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563a25841af0 0x563a2583aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdad9391700 1 -- 172.21.15.92:0/1411472653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563a257fda00 msgr2=0x563a2583f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdad9391700 1 --2- 172.21.15.92:0/1411472653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563a257fda00 0x563a2583f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdad9391700 1 -- 172.21.15.92:0/1411472653 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563a25841af0 msgr2=0x563a2583aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdad9391700 1 --2- 172.21.15.92:0/1411472653 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563a25841af0 0x563a2583aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdad9391700 1 -- 172.21.15.92:0/1411472653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a25588040 con 0x563a257fbc30 2023-10-06T10:43:31.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdad9b92700 1 --2- 172.21.15.92:0/1411472653 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563a25841af0 0x563a2583aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:31.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdad9391700 1 --2- 172.21.15.92:0/1411472653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fbc30 0x563a2583f150 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fdad0014460 tx=0x7fdad0016070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed214d6c9656299 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdab37fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:31.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdab37fe700 1 -- 172.21.15.92:0/1411472653 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad003c070 con 0x563a257fbc30 2023-10-06T10:43:31.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdab37fe700 1 -- 172.21.15.92:0/1411472653 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fdad0041d60 con 0x563a257fbc30 2023-10-06T10:43:31.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdae48edb80 1 -- 172.21.15.92:0/1411472653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdac40590f0 con 0x563a257fbc30 2023-10-06T10:43:31.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdab37fe700 1 -- 172.21.15.92:0/1411472653 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdad0014dc0 con 0x563a257fbc30 2023-10-06T10:43:31.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdae48edb80 10 client.7482 fetch_fsmap learned FSMap version 74 2023-10-06T10:43:31.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.472+0000 7fdae48edb80 1 -- 172.21.15.92:0/1411472653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x563a257197e0 con 0x563a257fbc30 2023-10-06T10:43:31.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdab37fe700 1 -- 172.21.15.92:0/1411472653 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac40590f0 con 0x563a257fbc30 2023-10-06T10:43:31.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdab37fe700 1 -- 172.21.15.92:0/1411472653 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 74) v1 ==== 3136+0+0 (secure 0 0 0) 0x7fdad0037460 con 0x563a257fbc30 2023-10-06T10:43:31.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdae48edb80 10 client.7482 fetch_fsmap finished waiting for FSMap version 74 2023-10-06T10:43:31.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdae48edb80 10 client.7482 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:43:31.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdae48edb80 20 client.7482 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:31.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdae48edb80 1 --2- 172.21.15.92:0/1411472653 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x563a25844fb0 0x563a2583af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdae48edb80 4 client.7482 mds_command: new command op to 7149 tid=0[{"prefix": "scrub start", "path": "/d1/d2/d3"}] 2023-10-06T10:43:31.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdae48edb80 1 -- 172.21.15.92:0/1411472653 --> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3"}) v1 -- 0x563a256ede60 con 0x563a25844fb0 2023-10-06T10:43:31.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdad8b90700 1 --2- 172.21.15.92:0/1411472653 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x563a25844fb0 0x563a2583af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdab37fe700 10 client.7482 ms_handle_connect on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:31.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdad8b90700 1 --2- 172.21.15.92:0/1411472653 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x563a25844fb0 0x563a2583af80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2467300e536bb6af server_cookie=40470b8def7407ee in_seq=0 out_seq=0 2023-10-06T10:43:31.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdab37fe700 1 -- 172.21.15.92:0/1411472653 <== mds.1 v2:172.21.15.92:6834/3268908749 1 ==== command_reply(tid 0: -18 Not rank 0) v1 ==== 18+0+0 (crc 0 0 0) 0x563a256ede60 con 0x563a25844fb0 2023-10-06T10:43:31.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdab37fe700 10 client.7482 handle_command_reply: tid=0 2023-10-06T10:43:31.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdae48edb80 1 client.7482 shutdown 2023-10-06T10:43:31.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdae48edb80 1 -- 172.21.15.92:0/1411472653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fbc30 msgr2=0x563a2583f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdae48edb80 1 --2- 172.21.15.92:0/1411472653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fbc30 0x563a2583f150 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fdad0014460 tx=0x7fdad0016070 comp rx=0 tx=0).stop 2023-10-06T10:43:31.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdae48edb80 1 -- 172.21.15.92:0/1411472653 shutdown_connections 2023-10-06T10:43:31.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdae48edb80 1 --2- 172.21.15.92:0/1411472653 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x563a25844fb0 0x563a2583af80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdae48edb80 1 --2- 172.21.15.92:0/1411472653 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563a25841af0 0x563a2583aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdab37fe700 0 client.7482 ms_handle_reset on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:31.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdae48edb80 1 --2- 172.21.15.92:0/1411472653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563a257fbc30 0x563a2583f150 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdae48edb80 1 --2- 172.21.15.92:0/1411472653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563a257fda00 0x563a2583f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.473+0000 7fdae48edb80 1 -- 172.21.15.92:0/1411472653 >> 172.21.15.92:0/1411472653 conn(0x563a257fc0d0 msgr2=0x563a257f7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:31.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.474+0000 7fdae48edb80 1 -- 172.21.15.92:0/1411472653 shutdown_connections 2023-10-06T10:43:31.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.474+0000 7fdae48edb80 1 -- 172.21.15.92:0/1411472653 wait complete. 2023-10-06T10:43:31.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.474+0000 7fdae48edb80 20 client.7482 trim_cache size 0 max 16384 2023-10-06T10:43:31.485 INFO:teuthology.orchestra.run.smithi092.stderr:Error EXDEV: Not rank 0 2023-10-06T10:43:31.485 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub abort 2023-10-06T10:43:31.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.600+0000 7f3c7bc6d700 1 Processor -- start 2023-10-06T10:43:31.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.600+0000 7f3c7bc6d700 1 -- start start 2023-10-06T10:43:31.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.601+0000 7f3c7bc6d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3c74150940 0x7f3c74159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.601+0000 7f3c7bc6d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3c7415a160 0x7f3c740ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.601+0000 7f3c7bc6d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c740c5250 0x7f3c740cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.601+0000 7f3c7bc6d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c7415ac30 con 0x7f3c74150940 2023-10-06T10:43:31.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.601+0000 7f3c7bc6d700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c740cb6a0 con 0x7f3c7415a160 2023-10-06T10:43:31.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.601+0000 7f3c7bc6d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c740cb090 con 0x7f3c740c5250 2023-10-06T10:43:31.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.601+0000 7f3c79a09700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3c74150940 0x7f3c74159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.601+0000 7f3c79a09700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3c74150940 0x7f3c74159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60888/0 (socket says 172.21.15.92:60888) 2023-10-06T10:43:31.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.601+0000 7f3c79a09700 1 -- 172.21.15.92:0/181481483 learned_addr learned my addr 172.21.15.92:0/181481483 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.601+0000 7f3c79208700 1 --2- 172.21.15.92:0/181481483 >> v2:172.21.15.124:3300/0 conn(0x7f3c7415a160 0x7f3c740ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.601+0000 7f3c7a20a700 1 --2- 172.21.15.92:0/181481483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c740c5250 0x7f3c740cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.602+0000 7f3c79a09700 1 -- 172.21.15.92:0/181481483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c740c5250 msgr2=0x7f3c740cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.602+0000 7f3c79a09700 1 --2- 172.21.15.92:0/181481483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c740c5250 0x7f3c740cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.602+0000 7f3c79a09700 1 -- 172.21.15.92:0/181481483 >> v2:172.21.15.124:3300/0 conn(0x7f3c7415a160 msgr2=0x7f3c740ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.602+0000 7f3c79a09700 1 --2- 172.21.15.92:0/181481483 >> v2:172.21.15.124:3300/0 conn(0x7f3c7415a160 0x7f3c740ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.602+0000 7f3c79a09700 1 -- 172.21.15.92:0/181481483 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7415a670 con 0x7f3c74150940 2023-10-06T10:43:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.602+0000 7f3c79a09700 1 --2- 172.21.15.92:0/181481483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c74150940 0x7f3c74159c20 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f3c74089650 tx=0x7f3c68009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c821a506912f6527 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.602+0000 7f3c78a07700 1 -- 172.21.15.92:0/181481483 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6800e070 con 0x7f3c74150940 2023-10-06T10:43:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.602+0000 7f3c78a07700 1 -- 172.21.15.92:0/181481483 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6800b970 con 0x7f3c74150940 2023-10-06T10:43:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.602+0000 7f3c78a07700 1 -- 172.21.15.92:0/181481483 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c680092e0 con 0x7f3c74150940 2023-10-06T10:43:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.602+0000 7f3c7bc6d700 1 -- 172.21.15.92:0/181481483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c74150940 msgr2=0x7f3c74159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.602+0000 7f3c7bc6d700 1 --2- 172.21.15.92:0/181481483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c74150940 0x7f3c74159c20 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f3c74089650 tx=0x7f3c68009840 comp rx=0 tx=0).stop 2023-10-06T10:43:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.602+0000 7f3c7bc6d700 1 -- 172.21.15.92:0/181481483 shutdown_connections 2023-10-06T10:43:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.602+0000 7f3c7bc6d700 1 --2- 172.21.15.92:0/181481483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c74150940 0x7f3c74159c20 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.602+0000 7f3c7bc6d700 1 --2- 172.21.15.92:0/181481483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c740c5250 0x7f3c740cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.602+0000 7f3c7bc6d700 1 --2- 172.21.15.92:0/181481483 >> v2:172.21.15.124:3300/0 conn(0x7f3c7415a160 0x7f3c740ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.602+0000 7f3c7bc6d700 1 -- 172.21.15.92:0/181481483 >> 172.21.15.92:0/181481483 conn(0x7f3c74099700 msgr2=0x7f3c7406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.603+0000 7f3c7bc6d700 1 -- 172.21.15.92:0/181481483 shutdown_connections 2023-10-06T10:43:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.603+0000 7f3c7bc6d700 1 -- 172.21.15.92:0/181481483 wait complete. 2023-10-06T10:43:31.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.604+0000 7f3c7bc6d700 1 Processor -- start 2023-10-06T10:43:31.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.604+0000 7f3c7bc6d700 1 -- start start 2023-10-06T10:43:31.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.604+0000 7f3c7bc6d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c740c5250 0x7f3c74140060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.604+0000 7f3c7bc6d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c7415a160 0x7f3c7415bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.604+0000 7f3c7bc6d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c741590f0 0x7f3c740cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.604+0000 7f3c7bc6d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c74159570 con 0x7f3c7415a160 2023-10-06T10:43:31.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.604+0000 7f3c7bc6d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7415c550 con 0x7f3c741590f0 2023-10-06T10:43:31.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.604+0000 7f3c7bc6d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7415c830 con 0x7f3c740c5250 2023-10-06T10:43:31.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.604+0000 7f3c79208700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c7415a160 0x7f3c7415bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.604+0000 7f3c79208700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c7415a160 0x7f3c7415bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60896/0 (socket says 172.21.15.92:60896) 2023-10-06T10:43:31.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.604+0000 7f3c79208700 1 -- 172.21.15.92:0/632153137 learned_addr learned my addr 172.21.15.92:0/632153137 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.605+0000 7f3c79a09700 1 --2- 172.21.15.92:0/632153137 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c740c5250 0x7f3c74140060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.605+0000 7f3c7a20a700 1 --2- 172.21.15.92:0/632153137 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c741590f0 0x7f3c740cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.605+0000 7f3c79208700 1 -- 172.21.15.92:0/632153137 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c740c5250 msgr2=0x7f3c74140060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.605+0000 7f3c79208700 1 --2- 172.21.15.92:0/632153137 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c740c5250 0x7f3c74140060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.605+0000 7f3c79208700 1 -- 172.21.15.92:0/632153137 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c741590f0 msgr2=0x7f3c740cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.605+0000 7f3c79208700 1 --2- 172.21.15.92:0/632153137 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c741590f0 0x7f3c740cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.605+0000 7f3c79208700 1 -- 172.21.15.92:0/632153137 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c68009ae0 con 0x7f3c7415a160 2023-10-06T10:43:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.605+0000 7f3c7a20a700 1 --2- 172.21.15.92:0/632153137 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c741590f0 0x7f3c740cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.605+0000 7f3c79208700 1 --2- 172.21.15.92:0/632153137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c7415a160 0x7f3c7415bec0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000ec50 tx=0x7f3c600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.605+0000 7f3c71ffb700 1 -- 172.21.15.92:0/632153137 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6000c850 con 0x7f3c7415a160 2023-10-06T10:43:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.605+0000 7f3c71ffb700 1 -- 172.21.15.92:0/632153137 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6000c9b0 con 0x7f3c7415a160 2023-10-06T10:43:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.605+0000 7f3c71ffb700 1 -- 172.21.15.92:0/632153137 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6000cd70 con 0x7f3c7415a160 2023-10-06T10:43:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.605+0000 7f3c7bc6d700 1 -- 172.21.15.92:0/632153137 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c7415cb10 con 0x7f3c7415a160 2023-10-06T10:43:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.605+0000 7f3c7bc6d700 1 -- 172.21.15.92:0/632153137 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7415d0d0 con 0x7f3c7415a160 2023-10-06T10:43:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.607+0000 7f3c71ffb700 1 -- 172.21.15.92:0/632153137 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c60013680 con 0x7f3c7415a160 2023-10-06T10:43:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.607+0000 7f3c71ffb700 1 --2- 172.21.15.92:0/632153137 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c58066f30 0x7f3c580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.607+0000 7f3c71ffb700 1 -- 172.21.15.92:0/632153137 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3c60084aa0 con 0x7f3c7415a160 2023-10-06T10:43:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.607+0000 7f3c79a09700 1 --2- 172.21.15.92:0/632153137 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c58066f30 0x7f3c580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.607+0000 7f3c79a09700 1 --2- 172.21.15.92:0/632153137 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c58066f30 0x7f3c580693e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009fd0 tx=0x7f3c68018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.609+0000 7f3c84f65b80 1 Processor -- start 2023-10-06T10:43:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.609+0000 7f3c84f65b80 1 -- start start 2023-10-06T10:43:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.609+0000 7f3c84f65b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ff013c4ac0 0x55ff013c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.609+0000 7f3c84f65b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013809d0 0x55ff0137d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.609+0000 7f3c84f65b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ff0137ec00 0x55ff0137daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.609+0000 7f3c84f65b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff012fdcb0 con 0x55ff013809d0 2023-10-06T10:43:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.609+0000 7f3c84f65b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff012d4aa0 con 0x55ff013c4ac0 2023-10-06T10:43:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.609+0000 7f3c84f65b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff012d22a0 con 0x55ff0137ec00 2023-10-06T10:43:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.610+0000 7f3c79208700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013809d0 0x55ff0137d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.610+0000 7f3c79208700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013809d0 0x55ff0137d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60902/0 (socket says 172.21.15.92:60902) 2023-10-06T10:43:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.610+0000 7f3c79208700 1 -- 172.21.15.92:0/1742312238 learned_addr learned my addr 172.21.15.92:0/1742312238 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.610+0000 7f3c79a09700 1 --2- 172.21.15.92:0/1742312238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ff013c4ac0 0x55ff013c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.610+0000 7f3c7a20a700 1 --2- 172.21.15.92:0/1742312238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ff0137ec00 0x55ff0137daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.610+0000 7f3c79208700 1 -- 172.21.15.92:0/1742312238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ff0137ec00 msgr2=0x55ff0137daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.610+0000 7f3c79208700 1 --2- 172.21.15.92:0/1742312238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ff0137ec00 0x55ff0137daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.610+0000 7f3c79208700 1 -- 172.21.15.92:0/1742312238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ff013c4ac0 msgr2=0x55ff013c4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.610+0000 7f3c79208700 1 --2- 172.21.15.92:0/1742312238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ff013c4ac0 0x55ff013c4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.610+0000 7f3c79208700 1 -- 172.21.15.92:0/1742312238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff011682a0 con 0x55ff013809d0 2023-10-06T10:43:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.610+0000 7f3c79208700 1 --2- 172.21.15.92:0/1742312238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013809d0 0x55ff0137d5b0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f3c60009850 tx=0x7f3c60082c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c18d23912c0f211 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.610+0000 7f3c56ffd700 1 -- 172.21.15.92:0/1742312238 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c60058980 con 0x55ff013809d0 2023-10-06T10:43:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.610+0000 7f3c56ffd700 1 -- 172.21.15.92:0/1742312238 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c60084040 con 0x55ff013809d0 2023-10-06T10:43:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.610+0000 7f3c56ffd700 1 -- 172.21.15.92:0/1742312238 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6005c6d0 con 0x55ff013809d0 2023-10-06T10:43:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.611+0000 7f3c84f65b80 1 -- 172.21.15.92:0/1742312238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013809d0 msgr2=0x55ff0137d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.611+0000 7f3c84f65b80 1 --2- 172.21.15.92:0/1742312238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013809d0 0x55ff0137d5b0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f3c60009850 tx=0x7f3c60082c80 comp rx=0 tx=0).stop 2023-10-06T10:43:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.611+0000 7f3c84f65b80 1 -- 172.21.15.92:0/1742312238 shutdown_connections 2023-10-06T10:43:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.611+0000 7f3c84f65b80 1 --2- 172.21.15.92:0/1742312238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ff0137ec00 0x55ff0137daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.611+0000 7f3c84f65b80 1 --2- 172.21.15.92:0/1742312238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ff013c4ac0 0x55ff013c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.611+0000 7f3c84f65b80 1 --2- 172.21.15.92:0/1742312238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013809d0 0x55ff0137d5b0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.611+0000 7f3c84f65b80 1 -- 172.21.15.92:0/1742312238 >> 172.21.15.92:0/1742312238 conn(0x55ff0137f0a0 msgr2=0x55ff0137f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.611+0000 7f3c84f65b80 1 -- 172.21.15.92:0/1742312238 shutdown_connections 2023-10-06T10:43:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.611+0000 7f3c84f65b80 1 -- 172.21.15.92:0/1742312238 wait complete. 2023-10-06T10:43:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.611+0000 7f3c84f65b80 1 -- start start 2023-10-06T10:43:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.611+0000 7f3c84f65b80 1 Processor -- start 2023-10-06T10:43:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.612+0000 7f3c84f65b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ff0137ec00 0x55ff013bef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.612+0000 7f3c84f65b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013809d0 0x55ff013a76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.612+0000 7f3c84f65b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ff013c4ac0 0x55ff013a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.612+0000 7f3c84f65b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff012ce090 con 0x55ff013809d0 2023-10-06T10:43:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.612+0000 7f3c84f65b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff012798c0 con 0x55ff013c4ac0 2023-10-06T10:43:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.612+0000 7f3c84f65b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff01294ed0 con 0x55ff0137ec00 2023-10-06T10:43:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.612+0000 7f3c84f65b80 10 client.0 _ll_register_callbacks cb 0x55ff0116af60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.612+0000 7f3c79208700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013809d0 0x55ff013a76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.612+0000 7f3c79208700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013809d0 0x55ff013a76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60918/0 (socket says 172.21.15.92:60918) 2023-10-06T10:43:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.612+0000 7f3c79208700 1 -- 172.21.15.92:0/1340587257 learned_addr learned my addr 172.21.15.92:0/1340587257 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c7a20a700 1 --2- 172.21.15.92:0/1340587257 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ff013c4ac0 0x55ff013a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c79a09700 1 --2- 172.21.15.92:0/1340587257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ff0137ec00 0x55ff013bef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c79208700 1 -- 172.21.15.92:0/1340587257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ff0137ec00 msgr2=0x55ff013bef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c79208700 1 --2- 172.21.15.92:0/1340587257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ff0137ec00 0x55ff013bef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c79208700 1 -- 172.21.15.92:0/1340587257 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ff013c4ac0 msgr2=0x55ff013a3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c79208700 1 --2- 172.21.15.92:0/1340587257 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ff013c4ac0 0x55ff013a3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c79208700 1 -- 172.21.15.92:0/1340587257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7415d0d0 con 0x55ff013809d0 2023-10-06T10:43:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c79208700 1 --2- 172.21.15.92:0/1340587257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013809d0 0x55ff013a76a0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f3c60009850 tx=0x7f3c6006c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f1b48b91456243 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c4ffff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c4ffff700 1 -- 172.21.15.92:0/1340587257 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c60058980 con 0x55ff013809d0 2023-10-06T10:43:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c4ffff700 1 -- 172.21.15.92:0/1340587257 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3c6005c6d0 con 0x55ff013809d0 2023-10-06T10:43:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c84f65b80 1 -- 172.21.15.92:0/1340587257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff013726c0 con 0x55ff013809d0 2023-10-06T10:43:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c4ffff700 1 -- 172.21.15.92:0/1340587257 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c60071250 con 0x55ff013809d0 2023-10-06T10:43:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c84f65b80 10 client.7494 fetch_fsmap learned FSMap version 74 2023-10-06T10:43:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c84f65b80 1 -- 172.21.15.92:0/1340587257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x55ff0110b010 con 0x55ff013809d0 2023-10-06T10:43:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c4ffff700 1 -- 172.21.15.92:0/1340587257 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ff013726c0 con 0x55ff013809d0 2023-10-06T10:43:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c4ffff700 1 -- 172.21.15.92:0/1340587257 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 74) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f3c60054c20 con 0x55ff013809d0 2023-10-06T10:43:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c84f65b80 10 client.7494 fetch_fsmap finished waiting for FSMap version 74 2023-10-06T10:43:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c84f65b80 10 client.7494 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:43:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c84f65b80 20 client.7494 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c84f65b80 1 --2- 172.21.15.92:0/1340587257 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55ff013a41b0 0x55ff0137daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c84f65b80 4 client.7494 mds_command: new command op to 7149 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:43:31.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c84f65b80 1 -- 172.21.15.92:0/1340587257 --> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff0134f5f0 con 0x55ff013a41b0 2023-10-06T10:43:31.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.613+0000 7f3c79a09700 1 --2- 172.21.15.92:0/1340587257 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55ff013a41b0 0x55ff0137daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.614+0000 7f3c4ffff700 10 client.7494 ms_handle_connect on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:31.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.614+0000 7f3c79a09700 1 --2- 172.21.15.92:0/1340587257 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55ff013a41b0 0x55ff0137daa0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48555f9c189bc7e3 server_cookie=3c233bace9b987fe in_seq=0 out_seq=0 2023-10-06T10:43:31.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.614+0000 7f3c4ffff700 1 -- 172.21.15.92:0/1340587257 <== mds.1 v2:172.21.15.92:6834/3268908749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55ff0134f5f0 con 0x55ff013a41b0 2023-10-06T10:43:31.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.614+0000 7f3c4ffff700 10 client.7494 handle_command_reply: tid=0 2023-10-06T10:43:31.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.614+0000 7f3c84f65b80 1 client.7494 shutdown 2023-10-06T10:43:31.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.614+0000 7f3c84f65b80 1 -- 172.21.15.92:0/1340587257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013809d0 msgr2=0x55ff013a76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.614+0000 7f3c84f65b80 1 --2- 172.21.15.92:0/1340587257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013809d0 0x55ff013a76a0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f3c60009850 tx=0x7f3c6006c000 comp rx=0 tx=0).stop 2023-10-06T10:43:31.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.614+0000 7f3c84f65b80 1 -- 172.21.15.92:0/1340587257 shutdown_connections 2023-10-06T10:43:31.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.614+0000 7f3c84f65b80 1 --2- 172.21.15.92:0/1340587257 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55ff013a41b0 0x55ff0137daa0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.614+0000 7f3c84f65b80 1 --2- 172.21.15.92:0/1340587257 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ff013c4ac0 0x55ff013a3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.614+0000 7f3c84f65b80 1 --2- 172.21.15.92:0/1340587257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ff0137ec00 0x55ff013bef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.614+0000 7f3c4ffff700 0 client.7494 ms_handle_reset on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:31.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.614+0000 7f3c84f65b80 1 --2- 172.21.15.92:0/1340587257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013809d0 0x55ff013a76a0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.614+0000 7f3c84f65b80 1 -- 172.21.15.92:0/1340587257 >> 172.21.15.92:0/1340587257 conn(0x55ff0137f0a0 msgr2=0x55ff013c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:31.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.614+0000 7f3c84f65b80 1 -- 172.21.15.92:0/1340587257 shutdown_connections 2023-10-06T10:43:31.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.615+0000 7f3c84f65b80 1 -- 172.21.15.92:0/1340587257 wait complete. 2023-10-06T10:43:31.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.615+0000 7f3c84f65b80 20 client.7494 trim_cache size 0 max 16384 2023-10-06T10:43:31.640 DEBUG:teuthology.orchestra.run:got remote process result: 18 2023-10-06T10:43:31.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.630+0000 7f3c84f65b80 1 Processor -- start 2023-10-06T10:43:31.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.630+0000 7f3c84f65b80 1 -- start start 2023-10-06T10:43:31.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.631+0000 7f3c84f65b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ff0137ec00 0x55ff013c2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.631+0000 7f3c84f65b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ff013809d0 0x55ff013c2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.631+0000 7f3c84f65b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013c4ac0 0x55ff013bfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.631+0000 7f3c84f65b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff01150e60 con 0x55ff013c4ac0 2023-10-06T10:43:31.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.631+0000 7f3c84f65b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff013e67c0 con 0x55ff0137ec00 2023-10-06T10:43:31.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.631+0000 7f3c84f65b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff013806f0 con 0x55ff013809d0 2023-10-06T10:43:31.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.631+0000 7f3c7a20a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013c4ac0 0x55ff013bfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.631+0000 7f3c7a20a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013c4ac0 0x55ff013bfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60934/0 (socket says 172.21.15.92:60934) 2023-10-06T10:43:31.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.631+0000 7f3c7a20a700 1 -- 172.21.15.92:0/3602546213 learned_addr learned my addr 172.21.15.92:0/3602546213 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.631+0000 7f3c79208700 1 --2- 172.21.15.92:0/3602546213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ff013809d0 0x55ff013c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.631+0000 7f3c79a09700 1 --2- 172.21.15.92:0/3602546213 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ff0137ec00 0x55ff013c2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.631+0000 7f3c7a20a700 1 -- 172.21.15.92:0/3602546213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ff013809d0 msgr2=0x55ff013c2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.631+0000 7f3c7a20a700 1 --2- 172.21.15.92:0/3602546213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ff013809d0 0x55ff013c2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.631+0000 7f3c7a20a700 1 -- 172.21.15.92:0/3602546213 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ff0137ec00 msgr2=0x55ff013c2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.631+0000 7f3c7a20a700 1 --2- 172.21.15.92:0/3602546213 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ff0137ec00 0x55ff013c2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.631+0000 7f3c7a20a700 1 -- 172.21.15.92:0/3602546213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff0129c7b0 con 0x55ff013c4ac0 2023-10-06T10:43:31.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.631+0000 7f3c7a20a700 1 --2- 172.21.15.92:0/3602546213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013c4ac0 0x55ff013bfa30 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009bf0 tx=0x7f3c6c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0cbf33b78a0027 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.631+0000 7f3c577fe700 1 -- 172.21.15.92:0/3602546213 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0157c0 con 0x55ff013c4ac0 2023-10-06T10:43:31.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.631+0000 7f3c577fe700 1 -- 172.21.15.92:0/3602546213 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6c015920 con 0x55ff013c4ac0 2023-10-06T10:43:31.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.631+0000 7f3c577fe700 1 -- 172.21.15.92:0/3602546213 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c015ca0 con 0x55ff013c4ac0 2023-10-06T10:43:31.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.632+0000 7f3c84f65b80 1 -- 172.21.15.92:0/3602546213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013c4ac0 msgr2=0x55ff013bfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.632+0000 7f3c84f65b80 1 --2- 172.21.15.92:0/3602546213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013c4ac0 0x55ff013bfa30 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009bf0 tx=0x7f3c6c008040 comp rx=0 tx=0).stop 2023-10-06T10:43:31.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.632+0000 7f3c84f65b80 1 -- 172.21.15.92:0/3602546213 shutdown_connections 2023-10-06T10:43:31.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.632+0000 7f3c84f65b80 1 --2- 172.21.15.92:0/3602546213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013c4ac0 0x55ff013bfa30 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.632+0000 7f3c84f65b80 1 --2- 172.21.15.92:0/3602546213 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ff0137ec00 0x55ff013c2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.632+0000 7f3c84f65b80 1 --2- 172.21.15.92:0/3602546213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ff013809d0 0x55ff013c2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.632+0000 7f3c84f65b80 1 -- 172.21.15.92:0/3602546213 >> 172.21.15.92:0/3602546213 conn(0x55ff0137f0a0 msgr2=0x55ff01378c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:31.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.632+0000 7f3c84f65b80 1 -- 172.21.15.92:0/3602546213 shutdown_connections 2023-10-06T10:43:31.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.632+0000 7f3c84f65b80 1 -- 172.21.15.92:0/3602546213 wait complete. 2023-10-06T10:43:31.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.632+0000 7f3c84f65b80 1 -- start start 2023-10-06T10:43:31.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.633+0000 7f3c84f65b80 1 Processor -- start 2023-10-06T10:43:31.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.633+0000 7f3c84f65b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ff0137ec00 0x55ff014294c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.633+0000 7f3c84f65b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013809d0 0x55ff013c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.633+0000 7f3c84f65b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ff013c4ac0 0x55ff013c22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.633+0000 7f3c84f65b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff013a60f0 con 0x55ff013809d0 2023-10-06T10:43:31.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.633+0000 7f3c84f65b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff01379870 con 0x55ff013c4ac0 2023-10-06T10:43:31.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.633+0000 7f3c84f65b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff013ba8e0 con 0x55ff0137ec00 2023-10-06T10:43:31.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.633+0000 7f3c84f65b80 10 client.0 _ll_register_callbacks cb 0x55ff0116af60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:31.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.633+0000 7f3c79208700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013809d0 0x55ff013c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.633+0000 7f3c79208700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013809d0 0x55ff013c40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60946/0 (socket says 172.21.15.92:60946) 2023-10-06T10:43:31.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.633+0000 7f3c79208700 1 -- 172.21.15.92:0/4142706278 learned_addr learned my addr 172.21.15.92:0/4142706278 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c79a09700 1 --2- 172.21.15.92:0/4142706278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ff0137ec00 0x55ff014294c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c7a20a700 1 --2- 172.21.15.92:0/4142706278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ff013c4ac0 0x55ff013c22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c79208700 1 -- 172.21.15.92:0/4142706278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ff0137ec00 msgr2=0x55ff014294c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c79208700 1 --2- 172.21.15.92:0/4142706278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ff0137ec00 0x55ff014294c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c79208700 1 -- 172.21.15.92:0/4142706278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ff013c4ac0 msgr2=0x55ff013c22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c79208700 1 --2- 172.21.15.92:0/4142706278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ff013c4ac0 0x55ff013c22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c79208700 1 -- 172.21.15.92:0/4142706278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c6c0098f0 con 0x55ff013809d0 2023-10-06T10:43:31.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c7a20a700 1 --2- 172.21.15.92:0/4142706278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ff013c4ac0 0x55ff013c22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:31.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c79208700 1 --2- 172.21.15.92:0/4142706278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013809d0 0x55ff013c40b0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f3c600556c0 tx=0x7f3c6007e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4cb7049fd47adbe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c4ffff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:31.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c4ffff700 1 -- 172.21.15.92:0/4142706278 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c60056520 con 0x55ff013809d0 2023-10-06T10:43:31.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c4ffff700 1 -- 172.21.15.92:0/4142706278 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3c6005c6d0 con 0x55ff013809d0 2023-10-06T10:43:31.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c84f65b80 1 -- 172.21.15.92:0/4142706278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c60084040 con 0x55ff013809d0 2023-10-06T10:43:31.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c4ffff700 1 -- 172.21.15.92:0/4142706278 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c60058980 con 0x55ff013809d0 2023-10-06T10:43:31.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c84f65b80 10 client.7500 fetch_fsmap learned FSMap version 74 2023-10-06T10:43:31.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c84f65b80 1 -- 172.21.15.92:0/4142706278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x55ff012834a0 con 0x55ff013809d0 2023-10-06T10:43:31.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c4ffff700 1 -- 172.21.15.92:0/4142706278 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c60084040 con 0x55ff013809d0 2023-10-06T10:43:31.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c4ffff700 1 -- 172.21.15.92:0/4142706278 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 74) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f3c6005c240 con 0x55ff013809d0 2023-10-06T10:43:31.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c84f65b80 10 client.7500 fetch_fsmap finished waiting for FSMap version 74 2023-10-06T10:43:31.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c84f65b80 10 client.7500 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:43:31.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c84f65b80 20 client.7500 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:31.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c84f65b80 1 --2- 172.21.15.92:0/4142706278 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55ff013abe20 0x55ff013ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c84f65b80 4 client.7500 mds_command: new command op to 7149 tid=0[{"prefix": "scrub abort"}] 2023-10-06T10:43:31.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.634+0000 7f3c84f65b80 1 -- 172.21.15.92:0/4142706278 --> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x55ff01270e30 con 0x55ff013abe20 2023-10-06T10:43:31.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.635+0000 7f3c79a09700 1 --2- 172.21.15.92:0/4142706278 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55ff013abe20 0x55ff013ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.635+0000 7f3c4ffff700 10 client.7500 ms_handle_connect on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:31.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.635+0000 7f3c79a09700 1 --2- 172.21.15.92:0/4142706278 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55ff013abe20 0x55ff013ac1f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cba9cbf74277bbcb server_cookie=bcc936d0093916f6 in_seq=0 out_seq=0 2023-10-06T10:43:31.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.635+0000 7f3c4ffff700 1 -- 172.21.15.92:0/4142706278 <== mds.1 v2:172.21.15.92:6834/3268908749 1 ==== command_reply(tid 0: -18 Not rank 0) v1 ==== 18+0+0 (crc 0 0 0) 0x55ff01270e30 con 0x55ff013abe20 2023-10-06T10:43:31.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.635+0000 7f3c4ffff700 10 client.7500 handle_command_reply: tid=0 2023-10-06T10:43:31.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.635+0000 7f3c84f65b80 1 client.7500 shutdown 2023-10-06T10:43:31.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.635+0000 7f3c84f65b80 1 -- 172.21.15.92:0/4142706278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013809d0 msgr2=0x55ff013c40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.635+0000 7f3c84f65b80 1 --2- 172.21.15.92:0/4142706278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013809d0 0x55ff013c40b0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f3c600556c0 tx=0x7f3c6007e000 comp rx=0 tx=0).stop 2023-10-06T10:43:31.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.635+0000 7f3c84f65b80 1 -- 172.21.15.92:0/4142706278 shutdown_connections 2023-10-06T10:43:31.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.635+0000 7f3c84f65b80 1 --2- 172.21.15.92:0/4142706278 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55ff013abe20 0x55ff013ac1f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.635+0000 7f3c84f65b80 1 --2- 172.21.15.92:0/4142706278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ff013c4ac0 0x55ff013c22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.635+0000 7f3c4ffff700 0 client.7500 ms_handle_reset on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:31.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.635+0000 7f3c84f65b80 1 --2- 172.21.15.92:0/4142706278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ff0137ec00 0x55ff014294c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.635+0000 7f3c84f65b80 1 --2- 172.21.15.92:0/4142706278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff013809d0 0x55ff013c40b0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.635+0000 7f3c84f65b80 1 -- 172.21.15.92:0/4142706278 >> 172.21.15.92:0/4142706278 conn(0x55ff0137f0a0 msgr2=0x55ff013a0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:31.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.635+0000 7f3c84f65b80 1 -- 172.21.15.92:0/4142706278 shutdown_connections 2023-10-06T10:43:31.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.635+0000 7f3c84f65b80 1 -- 172.21.15.92:0/4142706278 wait complete. 2023-10-06T10:43:31.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.635+0000 7f3c84f65b80 20 client.7500 trim_cache size 0 max 16384 2023-10-06T10:43:31.652 INFO:teuthology.orchestra.run.smithi092.stderr:Error EXDEV: Not rank 0 2023-10-06T10:43:31.652 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub pause 2023-10-06T10:43:31.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.765+0000 7f6e9c5a9700 1 Processor -- start 2023-10-06T10:43:31.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.765+0000 7f6e9c5a9700 1 -- start start 2023-10-06T10:43:31.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.766+0000 7f6e9c5a9700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e940cd4d0 0x7f6e940b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.766+0000 7f6e9c5a9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e9415ac30 con 0x7f6e940c5250 2023-10-06T10:43:31.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.766+0000 7f6e9c5a9700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e940cb6a0 con 0x7f6e94150940 2023-10-06T10:43:31.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.766+0000 7f6e9c5a9700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e940cb090 con 0x7f6e940cd4d0 2023-10-06T10:43:31.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.766+0000 7f6e99b44700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6e940c5250 0x7f6e940ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53978/0 (socket says 172.21.15.92:53978) 2023-10-06T10:43:31.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.766+0000 7f6e99b44700 1 -- 172.21.15.92:0/189885373 learned_addr learned my addr 172.21.15.92:0/189885373 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:31.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e9ab46700 1 --2- 172.21.15.92:0/189885373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e940cd4d0 0x7f6e940b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e99343700 1 -- 172.21.15.92:0/189885373 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3818735100 0 0) 0x7f6e88001110 con 0x7f6e940c5250 2023-10-06T10:43:31.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e99343700 1 -- 172.21.15.92:0/189885373 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e7c003570 con 0x7f6e940c5250 2023-10-06T10:43:31.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e99343700 1 -- 172.21.15.92:0/189885373 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1233027349 0 0) 0x7f6e7c003570 con 0x7f6e940c5250 2023-10-06T10:43:31.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e99343700 1 -- 172.21.15.92:0/189885373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e940cd4d0 msgr2=0x7f6e940b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e99343700 1 --2- 172.21.15.92:0/189885373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e940cd4d0 0x7f6e940b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e99343700 1 -- 172.21.15.92:0/189885373 >> v1:172.21.15.124:6789/0 conn(0x7f6e94150940 legacy=0x7f6e94159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e99343700 1 -- 172.21.15.92:0/189885373 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9415a670 con 0x7f6e940c5250 2023-10-06T10:43:31.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e99343700 1 -- 172.21.15.92:0/189885373 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f6e88001600 con 0x7f6e940c5250 2023-10-06T10:43:31.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e99343700 1 -- 172.21.15.92:0/189885373 >> v1:172.21.15.92:6789/0 conn(0x7f6e940c5250 legacy=0x7f6e940ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e99343700 1 --2- 172.21.15.92:0/189885373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e7c003a90 0x7f6e7c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e99343700 1 --2- 172.21.15.92:0/189885373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e7c004810 0x7f6e7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e99343700 1 --2- 172.21.15.92:0/189885373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e940cd4d0 0x7f6e7c0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e99343700 1 -- 172.21.15.92:0/189885373 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e940cb090 con 0x7f6e7c004810 2023-10-06T10:43:31.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e99343700 1 -- 172.21.15.92:0/189885373 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c00bb60 con 0x7f6e7c003a90 2023-10-06T10:43:31.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e99343700 1 -- 172.21.15.92:0/189885373 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c00bd20 con 0x7f6e940cd4d0 2023-10-06T10:43:31.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e99b44700 1 --2- 172.21.15.92:0/189885373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e7c004810 0x7f6e7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e9a345700 1 --2- 172.21.15.92:0/189885373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e7c003a90 0x7f6e7c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e9ab46700 1 --2- 172.21.15.92:0/189885373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e940cd4d0 0x7f6e7c0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e99b44700 1 -- 172.21.15.92:0/189885373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e940cd4d0 msgr2=0x7f6e7c0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e99b44700 1 --2- 172.21.15.92:0/189885373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e940cd4d0 0x7f6e7c0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e99b44700 1 -- 172.21.15.92:0/189885373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e7c003a90 msgr2=0x7f6e7c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.767+0000 7f6e99b44700 1 --2- 172.21.15.92:0/189885373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e7c003a90 0x7f6e7c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.768+0000 7f6e99b44700 1 -- 172.21.15.92:0/189885373 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c003570 con 0x7f6e7c004810 2023-10-06T10:43:31.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.768+0000 7f6e99b44700 1 --2- 172.21.15.92:0/189885373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e7c004810 0x7f6e7c004c80 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800c5c0 tx=0x7f6e88007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab89e7bf27909f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.768+0000 7f6e99343700 1 -- 172.21.15.92:0/189885373 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8800cdc0 con 0x7f6e7c004810 2023-10-06T10:43:31.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.768+0000 7f6e99343700 1 -- 172.21.15.92:0/189885373 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e88005920 con 0x7f6e7c004810 2023-10-06T10:43:31.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.768+0000 7f6e99343700 1 -- 172.21.15.92:0/189885373 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e88005ce0 con 0x7f6e7c004810 2023-10-06T10:43:31.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.768+0000 7f6e9c5a9700 1 -- 172.21.15.92:0/189885373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e7c004810 msgr2=0x7f6e7c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.768+0000 7f6e9c5a9700 1 --2- 172.21.15.92:0/189885373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e7c004810 0x7f6e7c004c80 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800c5c0 tx=0x7f6e88007a50 comp rx=0 tx=0).stop 2023-10-06T10:43:31.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.768+0000 7f6e9ab46700 1 -- 172.21.15.92:0/189885373 reap_dead start 2023-10-06T10:43:31.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.768+0000 7f6e9c5a9700 1 -- 172.21.15.92:0/189885373 shutdown_connections 2023-10-06T10:43:31.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.768+0000 7f6e9c5a9700 1 -- 172.21.15.92:0/189885373 >> 172.21.15.92:0/189885373 conn(0x7f6e94099700 msgr2=0x7f6e9406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:31.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.768+0000 7f6e9c5a9700 1 -- 172.21.15.92:0/189885373 shutdown_connections 2023-10-06T10:43:31.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.768+0000 7f6e9c5a9700 1 -- 172.21.15.92:0/189885373 wait complete. 2023-10-06T10:43:31.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.769+0000 7f6e9c5a9700 1 Processor -- start 2023-10-06T10:43:31.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.770+0000 7f6e9c5a9700 1 -- start start 2023-10-06T10:43:31.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.770+0000 7f6e9c5a9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e7c004810 0x7f6e9414f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.770+0000 7f6e9c5a9700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e9416a910 0x7f6e94156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.770+0000 7f6e9c5a9700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e94156c50 0x7f6e9416d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.770+0000 7f6e9c5a9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9416e1f0 con 0x7f6e7c004810 2023-10-06T10:43:31.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.770+0000 7f6e9c5a9700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e94169c20 con 0x7f6e9416a910 2023-10-06T10:43:31.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.770+0000 7f6e9c5a9700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e941570d0 con 0x7f6e94156c50 2023-10-06T10:43:31.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.770+0000 7f6e9a345700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e7c004810 0x7f6e9414f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.770+0000 7f6e9a345700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e7c004810 0x7f6e9414f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60970/0 (socket says 172.21.15.92:60970) 2023-10-06T10:43:31.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.770+0000 7f6e9a345700 1 -- 172.21.15.92:0/1385859609 learned_addr learned my addr 172.21.15.92:0/1385859609 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.770+0000 7f6e99b44700 1 --2- 172.21.15.92:0/1385859609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e9416a910 0x7f6e94156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.770+0000 7f6e9ab46700 1 --2- 172.21.15.92:0/1385859609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e94156c50 0x7f6e9416d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.771+0000 7f6e9a345700 1 -- 172.21.15.92:0/1385859609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e94156c50 msgr2=0x7f6e9416d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.771+0000 7f6e9a345700 1 --2- 172.21.15.92:0/1385859609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e94156c50 0x7f6e9416d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.771+0000 7f6e9a345700 1 -- 172.21.15.92:0/1385859609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e9416a910 msgr2=0x7f6e94156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.771+0000 7f6e9a345700 1 --2- 172.21.15.92:0/1385859609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e9416a910 0x7f6e94156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.771+0000 7f6e9a345700 1 -- 172.21.15.92:0/1385859609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9416a480 con 0x7f6e7c004810 2023-10-06T10:43:31.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.771+0000 7f6e9a345700 1 --2- 172.21.15.92:0/1385859609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e7c004810 0x7f6e9414f3e0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00c440 tx=0x7f6e8c001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.771+0000 7f6e927fc700 1 -- 172.21.15.92:0/1385859609 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c014480 con 0x7f6e7c004810 2023-10-06T10:43:31.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.771+0000 7f6e927fc700 1 -- 172.21.15.92:0/1385859609 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8c0145e0 con 0x7f6e7c004810 2023-10-06T10:43:31.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.771+0000 7f6e9c5a9700 1 -- 172.21.15.92:0/1385859609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e9416df10 con 0x7f6e7c004810 2023-10-06T10:43:31.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.771+0000 7f6e927fc700 1 -- 172.21.15.92:0/1385859609 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c0149a0 con 0x7f6e7c004810 2023-10-06T10:43:31.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.771+0000 7f6e9c5a9700 1 -- 172.21.15.92:0/1385859609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9416a110 con 0x7f6e7c004810 2023-10-06T10:43:31.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.772+0000 7f6e927fc700 1 -- 172.21.15.92:0/1385859609 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6e8c01d400 con 0x7f6e7c004810 2023-10-06T10:43:31.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.773+0000 7f6e927fc700 1 --2- 172.21.15.92:0/1385859609 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6e80066f30 0x7f6e800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.773+0000 7f6e99b44700 1 --2- 172.21.15.92:0/1385859609 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6e80066f30 0x7f6e800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.773+0000 7f6e927fc700 1 -- 172.21.15.92:0/1385859609 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6e8c0850d0 con 0x7f6e7c004810 2023-10-06T10:43:31.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.773+0000 7f6e99b44700 1 --2- 172.21.15.92:0/1385859609 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6e80066f30 0x7f6e800693e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6e88007e70 tx=0x7f6e88015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.775+0000 7f6ea58a1b80 1 Processor -- start 2023-10-06T10:43:31.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.775+0000 7f6ea58a1b80 1 -- start start 2023-10-06T10:43:31.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.776+0000 7f6ea58a1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e755195ac0 0x55e755195e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.776+0000 7f6ea58a1b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e7551519d0 0x55e75514e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.776+0000 7f6ea58a1b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e75514fc00 0x55e75514eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.776+0000 7f6ea58a1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7550cecb0 con 0x55e755195ac0 2023-10-06T10:43:31.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.776+0000 7f6ea58a1b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7550a5aa0 con 0x55e75514fc00 2023-10-06T10:43:31.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.776+0000 7f6ea58a1b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7550a32a0 con 0x55e7551519d0 2023-10-06T10:43:31.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.776+0000 7f6e9a345700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e755195ac0 0x55e755195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.776+0000 7f6e9a345700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e755195ac0 0x55e755195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60978/0 (socket says 172.21.15.92:60978) 2023-10-06T10:43:31.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.776+0000 7f6e9a345700 1 -- 172.21.15.92:0/2535480856 learned_addr learned my addr 172.21.15.92:0/2535480856 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.776+0000 7f6e99b44700 1 --2- 172.21.15.92:0/2535480856 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e7551519d0 0x55e75514e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.776+0000 7f6e9ab46700 1 --2- 172.21.15.92:0/2535480856 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e75514fc00 0x55e75514eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.776+0000 7f6e9a345700 1 -- 172.21.15.92:0/2535480856 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e7551519d0 msgr2=0x55e75514e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.776+0000 7f6e9a345700 1 --2- 172.21.15.92:0/2535480856 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e7551519d0 0x55e75514e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.776+0000 7f6e9a345700 1 -- 172.21.15.92:0/2535480856 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e75514fc00 msgr2=0x55e75514eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.776+0000 7f6e9a345700 1 --2- 172.21.15.92:0/2535480856 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e75514fc00 0x55e75514eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.776+0000 7f6e9a345700 1 -- 172.21.15.92:0/2535480856 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e754f392a0 con 0x55e755195ac0 2023-10-06T10:43:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.776+0000 7f6e9ab46700 1 --2- 172.21.15.92:0/2535480856 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e75514fc00 0x55e75514eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.776+0000 7f6e9a345700 1 --2- 172.21.15.92:0/2535480856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e755195ac0 0x55e755195e90 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c0832b0 tx=0x7f6e8c083210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb797975ba480023 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.776+0000 7f6e7b7fe700 1 -- 172.21.15.92:0/2535480856 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c058f70 con 0x55e755195ac0 2023-10-06T10:43:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.776+0000 7f6e7b7fe700 1 -- 172.21.15.92:0/2535480856 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8c05cc60 con 0x55e755195ac0 2023-10-06T10:43:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.776+0000 7f6e7b7fe700 1 -- 172.21.15.92:0/2535480856 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c054110 con 0x55e755195ac0 2023-10-06T10:43:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.777+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/2535480856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e755195ac0 msgr2=0x55e755195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.777+0000 7f6ea58a1b80 1 --2- 172.21.15.92:0/2535480856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e755195ac0 0x55e755195e90 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c0832b0 tx=0x7f6e8c083210 comp rx=0 tx=0).stop 2023-10-06T10:43:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.777+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/2535480856 shutdown_connections 2023-10-06T10:43:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.777+0000 7f6ea58a1b80 1 --2- 172.21.15.92:0/2535480856 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e75514fc00 0x55e75514eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.777+0000 7f6ea58a1b80 1 --2- 172.21.15.92:0/2535480856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e755195ac0 0x55e755195e90 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.777+0000 7f6ea58a1b80 1 --2- 172.21.15.92:0/2535480856 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e7551519d0 0x55e75514e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.777+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/2535480856 >> 172.21.15.92:0/2535480856 conn(0x55e7551500a0 msgr2=0x55e755150470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.777+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/2535480856 shutdown_connections 2023-10-06T10:43:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.777+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/2535480856 wait complete. 2023-10-06T10:43:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.778+0000 7f6ea58a1b80 1 -- start start 2023-10-06T10:43:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.778+0000 7f6ea58a1b80 1 Processor -- start 2023-10-06T10:43:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.778+0000 7f6ea58a1b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e75514fc00 0x55e755192a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6ea58a1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7551519d0 0x55e75518ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6ea58a1b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e755195ac0 0x55e755190880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6ea58a1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e75509f090 con 0x55e7551519d0 2023-10-06T10:43:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6ea58a1b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e75504a8c0 con 0x55e75514fc00 2023-10-06T10:43:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6ea58a1b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e755065ed0 con 0x55e755195ac0 2023-10-06T10:43:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6ea58a1b80 10 client.0 _ll_register_callbacks cb 0x55e754f3bf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6e99b44700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7551519d0 0x55e75518ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6e99b44700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7551519d0 0x55e75518ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60980/0 (socket says 172.21.15.92:60980) 2023-10-06T10:43:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6e99b44700 1 -- 172.21.15.92:0/3304427869 learned_addr learned my addr 172.21.15.92:0/3304427869 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6e9a345700 1 --2- 172.21.15.92:0/3304427869 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e75514fc00 0x55e755192a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6e9ab46700 1 --2- 172.21.15.92:0/3304427869 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e755195ac0 0x55e755190880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6e99b44700 1 -- 172.21.15.92:0/3304427869 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e755195ac0 msgr2=0x55e755190880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6e99b44700 1 --2- 172.21.15.92:0/3304427869 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e755195ac0 0x55e755190880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6e99b44700 1 -- 172.21.15.92:0/3304427869 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e75514fc00 msgr2=0x55e755192a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6e99b44700 1 --2- 172.21.15.92:0/3304427869 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e75514fc00 0x55e755192a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6e99b44700 1 -- 172.21.15.92:0/3304427869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9416a110 con 0x55e7551519d0 2023-10-06T10:43:31.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6e99b44700 1 --2- 172.21.15.92:0/3304427869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7551519d0 0x55e75518ca00 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c00bb30 tx=0x7f6e88021070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22e46b00f903904 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6e78ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:31.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6e78ff9700 1 -- 172.21.15.92:0/3304427869 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e88005e20 con 0x55e7551519d0 2023-10-06T10:43:31.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6e78ff9700 1 -- 172.21.15.92:0/3304427869 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6e880076c0 con 0x55e7551519d0 2023-10-06T10:43:31.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/3304427869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7551436c0 con 0x55e7551519d0 2023-10-06T10:43:31.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6e78ff9700 1 -- 172.21.15.92:0/3304427869 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e8801f410 con 0x55e7551519d0 2023-10-06T10:43:31.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6ea58a1b80 10 client.7512 fetch_fsmap learned FSMap version 74 2023-10-06T10:43:31.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/3304427869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x55e754edc010 con 0x55e7551519d0 2023-10-06T10:43:31.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.779+0000 7f6e78ff9700 1 -- 172.21.15.92:0/3304427869 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e7551436c0 con 0x55e7551519d0 2023-10-06T10:43:31.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.780+0000 7f6e78ff9700 1 -- 172.21.15.92:0/3304427869 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 74) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f6e880343d0 con 0x55e7551519d0 2023-10-06T10:43:31.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.780+0000 7f6ea58a1b80 10 client.7512 fetch_fsmap finished waiting for FSMap version 74 2023-10-06T10:43:31.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.780+0000 7f6ea58a1b80 10 client.7512 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:43:31.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.780+0000 7f6ea58a1b80 20 client.7512 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:31.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.780+0000 7f6ea58a1b80 1 --2- 172.21.15.92:0/3304427869 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55e755179180 0x55e755179550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.780+0000 7f6ea58a1b80 4 client.7512 mds_command: new command op to 7149 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:43:31.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.780+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/3304427869 --> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7551205f0 con 0x55e755179180 2023-10-06T10:43:31.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.780+0000 7f6e9a345700 1 --2- 172.21.15.92:0/3304427869 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55e755179180 0x55e755179550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.780+0000 7f6e9a345700 1 --2- 172.21.15.92:0/3304427869 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55e755179180 0x55e755179550 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=60709e03816b22ae server_cookie=27879c76d2448999 in_seq=0 out_seq=0 2023-10-06T10:43:31.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.780+0000 7f6e78ff9700 10 client.7512 ms_handle_connect on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:31.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.780+0000 7f6e78ff9700 1 -- 172.21.15.92:0/3304427869 <== mds.1 v2:172.21.15.92:6834/3268908749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55e7551205f0 con 0x55e755179180 2023-10-06T10:43:31.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.780+0000 7f6e78ff9700 10 client.7512 handle_command_reply: tid=0 2023-10-06T10:43:31.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.780+0000 7f6ea58a1b80 1 client.7512 shutdown 2023-10-06T10:43:31.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.781+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/3304427869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7551519d0 msgr2=0x55e75518ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.781+0000 7f6ea58a1b80 1 --2- 172.21.15.92:0/3304427869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7551519d0 0x55e75518ca00 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c00bb30 tx=0x7f6e88021070 comp rx=0 tx=0).stop 2023-10-06T10:43:31.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.781+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/3304427869 shutdown_connections 2023-10-06T10:43:31.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.781+0000 7f6ea58a1b80 1 --2- 172.21.15.92:0/3304427869 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55e755179180 0x55e755179550 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.781+0000 7f6ea58a1b80 1 --2- 172.21.15.92:0/3304427869 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e755195ac0 0x55e755190880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.781+0000 7f6e78ff9700 0 client.7512 ms_handle_reset on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:31.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.781+0000 7f6ea58a1b80 1 --2- 172.21.15.92:0/3304427869 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e75514fc00 0x55e755192a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.781+0000 7f6ea58a1b80 1 --2- 172.21.15.92:0/3304427869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7551519d0 0x55e75518ca00 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.781+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/3304427869 >> 172.21.15.92:0/3304427869 conn(0x55e7551500a0 msgr2=0x55e75514b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:31.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.781+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/3304427869 shutdown_connections 2023-10-06T10:43:31.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.781+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/3304427869 wait complete. 2023-10-06T10:43:31.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.781+0000 7f6ea58a1b80 20 client.7512 trim_cache size 0 max 16384 2023-10-06T10:43:31.807 DEBUG:teuthology.orchestra.run:got remote process result: 18 2023-10-06T10:43:31.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.796+0000 7f6ea58a1b80 1 Processor -- start 2023-10-06T10:43:31.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.796+0000 7f6ea58a1b80 1 -- start start 2023-10-06T10:43:31.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.797+0000 7f6ea58a1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75514fc00 0x55e7551930c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.797+0000 7f6ea58a1b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e7551519d0 0x55e755193600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.797+0000 7f6ea58a1b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e755195ac0 0x55e755149770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.797+0000 7f6ea58a1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e754f21e60 con 0x55e75514fc00 2023-10-06T10:43:31.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.797+0000 7f6ea58a1b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e755178fb0 con 0x55e7551519d0 2023-10-06T10:43:31.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.797+0000 7f6ea58a1b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e755179930 con 0x55e755195ac0 2023-10-06T10:43:31.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.797+0000 7f6e9a345700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75514fc00 0x55e7551930c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.797+0000 7f6e9a345700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75514fc00 0x55e7551930c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60982/0 (socket says 172.21.15.92:60982) 2023-10-06T10:43:31.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.797+0000 7f6e9a345700 1 -- 172.21.15.92:0/1995821782 learned_addr learned my addr 172.21.15.92:0/1995821782 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.797+0000 7f6e9a345700 1 -- 172.21.15.92:0/1995821782 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e755195ac0 msgr2=0x55e755149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.797+0000 7f6e9a345700 1 --2- 172.21.15.92:0/1995821782 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e755195ac0 0x55e755149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.797+0000 7f6e9a345700 1 -- 172.21.15.92:0/1995821782 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e7551519d0 msgr2=0x55e755193600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.797+0000 7f6e9a345700 1 --2- 172.21.15.92:0/1995821782 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e7551519d0 0x55e755193600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.797+0000 7f6e9a345700 1 -- 172.21.15.92:0/1995821782 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e75506d7b0 con 0x55e75514fc00 2023-10-06T10:43:31.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.797+0000 7f6e9a345700 1 --2- 172.21.15.92:0/1995821782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75514fc00 0x55e7551930c0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00c410 tx=0x7f6e8c07f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fce5d5deb6ecb82 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.797+0000 7f6e7bfff700 1 -- 172.21.15.92:0/1995821782 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c0561e0 con 0x55e75514fc00 2023-10-06T10:43:31.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.798+0000 7f6e7bfff700 1 -- 172.21.15.92:0/1995821782 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8c053eb0 con 0x55e75514fc00 2023-10-06T10:43:31.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.798+0000 7f6e7bfff700 1 -- 172.21.15.92:0/1995821782 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c054110 con 0x55e75514fc00 2023-10-06T10:43:31.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.798+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/1995821782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75514fc00 msgr2=0x55e7551930c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.798+0000 7f6ea58a1b80 1 --2- 172.21.15.92:0/1995821782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75514fc00 0x55e7551930c0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00c410 tx=0x7f6e8c07f040 comp rx=0 tx=0).stop 2023-10-06T10:43:31.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.798+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/1995821782 shutdown_connections 2023-10-06T10:43:31.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.798+0000 7f6ea58a1b80 1 --2- 172.21.15.92:0/1995821782 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e755195ac0 0x55e755149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.798+0000 7f6ea58a1b80 1 --2- 172.21.15.92:0/1995821782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75514fc00 0x55e7551930c0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.798+0000 7f6ea58a1b80 1 --2- 172.21.15.92:0/1995821782 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e7551519d0 0x55e755193600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.798+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/1995821782 >> 172.21.15.92:0/1995821782 conn(0x55e7551500a0 msgr2=0x55e7551ae940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:31.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.798+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/1995821782 shutdown_connections 2023-10-06T10:43:31.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.798+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/1995821782 wait complete. 2023-10-06T10:43:31.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.799+0000 7f6ea58a1b80 1 -- start start 2023-10-06T10:43:31.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.799+0000 7f6ea58a1b80 1 Processor -- start 2023-10-06T10:43:31.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6ea58a1b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e75514fc00 0x55e75518fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6ea58a1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7551519d0 0x55e7551fdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6ea58a1b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e755195ac0 0x55e75514e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6ea58a1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7551fea70 con 0x55e7551519d0 2023-10-06T10:43:31.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6ea58a1b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e75518ef20 con 0x55e75514fc00 2023-10-06T10:43:31.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6ea58a1b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e75519d700 con 0x55e755195ac0 2023-10-06T10:43:31.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6ea58a1b80 10 client.0 _ll_register_callbacks cb 0x55e754f3bf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:31.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6e99b44700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7551519d0 0x55e7551fdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6e99b44700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7551519d0 0x55e7551fdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60992/0 (socket says 172.21.15.92:60992) 2023-10-06T10:43:31.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6e99b44700 1 -- 172.21.15.92:0/736024491 learned_addr learned my addr 172.21.15.92:0/736024491 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6e9ab46700 1 --2- 172.21.15.92:0/736024491 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e755195ac0 0x55e75514e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6e9a345700 1 --2- 172.21.15.92:0/736024491 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e75514fc00 0x55e75518fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6e99b44700 1 -- 172.21.15.92:0/736024491 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e755195ac0 msgr2=0x55e75514e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6e99b44700 1 --2- 172.21.15.92:0/736024491 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e755195ac0 0x55e75514e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6e99b44700 1 -- 172.21.15.92:0/736024491 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e75514fc00 msgr2=0x55e75518fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6e99b44700 1 --2- 172.21.15.92:0/736024491 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e75514fc00 0x55e75518fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6e99b44700 1 -- 172.21.15.92:0/736024491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e754f392a0 con 0x55e7551519d0 2023-10-06T10:43:31.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6e99b44700 1 --2- 172.21.15.92:0/736024491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7551519d0 0x55e7551fdf10 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800c5c0 tx=0x7f6e8802d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4dd983ced2dfa6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6e78ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:31.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6e78ff9700 1 -- 172.21.15.92:0/736024491 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e88034b70 con 0x55e7551519d0 2023-10-06T10:43:31.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6e78ff9700 1 -- 172.21.15.92:0/736024491 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6e88005e20 con 0x55e7551519d0 2023-10-06T10:43:31.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/736024491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e8c05cc60 con 0x55e7551519d0 2023-10-06T10:43:31.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6e78ff9700 1 -- 172.21.15.92:0/736024491 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e88000ce0 con 0x55e7551519d0 2023-10-06T10:43:31.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6ea58a1b80 10 client.7518 fetch_fsmap learned FSMap version 74 2023-10-06T10:43:31.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/736024491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x55e7550544a0 con 0x55e7551519d0 2023-10-06T10:43:31.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.800+0000 7f6e78ff9700 1 -- 172.21.15.92:0/736024491 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8c05cc60 con 0x55e7551519d0 2023-10-06T10:43:31.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6e78ff9700 1 -- 172.21.15.92:0/736024491 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 74) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f6e880343d0 con 0x55e7551519d0 2023-10-06T10:43:31.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6ea58a1b80 10 client.7518 fetch_fsmap finished waiting for FSMap version 74 2023-10-06T10:43:31.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6ea58a1b80 10 client.7518 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:43:31.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6ea58a1b80 20 client.7518 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:31.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6ea58a1b80 1 --2- 172.21.15.92:0/736024491 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55e755192510 0x55e7551930c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6ea58a1b80 4 client.7518 mds_command: new command op to 7149 tid=0[{"prefix": "scrub pause"}] 2023-10-06T10:43:31.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/736024491 --> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55e755041e30 con 0x55e755192510 2023-10-06T10:43:31.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6e9a345700 1 --2- 172.21.15.92:0/736024491 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55e755192510 0x55e7551930c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6e78ff9700 10 client.7518 ms_handle_connect on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:31.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6e9a345700 1 --2- 172.21.15.92:0/736024491 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55e755192510 0x55e7551930c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2fb1592509e267ea server_cookie=89a5c6e01eb2757c in_seq=0 out_seq=0 2023-10-06T10:43:31.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6e78ff9700 1 -- 172.21.15.92:0/736024491 <== mds.1 v2:172.21.15.92:6834/3268908749 1 ==== command_reply(tid 0: -18 Not rank 0) v1 ==== 18+0+0 (crc 0 0 0) 0x55e755041e30 con 0x55e755192510 2023-10-06T10:43:31.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6e78ff9700 10 client.7518 handle_command_reply: tid=0 2023-10-06T10:43:31.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6ea58a1b80 1 client.7518 shutdown 2023-10-06T10:43:31.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/736024491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7551519d0 msgr2=0x55e7551fdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6ea58a1b80 1 --2- 172.21.15.92:0/736024491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7551519d0 0x55e7551fdf10 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800c5c0 tx=0x7f6e8802d040 comp rx=0 tx=0).stop 2023-10-06T10:43:31.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/736024491 shutdown_connections 2023-10-06T10:43:31.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6ea58a1b80 1 --2- 172.21.15.92:0/736024491 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x55e755192510 0x55e7551930c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6ea58a1b80 1 --2- 172.21.15.92:0/736024491 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e755195ac0 0x55e75514e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6ea58a1b80 1 --2- 172.21.15.92:0/736024491 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e75514fc00 0x55e75518fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6e78ff9700 0 client.7518 ms_handle_reset on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:31.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6ea58a1b80 1 --2- 172.21.15.92:0/736024491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7551519d0 0x55e7551fdf10 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/736024491 >> 172.21.15.92:0/736024491 conn(0x55e7551500a0 msgr2=0x55e75514a910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:31.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/736024491 shutdown_connections 2023-10-06T10:43:31.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.801+0000 7f6ea58a1b80 1 -- 172.21.15.92:0/736024491 wait complete. 2023-10-06T10:43:31.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.802+0000 7f6ea58a1b80 20 client.7518 trim_cache size 0 max 16384 2023-10-06T10:43:31.815 INFO:teuthology.orchestra.run.smithi092.stderr:Error EXDEV: Not rank 0 2023-10-06T10:43:31.815 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub resume 2023-10-06T10:43:31.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.929+0000 7fa4d2c33700 1 Processor -- start 2023-10-06T10:43:31.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.929+0000 7fa4d2c33700 1 -- start start 2023-10-06T10:43:31.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.930+0000 7fa4d2c33700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa4cc15a6b0 0x7fa4cc0b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.930+0000 7fa4d2c33700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa4cc0b24b0 0x7fa4cc0ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.930+0000 7fa4d2c33700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4cc0b1ae0 con 0x7fa4cc15aa80 2023-10-06T10:43:31.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.930+0000 7fa4d2c33700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4cc0cb630 con 0x7fa4cc15a6b0 2023-10-06T10:43:31.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.930+0000 7fa4d2c33700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4cc0b11b0 con 0x7fa4cc0b24b0 2023-10-06T10:43:31.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.930+0000 7fa4d09cf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa4cc15a6b0 0x7fa4cc0b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.930+0000 7fa4d09cf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa4cc15a6b0 0x7fa4cc0b1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32776/0 (socket says 172.21.15.92:32776) 2023-10-06T10:43:31.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.930+0000 7fa4d09cf700 1 -- 172.21.15.92:0/3211875734 learned_addr learned my addr 172.21.15.92:0/3211875734 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.930+0000 7fa4c3fff700 1 --2- 172.21.15.92:0/3211875734 >> v2:172.21.15.124:3300/0 conn(0x7fa4cc0b24b0 0x7fa4cc0ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.931+0000 7fa4d09cf700 1 -- 172.21.15.92:0/3211875734 >> v2:172.21.15.124:3300/0 conn(0x7fa4cc0b24b0 msgr2=0x7fa4cc0ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.931+0000 7fa4d09cf700 1 --2- 172.21.15.92:0/3211875734 >> v2:172.21.15.124:3300/0 conn(0x7fa4cc0b24b0 0x7fa4cc0ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.931+0000 7fa4d09cf700 1 -- 172.21.15.92:0/3211875734 >> v1:172.21.15.124:6789/0 conn(0x7fa4cc15aa80 legacy=0x7fa4cc0b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.931+0000 7fa4d09cf700 1 -- 172.21.15.92:0/3211875734 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4cc0b1490 con 0x7fa4cc15a6b0 2023-10-06T10:43:31.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.931+0000 7fa4d09cf700 1 --2- 172.21.15.92:0/3211875734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4cc15a6b0 0x7fa4cc0b1f70 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc06fd50 tx=0x7fa4c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f842877fce957bca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.931+0000 7fa4c37fe700 1 -- 172.21.15.92:0/3211875734 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c800e070 con 0x7fa4cc15a6b0 2023-10-06T10:43:31.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.931+0000 7fa4c37fe700 1 -- 172.21.15.92:0/3211875734 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c800ba90 con 0x7fa4cc15a6b0 2023-10-06T10:43:31.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.931+0000 7fa4c37fe700 1 -- 172.21.15.92:0/3211875734 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c80092e0 con 0x7fa4cc15a6b0 2023-10-06T10:43:31.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.931+0000 7fa4d2c33700 1 -- 172.21.15.92:0/3211875734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4cc15a6b0 msgr2=0x7fa4cc0b1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.931+0000 7fa4d2c33700 1 --2- 172.21.15.92:0/3211875734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4cc15a6b0 0x7fa4cc0b1f70 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc06fd50 tx=0x7fa4c8009840 comp rx=0 tx=0).stop 2023-10-06T10:43:31.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.931+0000 7fa4d2c33700 1 -- 172.21.15.92:0/3211875734 shutdown_connections 2023-10-06T10:43:31.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.931+0000 7fa4d2c33700 1 --2- 172.21.15.92:0/3211875734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4cc15a6b0 0x7fa4cc0b1f70 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.931+0000 7fa4d2c33700 1 --2- 172.21.15.92:0/3211875734 >> v2:172.21.15.124:3300/0 conn(0x7fa4cc0b24b0 0x7fa4cc0ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.931+0000 7fa4d2c33700 1 -- 172.21.15.92:0/3211875734 >> 172.21.15.92:0/3211875734 conn(0x7fa4cc06b600 msgr2=0x7fa4cc06ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:31.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.931+0000 7fa4d2c33700 1 -- 172.21.15.92:0/3211875734 shutdown_connections 2023-10-06T10:43:31.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.931+0000 7fa4d2c33700 1 -- 172.21.15.92:0/3211875734 wait complete. 2023-10-06T10:43:31.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.932+0000 7fa4d2c33700 1 Processor -- start 2023-10-06T10:43:31.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.933+0000 7fa4d2c33700 1 -- start start 2023-10-06T10:43:31.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.933+0000 7fa4d2c33700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4cc0b24b0 0x7fa4cc0ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.933+0000 7fa4d2c33700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4cc15a6b0 0x7fa4cc1519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.933+0000 7fa4d2c33700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4cc15aa80 0x7fa4cc151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.933+0000 7fa4d2c33700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc0af6c0 con 0x7fa4cc15a6b0 2023-10-06T10:43:31.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.933+0000 7fa4d2c33700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc0aeab0 con 0x7fa4cc15aa80 2023-10-06T10:43:31.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.933+0000 7fa4d2c33700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc0aed90 con 0x7fa4cc0b24b0 2023-10-06T10:43:31.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.934+0000 7fa4c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4cc15a6b0 0x7fa4cc1519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.934+0000 7fa4c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4cc15a6b0 0x7fa4cc1519c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32784/0 (socket says 172.21.15.92:32784) 2023-10-06T10:43:31.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.934+0000 7fa4c3fff700 1 -- 172.21.15.92:0/2804284883 learned_addr learned my addr 172.21.15.92:0/2804284883 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.934+0000 7fa4d09cf700 1 --2- 172.21.15.92:0/2804284883 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4cc0b24b0 0x7fa4cc0ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.934+0000 7fa4d11d0700 1 --2- 172.21.15.92:0/2804284883 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4cc15aa80 0x7fa4cc151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.934+0000 7fa4c3fff700 1 -- 172.21.15.92:0/2804284883 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4cc0b24b0 msgr2=0x7fa4cc0ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.934+0000 7fa4c3fff700 1 --2- 172.21.15.92:0/2804284883 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4cc0b24b0 0x7fa4cc0ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.934+0000 7fa4c3fff700 1 -- 172.21.15.92:0/2804284883 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4cc15aa80 msgr2=0x7fa4cc151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.934+0000 7fa4c3fff700 1 --2- 172.21.15.92:0/2804284883 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4cc15aa80 0x7fa4cc151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.934+0000 7fa4c3fff700 1 -- 172.21.15.92:0/2804284883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c8009ae0 con 0x7fa4cc15a6b0 2023-10-06T10:43:31.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.934+0000 7fa4d09cf700 1 --2- 172.21.15.92:0/2804284883 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4cc0b24b0 0x7fa4cc0ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:31.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.934+0000 7fa4c3fff700 1 --2- 172.21.15.92:0/2804284883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4cc15a6b0 0x7fa4cc1519c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc00d890 tx=0x7fa4bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.934+0000 7fa4c0ff9700 1 -- 172.21.15.92:0/2804284883 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc00b600 con 0x7fa4cc15a6b0 2023-10-06T10:43:31.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.934+0000 7fa4c0ff9700 1 -- 172.21.15.92:0/2804284883 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4bc00b760 con 0x7fa4cc15a6b0 2023-10-06T10:43:31.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.934+0000 7fa4d2c33700 1 -- 172.21.15.92:0/2804284883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4cc0af070 con 0x7fa4cc15a6b0 2023-10-06T10:43:31.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.934+0000 7fa4c0ff9700 1 -- 172.21.15.92:0/2804284883 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc00bae0 con 0x7fa4cc15a6b0 2023-10-06T10:43:31.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.934+0000 7fa4d2c33700 1 -- 172.21.15.92:0/2804284883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4cc0ae7c0 con 0x7fa4cc15a6b0 2023-10-06T10:43:31.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.935+0000 7fa4c0ff9700 1 -- 172.21.15.92:0/2804284883 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa4bc01e460 con 0x7fa4cc15a6b0 2023-10-06T10:43:31.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.936+0000 7fa4c0ff9700 1 --2- 172.21.15.92:0/2804284883 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa4b8066f30 0x7fa4b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.936+0000 7fa4c0ff9700 1 -- 172.21.15.92:0/2804284883 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa4bc084eb0 con 0x7fa4cc15a6b0 2023-10-06T10:43:31.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.936+0000 7fa4d09cf700 1 --2- 172.21.15.92:0/2804284883 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa4b8066f30 0x7fa4b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.936+0000 7fa4d09cf700 1 --2- 172.21.15.92:0/2804284883 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa4b8066f30 0x7fa4b80693e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8009880 tx=0x7fa4c8017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.938+0000 7fa4dbf2bb80 1 Processor -- start 2023-10-06T10:43:31.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.938+0000 7fa4dbf2bb80 1 -- start start 2023-10-06T10:43:31.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.939+0000 7fa4dbf2bb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a14d061f0 0x559a14d20900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.939+0000 7fa4dbf2bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14d23a40 0x559a14d20e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.939+0000 7fa4dbf2bb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a14cdac10 0x559a14cd95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.939+0000 7fa4dbf2bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a14c59cc0 con 0x559a14d23a40 2023-10-06T10:43:31.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.939+0000 7fa4dbf2bb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a14c30ab0 con 0x559a14cdac10 2023-10-06T10:43:31.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.939+0000 7fa4dbf2bb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a14c2e2b0 con 0x559a14d061f0 2023-10-06T10:43:31.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.939+0000 7fa4c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14d23a40 0x559a14d20e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.939+0000 7fa4c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14d23a40 0x559a14d20e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32786/0 (socket says 172.21.15.92:32786) 2023-10-06T10:43:31.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.939+0000 7fa4c3fff700 1 -- 172.21.15.92:0/93621791 learned_addr learned my addr 172.21.15.92:0/93621791 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.939+0000 7fa4d09cf700 1 --2- 172.21.15.92:0/93621791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a14d061f0 0x559a14d20900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.939+0000 7fa4d11d0700 1 --2- 172.21.15.92:0/93621791 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a14cdac10 0x559a14cd95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.939+0000 7fa4c3fff700 1 -- 172.21.15.92:0/93621791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a14d061f0 msgr2=0x559a14d20900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.939+0000 7fa4c3fff700 1 --2- 172.21.15.92:0/93621791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a14d061f0 0x559a14d20900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.939+0000 7fa4c3fff700 1 -- 172.21.15.92:0/93621791 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a14cdac10 msgr2=0x559a14cd95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.939+0000 7fa4c3fff700 1 --2- 172.21.15.92:0/93621791 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a14cdac10 0x559a14cd95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.939+0000 7fa4c3fff700 1 -- 172.21.15.92:0/93621791 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a14ac42b0 con 0x559a14d23a40 2023-10-06T10:43:31.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.939+0000 7fa4d09cf700 1 --2- 172.21.15.92:0/93621791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a14d061f0 0x559a14d20900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:31.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.939+0000 7fa4c3fff700 1 --2- 172.21.15.92:0/93621791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14d23a40 0x559a14d20e40 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc052040 tx=0x7fa4bc083090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b43ebbe0345b2b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.939+0000 7fa4b1ffb700 1 -- 172.21.15.92:0/93621791 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc055680 con 0x559a14d23a40 2023-10-06T10:43:31.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.939+0000 7fa4b1ffb700 1 -- 172.21.15.92:0/93621791 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4bc05cae0 con 0x559a14d23a40 2023-10-06T10:43:31.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.939+0000 7fa4b1ffb700 1 -- 172.21.15.92:0/93621791 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc053ee0 con 0x559a14d23a40 2023-10-06T10:43:31.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.940+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/93621791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14d23a40 msgr2=0x559a14d20e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.940+0000 7fa4dbf2bb80 1 --2- 172.21.15.92:0/93621791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14d23a40 0x559a14d20e40 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc052040 tx=0x7fa4bc083090 comp rx=0 tx=0).stop 2023-10-06T10:43:31.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.940+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/93621791 shutdown_connections 2023-10-06T10:43:31.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.940+0000 7fa4dbf2bb80 1 --2- 172.21.15.92:0/93621791 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a14cdac10 0x559a14cd95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.940+0000 7fa4dbf2bb80 1 --2- 172.21.15.92:0/93621791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a14d061f0 0x559a14d20900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.940+0000 7fa4dbf2bb80 1 --2- 172.21.15.92:0/93621791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14d23a40 0x559a14d20e40 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.940+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/93621791 >> 172.21.15.92:0/93621791 conn(0x559a14cdb0b0 msgr2=0x559a14cdb480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:31.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.940+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/93621791 shutdown_connections 2023-10-06T10:43:31.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.940+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/93621791 wait complete. 2023-10-06T10:43:31.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.940+0000 7fa4dbf2bb80 1 -- start start 2023-10-06T10:43:31.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.941+0000 7fa4dbf2bb80 1 Processor -- start 2023-10-06T10:43:31.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.941+0000 7fa4dbf2bb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a14cdac10 0x559a14d19970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.941+0000 7fa4dbf2bb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a14d061f0 0x559a14cd5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.941+0000 7fa4dbf2bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14d23a40 0x559a14cd6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.941+0000 7fa4dbf2bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a14c2a0a0 con 0x559a14d23a40 2023-10-06T10:43:31.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.941+0000 7fa4dbf2bb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a14bd58d0 con 0x559a14d061f0 2023-10-06T10:43:31.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.941+0000 7fa4dbf2bb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a14bf0ee0 con 0x559a14cdac10 2023-10-06T10:43:31.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.941+0000 7fa4dbf2bb80 10 client.0 _ll_register_callbacks cb 0x559a14ac6f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:31.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.941+0000 7fa4d11d0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14d23a40 0x559a14cd6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.942+0000 7fa4d11d0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14d23a40 0x559a14cd6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32800/0 (socket says 172.21.15.92:32800) 2023-10-06T10:43:31.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.942+0000 7fa4d11d0700 1 -- 172.21.15.92:0/2189142965 learned_addr learned my addr 172.21.15.92:0/2189142965 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.942+0000 7fa4d09cf700 1 --2- 172.21.15.92:0/2189142965 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a14cdac10 0x559a14d19970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.942+0000 7fa4c3fff700 1 --2- 172.21.15.92:0/2189142965 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a14d061f0 0x559a14cd5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.942+0000 7fa4d11d0700 1 -- 172.21.15.92:0/2189142965 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a14cdac10 msgr2=0x559a14d19970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.942+0000 7fa4d11d0700 1 --2- 172.21.15.92:0/2189142965 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a14cdac10 0x559a14d19970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.942+0000 7fa4d11d0700 1 -- 172.21.15.92:0/2189142965 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a14d061f0 msgr2=0x559a14cd5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.942+0000 7fa4d11d0700 1 --2- 172.21.15.92:0/2189142965 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a14d061f0 0x559a14cd5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.942+0000 7fa4d11d0700 1 -- 172.21.15.92:0/2189142965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4cc0ae7c0 con 0x559a14d23a40 2023-10-06T10:43:31.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.942+0000 7fa4d11d0700 1 --2- 172.21.15.92:0/2189142965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14d23a40 0x559a14cd6410 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400e350 tx=0x7fa4c4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966a67e9edf12d79 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.942+0000 7fa4aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:31.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.942+0000 7fa4aaffd700 1 -- 172.21.15.92:0/2189142965 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c400ee90 con 0x559a14d23a40 2023-10-06T10:43:31.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.942+0000 7fa4aaffd700 1 -- 172.21.15.92:0/2189142965 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa4c40123f0 con 0x559a14d23a40 2023-10-06T10:43:31.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.942+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/2189142965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a14cce6d0 con 0x559a14d23a40 2023-10-06T10:43:31.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.942+0000 7fa4aaffd700 1 -- 172.21.15.92:0/2189142965 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4c400c410 con 0x559a14d23a40 2023-10-06T10:43:31.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.942+0000 7fa4dbf2bb80 10 client.7530 fetch_fsmap learned FSMap version 74 2023-10-06T10:43:31.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.942+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/2189142965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x559a14a67020 con 0x559a14d23a40 2023-10-06T10:43:31.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.942+0000 7fa4aaffd700 1 -- 172.21.15.92:0/2189142965 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559a14cce6d0 con 0x559a14d23a40 2023-10-06T10:43:31.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.942+0000 7fa4aaffd700 1 -- 172.21.15.92:0/2189142965 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 74) v1 ==== 3136+0+0 (secure 0 0 0) 0x7fa4c4011070 con 0x559a14d23a40 2023-10-06T10:43:31.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.942+0000 7fa4dbf2bb80 10 client.7530 fetch_fsmap finished waiting for FSMap version 74 2023-10-06T10:43:31.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.942+0000 7fa4dbf2bb80 10 client.7530 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:43:31.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.943+0000 7fa4dbf2bb80 20 client.7530 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:31.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.943+0000 7fa4dbf2bb80 1 --2- 172.21.15.92:0/2189142965 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x559a14d181b0 0x559a14d050e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.943+0000 7fa4dbf2bb80 4 client.7530 mds_command: new command op to 7149 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:43:31.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.943+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/2189142965 --> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a14cab600 con 0x559a14d181b0 2023-10-06T10:43:31.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.943+0000 7fa4d09cf700 1 --2- 172.21.15.92:0/2189142965 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x559a14d181b0 0x559a14d050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.943+0000 7fa4d09cf700 1 --2- 172.21.15.92:0/2189142965 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x559a14d181b0 0x559a14d050e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=18de68647fb5f5b6 server_cookie=f3c8caddb7de0a5f in_seq=0 out_seq=0 2023-10-06T10:43:31.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.943+0000 7fa4aaffd700 10 client.7530 ms_handle_connect on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:31.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.943+0000 7fa4aaffd700 1 -- 172.21.15.92:0/2189142965 <== mds.1 v2:172.21.15.92:6834/3268908749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x559a14cab600 con 0x559a14d181b0 2023-10-06T10:43:31.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.943+0000 7fa4aaffd700 10 client.7530 handle_command_reply: tid=0 2023-10-06T10:43:31.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.943+0000 7fa4dbf2bb80 1 client.7530 shutdown 2023-10-06T10:43:31.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.944+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/2189142965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14d23a40 msgr2=0x559a14cd6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.944+0000 7fa4dbf2bb80 1 --2- 172.21.15.92:0/2189142965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14d23a40 0x559a14cd6410 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400e350 tx=0x7fa4c4008040 comp rx=0 tx=0).stop 2023-10-06T10:43:31.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.944+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/2189142965 shutdown_connections 2023-10-06T10:43:31.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.944+0000 7fa4dbf2bb80 1 --2- 172.21.15.92:0/2189142965 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x559a14d181b0 0x559a14d050e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.944+0000 7fa4dbf2bb80 1 --2- 172.21.15.92:0/2189142965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14d23a40 0x559a14cd6410 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.944+0000 7fa4aaffd700 0 client.7530 ms_handle_reset on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:31.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.944+0000 7fa4dbf2bb80 1 --2- 172.21.15.92:0/2189142965 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a14cdac10 0x559a14d19970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.944+0000 7fa4dbf2bb80 1 --2- 172.21.15.92:0/2189142965 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a14d061f0 0x559a14cd5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.944+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/2189142965 >> 172.21.15.92:0/2189142965 conn(0x559a14cdb0b0 msgr2=0x559a14d1f300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:31.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.944+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/2189142965 shutdown_connections 2023-10-06T10:43:31.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.944+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/2189142965 wait complete. 2023-10-06T10:43:31.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.944+0000 7fa4dbf2bb80 20 client.7530 trim_cache size 0 max 16384 2023-10-06T10:43:31.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.960+0000 7fa4dbf2bb80 1 Processor -- start 2023-10-06T10:43:31.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.960+0000 7fa4dbf2bb80 1 -- start start 2023-10-06T10:43:31.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.960+0000 7fa4dbf2bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14cdac10 0x559a14d16e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.960+0000 7fa4dbf2bb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a14d061f0 0x559a14d05260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.960+0000 7fa4dbf2bb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a14d23a40 0x559a14d057a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.960+0000 7fa4dbf2bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a14aace70 con 0x559a14cdac10 2023-10-06T10:43:31.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.960+0000 7fa4dbf2bb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a14d1fb80 con 0x559a14d23a40 2023-10-06T10:43:31.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.960+0000 7fa4dbf2bb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a14cf4760 con 0x559a14d061f0 2023-10-06T10:43:31.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.961+0000 7fa4d09cf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14cdac10 0x559a14d16e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.961+0000 7fa4d09cf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14cdac10 0x559a14d16e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32806/0 (socket says 172.21.15.92:32806) 2023-10-06T10:43:31.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.961+0000 7fa4d09cf700 1 -- 172.21.15.92:0/1860114720 learned_addr learned my addr 172.21.15.92:0/1860114720 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.961+0000 7fa4c3fff700 1 --2- 172.21.15.92:0/1860114720 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a14d061f0 0x559a14d05260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.961+0000 7fa4d11d0700 1 --2- 172.21.15.92:0/1860114720 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a14d23a40 0x559a14d057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.961+0000 7fa4d09cf700 1 -- 172.21.15.92:0/1860114720 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a14d061f0 msgr2=0x559a14d05260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.961+0000 7fa4d09cf700 1 --2- 172.21.15.92:0/1860114720 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a14d061f0 0x559a14d05260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.961+0000 7fa4d09cf700 1 -- 172.21.15.92:0/1860114720 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a14d23a40 msgr2=0x559a14d057a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.961+0000 7fa4d09cf700 1 --2- 172.21.15.92:0/1860114720 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a14d23a40 0x559a14d057a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.961+0000 7fa4d09cf700 1 -- 172.21.15.92:0/1860114720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a14d17bc0 con 0x559a14cdac10 2023-10-06T10:43:31.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.961+0000 7fa4d09cf700 1 --2- 172.21.15.92:0/1860114720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14cdac10 0x559a14d16e40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc06e7f0 tx=0x7fa4c8032250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f920f30ddfed933 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.961+0000 7fa4b27fc700 1 -- 172.21.15.92:0/1860114720 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c800bc80 con 0x559a14cdac10 2023-10-06T10:43:31.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.961+0000 7fa4b27fc700 1 -- 172.21.15.92:0/1860114720 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c802c430 con 0x559a14cdac10 2023-10-06T10:43:31.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.961+0000 7fa4b27fc700 1 -- 172.21.15.92:0/1860114720 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c802c5c0 con 0x559a14cdac10 2023-10-06T10:43:31.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.962+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/1860114720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14cdac10 msgr2=0x559a14d16e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.962+0000 7fa4dbf2bb80 1 --2- 172.21.15.92:0/1860114720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14cdac10 0x559a14d16e40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc06e7f0 tx=0x7fa4c8032250 comp rx=0 tx=0).stop 2023-10-06T10:43:31.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.962+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/1860114720 shutdown_connections 2023-10-06T10:43:31.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.962+0000 7fa4dbf2bb80 1 --2- 172.21.15.92:0/1860114720 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a14d23a40 0x559a14d057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.962+0000 7fa4dbf2bb80 1 --2- 172.21.15.92:0/1860114720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14cdac10 0x559a14d16e40 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.962+0000 7fa4dbf2bb80 1 --2- 172.21.15.92:0/1860114720 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a14d061f0 0x559a14d05260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.962+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/1860114720 >> 172.21.15.92:0/1860114720 conn(0x559a14cdb0b0 msgr2=0x559a14d1f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:31.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.962+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/1860114720 shutdown_connections 2023-10-06T10:43:31.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.962+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/1860114720 wait complete. 2023-10-06T10:43:31.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.962+0000 7fa4dbf2bb80 1 -- start start 2023-10-06T10:43:31.971 DEBUG:teuthology.orchestra.run:got remote process result: 18 2023-10-06T10:43:31.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.963+0000 7fa4dbf2bb80 1 Processor -- start 2023-10-06T10:43:31.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.963+0000 7fa4dbf2bb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a14cdac10 0x559a14d05770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.963+0000 7fa4dbf2bb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a14d061f0 0x559a14cd4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.963+0000 7fa4dbf2bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14d23a40 0x559a14cd4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.963+0000 7fa4dbf2bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a14d05cb0 con 0x559a14d23a40 2023-10-06T10:43:31.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.963+0000 7fa4dbf2bb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a14d049e0 con 0x559a14d061f0 2023-10-06T10:43:31.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.963+0000 7fa4dbf2bb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a14d6ed90 con 0x559a14cdac10 2023-10-06T10:43:31.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.963+0000 7fa4dbf2bb80 10 client.0 _ll_register_callbacks cb 0x559a14ac6f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:31.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.963+0000 7fa4d11d0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14d23a40 0x559a14cd4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.963+0000 7fa4d11d0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14d23a40 0x559a14cd4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32820/0 (socket says 172.21.15.92:32820) 2023-10-06T10:43:31.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.963+0000 7fa4d11d0700 1 -- 172.21.15.92:0/125645086 learned_addr learned my addr 172.21.15.92:0/125645086 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:31.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.963+0000 7fa4d09cf700 1 --2- 172.21.15.92:0/125645086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a14cdac10 0x559a14d05770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.964+0000 7fa4c3fff700 1 --2- 172.21.15.92:0/125645086 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a14d061f0 0x559a14cd4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.964+0000 7fa4d11d0700 1 -- 172.21.15.92:0/125645086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a14cdac10 msgr2=0x559a14d05770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.964+0000 7fa4d11d0700 1 --2- 172.21.15.92:0/125645086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a14cdac10 0x559a14d05770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.964+0000 7fa4d11d0700 1 -- 172.21.15.92:0/125645086 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a14d061f0 msgr2=0x559a14cd4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:31.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.964+0000 7fa4d11d0700 1 --2- 172.21.15.92:0/125645086 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a14d061f0 0x559a14cd4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.964+0000 7fa4d11d0700 1 -- 172.21.15.92:0/125645086 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4cc0b1490 con 0x559a14d23a40 2023-10-06T10:43:31.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.964+0000 7fa4d11d0700 1 --2- 172.21.15.92:0/125645086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14d23a40 0x559a14cd4cc0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x559a14d1aec0 tx=0x7fa4c401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ca9b1271ecb5c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:31.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.964+0000 7fa4aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:31.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.964+0000 7fa4aaffd700 1 -- 172.21.15.92:0/125645086 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4029070 con 0x559a14d23a40 2023-10-06T10:43:31.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.964+0000 7fa4aaffd700 1 -- 172.21.15.92:0/125645086 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa4c4006d60 con 0x559a14d23a40 2023-10-06T10:43:31.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.964+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/125645086 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4bc05cae0 con 0x559a14d23a40 2023-10-06T10:43:31.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.964+0000 7fa4aaffd700 1 -- 172.21.15.92:0/125645086 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4c400eed0 con 0x559a14d23a40 2023-10-06T10:43:31.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.964+0000 7fa4dbf2bb80 10 client.7536 fetch_fsmap learned FSMap version 74 2023-10-06T10:43:31.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.964+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/125645086 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x559a14bf87c0 con 0x559a14d23a40 2023-10-06T10:43:31.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.964+0000 7fa4aaffd700 1 -- 172.21.15.92:0/125645086 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4bc05cae0 con 0x559a14d23a40 2023-10-06T10:43:31.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.964+0000 7fa4aaffd700 1 -- 172.21.15.92:0/125645086 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 74) v1 ==== 3136+0+0 (secure 0 0 0) 0x7fa4c402d3a0 con 0x559a14d23a40 2023-10-06T10:43:31.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.964+0000 7fa4dbf2bb80 10 client.7536 fetch_fsmap finished waiting for FSMap version 74 2023-10-06T10:43:31.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.964+0000 7fa4dbf2bb80 10 client.7536 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:43:31.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.964+0000 7fa4dbf2bb80 20 client.7536 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:31.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.964+0000 7fa4dbf2bb80 1 --2- 172.21.15.92:0/125645086 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x559a14d6e0f0 0x559a14d6e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:31.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.964+0000 7fa4dbf2bb80 4 client.7536 mds_command: new command op to 7149 tid=0[{"prefix": "scrub resume"}] 2023-10-06T10:43:31.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.964+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/125645086 --> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x559a14bcce40 con 0x559a14d6e0f0 2023-10-06T10:43:31.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.965+0000 7fa4d09cf700 1 --2- 172.21.15.92:0/125645086 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x559a14d6e0f0 0x559a14d6e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:31.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.965+0000 7fa4aaffd700 10 client.7536 ms_handle_connect on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:31.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.965+0000 7fa4d09cf700 1 --2- 172.21.15.92:0/125645086 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x559a14d6e0f0 0x559a14d6e4c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=571ea9cadfc155a4 server_cookie=de59bf41b2d383a5 in_seq=0 out_seq=0 2023-10-06T10:43:31.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.965+0000 7fa4aaffd700 1 -- 172.21.15.92:0/125645086 <== mds.1 v2:172.21.15.92:6834/3268908749 1 ==== command_reply(tid 0: -18 Not rank 0) v1 ==== 18+0+0 (crc 0 0 0) 0x559a14bcce40 con 0x559a14d6e0f0 2023-10-06T10:43:31.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.965+0000 7fa4aaffd700 10 client.7536 handle_command_reply: tid=0 2023-10-06T10:43:31.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.965+0000 7fa4dbf2bb80 1 client.7536 shutdown 2023-10-06T10:43:31.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.965+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/125645086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14d23a40 msgr2=0x559a14cd4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:31.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.965+0000 7fa4dbf2bb80 1 --2- 172.21.15.92:0/125645086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14d23a40 0x559a14cd4cc0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x559a14d1aec0 tx=0x7fa4c401e000 comp rx=0 tx=0).stop 2023-10-06T10:43:31.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.965+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/125645086 shutdown_connections 2023-10-06T10:43:31.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.965+0000 7fa4dbf2bb80 1 --2- 172.21.15.92:0/125645086 >> [v2:172.21.15.92:6834/3268908749,v1:172.21.15.92:6836/3268908749] conn(0x559a14d6e0f0 0x559a14d6e4c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.965+0000 7fa4dbf2bb80 1 --2- 172.21.15.92:0/125645086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a14d23a40 0x559a14cd4cc0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.965+0000 7fa4aaffd700 0 client.7536 ms_handle_reset on v2:172.21.15.92:6834/3268908749 2023-10-06T10:43:31.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.965+0000 7fa4dbf2bb80 1 --2- 172.21.15.92:0/125645086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559a14cdac10 0x559a14d05770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.965+0000 7fa4dbf2bb80 1 --2- 172.21.15.92:0/125645086 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559a14d061f0 0x559a14cd4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:31.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.965+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/125645086 >> 172.21.15.92:0/125645086 conn(0x559a14cdb0b0 msgr2=0x559a14d7d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:31.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.965+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/125645086 shutdown_connections 2023-10-06T10:43:31.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.965+0000 7fa4dbf2bb80 1 -- 172.21.15.92:0/125645086 wait complete. 2023-10-06T10:43:31.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:31.965+0000 7fa4dbf2bb80 20 client.7536 trim_cache size 0 max 16384 2023-10-06T10:43:31.976 INFO:teuthology.orchestra.run.smithi092.stderr:Error EXDEV: Not rank 0 2023-10-06T10:43:31.976 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T10:43:31.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:43:32.013 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:43:32.061 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T10:43:32.061 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T10:43:32.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:43:32.061 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T10:43:32.097 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T10:43:32.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:43:32.098 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T10:43:32.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:43:32.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.304+0000 7fa2202bd700 1 Processor -- start 2023-10-06T10:43:32.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.304+0000 7fa2202bd700 1 -- start start 2023-10-06T10:43:32.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.305+0000 7fa2202bd700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa2180b49a0 0x7fa2180aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:32.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.305+0000 7fa2202bd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa2180b0760 0x7fa2180b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:32.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.305+0000 7fa2202bd700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2180b0c60 con 0x7fa218159560 2023-10-06T10:43:32.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.305+0000 7fa2202bd700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2180b0f40 con 0x7fa2180b0760 2023-10-06T10:43:32.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.305+0000 7fa2202bd700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2180b1220 con 0x7fa2180b49a0 2023-10-06T10:43:32.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.305+0000 7fa21e85a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa2180b0760 0x7fa2180b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:32.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.305+0000 7fa21e059700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa218159560 0x7fa2180aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53980/0 (socket says 172.21.15.92:53980) 2023-10-06T10:43:32.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.305+0000 7fa21e059700 1 -- 172.21.15.92:0/55158502 learned_addr learned my addr 172.21.15.92:0/55158502 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:32.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.305+0000 7fa21d858700 1 --2- 172.21.15.92:0/55158502 >> v2:172.21.15.124:3300/0 conn(0x7fa2180b49a0 0x7fa2180aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:32.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.305+0000 7fa21e85a700 1 -- 172.21.15.92:0/55158502 >> v2:172.21.15.124:3300/0 conn(0x7fa2180b49a0 msgr2=0x7fa2180aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:32.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.305+0000 7fa21e85a700 1 --2- 172.21.15.92:0/55158502 >> v2:172.21.15.124:3300/0 conn(0x7fa2180b49a0 0x7fa2180aac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:32.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.305+0000 7fa21e85a700 1 -- 172.21.15.92:0/55158502 >> v1:172.21.15.92:6789/0 conn(0x7fa218159560 legacy=0x7fa2180aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:32.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.305+0000 7fa21e85a700 1 -- 172.21.15.92:0/55158502 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2180b1500 con 0x7fa2180b0760 2023-10-06T10:43:32.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.305+0000 7fa21d057700 1 -- 172.21.15.92:0/55158502 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422319063 0 0) 0x7fa2180b0c60 con 0x7fa218159560 2023-10-06T10:43:32.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.305+0000 7fa21d858700 1 --2- 172.21.15.92:0/55158502 >> v2:172.21.15.124:3300/0 conn(0x7fa2180b49a0 0x7fa2180aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:32.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.305+0000 7fa21e85a700 1 --2- 172.21.15.92:0/55158502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2180b0760 0x7fa2180b1bc0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fa210009e60 tx=0x7fa210009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da2adf6d00e1ba3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:32.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.305+0000 7fa21d057700 1 -- 172.21.15.92:0/55158502 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21000e070 con 0x7fa2180b0760 2023-10-06T10:43:32.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.305+0000 7fa21d057700 1 -- 172.21.15.92:0/55158502 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21000bbf0 con 0x7fa2180b0760 2023-10-06T10:43:32.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.305+0000 7fa21d057700 1 -- 172.21.15.92:0/55158502 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210012430 con 0x7fa2180b0760 2023-10-06T10:43:32.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.306+0000 7fa2202bd700 1 -- 172.21.15.92:0/55158502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2180b0760 msgr2=0x7fa2180b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:32.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.306+0000 7fa2202bd700 1 --2- 172.21.15.92:0/55158502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2180b0760 0x7fa2180b1bc0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fa210009e60 tx=0x7fa210009780 comp rx=0 tx=0).stop 2023-10-06T10:43:32.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.306+0000 7fa2202bd700 1 -- 172.21.15.92:0/55158502 shutdown_connections 2023-10-06T10:43:32.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.306+0000 7fa2202bd700 1 --2- 172.21.15.92:0/55158502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2180b0760 0x7fa2180b1bc0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:32.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.306+0000 7fa2202bd700 1 --2- 172.21.15.92:0/55158502 >> v2:172.21.15.124:3300/0 conn(0x7fa2180b49a0 0x7fa2180aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:32.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.306+0000 7fa2202bd700 1 -- 172.21.15.92:0/55158502 >> 172.21.15.92:0/55158502 conn(0x7fa2180997f0 msgr2=0x7fa21806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:32.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.306+0000 7fa2202bd700 1 -- 172.21.15.92:0/55158502 shutdown_connections 2023-10-06T10:43:32.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.306+0000 7fa2202bd700 1 -- 172.21.15.92:0/55158502 wait complete. 2023-10-06T10:43:32.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.307+0000 7fa2202bd700 1 Processor -- start 2023-10-06T10:43:32.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.307+0000 7fa2202bd700 1 -- start start 2023-10-06T10:43:32.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.308+0000 7fa2202bd700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2180b0760 0x7fa21809db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:32.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.308+0000 7fa2202bd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2180b49a0 0x7fa2181589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:32.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.308+0000 7fa2202bd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa218159560 0x7fa21809b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:32.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.308+0000 7fa2202bd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21809e0b0 con 0x7fa2180b49a0 2023-10-06T10:43:32.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.308+0000 7fa2202bd700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa218099f50 con 0x7fa2180b0760 2023-10-06T10:43:32.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.308+0000 7fa2202bd700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21809adb0 con 0x7fa218159560 2023-10-06T10:43:32.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.308+0000 7fa21d858700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2180b49a0 0x7fa2181589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:32.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.308+0000 7fa21d858700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2180b49a0 0x7fa2181589b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32830/0 (socket says 172.21.15.92:32830) 2023-10-06T10:43:32.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.308+0000 7fa21d858700 1 -- 172.21.15.92:0/409564180 learned_addr learned my addr 172.21.15.92:0/409564180 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:32.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.308+0000 7fa21d858700 1 -- 172.21.15.92:0/409564180 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa218159560 msgr2=0x7fa21809b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:32.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.308+0000 7fa21d858700 1 --2- 172.21.15.92:0/409564180 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa218159560 0x7fa21809b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:32.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.308+0000 7fa21d858700 1 -- 172.21.15.92:0/409564180 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2180b0760 msgr2=0x7fa21809db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:32.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.308+0000 7fa21e059700 1 --2- 172.21.15.92:0/409564180 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2180b0760 0x7fa21809db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:32.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.308+0000 7fa21d858700 1 --2- 172.21.15.92:0/409564180 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2180b0760 0x7fa21809db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:32.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.308+0000 7fa21d858700 1 -- 172.21.15.92:0/409564180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa210009a20 con 0x7fa2180b49a0 2023-10-06T10:43:32.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.308+0000 7fa21d858700 1 --2- 172.21.15.92:0/409564180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2180b49a0 0x7fa2181589b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fa20800c830 tx=0x7fa208008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.309+0000 7fa20e7fc700 1 -- 172.21.15.92:0/409564180 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa208011600 con 0x7fa2180b49a0 2023-10-06T10:43:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.309+0000 7fa20e7fc700 1 -- 172.21.15.92:0/409564180 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa208011760 con 0x7fa2180b49a0 2023-10-06T10:43:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.309+0000 7fa2202bd700 1 -- 172.21.15.92:0/409564180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21809a1a0 con 0x7fa2180b49a0 2023-10-06T10:43:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.309+0000 7fa20e7fc700 1 -- 172.21.15.92:0/409564180 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa208011b20 con 0x7fa2180b49a0 2023-10-06T10:43:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.309+0000 7fa2202bd700 1 -- 172.21.15.92:0/409564180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21809a7e0 con 0x7fa2180b49a0 2023-10-06T10:43:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.309+0000 7fa2202bd700 1 -- 172.21.15.92:0/409564180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa218064a90 con 0x7fa2180b49a0 2023-10-06T10:43:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.310+0000 7fa20e7fc700 1 -- 172.21.15.92:0/409564180 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa20801a450 con 0x7fa2180b49a0 2023-10-06T10:43:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.310+0000 7fa20e7fc700 1 --2- 172.21.15.92:0/409564180 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa1fc066f30 0x7fa1fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.310+0000 7fa21e059700 1 --2- 172.21.15.92:0/409564180 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa1fc066f30 0x7fa1fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.310+0000 7fa20e7fc700 1 -- 172.21.15.92:0/409564180 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa208084580 con 0x7fa2180b49a0 2023-10-06T10:43:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.311+0000 7fa21e059700 1 --2- 172.21.15.92:0/409564180 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa1fc066f30 0x7fa1fc0693e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa214001730 tx=0x7fa214009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:32.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.315+0000 7fa20e7fc700 1 -- 172.21.15.92:0/409564180 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa208051550 con 0x7fa2180b49a0 2023-10-06T10:43:32.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.541+0000 7fa2202bd700 1 -- 172.21.15.92:0/409564180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2180b17e0 con 0x7fa2180b49a0 2023-10-06T10:43:32.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.542+0000 7fa20e7fc700 1 -- 172.21.15.92:0/409564180 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4637 (secure 0 0 0) 0x7fa208054b70 con 0x7fa2180b49a0 2023-10-06T10:43:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:32.544 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":74,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3469511807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3469511807},{"type":"v1","addr":"172.21.15.124:6834","nonce":3469511807}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:03.173076+0000","modified":"2023-10-06T10:43:28.388741+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7164,"mds_1":7149,"mds_2":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"a","rank":2,"incarnation":71,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/4003376156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4003376156},{"type":"v1","addr":"172.21.15.92:6837","nonce":4003376156}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7149":{"gid":7149,"name":"c","rank":1,"incarnation":69,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6836/3268908749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3268908749},{"type":"v1","addr":"172.21.15.92:6836","nonce":3268908749}]},"join_fscid":-1,"export_targets":[2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7164":{"gid":7164,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/2971348516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2971348516},{"type":"v1","addr":"172.21.15.124:6835","nonce":2971348516}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T10:43:32.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.544+0000 7fa2202bd700 1 -- 172.21.15.92:0/409564180 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa1fc066f30 msgr2=0x7fa1fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:32.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.544+0000 7fa2202bd700 1 --2- 172.21.15.92:0/409564180 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa1fc066f30 0x7fa1fc0693e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa214001730 tx=0x7fa214009040 comp rx=0 tx=0).stop 2023-10-06T10:43:32.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.544+0000 7fa2202bd700 1 -- 172.21.15.92:0/409564180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2180b49a0 msgr2=0x7fa2181589b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:32.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.544+0000 7fa2202bd700 1 --2- 172.21.15.92:0/409564180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2180b49a0 0x7fa2181589b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fa20800c830 tx=0x7fa208008040 comp rx=0 tx=0).stop 2023-10-06T10:43:32.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.544+0000 7fa2202bd700 1 -- 172.21.15.92:0/409564180 shutdown_connections 2023-10-06T10:43:32.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.544+0000 7fa2202bd700 1 --2- 172.21.15.92:0/409564180 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa1fc066f30 0x7fa1fc0693e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:32.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.544+0000 7fa2202bd700 1 --2- 172.21.15.92:0/409564180 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa218159560 0x7fa21809b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:32.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.544+0000 7fa2202bd700 1 --2- 172.21.15.92:0/409564180 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2180b0760 0x7fa21809db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:32.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.544+0000 7fa2202bd700 1 --2- 172.21.15.92:0/409564180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2180b49a0 0x7fa2181589b0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:32.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.544+0000 7fa2202bd700 1 -- 172.21.15.92:0/409564180 >> 172.21.15.92:0/409564180 conn(0x7fa2180997f0 msgr2=0x7fa2180b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:32.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.544+0000 7fa2202bd700 1 -- 172.21.15.92:0/409564180 shutdown_connections 2023-10-06T10:43:32.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.544+0000 7fa2202bd700 1 -- 172.21.15.92:0/409564180 wait complete. 2023-10-06T10:43:32.547 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 74 2023-10-06T10:43:32.557 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:43:32.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.669+0000 7f8dbc318700 1 Processor -- start 2023-10-06T10:43:32.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.669+0000 7f8dbc318700 1 -- start start 2023-10-06T10:43:32.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.670+0000 7f8dbc318700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8db4159560 0x7f8db40aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:32.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.670+0000 7f8dbc318700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8db40b0760 0x7f8db40b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:32.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.670+0000 7f8dbc318700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db40b0c60 con 0x7f8db40b49a0 2023-10-06T10:43:32.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.670+0000 7f8dbc318700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db40b0f40 con 0x7f8db40b0760 2023-10-06T10:43:32.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.670+0000 7f8dbc318700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db40b1220 con 0x7f8db4159560 2023-10-06T10:43:32.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.670+0000 7f8dba8b5700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8db40b0760 0x7f8db40b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:32.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.670+0000 7f8dba0b4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8db4159560 0x7f8db40aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:32.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.670+0000 7f8db98b3700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f8db40b49a0 0x7f8db40aaa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:35274/0 (socket says 172.21.15.92:35274) 2023-10-06T10:43:32.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.670+0000 7f8db98b3700 1 -- 172.21.15.92:0/2248317266 learned_addr learned my addr 172.21.15.92:0/2248317266 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:32.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.671+0000 7f8dba0b4700 1 -- 172.21.15.92:0/2248317266 >> v2:172.21.15.124:3300/0 conn(0x7f8db40b0760 msgr2=0x7f8db40b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:32.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.671+0000 7f8dba0b4700 1 --2- 172.21.15.92:0/2248317266 >> v2:172.21.15.124:3300/0 conn(0x7f8db40b0760 0x7f8db40b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:32.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.671+0000 7f8dba0b4700 1 -- 172.21.15.92:0/2248317266 >> v1:172.21.15.124:6789/0 conn(0x7f8db40b49a0 legacy=0x7f8db40aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:32.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.671+0000 7f8db90b2700 1 -- 172.21.15.92:0/2248317266 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2164326147 0 0) 0x7f8db40b0c60 con 0x7f8db40b49a0 2023-10-06T10:43:32.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.671+0000 7f8dba0b4700 1 -- 172.21.15.92:0/2248317266 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db40b1500 con 0x7f8db4159560 2023-10-06T10:43:32.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.671+0000 7f8dba8b5700 1 --2- 172.21.15.92:0/2248317266 >> v2:172.21.15.124:3300/0 conn(0x7f8db40b0760 0x7f8db40b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:43:32.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.671+0000 7f8dba0b4700 1 --2- 172.21.15.92:0/2248317266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8db4159560 0x7f8db40aa530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8dac009eb0 tx=0x7f8dac009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a75de31eec5f3cd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:32.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.671+0000 7f8db90b2700 1 -- 172.21.15.92:0/2248317266 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac00e070 con 0x7f8db4159560 2023-10-06T10:43:32.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.671+0000 7f8db90b2700 1 -- 172.21.15.92:0/2248317266 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dac00bc30 con 0x7f8db4159560 2023-10-06T10:43:32.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.671+0000 7f8db90b2700 1 -- 172.21.15.92:0/2248317266 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac012430 con 0x7f8db4159560 2023-10-06T10:43:32.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.672+0000 7f8dbc318700 1 -- 172.21.15.92:0/2248317266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8db4159560 msgr2=0x7f8db40aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:32.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.672+0000 7f8dbc318700 1 --2- 172.21.15.92:0/2248317266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8db4159560 0x7f8db40aa530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8dac009eb0 tx=0x7f8dac009780 comp rx=0 tx=0).stop 2023-10-06T10:43:32.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.672+0000 7f8dbc318700 1 -- 172.21.15.92:0/2248317266 shutdown_connections 2023-10-06T10:43:32.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.672+0000 7f8dbc318700 1 --2- 172.21.15.92:0/2248317266 >> v2:172.21.15.124:3300/0 conn(0x7f8db40b0760 0x7f8db40b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:32.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.672+0000 7f8dbc318700 1 --2- 172.21.15.92:0/2248317266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8db4159560 0x7f8db40aa530 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:32.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.672+0000 7f8dbc318700 1 -- 172.21.15.92:0/2248317266 >> 172.21.15.92:0/2248317266 conn(0x7f8db40997f0 msgr2=0x7f8db406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:32.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.672+0000 7f8dbc318700 1 -- 172.21.15.92:0/2248317266 shutdown_connections 2023-10-06T10:43:32.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.672+0000 7f8dbc318700 1 -- 172.21.15.92:0/2248317266 wait complete. 2023-10-06T10:43:32.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.673+0000 7f8dbc318700 1 Processor -- start 2023-10-06T10:43:32.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.673+0000 7f8dbc318700 1 -- start start 2023-10-06T10:43:32.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.673+0000 7f8dbc318700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db40b0760 0x7f8db40abd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:32.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.673+0000 7f8dbc318700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8db40b49a0 0x7f8db40c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:32.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.673+0000 7f8dbc318700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8db4159560 0x7f8db409d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:32.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.673+0000 7f8dbc318700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db40ad120 con 0x7f8db40b0760 2023-10-06T10:43:32.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.673+0000 7f8dbc318700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db40c5770 con 0x7f8db40b49a0 2023-10-06T10:43:32.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.673+0000 7f8dbc318700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db40ad370 con 0x7f8db4159560 2023-10-06T10:43:32.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.674+0000 7f8dba0b4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db40b0760 0x7f8db40abd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:32.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.674+0000 7f8dba0b4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db40b0760 0x7f8db40abd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32836/0 (socket says 172.21.15.92:32836) 2023-10-06T10:43:32.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.674+0000 7f8dba0b4700 1 -- 172.21.15.92:0/3948162413 learned_addr learned my addr 172.21.15.92:0/3948162413 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:32.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.674+0000 7f8dba8b5700 1 --2- 172.21.15.92:0/3948162413 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8db4159560 0x7f8db409d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:32.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.674+0000 7f8db98b3700 1 --2- 172.21.15.92:0/3948162413 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8db40b49a0 0x7f8db40c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:32.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.674+0000 7f8dba0b4700 1 -- 172.21.15.92:0/3948162413 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8db4159560 msgr2=0x7f8db409d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:32.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.674+0000 7f8dba0b4700 1 --2- 172.21.15.92:0/3948162413 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8db4159560 0x7f8db409d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:32.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.674+0000 7f8dba0b4700 1 -- 172.21.15.92:0/3948162413 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8db40b49a0 msgr2=0x7f8db40c5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:32.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.674+0000 7f8dba0b4700 1 --2- 172.21.15.92:0/3948162413 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8db40b49a0 0x7f8db40c5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:32.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.674+0000 7f8dba0b4700 1 -- 172.21.15.92:0/3948162413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da4009420 con 0x7f8db40b0760 2023-10-06T10:43:32.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.674+0000 7f8dba0b4700 1 --2- 172.21.15.92:0/3948162413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db40b0760 0x7f8db40abd30 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f8dac012580 tx=0x7f8dac019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:32.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.674+0000 7f8db27fc700 1 -- 172.21.15.92:0/3948162413 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac012dd0 con 0x7f8db40b0760 2023-10-06T10:43:32.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.674+0000 7f8db27fc700 1 -- 172.21.15.92:0/3948162413 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dac027470 con 0x7f8db40b0760 2023-10-06T10:43:32.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.674+0000 7f8db27fc700 1 -- 172.21.15.92:0/3948162413 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac027830 con 0x7f8db40b0760 2023-10-06T10:43:32.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.674+0000 7f8dbc318700 1 -- 172.21.15.92:0/3948162413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dac009a20 con 0x7f8db40b0760 2023-10-06T10:43:32.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.674+0000 7f8dbc318700 1 -- 172.21.15.92:0/3948162413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db40ad930 con 0x7f8db40b0760 2023-10-06T10:43:32.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.675+0000 7f8dbc318700 1 -- 172.21.15.92:0/3948162413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db4064a90 con 0x7f8db40b0760 2023-10-06T10:43:32.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.675+0000 7f8db27fc700 1 -- 172.21.15.92:0/3948162413 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8dac024070 con 0x7f8db40b0760 2023-10-06T10:43:32.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.676+0000 7f8db27fc700 1 --2- 172.21.15.92:0/3948162413 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8da0066f30 0x7f8da00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:32.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.676+0000 7f8db27fc700 1 -- 172.21.15.92:0/3948162413 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8dac087110 con 0x7f8db40b0760 2023-10-06T10:43:32.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.676+0000 7f8db98b3700 1 --2- 172.21.15.92:0/3948162413 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8da0066f30 0x7f8da00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:32.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.676+0000 7f8db98b3700 1 --2- 172.21.15.92:0/3948162413 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8da0066f30 0x7f8da00693e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8db409e060 tx=0x7f8da8007d10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:32.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.680+0000 7f8db27fc700 1 -- 172.21.15.92:0/3948162413 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8dac0541a0 con 0x7f8db40b0760 2023-10-06T10:43:32.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.868+0000 7f8dbc318700 1 -- 172.21.15.92:0/3948162413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8db40b17e0 con 0x7f8db40b0760 2023-10-06T10:43:32.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.869+0000 7f8db27fc700 1 -- 172.21.15.92:0/3948162413 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+17074 (secure 0 0 0) 0x7f8dac0577c0 con 0x7f8db40b0760 2023-10-06T10:43:32.871 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:32.871 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":78,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:43:04.326791+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:43:00.816800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-06T10:43:02.128579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:43:32.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.871+0000 7f8dbc318700 1 -- 172.21.15.92:0/3948162413 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8da0066f30 msgr2=0x7f8da00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:32.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.871+0000 7f8dbc318700 1 --2- 172.21.15.92:0/3948162413 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8da0066f30 0x7f8da00693e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8db409e060 tx=0x7f8da8007d10 comp rx=0 tx=0).stop 2023-10-06T10:43:32.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.871+0000 7f8dbc318700 1 -- 172.21.15.92:0/3948162413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db40b0760 msgr2=0x7f8db40abd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:32.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.871+0000 7f8dbc318700 1 --2- 172.21.15.92:0/3948162413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db40b0760 0x7f8db40abd30 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f8dac012580 tx=0x7f8dac019000 comp rx=0 tx=0).stop 2023-10-06T10:43:32.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.871+0000 7f8dbc318700 1 -- 172.21.15.92:0/3948162413 shutdown_connections 2023-10-06T10:43:32.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.871+0000 7f8dbc318700 1 --2- 172.21.15.92:0/3948162413 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8da0066f30 0x7f8da00693e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:32.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.871+0000 7f8dbc318700 1 --2- 172.21.15.92:0/3948162413 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8db4159560 0x7f8db409d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:32.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.871+0000 7f8dbc318700 1 --2- 172.21.15.92:0/3948162413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db40b0760 0x7f8db40abd30 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:32.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.871+0000 7f8dbc318700 1 --2- 172.21.15.92:0/3948162413 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8db40b49a0 0x7f8db40c5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:32.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.871+0000 7f8dbc318700 1 -- 172.21.15.92:0/3948162413 >> 172.21.15.92:0/3948162413 conn(0x7f8db40997f0 msgr2=0x7f8db40683c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:32.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.871+0000 7f8dbc318700 1 -- 172.21.15.92:0/3948162413 shutdown_connections 2023-10-06T10:43:32.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.871+0000 7f8dbc318700 1 -- 172.21.15.92:0/3948162413 wait complete. 2023-10-06T10:43:32.884 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T10:38:09.680559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T10:38:10.120786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T10:43:00.816800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T10:43:02.128579+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T10:43:32.885 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T10:43:33.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.999+0000 7f829d19a700 1 Processor -- start 2023-10-06T10:43:33.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:32.999+0000 7f829d19a700 1 -- start start 2023-10-06T10:43:33.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.000+0000 7f829d19a700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f82980cd940 0x7f8298159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.000+0000 7f829d19a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82980b4a20 0x7f82980d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.000+0000 7f829d19a700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82980b5ef0 con 0x7f82980b4e70 2023-10-06T10:43:33.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.000+0000 7f829d19a700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82980b3600 con 0x7f82980cd940 2023-10-06T10:43:33.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.000+0000 7f829d19a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82980b6380 con 0x7f82980b4a20 2023-10-06T10:43:33.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.000+0000 7f8296d9d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f82980cd940 0x7f8298159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.000+0000 7f829659c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82980b4a20 0x7f82980d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.000+0000 7f8296d9d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f82980cd940 0x7f8298159bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:46308/0 (socket says 172.21.15.92:46308) 2023-10-06T10:43:33.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.000+0000 7f829759e700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f82980b4e70 0x7f829815ada0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:35284/0 (socket says 172.21.15.92:35284) 2023-10-06T10:43:33.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.000+0000 7f8296d9d700 1 -- 172.21.15.92:0/1294815924 learned_addr learned my addr 172.21.15.92:0/1294815924 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:33.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.001+0000 7f829659c700 1 -- 172.21.15.92:0/1294815924 >> v2:172.21.15.124:3300/0 conn(0x7f82980cd940 msgr2=0x7f8298159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:33.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.001+0000 7f829659c700 1 --2- 172.21.15.92:0/1294815924 >> v2:172.21.15.124:3300/0 conn(0x7f82980cd940 0x7f8298159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.001+0000 7f829659c700 1 -- 172.21.15.92:0/1294815924 >> v1:172.21.15.124:6789/0 conn(0x7f82980b4e70 legacy=0x7f829815ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:33.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.001+0000 7f829659c700 1 -- 172.21.15.92:0/1294815924 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82980b52e0 con 0x7f82980b4a20 2023-10-06T10:43:33.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.001+0000 7f8295d9b700 1 -- 172.21.15.92:0/1294815924 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1097250118 0 0) 0x7f82980b5ef0 con 0x7f82980b4e70 2023-10-06T10:43:33.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.001+0000 7f8296d9d700 1 --2- 172.21.15.92:0/1294815924 >> v2:172.21.15.124:3300/0 conn(0x7f82980cd940 0x7f8298159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:43:33.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.001+0000 7f829659c700 1 --2- 172.21.15.92:0/1294815924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82980b4a20 0x7f82980d02d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f828c0098e0 tx=0x7f828c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de73d22092051536 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:33.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.001+0000 7f8295d9b700 1 -- 172.21.15.92:0/1294815924 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c00b640 con 0x7f82980b4a20 2023-10-06T10:43:33.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.001+0000 7f8295d9b700 1 -- 172.21.15.92:0/1294815924 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828c00b7a0 con 0x7f82980b4a20 2023-10-06T10:43:33.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.001+0000 7f8295d9b700 1 -- 172.21.15.92:0/1294815924 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c00bb40 con 0x7f82980b4a20 2023-10-06T10:43:33.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.001+0000 7f829d19a700 1 -- 172.21.15.92:0/1294815924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82980b4a20 msgr2=0x7f82980d02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:33.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.001+0000 7f829d19a700 1 --2- 172.21.15.92:0/1294815924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82980b4a20 0x7f82980d02d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f828c0098e0 tx=0x7f828c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.002+0000 7f829d19a700 1 -- 172.21.15.92:0/1294815924 shutdown_connections 2023-10-06T10:43:33.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.002+0000 7f829d19a700 1 --2- 172.21.15.92:0/1294815924 >> v2:172.21.15.124:3300/0 conn(0x7f82980cd940 0x7f8298159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.002+0000 7f829d19a700 1 --2- 172.21.15.92:0/1294815924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82980b4a20 0x7f82980d02d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.002+0000 7f829d19a700 1 -- 172.21.15.92:0/1294815924 >> 172.21.15.92:0/1294815924 conn(0x7f829806ece0 msgr2=0x7f829806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:33.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.002+0000 7f829d19a700 1 -- 172.21.15.92:0/1294815924 shutdown_connections 2023-10-06T10:43:33.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.002+0000 7f829d19a700 1 -- 172.21.15.92:0/1294815924 wait complete. 2023-10-06T10:43:33.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.003+0000 7f829d19a700 1 Processor -- start 2023-10-06T10:43:33.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.003+0000 7f829d19a700 1 -- start start 2023-10-06T10:43:33.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f829d19a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82980b4a20 0x7f8298147c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f829d19a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82980b4e70 0x7f82981514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f829d19a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82980cd940 0x7f82981474d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f829d19a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8298148ea0 con 0x7f82980b4a20 2023-10-06T10:43:33.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f829d19a700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8298147a10 con 0x7f82980cd940 2023-10-06T10:43:33.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f829d19a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8298148290 con 0x7f82980b4e70 2023-10-06T10:43:33.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f8296d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82980b4a20 0x7f8298147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f8296d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82980b4a20 0x7f8298147c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32846/0 (socket says 172.21.15.92:32846) 2023-10-06T10:43:33.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f8296d9d700 1 -- 172.21.15.92:0/1319436441 learned_addr learned my addr 172.21.15.92:0/1319436441 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:33.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f829659c700 1 --2- 172.21.15.92:0/1319436441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82980b4e70 0x7f82981514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f829759e700 1 --2- 172.21.15.92:0/1319436441 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82980cd940 0x7f82981474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f8296d9d700 1 -- 172.21.15.92:0/1319436441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82980b4e70 msgr2=0x7f82981514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:33.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f8296d9d700 1 --2- 172.21.15.92:0/1319436441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82980b4e70 0x7f82981514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f8296d9d700 1 -- 172.21.15.92:0/1319436441 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82980cd940 msgr2=0x7f82981474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:33.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f8296d9d700 1 --2- 172.21.15.92:0/1319436441 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82980cd940 0x7f82981474d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f8296d9d700 1 -- 172.21.15.92:0/1319436441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82880094f0 con 0x7f82980b4a20 2023-10-06T10:43:33.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f829659c700 1 --2- 172.21.15.92:0/1319436441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82980b4e70 0x7f82981514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:33.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f829759e700 1 --2- 172.21.15.92:0/1319436441 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82980cd940 0x7f82981474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:33.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f8296d9d700 1 --2- 172.21.15.92:0/1319436441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82980b4a20 0x7f8298147c00 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f828800ec80 tx=0x7f8288009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:33.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f827effd700 1 -- 172.21.15.92:0/1319436441 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828800c9d0 con 0x7f82980b4a20 2023-10-06T10:43:33.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f827effd700 1 -- 172.21.15.92:0/1319436441 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828800cb30 con 0x7f82980b4a20 2023-10-06T10:43:33.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f829d19a700 1 -- 172.21.15.92:0/1319436441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828c009420 con 0x7f82980b4a20 2023-10-06T10:43:33.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f827effd700 1 -- 172.21.15.92:0/1319436441 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828800ceb0 con 0x7f82980b4a20 2023-10-06T10:43:33.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.004+0000 7f829d19a700 1 -- 172.21.15.92:0/1319436441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8298148850 con 0x7f82980b4a20 2023-10-06T10:43:33.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.005+0000 7f829d19a700 1 -- 172.21.15.92:0/1319436441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8298149180 con 0x7f82980b4a20 2023-10-06T10:43:33.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.006+0000 7f827effd700 1 -- 172.21.15.92:0/1319436441 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8288013680 con 0x7f82980b4a20 2023-10-06T10:43:33.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.006+0000 7f827effd700 1 --2- 172.21.15.92:0/1319436441 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8284066f30 0x7f82840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.006+0000 7f827effd700 1 -- 172.21.15.92:0/1319436441 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8288084ee0 con 0x7f82980b4a20 2023-10-06T10:43:33.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.006+0000 7f829659c700 1 --2- 172.21.15.92:0/1319436441 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8284066f30 0x7f82840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.007+0000 7f829659c700 1 --2- 172.21.15.92:0/1319436441 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8284066f30 0x7f82840693e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f828c0091e0 tx=0x7f828c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:33.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.010+0000 7f827effd700 1 -- 172.21.15.92:0/1319436441 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8288051eb0 con 0x7f82980b4a20 2023-10-06T10:43:33.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.193+0000 7f829d19a700 1 -- 172.21.15.92:0/1319436441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8298064a90 con 0x7f82980b4a20 2023-10-06T10:43:33.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.193+0000 7f827effd700 1 -- 172.21.15.92:0/1319436441 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v78) v1 ==== 92+0+10 (secure 0 0 0) 0x7f82880170a0 con 0x7f82980b4a20 2023-10-06T10:43:33.195 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T10:43:33.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.195+0000 7f829d19a700 1 -- 172.21.15.92:0/1319436441 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8284066f30 msgr2=0x7f82840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:33.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.195+0000 7f829d19a700 1 --2- 172.21.15.92:0/1319436441 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8284066f30 0x7f82840693e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f828c0091e0 tx=0x7f828c017000 comp rx=0 tx=0).stop 2023-10-06T10:43:33.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.195+0000 7f829d19a700 1 -- 172.21.15.92:0/1319436441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82980b4a20 msgr2=0x7f8298147c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:33.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.195+0000 7f829d19a700 1 --2- 172.21.15.92:0/1319436441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82980b4a20 0x7f8298147c00 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f828800ec80 tx=0x7f8288009250 comp rx=0 tx=0).stop 2023-10-06T10:43:33.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.195+0000 7f829d19a700 1 -- 172.21.15.92:0/1319436441 shutdown_connections 2023-10-06T10:43:33.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.195+0000 7f829d19a700 1 --2- 172.21.15.92:0/1319436441 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8284066f30 0x7f82840693e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.195+0000 7f829d19a700 1 --2- 172.21.15.92:0/1319436441 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82980cd940 0x7f82981474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.195+0000 7f829d19a700 1 --2- 172.21.15.92:0/1319436441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82980b4a20 0x7f8298147c00 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.195+0000 7f829d19a700 1 --2- 172.21.15.92:0/1319436441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82980b4e70 0x7f82981514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.195+0000 7f829d19a700 1 -- 172.21.15.92:0/1319436441 >> 172.21.15.92:0/1319436441 conn(0x7f829806ece0 msgr2=0x7f829815f360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:33.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.195+0000 7f829d19a700 1 -- 172.21.15.92:0/1319436441 shutdown_connections 2023-10-06T10:43:33.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.195+0000 7f829d19a700 1 -- 172.21.15.92:0/1319436441 wait complete. 2023-10-06T10:43:33.209 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T10:43:33.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.324+0000 7f33bee33700 1 Processor -- start 2023-10-06T10:43:33.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.324+0000 7f33bee33700 1 -- start start 2023-10-06T10:43:33.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.325+0000 7f33bee33700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f33b81594d0 0x7f33b80cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.325+0000 7f33bee33700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33b80cbaa0 0x7f33b80ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.325+0000 7f33bee33700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b80cbeb0 con 0x7f33b81598a0 2023-10-06T10:43:33.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.325+0000 7f33bee33700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b80cf5d0 con 0x7f33b81594d0 2023-10-06T10:43:33.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.325+0000 7f33bee33700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b80cf8b0 con 0x7f33b80cbaa0 2023-10-06T10:43:33.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.325+0000 7f33bcbcf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f33b81594d0 0x7f33b80cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.325+0000 7f33affff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f33b81598a0 0x7f33b80ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53988/0 (socket says 172.21.15.92:53988) 2023-10-06T10:43:33.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.325+0000 7f33affff700 1 -- 172.21.15.92:0/404654232 learned_addr learned my addr 172.21.15.92:0/404654232 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:33.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.325+0000 7f33bd3d0700 1 --2- 172.21.15.92:0/404654232 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33b80cbaa0 0x7f33b80ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.325+0000 7f33bcbcf700 1 -- 172.21.15.92:0/404654232 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33b80cbaa0 msgr2=0x7f33b80ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:33.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.325+0000 7f33bcbcf700 1 --2- 172.21.15.92:0/404654232 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33b80cbaa0 0x7f33b80ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.325+0000 7f33bcbcf700 1 -- 172.21.15.92:0/404654232 >> v1:172.21.15.92:6789/0 conn(0x7f33b81598a0 legacy=0x7f33b80ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:33.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.325+0000 7f33af7fe700 1 -- 172.21.15.92:0/404654232 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2700492936 0 0) 0x7f33b80cbeb0 con 0x7f33b81598a0 2023-10-06T10:43:33.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.325+0000 7f33bcbcf700 1 -- 172.21.15.92:0/404654232 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b80cfb90 con 0x7f33b81594d0 2023-10-06T10:43:33.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.326+0000 7f33bcbcf700 1 --2- 172.21.15.92:0/404654232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33b81594d0 0x7f33b80cb560 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f33b40098e0 tx=0x7f33b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892a81546679f82b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:33.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.326+0000 7f33af7fe700 1 -- 172.21.15.92:0/404654232 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b4009e80 con 0x7f33b81594d0 2023-10-06T10:43:33.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.326+0000 7f33af7fe700 1 -- 172.21.15.92:0/404654232 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b400b860 con 0x7f33b81594d0 2023-10-06T10:43:33.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.326+0000 7f33af7fe700 1 -- 172.21.15.92:0/404654232 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b400bbe0 con 0x7f33b81594d0 2023-10-06T10:43:33.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.326+0000 7f33bee33700 1 -- 172.21.15.92:0/404654232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33b81594d0 msgr2=0x7f33b80cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:33.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.326+0000 7f33bee33700 1 --2- 172.21.15.92:0/404654232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33b81594d0 0x7f33b80cb560 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f33b40098e0 tx=0x7f33b40091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.326+0000 7f33bee33700 1 -- 172.21.15.92:0/404654232 shutdown_connections 2023-10-06T10:43:33.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.326+0000 7f33bee33700 1 --2- 172.21.15.92:0/404654232 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33b80cbaa0 0x7f33b80ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.326+0000 7f33bee33700 1 --2- 172.21.15.92:0/404654232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33b81594d0 0x7f33b80cb560 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.326+0000 7f33bee33700 1 -- 172.21.15.92:0/404654232 >> 172.21.15.92:0/404654232 conn(0x7f33b8089e50 msgr2=0x7f33b8099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:33.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.326+0000 7f33bee33700 1 -- 172.21.15.92:0/404654232 shutdown_connections 2023-10-06T10:43:33.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.326+0000 7f33bee33700 1 -- 172.21.15.92:0/404654232 wait complete. 2023-10-06T10:43:33.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.327+0000 7f33bee33700 1 Processor -- start 2023-10-06T10:43:33.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.328+0000 7f33bee33700 1 -- start start 2023-10-06T10:43:33.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.328+0000 7f33bee33700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33b80cbaa0 0x7f33b80b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.328+0000 7f33bee33700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33b81594d0 0x7f33b816a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.328+0000 7f33bee33700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33b81598a0 0x7f33b80b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.328+0000 7f33bee33700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b80b43d0 con 0x7f33b81598a0 2023-10-06T10:43:33.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.328+0000 7f33bee33700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b80b5050 con 0x7f33b81594d0 2023-10-06T10:43:33.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.328+0000 7f33bee33700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b80b4130 con 0x7f33b80cbaa0 2023-10-06T10:43:33.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.328+0000 7f33bd3d0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33b81598a0 0x7f33b80b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.328+0000 7f33bd3d0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33b81598a0 0x7f33b80b3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32862/0 (socket says 172.21.15.92:32862) 2023-10-06T10:43:33.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.328+0000 7f33bd3d0700 1 -- 172.21.15.92:0/1748721262 learned_addr learned my addr 172.21.15.92:0/1748721262 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:33.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.329+0000 7f33bcbcf700 1 --2- 172.21.15.92:0/1748721262 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33b80cbaa0 0x7f33b80b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.329+0000 7f33affff700 1 --2- 172.21.15.92:0/1748721262 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33b81594d0 0x7f33b816a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.329+0000 7f33bd3d0700 1 -- 172.21.15.92:0/1748721262 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33b80cbaa0 msgr2=0x7f33b80b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:33.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.329+0000 7f33bd3d0700 1 --2- 172.21.15.92:0/1748721262 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33b80cbaa0 0x7f33b80b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.329+0000 7f33bd3d0700 1 -- 172.21.15.92:0/1748721262 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33b81594d0 msgr2=0x7f33b816a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:33.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.329+0000 7f33bd3d0700 1 --2- 172.21.15.92:0/1748721262 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33b81594d0 0x7f33b816a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.329+0000 7f33bd3d0700 1 -- 172.21.15.92:0/1748721262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b4009420 con 0x7f33b81598a0 2023-10-06T10:43:33.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.329+0000 7f33bd3d0700 1 --2- 172.21.15.92:0/1748721262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33b81598a0 0x7f33b80b3bf0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009fa0 tx=0x7f33b0009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:33.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.329+0000 7f33acff9700 1 -- 172.21.15.92:0/1748721262 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b000ebc0 con 0x7f33b81598a0 2023-10-06T10:43:33.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.329+0000 7f33acff9700 1 -- 172.21.15.92:0/1748721262 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b000ed20 con 0x7f33b81598a0 2023-10-06T10:43:33.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.330+0000 7f33bee33700 1 -- 172.21.15.92:0/1748721262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b80b4720 con 0x7f33b81598a0 2023-10-06T10:43:33.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.330+0000 7f33acff9700 1 -- 172.21.15.92:0/1748721262 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0014670 con 0x7f33b81598a0 2023-10-06T10:43:33.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.330+0000 7f33bee33700 1 -- 172.21.15.92:0/1748721262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b80b4ce0 con 0x7f33b81598a0 2023-10-06T10:43:33.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.330+0000 7f33bee33700 1 -- 172.21.15.92:0/1748721262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b80b51e0 con 0x7f33b81598a0 2023-10-06T10:43:33.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.331+0000 7f33acff9700 1 -- 172.21.15.92:0/1748721262 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f33b00149e0 con 0x7f33b81598a0 2023-10-06T10:43:33.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.331+0000 7f33acff9700 1 --2- 172.21.15.92:0/1748721262 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f33a4066f30 0x7f33a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.331+0000 7f33acff9700 1 -- 172.21.15.92:0/1748721262 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f33b0084e70 con 0x7f33b81598a0 2023-10-06T10:43:33.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.331+0000 7f33bcbcf700 1 --2- 172.21.15.92:0/1748721262 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f33a4066f30 0x7f33a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.332+0000 7f33bcbcf700 1 --2- 172.21.15.92:0/1748721262 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f33a4066f30 0x7f33a40693e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f33b40091e0 tx=0x7f33b4017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:33.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.336+0000 7f33acff9700 1 -- 172.21.15.92:0/1748721262 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f33b0051e40 con 0x7f33b81598a0 2023-10-06T10:43:33.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.519+0000 7f33bee33700 1 -- 172.21.15.92:0/1748721262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f33b8064a90 con 0x7f33b81598a0 2023-10-06T10:43:33.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.519+0000 7f33acff9700 1 -- 172.21.15.92:0/1748721262 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f33b0055460 con 0x7f33b81598a0 2023-10-06T10:43:33.521 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T10:43:33.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.521+0000 7f33bee33700 1 -- 172.21.15.92:0/1748721262 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f33a4066f30 msgr2=0x7f33a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:33.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.521+0000 7f33bee33700 1 --2- 172.21.15.92:0/1748721262 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f33a4066f30 0x7f33a40693e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f33b40091e0 tx=0x7f33b4017000 comp rx=0 tx=0).stop 2023-10-06T10:43:33.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.521+0000 7f33bee33700 1 -- 172.21.15.92:0/1748721262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33b81598a0 msgr2=0x7f33b80b3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:33.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.521+0000 7f33bee33700 1 --2- 172.21.15.92:0/1748721262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33b81598a0 0x7f33b80b3bf0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009fa0 tx=0x7f33b0009900 comp rx=0 tx=0).stop 2023-10-06T10:43:33.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.521+0000 7f33bee33700 1 -- 172.21.15.92:0/1748721262 shutdown_connections 2023-10-06T10:43:33.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.521+0000 7f33bee33700 1 --2- 172.21.15.92:0/1748721262 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f33a4066f30 0x7f33a40693e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.521+0000 7f33bee33700 1 --2- 172.21.15.92:0/1748721262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33b81598a0 0x7f33b80b3bf0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.521+0000 7f33bee33700 1 --2- 172.21.15.92:0/1748721262 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33b80cbaa0 0x7f33b80b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.521+0000 7f33bee33700 1 --2- 172.21.15.92:0/1748721262 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33b81594d0 0x7f33b816a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.521+0000 7f33bee33700 1 -- 172.21.15.92:0/1748721262 >> 172.21.15.92:0/1748721262 conn(0x7f33b8089e50 msgr2=0x7f33b815d940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:33.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.521+0000 7f33bee33700 1 -- 172.21.15.92:0/1748721262 shutdown_connections 2023-10-06T10:43:33.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.521+0000 7f33bee33700 1 -- 172.21.15.92:0/1748721262 wait complete. 2023-10-06T10:43:33.535 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T10:43:33.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.651+0000 7f0072570700 1 Processor -- start 2023-10-06T10:43:33.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.651+0000 7f0072570700 1 -- start start 2023-10-06T10:43:33.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.652+0000 7f0072570700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f006c0b2620 0x7f006c0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.652+0000 7f0072570700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f006c0cc970 con 0x7f006c0b29f0 2023-10-06T10:43:33.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.652+0000 7f0072570700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f006c0cbd60 con 0x7f006c0cf1b0 2023-10-06T10:43:33.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.652+0000 7f0072570700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f006c0cc040 con 0x7f006c0b2620 2023-10-06T10:43:33.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.652+0000 7f007156e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f006c0b2620 0x7f006c0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.652+0000 7f0070d6d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f006c0b29f0 0x7f006c0ceaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49114/0 (socket says 172.21.15.92:49114) 2023-10-06T10:43:33.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.652+0000 7f0070d6d700 1 -- 172.21.15.92:0/3943587556 learned_addr learned my addr 172.21.15.92:0/3943587556 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:33.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.652+0000 7f007156e700 1 -- 172.21.15.92:0/3943587556 >> v1:172.21.15.124:6789/0 conn(0x7f006c0cf1b0 legacy=0x7f006c0c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:33.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.652+0000 7f007156e700 1 -- 172.21.15.92:0/3943587556 >> v1:172.21.15.92:6789/0 conn(0x7f006c0b29f0 legacy=0x7f006c0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:33.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.652+0000 7f007156e700 1 -- 172.21.15.92:0/3943587556 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c0cc320 con 0x7f006c0b2620 2023-10-06T10:43:33.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.652+0000 7f007156e700 1 --2- 172.21.15.92:0/3943587556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f006c0b2620 0x7f006c0cccb0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f0064009eb0 tx=0x7f0064009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e247dbeb2e528d2c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:33.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.653+0000 7f0063fff700 1 -- 172.21.15.92:0/3943587556 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006400e070 con 0x7f006c0b2620 2023-10-06T10:43:33.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.653+0000 7f0063fff700 1 -- 172.21.15.92:0/3943587556 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006400bbf0 con 0x7f006c0b2620 2023-10-06T10:43:33.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.653+0000 7f0063fff700 1 -- 172.21.15.92:0/3943587556 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0064012430 con 0x7f006c0b2620 2023-10-06T10:43:33.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.653+0000 7f0072570700 1 -- 172.21.15.92:0/3943587556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f006c0b2620 msgr2=0x7f006c0cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:33.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.653+0000 7f0072570700 1 --2- 172.21.15.92:0/3943587556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f006c0b2620 0x7f006c0cccb0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f0064009eb0 tx=0x7f0064009780 comp rx=0 tx=0).stop 2023-10-06T10:43:33.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.653+0000 7f0072570700 1 -- 172.21.15.92:0/3943587556 shutdown_connections 2023-10-06T10:43:33.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.653+0000 7f0072570700 1 --2- 172.21.15.92:0/3943587556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f006c0b2620 0x7f006c0cccb0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.653+0000 7f0072570700 1 -- 172.21.15.92:0/3943587556 >> 172.21.15.92:0/3943587556 conn(0x7f006c099520 msgr2=0x7f006c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:33.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.653+0000 7f0072570700 1 -- 172.21.15.92:0/3943587556 shutdown_connections 2023-10-06T10:43:33.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.653+0000 7f0072570700 1 -- 172.21.15.92:0/3943587556 wait complete. 2023-10-06T10:43:33.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.654+0000 7f0072570700 1 Processor -- start 2023-10-06T10:43:33.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.654+0000 7f0072570700 1 -- start start 2023-10-06T10:43:33.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.655+0000 7f0072570700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f006c0b2620 0x7f006c14c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.655+0000 7f0072570700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f006c0b29f0 0x7f006c169ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.655+0000 7f0072570700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f006c0cf1b0 0x7f006c0b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.655+0000 7f0072570700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c0ca6c0 con 0x7f006c0b29f0 2023-10-06T10:43:33.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.655+0000 7f0072570700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c0b50b0 con 0x7f006c0b2620 2023-10-06T10:43:33.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.655+0000 7f0072570700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006c14d480 con 0x7f006c0cf1b0 2023-10-06T10:43:33.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.655+0000 7f0070d6d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f006c0b29f0 0x7f006c169ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.655+0000 7f0070d6d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f006c0b29f0 0x7f006c169ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40660/0 (socket says 172.21.15.92:40660) 2023-10-06T10:43:33.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.655+0000 7f0070d6d700 1 -- 172.21.15.92:0/3412110298 learned_addr learned my addr 172.21.15.92:0/3412110298 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:33.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.656+0000 7f007156e700 1 --2- 172.21.15.92:0/3412110298 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f006c0b2620 0x7f006c14c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.656+0000 7f0071d6f700 1 --2- 172.21.15.92:0/3412110298 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f006c0cf1b0 0x7f006c0b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.656+0000 7f0070d6d700 1 -- 172.21.15.92:0/3412110298 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f006c0cf1b0 msgr2=0x7f006c0b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:33.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.656+0000 7f0070d6d700 1 --2- 172.21.15.92:0/3412110298 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f006c0cf1b0 0x7f006c0b4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.656+0000 7f0070d6d700 1 -- 172.21.15.92:0/3412110298 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f006c0b2620 msgr2=0x7f006c14c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:33.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.656+0000 7f0070d6d700 1 --2- 172.21.15.92:0/3412110298 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f006c0b2620 0x7f006c14c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.656+0000 7f0070d6d700 1 -- 172.21.15.92:0/3412110298 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0064009a20 con 0x7f006c0b29f0 2023-10-06T10:43:33.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.656+0000 7f007156e700 1 --2- 172.21.15.92:0/3412110298 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f006c0b2620 0x7f006c14c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:33.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.656+0000 7f0070d6d700 1 --2- 172.21.15.92:0/3412110298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f006c0b29f0 0x7f006c169ba0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f005c00c790 tx=0x7f005c007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:33.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.656+0000 7f00617fa700 1 -- 172.21.15.92:0/3412110298 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c005b30 con 0x7f006c0b29f0 2023-10-06T10:43:33.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.656+0000 7f00617fa700 1 -- 172.21.15.92:0/3412110298 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005c005cb0 con 0x7f006c0b29f0 2023-10-06T10:43:33.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.656+0000 7f0072570700 1 -- 172.21.15.92:0/3412110298 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f006c0ca100 con 0x7f006c0b29f0 2023-10-06T10:43:33.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.656+0000 7f00617fa700 1 -- 172.21.15.92:0/3412110298 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c004a80 con 0x7f006c0b29f0 2023-10-06T10:43:33.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.656+0000 7f0072570700 1 -- 172.21.15.92:0/3412110298 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f006c0b4d40 con 0x7f006c0b29f0 2023-10-06T10:43:33.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.657+0000 7f0072570700 1 -- 172.21.15.92:0/3412110298 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006c0b4ed0 con 0x7f006c0b29f0 2023-10-06T10:43:33.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.657+0000 7f00617fa700 1 -- 172.21.15.92:0/3412110298 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f005c005030 con 0x7f006c0b29f0 2023-10-06T10:43:33.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.658+0000 7f00617fa700 1 --2- 172.21.15.92:0/3412110298 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0050066f30 0x7f00500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.658+0000 7f007156e700 1 --2- 172.21.15.92:0/3412110298 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0050066f30 0x7f00500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.658+0000 7f00617fa700 1 -- 172.21.15.92:0/3412110298 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f005c084190 con 0x7f006c0b29f0 2023-10-06T10:43:33.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.658+0000 7f007156e700 1 --2- 172.21.15.92:0/3412110298 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0050066f30 0x7f00500693e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f00640097c0 tx=0x7f0064016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:33.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.662+0000 7f00617fa700 1 -- 172.21.15.92:0/3412110298 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f005c051160 con 0x7f006c0b29f0 2023-10-06T10:43:33.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.842+0000 7f0072570700 1 -- 172.21.15.92:0/3412110298 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f006c0b5260 con 0x7f006c0b29f0 2023-10-06T10:43:33.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.843+0000 7f00617fa700 1 -- 172.21.15.92:0/3412110298 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) v1 ==== 104+0+11 (secure 0 0 0) 0x7f006c0b5260 con 0x7f006c0b29f0 2023-10-06T10:43:33.845 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T10:43:33.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.844+0000 7f0072570700 1 -- 172.21.15.92:0/3412110298 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0050066f30 msgr2=0x7f00500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:33.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.844+0000 7f0072570700 1 --2- 172.21.15.92:0/3412110298 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0050066f30 0x7f00500693e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f00640097c0 tx=0x7f0064016000 comp rx=0 tx=0).stop 2023-10-06T10:43:33.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.844+0000 7f0072570700 1 -- 172.21.15.92:0/3412110298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f006c0b29f0 msgr2=0x7f006c169ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:33.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.844+0000 7f0072570700 1 --2- 172.21.15.92:0/3412110298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f006c0b29f0 0x7f006c169ba0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f005c00c790 tx=0x7f005c007f40 comp rx=0 tx=0).stop 2023-10-06T10:43:33.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.844+0000 7f0072570700 1 -- 172.21.15.92:0/3412110298 shutdown_connections 2023-10-06T10:43:33.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.845+0000 7f0072570700 1 --2- 172.21.15.92:0/3412110298 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0050066f30 0x7f00500693e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.845+0000 7f0072570700 1 --2- 172.21.15.92:0/3412110298 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f006c0cf1b0 0x7f006c0b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.845+0000 7f0072570700 1 --2- 172.21.15.92:0/3412110298 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f006c0b2620 0x7f006c14c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.845+0000 7f0072570700 1 --2- 172.21.15.92:0/3412110298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f006c0b29f0 0x7f006c169ba0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.845+0000 7f0072570700 1 -- 172.21.15.92:0/3412110298 >> 172.21.15.92:0/3412110298 conn(0x7f006c099520 msgr2=0x7f006c158e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:33.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.845+0000 7f0072570700 1 -- 172.21.15.92:0/3412110298 shutdown_connections 2023-10-06T10:43:33.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.845+0000 7f0072570700 1 -- 172.21.15.92:0/3412110298 wait complete. 2023-10-06T10:43:33.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T10:43:33.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.974+0000 7fa25f59e700 1 Processor -- start 2023-10-06T10:43:33.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.974+0000 7fa25f59e700 1 -- start start 2023-10-06T10:43:33.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.975+0000 7fa25f59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa2600b2620 0x7fa2600cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.975+0000 7fa25f59e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2600b29f0 0x7fa2600cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.975+0000 7fa25f59e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2600cc970 con 0x7fa2600ceaa0 2023-10-06T10:43:33.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.975+0000 7fa25f59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2600cbd60 con 0x7fa2600b2620 2023-10-06T10:43:33.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.975+0000 7fa25f59e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2600cc040 con 0x7fa2600b29f0 2023-10-06T10:43:33.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.975+0000 7fa25e59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa2600b2620 0x7fa2600cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.975+0000 7fa25e59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa2600b2620 0x7fa2600cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40662/0 (socket says 172.21.15.92:40662) 2023-10-06T10:43:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.975+0000 7fa25e59c700 1 -- 172.21.15.92:0/1698714558 learned_addr learned my addr 172.21.15.92:0/1698714558 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.975+0000 7fa255d9b700 1 --2- 172.21.15.92:0/1698714558 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2600b29f0 0x7fa2600cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.975+0000 7fa25e59c700 1 -- 172.21.15.92:0/1698714558 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2600b29f0 msgr2=0x7fa2600cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.975+0000 7fa25e59c700 1 --2- 172.21.15.92:0/1698714558 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2600b29f0 0x7fa2600cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.975+0000 7fa25e59c700 1 -- 172.21.15.92:0/1698714558 >> v1:172.21.15.124:6789/0 conn(0x7fa2600ceaa0 legacy=0x7fa2600cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.975+0000 7fa25e59c700 1 -- 172.21.15.92:0/1698714558 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2600cc320 con 0x7fa2600b2620 2023-10-06T10:43:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.975+0000 7fa25e59c700 1 --2- 172.21.15.92:0/1698714558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2600b2620 0x7fa2600cccb0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fa250009990 tx=0x7fa250009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38577e188ed1b9ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.976+0000 7fa25dd9b700 1 -- 172.21.15.92:0/1698714558 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250005c10 con 0x7fa2600b2620 2023-10-06T10:43:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.976+0000 7fa25dd9b700 1 -- 172.21.15.92:0/1698714558 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa250005d70 con 0x7fa2600b2620 2023-10-06T10:43:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.976+0000 7fa25dd9b700 1 -- 172.21.15.92:0/1698714558 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25000b880 con 0x7fa2600b2620 2023-10-06T10:43:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.976+0000 7fa25f59e700 1 -- 172.21.15.92:0/1698714558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2600b2620 msgr2=0x7fa2600cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:33.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.976+0000 7fa25f59e700 1 --2- 172.21.15.92:0/1698714558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2600b2620 0x7fa2600cccb0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fa250009990 tx=0x7fa250009220 comp rx=0 tx=0).stop 2023-10-06T10:43:33.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.976+0000 7fa25f59e700 1 -- 172.21.15.92:0/1698714558 shutdown_connections 2023-10-06T10:43:33.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.976+0000 7fa25f59e700 1 --2- 172.21.15.92:0/1698714558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2600b2620 0x7fa2600cccb0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.976+0000 7fa25f59e700 1 --2- 172.21.15.92:0/1698714558 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2600b29f0 0x7fa2600cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.976+0000 7fa25f59e700 1 -- 172.21.15.92:0/1698714558 >> 172.21.15.92:0/1698714558 conn(0x7fa260099520 msgr2=0x7fa26006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:33.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.976+0000 7fa25f59e700 1 -- 172.21.15.92:0/1698714558 shutdown_connections 2023-10-06T10:43:33.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.976+0000 7fa25f59e700 1 -- 172.21.15.92:0/1698714558 wait complete. 2023-10-06T10:43:33.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.977+0000 7fa25f59e700 1 Processor -- start 2023-10-06T10:43:33.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.977+0000 7fa25f59e700 1 -- start start 2023-10-06T10:43:33.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.978+0000 7fa25f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2600b2620 0x7fa2601556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.978+0000 7fa25f59e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2600b29f0 0x7fa260153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.978+0000 7fa25f59e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2600ceaa0 0x7fa2600b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.978+0000 7fa25f59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2600c7d50 con 0x7fa2600b2620 2023-10-06T10:43:33.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.978+0000 7fa25f59e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2600b4a70 con 0x7fa2600b29f0 2023-10-06T10:43:33.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.978+0000 7fa25f59e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2600c7420 con 0x7fa2600ceaa0 2023-10-06T10:43:33.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.978+0000 7fa25e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2600b2620 0x7fa2601556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.978+0000 7fa25e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2600b2620 0x7fa2601556b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40666/0 (socket says 172.21.15.92:40666) 2023-10-06T10:43:33.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.978+0000 7fa25e59c700 1 -- 172.21.15.92:0/3014579165 learned_addr learned my addr 172.21.15.92:0/3014579165 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:33.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.978+0000 7fa255d9b700 1 --2- 172.21.15.92:0/3014579165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2600b29f0 0x7fa260153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.978+0000 7fa25ed9d700 1 --2- 172.21.15.92:0/3014579165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2600ceaa0 0x7fa2600b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.978+0000 7fa25e59c700 1 -- 172.21.15.92:0/3014579165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2600ceaa0 msgr2=0x7fa2600b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.978+0000 7fa25e59c700 1 --2- 172.21.15.92:0/3014579165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2600ceaa0 0x7fa2600b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.978+0000 7fa25e59c700 1 -- 172.21.15.92:0/3014579165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2600b29f0 msgr2=0x7fa260153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.978+0000 7fa25e59c700 1 --2- 172.21.15.92:0/3014579165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2600b29f0 0x7fa260153550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.978+0000 7fa25e59c700 1 -- 172.21.15.92:0/3014579165 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa250009480 con 0x7fa2600b2620 2023-10-06T10:43:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.978+0000 7fa255d9b700 1 --2- 172.21.15.92:0/3014579165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2600b29f0 0x7fa260153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.979+0000 7fa25e59c700 1 --2- 172.21.15.92:0/3014579165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2600b2620 0x7fa2601556b0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fa25000b3c0 tx=0x7fa250016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.979+0000 7fa256ffd700 1 -- 172.21.15.92:0/3014579165 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250011770 con 0x7fa2600b2620 2023-10-06T10:43:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.979+0000 7fa256ffd700 1 -- 172.21.15.92:0/3014579165 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa250020070 con 0x7fa2600b2620 2023-10-06T10:43:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.979+0000 7fa256ffd700 1 -- 172.21.15.92:0/3014579165 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250011b00 con 0x7fa2600b2620 2023-10-06T10:43:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.979+0000 7fa25f59e700 1 -- 172.21.15.92:0/3014579165 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2600c7700 con 0x7fa2600b2620 2023-10-06T10:43:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.979+0000 7fa25f59e700 1 -- 172.21.15.92:0/3014579165 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2600b32c0 con 0x7fa2600b2620 2023-10-06T10:43:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.980+0000 7fa25f59e700 1 -- 172.21.15.92:0/3014579165 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa26005f230 con 0x7fa2600b2620 2023-10-06T10:43:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.980+0000 7fa256ffd700 1 -- 172.21.15.92:0/3014579165 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa250022030 con 0x7fa2600b2620 2023-10-06T10:43:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.980+0000 7fa256ffd700 1 --2- 172.21.15.92:0/3014579165 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa240066f30 0x7fa2400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.981+0000 7fa256ffd700 1 -- 172.21.15.92:0/3014579165 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa250086410 con 0x7fa2600b2620 2023-10-06T10:43:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.981+0000 7fa255d9b700 1 --2- 172.21.15.92:0/3014579165 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa240066f30 0x7fa2400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.981+0000 7fa255d9b700 1 --2- 172.21.15.92:0/3014579165 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa240066f30 0x7fa2400693e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa2600c6c70 tx=0x7fa248008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:33.985+0000 7fa256ffd700 1 -- 172.21.15.92:0/3014579165 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa2500533e0 con 0x7fa2600b2620 2023-10-06T10:43:34.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.170+0000 7fa25f59e700 1 -- 172.21.15.92:0/3014579165 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa2600cc600 con 0x7fa2600b2620 2023-10-06T10:43:34.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.170+0000 7fa256ffd700 1 -- 172.21.15.92:0/3014579165 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa250056a00 con 0x7fa2600b2620 2023-10-06T10:43:34.173 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T10:43:34.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.173+0000 7fa25f59e700 1 -- 172.21.15.92:0/3014579165 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa240066f30 msgr2=0x7fa2400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:34.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.173+0000 7fa25f59e700 1 --2- 172.21.15.92:0/3014579165 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa240066f30 0x7fa2400693e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa2600c6c70 tx=0x7fa248008040 comp rx=0 tx=0).stop 2023-10-06T10:43:34.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.173+0000 7fa25f59e700 1 -- 172.21.15.92:0/3014579165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2600b2620 msgr2=0x7fa2601556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:34.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.173+0000 7fa25f59e700 1 --2- 172.21.15.92:0/3014579165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2600b2620 0x7fa2601556b0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fa25000b3c0 tx=0x7fa250016000 comp rx=0 tx=0).stop 2023-10-06T10:43:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.173+0000 7fa25f59e700 1 -- 172.21.15.92:0/3014579165 shutdown_connections 2023-10-06T10:43:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.173+0000 7fa25f59e700 1 --2- 172.21.15.92:0/3014579165 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa240066f30 0x7fa2400693e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.173+0000 7fa25f59e700 1 --2- 172.21.15.92:0/3014579165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2600ceaa0 0x7fa2600b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.173+0000 7fa25f59e700 1 --2- 172.21.15.92:0/3014579165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2600b2620 0x7fa2601556b0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.173+0000 7fa25f59e700 1 --2- 172.21.15.92:0/3014579165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2600b29f0 0x7fa260153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.173+0000 7fa25f59e700 1 -- 172.21.15.92:0/3014579165 >> 172.21.15.92:0/3014579165 conn(0x7fa260099520 msgr2=0x7fa2600cde80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.173+0000 7fa25f59e700 1 -- 172.21.15.92:0/3014579165 shutdown_connections 2023-10-06T10:43:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.173+0000 7fa25f59e700 1 -- 172.21.15.92:0/3014579165 wait complete. 2023-10-06T10:43:34.187 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:43:34.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.298+0000 7f29be59d700 1 Processor -- start 2023-10-06T10:43:34.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.299+0000 7f29be59d700 1 -- start start 2023-10-06T10:43:34.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.299+0000 7f29be59d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f29b8159440 0x7f29b80b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:34.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.299+0000 7f29be59d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29b80b2100 0x7f29b80b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:34.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.299+0000 7f29be59d700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b80cb7a0 con 0x7f29b80b35d0 2023-10-06T10:43:34.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.299+0000 7f29be59d700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b80b0860 con 0x7f29b8159440 2023-10-06T10:43:34.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.299+0000 7f29be59d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80cb150 con 0x7f29b80b2100 2023-10-06T10:43:34.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.300+0000 7f29bd59b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f29b8159440 0x7f29b80b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:34.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.300+0000 7f29bd59b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f29b8159440 0x7f29b80b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:36124/0 (socket says 172.21.15.92:36124) 2023-10-06T10:43:34.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.300+0000 7f29bd59b700 1 -- 172.21.15.92:0/1164996438 learned_addr learned my addr 172.21.15.92:0/1164996438 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:34.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.300+0000 7f29bcd9a700 1 --1- 172.21.15.92:0/1164996438 >> v1:172.21.15.124:6789/0 conn(0x7f29b80b35d0 0x7f29b80cbc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:41882/0 (socket says 172.21.15.92:41882) 2023-10-06T10:43:34.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.300+0000 7f29bdd9c700 1 --2- 172.21.15.92:0/1164996438 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29b80b2100 0x7f29b80b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:34.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.300+0000 7f29bdd9c700 1 -- 172.21.15.92:0/1164996438 >> v2:172.21.15.124:3300/0 conn(0x7f29b8159440 msgr2=0x7f29b80b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:34.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.300+0000 7f29bdd9c700 1 --2- 172.21.15.92:0/1164996438 >> v2:172.21.15.124:3300/0 conn(0x7f29b8159440 0x7f29b80b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.300+0000 7f29bdd9c700 1 -- 172.21.15.92:0/1164996438 >> v1:172.21.15.124:6789/0 conn(0x7f29b80b35d0 legacy=0x7f29b80cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:34.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.300+0000 7f29bdd9c700 1 -- 172.21.15.92:0/1164996438 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b80cb430 con 0x7f29b80b2100 2023-10-06T10:43:34.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.300+0000 7f29bd59b700 1 --2- 172.21.15.92:0/1164996438 >> v2:172.21.15.124:3300/0 conn(0x7f29b8159440 0x7f29b80b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:43:34.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.300+0000 7f29bdd9c700 1 --2- 172.21.15.92:0/1164996438 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29b80b2100 0x7f29b80b2560 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009e60 tx=0x7f29b0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dc858f32170be47 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:34.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.301+0000 7f29affff700 1 -- 172.21.15.92:0/1164996438 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b000e070 con 0x7f29b80b2100 2023-10-06T10:43:34.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.301+0000 7f29affff700 1 -- 172.21.15.92:0/1164996438 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b000bb60 con 0x7f29b80b2100 2023-10-06T10:43:34.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.301+0000 7f29affff700 1 -- 172.21.15.92:0/1164996438 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0012430 con 0x7f29b80b2100 2023-10-06T10:43:34.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.301+0000 7f29be59d700 1 -- 172.21.15.92:0/1164996438 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29b80b2100 msgr2=0x7f29b80b2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:34.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.301+0000 7f29be59d700 1 --2- 172.21.15.92:0/1164996438 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29b80b2100 0x7f29b80b2560 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009e60 tx=0x7f29b0009780 comp rx=0 tx=0).stop 2023-10-06T10:43:34.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.301+0000 7f29be59d700 1 -- 172.21.15.92:0/1164996438 shutdown_connections 2023-10-06T10:43:34.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.301+0000 7f29be59d700 1 --2- 172.21.15.92:0/1164996438 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29b80b2100 0x7f29b80b2560 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.301+0000 7f29be59d700 1 --2- 172.21.15.92:0/1164996438 >> v2:172.21.15.124:3300/0 conn(0x7f29b8159440 0x7f29b80b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.301+0000 7f29be59d700 1 -- 172.21.15.92:0/1164996438 >> 172.21.15.92:0/1164996438 conn(0x7f29b806ece0 msgr2=0x7f29b806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.301+0000 7f29be59d700 1 -- 172.21.15.92:0/1164996438 shutdown_connections 2023-10-06T10:43:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.301+0000 7f29be59d700 1 -- 172.21.15.92:0/1164996438 wait complete. 2023-10-06T10:43:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.302+0000 7f29be59d700 1 Processor -- start 2023-10-06T10:43:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.302+0000 7f29be59d700 1 -- start start 2023-10-06T10:43:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.303+0000 7f29be59d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29b80b2100 0x7f29b8155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.303+0000 7f29be59d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29b80b35d0 0x7f29b815ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.303+0000 7f29be59d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29b8159440 0x7f29b815f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.303+0000 7f29be59d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80c7470 con 0x7f29b80b35d0 2023-10-06T10:43:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.303+0000 7f29be59d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80c7750 con 0x7f29b80b2100 2023-10-06T10:43:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.303+0000 7f29be59d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80c7a30 con 0x7f29b8159440 2023-10-06T10:43:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.303+0000 7f29bcd9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29b80b35d0 0x7f29b815ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.303+0000 7f29bcd9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29b80b35d0 0x7f29b815ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40680/0 (socket says 172.21.15.92:40680) 2023-10-06T10:43:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.303+0000 7f29bcd9a700 1 -- 172.21.15.92:0/146430182 learned_addr learned my addr 172.21.15.92:0/146430182 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.304+0000 7f29bd59b700 1 --2- 172.21.15.92:0/146430182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29b80b2100 0x7f29b8155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.304+0000 7f29bdd9c700 1 --2- 172.21.15.92:0/146430182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29b8159440 0x7f29b815f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.304+0000 7f29bcd9a700 1 -- 172.21.15.92:0/146430182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29b8159440 msgr2=0x7f29b815f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.304+0000 7f29bcd9a700 1 --2- 172.21.15.92:0/146430182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29b8159440 0x7f29b815f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.304+0000 7f29bcd9a700 1 -- 172.21.15.92:0/146430182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29b80b2100 msgr2=0x7f29b8155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.304+0000 7f29bcd9a700 1 --2- 172.21.15.92:0/146430182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29b80b2100 0x7f29b8155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.304+0000 7f29bcd9a700 1 -- 172.21.15.92:0/146430182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b40094f0 con 0x7f29b80b35d0 2023-10-06T10:43:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.304+0000 7f29bcd9a700 1 --2- 172.21.15.92:0/146430182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29b80b35d0 0x7f29b815ef20 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f29a800c7c0 tx=0x7f29a8007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.304+0000 7f29ad7fa700 1 -- 172.21.15.92:0/146430182 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a8005a70 con 0x7f29b80b35d0 2023-10-06T10:43:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.304+0000 7f29ad7fa700 1 -- 172.21.15.92:0/146430182 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a8005bd0 con 0x7f29b80b35d0 2023-10-06T10:43:34.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.304+0000 7f29be59d700 1 -- 172.21.15.92:0/146430182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b0009a20 con 0x7f29b80b35d0 2023-10-06T10:43:34.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.304+0000 7f29ad7fa700 1 -- 172.21.15.92:0/146430182 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a80070d0 con 0x7f29b80b35d0 2023-10-06T10:43:34.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.304+0000 7f29be59d700 1 -- 172.21.15.92:0/146430182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b80c7ff0 con 0x7f29b80b35d0 2023-10-06T10:43:34.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.305+0000 7f29be59d700 1 -- 172.21.15.92:0/146430182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b80ca630 con 0x7f29b80b35d0 2023-10-06T10:43:34.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.305+0000 7f29ad7fa700 1 -- 172.21.15.92:0/146430182 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f29a8007440 con 0x7f29b80b35d0 2023-10-06T10:43:34.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.306+0000 7f29ad7fa700 1 --2- 172.21.15.92:0/146430182 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f299c066f30 0x7f299c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:34.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.306+0000 7f29bd59b700 1 --2- 172.21.15.92:0/146430182 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f299c066f30 0x7f299c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:34.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.306+0000 7f29ad7fa700 1 -- 172.21.15.92:0/146430182 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f29a8083c30 con 0x7f29b80b35d0 2023-10-06T10:43:34.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.306+0000 7f29bd59b700 1 --2- 172.21.15.92:0/146430182 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f299c066f30 0x7f299c0693e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f29b4006d20 tx=0x7f29b4009250 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:34.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.310+0000 7f29ad7fa700 1 -- 172.21.15.92:0/146430182 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29a8050c00 con 0x7f29b80b35d0 2023-10-06T10:43:34.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.535+0000 7f29be59d700 1 -- 172.21.15.92:0/146430182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29b80c8210 con 0x7f29b80b35d0 2023-10-06T10:43:34.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.536+0000 7f29ad7fa700 1 -- 172.21.15.92:0/146430182 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4639 (secure 0 0 0) 0x7f29a8017020 con 0x7f29b80b35d0 2023-10-06T10:43:34.538 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:34.538 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":75,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3469511807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3469511807},{"type":"v1","addr":"172.21.15.124:6834","nonce":3469511807}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:03.173076+0000","modified":"2023-10-06T10:43:33.397340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7164,"mds_1":7149,"mds_2":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"a","rank":2,"incarnation":71,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/4003376156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4003376156},{"type":"v1","addr":"172.21.15.92:6837","nonce":4003376156}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7149":{"gid":7149,"name":"c","rank":1,"incarnation":69,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6836/3268908749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3268908749},{"type":"v1","addr":"172.21.15.92:6836","nonce":3268908749}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7164":{"gid":7164,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/2971348516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2971348516},{"type":"v1","addr":"172.21.15.124:6835","nonce":2971348516}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T10:43:34.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.538+0000 7f29be59d700 1 -- 172.21.15.92:0/146430182 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f299c066f30 msgr2=0x7f299c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:34.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.538+0000 7f29be59d700 1 --2- 172.21.15.92:0/146430182 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f299c066f30 0x7f299c0693e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f29b4006d20 tx=0x7f29b4009250 comp rx=0 tx=0).stop 2023-10-06T10:43:34.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.538+0000 7f29be59d700 1 -- 172.21.15.92:0/146430182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29b80b35d0 msgr2=0x7f29b815ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:34.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.538+0000 7f29be59d700 1 --2- 172.21.15.92:0/146430182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29b80b35d0 0x7f29b815ef20 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f29a800c7c0 tx=0x7f29a8007f30 comp rx=0 tx=0).stop 2023-10-06T10:43:34.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.538+0000 7f29be59d700 1 -- 172.21.15.92:0/146430182 shutdown_connections 2023-10-06T10:43:34.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.538+0000 7f29be59d700 1 --2- 172.21.15.92:0/146430182 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f299c066f30 0x7f299c0693e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.538+0000 7f29be59d700 1 --2- 172.21.15.92:0/146430182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29b8159440 0x7f29b815f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.538+0000 7f29be59d700 1 --2- 172.21.15.92:0/146430182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29b80b2100 0x7f29b8155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.538+0000 7f29be59d700 1 --2- 172.21.15.92:0/146430182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29b80b35d0 0x7f29b815ef20 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.538+0000 7f29be59d700 1 -- 172.21.15.92:0/146430182 >> 172.21.15.92:0/146430182 conn(0x7f29b806ece0 msgr2=0x7f29b80b4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:34.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.538+0000 7f29be59d700 1 -- 172.21.15.92:0/146430182 shutdown_connections 2023-10-06T10:43:34.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.538+0000 7f29be59d700 1 -- 172.21.15.92:0/146430182 wait complete. 2023-10-06T10:43:34.541 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 75 2023-10-06T10:43:34.551 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:43:34.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.663+0000 7fda90f57700 1 Processor -- start 2023-10-06T10:43:34.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.663+0000 7fda90f57700 1 -- start start 2023-10-06T10:43:34.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.664+0000 7fda90f57700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fda8c159560 0x7fda8c0aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:34.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.664+0000 7fda90f57700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fda8c0b49a0 0x7fda8c0aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:34.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.664+0000 7fda90f57700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda8c0b0c60 con 0x7fda8c0b0760 2023-10-06T10:43:34.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.664+0000 7fda90f57700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda8c0b0f40 con 0x7fda8c0b49a0 2023-10-06T10:43:34.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.664+0000 7fda90f57700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda8c0b1220 con 0x7fda8c159560 2023-10-06T10:43:34.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.664+0000 7fda89d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fda8c0b49a0 0x7fda8c0aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:34.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.664+0000 7fda89d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fda8c0b49a0 0x7fda8c0aaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40694/0 (socket says 172.21.15.92:40694) 2023-10-06T10:43:34.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.664+0000 7fda89d9b700 1 -- 172.21.15.92:0/1297992768 learned_addr learned my addr 172.21.15.92:0/1297992768 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:34.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.664+0000 7fda8a59c700 1 --2- 172.21.15.92:0/1297992768 >> v2:172.21.15.124:3300/0 conn(0x7fda8c159560 0x7fda8c0aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:34.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.664+0000 7fda89d9b700 1 -- 172.21.15.92:0/1297992768 >> v2:172.21.15.124:3300/0 conn(0x7fda8c159560 msgr2=0x7fda8c0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:34.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.664+0000 7fda89d9b700 1 --2- 172.21.15.92:0/1297992768 >> v2:172.21.15.124:3300/0 conn(0x7fda8c159560 0x7fda8c0aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.664+0000 7fda89d9b700 1 -- 172.21.15.92:0/1297992768 >> v1:172.21.15.124:6789/0 conn(0x7fda8c0b0760 legacy=0x7fda8c0b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:34.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.664+0000 7fda89d9b700 1 -- 172.21.15.92:0/1297992768 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8c0b1500 con 0x7fda8c0b49a0 2023-10-06T10:43:34.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.664+0000 7fda89d9b700 1 --2- 172.21.15.92:0/1297992768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda8c0b49a0 0x7fda8c0aaa70 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fda740099b0 tx=0x7fda74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4d42640151d065 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:34.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.664+0000 7fda8959a700 1 -- 172.21.15.92:0/1297992768 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda74009ea0 con 0x7fda8c0b49a0 2023-10-06T10:43:34.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.664+0000 7fda8959a700 1 -- 172.21.15.92:0/1297992768 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7400b820 con 0x7fda8c0b49a0 2023-10-06T10:43:34.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.664+0000 7fda8959a700 1 -- 172.21.15.92:0/1297992768 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7400bbc0 con 0x7fda8c0b49a0 2023-10-06T10:43:34.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.665+0000 7fda90f57700 1 -- 172.21.15.92:0/1297992768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda8c0b49a0 msgr2=0x7fda8c0aaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:34.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.665+0000 7fda90f57700 1 --2- 172.21.15.92:0/1297992768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda8c0b49a0 0x7fda8c0aaa70 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fda740099b0 tx=0x7fda74009250 comp rx=0 tx=0).stop 2023-10-06T10:43:34.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.665+0000 7fda90f57700 1 -- 172.21.15.92:0/1297992768 shutdown_connections 2023-10-06T10:43:34.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.665+0000 7fda90f57700 1 --2- 172.21.15.92:0/1297992768 >> v2:172.21.15.124:3300/0 conn(0x7fda8c159560 0x7fda8c0aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.665+0000 7fda90f57700 1 --2- 172.21.15.92:0/1297992768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda8c0b49a0 0x7fda8c0aaa70 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.665+0000 7fda90f57700 1 -- 172.21.15.92:0/1297992768 >> 172.21.15.92:0/1297992768 conn(0x7fda8c0997f0 msgr2=0x7fda8c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:34.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.665+0000 7fda90f57700 1 -- 172.21.15.92:0/1297992768 shutdown_connections 2023-10-06T10:43:34.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.665+0000 7fda90f57700 1 -- 172.21.15.92:0/1297992768 wait complete. 2023-10-06T10:43:34.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.666+0000 7fda90f57700 1 Processor -- start 2023-10-06T10:43:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.666+0000 7fda90f57700 1 -- start start 2023-10-06T10:43:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.667+0000 7fda90f57700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda8c0b0760 0x7fda8c141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.667+0000 7fda90f57700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda8c0b49a0 0x7fda8c153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.667+0000 7fda90f57700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda8c159560 0x7fda8c154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.667+0000 7fda90f57700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c140050 con 0x7fda8c159560 2023-10-06T10:43:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.667+0000 7fda90f57700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c13f440 con 0x7fda8c0b49a0 2023-10-06T10:43:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.667+0000 7fda90f57700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c13f720 con 0x7fda8c0b0760 2023-10-06T10:43:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.667+0000 7fda8ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda8c159560 0x7fda8c154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.667+0000 7fda8ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda8c159560 0x7fda8c154440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40708/0 (socket says 172.21.15.92:40708) 2023-10-06T10:43:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.667+0000 7fda8ad9d700 1 -- 172.21.15.92:0/3016456573 learned_addr learned my addr 172.21.15.92:0/3016456573 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.667+0000 7fda8a59c700 1 --2- 172.21.15.92:0/3016456573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda8c0b0760 0x7fda8c141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:34.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.667+0000 7fda89d9b700 1 --2- 172.21.15.92:0/3016456573 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda8c0b49a0 0x7fda8c153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:34.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.667+0000 7fda8ad9d700 1 -- 172.21.15.92:0/3016456573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda8c0b0760 msgr2=0x7fda8c141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:34.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.667+0000 7fda8ad9d700 1 --2- 172.21.15.92:0/3016456573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda8c0b0760 0x7fda8c141da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.667+0000 7fda8ad9d700 1 -- 172.21.15.92:0/3016456573 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda8c0b49a0 msgr2=0x7fda8c153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:34.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.667+0000 7fda8ad9d700 1 --2- 172.21.15.92:0/3016456573 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda8c0b49a0 0x7fda8c153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.667+0000 7fda8ad9d700 1 -- 172.21.15.92:0/3016456573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda740094f0 con 0x7fda8c159560 2023-10-06T10:43:34.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.667+0000 7fda8a59c700 1 --2- 172.21.15.92:0/3016456573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda8c0b0760 0x7fda8c141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:34.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.668+0000 7fda8ad9d700 1 --2- 172.21.15.92:0/3016456573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda8c159560 0x7fda8c154440 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00c560 tx=0x7fda7c007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:34.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.668+0000 7fda827fc700 1 -- 172.21.15.92:0/3016456573 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c011590 con 0x7fda8c159560 2023-10-06T10:43:34.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.668+0000 7fda827fc700 1 -- 172.21.15.92:0/3016456573 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7c0116f0 con 0x7fda8c159560 2023-10-06T10:43:34.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.668+0000 7fda90f57700 1 -- 172.21.15.92:0/3016456573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda8c13fa00 con 0x7fda8c159560 2023-10-06T10:43:34.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.668+0000 7fda827fc700 1 -- 172.21.15.92:0/3016456573 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c011ab0 con 0x7fda8c159560 2023-10-06T10:43:34.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.668+0000 7fda90f57700 1 -- 172.21.15.92:0/3016456573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda8c154980 con 0x7fda8c159560 2023-10-06T10:43:34.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.669+0000 7fda90f57700 1 -- 172.21.15.92:0/3016456573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda8c064a90 con 0x7fda8c159560 2023-10-06T10:43:34.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.669+0000 7fda827fc700 1 -- 172.21.15.92:0/3016456573 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fda7c01b3b0 con 0x7fda8c159560 2023-10-06T10:43:34.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.669+0000 7fda827fc700 1 --2- 172.21.15.92:0/3016456573 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda6c066f30 0x7fda6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:34.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.669+0000 7fda827fc700 1 -- 172.21.15.92:0/3016456573 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fda7c083c30 con 0x7fda8c159560 2023-10-06T10:43:34.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.669+0000 7fda8a59c700 1 --2- 172.21.15.92:0/3016456573 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda6c066f30 0x7fda6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:34.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.670+0000 7fda8a59c700 1 --2- 172.21.15.92:0/3016456573 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda6c066f30 0x7fda6c0693e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fda8c0b4840 tx=0x7fda8400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:34.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.674+0000 7fda827fc700 1 -- 172.21.15.92:0/3016456573 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda7c050c00 con 0x7fda8c159560 2023-10-06T10:43:34.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.861+0000 7fda90f57700 1 -- 172.21.15.92:0/3016456573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda8c0b17e0 con 0x7fda8c159560 2023-10-06T10:43:34.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.862+0000 7fda827fc700 1 -- 172.21.15.92:0/3016456573 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+17074 (secure 0 0 0) 0x7fda7c017070 con 0x7fda8c159560 2023-10-06T10:43:34.864 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:34.864 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":78,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:43:04.326791+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:43:00.816800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-06T10:43:02.128579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:43:34.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.864+0000 7fda90f57700 1 -- 172.21.15.92:0/3016456573 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda6c066f30 msgr2=0x7fda6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:34.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.864+0000 7fda90f57700 1 --2- 172.21.15.92:0/3016456573 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda6c066f30 0x7fda6c0693e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fda8c0b4840 tx=0x7fda8400b040 comp rx=0 tx=0).stop 2023-10-06T10:43:34.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.864+0000 7fda90f57700 1 -- 172.21.15.92:0/3016456573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda8c159560 msgr2=0x7fda8c154440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:34.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.864+0000 7fda90f57700 1 --2- 172.21.15.92:0/3016456573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda8c159560 0x7fda8c154440 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00c560 tx=0x7fda7c007ef0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.864+0000 7fda90f57700 1 -- 172.21.15.92:0/3016456573 shutdown_connections 2023-10-06T10:43:34.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.864+0000 7fda90f57700 1 --2- 172.21.15.92:0/3016456573 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda6c066f30 0x7fda6c0693e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.864+0000 7fda90f57700 1 --2- 172.21.15.92:0/3016456573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda8c159560 0x7fda8c154440 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.864+0000 7fda90f57700 1 --2- 172.21.15.92:0/3016456573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda8c0b0760 0x7fda8c141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.864+0000 7fda90f57700 1 --2- 172.21.15.92:0/3016456573 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda8c0b49a0 0x7fda8c153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.864+0000 7fda90f57700 1 -- 172.21.15.92:0/3016456573 >> 172.21.15.92:0/3016456573 conn(0x7fda8c0997f0 msgr2=0x7fda8c0d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:34.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.864+0000 7fda90f57700 1 -- 172.21.15.92:0/3016456573 shutdown_connections 2023-10-06T10:43:34.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.864+0000 7fda90f57700 1 -- 172.21.15.92:0/3016456573 wait complete. 2023-10-06T10:43:34.878 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T10:43:34.878 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:43:34.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.990+0000 7f9866bbe700 1 Processor -- start 2023-10-06T10:43:34.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.990+0000 7f9866bbe700 1 -- start start 2023-10-06T10:43:34.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.991+0000 7f9866bbe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9860159440 0x7f98600b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:34.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.991+0000 7f9866bbe700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f98600b2100 0x7f98600b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:34.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.991+0000 7f9866bbe700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98600cb7a0 con 0x7f98600b35d0 2023-10-06T10:43:34.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.991+0000 7f9866bbe700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98600b0860 con 0x7f9860159440 2023-10-06T10:43:34.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.991+0000 7f9866bbe700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98600cb150 con 0x7f98600b2100 2023-10-06T10:43:34.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.991+0000 7f9865bbc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9860159440 0x7f98600b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:34.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.991+0000 7f9865bbc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9860159440 0x7f98600b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40712/0 (socket says 172.21.15.92:40712) 2023-10-06T10:43:34.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.991+0000 7f9865bbc700 1 -- 172.21.15.92:0/3363375842 learned_addr learned my addr 172.21.15.92:0/3363375842 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:34.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.991+0000 7f9865bbc700 1 -- 172.21.15.92:0/3363375842 >> v2:172.21.15.124:3300/0 conn(0x7f98600b2100 msgr2=0x7f98600b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:34.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.991+0000 7f9865bbc700 1 --2- 172.21.15.92:0/3363375842 >> v2:172.21.15.124:3300/0 conn(0x7f98600b2100 0x7f98600b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.991+0000 7f9865bbc700 1 -- 172.21.15.92:0/3363375842 >> v1:172.21.15.124:6789/0 conn(0x7f98600b35d0 legacy=0x7f98600cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:34.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.991+0000 7f9865bbc700 1 -- 172.21.15.92:0/3363375842 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98600cb430 con 0x7f9860159440 2023-10-06T10:43:34.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.992+0000 7f9865bbc700 1 --2- 172.21.15.92:0/3363375842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9860159440 0x7f98600b3090 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f98600ca7f0 tx=0x7f985c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3d89dfbb58e204 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:34.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.992+0000 7f9864bba700 1 -- 172.21.15.92:0/3363375842 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c00b820 con 0x7f9860159440 2023-10-06T10:43:34.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.992+0000 7f9864bba700 1 -- 172.21.15.92:0/3363375842 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985c00b980 con 0x7f9860159440 2023-10-06T10:43:34.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.992+0000 7f9864bba700 1 -- 172.21.15.92:0/3363375842 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c00bd20 con 0x7f9860159440 2023-10-06T10:43:34.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.992+0000 7f9866bbe700 1 -- 172.21.15.92:0/3363375842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9860159440 msgr2=0x7f98600b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:34.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.992+0000 7f9866bbe700 1 --2- 172.21.15.92:0/3363375842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9860159440 0x7f98600b3090 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f98600ca7f0 tx=0x7f985c009250 comp rx=0 tx=0).stop 2023-10-06T10:43:34.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.992+0000 7f9866bbe700 1 -- 172.21.15.92:0/3363375842 shutdown_connections 2023-10-06T10:43:34.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.992+0000 7f9866bbe700 1 --2- 172.21.15.92:0/3363375842 >> v2:172.21.15.124:3300/0 conn(0x7f98600b2100 0x7f98600b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.992+0000 7f9866bbe700 1 --2- 172.21.15.92:0/3363375842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9860159440 0x7f98600b3090 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:34.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.992+0000 7f9866bbe700 1 -- 172.21.15.92:0/3363375842 >> 172.21.15.92:0/3363375842 conn(0x7f986006ece0 msgr2=0x7f986006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:34.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.992+0000 7f9866bbe700 1 -- 172.21.15.92:0/3363375842 shutdown_connections 2023-10-06T10:43:34.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.992+0000 7f9866bbe700 1 -- 172.21.15.92:0/3363375842 wait complete. 2023-10-06T10:43:34.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.993+0000 7f9866bbe700 1 Processor -- start 2023-10-06T10:43:34.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.993+0000 7f9866bbe700 1 -- start start 2023-10-06T10:43:34.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.994+0000 7f9866bbe700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98600b2100 0x7f98600c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:34.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.994+0000 7f9866bbe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98600b35d0 0x7f9860159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:34.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.994+0000 7f9866bbe700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9860159440 0x7f986015c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:35.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.994+0000 7f9866bbe700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9860159fa0 con 0x7f98600b35d0 2023-10-06T10:43:35.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.994+0000 7f9866bbe700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98601510c0 con 0x7f9860159440 2023-10-06T10:43:35.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.994+0000 7f9866bbe700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98601513a0 con 0x7f98600b2100 2023-10-06T10:43:35.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.994+0000 7f98653bb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98600b35d0 0x7f9860159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:35.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.994+0000 7f98653bb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98600b35d0 0x7f9860159a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40726/0 (socket says 172.21.15.92:40726) 2023-10-06T10:43:35.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.994+0000 7f98653bb700 1 -- 172.21.15.92:0/3104708658 learned_addr learned my addr 172.21.15.92:0/3104708658 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:35.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.994+0000 7f9865bbc700 1 --2- 172.21.15.92:0/3104708658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98600b2100 0x7f98600c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:35.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.994+0000 7f98663bd700 1 --2- 172.21.15.92:0/3104708658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9860159440 0x7f986015c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:35.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.995+0000 7f98653bb700 1 -- 172.21.15.92:0/3104708658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98600b2100 msgr2=0x7f98600c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:35.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.995+0000 7f98653bb700 1 --2- 172.21.15.92:0/3104708658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98600b2100 0x7f98600c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.995+0000 7f98653bb700 1 -- 172.21.15.92:0/3104708658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9860159440 msgr2=0x7f986015c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:35.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.995+0000 7f98653bb700 1 --2- 172.21.15.92:0/3104708658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9860159440 0x7f986015c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.995+0000 7f98653bb700 1 -- 172.21.15.92:0/3104708658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c0094f0 con 0x7f98600b35d0 2023-10-06T10:43:35.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.995+0000 7f98663bd700 1 --2- 172.21.15.92:0/3104708658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9860159440 0x7f986015c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:35.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.995+0000 7f98653bb700 1 --2- 172.21.15.92:0/3104708658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98600b35d0 0x7f9860159a60 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f985000b630 tx=0x7f98500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:35.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.995+0000 7f9855ffb700 1 -- 172.21.15.92:0/3104708658 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985000f510 con 0x7f98600b35d0 2023-10-06T10:43:35.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.995+0000 7f9855ffb700 1 -- 172.21.15.92:0/3104708658 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985000f690 con 0x7f98600b35d0 2023-10-06T10:43:35.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.995+0000 7f9866bbe700 1 -- 172.21.15.92:0/3104708658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f986015a190 con 0x7f98600b35d0 2023-10-06T10:43:35.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.995+0000 7f9855ffb700 1 -- 172.21.15.92:0/3104708658 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985000fa50 con 0x7f98600b35d0 2023-10-06T10:43:35.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.995+0000 7f9866bbe700 1 -- 172.21.15.92:0/3104708658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986015a750 con 0x7f98600b35d0 2023-10-06T10:43:35.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.996+0000 7f9866bbe700 1 -- 172.21.15.92:0/3104708658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f986005f230 con 0x7f98600b35d0 2023-10-06T10:43:35.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.996+0000 7f9855ffb700 1 -- 172.21.15.92:0/3104708658 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f985001a400 con 0x7f98600b35d0 2023-10-06T10:43:35.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.997+0000 7f9855ffb700 1 --2- 172.21.15.92:0/3104708658 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9844066f30 0x7f98440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:35.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.997+0000 7f9855ffb700 1 -- 172.21.15.92:0/3104708658 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f98500841f0 con 0x7f98600b35d0 2023-10-06T10:43:35.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.997+0000 7f9865bbc700 1 --2- 172.21.15.92:0/3104708658 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9844066f30 0x7f98440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:35.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:34.997+0000 7f9865bbc700 1 --2- 172.21.15.92:0/3104708658 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9844066f30 0x7f98440693e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f98600ca7f0 tx=0x7f985c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:35.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.001+0000 7f9855ffb700 1 -- 172.21.15.92:0/3104708658 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9850051240 con 0x7f98600b35d0 2023-10-06T10:43:35.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.225+0000 7f9866bbe700 1 -- 172.21.15.92:0/3104708658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9860159810 con 0x7f98600b35d0 2023-10-06T10:43:35.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.225+0000 7f9855ffb700 1 -- 172.21.15.92:0/3104708658 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4639 (secure 0 0 0) 0x7f9850054860 con 0x7f98600b35d0 2023-10-06T10:43:35.227 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:35.228 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":75,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3469511807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3469511807},{"type":"v1","addr":"172.21.15.124:6834","nonce":3469511807}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:03.173076+0000","modified":"2023-10-06T10:43:33.397340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7164,"mds_1":7149,"mds_2":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"a","rank":2,"incarnation":71,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/4003376156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4003376156},{"type":"v1","addr":"172.21.15.92:6837","nonce":4003376156}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7149":{"gid":7149,"name":"c","rank":1,"incarnation":69,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6836/3268908749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3268908749},{"type":"v1","addr":"172.21.15.92:6836","nonce":3268908749}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7164":{"gid":7164,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/2971348516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2971348516},{"type":"v1","addr":"172.21.15.124:6835","nonce":2971348516}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T10:43:35.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.227+0000 7f9866bbe700 1 -- 172.21.15.92:0/3104708658 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9844066f30 msgr2=0x7f98440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:35.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.227+0000 7f9866bbe700 1 --2- 172.21.15.92:0/3104708658 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9844066f30 0x7f98440693e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f98600ca7f0 tx=0x7f985c016000 comp rx=0 tx=0).stop 2023-10-06T10:43:35.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.227+0000 7f9866bbe700 1 -- 172.21.15.92:0/3104708658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98600b35d0 msgr2=0x7f9860159a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:35.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.227+0000 7f9866bbe700 1 --2- 172.21.15.92:0/3104708658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98600b35d0 0x7f9860159a60 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f985000b630 tx=0x7f98500091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.227+0000 7f9866bbe700 1 -- 172.21.15.92:0/3104708658 shutdown_connections 2023-10-06T10:43:35.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.227+0000 7f9866bbe700 1 --2- 172.21.15.92:0/3104708658 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9844066f30 0x7f98440693e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.227+0000 7f9866bbe700 1 --2- 172.21.15.92:0/3104708658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9860159440 0x7f986015c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.227+0000 7f9866bbe700 1 --2- 172.21.15.92:0/3104708658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98600b2100 0x7f98600c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.227+0000 7f9866bbe700 1 --2- 172.21.15.92:0/3104708658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98600b35d0 0x7f9860159a60 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.227+0000 7f9866bbe700 1 -- 172.21.15.92:0/3104708658 >> 172.21.15.92:0/3104708658 conn(0x7f986006ece0 msgr2=0x7f9860154bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:35.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.227+0000 7f9866bbe700 1 -- 172.21.15.92:0/3104708658 shutdown_connections 2023-10-06T10:43:35.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.227+0000 7f9866bbe700 1 -- 172.21.15.92:0/3104708658 wait complete. 2023-10-06T10:43:35.231 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 75 2023-10-06T10:43:35.241 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:43:35.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.355+0000 7f35cb59e700 1 Processor -- start 2023-10-06T10:43:35.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.355+0000 7f35cb59e700 1 -- start start 2023-10-06T10:43:35.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.356+0000 7f35cb59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f35cc0cbc30 0x7f35cc0cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:35.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.356+0000 7f35cb59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35cc0cb7a0 con 0x7f35cc0b2100 2023-10-06T10:43:35.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.356+0000 7f35cb59e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35cc0b0860 con 0x7f35cc159440 2023-10-06T10:43:35.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.356+0000 7f35cb59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35cc0cb150 con 0x7f35cc0cbc30 2023-10-06T10:43:35.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.356+0000 7f35c9d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f35cc0cbc30 0x7f35cc0cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:35.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.356+0000 7f35cad9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f35cc0b2100 0x7f35cc0b2520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49130/0 (socket says 172.21.15.92:49130) 2023-10-06T10:43:35.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.356+0000 7f35cad9d700 1 -- 172.21.15.92:0/3124079468 learned_addr learned my addr 172.21.15.92:0/3124079468 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:35.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.356+0000 7f35c9d9b700 1 -- 172.21.15.92:0/3124079468 >> v1:172.21.15.124:6789/0 conn(0x7f35cc159440 legacy=0x7f35cc0b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:35.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.356+0000 7f35c9d9b700 1 -- 172.21.15.92:0/3124079468 >> v1:172.21.15.92:6789/0 conn(0x7f35cc0b2100 legacy=0x7f35cc0b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:35.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.356+0000 7f35c959a700 1 -- 172.21.15.92:0/3124079468 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249739485 0 0) 0x7f35cc0cb7a0 con 0x7f35cc0b2100 2023-10-06T10:43:35.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.356+0000 7f35c9d9b700 1 -- 172.21.15.92:0/3124079468 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35cc0cb430 con 0x7f35cc0cbc30 2023-10-06T10:43:35.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.356+0000 7f35c9d9b700 1 --2- 172.21.15.92:0/3124079468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35cc0cbc30 0x7f35cc0cc080 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f35c00098e0 tx=0x7f35c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbd07cac7a27a12 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:35.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.356+0000 7f35c959a700 1 -- 172.21.15.92:0/3124079468 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c0009e80 con 0x7f35cc0cbc30 2023-10-06T10:43:35.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.356+0000 7f35c959a700 1 -- 172.21.15.92:0/3124079468 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c000b860 con 0x7f35cc0cbc30 2023-10-06T10:43:35.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.356+0000 7f35c959a700 1 -- 172.21.15.92:0/3124079468 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c000bbe0 con 0x7f35cc0cbc30 2023-10-06T10:43:35.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.357+0000 7f35cb59e700 1 -- 172.21.15.92:0/3124079468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35cc0cbc30 msgr2=0x7f35cc0cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:35.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.357+0000 7f35cb59e700 1 --2- 172.21.15.92:0/3124079468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35cc0cbc30 0x7f35cc0cc080 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f35c00098e0 tx=0x7f35c00091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.357+0000 7f35cb59e700 1 -- 172.21.15.92:0/3124079468 shutdown_connections 2023-10-06T10:43:35.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.357+0000 7f35cb59e700 1 --2- 172.21.15.92:0/3124079468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35cc0cbc30 0x7f35cc0cc080 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.357+0000 7f35cb59e700 1 -- 172.21.15.92:0/3124079468 >> 172.21.15.92:0/3124079468 conn(0x7f35cc06ece0 msgr2=0x7f35cc06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:35.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.357+0000 7f35cb59e700 1 -- 172.21.15.92:0/3124079468 shutdown_connections 2023-10-06T10:43:35.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.357+0000 7f35cb59e700 1 -- 172.21.15.92:0/3124079468 wait complete. 2023-10-06T10:43:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.358+0000 7f35cb59e700 1 Processor -- start 2023-10-06T10:43:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.358+0000 7f35cb59e700 1 -- start start 2023-10-06T10:43:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.359+0000 7f35cb59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35cc0b2100 0x7f35cc153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.359+0000 7f35cb59e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35cc0cbc30 0x7f35cc0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.359+0000 7f35cb59e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35cc159440 0x7f35cc0b4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.359+0000 7f35cb59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc0c74e0 con 0x7f35cc0b2100 2023-10-06T10:43:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.359+0000 7f35cb59e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc16a480 con 0x7f35cc159440 2023-10-06T10:43:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.359+0000 7f35cb59e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc153f70 con 0x7f35cc0cbc30 2023-10-06T10:43:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.359+0000 7f35ca59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35cc0b2100 0x7f35cc153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.359+0000 7f35ca59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35cc0b2100 0x7f35cc153a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40744/0 (socket says 172.21.15.92:40744) 2023-10-06T10:43:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.359+0000 7f35ca59c700 1 -- 172.21.15.92:0/1533758661 learned_addr learned my addr 172.21.15.92:0/1533758661 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.359+0000 7f35cad9d700 1 --2- 172.21.15.92:0/1533758661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35cc159440 0x7f35cc0b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.359+0000 7f35c9d9b700 1 --2- 172.21.15.92:0/1533758661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35cc0cbc30 0x7f35cc0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.359+0000 7f35ca59c700 1 -- 172.21.15.92:0/1533758661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35cc0cbc30 msgr2=0x7f35cc0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.359+0000 7f35ca59c700 1 --2- 172.21.15.92:0/1533758661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35cc0cbc30 0x7f35cc0b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.359+0000 7f35ca59c700 1 -- 172.21.15.92:0/1533758661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35cc159440 msgr2=0x7f35cc0b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.359+0000 7f35ca59c700 1 --2- 172.21.15.92:0/1533758661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35cc159440 0x7f35cc0b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.359+0000 7f35ca59c700 1 -- 172.21.15.92:0/1533758661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c0009420 con 0x7f35cc0b2100 2023-10-06T10:43:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.359+0000 7f35ca59c700 1 --2- 172.21.15.92:0/1533758661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35cc0b2100 0x7f35cc153a30 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00c510 tx=0x7f35bc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.359+0000 7f35ba7fc700 1 -- 172.21.15.92:0/1533758661 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc011590 con 0x7f35cc0b2100 2023-10-06T10:43:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.359+0000 7f35ba7fc700 1 -- 172.21.15.92:0/1533758661 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35bc011710 con 0x7f35cc0b2100 2023-10-06T10:43:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.359+0000 7f35cb59e700 1 -- 172.21.15.92:0/1533758661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35cc169870 con 0x7f35cc0b2100 2023-10-06T10:43:35.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.360+0000 7f35ba7fc700 1 -- 172.21.15.92:0/1533758661 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc011ad0 con 0x7f35cc0b2100 2023-10-06T10:43:35.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.360+0000 7f35cb59e700 1 -- 172.21.15.92:0/1533758661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35cc169e30 con 0x7f35cc0b2100 2023-10-06T10:43:35.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.360+0000 7f35cb59e700 1 -- 172.21.15.92:0/1533758661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35cc16a810 con 0x7f35cc0b2100 2023-10-06T10:43:35.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.361+0000 7f35ba7fc700 1 -- 172.21.15.92:0/1533758661 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f35bc01c440 con 0x7f35cc0b2100 2023-10-06T10:43:35.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.361+0000 7f35ba7fc700 1 --2- 172.21.15.92:0/1533758661 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f35ac066f30 0x7f35ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:35.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.361+0000 7f35ba7fc700 1 -- 172.21.15.92:0/1533758661 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f35bc083ce0 con 0x7f35cc0b2100 2023-10-06T10:43:35.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.361+0000 7f35c9d9b700 1 --2- 172.21.15.92:0/1533758661 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f35ac066f30 0x7f35ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:35.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.362+0000 7f35c9d9b700 1 --2- 172.21.15.92:0/1533758661 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f35ac066f30 0x7f35ac0693e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f35c00093f0 tx=0x7f35c0017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:35.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.365+0000 7f35ba7fc700 1 -- 172.21.15.92:0/1533758661 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35cc16a810 con 0x7f35cc0b2100 2023-10-06T10:43:35.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.590+0000 7f35cb59e700 1 -- 172.21.15.92:0/1533758661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35cc16a810 con 0x7f35cc0b2100 2023-10-06T10:43:35.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.591+0000 7f35ba7fc700 1 -- 172.21.15.92:0/1533758661 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4639 (secure 0 0 0) 0x7f35cc16a810 con 0x7f35cc0b2100 2023-10-06T10:43:35.593 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:35.593 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":75,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3469511807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3469511807},{"type":"v1","addr":"172.21.15.124:6834","nonce":3469511807}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:03.173076+0000","modified":"2023-10-06T10:43:33.397340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7164,"mds_1":7149,"mds_2":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"a","rank":2,"incarnation":71,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/4003376156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4003376156},{"type":"v1","addr":"172.21.15.92:6837","nonce":4003376156}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7149":{"gid":7149,"name":"c","rank":1,"incarnation":69,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6836/3268908749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3268908749},{"type":"v1","addr":"172.21.15.92:6836","nonce":3268908749}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7164":{"gid":7164,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/2971348516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2971348516},{"type":"v1","addr":"172.21.15.124:6835","nonce":2971348516}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T10:43:35.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.592+0000 7f35cb59e700 1 -- 172.21.15.92:0/1533758661 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f35ac066f30 msgr2=0x7f35ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:35.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.592+0000 7f35cb59e700 1 --2- 172.21.15.92:0/1533758661 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f35ac066f30 0x7f35ac0693e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f35c00093f0 tx=0x7f35c0017000 comp rx=0 tx=0).stop 2023-10-06T10:43:35.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.592+0000 7f35cb59e700 1 -- 172.21.15.92:0/1533758661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35cc0b2100 msgr2=0x7f35cc153a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:35.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.592+0000 7f35cb59e700 1 --2- 172.21.15.92:0/1533758661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35cc0b2100 0x7f35cc153a30 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00c510 tx=0x7f35bc00c8d0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.592+0000 7f35cb59e700 1 -- 172.21.15.92:0/1533758661 shutdown_connections 2023-10-06T10:43:35.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.592+0000 7f35cb59e700 1 --2- 172.21.15.92:0/1533758661 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f35ac066f30 0x7f35ac0693e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.592+0000 7f35cb59e700 1 --2- 172.21.15.92:0/1533758661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35cc159440 0x7f35cc0b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.592+0000 7f35cb59e700 1 --2- 172.21.15.92:0/1533758661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35cc0b2100 0x7f35cc153a30 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.592+0000 7f35cb59e700 1 --2- 172.21.15.92:0/1533758661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35cc0cbc30 0x7f35cc0b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.592+0000 7f35cb59e700 1 -- 172.21.15.92:0/1533758661 >> 172.21.15.92:0/1533758661 conn(0x7f35cc06ece0 msgr2=0x7f35cc0b57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:35.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.593+0000 7f35cb59e700 1 -- 172.21.15.92:0/1533758661 shutdown_connections 2023-10-06T10:43:35.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.593+0000 7f35cb59e700 1 -- 172.21.15.92:0/1533758661 wait complete. 2023-10-06T10:43:35.596 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 75 2023-10-06T10:43:35.606 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:43:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.718+0000 7fc70a361700 1 Processor -- start 2023-10-06T10:43:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.718+0000 7fc70a361700 1 -- start start 2023-10-06T10:43:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.719+0000 7fc70a361700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc7040696e0 0x7fc704069ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.719+0000 7fc70a361700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc70406d350 0x7fc70406eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.719+0000 7fc70a361700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc70406e6d0 con 0x7fc7040ada60 2023-10-06T10:43:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.719+0000 7fc70a361700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7040cd000 con 0x7fc7040696e0 2023-10-06T10:43:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.719+0000 7fc70a361700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc70406dac0 con 0x7fc70406d350 2023-10-06T10:43:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.719+0000 7fc7037fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc7040ada60 0x7fc7040aeeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49132/0 (socket says 172.21.15.92:49132) 2023-10-06T10:43:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.719+0000 7fc7037fe700 1 -- 172.21.15.92:0/3984777133 learned_addr learned my addr 172.21.15.92:0/3984777133 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc7088fe700 1 --2- 172.21.15.92:0/3984777133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc70406d350 0x7fc70406eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc703fff700 1 --2- 172.21.15.92:0/3984777133 >> v2:172.21.15.124:3300/0 conn(0x7fc7040696e0 0x7fc704069ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc702ffd700 1 -- 172.21.15.92:0/3984777133 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 185880517 0 0) 0x7fc70406e6d0 con 0x7fc7040ada60 2023-10-06T10:43:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc702ffd700 1 -- 172.21.15.92:0/3984777133 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6ec003570 con 0x7fc7040ada60 2023-10-06T10:43:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc702ffd700 1 -- 172.21.15.92:0/3984777133 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 558692671 0 0) 0x7fc6ec003570 con 0x7fc7040ada60 2023-10-06T10:43:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc702ffd700 1 -- 172.21.15.92:0/3984777133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc70406d350 msgr2=0x7fc70406eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc702ffd700 1 --2- 172.21.15.92:0/3984777133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc70406d350 0x7fc70406eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc702ffd700 1 -- 172.21.15.92:0/3984777133 >> v2:172.21.15.124:3300/0 conn(0x7fc7040696e0 msgr2=0x7fc704069ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:35.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc702ffd700 1 --2- 172.21.15.92:0/3984777133 >> v2:172.21.15.124:3300/0 conn(0x7fc7040696e0 0x7fc704069ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc702ffd700 1 -- 172.21.15.92:0/3984777133 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70406dda0 con 0x7fc7040ada60 2023-10-06T10:43:35.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc702ffd700 1 -- 172.21.15.92:0/3984777133 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fc6f8001470 con 0x7fc7040ada60 2023-10-06T10:43:35.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc702ffd700 1 -- 172.21.15.92:0/3984777133 >> v1:172.21.15.92:6789/0 conn(0x7fc7040ada60 legacy=0x7fc7040aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:35.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc702ffd700 1 --2- 172.21.15.92:0/3984777133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6ec003a60 0x7fc6ec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:35.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc702ffd700 1 --2- 172.21.15.92:0/3984777133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6ec004810 0x7fc6ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:35.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc702ffd700 1 --2- 172.21.15.92:0/3984777133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc70406d350 0x7fc6ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:35.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc702ffd700 1 -- 172.21.15.92:0/3984777133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7040cd000 con 0x7fc6ec004810 2023-10-06T10:43:35.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc702ffd700 1 -- 172.21.15.92:0/3984777133 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70406dac0 con 0x7fc6ec003a60 2023-10-06T10:43:35.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc702ffd700 1 -- 172.21.15.92:0/3984777133 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec00ba60 con 0x7fc70406d350 2023-10-06T10:43:35.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc7037fe700 1 --2- 172.21.15.92:0/3984777133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6ec004810 0x7fc6ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:35.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc703fff700 1 --2- 172.21.15.92:0/3984777133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6ec003a60 0x7fc6ec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:35.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc7088fe700 1 --2- 172.21.15.92:0/3984777133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc70406d350 0x7fc6ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:35.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc7037fe700 1 -- 172.21.15.92:0/3984777133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc70406d350 msgr2=0x7fc6ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:35.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc7037fe700 1 --2- 172.21.15.92:0/3984777133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc70406d350 0x7fc6ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc7037fe700 1 -- 172.21.15.92:0/3984777133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6ec003a60 msgr2=0x7fc6ec003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:35.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc7037fe700 1 --2- 172.21.15.92:0/3984777133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6ec003a60 0x7fc6ec003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.720+0000 7fc7037fe700 1 -- 172.21.15.92:0/3984777133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ec003570 con 0x7fc6ec004810 2023-10-06T10:43:35.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.721+0000 7fc7037fe700 1 --2- 172.21.15.92:0/3984777133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6ec004810 0x7fc6ec004c80 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800c3f0 tx=0x7fc6f8001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16be40d86e2de110 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:35.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.721+0000 7fc702ffd700 1 -- 172.21.15.92:0/3984777133 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f800cc80 con 0x7fc6ec004810 2023-10-06T10:43:35.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.721+0000 7fc702ffd700 1 -- 172.21.15.92:0/3984777133 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f800ce00 con 0x7fc6ec004810 2023-10-06T10:43:35.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.721+0000 7fc702ffd700 1 -- 172.21.15.92:0/3984777133 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8005b70 con 0x7fc6ec004810 2023-10-06T10:43:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.721+0000 7fc70a361700 1 -- 172.21.15.92:0/3984777133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6ec004810 msgr2=0x7fc6ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.721+0000 7fc70a361700 1 --2- 172.21.15.92:0/3984777133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6ec004810 0x7fc6ec004c80 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800c3f0 tx=0x7fc6f8001e90 comp rx=0 tx=0).stop 2023-10-06T10:43:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.721+0000 7fc7088fe700 1 -- 172.21.15.92:0/3984777133 reap_dead start 2023-10-06T10:43:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.721+0000 7fc70a361700 1 -- 172.21.15.92:0/3984777133 shutdown_connections 2023-10-06T10:43:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.721+0000 7fc70a361700 1 -- 172.21.15.92:0/3984777133 >> 172.21.15.92:0/3984777133 conn(0x7fc704068080 msgr2=0x7fc7040cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.721+0000 7fc70a361700 1 -- 172.21.15.92:0/3984777133 shutdown_connections 2023-10-06T10:43:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.721+0000 7fc70a361700 1 -- 172.21.15.92:0/3984777133 wait complete. 2023-10-06T10:43:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.722+0000 7fc70a361700 1 Processor -- start 2023-10-06T10:43:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.723+0000 7fc70a361700 1 -- start start 2023-10-06T10:43:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.723+0000 7fc70a361700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6ec004810 0x7fc7040c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.723+0000 7fc70a361700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7040c5aa0 0x7fc7040c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.723+0000 7fc70a361700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7040c23d0 0x7fc704152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.723+0000 7fc70a361700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7040ac370 con 0x7fc7040c23d0 2023-10-06T10:43:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.723+0000 7fc70a361700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7040c2830 con 0x7fc6ec004810 2023-10-06T10:43:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.723+0000 7fc70a361700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7040c2b00 con 0x7fc7040c5aa0 2023-10-06T10:43:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.723+0000 7fc7088fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7040c23d0 0x7fc704152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.723+0000 7fc7088fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7040c23d0 0x7fc704152740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40762/0 (socket says 172.21.15.92:40762) 2023-10-06T10:43:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.723+0000 7fc7088fe700 1 -- 172.21.15.92:0/3078298596 learned_addr learned my addr 172.21.15.92:0/3078298596 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.723+0000 7fc7037fe700 1 --2- 172.21.15.92:0/3078298596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7040c5aa0 0x7fc7040c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.723+0000 7fc703fff700 1 --2- 172.21.15.92:0/3078298596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6ec004810 0x7fc7040c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.724+0000 7fc7088fe700 1 -- 172.21.15.92:0/3078298596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7040c5aa0 msgr2=0x7fc7040c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.724+0000 7fc7088fe700 1 --2- 172.21.15.92:0/3078298596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7040c5aa0 0x7fc7040c1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.724+0000 7fc7088fe700 1 -- 172.21.15.92:0/3078298596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6ec004810 msgr2=0x7fc7040c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.724+0000 7fc7088fe700 1 --2- 172.21.15.92:0/3078298596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6ec004810 0x7fc7040c5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.724+0000 7fc7088fe700 1 -- 172.21.15.92:0/3078298596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7040c2de0 con 0x7fc7040c23d0 2023-10-06T10:43:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.724+0000 7fc703fff700 1 --2- 172.21.15.92:0/3078298596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6ec004810 0x7fc7040c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.724+0000 7fc7088fe700 1 --2- 172.21.15.92:0/3078298596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7040c23d0 0x7fc704152740 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fc7040c9b60 tx=0x7fc6fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.724+0000 7fc6ebfff700 1 -- 172.21.15.92:0/3078298596 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc00e040 con 0x7fc7040c23d0 2023-10-06T10:43:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.724+0000 7fc6ebfff700 1 -- 172.21.15.92:0/3078298596 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6fc00f040 con 0x7fc7040c23d0 2023-10-06T10:43:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.724+0000 7fc70a361700 1 -- 172.21.15.92:0/3078298596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc704155130 con 0x7fc7040c23d0 2023-10-06T10:43:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.724+0000 7fc6ebfff700 1 -- 172.21.15.92:0/3078298596 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc00fe20 con 0x7fc7040c23d0 2023-10-06T10:43:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.724+0000 7fc70a361700 1 -- 172.21.15.92:0/3078298596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7041556f0 con 0x7fc7040c23d0 2023-10-06T10:43:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.725+0000 7fc70a361700 1 -- 172.21.15.92:0/3078298596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7041558e0 con 0x7fc7040c23d0 2023-10-06T10:43:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.725+0000 7fc6ebfff700 1 -- 172.21.15.92:0/3078298596 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc6fc015640 con 0x7fc7040c23d0 2023-10-06T10:43:35.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.726+0000 7fc6ebfff700 1 --2- 172.21.15.92:0/3078298596 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc6f0066f30 0x7fc6f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:35.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.726+0000 7fc6ebfff700 1 -- 172.21.15.92:0/3078298596 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc6fc0850f0 con 0x7fc7040c23d0 2023-10-06T10:43:35.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.726+0000 7fc703fff700 1 --2- 172.21.15.92:0/3078298596 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc6f0066f30 0x7fc6f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:35.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.726+0000 7fc703fff700 1 --2- 172.21.15.92:0/3078298596 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc6f0066f30 0x7fc6f00693e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc70415a100 tx=0x7fc6f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:35.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.730+0000 7fc6ebfff700 1 -- 172.21.15.92:0/3078298596 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc6fc0520c0 con 0x7fc7040c23d0 2023-10-06T10:43:35.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.912+0000 7fc70a361700 1 -- 172.21.15.92:0/3078298596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc704064a90 con 0x7fc7040c23d0 2023-10-06T10:43:35.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.913+0000 7fc6ebfff700 1 -- 172.21.15.92:0/3078298596 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+17074 (secure 0 0 0) 0x7fc6fc0556e0 con 0x7fc7040c23d0 2023-10-06T10:43:35.915 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:35.915 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":78,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:43:04.326791+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:43:00.816800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-06T10:43:02.128579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:43:35.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.915+0000 7fc70a361700 1 -- 172.21.15.92:0/3078298596 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc6f0066f30 msgr2=0x7fc6f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:35.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.915+0000 7fc70a361700 1 --2- 172.21.15.92:0/3078298596 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc6f0066f30 0x7fc6f00693e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc70415a100 tx=0x7fc6f4008040 comp rx=0 tx=0).stop 2023-10-06T10:43:35.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.915+0000 7fc70a361700 1 -- 172.21.15.92:0/3078298596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7040c23d0 msgr2=0x7fc704152740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:35.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.915+0000 7fc70a361700 1 --2- 172.21.15.92:0/3078298596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7040c23d0 0x7fc704152740 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fc7040c9b60 tx=0x7fc6fc00b040 comp rx=0 tx=0).stop 2023-10-06T10:43:35.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.915+0000 7fc70a361700 1 -- 172.21.15.92:0/3078298596 shutdown_connections 2023-10-06T10:43:35.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.915+0000 7fc70a361700 1 --2- 172.21.15.92:0/3078298596 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc6f0066f30 0x7fc6f00693e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.915+0000 7fc70a361700 1 --2- 172.21.15.92:0/3078298596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7040c23d0 0x7fc704152740 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.915+0000 7fc70a361700 1 --2- 172.21.15.92:0/3078298596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6ec004810 0x7fc7040c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.915+0000 7fc70a361700 1 --2- 172.21.15.92:0/3078298596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7040c5aa0 0x7fc7040c1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:35.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.915+0000 7fc70a361700 1 -- 172.21.15.92:0/3078298596 >> 172.21.15.92:0/3078298596 conn(0x7fc704068080 msgr2=0x7fc704153a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:35.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.915+0000 7fc70a361700 1 -- 172.21.15.92:0/3078298596 shutdown_connections 2023-10-06T10:43:35.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:35.915+0000 7fc70a361700 1 -- 172.21.15.92:0/3078298596 wait complete. 2023-10-06T10:43:35.930 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T10:43:36.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.044+0000 7fe797810700 1 Processor -- start 2023-10-06T10:43:36.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.044+0000 7fe797810700 1 -- start start 2023-10-06T10:43:36.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.045+0000 7fe797810700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe790159560 0x7fe7900aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:36.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.045+0000 7fe797810700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7900b49a0 0x7fe7900aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:36.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.045+0000 7fe797810700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7900b0c60 con 0x7fe7900b0760 2023-10-06T10:43:36.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.045+0000 7fe797810700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7900b0f40 con 0x7fe790159560 2023-10-06T10:43:36.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.045+0000 7fe797810700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7900b1220 con 0x7fe7900b49a0 2023-10-06T10:43:36.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.045+0000 7fe7955ac700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe790159560 0x7fe7900aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:36.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.045+0000 7fe795dad700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe7900b0760 0x7fe7900b3890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49138/0 (socket says 172.21.15.92:49138) 2023-10-06T10:43:36.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.045+0000 7fe795dad700 1 -- 172.21.15.92:0/376985629 learned_addr learned my addr 172.21.15.92:0/376985629 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:36.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.045+0000 7fe794dab700 1 --2- 172.21.15.92:0/376985629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7900b49a0 0x7fe7900aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:36.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.045+0000 7fe7955ac700 1 -- 172.21.15.92:0/376985629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7900b49a0 msgr2=0x7fe7900aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:36.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.045+0000 7fe7955ac700 1 --2- 172.21.15.92:0/376985629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7900b49a0 0x7fe7900aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:36.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.045+0000 7fe7955ac700 1 -- 172.21.15.92:0/376985629 >> v1:172.21.15.92:6789/0 conn(0x7fe7900b0760 legacy=0x7fe7900b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:36.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.045+0000 7fe78ffff700 1 -- 172.21.15.92:0/376985629 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996053618 0 0) 0x7fe7900b0c60 con 0x7fe7900b0760 2023-10-06T10:43:36.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.045+0000 7fe7955ac700 1 -- 172.21.15.92:0/376985629 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7900b1500 con 0x7fe790159560 2023-10-06T10:43:36.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.045+0000 7fe7955ac700 1 --2- 172.21.15.92:0/376985629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe790159560 0x7fe7900aa530 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fe7880098d0 tx=0x7fe788009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af19a334bbef89e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:36.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.045+0000 7fe78ffff700 1 -- 172.21.15.92:0/376985629 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788005c10 con 0x7fe790159560 2023-10-06T10:43:36.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.046+0000 7fe78ffff700 1 -- 172.21.15.92:0/376985629 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe788005d70 con 0x7fe790159560 2023-10-06T10:43:36.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.046+0000 7fe78ffff700 1 -- 172.21.15.92:0/376985629 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788006280 con 0x7fe790159560 2023-10-06T10:43:36.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.046+0000 7fe797810700 1 -- 172.21.15.92:0/376985629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe790159560 msgr2=0x7fe7900aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:36.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.046+0000 7fe797810700 1 --2- 172.21.15.92:0/376985629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe790159560 0x7fe7900aa530 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fe7880098d0 tx=0x7fe788009120 comp rx=0 tx=0).stop 2023-10-06T10:43:36.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.046+0000 7fe797810700 1 -- 172.21.15.92:0/376985629 shutdown_connections 2023-10-06T10:43:36.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.046+0000 7fe797810700 1 --2- 172.21.15.92:0/376985629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe790159560 0x7fe7900aa530 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:36.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.046+0000 7fe797810700 1 --2- 172.21.15.92:0/376985629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7900b49a0 0x7fe7900aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:36.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.046+0000 7fe797810700 1 -- 172.21.15.92:0/376985629 >> 172.21.15.92:0/376985629 conn(0x7fe7900997f0 msgr2=0x7fe79006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:36.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.046+0000 7fe797810700 1 -- 172.21.15.92:0/376985629 shutdown_connections 2023-10-06T10:43:36.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.046+0000 7fe797810700 1 -- 172.21.15.92:0/376985629 wait complete. 2023-10-06T10:43:36.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.047+0000 7fe797810700 1 Processor -- start 2023-10-06T10:43:36.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.047+0000 7fe797810700 1 -- start start 2023-10-06T10:43:36.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.048+0000 7fe797810700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7900b0760 0x7fe79016cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:36.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.048+0000 7fe797810700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7900b49a0 0x7fe790156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:36.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.048+0000 7fe797810700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe790159560 0x7fe790159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:36.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.048+0000 7fe797810700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79009a180 con 0x7fe7900b0760 2023-10-06T10:43:36.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.048+0000 7fe797810700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79016d330 con 0x7fe7900b49a0 2023-10-06T10:43:36.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.048+0000 7fe797810700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79016d610 con 0x7fe790159560 2023-10-06T10:43:36.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.048+0000 7fe7955ac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7900b0760 0x7fe79016cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:36.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.048+0000 7fe7955ac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7900b0760 0x7fe79016cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40780/0 (socket says 172.21.15.92:40780) 2023-10-06T10:43:36.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.048+0000 7fe7955ac700 1 -- 172.21.15.92:0/1901415638 learned_addr learned my addr 172.21.15.92:0/1901415638 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:36.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.048+0000 7fe7955ac700 1 -- 172.21.15.92:0/1901415638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe790159560 msgr2=0x7fe790159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:36.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.048+0000 7fe7955ac700 1 --2- 172.21.15.92:0/1901415638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe790159560 0x7fe790159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:36.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.048+0000 7fe7955ac700 1 -- 172.21.15.92:0/1901415638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7900b49a0 msgr2=0x7fe790156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:36.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.048+0000 7fe7955ac700 1 --2- 172.21.15.92:0/1901415638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7900b49a0 0x7fe790156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:36.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.048+0000 7fe7955ac700 1 -- 172.21.15.92:0/1901415638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7880093c0 con 0x7fe7900b0760 2023-10-06T10:43:36.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.048+0000 7fe7955ac700 1 --2- 172.21.15.92:0/1901415638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7900b0760 0x7fe79016cd50 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fe788009f70 tx=0x7fe788016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:36.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.049+0000 7fe78d7fa700 1 -- 172.21.15.92:0/1901415638 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78801d880 con 0x7fe7900b0760 2023-10-06T10:43:36.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.049+0000 7fe78d7fa700 1 -- 172.21.15.92:0/1901415638 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78801da00 con 0x7fe7900b0760 2023-10-06T10:43:36.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.049+0000 7fe78d7fa700 1 -- 172.21.15.92:0/1901415638 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78801ddc0 con 0x7fe7900b0760 2023-10-06T10:43:36.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.049+0000 7fe797810700 1 -- 172.21.15.92:0/1901415638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79016d8f0 con 0x7fe7900b0760 2023-10-06T10:43:36.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.049+0000 7fe797810700 1 -- 172.21.15.92:0/1901415638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79014a7a0 con 0x7fe7900b0760 2023-10-06T10:43:36.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.050+0000 7fe797810700 1 -- 172.21.15.92:0/1901415638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe79006a250 con 0x7fe7900b0760 2023-10-06T10:43:36.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.050+0000 7fe78d7fa700 1 -- 172.21.15.92:0/1901415638 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe78801b690 con 0x7fe7900b0760 2023-10-06T10:43:36.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.050+0000 7fe78d7fa700 1 --2- 172.21.15.92:0/1901415638 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe77c066f30 0x7fe77c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:36.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.050+0000 7fe78d7fa700 1 -- 172.21.15.92:0/1901415638 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe7880865a0 con 0x7fe7900b0760 2023-10-06T10:43:36.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.050+0000 7fe794dab700 1 --2- 172.21.15.92:0/1901415638 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe77c066f30 0x7fe77c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:36.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.051+0000 7fe794dab700 1 --2- 172.21.15.92:0/1901415638 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe77c066f30 0x7fe77c0693e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe79015c9a0 tx=0x7fe7840091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:36.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.055+0000 7fe78d7fa700 1 -- 172.21.15.92:0/1901415638 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe788053570 con 0x7fe7900b0760 2023-10-06T10:43:36.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.284+0000 7fe797810700 1 -- 172.21.15.92:0/1901415638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe790064a90 con 0x7fe7900b0760 2023-10-06T10:43:36.286 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T10:43:36.285+0000 7f9e7f3ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T10:43:36.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.289+0000 7fe78d7fa700 1 -- 172.21.15.92:0/1901415638 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v76) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe788056b90 con 0x7fe7900b0760 2023-10-06T10:43:36.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.292+0000 7fe797810700 1 -- 172.21.15.92:0/1901415638 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe77c066f30 msgr2=0x7fe77c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:36.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.292+0000 7fe797810700 1 --2- 172.21.15.92:0/1901415638 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe77c066f30 0x7fe77c0693e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe79015c9a0 tx=0x7fe7840091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:36.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.292+0000 7fe797810700 1 -- 172.21.15.92:0/1901415638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7900b0760 msgr2=0x7fe79016cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:36.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.292+0000 7fe797810700 1 --2- 172.21.15.92:0/1901415638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7900b0760 0x7fe79016cd50 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fe788009f70 tx=0x7fe788016000 comp rx=0 tx=0).stop 2023-10-06T10:43:36.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.292+0000 7fe797810700 1 -- 172.21.15.92:0/1901415638 shutdown_connections 2023-10-06T10:43:36.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.292+0000 7fe797810700 1 --2- 172.21.15.92:0/1901415638 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe77c066f30 0x7fe77c0693e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:36.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.292+0000 7fe797810700 1 --2- 172.21.15.92:0/1901415638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe790159560 0x7fe790159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:36.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.292+0000 7fe797810700 1 --2- 172.21.15.92:0/1901415638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7900b0760 0x7fe79016cd50 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:36.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.292+0000 7fe797810700 1 --2- 172.21.15.92:0/1901415638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7900b49a0 0x7fe790156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:36.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.292+0000 7fe797810700 1 -- 172.21.15.92:0/1901415638 >> 172.21.15.92:0/1901415638 conn(0x7fe7900997f0 msgr2=0x7fe7900d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:36.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.292+0000 7fe797810700 1 -- 172.21.15.92:0/1901415638 shutdown_connections 2023-10-06T10:43:36.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.292+0000 7fe797810700 1 -- 172.21.15.92:0/1901415638 wait complete. 2023-10-06T10:43:36.298 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T10:43:36.298 INFO:tasks.ceph.mds.d.smithi124.stderr: -7570> 2023-10-06T10:43:08.091+0000 7f984e348700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T10:43:36.298 INFO:tasks.ceph.mds.d.smithi124.stderr: -7144> 2023-10-06T10:43:13.091+0000 7f984e348700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-06T10:43:36.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T10:43:36.314 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T10:43:36.314 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T10:43:36.325 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-06T10:43:36.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.423+0000 7fed9d88e700 1 Processor -- start 2023-10-06T10:43:36.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.423+0000 7fed9d88e700 1 -- start start 2023-10-06T10:43:36.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.424+0000 7fed9d88e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fed98150940 0x7fed98159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:36.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.424+0000 7fed9d88e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed9815a160 0x7fed980ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:36.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.424+0000 7fed9d88e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fed980c5250 0x7fed980cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:36.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.424+0000 7fed9d88e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed9815ac30 con 0x7fed980c5250 2023-10-06T10:43:36.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.424+0000 7fed9d88e700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed980cb6a0 con 0x7fed98150940 2023-10-06T10:43:36.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.424+0000 7fed9d88e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed980cb090 con 0x7fed9815a160 2023-10-06T10:43:36.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.424+0000 7fed977fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fed980c5250 0x7fed980cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:36.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.424+0000 7fed977fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fed980c5250 0x7fed980cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40814/0 (socket says 172.21.15.92:40814) 2023-10-06T10:43:36.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.424+0000 7fed977fe700 1 -- 172.21.15.92:0/1210280533 learned_addr learned my addr 172.21.15.92:0/1210280533 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:36.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.424+0000 7fed96ffd700 1 --2- 172.21.15.92:0/1210280533 >> v2:172.21.15.124:3300/0 conn(0x7fed98150940 0x7fed98159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:36.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.424+0000 7fed967fc700 1 --2- 172.21.15.92:0/1210280533 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed9815a160 0x7fed980ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:36.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.424+0000 7fed977fe700 1 -- 172.21.15.92:0/1210280533 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed9815a160 msgr2=0x7fed980ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:36.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.424+0000 7fed977fe700 1 --2- 172.21.15.92:0/1210280533 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed9815a160 0x7fed980ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:36.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.424+0000 7fed977fe700 1 -- 172.21.15.92:0/1210280533 >> v2:172.21.15.124:3300/0 conn(0x7fed98150940 msgr2=0x7fed98159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:36.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.424+0000 7fed977fe700 1 --2- 172.21.15.92:0/1210280533 >> v2:172.21.15.124:3300/0 conn(0x7fed98150940 0x7fed98159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:36.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.424+0000 7fed977fe700 1 -- 172.21.15.92:0/1210280533 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9815a670 con 0x7fed980c5250 2023-10-06T10:43:36.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.424+0000 7fed96ffd700 1 --2- 172.21.15.92:0/1210280533 >> v2:172.21.15.124:3300/0 conn(0x7fed98150940 0x7fed98159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:36.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.424+0000 7fed977fe700 1 --2- 172.21.15.92:0/1210280533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed980c5250 0x7fed980cd300 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fed9808ab60 tx=0x7fed90009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6043c87e03321969 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:36.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.424+0000 7fed95ffb700 1 -- 172.21.15.92:0/1210280533 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9000e070 con 0x7fed980c5250 2023-10-06T10:43:36.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.424+0000 7fed95ffb700 1 -- 172.21.15.92:0/1210280533 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9000ba50 con 0x7fed980c5250 2023-10-06T10:43:36.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.424+0000 7fed95ffb700 1 -- 172.21.15.92:0/1210280533 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed900092e0 con 0x7fed980c5250 2023-10-06T10:43:36.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.425+0000 7fed9d88e700 1 -- 172.21.15.92:0/1210280533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed980c5250 msgr2=0x7fed980cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:36.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.425+0000 7fed9d88e700 1 --2- 172.21.15.92:0/1210280533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed980c5250 0x7fed980cd300 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fed9808ab60 tx=0x7fed90009840 comp rx=0 tx=0).stop 2023-10-06T10:43:36.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.425+0000 7fed9d88e700 1 -- 172.21.15.92:0/1210280533 shutdown_connections 2023-10-06T10:43:36.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.425+0000 7fed9d88e700 1 --2- 172.21.15.92:0/1210280533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed980c5250 0x7fed980cd300 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:36.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.425+0000 7fed9d88e700 1 --2- 172.21.15.92:0/1210280533 >> v2:172.21.15.124:3300/0 conn(0x7fed98150940 0x7fed98159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:36.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.425+0000 7fed9d88e700 1 --2- 172.21.15.92:0/1210280533 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed9815a160 0x7fed980ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:36.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.425+0000 7fed9d88e700 1 -- 172.21.15.92:0/1210280533 >> 172.21.15.92:0/1210280533 conn(0x7fed98099700 msgr2=0x7fed9806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:36.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.425+0000 7fed9d88e700 1 -- 172.21.15.92:0/1210280533 shutdown_connections 2023-10-06T10:43:36.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.425+0000 7fed9d88e700 1 -- 172.21.15.92:0/1210280533 wait complete. 2023-10-06T10:43:36.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.426+0000 7fed9d88e700 1 Processor -- start 2023-10-06T10:43:36.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.426+0000 7fed9d88e700 1 -- start start 2023-10-06T10:43:36.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.427+0000 7fed9d88e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed980c5250 0x7fed981697e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:36.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.427+0000 7fed9d88e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed9815a160 0x7fed9809bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:36.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.427+0000 7fed9d88e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed9809aee0 0x7fed9816c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:36.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.427+0000 7fed9d88e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9809c8d0 con 0x7fed9809aee0 2023-10-06T10:43:36.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.427+0000 7fed9d88e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9809c5f0 con 0x7fed980c5250 2023-10-06T10:43:36.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.427+0000 7fed9d88e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed980c7700 con 0x7fed9815a160 2023-10-06T10:43:36.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.427+0000 7fed977fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed9809aee0 0x7fed9816c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:36.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.427+0000 7fed977fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed9809aee0 0x7fed9816c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40820/0 (socket says 172.21.15.92:40820) 2023-10-06T10:43:36.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.427+0000 7fed977fe700 1 -- 172.21.15.92:0/3902032722 learned_addr learned my addr 172.21.15.92:0/3902032722 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:36.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.427+0000 7fed96ffd700 1 --2- 172.21.15.92:0/3902032722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed980c5250 0x7fed981697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:36.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.427+0000 7fed967fc700 1 --2- 172.21.15.92:0/3902032722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed9815a160 0x7fed9809bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:36.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.427+0000 7fed977fe700 1 -- 172.21.15.92:0/3902032722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed9815a160 msgr2=0x7fed9809bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:36.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.427+0000 7fed977fe700 1 --2- 172.21.15.92:0/3902032722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed9815a160 0x7fed9809bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:36.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.427+0000 7fed977fe700 1 -- 172.21.15.92:0/3902032722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed980c5250 msgr2=0x7fed981697e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:36.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.427+0000 7fed977fe700 1 --2- 172.21.15.92:0/3902032722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed980c5250 0x7fed981697e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:36.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.427+0000 7fed977fe700 1 -- 172.21.15.92:0/3902032722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed90009ae0 con 0x7fed9809aee0 2023-10-06T10:43:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.427+0000 7fed96ffd700 1 --2- 172.21.15.92:0/3902032722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed980c5250 0x7fed981697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.427+0000 7fed967fc700 1 --2- 172.21.15.92:0/3902032722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed9815a160 0x7fed9809bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.427+0000 7fed977fe700 1 --2- 172.21.15.92:0/3902032722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed9809aee0 0x7fed9816c560 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fed980d14c0 tx=0x7fed90018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.428+0000 7fed7effd700 1 -- 172.21.15.92:0/3902032722 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90008e80 con 0x7fed9809aee0 2023-10-06T10:43:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.428+0000 7fed7effd700 1 -- 172.21.15.92:0/3902032722 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed90026460 con 0x7fed9809aee0 2023-10-06T10:43:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.428+0000 7fed9d88e700 1 -- 172.21.15.92:0/3902032722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed980c79e0 con 0x7fed9809aee0 2023-10-06T10:43:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.428+0000 7fed7effd700 1 -- 172.21.15.92:0/3902032722 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90026800 con 0x7fed9809aee0 2023-10-06T10:43:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.428+0000 7fed9d88e700 1 -- 172.21.15.92:0/3902032722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9809c2e0 con 0x7fed9809aee0 2023-10-06T10:43:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.429+0000 7fed9d88e700 1 -- 172.21.15.92:0/3902032722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed980c7560 con 0x7fed9809aee0 2023-10-06T10:43:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.429+0000 7fed7effd700 1 -- 172.21.15.92:0/3902032722 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fed90003040 con 0x7fed9809aee0 2023-10-06T10:43:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.429+0000 7fed7effd700 1 --2- 172.21.15.92:0/3902032722 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fed84066f30 0x7fed840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.429+0000 7fed7effd700 1 -- 172.21.15.92:0/3902032722 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fed900869a0 con 0x7fed9809aee0 2023-10-06T10:43:36.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.429+0000 7fed96ffd700 1 --2- 172.21.15.92:0/3902032722 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fed84066f30 0x7fed840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:36.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.430+0000 7fed96ffd700 1 --2- 172.21.15.92:0/3902032722 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fed84066f30 0x7fed840693e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fed88008b00 tx=0x7fed88008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:36.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.433+0000 7fed7effd700 1 -- 172.21.15.92:0/3902032722 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed980c7560 con 0x7fed9809aee0 2023-10-06T10:43:36.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:36.659+0000 7fed9d88e700 1 -- 172.21.15.92:0/3902032722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fed980c7f00 con 0x7fed9809aee0 2023-10-06T10:43:37.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.289+0000 7fed7effd700 1 -- 172.21.15.92:0/3902032722 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v77) v1 ==== 104+0+0 (secure 0 0 0) 0x7fed980c7f00 con 0x7fed9809aee0 2023-10-06T10:43:37.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.292+0000 7fed9d88e700 1 -- 172.21.15.92:0/3902032722 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fed84066f30 msgr2=0x7fed840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:37.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.292+0000 7fed9d88e700 1 --2- 172.21.15.92:0/3902032722 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fed84066f30 0x7fed840693e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fed88008b00 tx=0x7fed88008040 comp rx=0 tx=0).stop 2023-10-06T10:43:37.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.292+0000 7fed9d88e700 1 -- 172.21.15.92:0/3902032722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed9809aee0 msgr2=0x7fed9816c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:37.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.292+0000 7fed9d88e700 1 --2- 172.21.15.92:0/3902032722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed9809aee0 0x7fed9816c560 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fed980d14c0 tx=0x7fed90018000 comp rx=0 tx=0).stop 2023-10-06T10:43:37.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.292+0000 7fed9d88e700 1 -- 172.21.15.92:0/3902032722 shutdown_connections 2023-10-06T10:43:37.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.292+0000 7fed9d88e700 1 --2- 172.21.15.92:0/3902032722 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fed84066f30 0x7fed840693e0 secure :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fed88008b00 tx=0x7fed88008040 comp rx=0 tx=0).stop 2023-10-06T10:43:37.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.292+0000 7fed9d88e700 1 --2- 172.21.15.92:0/3902032722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed9809aee0 0x7fed9816c560 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:37.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.292+0000 7fed9d88e700 1 --2- 172.21.15.92:0/3902032722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed980c5250 0x7fed981697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:37.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.292+0000 7fed9d88e700 1 --2- 172.21.15.92:0/3902032722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed9815a160 0x7fed9809bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:37.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.292+0000 7fed9d88e700 1 -- 172.21.15.92:0/3902032722 >> 172.21.15.92:0/3902032722 conn(0x7fed98099700 msgr2=0x7fed98069410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:37.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.293+0000 7fed9d88e700 1 -- 172.21.15.92:0/3902032722 shutdown_connections 2023-10-06T10:43:37.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.293+0000 7fed9d88e700 1 -- 172.21.15.92:0/3902032722 wait complete. 2023-10-06T10:43:37.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T10:43:37.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.466+0000 7f9b627aa700 1 Processor -- start 2023-10-06T10:43:37.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.466+0000 7f9b627aa700 1 -- start start 2023-10-06T10:43:37.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.467+0000 7f9b627aa700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9b5c15a740 0x7f9b5c0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:37.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.467+0000 7f9b627aa700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b5c0b2ce0 con 0x7f9b5c0b2360 2023-10-06T10:43:37.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.467+0000 7f9b627aa700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b5c0b0410 con 0x7f9b5c06ece0 2023-10-06T10:43:37.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.467+0000 7f9b627aa700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b5c06fb40 con 0x7f9b5c15a740 2023-10-06T10:43:37.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.467+0000 7f9b60d47700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9b5c15a740 0x7f9b5c0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:37.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.467+0000 7f9b5b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9b5c0b2360 0x7f9b5c0b08a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49146/0 (socket says 172.21.15.92:49146) 2023-10-06T10:43:37.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.467+0000 7f9b5b7fe700 1 -- 172.21.15.92:0/2356364951 learned_addr learned my addr 172.21.15.92:0/2356364951 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:37.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.467+0000 7f9b60d47700 1 -- 172.21.15.92:0/2356364951 >> v1:172.21.15.124:6789/0 conn(0x7f9b5c06ece0 legacy=0x7f9b5c06f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:37.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.467+0000 7f9b60d47700 1 -- 172.21.15.92:0/2356364951 >> v1:172.21.15.92:6789/0 conn(0x7f9b5c0b2360 legacy=0x7f9b5c0b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:37.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.467+0000 7f9b5affd700 1 -- 172.21.15.92:0/2356364951 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2464147229 0 0) 0x7f9b5c0b2ce0 con 0x7f9b5c0b2360 2023-10-06T10:43:37.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.467+0000 7f9b60d47700 1 -- 172.21.15.92:0/2356364951 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5c0b2a00 con 0x7f9b5c15a740 2023-10-06T10:43:37.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.467+0000 7f9b60d47700 1 --2- 172.21.15.92:0/2356364951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c15a740 0x7f9b5c0aede0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009e60 tx=0x7f9b54009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a5c728ead95d03 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:37.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.467+0000 7f9b5affd700 1 -- 172.21.15.92:0/2356364951 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5400e070 con 0x7f9b5c15a740 2023-10-06T10:43:37.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.467+0000 7f9b5affd700 1 -- 172.21.15.92:0/2356364951 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5400bc30 con 0x7f9b5c15a740 2023-10-06T10:43:37.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.467+0000 7f9b5affd700 1 -- 172.21.15.92:0/2356364951 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b54012430 con 0x7f9b5c15a740 2023-10-06T10:43:37.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.468+0000 7f9b627aa700 1 -- 172.21.15.92:0/2356364951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c15a740 msgr2=0x7f9b5c0aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:37.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.468+0000 7f9b627aa700 1 --2- 172.21.15.92:0/2356364951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c15a740 0x7f9b5c0aede0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009e60 tx=0x7f9b54009780 comp rx=0 tx=0).stop 2023-10-06T10:43:37.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.468+0000 7f9b627aa700 1 -- 172.21.15.92:0/2356364951 shutdown_connections 2023-10-06T10:43:37.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.468+0000 7f9b627aa700 1 --2- 172.21.15.92:0/2356364951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c15a740 0x7f9b5c0aede0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:37.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.468+0000 7f9b627aa700 1 -- 172.21.15.92:0/2356364951 >> 172.21.15.92:0/2356364951 conn(0x7f9b5c0682f0 msgr2=0x7f9b5c0686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:37.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.468+0000 7f9b627aa700 1 -- 172.21.15.92:0/2356364951 shutdown_connections 2023-10-06T10:43:37.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.468+0000 7f9b627aa700 1 -- 172.21.15.92:0/2356364951 wait complete. 2023-10-06T10:43:37.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.469+0000 7f9b627aa700 1 Processor -- start 2023-10-06T10:43:37.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.469+0000 7f9b627aa700 1 -- start start 2023-10-06T10:43:37.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.470+0000 7f9b627aa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c06ece0 0x7f9b5c14b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:37.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.470+0000 7f9b627aa700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b5c0b2360 0x7f9b5c0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:37.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.470+0000 7f9b627aa700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b5c15a740 0x7f9b5c0c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:37.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.470+0000 7f9b627aa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c0cf680 con 0x7f9b5c06ece0 2023-10-06T10:43:37.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.470+0000 7f9b627aa700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c0cea70 con 0x7f9b5c0b2360 2023-10-06T10:43:37.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.470+0000 7f9b627aa700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c0ced50 con 0x7f9b5c15a740 2023-10-06T10:43:37.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.470+0000 7f9b5bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c06ece0 0x7f9b5c14b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:37.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.470+0000 7f9b5bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c06ece0 0x7f9b5c14b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40834/0 (socket says 172.21.15.92:40834) 2023-10-06T10:43:37.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.470+0000 7f9b5bfff700 1 -- 172.21.15.92:0/1546620799 learned_addr learned my addr 172.21.15.92:0/1546620799 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:37.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.470+0000 7f9b5bfff700 1 -- 172.21.15.92:0/1546620799 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b5c15a740 msgr2=0x7f9b5c0c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:37.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.470+0000 7f9b60d47700 1 --2- 172.21.15.92:0/1546620799 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b5c15a740 0x7f9b5c0c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:37.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.470+0000 7f9b5b7fe700 1 --2- 172.21.15.92:0/1546620799 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b5c0b2360 0x7f9b5c0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:37.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.470+0000 7f9b5bfff700 1 --2- 172.21.15.92:0/1546620799 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b5c15a740 0x7f9b5c0c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:37.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.470+0000 7f9b5bfff700 1 -- 172.21.15.92:0/1546620799 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b5c0b2360 msgr2=0x7f9b5c0cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:37.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.470+0000 7f9b5bfff700 1 --2- 172.21.15.92:0/1546620799 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b5c0b2360 0x7f9b5c0cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:37.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.470+0000 7f9b5bfff700 1 -- 172.21.15.92:0/1546620799 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b54009a20 con 0x7f9b5c06ece0 2023-10-06T10:43:37.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.470+0000 7f9b5bfff700 1 --2- 172.21.15.92:0/1546620799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c06ece0 0x7f9b5c14b870 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00c4e0 tx=0x7f9b4c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:37.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.470+0000 7f9b43fff700 1 -- 172.21.15.92:0/1546620799 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c011590 con 0x7f9b5c06ece0 2023-10-06T10:43:37.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.471+0000 7f9b43fff700 1 -- 172.21.15.92:0/1546620799 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4c011710 con 0x7f9b5c06ece0 2023-10-06T10:43:37.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.471+0000 7f9b43fff700 1 -- 172.21.15.92:0/1546620799 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c011ad0 con 0x7f9b5c06ece0 2023-10-06T10:43:37.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.471+0000 7f9b627aa700 1 -- 172.21.15.92:0/1546620799 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b5c0cf030 con 0x7f9b5c06ece0 2023-10-06T10:43:37.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.471+0000 7f9b627aa700 1 -- 172.21.15.92:0/1546620799 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5c14bdb0 con 0x7f9b5c06ece0 2023-10-06T10:43:37.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.472+0000 7f9b627aa700 1 -- 172.21.15.92:0/1546620799 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5c064a90 con 0x7f9b5c06ece0 2023-10-06T10:43:37.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.472+0000 7f9b43fff700 1 -- 172.21.15.92:0/1546620799 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9b4c01b3b0 con 0x7f9b5c06ece0 2023-10-06T10:43:37.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.472+0000 7f9b43fff700 1 --2- 172.21.15.92:0/1546620799 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9b48066f30 0x7f9b480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:37.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.472+0000 7f9b43fff700 1 -- 172.21.15.92:0/1546620799 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f9b4c083e00 con 0x7f9b5c06ece0 2023-10-06T10:43:37.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.472+0000 7f9b5b7fe700 1 --2- 172.21.15.92:0/1546620799 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9b48066f30 0x7f9b480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:37.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.473+0000 7f9b5b7fe700 1 --2- 172.21.15.92:0/1546620799 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9b48066f30 0x7f9b480693e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9b50007fd0 tx=0x7f9b50009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:37.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.476+0000 7f9b43fff700 1 -- 172.21.15.92:0/1546620799 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9b4c050cd0 con 0x7f9b5c06ece0 2023-10-06T10:43:37.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:37.659+0000 7f9b627aa700 1 -- 172.21.15.92:0/1546620799 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9b5c06f0b0 con 0x7f9b5c06ece0 2023-10-06T10:43:38.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.419+0000 7f9b43fff700 1 -- 172.21.15.92:0/1546620799 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v80) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9b4c017070 con 0x7f9b5c06ece0 2023-10-06T10:43:38.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.422+0000 7f9b627aa700 1 -- 172.21.15.92:0/1546620799 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9b48066f30 msgr2=0x7f9b480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:38.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.422+0000 7f9b627aa700 1 --2- 172.21.15.92:0/1546620799 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9b48066f30 0x7f9b480693e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9b50007fd0 tx=0x7f9b50009040 comp rx=0 tx=0).stop 2023-10-06T10:43:38.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.422+0000 7f9b627aa700 1 -- 172.21.15.92:0/1546620799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c06ece0 msgr2=0x7f9b5c14b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:38.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.422+0000 7f9b627aa700 1 --2- 172.21.15.92:0/1546620799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c06ece0 0x7f9b5c14b870 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00c4e0 tx=0x7f9b4c00c8a0 comp rx=0 tx=0).stop 2023-10-06T10:43:38.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.423+0000 7f9b627aa700 1 -- 172.21.15.92:0/1546620799 shutdown_connections 2023-10-06T10:43:38.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.423+0000 7f9b627aa700 1 --2- 172.21.15.92:0/1546620799 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9b48066f30 0x7f9b480693e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:38.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.423+0000 7f9b627aa700 1 --2- 172.21.15.92:0/1546620799 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b5c15a740 0x7f9b5c0c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:38.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.423+0000 7f9b627aa700 1 --2- 172.21.15.92:0/1546620799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c06ece0 0x7f9b5c14b870 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:38.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.423+0000 7f9b627aa700 1 --2- 172.21.15.92:0/1546620799 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b5c0b2360 0x7f9b5c0cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:38.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.423+0000 7f9b627aa700 1 -- 172.21.15.92:0/1546620799 >> 172.21.15.92:0/1546620799 conn(0x7f9b5c0682f0 msgr2=0x7f9b5c156940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:38.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.423+0000 7f9b627aa700 1 -- 172.21.15.92:0/1546620799 shutdown_connections 2023-10-06T10:43:38.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.423+0000 7f9b627aa700 1 -- 172.21.15.92:0/1546620799 wait complete. 2023-10-06T10:43:38.426 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-10-06T10:43:38.437 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T10:43:38.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.560+0000 7f24c684a700 1 Processor -- start 2023-10-06T10:43:38.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.560+0000 7f24c684a700 1 -- start start 2023-10-06T10:43:38.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.561+0000 7f24c684a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f24c00b2360 0x7f24c00b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:38.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.561+0000 7f24c684a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c00b2ce0 con 0x7f24c006ece0 2023-10-06T10:43:38.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.561+0000 7f24c684a700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c00b0410 con 0x7f24c00aede0 2023-10-06T10:43:38.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.561+0000 7f24c684a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c006fb40 con 0x7f24c00b2360 2023-10-06T10:43:38.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.561+0000 7f24bf7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f24c00b2360 0x7f24c00b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:38.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.561+0000 7f24bf7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f24c00b2360 0x7f24c00b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40840/0 (socket says 172.21.15.92:40840) 2023-10-06T10:43:38.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.561+0000 7f24bffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f24c006ece0 0x7f24c006f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49152/0 (socket says 172.21.15.92:49152) 2023-10-06T10:43:38.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.561+0000 7f24bf7fe700 1 -- 172.21.15.92:0/1633072675 learned_addr learned my addr 172.21.15.92:0/1633072675 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:38.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.562+0000 7f24bf7fe700 1 -- 172.21.15.92:0/1633072675 >> v1:172.21.15.124:6789/0 conn(0x7f24c00aede0 legacy=0x7f24c00af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:38.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.562+0000 7f24bf7fe700 1 -- 172.21.15.92:0/1633072675 >> v1:172.21.15.92:6789/0 conn(0x7f24c006ece0 legacy=0x7f24c006f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:38.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.562+0000 7f24bf7fe700 1 -- 172.21.15.92:0/1633072675 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c00b2a00 con 0x7f24c00b2360 2023-10-06T10:43:38.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.562+0000 7f24beffd700 1 -- 172.21.15.92:0/1633072675 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 339105973 0 0) 0x7f24c00b2ce0 con 0x7f24c006ece0 2023-10-06T10:43:38.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.562+0000 7f24bf7fe700 1 --2- 172.21.15.92:0/1633072675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c00b2360 0x7f24c00b08a0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f24b40099b0 tx=0x7f24b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8675fc723e4ba46b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:38.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.562+0000 7f24beffd700 1 -- 172.21.15.92:0/1633072675 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b400b860 con 0x7f24c00b2360 2023-10-06T10:43:38.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.562+0000 7f24beffd700 1 -- 172.21.15.92:0/1633072675 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b400b9c0 con 0x7f24c00b2360 2023-10-06T10:43:38.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.562+0000 7f24beffd700 1 -- 172.21.15.92:0/1633072675 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b400bd60 con 0x7f24c00b2360 2023-10-06T10:43:38.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.562+0000 7f24c684a700 1 -- 172.21.15.92:0/1633072675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c00b2360 msgr2=0x7f24c00b08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:38.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.562+0000 7f24c684a700 1 --2- 172.21.15.92:0/1633072675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c00b2360 0x7f24c00b08a0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f24b40099b0 tx=0x7f24b4009250 comp rx=0 tx=0).stop 2023-10-06T10:43:38.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.563+0000 7f24c684a700 1 -- 172.21.15.92:0/1633072675 shutdown_connections 2023-10-06T10:43:38.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.563+0000 7f24c684a700 1 --2- 172.21.15.92:0/1633072675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c00b2360 0x7f24c00b08a0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:38.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.563+0000 7f24c684a700 1 -- 172.21.15.92:0/1633072675 >> 172.21.15.92:0/1633072675 conn(0x7f24c00682f0 msgr2=0x7f24c00686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:38.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.563+0000 7f24c684a700 1 -- 172.21.15.92:0/1633072675 shutdown_connections 2023-10-06T10:43:38.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.563+0000 7f24c684a700 1 -- 172.21.15.92:0/1633072675 wait complete. 2023-10-06T10:43:38.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.564+0000 7f24c684a700 1 Processor -- start 2023-10-06T10:43:38.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.564+0000 7f24c684a700 1 -- start start 2023-10-06T10:43:38.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.565+0000 7f24c684a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24c006ece0 0x7f24c00c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:38.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.565+0000 7f24c684a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24c00aede0 0x7f24c0158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:38.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.565+0000 7f24c684a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c00b2360 0x7f24c00cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:38.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.565+0000 7f24c684a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c0159ee0 con 0x7f24c00b2360 2023-10-06T10:43:38.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.565+0000 7f24c684a700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c01592d0 con 0x7f24c006ece0 2023-10-06T10:43:38.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.565+0000 7f24c684a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c01595b0 con 0x7f24c00aede0 2023-10-06T10:43:38.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.565+0000 7f24c4de7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c00b2360 0x7f24c00cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:38.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.565+0000 7f24c4de7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c00b2360 0x7f24c00cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40850/0 (socket says 172.21.15.92:40850) 2023-10-06T10:43:38.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.565+0000 7f24c4de7700 1 -- 172.21.15.92:0/1038983095 learned_addr learned my addr 172.21.15.92:0/1038983095 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:38.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.565+0000 7f24bffff700 1 --2- 172.21.15.92:0/1038983095 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24c006ece0 0x7f24c00c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:38.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.565+0000 7f24bf7fe700 1 --2- 172.21.15.92:0/1038983095 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24c00aede0 0x7f24c0158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:38.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.565+0000 7f24c4de7700 1 -- 172.21.15.92:0/1038983095 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24c00aede0 msgr2=0x7f24c0158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:38.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.565+0000 7f24c4de7700 1 --2- 172.21.15.92:0/1038983095 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24c00aede0 0x7f24c0158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:38.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.565+0000 7f24c4de7700 1 -- 172.21.15.92:0/1038983095 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24c006ece0 msgr2=0x7f24c00c68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:38.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.565+0000 7f24c4de7700 1 --2- 172.21.15.92:0/1038983095 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24c006ece0 0x7f24c00c68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:38.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.565+0000 7f24c4de7700 1 -- 172.21.15.92:0/1038983095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b40094f0 con 0x7f24c00b2360 2023-10-06T10:43:38.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.565+0000 7f24c4de7700 1 --2- 172.21.15.92:0/1038983095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c00b2360 0x7f24c00cc450 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f24b800c840 tx=0x7f24b8001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:38.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.565+0000 7f24a7fff700 1 -- 172.21.15.92:0/1038983095 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8012630 con 0x7f24c00b2360 2023-10-06T10:43:38.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.565+0000 7f24a7fff700 1 -- 172.21.15.92:0/1038983095 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b80127b0 con 0x7f24c00b2360 2023-10-06T10:43:38.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.565+0000 7f24c684a700 1 -- 172.21.15.92:0/1038983095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c0159890 con 0x7f24c00b2360 2023-10-06T10:43:38.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.565+0000 7f24a7fff700 1 -- 172.21.15.92:0/1038983095 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8012b70 con 0x7f24c00b2360 2023-10-06T10:43:38.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.565+0000 7f24c684a700 1 -- 172.21.15.92:0/1038983095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c00c6e10 con 0x7f24c00b2360 2023-10-06T10:43:38.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.566+0000 7f24c684a700 1 -- 172.21.15.92:0/1038983095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24c015a050 con 0x7f24c00b2360 2023-10-06T10:43:38.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.567+0000 7f24a7fff700 1 -- 172.21.15.92:0/1038983095 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f24b8003030 con 0x7f24c00b2360 2023-10-06T10:43:38.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.567+0000 7f24a7fff700 1 --2- 172.21.15.92:0/1038983095 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f24ac066f30 0x7f24ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:38.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.567+0000 7f24a7fff700 1 -- 172.21.15.92:0/1038983095 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f24b80846e0 con 0x7f24c00b2360 2023-10-06T10:43:38.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.567+0000 7f24bffff700 1 --2- 172.21.15.92:0/1038983095 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f24ac066f30 0x7f24ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:38.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.567+0000 7f24bffff700 1 --2- 172.21.15.92:0/1038983095 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f24ac066f30 0x7f24ac0693e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f24b0007fd0 tx=0x7f24b0007db0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:38.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.571+0000 7f24a7fff700 1 -- 172.21.15.92:0/1038983095 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24b8051760 con 0x7f24c00b2360 2023-10-06T10:43:38.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:38.754+0000 7f24c684a700 1 -- 172.21.15.92:0/1038983095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f24c0064a90 con 0x7f24c00b2360 2023-10-06T10:43:39.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.424+0000 7f24a7fff700 1 -- 172.21.15.92:0/1038983095 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v81) v1 ==== 169+0+0 (secure 0 0 0) 0x7f24b8054d80 con 0x7f24c00b2360 2023-10-06T10:43:39.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.427+0000 7f24c684a700 1 -- 172.21.15.92:0/1038983095 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f24ac066f30 msgr2=0x7f24ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:39.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.427+0000 7f24c684a700 1 --2- 172.21.15.92:0/1038983095 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f24ac066f30 0x7f24ac0693e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f24b0007fd0 tx=0x7f24b0007db0 comp rx=0 tx=0).stop 2023-10-06T10:43:39.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.427+0000 7f24c684a700 1 -- 172.21.15.92:0/1038983095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c00b2360 msgr2=0x7f24c00cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:39.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.427+0000 7f24c684a700 1 --2- 172.21.15.92:0/1038983095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c00b2360 0x7f24c00cc450 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f24b800c840 tx=0x7f24b8001670 comp rx=0 tx=0).stop 2023-10-06T10:43:39.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.428+0000 7f24c684a700 1 -- 172.21.15.92:0/1038983095 shutdown_connections 2023-10-06T10:43:39.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.428+0000 7f24c684a700 1 --2- 172.21.15.92:0/1038983095 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f24ac066f30 0x7f24ac0693e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:39.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.428+0000 7f24c684a700 1 --2- 172.21.15.92:0/1038983095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c00b2360 0x7f24c00cc450 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:39.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.428+0000 7f24c684a700 1 --2- 172.21.15.92:0/1038983095 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24c006ece0 0x7f24c00c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:39.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.428+0000 7f24c684a700 1 --2- 172.21.15.92:0/1038983095 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24c00aede0 0x7f24c0158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:39.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.428+0000 7f24c684a700 1 -- 172.21.15.92:0/1038983095 >> 172.21.15.92:0/1038983095 conn(0x7f24c00682f0 msgr2=0x7f24c00b4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:39.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.428+0000 7f24c684a700 1 -- 172.21.15.92:0/1038983095 shutdown_connections 2023-10-06T10:43:39.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.430+0000 7f24c684a700 1 -- 172.21.15.92:0/1038983095 wait complete. 2023-10-06T10:43:39.432 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-10-06T10:43:39.446 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T10:43:39.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.564+0000 7fda2d02a700 1 Processor -- start 2023-10-06T10:43:39.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.564+0000 7fda2d02a700 1 -- start start 2023-10-06T10:43:39.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.565+0000 7fda2d02a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda281595b0 0x7fda280af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:39.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.565+0000 7fda2d02a700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fda280c6f60 0x7fda280c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:39.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.565+0000 7fda2d02a700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda280c7910 con 0x7fda28068470 2023-10-06T10:43:39.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.565+0000 7fda2d02a700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda280678f0 con 0x7fda280c6f60 2023-10-06T10:43:39.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.565+0000 7fda2d02a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda28069d10 con 0x7fda281595b0 2023-10-06T10:43:39.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.565+0000 7fda26d9d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda281595b0 0x7fda280af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:39.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.565+0000 7fda2659c700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fda280c6f60 0x7fda280c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:39.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.565+0000 7fda2759e700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fda28068470 0x7fda280688e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:41952/0 (socket says 172.21.15.92:41952) 2023-10-06T10:43:39.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.565+0000 7fda2759e700 1 -- 172.21.15.92:0/2537872478 learned_addr learned my addr 172.21.15.92:0/2537872478 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:39.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.566+0000 7fda26d9d700 1 -- 172.21.15.92:0/2537872478 >> v2:172.21.15.124:3300/0 conn(0x7fda280c6f60 msgr2=0x7fda280c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:39.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.566+0000 7fda26d9d700 1 --2- 172.21.15.92:0/2537872478 >> v2:172.21.15.124:3300/0 conn(0x7fda280c6f60 0x7fda280c73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:39.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.566+0000 7fda26d9d700 1 -- 172.21.15.92:0/2537872478 >> v1:172.21.15.124:6789/0 conn(0x7fda28068470 legacy=0x7fda280688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:39.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.566+0000 7fda26d9d700 1 -- 172.21.15.92:0/2537872478 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2806a890 con 0x7fda281595b0 2023-10-06T10:43:39.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.566+0000 7fda2659c700 1 --2- 172.21.15.92:0/2537872478 >> v2:172.21.15.124:3300/0 conn(0x7fda280c6f60 0x7fda280c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:43:39.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.566+0000 7fda26d9d700 1 --2- 172.21.15.92:0/2537872478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda281595b0 0x7fda280af7b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0098e0 tx=0x7fda1c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3193be62ef5cdfa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:39.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.566+0000 7fda25d9b700 1 -- 172.21.15.92:0/2537872478 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c00b640 con 0x7fda281595b0 2023-10-06T10:43:39.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.566+0000 7fda25d9b700 1 -- 172.21.15.92:0/2537872478 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1c00b7a0 con 0x7fda281595b0 2023-10-06T10:43:39.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.566+0000 7fda25d9b700 1 -- 172.21.15.92:0/2537872478 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c00bb40 con 0x7fda281595b0 2023-10-06T10:43:39.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.567+0000 7fda2d02a700 1 -- 172.21.15.92:0/2537872478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda281595b0 msgr2=0x7fda280af7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:39.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.567+0000 7fda2d02a700 1 --2- 172.21.15.92:0/2537872478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda281595b0 0x7fda280af7b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0098e0 tx=0x7fda1c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:39.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.567+0000 7fda2d02a700 1 -- 172.21.15.92:0/2537872478 shutdown_connections 2023-10-06T10:43:39.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.567+0000 7fda2d02a700 1 --2- 172.21.15.92:0/2537872478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda281595b0 0x7fda280af7b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:39.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.567+0000 7fda2d02a700 1 --2- 172.21.15.92:0/2537872478 >> v2:172.21.15.124:3300/0 conn(0x7fda280c6f60 0x7fda280c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:39.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.567+0000 7fda2d02a700 1 -- 172.21.15.92:0/2537872478 >> 172.21.15.92:0/2537872478 conn(0x7fda280cd3f0 msgr2=0x7fda280c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:39.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.567+0000 7fda2d02a700 1 -- 172.21.15.92:0/2537872478 shutdown_connections 2023-10-06T10:43:39.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.567+0000 7fda2d02a700 1 -- 172.21.15.92:0/2537872478 wait complete. 2023-10-06T10:43:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.568+0000 7fda2d02a700 1 Processor -- start 2023-10-06T10:43:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.568+0000 7fda2d02a700 1 -- start start 2023-10-06T10:43:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.569+0000 7fda2d02a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda28068470 0x7fda28159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.569+0000 7fda2d02a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda280c6f60 0x7fda281580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.569+0000 7fda2d02a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda281595b0 0x7fda280a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.569+0000 7fda2d02a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda281585f0 con 0x7fda280c6f60 2023-10-06T10:43:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.569+0000 7fda2d02a700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda280ad5f0 con 0x7fda28068470 2023-10-06T10:43:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.569+0000 7fda2d02a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda280aa290 con 0x7fda281595b0 2023-10-06T10:43:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.569+0000 7fda2659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda280c6f60 0x7fda281580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.569+0000 7fda2659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda280c6f60 0x7fda281580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40862/0 (socket says 172.21.15.92:40862) 2023-10-06T10:43:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.569+0000 7fda26d9d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda28068470 0x7fda28159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.569+0000 7fda2659c700 1 -- 172.21.15.92:0/2924515205 learned_addr learned my addr 172.21.15.92:0/2924515205 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.569+0000 7fda2759e700 1 --2- 172.21.15.92:0/2924515205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda281595b0 0x7fda280a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.569+0000 7fda2659c700 1 -- 172.21.15.92:0/2924515205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda281595b0 msgr2=0x7fda280a98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.569+0000 7fda2659c700 1 --2- 172.21.15.92:0/2924515205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda281595b0 0x7fda280a98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.569+0000 7fda2659c700 1 -- 172.21.15.92:0/2924515205 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda28068470 msgr2=0x7fda28159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.569+0000 7fda2659c700 1 --2- 172.21.15.92:0/2924515205 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda28068470 0x7fda28159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.569+0000 7fda2659c700 1 -- 172.21.15.92:0/2924515205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda18009420 con 0x7fda280c6f60 2023-10-06T10:43:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.569+0000 7fda2659c700 1 --2- 172.21.15.92:0/2924515205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda280c6f60 0x7fda281580b0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fda1800ec50 tx=0x7fda180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.569+0000 7fda0effd700 1 -- 172.21.15.92:0/2924515205 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1800c850 con 0x7fda280c6f60 2023-10-06T10:43:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.569+0000 7fda0effd700 1 -- 172.21.15.92:0/2924515205 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1800c9b0 con 0x7fda280c6f60 2023-10-06T10:43:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.569+0000 7fda2d02a700 1 -- 172.21.15.92:0/2924515205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda1c009420 con 0x7fda280c6f60 2023-10-06T10:43:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.569+0000 7fda0effd700 1 -- 172.21.15.92:0/2924515205 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1800cd50 con 0x7fda280c6f60 2023-10-06T10:43:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.569+0000 7fda2d02a700 1 -- 172.21.15.92:0/2924515205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda280aa850 con 0x7fda280c6f60 2023-10-06T10:43:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.570+0000 7fda2d02a700 1 -- 172.21.15.92:0/2924515205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda280ad780 con 0x7fda280c6f60 2023-10-06T10:43:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.571+0000 7fda0effd700 1 -- 172.21.15.92:0/2924515205 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fda18013680 con 0x7fda280c6f60 2023-10-06T10:43:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.571+0000 7fda0effd700 1 --2- 172.21.15.92:0/2924515205 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda14066f30 0x7fda140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.571+0000 7fda0effd700 1 -- 172.21.15.92:0/2924515205 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5784+0+0 (secure 0 0 0) 0x7fda18084860 con 0x7fda280c6f60 2023-10-06T10:43:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.571+0000 7fda26d9d700 1 --2- 172.21.15.92:0/2924515205 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda14066f30 0x7fda140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.571+0000 7fda26d9d700 1 --2- 172.21.15.92:0/2924515205 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda14066f30 0x7fda140693e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0093f0 tx=0x7fda1c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:39.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.575+0000 7fda0effd700 1 -- 172.21.15.92:0/2924515205 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda18051a80 con 0x7fda280c6f60 2023-10-06T10:43:39.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.757+0000 7fda2d02a700 1 -- 172.21.15.92:0/2924515205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fda28064a90 con 0x7fda280c6f60 2023-10-06T10:43:39.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.760+0000 7fda0effd700 1 -- 172.21.15.92:0/2924515205 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fda180550a0 con 0x7fda280c6f60 2023-10-06T10:43:39.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.762+0000 7fda2d02a700 1 -- 172.21.15.92:0/2924515205 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda14066f30 msgr2=0x7fda140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:39.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.762+0000 7fda2d02a700 1 --2- 172.21.15.92:0/2924515205 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda14066f30 0x7fda140693e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0093f0 tx=0x7fda1c017000 comp rx=0 tx=0).stop 2023-10-06T10:43:39.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.762+0000 7fda2d02a700 1 -- 172.21.15.92:0/2924515205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda280c6f60 msgr2=0x7fda281580b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:39.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.762+0000 7fda2d02a700 1 --2- 172.21.15.92:0/2924515205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda280c6f60 0x7fda281580b0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fda1800ec50 tx=0x7fda180091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:39.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.762+0000 7fda2d02a700 1 -- 172.21.15.92:0/2924515205 shutdown_connections 2023-10-06T10:43:39.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.762+0000 7fda2d02a700 1 --2- 172.21.15.92:0/2924515205 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fda14066f30 0x7fda140693e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:39.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.762+0000 7fda2d02a700 1 --2- 172.21.15.92:0/2924515205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda281595b0 0x7fda280a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:39.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.762+0000 7fda2d02a700 1 --2- 172.21.15.92:0/2924515205 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda28068470 0x7fda28159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:39.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.762+0000 7fda2d02a700 1 --2- 172.21.15.92:0/2924515205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda280c6f60 0x7fda281580b0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:39.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.762+0000 7fda2d02a700 1 -- 172.21.15.92:0/2924515205 >> 172.21.15.92:0/2924515205 conn(0x7fda280cd3f0 msgr2=0x7fda280cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:39.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.762+0000 7fda2d02a700 1 -- 172.21.15.92:0/2924515205 shutdown_connections 2023-10-06T10:43:39.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.762+0000 7fda2d02a700 1 -- 172.21.15.92:0/2924515205 wait complete. 2023-10-06T10:43:39.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0' 2023-10-06T10:43:39.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.888+0000 7fd2134f5700 1 Processor -- start 2023-10-06T10:43:39.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.888+0000 7fd2134f5700 1 -- start start 2023-10-06T10:43:39.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.889+0000 7fd2134f5700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd20c159560 0x7fd20c0aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:39.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.889+0000 7fd2134f5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd20c0b0760 0x7fd20c0b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:39.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.889+0000 7fd2134f5700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd20c0b0c60 con 0x7fd20c0b49a0 2023-10-06T10:43:39.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.889+0000 7fd2134f5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd20c0b0f40 con 0x7fd20c0b0760 2023-10-06T10:43:39.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.889+0000 7fd2134f5700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd20c0b1220 con 0x7fd20c159560 2023-10-06T10:43:39.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.889+0000 7fd211a92700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd20c0b0760 0x7fd20c0b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:39.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.889+0000 7fd211a92700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd20c0b0760 0x7fd20c0b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40864/0 (socket says 172.21.15.92:40864) 2023-10-06T10:43:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.889+0000 7fd211a92700 1 -- 172.21.15.92:0/1187263139 learned_addr learned my addr 172.21.15.92:0/1187263139 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.889+0000 7fd211291700 1 --2- 172.21.15.92:0/1187263139 >> v2:172.21.15.124:3300/0 conn(0x7fd20c159560 0x7fd20c0aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.889+0000 7fd211a92700 1 -- 172.21.15.92:0/1187263139 >> v2:172.21.15.124:3300/0 conn(0x7fd20c159560 msgr2=0x7fd20c0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.889+0000 7fd211a92700 1 --2- 172.21.15.92:0/1187263139 >> v2:172.21.15.124:3300/0 conn(0x7fd20c159560 0x7fd20c0aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.889+0000 7fd211a92700 1 -- 172.21.15.92:0/1187263139 >> v1:172.21.15.124:6789/0 conn(0x7fd20c0b49a0 legacy=0x7fd20c0aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.889+0000 7fd211a92700 1 -- 172.21.15.92:0/1187263139 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd20c0b1500 con 0x7fd20c0b0760 2023-10-06T10:43:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.889+0000 7fd211a92700 1 --2- 172.21.15.92:0/1187263139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd20c0b0760 0x7fd20c0b1bc0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fd204009f20 tx=0x7fd204009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a3af28f7f4f387 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.889+0000 7fd203fff700 1 -- 172.21.15.92:0/1187263139 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20400e070 con 0x7fd20c0b0760 2023-10-06T10:43:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.889+0000 7fd203fff700 1 -- 172.21.15.92:0/1187263139 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20400ba00 con 0x7fd20c0b0760 2023-10-06T10:43:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.889+0000 7fd203fff700 1 -- 172.21.15.92:0/1187263139 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2040092e0 con 0x7fd20c0b0760 2023-10-06T10:43:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.890+0000 7fd2134f5700 1 -- 172.21.15.92:0/1187263139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd20c0b0760 msgr2=0x7fd20c0b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.890+0000 7fd2134f5700 1 --2- 172.21.15.92:0/1187263139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd20c0b0760 0x7fd20c0b1bc0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fd204009f20 tx=0x7fd204009840 comp rx=0 tx=0).stop 2023-10-06T10:43:39.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.890+0000 7fd2134f5700 1 -- 172.21.15.92:0/1187263139 shutdown_connections 2023-10-06T10:43:39.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.890+0000 7fd2134f5700 1 --2- 172.21.15.92:0/1187263139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd20c0b0760 0x7fd20c0b1bc0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:39.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.890+0000 7fd2134f5700 1 --2- 172.21.15.92:0/1187263139 >> v2:172.21.15.124:3300/0 conn(0x7fd20c159560 0x7fd20c0aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:39.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.890+0000 7fd2134f5700 1 -- 172.21.15.92:0/1187263139 >> 172.21.15.92:0/1187263139 conn(0x7fd20c0997f0 msgr2=0x7fd20c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:39.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.890+0000 7fd2134f5700 1 -- 172.21.15.92:0/1187263139 shutdown_connections 2023-10-06T10:43:39.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.890+0000 7fd2134f5700 1 -- 172.21.15.92:0/1187263139 wait complete. 2023-10-06T10:43:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.891+0000 7fd2134f5700 1 Processor -- start 2023-10-06T10:43:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.891+0000 7fd2134f5700 1 -- start start 2023-10-06T10:43:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.892+0000 7fd2134f5700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd20c0b0760 0x7fd20c169490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.892+0000 7fd2134f5700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd20c0b49a0 0x7fd20c0ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.892+0000 7fd2134f5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd20c159560 0x7fd20c0aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.892+0000 7fd2134f5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20c0af050 con 0x7fd20c159560 2023-10-06T10:43:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.892+0000 7fd2134f5700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20c0ada70 con 0x7fd20c0b0760 2023-10-06T10:43:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.892+0000 7fd2134f5700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd20c0add50 con 0x7fd20c0b49a0 2023-10-06T10:43:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.892+0000 7fd211a92700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd20c159560 0x7fd20c0aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.892+0000 7fd211a92700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd20c159560 0x7fd20c0aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40878/0 (socket says 172.21.15.92:40878) 2023-10-06T10:43:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.892+0000 7fd211a92700 1 -- 172.21.15.92:0/205958735 learned_addr learned my addr 172.21.15.92:0/205958735 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.892+0000 7fd211291700 1 --2- 172.21.15.92:0/205958735 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd20c0b0760 0x7fd20c169490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.892+0000 7fd210a90700 1 --2- 172.21.15.92:0/205958735 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd20c0b49a0 0x7fd20c0ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.893+0000 7fd211a92700 1 -- 172.21.15.92:0/205958735 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd20c0b49a0 msgr2=0x7fd20c0ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.893+0000 7fd211a92700 1 --2- 172.21.15.92:0/205958735 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd20c0b49a0 0x7fd20c0ad3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.893+0000 7fd211a92700 1 -- 172.21.15.92:0/205958735 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd20c0b0760 msgr2=0x7fd20c169490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.893+0000 7fd211a92700 1 --2- 172.21.15.92:0/205958735 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd20c0b0760 0x7fd20c169490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:39.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.893+0000 7fd211a92700 1 -- 172.21.15.92:0/205958735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd204009ae0 con 0x7fd20c159560 2023-10-06T10:43:39.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.893+0000 7fd211a92700 1 --2- 172.21.15.92:0/205958735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd20c159560 0x7fd20c0aeb10 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fd20c067c20 tx=0x7fd204017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:39.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.893+0000 7fd2017fa700 1 -- 172.21.15.92:0/205958735 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2040092e0 con 0x7fd20c159560 2023-10-06T10:43:39.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.893+0000 7fd2017fa700 1 -- 172.21.15.92:0/205958735 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd204012d00 con 0x7fd20c159560 2023-10-06T10:43:39.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.893+0000 7fd2134f5700 1 -- 172.21.15.92:0/205958735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd20c0ae030 con 0x7fd20c159560 2023-10-06T10:43:39.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.893+0000 7fd2017fa700 1 -- 172.21.15.92:0/205958735 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20401d670 con 0x7fd20c159560 2023-10-06T10:43:39.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.893+0000 7fd2134f5700 1 -- 172.21.15.92:0/205958735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd20c0ae5f0 con 0x7fd20c159560 2023-10-06T10:43:39.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.894+0000 7fd2134f5700 1 -- 172.21.15.92:0/205958735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd20c0cf430 con 0x7fd20c159560 2023-10-06T10:43:39.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.894+0000 7fd2017fa700 1 -- 172.21.15.92:0/205958735 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd20401da20 con 0x7fd20c159560 2023-10-06T10:43:39.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.895+0000 7fd2017fa700 1 --2- 172.21.15.92:0/205958735 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd1f0066f30 0x7fd1f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:39.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.895+0000 7fd2017fa700 1 -- 172.21.15.92:0/205958735 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5784+0+0 (secure 0 0 0) 0x7fd204085c20 con 0x7fd20c159560 2023-10-06T10:43:39.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.895+0000 7fd211291700 1 --2- 172.21.15.92:0/205958735 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd1f0066f30 0x7fd1f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:39.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.895+0000 7fd211291700 1 --2- 172.21.15.92:0/205958735 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd1f0066f30 0x7fd1f00693e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd2080061f0 tx=0x7fd208008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:39.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:39.899+0000 7fd2017fa700 1 -- 172.21.15.92:0/205958735 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd204052e40 con 0x7fd20c159560 2023-10-06T10:43:40.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.167+0000 7fd2134f5700 1 -- 172.21.15.92:0/205958735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]} v 0) v1 -- 0x7fd20c064a90 con 0x7fd20c159560 2023-10-06T10:43:40.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.426+0000 7fd2017fa700 1 -- 172.21.15.92:0/205958735 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]}]=0 v276) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd204056460 con 0x7fd20c159560 2023-10-06T10:43:40.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.428+0000 7fd2134f5700 1 -- 172.21.15.92:0/205958735 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd1f0066f30 msgr2=0x7fd1f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:40.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.428+0000 7fd2134f5700 1 --2- 172.21.15.92:0/205958735 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd1f0066f30 0x7fd1f00693e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd2080061f0 tx=0x7fd208008040 comp rx=0 tx=0).stop 2023-10-06T10:43:40.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.428+0000 7fd2134f5700 1 -- 172.21.15.92:0/205958735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd20c159560 msgr2=0x7fd20c0aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:40.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.428+0000 7fd2134f5700 1 --2- 172.21.15.92:0/205958735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd20c159560 0x7fd20c0aeb10 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fd20c067c20 tx=0x7fd204017000 comp rx=0 tx=0).stop 2023-10-06T10:43:40.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.428+0000 7fd2134f5700 1 -- 172.21.15.92:0/205958735 shutdown_connections 2023-10-06T10:43:40.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.428+0000 7fd2134f5700 1 --2- 172.21.15.92:0/205958735 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd1f0066f30 0x7fd1f00693e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:40.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.428+0000 7fd2134f5700 1 --2- 172.21.15.92:0/205958735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd20c159560 0x7fd20c0aeb10 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:40.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.428+0000 7fd2134f5700 1 --2- 172.21.15.92:0/205958735 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd20c0b0760 0x7fd20c169490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:40.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.428+0000 7fd2134f5700 1 --2- 172.21.15.92:0/205958735 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd20c0b49a0 0x7fd20c0ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:40.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.428+0000 7fd2134f5700 1 -- 172.21.15.92:0/205958735 >> 172.21.15.92:0/205958735 conn(0x7fd20c0997f0 msgr2=0x7fd20c1588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:40.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.428+0000 7fd2134f5700 1 -- 172.21.15.92:0/205958735 shutdown_connections 2023-10-06T10:43:40.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.428+0000 7fd2134f5700 1 -- 172.21.15.92:0/205958735 wait complete. 2023-10-06T10:43:40.442 INFO:tasks.cephfs_test_runner:test_scrub_non_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2023-10-06T10:43:40.442 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2023-10-06T10:43:40.442 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0' 2023-10-06T10:43:40.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.554+0000 7fc23eb85700 1 Processor -- start 2023-10-06T10:43:40.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.554+0000 7fc23eb85700 1 -- start start 2023-10-06T10:43:40.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.555+0000 7fc23eb85700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc23806a590 0x7fc2381536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:40.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.555+0000 7fc23eb85700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc23806e660 0x7fc238153c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:40.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.555+0000 7fc23eb85700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2380c9cc0 con 0x7fc23806a960 2023-10-06T10:43:40.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.555+0000 7fc23eb85700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc23806f560 con 0x7fc23806a590 2023-10-06T10:43:40.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.555+0000 7fc23eb85700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23806eac0 con 0x7fc23806e660 2023-10-06T10:43:40.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.555+0000 7fc23db83700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc23806a590 0x7fc2381536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:40.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.555+0000 7fc23d382700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc23806a960 0x7fc23806df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49160/0 (socket says 172.21.15.92:49160) 2023-10-06T10:43:40.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.555+0000 7fc23d382700 1 -- 172.21.15.92:0/3921287469 learned_addr learned my addr 172.21.15.92:0/3921287469 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:40.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.556+0000 7fc23e384700 1 --2- 172.21.15.92:0/3921287469 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc23806e660 0x7fc238153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:40.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.556+0000 7fc23db83700 1 -- 172.21.15.92:0/3921287469 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc23806e660 msgr2=0x7fc238153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:40.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.556+0000 7fc23db83700 1 --2- 172.21.15.92:0/3921287469 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc23806e660 0x7fc238153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:40.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.556+0000 7fc23cb81700 1 -- 172.21.15.92:0/3921287469 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 643340232 0 0) 0x7fc2380c9cc0 con 0x7fc23806a960 2023-10-06T10:43:40.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.556+0000 7fc23db83700 1 -- 172.21.15.92:0/3921287469 >> v1:172.21.15.92:6789/0 conn(0x7fc23806a960 legacy=0x7fc23806df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:40.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.556+0000 7fc23db83700 1 -- 172.21.15.92:0/3921287469 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2380c6710 con 0x7fc23806a590 2023-10-06T10:43:40.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.556+0000 7fc23e384700 1 --2- 172.21.15.92:0/3921287469 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc23806e660 0x7fc238153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:40.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.556+0000 7fc23db83700 1 --2- 172.21.15.92:0/3921287469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc23806a590 0x7fc2381536f0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fc234009910 tx=0x7fc2340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9914dce26d12298a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:40.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.556+0000 7fc23cb81700 1 -- 172.21.15.92:0/3921287469 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23400b820 con 0x7fc23806a590 2023-10-06T10:43:40.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.556+0000 7fc23cb81700 1 -- 172.21.15.92:0/3921287469 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23400b980 con 0x7fc23806a590 2023-10-06T10:43:40.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.556+0000 7fc23cb81700 1 -- 172.21.15.92:0/3921287469 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23400bd20 con 0x7fc23806a590 2023-10-06T10:43:40.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.557+0000 7fc23eb85700 1 -- 172.21.15.92:0/3921287469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc23806a590 msgr2=0x7fc2381536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:40.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.557+0000 7fc23eb85700 1 --2- 172.21.15.92:0/3921287469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc23806a590 0x7fc2381536f0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fc234009910 tx=0x7fc2340091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:40.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.557+0000 7fc23eb85700 1 -- 172.21.15.92:0/3921287469 shutdown_connections 2023-10-06T10:43:40.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.557+0000 7fc23eb85700 1 --2- 172.21.15.92:0/3921287469 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc23806e660 0x7fc238153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:40.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.557+0000 7fc23eb85700 1 --2- 172.21.15.92:0/3921287469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc23806a590 0x7fc2381536f0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:40.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.557+0000 7fc23eb85700 1 -- 172.21.15.92:0/3921287469 >> 172.21.15.92:0/3921287469 conn(0x7fc238068080 msgr2=0x7fc2380cbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:40.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.557+0000 7fc23eb85700 1 -- 172.21.15.92:0/3921287469 shutdown_connections 2023-10-06T10:43:40.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.557+0000 7fc23eb85700 1 -- 172.21.15.92:0/3921287469 wait complete. 2023-10-06T10:43:40.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.558+0000 7fc23eb85700 1 Processor -- start 2023-10-06T10:43:40.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.558+0000 7fc23eb85700 1 -- start start 2023-10-06T10:43:40.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.559+0000 7fc23eb85700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc23806a590 0x7fc238158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:40.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.559+0000 7fc23eb85700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc23806a960 0x7fc238159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:40.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.559+0000 7fc23eb85700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc23806e660 0x7fc2380cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:40.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.559+0000 7fc23eb85700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2380b0120 con 0x7fc23806e660 2023-10-06T10:43:40.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.559+0000 7fc23eb85700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc238169c60 con 0x7fc23806a590 2023-10-06T10:43:40.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.559+0000 7fc23eb85700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc238169f40 con 0x7fc23806a960 2023-10-06T10:43:40.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.559+0000 7fc23e384700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc23806e660 0x7fc2380cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:40.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.559+0000 7fc23e384700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc23806e660 0x7fc2380cb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40902/0 (socket says 172.21.15.92:40902) 2023-10-06T10:43:40.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.559+0000 7fc23e384700 1 -- 172.21.15.92:0/2504557367 learned_addr learned my addr 172.21.15.92:0/2504557367 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:40.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.559+0000 7fc23d382700 1 --2- 172.21.15.92:0/2504557367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc23806a960 0x7fc238159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:40.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.559+0000 7fc23db83700 1 --2- 172.21.15.92:0/2504557367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc23806a590 0x7fc238158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:40.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.559+0000 7fc23e384700 1 -- 172.21.15.92:0/2504557367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc23806a960 msgr2=0x7fc238159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:40.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.559+0000 7fc23e384700 1 --2- 172.21.15.92:0/2504557367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc23806a960 0x7fc238159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:40.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.559+0000 7fc23e384700 1 -- 172.21.15.92:0/2504557367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc23806a590 msgr2=0x7fc238158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:40.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.559+0000 7fc23e384700 1 --2- 172.21.15.92:0/2504557367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc23806a590 0x7fc238158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:40.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.559+0000 7fc23e384700 1 -- 172.21.15.92:0/2504557367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc234009420 con 0x7fc23806e660 2023-10-06T10:43:40.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.559+0000 7fc23e384700 1 --2- 172.21.15.92:0/2504557367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc23806e660 0x7fc2380cb120 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fc230009fc0 tx=0x7fc2300099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:40.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.559+0000 7fc22dffb700 1 -- 172.21.15.92:0/2504557367 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23000dd40 con 0x7fc23806e660 2023-10-06T10:43:40.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.559+0000 7fc22dffb700 1 -- 172.21.15.92:0/2504557367 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23000f040 con 0x7fc23806e660 2023-10-06T10:43:40.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.559+0000 7fc23eb85700 1 -- 172.21.15.92:0/2504557367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc23816a220 con 0x7fc23806e660 2023-10-06T10:43:40.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.559+0000 7fc22dffb700 1 -- 172.21.15.92:0/2504557367 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23000dea0 con 0x7fc23806e660 2023-10-06T10:43:40.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.559+0000 7fc23eb85700 1 -- 172.21.15.92:0/2504557367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc23816a7e0 con 0x7fc23806e660 2023-10-06T10:43:40.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.560+0000 7fc23eb85700 1 -- 172.21.15.92:0/2504557367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc23805f230 con 0x7fc23806e660 2023-10-06T10:43:40.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.561+0000 7fc22dffb700 1 -- 172.21.15.92:0/2504557367 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc23001e3b0 con 0x7fc23806e660 2023-10-06T10:43:40.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.561+0000 7fc22dffb700 1 --2- 172.21.15.92:0/2504557367 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc21c066f30 0x7fc21c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:40.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.561+0000 7fc23db83700 1 --2- 172.21.15.92:0/2504557367 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc21c066f30 0x7fc21c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:40.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.561+0000 7fc22dffb700 1 -- 172.21.15.92:0/2504557367 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5784+0+0 (secure 0 0 0) 0x7fc230084a50 con 0x7fc23806e660 2023-10-06T10:43:40.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.562+0000 7fc23db83700 1 --2- 172.21.15.92:0/2504557367 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc21c066f30 0x7fc21c0693e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc234009860 tx=0x7fc234017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:40.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.566+0000 7fc22dffb700 1 -- 172.21.15.92:0/2504557367 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc230051c70 con 0x7fc23806e660 2023-10-06T10:43:40.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:40.826+0000 7fc23eb85700 1 -- 172.21.15.92:0/2504557367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]} v 0) v1 -- 0x7fc2380ccb90 con 0x7fc23806e660 2023-10-06T10:43:41.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.427+0000 7fc22dffb700 1 -- 172.21.15.92:0/2504557367 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]}]=0 v277) v1 ==== 157+0+0 (secure 0 0 0) 0x7fc23001a0e0 con 0x7fc23806e660 2023-10-06T10:43:41.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.429+0000 7fc23eb85700 1 -- 172.21.15.92:0/2504557367 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc21c066f30 msgr2=0x7fc21c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:41.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.429+0000 7fc23eb85700 1 --2- 172.21.15.92:0/2504557367 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc21c066f30 0x7fc21c0693e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc234009860 tx=0x7fc234017000 comp rx=0 tx=0).stop 2023-10-06T10:43:41.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.429+0000 7fc23eb85700 1 -- 172.21.15.92:0/2504557367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc23806e660 msgr2=0x7fc2380cb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:41.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.429+0000 7fc23eb85700 1 --2- 172.21.15.92:0/2504557367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc23806e660 0x7fc2380cb120 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fc230009fc0 tx=0x7fc2300099b0 comp rx=0 tx=0).stop 2023-10-06T10:43:41.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.429+0000 7fc23eb85700 1 -- 172.21.15.92:0/2504557367 shutdown_connections 2023-10-06T10:43:41.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.429+0000 7fc23eb85700 1 --2- 172.21.15.92:0/2504557367 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc21c066f30 0x7fc21c0693e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:41.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.429+0000 7fc23eb85700 1 --2- 172.21.15.92:0/2504557367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc23806e660 0x7fc2380cb120 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:41.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.429+0000 7fc23eb85700 1 --2- 172.21.15.92:0/2504557367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc23806a590 0x7fc238158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:41.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.429+0000 7fc23eb85700 1 --2- 172.21.15.92:0/2504557367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc23806a960 0x7fc238159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:41.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.429+0000 7fc23eb85700 1 -- 172.21.15.92:0/2504557367 >> 172.21.15.92:0/2504557367 conn(0x7fc238068080 msgr2=0x7fc2381534c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:41.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.429+0000 7fc23eb85700 1 -- 172.21.15.92:0/2504557367 shutdown_connections 2023-10-06T10:43:41.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.430+0000 7fc23eb85700 1 -- 172.21.15.92:0/2504557367 wait complete. 2023-10-06T10:43:41.444 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T10:43:41.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.559+0000 7fd91add7700 1 Processor -- start 2023-10-06T10:43:41.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.559+0000 7fd91add7700 1 -- start start 2023-10-06T10:43:41.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.560+0000 7fd91add7700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd9140b2620 0x7fd9140cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:41.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.560+0000 7fd91add7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd9140cf1b0 0x7fd9140cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:41.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.560+0000 7fd91add7700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9140cc970 con 0x7fd9140b29f0 2023-10-06T10:43:41.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.560+0000 7fd91add7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9140cbd60 con 0x7fd9140cf1b0 2023-10-06T10:43:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.560+0000 7fd91add7700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9140cc040 con 0x7fd9140b2620 2023-10-06T10:43:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.560+0000 7fd91a5d6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd9140cf1b0 0x7fd9140cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.560+0000 7fd91a5d6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd9140cf1b0 0x7fd9140cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40912/0 (socket says 172.21.15.92:40912) 2023-10-06T10:43:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.560+0000 7fd91a5d6700 1 -- 172.21.15.92:0/372006931 learned_addr learned my addr 172.21.15.92:0/372006931 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.560+0000 7fd919dd5700 1 --2- 172.21.15.92:0/372006931 >> v2:172.21.15.124:3300/0 conn(0x7fd9140b2620 0x7fd9140cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.560+0000 7fd91a5d6700 1 -- 172.21.15.92:0/372006931 >> v2:172.21.15.124:3300/0 conn(0x7fd9140b2620 msgr2=0x7fd9140cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.560+0000 7fd91a5d6700 1 --2- 172.21.15.92:0/372006931 >> v2:172.21.15.124:3300/0 conn(0x7fd9140b2620 0x7fd9140cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.560+0000 7fd91a5d6700 1 -- 172.21.15.92:0/372006931 >> v1:172.21.15.124:6789/0 conn(0x7fd9140b29f0 legacy=0x7fd9140ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.560+0000 7fd91a5d6700 1 -- 172.21.15.92:0/372006931 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9140cc320 con 0x7fd9140cf1b0 2023-10-06T10:43:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.561+0000 7fd91a5d6700 1 --2- 172.21.15.92:0/372006931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9140cf1b0 0x7fd9140cd1f0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fd9140894c0 tx=0x7fd90c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af1f05117eb968a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.561+0000 7fd918dd3700 1 -- 172.21.15.92:0/372006931 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00e070 con 0x7fd9140cf1b0 2023-10-06T10:43:41.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.561+0000 7fd918dd3700 1 -- 172.21.15.92:0/372006931 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c00ba10 con 0x7fd9140cf1b0 2023-10-06T10:43:41.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.561+0000 7fd918dd3700 1 -- 172.21.15.92:0/372006931 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c0092e0 con 0x7fd9140cf1b0 2023-10-06T10:43:41.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.561+0000 7fd91add7700 1 -- 172.21.15.92:0/372006931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9140cf1b0 msgr2=0x7fd9140cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:41.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.561+0000 7fd91add7700 1 --2- 172.21.15.92:0/372006931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9140cf1b0 0x7fd9140cd1f0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fd9140894c0 tx=0x7fd90c009840 comp rx=0 tx=0).stop 2023-10-06T10:43:41.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.561+0000 7fd91add7700 1 -- 172.21.15.92:0/372006931 shutdown_connections 2023-10-06T10:43:41.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.561+0000 7fd91add7700 1 --2- 172.21.15.92:0/372006931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9140cf1b0 0x7fd9140cd1f0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:41.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.561+0000 7fd91add7700 1 --2- 172.21.15.92:0/372006931 >> v2:172.21.15.124:3300/0 conn(0x7fd9140b2620 0x7fd9140cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:41.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.561+0000 7fd91add7700 1 -- 172.21.15.92:0/372006931 >> 172.21.15.92:0/372006931 conn(0x7fd914099520 msgr2=0x7fd91406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:41.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.561+0000 7fd91add7700 1 -- 172.21.15.92:0/372006931 shutdown_connections 2023-10-06T10:43:41.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.561+0000 7fd91add7700 1 -- 172.21.15.92:0/372006931 wait complete. 2023-10-06T10:43:41.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.562+0000 7fd91add7700 1 Processor -- start 2023-10-06T10:43:41.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.562+0000 7fd91add7700 1 -- start start 2023-10-06T10:43:41.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.563+0000 7fd91add7700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9140b2620 0x7fd91413fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:41.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.563+0000 7fd91add7700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9140b29f0 0x7fd9140b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:41.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.563+0000 7fd91add7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9140cf1b0 0x7fd91415f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:41.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.563+0000 7fd91add7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9140b4a30 con 0x7fd9140cf1b0 2023-10-06T10:43:41.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.563+0000 7fd91add7700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd914154580 con 0x7fd9140b29f0 2023-10-06T10:43:41.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.563+0000 7fd91add7700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd914140190 con 0x7fd9140b2620 2023-10-06T10:43:41.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.563+0000 7fd91a5d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9140cf1b0 0x7fd91415f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:41.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.563+0000 7fd91a5d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9140cf1b0 0x7fd91415f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40918/0 (socket says 172.21.15.92:40918) 2023-10-06T10:43:41.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.563+0000 7fd91a5d6700 1 -- 172.21.15.92:0/3629729544 learned_addr learned my addr 172.21.15.92:0/3629729544 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:41.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.563+0000 7fd9195d4700 1 --2- 172.21.15.92:0/3629729544 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9140b29f0 0x7fd9140b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:41.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.563+0000 7fd919dd5700 1 --2- 172.21.15.92:0/3629729544 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9140b2620 0x7fd91413fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:41.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.563+0000 7fd91a5d6700 1 -- 172.21.15.92:0/3629729544 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9140b2620 msgr2=0x7fd91413fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:41.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.563+0000 7fd91a5d6700 1 --2- 172.21.15.92:0/3629729544 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9140b2620 0x7fd91413fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:41.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.563+0000 7fd91a5d6700 1 -- 172.21.15.92:0/3629729544 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9140b29f0 msgr2=0x7fd9140b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:41.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.563+0000 7fd91a5d6700 1 --2- 172.21.15.92:0/3629729544 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9140b29f0 0x7fd9140b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:41.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.563+0000 7fd91a5d6700 1 -- 172.21.15.92:0/3629729544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c009ae0 con 0x7fd9140cf1b0 2023-10-06T10:43:41.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.563+0000 7fd9195d4700 1 --2- 172.21.15.92:0/3629729544 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9140b29f0 0x7fd9140b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:41.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.564+0000 7fd91a5d6700 1 --2- 172.21.15.92:0/3629729544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9140cf1b0 0x7fd91415f8c0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fd91406fcc0 tx=0x7fd90c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:41.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.564+0000 7fd909ffb700 1 -- 172.21.15.92:0/3629729544 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c012e60 con 0x7fd9140cf1b0 2023-10-06T10:43:41.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.564+0000 7fd909ffb700 1 -- 172.21.15.92:0/3629729544 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c0092e0 con 0x7fd9140cf1b0 2023-10-06T10:43:41.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.564+0000 7fd909ffb700 1 -- 172.21.15.92:0/3629729544 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c01d670 con 0x7fd9140cf1b0 2023-10-06T10:43:41.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.564+0000 7fd91add7700 1 -- 172.21.15.92:0/3629729544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9140b4100 con 0x7fd9140cf1b0 2023-10-06T10:43:41.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.564+0000 7fd91add7700 1 -- 172.21.15.92:0/3629729544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9140b46c0 con 0x7fd9140cf1b0 2023-10-06T10:43:41.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.565+0000 7fd91add7700 1 -- 172.21.15.92:0/3629729544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd91405f230 con 0x7fd9140cf1b0 2023-10-06T10:43:41.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.565+0000 7fd909ffb700 1 -- 172.21.15.92:0/3629729544 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd90c01d9e0 con 0x7fd9140cf1b0 2023-10-06T10:43:41.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.566+0000 7fd909ffb700 1 --2- 172.21.15.92:0/3629729544 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd8f8066f30 0x7fd8f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:41.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.566+0000 7fd909ffb700 1 -- 172.21.15.92:0/3629729544 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5784+0+0 (secure 0 0 0) 0x7fd90c085e00 con 0x7fd9140cf1b0 2023-10-06T10:43:41.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.566+0000 7fd919dd5700 1 --2- 172.21.15.92:0/3629729544 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd8f8066f30 0x7fd8f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:41.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.566+0000 7fd919dd5700 1 --2- 172.21.15.92:0/3629729544 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd8f8066f30 0x7fd8f80693e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd910006520 tx=0x7fd910008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:41.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.570+0000 7fd909ffb700 1 -- 172.21.15.92:0/3629729544 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd90c053020 con 0x7fd9140cf1b0 2023-10-06T10:43:41.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.752+0000 7fd91add7700 1 -- 172.21.15.92:0/3629729544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd9140cc600 con 0x7fd9140cf1b0 2023-10-06T10:43:41.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.756+0000 7fd909ffb700 1 -- 172.21.15.92:0/3629729544 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd90c056640 con 0x7fd9140cf1b0 2023-10-06T10:43:41.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.758+0000 7fd91add7700 1 -- 172.21.15.92:0/3629729544 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd8f8066f30 msgr2=0x7fd8f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:41.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.758+0000 7fd91add7700 1 --2- 172.21.15.92:0/3629729544 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd8f8066f30 0x7fd8f80693e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd910006520 tx=0x7fd910008040 comp rx=0 tx=0).stop 2023-10-06T10:43:41.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.758+0000 7fd91add7700 1 -- 172.21.15.92:0/3629729544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9140cf1b0 msgr2=0x7fd91415f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:41.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.758+0000 7fd91add7700 1 --2- 172.21.15.92:0/3629729544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9140cf1b0 0x7fd91415f8c0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fd91406fcc0 tx=0x7fd90c017000 comp rx=0 tx=0).stop 2023-10-06T10:43:41.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.758+0000 7fd91add7700 1 -- 172.21.15.92:0/3629729544 shutdown_connections 2023-10-06T10:43:41.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.758+0000 7fd91add7700 1 --2- 172.21.15.92:0/3629729544 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd8f8066f30 0x7fd8f80693e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:41.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.758+0000 7fd91add7700 1 --2- 172.21.15.92:0/3629729544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9140cf1b0 0x7fd91415f8c0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:41.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.758+0000 7fd91add7700 1 --2- 172.21.15.92:0/3629729544 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9140b2620 0x7fd91413fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:41.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.758+0000 7fd91add7700 1 --2- 172.21.15.92:0/3629729544 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9140b29f0 0x7fd9140b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:41.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.758+0000 7fd91add7700 1 -- 172.21.15.92:0/3629729544 >> 172.21.15.92:0/3629729544 conn(0x7fd914099520 msgr2=0x7fd9140cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:41.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.758+0000 7fd91add7700 1 -- 172.21.15.92:0/3629729544 shutdown_connections 2023-10-06T10:43:41.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.758+0000 7fd91add7700 1 -- 172.21.15.92:0/3629729544 wait complete. 2023-10-06T10:43:41.772 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T10:43:41.772 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:43:41.808 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:43:41.850 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T10:43:41.850 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T10:43:41.850 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:43:41.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.961+0000 7f803171a700 1 Processor -- start 2023-10-06T10:43:41.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.961+0000 7f803171a700 1 -- start start 2023-10-06T10:43:41.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.962+0000 7f803171a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f802c159440 0x7f802c0b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:41.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.962+0000 7f803171a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f802c0b35d0 0x7f802c0cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:41.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.962+0000 7f803171a700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f802c0cb7a0 con 0x7f802c0cc170 2023-10-06T10:43:41.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.962+0000 7f803171a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f802c0b0860 con 0x7f802c159440 2023-10-06T10:43:41.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.962+0000 7f803171a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0cb150 con 0x7f802c0b35d0 2023-10-06T10:43:41.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.962+0000 7f802bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f802c159440 0x7f802c0b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:41.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.962+0000 7f802bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f802c159440 0x7f802c0b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40928/0 (socket says 172.21.15.92:40928) 2023-10-06T10:43:41.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.962+0000 7f802bfff700 1 -- 172.21.15.92:0/848548471 learned_addr learned my addr 172.21.15.92:0/848548471 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:41.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.962+0000 7f802b7fe700 1 --2- 172.21.15.92:0/848548471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f802c0b35d0 0x7f802c0cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:41.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.962+0000 7f802bfff700 1 -- 172.21.15.92:0/848548471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f802c0b35d0 msgr2=0x7f802c0cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:41.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.962+0000 7f802bfff700 1 --2- 172.21.15.92:0/848548471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f802c0b35d0 0x7f802c0cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:41.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.962+0000 7f802bfff700 1 -- 172.21.15.92:0/848548471 >> v1:172.21.15.124:6789/0 conn(0x7f802c0cc170 legacy=0x7f802c0b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:41.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.962+0000 7f802bfff700 1 -- 172.21.15.92:0/848548471 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c0cb430 con 0x7f802c159440 2023-10-06T10:43:41.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.962+0000 7f802b7fe700 1 --2- 172.21.15.92:0/848548471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f802c0b35d0 0x7f802c0cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:41.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.962+0000 7f802bfff700 1 --2- 172.21.15.92:0/848548471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f802c159440 0x7f802c0b3090 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f801c0099b0 tx=0x7f801c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2da87ae869c4897 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:41.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.963+0000 7f802affd700 1 -- 172.21.15.92:0/848548471 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c00b860 con 0x7f802c159440 2023-10-06T10:43:41.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.963+0000 7f802affd700 1 -- 172.21.15.92:0/848548471 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801c00b9c0 con 0x7f802c159440 2023-10-06T10:43:41.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.963+0000 7f802affd700 1 -- 172.21.15.92:0/848548471 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c00bd60 con 0x7f802c159440 2023-10-06T10:43:41.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.963+0000 7f803171a700 1 -- 172.21.15.92:0/848548471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f802c159440 msgr2=0x7f802c0b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:41.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.963+0000 7f803171a700 1 --2- 172.21.15.92:0/848548471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f802c159440 0x7f802c0b3090 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f801c0099b0 tx=0x7f801c009250 comp rx=0 tx=0).stop 2023-10-06T10:43:41.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.963+0000 7f803171a700 1 -- 172.21.15.92:0/848548471 shutdown_connections 2023-10-06T10:43:41.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.963+0000 7f803171a700 1 --2- 172.21.15.92:0/848548471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f802c159440 0x7f802c0b3090 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:41.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.963+0000 7f803171a700 1 --2- 172.21.15.92:0/848548471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f802c0b35d0 0x7f802c0cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:41.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.963+0000 7f803171a700 1 -- 172.21.15.92:0/848548471 >> 172.21.15.92:0/848548471 conn(0x7f802c06ece0 msgr2=0x7f802c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:41.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.963+0000 7f803171a700 1 -- 172.21.15.92:0/848548471 shutdown_connections 2023-10-06T10:43:41.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.963+0000 7f803171a700 1 -- 172.21.15.92:0/848548471 wait complete. 2023-10-06T10:43:41.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.964+0000 7f803171a700 1 Processor -- start 2023-10-06T10:43:41.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.964+0000 7f803171a700 1 -- start start 2023-10-06T10:43:41.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.965+0000 7f803171a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f802c0b35d0 0x7f802c154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:41.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.965+0000 7f803171a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f802c0cc170 0x7f802c15a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:41.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.965+0000 7f803171a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f802c159440 0x7f802c0b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:41.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.965+0000 7f803171a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0d0980 con 0x7f802c159440 2023-10-06T10:43:41.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.965+0000 7f803171a700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c154fa0 con 0x7f802c0b35d0 2023-10-06T10:43:41.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.965+0000 7f803171a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0cfd70 con 0x7f802c0cc170 2023-10-06T10:43:41.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.965+0000 7f8030f19700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f802c159440 0x7f802c0b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:41.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.965+0000 7f802bfff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f802c0b35d0 0x7f802c154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:41.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.965+0000 7f8030f19700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f802c159440 0x7f802c0b4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40942/0 (socket says 172.21.15.92:40942) 2023-10-06T10:43:41.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.965+0000 7f8030f19700 1 -- 172.21.15.92:0/2827907757 learned_addr learned my addr 172.21.15.92:0/2827907757 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:41.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.965+0000 7f802b7fe700 1 --2- 172.21.15.92:0/2827907757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f802c0cc170 0x7f802c15a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:41.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.966+0000 7f8030f19700 1 -- 172.21.15.92:0/2827907757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f802c0cc170 msgr2=0x7f802c15a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:41.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.966+0000 7f8030f19700 1 --2- 172.21.15.92:0/2827907757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f802c0cc170 0x7f802c15a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:41.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.966+0000 7f8030f19700 1 -- 172.21.15.92:0/2827907757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f802c0b35d0 msgr2=0x7f802c154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:41.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.966+0000 7f8030f19700 1 --2- 172.21.15.92:0/2827907757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f802c0b35d0 0x7f802c154a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:41.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.966+0000 7f8030f19700 1 -- 172.21.15.92:0/2827907757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c0094f0 con 0x7f802c159440 2023-10-06T10:43:41.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.966+0000 7f8030f19700 1 --2- 172.21.15.92:0/2827907757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f802c159440 0x7f802c0b4ac0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f802400c910 tx=0x7f8024001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:41.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.966+0000 7f8013fff700 1 -- 172.21.15.92:0/2827907757 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024012630 con 0x7f802c159440 2023-10-06T10:43:41.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.966+0000 7f8013fff700 1 -- 172.21.15.92:0/2827907757 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8024012790 con 0x7f802c159440 2023-10-06T10:43:41.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.966+0000 7f803171a700 1 -- 172.21.15.92:0/2827907757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802c0d0050 con 0x7f802c159440 2023-10-06T10:43:41.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.966+0000 7f8013fff700 1 -- 172.21.15.92:0/2827907757 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024012b50 con 0x7f802c159440 2023-10-06T10:43:41.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.966+0000 7f803171a700 1 -- 172.21.15.92:0/2827907757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802c0d0610 con 0x7f802c159440 2023-10-06T10:43:41.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.967+0000 7f803171a700 1 -- 172.21.15.92:0/2827907757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802c0d0b30 con 0x7f802c159440 2023-10-06T10:43:41.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.967+0000 7f8013fff700 1 -- 172.21.15.92:0/2827907757 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f802401a3a0 con 0x7f802c159440 2023-10-06T10:43:41.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.968+0000 7f8013fff700 1 --2- 172.21.15.92:0/2827907757 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8018066f30 0x7f80180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:41.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.968+0000 7f8013fff700 1 -- 172.21.15.92:0/2827907757 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f8024084770 con 0x7f802c159440 2023-10-06T10:43:41.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.968+0000 7f802bfff700 1 --2- 172.21.15.92:0/2827907757 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8018066f30 0x7f80180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:41.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.968+0000 7f802bfff700 1 --2- 172.21.15.92:0/2827907757 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8018066f30 0x7f80180693e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f801c0099b0 tx=0x7f801c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:41.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:41.972+0000 7f8013fff700 1 -- 172.21.15.92:0/2827907757 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f802c0d0b30 con 0x7f802c159440 2023-10-06T10:43:42.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.200+0000 7f803171a700 1 -- 172.21.15.92:0/2827907757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f802c0d0b30 con 0x7f802c159440 2023-10-06T10:43:42.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.201+0000 7f8013fff700 1 -- 172.21.15.92:0/2827907757 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f802c0d0b30 con 0x7f802c159440 2023-10-06T10:43:42.203 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:42.203 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":77,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/3469511807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3469511807},{"type":"v1","addr":"172.21.15.124:6834","nonce":3469511807}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1186839451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":1186839451},{"type":"v1","addr":"172.21.15.124:6835","nonce":1186839451}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":7623,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/96716863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":96716863},{"type":"v1","addr":"172.21.15.92:6837","nonce":96716863}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":7626,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3836247068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3836247068},{"type":"v1","addr":"172.21.15.92:6836","nonce":3836247068}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[]} 2023-10-06T10:43:42.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.203+0000 7f803171a700 1 -- 172.21.15.92:0/2827907757 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8018066f30 msgr2=0x7f80180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:42.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.203+0000 7f803171a700 1 --2- 172.21.15.92:0/2827907757 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8018066f30 0x7f80180693e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f801c0099b0 tx=0x7f801c017000 comp rx=0 tx=0).stop 2023-10-06T10:43:42.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.203+0000 7f803171a700 1 -- 172.21.15.92:0/2827907757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f802c159440 msgr2=0x7f802c0b4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:42.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.203+0000 7f803171a700 1 --2- 172.21.15.92:0/2827907757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f802c159440 0x7f802c0b4ac0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f802400c910 tx=0x7f8024001710 comp rx=0 tx=0).stop 2023-10-06T10:43:42.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.203+0000 7f803171a700 1 -- 172.21.15.92:0/2827907757 shutdown_connections 2023-10-06T10:43:42.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.203+0000 7f803171a700 1 --2- 172.21.15.92:0/2827907757 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8018066f30 0x7f80180693e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:42.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.203+0000 7f803171a700 1 --2- 172.21.15.92:0/2827907757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f802c159440 0x7f802c0b4ac0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:42.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.203+0000 7f803171a700 1 --2- 172.21.15.92:0/2827907757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f802c0b35d0 0x7f802c154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:42.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.203+0000 7f803171a700 1 --2- 172.21.15.92:0/2827907757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f802c0cc170 0x7f802c15a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:42.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.203+0000 7f803171a700 1 -- 172.21.15.92:0/2827907757 >> 172.21.15.92:0/2827907757 conn(0x7f802c06ece0 msgr2=0x7f802c0cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:42.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.203+0000 7f803171a700 1 -- 172.21.15.92:0/2827907757 shutdown_connections 2023-10-06T10:43:42.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.203+0000 7f803171a700 1 -- 172.21.15.92:0/2827907757 wait complete. 2023-10-06T10:43:42.206 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 77 2023-10-06T10:43:42.217 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T10:43:42.217 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T10:43:42.217 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T10:43:42.217 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:43:42.218 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T10:43:42.218 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T10:43:42.218 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T10:43:42.218 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:43:42.218 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T10:43:42.218 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T10:43:42.219 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T10:43:42.219 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:43:42.219 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T10:43:42.219 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T10:43:42.219 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T10:43:42.219 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:43:42.226 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T10:43:42.226 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T10:43:42.227 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T10:43:42.227 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T10:43:42.229 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T10:43:42.230 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T10:43:42.230 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T10:43:42.230 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T10:43:42.269 INFO:tasks.ceph.mds.a:Started 2023-10-06T10:43:42.269 DEBUG:teuthology.parallel:result is None 2023-10-06T10:43:42.269 INFO:tasks.ceph.mds.c:Started 2023-10-06T10:43:42.270 DEBUG:teuthology.parallel:result is None 2023-10-06T10:43:42.272 INFO:tasks.ceph.mds.d:Started 2023-10-06T10:43:42.272 DEBUG:teuthology.parallel:result is None 2023-10-06T10:43:42.273 INFO:tasks.ceph.mds.b:Started 2023-10-06T10:43:42.273 DEBUG:teuthology.parallel:result is None 2023-10-06T10:43:42.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T10:43:42.341 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T10:43:42.342 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T10:43:42.357 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-06T10:43:42.358 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-06T10:43:42.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.391+0000 7f079ca4f700 1 Processor -- start 2023-10-06T10:43:42.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.391+0000 7f079ca4f700 1 -- start start 2023-10-06T10:43:42.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.392+0000 7f079ca4f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0798067cc0 0x7f079806e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:42.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.392+0000 7f079ca4f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079806bcc0 con 0x7f0798157600 2023-10-06T10:43:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.392+0000 7f079ca4f700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07980ca6f0 con 0x7f07980ca940 2023-10-06T10:43:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.392+0000 7f079ca4f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f079806ecd0 con 0x7f0798067cc0 2023-10-06T10:43:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.392+0000 7f0795d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0798067cc0 0x7f079806e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.392+0000 7f079659c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0798157600 0x7f07980675b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49176/0 (socket says 172.21.15.92:49176) 2023-10-06T10:43:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.392+0000 7f0795d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0798067cc0 0x7f079806e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40970/0 (socket says 172.21.15.92:40970) 2023-10-06T10:43:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.392+0000 7f079659c700 1 -- 172.21.15.92:0/4213979984 learned_addr learned my addr 172.21.15.92:0/4213979984 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.393+0000 7f0795d9b700 1 -- 172.21.15.92:0/4213979984 >> v1:172.21.15.124:6789/0 conn(0x7f07980ca940 legacy=0x7f079806c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.393+0000 7f0795d9b700 1 -- 172.21.15.92:0/4213979984 >> v1:172.21.15.92:6789/0 conn(0x7f0798157600 legacy=0x7f07980675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.393+0000 7f079559a700 1 -- 172.21.15.92:0/4213979984 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 335661847 0 0) 0x7f079806bcc0 con 0x7f0798157600 2023-10-06T10:43:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.393+0000 7f0795d9b700 1 -- 172.21.15.92:0/4213979984 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079806efb0 con 0x7f0798067cc0 2023-10-06T10:43:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.393+0000 7f0795d9b700 1 --2- 172.21.15.92:0/4213979984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0798067cc0 0x7f079806e680 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f07800098e0 tx=0x7f07800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89167fd224246d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:42.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.393+0000 7f079559a700 1 -- 172.21.15.92:0/4213979984 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780009e80 con 0x7f0798067cc0 2023-10-06T10:43:42.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.393+0000 7f079559a700 1 -- 172.21.15.92:0/4213979984 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078000b860 con 0x7f0798067cc0 2023-10-06T10:43:42.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.393+0000 7f079559a700 1 -- 172.21.15.92:0/4213979984 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078000bbe0 con 0x7f0798067cc0 2023-10-06T10:43:42.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.394+0000 7f079ca4f700 1 -- 172.21.15.92:0/4213979984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0798067cc0 msgr2=0x7f079806e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:42.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.394+0000 7f079ca4f700 1 --2- 172.21.15.92:0/4213979984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0798067cc0 0x7f079806e680 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f07800098e0 tx=0x7f07800091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:42.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.394+0000 7f079ca4f700 1 -- 172.21.15.92:0/4213979984 shutdown_connections 2023-10-06T10:43:42.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.394+0000 7f079ca4f700 1 --2- 172.21.15.92:0/4213979984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0798067cc0 0x7f079806e680 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:42.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.394+0000 7f079ca4f700 1 -- 172.21.15.92:0/4213979984 >> 172.21.15.92:0/4213979984 conn(0x7f07980cdeb0 msgr2=0x7f07980c9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:42.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.394+0000 7f079ca4f700 1 -- 172.21.15.92:0/4213979984 shutdown_connections 2023-10-06T10:43:42.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.394+0000 7f079ca4f700 1 -- 172.21.15.92:0/4213979984 wait complete. 2023-10-06T10:43:42.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.395+0000 7f079ca4f700 1 Processor -- start 2023-10-06T10:43:42.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.395+0000 7f079ca4f700 1 -- start start 2023-10-06T10:43:42.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.395+0000 7f079ca4f700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0798067cc0 0x7f07981579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:42.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.395+0000 7f079ca4f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07980ca940 0x7f07980aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:42.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.395+0000 7f079ca4f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0798157600 0x7f07981500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:42.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.396+0000 7f079ca4f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0798153e80 con 0x7f0798157600 2023-10-06T10:43:42.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.396+0000 7f079ca4f700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07980af000 con 0x7f0798067cc0 2023-10-06T10:43:42.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.396+0000 7f079ca4f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0798154010 con 0x7f07980ca940 2023-10-06T10:43:42.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.396+0000 7f0796d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0798157600 0x7f07981500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:42.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.396+0000 7f079659c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0798067cc0 0x7f07981579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:42.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.396+0000 7f0796d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0798157600 0x7f07981500d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40982/0 (socket says 172.21.15.92:40982) 2023-10-06T10:43:42.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.396+0000 7f0796d9d700 1 -- 172.21.15.92:0/2720700720 learned_addr learned my addr 172.21.15.92:0/2720700720 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:42.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.396+0000 7f0795d9b700 1 --2- 172.21.15.92:0/2720700720 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07980ca940 0x7f07980aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:42.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.396+0000 7f0796d9d700 1 -- 172.21.15.92:0/2720700720 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07980ca940 msgr2=0x7f07980aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:42.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.396+0000 7f0796d9d700 1 --2- 172.21.15.92:0/2720700720 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07980ca940 0x7f07980aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:42.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.396+0000 7f0796d9d700 1 -- 172.21.15.92:0/2720700720 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0798067cc0 msgr2=0x7f07981579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:42.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.396+0000 7f0796d9d700 1 --2- 172.21.15.92:0/2720700720 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0798067cc0 0x7f07981579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:42.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.396+0000 7f0796d9d700 1 -- 172.21.15.92:0/2720700720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0780009420 con 0x7f0798157600 2023-10-06T10:43:42.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.396+0000 7f079659c700 1 --2- 172.21.15.92:0/2720700720 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0798067cc0 0x7f07981579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:43:42.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.396+0000 7f0796d9d700 1 --2- 172.21.15.92:0/2720700720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0798157600 0x7f07981500d0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f078800c560 tx=0x7f0788007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:42.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.396+0000 7f078e7fc700 1 -- 172.21.15.92:0/2720700720 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788011590 con 0x7f0798157600 2023-10-06T10:43:42.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.396+0000 7f078e7fc700 1 -- 172.21.15.92:0/2720700720 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07880116f0 con 0x7f0798157600 2023-10-06T10:43:42.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.396+0000 7f079ca4f700 1 -- 172.21.15.92:0/2720700720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0798158060 con 0x7f0798157600 2023-10-06T10:43:42.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.396+0000 7f078e7fc700 1 -- 172.21.15.92:0/2720700720 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788011ab0 con 0x7f0798157600 2023-10-06T10:43:42.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.396+0000 7f079ca4f700 1 -- 172.21.15.92:0/2720700720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0798158620 con 0x7f0798157600 2023-10-06T10:43:42.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.397+0000 7f079ca4f700 1 -- 172.21.15.92:0/2720700720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0798154180 con 0x7f0798157600 2023-10-06T10:43:42.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.398+0000 7f078e7fc700 1 -- 172.21.15.92:0/2720700720 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f078801b3b0 con 0x7f0798157600 2023-10-06T10:43:42.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.398+0000 7f078e7fc700 1 --2- 172.21.15.92:0/2720700720 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0778066f30 0x7f07780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:42.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.398+0000 7f078e7fc700 1 -- 172.21.15.92:0/2720700720 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f0788083940 con 0x7f0798157600 2023-10-06T10:43:42.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.398+0000 7f079659c700 1 --2- 172.21.15.92:0/2720700720 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0778066f30 0x7f07780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:42.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.398+0000 7f079659c700 1 --2- 172.21.15.92:0/2720700720 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0778066f30 0x7f07780693e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0790001560 tx=0x7f0790009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:42.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.402+0000 7f078e7fc700 1 -- 172.21.15.92:0/2720700720 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0788050b60 con 0x7f0798157600 2023-10-06T10:43:42.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:42.602+0000 7f079ca4f700 1 -- 172.21.15.92:0/2720700720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0798064a90 con 0x7f0798157600 2023-10-06T10:43:43.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.451+0000 7f078e7fc700 1 -- 172.21.15.92:0/2720700720 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v86) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0788017070 con 0x7f0798157600 2023-10-06T10:43:43.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.453+0000 7f079ca4f700 1 -- 172.21.15.92:0/2720700720 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0778066f30 msgr2=0x7f07780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:43.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.453+0000 7f079ca4f700 1 --2- 172.21.15.92:0/2720700720 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0778066f30 0x7f07780693e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0790001560 tx=0x7f0790009040 comp rx=0 tx=0).stop 2023-10-06T10:43:43.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.453+0000 7f079ca4f700 1 -- 172.21.15.92:0/2720700720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0798157600 msgr2=0x7f07981500d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:43.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.453+0000 7f079ca4f700 1 --2- 172.21.15.92:0/2720700720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0798157600 0x7f07981500d0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f078800c560 tx=0x7f0788007ef0 comp rx=0 tx=0).stop 2023-10-06T10:43:43.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.453+0000 7f079ca4f700 1 -- 172.21.15.92:0/2720700720 shutdown_connections 2023-10-06T10:43:43.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.453+0000 7f079ca4f700 1 --2- 172.21.15.92:0/2720700720 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0778066f30 0x7f07780693e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:43.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.453+0000 7f079ca4f700 1 --2- 172.21.15.92:0/2720700720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0798157600 0x7f07981500d0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:43.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.453+0000 7f079ca4f700 1 --2- 172.21.15.92:0/2720700720 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0798067cc0 0x7f07981579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:43.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.453+0000 7f079ca4f700 1 --2- 172.21.15.92:0/2720700720 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07980ca940 0x7f07980aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:43.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.453+0000 7f079ca4f700 1 -- 172.21.15.92:0/2720700720 >> 172.21.15.92:0/2720700720 conn(0x7f07980cdeb0 msgr2=0x7f07980b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:43.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.453+0000 7f079ca4f700 1 -- 172.21.15.92:0/2720700720 shutdown_connections 2023-10-06T10:43:43.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.453+0000 7f079ca4f700 1 -- 172.21.15.92:0/2720700720 wait complete. 2023-10-06T10:43:43.456 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-10-06T10:43:43.467 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T10:43:43.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.579+0000 7f283fff2700 1 Processor -- start 2023-10-06T10:43:43.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.579+0000 7f283fff2700 1 -- start start 2023-10-06T10:43:43.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.580+0000 7f283fff2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2838159560 0x7f28380aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:43.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.580+0000 7f283fff2700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f28380b49a0 0x7f28380aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:43.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.580+0000 7f283fff2700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28380b0760 0x7f28380b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:43.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.580+0000 7f283fff2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28380b0c60 con 0x7f2838159560 2023-10-06T10:43:43.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.580+0000 7f283fff2700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28380b0f40 con 0x7f28380b49a0 2023-10-06T10:43:43.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.580+0000 7f283fff2700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28380b1220 con 0x7f28380b0760 2023-10-06T10:43:43.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.580+0000 7f283dd8e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2838159560 0x7f28380aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:43.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.580+0000 7f283dd8e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2838159560 0x7f28380aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44768/0 (socket says 172.21.15.92:44768) 2023-10-06T10:43:43.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.580+0000 7f283dd8e700 1 -- 172.21.15.92:0/655124027 learned_addr learned my addr 172.21.15.92:0/655124027 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:43.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.581+0000 7f283dd8e700 1 -- 172.21.15.92:0/655124027 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28380b0760 msgr2=0x7f28380b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:43.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.581+0000 7f283dd8e700 1 --2- 172.21.15.92:0/655124027 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28380b0760 0x7f28380b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:43.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.581+0000 7f283dd8e700 1 -- 172.21.15.92:0/655124027 >> v2:172.21.15.124:3300/0 conn(0x7f28380b49a0 msgr2=0x7f28380aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:43.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.581+0000 7f283dd8e700 1 --2- 172.21.15.92:0/655124027 >> v2:172.21.15.124:3300/0 conn(0x7f28380b49a0 0x7f28380aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:43.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.581+0000 7f283dd8e700 1 -- 172.21.15.92:0/655124027 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28380b1500 con 0x7f2838159560 2023-10-06T10:43:43.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.581+0000 7f283dd8e700 1 --2- 172.21.15.92:0/655124027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2838159560 0x7f28380aa530 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f2834009990 tx=0x7f2834009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193bd8efeb4fe0e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:43.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.581+0000 7f283cd8c700 1 -- 172.21.15.92:0/655124027 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2834005c10 con 0x7f2838159560 2023-10-06T10:43:43.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.581+0000 7f283cd8c700 1 -- 172.21.15.92:0/655124027 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2834005d70 con 0x7f2838159560 2023-10-06T10:43:43.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.581+0000 7f283cd8c700 1 -- 172.21.15.92:0/655124027 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283400b840 con 0x7f2838159560 2023-10-06T10:43:43.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.581+0000 7f283fff2700 1 -- 172.21.15.92:0/655124027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2838159560 msgr2=0x7f28380aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:43.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.581+0000 7f283fff2700 1 --2- 172.21.15.92:0/655124027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2838159560 0x7f28380aa530 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f2834009990 tx=0x7f2834009220 comp rx=0 tx=0).stop 2023-10-06T10:43:43.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.581+0000 7f283fff2700 1 -- 172.21.15.92:0/655124027 shutdown_connections 2023-10-06T10:43:43.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.581+0000 7f283fff2700 1 --2- 172.21.15.92:0/655124027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2838159560 0x7f28380aa530 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:43.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.581+0000 7f283fff2700 1 --2- 172.21.15.92:0/655124027 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28380b0760 0x7f28380b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:43.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.581+0000 7f283fff2700 1 --2- 172.21.15.92:0/655124027 >> v2:172.21.15.124:3300/0 conn(0x7f28380b49a0 0x7f28380aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:43.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.581+0000 7f283fff2700 1 -- 172.21.15.92:0/655124027 >> 172.21.15.92:0/655124027 conn(0x7f28380997f0 msgr2=0x7f283806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:43.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.582+0000 7f283fff2700 1 -- 172.21.15.92:0/655124027 shutdown_connections 2023-10-06T10:43:43.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.582+0000 7f283fff2700 1 -- 172.21.15.92:0/655124027 wait complete. 2023-10-06T10:43:43.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.583+0000 7f283fff2700 1 Processor -- start 2023-10-06T10:43:43.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.583+0000 7f283fff2700 1 -- start start 2023-10-06T10:43:43.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.583+0000 7f283fff2700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28380b0760 0x7f283809bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:43.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.583+0000 7f283fff2700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28380b49a0 0x7f283815c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:43.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.583+0000 7f283fff2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2838159560 0x7f2838156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:43.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.583+0000 7f283fff2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283815c7d0 con 0x7f2838159560 2023-10-06T10:43:43.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.583+0000 7f283fff2700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2838159dc0 con 0x7f28380b49a0 2023-10-06T10:43:43.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.583+0000 7f283fff2700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f283809c3b0 con 0x7f28380b0760 2023-10-06T10:43:43.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.583+0000 7f283e58f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2838159560 0x7f2838156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:43.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.583+0000 7f283e58f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2838159560 0x7f2838156990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44782/0 (socket says 172.21.15.92:44782) 2023-10-06T10:43:43.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.584+0000 7f283e58f700 1 -- 172.21.15.92:0/3695853801 learned_addr learned my addr 172.21.15.92:0/3695853801 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:43.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.584+0000 7f283dd8e700 1 --2- 172.21.15.92:0/3695853801 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28380b0760 0x7f283809bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:43.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.584+0000 7f283d58d700 1 --2- 172.21.15.92:0/3695853801 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28380b49a0 0x7f283815c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:43.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.584+0000 7f283e58f700 1 -- 172.21.15.92:0/3695853801 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28380b0760 msgr2=0x7f283809bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:43.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.584+0000 7f283e58f700 1 --2- 172.21.15.92:0/3695853801 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28380b0760 0x7f283809bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:43.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.584+0000 7f283e58f700 1 -- 172.21.15.92:0/3695853801 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28380b49a0 msgr2=0x7f283815c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:43.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.584+0000 7f283e58f700 1 --2- 172.21.15.92:0/3695853801 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28380b49a0 0x7f283815c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:43.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.584+0000 7f283e58f700 1 -- 172.21.15.92:0/3695853801 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2834009480 con 0x7f2838159560 2023-10-06T10:43:43.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.584+0000 7f283e58f700 1 --2- 172.21.15.92:0/3695853801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2838159560 0x7f2838156990 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f283000bb60 tx=0x7f2830009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:43.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.584+0000 7f282dffb700 1 -- 172.21.15.92:0/3695853801 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283000c8c0 con 0x7f2838159560 2023-10-06T10:43:43.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.584+0000 7f282dffb700 1 -- 172.21.15.92:0/3695853801 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283000ca40 con 0x7f2838159560 2023-10-06T10:43:43.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.584+0000 7f283fff2700 1 -- 172.21.15.92:0/3695853801 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283809c690 con 0x7f2838159560 2023-10-06T10:43:43.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.584+0000 7f282dffb700 1 -- 172.21.15.92:0/3695853801 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283000ce00 con 0x7f2838159560 2023-10-06T10:43:43.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.584+0000 7f283fff2700 1 -- 172.21.15.92:0/3695853801 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2838159a50 con 0x7f2838159560 2023-10-06T10:43:43.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.585+0000 7f283fff2700 1 -- 172.21.15.92:0/3695853801 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2838159f50 con 0x7f2838159560 2023-10-06T10:43:43.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.585+0000 7f282dffb700 1 -- 172.21.15.92:0/3695853801 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2830014620 con 0x7f2838159560 2023-10-06T10:43:43.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.586+0000 7f282dffb700 1 --2- 172.21.15.92:0/3695853801 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f281c066f30 0x7f281c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:43.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.586+0000 7f282dffb700 1 -- 172.21.15.92:0/3695853801 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f2830084380 con 0x7f2838159560 2023-10-06T10:43:43.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.586+0000 7f283dd8e700 1 --2- 172.21.15.92:0/3695853801 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f281c066f30 0x7f281c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:43.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.586+0000 7f283dd8e700 1 --2- 172.21.15.92:0/3695853801 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f281c066f30 0x7f281c0693e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2834009260 tx=0x7f2834016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:43.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.590+0000 7f282dffb700 1 -- 172.21.15.92:0/3695853801 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f283004c5a0 con 0x7f2838159560 2023-10-06T10:43:43.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.845+0000 7f283fff2700 1 -- 172.21.15.92:0/3695853801 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2838064a90 con 0x7f2838159560 2023-10-06T10:43:43.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.845+0000 7f282dffb700 1 -- 172.21.15.92:0/3695853801 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f2830051760 con 0x7f2838159560 2023-10-06T10:43:43.847 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:43.847 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:43:43.847 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-10-06T10:43:43.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:43.847 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-10-06T10:43:43.847 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lc28gCBAAVrjhXIawHuZEox0PE3/Rsg==", 2023-10-06T10:43:43.848 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:43:43.848 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:43:43.848 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:43:43.848 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:43:43.848 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:43:43.848 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:43:43.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:43:43.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:43.848 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-10-06T10:43:43.848 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lQpdOExAAm44s0n4Jekhy8AJk/o6Waw==", 2023-10-06T10:43:43.848 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:43:43.849 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:43:43.849 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:43:43.849 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:43:43.849 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:43:43.849 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:43:43.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:43:43.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:43.849 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-10-06T10:43:43.849 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lrCrsDhAAvp++wvvnSpZRGzvRTrGWEw==", 2023-10-06T10:43:43.849 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:43:43.849 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:43:43.849 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:43:43.850 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:43:43.850 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:43:43.850 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:43:43.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:43:43.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:43.850 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-10-06T10:43:43.850 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9l/DgWGhAAy3JMA5ntn/UaRkuTCyPdkg==", 2023-10-06T10:43:43.850 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:43:43.850 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:43:43.850 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:43:43.850 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:43:43.850 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:43:43.851 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:43:43.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:43:43.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:43.851 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-10-06T10:43:43.851 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBo4x9lDEIFMBAAc3DY1fYpY2UiY7MX7qSP5A==", 2023-10-06T10:43:43.851 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:43:43.851 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:43:43.851 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:43:43.851 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:43:43.851 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:43:43.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:43:43.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:43.852 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-10-06T10:43:43.852 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBr4x9lKzX8LRAApryLtFGkK6urkE8W70l5Cg==", 2023-10-06T10:43:43.852 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:43:43.852 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:43:43.852 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:43:43.852 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:43:43.852 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:43:43.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:43:43.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:43.852 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-10-06T10:43:43.852 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBu4x9lCMZpLhAAEBPa40Wnb820p6j+lv+PsA==", 2023-10-06T10:43:43.852 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:43:43.853 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:43:43.853 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:43:43.853 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:43:43.853 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:43:43.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:43:43.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:43.853 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-10-06T10:43:43.853 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBx4x9l4AHrLRAAFn6SnNkHnOiVXmmiU8qGyA==", 2023-10-06T10:43:43.853 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:43:43.853 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:43:43.853 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:43:43.853 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:43:43.853 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:43:43.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:43:43.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:43.854 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-10-06T10:43:43.854 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB34x9l7V7WMBAAvs5kvBP5n0Th478l/idjeQ==", 2023-10-06T10:43:43.854 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:43:43.854 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:43:43.854 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:43:43.854 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:43:43.854 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:43:43.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:43:43.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:43.854 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-10-06T10:43:43.855 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB64x9lfHIlLhAADCuqt+O/zPEETgoj//hwVg==", 2023-10-06T10:43:43.855 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:43:43.855 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:43:43.855 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:43:43.855 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:43:43.855 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:43:43.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:43:43.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:43.855 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-10-06T10:43:43.855 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB94x9lUmU7LhAAvxmtGmJxrI8TlOOD7taHOQ==", 2023-10-06T10:43:43.855 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:43:43.855 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:43:43.856 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:43:43.856 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:43:43.856 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:43:43.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:43:43.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:43.856 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-10-06T10:43:43.856 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQCA4x9lJE3nLBAAnREk9yNK+FhX4fnG+Za5Ug==", 2023-10-06T10:43:43.856 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:43:43.856 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:43:43.856 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:43:43.856 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:43:43.856 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:43:43.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:43:43.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:43.857 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-10-06T10:43:43.857 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lMePcHRAAMyftXWJWk7rwmWFkq18Rqg==", 2023-10-06T10:43:43.857 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:43:43.857 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:43:43.857 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-10-06T10:43:43.857 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T10:43:43.857 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:43:43.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:43:43.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:43.857 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-10-06T10:43:43.858 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lp1s0JhAAKvZ23MYjObxOK83q3UqJBQ==", 2023-10-06T10:43:43.858 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:43:43.858 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:43:43.858 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T10:43:43.858 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T10:43:43.858 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T10:43:43.858 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:43:43.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:43:43.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:43.858 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-10-06T10:43:43.858 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lc6o9IxAAaOY7S5rYn+OdeMNWK1UYMA==", 2023-10-06T10:43:43.858 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:43:43.858 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:43:43.859 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T10:43:43.859 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T10:43:43.859 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T10:43:43.859 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:43:43.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:43:43.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:43.859 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-10-06T10:43:43.859 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lwoyOKxAA811lehuQQ8MQQQzuTFAApg==", 2023-10-06T10:43:43.859 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:43:43.859 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:43:43.859 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T10:43:43.859 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T10:43:43.860 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T10:43:43.860 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:43:43.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:43:43.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:43.860 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-10-06T10:43:43.860 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBk4x9lONCoJhAAepF/sou/5M2GBvvfNAs5wQ==", 2023-10-06T10:43:43.860 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:43:43.860 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T10:43:43.860 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:43:43.860 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:43:43.860 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:43:43.860 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:43:43.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:43:43.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:43.861 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-10-06T10:43:43.861 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9ll8wgBRAAQUrHERQ3Igv1tosa+wryIg==", 2023-10-06T10:43:43.861 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:43:43.861 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T10:43:43.861 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T10:43:43.861 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:43:43.861 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:43:43.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:43:43.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:43.861 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-10-06T10:43:43.861 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBk4x9lgYa/OhAAyoI717Ga+Wv72srZ4FK+lQ==", 2023-10-06T10:43:43.862 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:43:43.862 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T10:43:43.862 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T10:43:43.862 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:43:43.862 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:43:43.862 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:43:43.862 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T10:43:43.862 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:43:43.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.847+0000 7f283fff2700 1 -- 172.21.15.92:0/3695853801 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f281c066f30 msgr2=0x7f281c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:43.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.847+0000 7f283fff2700 1 --2- 172.21.15.92:0/3695853801 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f281c066f30 0x7f281c0693e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2834009260 tx=0x7f2834016000 comp rx=0 tx=0).stop 2023-10-06T10:43:43.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.847+0000 7f283fff2700 1 -- 172.21.15.92:0/3695853801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2838159560 msgr2=0x7f2838156990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:43.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.847+0000 7f283fff2700 1 --2- 172.21.15.92:0/3695853801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2838159560 0x7f2838156990 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f283000bb60 tx=0x7f2830009810 comp rx=0 tx=0).stop 2023-10-06T10:43:43.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.847+0000 7f283fff2700 1 -- 172.21.15.92:0/3695853801 shutdown_connections 2023-10-06T10:43:43.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.847+0000 7f283fff2700 1 --2- 172.21.15.92:0/3695853801 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f281c066f30 0x7f281c0693e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:43.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.847+0000 7f283fff2700 1 --2- 172.21.15.92:0/3695853801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2838159560 0x7f2838156990 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:43.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.847+0000 7f283fff2700 1 --2- 172.21.15.92:0/3695853801 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28380b0760 0x7f283809bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:43.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.847+0000 7f283fff2700 1 --2- 172.21.15.92:0/3695853801 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28380b49a0 0x7f283815c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:43.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.847+0000 7f283fff2700 1 -- 172.21.15.92:0/3695853801 >> 172.21.15.92:0/3695853801 conn(0x7f28380997f0 msgr2=0x7f28380694c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:43.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.847+0000 7f283fff2700 1 -- 172.21.15.92:0/3695853801 shutdown_connections 2023-10-06T10:43:43.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.847+0000 7f283fff2700 1 -- 172.21.15.92:0/3695853801 wait complete. 2023-10-06T10:43:43.865 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:43:43.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.978+0000 7f24a3442700 1 Processor -- start 2023-10-06T10:43:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.978+0000 7f24a3442700 1 -- start start 2023-10-06T10:43:43.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.979+0000 7f24a3442700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f249c159560 0x7f249c0aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:43.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.979+0000 7f24a3442700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f249c0b0c60 con 0x7f249c0b49a0 2023-10-06T10:43:43.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.979+0000 7f24a3442700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f249c0b0f40 con 0x7f249c0b0760 2023-10-06T10:43:43.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.979+0000 7f24a3442700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f249c0b1220 con 0x7f249c159560 2023-10-06T10:43:43.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.979+0000 7f24a11de700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f249c159560 0x7f249c0aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:43.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.979+0000 7f24a09dd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f249c0b49a0 0x7f249c0aaa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37536/0 (socket says 172.21.15.92:37536) 2023-10-06T10:43:43.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.979+0000 7f24a09dd700 1 -- 172.21.15.92:0/4152453690 learned_addr learned my addr 172.21.15.92:0/4152453690 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:43.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.979+0000 7f24a11de700 1 -- 172.21.15.92:0/4152453690 >> v1:172.21.15.124:6789/0 conn(0x7f249c0b0760 legacy=0x7f249c0b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:43.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.979+0000 7f24a11de700 1 -- 172.21.15.92:0/4152453690 >> v1:172.21.15.92:6789/0 conn(0x7f249c0b49a0 legacy=0x7f249c0aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:43.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.979+0000 7f24a11de700 1 -- 172.21.15.92:0/4152453690 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249c0b1500 con 0x7f249c159560 2023-10-06T10:43:43.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.979+0000 7f2493fff700 1 -- 172.21.15.92:0/4152453690 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 661303119 0 0) 0x7f249c0b0c60 con 0x7f249c0b49a0 2023-10-06T10:43:43.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.979+0000 7f24a11de700 1 --2- 172.21.15.92:0/4152453690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f249c159560 0x7f249c0aa530 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f2494009900 tx=0x7f2494009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51725d9b90f6fb97 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:43.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.979+0000 7f2493fff700 1 -- 172.21.15.92:0/4152453690 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2494009ea0 con 0x7f249c159560 2023-10-06T10:43:43.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.980+0000 7f2493fff700 1 -- 172.21.15.92:0/4152453690 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2494005c10 con 0x7f249c159560 2023-10-06T10:43:43.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.980+0000 7f2493fff700 1 -- 172.21.15.92:0/4152453690 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249400b5d0 con 0x7f249c159560 2023-10-06T10:43:43.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.980+0000 7f24a3442700 1 -- 172.21.15.92:0/4152453690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f249c159560 msgr2=0x7f249c0aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:43.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.980+0000 7f24a3442700 1 --2- 172.21.15.92:0/4152453690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f249c159560 0x7f249c0aa530 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f2494009900 tx=0x7f2494009120 comp rx=0 tx=0).stop 2023-10-06T10:43:43.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.980+0000 7f24a3442700 1 -- 172.21.15.92:0/4152453690 shutdown_connections 2023-10-06T10:43:43.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.980+0000 7f24a3442700 1 --2- 172.21.15.92:0/4152453690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f249c159560 0x7f249c0aa530 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:43.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.980+0000 7f24a3442700 1 -- 172.21.15.92:0/4152453690 >> 172.21.15.92:0/4152453690 conn(0x7f249c0997f0 msgr2=0x7f249c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:43.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.980+0000 7f24a3442700 1 -- 172.21.15.92:0/4152453690 shutdown_connections 2023-10-06T10:43:43.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.980+0000 7f24a3442700 1 -- 172.21.15.92:0/4152453690 wait complete. 2023-10-06T10:43:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.981+0000 7f24a3442700 1 Processor -- start 2023-10-06T10:43:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.981+0000 7f24a3442700 1 -- start start 2023-10-06T10:43:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.982+0000 7f24a3442700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f249c0b0760 0x7f249c0ac020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.982+0000 7f24a3442700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f249c0b49a0 0x7f249c09a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.982+0000 7f24a3442700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f249c159560 0x7f249c0aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.982+0000 7f24a3442700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c0ac900 con 0x7f249c0b49a0 2023-10-06T10:43:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.982+0000 7f24a3442700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c0ae2f0 con 0x7f249c159560 2023-10-06T10:43:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.982+0000 7f24a3442700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f249c0ae5d0 con 0x7f249c0b0760 2023-10-06T10:43:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.982+0000 7f24a09dd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f249c0b49a0 0x7f249c09a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.982+0000 7f24a09dd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f249c0b49a0 0x7f249c09a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44800/0 (socket says 172.21.15.92:44800) 2023-10-06T10:43:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.982+0000 7f24a09dd700 1 -- 172.21.15.92:0/665684157 learned_addr learned my addr 172.21.15.92:0/665684157 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.982+0000 7f24a11de700 1 --2- 172.21.15.92:0/665684157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f249c0b0760 0x7f249c0ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.982+0000 7f24a19df700 1 --2- 172.21.15.92:0/665684157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f249c159560 0x7f249c0aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.983+0000 7f24a09dd700 1 -- 172.21.15.92:0/665684157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f249c0b0760 msgr2=0x7f249c0ac020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.983+0000 7f24a09dd700 1 --2- 172.21.15.92:0/665684157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f249c0b0760 0x7f249c0ac020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.983+0000 7f24a09dd700 1 -- 172.21.15.92:0/665684157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f249c159560 msgr2=0x7f249c0aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.983+0000 7f24a09dd700 1 --2- 172.21.15.92:0/665684157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f249c159560 0x7f249c0aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.983+0000 7f24a09dd700 1 -- 172.21.15.92:0/665684157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24940093c0 con 0x7f249c0b49a0 2023-10-06T10:43:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.983+0000 7f24a09dd700 1 --2- 172.21.15.92:0/665684157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f249c0b49a0 0x7f249c09a050 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f248c00c7e0 tx=0x7f248c007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.983+0000 7f24917fa700 1 -- 172.21.15.92:0/665684157 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c005c00 con 0x7f249c0b49a0 2023-10-06T10:43:43.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.983+0000 7f24917fa700 1 -- 172.21.15.92:0/665684157 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248c005d60 con 0x7f249c0b49a0 2023-10-06T10:43:43.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.983+0000 7f24a3442700 1 -- 172.21.15.92:0/665684157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f249c0ae8b0 con 0x7f249c0b49a0 2023-10-06T10:43:43.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.983+0000 7f24917fa700 1 -- 172.21.15.92:0/665684157 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c0072e0 con 0x7f249c0b49a0 2023-10-06T10:43:43.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.983+0000 7f24a3442700 1 -- 172.21.15.92:0/665684157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249c0adfa0 con 0x7f249c0b49a0 2023-10-06T10:43:43.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.984+0000 7f24a3442700 1 -- 172.21.15.92:0/665684157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f249c0aeaa0 con 0x7f249c0b49a0 2023-10-06T10:43:43.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.984+0000 7f24917fa700 1 -- 172.21.15.92:0/665684157 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f248c01b3b0 con 0x7f249c0b49a0 2023-10-06T10:43:43.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.985+0000 7f24917fa700 1 --2- 172.21.15.92:0/665684157 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2480066f30 0x7f24800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:43.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.985+0000 7f24917fa700 1 -- 172.21.15.92:0/665684157 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f248c083970 con 0x7f249c0b49a0 2023-10-06T10:43:43.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.985+0000 7f24a11de700 1 --2- 172.21.15.92:0/665684157 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2480066f30 0x7f24800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:43.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.985+0000 7f24a11de700 1 --2- 172.21.15.92:0/665684157 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2480066f30 0x7f24800693e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2494003080 tx=0x7f2494015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:43.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:43.989+0000 7f24917fa700 1 -- 172.21.15.92:0/665684157 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f248c04b5a0 con 0x7f249c0b49a0 2023-10-06T10:43:44.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.174+0000 7f24a3442700 1 -- 172.21.15.92:0/665684157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f249c064a90 con 0x7f249c0b49a0 2023-10-06T10:43:44.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.175+0000 7f24917fa700 1 -- 172.21.15.92:0/665684157 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+13856 (secure 0 0 0) 0x7f248c017090 con 0x7f249c0b49a0 2023-10-06T10:43:44.177 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:44.177 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":86,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:43:43.446862+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:43:44.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.177+0000 7f24a3442700 1 -- 172.21.15.92:0/665684157 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2480066f30 msgr2=0x7f24800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:44.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.177+0000 7f24a3442700 1 --2- 172.21.15.92:0/665684157 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2480066f30 0x7f24800693e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2494003080 tx=0x7f2494015000 comp rx=0 tx=0).stop 2023-10-06T10:43:44.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.177+0000 7f24a3442700 1 -- 172.21.15.92:0/665684157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f249c0b49a0 msgr2=0x7f249c09a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:44.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.177+0000 7f24a3442700 1 --2- 172.21.15.92:0/665684157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f249c0b49a0 0x7f249c09a050 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f248c00c7e0 tx=0x7f248c007d90 comp rx=0 tx=0).stop 2023-10-06T10:43:44.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.177+0000 7f24a3442700 1 -- 172.21.15.92:0/665684157 shutdown_connections 2023-10-06T10:43:44.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.177+0000 7f24a3442700 1 --2- 172.21.15.92:0/665684157 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2480066f30 0x7f24800693e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.177+0000 7f24a3442700 1 --2- 172.21.15.92:0/665684157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f249c159560 0x7f249c0aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.177+0000 7f24a3442700 1 --2- 172.21.15.92:0/665684157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f249c0b0760 0x7f249c0ac020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.177+0000 7f24a3442700 1 --2- 172.21.15.92:0/665684157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f249c0b49a0 0x7f249c09a050 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.177+0000 7f24a3442700 1 -- 172.21.15.92:0/665684157 >> 172.21.15.92:0/665684157 conn(0x7f249c0997f0 msgr2=0x7f249c0b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:44.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.177+0000 7f24a3442700 1 -- 172.21.15.92:0/665684157 shutdown_connections 2023-10-06T10:43:44.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.177+0000 7f24a3442700 1 -- 172.21.15.92:0/665684157 wait complete. 2023-10-06T10:43:44.191 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T10:38:09.680559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T10:38:10.120786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T10:43:44.191 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T10:43:44.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.306+0000 7f114afe0700 1 Processor -- start 2023-10-06T10:43:44.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.306+0000 7f114afe0700 1 -- start start 2023-10-06T10:43:44.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.307+0000 7f114afe0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11441598a0 0x7f11440ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:44.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.307+0000 7f114afe0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11440cbc70 0x7f11440ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:44.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.307+0000 7f114afe0700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11440cf5d0 con 0x7f11441594d0 2023-10-06T10:43:44.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.307+0000 7f114afe0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11440cf8b0 con 0x7f11440cbc70 2023-10-06T10:43:44.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.307+0000 7f114afe0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11440cfb90 con 0x7f11441598a0 2023-10-06T10:43:44.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.307+0000 7f114957d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11440cbc70 0x7f11440ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:44.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.307+0000 7f114957d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11440cbc70 0x7f11440ced20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44804/0 (socket says 172.21.15.92:44804) 2023-10-06T10:43:44.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.307+0000 7f114957d700 1 -- 172.21.15.92:0/1316128628 learned_addr learned my addr 172.21.15.92:0/1316128628 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:44.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.307+0000 7f113bfff700 1 --2- 172.21.15.92:0/1316128628 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11441598a0 0x7f11440ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:44.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.307+0000 7f114957d700 1 -- 172.21.15.92:0/1316128628 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11441598a0 msgr2=0x7f11440ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:44.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.307+0000 7f114957d700 1 --2- 172.21.15.92:0/1316128628 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11441598a0 0x7f11440ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.307+0000 7f114957d700 1 -- 172.21.15.92:0/1316128628 >> v1:172.21.15.124:6789/0 conn(0x7f11441594d0 legacy=0x7f11440cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:44.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.307+0000 7f114957d700 1 -- 172.21.15.92:0/1316128628 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11440cfe70 con 0x7f11440cbc70 2023-10-06T10:43:44.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.307+0000 7f113bfff700 1 --2- 172.21.15.92:0/1316128628 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11441598a0 0x7f11440ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:44.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.308+0000 7f114957d700 1 --2- 172.21.15.92:0/1316128628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11440cbc70 0x7f11440ced20 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f113c009f20 tx=0x7f113c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac2e0e1956914d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:44.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.308+0000 7f113b7fe700 1 -- 172.21.15.92:0/1316128628 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c00e070 con 0x7f11440cbc70 2023-10-06T10:43:44.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.308+0000 7f113b7fe700 1 -- 172.21.15.92:0/1316128628 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113c00ba80 con 0x7f11440cbc70 2023-10-06T10:43:44.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.308+0000 7f113b7fe700 1 -- 172.21.15.92:0/1316128628 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c0092e0 con 0x7f11440cbc70 2023-10-06T10:43:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.308+0000 7f114afe0700 1 -- 172.21.15.92:0/1316128628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11440cbc70 msgr2=0x7f11440ced20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.308+0000 7f114afe0700 1 --2- 172.21.15.92:0/1316128628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11440cbc70 0x7f11440ced20 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f113c009f20 tx=0x7f113c009840 comp rx=0 tx=0).stop 2023-10-06T10:43:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.308+0000 7f114afe0700 1 -- 172.21.15.92:0/1316128628 shutdown_connections 2023-10-06T10:43:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.308+0000 7f114afe0700 1 --2- 172.21.15.92:0/1316128628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11440cbc70 0x7f11440ced20 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.308+0000 7f114afe0700 1 --2- 172.21.15.92:0/1316128628 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11441598a0 0x7f11440ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.308+0000 7f114afe0700 1 -- 172.21.15.92:0/1316128628 >> 172.21.15.92:0/1316128628 conn(0x7f1144089e50 msgr2=0x7f1144099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.308+0000 7f114afe0700 1 -- 172.21.15.92:0/1316128628 shutdown_connections 2023-10-06T10:43:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.308+0000 7f114afe0700 1 -- 172.21.15.92:0/1316128628 wait complete. 2023-10-06T10:43:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.309+0000 7f114afe0700 1 Processor -- start 2023-10-06T10:43:44.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.310+0000 7f114afe0700 1 -- start start 2023-10-06T10:43:44.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.310+0000 7f114afe0700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11440cbc70 0x7f114414f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:44.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.310+0000 7f114afe0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11441594d0 0x7f11440b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:44.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.310+0000 7f114afe0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11441598a0 0x7f11440c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:44.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.310+0000 7f114afe0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1144149910 con 0x7f11441594d0 2023-10-06T10:43:44.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.310+0000 7f114afe0700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1144151ad0 con 0x7f11440cbc70 2023-10-06T10:43:44.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.310+0000 7f114afe0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1144150ec0 con 0x7f11441598a0 2023-10-06T10:43:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.310+0000 7f113bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11441594d0 0x7f11440b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.310+0000 7f113bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11441594d0 0x7f11440b4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44810/0 (socket says 172.21.15.92:44810) 2023-10-06T10:43:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.310+0000 7f113bfff700 1 -- 172.21.15.92:0/926794941 learned_addr learned my addr 172.21.15.92:0/926794941 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.311+0000 7f1148d7c700 1 --2- 172.21.15.92:0/926794941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11440cbc70 0x7f114414f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.311+0000 7f114957d700 1 --2- 172.21.15.92:0/926794941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11441598a0 0x7f11440c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.311+0000 7f113bfff700 1 -- 172.21.15.92:0/926794941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11441598a0 msgr2=0x7f11440c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.311+0000 7f113bfff700 1 --2- 172.21.15.92:0/926794941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11441598a0 0x7f11440c5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.311+0000 7f113bfff700 1 -- 172.21.15.92:0/926794941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11440cbc70 msgr2=0x7f114414f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.311+0000 7f113bfff700 1 --2- 172.21.15.92:0/926794941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11440cbc70 0x7f114414f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.311+0000 7f113bfff700 1 -- 172.21.15.92:0/926794941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113c009ae0 con 0x7f11441594d0 2023-10-06T10:43:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.311+0000 7f1148d7c700 1 --2- 172.21.15.92:0/926794941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11440cbc70 0x7f114414f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.311+0000 7f113bfff700 1 --2- 172.21.15.92:0/926794941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11441594d0 0x7f11440b4ce0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f113400d890 tx=0x7f1134008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.311+0000 7f1138ff9700 1 -- 172.21.15.92:0/926794941 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11340125f0 con 0x7f11441594d0 2023-10-06T10:43:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.311+0000 7f1138ff9700 1 -- 172.21.15.92:0/926794941 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1134012750 con 0x7f11441594d0 2023-10-06T10:43:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.311+0000 7f114afe0700 1 -- 172.21.15.92:0/926794941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11441511a0 con 0x7f11441594d0 2023-10-06T10:43:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.311+0000 7f1138ff9700 1 -- 172.21.15.92:0/926794941 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134012af0 con 0x7f11441594d0 2023-10-06T10:43:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.311+0000 7f114afe0700 1 -- 172.21.15.92:0/926794941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1144151760 con 0x7f11441594d0 2023-10-06T10:43:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.312+0000 7f114afe0700 1 -- 172.21.15.92:0/926794941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1144064a90 con 0x7f11441594d0 2023-10-06T10:43:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.312+0000 7f1138ff9700 1 -- 172.21.15.92:0/926794941 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1134014430 con 0x7f11441594d0 2023-10-06T10:43:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.313+0000 7f1138ff9700 1 --2- 172.21.15.92:0/926794941 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1130066f30 0x7f11300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.313+0000 7f1138ff9700 1 -- 172.21.15.92:0/926794941 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f1134084880 con 0x7f11441594d0 2023-10-06T10:43:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.313+0000 7f1148d7c700 1 --2- 172.21.15.92:0/926794941 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1130066f30 0x7f11300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:44.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.313+0000 7f1148d7c700 1 --2- 172.21.15.92:0/926794941 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1130066f30 0x7f11300693e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1140007fd0 tx=0x7f1140009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:44.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.317+0000 7f1138ff9700 1 -- 172.21.15.92:0/926794941 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1134051ba0 con 0x7f11441594d0 2023-10-06T10:43:44.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.499+0000 7f114afe0700 1 -- 172.21.15.92:0/926794941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f114404bb40 con 0x7f11441594d0 2023-10-06T10:43:44.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.499+0000 7f1138ff9700 1 -- 172.21.15.92:0/926794941 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v86) v1 ==== 92+0+10 (secure 0 0 0) 0x7f11340551c0 con 0x7f11441594d0 2023-10-06T10:43:44.501 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T10:43:44.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.501+0000 7f114afe0700 1 -- 172.21.15.92:0/926794941 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1130066f30 msgr2=0x7f11300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:44.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.501+0000 7f114afe0700 1 --2- 172.21.15.92:0/926794941 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1130066f30 0x7f11300693e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1140007fd0 tx=0x7f1140009040 comp rx=0 tx=0).stop 2023-10-06T10:43:44.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.501+0000 7f114afe0700 1 -- 172.21.15.92:0/926794941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11441594d0 msgr2=0x7f11440b4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:44.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.501+0000 7f114afe0700 1 --2- 172.21.15.92:0/926794941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11441594d0 0x7f11440b4ce0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f113400d890 tx=0x7f1134008040 comp rx=0 tx=0).stop 2023-10-06T10:43:44.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.501+0000 7f114afe0700 1 -- 172.21.15.92:0/926794941 shutdown_connections 2023-10-06T10:43:44.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.501+0000 7f114afe0700 1 --2- 172.21.15.92:0/926794941 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1130066f30 0x7f11300693e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.501+0000 7f114afe0700 1 --2- 172.21.15.92:0/926794941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11441598a0 0x7f11440c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.501+0000 7f114afe0700 1 --2- 172.21.15.92:0/926794941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11440cbc70 0x7f114414f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.501+0000 7f114afe0700 1 --2- 172.21.15.92:0/926794941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11441594d0 0x7f11440b4ce0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.501+0000 7f114afe0700 1 -- 172.21.15.92:0/926794941 >> 172.21.15.92:0/926794941 conn(0x7f1144089e50 msgr2=0x7f1144068400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:44.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.501+0000 7f114afe0700 1 -- 172.21.15.92:0/926794941 shutdown_connections 2023-10-06T10:43:44.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.501+0000 7f114afe0700 1 -- 172.21.15.92:0/926794941 wait complete. 2023-10-06T10:43:44.515 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T10:43:44.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.629+0000 7f043dbe4700 1 Processor -- start 2023-10-06T10:43:44.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.629+0000 7f043dbe4700 1 -- start start 2023-10-06T10:43:44.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.630+0000 7f043dbe4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04381598a0 0x7f04380ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:44.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.630+0000 7f043dbe4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04380cf5d0 con 0x7f04380cbc70 2023-10-06T10:43:44.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.630+0000 7f043dbe4700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04380cf8b0 con 0x7f04381594d0 2023-10-06T10:43:44.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.630+0000 7f043dbe4700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04380cfb90 con 0x7f04381598a0 2023-10-06T10:43:44.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.630+0000 7f0437fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f04380cbc70 0x7f04380ced20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37542/0 (socket says 172.21.15.92:37542) 2023-10-06T10:43:44.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.630+0000 7f0437fff700 1 -- 172.21.15.92:0/3270401269 learned_addr learned my addr 172.21.15.92:0/3270401269 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:44.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.630+0000 7f0436ffd700 1 --2- 172.21.15.92:0/3270401269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04381598a0 0x7f04380ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:44.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.630+0000 7f04367fc700 1 -- 172.21.15.92:0/3270401269 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378487342 0 0) 0x7f04380cf5d0 con 0x7f04380cbc70 2023-10-06T10:43:44.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.630+0000 7f04367fc700 1 -- 172.21.15.92:0/3270401269 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0420003570 con 0x7f04380cbc70 2023-10-06T10:43:44.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.631+0000 7f04367fc700 1 -- 172.21.15.92:0/3270401269 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347225156 0 0) 0x7f04380cf8b0 con 0x7f04381594d0 2023-10-06T10:43:44.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.631+0000 7f04367fc700 1 -- 172.21.15.92:0/3270401269 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04380cf5d0 con 0x7f04381594d0 2023-10-06T10:43:44.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.631+0000 7f0436ffd700 1 -- 172.21.15.92:0/3270401269 >> v1:172.21.15.124:6789/0 conn(0x7f04381594d0 legacy=0x7f04380cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:44.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.631+0000 7f04367fc700 1 -- 172.21.15.92:0/3270401269 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3300005327 0 0) 0x7f0420003570 con 0x7f04380cbc70 2023-10-06T10:43:44.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.631+0000 7f0436ffd700 1 -- 172.21.15.92:0/3270401269 >> v1:172.21.15.92:6789/0 conn(0x7f04380cbc70 legacy=0x7f04380ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:44.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.631+0000 7f0436ffd700 1 -- 172.21.15.92:0/3270401269 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04380cfe70 con 0x7f04381598a0 2023-10-06T10:43:44.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.631+0000 7f0436ffd700 1 --2- 172.21.15.92:0/3270401269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04381598a0 0x7f04380ce7e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f042c009910 tx=0x7f042c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=346b367a03e176e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:44.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.631+0000 7f04367fc700 1 -- 172.21.15.92:0/3270401269 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0430001470 con 0x7f04381598a0 2023-10-06T10:43:44.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.631+0000 7f04367fc700 1 -- 172.21.15.92:0/3270401269 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042c011420 con 0x7f04381598a0 2023-10-06T10:43:44.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.631+0000 7f04367fc700 1 -- 172.21.15.92:0/3270401269 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c0117a0 con 0x7f04381598a0 2023-10-06T10:43:44.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.632+0000 7f043dbe4700 1 -- 172.21.15.92:0/3270401269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04381598a0 msgr2=0x7f04380ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:44.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.632+0000 7f043dbe4700 1 --2- 172.21.15.92:0/3270401269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04381598a0 0x7f04380ce7e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f042c009910 tx=0x7f042c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.632+0000 7f043dbe4700 1 -- 172.21.15.92:0/3270401269 shutdown_connections 2023-10-06T10:43:44.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.632+0000 7f043dbe4700 1 --2- 172.21.15.92:0/3270401269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04381598a0 0x7f04380ce7e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.632+0000 7f043dbe4700 1 -- 172.21.15.92:0/3270401269 >> 172.21.15.92:0/3270401269 conn(0x7f0438089e50 msgr2=0x7f0438099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:44.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.632+0000 7f043dbe4700 1 -- 172.21.15.92:0/3270401269 shutdown_connections 2023-10-06T10:43:44.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.632+0000 7f043dbe4700 1 -- 172.21.15.92:0/3270401269 wait complete. 2023-10-06T10:43:44.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.633+0000 7f043dbe4700 1 Processor -- start 2023-10-06T10:43:44.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.633+0000 7f043dbe4700 1 -- start start 2023-10-06T10:43:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.634+0000 7f043dbe4700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04380cbc70 0x7f043814ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.634+0000 7f043dbe4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04381594d0 0x7f043816d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.634+0000 7f043dbe4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04381598a0 0x7f043816d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.634+0000 7f043dbe4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0438149880 con 0x7f04381594d0 2023-10-06T10:43:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.634+0000 7f043dbe4700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043814f860 con 0x7f04380cbc70 2023-10-06T10:43:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.634+0000 7f043dbe4700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f043814efa0 con 0x7f04381598a0 2023-10-06T10:43:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.634+0000 7f0436ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04381594d0 0x7f043816d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.634+0000 7f0436ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04381594d0 0x7f043816d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44816/0 (socket says 172.21.15.92:44816) 2023-10-06T10:43:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.634+0000 7f0436ffd700 1 -- 172.21.15.92:0/3226651457 learned_addr learned my addr 172.21.15.92:0/3226651457 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.634+0000 7f04377fe700 1 --2- 172.21.15.92:0/3226651457 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04380cbc70 0x7f043814ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.634+0000 7f0437fff700 1 --2- 172.21.15.92:0/3226651457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04381598a0 0x7f043816d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.634+0000 7f0436ffd700 1 -- 172.21.15.92:0/3226651457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04381598a0 msgr2=0x7f043816d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.634+0000 7f0436ffd700 1 --2- 172.21.15.92:0/3226651457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04381598a0 0x7f043816d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.634+0000 7f0436ffd700 1 -- 172.21.15.92:0/3226651457 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04380cbc70 msgr2=0x7f043814ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.634+0000 7f0436ffd700 1 --2- 172.21.15.92:0/3226651457 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04380cbc70 0x7f043814ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.634+0000 7f0436ffd700 1 -- 172.21.15.92:0/3226651457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042c009420 con 0x7f04381594d0 2023-10-06T10:43:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.634+0000 7f0436ffd700 1 --2- 172.21.15.92:0/3226651457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04381594d0 0x7f043816d460 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f042c009860 tx=0x7f042c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.634+0000 7f041f7fe700 1 -- 172.21.15.92:0/3226651457 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c01cc50 con 0x7f04381594d0 2023-10-06T10:43:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.634+0000 7f041f7fe700 1 -- 172.21.15.92:0/3226651457 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042c01cdb0 con 0x7f04381594d0 2023-10-06T10:43:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.634+0000 7f043dbe4700 1 -- 172.21.15.92:0/3226651457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043814cf60 con 0x7f04381594d0 2023-10-06T10:43:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.634+0000 7f041f7fe700 1 -- 172.21.15.92:0/3226651457 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c01a6a0 con 0x7f04381594d0 2023-10-06T10:43:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.634+0000 7f043dbe4700 1 -- 172.21.15.92:0/3226651457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043814f210 con 0x7f04381594d0 2023-10-06T10:43:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.635+0000 7f043dbe4700 1 -- 172.21.15.92:0/3226651457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0438064a90 con 0x7f04381594d0 2023-10-06T10:43:44.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.636+0000 7f041f7fe700 1 -- 172.21.15.92:0/3226651457 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f042c01aad0 con 0x7f04381594d0 2023-10-06T10:43:44.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.636+0000 7f041f7fe700 1 --2- 172.21.15.92:0/3226651457 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0424066f30 0x7f04240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:44.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.636+0000 7f041f7fe700 1 -- 172.21.15.92:0/3226651457 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f042c085db0 con 0x7f04381594d0 2023-10-06T10:43:44.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.636+0000 7f04377fe700 1 --2- 172.21.15.92:0/3226651457 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0424066f30 0x7f04240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:44.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.637+0000 7f04377fe700 1 --2- 172.21.15.92:0/3226651457 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0424066f30 0x7f04240693e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0428003f70 tx=0x7f0428009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:44.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.640+0000 7f041f7fe700 1 -- 172.21.15.92:0/3226651457 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f042c0530d0 con 0x7f04381594d0 2023-10-06T10:43:44.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.822+0000 7f043dbe4700 1 -- 172.21.15.92:0/3226651457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f04380c66a0 con 0x7f04381594d0 2023-10-06T10:43:44.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.822+0000 7f041f7fe700 1 -- 172.21.15.92:0/3226651457 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f042c0566f0 con 0x7f04381594d0 2023-10-06T10:43:44.824 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T10:43:44.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.824+0000 7f043dbe4700 1 -- 172.21.15.92:0/3226651457 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0424066f30 msgr2=0x7f04240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:44.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.824+0000 7f043dbe4700 1 --2- 172.21.15.92:0/3226651457 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0424066f30 0x7f04240693e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0428003f70 tx=0x7f0428009040 comp rx=0 tx=0).stop 2023-10-06T10:43:44.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.824+0000 7f043dbe4700 1 -- 172.21.15.92:0/3226651457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04381594d0 msgr2=0x7f043816d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:44.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.824+0000 7f043dbe4700 1 --2- 172.21.15.92:0/3226651457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04381594d0 0x7f043816d460 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f042c009860 tx=0x7f042c016000 comp rx=0 tx=0).stop 2023-10-06T10:43:44.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.824+0000 7f043dbe4700 1 -- 172.21.15.92:0/3226651457 shutdown_connections 2023-10-06T10:43:44.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.824+0000 7f043dbe4700 1 --2- 172.21.15.92:0/3226651457 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0424066f30 0x7f04240693e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.824+0000 7f043dbe4700 1 --2- 172.21.15.92:0/3226651457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04381598a0 0x7f043816d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.824+0000 7f043dbe4700 1 --2- 172.21.15.92:0/3226651457 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04380cbc70 0x7f043814ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.824+0000 7f043dbe4700 1 --2- 172.21.15.92:0/3226651457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04381594d0 0x7f043816d460 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.824+0000 7f043dbe4700 1 -- 172.21.15.92:0/3226651457 >> 172.21.15.92:0/3226651457 conn(0x7f0438089e50 msgr2=0x7f043815d870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:44.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.824+0000 7f043dbe4700 1 -- 172.21.15.92:0/3226651457 shutdown_connections 2023-10-06T10:43:44.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.824+0000 7f043dbe4700 1 -- 172.21.15.92:0/3226651457 wait complete. 2023-10-06T10:43:44.838 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T10:43:44.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.951+0000 7fce5b34a700 1 Processor -- start 2023-10-06T10:43:44.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.951+0000 7fce5b34a700 1 -- start start 2023-10-06T10:43:44.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.952+0000 7fce5b34a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce54159440 0x7fce540b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:44.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.952+0000 7fce5b34a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce540b35d0 0x7fce540cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:44.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.952+0000 7fce5b34a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce540cb7a0 con 0x7fce540cc170 2023-10-06T10:43:44.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.952+0000 7fce5b34a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce540b0860 con 0x7fce54159440 2023-10-06T10:43:44.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.952+0000 7fce5b34a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce540cb150 con 0x7fce540b35d0 2023-10-06T10:43:44.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.952+0000 7fce5a348700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce54159440 0x7fce540b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:44.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.952+0000 7fce5a348700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce54159440 0x7fce540b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44828/0 (socket says 172.21.15.92:44828) 2023-10-06T10:43:44.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.952+0000 7fce5ab49700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fce540cc170 0x7fce540b2100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37552/0 (socket says 172.21.15.92:37552) 2023-10-06T10:43:44.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.952+0000 7fce5a348700 1 -- 172.21.15.92:0/3805966715 learned_addr learned my addr 172.21.15.92:0/3805966715 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:44.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.952+0000 7fce59b47700 1 --2- 172.21.15.92:0/3805966715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce540b35d0 0x7fce540cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:44.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.952+0000 7fce5a348700 1 -- 172.21.15.92:0/3805966715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce540b35d0 msgr2=0x7fce540cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:44.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.952+0000 7fce5a348700 1 --2- 172.21.15.92:0/3805966715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce540b35d0 0x7fce540cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.952+0000 7fce5a348700 1 -- 172.21.15.92:0/3805966715 >> v1:172.21.15.92:6789/0 conn(0x7fce540cc170 legacy=0x7fce540b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:44.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.952+0000 7fce5a348700 1 -- 172.21.15.92:0/3805966715 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce540cb430 con 0x7fce54159440 2023-10-06T10:43:44.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.952+0000 7fce59346700 1 -- 172.21.15.92:0/3805966715 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404450626 0 0) 0x7fce540cb7a0 con 0x7fce540cc170 2023-10-06T10:43:44.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.953+0000 7fce5a348700 1 --2- 172.21.15.92:0/3805966715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce54159440 0x7fce540b3090 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fce500099b0 tx=0x7fce50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3deda5d1828ce5e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:44.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.953+0000 7fce59346700 1 -- 172.21.15.92:0/3805966715 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5000b860 con 0x7fce54159440 2023-10-06T10:43:44.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.953+0000 7fce59346700 1 -- 172.21.15.92:0/3805966715 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5000b9c0 con 0x7fce54159440 2023-10-06T10:43:44.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.953+0000 7fce59346700 1 -- 172.21.15.92:0/3805966715 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5000bd60 con 0x7fce54159440 2023-10-06T10:43:44.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.953+0000 7fce5b34a700 1 -- 172.21.15.92:0/3805966715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce54159440 msgr2=0x7fce540b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:44.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.953+0000 7fce5b34a700 1 --2- 172.21.15.92:0/3805966715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce54159440 0x7fce540b3090 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fce500099b0 tx=0x7fce50009250 comp rx=0 tx=0).stop 2023-10-06T10:43:44.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.953+0000 7fce5b34a700 1 -- 172.21.15.92:0/3805966715 shutdown_connections 2023-10-06T10:43:44.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.953+0000 7fce5b34a700 1 --2- 172.21.15.92:0/3805966715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce54159440 0x7fce540b3090 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.953+0000 7fce5b34a700 1 --2- 172.21.15.92:0/3805966715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce540b35d0 0x7fce540cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.953+0000 7fce5b34a700 1 -- 172.21.15.92:0/3805966715 >> 172.21.15.92:0/3805966715 conn(0x7fce5406ece0 msgr2=0x7fce5406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:44.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.953+0000 7fce5b34a700 1 -- 172.21.15.92:0/3805966715 shutdown_connections 2023-10-06T10:43:44.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.953+0000 7fce5b34a700 1 -- 172.21.15.92:0/3805966715 wait complete. 2023-10-06T10:43:44.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.954+0000 7fce5b34a700 1 Processor -- start 2023-10-06T10:43:44.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.955+0000 7fce5b34a700 1 -- start start 2023-10-06T10:43:44.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.955+0000 7fce5b34a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce540b35d0 0x7fce54154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:44.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.955+0000 7fce5b34a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce540cc170 0x7fce540d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:44.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.955+0000 7fce5b34a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce54159440 0x7fce540d0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:44.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.955+0000 7fce5b34a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce540c8780 con 0x7fce540b35d0 2023-10-06T10:43:44.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.955+0000 7fce5b34a700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce540c7e50 con 0x7fce54159440 2023-10-06T10:43:44.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.955+0000 7fce5b34a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce540c8130 con 0x7fce540cc170 2023-10-06T10:43:44.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.955+0000 7fce5a348700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce540b35d0 0x7fce54154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:44.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.955+0000 7fce5a348700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce540b35d0 0x7fce54154940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44842/0 (socket says 172.21.15.92:44842) 2023-10-06T10:43:44.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.955+0000 7fce5a348700 1 -- 172.21.15.92:0/1849975671 learned_addr learned my addr 172.21.15.92:0/1849975671 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:44.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.955+0000 7fce5ab49700 1 --2- 172.21.15.92:0/1849975671 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce54159440 0x7fce540d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:44.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.955+0000 7fce59b47700 1 --2- 172.21.15.92:0/1849975671 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce540cc170 0x7fce540d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:44.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.956+0000 7fce5a348700 1 -- 172.21.15.92:0/1849975671 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce540cc170 msgr2=0x7fce540d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:44.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.956+0000 7fce5a348700 1 --2- 172.21.15.92:0/1849975671 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce540cc170 0x7fce540d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.956+0000 7fce5a348700 1 -- 172.21.15.92:0/1849975671 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce54159440 msgr2=0x7fce540d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:44.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.956+0000 7fce5a348700 1 --2- 172.21.15.92:0/1849975671 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce54159440 0x7fce540d0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:44.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.956+0000 7fce5a348700 1 -- 172.21.15.92:0/1849975671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce500094f0 con 0x7fce540b35d0 2023-10-06T10:43:44.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.956+0000 7fce5a348700 1 --2- 172.21.15.92:0/1849975671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce540b35d0 0x7fce54154940 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fce54069150 tx=0x7fce50017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:44.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.956+0000 7fce4a7fc700 1 -- 172.21.15.92:0/1849975671 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50011d70 con 0x7fce540b35d0 2023-10-06T10:43:44.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.956+0000 7fce4a7fc700 1 -- 172.21.15.92:0/1849975671 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce50024440 con 0x7fce540b35d0 2023-10-06T10:43:44.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.956+0000 7fce4a7fc700 1 -- 172.21.15.92:0/1849975671 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50024800 con 0x7fce540b35d0 2023-10-06T10:43:44.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.956+0000 7fce5b34a700 1 -- 172.21.15.92:0/1849975671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce540c8410 con 0x7fce540b35d0 2023-10-06T10:43:44.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.956+0000 7fce5b34a700 1 -- 172.21.15.92:0/1849975671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce540c7b00 con 0x7fce540b35d0 2023-10-06T10:43:44.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.957+0000 7fce5b34a700 1 -- 172.21.15.92:0/1849975671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5405f230 con 0x7fce540b35d0 2023-10-06T10:43:44.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.957+0000 7fce4a7fc700 1 -- 172.21.15.92:0/1849975671 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fce5001a420 con 0x7fce540b35d0 2023-10-06T10:43:44.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.958+0000 7fce4a7fc700 1 --2- 172.21.15.92:0/1849975671 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fce38066f30 0x7fce380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:44.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.958+0000 7fce4a7fc700 1 -- 172.21.15.92:0/1849975671 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fce50086360 con 0x7fce540b35d0 2023-10-06T10:43:44.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.958+0000 7fce59b47700 1 --2- 172.21.15.92:0/1849975671 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fce38066f30 0x7fce380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:44.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.958+0000 7fce59b47700 1 --2- 172.21.15.92:0/1849975671 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fce38066f30 0x7fce380693e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fce540ca7c0 tx=0x7fce44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:44.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:44.962+0000 7fce4a7fc700 1 -- 172.21.15.92:0/1849975671 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce50053660 con 0x7fce540b35d0 2023-10-06T10:43:45.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.147+0000 7fce5b34a700 1 -- 172.21.15.92:0/1849975671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fce54159810 con 0x7fce540b35d0 2023-10-06T10:43:45.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.148+0000 7fce4a7fc700 1 -- 172.21.15.92:0/1849975671 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v86) v1 ==== 84+0+126 (secure 0 0 0) 0x7fce50056c80 con 0x7fce540b35d0 2023-10-06T10:43:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:45.150 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T10:43:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-10-06T10:43:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-10-06T10:43:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:43:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:43:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-10-06T10:43:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-10-06T10:43:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:43:45.151 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T10:43:45.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.150+0000 7fce5b34a700 1 -- 172.21.15.92:0/1849975671 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fce38066f30 msgr2=0x7fce380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:45.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.150+0000 7fce5b34a700 1 --2- 172.21.15.92:0/1849975671 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fce38066f30 0x7fce380693e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fce540ca7c0 tx=0x7fce44008040 comp rx=0 tx=0).stop 2023-10-06T10:43:45.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.150+0000 7fce5b34a700 1 -- 172.21.15.92:0/1849975671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce540b35d0 msgr2=0x7fce54154940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:45.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.150+0000 7fce5b34a700 1 --2- 172.21.15.92:0/1849975671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce540b35d0 0x7fce54154940 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fce54069150 tx=0x7fce50017000 comp rx=0 tx=0).stop 2023-10-06T10:43:45.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.150+0000 7fce5b34a700 1 -- 172.21.15.92:0/1849975671 shutdown_connections 2023-10-06T10:43:45.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.150+0000 7fce5b34a700 1 --2- 172.21.15.92:0/1849975671 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fce38066f30 0x7fce380693e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:45.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.150+0000 7fce5b34a700 1 --2- 172.21.15.92:0/1849975671 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce54159440 0x7fce540d0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:45.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.150+0000 7fce5b34a700 1 --2- 172.21.15.92:0/1849975671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce540b35d0 0x7fce54154940 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:45.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.150+0000 7fce5b34a700 1 --2- 172.21.15.92:0/1849975671 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce540cc170 0x7fce540d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:45.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.150+0000 7fce5b34a700 1 -- 172.21.15.92:0/1849975671 >> 172.21.15.92:0/1849975671 conn(0x7fce5406ece0 msgr2=0x7fce540b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:45.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.150+0000 7fce5b34a700 1 -- 172.21.15.92:0/1849975671 shutdown_connections 2023-10-06T10:43:45.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.150+0000 7fce5b34a700 1 -- 172.21.15.92:0/1849975671 wait complete. 2023-10-06T10:43:45.163 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T10:43:45.163 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T10:43:45.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.279+0000 7f580fbb7700 1 Processor -- start 2023-10-06T10:43:45.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.280+0000 7f580fbb7700 1 -- start start 2023-10-06T10:43:45.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.280+0000 7f580fbb7700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f580806ece0 0x7f58080b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:45.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.280+0000 7f580fbb7700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f580806f0b0 0x7f58080b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:45.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.280+0000 7f580fbb7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58080b2ce0 con 0x7f58080aede0 2023-10-06T10:43:45.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.280+0000 7f580fbb7700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580806fb40 con 0x7f580806f0b0 2023-10-06T10:43:45.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.280+0000 7f580fbb7700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58080b0410 con 0x7f580806ece0 2023-10-06T10:43:45.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.280+0000 7f580e154700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f58080aede0 0x7f58080af240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37562/0 (socket says 172.21.15.92:37562) 2023-10-06T10:43:45.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.280+0000 7f580e154700 1 -- 172.21.15.92:0/3349372436 learned_addr learned my addr 172.21.15.92:0/3349372436 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:45.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.281+0000 7f580d152700 1 --2- 172.21.15.92:0/3349372436 >> v2:172.21.15.124:3300/0 conn(0x7f580806f0b0 0x7f58080b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:45.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.281+0000 7f580d953700 1 --2- 172.21.15.92:0/3349372436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f580806ece0 0x7f58080b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:45.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.281+0000 7f580c951700 1 -- 172.21.15.92:0/3349372436 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1549392729 0 0) 0x7f58080b2ce0 con 0x7f58080aede0 2023-10-06T10:43:45.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.281+0000 7f580c951700 1 -- 172.21.15.92:0/3349372436 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57f4003570 con 0x7f58080aede0 2023-10-06T10:43:45.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.281+0000 7f580d152700 1 -- 172.21.15.92:0/3349372436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f580806ece0 msgr2=0x7f58080b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:45.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.281+0000 7f580d152700 1 --2- 172.21.15.92:0/3349372436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f580806ece0 0x7f58080b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:45.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.281+0000 7f580d152700 1 -- 172.21.15.92:0/3349372436 >> v1:172.21.15.92:6789/0 conn(0x7f58080aede0 legacy=0x7f58080af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:45.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.281+0000 7f580d152700 1 -- 172.21.15.92:0/3349372436 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58080b2a00 con 0x7f580806f0b0 2023-10-06T10:43:45.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.281+0000 7f580d953700 1 --2- 172.21.15.92:0/3349372436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f580806ece0 0x7f58080b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:43:45.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.281+0000 7f580d152700 1 --2- 172.21.15.92:0/3349372436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f580806f0b0 0x7f58080b08a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009910 tx=0x7f57f80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f85451ff6e594337 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:45.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.281+0000 7f580c951700 1 -- 172.21.15.92:0/3349372436 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8011630 con 0x7f580806f0b0 2023-10-06T10:43:45.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.281+0000 7f580c951700 1 -- 172.21.15.92:0/3349372436 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f8011790 con 0x7f580806f0b0 2023-10-06T10:43:45.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.281+0000 7f580c951700 1 -- 172.21.15.92:0/3349372436 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8011b30 con 0x7f580806f0b0 2023-10-06T10:43:45.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.282+0000 7f580fbb7700 1 -- 172.21.15.92:0/3349372436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f580806f0b0 msgr2=0x7f58080b08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:45.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.282+0000 7f580fbb7700 1 --2- 172.21.15.92:0/3349372436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f580806f0b0 0x7f58080b08a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009910 tx=0x7f57f80091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:45.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.282+0000 7f580fbb7700 1 -- 172.21.15.92:0/3349372436 shutdown_connections 2023-10-06T10:43:45.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.282+0000 7f580fbb7700 1 --2- 172.21.15.92:0/3349372436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f580806ece0 0x7f58080b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:45.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.282+0000 7f580fbb7700 1 --2- 172.21.15.92:0/3349372436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f580806f0b0 0x7f58080b08a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:45.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.282+0000 7f580fbb7700 1 -- 172.21.15.92:0/3349372436 >> 172.21.15.92:0/3349372436 conn(0x7f58080682f0 msgr2=0x7f58080686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:45.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.282+0000 7f580fbb7700 1 -- 172.21.15.92:0/3349372436 shutdown_connections 2023-10-06T10:43:45.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.282+0000 7f580fbb7700 1 -- 172.21.15.92:0/3349372436 wait complete. 2023-10-06T10:43:45.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.283+0000 7f580fbb7700 1 Processor -- start 2023-10-06T10:43:45.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.283+0000 7f580fbb7700 1 -- start start 2023-10-06T10:43:45.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.284+0000 7f580fbb7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f580806ece0 0x7f58081544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:45.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.284+0000 7f580fbb7700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f580806f0b0 0x7f58080b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:45.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.284+0000 7f580fbb7700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58080aede0 0x7f58080b4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:45.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.284+0000 7f580fbb7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5808155030 con 0x7f580806ece0 2023-10-06T10:43:45.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.284+0000 7f580fbb7700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58080c9380 con 0x7f580806f0b0 2023-10-06T10:43:45.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.284+0000 7f580fbb7700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58080c9660 con 0x7f58080aede0 2023-10-06T10:43:45.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.284+0000 7f580d953700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f580806ece0 0x7f58081544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:45.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.284+0000 7f580d953700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f580806ece0 0x7f58081544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44854/0 (socket says 172.21.15.92:44854) 2023-10-06T10:43:45.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.284+0000 7f580d953700 1 -- 172.21.15.92:0/3505814344 learned_addr learned my addr 172.21.15.92:0/3505814344 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:45.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.284+0000 7f580d953700 1 -- 172.21.15.92:0/3505814344 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58080aede0 msgr2=0x7f58080b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:45.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.284+0000 7f580d152700 1 --2- 172.21.15.92:0/3505814344 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f580806f0b0 0x7f58080b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:45.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.284+0000 7f580d953700 1 --2- 172.21.15.92:0/3505814344 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58080aede0 0x7f58080b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:45.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.284+0000 7f580d953700 1 -- 172.21.15.92:0/3505814344 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f580806f0b0 msgr2=0x7f58080b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:45.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.284+0000 7f580d953700 1 --2- 172.21.15.92:0/3505814344 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f580806f0b0 0x7f58080b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:45.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.284+0000 7f580d953700 1 -- 172.21.15.92:0/3505814344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5804009a20 con 0x7f580806ece0 2023-10-06T10:43:45.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.284+0000 7f580d152700 1 --2- 172.21.15.92:0/3505814344 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f580806f0b0 0x7f58080b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:45.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.284+0000 7f580d953700 1 --2- 172.21.15.92:0/3505814344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f580806ece0 0x7f58081544b0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f580400c0c0 tx=0x7f580400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:45.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.285+0000 7f57fdffb700 1 -- 172.21.15.92:0/3505814344 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580400cd60 con 0x7f580806ece0 2023-10-06T10:43:45.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.285+0000 7f57fdffb700 1 -- 172.21.15.92:0/3505814344 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580400f040 con 0x7f580806ece0 2023-10-06T10:43:45.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.285+0000 7f580fbb7700 1 -- 172.21.15.92:0/3505814344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f8009420 con 0x7f580806ece0 2023-10-06T10:43:45.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.285+0000 7f57fdffb700 1 -- 172.21.15.92:0/3505814344 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804014690 con 0x7f580806ece0 2023-10-06T10:43:45.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.285+0000 7f580fbb7700 1 -- 172.21.15.92:0/3505814344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58080c9c20 con 0x7f580806ece0 2023-10-06T10:43:45.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.286+0000 7f580fbb7700 1 -- 172.21.15.92:0/3505814344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5808064a90 con 0x7f580806ece0 2023-10-06T10:43:45.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.286+0000 7f57fdffb700 1 -- 172.21.15.92:0/3505814344 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5804014a00 con 0x7f580806ece0 2023-10-06T10:43:45.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.286+0000 7f57fdffb700 1 --2- 172.21.15.92:0/3505814344 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f57ec066f30 0x7f57ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:45.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.286+0000 7f57fdffb700 1 -- 172.21.15.92:0/3505814344 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f58040852f0 con 0x7f580806ece0 2023-10-06T10:43:45.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.287+0000 7f580d152700 1 --2- 172.21.15.92:0/3505814344 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f57ec066f30 0x7f57ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:45.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.287+0000 7f580d152700 1 --2- 172.21.15.92:0/3505814344 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f57ec066f30 0x7f57ec0693e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f580815cf10 tx=0x7f57f8017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:45.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.291+0000 7f57fdffb700 1 -- 172.21.15.92:0/3505814344 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5804052610 con 0x7f580806ece0 2023-10-06T10:43:45.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:45.473+0000 7f580fbb7700 1 -- 172.21.15.92:0/3505814344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f580804bb40 con 0x7f580806ece0 2023-10-06T10:43:46.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.457+0000 7f57fdffb700 1 -- 172.21.15.92:0/3505814344 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v87) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5804014cb0 con 0x7f580806ece0 2023-10-06T10:43:46.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.461+0000 7f580fbb7700 1 -- 172.21.15.92:0/3505814344 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f57ec066f30 msgr2=0x7f57ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:46.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.461+0000 7f580fbb7700 1 --2- 172.21.15.92:0/3505814344 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f57ec066f30 0x7f57ec0693e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f580815cf10 tx=0x7f57f8017000 comp rx=0 tx=0).stop 2023-10-06T10:43:46.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.461+0000 7f580fbb7700 1 -- 172.21.15.92:0/3505814344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f580806ece0 msgr2=0x7f58081544b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:46.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.461+0000 7f580fbb7700 1 --2- 172.21.15.92:0/3505814344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f580806ece0 0x7f58081544b0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f580400c0c0 tx=0x7f580400b040 comp rx=0 tx=0).stop 2023-10-06T10:43:46.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.461+0000 7f580fbb7700 1 -- 172.21.15.92:0/3505814344 shutdown_connections 2023-10-06T10:43:46.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.461+0000 7f580fbb7700 1 --2- 172.21.15.92:0/3505814344 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f57ec066f30 0x7f57ec0693e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:46.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.461+0000 7f580fbb7700 1 --2- 172.21.15.92:0/3505814344 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58080aede0 0x7f58080b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:46.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.461+0000 7f580fbb7700 1 --2- 172.21.15.92:0/3505814344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f580806ece0 0x7f58081544b0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:46.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.461+0000 7f580fbb7700 1 --2- 172.21.15.92:0/3505814344 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f580806f0b0 0x7f58080b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:46.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.461+0000 7f580fbb7700 1 -- 172.21.15.92:0/3505814344 >> 172.21.15.92:0/3505814344 conn(0x7f58080682f0 msgr2=0x7f5808156940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:46.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.461+0000 7f580fbb7700 1 -- 172.21.15.92:0/3505814344 shutdown_connections 2023-10-06T10:43:46.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.461+0000 7f580fbb7700 1 -- 172.21.15.92:0/3505814344 wait complete. 2023-10-06T10:43:46.464 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-10-06T10:43:46.478 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T10:43:46.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.598+0000 7f4984750700 1 Processor -- start 2023-10-06T10:43:46.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.598+0000 7f4984750700 1 -- start start 2023-10-06T10:43:46.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.599+0000 7f4984750700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f497c0cc820 0x7f497c0ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:46.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.599+0000 7f4984750700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f497c0675b0 0x7f497c0679f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:46.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.599+0000 7f4984750700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f497c067f30 con 0x7f497c06efb0 2023-10-06T10:43:46.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.599+0000 7f4984750700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f497c159bc0 con 0x7f497c0675b0 2023-10-06T10:43:46.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.599+0000 7f4984750700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f497c06f420 con 0x7f497c0cc820 2023-10-06T10:43:46.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.600+0000 7f4982ced700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f497c0675b0 0x7f497c0679f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:46.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.600+0000 7f4982ced700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f497c0675b0 0x7f497c0679f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44866/0 (socket says 172.21.15.92:44866) 2023-10-06T10:43:46.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.600+0000 7f4982ced700 1 -- 172.21.15.92:0/86422958 learned_addr learned my addr 172.21.15.92:0/86422958 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:46.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.600+0000 7f49824ec700 1 --2- 172.21.15.92:0/86422958 >> v2:172.21.15.124:3300/0 conn(0x7f497c0cc820 0x7f497c0ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:46.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.600+0000 7f4982ced700 1 -- 172.21.15.92:0/86422958 >> v2:172.21.15.124:3300/0 conn(0x7f497c0cc820 msgr2=0x7f497c0ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:46.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.600+0000 7f4982ced700 1 --2- 172.21.15.92:0/86422958 >> v2:172.21.15.124:3300/0 conn(0x7f497c0cc820 0x7f497c0ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:46.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.600+0000 7f4982ced700 1 -- 172.21.15.92:0/86422958 >> v1:172.21.15.124:6789/0 conn(0x7f497c06efb0 legacy=0x7f497c0c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:46.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.600+0000 7f4982ced700 1 -- 172.21.15.92:0/86422958 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497c0cbf70 con 0x7f497c0675b0 2023-10-06T10:43:46.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.600+0000 7f4982ced700 1 --2- 172.21.15.92:0/86422958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f497c0675b0 0x7f497c0679f0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f497c0cc160 tx=0x7f4974009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be70082a60a41ad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:46.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.600+0000 7f49814ea700 1 -- 172.21.15.92:0/86422958 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497400e070 con 0x7f497c0675b0 2023-10-06T10:43:46.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.600+0000 7f49814ea700 1 -- 172.21.15.92:0/86422958 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497400ba10 con 0x7f497c0675b0 2023-10-06T10:43:46.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.600+0000 7f49814ea700 1 -- 172.21.15.92:0/86422958 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49740092e0 con 0x7f497c0675b0 2023-10-06T10:43:46.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.601+0000 7f4984750700 1 -- 172.21.15.92:0/86422958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f497c0675b0 msgr2=0x7f497c0679f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:46.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.601+0000 7f4984750700 1 --2- 172.21.15.92:0/86422958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f497c0675b0 0x7f497c0679f0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f497c0cc160 tx=0x7f4974009840 comp rx=0 tx=0).stop 2023-10-06T10:43:46.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.601+0000 7f4984750700 1 -- 172.21.15.92:0/86422958 shutdown_connections 2023-10-06T10:43:46.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.601+0000 7f4984750700 1 --2- 172.21.15.92:0/86422958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f497c0675b0 0x7f497c0679f0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:46.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.601+0000 7f4984750700 1 --2- 172.21.15.92:0/86422958 >> v2:172.21.15.124:3300/0 conn(0x7f497c0cc820 0x7f497c0ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:46.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.601+0000 7f4984750700 1 -- 172.21.15.92:0/86422958 >> 172.21.15.92:0/86422958 conn(0x7f497c069190 msgr2=0x7f497c0cd180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:46.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.601+0000 7f4984750700 1 -- 172.21.15.92:0/86422958 shutdown_connections 2023-10-06T10:43:46.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.601+0000 7f4984750700 1 -- 172.21.15.92:0/86422958 wait complete. 2023-10-06T10:43:46.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.602+0000 7f4984750700 1 Processor -- start 2023-10-06T10:43:46.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.602+0000 7f4984750700 1 -- start start 2023-10-06T10:43:46.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f4984750700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f497c0675b0 0x7f497c1577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:46.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f4984750700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f497c06efb0 0x7f497c0b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:46.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f4984750700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f497c0cc820 0x7f497c1550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:46.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f4984750700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497c1590b0 con 0x7f497c0cc820 2023-10-06T10:43:46.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f4984750700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497c158010 con 0x7f497c06efb0 2023-10-06T10:43:46.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f4984750700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497c1582f0 con 0x7f497c0675b0 2023-10-06T10:43:46.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f4982ced700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f497c0cc820 0x7f497c1550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:46.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f4982ced700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f497c0cc820 0x7f497c1550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44874/0 (socket says 172.21.15.92:44874) 2023-10-06T10:43:46.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f4982ced700 1 -- 172.21.15.92:0/1006277781 learned_addr learned my addr 172.21.15.92:0/1006277781 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:46.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f49824ec700 1 --2- 172.21.15.92:0/1006277781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f497c0675b0 0x7f497c1577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:46.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f4981ceb700 1 --2- 172.21.15.92:0/1006277781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f497c06efb0 0x7f497c0b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:46.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f4982ced700 1 -- 172.21.15.92:0/1006277781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f497c0675b0 msgr2=0x7f497c1577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:46.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f4982ced700 1 --2- 172.21.15.92:0/1006277781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f497c0675b0 0x7f497c1577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:46.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f4982ced700 1 -- 172.21.15.92:0/1006277781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f497c06efb0 msgr2=0x7f497c0b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:46.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f4982ced700 1 --2- 172.21.15.92:0/1006277781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f497c06efb0 0x7f497c0b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:46.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f4982ced700 1 -- 172.21.15.92:0/1006277781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4974009ae0 con 0x7f497c0cc820 2023-10-06T10:43:46.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f4982ced700 1 --2- 172.21.15.92:0/1006277781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f497c0cc820 0x7f497c1550b0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f497c0680c0 tx=0x7f4974018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:46.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f496e7fc700 1 -- 172.21.15.92:0/1006277781 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4974008cb0 con 0x7f497c0cc820 2023-10-06T10:43:46.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f496e7fc700 1 -- 172.21.15.92:0/1006277781 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4974008e10 con 0x7f497c0cc820 2023-10-06T10:43:46.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f4984750700 1 -- 172.21.15.92:0/1006277781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f497c1585d0 con 0x7f497c0cc820 2023-10-06T10:43:46.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f496e7fc700 1 -- 172.21.15.92:0/1006277781 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49740266b0 con 0x7f497c0cc820 2023-10-06T10:43:46.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f4984750700 1 -- 172.21.15.92:0/1006277781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f497c158b90 con 0x7f497c0cc820 2023-10-06T10:43:46.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.604+0000 7f4984750700 1 -- 172.21.15.92:0/1006277781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497c064a90 con 0x7f497c0cc820 2023-10-06T10:43:46.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.605+0000 7f496e7fc700 1 -- 172.21.15.92:0/1006277781 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4974003040 con 0x7f497c0cc820 2023-10-06T10:43:46.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.605+0000 7f496e7fc700 1 --2- 172.21.15.92:0/1006277781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4960066f30 0x7f49600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:46.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.605+0000 7f496e7fc700 1 -- 172.21.15.92:0/1006277781 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5896+0+0 (secure 0 0 0) 0x7f4974086650 con 0x7f497c0cc820 2023-10-06T10:43:46.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.605+0000 7f49824ec700 1 --2- 172.21.15.92:0/1006277781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4960066f30 0x7f49600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:46.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.605+0000 7f49824ec700 1 --2- 172.21.15.92:0/1006277781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4960066f30 0x7f49600693e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4978006fd0 tx=0x7f4978008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:46.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.609+0000 7f496e7fc700 1 -- 172.21.15.92:0/1006277781 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4974053800 con 0x7f497c0cc820 2023-10-06T10:43:46.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:46.792+0000 7f4984750700 1 -- 172.21.15.92:0/1006277781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f497c04bb40 con 0x7f497c0cc820 2023-10-06T10:43:47.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.466+0000 7f496e7fc700 1 -- 172.21.15.92:0/1006277781 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v88) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4974056e20 con 0x7f497c0cc820 2023-10-06T10:43:47.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.471+0000 7f4984750700 1 -- 172.21.15.92:0/1006277781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4960066f30 msgr2=0x7f49600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:47.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.471+0000 7f4984750700 1 --2- 172.21.15.92:0/1006277781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4960066f30 0x7f49600693e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4978006fd0 tx=0x7f4978008040 comp rx=0 tx=0).stop 2023-10-06T10:43:47.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.471+0000 7f4984750700 1 -- 172.21.15.92:0/1006277781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f497c0cc820 msgr2=0x7f497c1550b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:47.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.471+0000 7f4984750700 1 --2- 172.21.15.92:0/1006277781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f497c0cc820 0x7f497c1550b0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f497c0680c0 tx=0x7f4974018000 comp rx=0 tx=0).stop 2023-10-06T10:43:47.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.472+0000 7f4984750700 1 -- 172.21.15.92:0/1006277781 shutdown_connections 2023-10-06T10:43:47.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.472+0000 7f4984750700 1 --2- 172.21.15.92:0/1006277781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4960066f30 0x7f49600693e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:47.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.472+0000 7f4984750700 1 --2- 172.21.15.92:0/1006277781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f497c0cc820 0x7f497c1550b0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:47.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.472+0000 7f4984750700 1 --2- 172.21.15.92:0/1006277781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f497c0675b0 0x7f497c1577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:47.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.472+0000 7f4984750700 1 --2- 172.21.15.92:0/1006277781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f497c06efb0 0x7f497c0b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:47.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.472+0000 7f4984750700 1 -- 172.21.15.92:0/1006277781 >> 172.21.15.92:0/1006277781 conn(0x7f497c069190 msgr2=0x7f497c063410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:47.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.475+0000 7f4984750700 1 -- 172.21.15.92:0/1006277781 shutdown_connections 2023-10-06T10:43:47.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.478+0000 7f4984750700 1 -- 172.21.15.92:0/1006277781 wait complete. 2023-10-06T10:43:47.483 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-10-06T10:43:47.496 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T10:43:47.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.611+0000 7f568855d700 1 Processor -- start 2023-10-06T10:43:47.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.611+0000 7f568855d700 1 -- start start 2023-10-06T10:43:47.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.612+0000 7f568855d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f56801594d0 0x7f56800cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:47.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.612+0000 7f568855d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f56801598a0 0x7f56800cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:47.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.612+0000 7f568855d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56800cf330 con 0x7f56800ce7e0 2023-10-06T10:43:47.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.612+0000 7f568855d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56800cf5d0 con 0x7f56801594d0 2023-10-06T10:43:47.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.612+0000 7f568855d700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56800cf8b0 con 0x7f56801598a0 2023-10-06T10:43:47.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.612+0000 7f56862f9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f56801594d0 0x7f56800cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:47.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.613+0000 7f5686afa700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f56800ce7e0 0x7f56800cec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37570/0 (socket says 172.21.15.92:37570) 2023-10-06T10:43:47.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.613+0000 7f5686afa700 1 -- 172.21.15.92:0/4211628915 learned_addr learned my addr 172.21.15.92:0/4211628915 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:47.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.613+0000 7f5685af8700 1 --2- 172.21.15.92:0/4211628915 >> v2:172.21.15.124:3300/0 conn(0x7f56801598a0 0x7f56800cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.613+0000 7f56862f9700 1 -- 172.21.15.92:0/4211628915 >> v2:172.21.15.124:3300/0 conn(0x7f56801598a0 msgr2=0x7f56800cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.613+0000 7f56862f9700 1 --2- 172.21.15.92:0/4211628915 >> v2:172.21.15.124:3300/0 conn(0x7f56801598a0 0x7f56800cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.613+0000 7f56862f9700 1 -- 172.21.15.92:0/4211628915 >> v1:172.21.15.92:6789/0 conn(0x7f56800ce7e0 legacy=0x7f56800cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.613+0000 7f56862f9700 1 -- 172.21.15.92:0/4211628915 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56800cfb90 con 0x7f56801594d0 2023-10-06T10:43:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.613+0000 7f56852f7700 1 -- 172.21.15.92:0/4211628915 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1051877913 0 0) 0x7f56800cf330 con 0x7f56800ce7e0 2023-10-06T10:43:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.613+0000 7f5685af8700 1 --2- 172.21.15.92:0/4211628915 >> v2:172.21.15.124:3300/0 conn(0x7f56801598a0 0x7f56800cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.613+0000 7f56862f9700 1 --2- 172.21.15.92:0/4211628915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56801594d0 0x7f56800cb560 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f56800c7370 tx=0x7f567c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9710ce1b485d73 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.613+0000 7f56852f7700 1 -- 172.21.15.92:0/4211628915 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c00b6b0 con 0x7f56801594d0 2023-10-06T10:43:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.613+0000 7f56852f7700 1 -- 172.21.15.92:0/4211628915 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c00b810 con 0x7f56801594d0 2023-10-06T10:43:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.613+0000 7f56852f7700 1 -- 172.21.15.92:0/4211628915 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c00bbb0 con 0x7f56801594d0 2023-10-06T10:43:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.614+0000 7f568855d700 1 -- 172.21.15.92:0/4211628915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56801594d0 msgr2=0x7f56800cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.614+0000 7f568855d700 1 --2- 172.21.15.92:0/4211628915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56801594d0 0x7f56800cb560 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f56800c7370 tx=0x7f567c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.614+0000 7f568855d700 1 -- 172.21.15.92:0/4211628915 shutdown_connections 2023-10-06T10:43:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.614+0000 7f568855d700 1 --2- 172.21.15.92:0/4211628915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56801594d0 0x7f56800cb560 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.614+0000 7f568855d700 1 --2- 172.21.15.92:0/4211628915 >> v2:172.21.15.124:3300/0 conn(0x7f56801598a0 0x7f56800cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.614+0000 7f568855d700 1 -- 172.21.15.92:0/4211628915 >> 172.21.15.92:0/4211628915 conn(0x7f5680089e50 msgr2=0x7f5680099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.614+0000 7f568855d700 1 -- 172.21.15.92:0/4211628915 shutdown_connections 2023-10-06T10:43:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.614+0000 7f568855d700 1 -- 172.21.15.92:0/4211628915 wait complete. 2023-10-06T10:43:47.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.615+0000 7f568855d700 1 Processor -- start 2023-10-06T10:43:47.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.615+0000 7f568855d700 1 -- start start 2023-10-06T10:43:47.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.615+0000 7f568855d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56800ce7e0 0x7f568016a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:47.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.615+0000 7f568855d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56801594d0 0x7f568009c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:47.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.615+0000 7f568855d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56801598a0 0x7f568009a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:47.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.615+0000 7f568855d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568016ad90 con 0x7f56801594d0 2023-10-06T10:43:47.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.615+0000 7f568855d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568009cc60 con 0x7f56801598a0 2023-10-06T10:43:47.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.615+0000 7f568855d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568009cf40 con 0x7f56800ce7e0 2023-10-06T10:43:47.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.616+0000 7f5685af8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56801594d0 0x7f568009c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:47.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.616+0000 7f5685af8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56801594d0 0x7f568009c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44892/0 (socket says 172.21.15.92:44892) 2023-10-06T10:43:47.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.616+0000 7f5685af8700 1 -- 172.21.15.92:0/2089146549 learned_addr learned my addr 172.21.15.92:0/2089146549 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:47.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.616+0000 7f56862f9700 1 --2- 172.21.15.92:0/2089146549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56800ce7e0 0x7f568016a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:47.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.616+0000 7f5686afa700 1 --2- 172.21.15.92:0/2089146549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56801598a0 0x7f568009a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:47.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.616+0000 7f5685af8700 1 -- 172.21.15.92:0/2089146549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56800ce7e0 msgr2=0x7f568016a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:47.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.616+0000 7f5685af8700 1 --2- 172.21.15.92:0/2089146549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56800ce7e0 0x7f568016a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:47.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.616+0000 7f5685af8700 1 -- 172.21.15.92:0/2089146549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56801598a0 msgr2=0x7f568009a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:47.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.616+0000 7f5685af8700 1 --2- 172.21.15.92:0/2089146549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56801598a0 0x7f568009a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:47.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.616+0000 7f5685af8700 1 -- 172.21.15.92:0/2089146549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567c009420 con 0x7f56801594d0 2023-10-06T10:43:47.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.616+0000 7f5685af8700 1 --2- 172.21.15.92:0/2089146549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56801594d0 0x7f568009c180 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f567000c880 tx=0x7f5670008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:47.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.616+0000 7f56767fc700 1 -- 172.21.15.92:0/2089146549 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5670011600 con 0x7f56801594d0 2023-10-06T10:43:47.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.616+0000 7f56767fc700 1 -- 172.21.15.92:0/2089146549 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5670011760 con 0x7f56801594d0 2023-10-06T10:43:47.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.616+0000 7f568855d700 1 -- 172.21.15.92:0/2089146549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568009d220 con 0x7f56801594d0 2023-10-06T10:43:47.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.616+0000 7f56767fc700 1 -- 172.21.15.92:0/2089146549 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5670011b00 con 0x7f56801594d0 2023-10-06T10:43:47.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.616+0000 7f568855d700 1 -- 172.21.15.92:0/2089146549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568009d7e0 con 0x7f56801594d0 2023-10-06T10:43:47.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.617+0000 7f568855d700 1 -- 172.21.15.92:0/2089146549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5680064a90 con 0x7f56801594d0 2023-10-06T10:43:47.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.618+0000 7f56767fc700 1 -- 172.21.15.92:0/2089146549 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f56700193b0 con 0x7f56801594d0 2023-10-06T10:43:47.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.618+0000 7f56767fc700 1 --2- 172.21.15.92:0/2089146549 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5664066f30 0x7f56640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:47.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.618+0000 7f56767fc700 1 -- 172.21.15.92:0/2089146549 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6278+0+0 (secure 0 0 0) 0x7f5670084780 con 0x7f56801594d0 2023-10-06T10:43:47.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.618+0000 7f56862f9700 1 --2- 172.21.15.92:0/2089146549 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5664066f30 0x7f56640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:47.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.619+0000 7f56862f9700 1 --2- 172.21.15.92:0/2089146549 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5664066f30 0x7f56640693e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f567c0093f0 tx=0x7f567c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:47.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.623+0000 7f56767fc700 1 -- 172.21.15.92:0/2089146549 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f56700517b0 con 0x7f56801594d0 2023-10-06T10:43:47.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:47.849+0000 7f568855d700 1 -- 172.21.15.92:0/2089146549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f56800cfe70 con 0x7f56801594d0 2023-10-06T10:43:48.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.488+0000 7f56767fc700 1 -- 172.21.15.92:0/2089146549 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v79) v1 ==== 174+0+0 (secure 0 0 0) 0x7f5670054dd0 con 0x7f56801594d0 2023-10-06T10:43:48.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.493+0000 7f568855d700 1 -- 172.21.15.92:0/2089146549 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5664066f30 msgr2=0x7f56640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:48.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.493+0000 7f568855d700 1 --2- 172.21.15.92:0/2089146549 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5664066f30 0x7f56640693e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f567c0093f0 tx=0x7f567c017000 comp rx=0 tx=0).stop 2023-10-06T10:43:48.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.493+0000 7f568855d700 1 -- 172.21.15.92:0/2089146549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56801594d0 msgr2=0x7f568009c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:48.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.493+0000 7f568855d700 1 --2- 172.21.15.92:0/2089146549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56801594d0 0x7f568009c180 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f567000c880 tx=0x7f5670008040 comp rx=0 tx=0).stop 2023-10-06T10:43:48.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.493+0000 7f568855d700 1 -- 172.21.15.92:0/2089146549 shutdown_connections 2023-10-06T10:43:48.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.493+0000 7f568855d700 1 --2- 172.21.15.92:0/2089146549 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5664066f30 0x7f56640693e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:48.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.493+0000 7f568855d700 1 --2- 172.21.15.92:0/2089146549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56801598a0 0x7f568009a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:48.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.493+0000 7f568855d700 1 --2- 172.21.15.92:0/2089146549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56800ce7e0 0x7f568016a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:48.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.493+0000 7f568855d700 1 --2- 172.21.15.92:0/2089146549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56801594d0 0x7f568009c180 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:48.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.493+0000 7f568855d700 1 -- 172.21.15.92:0/2089146549 >> 172.21.15.92:0/2089146549 conn(0x7f5680089e50 msgr2=0x7f56800cad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:48.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.493+0000 7f568855d700 1 -- 172.21.15.92:0/2089146549 shutdown_connections 2023-10-06T10:43:48.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.493+0000 7f568855d700 1 -- 172.21.15.92:0/2089146549 wait complete. 2023-10-06T10:43:48.496 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 14 and data pool 15 2023-10-06T10:43:48.507 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:43:48.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.620+0000 7f4f1519a700 1 Processor -- start 2023-10-06T10:43:48.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.620+0000 7f4f1519a700 1 -- start start 2023-10-06T10:43:48.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.621+0000 7f4f1519a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4f100adf20 0x7f4f100ac000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:48.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.621+0000 7f4f1519a700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4f1006e240 0x7f4f1006e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:48.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.621+0000 7f4f1519a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f100cc7f0 0x7f4f10069520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:48.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.621+0000 7f4f1519a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f10156110 con 0x7f4f100adf20 2023-10-06T10:43:48.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.621+0000 7f4f1519a700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f100c7330 con 0x7f4f1006e240 2023-10-06T10:43:48.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.621+0000 7f4f1519a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f100ac540 con 0x7f4f100cc7f0 2023-10-06T10:43:48.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.621+0000 7f4f0ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4f100adf20 0x7f4f100ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:48.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.621+0000 7f4f0ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4f100adf20 0x7f4f100ac000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44896/0 (socket says 172.21.15.92:44896) 2023-10-06T10:43:48.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.621+0000 7f4f0ed9d700 1 -- 172.21.15.92:0/3574213276 learned_addr learned my addr 172.21.15.92:0/3574213276 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:48.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.621+0000 7f4f0659c700 1 --2- 172.21.15.92:0/3574213276 >> v2:172.21.15.124:3300/0 conn(0x7f4f1006e240 0x7f4f1006e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:48.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.621+0000 7f4f0f59e700 1 --2- 172.21.15.92:0/3574213276 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f100cc7f0 0x7f4f10069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:48.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.621+0000 7f4f0ed9d700 1 -- 172.21.15.92:0/3574213276 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f100cc7f0 msgr2=0x7f4f10069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:48.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.621+0000 7f4f0ed9d700 1 --2- 172.21.15.92:0/3574213276 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f100cc7f0 0x7f4f10069520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:48.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.621+0000 7f4f0ed9d700 1 -- 172.21.15.92:0/3574213276 >> v2:172.21.15.124:3300/0 conn(0x7f4f1006e240 msgr2=0x7f4f1006e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:48.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.621+0000 7f4f0ed9d700 1 --2- 172.21.15.92:0/3574213276 >> v2:172.21.15.124:3300/0 conn(0x7f4f1006e240 0x7f4f1006e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:48.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.621+0000 7f4f0ed9d700 1 -- 172.21.15.92:0/3574213276 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f100c6720 con 0x7f4f100adf20 2023-10-06T10:43:48.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.621+0000 7f4f0659c700 1 --2- 172.21.15.92:0/3574213276 >> v2:172.21.15.124:3300/0 conn(0x7f4f1006e240 0x7f4f1006e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:48.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.622+0000 7f4f0ed9d700 1 --2- 172.21.15.92:0/3574213276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f100adf20 0x7f4f100ac000 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f4f100cc040 tx=0x7f4f00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcce5fd96edf33e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:48.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.622+0000 7f4f0e59c700 1 -- 172.21.15.92:0/3574213276 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0000b860 con 0x7f4f100adf20 2023-10-06T10:43:48.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.622+0000 7f4f0e59c700 1 -- 172.21.15.92:0/3574213276 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0000b9c0 con 0x7f4f100adf20 2023-10-06T10:43:48.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.622+0000 7f4f0e59c700 1 -- 172.21.15.92:0/3574213276 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0000bd60 con 0x7f4f100adf20 2023-10-06T10:43:48.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.622+0000 7f4f1519a700 1 -- 172.21.15.92:0/3574213276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f100adf20 msgr2=0x7f4f100ac000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:48.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.622+0000 7f4f1519a700 1 --2- 172.21.15.92:0/3574213276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f100adf20 0x7f4f100ac000 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f4f100cc040 tx=0x7f4f00009250 comp rx=0 tx=0).stop 2023-10-06T10:43:48.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.622+0000 7f4f1519a700 1 -- 172.21.15.92:0/3574213276 shutdown_connections 2023-10-06T10:43:48.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.622+0000 7f4f1519a700 1 --2- 172.21.15.92:0/3574213276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f100adf20 0x7f4f100ac000 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:48.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.622+0000 7f4f1519a700 1 --2- 172.21.15.92:0/3574213276 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f100cc7f0 0x7f4f10069520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:48.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.622+0000 7f4f1519a700 1 --2- 172.21.15.92:0/3574213276 >> v2:172.21.15.124:3300/0 conn(0x7f4f1006e240 0x7f4f1006e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:48.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.622+0000 7f4f1519a700 1 -- 172.21.15.92:0/3574213276 >> 172.21.15.92:0/3574213276 conn(0x7f4f10068080 msgr2=0x7f4f100cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:48.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.622+0000 7f4f1519a700 1 -- 172.21.15.92:0/3574213276 shutdown_connections 2023-10-06T10:43:48.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.622+0000 7f4f1519a700 1 -- 172.21.15.92:0/3574213276 wait complete. 2023-10-06T10:43:48.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.624+0000 7f4f1519a700 1 Processor -- start 2023-10-06T10:43:48.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.624+0000 7f4f1519a700 1 -- start start 2023-10-06T10:43:48.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.624+0000 7f4f1519a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f1006e240 0x7f4f100b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:48.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.624+0000 7f4f1519a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f100adf20 0x7f4f100c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:48.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.624+0000 7f4f1519a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f100cc7f0 0x7f4f100c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:48.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.624+0000 7f4f1519a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f100c4ee0 con 0x7f4f1006e240 2023-10-06T10:43:48.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.624+0000 7f4f1519a700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f100c33b0 con 0x7f4f100adf20 2023-10-06T10:43:48.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.624+0000 7f4f1519a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f100c48d0 con 0x7f4f100cc7f0 2023-10-06T10:43:48.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.624+0000 7f4f0ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f1006e240 0x7f4f100b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:48.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.624+0000 7f4f0ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f1006e240 0x7f4f100b0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44898/0 (socket says 172.21.15.92:44898) 2023-10-06T10:43:48.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.624+0000 7f4f0ed9d700 1 -- 172.21.15.92:0/4114707664 learned_addr learned my addr 172.21.15.92:0/4114707664 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:48.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.625+0000 7f4f0659c700 1 --2- 172.21.15.92:0/4114707664 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f100adf20 0x7f4f100c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:48.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.625+0000 7f4f0f59e700 1 --2- 172.21.15.92:0/4114707664 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f100cc7f0 0x7f4f100c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:48.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.625+0000 7f4f0ed9d700 1 -- 172.21.15.92:0/4114707664 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f100cc7f0 msgr2=0x7f4f100c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:48.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.625+0000 7f4f0ed9d700 1 --2- 172.21.15.92:0/4114707664 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f100cc7f0 0x7f4f100c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:48.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.625+0000 7f4f0ed9d700 1 -- 172.21.15.92:0/4114707664 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f100adf20 msgr2=0x7f4f100c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:48.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.625+0000 7f4f0ed9d700 1 --2- 172.21.15.92:0/4114707664 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f100adf20 0x7f4f100c8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:48.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.625+0000 7f4f0ed9d700 1 -- 172.21.15.92:0/4114707664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f000094f0 con 0x7f4f1006e240 2023-10-06T10:43:48.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.625+0000 7f4f0ed9d700 1 --2- 172.21.15.92:0/4114707664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f1006e240 0x7f4f100b0010 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009290 tx=0x7f4f00017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:48.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.625+0000 7f4f077fe700 1 -- 172.21.15.92:0/4114707664 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0001b5c0 con 0x7f4f1006e240 2023-10-06T10:43:48.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.625+0000 7f4f077fe700 1 -- 172.21.15.92:0/4114707664 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0001b720 con 0x7f4f1006e240 2023-10-06T10:43:48.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.625+0000 7f4f1519a700 1 -- 172.21.15.92:0/4114707664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f100c30f0 con 0x7f4f1006e240 2023-10-06T10:43:48.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.625+0000 7f4f077fe700 1 -- 172.21.15.92:0/4114707664 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0001bae0 con 0x7f4f1006e240 2023-10-06T10:43:48.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.625+0000 7f4f1519a700 1 -- 172.21.15.92:0/4114707664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f100c4b70 con 0x7f4f1006e240 2023-10-06T10:43:48.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.626+0000 7f4f1519a700 1 -- 172.21.15.92:0/4114707664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f100c51c0 con 0x7f4f1006e240 2023-10-06T10:43:48.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.626+0000 7f4f077fe700 1 -- 172.21.15.92:0/4114707664 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4f0002e440 con 0x7f4f1006e240 2023-10-06T10:43:48.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.627+0000 7f4f077fe700 1 --2- 172.21.15.92:0/4114707664 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4ef0066f30 0x7f4ef00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:48.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.627+0000 7f4f0659c700 1 --2- 172.21.15.92:0/4114707664 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4ef0066f30 0x7f4ef00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:48.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.627+0000 7f4f077fe700 1 -- 172.21.15.92:0/4114707664 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f4f00088050 con 0x7f4f1006e240 2023-10-06T10:43:48.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.627+0000 7f4f0659c700 1 --2- 172.21.15.92:0/4114707664 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4ef0066f30 0x7f4ef00693e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8006f00 tx=0x7f4ef8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:48.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.631+0000 7f4f077fe700 1 -- 172.21.15.92:0/4114707664 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f00055020 con 0x7f4f1006e240 2023-10-06T10:43:48.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.815+0000 7f4f1519a700 1 -- 172.21.15.92:0/4114707664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f10064a90 con 0x7f4f1006e240 2023-10-06T10:43:48.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.815+0000 7f4f077fe700 1 -- 172.21.15.92:0/4114707664 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+17087 (secure 0 0 0) 0x7f4f00058640 con 0x7f4f1006e240 2023-10-06T10:43:48.817 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:48.817 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":89,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:43:47.850525+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:43:45.492396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-06T10:43:46.811317+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:43:48.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.817+0000 7f4f1519a700 1 -- 172.21.15.92:0/4114707664 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4ef0066f30 msgr2=0x7f4ef00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:48.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.817+0000 7f4f1519a700 1 --2- 172.21.15.92:0/4114707664 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4ef0066f30 0x7f4ef00693e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8006f00 tx=0x7f4ef8008040 comp rx=0 tx=0).stop 2023-10-06T10:43:48.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.817+0000 7f4f1519a700 1 -- 172.21.15.92:0/4114707664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f1006e240 msgr2=0x7f4f100b0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:48.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.817+0000 7f4f1519a700 1 --2- 172.21.15.92:0/4114707664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f1006e240 0x7f4f100b0010 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009290 tx=0x7f4f00017000 comp rx=0 tx=0).stop 2023-10-06T10:43:48.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.817+0000 7f4f1519a700 1 -- 172.21.15.92:0/4114707664 shutdown_connections 2023-10-06T10:43:48.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.817+0000 7f4f1519a700 1 --2- 172.21.15.92:0/4114707664 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4ef0066f30 0x7f4ef00693e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:48.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.817+0000 7f4f1519a700 1 --2- 172.21.15.92:0/4114707664 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f100cc7f0 0x7f4f100c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:48.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.817+0000 7f4f1519a700 1 --2- 172.21.15.92:0/4114707664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f1006e240 0x7f4f100b0010 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:48.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.817+0000 7f4f1519a700 1 --2- 172.21.15.92:0/4114707664 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f100adf20 0x7f4f100c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:48.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.817+0000 7f4f1519a700 1 -- 172.21.15.92:0/4114707664 >> 172.21.15.92:0/4114707664 conn(0x7f4f10068080 msgr2=0x7f4f1006d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:48.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.817+0000 7f4f1519a700 1 -- 172.21.15.92:0/4114707664 shutdown_connections 2023-10-06T10:43:48.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.817+0000 7f4f1519a700 1 -- 172.21.15.92:0/4114707664 wait complete. 2023-10-06T10:43:48.831 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:43:48.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.944+0000 7f9b3a534700 1 Processor -- start 2023-10-06T10:43:48.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.944+0000 7f9b3a534700 1 -- start start 2023-10-06T10:43:48.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.945+0000 7f9b3a534700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9b340b49a0 0x7f9b340aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:48.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.945+0000 7f9b3a534700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9b340b0760 0x7f9b340b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:48.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.945+0000 7f9b3a534700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b340b0c60 con 0x7f9b34159560 2023-10-06T10:43:48.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.945+0000 7f9b3a534700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b340b0f40 con 0x7f9b340b49a0 2023-10-06T10:43:48.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.945+0000 7f9b3a534700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b340b1220 con 0x7f9b340b0760 2023-10-06T10:43:48.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.945+0000 7f9b337fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9b340b49a0 0x7f9b340aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:48.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.945+0000 7f9b337fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9b340b49a0 0x7f9b340aac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44904/0 (socket says 172.21.15.92:44904) 2023-10-06T10:43:48.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.945+0000 7f9b337fe700 1 -- 172.21.15.92:0/1449836952 learned_addr learned my addr 172.21.15.92:0/1449836952 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:48.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.945+0000 7f9b38ad1700 1 --2- 172.21.15.92:0/1449836952 >> v2:172.21.15.124:3300/0 conn(0x7f9b340b0760 0x7f9b340b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:48.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.945+0000 7f9b337fe700 1 -- 172.21.15.92:0/1449836952 >> v2:172.21.15.124:3300/0 conn(0x7f9b340b0760 msgr2=0x7f9b340b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:48.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.945+0000 7f9b337fe700 1 --2- 172.21.15.92:0/1449836952 >> v2:172.21.15.124:3300/0 conn(0x7f9b340b0760 0x7f9b340b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:48.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.945+0000 7f9b337fe700 1 -- 172.21.15.92:0/1449836952 >> v1:172.21.15.92:6789/0 conn(0x7f9b34159560 legacy=0x7f9b340aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:48.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.945+0000 7f9b337fe700 1 -- 172.21.15.92:0/1449836952 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b340b1500 con 0x7f9b340b49a0 2023-10-06T10:43:48.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.945+0000 7f9b32ffd700 1 -- 172.21.15.92:0/1449836952 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1206561722 0 0) 0x7f9b340b0c60 con 0x7f9b34159560 2023-10-06T10:43:48.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.945+0000 7f9b337fe700 1 --2- 172.21.15.92:0/1449836952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b340b49a0 0x7f9b340aac40 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f9b240099b0 tx=0x7f9b24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59fa91813b1e69c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:48.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.945+0000 7f9b32ffd700 1 -- 172.21.15.92:0/1449836952 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2400b820 con 0x7f9b340b49a0 2023-10-06T10:43:48.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.945+0000 7f9b32ffd700 1 -- 172.21.15.92:0/1449836952 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2400b980 con 0x7f9b340b49a0 2023-10-06T10:43:48.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.945+0000 7f9b32ffd700 1 -- 172.21.15.92:0/1449836952 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2400bd20 con 0x7f9b340b49a0 2023-10-06T10:43:48.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.946+0000 7f9b3a534700 1 -- 172.21.15.92:0/1449836952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b340b49a0 msgr2=0x7f9b340aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:48.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.946+0000 7f9b3a534700 1 --2- 172.21.15.92:0/1449836952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b340b49a0 0x7f9b340aac40 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f9b240099b0 tx=0x7f9b24009250 comp rx=0 tx=0).stop 2023-10-06T10:43:48.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.946+0000 7f9b3a534700 1 -- 172.21.15.92:0/1449836952 shutdown_connections 2023-10-06T10:43:48.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.946+0000 7f9b3a534700 1 --2- 172.21.15.92:0/1449836952 >> v2:172.21.15.124:3300/0 conn(0x7f9b340b0760 0x7f9b340b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:48.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.946+0000 7f9b3a534700 1 --2- 172.21.15.92:0/1449836952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b340b49a0 0x7f9b340aac40 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:48.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.946+0000 7f9b3a534700 1 -- 172.21.15.92:0/1449836952 >> 172.21.15.92:0/1449836952 conn(0x7f9b340997f0 msgr2=0x7f9b3406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:48.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.946+0000 7f9b3a534700 1 -- 172.21.15.92:0/1449836952 shutdown_connections 2023-10-06T10:43:48.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.946+0000 7f9b3a534700 1 -- 172.21.15.92:0/1449836952 wait complete. 2023-10-06T10:43:48.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.947+0000 7f9b3a534700 1 Processor -- start 2023-10-06T10:43:48.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.947+0000 7f9b3a534700 1 -- start start 2023-10-06T10:43:48.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.948+0000 7f9b3a534700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b340b0760 0x7f9b340b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:48.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.948+0000 7f9b3a534700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b340b49a0 0x7f9b3409be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:48.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.948+0000 7f9b3a534700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b34159560 0x7f9b3409c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:48.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.948+0000 7f9b3a534700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b34157f50 con 0x7f9b340b0760 2023-10-06T10:43:48.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.948+0000 7f9b3a534700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3409ca10 con 0x7f9b340b49a0 2023-10-06T10:43:48.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.948+0000 7f9b3a534700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3416d940 con 0x7f9b34159560 2023-10-06T10:43:48.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.948+0000 7f9b33fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b340b0760 0x7f9b340b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:48.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.948+0000 7f9b33fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b340b0760 0x7f9b340b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44914/0 (socket says 172.21.15.92:44914) 2023-10-06T10:43:48.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.948+0000 7f9b33fff700 1 -- 172.21.15.92:0/3581466754 learned_addr learned my addr 172.21.15.92:0/3581466754 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:48.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.948+0000 7f9b33fff700 1 -- 172.21.15.92:0/3581466754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b34159560 msgr2=0x7f9b3409c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:48.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.948+0000 7f9b33fff700 1 --2- 172.21.15.92:0/3581466754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b34159560 0x7f9b3409c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:48.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.948+0000 7f9b33fff700 1 -- 172.21.15.92:0/3581466754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b340b49a0 msgr2=0x7f9b3409be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:48.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.948+0000 7f9b33fff700 1 --2- 172.21.15.92:0/3581466754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b340b49a0 0x7f9b3409be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:48.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.948+0000 7f9b33fff700 1 -- 172.21.15.92:0/3581466754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b240094f0 con 0x7f9b340b0760 2023-10-06T10:43:48.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.948+0000 7f9b33fff700 1 --2- 172.21.15.92:0/3581466754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b340b0760 0x7f9b340b3480 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9b28001eb0 tx=0x7f9b28007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:48.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.948+0000 7f9b1bfff700 1 -- 172.21.15.92:0/3581466754 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2800e4a0 con 0x7f9b340b0760 2023-10-06T10:43:48.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.949+0000 7f9b1bfff700 1 -- 172.21.15.92:0/3581466754 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2800e600 con 0x7f9b340b0760 2023-10-06T10:43:48.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.949+0000 7f9b3a534700 1 -- 172.21.15.92:0/3581466754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b3416ddd0 con 0x7f9b340b0760 2023-10-06T10:43:48.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.949+0000 7f9b1bfff700 1 -- 172.21.15.92:0/3581466754 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2800e9c0 con 0x7f9b340b0760 2023-10-06T10:43:48.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.949+0000 7f9b3a534700 1 -- 172.21.15.92:0/3581466754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3416d2f0 con 0x7f9b340b0760 2023-10-06T10:43:48.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.949+0000 7f9b3a534700 1 -- 172.21.15.92:0/3581466754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b34064a90 con 0x7f9b340b0760 2023-10-06T10:43:48.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.950+0000 7f9b1bfff700 1 -- 172.21.15.92:0/3581466754 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9b2801b3b0 con 0x7f9b340b0760 2023-10-06T10:43:48.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.950+0000 7f9b1bfff700 1 --2- 172.21.15.92:0/3581466754 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9b20066f30 0x7f9b200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:48.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.950+0000 7f9b1bfff700 1 -- 172.21.15.92:0/3581466754 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9b28083e50 con 0x7f9b340b0760 2023-10-06T10:43:48.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.950+0000 7f9b337fe700 1 --2- 172.21.15.92:0/3581466754 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9b20066f30 0x7f9b200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:48.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.951+0000 7f9b337fe700 1 --2- 172.21.15.92:0/3581466754 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9b20066f30 0x7f9b200693e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9b34088de0 tx=0x7f9b24017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:48.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:48.955+0000 7f9b1bfff700 1 -- 172.21.15.92:0/3581466754 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9b28050e20 con 0x7f9b340b0760 2023-10-06T10:43:49.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.138+0000 7f9b3a534700 1 -- 172.21.15.92:0/3581466754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b340b17e0 con 0x7f9b340b0760 2023-10-06T10:43:49.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.139+0000 7f9b1bfff700 1 -- 172.21.15.92:0/3581466754 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+17087 (secure 0 0 0) 0x7f9b28017070 con 0x7f9b340b0760 2023-10-06T10:43:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:49.140 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":89,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:43:47.850525+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:43:45.492396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-06T10:43:46.811317+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:43:49.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.141+0000 7f9b3a534700 1 -- 172.21.15.92:0/3581466754 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9b20066f30 msgr2=0x7f9b200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:49.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.141+0000 7f9b3a534700 1 --2- 172.21.15.92:0/3581466754 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9b20066f30 0x7f9b200693e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9b34088de0 tx=0x7f9b24017000 comp rx=0 tx=0).stop 2023-10-06T10:43:49.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.141+0000 7f9b3a534700 1 -- 172.21.15.92:0/3581466754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b340b0760 msgr2=0x7f9b340b3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:49.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.141+0000 7f9b3a534700 1 --2- 172.21.15.92:0/3581466754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b340b0760 0x7f9b340b3480 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9b28001eb0 tx=0x7f9b28007f60 comp rx=0 tx=0).stop 2023-10-06T10:43:49.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.141+0000 7f9b3a534700 1 -- 172.21.15.92:0/3581466754 shutdown_connections 2023-10-06T10:43:49.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.141+0000 7f9b3a534700 1 --2- 172.21.15.92:0/3581466754 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9b20066f30 0x7f9b200693e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:49.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.141+0000 7f9b3a534700 1 --2- 172.21.15.92:0/3581466754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b34159560 0x7f9b3409c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:49.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.141+0000 7f9b3a534700 1 --2- 172.21.15.92:0/3581466754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b340b0760 0x7f9b340b3480 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:49.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.141+0000 7f9b3a534700 1 --2- 172.21.15.92:0/3581466754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b340b49a0 0x7f9b3409be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:49.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.141+0000 7f9b3a534700 1 -- 172.21.15.92:0/3581466754 >> 172.21.15.92:0/3581466754 conn(0x7f9b340997f0 msgr2=0x7f9b34069980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:49.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.141+0000 7f9b3a534700 1 -- 172.21.15.92:0/3581466754 shutdown_connections 2023-10-06T10:43:49.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.141+0000 7f9b3a534700 1 -- 172.21.15.92:0/3581466754 wait complete. 2023-10-06T10:43:49.154 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T10:43:49.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.270+0000 7f53dac7d700 1 Processor -- start 2023-10-06T10:43:49.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.270+0000 7f53dac7d700 1 -- start start 2023-10-06T10:43:49.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.271+0000 7f53dac7d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f53d40b2620 0x7f53d40cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:49.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.271+0000 7f53dac7d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53d40cf1b0 0x7f53d40cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:49.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.271+0000 7f53dac7d700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d40cc970 con 0x7f53d40b29f0 2023-10-06T10:43:49.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.271+0000 7f53dac7d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d40cbd60 con 0x7f53d40b2620 2023-10-06T10:43:49.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.271+0000 7f53dac7d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d40cc040 con 0x7f53d40cf1b0 2023-10-06T10:43:49.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.271+0000 7f53d9c7b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f53d40b2620 0x7f53d40cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:49.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.271+0000 7f53d9c7b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f53d40b2620 0x7f53d40cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44926/0 (socket says 172.21.15.92:44926) 2023-10-06T10:43:49.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.271+0000 7f53d9c7b700 1 -- 172.21.15.92:0/224866377 learned_addr learned my addr 172.21.15.92:0/224866377 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:49.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.271+0000 7f53da47c700 1 --2- 172.21.15.92:0/224866377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53d40cf1b0 0x7f53d40cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:49.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.271+0000 7f53d9c7b700 1 -- 172.21.15.92:0/224866377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53d40cf1b0 msgr2=0x7f53d40cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:49.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.271+0000 7f53d9c7b700 1 --2- 172.21.15.92:0/224866377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53d40cf1b0 0x7f53d40cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:49.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.271+0000 7f53d9c7b700 1 -- 172.21.15.92:0/224866377 >> v1:172.21.15.124:6789/0 conn(0x7f53d40b29f0 legacy=0x7f53d40ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:49.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.271+0000 7f53d9c7b700 1 -- 172.21.15.92:0/224866377 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d40cc320 con 0x7f53d40b2620 2023-10-06T10:43:49.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.271+0000 7f53d9c7b700 1 --2- 172.21.15.92:0/224866377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d40b2620 0x7f53d40cccb0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009f70 tx=0x7f53d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be4d645a9cc2080 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:49.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.271+0000 7f53d8c79700 1 -- 172.21.15.92:0/224866377 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d000e070 con 0x7f53d40b2620 2023-10-06T10:43:49.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.271+0000 7f53d8c79700 1 -- 172.21.15.92:0/224866377 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d000ba20 con 0x7f53d40b2620 2023-10-06T10:43:49.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.271+0000 7f53d8c79700 1 -- 172.21.15.92:0/224866377 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d00092e0 con 0x7f53d40b2620 2023-10-06T10:43:49.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.272+0000 7f53dac7d700 1 -- 172.21.15.92:0/224866377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d40b2620 msgr2=0x7f53d40cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:49.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.272+0000 7f53dac7d700 1 --2- 172.21.15.92:0/224866377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d40b2620 0x7f53d40cccb0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009f70 tx=0x7f53d0009840 comp rx=0 tx=0).stop 2023-10-06T10:43:49.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.272+0000 7f53dac7d700 1 -- 172.21.15.92:0/224866377 shutdown_connections 2023-10-06T10:43:49.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.272+0000 7f53dac7d700 1 --2- 172.21.15.92:0/224866377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53d40cf1b0 0x7f53d40cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:49.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.272+0000 7f53dac7d700 1 --2- 172.21.15.92:0/224866377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d40b2620 0x7f53d40cccb0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:49.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.272+0000 7f53dac7d700 1 -- 172.21.15.92:0/224866377 >> 172.21.15.92:0/224866377 conn(0x7f53d4099520 msgr2=0x7f53d406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:49.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.272+0000 7f53dac7d700 1 -- 172.21.15.92:0/224866377 shutdown_connections 2023-10-06T10:43:49.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.272+0000 7f53dac7d700 1 -- 172.21.15.92:0/224866377 wait complete. 2023-10-06T10:43:49.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.273+0000 7f53dac7d700 1 Processor -- start 2023-10-06T10:43:49.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.273+0000 7f53dac7d700 1 -- start start 2023-10-06T10:43:49.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.274+0000 7f53dac7d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53d40b2620 0x7f53d4157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:49.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.274+0000 7f53dac7d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d40b29f0 0x7f53d415d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:49.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.274+0000 7f53dac7d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53d40cf1b0 0x7f53d415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:49.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.274+0000 7f53dac7d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d415b550 con 0x7f53d40b29f0 2023-10-06T10:43:49.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.274+0000 7f53dac7d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d415a940 con 0x7f53d40cf1b0 2023-10-06T10:43:49.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.274+0000 7f53dac7d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d415ac20 con 0x7f53d40b2620 2023-10-06T10:43:49.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.274+0000 7f53d947a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d40b29f0 0x7f53d415d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:49.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.274+0000 7f53d947a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d40b29f0 0x7f53d415d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44928/0 (socket says 172.21.15.92:44928) 2023-10-06T10:43:49.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.274+0000 7f53d947a700 1 -- 172.21.15.92:0/3720953873 learned_addr learned my addr 172.21.15.92:0/3720953873 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.274+0000 7f53d9c7b700 1 --2- 172.21.15.92:0/3720953873 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53d40b2620 0x7f53d4157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.274+0000 7f53da47c700 1 --2- 172.21.15.92:0/3720953873 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53d40cf1b0 0x7f53d415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.274+0000 7f53d947a700 1 -- 172.21.15.92:0/3720953873 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53d40b2620 msgr2=0x7f53d4157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.274+0000 7f53d947a700 1 --2- 172.21.15.92:0/3720953873 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53d40b2620 0x7f53d4157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.274+0000 7f53d947a700 1 -- 172.21.15.92:0/3720953873 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53d40cf1b0 msgr2=0x7f53d415d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.274+0000 7f53d947a700 1 --2- 172.21.15.92:0/3720953873 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53d40cf1b0 0x7f53d415d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.274+0000 7f53d947a700 1 -- 172.21.15.92:0/3720953873 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d0009ae0 con 0x7f53d40b29f0 2023-10-06T10:43:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.274+0000 7f53da47c700 1 --2- 172.21.15.92:0/3720953873 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53d40cf1b0 0x7f53d415d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.274+0000 7f53d9c7b700 1 --2- 172.21.15.92:0/3720953873 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53d40b2620 0x7f53d4157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.274+0000 7f53d947a700 1 --2- 172.21.15.92:0/3720953873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d40b29f0 0x7f53d415d180 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f53c400c7c0 tx=0x7f53c4007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.275+0000 7f53c9ffb700 1 -- 172.21.15.92:0/3720953873 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c4011590 con 0x7f53d40b29f0 2023-10-06T10:43:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.275+0000 7f53c9ffb700 1 -- 172.21.15.92:0/3720953873 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c40116f0 con 0x7f53d40b29f0 2023-10-06T10:43:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.275+0000 7f53c9ffb700 1 -- 172.21.15.92:0/3720953873 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c4011ab0 con 0x7f53d40b29f0 2023-10-06T10:43:49.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.275+0000 7f53dac7d700 1 -- 172.21.15.92:0/3720953873 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d415af00 con 0x7f53d40b29f0 2023-10-06T10:43:49.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.275+0000 7f53dac7d700 1 -- 172.21.15.92:0/3720953873 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d41581f0 con 0x7f53d40b29f0 2023-10-06T10:43:49.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.276+0000 7f53dac7d700 1 -- 172.21.15.92:0/3720953873 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d415b700 con 0x7f53d40b29f0 2023-10-06T10:43:49.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.276+0000 7f53c9ffb700 1 -- 172.21.15.92:0/3720953873 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f53c401b3b0 con 0x7f53d40b29f0 2023-10-06T10:43:49.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.276+0000 7f53c9ffb700 1 --2- 172.21.15.92:0/3720953873 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f53b8066f30 0x7f53b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:49.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.276+0000 7f53c9ffb700 1 -- 172.21.15.92:0/3720953873 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f53c4083cf0 con 0x7f53d40b29f0 2023-10-06T10:43:49.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.276+0000 7f53d9c7b700 1 --2- 172.21.15.92:0/3720953873 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f53b8066f30 0x7f53b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:49.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.277+0000 7f53d9c7b700 1 --2- 172.21.15.92:0/3720953873 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f53b8066f30 0x7f53b80693e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009880 tx=0x7f53d0018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:49.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.281+0000 7f53c9ffb700 1 -- 172.21.15.92:0/3720953873 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53d415b700 con 0x7f53d40b29f0 2023-10-06T10:43:49.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:49.508+0000 7f53dac7d700 1 -- 172.21.15.92:0/3720953873 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f53d405f230 con 0x7f53d40b29f0 2023-10-06T10:43:50.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.499+0000 7f53c9ffb700 1 -- 172.21.15.92:0/3720953873 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v82) v1 ==== 118+0+0 (secure 0 0 0) 0x7f53c4017070 con 0x7f53d40b29f0 2023-10-06T10:43:50.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.501+0000 7f53dac7d700 1 -- 172.21.15.92:0/3720953873 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f53b8066f30 msgr2=0x7f53b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:50.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.501+0000 7f53dac7d700 1 --2- 172.21.15.92:0/3720953873 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f53b8066f30 0x7f53b80693e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009880 tx=0x7f53d0018000 comp rx=0 tx=0).stop 2023-10-06T10:43:50.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.501+0000 7f53dac7d700 1 -- 172.21.15.92:0/3720953873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d40b29f0 msgr2=0x7f53d415d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:50.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.501+0000 7f53dac7d700 1 --2- 172.21.15.92:0/3720953873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d40b29f0 0x7f53d415d180 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f53c400c7c0 tx=0x7f53c4007f50 comp rx=0 tx=0).stop 2023-10-06T10:43:50.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.501+0000 7f53dac7d700 1 -- 172.21.15.92:0/3720953873 shutdown_connections 2023-10-06T10:43:50.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.501+0000 7f53dac7d700 1 --2- 172.21.15.92:0/3720953873 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f53b8066f30 0x7f53b80693e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:50.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.501+0000 7f53dac7d700 1 --2- 172.21.15.92:0/3720953873 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53d40cf1b0 0x7f53d415d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:50.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.501+0000 7f53dac7d700 1 --2- 172.21.15.92:0/3720953873 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53d40b2620 0x7f53d4157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:50.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.501+0000 7f53dac7d700 1 --2- 172.21.15.92:0/3720953873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d40b29f0 0x7f53d415d180 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:50.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.501+0000 7f53dac7d700 1 -- 172.21.15.92:0/3720953873 >> 172.21.15.92:0/3720953873 conn(0x7f53d4099520 msgr2=0x7f53d40693f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:50.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.502+0000 7f53dac7d700 1 -- 172.21.15.92:0/3720953873 shutdown_connections 2023-10-06T10:43:50.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.502+0000 7f53dac7d700 1 -- 172.21.15.92:0/3720953873 wait complete. 2023-10-06T10:43:50.517 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T10:43:50.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.639+0000 7f36f0558700 1 Processor -- start 2023-10-06T10:43:50.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.639+0000 7f36f0558700 1 -- start start 2023-10-06T10:43:50.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.640+0000 7f36f0558700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f36e8068160 0x7f36e80c8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:50.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.640+0000 7f36f0558700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f36e81599e0 0x7f36e80c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:50.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.640+0000 7f36f0558700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e8069ab0 con 0x7f36e8159610 2023-10-06T10:43:50.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.640+0000 7f36f0558700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e80c3220 con 0x7f36e81599e0 2023-10-06T10:43:50.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.640+0000 7f36f0558700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e80c3500 con 0x7f36e8068160 2023-10-06T10:43:50.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.641+0000 7f36eeaf5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f36e81599e0 0x7f36e80c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:50.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.641+0000 7f36eeaf5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f36e81599e0 0x7f36e80c3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44930/0 (socket says 172.21.15.92:44930) 2023-10-06T10:43:50.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.641+0000 7f36eeaf5700 1 -- 172.21.15.92:0/2814741714 learned_addr learned my addr 172.21.15.92:0/2814741714 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:50.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.641+0000 7f36edaf3700 1 --2- 172.21.15.92:0/2814741714 >> v2:172.21.15.124:3300/0 conn(0x7f36e8068160 0x7f36e80c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:50.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.641+0000 7f36eeaf5700 1 -- 172.21.15.92:0/2814741714 >> v2:172.21.15.124:3300/0 conn(0x7f36e8068160 msgr2=0x7f36e80c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:50.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.641+0000 7f36eeaf5700 1 --2- 172.21.15.92:0/2814741714 >> v2:172.21.15.124:3300/0 conn(0x7f36e8068160 0x7f36e80c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:50.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.641+0000 7f36eeaf5700 1 -- 172.21.15.92:0/2814741714 >> v1:172.21.15.124:6789/0 conn(0x7f36e8159610 legacy=0x7f36e80cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:50.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.641+0000 7f36eeaf5700 1 -- 172.21.15.92:0/2814741714 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e80c37e0 con 0x7f36e81599e0 2023-10-06T10:43:50.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.641+0000 7f36edaf3700 1 --2- 172.21.15.92:0/2814741714 >> v2:172.21.15.124:3300/0 conn(0x7f36e8068160 0x7f36e80c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:50.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.641+0000 7f36eeaf5700 1 --2- 172.21.15.92:0/2814741714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e81599e0 0x7f36e80c3d00 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f36e806ed90 tx=0x7f36e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c9d9c8f4cb2a2e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:50.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.641+0000 7f36ed2f2700 1 -- 172.21.15.92:0/2814741714 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e000e070 con 0x7f36e81599e0 2023-10-06T10:43:50.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.641+0000 7f36ed2f2700 1 -- 172.21.15.92:0/2814741714 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e000ba50 con 0x7f36e81599e0 2023-10-06T10:43:50.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.641+0000 7f36ed2f2700 1 -- 172.21.15.92:0/2814741714 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e00092e0 con 0x7f36e81599e0 2023-10-06T10:43:50.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.642+0000 7f36f0558700 1 -- 172.21.15.92:0/2814741714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e81599e0 msgr2=0x7f36e80c3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:50.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.642+0000 7f36f0558700 1 --2- 172.21.15.92:0/2814741714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e81599e0 0x7f36e80c3d00 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f36e806ed90 tx=0x7f36e0009840 comp rx=0 tx=0).stop 2023-10-06T10:43:50.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.642+0000 7f36f0558700 1 -- 172.21.15.92:0/2814741714 shutdown_connections 2023-10-06T10:43:50.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.642+0000 7f36f0558700 1 --2- 172.21.15.92:0/2814741714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e81599e0 0x7f36e80c3d00 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:50.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.642+0000 7f36f0558700 1 --2- 172.21.15.92:0/2814741714 >> v2:172.21.15.124:3300/0 conn(0x7f36e8068160 0x7f36e80c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:50.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.642+0000 7f36f0558700 1 -- 172.21.15.92:0/2814741714 >> 172.21.15.92:0/2814741714 conn(0x7f36e8067d90 msgr2=0x7f36e80ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:50.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.642+0000 7f36f0558700 1 -- 172.21.15.92:0/2814741714 shutdown_connections 2023-10-06T10:43:50.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.642+0000 7f36f0558700 1 -- 172.21.15.92:0/2814741714 wait complete. 2023-10-06T10:43:50.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.643+0000 7f36f0558700 1 Processor -- start 2023-10-06T10:43:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.643+0000 7f36f0558700 1 -- start start 2023-10-06T10:43:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.644+0000 7f36f0558700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36e8068160 0x7f36e8153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.644+0000 7f36f0558700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36e8159610 0x7f36e80686c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.644+0000 7f36f0558700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e81599e0 0x7f36e8068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.644+0000 7f36f0558700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e80c5dc0 con 0x7f36e81599e0 2023-10-06T10:43:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.644+0000 7f36f0558700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e80c5fb0 con 0x7f36e8068160 2023-10-06T10:43:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.644+0000 7f36f0558700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e80c6290 con 0x7f36e8159610 2023-10-06T10:43:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.644+0000 7f36eeaf5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e81599e0 0x7f36e8068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.644+0000 7f36eeaf5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e81599e0 0x7f36e8068c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44938/0 (socket says 172.21.15.92:44938) 2023-10-06T10:43:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.644+0000 7f36eeaf5700 1 -- 172.21.15.92:0/3083687953 learned_addr learned my addr 172.21.15.92:0/3083687953 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.644+0000 7f36edaf3700 1 --2- 172.21.15.92:0/3083687953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36e8159610 0x7f36e80686c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.644+0000 7f36ee2f4700 1 --2- 172.21.15.92:0/3083687953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36e8068160 0x7f36e8153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.644+0000 7f36eeaf5700 1 -- 172.21.15.92:0/3083687953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36e8159610 msgr2=0x7f36e80686c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.644+0000 7f36eeaf5700 1 --2- 172.21.15.92:0/3083687953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36e8159610 0x7f36e80686c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.644+0000 7f36eeaf5700 1 -- 172.21.15.92:0/3083687953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36e8068160 msgr2=0x7f36e8153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.644+0000 7f36eeaf5700 1 --2- 172.21.15.92:0/3083687953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36e8068160 0x7f36e8153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.644+0000 7f36eeaf5700 1 -- 172.21.15.92:0/3083687953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e0009ae0 con 0x7f36e81599e0 2023-10-06T10:43:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.644+0000 7f36eeaf5700 1 --2- 172.21.15.92:0/3083687953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e81599e0 0x7f36e8068c00 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f36e806e7d0 tx=0x7f36e0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.644+0000 7f36de7fc700 1 -- 172.21.15.92:0/3083687953 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0008d20 con 0x7f36e81599e0 2023-10-06T10:43:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.644+0000 7f36de7fc700 1 -- 172.21.15.92:0/3083687953 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e0026460 con 0x7f36e81599e0 2023-10-06T10:43:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.644+0000 7f36f0558700 1 -- 172.21.15.92:0/3083687953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e80c6570 con 0x7f36e81599e0 2023-10-06T10:43:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.644+0000 7f36de7fc700 1 -- 172.21.15.92:0/3083687953 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0008ea0 con 0x7f36e81599e0 2023-10-06T10:43:50.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.644+0000 7f36f0558700 1 -- 172.21.15.92:0/3083687953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e80c6b30 con 0x7f36e81599e0 2023-10-06T10:43:50.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.645+0000 7f36f0558700 1 -- 172.21.15.92:0/3083687953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e8064a90 con 0x7f36e81599e0 2023-10-06T10:43:50.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.646+0000 7f36de7fc700 1 -- 172.21.15.92:0/3083687953 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f36e0003040 con 0x7f36e81599e0 2023-10-06T10:43:50.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.646+0000 7f36de7fc700 1 --2- 172.21.15.92:0/3083687953 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f36cc066f30 0x7f36cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:50.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.646+0000 7f36ee2f4700 1 --2- 172.21.15.92:0/3083687953 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f36cc066f30 0x7f36cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:50.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.646+0000 7f36de7fc700 1 -- 172.21.15.92:0/3083687953 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f36e00867c0 con 0x7f36e81599e0 2023-10-06T10:43:50.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.647+0000 7f36ee2f4700 1 --2- 172.21.15.92:0/3083687953 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f36cc066f30 0x7f36cc0693e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f36e4003a20 tx=0x7f36e4007f10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:50.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.651+0000 7f36de7fc700 1 -- 172.21.15.92:0/3083687953 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36e0053840 con 0x7f36e81599e0 2023-10-06T10:43:50.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:50.875+0000 7f36f0558700 1 -- 172.21.15.92:0/3083687953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f36e8069d90 con 0x7f36e81599e0 2023-10-06T10:43:51.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.500+0000 7f36de7fc700 1 -- 172.21.15.92:0/3083687953 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v83) v1 ==== 122+0+0 (secure 0 0 0) 0x7f36e0084e60 con 0x7f36e81599e0 2023-10-06T10:43:51.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.502+0000 7f36f0558700 1 -- 172.21.15.92:0/3083687953 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f36cc066f30 msgr2=0x7f36cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:51.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.502+0000 7f36f0558700 1 --2- 172.21.15.92:0/3083687953 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f36cc066f30 0x7f36cc0693e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f36e4003a20 tx=0x7f36e4007f10 comp rx=0 tx=0).stop 2023-10-06T10:43:51.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.502+0000 7f36f0558700 1 -- 172.21.15.92:0/3083687953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e81599e0 msgr2=0x7f36e8068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:51.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.502+0000 7f36f0558700 1 --2- 172.21.15.92:0/3083687953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e81599e0 0x7f36e8068c00 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f36e806e7d0 tx=0x7f36e0018000 comp rx=0 tx=0).stop 2023-10-06T10:43:51.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.502+0000 7f36f0558700 1 -- 172.21.15.92:0/3083687953 shutdown_connections 2023-10-06T10:43:51.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.502+0000 7f36f0558700 1 --2- 172.21.15.92:0/3083687953 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f36cc066f30 0x7f36cc0693e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:51.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.502+0000 7f36f0558700 1 --2- 172.21.15.92:0/3083687953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e81599e0 0x7f36e8068c00 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:51.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.502+0000 7f36f0558700 1 --2- 172.21.15.92:0/3083687953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36e8068160 0x7f36e8153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:51.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.502+0000 7f36f0558700 1 --2- 172.21.15.92:0/3083687953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36e8159610 0x7f36e80686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:51.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.502+0000 7f36f0558700 1 -- 172.21.15.92:0/3083687953 >> 172.21.15.92:0/3083687953 conn(0x7f36e8067d90 msgr2=0x7f36e81588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:51.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.502+0000 7f36f0558700 1 -- 172.21.15.92:0/3083687953 shutdown_connections 2023-10-06T10:43:51.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.502+0000 7f36f0558700 1 -- 172.21.15.92:0/3083687953 wait complete. 2023-10-06T10:43:51.515 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:43:51.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.628+0000 7f8557c7d700 1 Processor -- start 2023-10-06T10:43:51.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.629+0000 7f8557c7d700 1 -- start start 2023-10-06T10:43:51.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.629+0000 7f8557c7d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85500ca370 0x7f8550158270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:51.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.629+0000 7f8557c7d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8550157de0 con 0x7f85500adb60 2023-10-06T10:43:51.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.629+0000 7f8557c7d700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85500cb5d0 con 0x7f85500ce8a0 2023-10-06T10:43:51.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.629+0000 7f8557c7d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85500cbb90 con 0x7f85500ca370 2023-10-06T10:43:51.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.630+0000 7f8555218700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f85500adb60 0x7f85500ce190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37584/0 (socket says 172.21.15.92:37584) 2023-10-06T10:43:51.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.630+0000 7f8555218700 1 -- 172.21.15.92:0/2846073914 learned_addr learned my addr 172.21.15.92:0/2846073914 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:51.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.630+0000 7f8555a19700 1 --2- 172.21.15.92:0/2846073914 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85500ca370 0x7f8550158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:51.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.630+0000 7f8554a17700 1 -- 172.21.15.92:0/2846073914 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1337856935 0 0) 0x7f8550157de0 con 0x7f85500adb60 2023-10-06T10:43:51.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.630+0000 7f8554a17700 1 -- 172.21.15.92:0/2846073914 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f853c003570 con 0x7f85500adb60 2023-10-06T10:43:51.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.630+0000 7f8554a17700 1 -- 172.21.15.92:0/2846073914 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1475624456 0 0) 0x7f853c003570 con 0x7f85500adb60 2023-10-06T10:43:51.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.630+0000 7f8554a17700 1 -- 172.21.15.92:0/2846073914 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85500ca370 msgr2=0x7f8550158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:51.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.630+0000 7f8554a17700 1 --2- 172.21.15.92:0/2846073914 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85500ca370 0x7f8550158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:51.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.630+0000 7f8554a17700 1 -- 172.21.15.92:0/2846073914 >> v1:172.21.15.124:6789/0 conn(0x7f85500ce8a0 legacy=0x7f855015aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:51.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.630+0000 7f8554a17700 1 -- 172.21.15.92:0/2846073914 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85500cb8b0 con 0x7f85500adb60 2023-10-06T10:43:51.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.630+0000 7f8554a17700 1 -- 172.21.15.92:0/2846073914 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f8540001470 con 0x7f85500adb60 2023-10-06T10:43:51.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.630+0000 7f8554a17700 1 -- 172.21.15.92:0/2846073914 >> v1:172.21.15.92:6789/0 conn(0x7f85500adb60 legacy=0x7f85500ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:51.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.630+0000 7f8554a17700 1 --2- 172.21.15.92:0/2846073914 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f853c003a90 0x7f853c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:51.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.630+0000 7f8554a17700 1 --2- 172.21.15.92:0/2846073914 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85500ca370 0x7f853c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:51.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.630+0000 7f8554a17700 1 --2- 172.21.15.92:0/2846073914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f853c004d50 0x7f853c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:51.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.630+0000 7f8554a17700 1 -- 172.21.15.92:0/2846073914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85500cbb90 con 0x7f853c004d50 2023-10-06T10:43:51.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.630+0000 7f8554a17700 1 -- 172.21.15.92:0/2846073914 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c00bb10 con 0x7f853c003a90 2023-10-06T10:43:51.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.630+0000 7f8554a17700 1 -- 172.21.15.92:0/2846073914 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853c00bcd0 con 0x7f85500ca370 2023-10-06T10:43:51.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.630+0000 7f855621a700 1 --2- 172.21.15.92:0/2846073914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f853c004d50 0x7f853c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:51.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.630+0000 7f8555a19700 1 --2- 172.21.15.92:0/2846073914 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f853c003a90 0x7f853c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.630+0000 7f8555218700 1 --2- 172.21.15.92:0/2846073914 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85500ca370 0x7f853c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.631+0000 7f855621a700 1 -- 172.21.15.92:0/2846073914 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85500ca370 msgr2=0x7f853c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.631+0000 7f855621a700 1 --2- 172.21.15.92:0/2846073914 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85500ca370 0x7f853c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.631+0000 7f855621a700 1 -- 172.21.15.92:0/2846073914 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f853c003a90 msgr2=0x7f853c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.631+0000 7f855621a700 1 --2- 172.21.15.92:0/2846073914 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f853c003a90 0x7f853c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.631+0000 7f855621a700 1 -- 172.21.15.92:0/2846073914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853c003570 con 0x7f853c004d50 2023-10-06T10:43:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.631+0000 7f855621a700 1 --2- 172.21.15.92:0/2846073914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f853c004d50 0x7f853c0051c0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f8548002070 tx=0x7f85480016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e40c4c7e36b9a7f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.631+0000 7f8554a17700 1 -- 172.21.15.92:0/2846073914 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85480019e0 con 0x7f853c004d50 2023-10-06T10:43:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.631+0000 7f8554a17700 1 -- 172.21.15.92:0/2846073914 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854800c8b0 con 0x7f853c004d50 2023-10-06T10:43:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.631+0000 7f8554a17700 1 -- 172.21.15.92:0/2846073914 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854800cc50 con 0x7f853c004d50 2023-10-06T10:43:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.631+0000 7f8557c7d700 1 -- 172.21.15.92:0/2846073914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f853c004d50 msgr2=0x7f853c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.631+0000 7f8557c7d700 1 --2- 172.21.15.92:0/2846073914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f853c004d50 0x7f853c0051c0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f8548002070 tx=0x7f85480016d0 comp rx=0 tx=0).stop 2023-10-06T10:43:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.631+0000 7f855621a700 1 -- 172.21.15.92:0/2846073914 reap_dead start 2023-10-06T10:43:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.631+0000 7f8557c7d700 1 -- 172.21.15.92:0/2846073914 shutdown_connections 2023-10-06T10:43:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.631+0000 7f8557c7d700 1 -- 172.21.15.92:0/2846073914 >> 172.21.15.92:0/2846073914 conn(0x7f85500707d0 msgr2=0x7f8550070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.632+0000 7f8557c7d700 1 -- 172.21.15.92:0/2846073914 shutdown_connections 2023-10-06T10:43:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.632+0000 7f8557c7d700 1 -- 172.21.15.92:0/2846073914 wait complete. 2023-10-06T10:43:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.633+0000 7f8557c7d700 1 Processor -- start 2023-10-06T10:43:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.633+0000 7f8557c7d700 1 -- start start 2023-10-06T10:43:51.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.633+0000 7f8557c7d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f853c004d50 0x7f85500b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:51.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.633+0000 7f8557c7d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85501522c0 0x7f85500b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:51.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.633+0000 7f8557c7d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85500b1f10 0x7f855016ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:51.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.633+0000 7f8557c7d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85500b3c50 con 0x7f85500b1f10 2023-10-06T10:43:51.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.633+0000 7f8557c7d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85500b2950 con 0x7f85501522c0 2023-10-06T10:43:51.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.633+0000 7f8557c7d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85500b2c30 con 0x7f853c004d50 2023-10-06T10:43:51.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.633+0000 7f855621a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85500b1f10 0x7f855016ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:51.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.633+0000 7f855621a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85500b1f10 0x7f855016ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44956/0 (socket says 172.21.15.92:44956) 2023-10-06T10:43:51.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.633+0000 7f855621a700 1 -- 172.21.15.92:0/2661618808 learned_addr learned my addr 172.21.15.92:0/2661618808 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:51.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.633+0000 7f8555a19700 1 --2- 172.21.15.92:0/2661618808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f853c004d50 0x7f85500b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:51.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.633+0000 7f8555218700 1 --2- 172.21.15.92:0/2661618808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85501522c0 0x7f85500b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:51.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.634+0000 7f855621a700 1 -- 172.21.15.92:0/2661618808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f853c004d50 msgr2=0x7f85500b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:51.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.634+0000 7f855621a700 1 --2- 172.21.15.92:0/2661618808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f853c004d50 0x7f85500b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:51.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.634+0000 7f855621a700 1 -- 172.21.15.92:0/2661618808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85501522c0 msgr2=0x7f85500b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:51.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.634+0000 7f855621a700 1 --2- 172.21.15.92:0/2661618808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85501522c0 0x7f85500b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:51.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.634+0000 7f855621a700 1 -- 172.21.15.92:0/2661618808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85500b2f10 con 0x7f85500b1f10 2023-10-06T10:43:51.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.634+0000 7f855621a700 1 --2- 172.21.15.92:0/2661618808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85500b1f10 0x7f855016ad20 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f8548004000 tx=0x7f8548015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:51.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.634+0000 7f8545ffb700 1 -- 172.21.15.92:0/2661618808 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85480019e0 con 0x7f85500b1f10 2023-10-06T10:43:51.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.634+0000 7f8545ffb700 1 -- 172.21.15.92:0/2661618808 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854800c6d0 con 0x7f85500b1f10 2023-10-06T10:43:51.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.634+0000 7f8545ffb700 1 -- 172.21.15.92:0/2661618808 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548006db0 con 0x7f85500b1f10 2023-10-06T10:43:51.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.634+0000 7f8557c7d700 1 -- 172.21.15.92:0/2661618808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85500b31f0 con 0x7f85500b1f10 2023-10-06T10:43:51.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.634+0000 7f8557c7d700 1 -- 172.21.15.92:0/2661618808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855016bc20 con 0x7f85500b1f10 2023-10-06T10:43:51.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.635+0000 7f8557c7d700 1 -- 172.21.15.92:0/2661618808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85500b3410 con 0x7f85500b1f10 2023-10-06T10:43:51.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.635+0000 7f8545ffb700 1 -- 172.21.15.92:0/2661618808 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f854801f620 con 0x7f85500b1f10 2023-10-06T10:43:51.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.636+0000 7f8545ffb700 1 --2- 172.21.15.92:0/2661618808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8534066f30 0x7f85340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:51.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.636+0000 7f8545ffb700 1 -- 172.21.15.92:0/2661618808 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8548085a70 con 0x7f85500b1f10 2023-10-06T10:43:51.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.636+0000 7f8555a19700 1 --2- 172.21.15.92:0/2661618808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8534066f30 0x7f85340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:51.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.636+0000 7f8555a19700 1 --2- 172.21.15.92:0/2661618808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8534066f30 0x7f85340693e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f854c006f00 tx=0x7f854c0091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:51.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.640+0000 7f8545ffb700 1 -- 172.21.15.92:0/2661618808 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f85480529a0 con 0x7f85500b1f10 2023-10-06T10:43:51.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.864+0000 7f8557c7d700 1 -- 172.21.15.92:0/2661618808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f855016bc20 con 0x7f85500b1f10 2023-10-06T10:43:51.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.864+0000 7f8545ffb700 1 -- 172.21.15.92:0/2661618808 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f855016bc20 con 0x7f85500b1f10 2023-10-06T10:43:51.865 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:51.865 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":83,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2122441982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2122441982},{"type":"v1","addr":"172.21.15.124:6835","nonce":2122441982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78},{"gid":7686,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3605297930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3605297930},{"type":"v1","addr":"172.21.15.92:6836","nonce":3605297930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78},{"gid":7689,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/4047304116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4047304116},{"type":"v1","addr":"172.21.15.92:6837","nonce":4047304116}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:47.850832+0000","modified":"2023-10-06T10:43:51.497396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7701":{"gid":7701,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6834/4203118992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":4203118992},{"type":"v1","addr":"172.21.15.124:6834","nonce":4203118992}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T10:43:51.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.866+0000 7f8557c7d700 1 -- 172.21.15.92:0/2661618808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8534066f30 msgr2=0x7f85340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:51.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.866+0000 7f8557c7d700 1 --2- 172.21.15.92:0/2661618808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8534066f30 0x7f85340693e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f854c006f00 tx=0x7f854c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:51.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.866+0000 7f8557c7d700 1 -- 172.21.15.92:0/2661618808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85500b1f10 msgr2=0x7f855016ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:51.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.866+0000 7f8557c7d700 1 --2- 172.21.15.92:0/2661618808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85500b1f10 0x7f855016ad20 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f8548004000 tx=0x7f8548015000 comp rx=0 tx=0).stop 2023-10-06T10:43:51.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.866+0000 7f8557c7d700 1 -- 172.21.15.92:0/2661618808 shutdown_connections 2023-10-06T10:43:51.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.866+0000 7f8557c7d700 1 --2- 172.21.15.92:0/2661618808 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8534066f30 0x7f85340693e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:51.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.866+0000 7f8557c7d700 1 --2- 172.21.15.92:0/2661618808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85500b1f10 0x7f855016ad20 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:51.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.866+0000 7f8557c7d700 1 --2- 172.21.15.92:0/2661618808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f853c004d50 0x7f85500b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:51.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.866+0000 7f8557c7d700 1 --2- 172.21.15.92:0/2661618808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85501522c0 0x7f85500b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:51.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.867+0000 7f8557c7d700 1 -- 172.21.15.92:0/2661618808 >> 172.21.15.92:0/2661618808 conn(0x7f85500707d0 msgr2=0x7f855015a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:51.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.867+0000 7f8557c7d700 1 -- 172.21.15.92:0/2661618808 shutdown_connections 2023-10-06T10:43:51.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.867+0000 7f8557c7d700 1 -- 172.21.15.92:0/2661618808 wait complete. 2023-10-06T10:43:51.869 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 83 2023-10-06T10:43:51.879 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:43:51.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.992+0000 7f957d285700 1 Processor -- start 2023-10-06T10:43:51.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.992+0000 7f957d285700 1 -- start start 2023-10-06T10:43:51.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.993+0000 7f957d285700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f95780ca370 0x7f9578158270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:51.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.993+0000 7f957d285700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f95780ce8a0 0x7f957815aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:51.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.993+0000 7f957d285700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9578157de0 con 0x7f95780adb60 2023-10-06T10:43:51.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.993+0000 7f957d285700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95780cbb90 con 0x7f95780ce8a0 2023-10-06T10:43:51.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.993+0000 7f957d285700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95780cb5d0 con 0x7f95780ca370 2023-10-06T10:43:51.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.993+0000 7f95777fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f95780ce8a0 0x7f957815aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:51.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.993+0000 7f95777fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f95780ce8a0 0x7f957815aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44968/0 (socket says 172.21.15.92:44968) 2023-10-06T10:43:51.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.993+0000 7f95777fe700 1 -- 172.21.15.92:0/2666824660 learned_addr learned my addr 172.21.15.92:0/2666824660 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:51.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.993+0000 7f9576ffd700 1 --2- 172.21.15.92:0/2666824660 >> v2:172.21.15.124:3300/0 conn(0x7f95780ca370 0x7f9578158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:51.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.994+0000 7f95777fe700 1 -- 172.21.15.92:0/2666824660 >> v2:172.21.15.124:3300/0 conn(0x7f95780ca370 msgr2=0x7f9578158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:51.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.994+0000 7f95777fe700 1 --2- 172.21.15.92:0/2666824660 >> v2:172.21.15.124:3300/0 conn(0x7f95780ca370 0x7f9578158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:51.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.994+0000 7f95777fe700 1 -- 172.21.15.92:0/2666824660 >> v1:172.21.15.124:6789/0 conn(0x7f95780adb60 legacy=0x7f95780ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:51.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.994+0000 7f95777fe700 1 -- 172.21.15.92:0/2666824660 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95780cb8b0 con 0x7f95780ce8a0 2023-10-06T10:43:51.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.994+0000 7f95767fc700 1 -- 172.21.15.92:0/2666824660 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473174783 0 0) 0x7f9578157de0 con 0x7f95780adb60 2023-10-06T10:43:51.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.994+0000 7f95777fe700 1 --2- 172.21.15.92:0/2666824660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95780ce8a0 0x7f957815aed0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f957806f2c0 tx=0x7f9570009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166fd157b472e41b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:51.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.994+0000 7f95767fc700 1 -- 172.21.15.92:0/2666824660 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957000e070 con 0x7f95780ce8a0 2023-10-06T10:43:51.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.994+0000 7f95767fc700 1 -- 172.21.15.92:0/2666824660 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957000ba20 con 0x7f95780ce8a0 2023-10-06T10:43:51.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.994+0000 7f95767fc700 1 -- 172.21.15.92:0/2666824660 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95700092e0 con 0x7f95780ce8a0 2023-10-06T10:43:51.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.994+0000 7f957d285700 1 -- 172.21.15.92:0/2666824660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95780ce8a0 msgr2=0x7f957815aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:51.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.994+0000 7f957d285700 1 --2- 172.21.15.92:0/2666824660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95780ce8a0 0x7f957815aed0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f957806f2c0 tx=0x7f9570009840 comp rx=0 tx=0).stop 2023-10-06T10:43:51.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.995+0000 7f957d285700 1 -- 172.21.15.92:0/2666824660 shutdown_connections 2023-10-06T10:43:51.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.995+0000 7f957d285700 1 --2- 172.21.15.92:0/2666824660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95780ce8a0 0x7f957815aed0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:51.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.995+0000 7f957d285700 1 --2- 172.21.15.92:0/2666824660 >> v2:172.21.15.124:3300/0 conn(0x7f95780ca370 0x7f9578158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:51.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.995+0000 7f957d285700 1 -- 172.21.15.92:0/2666824660 >> 172.21.15.92:0/2666824660 conn(0x7f95780707d0 msgr2=0x7f9578070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:51.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.995+0000 7f957d285700 1 -- 172.21.15.92:0/2666824660 shutdown_connections 2023-10-06T10:43:51.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.995+0000 7f957d285700 1 -- 172.21.15.92:0/2666824660 wait complete. 2023-10-06T10:43:52.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.996+0000 7f957d285700 1 Processor -- start 2023-10-06T10:43:52.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.996+0000 7f957d285700 1 -- start start 2023-10-06T10:43:52.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.997+0000 7f957d285700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95780adb60 0x7f95780b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:52.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.997+0000 7f957d285700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95780ca370 0x7f95780c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:52.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.997+0000 7f957d285700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95780ce8a0 0x7f9578168570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:52.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.997+0000 7f957d285700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95780c2fc0 con 0x7f95780ca370 2023-10-06T10:43:52.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.997+0000 7f957d285700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957814c0a0 con 0x7f95780ce8a0 2023-10-06T10:43:52.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.997+0000 7f957d285700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95780c2720 con 0x7f95780adb60 2023-10-06T10:43:52.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.997+0000 7f956e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95780ca370 0x7f95780c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:52.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.997+0000 7f956e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95780ca370 0x7f95780c4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44984/0 (socket says 172.21.15.92:44984) 2023-10-06T10:43:52.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.997+0000 7f956e7fc700 1 -- 172.21.15.92:0/4130315446 learned_addr learned my addr 172.21.15.92:0/4130315446 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:52.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.997+0000 7f9576ffd700 1 --2- 172.21.15.92:0/4130315446 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95780adb60 0x7f95780b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:52.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.997+0000 7f95777fe700 1 --2- 172.21.15.92:0/4130315446 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95780ce8a0 0x7f9578168570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:52.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.997+0000 7f956e7fc700 1 -- 172.21.15.92:0/4130315446 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95780adb60 msgr2=0x7f95780b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:52.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.997+0000 7f956e7fc700 1 --2- 172.21.15.92:0/4130315446 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95780adb60 0x7f95780b3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.997+0000 7f956e7fc700 1 -- 172.21.15.92:0/4130315446 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95780ce8a0 msgr2=0x7f9578168570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:52.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.997+0000 7f956e7fc700 1 --2- 172.21.15.92:0/4130315446 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95780ce8a0 0x7f9578168570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.997+0000 7f956e7fc700 1 -- 172.21.15.92:0/4130315446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9570009ae0 con 0x7f95780ca370 2023-10-06T10:43:52.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.997+0000 7f956e7fc700 1 --2- 172.21.15.92:0/4130315446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95780ca370 0x7f95780c4390 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f956000c560 tx=0x7f9560007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:52.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.997+0000 7f956f7fe700 1 -- 172.21.15.92:0/4130315446 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560011590 con 0x7f95780ca370 2023-10-06T10:43:52.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.997+0000 7f956f7fe700 1 -- 172.21.15.92:0/4130315446 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9560011710 con 0x7f95780ca370 2023-10-06T10:43:52.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.997+0000 7f957d285700 1 -- 172.21.15.92:0/4130315446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9578168ab0 con 0x7f95780ca370 2023-10-06T10:43:52.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.997+0000 7f956f7fe700 1 -- 172.21.15.92:0/4130315446 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560011ad0 con 0x7f95780ca370 2023-10-06T10:43:52.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.997+0000 7f957d285700 1 -- 172.21.15.92:0/4130315446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95780c2970 con 0x7f95780ca370 2023-10-06T10:43:52.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.998+0000 7f957d285700 1 -- 172.21.15.92:0/4130315446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9578064a90 con 0x7f95780ca370 2023-10-06T10:43:52.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.999+0000 7f956f7fe700 1 -- 172.21.15.92:0/4130315446 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f956001b3b0 con 0x7f95780ca370 2023-10-06T10:43:52.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.999+0000 7f956f7fe700 1 --2- 172.21.15.92:0/4130315446 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9558066f30 0x7f95580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:52.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.999+0000 7f956f7fe700 1 -- 172.21.15.92:0/4130315446 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9560083c10 con 0x7f95780ca370 2023-10-06T10:43:52.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.999+0000 7f9576ffd700 1 --2- 172.21.15.92:0/4130315446 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9558066f30 0x7f95580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:52.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:51.999+0000 7f9576ffd700 1 --2- 172.21.15.92:0/4130315446 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9558066f30 0x7f95580693e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9568009db0 tx=0x7f95680091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:52.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.003+0000 7f956f7fe700 1 -- 172.21.15.92:0/4130315446 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9560050be0 con 0x7f95780ca370 2023-10-06T10:43:52.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.188+0000 7f957d285700 1 -- 172.21.15.92:0/4130315446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9578157a70 con 0x7f95780ca370 2023-10-06T10:43:52.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.189+0000 7f956f7fe700 1 -- 172.21.15.92:0/4130315446 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+17074 (secure 0 0 0) 0x7f9560017070 con 0x7f95780ca370 2023-10-06T10:43:52.190 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:52.190 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":90,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:43:49.334267+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:43:45.492396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-06T10:43:46.811317+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:43:52.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.191+0000 7f957d285700 1 -- 172.21.15.92:0/4130315446 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9558066f30 msgr2=0x7f95580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:52.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.191+0000 7f957d285700 1 --2- 172.21.15.92:0/4130315446 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9558066f30 0x7f95580693e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9568009db0 tx=0x7f95680091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.191+0000 7f957d285700 1 -- 172.21.15.92:0/4130315446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95780ca370 msgr2=0x7f95780c4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:52.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.191+0000 7f957d285700 1 --2- 172.21.15.92:0/4130315446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95780ca370 0x7f95780c4390 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f956000c560 tx=0x7f9560007f50 comp rx=0 tx=0).stop 2023-10-06T10:43:52.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.191+0000 7f957d285700 1 -- 172.21.15.92:0/4130315446 shutdown_connections 2023-10-06T10:43:52.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.191+0000 7f957d285700 1 --2- 172.21.15.92:0/4130315446 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9558066f30 0x7f95580693e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.191+0000 7f957d285700 1 --2- 172.21.15.92:0/4130315446 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95780ce8a0 0x7f9578168570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.191+0000 7f957d285700 1 --2- 172.21.15.92:0/4130315446 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95780adb60 0x7f95780b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.191+0000 7f957d285700 1 --2- 172.21.15.92:0/4130315446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95780ca370 0x7f95780c4390 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.191+0000 7f957d285700 1 -- 172.21.15.92:0/4130315446 >> 172.21.15.92:0/4130315446 conn(0x7f95780707d0 msgr2=0x7f957814d940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:52.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.191+0000 7f957d285700 1 -- 172.21.15.92:0/4130315446 shutdown_connections 2023-10-06T10:43:52.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.191+0000 7f957d285700 1 -- 172.21.15.92:0/4130315446 wait complete. 2023-10-06T10:43:52.205 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T10:43:52.205 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:43:52.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.318+0000 7f01f0674700 1 Processor -- start 2023-10-06T10:43:52.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.318+0000 7f01f0674700 1 -- start start 2023-10-06T10:43:52.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.319+0000 7f01f0674700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f01e8159560 0x7f01e80aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:52.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.319+0000 7f01f0674700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e80b0c60 con 0x7f01e80b0760 2023-10-06T10:43:52.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.319+0000 7f01f0674700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e80b0f40 con 0x7f01e80b49a0 2023-10-06T10:43:52.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.319+0000 7f01f0674700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e80b1220 con 0x7f01e8159560 2023-10-06T10:43:52.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.319+0000 7f01eec11700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f01e80b0760 0x7f01e80b1bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37586/0 (socket says 172.21.15.92:37586) 2023-10-06T10:43:52.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.319+0000 7f01eec11700 1 -- 172.21.15.92:0/1399310158 learned_addr learned my addr 172.21.15.92:0/1399310158 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:52.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.319+0000 7f01ee410700 1 --2- 172.21.15.92:0/1399310158 >> v2:172.21.15.124:3300/0 conn(0x7f01e8159560 0x7f01e80aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:52.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.319+0000 7f01ed40e700 1 -- 172.21.15.92:0/1399310158 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 785305632 0 0) 0x7f01e80b0c60 con 0x7f01e80b0760 2023-10-06T10:43:52.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.319+0000 7f01ed40e700 1 -- 172.21.15.92:0/1399310158 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01d4003570 con 0x7f01e80b0760 2023-10-06T10:43:52.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.319+0000 7f01ed40e700 1 -- 172.21.15.92:0/1399310158 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4063343900 0 0) 0x7f01d4003570 con 0x7f01e80b0760 2023-10-06T10:43:52.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.319+0000 7f01ed40e700 1 -- 172.21.15.92:0/1399310158 >> v2:172.21.15.124:3300/0 conn(0x7f01e8159560 msgr2=0x7f01e80aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:52.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.319+0000 7f01ed40e700 1 --2- 172.21.15.92:0/1399310158 >> v2:172.21.15.124:3300/0 conn(0x7f01e8159560 0x7f01e80aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.319+0000 7f01ed40e700 1 -- 172.21.15.92:0/1399310158 >> v1:172.21.15.124:6789/0 conn(0x7f01e80b49a0 legacy=0x7f01e80aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:52.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.319+0000 7f01ed40e700 1 -- 172.21.15.92:0/1399310158 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e80b1500 con 0x7f01e80b0760 2023-10-06T10:43:52.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.319+0000 7f01ed40e700 1 -- 172.21.15.92:0/1399310158 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f01e4001470 con 0x7f01e80b0760 2023-10-06T10:43:52.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.320+0000 7f01ed40e700 1 -- 172.21.15.92:0/1399310158 >> v1:172.21.15.92:6789/0 conn(0x7f01e80b0760 legacy=0x7f01e80b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:52.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.320+0000 7f01ed40e700 1 --2- 172.21.15.92:0/1399310158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01d4003a60 0x7f01d4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:52.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.320+0000 7f01ed40e700 1 --2- 172.21.15.92:0/1399310158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01d4004810 0x7f01d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:52.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.320+0000 7f01ed40e700 1 --2- 172.21.15.92:0/1399310158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01d40051c0 0x7f01d400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:52.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.320+0000 7f01ed40e700 1 -- 172.21.15.92:0/1399310158 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e80b1220 con 0x7f01d40051c0 2023-10-06T10:43:52.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.320+0000 7f01ed40e700 1 -- 172.21.15.92:0/1399310158 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d400c0d0 con 0x7f01d4004810 2023-10-06T10:43:52.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.320+0000 7f01ed40e700 1 -- 172.21.15.92:0/1399310158 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d400c290 con 0x7f01d4003a60 2023-10-06T10:43:52.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.320+0000 7f01eec11700 1 --2- 172.21.15.92:0/1399310158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01d40051c0 0x7f01d400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:52.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.320+0000 7f01ee410700 1 --2- 172.21.15.92:0/1399310158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01d4003a60 0x7f01d4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:52.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.320+0000 7f01edc0f700 1 --2- 172.21.15.92:0/1399310158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01d4004810 0x7f01d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:52.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.320+0000 7f01eec11700 1 -- 172.21.15.92:0/1399310158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01d4003a60 msgr2=0x7f01d4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:52.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.320+0000 7f01eec11700 1 --2- 172.21.15.92:0/1399310158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01d4003a60 0x7f01d4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.320+0000 7f01eec11700 1 -- 172.21.15.92:0/1399310158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01d4004810 msgr2=0x7f01d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:52.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.320+0000 7f01eec11700 1 --2- 172.21.15.92:0/1399310158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01d4004810 0x7f01d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.320+0000 7f01eec11700 1 -- 172.21.15.92:0/1399310158 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d4003570 con 0x7f01d40051c0 2023-10-06T10:43:52.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.320+0000 7f01eec11700 1 -- 172.21.15.92:0/1399310158 reap_dead start 2023-10-06T10:43:52.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.320+0000 7f01eec11700 1 --2- 172.21.15.92:0/1399310158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01d40051c0 0x7f01d400ba60 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f01e400c3c0 tx=0x7f01e4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c87a935ab06139 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:52.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.320+0000 7f01ed40e700 1 -- 172.21.15.92:0/1399310158 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e400cc90 con 0x7f01d40051c0 2023-10-06T10:43:52.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.320+0000 7f01ed40e700 1 -- 172.21.15.92:0/1399310158 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e400cdf0 con 0x7f01d40051c0 2023-10-06T10:43:52.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.320+0000 7f01ed40e700 1 -- 172.21.15.92:0/1399310158 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4005b50 con 0x7f01d40051c0 2023-10-06T10:43:52.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.321+0000 7f01f0674700 1 -- 172.21.15.92:0/1399310158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01d40051c0 msgr2=0x7f01d400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:52.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.321+0000 7f01f0674700 1 --2- 172.21.15.92:0/1399310158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01d40051c0 0x7f01d400ba60 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f01e400c3c0 tx=0x7f01e4001e50 comp rx=0 tx=0).stop 2023-10-06T10:43:52.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.321+0000 7f01f0674700 1 -- 172.21.15.92:0/1399310158 shutdown_connections 2023-10-06T10:43:52.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.321+0000 7f01f0674700 1 --2- 172.21.15.92:0/1399310158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01d40051c0 0x7f01d400ba60 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.321+0000 7f01f0674700 1 -- 172.21.15.92:0/1399310158 >> 172.21.15.92:0/1399310158 conn(0x7f01e80997f0 msgr2=0x7f01e806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:52.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.321+0000 7f01f0674700 1 -- 172.21.15.92:0/1399310158 shutdown_connections 2023-10-06T10:43:52.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.321+0000 7f01f0674700 1 -- 172.21.15.92:0/1399310158 wait complete. 2023-10-06T10:43:52.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.322+0000 7f01f0674700 1 Processor -- start 2023-10-06T10:43:52.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.322+0000 7f01f0674700 1 -- start start 2023-10-06T10:43:52.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.323+0000 7f01f0674700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01d40051c0 0x7f01e815b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:52.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.323+0000 7f01f0674700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01e8159f90 0x7f01e809b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:52.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.323+0000 7f01f0674700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01e8169870 0x7f01e8169ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:52.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.323+0000 7f01f0674700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e815b110 con 0x7f01e8159f90 2023-10-06T10:43:52.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.323+0000 7f01f0674700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e815a500 con 0x7f01e8169870 2023-10-06T10:43:52.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.323+0000 7f01f0674700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e815a7e0 con 0x7f01d40051c0 2023-10-06T10:43:52.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.323+0000 7f01edc0f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01e8159f90 0x7f01e809b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:52.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.323+0000 7f01edc0f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01e8159f90 0x7f01e809b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45006/0 (socket says 172.21.15.92:45006) 2023-10-06T10:43:52.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.323+0000 7f01edc0f700 1 -- 172.21.15.92:0/1067590571 learned_addr learned my addr 172.21.15.92:0/1067590571 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:52.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.323+0000 7f01ee410700 1 --2- 172.21.15.92:0/1067590571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01d40051c0 0x7f01e815b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:52.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.323+0000 7f01eec11700 1 --2- 172.21.15.92:0/1067590571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01e8169870 0x7f01e8169ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:52.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.323+0000 7f01edc0f700 1 -- 172.21.15.92:0/1067590571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01d40051c0 msgr2=0x7f01e815b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:52.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.323+0000 7f01edc0f700 1 --2- 172.21.15.92:0/1067590571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01d40051c0 0x7f01e815b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.323+0000 7f01edc0f700 1 -- 172.21.15.92:0/1067590571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01e8169870 msgr2=0x7f01e8169ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:52.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.323+0000 7f01edc0f700 1 --2- 172.21.15.92:0/1067590571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01e8169870 0x7f01e8169ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.323+0000 7f01edc0f700 1 -- 172.21.15.92:0/1067590571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e815aac0 con 0x7f01e8159f90 2023-10-06T10:43:52.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.323+0000 7f01edc0f700 1 --2- 172.21.15.92:0/1067590571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01e8159f90 0x7f01e809b1a0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f01d800e570 tx=0x7f01d8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:52.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.323+0000 7f01de7fc700 1 -- 172.21.15.92:0/1067590571 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8012500 con 0x7f01e8159f90 2023-10-06T10:43:52.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.323+0000 7f01de7fc700 1 -- 172.21.15.92:0/1067590571 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d8010040 con 0x7f01e8159f90 2023-10-06T10:43:52.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.323+0000 7f01f0674700 1 -- 172.21.15.92:0/1067590571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e815ada0 con 0x7f01e8159f90 2023-10-06T10:43:52.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.323+0000 7f01de7fc700 1 -- 172.21.15.92:0/1067590571 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d80128b0 con 0x7f01e8159f90 2023-10-06T10:43:52.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.324+0000 7f01f0674700 1 -- 172.21.15.92:0/1067590571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e8158c80 con 0x7f01e8159f90 2023-10-06T10:43:52.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.324+0000 7f01f0674700 1 -- 172.21.15.92:0/1067590571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e8070c40 con 0x7f01e8159f90 2023-10-06T10:43:52.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.325+0000 7f01de7fc700 1 -- 172.21.15.92:0/1067590571 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f01d8012c20 con 0x7f01e8159f90 2023-10-06T10:43:52.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.325+0000 7f01de7fc700 1 --2- 172.21.15.92:0/1067590571 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f01cc066f30 0x7f01cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:52.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.325+0000 7f01de7fc700 1 -- 172.21.15.92:0/1067590571 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f01d8084300 con 0x7f01e8159f90 2023-10-06T10:43:52.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.325+0000 7f01ee410700 1 --2- 172.21.15.92:0/1067590571 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f01cc066f30 0x7f01cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:52.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.326+0000 7f01ee410700 1 --2- 172.21.15.92:0/1067590571 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f01cc066f30 0x7f01cc0693e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f01e00094a0 tx=0x7f01e000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:52.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.329+0000 7f01de7fc700 1 -- 172.21.15.92:0/1067590571 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01d8051380 con 0x7f01e8159f90 2023-10-06T10:43:52.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.507+0000 7f01f0674700 1 -- 172.21.15.92:0/1067590571 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f01e8064a90 con 0x7f01cc066f30 2023-10-06T10:43:52.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.516+0000 7f01de7fc700 1 -- 172.21.15.92:0/1067590571 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311470 (secure 0 0 0) 0x7f01e8064a90 con 0x7f01cc066f30 2023-10-06T10:43:52.520 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:52.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.519+0000 7f01f0674700 1 -- 172.21.15.92:0/1067590571 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f01cc066f30 msgr2=0x7f01cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:52.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.519+0000 7f01f0674700 1 --2- 172.21.15.92:0/1067590571 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f01cc066f30 0x7f01cc0693e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f01e00094a0 tx=0x7f01e000b040 comp rx=0 tx=0).stop 2023-10-06T10:43:52.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.519+0000 7f01f0674700 1 -- 172.21.15.92:0/1067590571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01e8159f90 msgr2=0x7f01e809b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:52.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.519+0000 7f01f0674700 1 --2- 172.21.15.92:0/1067590571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01e8159f90 0x7f01e809b1a0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f01d800e570 tx=0x7f01d8009040 comp rx=0 tx=0).stop 2023-10-06T10:43:52.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.519+0000 7f01f0674700 1 -- 172.21.15.92:0/1067590571 shutdown_connections 2023-10-06T10:43:52.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.519+0000 7f01f0674700 1 --2- 172.21.15.92:0/1067590571 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f01cc066f30 0x7f01cc0693e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.519+0000 7f01f0674700 1 --2- 172.21.15.92:0/1067590571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01e8169870 0x7f01e8169ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.519+0000 7f01f0674700 1 --2- 172.21.15.92:0/1067590571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01d40051c0 0x7f01e815b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.519+0000 7f01f0674700 1 --2- 172.21.15.92:0/1067590571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01e8159f90 0x7f01e809b1a0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.519+0000 7f01f0674700 1 -- 172.21.15.92:0/1067590571 >> 172.21.15.92:0/1067590571 conn(0x7f01e80997f0 msgr2=0x7f01e80cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.519+0000 7f01f0674700 1 -- 172.21.15.92:0/1067590571 shutdown_connections 2023-10-06T10:43:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.519+0000 7f01f0674700 1 -- 172.21.15.92:0/1067590571 wait complete. 2023-10-06T10:43:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:43:52.532 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2023-10-06T10:43:52.155031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2388092,"kb_used_data":9788,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752586628,"statfs":{"total":773094113280,"available":770648707072,"internally_reserved":0,"allocated":10022912,"data_stored":5405539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.697641"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351426+0000","last_change":"2023-10-06T10:43:48.351426+0000","last_active":"2023-10-06T10:43:48.351426+0000","last_peered":"2023-10-06T10:43:48.351426+0000","last_clean":"2023-10-06T10:43:48.351426+0000","last_became_active":"2023-10-06T10:43:48.351185+0000","last_became_peered":"2023-10-06T10:43:48.351185+0000","last_unstale":"2023-10-06T10:43:48.351426+0000","last_undegraded":"2023-10-06T10:43:48.351426+0000","last_fullsized":"2023-10-06T10:43:48.351426+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:36:34.819923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.347665+0000","last_change":"2023-10-06T10:43:48.347665+0000","last_active":"2023-10-06T10:43:48.347665+0000","last_peered":"2023-10-06T10:43:48.347665+0000","last_clean":"2023-10-06T10:43:48.347665+0000","last_became_active":"2023-10-06T10:43:48.347420+0000","last_became_peered":"2023-10-06T10:43:48.347420+0000","last_unstale":"2023-10-06T10:43:48.347665+0000","last_undegraded":"2023-10-06T10:43:48.347665+0000","last_fullsized":"2023-10-06T10:43:48.347665+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:48:04.646078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351418+0000","last_change":"2023-10-06T10:43:48.351418+0000","last_active":"2023-10-06T10:43:48.351418+0000","last_peered":"2023-10-06T10:43:48.351418+0000","last_clean":"2023-10-06T10:43:48.351418+0000","last_became_active":"2023-10-06T10:43:48.351140+0000","last_became_peered":"2023-10-06T10:43:48.351140+0000","last_unstale":"2023-10-06T10:43:48.351418+0000","last_undegraded":"2023-10-06T10:43:48.351418+0000","last_fullsized":"2023-10-06T10:43:48.351418+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:04.984298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.352355+0000","last_change":"2023-10-06T10:43:48.352355+0000","last_active":"2023-10-06T10:43:48.352355+0000","last_peered":"2023-10-06T10:43:48.352355+0000","last_clean":"2023-10-06T10:43:48.352355+0000","last_became_active":"2023-10-06T10:43:48.352104+0000","last_became_peered":"2023-10-06T10:43:48.352104+0000","last_unstale":"2023-10-06T10:43:48.352355+0000","last_undegraded":"2023-10-06T10:43:48.352355+0000","last_fullsized":"2023-10-06T10:43:48.352355+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:44.549060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.355530+0000","last_change":"2023-10-06T10:43:48.355530+0000","last_active":"2023-10-06T10:43:48.355530+0000","last_peered":"2023-10-06T10:43:48.355530+0000","last_clean":"2023-10-06T10:43:48.355530+0000","last_became_active":"2023-10-06T10:43:48.355239+0000","last_became_peered":"2023-10-06T10:43:48.355239+0000","last_unstale":"2023-10-06T10:43:48.355530+0000","last_undegraded":"2023-10-06T10:43:48.355530+0000","last_fullsized":"2023-10-06T10:43:48.355530+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:08.040718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351912+0000","last_change":"2023-10-06T10:43:48.351912+0000","last_active":"2023-10-06T10:43:48.351912+0000","last_peered":"2023-10-06T10:43:48.351912+0000","last_clean":"2023-10-06T10:43:48.351912+0000","last_became_active":"2023-10-06T10:43:48.351212+0000","last_became_peered":"2023-10-06T10:43:48.351212+0000","last_unstale":"2023-10-06T10:43:48.351912+0000","last_undegraded":"2023-10-06T10:43:48.351912+0000","last_fullsized":"2023-10-06T10:43:48.351912+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:57:52.254017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351472+0000","last_change":"2023-10-06T10:43:48.351472+0000","last_active":"2023-10-06T10:43:48.351472+0000","last_peered":"2023-10-06T10:43:48.351472+0000","last_clean":"2023-10-06T10:43:48.351472+0000","last_became_active":"2023-10-06T10:43:48.351200+0000","last_became_peered":"2023-10-06T10:43:48.351200+0000","last_unstale":"2023-10-06T10:43:48.351472+0000","last_undegraded":"2023-10-06T10:43:48.351472+0000","last_fullsized":"2023-10-06T10:43:48.351472+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:37:26.353942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.352913+0000","last_change":"2023-10-06T10:43:48.352913+0000","last_active":"2023-10-06T10:43:48.352913+0000","last_peered":"2023-10-06T10:43:48.352913+0000","last_clean":"2023-10-06T10:43:48.352913+0000","last_became_active":"2023-10-06T10:43:48.352630+0000","last_became_peered":"2023-10-06T10:43:48.352630+0000","last_unstale":"2023-10-06T10:43:48.352913+0000","last_undegraded":"2023-10-06T10:43:48.352913+0000","last_fullsized":"2023-10-06T10:43:48.352913+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:25.393203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.358898+0000","last_change":"2023-10-06T10:43:48.358898+0000","last_active":"2023-10-06T10:43:48.358898+0000","last_peered":"2023-10-06T10:43:48.358898+0000","last_clean":"2023-10-06T10:43:48.358898+0000","last_became_active":"2023-10-06T10:43:48.358638+0000","last_became_peered":"2023-10-06T10:43:48.358638+0000","last_unstale":"2023-10-06T10:43:48.358898+0000","last_undegraded":"2023-10-06T10:43:48.358898+0000","last_fullsized":"2023-10-06T10:43:48.358898+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:28:19.852395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.355333+0000","last_change":"2023-10-06T10:43:48.355333+0000","last_active":"2023-10-06T10:43:48.355333+0000","last_peered":"2023-10-06T10:43:48.355333+0000","last_clean":"2023-10-06T10:43:48.355333+0000","last_became_active":"2023-10-06T10:43:48.355026+0000","last_became_peered":"2023-10-06T10:43:48.355026+0000","last_unstale":"2023-10-06T10:43:48.355333+0000","last_undegraded":"2023-10-06T10:43:48.355333+0000","last_fullsized":"2023-10-06T10:43:48.355333+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:56.460307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.358429+0000","last_change":"2023-10-06T10:43:48.358429+0000","last_active":"2023-10-06T10:43:48.358429+0000","last_peered":"2023-10-06T10:43:48.358429+0000","last_clean":"2023-10-06T10:43:48.358429+0000","last_became_active":"2023-10-06T10:43:48.357519+0000","last_became_peered":"2023-10-06T10:43:48.357519+0000","last_unstale":"2023-10-06T10:43:48.358429+0000","last_undegraded":"2023-10-06T10:43:48.358429+0000","last_fullsized":"2023-10-06T10:43:48.358429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:12:22.078056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.352000+0000","last_change":"2023-10-06T10:43:48.352000+0000","last_active":"2023-10-06T10:43:48.352000+0000","last_peered":"2023-10-06T10:43:48.352000+0000","last_clean":"2023-10-06T10:43:48.352000+0000","last_became_active":"2023-10-06T10:43:48.351741+0000","last_became_peered":"2023-10-06T10:43:48.351741+0000","last_unstale":"2023-10-06T10:43:48.352000+0000","last_undegraded":"2023-10-06T10:43:48.352000+0000","last_fullsized":"2023-10-06T10:43:48.352000+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:41:32.626627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.357087+0000","last_change":"2023-10-06T10:43:48.357087+0000","last_active":"2023-10-06T10:43:48.357087+0000","last_peered":"2023-10-06T10:43:48.357087+0000","last_clean":"2023-10-06T10:43:48.357087+0000","last_became_active":"2023-10-06T10:43:48.356841+0000","last_became_peered":"2023-10-06T10:43:48.356841+0000","last_unstale":"2023-10-06T10:43:48.357087+0000","last_undegraded":"2023-10-06T10:43:48.357087+0000","last_fullsized":"2023-10-06T10:43:48.357087+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:41:15.115031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.352548+0000","last_change":"2023-10-06T10:43:48.352548+0000","last_active":"2023-10-06T10:43:48.352548+0000","last_peered":"2023-10-06T10:43:48.352548+0000","last_clean":"2023-10-06T10:43:48.352548+0000","last_became_active":"2023-10-06T10:43:48.352317+0000","last_became_peered":"2023-10-06T10:43:48.352317+0000","last_unstale":"2023-10-06T10:43:48.352548+0000","last_undegraded":"2023-10-06T10:43:48.352548+0000","last_fullsized":"2023-10-06T10:43:48.352548+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:10.707456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.353097+0000","last_change":"2023-10-06T10:43:48.353097+0000","last_active":"2023-10-06T10:43:48.353097+0000","last_peered":"2023-10-06T10:43:48.353097+0000","last_clean":"2023-10-06T10:43:48.353097+0000","last_became_active":"2023-10-06T10:43:48.352863+0000","last_became_peered":"2023-10-06T10:43:48.352863+0000","last_unstale":"2023-10-06T10:43:48.353097+0000","last_undegraded":"2023-10-06T10:43:48.353097+0000","last_fullsized":"2023-10-06T10:43:48.353097+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:44.794731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.357122+0000","last_change":"2023-10-06T10:43:48.357122+0000","last_active":"2023-10-06T10:43:48.357122+0000","last_peered":"2023-10-06T10:43:48.357122+0000","last_clean":"2023-10-06T10:43:48.357122+0000","last_became_active":"2023-10-06T10:43:48.356789+0000","last_became_peered":"2023-10-06T10:43:48.356789+0000","last_unstale":"2023-10-06T10:43:48.357122+0000","last_undegraded":"2023-10-06T10:43:48.357122+0000","last_fullsized":"2023-10-06T10:43:48.357122+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:57:21.913042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.349105+0000","last_change":"2023-10-06T10:43:48.349105+0000","last_active":"2023-10-06T10:43:48.349105+0000","last_peered":"2023-10-06T10:43:48.349105+0000","last_clean":"2023-10-06T10:43:48.349105+0000","last_became_active":"2023-10-06T10:43:48.348842+0000","last_became_peered":"2023-10-06T10:43:48.348842+0000","last_unstale":"2023-10-06T10:43:48.349105+0000","last_undegraded":"2023-10-06T10:43:48.349105+0000","last_fullsized":"2023-10-06T10:43:48.349105+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:39:00.691728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.352184+0000","last_change":"2023-10-06T10:43:48.352184+0000","last_active":"2023-10-06T10:43:48.352184+0000","last_peered":"2023-10-06T10:43:48.352184+0000","last_clean":"2023-10-06T10:43:48.352184+0000","last_became_active":"2023-10-06T10:43:48.351926+0000","last_became_peered":"2023-10-06T10:43:48.351926+0000","last_unstale":"2023-10-06T10:43:48.352184+0000","last_undegraded":"2023-10-06T10:43:48.352184+0000","last_fullsized":"2023-10-06T10:43:48.352184+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:36.582208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.352025+0000","last_change":"2023-10-06T10:43:48.352025+0000","last_active":"2023-10-06T10:43:48.352025+0000","last_peered":"2023-10-06T10:43:48.352025+0000","last_clean":"2023-10-06T10:43:48.352025+0000","last_became_active":"2023-10-06T10:43:48.351767+0000","last_became_peered":"2023-10-06T10:43:48.351767+0000","last_unstale":"2023-10-06T10:43:48.352025+0000","last_undegraded":"2023-10-06T10:43:48.352025+0000","last_fullsized":"2023-10-06T10:43:48.352025+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:45:02.178691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.349548+0000","last_change":"2023-10-06T10:43:48.349548+0000","last_active":"2023-10-06T10:43:48.349548+0000","last_peered":"2023-10-06T10:43:48.349548+0000","last_clean":"2023-10-06T10:43:48.349548+0000","last_became_active":"2023-10-06T10:43:48.349293+0000","last_became_peered":"2023-10-06T10:43:48.349293+0000","last_unstale":"2023-10-06T10:43:48.349548+0000","last_undegraded":"2023-10-06T10:43:48.349548+0000","last_fullsized":"2023-10-06T10:43:48.349548+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:30.190139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351362+0000","last_change":"2023-10-06T10:43:48.351362+0000","last_active":"2023-10-06T10:43:48.351362+0000","last_peered":"2023-10-06T10:43:48.351362+0000","last_clean":"2023-10-06T10:43:48.351362+0000","last_became_active":"2023-10-06T10:43:48.350937+0000","last_became_peered":"2023-10-06T10:43:48.350937+0000","last_unstale":"2023-10-06T10:43:48.351362+0000","last_undegraded":"2023-10-06T10:43:48.351362+0000","last_fullsized":"2023-10-06T10:43:48.351362+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:22:32.640917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.344804+0000","last_change":"2023-10-06T10:43:48.344804+0000","last_active":"2023-10-06T10:43:48.344804+0000","last_peered":"2023-10-06T10:43:48.344804+0000","last_clean":"2023-10-06T10:43:48.344804+0000","last_became_active":"2023-10-06T10:43:48.344565+0000","last_became_peered":"2023-10-06T10:43:48.344565+0000","last_unstale":"2023-10-06T10:43:48.344804+0000","last_undegraded":"2023-10-06T10:43:48.344804+0000","last_fullsized":"2023-10-06T10:43:48.344804+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:44.794731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.345181+0000","last_change":"2023-10-06T10:43:48.345181+0000","last_active":"2023-10-06T10:43:48.345181+0000","last_peered":"2023-10-06T10:43:48.345181+0000","last_clean":"2023-10-06T10:43:48.345181+0000","last_became_active":"2023-10-06T10:43:48.344909+0000","last_became_peered":"2023-10-06T10:43:48.344909+0000","last_unstale":"2023-10-06T10:43:48.345181+0000","last_undegraded":"2023-10-06T10:43:48.345181+0000","last_fullsized":"2023-10-06T10:43:48.345181+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:11:07.128574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.353490+0000","last_change":"2023-10-06T10:43:48.353490+0000","last_active":"2023-10-06T10:43:48.353490+0000","last_peered":"2023-10-06T10:43:48.353490+0000","last_clean":"2023-10-06T10:43:48.353490+0000","last_became_active":"2023-10-06T10:43:48.352934+0000","last_became_peered":"2023-10-06T10:43:48.352934+0000","last_unstale":"2023-10-06T10:43:48.353490+0000","last_undegraded":"2023-10-06T10:43:48.353490+0000","last_fullsized":"2023-10-06T10:43:48.353490+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:06:22.716853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351676+0000","last_change":"2023-10-06T10:43:48.351676+0000","last_active":"2023-10-06T10:43:48.351676+0000","last_peered":"2023-10-06T10:43:48.351676+0000","last_clean":"2023-10-06T10:43:48.351676+0000","last_became_active":"2023-10-06T10:43:48.351406+0000","last_became_peered":"2023-10-06T10:43:48.351406+0000","last_unstale":"2023-10-06T10:43:48.351676+0000","last_undegraded":"2023-10-06T10:43:48.351676+0000","last_fullsized":"2023-10-06T10:43:48.351676+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:25.567935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.349351+0000","last_change":"2023-10-06T10:43:48.349351+0000","last_active":"2023-10-06T10:43:48.349351+0000","last_peered":"2023-10-06T10:43:48.349351+0000","last_clean":"2023-10-06T10:43:48.349351+0000","last_became_active":"2023-10-06T10:43:48.349110+0000","last_became_peered":"2023-10-06T10:43:48.349110+0000","last_unstale":"2023-10-06T10:43:48.349351+0000","last_undegraded":"2023-10-06T10:43:48.349351+0000","last_fullsized":"2023-10-06T10:43:48.349351+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:39.267032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351855+0000","last_change":"2023-10-06T10:43:48.351855+0000","last_active":"2023-10-06T10:43:48.351855+0000","last_peered":"2023-10-06T10:43:48.351855+0000","last_clean":"2023-10-06T10:43:48.351855+0000","last_became_active":"2023-10-06T10:43:48.351606+0000","last_became_peered":"2023-10-06T10:43:48.351606+0000","last_unstale":"2023-10-06T10:43:48.351855+0000","last_undegraded":"2023-10-06T10:43:48.351855+0000","last_fullsized":"2023-10-06T10:43:48.351855+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:11:41.020256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.356027+0000","last_change":"2023-10-06T10:43:48.356027+0000","last_active":"2023-10-06T10:43:48.356027+0000","last_peered":"2023-10-06T10:43:48.356027+0000","last_clean":"2023-10-06T10:43:48.356027+0000","last_became_active":"2023-10-06T10:43:48.355758+0000","last_became_peered":"2023-10-06T10:43:48.355758+0000","last_unstale":"2023-10-06T10:43:48.356027+0000","last_undegraded":"2023-10-06T10:43:48.356027+0000","last_fullsized":"2023-10-06T10:43:48.356027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:00.307376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.348641+0000","last_change":"2023-10-06T10:43:48.348641+0000","last_active":"2023-10-06T10:43:48.348641+0000","last_peered":"2023-10-06T10:43:48.348641+0000","last_clean":"2023-10-06T10:43:48.348641+0000","last_became_active":"2023-10-06T10:43:48.348345+0000","last_became_peered":"2023-10-06T10:43:48.348345+0000","last_unstale":"2023-10-06T10:43:48.348641+0000","last_undegraded":"2023-10-06T10:43:48.348641+0000","last_fullsized":"2023-10-06T10:43:48.348641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:07:38.821665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.354294+0000","last_change":"2023-10-06T10:43:48.354294+0000","last_active":"2023-10-06T10:43:48.354294+0000","last_peered":"2023-10-06T10:43:48.354294+0000","last_clean":"2023-10-06T10:43:48.354294+0000","last_became_active":"2023-10-06T10:43:48.354009+0000","last_became_peered":"2023-10-06T10:43:48.354009+0000","last_unstale":"2023-10-06T10:43:48.354294+0000","last_undegraded":"2023-10-06T10:43:48.354294+0000","last_fullsized":"2023-10-06T10:43:48.354294+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:01:23.602658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.350744+0000","last_change":"2023-10-06T10:43:48.350744+0000","last_active":"2023-10-06T10:43:48.350744+0000","last_peered":"2023-10-06T10:43:48.350744+0000","last_clean":"2023-10-06T10:43:48.350744+0000","last_became_active":"2023-10-06T10:43:48.350490+0000","last_became_peered":"2023-10-06T10:43:48.350490+0000","last_unstale":"2023-10-06T10:43:48.350744+0000","last_undegraded":"2023-10-06T10:43:48.350744+0000","last_fullsized":"2023-10-06T10:43:48.350744+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:22:33.960812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.348417+0000","last_change":"2023-10-06T10:43:48.348417+0000","last_active":"2023-10-06T10:43:48.348417+0000","last_peered":"2023-10-06T10:43:48.348417+0000","last_clean":"2023-10-06T10:43:48.348417+0000","last_became_active":"2023-10-06T10:43:48.348181+0000","last_became_peered":"2023-10-06T10:43:48.348181+0000","last_unstale":"2023-10-06T10:43:48.348417+0000","last_undegraded":"2023-10-06T10:43:48.348417+0000","last_fullsized":"2023-10-06T10:43:48.348417+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:52.926879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.353701+0000","last_change":"2023-10-06T10:43:48.353701+0000","last_active":"2023-10-06T10:43:48.353701+0000","last_peered":"2023-10-06T10:43:48.353701+0000","last_clean":"2023-10-06T10:43:48.353701+0000","last_became_active":"2023-10-06T10:43:48.351714+0000","last_became_peered":"2023-10-06T10:43:48.351714+0000","last_unstale":"2023-10-06T10:43:48.353701+0000","last_undegraded":"2023-10-06T10:43:48.353701+0000","last_fullsized":"2023-10-06T10:43:48.353701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:36:57.274188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"89'4","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.498122+0000","last_change":"2023-10-06T10:43:47.483577+0000","last_active":"2023-10-06T10:43:48.498122+0000","last_peered":"2023-10-06T10:43:48.498122+0000","last_clean":"2023-10-06T10:43:48.498122+0000","last_became_active":"2023-10-06T10:43:47.483355+0000","last_became_peered":"2023-10-06T10:43:47.483355+0000","last_unstale":"2023-10-06T10:43:48.498122+0000","last_undegraded":"2023-10-06T10:43:48.498122+0000","last_fullsized":"2023-10-06T10:43:48.498122+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:11:40.016496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.355144+0000","last_change":"2023-10-06T10:43:48.355144+0000","last_active":"2023-10-06T10:43:48.355144+0000","last_peered":"2023-10-06T10:43:48.355144+0000","last_clean":"2023-10-06T10:43:48.355144+0000","last_became_active":"2023-10-06T10:43:48.354870+0000","last_became_peered":"2023-10-06T10:43:48.354870+0000","last_unstale":"2023-10-06T10:43:48.355144+0000","last_undegraded":"2023-10-06T10:43:48.355144+0000","last_fullsized":"2023-10-06T10:43:48.355144+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:29:27.392646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"89'1","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.343744+0000","last_change":"2023-10-06T10:43:47.481283+0000","last_active":"2023-10-06T10:43:48.343744+0000","last_peered":"2023-10-06T10:43:48.343744+0000","last_clean":"2023-10-06T10:43:48.343744+0000","last_became_active":"2023-10-06T10:43:47.481026+0000","last_became_peered":"2023-10-06T10:43:47.481026+0000","last_unstale":"2023-10-06T10:43:48.343744+0000","last_undegraded":"2023-10-06T10:43:48.343744+0000","last_fullsized":"2023-10-06T10:43:48.343744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:40:10.471619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.357292+0000","last_change":"2023-10-06T10:43:48.357292+0000","last_active":"2023-10-06T10:43:48.357292+0000","last_peered":"2023-10-06T10:43:48.357292+0000","last_clean":"2023-10-06T10:43:48.357292+0000","last_became_active":"2023-10-06T10:43:48.356997+0000","last_became_peered":"2023-10-06T10:43:48.356997+0000","last_unstale":"2023-10-06T10:43:48.357292+0000","last_undegraded":"2023-10-06T10:43:48.357292+0000","last_fullsized":"2023-10-06T10:43:48.357292+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:10.707456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"89'2","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.499403+0000","last_change":"2023-10-06T10:43:47.481202+0000","last_active":"2023-10-06T10:43:48.499403+0000","last_peered":"2023-10-06T10:43:48.499403+0000","last_clean":"2023-10-06T10:43:48.499403+0000","last_became_active":"2023-10-06T10:43:47.480948+0000","last_became_peered":"2023-10-06T10:43:47.480948+0000","last_unstale":"2023-10-06T10:43:48.499403+0000","last_undegraded":"2023-10-06T10:43:48.499403+0000","last_fullsized":"2023-10-06T10:43:48.499403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:29:26.388885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.349180+0000","last_change":"2023-10-06T10:43:48.349180+0000","last_active":"2023-10-06T10:43:48.349180+0000","last_peered":"2023-10-06T10:43:48.349180+0000","last_clean":"2023-10-06T10:43:48.349180+0000","last_became_active":"2023-10-06T10:43:48.348915+0000","last_became_peered":"2023-10-06T10:43:48.348915+0000","last_unstale":"2023-10-06T10:43:48.349180+0000","last_undegraded":"2023-10-06T10:43:48.349180+0000","last_fullsized":"2023-10-06T10:43:48.349180+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:44.794731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"89'2","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.495361+0000","last_change":"2023-10-06T10:43:47.478977+0000","last_active":"2023-10-06T10:43:48.495361+0000","last_peered":"2023-10-06T10:43:48.495361+0000","last_clean":"2023-10-06T10:43:48.495361+0000","last_became_active":"2023-10-06T10:43:47.478586+0000","last_became_peered":"2023-10-06T10:43:47.478586+0000","last_unstale":"2023-10-06T10:43:48.495361+0000","last_undegraded":"2023-10-06T10:43:48.495361+0000","last_fullsized":"2023-10-06T10:43:48.495361+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:55.456547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351331+0000","last_change":"2023-10-06T10:43:48.351331+0000","last_active":"2023-10-06T10:43:48.351331+0000","last_peered":"2023-10-06T10:43:48.351331+0000","last_clean":"2023-10-06T10:43:48.351331+0000","last_became_active":"2023-10-06T10:43:48.351064+0000","last_became_peered":"2023-10-06T10:43:48.351064+0000","last_unstale":"2023-10-06T10:43:48.351331+0000","last_undegraded":"2023-10-06T10:43:48.351331+0000","last_fullsized":"2023-10-06T10:43:48.351331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:02:28.611301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"89'1","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.342913+0000","last_change":"2023-10-06T10:43:47.477361+0000","last_active":"2023-10-06T10:43:48.342913+0000","last_peered":"2023-10-06T10:43:48.342913+0000","last_clean":"2023-10-06T10:43:48.342913+0000","last_became_active":"2023-10-06T10:43:47.477114+0000","last_became_peered":"2023-10-06T10:43:47.477114+0000","last_unstale":"2023-10-06T10:43:48.342913+0000","last_undegraded":"2023-10-06T10:43:48.342913+0000","last_fullsized":"2023-10-06T10:43:48.342913+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:23.491319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.352595+0000","last_change":"2023-10-06T10:43:48.352595+0000","last_active":"2023-10-06T10:43:48.352595+0000","last_peered":"2023-10-06T10:43:48.352595+0000","last_clean":"2023-10-06T10:43:48.352595+0000","last_became_active":"2023-10-06T10:43:48.352336+0000","last_became_peered":"2023-10-06T10:43:48.352336+0000","last_unstale":"2023-10-06T10:43:48.352595+0000","last_undegraded":"2023-10-06T10:43:48.352595+0000","last_fullsized":"2023-10-06T10:43:48.352595+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:13:21.303231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"89'1","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.338289+0000","last_change":"2023-10-06T10:43:47.481917+0000","last_active":"2023-10-06T10:43:48.338289+0000","last_peered":"2023-10-06T10:43:48.338289+0000","last_clean":"2023-10-06T10:43:48.338289+0000","last_became_active":"2023-10-06T10:43:47.481681+0000","last_became_peered":"2023-10-06T10:43:47.481681+0000","last_unstale":"2023-10-06T10:43:48.338289+0000","last_undegraded":"2023-10-06T10:43:48.338289+0000","last_fullsized":"2023-10-06T10:43:48.338289+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:24.564175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.353199+0000","last_change":"2023-10-06T10:43:48.353199+0000","last_active":"2023-10-06T10:43:48.353199+0000","last_peered":"2023-10-06T10:43:48.353199+0000","last_clean":"2023-10-06T10:43:48.353199+0000","last_became_active":"2023-10-06T10:43:48.352967+0000","last_became_peered":"2023-10-06T10:43:48.352967+0000","last_unstale":"2023-10-06T10:43:48.353199+0000","last_undegraded":"2023-10-06T10:43:48.353199+0000","last_fullsized":"2023-10-06T10:43:48.353199+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:52:45.733863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.482698+0000","last_change":"2023-10-06T10:43:47.476010+0000","last_active":"2023-10-06T10:43:47.482698+0000","last_peered":"2023-10-06T10:43:47.482698+0000","last_clean":"2023-10-06T10:43:47.482698+0000","last_became_active":"2023-10-06T10:43:47.475749+0000","last_became_peered":"2023-10-06T10:43:47.475749+0000","last_unstale":"2023-10-06T10:43:47.482698+0000","last_undegraded":"2023-10-06T10:43:47.482698+0000","last_fullsized":"2023-10-06T10:43:47.482698+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:29:27.051793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.349220+0000","last_change":"2023-10-06T10:43:48.349220+0000","last_active":"2023-10-06T10:43:48.349220+0000","last_peered":"2023-10-06T10:43:48.349220+0000","last_clean":"2023-10-06T10:43:48.349220+0000","last_became_active":"2023-10-06T10:43:48.348948+0000","last_became_peered":"2023-10-06T10:43:48.348948+0000","last_unstale":"2023-10-06T10:43:48.349220+0000","last_undegraded":"2023-10-06T10:43:48.349220+0000","last_fullsized":"2023-10-06T10:43:48.349220+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:25.393203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.481114+0000","last_change":"2023-10-06T10:43:47.475799+0000","last_active":"2023-10-06T10:43:47.481114+0000","last_peered":"2023-10-06T10:43:47.481114+0000","last_clean":"2023-10-06T10:43:47.481114+0000","last_became_active":"2023-10-06T10:43:47.475576+0000","last_became_peered":"2023-10-06T10:43:47.475576+0000","last_unstale":"2023-10-06T10:43:47.481114+0000","last_undegraded":"2023-10-06T10:43:47.481114+0000","last_fullsized":"2023-10-06T10:43:47.481114+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:43:16.112346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351335+0000","last_change":"2023-10-06T10:43:48.351335+0000","last_active":"2023-10-06T10:43:48.351335+0000","last_peered":"2023-10-06T10:43:48.351335+0000","last_clean":"2023-10-06T10:43:48.351335+0000","last_became_active":"2023-10-06T10:43:48.351068+0000","last_became_peered":"2023-10-06T10:43:48.351068+0000","last_unstale":"2023-10-06T10:43:48.351335+0000","last_undegraded":"2023-10-06T10:43:48.351335+0000","last_fullsized":"2023-10-06T10:43:48.351335+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:21:58.749235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.477892+0000","last_change":"2023-10-06T10:43:47.473232+0000","last_active":"2023-10-06T10:43:47.477892+0000","last_peered":"2023-10-06T10:43:47.477892+0000","last_clean":"2023-10-06T10:43:47.477892+0000","last_became_active":"2023-10-06T10:43:47.472990+0000","last_became_peered":"2023-10-06T10:43:47.472990+0000","last_unstale":"2023-10-06T10:43:47.477892+0000","last_undegraded":"2023-10-06T10:43:47.477892+0000","last_fullsized":"2023-10-06T10:43:47.477892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:10.651805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351703+0000","last_change":"2023-10-06T10:43:48.351703+0000","last_active":"2023-10-06T10:43:48.351703+0000","last_peered":"2023-10-06T10:43:48.351703+0000","last_clean":"2023-10-06T10:43:48.351703+0000","last_became_active":"2023-10-06T10:43:48.351389+0000","last_became_peered":"2023-10-06T10:43:48.351389+0000","last_unstale":"2023-10-06T10:43:48.351703+0000","last_undegraded":"2023-10-06T10:43:48.351703+0000","last_fullsized":"2023-10-06T10:43:48.351703+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:17.993552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.482702+0000","last_change":"2023-10-06T10:43:47.479795+0000","last_active":"2023-10-06T10:43:47.482702+0000","last_peered":"2023-10-06T10:43:47.482702+0000","last_clean":"2023-10-06T10:43:47.482702+0000","last_became_active":"2023-10-06T10:43:47.479486+0000","last_became_peered":"2023-10-06T10:43:47.479486+0000","last_unstale":"2023-10-06T10:43:47.482702+0000","last_undegraded":"2023-10-06T10:43:47.482702+0000","last_fullsized":"2023-10-06T10:43:47.482702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:12:21.074296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351635+0000","last_change":"2023-10-06T10:43:48.351635+0000","last_active":"2023-10-06T10:43:48.351635+0000","last_peered":"2023-10-06T10:43:48.351635+0000","last_clean":"2023-10-06T10:43:48.351635+0000","last_became_active":"2023-10-06T10:43:48.351340+0000","last_became_peered":"2023-10-06T10:43:48.351340+0000","last_unstale":"2023-10-06T10:43:48.351635+0000","last_undegraded":"2023-10-06T10:43:48.351635+0000","last_fullsized":"2023-10-06T10:43:48.351635+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:04.472718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"89'3","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.496698+0000","last_change":"2023-10-06T10:43:47.480672+0000","last_active":"2023-10-06T10:43:48.496698+0000","last_peered":"2023-10-06T10:43:48.496698+0000","last_clean":"2023-10-06T10:43:48.496698+0000","last_became_active":"2023-10-06T10:43:47.480407+0000","last_became_peered":"2023-10-06T10:43:47.480407+0000","last_unstale":"2023-10-06T10:43:48.496698+0000","last_undegraded":"2023-10-06T10:43:48.496698+0000","last_fullsized":"2023-10-06T10:43:48.496698+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:18:26.575242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.357051+0000","last_change":"2023-10-06T10:43:48.357051+0000","last_active":"2023-10-06T10:43:48.357051+0000","last_peered":"2023-10-06T10:43:48.357051+0000","last_clean":"2023-10-06T10:43:48.357051+0000","last_became_active":"2023-10-06T10:43:48.356799+0000","last_became_peered":"2023-10-06T10:43:48.356799+0000","last_unstale":"2023-10-06T10:43:48.357051+0000","last_undegraded":"2023-10-06T10:43:48.357051+0000","last_fullsized":"2023-10-06T10:43:48.357051+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:07:37.438442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"89'6","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.502502+0000","last_change":"2023-10-06T10:43:47.474744+0000","last_active":"2023-10-06T10:43:48.502502+0000","last_peered":"2023-10-06T10:43:48.502502+0000","last_clean":"2023-10-06T10:43:48.502502+0000","last_became_active":"2023-10-06T10:43:47.474483+0000","last_became_peered":"2023-10-06T10:43:47.474483+0000","last_unstale":"2023-10-06T10:43:48.502502+0000","last_undegraded":"2023-10-06T10:43:48.502502+0000","last_fullsized":"2023-10-06T10:43:48.502502+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:07.036958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351834+0000","last_change":"2023-10-06T10:43:48.351834+0000","last_active":"2023-10-06T10:43:48.351834+0000","last_peered":"2023-10-06T10:43:48.351834+0000","last_clean":"2023-10-06T10:43:48.351834+0000","last_became_active":"2023-10-06T10:43:48.351591+0000","last_became_peered":"2023-10-06T10:43:48.351591+0000","last_unstale":"2023-10-06T10:43:48.351834+0000","last_undegraded":"2023-10-06T10:43:48.351834+0000","last_fullsized":"2023-10-06T10:43:48.351834+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:03:27.434250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"89'2","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.496250+0000","last_change":"2023-10-06T10:43:47.479328+0000","last_active":"2023-10-06T10:43:48.496250+0000","last_peered":"2023-10-06T10:43:48.496250+0000","last_clean":"2023-10-06T10:43:48.496250+0000","last_became_active":"2023-10-06T10:43:47.479092+0000","last_became_peered":"2023-10-06T10:43:47.479092+0000","last_unstale":"2023-10-06T10:43:48.496250+0000","last_undegraded":"2023-10-06T10:43:48.496250+0000","last_fullsized":"2023-10-06T10:43:48.496250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:13:30.750796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.350086+0000","last_change":"2023-10-06T10:43:48.350086+0000","last_active":"2023-10-06T10:43:48.350086+0000","last_peered":"2023-10-06T10:43:48.350086+0000","last_clean":"2023-10-06T10:43:48.350086+0000","last_became_active":"2023-10-06T10:43:48.349833+0000","last_became_peered":"2023-10-06T10:43:48.349833+0000","last_unstale":"2023-10-06T10:43:48.350086+0000","last_undegraded":"2023-10-06T10:43:48.350086+0000","last_fullsized":"2023-10-06T10:43:48.350086+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:31:49.281793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.482020+0000","last_change":"2023-10-06T10:43:47.477417+0000","last_active":"2023-10-06T10:43:47.482020+0000","last_peered":"2023-10-06T10:43:47.482020+0000","last_clean":"2023-10-06T10:43:47.482020+0000","last_became_active":"2023-10-06T10:43:47.477164+0000","last_became_peered":"2023-10-06T10:43:47.477164+0000","last_unstale":"2023-10-06T10:43:47.482020+0000","last_undegraded":"2023-10-06T10:43:47.482020+0000","last_fullsized":"2023-10-06T10:43:47.482020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:05.001802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.348736+0000","last_change":"2023-10-06T10:43:48.348736+0000","last_active":"2023-10-06T10:43:48.348736+0000","last_peered":"2023-10-06T10:43:48.348736+0000","last_clean":"2023-10-06T10:43:48.348736+0000","last_became_active":"2023-10-06T10:43:48.348420+0000","last_became_peered":"2023-10-06T10:43:48.348420+0000","last_unstale":"2023-10-06T10:43:48.348736+0000","last_undegraded":"2023-10-06T10:43:48.348736+0000","last_fullsized":"2023-10-06T10:43:48.348736+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:18.395189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.475299+0000","last_change":"2023-10-06T10:43:47.469427+0000","last_active":"2023-10-06T10:43:47.475299+0000","last_peered":"2023-10-06T10:43:47.475299+0000","last_clean":"2023-10-06T10:43:47.475299+0000","last_became_active":"2023-10-06T10:43:47.469159+0000","last_became_peered":"2023-10-06T10:43:47.469159+0000","last_unstale":"2023-10-06T10:43:47.475299+0000","last_undegraded":"2023-10-06T10:43:47.475299+0000","last_fullsized":"2023-10-06T10:43:47.475299+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:35:08.093726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.357617+0000","last_change":"2023-10-06T10:43:48.357617+0000","last_active":"2023-10-06T10:43:48.357617+0000","last_peered":"2023-10-06T10:43:48.357617+0000","last_clean":"2023-10-06T10:43:48.357617+0000","last_became_active":"2023-10-06T10:43:48.357307+0000","last_became_peered":"2023-10-06T10:43:48.357307+0000","last_unstale":"2023-10-06T10:43:48.357617+0000","last_undegraded":"2023-10-06T10:43:48.357617+0000","last_fullsized":"2023-10-06T10:43:48.357617+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:51.781585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"89'8","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.499897+0000","last_change":"2023-10-06T10:43:47.477030+0000","last_active":"2023-10-06T10:43:48.499897+0000","last_peered":"2023-10-06T10:43:48.499897+0000","last_clean":"2023-10-06T10:43:48.499897+0000","last_became_active":"2023-10-06T10:43:47.476777+0000","last_became_peered":"2023-10-06T10:43:47.476777+0000","last_unstale":"2023-10-06T10:43:48.499897+0000","last_undegraded":"2023-10-06T10:43:48.499897+0000","last_fullsized":"2023-10-06T10:43:48.499897+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:33:03.157191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.344777+0000","last_change":"2023-10-06T10:43:48.344777+0000","last_active":"2023-10-06T10:43:48.344777+0000","last_peered":"2023-10-06T10:43:48.344777+0000","last_clean":"2023-10-06T10:43:48.344777+0000","last_became_active":"2023-10-06T10:43:48.344463+0000","last_became_peered":"2023-10-06T10:43:48.344463+0000","last_unstale":"2023-10-06T10:43:48.344777+0000","last_undegraded":"2023-10-06T10:43:48.344777+0000","last_fullsized":"2023-10-06T10:43:48.344777+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:45:03.561894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"89'2","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.494969+0000","last_change":"2023-10-06T10:43:47.471640+0000","last_active":"2023-10-06T10:43:48.494969+0000","last_peered":"2023-10-06T10:43:48.494969+0000","last_clean":"2023-10-06T10:43:48.494969+0000","last_became_active":"2023-10-06T10:43:47.471411+0000","last_became_peered":"2023-10-06T10:43:47.471411+0000","last_unstale":"2023-10-06T10:43:48.494969+0000","last_undegraded":"2023-10-06T10:43:48.494969+0000","last_fullsized":"2023-10-06T10:43:48.494969+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:31:48.278032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.350041+0000","last_change":"2023-10-06T10:43:48.350041+0000","last_active":"2023-10-06T10:43:48.350041+0000","last_peered":"2023-10-06T10:43:48.350041+0000","last_clean":"2023-10-06T10:43:48.350041+0000","last_became_active":"2023-10-06T10:43:48.349786+0000","last_became_peered":"2023-10-06T10:43:48.349786+0000","last_unstale":"2023-10-06T10:43:48.350041+0000","last_undegraded":"2023-10-06T10:43:48.350041+0000","last_fullsized":"2023-10-06T10:43:48.350041+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:06.744223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.483035+0000","last_change":"2023-10-06T10:43:47.481804+0000","last_active":"2023-10-06T10:43:47.483035+0000","last_peered":"2023-10-06T10:43:47.483035+0000","last_clean":"2023-10-06T10:43:47.483035+0000","last_became_active":"2023-10-06T10:43:47.481553+0000","last_became_peered":"2023-10-06T10:43:47.481553+0000","last_unstale":"2023-10-06T10:43:47.483035+0000","last_undegraded":"2023-10-06T10:43:47.483035+0000","last_fullsized":"2023-10-06T10:43:47.483035+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:35.578447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351356+0000","last_change":"2023-10-06T10:43:48.351356+0000","last_active":"2023-10-06T10:43:48.351356+0000","last_peered":"2023-10-06T10:43:48.351356+0000","last_clean":"2023-10-06T10:43:48.351356+0000","last_became_active":"2023-10-06T10:43:48.351071+0000","last_became_peered":"2023-10-06T10:43:48.351071+0000","last_unstale":"2023-10-06T10:43:48.351356+0000","last_undegraded":"2023-10-06T10:43:48.351356+0000","last_fullsized":"2023-10-06T10:43:48.351356+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:56.247192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.482238+0000","last_change":"2023-10-06T10:43:47.479767+0000","last_active":"2023-10-06T10:43:47.482238+0000","last_peered":"2023-10-06T10:43:47.482238+0000","last_clean":"2023-10-06T10:43:47.482238+0000","last_became_active":"2023-10-06T10:43:47.479492+0000","last_became_peered":"2023-10-06T10:43:47.479492+0000","last_unstale":"2023-10-06T10:43:47.482238+0000","last_undegraded":"2023-10-06T10:43:47.482238+0000","last_fullsized":"2023-10-06T10:43:47.482238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:59.303616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.355519+0000","last_change":"2023-10-06T10:43:48.355519+0000","last_active":"2023-10-06T10:43:48.355519+0000","last_peered":"2023-10-06T10:43:48.355519+0000","last_clean":"2023-10-06T10:43:48.355519+0000","last_became_active":"2023-10-06T10:43:48.355202+0000","last_became_peered":"2023-10-06T10:43:48.355202+0000","last_unstale":"2023-10-06T10:43:48.355519+0000","last_undegraded":"2023-10-06T10:43:48.355519+0000","last_fullsized":"2023-10-06T10:43:48.355519+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:17:41.222364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.482791+0000","last_change":"2023-10-06T10:43:47.476477+0000","last_active":"2023-10-06T10:43:47.482791+0000","last_peered":"2023-10-06T10:43:47.482791+0000","last_clean":"2023-10-06T10:43:47.482791+0000","last_became_active":"2023-10-06T10:43:47.476196+0000","last_became_peered":"2023-10-06T10:43:47.476196+0000","last_unstale":"2023-10-06T10:43:47.482791+0000","last_undegraded":"2023-10-06T10:43:47.482791+0000","last_fullsized":"2023-10-06T10:43:47.482791+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:49.331843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.357962+0000","last_change":"2023-10-06T10:43:48.357962+0000","last_active":"2023-10-06T10:43:48.357962+0000","last_peered":"2023-10-06T10:43:48.357962+0000","last_clean":"2023-10-06T10:43:48.357962+0000","last_became_active":"2023-10-06T10:43:48.357714+0000","last_became_peered":"2023-10-06T10:43:48.357714+0000","last_unstale":"2023-10-06T10:43:48.357962+0000","last_undegraded":"2023-10-06T10:43:48.357962+0000","last_fullsized":"2023-10-06T10:43:48.357962+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:16.939561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.482122+0000","last_change":"2023-10-06T10:43:47.479993+0000","last_active":"2023-10-06T10:43:47.482122+0000","last_peered":"2023-10-06T10:43:47.482122+0000","last_clean":"2023-10-06T10:43:47.482122+0000","last_became_active":"2023-10-06T10:43:47.479742+0000","last_became_peered":"2023-10-06T10:43:47.479742+0000","last_unstale":"2023-10-06T10:43:47.482122+0000","last_undegraded":"2023-10-06T10:43:47.482122+0000","last_fullsized":"2023-10-06T10:43:47.482122+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:03.468957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.352570+0000","last_change":"2023-10-06T10:43:48.352570+0000","last_active":"2023-10-06T10:43:48.352570+0000","last_peered":"2023-10-06T10:43:48.352570+0000","last_clean":"2023-10-06T10:43:48.352570+0000","last_became_active":"2023-10-06T10:43:48.352295+0000","last_became_peered":"2023-10-06T10:43:48.352295+0000","last_unstale":"2023-10-06T10:43:48.352570+0000","last_undegraded":"2023-10-06T10:43:48.352570+0000","last_fullsized":"2023-10-06T10:43:48.352570+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:21:58.749235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.485388+0000","last_change":"2023-10-06T10:43:47.481914+0000","last_active":"2023-10-06T10:43:47.485388+0000","last_peered":"2023-10-06T10:43:47.485388+0000","last_clean":"2023-10-06T10:43:47.485388+0000","last_became_active":"2023-10-06T10:43:47.481686+0000","last_became_peered":"2023-10-06T10:43:47.481686+0000","last_unstale":"2023-10-06T10:43:47.485388+0000","last_undegraded":"2023-10-06T10:43:47.485388+0000","last_fullsized":"2023-10-06T10:43:47.485388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:20:48.312164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.357650+0000","last_change":"2023-10-06T10:43:48.357650+0000","last_active":"2023-10-06T10:43:48.357650+0000","last_peered":"2023-10-06T10:43:48.357650+0000","last_clean":"2023-10-06T10:43:48.357650+0000","last_became_active":"2023-10-06T10:43:48.357232+0000","last_became_peered":"2023-10-06T10:43:48.357232+0000","last_unstale":"2023-10-06T10:43:48.357650+0000","last_undegraded":"2023-10-06T10:43:48.357650+0000","last_fullsized":"2023-10-06T10:43:48.357650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:43:17.116106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.477918+0000","last_change":"2023-10-06T10:43:47.474567+0000","last_active":"2023-10-06T10:43:47.477918+0000","last_peered":"2023-10-06T10:43:47.477918+0000","last_clean":"2023-10-06T10:43:47.477918+0000","last_became_active":"2023-10-06T10:43:47.474321+0000","last_became_peered":"2023-10-06T10:43:47.474321+0000","last_unstale":"2023-10-06T10:43:47.477918+0000","last_undegraded":"2023-10-06T10:43:47.477918+0000","last_fullsized":"2023-10-06T10:43:47.477918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:11:40.016496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.355771+0000","last_change":"2023-10-06T10:43:48.355771+0000","last_active":"2023-10-06T10:43:48.355771+0000","last_peered":"2023-10-06T10:43:48.355771+0000","last_clean":"2023-10-06T10:43:48.355771+0000","last_became_active":"2023-10-06T10:43:48.355522+0000","last_became_peered":"2023-10-06T10:43:48.355522+0000","last_unstale":"2023-10-06T10:43:48.355771+0000","last_undegraded":"2023-10-06T10:43:48.355771+0000","last_fullsized":"2023-10-06T10:43:48.355771+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:35:09.097486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.481013+0000","last_change":"2023-10-06T10:43:47.478822+0000","last_active":"2023-10-06T10:43:47.481013+0000","last_peered":"2023-10-06T10:43:47.481013+0000","last_clean":"2023-10-06T10:43:47.481013+0000","last_became_active":"2023-10-06T10:43:47.478590+0000","last_became_peered":"2023-10-06T10:43:47.478590+0000","last_unstale":"2023-10-06T10:43:47.481013+0000","last_undegraded":"2023-10-06T10:43:47.481013+0000","last_fullsized":"2023-10-06T10:43:47.481013+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:44.670215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.349477+0000","last_change":"2023-10-06T10:43:48.349477+0000","last_active":"2023-10-06T10:43:48.349477+0000","last_peered":"2023-10-06T10:43:48.349477+0000","last_clean":"2023-10-06T10:43:48.349477+0000","last_became_active":"2023-10-06T10:43:48.349222+0000","last_became_peered":"2023-10-06T10:43:48.349222+0000","last_unstale":"2023-10-06T10:43:48.349477+0000","last_undegraded":"2023-10-06T10:43:48.349477+0000","last_fullsized":"2023-10-06T10:43:48.349477+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:53:51.241025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.477368+0000","last_change":"2023-10-06T10:43:47.469428+0000","last_active":"2023-10-06T10:43:47.477368+0000","last_peered":"2023-10-06T10:43:47.477368+0000","last_clean":"2023-10-06T10:43:47.477368+0000","last_became_active":"2023-10-06T10:43:47.469059+0000","last_became_peered":"2023-10-06T10:43:47.469059+0000","last_unstale":"2023-10-06T10:43:47.477368+0000","last_undegraded":"2023-10-06T10:43:47.477368+0000","last_fullsized":"2023-10-06T10:43:47.477368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:18:26.575242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.463333+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:43:47.463333+0000","last_peered":"2023-10-06T10:43:47.463333+0000","last_clean":"2023-10-06T10:43:47.463333+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:43:47.463333+0000","last_undegraded":"2023-10-06T10:43:47.463333+0000","last_fullsized":"2023-10-06T10:43:47.463333+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.349870+0000","last_change":"2023-10-06T10:43:48.349870+0000","last_active":"2023-10-06T10:43:48.349870+0000","last_peered":"2023-10-06T10:43:48.349870+0000","last_clean":"2023-10-06T10:43:48.349870+0000","last_became_active":"2023-10-06T10:43:48.349630+0000","last_became_peered":"2023-10-06T10:43:48.349630+0000","last_unstale":"2023-10-06T10:43:48.349870+0000","last_undegraded":"2023-10-06T10:43:48.349870+0000","last_fullsized":"2023-10-06T10:43:48.349870+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:56.247192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"89'2","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.499397+0000","last_change":"2023-10-06T10:43:47.481379+0000","last_active":"2023-10-06T10:43:48.499397+0000","last_peered":"2023-10-06T10:43:48.499397+0000","last_clean":"2023-10-06T10:43:48.499397+0000","last_became_active":"2023-10-06T10:43:47.481120+0000","last_became_peered":"2023-10-06T10:43:47.481120+0000","last_unstale":"2023-10-06T10:43:48.499397+0000","last_undegraded":"2023-10-06T10:43:48.499397+0000","last_fullsized":"2023-10-06T10:43:48.499397+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:07.036958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.465691+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:43:47.465691+0000","last_peered":"2023-10-06T10:43:47.465691+0000","last_clean":"2023-10-06T10:43:47.465691+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:43:47.465691+0000","last_undegraded":"2023-10-06T10:43:47.465691+0000","last_fullsized":"2023-10-06T10:43:47.465691+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.358332+0000","last_change":"2023-10-06T10:43:48.358332+0000","last_active":"2023-10-06T10:43:48.358332+0000","last_peered":"2023-10-06T10:43:48.358332+0000","last_clean":"2023-10-06T10:43:48.358332+0000","last_became_active":"2023-10-06T10:43:48.357397+0000","last_became_peered":"2023-10-06T10:43:48.357397+0000","last_unstale":"2023-10-06T10:43:48.358332+0000","last_undegraded":"2023-10-06T10:43:48.358332+0000","last_fullsized":"2023-10-06T10:43:48.358332+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:55.836401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.481461+0000","last_change":"2023-10-06T10:43:47.478819+0000","last_active":"2023-10-06T10:43:47.481461+0000","last_peered":"2023-10-06T10:43:47.481461+0000","last_clean":"2023-10-06T10:43:47.481461+0000","last_became_active":"2023-10-06T10:43:47.478332+0000","last_became_peered":"2023-10-06T10:43:47.478332+0000","last_unstale":"2023-10-06T10:43:47.481461+0000","last_undegraded":"2023-10-06T10:43:47.481461+0000","last_fullsized":"2023-10-06T10:43:47.481461+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:35:08.093726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":73,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.466100+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:43:47.466100+0000","last_peered":"2023-10-06T10:43:47.466100+0000","last_clean":"2023-10-06T10:43:47.466100+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:43:47.466100+0000","last_undegraded":"2023-10-06T10:43:47.466100+0000","last_fullsized":"2023-10-06T10:43:47.466100+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":216,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.473481+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:43:47.473481+0000","last_peered":"2023-10-06T10:43:47.473481+0000","last_clean":"2023-10-06T10:43:47.473481+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:43:47.473481+0000","last_undegraded":"2023-10-06T10:43:47.473481+0000","last_fullsized":"2023-10-06T10:43:47.473481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.349957+0000","last_change":"2023-10-06T10:43:48.349957+0000","last_active":"2023-10-06T10:43:48.349957+0000","last_peered":"2023-10-06T10:43:48.349957+0000","last_clean":"2023-10-06T10:43:48.349957+0000","last_became_active":"2023-10-06T10:43:48.349677+0000","last_became_peered":"2023-10-06T10:43:48.349677+0000","last_unstale":"2023-10-06T10:43:48.349957+0000","last_undegraded":"2023-10-06T10:43:48.349957+0000","last_fullsized":"2023-10-06T10:43:48.349957+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:38:04.571912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"89'1","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.346954+0000","last_change":"2023-10-06T10:43:47.473629+0000","last_active":"2023-10-06T10:43:48.346954+0000","last_peered":"2023-10-06T10:43:48.346954+0000","last_clean":"2023-10-06T10:43:48.346954+0000","last_became_active":"2023-10-06T10:43:47.473135+0000","last_became_peered":"2023-10-06T10:43:47.473135+0000","last_unstale":"2023-10-06T10:43:48.346954+0000","last_undegraded":"2023-10-06T10:43:48.346954+0000","last_fullsized":"2023-10-06T10:43:48.346954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:13:20.299471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":66,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.473850+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:43:47.473850+0000","last_peered":"2023-10-06T10:43:47.473850+0000","last_clean":"2023-10-06T10:43:47.473850+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:43:47.473850+0000","last_undegraded":"2023-10-06T10:43:47.473850+0000","last_fullsized":"2023-10-06T10:43:47.473850+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.354545+0000","last_change":"2023-10-06T10:43:48.354545+0000","last_active":"2023-10-06T10:43:48.354545+0000","last_peered":"2023-10-06T10:43:48.354545+0000","last_clean":"2023-10-06T10:43:48.354545+0000","last_became_active":"2023-10-06T10:43:48.354132+0000","last_became_peered":"2023-10-06T10:43:48.354132+0000","last_unstale":"2023-10-06T10:43:48.354545+0000","last_undegraded":"2023-10-06T10:43:48.354545+0000","last_fullsized":"2023-10-06T10:43:48.354545+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:04.984298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"89'1","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.344071+0000","last_change":"2023-10-06T10:43:47.479218+0000","last_active":"2023-10-06T10:43:48.344071+0000","last_peered":"2023-10-06T10:43:48.344071+0000","last_clean":"2023-10-06T10:43:48.344071+0000","last_became_active":"2023-10-06T10:43:47.479015+0000","last_became_peered":"2023-10-06T10:43:47.479015+0000","last_unstale":"2023-10-06T10:43:48.344071+0000","last_undegraded":"2023-10-06T10:43:48.344071+0000","last_fullsized":"2023-10-06T10:43:48.344071+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:31:48.278032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.473386+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:43:47.473386+0000","last_peered":"2023-10-06T10:43:47.473386+0000","last_clean":"2023-10-06T10:43:47.473386+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:43:47.473386+0000","last_undegraded":"2023-10-06T10:43:47.473386+0000","last_fullsized":"2023-10-06T10:43:47.473386+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.357698+0000","last_change":"2023-10-06T10:43:48.357698+0000","last_active":"2023-10-06T10:43:48.357698+0000","last_peered":"2023-10-06T10:43:48.357698+0000","last_clean":"2023-10-06T10:43:48.357698+0000","last_became_active":"2023-10-06T10:43:48.357451+0000","last_became_peered":"2023-10-06T10:43:48.357451+0000","last_unstale":"2023-10-06T10:43:48.357698+0000","last_undegraded":"2023-10-06T10:43:48.357698+0000","last_fullsized":"2023-10-06T10:43:48.357698+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:31.895142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"89'2","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.495116+0000","last_change":"2023-10-06T10:43:47.472980+0000","last_active":"2023-10-06T10:43:48.495116+0000","last_peered":"2023-10-06T10:43:48.495116+0000","last_clean":"2023-10-06T10:43:48.495116+0000","last_became_active":"2023-10-06T10:43:47.472715+0000","last_became_peered":"2023-10-06T10:43:47.472715+0000","last_unstale":"2023-10-06T10:43:48.495116+0000","last_undegraded":"2023-10-06T10:43:48.495116+0000","last_fullsized":"2023-10-06T10:43:48.495116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:03.468957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.465031+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:43:47.465031+0000","last_peered":"2023-10-06T10:43:47.465031+0000","last_clean":"2023-10-06T10:43:47.465031+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:43:47.465031+0000","last_undegraded":"2023-10-06T10:43:47.465031+0000","last_fullsized":"2023-10-06T10:43:47.465031+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.347841+0000","last_change":"2023-10-06T10:43:48.347841+0000","last_active":"2023-10-06T10:43:48.347841+0000","last_peered":"2023-10-06T10:43:48.347841+0000","last_clean":"2023-10-06T10:43:48.347841+0000","last_became_active":"2023-10-06T10:43:48.347112+0000","last_became_peered":"2023-10-06T10:43:48.347112+0000","last_unstale":"2023-10-06T10:43:48.347841+0000","last_undegraded":"2023-10-06T10:43:48.347841+0000","last_fullsized":"2023-10-06T10:43:48.347841+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:06.744223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.482537+0000","last_change":"2023-10-06T10:43:47.477258+0000","last_active":"2023-10-06T10:43:47.482537+0000","last_peered":"2023-10-06T10:43:47.482537+0000","last_clean":"2023-10-06T10:43:47.482537+0000","last_became_active":"2023-10-06T10:43:47.476970+0000","last_became_peered":"2023-10-06T10:43:47.476970+0000","last_unstale":"2023-10-06T10:43:47.482537+0000","last_undegraded":"2023-10-06T10:43:47.482537+0000","last_fullsized":"2023-10-06T10:43:47.482537+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:51.923119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.466942+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:43:47.466942+0000","last_peered":"2023-10-06T10:43:47.466942+0000","last_clean":"2023-10-06T10:43:47.466942+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:43:47.466942+0000","last_undegraded":"2023-10-06T10:43:47.466942+0000","last_fullsized":"2023-10-06T10:43:47.466942+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.355316+0000","last_change":"2023-10-06T10:43:48.355316+0000","last_active":"2023-10-06T10:43:48.355316+0000","last_peered":"2023-10-06T10:43:48.355316+0000","last_clean":"2023-10-06T10:43:48.355316+0000","last_became_active":"2023-10-06T10:43:48.355011+0000","last_became_peered":"2023-10-06T10:43:48.355011+0000","last_unstale":"2023-10-06T10:43:48.355316+0000","last_undegraded":"2023-10-06T10:43:48.355316+0000","last_fullsized":"2023-10-06T10:43:48.355316+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:09.942535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.483582+0000","last_change":"2023-10-06T10:43:47.481933+0000","last_active":"2023-10-06T10:43:47.483582+0000","last_peered":"2023-10-06T10:43:47.483582+0000","last_clean":"2023-10-06T10:43:47.483582+0000","last_became_active":"2023-10-06T10:43:47.481690+0000","last_became_peered":"2023-10-06T10:43:47.481690+0000","last_unstale":"2023-10-06T10:43:47.483582+0000","last_undegraded":"2023-10-06T10:43:47.483582+0000","last_fullsized":"2023-10-06T10:43:47.483582+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:08.938775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.467138+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:43:47.467138+0000","last_peered":"2023-10-06T10:43:47.467138+0000","last_clean":"2023-10-06T10:43:47.467138+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:43:47.467138+0000","last_undegraded":"2023-10-06T10:43:47.467138+0000","last_fullsized":"2023-10-06T10:43:47.467138+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574921,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298948,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072892,"statfs":{"total":96636764160,"available":96330641408,"internally_reserved":0,"allocated":1699840,"data_stored":1118251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51600000000000001,"15min":0.48999999999999999},"min":{"1min":0.36299999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70599999999999996,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.51400000000000001,"15min":0.51300000000000001},"min":{"1min":0.41899999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73099999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.63}]},{"osd":1,"last update":"Fri Oct 6 10:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.49199999999999999,"15min":0.46600000000000003},"min":{"1min":0.318,"5min":0.252,"15min":0.252},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51800000000000002,"15min":0.504},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67500000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.53800000000000003}]},{"osd":2,"last update":"Fri Oct 6 10:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.54500000000000004,"15min":0.51500000000000001},"min":{"1min":0.434,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78600000000000003,"5min":1.24,"15min":1.24},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.51700000000000002,"15min":0.499},"min":{"1min":0.45100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.73199999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.52100000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54800000000000004,"15min":0.51500000000000001},"min":{"1min":0.41099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.746,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.52500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.76100000000000001,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.59499999999999997}]},{"osd":4,"last update":"Fri Oct 6 10:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.46500000000000002},"min":{"1min":0.33500000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64900000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.373},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48399999999999999,"15min":0.503},"min":{"1min":0.311,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81200000000000006,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.40100000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.47399999999999998,"15min":0.48399999999999999},"min":{"1min":0.35599999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.436},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52300000000000002,"15min":0.54500000000000004},"min":{"1min":0.33800000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.48799999999999999}]},{"osd":6,"last update":"Fri Oct 6 10:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54000000000000004,"15min":0.56799999999999995},"min":{"1min":0.40899999999999997,"5min":0.308,"15min":0.308},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.39700000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.82099999999999995,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.45100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574919,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298356,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073484,"statfs":{"total":96636764160,"available":96331247616,"internally_reserved":0,"allocated":1093632,"data_stored":526418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.503,"15min":0.49199999999999999},"min":{"1min":0.372,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56399999999999995,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.498,"15min":0.44700000000000001},"min":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69699999999999995,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.502}]},{"osd":1,"last update":"Fri Oct 6 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.499,"15min":0.45300000000000001},"min":{"1min":0.371,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66100000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52600000000000002,"15min":0.48299999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71199999999999997,"5min":0.751,"15min":0.751},"last":0.433}]},{"osd":2,"last update":"Fri Oct 6 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51300000000000001,"15min":0.47899999999999998},"min":{"1min":0.40500000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.66500000000000004,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52200000000000002,"15min":0.47899999999999998},"min":{"1min":0.36299999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.53000000000000003}]},{"osd":3,"last update":"Fri Oct 6 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.5},"min":{"1min":0.38500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53700000000000003,"15min":0.51300000000000001},"min":{"1min":0.39400000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66900000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.40699999999999997}]},{"osd":4,"last update":"Fri Oct 6 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47399999999999998,"15min":0.47599999999999998},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.375},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48699999999999999,"15min":0.48499999999999999},"min":{"1min":0.308,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69299999999999995,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.44700000000000001}]},{"osd":5,"last update":"Fri Oct 6 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.496,"15min":0.48799999999999999},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53100000000000003,"15min":0.53600000000000003},"min":{"1min":0.373,"5min":0.308,"15min":0.308},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.51500000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73499999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55800000000000005,"15min":0.54200000000000004},"min":{"1min":0.33000000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.56200000000000006}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298940,"kb_used_data":1652,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072900,"statfs":{"total":96636764160,"available":96330649600,"internally_reserved":0,"allocated":1691648,"data_stored":1118139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.219,"5min":0.188,"15min":0.188},"max":{"1min":0.48099999999999998,"5min":0.505,"15min":0.505},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.35899999999999999,"15min":0.34899999999999998},"min":{"1min":0.27900000000000003,"5min":0.189,"15min":0.189},"max":{"1min":0.56399999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.27200000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.33400000000000002,"15min":0.32200000000000001},"min":{"1min":0.23499999999999999,"5min":0.218,"15min":0.218},"max":{"1min":0.441,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.248},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.376,"15min":0.36499999999999999},"min":{"1min":0.26400000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.48799999999999999,"5min":0.62,"15min":0.62},"last":0.35299999999999998}]},{"osd":3,"last update":"Fri Oct 6 10:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39600000000000002,"15min":0.38},"min":{"1min":0.28199999999999997,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.56100000000000005,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.379,"15min":0.375},"min":{"1min":0.27000000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.50600000000000001,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.312}]},{"osd":4,"last update":"Fri Oct 6 10:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.40500000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77400000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56999999999999995,"15min":0.56000000000000005},"min":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71199999999999997,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.47199999999999998}]},{"osd":5,"last update":"Fri Oct 6 10:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.44600000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.749,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.54900000000000004},"min":{"1min":0.42699999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73199999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.48299999999999998}]},{"osd":6,"last update":"Fri Oct 6 10:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.46600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76600000000000001,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61199999999999999,"15min":0.60699999999999998},"min":{"1min":0.46300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73999999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.52000000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.42199999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.73599999999999999,"5min":0.996,"15min":0.996},"last":0.499},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.49299999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74399999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.434}]}]},{"osd":0,"up_from":13,"seq":55834574921,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298364,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073476,"statfs":{"total":96636764160,"available":96331239424,"internally_reserved":0,"allocated":1101824,"data_stored":528325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.33000000000000002,"15min":0.32000000000000001},"min":{"1min":0.21199999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.372,"15min":0.34799999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.432}]},{"osd":2,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.31900000000000001,"15min":0.308},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.35199999999999998,"15min":0.33100000000000002},"min":{"1min":0.251,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.40799999999999997}]},{"osd":3,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.35299999999999998,"15min":0.34399999999999997},"min":{"1min":0.245,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52100000000000002,"5min":0.622,"15min":0.622},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.36299999999999999,"15min":0.35199999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.38200000000000001}]},{"osd":4,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.40000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76300000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56200000000000006,"15min":0.54000000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.377,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85699999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.45600000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77500000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.66800000000000004}]},{"osd":6,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.41399999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80000000000000004,"5min":1.075,"15min":1.075},"last":0.76000000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57999999999999996,"15min":0.57899999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83499999999999996,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46200000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.77200000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.64300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574919,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298364,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073476,"statfs":{"total":96636764160,"available":96331239424,"internally_reserved":0,"allocated":1101824,"data_stored":527805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.314,"15min":0.311},"min":{"1min":0.20799999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.47299999999999998,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.34999999999999998,"15min":0.32500000000000001},"min":{"1min":0.20200000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.55600000000000005,"5min":0.622,"15min":0.622},"last":0.49299999999999999}]},{"osd":1,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.34100000000000003,"15min":0.32700000000000001},"min":{"1min":0.23000000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.56599999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.372,"15min":0.36599999999999999},"min":{"1min":0.28100000000000003,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.45100000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.374,"15min":0.35299999999999998},"min":{"1min":0.26100000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.374,"15min":0.35999999999999999},"min":{"1min":0.25,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.50600000000000001}]},{"osd":4,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.42599999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.38500000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82399999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56599999999999995}]},{"osd":5,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57599999999999996,"15min":0.57999999999999996},"min":{"1min":0.41299999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.83799999999999997,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.373,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81799999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.69599999999999995}]},{"osd":6,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60599999999999998,"15min":0.61799999999999999},"min":{"1min":0.44400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.88700000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.62,"15min":0.622},"min":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79700000000000004,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.71299999999999997}]},{"osd":7,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59299999999999997,"15min":0.60899999999999999},"min":{"1min":0.38600000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.42299999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81100000000000005,"5min":0.874,"15min":0.874},"last":0.66800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574919,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298376,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073464,"statfs":{"total":96636764160,"available":96331227136,"internally_reserved":0,"allocated":1114112,"data_stored":529329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34799999999999998,"15min":0.42599999999999999},"min":{"1min":0.20799999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.52700000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.433},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.35099999999999998,"15min":0.35899999999999999},"min":{"1min":0.22900000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.56899999999999995,"5min":1.218,"15min":1.218},"last":0.40500000000000003}]},{"osd":1,"last update":"Fri Oct 6 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.34200000000000003,"15min":0.34000000000000002},"min":{"1min":0.26100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.63600000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.39600000000000002,"15min":0.46600000000000003},"min":{"1min":0.28000000000000003,"5min":0.218,"15min":0.218},"max":{"1min":0.66000000000000003,"5min":3.359,"15min":3.359},"last":0.45900000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.372,"15min":0.38},"min":{"1min":0.26100000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.71099999999999997,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.44600000000000001,"15min":0.60999999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59899999999999998,"5min":7.069,"15min":7.069},"last":0.44700000000000001}]},{"osd":4,"last update":"Fri Oct 6 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.58499999999999996},"min":{"1min":0.39900000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84699999999999998,"5min":1.165,"15min":1.165},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60099999999999998,"15min":0.59799999999999998},"min":{"1min":0.41299999999999998,"5min":0.375,"15min":0.375},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.54500000000000004}]},{"osd":5,"last update":"Fri Oct 6 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.622,"15min":0.68799999999999994},"min":{"1min":0.39000000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84399999999999997,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.436,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80000000000000004,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.71599999999999997}]},{"osd":6,"last update":"Fri Oct 6 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.42699999999999999,"5min":0.378,"15min":0.378},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65400000000000003,"15min":0.71799999999999997},"min":{"1min":0.46999999999999997,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.876,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Oct 6 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.46000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83199999999999996,"5min":1.137,"15min":1.137},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.40999999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.97899999999999998,"5min":1.014,"15min":1.014},"last":0.67100000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298380,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073460,"statfs":{"total":96636764160,"available":96331223040,"internally_reserved":0,"allocated":1118208,"data_stored":529431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.48799999999999999},"min":{"1min":0.28799999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66600000000000004,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.378},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.34100000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73099999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.45400000000000001}]},{"osd":1,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.50900000000000001,"15min":0.501},"min":{"1min":0.36499999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68100000000000005,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.44},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52000000000000002,"15min":0.53500000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71799999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.48799999999999999}]},{"osd":2,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.377,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.60999999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53200000000000003,"15min":0.52100000000000002},"min":{"1min":0.376,"5min":0.315,"15min":0.315},"max":{"1min":0.75800000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.40999999999999998}]},{"osd":3,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54400000000000004,"15min":0.52200000000000002},"min":{"1min":0.39400000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.36599999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77500000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.47499999999999998}]},{"osd":5,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.45300000000000001,"15min":0.49399999999999999},"min":{"1min":0.29499999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67900000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.47599999999999998,"15min":0.496},"min":{"1min":0.315,"5min":0.255,"15min":0.255},"max":{"1min":0.625,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.35099999999999998}]},{"osd":6,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51100000000000001,"15min":0.53800000000000003},"min":{"1min":0.33400000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.77100000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54600000000000004,"15min":0.57199999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71799999999999997,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.53100000000000003}]},{"osd":7,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53400000000000003,"15min":0.56799999999999995},"min":{"1min":0.34599999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71399999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.55300000000000005,"15min":0.58899999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.745,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.505}]}]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298364,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073476,"statfs":{"total":96636764160,"available":96331239424,"internally_reserved":0,"allocated":1101824,"data_stored":527841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.504,"15min":0.47299999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52100000000000002,"15min":0.52800000000000002},"min":{"1min":0.33400000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.66900000000000004,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.44}]},{"osd":1,"last update":"Fri Oct 6 10:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.50700000000000001,"15min":0.48899999999999999},"min":{"1min":0.34599999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68899999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52800000000000002,"15min":0.51800000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.372}]},{"osd":2,"last update":"Fri Oct 6 10:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.50700000000000001},"min":{"1min":0.31900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77600000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.40699999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.40400000000000003}]},{"osd":3,"last update":"Fri Oct 6 10:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55900000000000005,"15min":0.57299999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79900000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53200000000000003,"15min":0.55300000000000005},"min":{"1min":0.35099999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.79000000000000004,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.33200000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.47699999999999998,"15min":0.48599999999999999},"min":{"1min":0.318,"5min":0.317,"15min":0.317},"max":{"1min":0.70099999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.313},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.503,"15min":0.50700000000000001},"min":{"1min":0.35999999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.68100000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.35199999999999998}]},{"osd":6,"last update":"Fri Oct 6 10:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.503,"15min":0.51800000000000002},"min":{"1min":0.30499999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.46400000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55900000000000005,"15min":0.57799999999999996},"min":{"1min":0.38200000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73899999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.498},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.42499999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76200000000000001,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.41699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:43:52.540 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:43:52.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.685+0000 7fb5e359e700 1 Processor -- start 2023-10-06T10:43:52.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.685+0000 7fb5e359e700 1 -- start start 2023-10-06T10:43:52.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.686+0000 7fb5e359e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb5e4159440 0x7fb5e40b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:52.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.686+0000 7fb5e359e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5e40b2100 0x7fb5e40b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:52.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.686+0000 7fb5e359e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e40cb7a0 con 0x7fb5e40b35d0 2023-10-06T10:43:52.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.686+0000 7fb5e359e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e40b0860 con 0x7fb5e4159440 2023-10-06T10:43:52.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.686+0000 7fb5e359e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e40cb150 con 0x7fb5e40b2100 2023-10-06T10:43:52.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.686+0000 7fb5e259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb5e4159440 0x7fb5e40b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:52.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.686+0000 7fb5d9d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb5e40b35d0 0x7fb5e40cbc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37598/0 (socket says 172.21.15.92:37598) 2023-10-06T10:43:52.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.686+0000 7fb5d9d9b700 1 -- 172.21.15.92:0/129138712 learned_addr learned my addr 172.21.15.92:0/129138712 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:52.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.686+0000 7fb5e2d9d700 1 --2- 172.21.15.92:0/129138712 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5e40b2100 0x7fb5e40b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:52.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.686+0000 7fb5e259c700 1 -- 172.21.15.92:0/129138712 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5e40b2100 msgr2=0x7fb5e40b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:52.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.686+0000 7fb5e259c700 1 --2- 172.21.15.92:0/129138712 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5e40b2100 0x7fb5e40b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.686+0000 7fb5e259c700 1 -- 172.21.15.92:0/129138712 >> v1:172.21.15.92:6789/0 conn(0x7fb5e40b35d0 legacy=0x7fb5e40cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:52.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.686+0000 7fb5e259c700 1 -- 172.21.15.92:0/129138712 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e40cb430 con 0x7fb5e4159440 2023-10-06T10:43:52.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.686+0000 7fb5e1d9b700 1 -- 172.21.15.92:0/129138712 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3460139751 0 0) 0x7fb5e40cb7a0 con 0x7fb5e40b35d0 2023-10-06T10:43:52.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.686+0000 7fb5e2d9d700 1 --2- 172.21.15.92:0/129138712 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5e40b2100 0x7fb5e40b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:52.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.686+0000 7fb5e259c700 1 --2- 172.21.15.92:0/129138712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e4159440 0x7fb5e40b3090 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40098e0 tx=0x7fb5d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b161745e2792ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:52.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.687+0000 7fb5e1d9b700 1 -- 172.21.15.92:0/129138712 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4009e80 con 0x7fb5e4159440 2023-10-06T10:43:52.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.687+0000 7fb5e1d9b700 1 -- 172.21.15.92:0/129138712 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d400b860 con 0x7fb5e4159440 2023-10-06T10:43:52.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.687+0000 7fb5e1d9b700 1 -- 172.21.15.92:0/129138712 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d400bbe0 con 0x7fb5e4159440 2023-10-06T10:43:52.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.687+0000 7fb5e359e700 1 -- 172.21.15.92:0/129138712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e4159440 msgr2=0x7fb5e40b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:52.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.687+0000 7fb5e359e700 1 --2- 172.21.15.92:0/129138712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e4159440 0x7fb5e40b3090 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40098e0 tx=0x7fb5d40091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.687+0000 7fb5e359e700 1 -- 172.21.15.92:0/129138712 shutdown_connections 2023-10-06T10:43:52.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.687+0000 7fb5e359e700 1 --2- 172.21.15.92:0/129138712 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5e40b2100 0x7fb5e40b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.687+0000 7fb5e359e700 1 --2- 172.21.15.92:0/129138712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e4159440 0x7fb5e40b3090 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.687+0000 7fb5e359e700 1 -- 172.21.15.92:0/129138712 >> 172.21.15.92:0/129138712 conn(0x7fb5e406ece0 msgr2=0x7fb5e406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:52.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.687+0000 7fb5e359e700 1 -- 172.21.15.92:0/129138712 shutdown_connections 2023-10-06T10:43:52.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.687+0000 7fb5e359e700 1 -- 172.21.15.92:0/129138712 wait complete. 2023-10-06T10:43:52.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.688+0000 7fb5e359e700 1 Processor -- start 2023-10-06T10:43:52.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.688+0000 7fb5e359e700 1 -- start start 2023-10-06T10:43:52.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.689+0000 7fb5e359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e40b2100 0x7fb5e40c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:52.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.689+0000 7fb5e359e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5e40b35d0 0x7fb5e40cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:52.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.689+0000 7fb5e359e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5e4159440 0x7fb5e40c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:52.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.689+0000 7fb5e359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e415e470 con 0x7fb5e40b2100 2023-10-06T10:43:52.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.689+0000 7fb5e359e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e40c9cd0 con 0x7fb5e4159440 2023-10-06T10:43:52.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.689+0000 7fb5e359e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e40c9fb0 con 0x7fb5e40b35d0 2023-10-06T10:43:52.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.689+0000 7fb5e259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e40b2100 0x7fb5e40c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:52.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.689+0000 7fb5e259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e40b2100 0x7fb5e40c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45034/0 (socket says 172.21.15.92:45034) 2023-10-06T10:43:52.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.689+0000 7fb5e259c700 1 -- 172.21.15.92:0/1191809777 learned_addr learned my addr 172.21.15.92:0/1191809777 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:52.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.689+0000 7fb5e2d9d700 1 --2- 172.21.15.92:0/1191809777 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5e4159440 0x7fb5e40c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:52.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.689+0000 7fb5d9d9b700 1 --2- 172.21.15.92:0/1191809777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5e40b35d0 0x7fb5e40cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:52.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.689+0000 7fb5e259c700 1 -- 172.21.15.92:0/1191809777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5e40b35d0 msgr2=0x7fb5e40cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:52.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.689+0000 7fb5e259c700 1 --2- 172.21.15.92:0/1191809777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5e40b35d0 0x7fb5e40cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.689+0000 7fb5e259c700 1 -- 172.21.15.92:0/1191809777 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5e4159440 msgr2=0x7fb5e40c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:52.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.689+0000 7fb5e259c700 1 --2- 172.21.15.92:0/1191809777 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5e4159440 0x7fb5e40c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.689+0000 7fb5e259c700 1 -- 172.21.15.92:0/1191809777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d4009420 con 0x7fb5e40b2100 2023-10-06T10:43:52.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.689+0000 7fb5d9d9b700 1 --2- 172.21.15.92:0/1191809777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5e40b35d0 0x7fb5e40cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:52.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.689+0000 7fb5e2d9d700 1 --2- 172.21.15.92:0/1191809777 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5e4159440 0x7fb5e40c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:52.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.690+0000 7fb5e259c700 1 --2- 172.21.15.92:0/1191809777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e40b2100 0x7fb5e40c9480 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4069150 tx=0x7fb5d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:52.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.690+0000 7fb5daffd700 1 -- 172.21.15.92:0/1191809777 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d400bbe0 con 0x7fb5e40b2100 2023-10-06T10:43:52.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.690+0000 7fb5daffd700 1 -- 172.21.15.92:0/1191809777 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d4011420 con 0x7fb5e40b2100 2023-10-06T10:43:52.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.690+0000 7fb5e359e700 1 -- 172.21.15.92:0/1191809777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e40ca290 con 0x7fb5e40b2100 2023-10-06T10:43:52.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.690+0000 7fb5daffd700 1 -- 172.21.15.92:0/1191809777 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d40117a0 con 0x7fb5e40b2100 2023-10-06T10:43:52.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.690+0000 7fb5e359e700 1 -- 172.21.15.92:0/1191809777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e40c99c0 con 0x7fb5e40b2100 2023-10-06T10:43:52.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.691+0000 7fb5e359e700 1 -- 172.21.15.92:0/1191809777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e40ca0f0 con 0x7fb5e40b2100 2023-10-06T10:43:52.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.691+0000 7fb5daffd700 1 -- 172.21.15.92:0/1191809777 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb5d4011bd0 con 0x7fb5e40b2100 2023-10-06T10:43:52.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.691+0000 7fb5daffd700 1 --2- 172.21.15.92:0/1191809777 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb5c4066f30 0x7fb5c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:52.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.691+0000 7fb5daffd700 1 -- 172.21.15.92:0/1191809777 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb5d4085cb0 con 0x7fb5e40b2100 2023-10-06T10:43:52.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.692+0000 7fb5d9d9b700 1 --2- 172.21.15.92:0/1191809777 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb5c4066f30 0x7fb5c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:52.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.692+0000 7fb5d9d9b700 1 --2- 172.21.15.92:0/1191809777 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb5c4066f30 0x7fb5c40693e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb5e40d0d60 tx=0x7fb5cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:52.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.696+0000 7fb5daffd700 1 -- 172.21.15.92:0/1191809777 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5e40ca0f0 con 0x7fb5e40b2100 2023-10-06T10:43:52.766 INFO:tasks.ceph.mds.b.smithi124.stderr:2023-10-06T10:43:52.764+0000 7f557baa9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T10:43:52.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.874+0000 7fb5e359e700 1 -- 172.21.15.92:0/1191809777 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb5e40ca0f0 con 0x7fb5c4066f30 2023-10-06T10:43:52.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.882+0000 7fb5daffd700 1 -- 172.21.15.92:0/1191809777 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311470 (secure 0 0 0) 0x7fb5e40ca0f0 con 0x7fb5c4066f30 2023-10-06T10:43:52.887 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:52.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.886+0000 7fb5e359e700 1 -- 172.21.15.92:0/1191809777 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb5c4066f30 msgr2=0x7fb5c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:52.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.886+0000 7fb5e359e700 1 --2- 172.21.15.92:0/1191809777 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb5c4066f30 0x7fb5c40693e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb5e40d0d60 tx=0x7fb5cc009040 comp rx=0 tx=0).stop 2023-10-06T10:43:52.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.886+0000 7fb5e359e700 1 -- 172.21.15.92:0/1191809777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e40b2100 msgr2=0x7fb5e40c9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:52.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.886+0000 7fb5e359e700 1 --2- 172.21.15.92:0/1191809777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e40b2100 0x7fb5e40c9480 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4069150 tx=0x7fb5d4017000 comp rx=0 tx=0).stop 2023-10-06T10:43:52.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.886+0000 7fb5e359e700 1 -- 172.21.15.92:0/1191809777 shutdown_connections 2023-10-06T10:43:52.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.886+0000 7fb5e359e700 1 --2- 172.21.15.92:0/1191809777 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb5c4066f30 0x7fb5c40693e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.886+0000 7fb5e359e700 1 --2- 172.21.15.92:0/1191809777 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5e4159440 0x7fb5e40c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.886+0000 7fb5e359e700 1 --2- 172.21.15.92:0/1191809777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e40b2100 0x7fb5e40c9480 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.886+0000 7fb5e359e700 1 --2- 172.21.15.92:0/1191809777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5e40b35d0 0x7fb5e40cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:52.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.886+0000 7fb5e359e700 1 -- 172.21.15.92:0/1191809777 >> 172.21.15.92:0/1191809777 conn(0x7fb5e406ece0 msgr2=0x7fb5e40ccf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:52.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.886+0000 7fb5e359e700 1 -- 172.21.15.92:0/1191809777 shutdown_connections 2023-10-06T10:43:52.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:52.886+0000 7fb5e359e700 1 -- 172.21.15.92:0/1191809777 wait complete. 2023-10-06T10:43:52.891 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:43:52.899 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2023-10-06T10:43:52.155031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2388092,"kb_used_data":9788,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752586628,"statfs":{"total":773094113280,"available":770648707072,"internally_reserved":0,"allocated":10022912,"data_stored":5405539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.697641"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351426+0000","last_change":"2023-10-06T10:43:48.351426+0000","last_active":"2023-10-06T10:43:48.351426+0000","last_peered":"2023-10-06T10:43:48.351426+0000","last_clean":"2023-10-06T10:43:48.351426+0000","last_became_active":"2023-10-06T10:43:48.351185+0000","last_became_peered":"2023-10-06T10:43:48.351185+0000","last_unstale":"2023-10-06T10:43:48.351426+0000","last_undegraded":"2023-10-06T10:43:48.351426+0000","last_fullsized":"2023-10-06T10:43:48.351426+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:36:34.819923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.347665+0000","last_change":"2023-10-06T10:43:48.347665+0000","last_active":"2023-10-06T10:43:48.347665+0000","last_peered":"2023-10-06T10:43:48.347665+0000","last_clean":"2023-10-06T10:43:48.347665+0000","last_became_active":"2023-10-06T10:43:48.347420+0000","last_became_peered":"2023-10-06T10:43:48.347420+0000","last_unstale":"2023-10-06T10:43:48.347665+0000","last_undegraded":"2023-10-06T10:43:48.347665+0000","last_fullsized":"2023-10-06T10:43:48.347665+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:48:04.646078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351418+0000","last_change":"2023-10-06T10:43:48.351418+0000","last_active":"2023-10-06T10:43:48.351418+0000","last_peered":"2023-10-06T10:43:48.351418+0000","last_clean":"2023-10-06T10:43:48.351418+0000","last_became_active":"2023-10-06T10:43:48.351140+0000","last_became_peered":"2023-10-06T10:43:48.351140+0000","last_unstale":"2023-10-06T10:43:48.351418+0000","last_undegraded":"2023-10-06T10:43:48.351418+0000","last_fullsized":"2023-10-06T10:43:48.351418+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:04.984298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.352355+0000","last_change":"2023-10-06T10:43:48.352355+0000","last_active":"2023-10-06T10:43:48.352355+0000","last_peered":"2023-10-06T10:43:48.352355+0000","last_clean":"2023-10-06T10:43:48.352355+0000","last_became_active":"2023-10-06T10:43:48.352104+0000","last_became_peered":"2023-10-06T10:43:48.352104+0000","last_unstale":"2023-10-06T10:43:48.352355+0000","last_undegraded":"2023-10-06T10:43:48.352355+0000","last_fullsized":"2023-10-06T10:43:48.352355+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:44.549060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.355530+0000","last_change":"2023-10-06T10:43:48.355530+0000","last_active":"2023-10-06T10:43:48.355530+0000","last_peered":"2023-10-06T10:43:48.355530+0000","last_clean":"2023-10-06T10:43:48.355530+0000","last_became_active":"2023-10-06T10:43:48.355239+0000","last_became_peered":"2023-10-06T10:43:48.355239+0000","last_unstale":"2023-10-06T10:43:48.355530+0000","last_undegraded":"2023-10-06T10:43:48.355530+0000","last_fullsized":"2023-10-06T10:43:48.355530+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:08.040718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351912+0000","last_change":"2023-10-06T10:43:48.351912+0000","last_active":"2023-10-06T10:43:48.351912+0000","last_peered":"2023-10-06T10:43:48.351912+0000","last_clean":"2023-10-06T10:43:48.351912+0000","last_became_active":"2023-10-06T10:43:48.351212+0000","last_became_peered":"2023-10-06T10:43:48.351212+0000","last_unstale":"2023-10-06T10:43:48.351912+0000","last_undegraded":"2023-10-06T10:43:48.351912+0000","last_fullsized":"2023-10-06T10:43:48.351912+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:57:52.254017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351472+0000","last_change":"2023-10-06T10:43:48.351472+0000","last_active":"2023-10-06T10:43:48.351472+0000","last_peered":"2023-10-06T10:43:48.351472+0000","last_clean":"2023-10-06T10:43:48.351472+0000","last_became_active":"2023-10-06T10:43:48.351200+0000","last_became_peered":"2023-10-06T10:43:48.351200+0000","last_unstale":"2023-10-06T10:43:48.351472+0000","last_undegraded":"2023-10-06T10:43:48.351472+0000","last_fullsized":"2023-10-06T10:43:48.351472+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:37:26.353942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.352913+0000","last_change":"2023-10-06T10:43:48.352913+0000","last_active":"2023-10-06T10:43:48.352913+0000","last_peered":"2023-10-06T10:43:48.352913+0000","last_clean":"2023-10-06T10:43:48.352913+0000","last_became_active":"2023-10-06T10:43:48.352630+0000","last_became_peered":"2023-10-06T10:43:48.352630+0000","last_unstale":"2023-10-06T10:43:48.352913+0000","last_undegraded":"2023-10-06T10:43:48.352913+0000","last_fullsized":"2023-10-06T10:43:48.352913+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:25.393203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.358898+0000","last_change":"2023-10-06T10:43:48.358898+0000","last_active":"2023-10-06T10:43:48.358898+0000","last_peered":"2023-10-06T10:43:48.358898+0000","last_clean":"2023-10-06T10:43:48.358898+0000","last_became_active":"2023-10-06T10:43:48.358638+0000","last_became_peered":"2023-10-06T10:43:48.358638+0000","last_unstale":"2023-10-06T10:43:48.358898+0000","last_undegraded":"2023-10-06T10:43:48.358898+0000","last_fullsized":"2023-10-06T10:43:48.358898+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:28:19.852395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.355333+0000","last_change":"2023-10-06T10:43:48.355333+0000","last_active":"2023-10-06T10:43:48.355333+0000","last_peered":"2023-10-06T10:43:48.355333+0000","last_clean":"2023-10-06T10:43:48.355333+0000","last_became_active":"2023-10-06T10:43:48.355026+0000","last_became_peered":"2023-10-06T10:43:48.355026+0000","last_unstale":"2023-10-06T10:43:48.355333+0000","last_undegraded":"2023-10-06T10:43:48.355333+0000","last_fullsized":"2023-10-06T10:43:48.355333+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:56.460307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.358429+0000","last_change":"2023-10-06T10:43:48.358429+0000","last_active":"2023-10-06T10:43:48.358429+0000","last_peered":"2023-10-06T10:43:48.358429+0000","last_clean":"2023-10-06T10:43:48.358429+0000","last_became_active":"2023-10-06T10:43:48.357519+0000","last_became_peered":"2023-10-06T10:43:48.357519+0000","last_unstale":"2023-10-06T10:43:48.358429+0000","last_undegraded":"2023-10-06T10:43:48.358429+0000","last_fullsized":"2023-10-06T10:43:48.358429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:12:22.078056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.352000+0000","last_change":"2023-10-06T10:43:48.352000+0000","last_active":"2023-10-06T10:43:48.352000+0000","last_peered":"2023-10-06T10:43:48.352000+0000","last_clean":"2023-10-06T10:43:48.352000+0000","last_became_active":"2023-10-06T10:43:48.351741+0000","last_became_peered":"2023-10-06T10:43:48.351741+0000","last_unstale":"2023-10-06T10:43:48.352000+0000","last_undegraded":"2023-10-06T10:43:48.352000+0000","last_fullsized":"2023-10-06T10:43:48.352000+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:41:32.626627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.357087+0000","last_change":"2023-10-06T10:43:48.357087+0000","last_active":"2023-10-06T10:43:48.357087+0000","last_peered":"2023-10-06T10:43:48.357087+0000","last_clean":"2023-10-06T10:43:48.357087+0000","last_became_active":"2023-10-06T10:43:48.356841+0000","last_became_peered":"2023-10-06T10:43:48.356841+0000","last_unstale":"2023-10-06T10:43:48.357087+0000","last_undegraded":"2023-10-06T10:43:48.357087+0000","last_fullsized":"2023-10-06T10:43:48.357087+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:41:15.115031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.352548+0000","last_change":"2023-10-06T10:43:48.352548+0000","last_active":"2023-10-06T10:43:48.352548+0000","last_peered":"2023-10-06T10:43:48.352548+0000","last_clean":"2023-10-06T10:43:48.352548+0000","last_became_active":"2023-10-06T10:43:48.352317+0000","last_became_peered":"2023-10-06T10:43:48.352317+0000","last_unstale":"2023-10-06T10:43:48.352548+0000","last_undegraded":"2023-10-06T10:43:48.352548+0000","last_fullsized":"2023-10-06T10:43:48.352548+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:10.707456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.353097+0000","last_change":"2023-10-06T10:43:48.353097+0000","last_active":"2023-10-06T10:43:48.353097+0000","last_peered":"2023-10-06T10:43:48.353097+0000","last_clean":"2023-10-06T10:43:48.353097+0000","last_became_active":"2023-10-06T10:43:48.352863+0000","last_became_peered":"2023-10-06T10:43:48.352863+0000","last_unstale":"2023-10-06T10:43:48.353097+0000","last_undegraded":"2023-10-06T10:43:48.353097+0000","last_fullsized":"2023-10-06T10:43:48.353097+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:44.794731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.357122+0000","last_change":"2023-10-06T10:43:48.357122+0000","last_active":"2023-10-06T10:43:48.357122+0000","last_peered":"2023-10-06T10:43:48.357122+0000","last_clean":"2023-10-06T10:43:48.357122+0000","last_became_active":"2023-10-06T10:43:48.356789+0000","last_became_peered":"2023-10-06T10:43:48.356789+0000","last_unstale":"2023-10-06T10:43:48.357122+0000","last_undegraded":"2023-10-06T10:43:48.357122+0000","last_fullsized":"2023-10-06T10:43:48.357122+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:57:21.913042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.349105+0000","last_change":"2023-10-06T10:43:48.349105+0000","last_active":"2023-10-06T10:43:48.349105+0000","last_peered":"2023-10-06T10:43:48.349105+0000","last_clean":"2023-10-06T10:43:48.349105+0000","last_became_active":"2023-10-06T10:43:48.348842+0000","last_became_peered":"2023-10-06T10:43:48.348842+0000","last_unstale":"2023-10-06T10:43:48.349105+0000","last_undegraded":"2023-10-06T10:43:48.349105+0000","last_fullsized":"2023-10-06T10:43:48.349105+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:39:00.691728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.352184+0000","last_change":"2023-10-06T10:43:48.352184+0000","last_active":"2023-10-06T10:43:48.352184+0000","last_peered":"2023-10-06T10:43:48.352184+0000","last_clean":"2023-10-06T10:43:48.352184+0000","last_became_active":"2023-10-06T10:43:48.351926+0000","last_became_peered":"2023-10-06T10:43:48.351926+0000","last_unstale":"2023-10-06T10:43:48.352184+0000","last_undegraded":"2023-10-06T10:43:48.352184+0000","last_fullsized":"2023-10-06T10:43:48.352184+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:36.582208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.352025+0000","last_change":"2023-10-06T10:43:48.352025+0000","last_active":"2023-10-06T10:43:48.352025+0000","last_peered":"2023-10-06T10:43:48.352025+0000","last_clean":"2023-10-06T10:43:48.352025+0000","last_became_active":"2023-10-06T10:43:48.351767+0000","last_became_peered":"2023-10-06T10:43:48.351767+0000","last_unstale":"2023-10-06T10:43:48.352025+0000","last_undegraded":"2023-10-06T10:43:48.352025+0000","last_fullsized":"2023-10-06T10:43:48.352025+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:45:02.178691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.349548+0000","last_change":"2023-10-06T10:43:48.349548+0000","last_active":"2023-10-06T10:43:48.349548+0000","last_peered":"2023-10-06T10:43:48.349548+0000","last_clean":"2023-10-06T10:43:48.349548+0000","last_became_active":"2023-10-06T10:43:48.349293+0000","last_became_peered":"2023-10-06T10:43:48.349293+0000","last_unstale":"2023-10-06T10:43:48.349548+0000","last_undegraded":"2023-10-06T10:43:48.349548+0000","last_fullsized":"2023-10-06T10:43:48.349548+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:30.190139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351362+0000","last_change":"2023-10-06T10:43:48.351362+0000","last_active":"2023-10-06T10:43:48.351362+0000","last_peered":"2023-10-06T10:43:48.351362+0000","last_clean":"2023-10-06T10:43:48.351362+0000","last_became_active":"2023-10-06T10:43:48.350937+0000","last_became_peered":"2023-10-06T10:43:48.350937+0000","last_unstale":"2023-10-06T10:43:48.351362+0000","last_undegraded":"2023-10-06T10:43:48.351362+0000","last_fullsized":"2023-10-06T10:43:48.351362+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:22:32.640917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.344804+0000","last_change":"2023-10-06T10:43:48.344804+0000","last_active":"2023-10-06T10:43:48.344804+0000","last_peered":"2023-10-06T10:43:48.344804+0000","last_clean":"2023-10-06T10:43:48.344804+0000","last_became_active":"2023-10-06T10:43:48.344565+0000","last_became_peered":"2023-10-06T10:43:48.344565+0000","last_unstale":"2023-10-06T10:43:48.344804+0000","last_undegraded":"2023-10-06T10:43:48.344804+0000","last_fullsized":"2023-10-06T10:43:48.344804+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:44.794731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.345181+0000","last_change":"2023-10-06T10:43:48.345181+0000","last_active":"2023-10-06T10:43:48.345181+0000","last_peered":"2023-10-06T10:43:48.345181+0000","last_clean":"2023-10-06T10:43:48.345181+0000","last_became_active":"2023-10-06T10:43:48.344909+0000","last_became_peered":"2023-10-06T10:43:48.344909+0000","last_unstale":"2023-10-06T10:43:48.345181+0000","last_undegraded":"2023-10-06T10:43:48.345181+0000","last_fullsized":"2023-10-06T10:43:48.345181+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:11:07.128574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.353490+0000","last_change":"2023-10-06T10:43:48.353490+0000","last_active":"2023-10-06T10:43:48.353490+0000","last_peered":"2023-10-06T10:43:48.353490+0000","last_clean":"2023-10-06T10:43:48.353490+0000","last_became_active":"2023-10-06T10:43:48.352934+0000","last_became_peered":"2023-10-06T10:43:48.352934+0000","last_unstale":"2023-10-06T10:43:48.353490+0000","last_undegraded":"2023-10-06T10:43:48.353490+0000","last_fullsized":"2023-10-06T10:43:48.353490+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:06:22.716853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351676+0000","last_change":"2023-10-06T10:43:48.351676+0000","last_active":"2023-10-06T10:43:48.351676+0000","last_peered":"2023-10-06T10:43:48.351676+0000","last_clean":"2023-10-06T10:43:48.351676+0000","last_became_active":"2023-10-06T10:43:48.351406+0000","last_became_peered":"2023-10-06T10:43:48.351406+0000","last_unstale":"2023-10-06T10:43:48.351676+0000","last_undegraded":"2023-10-06T10:43:48.351676+0000","last_fullsized":"2023-10-06T10:43:48.351676+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:25.567935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.349351+0000","last_change":"2023-10-06T10:43:48.349351+0000","last_active":"2023-10-06T10:43:48.349351+0000","last_peered":"2023-10-06T10:43:48.349351+0000","last_clean":"2023-10-06T10:43:48.349351+0000","last_became_active":"2023-10-06T10:43:48.349110+0000","last_became_peered":"2023-10-06T10:43:48.349110+0000","last_unstale":"2023-10-06T10:43:48.349351+0000","last_undegraded":"2023-10-06T10:43:48.349351+0000","last_fullsized":"2023-10-06T10:43:48.349351+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:39.267032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351855+0000","last_change":"2023-10-06T10:43:48.351855+0000","last_active":"2023-10-06T10:43:48.351855+0000","last_peered":"2023-10-06T10:43:48.351855+0000","last_clean":"2023-10-06T10:43:48.351855+0000","last_became_active":"2023-10-06T10:43:48.351606+0000","last_became_peered":"2023-10-06T10:43:48.351606+0000","last_unstale":"2023-10-06T10:43:48.351855+0000","last_undegraded":"2023-10-06T10:43:48.351855+0000","last_fullsized":"2023-10-06T10:43:48.351855+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:11:41.020256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.356027+0000","last_change":"2023-10-06T10:43:48.356027+0000","last_active":"2023-10-06T10:43:48.356027+0000","last_peered":"2023-10-06T10:43:48.356027+0000","last_clean":"2023-10-06T10:43:48.356027+0000","last_became_active":"2023-10-06T10:43:48.355758+0000","last_became_peered":"2023-10-06T10:43:48.355758+0000","last_unstale":"2023-10-06T10:43:48.356027+0000","last_undegraded":"2023-10-06T10:43:48.356027+0000","last_fullsized":"2023-10-06T10:43:48.356027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:00.307376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.348641+0000","last_change":"2023-10-06T10:43:48.348641+0000","last_active":"2023-10-06T10:43:48.348641+0000","last_peered":"2023-10-06T10:43:48.348641+0000","last_clean":"2023-10-06T10:43:48.348641+0000","last_became_active":"2023-10-06T10:43:48.348345+0000","last_became_peered":"2023-10-06T10:43:48.348345+0000","last_unstale":"2023-10-06T10:43:48.348641+0000","last_undegraded":"2023-10-06T10:43:48.348641+0000","last_fullsized":"2023-10-06T10:43:48.348641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:07:38.821665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.354294+0000","last_change":"2023-10-06T10:43:48.354294+0000","last_active":"2023-10-06T10:43:48.354294+0000","last_peered":"2023-10-06T10:43:48.354294+0000","last_clean":"2023-10-06T10:43:48.354294+0000","last_became_active":"2023-10-06T10:43:48.354009+0000","last_became_peered":"2023-10-06T10:43:48.354009+0000","last_unstale":"2023-10-06T10:43:48.354294+0000","last_undegraded":"2023-10-06T10:43:48.354294+0000","last_fullsized":"2023-10-06T10:43:48.354294+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:01:23.602658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.350744+0000","last_change":"2023-10-06T10:43:48.350744+0000","last_active":"2023-10-06T10:43:48.350744+0000","last_peered":"2023-10-06T10:43:48.350744+0000","last_clean":"2023-10-06T10:43:48.350744+0000","last_became_active":"2023-10-06T10:43:48.350490+0000","last_became_peered":"2023-10-06T10:43:48.350490+0000","last_unstale":"2023-10-06T10:43:48.350744+0000","last_undegraded":"2023-10-06T10:43:48.350744+0000","last_fullsized":"2023-10-06T10:43:48.350744+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:22:33.960812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.348417+0000","last_change":"2023-10-06T10:43:48.348417+0000","last_active":"2023-10-06T10:43:48.348417+0000","last_peered":"2023-10-06T10:43:48.348417+0000","last_clean":"2023-10-06T10:43:48.348417+0000","last_became_active":"2023-10-06T10:43:48.348181+0000","last_became_peered":"2023-10-06T10:43:48.348181+0000","last_unstale":"2023-10-06T10:43:48.348417+0000","last_undegraded":"2023-10-06T10:43:48.348417+0000","last_fullsized":"2023-10-06T10:43:48.348417+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:52.926879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.353701+0000","last_change":"2023-10-06T10:43:48.353701+0000","last_active":"2023-10-06T10:43:48.353701+0000","last_peered":"2023-10-06T10:43:48.353701+0000","last_clean":"2023-10-06T10:43:48.353701+0000","last_became_active":"2023-10-06T10:43:48.351714+0000","last_became_peered":"2023-10-06T10:43:48.351714+0000","last_unstale":"2023-10-06T10:43:48.353701+0000","last_undegraded":"2023-10-06T10:43:48.353701+0000","last_fullsized":"2023-10-06T10:43:48.353701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:36:57.274188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"89'4","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.498122+0000","last_change":"2023-10-06T10:43:47.483577+0000","last_active":"2023-10-06T10:43:48.498122+0000","last_peered":"2023-10-06T10:43:48.498122+0000","last_clean":"2023-10-06T10:43:48.498122+0000","last_became_active":"2023-10-06T10:43:47.483355+0000","last_became_peered":"2023-10-06T10:43:47.483355+0000","last_unstale":"2023-10-06T10:43:48.498122+0000","last_undegraded":"2023-10-06T10:43:48.498122+0000","last_fullsized":"2023-10-06T10:43:48.498122+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:11:40.016496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.355144+0000","last_change":"2023-10-06T10:43:48.355144+0000","last_active":"2023-10-06T10:43:48.355144+0000","last_peered":"2023-10-06T10:43:48.355144+0000","last_clean":"2023-10-06T10:43:48.355144+0000","last_became_active":"2023-10-06T10:43:48.354870+0000","last_became_peered":"2023-10-06T10:43:48.354870+0000","last_unstale":"2023-10-06T10:43:48.355144+0000","last_undegraded":"2023-10-06T10:43:48.355144+0000","last_fullsized":"2023-10-06T10:43:48.355144+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:29:27.392646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"89'1","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.343744+0000","last_change":"2023-10-06T10:43:47.481283+0000","last_active":"2023-10-06T10:43:48.343744+0000","last_peered":"2023-10-06T10:43:48.343744+0000","last_clean":"2023-10-06T10:43:48.343744+0000","last_became_active":"2023-10-06T10:43:47.481026+0000","last_became_peered":"2023-10-06T10:43:47.481026+0000","last_unstale":"2023-10-06T10:43:48.343744+0000","last_undegraded":"2023-10-06T10:43:48.343744+0000","last_fullsized":"2023-10-06T10:43:48.343744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:40:10.471619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.357292+0000","last_change":"2023-10-06T10:43:48.357292+0000","last_active":"2023-10-06T10:43:48.357292+0000","last_peered":"2023-10-06T10:43:48.357292+0000","last_clean":"2023-10-06T10:43:48.357292+0000","last_became_active":"2023-10-06T10:43:48.356997+0000","last_became_peered":"2023-10-06T10:43:48.356997+0000","last_unstale":"2023-10-06T10:43:48.357292+0000","last_undegraded":"2023-10-06T10:43:48.357292+0000","last_fullsized":"2023-10-06T10:43:48.357292+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:10.707456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"89'2","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.499403+0000","last_change":"2023-10-06T10:43:47.481202+0000","last_active":"2023-10-06T10:43:48.499403+0000","last_peered":"2023-10-06T10:43:48.499403+0000","last_clean":"2023-10-06T10:43:48.499403+0000","last_became_active":"2023-10-06T10:43:47.480948+0000","last_became_peered":"2023-10-06T10:43:47.480948+0000","last_unstale":"2023-10-06T10:43:48.499403+0000","last_undegraded":"2023-10-06T10:43:48.499403+0000","last_fullsized":"2023-10-06T10:43:48.499403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:29:26.388885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.349180+0000","last_change":"2023-10-06T10:43:48.349180+0000","last_active":"2023-10-06T10:43:48.349180+0000","last_peered":"2023-10-06T10:43:48.349180+0000","last_clean":"2023-10-06T10:43:48.349180+0000","last_became_active":"2023-10-06T10:43:48.348915+0000","last_became_peered":"2023-10-06T10:43:48.348915+0000","last_unstale":"2023-10-06T10:43:48.349180+0000","last_undegraded":"2023-10-06T10:43:48.349180+0000","last_fullsized":"2023-10-06T10:43:48.349180+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:44.794731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"89'2","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.495361+0000","last_change":"2023-10-06T10:43:47.478977+0000","last_active":"2023-10-06T10:43:48.495361+0000","last_peered":"2023-10-06T10:43:48.495361+0000","last_clean":"2023-10-06T10:43:48.495361+0000","last_became_active":"2023-10-06T10:43:47.478586+0000","last_became_peered":"2023-10-06T10:43:47.478586+0000","last_unstale":"2023-10-06T10:43:48.495361+0000","last_undegraded":"2023-10-06T10:43:48.495361+0000","last_fullsized":"2023-10-06T10:43:48.495361+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:55.456547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351331+0000","last_change":"2023-10-06T10:43:48.351331+0000","last_active":"2023-10-06T10:43:48.351331+0000","last_peered":"2023-10-06T10:43:48.351331+0000","last_clean":"2023-10-06T10:43:48.351331+0000","last_became_active":"2023-10-06T10:43:48.351064+0000","last_became_peered":"2023-10-06T10:43:48.351064+0000","last_unstale":"2023-10-06T10:43:48.351331+0000","last_undegraded":"2023-10-06T10:43:48.351331+0000","last_fullsized":"2023-10-06T10:43:48.351331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:02:28.611301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"89'1","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.342913+0000","last_change":"2023-10-06T10:43:47.477361+0000","last_active":"2023-10-06T10:43:48.342913+0000","last_peered":"2023-10-06T10:43:48.342913+0000","last_clean":"2023-10-06T10:43:48.342913+0000","last_became_active":"2023-10-06T10:43:47.477114+0000","last_became_peered":"2023-10-06T10:43:47.477114+0000","last_unstale":"2023-10-06T10:43:48.342913+0000","last_undegraded":"2023-10-06T10:43:48.342913+0000","last_fullsized":"2023-10-06T10:43:48.342913+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:23.491319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.352595+0000","last_change":"2023-10-06T10:43:48.352595+0000","last_active":"2023-10-06T10:43:48.352595+0000","last_peered":"2023-10-06T10:43:48.352595+0000","last_clean":"2023-10-06T10:43:48.352595+0000","last_became_active":"2023-10-06T10:43:48.352336+0000","last_became_peered":"2023-10-06T10:43:48.352336+0000","last_unstale":"2023-10-06T10:43:48.352595+0000","last_undegraded":"2023-10-06T10:43:48.352595+0000","last_fullsized":"2023-10-06T10:43:48.352595+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:13:21.303231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"89'1","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.338289+0000","last_change":"2023-10-06T10:43:47.481917+0000","last_active":"2023-10-06T10:43:48.338289+0000","last_peered":"2023-10-06T10:43:48.338289+0000","last_clean":"2023-10-06T10:43:48.338289+0000","last_became_active":"2023-10-06T10:43:47.481681+0000","last_became_peered":"2023-10-06T10:43:47.481681+0000","last_unstale":"2023-10-06T10:43:48.338289+0000","last_undegraded":"2023-10-06T10:43:48.338289+0000","last_fullsized":"2023-10-06T10:43:48.338289+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:24.564175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.353199+0000","last_change":"2023-10-06T10:43:48.353199+0000","last_active":"2023-10-06T10:43:48.353199+0000","last_peered":"2023-10-06T10:43:48.353199+0000","last_clean":"2023-10-06T10:43:48.353199+0000","last_became_active":"2023-10-06T10:43:48.352967+0000","last_became_peered":"2023-10-06T10:43:48.352967+0000","last_unstale":"2023-10-06T10:43:48.353199+0000","last_undegraded":"2023-10-06T10:43:48.353199+0000","last_fullsized":"2023-10-06T10:43:48.353199+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:52:45.733863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.482698+0000","last_change":"2023-10-06T10:43:47.476010+0000","last_active":"2023-10-06T10:43:47.482698+0000","last_peered":"2023-10-06T10:43:47.482698+0000","last_clean":"2023-10-06T10:43:47.482698+0000","last_became_active":"2023-10-06T10:43:47.475749+0000","last_became_peered":"2023-10-06T10:43:47.475749+0000","last_unstale":"2023-10-06T10:43:47.482698+0000","last_undegraded":"2023-10-06T10:43:47.482698+0000","last_fullsized":"2023-10-06T10:43:47.482698+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:29:27.051793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.349220+0000","last_change":"2023-10-06T10:43:48.349220+0000","last_active":"2023-10-06T10:43:48.349220+0000","last_peered":"2023-10-06T10:43:48.349220+0000","last_clean":"2023-10-06T10:43:48.349220+0000","last_became_active":"2023-10-06T10:43:48.348948+0000","last_became_peered":"2023-10-06T10:43:48.348948+0000","last_unstale":"2023-10-06T10:43:48.349220+0000","last_undegraded":"2023-10-06T10:43:48.349220+0000","last_fullsized":"2023-10-06T10:43:48.349220+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:25.393203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.481114+0000","last_change":"2023-10-06T10:43:47.475799+0000","last_active":"2023-10-06T10:43:47.481114+0000","last_peered":"2023-10-06T10:43:47.481114+0000","last_clean":"2023-10-06T10:43:47.481114+0000","last_became_active":"2023-10-06T10:43:47.475576+0000","last_became_peered":"2023-10-06T10:43:47.475576+0000","last_unstale":"2023-10-06T10:43:47.481114+0000","last_undegraded":"2023-10-06T10:43:47.481114+0000","last_fullsized":"2023-10-06T10:43:47.481114+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:43:16.112346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351335+0000","last_change":"2023-10-06T10:43:48.351335+0000","last_active":"2023-10-06T10:43:48.351335+0000","last_peered":"2023-10-06T10:43:48.351335+0000","last_clean":"2023-10-06T10:43:48.351335+0000","last_became_active":"2023-10-06T10:43:48.351068+0000","last_became_peered":"2023-10-06T10:43:48.351068+0000","last_unstale":"2023-10-06T10:43:48.351335+0000","last_undegraded":"2023-10-06T10:43:48.351335+0000","last_fullsized":"2023-10-06T10:43:48.351335+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:21:58.749235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.477892+0000","last_change":"2023-10-06T10:43:47.473232+0000","last_active":"2023-10-06T10:43:47.477892+0000","last_peered":"2023-10-06T10:43:47.477892+0000","last_clean":"2023-10-06T10:43:47.477892+0000","last_became_active":"2023-10-06T10:43:47.472990+0000","last_became_peered":"2023-10-06T10:43:47.472990+0000","last_unstale":"2023-10-06T10:43:47.477892+0000","last_undegraded":"2023-10-06T10:43:47.477892+0000","last_fullsized":"2023-10-06T10:43:47.477892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:10.651805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351703+0000","last_change":"2023-10-06T10:43:48.351703+0000","last_active":"2023-10-06T10:43:48.351703+0000","last_peered":"2023-10-06T10:43:48.351703+0000","last_clean":"2023-10-06T10:43:48.351703+0000","last_became_active":"2023-10-06T10:43:48.351389+0000","last_became_peered":"2023-10-06T10:43:48.351389+0000","last_unstale":"2023-10-06T10:43:48.351703+0000","last_undegraded":"2023-10-06T10:43:48.351703+0000","last_fullsized":"2023-10-06T10:43:48.351703+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:17.993552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.482702+0000","last_change":"2023-10-06T10:43:47.479795+0000","last_active":"2023-10-06T10:43:47.482702+0000","last_peered":"2023-10-06T10:43:47.482702+0000","last_clean":"2023-10-06T10:43:47.482702+0000","last_became_active":"2023-10-06T10:43:47.479486+0000","last_became_peered":"2023-10-06T10:43:47.479486+0000","last_unstale":"2023-10-06T10:43:47.482702+0000","last_undegraded":"2023-10-06T10:43:47.482702+0000","last_fullsized":"2023-10-06T10:43:47.482702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:12:21.074296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351635+0000","last_change":"2023-10-06T10:43:48.351635+0000","last_active":"2023-10-06T10:43:48.351635+0000","last_peered":"2023-10-06T10:43:48.351635+0000","last_clean":"2023-10-06T10:43:48.351635+0000","last_became_active":"2023-10-06T10:43:48.351340+0000","last_became_peered":"2023-10-06T10:43:48.351340+0000","last_unstale":"2023-10-06T10:43:48.351635+0000","last_undegraded":"2023-10-06T10:43:48.351635+0000","last_fullsized":"2023-10-06T10:43:48.351635+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:04.472718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"89'3","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.496698+0000","last_change":"2023-10-06T10:43:47.480672+0000","last_active":"2023-10-06T10:43:48.496698+0000","last_peered":"2023-10-06T10:43:48.496698+0000","last_clean":"2023-10-06T10:43:48.496698+0000","last_became_active":"2023-10-06T10:43:47.480407+0000","last_became_peered":"2023-10-06T10:43:47.480407+0000","last_unstale":"2023-10-06T10:43:48.496698+0000","last_undegraded":"2023-10-06T10:43:48.496698+0000","last_fullsized":"2023-10-06T10:43:48.496698+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:18:26.575242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.357051+0000","last_change":"2023-10-06T10:43:48.357051+0000","last_active":"2023-10-06T10:43:48.357051+0000","last_peered":"2023-10-06T10:43:48.357051+0000","last_clean":"2023-10-06T10:43:48.357051+0000","last_became_active":"2023-10-06T10:43:48.356799+0000","last_became_peered":"2023-10-06T10:43:48.356799+0000","last_unstale":"2023-10-06T10:43:48.357051+0000","last_undegraded":"2023-10-06T10:43:48.357051+0000","last_fullsized":"2023-10-06T10:43:48.357051+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:07:37.438442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"89'6","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.502502+0000","last_change":"2023-10-06T10:43:47.474744+0000","last_active":"2023-10-06T10:43:48.502502+0000","last_peered":"2023-10-06T10:43:48.502502+0000","last_clean":"2023-10-06T10:43:48.502502+0000","last_became_active":"2023-10-06T10:43:47.474483+0000","last_became_peered":"2023-10-06T10:43:47.474483+0000","last_unstale":"2023-10-06T10:43:48.502502+0000","last_undegraded":"2023-10-06T10:43:48.502502+0000","last_fullsized":"2023-10-06T10:43:48.502502+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:07.036958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351834+0000","last_change":"2023-10-06T10:43:48.351834+0000","last_active":"2023-10-06T10:43:48.351834+0000","last_peered":"2023-10-06T10:43:48.351834+0000","last_clean":"2023-10-06T10:43:48.351834+0000","last_became_active":"2023-10-06T10:43:48.351591+0000","last_became_peered":"2023-10-06T10:43:48.351591+0000","last_unstale":"2023-10-06T10:43:48.351834+0000","last_undegraded":"2023-10-06T10:43:48.351834+0000","last_fullsized":"2023-10-06T10:43:48.351834+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:03:27.434250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"89'2","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.496250+0000","last_change":"2023-10-06T10:43:47.479328+0000","last_active":"2023-10-06T10:43:48.496250+0000","last_peered":"2023-10-06T10:43:48.496250+0000","last_clean":"2023-10-06T10:43:48.496250+0000","last_became_active":"2023-10-06T10:43:47.479092+0000","last_became_peered":"2023-10-06T10:43:47.479092+0000","last_unstale":"2023-10-06T10:43:48.496250+0000","last_undegraded":"2023-10-06T10:43:48.496250+0000","last_fullsized":"2023-10-06T10:43:48.496250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:13:30.750796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.350086+0000","last_change":"2023-10-06T10:43:48.350086+0000","last_active":"2023-10-06T10:43:48.350086+0000","last_peered":"2023-10-06T10:43:48.350086+0000","last_clean":"2023-10-06T10:43:48.350086+0000","last_became_active":"2023-10-06T10:43:48.349833+0000","last_became_peered":"2023-10-06T10:43:48.349833+0000","last_unstale":"2023-10-06T10:43:48.350086+0000","last_undegraded":"2023-10-06T10:43:48.350086+0000","last_fullsized":"2023-10-06T10:43:48.350086+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:31:49.281793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.482020+0000","last_change":"2023-10-06T10:43:47.477417+0000","last_active":"2023-10-06T10:43:47.482020+0000","last_peered":"2023-10-06T10:43:47.482020+0000","last_clean":"2023-10-06T10:43:47.482020+0000","last_became_active":"2023-10-06T10:43:47.477164+0000","last_became_peered":"2023-10-06T10:43:47.477164+0000","last_unstale":"2023-10-06T10:43:47.482020+0000","last_undegraded":"2023-10-06T10:43:47.482020+0000","last_fullsized":"2023-10-06T10:43:47.482020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:05.001802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.348736+0000","last_change":"2023-10-06T10:43:48.348736+0000","last_active":"2023-10-06T10:43:48.348736+0000","last_peered":"2023-10-06T10:43:48.348736+0000","last_clean":"2023-10-06T10:43:48.348736+0000","last_became_active":"2023-10-06T10:43:48.348420+0000","last_became_peered":"2023-10-06T10:43:48.348420+0000","last_unstale":"2023-10-06T10:43:48.348736+0000","last_undegraded":"2023-10-06T10:43:48.348736+0000","last_fullsized":"2023-10-06T10:43:48.348736+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:18.395189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.475299+0000","last_change":"2023-10-06T10:43:47.469427+0000","last_active":"2023-10-06T10:43:47.475299+0000","last_peered":"2023-10-06T10:43:47.475299+0000","last_clean":"2023-10-06T10:43:47.475299+0000","last_became_active":"2023-10-06T10:43:47.469159+0000","last_became_peered":"2023-10-06T10:43:47.469159+0000","last_unstale":"2023-10-06T10:43:47.475299+0000","last_undegraded":"2023-10-06T10:43:47.475299+0000","last_fullsized":"2023-10-06T10:43:47.475299+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:35:08.093726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.357617+0000","last_change":"2023-10-06T10:43:48.357617+0000","last_active":"2023-10-06T10:43:48.357617+0000","last_peered":"2023-10-06T10:43:48.357617+0000","last_clean":"2023-10-06T10:43:48.357617+0000","last_became_active":"2023-10-06T10:43:48.357307+0000","last_became_peered":"2023-10-06T10:43:48.357307+0000","last_unstale":"2023-10-06T10:43:48.357617+0000","last_undegraded":"2023-10-06T10:43:48.357617+0000","last_fullsized":"2023-10-06T10:43:48.357617+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:51.781585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"89'8","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.499897+0000","last_change":"2023-10-06T10:43:47.477030+0000","last_active":"2023-10-06T10:43:48.499897+0000","last_peered":"2023-10-06T10:43:48.499897+0000","last_clean":"2023-10-06T10:43:48.499897+0000","last_became_active":"2023-10-06T10:43:47.476777+0000","last_became_peered":"2023-10-06T10:43:47.476777+0000","last_unstale":"2023-10-06T10:43:48.499897+0000","last_undegraded":"2023-10-06T10:43:48.499897+0000","last_fullsized":"2023-10-06T10:43:48.499897+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:33:03.157191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.344777+0000","last_change":"2023-10-06T10:43:48.344777+0000","last_active":"2023-10-06T10:43:48.344777+0000","last_peered":"2023-10-06T10:43:48.344777+0000","last_clean":"2023-10-06T10:43:48.344777+0000","last_became_active":"2023-10-06T10:43:48.344463+0000","last_became_peered":"2023-10-06T10:43:48.344463+0000","last_unstale":"2023-10-06T10:43:48.344777+0000","last_undegraded":"2023-10-06T10:43:48.344777+0000","last_fullsized":"2023-10-06T10:43:48.344777+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:45:03.561894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"89'2","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.494969+0000","last_change":"2023-10-06T10:43:47.471640+0000","last_active":"2023-10-06T10:43:48.494969+0000","last_peered":"2023-10-06T10:43:48.494969+0000","last_clean":"2023-10-06T10:43:48.494969+0000","last_became_active":"2023-10-06T10:43:47.471411+0000","last_became_peered":"2023-10-06T10:43:47.471411+0000","last_unstale":"2023-10-06T10:43:48.494969+0000","last_undegraded":"2023-10-06T10:43:48.494969+0000","last_fullsized":"2023-10-06T10:43:48.494969+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:31:48.278032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.350041+0000","last_change":"2023-10-06T10:43:48.350041+0000","last_active":"2023-10-06T10:43:48.350041+0000","last_peered":"2023-10-06T10:43:48.350041+0000","last_clean":"2023-10-06T10:43:48.350041+0000","last_became_active":"2023-10-06T10:43:48.349786+0000","last_became_peered":"2023-10-06T10:43:48.349786+0000","last_unstale":"2023-10-06T10:43:48.350041+0000","last_undegraded":"2023-10-06T10:43:48.350041+0000","last_fullsized":"2023-10-06T10:43:48.350041+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:06.744223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.483035+0000","last_change":"2023-10-06T10:43:47.481804+0000","last_active":"2023-10-06T10:43:47.483035+0000","last_peered":"2023-10-06T10:43:47.483035+0000","last_clean":"2023-10-06T10:43:47.483035+0000","last_became_active":"2023-10-06T10:43:47.481553+0000","last_became_peered":"2023-10-06T10:43:47.481553+0000","last_unstale":"2023-10-06T10:43:47.483035+0000","last_undegraded":"2023-10-06T10:43:47.483035+0000","last_fullsized":"2023-10-06T10:43:47.483035+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:35.578447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.351356+0000","last_change":"2023-10-06T10:43:48.351356+0000","last_active":"2023-10-06T10:43:48.351356+0000","last_peered":"2023-10-06T10:43:48.351356+0000","last_clean":"2023-10-06T10:43:48.351356+0000","last_became_active":"2023-10-06T10:43:48.351071+0000","last_became_peered":"2023-10-06T10:43:48.351071+0000","last_unstale":"2023-10-06T10:43:48.351356+0000","last_undegraded":"2023-10-06T10:43:48.351356+0000","last_fullsized":"2023-10-06T10:43:48.351356+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:56.247192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.482238+0000","last_change":"2023-10-06T10:43:47.479767+0000","last_active":"2023-10-06T10:43:47.482238+0000","last_peered":"2023-10-06T10:43:47.482238+0000","last_clean":"2023-10-06T10:43:47.482238+0000","last_became_active":"2023-10-06T10:43:47.479492+0000","last_became_peered":"2023-10-06T10:43:47.479492+0000","last_unstale":"2023-10-06T10:43:47.482238+0000","last_undegraded":"2023-10-06T10:43:47.482238+0000","last_fullsized":"2023-10-06T10:43:47.482238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:59.303616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.355519+0000","last_change":"2023-10-06T10:43:48.355519+0000","last_active":"2023-10-06T10:43:48.355519+0000","last_peered":"2023-10-06T10:43:48.355519+0000","last_clean":"2023-10-06T10:43:48.355519+0000","last_became_active":"2023-10-06T10:43:48.355202+0000","last_became_peered":"2023-10-06T10:43:48.355202+0000","last_unstale":"2023-10-06T10:43:48.355519+0000","last_undegraded":"2023-10-06T10:43:48.355519+0000","last_fullsized":"2023-10-06T10:43:48.355519+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:17:41.222364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.482791+0000","last_change":"2023-10-06T10:43:47.476477+0000","last_active":"2023-10-06T10:43:47.482791+0000","last_peered":"2023-10-06T10:43:47.482791+0000","last_clean":"2023-10-06T10:43:47.482791+0000","last_became_active":"2023-10-06T10:43:47.476196+0000","last_became_peered":"2023-10-06T10:43:47.476196+0000","last_unstale":"2023-10-06T10:43:47.482791+0000","last_undegraded":"2023-10-06T10:43:47.482791+0000","last_fullsized":"2023-10-06T10:43:47.482791+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:49.331843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.357962+0000","last_change":"2023-10-06T10:43:48.357962+0000","last_active":"2023-10-06T10:43:48.357962+0000","last_peered":"2023-10-06T10:43:48.357962+0000","last_clean":"2023-10-06T10:43:48.357962+0000","last_became_active":"2023-10-06T10:43:48.357714+0000","last_became_peered":"2023-10-06T10:43:48.357714+0000","last_unstale":"2023-10-06T10:43:48.357962+0000","last_undegraded":"2023-10-06T10:43:48.357962+0000","last_fullsized":"2023-10-06T10:43:48.357962+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:16.939561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.482122+0000","last_change":"2023-10-06T10:43:47.479993+0000","last_active":"2023-10-06T10:43:47.482122+0000","last_peered":"2023-10-06T10:43:47.482122+0000","last_clean":"2023-10-06T10:43:47.482122+0000","last_became_active":"2023-10-06T10:43:47.479742+0000","last_became_peered":"2023-10-06T10:43:47.479742+0000","last_unstale":"2023-10-06T10:43:47.482122+0000","last_undegraded":"2023-10-06T10:43:47.482122+0000","last_fullsized":"2023-10-06T10:43:47.482122+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:03.468957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.352570+0000","last_change":"2023-10-06T10:43:48.352570+0000","last_active":"2023-10-06T10:43:48.352570+0000","last_peered":"2023-10-06T10:43:48.352570+0000","last_clean":"2023-10-06T10:43:48.352570+0000","last_became_active":"2023-10-06T10:43:48.352295+0000","last_became_peered":"2023-10-06T10:43:48.352295+0000","last_unstale":"2023-10-06T10:43:48.352570+0000","last_undegraded":"2023-10-06T10:43:48.352570+0000","last_fullsized":"2023-10-06T10:43:48.352570+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:21:58.749235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.485388+0000","last_change":"2023-10-06T10:43:47.481914+0000","last_active":"2023-10-06T10:43:47.485388+0000","last_peered":"2023-10-06T10:43:47.485388+0000","last_clean":"2023-10-06T10:43:47.485388+0000","last_became_active":"2023-10-06T10:43:47.481686+0000","last_became_peered":"2023-10-06T10:43:47.481686+0000","last_unstale":"2023-10-06T10:43:47.485388+0000","last_undegraded":"2023-10-06T10:43:47.485388+0000","last_fullsized":"2023-10-06T10:43:47.485388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:20:48.312164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.357650+0000","last_change":"2023-10-06T10:43:48.357650+0000","last_active":"2023-10-06T10:43:48.357650+0000","last_peered":"2023-10-06T10:43:48.357650+0000","last_clean":"2023-10-06T10:43:48.357650+0000","last_became_active":"2023-10-06T10:43:48.357232+0000","last_became_peered":"2023-10-06T10:43:48.357232+0000","last_unstale":"2023-10-06T10:43:48.357650+0000","last_undegraded":"2023-10-06T10:43:48.357650+0000","last_fullsized":"2023-10-06T10:43:48.357650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:43:17.116106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.477918+0000","last_change":"2023-10-06T10:43:47.474567+0000","last_active":"2023-10-06T10:43:47.477918+0000","last_peered":"2023-10-06T10:43:47.477918+0000","last_clean":"2023-10-06T10:43:47.477918+0000","last_became_active":"2023-10-06T10:43:47.474321+0000","last_became_peered":"2023-10-06T10:43:47.474321+0000","last_unstale":"2023-10-06T10:43:47.477918+0000","last_undegraded":"2023-10-06T10:43:47.477918+0000","last_fullsized":"2023-10-06T10:43:47.477918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:11:40.016496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.355771+0000","last_change":"2023-10-06T10:43:48.355771+0000","last_active":"2023-10-06T10:43:48.355771+0000","last_peered":"2023-10-06T10:43:48.355771+0000","last_clean":"2023-10-06T10:43:48.355771+0000","last_became_active":"2023-10-06T10:43:48.355522+0000","last_became_peered":"2023-10-06T10:43:48.355522+0000","last_unstale":"2023-10-06T10:43:48.355771+0000","last_undegraded":"2023-10-06T10:43:48.355771+0000","last_fullsized":"2023-10-06T10:43:48.355771+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:35:09.097486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.481013+0000","last_change":"2023-10-06T10:43:47.478822+0000","last_active":"2023-10-06T10:43:47.481013+0000","last_peered":"2023-10-06T10:43:47.481013+0000","last_clean":"2023-10-06T10:43:47.481013+0000","last_became_active":"2023-10-06T10:43:47.478590+0000","last_became_peered":"2023-10-06T10:43:47.478590+0000","last_unstale":"2023-10-06T10:43:47.481013+0000","last_undegraded":"2023-10-06T10:43:47.481013+0000","last_fullsized":"2023-10-06T10:43:47.481013+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:44.670215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.349477+0000","last_change":"2023-10-06T10:43:48.349477+0000","last_active":"2023-10-06T10:43:48.349477+0000","last_peered":"2023-10-06T10:43:48.349477+0000","last_clean":"2023-10-06T10:43:48.349477+0000","last_became_active":"2023-10-06T10:43:48.349222+0000","last_became_peered":"2023-10-06T10:43:48.349222+0000","last_unstale":"2023-10-06T10:43:48.349477+0000","last_undegraded":"2023-10-06T10:43:48.349477+0000","last_fullsized":"2023-10-06T10:43:48.349477+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:53:51.241025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.477368+0000","last_change":"2023-10-06T10:43:47.469428+0000","last_active":"2023-10-06T10:43:47.477368+0000","last_peered":"2023-10-06T10:43:47.477368+0000","last_clean":"2023-10-06T10:43:47.477368+0000","last_became_active":"2023-10-06T10:43:47.469059+0000","last_became_peered":"2023-10-06T10:43:47.469059+0000","last_unstale":"2023-10-06T10:43:47.477368+0000","last_undegraded":"2023-10-06T10:43:47.477368+0000","last_fullsized":"2023-10-06T10:43:47.477368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:18:26.575242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.463333+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:43:47.463333+0000","last_peered":"2023-10-06T10:43:47.463333+0000","last_clean":"2023-10-06T10:43:47.463333+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:43:47.463333+0000","last_undegraded":"2023-10-06T10:43:47.463333+0000","last_fullsized":"2023-10-06T10:43:47.463333+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.349870+0000","last_change":"2023-10-06T10:43:48.349870+0000","last_active":"2023-10-06T10:43:48.349870+0000","last_peered":"2023-10-06T10:43:48.349870+0000","last_clean":"2023-10-06T10:43:48.349870+0000","last_became_active":"2023-10-06T10:43:48.349630+0000","last_became_peered":"2023-10-06T10:43:48.349630+0000","last_unstale":"2023-10-06T10:43:48.349870+0000","last_undegraded":"2023-10-06T10:43:48.349870+0000","last_fullsized":"2023-10-06T10:43:48.349870+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:56.247192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"89'2","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.499397+0000","last_change":"2023-10-06T10:43:47.481379+0000","last_active":"2023-10-06T10:43:48.499397+0000","last_peered":"2023-10-06T10:43:48.499397+0000","last_clean":"2023-10-06T10:43:48.499397+0000","last_became_active":"2023-10-06T10:43:47.481120+0000","last_became_peered":"2023-10-06T10:43:47.481120+0000","last_unstale":"2023-10-06T10:43:48.499397+0000","last_undegraded":"2023-10-06T10:43:48.499397+0000","last_fullsized":"2023-10-06T10:43:48.499397+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:07.036958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.465691+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:43:47.465691+0000","last_peered":"2023-10-06T10:43:47.465691+0000","last_clean":"2023-10-06T10:43:47.465691+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:43:47.465691+0000","last_undegraded":"2023-10-06T10:43:47.465691+0000","last_fullsized":"2023-10-06T10:43:47.465691+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.358332+0000","last_change":"2023-10-06T10:43:48.358332+0000","last_active":"2023-10-06T10:43:48.358332+0000","last_peered":"2023-10-06T10:43:48.358332+0000","last_clean":"2023-10-06T10:43:48.358332+0000","last_became_active":"2023-10-06T10:43:48.357397+0000","last_became_peered":"2023-10-06T10:43:48.357397+0000","last_unstale":"2023-10-06T10:43:48.358332+0000","last_undegraded":"2023-10-06T10:43:48.358332+0000","last_fullsized":"2023-10-06T10:43:48.358332+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:55.836401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.481461+0000","last_change":"2023-10-06T10:43:47.478819+0000","last_active":"2023-10-06T10:43:47.481461+0000","last_peered":"2023-10-06T10:43:47.481461+0000","last_clean":"2023-10-06T10:43:47.481461+0000","last_became_active":"2023-10-06T10:43:47.478332+0000","last_became_peered":"2023-10-06T10:43:47.478332+0000","last_unstale":"2023-10-06T10:43:47.481461+0000","last_undegraded":"2023-10-06T10:43:47.481461+0000","last_fullsized":"2023-10-06T10:43:47.481461+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:35:08.093726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":73,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.466100+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:43:47.466100+0000","last_peered":"2023-10-06T10:43:47.466100+0000","last_clean":"2023-10-06T10:43:47.466100+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:43:47.466100+0000","last_undegraded":"2023-10-06T10:43:47.466100+0000","last_fullsized":"2023-10-06T10:43:47.466100+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":216,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.473481+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:43:47.473481+0000","last_peered":"2023-10-06T10:43:47.473481+0000","last_clean":"2023-10-06T10:43:47.473481+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:43:47.473481+0000","last_undegraded":"2023-10-06T10:43:47.473481+0000","last_fullsized":"2023-10-06T10:43:47.473481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.349957+0000","last_change":"2023-10-06T10:43:48.349957+0000","last_active":"2023-10-06T10:43:48.349957+0000","last_peered":"2023-10-06T10:43:48.349957+0000","last_clean":"2023-10-06T10:43:48.349957+0000","last_became_active":"2023-10-06T10:43:48.349677+0000","last_became_peered":"2023-10-06T10:43:48.349677+0000","last_unstale":"2023-10-06T10:43:48.349957+0000","last_undegraded":"2023-10-06T10:43:48.349957+0000","last_fullsized":"2023-10-06T10:43:48.349957+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:38:04.571912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"89'1","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.346954+0000","last_change":"2023-10-06T10:43:47.473629+0000","last_active":"2023-10-06T10:43:48.346954+0000","last_peered":"2023-10-06T10:43:48.346954+0000","last_clean":"2023-10-06T10:43:48.346954+0000","last_became_active":"2023-10-06T10:43:47.473135+0000","last_became_peered":"2023-10-06T10:43:47.473135+0000","last_unstale":"2023-10-06T10:43:48.346954+0000","last_undegraded":"2023-10-06T10:43:48.346954+0000","last_fullsized":"2023-10-06T10:43:48.346954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:13:20.299471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":66,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.473850+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:43:47.473850+0000","last_peered":"2023-10-06T10:43:47.473850+0000","last_clean":"2023-10-06T10:43:47.473850+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:43:47.473850+0000","last_undegraded":"2023-10-06T10:43:47.473850+0000","last_fullsized":"2023-10-06T10:43:47.473850+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.354545+0000","last_change":"2023-10-06T10:43:48.354545+0000","last_active":"2023-10-06T10:43:48.354545+0000","last_peered":"2023-10-06T10:43:48.354545+0000","last_clean":"2023-10-06T10:43:48.354545+0000","last_became_active":"2023-10-06T10:43:48.354132+0000","last_became_peered":"2023-10-06T10:43:48.354132+0000","last_unstale":"2023-10-06T10:43:48.354545+0000","last_undegraded":"2023-10-06T10:43:48.354545+0000","last_fullsized":"2023-10-06T10:43:48.354545+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:04.984298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"89'1","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.344071+0000","last_change":"2023-10-06T10:43:47.479218+0000","last_active":"2023-10-06T10:43:48.344071+0000","last_peered":"2023-10-06T10:43:48.344071+0000","last_clean":"2023-10-06T10:43:48.344071+0000","last_became_active":"2023-10-06T10:43:47.479015+0000","last_became_peered":"2023-10-06T10:43:47.479015+0000","last_unstale":"2023-10-06T10:43:48.344071+0000","last_undegraded":"2023-10-06T10:43:48.344071+0000","last_fullsized":"2023-10-06T10:43:48.344071+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:31:48.278032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.473386+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:43:47.473386+0000","last_peered":"2023-10-06T10:43:47.473386+0000","last_clean":"2023-10-06T10:43:47.473386+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:43:47.473386+0000","last_undegraded":"2023-10-06T10:43:47.473386+0000","last_fullsized":"2023-10-06T10:43:47.473386+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.357698+0000","last_change":"2023-10-06T10:43:48.357698+0000","last_active":"2023-10-06T10:43:48.357698+0000","last_peered":"2023-10-06T10:43:48.357698+0000","last_clean":"2023-10-06T10:43:48.357698+0000","last_became_active":"2023-10-06T10:43:48.357451+0000","last_became_peered":"2023-10-06T10:43:48.357451+0000","last_unstale":"2023-10-06T10:43:48.357698+0000","last_undegraded":"2023-10-06T10:43:48.357698+0000","last_fullsized":"2023-10-06T10:43:48.357698+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:31.895142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"89'2","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.495116+0000","last_change":"2023-10-06T10:43:47.472980+0000","last_active":"2023-10-06T10:43:48.495116+0000","last_peered":"2023-10-06T10:43:48.495116+0000","last_clean":"2023-10-06T10:43:48.495116+0000","last_became_active":"2023-10-06T10:43:47.472715+0000","last_became_peered":"2023-10-06T10:43:47.472715+0000","last_unstale":"2023-10-06T10:43:48.495116+0000","last_undegraded":"2023-10-06T10:43:48.495116+0000","last_fullsized":"2023-10-06T10:43:48.495116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:03.468957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.465031+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:43:47.465031+0000","last_peered":"2023-10-06T10:43:47.465031+0000","last_clean":"2023-10-06T10:43:47.465031+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:43:47.465031+0000","last_undegraded":"2023-10-06T10:43:47.465031+0000","last_fullsized":"2023-10-06T10:43:47.465031+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.347841+0000","last_change":"2023-10-06T10:43:48.347841+0000","last_active":"2023-10-06T10:43:48.347841+0000","last_peered":"2023-10-06T10:43:48.347841+0000","last_clean":"2023-10-06T10:43:48.347841+0000","last_became_active":"2023-10-06T10:43:48.347112+0000","last_became_peered":"2023-10-06T10:43:48.347112+0000","last_unstale":"2023-10-06T10:43:48.347841+0000","last_undegraded":"2023-10-06T10:43:48.347841+0000","last_fullsized":"2023-10-06T10:43:48.347841+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:06.744223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.482537+0000","last_change":"2023-10-06T10:43:47.477258+0000","last_active":"2023-10-06T10:43:47.482537+0000","last_peered":"2023-10-06T10:43:47.482537+0000","last_clean":"2023-10-06T10:43:47.482537+0000","last_became_active":"2023-10-06T10:43:47.476970+0000","last_became_peered":"2023-10-06T10:43:47.476970+0000","last_unstale":"2023-10-06T10:43:47.482537+0000","last_undegraded":"2023-10-06T10:43:47.482537+0000","last_fullsized":"2023-10-06T10:43:47.482537+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:51.923119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.466942+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:43:47.466942+0000","last_peered":"2023-10-06T10:43:47.466942+0000","last_clean":"2023-10-06T10:43:47.466942+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:43:47.466942+0000","last_undegraded":"2023-10-06T10:43:47.466942+0000","last_fullsized":"2023-10-06T10:43:47.466942+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T10:43:48.355316+0000","last_change":"2023-10-06T10:43:48.355316+0000","last_active":"2023-10-06T10:43:48.355316+0000","last_peered":"2023-10-06T10:43:48.355316+0000","last_clean":"2023-10-06T10:43:48.355316+0000","last_became_active":"2023-10-06T10:43:48.355011+0000","last_became_peered":"2023-10-06T10:43:48.355011+0000","last_unstale":"2023-10-06T10:43:48.355316+0000","last_undegraded":"2023-10-06T10:43:48.355316+0000","last_fullsized":"2023-10-06T10:43:48.355316+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:47.457082+0000","last_clean_scrub_stamp":"2023-10-06T10:43:47.457082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:09.942535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.483582+0000","last_change":"2023-10-06T10:43:47.481933+0000","last_active":"2023-10-06T10:43:47.483582+0000","last_peered":"2023-10-06T10:43:47.483582+0000","last_clean":"2023-10-06T10:43:47.483582+0000","last_became_active":"2023-10-06T10:43:47.481690+0000","last_became_peered":"2023-10-06T10:43:47.481690+0000","last_unstale":"2023-10-06T10:43:47.483582+0000","last_undegraded":"2023-10-06T10:43:47.483582+0000","last_fullsized":"2023-10-06T10:43:47.483582+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:43:46.453322+0000","last_clean_scrub_stamp":"2023-10-06T10:43:46.453322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:08.938775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T10:43:47.467138+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:43:47.467138+0000","last_peered":"2023-10-06T10:43:47.467138+0000","last_clean":"2023-10-06T10:43:47.467138+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:43:47.467138+0000","last_undegraded":"2023-10-06T10:43:47.467138+0000","last_fullsized":"2023-10-06T10:43:47.467138+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574921,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298948,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072892,"statfs":{"total":96636764160,"available":96330641408,"internally_reserved":0,"allocated":1699840,"data_stored":1118251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51600000000000001,"15min":0.48999999999999999},"min":{"1min":0.36299999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70599999999999996,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.51400000000000001,"15min":0.51300000000000001},"min":{"1min":0.41899999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73099999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.63}]},{"osd":1,"last update":"Fri Oct 6 10:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.49199999999999999,"15min":0.46600000000000003},"min":{"1min":0.318,"5min":0.252,"15min":0.252},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51800000000000002,"15min":0.504},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67500000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.53800000000000003}]},{"osd":2,"last update":"Fri Oct 6 10:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.54500000000000004,"15min":0.51500000000000001},"min":{"1min":0.434,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78600000000000003,"5min":1.24,"15min":1.24},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.51700000000000002,"15min":0.499},"min":{"1min":0.45100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.73199999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.52100000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54800000000000004,"15min":0.51500000000000001},"min":{"1min":0.41099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.746,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.52500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.76100000000000001,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.59499999999999997}]},{"osd":4,"last update":"Fri Oct 6 10:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.46500000000000002},"min":{"1min":0.33500000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64900000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.373},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48399999999999999,"15min":0.503},"min":{"1min":0.311,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81200000000000006,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.40100000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.47399999999999998,"15min":0.48399999999999999},"min":{"1min":0.35599999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.436},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52300000000000002,"15min":0.54500000000000004},"min":{"1min":0.33800000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.48799999999999999}]},{"osd":6,"last update":"Fri Oct 6 10:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54000000000000004,"15min":0.56799999999999995},"min":{"1min":0.40899999999999997,"5min":0.308,"15min":0.308},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.39700000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.82099999999999995,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.45100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574919,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298356,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073484,"statfs":{"total":96636764160,"available":96331247616,"internally_reserved":0,"allocated":1093632,"data_stored":526418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.503,"15min":0.49199999999999999},"min":{"1min":0.372,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56399999999999995,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.498,"15min":0.44700000000000001},"min":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69699999999999995,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.502}]},{"osd":1,"last update":"Fri Oct 6 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.499,"15min":0.45300000000000001},"min":{"1min":0.371,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66100000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52600000000000002,"15min":0.48299999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71199999999999997,"5min":0.751,"15min":0.751},"last":0.433}]},{"osd":2,"last update":"Fri Oct 6 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51300000000000001,"15min":0.47899999999999998},"min":{"1min":0.40500000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.66500000000000004,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52200000000000002,"15min":0.47899999999999998},"min":{"1min":0.36299999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.53000000000000003}]},{"osd":3,"last update":"Fri Oct 6 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.5},"min":{"1min":0.38500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53700000000000003,"15min":0.51300000000000001},"min":{"1min":0.39400000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66900000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.40699999999999997}]},{"osd":4,"last update":"Fri Oct 6 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47399999999999998,"15min":0.47599999999999998},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.375},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48699999999999999,"15min":0.48499999999999999},"min":{"1min":0.308,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69299999999999995,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.44700000000000001}]},{"osd":5,"last update":"Fri Oct 6 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.496,"15min":0.48799999999999999},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53100000000000003,"15min":0.53600000000000003},"min":{"1min":0.373,"5min":0.308,"15min":0.308},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.51500000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73499999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55800000000000005,"15min":0.54200000000000004},"min":{"1min":0.33000000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.56200000000000006}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298940,"kb_used_data":1652,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072900,"statfs":{"total":96636764160,"available":96330649600,"internally_reserved":0,"allocated":1691648,"data_stored":1118139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.219,"5min":0.188,"15min":0.188},"max":{"1min":0.48099999999999998,"5min":0.505,"15min":0.505},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.35899999999999999,"15min":0.34899999999999998},"min":{"1min":0.27900000000000003,"5min":0.189,"15min":0.189},"max":{"1min":0.56399999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.27200000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.33400000000000002,"15min":0.32200000000000001},"min":{"1min":0.23499999999999999,"5min":0.218,"15min":0.218},"max":{"1min":0.441,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.248},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.376,"15min":0.36499999999999999},"min":{"1min":0.26400000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.48799999999999999,"5min":0.62,"15min":0.62},"last":0.35299999999999998}]},{"osd":3,"last update":"Fri Oct 6 10:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39600000000000002,"15min":0.38},"min":{"1min":0.28199999999999997,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.56100000000000005,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.379,"15min":0.375},"min":{"1min":0.27000000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.50600000000000001,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.312}]},{"osd":4,"last update":"Fri Oct 6 10:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.40500000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77400000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56999999999999995,"15min":0.56000000000000005},"min":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71199999999999997,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.47199999999999998}]},{"osd":5,"last update":"Fri Oct 6 10:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.44600000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.749,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.54900000000000004},"min":{"1min":0.42699999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73199999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.48299999999999998}]},{"osd":6,"last update":"Fri Oct 6 10:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.46600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76600000000000001,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61199999999999999,"15min":0.60699999999999998},"min":{"1min":0.46300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73999999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.52000000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.42199999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.73599999999999999,"5min":0.996,"15min":0.996},"last":0.499},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.49299999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74399999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.434}]}]},{"osd":0,"up_from":13,"seq":55834574921,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298364,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073476,"statfs":{"total":96636764160,"available":96331239424,"internally_reserved":0,"allocated":1101824,"data_stored":528325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.33000000000000002,"15min":0.32000000000000001},"min":{"1min":0.21199999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.372,"15min":0.34799999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.432}]},{"osd":2,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.31900000000000001,"15min":0.308},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.35199999999999998,"15min":0.33100000000000002},"min":{"1min":0.251,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.40799999999999997}]},{"osd":3,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.35299999999999998,"15min":0.34399999999999997},"min":{"1min":0.245,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52100000000000002,"5min":0.622,"15min":0.622},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.36299999999999999,"15min":0.35199999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.38200000000000001}]},{"osd":4,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.40000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76300000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56200000000000006,"15min":0.54000000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.377,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85699999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.45600000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77500000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.66800000000000004}]},{"osd":6,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.41399999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80000000000000004,"5min":1.075,"15min":1.075},"last":0.76000000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57999999999999996,"15min":0.57899999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83499999999999996,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46200000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.77200000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.64300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574919,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298364,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073476,"statfs":{"total":96636764160,"available":96331239424,"internally_reserved":0,"allocated":1101824,"data_stored":527805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.314,"15min":0.311},"min":{"1min":0.20799999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.47299999999999998,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.34999999999999998,"15min":0.32500000000000001},"min":{"1min":0.20200000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.55600000000000005,"5min":0.622,"15min":0.622},"last":0.49299999999999999}]},{"osd":1,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.34100000000000003,"15min":0.32700000000000001},"min":{"1min":0.23000000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.56599999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.372,"15min":0.36599999999999999},"min":{"1min":0.28100000000000003,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.45100000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.374,"15min":0.35299999999999998},"min":{"1min":0.26100000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.374,"15min":0.35999999999999999},"min":{"1min":0.25,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.50600000000000001}]},{"osd":4,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.42599999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.38500000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82399999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56599999999999995}]},{"osd":5,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57599999999999996,"15min":0.57999999999999996},"min":{"1min":0.41299999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.83799999999999997,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.373,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81799999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.69599999999999995}]},{"osd":6,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60599999999999998,"15min":0.61799999999999999},"min":{"1min":0.44400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.88700000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.62,"15min":0.622},"min":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79700000000000004,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.71299999999999997}]},{"osd":7,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59299999999999997,"15min":0.60899999999999999},"min":{"1min":0.38600000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.42299999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81100000000000005,"5min":0.874,"15min":0.874},"last":0.66800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574919,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298376,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073464,"statfs":{"total":96636764160,"available":96331227136,"internally_reserved":0,"allocated":1114112,"data_stored":529329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34799999999999998,"15min":0.42599999999999999},"min":{"1min":0.20799999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.52700000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.433},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.35099999999999998,"15min":0.35899999999999999},"min":{"1min":0.22900000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.56899999999999995,"5min":1.218,"15min":1.218},"last":0.40500000000000003}]},{"osd":1,"last update":"Fri Oct 6 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.34200000000000003,"15min":0.34000000000000002},"min":{"1min":0.26100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.63600000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.39600000000000002,"15min":0.46600000000000003},"min":{"1min":0.28000000000000003,"5min":0.218,"15min":0.218},"max":{"1min":0.66000000000000003,"5min":3.359,"15min":3.359},"last":0.45900000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.372,"15min":0.38},"min":{"1min":0.26100000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.71099999999999997,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.44600000000000001,"15min":0.60999999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59899999999999998,"5min":7.069,"15min":7.069},"last":0.44700000000000001}]},{"osd":4,"last update":"Fri Oct 6 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.58499999999999996},"min":{"1min":0.39900000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84699999999999998,"5min":1.165,"15min":1.165},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60099999999999998,"15min":0.59799999999999998},"min":{"1min":0.41299999999999998,"5min":0.375,"15min":0.375},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.54500000000000004}]},{"osd":5,"last update":"Fri Oct 6 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.622,"15min":0.68799999999999994},"min":{"1min":0.39000000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84399999999999997,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.436,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80000000000000004,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.71599999999999997}]},{"osd":6,"last update":"Fri Oct 6 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.42699999999999999,"5min":0.378,"15min":0.378},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65400000000000003,"15min":0.71799999999999997},"min":{"1min":0.46999999999999997,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.876,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Oct 6 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.46000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83199999999999996,"5min":1.137,"15min":1.137},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.40999999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.97899999999999998,"5min":1.014,"15min":1.014},"last":0.67100000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298380,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073460,"statfs":{"total":96636764160,"available":96331223040,"internally_reserved":0,"allocated":1118208,"data_stored":529431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.48799999999999999},"min":{"1min":0.28799999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66600000000000004,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.378},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.34100000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73099999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.45400000000000001}]},{"osd":1,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.50900000000000001,"15min":0.501},"min":{"1min":0.36499999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68100000000000005,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.44},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52000000000000002,"15min":0.53500000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71799999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.48799999999999999}]},{"osd":2,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.377,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.60999999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53200000000000003,"15min":0.52100000000000002},"min":{"1min":0.376,"5min":0.315,"15min":0.315},"max":{"1min":0.75800000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.40999999999999998}]},{"osd":3,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54400000000000004,"15min":0.52200000000000002},"min":{"1min":0.39400000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.36599999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77500000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.47499999999999998}]},{"osd":5,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.45300000000000001,"15min":0.49399999999999999},"min":{"1min":0.29499999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67900000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.47599999999999998,"15min":0.496},"min":{"1min":0.315,"5min":0.255,"15min":0.255},"max":{"1min":0.625,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.35099999999999998}]},{"osd":6,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51100000000000001,"15min":0.53800000000000003},"min":{"1min":0.33400000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.77100000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54600000000000004,"15min":0.57199999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71799999999999997,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.53100000000000003}]},{"osd":7,"last update":"Fri Oct 6 10:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53400000000000003,"15min":0.56799999999999995},"min":{"1min":0.34599999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71399999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.55300000000000005,"15min":0.58899999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.745,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.505}]}]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298364,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073476,"statfs":{"total":96636764160,"available":96331239424,"internally_reserved":0,"allocated":1101824,"data_stored":527841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.504,"15min":0.47299999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52100000000000002,"15min":0.52800000000000002},"min":{"1min":0.33400000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.66900000000000004,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.44}]},{"osd":1,"last update":"Fri Oct 6 10:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.50700000000000001,"15min":0.48899999999999999},"min":{"1min":0.34599999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68899999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52800000000000002,"15min":0.51800000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.372}]},{"osd":2,"last update":"Fri Oct 6 10:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.50700000000000001},"min":{"1min":0.31900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77600000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.40699999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.40400000000000003}]},{"osd":3,"last update":"Fri Oct 6 10:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55900000000000005,"15min":0.57299999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79900000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53200000000000003,"15min":0.55300000000000005},"min":{"1min":0.35099999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.79000000000000004,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.33200000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.47699999999999998,"15min":0.48599999999999999},"min":{"1min":0.318,"5min":0.317,"15min":0.317},"max":{"1min":0.70099999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.313},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.503,"15min":0.50700000000000001},"min":{"1min":0.35999999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.68100000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.35199999999999998}]},{"osd":6,"last update":"Fri Oct 6 10:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.503,"15min":0.51800000000000002},"min":{"1min":0.30499999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.46400000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55900000000000005,"15min":0.57799999999999996},"min":{"1min":0.38200000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73899999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.498},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.42499999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76200000000000001,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.41699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:43:52.905 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T10:43:52.906 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T10:43:53.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.058+0000 7fec56b57700 1 Processor -- start 2023-10-06T10:43:53.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.058+0000 7fec56b57700 1 -- start start 2023-10-06T10:43:53.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.059+0000 7fec56b57700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fec5006ce80 0x7fec5015b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:53.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.059+0000 7fec56b57700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fec500c88e0 0x7fec500cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:53.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.059+0000 7fec56b57700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec500cb430 con 0x7fec5006d250 2023-10-06T10:43:53.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.059+0000 7fec56b57700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec5015be00 con 0x7fec500c88e0 2023-10-06T10:43:53.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.059+0000 7fec56b57700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec500ca5c0 con 0x7fec5006ce80 2023-10-06T10:43:53.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.059+0000 7fec4ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fec500c88e0 0x7fec500cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:53.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.059+0000 7fec550f4700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fec5006d250 0x7fec500cc3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37610/0 (socket says 172.21.15.92:37610) 2023-10-06T10:43:53.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.059+0000 7fec550f4700 1 -- 172.21.15.92:0/826166851 learned_addr learned my addr 172.21.15.92:0/826166851 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:53.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.060+0000 7fec548f3700 1 --2- 172.21.15.92:0/826166851 >> v2:172.21.15.124:3300/0 conn(0x7fec5006ce80 0x7fec5015b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:53.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.060+0000 7fec4ffff700 1 -- 172.21.15.92:0/826166851 >> v2:172.21.15.124:3300/0 conn(0x7fec5006ce80 msgr2=0x7fec5015b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:53.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.060+0000 7fec4ffff700 1 --2- 172.21.15.92:0/826166851 >> v2:172.21.15.124:3300/0 conn(0x7fec5006ce80 0x7fec5015b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:53.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.060+0000 7fec4ffff700 1 -- 172.21.15.92:0/826166851 >> v1:172.21.15.92:6789/0 conn(0x7fec5006d250 legacy=0x7fec500cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:53.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.060+0000 7fec4f7fe700 1 -- 172.21.15.92:0/826166851 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593264511 0 0) 0x7fec500cb430 con 0x7fec5006d250 2023-10-06T10:43:53.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.060+0000 7fec4ffff700 1 -- 172.21.15.92:0/826166851 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec500ca820 con 0x7fec500c88e0 2023-10-06T10:43:53.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.060+0000 7fec4ffff700 1 --2- 172.21.15.92:0/826166851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec500c88e0 0x7fec500cbe70 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fec5006e990 tx=0x7fec3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c8791a75ee7bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:53.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.060+0000 7fec4f7fe700 1 -- 172.21.15.92:0/826166851 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c00b6b0 con 0x7fec500c88e0 2023-10-06T10:43:53.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.060+0000 7fec4f7fe700 1 -- 172.21.15.92:0/826166851 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c00b810 con 0x7fec500c88e0 2023-10-06T10:43:53.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.060+0000 7fec4f7fe700 1 -- 172.21.15.92:0/826166851 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c00bbb0 con 0x7fec500c88e0 2023-10-06T10:43:53.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.060+0000 7fec56b57700 1 -- 172.21.15.92:0/826166851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec500c88e0 msgr2=0x7fec500cbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:53.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.060+0000 7fec56b57700 1 --2- 172.21.15.92:0/826166851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec500c88e0 0x7fec500cbe70 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fec5006e990 tx=0x7fec3c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:53.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.060+0000 7fec56b57700 1 -- 172.21.15.92:0/826166851 shutdown_connections 2023-10-06T10:43:53.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.060+0000 7fec56b57700 1 --2- 172.21.15.92:0/826166851 >> v2:172.21.15.124:3300/0 conn(0x7fec5006ce80 0x7fec5015b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:53.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.060+0000 7fec56b57700 1 --2- 172.21.15.92:0/826166851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec500c88e0 0x7fec500cbe70 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:53.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.060+0000 7fec56b57700 1 -- 172.21.15.92:0/826166851 >> 172.21.15.92:0/826166851 conn(0x7fec5006b610 msgr2=0x7fec5006ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:53.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.060+0000 7fec56b57700 1 -- 172.21.15.92:0/826166851 shutdown_connections 2023-10-06T10:43:53.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.061+0000 7fec56b57700 1 -- 172.21.15.92:0/826166851 wait complete. 2023-10-06T10:43:53.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.061+0000 7fec56b57700 1 Processor -- start 2023-10-06T10:43:53.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.062+0000 7fec56b57700 1 -- start start 2023-10-06T10:43:53.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.062+0000 7fec56b57700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec5006ce80 0x7fec500ace50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:53.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.062+0000 7fec56b57700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec5006d250 0x7fec500b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:53.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.062+0000 7fec56b57700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec500c88e0 0x7fec500adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:53.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.062+0000 7fec56b57700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec500687f0 con 0x7fec500c88e0 2023-10-06T10:43:53.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.062+0000 7fec56b57700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec50159890 con 0x7fec5006d250 2023-10-06T10:43:53.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.062+0000 7fec56b57700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec50159b70 con 0x7fec5006ce80 2023-10-06T10:43:53.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.063+0000 7fec550f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec500c88e0 0x7fec500adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:53.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.063+0000 7fec550f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec500c88e0 0x7fec500adde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45052/0 (socket says 172.21.15.92:45052) 2023-10-06T10:43:53.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.063+0000 7fec550f4700 1 -- 172.21.15.92:0/3895195652 learned_addr learned my addr 172.21.15.92:0/3895195652 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:53.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.063+0000 7fec548f3700 1 --2- 172.21.15.92:0/3895195652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec5006ce80 0x7fec500ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:53.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.063+0000 7fec4ffff700 1 --2- 172.21.15.92:0/3895195652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec5006d250 0x7fec500b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:53.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.063+0000 7fec550f4700 1 -- 172.21.15.92:0/3895195652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec5006ce80 msgr2=0x7fec500ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:53.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.063+0000 7fec550f4700 1 --2- 172.21.15.92:0/3895195652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec5006ce80 0x7fec500ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:53.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.063+0000 7fec550f4700 1 -- 172.21.15.92:0/3895195652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec5006d250 msgr2=0x7fec500b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:53.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.063+0000 7fec550f4700 1 --2- 172.21.15.92:0/3895195652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec5006d250 0x7fec500b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:53.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.063+0000 7fec550f4700 1 -- 172.21.15.92:0/3895195652 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c009420 con 0x7fec500c88e0 2023-10-06T10:43:53.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.063+0000 7fec550f4700 1 --2- 172.21.15.92:0/3895195652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec500c88e0 0x7fec500adde0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fec4800cc90 tx=0x7fec48001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:53.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.063+0000 7fec4cff9700 1 -- 172.21.15.92:0/3895195652 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4800a8b0 con 0x7fec500c88e0 2023-10-06T10:43:53.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.063+0000 7fec4cff9700 1 -- 172.21.15.92:0/3895195652 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4800aa10 con 0x7fec500c88e0 2023-10-06T10:43:53.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.063+0000 7fec4cff9700 1 -- 172.21.15.92:0/3895195652 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4800add0 con 0x7fec500c88e0 2023-10-06T10:43:53.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.063+0000 7fec56b57700 1 -- 172.21.15.92:0/3895195652 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec50159e50 con 0x7fec500c88e0 2023-10-06T10:43:53.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.063+0000 7fec56b57700 1 -- 172.21.15.92:0/3895195652 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec500ad390 con 0x7fec500c88e0 2023-10-06T10:43:53.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.064+0000 7fec56b57700 1 -- 172.21.15.92:0/3895195652 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec50064a90 con 0x7fec500c88e0 2023-10-06T10:43:53.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.064+0000 7fec4cff9700 1 -- 172.21.15.92:0/3895195652 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec48007300 con 0x7fec500c88e0 2023-10-06T10:43:53.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.065+0000 7fec4cff9700 1 --2- 172.21.15.92:0/3895195652 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fec34066f30 0x7fec340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:53.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.065+0000 7fec4cff9700 1 -- 172.21.15.92:0/3895195652 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fec48084af0 con 0x7fec500c88e0 2023-10-06T10:43:53.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.065+0000 7fec548f3700 1 --2- 172.21.15.92:0/3895195652 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fec34066f30 0x7fec340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:53.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.065+0000 7fec548f3700 1 --2- 172.21.15.92:0/3895195652 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fec34066f30 0x7fec340693e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fec44006fd0 tx=0x7fec44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:53.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.069+0000 7fec4cff9700 1 -- 172.21.15.92:0/3895195652 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec48051b40 con 0x7fec500c88e0 2023-10-06T10:43:53.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.323+0000 7fec56b57700 1 -- 172.21.15.92:0/3895195652 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fec500cab00 con 0x7fec500c88e0 2023-10-06T10:43:53.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.327+0000 7fec4cff9700 1 -- 172.21.15.92:0/3895195652 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7fec48055160 con 0x7fec500c88e0 2023-10-06T10:43:53.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.329+0000 7fec56b57700 1 -- 172.21.15.92:0/3895195652 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fec34066f30 msgr2=0x7fec340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:53.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.329+0000 7fec56b57700 1 --2- 172.21.15.92:0/3895195652 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fec34066f30 0x7fec340693e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fec44006fd0 tx=0x7fec44008040 comp rx=0 tx=0).stop 2023-10-06T10:43:53.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.329+0000 7fec56b57700 1 -- 172.21.15.92:0/3895195652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec500c88e0 msgr2=0x7fec500adde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:53.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.329+0000 7fec56b57700 1 --2- 172.21.15.92:0/3895195652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec500c88e0 0x7fec500adde0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fec4800cc90 tx=0x7fec48001a80 comp rx=0 tx=0).stop 2023-10-06T10:43:53.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.329+0000 7fec56b57700 1 -- 172.21.15.92:0/3895195652 shutdown_connections 2023-10-06T10:43:53.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.329+0000 7fec56b57700 1 --2- 172.21.15.92:0/3895195652 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fec34066f30 0x7fec340693e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:53.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.329+0000 7fec56b57700 1 --2- 172.21.15.92:0/3895195652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec500c88e0 0x7fec500adde0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:53.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.329+0000 7fec56b57700 1 --2- 172.21.15.92:0/3895195652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec5006ce80 0x7fec500ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:53.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.329+0000 7fec56b57700 1 --2- 172.21.15.92:0/3895195652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec5006d250 0x7fec500b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:53.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.329+0000 7fec56b57700 1 -- 172.21.15.92:0/3895195652 >> 172.21.15.92:0/3895195652 conn(0x7fec5006b610 msgr2=0x7fec50155e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:53.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.329+0000 7fec56b57700 1 -- 172.21.15.92:0/3895195652 shutdown_connections 2023-10-06T10:43:53.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.329+0000 7fec56b57700 1 -- 172.21.15.92:0/3895195652 wait complete. 2023-10-06T10:43:53.331 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-10-06T10:43:53.342 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:43:53.357 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:43:53.388 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:43:53.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.459+0000 7f0b52b00700 1 Processor -- start 2023-10-06T10:43:53.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.459+0000 7f0b52b00700 1 -- start start 2023-10-06T10:43:53.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.460+0000 7f0b52b00700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0b4c159560 0x7f0b4c0aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:53.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.460+0000 7f0b52b00700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b4c0b0760 0x7f0b4c0b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:53.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.460+0000 7f0b52b00700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b4c0b0c60 con 0x7f0b4c0b49a0 2023-10-06T10:43:53.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.460+0000 7f0b52b00700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b4c0b0f40 con 0x7f0b4c159560 2023-10-06T10:43:53.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.460+0000 7f0b52b00700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0b1220 con 0x7f0b4c0b0760 2023-10-06T10:43:53.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.460+0000 7f0b5089c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0b4c159560 0x7f0b4c0aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:53.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.460+0000 7f0b5089c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0b4c159560 0x7f0b4c0aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36184/0 (socket says 172.21.15.92:36184) 2023-10-06T10:43:53.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.460+0000 7f0b5089c700 1 -- 172.21.15.92:0/4039492441 learned_addr learned my addr 172.21.15.92:0/4039492441 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:53.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.460+0000 7f0b5089c700 1 -- 172.21.15.92:0/4039492441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b4c0b0760 msgr2=0x7f0b4c0b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:53.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.460+0000 7f0b5109d700 1 --2- 172.21.15.92:0/4039492441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b4c0b0760 0x7f0b4c0b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:53.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.460+0000 7f0b5089c700 1 --2- 172.21.15.92:0/4039492441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b4c0b0760 0x7f0b4c0b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:53.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.460+0000 7f0b5089c700 1 -- 172.21.15.92:0/4039492441 >> v1:172.21.15.124:6789/0 conn(0x7f0b4c0b49a0 legacy=0x7f0b4c0aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:53.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.460+0000 7f0b5089c700 1 -- 172.21.15.92:0/4039492441 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4c0b1500 con 0x7f0b4c159560 2023-10-06T10:43:53.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.460+0000 7f0b5089c700 1 --2- 172.21.15.92:0/4039492441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b4c159560 0x7f0b4c0aa530 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f0b400099c0 tx=0x7f0b40009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0c314baec3a58f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:53.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.460+0000 7f0b4b7fe700 1 -- 172.21.15.92:0/4039492441 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40005c10 con 0x7f0b4c159560 2023-10-06T10:43:53.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.461+0000 7f0b4b7fe700 1 -- 172.21.15.92:0/4039492441 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b40005d70 con 0x7f0b4c159560 2023-10-06T10:43:53.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.461+0000 7f0b4b7fe700 1 -- 172.21.15.92:0/4039492441 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4000b800 con 0x7f0b4c159560 2023-10-06T10:43:53.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.461+0000 7f0b52b00700 1 -- 172.21.15.92:0/4039492441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b4c159560 msgr2=0x7f0b4c0aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:53.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.461+0000 7f0b52b00700 1 --2- 172.21.15.92:0/4039492441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b4c159560 0x7f0b4c0aa530 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f0b400099c0 tx=0x7f0b40009220 comp rx=0 tx=0).stop 2023-10-06T10:43:53.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.461+0000 7f0b52b00700 1 -- 172.21.15.92:0/4039492441 shutdown_connections 2023-10-06T10:43:53.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.461+0000 7f0b52b00700 1 --2- 172.21.15.92:0/4039492441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b4c0b0760 0x7f0b4c0b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:53.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.461+0000 7f0b52b00700 1 --2- 172.21.15.92:0/4039492441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b4c159560 0x7f0b4c0aa530 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:53.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.461+0000 7f0b52b00700 1 -- 172.21.15.92:0/4039492441 >> 172.21.15.92:0/4039492441 conn(0x7f0b4c0997f0 msgr2=0x7f0b4c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:53.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.461+0000 7f0b52b00700 1 -- 172.21.15.92:0/4039492441 shutdown_connections 2023-10-06T10:43:53.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.461+0000 7f0b52b00700 1 -- 172.21.15.92:0/4039492441 wait complete. 2023-10-06T10:43:53.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.462+0000 7f0b52b00700 1 Processor -- start 2023-10-06T10:43:53.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.462+0000 7f0b52b00700 1 -- start start 2023-10-06T10:43:53.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.463+0000 7f0b52b00700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b4c0b0760 0x7f0b4c0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:53.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.463+0000 7f0b52b00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b4c0b49a0 0x7f0b4c16ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:53.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.463+0000 7f0b52b00700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b4c159560 0x7f0b4c157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:53.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.463+0000 7f0b52b00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0adc40 con 0x7f0b4c0b49a0 2023-10-06T10:43:53.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.463+0000 7f0b52b00700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c157fe0 con 0x7f0b4c0b0760 2023-10-06T10:43:53.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.463+0000 7f0b52b00700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0abc10 con 0x7f0b4c159560 2023-10-06T10:43:53.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.463+0000 7f0b4bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b4c0b49a0 0x7f0b4c16ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:53.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.463+0000 7f0b4bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b4c0b49a0 0x7f0b4c16ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36196/0 (socket says 172.21.15.92:36196) 2023-10-06T10:43:53.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.463+0000 7f0b4bfff700 1 -- 172.21.15.92:0/2593818842 learned_addr learned my addr 172.21.15.92:0/2593818842 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:53.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.463+0000 7f0b5089c700 1 --2- 172.21.15.92:0/2593818842 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b4c0b0760 0x7f0b4c0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:53.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.463+0000 7f0b5109d700 1 --2- 172.21.15.92:0/2593818842 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b4c159560 0x7f0b4c157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:53.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.463+0000 7f0b4bfff700 1 -- 172.21.15.92:0/2593818842 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b4c159560 msgr2=0x7f0b4c157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:53.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.463+0000 7f0b4bfff700 1 --2- 172.21.15.92:0/2593818842 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b4c159560 0x7f0b4c157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:53.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.463+0000 7f0b4bfff700 1 -- 172.21.15.92:0/2593818842 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b4c0b0760 msgr2=0x7f0b4c0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:53.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.463+0000 7f0b4bfff700 1 --2- 172.21.15.92:0/2593818842 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b4c0b0760 0x7f0b4c0af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:53.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.464+0000 7f0b4bfff700 1 -- 172.21.15.92:0/2593818842 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b40009480 con 0x7f0b4c0b49a0 2023-10-06T10:43:53.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.464+0000 7f0b4bfff700 1 --2- 172.21.15.92:0/2593818842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b4c0b49a0 0x7f0b4c16ab30 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f0b38002ee0 tx=0x7f0b38007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:53.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.464+0000 7f0b48ff9700 1 -- 172.21.15.92:0/2593818842 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b38016070 con 0x7f0b4c0b49a0 2023-10-06T10:43:53.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.464+0000 7f0b48ff9700 1 -- 172.21.15.92:0/2593818842 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b38006820 con 0x7f0b4c0b49a0 2023-10-06T10:43:53.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.464+0000 7f0b48ff9700 1 -- 172.21.15.92:0/2593818842 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b38006bc0 con 0x7f0b4c0b49a0 2023-10-06T10:43:53.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.464+0000 7f0b52b00700 1 -- 172.21.15.92:0/2593818842 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b4c0abef0 con 0x7f0b4c0b49a0 2023-10-06T10:43:53.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.464+0000 7f0b52b00700 1 -- 172.21.15.92:0/2593818842 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b4c0ad810 con 0x7f0b4c0b49a0 2023-10-06T10:43:53.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.465+0000 7f0b52b00700 1 -- 172.21.15.92:0/2593818842 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b4c0ac390 con 0x7f0b4c0b49a0 2023-10-06T10:43:53.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.465+0000 7f0b48ff9700 1 -- 172.21.15.92:0/2593818842 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0b38007020 con 0x7f0b4c0b49a0 2023-10-06T10:43:53.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.466+0000 7f0b48ff9700 1 --2- 172.21.15.92:0/2593818842 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0b30066f30 0x7f0b300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:53.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.466+0000 7f0b48ff9700 1 -- 172.21.15.92:0/2593818842 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0b38083f40 con 0x7f0b4c0b49a0 2023-10-06T10:43:53.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.466+0000 7f0b5089c700 1 --2- 172.21.15.92:0/2593818842 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0b30066f30 0x7f0b300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:53.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.466+0000 7f0b5089c700 1 --2- 172.21.15.92:0/2593818842 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0b30066f30 0x7f0b300693e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0b40007fd0 tx=0x7f0b40015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:53.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.470+0000 7f0b48ff9700 1 -- 172.21.15.92:0/2593818842 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b38050f10 con 0x7f0b4c0b49a0 2023-10-06T10:43:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.698+0000 7f0b52b00700 1 -- 172.21.15.92:0/2593818842 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b4c064a90 con 0x7f0b4c0b49a0 2023-10-06T10:43:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.699+0000 7f0b48ff9700 1 -- 172.21.15.92:0/2593818842 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f0b38054530 con 0x7f0b4c0b49a0 2023-10-06T10:43:53.700 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:53.700 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":83,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2122441982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2122441982},{"type":"v1","addr":"172.21.15.124:6835","nonce":2122441982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78},{"gid":7686,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3605297930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3605297930},{"type":"v1","addr":"172.21.15.92:6836","nonce":3605297930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78},{"gid":7689,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/4047304116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4047304116},{"type":"v1","addr":"172.21.15.92:6837","nonce":4047304116}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:47.850832+0000","modified":"2023-10-06T10:43:51.497396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7701":{"gid":7701,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6834/4203118992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":4203118992},{"type":"v1","addr":"172.21.15.124:6834","nonce":4203118992}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T10:43:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.701+0000 7f0b52b00700 1 -- 172.21.15.92:0/2593818842 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0b30066f30 msgr2=0x7f0b300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.701+0000 7f0b52b00700 1 --2- 172.21.15.92:0/2593818842 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0b30066f30 0x7f0b300693e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0b40007fd0 tx=0x7f0b40015000 comp rx=0 tx=0).stop 2023-10-06T10:43:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.701+0000 7f0b52b00700 1 -- 172.21.15.92:0/2593818842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b4c0b49a0 msgr2=0x7f0b4c16ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.701+0000 7f0b52b00700 1 --2- 172.21.15.92:0/2593818842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b4c0b49a0 0x7f0b4c16ab30 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f0b38002ee0 tx=0x7f0b38007f20 comp rx=0 tx=0).stop 2023-10-06T10:43:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.701+0000 7f0b52b00700 1 -- 172.21.15.92:0/2593818842 shutdown_connections 2023-10-06T10:43:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.701+0000 7f0b52b00700 1 --2- 172.21.15.92:0/2593818842 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0b30066f30 0x7f0b300693e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.701+0000 7f0b52b00700 1 --2- 172.21.15.92:0/2593818842 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b4c159560 0x7f0b4c157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.701+0000 7f0b52b00700 1 --2- 172.21.15.92:0/2593818842 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b4c0b0760 0x7f0b4c0af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.701+0000 7f0b52b00700 1 --2- 172.21.15.92:0/2593818842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b4c0b49a0 0x7f0b4c16ab30 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.701+0000 7f0b52b00700 1 -- 172.21.15.92:0/2593818842 >> 172.21.15.92:0/2593818842 conn(0x7f0b4c0997f0 msgr2=0x7f0b4c158960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.701+0000 7f0b52b00700 1 -- 172.21.15.92:0/2593818842 shutdown_connections 2023-10-06T10:43:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:53.701+0000 7f0b52b00700 1 -- 172.21.15.92:0/2593818842 wait complete. 2023-10-06T10:43:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 83 2023-10-06T10:43:54.116 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T10:43:47.850832+0000', 'modified': '2023-10-06T10:43:51.497396+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7701}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7701': {'gid': 7701, 'name': 'b', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.124:6834/4203118992', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6832', 'nonce': 4203118992}, {'type': 'v1', 'addr': '172.21.15.124:6834', 'nonce': 4203118992}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T10:43:54.116 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T10:43:54.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T10:43:54.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.234+0000 7ff122996700 1 Processor -- start 2023-10-06T10:43:54.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.234+0000 7ff122996700 1 -- start start 2023-10-06T10:43:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.235+0000 7ff122996700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff11c0cbc30 0x7ff11c0cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.235+0000 7ff122996700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff11c0cb7a0 con 0x7ff11c159440 2023-10-06T10:43:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.235+0000 7ff122996700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff11c0b0860 con 0x7ff11c0b2100 2023-10-06T10:43:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.235+0000 7ff122996700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c0cb150 con 0x7ff11c0cbc30 2023-10-06T10:43:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.235+0000 7ff121994700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff11c159440 0x7ff11c0b3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57250/0 (socket says 172.21.15.92:57250) 2023-10-06T10:43:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.235+0000 7ff121994700 1 -- 172.21.15.92:0/4285727516 learned_addr learned my addr 172.21.15.92:0/4285727516 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.235+0000 7ff120992700 1 -- 172.21.15.92:0/4285727516 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3921100742 0 0) 0x7ff11c0cb7a0 con 0x7ff11c159440 2023-10-06T10:43:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.235+0000 7ff120992700 1 -- 172.21.15.92:0/4285727516 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff108003570 con 0x7ff11c159440 2023-10-06T10:43:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.235+0000 7ff121193700 1 --2- 172.21.15.92:0/4285727516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff11c0cbc30 0x7ff11c0cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff120992700 1 -- 172.21.15.92:0/4285727516 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3384111922 0 0) 0x7ff108003570 con 0x7ff11c159440 2023-10-06T10:43:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff120992700 1 -- 172.21.15.92:0/4285727516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff11c0cbc30 msgr2=0x7ff11c0cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:54.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff120992700 1 --2- 172.21.15.92:0/4285727516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff11c0cbc30 0x7ff11c0cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff120992700 1 -- 172.21.15.92:0/4285727516 >> v1:172.21.15.124:6789/0 conn(0x7ff11c0b2100 legacy=0x7ff11c0b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:54.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff120992700 1 -- 172.21.15.92:0/4285727516 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11c0cb430 con 0x7ff11c159440 2023-10-06T10:43:54.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff120992700 1 -- 172.21.15.92:0/4285727516 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7ff118001470 con 0x7ff11c159440 2023-10-06T10:43:54.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff120992700 1 -- 172.21.15.92:0/4285727516 >> v1:172.21.15.92:6789/0 conn(0x7ff11c159440 legacy=0x7ff11c0b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:54.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff120992700 1 --2- 172.21.15.92:0/4285727516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff108003a90 0x7ff108003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff120992700 1 --2- 172.21.15.92:0/4285727516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff108004810 0x7ff108004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff120992700 1 --2- 172.21.15.92:0/4285727516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff11c0cbc30 0x7ff1080051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff120992700 1 -- 172.21.15.92:0/4285727516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c0cb150 con 0x7ff108004810 2023-10-06T10:43:54.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff120992700 1 -- 172.21.15.92:0/4285727516 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10800bb60 con 0x7ff108003a90 2023-10-06T10:43:54.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff120992700 1 -- 172.21.15.92:0/4285727516 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff10800bd20 con 0x7ff11c0cbc30 2023-10-06T10:43:54.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff121193700 1 --2- 172.21.15.92:0/4285727516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff108004810 0x7ff108004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff121994700 1 --2- 172.21.15.92:0/4285727516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff108003a90 0x7ff108003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff122195700 1 --2- 172.21.15.92:0/4285727516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff11c0cbc30 0x7ff1080051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff121193700 1 -- 172.21.15.92:0/4285727516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff11c0cbc30 msgr2=0x7ff1080051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:54.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff121193700 1 --2- 172.21.15.92:0/4285727516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff11c0cbc30 0x7ff1080051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff121193700 1 -- 172.21.15.92:0/4285727516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff108003a90 msgr2=0x7ff108003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:54.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff121193700 1 --2- 172.21.15.92:0/4285727516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff108003a90 0x7ff108003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff121193700 1 -- 172.21.15.92:0/4285727516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff108003570 con 0x7ff108004810 2023-10-06T10:43:54.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff121994700 1 --2- 172.21.15.92:0/4285727516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff108003a90 0x7ff108003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:54.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff121193700 1 --2- 172.21.15.92:0/4285727516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff108004810 0x7ff108004c80 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009c00 tx=0x7ff10c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c10adcee5c3ebbf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:54.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff120992700 1 -- 172.21.15.92:0/4285727516 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c00e710 con 0x7ff108004810 2023-10-06T10:43:54.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff120992700 1 -- 172.21.15.92:0/4285727516 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10c00e870 con 0x7ff108004810 2023-10-06T10:43:54.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.236+0000 7ff120992700 1 -- 172.21.15.92:0/4285727516 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c00ec30 con 0x7ff108004810 2023-10-06T10:43:54.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.237+0000 7ff122996700 1 -- 172.21.15.92:0/4285727516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff108004810 msgr2=0x7ff108004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:54.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.237+0000 7ff122996700 1 --2- 172.21.15.92:0/4285727516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff108004810 0x7ff108004c80 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009c00 tx=0x7ff10c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.237+0000 7ff122195700 1 -- 172.21.15.92:0/4285727516 reap_dead start 2023-10-06T10:43:54.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.237+0000 7ff122996700 1 -- 172.21.15.92:0/4285727516 shutdown_connections 2023-10-06T10:43:54.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.237+0000 7ff122996700 1 -- 172.21.15.92:0/4285727516 >> 172.21.15.92:0/4285727516 conn(0x7ff11c06ece0 msgr2=0x7ff11c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:54.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.237+0000 7ff122996700 1 -- 172.21.15.92:0/4285727516 shutdown_connections 2023-10-06T10:43:54.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.237+0000 7ff122996700 1 -- 172.21.15.92:0/4285727516 wait complete. 2023-10-06T10:43:54.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.238+0000 7ff122996700 1 Processor -- start 2023-10-06T10:43:54.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.238+0000 7ff122996700 1 -- start start 2023-10-06T10:43:54.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.239+0000 7ff122996700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff108004810 0x7ff11c15a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.239+0000 7ff122996700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff11c15c230 0x7ff11c15c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.239+0000 7ff122996700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff11c15cf50 0x7ff11c15d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.239+0000 7ff122996700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c0b05a0 con 0x7ff11c15c230 2023-10-06T10:43:54.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.239+0000 7ff122996700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c16a390 con 0x7ff108004810 2023-10-06T10:43:54.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.239+0000 7ff122996700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c169780 con 0x7ff11c15cf50 2023-10-06T10:43:54.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.239+0000 7ff121193700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff11c15c230 0x7ff11c15c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.239+0000 7ff121193700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff11c15c230 0x7ff11c15c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36202/0 (socket says 172.21.15.92:36202) 2023-10-06T10:43:54.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.239+0000 7ff121193700 1 -- 172.21.15.92:0/1170116887 learned_addr learned my addr 172.21.15.92:0/1170116887 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:54.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.239+0000 7ff121994700 1 --2- 172.21.15.92:0/1170116887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff108004810 0x7ff11c15a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.239+0000 7ff122195700 1 --2- 172.21.15.92:0/1170116887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff11c15cf50 0x7ff11c15d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.239+0000 7ff121193700 1 -- 172.21.15.92:0/1170116887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff11c15cf50 msgr2=0x7ff11c15d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:54.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.239+0000 7ff121193700 1 --2- 172.21.15.92:0/1170116887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff11c15cf50 0x7ff11c15d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.239+0000 7ff121193700 1 -- 172.21.15.92:0/1170116887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff108004810 msgr2=0x7ff11c15a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:54.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.239+0000 7ff121193700 1 --2- 172.21.15.92:0/1170116887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff108004810 0x7ff11c15a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.239+0000 7ff121193700 1 -- 172.21.15.92:0/1170116887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11c169a60 con 0x7ff11c15c230 2023-10-06T10:43:54.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.239+0000 7ff121193700 1 --2- 172.21.15.92:0/1170116887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff11c15c230 0x7ff11c15c6a0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7ff10800bb30 tx=0x7ff10c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:54.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.239+0000 7ff111ffb700 1 -- 172.21.15.92:0/1170116887 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c022070 con 0x7ff11c15c230 2023-10-06T10:43:54.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.240+0000 7ff111ffb700 1 -- 172.21.15.92:0/1170116887 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10c01c450 con 0x7ff11c15c230 2023-10-06T10:43:54.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.240+0000 7ff122996700 1 -- 172.21.15.92:0/1170116887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff11c169d40 con 0x7ff11c15c230 2023-10-06T10:43:54.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.240+0000 7ff111ffb700 1 -- 172.21.15.92:0/1170116887 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c01c830 con 0x7ff11c15c230 2023-10-06T10:43:54.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.240+0000 7ff122996700 1 -- 172.21.15.92:0/1170116887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11c15cbe0 con 0x7ff11c15c230 2023-10-06T10:43:54.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.241+0000 7ff111ffb700 1 -- 172.21.15.92:0/1170116887 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff10c01cc20 con 0x7ff11c15c230 2023-10-06T10:43:54.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.241+0000 7ff111ffb700 1 --2- 172.21.15.92:0/1170116887 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff100066f30 0x7ff1000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.242+0000 7ff111ffb700 1 -- 172.21.15.92:0/1170116887 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff10c0863f0 con 0x7ff11c15c230 2023-10-06T10:43:54.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.242+0000 7ff121994700 1 --2- 172.21.15.92:0/1170116887 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff100066f30 0x7ff1000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.242+0000 7ff121994700 1 --2- 172.21.15.92:0/1170116887 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff100066f30 0x7ff1000693e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff1180015a0 tx=0x7ff118000ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:54.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.243+0000 7ff12cef0b80 1 Processor -- start 2023-10-06T10:43:54.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.243+0000 7ff12cef0b80 1 -- start start 2023-10-06T10:43:54.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.244+0000 7ff12cef0b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561298ae0950 0x561298ae0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.244+0000 7ff12cef0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298b2abc0 0x561298b09920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.244+0000 7ff12cef0b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561298b09e60 0x561298adc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.244+0000 7ff12cef0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561298a60c80 con 0x561298b2abc0 2023-10-06T10:43:54.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.244+0000 7ff12cef0b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561298a37a70 con 0x561298ae0950 2023-10-06T10:43:54.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.244+0000 7ff12cef0b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561298a35270 con 0x561298b09e60 2023-10-06T10:43:54.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.244+0000 7ff121193700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298b2abc0 0x561298b09920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.244+0000 7ff121193700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298b2abc0 0x561298b09920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36208/0 (socket says 172.21.15.92:36208) 2023-10-06T10:43:54.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.244+0000 7ff121193700 1 -- 172.21.15.92:0/933673076 learned_addr learned my addr 172.21.15.92:0/933673076 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:54.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.244+0000 7ff121994700 1 --2- 172.21.15.92:0/933673076 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561298ae0950 0x561298ae0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.244+0000 7ff122195700 1 --2- 172.21.15.92:0/933673076 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561298b09e60 0x561298adc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.244+0000 7ff121193700 1 -- 172.21.15.92:0/933673076 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561298b09e60 msgr2=0x561298adc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:54.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.244+0000 7ff121193700 1 --2- 172.21.15.92:0/933673076 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561298b09e60 0x561298adc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.244+0000 7ff121193700 1 -- 172.21.15.92:0/933673076 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561298ae0950 msgr2=0x561298ae0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:54.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.244+0000 7ff121193700 1 --2- 172.21.15.92:0/933673076 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561298ae0950 0x561298ae0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.244+0000 7ff121193700 1 -- 172.21.15.92:0/933673076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612988cb270 con 0x561298b2abc0 2023-10-06T10:43:54.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.244+0000 7ff121994700 1 --2- 172.21.15.92:0/933673076 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561298ae0950 0x561298ae0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:54.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.244+0000 7ff121193700 1 --2- 172.21.15.92:0/933673076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298b2abc0 0x561298b09920 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00ed60 tx=0x7ff10c01f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd840008f43000e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:54.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.245+0000 7ff106ffd700 1 -- 172.21.15.92:0/933673076 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c0576e0 con 0x561298b2abc0 2023-10-06T10:43:54.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.245+0000 7ff106ffd700 1 -- 172.21.15.92:0/933673076 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10c056fe0 con 0x561298b2abc0 2023-10-06T10:43:54.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.245+0000 7ff106ffd700 1 -- 172.21.15.92:0/933673076 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c05a230 con 0x561298b2abc0 2023-10-06T10:43:54.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.245+0000 7ff12cef0b80 1 -- 172.21.15.92:0/933673076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298b2abc0 msgr2=0x561298b09920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:54.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.245+0000 7ff12cef0b80 1 --2- 172.21.15.92:0/933673076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298b2abc0 0x561298b09920 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00ed60 tx=0x7ff10c01f040 comp rx=0 tx=0).stop 2023-10-06T10:43:54.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.245+0000 7ff12cef0b80 1 -- 172.21.15.92:0/933673076 shutdown_connections 2023-10-06T10:43:54.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.245+0000 7ff12cef0b80 1 --2- 172.21.15.92:0/933673076 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561298b09e60 0x561298adc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.245+0000 7ff12cef0b80 1 --2- 172.21.15.92:0/933673076 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561298ae0950 0x561298ae0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.245+0000 7ff12cef0b80 1 --2- 172.21.15.92:0/933673076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298b2abc0 0x561298b09920 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.245+0000 7ff12cef0b80 1 -- 172.21.15.92:0/933673076 >> 172.21.15.92:0/933673076 conn(0x561298ae2270 msgr2=0x561298ae2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:54.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.245+0000 7ff12cef0b80 1 -- 172.21.15.92:0/933673076 shutdown_connections 2023-10-06T10:43:54.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.245+0000 7ff12cef0b80 1 -- 172.21.15.92:0/933673076 wait complete. 2023-10-06T10:43:54.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.246+0000 7ff12cef0b80 1 -- start start 2023-10-06T10:43:54.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.246+0000 7ff12cef0b80 1 Processor -- start 2023-10-06T10:43:54.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.247+0000 7ff12cef0b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561298ae0950 0x561298b37440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.247+0000 7ff12cef0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298b09e60 0x561298adcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.247+0000 7ff12cef0b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561298b2abc0 0x561298add4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.247+0000 7ff12cef0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561298a31060 con 0x561298b09e60 2023-10-06T10:43:54.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.247+0000 7ff12cef0b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612989dc890 con 0x561298ae0950 2023-10-06T10:43:54.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.247+0000 7ff12cef0b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612989f7ea0 con 0x561298b2abc0 2023-10-06T10:43:54.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.247+0000 7ff12cef0b80 10 client.0 _ll_register_callbacks cb 0x5612988cdf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:54.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.247+0000 7ff121193700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298b09e60 0x561298adcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.247+0000 7ff121193700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298b09e60 0x561298adcf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36222/0 (socket says 172.21.15.92:36222) 2023-10-06T10:43:54.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.247+0000 7ff121193700 1 -- 172.21.15.92:0/2322993017 learned_addr learned my addr 172.21.15.92:0/2322993017 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:54.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.247+0000 7ff121994700 1 --2- 172.21.15.92:0/2322993017 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561298ae0950 0x561298b37440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.247+0000 7ff122195700 1 --2- 172.21.15.92:0/2322993017 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561298b2abc0 0x561298add4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.247+0000 7ff121193700 1 -- 172.21.15.92:0/2322993017 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561298b2abc0 msgr2=0x561298add4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:54.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.247+0000 7ff121193700 1 --2- 172.21.15.92:0/2322993017 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561298b2abc0 0x561298add4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.247+0000 7ff121193700 1 -- 172.21.15.92:0/2322993017 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561298ae0950 msgr2=0x561298b37440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:54.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.247+0000 7ff121193700 1 --2- 172.21.15.92:0/2322993017 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561298ae0950 0x561298b37440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.247+0000 7ff121193700 1 -- 172.21.15.92:0/2322993017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11c15cbe0 con 0x561298b09e60 2023-10-06T10:43:54.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.247+0000 7ff121193700 1 --2- 172.21.15.92:0/2322993017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298b09e60 0x561298adcf60 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7ff10c0093f0 tx=0x7ff10c05a030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7fd86448b27709 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:54.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.247+0000 7ff0fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:54.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.248+0000 7ff0fbfff700 1 -- 172.21.15.92:0/2322993017 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c0576e0 con 0x561298b09e60 2023-10-06T10:43:54.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.248+0000 7ff12cef0b80 1 -- 172.21.15.92:0/2322993017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561298ad5690 con 0x561298b09e60 2023-10-06T10:43:54.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.248+0000 7ff0fbfff700 1 -- 172.21.15.92:0/2322993017 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff10c05df80 con 0x561298b09e60 2023-10-06T10:43:54.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.248+0000 7ff0fbfff700 1 -- 172.21.15.92:0/2322993017 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff10c05a230 con 0x561298b09e60 2023-10-06T10:43:54.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.248+0000 7ff12cef0b80 10 client.7851 fetch_fsmap learned FSMap version 83 2023-10-06T10:43:54.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.248+0000 7ff12cef0b80 1 -- 172.21.15.92:0/2322993017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x56129886dfe0 con 0x561298b09e60 2023-10-06T10:43:54.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.248+0000 7ff0fbfff700 1 -- 172.21.15.92:0/2322993017 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561298ad5690 con 0x561298b09e60 2023-10-06T10:43:54.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.248+0000 7ff0fbfff700 1 -- 172.21.15.92:0/2322993017 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 83) v1 ==== 3100+0+0 (secure 0 0 0) 0x7ff10c06d1b0 con 0x561298b09e60 2023-10-06T10:43:54.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.248+0000 7ff12cef0b80 10 client.7851 fetch_fsmap finished waiting for FSMap version 83 2023-10-06T10:43:54.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.248+0000 7ff12cef0b80 10 client.7851 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T10:43:54.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.248+0000 7ff12cef0b80 20 client.7851 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:54.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.248+0000 7ff12cef0b80 1 --2- 172.21.15.92:0/2322993017 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x561298b08500 0x561298b088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.248+0000 7ff12cef0b80 4 client.7851 mds_command: new command op to 7701 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:43:54.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.248+0000 7ff12cef0b80 1 -- 172.21.15.92:0/2322993017 --> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561298ab25c0 con 0x561298b08500 2023-10-06T10:43:54.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.248+0000 7ff121994700 1 --2- 172.21.15.92:0/2322993017 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x561298b08500 0x561298b088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.249+0000 7ff0fbfff700 10 client.7851 ms_handle_connect on v2:172.21.15.124:6832/4203118992 2023-10-06T10:43:54.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.249+0000 7ff121994700 1 --2- 172.21.15.92:0/2322993017 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x561298b08500 0x561298b088d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acdc74b047728960 server_cookie=c42995e77bbe251f in_seq=0 out_seq=0 2023-10-06T10:43:54.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.249+0000 7ff0fbfff700 1 -- 172.21.15.92:0/2322993017 <== mds.0 v2:172.21.15.124:6832/4203118992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x561298ab25c0 con 0x561298b08500 2023-10-06T10:43:54.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.249+0000 7ff0fbfff700 10 client.7851 handle_command_reply: tid=0 2023-10-06T10:43:54.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.249+0000 7ff12cef0b80 1 client.7851 shutdown 2023-10-06T10:43:54.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.249+0000 7ff12cef0b80 1 -- 172.21.15.92:0/2322993017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298b09e60 msgr2=0x561298adcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:54.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.249+0000 7ff12cef0b80 1 --2- 172.21.15.92:0/2322993017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298b09e60 0x561298adcf60 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7ff10c0093f0 tx=0x7ff10c05a030 comp rx=0 tx=0).stop 2023-10-06T10:43:54.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.250+0000 7ff12cef0b80 1 -- 172.21.15.92:0/2322993017 shutdown_connections 2023-10-06T10:43:54.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.250+0000 7ff12cef0b80 1 --2- 172.21.15.92:0/2322993017 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x561298b08500 0x561298b088d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.250+0000 7ff12cef0b80 1 --2- 172.21.15.92:0/2322993017 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561298b2abc0 0x561298add4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.250+0000 7ff0fbfff700 0 client.7851 ms_handle_reset on v2:172.21.15.124:6832/4203118992 2023-10-06T10:43:54.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.250+0000 7ff12cef0b80 1 --2- 172.21.15.92:0/2322993017 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561298ae0950 0x561298b37440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.250+0000 7ff12cef0b80 1 --2- 172.21.15.92:0/2322993017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298b09e60 0x561298adcf60 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.250+0000 7ff12cef0b80 1 -- 172.21.15.92:0/2322993017 >> 172.21.15.92:0/2322993017 conn(0x561298ae2270 msgr2=0x561298b067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:54.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.250+0000 7ff12cef0b80 1 -- 172.21.15.92:0/2322993017 shutdown_connections 2023-10-06T10:43:54.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.250+0000 7ff12cef0b80 1 -- 172.21.15.92:0/2322993017 wait complete. 2023-10-06T10:43:54.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.250+0000 7ff12cef0b80 20 client.7851 trim_cache size 0 max 16384 2023-10-06T10:43:54.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.266+0000 7ff12cef0b80 1 Processor -- start 2023-10-06T10:43:54.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.266+0000 7ff12cef0b80 1 -- start start 2023-10-06T10:43:54.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.267+0000 7ff12cef0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298ae0950 0x561298ae2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.267+0000 7ff12cef0b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561298b09e60 0x561298b22790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.267+0000 7ff12cef0b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561298b2abc0 0x561298b22cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.267+0000 7ff12cef0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612988b3e30 con 0x561298ae0950 2023-10-06T10:43:54.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.267+0000 7ff12cef0b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561298ad7f40 con 0x561298b09e60 2023-10-06T10:43:54.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.267+0000 7ff12cef0b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561298b25580 con 0x561298b2abc0 2023-10-06T10:43:54.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.267+0000 7ff121994700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298ae0950 0x561298ae2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.267+0000 7ff121994700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298ae0950 0x561298ae2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36238/0 (socket says 172.21.15.92:36238) 2023-10-06T10:43:54.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.267+0000 7ff121994700 1 -- 172.21.15.92:0/1574143028 learned_addr learned my addr 172.21.15.92:0/1574143028 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:54.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.267+0000 7ff121994700 1 -- 172.21.15.92:0/1574143028 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561298b2abc0 msgr2=0x561298b22cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:54.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.267+0000 7ff121994700 1 --2- 172.21.15.92:0/1574143028 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561298b2abc0 0x561298b22cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.267+0000 7ff121994700 1 -- 172.21.15.92:0/1574143028 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561298b09e60 msgr2=0x561298b22790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:54.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.267+0000 7ff121994700 1 --2- 172.21.15.92:0/1574143028 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561298b09e60 0x561298b22790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.267+0000 7ff121994700 1 -- 172.21.15.92:0/1574143028 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612989ff780 con 0x561298ae0950 2023-10-06T10:43:54.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.268+0000 7ff121994700 1 --2- 172.21.15.92:0/1574143028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298ae0950 0x561298ae2640 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x561298addb00 tx=0x7ff118016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99adefedee108dfa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:54.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.268+0000 7ff1077fe700 1 -- 172.21.15.92:0/1574143028 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118005500 con 0x561298ae0950 2023-10-06T10:43:54.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.268+0000 7ff1077fe700 1 -- 172.21.15.92:0/1574143028 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11801c040 con 0x561298ae0950 2023-10-06T10:43:54.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.268+0000 7ff1077fe700 1 -- 172.21.15.92:0/1574143028 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118005680 con 0x561298ae0950 2023-10-06T10:43:54.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.268+0000 7ff12cef0b80 1 -- 172.21.15.92:0/1574143028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298ae0950 msgr2=0x561298ae2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:54.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.268+0000 7ff12cef0b80 1 --2- 172.21.15.92:0/1574143028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298ae0950 0x561298ae2640 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x561298addb00 tx=0x7ff118016040 comp rx=0 tx=0).stop 2023-10-06T10:43:54.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.268+0000 7ff12cef0b80 1 -- 172.21.15.92:0/1574143028 shutdown_connections 2023-10-06T10:43:54.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.268+0000 7ff12cef0b80 1 --2- 172.21.15.92:0/1574143028 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561298b2abc0 0x561298b22cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.268+0000 7ff12cef0b80 1 --2- 172.21.15.92:0/1574143028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298ae0950 0x561298ae2640 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.268+0000 7ff12cef0b80 1 --2- 172.21.15.92:0/1574143028 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561298b09e60 0x561298b22790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.268+0000 7ff12cef0b80 1 -- 172.21.15.92:0/1574143028 >> 172.21.15.92:0/1574143028 conn(0x561298ae2270 msgr2=0x561298b39c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:54.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.268+0000 7ff12cef0b80 1 -- 172.21.15.92:0/1574143028 shutdown_connections 2023-10-06T10:43:54.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.268+0000 7ff12cef0b80 1 -- 172.21.15.92:0/1574143028 wait complete. 2023-10-06T10:43:54.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.269+0000 7ff12cef0b80 1 -- start start 2023-10-06T10:43:54.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.269+0000 7ff12cef0b80 1 Processor -- start 2023-10-06T10:43:54.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.270+0000 7ff12cef0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298ae0950 0x561298b22790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.270+0000 7ff12cef0b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561298b09e60 0x561298b22cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.270+0000 7ff12cef0b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561298b2abc0 0x561298b2aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.270+0000 7ff12cef0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561298b8d930 con 0x561298ae0950 2023-10-06T10:43:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.270+0000 7ff12cef0b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561298b1d5b0 con 0x561298b09e60 2023-10-06T10:43:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.270+0000 7ff12cef0b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561298add9d0 con 0x561298b2abc0 2023-10-06T10:43:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.270+0000 7ff12cef0b80 10 client.0 _ll_register_callbacks cb 0x5612988cdf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.270+0000 7ff121994700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298ae0950 0x561298b22790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.270+0000 7ff121994700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298ae0950 0x561298b22790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36248/0 (socket says 172.21.15.92:36248) 2023-10-06T10:43:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.270+0000 7ff121994700 1 -- 172.21.15.92:0/3358166120 learned_addr learned my addr 172.21.15.92:0/3358166120 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.270+0000 7ff121994700 1 -- 172.21.15.92:0/3358166120 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561298b2abc0 msgr2=0x561298b2aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.270+0000 7ff121193700 1 --2- 172.21.15.92:0/3358166120 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561298b09e60 0x561298b22cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.270+0000 7ff122195700 1 --2- 172.21.15.92:0/3358166120 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561298b2abc0 0x561298b2aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.270+0000 7ff121994700 1 --2- 172.21.15.92:0/3358166120 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561298b2abc0 0x561298b2aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.270+0000 7ff121994700 1 -- 172.21.15.92:0/3358166120 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561298b09e60 msgr2=0x561298b22cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.270+0000 7ff121994700 1 --2- 172.21.15.92:0/3358166120 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561298b09e60 0x561298b22cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.270+0000 7ff121994700 1 -- 172.21.15.92:0/3358166120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11c0cb430 con 0x561298ae0950 2023-10-06T10:43:54.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.270+0000 7ff121994700 1 --2- 172.21.15.92:0/3358166120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298ae0950 0x561298b22790 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7ff118004a20 tx=0x7ff118028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198662fba57fecae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:54.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.270+0000 7ff0fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:54.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.270+0000 7ff0fbfff700 1 -- 172.21.15.92:0/3358166120 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118025990 con 0x561298ae0950 2023-10-06T10:43:54.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.270+0000 7ff0fbfff700 1 -- 172.21.15.92:0/3358166120 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff118025af0 con 0x561298ae0950 2023-10-06T10:43:54.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.270+0000 7ff12cef0b80 1 -- 172.21.15.92:0/3358166120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff10c056fe0 con 0x561298ae0950 2023-10-06T10:43:54.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.271+0000 7ff0fbfff700 1 -- 172.21.15.92:0/3358166120 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff11800f540 con 0x561298ae0950 2023-10-06T10:43:54.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.271+0000 7ff12cef0b80 10 client.7857 fetch_fsmap learned FSMap version 83 2023-10-06T10:43:54.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.271+0000 7ff12cef0b80 1 -- 172.21.15.92:0/3358166120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x5612989e6470 con 0x561298ae0950 2023-10-06T10:43:54.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.271+0000 7ff0fbfff700 1 -- 172.21.15.92:0/3358166120 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10c056fe0 con 0x561298ae0950 2023-10-06T10:43:54.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.271+0000 7ff0fbfff700 1 -- 172.21.15.92:0/3358166120 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 83) v1 ==== 3100+0+0 (secure 0 0 0) 0x7ff11802d020 con 0x561298ae0950 2023-10-06T10:43:54.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.271+0000 7ff12cef0b80 10 client.7857 fetch_fsmap finished waiting for FSMap version 83 2023-10-06T10:43:54.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.271+0000 7ff12cef0b80 10 client.7857 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T10:43:54.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.271+0000 7ff12cef0b80 20 client.7857 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:54.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.271+0000 7ff12cef0b80 1 --2- 172.21.15.92:0/3358166120 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x561298b2b530 0x561298b2b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.271+0000 7ff12cef0b80 4 client.7857 mds_command: new command op to 7701 tid=0[{"prefix": "status"}] 2023-10-06T10:43:54.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.271+0000 7ff12cef0b80 1 -- 172.21.15.92:0/3358166120 --> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5612989d3e00 con 0x561298b2b530 2023-10-06T10:43:54.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.271+0000 7ff121193700 1 --2- 172.21.15.92:0/3358166120 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x561298b2b530 0x561298b2b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.271+0000 7ff0fbfff700 10 client.7857 ms_handle_connect on v2:172.21.15.124:6832/4203118992 2023-10-06T10:43:54.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.271+0000 7ff121193700 1 --2- 172.21.15.92:0/3358166120 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x561298b2b530 0x561298b2b900 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5273ba2c0db9afe4 server_cookie=2843284281045752 in_seq=0 out_seq=0 2023-10-06T10:43:54.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.272+0000 7ff0fbfff700 1 -- 172.21.15.92:0/3358166120 <== mds.0 v2:172.21.15.124:6832/4203118992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5612989d3e00 con 0x561298b2b530 2023-10-06T10:43:54.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.272+0000 7ff0fbfff700 10 client.7857 handle_command_reply: tid=0 2023-10-06T10:43:54.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.272+0000 7ff12cef0b80 1 client.7857 shutdown 2023-10-06T10:43:54.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.272+0000 7ff12cef0b80 1 -- 172.21.15.92:0/3358166120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298ae0950 msgr2=0x561298b22790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:54.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.272+0000 7ff12cef0b80 1 --2- 172.21.15.92:0/3358166120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298ae0950 0x561298b22790 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7ff118004a20 tx=0x7ff118028000 comp rx=0 tx=0).stop 2023-10-06T10:43:54.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.272+0000 7ff12cef0b80 1 -- 172.21.15.92:0/3358166120 shutdown_connections 2023-10-06T10:43:54.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.272+0000 7ff12cef0b80 1 --2- 172.21.15.92:0/3358166120 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x561298b2b530 0x561298b2b900 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.272+0000 7ff12cef0b80 1 --2- 172.21.15.92:0/3358166120 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561298b2abc0 0x561298b2aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.272+0000 7ff0fbfff700 0 client.7857 ms_handle_reset on v2:172.21.15.124:6832/4203118992 2023-10-06T10:43:54.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.272+0000 7ff12cef0b80 1 --2- 172.21.15.92:0/3358166120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561298ae0950 0x561298b22790 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.272+0000 7ff12cef0b80 1 --2- 172.21.15.92:0/3358166120 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561298b09e60 0x561298b22cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.272+0000 7ff12cef0b80 1 -- 172.21.15.92:0/3358166120 >> 172.21.15.92:0/3358166120 conn(0x561298ae2270 msgr2=0x561298b36b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:54.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.272+0000 7ff12cef0b80 1 -- 172.21.15.92:0/3358166120 shutdown_connections 2023-10-06T10:43:54.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.272+0000 7ff12cef0b80 1 -- 172.21.15.92:0/3358166120 wait complete. 2023-10-06T10:43:54.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.272+0000 7ff12cef0b80 20 client.7857 trim_cache size 0 max 16384 2023-10-06T10:43:54.279 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:43:54.280 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:43:54.280 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T10:43:54.280 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 7701, 2023-10-06T10:43:54.280 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:43:54.280 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:43:54.280 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:43:54.280 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 5.7844646729999996, 2023-10-06T10:43:54.280 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 83, 2023-10-06T10:43:54.280 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 90, 2023-10-06T10:43:54.280 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 90, 2023-10-06T10:43:54.280 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 11.91365439 2023-10-06T10:43:54.280 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:43:54.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.272+0000 7ff122996700 1 -- 172.21.15.92:0/1170116887 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff100066f30 msgr2=0x7ff1000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:54.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.272+0000 7ff122996700 1 --2- 172.21.15.92:0/1170116887 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff100066f30 0x7ff1000693e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff1180015a0 tx=0x7ff118000ed0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.272+0000 7ff122996700 1 -- 172.21.15.92:0/1170116887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff11c15c230 msgr2=0x7ff11c15c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:54.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.273+0000 7ff122996700 1 --2- 172.21.15.92:0/1170116887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff11c15c230 0x7ff11c15c6a0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7ff10800bb30 tx=0x7ff10c01a000 comp rx=0 tx=0).stop 2023-10-06T10:43:54.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.273+0000 7ff122996700 1 -- 172.21.15.92:0/1170116887 shutdown_connections 2023-10-06T10:43:54.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.273+0000 7ff122996700 1 --2- 172.21.15.92:0/1170116887 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff100066f30 0x7ff1000693e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.273+0000 7ff122996700 1 --2- 172.21.15.92:0/1170116887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff11c15cf50 0x7ff11c15d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.273+0000 7ff122996700 1 --2- 172.21.15.92:0/1170116887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff108004810 0x7ff11c15a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.273+0000 7ff122996700 1 --2- 172.21.15.92:0/1170116887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff11c15c230 0x7ff11c15c6a0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.273+0000 7ff122996700 1 -- 172.21.15.92:0/1170116887 >> 172.21.15.92:0/1170116887 conn(0x7ff11c06ece0 msgr2=0x7ff11c0b5080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:54.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.273+0000 7ff122996700 1 -- 172.21.15.92:0/1170116887 shutdown_connections 2023-10-06T10:43:54.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.273+0000 7ff122996700 1 -- 172.21.15.92:0/1170116887 wait complete. 2023-10-06T10:43:54.285 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T10:43:54.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.399+0000 7f33957c2700 1 Processor -- start 2023-10-06T10:43:54.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.399+0000 7f33957c2700 1 -- start start 2023-10-06T10:43:54.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.400+0000 7f33957c2700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33900adb60 0x7f33900ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.400+0000 7f33957c2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33900ceb30 con 0x7f33900ce6d0 2023-10-06T10:43:54.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.400+0000 7f33957c2700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3390157de0 con 0x7f33900ca370 2023-10-06T10:43:54.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.400+0000 7f33957c2700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33900cbb90 con 0x7f33900adb60 2023-10-06T10:43:54.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.400+0000 7f338f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f33900ce6d0 0x7f339015aed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57266/0 (socket says 172.21.15.92:57266) 2023-10-06T10:43:54.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.400+0000 7f338f7fe700 1 -- 172.21.15.92:0/3092736759 learned_addr learned my addr 172.21.15.92:0/3092736759 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:54.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.401+0000 7f33867fc700 1 --2- 172.21.15.92:0/3092736759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33900adb60 0x7f33900ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.401+0000 7f338e7fc700 1 -- 172.21.15.92:0/3092736759 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147600834 0 0) 0x7f3388001490 con 0x7f33900ce6d0 2023-10-06T10:43:54.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.401+0000 7f338e7fc700 1 -- 172.21.15.92:0/3092736759 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f337c003570 con 0x7f33900ce6d0 2023-10-06T10:43:54.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.401+0000 7f338e7fc700 1 -- 172.21.15.92:0/3092736759 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764690670 0 0) 0x7f3390157de0 con 0x7f33900ca370 2023-10-06T10:43:54.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.401+0000 7f338e7fc700 1 -- 172.21.15.92:0/3092736759 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3388001490 con 0x7f33900ca370 2023-10-06T10:43:54.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.401+0000 7f338e7fc700 1 -- 172.21.15.92:0/3092736759 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1032004445 0 0) 0x7f337c003570 con 0x7f33900ce6d0 2023-10-06T10:43:54.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.401+0000 7f33867fc700 1 -- 172.21.15.92:0/3092736759 >> v1:172.21.15.124:6789/0 conn(0x7f33900ca370 legacy=0x7f3390158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:54.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.401+0000 7f33867fc700 1 -- 172.21.15.92:0/3092736759 >> v1:172.21.15.92:6789/0 conn(0x7f33900ce6d0 legacy=0x7f339015aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:54.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.401+0000 7f33867fc700 1 -- 172.21.15.92:0/3092736759 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33900cb5d0 con 0x7f33900adb60 2023-10-06T10:43:54.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.401+0000 7f33867fc700 1 --2- 172.21.15.92:0/3092736759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33900adb60 0x7f33900ce190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3378009910 tx=0x7f33780091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60bbc2b48aea5125 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:54.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.401+0000 7f338e7fc700 1 -- 172.21.15.92:0/3092736759 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3388001980 con 0x7f33900adb60 2023-10-06T10:43:54.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.401+0000 7f338e7fc700 1 -- 172.21.15.92:0/3092736759 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3378011420 con 0x7f33900adb60 2023-10-06T10:43:54.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.401+0000 7f338e7fc700 1 -- 172.21.15.92:0/3092736759 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33780117c0 con 0x7f33900adb60 2023-10-06T10:43:54.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.402+0000 7f33957c2700 1 -- 172.21.15.92:0/3092736759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33900adb60 msgr2=0x7f33900ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:54.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.402+0000 7f33957c2700 1 --2- 172.21.15.92:0/3092736759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33900adb60 0x7f33900ce190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3378009910 tx=0x7f33780091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.402+0000 7f33957c2700 1 -- 172.21.15.92:0/3092736759 shutdown_connections 2023-10-06T10:43:54.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.402+0000 7f33957c2700 1 --2- 172.21.15.92:0/3092736759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33900adb60 0x7f33900ce190 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.402+0000 7f33957c2700 1 -- 172.21.15.92:0/3092736759 >> 172.21.15.92:0/3092736759 conn(0x7f33900707d0 msgr2=0x7f3390070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:54.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.402+0000 7f33957c2700 1 -- 172.21.15.92:0/3092736759 shutdown_connections 2023-10-06T10:43:54.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.402+0000 7f33957c2700 1 -- 172.21.15.92:0/3092736759 wait complete. 2023-10-06T10:43:54.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.403+0000 7f33957c2700 1 Processor -- start 2023-10-06T10:43:54.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.403+0000 7f33957c2700 1 -- start start 2023-10-06T10:43:54.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f33957c2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33900adb60 0x7f33900c1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f33957c2700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33900ca370 0x7f33900c0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f33957c2700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33900ce6d0 0x7f339015ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f33957c2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33900b1300 con 0x7f33900adb60 2023-10-06T10:43:54.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f33957c2700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33900b18c0 con 0x7f33900ce6d0 2023-10-06T10:43:54.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f33957c2700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33900c4c00 con 0x7f33900ca370 2023-10-06T10:43:54.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f338effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33900adb60 0x7f33900c1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f338effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33900adb60 0x7f33900c1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36264/0 (socket says 172.21.15.92:36264) 2023-10-06T10:43:54.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f338effd700 1 -- 172.21.15.92:0/4207737752 learned_addr learned my addr 172.21.15.92:0/4207737752 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:54.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f338f7fe700 1 --2- 172.21.15.92:0/4207737752 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33900ce6d0 0x7f339015ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f33867fc700 1 --2- 172.21.15.92:0/4207737752 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33900ca370 0x7f33900c0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f338effd700 1 -- 172.21.15.92:0/4207737752 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33900ca370 msgr2=0x7f33900c0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:54.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f338effd700 1 --2- 172.21.15.92:0/4207737752 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33900ca370 0x7f33900c0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f338effd700 1 -- 172.21.15.92:0/4207737752 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33900ce6d0 msgr2=0x7f339015ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:54.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f338effd700 1 --2- 172.21.15.92:0/4207737752 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33900ce6d0 0x7f339015ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f338effd700 1 -- 172.21.15.92:0/4207737752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3378009420 con 0x7f33900adb60 2023-10-06T10:43:54.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f338f7fe700 1 --2- 172.21.15.92:0/4207737752 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33900ce6d0 0x7f339015ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:54.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f338effd700 1 --2- 172.21.15.92:0/4207737752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33900adb60 0x7f33900c1f40 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f338000c7f0 tx=0x7f3380007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:54.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f33877fe700 1 -- 172.21.15.92:0/4207737752 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3380016070 con 0x7f33900adb60 2023-10-06T10:43:54.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f33877fe700 1 -- 172.21.15.92:0/4207737752 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3380011680 con 0x7f33900adb60 2023-10-06T10:43:54.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f33957c2700 1 -- 172.21.15.92:0/4207737752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33900c4ee0 con 0x7f33900adb60 2023-10-06T10:43:54.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f33877fe700 1 -- 172.21.15.92:0/4207737752 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3380011a20 con 0x7f33900adb60 2023-10-06T10:43:54.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.404+0000 7f33957c2700 1 -- 172.21.15.92:0/4207737752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33900c1480 con 0x7f33900adb60 2023-10-06T10:43:54.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.405+0000 7f33957c2700 1 -- 172.21.15.92:0/4207737752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33900c1670 con 0x7f33900adb60 2023-10-06T10:43:54.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.406+0000 7f33877fe700 1 -- 172.21.15.92:0/4207737752 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3380012440 con 0x7f33900adb60 2023-10-06T10:43:54.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.406+0000 7f33877fe700 1 --2- 172.21.15.92:0/4207737752 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3370066fb0 0x7f3370069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:54.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.406+0000 7f33867fc700 1 --2- 172.21.15.92:0/4207737752 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3370066fb0 0x7f3370069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:54.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.406+0000 7f33877fe700 1 -- 172.21.15.92:0/4207737752 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3380084690 con 0x7f33900adb60 2023-10-06T10:43:54.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.407+0000 7f33867fc700 1 --2- 172.21.15.92:0/4207737752 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3370066fb0 0x7f3370069460 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f33780030c0 tx=0x7f3378016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:54.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.411+0000 7f33877fe700 1 -- 172.21.15.92:0/4207737752 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f33800515c0 con 0x7f33900adb60 2023-10-06T10:43:54.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.638+0000 7f33957c2700 1 -- 172.21.15.92:0/4207737752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3390064a90 con 0x7f33900adb60 2023-10-06T10:43:54.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.639+0000 7f33877fe700 1 -- 172.21.15.92:0/4207737752 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v83) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3380054be0 con 0x7f33900adb60 2023-10-06T10:43:54.640 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T10:43:54.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.640+0000 7f33957c2700 1 -- 172.21.15.92:0/4207737752 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3370066fb0 msgr2=0x7f3370069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:54.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.640+0000 7f33957c2700 1 --2- 172.21.15.92:0/4207737752 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3370066fb0 0x7f3370069460 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f33780030c0 tx=0x7f3378016000 comp rx=0 tx=0).stop 2023-10-06T10:43:54.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.640+0000 7f33957c2700 1 -- 172.21.15.92:0/4207737752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33900adb60 msgr2=0x7f33900c1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:54.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.640+0000 7f33957c2700 1 --2- 172.21.15.92:0/4207737752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33900adb60 0x7f33900c1f40 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f338000c7f0 tx=0x7f3380007dd0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.640+0000 7f33957c2700 1 -- 172.21.15.92:0/4207737752 shutdown_connections 2023-10-06T10:43:54.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.640+0000 7f33957c2700 1 --2- 172.21.15.92:0/4207737752 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3370066fb0 0x7f3370069460 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.640+0000 7f33957c2700 1 --2- 172.21.15.92:0/4207737752 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33900ce6d0 0x7f339015ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.640+0000 7f33957c2700 1 --2- 172.21.15.92:0/4207737752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33900adb60 0x7f33900c1f40 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.640+0000 7f33957c2700 1 --2- 172.21.15.92:0/4207737752 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33900ca370 0x7f33900c0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:54.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.641+0000 7f33957c2700 1 -- 172.21.15.92:0/4207737752 >> 172.21.15.92:0/4207737752 conn(0x7f33900707d0 msgr2=0x7f33901534f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:54.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.641+0000 7f33957c2700 1 -- 172.21.15.92:0/4207737752 shutdown_connections 2023-10-06T10:43:54.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:54.641+0000 7f33957c2700 1 -- 172.21.15.92:0/4207737752 wait complete. 2023-10-06T10:43:54.653 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T10:43:54.653 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T10:43:54.653 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T10:43:54.654 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T10:43:54.654 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T10:43:54.654 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T10:43:54.654 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T10:43:54.654 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T10:43:54.654 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T10:43:54.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:43:54.654 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T10:43:54.670 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T10:43:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T10:43:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T10:43:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:43:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T10:43:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:43:54.670 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:43:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:43:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T10:43:54.671 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:43:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:43:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T10:43:54.671 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:43:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:43:54.671 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T10:43:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:43:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T10:43:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42175sec preferred_lft 42175sec 2023-10-06T10:43:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T10:43:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:43:54.671 INFO:teuthology.orchestra.run.smithi092.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T10:43:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 92:7f:88:24:4e:ca brd ff:ff:ff:ff:ff:ff 2023-10-06T10:43:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T10:43:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:43:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::9c9d:23ff:fee4:30ae/64 scope link 2023-10-06T10:43:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:43:54.672 INFO:teuthology.orchestra.run.smithi092.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T10:43:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 92:7f:88:24:4e:ca brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T10:43:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::907f:88ff:fe24:4eca/64 scope link 2023-10-06T10:43:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:43:54.672 INFO:teuthology.orchestra.run.smithi092.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T10:43:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether b6:32:f9:d9:96:1b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T10:43:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::b432:f9ff:fed9:961b/64 scope link 2023-10-06T10:43:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:43:54.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:43:54.673 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T10:43:54.733 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T10:43:54.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:43:54.735 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T10:43:54.790 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T10:43:54.791 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T10:43:54.791 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T10:43:54.791 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T10:43:54.791 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T10:43:54.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:43:54.791 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T10:43:54.791 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T10:43:54.792 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T10:43:54.792 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T10:43:54.792 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T10:43:54.792 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T10:43:54.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:43:54.890 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T10:43:54.904 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T10:43:54.904 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T10:43:54.905 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T10:43:54.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:43:54.905 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-10-06T10:43:54.968 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T10:43:54.969 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T10:43:55.023 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T10:43:55.023 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T10:43:55.023 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T10:43:55.076 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T10:43:55.076 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T10:43:55.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:43:55.156 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:43:55.157 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:43:55.175 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:43:55.175 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:43:55.176 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-06T10:43:55.176 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@8f307bfb-c4cb-4f66-aade-46fffe0c6e0c.cephfs=/ 2023-10-06T10:43:55.176 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-06T10:43:55.176 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T10:43:55.176 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T10:43:55.176 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T10:43:55.198 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:43:55.198 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:43:55.198 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-06T10:43:55.198 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@8f307bfb-c4cb-4f66-aade-46fffe0c6e0c.cephfs=/ 2023-10-06T10:43:55.198 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-06T10:43:55.198 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T10:43:55.198 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T10:43:55.198 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T10:43:55.199 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.124:6789,172.21.15.124:6790" 2023-10-06T10:43:55.199 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.124:6789,172.21.15.124:6790:/ 2023-10-06T10:43:55.199 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=8f307bfb-c4cb-4f66-aade-46fffe0c6e0c" will pass to kernel 2023-10-06T10:43:55.200 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T10:43:55.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:43:55.200 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T10:43:55.265 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-10-06T10:43:55.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.421+0000 7fe2b55fa700 1 Processor -- start 2023-10-06T10:43:55.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.421+0000 7fe2b55fa700 1 -- start start 2023-10-06T10:43:55.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.422+0000 7fe2b55fa700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe2b0155800 0x7fe2b006b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:55.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.422+0000 7fe2b55fa700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b006f020 con 0x7fe2b0159530 2023-10-06T10:43:55.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.422+0000 7fe2b55fa700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b01599c0 con 0x7fe2b006a4c0 2023-10-06T10:43:55.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.422+0000 7fe2b55fa700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b0159ca0 con 0x7fe2b0155800 2023-10-06T10:43:55.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.422+0000 7fe2a67fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe2b0159530 0x7fe2b0069db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57270/0 (socket says 172.21.15.92:57270) 2023-10-06T10:43:55.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.422+0000 7fe2a67fc700 1 -- 172.21.15.92:0/3602257563 learned_addr learned my addr 172.21.15.92:0/3602257563 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:55.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.422+0000 7fe2ae7fc700 1 -- 172.21.15.92:0/3602257563 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 436546496 0 0) 0x7fe2b006f020 con 0x7fe2b0159530 2023-10-06T10:43:55.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.422+0000 7fe2ae7fc700 1 -- 172.21.15.92:0/3602257563 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe29c003570 con 0x7fe2b0159530 2023-10-06T10:43:55.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.422+0000 7fe2aeffd700 1 --2- 172.21.15.92:0/3602257563 >> v2:172.21.15.124:3300/0 conn(0x7fe2b0155800 0x7fe2b006b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:55.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2ae7fc700 1 -- 172.21.15.92:0/3602257563 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2987933619 0 0) 0x7fe29c003570 con 0x7fe2b0159530 2023-10-06T10:43:55.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2ae7fc700 1 -- 172.21.15.92:0/3602257563 >> v2:172.21.15.124:3300/0 conn(0x7fe2b0155800 msgr2=0x7fe2b006b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:55.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2ae7fc700 1 --2- 172.21.15.92:0/3602257563 >> v2:172.21.15.124:3300/0 conn(0x7fe2b0155800 0x7fe2b006b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:55.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2ae7fc700 1 -- 172.21.15.92:0/3602257563 >> v1:172.21.15.124:6789/0 conn(0x7fe2b006a4c0 legacy=0x7fe2b0155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:55.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2ae7fc700 1 -- 172.21.15.92:0/3602257563 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b0159f80 con 0x7fe2b0159530 2023-10-06T10:43:55.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2ae7fc700 1 -- 172.21.15.92:0/3602257563 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fe298001470 con 0x7fe2b0159530 2023-10-06T10:43:55.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2ae7fc700 1 -- 172.21.15.92:0/3602257563 >> v1:172.21.15.92:6789/0 conn(0x7fe2b0159530 legacy=0x7fe2b0069db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:55.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2ae7fc700 1 --2- 172.21.15.92:0/3602257563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe29c003a90 0x7fe29c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:55.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2ae7fc700 1 --2- 172.21.15.92:0/3602257563 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe29c004810 0x7fe29c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:55.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2ae7fc700 1 --2- 172.21.15.92:0/3602257563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe29c0052e0 0x7fe29c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:55.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2ae7fc700 1 -- 172.21.15.92:0/3602257563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b0159ca0 con 0x7fe29c0052e0 2023-10-06T10:43:55.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2ae7fc700 1 -- 172.21.15.92:0/3602257563 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c00c1b0 con 0x7fe29c004810 2023-10-06T10:43:55.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2ae7fc700 1 -- 172.21.15.92:0/3602257563 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c00c370 con 0x7fe29c003a90 2023-10-06T10:43:55.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2aeffd700 1 --2- 172.21.15.92:0/3602257563 >> v2:172.21.15.124:3300/0 conn(0x7fe2b0155800 0x7fe2b006b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:55.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2af7fe700 1 --2- 172.21.15.92:0/3602257563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe29c0052e0 0x7fe29c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:55.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2a67fc700 1 --2- 172.21.15.92:0/3602257563 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe29c004810 0x7fe29c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:55.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2aeffd700 1 --2- 172.21.15.92:0/3602257563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe29c003a90 0x7fe29c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:55.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2af7fe700 1 -- 172.21.15.92:0/3602257563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe29c003a90 msgr2=0x7fe29c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:55.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2af7fe700 1 --2- 172.21.15.92:0/3602257563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe29c003a90 0x7fe29c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:55.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2af7fe700 1 -- 172.21.15.92:0/3602257563 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe29c004810 msgr2=0x7fe29c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:55.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2af7fe700 1 --2- 172.21.15.92:0/3602257563 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe29c004810 0x7fe29c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:55.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2af7fe700 1 -- 172.21.15.92:0/3602257563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c003570 con 0x7fe29c0052e0 2023-10-06T10:43:55.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2af7fe700 1 -- 172.21.15.92:0/3602257563 reap_dead start 2023-10-06T10:43:55.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2af7fe700 1 --2- 172.21.15.92:0/3602257563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe29c0052e0 0x7fe29c00ba60 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8001f90 tx=0x7fe2a8001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1066e830e69083a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:55.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2ae7fc700 1 -- 172.21.15.92:0/3602257563 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8001950 con 0x7fe29c0052e0 2023-10-06T10:43:55.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2ae7fc700 1 -- 172.21.15.92:0/3602257563 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a800c8b0 con 0x7fe29c0052e0 2023-10-06T10:43:55.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.423+0000 7fe2ae7fc700 1 -- 172.21.15.92:0/3602257563 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a800cc50 con 0x7fe29c0052e0 2023-10-06T10:43:55.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.424+0000 7fe2b55fa700 1 -- 172.21.15.92:0/3602257563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe29c0052e0 msgr2=0x7fe29c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:55.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.424+0000 7fe2b55fa700 1 --2- 172.21.15.92:0/3602257563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe29c0052e0 0x7fe29c00ba60 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8001f90 tx=0x7fe2a8001640 comp rx=0 tx=0).stop 2023-10-06T10:43:55.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.424+0000 7fe2b55fa700 1 -- 172.21.15.92:0/3602257563 shutdown_connections 2023-10-06T10:43:55.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.424+0000 7fe2b55fa700 1 --2- 172.21.15.92:0/3602257563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe29c0052e0 0x7fe29c00ba60 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:55.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.424+0000 7fe2b55fa700 1 -- 172.21.15.92:0/3602257563 >> 172.21.15.92:0/3602257563 conn(0x7fe2b00cc0a0 msgr2=0x7fe2b00cc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:55.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.424+0000 7fe2b55fa700 1 -- 172.21.15.92:0/3602257563 shutdown_connections 2023-10-06T10:43:55.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.424+0000 7fe2b55fa700 1 -- 172.21.15.92:0/3602257563 wait complete. 2023-10-06T10:43:55.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.425+0000 7fe2b55fa700 1 Processor -- start 2023-10-06T10:43:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.425+0000 7fe2b55fa700 1 -- start start 2023-10-06T10:43:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.426+0000 7fe2b55fa700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe29c0052e0 0x7fe2b014ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.426+0000 7fe2b55fa700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2b0157300 0x7fe2b01579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.426+0000 7fe2b55fa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b0156190 0x7fe2b0156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.426+0000 7fe2b55fa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b0156b70 con 0x7fe2b0156190 2023-10-06T10:43:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.426+0000 7fe2b55fa700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b014f230 con 0x7fe2b0157300 2023-10-06T10:43:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.426+0000 7fe2b55fa700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b0152380 con 0x7fe29c0052e0 2023-10-06T10:43:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.426+0000 7fe2af7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b0156190 0x7fe2b0156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.426+0000 7fe2af7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b0156190 0x7fe2b0156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36290/0 (socket says 172.21.15.92:36290) 2023-10-06T10:43:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.426+0000 7fe2af7fe700 1 -- 172.21.15.92:0/543945238 learned_addr learned my addr 172.21.15.92:0/543945238 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.426+0000 7fe2a67fc700 1 --2- 172.21.15.92:0/543945238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2b0157300 0x7fe2b01579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.426+0000 7fe2aeffd700 1 --2- 172.21.15.92:0/543945238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe29c0052e0 0x7fe2b014ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.426+0000 7fe2af7fe700 1 -- 172.21.15.92:0/543945238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe29c0052e0 msgr2=0x7fe2b014ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.426+0000 7fe2af7fe700 1 --2- 172.21.15.92:0/543945238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe29c0052e0 0x7fe2b014ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.426+0000 7fe2af7fe700 1 -- 172.21.15.92:0/543945238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2b0157300 msgr2=0x7fe2b01579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.426+0000 7fe2af7fe700 1 --2- 172.21.15.92:0/543945238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2b0157300 0x7fe2b01579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.426+0000 7fe2af7fe700 1 -- 172.21.15.92:0/543945238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b0152810 con 0x7fe2b0156190 2023-10-06T10:43:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.426+0000 7fe2a67fc700 1 --2- 172.21.15.92:0/543945238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2b0157300 0x7fe2b01579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.426+0000 7fe2af7fe700 1 --2- 172.21.15.92:0/543945238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b0156190 0x7fe2b0156630 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00c8740 tx=0x7fe2a8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.427+0000 7fe2a77fe700 1 -- 172.21.15.92:0/543945238 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8001950 con 0x7fe2b0156190 2023-10-06T10:43:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.427+0000 7fe2a77fe700 1 -- 172.21.15.92:0/543945238 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a800cea0 con 0x7fe2b0156190 2023-10-06T10:43:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.427+0000 7fe2b55fa700 1 -- 172.21.15.92:0/543945238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b0151770 con 0x7fe2b0156190 2023-10-06T10:43:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.427+0000 7fe2a77fe700 1 -- 172.21.15.92:0/543945238 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a801e850 con 0x7fe2b0156190 2023-10-06T10:43:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.427+0000 7fe2b55fa700 1 -- 172.21.15.92:0/543945238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b0151d30 con 0x7fe2b0156190 2023-10-06T10:43:55.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.428+0000 7fe2b55fa700 1 -- 172.21.15.92:0/543945238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b00ae7b0 con 0x7fe2b0156190 2023-10-06T10:43:55.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.428+0000 7fe2a77fe700 1 -- 172.21.15.92:0/543945238 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe2a80273b0 con 0x7fe2b0156190 2023-10-06T10:43:55.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.428+0000 7fe2a77fe700 1 --2- 172.21.15.92:0/543945238 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe290066f30 0x7fe2900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:55.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.428+0000 7fe2a77fe700 1 -- 172.21.15.92:0/543945238 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe2a8085690 con 0x7fe2b0156190 2023-10-06T10:43:55.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.428+0000 7fe2aeffd700 1 --2- 172.21.15.92:0/543945238 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe290066f30 0x7fe2900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:55.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.429+0000 7fe2aeffd700 1 --2- 172.21.15.92:0/543945238 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe290066f30 0x7fe2900693e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000abf0 tx=0x7fe2a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:55.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.432+0000 7fe2a77fe700 1 -- 172.21.15.92:0/543945238 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2a8052710 con 0x7fe2b0156190 2023-10-06T10:43:55.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:55.661+0000 7fe2b55fa700 1 -- 172.21.15.92:0/543945238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7fe2b01524f0 con 0x7fe2b0156190 2023-10-06T10:43:56.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.521+0000 7fe2a77fe700 1 -- 172.21.15.92:0/543945238 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v84) v1 ==== 105+0+0 (secure 0 0 0) 0x7fe2a8055d30 con 0x7fe2b0156190 2023-10-06T10:43:56.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.524+0000 7fe2b55fa700 1 -- 172.21.15.92:0/543945238 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe290066f30 msgr2=0x7fe2900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:56.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.524+0000 7fe2b55fa700 1 --2- 172.21.15.92:0/543945238 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe290066f30 0x7fe2900693e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000abf0 tx=0x7fe2a0008040 comp rx=0 tx=0).stop 2023-10-06T10:43:56.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.524+0000 7fe2b55fa700 1 -- 172.21.15.92:0/543945238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b0156190 msgr2=0x7fe2b0156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:56.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.524+0000 7fe2b55fa700 1 --2- 172.21.15.92:0/543945238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b0156190 0x7fe2b0156630 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00c8740 tx=0x7fe2a8015000 comp rx=0 tx=0).stop 2023-10-06T10:43:56.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.524+0000 7fe2b55fa700 1 -- 172.21.15.92:0/543945238 shutdown_connections 2023-10-06T10:43:56.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.524+0000 7fe2b55fa700 1 --2- 172.21.15.92:0/543945238 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe290066f30 0x7fe2900693e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:56.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.524+0000 7fe2b55fa700 1 --2- 172.21.15.92:0/543945238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b0156190 0x7fe2b0156630 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:56.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.524+0000 7fe2b55fa700 1 --2- 172.21.15.92:0/543945238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe29c0052e0 0x7fe2b014ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:56.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.524+0000 7fe2b55fa700 1 --2- 172.21.15.92:0/543945238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2b0157300 0x7fe2b01579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:56.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.524+0000 7fe2b55fa700 1 -- 172.21.15.92:0/543945238 >> 172.21.15.92:0/543945238 conn(0x7fe2b00cc0a0 msgr2=0x7fe2b006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:56.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.524+0000 7fe2b55fa700 1 -- 172.21.15.92:0/543945238 shutdown_connections 2023-10-06T10:43:56.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.524+0000 7fe2b55fa700 1 -- 172.21.15.92:0/543945238 wait complete. 2023-10-06T10:43:56.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:43:56.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.664+0000 7f2692c9b700 1 Processor -- start 2023-10-06T10:43:56.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.664+0000 7f2692c9b700 1 -- start start 2023-10-06T10:43:56.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.665+0000 7f2692c9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f268c159560 0x7f268c0aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:56.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.665+0000 7f2692c9b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f268c0b0760 0x7f268c0b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:56.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.665+0000 7f2692c9b700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f268c0b0c60 con 0x7f268c0b49a0 2023-10-06T10:43:56.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.665+0000 7f2692c9b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f268c0b0f40 con 0x7f268c159560 2023-10-06T10:43:56.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.665+0000 7f2692c9b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268c0b1220 con 0x7f268c0b0760 2023-10-06T10:43:56.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.665+0000 7f2690a37700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f268c159560 0x7f268c0aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:56.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.665+0000 7f2690a37700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f268c159560 0x7f268c0aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36300/0 (socket says 172.21.15.92:36300) 2023-10-06T10:43:56.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.665+0000 7f2690a37700 1 -- 172.21.15.92:0/2604939501 learned_addr learned my addr 172.21.15.92:0/2604939501 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:56.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.665+0000 7f2691238700 1 --2- 172.21.15.92:0/2604939501 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f268c0b0760 0x7f268c0b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:56.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.665+0000 7f2690a37700 1 -- 172.21.15.92:0/2604939501 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f268c0b0760 msgr2=0x7f268c0b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:56.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.665+0000 7f2690a37700 1 --2- 172.21.15.92:0/2604939501 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f268c0b0760 0x7f268c0b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:56.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.665+0000 7f2690a37700 1 -- 172.21.15.92:0/2604939501 >> v1:172.21.15.124:6789/0 conn(0x7f268c0b49a0 legacy=0x7f268c0aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:56.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.665+0000 7f2690a37700 1 -- 172.21.15.92:0/2604939501 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c0b1500 con 0x7f268c159560 2023-10-06T10:43:56.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.665+0000 7f2691238700 1 --2- 172.21.15.92:0/2604939501 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f268c0b0760 0x7f268c0b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:56.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.665+0000 7f2690a37700 1 --2- 172.21.15.92:0/2604939501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c159560 0x7f268c0aa530 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f26800099c0 tx=0x7f2680009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56f1a358c8a5f62 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:56.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.665+0000 7f268b7fe700 1 -- 172.21.15.92:0/2604939501 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2680005c10 con 0x7f268c159560 2023-10-06T10:43:56.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.665+0000 7f268b7fe700 1 -- 172.21.15.92:0/2604939501 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2680005d70 con 0x7f268c159560 2023-10-06T10:43:56.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.665+0000 7f268b7fe700 1 -- 172.21.15.92:0/2604939501 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268000b800 con 0x7f268c159560 2023-10-06T10:43:56.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.666+0000 7f2692c9b700 1 -- 172.21.15.92:0/2604939501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c159560 msgr2=0x7f268c0aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:56.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.666+0000 7f2692c9b700 1 --2- 172.21.15.92:0/2604939501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c159560 0x7f268c0aa530 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f26800099c0 tx=0x7f2680009220 comp rx=0 tx=0).stop 2023-10-06T10:43:56.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.666+0000 7f2692c9b700 1 -- 172.21.15.92:0/2604939501 shutdown_connections 2023-10-06T10:43:56.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.666+0000 7f2692c9b700 1 --2- 172.21.15.92:0/2604939501 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f268c0b0760 0x7f268c0b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:56.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.666+0000 7f2692c9b700 1 --2- 172.21.15.92:0/2604939501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c159560 0x7f268c0aa530 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:56.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.666+0000 7f2692c9b700 1 -- 172.21.15.92:0/2604939501 >> 172.21.15.92:0/2604939501 conn(0x7f268c0997f0 msgr2=0x7f268c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:56.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.666+0000 7f2692c9b700 1 -- 172.21.15.92:0/2604939501 shutdown_connections 2023-10-06T10:43:56.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.666+0000 7f2692c9b700 1 -- 172.21.15.92:0/2604939501 wait complete. 2023-10-06T10:43:56.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.667+0000 7f2692c9b700 1 Processor -- start 2023-10-06T10:43:56.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.667+0000 7f2692c9b700 1 -- start start 2023-10-06T10:43:56.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.668+0000 7f2692c9b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f268c0b0760 0x7f268c0acb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:56.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.668+0000 7f2692c9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c0b49a0 0x7f268c0abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:56.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.668+0000 7f2692c9b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f268c159560 0x7f268c15bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:56.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.668+0000 7f2692c9b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c09ce40 con 0x7f268c0b49a0 2023-10-06T10:43:56.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.668+0000 7f2692c9b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c09c230 con 0x7f268c0b0760 2023-10-06T10:43:56.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.668+0000 7f2692c9b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268c09c510 con 0x7f268c159560 2023-10-06T10:43:56.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.668+0000 7f268bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c0b49a0 0x7f268c0abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:56.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.668+0000 7f268bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c0b49a0 0x7f268c0abbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36310/0 (socket says 172.21.15.92:36310) 2023-10-06T10:43:56.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.668+0000 7f268bfff700 1 -- 172.21.15.92:0/4143038826 learned_addr learned my addr 172.21.15.92:0/4143038826 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:56.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.668+0000 7f2690a37700 1 --2- 172.21.15.92:0/4143038826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f268c0b0760 0x7f268c0acb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:56.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.668+0000 7f2691238700 1 --2- 172.21.15.92:0/4143038826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f268c159560 0x7f268c15bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:56.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.668+0000 7f268bfff700 1 -- 172.21.15.92:0/4143038826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f268c159560 msgr2=0x7f268c15bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:56.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.668+0000 7f268bfff700 1 --2- 172.21.15.92:0/4143038826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f268c159560 0x7f268c15bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:56.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.668+0000 7f268bfff700 1 -- 172.21.15.92:0/4143038826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f268c0b0760 msgr2=0x7f268c0acb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:56.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.668+0000 7f268bfff700 1 --2- 172.21.15.92:0/4143038826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f268c0b0760 0x7f268c0acb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:56.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.668+0000 7f268bfff700 1 -- 172.21.15.92:0/4143038826 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2680009480 con 0x7f268c0b49a0 2023-10-06T10:43:56.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.669+0000 7f268bfff700 1 --2- 172.21.15.92:0/4143038826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c0b49a0 0x7f268c0abbb0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f2678002ee0 tx=0x7f2678007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:56.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.669+0000 7f2688ff9700 1 -- 172.21.15.92:0/4143038826 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2678016070 con 0x7f268c0b49a0 2023-10-06T10:43:56.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.669+0000 7f2688ff9700 1 -- 172.21.15.92:0/4143038826 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2678006820 con 0x7f268c0b49a0 2023-10-06T10:43:56.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.669+0000 7f2692c9b700 1 -- 172.21.15.92:0/4143038826 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268c09c7f0 con 0x7f268c0b49a0 2023-10-06T10:43:56.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.669+0000 7f2688ff9700 1 -- 172.21.15.92:0/4143038826 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2678006be0 con 0x7f268c0b49a0 2023-10-06T10:43:56.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.669+0000 7f2692c9b700 1 -- 172.21.15.92:0/4143038826 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268c0ac890 con 0x7f268c0b49a0 2023-10-06T10:43:56.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.670+0000 7f2692c9b700 1 -- 172.21.15.92:0/4143038826 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f268c064a90 con 0x7f268c0b49a0 2023-10-06T10:43:56.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.670+0000 7f2688ff9700 1 -- 172.21.15.92:0/4143038826 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2678007020 con 0x7f268c0b49a0 2023-10-06T10:43:56.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.670+0000 7f2688ff9700 1 --2- 172.21.15.92:0/4143038826 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2670066f30 0x7f26700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:56.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.671+0000 7f2688ff9700 1 -- 172.21.15.92:0/4143038826 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f26780840a0 con 0x7f268c0b49a0 2023-10-06T10:43:56.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.671+0000 7f2690a37700 1 --2- 172.21.15.92:0/4143038826 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2670066f30 0x7f26700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:56.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.671+0000 7f2690a37700 1 --2- 172.21.15.92:0/4143038826 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2670066f30 0x7f26700693e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2680007fd0 tx=0x7f2680015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:56.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.675+0000 7f2688ff9700 1 -- 172.21.15.92:0/4143038826 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2678050fd0 con 0x7f268c0b49a0 2023-10-06T10:43:56.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.896+0000 7f2692c9b700 1 -- 172.21.15.92:0/4143038826 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f268c09cf80 con 0x7f268c0b49a0 2023-10-06T10:43:56.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.896+0000 7f2688ff9700 1 -- 172.21.15.92:0/4143038826 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4621 (secure 0 0 0) 0x7f26780545f0 con 0x7f268c0b49a0 2023-10-06T10:43:56.897 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:56.898 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":85,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2122441982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2122441982},{"type":"v1","addr":"172.21.15.124:6835","nonce":2122441982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78},{"gid":7686,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3605297930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3605297930},{"type":"v1","addr":"172.21.15.92:6836","nonce":3605297930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:47.850832+0000","modified":"2023-10-06T10:43:56.522104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":7701,"mds_1":7689},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7689":{"gid":7689,"name":"c","rank":1,"incarnation":85,"state":"up:creating","state_seq":1,"addr":"172.21.15.92:6837/4047304116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4047304116},{"type":"v1","addr":"172.21.15.92:6837","nonce":4047304116}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7701":{"gid":7701,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6834/4203118992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":4203118992},{"type":"v1","addr":"172.21.15.124:6834","nonce":4203118992}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T10:43:56.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.899+0000 7f2692c9b700 1 -- 172.21.15.92:0/4143038826 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2670066f30 msgr2=0x7f26700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:56.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.899+0000 7f2692c9b700 1 --2- 172.21.15.92:0/4143038826 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2670066f30 0x7f26700693e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2680007fd0 tx=0x7f2680015000 comp rx=0 tx=0).stop 2023-10-06T10:43:56.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.899+0000 7f2692c9b700 1 -- 172.21.15.92:0/4143038826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c0b49a0 msgr2=0x7f268c0abbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:56.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.899+0000 7f2692c9b700 1 --2- 172.21.15.92:0/4143038826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c0b49a0 0x7f268c0abbb0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f2678002ee0 tx=0x7f2678007f20 comp rx=0 tx=0).stop 2023-10-06T10:43:56.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.899+0000 7f2692c9b700 1 -- 172.21.15.92:0/4143038826 shutdown_connections 2023-10-06T10:43:56.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.899+0000 7f2692c9b700 1 --2- 172.21.15.92:0/4143038826 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2670066f30 0x7f26700693e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:56.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.899+0000 7f2692c9b700 1 --2- 172.21.15.92:0/4143038826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f268c159560 0x7f268c15bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:56.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.899+0000 7f2692c9b700 1 --2- 172.21.15.92:0/4143038826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f268c0b0760 0x7f268c0acb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:56.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.899+0000 7f2692c9b700 1 --2- 172.21.15.92:0/4143038826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c0b49a0 0x7f268c0abbb0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:56.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.899+0000 7f2692c9b700 1 -- 172.21.15.92:0/4143038826 >> 172.21.15.92:0/4143038826 conn(0x7f268c0997f0 msgr2=0x7f268c0aa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:56.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.899+0000 7f2692c9b700 1 -- 172.21.15.92:0/4143038826 shutdown_connections 2023-10-06T10:43:56.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:56.899+0000 7f2692c9b700 1 -- 172.21.15.92:0/4143038826 wait complete. 2023-10-06T10:43:56.901 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 85 2023-10-06T10:43:57.314 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T10:43:47.850832+0000', 'modified': '2023-10-06T10:43:56.522104+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 7701, 'mds_1': 7689}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7689': {'gid': 7689, 'name': 'c', 'rank': 1, 'incarnation': 85, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.92:6837/4047304116', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 4047304116}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 4047304116}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_7701': {'gid': 7701, 'name': 'b', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.124:6834/4203118992', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6832', 'nonce': 4203118992}, {'type': 'v1', 'addr': '172.21.15.124:6834', 'nonce': 4203118992}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T10:43:57.314 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7689:up:creating 2023-10-06T10:43:57.766 INFO:tasks.ceph.mds.b.smithi124.stderr:2023-10-06T10:43:57.765+0000 7f557baa9700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-06T10:43:58.315 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:43:58.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.436+0000 7fc6c5564700 1 Processor -- start 2023-10-06T10:43:58.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.436+0000 7fc6c5564700 1 -- start start 2023-10-06T10:43:58.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.436+0000 7fc6c5564700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc6c0159560 0x7fc6c00aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:58.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.436+0000 7fc6c5564700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c00b0c60 con 0x7fc6c00b49a0 2023-10-06T10:43:58.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.436+0000 7fc6c5564700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c00b0f40 con 0x7fc6c00b0760 2023-10-06T10:43:58.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.436+0000 7fc6c5564700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c00b1220 con 0x7fc6c0159560 2023-10-06T10:43:58.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.437+0000 7fc6b67fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc6c00b49a0 0x7fc6c00aaa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57274/0 (socket says 172.21.15.92:57274) 2023-10-06T10:43:58.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.437+0000 7fc6beffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc6c0159560 0x7fc6c00aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:58.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.437+0000 7fc6b67fc700 1 -- 172.21.15.92:0/1171554697 learned_addr learned my addr 172.21.15.92:0/1171554697 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:58.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.437+0000 7fc6beffd700 1 -- 172.21.15.92:0/1171554697 >> v1:172.21.15.124:6789/0 conn(0x7fc6c00b0760 legacy=0x7fc6c00b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:58.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.437+0000 7fc6beffd700 1 -- 172.21.15.92:0/1171554697 >> v1:172.21.15.92:6789/0 conn(0x7fc6c00b49a0 legacy=0x7fc6c00aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:58.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.437+0000 7fc6be7fc700 1 -- 172.21.15.92:0/1171554697 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3281489350 0 0) 0x7fc6c00b0c60 con 0x7fc6c00b49a0 2023-10-06T10:43:58.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.437+0000 7fc6beffd700 1 -- 172.21.15.92:0/1171554697 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c00b1500 con 0x7fc6c0159560 2023-10-06T10:43:58.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.437+0000 7fc6beffd700 1 --2- 172.21.15.92:0/1171554697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c0159560 0x7fc6c00aa530 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009900 tx=0x7fc6b0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d25f4c53972082 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:58.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.437+0000 7fc6be7fc700 1 -- 172.21.15.92:0/1171554697 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0009ea0 con 0x7fc6c0159560 2023-10-06T10:43:58.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.437+0000 7fc6be7fc700 1 -- 172.21.15.92:0/1171554697 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b0005c10 con 0x7fc6c0159560 2023-10-06T10:43:58.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.437+0000 7fc6be7fc700 1 -- 172.21.15.92:0/1171554697 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b000b5d0 con 0x7fc6c0159560 2023-10-06T10:43:58.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.438+0000 7fc6c5564700 1 -- 172.21.15.92:0/1171554697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c0159560 msgr2=0x7fc6c00aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:58.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.438+0000 7fc6c5564700 1 --2- 172.21.15.92:0/1171554697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c0159560 0x7fc6c00aa530 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009900 tx=0x7fc6b0009120 comp rx=0 tx=0).stop 2023-10-06T10:43:58.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.438+0000 7fc6c5564700 1 -- 172.21.15.92:0/1171554697 shutdown_connections 2023-10-06T10:43:58.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.438+0000 7fc6c5564700 1 --2- 172.21.15.92:0/1171554697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c0159560 0x7fc6c00aa530 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:58.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.438+0000 7fc6c5564700 1 -- 172.21.15.92:0/1171554697 >> 172.21.15.92:0/1171554697 conn(0x7fc6c00997f0 msgr2=0x7fc6c006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:58.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.438+0000 7fc6c5564700 1 -- 172.21.15.92:0/1171554697 shutdown_connections 2023-10-06T10:43:58.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.438+0000 7fc6c5564700 1 -- 172.21.15.92:0/1171554697 wait complete. 2023-10-06T10:43:58.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.439+0000 7fc6c5564700 1 Processor -- start 2023-10-06T10:43:58.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.439+0000 7fc6c5564700 1 -- start start 2023-10-06T10:43:58.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.440+0000 7fc6c5564700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c00b0760 0x7fc6c016a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:58.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.440+0000 7fc6c5564700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6c00b49a0 0x7fc6c015bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:58.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.440+0000 7fc6c5564700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6c0159560 0x7fc6c015c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:58.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.440+0000 7fc6c5564700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c016ddc0 con 0x7fc6c00b0760 2023-10-06T10:43:58.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.440+0000 7fc6c5564700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c00abee0 con 0x7fc6c0159560 2023-10-06T10:43:58.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.440+0000 7fc6c5564700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c016d490 con 0x7fc6c00b49a0 2023-10-06T10:43:58.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.440+0000 7fc6beffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c00b0760 0x7fc6c016a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:58.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.440+0000 7fc6beffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c00b0760 0x7fc6c016a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36320/0 (socket says 172.21.15.92:36320) 2023-10-06T10:43:58.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.440+0000 7fc6beffd700 1 -- 172.21.15.92:0/3832353976 learned_addr learned my addr 172.21.15.92:0/3832353976 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:58.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.440+0000 7fc6b67fc700 1 --2- 172.21.15.92:0/3832353976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6c00b49a0 0x7fc6c015bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:58.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.440+0000 7fc6bf7fe700 1 --2- 172.21.15.92:0/3832353976 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6c0159560 0x7fc6c015c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:58.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.440+0000 7fc6beffd700 1 -- 172.21.15.92:0/3832353976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6c00b49a0 msgr2=0x7fc6c015bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:58.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.440+0000 7fc6beffd700 1 --2- 172.21.15.92:0/3832353976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6c00b49a0 0x7fc6c015bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:58.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.440+0000 7fc6beffd700 1 -- 172.21.15.92:0/3832353976 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6c0159560 msgr2=0x7fc6c015c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:58.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.440+0000 7fc6beffd700 1 --2- 172.21.15.92:0/3832353976 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6c0159560 0x7fc6c015c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:58.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.440+0000 7fc6beffd700 1 -- 172.21.15.92:0/3832353976 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b00093c0 con 0x7fc6c00b0760 2023-10-06T10:43:58.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.440+0000 7fc6beffd700 1 --2- 172.21.15.92:0/3832353976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c00b0760 0x7fc6c016a4a0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0003080 tx=0x7fc6b0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:58.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.440+0000 7fc6b77fe700 1 -- 172.21.15.92:0/3832353976 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0021070 con 0x7fc6c00b0760 2023-10-06T10:43:58.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.440+0000 7fc6b77fe700 1 -- 172.21.15.92:0/3832353976 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b00065f0 con 0x7fc6c00b0760 2023-10-06T10:43:58.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.440+0000 7fc6b77fe700 1 -- 172.21.15.92:0/3832353976 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b001d430 con 0x7fc6c00b0760 2023-10-06T10:43:58.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.440+0000 7fc6c5564700 1 -- 172.21.15.92:0/3832353976 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c016d770 con 0x7fc6c00b0760 2023-10-06T10:43:58.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.441+0000 7fc6c5564700 1 -- 172.21.15.92:0/3832353976 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c016a9e0 con 0x7fc6c00b0760 2023-10-06T10:43:58.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.441+0000 7fc6c5564700 1 -- 172.21.15.92:0/3832353976 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c016ac30 con 0x7fc6c00b0760 2023-10-06T10:43:58.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.442+0000 7fc6b77fe700 1 -- 172.21.15.92:0/3832353976 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc6b001d820 con 0x7fc6c00b0760 2023-10-06T10:43:58.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.442+0000 7fc6b77fe700 1 --2- 172.21.15.92:0/3832353976 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc6a0066f30 0x7fc6a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:58.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.442+0000 7fc6b77fe700 1 -- 172.21.15.92:0/3832353976 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc6b0086250 con 0x7fc6c00b0760 2023-10-06T10:43:58.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.442+0000 7fc6b67fc700 1 --2- 172.21.15.92:0/3832353976 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc6a0066f30 0x7fc6a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:58.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.443+0000 7fc6b67fc700 1 --2- 172.21.15.92:0/3832353976 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc6a0066f30 0x7fc6a00693e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc6c00afdb0 tx=0x7fc6a8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:58.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.447+0000 7fc6b77fe700 1 -- 172.21.15.92:0/3832353976 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc6b0053180 con 0x7fc6c00b0760 2023-10-06T10:43:58.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.676+0000 7fc6c5564700 1 -- 172.21.15.92:0/3832353976 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6c016e0a0 con 0x7fc6c00b0760 2023-10-06T10:43:58.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.676+0000 7fc6b77fe700 1 -- 172.21.15.92:0/3832353976 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4633 (secure 0 0 0) 0x7fc6b00567a0 con 0x7fc6c00b0760 2023-10-06T10:43:58.678 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:58.678 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":88,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2122441982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2122441982},{"type":"v1","addr":"172.21.15.124:6835","nonce":2122441982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:47.850832+0000","modified":"2023-10-06T10:43:58.531567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7701,"mds_1":7689,"mds_2":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6836/3605297930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3605297930},{"type":"v1","addr":"172.21.15.92:6836","nonce":3605297930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7689":{"gid":7689,"name":"c","rank":1,"incarnation":85,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/4047304116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4047304116},{"type":"v1","addr":"172.21.15.92:6837","nonce":4047304116}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7701":{"gid":7701,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6834/4203118992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":4203118992},{"type":"v1","addr":"172.21.15.124:6834","nonce":4203118992}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T10:43:58.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.678+0000 7fc6c5564700 1 -- 172.21.15.92:0/3832353976 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc6a0066f30 msgr2=0x7fc6a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:58.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.678+0000 7fc6c5564700 1 --2- 172.21.15.92:0/3832353976 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc6a0066f30 0x7fc6a00693e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc6c00afdb0 tx=0x7fc6a8009040 comp rx=0 tx=0).stop 2023-10-06T10:43:58.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.679+0000 7fc6c5564700 1 -- 172.21.15.92:0/3832353976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c00b0760 msgr2=0x7fc6c016a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:58.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.679+0000 7fc6c5564700 1 --2- 172.21.15.92:0/3832353976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c00b0760 0x7fc6c016a4a0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0003080 tx=0x7fc6b0015000 comp rx=0 tx=0).stop 2023-10-06T10:43:58.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.679+0000 7fc6c5564700 1 -- 172.21.15.92:0/3832353976 shutdown_connections 2023-10-06T10:43:58.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.679+0000 7fc6c5564700 1 --2- 172.21.15.92:0/3832353976 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc6a0066f30 0x7fc6a00693e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:58.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.679+0000 7fc6c5564700 1 --2- 172.21.15.92:0/3832353976 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6c0159560 0x7fc6c015c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:58.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.679+0000 7fc6c5564700 1 --2- 172.21.15.92:0/3832353976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c00b0760 0x7fc6c016a4a0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:58.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.679+0000 7fc6c5564700 1 --2- 172.21.15.92:0/3832353976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6c00b49a0 0x7fc6c015bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:58.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.679+0000 7fc6c5564700 1 -- 172.21.15.92:0/3832353976 >> 172.21.15.92:0/3832353976 conn(0x7fc6c00997f0 msgr2=0x7fc6c00699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:58.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.679+0000 7fc6c5564700 1 -- 172.21.15.92:0/3832353976 shutdown_connections 2023-10-06T10:43:58.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:58.679+0000 7fc6c5564700 1 -- 172.21.15.92:0/3832353976 wait complete. 2023-10-06T10:43:58.682 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 88 2023-10-06T10:43:59.095 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T10:43:47.850832+0000', 'modified': '2023-10-06T10:43:58.531567+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 7701, 'mds_1': 7689, 'mds_2': 7686}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7686': {'gid': 7686, 'name': 'a', 'rank': 2, 'incarnation': 87, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.92:6836/3605297930', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 3605297930}, {'type': 'v1', 'addr': '172.21.15.92:6836', 'nonce': 3605297930}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_7689': {'gid': 7689, 'name': 'c', 'rank': 1, 'incarnation': 85, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.92:6837/4047304116', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 4047304116}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 4047304116}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_7701': {'gid': 7701, 'name': 'b', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.124:6834/4203118992', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6832', 'nonce': 4203118992}, {'type': 'v1', 'addr': '172.21.15.124:6834', 'nonce': 4203118992}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T10:43:59.096 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-10-06T10:43:59.096 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T10:43:59.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.213+0000 7f69b6a3e700 1 Processor -- start 2023-10-06T10:43:59.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.213+0000 7f69b6a3e700 1 -- start start 2023-10-06T10:43:59.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.214+0000 7f69b6a3e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f69b01570f0 0x7f69b0069ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.214+0000 7f69b6a3e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69b006e8b0 0x7f69b00685c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.214+0000 7f69b6a3e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b0068f40 con 0x7f69b0068b00 2023-10-06T10:43:59.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.214+0000 7f69b6a3e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b00cad20 con 0x7f69b01570f0 2023-10-06T10:43:59.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.214+0000 7f69b6a3e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b00cc4f0 con 0x7f69b006e8b0 2023-10-06T10:43:59.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.214+0000 7f69b5a3c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f69b01570f0 0x7f69b0069ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.214+0000 7f69b5a3c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f69b01570f0 0x7f69b0069ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36336/0 (socket says 172.21.15.92:36336) 2023-10-06T10:43:59.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.214+0000 7f69b5a3c700 1 -- 172.21.15.92:0/866160746 learned_addr learned my addr 172.21.15.92:0/866160746 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:59.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.215+0000 7f69b523b700 1 --2- 172.21.15.92:0/866160746 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69b006e8b0 0x7f69b00685c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.215+0000 7f69b5a3c700 1 -- 172.21.15.92:0/866160746 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69b006e8b0 msgr2=0x7f69b00685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.215+0000 7f69b5a3c700 1 --2- 172.21.15.92:0/866160746 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69b006e8b0 0x7f69b00685c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.215+0000 7f69b5a3c700 1 -- 172.21.15.92:0/866160746 >> v1:172.21.15.124:6789/0 conn(0x7f69b0068b00 legacy=0x7f69b015ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.215+0000 7f69b5a3c700 1 -- 172.21.15.92:0/866160746 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b006a560 con 0x7f69b01570f0 2023-10-06T10:43:59.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.215+0000 7f69b5a3c700 1 --2- 172.21.15.92:0/866160746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b01570f0 0x7f69b0069ab0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0099b0 tx=0x7f69ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf750c60bd2e089e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.215+0000 7f69b4a3a700 1 -- 172.21.15.92:0/866160746 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac00b820 con 0x7f69b01570f0 2023-10-06T10:43:59.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.215+0000 7f69b4a3a700 1 -- 172.21.15.92:0/866160746 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ac00b980 con 0x7f69b01570f0 2023-10-06T10:43:59.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.215+0000 7f69b4a3a700 1 -- 172.21.15.92:0/866160746 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac00bd40 con 0x7f69b01570f0 2023-10-06T10:43:59.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.215+0000 7f69b6a3e700 1 -- 172.21.15.92:0/866160746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b01570f0 msgr2=0x7f69b0069ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.215+0000 7f69b6a3e700 1 --2- 172.21.15.92:0/866160746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b01570f0 0x7f69b0069ab0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0099b0 tx=0x7f69ac009250 comp rx=0 tx=0).stop 2023-10-06T10:43:59.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.216+0000 7f69b6a3e700 1 -- 172.21.15.92:0/866160746 shutdown_connections 2023-10-06T10:43:59.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.216+0000 7f69b6a3e700 1 --2- 172.21.15.92:0/866160746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b01570f0 0x7f69b0069ab0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.216+0000 7f69b6a3e700 1 --2- 172.21.15.92:0/866160746 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69b006e8b0 0x7f69b00685c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.216+0000 7f69b6a3e700 1 -- 172.21.15.92:0/866160746 >> 172.21.15.92:0/866160746 conn(0x7f69b00c9c10 msgr2=0x7f69b006dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:59.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.216+0000 7f69b6a3e700 1 -- 172.21.15.92:0/866160746 shutdown_connections 2023-10-06T10:43:59.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.216+0000 7f69b6a3e700 1 -- 172.21.15.92:0/866160746 wait complete. 2023-10-06T10:43:59.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.217+0000 7f69b6a3e700 1 Processor -- start 2023-10-06T10:43:59.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.217+0000 7f69b6a3e700 1 -- start start 2023-10-06T10:43:59.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.217+0000 7f69b6a3e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69b0068b00 0x7f69b014b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.217+0000 7f69b6a3e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69b006e8b0 0x7f69b00add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.217+0000 7f69b6a3e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b01570f0 0x7f69b0151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.217+0000 7f69b6a3e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b0169f30 con 0x7f69b01570f0 2023-10-06T10:43:59.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.217+0000 7f69b6a3e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b0169320 con 0x7f69b0068b00 2023-10-06T10:43:59.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.217+0000 7f69b6a3e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b0169600 con 0x7f69b006e8b0 2023-10-06T10:43:59.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.218+0000 7f69b623d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b01570f0 0x7f69b0151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.218+0000 7f69b623d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b01570f0 0x7f69b0151fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36348/0 (socket says 172.21.15.92:36348) 2023-10-06T10:43:59.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.218+0000 7f69b623d700 1 -- 172.21.15.92:0/1580009231 learned_addr learned my addr 172.21.15.92:0/1580009231 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:59.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.218+0000 7f69b5a3c700 1 --2- 172.21.15.92:0/1580009231 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69b0068b00 0x7f69b014b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.218+0000 7f69b523b700 1 --2- 172.21.15.92:0/1580009231 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69b006e8b0 0x7f69b00add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.218+0000 7f69b623d700 1 -- 172.21.15.92:0/1580009231 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69b006e8b0 msgr2=0x7f69b00add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.218+0000 7f69b623d700 1 --2- 172.21.15.92:0/1580009231 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69b006e8b0 0x7f69b00add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.218+0000 7f69b623d700 1 -- 172.21.15.92:0/1580009231 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69b0068b00 msgr2=0x7f69b014b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.218+0000 7f69b623d700 1 --2- 172.21.15.92:0/1580009231 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69b0068b00 0x7f69b014b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.218+0000 7f69b623d700 1 -- 172.21.15.92:0/1580009231 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ac0094f0 con 0x7f69b01570f0 2023-10-06T10:43:59.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.218+0000 7f69b523b700 1 --2- 172.21.15.92:0/1580009231 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69b006e8b0 0x7f69b00add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:59.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.218+0000 7f69b623d700 1 --2- 172.21.15.92:0/1580009231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b01570f0 0x7f69b0151fe0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f69a800c930 tx=0x7f69a80026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.218+0000 7f69a5ffb700 1 -- 172.21.15.92:0/1580009231 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a800ce70 con 0x7f69b01570f0 2023-10-06T10:43:59.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.218+0000 7f69a5ffb700 1 -- 172.21.15.92:0/1580009231 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a800a7e0 con 0x7f69b01570f0 2023-10-06T10:43:59.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.218+0000 7f69a5ffb700 1 -- 172.21.15.92:0/1580009231 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a800aba0 con 0x7f69b01570f0 2023-10-06T10:43:59.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.218+0000 7f69b6a3e700 1 -- 172.21.15.92:0/1580009231 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b01698e0 con 0x7f69b01570f0 2023-10-06T10:43:59.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.218+0000 7f69b6a3e700 1 -- 172.21.15.92:0/1580009231 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b00ae2a0 con 0x7f69b01570f0 2023-10-06T10:43:59.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.220+0000 7f69a5ffb700 1 -- 172.21.15.92:0/1580009231 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f69a8012420 con 0x7f69b01570f0 2023-10-06T10:43:59.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.220+0000 7f69a5ffb700 1 --2- 172.21.15.92:0/1580009231 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6994066f30 0x7f69940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.220+0000 7f69a5ffb700 1 -- 172.21.15.92:0/1580009231 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f69a8084c10 con 0x7f69b01570f0 2023-10-06T10:43:59.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.220+0000 7f69b5a3c700 1 --2- 172.21.15.92:0/1580009231 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6994066f30 0x7f69940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.220+0000 7f69b5a3c700 1 --2- 172.21.15.92:0/1580009231 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6994066f30 0x7f69940693e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0099b0 tx=0x7f69ac016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.222+0000 7f69c0f98b80 1 Processor -- start 2023-10-06T10:43:59.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.222+0000 7f69c0f98b80 1 -- start start 2023-10-06T10:43:59.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.223+0000 7f69c0f98b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567a8a6d950 0x5567a8a6dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.223+0000 7f69c0f98b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567a8ab7bc0 0x5567a8a96920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.223+0000 7f69c0f98b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a96e60 0x5567a8a69990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.223+0000 7f69c0f98b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567a89edc80 con 0x5567a8a96e60 2023-10-06T10:43:59.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.223+0000 7f69c0f98b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567a89c4a70 con 0x5567a8a6d950 2023-10-06T10:43:59.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.223+0000 7f69c0f98b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567a89c2270 con 0x5567a8ab7bc0 2023-10-06T10:43:59.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.223+0000 7f69b623d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a96e60 0x5567a8a69990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.223+0000 7f69b623d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a96e60 0x5567a8a69990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36356/0 (socket says 172.21.15.92:36356) 2023-10-06T10:43:59.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.223+0000 7f69b623d700 1 -- 172.21.15.92:0/891034436 learned_addr learned my addr 172.21.15.92:0/891034436 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:59.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.223+0000 7f69b5a3c700 1 --2- 172.21.15.92:0/891034436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567a8a6d950 0x5567a8a6dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.223+0000 7f69b523b700 1 --2- 172.21.15.92:0/891034436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567a8ab7bc0 0x5567a8a96920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.223+0000 7f69b623d700 1 -- 172.21.15.92:0/891034436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567a8ab7bc0 msgr2=0x5567a8a96920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.223+0000 7f69b623d700 1 --2- 172.21.15.92:0/891034436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567a8ab7bc0 0x5567a8a96920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.223+0000 7f69b623d700 1 -- 172.21.15.92:0/891034436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567a8a6d950 msgr2=0x5567a8a6dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.223+0000 7f69b623d700 1 --2- 172.21.15.92:0/891034436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567a8a6d950 0x5567a8a6dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.223+0000 7f69b623d700 1 -- 172.21.15.92:0/891034436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567a8858270 con 0x5567a8a96e60 2023-10-06T10:43:59.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.223+0000 7f69b5a3c700 1 --2- 172.21.15.92:0/891034436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567a8a6d950 0x5567a8a6dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:59.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.223+0000 7f69b523b700 1 --2- 172.21.15.92:0/891034436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567a8ab7bc0 0x5567a8a96920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:59.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.223+0000 7f69b623d700 1 --2- 172.21.15.92:0/891034436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a96e60 0x5567a8a69990 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f69a8051e50 tx=0x7f69a8082df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01b96eb8636e468 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.223+0000 7f6992ffd700 1 -- 172.21.15.92:0/891034436 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a8053a90 con 0x5567a8a96e60 2023-10-06T10:43:59.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.223+0000 7f6992ffd700 1 -- 172.21.15.92:0/891034436 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a8053cf0 con 0x5567a8a96e60 2023-10-06T10:43:59.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.223+0000 7f6992ffd700 1 -- 172.21.15.92:0/891034436 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a8055dc0 con 0x5567a8a96e60 2023-10-06T10:43:59.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.224+0000 7f69c0f98b80 1 -- 172.21.15.92:0/891034436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a96e60 msgr2=0x5567a8a69990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.224+0000 7f69c0f98b80 1 --2- 172.21.15.92:0/891034436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a96e60 0x5567a8a69990 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f69a8051e50 tx=0x7f69a8082df0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.224+0000 7f69c0f98b80 1 -- 172.21.15.92:0/891034436 shutdown_connections 2023-10-06T10:43:59.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.224+0000 7f69c0f98b80 1 --2- 172.21.15.92:0/891034436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a96e60 0x5567a8a69990 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.224+0000 7f69c0f98b80 1 --2- 172.21.15.92:0/891034436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567a8a6d950 0x5567a8a6dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.224+0000 7f69c0f98b80 1 --2- 172.21.15.92:0/891034436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567a8ab7bc0 0x5567a8a96920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.224+0000 7f69c0f98b80 1 -- 172.21.15.92:0/891034436 >> 172.21.15.92:0/891034436 conn(0x5567a8a6f270 msgr2=0x5567a8a6f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:59.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.224+0000 7f69c0f98b80 1 -- 172.21.15.92:0/891034436 shutdown_connections 2023-10-06T10:43:59.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.224+0000 7f69c0f98b80 1 -- 172.21.15.92:0/891034436 wait complete. 2023-10-06T10:43:59.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.224+0000 7f69c0f98b80 1 -- start start 2023-10-06T10:43:59.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.225+0000 7f69c0f98b80 1 Processor -- start 2023-10-06T10:43:59.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.225+0000 7f69c0f98b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567a8a6d950 0x5567a8aaf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.225+0000 7f69c0f98b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a96e60 0x5567a8a95660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.225+0000 7f69c0f98b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567a8ab7bc0 0x5567a8aacb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.225+0000 7f69c0f98b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567a89be060 con 0x5567a8a96e60 2023-10-06T10:43:59.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.225+0000 7f69c0f98b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567a8969890 con 0x5567a8ab7bc0 2023-10-06T10:43:59.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.225+0000 7f69c0f98b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567a8984ea0 con 0x5567a8a6d950 2023-10-06T10:43:59.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.226+0000 7f69c0f98b80 10 client.0 _ll_register_callbacks cb 0x5567a885af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:59.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.226+0000 7f69b523b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a96e60 0x5567a8a95660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.226+0000 7f69b523b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a96e60 0x5567a8a95660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36362/0 (socket says 172.21.15.92:36362) 2023-10-06T10:43:59.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.226+0000 7f69b523b700 1 -- 172.21.15.92:0/4113122122 learned_addr learned my addr 172.21.15.92:0/4113122122 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:59.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.226+0000 7f69b623d700 1 --2- 172.21.15.92:0/4113122122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567a8ab7bc0 0x5567a8aacb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.226+0000 7f69b5a3c700 1 --2- 172.21.15.92:0/4113122122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567a8a6d950 0x5567a8aaf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.226+0000 7f69b523b700 1 -- 172.21.15.92:0/4113122122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567a8a6d950 msgr2=0x5567a8aaf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.226+0000 7f69b523b700 1 --2- 172.21.15.92:0/4113122122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567a8a6d950 0x5567a8aaf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.226+0000 7f69b523b700 1 -- 172.21.15.92:0/4113122122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567a8ab7bc0 msgr2=0x5567a8aacb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.226+0000 7f69b523b700 1 --2- 172.21.15.92:0/4113122122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567a8ab7bc0 0x5567a8aacb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.226+0000 7f69b523b700 1 -- 172.21.15.92:0/4113122122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b00ae2a0 con 0x5567a8a96e60 2023-10-06T10:43:59.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.226+0000 7f69b523b700 1 --2- 172.21.15.92:0/4113122122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a96e60 0x5567a8a95660 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f69a000e5b0 tx=0x7f69a0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f483d049a1b39eb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.226+0000 7f698bfff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:59.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.226+0000 7f698bfff700 1 -- 172.21.15.92:0/4113122122 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a000ee80 con 0x5567a8a96e60 2023-10-06T10:43:59.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.226+0000 7f69c0f98b80 1 -- 172.21.15.92:0/4113122122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567a8a62690 con 0x5567a8a96e60 2023-10-06T10:43:59.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.226+0000 7f698bfff700 1 -- 172.21.15.92:0/4113122122 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f69a000b030 con 0x5567a8a96e60 2023-10-06T10:43:59.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.226+0000 7f698bfff700 1 -- 172.21.15.92:0/4113122122 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69a0012850 con 0x5567a8a96e60 2023-10-06T10:43:59.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.226+0000 7f69c0f98b80 10 client.7899 fetch_fsmap learned FSMap version 88 2023-10-06T10:43:59.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.226+0000 7f69c0f98b80 1 -- 172.21.15.92:0/4113122122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x5567a87fafe0 con 0x5567a8a96e60 2023-10-06T10:43:59.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.226+0000 7f698bfff700 1 -- 172.21.15.92:0/4113122122 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5567a8a62690 con 0x5567a8a96e60 2023-10-06T10:43:59.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.227+0000 7f698bfff700 1 -- 172.21.15.92:0/4113122122 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 88) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f69a00129b0 con 0x5567a8a96e60 2023-10-06T10:43:59.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.227+0000 7f69c0f98b80 10 client.7899 fetch_fsmap finished waiting for FSMap version 88 2023-10-06T10:43:59.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.227+0000 7f69c0f98b80 10 client.7899 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T10:43:59.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.227+0000 7f69c0f98b80 20 client.7899 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:59.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.227+0000 7f69c0f98b80 1 --2- 172.21.15.92:0/4113122122 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x5567a8aadaf0 0x5567a8aadec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.227+0000 7f69c0f98b80 4 client.7899 mds_command: new command op to 7686 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:43:59.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.227+0000 7f69c0f98b80 1 -- 172.21.15.92:0/4113122122 --> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567a8a3f5c0 con 0x5567a8aadaf0 2023-10-06T10:43:59.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.227+0000 7f69b5a3c700 1 --2- 172.21.15.92:0/4113122122 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x5567a8aadaf0 0x5567a8aadec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.227+0000 7f698bfff700 10 client.7899 ms_handle_connect on v2:172.21.15.92:6834/3605297930 2023-10-06T10:43:59.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.227+0000 7f69b5a3c700 1 --2- 172.21.15.92:0/4113122122 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x5567a8aadaf0 0x5567a8aadec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99e413e0321c09cb server_cookie=9e7f70958775bb7c in_seq=0 out_seq=0 2023-10-06T10:43:59.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.228+0000 7f698bfff700 1 -- 172.21.15.92:0/4113122122 <== mds.2 v2:172.21.15.92:6834/3605297930 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5567a8a3f5c0 con 0x5567a8aadaf0 2023-10-06T10:43:59.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.228+0000 7f698bfff700 10 client.7899 handle_command_reply: tid=0 2023-10-06T10:43:59.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.228+0000 7f69c0f98b80 1 client.7899 shutdown 2023-10-06T10:43:59.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.228+0000 7f69c0f98b80 1 -- 172.21.15.92:0/4113122122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a96e60 msgr2=0x5567a8a95660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.228+0000 7f69c0f98b80 1 --2- 172.21.15.92:0/4113122122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a96e60 0x5567a8a95660 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f69a000e5b0 tx=0x7f69a0008040 comp rx=0 tx=0).stop 2023-10-06T10:43:59.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.228+0000 7f69c0f98b80 1 -- 172.21.15.92:0/4113122122 shutdown_connections 2023-10-06T10:43:59.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.228+0000 7f69c0f98b80 1 --2- 172.21.15.92:0/4113122122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567a8ab7bc0 0x5567a8aacb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.228+0000 7f69c0f98b80 1 --2- 172.21.15.92:0/4113122122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567a8a6d950 0x5567a8aaf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.228+0000 7f69c0f98b80 1 --2- 172.21.15.92:0/4113122122 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x5567a8aadaf0 0x5567a8aadec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.228+0000 7f69c0f98b80 1 --2- 172.21.15.92:0/4113122122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a96e60 0x5567a8a95660 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.228+0000 7f698bfff700 0 client.7899 ms_handle_reset on v2:172.21.15.92:6834/3605297930 2023-10-06T10:43:59.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.228+0000 7f69c0f98b80 1 -- 172.21.15.92:0/4113122122 >> 172.21.15.92:0/4113122122 conn(0x5567a8a6f270 msgr2=0x5567a8a641a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:59.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.228+0000 7f69c0f98b80 1 -- 172.21.15.92:0/4113122122 shutdown_connections 2023-10-06T10:43:59.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.228+0000 7f69c0f98b80 1 -- 172.21.15.92:0/4113122122 wait complete. 2023-10-06T10:43:59.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.228+0000 7f69c0f98b80 20 client.7899 trim_cache size 0 max 16384 2023-10-06T10:43:59.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.244+0000 7f69c0f98b80 1 Processor -- start 2023-10-06T10:43:59.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.245+0000 7f69c0f98b80 1 -- start start 2023-10-06T10:43:59.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.245+0000 7f69c0f98b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567a8a6d950 0x5567a8ac3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.245+0000 7f69c0f98b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a96e60 0x5567a8ac4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.245+0000 7f69c0f98b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567a8ab7bc0 0x5567a8a641a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.245+0000 7f69c0f98b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567a8840e30 con 0x5567a8a96e60 2023-10-06T10:43:59.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.245+0000 7f69c0f98b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567a8aaf4a0 con 0x5567a8a6d950 2023-10-06T10:43:59.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.245+0000 7f69c0f98b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567a8a4d0d0 con 0x5567a8ab7bc0 2023-10-06T10:43:59.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.245+0000 7f69b523b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a96e60 0x5567a8ac4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.245+0000 7f69b523b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a96e60 0x5567a8ac4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36372/0 (socket says 172.21.15.92:36372) 2023-10-06T10:43:59.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.245+0000 7f69b523b700 1 -- 172.21.15.92:0/4149134933 learned_addr learned my addr 172.21.15.92:0/4149134933 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:59.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.246+0000 7f69b523b700 1 -- 172.21.15.92:0/4149134933 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567a8ab7bc0 msgr2=0x5567a8a641a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.246+0000 7f69b523b700 1 --2- 172.21.15.92:0/4149134933 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567a8ab7bc0 0x5567a8a641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.246+0000 7f69b523b700 1 -- 172.21.15.92:0/4149134933 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567a8a6d950 msgr2=0x5567a8ac3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.246+0000 7f69b523b700 1 --2- 172.21.15.92:0/4149134933 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567a8a6d950 0x5567a8ac3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.246+0000 7f69b523b700 1 -- 172.21.15.92:0/4149134933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567a898c780 con 0x5567a8a96e60 2023-10-06T10:43:59.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.246+0000 7f69b523b700 1 --2- 172.21.15.92:0/4149134933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a96e60 0x5567a8ac4030 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f69a000eca0 tx=0x7f69a001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c48c3d2a961356 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.246+0000 7f69937fe700 1 -- 172.21.15.92:0/4149134933 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a0012e20 con 0x5567a8a96e60 2023-10-06T10:43:59.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.246+0000 7f69937fe700 1 -- 172.21.15.92:0/4149134933 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a0009470 con 0x5567a8a96e60 2023-10-06T10:43:59.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.246+0000 7f69937fe700 1 -- 172.21.15.92:0/4149134933 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a0000e10 con 0x5567a8a96e60 2023-10-06T10:43:59.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.246+0000 7f69c0f98b80 1 -- 172.21.15.92:0/4149134933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a96e60 msgr2=0x5567a8ac4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.246+0000 7f69c0f98b80 1 --2- 172.21.15.92:0/4149134933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a96e60 0x5567a8ac4030 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f69a000eca0 tx=0x7f69a001f000 comp rx=0 tx=0).stop 2023-10-06T10:43:59.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.246+0000 7f69c0f98b80 1 -- 172.21.15.92:0/4149134933 shutdown_connections 2023-10-06T10:43:59.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.246+0000 7f69c0f98b80 1 --2- 172.21.15.92:0/4149134933 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567a8ab7bc0 0x5567a8a641a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.246+0000 7f69c0f98b80 1 --2- 172.21.15.92:0/4149134933 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567a8a6d950 0x5567a8ac3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.246+0000 7f69c0f98b80 1 --2- 172.21.15.92:0/4149134933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a96e60 0x5567a8ac4030 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.246+0000 7f69c0f98b80 1 -- 172.21.15.92:0/4149134933 >> 172.21.15.92:0/4149134933 conn(0x5567a8a6f270 msgr2=0x5567a8a6f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:59.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.246+0000 7f69c0f98b80 1 -- 172.21.15.92:0/4149134933 shutdown_connections 2023-10-06T10:43:59.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.246+0000 7f69c0f98b80 1 -- 172.21.15.92:0/4149134933 wait complete. 2023-10-06T10:43:59.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.247+0000 7f69c0f98b80 1 -- start start 2023-10-06T10:43:59.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.247+0000 7f69c0f98b80 1 Processor -- start 2023-10-06T10:43:59.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.248+0000 7f69c0f98b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a6d950 0x5567a8a64600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.248+0000 7f69c0f98b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567a8a96e60 0x5567a8a6a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.248+0000 7f69c0f98b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567a8ab7bc0 0x5567a8a6a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.248+0000 7f69c0f98b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567a8b1aa30 con 0x5567a8a6d950 2023-10-06T10:43:59.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.248+0000 7f69c0f98b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567a8a6de50 con 0x5567a8ab7bc0 2023-10-06T10:43:59.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.248+0000 7f69c0f98b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567a8ab1da0 con 0x5567a8a96e60 2023-10-06T10:43:59.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.248+0000 7f69c0f98b80 10 client.0 _ll_register_callbacks cb 0x5567a885af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:59.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.248+0000 7f69b5a3c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a6d950 0x5567a8a64600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.248+0000 7f69b5a3c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a6d950 0x5567a8a64600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36382/0 (socket says 172.21.15.92:36382) 2023-10-06T10:43:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.248+0000 7f69b5a3c700 1 -- 172.21.15.92:0/3368272398 learned_addr learned my addr 172.21.15.92:0/3368272398 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.248+0000 7f69b623d700 1 --2- 172.21.15.92:0/3368272398 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567a8ab7bc0 0x5567a8a6a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.248+0000 7f69b523b700 1 --2- 172.21.15.92:0/3368272398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567a8a96e60 0x5567a8a6a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.248+0000 7f69b5a3c700 1 -- 172.21.15.92:0/3368272398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567a8a96e60 msgr2=0x5567a8a6a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.248+0000 7f69b5a3c700 1 --2- 172.21.15.92:0/3368272398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567a8a96e60 0x5567a8a6a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.248+0000 7f69b5a3c700 1 -- 172.21.15.92:0/3368272398 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567a8ab7bc0 msgr2=0x5567a8a6a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.248+0000 7f69b5a3c700 1 --2- 172.21.15.92:0/3368272398 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567a8ab7bc0 0x5567a8a6a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.248+0000 7f69b5a3c700 1 -- 172.21.15.92:0/3368272398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567a87fafe0 con 0x5567a8a6d950 2023-10-06T10:43:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.248+0000 7f69b623d700 1 --2- 172.21.15.92:0/3368272398 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567a8ab7bc0 0x5567a8a6a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.248+0000 7f69b5a3c700 1 --2- 172.21.15.92:0/3368272398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a6d950 0x5567a8a64600 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00b3c0 tx=0x7f69ac02b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc29d42e4badef24 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.248+0000 7f698bfff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.249+0000 7f698bfff700 1 -- 172.21.15.92:0/3368272398 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac011ac0 con 0x5567a8a6d950 2023-10-06T10:43:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.249+0000 7f698bfff700 1 -- 172.21.15.92:0/3368272398 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f69ac011c20 con 0x5567a8a6d950 2023-10-06T10:43:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.249+0000 7f69c0f98b80 1 -- 172.21.15.92:0/3368272398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69a8053cf0 con 0x5567a8a6d950 2023-10-06T10:43:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.249+0000 7f698bfff700 1 -- 172.21.15.92:0/3368272398 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69ac01d460 con 0x5567a8a6d950 2023-10-06T10:43:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.249+0000 7f69c0f98b80 10 client.7905 fetch_fsmap learned FSMap version 88 2023-10-06T10:43:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.249+0000 7f69c0f98b80 1 -- 172.21.15.92:0/3368272398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x5567a8973470 con 0x5567a8a6d950 2023-10-06T10:43:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.249+0000 7f698bfff700 1 -- 172.21.15.92:0/3368272398 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a8053cf0 con 0x5567a8a6d950 2023-10-06T10:43:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.249+0000 7f698bfff700 1 -- 172.21.15.92:0/3368272398 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 88) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f69ac01d5c0 con 0x5567a8a6d950 2023-10-06T10:43:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.249+0000 7f69c0f98b80 10 client.7905 fetch_fsmap finished waiting for FSMap version 88 2023-10-06T10:43:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.249+0000 7f69c0f98b80 10 client.7905 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T10:43:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.249+0000 7f69c0f98b80 20 client.7905 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.249+0000 7f69c0f98b80 1 --2- 172.21.15.92:0/3368272398 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x5567a8b19a60 0x5567a8b19e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.249+0000 7f69c0f98b80 4 client.7905 mds_command: new command op to 7686 tid=0[{"prefix": "status"}] 2023-10-06T10:43:59.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.249+0000 7f69c0f98b80 1 -- 172.21.15.92:0/3368272398 --> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5567a8960e00 con 0x5567a8b19a60 2023-10-06T10:43:59.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.249+0000 7f69b523b700 1 --2- 172.21.15.92:0/3368272398 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x5567a8b19a60 0x5567a8b19e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.249+0000 7f698bfff700 10 client.7905 ms_handle_connect on v2:172.21.15.92:6834/3605297930 2023-10-06T10:43:59.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.249+0000 7f69b523b700 1 --2- 172.21.15.92:0/3368272398 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x5567a8b19a60 0x5567a8b19e30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c5598c7b5ec8e779 server_cookie=bf2b18c6d025cfce in_seq=0 out_seq=0 2023-10-06T10:43:59.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.250+0000 7f698bfff700 1 -- 172.21.15.92:0/3368272398 <== mds.2 v2:172.21.15.92:6834/3605297930 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5567a8960e00 con 0x5567a8b19a60 2023-10-06T10:43:59.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.250+0000 7f698bfff700 10 client.7905 handle_command_reply: tid=0 2023-10-06T10:43:59.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.250+0000 7f69c0f98b80 1 client.7905 shutdown 2023-10-06T10:43:59.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.250+0000 7f69c0f98b80 1 -- 172.21.15.92:0/3368272398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a6d950 msgr2=0x5567a8a64600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.250+0000 7f69c0f98b80 1 --2- 172.21.15.92:0/3368272398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a6d950 0x5567a8a64600 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00b3c0 tx=0x7f69ac02b040 comp rx=0 tx=0).stop 2023-10-06T10:43:59.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.250+0000 7f69c0f98b80 1 -- 172.21.15.92:0/3368272398 shutdown_connections 2023-10-06T10:43:59.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.250+0000 7f69c0f98b80 1 --2- 172.21.15.92:0/3368272398 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5567a8ab7bc0 0x5567a8a6a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.250+0000 7f69c0f98b80 1 --2- 172.21.15.92:0/3368272398 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x5567a8b19a60 0x5567a8b19e30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.250+0000 7f69c0f98b80 1 --2- 172.21.15.92:0/3368272398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567a8a6d950 0x5567a8a64600 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.250+0000 7f698bfff700 0 client.7905 ms_handle_reset on v2:172.21.15.92:6834/3605297930 2023-10-06T10:43:59.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.250+0000 7f69c0f98b80 1 --2- 172.21.15.92:0/3368272398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5567a8a96e60 0x5567a8a6a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.250+0000 7f69c0f98b80 1 -- 172.21.15.92:0/3368272398 >> 172.21.15.92:0/3368272398 conn(0x5567a8a6f270 msgr2=0x5567a8aad3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:59.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.250+0000 7f69c0f98b80 1 -- 172.21.15.92:0/3368272398 shutdown_connections 2023-10-06T10:43:59.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.250+0000 7f69c0f98b80 1 -- 172.21.15.92:0/3368272398 wait complete. 2023-10-06T10:43:59.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.250+0000 7f69c0f98b80 20 client.7905 trim_cache size 0 max 16384 2023-10-06T10:43:59.259 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:43:59.259 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:43:59.260 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 2, 2023-10-06T10:43:59.260 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 7686, 2023-10-06T10:43:59.260 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:43:59.260 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:43:59.260 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:43:59.260 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 1.720473556, 2023-10-06T10:43:59.260 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 88, 2023-10-06T10:43:59.260 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 90, 2023-10-06T10:43:59.260 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 90, 2023-10-06T10:43:59.260 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 16.908566522000001 2023-10-06T10:43:59.260 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:43:59.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.250+0000 7f69b6a3e700 1 -- 172.21.15.92:0/1580009231 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6994066f30 msgr2=0x7f69940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.250+0000 7f69b6a3e700 1 --2- 172.21.15.92:0/1580009231 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6994066f30 0x7f69940693e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0099b0 tx=0x7f69ac016000 comp rx=0 tx=0).stop 2023-10-06T10:43:59.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.250+0000 7f69b6a3e700 1 -- 172.21.15.92:0/1580009231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b01570f0 msgr2=0x7f69b0151fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.250+0000 7f69b6a3e700 1 --2- 172.21.15.92:0/1580009231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b01570f0 0x7f69b0151fe0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f69a800c930 tx=0x7f69a80026e0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.251+0000 7f69b6a3e700 1 -- 172.21.15.92:0/1580009231 shutdown_connections 2023-10-06T10:43:59.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.251+0000 7f69b6a3e700 1 --2- 172.21.15.92:0/1580009231 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6994066f30 0x7f69940693e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.251+0000 7f69b6a3e700 1 --2- 172.21.15.92:0/1580009231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b01570f0 0x7f69b0151fe0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.251+0000 7f69b6a3e700 1 --2- 172.21.15.92:0/1580009231 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69b0068b00 0x7f69b014b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.251+0000 7f69b6a3e700 1 --2- 172.21.15.92:0/1580009231 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69b006e8b0 0x7f69b00add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.251+0000 7f69b6a3e700 1 -- 172.21.15.92:0/1580009231 >> 172.21.15.92:0/1580009231 conn(0x7f69b00c9c10 msgr2=0x7f69b00c3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:59.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.251+0000 7f69b6a3e700 1 -- 172.21.15.92:0/1580009231 shutdown_connections 2023-10-06T10:43:59.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.251+0000 7f69b6a3e700 1 -- 172.21.15.92:0/1580009231 wait complete. 2023-10-06T10:43:59.264 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T10:43:59.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.378+0000 7fadb0090700 1 Processor -- start 2023-10-06T10:43:59.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.378+0000 7fadb0090700 1 -- start start 2023-10-06T10:43:59.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.379+0000 7fadb0090700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fada8159560 0x7fada80aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.379+0000 7fadb0090700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fada80b49a0 0x7fada80aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.379+0000 7fadb0090700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fada80b0c60 con 0x7fada80b0760 2023-10-06T10:43:59.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.379+0000 7fadb0090700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fada80b0f40 con 0x7fada8159560 2023-10-06T10:43:59.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.379+0000 7fadb0090700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada80b1220 con 0x7fada80b49a0 2023-10-06T10:43:59.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.379+0000 7fadae62d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fada80b0760 0x7fada80b3890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57284/0 (socket says 172.21.15.92:57284) 2023-10-06T10:43:59.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.379+0000 7fadae62d700 1 -- 172.21.15.92:0/1898506987 learned_addr learned my addr 172.21.15.92:0/1898506987 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:59.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.379+0000 7fadad62b700 1 --2- 172.21.15.92:0/1898506987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fada80b49a0 0x7fada80aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.379+0000 7fadade2c700 1 --2- 172.21.15.92:0/1898506987 >> v2:172.21.15.124:3300/0 conn(0x7fada8159560 0x7fada80aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.379+0000 7fadace2a700 1 -- 172.21.15.92:0/1898506987 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 190839375 0 0) 0x7fada80b0c60 con 0x7fada80b0760 2023-10-06T10:43:59.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.379+0000 7fadace2a700 1 -- 172.21.15.92:0/1898506987 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad94003570 con 0x7fada80b0760 2023-10-06T10:43:59.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadace2a700 1 -- 172.21.15.92:0/1898506987 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3281450917 0 0) 0x7fad94003570 con 0x7fada80b0760 2023-10-06T10:43:59.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadace2a700 1 -- 172.21.15.92:0/1898506987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fada80b49a0 msgr2=0x7fada80aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadace2a700 1 --2- 172.21.15.92:0/1898506987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fada80b49a0 0x7fada80aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadace2a700 1 -- 172.21.15.92:0/1898506987 >> v2:172.21.15.124:3300/0 conn(0x7fada8159560 msgr2=0x7fada80aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadace2a700 1 --2- 172.21.15.92:0/1898506987 >> v2:172.21.15.124:3300/0 conn(0x7fada8159560 0x7fada80aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadace2a700 1 -- 172.21.15.92:0/1898506987 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada80b1500 con 0x7fada80b0760 2023-10-06T10:43:59.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadace2a700 1 -- 172.21.15.92:0/1898506987 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fada00017f0 con 0x7fada80b0760 2023-10-06T10:43:59.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadace2a700 1 -- 172.21.15.92:0/1898506987 >> v1:172.21.15.92:6789/0 conn(0x7fada80b0760 legacy=0x7fada80b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadace2a700 1 --2- 172.21.15.92:0/1898506987 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad94003a60 0x7fad94003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadace2a700 1 --2- 172.21.15.92:0/1898506987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad94004810 0x7fad94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadace2a700 1 --2- 172.21.15.92:0/1898506987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fada80b49a0 0x7fad940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadace2a700 1 -- 172.21.15.92:0/1898506987 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada80b0f40 con 0x7fad94004810 2023-10-06T10:43:59.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadace2a700 1 -- 172.21.15.92:0/1898506987 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada80b1220 con 0x7fad94003a60 2023-10-06T10:43:59.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadace2a700 1 -- 172.21.15.92:0/1898506987 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9400ba60 con 0x7fada80b49a0 2023-10-06T10:43:59.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadad62b700 1 --2- 172.21.15.92:0/1898506987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad94004810 0x7fad94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadade2c700 1 --2- 172.21.15.92:0/1898506987 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad94003a60 0x7fad94003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadae62d700 1 --2- 172.21.15.92:0/1898506987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fada80b49a0 0x7fad940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadad62b700 1 -- 172.21.15.92:0/1898506987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fada80b49a0 msgr2=0x7fad940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadad62b700 1 --2- 172.21.15.92:0/1898506987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fada80b49a0 0x7fad940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadad62b700 1 -- 172.21.15.92:0/1898506987 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad94003a60 msgr2=0x7fad94003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadad62b700 1 --2- 172.21.15.92:0/1898506987 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad94003a60 0x7fad94003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadad62b700 1 -- 172.21.15.92:0/1898506987 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad94003570 con 0x7fad94004810 2023-10-06T10:43:59.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadade2c700 1 --2- 172.21.15.92:0/1898506987 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad94003a60 0x7fad94003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:59.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadad62b700 1 --2- 172.21.15.92:0/1898506987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad94004810 0x7fad94004c80 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fad98009c30 tx=0x7fad980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88361f9f11ffa4f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadace2a700 1 -- 172.21.15.92:0/1898506987 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9800e710 con 0x7fad94004810 2023-10-06T10:43:59.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadace2a700 1 -- 172.21.15.92:0/1898506987 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9800e870 con 0x7fad94004810 2023-10-06T10:43:59.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.380+0000 7fadace2a700 1 -- 172.21.15.92:0/1898506987 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9800ec30 con 0x7fad94004810 2023-10-06T10:43:59.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.381+0000 7fadb0090700 1 -- 172.21.15.92:0/1898506987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad94004810 msgr2=0x7fad94004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.381+0000 7fadb0090700 1 --2- 172.21.15.92:0/1898506987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad94004810 0x7fad94004c80 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fad98009c30 tx=0x7fad980091a0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.381+0000 7fadae62d700 1 -- 172.21.15.92:0/1898506987 reap_dead start 2023-10-06T10:43:59.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.381+0000 7fadb0090700 1 -- 172.21.15.92:0/1898506987 shutdown_connections 2023-10-06T10:43:59.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.381+0000 7fadb0090700 1 -- 172.21.15.92:0/1898506987 >> 172.21.15.92:0/1898506987 conn(0x7fada80997f0 msgr2=0x7fada806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:59.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.381+0000 7fadb0090700 1 -- 172.21.15.92:0/1898506987 shutdown_connections 2023-10-06T10:43:59.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.381+0000 7fadb0090700 1 -- 172.21.15.92:0/1898506987 wait complete. 2023-10-06T10:43:59.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.382+0000 7fadb0090700 1 Processor -- start 2023-10-06T10:43:59.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.382+0000 7fadb0090700 1 -- start start 2023-10-06T10:43:59.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.383+0000 7fadb0090700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad94004810 0x7fada816d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.383+0000 7fadb0090700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada815ad80 0x7fada815b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.383+0000 7fadb0090700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fada80cc970 0x7fada80ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.383+0000 7fadb0090700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada815baf0 con 0x7fada815ad80 2023-10-06T10:43:59.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.383+0000 7fadb0090700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada815bdd0 con 0x7fad94004810 2023-10-06T10:43:59.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.383+0000 7fadb0090700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada815c0b0 con 0x7fada80cc970 2023-10-06T10:43:59.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.383+0000 7fadad62b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada815ad80 0x7fada815b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.383+0000 7fadad62b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada815ad80 0x7fada815b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36404/0 (socket says 172.21.15.92:36404) 2023-10-06T10:43:59.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.383+0000 7fadad62b700 1 -- 172.21.15.92:0/2341715209 learned_addr learned my addr 172.21.15.92:0/2341715209 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:59.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.383+0000 7fadade2c700 1 --2- 172.21.15.92:0/2341715209 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad94004810 0x7fada816d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.383+0000 7fadae62d700 1 --2- 172.21.15.92:0/2341715209 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fada80cc970 0x7fada80ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.383+0000 7fadad62b700 1 -- 172.21.15.92:0/2341715209 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fada80cc970 msgr2=0x7fada80ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.383+0000 7fadad62b700 1 --2- 172.21.15.92:0/2341715209 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fada80cc970 0x7fada80ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.383+0000 7fadad62b700 1 -- 172.21.15.92:0/2341715209 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad94004810 msgr2=0x7fada816d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.383+0000 7fadad62b700 1 --2- 172.21.15.92:0/2341715209 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad94004810 0x7fada816d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.383+0000 7fadad62b700 1 -- 172.21.15.92:0/2341715209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada815c390 con 0x7fada815ad80 2023-10-06T10:43:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.383+0000 7fadade2c700 1 --2- 172.21.15.92:0/2341715209 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad94004810 0x7fada816d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.383+0000 7fadae62d700 1 --2- 172.21.15.92:0/2341715209 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fada80cc970 0x7fada80ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.383+0000 7fadad62b700 1 --2- 172.21.15.92:0/2341715209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada815ad80 0x7fada815b1f0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fad940046d0 tx=0x7fad9801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.383+0000 7fad9dffb700 1 -- 172.21.15.92:0/2341715209 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad98022070 con 0x7fada815ad80 2023-10-06T10:43:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.383+0000 7fad9dffb700 1 -- 172.21.15.92:0/2341715209 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9800e9e0 con 0x7fada815ad80 2023-10-06T10:43:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.384+0000 7fadb0090700 1 -- 172.21.15.92:0/2341715209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada815b730 con 0x7fada815ad80 2023-10-06T10:43:59.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.384+0000 7fad9dffb700 1 -- 172.21.15.92:0/2341715209 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9801c450 con 0x7fada815ad80 2023-10-06T10:43:59.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.384+0000 7fadb0090700 1 -- 172.21.15.92:0/2341715209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada80cd320 con 0x7fada815ad80 2023-10-06T10:43:59.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.385+0000 7fad9dffb700 1 -- 172.21.15.92:0/2341715209 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fad9801c840 con 0x7fada815ad80 2023-10-06T10:43:59.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.386+0000 7fad9dffb700 1 --2- 172.21.15.92:0/2341715209 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fad8c066f30 0x7fad8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.386+0000 7fad9dffb700 1 -- 172.21.15.92:0/2341715209 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fad98085fd0 con 0x7fada815ad80 2023-10-06T10:43:59.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.386+0000 7fadade2c700 1 --2- 172.21.15.92:0/2341715209 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fad8c066f30 0x7fad8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.386+0000 7fadade2c700 1 --2- 172.21.15.92:0/2341715209 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fad8c066f30 0x7fad8c0693e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fada4006aa0 tx=0x7fada4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.388+0000 7fadb9388b80 1 Processor -- start 2023-10-06T10:43:59.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.388+0000 7fadb9388b80 1 -- start start 2023-10-06T10:43:59.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.388+0000 7fadb9388b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55998ea46a90 0x55998ea46e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.388+0000 7fadb9388b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55998ea029a0 0x55998e9ff580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.388+0000 7fadb9388b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea00bd0 0x55998e9ffac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.388+0000 7fadb9388b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55998e97fc80 con 0x55998ea00bd0 2023-10-06T10:43:59.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.388+0000 7fadb9388b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55998e956a70 con 0x55998ea46a90 2023-10-06T10:43:59.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.388+0000 7fadb9388b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55998e954270 con 0x55998ea029a0 2023-10-06T10:43:59.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.388+0000 7fadae62d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea00bd0 0x55998e9ffac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.388+0000 7fadae62d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea00bd0 0x55998e9ffac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36412/0 (socket says 172.21.15.92:36412) 2023-10-06T10:43:59.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.388+0000 7fadae62d700 1 -- 172.21.15.92:0/3664456696 learned_addr learned my addr 172.21.15.92:0/3664456696 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:59.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.389+0000 7fadae62d700 1 -- 172.21.15.92:0/3664456696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55998ea029a0 msgr2=0x55998e9ff580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.389+0000 7fadae62d700 1 --2- 172.21.15.92:0/3664456696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55998ea029a0 0x55998e9ff580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.389+0000 7fadae62d700 1 -- 172.21.15.92:0/3664456696 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55998ea46a90 msgr2=0x55998ea46e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.389+0000 7fadae62d700 1 --2- 172.21.15.92:0/3664456696 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55998ea46a90 0x55998ea46e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.389+0000 7fadae62d700 1 -- 172.21.15.92:0/3664456696 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55998e7ea270 con 0x55998ea00bd0 2023-10-06T10:43:59.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.389+0000 7fadae62d700 1 --2- 172.21.15.92:0/3664456696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea00bd0 0x55998e9ffac0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fada808ac50 tx=0x7fada0006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30576cf936099fdc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.389+0000 7fad92ffd700 1 -- 172.21.15.92:0/3664456696 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0007860 con 0x55998ea00bd0 2023-10-06T10:43:59.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.389+0000 7fad92ffd700 1 -- 172.21.15.92:0/3664456696 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada00079c0 con 0x55998ea00bd0 2023-10-06T10:43:59.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.389+0000 7fad92ffd700 1 -- 172.21.15.92:0/3664456696 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada00156a0 con 0x55998ea00bd0 2023-10-06T10:43:59.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.389+0000 7fadb9388b80 1 -- 172.21.15.92:0/3664456696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea00bd0 msgr2=0x55998e9ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.389+0000 7fadb9388b80 1 --2- 172.21.15.92:0/3664456696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea00bd0 0x55998e9ffac0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fada808ac50 tx=0x7fada0006040 comp rx=0 tx=0).stop 2023-10-06T10:43:59.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.389+0000 7fadb9388b80 1 -- 172.21.15.92:0/3664456696 shutdown_connections 2023-10-06T10:43:59.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.389+0000 7fadb9388b80 1 --2- 172.21.15.92:0/3664456696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea00bd0 0x55998e9ffac0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.389+0000 7fadb9388b80 1 --2- 172.21.15.92:0/3664456696 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55998ea46a90 0x55998ea46e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.389+0000 7fadb9388b80 1 --2- 172.21.15.92:0/3664456696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55998ea029a0 0x55998e9ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.389+0000 7fadb9388b80 1 -- 172.21.15.92:0/3664456696 >> 172.21.15.92:0/3664456696 conn(0x55998ea01070 msgr2=0x55998ea01440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:59.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.390+0000 7fadb9388b80 1 -- 172.21.15.92:0/3664456696 shutdown_connections 2023-10-06T10:43:59.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.390+0000 7fadb9388b80 1 -- 172.21.15.92:0/3664456696 wait complete. 2023-10-06T10:43:59.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.390+0000 7fadb9388b80 1 -- start start 2023-10-06T10:43:59.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.390+0000 7fadb9388b80 1 Processor -- start 2023-10-06T10:43:59.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.391+0000 7fadb9388b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55998ea00bd0 0x55998ea40900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.391+0000 7fadb9388b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea029a0 0x55998ea2bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.391+0000 7fadb9388b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55998ea46a90 0x55998ea27ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.391+0000 7fadb9388b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55998e950060 con 0x55998ea029a0 2023-10-06T10:43:59.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.391+0000 7fadb9388b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55998e8fb890 con 0x55998ea00bd0 2023-10-06T10:43:59.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.391+0000 7fadb9388b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55998e916ea0 con 0x55998ea46a90 2023-10-06T10:43:59.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.391+0000 7fadb9388b80 10 client.0 _ll_register_callbacks cb 0x55998e7ecf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:59.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.391+0000 7fadad62b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea029a0 0x55998ea2bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.391+0000 7fadad62b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea029a0 0x55998ea2bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36418/0 (socket says 172.21.15.92:36418) 2023-10-06T10:43:59.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.391+0000 7fadad62b700 1 -- 172.21.15.92:0/823695412 learned_addr learned my addr 172.21.15.92:0/823695412 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:59.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.391+0000 7fadad62b700 1 -- 172.21.15.92:0/823695412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55998ea46a90 msgr2=0x55998ea27ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.391+0000 7fadad62b700 1 --2- 172.21.15.92:0/823695412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55998ea46a90 0x55998ea27ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.391+0000 7fadade2c700 1 --2- 172.21.15.92:0/823695412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55998ea00bd0 0x55998ea40900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.391+0000 7fadad62b700 1 -- 172.21.15.92:0/823695412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55998ea00bd0 msgr2=0x55998ea40900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.391+0000 7fadad62b700 1 --2- 172.21.15.92:0/823695412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55998ea00bd0 0x55998ea40900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.391+0000 7fadad62b700 1 -- 172.21.15.92:0/823695412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada80b1500 con 0x55998ea029a0 2023-10-06T10:43:59.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.392+0000 7fadad62b700 1 --2- 172.21.15.92:0/823695412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea029a0 0x55998ea2bdf0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fad9800e4b0 tx=0x7fad98084110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360ac7ee236d4f87 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.392+0000 7fad87fff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:59.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.392+0000 7fad87fff700 1 -- 172.21.15.92:0/823695412 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad980091e0 con 0x55998ea029a0 2023-10-06T10:43:59.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.392+0000 7fad87fff700 1 -- 172.21.15.92:0/823695412 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fad98056bc0 con 0x55998ea029a0 2023-10-06T10:43:59.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.392+0000 7fadb9388b80 1 -- 172.21.15.92:0/823695412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55998e9f4690 con 0x55998ea029a0 2023-10-06T10:43:59.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.392+0000 7fad87fff700 1 -- 172.21.15.92:0/823695412 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad98059e10 con 0x55998ea029a0 2023-10-06T10:43:59.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.392+0000 7fadb9388b80 10 client.7917 fetch_fsmap learned FSMap version 88 2023-10-06T10:43:59.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.392+0000 7fadb9388b80 1 -- 172.21.15.92:0/823695412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x55998e78cfe0 con 0x55998ea029a0 2023-10-06T10:43:59.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.392+0000 7fad87fff700 1 -- 172.21.15.92:0/823695412 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55998e9f4690 con 0x55998ea029a0 2023-10-06T10:43:59.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.392+0000 7fad87fff700 1 -- 172.21.15.92:0/823695412 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 88) v1 ==== 3124+0+0 (secure 0 0 0) 0x7fad980560b0 con 0x55998ea029a0 2023-10-06T10:43:59.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.392+0000 7fadb9388b80 10 client.7917 fetch_fsmap finished waiting for FSMap version 88 2023-10-06T10:43:59.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.392+0000 7fadb9388b80 10 client.7917 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:43:59.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.392+0000 7fadb9388b80 20 client.7917 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:59.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.392+0000 7fadb9388b80 1 --2- 172.21.15.92:0/823695412 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x55998ea29900 0x55998ea29cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.392+0000 7fadb9388b80 4 client.7917 mds_command: new command op to 7689 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:43:59.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.392+0000 7fadb9388b80 1 -- 172.21.15.92:0/823695412 --> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55998e9d15c0 con 0x55998ea29900 2023-10-06T10:43:59.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.392+0000 7fadade2c700 1 --2- 172.21.15.92:0/823695412 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x55998ea29900 0x55998ea29cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.393+0000 7fad87fff700 10 client.7917 ms_handle_connect on v2:172.21.15.92:6835/4047304116 2023-10-06T10:43:59.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.393+0000 7fadade2c700 1 --2- 172.21.15.92:0/823695412 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x55998ea29900 0x55998ea29cd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d532fb76376dbeef server_cookie=a978386d81a3132d in_seq=0 out_seq=0 2023-10-06T10:43:59.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.393+0000 7fad87fff700 1 -- 172.21.15.92:0/823695412 <== mds.1 v2:172.21.15.92:6835/4047304116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55998e9d15c0 con 0x55998ea29900 2023-10-06T10:43:59.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.393+0000 7fad87fff700 10 client.7917 handle_command_reply: tid=0 2023-10-06T10:43:59.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.393+0000 7fadb9388b80 1 client.7917 shutdown 2023-10-06T10:43:59.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.393+0000 7fadb9388b80 1 -- 172.21.15.92:0/823695412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea029a0 msgr2=0x55998ea2bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.393+0000 7fadb9388b80 1 --2- 172.21.15.92:0/823695412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea029a0 0x55998ea2bdf0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fad9800e4b0 tx=0x7fad98084110 comp rx=0 tx=0).stop 2023-10-06T10:43:59.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.393+0000 7fadb9388b80 1 -- 172.21.15.92:0/823695412 shutdown_connections 2023-10-06T10:43:59.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.393+0000 7fadb9388b80 1 --2- 172.21.15.92:0/823695412 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x55998ea29900 0x55998ea29cd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.393+0000 7fadb9388b80 1 --2- 172.21.15.92:0/823695412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55998ea46a90 0x55998ea27ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.393+0000 7fad87fff700 0 client.7917 ms_handle_reset on v2:172.21.15.92:6835/4047304116 2023-10-06T10:43:59.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.393+0000 7fadb9388b80 1 --2- 172.21.15.92:0/823695412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55998ea00bd0 0x55998ea40900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.393+0000 7fadb9388b80 1 --2- 172.21.15.92:0/823695412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea029a0 0x55998ea2bdf0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.393+0000 7fadb9388b80 1 -- 172.21.15.92:0/823695412 >> 172.21.15.92:0/823695412 conn(0x55998ea01070 msgr2=0x55998e9fba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:59.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.393+0000 7fadb9388b80 1 -- 172.21.15.92:0/823695412 shutdown_connections 2023-10-06T10:43:59.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.393+0000 7fadb9388b80 1 -- 172.21.15.92:0/823695412 wait complete. 2023-10-06T10:43:59.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.394+0000 7fadb9388b80 20 client.7917 trim_cache size 0 max 16384 2023-10-06T10:43:59.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.410+0000 7fadb9388b80 1 Processor -- start 2023-10-06T10:43:59.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.410+0000 7fadb9388b80 1 -- start start 2023-10-06T10:43:59.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.411+0000 7fadb9388b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea00bd0 0x55998ea41f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.411+0000 7fadb9388b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55998ea029a0 0x55998e9fbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.411+0000 7fadb9388b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55998ea46a90 0x55998ea45350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.411+0000 7fadb9388b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55998e7d2e30 con 0x55998ea00bd0 2023-10-06T10:43:59.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.411+0000 7fadb9388b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55998e9fa0c0 con 0x55998ea029a0 2023-10-06T10:43:59.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.411+0000 7fadb9388b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55998ea470a0 con 0x55998ea46a90 2023-10-06T10:43:59.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.411+0000 7fadade2c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea00bd0 0x55998ea41f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.411+0000 7fadade2c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea00bd0 0x55998ea41f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36432/0 (socket says 172.21.15.92:36432) 2023-10-06T10:43:59.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.411+0000 7fadade2c700 1 -- 172.21.15.92:0/1853647959 learned_addr learned my addr 172.21.15.92:0/1853647959 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:59.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.411+0000 7fadade2c700 1 -- 172.21.15.92:0/1853647959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55998ea46a90 msgr2=0x55998ea45350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.411+0000 7fadade2c700 1 --2- 172.21.15.92:0/1853647959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55998ea46a90 0x55998ea45350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.411+0000 7fadade2c700 1 -- 172.21.15.92:0/1853647959 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55998ea029a0 msgr2=0x55998e9fbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.411+0000 7fadade2c700 1 --2- 172.21.15.92:0/1853647959 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55998ea029a0 0x55998e9fbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.411+0000 7fadade2c700 1 -- 172.21.15.92:0/1853647959 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55998e91e780 con 0x55998ea00bd0 2023-10-06T10:43:59.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.411+0000 7fadade2c700 1 --2- 172.21.15.92:0/1853647959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea00bd0 0x55998ea41f60 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fada4009fd0 tx=0x7fada4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8f354847aad8aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.411+0000 7fad937fe700 1 -- 172.21.15.92:0/1853647959 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada4029e20 con 0x55998ea00bd0 2023-10-06T10:43:59.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.411+0000 7fad937fe700 1 -- 172.21.15.92:0/1853647959 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada4013e30 con 0x55998ea00bd0 2023-10-06T10:43:59.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.411+0000 7fad937fe700 1 -- 172.21.15.92:0/1853647959 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada4005a00 con 0x55998ea00bd0 2023-10-06T10:43:59.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.412+0000 7fadb9388b80 1 -- 172.21.15.92:0/1853647959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea00bd0 msgr2=0x55998ea41f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.412+0000 7fadb9388b80 1 --2- 172.21.15.92:0/1853647959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea00bd0 0x55998ea41f60 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fada4009fd0 tx=0x7fada4018000 comp rx=0 tx=0).stop 2023-10-06T10:43:59.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.412+0000 7fadb9388b80 1 -- 172.21.15.92:0/1853647959 shutdown_connections 2023-10-06T10:43:59.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.412+0000 7fadb9388b80 1 --2- 172.21.15.92:0/1853647959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55998ea46a90 0x55998ea45350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.412+0000 7fadb9388b80 1 --2- 172.21.15.92:0/1853647959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea00bd0 0x55998ea41f60 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.412+0000 7fadb9388b80 1 --2- 172.21.15.92:0/1853647959 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55998ea029a0 0x55998e9fbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.412+0000 7fadb9388b80 1 -- 172.21.15.92:0/1853647959 >> 172.21.15.92:0/1853647959 conn(0x55998ea01070 msgr2=0x55998ea441f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:59.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.412+0000 7fadb9388b80 1 -- 172.21.15.92:0/1853647959 shutdown_connections 2023-10-06T10:43:59.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.412+0000 7fadb9388b80 1 -- 172.21.15.92:0/1853647959 wait complete. 2023-10-06T10:43:59.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.412+0000 7fadb9388b80 1 -- start start 2023-10-06T10:43:59.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.413+0000 7fadb9388b80 1 Processor -- start 2023-10-06T10:43:59.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.413+0000 7fadb9388b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea00bd0 0x55998ea4a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.413+0000 7fadb9388b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55998ea029a0 0x55998eaaea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.413+0000 7fadb9388b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55998ea46a90 0x55998ea3c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.413+0000 7fadb9388b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55998ea3cf80 con 0x55998ea00bd0 2023-10-06T10:43:59.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.413+0000 7fadb9388b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55998ea2f900 con 0x55998ea029a0 2023-10-06T10:43:59.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.413+0000 7fadb9388b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55998ea49e50 con 0x55998ea46a90 2023-10-06T10:43:59.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.414+0000 7fadb9388b80 10 client.0 _ll_register_callbacks cb 0x55998e7ecf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:59.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.414+0000 7fadade2c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea00bd0 0x55998ea4a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.414+0000 7fadade2c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea00bd0 0x55998ea4a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36446/0 (socket says 172.21.15.92:36446) 2023-10-06T10:43:59.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.414+0000 7fadade2c700 1 -- 172.21.15.92:0/1686097004 learned_addr learned my addr 172.21.15.92:0/1686097004 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:59.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.414+0000 7fadad62b700 1 --2- 172.21.15.92:0/1686097004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55998ea029a0 0x55998eaaea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.414+0000 7fadae62d700 1 --2- 172.21.15.92:0/1686097004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55998ea46a90 0x55998ea3c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.414+0000 7fadade2c700 1 -- 172.21.15.92:0/1686097004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55998ea46a90 msgr2=0x55998ea3c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.414+0000 7fadade2c700 1 --2- 172.21.15.92:0/1686097004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55998ea46a90 0x55998ea3c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.414+0000 7fadade2c700 1 -- 172.21.15.92:0/1686097004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55998ea029a0 msgr2=0x55998eaaea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.414+0000 7fadade2c700 1 --2- 172.21.15.92:0/1686097004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55998ea029a0 0x55998eaaea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.414+0000 7fadade2c700 1 -- 172.21.15.92:0/1686097004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada401e040 con 0x55998ea00bd0 2023-10-06T10:43:59.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.414+0000 7fadade2c700 1 --2- 172.21.15.92:0/1686097004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea00bd0 0x55998ea4a8a0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fada4008010 tx=0x7fada4026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26da9887177434b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.414+0000 7fad87fff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:59.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.414+0000 7fad87fff700 1 -- 172.21.15.92:0/1686097004 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada4006050 con 0x55998ea00bd0 2023-10-06T10:43:59.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.414+0000 7fad87fff700 1 -- 172.21.15.92:0/1686097004 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fada40061b0 con 0x55998ea00bd0 2023-10-06T10:43:59.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.414+0000 7fadb9388b80 1 -- 172.21.15.92:0/1686097004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fada00079c0 con 0x55998ea00bd0 2023-10-06T10:43:59.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.414+0000 7fad87fff700 1 -- 172.21.15.92:0/1686097004 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fada4005eb0 con 0x55998ea00bd0 2023-10-06T10:43:59.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.414+0000 7fadb9388b80 10 client.7923 fetch_fsmap learned FSMap version 88 2023-10-06T10:43:59.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.414+0000 7fadb9388b80 1 -- 172.21.15.92:0/1686097004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x55998e905470 con 0x55998ea00bd0 2023-10-06T10:43:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.414+0000 7fad87fff700 1 -- 172.21.15.92:0/1686097004 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada00079c0 con 0x55998ea00bd0 2023-10-06T10:43:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fad87fff700 1 -- 172.21.15.92:0/1686097004 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 88) v1 ==== 3124+0+0 (secure 0 0 0) 0x7fada4006580 con 0x55998ea00bd0 2023-10-06T10:43:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fadb9388b80 10 client.7923 fetch_fsmap finished waiting for FSMap version 88 2023-10-06T10:43:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fadb9388b80 10 client.7923 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:43:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fadb9388b80 20 client.7923 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fadb9388b80 1 --2- 172.21.15.92:0/1686097004 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x55998ea2b6d0 0x55998ea2baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fadb9388b80 4 client.7923 mds_command: new command op to 7689 tid=0[{"prefix": "status"}] 2023-10-06T10:43:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fadb9388b80 1 -- 172.21.15.92:0/1686097004 --> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55998e8f2e00 con 0x55998ea2b6d0 2023-10-06T10:43:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fadad62b700 1 --2- 172.21.15.92:0/1686097004 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x55998ea2b6d0 0x55998ea2baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fad87fff700 10 client.7923 ms_handle_connect on v2:172.21.15.92:6835/4047304116 2023-10-06T10:43:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fadad62b700 1 --2- 172.21.15.92:0/1686097004 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x55998ea2b6d0 0x55998ea2baa0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61cf95417f65634 server_cookie=9603f34062a0b791 in_seq=0 out_seq=0 2023-10-06T10:43:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fad87fff700 1 -- 172.21.15.92:0/1686097004 <== mds.1 v2:172.21.15.92:6835/4047304116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55998e8f2e00 con 0x55998ea2b6d0 2023-10-06T10:43:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fad87fff700 10 client.7923 handle_command_reply: tid=0 2023-10-06T10:43:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fadb9388b80 1 client.7923 shutdown 2023-10-06T10:43:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fadb9388b80 1 -- 172.21.15.92:0/1686097004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea00bd0 msgr2=0x55998ea4a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fadb9388b80 1 --2- 172.21.15.92:0/1686097004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea00bd0 0x55998ea4a8a0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fada4008010 tx=0x7fada4026000 comp rx=0 tx=0).stop 2023-10-06T10:43:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fadb9388b80 1 -- 172.21.15.92:0/1686097004 shutdown_connections 2023-10-06T10:43:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fadb9388b80 1 --2- 172.21.15.92:0/1686097004 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x55998ea2b6d0 0x55998ea2baa0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fadb9388b80 1 --2- 172.21.15.92:0/1686097004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55998ea46a90 0x55998ea3c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fadb9388b80 1 --2- 172.21.15.92:0/1686097004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55998ea00bd0 0x55998ea4a8a0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fad87fff700 0 client.7923 ms_handle_reset on v2:172.21.15.92:6835/4047304116 2023-10-06T10:43:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fadb9388b80 1 --2- 172.21.15.92:0/1686097004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55998ea029a0 0x55998eaaea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fadb9388b80 1 -- 172.21.15.92:0/1686097004 >> 172.21.15.92:0/1686097004 conn(0x55998ea01070 msgr2=0x55998ea42ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.415+0000 7fadb9388b80 1 -- 172.21.15.92:0/1686097004 shutdown_connections 2023-10-06T10:43:59.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.416+0000 7fadb9388b80 1 -- 172.21.15.92:0/1686097004 wait complete. 2023-10-06T10:43:59.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.416+0000 7fadb9388b80 20 client.7923 trim_cache size 0 max 16384 2023-10-06T10:43:59.425 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:43:59.425 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:43:59.425 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 1, 2023-10-06T10:43:59.425 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 7689, 2023-10-06T10:43:59.425 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:43:59.425 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:43:59.425 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:43:59.426 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 2.8925712639999999, 2023-10-06T10:43:59.426 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 88, 2023-10-06T10:43:59.426 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 90, 2023-10-06T10:43:59.426 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 90, 2023-10-06T10:43:59.426 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 17.073687183000001 2023-10-06T10:43:59.426 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:43:59.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.416+0000 7fadb0090700 1 -- 172.21.15.92:0/2341715209 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fad8c066f30 msgr2=0x7fad8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.416+0000 7fadb0090700 1 --2- 172.21.15.92:0/2341715209 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fad8c066f30 0x7fad8c0693e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fada4006aa0 tx=0x7fada4008040 comp rx=0 tx=0).stop 2023-10-06T10:43:59.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.416+0000 7fadb0090700 1 -- 172.21.15.92:0/2341715209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada815ad80 msgr2=0x7fada815b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.416+0000 7fadb0090700 1 --2- 172.21.15.92:0/2341715209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada815ad80 0x7fada815b1f0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fad940046d0 tx=0x7fad9801a000 comp rx=0 tx=0).stop 2023-10-06T10:43:59.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.416+0000 7fadb0090700 1 -- 172.21.15.92:0/2341715209 shutdown_connections 2023-10-06T10:43:59.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.416+0000 7fadb0090700 1 --2- 172.21.15.92:0/2341715209 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fad8c066f30 0x7fad8c0693e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.416+0000 7fadb0090700 1 --2- 172.21.15.92:0/2341715209 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fada80cc970 0x7fada80ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.416+0000 7fadb0090700 1 --2- 172.21.15.92:0/2341715209 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad94004810 0x7fada816d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.416+0000 7fadb0090700 1 --2- 172.21.15.92:0/2341715209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada815ad80 0x7fada815b1f0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.416+0000 7fadb0090700 1 -- 172.21.15.92:0/2341715209 >> 172.21.15.92:0/2341715209 conn(0x7fada80997f0 msgr2=0x7fada81576a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:59.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.416+0000 7fadb0090700 1 -- 172.21.15.92:0/2341715209 shutdown_connections 2023-10-06T10:43:59.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.416+0000 7fadb0090700 1 -- 172.21.15.92:0/2341715209 wait complete. 2023-10-06T10:43:59.430 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T10:43:59.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.542+0000 7fa4db2da700 1 Processor -- start 2023-10-06T10:43:59.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.542+0000 7fa4db2da700 1 -- start start 2023-10-06T10:43:59.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.543+0000 7fa4db2da700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa4d40b25e0 0x7fa4d40b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.543+0000 7fa4db2da700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa4d40b29b0 0x7fa4d4159120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.543+0000 7fa4db2da700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4d40cbfa0 con 0x7fa4d4159660 2023-10-06T10:43:59.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.543+0000 7fa4db2da700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4d40cb390 con 0x7fa4d40b25e0 2023-10-06T10:43:59.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.543+0000 7fa4db2da700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4d40cb670 con 0x7fa4d40b29b0 2023-10-06T10:43:59.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.543+0000 7fa4d9076700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa4d40b25e0 0x7fa4d40b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.543+0000 7fa4d9877700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa4d4159660 0x7fa4d4068540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57294/0 (socket says 172.21.15.92:57294) 2023-10-06T10:43:59.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.543+0000 7fa4d9877700 1 -- 172.21.15.92:0/3653504969 learned_addr learned my addr 172.21.15.92:0/3653504969 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:59.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.544+0000 7fa4d8875700 1 --2- 172.21.15.92:0/3653504969 >> v2:172.21.15.124:3300/0 conn(0x7fa4d40b29b0 0x7fa4d4159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.544+0000 7fa4d9076700 1 -- 172.21.15.92:0/3653504969 >> v2:172.21.15.124:3300/0 conn(0x7fa4d40b29b0 msgr2=0x7fa4d4159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.544+0000 7fa4d9076700 1 --2- 172.21.15.92:0/3653504969 >> v2:172.21.15.124:3300/0 conn(0x7fa4d40b29b0 0x7fa4d4159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.544+0000 7fa4cbfff700 1 -- 172.21.15.92:0/3653504969 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1942512647 0 0) 0x7fa4d40cbfa0 con 0x7fa4d4159660 2023-10-06T10:43:59.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.544+0000 7fa4d9076700 1 -- 172.21.15.92:0/3653504969 >> v1:172.21.15.92:6789/0 conn(0x7fa4d4159660 legacy=0x7fa4d4068540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.544+0000 7fa4d9076700 1 -- 172.21.15.92:0/3653504969 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d40cb950 con 0x7fa4d40b25e0 2023-10-06T10:43:59.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.544+0000 7fa4d8875700 1 --2- 172.21.15.92:0/3653504969 >> v2:172.21.15.124:3300/0 conn(0x7fa4d40b29b0 0x7fa4d4159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:59.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.544+0000 7fa4d9076700 1 --2- 172.21.15.92:0/3653504969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4d40b25e0 0x7fa4d40b0690 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40cab10 tx=0x7fa4d0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b063725fe05c6f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.544+0000 7fa4cbfff700 1 -- 172.21.15.92:0/3653504969 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0005c10 con 0x7fa4d40b25e0 2023-10-06T10:43:59.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.544+0000 7fa4cbfff700 1 -- 172.21.15.92:0/3653504969 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d0005d70 con 0x7fa4d40b25e0 2023-10-06T10:43:59.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.544+0000 7fa4cbfff700 1 -- 172.21.15.92:0/3653504969 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0006260 con 0x7fa4d40b25e0 2023-10-06T10:43:59.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.544+0000 7fa4db2da700 1 -- 172.21.15.92:0/3653504969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4d40b25e0 msgr2=0x7fa4d40b0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.544+0000 7fa4db2da700 1 --2- 172.21.15.92:0/3653504969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4d40b25e0 0x7fa4d40b0690 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40cab10 tx=0x7fa4d0009120 comp rx=0 tx=0).stop 2023-10-06T10:43:59.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.545+0000 7fa4db2da700 1 -- 172.21.15.92:0/3653504969 shutdown_connections 2023-10-06T10:43:59.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.545+0000 7fa4db2da700 1 --2- 172.21.15.92:0/3653504969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4d40b25e0 0x7fa4d40b0690 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.545+0000 7fa4db2da700 1 --2- 172.21.15.92:0/3653504969 >> v2:172.21.15.124:3300/0 conn(0x7fa4d40b29b0 0x7fa4d4159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.545+0000 7fa4db2da700 1 -- 172.21.15.92:0/3653504969 >> 172.21.15.92:0/3653504969 conn(0x7fa4d406ece0 msgr2=0x7fa4d406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:59.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.545+0000 7fa4db2da700 1 -- 172.21.15.92:0/3653504969 shutdown_connections 2023-10-06T10:43:59.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.545+0000 7fa4db2da700 1 -- 172.21.15.92:0/3653504969 wait complete. 2023-10-06T10:43:59.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.546+0000 7fa4db2da700 1 Processor -- start 2023-10-06T10:43:59.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.546+0000 7fa4db2da700 1 -- start start 2023-10-06T10:43:59.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.547+0000 7fa4db2da700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4d40b25e0 0x7fa4d40cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.547+0000 7fa4db2da700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4d40b29b0 0x7fa4d4146440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.547+0000 7fa4db2da700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4d4159660 0x7fa4d4146980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.547+0000 7fa4db2da700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d41555c0 con 0x7fa4d40b25e0 2023-10-06T10:43:59.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.547+0000 7fa4db2da700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d4155860 con 0x7fa4d4159660 2023-10-06T10:43:59.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.547+0000 7fa4db2da700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d4155b40 con 0x7fa4d40b29b0 2023-10-06T10:43:59.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.547+0000 7fa4d9076700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4d40b25e0 0x7fa4d40cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.547+0000 7fa4d9076700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4d40b25e0 0x7fa4d40cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36452/0 (socket says 172.21.15.92:36452) 2023-10-06T10:43:59.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.547+0000 7fa4d9076700 1 -- 172.21.15.92:0/2095050521 learned_addr learned my addr 172.21.15.92:0/2095050521 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:59.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.547+0000 7fa4d8875700 1 --2- 172.21.15.92:0/2095050521 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4d40b29b0 0x7fa4d4146440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.547+0000 7fa4d9877700 1 --2- 172.21.15.92:0/2095050521 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4d4159660 0x7fa4d4146980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.547+0000 7fa4d9076700 1 -- 172.21.15.92:0/2095050521 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4d40b29b0 msgr2=0x7fa4d4146440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.547+0000 7fa4d9076700 1 --2- 172.21.15.92:0/2095050521 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4d40b29b0 0x7fa4d4146440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.547+0000 7fa4d9076700 1 -- 172.21.15.92:0/2095050521 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4d4159660 msgr2=0x7fa4d4146980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.547+0000 7fa4d9076700 1 --2- 172.21.15.92:0/2095050521 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4d4159660 0x7fa4d4146980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.547+0000 7fa4d9076700 1 -- 172.21.15.92:0/2095050521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d00093c0 con 0x7fa4d40b25e0 2023-10-06T10:43:59.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.547+0000 7fa4d9076700 1 --2- 172.21.15.92:0/2095050521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4d40b25e0 0x7fa4d40cc680 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4069690 tx=0x7fa4d0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.547+0000 7fa4c97fa700 1 -- 172.21.15.92:0/2095050521 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d000bc60 con 0x7fa4d40b25e0 2023-10-06T10:43:59.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.547+0000 7fa4c97fa700 1 -- 172.21.15.92:0/2095050521 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d000bde0 con 0x7fa4d40b25e0 2023-10-06T10:43:59.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.547+0000 7fa4db2da700 1 -- 172.21.15.92:0/2095050521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4d4155e20 con 0x7fa4d40b25e0 2023-10-06T10:43:59.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.547+0000 7fa4c97fa700 1 -- 172.21.15.92:0/2095050521 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d001a670 con 0x7fa4d40b25e0 2023-10-06T10:43:59.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.548+0000 7fa4db2da700 1 -- 172.21.15.92:0/2095050521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d40ccbc0 con 0x7fa4d40b25e0 2023-10-06T10:43:59.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.549+0000 7fa4c97fa700 1 -- 172.21.15.92:0/2095050521 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa4d001a9e0 con 0x7fa4d40b25e0 2023-10-06T10:43:59.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.549+0000 7fa4c97fa700 1 --2- 172.21.15.92:0/2095050521 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa4b8066f30 0x7fa4b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.549+0000 7fa4d8875700 1 --2- 172.21.15.92:0/2095050521 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa4b8066f30 0x7fa4b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.549+0000 7fa4c97fa700 1 -- 172.21.15.92:0/2095050521 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa4d00862e0 con 0x7fa4d40b25e0 2023-10-06T10:43:59.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.550+0000 7fa4d8875700 1 --2- 172.21.15.92:0/2095050521 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa4b8066f30 0x7fa4b80693e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40c6f20 tx=0x7fa4c4006f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.551+0000 7fa4e45d2b80 1 Processor -- start 2023-10-06T10:43:59.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.551+0000 7fa4e45d2b80 1 -- start start 2023-10-06T10:43:59.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.552+0000 7fa4e45d2b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c3b115da90 0x55c3b115de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.552+0000 7fa4e45d2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b11199a0 0x55c3b1116580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.552+0000 7fa4e45d2b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c3b1117bd0 0x55c3b1116ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.552+0000 7fa4e45d2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b1096c80 con 0x55c3b11199a0 2023-10-06T10:43:59.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.552+0000 7fa4e45d2b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b106da70 con 0x55c3b1117bd0 2023-10-06T10:43:59.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.552+0000 7fa4e45d2b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c3b106b270 con 0x55c3b115da90 2023-10-06T10:43:59.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.552+0000 7fa4d8875700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b11199a0 0x55c3b1116580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.552+0000 7fa4d8875700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b11199a0 0x55c3b1116580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36456/0 (socket says 172.21.15.92:36456) 2023-10-06T10:43:59.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.552+0000 7fa4d8875700 1 -- 172.21.15.92:0/350478819 learned_addr learned my addr 172.21.15.92:0/350478819 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:59.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.552+0000 7fa4d9877700 1 --2- 172.21.15.92:0/350478819 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c3b1117bd0 0x55c3b1116ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.552+0000 7fa4d9076700 1 --2- 172.21.15.92:0/350478819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c3b115da90 0x55c3b115de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.552+0000 7fa4d8875700 1 -- 172.21.15.92:0/350478819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c3b115da90 msgr2=0x55c3b115de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.552+0000 7fa4d8875700 1 --2- 172.21.15.92:0/350478819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c3b115da90 0x55c3b115de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.552+0000 7fa4d8875700 1 -- 172.21.15.92:0/350478819 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c3b1117bd0 msgr2=0x55c3b1116ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.552+0000 7fa4d8875700 1 --2- 172.21.15.92:0/350478819 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c3b1117bd0 0x55c3b1116ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.552+0000 7fa4d8875700 1 -- 172.21.15.92:0/350478819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3b0f01270 con 0x55c3b11199a0 2023-10-06T10:43:59.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.552+0000 7fa4d8875700 1 --2- 172.21.15.92:0/350478819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b11199a0 0x55c3b1116580 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400feb0 tx=0x7fa4c4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66adb49f11a54487 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.553+0000 7fa4be7fc700 1 -- 172.21.15.92:0/350478819 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4013ca0 con 0x55c3b11199a0 2023-10-06T10:43:59.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.553+0000 7fa4be7fc700 1 -- 172.21.15.92:0/350478819 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c4019070 con 0x55c3b11199a0 2023-10-06T10:43:59.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.553+0000 7fa4be7fc700 1 -- 172.21.15.92:0/350478819 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4015660 con 0x55c3b11199a0 2023-10-06T10:43:59.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.553+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/350478819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b11199a0 msgr2=0x55c3b1116580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.553+0000 7fa4e45d2b80 1 --2- 172.21.15.92:0/350478819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b11199a0 0x55c3b1116580 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400feb0 tx=0x7fa4c4014000 comp rx=0 tx=0).stop 2023-10-06T10:43:59.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.553+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/350478819 shutdown_connections 2023-10-06T10:43:59.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.553+0000 7fa4e45d2b80 1 --2- 172.21.15.92:0/350478819 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c3b1117bd0 0x55c3b1116ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.553+0000 7fa4e45d2b80 1 --2- 172.21.15.92:0/350478819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c3b115da90 0x55c3b115de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.553+0000 7fa4e45d2b80 1 --2- 172.21.15.92:0/350478819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b11199a0 0x55c3b1116580 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.553+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/350478819 >> 172.21.15.92:0/350478819 conn(0x55c3b1118070 msgr2=0x55c3b1118440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:59.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.553+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/350478819 shutdown_connections 2023-10-06T10:43:59.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.553+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/350478819 wait complete. 2023-10-06T10:43:59.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.554+0000 7fa4e45d2b80 1 -- start start 2023-10-06T10:43:59.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.554+0000 7fa4e45d2b80 1 Processor -- start 2023-10-06T10:43:59.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4e45d2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b1117bd0 0x55c3b113d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4e45d2b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c3b11199a0 0x55c3b1170080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4e45d2b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c3b115da90 0x55c3b113ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4e45d2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b1067060 con 0x55c3b1117bd0 2023-10-06T10:43:59.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4e45d2b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b1012890 con 0x55c3b11199a0 2023-10-06T10:43:59.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4e45d2b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c3b102dea0 con 0x55c3b115da90 2023-10-06T10:43:59.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4e45d2b80 10 client.0 _ll_register_callbacks cb 0x55c3b0f03f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:59.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4d9076700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b1117bd0 0x55c3b113d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4d9076700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b1117bd0 0x55c3b113d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36464/0 (socket says 172.21.15.92:36464) 2023-10-06T10:43:59.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4d9076700 1 -- 172.21.15.92:0/3096282918 learned_addr learned my addr 172.21.15.92:0/3096282918 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:59.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4d8875700 1 --2- 172.21.15.92:0/3096282918 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c3b11199a0 0x55c3b1170080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4d9877700 1 --2- 172.21.15.92:0/3096282918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c3b115da90 0x55c3b113ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4d9076700 1 -- 172.21.15.92:0/3096282918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c3b115da90 msgr2=0x55c3b113ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4d9076700 1 --2- 172.21.15.92:0/3096282918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c3b115da90 0x55c3b113ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4d9076700 1 -- 172.21.15.92:0/3096282918 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c3b11199a0 msgr2=0x55c3b1170080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4d9076700 1 --2- 172.21.15.92:0/3096282918 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c3b11199a0 0x55c3b1170080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4d9076700 1 -- 172.21.15.92:0/3096282918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c400fae0 con 0x55c3b1117bd0 2023-10-06T10:43:59.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4d8875700 1 --2- 172.21.15.92:0/3096282918 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c3b11199a0 0x55c3b1170080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:59.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4d9877700 1 --2- 172.21.15.92:0/3096282918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c3b115da90 0x55c3b113ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:59.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4d9076700 1 --2- 172.21.15.92:0/3096282918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b1117bd0 0x55c3b113d550 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000b0e0 tx=0x7fa4d005c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8de3ee7aa7b2ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4b37fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:59.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4b37fe700 1 -- 172.21.15.92:0/3096282918 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d00532b0 con 0x55c3b1117bd0 2023-10-06T10:43:59.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4b37fe700 1 -- 172.21.15.92:0/3096282918 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa4d005a1c0 con 0x55c3b1117bd0 2023-10-06T10:43:59.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.555+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/3096282918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3b110b690 con 0x55c3b1117bd0 2023-10-06T10:43:59.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.556+0000 7fa4b37fe700 1 -- 172.21.15.92:0/3096282918 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4d005c2c0 con 0x55c3b1117bd0 2023-10-06T10:43:59.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.556+0000 7fa4e45d2b80 10 client.7938 fetch_fsmap learned FSMap version 88 2023-10-06T10:43:59.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.556+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/3096282918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x55c3b0ea3fe0 con 0x55c3b1117bd0 2023-10-06T10:43:59.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.556+0000 7fa4b37fe700 1 -- 172.21.15.92:0/3096282918 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d0082020 con 0x55c3b1117bd0 2023-10-06T10:43:59.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.556+0000 7fa4b37fe700 1 -- 172.21.15.92:0/3096282918 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 88) v1 ==== 3124+0+0 (secure 0 0 0) 0x7fa4d006d170 con 0x55c3b1117bd0 2023-10-06T10:43:59.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.556+0000 7fa4e45d2b80 10 client.7938 fetch_fsmap finished waiting for FSMap version 88 2023-10-06T10:43:59.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.556+0000 7fa4e45d2b80 10 client.7938 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T10:43:59.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.556+0000 7fa4e45d2b80 20 client.7938 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:59.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.556+0000 7fa4e45d2b80 1 --2- 172.21.15.92:0/3096282918 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x55c3b1158330 0x55c3b1158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.556+0000 7fa4e45d2b80 4 client.7938 mds_command: new command op to 7701 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:43:59.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.556+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/3096282918 --> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3b10e85c0 con 0x55c3b1158330 2023-10-06T10:43:59.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.556+0000 7fa4d8875700 1 --2- 172.21.15.92:0/3096282918 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x55c3b1158330 0x55c3b1158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.557+0000 7fa4b37fe700 10 client.7938 ms_handle_connect on v2:172.21.15.124:6832/4203118992 2023-10-06T10:43:59.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.557+0000 7fa4d8875700 1 --2- 172.21.15.92:0/3096282918 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x55c3b1158330 0x55c3b1158700 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fba2bd5783f2560b server_cookie=a42f52b7ad048525 in_seq=0 out_seq=0 2023-10-06T10:43:59.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.557+0000 7fa4b37fe700 1 -- 172.21.15.92:0/3096282918 <== mds.0 v2:172.21.15.124:6832/4203118992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55c3b10e85c0 con 0x55c3b1158330 2023-10-06T10:43:59.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.557+0000 7fa4b37fe700 10 client.7938 handle_command_reply: tid=0 2023-10-06T10:43:59.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.557+0000 7fa4e45d2b80 1 client.7938 shutdown 2023-10-06T10:43:59.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.557+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/3096282918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b1117bd0 msgr2=0x55c3b113d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.557+0000 7fa4e45d2b80 1 --2- 172.21.15.92:0/3096282918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b1117bd0 0x55c3b113d550 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000b0e0 tx=0x7fa4d005c040 comp rx=0 tx=0).stop 2023-10-06T10:43:59.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.557+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/3096282918 shutdown_connections 2023-10-06T10:43:59.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.557+0000 7fa4e45d2b80 1 --2- 172.21.15.92:0/3096282918 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x55c3b1158330 0x55c3b1158700 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.557+0000 7fa4e45d2b80 1 --2- 172.21.15.92:0/3096282918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c3b115da90 0x55c3b113ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.557+0000 7fa4b37fe700 0 client.7938 ms_handle_reset on v2:172.21.15.124:6832/4203118992 2023-10-06T10:43:59.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.557+0000 7fa4e45d2b80 1 --2- 172.21.15.92:0/3096282918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b1117bd0 0x55c3b113d550 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.557+0000 7fa4e45d2b80 1 --2- 172.21.15.92:0/3096282918 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c3b11199a0 0x55c3b1170080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.557+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/3096282918 >> 172.21.15.92:0/3096282918 conn(0x55c3b1118070 msgr2=0x55c3b115b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:59.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.558+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/3096282918 shutdown_connections 2023-10-06T10:43:59.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.558+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/3096282918 wait complete. 2023-10-06T10:43:59.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.558+0000 7fa4e45d2b80 20 client.7938 trim_cache size 0 max 16384 2023-10-06T10:43:59.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.574+0000 7fa4e45d2b80 1 Processor -- start 2023-10-06T10:43:59.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.574+0000 7fa4e45d2b80 1 -- start start 2023-10-06T10:43:59.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.575+0000 7fa4e45d2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b1117bd0 0x55c3b1112850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.575+0000 7fa4e45d2b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c3b11199a0 0x55c3b1116580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.575+0000 7fa4e45d2b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c3b1158330 0x55c3b1116ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.575+0000 7fa4e45d2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b0ee9e30 con 0x55c3b1117bd0 2023-10-06T10:43:59.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.575+0000 7fa4e45d2b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b1157c00 con 0x55c3b1158330 2023-10-06T10:43:59.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.575+0000 7fa4e45d2b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c3b1159860 con 0x55c3b11199a0 2023-10-06T10:43:59.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.575+0000 7fa4d9076700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b1117bd0 0x55c3b1112850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.575+0000 7fa4d9076700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b1117bd0 0x55c3b1112850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36472/0 (socket says 172.21.15.92:36472) 2023-10-06T10:43:59.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.575+0000 7fa4d9076700 1 -- 172.21.15.92:0/1278937499 learned_addr learned my addr 172.21.15.92:0/1278937499 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:59.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.575+0000 7fa4d8875700 1 --2- 172.21.15.92:0/1278937499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c3b11199a0 0x55c3b1116580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.575+0000 7fa4d9076700 1 -- 172.21.15.92:0/1278937499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c3b11199a0 msgr2=0x55c3b1116580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.575+0000 7fa4d9877700 1 --2- 172.21.15.92:0/1278937499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c3b1158330 0x55c3b1116ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.575+0000 7fa4d9076700 1 --2- 172.21.15.92:0/1278937499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c3b11199a0 0x55c3b1116580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.575+0000 7fa4d9076700 1 -- 172.21.15.92:0/1278937499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c3b1158330 msgr2=0x55c3b1116ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.575+0000 7fa4d9076700 1 --2- 172.21.15.92:0/1278937499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c3b1158330 0x55c3b1116ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.575+0000 7fa4d9076700 1 -- 172.21.15.92:0/1278937499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3b1035780 con 0x55c3b1117bd0 2023-10-06T10:43:59.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.575+0000 7fa4d8875700 1 --2- 172.21.15.92:0/1278937499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c3b11199a0 0x55c3b1116580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:59.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.576+0000 7fa4d9076700 1 --2- 172.21.15.92:0/1278937499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b1117bd0 0x55c3b1112850 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0006aa0 tx=0x7fa4d0008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527b773d85b64b7b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.576+0000 7fa4beffd700 1 -- 172.21.15.92:0/1278937499 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d00532b0 con 0x55c3b1117bd0 2023-10-06T10:43:59.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.576+0000 7fa4beffd700 1 -- 172.21.15.92:0/1278937499 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d0082070 con 0x55c3b1117bd0 2023-10-06T10:43:59.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.576+0000 7fa4beffd700 1 -- 172.21.15.92:0/1278937499 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d005c2c0 con 0x55c3b1117bd0 2023-10-06T10:43:59.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.576+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/1278937499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b1117bd0 msgr2=0x55c3b1112850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.576+0000 7fa4e45d2b80 1 --2- 172.21.15.92:0/1278937499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b1117bd0 0x55c3b1112850 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0006aa0 tx=0x7fa4d0008000 comp rx=0 tx=0).stop 2023-10-06T10:43:59.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.576+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/1278937499 shutdown_connections 2023-10-06T10:43:59.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.576+0000 7fa4e45d2b80 1 --2- 172.21.15.92:0/1278937499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c3b1158330 0x55c3b1116ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.576+0000 7fa4e45d2b80 1 --2- 172.21.15.92:0/1278937499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b1117bd0 0x55c3b1112850 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.576+0000 7fa4e45d2b80 1 --2- 172.21.15.92:0/1278937499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c3b11199a0 0x55c3b1116580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.576+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/1278937499 >> 172.21.15.92:0/1278937499 conn(0x55c3b1118070 msgr2=0x55c3b115c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:59.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.576+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/1278937499 shutdown_connections 2023-10-06T10:43:59.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.576+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/1278937499 wait complete. 2023-10-06T10:43:59.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.577+0000 7fa4e45d2b80 1 -- start start 2023-10-06T10:43:59.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.577+0000 7fa4e45d2b80 1 Processor -- start 2023-10-06T10:43:59.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4e45d2b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c3b1117bd0 0x55c3b1145aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4e45d2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b11199a0 0x55c3b1145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4e45d2b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c3b1158330 0x55c3b1141440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4e45d2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b1154720 con 0x55c3b11199a0 2023-10-06T10:43:59.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4e45d2b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b1111cc0 con 0x55c3b1158330 2023-10-06T10:43:59.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4e45d2b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c3b1116580 con 0x55c3b1117bd0 2023-10-06T10:43:59.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4e45d2b80 10 client.0 _ll_register_callbacks cb 0x55c3b0f03f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:43:59.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4d8875700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b11199a0 0x55c3b1145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4d8875700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b11199a0 0x55c3b1145fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36486/0 (socket says 172.21.15.92:36486) 2023-10-06T10:43:59.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4d8875700 1 -- 172.21.15.92:0/3672127748 learned_addr learned my addr 172.21.15.92:0/3672127748 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:59.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4d9076700 1 --2- 172.21.15.92:0/3672127748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c3b1117bd0 0x55c3b1145aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4d9877700 1 --2- 172.21.15.92:0/3672127748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c3b1158330 0x55c3b1141440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4d8875700 1 -- 172.21.15.92:0/3672127748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c3b1117bd0 msgr2=0x55c3b1145aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4d8875700 1 --2- 172.21.15.92:0/3672127748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c3b1117bd0 0x55c3b1145aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4d8875700 1 -- 172.21.15.92:0/3672127748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c3b1158330 msgr2=0x55c3b1141440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4d8875700 1 --2- 172.21.15.92:0/3672127748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c3b1158330 0x55c3b1141440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4d8875700 1 -- 172.21.15.92:0/3672127748 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3b0ea3fe0 con 0x55c3b11199a0 2023-10-06T10:43:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4d9877700 1 --2- 172.21.15.92:0/3672127748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c3b1158330 0x55c3b1141440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:43:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4d8875700 1 --2- 172.21.15.92:0/3672127748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b11199a0 0x55c3b1145fe0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400ab90 tx=0x7fa4c4031250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3124c34c04910dad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4b37fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:43:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4b37fe700 1 -- 172.21.15.92:0/3672127748 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4015450 con 0x55c3b11199a0 2023-10-06T10:43:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4b37fe700 1 -- 172.21.15.92:0/3672127748 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa4c40156b0 con 0x55c3b11199a0 2023-10-06T10:43:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.578+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/3672127748 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4c4019070 con 0x55c3b11199a0 2023-10-06T10:43:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.579+0000 7fa4b37fe700 1 -- 172.21.15.92:0/3672127748 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4c400a5f0 con 0x55c3b11199a0 2023-10-06T10:43:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.579+0000 7fa4e45d2b80 10 client.7944 fetch_fsmap learned FSMap version 88 2023-10-06T10:43:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.579+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/3672127748 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x55c3b101c470 con 0x55c3b11199a0 2023-10-06T10:43:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.579+0000 7fa4b37fe700 1 -- 172.21.15.92:0/3672127748 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c4019070 con 0x55c3b11199a0 2023-10-06T10:43:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.579+0000 7fa4b37fe700 1 -- 172.21.15.92:0/3672127748 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 88) v1 ==== 3124+0+0 (secure 0 0 0) 0x7fa4c4005a00 con 0x55c3b11199a0 2023-10-06T10:43:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.579+0000 7fa4e45d2b80 10 client.7944 fetch_fsmap finished waiting for FSMap version 88 2023-10-06T10:43:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.579+0000 7fa4e45d2b80 10 client.7944 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T10:43:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.579+0000 7fa4e45d2b80 20 client.7944 populate_metadata read hostname 'smithi092' 2023-10-06T10:43:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.579+0000 7fa4e45d2b80 1 --2- 172.21.15.92:0/3672127748 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x55c3b115da90 0x55c3b1141b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.579+0000 7fa4e45d2b80 4 client.7944 mds_command: new command op to 7701 tid=0[{"prefix": "status"}] 2023-10-06T10:43:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.579+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/3672127748 --> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c3b1009e00 con 0x55c3b115da90 2023-10-06T10:43:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.579+0000 7fa4d9076700 1 --2- 172.21.15.92:0/3672127748 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x55c3b115da90 0x55c3b1141b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.579+0000 7fa4b37fe700 10 client.7944 ms_handle_connect on v2:172.21.15.124:6832/4203118992 2023-10-06T10:43:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.579+0000 7fa4d9076700 1 --2- 172.21.15.92:0/3672127748 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x55c3b115da90 0x55c3b1141b10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=238da975c4aac43b server_cookie=ef8875241167103e in_seq=0 out_seq=0 2023-10-06T10:43:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4b37fe700 1 -- 172.21.15.92:0/3672127748 <== mds.0 v2:172.21.15.124:6832/4203118992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x55c3b1009e00 con 0x55c3b115da90 2023-10-06T10:43:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4b37fe700 10 client.7944 handle_command_reply: tid=0 2023-10-06T10:43:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4e45d2b80 1 client.7944 shutdown 2023-10-06T10:43:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/3672127748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b11199a0 msgr2=0x55c3b1145fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4e45d2b80 1 --2- 172.21.15.92:0/3672127748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b11199a0 0x55c3b1145fe0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400ab90 tx=0x7fa4c4031250 comp rx=0 tx=0).stop 2023-10-06T10:43:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/3672127748 shutdown_connections 2023-10-06T10:43:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4e45d2b80 1 --2- 172.21.15.92:0/3672127748 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x55c3b115da90 0x55c3b1141b10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4e45d2b80 1 --2- 172.21.15.92:0/3672127748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c3b1158330 0x55c3b1141440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4e45d2b80 1 --2- 172.21.15.92:0/3672127748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c3b1117bd0 0x55c3b1145aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4e45d2b80 1 --2- 172.21.15.92:0/3672127748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3b11199a0 0x55c3b1145fe0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4b37fe700 0 client.7944 ms_handle_reset on v2:172.21.15.124:6832/4203118992 2023-10-06T10:43:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/3672127748 >> 172.21.15.92:0/3672127748 conn(0x55c3b1118070 msgr2=0x55c3b116ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/3672127748 shutdown_connections 2023-10-06T10:43:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4e45d2b80 1 -- 172.21.15.92:0/3672127748 wait complete. 2023-10-06T10:43:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4e45d2b80 20 client.7944 trim_cache size 0 max 16384 2023-10-06T10:43:59.590 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:43:59.590 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:43:59.590 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T10:43:59.590 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 7701, 2023-10-06T10:43:59.590 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:43:59.590 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:43:59.590 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:43:59.590 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 11.093358533, 2023-10-06T10:43:59.590 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 88, 2023-10-06T10:43:59.590 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 90, 2023-10-06T10:43:59.591 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 90, 2023-10-06T10:43:59.591 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 17.222547743 2023-10-06T10:43:59.591 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:43:59.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4db2da700 1 -- 172.21.15.92:0/2095050521 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa4b8066f30 msgr2=0x7fa4b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4db2da700 1 --2- 172.21.15.92:0/2095050521 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa4b8066f30 0x7fa4b80693e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40c6f20 tx=0x7fa4c4006f90 comp rx=0 tx=0).stop 2023-10-06T10:43:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4db2da700 1 -- 172.21.15.92:0/2095050521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4d40b25e0 msgr2=0x7fa4d40cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4db2da700 1 --2- 172.21.15.92:0/2095050521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4d40b25e0 0x7fa4d40cc680 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4069690 tx=0x7fa4d0016000 comp rx=0 tx=0).stop 2023-10-06T10:43:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4db2da700 1 -- 172.21.15.92:0/2095050521 shutdown_connections 2023-10-06T10:43:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4db2da700 1 --2- 172.21.15.92:0/2095050521 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa4b8066f30 0x7fa4b80693e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4db2da700 1 --2- 172.21.15.92:0/2095050521 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4d4159660 0x7fa4d4146980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4db2da700 1 --2- 172.21.15.92:0/2095050521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4d40b25e0 0x7fa4d40cc680 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4db2da700 1 --2- 172.21.15.92:0/2095050521 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4d40b29b0 0x7fa4d4146440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4db2da700 1 -- 172.21.15.92:0/2095050521 >> 172.21.15.92:0/2095050521 conn(0x7fa4d406ece0 msgr2=0x7fa4d415c950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4db2da700 1 -- 172.21.15.92:0/2095050521 shutdown_connections 2023-10-06T10:43:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.580+0000 7fa4db2da700 1 -- 172.21.15.92:0/2095050521 wait complete. 2023-10-06T10:43:59.594 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:43:59.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.706+0000 7ff6edeff700 1 Processor -- start 2023-10-06T10:43:59.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.706+0000 7ff6edeff700 1 -- start start 2023-10-06T10:43:59.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.707+0000 7ff6edeff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff6e8159560 0x7ff6e80aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.707+0000 7ff6edeff700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff6e80b0760 0x7ff6e80b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.707+0000 7ff6edeff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e80b0c60 con 0x7ff6e80b49a0 2023-10-06T10:43:59.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.707+0000 7ff6edeff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e80b0f40 con 0x7ff6e8159560 2023-10-06T10:43:59.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.707+0000 7ff6edeff700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e80b1220 con 0x7ff6e80b0760 2023-10-06T10:43:59.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.707+0000 7ff6e77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff6e8159560 0x7ff6e80aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.707+0000 7ff6dffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff6e80b49a0 0x7ff6e80aaa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57302/0 (socket says 172.21.15.92:57302) 2023-10-06T10:43:59.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.707+0000 7ff6dffff700 1 -- 172.21.15.92:0/586131717 learned_addr learned my addr 172.21.15.92:0/586131717 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:43:59.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.707+0000 7ff6e77fe700 1 -- 172.21.15.92:0/586131717 >> v2:172.21.15.124:3300/0 conn(0x7ff6e80b0760 msgr2=0x7ff6e80b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:43:59.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.707+0000 7ff6e7fff700 1 --2- 172.21.15.92:0/586131717 >> v2:172.21.15.124:3300/0 conn(0x7ff6e80b0760 0x7ff6e80b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.707+0000 7ff6e77fe700 1 --2- 172.21.15.92:0/586131717 >> v2:172.21.15.124:3300/0 conn(0x7ff6e80b0760 0x7ff6e80b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.707+0000 7ff6e77fe700 1 -- 172.21.15.92:0/586131717 >> v1:172.21.15.92:6789/0 conn(0x7ff6e80b49a0 legacy=0x7ff6e80aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.707+0000 7ff6e77fe700 1 -- 172.21.15.92:0/586131717 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e80b1500 con 0x7ff6e8159560 2023-10-06T10:43:59.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.707+0000 7ff6e6ffd700 1 -- 172.21.15.92:0/586131717 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1497927510 0 0) 0x7ff6e80b0c60 con 0x7ff6e80b49a0 2023-10-06T10:43:59.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.707+0000 7ff6e7fff700 1 --2- 172.21.15.92:0/586131717 >> v2:172.21.15.124:3300/0 conn(0x7ff6e80b0760 0x7ff6e80b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:43:59.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.707+0000 7ff6e77fe700 1 --2- 172.21.15.92:0/586131717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6e8159560 0x7ff6e80aa530 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7ff6d80098d0 tx=0x7ff6d8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfed53292c5275a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.708+0000 7ff6e6ffd700 1 -- 172.21.15.92:0/586131717 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8009e70 con 0x7ff6e8159560 2023-10-06T10:43:59.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.708+0000 7ff6e6ffd700 1 -- 172.21.15.92:0/586131717 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d8005c10 con 0x7ff6e8159560 2023-10-06T10:43:59.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.708+0000 7ff6e6ffd700 1 -- 172.21.15.92:0/586131717 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d800b610 con 0x7ff6e8159560 2023-10-06T10:43:59.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.708+0000 7ff6edeff700 1 -- 172.21.15.92:0/586131717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6e8159560 msgr2=0x7ff6e80aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.708+0000 7ff6edeff700 1 --2- 172.21.15.92:0/586131717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6e8159560 0x7ff6e80aa530 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7ff6d80098d0 tx=0x7ff6d8009120 comp rx=0 tx=0).stop 2023-10-06T10:43:59.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.708+0000 7ff6edeff700 1 -- 172.21.15.92:0/586131717 shutdown_connections 2023-10-06T10:43:59.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.708+0000 7ff6edeff700 1 --2- 172.21.15.92:0/586131717 >> v2:172.21.15.124:3300/0 conn(0x7ff6e80b0760 0x7ff6e80b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.708+0000 7ff6edeff700 1 --2- 172.21.15.92:0/586131717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6e8159560 0x7ff6e80aa530 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.708+0000 7ff6edeff700 1 -- 172.21.15.92:0/586131717 >> 172.21.15.92:0/586131717 conn(0x7ff6e80997f0 msgr2=0x7ff6e806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:59.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.708+0000 7ff6edeff700 1 -- 172.21.15.92:0/586131717 shutdown_connections 2023-10-06T10:43:59.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.708+0000 7ff6edeff700 1 -- 172.21.15.92:0/586131717 wait complete. 2023-10-06T10:43:59.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.709+0000 7ff6edeff700 1 Processor -- start 2023-10-06T10:43:59.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.709+0000 7ff6edeff700 1 -- start start 2023-10-06T10:43:59.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.710+0000 7ff6edeff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e80b0760 0x7ff6e815a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.710+0000 7ff6edeff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6e80b49a0 0x7ff6e815cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.710+0000 7ff6edeff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6e8159560 0x7ff6e8158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.710+0000 7ff6edeff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e815bab0 con 0x7ff6e80b49a0 2023-10-06T10:43:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.710+0000 7ff6edeff700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e815a900 con 0x7ff6e8159560 2023-10-06T10:43:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.710+0000 7ff6edeff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e815abe0 con 0x7ff6e80b0760 2023-10-06T10:43:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.710+0000 7ff6dffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6e80b49a0 0x7ff6e815cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.710+0000 7ff6dffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6e80b49a0 0x7ff6e815cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36496/0 (socket says 172.21.15.92:36496) 2023-10-06T10:43:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.710+0000 7ff6dffff700 1 -- 172.21.15.92:0/251057413 learned_addr learned my addr 172.21.15.92:0/251057413 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:43:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.710+0000 7ff6e77fe700 1 --2- 172.21.15.92:0/251057413 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e80b0760 0x7ff6e815a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.710+0000 7ff6e7fff700 1 --2- 172.21.15.92:0/251057413 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6e8159560 0x7ff6e8158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.710+0000 7ff6dffff700 1 -- 172.21.15.92:0/251057413 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e80b0760 msgr2=0x7ff6e815a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.710+0000 7ff6dffff700 1 --2- 172.21.15.92:0/251057413 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e80b0760 0x7ff6e815a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.710+0000 7ff6dffff700 1 -- 172.21.15.92:0/251057413 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6e8159560 msgr2=0x7ff6e8158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.710+0000 7ff6dffff700 1 --2- 172.21.15.92:0/251057413 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6e8159560 0x7ff6e8158800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.710+0000 7ff6dffff700 1 -- 172.21.15.92:0/251057413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d80093c0 con 0x7ff6e80b49a0 2023-10-06T10:43:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.711+0000 7ff6dffff700 1 --2- 172.21.15.92:0/251057413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6e80b49a0 0x7ff6e815cd20 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000c7e0 tx=0x7ff6d0007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.711+0000 7ff6df7fe700 1 -- 172.21.15.92:0/251057413 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0005c00 con 0x7ff6e80b49a0 2023-10-06T10:43:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.711+0000 7ff6df7fe700 1 -- 172.21.15.92:0/251057413 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d0005d60 con 0x7ff6e80b49a0 2023-10-06T10:43:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.711+0000 7ff6edeff700 1 -- 172.21.15.92:0/251057413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e815aec0 con 0x7ff6e80b49a0 2023-10-06T10:43:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.711+0000 7ff6df7fe700 1 -- 172.21.15.92:0/251057413 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0007300 con 0x7ff6e80b49a0 2023-10-06T10:43:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.711+0000 7ff6edeff700 1 -- 172.21.15.92:0/251057413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e8158d40 con 0x7ff6e80b49a0 2023-10-06T10:43:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.712+0000 7ff6edeff700 1 -- 172.21.15.92:0/251057413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e8064a90 con 0x7ff6e80b49a0 2023-10-06T10:43:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.712+0000 7ff6df7fe700 1 -- 172.21.15.92:0/251057413 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff6d001b3b0 con 0x7ff6e80b49a0 2023-10-06T10:43:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.712+0000 7ff6df7fe700 1 --2- 172.21.15.92:0/251057413 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff6c8066f30 0x7ff6c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:43:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.713+0000 7ff6df7fe700 1 -- 172.21.15.92:0/251057413 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff6d0083c30 con 0x7ff6e80b49a0 2023-10-06T10:43:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.713+0000 7ff6e77fe700 1 --2- 172.21.15.92:0/251057413 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff6c8066f30 0x7ff6c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:43:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.713+0000 7ff6e77fe700 1 --2- 172.21.15.92:0/251057413 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff6c8066f30 0x7ff6c80693e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8009160 tx=0x7ff6d8016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:43:59.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.717+0000 7ff6df7fe700 1 -- 172.21.15.92:0/251057413 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6d0050c00 con 0x7ff6e80b49a0 2023-10-06T10:43:59.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.944+0000 7ff6edeff700 1 -- 172.21.15.92:0/251057413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6e80b17e0 con 0x7ff6e80b49a0 2023-10-06T10:43:59.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.944+0000 7ff6df7fe700 1 -- 172.21.15.92:0/251057413 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4633 (secure 0 0 0) 0x7ff6d0017020 con 0x7ff6e80b49a0 2023-10-06T10:43:59.947 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:43:59.947 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":88,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2122441982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2122441982},{"type":"v1","addr":"172.21.15.124:6835","nonce":2122441982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:47.850832+0000","modified":"2023-10-06T10:43:58.531567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7701,"mds_1":7689,"mds_2":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6836/3605297930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3605297930},{"type":"v1","addr":"172.21.15.92:6836","nonce":3605297930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7689":{"gid":7689,"name":"c","rank":1,"incarnation":85,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/4047304116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4047304116},{"type":"v1","addr":"172.21.15.92:6837","nonce":4047304116}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7701":{"gid":7701,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6834/4203118992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":4203118992},{"type":"v1","addr":"172.21.15.124:6834","nonce":4203118992}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T10:43:59.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.947+0000 7ff6edeff700 1 -- 172.21.15.92:0/251057413 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff6c8066f30 msgr2=0x7ff6c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.947+0000 7ff6edeff700 1 --2- 172.21.15.92:0/251057413 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff6c8066f30 0x7ff6c80693e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8009160 tx=0x7ff6d8016000 comp rx=0 tx=0).stop 2023-10-06T10:43:59.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.947+0000 7ff6edeff700 1 -- 172.21.15.92:0/251057413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6e80b49a0 msgr2=0x7ff6e815cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:43:59.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.947+0000 7ff6edeff700 1 --2- 172.21.15.92:0/251057413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6e80b49a0 0x7ff6e815cd20 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000c7e0 tx=0x7ff6d0007d90 comp rx=0 tx=0).stop 2023-10-06T10:43:59.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.947+0000 7ff6edeff700 1 -- 172.21.15.92:0/251057413 shutdown_connections 2023-10-06T10:43:59.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.947+0000 7ff6edeff700 1 --2- 172.21.15.92:0/251057413 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff6c8066f30 0x7ff6c80693e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.947+0000 7ff6edeff700 1 --2- 172.21.15.92:0/251057413 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6e8159560 0x7ff6e8158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.947+0000 7ff6edeff700 1 --2- 172.21.15.92:0/251057413 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e80b0760 0x7ff6e815a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.947+0000 7ff6edeff700 1 --2- 172.21.15.92:0/251057413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6e80b49a0 0x7ff6e815cd20 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:43:59.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.947+0000 7ff6edeff700 1 -- 172.21.15.92:0/251057413 >> 172.21.15.92:0/251057413 conn(0x7ff6e80997f0 msgr2=0x7ff6e80b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:43:59.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.947+0000 7ff6edeff700 1 -- 172.21.15.92:0/251057413 shutdown_connections 2023-10-06T10:43:59.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:43:59.947+0000 7ff6edeff700 1 -- 172.21.15.92:0/251057413 wait complete. 2023-10-06T10:43:59.950 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 88 2023-10-06T10:43:59.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:43:59.961 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2023-10-06T10:43:59.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:43:59.986 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2023-10-06T10:44:00.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:44:00.046 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2023-10-06T10:44:00.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:44:00.102 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2023-10-06T10:44:00.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:44:00.158 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2023-10-06T10:44:00.214 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-10-06T10:44:00.353 INFO:teuthology.orchestra.run.smithi124.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:44:00.354 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:44:00.354 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:44:02.355 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-10-06T10:44:02.496 INFO:teuthology.orchestra.run.smithi124.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:44:02.496 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:44:02.496 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:44:04.497 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-10-06T10:44:04.636 INFO:teuthology.orchestra.run.smithi124.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:44:04.636 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:44:04.637 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:44:06.637 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-10-06T10:44:06.778 INFO:teuthology.orchestra.run.smithi124.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:44:06.779 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:44:06.779 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:44:08.780 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-10-06T10:44:08.921 INFO:teuthology.orchestra.run.smithi124.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2023-10-06T10:44:08.922 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2023-10-06T10:44:08.923 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:44:08.923 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:44:09.061 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-10-06T10:44:09.061 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:44:09.062 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:44:11.063 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:44:11.211 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-10-06T10:44:11.212 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:44:11.212 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:44:13.213 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:44:13.351 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2023-10-06T10:44:13.352 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2023-10-06T10:44:13.352 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:44:13.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-10-06T10:44:13.482 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-10-06T10:44:13.483 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:44:13.483 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:44:13.483 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 flush journal 2023-10-06T10:44:13.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.600+0000 7ff0c5ccc700 1 Processor -- start 2023-10-06T10:44:13.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.600+0000 7ff0c5ccc700 1 -- start start 2023-10-06T10:44:13.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0c5ccc700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff0c00cd4d0 0x7ff0c00b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0c5ccc700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c015ac30 con 0x7ff0c0150940 2023-10-06T10:44:13.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0c5ccc700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c00cb6a0 con 0x7ff0c00c5250 2023-10-06T10:44:13.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0c5ccc700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c00cb090 con 0x7ff0c00cd4d0 2023-10-06T10:44:13.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0bf7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff0c0150940 0x7ff0c0159c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44562/0 (socket says 172.21.15.92:44562) 2023-10-06T10:44:13.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0bf7fe700 1 -- 172.21.15.92:0/1719211639 learned_addr learned my addr 172.21.15.92:0/1719211639 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:13.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0bffff700 1 --2- 172.21.15.92:0/1719211639 >> v2:172.21.15.124:3300/0 conn(0x7ff0c00cd4d0 0x7ff0c00b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0be7fc700 1 -- 172.21.15.92:0/1719211639 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153819494 0 0) 0x7ff0b8001610 con 0x7ff0c0150940 2023-10-06T10:44:13.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0be7fc700 1 -- 172.21.15.92:0/1719211639 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0ac003570 con 0x7ff0c0150940 2023-10-06T10:44:13.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0be7fc700 1 -- 172.21.15.92:0/1719211639 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2808076584 0 0) 0x7ff0ac003570 con 0x7ff0c0150940 2023-10-06T10:44:13.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0be7fc700 1 -- 172.21.15.92:0/1719211639 >> v2:172.21.15.124:3300/0 conn(0x7ff0c00cd4d0 msgr2=0x7ff0c00b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0be7fc700 1 --2- 172.21.15.92:0/1719211639 >> v2:172.21.15.124:3300/0 conn(0x7ff0c00cd4d0 0x7ff0c00b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0be7fc700 1 -- 172.21.15.92:0/1719211639 >> v1:172.21.15.124:6789/0 conn(0x7ff0c00c5250 legacy=0x7ff0c00ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0be7fc700 1 -- 172.21.15.92:0/1719211639 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c015a670 con 0x7ff0c0150940 2023-10-06T10:44:13.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0be7fc700 1 -- 172.21.15.92:0/1719211639 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7ff0b8001b00 con 0x7ff0c0150940 2023-10-06T10:44:13.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0be7fc700 1 -- 172.21.15.92:0/1719211639 >> v1:172.21.15.92:6789/0 conn(0x7ff0c0150940 legacy=0x7ff0c0159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0be7fc700 1 --2- 172.21.15.92:0/1719211639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0ac003a60 0x7ff0ac003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0be7fc700 1 --2- 172.21.15.92:0/1719211639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0ac004810 0x7ff0ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0be7fc700 1 --2- 172.21.15.92:0/1719211639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0ac0051c0 0x7ff0ac00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0be7fc700 1 -- 172.21.15.92:0/1719211639 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c00cb090 con 0x7ff0ac0051c0 2023-10-06T10:44:13.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0be7fc700 1 -- 172.21.15.92:0/1719211639 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac00c0d0 con 0x7ff0ac003a60 2023-10-06T10:44:13.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0be7fc700 1 -- 172.21.15.92:0/1719211639 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac00c290 con 0x7ff0ac004810 2023-10-06T10:44:13.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.601+0000 7ff0bffff700 1 --2- 172.21.15.92:0/1719211639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0ac0051c0 0x7ff0ac00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.602+0000 7ff0bf7fe700 1 --2- 172.21.15.92:0/1719211639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0ac003a60 0x7ff0ac003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.602+0000 7ff0beffd700 1 --2- 172.21.15.92:0/1719211639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0ac004810 0x7ff0ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.602+0000 7ff0bffff700 1 -- 172.21.15.92:0/1719211639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0ac004810 msgr2=0x7ff0ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.602+0000 7ff0bffff700 1 --2- 172.21.15.92:0/1719211639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0ac004810 0x7ff0ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.602+0000 7ff0bffff700 1 -- 172.21.15.92:0/1719211639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0ac003a60 msgr2=0x7ff0ac003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.602+0000 7ff0bffff700 1 --2- 172.21.15.92:0/1719211639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0ac003a60 0x7ff0ac003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.602+0000 7ff0bffff700 1 -- 172.21.15.92:0/1719211639 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ac003570 con 0x7ff0ac0051c0 2023-10-06T10:44:13.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.602+0000 7ff0bffff700 1 -- 172.21.15.92:0/1719211639 reap_dead start 2023-10-06T10:44:13.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.602+0000 7ff0bffff700 1 --2- 172.21.15.92:0/1719211639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0ac0051c0 0x7ff0ac00ba60 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009c00 tx=0x7ff0b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1b91b55b8bc34a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:13.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.602+0000 7ff0be7fc700 1 -- 172.21.15.92:0/1719211639 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b000e710 con 0x7ff0ac0051c0 2023-10-06T10:44:13.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.602+0000 7ff0be7fc700 1 -- 172.21.15.92:0/1719211639 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b000e890 con 0x7ff0ac0051c0 2023-10-06T10:44:13.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.602+0000 7ff0be7fc700 1 -- 172.21.15.92:0/1719211639 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b000ec50 con 0x7ff0ac0051c0 2023-10-06T10:44:13.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.602+0000 7ff0c5ccc700 1 -- 172.21.15.92:0/1719211639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0ac0051c0 msgr2=0x7ff0ac00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.602+0000 7ff0c5ccc700 1 --2- 172.21.15.92:0/1719211639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0ac0051c0 0x7ff0ac00ba60 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009c00 tx=0x7ff0b00091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.603+0000 7ff0c5ccc700 1 -- 172.21.15.92:0/1719211639 shutdown_connections 2023-10-06T10:44:13.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.603+0000 7ff0c5ccc700 1 --2- 172.21.15.92:0/1719211639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0ac0051c0 0x7ff0ac00ba60 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.603+0000 7ff0c5ccc700 1 -- 172.21.15.92:0/1719211639 >> 172.21.15.92:0/1719211639 conn(0x7ff0c0099700 msgr2=0x7ff0c006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:13.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.603+0000 7ff0c5ccc700 1 -- 172.21.15.92:0/1719211639 shutdown_connections 2023-10-06T10:44:13.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.603+0000 7ff0c5ccc700 1 -- 172.21.15.92:0/1719211639 wait complete. 2023-10-06T10:44:13.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.604+0000 7ff0c5ccc700 1 Processor -- start 2023-10-06T10:44:13.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.604+0000 7ff0c5ccc700 1 -- start start 2023-10-06T10:44:13.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.605+0000 7ff0c5ccc700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0ac0051c0 0x7ff0c009ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.605+0000 7ff0c5ccc700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0c009d180 0x7ff0c009c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.605+0000 7ff0c5ccc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0c00ce830 0x7ff0c00cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.605+0000 7ff0c5ccc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c009c8f0 con 0x7ff0c00ce830 2023-10-06T10:44:13.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.605+0000 7ff0c5ccc700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c00cf210 con 0x7ff0c009d180 2023-10-06T10:44:13.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.605+0000 7ff0c5ccc700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c009d5f0 con 0x7ff0ac0051c0 2023-10-06T10:44:13.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.605+0000 7ff0bffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0c00ce830 0x7ff0c00cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.605+0000 7ff0bffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0c00ce830 0x7ff0c00cecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48122/0 (socket says 172.21.15.92:48122) 2023-10-06T10:44:13.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.605+0000 7ff0bffff700 1 -- 172.21.15.92:0/1877790247 learned_addr learned my addr 172.21.15.92:0/1877790247 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:13.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.605+0000 7ff0bf7fe700 1 --2- 172.21.15.92:0/1877790247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0ac0051c0 0x7ff0c009ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.605+0000 7ff0beffd700 1 --2- 172.21.15.92:0/1877790247 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0c009d180 0x7ff0c009c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.605+0000 7ff0bffff700 1 -- 172.21.15.92:0/1877790247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0ac0051c0 msgr2=0x7ff0c009ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.605+0000 7ff0bffff700 1 --2- 172.21.15.92:0/1877790247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0ac0051c0 0x7ff0c009ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.605+0000 7ff0bffff700 1 -- 172.21.15.92:0/1877790247 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0c009d180 msgr2=0x7ff0c009c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.605+0000 7ff0bffff700 1 --2- 172.21.15.92:0/1877790247 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0c009d180 0x7ff0c009c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.605+0000 7ff0bffff700 1 -- 172.21.15.92:0/1877790247 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c009b310 con 0x7ff0c00ce830 2023-10-06T10:44:13.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.605+0000 7ff0bffff700 1 --2- 172.21.15.92:0/1877790247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0c00ce830 0x7ff0c00cecd0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0071190 tx=0x7ff0b0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:13.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.605+0000 7ff0b77fe700 1 -- 172.21.15.92:0/1877790247 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b000ec10 con 0x7ff0c00ce830 2023-10-06T10:44:13.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.605+0000 7ff0b77fe700 1 -- 172.21.15.92:0/1877790247 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b001c450 con 0x7ff0c00ce830 2023-10-06T10:44:13.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.605+0000 7ff0c5ccc700 1 -- 172.21.15.92:0/1877790247 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c009b5f0 con 0x7ff0c00ce830 2023-10-06T10:44:13.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.605+0000 7ff0b77fe700 1 -- 172.21.15.92:0/1877790247 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b001c7f0 con 0x7ff0c00ce830 2023-10-06T10:44:13.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.605+0000 7ff0c5ccc700 1 -- 172.21.15.92:0/1877790247 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c009bbb0 con 0x7ff0c00ce830 2023-10-06T10:44:13.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.607+0000 7ff0b77fe700 1 -- 172.21.15.92:0/1877790247 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff0b001cc20 con 0x7ff0c00ce830 2023-10-06T10:44:13.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.607+0000 7ff0b77fe700 1 --2- 172.21.15.92:0/1877790247 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff0a0066f30 0x7ff0a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.607+0000 7ff0b77fe700 1 -- 172.21.15.92:0/1877790247 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff0b0086a60 con 0x7ff0c00ce830 2023-10-06T10:44:13.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.607+0000 7ff0bf7fe700 1 --2- 172.21.15.92:0/1877790247 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff0a0066f30 0x7ff0a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.607+0000 7ff0bf7fe700 1 --2- 172.21.15.92:0/1877790247 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff0a0066f30 0x7ff0a00693e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8007f20 tx=0x7ff0b8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:13.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.609+0000 7ff0cefc4b80 1 Processor -- start 2023-10-06T10:44:13.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.609+0000 7ff0cefc4b80 1 -- start start 2023-10-06T10:44:13.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.610+0000 7ff0cefc4b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561f13a3dae0 0x561f13a3deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.610+0000 7ff0cefc4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f139f99f0 0x561f139f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.610+0000 7ff0cefc4b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561f139f7c20 0x561f139f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.610+0000 7ff0cefc4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f13976cd0 con 0x561f139f99f0 2023-10-06T10:44:13.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.610+0000 7ff0cefc4b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1394dac0 con 0x561f139f7c20 2023-10-06T10:44:13.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.610+0000 7ff0cefc4b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561f1394b2c0 con 0x561f13a3dae0 2023-10-06T10:44:13.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.610+0000 7ff0beffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f139f99f0 0x561f139f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.610+0000 7ff0beffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f139f99f0 0x561f139f65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48136/0 (socket says 172.21.15.92:48136) 2023-10-06T10:44:13.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.610+0000 7ff0beffd700 1 -- 172.21.15.92:0/2857811344 learned_addr learned my addr 172.21.15.92:0/2857811344 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:13.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.610+0000 7ff0bf7fe700 1 --2- 172.21.15.92:0/2857811344 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561f13a3dae0 0x561f13a3deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.610+0000 7ff0bffff700 1 --2- 172.21.15.92:0/2857811344 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561f139f7c20 0x561f139f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.610+0000 7ff0beffd700 1 -- 172.21.15.92:0/2857811344 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561f13a3dae0 msgr2=0x561f13a3deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.610+0000 7ff0beffd700 1 --2- 172.21.15.92:0/2857811344 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561f13a3dae0 0x561f13a3deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.610+0000 7ff0beffd700 1 -- 172.21.15.92:0/2857811344 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561f139f7c20 msgr2=0x561f139f6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.610+0000 7ff0beffd700 1 --2- 172.21.15.92:0/2857811344 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561f139f7c20 0x561f139f6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.610+0000 7ff0beffd700 1 -- 172.21.15.92:0/2857811344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f137e12c0 con 0x561f139f99f0 2023-10-06T10:44:13.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.610+0000 7ff0beffd700 1 --2- 172.21.15.92:0/2857811344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f139f99f0 0x561f139f65d0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8003690 tx=0x7ff0a8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ac701401360d72 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:13.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.611+0000 7ff0b4ff9700 1 -- 172.21.15.92:0/2857811344 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8006820 con 0x561f139f99f0 2023-10-06T10:44:13.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.611+0000 7ff0b4ff9700 1 -- 172.21.15.92:0/2857811344 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a8010050 con 0x561f139f99f0 2023-10-06T10:44:13.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.611+0000 7ff0b4ff9700 1 -- 172.21.15.92:0/2857811344 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8006bf0 con 0x561f139f99f0 2023-10-06T10:44:13.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.611+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/2857811344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f139f99f0 msgr2=0x561f139f65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.611+0000 7ff0cefc4b80 1 --2- 172.21.15.92:0/2857811344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f139f99f0 0x561f139f65d0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8003690 tx=0x7ff0a8009040 comp rx=0 tx=0).stop 2023-10-06T10:44:13.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.611+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/2857811344 shutdown_connections 2023-10-06T10:44:13.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.611+0000 7ff0cefc4b80 1 --2- 172.21.15.92:0/2857811344 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561f139f7c20 0x561f139f6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.611+0000 7ff0cefc4b80 1 --2- 172.21.15.92:0/2857811344 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561f13a3dae0 0x561f13a3deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.611+0000 7ff0cefc4b80 1 --2- 172.21.15.92:0/2857811344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f139f99f0 0x561f139f65d0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.611+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/2857811344 >> 172.21.15.92:0/2857811344 conn(0x561f139f80c0 msgr2=0x561f139f8490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:13.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.611+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/2857811344 shutdown_connections 2023-10-06T10:44:13.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.611+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/2857811344 wait complete. 2023-10-06T10:44:13.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.612+0000 7ff0cefc4b80 1 -- start start 2023-10-06T10:44:13.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.612+0000 7ff0cefc4b80 1 Processor -- start 2023-10-06T10:44:13.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff0cefc4b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561f139f7c20 0x561f13a3a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff0cefc4b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561f139f99f0 0x561f13a1ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff0cefc4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f13a3dae0 0x561f13a1f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff0cefc4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f139470b0 con 0x561f13a3dae0 2023-10-06T10:44:13.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff0cefc4b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f138f28e0 con 0x561f139f7c20 2023-10-06T10:44:13.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff0cefc4b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561f1390def0 con 0x561f139f99f0 2023-10-06T10:44:13.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff0cefc4b80 10 client.0 _ll_register_callbacks cb 0x561f137e3f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:13.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff0bffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f13a3dae0 0x561f13a1f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff0bffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f13a3dae0 0x561f13a1f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48144/0 (socket says 172.21.15.92:48144) 2023-10-06T10:44:13.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff0bffff700 1 -- 172.21.15.92:0/3417580600 learned_addr learned my addr 172.21.15.92:0/3417580600 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:13.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff0bffff700 1 -- 172.21.15.92:0/3417580600 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561f139f99f0 msgr2=0x561f13a1ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff0bf7fe700 1 --2- 172.21.15.92:0/3417580600 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561f139f7c20 0x561f13a3a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff0beffd700 1 --2- 172.21.15.92:0/3417580600 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561f139f99f0 0x561f13a1ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff0bffff700 1 --2- 172.21.15.92:0/3417580600 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561f139f99f0 0x561f13a1ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff0bffff700 1 -- 172.21.15.92:0/3417580600 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561f139f7c20 msgr2=0x561f13a3a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff0bffff700 1 --2- 172.21.15.92:0/3417580600 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561f139f7c20 0x561f13a3a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff0bffff700 1 -- 172.21.15.92:0/3417580600 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c00cb6a0 con 0x561f13a3dae0 2023-10-06T10:44:13.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff0beffd700 1 --2- 172.21.15.92:0/3417580600 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561f139f99f0 0x561f13a1ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:13.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff0bf7fe700 1 --2- 172.21.15.92:0/3417580600 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561f139f7c20 0x561f13a3a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:13.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff0bffff700 1 --2- 172.21.15.92:0/3417580600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f13a3dae0 0x561f13a1f1d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0084d70 tx=0x7ff0b0084c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21dc925f28a9927e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:13.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff09dffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:13.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff09dffb700 1 -- 172.21.15.92:0/3417580600 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b00576f0 con 0x561f13a3dae0 2023-10-06T10:44:13.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff09dffb700 1 -- 172.21.15.92:0/3417580600 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff0b005a940 con 0x561f13a3dae0 2023-10-06T10:44:13.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.613+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/3417580600 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f139eb6e0 con 0x561f13a3dae0 2023-10-06T10:44:13.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.614+0000 7ff09dffb700 1 -- 172.21.15.92:0/3417580600 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0b005e690 con 0x561f13a3dae0 2023-10-06T10:44:13.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.614+0000 7ff0cefc4b80 10 client.7965 fetch_fsmap learned FSMap version 90 2023-10-06T10:44:13.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.614+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/3417580600 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x561f13784030 con 0x561f13a3dae0 2023-10-06T10:44:13.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.614+0000 7ff09dffb700 1 -- 172.21.15.92:0/3417580600 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561f139eb6e0 con 0x561f13a3dae0 2023-10-06T10:44:13.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.614+0000 7ff09dffb700 1 -- 172.21.15.92:0/3417580600 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 90) v1 ==== 3136+0+0 (secure 0 0 0) 0x7ff0b005e200 con 0x561f13a3dae0 2023-10-06T10:44:13.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.614+0000 7ff0cefc4b80 10 client.7965 fetch_fsmap finished waiting for FSMap version 90 2023-10-06T10:44:13.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.614+0000 7ff0cefc4b80 10 client.7965 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2023-10-06T10:44:13.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.614+0000 7ff0cefc4b80 20 client.7965 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:13.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.614+0000 7ff0cefc4b80 1 --2- 172.21.15.92:0/3417580600 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x561f13a510e0 0x561f13a4ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.614+0000 7ff0cefc4b80 4 client.7965 mds_command: new command op to 7701 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:44:13.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.614+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/3417580600 --> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f139c8610 con 0x561f13a510e0 2023-10-06T10:44:13.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.614+0000 7ff0bf7fe700 1 --2- 172.21.15.92:0/3417580600 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x561f13a510e0 0x561f13a4ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.614+0000 7ff09dffb700 10 client.7965 ms_handle_connect on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:13.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.614+0000 7ff0bf7fe700 1 --2- 172.21.15.92:0/3417580600 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x561f13a510e0 0x561f13a4ce40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a275260d68585a96 server_cookie=7f0bc2973dfb90db in_seq=0 out_seq=0 2023-10-06T10:44:13.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.615+0000 7ff09dffb700 1 -- 172.21.15.92:0/3417580600 <== mds.0 v2:172.21.15.124:6832/4203118992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x561f139c8610 con 0x561f13a510e0 2023-10-06T10:44:13.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.615+0000 7ff09dffb700 10 client.7965 handle_command_reply: tid=0 2023-10-06T10:44:13.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.615+0000 7ff0cefc4b80 1 client.7965 shutdown 2023-10-06T10:44:13.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.615+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/3417580600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f13a3dae0 msgr2=0x561f13a1f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.615+0000 7ff0cefc4b80 1 --2- 172.21.15.92:0/3417580600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f13a3dae0 0x561f13a1f1d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0084d70 tx=0x7ff0b0084c40 comp rx=0 tx=0).stop 2023-10-06T10:44:13.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.615+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/3417580600 shutdown_connections 2023-10-06T10:44:13.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.615+0000 7ff0cefc4b80 1 --2- 172.21.15.92:0/3417580600 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x561f13a510e0 0x561f13a4ce40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.615+0000 7ff0cefc4b80 1 --2- 172.21.15.92:0/3417580600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f13a3dae0 0x561f13a1f1d0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.615+0000 7ff0cefc4b80 1 --2- 172.21.15.92:0/3417580600 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561f139f7c20 0x561f13a3a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.615+0000 7ff09dffb700 0 client.7965 ms_handle_reset on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:13.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.615+0000 7ff0cefc4b80 1 --2- 172.21.15.92:0/3417580600 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561f139f99f0 0x561f13a1ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.615+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/3417580600 >> 172.21.15.92:0/3417580600 conn(0x561f139f80c0 msgr2=0x561f13a3b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:13.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.615+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/3417580600 shutdown_connections 2023-10-06T10:44:13.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.615+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/3417580600 wait complete. 2023-10-06T10:44:13.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.615+0000 7ff0cefc4b80 20 client.7965 trim_cache size 0 max 16384 2023-10-06T10:44:13.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.629+0000 7ff0cefc4b80 1 Processor -- start 2023-10-06T10:44:13.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.629+0000 7ff0cefc4b80 1 -- start start 2023-10-06T10:44:13.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.630+0000 7ff0cefc4b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561f139f7c20 0x561f13a4f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.630+0000 7ff0cefc4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f139f99f0 0x561f13a3c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.630+0000 7ff0cefc4b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561f13a3dae0 0x561f13a3c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.630+0000 7ff0cefc4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f137c9e80 con 0x561f139f99f0 2023-10-06T10:44:13.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.630+0000 7ff0cefc4b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f13a507a0 con 0x561f139f7c20 2023-10-06T10:44:13.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.630+0000 7ff0cefc4b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561f13a3e760 con 0x561f13a3dae0 2023-10-06T10:44:13.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.630+0000 7ff0beffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f139f99f0 0x561f13a3c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.630+0000 7ff0beffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f139f99f0 0x561f13a3c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48150/0 (socket says 172.21.15.92:48150) 2023-10-06T10:44:13.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.630+0000 7ff0beffd700 1 -- 172.21.15.92:0/3939433675 learned_addr learned my addr 172.21.15.92:0/3939433675 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:13.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.630+0000 7ff0bf7fe700 1 --2- 172.21.15.92:0/3939433675 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561f139f7c20 0x561f13a4f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.630+0000 7ff0bffff700 1 --2- 172.21.15.92:0/3939433675 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561f13a3dae0 0x561f13a3c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.630+0000 7ff0beffd700 1 -- 172.21.15.92:0/3939433675 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561f13a3dae0 msgr2=0x561f13a3c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.630+0000 7ff0beffd700 1 --2- 172.21.15.92:0/3939433675 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561f13a3dae0 0x561f13a3c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.630+0000 7ff0beffd700 1 -- 172.21.15.92:0/3939433675 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561f139f7c20 msgr2=0x561f13a4f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.630+0000 7ff0beffd700 1 --2- 172.21.15.92:0/3939433675 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561f139f7c20 0x561f13a4f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.630+0000 7ff0beffd700 1 -- 172.21.15.92:0/3939433675 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f139157d0 con 0x561f139f99f0 2023-10-06T10:44:13.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.630+0000 7ff0beffd700 1 --2- 172.21.15.92:0/3939433675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f139f99f0 0x561f13a3c2d0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8003690 tx=0x7ff0a8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518c0757ba3d81af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:13.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.630+0000 7ff0b57fa700 1 -- 172.21.15.92:0/3939433675 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8007cc0 con 0x561f139f99f0 2023-10-06T10:44:13.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.631+0000 7ff0b57fa700 1 -- 172.21.15.92:0/3939433675 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a8000c00 con 0x561f139f99f0 2023-10-06T10:44:13.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.631+0000 7ff0b57fa700 1 -- 172.21.15.92:0/3939433675 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8006a70 con 0x561f139f99f0 2023-10-06T10:44:13.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.631+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/3939433675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f139f99f0 msgr2=0x561f13a3c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.631+0000 7ff0cefc4b80 1 --2- 172.21.15.92:0/3939433675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f139f99f0 0x561f13a3c2d0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8003690 tx=0x7ff0a8019000 comp rx=0 tx=0).stop 2023-10-06T10:44:13.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.631+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/3939433675 shutdown_connections 2023-10-06T10:44:13.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.631+0000 7ff0cefc4b80 1 --2- 172.21.15.92:0/3939433675 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561f13a3dae0 0x561f13a3c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.631+0000 7ff0cefc4b80 1 --2- 172.21.15.92:0/3939433675 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561f139f7c20 0x561f13a4f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.631+0000 7ff0cefc4b80 1 --2- 172.21.15.92:0/3939433675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f139f99f0 0x561f13a3c2d0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.631+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/3939433675 >> 172.21.15.92:0/3939433675 conn(0x561f139f80c0 msgr2=0x561f13a1f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:13.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.631+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/3939433675 shutdown_connections 2023-10-06T10:44:13.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.631+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/3939433675 wait complete. 2023-10-06T10:44:13.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.632+0000 7ff0cefc4b80 1 -- start start 2023-10-06T10:44:13.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.632+0000 7ff0cefc4b80 1 Processor -- start 2023-10-06T10:44:13.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.632+0000 7ff0cefc4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f139f7c20 0x561f139f2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.632+0000 7ff0cefc4b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561f139f99f0 0x561f139f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.632+0000 7ff0cefc4b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561f13a3dae0 0x561f139f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.632+0000 7ff0cefc4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f13a4fb50 con 0x561f139f7c20 2023-10-06T10:44:13.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.632+0000 7ff0cefc4b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f13a34560 con 0x561f139f99f0 2023-10-06T10:44:13.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.632+0000 7ff0cefc4b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561f13a33c80 con 0x561f13a3dae0 2023-10-06T10:44:13.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.632+0000 7ff0cefc4b80 10 client.0 _ll_register_callbacks cb 0x561f137e3f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:13.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.632+0000 7ff0bf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f139f7c20 0x561f139f2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.632+0000 7ff0bf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f139f7c20 0x561f139f2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48166/0 (socket says 172.21.15.92:48166) 2023-10-06T10:44:13.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.632+0000 7ff0bf7fe700 1 -- 172.21.15.92:0/2570698888 learned_addr learned my addr 172.21.15.92:0/2570698888 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:13.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff0bffff700 1 --2- 172.21.15.92:0/2570698888 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561f13a3dae0 0x561f139f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff0beffd700 1 --2- 172.21.15.92:0/2570698888 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561f139f99f0 0x561f139f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff0bf7fe700 1 -- 172.21.15.92:0/2570698888 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561f13a3dae0 msgr2=0x561f139f65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff0bf7fe700 1 --2- 172.21.15.92:0/2570698888 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561f13a3dae0 0x561f139f65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff0bf7fe700 1 -- 172.21.15.92:0/2570698888 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561f139f99f0 msgr2=0x561f139f2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff0bf7fe700 1 --2- 172.21.15.92:0/2570698888 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561f139f99f0 0x561f139f2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff0bf7fe700 1 -- 172.21.15.92:0/2570698888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f137e12c0 con 0x561f139f7c20 2023-10-06T10:44:13.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff0bffff700 1 --2- 172.21.15.92:0/2570698888 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561f13a3dae0 0x561f139f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:13.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff0beffd700 1 --2- 172.21.15.92:0/2570698888 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561f139f99f0 0x561f139f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:13.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff0bf7fe700 1 --2- 172.21.15.92:0/2570698888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f139f7c20 0x561f139f2970 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8001c30 tx=0x7ff0b8025040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e88ac2ac3c04ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:13.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff09dffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:13.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff09dffb700 1 -- 172.21.15.92:0/2570698888 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8028040 con 0x561f139f7c20 2023-10-06T10:44:13.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff09dffb700 1 -- 172.21.15.92:0/2570698888 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff0b801b040 con 0x561f139f7c20 2023-10-06T10:44:13.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/2570698888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0a8010050 con 0x561f139f7c20 2023-10-06T10:44:13.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff09dffb700 1 -- 172.21.15.92:0/2570698888 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0b802b680 con 0x561f139f7c20 2023-10-06T10:44:13.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff0cefc4b80 10 client.7971 fetch_fsmap learned FSMap version 90 2023-10-06T10:44:13.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/2570698888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x561f138fc4c0 con 0x561f139f7c20 2023-10-06T10:44:13.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff09dffb700 1 -- 172.21.15.92:0/2570698888 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a8010050 con 0x561f139f7c20 2023-10-06T10:44:13.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff09dffb700 1 -- 172.21.15.92:0/2570698888 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 90) v1 ==== 3136+0+0 (secure 0 0 0) 0x7ff0b802b7e0 con 0x561f139f7c20 2023-10-06T10:44:13.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff0cefc4b80 10 client.7971 fetch_fsmap finished waiting for FSMap version 90 2023-10-06T10:44:13.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff0cefc4b80 10 client.7971 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2023-10-06T10:44:13.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff0cefc4b80 20 client.7971 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:13.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff0cefc4b80 1 --2- 172.21.15.92:0/2570698888 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x561f13a40980 0x561f139f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff0cefc4b80 4 client.7971 mds_command: new command op to 7701 tid=0[{"prefix": "flush journal"}] 2023-10-06T10:44:13.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.633+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/2570698888 --> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561f138e9e50 con 0x561f13a40980 2023-10-06T10:44:13.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.634+0000 7ff0beffd700 1 --2- 172.21.15.92:0/2570698888 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x561f13a40980 0x561f139f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.634+0000 7ff09dffb700 10 client.7971 ms_handle_connect on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:13.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.634+0000 7ff0beffd700 1 --2- 172.21.15.92:0/2570698888 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x561f13a40980 0x561f139f6b10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7396bd1b8c597b42 server_cookie=93cc8cd8e53624e0 in_seq=0 out_seq=0 2023-10-06T10:44:13.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.651+0000 7ff09dffb700 1 -- 172.21.15.92:0/2570698888 <== mds.0 v2:172.21.15.124:6832/4203118992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561f138e9e50 con 0x561f13a40980 2023-10-06T10:44:13.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.651+0000 7ff09dffb700 10 client.7971 handle_command_reply: tid=0 2023-10-06T10:44:13.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.651+0000 7ff0cefc4b80 1 client.7971 shutdown 2023-10-06T10:44:13.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.651+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/2570698888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f139f7c20 msgr2=0x561f139f2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.651+0000 7ff0cefc4b80 1 --2- 172.21.15.92:0/2570698888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f139f7c20 0x561f139f2970 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8001c30 tx=0x7ff0b8025040 comp rx=0 tx=0).stop 2023-10-06T10:44:13.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.651+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/2570698888 shutdown_connections 2023-10-06T10:44:13.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.651+0000 7ff0cefc4b80 1 --2- 172.21.15.92:0/2570698888 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x561f13a40980 0x561f139f6b10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.651+0000 7ff0cefc4b80 1 --2- 172.21.15.92:0/2570698888 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561f13a3dae0 0x561f139f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.651+0000 7ff09dffb700 0 client.7971 ms_handle_reset on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:13.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.651+0000 7ff0cefc4b80 1 --2- 172.21.15.92:0/2570698888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f139f7c20 0x561f139f2970 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.651+0000 7ff0cefc4b80 1 --2- 172.21.15.92:0/2570698888 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561f139f99f0 0x561f139f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.651+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/2570698888 >> 172.21.15.92:0/2570698888 conn(0x561f139f80c0 msgr2=0x561f13a36a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:13.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.651+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/2570698888 shutdown_connections 2023-10-06T10:44:13.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.651+0000 7ff0cefc4b80 1 -- 172.21.15.92:0/2570698888 wait complete. 2023-10-06T10:44:13.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.652+0000 7ff0cefc4b80 20 client.7971 trim_cache size 0 max 16384 2023-10-06T10:44:13.656 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:44:13.656 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2023-10-06T10:44:13.657 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:44:13.657 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:44:13.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.652+0000 7ff0c5ccc700 1 -- 172.21.15.92:0/1877790247 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff0a0066f30 msgr2=0x7ff0a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.652+0000 7ff0c5ccc700 1 --2- 172.21.15.92:0/1877790247 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff0a0066f30 0x7ff0a00693e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8007f20 tx=0x7ff0b8009040 comp rx=0 tx=0).stop 2023-10-06T10:44:13.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.652+0000 7ff0c5ccc700 1 -- 172.21.15.92:0/1877790247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0c00ce830 msgr2=0x7ff0c00cecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.652+0000 7ff0c5ccc700 1 --2- 172.21.15.92:0/1877790247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0c00ce830 0x7ff0c00cecd0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0071190 tx=0x7ff0b0019000 comp rx=0 tx=0).stop 2023-10-06T10:44:13.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.652+0000 7ff0c5ccc700 1 -- 172.21.15.92:0/1877790247 shutdown_connections 2023-10-06T10:44:13.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.652+0000 7ff0c5ccc700 1 --2- 172.21.15.92:0/1877790247 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff0a0066f30 0x7ff0a00693e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.652+0000 7ff0c5ccc700 1 --2- 172.21.15.92:0/1877790247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0c00ce830 0x7ff0c00cecd0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.652+0000 7ff0c5ccc700 1 --2- 172.21.15.92:0/1877790247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0ac0051c0 0x7ff0c009ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.652+0000 7ff0c5ccc700 1 --2- 172.21.15.92:0/1877790247 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0c009d180 0x7ff0c009c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.652+0000 7ff0c5ccc700 1 -- 172.21.15.92:0/1877790247 >> 172.21.15.92:0/1877790247 conn(0x7ff0c0099700 msgr2=0x7ff0c00b58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:13.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.652+0000 7ff0c5ccc700 1 -- 172.21.15.92:0/1877790247 shutdown_connections 2023-10-06T10:44:13.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.652+0000 7ff0c5ccc700 1 -- 172.21.15.92:0/1877790247 wait complete. 2023-10-06T10:44:13.666 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:1 flush journal 2023-10-06T10:44:13.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.779+0000 7ff7dd754700 1 Processor -- start 2023-10-06T10:44:13.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.779+0000 7ff7dd754700 1 -- start start 2023-10-06T10:44:13.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.779+0000 7ff7dd754700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff7d8150940 0x7ff7d8159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.779+0000 7ff7dd754700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff7d80cd4d0 0x7ff7d80b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.779+0000 7ff7dd754700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d815ac30 con 0x7ff7d815a160 2023-10-06T10:44:13.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.779+0000 7ff7dd754700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d80cb6a0 con 0x7ff7d80cd4d0 2023-10-06T10:44:13.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.779+0000 7ff7dd754700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d80cb090 con 0x7ff7d8150940 2023-10-06T10:44:13.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.780+0000 7ff7d77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff7d80cd4d0 0x7ff7d80b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.780+0000 7ff7d77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff7d80cd4d0 0x7ff7d80b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48178/0 (socket says 172.21.15.92:48178) 2023-10-06T10:44:13.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.780+0000 7ff7d77fe700 1 -- 172.21.15.92:0/1243338470 learned_addr learned my addr 172.21.15.92:0/1243338470 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:13.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.780+0000 7ff7d6ffd700 1 --2- 172.21.15.92:0/1243338470 >> v2:172.21.15.124:3300/0 conn(0x7ff7d8150940 0x7ff7d8159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.780+0000 7ff7d77fe700 1 -- 172.21.15.92:0/1243338470 >> v2:172.21.15.124:3300/0 conn(0x7ff7d8150940 msgr2=0x7ff7d8159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.780+0000 7ff7d77fe700 1 --2- 172.21.15.92:0/1243338470 >> v2:172.21.15.124:3300/0 conn(0x7ff7d8150940 0x7ff7d8159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.780+0000 7ff7d77fe700 1 -- 172.21.15.92:0/1243338470 >> v1:172.21.15.124:6789/0 conn(0x7ff7d815a160 legacy=0x7ff7d80ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.780+0000 7ff7d77fe700 1 -- 172.21.15.92:0/1243338470 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d815a670 con 0x7ff7d80cd4d0 2023-10-06T10:44:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.780+0000 7ff7d77fe700 1 --2- 172.21.15.92:0/1243338470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7d80cd4d0 0x7ff7d80b2050 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7ff7d808ab60 tx=0x7ff7c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369c20bc2de2a74e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.780+0000 7ff7d5ffb700 1 -- 172.21.15.92:0/1243338470 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c800e070 con 0x7ff7d80cd4d0 2023-10-06T10:44:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.780+0000 7ff7d5ffb700 1 -- 172.21.15.92:0/1243338470 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c800ba20 con 0x7ff7d80cd4d0 2023-10-06T10:44:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.780+0000 7ff7d5ffb700 1 -- 172.21.15.92:0/1243338470 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c80092e0 con 0x7ff7d80cd4d0 2023-10-06T10:44:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.781+0000 7ff7dd754700 1 -- 172.21.15.92:0/1243338470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7d80cd4d0 msgr2=0x7ff7d80b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.781+0000 7ff7dd754700 1 --2- 172.21.15.92:0/1243338470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7d80cd4d0 0x7ff7d80b2050 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7ff7d808ab60 tx=0x7ff7c8009840 comp rx=0 tx=0).stop 2023-10-06T10:44:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.781+0000 7ff7dd754700 1 -- 172.21.15.92:0/1243338470 shutdown_connections 2023-10-06T10:44:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.781+0000 7ff7dd754700 1 --2- 172.21.15.92:0/1243338470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7d80cd4d0 0x7ff7d80b2050 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.781+0000 7ff7dd754700 1 --2- 172.21.15.92:0/1243338470 >> v2:172.21.15.124:3300/0 conn(0x7ff7d8150940 0x7ff7d8159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.781+0000 7ff7dd754700 1 -- 172.21.15.92:0/1243338470 >> 172.21.15.92:0/1243338470 conn(0x7ff7d8099700 msgr2=0x7ff7d806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.781+0000 7ff7dd754700 1 -- 172.21.15.92:0/1243338470 shutdown_connections 2023-10-06T10:44:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.781+0000 7ff7dd754700 1 -- 172.21.15.92:0/1243338470 wait complete. 2023-10-06T10:44:13.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.782+0000 7ff7dd754700 1 Processor -- start 2023-10-06T10:44:13.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.782+0000 7ff7dd754700 1 -- start start 2023-10-06T10:44:13.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.783+0000 7ff7dd754700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7d80cd4d0 0x7ff7d80c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.783+0000 7ff7dd754700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7d815a160 0x7ff7d814eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.783+0000 7ff7dd754700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7d80b43d0 0x7ff7d80b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.783+0000 7ff7dd754700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d80c8600 con 0x7ff7d80cd4d0 2023-10-06T10:44:13.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.783+0000 7ff7dd754700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d814f620 con 0x7ff7d80b43d0 2023-10-06T10:44:13.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.783+0000 7ff7dd754700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d814f090 con 0x7ff7d815a160 2023-10-06T10:44:13.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.783+0000 7ff7d6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7d80cd4d0 0x7ff7d80c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.783+0000 7ff7d6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7d80cd4d0 0x7ff7d80c7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48184/0 (socket says 172.21.15.92:48184) 2023-10-06T10:44:13.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.783+0000 7ff7d6ffd700 1 -- 172.21.15.92:0/3577227658 learned_addr learned my addr 172.21.15.92:0/3577227658 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.783+0000 7ff7d77fe700 1 --2- 172.21.15.92:0/3577227658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7d80b43d0 0x7ff7d80b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.783+0000 7ff7d67fc700 1 --2- 172.21.15.92:0/3577227658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7d815a160 0x7ff7d814eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.783+0000 7ff7d6ffd700 1 -- 172.21.15.92:0/3577227658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7d815a160 msgr2=0x7ff7d814eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.783+0000 7ff7d6ffd700 1 --2- 172.21.15.92:0/3577227658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7d815a160 0x7ff7d814eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.783+0000 7ff7d6ffd700 1 -- 172.21.15.92:0/3577227658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7d80b43d0 msgr2=0x7ff7d80b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.783+0000 7ff7d6ffd700 1 --2- 172.21.15.92:0/3577227658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7d80b43d0 0x7ff7d80b4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.783+0000 7ff7d6ffd700 1 -- 172.21.15.92:0/3577227658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c8009ae0 con 0x7ff7d80cd4d0 2023-10-06T10:44:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.783+0000 7ff7d6ffd700 1 --2- 172.21.15.92:0/3577227658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7d80cd4d0 0x7ff7d80c7f70 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000ec80 tx=0x7ff7d0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.783+0000 7ff7beffd700 1 -- 172.21.15.92:0/3577227658 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d000c9d0 con 0x7ff7d80cd4d0 2023-10-06T10:44:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.784+0000 7ff7beffd700 1 -- 172.21.15.92:0/3577227658 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d000cb30 con 0x7ff7d80cd4d0 2023-10-06T10:44:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.784+0000 7ff7dd754700 1 -- 172.21.15.92:0/3577227658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d80b5510 con 0x7ff7d80cd4d0 2023-10-06T10:44:13.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.784+0000 7ff7beffd700 1 -- 172.21.15.92:0/3577227658 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0013470 con 0x7ff7d80cd4d0 2023-10-06T10:44:13.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.784+0000 7ff7dd754700 1 -- 172.21.15.92:0/3577227658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d80b5ad0 con 0x7ff7d80cd4d0 2023-10-06T10:44:13.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.785+0000 7ff7beffd700 1 -- 172.21.15.92:0/3577227658 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff7d0013860 con 0x7ff7d80cd4d0 2023-10-06T10:44:13.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.785+0000 7ff7beffd700 1 --2- 172.21.15.92:0/3577227658 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff7c4066f30 0x7ff7c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.785+0000 7ff7beffd700 1 -- 172.21.15.92:0/3577227658 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff7d0084ec0 con 0x7ff7d80cd4d0 2023-10-06T10:44:13.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.785+0000 7ff7d67fc700 1 --2- 172.21.15.92:0/3577227658 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff7c4066f30 0x7ff7c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.786+0000 7ff7d67fc700 1 --2- 172.21.15.92:0/3577227658 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff7c4066f30 0x7ff7c40693e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff7d816cf00 tx=0x7ff7cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:13.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.787+0000 7ff7e6a4cb80 1 Processor -- start 2023-10-06T10:44:13.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.787+0000 7ff7e6a4cb80 1 -- start start 2023-10-06T10:44:13.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.788+0000 7ff7e6a4cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5582bc932ae0 0x5582bc932eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.788+0000 7ff7e6a4cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5582bc8ee9f0 0x5582bc8eb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.788+0000 7ff7e6a4cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ecc20 0x5582bc8ebb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.788+0000 7ff7e6a4cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582bc86bcd0 con 0x5582bc8ecc20 2023-10-06T10:44:13.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.788+0000 7ff7e6a4cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582bc842ac0 con 0x5582bc8ee9f0 2023-10-06T10:44:13.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.788+0000 7ff7e6a4cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582bc8402c0 con 0x5582bc932ae0 2023-10-06T10:44:13.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.788+0000 7ff7d77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ecc20 0x5582bc8ebb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.788+0000 7ff7d77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ecc20 0x5582bc8ebb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48196/0 (socket says 172.21.15.92:48196) 2023-10-06T10:44:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.788+0000 7ff7d77fe700 1 -- 172.21.15.92:0/1372076176 learned_addr learned my addr 172.21.15.92:0/1372076176 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.788+0000 7ff7d77fe700 1 -- 172.21.15.92:0/1372076176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5582bc932ae0 msgr2=0x5582bc932eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.788+0000 7ff7d6ffd700 1 --2- 172.21.15.92:0/1372076176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5582bc932ae0 0x5582bc932eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.788+0000 7ff7d67fc700 1 --2- 172.21.15.92:0/1372076176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5582bc8ee9f0 0x5582bc8eb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.788+0000 7ff7d77fe700 1 --2- 172.21.15.92:0/1372076176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5582bc932ae0 0x5582bc932eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.788+0000 7ff7d77fe700 1 -- 172.21.15.92:0/1372076176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5582bc8ee9f0 msgr2=0x5582bc8eb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.788+0000 7ff7d77fe700 1 --2- 172.21.15.92:0/1372076176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5582bc8ee9f0 0x5582bc8eb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.788+0000 7ff7d77fe700 1 -- 172.21.15.92:0/1372076176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582bc6d62c0 con 0x5582bc8ecc20 2023-10-06T10:44:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.789+0000 7ff7d6ffd700 1 --2- 172.21.15.92:0/1372076176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5582bc932ae0 0x5582bc932eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.789+0000 7ff7d67fc700 1 --2- 172.21.15.92:0/1372076176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5582bc8ee9f0 0x5582bc8eb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.789+0000 7ff7d77fe700 1 --2- 172.21.15.92:0/1372076176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ecc20 0x5582bc8ebb10 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800b6c0 tx=0x7ff7c8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684106dae94fbf7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.789+0000 7ff7b7fff700 1 -- 172.21.15.92:0/1372076176 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8007200 con 0x5582bc8ecc20 2023-10-06T10:44:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.789+0000 7ff7b7fff700 1 -- 172.21.15.92:0/1372076176 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c800be30 con 0x5582bc8ecc20 2023-10-06T10:44:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.789+0000 7ff7b7fff700 1 -- 172.21.15.92:0/1372076176 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8007610 con 0x5582bc8ecc20 2023-10-06T10:44:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.789+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/1372076176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ecc20 msgr2=0x5582bc8ebb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.789+0000 7ff7e6a4cb80 1 --2- 172.21.15.92:0/1372076176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ecc20 0x5582bc8ebb10 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800b6c0 tx=0x7ff7c8018000 comp rx=0 tx=0).stop 2023-10-06T10:44:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.789+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/1372076176 shutdown_connections 2023-10-06T10:44:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.789+0000 7ff7e6a4cb80 1 --2- 172.21.15.92:0/1372076176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ecc20 0x5582bc8ebb10 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.789+0000 7ff7e6a4cb80 1 --2- 172.21.15.92:0/1372076176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5582bc932ae0 0x5582bc932eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.789+0000 7ff7e6a4cb80 1 --2- 172.21.15.92:0/1372076176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5582bc8ee9f0 0x5582bc8eb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.789+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/1372076176 >> 172.21.15.92:0/1372076176 conn(0x5582bc8ed0c0 msgr2=0x5582bc8ed490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.790+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/1372076176 shutdown_connections 2023-10-06T10:44:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.790+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/1372076176 wait complete. 2023-10-06T10:44:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.790+0000 7ff7e6a4cb80 1 -- start start 2023-10-06T10:44:13.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.790+0000 7ff7e6a4cb80 1 Processor -- start 2023-10-06T10:44:13.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.791+0000 7ff7e6a4cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ecc20 0x5582bc92cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.791+0000 7ff7e6a4cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5582bc8ee9f0 0x5582bc92d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.791+0000 7ff7e6a4cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5582bc932ae0 0x5582bc913b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.791+0000 7ff7e6a4cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582bc83c0b0 con 0x5582bc8ecc20 2023-10-06T10:44:13.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.791+0000 7ff7e6a4cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582bc7e78e0 con 0x5582bc8ee9f0 2023-10-06T10:44:13.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.791+0000 7ff7e6a4cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582bc802ef0 con 0x5582bc932ae0 2023-10-06T10:44:13.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.791+0000 7ff7e6a4cb80 10 client.0 _ll_register_callbacks cb 0x5582bc6d8f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:13.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.791+0000 7ff7d6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ecc20 0x5582bc92cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.791+0000 7ff7d6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ecc20 0x5582bc92cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48200/0 (socket says 172.21.15.92:48200) 2023-10-06T10:44:13.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.791+0000 7ff7d6ffd700 1 -- 172.21.15.92:0/3669038708 learned_addr learned my addr 172.21.15.92:0/3669038708 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:13.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.791+0000 7ff7d67fc700 1 --2- 172.21.15.92:0/3669038708 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5582bc8ee9f0 0x5582bc92d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.791+0000 7ff7d77fe700 1 --2- 172.21.15.92:0/3669038708 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5582bc932ae0 0x5582bc913b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.791+0000 7ff7d6ffd700 1 -- 172.21.15.92:0/3669038708 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5582bc932ae0 msgr2=0x5582bc913b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.791+0000 7ff7d6ffd700 1 --2- 172.21.15.92:0/3669038708 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5582bc932ae0 0x5582bc913b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.791+0000 7ff7d6ffd700 1 -- 172.21.15.92:0/3669038708 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5582bc8ee9f0 msgr2=0x5582bc92d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.791+0000 7ff7d6ffd700 1 --2- 172.21.15.92:0/3669038708 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5582bc8ee9f0 0x5582bc92d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.791+0000 7ff7d6ffd700 1 -- 172.21.15.92:0/3669038708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d815a670 con 0x5582bc8ecc20 2023-10-06T10:44:13.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.791+0000 7ff7d67fc700 1 --2- 172.21.15.92:0/3669038708 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5582bc8ee9f0 0x5582bc92d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:13.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.792+0000 7ff7d6ffd700 1 --2- 172.21.15.92:0/3669038708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ecc20 0x5582bc92cf20 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000c7c0 tx=0x7ff7d00830a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97ca00fc3f0ee05 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:13.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.792+0000 7ff7b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:13.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.792+0000 7ff7b57fa700 1 -- 172.21.15.92:0/3669038708 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0051e90 con 0x5582bc8ecc20 2023-10-06T10:44:13.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.792+0000 7ff7b57fa700 1 -- 172.21.15.92:0/3669038708 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff7d0058da0 con 0x5582bc8ecc20 2023-10-06T10:44:13.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.792+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/3669038708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5582bc8e06e0 con 0x5582bc8ecc20 2023-10-06T10:44:13.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.792+0000 7ff7b57fa700 1 -- 172.21.15.92:0/3669038708 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7d005caf0 con 0x5582bc8ecc20 2023-10-06T10:44:13.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.792+0000 7ff7e6a4cb80 10 client.7983 fetch_fsmap learned FSMap version 90 2023-10-06T10:44:13.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.792+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/3669038708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x5582bc679030 con 0x5582bc8ecc20 2023-10-06T10:44:13.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.792+0000 7ff7b57fa700 1 -- 172.21.15.92:0/3669038708 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5582bc8e06e0 con 0x5582bc8ecc20 2023-10-06T10:44:13.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.792+0000 7ff7b57fa700 1 -- 172.21.15.92:0/3669038708 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 90) v1 ==== 3136+0+0 (secure 0 0 0) 0x7ff7d005c660 con 0x5582bc8ecc20 2023-10-06T10:44:13.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.792+0000 7ff7e6a4cb80 10 client.7983 fetch_fsmap finished waiting for FSMap version 90 2023-10-06T10:44:13.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.792+0000 7ff7e6a4cb80 10 client.7983 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.c 2023-10-06T10:44:13.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.792+0000 7ff7e6a4cb80 20 client.7983 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:13.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.792+0000 7ff7e6a4cb80 1 --2- 172.21.15.92:0/3669038708 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5582bc9183b0 0x5582bc914050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.792+0000 7ff7e6a4cb80 4 client.7983 mds_command: new command op to 7689 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:44:13.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.792+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/3669038708 --> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582bc8bd610 con 0x5582bc9183b0 2023-10-06T10:44:13.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.792+0000 7ff7d67fc700 1 --2- 172.21.15.92:0/3669038708 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5582bc9183b0 0x5582bc914050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.793+0000 7ff7b57fa700 10 client.7983 ms_handle_connect on v2:172.21.15.92:6835/4047304116 2023-10-06T10:44:13.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.793+0000 7ff7d67fc700 1 --2- 172.21.15.92:0/3669038708 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5582bc9183b0 0x5582bc914050 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61f71a14c3c7b428 server_cookie=d4aedae8f493ed00 in_seq=0 out_seq=0 2023-10-06T10:44:13.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.793+0000 7ff7b57fa700 1 -- 172.21.15.92:0/3669038708 <== mds.1 v2:172.21.15.92:6835/4047304116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5582bc8bd610 con 0x5582bc9183b0 2023-10-06T10:44:13.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.793+0000 7ff7b57fa700 10 client.7983 handle_command_reply: tid=0 2023-10-06T10:44:13.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.793+0000 7ff7e6a4cb80 1 client.7983 shutdown 2023-10-06T10:44:13.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.793+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/3669038708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ecc20 msgr2=0x5582bc92cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.793+0000 7ff7e6a4cb80 1 --2- 172.21.15.92:0/3669038708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ecc20 0x5582bc92cf20 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000c7c0 tx=0x7ff7d00830a0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.793+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/3669038708 shutdown_connections 2023-10-06T10:44:13.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.793+0000 7ff7e6a4cb80 1 --2- 172.21.15.92:0/3669038708 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5582bc9183b0 0x5582bc914050 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.793+0000 7ff7e6a4cb80 1 --2- 172.21.15.92:0/3669038708 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5582bc932ae0 0x5582bc913b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.793+0000 7ff7b57fa700 0 client.7983 ms_handle_reset on v2:172.21.15.92:6835/4047304116 2023-10-06T10:44:13.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.793+0000 7ff7e6a4cb80 1 --2- 172.21.15.92:0/3669038708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ecc20 0x5582bc92cf20 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.793+0000 7ff7e6a4cb80 1 --2- 172.21.15.92:0/3669038708 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5582bc8ee9f0 0x5582bc92d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.793+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/3669038708 >> 172.21.15.92:0/3669038708 conn(0x5582bc8ed0c0 msgr2=0x5582bc8e8870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:13.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.793+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/3669038708 shutdown_connections 2023-10-06T10:44:13.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.793+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/3669038708 wait complete. 2023-10-06T10:44:13.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.794+0000 7ff7e6a4cb80 20 client.7983 trim_cache size 0 max 16384 2023-10-06T10:44:13.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.807+0000 7ff7e6a4cb80 1 Processor -- start 2023-10-06T10:44:13.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.807+0000 7ff7e6a4cb80 1 -- start start 2023-10-06T10:44:13.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.808+0000 7ff7e6a4cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5582bc8ecc20 0x5582bc92d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.808+0000 7ff7e6a4cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ee9f0 0x5582bc9282b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.808+0000 7ff7e6a4cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5582bc932ae0 0x5582bc9287f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.808+0000 7ff7e6a4cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582bc6bee80 con 0x5582bc8ee9f0 2023-10-06T10:44:13.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.808+0000 7ff7e6a4cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582bc92df20 con 0x5582bc8ecc20 2023-10-06T10:44:13.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.808+0000 7ff7e6a4cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582bc942750 con 0x5582bc932ae0 2023-10-06T10:44:13.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.808+0000 7ff7d67fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ee9f0 0x5582bc9282b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.808+0000 7ff7d67fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ee9f0 0x5582bc9282b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48202/0 (socket says 172.21.15.92:48202) 2023-10-06T10:44:13.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.808+0000 7ff7d67fc700 1 -- 172.21.15.92:0/3849858141 learned_addr learned my addr 172.21.15.92:0/3849858141 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:13.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.808+0000 7ff7d67fc700 1 -- 172.21.15.92:0/3849858141 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5582bc932ae0 msgr2=0x5582bc9287f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.808+0000 7ff7d67fc700 1 --2- 172.21.15.92:0/3849858141 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5582bc932ae0 0x5582bc9287f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.808+0000 7ff7d67fc700 1 -- 172.21.15.92:0/3849858141 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5582bc8ecc20 msgr2=0x5582bc92d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.808+0000 7ff7d6ffd700 1 --2- 172.21.15.92:0/3849858141 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5582bc8ecc20 0x5582bc92d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.808+0000 7ff7d67fc700 1 --2- 172.21.15.92:0/3849858141 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5582bc8ecc20 0x5582bc92d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.808+0000 7ff7d67fc700 1 -- 172.21.15.92:0/3849858141 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582bc80a7d0 con 0x5582bc8ee9f0 2023-10-06T10:44:13.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.808+0000 7ff7d67fc700 1 --2- 172.21.15.92:0/3849858141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ee9f0 0x5582bc9282b0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0030f0 tx=0x7ff7cc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4144ace7062475 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:13.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.808+0000 7ff7bcff9700 1 -- 172.21.15.92:0/3849858141 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc0078b0 con 0x5582bc8ee9f0 2023-10-06T10:44:13.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.809+0000 7ff7bcff9700 1 -- 172.21.15.92:0/3849858141 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7cc025750 con 0x5582bc8ee9f0 2023-10-06T10:44:13.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.809+0000 7ff7bcff9700 1 -- 172.21.15.92:0/3849858141 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc0258e0 con 0x5582bc8ee9f0 2023-10-06T10:44:13.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.809+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/3849858141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ee9f0 msgr2=0x5582bc9282b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.809+0000 7ff7e6a4cb80 1 --2- 172.21.15.92:0/3849858141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ee9f0 0x5582bc9282b0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0030f0 tx=0x7ff7cc013000 comp rx=0 tx=0).stop 2023-10-06T10:44:13.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.809+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/3849858141 shutdown_connections 2023-10-06T10:44:13.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.809+0000 7ff7e6a4cb80 1 --2- 172.21.15.92:0/3849858141 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5582bc932ae0 0x5582bc9287f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.809+0000 7ff7e6a4cb80 1 --2- 172.21.15.92:0/3849858141 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5582bc8ecc20 0x5582bc92d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.809+0000 7ff7e6a4cb80 1 --2- 172.21.15.92:0/3849858141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ee9f0 0x5582bc9282b0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.809+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/3849858141 >> 172.21.15.92:0/3849858141 conn(0x5582bc8ed0c0 msgr2=0x5582bc8e8870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:13.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.809+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/3849858141 shutdown_connections 2023-10-06T10:44:13.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.809+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/3849858141 wait complete. 2023-10-06T10:44:13.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.810+0000 7ff7e6a4cb80 1 -- start start 2023-10-06T10:44:13.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.810+0000 7ff7e6a4cb80 1 Processor -- start 2023-10-06T10:44:13.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7e6a4cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ecc20 0x5582bc997f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7e6a4cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5582bc8ee9f0 0x5582bc935870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7e6a4cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5582bc932ae0 0x5582bc9282b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7e6a4cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582bc9364f0 con 0x5582bc8ecc20 2023-10-06T10:44:13.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7e6a4cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582bc997af0 con 0x5582bc932ae0 2023-10-06T10:44:13.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7e6a4cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582bc930e40 con 0x5582bc8ee9f0 2023-10-06T10:44:13.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7e6a4cb80 10 client.0 _ll_register_callbacks cb 0x5582bc6d8f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:13.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7d6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ecc20 0x5582bc997f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7d6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ecc20 0x5582bc997f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48206/0 (socket says 172.21.15.92:48206) 2023-10-06T10:44:13.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7d6ffd700 1 -- 172.21.15.92:0/2615228964 learned_addr learned my addr 172.21.15.92:0/2615228964 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:13.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7d77fe700 1 --2- 172.21.15.92:0/2615228964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5582bc932ae0 0x5582bc9282b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7d67fc700 1 --2- 172.21.15.92:0/2615228964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5582bc8ee9f0 0x5582bc935870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7d6ffd700 1 -- 172.21.15.92:0/2615228964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5582bc8ee9f0 msgr2=0x5582bc935870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7d6ffd700 1 --2- 172.21.15.92:0/2615228964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5582bc8ee9f0 0x5582bc935870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7d6ffd700 1 -- 172.21.15.92:0/2615228964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5582bc932ae0 msgr2=0x5582bc9282b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7d6ffd700 1 --2- 172.21.15.92:0/2615228964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5582bc932ae0 0x5582bc9282b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7d6ffd700 1 -- 172.21.15.92:0/2615228964 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7cc018040 con 0x5582bc8ecc20 2023-10-06T10:44:13.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7d6ffd700 1 --2- 172.21.15.92:0/2615228964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ecc20 0x5582bc997f80 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0052050 tx=0x7ff7d006e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513f00c6b99d26d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:13.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:13.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7b57fa700 1 -- 172.21.15.92:0/2615228964 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0051e90 con 0x5582bc8ecc20 2023-10-06T10:44:13.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7b57fa700 1 -- 172.21.15.92:0/2615228964 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff7d0058da0 con 0x5582bc8ecc20 2023-10-06T10:44:13.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/2615228964 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7c800be30 con 0x5582bc8ecc20 2023-10-06T10:44:13.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7b57fa700 1 -- 172.21.15.92:0/2615228964 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7d005caf0 con 0x5582bc8ecc20 2023-10-06T10:44:13.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7e6a4cb80 10 client.7989 fetch_fsmap learned FSMap version 90 2023-10-06T10:44:13.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/2615228964 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x5582bc7f14c0 con 0x5582bc8ecc20 2023-10-06T10:44:13.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.811+0000 7ff7b57fa700 1 -- 172.21.15.92:0/2615228964 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c800be30 con 0x5582bc8ecc20 2023-10-06T10:44:13.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.812+0000 7ff7b57fa700 1 -- 172.21.15.92:0/2615228964 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 90) v1 ==== 3136+0+0 (secure 0 0 0) 0x7ff7d005c660 con 0x5582bc8ecc20 2023-10-06T10:44:13.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.812+0000 7ff7e6a4cb80 10 client.7989 fetch_fsmap finished waiting for FSMap version 90 2023-10-06T10:44:13.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.812+0000 7ff7e6a4cb80 10 client.7989 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.c 2023-10-06T10:44:13.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.812+0000 7ff7e6a4cb80 20 client.7989 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:13.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.812+0000 7ff7e6a4cb80 1 --2- 172.21.15.92:0/2615228964 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5582bc94be30 0x5582bc9287f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.812+0000 7ff7e6a4cb80 4 client.7989 mds_command: new command op to 7689 tid=0[{"prefix": "flush journal"}] 2023-10-06T10:44:13.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.812+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/2615228964 --> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5582bc7dee50 con 0x5582bc94be30 2023-10-06T10:44:13.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.812+0000 7ff7d67fc700 1 --2- 172.21.15.92:0/2615228964 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5582bc94be30 0x5582bc9287f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.812+0000 7ff7b57fa700 10 client.7989 ms_handle_connect on v2:172.21.15.92:6835/4047304116 2023-10-06T10:44:13.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.812+0000 7ff7d67fc700 1 --2- 172.21.15.92:0/2615228964 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5582bc94be30 0x5582bc9287f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6cc108e4375af5b4 server_cookie=6425d11f4cf2feee in_seq=0 out_seq=0 2023-10-06T10:44:13.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.822+0000 7ff7b57fa700 1 -- 172.21.15.92:0/2615228964 <== mds.1 v2:172.21.15.92:6835/4047304116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5582bc7dee50 con 0x5582bc94be30 2023-10-06T10:44:13.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.822+0000 7ff7b57fa700 10 client.7989 handle_command_reply: tid=0 2023-10-06T10:44:13.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.822+0000 7ff7e6a4cb80 1 client.7989 shutdown 2023-10-06T10:44:13.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.822+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/2615228964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ecc20 msgr2=0x5582bc997f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.822+0000 7ff7e6a4cb80 1 --2- 172.21.15.92:0/2615228964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ecc20 0x5582bc997f80 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0052050 tx=0x7ff7d006e000 comp rx=0 tx=0).stop 2023-10-06T10:44:13.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.822+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/2615228964 shutdown_connections 2023-10-06T10:44:13.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.822+0000 7ff7e6a4cb80 1 --2- 172.21.15.92:0/2615228964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5582bc932ae0 0x5582bc9282b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.822+0000 7ff7e6a4cb80 1 --2- 172.21.15.92:0/2615228964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582bc8ecc20 0x5582bc997f80 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.822+0000 7ff7e6a4cb80 1 --2- 172.21.15.92:0/2615228964 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5582bc94be30 0x5582bc9287f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.822+0000 7ff7e6a4cb80 1 --2- 172.21.15.92:0/2615228964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5582bc8ee9f0 0x5582bc935870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.822+0000 7ff7b57fa700 0 client.7989 ms_handle_reset on v2:172.21.15.92:6835/4047304116 2023-10-06T10:44:13.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.822+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/2615228964 >> 172.21.15.92:0/2615228964 conn(0x5582bc8ed0c0 msgr2=0x5582bc92d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:13.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.822+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/2615228964 shutdown_connections 2023-10-06T10:44:13.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.822+0000 7ff7e6a4cb80 1 -- 172.21.15.92:0/2615228964 wait complete. 2023-10-06T10:44:13.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.823+0000 7ff7e6a4cb80 20 client.7989 trim_cache size 0 max 16384 2023-10-06T10:44:13.829 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:44:13.829 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2023-10-06T10:44:13.830 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:44:13.830 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:44:13.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.823+0000 7ff7dd754700 1 -- 172.21.15.92:0/3577227658 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff7c4066f30 msgr2=0x7ff7c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.823+0000 7ff7dd754700 1 --2- 172.21.15.92:0/3577227658 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff7c4066f30 0x7ff7c40693e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff7d816cf00 tx=0x7ff7cc009040 comp rx=0 tx=0).stop 2023-10-06T10:44:13.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.823+0000 7ff7dd754700 1 -- 172.21.15.92:0/3577227658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7d80cd4d0 msgr2=0x7ff7d80c7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.823+0000 7ff7dd754700 1 --2- 172.21.15.92:0/3577227658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7d80cd4d0 0x7ff7d80c7f70 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000ec80 tx=0x7ff7d0009120 comp rx=0 tx=0).stop 2023-10-06T10:44:13.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.823+0000 7ff7dd754700 1 -- 172.21.15.92:0/3577227658 shutdown_connections 2023-10-06T10:44:13.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.823+0000 7ff7dd754700 1 --2- 172.21.15.92:0/3577227658 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff7c4066f30 0x7ff7c40693e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.823+0000 7ff7dd754700 1 --2- 172.21.15.92:0/3577227658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7d80b43d0 0x7ff7d80b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.823+0000 7ff7dd754700 1 --2- 172.21.15.92:0/3577227658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7d80cd4d0 0x7ff7d80c7f70 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.823+0000 7ff7dd754700 1 --2- 172.21.15.92:0/3577227658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7d815a160 0x7ff7d814eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.823+0000 7ff7dd754700 1 -- 172.21.15.92:0/3577227658 >> 172.21.15.92:0/3577227658 conn(0x7ff7d8099700 msgr2=0x7ff7d80cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:13.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.823+0000 7ff7dd754700 1 -- 172.21.15.92:0/3577227658 shutdown_connections 2023-10-06T10:44:13.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.823+0000 7ff7dd754700 1 -- 172.21.15.92:0/3577227658 wait complete. 2023-10-06T10:44:13.837 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:2 flush journal 2023-10-06T10:44:13.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.951+0000 7f49ded16700 1 Processor -- start 2023-10-06T10:44:13.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.951+0000 7f49ded16700 1 -- start start 2023-10-06T10:44:13.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.952+0000 7f49ded16700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f49d80cd3c0 0x7f49d80ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.952+0000 7f49ded16700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d80cc970 con 0x7f49d80b2620 2023-10-06T10:44:13.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.952+0000 7f49ded16700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d80cbd60 con 0x7f49d80b29f0 2023-10-06T10:44:13.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.952+0000 7f49ded16700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d80cc040 con 0x7f49d80cd3c0 2023-10-06T10:44:13.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.952+0000 7f49dd513700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f49d80cd3c0 0x7f49d80ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.952+0000 7f49dd513700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f49d80cd3c0 0x7f49d80ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48216/0 (socket says 172.21.15.92:48216) 2023-10-06T10:44:13.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.952+0000 7f49ddd14700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f49d80b2620 0x7f49d80cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44568/0 (socket says 172.21.15.92:44568) 2023-10-06T10:44:13.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.952+0000 7f49dd513700 1 -- 172.21.15.92:0/347939552 learned_addr learned my addr 172.21.15.92:0/347939552 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:13.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.952+0000 7f49dd513700 1 -- 172.21.15.92:0/347939552 >> v1:172.21.15.124:6789/0 conn(0x7f49d80b29f0 legacy=0x7f49d80c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.952+0000 7f49dd513700 1 -- 172.21.15.92:0/347939552 >> v1:172.21.15.92:6789/0 conn(0x7f49d80b2620 legacy=0x7f49d80cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.952+0000 7f49dd513700 1 -- 172.21.15.92:0/347939552 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d80cc320 con 0x7f49d80cd3c0 2023-10-06T10:44:13.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.952+0000 7f49dcd12700 1 -- 172.21.15.92:0/347939552 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 564323495 0 0) 0x7f49d80cc970 con 0x7f49d80b2620 2023-10-06T10:44:13.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.953+0000 7f49dd513700 1 --2- 172.21.15.92:0/347939552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49d80cd3c0 0x7f49d80ceaa0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f49c80099b0 tx=0x7f49c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c92bccbb9080a77 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:13.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.953+0000 7f49dcd12700 1 -- 172.21.15.92:0/347939552 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c800b820 con 0x7f49d80cd3c0 2023-10-06T10:44:13.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.953+0000 7f49dcd12700 1 -- 172.21.15.92:0/347939552 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c800b980 con 0x7f49d80cd3c0 2023-10-06T10:44:13.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.953+0000 7f49dcd12700 1 -- 172.21.15.92:0/347939552 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c800bd20 con 0x7f49d80cd3c0 2023-10-06T10:44:13.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.953+0000 7f49ded16700 1 -- 172.21.15.92:0/347939552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49d80cd3c0 msgr2=0x7f49d80ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.953+0000 7f49ded16700 1 --2- 172.21.15.92:0/347939552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49d80cd3c0 0x7f49d80ceaa0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f49c80099b0 tx=0x7f49c8009250 comp rx=0 tx=0).stop 2023-10-06T10:44:13.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.953+0000 7f49ded16700 1 -- 172.21.15.92:0/347939552 shutdown_connections 2023-10-06T10:44:13.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.953+0000 7f49ded16700 1 --2- 172.21.15.92:0/347939552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49d80cd3c0 0x7f49d80ceaa0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.953+0000 7f49ded16700 1 -- 172.21.15.92:0/347939552 >> 172.21.15.92:0/347939552 conn(0x7f49d8099520 msgr2=0x7f49d806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:13.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.953+0000 7f49ded16700 1 -- 172.21.15.92:0/347939552 shutdown_connections 2023-10-06T10:44:13.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.953+0000 7f49ded16700 1 -- 172.21.15.92:0/347939552 wait complete. 2023-10-06T10:44:13.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.954+0000 7f49ded16700 1 Processor -- start 2023-10-06T10:44:13.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.955+0000 7f49ded16700 1 -- start start 2023-10-06T10:44:13.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.955+0000 7f49ded16700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49d80b2620 0x7f49d8147c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.955+0000 7f49ded16700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49d80b29f0 0x7f49d815adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.955+0000 7f49ded16700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49d80cd3c0 0x7f49d815b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.955+0000 7f49ded16700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d8169520 con 0x7f49d80cd3c0 2023-10-06T10:44:13.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.955+0000 7f49ded16700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d81697e0 con 0x7f49d80b29f0 2023-10-06T10:44:13.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.955+0000 7f49ded16700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d8169ac0 con 0x7f49d80b2620 2023-10-06T10:44:13.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.955+0000 7f49de515700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49d80cd3c0 0x7f49d815b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.955+0000 7f49de515700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49d80cd3c0 0x7f49d815b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48218/0 (socket says 172.21.15.92:48218) 2023-10-06T10:44:13.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.955+0000 7f49de515700 1 -- 172.21.15.92:0/2384915157 learned_addr learned my addr 172.21.15.92:0/2384915157 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:13.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.956+0000 7f49de515700 1 -- 172.21.15.92:0/2384915157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49d80b2620 msgr2=0x7f49d8147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.956+0000 7f49de515700 1 --2- 172.21.15.92:0/2384915157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49d80b2620 0x7f49d8147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.956+0000 7f49de515700 1 -- 172.21.15.92:0/2384915157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49d80b29f0 msgr2=0x7f49d815adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.956+0000 7f49dd513700 1 --2- 172.21.15.92:0/2384915157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49d80b29f0 0x7f49d815adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.956+0000 7f49de515700 1 --2- 172.21.15.92:0/2384915157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49d80b29f0 0x7f49d815adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.956+0000 7f49de515700 1 -- 172.21.15.92:0/2384915157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c80094f0 con 0x7f49d80cd3c0 2023-10-06T10:44:13.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.956+0000 7f49de515700 1 --2- 172.21.15.92:0/2384915157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49d80cd3c0 0x7f49d815b300 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f49d000c890 tx=0x7f49d00026a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:13.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.956+0000 7f49cdffb700 1 -- 172.21.15.92:0/2384915157 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0012630 con 0x7f49d80cd3c0 2023-10-06T10:44:13.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.956+0000 7f49cdffb700 1 -- 172.21.15.92:0/2384915157 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d0012790 con 0x7f49d80cd3c0 2023-10-06T10:44:13.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.956+0000 7f49ded16700 1 -- 172.21.15.92:0/2384915157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d8169da0 con 0x7f49d80cd3c0 2023-10-06T10:44:13.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.956+0000 7f49cdffb700 1 -- 172.21.15.92:0/2384915157 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0012b50 con 0x7f49d80cd3c0 2023-10-06T10:44:13.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.956+0000 7f49ded16700 1 -- 172.21.15.92:0/2384915157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d816a360 con 0x7f49d80cd3c0 2023-10-06T10:44:13.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.957+0000 7f49cdffb700 1 -- 172.21.15.92:0/2384915157 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f49d000a420 con 0x7f49d80cd3c0 2023-10-06T10:44:13.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.957+0000 7f49cdffb700 1 --2- 172.21.15.92:0/2384915157 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f49bc066f30 0x7f49bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.958+0000 7f49cdffb700 1 -- 172.21.15.92:0/2384915157 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f49d0084cb0 con 0x7f49d80cd3c0 2023-10-06T10:44:13.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.958+0000 7f49ddd14700 1 --2- 172.21.15.92:0/2384915157 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f49bc066f30 0x7f49bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.958+0000 7f49ddd14700 1 --2- 172.21.15.92:0/2384915157 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f49bc066f30 0x7f49bc0693e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f49d4007fd0 tx=0x7f49d4007b00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:13.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.960+0000 7f49e9270b80 1 Processor -- start 2023-10-06T10:44:13.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.960+0000 7f49e9270b80 1 -- start start 2023-10-06T10:44:13.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.960+0000 7f49e9270b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34949a0 0x55c7e3494d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.960+0000 7f49e9270b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7e34dec10 0x55c7e34bd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.960+0000 7f49e9270b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7e34bdeb0 0x55c7e34909e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.960+0000 7f49e9270b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7e3414cd0 con 0x55c7e34949a0 2023-10-06T10:44:13.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.960+0000 7f49e9270b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7e33ebac0 con 0x55c7e34dec10 2023-10-06T10:44:13.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.960+0000 7f49e9270b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7e33e92c0 con 0x55c7e34bdeb0 2023-10-06T10:44:13.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.961+0000 7f49ddd14700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34949a0 0x55c7e3494d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.961+0000 7f49ddd14700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34949a0 0x55c7e3494d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48224/0 (socket says 172.21.15.92:48224) 2023-10-06T10:44:13.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.961+0000 7f49ddd14700 1 -- 172.21.15.92:0/959562327 learned_addr learned my addr 172.21.15.92:0/959562327 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:13.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.961+0000 7f49de515700 1 --2- 172.21.15.92:0/959562327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7e34bdeb0 0x55c7e34909e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.961+0000 7f49dd513700 1 --2- 172.21.15.92:0/959562327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7e34dec10 0x55c7e34bd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.961+0000 7f49ddd14700 1 -- 172.21.15.92:0/959562327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7e34bdeb0 msgr2=0x55c7e34909e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.961+0000 7f49ddd14700 1 --2- 172.21.15.92:0/959562327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7e34bdeb0 0x55c7e34909e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.961+0000 7f49ddd14700 1 -- 172.21.15.92:0/959562327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7e34dec10 msgr2=0x55c7e34bd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.961+0000 7f49ddd14700 1 --2- 172.21.15.92:0/959562327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7e34dec10 0x55c7e34bd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.961+0000 7f49ddd14700 1 -- 172.21.15.92:0/959562327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7e327f2c0 con 0x55c7e34949a0 2023-10-06T10:44:13.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.961+0000 7f49ddd14700 1 --2- 172.21.15.92:0/959562327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34949a0 0x55c7e3494d70 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f49d4007640 tx=0x7f49d4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f5d5659b435976 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:13.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.961+0000 7f49c2ffd700 1 -- 172.21.15.92:0/959562327 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d401b070 con 0x55c7e34949a0 2023-10-06T10:44:13.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.961+0000 7f49c2ffd700 1 -- 172.21.15.92:0/959562327 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d4007670 con 0x55c7e34949a0 2023-10-06T10:44:13.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.961+0000 7f49c2ffd700 1 -- 172.21.15.92:0/959562327 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d40128a0 con 0x55c7e34949a0 2023-10-06T10:44:13.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.962+0000 7f49e9270b80 1 -- 172.21.15.92:0/959562327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34949a0 msgr2=0x55c7e3494d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.962+0000 7f49e9270b80 1 --2- 172.21.15.92:0/959562327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34949a0 0x55c7e3494d70 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f49d4007640 tx=0x7f49d4011000 comp rx=0 tx=0).stop 2023-10-06T10:44:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.962+0000 7f49e9270b80 1 -- 172.21.15.92:0/959562327 shutdown_connections 2023-10-06T10:44:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.962+0000 7f49e9270b80 1 --2- 172.21.15.92:0/959562327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7e34bdeb0 0x55c7e34909e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.962+0000 7f49e9270b80 1 --2- 172.21.15.92:0/959562327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34949a0 0x55c7e3494d70 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.962+0000 7f49e9270b80 1 --2- 172.21.15.92:0/959562327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7e34dec10 0x55c7e34bd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.962+0000 7f49e9270b80 1 -- 172.21.15.92:0/959562327 >> 172.21.15.92:0/959562327 conn(0x55c7e34962c0 msgr2=0x55c7e3496690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.962+0000 7f49e9270b80 1 -- 172.21.15.92:0/959562327 shutdown_connections 2023-10-06T10:44:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.962+0000 7f49e9270b80 1 -- 172.21.15.92:0/959562327 wait complete. 2023-10-06T10:44:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.962+0000 7f49e9270b80 1 -- start start 2023-10-06T10:44:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.962+0000 7f49e9270b80 1 Processor -- start 2023-10-06T10:44:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.963+0000 7f49e9270b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7e34949a0 0x55c7e34d8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.963+0000 7f49e9270b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7e34bdeb0 0x55c7e34d62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.963+0000 7f49e9270b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34dec10 0x55c7e34d67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.963+0000 7f49e9270b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7e33e50b0 con 0x55c7e34dec10 2023-10-06T10:44:13.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.963+0000 7f49e9270b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7e33908e0 con 0x55c7e34bdeb0 2023-10-06T10:44:13.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.963+0000 7f49e9270b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7e33abef0 con 0x55c7e34949a0 2023-10-06T10:44:13.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.963+0000 7f49e9270b80 10 client.0 _ll_register_callbacks cb 0x55c7e3281f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:13.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.963+0000 7f49de515700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34dec10 0x55c7e34d67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.963+0000 7f49de515700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34dec10 0x55c7e34d67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48238/0 (socket says 172.21.15.92:48238) 2023-10-06T10:44:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.963+0000 7f49de515700 1 -- 172.21.15.92:0/894956021 learned_addr learned my addr 172.21.15.92:0/894956021 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49ddd14700 1 --2- 172.21.15.92:0/894956021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7e34949a0 0x55c7e34d8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49dd513700 1 --2- 172.21.15.92:0/894956021 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7e34bdeb0 0x55c7e34d62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49de515700 1 -- 172.21.15.92:0/894956021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7e34949a0 msgr2=0x55c7e34d8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49de515700 1 --2- 172.21.15.92:0/894956021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7e34949a0 0x55c7e34d8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49de515700 1 -- 172.21.15.92:0/894956021 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7e34bdeb0 msgr2=0x55c7e34d62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49de515700 1 --2- 172.21.15.92:0/894956021 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7e34bdeb0 0x55c7e34d62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49de515700 1 -- 172.21.15.92:0/894956021 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d40072d0 con 0x55c7e34dec10 2023-10-06T10:44:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49de515700 1 --2- 172.21.15.92:0/894956021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34dec10 0x55c7e34d67f0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f49d000c860 tx=0x7f49d0014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a33aaf543938a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49b7fff700 1 -- 172.21.15.92:0/894956021 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d005c840 con 0x55c7e34dec10 2023-10-06T10:44:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49b7fff700 1 -- 172.21.15.92:0/894956021 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f49d0054d90 con 0x55c7e34dec10 2023-10-06T10:44:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49e9270b80 1 -- 172.21.15.92:0/894956021 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7e34896e0 con 0x55c7e34dec10 2023-10-06T10:44:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49b7fff700 1 -- 172.21.15.92:0/894956021 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49d0071270 con 0x55c7e34dec10 2023-10-06T10:44:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49e9270b80 10 client.8004 fetch_fsmap learned FSMap version 90 2023-10-06T10:44:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49e9270b80 1 -- 172.21.15.92:0/894956021 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x55c7e3222030 con 0x55c7e34dec10 2023-10-06T10:44:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49b7fff700 1 -- 172.21.15.92:0/894956021 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d0060020 con 0x55c7e34dec10 2023-10-06T10:44:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49b7fff700 1 -- 172.21.15.92:0/894956021 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 90) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f49d005c3b0 con 0x55c7e34dec10 2023-10-06T10:44:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49e9270b80 10 client.8004 fetch_fsmap finished waiting for FSMap version 90 2023-10-06T10:44:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49e9270b80 10 client.8004 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.a 2023-10-06T10:44:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49e9270b80 20 client.8004 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49e9270b80 1 --2- 172.21.15.92:0/894956021 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x55c7e348bbd0 0x55c7e34ddb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49e9270b80 4 client.8004 mds_command: new command op to 7686 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:44:13.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.964+0000 7f49e9270b80 1 -- 172.21.15.92:0/894956021 --> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7e3466610 con 0x55c7e348bbd0 2023-10-06T10:44:13.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.965+0000 7f49ddd14700 1 --2- 172.21.15.92:0/894956021 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x55c7e348bbd0 0x55c7e34ddb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.965+0000 7f49b7fff700 10 client.8004 ms_handle_connect on v2:172.21.15.92:6834/3605297930 2023-10-06T10:44:13.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.965+0000 7f49ddd14700 1 --2- 172.21.15.92:0/894956021 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x55c7e348bbd0 0x55c7e34ddb20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=79c79dca59740132 server_cookie=4bb8b250e6a452af in_seq=0 out_seq=0 2023-10-06T10:44:13.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.965+0000 7f49b7fff700 1 -- 172.21.15.92:0/894956021 <== mds.2 v2:172.21.15.92:6834/3605297930 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55c7e3466610 con 0x55c7e348bbd0 2023-10-06T10:44:13.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.965+0000 7f49b7fff700 10 client.8004 handle_command_reply: tid=0 2023-10-06T10:44:13.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.965+0000 7f49e9270b80 1 client.8004 shutdown 2023-10-06T10:44:13.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.965+0000 7f49e9270b80 1 -- 172.21.15.92:0/894956021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34dec10 msgr2=0x55c7e34d67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.965+0000 7f49e9270b80 1 --2- 172.21.15.92:0/894956021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34dec10 0x55c7e34d67f0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f49d000c860 tx=0x7f49d0014040 comp rx=0 tx=0).stop 2023-10-06T10:44:13.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.965+0000 7f49e9270b80 1 -- 172.21.15.92:0/894956021 shutdown_connections 2023-10-06T10:44:13.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.965+0000 7f49e9270b80 1 --2- 172.21.15.92:0/894956021 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x55c7e348bbd0 0x55c7e34ddb20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.965+0000 7f49e9270b80 1 --2- 172.21.15.92:0/894956021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34dec10 0x55c7e34d67f0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.965+0000 7f49e9270b80 1 --2- 172.21.15.92:0/894956021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7e34949a0 0x55c7e34d8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.965+0000 7f49b7fff700 0 client.8004 ms_handle_reset on v2:172.21.15.92:6834/3605297930 2023-10-06T10:44:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.965+0000 7f49e9270b80 1 --2- 172.21.15.92:0/894956021 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7e34bdeb0 0x55c7e34d62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.966+0000 7f49e9270b80 1 -- 172.21.15.92:0/894956021 >> 172.21.15.92:0/894956021 conn(0x55c7e34962c0 msgr2=0x55c7e34da4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.966+0000 7f49e9270b80 1 -- 172.21.15.92:0/894956021 shutdown_connections 2023-10-06T10:44:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.966+0000 7f49e9270b80 1 -- 172.21.15.92:0/894956021 wait complete. 2023-10-06T10:44:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.966+0000 7f49e9270b80 20 client.8004 trim_cache size 0 max 16384 2023-10-06T10:44:13.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.979+0000 7f49e9270b80 1 Processor -- start 2023-10-06T10:44:13.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.979+0000 7f49e9270b80 1 -- start start 2023-10-06T10:44:13.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.980+0000 7f49e9270b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7e34949a0 0x55c7e3491460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.980+0000 7f49e9270b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7e34bdeb0 0x55c7e34d75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.980+0000 7f49e9270b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34dec10 0x55c7e34d7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.980+0000 7f49e9270b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7e3267e80 con 0x55c7e34dec10 2023-10-06T10:44:13.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.980+0000 7f49e9270b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7e34dd3f0 con 0x55c7e34949a0 2023-10-06T10:44:13.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.980+0000 7f49e9270b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7e34d90d0 con 0x55c7e34bdeb0 2023-10-06T10:44:13.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.980+0000 7f49de515700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34dec10 0x55c7e34d7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.980+0000 7f49de515700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34dec10 0x55c7e34d7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48242/0 (socket says 172.21.15.92:48242) 2023-10-06T10:44:13.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.980+0000 7f49de515700 1 -- 172.21.15.92:0/3405661964 learned_addr learned my addr 172.21.15.92:0/3405661964 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:13.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.980+0000 7f49de515700 1 -- 172.21.15.92:0/3405661964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7e34bdeb0 msgr2=0x55c7e34d75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.980+0000 7f49dd513700 1 --2- 172.21.15.92:0/3405661964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7e34bdeb0 0x55c7e34d75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.980+0000 7f49de515700 1 --2- 172.21.15.92:0/3405661964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7e34bdeb0 0x55c7e34d75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.980+0000 7f49de515700 1 -- 172.21.15.92:0/3405661964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7e34949a0 msgr2=0x55c7e3491460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.981+0000 7f49de515700 1 --2- 172.21.15.92:0/3405661964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7e34949a0 0x55c7e3491460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.981+0000 7f49de515700 1 -- 172.21.15.92:0/3405661964 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7e3490b40 con 0x55c7e34dec10 2023-10-06T10:44:13.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.981+0000 7f49de515700 1 --2- 172.21.15.92:0/3405661964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34dec10 0x55c7e34d7b20 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f49d0053bf0 tx=0x7f49d006d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef88c200b8d21d7b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:13.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.981+0000 7f49c37fe700 1 -- 172.21.15.92:0/3405661964 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0054d90 con 0x55c7e34dec10 2023-10-06T10:44:13.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.981+0000 7f49c37fe700 1 -- 172.21.15.92:0/3405661964 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d0060040 con 0x55c7e34dec10 2023-10-06T10:44:13.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.981+0000 7f49c37fe700 1 -- 172.21.15.92:0/3405661964 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d005c840 con 0x55c7e34dec10 2023-10-06T10:44:13.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.981+0000 7f49e9270b80 1 -- 172.21.15.92:0/3405661964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34dec10 msgr2=0x55c7e34d7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.981+0000 7f49e9270b80 1 --2- 172.21.15.92:0/3405661964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34dec10 0x55c7e34d7b20 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f49d0053bf0 tx=0x7f49d006d000 comp rx=0 tx=0).stop 2023-10-06T10:44:13.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.981+0000 7f49e9270b80 1 -- 172.21.15.92:0/3405661964 shutdown_connections 2023-10-06T10:44:13.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.981+0000 7f49e9270b80 1 --2- 172.21.15.92:0/3405661964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34dec10 0x55c7e34d7b20 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.981+0000 7f49e9270b80 1 --2- 172.21.15.92:0/3405661964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7e34949a0 0x55c7e3491460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.981+0000 7f49e9270b80 1 --2- 172.21.15.92:0/3405661964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7e34bdeb0 0x55c7e34d75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.981+0000 7f49e9270b80 1 -- 172.21.15.92:0/3405661964 >> 172.21.15.92:0/3405661964 conn(0x55c7e34962c0 msgr2=0x55c7e34d62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:13.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.981+0000 7f49e9270b80 1 -- 172.21.15.92:0/3405661964 shutdown_connections 2023-10-06T10:44:13.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.981+0000 7f49e9270b80 1 -- 172.21.15.92:0/3405661964 wait complete. 2023-10-06T10:44:13.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.982+0000 7f49e9270b80 1 -- start start 2023-10-06T10:44:13.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.982+0000 7f49e9270b80 1 Processor -- start 2023-10-06T10:44:13.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.983+0000 7f49e9270b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7e34949a0 0x55c7e3543ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.983+0000 7f49e9270b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34bdeb0 0x55c7e34ec290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.983+0000 7f49e9270b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7e34dec10 0x55c7e34ec7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.983+0000 7f49e9270b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7e3542100 con 0x55c7e34bdeb0 2023-10-06T10:44:13.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.983+0000 7f49e9270b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7e34b9080 con 0x55c7e34949a0 2023-10-06T10:44:13.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.983+0000 7f49e9270b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7e34b5da0 con 0x55c7e34dec10 2023-10-06T10:44:13.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.983+0000 7f49e9270b80 10 client.0 _ll_register_callbacks cb 0x55c7e3281f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:13.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.983+0000 7f49dd513700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34bdeb0 0x55c7e34ec290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.983+0000 7f49dd513700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34bdeb0 0x55c7e34ec290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48256/0 (socket says 172.21.15.92:48256) 2023-10-06T10:44:13.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.983+0000 7f49dd513700 1 -- 172.21.15.92:0/36754495 learned_addr learned my addr 172.21.15.92:0/36754495 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:13.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.983+0000 7f49ddd14700 1 --2- 172.21.15.92:0/36754495 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7e34949a0 0x55c7e3543ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.983+0000 7f49de515700 1 --2- 172.21.15.92:0/36754495 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7e34dec10 0x55c7e34ec7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.983+0000 7f49dd513700 1 -- 172.21.15.92:0/36754495 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7e34dec10 msgr2=0x55c7e34ec7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.983+0000 7f49dd513700 1 --2- 172.21.15.92:0/36754495 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7e34dec10 0x55c7e34ec7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.983+0000 7f49dd513700 1 -- 172.21.15.92:0/36754495 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7e34949a0 msgr2=0x55c7e3543ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:13.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.983+0000 7f49dd513700 1 --2- 172.21.15.92:0/36754495 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7e34949a0 0x55c7e3543ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.983+0000 7f49dd513700 1 -- 172.21.15.92:0/36754495 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7e3222030 con 0x55c7e34bdeb0 2023-10-06T10:44:13.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.983+0000 7f49dd513700 1 --2- 172.21.15.92:0/36754495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34bdeb0 0x55c7e34ec290 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f49c8006fa0 tx=0x7f49c8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2748d5d320797396 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:13.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.983+0000 7f49b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:13.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.984+0000 7f49b7fff700 1 -- 172.21.15.92:0/36754495 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c8023050 con 0x55c7e34bdeb0 2023-10-06T10:44:13.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.984+0000 7f49b7fff700 1 -- 172.21.15.92:0/36754495 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f49c8011840 con 0x55c7e34bdeb0 2023-10-06T10:44:13.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.984+0000 7f49e9270b80 1 -- 172.21.15.92:0/36754495 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49d4007670 con 0x55c7e34bdeb0 2023-10-06T10:44:13.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.984+0000 7f49b7fff700 1 -- 172.21.15.92:0/36754495 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49c8011c10 con 0x55c7e34bdeb0 2023-10-06T10:44:13.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.984+0000 7f49e9270b80 10 client.8010 fetch_fsmap learned FSMap version 90 2023-10-06T10:44:13.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.984+0000 7f49e9270b80 1 -- 172.21.15.92:0/36754495 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x55c7e33b37d0 con 0x55c7e34bdeb0 2023-10-06T10:44:13.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.984+0000 7f49b7fff700 1 -- 172.21.15.92:0/36754495 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d4007670 con 0x55c7e34bdeb0 2023-10-06T10:44:13.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.984+0000 7f49b7fff700 1 -- 172.21.15.92:0/36754495 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 90) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f49c8018070 con 0x55c7e34bdeb0 2023-10-06T10:44:13.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.984+0000 7f49e9270b80 10 client.8010 fetch_fsmap finished waiting for FSMap version 90 2023-10-06T10:44:13.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.984+0000 7f49e9270b80 10 client.8010 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.a 2023-10-06T10:44:13.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.984+0000 7f49e9270b80 20 client.8010 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:13.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.984+0000 7f49e9270b80 1 --2- 172.21.15.92:0/36754495 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x55c7e34bd990 0x55c7e3539f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:13.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.984+0000 7f49e9270b80 4 client.8010 mds_command: new command op to 7686 tid=0[{"prefix": "flush journal"}] 2023-10-06T10:44:13.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.984+0000 7f49e9270b80 1 -- 172.21.15.92:0/36754495 --> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c7e3387e50 con 0x55c7e34bd990 2023-10-06T10:44:13.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.984+0000 7f49ddd14700 1 --2- 172.21.15.92:0/36754495 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x55c7e34bd990 0x55c7e3539f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:13.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.984+0000 7f49ddd14700 1 --2- 172.21.15.92:0/36754495 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x55c7e34bd990 0x55c7e3539f10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e1c9f4ee9cba922e server_cookie=d9eb1a624c7977fe in_seq=0 out_seq=0 2023-10-06T10:44:13.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.984+0000 7f49b7fff700 10 client.8010 ms_handle_connect on v2:172.21.15.92:6834/3605297930 2023-10-06T10:44:13.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.994+0000 7f49b7fff700 1 -- 172.21.15.92:0/36754495 <== mds.2 v2:172.21.15.92:6834/3605297930 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c7e3387e50 con 0x55c7e34bd990 2023-10-06T10:44:13.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.994+0000 7f49b7fff700 10 client.8010 handle_command_reply: tid=0 2023-10-06T10:44:13.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.995+0000 7f49e9270b80 1 client.8010 shutdown 2023-10-06T10:44:13.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.995+0000 7f49e9270b80 1 -- 172.21.15.92:0/36754495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34bdeb0 msgr2=0x55c7e34ec290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:13.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.995+0000 7f49e9270b80 1 --2- 172.21.15.92:0/36754495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34bdeb0 0x55c7e34ec290 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f49c8006fa0 tx=0x7f49c8016000 comp rx=0 tx=0).stop 2023-10-06T10:44:13.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.995+0000 7f49e9270b80 1 -- 172.21.15.92:0/36754495 shutdown_connections 2023-10-06T10:44:13.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.995+0000 7f49e9270b80 1 --2- 172.21.15.92:0/36754495 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c7e34dec10 0x55c7e34ec7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.995+0000 7f49e9270b80 1 --2- 172.21.15.92:0/36754495 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c7e34949a0 0x55c7e3543ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.995+0000 7f49e9270b80 1 --2- 172.21.15.92:0/36754495 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x55c7e34bd990 0x55c7e3539f10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.995+0000 7f49e9270b80 1 --2- 172.21.15.92:0/36754495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7e34bdeb0 0x55c7e34ec290 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:13.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.995+0000 7f49e9270b80 1 -- 172.21.15.92:0/36754495 >> 172.21.15.92:0/36754495 conn(0x55c7e34962c0 msgr2=0x55c7e34d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:13.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.995+0000 7f49b7fff700 0 client.8010 ms_handle_reset on v2:172.21.15.92:6834/3605297930 2023-10-06T10:44:13.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.995+0000 7f49e9270b80 1 -- 172.21.15.92:0/36754495 shutdown_connections 2023-10-06T10:44:13.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.995+0000 7f49e9270b80 1 -- 172.21.15.92:0/36754495 wait complete. 2023-10-06T10:44:13.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.995+0000 7f49e9270b80 20 client.8010 trim_cache size 0 max 16384 2023-10-06T10:44:13.998 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:44:13.999 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2023-10-06T10:44:13.999 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:44:13.999 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:44:14.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.995+0000 7f49ded16700 1 -- 172.21.15.92:0/2384915157 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f49bc066f30 msgr2=0x7f49bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:14.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.995+0000 7f49ded16700 1 --2- 172.21.15.92:0/2384915157 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f49bc066f30 0x7f49bc0693e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f49d4007fd0 tx=0x7f49d4007b00 comp rx=0 tx=0).stop 2023-10-06T10:44:14.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.995+0000 7f49ded16700 1 -- 172.21.15.92:0/2384915157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49d80cd3c0 msgr2=0x7f49d815b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:14.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.995+0000 7f49ded16700 1 --2- 172.21.15.92:0/2384915157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49d80cd3c0 0x7f49d815b300 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f49d000c890 tx=0x7f49d00026a0 comp rx=0 tx=0).stop 2023-10-06T10:44:14.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.996+0000 7f49ded16700 1 -- 172.21.15.92:0/2384915157 shutdown_connections 2023-10-06T10:44:14.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.996+0000 7f49ded16700 1 --2- 172.21.15.92:0/2384915157 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f49bc066f30 0x7f49bc0693e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:14.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.996+0000 7f49ded16700 1 --2- 172.21.15.92:0/2384915157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49d80cd3c0 0x7f49d815b300 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:14.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.996+0000 7f49ded16700 1 --2- 172.21.15.92:0/2384915157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49d80b2620 0x7f49d8147c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:14.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.996+0000 7f49ded16700 1 --2- 172.21.15.92:0/2384915157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49d80b29f0 0x7f49d815adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:14.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.996+0000 7f49ded16700 1 -- 172.21.15.92:0/2384915157 >> 172.21.15.92:0/2384915157 conn(0x7f49d8099520 msgr2=0x7f49d80c5350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:14.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.996+0000 7f49ded16700 1 -- 172.21.15.92:0/2384915157 shutdown_connections 2023-10-06T10:44:14.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:13.996+0000 7f49ded16700 1 -- 172.21.15.92:0/2384915157 wait complete. 2023-10-06T10:44:14.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:44:14.009 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2023-10-06T10:44:14.031 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/ 2023-10-06T10:44:14.031 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4 2023-10-06T10:44:14.031 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4/d5 2023-10-06T10:44:14.031 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4/d5/d6 2023-10-06T10:44:14.031 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4/d5/d6/d7 2023-10-06T10:44:14.031 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2023-10-06T10:44:14.032 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:44:14.032 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:44:14.032 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:44:14.032 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:44:14.032 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2023-10-06T10:44:14.033 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:44:14.134 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627778 2023-10-06T10:44:14.323 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:44:14.323 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:44:14.324 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:44:14.324 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:44:14.324 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2023-10-06T10:44:14.324 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:44:14.428 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627779 2023-10-06T10:44:14.617 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:44:14.617 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:44:14.617 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:44:14.617 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:44:14.617 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2023-10-06T10:44:14.618 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:44:14.721 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627780 2023-10-06T10:44:14.911 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:44:14.911 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:44:14.911 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:44:14.911 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:44:14.911 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2023-10-06T10:44:14.911 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:44:15.016 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627781 2023-10-06T10:44:15.204 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:44:15.204 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:44:15.204 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:44:15.204 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:44:15.205 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2023-10-06T10:44:15.205 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:44:15.309 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627782 2023-10-06T10:44:15.498 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:44:15.498 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:44:15.498 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:44:15.498 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:44:15.498 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2023-10-06T10:44:15.498 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:44:15.603 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627783 2023-10-06T10:44:15.792 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2023-10-06T10:44:15.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.870+0000 7fe36372e000 1 Processor -- start 2023-10-06T10:44:15.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.870+0000 7fe36372e000 1 -- start start 2023-10-06T10:44:15.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.871+0000 7fe36372e000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563d47dd2270 0x563d47dcefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:15.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.871+0000 7fe36372e000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563d47ded3b0 0x563d47dcf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:15.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.871+0000 7fe36372e000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d47e60ab0 0x563d47e83a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:15.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.871+0000 7fe36372e000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d47cee980 con 0x563d47e60ab0 2023-10-06T10:44:15.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.871+0000 7fe36372e000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d47cee620 con 0x563d47ded3b0 2023-10-06T10:44:15.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.871+0000 7fe36372e000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d47ce5400 con 0x563d47dd2270 2023-10-06T10:44:15.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.871+0000 7fe35d16a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d47e60ab0 0x563d47e83a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:15.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.871+0000 7fe35d16a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d47e60ab0 0x563d47e83a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48260/0 (socket says 172.21.15.92:48260) 2023-10-06T10:44:15.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.871+0000 7fe35d16a700 1 -- 172.21.15.92:0/2300543594 learned_addr learned my addr 172.21.15.92:0/2300543594 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:15.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.871+0000 7fe35c969700 1 --2- 172.21.15.92:0/2300543594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563d47dd2270 0x563d47dcefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:15.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.871+0000 7fe35c168700 1 --2- 172.21.15.92:0/2300543594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563d47ded3b0 0x563d47dcf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:15.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.871+0000 7fe35d16a700 1 -- 172.21.15.92:0/2300543594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563d47dd2270 msgr2=0x563d47dcefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:15.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.871+0000 7fe35d16a700 1 --2- 172.21.15.92:0/2300543594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563d47dd2270 0x563d47dcefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:15.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.871+0000 7fe35d16a700 1 -- 172.21.15.92:0/2300543594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563d47ded3b0 msgr2=0x563d47dcf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:15.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.871+0000 7fe35d16a700 1 --2- 172.21.15.92:0/2300543594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563d47ded3b0 0x563d47dcf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:15.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.871+0000 7fe35d16a700 1 -- 172.21.15.92:0/2300543594 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe350005cf0 con 0x563d47e60ab0 2023-10-06T10:44:15.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.871+0000 7fe35c168700 1 --2- 172.21.15.92:0/2300543594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563d47ded3b0 0x563d47dcf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:15.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.871+0000 7fe35c969700 1 --2- 172.21.15.92:0/2300543594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563d47dd2270 0x563d47dcefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:15.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.871+0000 7fe35d16a700 1 --2- 172.21.15.92:0/2300543594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d47e60ab0 0x563d47e83a40 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fe3540016e0 tx=0x7fe354006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85220974e6afad47 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:15.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.872+0000 7fe35b166700 1 -- 172.21.15.92:0/2300543594 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe354006320 con 0x563d47e60ab0 2023-10-06T10:44:15.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.872+0000 7fe35b166700 1 -- 172.21.15.92:0/2300543594 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe354004520 con 0x563d47e60ab0 2023-10-06T10:44:15.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.872+0000 7fe35b166700 1 -- 172.21.15.92:0/2300543594 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3540048e0 con 0x563d47e60ab0 2023-10-06T10:44:15.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.872+0000 7fe36372e000 1 -- 172.21.15.92:0/2300543594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d47e60ab0 msgr2=0x563d47e83a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:15.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.872+0000 7fe36372e000 1 --2- 172.21.15.92:0/2300543594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d47e60ab0 0x563d47e83a40 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fe3540016e0 tx=0x7fe354006950 comp rx=0 tx=0).stop 2023-10-06T10:44:15.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.872+0000 7fe36372e000 1 -- 172.21.15.92:0/2300543594 shutdown_connections 2023-10-06T10:44:15.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.872+0000 7fe36372e000 1 --2- 172.21.15.92:0/2300543594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d47e60ab0 0x563d47e83a40 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:15.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.872+0000 7fe36372e000 1 --2- 172.21.15.92:0/2300543594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563d47dd2270 0x563d47dcefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:15.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.872+0000 7fe36372e000 1 --2- 172.21.15.92:0/2300543594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563d47ded3b0 0x563d47dcf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:15.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.872+0000 7fe36372e000 1 -- 172.21.15.92:0/2300543594 >> 172.21.15.92:0/2300543594 conn(0x563d47da4520 msgr2=0x563d47da6900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:15.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.872+0000 7fe36372e000 1 -- 172.21.15.92:0/2300543594 shutdown_connections 2023-10-06T10:44:15.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.872+0000 7fe36372e000 1 -- 172.21.15.92:0/2300543594 wait complete. 2023-10-06T10:44:15.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.873+0000 7fe36372e000 1 Processor -- start 2023-10-06T10:44:15.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.873+0000 7fe36372e000 1 -- start start 2023-10-06T10:44:15.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.873+0000 7fe36372e000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563d47dd2270 0x563d47e79570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:15.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.873+0000 7fe36372e000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d47ded3b0 0x563d47e872c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:15.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.873+0000 7fe36372e000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563d47e60ab0 0x563d47deb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:15.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.873+0000 7fe36372e000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d47e860f0 con 0x563d47ded3b0 2023-10-06T10:44:15.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.873+0000 7fe36372e000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d47e79ab0 con 0x563d47dd2270 2023-10-06T10:44:15.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.873+0000 7fe36372e000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d47dcf2f0 con 0x563d47e60ab0 2023-10-06T10:44:15.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.874+0000 7fe35c168700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d47ded3b0 0x563d47e872c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:15.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.874+0000 7fe35c168700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d47ded3b0 0x563d47e872c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48264/0 (socket says 172.21.15.92:48264) 2023-10-06T10:44:15.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.874+0000 7fe35c168700 1 -- 172.21.15.92:0/2698165384 learned_addr learned my addr 172.21.15.92:0/2698165384 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:15.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.874+0000 7fe35d16a700 1 --2- 172.21.15.92:0/2698165384 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563d47e60ab0 0x563d47deb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:15.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.874+0000 7fe35c969700 1 --2- 172.21.15.92:0/2698165384 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563d47dd2270 0x563d47e79570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:15.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.874+0000 7fe35c168700 1 -- 172.21.15.92:0/2698165384 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563d47e60ab0 msgr2=0x563d47deb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:15.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.874+0000 7fe35c168700 1 --2- 172.21.15.92:0/2698165384 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563d47e60ab0 0x563d47deb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:15.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.874+0000 7fe35c168700 1 -- 172.21.15.92:0/2698165384 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563d47dd2270 msgr2=0x563d47e79570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:15.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.874+0000 7fe35c168700 1 --2- 172.21.15.92:0/2698165384 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563d47dd2270 0x563d47e79570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:15.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.874+0000 7fe35c168700 1 -- 172.21.15.92:0/2698165384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe354006c30 con 0x563d47ded3b0 2023-10-06T10:44:15.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.874+0000 7fe35c168700 1 --2- 172.21.15.92:0/2698165384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d47ded3b0 0x563d47e872c0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fe350009c30 tx=0x7fe35000c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:15.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.874+0000 7fe359963700 1 -- 172.21.15.92:0/2698165384 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe350011930 con 0x563d47ded3b0 2023-10-06T10:44:15.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.874+0000 7fe359963700 1 -- 172.21.15.92:0/2698165384 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe350011a90 con 0x563d47ded3b0 2023-10-06T10:44:15.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.874+0000 7fe36372e000 1 -- 172.21.15.92:0/2698165384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34c005cf0 con 0x563d47ded3b0 2023-10-06T10:44:15.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.874+0000 7fe359963700 1 -- 172.21.15.92:0/2698165384 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe350011e50 con 0x563d47ded3b0 2023-10-06T10:44:15.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.874+0000 7fe36372e000 1 -- 172.21.15.92:0/2698165384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d47cfd3d0 con 0x563d47ded3b0 2023-10-06T10:44:15.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.875+0000 7fe359963700 1 -- 172.21.15.92:0/2698165384 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe350017690 con 0x563d47ded3b0 2023-10-06T10:44:15.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.876+0000 7fe359963700 1 --2- 172.21.15.92:0/2698165384 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe348065890 0x7fe348067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:15.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.876+0000 7fe359963700 1 -- 172.21.15.92:0/2698165384 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe350085de0 con 0x563d47ded3b0 2023-10-06T10:44:15.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.876+0000 7fe35c969700 1 --2- 172.21.15.92:0/2698165384 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe348065890 0x7fe348067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:15.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.876+0000 7fe36372e000 1 --2- 172.21.15.92:0/2698165384 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x563d47e87800 0x563d47e86c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:15.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.876+0000 7fe36372e000 1 -- 172.21.15.92:0/2698165384 --> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] -- osd_op(unknown.0.0:1 14.12 14:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) v8 -- 0x563d47e5f370 con 0x563d47e87800 2023-10-06T10:44:15.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.876+0000 7fe35d16a700 1 --2- 172.21.15.92:0/2698165384 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x563d47e87800 0x563d47e86c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:15.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.876+0000 7fe35c969700 1 --2- 172.21.15.92:0/2698165384 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe348065890 0x7fe348067d40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00a520 tx=0x7fe34c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:15.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.876+0000 7fe35d16a700 1 --2- 172.21.15.92:0/2698165384 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x563d47e87800 0x563d47e86c90 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:15.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.877+0000 7fe35d16a700 1 -- 172.21.15.92:0/2698165384 <== osd.2 v2:172.21.15.92:6824/90615 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v90'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe354004ce0 con 0x563d47e87800 2023-10-06T10:44:15.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.877+0000 7fe36372e000 1 -- 172.21.15.92:0/2698165384 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x563d47e87800 msgr2=0x563d47e86c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:15.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.877+0000 7fe36372e000 1 --2- 172.21.15.92:0/2698165384 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x563d47e87800 0x563d47e86c90 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:15.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.877+0000 7fe36372e000 1 -- 172.21.15.92:0/2698165384 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe348065890 msgr2=0x7fe348067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:15.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.877+0000 7fe36372e000 1 --2- 172.21.15.92:0/2698165384 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe348065890 0x7fe348067d40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00a520 tx=0x7fe34c016000 comp rx=0 tx=0).stop 2023-10-06T10:44:15.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.877+0000 7fe36372e000 1 -- 172.21.15.92:0/2698165384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d47ded3b0 msgr2=0x563d47e872c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:15.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.878+0000 7fe36372e000 1 --2- 172.21.15.92:0/2698165384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d47ded3b0 0x563d47e872c0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fe350009c30 tx=0x7fe35000c040 comp rx=0 tx=0).stop 2023-10-06T10:44:15.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.878+0000 7fe35d16a700 1 -- 172.21.15.92:0/2698165384 reap_dead start 2023-10-06T10:44:15.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.878+0000 7fe36372e000 1 -- 172.21.15.92:0/2698165384 shutdown_connections 2023-10-06T10:44:15.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.878+0000 7fe36372e000 1 -- 172.21.15.92:0/2698165384 >> 172.21.15.92:0/2698165384 conn(0x563d47da4520 msgr2=0x563d47da6900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:15.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.878+0000 7fe36372e000 1 -- 172.21.15.92:0/2698165384 shutdown_connections 2023-10-06T10:44:15.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.878+0000 7fe36372e000 1 -- 172.21.15.92:0/2698165384 wait complete. 2023-10-06T10:44:15.887 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2023-10-06T10:44:15.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.961+0000 7fe6c9eef000 1 Processor -- start 2023-10-06T10:44:15.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.961+0000 7fe6c9eef000 1 -- start start 2023-10-06T10:44:15.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.962+0000 7fe6c9eef000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559ec0c06270 0x559ec0cd9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:15.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.962+0000 7fe6c9eef000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559ec0c4c600 0x559ec0cd5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:15.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.962+0000 7fe6c9eef000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ec0c4c9d0 0x559ec0cdc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:15.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.962+0000 7fe6c9eef000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec0b4e980 con 0x559ec0c4c9d0 2023-10-06T10:44:15.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.962+0000 7fe6c9eef000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec0b4e620 con 0x559ec0c06270 2023-10-06T10:44:15.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.962+0000 7fe6c9eef000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ec0b45400 con 0x559ec0c4c600 2023-10-06T10:44:15.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.962+0000 7fe6c392b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ec0c4c9d0 0x559ec0cdc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:15.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.962+0000 7fe6c392b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ec0c4c9d0 0x559ec0cdc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48284/0 (socket says 172.21.15.92:48284) 2023-10-06T10:44:15.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.962+0000 7fe6c392b700 1 -- 172.21.15.92:0/2431804402 learned_addr learned my addr 172.21.15.92:0/2431804402 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:15.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.962+0000 7fe6c392b700 1 -- 172.21.15.92:0/2431804402 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559ec0c4c600 msgr2=0x559ec0cd5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:15.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.962+0000 7fe6c392b700 1 --2- 172.21.15.92:0/2431804402 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559ec0c4c600 0x559ec0cd5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:15.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.962+0000 7fe6c392b700 1 -- 172.21.15.92:0/2431804402 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559ec0c06270 msgr2=0x559ec0cd9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:15.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.962+0000 7fe6c392b700 1 --2- 172.21.15.92:0/2431804402 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559ec0c06270 0x559ec0cd9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:15.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.962+0000 7fe6c392b700 1 -- 172.21.15.92:0/2431804402 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b4005cf0 con 0x559ec0c4c9d0 2023-10-06T10:44:15.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.962+0000 7fe6c392b700 1 --2- 172.21.15.92:0/2431804402 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ec0c4c9d0 0x559ec0cdc7a0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00a570 tx=0x7fe6bc007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9253d76733b290a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:15.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.963+0000 7fe6c2128700 1 -- 172.21.15.92:0/2431804402 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc0115f0 con 0x559ec0c4c9d0 2023-10-06T10:44:15.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.963+0000 7fe6c2128700 1 -- 172.21.15.92:0/2431804402 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6bc011750 con 0x559ec0c4c9d0 2023-10-06T10:44:15.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.963+0000 7fe6c2128700 1 -- 172.21.15.92:0/2431804402 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc011af0 con 0x559ec0c4c9d0 2023-10-06T10:44:15.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.963+0000 7fe6c9eef000 1 -- 172.21.15.92:0/2431804402 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ec0c4c9d0 msgr2=0x559ec0cdc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:15.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.963+0000 7fe6c9eef000 1 --2- 172.21.15.92:0/2431804402 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ec0c4c9d0 0x559ec0cdc7a0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00a570 tx=0x7fe6bc007040 comp rx=0 tx=0).stop 2023-10-06T10:44:15.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.963+0000 7fe6c9eef000 1 -- 172.21.15.92:0/2431804402 shutdown_connections 2023-10-06T10:44:15.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.963+0000 7fe6c9eef000 1 --2- 172.21.15.92:0/2431804402 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ec0c4c9d0 0x559ec0cdc7a0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:15.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.963+0000 7fe6c9eef000 1 --2- 172.21.15.92:0/2431804402 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559ec0c06270 0x559ec0cd9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:15.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.963+0000 7fe6c9eef000 1 --2- 172.21.15.92:0/2431804402 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559ec0c4c600 0x559ec0cd5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:15.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.963+0000 7fe6c9eef000 1 -- 172.21.15.92:0/2431804402 >> 172.21.15.92:0/2431804402 conn(0x559ec0c04750 msgr2=0x559ec0cd8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:15.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.963+0000 7fe6c9eef000 1 -- 172.21.15.92:0/2431804402 shutdown_connections 2023-10-06T10:44:15.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.963+0000 7fe6c9eef000 1 -- 172.21.15.92:0/2431804402 wait complete. 2023-10-06T10:44:15.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.964+0000 7fe6c9eef000 1 Processor -- start 2023-10-06T10:44:15.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.964+0000 7fe6c9eef000 1 -- start start 2023-10-06T10:44:15.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.964+0000 7fe6c9eef000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559ec0c06270 0x559ec0cdc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:15.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.964+0000 7fe6c9eef000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559ec0c4c600 0x559ec0cd7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:15.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.964+0000 7fe6c9eef000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ec0c4c9d0 0x559ec0cd7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:15.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.964+0000 7fe6c9eef000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec0ce50b0 con 0x559ec0c4c9d0 2023-10-06T10:44:15.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.964+0000 7fe6c9eef000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec0ce5350 con 0x559ec0c06270 2023-10-06T10:44:15.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.964+0000 7fe6c9eef000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ec0ce55f0 con 0x559ec0c4c600 2023-10-06T10:44:15.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.965+0000 7fe6c392b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ec0c4c9d0 0x559ec0cd7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:15.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.965+0000 7fe6c392b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ec0c4c9d0 0x559ec0cd7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48288/0 (socket says 172.21.15.92:48288) 2023-10-06T10:44:15.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.965+0000 7fe6c392b700 1 -- 172.21.15.92:0/332598354 learned_addr learned my addr 172.21.15.92:0/332598354 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:15.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.965+0000 7fe6c312a700 1 --2- 172.21.15.92:0/332598354 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559ec0c06270 0x559ec0cdc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:15.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.965+0000 7fe6ba929700 1 --2- 172.21.15.92:0/332598354 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559ec0c4c600 0x559ec0cd7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:15.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.965+0000 7fe6c392b700 1 -- 172.21.15.92:0/332598354 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559ec0c4c600 msgr2=0x559ec0cd7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:15.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.965+0000 7fe6c392b700 1 --2- 172.21.15.92:0/332598354 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559ec0c4c600 0x559ec0cd7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:15.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.965+0000 7fe6c392b700 1 -- 172.21.15.92:0/332598354 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559ec0c06270 msgr2=0x559ec0cdc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:15.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.965+0000 7fe6c392b700 1 --2- 172.21.15.92:0/332598354 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559ec0c06270 0x559ec0cdc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:15.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.965+0000 7fe6c392b700 1 -- 172.21.15.92:0/332598354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc00d040 con 0x559ec0c4c9d0 2023-10-06T10:44:15.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.965+0000 7fe6c392b700 1 --2- 172.21.15.92:0/332598354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ec0c4c9d0 0x559ec0cd7a60 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00acc0 tx=0x7fe6bc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:15.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.965+0000 7fe6c0925700 1 -- 172.21.15.92:0/332598354 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc0115f0 con 0x559ec0c4c9d0 2023-10-06T10:44:15.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.965+0000 7fe6c0925700 1 -- 172.21.15.92:0/332598354 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6bc021070 con 0x559ec0c4c9d0 2023-10-06T10:44:15.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.965+0000 7fe6c0925700 1 -- 172.21.15.92:0/332598354 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc005c70 con 0x559ec0c4c9d0 2023-10-06T10:44:15.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.965+0000 7fe6c9eef000 1 -- 172.21.15.92:0/332598354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559ec0b5d3d0 con 0x559ec0c4c9d0 2023-10-06T10:44:15.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.965+0000 7fe6c9eef000 1 -- 172.21.15.92:0/332598354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ec0b50ae0 con 0x559ec0c4c9d0 2023-10-06T10:44:15.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.967+0000 7fe6c0925700 1 -- 172.21.15.92:0/332598354 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe6bc023030 con 0x559ec0c4c9d0 2023-10-06T10:44:15.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.967+0000 7fe6c0925700 1 --2- 172.21.15.92:0/332598354 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe6a4065890 0x7fe6a4067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:15.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.967+0000 7fe6c0925700 1 -- 172.21.15.92:0/332598354 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe6bc085770 con 0x559ec0c4c9d0 2023-10-06T10:44:15.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.967+0000 7fe6c312a700 1 --2- 172.21.15.92:0/332598354 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe6a4065890 0x7fe6a4067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:15.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.967+0000 7fe6c9eef000 1 --2- 172.21.15.92:0/332598354 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x559ec0cdcec0 0x559ec0ceb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:15.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.967+0000 7fe6c9eef000 1 -- 172.21.15.92:0/332598354 --> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] -- osd_op(unknown.0.0:1 14.12 14:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) v8 -- 0x559ec0ceb7a0 con 0x559ec0cdcec0 2023-10-06T10:44:15.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.967+0000 7fe6ba929700 1 --2- 172.21.15.92:0/332598354 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x559ec0cdcec0 0x559ec0ceb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:15.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.967+0000 7fe6c312a700 1 --2- 172.21.15.92:0/332598354 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe6a4065890 0x7fe6a4067d40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4008fd0 tx=0x7fe6b4016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:15.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.967+0000 7fe6ba929700 1 --2- 172.21.15.92:0/332598354 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x559ec0cdcec0 0x559ec0ceb260 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:15.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.969+0000 7fe6ba929700 1 -- 172.21.15.92:0/332598354 <== osd.2 v2:172.21.15.92:6824/90615 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v90'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe6ac00e6f0 con 0x559ec0cdcec0 2023-10-06T10:44:15.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.969+0000 7fe6c9eef000 1 -- 172.21.15.92:0/332598354 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x559ec0cdcec0 msgr2=0x559ec0ceb260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:15.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.969+0000 7fe6c9eef000 1 --2- 172.21.15.92:0/332598354 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x559ec0cdcec0 0x559ec0ceb260 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:15.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.969+0000 7fe6c9eef000 1 -- 172.21.15.92:0/332598354 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe6a4065890 msgr2=0x7fe6a4067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:15.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.969+0000 7fe6c9eef000 1 --2- 172.21.15.92:0/332598354 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe6a4065890 0x7fe6a4067d40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4008fd0 tx=0x7fe6b4016000 comp rx=0 tx=0).stop 2023-10-06T10:44:15.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.969+0000 7fe6c9eef000 1 -- 172.21.15.92:0/332598354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ec0c4c9d0 msgr2=0x559ec0cd7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:15.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.969+0000 7fe6c9eef000 1 --2- 172.21.15.92:0/332598354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ec0c4c9d0 0x559ec0cd7a60 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00acc0 tx=0x7fe6bc018000 comp rx=0 tx=0).stop 2023-10-06T10:44:15.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.969+0000 7fe6c392b700 1 -- 172.21.15.92:0/332598354 reap_dead start 2023-10-06T10:44:15.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.969+0000 7fe6c9eef000 1 -- 172.21.15.92:0/332598354 shutdown_connections 2023-10-06T10:44:15.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.969+0000 7fe6c9eef000 1 -- 172.21.15.92:0/332598354 >> 172.21.15.92:0/332598354 conn(0x559ec0c04750 msgr2=0x559ec0c46dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:15.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.969+0000 7fe6c9eef000 1 -- 172.21.15.92:0/332598354 shutdown_connections 2023-10-06T10:44:15.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:15.969+0000 7fe6c9eef000 1 -- 172.21.15.92:0/332598354 wait complete. 2023-10-06T10:44:15.978 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2023-10-06T10:44:16.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.053+0000 7f57f67ab000 1 Processor -- start 2023-10-06T10:44:16.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.053+0000 7f57f67ab000 1 -- start start 2023-10-06T10:44:16.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.054+0000 7f57f67ab000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55784672de70 0x55784679fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.054+0000 7f57f67ab000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578467a4990 0x55784670fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.054+0000 7f57f67ab000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5578467bbe30 0x557846710280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.054+0000 7f57f67ab000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55784662f980 con 0x5578467a4990 2023-10-06T10:44:16.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.054+0000 7f57f67ab000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55784662f620 con 0x5578467bbe30 2023-10-06T10:44:16.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.054+0000 7f57f67ab000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x557846626400 con 0x55784672de70 2023-10-06T10:44:16.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.054+0000 7f57ef1e5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578467a4990 0x55784670fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.054+0000 7f57ef1e5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578467a4990 0x55784670fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48302/0 (socket says 172.21.15.92:48302) 2023-10-06T10:44:16.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.054+0000 7f57ef1e5700 1 -- 172.21.15.92:0/3492973510 learned_addr learned my addr 172.21.15.92:0/3492973510 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.054+0000 7f57f01e7700 1 --2- 172.21.15.92:0/3492973510 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5578467bbe30 0x557846710280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.054+0000 7f57ef9e6700 1 --2- 172.21.15.92:0/3492973510 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55784672de70 0x55784679fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.054+0000 7f57ef1e5700 1 -- 172.21.15.92:0/3492973510 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55784672de70 msgr2=0x55784679fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.054+0000 7f57ef1e5700 1 --2- 172.21.15.92:0/3492973510 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55784672de70 0x55784679fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.055+0000 7f57ef1e5700 1 -- 172.21.15.92:0/3492973510 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5578467bbe30 msgr2=0x557846710280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.055+0000 7f57ef1e5700 1 --2- 172.21.15.92:0/3492973510 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5578467bbe30 0x557846710280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.055+0000 7f57ef1e5700 1 -- 172.21.15.92:0/3492973510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e4005cf0 con 0x5578467a4990 2023-10-06T10:44:16.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.055+0000 7f57f01e7700 1 --2- 172.21.15.92:0/3492973510 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5578467bbe30 0x557846710280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:16.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.055+0000 7f57ef1e5700 1 --2- 172.21.15.92:0/3492973510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578467a4990 0x55784670fd40 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x5578466e3c00 tx=0x7f57e4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4433f6f44e658b16 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.055+0000 7f57ee1e3700 1 -- 172.21.15.92:0/3492973510 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e4000c90 con 0x5578467a4990 2023-10-06T10:44:16.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.055+0000 7f57ee1e3700 1 -- 172.21.15.92:0/3492973510 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e40089d0 con 0x5578467a4990 2023-10-06T10:44:16.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.055+0000 7f57ee1e3700 1 -- 172.21.15.92:0/3492973510 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e4026460 con 0x5578467a4990 2023-10-06T10:44:16.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.055+0000 7f57f67ab000 1 -- 172.21.15.92:0/3492973510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578467a4990 msgr2=0x55784670fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.055+0000 7f57f67ab000 1 --2- 172.21.15.92:0/3492973510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578467a4990 0x55784670fd40 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x5578466e3c00 tx=0x7f57e4016000 comp rx=0 tx=0).stop 2023-10-06T10:44:16.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.055+0000 7f57f67ab000 1 -- 172.21.15.92:0/3492973510 shutdown_connections 2023-10-06T10:44:16.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.055+0000 7f57f67ab000 1 --2- 172.21.15.92:0/3492973510 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5578467bbe30 0x557846710280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.055+0000 7f57f67ab000 1 --2- 172.21.15.92:0/3492973510 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55784672de70 0x55784679fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.055+0000 7f57f67ab000 1 --2- 172.21.15.92:0/3492973510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578467a4990 0x55784670fd40 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.055+0000 7f57f67ab000 1 -- 172.21.15.92:0/3492973510 >> 172.21.15.92:0/3492973510 conn(0x5578466e5520 msgr2=0x55784672abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.055+0000 7f57f67ab000 1 -- 172.21.15.92:0/3492973510 shutdown_connections 2023-10-06T10:44:16.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.055+0000 7f57f67ab000 1 -- 172.21.15.92:0/3492973510 wait complete. 2023-10-06T10:44:16.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.056+0000 7f57f67ab000 1 Processor -- start 2023-10-06T10:44:16.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.056+0000 7f57f67ab000 1 -- start start 2023-10-06T10:44:16.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57f67ab000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55784672de70 0x55784679db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57f67ab000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5578467a4990 0x5578467ba8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57f67ab000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5578467bbe30 0x557846714740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57f67ab000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55784679e210 con 0x55784672de70 2023-10-06T10:44:16.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57f67ab000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578466e3710 con 0x5578467a4990 2023-10-06T10:44:16.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57f67ab000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x557846706900 con 0x5578467bbe30 2023-10-06T10:44:16.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57ef9e6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55784672de70 0x55784679db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57ef9e6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55784672de70 0x55784679db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48314/0 (socket says 172.21.15.92:48314) 2023-10-06T10:44:16.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57ef9e6700 1 -- 172.21.15.92:0/4002462009 learned_addr learned my addr 172.21.15.92:0/4002462009 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57f01e7700 1 --2- 172.21.15.92:0/4002462009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5578467bbe30 0x557846714740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57ef1e5700 1 --2- 172.21.15.92:0/4002462009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5578467a4990 0x5578467ba8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57ef9e6700 1 -- 172.21.15.92:0/4002462009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5578467bbe30 msgr2=0x557846714740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57ef9e6700 1 --2- 172.21.15.92:0/4002462009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5578467bbe30 0x557846714740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57ef9e6700 1 -- 172.21.15.92:0/4002462009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5578467a4990 msgr2=0x5578467ba8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57ef9e6700 1 --2- 172.21.15.92:0/4002462009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5578467a4990 0x5578467ba8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57ef9e6700 1 -- 172.21.15.92:0/4002462009 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578466581c0 con 0x55784672de70 2023-10-06T10:44:16.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57ef1e5700 1 --2- 172.21.15.92:0/4002462009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5578467a4990 0x5578467ba8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:16.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57f01e7700 1 --2- 172.21.15.92:0/4002462009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5578467bbe30 0x557846714740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:16.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57ef9e6700 1 --2- 172.21.15.92:0/4002462009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55784672de70 0x55784679db20 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f57e000ce40 tx=0x7f57e0006930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57ec9e0700 1 -- 172.21.15.92:0/4002462009 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e000ae80 con 0x55784672de70 2023-10-06T10:44:16.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57ec9e0700 1 -- 172.21.15.92:0/4002462009 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e000e440 con 0x55784672de70 2023-10-06T10:44:16.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57ec9e0700 1 -- 172.21.15.92:0/4002462009 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e000e7e0 con 0x55784672de70 2023-10-06T10:44:16.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57f67ab000 1 -- 172.21.15.92:0/4002462009 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55784663e3d0 con 0x55784672de70 2023-10-06T10:44:16.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.057+0000 7f57f67ab000 1 -- 172.21.15.92:0/4002462009 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557846631ae0 con 0x55784672de70 2023-10-06T10:44:16.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.059+0000 7f57ec9e0700 1 -- 172.21.15.92:0/4002462009 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f57e000ebd0 con 0x55784672de70 2023-10-06T10:44:16.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.059+0000 7f57ec9e0700 1 --2- 172.21.15.92:0/4002462009 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f57dc0658c0 0x7f57dc067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.059+0000 7f57ec9e0700 1 -- 172.21.15.92:0/4002462009 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f57e0083b70 con 0x55784672de70 2023-10-06T10:44:16.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.059+0000 7f57ef1e5700 1 --2- 172.21.15.92:0/4002462009 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f57dc0658c0 0x7f57dc067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.059+0000 7f57f67ab000 1 --2- 172.21.15.92:0/4002462009 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x5578467bae00 0x5578467cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.059+0000 7f57f67ab000 1 -- 172.21.15.92:0/4002462009 --> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] -- osd_op(unknown.0.0:1 14.5 14:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) v8 -- 0x55784672b0d0 con 0x5578467bae00 2023-10-06T10:44:16.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.059+0000 7f57f01e7700 1 --2- 172.21.15.92:0/4002462009 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x5578467bae00 0x5578467cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.060+0000 7f57ef1e5700 1 --2- 172.21.15.92:0/4002462009 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f57dc0658c0 0x7f57dc067d70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f57e4000e10 tx=0x7f57e401e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.060+0000 7f57f01e7700 1 --2- 172.21.15.92:0/4002462009 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x5578467bae00 0x5578467cc500 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.061+0000 7f57f01e7700 1 -- 172.21.15.92:0/4002462009 <== osd.0 v2:172.21.15.92:6800/90604 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v90'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f57e800ca00 con 0x5578467bae00 2023-10-06T10:44:16.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.061+0000 7f57f67ab000 1 -- 172.21.15.92:0/4002462009 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x5578467bae00 msgr2=0x5578467cc500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.061+0000 7f57f67ab000 1 --2- 172.21.15.92:0/4002462009 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x5578467bae00 0x5578467cc500 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.061+0000 7f57f67ab000 1 -- 172.21.15.92:0/4002462009 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f57dc0658c0 msgr2=0x7f57dc067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.061+0000 7f57f67ab000 1 --2- 172.21.15.92:0/4002462009 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f57dc0658c0 0x7f57dc067d70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f57e4000e10 tx=0x7f57e401e000 comp rx=0 tx=0).stop 2023-10-06T10:44:16.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.061+0000 7f57f67ab000 1 -- 172.21.15.92:0/4002462009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55784672de70 msgr2=0x55784679db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.061+0000 7f57f67ab000 1 --2- 172.21.15.92:0/4002462009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55784672de70 0x55784679db20 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f57e000ce40 tx=0x7f57e0006930 comp rx=0 tx=0).stop 2023-10-06T10:44:16.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.061+0000 7f57f01e7700 1 -- 172.21.15.92:0/4002462009 reap_dead start 2023-10-06T10:44:16.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.061+0000 7f57f67ab000 1 -- 172.21.15.92:0/4002462009 shutdown_connections 2023-10-06T10:44:16.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.061+0000 7f57f67ab000 1 -- 172.21.15.92:0/4002462009 >> 172.21.15.92:0/4002462009 conn(0x5578466e5520 msgr2=0x5578467bb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.062+0000 7f57f67ab000 1 -- 172.21.15.92:0/4002462009 shutdown_connections 2023-10-06T10:44:16.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.062+0000 7f57f67ab000 1 -- 172.21.15.92:0/4002462009 wait complete. 2023-10-06T10:44:16.069 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2023-10-06T10:44:16.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.144+0000 7f42e0391000 1 Processor -- start 2023-10-06T10:44:16.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.144+0000 7f42e0391000 1 -- start start 2023-10-06T10:44:16.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.145+0000 7f42e0391000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42c80030b0 0x5640abdd9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.145+0000 7f42e0391000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640abdf5c90 0x5640abdf6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.145+0000 7f42e0391000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5640abdf6960 0x5640abd20d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.145+0000 7f42e0391000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640abcc9320 con 0x5640abdf5c90 2023-10-06T10:44:16.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.145+0000 7f42e0391000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640abc6d980 con 0x7f42c80030b0 2023-10-06T10:44:16.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.145+0000 7f42e0391000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640abc6d620 con 0x5640abdf6960 2023-10-06T10:44:16.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.145+0000 7f42d3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640abdf5c90 0x5640abdf6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.145+0000 7f42d3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640abdf5c90 0x5640abdf6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48332/0 (socket says 172.21.15.92:48332) 2023-10-06T10:44:16.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.145+0000 7f42d3fff700 1 -- 172.21.15.92:0/1347740756 learned_addr learned my addr 172.21.15.92:0/1347740756 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.145+0000 7f42d9dcd700 1 --2- 172.21.15.92:0/1347740756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5640abdf6960 0x5640abd20d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.145+0000 7f42d95cc700 1 --2- 172.21.15.92:0/1347740756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42c80030b0 0x5640abdd9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.145+0000 7f42d3fff700 1 -- 172.21.15.92:0/1347740756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5640abdf6960 msgr2=0x5640abd20d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.145+0000 7f42d3fff700 1 --2- 172.21.15.92:0/1347740756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5640abdf6960 0x5640abd20d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.145+0000 7f42d3fff700 1 -- 172.21.15.92:0/1347740756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42c80030b0 msgr2=0x5640abdd9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.145+0000 7f42d3fff700 1 --2- 172.21.15.92:0/1347740756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42c80030b0 0x5640abdd9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.145+0000 7f42d3fff700 1 -- 172.21.15.92:0/1347740756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640abc6fae0 con 0x5640abdf5c90 2023-10-06T10:44:16.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.145+0000 7f42d3fff700 1 --2- 172.21.15.92:0/1347740756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640abdf5c90 0x5640abdf6420 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f42c400b9f0 tx=0x7f42c4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4279d1cd8dd0df6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.145+0000 7f42d37fe700 1 -- 172.21.15.92:0/1347740756 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c4000bc0 con 0x5640abdf5c90 2023-10-06T10:44:16.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.145+0000 7f42d37fe700 1 -- 172.21.15.92:0/1347740756 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c400bcd0 con 0x5640abdf5c90 2023-10-06T10:44:16.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.145+0000 7f42d37fe700 1 -- 172.21.15.92:0/1347740756 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c400be80 con 0x5640abdf5c90 2023-10-06T10:44:16.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.146+0000 7f42e0391000 1 -- 172.21.15.92:0/1347740756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640abdf5c90 msgr2=0x5640abdf6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.146+0000 7f42e0391000 1 --2- 172.21.15.92:0/1347740756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640abdf5c90 0x5640abdf6420 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f42c400b9f0 tx=0x7f42c4016000 comp rx=0 tx=0).stop 2023-10-06T10:44:16.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.146+0000 7f42e0391000 1 -- 172.21.15.92:0/1347740756 shutdown_connections 2023-10-06T10:44:16.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.146+0000 7f42e0391000 1 --2- 172.21.15.92:0/1347740756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5640abdf6960 0x5640abd20d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.146+0000 7f42e0391000 1 --2- 172.21.15.92:0/1347740756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42c80030b0 0x5640abdd9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.146+0000 7f42e0391000 1 --2- 172.21.15.92:0/1347740756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640abdf5c90 0x5640abdf6420 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.146+0000 7f42e0391000 1 -- 172.21.15.92:0/1347740756 >> 172.21.15.92:0/1347740756 conn(0x5640abd23520 msgr2=0x5640abd4dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.146+0000 7f42e0391000 1 -- 172.21.15.92:0/1347740756 shutdown_connections 2023-10-06T10:44:16.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.146+0000 7f42e0391000 1 -- 172.21.15.92:0/1347740756 wait complete. 2023-10-06T10:44:16.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.146+0000 7f42e0391000 1 Processor -- start 2023-10-06T10:44:16.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.147+0000 7f42e0391000 1 -- start start 2023-10-06T10:44:16.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.147+0000 7f42e0391000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5640abdf5c90 0x5640abdd8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.147+0000 7f42e0391000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640abdf6960 0x5640abdff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.147+0000 7f42e0391000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42c80030b0 0x5640abdff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.147+0000 7f42e0391000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640abc64400 con 0x5640abdf6960 2023-10-06T10:44:16.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.147+0000 7f42e0391000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640abe02480 con 0x5640abdf5c90 2023-10-06T10:44:16.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.147+0000 7f42e0391000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640abdd9270 con 0x7f42c80030b0 2023-10-06T10:44:16.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.147+0000 7f42d3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640abdf6960 0x5640abdff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.147+0000 7f42d3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640abdf6960 0x5640abdff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48344/0 (socket says 172.21.15.92:48344) 2023-10-06T10:44:16.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.147+0000 7f42d3fff700 1 -- 172.21.15.92:0/847678480 learned_addr learned my addr 172.21.15.92:0/847678480 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.148+0000 7f42d9dcd700 1 --2- 172.21.15.92:0/847678480 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42c80030b0 0x5640abdff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.148+0000 7f42d95cc700 1 --2- 172.21.15.92:0/847678480 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5640abdf5c90 0x5640abdd8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.148+0000 7f42d3fff700 1 -- 172.21.15.92:0/847678480 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42c80030b0 msgr2=0x5640abdff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.148+0000 7f42d3fff700 1 --2- 172.21.15.92:0/847678480 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42c80030b0 0x5640abdff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.148+0000 7f42d3fff700 1 -- 172.21.15.92:0/847678480 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5640abdf5c90 msgr2=0x5640abdd8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.148+0000 7f42d3fff700 1 --2- 172.21.15.92:0/847678480 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5640abdf5c90 0x5640abdd8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.148+0000 7f42d3fff700 1 -- 172.21.15.92:0/847678480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c8001e10 con 0x5640abdf6960 2023-10-06T10:44:16.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.148+0000 7f42d3fff700 1 --2- 172.21.15.92:0/847678480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640abdf6960 0x5640abdff2a0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f42c4001ab0 tx=0x7f42c401f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.148+0000 7f42d1ffb700 1 -- 172.21.15.92:0/847678480 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c402d070 con 0x5640abdf6960 2023-10-06T10:44:16.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.148+0000 7f42d1ffb700 1 -- 172.21.15.92:0/847678480 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c401a6d0 con 0x5640abdf6960 2023-10-06T10:44:16.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.148+0000 7f42d1ffb700 1 -- 172.21.15.92:0/847678480 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c401aa70 con 0x5640abdf6960 2023-10-06T10:44:16.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.148+0000 7f42e0391000 1 -- 172.21.15.92:0/847678480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5640abc6f540 con 0x5640abdf6960 2023-10-06T10:44:16.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.148+0000 7f42e0391000 1 -- 172.21.15.92:0/847678480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640abe02910 con 0x5640abdf6960 2023-10-06T10:44:16.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.149+0000 7f42d1ffb700 1 -- 172.21.15.92:0/847678480 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f42c4027470 con 0x5640abdf6960 2023-10-06T10:44:16.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.149+0000 7f42d1ffb700 1 --2- 172.21.15.92:0/847678480 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f42bc065890 0x7f42bc067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.150+0000 7f42d1ffb700 1 -- 172.21.15.92:0/847678480 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f42c4022020 con 0x5640abdf6960 2023-10-06T10:44:16.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.150+0000 7f42d95cc700 1 --2- 172.21.15.92:0/847678480 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f42bc065890 0x7f42bc067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.150+0000 7f42e0391000 1 --2- 172.21.15.92:0/847678480 >> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] conn(0x5640abddeb80 0x5640abe14040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.150+0000 7f42e0391000 1 -- 172.21.15.92:0/847678480 --> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] -- osd_op(unknown.0.0:1 14.9 14:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) v8 -- 0x5640abdf4510 con 0x5640abddeb80 2023-10-06T10:44:16.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.150+0000 7f42d95cc700 1 --2- 172.21.15.92:0/847678480 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f42bc065890 0x7f42bc067d40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f42cc005fd0 tx=0x7f42cc005b30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.150+0000 7f42d9dcd700 1 --2- 172.21.15.92:0/847678480 >> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] conn(0x5640abddeb80 0x5640abe14040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.150+0000 7f42d9dcd700 1 --2- 172.21.15.92:0/847678480 >> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] conn(0x5640abddeb80 0x5640abe14040 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.152+0000 7f42d9dcd700 1 -- 172.21.15.92:0/847678480 <== osd.5 v2:172.21.15.124:6816/90907 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v90'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f42d40037a0 con 0x5640abddeb80 2023-10-06T10:44:16.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.152+0000 7f42e0391000 1 -- 172.21.15.92:0/847678480 >> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] conn(0x5640abddeb80 msgr2=0x5640abe14040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.152+0000 7f42e0391000 1 --2- 172.21.15.92:0/847678480 >> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] conn(0x5640abddeb80 0x5640abe14040 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.152+0000 7f42e0391000 1 -- 172.21.15.92:0/847678480 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f42bc065890 msgr2=0x7f42bc067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.152+0000 7f42e0391000 1 --2- 172.21.15.92:0/847678480 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f42bc065890 0x7f42bc067d40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f42cc005fd0 tx=0x7f42cc005b30 comp rx=0 tx=0).stop 2023-10-06T10:44:16.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.152+0000 7f42e0391000 1 -- 172.21.15.92:0/847678480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640abdf6960 msgr2=0x5640abdff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.152+0000 7f42e0391000 1 --2- 172.21.15.92:0/847678480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640abdf6960 0x5640abdff2a0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f42c4001ab0 tx=0x7f42c401f070 comp rx=0 tx=0).stop 2023-10-06T10:44:16.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.152+0000 7f42d9dcd700 1 -- 172.21.15.92:0/847678480 reap_dead start 2023-10-06T10:44:16.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.152+0000 7f42e0391000 1 -- 172.21.15.92:0/847678480 shutdown_connections 2023-10-06T10:44:16.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.152+0000 7f42e0391000 1 -- 172.21.15.92:0/847678480 >> 172.21.15.92:0/847678480 conn(0x5640abd23520 msgr2=0x5640abdd9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.152+0000 7f42e0391000 1 -- 172.21.15.92:0/847678480 shutdown_connections 2023-10-06T10:44:16.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.152+0000 7f42e0391000 1 -- 172.21.15.92:0/847678480 wait complete. 2023-10-06T10:44:16.159 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2023-10-06T10:44:16.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.233+0000 7f84daa82000 1 Processor -- start 2023-10-06T10:44:16.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.233+0000 7f84daa82000 1 -- start start 2023-10-06T10:44:16.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.234+0000 7f84daa82000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55954202c270 0x5595420f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.234+0000 7f84daa82000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5595420fcc80 0x5595420f8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.234+0000 7f84daa82000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5595420fd050 0x5595421009a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.234+0000 7f84daa82000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559541fd0320 con 0x55954202c270 2023-10-06T10:44:16.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.234+0000 7f84daa82000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559541f74980 con 0x5595420fd050 2023-10-06T10:44:16.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.234+0000 7f84daa82000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559541f74620 con 0x5595420fcc80 2023-10-06T10:44:16.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.234+0000 7f84d3cbd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55954202c270 0x5595420f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.234+0000 7f84d3cbd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55954202c270 0x5595420f6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48372/0 (socket says 172.21.15.92:48372) 2023-10-06T10:44:16.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.234+0000 7f84d3cbd700 1 -- 172.21.15.92:0/3055839971 learned_addr learned my addr 172.21.15.92:0/3055839971 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.234+0000 7f84d44be700 1 --2- 172.21.15.92:0/3055839971 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5595420fd050 0x5595421009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.234+0000 7f84d34bc700 1 --2- 172.21.15.92:0/3055839971 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5595420fcc80 0x5595420f8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.234+0000 7f84d3cbd700 1 -- 172.21.15.92:0/3055839971 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5595420fcc80 msgr2=0x5595420f8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.234+0000 7f84d3cbd700 1 --2- 172.21.15.92:0/3055839971 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5595420fcc80 0x5595420f8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.234+0000 7f84d3cbd700 1 -- 172.21.15.92:0/3055839971 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5595420fd050 msgr2=0x5595421009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.234+0000 7f84d3cbd700 1 --2- 172.21.15.92:0/3055839971 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5595420fd050 0x5595421009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.234+0000 7f84d3cbd700 1 -- 172.21.15.92:0/3055839971 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c8005cf0 con 0x55954202c270 2023-10-06T10:44:16.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.234+0000 7f84d44be700 1 --2- 172.21.15.92:0/3055839971 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5595420fd050 0x5595421009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:16.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.234+0000 7f84d3cbd700 1 --2- 172.21.15.92:0/3055839971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55954202c270 0x5595420f6fd0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f84c400b2f0 tx=0x7f84c4001790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6648f7a2e80f7c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.234+0000 7f84d24ba700 1 -- 172.21.15.92:0/3055839971 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c400be90 con 0x55954202c270 2023-10-06T10:44:16.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.234+0000 7f84d24ba700 1 -- 172.21.15.92:0/3055839971 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c400e470 con 0x55954202c270 2023-10-06T10:44:16.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.234+0000 7f84d24ba700 1 -- 172.21.15.92:0/3055839971 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c400e830 con 0x55954202c270 2023-10-06T10:44:16.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.235+0000 7f84daa82000 1 -- 172.21.15.92:0/3055839971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55954202c270 msgr2=0x5595420f6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.235+0000 7f84daa82000 1 --2- 172.21.15.92:0/3055839971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55954202c270 0x5595420f6fd0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f84c400b2f0 tx=0x7f84c4001790 comp rx=0 tx=0).stop 2023-10-06T10:44:16.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.235+0000 7f84daa82000 1 -- 172.21.15.92:0/3055839971 shutdown_connections 2023-10-06T10:44:16.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.235+0000 7f84daa82000 1 --2- 172.21.15.92:0/3055839971 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5595420fd050 0x5595421009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.235+0000 7f84daa82000 1 --2- 172.21.15.92:0/3055839971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55954202c270 0x5595420f6fd0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.235+0000 7f84daa82000 1 --2- 172.21.15.92:0/3055839971 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5595420fcc80 0x5595420f8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.235+0000 7f84daa82000 1 -- 172.21.15.92:0/3055839971 >> 172.21.15.92:0/3055839971 conn(0x55954202a750 msgr2=0x5595420587a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.235+0000 7f84daa82000 1 -- 172.21.15.92:0/3055839971 shutdown_connections 2023-10-06T10:44:16.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.235+0000 7f84daa82000 1 -- 172.21.15.92:0/3055839971 wait complete. 2023-10-06T10:44:16.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.235+0000 7f84daa82000 1 Processor -- start 2023-10-06T10:44:16.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.236+0000 7f84daa82000 1 -- start start 2023-10-06T10:44:16.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.236+0000 7f84daa82000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55954202c270 0x559542102960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.236+0000 7f84daa82000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5595420fcc80 0x5595420f0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.236+0000 7f84daa82000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5595420fd050 0x5595421009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.236+0000 7f84daa82000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559541f6b400 con 0x55954202c270 2023-10-06T10:44:16.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.236+0000 7f84daa82000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559542104970 con 0x5595420fcc80 2023-10-06T10:44:16.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.236+0000 7f84daa82000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559542103150 con 0x5595420fd050 2023-10-06T10:44:16.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.236+0000 7f84d3cbd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55954202c270 0x559542102960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.236+0000 7f84d3cbd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55954202c270 0x559542102960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48386/0 (socket says 172.21.15.92:48386) 2023-10-06T10:44:16.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.236+0000 7f84d3cbd700 1 -- 172.21.15.92:0/2125916020 learned_addr learned my addr 172.21.15.92:0/2125916020 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.236+0000 7f84d34bc700 1 --2- 172.21.15.92:0/2125916020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5595420fcc80 0x5595420f0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.236+0000 7f84d44be700 1 --2- 172.21.15.92:0/2125916020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5595420fd050 0x5595421009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.237+0000 7f84d3cbd700 1 -- 172.21.15.92:0/2125916020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5595420fd050 msgr2=0x5595421009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.237+0000 7f84d3cbd700 1 --2- 172.21.15.92:0/2125916020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5595420fd050 0x5595421009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.237+0000 7f84d3cbd700 1 -- 172.21.15.92:0/2125916020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5595420fcc80 msgr2=0x5595420f0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.237+0000 7f84d3cbd700 1 --2- 172.21.15.92:0/2125916020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5595420fcc80 0x5595420f0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.237+0000 7f84d3cbd700 1 -- 172.21.15.92:0/2125916020 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c4001a70 con 0x55954202c270 2023-10-06T10:44:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.237+0000 7f84d3cbd700 1 --2- 172.21.15.92:0/2125916020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55954202c270 0x559542102960 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f84c400ba40 tx=0x7f84c4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.237+0000 7f84d0cb7700 1 -- 172.21.15.92:0/2125916020 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c401f070 con 0x55954202c270 2023-10-06T10:44:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.237+0000 7f84d0cb7700 1 -- 172.21.15.92:0/2125916020 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c4001430 con 0x55954202c270 2023-10-06T10:44:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.237+0000 7f84d0cb7700 1 -- 172.21.15.92:0/2125916020 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4018870 con 0x55954202c270 2023-10-06T10:44:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.237+0000 7f84daa82000 1 -- 172.21.15.92:0/2125916020 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559541f76ae0 con 0x55954202c270 2023-10-06T10:44:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.237+0000 7f84daa82000 1 -- 172.21.15.92:0/2125916020 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559541f76540 con 0x55954202c270 2023-10-06T10:44:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.238+0000 7f84d0cb7700 1 -- 172.21.15.92:0/2125916020 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f84c4018be0 con 0x55954202c270 2023-10-06T10:44:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.238+0000 7f84d0cb7700 1 --2- 172.21.15.92:0/2125916020 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f84c0065890 0x7f84c0067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.239+0000 7f84d0cb7700 1 -- 172.21.15.92:0/2125916020 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f84c4085140 con 0x55954202c270 2023-10-06T10:44:16.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.239+0000 7f84d34bc700 1 --2- 172.21.15.92:0/2125916020 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f84c0065890 0x7f84c0067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.239+0000 7f84daa82000 1 --2- 172.21.15.92:0/2125916020 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x5595420f81a0 0x559542053d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.239+0000 7f84daa82000 1 -- 172.21.15.92:0/2125916020 --> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] -- osd_op(unknown.0.0:1 14.5 14:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) v8 -- 0x5595420542c0 con 0x5595420f81a0 2023-10-06T10:44:16.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.239+0000 7f84d44be700 1 --2- 172.21.15.92:0/2125916020 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x5595420f81a0 0x559542053d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.239+0000 7f84d34bc700 1 --2- 172.21.15.92:0/2125916020 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f84c0065890 0x7f84c0067d40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f84c800a5d0 tx=0x7f84c8016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.239+0000 7f84d44be700 1 --2- 172.21.15.92:0/2125916020 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x5595420f81a0 0x559542053d80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.240+0000 7f84d44be700 1 -- 172.21.15.92:0/2125916020 <== osd.0 v2:172.21.15.92:6800/90604 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v90'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f84cc00e7b0 con 0x5595420f81a0 2023-10-06T10:44:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.240+0000 7f84daa82000 1 -- 172.21.15.92:0/2125916020 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x5595420f81a0 msgr2=0x559542053d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.240+0000 7f84daa82000 1 --2- 172.21.15.92:0/2125916020 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x5595420f81a0 0x559542053d80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.240+0000 7f84daa82000 1 -- 172.21.15.92:0/2125916020 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f84c0065890 msgr2=0x7f84c0067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.240+0000 7f84daa82000 1 --2- 172.21.15.92:0/2125916020 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f84c0065890 0x7f84c0067d40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f84c800a5d0 tx=0x7f84c8016000 comp rx=0 tx=0).stop 2023-10-06T10:44:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.240+0000 7f84daa82000 1 -- 172.21.15.92:0/2125916020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55954202c270 msgr2=0x559542102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.240+0000 7f84daa82000 1 --2- 172.21.15.92:0/2125916020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55954202c270 0x559542102960 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f84c400ba40 tx=0x7f84c4016000 comp rx=0 tx=0).stop 2023-10-06T10:44:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.241+0000 7f84d44be700 1 -- 172.21.15.92:0/2125916020 reap_dead start 2023-10-06T10:44:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.241+0000 7f84daa82000 1 -- 172.21.15.92:0/2125916020 shutdown_connections 2023-10-06T10:44:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.241+0000 7f84daa82000 1 -- 172.21.15.92:0/2125916020 >> 172.21.15.92:0/2125916020 conn(0x55954202a750 msgr2=0x559542017830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.241+0000 7f84daa82000 1 -- 172.21.15.92:0/2125916020 shutdown_connections 2023-10-06T10:44:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.241+0000 7f84daa82000 1 -- 172.21.15.92:0/2125916020 wait complete. 2023-10-06T10:44:16.250 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2023-10-06T10:44:16.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.325+0000 7f454443f000 1 Processor -- start 2023-10-06T10:44:16.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.325+0000 7f454443f000 1 -- start start 2023-10-06T10:44:16.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.326+0000 7f454443f000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55910156f270 0x55910163e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.326+0000 7f454443f000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55910163fc80 0x559101639d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.326+0000 7f454443f000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559101640050 0x559101598680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.326+0000 7f454443f000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591014b7980 con 0x55910163fc80 2023-10-06T10:44:16.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.326+0000 7f454443f000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591014b7620 con 0x55910156f270 2023-10-06T10:44:16.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.326+0000 7f454443f000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591014ae400 con 0x559101640050 2023-10-06T10:44:16.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.326+0000 7f453ce79700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55910163fc80 0x559101639d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.326+0000 7f453ce79700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55910163fc80 0x559101639d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48412/0 (socket says 172.21.15.92:48412) 2023-10-06T10:44:16.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.326+0000 7f453ce79700 1 -- 172.21.15.92:0/2037864191 learned_addr learned my addr 172.21.15.92:0/2037864191 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.326+0000 7f453d67a700 1 --2- 172.21.15.92:0/2037864191 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55910156f270 0x55910163e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.326+0000 7f453de7b700 1 --2- 172.21.15.92:0/2037864191 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559101640050 0x559101598680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.326+0000 7f453ce79700 1 -- 172.21.15.92:0/2037864191 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559101640050 msgr2=0x559101598680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.326+0000 7f453ce79700 1 --2- 172.21.15.92:0/2037864191 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559101640050 0x559101598680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.326+0000 7f453ce79700 1 -- 172.21.15.92:0/2037864191 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55910156f270 msgr2=0x55910163e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.326+0000 7f453ce79700 1 --2- 172.21.15.92:0/2037864191 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55910156f270 0x55910163e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.326+0000 7f453ce79700 1 -- 172.21.15.92:0/2037864191 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453800a0e0 con 0x55910163fc80 2023-10-06T10:44:16.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.326+0000 7f453ce79700 1 --2- 172.21.15.92:0/2037864191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55910163fc80 0x559101639d60 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f453400a780 tx=0x7f4534005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63b884d23f89bf7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.327+0000 7f452f7fe700 1 -- 172.21.15.92:0/2037864191 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453400f070 con 0x55910163fc80 2023-10-06T10:44:16.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.327+0000 7f452f7fe700 1 -- 172.21.15.92:0/2037864191 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4534008660 con 0x55910163fc80 2023-10-06T10:44:16.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.327+0000 7f452f7fe700 1 -- 172.21.15.92:0/2037864191 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534008a00 con 0x55910163fc80 2023-10-06T10:44:16.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.327+0000 7f454443f000 1 -- 172.21.15.92:0/2037864191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55910163fc80 msgr2=0x559101639d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.327+0000 7f454443f000 1 --2- 172.21.15.92:0/2037864191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55910163fc80 0x559101639d60 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f453400a780 tx=0x7f4534005c80 comp rx=0 tx=0).stop 2023-10-06T10:44:16.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.327+0000 7f454443f000 1 -- 172.21.15.92:0/2037864191 shutdown_connections 2023-10-06T10:44:16.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.327+0000 7f454443f000 1 --2- 172.21.15.92:0/2037864191 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559101640050 0x559101598680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.327+0000 7f454443f000 1 --2- 172.21.15.92:0/2037864191 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55910156f270 0x55910163e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.327+0000 7f454443f000 1 --2- 172.21.15.92:0/2037864191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55910163fc80 0x559101639d60 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.327+0000 7f454443f000 1 -- 172.21.15.92:0/2037864191 >> 172.21.15.92:0/2037864191 conn(0x55910156d750 msgr2=0x55910163efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.327+0000 7f454443f000 1 -- 172.21.15.92:0/2037864191 shutdown_connections 2023-10-06T10:44:16.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.327+0000 7f454443f000 1 -- 172.21.15.92:0/2037864191 wait complete. 2023-10-06T10:44:16.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.328+0000 7f454443f000 1 Processor -- start 2023-10-06T10:44:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.328+0000 7f454443f000 1 -- start start 2023-10-06T10:44:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.328+0000 7f454443f000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55910156f270 0x559101639d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.328+0000 7f454443f000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55910163fc80 0x559101598680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.328+0000 7f454443f000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559101640050 0x559101598bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.328+0000 7f454443f000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55910163a2a0 con 0x55910163fc80 2023-10-06T10:44:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.328+0000 7f454443f000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559101637250 con 0x55910156f270 2023-10-06T10:44:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.328+0000 7f454443f000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559101642f60 con 0x559101640050 2023-10-06T10:44:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.329+0000 7f453ce79700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55910163fc80 0x559101598680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.329+0000 7f453ce79700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55910163fc80 0x559101598680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48424/0 (socket says 172.21.15.92:48424) 2023-10-06T10:44:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.329+0000 7f453d67a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55910156f270 0x559101639d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.329+0000 7f453ce79700 1 -- 172.21.15.92:0/732374565 learned_addr learned my addr 172.21.15.92:0/732374565 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.329+0000 7f453de7b700 1 --2- 172.21.15.92:0/732374565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559101640050 0x559101598bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.329+0000 7f453ce79700 1 -- 172.21.15.92:0/732374565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559101640050 msgr2=0x559101598bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.329+0000 7f453ce79700 1 --2- 172.21.15.92:0/732374565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559101640050 0x559101598bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.329+0000 7f453ce79700 1 -- 172.21.15.92:0/732374565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55910156f270 msgr2=0x559101639d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.329+0000 7f453ce79700 1 --2- 172.21.15.92:0/732374565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55910156f270 0x559101639d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.329+0000 7f453ce79700 1 -- 172.21.15.92:0/732374565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453400a440 con 0x55910163fc80 2023-10-06T10:44:16.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.329+0000 7f453ce79700 1 --2- 172.21.15.92:0/732374565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55910163fc80 0x559101598680 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f453400aed0 tx=0x7f4534018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.329+0000 7f452dffb700 1 -- 172.21.15.92:0/732374565 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534020630 con 0x55910163fc80 2023-10-06T10:44:16.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.329+0000 7f452dffb700 1 -- 172.21.15.92:0/732374565 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4534020790 con 0x55910163fc80 2023-10-06T10:44:16.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.329+0000 7f454443f000 1 -- 172.21.15.92:0/732374565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5591014c63d0 con 0x55910163fc80 2023-10-06T10:44:16.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.329+0000 7f452dffb700 1 -- 172.21.15.92:0/732374565 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534020b50 con 0x55910163fc80 2023-10-06T10:44:16.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.329+0000 7f454443f000 1 -- 172.21.15.92:0/732374565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591014b9ae0 con 0x55910163fc80 2023-10-06T10:44:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.331+0000 7f452dffb700 1 -- 172.21.15.92:0/732374565 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4534013070 con 0x55910163fc80 2023-10-06T10:44:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.331+0000 7f452dffb700 1 --2- 172.21.15.92:0/732374565 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f45200658c0 0x7f4520067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.331+0000 7f452dffb700 1 -- 172.21.15.92:0/732374565 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f4534084ac0 con 0x55910163fc80 2023-10-06T10:44:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.331+0000 7f453d67a700 1 --2- 172.21.15.92:0/732374565 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f45200658c0 0x7f4520067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.331+0000 7f454443f000 1 --2- 172.21.15.92:0/732374565 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x5591016337d0 0x5591016421c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.331+0000 7f454443f000 1 -- 172.21.15.92:0/732374565 --> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] -- osd_op(unknown.0.0:1 14.1d 14:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) v8 -- 0x559101642700 con 0x5591016337d0 2023-10-06T10:44:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.331+0000 7f453de7b700 1 --2- 172.21.15.92:0/732374565 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x5591016337d0 0x5591016421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.331+0000 7f453d67a700 1 --2- 172.21.15.92:0/732374565 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f45200658c0 0x7f4520067d70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4530000c00 tx=0x7f4530001900 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.332+0000 7f453de7b700 1 --2- 172.21.15.92:0/732374565 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x5591016337d0 0x5591016421c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.333+0000 7f453de7b700 1 -- 172.21.15.92:0/732374565 <== osd.0 v2:172.21.15.92:6800/90604 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v90'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4538016020 con 0x5591016337d0 2023-10-06T10:44:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.333+0000 7f454443f000 1 -- 172.21.15.92:0/732374565 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x5591016337d0 msgr2=0x5591016421c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.333+0000 7f454443f000 1 --2- 172.21.15.92:0/732374565 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x5591016337d0 0x5591016421c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.333+0000 7f454443f000 1 -- 172.21.15.92:0/732374565 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f45200658c0 msgr2=0x7f4520067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.333+0000 7f454443f000 1 --2- 172.21.15.92:0/732374565 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f45200658c0 0x7f4520067d70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4530000c00 tx=0x7f4530001900 comp rx=0 tx=0).stop 2023-10-06T10:44:16.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.333+0000 7f454443f000 1 -- 172.21.15.92:0/732374565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55910163fc80 msgr2=0x559101598680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.333+0000 7f454443f000 1 --2- 172.21.15.92:0/732374565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55910163fc80 0x559101598680 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f453400aed0 tx=0x7f4534018000 comp rx=0 tx=0).stop 2023-10-06T10:44:16.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.333+0000 7f453de7b700 1 -- 172.21.15.92:0/732374565 reap_dead start 2023-10-06T10:44:16.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.333+0000 7f454443f000 1 -- 172.21.15.92:0/732374565 shutdown_connections 2023-10-06T10:44:16.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.333+0000 7f454443f000 1 -- 172.21.15.92:0/732374565 >> 172.21.15.92:0/732374565 conn(0x55910156d750 msgr2=0x55910163b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.333+0000 7f454443f000 1 -- 172.21.15.92:0/732374565 shutdown_connections 2023-10-06T10:44:16.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.333+0000 7f454443f000 1 -- 172.21.15.92:0/732374565 wait complete. 2023-10-06T10:44:16.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub start /d1/d2/d3 recursive,force 2023-10-06T10:44:16.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.495+0000 7f2bdccde700 1 Processor -- start 2023-10-06T10:44:16.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.495+0000 7f2bdccde700 1 -- start start 2023-10-06T10:44:16.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.496+0000 7f2bdccde700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2bd8153990 0x7f2bd815a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.496+0000 7f2bdccde700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2bd80d09c0 0x7f2bd80ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.496+0000 7f2bdccde700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd80cdbf0 con 0x7f2bd80cca70 2023-10-06T10:44:16.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.496+0000 7f2bdccde700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd80cb4c0 con 0x7f2bd80d09c0 2023-10-06T10:44:16.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.496+0000 7f2bdccde700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd815a8b0 con 0x7f2bd8153990 2023-10-06T10:44:16.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.496+0000 7f2bd7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2bd80d09c0 0x7f2bd80ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.496+0000 7f2bd7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2bd80d09c0 0x7f2bd80ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48436/0 (socket says 172.21.15.92:48436) 2023-10-06T10:44:16.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.496+0000 7f2bd7fff700 1 -- 172.21.15.92:0/1400238767 learned_addr learned my addr 172.21.15.92:0/1400238767 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.496+0000 7f2bd77fe700 1 --2- 172.21.15.92:0/1400238767 >> v2:172.21.15.124:3300/0 conn(0x7f2bd8153990 0x7f2bd815a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.496+0000 7f2bd7fff700 1 -- 172.21.15.92:0/1400238767 >> v2:172.21.15.124:3300/0 conn(0x7f2bd8153990 msgr2=0x7f2bd815a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.496+0000 7f2bd7fff700 1 --2- 172.21.15.92:0/1400238767 >> v2:172.21.15.124:3300/0 conn(0x7f2bd8153990 0x7f2bd815a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.496+0000 7f2bd7fff700 1 -- 172.21.15.92:0/1400238767 >> v1:172.21.15.124:6789/0 conn(0x7f2bd80cca70 legacy=0x7f2bd80d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.496+0000 7f2bd7fff700 1 -- 172.21.15.92:0/1400238767 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd80ccfe0 con 0x7f2bd80d09c0 2023-10-06T10:44:16.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.496+0000 7f2bd7fff700 1 --2- 172.21.15.92:0/1400238767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bd80d09c0 0x7f2bd80ce860 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80d0280 tx=0x7f2bd0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb4564e084d3a78 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.496+0000 7f2bd6ffd700 1 -- 172.21.15.92:0/1400238767 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd000e070 con 0x7f2bd80d09c0 2023-10-06T10:44:16.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.496+0000 7f2bd6ffd700 1 -- 172.21.15.92:0/1400238767 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd000ba20 con 0x7f2bd80d09c0 2023-10-06T10:44:16.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.497+0000 7f2bd6ffd700 1 -- 172.21.15.92:0/1400238767 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd00092e0 con 0x7f2bd80d09c0 2023-10-06T10:44:16.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.497+0000 7f2bdccde700 1 -- 172.21.15.92:0/1400238767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bd80d09c0 msgr2=0x7f2bd80ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.497+0000 7f2bdccde700 1 --2- 172.21.15.92:0/1400238767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bd80d09c0 0x7f2bd80ce860 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80d0280 tx=0x7f2bd0009840 comp rx=0 tx=0).stop 2023-10-06T10:44:16.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.497+0000 7f2bdccde700 1 -- 172.21.15.92:0/1400238767 shutdown_connections 2023-10-06T10:44:16.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.497+0000 7f2bdccde700 1 --2- 172.21.15.92:0/1400238767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bd80d09c0 0x7f2bd80ce860 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.497+0000 7f2bdccde700 1 --2- 172.21.15.92:0/1400238767 >> v2:172.21.15.124:3300/0 conn(0x7f2bd8153990 0x7f2bd815a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.497+0000 7f2bdccde700 1 -- 172.21.15.92:0/1400238767 >> 172.21.15.92:0/1400238767 conn(0x7f2bd80680c0 msgr2=0x7f2bd8068490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.497+0000 7f2bdccde700 1 -- 172.21.15.92:0/1400238767 shutdown_connections 2023-10-06T10:44:16.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.497+0000 7f2bdccde700 1 -- 172.21.15.92:0/1400238767 wait complete. 2023-10-06T10:44:16.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.498+0000 7f2bdccde700 1 Processor -- start 2023-10-06T10:44:16.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.498+0000 7f2bdccde700 1 -- start start 2023-10-06T10:44:16.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.499+0000 7f2bdccde700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bd80cca70 0x7f2bd8151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.499+0000 7f2bdccde700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bd80d09c0 0x7f2bd80c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.499+0000 7f2bdccde700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2bd8153990 0x7f2bd815ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.499+0000 7f2bdccde700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd80c7bc0 con 0x7f2bd80cca70 2023-10-06T10:44:16.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.499+0000 7f2bdccde700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd80b0eb0 con 0x7f2bd8153990 2023-10-06T10:44:16.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.499+0000 7f2bdccde700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd80affc0 con 0x7f2bd80d09c0 2023-10-06T10:44:16.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.499+0000 7f2bd77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bd80cca70 0x7f2bd8151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.499+0000 7f2bd77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bd80cca70 0x7f2bd8151420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48454/0 (socket says 172.21.15.92:48454) 2023-10-06T10:44:16.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.499+0000 7f2bd77fe700 1 -- 172.21.15.92:0/2433008444 learned_addr learned my addr 172.21.15.92:0/2433008444 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.499+0000 7f2bd7fff700 1 --2- 172.21.15.92:0/2433008444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2bd8153990 0x7f2bd815ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.499+0000 7f2bceffd700 1 --2- 172.21.15.92:0/2433008444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bd80d09c0 0x7f2bd80c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.499+0000 7f2bd77fe700 1 -- 172.21.15.92:0/2433008444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bd80d09c0 msgr2=0x7f2bd80c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.499+0000 7f2bd77fe700 1 --2- 172.21.15.92:0/2433008444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bd80d09c0 0x7f2bd80c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.499+0000 7f2bd77fe700 1 -- 172.21.15.92:0/2433008444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2bd8153990 msgr2=0x7f2bd815ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.499+0000 7f2bd77fe700 1 --2- 172.21.15.92:0/2433008444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2bd8153990 0x7f2bd815ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.499+0000 7f2bd77fe700 1 -- 172.21.15.92:0/2433008444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd0009ae0 con 0x7f2bd80cca70 2023-10-06T10:44:16.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.500+0000 7f2bd77fe700 1 --2- 172.21.15.92:0/2433008444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bd80cca70 0x7f2bd8151420 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800ecb0 tx=0x7f2bc80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.500+0000 7f2bcffff700 1 -- 172.21.15.92:0/2433008444 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800c9d0 con 0x7f2bd80cca70 2023-10-06T10:44:16.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.500+0000 7f2bcffff700 1 -- 172.21.15.92:0/2433008444 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc800cb30 con 0x7f2bd80cca70 2023-10-06T10:44:16.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.500+0000 7f2bdccde700 1 -- 172.21.15.92:0/2433008444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd80b02a0 con 0x7f2bd80cca70 2023-10-06T10:44:16.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.500+0000 7f2bcffff700 1 -- 172.21.15.92:0/2433008444 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800ced0 con 0x7f2bd80cca70 2023-10-06T10:44:16.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.500+0000 7f2bdccde700 1 -- 172.21.15.92:0/2433008444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd80b0860 con 0x7f2bd80cca70 2023-10-06T10:44:16.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.501+0000 7f2bcffff700 1 -- 172.21.15.92:0/2433008444 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2bc8013680 con 0x7f2bd80cca70 2023-10-06T10:44:16.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.501+0000 7f2bcffff700 1 --2- 172.21.15.92:0/2433008444 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2bb8066f30 0x7f2bb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.501+0000 7f2bcffff700 1 -- 172.21.15.92:0/2433008444 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2bc8084b90 con 0x7f2bd80cca70 2023-10-06T10:44:16.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.502+0000 7f2bceffd700 1 --2- 172.21.15.92:0/2433008444 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2bb8066f30 0x7f2bb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.502+0000 7f2bceffd700 1 --2- 172.21.15.92:0/2433008444 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2bb8066f30 0x7f2bb80693e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2bd816cd40 tx=0x7f2bc0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.503+0000 7f2be7238b80 1 Processor -- start 2023-10-06T10:44:16.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.504+0000 7f2be7238b80 1 -- start start 2023-10-06T10:44:16.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.504+0000 7f2be7238b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623134f8a30 0x5623134f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.504+0000 7f2be7238b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562313542ca0 0x562313521a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.504+0000 7f2be7238b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x5623134f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.504+0000 7f2be7238b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562313478d60 con 0x562313521f40 2023-10-06T10:44:16.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.504+0000 7f2be7238b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56231344fb50 con 0x5623134f8a30 2023-10-06T10:44:16.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.504+0000 7f2be7238b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56231344d350 con 0x562313542ca0 2023-10-06T10:44:16.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.504+0000 7f2bd7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x5623134f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.504+0000 7f2bd7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x5623134f4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48460/0 (socket says 172.21.15.92:48460) 2023-10-06T10:44:16.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.504+0000 7f2bd7fff700 1 -- 172.21.15.92:0/599810791 learned_addr learned my addr 172.21.15.92:0/599810791 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.505+0000 7f2bd77fe700 1 --2- 172.21.15.92:0/599810791 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623134f8a30 0x5623134f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.505+0000 7f2bceffd700 1 --2- 172.21.15.92:0/599810791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562313542ca0 0x562313521a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.505+0000 7f2bd7fff700 1 -- 172.21.15.92:0/599810791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562313542ca0 msgr2=0x562313521a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.505+0000 7f2bd7fff700 1 --2- 172.21.15.92:0/599810791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562313542ca0 0x562313521a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.505+0000 7f2bd7fff700 1 -- 172.21.15.92:0/599810791 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623134f8a30 msgr2=0x5623134f8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.505+0000 7f2bd7fff700 1 --2- 172.21.15.92:0/599810791 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623134f8a30 0x5623134f8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.505+0000 7f2bd7fff700 1 -- 172.21.15.92:0/599810791 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623132e3350 con 0x562313521f40 2023-10-06T10:44:16.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.505+0000 7f2bceffd700 1 --2- 172.21.15.92:0/599810791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562313542ca0 0x562313521a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:16.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.505+0000 7f2bd7fff700 1 --2- 172.21.15.92:0/599810791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x5623134f4a70 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0012000 tx=0x7f2bd0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8cede9287ca88e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.505+0000 7f2bccff9700 1 -- 172.21.15.92:0/599810791 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0007200 con 0x562313521f40 2023-10-06T10:44:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.505+0000 7f2bccff9700 1 -- 172.21.15.92:0/599810791 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd000be30 con 0x562313521f40 2023-10-06T10:44:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.505+0000 7f2bccff9700 1 -- 172.21.15.92:0/599810791 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0007610 con 0x562313521f40 2023-10-06T10:44:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.506+0000 7f2be7238b80 1 -- 172.21.15.92:0/599810791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 msgr2=0x5623134f4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.506+0000 7f2be7238b80 1 --2- 172.21.15.92:0/599810791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x5623134f4a70 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0012000 tx=0x7f2bd0018000 comp rx=0 tx=0).stop 2023-10-06T10:44:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.506+0000 7f2be7238b80 1 -- 172.21.15.92:0/599810791 shutdown_connections 2023-10-06T10:44:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.506+0000 7f2be7238b80 1 --2- 172.21.15.92:0/599810791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x5623134f4a70 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.506+0000 7f2be7238b80 1 --2- 172.21.15.92:0/599810791 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623134f8a30 0x5623134f8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.506+0000 7f2be7238b80 1 --2- 172.21.15.92:0/599810791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562313542ca0 0x562313521a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.506+0000 7f2be7238b80 1 -- 172.21.15.92:0/599810791 >> 172.21.15.92:0/599810791 conn(0x5623134fa350 msgr2=0x5623134fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.506+0000 7f2be7238b80 1 -- 172.21.15.92:0/599810791 shutdown_connections 2023-10-06T10:44:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.506+0000 7f2be7238b80 1 -- 172.21.15.92:0/599810791 wait complete. 2023-10-06T10:44:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.506+0000 7f2be7238b80 1 -- start start 2023-10-06T10:44:16.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.506+0000 7f2be7238b80 1 Processor -- start 2023-10-06T10:44:16.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.507+0000 7f2be7238b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623134f8a30 0x56231354fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.507+0000 7f2be7238b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x562313538ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.507+0000 7f2be7238b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562313542ca0 0x562313536bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.507+0000 7f2be7238b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562313449140 con 0x562313521f40 2023-10-06T10:44:16.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.507+0000 7f2be7238b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623133f4970 con 0x5623134f8a30 2023-10-06T10:44:16.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.507+0000 7f2be7238b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56231340ff80 con 0x562313542ca0 2023-10-06T10:44:16.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.507+0000 7f2be7238b80 10 client.0 _ll_register_callbacks cb 0x5623132e6010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:16.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.507+0000 7f2bceffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x562313538ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.507+0000 7f2bceffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x562313538ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48476/0 (socket says 172.21.15.92:48476) 2023-10-06T10:44:16.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.507+0000 7f2bceffd700 1 -- 172.21.15.92:0/502527097 learned_addr learned my addr 172.21.15.92:0/502527097 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.507+0000 7f2bd7fff700 1 --2- 172.21.15.92:0/502527097 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562313542ca0 0x562313536bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.507+0000 7f2bd77fe700 1 --2- 172.21.15.92:0/502527097 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623134f8a30 0x56231354fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.508+0000 7f2bceffd700 1 -- 172.21.15.92:0/502527097 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562313542ca0 msgr2=0x562313536bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.508+0000 7f2bceffd700 1 --2- 172.21.15.92:0/502527097 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562313542ca0 0x562313536bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.508+0000 7f2bceffd700 1 -- 172.21.15.92:0/502527097 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623134f8a30 msgr2=0x56231354fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.508+0000 7f2bceffd700 1 --2- 172.21.15.92:0/502527097 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623134f8a30 0x56231354fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.508+0000 7f2bceffd700 1 -- 172.21.15.92:0/502527097 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd80ccfe0 con 0x562313521f40 2023-10-06T10:44:16.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.508+0000 7f2bceffd700 1 --2- 172.21.15.92:0/502527097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x562313538ee0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0015f00 tx=0x7f2bc0017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2c5613c8df7b29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.508+0000 7f2bb5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:16.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.508+0000 7f2bb5ffb700 1 -- 172.21.15.92:0/502527097 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0012af0 con 0x562313521f40 2023-10-06T10:44:16.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.508+0000 7f2bb5ffb700 1 -- 172.21.15.92:0/502527097 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2bc0023d90 con 0x562313521f40 2023-10-06T10:44:16.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.508+0000 7f2be7238b80 1 -- 172.21.15.92:0/502527097 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623134ed770 con 0x562313521f40 2023-10-06T10:44:16.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.508+0000 7f2bb5ffb700 1 -- 172.21.15.92:0/502527097 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bc0012e60 con 0x562313521f40 2023-10-06T10:44:16.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.508+0000 7f2be7238b80 10 client.8085 fetch_fsmap learned FSMap version 90 2023-10-06T10:44:16.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.508+0000 7f2be7238b80 1 -- 172.21.15.92:0/502527097 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x5623132860c0 con 0x562313521f40 2023-10-06T10:44:16.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.508+0000 7f2bb5ffb700 1 -- 172.21.15.92:0/502527097 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5623134ed770 con 0x562313521f40 2023-10-06T10:44:16.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.508+0000 7f2bb5ffb700 1 -- 172.21.15.92:0/502527097 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 90) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f2bc0022430 con 0x562313521f40 2023-10-06T10:44:16.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.508+0000 7f2be7238b80 10 client.8085 fetch_fsmap finished waiting for FSMap version 90 2023-10-06T10:44:16.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.508+0000 7f2be7238b80 10 client.8085 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2023-10-06T10:44:16.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.508+0000 7f2be7238b80 20 client.8085 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:16.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.508+0000 7f2be7238b80 1 --2- 172.21.15.92:0/502527097 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x562313539420 0x562313551480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.508+0000 7f2be7238b80 4 client.8085 mds_command: new command op to 7701 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:44:16.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.508+0000 7f2be7238b80 1 -- 172.21.15.92:0/502527097 --> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623134ca6a0 con 0x562313539420 2023-10-06T10:44:16.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.508+0000 7f2bd77fe700 1 --2- 172.21.15.92:0/502527097 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x562313539420 0x562313551480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.509+0000 7f2bb5ffb700 10 client.8085 ms_handle_connect on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:16.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.509+0000 7f2bd77fe700 1 --2- 172.21.15.92:0/502527097 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x562313539420 0x562313551480 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb3b5a827e7bbe77 server_cookie=48548c932c612c48 in_seq=0 out_seq=0 2023-10-06T10:44:16.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.509+0000 7f2bb5ffb700 1 -- 172.21.15.92:0/502527097 <== mds.0 v2:172.21.15.124:6832/4203118992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5623134ca6a0 con 0x562313539420 2023-10-06T10:44:16.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.509+0000 7f2bb5ffb700 10 client.8085 handle_command_reply: tid=0 2023-10-06T10:44:16.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.509+0000 7f2be7238b80 1 client.8085 shutdown 2023-10-06T10:44:16.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.510+0000 7f2be7238b80 1 -- 172.21.15.92:0/502527097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 msgr2=0x562313538ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.510+0000 7f2be7238b80 1 --2- 172.21.15.92:0/502527097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x562313538ee0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0015f00 tx=0x7f2bc0017040 comp rx=0 tx=0).stop 2023-10-06T10:44:16.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.510+0000 7f2be7238b80 1 -- 172.21.15.92:0/502527097 shutdown_connections 2023-10-06T10:44:16.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.510+0000 7f2be7238b80 1 --2- 172.21.15.92:0/502527097 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x562313539420 0x562313551480 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.510+0000 7f2be7238b80 1 --2- 172.21.15.92:0/502527097 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562313542ca0 0x562313536bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.510+0000 7f2be7238b80 1 --2- 172.21.15.92:0/502527097 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623134f8a30 0x56231354fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.510+0000 7f2be7238b80 1 --2- 172.21.15.92:0/502527097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x562313538ee0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.510+0000 7f2be7238b80 1 -- 172.21.15.92:0/502527097 >> 172.21.15.92:0/502527097 conn(0x5623134fa350 msgr2=0x5623134ef280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.510+0000 7f2bb5ffb700 0 client.8085 ms_handle_reset on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:16.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.510+0000 7f2be7238b80 1 -- 172.21.15.92:0/502527097 shutdown_connections 2023-10-06T10:44:16.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.510+0000 7f2be7238b80 1 -- 172.21.15.92:0/502527097 wait complete. 2023-10-06T10:44:16.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.510+0000 7f2be7238b80 20 client.8085 trim_cache size 0 max 16384 2023-10-06T10:44:16.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.526+0000 7f2be7238b80 1 Processor -- start 2023-10-06T10:44:16.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.526+0000 7f2be7238b80 1 -- start start 2023-10-06T10:44:16.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.527+0000 7f2be7238b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623134f8a30 0x5623134ef280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.527+0000 7f2be7238b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x5623134ef7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.527+0000 7f2be7238b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562313542ca0 0x56231354d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.527+0000 7f2be7238b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623134f7d20 con 0x562313521f40 2023-10-06T10:44:16.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.527+0000 7f2be7238b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623132cbf10 con 0x5623134f8a30 2023-10-06T10:44:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.527+0000 7f2be7238b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56231351f0a0 con 0x562313542ca0 2023-10-06T10:44:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.527+0000 7f2bceffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x5623134ef7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.527+0000 7f2bceffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x5623134ef7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48478/0 (socket says 172.21.15.92:48478) 2023-10-06T10:44:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.527+0000 7f2bceffd700 1 -- 172.21.15.92:0/3283951479 learned_addr learned my addr 172.21.15.92:0/3283951479 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.527+0000 7f2bd7fff700 1 --2- 172.21.15.92:0/3283951479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562313542ca0 0x56231354d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.527+0000 7f2bd77fe700 1 --2- 172.21.15.92:0/3283951479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623134f8a30 0x5623134ef280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.527+0000 7f2bceffd700 1 -- 172.21.15.92:0/3283951479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562313542ca0 msgr2=0x56231354d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.527+0000 7f2bceffd700 1 --2- 172.21.15.92:0/3283951479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562313542ca0 0x56231354d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.527+0000 7f2bceffd700 1 -- 172.21.15.92:0/3283951479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623134f8a30 msgr2=0x5623134ef280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.527+0000 7f2bceffd700 1 --2- 172.21.15.92:0/3283951479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623134f8a30 0x5623134ef280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.527+0000 7f2bceffd700 1 -- 172.21.15.92:0/3283951479 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562313417860 con 0x562313521f40 2023-10-06T10:44:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.527+0000 7f2bd77fe700 1 --2- 172.21.15.92:0/3283951479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623134f8a30 0x5623134ef280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:16.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.527+0000 7f2bceffd700 1 --2- 172.21.15.92:0/3283951479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x5623134ef7c0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00154f0 tx=0x7f2bc001a030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a1954078188575 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.527+0000 7f2bcd7fa700 1 -- 172.21.15.92:0/3283951479 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0022990 con 0x562313521f40 2023-10-06T10:44:16.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.527+0000 7f2bcd7fa700 1 -- 172.21.15.92:0/3283951479 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc0007e90 con 0x562313521f40 2023-10-06T10:44:16.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.527+0000 7f2bcd7fa700 1 -- 172.21.15.92:0/3283951479 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0012940 con 0x562313521f40 2023-10-06T10:44:16.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.528+0000 7f2be7238b80 1 -- 172.21.15.92:0/3283951479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 msgr2=0x5623134ef7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.528+0000 7f2be7238b80 1 --2- 172.21.15.92:0/3283951479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x5623134ef7c0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00154f0 tx=0x7f2bc001a030 comp rx=0 tx=0).stop 2023-10-06T10:44:16.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.528+0000 7f2be7238b80 1 -- 172.21.15.92:0/3283951479 shutdown_connections 2023-10-06T10:44:16.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.528+0000 7f2be7238b80 1 --2- 172.21.15.92:0/3283951479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562313542ca0 0x56231354d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.528+0000 7f2be7238b80 1 --2- 172.21.15.92:0/3283951479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623134f8a30 0x5623134ef280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.528+0000 7f2be7238b80 1 --2- 172.21.15.92:0/3283951479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x5623134ef7c0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.528+0000 7f2be7238b80 1 -- 172.21.15.92:0/3283951479 >> 172.21.15.92:0/3283951479 conn(0x5623134fa350 msgr2=0x56231353e580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.528+0000 7f2be7238b80 1 -- 172.21.15.92:0/3283951479 shutdown_connections 2023-10-06T10:44:16.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.528+0000 7f2be7238b80 1 -- 172.21.15.92:0/3283951479 wait complete. 2023-10-06T10:44:16.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.528+0000 7f2be7238b80 1 -- start start 2023-10-06T10:44:16.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.528+0000 7f2be7238b80 1 Processor -- start 2023-10-06T10:44:16.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.529+0000 7f2be7238b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623134f8a30 0x5623134ef280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.529+0000 7f2be7238b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x5623134ef7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.529+0000 7f2be7238b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562313542ca0 0x56231353f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.529+0000 7f2be7238b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562313537fe0 con 0x562313521f40 2023-10-06T10:44:16.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.529+0000 7f2be7238b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623134fb5c0 con 0x5623134f8a30 2023-10-06T10:44:16.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.529+0000 7f2be7238b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623135382c0 con 0x562313542ca0 2023-10-06T10:44:16.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.529+0000 7f2be7238b80 10 client.0 _ll_register_callbacks cb 0x5623132e6010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:16.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.529+0000 7f2bceffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x5623134ef7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.529+0000 7f2bceffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x5623134ef7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48492/0 (socket says 172.21.15.92:48492) 2023-10-06T10:44:16.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.529+0000 7f2bceffd700 1 -- 172.21.15.92:0/3927788314 learned_addr learned my addr 172.21.15.92:0/3927788314 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.529+0000 7f2bd77fe700 1 --2- 172.21.15.92:0/3927788314 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623134f8a30 0x5623134ef280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.529+0000 7f2bd7fff700 1 --2- 172.21.15.92:0/3927788314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562313542ca0 0x56231353f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.530+0000 7f2bceffd700 1 -- 172.21.15.92:0/3927788314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562313542ca0 msgr2=0x56231353f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.530+0000 7f2bceffd700 1 --2- 172.21.15.92:0/3927788314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562313542ca0 0x56231353f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.530+0000 7f2bceffd700 1 -- 172.21.15.92:0/3927788314 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623134f8a30 msgr2=0x5623134ef280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.530+0000 7f2bceffd700 1 --2- 172.21.15.92:0/3927788314 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623134f8a30 0x5623134ef280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.530+0000 7f2bceffd700 1 -- 172.21.15.92:0/3927788314 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623132860c0 con 0x562313521f40 2023-10-06T10:44:16.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.530+0000 7f2bceffd700 1 --2- 172.21.15.92:0/3927788314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x5623134ef7c0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0015fd0 tx=0x7f2bc002e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6520f2259770df51 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.530+0000 7f2bb5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:16.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.530+0000 7f2bb5ffb700 1 -- 172.21.15.92:0/3927788314 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc003a070 con 0x562313521f40 2023-10-06T10:44:16.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.530+0000 7f2bb5ffb700 1 -- 172.21.15.92:0/3927788314 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2bc00228a0 con 0x562313521f40 2023-10-06T10:44:16.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.530+0000 7f2be7238b80 1 -- 172.21.15.92:0/3927788314 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2bd000be30 con 0x562313521f40 2023-10-06T10:44:16.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.530+0000 7f2bb5ffb700 1 -- 172.21.15.92:0/3927788314 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bc0015c50 con 0x562313521f40 2023-10-06T10:44:16.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.530+0000 7f2be7238b80 10 client.8091 fetch_fsmap learned FSMap version 90 2023-10-06T10:44:16.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.530+0000 7f2be7238b80 1 -- 172.21.15.92:0/3927788314 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x5623133fe550 con 0x562313521f40 2023-10-06T10:44:16.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.530+0000 7f2bb5ffb700 1 -- 172.21.15.92:0/3927788314 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd000be30 con 0x562313521f40 2023-10-06T10:44:16.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.530+0000 7f2bb5ffb700 1 -- 172.21.15.92:0/3927788314 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 90) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f2bc0006820 con 0x562313521f40 2023-10-06T10:44:16.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.530+0000 7f2be7238b80 10 client.8091 fetch_fsmap finished waiting for FSMap version 90 2023-10-06T10:44:16.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.530+0000 7f2be7238b80 10 client.8091 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2023-10-06T10:44:16.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.530+0000 7f2be7238b80 20 client.8091 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:16.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.530+0000 7f2be7238b80 1 --2- 172.21.15.92:0/3927788314 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x562313559ea0 0x56231353f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.530+0000 7f2be7238b80 4 client.8091 mds_command: new command op to 7701 tid=0[{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2023-10-06T10:44:16.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.530+0000 7f2be7238b80 1 -- 172.21.15.92:0/3927788314 --> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) v1 -- 0x5623133ebee0 con 0x562313559ea0 2023-10-06T10:44:16.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.530+0000 7f2bd77fe700 1 --2- 172.21.15.92:0/3927788314 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x562313559ea0 0x56231353f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.531+0000 7f2bb5ffb700 10 client.8091 ms_handle_connect on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:16.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.531+0000 7f2bd77fe700 1 --2- 172.21.15.92:0/3927788314 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x562313559ea0 0x56231353f9b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=837df38f760d2867 server_cookie=b3497a44ecd16c0a in_seq=0 out_seq=0 2023-10-06T10:44:16.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.531+0000 7f2bb5ffb700 1 -- 172.21.15.92:0/3927788314 <== mds.0 v2:172.21.15.124:6832/4203118992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5623133ebee0 con 0x562313559ea0 2023-10-06T10:44:16.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.531+0000 7f2bb5ffb700 10 client.8091 handle_command_reply: tid=0 2023-10-06T10:44:16.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.531+0000 7f2be7238b80 1 client.8091 shutdown 2023-10-06T10:44:16.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.532+0000 7f2be7238b80 1 -- 172.21.15.92:0/3927788314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 msgr2=0x5623134ef7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.532+0000 7f2be7238b80 1 --2- 172.21.15.92:0/3927788314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x5623134ef7c0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0015fd0 tx=0x7f2bc002e040 comp rx=0 tx=0).stop 2023-10-06T10:44:16.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.532+0000 7f2be7238b80 1 -- 172.21.15.92:0/3927788314 shutdown_connections 2023-10-06T10:44:16.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.532+0000 7f2be7238b80 1 --2- 172.21.15.92:0/3927788314 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x562313559ea0 0x56231353f9b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.532+0000 7f2be7238b80 1 --2- 172.21.15.92:0/3927788314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562313542ca0 0x56231353f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.532+0000 7f2bb5ffb700 0 client.8091 ms_handle_reset on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:16.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.532+0000 7f2be7238b80 1 --2- 172.21.15.92:0/3927788314 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623134f8a30 0x5623134ef280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.532+0000 7f2be7238b80 1 --2- 172.21.15.92:0/3927788314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562313521f40 0x5623134ef7c0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.532+0000 7f2be7238b80 1 -- 172.21.15.92:0/3927788314 >> 172.21.15.92:0/3927788314 conn(0x5623134fa350 msgr2=0x562313528a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.532+0000 7f2be7238b80 1 -- 172.21.15.92:0/3927788314 shutdown_connections 2023-10-06T10:44:16.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.532+0000 7f2be7238b80 1 -- 172.21.15.92:0/3927788314 wait complete. 2023-10-06T10:44:16.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.532+0000 7f2be7238b80 20 client.8091 trim_cache size 0 max 16384 2023-10-06T10:44:16.541 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:44:16.541 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0, 2023-10-06T10:44:16.541 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_tag": "9669003b-2b0f-4975-abcd-f07cf375c689", 2023-10-06T10:44:16.541 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": "asynchronous" 2023-10-06T10:44:16.542 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:44:16.542 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-10-06T10:44:16.532+0000 7f13de217700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2023-10-06T10:43:59.981922+0000 1=0+1) n(v2 rc2023-10-06T10:44:00.210915+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={7866=pAsLsXsFsx/pAsLsXsFsx@4},l=7866 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x558f21c5d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:43:59.981922+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:44:00.210915+0000 4=0+4)","memory_value.dirstat":"f(v0 m2023-10-06T10:43:59.981922+0000 1=0+1)","memory_value.rstat":"n(v2 rc2023-10-06T10:44:00.210915+0000 4=0+4)","error_str":""},"return_code":-61} 2023-10-06T10:44:16.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.532+0000 7f2bdccde700 1 -- 172.21.15.92:0/2433008444 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2bb8066f30 msgr2=0x7f2bb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.532+0000 7f2bdccde700 1 --2- 172.21.15.92:0/2433008444 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2bb8066f30 0x7f2bb80693e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2bd816cd40 tx=0x7f2bc0009040 comp rx=0 tx=0).stop 2023-10-06T10:44:16.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.532+0000 7f2bdccde700 1 -- 172.21.15.92:0/2433008444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bd80cca70 msgr2=0x7f2bd8151420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.532+0000 7f2bdccde700 1 --2- 172.21.15.92:0/2433008444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bd80cca70 0x7f2bd8151420 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800ecb0 tx=0x7f2bc80091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.533+0000 7f2bdccde700 1 -- 172.21.15.92:0/2433008444 shutdown_connections 2023-10-06T10:44:16.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.533+0000 7f2bdccde700 1 --2- 172.21.15.92:0/2433008444 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2bb8066f30 0x7f2bb80693e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.533+0000 7f2bdccde700 1 --2- 172.21.15.92:0/2433008444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2bd8153990 0x7f2bd815ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.533+0000 7f2bdccde700 1 --2- 172.21.15.92:0/2433008444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bd80cca70 0x7f2bd8151420 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.533+0000 7f2bdccde700 1 --2- 172.21.15.92:0/2433008444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bd80d09c0 0x7f2bd80c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.533+0000 7f2bdccde700 1 -- 172.21.15.92:0/2433008444 >> 172.21.15.92:0/2433008444 conn(0x7f2bd80680c0 msgr2=0x7f2bd80cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.544 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-10-06T10:44:16.533+0000 7fc921715700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2023-10-06T10:43:59.982922+0000 1=0+1) n(v0 rc2023-10-06T10:43:59.982922+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={7866=pAsLsXsFsx/pAsLsXsFsx@4},l=7866 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x56472c3c9080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:43:59.982922+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:43:59.982922+0000 2=0+2)","memory_value.dirstat":"f(v0 m2023-10-06T10:43:59.982922+0000 1=0+1)","memory_value.rstat":"n(v0 rc2023-10-06T10:43:59.982922+0000 2=0+2)","error_str":""},"return_code":-61} 2023-10-06T10:44:16.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.533+0000 7f2bdccde700 1 -- 172.21.15.92:0/2433008444 shutdown_connections 2023-10-06T10:44:16.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.533+0000 7f2bdccde700 1 -- 172.21.15.92:0/2433008444 wait complete. 2023-10-06T10:44:16.545 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-10-06T10:44:16.533+0000 7fc921715700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={7866=pAsLsXsFsx/AsLsXsFsx@2},l=7866 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x56472c3c9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2023-10-06T10:44:16.545 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-10-06T10:44:16.534+0000 7f13e4223700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2023-10-06T10:43:59.982922+0000 1=0+1) n(v1 rc2023-10-06T10:44:00.210915+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={7866=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x558f21c5c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:43:59.982922+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:43:59.982922+0000 3=0+3)","memory_value.dirstat":"f(v0 m2023-10-06T10:43:59.982922+0000 1=0+1)","memory_value.rstat":"n(v1 rc2023-10-06T10:44:00.210915+0000 3=0+3)","error_str":""},"return_code":-61} 2023-10-06T10:44:16.545 INFO:tasks.ceph.mds.b.smithi124.stderr:2023-10-06T10:44:16.532+0000 7f55782a2700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2023-10-06T10:43:59.980922+0000 1=0+1) n(v3 rc2023-10-06T10:44:00.210915+0000 6=0+6) (inest mix dirty) (ifile excl) (iversion lock) caps={7866=pAsLsXsFsx/pAsLsXsFsx@5},l=7866 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55cab959b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:43:59.980922+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:44:00.210915+0000 6=0+6)","memory_value.dirstat":"f(v0 m2023-10-06T10:43:59.980922+0000 1=0+1)","memory_value.rstat":"n(v3 rc2023-10-06T10:44:00.210915+0000 6=0+6)","error_str":""},"return_code":-61} 2023-10-06T10:44:16.545 INFO:tasks.ceph.mds.b.smithi124.stderr:2023-10-06T10:44:16.533+0000 7f557e2ae700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2023-10-06T10:43:59.980922+0000 1=0+1) n(v4 rc2023-10-06T10:44:00.210915+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={7866=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x55cab9672000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:43:59.980922+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:44:00.210915+0000 5=0+5)","memory_value.dirstat":"f(v0 m2023-10-06T10:43:59.980922+0000 1=0+1)","memory_value.rstat":"n(v4 rc2023-10-06T10:44:00.210915+0000 5=0+5)","error_str":""},"return_code":-61} 2023-10-06T10:44:16.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub pause 2023-10-06T10:44:16.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.661+0000 7f66de3cb700 1 Processor -- start 2023-10-06T10:44:16.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.661+0000 7f66de3cb700 1 -- start start 2023-10-06T10:44:16.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.662+0000 7f66de3cb700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f66d80c5250 0x7f66d80ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.662+0000 7f66de3cb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f66d80cd300 0x7f66d80b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.662+0000 7f66de3cb700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66d815ac30 con 0x7f66d8150940 2023-10-06T10:44:16.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.662+0000 7f66de3cb700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66d80cb6a0 con 0x7f66d80cd300 2023-10-06T10:44:16.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.662+0000 7f66de3cb700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66d80cb090 con 0x7f66d80c5250 2023-10-06T10:44:16.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.662+0000 7f66dc968700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f66d80cd300 0x7f66d80b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.662+0000 7f66dc968700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f66d80cd300 0x7f66d80b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48502/0 (socket says 172.21.15.92:48502) 2023-10-06T10:44:16.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.662+0000 7f66dc968700 1 -- 172.21.15.92:0/422156325 learned_addr learned my addr 172.21.15.92:0/422156325 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.663+0000 7f66cffff700 1 --2- 172.21.15.92:0/422156325 >> v2:172.21.15.124:3300/0 conn(0x7f66d80c5250 0x7f66d80ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.663+0000 7f66dc968700 1 -- 172.21.15.92:0/422156325 >> v2:172.21.15.124:3300/0 conn(0x7f66d80c5250 msgr2=0x7f66d80ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.663+0000 7f66dc968700 1 --2- 172.21.15.92:0/422156325 >> v2:172.21.15.124:3300/0 conn(0x7f66d80c5250 0x7f66d80ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.663+0000 7f66dc968700 1 -- 172.21.15.92:0/422156325 >> v1:172.21.15.124:6789/0 conn(0x7f66d8150940 legacy=0x7f66d8159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.663+0000 7f66dc968700 1 -- 172.21.15.92:0/422156325 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d80cd700 con 0x7f66d80cd300 2023-10-06T10:44:16.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.663+0000 7f66dc968700 1 --2- 172.21.15.92:0/422156325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66d80cd300 0x7f66d80b2050 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f66d8088f00 tx=0x7f66d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ce54e7771a6f45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.663+0000 7f66d77fe700 1 -- 172.21.15.92:0/422156325 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d000e070 con 0x7f66d80cd300 2023-10-06T10:44:16.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.663+0000 7f66d77fe700 1 -- 172.21.15.92:0/422156325 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d000ba50 con 0x7f66d80cd300 2023-10-06T10:44:16.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.663+0000 7f66d77fe700 1 -- 172.21.15.92:0/422156325 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d00092e0 con 0x7f66d80cd300 2023-10-06T10:44:16.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.664+0000 7f66de3cb700 1 -- 172.21.15.92:0/422156325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66d80cd300 msgr2=0x7f66d80b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.664+0000 7f66de3cb700 1 --2- 172.21.15.92:0/422156325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66d80cd300 0x7f66d80b2050 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f66d8088f00 tx=0x7f66d0009840 comp rx=0 tx=0).stop 2023-10-06T10:44:16.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.664+0000 7f66de3cb700 1 -- 172.21.15.92:0/422156325 shutdown_connections 2023-10-06T10:44:16.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.664+0000 7f66de3cb700 1 --2- 172.21.15.92:0/422156325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66d80cd300 0x7f66d80b2050 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.664+0000 7f66de3cb700 1 --2- 172.21.15.92:0/422156325 >> v2:172.21.15.124:3300/0 conn(0x7f66d80c5250 0x7f66d80ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.664+0000 7f66de3cb700 1 -- 172.21.15.92:0/422156325 >> 172.21.15.92:0/422156325 conn(0x7f66d8099700 msgr2=0x7f66d806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.664+0000 7f66de3cb700 1 -- 172.21.15.92:0/422156325 shutdown_connections 2023-10-06T10:44:16.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.664+0000 7f66de3cb700 1 -- 172.21.15.92:0/422156325 wait complete. 2023-10-06T10:44:16.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.665+0000 7f66de3cb700 1 Processor -- start 2023-10-06T10:44:16.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.665+0000 7f66de3cb700 1 -- start start 2023-10-06T10:44:16.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.665+0000 7f66de3cb700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66d80c5250 0x7f66d80b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.665+0000 7f66de3cb700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66d80cd300 0x7f66d8156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.665+0000 7f66de3cb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66d8155350 0x7f66d8158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.665+0000 7f66de3cb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d8155ba0 con 0x7f66d8155350 2023-10-06T10:44:16.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.665+0000 7f66de3cb700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d8155e80 con 0x7f66d80cd300 2023-10-06T10:44:16.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.665+0000 7f66de3cb700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d8156160 con 0x7f66d80c5250 2023-10-06T10:44:16.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.666+0000 7f66dc968700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66d8155350 0x7f66d8158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.666+0000 7f66dc968700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66d8155350 0x7f66d8158cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48506/0 (socket says 172.21.15.92:48506) 2023-10-06T10:44:16.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.666+0000 7f66dc968700 1 -- 172.21.15.92:0/1883795685 learned_addr learned my addr 172.21.15.92:0/1883795685 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.666+0000 7f66cffff700 1 --2- 172.21.15.92:0/1883795685 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66d80cd300 0x7f66d8156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.666+0000 7f66d7fff700 1 --2- 172.21.15.92:0/1883795685 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66d80c5250 0x7f66d80b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.666+0000 7f66dc968700 1 -- 172.21.15.92:0/1883795685 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66d80c5250 msgr2=0x7f66d80b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.666+0000 7f66dc968700 1 --2- 172.21.15.92:0/1883795685 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66d80c5250 0x7f66d80b4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.666+0000 7f66dc968700 1 -- 172.21.15.92:0/1883795685 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66d80cd300 msgr2=0x7f66d8156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.666+0000 7f66dc968700 1 --2- 172.21.15.92:0/1883795685 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66d80cd300 0x7f66d8156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.666+0000 7f66dc968700 1 -- 172.21.15.92:0/1883795685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d0009ae0 con 0x7f66d8155350 2023-10-06T10:44:16.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.666+0000 7f66dc968700 1 --2- 172.21.15.92:0/1883795685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66d8155350 0x7f66d8158cc0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f66d80d14c0 tx=0x7f66d0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.666+0000 7f66d4ff9700 1 -- 172.21.15.92:0/1883795685 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d000be60 con 0x7f66d8155350 2023-10-06T10:44:16.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.666+0000 7f66d4ff9700 1 -- 172.21.15.92:0/1883795685 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d0008d20 con 0x7f66d8155350 2023-10-06T10:44:16.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.666+0000 7f66de3cb700 1 -- 172.21.15.92:0/1883795685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d8156440 con 0x7f66d8155350 2023-10-06T10:44:16.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.666+0000 7f66d4ff9700 1 -- 172.21.15.92:0/1883795685 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d00266b0 con 0x7f66d8155350 2023-10-06T10:44:16.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.666+0000 7f66de3cb700 1 -- 172.21.15.92:0/1883795685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d81557c0 con 0x7f66d8155350 2023-10-06T10:44:16.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.667+0000 7f66d4ff9700 1 -- 172.21.15.92:0/1883795685 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f66d0003040 con 0x7f66d8155350 2023-10-06T10:44:16.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.668+0000 7f66d4ff9700 1 --2- 172.21.15.92:0/1883795685 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f66b8066f30 0x7f66b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.668+0000 7f66d4ff9700 1 -- 172.21.15.92:0/1883795685 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f66d00867c0 con 0x7f66d8155350 2023-10-06T10:44:16.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.668+0000 7f66d7fff700 1 --2- 172.21.15.92:0/1883795685 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f66b8066f30 0x7f66b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.668+0000 7f66d7fff700 1 --2- 172.21.15.92:0/1883795685 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f66b8066f30 0x7f66b80693e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f66d8089bb0 tx=0x7f66c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.670+0000 7f66e76c3b80 1 Processor -- start 2023-10-06T10:44:16.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.670+0000 7f66e76c3b80 1 -- start start 2023-10-06T10:44:16.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.671+0000 7f66e76c3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b53222aad0 0x55b53222aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.671+0000 7f66e76c3b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5321e69e0 0x55b5321e35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.671+0000 7f66e76c3b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5321e4c10 0x55b5321e3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.671+0000 7f66e76c3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b532163cc0 con 0x55b53222aad0 2023-10-06T10:44:16.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.671+0000 7f66e76c3b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b53213aab0 con 0x55b5321e4c10 2023-10-06T10:44:16.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.671+0000 7f66e76c3b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5321382b0 con 0x55b5321e69e0 2023-10-06T10:44:16.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.671+0000 7f66d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b53222aad0 0x55b53222aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.671+0000 7f66d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b53222aad0 0x55b53222aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48518/0 (socket says 172.21.15.92:48518) 2023-10-06T10:44:16.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.671+0000 7f66d7fff700 1 -- 172.21.15.92:0/1207265174 learned_addr learned my addr 172.21.15.92:0/1207265174 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.671+0000 7f66cffff700 1 --2- 172.21.15.92:0/1207265174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5321e69e0 0x55b5321e35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.671+0000 7f66dc968700 1 --2- 172.21.15.92:0/1207265174 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5321e4c10 0x55b5321e3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.671+0000 7f66d7fff700 1 -- 172.21.15.92:0/1207265174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5321e69e0 msgr2=0x55b5321e35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.671+0000 7f66d7fff700 1 --2- 172.21.15.92:0/1207265174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5321e69e0 0x55b5321e35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.671+0000 7f66d7fff700 1 -- 172.21.15.92:0/1207265174 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5321e4c10 msgr2=0x55b5321e3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.671+0000 7f66d7fff700 1 --2- 172.21.15.92:0/1207265174 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5321e4c10 0x55b5321e3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.671+0000 7f66d7fff700 1 -- 172.21.15.92:0/1207265174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b531fce2b0 con 0x55b53222aad0 2023-10-06T10:44:16.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.671+0000 7f66d7fff700 1 --2- 172.21.15.92:0/1207265174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b53222aad0 0x55b53222aea0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f66c800fac0 tx=0x7f66c8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b4bfbb8e5dbe3c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.672+0000 7f66cd7fa700 1 -- 172.21.15.92:0/1207265174 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c8014760 con 0x55b53222aad0 2023-10-06T10:44:16.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.672+0000 7f66cd7fa700 1 -- 172.21.15.92:0/1207265174 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c80148c0 con 0x55b53222aad0 2023-10-06T10:44:16.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.672+0000 7f66cd7fa700 1 -- 172.21.15.92:0/1207265174 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c8014c40 con 0x55b53222aad0 2023-10-06T10:44:16.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.672+0000 7f66e76c3b80 1 -- 172.21.15.92:0/1207265174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b53222aad0 msgr2=0x55b53222aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.672+0000 7f66e76c3b80 1 --2- 172.21.15.92:0/1207265174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b53222aad0 0x55b53222aea0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f66c800fac0 tx=0x7f66c8015000 comp rx=0 tx=0).stop 2023-10-06T10:44:16.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.672+0000 7f66e76c3b80 1 -- 172.21.15.92:0/1207265174 shutdown_connections 2023-10-06T10:44:16.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.672+0000 7f66e76c3b80 1 --2- 172.21.15.92:0/1207265174 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5321e4c10 0x55b5321e3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.672+0000 7f66e76c3b80 1 --2- 172.21.15.92:0/1207265174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b53222aad0 0x55b53222aea0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.672+0000 7f66e76c3b80 1 --2- 172.21.15.92:0/1207265174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5321e69e0 0x55b5321e35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.672+0000 7f66e76c3b80 1 -- 172.21.15.92:0/1207265174 >> 172.21.15.92:0/1207265174 conn(0x55b5321e50b0 msgr2=0x55b5321e5480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.672+0000 7f66e76c3b80 1 -- 172.21.15.92:0/1207265174 shutdown_connections 2023-10-06T10:44:16.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.672+0000 7f66e76c3b80 1 -- 172.21.15.92:0/1207265174 wait complete. 2023-10-06T10:44:16.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.673+0000 7f66e76c3b80 1 -- start start 2023-10-06T10:44:16.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.673+0000 7f66e76c3b80 1 Processor -- start 2023-10-06T10:44:16.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66e76c3b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5321e4c10 0x55b532226210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66e76c3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5321e69e0 0x55b53220bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66e76c3b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b53222aad0 0x55b53220c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66e76c3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5321340a0 con 0x55b5321e69e0 2023-10-06T10:44:16.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66e76c3b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5320df8d0 con 0x55b53222aad0 2023-10-06T10:44:16.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66e76c3b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5320faee0 con 0x55b5321e4c10 2023-10-06T10:44:16.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66e76c3b80 10 client.0 _ll_register_callbacks cb 0x55b531fd0f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:16.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66cffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5321e69e0 0x55b53220bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66cffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5321e69e0 0x55b53220bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48526/0 (socket says 172.21.15.92:48526) 2023-10-06T10:44:16.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66cffff700 1 -- 172.21.15.92:0/518576335 learned_addr learned my addr 172.21.15.92:0/518576335 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66dc968700 1 --2- 172.21.15.92:0/518576335 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b53222aad0 0x55b53220c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66d7fff700 1 --2- 172.21.15.92:0/518576335 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5321e4c10 0x55b532226210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66cffff700 1 -- 172.21.15.92:0/518576335 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5321e4c10 msgr2=0x55b532226210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66cffff700 1 --2- 172.21.15.92:0/518576335 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5321e4c10 0x55b532226210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66cffff700 1 -- 172.21.15.92:0/518576335 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b53222aad0 msgr2=0x55b53220c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66cffff700 1 --2- 172.21.15.92:0/518576335 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b53222aad0 0x55b53220c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66cffff700 1 -- 172.21.15.92:0/518576335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c800f650 con 0x55b5321e69e0 2023-10-06T10:44:16.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66dc968700 1 --2- 172.21.15.92:0/518576335 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b53222aad0 0x55b53220c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:16.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66d7fff700 1 --2- 172.21.15.92:0/518576335 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5321e4c10 0x55b532226210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:16.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66cffff700 1 --2- 172.21.15.92:0/518576335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5321e69e0 0x55b53220bd10 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f66c0012780 tx=0x7f66c0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282083c63cd515b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66b67fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:16.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66b67fc700 1 -- 172.21.15.92:0/518576335 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c0014690 con 0x55b5321e69e0 2023-10-06T10:44:16.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66b67fc700 1 -- 172.21.15.92:0/518576335 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f66c00147f0 con 0x55b5321e69e0 2023-10-06T10:44:16.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66e76c3b80 1 -- 172.21.15.92:0/518576335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b5321d86d0 con 0x55b5321e69e0 2023-10-06T10:44:16.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66b67fc700 1 -- 172.21.15.92:0/518576335 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66c0014bc0 con 0x55b5321e69e0 2023-10-06T10:44:16.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66e76c3b80 10 client.8103 fetch_fsmap learned FSMap version 90 2023-10-06T10:44:16.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.674+0000 7f66e76c3b80 1 -- 172.21.15.92:0/518576335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x55b531f71020 con 0x55b5321e69e0 2023-10-06T10:44:16.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.675+0000 7f66b67fc700 1 -- 172.21.15.92:0/518576335 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b5321d86d0 con 0x55b5321e69e0 2023-10-06T10:44:16.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.675+0000 7f66b67fc700 1 -- 172.21.15.92:0/518576335 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 90) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f66c0014d20 con 0x55b5321e69e0 2023-10-06T10:44:16.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.675+0000 7f66e76c3b80 10 client.8103 fetch_fsmap finished waiting for FSMap version 90 2023-10-06T10:44:16.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.675+0000 7f66e76c3b80 10 client.8103 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2023-10-06T10:44:16.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.675+0000 7f66e76c3b80 20 client.8103 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:16.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.675+0000 7f66e76c3b80 1 --2- 172.21.15.92:0/518576335 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x55b532227850 0x55b532224ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.675+0000 7f66e76c3b80 4 client.8103 mds_command: new command op to 7701 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:44:16.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.675+0000 7f66e76c3b80 1 -- 172.21.15.92:0/518576335 --> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5321b5600 con 0x55b532227850 2023-10-06T10:44:16.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.675+0000 7f66d7fff700 1 --2- 172.21.15.92:0/518576335 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x55b532227850 0x55b532224ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.675+0000 7f66b67fc700 10 client.8103 ms_handle_connect on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:16.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.675+0000 7f66d7fff700 1 --2- 172.21.15.92:0/518576335 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x55b532227850 0x55b532224ef0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4510812969092ff server_cookie=691914bd36de69ae in_seq=0 out_seq=0 2023-10-06T10:44:16.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.676+0000 7f66b67fc700 1 -- 172.21.15.92:0/518576335 <== mds.0 v2:172.21.15.124:6832/4203118992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55b5321b5600 con 0x55b532227850 2023-10-06T10:44:16.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.676+0000 7f66b67fc700 10 client.8103 handle_command_reply: tid=0 2023-10-06T10:44:16.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.676+0000 7f66e76c3b80 1 client.8103 shutdown 2023-10-06T10:44:16.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.676+0000 7f66e76c3b80 1 -- 172.21.15.92:0/518576335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5321e69e0 msgr2=0x55b53220bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.676+0000 7f66e76c3b80 1 --2- 172.21.15.92:0/518576335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5321e69e0 0x55b53220bd10 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f66c0012780 tx=0x7f66c0008040 comp rx=0 tx=0).stop 2023-10-06T10:44:16.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.676+0000 7f66e76c3b80 1 -- 172.21.15.92:0/518576335 shutdown_connections 2023-10-06T10:44:16.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.676+0000 7f66e76c3b80 1 --2- 172.21.15.92:0/518576335 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x55b532227850 0x55b532224ef0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.676+0000 7f66e76c3b80 1 --2- 172.21.15.92:0/518576335 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b53222aad0 0x55b53220c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.676+0000 7f66e76c3b80 1 --2- 172.21.15.92:0/518576335 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5321e4c10 0x55b532226210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.676+0000 7f66b67fc700 0 client.8103 ms_handle_reset on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:16.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.676+0000 7f66e76c3b80 1 --2- 172.21.15.92:0/518576335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5321e69e0 0x55b53220bd10 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.676+0000 7f66e76c3b80 1 -- 172.21.15.92:0/518576335 >> 172.21.15.92:0/518576335 conn(0x55b5321e50b0 msgr2=0x55b5321dfab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.676+0000 7f66e76c3b80 1 -- 172.21.15.92:0/518576335 shutdown_connections 2023-10-06T10:44:16.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.676+0000 7f66e76c3b80 1 -- 172.21.15.92:0/518576335 wait complete. 2023-10-06T10:44:16.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.676+0000 7f66e76c3b80 20 client.8103 trim_cache size 0 max 16384 2023-10-06T10:44:16.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.692+0000 7f66e76c3b80 1 Processor -- start 2023-10-06T10:44:16.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.692+0000 7f66e76c3b80 1 -- start start 2023-10-06T10:44:16.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.692+0000 7f66e76c3b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5321e4c10 0x55b53220bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.692+0000 7f66e76c3b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5321e69e0 0x55b53220c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.692+0000 7f66e76c3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b53222aad0 0x55b532228310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.692+0000 7f66e76c3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b531fb6e70 con 0x55b53222aad0 2023-10-06T10:44:16.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.692+0000 7f66e76c3b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b53222dc50 con 0x55b5321e69e0 2023-10-06T10:44:16.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.692+0000 7f66e76c3b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b532227e70 con 0x55b5321e4c10 2023-10-06T10:44:16.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.693+0000 7f66dc968700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b53222aad0 0x55b532228310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.693+0000 7f66dc968700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b53222aad0 0x55b532228310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48542/0 (socket says 172.21.15.92:48542) 2023-10-06T10:44:16.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.693+0000 7f66dc968700 1 -- 172.21.15.92:0/4010480041 learned_addr learned my addr 172.21.15.92:0/4010480041 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.693+0000 7f66d7fff700 1 --2- 172.21.15.92:0/4010480041 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5321e4c10 0x55b53220bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.693+0000 7f66cffff700 1 --2- 172.21.15.92:0/4010480041 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5321e69e0 0x55b53220c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.693+0000 7f66dc968700 1 -- 172.21.15.92:0/4010480041 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5321e4c10 msgr2=0x55b53220bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.693+0000 7f66dc968700 1 --2- 172.21.15.92:0/4010480041 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5321e4c10 0x55b53220bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.693+0000 7f66dc968700 1 -- 172.21.15.92:0/4010480041 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5321e69e0 msgr2=0x55b53220c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.693+0000 7f66dc968700 1 --2- 172.21.15.92:0/4010480041 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5321e69e0 0x55b53220c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.693+0000 7f66dc968700 1 -- 172.21.15.92:0/4010480041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5321027c0 con 0x55b53222aad0 2023-10-06T10:44:16.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.693+0000 7f66dc968700 1 --2- 172.21.15.92:0/4010480041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b53222aad0 0x55b532228310 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f66d0057490 tx=0x7f66d0086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beadc7d38b801010 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.693+0000 7f66cdffb700 1 -- 172.21.15.92:0/4010480041 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d000e040 con 0x55b53222aad0 2023-10-06T10:44:16.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.693+0000 7f66cdffb700 1 -- 172.21.15.92:0/4010480041 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d005a6a0 con 0x55b53222aad0 2023-10-06T10:44:16.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.693+0000 7f66cdffb700 1 -- 172.21.15.92:0/4010480041 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d005e3f0 con 0x55b53222aad0 2023-10-06T10:44:16.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.694+0000 7f66e76c3b80 1 -- 172.21.15.92:0/4010480041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b53222aad0 msgr2=0x55b532228310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.694+0000 7f66e76c3b80 1 --2- 172.21.15.92:0/4010480041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b53222aad0 0x55b532228310 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f66d0057490 tx=0x7f66d0086040 comp rx=0 tx=0).stop 2023-10-06T10:44:16.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.694+0000 7f66e76c3b80 1 -- 172.21.15.92:0/4010480041 shutdown_connections 2023-10-06T10:44:16.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.694+0000 7f66e76c3b80 1 --2- 172.21.15.92:0/4010480041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b53222aad0 0x55b532228310 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.694+0000 7f66e76c3b80 1 --2- 172.21.15.92:0/4010480041 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5321e4c10 0x55b53220bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.694+0000 7f66e76c3b80 1 --2- 172.21.15.92:0/4010480041 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5321e69e0 0x55b53220c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.694+0000 7f66e76c3b80 1 -- 172.21.15.92:0/4010480041 >> 172.21.15.92:0/4010480041 conn(0x55b5321e50b0 msgr2=0x55b532209410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.694+0000 7f66e76c3b80 1 -- 172.21.15.92:0/4010480041 shutdown_connections 2023-10-06T10:44:16.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.694+0000 7f66e76c3b80 1 -- 172.21.15.92:0/4010480041 wait complete. 2023-10-06T10:44:16.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.694+0000 7f66e76c3b80 1 -- start start 2023-10-06T10:44:16.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.694+0000 7f66e76c3b80 1 Processor -- start 2023-10-06T10:44:16.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.695+0000 7f66e76c3b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5321e4c10 0x55b5322911c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.695+0000 7f66e76c3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5321e69e0 0x55b5322202a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.695+0000 7f66e76c3b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b53222aad0 0x55b5322207e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.695+0000 7f66e76c3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b53223b8a0 con 0x55b5321e69e0 2023-10-06T10:44:16.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.695+0000 7f66e76c3b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b532291790 con 0x55b53222aad0 2023-10-06T10:44:16.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.695+0000 7f66e76c3b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5321e1c10 con 0x55b5321e4c10 2023-10-06T10:44:16.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.695+0000 7f66e76c3b80 10 client.0 _ll_register_callbacks cb 0x55b531fd0f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:16.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.695+0000 7f66cffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5321e69e0 0x55b5322202a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.695+0000 7f66cffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5321e69e0 0x55b5322202a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48556/0 (socket says 172.21.15.92:48556) 2023-10-06T10:44:16.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.695+0000 7f66cffff700 1 -- 172.21.15.92:0/1104093883 learned_addr learned my addr 172.21.15.92:0/1104093883 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.695+0000 7f66d7fff700 1 --2- 172.21.15.92:0/1104093883 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5321e4c10 0x55b5322911c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.695+0000 7f66dc968700 1 --2- 172.21.15.92:0/1104093883 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b53222aad0 0x55b5322207e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.695+0000 7f66cffff700 1 -- 172.21.15.92:0/1104093883 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5321e4c10 msgr2=0x55b5322911c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.696+0000 7f66cffff700 1 --2- 172.21.15.92:0/1104093883 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5321e4c10 0x55b5322911c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.696+0000 7f66cffff700 1 -- 172.21.15.92:0/1104093883 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b53222aad0 msgr2=0x55b5322207e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.696+0000 7f66cffff700 1 --2- 172.21.15.92:0/1104093883 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b53222aad0 0x55b5322207e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.696+0000 7f66cffff700 1 -- 172.21.15.92:0/1104093883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d81557c0 con 0x55b5321e69e0 2023-10-06T10:44:16.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.696+0000 7f66dc968700 1 --2- 172.21.15.92:0/1104093883 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b53222aad0 0x55b5322207e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:16.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.696+0000 7f66cffff700 1 --2- 172.21.15.92:0/1104093883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5321e69e0 0x55b5322202a0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f66c0012780 tx=0x7f66c0022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db1b29bc3c5c1a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.696+0000 7f66b67fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:16.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.696+0000 7f66b67fc700 1 -- 172.21.15.92:0/1104093883 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c002f070 con 0x55b5321e69e0 2023-10-06T10:44:16.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.696+0000 7f66b67fc700 1 -- 172.21.15.92:0/1104093883 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f66c0000e40 con 0x55b5321e69e0 2023-10-06T10:44:16.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.696+0000 7f66e76c3b80 1 -- 172.21.15.92:0/1104093883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66c80148c0 con 0x55b5321e69e0 2023-10-06T10:44:16.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.696+0000 7f66b67fc700 1 -- 172.21.15.92:0/1104093883 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66c0012ed0 con 0x55b5321e69e0 2023-10-06T10:44:16.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.696+0000 7f66e76c3b80 10 client.8109 fetch_fsmap learned FSMap version 90 2023-10-06T10:44:16.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.696+0000 7f66e76c3b80 1 -- 172.21.15.92:0/1104093883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x55b5320e94b0 con 0x55b5321e69e0 2023-10-06T10:44:16.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.696+0000 7f66b67fc700 1 -- 172.21.15.92:0/1104093883 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c80148c0 con 0x55b5321e69e0 2023-10-06T10:44:16.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.696+0000 7f66b67fc700 1 -- 172.21.15.92:0/1104093883 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 90) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f66c0013030 con 0x55b5321e69e0 2023-10-06T10:44:16.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.696+0000 7f66e76c3b80 10 client.8109 fetch_fsmap finished waiting for FSMap version 90 2023-10-06T10:44:16.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.696+0000 7f66e76c3b80 10 client.8109 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2023-10-06T10:44:16.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.696+0000 7f66e76c3b80 20 client.8109 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:16.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.696+0000 7f66e76c3b80 1 --2- 172.21.15.92:0/1104093883 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x55b5321e35c0 0x55b53223bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.696+0000 7f66e76c3b80 4 client.8109 mds_command: new command op to 7701 tid=0[{"prefix": "scrub pause"}] 2023-10-06T10:44:16.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.696+0000 7f66e76c3b80 1 -- 172.21.15.92:0/1104093883 --> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55b5320d6e40 con 0x55b5321e35c0 2023-10-06T10:44:16.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.696+0000 7f66d7fff700 1 --2- 172.21.15.92:0/1104093883 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x55b5321e35c0 0x55b53223bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.697+0000 7f66b67fc700 10 client.8109 ms_handle_connect on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:16.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.697+0000 7f66d7fff700 1 --2- 172.21.15.92:0/1104093883 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x55b5321e35c0 0x55b53223bcf0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=889c53c77c14366 server_cookie=db3713050caa7fcb in_seq=0 out_seq=0 2023-10-06T10:44:16.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.697+0000 7f66b67fc700 1 -- 172.21.15.92:0/1104093883 <== mds.0 v2:172.21.15.124:6832/4203118992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55b5320d6e40 con 0x55b5321e35c0 2023-10-06T10:44:16.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.697+0000 7f66b67fc700 10 client.8109 handle_command_reply: tid=0 2023-10-06T10:44:16.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.697+0000 7f66e76c3b80 1 client.8109 shutdown 2023-10-06T10:44:16.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.697+0000 7f66e76c3b80 1 -- 172.21.15.92:0/1104093883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5321e69e0 msgr2=0x55b5322202a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.697+0000 7f66e76c3b80 1 --2- 172.21.15.92:0/1104093883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5321e69e0 0x55b5322202a0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f66c0012780 tx=0x7f66c0022000 comp rx=0 tx=0).stop 2023-10-06T10:44:16.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.697+0000 7f66e76c3b80 1 -- 172.21.15.92:0/1104093883 shutdown_connections 2023-10-06T10:44:16.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.697+0000 7f66e76c3b80 1 --2- 172.21.15.92:0/1104093883 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x55b5321e35c0 0x55b53223bcf0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.697+0000 7f66e76c3b80 1 --2- 172.21.15.92:0/1104093883 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b53222aad0 0x55b5322207e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.697+0000 7f66b67fc700 0 client.8109 ms_handle_reset on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:16.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.697+0000 7f66e76c3b80 1 --2- 172.21.15.92:0/1104093883 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5321e4c10 0x55b5322911c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.697+0000 7f66e76c3b80 1 --2- 172.21.15.92:0/1104093883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5321e69e0 0x55b5322202a0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.697+0000 7f66e76c3b80 1 -- 172.21.15.92:0/1104093883 >> 172.21.15.92:0/1104093883 conn(0x55b5321e50b0 msgr2=0x55b5322282b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.697+0000 7f66e76c3b80 1 -- 172.21.15.92:0/1104093883 shutdown_connections 2023-10-06T10:44:16.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.697+0000 7f66e76c3b80 1 -- 172.21.15.92:0/1104093883 wait complete. 2023-10-06T10:44:16.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.697+0000 7f66e76c3b80 20 client.8109 trim_cache size 0 max 16384 2023-10-06T10:44:16.706 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:44:16.707 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:44:16.707 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:44:16.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.698+0000 7f66de3cb700 1 -- 172.21.15.92:0/1883795685 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f66b8066f30 msgr2=0x7f66b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.698+0000 7f66de3cb700 1 --2- 172.21.15.92:0/1883795685 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f66b8066f30 0x7f66b80693e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f66d8089bb0 tx=0x7f66c8009040 comp rx=0 tx=0).stop 2023-10-06T10:44:16.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.698+0000 7f66de3cb700 1 -- 172.21.15.92:0/1883795685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66d8155350 msgr2=0x7f66d8158cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.698+0000 7f66de3cb700 1 --2- 172.21.15.92:0/1883795685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66d8155350 0x7f66d8158cc0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f66d80d14c0 tx=0x7f66d0018000 comp rx=0 tx=0).stop 2023-10-06T10:44:16.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.698+0000 7f66de3cb700 1 -- 172.21.15.92:0/1883795685 shutdown_connections 2023-10-06T10:44:16.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.698+0000 7f66de3cb700 1 --2- 172.21.15.92:0/1883795685 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f66b8066f30 0x7f66b80693e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.698+0000 7f66de3cb700 1 --2- 172.21.15.92:0/1883795685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66d8155350 0x7f66d8158cc0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.698+0000 7f66de3cb700 1 --2- 172.21.15.92:0/1883795685 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66d80c5250 0x7f66d80b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.698+0000 7f66de3cb700 1 --2- 172.21.15.92:0/1883795685 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66d80cd300 0x7f66d8156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.698+0000 7f66de3cb700 1 -- 172.21.15.92:0/1883795685 >> 172.21.15.92:0/1883795685 conn(0x7f66d8099700 msgr2=0x7f66d814fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.698+0000 7f66de3cb700 1 -- 172.21.15.92:0/1883795685 shutdown_connections 2023-10-06T10:44:16.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.698+0000 7f66de3cb700 1 -- 172.21.15.92:0/1883795685 wait complete. 2023-10-06T10:44:16.711 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:1 scrub status 2023-10-06T10:44:16.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.826+0000 7f9f5616d700 1 Processor -- start 2023-10-06T10:44:16.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.826+0000 7f9f5616d700 1 -- start start 2023-10-06T10:44:16.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.827+0000 7f9f5616d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f500b2620 0x7f9f500cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.827+0000 7f9f5616d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f500b29f0 0x7f9f500cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.827+0000 7f9f5616d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9f500ceaa0 0x7f9f500cef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.827+0000 7f9f5616d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f500cc970 con 0x7f9f500b29f0 2023-10-06T10:44:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.827+0000 7f9f5616d700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f500cf480 con 0x7f9f500ceaa0 2023-10-06T10:44:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.827+0000 7f9f5616d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f500cbd60 con 0x7f9f500b2620 2023-10-06T10:44:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.827+0000 7f9f5496a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f500b29f0 0x7f9f500cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.827+0000 7f9f5496a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f500b29f0 0x7f9f500cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48564/0 (socket says 172.21.15.92:48564) 2023-10-06T10:44:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.827+0000 7f9f5496a700 1 -- 172.21.15.92:0/3751530133 learned_addr learned my addr 172.21.15.92:0/3751530133 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.827+0000 7f9f5516b700 1 --2- 172.21.15.92:0/3751530133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f500b2620 0x7f9f500cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.827+0000 7f9f5596c700 1 --2- 172.21.15.92:0/3751530133 >> v2:172.21.15.124:3300/0 conn(0x7f9f500ceaa0 0x7f9f500cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.827+0000 7f9f5496a700 1 -- 172.21.15.92:0/3751530133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f500b2620 msgr2=0x7f9f500cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.827+0000 7f9f5496a700 1 --2- 172.21.15.92:0/3751530133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f500b2620 0x7f9f500cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.827+0000 7f9f5496a700 1 -- 172.21.15.92:0/3751530133 >> v2:172.21.15.124:3300/0 conn(0x7f9f500ceaa0 msgr2=0x7f9f500cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.827+0000 7f9f5496a700 1 --2- 172.21.15.92:0/3751530133 >> v2:172.21.15.124:3300/0 conn(0x7f9f500ceaa0 0x7f9f500cef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.827+0000 7f9f5496a700 1 -- 172.21.15.92:0/3751530133 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f500cc040 con 0x7f9f500b29f0 2023-10-06T10:44:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.828+0000 7f9f5496a700 1 --2- 172.21.15.92:0/3751530133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f500b29f0 0x7f9f500cd1f0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f9f50089650 tx=0x7f9f40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f72b6476a9194 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.828+0000 7f9f47fff700 1 -- 172.21.15.92:0/3751530133 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4000b860 con 0x7f9f500b29f0 2023-10-06T10:44:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.828+0000 7f9f47fff700 1 -- 172.21.15.92:0/3751530133 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4000b9c0 con 0x7f9f500b29f0 2023-10-06T10:44:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.828+0000 7f9f47fff700 1 -- 172.21.15.92:0/3751530133 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4000bd60 con 0x7f9f500b29f0 2023-10-06T10:44:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.828+0000 7f9f5616d700 1 -- 172.21.15.92:0/3751530133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f500b29f0 msgr2=0x7f9f500cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.828+0000 7f9f5616d700 1 --2- 172.21.15.92:0/3751530133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f500b29f0 0x7f9f500cd1f0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f9f50089650 tx=0x7f9f40009250 comp rx=0 tx=0).stop 2023-10-06T10:44:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.828+0000 7f9f5616d700 1 -- 172.21.15.92:0/3751530133 shutdown_connections 2023-10-06T10:44:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.828+0000 7f9f5616d700 1 --2- 172.21.15.92:0/3751530133 >> v2:172.21.15.124:3300/0 conn(0x7f9f500ceaa0 0x7f9f500cef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.828+0000 7f9f5616d700 1 --2- 172.21.15.92:0/3751530133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f500b2620 0x7f9f500cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.828+0000 7f9f5616d700 1 --2- 172.21.15.92:0/3751530133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f500b29f0 0x7f9f500cd1f0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.828+0000 7f9f5616d700 1 -- 172.21.15.92:0/3751530133 >> 172.21.15.92:0/3751530133 conn(0x7f9f50099520 msgr2=0x7f9f5006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.828+0000 7f9f5616d700 1 -- 172.21.15.92:0/3751530133 shutdown_connections 2023-10-06T10:44:16.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.828+0000 7f9f5616d700 1 -- 172.21.15.92:0/3751530133 wait complete. 2023-10-06T10:44:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.829+0000 7f9f5616d700 1 Processor -- start 2023-10-06T10:44:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.830+0000 7f9f5616d700 1 -- start start 2023-10-06T10:44:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.830+0000 7f9f5616d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f500b2620 0x7f9f5016a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.830+0000 7f9f5616d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f500b29f0 0x7f9f50099f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.830+0000 7f9f5616d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f500ceaa0 0x7f9f5009d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.830+0000 7f9f5616d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5009b010 con 0x7f9f500ceaa0 2023-10-06T10:44:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.830+0000 7f9f5616d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5016a940 con 0x7f9f500b2620 2023-10-06T10:44:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.830+0000 7f9f5616d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5016acd0 con 0x7f9f500b29f0 2023-10-06T10:44:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.830+0000 7f9f5596c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f500ceaa0 0x7f9f5009d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.831+0000 7f9f5596c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f500ceaa0 0x7f9f5009d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48576/0 (socket says 172.21.15.92:48576) 2023-10-06T10:44:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.831+0000 7f9f5596c700 1 -- 172.21.15.92:0/920445980 learned_addr learned my addr 172.21.15.92:0/920445980 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.831+0000 7f9f5496a700 1 --2- 172.21.15.92:0/920445980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f500b29f0 0x7f9f50099f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.831+0000 7f9f5516b700 1 --2- 172.21.15.92:0/920445980 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f500b2620 0x7f9f5016a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.831+0000 7f9f5596c700 1 -- 172.21.15.92:0/920445980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f500b29f0 msgr2=0x7f9f50099f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.831+0000 7f9f5596c700 1 --2- 172.21.15.92:0/920445980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f500b29f0 0x7f9f50099f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.831+0000 7f9f5596c700 1 -- 172.21.15.92:0/920445980 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f500b2620 msgr2=0x7f9f5016a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.831+0000 7f9f5596c700 1 --2- 172.21.15.92:0/920445980 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f500b2620 0x7f9f5016a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.831+0000 7f9f5596c700 1 -- 172.21.15.92:0/920445980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f400094f0 con 0x7f9f500ceaa0 2023-10-06T10:44:16.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.831+0000 7f9f5496a700 1 --2- 172.21.15.92:0/920445980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f500b29f0 0x7f9f50099f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:16.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.831+0000 7f9f5516b700 1 --2- 172.21.15.92:0/920445980 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f500b2620 0x7f9f5016a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:16.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.831+0000 7f9f5596c700 1 --2- 172.21.15.92:0/920445980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f500ceaa0 0x7f9f5009d880 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009fa0 tx=0x7f9f48009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.831+0000 7f9f457fa700 1 -- 172.21.15.92:0/920445980 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4800ed20 con 0x7f9f500ceaa0 2023-10-06T10:44:16.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.831+0000 7f9f457fa700 1 -- 172.21.15.92:0/920445980 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4800ee80 con 0x7f9f500ceaa0 2023-10-06T10:44:16.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.831+0000 7f9f457fa700 1 -- 172.21.15.92:0/920445980 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48014670 con 0x7f9f500ceaa0 2023-10-06T10:44:16.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.831+0000 7f9f5616d700 1 -- 172.21.15.92:0/920445980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5009a4d0 con 0x7f9f500ceaa0 2023-10-06T10:44:16.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.831+0000 7f9f5616d700 1 -- 172.21.15.92:0/920445980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5009ad30 con 0x7f9f500ceaa0 2023-10-06T10:44:16.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.833+0000 7f9f457fa700 1 -- 172.21.15.92:0/920445980 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f480149e0 con 0x7f9f500ceaa0 2023-10-06T10:44:16.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.833+0000 7f9f457fa700 1 --2- 172.21.15.92:0/920445980 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f34066f30 0x7f9f340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.833+0000 7f9f457fa700 1 -- 172.21.15.92:0/920445980 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9f48084fc0 con 0x7f9f500ceaa0 2023-10-06T10:44:16.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.833+0000 7f9f5516b700 1 --2- 172.21.15.92:0/920445980 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f34066f30 0x7f9f340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.833+0000 7f9f5516b700 1 --2- 172.21.15.92:0/920445980 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f34066f30 0x7f9f340693e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c006aa0 tx=0x7f9f4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.835+0000 7f9f606c7b80 1 Processor -- start 2023-10-06T10:44:16.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.835+0000 7f9f606c7b80 1 -- start start 2023-10-06T10:44:16.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.835+0000 7f9f606c7b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5645dc9e09a0 0x5645dc9e0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.835+0000 7f9f606c7b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5645dca2ac10 0x5645dca09970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.835+0000 7f9f606c7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dca09eb0 0x5645dc9dc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.835+0000 7f9f606c7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645dc960cd0 con 0x5645dca09eb0 2023-10-06T10:44:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.835+0000 7f9f606c7b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645dc937ac0 con 0x5645dc9e09a0 2023-10-06T10:44:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.835+0000 7f9f606c7b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645dc9352c0 con 0x5645dca2ac10 2023-10-06T10:44:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.835+0000 7f9f5596c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dca09eb0 0x5645dc9dc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.835+0000 7f9f5596c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dca09eb0 0x5645dc9dc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48592/0 (socket says 172.21.15.92:48592) 2023-10-06T10:44:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.835+0000 7f9f5596c700 1 -- 172.21.15.92:0/3095973555 learned_addr learned my addr 172.21.15.92:0/3095973555 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.836+0000 7f9f5516b700 1 --2- 172.21.15.92:0/3095973555 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5645dc9e09a0 0x5645dc9e0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.836+0000 7f9f5496a700 1 --2- 172.21.15.92:0/3095973555 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5645dca2ac10 0x5645dca09970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.836+0000 7f9f5596c700 1 -- 172.21.15.92:0/3095973555 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5645dca2ac10 msgr2=0x5645dca09970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.836+0000 7f9f5596c700 1 --2- 172.21.15.92:0/3095973555 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5645dca2ac10 0x5645dca09970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.836+0000 7f9f5596c700 1 -- 172.21.15.92:0/3095973555 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5645dc9e09a0 msgr2=0x5645dc9e0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.836+0000 7f9f5596c700 1 --2- 172.21.15.92:0/3095973555 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5645dc9e09a0 0x5645dc9e0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.836+0000 7f9f5596c700 1 -- 172.21.15.92:0/3095973555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645dc7cb2c0 con 0x5645dca09eb0 2023-10-06T10:44:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.836+0000 7f9f5596c700 1 --2- 172.21.15.92:0/3095973555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dca09eb0 0x5645dc9dc9e0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f9f48052150 tx=0x7f9f480831a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2e620b1f954fff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.836+0000 7f9f3a7fc700 1 -- 172.21.15.92:0/3095973555 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48055c50 con 0x5645dca09eb0 2023-10-06T10:44:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.836+0000 7f9f3a7fc700 1 -- 172.21.15.92:0/3095973555 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f48058ea0 con 0x5645dca09eb0 2023-10-06T10:44:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.836+0000 7f9f3a7fc700 1 -- 172.21.15.92:0/3095973555 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4805cbf0 con 0x5645dca09eb0 2023-10-06T10:44:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.837+0000 7f9f606c7b80 1 -- 172.21.15.92:0/3095973555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dca09eb0 msgr2=0x5645dc9dc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.837+0000 7f9f606c7b80 1 --2- 172.21.15.92:0/3095973555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dca09eb0 0x5645dc9dc9e0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f9f48052150 tx=0x7f9f480831a0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.837+0000 7f9f606c7b80 1 -- 172.21.15.92:0/3095973555 shutdown_connections 2023-10-06T10:44:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.837+0000 7f9f606c7b80 1 --2- 172.21.15.92:0/3095973555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dca09eb0 0x5645dc9dc9e0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.837+0000 7f9f606c7b80 1 --2- 172.21.15.92:0/3095973555 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5645dc9e09a0 0x5645dc9e0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.837+0000 7f9f606c7b80 1 --2- 172.21.15.92:0/3095973555 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5645dca2ac10 0x5645dca09970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.837+0000 7f9f606c7b80 1 -- 172.21.15.92:0/3095973555 >> 172.21.15.92:0/3095973555 conn(0x5645dc9e22c0 msgr2=0x5645dc9e2690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.837+0000 7f9f606c7b80 1 -- 172.21.15.92:0/3095973555 shutdown_connections 2023-10-06T10:44:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.837+0000 7f9f606c7b80 1 -- 172.21.15.92:0/3095973555 wait complete. 2023-10-06T10:44:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.837+0000 7f9f606c7b80 1 -- start start 2023-10-06T10:44:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.837+0000 7f9f606c7b80 1 Processor -- start 2023-10-06T10:44:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.838+0000 7f9f606c7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dc9e09a0 0x5645dca08770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.838+0000 7f9f606c7b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5645dca09eb0 0x5645dca1ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.838+0000 7f9f606c7b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5645dca2ac10 0x5645dca22a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.838+0000 7f9f606c7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645dc9310b0 con 0x5645dc9e09a0 2023-10-06T10:44:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.838+0000 7f9f606c7b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645dc8dc8e0 con 0x5645dca2ac10 2023-10-06T10:44:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.838+0000 7f9f606c7b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645dc8f7ef0 con 0x5645dca09eb0 2023-10-06T10:44:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.838+0000 7f9f606c7b80 10 client.0 _ll_register_callbacks cb 0x5645dc7cdf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.838+0000 7f9f5516b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dc9e09a0 0x5645dca08770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.838+0000 7f9f5516b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dc9e09a0 0x5645dca08770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48602/0 (socket says 172.21.15.92:48602) 2023-10-06T10:44:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.838+0000 7f9f5516b700 1 -- 172.21.15.92:0/1000327011 learned_addr learned my addr 172.21.15.92:0/1000327011 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.838+0000 7f9f5596c700 1 --2- 172.21.15.92:0/1000327011 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5645dca2ac10 0x5645dca22a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.838+0000 7f9f5496a700 1 --2- 172.21.15.92:0/1000327011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5645dca09eb0 0x5645dca1ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.839+0000 7f9f5516b700 1 -- 172.21.15.92:0/1000327011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5645dca09eb0 msgr2=0x5645dca1ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.839+0000 7f9f5516b700 1 --2- 172.21.15.92:0/1000327011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5645dca09eb0 0x5645dca1ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.839+0000 7f9f5516b700 1 -- 172.21.15.92:0/1000327011 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5645dca2ac10 msgr2=0x5645dca22a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.839+0000 7f9f5516b700 1 --2- 172.21.15.92:0/1000327011 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5645dca2ac10 0x5645dca22a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.839+0000 7f9f5516b700 1 -- 172.21.15.92:0/1000327011 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5009ad30 con 0x5645dc9e09a0 2023-10-06T10:44:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.839+0000 7f9f5516b700 1 --2- 172.21.15.92:0/1000327011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dc9e09a0 0x5645dca08770 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00ef50 tx=0x7f9f4c017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee5d0c7a8493e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.839+0000 7f9f2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.839+0000 7f9f2f7fe700 1 -- 172.21.15.92:0/1000327011 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c015dc0 con 0x5645dc9e09a0 2023-10-06T10:44:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.839+0000 7f9f2f7fe700 1 -- 172.21.15.92:0/1000327011 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9f4c021420 con 0x5645dc9e09a0 2023-10-06T10:44:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.839+0000 7f9f606c7b80 1 -- 172.21.15.92:0/1000327011 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645dc9d56e0 con 0x5645dc9e09a0 2023-10-06T10:44:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.839+0000 7f9f2f7fe700 1 -- 172.21.15.92:0/1000327011 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f4c0217f0 con 0x5645dc9e09a0 2023-10-06T10:44:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.839+0000 7f9f606c7b80 10 client.8121 fetch_fsmap learned FSMap version 90 2023-10-06T10:44:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.839+0000 7f9f606c7b80 1 -- 172.21.15.92:0/1000327011 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x5645dc76e030 con 0x5645dc9e09a0 2023-10-06T10:44:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.839+0000 7f9f2f7fe700 1 -- 172.21.15.92:0/1000327011 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5645dc9d56e0 con 0x5645dc9e09a0 2023-10-06T10:44:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.839+0000 7f9f2f7fe700 1 -- 172.21.15.92:0/1000327011 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 90) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f9f4c021950 con 0x5645dc9e09a0 2023-10-06T10:44:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.839+0000 7f9f606c7b80 10 client.8121 fetch_fsmap finished waiting for FSMap version 90 2023-10-06T10:44:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.839+0000 7f9f606c7b80 10 client.8121 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.c 2023-10-06T10:44:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.839+0000 7f9f606c7b80 20 client.8121 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:16.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.839+0000 7f9f606c7b80 1 --2- 172.21.15.92:0/1000327011 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5645dca3a4c0 0x5645dca3a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.839+0000 7f9f606c7b80 4 client.8121 mds_command: new command op to 7689 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:44:16.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.839+0000 7f9f606c7b80 1 -- 172.21.15.92:0/1000327011 --> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645dc9b2610 con 0x5645dca3a4c0 2023-10-06T10:44:16.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.839+0000 7f9f5496a700 1 --2- 172.21.15.92:0/1000327011 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5645dca3a4c0 0x5645dca3a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.840+0000 7f9f2f7fe700 10 client.8121 ms_handle_connect on v2:172.21.15.92:6835/4047304116 2023-10-06T10:44:16.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.840+0000 7f9f5496a700 1 --2- 172.21.15.92:0/1000327011 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5645dca3a4c0 0x5645dca3a8c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=24531c80879ca734 server_cookie=672f59898e82f055 in_seq=0 out_seq=0 2023-10-06T10:44:16.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.840+0000 7f9f2f7fe700 1 -- 172.21.15.92:0/1000327011 <== mds.1 v2:172.21.15.92:6835/4047304116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5645dc9b2610 con 0x5645dca3a4c0 2023-10-06T10:44:16.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.840+0000 7f9f2f7fe700 10 client.8121 handle_command_reply: tid=0 2023-10-06T10:44:16.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.840+0000 7f9f606c7b80 1 client.8121 shutdown 2023-10-06T10:44:16.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.840+0000 7f9f606c7b80 1 -- 172.21.15.92:0/1000327011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dc9e09a0 msgr2=0x5645dca08770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.840+0000 7f9f606c7b80 1 --2- 172.21.15.92:0/1000327011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dc9e09a0 0x5645dca08770 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00ef50 tx=0x7f9f4c017040 comp rx=0 tx=0).stop 2023-10-06T10:44:16.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.840+0000 7f9f606c7b80 1 -- 172.21.15.92:0/1000327011 shutdown_connections 2023-10-06T10:44:16.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.840+0000 7f9f606c7b80 1 --2- 172.21.15.92:0/1000327011 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5645dca2ac10 0x5645dca22a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.840+0000 7f9f606c7b80 1 --2- 172.21.15.92:0/1000327011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dc9e09a0 0x5645dca08770 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.840+0000 7f9f606c7b80 1 --2- 172.21.15.92:0/1000327011 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5645dca3a4c0 0x5645dca3a8c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.841+0000 7f9f606c7b80 1 --2- 172.21.15.92:0/1000327011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5645dca09eb0 0x5645dca1ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.841+0000 7f9f2f7fe700 0 client.8121 ms_handle_reset on v2:172.21.15.92:6835/4047304116 2023-10-06T10:44:16.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.841+0000 7f9f606c7b80 1 -- 172.21.15.92:0/1000327011 >> 172.21.15.92:0/1000327011 conn(0x5645dc9e22c0 msgr2=0x5645dc9d71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.841+0000 7f9f606c7b80 1 -- 172.21.15.92:0/1000327011 shutdown_connections 2023-10-06T10:44:16.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.841+0000 7f9f606c7b80 1 -- 172.21.15.92:0/1000327011 wait complete. 2023-10-06T10:44:16.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.841+0000 7f9f606c7b80 20 client.8121 trim_cache size 0 max 16384 2023-10-06T10:44:16.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.856+0000 7f9f606c7b80 1 Processor -- start 2023-10-06T10:44:16.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.856+0000 7f9f606c7b80 1 -- start start 2023-10-06T10:44:16.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.857+0000 7f9f606c7b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5645dc9e09a0 0x5645dca408c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.857+0000 7f9f606c7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dca09eb0 0x5645dca40e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.857+0000 7f9f606c7b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5645dca2ac10 0x5645dca2afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.857+0000 7f9f606c7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645dc9dfc90 con 0x5645dca09eb0 2023-10-06T10:44:16.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.857+0000 7f9f606c7b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645dc7b3e80 con 0x5645dc9e09a0 2023-10-06T10:44:16.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.857+0000 7f9f606c7b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645dca069d0 con 0x5645dca2ac10 2023-10-06T10:44:16.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.857+0000 7f9f5496a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dca09eb0 0x5645dca40e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.857+0000 7f9f5496a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dca09eb0 0x5645dca40e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48618/0 (socket says 172.21.15.92:48618) 2023-10-06T10:44:16.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.857+0000 7f9f5496a700 1 -- 172.21.15.92:0/1865250693 learned_addr learned my addr 172.21.15.92:0/1865250693 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.858+0000 7f9f5596c700 1 --2- 172.21.15.92:0/1865250693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5645dca2ac10 0x5645dca2afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.858+0000 7f9f5516b700 1 --2- 172.21.15.92:0/1865250693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5645dc9e09a0 0x5645dca408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.858+0000 7f9f5496a700 1 -- 172.21.15.92:0/1865250693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5645dca2ac10 msgr2=0x5645dca2afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.858+0000 7f9f5496a700 1 --2- 172.21.15.92:0/1865250693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5645dca2ac10 0x5645dca2afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.858+0000 7f9f5496a700 1 -- 172.21.15.92:0/1865250693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5645dc9e09a0 msgr2=0x5645dca408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.858+0000 7f9f5496a700 1 --2- 172.21.15.92:0/1865250693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5645dc9e09a0 0x5645dca408c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.858+0000 7f9f5496a700 1 -- 172.21.15.92:0/1865250693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645dc8ff7d0 con 0x5645dca09eb0 2023-10-06T10:44:16.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.858+0000 7f9f5496a700 1 --2- 172.21.15.92:0/1865250693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dca09eb0 0x5645dca40e00 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f9f40006fd0 tx=0x7f9f4001d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff72ca8994e7ad4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.858+0000 7f9f3affd700 1 -- 172.21.15.92:0/1865250693 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4002c070 con 0x5645dca09eb0 2023-10-06T10:44:16.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.858+0000 7f9f3affd700 1 -- 172.21.15.92:0/1865250693 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f40022040 con 0x5645dca09eb0 2023-10-06T10:44:16.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.858+0000 7f9f3affd700 1 -- 172.21.15.92:0/1865250693 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40021040 con 0x5645dca09eb0 2023-10-06T10:44:16.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.858+0000 7f9f606c7b80 1 -- 172.21.15.92:0/1865250693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dca09eb0 msgr2=0x5645dca40e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.858+0000 7f9f606c7b80 1 --2- 172.21.15.92:0/1865250693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dca09eb0 0x5645dca40e00 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f9f40006fd0 tx=0x7f9f4001d040 comp rx=0 tx=0).stop 2023-10-06T10:44:16.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.858+0000 7f9f606c7b80 1 -- 172.21.15.92:0/1865250693 shutdown_connections 2023-10-06T10:44:16.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.858+0000 7f9f606c7b80 1 --2- 172.21.15.92:0/1865250693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5645dca2ac10 0x5645dca2afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.858+0000 7f9f606c7b80 1 --2- 172.21.15.92:0/1865250693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5645dc9e09a0 0x5645dca408c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.858+0000 7f9f606c7b80 1 --2- 172.21.15.92:0/1865250693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dca09eb0 0x5645dca40e00 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.858+0000 7f9f606c7b80 1 -- 172.21.15.92:0/1865250693 >> 172.21.15.92:0/1865250693 conn(0x5645dc9e22c0 msgr2=0x5645dca06b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.858+0000 7f9f606c7b80 1 -- 172.21.15.92:0/1865250693 shutdown_connections 2023-10-06T10:44:16.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.859+0000 7f9f606c7b80 1 -- 172.21.15.92:0/1865250693 wait complete. 2023-10-06T10:44:16.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.859+0000 7f9f606c7b80 1 -- start start 2023-10-06T10:44:16.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.859+0000 7f9f606c7b80 1 Processor -- start 2023-10-06T10:44:16.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.860+0000 7f9f606c7b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5645dc9e09a0 0x5645dca39530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.860+0000 7f9f606c7b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5645dca09eb0 0x5645dca90980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.860+0000 7f9f606c7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dca2ac10 0x5645dca90ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.860+0000 7f9f606c7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645dc9d74d0 con 0x5645dca2ac10 2023-10-06T10:44:16.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.860+0000 7f9f606c7b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645dca8d6b0 con 0x5645dca09eb0 2023-10-06T10:44:16.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.860+0000 7f9f606c7b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645dc9d7280 con 0x5645dc9e09a0 2023-10-06T10:44:16.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.860+0000 7f9f606c7b80 10 client.0 _ll_register_callbacks cb 0x5645dc7cdf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:16.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.860+0000 7f9f5596c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dca2ac10 0x5645dca90ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.860+0000 7f9f5596c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dca2ac10 0x5645dca90ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48626/0 (socket says 172.21.15.92:48626) 2023-10-06T10:44:16.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.860+0000 7f9f5596c700 1 -- 172.21.15.92:0/2845720947 learned_addr learned my addr 172.21.15.92:0/2845720947 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.860+0000 7f9f5496a700 1 --2- 172.21.15.92:0/2845720947 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5645dca09eb0 0x5645dca90980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.860+0000 7f9f5516b700 1 --2- 172.21.15.92:0/2845720947 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5645dc9e09a0 0x5645dca39530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.860+0000 7f9f5596c700 1 -- 172.21.15.92:0/2845720947 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5645dc9e09a0 msgr2=0x5645dca39530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.860+0000 7f9f5596c700 1 --2- 172.21.15.92:0/2845720947 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5645dc9e09a0 0x5645dca39530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.860+0000 7f9f5596c700 1 -- 172.21.15.92:0/2845720947 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5645dca09eb0 msgr2=0x5645dca90980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.860+0000 7f9f5596c700 1 --2- 172.21.15.92:0/2845720947 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5645dca09eb0 0x5645dca90980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.860+0000 7f9f5596c700 1 -- 172.21.15.92:0/2845720947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f500cc040 con 0x5645dca2ac10 2023-10-06T10:44:16.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.861+0000 7f9f5596c700 1 --2- 172.21.15.92:0/2845720947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dca2ac10 0x5645dca90ec0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f9f48055be0 tx=0x7f9f4806e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63dab987ca60fd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.861+0000 7f9f2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:16.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.861+0000 7f9f2f7fe700 1 -- 172.21.15.92:0/2845720947 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48055c50 con 0x5645dca2ac10 2023-10-06T10:44:16.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.861+0000 7f9f2f7fe700 1 -- 172.21.15.92:0/2845720947 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9f4805ce40 con 0x5645dca2ac10 2023-10-06T10:44:16.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.861+0000 7f9f606c7b80 1 -- 172.21.15.92:0/2845720947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f48058ea0 con 0x5645dca2ac10 2023-10-06T10:44:16.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.861+0000 7f9f2f7fe700 1 -- 172.21.15.92:0/2845720947 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f4805cbf0 con 0x5645dca2ac10 2023-10-06T10:44:16.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.861+0000 7f9f606c7b80 10 client.8127 fetch_fsmap learned FSMap version 90 2023-10-06T10:44:16.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.861+0000 7f9f606c7b80 1 -- 172.21.15.92:0/2845720947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x5645dc8e64c0 con 0x5645dca2ac10 2023-10-06T10:44:16.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.861+0000 7f9f2f7fe700 1 -- 172.21.15.92:0/2845720947 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f48058ea0 con 0x5645dca2ac10 2023-10-06T10:44:16.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.861+0000 7f9f2f7fe700 1 -- 172.21.15.92:0/2845720947 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 90) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f9f48055140 con 0x5645dca2ac10 2023-10-06T10:44:16.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.861+0000 7f9f606c7b80 10 client.8127 fetch_fsmap finished waiting for FSMap version 90 2023-10-06T10:44:16.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.861+0000 7f9f606c7b80 10 client.8127 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.c 2023-10-06T10:44:16.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.861+0000 7f9f606c7b80 20 client.8127 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:16.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.861+0000 7f9f606c7b80 1 --2- 172.21.15.92:0/2845720947 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5645dca08cf0 0x5645dca8c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.861+0000 7f9f606c7b80 4 client.8127 mds_command: new command op to 7689 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:44:16.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.861+0000 7f9f606c7b80 1 -- 172.21.15.92:0/2845720947 --> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5645dc8d3e50 con 0x5645dca08cf0 2023-10-06T10:44:16.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.861+0000 7f9f5516b700 1 --2- 172.21.15.92:0/2845720947 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5645dca08cf0 0x5645dca8c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.861+0000 7f9f2f7fe700 10 client.8127 ms_handle_connect on v2:172.21.15.92:6835/4047304116 2023-10-06T10:44:16.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.861+0000 7f9f5516b700 1 --2- 172.21.15.92:0/2845720947 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5645dca08cf0 0x5645dca8c3e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1484781db7f46ae server_cookie=18588bdd592b2127 in_seq=0 out_seq=0 2023-10-06T10:44:16.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.862+0000 7f9f2f7fe700 1 -- 172.21.15.92:0/2845720947 <== mds.1 v2:172.21.15.92:6835/4047304116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5645dc8d3e50 con 0x5645dca08cf0 2023-10-06T10:44:16.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.862+0000 7f9f2f7fe700 10 client.8127 handle_command_reply: tid=0 2023-10-06T10:44:16.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.862+0000 7f9f606c7b80 1 client.8127 shutdown 2023-10-06T10:44:16.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.862+0000 7f9f606c7b80 1 -- 172.21.15.92:0/2845720947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dca2ac10 msgr2=0x5645dca90ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.862+0000 7f9f606c7b80 1 --2- 172.21.15.92:0/2845720947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dca2ac10 0x5645dca90ec0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f9f48055be0 tx=0x7f9f4806e000 comp rx=0 tx=0).stop 2023-10-06T10:44:16.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.862+0000 7f9f606c7b80 1 -- 172.21.15.92:0/2845720947 shutdown_connections 2023-10-06T10:44:16.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.862+0000 7f9f606c7b80 1 --2- 172.21.15.92:0/2845720947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645dca2ac10 0x5645dca90ec0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.862+0000 7f9f606c7b80 1 --2- 172.21.15.92:0/2845720947 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5645dca08cf0 0x5645dca8c3e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.862+0000 7f9f606c7b80 1 --2- 172.21.15.92:0/2845720947 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5645dc9e09a0 0x5645dca39530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.862+0000 7f9f2f7fe700 0 client.8127 ms_handle_reset on v2:172.21.15.92:6835/4047304116 2023-10-06T10:44:16.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.862+0000 7f9f606c7b80 1 --2- 172.21.15.92:0/2845720947 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5645dca09eb0 0x5645dca90980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.862+0000 7f9f606c7b80 1 -- 172.21.15.92:0/2845720947 >> 172.21.15.92:0/2845720947 conn(0x5645dc9e22c0 msgr2=0x5645dca408c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.862+0000 7f9f606c7b80 1 -- 172.21.15.92:0/2845720947 shutdown_connections 2023-10-06T10:44:16.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.862+0000 7f9f606c7b80 1 -- 172.21.15.92:0/2845720947 wait complete. 2023-10-06T10:44:16.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.862+0000 7f9f606c7b80 20 client.8127 trim_cache size 0 max 16384 2023-10-06T10:44:16.871 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:44:16.871 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "PAUSED (0 inodes in the stack)", 2023-10-06T10:44:16.871 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:44:16.871 INFO:teuthology.orchestra.run.smithi092.stdout: "9669003b-2b0f-4975-abcd-f07cf375c689": { 2023-10-06T10:44:16.871 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:44:16.872 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "9669003b-2b0f-4975-abcd-f07cf375c689", 2023-10-06T10:44:16.872 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:44:16.872 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:16.872 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:16.872 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:44:16.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.862+0000 7f9f5616d700 1 -- 172.21.15.92:0/920445980 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f34066f30 msgr2=0x7f9f340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.862+0000 7f9f5616d700 1 --2- 172.21.15.92:0/920445980 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f34066f30 0x7f9f340693e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c006aa0 tx=0x7f9f4c008040 comp rx=0 tx=0).stop 2023-10-06T10:44:16.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.862+0000 7f9f5616d700 1 -- 172.21.15.92:0/920445980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f500ceaa0 msgr2=0x7f9f5009d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.862+0000 7f9f5616d700 1 --2- 172.21.15.92:0/920445980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f500ceaa0 0x7f9f5009d880 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009fa0 tx=0x7f9f48009900 comp rx=0 tx=0).stop 2023-10-06T10:44:16.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.863+0000 7f9f5616d700 1 -- 172.21.15.92:0/920445980 shutdown_connections 2023-10-06T10:44:16.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.863+0000 7f9f5616d700 1 --2- 172.21.15.92:0/920445980 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f34066f30 0x7f9f340693e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.863+0000 7f9f5616d700 1 --2- 172.21.15.92:0/920445980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f500ceaa0 0x7f9f5009d880 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.863+0000 7f9f5616d700 1 --2- 172.21.15.92:0/920445980 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f500b2620 0x7f9f5016a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.863+0000 7f9f5616d700 1 --2- 172.21.15.92:0/920445980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f500b29f0 0x7f9f50099f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.863+0000 7f9f5616d700 1 -- 172.21.15.92:0/920445980 >> 172.21.15.92:0/920445980 conn(0x7f9f50099520 msgr2=0x7f9f500c62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.863+0000 7f9f5616d700 1 -- 172.21.15.92:0/920445980 shutdown_connections 2023-10-06T10:44:16.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.863+0000 7f9f5616d700 1 -- 172.21.15.92:0/920445980 wait complete. 2023-10-06T10:44:16.876 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:44:16.876 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:2 scrub status 2023-10-06T10:44:16.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.989+0000 7fde8623c700 1 Processor -- start 2023-10-06T10:44:16.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.989+0000 7fde8623c700 1 -- start start 2023-10-06T10:44:16.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.990+0000 7fde8623c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde80153990 0x7fde8015a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.990+0000 7fde8623c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fde800cca70 0x7fde800d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:16.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.990+0000 7fde8623c700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde800d0c50 con 0x7fde800d07f0 2023-10-06T10:44:16.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.990+0000 7fde8623c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde800cdbf0 con 0x7fde800cca70 2023-10-06T10:44:16.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.990+0000 7fde8623c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde800cb4c0 con 0x7fde80153990 2023-10-06T10:44:16.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.990+0000 7fde84a39700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fde800cca70 0x7fde800d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.990+0000 7fde84a39700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fde800cca70 0x7fde800d02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48634/0 (socket says 172.21.15.92:48634) 2023-10-06T10:44:16.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.990+0000 7fde84a39700 1 -- 172.21.15.92:0/795492878 learned_addr learned my addr 172.21.15.92:0/795492878 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:16.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.990+0000 7fde8523a700 1 --2- 172.21.15.92:0/795492878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde80153990 0x7fde8015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:16.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.990+0000 7fde84a39700 1 -- 172.21.15.92:0/795492878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde80153990 msgr2=0x7fde8015a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.990+0000 7fde84a39700 1 --2- 172.21.15.92:0/795492878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde80153990 0x7fde8015a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.990+0000 7fde84a39700 1 -- 172.21.15.92:0/795492878 >> v1:172.21.15.124:6789/0 conn(0x7fde800d07f0 legacy=0x7fde8015cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:16.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.990+0000 7fde84a39700 1 -- 172.21.15.92:0/795492878 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8015a8b0 con 0x7fde800cca70 2023-10-06T10:44:16.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.991+0000 7fde8523a700 1 --2- 172.21.15.92:0/795492878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde80153990 0x7fde8015a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:16.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.991+0000 7fde84a39700 1 --2- 172.21.15.92:0/795492878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde800cca70 0x7fde800d02b0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fde740099b0 tx=0x7fde74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15210261fffd63e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:16.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.991+0000 7fde7ffff700 1 -- 172.21.15.92:0/795492878 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7400b820 con 0x7fde800cca70 2023-10-06T10:44:16.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.991+0000 7fde7ffff700 1 -- 172.21.15.92:0/795492878 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7400b980 con 0x7fde800cca70 2023-10-06T10:44:16.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.991+0000 7fde7ffff700 1 -- 172.21.15.92:0/795492878 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7400bd20 con 0x7fde800cca70 2023-10-06T10:44:16.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.991+0000 7fde8623c700 1 -- 172.21.15.92:0/795492878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde800cca70 msgr2=0x7fde800d02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:16.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.991+0000 7fde8623c700 1 --2- 172.21.15.92:0/795492878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde800cca70 0x7fde800d02b0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fde740099b0 tx=0x7fde74009250 comp rx=0 tx=0).stop 2023-10-06T10:44:16.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.991+0000 7fde8623c700 1 -- 172.21.15.92:0/795492878 shutdown_connections 2023-10-06T10:44:16.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.991+0000 7fde8623c700 1 --2- 172.21.15.92:0/795492878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde80153990 0x7fde8015a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.991+0000 7fde8623c700 1 --2- 172.21.15.92:0/795492878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde800cca70 0x7fde800d02b0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:16.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.991+0000 7fde8623c700 1 -- 172.21.15.92:0/795492878 >> 172.21.15.92:0/795492878 conn(0x7fde800680c0 msgr2=0x7fde80068490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:16.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.991+0000 7fde8623c700 1 -- 172.21.15.92:0/795492878 shutdown_connections 2023-10-06T10:44:16.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.991+0000 7fde8623c700 1 -- 172.21.15.92:0/795492878 wait complete. 2023-10-06T10:44:16.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.992+0000 7fde8623c700 1 Processor -- start 2023-10-06T10:44:16.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.993+0000 7fde8623c700 1 -- start start 2023-10-06T10:44:17.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.993+0000 7fde8623c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde800cca70 0x7fde800c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.993+0000 7fde8623c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde800d07f0 0x7fde800b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.993+0000 7fde8623c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde80153990 0x7fde800c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.993+0000 7fde8623c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde800b4f30 con 0x7fde800cca70 2023-10-06T10:44:17.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.993+0000 7fde8623c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde800b4320 con 0x7fde800d07f0 2023-10-06T10:44:17.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.993+0000 7fde8623c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde800b4600 con 0x7fde80153990 2023-10-06T10:44:17.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.994+0000 7fde8523a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde800cca70 0x7fde800c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.994+0000 7fde8523a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde800cca70 0x7fde800c7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48648/0 (socket says 172.21.15.92:48648) 2023-10-06T10:44:17.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.994+0000 7fde8523a700 1 -- 172.21.15.92:0/3975877704 learned_addr learned my addr 172.21.15.92:0/3975877704 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:17.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.994+0000 7fde84a39700 1 --2- 172.21.15.92:0/3975877704 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde800d07f0 0x7fde800b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.994+0000 7fde85a3b700 1 --2- 172.21.15.92:0/3975877704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde80153990 0x7fde800c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.994+0000 7fde8523a700 1 -- 172.21.15.92:0/3975877704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde80153990 msgr2=0x7fde800c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.994+0000 7fde8523a700 1 --2- 172.21.15.92:0/3975877704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde80153990 0x7fde800c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.994+0000 7fde8523a700 1 -- 172.21.15.92:0/3975877704 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde800d07f0 msgr2=0x7fde800b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.994+0000 7fde8523a700 1 --2- 172.21.15.92:0/3975877704 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde800d07f0 0x7fde800b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.994+0000 7fde8523a700 1 -- 172.21.15.92:0/3975877704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde740094f0 con 0x7fde800cca70 2023-10-06T10:44:17.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.994+0000 7fde85a3b700 1 --2- 172.21.15.92:0/3975877704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde80153990 0x7fde800c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:17.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.994+0000 7fde8523a700 1 --2- 172.21.15.92:0/3975877704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde800cca70 0x7fde800c7810 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fde7000cfb0 tx=0x7fde70007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:17.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.994+0000 7fde7d7fa700 1 -- 172.21.15.92:0/3975877704 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7000ae30 con 0x7fde800cca70 2023-10-06T10:44:17.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.994+0000 7fde7d7fa700 1 -- 172.21.15.92:0/3975877704 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde70016420 con 0x7fde800cca70 2023-10-06T10:44:17.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.994+0000 7fde8623c700 1 -- 172.21.15.92:0/3975877704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde800b48e0 con 0x7fde800cca70 2023-10-06T10:44:17.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.994+0000 7fde7d7fa700 1 -- 172.21.15.92:0/3975877704 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde700167e0 con 0x7fde800cca70 2023-10-06T10:44:17.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.994+0000 7fde8623c700 1 -- 172.21.15.92:0/3975877704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde800b3fb0 con 0x7fde800cca70 2023-10-06T10:44:17.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.995+0000 7fde7d7fa700 1 -- 172.21.15.92:0/3975877704 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fde70016bd0 con 0x7fde800cca70 2023-10-06T10:44:17.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.996+0000 7fde7d7fa700 1 --2- 172.21.15.92:0/3975877704 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fde6c066f30 0x7fde6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.996+0000 7fde7d7fa700 1 -- 172.21.15.92:0/3975877704 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fde70012070 con 0x7fde800cca70 2023-10-06T10:44:17.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.996+0000 7fde84a39700 1 --2- 172.21.15.92:0/3975877704 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fde6c066f30 0x7fde6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.996+0000 7fde84a39700 1 --2- 172.21.15.92:0/3975877704 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fde6c066f30 0x7fde6c0693e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fde740099b0 tx=0x7fde74016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:17.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.998+0000 7fde90796b80 1 Processor -- start 2023-10-06T10:44:17.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.998+0000 7fde90796b80 1 -- start start 2023-10-06T10:44:17.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.999+0000 7fde90796b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8f8f9a0 0x562db8f8fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.999+0000 7fde90796b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8fd9c10 0x562db8fb8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.999+0000 7fde90796b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562db8fb8eb0 0x562db8f8b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.999+0000 7fde90796b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db8f0fcd0 con 0x562db8fd9c10 2023-10-06T10:44:17.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.999+0000 7fde90796b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db8ee6ac0 con 0x562db8f8f9a0 2023-10-06T10:44:17.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.999+0000 7fde90796b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562db8ee42c0 con 0x562db8fb8eb0 2023-10-06T10:44:17.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.999+0000 7fde84a39700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8fd9c10 0x562db8fb8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.999+0000 7fde84a39700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8fd9c10 0x562db8fb8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48662/0 (socket says 172.21.15.92:48662) 2023-10-06T10:44:17.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.999+0000 7fde84a39700 1 -- 172.21.15.92:0/764416136 learned_addr learned my addr 172.21.15.92:0/764416136 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:17.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.999+0000 7fde8523a700 1 --2- 172.21.15.92:0/764416136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8f8f9a0 0x562db8f8fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.999+0000 7fde85a3b700 1 --2- 172.21.15.92:0/764416136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562db8fb8eb0 0x562db8f8b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.999+0000 7fde84a39700 1 -- 172.21.15.92:0/764416136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562db8fb8eb0 msgr2=0x562db8f8b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:17.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.999+0000 7fde84a39700 1 --2- 172.21.15.92:0/764416136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562db8fb8eb0 0x562db8f8b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.999+0000 7fde84a39700 1 -- 172.21.15.92:0/764416136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8f8f9a0 msgr2=0x562db8f8fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:17.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.999+0000 7fde84a39700 1 --2- 172.21.15.92:0/764416136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8f8f9a0 0x562db8f8fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.999+0000 7fde84a39700 1 -- 172.21.15.92:0/764416136 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562db8d7a2c0 con 0x562db8fd9c10 2023-10-06T10:44:17.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.999+0000 7fde8523a700 1 --2- 172.21.15.92:0/764416136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8f8f9a0 0x562db8f8fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:17.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.999+0000 7fde85a3b700 1 --2- 172.21.15.92:0/764416136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562db8fb8eb0 0x562db8f8b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:17.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.999+0000 7fde84a39700 1 --2- 172.21.15.92:0/764416136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8fd9c10 0x562db8fb8970 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fde74006e80 tx=0x7fde74021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0d32725ca2919e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:17.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.999+0000 7fde667fc700 1 -- 172.21.15.92:0/764416136 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde74021070 con 0x562db8fd9c10 2023-10-06T10:44:17.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.999+0000 7fde667fc700 1 -- 172.21.15.92:0/764416136 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7400beb0 con 0x562db8fd9c10 2023-10-06T10:44:17.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:16.999+0000 7fde667fc700 1 -- 172.21.15.92:0/764416136 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde74011a90 con 0x562db8fd9c10 2023-10-06T10:44:17.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.000+0000 7fde90796b80 1 -- 172.21.15.92:0/764416136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8fd9c10 msgr2=0x562db8fb8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.000+0000 7fde90796b80 1 --2- 172.21.15.92:0/764416136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8fd9c10 0x562db8fb8970 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fde74006e80 tx=0x7fde74021000 comp rx=0 tx=0).stop 2023-10-06T10:44:17.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.000+0000 7fde90796b80 1 -- 172.21.15.92:0/764416136 shutdown_connections 2023-10-06T10:44:17.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.000+0000 7fde90796b80 1 --2- 172.21.15.92:0/764416136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562db8fb8eb0 0x562db8f8b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.000+0000 7fde90796b80 1 --2- 172.21.15.92:0/764416136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8f8f9a0 0x562db8f8fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.000+0000 7fde90796b80 1 --2- 172.21.15.92:0/764416136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8fd9c10 0x562db8fb8970 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.000+0000 7fde90796b80 1 -- 172.21.15.92:0/764416136 >> 172.21.15.92:0/764416136 conn(0x562db8f912c0 msgr2=0x562db8f91690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:17.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.000+0000 7fde90796b80 1 -- 172.21.15.92:0/764416136 shutdown_connections 2023-10-06T10:44:17.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.000+0000 7fde90796b80 1 -- 172.21.15.92:0/764416136 wait complete. 2023-10-06T10:44:17.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.000+0000 7fde90796b80 1 -- start start 2023-10-06T10:44:17.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.001+0000 7fde90796b80 1 Processor -- start 2023-10-06T10:44:17.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.001+0000 7fde90796b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8f8f9a0 0x562db8fe6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.001+0000 7fde90796b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562db8fb8eb0 0x562db8fe9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.001+0000 7fde90796b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8fd9c10 0x562db8fcf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.001+0000 7fde90796b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db8ee00b0 con 0x562db8fd9c10 2023-10-06T10:44:17.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.001+0000 7fde90796b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db8e8b8e0 con 0x562db8f8f9a0 2023-10-06T10:44:17.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.001+0000 7fde90796b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562db8ea6ef0 con 0x562db8fb8eb0 2023-10-06T10:44:17.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.001+0000 7fde90796b80 10 client.0 _ll_register_callbacks cb 0x562db8d7cf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:17.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde85a3b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8fd9c10 0x562db8fcf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde85a3b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8fd9c10 0x562db8fcf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48666/0 (socket says 172.21.15.92:48666) 2023-10-06T10:44:17.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde85a3b700 1 -- 172.21.15.92:0/4102752812 learned_addr learned my addr 172.21.15.92:0/4102752812 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:17.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde84a39700 1 --2- 172.21.15.92:0/4102752812 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562db8fb8eb0 0x562db8fe9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde8523a700 1 --2- 172.21.15.92:0/4102752812 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8f8f9a0 0x562db8fe6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde85a3b700 1 -- 172.21.15.92:0/4102752812 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562db8fb8eb0 msgr2=0x562db8fe9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:17.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde85a3b700 1 --2- 172.21.15.92:0/4102752812 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562db8fb8eb0 0x562db8fe9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde85a3b700 1 -- 172.21.15.92:0/4102752812 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8f8f9a0 msgr2=0x562db8fe6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:17.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde85a3b700 1 --2- 172.21.15.92:0/4102752812 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8f8f9a0 0x562db8fe6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde85a3b700 1 -- 172.21.15.92:0/4102752812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde8015a8b0 con 0x562db8fd9c10 2023-10-06T10:44:17.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde84a39700 1 --2- 172.21.15.92:0/4102752812 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562db8fb8eb0 0x562db8fe9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:17.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde8523a700 1 --2- 172.21.15.92:0/4102752812 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8f8f9a0 0x562db8fe6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:17.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde85a3b700 1 --2- 172.21.15.92:0/4102752812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8fd9c10 0x562db8fcf5e0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fde7800f390 tx=0x7fde78008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b079065ef466f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:17.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde5f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:17.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde5f7fe700 1 -- 172.21.15.92:0/4102752812 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde78013400 con 0x562db8fd9c10 2023-10-06T10:44:17.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde5f7fe700 1 -- 172.21.15.92:0/4102752812 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fde78013560 con 0x562db8fd9c10 2023-10-06T10:44:17.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde90796b80 1 -- 172.21.15.92:0/4102752812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562db8f846e0 con 0x562db8fd9c10 2023-10-06T10:44:17.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde5f7fe700 1 -- 172.21.15.92:0/4102752812 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde7800b5e0 con 0x562db8fd9c10 2023-10-06T10:44:17.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde90796b80 10 client.8139 fetch_fsmap learned FSMap version 90 2023-10-06T10:44:17.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde90796b80 1 -- 172.21.15.92:0/4102752812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x562db8d1d030 con 0x562db8fd9c10 2023-10-06T10:44:17.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde5f7fe700 1 -- 172.21.15.92:0/4102752812 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562db8f846e0 con 0x562db8fd9c10 2023-10-06T10:44:17.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde5f7fe700 1 -- 172.21.15.92:0/4102752812 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 90) v1 ==== 3136+0+0 (secure 0 0 0) 0x7fde78013b40 con 0x562db8fd9c10 2023-10-06T10:44:17.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde90796b80 10 client.8139 fetch_fsmap finished waiting for FSMap version 90 2023-10-06T10:44:17.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.002+0000 7fde90796b80 10 client.8139 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.a 2023-10-06T10:44:17.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.003+0000 7fde90796b80 20 client.8139 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:17.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.003+0000 7fde90796b80 1 --2- 172.21.15.92:0/4102752812 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x562db8fcfb20 0x562db8f861f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.003+0000 7fde90796b80 4 client.8139 mds_command: new command op to 7686 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:44:17.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.003+0000 7fde90796b80 1 -- 172.21.15.92:0/4102752812 --> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562db8f61610 con 0x562db8fcfb20 2023-10-06T10:44:17.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.003+0000 7fde8523a700 1 --2- 172.21.15.92:0/4102752812 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x562db8fcfb20 0x562db8f861f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.003+0000 7fde5f7fe700 10 client.8139 ms_handle_connect on v2:172.21.15.92:6834/3605297930 2023-10-06T10:44:17.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.003+0000 7fde8523a700 1 --2- 172.21.15.92:0/4102752812 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x562db8fcfb20 0x562db8f861f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb273c0c2d148e47 server_cookie=42ac73ed052d147f in_seq=0 out_seq=0 2023-10-06T10:44:17.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.003+0000 7fde5f7fe700 1 -- 172.21.15.92:0/4102752812 <== mds.2 v2:172.21.15.92:6834/3605297930 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x562db8f61610 con 0x562db8fcfb20 2023-10-06T10:44:17.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.003+0000 7fde5f7fe700 10 client.8139 handle_command_reply: tid=0 2023-10-06T10:44:17.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.003+0000 7fde90796b80 1 client.8139 shutdown 2023-10-06T10:44:17.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.004+0000 7fde90796b80 1 -- 172.21.15.92:0/4102752812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8fd9c10 msgr2=0x562db8fcf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.004+0000 7fde90796b80 1 --2- 172.21.15.92:0/4102752812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8fd9c10 0x562db8fcf5e0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fde7800f390 tx=0x7fde78008040 comp rx=0 tx=0).stop 2023-10-06T10:44:17.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.004+0000 7fde90796b80 1 -- 172.21.15.92:0/4102752812 shutdown_connections 2023-10-06T10:44:17.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.004+0000 7fde90796b80 1 --2- 172.21.15.92:0/4102752812 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x562db8fcfb20 0x562db8f861f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.004+0000 7fde90796b80 1 --2- 172.21.15.92:0/4102752812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8fd9c10 0x562db8fcf5e0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.004+0000 7fde90796b80 1 --2- 172.21.15.92:0/4102752812 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8f8f9a0 0x562db8fe6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.004+0000 7fde90796b80 1 --2- 172.21.15.92:0/4102752812 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562db8fb8eb0 0x562db8fe9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.004+0000 7fde5f7fe700 0 client.8139 ms_handle_reset on v2:172.21.15.92:6834/3605297930 2023-10-06T10:44:17.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.004+0000 7fde90796b80 1 -- 172.21.15.92:0/4102752812 >> 172.21.15.92:0/4102752812 conn(0x562db8f912c0 msgr2=0x562db8fb5720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:17.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.004+0000 7fde90796b80 1 -- 172.21.15.92:0/4102752812 shutdown_connections 2023-10-06T10:44:17.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.004+0000 7fde90796b80 1 -- 172.21.15.92:0/4102752812 wait complete. 2023-10-06T10:44:17.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.004+0000 7fde90796b80 20 client.8139 trim_cache size 0 max 16384 2023-10-06T10:44:17.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.019+0000 7fde90796b80 1 Processor -- start 2023-10-06T10:44:17.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.020+0000 7fde90796b80 1 -- start start 2023-10-06T10:44:17.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.020+0000 7fde90796b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8f8f9a0 0x562db8fe5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.020+0000 7fde90796b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8fb8eb0 0x562db8f91690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.020+0000 7fde90796b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562db8fd9c10 0x562db8fd0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.020+0000 7fde90796b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db8f8ec90 con 0x562db8fb8eb0 2023-10-06T10:44:17.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.020+0000 7fde90796b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db8d62e80 con 0x562db8f8f9a0 2023-10-06T10:44:17.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.020+0000 7fde90796b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562db8fe8aa0 con 0x562db8fd9c10 2023-10-06T10:44:17.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.020+0000 7fde84a39700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8fb8eb0 0x562db8f91690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.020+0000 7fde84a39700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8fb8eb0 0x562db8f91690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48676/0 (socket says 172.21.15.92:48676) 2023-10-06T10:44:17.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.020+0000 7fde84a39700 1 -- 172.21.15.92:0/2809238114 learned_addr learned my addr 172.21.15.92:0/2809238114 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:17.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.020+0000 7fde85a3b700 1 --2- 172.21.15.92:0/2809238114 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562db8fd9c10 0x562db8fd0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.020+0000 7fde8523a700 1 --2- 172.21.15.92:0/2809238114 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8f8f9a0 0x562db8fe5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.021+0000 7fde84a39700 1 -- 172.21.15.92:0/2809238114 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562db8fd9c10 msgr2=0x562db8fd0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:17.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.021+0000 7fde84a39700 1 --2- 172.21.15.92:0/2809238114 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562db8fd9c10 0x562db8fd0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.021+0000 7fde84a39700 1 -- 172.21.15.92:0/2809238114 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8f8f9a0 msgr2=0x562db8fe5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:17.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.021+0000 7fde84a39700 1 --2- 172.21.15.92:0/2809238114 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8f8f9a0 0x562db8fe5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.021+0000 7fde84a39700 1 -- 172.21.15.92:0/2809238114 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562db8eae7d0 con 0x562db8fb8eb0 2023-10-06T10:44:17.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.021+0000 7fde8523a700 1 --2- 172.21.15.92:0/2809238114 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8f8f9a0 0x562db8fe5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:17.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.021+0000 7fde84a39700 1 --2- 172.21.15.92:0/2809238114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8fb8eb0 0x562db8f91690 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fde740090b0 tx=0x7fde7402c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ee37f2337dfd31 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:17.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.021+0000 7fde66ffd700 1 -- 172.21.15.92:0/2809238114 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde74038070 con 0x562db8fb8eb0 2023-10-06T10:44:17.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.021+0000 7fde66ffd700 1 -- 172.21.15.92:0/2809238114 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde74011e40 con 0x562db8fb8eb0 2023-10-06T10:44:17.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.021+0000 7fde66ffd700 1 -- 172.21.15.92:0/2809238114 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde74021040 con 0x562db8fb8eb0 2023-10-06T10:44:17.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.021+0000 7fde90796b80 1 -- 172.21.15.92:0/2809238114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8fb8eb0 msgr2=0x562db8f91690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.021+0000 7fde90796b80 1 --2- 172.21.15.92:0/2809238114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8fb8eb0 0x562db8f91690 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fde740090b0 tx=0x7fde7402c040 comp rx=0 tx=0).stop 2023-10-06T10:44:17.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.021+0000 7fde90796b80 1 -- 172.21.15.92:0/2809238114 shutdown_connections 2023-10-06T10:44:17.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.021+0000 7fde90796b80 1 --2- 172.21.15.92:0/2809238114 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562db8fd9c10 0x562db8fd0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.021+0000 7fde90796b80 1 --2- 172.21.15.92:0/2809238114 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8f8f9a0 0x562db8fe5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.021+0000 7fde90796b80 1 --2- 172.21.15.92:0/2809238114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8fb8eb0 0x562db8f91690 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.021+0000 7fde90796b80 1 -- 172.21.15.92:0/2809238114 >> 172.21.15.92:0/2809238114 conn(0x562db8f912c0 msgr2=0x562db8fd2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:17.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.022+0000 7fde90796b80 1 -- 172.21.15.92:0/2809238114 shutdown_connections 2023-10-06T10:44:17.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.022+0000 7fde90796b80 1 -- 172.21.15.92:0/2809238114 wait complete. 2023-10-06T10:44:17.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.022+0000 7fde90796b80 1 -- start start 2023-10-06T10:44:17.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.022+0000 7fde90796b80 1 Processor -- start 2023-10-06T10:44:17.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.023+0000 7fde90796b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8f8f9a0 0x562db8fd0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.023+0000 7fde90796b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562db8fb8eb0 0x562db8fd06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.023+0000 7fde90796b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8fd9c10 0x562db8fed880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.023+0000 7fde90796b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db8fcd480 con 0x562db8f8f9a0 2023-10-06T10:44:17.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.023+0000 7fde90796b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db8feeab0 con 0x562db8fd9c10 2023-10-06T10:44:17.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.023+0000 7fde90796b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562db8fd5e70 con 0x562db8fb8eb0 2023-10-06T10:44:17.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.023+0000 7fde90796b80 10 client.0 _ll_register_callbacks cb 0x562db8d7cf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:17.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.023+0000 7fde8523a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8f8f9a0 0x562db8fd0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.023+0000 7fde8523a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8f8f9a0 0x562db8fd0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48690/0 (socket says 172.21.15.92:48690) 2023-10-06T10:44:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.023+0000 7fde8523a700 1 -- 172.21.15.92:0/4117708955 learned_addr learned my addr 172.21.15.92:0/4117708955 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.023+0000 7fde84a39700 1 --2- 172.21.15.92:0/4117708955 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562db8fb8eb0 0x562db8fd06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.023+0000 7fde85a3b700 1 --2- 172.21.15.92:0/4117708955 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8fd9c10 0x562db8fed880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.023+0000 7fde8523a700 1 -- 172.21.15.92:0/4117708955 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562db8fb8eb0 msgr2=0x562db8fd06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.023+0000 7fde8523a700 1 --2- 172.21.15.92:0/4117708955 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562db8fb8eb0 0x562db8fd06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.023+0000 7fde8523a700 1 -- 172.21.15.92:0/4117708955 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8fd9c10 msgr2=0x562db8fed880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.023+0000 7fde8523a700 1 --2- 172.21.15.92:0/4117708955 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8fd9c10 0x562db8fed880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.023+0000 7fde8523a700 1 -- 172.21.15.92:0/4117708955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562db8d7a2c0 con 0x562db8f8f9a0 2023-10-06T10:44:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.024+0000 7fde8523a700 1 --2- 172.21.15.92:0/4117708955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8f8f9a0 0x562db8fd0170 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fde7000a900 tx=0x7fde70018040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2a94bc0c6e4545 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.024+0000 7fde5f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.024+0000 7fde5f7fe700 1 -- 172.21.15.92:0/4117708955 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde700540f0 con 0x562db8f8f9a0 2023-10-06T10:44:17.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.024+0000 7fde5f7fe700 1 -- 172.21.15.92:0/4117708955 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fde70055240 con 0x562db8f8f9a0 2023-10-06T10:44:17.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.024+0000 7fde90796b80 1 -- 172.21.15.92:0/4117708955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde7400beb0 con 0x562db8f8f9a0 2023-10-06T10:44:17.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.024+0000 7fde5f7fe700 1 -- 172.21.15.92:0/4117708955 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde700830d0 con 0x562db8f8f9a0 2023-10-06T10:44:17.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.024+0000 7fde90796b80 10 client.8145 fetch_fsmap learned FSMap version 90 2023-10-06T10:44:17.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.024+0000 7fde90796b80 1 -- 172.21.15.92:0/4117708955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x562db8e954c0 con 0x562db8f8f9a0 2023-10-06T10:44:17.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.024+0000 7fde5f7fe700 1 -- 172.21.15.92:0/4117708955 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde70060020 con 0x562db8f8f9a0 2023-10-06T10:44:17.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.024+0000 7fde5f7fe700 1 -- 172.21.15.92:0/4117708955 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 90) v1 ==== 3136+0+0 (secure 0 0 0) 0x7fde7005c860 con 0x562db8f8f9a0 2023-10-06T10:44:17.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.024+0000 7fde90796b80 10 client.8145 fetch_fsmap finished waiting for FSMap version 90 2023-10-06T10:44:17.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.024+0000 7fde90796b80 10 client.8145 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.a 2023-10-06T10:44:17.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.024+0000 7fde90796b80 20 client.8145 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:17.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.024+0000 7fde90796b80 1 --2- 172.21.15.92:0/4117708955 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x562db8fd4f60 0x562db8feddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.024+0000 7fde90796b80 4 client.8145 mds_command: new command op to 7686 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:44:17.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.024+0000 7fde90796b80 1 -- 172.21.15.92:0/4117708955 --> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562db8e82e50 con 0x562db8fd4f60 2023-10-06T10:44:17.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.024+0000 7fde84a39700 1 --2- 172.21.15.92:0/4117708955 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x562db8fd4f60 0x562db8feddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.024+0000 7fde5f7fe700 10 client.8145 ms_handle_connect on v2:172.21.15.92:6834/3605297930 2023-10-06T10:44:17.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.024+0000 7fde84a39700 1 --2- 172.21.15.92:0/4117708955 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x562db8fd4f60 0x562db8feddc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f30d0314ea5a6e84 server_cookie=32b73b8e6c71da03 in_seq=0 out_seq=0 2023-10-06T10:44:17.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.025+0000 7fde5f7fe700 1 -- 172.21.15.92:0/4117708955 <== mds.2 v2:172.21.15.92:6834/3605297930 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x562db8e82e50 con 0x562db8fd4f60 2023-10-06T10:44:17.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.025+0000 7fde5f7fe700 10 client.8145 handle_command_reply: tid=0 2023-10-06T10:44:17.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.025+0000 7fde90796b80 1 client.8145 shutdown 2023-10-06T10:44:17.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.025+0000 7fde90796b80 1 -- 172.21.15.92:0/4117708955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8f8f9a0 msgr2=0x562db8fd0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.025+0000 7fde90796b80 1 --2- 172.21.15.92:0/4117708955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8f8f9a0 0x562db8fd0170 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fde7000a900 tx=0x7fde70018040 comp rx=0 tx=0).stop 2023-10-06T10:44:17.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.025+0000 7fde90796b80 1 -- 172.21.15.92:0/4117708955 shutdown_connections 2023-10-06T10:44:17.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.025+0000 7fde90796b80 1 --2- 172.21.15.92:0/4117708955 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562db8fd9c10 0x562db8fed880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.025+0000 7fde90796b80 1 --2- 172.21.15.92:0/4117708955 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x562db8fd4f60 0x562db8feddc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.025+0000 7fde90796b80 1 --2- 172.21.15.92:0/4117708955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562db8f8f9a0 0x562db8fd0170 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.025+0000 7fde5f7fe700 0 client.8145 ms_handle_reset on v2:172.21.15.92:6834/3605297930 2023-10-06T10:44:17.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.025+0000 7fde90796b80 1 --2- 172.21.15.92:0/4117708955 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562db8fb8eb0 0x562db8fd06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.025+0000 7fde90796b80 1 -- 172.21.15.92:0/4117708955 >> 172.21.15.92:0/4117708955 conn(0x562db8f912c0 msgr2=0x562db8f861f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:17.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.025+0000 7fde90796b80 1 -- 172.21.15.92:0/4117708955 shutdown_connections 2023-10-06T10:44:17.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.025+0000 7fde90796b80 1 -- 172.21.15.92:0/4117708955 wait complete. 2023-10-06T10:44:17.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.025+0000 7fde90796b80 20 client.8145 trim_cache size 0 max 16384 2023-10-06T10:44:17.038 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:44:17.038 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "PAUSED (0 inodes in the stack)", 2023-10-06T10:44:17.038 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:44:17.038 INFO:teuthology.orchestra.run.smithi092.stdout: "9669003b-2b0f-4975-abcd-f07cf375c689": { 2023-10-06T10:44:17.038 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:44:17.038 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "9669003b-2b0f-4975-abcd-f07cf375c689", 2023-10-06T10:44:17.038 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:44:17.038 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:17.038 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:17.039 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:44:17.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.025+0000 7fde8623c700 1 -- 172.21.15.92:0/3975877704 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fde6c066f30 msgr2=0x7fde6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.025+0000 7fde8623c700 1 --2- 172.21.15.92:0/3975877704 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fde6c066f30 0x7fde6c0693e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fde740099b0 tx=0x7fde74016000 comp rx=0 tx=0).stop 2023-10-06T10:44:17.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.025+0000 7fde8623c700 1 -- 172.21.15.92:0/3975877704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde800cca70 msgr2=0x7fde800c7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.025+0000 7fde8623c700 1 --2- 172.21.15.92:0/3975877704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde800cca70 0x7fde800c7810 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fde7000cfb0 tx=0x7fde70007870 comp rx=0 tx=0).stop 2023-10-06T10:44:17.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.025+0000 7fde8623c700 1 -- 172.21.15.92:0/3975877704 shutdown_connections 2023-10-06T10:44:17.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.026+0000 7fde8623c700 1 --2- 172.21.15.92:0/3975877704 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fde6c066f30 0x7fde6c0693e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.026+0000 7fde8623c700 1 --2- 172.21.15.92:0/3975877704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde80153990 0x7fde800c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.026+0000 7fde8623c700 1 --2- 172.21.15.92:0/3975877704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde800cca70 0x7fde800c7810 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.026+0000 7fde8623c700 1 --2- 172.21.15.92:0/3975877704 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde800d07f0 0x7fde800b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.026+0000 7fde8623c700 1 -- 172.21.15.92:0/3975877704 >> 172.21.15.92:0/3975877704 conn(0x7fde800680c0 msgr2=0x7fde800ce860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:17.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.026+0000 7fde8623c700 1 -- 172.21.15.92:0/3975877704 shutdown_connections 2023-10-06T10:44:17.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.026+0000 7fde8623c700 1 -- 172.21.15.92:0/3975877704 wait complete. 2023-10-06T10:44:17.041 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:44:17.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub status 2023-10-06T10:44:17.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.155+0000 7f7f8ee0e700 1 Processor -- start 2023-10-06T10:44:17.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.155+0000 7f7f8ee0e700 1 -- start start 2023-10-06T10:44:17.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.156+0000 7f7f8ee0e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7f880b2620 0x7f7f880cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.156+0000 7f7f8ee0e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f880cc970 con 0x7f7f880b29f0 2023-10-06T10:44:17.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.156+0000 7f7f8ee0e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f880cbd60 con 0x7f7f880cf1b0 2023-10-06T10:44:17.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.156+0000 7f7f8ee0e700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f880cc040 con 0x7f7f880b2620 2023-10-06T10:44:17.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.156+0000 7f7f8d60b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7f880b29f0 0x7f7f880ceaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44616/0 (socket says 172.21.15.92:44616) 2023-10-06T10:44:17.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.156+0000 7f7f8d60b700 1 -- 172.21.15.92:0/1194157307 learned_addr learned my addr 172.21.15.92:0/1194157307 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:17.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.156+0000 7f7f8de0c700 1 --2- 172.21.15.92:0/1194157307 >> v2:172.21.15.124:3300/0 conn(0x7f7f880b2620 0x7f7f880cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.157+0000 7f7f8ce0a700 1 -- 172.21.15.92:0/1194157307 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1159608911 0 0) 0x7f7f880cc970 con 0x7f7f880b29f0 2023-10-06T10:44:17.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.157+0000 7f7f8ce0a700 1 -- 172.21.15.92:0/1194157307 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f74003570 con 0x7f7f880b29f0 2023-10-06T10:44:17.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.157+0000 7f7f8ce0a700 1 -- 172.21.15.92:0/1194157307 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3843195423 0 0) 0x7f7f74003570 con 0x7f7f880b29f0 2023-10-06T10:44:17.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.157+0000 7f7f8de0c700 1 -- 172.21.15.92:0/1194157307 >> v1:172.21.15.124:6789/0 conn(0x7f7f880cf1b0 legacy=0x7f7f880c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.157+0000 7f7f8de0c700 1 -- 172.21.15.92:0/1194157307 >> v1:172.21.15.92:6789/0 conn(0x7f7f880b29f0 legacy=0x7f7f880ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.157+0000 7f7f8de0c700 1 -- 172.21.15.92:0/1194157307 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f880cc320 con 0x7f7f880b2620 2023-10-06T10:44:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.157+0000 7f7f8de0c700 1 --2- 172.21.15.92:0/1194157307 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f880b2620 0x7f7f880cccb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7f84009eb0 tx=0x7f7f84009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd8de861e8f3288c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.157+0000 7f7f8ce0a700 1 -- 172.21.15.92:0/1194157307 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8400e070 con 0x7f7f880b2620 2023-10-06T10:44:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.157+0000 7f7f8ce0a700 1 -- 172.21.15.92:0/1194157307 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8400b920 con 0x7f7f880b2620 2023-10-06T10:44:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.157+0000 7f7f8ce0a700 1 -- 172.21.15.92:0/1194157307 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f78001470 con 0x7f7f880b2620 2023-10-06T10:44:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.158+0000 7f7f8ee0e700 1 -- 172.21.15.92:0/1194157307 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f880b2620 msgr2=0x7f7f880cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.158+0000 7f7f8ee0e700 1 --2- 172.21.15.92:0/1194157307 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f880b2620 0x7f7f880cccb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7f84009eb0 tx=0x7f7f84009780 comp rx=0 tx=0).stop 2023-10-06T10:44:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.158+0000 7f7f8ee0e700 1 -- 172.21.15.92:0/1194157307 shutdown_connections 2023-10-06T10:44:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.158+0000 7f7f8ee0e700 1 --2- 172.21.15.92:0/1194157307 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f880b2620 0x7f7f880cccb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.158+0000 7f7f8ee0e700 1 -- 172.21.15.92:0/1194157307 >> 172.21.15.92:0/1194157307 conn(0x7f7f88099520 msgr2=0x7f7f8806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.158+0000 7f7f8ee0e700 1 -- 172.21.15.92:0/1194157307 shutdown_connections 2023-10-06T10:44:17.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.158+0000 7f7f8ee0e700 1 -- 172.21.15.92:0/1194157307 wait complete. 2023-10-06T10:44:17.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.159+0000 7f7f8ee0e700 1 Processor -- start 2023-10-06T10:44:17.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.159+0000 7f7f8ee0e700 1 -- start start 2023-10-06T10:44:17.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.159+0000 7f7f8ee0e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f880b2620 0x7f7f881551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.159+0000 7f7f8ee0e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f880b29f0 0x7f7f880b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.160+0000 7f7f8ee0e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f880cf1b0 0x7f7f880b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.160+0000 7f7f8ee0e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f881691d0 con 0x7f7f880b2620 2023-10-06T10:44:17.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.160+0000 7f7f8ee0e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f880b3f50 con 0x7f7f880cf1b0 2023-10-06T10:44:17.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.160+0000 7f7f8ee0e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f880b4230 con 0x7f7f880b29f0 2023-10-06T10:44:17.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.160+0000 7f7f8de0c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f880b2620 0x7f7f881551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.160+0000 7f7f8de0c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f880b2620 0x7f7f881551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48704/0 (socket says 172.21.15.92:48704) 2023-10-06T10:44:17.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.160+0000 7f7f8de0c700 1 -- 172.21.15.92:0/1947424741 learned_addr learned my addr 172.21.15.92:0/1947424741 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:17.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.160+0000 7f7f8d60b700 1 --2- 172.21.15.92:0/1947424741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f880b29f0 0x7f7f880b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.160+0000 7f7f8e60d700 1 --2- 172.21.15.92:0/1947424741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f880cf1b0 0x7f7f880b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.160+0000 7f7f8de0c700 1 -- 172.21.15.92:0/1947424741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f880b29f0 msgr2=0x7f7f880b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.160+0000 7f7f8de0c700 1 --2- 172.21.15.92:0/1947424741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f880b29f0 0x7f7f880b54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.160+0000 7f7f8de0c700 1 -- 172.21.15.92:0/1947424741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f880cf1b0 msgr2=0x7f7f880b59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.160+0000 7f7f8de0c700 1 --2- 172.21.15.92:0/1947424741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f880cf1b0 0x7f7f880b59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.160+0000 7f7f8de0c700 1 -- 172.21.15.92:0/1947424741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f84009a20 con 0x7f7f880b2620 2023-10-06T10:44:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.160+0000 7f7f8de0c700 1 --2- 172.21.15.92:0/1947424741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f880b2620 0x7f7f881551f0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f7f840099f0 tx=0x7f7f84017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.160+0000 7f7f7dffb700 1 -- 172.21.15.92:0/1947424741 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f84012c50 con 0x7f7f880b2620 2023-10-06T10:44:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.160+0000 7f7f7dffb700 1 -- 172.21.15.92:0/1947424741 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f84012db0 con 0x7f7f880b2620 2023-10-06T10:44:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.160+0000 7f7f8ee0e700 1 -- 172.21.15.92:0/1947424741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f880b4510 con 0x7f7f880b2620 2023-10-06T10:44:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.160+0000 7f7f7dffb700 1 -- 172.21.15.92:0/1947424741 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8401f660 con 0x7f7f880b2620 2023-10-06T10:44:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.160+0000 7f7f8ee0e700 1 -- 172.21.15.92:0/1947424741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f88155730 con 0x7f7f880b2620 2023-10-06T10:44:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.162+0000 7f7f7dffb700 1 -- 172.21.15.92:0/1947424741 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7f8401fa90 con 0x7f7f880b2620 2023-10-06T10:44:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.162+0000 7f7f7dffb700 1 --2- 172.21.15.92:0/1947424741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7f6c066f30 0x7f7f6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.162+0000 7f7f7dffb700 1 -- 172.21.15.92:0/1947424741 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7f84086910 con 0x7f7f880b2620 2023-10-06T10:44:17.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.162+0000 7f7f8d60b700 1 --2- 172.21.15.92:0/1947424741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7f6c066f30 0x7f7f6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.162+0000 7f7f8d60b700 1 --2- 172.21.15.92:0/1947424741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7f6c066f30 0x7f7f6c0693e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7f880b5190 tx=0x7f7f78009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.164+0000 7f7f99368b80 1 Processor -- start 2023-10-06T10:44:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.164+0000 7f7f99368b80 1 -- start start 2023-10-06T10:44:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.165+0000 7f7f99368b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e309a0 0x562847e30d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.165+0000 7f7f99368b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e7ac10 0x562847e59970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.165+0000 7f7f99368b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562847e59eb0 0x562847e2c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.165+0000 7f7f99368b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562847db0cd0 con 0x562847e309a0 2023-10-06T10:44:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.165+0000 7f7f99368b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562847d87ac0 con 0x562847e7ac10 2023-10-06T10:44:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.165+0000 7f7f99368b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562847d852c0 con 0x562847e59eb0 2023-10-06T10:44:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.165+0000 7f7f8de0c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e309a0 0x562847e30d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.165+0000 7f7f8de0c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e309a0 0x562847e30d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48710/0 (socket says 172.21.15.92:48710) 2023-10-06T10:44:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.165+0000 7f7f8de0c700 1 -- 172.21.15.92:0/3268931278 learned_addr learned my addr 172.21.15.92:0/3268931278 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.165+0000 7f7f8d60b700 1 --2- 172.21.15.92:0/3268931278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e7ac10 0x562847e59970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.165+0000 7f7f8e60d700 1 --2- 172.21.15.92:0/3268931278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562847e59eb0 0x562847e2c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.165+0000 7f7f8de0c700 1 -- 172.21.15.92:0/3268931278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562847e59eb0 msgr2=0x562847e2c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.165+0000 7f7f8de0c700 1 --2- 172.21.15.92:0/3268931278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562847e59eb0 0x562847e2c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.165+0000 7f7f8de0c700 1 -- 172.21.15.92:0/3268931278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e7ac10 msgr2=0x562847e59970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.165+0000 7f7f8de0c700 1 --2- 172.21.15.92:0/3268931278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e7ac10 0x562847e59970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.165+0000 7f7f8de0c700 1 -- 172.21.15.92:0/3268931278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562847c1b2c0 con 0x562847e309a0 2023-10-06T10:44:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.165+0000 7f7f8d60b700 1 --2- 172.21.15.92:0/3268931278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e7ac10 0x562847e59970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.165+0000 7f7f8de0c700 1 --2- 172.21.15.92:0/3268931278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e309a0 0x562847e30d70 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f7f84007210 tx=0x7f7f84084af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91733ddf03fb6033 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.165+0000 7f7f72ffd700 1 -- 172.21.15.92:0/3268931278 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f84071070 con 0x562847e309a0 2023-10-06T10:44:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.165+0000 7f7f72ffd700 1 -- 172.21.15.92:0/3268931278 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8405a8b0 con 0x562847e309a0 2023-10-06T10:44:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.165+0000 7f7f72ffd700 1 -- 172.21.15.92:0/3268931278 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8400e040 con 0x562847e309a0 2023-10-06T10:44:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.166+0000 7f7f99368b80 1 -- 172.21.15.92:0/3268931278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e309a0 msgr2=0x562847e30d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.166+0000 7f7f99368b80 1 --2- 172.21.15.92:0/3268931278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e309a0 0x562847e30d70 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f7f84007210 tx=0x7f7f84084af0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.166+0000 7f7f99368b80 1 -- 172.21.15.92:0/3268931278 shutdown_connections 2023-10-06T10:44:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.166+0000 7f7f99368b80 1 --2- 172.21.15.92:0/3268931278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562847e59eb0 0x562847e2c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.166+0000 7f7f99368b80 1 --2- 172.21.15.92:0/3268931278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e309a0 0x562847e30d70 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.166+0000 7f7f99368b80 1 --2- 172.21.15.92:0/3268931278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e7ac10 0x562847e59970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.166+0000 7f7f99368b80 1 -- 172.21.15.92:0/3268931278 >> 172.21.15.92:0/3268931278 conn(0x562847e322c0 msgr2=0x562847e32690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:17.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.166+0000 7f7f99368b80 1 -- 172.21.15.92:0/3268931278 shutdown_connections 2023-10-06T10:44:17.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.166+0000 7f7f99368b80 1 -- 172.21.15.92:0/3268931278 wait complete. 2023-10-06T10:44:17.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.166+0000 7f7f99368b80 1 -- start start 2023-10-06T10:44:17.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.166+0000 7f7f99368b80 1 Processor -- start 2023-10-06T10:44:17.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.167+0000 7f7f99368b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e309a0 0x562847e56060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.167+0000 7f7f99368b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562847e59eb0 0x562847e74b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.167+0000 7f7f99368b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e7ac10 0x562847e2cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.167+0000 7f7f99368b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562847d810b0 con 0x562847e7ac10 2023-10-06T10:44:17.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.167+0000 7f7f99368b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562847d2c8e0 con 0x562847e309a0 2023-10-06T10:44:17.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.167+0000 7f7f99368b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562847d47ef0 con 0x562847e59eb0 2023-10-06T10:44:17.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.167+0000 7f7f99368b80 10 client.0 _ll_register_callbacks cb 0x562847c1df80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:17.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.167+0000 7f7f8e60d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e7ac10 0x562847e2cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.167+0000 7f7f8e60d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e7ac10 0x562847e2cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48718/0 (socket says 172.21.15.92:48718) 2023-10-06T10:44:17.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.167+0000 7f7f8e60d700 1 -- 172.21.15.92:0/552707487 learned_addr learned my addr 172.21.15.92:0/552707487 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:17.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.167+0000 7f7f8de0c700 1 --2- 172.21.15.92:0/552707487 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e309a0 0x562847e56060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.168+0000 7f7f8d60b700 1 --2- 172.21.15.92:0/552707487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562847e59eb0 0x562847e74b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.168+0000 7f7f8e60d700 1 -- 172.21.15.92:0/552707487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562847e59eb0 msgr2=0x562847e74b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:17.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.168+0000 7f7f8e60d700 1 --2- 172.21.15.92:0/552707487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562847e59eb0 0x562847e74b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.168+0000 7f7f8e60d700 1 -- 172.21.15.92:0/552707487 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e309a0 msgr2=0x562847e56060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:17.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.168+0000 7f7f8e60d700 1 --2- 172.21.15.92:0/552707487 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e309a0 0x562847e56060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.168+0000 7f7f8e60d700 1 -- 172.21.15.92:0/552707487 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f88155730 con 0x562847e7ac10 2023-10-06T10:44:17.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.168+0000 7f7f8e60d700 1 --2- 172.21.15.92:0/552707487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e7ac10 0x562847e2cf20 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f7f80003ef0 tx=0x7f7f80009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18747c58ad4dd777 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:17.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.168+0000 7f7f67fff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:17.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.168+0000 7f7f67fff700 1 -- 172.21.15.92:0/552707487 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f80010cf0 con 0x562847e7ac10 2023-10-06T10:44:17.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.168+0000 7f7f67fff700 1 -- 172.21.15.92:0/552707487 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7f80010e50 con 0x562847e7ac10 2023-10-06T10:44:17.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.168+0000 7f7f99368b80 1 -- 172.21.15.92:0/552707487 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562847e256e0 con 0x562847e7ac10 2023-10-06T10:44:17.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.168+0000 7f7f67fff700 1 -- 172.21.15.92:0/552707487 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f80014610 con 0x562847e7ac10 2023-10-06T10:44:17.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.168+0000 7f7f99368b80 10 client.8157 fetch_fsmap learned FSMap version 90 2023-10-06T10:44:17.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.168+0000 7f7f99368b80 1 -- 172.21.15.92:0/552707487 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x562847bbe030 con 0x562847e7ac10 2023-10-06T10:44:17.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.168+0000 7f7f67fff700 1 -- 172.21.15.92:0/552707487 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562847e256e0 con 0x562847e7ac10 2023-10-06T10:44:17.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.168+0000 7f7f67fff700 1 -- 172.21.15.92:0/552707487 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 90) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f7f80014770 con 0x562847e7ac10 2023-10-06T10:44:17.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.168+0000 7f7f99368b80 10 client.8157 fetch_fsmap finished waiting for FSMap version 90 2023-10-06T10:44:17.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.168+0000 7f7f99368b80 10 client.8157 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2023-10-06T10:44:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.168+0000 7f7f99368b80 20 client.8157 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.168+0000 7f7f99368b80 1 --2- 172.21.15.92:0/552707487 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x562847e58060 0x562847e58460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.168+0000 7f7f99368b80 4 client.8157 mds_command: new command op to 7701 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:44:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.168+0000 7f7f99368b80 1 -- 172.21.15.92:0/552707487 --> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562847e02610 con 0x562847e58060 2023-10-06T10:44:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.169+0000 7f7f8de0c700 1 --2- 172.21.15.92:0/552707487 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x562847e58060 0x562847e58460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.169+0000 7f7f67fff700 10 client.8157 ms_handle_connect on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.169+0000 7f7f8de0c700 1 --2- 172.21.15.92:0/552707487 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x562847e58060 0x562847e58460 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2de8e50172bde62 server_cookie=1443ba8504064fe2 in_seq=0 out_seq=0 2023-10-06T10:44:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.169+0000 7f7f67fff700 1 -- 172.21.15.92:0/552707487 <== mds.0 v2:172.21.15.124:6832/4203118992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x562847e02610 con 0x562847e58060 2023-10-06T10:44:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.169+0000 7f7f67fff700 10 client.8157 handle_command_reply: tid=0 2023-10-06T10:44:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.170+0000 7f7f99368b80 1 client.8157 shutdown 2023-10-06T10:44:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.170+0000 7f7f99368b80 1 -- 172.21.15.92:0/552707487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e7ac10 msgr2=0x562847e2cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.170+0000 7f7f99368b80 1 --2- 172.21.15.92:0/552707487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e7ac10 0x562847e2cf20 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f7f80003ef0 tx=0x7f7f80009040 comp rx=0 tx=0).stop 2023-10-06T10:44:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.170+0000 7f7f99368b80 1 -- 172.21.15.92:0/552707487 shutdown_connections 2023-10-06T10:44:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.170+0000 7f7f99368b80 1 --2- 172.21.15.92:0/552707487 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x562847e58060 0x562847e58460 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.170+0000 7f7f99368b80 1 --2- 172.21.15.92:0/552707487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e7ac10 0x562847e2cf20 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.170+0000 7f7f67fff700 0 client.8157 ms_handle_reset on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.170+0000 7f7f99368b80 1 --2- 172.21.15.92:0/552707487 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e309a0 0x562847e56060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.170+0000 7f7f99368b80 1 --2- 172.21.15.92:0/552707487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562847e59eb0 0x562847e74b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.170+0000 7f7f99368b80 1 -- 172.21.15.92:0/552707487 >> 172.21.15.92:0/552707487 conn(0x562847e322c0 msgr2=0x562847e764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.170+0000 7f7f99368b80 1 -- 172.21.15.92:0/552707487 shutdown_connections 2023-10-06T10:44:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.170+0000 7f7f99368b80 1 -- 172.21.15.92:0/552707487 wait complete. 2023-10-06T10:44:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.170+0000 7f7f99368b80 20 client.8157 trim_cache size 0 max 16384 2023-10-06T10:44:17.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.186+0000 7f7f99368b80 1 Processor -- start 2023-10-06T10:44:17.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.186+0000 7f7f99368b80 1 -- start start 2023-10-06T10:44:17.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.186+0000 7f7f99368b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e309a0 0x562847e271f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.186+0000 7f7f99368b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562847e59eb0 0x562847e27730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.186+0000 7f7f99368b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e7ac10 0x562847e56950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.186+0000 7f7f99368b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562847e32090 con 0x562847e309a0 2023-10-06T10:44:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.186+0000 7f7f99368b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562847c03e80 con 0x562847e7ac10 2023-10-06T10:44:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.186+0000 7f7f99368b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562847e31160 con 0x562847e59eb0 2023-10-06T10:44:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.186+0000 7f7f8de0c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e309a0 0x562847e271f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.187+0000 7f7f8de0c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e309a0 0x562847e271f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48720/0 (socket says 172.21.15.92:48720) 2023-10-06T10:44:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.187+0000 7f7f8de0c700 1 -- 172.21.15.92:0/559656157 learned_addr learned my addr 172.21.15.92:0/559656157 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.187+0000 7f7f8d60b700 1 --2- 172.21.15.92:0/559656157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562847e59eb0 0x562847e27730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.187+0000 7f7f8e60d700 1 --2- 172.21.15.92:0/559656157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e7ac10 0x562847e56950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.187+0000 7f7f8de0c700 1 -- 172.21.15.92:0/559656157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562847e59eb0 msgr2=0x562847e27730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.187+0000 7f7f8de0c700 1 --2- 172.21.15.92:0/559656157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562847e59eb0 0x562847e27730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.187+0000 7f7f8de0c700 1 -- 172.21.15.92:0/559656157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e7ac10 msgr2=0x562847e56950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.187+0000 7f7f8de0c700 1 --2- 172.21.15.92:0/559656157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e7ac10 0x562847e56950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.187+0000 7f7f8de0c700 1 -- 172.21.15.92:0/559656157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562847e2edf0 con 0x562847e309a0 2023-10-06T10:44:17.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.187+0000 7f7f8e60d700 1 --2- 172.21.15.92:0/559656157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e7ac10 0x562847e56950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:17.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.187+0000 7f7f8d60b700 1 --2- 172.21.15.92:0/559656157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562847e59eb0 0x562847e27730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:17.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.187+0000 7f7f8de0c700 1 --2- 172.21.15.92:0/559656157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e309a0 0x562847e271f0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f7f840097c0 tx=0x7f7f84079250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a842b277449d352e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:17.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.187+0000 7f7f737fe700 1 -- 172.21.15.92:0/559656157 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8400e040 con 0x562847e309a0 2023-10-06T10:44:17.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.187+0000 7f7f737fe700 1 -- 172.21.15.92:0/559656157 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8405e600 con 0x562847e309a0 2023-10-06T10:44:17.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.187+0000 7f7f737fe700 1 -- 172.21.15.92:0/559656157 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f84071040 con 0x562847e309a0 2023-10-06T10:44:17.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.188+0000 7f7f99368b80 1 -- 172.21.15.92:0/559656157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e309a0 msgr2=0x562847e271f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.188+0000 7f7f99368b80 1 --2- 172.21.15.92:0/559656157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e309a0 0x562847e271f0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f7f840097c0 tx=0x7f7f84079250 comp rx=0 tx=0).stop 2023-10-06T10:44:17.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.188+0000 7f7f99368b80 1 -- 172.21.15.92:0/559656157 shutdown_connections 2023-10-06T10:44:17.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.188+0000 7f7f99368b80 1 --2- 172.21.15.92:0/559656157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e7ac10 0x562847e56950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.188+0000 7f7f99368b80 1 --2- 172.21.15.92:0/559656157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e309a0 0x562847e271f0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.188+0000 7f7f99368b80 1 --2- 172.21.15.92:0/559656157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562847e59eb0 0x562847e27730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.188+0000 7f7f99368b80 1 -- 172.21.15.92:0/559656157 >> 172.21.15.92:0/559656157 conn(0x562847e322c0 msgr2=0x562847e6ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.188+0000 7f7f99368b80 1 -- 172.21.15.92:0/559656157 shutdown_connections 2023-10-06T10:44:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.188+0000 7f7f99368b80 1 -- 172.21.15.92:0/559656157 wait complete. 2023-10-06T10:44:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.188+0000 7f7f99368b80 1 -- start start 2023-10-06T10:44:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.188+0000 7f7f99368b80 1 Processor -- start 2023-10-06T10:44:17.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.189+0000 7f7f99368b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562847e309a0 0x562847edc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.189+0000 7f7f99368b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e59eb0 0x562847e90090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.189+0000 7f7f99368b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e7ac10 0x562847e905d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.189+0000 7f7f99368b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562847e78d80 con 0x562847e7ac10 2023-10-06T10:44:17.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.189+0000 7f7f99368b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562847e742a0 con 0x562847e59eb0 2023-10-06T10:44:17.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.189+0000 7f7f99368b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562847e2d120 con 0x562847e309a0 2023-10-06T10:44:17.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.189+0000 7f7f99368b80 10 client.0 _ll_register_callbacks cb 0x562847c1df80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:17.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.189+0000 7f7f8e60d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e7ac10 0x562847e905d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.189+0000 7f7f8e60d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e7ac10 0x562847e905d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48730/0 (socket says 172.21.15.92:48730) 2023-10-06T10:44:17.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.189+0000 7f7f8e60d700 1 -- 172.21.15.92:0/2828683625 learned_addr learned my addr 172.21.15.92:0/2828683625 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:17.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.189+0000 7f7f8de0c700 1 --2- 172.21.15.92:0/2828683625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562847e309a0 0x562847edc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.189+0000 7f7f8d60b700 1 --2- 172.21.15.92:0/2828683625 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e59eb0 0x562847e90090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.189+0000 7f7f8e60d700 1 -- 172.21.15.92:0/2828683625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562847e309a0 msgr2=0x562847edc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:17.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.189+0000 7f7f8e60d700 1 --2- 172.21.15.92:0/2828683625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562847e309a0 0x562847edc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.189+0000 7f7f8e60d700 1 -- 172.21.15.92:0/2828683625 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e59eb0 msgr2=0x562847e90090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:17.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.190+0000 7f7f8e60d700 1 --2- 172.21.15.92:0/2828683625 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e59eb0 0x562847e90090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.190+0000 7f7f8e60d700 1 -- 172.21.15.92:0/2828683625 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562847c1b2c0 con 0x562847e7ac10 2023-10-06T10:44:17.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.190+0000 7f7f8d60b700 1 --2- 172.21.15.92:0/2828683625 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e59eb0 0x562847e90090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:17.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.190+0000 7f7f8e60d700 1 --2- 172.21.15.92:0/2828683625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e7ac10 0x562847e905d0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f7f80014fc0 tx=0x7f7f80020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e01034a50fd561 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:17.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.190+0000 7f7f67fff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:17.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.190+0000 7f7f67fff700 1 -- 172.21.15.92:0/2828683625 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f80003410 con 0x562847e7ac10 2023-10-06T10:44:17.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.190+0000 7f7f67fff700 1 -- 172.21.15.92:0/2828683625 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7f80010cf0 con 0x562847e7ac10 2023-10-06T10:44:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.190+0000 7f7f99368b80 1 -- 172.21.15.92:0/2828683625 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f8405a8b0 con 0x562847e7ac10 2023-10-06T10:44:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.190+0000 7f7f67fff700 1 -- 172.21.15.92:0/2828683625 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f80010eb0 con 0x562847e7ac10 2023-10-06T10:44:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.190+0000 7f7f99368b80 10 client.8163 fetch_fsmap learned FSMap version 90 2023-10-06T10:44:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.190+0000 7f7f99368b80 1 -- 172.21.15.92:0/2828683625 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x562847d4f7d0 con 0x562847e7ac10 2023-10-06T10:44:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.190+0000 7f7f67fff700 1 -- 172.21.15.92:0/2828683625 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8405a8b0 con 0x562847e7ac10 2023-10-06T10:44:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.190+0000 7f7f67fff700 1 -- 172.21.15.92:0/2828683625 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 90) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f7f8002e320 con 0x562847e7ac10 2023-10-06T10:44:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.190+0000 7f7f99368b80 10 client.8163 fetch_fsmap finished waiting for FSMap version 90 2023-10-06T10:44:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.190+0000 7f7f99368b80 10 client.8163 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2023-10-06T10:44:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.190+0000 7f7f99368b80 20 client.8163 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.190+0000 7f7f99368b80 1 --2- 172.21.15.92:0/2828683625 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x562847e6da60 0x562847e6de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.190+0000 7f7f99368b80 4 client.8163 mds_command: new command op to 7701 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:44:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.190+0000 7f7f99368b80 1 -- 172.21.15.92:0/2828683625 --> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562847d23e50 con 0x562847e6da60 2023-10-06T10:44:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.190+0000 7f7f8de0c700 1 --2- 172.21.15.92:0/2828683625 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x562847e6da60 0x562847e6de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.191+0000 7f7f67fff700 10 client.8163 ms_handle_connect on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.191+0000 7f7f8de0c700 1 --2- 172.21.15.92:0/2828683625 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x562847e6da60 0x562847e6de60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6b44d6613bc5a15 server_cookie=e3792d38c32eb74c in_seq=0 out_seq=0 2023-10-06T10:44:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.191+0000 7f7f67fff700 1 -- 172.21.15.92:0/2828683625 <== mds.0 v2:172.21.15.124:6832/4203118992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x562847d23e50 con 0x562847e6da60 2023-10-06T10:44:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.191+0000 7f7f67fff700 10 client.8163 handle_command_reply: tid=0 2023-10-06T10:44:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.191+0000 7f7f99368b80 1 client.8163 shutdown 2023-10-06T10:44:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.191+0000 7f7f99368b80 1 -- 172.21.15.92:0/2828683625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e7ac10 msgr2=0x562847e905d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.191+0000 7f7f99368b80 1 --2- 172.21.15.92:0/2828683625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e7ac10 0x562847e905d0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f7f80014fc0 tx=0x7f7f80020000 comp rx=0 tx=0).stop 2023-10-06T10:44:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.191+0000 7f7f99368b80 1 -- 172.21.15.92:0/2828683625 shutdown_connections 2023-10-06T10:44:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.191+0000 7f7f99368b80 1 --2- 172.21.15.92:0/2828683625 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x562847e6da60 0x562847e6de60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.191+0000 7f7f99368b80 1 --2- 172.21.15.92:0/2828683625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562847e7ac10 0x562847e905d0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.191+0000 7f7f67fff700 0 client.8163 ms_handle_reset on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:17.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.191+0000 7f7f99368b80 1 --2- 172.21.15.92:0/2828683625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562847e309a0 0x562847edc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.191+0000 7f7f99368b80 1 --2- 172.21.15.92:0/2828683625 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562847e59eb0 0x562847e90090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.191+0000 7f7f99368b80 1 -- 172.21.15.92:0/2828683625 >> 172.21.15.92:0/2828683625 conn(0x562847e322c0 msgr2=0x562847e271f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:17.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.191+0000 7f7f99368b80 1 -- 172.21.15.92:0/2828683625 shutdown_connections 2023-10-06T10:44:17.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.191+0000 7f7f99368b80 1 -- 172.21.15.92:0/2828683625 wait complete. 2023-10-06T10:44:17.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.191+0000 7f7f99368b80 20 client.8163 trim_cache size 0 max 16384 2023-10-06T10:44:17.201 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:44:17.201 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "PAUSED (0 inodes in the stack)", 2023-10-06T10:44:17.202 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:44:17.202 INFO:teuthology.orchestra.run.smithi092.stdout: "9669003b-2b0f-4975-abcd-f07cf375c689": { 2023-10-06T10:44:17.202 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:44:17.202 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "9669003b-2b0f-4975-abcd-f07cf375c689", 2023-10-06T10:44:17.202 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:44:17.202 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:17.202 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:17.202 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:44:17.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.192+0000 7f7f8ee0e700 1 -- 172.21.15.92:0/1947424741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7f6c066f30 msgr2=0x7f7f6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.192+0000 7f7f8ee0e700 1 --2- 172.21.15.92:0/1947424741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7f6c066f30 0x7f7f6c0693e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7f880b5190 tx=0x7f7f78009040 comp rx=0 tx=0).stop 2023-10-06T10:44:17.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.192+0000 7f7f8ee0e700 1 -- 172.21.15.92:0/1947424741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f880b2620 msgr2=0x7f7f881551f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.192+0000 7f7f8ee0e700 1 --2- 172.21.15.92:0/1947424741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f880b2620 0x7f7f881551f0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f7f840099f0 tx=0x7f7f84017000 comp rx=0 tx=0).stop 2023-10-06T10:44:17.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.192+0000 7f7f8ee0e700 1 -- 172.21.15.92:0/1947424741 shutdown_connections 2023-10-06T10:44:17.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.192+0000 7f7f8ee0e700 1 --2- 172.21.15.92:0/1947424741 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7f6c066f30 0x7f7f6c0693e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.192+0000 7f7f8ee0e700 1 --2- 172.21.15.92:0/1947424741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f880cf1b0 0x7f7f880b59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.192+0000 7f7f8ee0e700 1 --2- 172.21.15.92:0/1947424741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f880b2620 0x7f7f881551f0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.192+0000 7f7f8ee0e700 1 --2- 172.21.15.92:0/1947424741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f880b29f0 0x7f7f880b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.192+0000 7f7f8ee0e700 1 -- 172.21.15.92:0/1947424741 >> 172.21.15.92:0/1947424741 conn(0x7f7f88099520 msgr2=0x7f7f880ccb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:17.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.192+0000 7f7f8ee0e700 1 -- 172.21.15.92:0/1947424741 shutdown_connections 2023-10-06T10:44:17.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.192+0000 7f7f8ee0e700 1 -- 172.21.15.92:0/1947424741 wait complete. 2023-10-06T10:44:17.205 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:44:17.205 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:44:17.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.317+0000 7f1fcbfff700 1 Processor -- start 2023-10-06T10:44:17.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.317+0000 7f1fcbfff700 1 -- start start 2023-10-06T10:44:17.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.318+0000 7f1fcbfff700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1fcc159440 0x7f1fcc0b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.318+0000 7f1fcbfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1fcc0b2100 0x7f1fcc0b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.318+0000 7f1fcbfff700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fcc0cb7a0 con 0x7f1fcc0b35d0 2023-10-06T10:44:17.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.318+0000 7f1fcbfff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fcc0b0860 con 0x7f1fcc0b2100 2023-10-06T10:44:17.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.318+0000 7f1fcbfff700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fcc0cb150 con 0x7f1fcc159440 2023-10-06T10:44:17.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.318+0000 7f1fcb7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1fcc0b2100 0x7f1fcc0b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.318+0000 7f1fcb7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1fcc0b2100 0x7f1fcc0b2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48746/0 (socket says 172.21.15.92:48746) 2023-10-06T10:44:17.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.318+0000 7f1fcb7fe700 1 -- 172.21.15.92:0/1092986108 learned_addr learned my addr 172.21.15.92:0/1092986108 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:17.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.318+0000 7f1fcaffd700 1 --2- 172.21.15.92:0/1092986108 >> v2:172.21.15.124:3300/0 conn(0x7f1fcc159440 0x7f1fcc0b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.318+0000 7f1fcb7fe700 1 -- 172.21.15.92:0/1092986108 >> v2:172.21.15.124:3300/0 conn(0x7f1fcc159440 msgr2=0x7f1fcc0b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:17.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.318+0000 7f1fcb7fe700 1 --2- 172.21.15.92:0/1092986108 >> v2:172.21.15.124:3300/0 conn(0x7f1fcc159440 0x7f1fcc0b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.318+0000 7f1fcb7fe700 1 -- 172.21.15.92:0/1092986108 >> v1:172.21.15.124:6789/0 conn(0x7f1fcc0b35d0 legacy=0x7f1fcc0cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:17.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.318+0000 7f1fcb7fe700 1 -- 172.21.15.92:0/1092986108 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc0cb430 con 0x7f1fcc0b2100 2023-10-06T10:44:17.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.318+0000 7f1fcaffd700 1 --2- 172.21.15.92:0/1092986108 >> v2:172.21.15.124:3300/0 conn(0x7f1fcc159440 0x7f1fcc0b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:17.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.319+0000 7f1fcb7fe700 1 --2- 172.21.15.92:0/1092986108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fcc0b2100 0x7f1fcc0b2520 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc06f9f0 tx=0x7f1fc4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9651facd68eeac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:17.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.319+0000 7f1fca7fc700 1 -- 172.21.15.92:0/1092986108 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc400e070 con 0x7f1fcc0b2100 2023-10-06T10:44:17.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.319+0000 7f1fca7fc700 1 -- 172.21.15.92:0/1092986108 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc400ba10 con 0x7f1fcc0b2100 2023-10-06T10:44:17.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.319+0000 7f1fca7fc700 1 -- 172.21.15.92:0/1092986108 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc40092e0 con 0x7f1fcc0b2100 2023-10-06T10:44:17.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.319+0000 7f1fcbfff700 1 -- 172.21.15.92:0/1092986108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fcc0b2100 msgr2=0x7f1fcc0b2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.319+0000 7f1fcbfff700 1 --2- 172.21.15.92:0/1092986108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fcc0b2100 0x7f1fcc0b2520 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc06f9f0 tx=0x7f1fc4009840 comp rx=0 tx=0).stop 2023-10-06T10:44:17.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.319+0000 7f1fcbfff700 1 -- 172.21.15.92:0/1092986108 shutdown_connections 2023-10-06T10:44:17.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.319+0000 7f1fcbfff700 1 --2- 172.21.15.92:0/1092986108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fcc0b2100 0x7f1fcc0b2520 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.319+0000 7f1fcbfff700 1 --2- 172.21.15.92:0/1092986108 >> v2:172.21.15.124:3300/0 conn(0x7f1fcc159440 0x7f1fcc0b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.319+0000 7f1fcbfff700 1 -- 172.21.15.92:0/1092986108 >> 172.21.15.92:0/1092986108 conn(0x7f1fcc06ece0 msgr2=0x7f1fcc06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:17.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.319+0000 7f1fcbfff700 1 -- 172.21.15.92:0/1092986108 shutdown_connections 2023-10-06T10:44:17.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.319+0000 7f1fcbfff700 1 -- 172.21.15.92:0/1092986108 wait complete. 2023-10-06T10:44:17.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.320+0000 7f1fcbfff700 1 Processor -- start 2023-10-06T10:44:17.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.321+0000 7f1fcbfff700 1 -- start start 2023-10-06T10:44:17.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.321+0000 7f1fcbfff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fcc0b2100 0x7f1fcc0c2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.321+0000 7f1fcbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fcc0b35d0 0x7f1fcc0c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.321+0000 7f1fcbfff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1fcc159440 0x7f1fcc0c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.321+0000 7f1fcbfff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc159c90 con 0x7f1fcc0b35d0 2023-10-06T10:44:17.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.321+0000 7f1fcbfff700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc0c6970 con 0x7f1fcc159440 2023-10-06T10:44:17.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.321+0000 7f1fcbfff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc0c34e0 con 0x7f1fcc0b2100 2023-10-06T10:44:17.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.321+0000 7f1fc27fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fcc0b35d0 0x7f1fcc0c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.321+0000 7f1fc27fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fcc0b35d0 0x7f1fcc0c55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48754/0 (socket says 172.21.15.92:48754) 2023-10-06T10:44:17.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.321+0000 7f1fc27fc700 1 -- 172.21.15.92:0/3295163467 learned_addr learned my addr 172.21.15.92:0/3295163467 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:17.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.321+0000 7f1fcb7fe700 1 --2- 172.21.15.92:0/3295163467 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1fcc159440 0x7f1fcc0c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.321+0000 7f1fcaffd700 1 --2- 172.21.15.92:0/3295163467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fcc0b2100 0x7f1fcc0c2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.322+0000 7f1fc27fc700 1 -- 172.21.15.92:0/3295163467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fcc0b2100 msgr2=0x7f1fcc0c2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.322+0000 7f1fc27fc700 1 --2- 172.21.15.92:0/3295163467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fcc0b2100 0x7f1fcc0c2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.322+0000 7f1fc27fc700 1 -- 172.21.15.92:0/3295163467 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1fcc159440 msgr2=0x7f1fcc0c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.322+0000 7f1fc27fc700 1 --2- 172.21.15.92:0/3295163467 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1fcc159440 0x7f1fcc0c5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.322+0000 7f1fc27fc700 1 -- 172.21.15.92:0/3295163467 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc4009ae0 con 0x7f1fcc0b35d0 2023-10-06T10:44:17.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.322+0000 7f1fc27fc700 1 --2- 172.21.15.92:0/3295163467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fcc0b35d0 0x7f1fcc0c55a0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400b630 tx=0x7f1fb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:17.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.322+0000 7f1fc37fe700 1 -- 172.21.15.92:0/3295163467 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb400f510 con 0x7f1fcc0b35d0 2023-10-06T10:44:17.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.322+0000 7f1fc37fe700 1 -- 172.21.15.92:0/3295163467 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb400f690 con 0x7f1fcc0b35d0 2023-10-06T10:44:17.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.322+0000 7f1fc37fe700 1 -- 172.21.15.92:0/3295163467 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb400fa50 con 0x7f1fcc0b35d0 2023-10-06T10:44:17.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.322+0000 7f1fcbfff700 1 -- 172.21.15.92:0/3295163467 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fcc0c60d0 con 0x7f1fcc0b35d0 2023-10-06T10:44:17.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.322+0000 7f1fcbfff700 1 -- 172.21.15.92:0/3295163467 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fcc0c6690 con 0x7f1fcc0b35d0 2023-10-06T10:44:17.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.323+0000 7f1fcbfff700 1 -- 172.21.15.92:0/3295163467 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fcc159a60 con 0x7f1fcc0b35d0 2023-10-06T10:44:17.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.323+0000 7f1fc37fe700 1 -- 172.21.15.92:0/3295163467 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1fb401a400 con 0x7f1fcc0b35d0 2023-10-06T10:44:17.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.324+0000 7f1fc37fe700 1 --2- 172.21.15.92:0/3295163467 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1fac066f30 0x7f1fac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.324+0000 7f1fc37fe700 1 -- 172.21.15.92:0/3295163467 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1fb4084200 con 0x7f1fcc0b35d0 2023-10-06T10:44:17.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.324+0000 7f1fcaffd700 1 --2- 172.21.15.92:0/3295163467 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1fac066f30 0x7f1fac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.324+0000 7f1fcaffd700 1 --2- 172.21.15.92:0/3295163467 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1fac066f30 0x7f1fac0693e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc006fd0 tx=0x7f1fbc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:17.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.328+0000 7f1fc37fe700 1 -- 172.21.15.92:0/3295163467 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1fcc159a60 con 0x7f1fcc0b35d0 2023-10-06T10:44:17.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.555+0000 7f1fcbfff700 1 -- 172.21.15.92:0/3295163467 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fcc159a60 con 0x7f1fcc0b35d0 2023-10-06T10:44:17.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.555+0000 7f1fc37fe700 1 -- 172.21.15.92:0/3295163467 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4637 (secure 0 0 0) 0x7f1fcc159a60 con 0x7f1fcc0b35d0 2023-10-06T10:44:17.557 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:17.557 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":90,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2122441982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2122441982},{"type":"v1","addr":"172.21.15.124:6835","nonce":2122441982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:47.850832+0000","modified":"2023-10-06T10:44:12.564354+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7701,"mds_1":7689,"mds_2":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6836/3605297930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3605297930},{"type":"v1","addr":"172.21.15.92:6836","nonce":3605297930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7689":{"gid":7689,"name":"c","rank":1,"incarnation":85,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/4047304116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4047304116},{"type":"v1","addr":"172.21.15.92:6837","nonce":4047304116}]},"join_fscid":-1,"export_targets":[2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7701":{"gid":7701,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6834/4203118992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":4203118992},{"type":"v1","addr":"172.21.15.124:6834","nonce":4203118992}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T10:44:17.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.557+0000 7f1fcbfff700 1 -- 172.21.15.92:0/3295163467 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1fac066f30 msgr2=0x7f1fac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.557+0000 7f1fcbfff700 1 --2- 172.21.15.92:0/3295163467 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1fac066f30 0x7f1fac0693e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc006fd0 tx=0x7f1fbc008040 comp rx=0 tx=0).stop 2023-10-06T10:44:17.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.557+0000 7f1fcbfff700 1 -- 172.21.15.92:0/3295163467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fcc0b35d0 msgr2=0x7f1fcc0c55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.557+0000 7f1fcbfff700 1 --2- 172.21.15.92:0/3295163467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fcc0b35d0 0x7f1fcc0c55a0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400b630 tx=0x7f1fb40091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.557+0000 7f1fcbfff700 1 -- 172.21.15.92:0/3295163467 shutdown_connections 2023-10-06T10:44:17.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.557+0000 7f1fcbfff700 1 --2- 172.21.15.92:0/3295163467 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1fac066f30 0x7f1fac0693e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.557+0000 7f1fcbfff700 1 --2- 172.21.15.92:0/3295163467 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1fcc159440 0x7f1fcc0c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.557+0000 7f1fcbfff700 1 --2- 172.21.15.92:0/3295163467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fcc0b2100 0x7f1fcc0c2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.557+0000 7f1fcbfff700 1 --2- 172.21.15.92:0/3295163467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fcc0b35d0 0x7f1fcc0c55a0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.557+0000 7f1fcbfff700 1 -- 172.21.15.92:0/3295163467 >> 172.21.15.92:0/3295163467 conn(0x7f1fcc06ece0 msgr2=0x7f1fcc1586f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:17.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.557+0000 7f1fcbfff700 1 -- 172.21.15.92:0/3295163467 shutdown_connections 2023-10-06T10:44:17.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.557+0000 7f1fcbfff700 1 -- 172.21.15.92:0/3295163467 wait complete. 2023-10-06T10:44:17.561 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 90 2023-10-06T10:44:17.571 DEBUG:tasks.cephfs.test_multimds_misc:current active=['b', 'c', 'a'] 2023-10-06T10:44:17.571 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:44:17.577 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T10:44:17.575+0000 7f9e81bcf700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T10:44:17.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.682+0000 7f22d5a36700 1 Processor -- start 2023-10-06T10:44:17.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.682+0000 7f22d5a36700 1 -- start start 2023-10-06T10:44:17.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.683+0000 7f22d5a36700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f22d00ad210 0x7f22d00cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.683+0000 7f22d5a36700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f22d015db10 0x7f22d00ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.683+0000 7f22d5a36700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d00ca600 con 0x7f22d00ad5e0 2023-10-06T10:44:17.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.683+0000 7f22d5a36700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d00cecf0 con 0x7f22d015db10 2023-10-06T10:44:17.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.683+0000 7f22d5a36700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d00cefd0 con 0x7f22d00ad210 2023-10-06T10:44:17.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.683+0000 7f22c67fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f22d00ad5e0 0x7f22d015d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44622/0 (socket says 172.21.15.92:44622) 2023-10-06T10:44:17.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.683+0000 7f22c67fc700 1 -- 172.21.15.92:0/3892395597 learned_addr learned my addr 172.21.15.92:0/3892395597 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:17.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.683+0000 7f22cf7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f22d015db10 0x7f22d00ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.684+0000 7f22ceffd700 1 --2- 172.21.15.92:0/3892395597 >> v2:172.21.15.124:3300/0 conn(0x7f22d00ad210 0x7f22d00cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.684+0000 7f22cf7fe700 1 -- 172.21.15.92:0/3892395597 >> v2:172.21.15.124:3300/0 conn(0x7f22d00ad210 msgr2=0x7f22d00cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:17.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.684+0000 7f22cf7fe700 1 --2- 172.21.15.92:0/3892395597 >> v2:172.21.15.124:3300/0 conn(0x7f22d00ad210 0x7f22d00cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.684+0000 7f22ce7fc700 1 -- 172.21.15.92:0/3892395597 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2616272175 0 0) 0x7f22d00ca600 con 0x7f22d00ad5e0 2023-10-06T10:44:17.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.684+0000 7f22cf7fe700 1 -- 172.21.15.92:0/3892395597 >> v1:172.21.15.92:6789/0 conn(0x7f22d00ad5e0 legacy=0x7f22d015d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.684+0000 7f22cf7fe700 1 -- 172.21.15.92:0/3892395597 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d00cca90 con 0x7f22d015db10 2023-10-06T10:44:17.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.684+0000 7f22cf7fe700 1 --2- 172.21.15.92:0/3892395597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22d015db10 0x7f22d00ca0c0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f22d00cacb0 tx=0x7f22c8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934a54cb1ce2c64e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:17.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.684+0000 7f22ce7fc700 1 -- 172.21.15.92:0/3892395597 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c800e070 con 0x7f22d015db10 2023-10-06T10:44:17.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.684+0000 7f22ce7fc700 1 -- 172.21.15.92:0/3892395597 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c800bc30 con 0x7f22d015db10 2023-10-06T10:44:17.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.684+0000 7f22ce7fc700 1 -- 172.21.15.92:0/3892395597 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c8012430 con 0x7f22d015db10 2023-10-06T10:44:17.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.684+0000 7f22d5a36700 1 -- 172.21.15.92:0/3892395597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22d015db10 msgr2=0x7f22d00ca0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.684+0000 7f22d5a36700 1 --2- 172.21.15.92:0/3892395597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22d015db10 0x7f22d00ca0c0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f22d00cacb0 tx=0x7f22c8009780 comp rx=0 tx=0).stop 2023-10-06T10:44:17.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.684+0000 7f22d5a36700 1 -- 172.21.15.92:0/3892395597 shutdown_connections 2023-10-06T10:44:17.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.684+0000 7f22d5a36700 1 --2- 172.21.15.92:0/3892395597 >> v2:172.21.15.124:3300/0 conn(0x7f22d00ad210 0x7f22d00cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.684+0000 7f22d5a36700 1 --2- 172.21.15.92:0/3892395597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22d015db10 0x7f22d00ca0c0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.684+0000 7f22d5a36700 1 -- 172.21.15.92:0/3892395597 >> 172.21.15.92:0/3892395597 conn(0x7f22d0067d90 msgr2=0x7f22d00ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:17.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.684+0000 7f22d5a36700 1 -- 172.21.15.92:0/3892395597 shutdown_connections 2023-10-06T10:44:17.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.684+0000 7f22d5a36700 1 -- 172.21.15.92:0/3892395597 wait complete. 2023-10-06T10:44:17.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.686+0000 7f22d5a36700 1 Processor -- start 2023-10-06T10:44:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.686+0000 7f22d5a36700 1 -- start start 2023-10-06T10:44:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22d5a36700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22d00ad210 0x7f22d014c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22d5a36700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22d00ad5e0 0x7f22d0069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22d5a36700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22d015db10 0x7f22d00adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22d5a36700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d015afa0 con 0x7f22d00ad5e0 2023-10-06T10:44:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22d5a36700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d00c6170 con 0x7f22d015db10 2023-10-06T10:44:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22d5a36700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d00c5280 con 0x7f22d00ad210 2023-10-06T10:44:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22c67fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22d00ad5e0 0x7f22d0069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22c67fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22d00ad5e0 0x7f22d0069660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48768/0 (socket says 172.21.15.92:48768) 2023-10-06T10:44:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22c67fc700 1 -- 172.21.15.92:0/2489561516 learned_addr learned my addr 172.21.15.92:0/2489561516 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22ceffd700 1 --2- 172.21.15.92:0/2489561516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22d00ad210 0x7f22d014c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22cf7fe700 1 --2- 172.21.15.92:0/2489561516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22d015db10 0x7f22d00adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22c67fc700 1 -- 172.21.15.92:0/2489561516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22d00ad210 msgr2=0x7f22d014c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22c67fc700 1 --2- 172.21.15.92:0/2489561516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22d00ad210 0x7f22d014c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22c67fc700 1 -- 172.21.15.92:0/2489561516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22d015db10 msgr2=0x7f22d00adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22c67fc700 1 --2- 172.21.15.92:0/2489561516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22d015db10 0x7f22d00adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22c67fc700 1 -- 172.21.15.92:0/2489561516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c8009a20 con 0x7f22d00ad5e0 2023-10-06T10:44:17.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22cf7fe700 1 --2- 172.21.15.92:0/2489561516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22d015db10 0x7f22d00adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:17.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22ceffd700 1 --2- 172.21.15.92:0/2489561516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22d00ad210 0x7f22d014c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:17.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22c67fc700 1 --2- 172.21.15.92:0/2489561516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22d00ad5e0 0x7f22d0069660 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f22b800c7e0 tx=0x7f22b8007d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:17.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22c77fe700 1 -- 172.21.15.92:0/2489561516 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b8005c00 con 0x7f22d00ad5e0 2023-10-06T10:44:17.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22c77fe700 1 -- 172.21.15.92:0/2489561516 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b8005d80 con 0x7f22d00ad5e0 2023-10-06T10:44:17.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22d5a36700 1 -- 172.21.15.92:0/2489561516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d00c5560 con 0x7f22d00ad5e0 2023-10-06T10:44:17.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22c77fe700 1 -- 172.21.15.92:0/2489561516 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b8007320 con 0x7f22d00ad5e0 2023-10-06T10:44:17.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.687+0000 7f22d5a36700 1 -- 172.21.15.92:0/2489561516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d00c5b20 con 0x7f22d00ad5e0 2023-10-06T10:44:17.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.688+0000 7f22d5a36700 1 -- 172.21.15.92:0/2489561516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d0064a90 con 0x7f22d00ad5e0 2023-10-06T10:44:17.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.689+0000 7f22c77fe700 1 -- 172.21.15.92:0/2489561516 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f22b801b3b0 con 0x7f22d00ad5e0 2023-10-06T10:44:17.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.689+0000 7f22c77fe700 1 --2- 172.21.15.92:0/2489561516 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f22b0066f30 0x7f22b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:17.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.689+0000 7f22c77fe700 1 -- 172.21.15.92:0/2489561516 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f22b8083d80 con 0x7f22d00ad5e0 2023-10-06T10:44:17.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.689+0000 7f22ceffd700 1 --2- 172.21.15.92:0/2489561516 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f22b0066f30 0x7f22b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:17.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.689+0000 7f22ceffd700 1 --2- 172.21.15.92:0/2489561516 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f22b0066f30 0x7f22b00693e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f22c0006fd0 tx=0x7f22c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:17.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.693+0000 7f22c77fe700 1 -- 172.21.15.92:0/2489561516 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22b8050d50 con 0x7f22d00ad5e0 2023-10-06T10:44:17.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.951+0000 7f22d5a36700 1 -- 172.21.15.92:0/2489561516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f22d00ccd70 con 0x7f22d00ad5e0 2023-10-06T10:44:17.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.952+0000 7f22c77fe700 1 -- 172.21.15.92:0/2489561516 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1300 (secure 0 0 0) 0x7f22b8017080 con 0x7f22d00ad5e0 2023-10-06T10:44:17.954 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:17.954 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":379,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":698400,"bytes_used":2445787136,"bytes_avail":770648326144,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":11773,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":91,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"a","status":"up:active","gid":7686},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7689},{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":7701}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2023-10-06T10:42:52.143574+0000","services":{}},"progress_events":{}} 2023-10-06T10:44:17.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.954+0000 7f22d5a36700 1 -- 172.21.15.92:0/2489561516 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f22b0066f30 msgr2=0x7f22b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.954+0000 7f22d5a36700 1 --2- 172.21.15.92:0/2489561516 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f22b0066f30 0x7f22b00693e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f22c0006fd0 tx=0x7f22c0008040 comp rx=0 tx=0).stop 2023-10-06T10:44:17.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.954+0000 7f22d5a36700 1 -- 172.21.15.92:0/2489561516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22d00ad5e0 msgr2=0x7f22d0069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:17.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.954+0000 7f22d5a36700 1 --2- 172.21.15.92:0/2489561516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22d00ad5e0 0x7f22d0069660 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f22b800c7e0 tx=0x7f22b8007d70 comp rx=0 tx=0).stop 2023-10-06T10:44:17.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.954+0000 7f22d5a36700 1 -- 172.21.15.92:0/2489561516 shutdown_connections 2023-10-06T10:44:17.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.954+0000 7f22d5a36700 1 --2- 172.21.15.92:0/2489561516 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f22b0066f30 0x7f22b00693e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.954+0000 7f22d5a36700 1 --2- 172.21.15.92:0/2489561516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22d015db10 0x7f22d00adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.954+0000 7f22d5a36700 1 --2- 172.21.15.92:0/2489561516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22d00ad210 0x7f22d014c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.954+0000 7f22d5a36700 1 --2- 172.21.15.92:0/2489561516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22d00ad5e0 0x7f22d0069660 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:17.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.954+0000 7f22d5a36700 1 -- 172.21.15.92:0/2489561516 >> 172.21.15.92:0/2489561516 conn(0x7f22d0067d90 msgr2=0x7f22d00ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:17.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.954+0000 7f22d5a36700 1 -- 172.21.15.92:0/2489561516 shutdown_connections 2023-10-06T10:44:17.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:17.954+0000 7f22d5a36700 1 -- 172.21.15.92:0/2489561516 wait complete. 2023-10-06T10:44:18.968 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:44:19.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.090+0000 7f568d7bc700 1 Processor -- start 2023-10-06T10:44:19.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.090+0000 7f568d7bc700 1 -- start start 2023-10-06T10:44:19.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.090+0000 7f568d7bc700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f56880b49a0 0x7f56880aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:19.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.090+0000 7f568d7bc700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56880b0c60 con 0x7f5688159560 2023-10-06T10:44:19.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.090+0000 7f568d7bc700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56880b0f40 con 0x7f56880b0760 2023-10-06T10:44:19.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.090+0000 7f568d7bc700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56880b1220 con 0x7f56880b49a0 2023-10-06T10:44:19.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f5686ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5688159560 0x7f56880aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44634/0 (socket says 172.21.15.92:44634) 2023-10-06T10:44:19.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f5686ffd700 1 -- 172.21.15.92:0/3422160388 learned_addr learned my addr 172.21.15.92:0/3422160388 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:19.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f567e7fc700 1 --2- 172.21.15.92:0/3422160388 >> v2:172.21.15.124:3300/0 conn(0x7f56880b49a0 0x7f56880aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:19.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f56867fc700 1 -- 172.21.15.92:0/3422160388 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3900310998 0 0) 0x7f56880b0c60 con 0x7f5688159560 2023-10-06T10:44:19.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f56867fc700 1 -- 172.21.15.92:0/3422160388 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5674003570 con 0x7f5688159560 2023-10-06T10:44:19.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f56867fc700 1 -- 172.21.15.92:0/3422160388 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1841908859 0 0) 0x7f5674003570 con 0x7f5688159560 2023-10-06T10:44:19.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f56867fc700 1 -- 172.21.15.92:0/3422160388 >> v2:172.21.15.124:3300/0 conn(0x7f56880b49a0 msgr2=0x7f56880aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:19.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f56867fc700 1 --2- 172.21.15.92:0/3422160388 >> v2:172.21.15.124:3300/0 conn(0x7f56880b49a0 0x7f56880aac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:19.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f56867fc700 1 -- 172.21.15.92:0/3422160388 >> v1:172.21.15.124:6789/0 conn(0x7f56880b0760 legacy=0x7f56880b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:19.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f56867fc700 1 -- 172.21.15.92:0/3422160388 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56880b1500 con 0x7f5688159560 2023-10-06T10:44:19.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f56867fc700 1 -- 172.21.15.92:0/3422160388 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f56780015f0 con 0x7f5688159560 2023-10-06T10:44:19.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f56867fc700 1 -- 172.21.15.92:0/3422160388 >> v1:172.21.15.92:6789/0 conn(0x7f5688159560 legacy=0x7f56880aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:19.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f56867fc700 1 --2- 172.21.15.92:0/3422160388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5674003a90 0x7f5674003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:19.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f56867fc700 1 --2- 172.21.15.92:0/3422160388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5674004810 0x7f5674004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:19.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f56867fc700 1 --2- 172.21.15.92:0/3422160388 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5674005210 0x7f567400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:19.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f56867fc700 1 -- 172.21.15.92:0/3422160388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56880b1220 con 0x7f5674004810 2023-10-06T10:44:19.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f56867fc700 1 -- 172.21.15.92:0/3422160388 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567400c130 con 0x7f5674005210 2023-10-06T10:44:19.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f56867fc700 1 -- 172.21.15.92:0/3422160388 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567400c2c0 con 0x7f5674003a90 2023-10-06T10:44:19.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f567e7fc700 1 --2- 172.21.15.92:0/3422160388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5674004810 0x7f5674004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:19.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f5686ffd700 1 --2- 172.21.15.92:0/3422160388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5674003a90 0x7f5674003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:19.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f56877fe700 1 --2- 172.21.15.92:0/3422160388 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5674005210 0x7f567400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:19.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f567e7fc700 1 -- 172.21.15.92:0/3422160388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5674003a90 msgr2=0x7f5674003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:19.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f567e7fc700 1 --2- 172.21.15.92:0/3422160388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5674003a90 0x7f5674003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:19.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f567e7fc700 1 -- 172.21.15.92:0/3422160388 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5674005210 msgr2=0x7f567400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:19.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f567e7fc700 1 --2- 172.21.15.92:0/3422160388 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5674005210 0x7f567400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:19.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f56877fe700 1 -- 172.21.15.92:0/3422160388 reap_dead start 2023-10-06T10:44:19.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.091+0000 7f567e7fc700 1 -- 172.21.15.92:0/3422160388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5674003570 con 0x7f5674004810 2023-10-06T10:44:19.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.092+0000 7f567e7fc700 1 --2- 172.21.15.92:0/3422160388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5674004810 0x7f5674004c80 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f5688088de0 tx=0x7f56700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4bc837415e5e4c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:19.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.092+0000 7f56867fc700 1 -- 172.21.15.92:0/3422160388 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567000e6e0 con 0x7f5674004810 2023-10-06T10:44:19.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.092+0000 7f56867fc700 1 -- 172.21.15.92:0/3422160388 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567000e840 con 0x7f5674004810 2023-10-06T10:44:19.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.092+0000 7f56867fc700 1 -- 172.21.15.92:0/3422160388 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567000ec00 con 0x7f5674004810 2023-10-06T10:44:19.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.092+0000 7f568d7bc700 1 -- 172.21.15.92:0/3422160388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5674004810 msgr2=0x7f5674004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:19.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.092+0000 7f568d7bc700 1 --2- 172.21.15.92:0/3422160388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5674004810 0x7f5674004c80 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f5688088de0 tx=0x7f56700091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:19.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.092+0000 7f568d7bc700 1 -- 172.21.15.92:0/3422160388 shutdown_connections 2023-10-06T10:44:19.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.092+0000 7f568d7bc700 1 --2- 172.21.15.92:0/3422160388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5674004810 0x7f5674004c80 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:19.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.092+0000 7f568d7bc700 1 -- 172.21.15.92:0/3422160388 >> 172.21.15.92:0/3422160388 conn(0x7f56880997f0 msgr2=0x7f568806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:19.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.092+0000 7f568d7bc700 1 -- 172.21.15.92:0/3422160388 shutdown_connections 2023-10-06T10:44:19.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.092+0000 7f568d7bc700 1 -- 172.21.15.92:0/3422160388 wait complete. 2023-10-06T10:44:19.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.094+0000 7f568d7bc700 1 Processor -- start 2023-10-06T10:44:19.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.094+0000 7f568d7bc700 1 -- start start 2023-10-06T10:44:19.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.094+0000 7f568d7bc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5674004810 0x7f568815caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:19.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.094+0000 7f568d7bc700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f568815a5c0 0x7f568816d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:19.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.094+0000 7f568d7bc700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f568816daf0 0x7f56880ce150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:19.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.094+0000 7f568d7bc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568816df70 con 0x7f5674004810 2023-10-06T10:44:19.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.094+0000 7f568d7bc700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568816e450 con 0x7f568815a5c0 2023-10-06T10:44:19.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.094+0000 7f568d7bc700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5688155eb0 con 0x7f568816daf0 2023-10-06T10:44:19.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.094+0000 7f5686ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5674004810 0x7f568815caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:19.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.094+0000 7f5686ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5674004810 0x7f568815caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48786/0 (socket says 172.21.15.92:48786) 2023-10-06T10:44:19.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.094+0000 7f5686ffd700 1 -- 172.21.15.92:0/4090115772 learned_addr learned my addr 172.21.15.92:0/4090115772 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:19.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.095+0000 7f56877fe700 1 --2- 172.21.15.92:0/4090115772 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f568816daf0 0x7f56880ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:19.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.095+0000 7f567e7fc700 1 --2- 172.21.15.92:0/4090115772 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f568815a5c0 0x7f568816d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:19.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.095+0000 7f5686ffd700 1 -- 172.21.15.92:0/4090115772 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f568816daf0 msgr2=0x7f56880ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:19.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.095+0000 7f5686ffd700 1 --2- 172.21.15.92:0/4090115772 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f568816daf0 0x7f56880ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:19.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.095+0000 7f5686ffd700 1 -- 172.21.15.92:0/4090115772 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f568815a5c0 msgr2=0x7f568816d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:19.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.095+0000 7f5686ffd700 1 --2- 172.21.15.92:0/4090115772 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f568815a5c0 0x7f568816d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:19.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.095+0000 7f5686ffd700 1 -- 172.21.15.92:0/4090115772 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568816e170 con 0x7f5674004810 2023-10-06T10:44:19.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.095+0000 7f56877fe700 1 --2- 172.21.15.92:0/4090115772 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f568816daf0 0x7f56880ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:19.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.095+0000 7f567e7fc700 1 --2- 172.21.15.92:0/4090115772 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f568815a5c0 0x7f568816d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:19.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.095+0000 7f5686ffd700 1 --2- 172.21.15.92:0/4090115772 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5674004810 0x7f568815caf0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f56780011d0 tx=0x7f5678009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:19.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.095+0000 7f567f7fe700 1 -- 172.21.15.92:0/4090115772 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56780072e0 con 0x7f5674004810 2023-10-06T10:44:19.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.095+0000 7f567f7fe700 1 -- 172.21.15.92:0/4090115772 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5678007440 con 0x7f5674004810 2023-10-06T10:44:19.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.095+0000 7f567f7fe700 1 -- 172.21.15.92:0/4090115772 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56780057f0 con 0x7f5674004810 2023-10-06T10:44:19.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.095+0000 7f568d7bc700 1 -- 172.21.15.92:0/4090115772 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5688159ae0 con 0x7f5674004810 2023-10-06T10:44:19.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.095+0000 7f568d7bc700 1 -- 172.21.15.92:0/4090115772 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568815a0a0 con 0x7f5674004810 2023-10-06T10:44:19.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.096+0000 7f568d7bc700 1 -- 172.21.15.92:0/4090115772 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568815a2c0 con 0x7f5674004810 2023-10-06T10:44:19.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.096+0000 7f567f7fe700 1 -- 172.21.15.92:0/4090115772 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5678005b80 con 0x7f5674004810 2023-10-06T10:44:19.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.097+0000 7f567f7fe700 1 --2- 172.21.15.92:0/4090115772 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5668066f30 0x7f56680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:19.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.097+0000 7f567f7fe700 1 -- 172.21.15.92:0/4090115772 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f56780849e0 con 0x7f5674004810 2023-10-06T10:44:19.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.097+0000 7f567e7fc700 1 --2- 172.21.15.92:0/4090115772 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5668066f30 0x7f56680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:19.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.097+0000 7f567e7fc700 1 --2- 172.21.15.92:0/4090115772 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5668066f30 0x7f56680693e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f56700091e0 tx=0x7f567001a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:19.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.101+0000 7f567f7fe700 1 -- 172.21.15.92:0/4090115772 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f56780519b0 con 0x7f5674004810 2023-10-06T10:44:19.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.327+0000 7f568d7bc700 1 -- 172.21.15.92:0/4090115772 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5688064a90 con 0x7f5674004810 2023-10-06T10:44:19.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.328+0000 7f567f7fe700 1 -- 172.21.15.92:0/4090115772 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4640 (secure 0 0 0) 0x7f5678054fd0 con 0x7f5674004810 2023-10-06T10:44:19.330 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:19.330 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":91,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2122441982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2122441982},{"type":"v1","addr":"172.21.15.124:6835","nonce":2122441982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:47.850832+0000","modified":"2023-10-06T10:44:17.577192+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7701,"mds_1":7689,"mds_2":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6836/3605297930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3605297930},{"type":"v1","addr":"172.21.15.92:6836","nonce":3605297930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7689":{"gid":7689,"name":"c","rank":1,"incarnation":85,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/4047304116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4047304116},{"type":"v1","addr":"172.21.15.92:6837","nonce":4047304116}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7701":{"gid":7701,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6834/4203118992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":4203118992},{"type":"v1","addr":"172.21.15.124:6834","nonce":4203118992}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T10:44:19.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.330+0000 7f568d7bc700 1 -- 172.21.15.92:0/4090115772 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5668066f30 msgr2=0x7f56680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:19.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.330+0000 7f568d7bc700 1 --2- 172.21.15.92:0/4090115772 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5668066f30 0x7f56680693e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f56700091e0 tx=0x7f567001a000 comp rx=0 tx=0).stop 2023-10-06T10:44:19.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.330+0000 7f568d7bc700 1 -- 172.21.15.92:0/4090115772 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5674004810 msgr2=0x7f568815caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:19.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.330+0000 7f568d7bc700 1 --2- 172.21.15.92:0/4090115772 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5674004810 0x7f568815caf0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f56780011d0 tx=0x7f5678009040 comp rx=0 tx=0).stop 2023-10-06T10:44:19.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.330+0000 7f568d7bc700 1 -- 172.21.15.92:0/4090115772 shutdown_connections 2023-10-06T10:44:19.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.330+0000 7f568d7bc700 1 --2- 172.21.15.92:0/4090115772 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5668066f30 0x7f56680693e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:19.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.330+0000 7f568d7bc700 1 --2- 172.21.15.92:0/4090115772 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f568816daf0 0x7f56880ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:19.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.330+0000 7f568d7bc700 1 --2- 172.21.15.92:0/4090115772 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5674004810 0x7f568815caf0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:19.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.330+0000 7f568d7bc700 1 --2- 172.21.15.92:0/4090115772 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f568815a5c0 0x7f568816d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:19.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.330+0000 7f568d7bc700 1 -- 172.21.15.92:0/4090115772 >> 172.21.15.92:0/4090115772 conn(0x7f56880997f0 msgr2=0x7f5688157850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:19.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.330+0000 7f568d7bc700 1 -- 172.21.15.92:0/4090115772 shutdown_connections 2023-10-06T10:44:19.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.330+0000 7f568d7bc700 1 -- 172.21.15.92:0/4090115772 wait complete. 2023-10-06T10:44:19.333 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 91 2023-10-06T10:44:19.344 DEBUG:tasks.cephfs.test_multimds_misc:current active=['b', 'c', 'a'] 2023-10-06T10:44:19.344 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:44:19.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.455+0000 7feeb8960700 1 Processor -- start 2023-10-06T10:44:19.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.455+0000 7feeb8960700 1 -- start start 2023-10-06T10:44:19.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.456+0000 7feeb8960700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7feeb4159560 0x7feeb40aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:19.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.456+0000 7feeb8960700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7feeb40b49a0 0x7feeb40aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:19.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.456+0000 7feeb8960700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feeb40b0760 0x7feeb40b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:19.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.456+0000 7feeb8960700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb40b0c60 con 0x7feeb4159560 2023-10-06T10:44:19.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.456+0000 7feeb8960700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb40b0f40 con 0x7feeb40b49a0 2023-10-06T10:44:19.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.456+0000 7feeb8960700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb40b1220 con 0x7feeb40b0760 2023-10-06T10:44:19.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.456+0000 7feeb259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7feeb4159560 0x7feeb40aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:19.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.456+0000 7feeb259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7feeb4159560 0x7feeb40aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48802/0 (socket says 172.21.15.92:48802) 2023-10-06T10:44:19.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.456+0000 7feeb259c700 1 -- 172.21.15.92:0/348968801 learned_addr learned my addr 172.21.15.92:0/348968801 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:19.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.456+0000 7feeb259c700 1 -- 172.21.15.92:0/348968801 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feeb40b0760 msgr2=0x7feeb40b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:19.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.456+0000 7feeb2d9d700 1 --2- 172.21.15.92:0/348968801 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feeb40b0760 0x7feeb40b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.456+0000 7feeb259c700 1 --2- 172.21.15.92:0/348968801 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feeb40b0760 0x7feeb40b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.456+0000 7feea9d9b700 1 --2- 172.21.15.92:0/348968801 >> v2:172.21.15.124:3300/0 conn(0x7feeb40b49a0 0x7feeb40aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.456+0000 7feeb259c700 1 -- 172.21.15.92:0/348968801 >> v2:172.21.15.124:3300/0 conn(0x7feeb40b49a0 msgr2=0x7feeb40aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.456+0000 7feeb259c700 1 --2- 172.21.15.92:0/348968801 >> v2:172.21.15.124:3300/0 conn(0x7feeb40b49a0 0x7feeb40aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.456+0000 7feeb259c700 1 -- 172.21.15.92:0/348968801 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb40b1500 con 0x7feeb4159560 2023-10-06T10:44:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.456+0000 7feea9d9b700 1 --2- 172.21.15.92:0/348968801 >> v2:172.21.15.124:3300/0 conn(0x7feeb40b49a0 0x7feeb40aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.456+0000 7feeb259c700 1 --2- 172.21.15.92:0/348968801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeb4159560 0x7feeb40aa530 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7feea4009990 tx=0x7feea4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94bd3f5a562203c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.457+0000 7feeb1d9b700 1 -- 172.21.15.92:0/348968801 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4005c10 con 0x7feeb4159560 2023-10-06T10:44:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.457+0000 7feeb1d9b700 1 -- 172.21.15.92:0/348968801 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea4005d70 con 0x7feeb4159560 2023-10-06T10:44:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.457+0000 7feeb1d9b700 1 -- 172.21.15.92:0/348968801 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea400b840 con 0x7feeb4159560 2023-10-06T10:44:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.457+0000 7feeb8960700 1 -- 172.21.15.92:0/348968801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeb4159560 msgr2=0x7feeb40aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.457+0000 7feeb8960700 1 --2- 172.21.15.92:0/348968801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeb4159560 0x7feeb40aa530 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7feea4009990 tx=0x7feea4009220 comp rx=0 tx=0).stop 2023-10-06T10:44:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.457+0000 7feeb8960700 1 -- 172.21.15.92:0/348968801 shutdown_connections 2023-10-06T10:44:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.457+0000 7feeb8960700 1 --2- 172.21.15.92:0/348968801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeb4159560 0x7feeb40aa530 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.457+0000 7feeb8960700 1 --2- 172.21.15.92:0/348968801 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feeb40b0760 0x7feeb40b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.457+0000 7feeb8960700 1 --2- 172.21.15.92:0/348968801 >> v2:172.21.15.124:3300/0 conn(0x7feeb40b49a0 0x7feeb40aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.457+0000 7feeb8960700 1 -- 172.21.15.92:0/348968801 >> 172.21.15.92:0/348968801 conn(0x7feeb40997f0 msgr2=0x7feeb406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.457+0000 7feeb8960700 1 -- 172.21.15.92:0/348968801 shutdown_connections 2023-10-06T10:44:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.457+0000 7feeb8960700 1 -- 172.21.15.92:0/348968801 wait complete. 2023-10-06T10:44:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.458+0000 7feeb8960700 1 Processor -- start 2023-10-06T10:44:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.458+0000 7feeb8960700 1 -- start start 2023-10-06T10:44:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.459+0000 7feeb8960700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeb40b0760 0x7feeb4151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:19.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.459+0000 7feeb8960700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feeb40b49a0 0x7feeb41599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:19.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.459+0000 7feeb8960700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feeb4159560 0x7feeb415b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:19.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.459+0000 7feeb8960700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb4154f40 con 0x7feeb40b0760 2023-10-06T10:44:19.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.459+0000 7feeb8960700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb415bdd0 con 0x7feeb40b49a0 2023-10-06T10:44:19.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.459+0000 7feeb8960700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb415c0b0 con 0x7feeb4159560 2023-10-06T10:44:19.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.459+0000 7feeb259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeb40b0760 0x7feeb4151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:19.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.459+0000 7feeb259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeb40b0760 0x7feeb4151030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48816/0 (socket says 172.21.15.92:48816) 2023-10-06T10:44:19.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.459+0000 7feeb259c700 1 -- 172.21.15.92:0/1154471805 learned_addr learned my addr 172.21.15.92:0/1154471805 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:19.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.459+0000 7feea9d9b700 1 --2- 172.21.15.92:0/1154471805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feeb40b49a0 0x7feeb41599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:19.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.459+0000 7feeb2d9d700 1 --2- 172.21.15.92:0/1154471805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feeb4159560 0x7feeb415b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:19.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.460+0000 7feeb259c700 1 -- 172.21.15.92:0/1154471805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feeb4159560 msgr2=0x7feeb415b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:19.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.460+0000 7feeb259c700 1 --2- 172.21.15.92:0/1154471805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feeb4159560 0x7feeb415b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:19.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.460+0000 7feeb259c700 1 -- 172.21.15.92:0/1154471805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feeb40b49a0 msgr2=0x7feeb41599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:19.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.460+0000 7feeb259c700 1 --2- 172.21.15.92:0/1154471805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feeb40b49a0 0x7feeb41599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:19.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.460+0000 7feeb259c700 1 -- 172.21.15.92:0/1154471805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea4009480 con 0x7feeb40b0760 2023-10-06T10:44:19.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.460+0000 7feeb259c700 1 --2- 172.21.15.92:0/1154471805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeb40b0760 0x7feeb4151030 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7feeb4067c20 tx=0x7feea4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:19.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.460+0000 7feeaaffd700 1 -- 172.21.15.92:0/1154471805 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4011940 con 0x7feeb40b0760 2023-10-06T10:44:19.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.460+0000 7feeaaffd700 1 -- 172.21.15.92:0/1154471805 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea4011aa0 con 0x7feeb40b0760 2023-10-06T10:44:19.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.460+0000 7feeb8960700 1 -- 172.21.15.92:0/1154471805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb415c390 con 0x7feeb40b0760 2023-10-06T10:44:19.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.460+0000 7feeaaffd700 1 -- 172.21.15.92:0/1154471805 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4011e60 con 0x7feeb40b0760 2023-10-06T10:44:19.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.460+0000 7feeb8960700 1 -- 172.21.15.92:0/1154471805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb4156ad0 con 0x7feeb40b0760 2023-10-06T10:44:19.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.461+0000 7feeb8960700 1 -- 172.21.15.92:0/1154471805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb4064a90 con 0x7feeb40b0760 2023-10-06T10:44:19.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.461+0000 7feeaaffd700 1 -- 172.21.15.92:0/1154471805 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feea401b640 con 0x7feeb40b0760 2023-10-06T10:44:19.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.461+0000 7feeaaffd700 1 --2- 172.21.15.92:0/1154471805 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fee94066f30 0x7fee940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:19.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.462+0000 7feeaaffd700 1 -- 172.21.15.92:0/1154471805 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7feea4086430 con 0x7feeb40b0760 2023-10-06T10:44:19.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.462+0000 7feea9d9b700 1 --2- 172.21.15.92:0/1154471805 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fee94066f30 0x7fee940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:19.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.462+0000 7feea9d9b700 1 --2- 172.21.15.92:0/1154471805 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fee94066f30 0x7fee940693e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7feeb415b030 tx=0x7fee9c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:19.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.466+0000 7feeaaffd700 1 -- 172.21.15.92:0/1154471805 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feea4053400 con 0x7feeb40b0760 2023-10-06T10:44:19.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.728+0000 7feeb8960700 1 -- 172.21.15.92:0/1154471805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7feeb40b17e0 con 0x7feeb40b0760 2023-10-06T10:44:19.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.728+0000 7feeaaffd700 1 -- 172.21.15.92:0/1154471805 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1888 (secure 0 0 0) 0x7feea402a770 con 0x7feeb40b0760 2023-10-06T10:44:19.730 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:19.731 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":381,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":698400,"bytes_used":2445787136,"bytes_avail":770648326144,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":9555,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":91,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"a","status":"up:active","gid":7686},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7689},{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":7701}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2023-10-06T10:44:18.160883+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T10:44:19.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.730+0000 7feeb8960700 1 -- 172.21.15.92:0/1154471805 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fee94066f30 msgr2=0x7fee940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:19.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.730+0000 7feeb8960700 1 --2- 172.21.15.92:0/1154471805 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fee94066f30 0x7fee940693e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7feeb415b030 tx=0x7fee9c008040 comp rx=0 tx=0).stop 2023-10-06T10:44:19.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.730+0000 7feeb8960700 1 -- 172.21.15.92:0/1154471805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeb40b0760 msgr2=0x7feeb4151030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:19.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.730+0000 7feeb8960700 1 --2- 172.21.15.92:0/1154471805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeb40b0760 0x7feeb4151030 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7feeb4067c20 tx=0x7feea4016000 comp rx=0 tx=0).stop 2023-10-06T10:44:19.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.730+0000 7feeb8960700 1 -- 172.21.15.92:0/1154471805 shutdown_connections 2023-10-06T10:44:19.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.730+0000 7feeb8960700 1 --2- 172.21.15.92:0/1154471805 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fee94066f30 0x7fee940693e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:19.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.730+0000 7feeb8960700 1 --2- 172.21.15.92:0/1154471805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feeb4159560 0x7feeb415b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:19.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.730+0000 7feeb8960700 1 --2- 172.21.15.92:0/1154471805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeb40b0760 0x7feeb4151030 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:19.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.730+0000 7feeb8960700 1 --2- 172.21.15.92:0/1154471805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feeb40b49a0 0x7feeb41599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:19.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.730+0000 7feeb8960700 1 -- 172.21.15.92:0/1154471805 >> 172.21.15.92:0/1154471805 conn(0x7feeb40997f0 msgr2=0x7feeb40b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:19.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.730+0000 7feeb8960700 1 -- 172.21.15.92:0/1154471805 shutdown_connections 2023-10-06T10:44:19.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:19.730+0000 7feeb8960700 1 -- 172.21.15.92:0/1154471805 wait complete. 2023-10-06T10:44:19.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]'} 2023-10-06T10:44:20.745 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:44:20.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.865+0000 7f433b59e700 1 Processor -- start 2023-10-06T10:44:20.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.865+0000 7f433b59e700 1 -- start start 2023-10-06T10:44:20.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.866+0000 7f433b59e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f433c0ab410 0x7f433c0afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:20.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.866+0000 7f433b59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f433c0686c0 con 0x7f433c159390 2023-10-06T10:44:20.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.866+0000 7f433b59e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f433c068c80 con 0x7f433c159760 2023-10-06T10:44:20.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.866+0000 7f433b59e700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f433c0ad0f0 con 0x7f433c0ab410 2023-10-06T10:44:20.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.866+0000 7f433a59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f433c159390 0x7f433c0ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44638/0 (socket says 172.21.15.92:44638) 2023-10-06T10:44:20.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.866+0000 7f433a59c700 1 -- 172.21.15.92:0/704500541 learned_addr learned my addr 172.21.15.92:0/704500541 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:20.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.866+0000 7f433959a700 1 -- 172.21.15.92:0/704500541 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782060109 0 0) 0x7f433c0686c0 con 0x7f433c159390 2023-10-06T10:44:20.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f433959a700 1 -- 172.21.15.92:0/704500541 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4324003570 con 0x7f433c159390 2023-10-06T10:44:20.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f433ad9d700 1 --2- 172.21.15.92:0/704500541 >> v2:172.21.15.124:3300/0 conn(0x7f433c0ab410 0x7f433c0afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:20.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f433959a700 1 -- 172.21.15.92:0/704500541 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 523360124 0 0) 0x7f4324003570 con 0x7f433c159390 2023-10-06T10:44:20.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f433959a700 1 -- 172.21.15.92:0/704500541 >> v2:172.21.15.124:3300/0 conn(0x7f433c0ab410 msgr2=0x7f433c0afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:20.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f433959a700 1 --2- 172.21.15.92:0/704500541 >> v2:172.21.15.124:3300/0 conn(0x7f433c0ab410 0x7f433c0afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:20.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f433959a700 1 -- 172.21.15.92:0/704500541 >> v1:172.21.15.124:6789/0 conn(0x7f433c159760 legacy=0x7f433c0aad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:20.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f433959a700 1 -- 172.21.15.92:0/704500541 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c0ad3d0 con 0x7f433c159390 2023-10-06T10:44:20.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f433959a700 1 -- 172.21.15.92:0/704500541 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f432c0017f0 con 0x7f433c159390 2023-10-06T10:44:20.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f433959a700 1 -- 172.21.15.92:0/704500541 >> v1:172.21.15.92:6789/0 conn(0x7f433c159390 legacy=0x7f433c0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:20.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f433959a700 1 --2- 172.21.15.92:0/704500541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4324003a60 0x7f4324003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:20.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f433959a700 1 --2- 172.21.15.92:0/704500541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4324004810 0x7f4324004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:20.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f433959a700 1 --2- 172.21.15.92:0/704500541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43240051c0 0x7f432400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:20.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f433959a700 1 -- 172.21.15.92:0/704500541 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c0ad0f0 con 0x7f4324004810 2023-10-06T10:44:20.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f433959a700 1 -- 172.21.15.92:0/704500541 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432400c0d0 con 0x7f4324003a60 2023-10-06T10:44:20.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f433959a700 1 -- 172.21.15.92:0/704500541 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432400c290 con 0x7f43240051c0 2023-10-06T10:44:20.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f4339d9b700 1 --2- 172.21.15.92:0/704500541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4324004810 0x7f4324004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:20.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f433a59c700 1 --2- 172.21.15.92:0/704500541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4324003a60 0x7f4324003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:20.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f433ad9d700 1 --2- 172.21.15.92:0/704500541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43240051c0 0x7f432400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:20.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f4339d9b700 1 -- 172.21.15.92:0/704500541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43240051c0 msgr2=0x7f432400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:20.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f4339d9b700 1 --2- 172.21.15.92:0/704500541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43240051c0 0x7f432400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:20.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f4339d9b700 1 -- 172.21.15.92:0/704500541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4324003a60 msgr2=0x7f4324003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:20.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f4339d9b700 1 --2- 172.21.15.92:0/704500541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4324003a60 0x7f4324003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:20.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f433ad9d700 1 -- 172.21.15.92:0/704500541 reap_dead start 2023-10-06T10:44:20.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f4339d9b700 1 -- 172.21.15.92:0/704500541 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4324003570 con 0x7f4324004810 2023-10-06T10:44:20.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.867+0000 7f4339d9b700 1 --2- 172.21.15.92:0/704500541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4324004810 0x7f4324004c80 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f43300018b0 tx=0x7f4330007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d38678dcf4e05dc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:20.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.868+0000 7f433959a700 1 -- 172.21.15.92:0/704500541 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4330012420 con 0x7f4324004810 2023-10-06T10:44:20.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.868+0000 7f433959a700 1 -- 172.21.15.92:0/704500541 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4330012580 con 0x7f4324004810 2023-10-06T10:44:20.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.868+0000 7f433959a700 1 -- 172.21.15.92:0/704500541 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4330012920 con 0x7f4324004810 2023-10-06T10:44:20.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.868+0000 7f433b59e700 1 -- 172.21.15.92:0/704500541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4324004810 msgr2=0x7f4324004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:20.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.868+0000 7f433b59e700 1 --2- 172.21.15.92:0/704500541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4324004810 0x7f4324004c80 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f43300018b0 tx=0x7f4330007d30 comp rx=0 tx=0).stop 2023-10-06T10:44:20.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.868+0000 7f433b59e700 1 -- 172.21.15.92:0/704500541 shutdown_connections 2023-10-06T10:44:20.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.868+0000 7f433b59e700 1 --2- 172.21.15.92:0/704500541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4324004810 0x7f4324004c80 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:20.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.868+0000 7f433b59e700 1 -- 172.21.15.92:0/704500541 >> 172.21.15.92:0/704500541 conn(0x7f433c097410 msgr2=0x7f433c067d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:20.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.868+0000 7f433b59e700 1 -- 172.21.15.92:0/704500541 shutdown_connections 2023-10-06T10:44:20.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.868+0000 7f433b59e700 1 -- 172.21.15.92:0/704500541 wait complete. 2023-10-06T10:44:20.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.869+0000 7f433b59e700 1 Processor -- start 2023-10-06T10:44:20.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.869+0000 7f433b59e700 1 -- start start 2023-10-06T10:44:20.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.870+0000 7f433b59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4324004810 0x7f433c0b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:20.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.870+0000 7f433b59e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f433c15be10 0x7f433c153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:20.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.870+0000 7f433b59e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f433c15a560 0x7f433c15c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:20.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.870+0000 7f433b59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c15b980 con 0x7f4324004810 2023-10-06T10:44:20.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.870+0000 7f433b59e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c15ad70 con 0x7f433c15a560 2023-10-06T10:44:20.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.870+0000 7f433b59e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433c15b050 con 0x7f433c15be10 2023-10-06T10:44:20.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.870+0000 7f433a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4324004810 0x7f433c0b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:20.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.870+0000 7f433a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4324004810 0x7f433c0b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48826/0 (socket says 172.21.15.92:48826) 2023-10-06T10:44:20.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.870+0000 7f433a59c700 1 -- 172.21.15.92:0/1482634971 learned_addr learned my addr 172.21.15.92:0/1482634971 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:20.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.870+0000 7f4339d9b700 1 --2- 172.21.15.92:0/1482634971 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f433c15be10 0x7f433c153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:20.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.870+0000 7f433ad9d700 1 --2- 172.21.15.92:0/1482634971 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f433c15a560 0x7f433c15c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:20.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.870+0000 7f433a59c700 1 -- 172.21.15.92:0/1482634971 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f433c15be10 msgr2=0x7f433c153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:20.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.870+0000 7f433a59c700 1 --2- 172.21.15.92:0/1482634971 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f433c15be10 0x7f433c153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:20.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.870+0000 7f433a59c700 1 -- 172.21.15.92:0/1482634971 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f433c15a560 msgr2=0x7f433c15c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:20.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.870+0000 7f433a59c700 1 --2- 172.21.15.92:0/1482634971 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f433c15a560 0x7f433c15c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:20.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.870+0000 7f433a59c700 1 -- 172.21.15.92:0/1482634971 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c15b330 con 0x7f4324004810 2023-10-06T10:44:20.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.870+0000 7f433a59c700 1 --2- 172.21.15.92:0/1482634971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4324004810 0x7f433c0b27e0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f432c007a90 tx=0x7f433c0aeac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:20.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.871+0000 7f432a7fc700 1 -- 172.21.15.92:0/1482634971 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c00ae40 con 0x7f4324004810 2023-10-06T10:44:20.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.871+0000 7f432a7fc700 1 -- 172.21.15.92:0/1482634971 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432c0133f0 con 0x7f4324004810 2023-10-06T10:44:20.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.871+0000 7f433b59e700 1 -- 172.21.15.92:0/1482634971 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f433c15b610 con 0x7f4324004810 2023-10-06T10:44:20.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.871+0000 7f432a7fc700 1 -- 172.21.15.92:0/1482634971 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c0137b0 con 0x7f4324004810 2023-10-06T10:44:20.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.871+0000 7f433b59e700 1 -- 172.21.15.92:0/1482634971 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f433c15c830 con 0x7f4324004810 2023-10-06T10:44:20.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.872+0000 7f433b59e700 1 -- 172.21.15.92:0/1482634971 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f433c05f230 con 0x7f4324004810 2023-10-06T10:44:20.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.872+0000 7f432a7fc700 1 -- 172.21.15.92:0/1482634971 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f432c013be0 con 0x7f4324004810 2023-10-06T10:44:20.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.872+0000 7f432a7fc700 1 --2- 172.21.15.92:0/1482634971 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f431c066f30 0x7f431c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:20.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.873+0000 7f4339d9b700 1 --2- 172.21.15.92:0/1482634971 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f431c066f30 0x7f431c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:20.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.873+0000 7f432a7fc700 1 -- 172.21.15.92:0/1482634971 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f432c085620 con 0x7f4324004810 2023-10-06T10:44:20.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.873+0000 7f4339d9b700 1 --2- 172.21.15.92:0/1482634971 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f431c066f30 0x7f431c0693e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4330001690 tx=0x7f4330015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:20.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:20.878+0000 7f432a7fc700 1 -- 172.21.15.92:0/1482634971 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f432c0525f0 con 0x7f4324004810 2023-10-06T10:44:21.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.101+0000 7f433b59e700 1 -- 172.21.15.92:0/1482634971 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f433c154420 con 0x7f4324004810 2023-10-06T10:44:21.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.101+0000 7f432a7fc700 1 -- 172.21.15.92:0/1482634971 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4642 (secure 0 0 0) 0x7f432c055c10 con 0x7f4324004810 2023-10-06T10:44:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:21.104 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2122441982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2122441982},{"type":"v1","addr":"172.21.15.124:6835","nonce":2122441982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:47.850832+0000","modified":"2023-10-06T10:44:20.587653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7701,"mds_1":7689,"mds_2":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6836/3605297930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3605297930},{"type":"v1","addr":"172.21.15.92:6836","nonce":3605297930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7689":{"gid":7689,"name":"c","rank":1,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.92:6837/4047304116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4047304116},{"type":"v1","addr":"172.21.15.92:6837","nonce":4047304116}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7701":{"gid":7701,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":11,"addr":"172.21.15.124:6834/4203118992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":4203118992},{"type":"v1","addr":"172.21.15.124:6834","nonce":4203118992}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T10:44:21.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.103+0000 7f433b59e700 1 -- 172.21.15.92:0/1482634971 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f431c066f30 msgr2=0x7f431c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:21.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.103+0000 7f433b59e700 1 --2- 172.21.15.92:0/1482634971 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f431c066f30 0x7f431c0693e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4330001690 tx=0x7f4330015000 comp rx=0 tx=0).stop 2023-10-06T10:44:21.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.103+0000 7f433b59e700 1 -- 172.21.15.92:0/1482634971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4324004810 msgr2=0x7f433c0b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:21.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.103+0000 7f433b59e700 1 --2- 172.21.15.92:0/1482634971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4324004810 0x7f433c0b27e0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f432c007a90 tx=0x7f433c0aeac0 comp rx=0 tx=0).stop 2023-10-06T10:44:21.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.104+0000 7f433b59e700 1 -- 172.21.15.92:0/1482634971 shutdown_connections 2023-10-06T10:44:21.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.104+0000 7f433b59e700 1 --2- 172.21.15.92:0/1482634971 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f431c066f30 0x7f431c0693e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:21.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.104+0000 7f433b59e700 1 --2- 172.21.15.92:0/1482634971 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f433c15a560 0x7f433c15c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:21.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.104+0000 7f433b59e700 1 --2- 172.21.15.92:0/1482634971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4324004810 0x7f433c0b27e0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:21.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.104+0000 7f433b59e700 1 --2- 172.21.15.92:0/1482634971 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f433c15be10 0x7f433c153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:21.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.104+0000 7f433b59e700 1 -- 172.21.15.92:0/1482634971 >> 172.21.15.92:0/1482634971 conn(0x7f433c097410 msgr2=0x7f433c0afc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:21.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.104+0000 7f433b59e700 1 -- 172.21.15.92:0/1482634971 shutdown_connections 2023-10-06T10:44:21.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.104+0000 7f433b59e700 1 -- 172.21.15.92:0/1482634971 wait complete. 2023-10-06T10:44:21.107 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 92 2023-10-06T10:44:21.117 DEBUG:tasks.cephfs.test_multimds_misc:current active=['b', 'c', 'a'] 2023-10-06T10:44:21.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:44:21.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.230+0000 7f89e09a2700 1 Processor -- start 2023-10-06T10:44:21.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.230+0000 7f89e09a2700 1 -- start start 2023-10-06T10:44:21.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.231+0000 7f89e09a2700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89dc0ad210 0x7f89dc0cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:21.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.231+0000 7f89e09a2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f89dc0ad5e0 0x7f89dc15d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:21.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.231+0000 7f89e09a2700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89dc0cecf0 con 0x7f89dc15d940 2023-10-06T10:44:21.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.231+0000 7f89e09a2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89dc15ddc0 con 0x7f89dc0ad5e0 2023-10-06T10:44:21.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.231+0000 7f89e09a2700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc0cefd0 con 0x7f89dc0ad210 2023-10-06T10:44:21.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.231+0000 7f89d1d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f89dc0ad5e0 0x7f89dc15d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:21.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.231+0000 7f89d1d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f89dc0ad5e0 0x7f89dc15d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48832/0 (socket says 172.21.15.92:48832) 2023-10-06T10:44:21.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.231+0000 7f89d1d9b700 1 -- 172.21.15.92:0/2197836818 learned_addr learned my addr 172.21.15.92:0/2197836818 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:21.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.231+0000 7f89da59c700 1 --2- 172.21.15.92:0/2197836818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89dc0ad210 0x7f89dc0cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:21.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.232+0000 7f89d1d9b700 1 -- 172.21.15.92:0/2197836818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89dc0ad210 msgr2=0x7f89dc0cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:21.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.232+0000 7f89d1d9b700 1 --2- 172.21.15.92:0/2197836818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89dc0ad210 0x7f89dc0cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:21.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.232+0000 7f89d1d9b700 1 -- 172.21.15.92:0/2197836818 >> v1:172.21.15.124:6789/0 conn(0x7f89dc15d940 legacy=0x7f89dc0686c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:21.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.232+0000 7f89d1d9b700 1 -- 172.21.15.92:0/2197836818 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89dc0cca90 con 0x7f89dc0ad5e0 2023-10-06T10:44:21.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.232+0000 7f89d1d9b700 1 --2- 172.21.15.92:0/2197836818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89dc0ad5e0 0x7f89dc15d400 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f89c40099b0 tx=0x7f89c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ea5edf0e4ce936 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:21.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.232+0000 7f89d9d9b700 1 -- 172.21.15.92:0/2197836818 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c400b860 con 0x7f89dc0ad5e0 2023-10-06T10:44:21.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.232+0000 7f89d9d9b700 1 -- 172.21.15.92:0/2197836818 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c400b9c0 con 0x7f89dc0ad5e0 2023-10-06T10:44:21.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.232+0000 7f89d9d9b700 1 -- 172.21.15.92:0/2197836818 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c400bd60 con 0x7f89dc0ad5e0 2023-10-06T10:44:21.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.232+0000 7f89e09a2700 1 -- 172.21.15.92:0/2197836818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89dc0ad5e0 msgr2=0x7f89dc15d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:21.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.232+0000 7f89e09a2700 1 --2- 172.21.15.92:0/2197836818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89dc0ad5e0 0x7f89dc15d400 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f89c40099b0 tx=0x7f89c4009250 comp rx=0 tx=0).stop 2023-10-06T10:44:21.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.232+0000 7f89e09a2700 1 -- 172.21.15.92:0/2197836818 shutdown_connections 2023-10-06T10:44:21.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.232+0000 7f89e09a2700 1 --2- 172.21.15.92:0/2197836818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89dc0ad210 0x7f89dc0cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:21.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.232+0000 7f89e09a2700 1 --2- 172.21.15.92:0/2197836818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89dc0ad5e0 0x7f89dc15d400 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:21.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.232+0000 7f89e09a2700 1 -- 172.21.15.92:0/2197836818 >> 172.21.15.92:0/2197836818 conn(0x7f89dc067d90 msgr2=0x7f89dc0ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:21.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.232+0000 7f89e09a2700 1 -- 172.21.15.92:0/2197836818 shutdown_connections 2023-10-06T10:44:21.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.232+0000 7f89e09a2700 1 -- 172.21.15.92:0/2197836818 wait complete. 2023-10-06T10:44:21.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.234+0000 7f89e09a2700 1 Processor -- start 2023-10-06T10:44:21.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.234+0000 7f89e09a2700 1 -- start start 2023-10-06T10:44:21.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.234+0000 7f89e09a2700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89dc0ad210 0x7f89dc0b0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:21.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.234+0000 7f89e09a2700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89dc0ad5e0 0x7f89dc16a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:21.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.234+0000 7f89e09a2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89dc15d940 0x7f89dc159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:21.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.234+0000 7f89e09a2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc16b370 con 0x7f89dc15d940 2023-10-06T10:44:21.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.234+0000 7f89e09a2700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc16adb0 con 0x7f89dc0ad5e0 2023-10-06T10:44:21.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.234+0000 7f89e09a2700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc16b090 con 0x7f89dc0ad210 2023-10-06T10:44:21.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.234+0000 7f89dad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89dc15d940 0x7f89dc159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:21.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.234+0000 7f89dad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89dc15d940 0x7f89dc159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48836/0 (socket says 172.21.15.92:48836) 2023-10-06T10:44:21.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.235+0000 7f89dad9d700 1 -- 172.21.15.92:0/898769356 learned_addr learned my addr 172.21.15.92:0/898769356 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:21.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.235+0000 7f89da59c700 1 --2- 172.21.15.92:0/898769356 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89dc0ad210 0x7f89dc0b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:21.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.235+0000 7f89d1d9b700 1 --2- 172.21.15.92:0/898769356 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89dc0ad5e0 0x7f89dc16a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:21.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.235+0000 7f89dad9d700 1 -- 172.21.15.92:0/898769356 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89dc0ad210 msgr2=0x7f89dc0b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:21.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.235+0000 7f89dad9d700 1 --2- 172.21.15.92:0/898769356 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89dc0ad210 0x7f89dc0b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:21.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.235+0000 7f89dad9d700 1 -- 172.21.15.92:0/898769356 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89dc0ad5e0 msgr2=0x7f89dc16a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:21.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.235+0000 7f89dad9d700 1 --2- 172.21.15.92:0/898769356 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89dc0ad5e0 0x7f89dc16a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:21.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.235+0000 7f89dad9d700 1 -- 172.21.15.92:0/898769356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c40094f0 con 0x7f89dc15d940 2023-10-06T10:44:21.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.235+0000 7f89dad9d700 1 --2- 172.21.15.92:0/898769356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89dc15d940 0x7f89dc159400 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f89d400c840 tx=0x7f89d4001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:21.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.235+0000 7f89d2ffd700 1 -- 172.21.15.92:0/898769356 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4012630 con 0x7f89dc15d940 2023-10-06T10:44:21.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.235+0000 7f89d2ffd700 1 -- 172.21.15.92:0/898769356 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d40127b0 con 0x7f89dc15d940 2023-10-06T10:44:21.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.235+0000 7f89e09a2700 1 -- 172.21.15.92:0/898769356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89dc16b610 con 0x7f89dc15d940 2023-10-06T10:44:21.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.235+0000 7f89d2ffd700 1 -- 172.21.15.92:0/898769356 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4012b70 con 0x7f89dc15d940 2023-10-06T10:44:21.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.235+0000 7f89e09a2700 1 -- 172.21.15.92:0/898769356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89dc0b06d0 con 0x7f89dc15d940 2023-10-06T10:44:21.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.236+0000 7f89e09a2700 1 -- 172.21.15.92:0/898769356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89dc16bb30 con 0x7f89dc15d940 2023-10-06T10:44:21.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.236+0000 7f89d2ffd700 1 -- 172.21.15.92:0/898769356 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f89d401a3a0 con 0x7f89dc15d940 2023-10-06T10:44:21.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.237+0000 7f89d2ffd700 1 --2- 172.21.15.92:0/898769356 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f89bc066f30 0x7f89bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:21.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.237+0000 7f89d2ffd700 1 -- 172.21.15.92:0/898769356 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f89d40848b0 con 0x7f89dc15d940 2023-10-06T10:44:21.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.237+0000 7f89da59c700 1 --2- 172.21.15.92:0/898769356 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f89bc066f30 0x7f89bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:21.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.237+0000 7f89da59c700 1 --2- 172.21.15.92:0/898769356 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f89bc066f30 0x7f89bc0693e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f89dc06b9c0 tx=0x7f89cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:21.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.241+0000 7f89d2ffd700 1 -- 172.21.15.92:0/898769356 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89dc16bb30 con 0x7f89dc15d940 2023-10-06T10:44:21.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.497+0000 7f89e09a2700 1 -- 172.21.15.92:0/898769356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f89dc16bb30 con 0x7f89dc15d940 2023-10-06T10:44:21.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.498+0000 7f89d2ffd700 1 -- 172.21.15.92:0/898769356 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1888 (secure 0 0 0) 0x7f89dc16bb30 con 0x7f89dc15d940 2023-10-06T10:44:21.499 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:21.499 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":382,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":698974,"bytes_used":2445807616,"bytes_avail":770648305664,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":9811,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":92,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"a","status":"up:active","gid":7686},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7689},{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":7701}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2023-10-06T10:44:18.160883+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T10:44:21.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.500+0000 7f89e09a2700 1 -- 172.21.15.92:0/898769356 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f89bc066f30 msgr2=0x7f89bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:21.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.500+0000 7f89e09a2700 1 --2- 172.21.15.92:0/898769356 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f89bc066f30 0x7f89bc0693e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f89dc06b9c0 tx=0x7f89cc008040 comp rx=0 tx=0).stop 2023-10-06T10:44:21.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.500+0000 7f89e09a2700 1 -- 172.21.15.92:0/898769356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89dc15d940 msgr2=0x7f89dc159400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:21.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.500+0000 7f89e09a2700 1 --2- 172.21.15.92:0/898769356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89dc15d940 0x7f89dc159400 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f89d400c840 tx=0x7f89d4001670 comp rx=0 tx=0).stop 2023-10-06T10:44:21.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.500+0000 7f89e09a2700 1 -- 172.21.15.92:0/898769356 shutdown_connections 2023-10-06T10:44:21.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.500+0000 7f89e09a2700 1 --2- 172.21.15.92:0/898769356 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f89bc066f30 0x7f89bc0693e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:21.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.500+0000 7f89e09a2700 1 --2- 172.21.15.92:0/898769356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89dc15d940 0x7f89dc159400 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:21.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.500+0000 7f89e09a2700 1 --2- 172.21.15.92:0/898769356 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89dc0ad210 0x7f89dc0b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:21.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.500+0000 7f89e09a2700 1 --2- 172.21.15.92:0/898769356 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89dc0ad5e0 0x7f89dc16a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:21.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.500+0000 7f89e09a2700 1 -- 172.21.15.92:0/898769356 >> 172.21.15.92:0/898769356 conn(0x7f89dc067d90 msgr2=0x7f89dc0ca140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:21.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.500+0000 7f89e09a2700 1 -- 172.21.15.92:0/898769356 shutdown_connections 2023-10-06T10:44:21.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:21.500+0000 7f89e09a2700 1 -- 172.21.15.92:0/898769356 wait complete. 2023-10-06T10:44:21.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]'} 2023-10-06T10:44:22.513 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:44:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.632+0000 7f87edfc9700 1 Processor -- start 2023-10-06T10:44:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.633+0000 7f87edfc9700 1 -- start start 2023-10-06T10:44:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.633+0000 7f87edfc9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f87e80ad5e0 0x7f87e80cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.633+0000 7f87edfc9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e80cecf0 con 0x7f87e80ad210 2023-10-06T10:44:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.633+0000 7f87edfc9700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e80cefd0 con 0x7f87e815db10 2023-10-06T10:44:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.633+0000 7f87edfc9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87e80cca90 con 0x7f87e80ad5e0 2023-10-06T10:44:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.633+0000 7f87e77fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f87e80ad210 0x7f87e815d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44654/0 (socket says 172.21.15.92:44654) 2023-10-06T10:44:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.633+0000 7f87e77fe700 1 -- 172.21.15.92:0/1015943892 learned_addr learned my addr 172.21.15.92:0/1015943892 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.634+0000 7f87e7fff700 1 --2- 172.21.15.92:0/1015943892 >> v2:172.21.15.92:3300/0 conn(0x7f87e80ad5e0 0x7f87e80cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.634+0000 7f87e6ffd700 1 -- 172.21.15.92:0/1015943892 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2546855887 0 0) 0x7f87e80cecf0 con 0x7f87e80ad210 2023-10-06T10:44:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.634+0000 7f87e6ffd700 1 -- 172.21.15.92:0/1015943892 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87d4003570 con 0x7f87e80ad210 2023-10-06T10:44:22.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.634+0000 7f87e7fff700 1 -- 172.21.15.92:0/1015943892 >> v1:172.21.15.124:6789/0 conn(0x7f87e815db10 legacy=0x7f87e80ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:22.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.634+0000 7f87e7fff700 1 -- 172.21.15.92:0/1015943892 >> v1:172.21.15.92:6789/0 conn(0x7f87e80ad210 legacy=0x7f87e815d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:22.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.634+0000 7f87e7fff700 1 -- 172.21.15.92:0/1015943892 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e80ccd70 con 0x7f87e80ad5e0 2023-10-06T10:44:22.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.634+0000 7f87e7fff700 1 --2- 172.21.15.92:0/1015943892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e80ad5e0 0x7f87e80cd290 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009e60 tx=0x7f87e0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd1738722ac387b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:22.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.634+0000 7f87e6ffd700 1 -- 172.21.15.92:0/1015943892 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e000e070 con 0x7f87e80ad5e0 2023-10-06T10:44:22.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.634+0000 7f87e6ffd700 1 -- 172.21.15.92:0/1015943892 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e000bb20 con 0x7f87e80ad5e0 2023-10-06T10:44:22.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.634+0000 7f87e6ffd700 1 -- 172.21.15.92:0/1015943892 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e0012430 con 0x7f87e80ad5e0 2023-10-06T10:44:22.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.635+0000 7f87edfc9700 1 -- 172.21.15.92:0/1015943892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e80ad5e0 msgr2=0x7f87e80cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:22.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.635+0000 7f87edfc9700 1 --2- 172.21.15.92:0/1015943892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e80ad5e0 0x7f87e80cd290 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009e60 tx=0x7f87e0009780 comp rx=0 tx=0).stop 2023-10-06T10:44:22.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.635+0000 7f87edfc9700 1 -- 172.21.15.92:0/1015943892 shutdown_connections 2023-10-06T10:44:22.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.635+0000 7f87edfc9700 1 --2- 172.21.15.92:0/1015943892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e80ad5e0 0x7f87e80cd290 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:22.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.635+0000 7f87edfc9700 1 -- 172.21.15.92:0/1015943892 >> 172.21.15.92:0/1015943892 conn(0x7f87e8067d90 msgr2=0x7f87e80ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:22.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.635+0000 7f87edfc9700 1 -- 172.21.15.92:0/1015943892 shutdown_connections 2023-10-06T10:44:22.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.635+0000 7f87edfc9700 1 -- 172.21.15.92:0/1015943892 wait complete. 2023-10-06T10:44:22.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.636+0000 7f87edfc9700 1 Processor -- start 2023-10-06T10:44:22.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.636+0000 7f87edfc9700 1 -- start start 2023-10-06T10:44:22.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87edfc9700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87e80ad210 0x7f87e80c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:22.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87edfc9700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87e80ad5e0 0x7f87e80b2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:22.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87edfc9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e815db10 0x7f87e80b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:22.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87edfc9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e80c6e30 con 0x7f87e815db10 2023-10-06T10:44:22.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87edfc9700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e80c6250 con 0x7f87e80ad5e0 2023-10-06T10:44:22.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87edfc9700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e80c5f70 con 0x7f87e80ad210 2023-10-06T10:44:22.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87e7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e815db10 0x7f87e80b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:22.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87e7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e815db10 0x7f87e80b3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48846/0 (socket says 172.21.15.92:48846) 2023-10-06T10:44:22.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87e7fff700 1 -- 172.21.15.92:0/3248404474 learned_addr learned my addr 172.21.15.92:0/3248404474 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:22.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87deffd700 1 --2- 172.21.15.92:0/3248404474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87e80ad5e0 0x7f87e80b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:22.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87e7fff700 1 -- 172.21.15.92:0/3248404474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87e80ad210 msgr2=0x7f87e80c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:22.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87e77fe700 1 --2- 172.21.15.92:0/3248404474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87e80ad210 0x7f87e80c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:22.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87e7fff700 1 --2- 172.21.15.92:0/3248404474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87e80ad210 0x7f87e80c72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:22.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87e7fff700 1 -- 172.21.15.92:0/3248404474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87e80ad5e0 msgr2=0x7f87e80b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:22.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87e7fff700 1 --2- 172.21.15.92:0/3248404474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87e80ad5e0 0x7f87e80b2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:22.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87e7fff700 1 -- 172.21.15.92:0/3248404474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e0009a20 con 0x7f87e815db10 2023-10-06T10:44:22.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87deffd700 1 --2- 172.21.15.92:0/3248404474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87e80ad5e0 0x7f87e80b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:22.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87e7fff700 1 --2- 172.21.15.92:0/3248404474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e815db10 0x7f87e80b3010 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f87e0003000 tx=0x7f87e0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:22.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87dffff700 1 -- 172.21.15.92:0/3248404474 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e0012e00 con 0x7f87e815db10 2023-10-06T10:44:22.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87dffff700 1 -- 172.21.15.92:0/3248404474 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e001e640 con 0x7f87e815db10 2023-10-06T10:44:22.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87edfc9700 1 -- 172.21.15.92:0/3248404474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e80c67e0 con 0x7f87e815db10 2023-10-06T10:44:22.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87dffff700 1 -- 172.21.15.92:0/3248404474 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e001ea00 con 0x7f87e815db10 2023-10-06T10:44:22.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.637+0000 7f87edfc9700 1 -- 172.21.15.92:0/3248404474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e80b2bd0 con 0x7f87e815db10 2023-10-06T10:44:22.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.638+0000 7f87edfc9700 1 -- 172.21.15.92:0/3248404474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e8064a90 con 0x7f87e815db10 2023-10-06T10:44:22.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.639+0000 7f87dffff700 1 -- 172.21.15.92:0/3248404474 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f87e0024070 con 0x7f87e815db10 2023-10-06T10:44:22.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.639+0000 7f87dffff700 1 --2- 172.21.15.92:0/3248404474 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f87c8066f30 0x7f87c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:22.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.639+0000 7f87dffff700 1 -- 172.21.15.92:0/3248404474 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f87e0087170 con 0x7f87e815db10 2023-10-06T10:44:22.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.639+0000 7f87e77fe700 1 --2- 172.21.15.92:0/3248404474 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f87c8066f30 0x7f87c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:22.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.639+0000 7f87e77fe700 1 --2- 172.21.15.92:0/3248404474 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f87c8066f30 0x7f87c80693e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f87d8007fd0 tx=0x7f87d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:22.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.643+0000 7f87dffff700 1 -- 172.21.15.92:0/3248404474 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87e0054140 con 0x7f87e815db10 2023-10-06T10:44:22.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.869+0000 7f87edfc9700 1 -- 172.21.15.92:0/3248404474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87e80cc760 con 0x7f87e815db10 2023-10-06T10:44:22.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.870+0000 7f87dffff700 1 -- 172.21.15.92:0/3248404474 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4642 (secure 0 0 0) 0x7f87e0030e80 con 0x7f87e815db10 2023-10-06T10:44:22.871 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:22.871 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2122441982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2122441982},{"type":"v1","addr":"172.21.15.124:6835","nonce":2122441982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:47.850832+0000","modified":"2023-10-06T10:44:20.587653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7701,"mds_1":7689,"mds_2":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6836/3605297930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3605297930},{"type":"v1","addr":"172.21.15.92:6836","nonce":3605297930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7689":{"gid":7689,"name":"c","rank":1,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.92:6837/4047304116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4047304116},{"type":"v1","addr":"172.21.15.92:6837","nonce":4047304116}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7701":{"gid":7701,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":11,"addr":"172.21.15.124:6834/4203118992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":4203118992},{"type":"v1","addr":"172.21.15.124:6834","nonce":4203118992}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T10:44:22.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.872+0000 7f87edfc9700 1 -- 172.21.15.92:0/3248404474 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f87c8066f30 msgr2=0x7f87c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:22.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.872+0000 7f87edfc9700 1 --2- 172.21.15.92:0/3248404474 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f87c8066f30 0x7f87c80693e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f87d8007fd0 tx=0x7f87d8009040 comp rx=0 tx=0).stop 2023-10-06T10:44:22.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.872+0000 7f87edfc9700 1 -- 172.21.15.92:0/3248404474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e815db10 msgr2=0x7f87e80b3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:22.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.872+0000 7f87edfc9700 1 --2- 172.21.15.92:0/3248404474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e815db10 0x7f87e80b3010 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f87e0003000 tx=0x7f87e0018000 comp rx=0 tx=0).stop 2023-10-06T10:44:22.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.872+0000 7f87edfc9700 1 -- 172.21.15.92:0/3248404474 shutdown_connections 2023-10-06T10:44:22.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.872+0000 7f87edfc9700 1 --2- 172.21.15.92:0/3248404474 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f87c8066f30 0x7f87c80693e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:22.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.872+0000 7f87edfc9700 1 --2- 172.21.15.92:0/3248404474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e815db10 0x7f87e80b3010 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:22.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.872+0000 7f87edfc9700 1 --2- 172.21.15.92:0/3248404474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87e80ad210 0x7f87e80c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:22.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.872+0000 7f87edfc9700 1 --2- 172.21.15.92:0/3248404474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87e80ad5e0 0x7f87e80b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:22.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.872+0000 7f87edfc9700 1 -- 172.21.15.92:0/3248404474 >> 172.21.15.92:0/3248404474 conn(0x7f87e8067d90 msgr2=0x7f87e80ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:22.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.872+0000 7f87edfc9700 1 -- 172.21.15.92:0/3248404474 shutdown_connections 2023-10-06T10:44:22.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.872+0000 7f87edfc9700 1 -- 172.21.15.92:0/3248404474 wait complete. 2023-10-06T10:44:22.874 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 92 2023-10-06T10:44:22.885 DEBUG:tasks.cephfs.test_multimds_misc:current active=['b', 'c', 'a'] 2023-10-06T10:44:22.885 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:44:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.997+0000 7efd3dd88700 1 Processor -- start 2023-10-06T10:44:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.997+0000 7efd3dd88700 1 -- start start 2023-10-06T10:44:23.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.998+0000 7efd3dd88700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efd380b49a0 0x7efd380aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:23.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.998+0000 7efd3dd88700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd380b0c60 con 0x7efd38159560 2023-10-06T10:44:23.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.998+0000 7efd3dd88700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd380b0f40 con 0x7efd380b0760 2023-10-06T10:44:23.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.998+0000 7efd3dd88700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd380b1220 con 0x7efd380b49a0 2023-10-06T10:44:23.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.998+0000 7efd36ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efd380b49a0 0x7efd380aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:23.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.998+0000 7efd377fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7efd38159560 0x7efd380aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44670/0 (socket says 172.21.15.92:44670) 2023-10-06T10:44:23.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.998+0000 7efd377fe700 1 -- 172.21.15.92:0/885753922 learned_addr learned my addr 172.21.15.92:0/885753922 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:23.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.999+0000 7efd367fc700 1 -- 172.21.15.92:0/885753922 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1417850579 0 0) 0x7efd380b0c60 con 0x7efd38159560 2023-10-06T10:44:23.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.999+0000 7efd367fc700 1 -- 172.21.15.92:0/885753922 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd20003570 con 0x7efd38159560 2023-10-06T10:44:23.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.999+0000 7efd36ffd700 1 -- 172.21.15.92:0/885753922 >> v1:172.21.15.124:6789/0 conn(0x7efd380b0760 legacy=0x7efd380b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:23.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.999+0000 7efd36ffd700 1 -- 172.21.15.92:0/885753922 >> v1:172.21.15.92:6789/0 conn(0x7efd38159560 legacy=0x7efd380aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:23.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.999+0000 7efd36ffd700 1 -- 172.21.15.92:0/885753922 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd380b1500 con 0x7efd380b49a0 2023-10-06T10:44:23.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.999+0000 7efd36ffd700 1 --2- 172.21.15.92:0/885753922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd380b49a0 0x7efd380aac40 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009910 tx=0x7efd2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc1848b16c681a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:23.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.999+0000 7efd367fc700 1 -- 172.21.15.92:0/885753922 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c00b640 con 0x7efd380b49a0 2023-10-06T10:44:23.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.999+0000 7efd367fc700 1 -- 172.21.15.92:0/885753922 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2c00b7a0 con 0x7efd380b49a0 2023-10-06T10:44:23.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.999+0000 7efd367fc700 1 -- 172.21.15.92:0/885753922 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c00bb60 con 0x7efd380b49a0 2023-10-06T10:44:23.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.999+0000 7efd3dd88700 1 -- 172.21.15.92:0/885753922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd380b49a0 msgr2=0x7efd380aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:23.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.999+0000 7efd3dd88700 1 --2- 172.21.15.92:0/885753922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd380b49a0 0x7efd380aac40 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009910 tx=0x7efd2c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:23.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.999+0000 7efd3dd88700 1 -- 172.21.15.92:0/885753922 shutdown_connections 2023-10-06T10:44:23.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.999+0000 7efd3dd88700 1 --2- 172.21.15.92:0/885753922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd380b49a0 0x7efd380aac40 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:23.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.999+0000 7efd3dd88700 1 -- 172.21.15.92:0/885753922 >> 172.21.15.92:0/885753922 conn(0x7efd380997f0 msgr2=0x7efd3806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:23.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.999+0000 7efd3dd88700 1 -- 172.21.15.92:0/885753922 shutdown_connections 2023-10-06T10:44:23.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:22.999+0000 7efd3dd88700 1 -- 172.21.15.92:0/885753922 wait complete. 2023-10-06T10:44:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.001+0000 7efd3dd88700 1 Processor -- start 2023-10-06T10:44:23.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.001+0000 7efd3dd88700 1 -- start start 2023-10-06T10:44:23.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.001+0000 7efd3dd88700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd380b0760 0x7efd38156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:23.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.001+0000 7efd3dd88700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd380b49a0 0x7efd3815b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:23.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.001+0000 7efd3dd88700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd38159560 0x7efd3809aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:23.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.001+0000 7efd3dd88700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3809c930 con 0x7efd38159560 2023-10-06T10:44:23.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.001+0000 7efd3dd88700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3816cd80 con 0x7efd380b0760 2023-10-06T10:44:23.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.001+0000 7efd3dd88700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3816cfd0 con 0x7efd380b49a0 2023-10-06T10:44:23.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.002+0000 7efd37fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd38159560 0x7efd3809aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:23.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.002+0000 7efd37fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd38159560 0x7efd3809aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48858/0 (socket says 172.21.15.92:48858) 2023-10-06T10:44:23.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.002+0000 7efd37fff700 1 -- 172.21.15.92:0/3700676203 learned_addr learned my addr 172.21.15.92:0/3700676203 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:23.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.002+0000 7efd377fe700 1 --2- 172.21.15.92:0/3700676203 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd380b0760 0x7efd38156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:23.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.002+0000 7efd36ffd700 1 --2- 172.21.15.92:0/3700676203 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd380b49a0 0x7efd3815b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:23.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.002+0000 7efd37fff700 1 -- 172.21.15.92:0/3700676203 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd380b49a0 msgr2=0x7efd3815b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:23.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.002+0000 7efd37fff700 1 --2- 172.21.15.92:0/3700676203 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd380b49a0 0x7efd3815b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:23.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.002+0000 7efd37fff700 1 -- 172.21.15.92:0/3700676203 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd380b0760 msgr2=0x7efd38156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:23.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.002+0000 7efd37fff700 1 --2- 172.21.15.92:0/3700676203 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd380b0760 0x7efd38156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:23.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.002+0000 7efd37fff700 1 -- 172.21.15.92:0/3700676203 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2c009420 con 0x7efd38159560 2023-10-06T10:44:23.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.002+0000 7efd37fff700 1 --2- 172.21.15.92:0/3700676203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd38159560 0x7efd3809aa60 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7efd3000c8e0 tx=0x7efd30001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:23.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.002+0000 7efd1f7fe700 1 -- 172.21.15.92:0/3700676203 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd30012630 con 0x7efd38159560 2023-10-06T10:44:23.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.002+0000 7efd1f7fe700 1 -- 172.21.15.92:0/3700676203 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd30012790 con 0x7efd38159560 2023-10-06T10:44:23.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.002+0000 7efd3dd88700 1 -- 172.21.15.92:0/3700676203 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3816d2b0 con 0x7efd38159560 2023-10-06T10:44:23.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.002+0000 7efd1f7fe700 1 -- 172.21.15.92:0/3700676203 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd30012b50 con 0x7efd38159560 2023-10-06T10:44:23.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.002+0000 7efd3dd88700 1 -- 172.21.15.92:0/3700676203 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3816d870 con 0x7efd38159560 2023-10-06T10:44:23.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.003+0000 7efd3dd88700 1 -- 172.21.15.92:0/3700676203 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd38064a90 con 0x7efd38159560 2023-10-06T10:44:23.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.003+0000 7efd1f7fe700 1 -- 172.21.15.92:0/3700676203 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efd3001a3a0 con 0x7efd38159560 2023-10-06T10:44:23.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.004+0000 7efd1f7fe700 1 --2- 172.21.15.92:0/3700676203 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efd24066f30 0x7efd240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:23.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.004+0000 7efd1f7fe700 1 -- 172.21.15.92:0/3700676203 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7efd30084940 con 0x7efd38159560 2023-10-06T10:44:23.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.004+0000 7efd377fe700 1 --2- 172.21.15.92:0/3700676203 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efd24066f30 0x7efd240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:23.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.004+0000 7efd377fe700 1 --2- 172.21.15.92:0/3700676203 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efd24066f30 0x7efd240693e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7efd28001360 tx=0x7efd28009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:23.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.008+0000 7efd1f7fe700 1 -- 172.21.15.92:0/3700676203 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd300553a0 con 0x7efd38159560 2023-10-06T10:44:23.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.267+0000 7efd3dd88700 1 -- 172.21.15.92:0/3700676203 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7efd3815d070 con 0x7efd38159560 2023-10-06T10:44:23.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.267+0000 7efd1f7fe700 1 -- 172.21.15.92:0/3700676203 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1888 (secure 0 0 0) 0x7efd30054f30 con 0x7efd38159560 2023-10-06T10:44:23.269 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:23.269 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":384,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":698974,"bytes_used":2445807616,"bytes_avail":770648305664,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6654,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":92,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"a","status":"up:active","gid":7686},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7689},{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":7701}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2023-10-06T10:44:18.160883+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T10:44:23.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.269+0000 7efd3dd88700 1 -- 172.21.15.92:0/3700676203 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efd24066f30 msgr2=0x7efd240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:23.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.269+0000 7efd3dd88700 1 --2- 172.21.15.92:0/3700676203 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efd24066f30 0x7efd240693e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7efd28001360 tx=0x7efd28009040 comp rx=0 tx=0).stop 2023-10-06T10:44:23.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.269+0000 7efd3dd88700 1 -- 172.21.15.92:0/3700676203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd38159560 msgr2=0x7efd3809aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:23.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.269+0000 7efd3dd88700 1 --2- 172.21.15.92:0/3700676203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd38159560 0x7efd3809aa60 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7efd3000c8e0 tx=0x7efd30001710 comp rx=0 tx=0).stop 2023-10-06T10:44:23.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.269+0000 7efd3dd88700 1 -- 172.21.15.92:0/3700676203 shutdown_connections 2023-10-06T10:44:23.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.269+0000 7efd3dd88700 1 --2- 172.21.15.92:0/3700676203 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efd24066f30 0x7efd240693e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:23.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.269+0000 7efd3dd88700 1 --2- 172.21.15.92:0/3700676203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd38159560 0x7efd3809aa60 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:23.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.269+0000 7efd3dd88700 1 --2- 172.21.15.92:0/3700676203 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd380b0760 0x7efd38156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:23.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.269+0000 7efd3dd88700 1 --2- 172.21.15.92:0/3700676203 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd380b49a0 0x7efd3815b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:23.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.269+0000 7efd3dd88700 1 -- 172.21.15.92:0/3700676203 >> 172.21.15.92:0/3700676203 conn(0x7efd380997f0 msgr2=0x7efd38158a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:23.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.269+0000 7efd3dd88700 1 -- 172.21.15.92:0/3700676203 shutdown_connections 2023-10-06T10:44:23.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:23.269+0000 7efd3dd88700 1 -- 172.21.15.92:0/3700676203 wait complete. 2023-10-06T10:44:23.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]'} 2023-10-06T10:44:23.339 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T10:44:23.339+0000 7f9e81bcf700 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T10:44:23.454 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:44:23.457 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:44:24.282 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:44:24.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.399+0000 7f85b11e5700 1 Processor -- start 2023-10-06T10:44:24.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.399+0000 7f85b11e5700 1 -- start start 2023-10-06T10:44:24.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.400+0000 7f85b11e5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f85ac0b49a0 0x7f85ac0aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:24.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.400+0000 7f85b11e5700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f85ac0b0760 0x7f85ac0b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:24.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.400+0000 7f85b11e5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85ac0b0c60 con 0x7f85ac159560 2023-10-06T10:44:24.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.400+0000 7f85b11e5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85ac0b0f40 con 0x7f85ac0b49a0 2023-10-06T10:44:24.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.400+0000 7f85b11e5700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85ac0b1220 con 0x7f85ac0b0760 2023-10-06T10:44:24.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.400+0000 7f85aa59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f85ac0b49a0 0x7f85ac0aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:24.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.400+0000 7f85aad9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f85ac159560 0x7f85ac0aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44136/0 (socket says 172.21.15.92:44136) 2023-10-06T10:44:24.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.400+0000 7f85aad9d700 1 -- 172.21.15.92:0/3162734560 learned_addr learned my addr 172.21.15.92:0/3162734560 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:24.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.400+0000 7f85ab59e700 1 --2- 172.21.15.92:0/3162734560 >> v2:172.21.15.124:3300/0 conn(0x7f85ac0b0760 0x7f85ac0b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:24.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.400+0000 7f85aa59c700 1 -- 172.21.15.92:0/3162734560 >> v2:172.21.15.124:3300/0 conn(0x7f85ac0b0760 msgr2=0x7f85ac0b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:24.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.400+0000 7f85aa59c700 1 --2- 172.21.15.92:0/3162734560 >> v2:172.21.15.124:3300/0 conn(0x7f85ac0b0760 0x7f85ac0b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:24.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.400+0000 7f85aa59c700 1 -- 172.21.15.92:0/3162734560 >> v1:172.21.15.92:6789/0 conn(0x7f85ac159560 legacy=0x7f85ac0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:24.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.400+0000 7f85aa59c700 1 -- 172.21.15.92:0/3162734560 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ac0b1500 con 0x7f85ac0b49a0 2023-10-06T10:44:24.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.400+0000 7f85a9d9b700 1 -- 172.21.15.92:0/3162734560 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3144003576 0 0) 0x7f85ac0b0c60 con 0x7f85ac159560 2023-10-06T10:44:24.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.400+0000 7f85aa59c700 1 --2- 172.21.15.92:0/3162734560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85ac0b49a0 0x7f85ac0aac40 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f85a00098e0 tx=0x7f85a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe5917c71d19584 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:24.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.400+0000 7f85a9d9b700 1 -- 172.21.15.92:0/3162734560 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0011630 con 0x7f85ac0b49a0 2023-10-06T10:44:24.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.400+0000 7f85a9d9b700 1 -- 172.21.15.92:0/3162734560 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a0011790 con 0x7f85ac0b49a0 2023-10-06T10:44:24.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.400+0000 7f85a9d9b700 1 -- 172.21.15.92:0/3162734560 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0011b30 con 0x7f85ac0b49a0 2023-10-06T10:44:24.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.401+0000 7f85b11e5700 1 -- 172.21.15.92:0/3162734560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85ac0b49a0 msgr2=0x7f85ac0aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:24.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.401+0000 7f85b11e5700 1 --2- 172.21.15.92:0/3162734560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85ac0b49a0 0x7f85ac0aac40 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f85a00098e0 tx=0x7f85a00091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:24.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.401+0000 7f85b11e5700 1 -- 172.21.15.92:0/3162734560 shutdown_connections 2023-10-06T10:44:24.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.401+0000 7f85b11e5700 1 --2- 172.21.15.92:0/3162734560 >> v2:172.21.15.124:3300/0 conn(0x7f85ac0b0760 0x7f85ac0b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:24.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.401+0000 7f85b11e5700 1 --2- 172.21.15.92:0/3162734560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85ac0b49a0 0x7f85ac0aac40 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:24.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.401+0000 7f85b11e5700 1 -- 172.21.15.92:0/3162734560 >> 172.21.15.92:0/3162734560 conn(0x7f85ac0997f0 msgr2=0x7f85ac06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:24.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.401+0000 7f85b11e5700 1 -- 172.21.15.92:0/3162734560 shutdown_connections 2023-10-06T10:44:24.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.401+0000 7f85b11e5700 1 -- 172.21.15.92:0/3162734560 wait complete. 2023-10-06T10:44:24.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.402+0000 7f85b11e5700 1 Processor -- start 2023-10-06T10:44:24.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.402+0000 7f85b11e5700 1 -- start start 2023-10-06T10:44:24.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.403+0000 7f85b11e5700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85ac0b0760 0x7f85ac0b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:24.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.403+0000 7f85b11e5700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85ac0b49a0 0x7f85ac09be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:24.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.403+0000 7f85b11e5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85ac159560 0x7f85ac09c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:24.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.403+0000 7f85b11e5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac157f50 con 0x7f85ac159560 2023-10-06T10:44:24.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.403+0000 7f85b11e5700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac09ca10 con 0x7f85ac0b49a0 2023-10-06T10:44:24.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.403+0000 7f85b11e5700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac16d940 con 0x7f85ac0b0760 2023-10-06T10:44:24.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.403+0000 7f85ab59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85ac159560 0x7f85ac09c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:24.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.403+0000 7f85ab59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85ac159560 0x7f85ac09c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37444/0 (socket says 172.21.15.92:37444) 2023-10-06T10:44:24.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.403+0000 7f85ab59e700 1 -- 172.21.15.92:0/445997994 learned_addr learned my addr 172.21.15.92:0/445997994 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:24.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.403+0000 7f85aa59c700 1 --2- 172.21.15.92:0/445997994 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85ac0b49a0 0x7f85ac09be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:24.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.403+0000 7f85aad9d700 1 --2- 172.21.15.92:0/445997994 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85ac0b0760 0x7f85ac0b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:24.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.403+0000 7f85ab59e700 1 -- 172.21.15.92:0/445997994 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85ac0b0760 msgr2=0x7f85ac0b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:24.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.403+0000 7f85ab59e700 1 --2- 172.21.15.92:0/445997994 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85ac0b0760 0x7f85ac0b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:24.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.403+0000 7f85ab59e700 1 -- 172.21.15.92:0/445997994 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85ac0b49a0 msgr2=0x7f85ac09be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:24.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.403+0000 7f85ab59e700 1 --2- 172.21.15.92:0/445997994 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85ac0b49a0 0x7f85ac09be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:24.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.403+0000 7f85ab59e700 1 -- 172.21.15.92:0/445997994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a0009420 con 0x7f85ac159560 2023-10-06T10:44:24.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.404+0000 7f85ab59e700 1 --2- 172.21.15.92:0/445997994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85ac159560 0x7f85ac09c340 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f859c009e90 tx=0x7f859c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:24.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.404+0000 7f8592ffd700 1 -- 172.21.15.92:0/445997994 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c00cb70 con 0x7f85ac159560 2023-10-06T10:44:24.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.404+0000 7f8592ffd700 1 -- 172.21.15.92:0/445997994 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859c00f040 con 0x7f85ac159560 2023-10-06T10:44:24.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.404+0000 7f85b11e5700 1 -- 172.21.15.92:0/445997994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ac16ddd0 con 0x7f85ac159560 2023-10-06T10:44:24.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.404+0000 7f8592ffd700 1 -- 172.21.15.92:0/445997994 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c014480 con 0x7f85ac159560 2023-10-06T10:44:24.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.404+0000 7f85b11e5700 1 -- 172.21.15.92:0/445997994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ac16d2f0 con 0x7f85ac159560 2023-10-06T10:44:24.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.405+0000 7f85b11e5700 1 -- 172.21.15.92:0/445997994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85ac064a90 con 0x7f85ac159560 2023-10-06T10:44:24.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.405+0000 7f8592ffd700 1 -- 172.21.15.92:0/445997994 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f859c014830 con 0x7f85ac159560 2023-10-06T10:44:24.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.405+0000 7f8592ffd700 1 --2- 172.21.15.92:0/445997994 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8598066f30 0x7f85980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:24.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.405+0000 7f8592ffd700 1 -- 172.21.15.92:0/445997994 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f859c0851a0 con 0x7f85ac159560 2023-10-06T10:44:24.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.405+0000 7f85aad9d700 1 --2- 172.21.15.92:0/445997994 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8598066f30 0x7f85980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:24.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.406+0000 7f85aad9d700 1 --2- 172.21.15.92:0/445997994 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8598066f30 0x7f85980693e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f85ac09e200 tx=0x7f85a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:24.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.410+0000 7f8592ffd700 1 -- 172.21.15.92:0/445997994 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f859c052170 con 0x7f85ac159560 2023-10-06T10:44:24.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.637+0000 7f85b11e5700 1 -- 172.21.15.92:0/445997994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85ac0895a0 con 0x7f85ac159560 2023-10-06T10:44:24.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.637+0000 7f8592ffd700 1 -- 172.21.15.92:0/445997994 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4642 (secure 0 0 0) 0x7f859c055790 con 0x7f85ac159560 2023-10-06T10:44:24.638 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:24.638 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2122441982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2122441982},{"type":"v1","addr":"172.21.15.124:6835","nonce":2122441982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:47.850832+0000","modified":"2023-10-06T10:44:20.587653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7701,"mds_1":7689,"mds_2":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6836/3605297930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3605297930},{"type":"v1","addr":"172.21.15.92:6836","nonce":3605297930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7689":{"gid":7689,"name":"c","rank":1,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.92:6837/4047304116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4047304116},{"type":"v1","addr":"172.21.15.92:6837","nonce":4047304116}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7701":{"gid":7701,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":11,"addr":"172.21.15.124:6834/4203118992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":4203118992},{"type":"v1","addr":"172.21.15.124:6834","nonce":4203118992}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T10:44:24.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.639+0000 7f85b11e5700 1 -- 172.21.15.92:0/445997994 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8598066f30 msgr2=0x7f85980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:24.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.639+0000 7f85b11e5700 1 --2- 172.21.15.92:0/445997994 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8598066f30 0x7f85980693e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f85ac09e200 tx=0x7f85a4009040 comp rx=0 tx=0).stop 2023-10-06T10:44:24.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.639+0000 7f85b11e5700 1 -- 172.21.15.92:0/445997994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85ac159560 msgr2=0x7f85ac09c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:24.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.639+0000 7f85b11e5700 1 --2- 172.21.15.92:0/445997994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85ac159560 0x7f85ac09c340 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f859c009e90 tx=0x7f859c00b040 comp rx=0 tx=0).stop 2023-10-06T10:44:24.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.639+0000 7f85b11e5700 1 -- 172.21.15.92:0/445997994 shutdown_connections 2023-10-06T10:44:24.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.639+0000 7f85b11e5700 1 --2- 172.21.15.92:0/445997994 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8598066f30 0x7f85980693e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:24.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.639+0000 7f85b11e5700 1 --2- 172.21.15.92:0/445997994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85ac159560 0x7f85ac09c340 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:24.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.639+0000 7f85b11e5700 1 --2- 172.21.15.92:0/445997994 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85ac0b0760 0x7f85ac0b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:24.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.639+0000 7f85b11e5700 1 --2- 172.21.15.92:0/445997994 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85ac0b49a0 0x7f85ac09be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:24.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.639+0000 7f85b11e5700 1 -- 172.21.15.92:0/445997994 >> 172.21.15.92:0/445997994 conn(0x7f85ac0997f0 msgr2=0x7f85ac069980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:24.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.639+0000 7f85b11e5700 1 -- 172.21.15.92:0/445997994 shutdown_connections 2023-10-06T10:44:24.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.639+0000 7f85b11e5700 1 -- 172.21.15.92:0/445997994 wait complete. 2023-10-06T10:44:24.642 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 92 2023-10-06T10:44:24.652 DEBUG:tasks.cephfs.test_multimds_misc:current active=['b', 'c', 'a'] 2023-10-06T10:44:24.652 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:44:24.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.764+0000 7f3cb7024700 1 Processor -- start 2023-10-06T10:44:24.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.764+0000 7f3cb7024700 1 -- start start 2023-10-06T10:44:24.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.765+0000 7f3cb7024700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3cb0159950 0x7f3cb00675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:24.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.765+0000 7f3cb7024700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3cb0067af0 0x7f3cb00b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:24.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.765+0000 7f3cb7024700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb0067f00 con 0x7f3cb0159580 2023-10-06T10:44:24.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.765+0000 7f3cb7024700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb00cde30 con 0x7f3cb0159950 2023-10-06T10:44:24.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.765+0000 7f3cb7024700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb006ef20 con 0x7f3cb0067af0 2023-10-06T10:44:24.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.765+0000 7f3caffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3cb0159950 0x7f3cb00675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:24.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.765+0000 7f3caffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3cb0159950 0x7f3cb00675b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37460/0 (socket says 172.21.15.92:37460) 2023-10-06T10:44:24.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.765+0000 7f3caffff700 1 -- 172.21.15.92:0/1866182988 learned_addr learned my addr 172.21.15.92:0/1866182988 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:24.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.765+0000 7f3cb55c1700 1 --2- 172.21.15.92:0/1866182988 >> v2:172.21.15.124:3300/0 conn(0x7f3cb0067af0 0x7f3cb00b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:24.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.765+0000 7f3caffff700 1 -- 172.21.15.92:0/1866182988 >> v2:172.21.15.124:3300/0 conn(0x7f3cb0067af0 msgr2=0x7f3cb00b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:24.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.765+0000 7f3caffff700 1 --2- 172.21.15.92:0/1866182988 >> v2:172.21.15.124:3300/0 conn(0x7f3cb0067af0 0x7f3cb00b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:24.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.765+0000 7f3caffff700 1 -- 172.21.15.92:0/1866182988 >> v1:172.21.15.124:6789/0 conn(0x7f3cb0159580 legacy=0x7f3cb006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:24.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.765+0000 7f3caffff700 1 -- 172.21.15.92:0/1866182988 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb006f200 con 0x7f3cb0159950 2023-10-06T10:44:24.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.765+0000 7f3caffff700 1 --2- 172.21.15.92:0/1866182988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cb0159950 0x7f3cb00675b0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00b3290 tx=0x7f3ca0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c755dc848ff14f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:24.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.766+0000 7f3caf7fe700 1 -- 172.21.15.92:0/1866182988 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca000e070 con 0x7f3cb0159950 2023-10-06T10:44:24.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.766+0000 7f3caf7fe700 1 -- 172.21.15.92:0/1866182988 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca000ba20 con 0x7f3cb0159950 2023-10-06T10:44:24.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.766+0000 7f3caf7fe700 1 -- 172.21.15.92:0/1866182988 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca00092e0 con 0x7f3cb0159950 2023-10-06T10:44:24.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.766+0000 7f3cb7024700 1 -- 172.21.15.92:0/1866182988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cb0159950 msgr2=0x7f3cb00675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:24.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.766+0000 7f3cb7024700 1 --2- 172.21.15.92:0/1866182988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cb0159950 0x7f3cb00675b0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00b3290 tx=0x7f3ca0009840 comp rx=0 tx=0).stop 2023-10-06T10:44:24.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.766+0000 7f3cb7024700 1 -- 172.21.15.92:0/1866182988 shutdown_connections 2023-10-06T10:44:24.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.766+0000 7f3cb7024700 1 --2- 172.21.15.92:0/1866182988 >> v2:172.21.15.124:3300/0 conn(0x7f3cb0067af0 0x7f3cb00b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:24.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.766+0000 7f3cb7024700 1 --2- 172.21.15.92:0/1866182988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cb0159950 0x7f3cb00675b0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:24.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.766+0000 7f3cb7024700 1 -- 172.21.15.92:0/1866182988 >> 172.21.15.92:0/1866182988 conn(0x7f3cb0069140 msgr2=0x7f3cb00cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:24.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.766+0000 7f3cb7024700 1 -- 172.21.15.92:0/1866182988 shutdown_connections 2023-10-06T10:44:24.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.766+0000 7f3cb7024700 1 -- 172.21.15.92:0/1866182988 wait complete. 2023-10-06T10:44:24.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.767+0000 7f3cb7024700 1 Processor -- start 2023-10-06T10:44:24.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.767+0000 7f3cb7024700 1 -- start start 2023-10-06T10:44:24.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.768+0000 7f3cb7024700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3cb0067af0 0x7f3cb015cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:24.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.768+0000 7f3cb7024700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3cb0159580 0x7f3cb0156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:24.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.768+0000 7f3cb7024700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cb0159950 0x7f3cb015b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:24.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.768+0000 7f3cb7024700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb0156760 con 0x7f3cb0159950 2023-10-06T10:44:24.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.768+0000 7f3cb7024700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb01569e0 con 0x7f3cb0159580 2023-10-06T10:44:24.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.768+0000 7f3cb7024700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb0156cc0 con 0x7f3cb0067af0 2023-10-06T10:44:24.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.768+0000 7f3cb55c1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cb0159950 0x7f3cb015b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:24.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.768+0000 7f3cb55c1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cb0159950 0x7f3cb015b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37472/0 (socket says 172.21.15.92:37472) 2023-10-06T10:44:24.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.768+0000 7f3cb55c1700 1 -- 172.21.15.92:0/3030999624 learned_addr learned my addr 172.21.15.92:0/3030999624 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:24.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.768+0000 7f3caffff700 1 --2- 172.21.15.92:0/3030999624 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3cb0159580 0x7f3cb0156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:24.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.768+0000 7f3cb4dc0700 1 --2- 172.21.15.92:0/3030999624 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3cb0067af0 0x7f3cb015cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:24.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.768+0000 7f3cb55c1700 1 -- 172.21.15.92:0/3030999624 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3cb0067af0 msgr2=0x7f3cb015cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:24.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.768+0000 7f3cb55c1700 1 --2- 172.21.15.92:0/3030999624 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3cb0067af0 0x7f3cb015cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:24.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.768+0000 7f3cb55c1700 1 -- 172.21.15.92:0/3030999624 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3cb0159580 msgr2=0x7f3cb0156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:24.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.768+0000 7f3cb55c1700 1 --2- 172.21.15.92:0/3030999624 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3cb0159580 0x7f3cb0156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:24.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.768+0000 7f3cb55c1700 1 -- 172.21.15.92:0/3030999624 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca0009ae0 con 0x7f3cb0159950 2023-10-06T10:44:24.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.768+0000 7f3cb4dc0700 1 --2- 172.21.15.92:0/3030999624 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3cb0067af0 0x7f3cb015cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:24.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.769+0000 7f3cb55c1700 1 --2- 172.21.15.92:0/3030999624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cb0159950 0x7f3cb015b720 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800e930 tx=0x7f3c980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:24.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.769+0000 7f3cacff9700 1 -- 172.21.15.92:0/3030999624 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c98013680 con 0x7f3cb0159950 2023-10-06T10:44:24.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.769+0000 7f3cacff9700 1 -- 172.21.15.92:0/3030999624 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c98013800 con 0x7f3cb0159950 2023-10-06T10:44:24.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.769+0000 7f3cb7024700 1 -- 172.21.15.92:0/3030999624 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb0156fa0 con 0x7f3cb0159950 2023-10-06T10:44:24.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.769+0000 7f3cacff9700 1 -- 172.21.15.92:0/3030999624 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c98013bc0 con 0x7f3cb0159950 2023-10-06T10:44:24.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.769+0000 7f3cb7024700 1 -- 172.21.15.92:0/3030999624 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb0151af0 con 0x7f3cb0159950 2023-10-06T10:44:24.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.770+0000 7f3cb7024700 1 -- 172.21.15.92:0/3030999624 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb00c0c40 con 0x7f3cb0159950 2023-10-06T10:44:24.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.770+0000 7f3cacff9700 1 -- 172.21.15.92:0/3030999624 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c9800c460 con 0x7f3cb0159950 2023-10-06T10:44:24.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.770+0000 7f3cacff9700 1 --2- 172.21.15.92:0/3030999624 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c90066f30 0x7f3c900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:24.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.771+0000 7f3cacff9700 1 -- 172.21.15.92:0/3030999624 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3c98084d60 con 0x7f3cb0159950 2023-10-06T10:44:24.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.771+0000 7f3cb4dc0700 1 --2- 172.21.15.92:0/3030999624 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c90066f30 0x7f3c900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:24.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.771+0000 7f3cb4dc0700 1 --2- 172.21.15.92:0/3030999624 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c90066f30 0x7f3c900693e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8007fd0 tx=0x7f3ca8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:24.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:24.775+0000 7f3cacff9700 1 -- 172.21.15.92:0/3030999624 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cb00c0c40 con 0x7f3cb0159950 2023-10-06T10:44:25.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.028+0000 7f3cb7024700 1 -- 172.21.15.92:0/3030999624 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3cb00c0c40 con 0x7f3cb0159950 2023-10-06T10:44:25.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.029+0000 7f3cacff9700 1 -- 172.21.15.92:0/3030999624 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1970 (secure 0 0 0) 0x7f3c980217d0 con 0x7f3cb0159950 2023-10-06T10:44:25.030 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:25.030 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":386,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":698974,"bytes_used":2445807616,"bytes_avail":770648305664,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6654,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":92,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"a","status":"up:active","gid":7686},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7689},{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":7701}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2023-10-06T10:44:24.162386+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T10:44:25.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.031+0000 7f3cb7024700 1 -- 172.21.15.92:0/3030999624 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c90066f30 msgr2=0x7f3c900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.031+0000 7f3cb7024700 1 --2- 172.21.15.92:0/3030999624 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c90066f30 0x7f3c900693e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8007fd0 tx=0x7f3ca8009040 comp rx=0 tx=0).stop 2023-10-06T10:44:25.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.031+0000 7f3cb7024700 1 -- 172.21.15.92:0/3030999624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cb0159950 msgr2=0x7f3cb015b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.031+0000 7f3cb7024700 1 --2- 172.21.15.92:0/3030999624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cb0159950 0x7f3cb015b720 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800e930 tx=0x7f3c980091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.031+0000 7f3cb7024700 1 -- 172.21.15.92:0/3030999624 shutdown_connections 2023-10-06T10:44:25.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.031+0000 7f3cb7024700 1 --2- 172.21.15.92:0/3030999624 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c90066f30 0x7f3c900693e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.031+0000 7f3cb7024700 1 --2- 172.21.15.92:0/3030999624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cb0159950 0x7f3cb015b720 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.031+0000 7f3cb7024700 1 --2- 172.21.15.92:0/3030999624 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3cb0067af0 0x7f3cb015cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.031+0000 7f3cb7024700 1 --2- 172.21.15.92:0/3030999624 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3cb0159580 0x7f3cb0156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.031+0000 7f3cb7024700 1 -- 172.21.15.92:0/3030999624 >> 172.21.15.92:0/3030999624 conn(0x7f3cb0069140 msgr2=0x7f3cb015f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.031+0000 7f3cb7024700 1 -- 172.21.15.92:0/3030999624 shutdown_connections 2023-10-06T10:44:25.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.031+0000 7f3cb7024700 1 -- 172.21.15.92:0/3030999624 wait complete. 2023-10-06T10:44:25.043 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2023-10-06T10:44:25.044 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub resume 2023-10-06T10:44:25.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.159+0000 7f3c2500c700 1 Processor -- start 2023-10-06T10:44:25.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.159+0000 7f3c2500c700 1 -- start start 2023-10-06T10:44:25.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.160+0000 7f3c2500c700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3c200c5250 0x7f3c200ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.160+0000 7f3c2500c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c200cd300 0x7f3c200b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.160+0000 7f3c2500c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c2015ac30 con 0x7f3c20150940 2023-10-06T10:44:25.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.160+0000 7f3c2500c700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c200cb6a0 con 0x7f3c200c5250 2023-10-06T10:44:25.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.160+0000 7f3c2500c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c200cd720 con 0x7f3c200cd300 2023-10-06T10:44:25.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.160+0000 7f3c1ed9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3c20150940 0x7f3c20159c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44152/0 (socket says 172.21.15.92:44152) 2023-10-06T10:44:25.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.160+0000 7f3c1ed9d700 1 -- 172.21.15.92:0/602873644 learned_addr learned my addr 172.21.15.92:0/602873644 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:25.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.160+0000 7f3c1f59e700 1 --2- 172.21.15.92:0/602873644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c200cd300 0x7f3c200b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.160+0000 7f3c1659c700 1 --2- 172.21.15.92:0/602873644 >> v2:172.21.15.124:3300/0 conn(0x7f3c200c5250 0x7f3c200ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.160+0000 7f3c1e59c700 1 -- 172.21.15.92:0/602873644 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874243577 0 0) 0x7f3c10001290 con 0x7f3c20150940 2023-10-06T10:44:25.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.160+0000 7f3c1e59c700 1 -- 172.21.15.92:0/602873644 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c0c003570 con 0x7f3c20150940 2023-10-06T10:44:25.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1e59c700 1 -- 172.21.15.92:0/602873644 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4288254218 0 0) 0x7f3c0c003570 con 0x7f3c20150940 2023-10-06T10:44:25.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1e59c700 1 -- 172.21.15.92:0/602873644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c200cd300 msgr2=0x7f3c200b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1e59c700 1 --2- 172.21.15.92:0/602873644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c200cd300 0x7f3c200b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1e59c700 1 -- 172.21.15.92:0/602873644 >> v2:172.21.15.124:3300/0 conn(0x7f3c200c5250 msgr2=0x7f3c200ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1e59c700 1 --2- 172.21.15.92:0/602873644 >> v2:172.21.15.124:3300/0 conn(0x7f3c200c5250 0x7f3c200ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1e59c700 1 -- 172.21.15.92:0/602873644 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c200cb090 con 0x7f3c20150940 2023-10-06T10:44:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1e59c700 1 -- 172.21.15.92:0/602873644 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f3c10001780 con 0x7f3c20150940 2023-10-06T10:44:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1e59c700 1 -- 172.21.15.92:0/602873644 >> v1:172.21.15.92:6789/0 conn(0x7f3c20150940 legacy=0x7f3c20159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1e59c700 1 --2- 172.21.15.92:0/602873644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c200cd300 0x7f3c0c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1e59c700 1 --2- 172.21.15.92:0/602873644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c0c003fe0 0x7f3c0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1e59c700 1 --2- 172.21.15.92:0/602873644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c0c004d50 0x7f3c0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1e59c700 1 -- 172.21.15.92:0/602873644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c200cb6a0 con 0x7f3c0c003fe0 2023-10-06T10:44:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1e59c700 1 -- 172.21.15.92:0/602873644 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c200cd720 con 0x7f3c0c004d50 2023-10-06T10:44:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1e59c700 1 -- 172.21.15.92:0/602873644 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c00ba90 con 0x7f3c200cd300 2023-10-06T10:44:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1659c700 1 --2- 172.21.15.92:0/602873644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c0c003fe0 0x7f3c0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1f59e700 1 --2- 172.21.15.92:0/602873644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c0c004d50 0x7f3c0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1ed9d700 1 --2- 172.21.15.92:0/602873644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c200cd300 0x7f3c0c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1659c700 1 -- 172.21.15.92:0/602873644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c200cd300 msgr2=0x7f3c0c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1659c700 1 --2- 172.21.15.92:0/602873644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c200cd300 0x7f3c0c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1659c700 1 -- 172.21.15.92:0/602873644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c0c004d50 msgr2=0x7f3c0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1659c700 1 --2- 172.21.15.92:0/602873644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c0c004d50 0x7f3c0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1659c700 1 -- 172.21.15.92:0/602873644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0c003570 con 0x7f3c0c003fe0 2023-10-06T10:44:25.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1ed9d700 1 --2- 172.21.15.92:0/602873644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c200cd300 0x7f3c0c003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:25.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1659c700 1 --2- 172.21.15.92:0/602873644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c0c003fe0 0x7f3c0c004810 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009c00 tx=0x7f3c080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef04803264eaad68 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1e59c700 1 -- 172.21.15.92:0/602873644 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0800e6b0 con 0x7f3c0c003fe0 2023-10-06T10:44:25.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1e59c700 1 -- 172.21.15.92:0/602873644 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0800e810 con 0x7f3c0c003fe0 2023-10-06T10:44:25.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.161+0000 7f3c1e59c700 1 -- 172.21.15.92:0/602873644 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0800ebd0 con 0x7f3c0c003fe0 2023-10-06T10:44:25.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.162+0000 7f3c2500c700 1 -- 172.21.15.92:0/602873644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c0c003fe0 msgr2=0x7f3c0c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.162+0000 7f3c2500c700 1 --2- 172.21.15.92:0/602873644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c0c003fe0 0x7f3c0c004810 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009c00 tx=0x7f3c080091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.162+0000 7f3c1f59e700 1 -- 172.21.15.92:0/602873644 reap_dead start 2023-10-06T10:44:25.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.162+0000 7f3c2500c700 1 -- 172.21.15.92:0/602873644 shutdown_connections 2023-10-06T10:44:25.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.162+0000 7f3c2500c700 1 -- 172.21.15.92:0/602873644 >> 172.21.15.92:0/602873644 conn(0x7f3c20099700 msgr2=0x7f3c2006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.162+0000 7f3c2500c700 1 -- 172.21.15.92:0/602873644 shutdown_connections 2023-10-06T10:44:25.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.162+0000 7f3c2500c700 1 -- 172.21.15.92:0/602873644 wait complete. 2023-10-06T10:44:25.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.163+0000 7f3c2500c700 1 Processor -- start 2023-10-06T10:44:25.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.163+0000 7f3c2500c700 1 -- start start 2023-10-06T10:44:25.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.164+0000 7f3c2500c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c20169a30 0x7f3c2009aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.164+0000 7f3c2500c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c2009d320 0x7f3c20169e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.164+0000 7f3c2500c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c20068a30 0x7f3c20068ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.164+0000 7f3c2500c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c20069410 con 0x7f3c20169a30 2023-10-06T10:44:25.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.164+0000 7f3c2500c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2009dd70 con 0x7f3c20068a30 2023-10-06T10:44:25.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.164+0000 7f3c2500c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2016a340 con 0x7f3c2009d320 2023-10-06T10:44:25.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.164+0000 7f3c1ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c20169a30 0x7f3c2009aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.164+0000 7f3c1ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c20169a30 0x7f3c2009aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37502/0 (socket says 172.21.15.92:37502) 2023-10-06T10:44:25.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.164+0000 7f3c1ed9d700 1 -- 172.21.15.92:0/1062607593 learned_addr learned my addr 172.21.15.92:0/1062607593 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:25.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.164+0000 7f3c1f59e700 1 --2- 172.21.15.92:0/1062607593 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c20068a30 0x7f3c20068ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.164+0000 7f3c1659c700 1 --2- 172.21.15.92:0/1062607593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c2009d320 0x7f3c20169e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.164+0000 7f3c1ed9d700 1 -- 172.21.15.92:0/1062607593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c2009d320 msgr2=0x7f3c20169e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.164+0000 7f3c1ed9d700 1 --2- 172.21.15.92:0/1062607593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c2009d320 0x7f3c20169e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.164+0000 7f3c1ed9d700 1 -- 172.21.15.92:0/1062607593 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c20068a30 msgr2=0x7f3c20068ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.164+0000 7f3c1ed9d700 1 --2- 172.21.15.92:0/1062607593 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c20068a30 0x7f3c20068ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.164+0000 7f3c1ed9d700 1 -- 172.21.15.92:0/1062607593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2009da90 con 0x7f3c20169a30 2023-10-06T10:44:25.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.164+0000 7f3c1659c700 1 --2- 172.21.15.92:0/1062607593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c2009d320 0x7f3c20169e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:25.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.164+0000 7f3c1ed9d700 1 --2- 172.21.15.92:0/1062607593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c20169a30 0x7f3c2009aeb0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f3c10007860 tx=0x7f3c10009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.165+0000 7f3c177fe700 1 -- 172.21.15.92:0/1062607593 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c100072e0 con 0x7f3c20169a30 2023-10-06T10:44:25.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.165+0000 7f3c177fe700 1 -- 172.21.15.92:0/1062607593 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c10007440 con 0x7f3c20169a30 2023-10-06T10:44:25.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.165+0000 7f3c177fe700 1 -- 172.21.15.92:0/1062607593 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c100057f0 con 0x7f3c20169a30 2023-10-06T10:44:25.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.165+0000 7f3c2500c700 1 -- 172.21.15.92:0/1062607593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c200b46d0 con 0x7f3c20169a30 2023-10-06T10:44:25.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.165+0000 7f3c2500c700 1 -- 172.21.15.92:0/1062607593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c200b4c90 con 0x7f3c20169a30 2023-10-06T10:44:25.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.166+0000 7f3c177fe700 1 -- 172.21.15.92:0/1062607593 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c10005b60 con 0x7f3c20169a30 2023-10-06T10:44:25.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.166+0000 7f3c177fe700 1 --2- 172.21.15.92:0/1062607593 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c00066f30 0x7f3c000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.166+0000 7f3c177fe700 1 -- 172.21.15.92:0/1062607593 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3c10084810 con 0x7f3c20169a30 2023-10-06T10:44:25.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.167+0000 7f3c1659c700 1 --2- 172.21.15.92:0/1062607593 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c00066f30 0x7f3c000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.167+0000 7f3c1659c700 1 --2- 172.21.15.92:0/1062607593 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c00066f30 0x7f3c000693e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3c080091e0 tx=0x7f3c0801a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.168+0000 7f3c2e304b80 1 Processor -- start 2023-10-06T10:44:25.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.169+0000 7f3c2e304b80 1 -- start start 2023-10-06T10:44:25.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.169+0000 7f3c2e304b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993db8ae0 0x559993db8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.169+0000 7f3c2e304b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559993d749f0 0x559993d715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.169+0000 7f3c2e304b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559993d72c20 0x559993d71b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.169+0000 7f3c2e304b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559993cf1cd0 con 0x559993db8ae0 2023-10-06T10:44:25.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.169+0000 7f3c2e304b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559993cc8ac0 con 0x559993d72c20 2023-10-06T10:44:25.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.169+0000 7f3c2e304b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559993cc62c0 con 0x559993d749f0 2023-10-06T10:44:25.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.169+0000 7f3c1ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993db8ae0 0x559993db8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.169+0000 7f3c1ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993db8ae0 0x559993db8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37506/0 (socket says 172.21.15.92:37506) 2023-10-06T10:44:25.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.169+0000 7f3c1ed9d700 1 -- 172.21.15.92:0/1588655803 learned_addr learned my addr 172.21.15.92:0/1588655803 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:25.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.170+0000 7f3c1f59e700 1 --2- 172.21.15.92:0/1588655803 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559993d72c20 0x559993d71b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.170+0000 7f3c1659c700 1 --2- 172.21.15.92:0/1588655803 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559993d749f0 0x559993d715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.170+0000 7f3c1ed9d700 1 -- 172.21.15.92:0/1588655803 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559993d749f0 msgr2=0x559993d715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.170+0000 7f3c1ed9d700 1 --2- 172.21.15.92:0/1588655803 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559993d749f0 0x559993d715d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.170+0000 7f3c1ed9d700 1 -- 172.21.15.92:0/1588655803 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559993d72c20 msgr2=0x559993d71b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.170+0000 7f3c1ed9d700 1 --2- 172.21.15.92:0/1588655803 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559993d72c20 0x559993d71b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.170+0000 7f3c1ed9d700 1 -- 172.21.15.92:0/1588655803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559993b5c2c0 con 0x559993db8ae0 2023-10-06T10:44:25.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.170+0000 7f3c1f59e700 1 --2- 172.21.15.92:0/1588655803 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559993d72c20 0x559993d71b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:25.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.170+0000 7f3c1ed9d700 1 --2- 172.21.15.92:0/1588655803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993db8ae0 0x559993db8eb0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f3c10051900 tx=0x7f3c10082950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf07c8f054168bdf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.170+0000 7f3bfffff700 1 -- 172.21.15.92:0/1588655803 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c100551d0 con 0x559993db8ae0 2023-10-06T10:44:25.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.170+0000 7f3bfffff700 1 -- 172.21.15.92:0/1588655803 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c10055b00 con 0x559993db8ae0 2023-10-06T10:44:25.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.170+0000 7f3bfffff700 1 -- 172.21.15.92:0/1588655803 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c10055400 con 0x559993db8ae0 2023-10-06T10:44:25.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.171+0000 7f3c2e304b80 1 -- 172.21.15.92:0/1588655803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993db8ae0 msgr2=0x559993db8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.171+0000 7f3c2e304b80 1 --2- 172.21.15.92:0/1588655803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993db8ae0 0x559993db8eb0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f3c10051900 tx=0x7f3c10082950 comp rx=0 tx=0).stop 2023-10-06T10:44:25.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.171+0000 7f3c2e304b80 1 -- 172.21.15.92:0/1588655803 shutdown_connections 2023-10-06T10:44:25.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.171+0000 7f3c2e304b80 1 --2- 172.21.15.92:0/1588655803 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559993d72c20 0x559993d71b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.171+0000 7f3c2e304b80 1 --2- 172.21.15.92:0/1588655803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993db8ae0 0x559993db8eb0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.171+0000 7f3c2e304b80 1 --2- 172.21.15.92:0/1588655803 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559993d749f0 0x559993d715d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.171+0000 7f3c2e304b80 1 -- 172.21.15.92:0/1588655803 >> 172.21.15.92:0/1588655803 conn(0x559993d730c0 msgr2=0x559993d73490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.171+0000 7f3c2e304b80 1 -- 172.21.15.92:0/1588655803 shutdown_connections 2023-10-06T10:44:25.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.171+0000 7f3c2e304b80 1 -- 172.21.15.92:0/1588655803 wait complete. 2023-10-06T10:44:25.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.171+0000 7f3c2e304b80 1 -- start start 2023-10-06T10:44:25.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.171+0000 7f3c2e304b80 1 Processor -- start 2023-10-06T10:44:25.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.172+0000 7f3c2e304b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559993d72c20 0x559993db5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.172+0000 7f3c2e304b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559993d749f0 0x559993d9b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.172+0000 7f3c2e304b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993db8ae0 0x559993d715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.172+0000 7f3c2e304b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559993cc20b0 con 0x559993db8ae0 2023-10-06T10:44:25.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.172+0000 7f3c2e304b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559993c6d8e0 con 0x559993d72c20 2023-10-06T10:44:25.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.172+0000 7f3c2e304b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559993c88ef0 con 0x559993d749f0 2023-10-06T10:44:25.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.172+0000 7f3c2e304b80 10 client.0 _ll_register_callbacks cb 0x559993b5ef80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:25.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.172+0000 7f3c1f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993db8ae0 0x559993d715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.172+0000 7f3c1f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993db8ae0 0x559993d715d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37512/0 (socket says 172.21.15.92:37512) 2023-10-06T10:44:25.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.172+0000 7f3c1f59e700 1 -- 172.21.15.92:0/3374052629 learned_addr learned my addr 172.21.15.92:0/3374052629 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:25.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3c1659c700 1 --2- 172.21.15.92:0/3374052629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559993d749f0 0x559993d9b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3c1ed9d700 1 --2- 172.21.15.92:0/3374052629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559993d72c20 0x559993db5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3c1f59e700 1 -- 172.21.15.92:0/3374052629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559993d749f0 msgr2=0x559993d9b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3c1f59e700 1 --2- 172.21.15.92:0/3374052629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559993d749f0 0x559993d9b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3c1f59e700 1 -- 172.21.15.92:0/3374052629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559993d72c20 msgr2=0x559993db5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3c1f59e700 1 --2- 172.21.15.92:0/3374052629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559993d72c20 0x559993db5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3c1f59e700 1 -- 172.21.15.92:0/3374052629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c200b4c90 con 0x559993db8ae0 2023-10-06T10:44:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3c1659c700 1 --2- 172.21.15.92:0/3374052629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559993d749f0 0x559993d9b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3c1ed9d700 1 --2- 172.21.15.92:0/3374052629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559993d72c20 0x559993db5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3c1f59e700 1 --2- 172.21.15.92:0/3374052629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993db8ae0 0x559993d715d0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f3c20071190 tx=0x7f3c1800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39bcf994649ad0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3bfd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3bfd7fa700 1 -- 172.21.15.92:0/3374052629 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1800e040 con 0x559993db8ae0 2023-10-06T10:44:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3bfd7fa700 1 -- 172.21.15.92:0/3374052629 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3c18013920 con 0x559993db8ae0 2023-10-06T10:44:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3c2e304b80 1 -- 172.21.15.92:0/3374052629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559993d666e0 con 0x559993db8ae0 2023-10-06T10:44:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3bfd7fa700 1 -- 172.21.15.92:0/3374052629 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c18013d10 con 0x559993db8ae0 2023-10-06T10:44:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3c2e304b80 10 client.8247 fetch_fsmap learned FSMap version 92 2023-10-06T10:44:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3c2e304b80 1 -- 172.21.15.92:0/3374052629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) v3 -- 0x559993aff030 con 0x559993db8ae0 2023-10-06T10:44:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3bfd7fa700 1 -- 172.21.15.92:0/3374052629 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559993d666e0 con 0x559993db8ae0 2023-10-06T10:44:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3bfd7fa700 1 -- 172.21.15.92:0/3374052629 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 92) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f3c1801f030 con 0x559993db8ae0 2023-10-06T10:44:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3c2e304b80 10 client.8247 fetch_fsmap finished waiting for FSMap version 92 2023-10-06T10:44:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3c2e304b80 10 client.8247 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2023-10-06T10:44:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3c2e304b80 20 client.8247 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3c2e304b80 1 --2- 172.21.15.92:0/3374052629 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x559993d987c0 0x559993d71b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3c2e304b80 4 client.8247 mds_command: new command op to 7701 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:44:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.173+0000 7f3c2e304b80 1 -- 172.21.15.92:0/3374052629 --> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559993d43610 con 0x559993d987c0 2023-10-06T10:44:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.174+0000 7f3c1ed9d700 1 --2- 172.21.15.92:0/3374052629 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x559993d987c0 0x559993d71b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.174+0000 7f3bfd7fa700 10 client.8247 ms_handle_connect on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.174+0000 7f3c1ed9d700 1 --2- 172.21.15.92:0/3374052629 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x559993d987c0 0x559993d71b10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c8bb5811e45d5ff server_cookie=ff504de16514bfb5 in_seq=0 out_seq=0 2023-10-06T10:44:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.174+0000 7f3bfd7fa700 1 -- 172.21.15.92:0/3374052629 <== mds.0 v2:172.21.15.124:6832/4203118992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x559993d43610 con 0x559993d987c0 2023-10-06T10:44:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.174+0000 7f3bfd7fa700 10 client.8247 handle_command_reply: tid=0 2023-10-06T10:44:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.174+0000 7f3c2e304b80 1 client.8247 shutdown 2023-10-06T10:44:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.175+0000 7f3c2e304b80 1 -- 172.21.15.92:0/3374052629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993db8ae0 msgr2=0x559993d715d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.175+0000 7f3c2e304b80 1 --2- 172.21.15.92:0/3374052629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993db8ae0 0x559993d715d0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f3c20071190 tx=0x7f3c1800b040 comp rx=0 tx=0).stop 2023-10-06T10:44:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.175+0000 7f3c2e304b80 1 -- 172.21.15.92:0/3374052629 shutdown_connections 2023-10-06T10:44:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.175+0000 7f3c2e304b80 1 --2- 172.21.15.92:0/3374052629 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x559993d987c0 0x559993d71b10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.175+0000 7f3c2e304b80 1 --2- 172.21.15.92:0/3374052629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993db8ae0 0x559993d715d0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.175+0000 7f3bfd7fa700 0 client.8247 ms_handle_reset on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.175+0000 7f3c2e304b80 1 --2- 172.21.15.92:0/3374052629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559993d72c20 0x559993db5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.175+0000 7f3c2e304b80 1 --2- 172.21.15.92:0/3374052629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559993d749f0 0x559993d9b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.175+0000 7f3c2e304b80 1 -- 172.21.15.92:0/3374052629 >> 172.21.15.92:0/3374052629 conn(0x559993d730c0 msgr2=0x559993d6dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.175+0000 7f3c2e304b80 1 -- 172.21.15.92:0/3374052629 shutdown_connections 2023-10-06T10:44:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.175+0000 7f3c2e304b80 1 -- 172.21.15.92:0/3374052629 wait complete. 2023-10-06T10:44:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.175+0000 7f3c2e304b80 20 client.8247 trim_cache size 0 max 16384 2023-10-06T10:44:25.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.190+0000 7f3c2e304b80 1 Processor -- start 2023-10-06T10:44:25.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.190+0000 7f3c2e304b80 1 -- start start 2023-10-06T10:44:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.191+0000 7f3c2e304b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559993d72c20 0x559993d6d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.191+0000 7f3c2e304b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559993d749f0 0x559993dbb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.191+0000 7f3c2e304b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993d987c0 0x559993db65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.191+0000 7f3c2e304b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559993b44e80 con 0x559993d987c0 2023-10-06T10:44:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.191+0000 7f3c2e304b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559993db8620 con 0x559993d749f0 2023-10-06T10:44:25.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.191+0000 7f3c2e304b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559993d74350 con 0x559993d72c20 2023-10-06T10:44:25.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.191+0000 7f3c1f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993d987c0 0x559993db65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.191+0000 7f3c1f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993d987c0 0x559993db65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37518/0 (socket says 172.21.15.92:37518) 2023-10-06T10:44:25.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.191+0000 7f3c1f59e700 1 -- 172.21.15.92:0/1884737077 learned_addr learned my addr 172.21.15.92:0/1884737077 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:25.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.191+0000 7f3c1f59e700 1 -- 172.21.15.92:0/1884737077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559993d72c20 msgr2=0x559993d6d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.191+0000 7f3c1f59e700 1 --2- 172.21.15.92:0/1884737077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559993d72c20 0x559993d6d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.191+0000 7f3c1f59e700 1 -- 172.21.15.92:0/1884737077 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559993d749f0 msgr2=0x559993dbb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.191+0000 7f3c1f59e700 1 --2- 172.21.15.92:0/1884737077 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559993d749f0 0x559993dbb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.191+0000 7f3c1f59e700 1 -- 172.21.15.92:0/1884737077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559993c907d0 con 0x559993d987c0 2023-10-06T10:44:25.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.192+0000 7f3c1f59e700 1 --2- 172.21.15.92:0/1884737077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993d987c0 0x559993db65e0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f3c20071190 tx=0x7f3c18020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4091d03ec7b77665 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.192+0000 7f3c14d99700 1 -- 172.21.15.92:0/1884737077 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1800e070 con 0x559993d987c0 2023-10-06T10:44:25.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.192+0000 7f3c14d99700 1 -- 172.21.15.92:0/1884737077 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1800f040 con 0x559993d987c0 2023-10-06T10:44:25.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.192+0000 7f3c14d99700 1 -- 172.21.15.92:0/1884737077 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c18021d70 con 0x559993d987c0 2023-10-06T10:44:25.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.192+0000 7f3c2e304b80 1 -- 172.21.15.92:0/1884737077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993d987c0 msgr2=0x559993db65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.192+0000 7f3c2e304b80 1 --2- 172.21.15.92:0/1884737077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993d987c0 0x559993db65e0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f3c20071190 tx=0x7f3c18020000 comp rx=0 tx=0).stop 2023-10-06T10:44:25.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.192+0000 7f3c2e304b80 1 -- 172.21.15.92:0/1884737077 shutdown_connections 2023-10-06T10:44:25.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.192+0000 7f3c2e304b80 1 --2- 172.21.15.92:0/1884737077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993d987c0 0x559993db65e0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.192+0000 7f3c2e304b80 1 --2- 172.21.15.92:0/1884737077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559993d72c20 0x559993d6d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.192+0000 7f3c2e304b80 1 --2- 172.21.15.92:0/1884737077 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559993d749f0 0x559993dbb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.192+0000 7f3c2e304b80 1 -- 172.21.15.92:0/1884737077 >> 172.21.15.92:0/1884737077 conn(0x559993d730c0 msgr2=0x559993d6ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.192+0000 7f3c2e304b80 1 -- 172.21.15.92:0/1884737077 shutdown_connections 2023-10-06T10:44:25.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.192+0000 7f3c2e304b80 1 -- 172.21.15.92:0/1884737077 wait complete. 2023-10-06T10:44:25.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.193+0000 7f3c2e304b80 1 -- start start 2023-10-06T10:44:25.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.193+0000 7f3c2e304b80 1 Processor -- start 2023-10-06T10:44:25.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.194+0000 7f3c2e304b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559993d72c20 0x559993d9ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.194+0000 7f3c2e304b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559993d749f0 0x559993e06610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.194+0000 7f3c2e304b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993d987c0 0x559993d97740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.194+0000 7f3c2e304b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559993e1d830 con 0x559993d987c0 2023-10-06T10:44:25.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.194+0000 7f3c2e304b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x559993e06b50 con 0x559993d72c20 2023-10-06T10:44:25.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.194+0000 7f3c2e304b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x559993d72470 con 0x559993d749f0 2023-10-06T10:44:25.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.194+0000 7f3c2e304b80 10 client.0 _ll_register_callbacks cb 0x559993b5ef80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:25.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.194+0000 7f3c1f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993d987c0 0x559993d97740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.194+0000 7f3c1f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993d987c0 0x559993d97740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37532/0 (socket says 172.21.15.92:37532) 2023-10-06T10:44:25.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.194+0000 7f3c1f59e700 1 -- 172.21.15.92:0/113159900 learned_addr learned my addr 172.21.15.92:0/113159900 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:25.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.194+0000 7f3c1ed9d700 1 --2- 172.21.15.92:0/113159900 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559993d72c20 0x559993d9ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.194+0000 7f3c1659c700 1 --2- 172.21.15.92:0/113159900 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559993d749f0 0x559993e06610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.194+0000 7f3c1f59e700 1 -- 172.21.15.92:0/113159900 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559993d749f0 msgr2=0x559993e06610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.194+0000 7f3c1f59e700 1 --2- 172.21.15.92:0/113159900 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559993d749f0 0x559993e06610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.194+0000 7f3c1f59e700 1 -- 172.21.15.92:0/113159900 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559993d72c20 msgr2=0x559993d9ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.194+0000 7f3c1f59e700 1 --2- 172.21.15.92:0/113159900 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559993d72c20 0x559993d9ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.194+0000 7f3c1f59e700 1 -- 172.21.15.92:0/113159900 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559993aff030 con 0x559993d987c0 2023-10-06T10:44:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.194+0000 7f3c1ed9d700 1 --2- 172.21.15.92:0/113159900 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559993d72c20 0x559993d9ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.194+0000 7f3c1f59e700 1 --2- 172.21.15.92:0/113159900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993d987c0 0x559993d97740 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f3c18030040 tx=0x7f3c1802d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d448f1fa30ec8c27 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.194+0000 7f3bfd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.194+0000 7f3bfd7fa700 1 -- 172.21.15.92:0/113159900 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1800e040 con 0x559993d987c0 2023-10-06T10:44:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.195+0000 7f3bfd7fa700 1 -- 172.21.15.92:0/113159900 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3c1803adb0 con 0x559993d987c0 2023-10-06T10:44:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.195+0000 7f3c2e304b80 1 -- 172.21.15.92:0/113159900 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c10055b00 con 0x559993d987c0 2023-10-06T10:44:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.195+0000 7f3bfd7fa700 1 -- 172.21.15.92:0/113159900 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c180137a0 con 0x559993d987c0 2023-10-06T10:44:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.195+0000 7f3c2e304b80 10 client.8253 fetch_fsmap learned FSMap version 92 2023-10-06T10:44:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.195+0000 7f3c2e304b80 1 -- 172.21.15.92:0/113159900 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) v3 -- 0x559993c774c0 con 0x559993d987c0 2023-10-06T10:44:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.195+0000 7f3bfd7fa700 1 -- 172.21.15.92:0/113159900 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c10055b00 con 0x559993d987c0 2023-10-06T10:44:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.195+0000 7f3bfd7fa700 1 -- 172.21.15.92:0/113159900 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 92) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f3c18040020 con 0x559993d987c0 2023-10-06T10:44:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.195+0000 7f3c2e304b80 10 client.8253 fetch_fsmap finished waiting for FSMap version 92 2023-10-06T10:44:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.195+0000 7f3c2e304b80 10 client.8253 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2023-10-06T10:44:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.195+0000 7f3c2e304b80 20 client.8253 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.195+0000 7f3c2e304b80 1 --2- 172.21.15.92:0/113159900 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x559993db8ae0 0x559993d97c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.195+0000 7f3c2e304b80 4 client.8253 mds_command: new command op to 7701 tid=0[{"prefix": "scrub resume"}] 2023-10-06T10:44:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.195+0000 7f3c2e304b80 1 -- 172.21.15.92:0/113159900 --> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x559993c64e50 con 0x559993db8ae0 2023-10-06T10:44:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.195+0000 7f3c1ed9d700 1 --2- 172.21.15.92:0/113159900 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x559993db8ae0 0x559993d97c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.196+0000 7f3c1ed9d700 1 --2- 172.21.15.92:0/113159900 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x559993db8ae0 0x559993d97c80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c4bd8f97df5c9e4 server_cookie=1bb48476e1f78d78 in_seq=0 out_seq=0 2023-10-06T10:44:25.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.196+0000 7f3bfd7fa700 10 client.8253 ms_handle_connect on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:25.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.196+0000 7f3bfd7fa700 1 -- 172.21.15.92:0/113159900 <== mds.0 v2:172.21.15.124:6832/4203118992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x559993c64e50 con 0x559993db8ae0 2023-10-06T10:44:25.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.196+0000 7f3bfd7fa700 10 client.8253 handle_command_reply: tid=0 2023-10-06T10:44:25.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.196+0000 7f3c2e304b80 1 client.8253 shutdown 2023-10-06T10:44:25.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.196+0000 7f3c2e304b80 1 -- 172.21.15.92:0/113159900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993d987c0 msgr2=0x559993d97740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.196+0000 7f3c2e304b80 1 --2- 172.21.15.92:0/113159900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993d987c0 0x559993d97740 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f3c18030040 tx=0x7f3c1802d000 comp rx=0 tx=0).stop 2023-10-06T10:44:25.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.196+0000 7f3c2e304b80 1 -- 172.21.15.92:0/113159900 shutdown_connections 2023-10-06T10:44:25.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.196+0000 7f3c2e304b80 1 --2- 172.21.15.92:0/113159900 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x559993db8ae0 0x559993d97c80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.196+0000 7f3c2e304b80 1 --2- 172.21.15.92:0/113159900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559993d987c0 0x559993d97740 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.196+0000 7f3bfd7fa700 0 client.8253 ms_handle_reset on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:25.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.196+0000 7f3c2e304b80 1 --2- 172.21.15.92:0/113159900 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x559993d72c20 0x559993d9ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.196+0000 7f3c2e304b80 1 --2- 172.21.15.92:0/113159900 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x559993d749f0 0x559993e06610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.196+0000 7f3c2e304b80 1 -- 172.21.15.92:0/113159900 >> 172.21.15.92:0/113159900 conn(0x559993d730c0 msgr2=0x559993db72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.196+0000 7f3c2e304b80 1 -- 172.21.15.92:0/113159900 shutdown_connections 2023-10-06T10:44:25.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.196+0000 7f3c2e304b80 1 -- 172.21.15.92:0/113159900 wait complete. 2023-10-06T10:44:25.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.196+0000 7f3c2e304b80 20 client.8253 trim_cache size 0 max 16384 2023-10-06T10:44:25.205 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:44:25.205 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:44:25.205 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:44:25.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.197+0000 7f3c2500c700 1 -- 172.21.15.92:0/1062607593 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c00066f30 msgr2=0x7f3c000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.197+0000 7f3c2500c700 1 --2- 172.21.15.92:0/1062607593 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c00066f30 0x7f3c000693e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3c080091e0 tx=0x7f3c0801a000 comp rx=0 tx=0).stop 2023-10-06T10:44:25.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.197+0000 7f3c2500c700 1 -- 172.21.15.92:0/1062607593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c20169a30 msgr2=0x7f3c2009aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.197+0000 7f3c2500c700 1 --2- 172.21.15.92:0/1062607593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c20169a30 0x7f3c2009aeb0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f3c10007860 tx=0x7f3c10009040 comp rx=0 tx=0).stop 2023-10-06T10:44:25.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.197+0000 7f3c2500c700 1 -- 172.21.15.92:0/1062607593 shutdown_connections 2023-10-06T10:44:25.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.197+0000 7f3c2500c700 1 --2- 172.21.15.92:0/1062607593 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c00066f30 0x7f3c000693e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.197+0000 7f3c2500c700 1 --2- 172.21.15.92:0/1062607593 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c20068a30 0x7f3c20068ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.197+0000 7f3c2500c700 1 --2- 172.21.15.92:0/1062607593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c20169a30 0x7f3c2009aeb0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.197+0000 7f3c2500c700 1 --2- 172.21.15.92:0/1062607593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c2009d320 0x7f3c20169e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.197+0000 7f3c2500c700 1 -- 172.21.15.92:0/1062607593 >> 172.21.15.92:0/1062607593 conn(0x7f3c20099700 msgr2=0x7f3c20069900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.197+0000 7f3c2500c700 1 -- 172.21.15.92:0/1062607593 shutdown_connections 2023-10-06T10:44:25.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.197+0000 7f3c2500c700 1 -- 172.21.15.92:0/1062607593 wait complete. 2023-10-06T10:44:25.209 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:1 scrub status 2023-10-06T10:44:25.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.324+0000 7ff9f0249700 1 Processor -- start 2023-10-06T10:44:25.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.324+0000 7ff9f0249700 1 -- start start 2023-10-06T10:44:25.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.325+0000 7ff9f0249700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9e8150940 0x7ff9e8159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.325+0000 7ff9f0249700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff9e815a160 0x7ff9e80ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.325+0000 7ff9f0249700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff9e80c5250 0x7ff9e80cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.325+0000 7ff9f0249700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e815ac30 con 0x7ff9e80c5250 2023-10-06T10:44:25.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.325+0000 7ff9f0249700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e80cb6a0 con 0x7ff9e815a160 2023-10-06T10:44:25.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.325+0000 7ff9f0249700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e80cb090 con 0x7ff9e8150940 2023-10-06T10:44:25.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.325+0000 7ff9ee7e6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff9e80c5250 0x7ff9e80cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.325+0000 7ff9ee7e6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff9e80c5250 0x7ff9e80cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37536/0 (socket says 172.21.15.92:37536) 2023-10-06T10:44:25.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.325+0000 7ff9ee7e6700 1 -- 172.21.15.92:0/1503047850 learned_addr learned my addr 172.21.15.92:0/1503047850 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:25.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.325+0000 7ff9edfe5700 1 --2- 172.21.15.92:0/1503047850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9e8150940 0x7ff9e8159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.325+0000 7ff9ed7e4700 1 --2- 172.21.15.92:0/1503047850 >> v2:172.21.15.124:3300/0 conn(0x7ff9e815a160 0x7ff9e80ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.325+0000 7ff9ee7e6700 1 -- 172.21.15.92:0/1503047850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9e8150940 msgr2=0x7ff9e8159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.325+0000 7ff9ee7e6700 1 --2- 172.21.15.92:0/1503047850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9e8150940 0x7ff9e8159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.325+0000 7ff9ee7e6700 1 -- 172.21.15.92:0/1503047850 >> v2:172.21.15.124:3300/0 conn(0x7ff9e815a160 msgr2=0x7ff9e80ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.326+0000 7ff9ee7e6700 1 --2- 172.21.15.92:0/1503047850 >> v2:172.21.15.124:3300/0 conn(0x7ff9e815a160 0x7ff9e80ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.326+0000 7ff9ee7e6700 1 -- 172.21.15.92:0/1503047850 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e815a670 con 0x7ff9e80c5250 2023-10-06T10:44:25.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.326+0000 7ff9ed7e4700 1 --2- 172.21.15.92:0/1503047850 >> v2:172.21.15.124:3300/0 conn(0x7ff9e815a160 0x7ff9e80ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:25.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.326+0000 7ff9edfe5700 1 --2- 172.21.15.92:0/1503047850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9e8150940 0x7ff9e8159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:25.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.326+0000 7ff9ee7e6700 1 --2- 172.21.15.92:0/1503047850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e80c5250 0x7ff9e80cd300 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8089590 tx=0x7ff9e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821501b659f8471 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.326+0000 7ff9ecfe3700 1 -- 172.21.15.92:0/1503047850 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e000b860 con 0x7ff9e80c5250 2023-10-06T10:44:25.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.326+0000 7ff9ecfe3700 1 -- 172.21.15.92:0/1503047850 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e000b9c0 con 0x7ff9e80c5250 2023-10-06T10:44:25.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.326+0000 7ff9ecfe3700 1 -- 172.21.15.92:0/1503047850 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e000bd60 con 0x7ff9e80c5250 2023-10-06T10:44:25.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.326+0000 7ff9f0249700 1 -- 172.21.15.92:0/1503047850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e80c5250 msgr2=0x7ff9e80cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.326+0000 7ff9f0249700 1 --2- 172.21.15.92:0/1503047850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e80c5250 0x7ff9e80cd300 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8089590 tx=0x7ff9e0009250 comp rx=0 tx=0).stop 2023-10-06T10:44:25.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.326+0000 7ff9f0249700 1 -- 172.21.15.92:0/1503047850 shutdown_connections 2023-10-06T10:44:25.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.326+0000 7ff9f0249700 1 --2- 172.21.15.92:0/1503047850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e80c5250 0x7ff9e80cd300 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.326+0000 7ff9f0249700 1 --2- 172.21.15.92:0/1503047850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9e8150940 0x7ff9e8159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.326+0000 7ff9f0249700 1 --2- 172.21.15.92:0/1503047850 >> v2:172.21.15.124:3300/0 conn(0x7ff9e815a160 0x7ff9e80ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.326+0000 7ff9f0249700 1 -- 172.21.15.92:0/1503047850 >> 172.21.15.92:0/1503047850 conn(0x7ff9e8099700 msgr2=0x7ff9e806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.326+0000 7ff9f0249700 1 -- 172.21.15.92:0/1503047850 shutdown_connections 2023-10-06T10:44:25.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.326+0000 7ff9f0249700 1 -- 172.21.15.92:0/1503047850 wait complete. 2023-10-06T10:44:25.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.328+0000 7ff9f0249700 1 Processor -- start 2023-10-06T10:44:25.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.328+0000 7ff9f0249700 1 -- start start 2023-10-06T10:44:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.328+0000 7ff9f0249700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e80c5250 0x7ff9e81542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.328+0000 7ff9f0249700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9e815a160 0x7ff9e813d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.328+0000 7ff9f0249700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9e8155b50 0x7ff9e8151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.328+0000 7ff9f0249700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e81556c0 con 0x7ff9e80c5250 2023-10-06T10:44:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.328+0000 7ff9f0249700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e8154810 con 0x7ff9e815a160 2023-10-06T10:44:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.328+0000 7ff9f0249700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e8154ab0 con 0x7ff9e8155b50 2023-10-06T10:44:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.328+0000 7ff9edfe5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e80c5250 0x7ff9e81542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.328+0000 7ff9edfe5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e80c5250 0x7ff9e81542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37548/0 (socket says 172.21.15.92:37548) 2023-10-06T10:44:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.328+0000 7ff9edfe5700 1 -- 172.21.15.92:0/3901605040 learned_addr learned my addr 172.21.15.92:0/3901605040 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.328+0000 7ff9ed7e4700 1 --2- 172.21.15.92:0/3901605040 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9e815a160 0x7ff9e813d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.329+0000 7ff9ee7e6700 1 --2- 172.21.15.92:0/3901605040 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9e8155b50 0x7ff9e8151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.329+0000 7ff9edfe5700 1 -- 172.21.15.92:0/3901605040 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9e8155b50 msgr2=0x7ff9e8151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.329+0000 7ff9edfe5700 1 --2- 172.21.15.92:0/3901605040 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9e8155b50 0x7ff9e8151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.329+0000 7ff9edfe5700 1 -- 172.21.15.92:0/3901605040 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9e815a160 msgr2=0x7ff9e813d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.329+0000 7ff9edfe5700 1 --2- 172.21.15.92:0/3901605040 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9e815a160 0x7ff9e813d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.329+0000 7ff9edfe5700 1 -- 172.21.15.92:0/3901605040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e00094f0 con 0x7ff9e80c5250 2023-10-06T10:44:25.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.329+0000 7ff9ed7e4700 1 --2- 172.21.15.92:0/3901605040 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9e815a160 0x7ff9e813d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:25.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.329+0000 7ff9edfe5700 1 --2- 172.21.15.92:0/3901605040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e80c5250 0x7ff9e81542d0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800a4d0 tx=0x7ff9d80078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.329+0000 7ff9e5ffb700 1 -- 172.21.15.92:0/3901605040 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8008590 con 0x7ff9e80c5250 2023-10-06T10:44:25.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.329+0000 7ff9e5ffb700 1 -- 172.21.15.92:0/3901605040 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d80086f0 con 0x7ff9e80c5250 2023-10-06T10:44:25.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.329+0000 7ff9f0249700 1 -- 172.21.15.92:0/3901605040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e8154d90 con 0x7ff9e80c5250 2023-10-06T10:44:25.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.329+0000 7ff9e5ffb700 1 -- 172.21.15.92:0/3901605040 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8008a90 con 0x7ff9e80c5250 2023-10-06T10:44:25.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.329+0000 7ff9f0249700 1 -- 172.21.15.92:0/3901605040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e8155350 con 0x7ff9e80c5250 2023-10-06T10:44:25.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.330+0000 7ff9e5ffb700 1 -- 172.21.15.92:0/3901605040 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff9d8014410 con 0x7ff9e80c5250 2023-10-06T10:44:25.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.331+0000 7ff9e5ffb700 1 --2- 172.21.15.92:0/3901605040 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff9d4066f30 0x7ff9d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.331+0000 7ff9e5ffb700 1 -- 172.21.15.92:0/3901605040 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff9d8085420 con 0x7ff9e80c5250 2023-10-06T10:44:25.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.331+0000 7ff9ed7e4700 1 --2- 172.21.15.92:0/3901605040 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff9d4066f30 0x7ff9d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.331+0000 7ff9ed7e4700 1 --2- 172.21.15.92:0/3901605040 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff9d4066f30 0x7ff9d40693e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff9e81506c0 tx=0x7ff9dc006f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.333+0000 7ff9f9541b80 1 Processor -- start 2023-10-06T10:44:25.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.333+0000 7ff9f9541b80 1 -- start start 2023-10-06T10:44:25.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.333+0000 7ff9f9541b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635c452aae0 0x5635c452aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.333+0000 7ff9f9541b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c44e69f0 0x5635c44e35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.333+0000 7ff9f9541b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635c44e4c20 0x5635c44e3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.333+0000 7ff9f9541b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635c4463cd0 con 0x5635c44e69f0 2023-10-06T10:44:25.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.333+0000 7ff9f9541b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635c443aac0 con 0x5635c44e4c20 2023-10-06T10:44:25.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.333+0000 7ff9f9541b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635c44382c0 con 0x5635c452aae0 2023-10-06T10:44:25.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.334+0000 7ff9ed7e4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c44e69f0 0x5635c44e35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.334+0000 7ff9ed7e4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c44e69f0 0x5635c44e35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37554/0 (socket says 172.21.15.92:37554) 2023-10-06T10:44:25.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.334+0000 7ff9ed7e4700 1 -- 172.21.15.92:0/3978728325 learned_addr learned my addr 172.21.15.92:0/3978728325 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:25.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.334+0000 7ff9edfe5700 1 --2- 172.21.15.92:0/3978728325 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635c452aae0 0x5635c452aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.334+0000 7ff9ee7e6700 1 --2- 172.21.15.92:0/3978728325 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635c44e4c20 0x5635c44e3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.334+0000 7ff9ed7e4700 1 -- 172.21.15.92:0/3978728325 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635c452aae0 msgr2=0x5635c452aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.334+0000 7ff9ed7e4700 1 --2- 172.21.15.92:0/3978728325 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635c452aae0 0x5635c452aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.334+0000 7ff9ed7e4700 1 -- 172.21.15.92:0/3978728325 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635c44e4c20 msgr2=0x5635c44e3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.334+0000 7ff9ed7e4700 1 --2- 172.21.15.92:0/3978728325 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635c44e4c20 0x5635c44e3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.334+0000 7ff9ed7e4700 1 -- 172.21.15.92:0/3978728325 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635c42ce2c0 con 0x5635c44e69f0 2023-10-06T10:44:25.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.334+0000 7ff9ee7e6700 1 --2- 172.21.15.92:0/3978728325 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635c44e4c20 0x5635c44e3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:25.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.334+0000 7ff9ed7e4700 1 --2- 172.21.15.92:0/3978728325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c44e69f0 0x5635c44e35d0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc00fb40 tx=0x7ff9dc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac85c12e50a0c45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.334+0000 7ff9ceffd700 1 -- 172.21.15.92:0/3978728325 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc0148f0 con 0x5635c44e69f0 2023-10-06T10:44:25.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.334+0000 7ff9ceffd700 1 -- 172.21.15.92:0/3978728325 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9dc019070 con 0x5635c44e69f0 2023-10-06T10:44:25.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.334+0000 7ff9ceffd700 1 -- 172.21.15.92:0/3978728325 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc014c60 con 0x5635c44e69f0 2023-10-06T10:44:25.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.335+0000 7ff9f9541b80 1 -- 172.21.15.92:0/3978728325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c44e69f0 msgr2=0x5635c44e35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.335+0000 7ff9f9541b80 1 --2- 172.21.15.92:0/3978728325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c44e69f0 0x5635c44e35d0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc00fb40 tx=0x7ff9dc016000 comp rx=0 tx=0).stop 2023-10-06T10:44:25.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.335+0000 7ff9f9541b80 1 -- 172.21.15.92:0/3978728325 shutdown_connections 2023-10-06T10:44:25.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.335+0000 7ff9f9541b80 1 --2- 172.21.15.92:0/3978728325 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635c44e4c20 0x5635c44e3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.335+0000 7ff9f9541b80 1 --2- 172.21.15.92:0/3978728325 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635c452aae0 0x5635c452aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.335+0000 7ff9f9541b80 1 --2- 172.21.15.92:0/3978728325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c44e69f0 0x5635c44e35d0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.335+0000 7ff9f9541b80 1 -- 172.21.15.92:0/3978728325 >> 172.21.15.92:0/3978728325 conn(0x5635c44e50c0 msgr2=0x5635c44e5490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.335+0000 7ff9f9541b80 1 -- 172.21.15.92:0/3978728325 shutdown_connections 2023-10-06T10:44:25.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.335+0000 7ff9f9541b80 1 -- 172.21.15.92:0/3978728325 wait complete. 2023-10-06T10:44:25.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.335+0000 7ff9f9541b80 1 -- start start 2023-10-06T10:44:25.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.335+0000 7ff9f9541b80 1 Processor -- start 2023-10-06T10:44:25.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.336+0000 7ff9f9541b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635c44e4c20 0x5635c450d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.336+0000 7ff9f9541b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c44e69f0 0x5635c4507d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.336+0000 7ff9f9541b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635c452aae0 0x5635c450a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.336+0000 7ff9f9541b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635c44340b0 con 0x5635c44e69f0 2023-10-06T10:44:25.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.336+0000 7ff9f9541b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635c43df8e0 con 0x5635c452aae0 2023-10-06T10:44:25.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.336+0000 7ff9f9541b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635c43faef0 con 0x5635c44e4c20 2023-10-06T10:44:25.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.336+0000 7ff9f9541b80 10 client.0 _ll_register_callbacks cb 0x5635c42d0f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:25.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.336+0000 7ff9ed7e4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c44e69f0 0x5635c4507d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.336+0000 7ff9ed7e4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c44e69f0 0x5635c4507d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37568/0 (socket says 172.21.15.92:37568) 2023-10-06T10:44:25.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.336+0000 7ff9ed7e4700 1 -- 172.21.15.92:0/1345336948 learned_addr learned my addr 172.21.15.92:0/1345336948 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:25.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.337+0000 7ff9edfe5700 1 --2- 172.21.15.92:0/1345336948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635c44e4c20 0x5635c450d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.337+0000 7ff9ee7e6700 1 --2- 172.21.15.92:0/1345336948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635c452aae0 0x5635c450a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.337+0000 7ff9ed7e4700 1 -- 172.21.15.92:0/1345336948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635c44e4c20 msgr2=0x5635c450d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.337+0000 7ff9ed7e4700 1 --2- 172.21.15.92:0/1345336948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635c44e4c20 0x5635c450d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.337+0000 7ff9ed7e4700 1 -- 172.21.15.92:0/1345336948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635c452aae0 msgr2=0x5635c450a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.337+0000 7ff9ed7e4700 1 --2- 172.21.15.92:0/1345336948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635c452aae0 0x5635c450a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.337+0000 7ff9ed7e4700 1 -- 172.21.15.92:0/1345336948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9dc00e040 con 0x5635c44e69f0 2023-10-06T10:44:25.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.337+0000 7ff9ed7e4700 1 --2- 172.21.15.92:0/1345336948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c44e69f0 0x5635c4507d70 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc021000 tx=0x7ff9dc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d4cbd809d31d84 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.337+0000 7ff9c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:25.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.337+0000 7ff9c7fff700 1 -- 172.21.15.92:0/1345336948 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc020860 con 0x5635c44e69f0 2023-10-06T10:44:25.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.337+0000 7ff9c7fff700 1 -- 172.21.15.92:0/1345336948 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff9dc0209c0 con 0x5635c44e69f0 2023-10-06T10:44:25.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.337+0000 7ff9f9541b80 1 -- 172.21.15.92:0/1345336948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635c44d86e0 con 0x5635c44e69f0 2023-10-06T10:44:25.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.337+0000 7ff9c7fff700 1 -- 172.21.15.92:0/1345336948 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9dc020d90 con 0x5635c44e69f0 2023-10-06T10:44:25.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.337+0000 7ff9f9541b80 10 client.8265 fetch_fsmap learned FSMap version 92 2023-10-06T10:44:25.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.337+0000 7ff9f9541b80 1 -- 172.21.15.92:0/1345336948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) v3 -- 0x5635c4271030 con 0x5635c44e69f0 2023-10-06T10:44:25.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.337+0000 7ff9c7fff700 1 -- 172.21.15.92:0/1345336948 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5635c44d86e0 con 0x5635c44e69f0 2023-10-06T10:44:25.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.337+0000 7ff9c7fff700 1 -- 172.21.15.92:0/1345336948 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 92) v1 ==== 3140+0+0 (secure 0 0 0) 0x7ff9dc030020 con 0x5635c44e69f0 2023-10-06T10:44:25.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.337+0000 7ff9f9541b80 10 client.8265 fetch_fsmap finished waiting for FSMap version 92 2023-10-06T10:44:25.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.337+0000 7ff9f9541b80 10 client.8265 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.c 2023-10-06T10:44:25.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.337+0000 7ff9f9541b80 20 client.8265 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:25.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.337+0000 7ff9f9541b80 1 --2- 172.21.15.92:0/1345336948 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5635c450a440 0x5635c450adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.338+0000 7ff9f9541b80 4 client.8265 mds_command: new command op to 7689 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:44:25.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.338+0000 7ff9f9541b80 1 -- 172.21.15.92:0/1345336948 --> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635c44b5610 con 0x5635c450a440 2023-10-06T10:44:25.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.338+0000 7ff9edfe5700 1 --2- 172.21.15.92:0/1345336948 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5635c450a440 0x5635c450adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.338+0000 7ff9c7fff700 10 client.8265 ms_handle_connect on v2:172.21.15.92:6835/4047304116 2023-10-06T10:44:25.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.338+0000 7ff9edfe5700 1 --2- 172.21.15.92:0/1345336948 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5635c450a440 0x5635c450adb0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d35e5d836324bbd0 server_cookie=2f1f0f8b83ef9475 in_seq=0 out_seq=0 2023-10-06T10:44:25.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.338+0000 7ff9c7fff700 1 -- 172.21.15.92:0/1345336948 <== mds.1 v2:172.21.15.92:6835/4047304116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5635c44b5610 con 0x5635c450a440 2023-10-06T10:44:25.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.338+0000 7ff9c7fff700 10 client.8265 handle_command_reply: tid=0 2023-10-06T10:44:25.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.338+0000 7ff9f9541b80 1 client.8265 shutdown 2023-10-06T10:44:25.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.338+0000 7ff9f9541b80 1 -- 172.21.15.92:0/1345336948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c44e69f0 msgr2=0x5635c4507d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.338+0000 7ff9f9541b80 1 --2- 172.21.15.92:0/1345336948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c44e69f0 0x5635c4507d70 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc021000 tx=0x7ff9dc019000 comp rx=0 tx=0).stop 2023-10-06T10:44:25.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.338+0000 7ff9f9541b80 1 -- 172.21.15.92:0/1345336948 shutdown_connections 2023-10-06T10:44:25.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.338+0000 7ff9f9541b80 1 --2- 172.21.15.92:0/1345336948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635c452aae0 0x5635c450a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.339+0000 7ff9f9541b80 1 --2- 172.21.15.92:0/1345336948 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5635c450a440 0x5635c450adb0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.339+0000 7ff9f9541b80 1 --2- 172.21.15.92:0/1345336948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635c44e4c20 0x5635c450d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.339+0000 7ff9f9541b80 1 --2- 172.21.15.92:0/1345336948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c44e69f0 0x5635c4507d70 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.339+0000 7ff9c7fff700 0 client.8265 ms_handle_reset on v2:172.21.15.92:6835/4047304116 2023-10-06T10:44:25.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.339+0000 7ff9f9541b80 1 -- 172.21.15.92:0/1345336948 >> 172.21.15.92:0/1345336948 conn(0x5635c44e50c0 msgr2=0x5635c44e0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.339+0000 7ff9f9541b80 1 -- 172.21.15.92:0/1345336948 shutdown_connections 2023-10-06T10:44:25.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.339+0000 7ff9f9541b80 1 -- 172.21.15.92:0/1345336948 wait complete. 2023-10-06T10:44:25.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.339+0000 7ff9f9541b80 20 client.8265 trim_cache size 0 max 16384 2023-10-06T10:44:25.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.354+0000 7ff9f9541b80 1 Processor -- start 2023-10-06T10:44:25.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.355+0000 7ff9f9541b80 1 -- start start 2023-10-06T10:44:25.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.355+0000 7ff9f9541b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635c44e4c20 0x5635c450eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.355+0000 7ff9f9541b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635c44e69f0 0x5635c450a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.355+0000 7ff9f9541b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c452aae0 0x5635c4512530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.355+0000 7ff9f9541b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635c42b6e80 con 0x5635c452aae0 2023-10-06T10:44:25.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.355+0000 7ff9f9541b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635c454bf50 con 0x5635c44e4c20 2023-10-06T10:44:25.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.355+0000 7ff9f9541b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635c450b1b0 con 0x5635c44e69f0 2023-10-06T10:44:25.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.355+0000 7ff9ee7e6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c452aae0 0x5635c4512530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.355+0000 7ff9ee7e6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c452aae0 0x5635c4512530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37580/0 (socket says 172.21.15.92:37580) 2023-10-06T10:44:25.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.355+0000 7ff9ee7e6700 1 -- 172.21.15.92:0/545406659 learned_addr learned my addr 172.21.15.92:0/545406659 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:25.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.356+0000 7ff9edfe5700 1 --2- 172.21.15.92:0/545406659 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635c44e4c20 0x5635c450eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.356+0000 7ff9ee7e6700 1 -- 172.21.15.92:0/545406659 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635c44e69f0 msgr2=0x5635c450a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.356+0000 7ff9ed7e4700 1 --2- 172.21.15.92:0/545406659 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635c44e69f0 0x5635c450a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.356+0000 7ff9ee7e6700 1 --2- 172.21.15.92:0/545406659 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635c44e69f0 0x5635c450a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.356+0000 7ff9ee7e6700 1 -- 172.21.15.92:0/545406659 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635c44e4c20 msgr2=0x5635c450eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.356+0000 7ff9ee7e6700 1 --2- 172.21.15.92:0/545406659 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635c44e4c20 0x5635c450eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.356+0000 7ff9ee7e6700 1 -- 172.21.15.92:0/545406659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635c44027d0 con 0x5635c452aae0 2023-10-06T10:44:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.356+0000 7ff9ee7e6700 1 --2- 172.21.15.92:0/545406659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c452aae0 0x5635c4512530 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80d14c0 tx=0x7ff9e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cde60b2a3c4f29a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.356+0000 7ff9cf7fe700 1 -- 172.21.15.92:0/545406659 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e000bcd0 con 0x5635c452aae0 2023-10-06T10:44:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.356+0000 7ff9cf7fe700 1 -- 172.21.15.92:0/545406659 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e0021050 con 0x5635c452aae0 2023-10-06T10:44:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.356+0000 7ff9cf7fe700 1 -- 172.21.15.92:0/545406659 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0011860 con 0x5635c452aae0 2023-10-06T10:44:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.356+0000 7ff9f9541b80 1 -- 172.21.15.92:0/545406659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c452aae0 msgr2=0x5635c4512530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.356+0000 7ff9f9541b80 1 --2- 172.21.15.92:0/545406659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c452aae0 0x5635c4512530 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80d14c0 tx=0x7ff9e0017000 comp rx=0 tx=0).stop 2023-10-06T10:44:25.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.356+0000 7ff9f9541b80 1 -- 172.21.15.92:0/545406659 shutdown_connections 2023-10-06T10:44:25.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.356+0000 7ff9f9541b80 1 --2- 172.21.15.92:0/545406659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c452aae0 0x5635c4512530 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.356+0000 7ff9f9541b80 1 --2- 172.21.15.92:0/545406659 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635c44e4c20 0x5635c450eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.356+0000 7ff9f9541b80 1 --2- 172.21.15.92:0/545406659 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635c44e69f0 0x5635c450a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.356+0000 7ff9f9541b80 1 -- 172.21.15.92:0/545406659 >> 172.21.15.92:0/545406659 conn(0x5635c44e50c0 msgr2=0x5635c44e15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.357+0000 7ff9f9541b80 1 -- 172.21.15.92:0/545406659 shutdown_connections 2023-10-06T10:44:25.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.357+0000 7ff9f9541b80 1 -- 172.21.15.92:0/545406659 wait complete. 2023-10-06T10:44:25.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.357+0000 7ff9f9541b80 1 -- start start 2023-10-06T10:44:25.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.357+0000 7ff9f9541b80 1 Processor -- start 2023-10-06T10:44:25.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.358+0000 7ff9f9541b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635c44e4c20 0x5635c458fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.358+0000 7ff9f9541b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635c44e69f0 0x5635c44e39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.358+0000 7ff9f9541b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c452aae0 0x5635c450bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.358+0000 7ff9f9541b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635c450ba80 con 0x5635c452aae0 2023-10-06T10:44:25.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.358+0000 7ff9f9541b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635c44dc2f0 con 0x5635c44e69f0 2023-10-06T10:44:25.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.358+0000 7ff9f9541b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635c45902f0 con 0x5635c44e4c20 2023-10-06T10:44:25.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.358+0000 7ff9f9541b80 10 client.0 _ll_register_callbacks cb 0x5635c42d0f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:25.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.358+0000 7ff9ee7e6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c452aae0 0x5635c450bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.358+0000 7ff9ee7e6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c452aae0 0x5635c450bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37592/0 (socket says 172.21.15.92:37592) 2023-10-06T10:44:25.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.358+0000 7ff9ee7e6700 1 -- 172.21.15.92:0/3222436059 learned_addr learned my addr 172.21.15.92:0/3222436059 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:25.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.358+0000 7ff9ed7e4700 1 --2- 172.21.15.92:0/3222436059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635c44e69f0 0x5635c44e39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.358+0000 7ff9edfe5700 1 --2- 172.21.15.92:0/3222436059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635c44e4c20 0x5635c458fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.358+0000 7ff9ee7e6700 1 -- 172.21.15.92:0/3222436059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635c44e4c20 msgr2=0x5635c458fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.358+0000 7ff9ee7e6700 1 --2- 172.21.15.92:0/3222436059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635c44e4c20 0x5635c458fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.358+0000 7ff9ee7e6700 1 -- 172.21.15.92:0/3222436059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635c44e69f0 msgr2=0x5635c44e39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.358+0000 7ff9ee7e6700 1 --2- 172.21.15.92:0/3222436059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635c44e69f0 0x5635c44e39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.358+0000 7ff9ee7e6700 1 -- 172.21.15.92:0/3222436059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e815a670 con 0x5635c452aae0 2023-10-06T10:44:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.358+0000 7ff9ed7e4700 1 --2- 172.21.15.92:0/3222436059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635c44e69f0 0x5635c44e39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.359+0000 7ff9ee7e6700 1 --2- 172.21.15.92:0/3222436059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c452aae0 0x5635c450bed0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0006fd0 tx=0x7ff9e001d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a94db173b5b055 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.359+0000 7ff9c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.359+0000 7ff9c7fff700 1 -- 172.21.15.92:0/3222436059 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0011630 con 0x5635c452aae0 2023-10-06T10:44:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.359+0000 7ff9c7fff700 1 -- 172.21.15.92:0/3222436059 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff9e0011790 con 0x5635c452aae0 2023-10-06T10:44:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.359+0000 7ff9f9541b80 1 -- 172.21.15.92:0/3222436059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9dc019070 con 0x5635c452aae0 2023-10-06T10:44:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.359+0000 7ff9c7fff700 1 -- 172.21.15.92:0/3222436059 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9e0027680 con 0x5635c452aae0 2023-10-06T10:44:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.359+0000 7ff9f9541b80 10 client.8271 fetch_fsmap learned FSMap version 92 2023-10-06T10:44:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.359+0000 7ff9f9541b80 1 -- 172.21.15.92:0/3222436059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) v3 -- 0x5635c43e94c0 con 0x5635c452aae0 2023-10-06T10:44:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.359+0000 7ff9c7fff700 1 -- 172.21.15.92:0/3222436059 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9dc019070 con 0x5635c452aae0 2023-10-06T10:44:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.359+0000 7ff9c7fff700 1 -- 172.21.15.92:0/3222436059 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 92) v1 ==== 3140+0+0 (secure 0 0 0) 0x7ff9e00277e0 con 0x5635c452aae0 2023-10-06T10:44:25.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.359+0000 7ff9f9541b80 10 client.8271 fetch_fsmap finished waiting for FSMap version 92 2023-10-06T10:44:25.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.359+0000 7ff9f9541b80 10 client.8271 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.c 2023-10-06T10:44:25.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.359+0000 7ff9f9541b80 20 client.8271 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:25.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.359+0000 7ff9f9541b80 1 --2- 172.21.15.92:0/3222436059 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5635c450c410 0x5635c45277c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.359+0000 7ff9f9541b80 4 client.8271 mds_command: new command op to 7689 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:44:25.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.359+0000 7ff9f9541b80 1 -- 172.21.15.92:0/3222436059 --> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5635c43d6e50 con 0x5635c450c410 2023-10-06T10:44:25.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.359+0000 7ff9edfe5700 1 --2- 172.21.15.92:0/3222436059 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5635c450c410 0x5635c45277c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.360+0000 7ff9c7fff700 10 client.8271 ms_handle_connect on v2:172.21.15.92:6835/4047304116 2023-10-06T10:44:25.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.360+0000 7ff9edfe5700 1 --2- 172.21.15.92:0/3222436059 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5635c450c410 0x5635c45277c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe1e932b1a29a3e0 server_cookie=46e2a28b76b1ae22 in_seq=0 out_seq=0 2023-10-06T10:44:25.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.360+0000 7ff9c7fff700 1 -- 172.21.15.92:0/3222436059 <== mds.1 v2:172.21.15.92:6835/4047304116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5635c43d6e50 con 0x5635c450c410 2023-10-06T10:44:25.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.360+0000 7ff9c7fff700 10 client.8271 handle_command_reply: tid=0 2023-10-06T10:44:25.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.360+0000 7ff9f9541b80 1 client.8271 shutdown 2023-10-06T10:44:25.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.360+0000 7ff9f9541b80 1 -- 172.21.15.92:0/3222436059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c452aae0 msgr2=0x5635c450bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.360+0000 7ff9f9541b80 1 --2- 172.21.15.92:0/3222436059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c452aae0 0x5635c450bed0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0006fd0 tx=0x7ff9e001d070 comp rx=0 tx=0).stop 2023-10-06T10:44:25.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.360+0000 7ff9f9541b80 1 -- 172.21.15.92:0/3222436059 shutdown_connections 2023-10-06T10:44:25.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.360+0000 7ff9f9541b80 1 --2- 172.21.15.92:0/3222436059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635c452aae0 0x5635c450bed0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.360+0000 7ff9f9541b80 1 --2- 172.21.15.92:0/3222436059 >> [v2:172.21.15.92:6835/4047304116,v1:172.21.15.92:6837/4047304116] conn(0x5635c450c410 0x5635c45277c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.360+0000 7ff9f9541b80 1 --2- 172.21.15.92:0/3222436059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635c44e4c20 0x5635c458fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.360+0000 7ff9f9541b80 1 --2- 172.21.15.92:0/3222436059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635c44e69f0 0x5635c44e39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.360+0000 7ff9c7fff700 0 client.8271 ms_handle_reset on v2:172.21.15.92:6835/4047304116 2023-10-06T10:44:25.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.360+0000 7ff9f9541b80 1 -- 172.21.15.92:0/3222436059 >> 172.21.15.92:0/3222436059 conn(0x5635c44e50c0 msgr2=0x5635c450a810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.360+0000 7ff9f9541b80 1 -- 172.21.15.92:0/3222436059 shutdown_connections 2023-10-06T10:44:25.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.360+0000 7ff9f9541b80 1 -- 172.21.15.92:0/3222436059 wait complete. 2023-10-06T10:44:25.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.360+0000 7ff9f9541b80 20 client.8271 trim_cache size 0 max 16384 2023-10-06T10:44:25.369 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:44:25.369 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T10:44:25.369 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:44:25.369 INFO:teuthology.orchestra.run.smithi092.stdout: "9669003b-2b0f-4975-abcd-f07cf375c689": { 2023-10-06T10:44:25.369 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:44:25.369 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "9669003b-2b0f-4975-abcd-f07cf375c689", 2023-10-06T10:44:25.370 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:44:25.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:25.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:25.370 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:44:25.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.361+0000 7ff9f0249700 1 -- 172.21.15.92:0/3901605040 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff9d4066f30 msgr2=0x7ff9d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.361+0000 7ff9f0249700 1 --2- 172.21.15.92:0/3901605040 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff9d4066f30 0x7ff9d40693e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff9e81506c0 tx=0x7ff9dc006f90 comp rx=0 tx=0).stop 2023-10-06T10:44:25.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.361+0000 7ff9f0249700 1 -- 172.21.15.92:0/3901605040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e80c5250 msgr2=0x7ff9e81542d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.361+0000 7ff9f0249700 1 --2- 172.21.15.92:0/3901605040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e80c5250 0x7ff9e81542d0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800a4d0 tx=0x7ff9d80078d0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.361+0000 7ff9f0249700 1 -- 172.21.15.92:0/3901605040 shutdown_connections 2023-10-06T10:44:25.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.361+0000 7ff9f0249700 1 --2- 172.21.15.92:0/3901605040 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff9d4066f30 0x7ff9d40693e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.361+0000 7ff9f0249700 1 --2- 172.21.15.92:0/3901605040 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9e8155b50 0x7ff9e8151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.361+0000 7ff9f0249700 1 --2- 172.21.15.92:0/3901605040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e80c5250 0x7ff9e81542d0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.361+0000 7ff9f0249700 1 --2- 172.21.15.92:0/3901605040 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9e815a160 0x7ff9e813d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.361+0000 7ff9f0249700 1 -- 172.21.15.92:0/3901605040 >> 172.21.15.92:0/3901605040 conn(0x7ff9e8099700 msgr2=0x7ff9e8068c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.361+0000 7ff9f0249700 1 -- 172.21.15.92:0/3901605040 shutdown_connections 2023-10-06T10:44:25.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.361+0000 7ff9f0249700 1 -- 172.21.15.92:0/3901605040 wait complete. 2023-10-06T10:44:25.373 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:44:25.373 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:2 scrub status 2023-10-06T10:44:25.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.488+0000 7f476e3ae700 1 Processor -- start 2023-10-06T10:44:25.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.488+0000 7f476e3ae700 1 -- start start 2023-10-06T10:44:25.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.489+0000 7f476e3ae700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f47680b2620 0x7f47680cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.489+0000 7f476e3ae700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47680cc970 con 0x7f47680cf1b0 2023-10-06T10:44:25.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.489+0000 7f476e3ae700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47680cbd60 con 0x7f47680b29f0 2023-10-06T10:44:25.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.489+0000 7f476e3ae700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47680cc040 con 0x7f47680b2620 2023-10-06T10:44:25.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.489+0000 7f476d3ac700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f47680b2620 0x7f47680cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.489+0000 7f476dbad700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f47680cf1b0 0x7f47680c5350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44160/0 (socket says 172.21.15.92:44160) 2023-10-06T10:44:25.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.489+0000 7f476dbad700 1 -- 172.21.15.92:0/4260189791 learned_addr learned my addr 172.21.15.92:0/4260189791 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:25.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.489+0000 7f476d3ac700 1 -- 172.21.15.92:0/4260189791 >> v1:172.21.15.124:6789/0 conn(0x7f47680b29f0 legacy=0x7f47680ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.489+0000 7f476d3ac700 1 -- 172.21.15.92:0/4260189791 >> v1:172.21.15.92:6789/0 conn(0x7f47680cf1b0 legacy=0x7f47680c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.489+0000 7f476d3ac700 1 -- 172.21.15.92:0/4260189791 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47680cc320 con 0x7f47680b2620 2023-10-06T10:44:25.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.490+0000 7f476d3ac700 1 --2- 172.21.15.92:0/4260189791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47680b2620 0x7f47680cccb0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f4764009eb0 tx=0x7f4764009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7573901e165edf4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.490+0000 7f475ffff700 1 -- 172.21.15.92:0/4260189791 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476400e070 con 0x7f47680b2620 2023-10-06T10:44:25.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.490+0000 7f475ffff700 1 -- 172.21.15.92:0/4260189791 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476400bbf0 con 0x7f47680b2620 2023-10-06T10:44:25.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.490+0000 7f475ffff700 1 -- 172.21.15.92:0/4260189791 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764012430 con 0x7f47680b2620 2023-10-06T10:44:25.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.490+0000 7f476e3ae700 1 -- 172.21.15.92:0/4260189791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47680b2620 msgr2=0x7f47680cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.490+0000 7f476e3ae700 1 --2- 172.21.15.92:0/4260189791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47680b2620 0x7f47680cccb0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f4764009eb0 tx=0x7f4764009780 comp rx=0 tx=0).stop 2023-10-06T10:44:25.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.490+0000 7f476e3ae700 1 -- 172.21.15.92:0/4260189791 shutdown_connections 2023-10-06T10:44:25.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.490+0000 7f476e3ae700 1 --2- 172.21.15.92:0/4260189791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47680b2620 0x7f47680cccb0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.490+0000 7f476e3ae700 1 -- 172.21.15.92:0/4260189791 >> 172.21.15.92:0/4260189791 conn(0x7f4768099520 msgr2=0x7f476806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.490+0000 7f476e3ae700 1 -- 172.21.15.92:0/4260189791 shutdown_connections 2023-10-06T10:44:25.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.490+0000 7f476e3ae700 1 -- 172.21.15.92:0/4260189791 wait complete. 2023-10-06T10:44:25.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.491+0000 7f476e3ae700 1 Processor -- start 2023-10-06T10:44:25.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.491+0000 7f476e3ae700 1 -- start start 2023-10-06T10:44:25.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.492+0000 7f476e3ae700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47680b2620 0x7f4768159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.492+0000 7f476e3ae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47680b29f0 0x7f47680ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.492+0000 7f476e3ae700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47680cf1b0 0x7f47681579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.492+0000 7f476e3ae700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476815ad80 con 0x7f47680b29f0 2023-10-06T10:44:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.492+0000 7f476e3ae700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476815a410 con 0x7f47680cf1b0 2023-10-06T10:44:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.492+0000 7f476e3ae700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476815a6f0 con 0x7f47680b2620 2023-10-06T10:44:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.492+0000 7f476cbab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47680b29f0 0x7f47680ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.492+0000 7f476cbab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47680b29f0 0x7f47680ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37604/0 (socket says 172.21.15.92:37604) 2023-10-06T10:44:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.492+0000 7f476cbab700 1 -- 172.21.15.92:0/1327413284 learned_addr learned my addr 172.21.15.92:0/1327413284 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.493+0000 7f476d3ac700 1 --2- 172.21.15.92:0/1327413284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47680b2620 0x7f4768159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.493+0000 7f476dbad700 1 --2- 172.21.15.92:0/1327413284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47680cf1b0 0x7f47681579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.493+0000 7f476cbab700 1 -- 172.21.15.92:0/1327413284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47680b2620 msgr2=0x7f4768159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.493+0000 7f476cbab700 1 --2- 172.21.15.92:0/1327413284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47680b2620 0x7f4768159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.493+0000 7f476cbab700 1 -- 172.21.15.92:0/1327413284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47680cf1b0 msgr2=0x7f47681579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.493+0000 7f476cbab700 1 --2- 172.21.15.92:0/1327413284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47680cf1b0 0x7f47681579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.493+0000 7f476cbab700 1 -- 172.21.15.92:0/1327413284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4764009a20 con 0x7f47680b29f0 2023-10-06T10:44:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.493+0000 7f476dbad700 1 --2- 172.21.15.92:0/1327413284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47680cf1b0 0x7f47681579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.493+0000 7f476d3ac700 1 --2- 172.21.15.92:0/1327413284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47680b2620 0x7f4768159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.493+0000 7f476cbab700 1 --2- 172.21.15.92:0/1327413284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47680b29f0 0x7f47680ccd50 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f475800c590 tx=0x7f4758007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.493+0000 7f475d7fa700 1 -- 172.21.15.92:0/1327413284 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758011590 con 0x7f47680b29f0 2023-10-06T10:44:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.493+0000 7f475d7fa700 1 -- 172.21.15.92:0/1327413284 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4758011730 con 0x7f47680b29f0 2023-10-06T10:44:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.493+0000 7f475d7fa700 1 -- 172.21.15.92:0/1327413284 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758011af0 con 0x7f47680b29f0 2023-10-06T10:44:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.493+0000 7f476e3ae700 1 -- 172.21.15.92:0/1327413284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f476815a9d0 con 0x7f47680b29f0 2023-10-06T10:44:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.493+0000 7f476e3ae700 1 -- 172.21.15.92:0/1327413284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f476815a100 con 0x7f47680b29f0 2023-10-06T10:44:25.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.494+0000 7f475d7fa700 1 -- 172.21.15.92:0/1327413284 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f475801c460 con 0x7f47680b29f0 2023-10-06T10:44:25.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.495+0000 7f475d7fa700 1 --2- 172.21.15.92:0/1327413284 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f474c066f30 0x7f474c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.495+0000 7f475d7fa700 1 -- 172.21.15.92:0/1327413284 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f4758084580 con 0x7f47680b29f0 2023-10-06T10:44:25.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.495+0000 7f476d3ac700 1 --2- 172.21.15.92:0/1327413284 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f474c066f30 0x7f474c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.495+0000 7f476d3ac700 1 --2- 172.21.15.92:0/1327413284 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f474c066f30 0x7f474c0693e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f47640099c0 tx=0x7f4764016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.497+0000 7f4778908b80 1 Processor -- start 2023-10-06T10:44:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.497+0000 7f4778908b80 1 -- start start 2023-10-06T10:44:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.497+0000 7f4778908b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0d49a0 0x5622cc0d4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.497+0000 7f4778908b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622cc11ec10 0x5622cc0fd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.497+0000 7f4778908b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622cc0fdeb0 0x5622cc0d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.497+0000 7f4778908b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622cc054cd0 con 0x5622cc0d49a0 2023-10-06T10:44:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.497+0000 7f4778908b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622cc02bac0 con 0x5622cc0fdeb0 2023-10-06T10:44:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.497+0000 7f4778908b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622cc0292c0 con 0x5622cc11ec10 2023-10-06T10:44:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.498+0000 7f476d3ac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0d49a0 0x5622cc0d4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.498+0000 7f476d3ac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0d49a0 0x5622cc0d4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37620/0 (socket says 172.21.15.92:37620) 2023-10-06T10:44:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.498+0000 7f476d3ac700 1 -- 172.21.15.92:0/4103761647 learned_addr learned my addr 172.21.15.92:0/4103761647 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.498+0000 7f476dbad700 1 --2- 172.21.15.92:0/4103761647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622cc0fdeb0 0x5622cc0d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.498+0000 7f476cbab700 1 --2- 172.21.15.92:0/4103761647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622cc11ec10 0x5622cc0fd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.498+0000 7f476d3ac700 1 -- 172.21.15.92:0/4103761647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622cc11ec10 msgr2=0x5622cc0fd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.498+0000 7f476d3ac700 1 --2- 172.21.15.92:0/4103761647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622cc11ec10 0x5622cc0fd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.498+0000 7f476d3ac700 1 -- 172.21.15.92:0/4103761647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622cc0fdeb0 msgr2=0x5622cc0d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.498+0000 7f476d3ac700 1 --2- 172.21.15.92:0/4103761647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622cc0fdeb0 0x5622cc0d09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.498+0000 7f476d3ac700 1 -- 172.21.15.92:0/4103761647 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622cbebf2c0 con 0x5622cc0d49a0 2023-10-06T10:44:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.498+0000 7f476d3ac700 1 --2- 172.21.15.92:0/4103761647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0d49a0 0x5622cc0d4d70 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f4764001050 tx=0x7f4764020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e34917181db2fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.498+0000 7f47527fc700 1 -- 172.21.15.92:0/4103761647 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764027460 con 0x5622cc0d49a0 2023-10-06T10:44:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.498+0000 7f47527fc700 1 -- 172.21.15.92:0/4103761647 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47640275c0 con 0x5622cc0d49a0 2023-10-06T10:44:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.498+0000 7f47527fc700 1 -- 172.21.15.92:0/4103761647 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764027980 con 0x5622cc0d49a0 2023-10-06T10:44:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.499+0000 7f4778908b80 1 -- 172.21.15.92:0/4103761647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0d49a0 msgr2=0x5622cc0d4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.499+0000 7f4778908b80 1 --2- 172.21.15.92:0/4103761647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0d49a0 0x5622cc0d4d70 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f4764001050 tx=0x7f4764020040 comp rx=0 tx=0).stop 2023-10-06T10:44:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.499+0000 7f4778908b80 1 -- 172.21.15.92:0/4103761647 shutdown_connections 2023-10-06T10:44:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.499+0000 7f4778908b80 1 --2- 172.21.15.92:0/4103761647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622cc0fdeb0 0x5622cc0d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.499+0000 7f4778908b80 1 --2- 172.21.15.92:0/4103761647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0d49a0 0x5622cc0d4d70 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.499+0000 7f4778908b80 1 --2- 172.21.15.92:0/4103761647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622cc11ec10 0x5622cc0fd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.499+0000 7f4778908b80 1 -- 172.21.15.92:0/4103761647 >> 172.21.15.92:0/4103761647 conn(0x5622cc0d62c0 msgr2=0x5622cc0d6690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.499+0000 7f4778908b80 1 -- 172.21.15.92:0/4103761647 shutdown_connections 2023-10-06T10:44:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.499+0000 7f4778908b80 1 -- 172.21.15.92:0/4103761647 wait complete. 2023-10-06T10:44:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.499+0000 7f4778908b80 1 -- start start 2023-10-06T10:44:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.500+0000 7f4778908b80 1 Processor -- start 2023-10-06T10:44:25.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.500+0000 7f4778908b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0d49a0 0x5622cc116890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.500+0000 7f4778908b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622cc0fdeb0 0x5622cc118c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.500+0000 7f4778908b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622cc11ec10 0x5622cc0cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.500+0000 7f4778908b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622cc0250b0 con 0x5622cc0d49a0 2023-10-06T10:44:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.500+0000 7f4778908b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622cbfd08e0 con 0x5622cc11ec10 2023-10-06T10:44:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.500+0000 7f4778908b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622cbfebef0 con 0x5622cc0fdeb0 2023-10-06T10:44:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.500+0000 7f476d3ac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0d49a0 0x5622cc116890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.501+0000 7f4778908b80 10 client.0 _ll_register_callbacks cb 0x5622cbec1f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.501+0000 7f476d3ac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0d49a0 0x5622cc116890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37626/0 (socket says 172.21.15.92:37626) 2023-10-06T10:44:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.501+0000 7f476d3ac700 1 -- 172.21.15.92:0/341397032 learned_addr learned my addr 172.21.15.92:0/341397032 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.501+0000 7f476dbad700 1 --2- 172.21.15.92:0/341397032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622cc11ec10 0x5622cc0cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.501+0000 7f476cbab700 1 --2- 172.21.15.92:0/341397032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622cc0fdeb0 0x5622cc118c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.501+0000 7f476d3ac700 1 -- 172.21.15.92:0/341397032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622cc0fdeb0 msgr2=0x5622cc118c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.501+0000 7f476d3ac700 1 --2- 172.21.15.92:0/341397032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622cc0fdeb0 0x5622cc118c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.501+0000 7f476d3ac700 1 -- 172.21.15.92:0/341397032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622cc11ec10 msgr2=0x5622cc0cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.501+0000 7f476d3ac700 1 --2- 172.21.15.92:0/341397032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622cc11ec10 0x5622cc0cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.501+0000 7f476d3ac700 1 -- 172.21.15.92:0/341397032 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47680cc320 con 0x5622cc0d49a0 2023-10-06T10:44:25.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.501+0000 7f476cbab700 1 --2- 172.21.15.92:0/341397032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622cc0fdeb0 0x5622cc118c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:25.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.501+0000 7f476d3ac700 1 --2- 172.21.15.92:0/341397032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0d49a0 0x5622cc116890 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f476400d010 tx=0x7f476402b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81d38748c59af67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.501+0000 7f47477fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:25.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.501+0000 7f47477fe700 1 -- 172.21.15.92:0/341397032 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764032440 con 0x5622cc0d49a0 2023-10-06T10:44:25.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.501+0000 7f47477fe700 1 -- 172.21.15.92:0/341397032 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f476403bd70 con 0x5622cc0d49a0 2023-10-06T10:44:25.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.501+0000 7f4778908b80 1 -- 172.21.15.92:0/341397032 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622cc0c96e0 con 0x5622cc0d49a0 2023-10-06T10:44:25.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.501+0000 7f47477fe700 1 -- 172.21.15.92:0/341397032 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47640327b0 con 0x5622cc0d49a0 2023-10-06T10:44:25.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.501+0000 7f4778908b80 10 client.8286 fetch_fsmap learned FSMap version 92 2023-10-06T10:44:25.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.501+0000 7f4778908b80 1 -- 172.21.15.92:0/341397032 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) v3 -- 0x5622cbe62030 con 0x5622cc0d49a0 2023-10-06T10:44:25.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.501+0000 7f47477fe700 1 -- 172.21.15.92:0/341397032 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5622cc0c96e0 con 0x5622cc0d49a0 2023-10-06T10:44:25.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.502+0000 7f47477fe700 1 -- 172.21.15.92:0/341397032 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 92) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f4764041020 con 0x5622cc0d49a0 2023-10-06T10:44:25.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.502+0000 7f4778908b80 10 client.8286 fetch_fsmap finished waiting for FSMap version 92 2023-10-06T10:44:25.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.502+0000 7f4778908b80 10 client.8286 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.a 2023-10-06T10:44:25.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.502+0000 7f4778908b80 20 client.8286 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:25.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.502+0000 7f4778908b80 1 --2- 172.21.15.92:0/341397032 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x5622cc0ed6b0 0x5622cc0edab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.502+0000 7f4778908b80 4 client.8286 mds_command: new command op to 7686 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:44:25.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.502+0000 7f4778908b80 1 -- 172.21.15.92:0/341397032 --> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622cc0a6610 con 0x5622cc0ed6b0 2023-10-06T10:44:25.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.502+0000 7f476cbab700 1 --2- 172.21.15.92:0/341397032 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x5622cc0ed6b0 0x5622cc0edab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.502+0000 7f47477fe700 10 client.8286 ms_handle_connect on v2:172.21.15.92:6834/3605297930 2023-10-06T10:44:25.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.502+0000 7f476cbab700 1 --2- 172.21.15.92:0/341397032 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x5622cc0ed6b0 0x5622cc0edab0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=693922cfa755f2a8 server_cookie=233c99c1b2331de0 in_seq=0 out_seq=0 2023-10-06T10:44:25.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.502+0000 7f47477fe700 1 -- 172.21.15.92:0/341397032 <== mds.2 v2:172.21.15.92:6834/3605297930 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5622cc0a6610 con 0x5622cc0ed6b0 2023-10-06T10:44:25.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.502+0000 7f47477fe700 10 client.8286 handle_command_reply: tid=0 2023-10-06T10:44:25.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.502+0000 7f4778908b80 1 client.8286 shutdown 2023-10-06T10:44:25.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.503+0000 7f4778908b80 1 -- 172.21.15.92:0/341397032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0d49a0 msgr2=0x5622cc116890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.503+0000 7f4778908b80 1 --2- 172.21.15.92:0/341397032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0d49a0 0x5622cc116890 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f476400d010 tx=0x7f476402b040 comp rx=0 tx=0).stop 2023-10-06T10:44:25.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.503+0000 7f4778908b80 1 -- 172.21.15.92:0/341397032 shutdown_connections 2023-10-06T10:44:25.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.503+0000 7f4778908b80 1 --2- 172.21.15.92:0/341397032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622cc11ec10 0x5622cc0cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.503+0000 7f4778908b80 1 --2- 172.21.15.92:0/341397032 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x5622cc0ed6b0 0x5622cc0edab0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.503+0000 7f4778908b80 1 --2- 172.21.15.92:0/341397032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0d49a0 0x5622cc116890 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.503+0000 7f4778908b80 1 --2- 172.21.15.92:0/341397032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622cc0fdeb0 0x5622cc118c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.503+0000 7f47477fe700 0 client.8286 ms_handle_reset on v2:172.21.15.92:6834/3605297930 2023-10-06T10:44:25.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.503+0000 7f4778908b80 1 -- 172.21.15.92:0/341397032 >> 172.21.15.92:0/341397032 conn(0x5622cc0d62c0 msgr2=0x5622cc0d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.503+0000 7f4778908b80 1 -- 172.21.15.92:0/341397032 shutdown_connections 2023-10-06T10:44:25.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.503+0000 7f4778908b80 1 -- 172.21.15.92:0/341397032 wait complete. 2023-10-06T10:44:25.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.503+0000 7f4778908b80 20 client.8286 trim_cache size 0 max 16384 2023-10-06T10:44:25.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.519+0000 7f4778908b80 1 Processor -- start 2023-10-06T10:44:25.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.519+0000 7f4778908b80 1 -- start start 2023-10-06T10:44:25.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.519+0000 7f4778908b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0d49a0 0x5622cc118c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.519+0000 7f4778908b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622cc0fdeb0 0x5622cc12a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.519+0000 7f4778908b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622cc11ec10 0x5622cc1165a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.519+0000 7f4778908b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622cbea7e80 con 0x5622cc0d49a0 2023-10-06T10:44:25.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.519+0000 7f4778908b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622cc0d5730 con 0x5622cc11ec10 2023-10-06T10:44:25.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.519+0000 7f4778908b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622cc13e7e0 con 0x5622cc0fdeb0 2023-10-06T10:44:25.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.519+0000 7f476d3ac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0d49a0 0x5622cc118c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.519+0000 7f476d3ac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0d49a0 0x5622cc118c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37642/0 (socket says 172.21.15.92:37642) 2023-10-06T10:44:25.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.519+0000 7f476d3ac700 1 -- 172.21.15.92:0/3923787937 learned_addr learned my addr 172.21.15.92:0/3923787937 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:25.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.520+0000 7f476d3ac700 1 -- 172.21.15.92:0/3923787937 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622cc0fdeb0 msgr2=0x5622cc12a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.520+0000 7f476d3ac700 1 --2- 172.21.15.92:0/3923787937 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622cc0fdeb0 0x5622cc12a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.520+0000 7f476d3ac700 1 -- 172.21.15.92:0/3923787937 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622cc11ec10 msgr2=0x5622cc1165a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.520+0000 7f476d3ac700 1 --2- 172.21.15.92:0/3923787937 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622cc11ec10 0x5622cc1165a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.520+0000 7f476d3ac700 1 -- 172.21.15.92:0/3923787937 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622cc118660 con 0x5622cc0d49a0 2023-10-06T10:44:25.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.520+0000 7f476d3ac700 1 --2- 172.21.15.92:0/3923787937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0d49a0 0x5622cc118c50 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f476401ee00 tx=0x7f4764020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72af4b615a406c91 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.520+0000 7f4752ffd700 1 -- 172.21.15.92:0/3923787937 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764032960 con 0x5622cc0d49a0 2023-10-06T10:44:25.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.520+0000 7f4752ffd700 1 -- 172.21.15.92:0/3923787937 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4764032ac0 con 0x5622cc0d49a0 2023-10-06T10:44:25.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.520+0000 7f4752ffd700 1 -- 172.21.15.92:0/3923787937 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764032e40 con 0x5622cc0d49a0 2023-10-06T10:44:25.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.520+0000 7f4778908b80 1 -- 172.21.15.92:0/3923787937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0d49a0 msgr2=0x5622cc118c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.520+0000 7f4778908b80 1 --2- 172.21.15.92:0/3923787937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0d49a0 0x5622cc118c50 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f476401ee00 tx=0x7f4764020070 comp rx=0 tx=0).stop 2023-10-06T10:44:25.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.520+0000 7f4778908b80 1 -- 172.21.15.92:0/3923787937 shutdown_connections 2023-10-06T10:44:25.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.520+0000 7f4778908b80 1 --2- 172.21.15.92:0/3923787937 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622cc11ec10 0x5622cc1165a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.520+0000 7f4778908b80 1 --2- 172.21.15.92:0/3923787937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0d49a0 0x5622cc118c50 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.520+0000 7f4778908b80 1 --2- 172.21.15.92:0/3923787937 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622cc0fdeb0 0x5622cc12a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.520+0000 7f4778908b80 1 -- 172.21.15.92:0/3923787937 >> 172.21.15.92:0/3923787937 conn(0x5622cc0d62c0 msgr2=0x5622cc0d6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.521+0000 7f4778908b80 1 -- 172.21.15.92:0/3923787937 shutdown_connections 2023-10-06T10:44:25.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.521+0000 7f4778908b80 1 -- 172.21.15.92:0/3923787937 wait complete. 2023-10-06T10:44:25.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.521+0000 7f4778908b80 1 -- start start 2023-10-06T10:44:25.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.521+0000 7f4778908b80 1 Processor -- start 2023-10-06T10:44:25.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.522+0000 7f4778908b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622cc0d49a0 0x5622cc135210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.522+0000 7f4778908b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0fdeb0 0x5622cc111b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.522+0000 7f4778908b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622cc11ec10 0x5622cc112070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.522+0000 7f4778908b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622cc181160 con 0x5622cc0fdeb0 2023-10-06T10:44:25.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.522+0000 7f4778908b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622cc1815c0 con 0x5622cc11ec10 2023-10-06T10:44:25.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.522+0000 7f4778908b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622cc135750 con 0x5622cc0d49a0 2023-10-06T10:44:25.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.522+0000 7f4778908b80 10 client.0 _ll_register_callbacks cb 0x5622cbec1f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:25.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.522+0000 7f476cbab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0fdeb0 0x5622cc111b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.522+0000 7f476cbab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0fdeb0 0x5622cc111b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37646/0 (socket says 172.21.15.92:37646) 2023-10-06T10:44:25.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.522+0000 7f476cbab700 1 -- 172.21.15.92:0/59678376 learned_addr learned my addr 172.21.15.92:0/59678376 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:25.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.522+0000 7f476dbad700 1 --2- 172.21.15.92:0/59678376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622cc11ec10 0x5622cc112070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.522+0000 7f476d3ac700 1 --2- 172.21.15.92:0/59678376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622cc0d49a0 0x5622cc135210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.522+0000 7f476cbab700 1 -- 172.21.15.92:0/59678376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622cc0d49a0 msgr2=0x5622cc135210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.522+0000 7f476cbab700 1 --2- 172.21.15.92:0/59678376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622cc0d49a0 0x5622cc135210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.522+0000 7f476cbab700 1 -- 172.21.15.92:0/59678376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622cc11ec10 msgr2=0x5622cc112070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.522+0000 7f476cbab700 1 --2- 172.21.15.92:0/59678376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622cc11ec10 0x5622cc112070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.522+0000 7f476cbab700 1 -- 172.21.15.92:0/59678376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622cbe62030 con 0x5622cc0fdeb0 2023-10-06T10:44:25.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.522+0000 7f476dbad700 1 --2- 172.21.15.92:0/59678376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622cc11ec10 0x5622cc112070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:25.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.522+0000 7f476cbab700 1 --2- 172.21.15.92:0/59678376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0fdeb0 0x5622cc111b30 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f4758051710 tx=0x7f4758014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc402a2dc025902e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.522+0000 7f47477fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:25.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.523+0000 7f47477fe700 1 -- 172.21.15.92:0/59678376 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475806f070 con 0x5622cc0fdeb0 2023-10-06T10:44:25.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.523+0000 7f47477fe700 1 -- 172.21.15.92:0/59678376 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f47580535b0 con 0x5622cc0fdeb0 2023-10-06T10:44:25.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.523+0000 7f4778908b80 1 -- 172.21.15.92:0/59678376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47640275c0 con 0x5622cc0fdeb0 2023-10-06T10:44:25.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.523+0000 7f47477fe700 1 -- 172.21.15.92:0/59678376 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f475805c230 con 0x5622cc0fdeb0 2023-10-06T10:44:25.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.523+0000 7f4778908b80 10 client.8292 fetch_fsmap learned FSMap version 92 2023-10-06T10:44:25.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.523+0000 7f4778908b80 1 -- 172.21.15.92:0/59678376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) v3 -- 0x5622cbff37d0 con 0x5622cc0fdeb0 2023-10-06T10:44:25.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.523+0000 7f47477fe700 1 -- 172.21.15.92:0/59678376 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47640275c0 con 0x5622cc0fdeb0 2023-10-06T10:44:25.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.523+0000 7f47477fe700 1 -- 172.21.15.92:0/59678376 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 92) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f4758054700 con 0x5622cc0fdeb0 2023-10-06T10:44:25.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.523+0000 7f4778908b80 10 client.8292 fetch_fsmap finished waiting for FSMap version 92 2023-10-06T10:44:25.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.523+0000 7f4778908b80 10 client.8292 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.a 2023-10-06T10:44:25.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.523+0000 7f4778908b80 20 client.8292 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:25.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.523+0000 7f4778908b80 1 --2- 172.21.15.92:0/59678376 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x5622cc1125b0 0x5622cc17f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.523+0000 7f4778908b80 4 client.8292 mds_command: new command op to 7686 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:44:25.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.523+0000 7f4778908b80 1 -- 172.21.15.92:0/59678376 --> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5622cbfc7e50 con 0x5622cc1125b0 2023-10-06T10:44:25.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.523+0000 7f476d3ac700 1 --2- 172.21.15.92:0/59678376 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x5622cc1125b0 0x5622cc17f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.523+0000 7f47477fe700 10 client.8292 ms_handle_connect on v2:172.21.15.92:6834/3605297930 2023-10-06T10:44:25.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.523+0000 7f476d3ac700 1 --2- 172.21.15.92:0/59678376 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x5622cc1125b0 0x5622cc17f3e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=386221ca03a8df86 server_cookie=8a568765444e2ba3 in_seq=0 out_seq=0 2023-10-06T10:44:25.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.524+0000 7f47477fe700 1 -- 172.21.15.92:0/59678376 <== mds.2 v2:172.21.15.92:6834/3605297930 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5622cbfc7e50 con 0x5622cc1125b0 2023-10-06T10:44:25.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.524+0000 7f47477fe700 10 client.8292 handle_command_reply: tid=0 2023-10-06T10:44:25.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.524+0000 7f4778908b80 1 client.8292 shutdown 2023-10-06T10:44:25.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.524+0000 7f4778908b80 1 -- 172.21.15.92:0/59678376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0fdeb0 msgr2=0x5622cc111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.524+0000 7f4778908b80 1 --2- 172.21.15.92:0/59678376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0fdeb0 0x5622cc111b30 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f4758051710 tx=0x7f4758014040 comp rx=0 tx=0).stop 2023-10-06T10:44:25.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.524+0000 7f4778908b80 1 -- 172.21.15.92:0/59678376 shutdown_connections 2023-10-06T10:44:25.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.524+0000 7f4778908b80 1 --2- 172.21.15.92:0/59678376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622cc11ec10 0x5622cc112070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.524+0000 7f4778908b80 1 --2- 172.21.15.92:0/59678376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622cc0d49a0 0x5622cc135210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.524+0000 7f4778908b80 1 --2- 172.21.15.92:0/59678376 >> [v2:172.21.15.92:6834/3605297930,v1:172.21.15.92:6836/3605297930] conn(0x5622cc1125b0 0x5622cc17f3e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.524+0000 7f4778908b80 1 --2- 172.21.15.92:0/59678376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cc0fdeb0 0x5622cc111b30 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.524+0000 7f47477fe700 0 client.8292 ms_handle_reset on v2:172.21.15.92:6834/3605297930 2023-10-06T10:44:25.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.524+0000 7f4778908b80 1 -- 172.21.15.92:0/59678376 >> 172.21.15.92:0/59678376 conn(0x5622cc0d62c0 msgr2=0x5622cc118bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.524+0000 7f4778908b80 1 -- 172.21.15.92:0/59678376 shutdown_connections 2023-10-06T10:44:25.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.524+0000 7f4778908b80 1 -- 172.21.15.92:0/59678376 wait complete. 2023-10-06T10:44:25.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.524+0000 7f4778908b80 20 client.8292 trim_cache size 0 max 16384 2023-10-06T10:44:25.532 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:44:25.533 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T10:44:25.533 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:44:25.533 INFO:teuthology.orchestra.run.smithi092.stdout: "9669003b-2b0f-4975-abcd-f07cf375c689": { 2023-10-06T10:44:25.533 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:44:25.533 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "9669003b-2b0f-4975-abcd-f07cf375c689", 2023-10-06T10:44:25.533 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:44:25.533 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:25.533 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:25.533 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:44:25.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.524+0000 7f476e3ae700 1 -- 172.21.15.92:0/1327413284 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f474c066f30 msgr2=0x7f474c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.524+0000 7f476e3ae700 1 --2- 172.21.15.92:0/1327413284 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f474c066f30 0x7f474c0693e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f47640099c0 tx=0x7f4764016000 comp rx=0 tx=0).stop 2023-10-06T10:44:25.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.524+0000 7f476e3ae700 1 -- 172.21.15.92:0/1327413284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47680b29f0 msgr2=0x7f47680ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.524+0000 7f476e3ae700 1 --2- 172.21.15.92:0/1327413284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47680b29f0 0x7f47680ccd50 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f475800c590 tx=0x7f4758007f30 comp rx=0 tx=0).stop 2023-10-06T10:44:25.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.525+0000 7f476e3ae700 1 -- 172.21.15.92:0/1327413284 shutdown_connections 2023-10-06T10:44:25.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.525+0000 7f476e3ae700 1 --2- 172.21.15.92:0/1327413284 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f474c066f30 0x7f474c0693e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.525+0000 7f476e3ae700 1 --2- 172.21.15.92:0/1327413284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47680cf1b0 0x7f47681579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.525+0000 7f476e3ae700 1 --2- 172.21.15.92:0/1327413284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47680b2620 0x7f4768159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.525+0000 7f476e3ae700 1 --2- 172.21.15.92:0/1327413284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47680b29f0 0x7f47680ccd50 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.525+0000 7f476e3ae700 1 -- 172.21.15.92:0/1327413284 >> 172.21.15.92:0/1327413284 conn(0x7f4768099520 msgr2=0x7f4768158c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.525+0000 7f476e3ae700 1 -- 172.21.15.92:0/1327413284 shutdown_connections 2023-10-06T10:44:25.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.525+0000 7f476e3ae700 1 -- 172.21.15.92:0/1327413284 wait complete. 2023-10-06T10:44:25.537 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:44:25.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub status 2023-10-06T10:44:25.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.654+0000 7fbff8620700 1 Processor -- start 2023-10-06T10:44:25.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.654+0000 7fbff8620700 1 -- start start 2023-10-06T10:44:25.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.655+0000 7fbff8620700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbff0067800 0x7fbff01584d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.655+0000 7fbff8620700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbff0067bd0 0x7fbff015af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.655+0000 7fbff8620700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff0157a60 con 0x7fbff015b490 2023-10-06T10:44:25.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.655+0000 7fbff8620700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff00c9930 con 0x7fbff0067800 2023-10-06T10:44:25.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.655+0000 7fbff8620700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff015ba80 con 0x7fbff0067bd0 2023-10-06T10:44:25.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.655+0000 7fbff63bc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbff0067800 0x7fbff01584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.655+0000 7fbff6bbd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbff015b490 0x7fbff01405f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44174/0 (socket says 172.21.15.92:44174) 2023-10-06T10:44:25.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.655+0000 7fbff6bbd700 1 -- 172.21.15.92:0/2618539024 learned_addr learned my addr 172.21.15.92:0/2618539024 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:25.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.655+0000 7fbff63bc700 1 -- 172.21.15.92:0/2618539024 >> v2:172.21.15.124:3300/0 conn(0x7fbff0067bd0 msgr2=0x7fbff015af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.655+0000 7fbff63bc700 1 --2- 172.21.15.92:0/2618539024 >> v2:172.21.15.124:3300/0 conn(0x7fbff0067bd0 0x7fbff015af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.655+0000 7fbff63bc700 1 -- 172.21.15.92:0/2618539024 >> v1:172.21.15.92:6789/0 conn(0x7fbff015b490 legacy=0x7fbff01405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.655+0000 7fbff63bc700 1 -- 172.21.15.92:0/2618539024 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff0157cd0 con 0x7fbff0067800 2023-10-06T10:44:25.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.655+0000 7fbff53ba700 1 -- 172.21.15.92:0/2618539024 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223923826 0 0) 0x7fbff0157a60 con 0x7fbff015b490 2023-10-06T10:44:25.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.656+0000 7fbff63bc700 1 --2- 172.21.15.92:0/2618539024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff0067800 0x7fbff01584d0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fbff006d290 tx=0x7fbfec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e975ca70c2c4904 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.656+0000 7fbff53ba700 1 -- 172.21.15.92:0/2618539024 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec00e070 con 0x7fbff0067800 2023-10-06T10:44:25.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.656+0000 7fbff53ba700 1 -- 172.21.15.92:0/2618539024 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec00bc30 con 0x7fbff0067800 2023-10-06T10:44:25.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.656+0000 7fbff53ba700 1 -- 172.21.15.92:0/2618539024 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec012430 con 0x7fbff0067800 2023-10-06T10:44:25.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.656+0000 7fbff8620700 1 -- 172.21.15.92:0/2618539024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff0067800 msgr2=0x7fbff01584d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.656+0000 7fbff8620700 1 --2- 172.21.15.92:0/2618539024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff0067800 0x7fbff01584d0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fbff006d290 tx=0x7fbfec009780 comp rx=0 tx=0).stop 2023-10-06T10:44:25.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.656+0000 7fbff8620700 1 -- 172.21.15.92:0/2618539024 shutdown_connections 2023-10-06T10:44:25.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.656+0000 7fbff8620700 1 --2- 172.21.15.92:0/2618539024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff0067800 0x7fbff01584d0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.656+0000 7fbff8620700 1 --2- 172.21.15.92:0/2618539024 >> v2:172.21.15.124:3300/0 conn(0x7fbff0067bd0 0x7fbff015af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.656+0000 7fbff8620700 1 -- 172.21.15.92:0/2618539024 >> 172.21.15.92:0/2618539024 conn(0x7fbff00cd1a0 msgr2=0x7fbff00cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.656+0000 7fbff8620700 1 -- 172.21.15.92:0/2618539024 shutdown_connections 2023-10-06T10:44:25.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.656+0000 7fbff8620700 1 -- 172.21.15.92:0/2618539024 wait complete. 2023-10-06T10:44:25.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.657+0000 7fbff8620700 1 Processor -- start 2023-10-06T10:44:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.657+0000 7fbff8620700 1 -- start start 2023-10-06T10:44:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.658+0000 7fbff8620700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff0067800 0x7fbff00c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.658+0000 7fbff8620700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbff0067bd0 0x7fbff00a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.658+0000 7fbff8620700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbff015b490 0x7fbff0156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.658+0000 7fbff8620700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff00a6ae0 con 0x7fbff0067800 2023-10-06T10:44:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.658+0000 7fbff8620700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff00a6800 con 0x7fbff0067bd0 2023-10-06T10:44:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.658+0000 7fbff8620700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff00c4380 con 0x7fbff015b490 2023-10-06T10:44:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.658+0000 7fbff63bc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff0067800 0x7fbff00c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.658+0000 7fbff63bc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff0067800 0x7fbff00c4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37666/0 (socket says 172.21.15.92:37666) 2023-10-06T10:44:25.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.658+0000 7fbff63bc700 1 -- 172.21.15.92:0/3383807195 learned_addr learned my addr 172.21.15.92:0/3383807195 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:25.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.658+0000 7fbff6bbd700 1 --2- 172.21.15.92:0/3383807195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbff015b490 0x7fbff0156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.658+0000 7fbff5bbb700 1 --2- 172.21.15.92:0/3383807195 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbff0067bd0 0x7fbff00a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.659+0000 7fbff63bc700 1 -- 172.21.15.92:0/3383807195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbff015b490 msgr2=0x7fbff0156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.659+0000 7fbff63bc700 1 --2- 172.21.15.92:0/3383807195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbff015b490 0x7fbff0156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.659+0000 7fbff63bc700 1 -- 172.21.15.92:0/3383807195 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbff0067bd0 msgr2=0x7fbff00a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.659+0000 7fbff63bc700 1 --2- 172.21.15.92:0/3383807195 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbff0067bd0 0x7fbff00a5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.659+0000 7fbff63bc700 1 -- 172.21.15.92:0/3383807195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfec009a20 con 0x7fbff0067800 2023-10-06T10:44:25.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.659+0000 7fbff63bc700 1 --2- 172.21.15.92:0/3383807195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff0067800 0x7fbff00c4e60 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fbfec013000 tx=0x7fbfec019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.659+0000 7fbfe67fc700 1 -- 172.21.15.92:0/3383807195 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec027470 con 0x7fbff0067800 2023-10-06T10:44:25.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.659+0000 7fbfe67fc700 1 -- 172.21.15.92:0/3383807195 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec027610 con 0x7fbff0067800 2023-10-06T10:44:25.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.659+0000 7fbff8620700 1 -- 172.21.15.92:0/3383807195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff00c4660 con 0x7fbff0067800 2023-10-06T10:44:25.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.659+0000 7fbfe67fc700 1 -- 172.21.15.92:0/3383807195 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0279d0 con 0x7fbff0067800 2023-10-06T10:44:25.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.659+0000 7fbff8620700 1 -- 172.21.15.92:0/3383807195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff00a6440 con 0x7fbff0067800 2023-10-06T10:44:25.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.660+0000 7fbfe67fc700 1 -- 172.21.15.92:0/3383807195 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbfec024070 con 0x7fbff0067800 2023-10-06T10:44:25.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.661+0000 7fbfe67fc700 1 --2- 172.21.15.92:0/3383807195 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfd4066f30 0x7fbfd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.661+0000 7fbfe67fc700 1 -- 172.21.15.92:0/3383807195 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fbfec086f20 con 0x7fbff0067800 2023-10-06T10:44:25.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.661+0000 7fbff5bbb700 1 --2- 172.21.15.92:0/3383807195 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfd4066f30 0x7fbfd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.661+0000 7fbff5bbb700 1 --2- 172.21.15.92:0/3383807195 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfd4066f30 0x7fbfd40693e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0006fd0 tx=0x7fbfe0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.663+0000 7fc001918b80 1 Processor -- start 2023-10-06T10:44:25.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.663+0000 7fc001918b80 1 -- start start 2023-10-06T10:44:25.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.663+0000 7fc001918b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee62cae0 0x5599ee62ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.663+0000 7fc001918b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599ee5e89f0 0x5599ee5e55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.663+0000 7fc001918b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599ee5e6c20 0x5599ee5e5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.663+0000 7fc001918b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ee565cd0 con 0x5599ee62cae0 2023-10-06T10:44:25.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.663+0000 7fc001918b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ee53cac0 con 0x5599ee5e6c20 2023-10-06T10:44:25.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.663+0000 7fc001918b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599ee53a2c0 con 0x5599ee5e89f0 2023-10-06T10:44:25.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.664+0000 7fbff63bc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee62cae0 0x5599ee62ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.664+0000 7fbff63bc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee62cae0 0x5599ee62ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37682/0 (socket says 172.21.15.92:37682) 2023-10-06T10:44:25.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.664+0000 7fbff63bc700 1 -- 172.21.15.92:0/561875084 learned_addr learned my addr 172.21.15.92:0/561875084 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:25.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.664+0000 7fbff5bbb700 1 --2- 172.21.15.92:0/561875084 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599ee5e89f0 0x5599ee5e55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.664+0000 7fbff6bbd700 1 --2- 172.21.15.92:0/561875084 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599ee5e6c20 0x5599ee5e5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.664+0000 7fbff63bc700 1 -- 172.21.15.92:0/561875084 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599ee5e89f0 msgr2=0x5599ee5e55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.664+0000 7fbff63bc700 1 --2- 172.21.15.92:0/561875084 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599ee5e89f0 0x5599ee5e55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.664+0000 7fbff63bc700 1 -- 172.21.15.92:0/561875084 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599ee5e6c20 msgr2=0x5599ee5e5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.664+0000 7fbff63bc700 1 --2- 172.21.15.92:0/561875084 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599ee5e6c20 0x5599ee5e5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.664+0000 7fbff63bc700 1 -- 172.21.15.92:0/561875084 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599ee3d02c0 con 0x5599ee62cae0 2023-10-06T10:44:25.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.664+0000 7fbff63bc700 1 --2- 172.21.15.92:0/561875084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee62cae0 0x5599ee62ceb0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0851d0 tx=0x7fbfec020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12fcaad7c8763f91 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.664+0000 7fbfd37fe700 1 -- 172.21.15.92:0/561875084 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec00e040 con 0x5599ee62cae0 2023-10-06T10:44:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.664+0000 7fbfd37fe700 1 -- 172.21.15.92:0/561875084 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec0582b0 con 0x5599ee62cae0 2023-10-06T10:44:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.664+0000 7fbfd37fe700 1 -- 172.21.15.92:0/561875084 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec057bb0 con 0x5599ee62cae0 2023-10-06T10:44:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.665+0000 7fc001918b80 1 -- 172.21.15.92:0/561875084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee62cae0 msgr2=0x5599ee62ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.665+0000 7fc001918b80 1 --2- 172.21.15.92:0/561875084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee62cae0 0x5599ee62ceb0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0851d0 tx=0x7fbfec020040 comp rx=0 tx=0).stop 2023-10-06T10:44:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.665+0000 7fc001918b80 1 -- 172.21.15.92:0/561875084 shutdown_connections 2023-10-06T10:44:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.665+0000 7fc001918b80 1 --2- 172.21.15.92:0/561875084 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599ee5e6c20 0x5599ee5e5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.665+0000 7fc001918b80 1 --2- 172.21.15.92:0/561875084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee62cae0 0x5599ee62ceb0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.665+0000 7fc001918b80 1 --2- 172.21.15.92:0/561875084 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599ee5e89f0 0x5599ee5e55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.665+0000 7fc001918b80 1 -- 172.21.15.92:0/561875084 >> 172.21.15.92:0/561875084 conn(0x5599ee5e70c0 msgr2=0x5599ee5e7490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.665+0000 7fc001918b80 1 -- 172.21.15.92:0/561875084 shutdown_connections 2023-10-06T10:44:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.665+0000 7fc001918b80 1 -- 172.21.15.92:0/561875084 wait complete. 2023-10-06T10:44:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.665+0000 7fc001918b80 1 -- start start 2023-10-06T10:44:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.666+0000 7fc001918b80 1 Processor -- start 2023-10-06T10:44:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.666+0000 7fc001918b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee5e6c20 0x5599ee60c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.666+0000 7fc001918b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599ee5e89f0 0x5599ee6298a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.666+0000 7fc001918b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599ee62cae0 0x5599ee60da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.666+0000 7fc001918b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ee5360b0 con 0x5599ee5e6c20 2023-10-06T10:44:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.666+0000 7fc001918b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ee4e18e0 con 0x5599ee5e89f0 2023-10-06T10:44:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.666+0000 7fc001918b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599ee4fcef0 con 0x5599ee62cae0 2023-10-06T10:44:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fc001918b80 10 client.0 _ll_register_callbacks cb 0x5599ee3d2f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fbff63bc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee5e6c20 0x5599ee60c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fbff63bc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee5e6c20 0x5599ee60c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37686/0 (socket says 172.21.15.92:37686) 2023-10-06T10:44:25.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fbff63bc700 1 -- 172.21.15.92:0/217228176 learned_addr learned my addr 172.21.15.92:0/217228176 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:25.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fbff5bbb700 1 --2- 172.21.15.92:0/217228176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599ee5e89f0 0x5599ee6298a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fbff6bbd700 1 --2- 172.21.15.92:0/217228176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599ee62cae0 0x5599ee60da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fbff63bc700 1 -- 172.21.15.92:0/217228176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599ee62cae0 msgr2=0x5599ee60da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fbff63bc700 1 --2- 172.21.15.92:0/217228176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599ee62cae0 0x5599ee60da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fbff63bc700 1 -- 172.21.15.92:0/217228176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599ee5e89f0 msgr2=0x5599ee6298a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fbff63bc700 1 --2- 172.21.15.92:0/217228176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599ee5e89f0 0x5599ee6298a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fbff63bc700 1 -- 172.21.15.92:0/217228176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff00a6440 con 0x5599ee5e6c20 2023-10-06T10:44:25.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fbff6bbd700 1 --2- 172.21.15.92:0/217228176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599ee62cae0 0x5599ee60da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:25.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fbff63bc700 1 --2- 172.21.15.92:0/217228176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee5e6c20 0x5599ee60c270 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fbfec085280 tx=0x7fbfec06d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54415b9accc628d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fbfd0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fbfd0ff9700 1 -- 172.21.15.92:0/217228176 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec057bb0 con 0x5599ee5e6c20 2023-10-06T10:44:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fbfd0ff9700 1 -- 172.21.15.92:0/217228176 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fbfec05e6c0 con 0x5599ee5e6c20 2023-10-06T10:44:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fc001918b80 1 -- 172.21.15.92:0/217228176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599ee5da6e0 con 0x5599ee5e6c20 2023-10-06T10:44:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fbfd0ff9700 1 -- 172.21.15.92:0/217228176 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfec058bc0 con 0x5599ee5e6c20 2023-10-06T10:44:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fc001918b80 10 client.8307 fetch_fsmap learned FSMap version 92 2023-10-06T10:44:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fc001918b80 1 -- 172.21.15.92:0/217228176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) v3 -- 0x5599ee373030 con 0x5599ee5e6c20 2023-10-06T10:44:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fbfd0ff9700 1 -- 172.21.15.92:0/217228176 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5599ee5da6e0 con 0x5599ee5e6c20 2023-10-06T10:44:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fbfd0ff9700 1 -- 172.21.15.92:0/217228176 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 92) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fbfec058d20 con 0x5599ee5e6c20 2023-10-06T10:44:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fc001918b80 10 client.8307 fetch_fsmap finished waiting for FSMap version 92 2023-10-06T10:44:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.667+0000 7fc001918b80 10 client.8307 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2023-10-06T10:44:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.668+0000 7fc001918b80 20 client.8307 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.668+0000 7fc001918b80 1 --2- 172.21.15.92:0/217228176 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x5599ee63f020 0x5599ee63f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.668+0000 7fc001918b80 4 client.8307 mds_command: new command op to 7701 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:44:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.668+0000 7fc001918b80 1 -- 172.21.15.92:0/217228176 --> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599ee5b7610 con 0x5599ee63f020 2023-10-06T10:44:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.668+0000 7fbff5bbb700 1 --2- 172.21.15.92:0/217228176 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x5599ee63f020 0x5599ee63f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.668+0000 7fbfd0ff9700 10 client.8307 ms_handle_connect on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.668+0000 7fbff5bbb700 1 --2- 172.21.15.92:0/217228176 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x5599ee63f020 0x5599ee63f3f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c191186833719d1a server_cookie=db60381d268ac28b in_seq=0 out_seq=0 2023-10-06T10:44:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.669+0000 7fbfd0ff9700 1 -- 172.21.15.92:0/217228176 <== mds.0 v2:172.21.15.124:6832/4203118992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5599ee5b7610 con 0x5599ee63f020 2023-10-06T10:44:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.669+0000 7fbfd0ff9700 10 client.8307 handle_command_reply: tid=0 2023-10-06T10:44:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.669+0000 7fc001918b80 1 client.8307 shutdown 2023-10-06T10:44:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.669+0000 7fc001918b80 1 -- 172.21.15.92:0/217228176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee5e6c20 msgr2=0x5599ee60c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.669+0000 7fc001918b80 1 --2- 172.21.15.92:0/217228176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee5e6c20 0x5599ee60c270 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fbfec085280 tx=0x7fbfec06d000 comp rx=0 tx=0).stop 2023-10-06T10:44:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.669+0000 7fc001918b80 1 -- 172.21.15.92:0/217228176 shutdown_connections 2023-10-06T10:44:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.669+0000 7fc001918b80 1 --2- 172.21.15.92:0/217228176 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x5599ee63f020 0x5599ee63f3f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.669+0000 7fc001918b80 1 --2- 172.21.15.92:0/217228176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599ee62cae0 0x5599ee60da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.669+0000 7fc001918b80 1 --2- 172.21.15.92:0/217228176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee5e6c20 0x5599ee60c270 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.669+0000 7fc001918b80 1 --2- 172.21.15.92:0/217228176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599ee5e89f0 0x5599ee6298a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.669+0000 7fbfd0ff9700 0 client.8307 ms_handle_reset on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.669+0000 7fc001918b80 1 -- 172.21.15.92:0/217228176 >> 172.21.15.92:0/217228176 conn(0x5599ee5e70c0 msgr2=0x5599ee5e2870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.669+0000 7fc001918b80 1 -- 172.21.15.92:0/217228176 shutdown_connections 2023-10-06T10:44:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.669+0000 7fc001918b80 1 -- 172.21.15.92:0/217228176 wait complete. 2023-10-06T10:44:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.669+0000 7fc001918b80 20 client.8307 trim_cache size 0 max 16384 2023-10-06T10:44:25.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.685+0000 7fc001918b80 1 Processor -- start 2023-10-06T10:44:25.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.685+0000 7fc001918b80 1 -- start start 2023-10-06T10:44:25.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.686+0000 7fc001918b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599ee5e6c20 0x5599ee5e1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.686+0000 7fc001918b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599ee5e89f0 0x5599ee645420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.686+0000 7fc001918b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee62cae0 0x5599ee60f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.686+0000 7fc001918b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ee3b8e80 con 0x5599ee62cae0 2023-10-06T10:44:25.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.686+0000 7fc001918b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ee5e0110 con 0x5599ee5e89f0 2023-10-06T10:44:25.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.686+0000 7fc001918b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599ee630460 con 0x5599ee5e6c20 2023-10-06T10:44:25.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.686+0000 7fbff6bbd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee62cae0 0x5599ee60f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.686+0000 7fbff6bbd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee62cae0 0x5599ee60f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37694/0 (socket says 172.21.15.92:37694) 2023-10-06T10:44:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.686+0000 7fbff6bbd700 1 -- 172.21.15.92:0/248832121 learned_addr learned my addr 172.21.15.92:0/248832121 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.686+0000 7fbff5bbb700 1 --2- 172.21.15.92:0/248832121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599ee5e89f0 0x5599ee645420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.686+0000 7fbff63bc700 1 --2- 172.21.15.92:0/248832121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599ee5e6c20 0x5599ee5e1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.686+0000 7fbff6bbd700 1 -- 172.21.15.92:0/248832121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599ee5e6c20 msgr2=0x5599ee5e1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.686+0000 7fbff6bbd700 1 --2- 172.21.15.92:0/248832121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599ee5e6c20 0x5599ee5e1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.686+0000 7fbff6bbd700 1 -- 172.21.15.92:0/248832121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599ee5e89f0 msgr2=0x5599ee645420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.686+0000 7fbff6bbd700 1 --2- 172.21.15.92:0/248832121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599ee5e89f0 0x5599ee645420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.686+0000 7fbff6bbd700 1 -- 172.21.15.92:0/248832121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599ee5047d0 con 0x5599ee62cae0 2023-10-06T10:44:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.686+0000 7fbff63bc700 1 --2- 172.21.15.92:0/248832121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599ee5e6c20 0x5599ee5e1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:25.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.686+0000 7fbff6bbd700 1 --2- 172.21.15.92:0/248832121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee62cae0 0x5599ee60f6c0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8005c10 tx=0x7fbfe8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5c3579e2074f32 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.686+0000 7fbfd3fff700 1 -- 172.21.15.92:0/248832121 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8016450 con 0x5599ee62cae0 2023-10-06T10:44:25.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.686+0000 7fbfd3fff700 1 -- 172.21.15.92:0/248832121 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe80165b0 con 0x5599ee62cae0 2023-10-06T10:44:25.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.686+0000 7fbfd3fff700 1 -- 172.21.15.92:0/248832121 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8016970 con 0x5599ee62cae0 2023-10-06T10:44:25.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.687+0000 7fc001918b80 1 -- 172.21.15.92:0/248832121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee62cae0 msgr2=0x5599ee60f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.687+0000 7fc001918b80 1 --2- 172.21.15.92:0/248832121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee62cae0 0x5599ee60f6c0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8005c10 tx=0x7fbfe8009040 comp rx=0 tx=0).stop 2023-10-06T10:44:25.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.687+0000 7fc001918b80 1 -- 172.21.15.92:0/248832121 shutdown_connections 2023-10-06T10:44:25.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.687+0000 7fc001918b80 1 --2- 172.21.15.92:0/248832121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee62cae0 0x5599ee60f6c0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.687+0000 7fc001918b80 1 --2- 172.21.15.92:0/248832121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599ee5e6c20 0x5599ee5e1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.687+0000 7fc001918b80 1 --2- 172.21.15.92:0/248832121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599ee5e89f0 0x5599ee645420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.687+0000 7fc001918b80 1 -- 172.21.15.92:0/248832121 >> 172.21.15.92:0/248832121 conn(0x5599ee5e70c0 msgr2=0x5599ee62b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.687+0000 7fc001918b80 1 -- 172.21.15.92:0/248832121 shutdown_connections 2023-10-06T10:44:25.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.687+0000 7fc001918b80 1 -- 172.21.15.92:0/248832121 wait complete. 2023-10-06T10:44:25.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.687+0000 7fc001918b80 1 -- start start 2023-10-06T10:44:25.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.687+0000 7fc001918b80 1 Processor -- start 2023-10-06T10:44:25.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.688+0000 7fc001918b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599ee5e6c20 0x5599ee628500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.688+0000 7fc001918b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599ee5e89f0 0x5599ee6222b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.688+0000 7fc001918b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee62cae0 0x5599ee5e1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.688+0000 7fc001918b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ee692a30 con 0x5599ee62cae0 2023-10-06T10:44:25.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.688+0000 7fc001918b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ee622ae0 con 0x5599ee5e6c20 2023-10-06T10:44:25.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.688+0000 7fc001918b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599ee6260b0 con 0x5599ee5e89f0 2023-10-06T10:44:25.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.688+0000 7fc001918b80 10 client.0 _ll_register_callbacks cb 0x5599ee3d2f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:44:25.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.688+0000 7fbff6bbd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee62cae0 0x5599ee5e1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.688+0000 7fbff6bbd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee62cae0 0x5599ee5e1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37698/0 (socket says 172.21.15.92:37698) 2023-10-06T10:44:25.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.688+0000 7fbff6bbd700 1 -- 172.21.15.92:0/3637542088 learned_addr learned my addr 172.21.15.92:0/3637542088 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:25.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.688+0000 7fbff5bbb700 1 --2- 172.21.15.92:0/3637542088 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599ee5e89f0 0x5599ee6222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.688+0000 7fbff63bc700 1 --2- 172.21.15.92:0/3637542088 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599ee5e6c20 0x5599ee628500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.689+0000 7fbff6bbd700 1 -- 172.21.15.92:0/3637542088 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599ee5e89f0 msgr2=0x5599ee6222b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.689+0000 7fbff6bbd700 1 --2- 172.21.15.92:0/3637542088 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599ee5e89f0 0x5599ee6222b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.689+0000 7fbff6bbd700 1 -- 172.21.15.92:0/3637542088 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599ee5e6c20 msgr2=0x5599ee628500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.689+0000 7fbff6bbd700 1 --2- 172.21.15.92:0/3637542088 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599ee5e6c20 0x5599ee628500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.689+0000 7fbff6bbd700 1 -- 172.21.15.92:0/3637542088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe800f040 con 0x5599ee62cae0 2023-10-06T10:44:25.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.689+0000 7fbff6bbd700 1 --2- 172.21.15.92:0/3637542088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee62cae0 0x5599ee5e1fc0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8016b20 tx=0x7fbfe801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc7b9202f7839a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.689+0000 7fbfd0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:44:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.689+0000 7fbfd0ff9700 1 -- 172.21.15.92:0/3637542088 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8016e30 con 0x5599ee62cae0 2023-10-06T10:44:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.689+0000 7fbfd0ff9700 1 -- 172.21.15.92:0/3637542088 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fbfe801f660 con 0x5599ee62cae0 2023-10-06T10:44:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.689+0000 7fc001918b80 1 -- 172.21.15.92:0/3637542088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbfec0582b0 con 0x5599ee62cae0 2023-10-06T10:44:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.689+0000 7fbfd0ff9700 1 -- 172.21.15.92:0/3637542088 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfe801fa70 con 0x5599ee62cae0 2023-10-06T10:44:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.689+0000 7fc001918b80 10 client.8313 fetch_fsmap learned FSMap version 92 2023-10-06T10:44:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.689+0000 7fc001918b80 1 -- 172.21.15.92:0/3637542088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) v3 -- 0x5599ee4eb4c0 con 0x5599ee62cae0 2023-10-06T10:44:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.689+0000 7fbfd0ff9700 1 -- 172.21.15.92:0/3637542088 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec0582b0 con 0x5599ee62cae0 2023-10-06T10:44:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.689+0000 7fbfd0ff9700 1 -- 172.21.15.92:0/3637542088 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 92) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fbfe801fbd0 con 0x5599ee62cae0 2023-10-06T10:44:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.689+0000 7fc001918b80 10 client.8313 fetch_fsmap finished waiting for FSMap version 92 2023-10-06T10:44:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.689+0000 7fc001918b80 10 client.8313 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2023-10-06T10:44:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.689+0000 7fc001918b80 20 client.8313 populate_metadata read hostname 'smithi092' 2023-10-06T10:44:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.689+0000 7fc001918b80 1 --2- 172.21.15.92:0/3637542088 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x5599ee626540 0x5599ee5e2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.689+0000 7fc001918b80 4 client.8313 mds_command: new command op to 7701 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:44:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.689+0000 7fc001918b80 1 -- 172.21.15.92:0/3637542088 --> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5599ee4d8e50 con 0x5599ee626540 2023-10-06T10:44:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.689+0000 7fbff63bc700 1 --2- 172.21.15.92:0/3637542088 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x5599ee626540 0x5599ee5e2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.690+0000 7fbff63bc700 1 --2- 172.21.15.92:0/3637542088 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x5599ee626540 0x5599ee5e2500 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df3413beab2be7a7 server_cookie=10b339268a0d8b69 in_seq=0 out_seq=0 2023-10-06T10:44:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.690+0000 7fbfd0ff9700 10 client.8313 ms_handle_connect on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.690+0000 7fbfd0ff9700 1 -- 172.21.15.92:0/3637542088 <== mds.0 v2:172.21.15.124:6832/4203118992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5599ee4d8e50 con 0x5599ee626540 2023-10-06T10:44:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.690+0000 7fbfd0ff9700 10 client.8313 handle_command_reply: tid=0 2023-10-06T10:44:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.690+0000 7fc001918b80 1 client.8313 shutdown 2023-10-06T10:44:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.690+0000 7fc001918b80 1 -- 172.21.15.92:0/3637542088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee62cae0 msgr2=0x5599ee5e1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.690+0000 7fc001918b80 1 --2- 172.21.15.92:0/3637542088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee62cae0 0x5599ee5e1fc0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8016b20 tx=0x7fbfe801f000 comp rx=0 tx=0).stop 2023-10-06T10:44:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.690+0000 7fc001918b80 1 -- 172.21.15.92:0/3637542088 shutdown_connections 2023-10-06T10:44:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.690+0000 7fc001918b80 1 --2- 172.21.15.92:0/3637542088 >> [v2:172.21.15.124:6832/4203118992,v1:172.21.15.124:6834/4203118992] conn(0x5599ee626540 0x5599ee5e2500 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.690+0000 7fc001918b80 1 --2- 172.21.15.92:0/3637542088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599ee62cae0 0x5599ee5e1fc0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.690+0000 7fc001918b80 1 --2- 172.21.15.92:0/3637542088 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5599ee5e6c20 0x5599ee628500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.690+0000 7fbfd0ff9700 0 client.8313 ms_handle_reset on v2:172.21.15.124:6832/4203118992 2023-10-06T10:44:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.690+0000 7fc001918b80 1 --2- 172.21.15.92:0/3637542088 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5599ee5e89f0 0x5599ee6222b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.690+0000 7fc001918b80 1 -- 172.21.15.92:0/3637542088 >> 172.21.15.92:0/3637542088 conn(0x5599ee5e70c0 msgr2=0x5599ee611c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.690+0000 7fc001918b80 1 -- 172.21.15.92:0/3637542088 shutdown_connections 2023-10-06T10:44:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.690+0000 7fc001918b80 1 -- 172.21.15.92:0/3637542088 wait complete. 2023-10-06T10:44:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.690+0000 7fc001918b80 20 client.8313 trim_cache size 0 max 16384 2023-10-06T10:44:25.700 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:44:25.700 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T10:44:25.700 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:44:25.700 INFO:teuthology.orchestra.run.smithi092.stdout: "9669003b-2b0f-4975-abcd-f07cf375c689": { 2023-10-06T10:44:25.700 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:44:25.700 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "9669003b-2b0f-4975-abcd-f07cf375c689", 2023-10-06T10:44:25.700 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:44:25.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:25.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:25.700 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:44:25.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.691+0000 7fbff8620700 1 -- 172.21.15.92:0/3383807195 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfd4066f30 msgr2=0x7fbfd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.691+0000 7fbff8620700 1 --2- 172.21.15.92:0/3383807195 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfd4066f30 0x7fbfd40693e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0006fd0 tx=0x7fbfe0008040 comp rx=0 tx=0).stop 2023-10-06T10:44:25.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.691+0000 7fbff8620700 1 -- 172.21.15.92:0/3383807195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff0067800 msgr2=0x7fbff00c4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.691+0000 7fbff8620700 1 --2- 172.21.15.92:0/3383807195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff0067800 0x7fbff00c4e60 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fbfec013000 tx=0x7fbfec019000 comp rx=0 tx=0).stop 2023-10-06T10:44:25.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.691+0000 7fbff8620700 1 -- 172.21.15.92:0/3383807195 shutdown_connections 2023-10-06T10:44:25.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.691+0000 7fbff8620700 1 --2- 172.21.15.92:0/3383807195 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfd4066f30 0x7fbfd40693e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.691+0000 7fbff8620700 1 --2- 172.21.15.92:0/3383807195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbff015b490 0x7fbff0156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.691+0000 7fbff8620700 1 --2- 172.21.15.92:0/3383807195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff0067800 0x7fbff00c4e60 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.691+0000 7fbff8620700 1 --2- 172.21.15.92:0/3383807195 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbff0067bd0 0x7fbff00a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.691+0000 7fbff8620700 1 -- 172.21.15.92:0/3383807195 >> 172.21.15.92:0/3383807195 conn(0x7fbff00cd1a0 msgr2=0x7fbff006b990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.691+0000 7fbff8620700 1 -- 172.21.15.92:0/3383807195 shutdown_connections 2023-10-06T10:44:25.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.691+0000 7fbff8620700 1 -- 172.21.15.92:0/3383807195 wait complete. 2023-10-06T10:44:25.703 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:44:25.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:44:25.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.816+0000 7f2954d3a700 1 Processor -- start 2023-10-06T10:44:25.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.816+0000 7f2954d3a700 1 -- start start 2023-10-06T10:44:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.817+0000 7f2954d3a700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f29500696e0 0x7f2950069ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.817+0000 7f2954d3a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f295006d350 0x7f295006eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.817+0000 7f2954d3a700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f295006e6d0 con 0x7f29500ada60 2023-10-06T10:44:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.817+0000 7f2954d3a700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29500cd000 con 0x7f29500696e0 2023-10-06T10:44:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.817+0000 7f2954d3a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295006dac0 con 0x7f295006d350 2023-10-06T10:44:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.817+0000 7f294ed9d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f295006d350 0x7f295006eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.817+0000 7f294e59c700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f29500696e0 0x7f2950069ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.817+0000 7f294dd9b700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f29500ada60 0x7f29500aeeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:37582/0 (socket says 172.21.15.92:37582) 2023-10-06T10:44:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.817+0000 7f294e59c700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f29500696e0 0x7f2950069ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:50164/0 (socket says 172.21.15.92:50164) 2023-10-06T10:44:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.817+0000 7f294ed9d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f295006d350 0x7f295006eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:40032/0 (socket says 172.21.15.92:40032) 2023-10-06T10:44:25.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.817+0000 7f294dd9b700 1 -- 172.21.15.92:0/4244954963 learned_addr learned my addr 172.21.15.92:0/4244954963 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:25.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.818+0000 7f294ed9d700 1 -- 172.21.15.92:0/4244954963 >> v2:172.21.15.124:3300/0 conn(0x7f29500696e0 msgr2=0x7f2950069ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:25.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.818+0000 7f294ed9d700 1 --2- 172.21.15.92:0/4244954963 >> v2:172.21.15.124:3300/0 conn(0x7f29500696e0 0x7f2950069ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.818+0000 7f294ed9d700 1 -- 172.21.15.92:0/4244954963 >> v1:172.21.15.124:6789/0 conn(0x7f29500ada60 legacy=0x7f29500aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.818+0000 7f294ed9d700 1 -- 172.21.15.92:0/4244954963 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295006dda0 con 0x7f295006d350 2023-10-06T10:44:25.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.818+0000 7f294ed9d700 1 --2- 172.21.15.92:0/4244954963 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f295006d350 0x7f295006eb60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f29480098e0 tx=0x7f29480091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beb112293b7d4d4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.818+0000 7f294d59a700 1 -- 172.21.15.92:0/4244954963 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294800b640 con 0x7f295006d350 2023-10-06T10:44:25.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.818+0000 7f294d59a700 1 -- 172.21.15.92:0/4244954963 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294800b7a0 con 0x7f295006d350 2023-10-06T10:44:25.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.818+0000 7f294d59a700 1 -- 172.21.15.92:0/4244954963 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294800bb40 con 0x7f295006d350 2023-10-06T10:44:25.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.819+0000 7f2954d3a700 1 -- 172.21.15.92:0/4244954963 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f295006d350 msgr2=0x7f295006eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.819+0000 7f2954d3a700 1 --2- 172.21.15.92:0/4244954963 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f295006d350 0x7f295006eb60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f29480098e0 tx=0x7f29480091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.819+0000 7f2954d3a700 1 -- 172.21.15.92:0/4244954963 shutdown_connections 2023-10-06T10:44:25.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.819+0000 7f2954d3a700 1 --2- 172.21.15.92:0/4244954963 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f295006d350 0x7f295006eb60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.819+0000 7f2954d3a700 1 --2- 172.21.15.92:0/4244954963 >> v2:172.21.15.124:3300/0 conn(0x7f29500696e0 0x7f2950069ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.819+0000 7f2954d3a700 1 -- 172.21.15.92:0/4244954963 >> 172.21.15.92:0/4244954963 conn(0x7f2950068080 msgr2=0x7f29500cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:25.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.819+0000 7f2954d3a700 1 -- 172.21.15.92:0/4244954963 shutdown_connections 2023-10-06T10:44:25.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.819+0000 7f2954d3a700 1 -- 172.21.15.92:0/4244954963 wait complete. 2023-10-06T10:44:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.820+0000 7f2954d3a700 1 Processor -- start 2023-10-06T10:44:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.820+0000 7f2954d3a700 1 -- start start 2023-10-06T10:44:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f2954d3a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29500696e0 0x7f29500ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f2954d3a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f295006d350 0x7f29501573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f2954d3a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29500ada60 0x7f29500c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f2954d3a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29500ac600 con 0x7f295006d350 2023-10-06T10:44:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f2954d3a700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2950145de0 con 0x7f29500696e0 2023-10-06T10:44:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f2954d3a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29500c0a60 con 0x7f29500ada60 2023-10-06T10:44:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f294dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f295006d350 0x7f29501573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f294e59c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29500696e0 0x7f29500ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f294dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f295006d350 0x7f29501573b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37704/0 (socket says 172.21.15.92:37704) 2023-10-06T10:44:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f294dd9b700 1 -- 172.21.15.92:0/2693383409 learned_addr learned my addr 172.21.15.92:0/2693383409 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f294ed9d700 1 --2- 172.21.15.92:0/2693383409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29500ada60 0x7f29500c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f294dd9b700 1 -- 172.21.15.92:0/2693383409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29500ada60 msgr2=0x7f29500c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f294dd9b700 1 --2- 172.21.15.92:0/2693383409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29500ada60 0x7f29500c2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f294dd9b700 1 -- 172.21.15.92:0/2693383409 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29500696e0 msgr2=0x7f29500ac0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f294dd9b700 1 --2- 172.21.15.92:0/2693383409 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29500696e0 0x7f29500ac0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f294dd9b700 1 -- 172.21.15.92:0/2693383409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2948009420 con 0x7f295006d350 2023-10-06T10:44:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f294e59c700 1 --2- 172.21.15.92:0/2693383409 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29500696e0 0x7f29500ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:44:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f294dd9b700 1 --2- 172.21.15.92:0/2693383409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f295006d350 0x7f29501573b0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f294400c560 tx=0x7f2944001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f293e7fc700 1 -- 172.21.15.92:0/2693383409 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944005b30 con 0x7f295006d350 2023-10-06T10:44:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f293e7fc700 1 -- 172.21.15.92:0/2693383409 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2944005c90 con 0x7f295006d350 2023-10-06T10:44:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f2954d3a700 1 -- 172.21.15.92:0/2693383409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29500bfe50 con 0x7f295006d350 2023-10-06T10:44:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f293e7fc700 1 -- 172.21.15.92:0/2693383409 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944007320 con 0x7f295006d350 2023-10-06T10:44:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.821+0000 7f2954d3a700 1 -- 172.21.15.92:0/2693383409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29500c0410 con 0x7f295006d350 2023-10-06T10:44:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.822+0000 7f2954d3a700 1 -- 172.21.15.92:0/2693383409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29500c0bd0 con 0x7f295006d350 2023-10-06T10:44:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.823+0000 7f293e7fc700 1 -- 172.21.15.92:0/2693383409 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2944007690 con 0x7f295006d350 2023-10-06T10:44:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.823+0000 7f293e7fc700 1 --2- 172.21.15.92:0/2693383409 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2930066f30 0x7f29300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.823+0000 7f294e59c700 1 --2- 172.21.15.92:0/2693383409 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2930066f30 0x7f29300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.823+0000 7f293e7fc700 1 -- 172.21.15.92:0/2693383409 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2944083b10 con 0x7f295006d350 2023-10-06T10:44:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.824+0000 7f294e59c700 1 --2- 172.21.15.92:0/2693383409 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2930066f30 0x7f29300693e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2940010d20 tx=0x7f2940012040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:25.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:25.827+0000 7f293e7fc700 1 -- 172.21.15.92:0/2693383409 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2944050ae0 con 0x7f295006d350 2023-10-06T10:44:26.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.050+0000 7f2954d3a700 1 -- 172.21.15.92:0/2693383409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2950064a90 con 0x7f295006d350 2023-10-06T10:44:26.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.051+0000 7f293e7fc700 1 -- 172.21.15.92:0/2693383409 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4642 (secure 0 0 0) 0x7f2944017090 con 0x7f295006d350 2023-10-06T10:44:26.052 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:26.052 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2122441982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2122441982},{"type":"v1","addr":"172.21.15.124:6835","nonce":2122441982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:47.850832+0000","modified":"2023-10-06T10:44:20.587653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7701,"mds_1":7689,"mds_2":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6836/3605297930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3605297930},{"type":"v1","addr":"172.21.15.92:6836","nonce":3605297930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7689":{"gid":7689,"name":"c","rank":1,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.92:6837/4047304116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4047304116},{"type":"v1","addr":"172.21.15.92:6837","nonce":4047304116}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7701":{"gid":7701,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":11,"addr":"172.21.15.124:6834/4203118992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":4203118992},{"type":"v1","addr":"172.21.15.124:6834","nonce":4203118992}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T10:44:26.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.053+0000 7f2954d3a700 1 -- 172.21.15.92:0/2693383409 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2930066f30 msgr2=0x7f29300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:26.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.053+0000 7f2954d3a700 1 --2- 172.21.15.92:0/2693383409 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2930066f30 0x7f29300693e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2940010d20 tx=0x7f2940012040 comp rx=0 tx=0).stop 2023-10-06T10:44:26.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.053+0000 7f2954d3a700 1 -- 172.21.15.92:0/2693383409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f295006d350 msgr2=0x7f29501573b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:26.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.053+0000 7f2954d3a700 1 --2- 172.21.15.92:0/2693383409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f295006d350 0x7f29501573b0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f294400c560 tx=0x7f2944001020 comp rx=0 tx=0).stop 2023-10-06T10:44:26.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.053+0000 7f2954d3a700 1 -- 172.21.15.92:0/2693383409 shutdown_connections 2023-10-06T10:44:26.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.053+0000 7f2954d3a700 1 --2- 172.21.15.92:0/2693383409 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2930066f30 0x7f29300693e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:26.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.053+0000 7f2954d3a700 1 --2- 172.21.15.92:0/2693383409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29500ada60 0x7f29500c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:26.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.053+0000 7f2954d3a700 1 --2- 172.21.15.92:0/2693383409 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29500696e0 0x7f29500ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:26.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.053+0000 7f2954d3a700 1 --2- 172.21.15.92:0/2693383409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f295006d350 0x7f29501573b0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:26.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.053+0000 7f2954d3a700 1 -- 172.21.15.92:0/2693383409 >> 172.21.15.92:0/2693383409 conn(0x7f2950068080 msgr2=0x7f29501592e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:26.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.053+0000 7f2954d3a700 1 -- 172.21.15.92:0/2693383409 shutdown_connections 2023-10-06T10:44:26.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.053+0000 7f2954d3a700 1 -- 172.21.15.92:0/2693383409 wait complete. 2023-10-06T10:44:26.055 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 92 2023-10-06T10:44:26.066 DEBUG:tasks.cephfs.test_multimds_misc:current active=['b', 'c', 'a'] 2023-10-06T10:44:26.066 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:44:26.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.177+0000 7f547503e700 1 Processor -- start 2023-10-06T10:44:26.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.177+0000 7f547503e700 1 -- start start 2023-10-06T10:44:26.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.178+0000 7f547503e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f54700b49a0 0x7f54700aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:26.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.178+0000 7f547503e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54700b0760 0x7f54700b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:26.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.178+0000 7f547503e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54700b0c60 con 0x7f5470159560 2023-10-06T10:44:26.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.178+0000 7f547503e700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54700b0f40 con 0x7f54700b49a0 2023-10-06T10:44:26.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.178+0000 7f547503e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54700b1220 con 0x7f54700b0760 2023-10-06T10:44:26.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.178+0000 7f546f59e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54700b0760 0x7f54700b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:26.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.178+0000 7f546659c700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f54700b49a0 0x7f54700aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:26.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.178+0000 7f546ed9d700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f5470159560 0x7f54700aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:37598/0 (socket says 172.21.15.92:37598) 2023-10-06T10:44:26.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.178+0000 7f546ed9d700 1 -- 172.21.15.92:0/2056850155 learned_addr learned my addr 172.21.15.92:0/2056850155 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:26.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.179+0000 7f546659c700 1 -- 172.21.15.92:0/2056850155 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54700b0760 msgr2=0x7f54700b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:26.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.179+0000 7f546659c700 1 --2- 172.21.15.92:0/2056850155 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54700b0760 0x7f54700b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:26.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.179+0000 7f546659c700 1 -- 172.21.15.92:0/2056850155 >> v1:172.21.15.124:6789/0 conn(0x7f5470159560 legacy=0x7f54700aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:26.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.179+0000 7f546659c700 1 -- 172.21.15.92:0/2056850155 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54700b1500 con 0x7f54700b49a0 2023-10-06T10:44:26.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.179+0000 7f546e59c700 1 -- 172.21.15.92:0/2056850155 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 151128292 0 0) 0x7f54700b0c60 con 0x7f5470159560 2023-10-06T10:44:26.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.179+0000 7f546f59e700 1 --2- 172.21.15.92:0/2056850155 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54700b0760 0x7f54700b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:44:26.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.179+0000 7f546659c700 1 --2- 172.21.15.92:0/2056850155 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54700b49a0 0x7f54700aac40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5458009910 tx=0x7f54580091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=414cbded25607716 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:26.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.179+0000 7f546e59c700 1 -- 172.21.15.92:0/2056850155 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545800b640 con 0x7f54700b49a0 2023-10-06T10:44:26.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.179+0000 7f546e59c700 1 -- 172.21.15.92:0/2056850155 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545800b7a0 con 0x7f54700b49a0 2023-10-06T10:44:26.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.179+0000 7f546e59c700 1 -- 172.21.15.92:0/2056850155 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545800bb60 con 0x7f54700b49a0 2023-10-06T10:44:26.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.179+0000 7f547503e700 1 -- 172.21.15.92:0/2056850155 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54700b49a0 msgr2=0x7f54700aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:26.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.179+0000 7f547503e700 1 --2- 172.21.15.92:0/2056850155 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54700b49a0 0x7f54700aac40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5458009910 tx=0x7f54580091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:26.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.180+0000 7f547503e700 1 -- 172.21.15.92:0/2056850155 shutdown_connections 2023-10-06T10:44:26.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.180+0000 7f547503e700 1 --2- 172.21.15.92:0/2056850155 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54700b0760 0x7f54700b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:26.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.180+0000 7f547503e700 1 --2- 172.21.15.92:0/2056850155 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54700b49a0 0x7f54700aac40 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:26.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.180+0000 7f547503e700 1 -- 172.21.15.92:0/2056850155 >> 172.21.15.92:0/2056850155 conn(0x7f54700997f0 msgr2=0x7f547006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:26.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.180+0000 7f547503e700 1 -- 172.21.15.92:0/2056850155 shutdown_connections 2023-10-06T10:44:26.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.180+0000 7f547503e700 1 -- 172.21.15.92:0/2056850155 wait complete. 2023-10-06T10:44:26.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.181+0000 7f547503e700 1 Processor -- start 2023-10-06T10:44:26.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.181+0000 7f547503e700 1 -- start start 2023-10-06T10:44:26.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.182+0000 7f547503e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54700b0760 0x7f547015c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:26.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.182+0000 7f547503e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54700b49a0 0x7f5470156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:26.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.182+0000 7f547503e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5470159560 0x7f547009d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:26.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.182+0000 7f547503e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54701696a0 con 0x7f54700b0760 2023-10-06T10:44:26.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.182+0000 7f547503e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547015b310 con 0x7f5470159560 2023-10-06T10:44:26.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.182+0000 7f547503e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547009df00 con 0x7f54700b49a0 2023-10-06T10:44:26.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.182+0000 7f546ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54700b0760 0x7f547015c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:26.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.182+0000 7f546ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54700b0760 0x7f547015c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37712/0 (socket says 172.21.15.92:37712) 2023-10-06T10:44:26.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.182+0000 7f546ed9d700 1 -- 172.21.15.92:0/3788205165 learned_addr learned my addr 172.21.15.92:0/3788205165 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:26.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.182+0000 7f546659c700 1 --2- 172.21.15.92:0/3788205165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54700b49a0 0x7f5470156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:26.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.182+0000 7f546f59e700 1 --2- 172.21.15.92:0/3788205165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5470159560 0x7f547009d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:26.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.182+0000 7f546ed9d700 1 -- 172.21.15.92:0/3788205165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54700b49a0 msgr2=0x7f5470156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:26.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.182+0000 7f546ed9d700 1 --2- 172.21.15.92:0/3788205165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54700b49a0 0x7f5470156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:26.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.182+0000 7f546ed9d700 1 -- 172.21.15.92:0/3788205165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5470159560 msgr2=0x7f547009d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:26.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.182+0000 7f546ed9d700 1 --2- 172.21.15.92:0/3788205165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5470159560 0x7f547009d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:26.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.182+0000 7f546ed9d700 1 -- 172.21.15.92:0/3788205165 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5468009a20 con 0x7f54700b0760 2023-10-06T10:44:26.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.182+0000 7f546ed9d700 1 --2- 172.21.15.92:0/3788205165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54700b0760 0x7f547015c3f0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f546000c5b0 tx=0x7f5460001030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:26.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.182+0000 7f54677fe700 1 -- 172.21.15.92:0/3788205165 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460005b80 con 0x7f54700b0760 2023-10-06T10:44:26.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.182+0000 7f54677fe700 1 -- 172.21.15.92:0/3788205165 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5460005ce0 con 0x7f54700b0760 2023-10-06T10:44:26.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.182+0000 7f547503e700 1 -- 172.21.15.92:0/3788205165 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5458009420 con 0x7f54700b0760 2023-10-06T10:44:26.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.182+0000 7f54677fe700 1 -- 172.21.15.92:0/3788205165 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54600072e0 con 0x7f54700b0760 2023-10-06T10:44:26.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.182+0000 7f547503e700 1 -- 172.21.15.92:0/3788205165 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547009d940 con 0x7f54700b0760 2023-10-06T10:44:26.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.183+0000 7f547503e700 1 -- 172.21.15.92:0/3788205165 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547009a5b0 con 0x7f54700b0760 2023-10-06T10:44:26.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.184+0000 7f54677fe700 1 -- 172.21.15.92:0/3788205165 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5460007690 con 0x7f54700b0760 2023-10-06T10:44:26.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.184+0000 7f54677fe700 1 --2- 172.21.15.92:0/3788205165 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5450066f30 0x7f54500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:26.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.184+0000 7f54677fe700 1 -- 172.21.15.92:0/3788205165 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5460083ac0 con 0x7f54700b0760 2023-10-06T10:44:26.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.184+0000 7f546659c700 1 --2- 172.21.15.92:0/3788205165 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5450066f30 0x7f54500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:26.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.185+0000 7f546659c700 1 --2- 172.21.15.92:0/3788205165 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5450066f30 0x7f54500693e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5458009910 tx=0x7f5458017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:26.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.188+0000 7f54677fe700 1 -- 172.21.15.92:0/3788205165 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5460050c00 con 0x7f54700b0760 2023-10-06T10:44:26.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.448+0000 7f547503e700 1 -- 172.21.15.92:0/3788205165 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f54700b17e0 con 0x7f54700b0760 2023-10-06T10:44:26.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.448+0000 7f54677fe700 1 -- 172.21.15.92:0/3788205165 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1970 (secure 0 0 0) 0x7f5460083e60 con 0x7f54700b0760 2023-10-06T10:44:26.450 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:26.450 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":387,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":698974,"bytes_used":2445807616,"bytes_avail":770648305664,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6654,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":92,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"a","status":"up:active","gid":7686},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7689},{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":7701}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2023-10-06T10:44:24.162386+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T10:44:26.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.450+0000 7f547503e700 1 -- 172.21.15.92:0/3788205165 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5450066f30 msgr2=0x7f54500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:26.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.450+0000 7f547503e700 1 --2- 172.21.15.92:0/3788205165 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5450066f30 0x7f54500693e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5458009910 tx=0x7f5458017000 comp rx=0 tx=0).stop 2023-10-06T10:44:26.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.450+0000 7f547503e700 1 -- 172.21.15.92:0/3788205165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54700b0760 msgr2=0x7f547015c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:26.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.450+0000 7f547503e700 1 --2- 172.21.15.92:0/3788205165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54700b0760 0x7f547015c3f0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f546000c5b0 tx=0x7f5460001030 comp rx=0 tx=0).stop 2023-10-06T10:44:26.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.450+0000 7f547503e700 1 -- 172.21.15.92:0/3788205165 shutdown_connections 2023-10-06T10:44:26.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.450+0000 7f547503e700 1 --2- 172.21.15.92:0/3788205165 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5450066f30 0x7f54500693e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:26.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.450+0000 7f547503e700 1 --2- 172.21.15.92:0/3788205165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5470159560 0x7f547009d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:26.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.450+0000 7f547503e700 1 --2- 172.21.15.92:0/3788205165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54700b0760 0x7f547015c3f0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:26.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.450+0000 7f547503e700 1 --2- 172.21.15.92:0/3788205165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54700b49a0 0x7f5470156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:26.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.450+0000 7f547503e700 1 -- 172.21.15.92:0/3788205165 >> 172.21.15.92:0/3788205165 conn(0x7f54700997f0 msgr2=0x7f5470069120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:26.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.450+0000 7f547503e700 1 -- 172.21.15.92:0/3788205165 shutdown_connections 2023-10-06T10:44:26.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:26.450+0000 7f547503e700 1 -- 172.21.15.92:0/3788205165 wait complete. 2023-10-06T10:44:26.463 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2023-10-06T10:44:27.464 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:44:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.582+0000 7fbe2d979700 1 Processor -- start 2023-10-06T10:44:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.582+0000 7fbe2d979700 1 -- start start 2023-10-06T10:44:27.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.583+0000 7fbe2d979700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbe28159650 0x7fbe280ac440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:27.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.583+0000 7fbe2d979700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe280685c0 0x7fbe28068a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:27.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.583+0000 7fbe2d979700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe28068f70 con 0x7fbe2815d350 2023-10-06T10:44:27.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.583+0000 7fbe2d979700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe280a9f90 con 0x7fbe28159650 2023-10-06T10:44:27.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.583+0000 7fbe2d979700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe280ac980 con 0x7fbe280685c0 2023-10-06T10:44:27.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.583+0000 7fbe26ffd700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbe28159650 0x7fbe280ac440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:27.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.583+0000 7fbe1ffff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe280685c0 0x7fbe28068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:27.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.583+0000 7fbe26ffd700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbe28159650 0x7fbe280ac440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:50212/0 (socket says 172.21.15.92:50212) 2023-10-06T10:44:27.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.583+0000 7fbe1ffff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe280685c0 0x7fbe28068a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:40056/0 (socket says 172.21.15.92:40056) 2023-10-06T10:44:27.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.583+0000 7fbe26ffd700 1 -- 172.21.15.92:0/852258967 learned_addr learned my addr 172.21.15.92:0/852258967 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:27.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.583+0000 7fbe277fe700 1 --1- 172.21.15.92:0/852258967 >> v1:172.21.15.124:6789/0 conn(0x7fbe2815d350 0x7fbe2815d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:37602/0 (socket says 172.21.15.92:37602) 2023-10-06T10:44:27.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.584+0000 7fbe26ffd700 1 -- 172.21.15.92:0/852258967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe280685c0 msgr2=0x7fbe28068a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:27.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.584+0000 7fbe26ffd700 1 --2- 172.21.15.92:0/852258967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe280685c0 0x7fbe28068a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:27.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.584+0000 7fbe26ffd700 1 -- 172.21.15.92:0/852258967 >> v1:172.21.15.124:6789/0 conn(0x7fbe2815d350 legacy=0x7fbe2815d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:27.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.584+0000 7fbe26ffd700 1 -- 172.21.15.92:0/852258967 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe280abfa0 con 0x7fbe28159650 2023-10-06T10:44:27.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.584+0000 7fbe1ffff700 1 --2- 172.21.15.92:0/852258967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe280685c0 0x7fbe28068a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:44:27.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.584+0000 7fbe26ffd700 1 --2- 172.21.15.92:0/852258967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe28159650 0x7fbe280ac440 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009990 tx=0x7fbe18009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2baa0b5c22f80cae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:27.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.584+0000 7fbe267fc700 1 -- 172.21.15.92:0/852258967 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe18009e80 con 0x7fbe28159650 2023-10-06T10:44:27.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.584+0000 7fbe267fc700 1 -- 172.21.15.92:0/852258967 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe18005c10 con 0x7fbe28159650 2023-10-06T10:44:27.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.584+0000 7fbe267fc700 1 -- 172.21.15.92:0/852258967 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1800b650 con 0x7fbe28159650 2023-10-06T10:44:27.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.585+0000 7fbe2d979700 1 -- 172.21.15.92:0/852258967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe28159650 msgr2=0x7fbe280ac440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:27.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.585+0000 7fbe2d979700 1 --2- 172.21.15.92:0/852258967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe28159650 0x7fbe280ac440 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009990 tx=0x7fbe18009220 comp rx=0 tx=0).stop 2023-10-06T10:44:27.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.585+0000 7fbe2d979700 1 -- 172.21.15.92:0/852258967 shutdown_connections 2023-10-06T10:44:27.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.585+0000 7fbe2d979700 1 --2- 172.21.15.92:0/852258967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe28159650 0x7fbe280ac440 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:27.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.585+0000 7fbe2d979700 1 --2- 172.21.15.92:0/852258967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe280685c0 0x7fbe28068a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:27.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.585+0000 7fbe2d979700 1 -- 172.21.15.92:0/852258967 >> 172.21.15.92:0/852258967 conn(0x7fbe2806de40 msgr2=0x7fbe2806e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:27.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.585+0000 7fbe2d979700 1 -- 172.21.15.92:0/852258967 shutdown_connections 2023-10-06T10:44:27.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.585+0000 7fbe2d979700 1 -- 172.21.15.92:0/852258967 wait complete. 2023-10-06T10:44:27.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.586+0000 7fbe2d979700 1 Processor -- start 2023-10-06T10:44:27.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.586+0000 7fbe2d979700 1 -- start start 2023-10-06T10:44:27.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.586+0000 7fbe2d979700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe280685c0 0x7fbe2816c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:27.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.586+0000 7fbe2d979700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe28159650 0x7fbe28153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:27.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.586+0000 7fbe2d979700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe2815d350 0x7fbe280c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:27.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.586+0000 7fbe2d979700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe280cbd80 con 0x7fbe280685c0 2023-10-06T10:44:27.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.586+0000 7fbe2d979700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2816c2d0 con 0x7fbe2815d350 2023-10-06T10:44:27.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.586+0000 7fbe2d979700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe28153d50 con 0x7fbe28159650 2023-10-06T10:44:27.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.587+0000 7fbe26ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe280685c0 0x7fbe2816c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:27.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.587+0000 7fbe26ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe280685c0 0x7fbe2816c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37716/0 (socket says 172.21.15.92:37716) 2023-10-06T10:44:27.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.587+0000 7fbe26ffd700 1 -- 172.21.15.92:0/3335457201 learned_addr learned my addr 172.21.15.92:0/3335457201 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:27.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.587+0000 7fbe1ffff700 1 --2- 172.21.15.92:0/3335457201 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe28159650 0x7fbe28153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:27.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.587+0000 7fbe277fe700 1 --2- 172.21.15.92:0/3335457201 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe2815d350 0x7fbe280c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:27.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.587+0000 7fbe26ffd700 1 -- 172.21.15.92:0/3335457201 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe28159650 msgr2=0x7fbe28153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:27.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.587+0000 7fbe26ffd700 1 --2- 172.21.15.92:0/3335457201 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe28159650 0x7fbe28153810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:27.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.587+0000 7fbe26ffd700 1 -- 172.21.15.92:0/3335457201 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe2815d350 msgr2=0x7fbe280c9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:27.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.587+0000 7fbe26ffd700 1 --2- 172.21.15.92:0/3335457201 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe2815d350 0x7fbe280c9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:27.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.587+0000 7fbe26ffd700 1 -- 172.21.15.92:0/3335457201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe10009420 con 0x7fbe280685c0 2023-10-06T10:44:27.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.587+0000 7fbe26ffd700 1 --2- 172.21.15.92:0/3335457201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe280685c0 0x7fbe2816c760 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800b3c0 tx=0x7fbe18016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:27.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.587+0000 7fbe1effd700 1 -- 172.21.15.92:0/3335457201 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe18011980 con 0x7fbe280685c0 2023-10-06T10:44:27.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.587+0000 7fbe1effd700 1 -- 172.21.15.92:0/3335457201 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe18020070 con 0x7fbe280685c0 2023-10-06T10:44:27.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.587+0000 7fbe2d979700 1 -- 172.21.15.92:0/3335457201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe18009480 con 0x7fbe280685c0 2023-10-06T10:44:27.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.587+0000 7fbe1effd700 1 -- 172.21.15.92:0/3335457201 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe18011d10 con 0x7fbe280685c0 2023-10-06T10:44:27.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.587+0000 7fbe2d979700 1 -- 172.21.15.92:0/3335457201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe2816b9a0 con 0x7fbe280685c0 2023-10-06T10:44:27.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.588+0000 7fbe2d979700 1 -- 172.21.15.92:0/3335457201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe28064a90 con 0x7fbe280685c0 2023-10-06T10:44:27.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.588+0000 7fbe1effd700 1 -- 172.21.15.92:0/3335457201 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbe18022030 con 0x7fbe280685c0 2023-10-06T10:44:27.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.589+0000 7fbe1effd700 1 --2- 172.21.15.92:0/3335457201 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbe08066f30 0x7fbe080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:27.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.589+0000 7fbe1effd700 1 -- 172.21.15.92:0/3335457201 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fbe180862b0 con 0x7fbe280685c0 2023-10-06T10:44:27.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.589+0000 7fbe1ffff700 1 --2- 172.21.15.92:0/3335457201 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbe08066f30 0x7fbe080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:27.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.589+0000 7fbe1ffff700 1 --2- 172.21.15.92:0/3335457201 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbe08066f30 0x7fbe080693e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fbe280cc8b0 tx=0x7fbe100091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:27.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.593+0000 7fbe1effd700 1 -- 172.21.15.92:0/3335457201 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe18053280 con 0x7fbe280685c0 2023-10-06T10:44:27.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.817+0000 7fbe2d979700 1 -- 172.21.15.92:0/3335457201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe2804bb40 con 0x7fbe280685c0 2023-10-06T10:44:27.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.817+0000 7fbe1effd700 1 -- 172.21.15.92:0/3335457201 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4642 (secure 0 0 0) 0x7fbe180568a0 con 0x7fbe280685c0 2023-10-06T10:44:27.818 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:27.819 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2122441982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2122441982},{"type":"v1","addr":"172.21.15.124:6835","nonce":2122441982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:47.850832+0000","modified":"2023-10-06T10:44:20.587653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7701,"mds_1":7689,"mds_2":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6836/3605297930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3605297930},{"type":"v1","addr":"172.21.15.92:6836","nonce":3605297930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7689":{"gid":7689,"name":"c","rank":1,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.92:6837/4047304116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4047304116},{"type":"v1","addr":"172.21.15.92:6837","nonce":4047304116}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7701":{"gid":7701,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":11,"addr":"172.21.15.124:6834/4203118992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":4203118992},{"type":"v1","addr":"172.21.15.124:6834","nonce":4203118992}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T10:44:27.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.819+0000 7fbe2d979700 1 -- 172.21.15.92:0/3335457201 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbe08066f30 msgr2=0x7fbe080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:27.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.819+0000 7fbe2d979700 1 --2- 172.21.15.92:0/3335457201 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbe08066f30 0x7fbe080693e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fbe280cc8b0 tx=0x7fbe100091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:27.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.819+0000 7fbe2d979700 1 -- 172.21.15.92:0/3335457201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe280685c0 msgr2=0x7fbe2816c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:27.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.819+0000 7fbe2d979700 1 --2- 172.21.15.92:0/3335457201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe280685c0 0x7fbe2816c760 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800b3c0 tx=0x7fbe18016000 comp rx=0 tx=0).stop 2023-10-06T10:44:27.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.819+0000 7fbe2d979700 1 -- 172.21.15.92:0/3335457201 shutdown_connections 2023-10-06T10:44:27.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.819+0000 7fbe2d979700 1 --2- 172.21.15.92:0/3335457201 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbe08066f30 0x7fbe080693e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:27.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.819+0000 7fbe2d979700 1 --2- 172.21.15.92:0/3335457201 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe2815d350 0x7fbe280c9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:27.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.819+0000 7fbe2d979700 1 --2- 172.21.15.92:0/3335457201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe280685c0 0x7fbe2816c760 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:27.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.819+0000 7fbe2d979700 1 --2- 172.21.15.92:0/3335457201 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe28159650 0x7fbe28153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:27.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.819+0000 7fbe2d979700 1 -- 172.21.15.92:0/3335457201 >> 172.21.15.92:0/3335457201 conn(0x7fbe2806de40 msgr2=0x7fbe280aa420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:27.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.819+0000 7fbe2d979700 1 -- 172.21.15.92:0/3335457201 shutdown_connections 2023-10-06T10:44:27.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.819+0000 7fbe2d979700 1 -- 172.21.15.92:0/3335457201 wait complete. 2023-10-06T10:44:27.822 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 92 2023-10-06T10:44:27.832 DEBUG:tasks.cephfs.test_multimds_misc:current active=['b', 'c', 'a'] 2023-10-06T10:44:27.832 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:44:27.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.945+0000 7f639a866700 1 Processor -- start 2023-10-06T10:44:27.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.945+0000 7f639a866700 1 -- start start 2023-10-06T10:44:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.946+0000 7f639a866700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f63940b49a0 0x7f63940aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.946+0000 7f639a866700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63940b0760 0x7f63940b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.946+0000 7f639a866700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63940b0c60 con 0x7f6394159560 2023-10-06T10:44:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.946+0000 7f639a866700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63940b0f40 con 0x7f63940b49a0 2023-10-06T10:44:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.946+0000 7f639a866700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63940b1220 con 0x7f63940b0760 2023-10-06T10:44:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.946+0000 7f63937fe700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f63940b49a0 0x7f63940aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.946+0000 7f6398e03700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63940b0760 0x7f63940b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.946+0000 7f6393fff700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f6394159560 0x7f63940aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:37616/0 (socket says 172.21.15.92:37616) 2023-10-06T10:44:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.946+0000 7f63937fe700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f63940b49a0 0x7f63940aac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:50230/0 (socket says 172.21.15.92:50230) 2023-10-06T10:44:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.946+0000 7f6393fff700 1 -- 172.21.15.92:0/1066225364 learned_addr learned my addr 172.21.15.92:0/1066225364 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:27.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.947+0000 7f6398e03700 1 -- 172.21.15.92:0/1066225364 >> v2:172.21.15.124:3300/0 conn(0x7f63940b49a0 msgr2=0x7f63940aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:27.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.947+0000 7f6398e03700 1 --2- 172.21.15.92:0/1066225364 >> v2:172.21.15.124:3300/0 conn(0x7f63940b49a0 0x7f63940aac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:27.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.947+0000 7f6398e03700 1 -- 172.21.15.92:0/1066225364 >> v1:172.21.15.124:6789/0 conn(0x7f6394159560 legacy=0x7f63940aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:27.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.947+0000 7f6398e03700 1 -- 172.21.15.92:0/1066225364 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63940b1500 con 0x7f63940b0760 2023-10-06T10:44:27.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.947+0000 7f63937fe700 1 --2- 172.21.15.92:0/1066225364 >> v2:172.21.15.124:3300/0 conn(0x7f63940b49a0 0x7f63940aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:44:27.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.947+0000 7f6398e03700 1 --2- 172.21.15.92:0/1066225364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63940b0760 0x7f63940b1bc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f638c009e90 tx=0x7f638c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da76f5d2863136ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:27.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.947+0000 7f6392ffd700 1 -- 172.21.15.92:0/1066225364 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c00e070 con 0x7f63940b0760 2023-10-06T10:44:27.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.947+0000 7f6392ffd700 1 -- 172.21.15.92:0/1066225364 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638c00bbf0 con 0x7f63940b0760 2023-10-06T10:44:27.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.947+0000 7f6392ffd700 1 -- 172.21.15.92:0/1066225364 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c012430 con 0x7f63940b0760 2023-10-06T10:44:27.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.947+0000 7f639a866700 1 -- 172.21.15.92:0/1066225364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63940b0760 msgr2=0x7f63940b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:27.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.947+0000 7f639a866700 1 --2- 172.21.15.92:0/1066225364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63940b0760 0x7f63940b1bc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f638c009e90 tx=0x7f638c009780 comp rx=0 tx=0).stop 2023-10-06T10:44:27.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.947+0000 7f639a866700 1 -- 172.21.15.92:0/1066225364 shutdown_connections 2023-10-06T10:44:27.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.947+0000 7f639a866700 1 --2- 172.21.15.92:0/1066225364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63940b0760 0x7f63940b1bc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:27.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.947+0000 7f639a866700 1 --2- 172.21.15.92:0/1066225364 >> v2:172.21.15.124:3300/0 conn(0x7f63940b49a0 0x7f63940aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:27.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.947+0000 7f639a866700 1 -- 172.21.15.92:0/1066225364 >> 172.21.15.92:0/1066225364 conn(0x7f63940997f0 msgr2=0x7f639406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:27.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.948+0000 7f639a866700 1 -- 172.21.15.92:0/1066225364 shutdown_connections 2023-10-06T10:44:27.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.948+0000 7f639a866700 1 -- 172.21.15.92:0/1066225364 wait complete. 2023-10-06T10:44:27.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.949+0000 7f639a866700 1 Processor -- start 2023-10-06T10:44:27.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.949+0000 7f639a866700 1 -- start start 2023-10-06T10:44:27.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.949+0000 7f639a866700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63940b0760 0x7f63940abb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:27.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.949+0000 7f639a866700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63940b49a0 0x7f63940adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:27.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.949+0000 7f639a866700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6394159560 0x7f63941579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:27.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.949+0000 7f639a866700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63940ae4b0 con 0x7f6394159560 2023-10-06T10:44:27.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.949+0000 7f639a866700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63940af340 con 0x7f63940b0760 2023-10-06T10:44:27.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.949+0000 7f639a866700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63940ae730 con 0x7f63940b49a0 2023-10-06T10:44:27.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.950+0000 7f6398e03700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6394159560 0x7f63941579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:27.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.950+0000 7f6398e03700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6394159560 0x7f63941579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37728/0 (socket says 172.21.15.92:37728) 2023-10-06T10:44:27.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.950+0000 7f6398e03700 1 -- 172.21.15.92:0/400561410 learned_addr learned my addr 172.21.15.92:0/400561410 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:27.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.950+0000 7f6393fff700 1 --2- 172.21.15.92:0/400561410 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63940b0760 0x7f63940abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:27.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.950+0000 7f63937fe700 1 --2- 172.21.15.92:0/400561410 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63940b49a0 0x7f63940adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:27.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.950+0000 7f6398e03700 1 -- 172.21.15.92:0/400561410 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63940b49a0 msgr2=0x7f63940adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:27.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.950+0000 7f6398e03700 1 --2- 172.21.15.92:0/400561410 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63940b49a0 0x7f63940adf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:27.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.950+0000 7f6398e03700 1 -- 172.21.15.92:0/400561410 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63940b0760 msgr2=0x7f63940abb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:27.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.950+0000 7f6398e03700 1 --2- 172.21.15.92:0/400561410 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63940b0760 0x7f63940abb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:27.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.950+0000 7f6398e03700 1 -- 172.21.15.92:0/400561410 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6388009420 con 0x7f6394159560 2023-10-06T10:44:27.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.950+0000 7f6398e03700 1 --2- 172.21.15.92:0/400561410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6394159560 0x7f63941579a0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f638c013000 tx=0x7f638c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:27.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.950+0000 7f637bfff700 1 -- 172.21.15.92:0/400561410 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c012ce0 con 0x7f6394159560 2023-10-06T10:44:27.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.950+0000 7f637bfff700 1 -- 172.21.15.92:0/400561410 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638c021070 con 0x7f6394159560 2023-10-06T10:44:27.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.950+0000 7f639a866700 1 -- 172.21.15.92:0/400561410 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f638c009a20 con 0x7f6394159560 2023-10-06T10:44:27.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.950+0000 7f637bfff700 1 -- 172.21.15.92:0/400561410 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c026680 con 0x7f6394159560 2023-10-06T10:44:27.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.950+0000 7f639a866700 1 -- 172.21.15.92:0/400561410 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63940aecf0 con 0x7f6394159560 2023-10-06T10:44:27.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.951+0000 7f639a866700 1 -- 172.21.15.92:0/400561410 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6394064a90 con 0x7f6394159560 2023-10-06T10:44:27.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.951+0000 7f637bfff700 1 -- 172.21.15.92:0/400561410 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f638c023030 con 0x7f6394159560 2023-10-06T10:44:27.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.952+0000 7f637bfff700 1 --2- 172.21.15.92:0/400561410 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6380066f30 0x7f63800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:27.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.952+0000 7f637bfff700 1 -- 172.21.15.92:0/400561410 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f638c0869b0 con 0x7f6394159560 2023-10-06T10:44:27.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.952+0000 7f6393fff700 1 --2- 172.21.15.92:0/400561410 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6380066f30 0x7f63800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:27.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.952+0000 7f6393fff700 1 --2- 172.21.15.92:0/400561410 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6380066f30 0x7f63800693e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6384007fd0 tx=0x7f6384007f00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:27.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:27.956+0000 7f637bfff700 1 -- 172.21.15.92:0/400561410 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f638c053980 con 0x7f6394159560 2023-10-06T10:44:28.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:28.211+0000 7f639a866700 1 -- 172.21.15.92:0/400561410 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f63940b17e0 con 0x7f6394159560 2023-10-06T10:44:28.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:28.212+0000 7f637bfff700 1 -- 172.21.15.92:0/400561410 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1970 (secure 0 0 0) 0x7f638c012e60 con 0x7f6394159560 2023-10-06T10:44:28.213 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:28.213 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":389,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":704416,"bytes_used":2445824000,"bytes_avail":770648289280,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":7337,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":92,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"a","status":"up:active","gid":7686},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7689},{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":7701}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2023-10-06T10:44:24.162386+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T10:44:28.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:28.214+0000 7f639a866700 1 -- 172.21.15.92:0/400561410 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6380066f30 msgr2=0x7f63800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:28.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:28.214+0000 7f639a866700 1 --2- 172.21.15.92:0/400561410 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6380066f30 0x7f63800693e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6384007fd0 tx=0x7f6384007f00 comp rx=0 tx=0).stop 2023-10-06T10:44:28.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:28.214+0000 7f639a866700 1 -- 172.21.15.92:0/400561410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6394159560 msgr2=0x7f63941579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:28.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:28.214+0000 7f639a866700 1 --2- 172.21.15.92:0/400561410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6394159560 0x7f63941579a0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f638c013000 tx=0x7f638c019000 comp rx=0 tx=0).stop 2023-10-06T10:44:28.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:28.214+0000 7f639a866700 1 -- 172.21.15.92:0/400561410 shutdown_connections 2023-10-06T10:44:28.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:28.214+0000 7f639a866700 1 --2- 172.21.15.92:0/400561410 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6380066f30 0x7f63800693e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:28.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:28.214+0000 7f639a866700 1 --2- 172.21.15.92:0/400561410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6394159560 0x7f63941579a0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:28.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:28.214+0000 7f639a866700 1 --2- 172.21.15.92:0/400561410 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63940b0760 0x7f63940abb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:28.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:28.214+0000 7f639a866700 1 --2- 172.21.15.92:0/400561410 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63940b49a0 0x7f63940adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:28.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:28.214+0000 7f639a866700 1 -- 172.21.15.92:0/400561410 >> 172.21.15.92:0/400561410 conn(0x7f63940997f0 msgr2=0x7f63941588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:28.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:28.214+0000 7f639a866700 1 -- 172.21.15.92:0/400561410 shutdown_connections 2023-10-06T10:44:28.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:28.214+0000 7f639a866700 1 -- 172.21.15.92:0/400561410 wait complete. 2023-10-06T10:44:28.227 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2023-10-06T10:44:29.227 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:44:29.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.342+0000 7f764b459700 1 Processor -- start 2023-10-06T10:44:29.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.343+0000 7f764b459700 1 -- start start 2023-10-06T10:44:29.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.343+0000 7f764b459700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7644159560 0x7f76440aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:29.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.343+0000 7f764b459700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f76440b0760 0x7f76440b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:29.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.343+0000 7f764b459700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76440b0c60 con 0x7f76440b49a0 2023-10-06T10:44:29.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.343+0000 7f764b459700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76440b0f40 con 0x7f76440b0760 2023-10-06T10:44:29.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.343+0000 7f764b459700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76440b1220 con 0x7f7644159560 2023-10-06T10:44:29.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.344+0000 7f76499f6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f76440b0760 0x7f76440b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:29.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.344+0000 7f76489f4700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f76440b49a0 0x7f76440aaa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44190/0 (socket says 172.21.15.92:44190) 2023-10-06T10:44:29.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.344+0000 7f76489f4700 1 -- 172.21.15.92:0/4231501922 learned_addr learned my addr 172.21.15.92:0/4231501922 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:29.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.344+0000 7f76491f5700 1 --2- 172.21.15.92:0/4231501922 >> v2:172.21.15.124:3300/0 conn(0x7f7644159560 0x7f76440aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:29.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.344+0000 7f76499f6700 1 -- 172.21.15.92:0/4231501922 >> v2:172.21.15.124:3300/0 conn(0x7f7644159560 msgr2=0x7f76440aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:29.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.344+0000 7f76499f6700 1 --2- 172.21.15.92:0/4231501922 >> v2:172.21.15.124:3300/0 conn(0x7f7644159560 0x7f76440aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:29.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.344+0000 7f76499f6700 1 -- 172.21.15.92:0/4231501922 >> v1:172.21.15.92:6789/0 conn(0x7f76440b49a0 legacy=0x7f76440aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:29.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.344+0000 7f76499f6700 1 -- 172.21.15.92:0/4231501922 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76440b1500 con 0x7f76440b0760 2023-10-06T10:44:29.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.344+0000 7f763bfff700 1 -- 172.21.15.92:0/4231501922 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105965767 0 0) 0x7f76440b0c60 con 0x7f76440b49a0 2023-10-06T10:44:29.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.344+0000 7f76499f6700 1 --2- 172.21.15.92:0/4231501922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76440b0760 0x7f76440b1bc0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f763c009e60 tx=0x7f763c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2daa35d8ccc24dde server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:29.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.344+0000 7f763bfff700 1 -- 172.21.15.92:0/4231501922 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c00e070 con 0x7f76440b0760 2023-10-06T10:44:29.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.344+0000 7f763bfff700 1 -- 172.21.15.92:0/4231501922 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763c00bbf0 con 0x7f76440b0760 2023-10-06T10:44:29.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.344+0000 7f763bfff700 1 -- 172.21.15.92:0/4231501922 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c012430 con 0x7f76440b0760 2023-10-06T10:44:29.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.345+0000 7f764b459700 1 -- 172.21.15.92:0/4231501922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76440b0760 msgr2=0x7f76440b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:29.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.345+0000 7f764b459700 1 --2- 172.21.15.92:0/4231501922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76440b0760 0x7f76440b1bc0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f763c009e60 tx=0x7f763c009780 comp rx=0 tx=0).stop 2023-10-06T10:44:29.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.345+0000 7f764b459700 1 -- 172.21.15.92:0/4231501922 shutdown_connections 2023-10-06T10:44:29.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.345+0000 7f764b459700 1 --2- 172.21.15.92:0/4231501922 >> v2:172.21.15.124:3300/0 conn(0x7f7644159560 0x7f76440aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:29.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.345+0000 7f764b459700 1 --2- 172.21.15.92:0/4231501922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76440b0760 0x7f76440b1bc0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:29.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.345+0000 7f764b459700 1 -- 172.21.15.92:0/4231501922 >> 172.21.15.92:0/4231501922 conn(0x7f76440997f0 msgr2=0x7f764406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:29.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.345+0000 7f764b459700 1 -- 172.21.15.92:0/4231501922 shutdown_connections 2023-10-06T10:44:29.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.345+0000 7f764b459700 1 -- 172.21.15.92:0/4231501922 wait complete. 2023-10-06T10:44:29.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.346+0000 7f764b459700 1 Processor -- start 2023-10-06T10:44:29.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.346+0000 7f764b459700 1 -- start start 2023-10-06T10:44:29.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.347+0000 7f764b459700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76440b0760 0x7f7644158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:29.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.347+0000 7f764b459700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76440b49a0 0x7f764415ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:29.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.347+0000 7f764b459700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7644159560 0x7f764409a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:29.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.347+0000 7f764b459700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7644169780 con 0x7f7644159560 2023-10-06T10:44:29.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.347+0000 7f764b459700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764415aad0 con 0x7f76440b49a0 2023-10-06T10:44:29.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.347+0000 7f764b459700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764415adb0 con 0x7f76440b0760 2023-10-06T10:44:29.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.347+0000 7f76499f6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7644159560 0x7f764409a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:29.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.347+0000 7f76499f6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7644159560 0x7f764409a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37748/0 (socket says 172.21.15.92:37748) 2023-10-06T10:44:29.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.347+0000 7f76499f6700 1 -- 172.21.15.92:0/4026309346 learned_addr learned my addr 172.21.15.92:0/4026309346 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:29.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.347+0000 7f76489f4700 1 --2- 172.21.15.92:0/4026309346 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76440b49a0 0x7f764415ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:29.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.347+0000 7f76491f5700 1 --2- 172.21.15.92:0/4026309346 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76440b0760 0x7f7644158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:29.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.347+0000 7f76499f6700 1 -- 172.21.15.92:0/4026309346 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76440b0760 msgr2=0x7f7644158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:29.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.347+0000 7f76499f6700 1 --2- 172.21.15.92:0/4026309346 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76440b0760 0x7f7644158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:29.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.347+0000 7f76499f6700 1 -- 172.21.15.92:0/4026309346 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76440b49a0 msgr2=0x7f764415ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:29.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.347+0000 7f76499f6700 1 --2- 172.21.15.92:0/4026309346 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76440b49a0 0x7f764415ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:29.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.347+0000 7f76499f6700 1 -- 172.21.15.92:0/4026309346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c009a20 con 0x7f7644159560 2023-10-06T10:44:29.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.348+0000 7f76499f6700 1 --2- 172.21.15.92:0/4026309346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7644159560 0x7f764409a520 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f763c00b6c0 tx=0x7f763c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:29.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.348+0000 7f76397fa700 1 -- 172.21.15.92:0/4026309346 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c027470 con 0x7f7644159560 2023-10-06T10:44:29.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.348+0000 7f76397fa700 1 -- 172.21.15.92:0/4026309346 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763c0275d0 con 0x7f7644159560 2023-10-06T10:44:29.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.348+0000 7f764b459700 1 -- 172.21.15.92:0/4026309346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764415b090 con 0x7f7644159560 2023-10-06T10:44:29.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.348+0000 7f76397fa700 1 -- 172.21.15.92:0/4026309346 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c027970 con 0x7f7644159560 2023-10-06T10:44:29.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.348+0000 7f764b459700 1 -- 172.21.15.92:0/4026309346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764415cfd0 con 0x7f7644159560 2023-10-06T10:44:29.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.349+0000 7f764b459700 1 -- 172.21.15.92:0/4026309346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76441698d0 con 0x7f7644159560 2023-10-06T10:44:29.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.349+0000 7f76397fa700 1 -- 172.21.15.92:0/4026309346 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f763c024070 con 0x7f7644159560 2023-10-06T10:44:29.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.349+0000 7f76397fa700 1 --2- 172.21.15.92:0/4026309346 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7628066f30 0x7f76280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:29.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.349+0000 7f76397fa700 1 -- 172.21.15.92:0/4026309346 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f763c0872c0 con 0x7f7644159560 2023-10-06T10:44:29.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.349+0000 7f76491f5700 1 --2- 172.21.15.92:0/4026309346 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7628066f30 0x7f76280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:29.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.350+0000 7f76491f5700 1 --2- 172.21.15.92:0/4026309346 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7628066f30 0x7f76280693e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f76400061f0 tx=0x7f7640008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:29.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.354+0000 7f76397fa700 1 -- 172.21.15.92:0/4026309346 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f76441698d0 con 0x7f7644159560 2023-10-06T10:44:29.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.577+0000 7f764b459700 1 -- 172.21.15.92:0/4026309346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76441698d0 con 0x7f7644159560 2023-10-06T10:44:29.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.577+0000 7f76397fa700 1 -- 172.21.15.92:0/4026309346 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4642 (secure 0 0 0) 0x7f76441698d0 con 0x7f7644159560 2023-10-06T10:44:29.579 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:29.579 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2122441982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2122441982},{"type":"v1","addr":"172.21.15.124:6835","nonce":2122441982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:47.850832+0000","modified":"2023-10-06T10:44:20.587653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7701,"mds_1":7689,"mds_2":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6836/3605297930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3605297930},{"type":"v1","addr":"172.21.15.92:6836","nonce":3605297930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7689":{"gid":7689,"name":"c","rank":1,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.92:6837/4047304116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4047304116},{"type":"v1","addr":"172.21.15.92:6837","nonce":4047304116}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7701":{"gid":7701,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":11,"addr":"172.21.15.124:6834/4203118992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":4203118992},{"type":"v1","addr":"172.21.15.124:6834","nonce":4203118992}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T10:44:29.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.579+0000 7f764b459700 1 -- 172.21.15.92:0/4026309346 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7628066f30 msgr2=0x7f76280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:29.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.579+0000 7f764b459700 1 --2- 172.21.15.92:0/4026309346 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7628066f30 0x7f76280693e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f76400061f0 tx=0x7f7640008040 comp rx=0 tx=0).stop 2023-10-06T10:44:29.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.579+0000 7f764b459700 1 -- 172.21.15.92:0/4026309346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7644159560 msgr2=0x7f764409a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:29.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.579+0000 7f764b459700 1 --2- 172.21.15.92:0/4026309346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7644159560 0x7f764409a520 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f763c00b6c0 tx=0x7f763c019000 comp rx=0 tx=0).stop 2023-10-06T10:44:29.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.579+0000 7f764b459700 1 -- 172.21.15.92:0/4026309346 shutdown_connections 2023-10-06T10:44:29.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.579+0000 7f764b459700 1 --2- 172.21.15.92:0/4026309346 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7628066f30 0x7f76280693e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:29.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.579+0000 7f764b459700 1 --2- 172.21.15.92:0/4026309346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7644159560 0x7f764409a520 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:29.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.579+0000 7f764b459700 1 --2- 172.21.15.92:0/4026309346 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76440b0760 0x7f7644158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:29.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.579+0000 7f764b459700 1 --2- 172.21.15.92:0/4026309346 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76440b49a0 0x7f764415ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:29.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.579+0000 7f764b459700 1 -- 172.21.15.92:0/4026309346 >> 172.21.15.92:0/4026309346 conn(0x7f76440997f0 msgr2=0x7f76440b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:29.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.580+0000 7f764b459700 1 -- 172.21.15.92:0/4026309346 shutdown_connections 2023-10-06T10:44:29.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.580+0000 7f764b459700 1 -- 172.21.15.92:0/4026309346 wait complete. 2023-10-06T10:44:29.582 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 92 2023-10-06T10:44:29.592 DEBUG:tasks.cephfs.test_multimds_misc:current active=['b', 'c', 'a'] 2023-10-06T10:44:29.592 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:44:29.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.705+0000 7fc1050cb700 1 Processor -- start 2023-10-06T10:44:29.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.705+0000 7fc1050cb700 1 -- start start 2023-10-06T10:44:29.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.706+0000 7fc1050cb700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc100159560 0x7fc1000aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:29.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.706+0000 7fc1050cb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc1000b0760 0x7fc1000b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:29.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.706+0000 7fc1050cb700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1000b0c60 con 0x7fc1000b49a0 2023-10-06T10:44:29.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.706+0000 7fc1050cb700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1000b0f40 con 0x7fc1000b0760 2023-10-06T10:44:29.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.706+0000 7fc1050cb700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1000b1220 con 0x7fc100159560 2023-10-06T10:44:29.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.706+0000 7fc0ff59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc1000b0760 0x7fc1000b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:29.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.706+0000 7fc0f659c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc1000b49a0 0x7fc1000aaa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44194/0 (socket says 172.21.15.92:44194) 2023-10-06T10:44:29.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.706+0000 7fc0f659c700 1 -- 172.21.15.92:0/3671761438 learned_addr learned my addr 172.21.15.92:0/3671761438 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:29.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.706+0000 7fc0fed9d700 1 --2- 172.21.15.92:0/3671761438 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc100159560 0x7fc1000aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:29.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.706+0000 7fc0ff59e700 1 -- 172.21.15.92:0/3671761438 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc100159560 msgr2=0x7fc1000aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:29.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.706+0000 7fc0ff59e700 1 --2- 172.21.15.92:0/3671761438 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc100159560 0x7fc1000aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:29.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.706+0000 7fc0ff59e700 1 -- 172.21.15.92:0/3671761438 >> v1:172.21.15.92:6789/0 conn(0x7fc1000b49a0 legacy=0x7fc1000aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:29.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.706+0000 7fc0fe59c700 1 -- 172.21.15.92:0/3671761438 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 557261040 0 0) 0x7fc1000b0c60 con 0x7fc1000b49a0 2023-10-06T10:44:29.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.706+0000 7fc0ff59e700 1 -- 172.21.15.92:0/3671761438 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1000b1500 con 0x7fc1000b0760 2023-10-06T10:44:29.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.707+0000 7fc0ff59e700 1 --2- 172.21.15.92:0/3671761438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1000b0760 0x7fc1000b1bc0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009e60 tx=0x7fc0f8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349063d494d35224 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:29.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.707+0000 7fc0fe59c700 1 -- 172.21.15.92:0/3671761438 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f800e070 con 0x7fc1000b0760 2023-10-06T10:44:29.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.707+0000 7fc0fe59c700 1 -- 172.21.15.92:0/3671761438 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f800bc30 con 0x7fc1000b0760 2023-10-06T10:44:29.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.707+0000 7fc0fe59c700 1 -- 172.21.15.92:0/3671761438 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8012430 con 0x7fc1000b0760 2023-10-06T10:44:29.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.707+0000 7fc1050cb700 1 -- 172.21.15.92:0/3671761438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1000b0760 msgr2=0x7fc1000b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:29.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.707+0000 7fc1050cb700 1 --2- 172.21.15.92:0/3671761438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1000b0760 0x7fc1000b1bc0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009e60 tx=0x7fc0f8009780 comp rx=0 tx=0).stop 2023-10-06T10:44:29.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.707+0000 7fc1050cb700 1 -- 172.21.15.92:0/3671761438 shutdown_connections 2023-10-06T10:44:29.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.707+0000 7fc1050cb700 1 --2- 172.21.15.92:0/3671761438 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc100159560 0x7fc1000aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:29.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.707+0000 7fc1050cb700 1 --2- 172.21.15.92:0/3671761438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1000b0760 0x7fc1000b1bc0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:29.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.707+0000 7fc1050cb700 1 -- 172.21.15.92:0/3671761438 >> 172.21.15.92:0/3671761438 conn(0x7fc1000997f0 msgr2=0x7fc10006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:29.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.707+0000 7fc1050cb700 1 -- 172.21.15.92:0/3671761438 shutdown_connections 2023-10-06T10:44:29.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.707+0000 7fc1050cb700 1 -- 172.21.15.92:0/3671761438 wait complete. 2023-10-06T10:44:29.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.708+0000 7fc1050cb700 1 Processor -- start 2023-10-06T10:44:29.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.708+0000 7fc1050cb700 1 -- start start 2023-10-06T10:44:29.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.709+0000 7fc1050cb700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1000b0760 0x7fc1000acb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:29.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.709+0000 7fc1050cb700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1000b49a0 0x7fc10016db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:29.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.709+0000 7fc1050cb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc100159560 0x7fc10009d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:29.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.709+0000 7fc1050cb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10016e060 con 0x7fc100159560 2023-10-06T10:44:29.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.709+0000 7fc1050cb700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1000ad200 con 0x7fc1000b49a0 2023-10-06T10:44:29.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.709+0000 7fc1050cb700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1000ad4e0 con 0x7fc1000b0760 2023-10-06T10:44:29.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.709+0000 7fc0ff59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc100159560 0x7fc10009d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:29.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.709+0000 7fc0ff59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc100159560 0x7fc10009d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37770/0 (socket says 172.21.15.92:37770) 2023-10-06T10:44:29.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.709+0000 7fc0ff59e700 1 -- 172.21.15.92:0/774956488 learned_addr learned my addr 172.21.15.92:0/774956488 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:29.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.709+0000 7fc0f659c700 1 --2- 172.21.15.92:0/774956488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1000b49a0 0x7fc10016db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:29.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.709+0000 7fc0ff59e700 1 -- 172.21.15.92:0/774956488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1000b0760 msgr2=0x7fc1000acb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:29.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.709+0000 7fc0ff59e700 1 --2- 172.21.15.92:0/774956488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1000b0760 0x7fc1000acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:29.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.709+0000 7fc0ff59e700 1 -- 172.21.15.92:0/774956488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1000b49a0 msgr2=0x7fc10016db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:29.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.709+0000 7fc0ff59e700 1 --2- 172.21.15.92:0/774956488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1000b49a0 0x7fc10016db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:29.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.709+0000 7fc0ff59e700 1 -- 172.21.15.92:0/774956488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f8009a20 con 0x7fc100159560 2023-10-06T10:44:29.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.709+0000 7fc0f659c700 1 --2- 172.21.15.92:0/774956488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1000b49a0 0x7fc10016db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:29.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.710+0000 7fc0ff59e700 1 --2- 172.21.15.92:0/774956488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc100159560 0x7fc10009d1f0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fc100067c20 tx=0x7fc0f8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:29.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.710+0000 7fc0f77fe700 1 -- 172.21.15.92:0/774956488 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f801d630 con 0x7fc100159560 2023-10-06T10:44:29.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.710+0000 7fc0f77fe700 1 -- 172.21.15.92:0/774956488 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f801d7b0 con 0x7fc100159560 2023-10-06T10:44:29.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.710+0000 7fc1050cb700 1 -- 172.21.15.92:0/774956488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1000ad7c0 con 0x7fc100159560 2023-10-06T10:44:29.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.710+0000 7fc0f77fe700 1 -- 172.21.15.92:0/774956488 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f801db70 con 0x7fc100159560 2023-10-06T10:44:29.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.710+0000 7fc1050cb700 1 -- 172.21.15.92:0/774956488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1000add80 con 0x7fc100159560 2023-10-06T10:44:29.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.711+0000 7fc1050cb700 1 -- 172.21.15.92:0/774956488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1000adfa0 con 0x7fc100159560 2023-10-06T10:44:29.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.711+0000 7fc0f77fe700 1 -- 172.21.15.92:0/774956488 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc0f8024070 con 0x7fc100159560 2023-10-06T10:44:29.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.712+0000 7fc0f77fe700 1 --2- 172.21.15.92:0/774956488 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc0e0066f30 0x7fc0e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:29.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.712+0000 7fc0fed9d700 1 --2- 172.21.15.92:0/774956488 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc0e0066f30 0x7fc0e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:29.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.712+0000 7fc0f77fe700 1 -- 172.21.15.92:0/774956488 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc0f80871b0 con 0x7fc100159560 2023-10-06T10:44:29.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.712+0000 7fc0fed9d700 1 --2- 172.21.15.92:0/774956488 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc0e0066f30 0x7fc0e00693e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0006fd0 tx=0x7fc0f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:29.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.716+0000 7fc0f77fe700 1 -- 172.21.15.92:0/774956488 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0f8054180 con 0x7fc100159560 2023-10-06T10:44:29.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.974+0000 7fc1050cb700 1 -- 172.21.15.92:0/774956488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc100064a90 con 0x7fc100159560 2023-10-06T10:44:29.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.975+0000 7fc0f77fe700 1 -- 172.21.15.92:0/774956488 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1921 (secure 0 0 0) 0x7fc0f80577a0 con 0x7fc100159560 2023-10-06T10:44:29.978 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:29.979 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":391,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":704416,"bytes_used":2445824000,"bytes_avail":770648289280,"bytes_total":773094113280,"write_bytes_sec":938,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":92,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"a","status":"up:active","gid":7686},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7689},{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":7701}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2023-10-06T10:44:28.163210+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T10:44:29.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.977+0000 7fc1050cb700 1 -- 172.21.15.92:0/774956488 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc0e0066f30 msgr2=0x7fc0e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:29.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.977+0000 7fc1050cb700 1 --2- 172.21.15.92:0/774956488 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc0e0066f30 0x7fc0e00693e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0006fd0 tx=0x7fc0f0008040 comp rx=0 tx=0).stop 2023-10-06T10:44:29.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.977+0000 7fc1050cb700 1 -- 172.21.15.92:0/774956488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc100159560 msgr2=0x7fc10009d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:29.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.977+0000 7fc1050cb700 1 --2- 172.21.15.92:0/774956488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc100159560 0x7fc10009d1f0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fc100067c20 tx=0x7fc0f8019000 comp rx=0 tx=0).stop 2023-10-06T10:44:29.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.977+0000 7fc1050cb700 1 -- 172.21.15.92:0/774956488 shutdown_connections 2023-10-06T10:44:29.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.977+0000 7fc1050cb700 1 --2- 172.21.15.92:0/774956488 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc0e0066f30 0x7fc0e00693e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:29.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.977+0000 7fc1050cb700 1 --2- 172.21.15.92:0/774956488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc100159560 0x7fc10009d1f0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:29.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.977+0000 7fc1050cb700 1 --2- 172.21.15.92:0/774956488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1000b0760 0x7fc1000acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:29.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.977+0000 7fc1050cb700 1 --2- 172.21.15.92:0/774956488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1000b49a0 0x7fc10016db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:29.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.977+0000 7fc1050cb700 1 -- 172.21.15.92:0/774956488 >> 172.21.15.92:0/774956488 conn(0x7fc1000997f0 msgr2=0x7fc1000b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:29.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.977+0000 7fc1050cb700 1 -- 172.21.15.92:0/774956488 shutdown_connections 2023-10-06T10:44:29.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:29.977+0000 7fc1050cb700 1 -- 172.21.15.92:0/774956488 wait complete. 2023-10-06T10:44:29.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'idle+waiting paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]'} 2023-10-06T10:44:30.990 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:44:31.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.107+0000 7f6197137700 1 Processor -- start 2023-10-06T10:44:31.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.107+0000 7f6197137700 1 -- start start 2023-10-06T10:44:31.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.107+0000 7f6197137700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6190159560 0x7f61900aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:31.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.107+0000 7f6197137700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f61900b49a0 0x7f61900aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:31.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.108+0000 7f6197137700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61900b0c60 con 0x7f61900b0760 2023-10-06T10:44:31.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.108+0000 7f6197137700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61900b0f40 con 0x7f61900b49a0 2023-10-06T10:44:31.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.108+0000 7f6197137700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61900b1220 con 0x7f6190159560 2023-10-06T10:44:31.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.108+0000 7f6187fff700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f61900b49a0 0x7f61900aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:31.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.108+0000 7f6194ed3700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6190159560 0x7f61900aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:31.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.108+0000 7f6187fff700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f61900b49a0 0x7f61900aaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:50292/0 (socket says 172.21.15.92:50292) 2023-10-06T10:44:31.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.108+0000 7f6187fff700 1 -- 172.21.15.92:0/3562193605 learned_addr learned my addr 172.21.15.92:0/3562193605 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:31.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.108+0000 7f6187fff700 1 -- 172.21.15.92:0/3562193605 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6190159560 msgr2=0x7f61900aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:31.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.108+0000 7f6187fff700 1 --2- 172.21.15.92:0/3562193605 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6190159560 0x7f61900aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:31.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.108+0000 7f6187fff700 1 -- 172.21.15.92:0/3562193605 >> v1:172.21.15.124:6789/0 conn(0x7f61900b0760 legacy=0x7f61900b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:31.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.108+0000 7f6187fff700 1 -- 172.21.15.92:0/3562193605 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61900b1500 con 0x7f61900b49a0 2023-10-06T10:44:31.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.108+0000 7f6194ed3700 1 --2- 172.21.15.92:0/3562193605 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6190159560 0x7f61900aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:44:31.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.109+0000 7f6187fff700 1 --2- 172.21.15.92:0/3562193605 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61900b49a0 0x7f61900aaa70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f61800099b0 tx=0x7f6180009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94f52a508387d11b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:31.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.109+0000 7f61877fe700 1 -- 172.21.15.92:0/3562193605 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6180009ea0 con 0x7f61900b49a0 2023-10-06T10:44:31.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.109+0000 7f61877fe700 1 -- 172.21.15.92:0/3562193605 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618000b860 con 0x7f61900b49a0 2023-10-06T10:44:31.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.109+0000 7f61877fe700 1 -- 172.21.15.92:0/3562193605 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618000bc00 con 0x7f61900b49a0 2023-10-06T10:44:31.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.109+0000 7f6197137700 1 -- 172.21.15.92:0/3562193605 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61900b49a0 msgr2=0x7f61900aaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:31.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.109+0000 7f6197137700 1 --2- 172.21.15.92:0/3562193605 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61900b49a0 0x7f61900aaa70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f61800099b0 tx=0x7f6180009250 comp rx=0 tx=0).stop 2023-10-06T10:44:31.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.109+0000 7f6197137700 1 -- 172.21.15.92:0/3562193605 shutdown_connections 2023-10-06T10:44:31.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.109+0000 7f6197137700 1 --2- 172.21.15.92:0/3562193605 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6190159560 0x7f61900aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:31.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.109+0000 7f6197137700 1 --2- 172.21.15.92:0/3562193605 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61900b49a0 0x7f61900aaa70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:31.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.109+0000 7f6197137700 1 -- 172.21.15.92:0/3562193605 >> 172.21.15.92:0/3562193605 conn(0x7f61900997f0 msgr2=0x7f619006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:31.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.109+0000 7f6197137700 1 -- 172.21.15.92:0/3562193605 shutdown_connections 2023-10-06T10:44:31.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.109+0000 7f6197137700 1 -- 172.21.15.92:0/3562193605 wait complete. 2023-10-06T10:44:31.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.110+0000 7f6197137700 1 Processor -- start 2023-10-06T10:44:31.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.111+0000 7f6197137700 1 -- start start 2023-10-06T10:44:31.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.111+0000 7f6197137700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61900b0760 0x7f61901579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:31.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.111+0000 7f6197137700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61900b49a0 0x7f619009c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:31.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.111+0000 7f6197137700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6190159560 0x7f619009cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:31.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.111+0000 7f6197137700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6190157ee0 con 0x7f61900b0760 2023-10-06T10:44:31.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.111+0000 7f6197137700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61900cbce0 con 0x7f6190159560 2023-10-06T10:44:31.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.111+0000 7f6197137700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61900cc170 con 0x7f61900b49a0 2023-10-06T10:44:31.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.111+0000 7f6194ed3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61900b0760 0x7f61901579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:31.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.111+0000 7f6194ed3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61900b0760 0x7f61901579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37780/0 (socket says 172.21.15.92:37780) 2023-10-06T10:44:31.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.111+0000 7f6194ed3700 1 -- 172.21.15.92:0/3397897550 learned_addr learned my addr 172.21.15.92:0/3397897550 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:31.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.112+0000 7f6187fff700 1 --2- 172.21.15.92:0/3397897550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61900b49a0 0x7f619009c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:31.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.112+0000 7f61956d4700 1 --2- 172.21.15.92:0/3397897550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6190159560 0x7f619009cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:31.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.112+0000 7f6194ed3700 1 -- 172.21.15.92:0/3397897550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61900b49a0 msgr2=0x7f619009c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:31.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.112+0000 7f6194ed3700 1 --2- 172.21.15.92:0/3397897550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61900b49a0 0x7f619009c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:31.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.112+0000 7f6194ed3700 1 -- 172.21.15.92:0/3397897550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6190159560 msgr2=0x7f619009cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:31.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.112+0000 7f6194ed3700 1 --2- 172.21.15.92:0/3397897550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6190159560 0x7f619009cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.112+0000 7f6194ed3700 1 -- 172.21.15.92:0/3397897550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c0093c0 con 0x7f61900b0760 2023-10-06T10:44:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.112+0000 7f61956d4700 1 --2- 172.21.15.92:0/3397897550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6190159560 0x7f619009cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.112+0000 7f6194ed3700 1 --2- 172.21.15.92:0/3397897550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61900b0760 0x7f61901579a0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f618c00ec80 tx=0x7f618c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.112+0000 7f6184ff9700 1 -- 172.21.15.92:0/3397897550 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c005a00 con 0x7f61900b0760 2023-10-06T10:44:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.112+0000 7f6184ff9700 1 -- 172.21.15.92:0/3397897550 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618c005b60 con 0x7f61900b0760 2023-10-06T10:44:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.112+0000 7f6197137700 1 -- 172.21.15.92:0/3397897550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61800094f0 con 0x7f61900b0760 2023-10-06T10:44:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.112+0000 7f6184ff9700 1 -- 172.21.15.92:0/3397897550 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c00c920 con 0x7f61900b0760 2023-10-06T10:44:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.112+0000 7f6197137700 1 -- 172.21.15.92:0/3397897550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61900cb690 con 0x7f61900b0760 2023-10-06T10:44:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.113+0000 7f6197137700 1 -- 172.21.15.92:0/3397897550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61900cbfc0 con 0x7f61900b0760 2023-10-06T10:44:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.113+0000 7f6184ff9700 1 -- 172.21.15.92:0/3397897550 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f618c00cc90 con 0x7f61900b0760 2023-10-06T10:44:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.114+0000 7f6184ff9700 1 --2- 172.21.15.92:0/3397897550 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f617c066f30 0x7f617c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:31.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.114+0000 7f6187fff700 1 --2- 172.21.15.92:0/3397897550 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f617c066f30 0x7f617c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:31.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.114+0000 7f6184ff9700 1 -- 172.21.15.92:0/3397897550 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f618c084d70 con 0x7f61900b0760 2023-10-06T10:44:31.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.114+0000 7f6187fff700 1 --2- 172.21.15.92:0/3397897550 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f617c066f30 0x7f617c0693e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f61800099b0 tx=0x7f6180017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:31.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.118+0000 7f6184ff9700 1 -- 172.21.15.92:0/3397897550 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f618c051d40 con 0x7f61900b0760 2023-10-06T10:44:31.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.344+0000 7f6197137700 1 -- 172.21.15.92:0/3397897550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6190064a90 con 0x7f61900b0760 2023-10-06T10:44:31.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.344+0000 7f6184ff9700 1 -- 172.21.15.92:0/3397897550 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4642 (secure 0 0 0) 0x7f618c055360 con 0x7f61900b0760 2023-10-06T10:44:31.345 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:31.345 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2122441982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2122441982},{"type":"v1","addr":"172.21.15.124:6835","nonce":2122441982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:47.850832+0000","modified":"2023-10-06T10:44:20.587653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7701,"mds_1":7689,"mds_2":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6836/3605297930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3605297930},{"type":"v1","addr":"172.21.15.92:6836","nonce":3605297930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7689":{"gid":7689,"name":"c","rank":1,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.92:6837/4047304116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4047304116},{"type":"v1","addr":"172.21.15.92:6837","nonce":4047304116}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7701":{"gid":7701,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":11,"addr":"172.21.15.124:6834/4203118992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":4203118992},{"type":"v1","addr":"172.21.15.124:6834","nonce":4203118992}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T10:44:31.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.346+0000 7f6197137700 1 -- 172.21.15.92:0/3397897550 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f617c066f30 msgr2=0x7f617c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:31.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.346+0000 7f6197137700 1 --2- 172.21.15.92:0/3397897550 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f617c066f30 0x7f617c0693e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f61800099b0 tx=0x7f6180017000 comp rx=0 tx=0).stop 2023-10-06T10:44:31.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.346+0000 7f6197137700 1 -- 172.21.15.92:0/3397897550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61900b0760 msgr2=0x7f61901579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:31.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.346+0000 7f6197137700 1 --2- 172.21.15.92:0/3397897550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61900b0760 0x7f61901579a0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f618c00ec80 tx=0x7f618c009120 comp rx=0 tx=0).stop 2023-10-06T10:44:31.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.346+0000 7f6197137700 1 -- 172.21.15.92:0/3397897550 shutdown_connections 2023-10-06T10:44:31.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.346+0000 7f6197137700 1 --2- 172.21.15.92:0/3397897550 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f617c066f30 0x7f617c0693e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:31.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.346+0000 7f6197137700 1 --2- 172.21.15.92:0/3397897550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6190159560 0x7f619009cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:31.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.346+0000 7f6197137700 1 --2- 172.21.15.92:0/3397897550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61900b0760 0x7f61901579a0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:31.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.346+0000 7f6197137700 1 --2- 172.21.15.92:0/3397897550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61900b49a0 0x7f619009c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:31.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.346+0000 7f6197137700 1 -- 172.21.15.92:0/3397897550 >> 172.21.15.92:0/3397897550 conn(0x7f61900997f0 msgr2=0x7f61900d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:31.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.346+0000 7f6197137700 1 -- 172.21.15.92:0/3397897550 shutdown_connections 2023-10-06T10:44:31.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.346+0000 7f6197137700 1 -- 172.21.15.92:0/3397897550 wait complete. 2023-10-06T10:44:31.349 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 92 2023-10-06T10:44:31.359 DEBUG:tasks.cephfs.test_multimds_misc:current active=['b', 'c', 'a'] 2023-10-06T10:44:31.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:44:31.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.471+0000 7fefd359e700 1 Processor -- start 2023-10-06T10:44:31.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.471+0000 7fefd359e700 1 -- start start 2023-10-06T10:44:31.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.472+0000 7fefd359e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fefd4159440 0x7fefd40b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:31.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.472+0000 7fefd359e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fefd40b2100 0x7fefd40b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:31.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.472+0000 7fefd359e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd40cb7a0 con 0x7fefd40b35d0 2023-10-06T10:44:31.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.472+0000 7fefd359e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd40b0860 con 0x7fefd40b2100 2023-10-06T10:44:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.472+0000 7fefd359e700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd40cb150 con 0x7fefd4159440 2023-10-06T10:44:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.472+0000 7fefd2d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fefd40b2100 0x7fefd40b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.472+0000 7fefd2d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fefd40b2100 0x7fefd40b2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37784/0 (socket says 172.21.15.92:37784) 2023-10-06T10:44:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.472+0000 7fefd2d9d700 1 -- 172.21.15.92:0/3261788107 learned_addr learned my addr 172.21.15.92:0/3261788107 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.472+0000 7fefd259c700 1 --2- 172.21.15.92:0/3261788107 >> v2:172.21.15.124:3300/0 conn(0x7fefd4159440 0x7fefd40b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.472+0000 7fefd2d9d700 1 -- 172.21.15.92:0/3261788107 >> v2:172.21.15.124:3300/0 conn(0x7fefd4159440 msgr2=0x7fefd40b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.472+0000 7fefd2d9d700 1 --2- 172.21.15.92:0/3261788107 >> v2:172.21.15.124:3300/0 conn(0x7fefd4159440 0x7fefd40b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.472+0000 7fefd2d9d700 1 -- 172.21.15.92:0/3261788107 >> v1:172.21.15.124:6789/0 conn(0x7fefd40b35d0 legacy=0x7fefd40cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.472+0000 7fefd2d9d700 1 -- 172.21.15.92:0/3261788107 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd40cb430 con 0x7fefd40b2100 2023-10-06T10:44:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.473+0000 7fefd2d9d700 1 --2- 172.21.15.92:0/3261788107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefd40b2100 0x7fefd40b2520 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fefd406f9f0 tx=0x7fefcc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8d94a213fd31f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.473+0000 7fefd1d9b700 1 -- 172.21.15.92:0/3261788107 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc00e070 con 0x7fefd40b2100 2023-10-06T10:44:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.473+0000 7fefd1d9b700 1 -- 172.21.15.92:0/3261788107 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefcc00ba20 con 0x7fefd40b2100 2023-10-06T10:44:31.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.473+0000 7fefd1d9b700 1 -- 172.21.15.92:0/3261788107 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc0092e0 con 0x7fefd40b2100 2023-10-06T10:44:31.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.473+0000 7fefd359e700 1 -- 172.21.15.92:0/3261788107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefd40b2100 msgr2=0x7fefd40b2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:31.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.473+0000 7fefd359e700 1 --2- 172.21.15.92:0/3261788107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefd40b2100 0x7fefd40b2520 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fefd406f9f0 tx=0x7fefcc009840 comp rx=0 tx=0).stop 2023-10-06T10:44:31.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.473+0000 7fefd359e700 1 -- 172.21.15.92:0/3261788107 shutdown_connections 2023-10-06T10:44:31.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.473+0000 7fefd359e700 1 --2- 172.21.15.92:0/3261788107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefd40b2100 0x7fefd40b2520 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:31.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.473+0000 7fefd359e700 1 --2- 172.21.15.92:0/3261788107 >> v2:172.21.15.124:3300/0 conn(0x7fefd4159440 0x7fefd40b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:31.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.473+0000 7fefd359e700 1 -- 172.21.15.92:0/3261788107 >> 172.21.15.92:0/3261788107 conn(0x7fefd406ece0 msgr2=0x7fefd406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:31.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.473+0000 7fefd359e700 1 -- 172.21.15.92:0/3261788107 shutdown_connections 2023-10-06T10:44:31.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.473+0000 7fefd359e700 1 -- 172.21.15.92:0/3261788107 wait complete. 2023-10-06T10:44:31.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.474+0000 7fefd359e700 1 Processor -- start 2023-10-06T10:44:31.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.474+0000 7fefd359e700 1 -- start start 2023-10-06T10:44:31.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.475+0000 7fefd359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefd40b2100 0x7fefd40c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:31.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.475+0000 7fefd359e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fefd40b35d0 0x7fefd41547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:31.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.475+0000 7fefd359e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefd4159440 0x7fefd415db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:31.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.475+0000 7fefd359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd40c9890 con 0x7fefd40b2100 2023-10-06T10:44:31.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.475+0000 7fefd359e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd40c8c80 con 0x7fefd40b35d0 2023-10-06T10:44:31.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.475+0000 7fefd359e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd40c8f60 con 0x7fefd4159440 2023-10-06T10:44:31.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.475+0000 7fefd259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefd40b2100 0x7fefd40c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:31.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.475+0000 7fefd259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefd40b2100 0x7fefd40c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37800/0 (socket says 172.21.15.92:37800) 2023-10-06T10:44:31.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.475+0000 7fefd259c700 1 -- 172.21.15.92:0/2598463750 learned_addr learned my addr 172.21.15.92:0/2598463750 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:31.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.475+0000 7fefd2d9d700 1 --2- 172.21.15.92:0/2598463750 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefd4159440 0x7fefd415db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:31.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.475+0000 7fefc9d9b700 1 --2- 172.21.15.92:0/2598463750 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fefd40b35d0 0x7fefd41547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:31.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.476+0000 7fefd259c700 1 -- 172.21.15.92:0/2598463750 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefd4159440 msgr2=0x7fefd415db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:31.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.476+0000 7fefd259c700 1 --2- 172.21.15.92:0/2598463750 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefd4159440 0x7fefd415db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:31.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.476+0000 7fefd259c700 1 -- 172.21.15.92:0/2598463750 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fefd40b35d0 msgr2=0x7fefd41547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:31.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.476+0000 7fefd259c700 1 --2- 172.21.15.92:0/2598463750 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fefd40b35d0 0x7fefd41547a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:31.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.476+0000 7fefd259c700 1 -- 172.21.15.92:0/2598463750 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefcc009ae0 con 0x7fefd40b2100 2023-10-06T10:44:31.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.476+0000 7fefd259c700 1 --2- 172.21.15.92:0/2598463750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefd40b2100 0x7fefd40c9d20 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fefc400e930 tx=0x7fefc40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:31.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.476+0000 7fefcaffd700 1 -- 172.21.15.92:0/2598463750 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc400eed0 con 0x7fefd40b2100 2023-10-06T10:44:31.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.476+0000 7fefcaffd700 1 -- 172.21.15.92:0/2598463750 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc400c850 con 0x7fefd40b2100 2023-10-06T10:44:31.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.476+0000 7fefd359e700 1 -- 172.21.15.92:0/2598463750 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd40c9240 con 0x7fefd40b2100 2023-10-06T10:44:31.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.476+0000 7fefcaffd700 1 -- 172.21.15.92:0/2598463750 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc400cbf0 con 0x7fefd40b2100 2023-10-06T10:44:31.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.476+0000 7fefd359e700 1 -- 172.21.15.92:0/2598463750 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd4154ce0 con 0x7fefd40b2100 2023-10-06T10:44:31.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.477+0000 7fefd359e700 1 -- 172.21.15.92:0/2598463750 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd405f230 con 0x7fefd40b2100 2023-10-06T10:44:31.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.477+0000 7fefcaffd700 1 -- 172.21.15.92:0/2598463750 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fefc4013470 con 0x7fefd40b2100 2023-10-06T10:44:31.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.477+0000 7fefcaffd700 1 --2- 172.21.15.92:0/2598463750 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fefb4066f30 0x7fefb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:31.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.478+0000 7fefcaffd700 1 -- 172.21.15.92:0/2598463750 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fefc4084db0 con 0x7fefd40b2100 2023-10-06T10:44:31.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.478+0000 7fefc9d9b700 1 --2- 172.21.15.92:0/2598463750 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fefb4066f30 0x7fefb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:31.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.478+0000 7fefc9d9b700 1 --2- 172.21.15.92:0/2598463750 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fefb4066f30 0x7fefb40693e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fefd40ceae0 tx=0x7fefbc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:31.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.482+0000 7fefcaffd700 1 -- 172.21.15.92:0/2598463750 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fefc4051d80 con 0x7fefd40b2100 2023-10-06T10:44:31.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.737+0000 7fefd359e700 1 -- 172.21.15.92:0/2598463750 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fefd4159810 con 0x7fefd40b2100 2023-10-06T10:44:31.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.737+0000 7fefcaffd700 1 -- 172.21.15.92:0/2598463750 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1921 (secure 0 0 0) 0x7fefc40177a0 con 0x7fefd40b2100 2023-10-06T10:44:31.740 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:31.740 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":393,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":704416,"bytes_used":2445824000,"bytes_avail":770648289280,"bytes_total":773094113280,"write_bytes_sec":938,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":92,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"a","status":"up:active","gid":7686},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7689},{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":7701}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2023-10-06T10:44:28.163210+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T10:44:31.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.739+0000 7fefd359e700 1 -- 172.21.15.92:0/2598463750 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fefb4066f30 msgr2=0x7fefb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:31.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.739+0000 7fefd359e700 1 --2- 172.21.15.92:0/2598463750 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fefb4066f30 0x7fefb40693e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fefd40ceae0 tx=0x7fefbc009040 comp rx=0 tx=0).stop 2023-10-06T10:44:31.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.740+0000 7fefd359e700 1 -- 172.21.15.92:0/2598463750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefd40b2100 msgr2=0x7fefd40c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:31.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.740+0000 7fefd359e700 1 --2- 172.21.15.92:0/2598463750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefd40b2100 0x7fefd40c9d20 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fefc400e930 tx=0x7fefc40091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:31.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.740+0000 7fefd359e700 1 -- 172.21.15.92:0/2598463750 shutdown_connections 2023-10-06T10:44:31.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.740+0000 7fefd359e700 1 --2- 172.21.15.92:0/2598463750 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fefb4066f30 0x7fefb40693e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:31.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.740+0000 7fefd359e700 1 --2- 172.21.15.92:0/2598463750 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefd4159440 0x7fefd415db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:31.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.740+0000 7fefd359e700 1 --2- 172.21.15.92:0/2598463750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefd40b2100 0x7fefd40c9d20 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:31.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.740+0000 7fefd359e700 1 --2- 172.21.15.92:0/2598463750 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fefd40b35d0 0x7fefd41547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:31.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.740+0000 7fefd359e700 1 -- 172.21.15.92:0/2598463750 >> 172.21.15.92:0/2598463750 conn(0x7fefd406ece0 msgr2=0x7fefd4158680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:31.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.740+0000 7fefd359e700 1 -- 172.21.15.92:0/2598463750 shutdown_connections 2023-10-06T10:44:31.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:31.740+0000 7fefd359e700 1 -- 172.21.15.92:0/2598463750 wait complete. 2023-10-06T10:44:31.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'idle+waiting paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]'} 2023-10-06T10:44:32.754 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:44:32.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.874+0000 7f5ac0c14700 1 Processor -- start 2023-10-06T10:44:32.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.874+0000 7f5ac0c14700 1 -- start start 2023-10-06T10:44:32.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.875+0000 7f5ac0c14700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5abc0b25e0 0x7f5abc0b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:32.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.875+0000 7f5ac0c14700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5abc159830 0x7f5abc0cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:32.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.875+0000 7f5ac0c14700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5abc0cbfa0 con 0x7f5abc0b29b0 2023-10-06T10:44:32.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.875+0000 7f5ac0c14700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5abc0cb390 con 0x7f5abc159830 2023-10-06T10:44:32.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.875+0000 7f5ac0c14700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc0cb670 con 0x7f5abc0b25e0 2023-10-06T10:44:32.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.876+0000 7f5abad9d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5abc159830 0x7f5abc0cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:32.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.876+0000 7f5aba59c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5abc0b25e0 0x7f5abc0b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:32.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.876+0000 7f5abad9d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5abc159830 0x7f5abc0cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:50344/0 (socket says 172.21.15.92:50344) 2023-10-06T10:44:32.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.876+0000 7f5abad9d700 1 -- 172.21.15.92:0/4214110514 learned_addr learned my addr 172.21.15.92:0/4214110514 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:32.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.876+0000 7f5ab9d9b700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f5abc0b29b0 0x7f5abc159120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:37628/0 (socket says 172.21.15.92:37628) 2023-10-06T10:44:32.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.876+0000 7f5abad9d700 1 -- 172.21.15.92:0/4214110514 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5abc0b25e0 msgr2=0x7f5abc0b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:32.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.876+0000 7f5abad9d700 1 --2- 172.21.15.92:0/4214110514 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5abc0b25e0 0x7f5abc0b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:32.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.876+0000 7f5abad9d700 1 -- 172.21.15.92:0/4214110514 >> v1:172.21.15.124:6789/0 conn(0x7f5abc0b29b0 legacy=0x7f5abc159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:32.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.876+0000 7f5abad9d700 1 -- 172.21.15.92:0/4214110514 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc0cb950 con 0x7f5abc159830 2023-10-06T10:44:32.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.876+0000 7f5aba59c700 1 --2- 172.21.15.92:0/4214110514 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5abc0b25e0 0x7f5abc0b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:44:32.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.876+0000 7f5abad9d700 1 --2- 172.21.15.92:0/4214110514 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5abc159830 0x7f5abc0cc430 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40099b0 tx=0x7f5ab4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4936278be01fdd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:32.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.876+0000 7f5ab959a700 1 -- 172.21.15.92:0/4214110514 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4009ea0 con 0x7f5abc159830 2023-10-06T10:44:32.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.876+0000 7f5ab959a700 1 -- 172.21.15.92:0/4214110514 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab400b860 con 0x7f5abc159830 2023-10-06T10:44:32.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.876+0000 7f5ab959a700 1 -- 172.21.15.92:0/4214110514 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab400bc00 con 0x7f5abc159830 2023-10-06T10:44:32.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.877+0000 7f5ac0c14700 1 -- 172.21.15.92:0/4214110514 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5abc159830 msgr2=0x7f5abc0cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:32.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.877+0000 7f5ac0c14700 1 --2- 172.21.15.92:0/4214110514 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5abc159830 0x7f5abc0cc430 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40099b0 tx=0x7f5ab4009250 comp rx=0 tx=0).stop 2023-10-06T10:44:32.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.877+0000 7f5ac0c14700 1 -- 172.21.15.92:0/4214110514 shutdown_connections 2023-10-06T10:44:32.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.877+0000 7f5ac0c14700 1 --2- 172.21.15.92:0/4214110514 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5abc159830 0x7f5abc0cc430 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:32.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.877+0000 7f5ac0c14700 1 --2- 172.21.15.92:0/4214110514 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5abc0b25e0 0x7f5abc0b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:32.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.877+0000 7f5ac0c14700 1 -- 172.21.15.92:0/4214110514 >> 172.21.15.92:0/4214110514 conn(0x7f5abc06ece0 msgr2=0x7f5abc06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:32.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.877+0000 7f5ac0c14700 1 -- 172.21.15.92:0/4214110514 shutdown_connections 2023-10-06T10:44:32.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.877+0000 7f5ac0c14700 1 -- 172.21.15.92:0/4214110514 wait complete. 2023-10-06T10:44:32.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.878+0000 7f5ac0c14700 1 Processor -- start 2023-10-06T10:44:32.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.878+0000 7f5ac0c14700 1 -- start start 2023-10-06T10:44:32.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.879+0000 7f5ac0c14700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5abc0b25e0 0x7f5abc0c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:32.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.879+0000 7f5ac0c14700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5abc0b29b0 0x7f5abc14a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:32.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.879+0000 7f5ac0c14700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5abc159830 0x7f5abc0b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:32.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.879+0000 7f5ac0c14700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc16a8a0 con 0x7f5abc0b25e0 2023-10-06T10:44:32.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.879+0000 7f5ac0c14700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc0c5df0 con 0x7f5abc159830 2023-10-06T10:44:32.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.879+0000 7f5ac0c14700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc169800 con 0x7f5abc0b29b0 2023-10-06T10:44:32.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.879+0000 7f5aba59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5abc0b25e0 0x7f5abc0c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:32.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.879+0000 7f5aba59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5abc0b25e0 0x7f5abc0c58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37814/0 (socket says 172.21.15.92:37814) 2023-10-06T10:44:32.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.879+0000 7f5aba59c700 1 -- 172.21.15.92:0/3498644327 learned_addr learned my addr 172.21.15.92:0/3498644327 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:32.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.879+0000 7f5ab9d9b700 1 --2- 172.21.15.92:0/3498644327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5abc0b29b0 0x7f5abc14a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:32.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.879+0000 7f5abad9d700 1 --2- 172.21.15.92:0/3498644327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5abc159830 0x7f5abc0b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:32.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.879+0000 7f5aba59c700 1 -- 172.21.15.92:0/3498644327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5abc0b29b0 msgr2=0x7f5abc14a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:32.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.879+0000 7f5aba59c700 1 --2- 172.21.15.92:0/3498644327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5abc0b29b0 0x7f5abc14a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:32.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.879+0000 7f5aba59c700 1 -- 172.21.15.92:0/3498644327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5abc159830 msgr2=0x7f5abc0b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:32.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.879+0000 7f5aba59c700 1 --2- 172.21.15.92:0/3498644327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5abc159830 0x7f5abc0b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:32.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.879+0000 7f5aba59c700 1 -- 172.21.15.92:0/3498644327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aac009a20 con 0x7f5abc0b25e0 2023-10-06T10:44:32.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.879+0000 7f5abad9d700 1 --2- 172.21.15.92:0/3498644327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5abc159830 0x7f5abc0b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:32.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.880+0000 7f5aba59c700 1 --2- 172.21.15.92:0/3498644327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5abc0b25e0 0x7f5abc0c58b0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00c1f0 tx=0x7f5aac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:32.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.880+0000 7f5aaa7fc700 1 -- 172.21.15.92:0/3498644327 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac00ce30 con 0x7f5abc0b25e0 2023-10-06T10:44:32.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.880+0000 7f5aaa7fc700 1 -- 172.21.15.92:0/3498644327 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aac00f040 con 0x7f5abc0b25e0 2023-10-06T10:44:32.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.880+0000 7f5aaa7fc700 1 -- 172.21.15.92:0/3498644327 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac014690 con 0x7f5abc0b25e0 2023-10-06T10:44:32.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.880+0000 7f5ac0c14700 1 -- 172.21.15.92:0/3498644327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab40094f0 con 0x7f5abc0b25e0 2023-10-06T10:44:32.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.880+0000 7f5ac0c14700 1 -- 172.21.15.92:0/3498644327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5abc169dc0 con 0x7f5abc0b25e0 2023-10-06T10:44:32.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.881+0000 7f5ac0c14700 1 -- 172.21.15.92:0/3498644327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5abc064a90 con 0x7f5abc0b25e0 2023-10-06T10:44:32.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.881+0000 7f5aaa7fc700 1 -- 172.21.15.92:0/3498644327 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5aac014a00 con 0x7f5abc0b25e0 2023-10-06T10:44:32.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.881+0000 7f5aaa7fc700 1 --2- 172.21.15.92:0/3498644327 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5a9c066f30 0x7f5a9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:32.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.881+0000 7f5aaa7fc700 1 -- 172.21.15.92:0/3498644327 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5aac085850 con 0x7f5abc0b25e0 2023-10-06T10:44:32.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.881+0000 7f5ab9d9b700 1 --2- 172.21.15.92:0/3498644327 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5a9c066f30 0x7f5a9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:32.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.882+0000 7f5ab9d9b700 1 --2- 172.21.15.92:0/3498644327 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5a9c066f30 0x7f5a9c0693e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0007fd0 tx=0x7f5ab0007d70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:32.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:32.886+0000 7f5aaa7fc700 1 -- 172.21.15.92:0/3498644327 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5aac0562b0 con 0x7f5abc0b25e0 2023-10-06T10:44:33.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.110+0000 7f5ac0c14700 1 -- 172.21.15.92:0/3498644327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5abc16a6f0 con 0x7f5abc0b25e0 2023-10-06T10:44:33.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.111+0000 7f5aaa7fc700 1 -- 172.21.15.92:0/3498644327 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4642 (secure 0 0 0) 0x7f5aac014cb0 con 0x7f5abc0b25e0 2023-10-06T10:44:33.113 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:33.113 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2122441982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2122441982},{"type":"v1","addr":"172.21.15.124:6835","nonce":2122441982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:47.850832+0000","modified":"2023-10-06T10:44:20.587653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7701,"mds_1":7689,"mds_2":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6836/3605297930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3605297930},{"type":"v1","addr":"172.21.15.92:6836","nonce":3605297930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7689":{"gid":7689,"name":"c","rank":1,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.92:6837/4047304116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4047304116},{"type":"v1","addr":"172.21.15.92:6837","nonce":4047304116}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7701":{"gid":7701,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":11,"addr":"172.21.15.124:6834/4203118992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":4203118992},{"type":"v1","addr":"172.21.15.124:6834","nonce":4203118992}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T10:44:33.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.113+0000 7f5ac0c14700 1 -- 172.21.15.92:0/3498644327 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5a9c066f30 msgr2=0x7f5a9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:33.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.113+0000 7f5ac0c14700 1 --2- 172.21.15.92:0/3498644327 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5a9c066f30 0x7f5a9c0693e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0007fd0 tx=0x7f5ab0007d70 comp rx=0 tx=0).stop 2023-10-06T10:44:33.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.113+0000 7f5ac0c14700 1 -- 172.21.15.92:0/3498644327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5abc0b25e0 msgr2=0x7f5abc0c58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:33.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.113+0000 7f5ac0c14700 1 --2- 172.21.15.92:0/3498644327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5abc0b25e0 0x7f5abc0c58b0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00c1f0 tx=0x7f5aac00b040 comp rx=0 tx=0).stop 2023-10-06T10:44:33.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.113+0000 7f5ac0c14700 1 -- 172.21.15.92:0/3498644327 shutdown_connections 2023-10-06T10:44:33.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.113+0000 7f5ac0c14700 1 --2- 172.21.15.92:0/3498644327 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5a9c066f30 0x7f5a9c0693e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:33.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.113+0000 7f5ac0c14700 1 --2- 172.21.15.92:0/3498644327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5abc159830 0x7f5abc0b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:33.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.113+0000 7f5ac0c14700 1 --2- 172.21.15.92:0/3498644327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5abc0b25e0 0x7f5abc0c58b0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:33.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.113+0000 7f5ac0c14700 1 --2- 172.21.15.92:0/3498644327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5abc0b29b0 0x7f5abc14a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:33.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.113+0000 7f5ac0c14700 1 -- 172.21.15.92:0/3498644327 >> 172.21.15.92:0/3498644327 conn(0x7f5abc06ece0 msgr2=0x7f5abc154140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:33.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.113+0000 7f5ac0c14700 1 -- 172.21.15.92:0/3498644327 shutdown_connections 2023-10-06T10:44:33.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.113+0000 7f5ac0c14700 1 -- 172.21.15.92:0/3498644327 wait complete. 2023-10-06T10:44:33.117 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 92 2023-10-06T10:44:33.127 DEBUG:tasks.cephfs.test_multimds_misc:current active=['b', 'c', 'a'] 2023-10-06T10:44:33.127 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:44:33.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.238+0000 7fec9b59e700 1 Processor -- start 2023-10-06T10:44:33.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.238+0000 7fec9b59e700 1 -- start start 2023-10-06T10:44:33.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.239+0000 7fec9b59e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c0b2810 0x7fec9c0cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:33.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.239+0000 7fec9b59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec9c0cb7a0 con 0x7fec9c0cbc30 2023-10-06T10:44:33.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.239+0000 7fec9b59e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec9c0b0860 con 0x7fec9c159440 2023-10-06T10:44:33.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.239+0000 7fec9b59e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0cb150 con 0x7fec9c0b2810 2023-10-06T10:44:33.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.239+0000 7fec91d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fec9c0cbc30 0x7fec9c0b2100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44202/0 (socket says 172.21.15.92:44202) 2023-10-06T10:44:33.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.239+0000 7fec91d9b700 1 -- 172.21.15.92:0/4075526769 learned_addr learned my addr 172.21.15.92:0/4075526769 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:33.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.239+0000 7fec99d9b700 1 -- 172.21.15.92:0/4075526769 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331148016 0 0) 0x7fec9c0cb7a0 con 0x7fec9c0cbc30 2023-10-06T10:44:33.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.239+0000 7fec9ad9d700 1 --2- 172.21.15.92:0/4075526769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c0b2810 0x7fec9c0cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:33.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.239+0000 7fec99d9b700 1 -- 172.21.15.92:0/4075526769 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec88003570 con 0x7fec9c0cbc30 2023-10-06T10:44:33.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.239+0000 7fec99d9b700 1 -- 172.21.15.92:0/4075526769 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2781850684 0 0) 0x7fec88003570 con 0x7fec9c0cbc30 2023-10-06T10:44:33.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.239+0000 7fec99d9b700 1 -- 172.21.15.92:0/4075526769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c0b2810 msgr2=0x7fec9c0cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:33.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.239+0000 7fec99d9b700 1 --2- 172.21.15.92:0/4075526769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c0b2810 0x7fec9c0cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:33.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.239+0000 7fec99d9b700 1 -- 172.21.15.92:0/4075526769 >> v1:172.21.15.124:6789/0 conn(0x7fec9c159440 legacy=0x7fec9c0b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:33.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.239+0000 7fec99d9b700 1 -- 172.21.15.92:0/4075526769 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c0cb430 con 0x7fec9c0cbc30 2023-10-06T10:44:33.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.239+0000 7fec99d9b700 1 -- 172.21.15.92:0/4075526769 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fec84001470 con 0x7fec9c0cbc30 2023-10-06T10:44:33.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.240+0000 7fec99d9b700 1 -- 172.21.15.92:0/4075526769 >> v1:172.21.15.92:6789/0 conn(0x7fec9c0cbc30 legacy=0x7fec9c0b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:33.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.240+0000 7fec99d9b700 1 --2- 172.21.15.92:0/4075526769 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec88003a90 0x7fec88003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:33.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.240+0000 7fec99d9b700 1 --2- 172.21.15.92:0/4075526769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c0b2810 0x7fec88004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:33.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.240+0000 7fec99d9b700 1 --2- 172.21.15.92:0/4075526769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec88004d50 0x7fec880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:33.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.240+0000 7fec99d9b700 1 -- 172.21.15.92:0/4075526769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0cb150 con 0x7fec88004d50 2023-10-06T10:44:33.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.240+0000 7fec99d9b700 1 -- 172.21.15.92:0/4075526769 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8800bb70 con 0x7fec88003a90 2023-10-06T10:44:33.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.240+0000 7fec99d9b700 1 -- 172.21.15.92:0/4075526769 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8800bd30 con 0x7fec9c0b2810 2023-10-06T10:44:33.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.240+0000 7fec9ad9d700 1 --2- 172.21.15.92:0/4075526769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec88004d50 0x7fec880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:33.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.240+0000 7fec9a59c700 1 --2- 172.21.15.92:0/4075526769 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec88003a90 0x7fec88003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:33.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.240+0000 7fec91d9b700 1 --2- 172.21.15.92:0/4075526769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c0b2810 0x7fec88004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:33.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.240+0000 7fec9ad9d700 1 -- 172.21.15.92:0/4075526769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c0b2810 msgr2=0x7fec88004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:33.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.240+0000 7fec9ad9d700 1 --2- 172.21.15.92:0/4075526769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c0b2810 0x7fec88004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:33.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.240+0000 7fec9ad9d700 1 -- 172.21.15.92:0/4075526769 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec88003a90 msgr2=0x7fec88003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:33.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.240+0000 7fec9ad9d700 1 --2- 172.21.15.92:0/4075526769 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec88003a90 0x7fec88003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:33.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.240+0000 7fec9ad9d700 1 -- 172.21.15.92:0/4075526769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec88003570 con 0x7fec88004d50 2023-10-06T10:44:33.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.240+0000 7fec9ad9d700 1 --2- 172.21.15.92:0/4075526769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec88004d50 0x7fec880051c0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fec9400e210 tx=0x7fec9400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d810bde7062a682 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:33.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.240+0000 7fec99d9b700 1 -- 172.21.15.92:0/4075526769 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9400eac0 con 0x7fec88004d50 2023-10-06T10:44:33.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.240+0000 7fec99d9b700 1 -- 172.21.15.92:0/4075526769 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9400f040 con 0x7fec88004d50 2023-10-06T10:44:33.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.240+0000 7fec99d9b700 1 -- 172.21.15.92:0/4075526769 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9400ee70 con 0x7fec88004d50 2023-10-06T10:44:33.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.241+0000 7fec9b59e700 1 -- 172.21.15.92:0/4075526769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec88004d50 msgr2=0x7fec880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:33.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.241+0000 7fec9b59e700 1 --2- 172.21.15.92:0/4075526769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec88004d50 0x7fec880051c0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fec9400e210 tx=0x7fec9400b040 comp rx=0 tx=0).stop 2023-10-06T10:44:33.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.241+0000 7fec9ad9d700 1 -- 172.21.15.92:0/4075526769 reap_dead start 2023-10-06T10:44:33.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.241+0000 7fec9b59e700 1 -- 172.21.15.92:0/4075526769 shutdown_connections 2023-10-06T10:44:33.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.241+0000 7fec9b59e700 1 -- 172.21.15.92:0/4075526769 >> 172.21.15.92:0/4075526769 conn(0x7fec9c06ece0 msgr2=0x7fec9c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:33.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.241+0000 7fec9b59e700 1 -- 172.21.15.92:0/4075526769 shutdown_connections 2023-10-06T10:44:33.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.241+0000 7fec9b59e700 1 -- 172.21.15.92:0/4075526769 wait complete. 2023-10-06T10:44:33.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.242+0000 7fec9b59e700 1 Processor -- start 2023-10-06T10:44:33.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.242+0000 7fec9b59e700 1 -- start start 2023-10-06T10:44:33.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec9b59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec9c1476d0 0x7fec9c15d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:33.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec9b59e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c15d6d0 0x7fec9c15b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:33.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec9b59e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec9c15b850 0x7fec9c0cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:33.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec9b59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c15df80 con 0x7fec9c1476d0 2023-10-06T10:44:33.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec9b59e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c15cc10 con 0x7fec9c15b850 2023-10-06T10:44:33.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec9b59e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c15c760 con 0x7fec9c15d6d0 2023-10-06T10:44:33.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec9a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec9c1476d0 0x7fec9c15d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:33.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec9a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec9c1476d0 0x7fec9c15d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37834/0 (socket says 172.21.15.92:37834) 2023-10-06T10:44:33.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec9a59c700 1 -- 172.21.15.92:0/199864612 learned_addr learned my addr 172.21.15.92:0/199864612 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:33.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec9ad9d700 1 --2- 172.21.15.92:0/199864612 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec9c15b850 0x7fec9c0cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:33.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec91d9b700 1 --2- 172.21.15.92:0/199864612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c15d6d0 0x7fec9c15b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:33.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec9a59c700 1 -- 172.21.15.92:0/199864612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c15d6d0 msgr2=0x7fec9c15b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:33.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec9a59c700 1 --2- 172.21.15.92:0/199864612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c15d6d0 0x7fec9c15b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:33.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec9a59c700 1 -- 172.21.15.92:0/199864612 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec9c15b850 msgr2=0x7fec9c0cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:33.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec9a59c700 1 --2- 172.21.15.92:0/199864612 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec9c15b850 0x7fec9c0cedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:33.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec9a59c700 1 -- 172.21.15.92:0/199864612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c15dc50 con 0x7fec9c1476d0 2023-10-06T10:44:33.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec9ad9d700 1 --2- 172.21.15.92:0/199864612 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec9c15b850 0x7fec9c0cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:33.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec9a59c700 1 --2- 172.21.15.92:0/199864612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec9c1476d0 0x7fec9c15d190 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fec8c002950 tx=0x7fec8c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:33.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec92ffd700 1 -- 172.21.15.92:0/199864612 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c002e90 con 0x7fec9c1476d0 2023-10-06T10:44:33.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec92ffd700 1 -- 172.21.15.92:0/199864612 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8c00f8d0 con 0x7fec9c1476d0 2023-10-06T10:44:33.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec9b59e700 1 -- 172.21.15.92:0/199864612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9c15c1a0 con 0x7fec9c1476d0 2023-10-06T10:44:33.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec92ffd700 1 -- 172.21.15.92:0/199864612 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c00fc90 con 0x7fec9c1476d0 2023-10-06T10:44:33.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.243+0000 7fec9b59e700 1 -- 172.21.15.92:0/199864612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9c147aa0 con 0x7fec9c1476d0 2023-10-06T10:44:33.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.244+0000 7fec9b59e700 1 -- 172.21.15.92:0/199864612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec9c05f230 con 0x7fec9c1476d0 2023-10-06T10:44:33.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.245+0000 7fec92ffd700 1 -- 172.21.15.92:0/199864612 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec8c013600 con 0x7fec9c1476d0 2023-10-06T10:44:33.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.245+0000 7fec92ffd700 1 --2- 172.21.15.92:0/199864612 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fec7c066f30 0x7fec7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:33.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.245+0000 7fec92ffd700 1 -- 172.21.15.92:0/199864612 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fec8c084610 con 0x7fec9c1476d0 2023-10-06T10:44:33.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.245+0000 7fec91d9b700 1 --2- 172.21.15.92:0/199864612 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fec7c066f30 0x7fec7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:33.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.245+0000 7fec91d9b700 1 --2- 172.21.15.92:0/199864612 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fec7c066f30 0x7fec7c0693e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fec84000e10 tx=0x7fec84009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:33.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.249+0000 7fec92ffd700 1 -- 172.21.15.92:0/199864612 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec8c0515e0 con 0x7fec9c1476d0 2023-10-06T10:44:33.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.507+0000 7fec9b59e700 1 -- 172.21.15.92:0/199864612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fec9c159810 con 0x7fec9c1476d0 2023-10-06T10:44:33.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.508+0000 7fec92ffd700 1 -- 172.21.15.92:0/199864612 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1921 (secure 0 0 0) 0x7fec8c00fdf0 con 0x7fec9c1476d0 2023-10-06T10:44:33.510 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:33.510 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":394,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":704416,"bytes_used":2445824000,"bytes_avail":770648289280,"bytes_total":773094113280,"write_bytes_sec":682,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":92,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"a","status":"up:active","gid":7686},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7689},{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":7701}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2023-10-06T10:44:28.163210+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T10:44:33.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.509+0000 7fec9b59e700 1 -- 172.21.15.92:0/199864612 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fec7c066f30 msgr2=0x7fec7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:33.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.509+0000 7fec9b59e700 1 --2- 172.21.15.92:0/199864612 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fec7c066f30 0x7fec7c0693e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fec84000e10 tx=0x7fec84009040 comp rx=0 tx=0).stop 2023-10-06T10:44:33.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.509+0000 7fec9b59e700 1 -- 172.21.15.92:0/199864612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec9c1476d0 msgr2=0x7fec9c15d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:33.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.509+0000 7fec9b59e700 1 --2- 172.21.15.92:0/199864612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec9c1476d0 0x7fec9c15d190 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fec8c002950 tx=0x7fec8c007d30 comp rx=0 tx=0).stop 2023-10-06T10:44:33.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.509+0000 7fec9b59e700 1 -- 172.21.15.92:0/199864612 shutdown_connections 2023-10-06T10:44:33.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.509+0000 7fec9b59e700 1 --2- 172.21.15.92:0/199864612 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fec7c066f30 0x7fec7c0693e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:33.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.509+0000 7fec9b59e700 1 --2- 172.21.15.92:0/199864612 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec9c15b850 0x7fec9c0cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:33.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.509+0000 7fec9b59e700 1 --2- 172.21.15.92:0/199864612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec9c1476d0 0x7fec9c15d190 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:33.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.509+0000 7fec9b59e700 1 --2- 172.21.15.92:0/199864612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c15d6d0 0x7fec9c15b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:33.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.509+0000 7fec9b59e700 1 -- 172.21.15.92:0/199864612 >> 172.21.15.92:0/199864612 conn(0x7fec9c06ece0 msgr2=0x7fec9c155780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:33.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.510+0000 7fec9b59e700 1 -- 172.21.15.92:0/199864612 shutdown_connections 2023-10-06T10:44:33.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:33.510+0000 7fec9b59e700 1 -- 172.21.15.92:0/199864612 wait complete. 2023-10-06T10:44:33.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'idle+waiting paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]'} 2023-10-06T10:44:34.524 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:44:34.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.641+0000 7f048df64700 1 Processor -- start 2023-10-06T10:44:34.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.641+0000 7f048df64700 1 -- start start 2023-10-06T10:44:34.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.642+0000 7f048df64700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04880ca370 0x7f0488158270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:34.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.642+0000 7f048df64700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f04880adb60 0x7f04880ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:34.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.642+0000 7f048df64700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0488157de0 con 0x7f04880ce6d0 2023-10-06T10:44:34.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.642+0000 7f048df64700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04880cbb90 con 0x7f04880adb60 2023-10-06T10:44:34.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.642+0000 7f048df64700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04880ceaf0 con 0x7f04880ca370 2023-10-06T10:44:34.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.642+0000 7f0486ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f04880adb60 0x7f04880ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:34.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.642+0000 7f0486ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f04880adb60 0x7f04880ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46282/0 (socket says 172.21.15.92:46282) 2023-10-06T10:44:34.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.642+0000 7f0486ffd700 1 -- 172.21.15.92:0/1246861508 learned_addr learned my addr 172.21.15.92:0/1246861508 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:34.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.642+0000 7f04877fe700 1 --2- 172.21.15.92:0/1246861508 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04880ca370 0x7f0488158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:34.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.642+0000 7f0486ffd700 1 -- 172.21.15.92:0/1246861508 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04880ca370 msgr2=0x7f0488158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:34.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.642+0000 7f0486ffd700 1 --2- 172.21.15.92:0/1246861508 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04880ca370 0x7f0488158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:34.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.642+0000 7f0486ffd700 1 -- 172.21.15.92:0/1246861508 >> v1:172.21.15.124:6789/0 conn(0x7f04880ce6d0 legacy=0x7f048815aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:34.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.642+0000 7f0486ffd700 1 -- 172.21.15.92:0/1246861508 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04880cb5d0 con 0x7f04880adb60 2023-10-06T10:44:34.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.642+0000 7f04877fe700 1 --2- 172.21.15.92:0/1246861508 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04880ca370 0x7f0488158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:34.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.643+0000 7f0486ffd700 1 --2- 172.21.15.92:0/1246861508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04880adb60 0x7f04880ce190 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f04780099b0 tx=0x7f0478009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f6b36e7724c4fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:34.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.643+0000 7f04867fc700 1 -- 172.21.15.92:0/1246861508 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047800b820 con 0x7f04880adb60 2023-10-06T10:44:34.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.643+0000 7f04867fc700 1 -- 172.21.15.92:0/1246861508 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047800b980 con 0x7f04880adb60 2023-10-06T10:44:34.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.643+0000 7f04867fc700 1 -- 172.21.15.92:0/1246861508 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047800bd20 con 0x7f04880adb60 2023-10-06T10:44:34.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.643+0000 7f048df64700 1 -- 172.21.15.92:0/1246861508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04880adb60 msgr2=0x7f04880ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:34.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.643+0000 7f048df64700 1 --2- 172.21.15.92:0/1246861508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04880adb60 0x7f04880ce190 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f04780099b0 tx=0x7f0478009250 comp rx=0 tx=0).stop 2023-10-06T10:44:34.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.643+0000 7f048df64700 1 -- 172.21.15.92:0/1246861508 shutdown_connections 2023-10-06T10:44:34.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.643+0000 7f048df64700 1 --2- 172.21.15.92:0/1246861508 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04880ca370 0x7f0488158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.643+0000 7f048df64700 1 --2- 172.21.15.92:0/1246861508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04880adb60 0x7f04880ce190 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.643+0000 7f048df64700 1 -- 172.21.15.92:0/1246861508 >> 172.21.15.92:0/1246861508 conn(0x7f04880707d0 msgr2=0x7f0488070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.643+0000 7f048df64700 1 -- 172.21.15.92:0/1246861508 shutdown_connections 2023-10-06T10:44:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.643+0000 7f048df64700 1 -- 172.21.15.92:0/1246861508 wait complete. 2023-10-06T10:44:34.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.645+0000 7f048df64700 1 Processor -- start 2023-10-06T10:44:34.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.645+0000 7f048df64700 1 -- start start 2023-10-06T10:44:34.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.645+0000 7f048df64700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04880adb60 0x7f04880c36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:34.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.645+0000 7f048df64700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04880ca370 0x7f04880afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:34.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.645+0000 7f048df64700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04880ce6d0 0x7f048814a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:34.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.645+0000 7f048df64700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04880b3640 con 0x7f04880ca370 2023-10-06T10:44:34.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.645+0000 7f048df64700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04880c4160 con 0x7f04880ce6d0 2023-10-06T10:44:34.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.645+0000 7f048df64700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04880b2a30 con 0x7f04880adb60 2023-10-06T10:44:34.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.646+0000 7f0486ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04880ca370 0x7f04880afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:34.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.646+0000 7f0486ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04880ca370 0x7f04880afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46298/0 (socket says 172.21.15.92:46298) 2023-10-06T10:44:34.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.646+0000 7f0486ffd700 1 -- 172.21.15.92:0/468385595 learned_addr learned my addr 172.21.15.92:0/468385595 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:34.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.646+0000 7f04877fe700 1 --2- 172.21.15.92:0/468385595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04880adb60 0x7f04880c36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:34.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.646+0000 7f0487fff700 1 --2- 172.21.15.92:0/468385595 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04880ce6d0 0x7f048814a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:34.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.646+0000 7f0486ffd700 1 -- 172.21.15.92:0/468385595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04880adb60 msgr2=0x7f04880c36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:34.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.646+0000 7f0486ffd700 1 --2- 172.21.15.92:0/468385595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04880adb60 0x7f04880c36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:34.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.646+0000 7f0486ffd700 1 -- 172.21.15.92:0/468385595 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04880ce6d0 msgr2=0x7f048814a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:34.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.646+0000 7f0486ffd700 1 --2- 172.21.15.92:0/468385595 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04880ce6d0 0x7f048814a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:34.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.646+0000 7f0486ffd700 1 -- 172.21.15.92:0/468385595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04780094f0 con 0x7f04880ca370 2023-10-06T10:44:34.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.646+0000 7f0486ffd700 1 --2- 172.21.15.92:0/468385595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04880ca370 0x7f04880afdf0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f0478009290 tx=0x7f0478016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:34.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.646+0000 7f047f7fe700 1 -- 172.21.15.92:0/468385595 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0478011c10 con 0x7f04880ca370 2023-10-06T10:44:34.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.646+0000 7f047f7fe700 1 -- 172.21.15.92:0/468385595 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0478020070 con 0x7f04880ca370 2023-10-06T10:44:34.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.646+0000 7f048df64700 1 -- 172.21.15.92:0/468385595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04880b2d10 con 0x7f04880ca370 2023-10-06T10:44:34.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.646+0000 7f047f7fe700 1 -- 172.21.15.92:0/468385595 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047801b450 con 0x7f04880ca370 2023-10-06T10:44:34.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.646+0000 7f048df64700 1 -- 172.21.15.92:0/468385595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04880b32d0 con 0x7f04880ca370 2023-10-06T10:44:34.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.647+0000 7f048df64700 1 -- 172.21.15.92:0/468385595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048814f360 con 0x7f04880ca370 2023-10-06T10:44:34.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.647+0000 7f047f7fe700 1 -- 172.21.15.92:0/468385595 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0478022030 con 0x7f04880ca370 2023-10-06T10:44:34.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.648+0000 7f047f7fe700 1 --2- 172.21.15.92:0/468385595 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0468066f30 0x7f04680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:34.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.648+0000 7f047f7fe700 1 -- 172.21.15.92:0/468385595 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f04780865c0 con 0x7f04880ca370 2023-10-06T10:44:34.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.648+0000 7f04877fe700 1 --2- 172.21.15.92:0/468385595 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0468066f30 0x7f04680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:34.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.648+0000 7f04877fe700 1 --2- 172.21.15.92:0/468385595 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0468066f30 0x7f04680693e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f04880ce5f0 tx=0x7f0470006f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:34.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.652+0000 7f047f7fe700 1 -- 172.21.15.92:0/468385595 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0478053590 con 0x7f04880ca370 2023-10-06T10:44:34.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.875+0000 7f048df64700 1 -- 172.21.15.92:0/468385595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f048814f360 con 0x7f04880ca370 2023-10-06T10:44:34.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.876+0000 7f047f7fe700 1 -- 172.21.15.92:0/468385595 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4642 (secure 0 0 0) 0x7f0478056bb0 con 0x7f04880ca370 2023-10-06T10:44:34.878 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:34.878 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2122441982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2122441982},{"type":"v1","addr":"172.21.15.124:6835","nonce":2122441982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:47.850832+0000","modified":"2023-10-06T10:44:20.587653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7701,"mds_1":7689,"mds_2":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6836/3605297930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3605297930},{"type":"v1","addr":"172.21.15.92:6836","nonce":3605297930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7689":{"gid":7689,"name":"c","rank":1,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.92:6837/4047304116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4047304116},{"type":"v1","addr":"172.21.15.92:6837","nonce":4047304116}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7701":{"gid":7701,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":11,"addr":"172.21.15.124:6834/4203118992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":4203118992},{"type":"v1","addr":"172.21.15.124:6834","nonce":4203118992}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T10:44:34.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.878+0000 7f048df64700 1 -- 172.21.15.92:0/468385595 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0468066f30 msgr2=0x7f04680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:34.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.878+0000 7f048df64700 1 --2- 172.21.15.92:0/468385595 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0468066f30 0x7f04680693e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f04880ce5f0 tx=0x7f0470006f90 comp rx=0 tx=0).stop 2023-10-06T10:44:34.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.878+0000 7f048df64700 1 -- 172.21.15.92:0/468385595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04880ca370 msgr2=0x7f04880afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:34.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.878+0000 7f048df64700 1 --2- 172.21.15.92:0/468385595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04880ca370 0x7f04880afdf0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f0478009290 tx=0x7f0478016000 comp rx=0 tx=0).stop 2023-10-06T10:44:34.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.878+0000 7f048df64700 1 -- 172.21.15.92:0/468385595 shutdown_connections 2023-10-06T10:44:34.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.878+0000 7f048df64700 1 --2- 172.21.15.92:0/468385595 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0468066f30 0x7f04680693e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:34.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.878+0000 7f048df64700 1 --2- 172.21.15.92:0/468385595 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04880ce6d0 0x7f048814a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:34.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.878+0000 7f048df64700 1 --2- 172.21.15.92:0/468385595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04880adb60 0x7f04880c36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:34.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.878+0000 7f048df64700 1 --2- 172.21.15.92:0/468385595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04880ca370 0x7f04880afdf0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:34.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.878+0000 7f048df64700 1 -- 172.21.15.92:0/468385595 >> 172.21.15.92:0/468385595 conn(0x7f04880707d0 msgr2=0x7f0488154500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:34.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.878+0000 7f048df64700 1 -- 172.21.15.92:0/468385595 shutdown_connections 2023-10-06T10:44:34.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:34.878+0000 7f048df64700 1 -- 172.21.15.92:0/468385595 wait complete. 2023-10-06T10:44:34.881 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 92 2023-10-06T10:44:34.892 DEBUG:tasks.cephfs.test_multimds_misc:current active=['b', 'c', 'a'] 2023-10-06T10:44:34.892 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:44:35.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.003+0000 7f671d556700 1 Processor -- start 2023-10-06T10:44:35.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.003+0000 7f671d556700 1 -- start start 2023-10-06T10:44:35.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.004+0000 7f671d556700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67180b49a0 0x7f67180aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:35.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.004+0000 7f671d556700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f67180b0760 0x7f67180b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:35.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.004+0000 7f671d556700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67180b0c60 con 0x7f6718159560 2023-10-06T10:44:35.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.004+0000 7f671d556700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67180b0f40 con 0x7f67180b0760 2023-10-06T10:44:35.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.004+0000 7f671d556700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67180b1220 con 0x7f67180b49a0 2023-10-06T10:44:35.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.004+0000 7f67177fe700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f67180b0760 0x7f67180b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:35.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.004+0000 7f67167fc700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67180b49a0 0x7f67180aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:35.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.004+0000 7f67167fc700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67180b49a0 0x7f67180aac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:53514/0 (socket says 172.21.15.92:53514) 2023-10-06T10:44:35.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.004+0000 7f67167fc700 1 -- 172.21.15.92:0/688879684 learned_addr learned my addr 172.21.15.92:0/688879684 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:35.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.004+0000 7f67177fe700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f67180b0760 0x7f67180b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:39312/0 (socket says 172.21.15.92:39312) 2023-10-06T10:44:35.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.004+0000 7f6716ffd700 1 --1- 172.21.15.92:0/688879684 >> v1:172.21.15.124:6789/0 conn(0x7f6718159560 0x7f67180aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:47492/0 (socket says 172.21.15.92:47492) 2023-10-06T10:44:35.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.004+0000 7f67167fc700 1 -- 172.21.15.92:0/688879684 >> v2:172.21.15.124:3300/0 conn(0x7f67180b0760 msgr2=0x7f67180b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:35.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.004+0000 7f67167fc700 1 --2- 172.21.15.92:0/688879684 >> v2:172.21.15.124:3300/0 conn(0x7f67180b0760 0x7f67180b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.004+0000 7f67167fc700 1 -- 172.21.15.92:0/688879684 >> v1:172.21.15.124:6789/0 conn(0x7f6718159560 legacy=0x7f67180aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:35.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.004+0000 7f67167fc700 1 -- 172.21.15.92:0/688879684 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67180b1500 con 0x7f67180b49a0 2023-10-06T10:44:35.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.004+0000 7f67177fe700 1 --2- 172.21.15.92:0/688879684 >> v2:172.21.15.124:3300/0 conn(0x7f67180b0760 0x7f67180b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:44:35.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.005+0000 7f67167fc700 1 --2- 172.21.15.92:0/688879684 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67180b49a0 0x7f67180aac40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f670c0099b0 tx=0x7f670c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5d432a9c546bce8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:35.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.005+0000 7f6715ffb700 1 -- 172.21.15.92:0/688879684 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c009ea0 con 0x7f67180b49a0 2023-10-06T10:44:35.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.005+0000 7f6715ffb700 1 -- 172.21.15.92:0/688879684 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670c00b860 con 0x7f67180b49a0 2023-10-06T10:44:35.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.005+0000 7f6715ffb700 1 -- 172.21.15.92:0/688879684 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c00bc00 con 0x7f67180b49a0 2023-10-06T10:44:35.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.005+0000 7f671d556700 1 -- 172.21.15.92:0/688879684 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67180b49a0 msgr2=0x7f67180aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:35.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.005+0000 7f671d556700 1 --2- 172.21.15.92:0/688879684 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67180b49a0 0x7f67180aac40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f670c0099b0 tx=0x7f670c009250 comp rx=0 tx=0).stop 2023-10-06T10:44:35.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.005+0000 7f671d556700 1 -- 172.21.15.92:0/688879684 shutdown_connections 2023-10-06T10:44:35.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.005+0000 7f671d556700 1 --2- 172.21.15.92:0/688879684 >> v2:172.21.15.124:3300/0 conn(0x7f67180b0760 0x7f67180b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.005+0000 7f671d556700 1 --2- 172.21.15.92:0/688879684 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67180b49a0 0x7f67180aac40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.005+0000 7f671d556700 1 -- 172.21.15.92:0/688879684 >> 172.21.15.92:0/688879684 conn(0x7f67180997f0 msgr2=0x7f671806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:35.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.005+0000 7f671d556700 1 -- 172.21.15.92:0/688879684 shutdown_connections 2023-10-06T10:44:35.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.005+0000 7f671d556700 1 -- 172.21.15.92:0/688879684 wait complete. 2023-10-06T10:44:35.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.006+0000 7f671d556700 1 Processor -- start 2023-10-06T10:44:35.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.007+0000 7f671d556700 1 -- start start 2023-10-06T10:44:35.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.007+0000 7f671d556700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67180b0760 0x7f671809a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:35.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.007+0000 7f671d556700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67180b49a0 0x7f67180aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:35.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.007+0000 7f671d556700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6718159560 0x7f6718157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:35.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.007+0000 7f671d556700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671809d9c0 con 0x7f6718159560 2023-10-06T10:44:35.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.007+0000 7f671d556700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671809d090 con 0x7f67180b49a0 2023-10-06T10:44:35.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.007+0000 7f671d556700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671809d370 con 0x7f67180b0760 2023-10-06T10:44:35.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.007+0000 7f67177fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6718159560 0x7f6718157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:35.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.007+0000 7f67177fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6718159560 0x7f6718157c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46308/0 (socket says 172.21.15.92:46308) 2023-10-06T10:44:35.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.007+0000 7f67177fe700 1 -- 172.21.15.92:0/8500104 learned_addr learned my addr 172.21.15.92:0/8500104 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:35.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.008+0000 7f6716ffd700 1 --2- 172.21.15.92:0/8500104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67180b0760 0x7f671809a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:35.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.008+0000 7f67167fc700 1 --2- 172.21.15.92:0/8500104 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67180b49a0 0x7f67180aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:35.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.008+0000 7f67177fe700 1 -- 172.21.15.92:0/8500104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67180b0760 msgr2=0x7f671809a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:35.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.008+0000 7f67177fe700 1 --2- 172.21.15.92:0/8500104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67180b0760 0x7f671809a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.008+0000 7f67177fe700 1 -- 172.21.15.92:0/8500104 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67180b49a0 msgr2=0x7f67180aa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:35.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.008+0000 7f67177fe700 1 --2- 172.21.15.92:0/8500104 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67180b49a0 0x7f67180aa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.008+0000 7f67177fe700 1 -- 172.21.15.92:0/8500104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6710009420 con 0x7f6718159560 2023-10-06T10:44:35.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.008+0000 7f67177fe700 1 --2- 172.21.15.92:0/8500104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6718159560 0x7f6718157c60 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f671000ec80 tx=0x7f67100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:35.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.008+0000 7f66feffd700 1 -- 172.21.15.92:0/8500104 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671000c850 con 0x7f6718159560 2023-10-06T10:44:35.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.008+0000 7f66feffd700 1 -- 172.21.15.92:0/8500104 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671000c9b0 con 0x7f6718159560 2023-10-06T10:44:35.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.008+0000 7f671d556700 1 -- 172.21.15.92:0/8500104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670c0094f0 con 0x7f6718159560 2023-10-06T10:44:35.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.008+0000 7f66feffd700 1 -- 172.21.15.92:0/8500104 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671000cd50 con 0x7f6718159560 2023-10-06T10:44:35.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.008+0000 7f671d556700 1 -- 172.21.15.92:0/8500104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67181581a0 con 0x7f6718159560 2023-10-06T10:44:35.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.009+0000 7f671d556700 1 -- 172.21.15.92:0/8500104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f671809db30 con 0x7f6718159560 2023-10-06T10:44:35.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.009+0000 7f66feffd700 1 -- 172.21.15.92:0/8500104 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6710013680 con 0x7f6718159560 2023-10-06T10:44:35.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.010+0000 7f66feffd700 1 --2- 172.21.15.92:0/8500104 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6704066f30 0x7f67040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:35.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.010+0000 7f66feffd700 1 -- 172.21.15.92:0/8500104 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f67100847c0 con 0x7f6718159560 2023-10-06T10:44:35.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.010+0000 7f6716ffd700 1 --2- 172.21.15.92:0/8500104 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6704066f30 0x7f67040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:35.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.010+0000 7f6716ffd700 1 --2- 172.21.15.92:0/8500104 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6704066f30 0x7f67040693e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f671806a030 tx=0x7f6708001a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:35.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.014+0000 7f66feffd700 1 -- 172.21.15.92:0/8500104 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6710051790 con 0x7f6718159560 2023-10-06T10:44:35.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.272+0000 7f671d556700 1 -- 172.21.15.92:0/8500104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6718064a90 con 0x7f6718159560 2023-10-06T10:44:35.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.272+0000 7f66feffd700 1 -- 172.21.15.92:0/8500104 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1827 (secure 0 0 0) 0x7f6710082e60 con 0x7f6718159560 2023-10-06T10:44:35.275 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:35.275 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":396,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":704416,"bytes_used":2445824000,"bytes_avail":770648289280,"bytes_total":773094113280,"write_bytes_sec":682,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":92,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"a","status":"up:active","gid":7686},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7689},{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":7701}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2023-10-06T10:44:34.164444+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T10:44:35.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.274+0000 7f671d556700 1 -- 172.21.15.92:0/8500104 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6704066f30 msgr2=0x7f67040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:35.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.274+0000 7f671d556700 1 --2- 172.21.15.92:0/8500104 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6704066f30 0x7f67040693e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f671806a030 tx=0x7f6708001a20 comp rx=0 tx=0).stop 2023-10-06T10:44:35.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.274+0000 7f671d556700 1 -- 172.21.15.92:0/8500104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6718159560 msgr2=0x7f6718157c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:35.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.274+0000 7f671d556700 1 --2- 172.21.15.92:0/8500104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6718159560 0x7f6718157c60 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f671000ec80 tx=0x7f67100091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.274+0000 7f671d556700 1 -- 172.21.15.92:0/8500104 shutdown_connections 2023-10-06T10:44:35.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.274+0000 7f671d556700 1 --2- 172.21.15.92:0/8500104 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6704066f30 0x7f67040693e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.274+0000 7f671d556700 1 --2- 172.21.15.92:0/8500104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6718159560 0x7f6718157c60 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.274+0000 7f671d556700 1 --2- 172.21.15.92:0/8500104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67180b0760 0x7f671809a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.274+0000 7f671d556700 1 --2- 172.21.15.92:0/8500104 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67180b49a0 0x7f67180aa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.274+0000 7f671d556700 1 -- 172.21.15.92:0/8500104 >> 172.21.15.92:0/8500104 conn(0x7f67180997f0 msgr2=0x7f6718158b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:35.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.274+0000 7f671d556700 1 -- 172.21.15.92:0/8500104 shutdown_connections 2023-10-06T10:44:35.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.274+0000 7f671d556700 1 -- 172.21.15.92:0/8500104 wait complete. 2023-10-06T10:44:35.288 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2023-10-06T10:44:35.288 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T10:44:35.288 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:44:35.329 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:44:35.378 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T10:44:35.379 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T10:44:35.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:44:35.379 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T10:44:35.419 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T10:44:35.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:44:35.419 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T10:44:35.475 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:44:35.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.628+0000 7f424b763700 1 Processor -- start 2023-10-06T10:44:35.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.628+0000 7f424b763700 1 -- start start 2023-10-06T10:44:35.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.629+0000 7f424b763700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4244159560 0x7f42440aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:35.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.629+0000 7f424b763700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f42440b49a0 0x7f42440aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:35.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.629+0000 7f424b763700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42440b0760 0x7f42440b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:35.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.629+0000 7f424b763700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42440b0c60 con 0x7f4244159560 2023-10-06T10:44:35.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.629+0000 7f424b763700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42440b0f40 con 0x7f42440b49a0 2023-10-06T10:44:35.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.629+0000 7f424b763700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42440b1220 con 0x7f42440b0760 2023-10-06T10:44:35.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.629+0000 7f42494ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4244159560 0x7f42440aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:35.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.629+0000 7f42494ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4244159560 0x7f42440aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46312/0 (socket says 172.21.15.92:46312) 2023-10-06T10:44:35.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.629+0000 7f42494ff700 1 -- 172.21.15.92:0/3365692699 learned_addr learned my addr 172.21.15.92:0/3365692699 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:35.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.629+0000 7f4248cfe700 1 --2- 172.21.15.92:0/3365692699 >> v2:172.21.15.124:3300/0 conn(0x7f42440b49a0 0x7f42440aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:35.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.630+0000 7f42494ff700 1 -- 172.21.15.92:0/3365692699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42440b0760 msgr2=0x7f42440b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:35.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.630+0000 7f4249d00700 1 --2- 172.21.15.92:0/3365692699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42440b0760 0x7f42440b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:35.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.630+0000 7f42494ff700 1 --2- 172.21.15.92:0/3365692699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42440b0760 0x7f42440b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.630+0000 7f42494ff700 1 -- 172.21.15.92:0/3365692699 >> v2:172.21.15.124:3300/0 conn(0x7f42440b49a0 msgr2=0x7f42440aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:35.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.630+0000 7f42494ff700 1 --2- 172.21.15.92:0/3365692699 >> v2:172.21.15.124:3300/0 conn(0x7f42440b49a0 0x7f42440aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.630+0000 7f42494ff700 1 -- 172.21.15.92:0/3365692699 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42440b1500 con 0x7f4244159560 2023-10-06T10:44:35.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.630+0000 7f4248cfe700 1 --2- 172.21.15.92:0/3365692699 >> v2:172.21.15.124:3300/0 conn(0x7f42440b49a0 0x7f42440aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:35.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.630+0000 7f42494ff700 1 --2- 172.21.15.92:0/3365692699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4244159560 0x7f42440aa530 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f4240009f70 tx=0x7f4240009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647f4db3b190ffb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:35.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.630+0000 7f423bfff700 1 -- 172.21.15.92:0/3365692699 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424000e070 con 0x7f4244159560 2023-10-06T10:44:35.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.630+0000 7f423bfff700 1 -- 172.21.15.92:0/3365692699 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424000ba90 con 0x7f4244159560 2023-10-06T10:44:35.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.630+0000 7f423bfff700 1 -- 172.21.15.92:0/3365692699 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42400092e0 con 0x7f4244159560 2023-10-06T10:44:35.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.630+0000 7f424b763700 1 -- 172.21.15.92:0/3365692699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4244159560 msgr2=0x7f42440aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:35.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.630+0000 7f424b763700 1 --2- 172.21.15.92:0/3365692699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4244159560 0x7f42440aa530 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f4240009f70 tx=0x7f4240009840 comp rx=0 tx=0).stop 2023-10-06T10:44:35.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.630+0000 7f424b763700 1 -- 172.21.15.92:0/3365692699 shutdown_connections 2023-10-06T10:44:35.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.630+0000 7f424b763700 1 --2- 172.21.15.92:0/3365692699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4244159560 0x7f42440aa530 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.630+0000 7f424b763700 1 --2- 172.21.15.92:0/3365692699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42440b0760 0x7f42440b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.630+0000 7f424b763700 1 --2- 172.21.15.92:0/3365692699 >> v2:172.21.15.124:3300/0 conn(0x7f42440b49a0 0x7f42440aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.630+0000 7f424b763700 1 -- 172.21.15.92:0/3365692699 >> 172.21.15.92:0/3365692699 conn(0x7f42440997f0 msgr2=0x7f424406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:35.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.631+0000 7f424b763700 1 -- 172.21.15.92:0/3365692699 shutdown_connections 2023-10-06T10:44:35.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.631+0000 7f424b763700 1 -- 172.21.15.92:0/3365692699 wait complete. 2023-10-06T10:44:35.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.632+0000 7f424b763700 1 Processor -- start 2023-10-06T10:44:35.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.632+0000 7f424b763700 1 -- start start 2023-10-06T10:44:35.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.632+0000 7f424b763700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42440b0760 0x7f42441574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:35.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.632+0000 7f424b763700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42440b49a0 0x7f42441579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:35.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.632+0000 7f424b763700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4244159560 0x7f424416a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:35.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.632+0000 7f424b763700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424416d560 con 0x7f42440b0760 2023-10-06T10:44:35.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.632+0000 7f424b763700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424416e0a0 con 0x7f4244159560 2023-10-06T10:44:35.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.632+0000 7f424b763700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f424416d770 con 0x7f42440b49a0 2023-10-06T10:44:35.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.633+0000 7f42494ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42440b0760 0x7f42441574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:35.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.633+0000 7f42494ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42440b0760 0x7f42441574b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46322/0 (socket says 172.21.15.92:46322) 2023-10-06T10:44:35.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.633+0000 7f42494ff700 1 -- 172.21.15.92:0/2036090951 learned_addr learned my addr 172.21.15.92:0/2036090951 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:35.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.633+0000 7f4249d00700 1 --2- 172.21.15.92:0/2036090951 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4244159560 0x7f424416a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:35.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.633+0000 7f4248cfe700 1 --2- 172.21.15.92:0/2036090951 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42440b49a0 0x7f42441579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:35.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.633+0000 7f42494ff700 1 -- 172.21.15.92:0/2036090951 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42440b49a0 msgr2=0x7f42441579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:35.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.633+0000 7f42494ff700 1 --2- 172.21.15.92:0/2036090951 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42440b49a0 0x7f42441579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.633+0000 7f42494ff700 1 -- 172.21.15.92:0/2036090951 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4244159560 msgr2=0x7f424416a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:35.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.633+0000 7f42494ff700 1 --2- 172.21.15.92:0/2036090951 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4244159560 0x7f424416a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.633+0000 7f42494ff700 1 -- 172.21.15.92:0/2036090951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4240009ae0 con 0x7f42440b0760 2023-10-06T10:44:35.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.633+0000 7f42494ff700 1 --2- 172.21.15.92:0/2036090951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42440b0760 0x7f42441574b0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f424000b770 tx=0x7f4240018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:35.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.633+0000 7f42397fa700 1 -- 172.21.15.92:0/2036090951 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42400089e0 con 0x7f42440b0760 2023-10-06T10:44:35.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.633+0000 7f42397fa700 1 -- 172.21.15.92:0/2036090951 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4240008b60 con 0x7f42440b0760 2023-10-06T10:44:35.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.633+0000 7f424b763700 1 -- 172.21.15.92:0/2036090951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f424416da50 con 0x7f42440b0760 2023-10-06T10:44:35.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.633+0000 7f42397fa700 1 -- 172.21.15.92:0/2036090951 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240025460 con 0x7f42440b0760 2023-10-06T10:44:35.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.633+0000 7f424b763700 1 -- 172.21.15.92:0/2036090951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4244156d30 con 0x7f42440b0760 2023-10-06T10:44:35.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.634+0000 7f424b763700 1 -- 172.21.15.92:0/2036090951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f424416e380 con 0x7f42440b0760 2023-10-06T10:44:35.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.634+0000 7f42397fa700 1 -- 172.21.15.92:0/2036090951 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4240003040 con 0x7f42440b0760 2023-10-06T10:44:35.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.635+0000 7f42397fa700 1 --2- 172.21.15.92:0/2036090951 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4228066f30 0x7f42280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:35.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.635+0000 7f42397fa700 1 -- 172.21.15.92:0/2036090951 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f4240085d40 con 0x7f42440b0760 2023-10-06T10:44:35.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.635+0000 7f4248cfe700 1 --2- 172.21.15.92:0/2036090951 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4228066f30 0x7f42280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:35.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.635+0000 7f4248cfe700 1 --2- 172.21.15.92:0/2036090951 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4228066f30 0x7f42280693e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f42441565c0 tx=0x7f4234008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:35.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.639+0000 7f42397fa700 1 -- 172.21.15.92:0/2036090951 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4240052dc0 con 0x7f42440b0760 2023-10-06T10:44:35.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.861+0000 7f424b763700 1 -- 172.21.15.92:0/2036090951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4244064a90 con 0x7f42440b0760 2023-10-06T10:44:35.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.862+0000 7f42397fa700 1 -- 172.21.15.92:0/2036090951 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4642 (secure 0 0 0) 0x7f42400563e0 con 0x7f42440b0760 2023-10-06T10:44:35.864 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:35.864 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2122441982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2122441982},{"type":"v1","addr":"172.21.15.124:6835","nonce":2122441982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:47.850832+0000","modified":"2023-10-06T10:44:20.587653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7701,"mds_1":7689,"mds_2":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6836/3605297930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3605297930},{"type":"v1","addr":"172.21.15.92:6836","nonce":3605297930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7689":{"gid":7689,"name":"c","rank":1,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.92:6837/4047304116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4047304116},{"type":"v1","addr":"172.21.15.92:6837","nonce":4047304116}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7701":{"gid":7701,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":11,"addr":"172.21.15.124:6834/4203118992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":4203118992},{"type":"v1","addr":"172.21.15.124:6834","nonce":4203118992}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T10:44:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.864+0000 7f424b763700 1 -- 172.21.15.92:0/2036090951 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4228066f30 msgr2=0x7f42280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.864+0000 7f424b763700 1 --2- 172.21.15.92:0/2036090951 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4228066f30 0x7f42280693e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f42441565c0 tx=0x7f4234008040 comp rx=0 tx=0).stop 2023-10-06T10:44:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.864+0000 7f424b763700 1 -- 172.21.15.92:0/2036090951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42440b0760 msgr2=0x7f42441574b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.864+0000 7f424b763700 1 --2- 172.21.15.92:0/2036090951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42440b0760 0x7f42441574b0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f424000b770 tx=0x7f4240018000 comp rx=0 tx=0).stop 2023-10-06T10:44:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.864+0000 7f424b763700 1 -- 172.21.15.92:0/2036090951 shutdown_connections 2023-10-06T10:44:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.864+0000 7f424b763700 1 --2- 172.21.15.92:0/2036090951 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4228066f30 0x7f42280693e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.864+0000 7f424b763700 1 --2- 172.21.15.92:0/2036090951 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4244159560 0x7f424416a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.864+0000 7f424b763700 1 --2- 172.21.15.92:0/2036090951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42440b0760 0x7f42441574b0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.864+0000 7f424b763700 1 --2- 172.21.15.92:0/2036090951 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42440b49a0 0x7f42441579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.864+0000 7f424b763700 1 -- 172.21.15.92:0/2036090951 >> 172.21.15.92:0/2036090951 conn(0x7f42440997f0 msgr2=0x7f42440b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.864+0000 7f424b763700 1 -- 172.21.15.92:0/2036090951 shutdown_connections 2023-10-06T10:44:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.864+0000 7f424b763700 1 -- 172.21.15.92:0/2036090951 wait complete. 2023-10-06T10:44:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 92 2023-10-06T10:44:35.878 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:44:35.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.990+0000 7f692517b700 1 Processor -- start 2023-10-06T10:44:35.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.990+0000 7f692517b700 1 -- start start 2023-10-06T10:44:35.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.991+0000 7f692517b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f69200b25e0 0x7f69200b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:35.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.991+0000 7f692517b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69200b29b0 0x7f6920159120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:35.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.991+0000 7f692517b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69200cbfa0 con 0x7f6920159660 2023-10-06T10:44:35.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.991+0000 7f692517b700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69200cb390 con 0x7f69200b25e0 2023-10-06T10:44:35.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.991+0000 7f692517b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69200cb670 con 0x7f69200b29b0 2023-10-06T10:44:35.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.991+0000 7f691f59e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6920159660 0x7f6920068540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:42944/0 (socket says 172.21.15.92:42944) 2023-10-06T10:44:35.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.991+0000 7f691f59e700 1 -- 172.21.15.92:0/537463713 learned_addr learned my addr 172.21.15.92:0/537463713 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:35.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.991+0000 7f691e59c700 1 --2- 172.21.15.92:0/537463713 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69200b29b0 0x7f6920159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:35.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.991+0000 7f691ed9d700 1 --2- 172.21.15.92:0/537463713 >> v2:172.21.15.124:3300/0 conn(0x7f69200b25e0 0x7f69200b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:35.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.991+0000 7f691dd9b700 1 -- 172.21.15.92:0/537463713 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780869840 0 0) 0x7f69200cbfa0 con 0x7f6920159660 2023-10-06T10:44:35.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691dd9b700 1 -- 172.21.15.92:0/537463713 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6908003570 con 0x7f6920159660 2023-10-06T10:44:35.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691dd9b700 1 -- 172.21.15.92:0/537463713 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3522559999 0 0) 0x7f6908003570 con 0x7f6920159660 2023-10-06T10:44:35.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691dd9b700 1 -- 172.21.15.92:0/537463713 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69200b29b0 msgr2=0x7f6920159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:35.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691dd9b700 1 --2- 172.21.15.92:0/537463713 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69200b29b0 0x7f6920159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691dd9b700 1 -- 172.21.15.92:0/537463713 >> v2:172.21.15.124:3300/0 conn(0x7f69200b25e0 msgr2=0x7f69200b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:35.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691dd9b700 1 --2- 172.21.15.92:0/537463713 >> v2:172.21.15.124:3300/0 conn(0x7f69200b25e0 0x7f69200b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691dd9b700 1 -- 172.21.15.92:0/537463713 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69200cb950 con 0x7f6920159660 2023-10-06T10:44:35.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691dd9b700 1 -- 172.21.15.92:0/537463713 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f69180017f0 con 0x7f6920159660 2023-10-06T10:44:35.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691dd9b700 1 -- 172.21.15.92:0/537463713 >> v1:172.21.15.92:6789/0 conn(0x7f6920159660 legacy=0x7f6920068540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:35.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691dd9b700 1 --2- 172.21.15.92:0/537463713 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69200b29b0 0x7f6908003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:35.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691dd9b700 1 --2- 172.21.15.92:0/537463713 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6908003fe0 0x7f6908004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:35.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691dd9b700 1 --2- 172.21.15.92:0/537463713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6908004d50 0x7f69080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:35.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691dd9b700 1 -- 172.21.15.92:0/537463713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69200cb390 con 0x7f6908004d50 2023-10-06T10:44:35.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691dd9b700 1 -- 172.21.15.92:0/537463713 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69200cb670 con 0x7f6908003fe0 2023-10-06T10:44:35.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691dd9b700 1 -- 172.21.15.92:0/537463713 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690800ba90 con 0x7f69200b29b0 2023-10-06T10:44:35.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691f59e700 1 --2- 172.21.15.92:0/537463713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6908004d50 0x7f69080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:35.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691ed9d700 1 --2- 172.21.15.92:0/537463713 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69200b29b0 0x7f6908003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:35.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691e59c700 1 --2- 172.21.15.92:0/537463713 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6908003fe0 0x7f6908004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:35.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691f59e700 1 -- 172.21.15.92:0/537463713 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69200b29b0 msgr2=0x7f6908003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:35.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691f59e700 1 --2- 172.21.15.92:0/537463713 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69200b29b0 0x7f6908003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691f59e700 1 -- 172.21.15.92:0/537463713 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6908003fe0 msgr2=0x7f6908004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:35.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691f59e700 1 --2- 172.21.15.92:0/537463713 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6908003fe0 0x7f6908004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:35.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691f59e700 1 -- 172.21.15.92:0/537463713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6908003570 con 0x7f6908004d50 2023-10-06T10:44:35.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691f59e700 1 --2- 172.21.15.92:0/537463713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6908004d50 0x7f69080051c0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f691800a970 tx=0x7f6918002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8849adc6193ae493 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:35.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.992+0000 7f691dd9b700 1 -- 172.21.15.92:0/537463713 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6918012440 con 0x7f6908004d50 2023-10-06T10:44:35.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.993+0000 7f691dd9b700 1 -- 172.21.15.92:0/537463713 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69180125c0 con 0x7f6908004d50 2023-10-06T10:44:35.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.993+0000 7f691dd9b700 1 -- 172.21.15.92:0/537463713 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6918012980 con 0x7f6908004d50 2023-10-06T10:44:35.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.993+0000 7f692517b700 1 -- 172.21.15.92:0/537463713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6908004d50 msgr2=0x7f69080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:35.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.993+0000 7f692517b700 1 --2- 172.21.15.92:0/537463713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6908004d50 0x7f69080051c0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f691800a970 tx=0x7f6918002210 comp rx=0 tx=0).stop 2023-10-06T10:44:35.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.993+0000 7f691f59e700 1 -- 172.21.15.92:0/537463713 reap_dead start 2023-10-06T10:44:35.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.993+0000 7f692517b700 1 -- 172.21.15.92:0/537463713 shutdown_connections 2023-10-06T10:44:36.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.993+0000 7f692517b700 1 -- 172.21.15.92:0/537463713 >> 172.21.15.92:0/537463713 conn(0x7f692006ece0 msgr2=0x7f692006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:36.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.993+0000 7f692517b700 1 -- 172.21.15.92:0/537463713 shutdown_connections 2023-10-06T10:44:36.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.993+0000 7f692517b700 1 -- 172.21.15.92:0/537463713 wait complete. 2023-10-06T10:44:36.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.994+0000 7f692517b700 1 Processor -- start 2023-10-06T10:44:36.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.994+0000 7f692517b700 1 -- start start 2023-10-06T10:44:36.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.995+0000 7f692517b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6920148b10 0x7f6920155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.995+0000 7f692517b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f692015bef0 0x7f69201562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.995+0000 7f692517b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6920154940 0x7f69201696d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.995+0000 7f692517b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692016d7b0 con 0x7f692015bef0 2023-10-06T10:44:36.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.995+0000 7f692517b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692016dc40 con 0x7f6920154940 2023-10-06T10:44:36.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.995+0000 7f692517b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69200c8ad0 con 0x7f6920148b10 2023-10-06T10:44:36.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.995+0000 7f691e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f692015bef0 0x7f69201562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:36.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.995+0000 7f691e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f692015bef0 0x7f69201562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46328/0 (socket says 172.21.15.92:46328) 2023-10-06T10:44:36.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.995+0000 7f691e59c700 1 -- 172.21.15.92:0/3785343459 learned_addr learned my addr 172.21.15.92:0/3785343459 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:36.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.995+0000 7f691f59e700 1 --2- 172.21.15.92:0/3785343459 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6920154940 0x7f69201696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:36.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.995+0000 7f691ed9d700 1 --2- 172.21.15.92:0/3785343459 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6920148b10 0x7f6920155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:36.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.995+0000 7f691e59c700 1 -- 172.21.15.92:0/3785343459 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6920148b10 msgr2=0x7f6920155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:36.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.995+0000 7f691e59c700 1 --2- 172.21.15.92:0/3785343459 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6920148b10 0x7f6920155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.995+0000 7f691e59c700 1 -- 172.21.15.92:0/3785343459 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6920154940 msgr2=0x7f69201696d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:36.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.995+0000 7f691e59c700 1 --2- 172.21.15.92:0/3785343459 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6920154940 0x7f69201696d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.995+0000 7f691e59c700 1 -- 172.21.15.92:0/3785343459 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692016ce80 con 0x7f692015bef0 2023-10-06T10:44:36.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.996+0000 7f691e59c700 1 --2- 172.21.15.92:0/3785343459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f692015bef0 0x7f69201562f0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f6914006ac0 tx=0x7f69140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:36.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.996+0000 7f6906ffd700 1 -- 172.21.15.92:0/3785343459 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691400f8e0 con 0x7f692015bef0 2023-10-06T10:44:36.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.996+0000 7f6906ffd700 1 -- 172.21.15.92:0/3785343459 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691400fa40 con 0x7f692015bef0 2023-10-06T10:44:36.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.996+0000 7f6906ffd700 1 -- 172.21.15.92:0/3785343459 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691400fdc0 con 0x7f692015bef0 2023-10-06T10:44:36.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.996+0000 7f692517b700 1 -- 172.21.15.92:0/3785343459 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692016d160 con 0x7f692015bef0 2023-10-06T10:44:36.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.996+0000 7f692517b700 1 -- 172.21.15.92:0/3785343459 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6920154ff0 con 0x7f692015bef0 2023-10-06T10:44:36.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.997+0000 7f692517b700 1 -- 172.21.15.92:0/3785343459 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6920064a90 con 0x7f692015bef0 2023-10-06T10:44:36.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.997+0000 7f6906ffd700 1 -- 172.21.15.92:0/3785343459 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6914014650 con 0x7f692015bef0 2023-10-06T10:44:36.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.997+0000 7f6906ffd700 1 --2- 172.21.15.92:0/3785343459 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f690c066f30 0x7f690c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.997+0000 7f6906ffd700 1 -- 172.21.15.92:0/3785343459 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6914018030 con 0x7f692015bef0 2023-10-06T10:44:36.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.998+0000 7f691ed9d700 1 --2- 172.21.15.92:0/3785343459 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f690c066f30 0x7f690c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:36.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:35.998+0000 7f691ed9d700 1 --2- 172.21.15.92:0/3785343459 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f690c066f30 0x7f690c0693e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6910006e50 tx=0x7f6910009120 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:36.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.002+0000 7f6906ffd700 1 -- 172.21.15.92:0/3785343459 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69140520d0 con 0x7f692015bef0 2023-10-06T10:44:36.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.187+0000 7f692517b700 1 -- 172.21.15.92:0/3785343459 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f692016d8f0 con 0x7f692015bef0 2023-10-06T10:44:36.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.188+0000 7f6906ffd700 1 -- 172.21.15.92:0/3785343459 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+17074 (secure 0 0 0) 0x7f69140556f0 con 0x7f692015bef0 2023-10-06T10:44:36.190 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:36.190 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":90,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:43:49.334267+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:43:45.492396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-06T10:43:46.811317+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:44:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.190+0000 7f692517b700 1 -- 172.21.15.92:0/3785343459 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f690c066f30 msgr2=0x7f690c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.190+0000 7f692517b700 1 --2- 172.21.15.92:0/3785343459 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f690c066f30 0x7f690c0693e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6910006e50 tx=0x7f6910009120 comp rx=0 tx=0).stop 2023-10-06T10:44:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.190+0000 7f692517b700 1 -- 172.21.15.92:0/3785343459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f692015bef0 msgr2=0x7f69201562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.190+0000 7f692517b700 1 --2- 172.21.15.92:0/3785343459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f692015bef0 0x7f69201562f0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f6914006ac0 tx=0x7f69140091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.190+0000 7f692517b700 1 -- 172.21.15.92:0/3785343459 shutdown_connections 2023-10-06T10:44:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.190+0000 7f692517b700 1 --2- 172.21.15.92:0/3785343459 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f690c066f30 0x7f690c0693e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.190+0000 7f692517b700 1 --2- 172.21.15.92:0/3785343459 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6920154940 0x7f69201696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.190+0000 7f692517b700 1 --2- 172.21.15.92:0/3785343459 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6920148b10 0x7f6920155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.190+0000 7f692517b700 1 --2- 172.21.15.92:0/3785343459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f692015bef0 0x7f69201562f0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.190+0000 7f692517b700 1 -- 172.21.15.92:0/3785343459 >> 172.21.15.92:0/3785343459 conn(0x7f692006ece0 msgr2=0x7f6920156d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.191+0000 7f692517b700 1 -- 172.21.15.92:0/3785343459 shutdown_connections 2023-10-06T10:44:36.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.191+0000 7f692517b700 1 -- 172.21.15.92:0/3785343459 wait complete. 2023-10-06T10:44:36.204 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T10:38:09.680559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T10:38:10.120786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T10:43:45.492396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T10:43:46.811317+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T10:44:36.205 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T10:44:36.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.320+0000 7f3c7759e700 1 Processor -- start 2023-10-06T10:44:36.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.320+0000 7f3c7759e700 1 -- start start 2023-10-06T10:44:36.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.321+0000 7f3c7759e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3c780cd3c0 0x7f3c780ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.321+0000 7f3c7759e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3c780b29f0 0x7f3c780cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.321+0000 7f3c7759e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c780cc970 con 0x7f3c780b2620 2023-10-06T10:44:36.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.321+0000 7f3c7759e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c780cf520 con 0x7f3c780cd3c0 2023-10-06T10:44:36.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.321+0000 7f3c7759e700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c780cbd60 con 0x7f3c780b29f0 2023-10-06T10:44:36.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.321+0000 7f3c75d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3c780cd3c0 0x7f3c780ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:36.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.321+0000 7f3c75d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3c780cd3c0 0x7f3c780ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46330/0 (socket says 172.21.15.92:46330) 2023-10-06T10:44:36.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.321+0000 7f3c75d9b700 1 -- 172.21.15.92:0/1305372401 learned_addr learned my addr 172.21.15.92:0/1305372401 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:36.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.321+0000 7f3c75d9b700 1 -- 172.21.15.92:0/1305372401 >> v2:172.21.15.124:3300/0 conn(0x7f3c780b29f0 msgr2=0x7f3c780cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:36.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.321+0000 7f3c75d9b700 1 --2- 172.21.15.92:0/1305372401 >> v2:172.21.15.124:3300/0 conn(0x7f3c780b29f0 0x7f3c780cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.321+0000 7f3c75d9b700 1 -- 172.21.15.92:0/1305372401 >> v1:172.21.15.124:6789/0 conn(0x7f3c780b2620 legacy=0x7f3c780cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:36.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.321+0000 7f3c75d9b700 1 -- 172.21.15.92:0/1305372401 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c780cc040 con 0x7f3c780cd3c0 2023-10-06T10:44:36.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.321+0000 7f3c75d9b700 1 --2- 172.21.15.92:0/1305372401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c780cd3c0 0x7f3c780ceaa0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f3c78099b90 tx=0x7f3c6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75af618ac39f5c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:36.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.321+0000 7f3c7559a700 1 -- 172.21.15.92:0/1305372401 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c00e070 con 0x7f3c780cd3c0 2023-10-06T10:44:36.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.322+0000 7f3c7559a700 1 -- 172.21.15.92:0/1305372401 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6c00ba10 con 0x7f3c780cd3c0 2023-10-06T10:44:36.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.322+0000 7f3c7559a700 1 -- 172.21.15.92:0/1305372401 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0092e0 con 0x7f3c780cd3c0 2023-10-06T10:44:36.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.322+0000 7f3c7759e700 1 -- 172.21.15.92:0/1305372401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c780cd3c0 msgr2=0x7f3c780ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:36.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.322+0000 7f3c7759e700 1 --2- 172.21.15.92:0/1305372401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c780cd3c0 0x7f3c780ceaa0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f3c78099b90 tx=0x7f3c6c009840 comp rx=0 tx=0).stop 2023-10-06T10:44:36.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.322+0000 7f3c7759e700 1 -- 172.21.15.92:0/1305372401 shutdown_connections 2023-10-06T10:44:36.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.322+0000 7f3c7759e700 1 --2- 172.21.15.92:0/1305372401 >> v2:172.21.15.124:3300/0 conn(0x7f3c780b29f0 0x7f3c780cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.322+0000 7f3c7759e700 1 --2- 172.21.15.92:0/1305372401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c780cd3c0 0x7f3c780ceaa0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.322+0000 7f3c7759e700 1 -- 172.21.15.92:0/1305372401 >> 172.21.15.92:0/1305372401 conn(0x7f3c78099520 msgr2=0x7f3c7806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:36.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.322+0000 7f3c7759e700 1 -- 172.21.15.92:0/1305372401 shutdown_connections 2023-10-06T10:44:36.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.322+0000 7f3c7759e700 1 -- 172.21.15.92:0/1305372401 wait complete. 2023-10-06T10:44:36.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.323+0000 7f3c7759e700 1 Processor -- start 2023-10-06T10:44:36.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.323+0000 7f3c7759e700 1 -- start start 2023-10-06T10:44:36.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.324+0000 7f3c7759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c780b2620 0x7f3c7815c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.324+0000 7f3c7759e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c780b29f0 0x7f3c780c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.324+0000 7f3c7759e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c780cd3c0 0x7f3c7814ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.324+0000 7f3c7759e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c780c9370 con 0x7f3c780b2620 2023-10-06T10:44:36.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.324+0000 7f3c7759e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c780c8fe0 con 0x7f3c780b29f0 2023-10-06T10:44:36.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.324+0000 7f3c7759e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7815cd60 con 0x7f3c780cd3c0 2023-10-06T10:44:36.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.324+0000 7f3c7659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c780b2620 0x7f3c7815c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:36.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.324+0000 7f3c7659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c780b2620 0x7f3c7815c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46336/0 (socket says 172.21.15.92:46336) 2023-10-06T10:44:36.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.324+0000 7f3c7659c700 1 -- 172.21.15.92:0/232952974 learned_addr learned my addr 172.21.15.92:0/232952974 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:36.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.324+0000 7f3c7659c700 1 -- 172.21.15.92:0/232952974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c780cd3c0 msgr2=0x7f3c7814ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.324+0000 7f3c7659c700 1 --2- 172.21.15.92:0/232952974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c780cd3c0 0x7f3c7814ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.324+0000 7f3c7659c700 1 -- 172.21.15.92:0/232952974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c780b29f0 msgr2=0x7f3c780c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.324+0000 7f3c7659c700 1 --2- 172.21.15.92:0/232952974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c780b29f0 0x7f3c780c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.324+0000 7f3c7659c700 1 -- 172.21.15.92:0/232952974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6c009ae0 con 0x7f3c780b2620 2023-10-06T10:44:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.324+0000 7f3c7659c700 1 --2- 172.21.15.92:0/232952974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c780b2620 0x7f3c7815c780 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000b5e0 tx=0x7f3c700090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.325+0000 7f3c667fc700 1 -- 172.21.15.92:0/232952974 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7000f510 con 0x7f3c780b2620 2023-10-06T10:44:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.325+0000 7f3c667fc700 1 -- 172.21.15.92:0/232952974 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7000f690 con 0x7f3c780b2620 2023-10-06T10:44:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.325+0000 7f3c7759e700 1 -- 172.21.15.92:0/232952974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c780c95c0 con 0x7f3c780b2620 2023-10-06T10:44:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.325+0000 7f3c667fc700 1 -- 172.21.15.92:0/232952974 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7000fa50 con 0x7f3c780b2620 2023-10-06T10:44:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.325+0000 7f3c7759e700 1 -- 172.21.15.92:0/232952974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c780c9b80 con 0x7f3c780b2620 2023-10-06T10:44:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.326+0000 7f3c7759e700 1 -- 172.21.15.92:0/232952974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c780c9d70 con 0x7f3c780b2620 2023-10-06T10:44:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.326+0000 7f3c667fc700 1 -- 172.21.15.92:0/232952974 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c7001a400 con 0x7f3c780b2620 2023-10-06T10:44:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.326+0000 7f3c667fc700 1 --2- 172.21.15.92:0/232952974 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c58066f30 0x7f3c580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.326+0000 7f3c667fc700 1 -- 172.21.15.92:0/232952974 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3c700842a0 con 0x7f3c780b2620 2023-10-06T10:44:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.326+0000 7f3c75d9b700 1 --2- 172.21.15.92:0/232952974 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c58066f30 0x7f3c580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.327+0000 7f3c75d9b700 1 --2- 172.21.15.92:0/232952974 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c58066f30 0x7f3c580693e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3c780cd7c0 tx=0x7f3c6c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:36.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.330+0000 7f3c667fc700 1 -- 172.21.15.92:0/232952974 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c700512f0 con 0x7f3c780b2620 2023-10-06T10:44:36.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.517+0000 7f3c7759e700 1 -- 172.21.15.92:0/232952974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3c7805f230 con 0x7f3c780b2620 2023-10-06T10:44:36.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.518+0000 7f3c667fc700 1 -- 172.21.15.92:0/232952974 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v90) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3c70054910 con 0x7f3c780b2620 2023-10-06T10:44:36.520 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T10:44:36.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.520+0000 7f3c7759e700 1 -- 172.21.15.92:0/232952974 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c58066f30 msgr2=0x7f3c580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:36.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.520+0000 7f3c7759e700 1 --2- 172.21.15.92:0/232952974 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c58066f30 0x7f3c580693e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3c780cd7c0 tx=0x7f3c6c017000 comp rx=0 tx=0).stop 2023-10-06T10:44:36.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.520+0000 7f3c7759e700 1 -- 172.21.15.92:0/232952974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c780b2620 msgr2=0x7f3c7815c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:36.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.520+0000 7f3c7759e700 1 --2- 172.21.15.92:0/232952974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c780b2620 0x7f3c7815c780 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000b5e0 tx=0x7f3c700090f0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.520+0000 7f3c7759e700 1 -- 172.21.15.92:0/232952974 shutdown_connections 2023-10-06T10:44:36.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.520+0000 7f3c7759e700 1 --2- 172.21.15.92:0/232952974 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c58066f30 0x7f3c580693e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.520+0000 7f3c7759e700 1 --2- 172.21.15.92:0/232952974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c780cd3c0 0x7f3c7814ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.520+0000 7f3c7759e700 1 --2- 172.21.15.92:0/232952974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c780b2620 0x7f3c7815c780 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.520+0000 7f3c7759e700 1 --2- 172.21.15.92:0/232952974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c780b29f0 0x7f3c780c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.520+0000 7f3c7759e700 1 -- 172.21.15.92:0/232952974 >> 172.21.15.92:0/232952974 conn(0x7f3c78099520 msgr2=0x7f3c780c6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:36.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.520+0000 7f3c7759e700 1 -- 172.21.15.92:0/232952974 shutdown_connections 2023-10-06T10:44:36.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.520+0000 7f3c7759e700 1 -- 172.21.15.92:0/232952974 wait complete. 2023-10-06T10:44:36.534 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T10:44:36.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.649+0000 7f221b32b700 1 Processor -- start 2023-10-06T10:44:36.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.649+0000 7f221b32b700 1 -- start start 2023-10-06T10:44:36.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.650+0000 7f221b32b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f221415aec0 0x7f2214157470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.650+0000 7f221b32b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f22140b2110 0x7f22140b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.650+0000 7f221b32b700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f221406a3c0 con 0x7f22141570a0 2023-10-06T10:44:36.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.650+0000 7f221b32b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22140b2a70 con 0x7f22140b2110 2023-10-06T10:44:36.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.650+0000 7f221b32b700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f221406ebf0 con 0x7f221415aec0 2023-10-06T10:44:36.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.651+0000 7f221ab2a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f22140b2110 0x7f22140b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:36.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.651+0000 7f221ab2a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f22140b2110 0x7f22140b2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46340/0 (socket says 172.21.15.92:46340) 2023-10-06T10:44:36.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.651+0000 7f221ab2a700 1 -- 172.21.15.92:0/2524487430 learned_addr learned my addr 172.21.15.92:0/2524487430 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:36.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.651+0000 7f2219b28700 1 --2- 172.21.15.92:0/2524487430 >> v2:172.21.15.124:3300/0 conn(0x7f221415aec0 0x7f2214157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:36.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.651+0000 7f221ab2a700 1 -- 172.21.15.92:0/2524487430 >> v2:172.21.15.124:3300/0 conn(0x7f221415aec0 msgr2=0x7f2214157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:36.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.651+0000 7f221ab2a700 1 --2- 172.21.15.92:0/2524487430 >> v2:172.21.15.124:3300/0 conn(0x7f221415aec0 0x7f2214157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.651+0000 7f221ab2a700 1 -- 172.21.15.92:0/2524487430 >> v1:172.21.15.124:6789/0 conn(0x7f22141570a0 legacy=0x7f221406e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:36.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.651+0000 7f221ab2a700 1 -- 172.21.15.92:0/2524487430 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221406eed0 con 0x7f22140b2110 2023-10-06T10:44:36.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.651+0000 7f221ab2a700 1 --2- 172.21.15.92:0/2524487430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22140b2110 0x7f22140b2530 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f221406e080 tx=0x7f220c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d7c327053db10e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.652+0000 7f2219327700 1 -- 172.21.15.92:0/2524487430 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c00e070 con 0x7f22140b2110 2023-10-06T10:44:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.652+0000 7f2219327700 1 -- 172.21.15.92:0/2524487430 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220c00ba20 con 0x7f22140b2110 2023-10-06T10:44:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.652+0000 7f2219327700 1 -- 172.21.15.92:0/2524487430 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c0092e0 con 0x7f22140b2110 2023-10-06T10:44:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.652+0000 7f221b32b700 1 -- 172.21.15.92:0/2524487430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22140b2110 msgr2=0x7f22140b2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.652+0000 7f221b32b700 1 --2- 172.21.15.92:0/2524487430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22140b2110 0x7f22140b2530 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f221406e080 tx=0x7f220c009840 comp rx=0 tx=0).stop 2023-10-06T10:44:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.653+0000 7f221b32b700 1 -- 172.21.15.92:0/2524487430 shutdown_connections 2023-10-06T10:44:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.653+0000 7f221b32b700 1 --2- 172.21.15.92:0/2524487430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22140b2110 0x7f22140b2530 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.653+0000 7f221b32b700 1 --2- 172.21.15.92:0/2524487430 >> v2:172.21.15.124:3300/0 conn(0x7f221415aec0 0x7f2214157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.653+0000 7f221b32b700 1 -- 172.21.15.92:0/2524487430 >> 172.21.15.92:0/2524487430 conn(0x7f2214069140 msgr2=0x7f22140aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.653+0000 7f221b32b700 1 -- 172.21.15.92:0/2524487430 shutdown_connections 2023-10-06T10:44:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.653+0000 7f221b32b700 1 -- 172.21.15.92:0/2524487430 wait complete. 2023-10-06T10:44:36.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.654+0000 7f221b32b700 1 Processor -- start 2023-10-06T10:44:36.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.654+0000 7f221b32b700 1 -- start start 2023-10-06T10:44:36.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.654+0000 7f221b32b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22140b2110 0x7f2214151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.654+0000 7f221b32b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22141570a0 0x7f22141671b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.654+0000 7f221b32b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f221415aec0 0x7f2214155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.654+0000 7f221b32b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22141583a0 con 0x7f221415aec0 2023-10-06T10:44:36.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.654+0000 7f221b32b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2214151bc0 con 0x7f22140b2110 2023-10-06T10:44:36.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.654+0000 7f221b32b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2214157a70 con 0x7f22141570a0 2023-10-06T10:44:36.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.654+0000 7f221ab2a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f221415aec0 0x7f2214155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:36.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.654+0000 7f221ab2a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f221415aec0 0x7f2214155f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46354/0 (socket says 172.21.15.92:46354) 2023-10-06T10:44:36.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.654+0000 7f221ab2a700 1 -- 172.21.15.92:0/2934339200 learned_addr learned my addr 172.21.15.92:0/2934339200 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:36.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.655+0000 7f221a329700 1 --2- 172.21.15.92:0/2934339200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22140b2110 0x7f2214151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:36.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.655+0000 7f2219b28700 1 --2- 172.21.15.92:0/2934339200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22141570a0 0x7f22141671b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:36.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.655+0000 7f221ab2a700 1 -- 172.21.15.92:0/2934339200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22141570a0 msgr2=0x7f22141671b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:36.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.655+0000 7f221ab2a700 1 --2- 172.21.15.92:0/2934339200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22141570a0 0x7f22141671b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.655+0000 7f221ab2a700 1 -- 172.21.15.92:0/2934339200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22140b2110 msgr2=0x7f2214151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:36.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.655+0000 7f221ab2a700 1 --2- 172.21.15.92:0/2934339200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22140b2110 0x7f2214151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.655+0000 7f221ab2a700 1 -- 172.21.15.92:0/2934339200 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220c009ae0 con 0x7f221415aec0 2023-10-06T10:44:36.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.655+0000 7f221ab2a700 1 --2- 172.21.15.92:0/2934339200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f221415aec0 0x7f2214155f60 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f220c009880 tx=0x7f220c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:36.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.655+0000 7f220a7fc700 1 -- 172.21.15.92:0/2934339200 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c008be0 con 0x7f221415aec0 2023-10-06T10:44:36.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.655+0000 7f220a7fc700 1 -- 172.21.15.92:0/2934339200 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220c008d60 con 0x7f221415aec0 2023-10-06T10:44:36.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.655+0000 7f220a7fc700 1 -- 172.21.15.92:0/2934339200 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c0266b0 con 0x7f221415aec0 2023-10-06T10:44:36.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.655+0000 7f221b32b700 1 -- 172.21.15.92:0/2934339200 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2214157d50 con 0x7f221415aec0 2023-10-06T10:44:36.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.655+0000 7f221b32b700 1 -- 172.21.15.92:0/2934339200 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2214154b40 con 0x7f221415aec0 2023-10-06T10:44:36.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.656+0000 7f221b32b700 1 -- 172.21.15.92:0/2934339200 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2214154cd0 con 0x7f221415aec0 2023-10-06T10:44:36.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.656+0000 7f220a7fc700 1 -- 172.21.15.92:0/2934339200 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f220c003040 con 0x7f221415aec0 2023-10-06T10:44:36.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.657+0000 7f220a7fc700 1 --2- 172.21.15.92:0/2934339200 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f21f8066f30 0x7f21f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.657+0000 7f220a7fc700 1 -- 172.21.15.92:0/2934339200 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f220c0861c0 con 0x7f221415aec0 2023-10-06T10:44:36.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.657+0000 7f221a329700 1 --2- 172.21.15.92:0/2934339200 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f21f8066f30 0x7f21f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:36.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.657+0000 7f221a329700 1 --2- 172.21.15.92:0/2934339200 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f21f8066f30 0x7f21f80693e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2210003880 tx=0x7f2210009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:36.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.661+0000 7f220a7fc700 1 -- 172.21.15.92:0/2934339200 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f220c053190 con 0x7f221415aec0 2023-10-06T10:44:36.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.843+0000 7f221b32b700 1 -- 172.21.15.92:0/2934339200 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f221405f230 con 0x7f221415aec0 2023-10-06T10:44:36.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.843+0000 7f220a7fc700 1 -- 172.21.15.92:0/2934339200 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f220c009940 con 0x7f221415aec0 2023-10-06T10:44:36.845 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T10:44:36.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.845+0000 7f221b32b700 1 -- 172.21.15.92:0/2934339200 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f21f8066f30 msgr2=0x7f21f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:36.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.845+0000 7f221b32b700 1 --2- 172.21.15.92:0/2934339200 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f21f8066f30 0x7f21f80693e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2210003880 tx=0x7f2210009040 comp rx=0 tx=0).stop 2023-10-06T10:44:36.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.845+0000 7f221b32b700 1 -- 172.21.15.92:0/2934339200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f221415aec0 msgr2=0x7f2214155f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:36.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.845+0000 7f221b32b700 1 --2- 172.21.15.92:0/2934339200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f221415aec0 0x7f2214155f60 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f220c009880 tx=0x7f220c018000 comp rx=0 tx=0).stop 2023-10-06T10:44:36.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.845+0000 7f221b32b700 1 -- 172.21.15.92:0/2934339200 shutdown_connections 2023-10-06T10:44:36.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.845+0000 7f221b32b700 1 --2- 172.21.15.92:0/2934339200 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f21f8066f30 0x7f21f80693e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.845+0000 7f221b32b700 1 --2- 172.21.15.92:0/2934339200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f221415aec0 0x7f2214155f60 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.845+0000 7f221b32b700 1 --2- 172.21.15.92:0/2934339200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22140b2110 0x7f2214151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.845+0000 7f221b32b700 1 --2- 172.21.15.92:0/2934339200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22141570a0 0x7f22141671b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.845+0000 7f221b32b700 1 -- 172.21.15.92:0/2934339200 >> 172.21.15.92:0/2934339200 conn(0x7f2214069140 msgr2=0x7f22140acab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:36.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.845+0000 7f221b32b700 1 -- 172.21.15.92:0/2934339200 shutdown_connections 2023-10-06T10:44:36.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.845+0000 7f221b32b700 1 -- 172.21.15.92:0/2934339200 wait complete. 2023-10-06T10:44:36.859 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T10:44:36.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.974+0000 7fb559b04700 1 Processor -- start 2023-10-06T10:44:36.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.974+0000 7fb559b04700 1 -- start start 2023-10-06T10:44:36.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.975+0000 7fb559b04700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5541594d0 0x7fb5540cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.975+0000 7fb559b04700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb5541598a0 0x7fb5540cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.975+0000 7fb559b04700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb5540ce7e0 0x7fb5540cec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.975+0000 7fb559b04700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5540cf5d0 con 0x7fb5540ce7e0 2023-10-06T10:44:36.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.975+0000 7fb559b04700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5540cf8b0 con 0x7fb5541598a0 2023-10-06T10:44:36.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.975+0000 7fb559b04700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5540cfb90 con 0x7fb5541594d0 2023-10-06T10:44:36.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.975+0000 7fb553fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb5540ce7e0 0x7fb5540cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:36.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.975+0000 7fb553fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb5540ce7e0 0x7fb5540cec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46362/0 (socket says 172.21.15.92:46362) 2023-10-06T10:44:36.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.975+0000 7fb553fff700 1 -- 172.21.15.92:0/2700141483 learned_addr learned my addr 172.21.15.92:0/2700141483 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:36.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.975+0000 7fb5537fe700 1 --2- 172.21.15.92:0/2700141483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5541594d0 0x7fb5540cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:36.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.975+0000 7fb552ffd700 1 --2- 172.21.15.92:0/2700141483 >> v2:172.21.15.124:3300/0 conn(0x7fb5541598a0 0x7fb5540cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:36.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.975+0000 7fb553fff700 1 -- 172.21.15.92:0/2700141483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5541594d0 msgr2=0x7fb5540cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:36.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.975+0000 7fb553fff700 1 --2- 172.21.15.92:0/2700141483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5541594d0 0x7fb5540cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.975+0000 7fb553fff700 1 -- 172.21.15.92:0/2700141483 >> v2:172.21.15.124:3300/0 conn(0x7fb5541598a0 msgr2=0x7fb5540cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:36.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.975+0000 7fb553fff700 1 --2- 172.21.15.92:0/2700141483 >> v2:172.21.15.124:3300/0 conn(0x7fb5541598a0 0x7fb5540cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.975+0000 7fb553fff700 1 -- 172.21.15.92:0/2700141483 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5540cfe70 con 0x7fb5540ce7e0 2023-10-06T10:44:36.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.975+0000 7fb5537fe700 1 --2- 172.21.15.92:0/2700141483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5541594d0 0x7fb5540cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:36.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.976+0000 7fb553fff700 1 --2- 172.21.15.92:0/2700141483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5540ce7e0 0x7fb5540cec00 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fb554089320 tx=0x7fb54c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a33670ce098d3e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:36.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.976+0000 7fb5527fc700 1 -- 172.21.15.92:0/2700141483 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c00e070 con 0x7fb5540ce7e0 2023-10-06T10:44:36.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.976+0000 7fb5527fc700 1 -- 172.21.15.92:0/2700141483 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54c00ba50 con 0x7fb5540ce7e0 2023-10-06T10:44:36.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.976+0000 7fb5527fc700 1 -- 172.21.15.92:0/2700141483 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c0092e0 con 0x7fb5540ce7e0 2023-10-06T10:44:36.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.976+0000 7fb559b04700 1 -- 172.21.15.92:0/2700141483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5540ce7e0 msgr2=0x7fb5540cec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:36.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.976+0000 7fb559b04700 1 --2- 172.21.15.92:0/2700141483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5540ce7e0 0x7fb5540cec00 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fb554089320 tx=0x7fb54c009840 comp rx=0 tx=0).stop 2023-10-06T10:44:36.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.976+0000 7fb559b04700 1 -- 172.21.15.92:0/2700141483 shutdown_connections 2023-10-06T10:44:36.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.976+0000 7fb559b04700 1 --2- 172.21.15.92:0/2700141483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5540ce7e0 0x7fb5540cec00 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.976+0000 7fb559b04700 1 --2- 172.21.15.92:0/2700141483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5541594d0 0x7fb5540cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.976+0000 7fb559b04700 1 --2- 172.21.15.92:0/2700141483 >> v2:172.21.15.124:3300/0 conn(0x7fb5541598a0 0x7fb5540cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.976+0000 7fb559b04700 1 -- 172.21.15.92:0/2700141483 >> 172.21.15.92:0/2700141483 conn(0x7fb554089e50 msgr2=0x7fb554099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:36.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.976+0000 7fb559b04700 1 -- 172.21.15.92:0/2700141483 shutdown_connections 2023-10-06T10:44:36.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.976+0000 7fb559b04700 1 -- 172.21.15.92:0/2700141483 wait complete. 2023-10-06T10:44:36.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.978+0000 7fb559b04700 1 Processor -- start 2023-10-06T10:44:36.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.978+0000 7fb559b04700 1 -- start start 2023-10-06T10:44:36.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.978+0000 7fb559b04700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5540ce7e0 0x7fb554148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.978+0000 7fb559b04700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5541594d0 0x7fb55414ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.978+0000 7fb559b04700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5541598a0 0x7fb5541504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.978+0000 7fb559b04700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55414e710 con 0x7fb5540ce7e0 2023-10-06T10:44:36.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.978+0000 7fb559b04700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb554150b20 con 0x7fb5541598a0 2023-10-06T10:44:36.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.978+0000 7fb559b04700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb554150e00 con 0x7fb5541594d0 2023-10-06T10:44:36.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.978+0000 7fb5537fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5540ce7e0 0x7fb554148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:36.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.978+0000 7fb5537fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5540ce7e0 0x7fb554148fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46370/0 (socket says 172.21.15.92:46370) 2023-10-06T10:44:36.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.978+0000 7fb5537fe700 1 -- 172.21.15.92:0/358900291 learned_addr learned my addr 172.21.15.92:0/358900291 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:36.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.979+0000 7fb552ffd700 1 --2- 172.21.15.92:0/358900291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5541594d0 0x7fb55414ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:36.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.979+0000 7fb553fff700 1 --2- 172.21.15.92:0/358900291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5541598a0 0x7fb5541504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:36.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.979+0000 7fb5537fe700 1 -- 172.21.15.92:0/358900291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5541594d0 msgr2=0x7fb55414ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:36.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.979+0000 7fb5537fe700 1 --2- 172.21.15.92:0/358900291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5541594d0 0x7fb55414ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.979+0000 7fb5537fe700 1 -- 172.21.15.92:0/358900291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5541598a0 msgr2=0x7fb5541504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:36.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.979+0000 7fb5537fe700 1 --2- 172.21.15.92:0/358900291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5541598a0 0x7fb5541504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:36.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.979+0000 7fb5537fe700 1 -- 172.21.15.92:0/358900291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54c009ae0 con 0x7fb5540ce7e0 2023-10-06T10:44:36.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.979+0000 7fb553fff700 1 --2- 172.21.15.92:0/358900291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5541598a0 0x7fb5541504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:36.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.979+0000 7fb5537fe700 1 --2- 172.21.15.92:0/358900291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5540ce7e0 0x7fb554148fc0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fb54400e8d0 tx=0x7fb544008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:36.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.979+0000 7fb53b7fe700 1 -- 172.21.15.92:0/358900291 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544013410 con 0x7fb5540ce7e0 2023-10-06T10:44:36.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.979+0000 7fb53b7fe700 1 -- 172.21.15.92:0/358900291 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb544013570 con 0x7fb5540ce7e0 2023-10-06T10:44:36.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.979+0000 7fb53b7fe700 1 -- 172.21.15.92:0/358900291 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544013910 con 0x7fb5540ce7e0 2023-10-06T10:44:36.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.979+0000 7fb559b04700 1 -- 172.21.15.92:0/358900291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5541510e0 con 0x7fb5540ce7e0 2023-10-06T10:44:36.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.979+0000 7fb559b04700 1 -- 172.21.15.92:0/358900291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5541516a0 con 0x7fb5540ce7e0 2023-10-06T10:44:36.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.980+0000 7fb559b04700 1 -- 172.21.15.92:0/358900291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb554064a90 con 0x7fb5540ce7e0 2023-10-06T10:44:36.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.980+0000 7fb53b7fe700 1 -- 172.21.15.92:0/358900291 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb544013d00 con 0x7fb5540ce7e0 2023-10-06T10:44:36.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.981+0000 7fb53b7fe700 1 --2- 172.21.15.92:0/358900291 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb540066f30 0x7fb5400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:36.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.981+0000 7fb53b7fe700 1 -- 172.21.15.92:0/358900291 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb544084540 con 0x7fb5540ce7e0 2023-10-06T10:44:36.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.981+0000 7fb552ffd700 1 --2- 172.21.15.92:0/358900291 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb540066f30 0x7fb5400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:36.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.981+0000 7fb552ffd700 1 --2- 172.21.15.92:0/358900291 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb540066f30 0x7fb5400693e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb5540ce470 tx=0x7fb548008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:36.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:36.985+0000 7fb53b7fe700 1 -- 172.21.15.92:0/358900291 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb544051510 con 0x7fb5540ce7e0 2023-10-06T10:44:37.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.167+0000 7fb559b04700 1 -- 172.21.15.92:0/358900291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb5540cf140 con 0x7fb5540ce7e0 2023-10-06T10:44:37.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.167+0000 7fb53b7fe700 1 -- 172.21.15.92:0/358900291 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb544054b30 con 0x7fb5540ce7e0 2023-10-06T10:44:37.169 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T10:44:37.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.169+0000 7fb559b04700 1 -- 172.21.15.92:0/358900291 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb540066f30 msgr2=0x7fb5400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:37.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.169+0000 7fb559b04700 1 --2- 172.21.15.92:0/358900291 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb540066f30 0x7fb5400693e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb5540ce470 tx=0x7fb548008040 comp rx=0 tx=0).stop 2023-10-06T10:44:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.169+0000 7fb559b04700 1 -- 172.21.15.92:0/358900291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5540ce7e0 msgr2=0x7fb554148fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.169+0000 7fb559b04700 1 --2- 172.21.15.92:0/358900291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5540ce7e0 0x7fb554148fc0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fb54400e8d0 tx=0x7fb544008040 comp rx=0 tx=0).stop 2023-10-06T10:44:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.169+0000 7fb559b04700 1 -- 172.21.15.92:0/358900291 shutdown_connections 2023-10-06T10:44:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.169+0000 7fb559b04700 1 --2- 172.21.15.92:0/358900291 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb540066f30 0x7fb5400693e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.169+0000 7fb559b04700 1 --2- 172.21.15.92:0/358900291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5541598a0 0x7fb5541504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.169+0000 7fb559b04700 1 --2- 172.21.15.92:0/358900291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5540ce7e0 0x7fb554148fc0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.169+0000 7fb559b04700 1 --2- 172.21.15.92:0/358900291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5541594d0 0x7fb55414ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.169+0000 7fb559b04700 1 -- 172.21.15.92:0/358900291 >> 172.21.15.92:0/358900291 conn(0x7fb554089e50 msgr2=0x7fb554099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.169+0000 7fb559b04700 1 -- 172.21.15.92:0/358900291 shutdown_connections 2023-10-06T10:44:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.169+0000 7fb559b04700 1 -- 172.21.15.92:0/358900291 wait complete. 2023-10-06T10:44:37.183 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T10:44:37.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.299+0000 7f8b2c8a9700 1 Processor -- start 2023-10-06T10:44:37.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.299+0000 7f8b2c8a9700 1 -- start start 2023-10-06T10:44:37.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b2c8a9700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b280b2620 0x7f8b280cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:37.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b2c8a9700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8b280b29f0 0x7f8b280cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:37.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b2c8a9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b280cc970 con 0x7f8b280ceaa0 2023-10-06T10:44:37.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b2c8a9700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b280cbd60 con 0x7f8b280b29f0 2023-10-06T10:44:37.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b2c8a9700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b280cc040 con 0x7f8b280b2620 2023-10-06T10:44:37.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b2759e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8b280ceaa0 0x7f8b280cef40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:42952/0 (socket says 172.21.15.92:42952) 2023-10-06T10:44:37.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b2759e700 1 -- 172.21.15.92:0/1137756200 learned_addr learned my addr 172.21.15.92:0/1137756200 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:37.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b26d9d700 1 --2- 172.21.15.92:0/1137756200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b280b2620 0x7f8b280cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:37.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b2659c700 1 --2- 172.21.15.92:0/1137756200 >> v2:172.21.15.124:3300/0 conn(0x7f8b280b29f0 0x7f8b280cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:37.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b25d9b700 1 -- 172.21.15.92:0/1137756200 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401620962 0 0) 0x7f8b280cc970 con 0x7f8b280ceaa0 2023-10-06T10:44:37.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b25d9b700 1 -- 172.21.15.92:0/1137756200 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b10003570 con 0x7f8b280ceaa0 2023-10-06T10:44:37.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b25d9b700 1 -- 172.21.15.92:0/1137756200 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2046465082 0 0) 0x7f8b10003570 con 0x7f8b280ceaa0 2023-10-06T10:44:37.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b25d9b700 1 -- 172.21.15.92:0/1137756200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b280b2620 msgr2=0x7f8b280cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:37.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b25d9b700 1 --2- 172.21.15.92:0/1137756200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b280b2620 0x7f8b280cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b25d9b700 1 -- 172.21.15.92:0/1137756200 >> v2:172.21.15.124:3300/0 conn(0x7f8b280b29f0 msgr2=0x7f8b280cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:37.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b25d9b700 1 --2- 172.21.15.92:0/1137756200 >> v2:172.21.15.124:3300/0 conn(0x7f8b280b29f0 0x7f8b280cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b25d9b700 1 -- 172.21.15.92:0/1137756200 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b280cc320 con 0x7f8b280ceaa0 2023-10-06T10:44:37.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b25d9b700 1 -- 172.21.15.92:0/1137756200 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f8b200017f0 con 0x7f8b280ceaa0 2023-10-06T10:44:37.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b25d9b700 1 -- 172.21.15.92:0/1137756200 >> v1:172.21.15.92:6789/0 conn(0x7f8b280ceaa0 legacy=0x7f8b280cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:37.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b25d9b700 1 --2- 172.21.15.92:0/1137756200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b280b2620 0x7f8b10003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:37.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b25d9b700 1 --2- 172.21.15.92:0/1137756200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b10003fe0 0x7f8b10004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:37.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b25d9b700 1 --2- 172.21.15.92:0/1137756200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b10004d50 0x7f8b100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:37.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b25d9b700 1 -- 172.21.15.92:0/1137756200 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b280cbd60 con 0x7f8b10003fe0 2023-10-06T10:44:37.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b25d9b700 1 -- 172.21.15.92:0/1137756200 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b280cc040 con 0x7f8b10004d50 2023-10-06T10:44:37.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.300+0000 7f8b25d9b700 1 -- 172.21.15.92:0/1137756200 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1000ba90 con 0x7f8b280b2620 2023-10-06T10:44:37.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.301+0000 7f8b2659c700 1 --2- 172.21.15.92:0/1137756200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b10003fe0 0x7f8b10004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:37.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.301+0000 7f8b26d9d700 1 --2- 172.21.15.92:0/1137756200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b280b2620 0x7f8b10003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:37.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.301+0000 7f8b2759e700 1 --2- 172.21.15.92:0/1137756200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b10004d50 0x7f8b100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:37.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.301+0000 7f8b2659c700 1 -- 172.21.15.92:0/1137756200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b280b2620 msgr2=0x7f8b10003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:37.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.301+0000 7f8b2659c700 1 --2- 172.21.15.92:0/1137756200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b280b2620 0x7f8b10003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.301+0000 7f8b2659c700 1 -- 172.21.15.92:0/1137756200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b10004d50 msgr2=0x7f8b100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:37.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.301+0000 7f8b2659c700 1 --2- 172.21.15.92:0/1137756200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b10004d50 0x7f8b100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.301+0000 7f8b2659c700 1 -- 172.21.15.92:0/1137756200 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b10003570 con 0x7f8b10003fe0 2023-10-06T10:44:37.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.301+0000 7f8b2659c700 1 --2- 172.21.15.92:0/1137756200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b10003fe0 0x7f8b10004810 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c009c00 tx=0x7f8b1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a438e7d69aa50e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:37.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.301+0000 7f8b25d9b700 1 -- 172.21.15.92:0/1137756200 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c00e6b0 con 0x7f8b10003fe0 2023-10-06T10:44:37.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.301+0000 7f8b25d9b700 1 -- 172.21.15.92:0/1137756200 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1c00e810 con 0x7f8b10003fe0 2023-10-06T10:44:37.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.301+0000 7f8b25d9b700 1 -- 172.21.15.92:0/1137756200 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c00ebd0 con 0x7f8b10003fe0 2023-10-06T10:44:37.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.302+0000 7f8b2c8a9700 1 -- 172.21.15.92:0/1137756200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b10003fe0 msgr2=0x7f8b10004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:37.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.302+0000 7f8b2c8a9700 1 --2- 172.21.15.92:0/1137756200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b10003fe0 0x7f8b10004810 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c009c00 tx=0x7f8b1c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.302+0000 7f8b2759e700 1 -- 172.21.15.92:0/1137756200 reap_dead start 2023-10-06T10:44:37.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.302+0000 7f8b2c8a9700 1 -- 172.21.15.92:0/1137756200 shutdown_connections 2023-10-06T10:44:37.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.302+0000 7f8b2c8a9700 1 -- 172.21.15.92:0/1137756200 >> 172.21.15.92:0/1137756200 conn(0x7f8b28099520 msgr2=0x7f8b2806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:37.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.302+0000 7f8b2c8a9700 1 -- 172.21.15.92:0/1137756200 shutdown_connections 2023-10-06T10:44:37.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.302+0000 7f8b2c8a9700 1 -- 172.21.15.92:0/1137756200 wait complete. 2023-10-06T10:44:37.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.303+0000 7f8b2c8a9700 1 Processor -- start 2023-10-06T10:44:37.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.303+0000 7f8b2c8a9700 1 -- start start 2023-10-06T10:44:37.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b2c8a9700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b10003fe0 0x7f8b280c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:37.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b2c8a9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b2815d3a0 0x7f8b28169260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:37.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b2c8a9700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b2815f2a0 0x7f8b2815f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:37.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b2c8a9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b280c7c00 con 0x7f8b2815d3a0 2023-10-06T10:44:37.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b2c8a9700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2815fc80 con 0x7f8b10003fe0 2023-10-06T10:44:37.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b2c8a9700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2816a290 con 0x7f8b2815f2a0 2023-10-06T10:44:37.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b2659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b2815d3a0 0x7f8b28169260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:37.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b2659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b2815d3a0 0x7f8b28169260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46392/0 (socket says 172.21.15.92:46392) 2023-10-06T10:44:37.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b2659c700 1 -- 172.21.15.92:0/1174168327 learned_addr learned my addr 172.21.15.92:0/1174168327 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:37.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b26d9d700 1 --2- 172.21.15.92:0/1174168327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b10003fe0 0x7f8b280c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:37.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b2759e700 1 --2- 172.21.15.92:0/1174168327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b2815f2a0 0x7f8b2815f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:37.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b2659c700 1 -- 172.21.15.92:0/1174168327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b2815f2a0 msgr2=0x7f8b2815f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:37.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b2659c700 1 --2- 172.21.15.92:0/1174168327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b2815f2a0 0x7f8b2815f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b2659c700 1 -- 172.21.15.92:0/1174168327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b10003fe0 msgr2=0x7f8b280c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:37.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b2659c700 1 --2- 172.21.15.92:0/1174168327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b10003fe0 0x7f8b280c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b2659c700 1 -- 172.21.15.92:0/1174168327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b280b4590 con 0x7f8b2815d3a0 2023-10-06T10:44:37.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b2759e700 1 --2- 172.21.15.92:0/1174168327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b2815f2a0 0x7f8b2815f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:37.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b2659c700 1 --2- 172.21.15.92:0/1174168327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b2815d3a0 0x7f8b28169260 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00e4b0 tx=0x7f8b1c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:37.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b0effd700 1 -- 172.21.15.92:0/1174168327 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c023070 con 0x7f8b2815d3a0 2023-10-06T10:44:37.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b0effd700 1 -- 172.21.15.92:0/1174168327 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1c00ebb0 con 0x7f8b2815d3a0 2023-10-06T10:44:37.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b2c8a9700 1 -- 172.21.15.92:0/1174168327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b281697a0 con 0x7f8b2815d3a0 2023-10-06T10:44:37.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b0effd700 1 -- 172.21.15.92:0/1174168327 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c00e6b0 con 0x7f8b2815d3a0 2023-10-06T10:44:37.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.304+0000 7f8b2c8a9700 1 -- 172.21.15.92:0/1174168327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b28169cd0 con 0x7f8b2815d3a0 2023-10-06T10:44:37.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.305+0000 7f8b2c8a9700 1 -- 172.21.15.92:0/1174168327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b2805f230 con 0x7f8b2815d3a0 2023-10-06T10:44:37.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.306+0000 7f8b0effd700 1 -- 172.21.15.92:0/1174168327 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8b1c013680 con 0x7f8b2815d3a0 2023-10-06T10:44:37.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.306+0000 7f8b0effd700 1 --2- 172.21.15.92:0/1174168327 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8b14066f30 0x7f8b140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:37.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.306+0000 7f8b0effd700 1 -- 172.21.15.92:0/1174168327 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8b1c085ff0 con 0x7f8b2815d3a0 2023-10-06T10:44:37.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.306+0000 7f8b26d9d700 1 --2- 172.21.15.92:0/1174168327 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8b14066f30 0x7f8b140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:37.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.306+0000 7f8b26d9d700 1 --2- 172.21.15.92:0/1174168327 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8b14066f30 0x7f8b140693e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8b18006ea0 tx=0x7f8b18009120 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:37.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.310+0000 7f8b0effd700 1 -- 172.21.15.92:0/1174168327 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b1c052fc0 con 0x7f8b2815d3a0 2023-10-06T10:44:37.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.492+0000 7f8b2c8a9700 1 -- 172.21.15.92:0/1174168327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8b280cc600 con 0x7f8b2815d3a0 2023-10-06T10:44:37.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.493+0000 7f8b0effd700 1 -- 172.21.15.92:0/1174168327 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8b1c0565e0 con 0x7f8b2815d3a0 2023-10-06T10:44:37.495 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T10:44:37.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.495+0000 7f8b2c8a9700 1 -- 172.21.15.92:0/1174168327 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8b14066f30 msgr2=0x7f8b140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:37.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.495+0000 7f8b2c8a9700 1 --2- 172.21.15.92:0/1174168327 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8b14066f30 0x7f8b140693e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8b18006ea0 tx=0x7f8b18009120 comp rx=0 tx=0).stop 2023-10-06T10:44:37.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.495+0000 7f8b2c8a9700 1 -- 172.21.15.92:0/1174168327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b2815d3a0 msgr2=0x7f8b28169260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:37.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.495+0000 7f8b2c8a9700 1 --2- 172.21.15.92:0/1174168327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b2815d3a0 0x7f8b28169260 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00e4b0 tx=0x7f8b1c01a000 comp rx=0 tx=0).stop 2023-10-06T10:44:37.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.495+0000 7f8b2c8a9700 1 -- 172.21.15.92:0/1174168327 shutdown_connections 2023-10-06T10:44:37.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.495+0000 7f8b2c8a9700 1 --2- 172.21.15.92:0/1174168327 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8b14066f30 0x7f8b140693e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.495+0000 7f8b2c8a9700 1 --2- 172.21.15.92:0/1174168327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b2815f2a0 0x7f8b2815f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.495+0000 7f8b2c8a9700 1 --2- 172.21.15.92:0/1174168327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b10003fe0 0x7f8b280c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.495+0000 7f8b2c8a9700 1 --2- 172.21.15.92:0/1174168327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b2815d3a0 0x7f8b28169260 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.495+0000 7f8b2c8a9700 1 -- 172.21.15.92:0/1174168327 >> 172.21.15.92:0/1174168327 conn(0x7f8b28099520 msgr2=0x7f8b280c6650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:37.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.495+0000 7f8b2c8a9700 1 -- 172.21.15.92:0/1174168327 shutdown_connections 2023-10-06T10:44:37.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.495+0000 7f8b2c8a9700 1 -- 172.21.15.92:0/1174168327 wait complete. 2023-10-06T10:44:37.508 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:44:37.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.621+0000 7efe18448700 1 Processor -- start 2023-10-06T10:44:37.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.621+0000 7efe18448700 1 -- start start 2023-10-06T10:44:37.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.622+0000 7efe18448700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efe10159560 0x7efe100aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:37.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.622+0000 7efe18448700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe100b0c60 con 0x7efe100b0760 2023-10-06T10:44:37.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.622+0000 7efe18448700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe100b0f40 con 0x7efe100b49a0 2023-10-06T10:44:37.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.622+0000 7efe18448700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe100b1220 con 0x7efe10159560 2023-10-06T10:44:37.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.622+0000 7efe169e5700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7efe100b0760 0x7efe100b1bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:42958/0 (socket says 172.21.15.92:42958) 2023-10-06T10:44:37.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.622+0000 7efe169e5700 1 -- 172.21.15.92:0/1581081620 learned_addr learned my addr 172.21.15.92:0/1581081620 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:37.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.622+0000 7efe161e4700 1 --2- 172.21.15.92:0/1581081620 >> v2:172.21.15.124:3300/0 conn(0x7efe10159560 0x7efe100aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:37.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.622+0000 7efe151e2700 1 -- 172.21.15.92:0/1581081620 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3237759843 0 0) 0x7efe100b0c60 con 0x7efe100b0760 2023-10-06T10:44:37.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.622+0000 7efe151e2700 1 -- 172.21.15.92:0/1581081620 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdf8003570 con 0x7efe100b0760 2023-10-06T10:44:37.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.622+0000 7efe151e2700 1 -- 172.21.15.92:0/1581081620 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407016966 0 0) 0x7efe100b0f40 con 0x7efe100b49a0 2023-10-06T10:44:37.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.622+0000 7efe151e2700 1 -- 172.21.15.92:0/1581081620 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe100b0c60 con 0x7efe100b49a0 2023-10-06T10:44:37.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.622+0000 7efe151e2700 1 -- 172.21.15.92:0/1581081620 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1034720722 0 0) 0x7efdf8003570 con 0x7efe100b0760 2023-10-06T10:44:37.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.622+0000 7efe161e4700 1 -- 172.21.15.92:0/1581081620 >> v1:172.21.15.124:6789/0 conn(0x7efe100b49a0 legacy=0x7efe100aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:37.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.622+0000 7efe161e4700 1 -- 172.21.15.92:0/1581081620 >> v1:172.21.15.92:6789/0 conn(0x7efe100b0760 legacy=0x7efe100b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:37.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.623+0000 7efe161e4700 1 -- 172.21.15.92:0/1581081620 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe100b1500 con 0x7efe10159560 2023-10-06T10:44:37.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.623+0000 7efe161e4700 1 --2- 172.21.15.92:0/1581081620 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe10159560 0x7efe100aa530 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efe08009900 tx=0x7efe08009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f20e61df545575 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:37.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.623+0000 7efe151e2700 1 -- 172.21.15.92:0/1581081620 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe000013f0 con 0x7efe10159560 2023-10-06T10:44:37.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.623+0000 7efe151e2700 1 -- 172.21.15.92:0/1581081620 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe08011420 con 0x7efe10159560 2023-10-06T10:44:37.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.623+0000 7efe151e2700 1 -- 172.21.15.92:0/1581081620 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe080117c0 con 0x7efe10159560 2023-10-06T10:44:37.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.623+0000 7efe18448700 1 -- 172.21.15.92:0/1581081620 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe10159560 msgr2=0x7efe100aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:37.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.623+0000 7efe18448700 1 --2- 172.21.15.92:0/1581081620 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe10159560 0x7efe100aa530 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efe08009900 tx=0x7efe08009120 comp rx=0 tx=0).stop 2023-10-06T10:44:37.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.624+0000 7efe18448700 1 -- 172.21.15.92:0/1581081620 shutdown_connections 2023-10-06T10:44:37.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.624+0000 7efe18448700 1 --2- 172.21.15.92:0/1581081620 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe10159560 0x7efe100aa530 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.624+0000 7efe18448700 1 -- 172.21.15.92:0/1581081620 >> 172.21.15.92:0/1581081620 conn(0x7efe100997f0 msgr2=0x7efe1006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:37.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.624+0000 7efe18448700 1 -- 172.21.15.92:0/1581081620 shutdown_connections 2023-10-06T10:44:37.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.624+0000 7efe18448700 1 -- 172.21.15.92:0/1581081620 wait complete. 2023-10-06T10:44:37.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.625+0000 7efe18448700 1 Processor -- start 2023-10-06T10:44:37.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.625+0000 7efe18448700 1 -- start start 2023-10-06T10:44:37.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.625+0000 7efe18448700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe100b0760 0x7efe100ae530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:37.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.625+0000 7efe18448700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe100b49a0 0x7efe10157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:37.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.625+0000 7efe18448700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe10159560 0x7efe100ad650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:37.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.625+0000 7efe18448700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1009bc50 con 0x7efe100b49a0 2023-10-06T10:44:37.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.625+0000 7efe18448700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1009b560 con 0x7efe10159560 2023-10-06T10:44:37.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.625+0000 7efe18448700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe1009b840 con 0x7efe100b0760 2023-10-06T10:44:37.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.626+0000 7efe159e3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe100b49a0 0x7efe10157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:37.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.626+0000 7efe159e3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe100b49a0 0x7efe10157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46408/0 (socket says 172.21.15.92:46408) 2023-10-06T10:44:37.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.626+0000 7efe159e3700 1 -- 172.21.15.92:0/2879113866 learned_addr learned my addr 172.21.15.92:0/2879113866 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:37.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.626+0000 7efe169e5700 1 --2- 172.21.15.92:0/2879113866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe10159560 0x7efe100ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:37.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.626+0000 7efe161e4700 1 --2- 172.21.15.92:0/2879113866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe100b0760 0x7efe100ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:37.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.626+0000 7efe159e3700 1 -- 172.21.15.92:0/2879113866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe100b0760 msgr2=0x7efe100ae530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:37.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.626+0000 7efe159e3700 1 --2- 172.21.15.92:0/2879113866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe100b0760 0x7efe100ae530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.626+0000 7efe159e3700 1 -- 172.21.15.92:0/2879113866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe10159560 msgr2=0x7efe100ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:37.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.626+0000 7efe159e3700 1 --2- 172.21.15.92:0/2879113866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe10159560 0x7efe100ad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.626+0000 7efe159e3700 1 -- 172.21.15.92:0/2879113866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe080093c0 con 0x7efe100b49a0 2023-10-06T10:44:37.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.626+0000 7efe159e3700 1 --2- 172.21.15.92:0/2879113866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe100b49a0 0x7efe10157c40 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7efe0400cb10 tx=0x7efe04002820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:37.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.626+0000 7efe0e7fc700 1 -- 172.21.15.92:0/2879113866 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0400a7f0 con 0x7efe100b49a0 2023-10-06T10:44:37.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.626+0000 7efe0e7fc700 1 -- 172.21.15.92:0/2879113866 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0400a950 con 0x7efe100b49a0 2023-10-06T10:44:37.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.626+0000 7efe18448700 1 -- 172.21.15.92:0/2879113866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe1009bea0 con 0x7efe100b49a0 2023-10-06T10:44:37.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.626+0000 7efe0e7fc700 1 -- 172.21.15.92:0/2879113866 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0400ad10 con 0x7efe100b49a0 2023-10-06T10:44:37.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.626+0000 7efe18448700 1 -- 172.21.15.92:0/2879113866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe100aea70 con 0x7efe100b49a0 2023-10-06T10:44:37.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.627+0000 7efe18448700 1 -- 172.21.15.92:0/2879113866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe10064a90 con 0x7efe100b49a0 2023-10-06T10:44:37.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.627+0000 7efe0e7fc700 1 -- 172.21.15.92:0/2879113866 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efe04012630 con 0x7efe100b49a0 2023-10-06T10:44:37.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.628+0000 7efe0e7fc700 1 --2- 172.21.15.92:0/2879113866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efdfc066f30 0x7efdfc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:37.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.628+0000 7efe161e4700 1 --2- 172.21.15.92:0/2879113866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efdfc066f30 0x7efdfc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:37.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.628+0000 7efe0e7fc700 1 -- 172.21.15.92:0/2879113866 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7efe04084ce0 con 0x7efe100b49a0 2023-10-06T10:44:37.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.628+0000 7efe161e4700 1 --2- 172.21.15.92:0/2879113866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efdfc066f30 0x7efdfc0693e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7efe080098d0 tx=0x7efe08016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:37.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.632+0000 7efe0e7fc700 1 -- 172.21.15.92:0/2879113866 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe04051cb0 con 0x7efe100b49a0 2023-10-06T10:44:37.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.859+0000 7efe18448700 1 -- 172.21.15.92:0/2879113866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe1009ba00 con 0x7efe100b49a0 2023-10-06T10:44:37.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.860+0000 7efe0e7fc700 1 -- 172.21.15.92:0/2879113866 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4642 (secure 0 0 0) 0x7efe040552d0 con 0x7efe100b49a0 2023-10-06T10:44:37.862 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:37.862 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2122441982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2122441982},{"type":"v1","addr":"172.21.15.124:6835","nonce":2122441982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:47.850832+0000","modified":"2023-10-06T10:44:20.587653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7701,"mds_1":7689,"mds_2":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6836/3605297930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3605297930},{"type":"v1","addr":"172.21.15.92:6836","nonce":3605297930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7689":{"gid":7689,"name":"c","rank":1,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.92:6837/4047304116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4047304116},{"type":"v1","addr":"172.21.15.92:6837","nonce":4047304116}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7701":{"gid":7701,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":11,"addr":"172.21.15.124:6834/4203118992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":4203118992},{"type":"v1","addr":"172.21.15.124:6834","nonce":4203118992}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T10:44:37.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.862+0000 7efe18448700 1 -- 172.21.15.92:0/2879113866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efdfc066f30 msgr2=0x7efdfc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:37.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.862+0000 7efe18448700 1 --2- 172.21.15.92:0/2879113866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efdfc066f30 0x7efdfc0693e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7efe080098d0 tx=0x7efe08016000 comp rx=0 tx=0).stop 2023-10-06T10:44:37.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.862+0000 7efe18448700 1 -- 172.21.15.92:0/2879113866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe100b49a0 msgr2=0x7efe10157c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:37.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.862+0000 7efe18448700 1 --2- 172.21.15.92:0/2879113866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe100b49a0 0x7efe10157c40 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7efe0400cb10 tx=0x7efe04002820 comp rx=0 tx=0).stop 2023-10-06T10:44:37.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.862+0000 7efe18448700 1 -- 172.21.15.92:0/2879113866 shutdown_connections 2023-10-06T10:44:37.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.862+0000 7efe18448700 1 --2- 172.21.15.92:0/2879113866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efdfc066f30 0x7efdfc0693e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.862+0000 7efe18448700 1 --2- 172.21.15.92:0/2879113866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe10159560 0x7efe100ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.862+0000 7efe18448700 1 --2- 172.21.15.92:0/2879113866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe100b0760 0x7efe100ae530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.862+0000 7efe18448700 1 --2- 172.21.15.92:0/2879113866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe100b49a0 0x7efe10157c40 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.862+0000 7efe18448700 1 -- 172.21.15.92:0/2879113866 >> 172.21.15.92:0/2879113866 conn(0x7efe100997f0 msgr2=0x7efe100683c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:37.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.862+0000 7efe18448700 1 -- 172.21.15.92:0/2879113866 shutdown_connections 2023-10-06T10:44:37.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.862+0000 7efe18448700 1 -- 172.21.15.92:0/2879113866 wait complete. 2023-10-06T10:44:37.865 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 92 2023-10-06T10:44:37.876 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:44:37.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.988+0000 7f3184535700 1 Processor -- start 2023-10-06T10:44:37.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.988+0000 7f3184535700 1 -- start start 2023-10-06T10:44:37.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.989+0000 7f3184535700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f317c159560 0x7f317c0aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:37.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.989+0000 7f3184535700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f317c0b49a0 0x7f317c0aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:37.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.989+0000 7f3184535700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f317c0b0760 0x7f317c0b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:37.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.989+0000 7f3184535700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f317c0b0c60 con 0x7f317c159560 2023-10-06T10:44:37.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.989+0000 7f3184535700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f317c0b0f40 con 0x7f317c0b49a0 2023-10-06T10:44:37.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.989+0000 7f3184535700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317c0b1220 con 0x7f317c0b0760 2023-10-06T10:44:37.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.989+0000 7f31822d1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f317c159560 0x7f317c0aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:37.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.989+0000 7f31822d1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f317c159560 0x7f317c0aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46414/0 (socket says 172.21.15.92:46414) 2023-10-06T10:44:37.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.989+0000 7f31822d1700 1 -- 172.21.15.92:0/3192897432 learned_addr learned my addr 172.21.15.92:0/3192897432 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:37.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.989+0000 7f3182ad2700 1 --2- 172.21.15.92:0/3192897432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f317c0b0760 0x7f317c0b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:37.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.989+0000 7f3181ad0700 1 --2- 172.21.15.92:0/3192897432 >> v2:172.21.15.124:3300/0 conn(0x7f317c0b49a0 0x7f317c0aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:37.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.989+0000 7f31822d1700 1 -- 172.21.15.92:0/3192897432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f317c0b0760 msgr2=0x7f317c0b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:37.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.989+0000 7f31822d1700 1 --2- 172.21.15.92:0/3192897432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f317c0b0760 0x7f317c0b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.989+0000 7f31822d1700 1 -- 172.21.15.92:0/3192897432 >> v2:172.21.15.124:3300/0 conn(0x7f317c0b49a0 msgr2=0x7f317c0aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:37.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.989+0000 7f31822d1700 1 --2- 172.21.15.92:0/3192897432 >> v2:172.21.15.124:3300/0 conn(0x7f317c0b49a0 0x7f317c0aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.989+0000 7f31822d1700 1 -- 172.21.15.92:0/3192897432 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317c0b1500 con 0x7f317c159560 2023-10-06T10:44:37.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.989+0000 7f3181ad0700 1 --2- 172.21.15.92:0/3192897432 >> v2:172.21.15.124:3300/0 conn(0x7f317c0b49a0 0x7f317c0aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:37.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.989+0000 7f31822d1700 1 --2- 172.21.15.92:0/3192897432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f317c159560 0x7f317c0aa530 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f3174009f70 tx=0x7f3174009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e742b167fa133f6a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:37.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.989+0000 7f31812cf700 1 -- 172.21.15.92:0/3192897432 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317400e070 con 0x7f317c159560 2023-10-06T10:44:37.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.989+0000 7f31812cf700 1 -- 172.21.15.92:0/3192897432 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317400ba90 con 0x7f317c159560 2023-10-06T10:44:37.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.989+0000 7f31812cf700 1 -- 172.21.15.92:0/3192897432 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31740092e0 con 0x7f317c159560 2023-10-06T10:44:37.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.990+0000 7f3184535700 1 -- 172.21.15.92:0/3192897432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f317c159560 msgr2=0x7f317c0aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:37.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.990+0000 7f3184535700 1 --2- 172.21.15.92:0/3192897432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f317c159560 0x7f317c0aa530 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f3174009f70 tx=0x7f3174009840 comp rx=0 tx=0).stop 2023-10-06T10:44:37.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.990+0000 7f3184535700 1 -- 172.21.15.92:0/3192897432 shutdown_connections 2023-10-06T10:44:37.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.990+0000 7f3184535700 1 --2- 172.21.15.92:0/3192897432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f317c159560 0x7f317c0aa530 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.990+0000 7f3184535700 1 --2- 172.21.15.92:0/3192897432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f317c0b0760 0x7f317c0b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.990+0000 7f3184535700 1 --2- 172.21.15.92:0/3192897432 >> v2:172.21.15.124:3300/0 conn(0x7f317c0b49a0 0x7f317c0aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:37.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.990+0000 7f3184535700 1 -- 172.21.15.92:0/3192897432 >> 172.21.15.92:0/3192897432 conn(0x7f317c0997f0 msgr2=0x7f317c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:37.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.990+0000 7f3184535700 1 -- 172.21.15.92:0/3192897432 shutdown_connections 2023-10-06T10:44:37.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.990+0000 7f3184535700 1 -- 172.21.15.92:0/3192897432 wait complete. 2023-10-06T10:44:38.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.991+0000 7f3184535700 1 Processor -- start 2023-10-06T10:44:38.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.991+0000 7f3184535700 1 -- start start 2023-10-06T10:44:38.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.992+0000 7f3184535700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f317c0b0760 0x7f317c1575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:38.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.992+0000 7f3184535700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f317c0b49a0 0x7f317c09c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:38.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.992+0000 7f3184535700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f317c159560 0x7f317c09a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:38.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.992+0000 7f3184535700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317c09c970 con 0x7f317c0b0760 2023-10-06T10:44:38.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.992+0000 7f3184535700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317c16a0f0 con 0x7f317c159560 2023-10-06T10:44:38.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.992+0000 7f3184535700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317c16a3d0 con 0x7f317c0b49a0 2023-10-06T10:44:38.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.992+0000 7f31822d1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f317c0b0760 0x7f317c1575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:38.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.992+0000 7f31822d1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f317c0b0760 0x7f317c1575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46430/0 (socket says 172.21.15.92:46430) 2023-10-06T10:44:38.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.992+0000 7f31822d1700 1 -- 172.21.15.92:0/3334648985 learned_addr learned my addr 172.21.15.92:0/3334648985 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:38.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.992+0000 7f31822d1700 1 -- 172.21.15.92:0/3334648985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f317c0b49a0 msgr2=0x7f317c09c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:38.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.992+0000 7f31822d1700 1 --2- 172.21.15.92:0/3334648985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f317c0b49a0 0x7f317c09c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.992+0000 7f31822d1700 1 -- 172.21.15.92:0/3334648985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f317c159560 msgr2=0x7f317c09a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:38.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.992+0000 7f31822d1700 1 --2- 172.21.15.92:0/3334648985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f317c159560 0x7f317c09a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.992+0000 7f31822d1700 1 -- 172.21.15.92:0/3334648985 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3174009ae0 con 0x7f317c0b0760 2023-10-06T10:44:38.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.992+0000 7f31822d1700 1 --2- 172.21.15.92:0/3334648985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f317c0b0760 0x7f317c1575c0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f317c067c20 tx=0x7f3174018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:38.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.992+0000 7f31727fc700 1 -- 172.21.15.92:0/3334648985 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3174008aa0 con 0x7f317c0b0760 2023-10-06T10:44:38.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.993+0000 7f31727fc700 1 -- 172.21.15.92:0/3334648985 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3174008c00 con 0x7f317c0b0760 2023-10-06T10:44:38.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.993+0000 7f3184535700 1 -- 172.21.15.92:0/3334648985 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317c16a6b0 con 0x7f317c0b0760 2023-10-06T10:44:38.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.993+0000 7f31727fc700 1 -- 172.21.15.92:0/3334648985 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3174025460 con 0x7f317c0b0760 2023-10-06T10:44:38.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.993+0000 7f3184535700 1 -- 172.21.15.92:0/3334648985 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317c157b00 con 0x7f317c0b0760 2023-10-06T10:44:38.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.993+0000 7f3184535700 1 -- 172.21.15.92:0/3334648985 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f317c064a90 con 0x7f317c0b0760 2023-10-06T10:44:38.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.994+0000 7f31727fc700 1 -- 172.21.15.92:0/3334648985 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3174003040 con 0x7f317c0b0760 2023-10-06T10:44:38.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.994+0000 7f31727fc700 1 --2- 172.21.15.92:0/3334648985 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3160066f30 0x7f31600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:38.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.994+0000 7f31727fc700 1 -- 172.21.15.92:0/3334648985 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3174086200 con 0x7f317c0b0760 2023-10-06T10:44:38.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.994+0000 7f3181ad0700 1 --2- 172.21.15.92:0/3334648985 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3160066f30 0x7f31600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:38.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.995+0000 7f3181ad0700 1 --2- 172.21.15.92:0/3334648985 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3160066f30 0x7f31600693e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f316c006f00 tx=0x7f316c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:38.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:37.999+0000 7f31727fc700 1 -- 172.21.15.92:0/3334648985 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31740531d0 con 0x7f317c0b0760 2023-10-06T10:44:38.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.183+0000 7f3184535700 1 -- 172.21.15.92:0/3334648985 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f317c0b17e0 con 0x7f317c0b0760 2023-10-06T10:44:38.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.184+0000 7f31727fc700 1 -- 172.21.15.92:0/3334648985 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+17074 (secure 0 0 0) 0x7f31740567f0 con 0x7f317c0b0760 2023-10-06T10:44:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:38.186 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":90,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:43:49.334267+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:43:45.492396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-06T10:43:46.811317+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:44:38.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.186+0000 7f3184535700 1 -- 172.21.15.92:0/3334648985 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3160066f30 msgr2=0x7f31600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:38.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.186+0000 7f3184535700 1 --2- 172.21.15.92:0/3334648985 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3160066f30 0x7f31600693e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f316c006f00 tx=0x7f316c008040 comp rx=0 tx=0).stop 2023-10-06T10:44:38.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.186+0000 7f3184535700 1 -- 172.21.15.92:0/3334648985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f317c0b0760 msgr2=0x7f317c1575c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:38.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.186+0000 7f3184535700 1 --2- 172.21.15.92:0/3334648985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f317c0b0760 0x7f317c1575c0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f317c067c20 tx=0x7f3174018000 comp rx=0 tx=0).stop 2023-10-06T10:44:38.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.186+0000 7f3184535700 1 -- 172.21.15.92:0/3334648985 shutdown_connections 2023-10-06T10:44:38.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.186+0000 7f3184535700 1 --2- 172.21.15.92:0/3334648985 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3160066f30 0x7f31600693e0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.186+0000 7f3184535700 1 --2- 172.21.15.92:0/3334648985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f317c159560 0x7f317c09a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.186+0000 7f3184535700 1 --2- 172.21.15.92:0/3334648985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f317c0b0760 0x7f317c1575c0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.186+0000 7f3184535700 1 --2- 172.21.15.92:0/3334648985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f317c0b49a0 0x7f317c09c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.186+0000 7f3184535700 1 -- 172.21.15.92:0/3334648985 >> 172.21.15.92:0/3334648985 conn(0x7f317c0997f0 msgr2=0x7f317c158960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:38.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.186+0000 7f3184535700 1 -- 172.21.15.92:0/3334648985 shutdown_connections 2023-10-06T10:44:38.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.186+0000 7f3184535700 1 -- 172.21.15.92:0/3334648985 wait complete. 2023-10-06T10:44:38.200 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T10:44:38.200 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:44:38.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.310+0000 7fe23fe23700 1 Processor -- start 2023-10-06T10:44:38.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.311+0000 7fe23fe23700 1 -- start start 2023-10-06T10:44:38.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.311+0000 7fe23fe23700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe238159560 0x7fe2380aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:38.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.311+0000 7fe23fe23700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe2380b49a0 0x7fe2380aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:38.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.311+0000 7fe23fe23700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2380b0c60 con 0x7fe2380b0760 2023-10-06T10:44:38.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.311+0000 7fe23fe23700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2380b0f40 con 0x7fe2380b49a0 2023-10-06T10:44:38.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.311+0000 7fe23fe23700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380b1220 con 0x7fe238159560 2023-10-06T10:44:38.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.312+0000 7fe23d3be700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe2380b49a0 0x7fe2380aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:38.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.312+0000 7fe23e3c0700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe2380b0760 0x7fe2380b3890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:42966/0 (socket says 172.21.15.92:42966) 2023-10-06T10:44:38.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.312+0000 7fe23e3c0700 1 -- 172.21.15.92:0/3059356211 learned_addr learned my addr 172.21.15.92:0/3059356211 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:38.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.312+0000 7fe23dbbf700 1 --2- 172.21.15.92:0/3059356211 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe238159560 0x7fe2380aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:38.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.312+0000 7fe23d3be700 1 -- 172.21.15.92:0/3059356211 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe238159560 msgr2=0x7fe2380aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:38.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.312+0000 7fe23d3be700 1 --2- 172.21.15.92:0/3059356211 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe238159560 0x7fe2380aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.312+0000 7fe23d3be700 1 -- 172.21.15.92:0/3059356211 >> v1:172.21.15.92:6789/0 conn(0x7fe2380b0760 legacy=0x7fe2380b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:38.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.312+0000 7fe23cbbd700 1 -- 172.21.15.92:0/3059356211 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 269518503 0 0) 0x7fe2380b0c60 con 0x7fe2380b0760 2023-10-06T10:44:38.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.312+0000 7fe23d3be700 1 -- 172.21.15.92:0/3059356211 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2380b1500 con 0x7fe2380b49a0 2023-10-06T10:44:38.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.312+0000 7fe23d3be700 1 --2- 172.21.15.92:0/3059356211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2380b49a0 0x7fe2380aaa70 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fe2280098e0 tx=0x7fe2280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545fa1d8d37830df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:38.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.312+0000 7fe23cbbd700 1 -- 172.21.15.92:0/3059356211 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe228009e80 con 0x7fe2380b49a0 2023-10-06T10:44:38.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.312+0000 7fe23cbbd700 1 -- 172.21.15.92:0/3059356211 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22800b860 con 0x7fe2380b49a0 2023-10-06T10:44:38.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.312+0000 7fe23cbbd700 1 -- 172.21.15.92:0/3059356211 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22800bbe0 con 0x7fe2380b49a0 2023-10-06T10:44:38.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.313+0000 7fe23fe23700 1 -- 172.21.15.92:0/3059356211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2380b49a0 msgr2=0x7fe2380aaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:38.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.313+0000 7fe23fe23700 1 --2- 172.21.15.92:0/3059356211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2380b49a0 0x7fe2380aaa70 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fe2280098e0 tx=0x7fe2280091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.313+0000 7fe23fe23700 1 -- 172.21.15.92:0/3059356211 shutdown_connections 2023-10-06T10:44:38.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.313+0000 7fe23fe23700 1 --2- 172.21.15.92:0/3059356211 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe238159560 0x7fe2380aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.313+0000 7fe23fe23700 1 --2- 172.21.15.92:0/3059356211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2380b49a0 0x7fe2380aaa70 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.313+0000 7fe23fe23700 1 -- 172.21.15.92:0/3059356211 >> 172.21.15.92:0/3059356211 conn(0x7fe2380997f0 msgr2=0x7fe23806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:38.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.313+0000 7fe23fe23700 1 -- 172.21.15.92:0/3059356211 shutdown_connections 2023-10-06T10:44:38.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.313+0000 7fe23fe23700 1 -- 172.21.15.92:0/3059356211 wait complete. 2023-10-06T10:44:38.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.314+0000 7fe23fe23700 1 Processor -- start 2023-10-06T10:44:38.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.314+0000 7fe23fe23700 1 -- start start 2023-10-06T10:44:38.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.315+0000 7fe23fe23700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2380b0760 0x7fe238099f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:38.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.315+0000 7fe23fe23700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2380b49a0 0x7fe23816a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:38.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.315+0000 7fe23fe23700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe238159560 0x7fe23815ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.315+0000 7fe23fe23700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23809da80 con 0x7fe2380b0760 2023-10-06T10:44:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.315+0000 7fe23fe23700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23809cef0 con 0x7fe238159560 2023-10-06T10:44:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.315+0000 7fe23fe23700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23809d150 con 0x7fe2380b49a0 2023-10-06T10:44:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.315+0000 7fe23dbbf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2380b0760 0x7fe238099f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.315+0000 7fe23dbbf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2380b0760 0x7fe238099f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46446/0 (socket says 172.21.15.92:46446) 2023-10-06T10:44:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.315+0000 7fe23dbbf700 1 -- 172.21.15.92:0/2026272268 learned_addr learned my addr 172.21.15.92:0/2026272268 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.315+0000 7fe23d3be700 1 --2- 172.21.15.92:0/2026272268 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2380b49a0 0x7fe23816a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.315+0000 7fe23dbbf700 1 -- 172.21.15.92:0/2026272268 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2380b49a0 msgr2=0x7fe23816a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.315+0000 7fe23e3c0700 1 --2- 172.21.15.92:0/2026272268 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe238159560 0x7fe23815ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.315+0000 7fe23dbbf700 1 --2- 172.21.15.92:0/2026272268 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2380b49a0 0x7fe23816a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.315+0000 7fe23dbbf700 1 -- 172.21.15.92:0/2026272268 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe238159560 msgr2=0x7fe23815ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.315+0000 7fe23dbbf700 1 --2- 172.21.15.92:0/2026272268 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe238159560 0x7fe23815ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.315+0000 7fe23dbbf700 1 -- 172.21.15.92:0/2026272268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe228009420 con 0x7fe2380b0760 2023-10-06T10:44:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.315+0000 7fe23dbbf700 1 --2- 172.21.15.92:0/2026272268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2380b0760 0x7fe238099f50 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fe23400e0e0 tx=0x7fe2340098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.315+0000 7fe22dffb700 1 -- 172.21.15.92:0/2026272268 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23400ed80 con 0x7fe2380b0760 2023-10-06T10:44:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.315+0000 7fe22dffb700 1 -- 172.21.15.92:0/2026272268 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe234014420 con 0x7fe2380b0760 2023-10-06T10:44:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.315+0000 7fe22dffb700 1 -- 172.21.15.92:0/2026272268 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2340147e0 con 0x7fe2380b0760 2023-10-06T10:44:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.315+0000 7fe23fe23700 1 -- 172.21.15.92:0/2026272268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23809d430 con 0x7fe2380b0760 2023-10-06T10:44:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.316+0000 7fe23fe23700 1 -- 172.21.15.92:0/2026272268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe23809df10 con 0x7fe2380b0760 2023-10-06T10:44:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.317+0000 7fe23fe23700 1 -- 172.21.15.92:0/2026272268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe238064a90 con 0x7fe2380b0760 2023-10-06T10:44:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.317+0000 7fe22dffb700 1 -- 172.21.15.92:0/2026272268 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe234014b90 con 0x7fe2380b0760 2023-10-06T10:44:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.317+0000 7fe22dffb700 1 --2- 172.21.15.92:0/2026272268 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe21c066f30 0x7fe21c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.317+0000 7fe22dffb700 1 -- 172.21.15.92:0/2026272268 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe234084f50 con 0x7fe2380b0760 2023-10-06T10:44:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.317+0000 7fe23d3be700 1 --2- 172.21.15.92:0/2026272268 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe21c066f30 0x7fe21c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:38.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.318+0000 7fe23d3be700 1 --2- 172.21.15.92:0/2026272268 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe21c066f30 0x7fe21c0693e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe2380b4db0 tx=0x7fe228017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:38.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.321+0000 7fe22dffb700 1 -- 172.21.15.92:0/2026272268 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe234051f20 con 0x7fe2380b0760 2023-10-06T10:44:38.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.548+0000 7fe23fe23700 1 -- 172.21.15.92:0/2026272268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2380b17e0 con 0x7fe2380b0760 2023-10-06T10:44:38.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.548+0000 7fe22dffb700 1 -- 172.21.15.92:0/2026272268 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4642 (secure 0 0 0) 0x7fe234055540 con 0x7fe2380b0760 2023-10-06T10:44:38.550 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:38.550 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2122441982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2122441982},{"type":"v1","addr":"172.21.15.124:6835","nonce":2122441982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:47.850832+0000","modified":"2023-10-06T10:44:20.587653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7701,"mds_1":7689,"mds_2":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6836/3605297930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3605297930},{"type":"v1","addr":"172.21.15.92:6836","nonce":3605297930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7689":{"gid":7689,"name":"c","rank":1,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.92:6837/4047304116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4047304116},{"type":"v1","addr":"172.21.15.92:6837","nonce":4047304116}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7701":{"gid":7701,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":11,"addr":"172.21.15.124:6834/4203118992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":4203118992},{"type":"v1","addr":"172.21.15.124:6834","nonce":4203118992}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T10:44:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.550+0000 7fe23fe23700 1 -- 172.21.15.92:0/2026272268 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe21c066f30 msgr2=0x7fe21c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.550+0000 7fe23fe23700 1 --2- 172.21.15.92:0/2026272268 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe21c066f30 0x7fe21c0693e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe2380b4db0 tx=0x7fe228017000 comp rx=0 tx=0).stop 2023-10-06T10:44:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.550+0000 7fe23fe23700 1 -- 172.21.15.92:0/2026272268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2380b0760 msgr2=0x7fe238099f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:38.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.550+0000 7fe23fe23700 1 --2- 172.21.15.92:0/2026272268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2380b0760 0x7fe238099f50 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fe23400e0e0 tx=0x7fe2340098d0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.550+0000 7fe23fe23700 1 -- 172.21.15.92:0/2026272268 shutdown_connections 2023-10-06T10:44:38.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.550+0000 7fe23fe23700 1 --2- 172.21.15.92:0/2026272268 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe21c066f30 0x7fe21c0693e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.550+0000 7fe23fe23700 1 --2- 172.21.15.92:0/2026272268 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe238159560 0x7fe23815ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.550+0000 7fe23fe23700 1 --2- 172.21.15.92:0/2026272268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2380b0760 0x7fe238099f50 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.550+0000 7fe23fe23700 1 --2- 172.21.15.92:0/2026272268 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2380b49a0 0x7fe23816a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.550+0000 7fe23fe23700 1 -- 172.21.15.92:0/2026272268 >> 172.21.15.92:0/2026272268 conn(0x7fe2380997f0 msgr2=0x7fe238158960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:38.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.550+0000 7fe23fe23700 1 -- 172.21.15.92:0/2026272268 shutdown_connections 2023-10-06T10:44:38.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.550+0000 7fe23fe23700 1 -- 172.21.15.92:0/2026272268 wait complete. 2023-10-06T10:44:38.554 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 92 2023-10-06T10:44:38.564 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:44:38.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.677+0000 7fc6b0781700 1 Processor -- start 2023-10-06T10:44:38.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.677+0000 7fc6b0781700 1 -- start start 2023-10-06T10:44:38.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.678+0000 7fc6b0781700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc6a8159560 0x7fc6a80aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:38.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.678+0000 7fc6b0781700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc6a80b49a0 0x7fc6a80aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:38.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.678+0000 7fc6b0781700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6a80b0c60 con 0x7fc6a80b0760 2023-10-06T10:44:38.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.678+0000 7fc6b0781700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a80b0f40 con 0x7fc6a80b49a0 2023-10-06T10:44:38.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.678+0000 7fc6b0781700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a80b1220 con 0x7fc6a8159560 2023-10-06T10:44:38.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.678+0000 7fc6add1c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc6a80b49a0 0x7fc6a80aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:38.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.678+0000 7fc6aed1e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc6a80b0760 0x7fc6a80b3890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:42970/0 (socket says 172.21.15.92:42970) 2023-10-06T10:44:38.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.678+0000 7fc6aed1e700 1 -- 172.21.15.92:0/2722969871 learned_addr learned my addr 172.21.15.92:0/2722969871 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:38.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.678+0000 7fc6ae51d700 1 --2- 172.21.15.92:0/2722969871 >> v2:172.21.15.124:3300/0 conn(0x7fc6a8159560 0x7fc6a80aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:38.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.678+0000 7fc6add1c700 1 -- 172.21.15.92:0/2722969871 >> v2:172.21.15.124:3300/0 conn(0x7fc6a8159560 msgr2=0x7fc6a80aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:38.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.678+0000 7fc6add1c700 1 --2- 172.21.15.92:0/2722969871 >> v2:172.21.15.124:3300/0 conn(0x7fc6a8159560 0x7fc6a80aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.678+0000 7fc6add1c700 1 -- 172.21.15.92:0/2722969871 >> v1:172.21.15.92:6789/0 conn(0x7fc6a80b0760 legacy=0x7fc6a80b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:38.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.678+0000 7fc6add1c700 1 -- 172.21.15.92:0/2722969871 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a80b1500 con 0x7fc6a80b49a0 2023-10-06T10:44:38.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.678+0000 7fc6ad51b700 1 -- 172.21.15.92:0/2722969871 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1349945966 0 0) 0x7fc6a80b0c60 con 0x7fc6a80b0760 2023-10-06T10:44:38.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.678+0000 7fc6ae51d700 1 --2- 172.21.15.92:0/2722969871 >> v2:172.21.15.124:3300/0 conn(0x7fc6a8159560 0x7fc6a80aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:38.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.678+0000 7fc6add1c700 1 --2- 172.21.15.92:0/2722969871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a80b49a0 0x7fc6a80aaa70 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fc6980098e0 tx=0x7fc6980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a99e635914a9af6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:38.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.678+0000 7fc6ad51b700 1 -- 172.21.15.92:0/2722969871 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc698009e80 con 0x7fc6a80b49a0 2023-10-06T10:44:38.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.678+0000 7fc6ad51b700 1 -- 172.21.15.92:0/2722969871 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69800b820 con 0x7fc6a80b49a0 2023-10-06T10:44:38.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.678+0000 7fc6ad51b700 1 -- 172.21.15.92:0/2722969871 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69800bba0 con 0x7fc6a80b49a0 2023-10-06T10:44:38.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.679+0000 7fc6b0781700 1 -- 172.21.15.92:0/2722969871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a80b49a0 msgr2=0x7fc6a80aaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:38.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.679+0000 7fc6b0781700 1 --2- 172.21.15.92:0/2722969871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a80b49a0 0x7fc6a80aaa70 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fc6980098e0 tx=0x7fc6980091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.679+0000 7fc6b0781700 1 -- 172.21.15.92:0/2722969871 shutdown_connections 2023-10-06T10:44:38.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.679+0000 7fc6b0781700 1 --2- 172.21.15.92:0/2722969871 >> v2:172.21.15.124:3300/0 conn(0x7fc6a8159560 0x7fc6a80aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.679+0000 7fc6b0781700 1 --2- 172.21.15.92:0/2722969871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a80b49a0 0x7fc6a80aaa70 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.679+0000 7fc6b0781700 1 -- 172.21.15.92:0/2722969871 >> 172.21.15.92:0/2722969871 conn(0x7fc6a80997f0 msgr2=0x7fc6a806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:38.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.679+0000 7fc6b0781700 1 -- 172.21.15.92:0/2722969871 shutdown_connections 2023-10-06T10:44:38.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.679+0000 7fc6b0781700 1 -- 172.21.15.92:0/2722969871 wait complete. 2023-10-06T10:44:38.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.680+0000 7fc6b0781700 1 Processor -- start 2023-10-06T10:44:38.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.680+0000 7fc6b0781700 1 -- start start 2023-10-06T10:44:38.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.681+0000 7fc6b0781700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6a80b0760 0x7fc6a8150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:38.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.681+0000 7fc6b0781700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a80b49a0 0x7fc6a8151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:38.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.681+0000 7fc6b0781700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6a8159560 0x7fc6a814db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:38.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.681+0000 7fc6b0781700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a8150b10 con 0x7fc6a80b49a0 2023-10-06T10:44:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.681+0000 7fc6b0781700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a81522c0 con 0x7fc6a8159560 2023-10-06T10:44:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.681+0000 7fc6b0781700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a81504c0 con 0x7fc6a80b0760 2023-10-06T10:44:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.681+0000 7fc6add1c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a80b49a0 0x7fc6a8151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.681+0000 7fc6add1c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a80b49a0 0x7fc6a8151990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46468/0 (socket says 172.21.15.92:46468) 2023-10-06T10:44:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.681+0000 7fc6add1c700 1 -- 172.21.15.92:0/1425298478 learned_addr learned my addr 172.21.15.92:0/1425298478 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.681+0000 7fc6ae51d700 1 --2- 172.21.15.92:0/1425298478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6a80b0760 0x7fc6a8150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.681+0000 7fc6aed1e700 1 --2- 172.21.15.92:0/1425298478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6a8159560 0x7fc6a814db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.681+0000 7fc6add1c700 1 -- 172.21.15.92:0/1425298478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6a80b0760 msgr2=0x7fc6a8150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.681+0000 7fc6add1c700 1 --2- 172.21.15.92:0/1425298478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6a80b0760 0x7fc6a8150fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.681+0000 7fc6add1c700 1 -- 172.21.15.92:0/1425298478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6a8159560 msgr2=0x7fc6a814db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.681+0000 7fc6add1c700 1 --2- 172.21.15.92:0/1425298478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6a8159560 0x7fc6a814db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.681+0000 7fc6add1c700 1 -- 172.21.15.92:0/1425298478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc698009420 con 0x7fc6a80b49a0 2023-10-06T10:44:38.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.682+0000 7fc6add1c700 1 --2- 172.21.15.92:0/1425298478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a80b49a0 0x7fc6a8151990 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fc698013000 tx=0x7fc698017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:38.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.682+0000 7fc69e7fc700 1 -- 172.21.15.92:0/1425298478 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69801f070 con 0x7fc6a80b49a0 2023-10-06T10:44:38.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.682+0000 7fc69e7fc700 1 -- 172.21.15.92:0/1425298478 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc698000cc0 con 0x7fc6a80b49a0 2023-10-06T10:44:38.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.682+0000 7fc6b0781700 1 -- 172.21.15.92:0/1425298478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a81507a0 con 0x7fc6a80b49a0 2023-10-06T10:44:38.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.682+0000 7fc69e7fc700 1 -- 172.21.15.92:0/1425298478 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc698025420 con 0x7fc6a80b49a0 2023-10-06T10:44:38.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.682+0000 7fc6b0781700 1 -- 172.21.15.92:0/1425298478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a81514e0 con 0x7fc6a80b49a0 2023-10-06T10:44:38.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.683+0000 7fc6b0781700 1 -- 172.21.15.92:0/1425298478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a8150c80 con 0x7fc6a80b49a0 2023-10-06T10:44:38.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.683+0000 7fc69e7fc700 1 -- 172.21.15.92:0/1425298478 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc698023070 con 0x7fc6a80b49a0 2023-10-06T10:44:38.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.683+0000 7fc69e7fc700 1 --2- 172.21.15.92:0/1425298478 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc68c066f30 0x7fc68c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:38.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.683+0000 7fc69e7fc700 1 -- 172.21.15.92:0/1425298478 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc698085df0 con 0x7fc6a80b49a0 2023-10-06T10:44:38.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.683+0000 7fc6ae51d700 1 --2- 172.21.15.92:0/1425298478 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc68c066f30 0x7fc68c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:38.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.684+0000 7fc6ae51d700 1 --2- 172.21.15.92:0/1425298478 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc68c066f30 0x7fc68c0693e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc6a815ba40 tx=0x7fc6a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:38.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.688+0000 7fc69e7fc700 1 -- 172.21.15.92:0/1425298478 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc698052dc0 con 0x7fc6a80b49a0 2023-10-06T10:44:38.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.910+0000 7fc6b0781700 1 -- 172.21.15.92:0/1425298478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6a8064a90 con 0x7fc6a80b49a0 2023-10-06T10:44:38.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.910+0000 7fc69e7fc700 1 -- 172.21.15.92:0/1425298478 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4642 (secure 0 0 0) 0x7fc698015090 con 0x7fc6a80b49a0 2023-10-06T10:44:38.912 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:38.912 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2122441982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2122441982},{"type":"v1","addr":"172.21.15.124:6835","nonce":2122441982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:43:47.850832+0000","modified":"2023-10-06T10:44:20.587653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7701,"mds_1":7689,"mds_2":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6836/3605297930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3605297930},{"type":"v1","addr":"172.21.15.92:6836","nonce":3605297930}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7689":{"gid":7689,"name":"c","rank":1,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.92:6837/4047304116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4047304116},{"type":"v1","addr":"172.21.15.92:6837","nonce":4047304116}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7701":{"gid":7701,"name":"b","rank":0,"incarnation":79,"state":"up:active","state_seq":11,"addr":"172.21.15.124:6834/4203118992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":4203118992},{"type":"v1","addr":"172.21.15.124:6834","nonce":4203118992}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T10:44:38.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.912+0000 7fc6b0781700 1 -- 172.21.15.92:0/1425298478 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc68c066f30 msgr2=0x7fc68c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:38.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.912+0000 7fc6b0781700 1 --2- 172.21.15.92:0/1425298478 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc68c066f30 0x7fc68c0693e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc6a815ba40 tx=0x7fc6a4008040 comp rx=0 tx=0).stop 2023-10-06T10:44:38.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.912+0000 7fc6b0781700 1 -- 172.21.15.92:0/1425298478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a80b49a0 msgr2=0x7fc6a8151990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:38.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.912+0000 7fc6b0781700 1 --2- 172.21.15.92:0/1425298478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a80b49a0 0x7fc6a8151990 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fc698013000 tx=0x7fc698017000 comp rx=0 tx=0).stop 2023-10-06T10:44:38.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.912+0000 7fc6b0781700 1 -- 172.21.15.92:0/1425298478 shutdown_connections 2023-10-06T10:44:38.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.912+0000 7fc6b0781700 1 --2- 172.21.15.92:0/1425298478 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc68c066f30 0x7fc68c0693e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.912+0000 7fc6b0781700 1 --2- 172.21.15.92:0/1425298478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6a8159560 0x7fc6a814db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.912+0000 7fc6b0781700 1 --2- 172.21.15.92:0/1425298478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6a80b0760 0x7fc6a8150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.912+0000 7fc6b0781700 1 --2- 172.21.15.92:0/1425298478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a80b49a0 0x7fc6a8151990 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:38.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.912+0000 7fc6b0781700 1 -- 172.21.15.92:0/1425298478 >> 172.21.15.92:0/1425298478 conn(0x7fc6a80997f0 msgr2=0x7fc6a80683c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:38.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.912+0000 7fc6b0781700 1 -- 172.21.15.92:0/1425298478 shutdown_connections 2023-10-06T10:44:38.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:38.912+0000 7fc6b0781700 1 -- 172.21.15.92:0/1425298478 wait complete. 2023-10-06T10:44:38.915 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 92 2023-10-06T10:44:38.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:44:39.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.037+0000 7f0528899700 1 Processor -- start 2023-10-06T10:44:39.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.037+0000 7f0528899700 1 -- start start 2023-10-06T10:44:39.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.038+0000 7f0528899700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0524159440 0x7f05240b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:39.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.038+0000 7f0528899700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05240b35d0 0x7f05240cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:39.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.038+0000 7f0528899700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05240cb7a0 con 0x7f05240cc170 2023-10-06T10:44:39.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.038+0000 7f0528899700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05240b0860 con 0x7f0524159440 2023-10-06T10:44:39.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.038+0000 7f0528899700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05240cb150 con 0x7f05240b35d0 2023-10-06T10:44:39.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.038+0000 7f05237fe700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0524159440 0x7f05240b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:39.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.038+0000 7f0522ffd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05240b35d0 0x7f05240cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:39.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.038+0000 7f0522ffd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05240b35d0 0x7f05240cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:53704/0 (socket says 172.21.15.92:53704) 2023-10-06T10:44:39.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.038+0000 7f05237fe700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0524159440 0x7f05240b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:39518/0 (socket says 172.21.15.92:39518) 2023-10-06T10:44:39.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.038+0000 7f0522ffd700 1 -- 172.21.15.92:0/858262037 learned_addr learned my addr 172.21.15.92:0/858262037 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:39.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.038+0000 7f0523fff700 1 --1- 172.21.15.92:0/858262037 >> v1:172.21.15.124:6789/0 conn(0x7f05240cc170 0x7f05240b2100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:47522/0 (socket says 172.21.15.92:47522) 2023-10-06T10:44:39.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.039+0000 7f05227fc700 1 -- 172.21.15.92:0/858262037 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2292713672 0 0) 0x7f05240cb7a0 con 0x7f05240cc170 2023-10-06T10:44:39.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.039+0000 7f05227fc700 1 -- 172.21.15.92:0/858262037 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f050c003570 con 0x7f05240cc170 2023-10-06T10:44:39.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.039+0000 7f05237fe700 1 -- 172.21.15.92:0/858262037 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05240b35d0 msgr2=0x7f05240cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:39.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.039+0000 7f05237fe700 1 --2- 172.21.15.92:0/858262037 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05240b35d0 0x7f05240cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:39.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.039+0000 7f05237fe700 1 -- 172.21.15.92:0/858262037 >> v1:172.21.15.124:6789/0 conn(0x7f05240cc170 legacy=0x7f05240b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:39.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.039+0000 7f05237fe700 1 -- 172.21.15.92:0/858262037 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05240cb430 con 0x7f0524159440 2023-10-06T10:44:39.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.039+0000 7f0522ffd700 1 --2- 172.21.15.92:0/858262037 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05240b35d0 0x7f05240cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:44:39.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.039+0000 7f05237fe700 1 --2- 172.21.15.92:0/858262037 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0524159440 0x7f05240b3090 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0514009e60 tx=0x7f0514009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee4859316691e35d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:39.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.039+0000 7f05227fc700 1 -- 172.21.15.92:0/858262037 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051400e070 con 0x7f0524159440 2023-10-06T10:44:39.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.039+0000 7f05227fc700 1 -- 172.21.15.92:0/858262037 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051400bab0 con 0x7f0524159440 2023-10-06T10:44:39.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.039+0000 7f05227fc700 1 -- 172.21.15.92:0/858262037 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051400be70 con 0x7f0524159440 2023-10-06T10:44:39.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.039+0000 7f0528899700 1 -- 172.21.15.92:0/858262037 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0524159440 msgr2=0x7f05240b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:39.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.040+0000 7f0528899700 1 --2- 172.21.15.92:0/858262037 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0524159440 0x7f05240b3090 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0514009e60 tx=0x7f0514009780 comp rx=0 tx=0).stop 2023-10-06T10:44:39.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.040+0000 7f0528899700 1 -- 172.21.15.92:0/858262037 shutdown_connections 2023-10-06T10:44:39.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.040+0000 7f0528899700 1 --2- 172.21.15.92:0/858262037 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0524159440 0x7f05240b3090 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:39.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.040+0000 7f0528899700 1 --2- 172.21.15.92:0/858262037 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05240b35d0 0x7f05240cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:39.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.040+0000 7f0528899700 1 -- 172.21.15.92:0/858262037 >> 172.21.15.92:0/858262037 conn(0x7f052406ece0 msgr2=0x7f052406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:39.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.040+0000 7f0528899700 1 -- 172.21.15.92:0/858262037 shutdown_connections 2023-10-06T10:44:39.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.040+0000 7f0528899700 1 -- 172.21.15.92:0/858262037 wait complete. 2023-10-06T10:44:39.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.041+0000 7f0528899700 1 Processor -- start 2023-10-06T10:44:39.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.041+0000 7f0528899700 1 -- start start 2023-10-06T10:44:39.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.042+0000 7f0528899700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05240b35d0 0x7f0524153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:39.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.042+0000 7f0528899700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05240cc170 0x7f05240d0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:39.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.042+0000 7f0528899700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0524159440 0x7f052415ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:39.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.042+0000 7f0528899700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052415c680 con 0x7f0524159440 2023-10-06T10:44:39.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.042+0000 7f0528899700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05240cfdb0 con 0x7f05240cc170 2023-10-06T10:44:39.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.042+0000 7f0528899700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0524154800 con 0x7f05240b35d0 2023-10-06T10:44:39.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.042+0000 7f0523fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0524159440 0x7f052415ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:39.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.042+0000 7f0523fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0524159440 0x7f052415ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46480/0 (socket says 172.21.15.92:46480) 2023-10-06T10:44:39.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.042+0000 7f0523fff700 1 -- 172.21.15.92:0/3567122168 learned_addr learned my addr 172.21.15.92:0/3567122168 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:39.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.042+0000 7f0522ffd700 1 --2- 172.21.15.92:0/3567122168 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05240cc170 0x7f05240d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:39.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.042+0000 7f05237fe700 1 --2- 172.21.15.92:0/3567122168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05240b35d0 0x7f0524153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:39.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.042+0000 7f0523fff700 1 -- 172.21.15.92:0/3567122168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05240b35d0 msgr2=0x7f0524153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:39.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.042+0000 7f0523fff700 1 --2- 172.21.15.92:0/3567122168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05240b35d0 0x7f0524153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:39.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.042+0000 7f0523fff700 1 -- 172.21.15.92:0/3567122168 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05240cc170 msgr2=0x7f05240d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:39.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.042+0000 7f0523fff700 1 --2- 172.21.15.92:0/3567122168 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05240cc170 0x7f05240d0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:39.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.042+0000 7f0523fff700 1 -- 172.21.15.92:0/3567122168 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05180094f0 con 0x7f0524159440 2023-10-06T10:44:39.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.042+0000 7f0523fff700 1 --2- 172.21.15.92:0/3567122168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0524159440 0x7f052415ec30 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f051c00c7c0 tx=0x7f051c007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:39.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.042+0000 7f050b7fe700 1 -- 172.21.15.92:0/3567122168 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c005b40 con 0x7f0524159440 2023-10-06T10:44:39.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.042+0000 7f050b7fe700 1 -- 172.21.15.92:0/3567122168 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051c005ca0 con 0x7f0524159440 2023-10-06T10:44:39.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.042+0000 7f0528899700 1 -- 172.21.15.92:0/3567122168 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0514009a20 con 0x7f0524159440 2023-10-06T10:44:39.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.042+0000 7f050b7fe700 1 -- 172.21.15.92:0/3567122168 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c0072e0 con 0x7f0524159440 2023-10-06T10:44:39.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.042+0000 7f0528899700 1 -- 172.21.15.92:0/3567122168 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052415cb70 con 0x7f0524159440 2023-10-06T10:44:39.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.043+0000 7f0528899700 1 -- 172.21.15.92:0/3567122168 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052405f230 con 0x7f0524159440 2023-10-06T10:44:39.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.044+0000 7f050b7fe700 1 -- 172.21.15.92:0/3567122168 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f051c012400 con 0x7f0524159440 2023-10-06T10:44:39.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.044+0000 7f050b7fe700 1 --2- 172.21.15.92:0/3567122168 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0510066f30 0x7f05100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:39.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.044+0000 7f050b7fe700 1 -- 172.21.15.92:0/3567122168 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f051c083ac0 con 0x7f0524159440 2023-10-06T10:44:39.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.044+0000 7f05237fe700 1 --2- 172.21.15.92:0/3567122168 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0510066f30 0x7f05100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:39.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.044+0000 7f05237fe700 1 --2- 172.21.15.92:0/3567122168 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0510066f30 0x7f05100693e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f05140097c0 tx=0x7f0514019000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:39.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.048+0000 7f050b7fe700 1 -- 172.21.15.92:0/3567122168 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f051c04b5a0 con 0x7f0524159440 2023-10-06T10:44:39.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.233+0000 7f0528899700 1 -- 172.21.15.92:0/3567122168 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05240d0520 con 0x7f0524159440 2023-10-06T10:44:39.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.234+0000 7f050b7fe700 1 -- 172.21.15.92:0/3567122168 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+17074 (secure 0 0 0) 0x7f051c083e60 con 0x7f0524159440 2023-10-06T10:44:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:39.236 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":90,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:43:49.334267+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:43:45.492396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-06T10:43:46.811317+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:44:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.236+0000 7f0528899700 1 -- 172.21.15.92:0/3567122168 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0510066f30 msgr2=0x7f05100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.236+0000 7f0528899700 1 --2- 172.21.15.92:0/3567122168 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0510066f30 0x7f05100693e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f05140097c0 tx=0x7f0514019000 comp rx=0 tx=0).stop 2023-10-06T10:44:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.236+0000 7f0528899700 1 -- 172.21.15.92:0/3567122168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0524159440 msgr2=0x7f052415ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.236+0000 7f0528899700 1 --2- 172.21.15.92:0/3567122168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0524159440 0x7f052415ec30 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f051c00c7c0 tx=0x7f051c007d90 comp rx=0 tx=0).stop 2023-10-06T10:44:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.236+0000 7f0528899700 1 -- 172.21.15.92:0/3567122168 shutdown_connections 2023-10-06T10:44:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.236+0000 7f0528899700 1 --2- 172.21.15.92:0/3567122168 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0510066f30 0x7f05100693e0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.236+0000 7f0528899700 1 --2- 172.21.15.92:0/3567122168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0524159440 0x7f052415ec30 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.236+0000 7f0528899700 1 --2- 172.21.15.92:0/3567122168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05240b35d0 0x7f0524153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:39.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.236+0000 7f0528899700 1 --2- 172.21.15.92:0/3567122168 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05240cc170 0x7f05240d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:39.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.236+0000 7f0528899700 1 -- 172.21.15.92:0/3567122168 >> 172.21.15.92:0/3567122168 conn(0x7f052406ece0 msgr2=0x7f05240b3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:39.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.236+0000 7f0528899700 1 -- 172.21.15.92:0/3567122168 shutdown_connections 2023-10-06T10:44:39.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.236+0000 7f0528899700 1 -- 172.21.15.92:0/3567122168 wait complete. 2023-10-06T10:44:39.250 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T10:44:39.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.364+0000 7fa83eccf700 1 Processor -- start 2023-10-06T10:44:39.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.364+0000 7fa83eccf700 1 -- start start 2023-10-06T10:44:39.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.365+0000 7fa83eccf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa8380b0760 0x7fa8380ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:39.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.365+0000 7fa83eccf700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8380b0c60 con 0x7fa838159560 2023-10-06T10:44:39.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.365+0000 7fa83eccf700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8380b0f40 con 0x7fa8380b49a0 2023-10-06T10:44:39.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.365+0000 7fa83eccf700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8380ab350 con 0x7fa8380b0760 2023-10-06T10:44:39.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.365+0000 7fa83d26c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa8380b0760 0x7fa8380ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:39.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.365+0000 7fa83d26c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa8380b0760 0x7fa8380ccf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46486/0 (socket says 172.21.15.92:46486) 2023-10-06T10:44:39.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.365+0000 7fa83d26c700 1 -- 172.21.15.92:0/2279157206 learned_addr learned my addr 172.21.15.92:0/2279157206 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:39.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.365+0000 7fa83ca6b700 1 --1- 172.21.15.92:0/2279157206 >> v1:172.21.15.92:6789/0 conn(0x7fa838159560 0x7fa8380aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:42972/0 (socket says 172.21.15.92:42972) 2023-10-06T10:44:39.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.365+0000 7fa83d26c700 1 -- 172.21.15.92:0/2279157206 >> v1:172.21.15.124:6789/0 conn(0x7fa8380b49a0 legacy=0x7fa8380aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:39.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.365+0000 7fa83d26c700 1 -- 172.21.15.92:0/2279157206 >> v1:172.21.15.92:6789/0 conn(0x7fa838159560 legacy=0x7fa8380aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:39.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.365+0000 7fa83d26c700 1 -- 172.21.15.92:0/2279157206 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8380b1220 con 0x7fa8380b0760 2023-10-06T10:44:39.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.365+0000 7fa82f7fe700 1 -- 172.21.15.92:0/2279157206 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1566382734 0 0) 0x7fa8380b0c60 con 0x7fa838159560 2023-10-06T10:44:39.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.365+0000 7fa83d26c700 1 --2- 172.21.15.92:0/2279157206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8380b0760 0x7fa8380ccf90 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fa830009f50 tx=0x7fa830009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7022aee5a987ba1b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:39.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.365+0000 7fa82f7fe700 1 -- 172.21.15.92:0/2279157206 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83000e070 con 0x7fa8380b0760 2023-10-06T10:44:39.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.365+0000 7fa82f7fe700 1 -- 172.21.15.92:0/2279157206 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83000ba70 con 0x7fa8380b0760 2023-10-06T10:44:39.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.365+0000 7fa82f7fe700 1 -- 172.21.15.92:0/2279157206 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8300092e0 con 0x7fa8380b0760 2023-10-06T10:44:39.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.366+0000 7fa83eccf700 1 -- 172.21.15.92:0/2279157206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8380b0760 msgr2=0x7fa8380ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:39.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.366+0000 7fa83eccf700 1 --2- 172.21.15.92:0/2279157206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8380b0760 0x7fa8380ccf90 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fa830009f50 tx=0x7fa830009840 comp rx=0 tx=0).stop 2023-10-06T10:44:39.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.366+0000 7fa83eccf700 1 -- 172.21.15.92:0/2279157206 shutdown_connections 2023-10-06T10:44:39.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.366+0000 7fa83eccf700 1 --2- 172.21.15.92:0/2279157206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8380b0760 0x7fa8380ccf90 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:39.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.366+0000 7fa83eccf700 1 -- 172.21.15.92:0/2279157206 >> 172.21.15.92:0/2279157206 conn(0x7fa8380997f0 msgr2=0x7fa83806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:39.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.366+0000 7fa83eccf700 1 -- 172.21.15.92:0/2279157206 shutdown_connections 2023-10-06T10:44:39.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.366+0000 7fa83eccf700 1 -- 172.21.15.92:0/2279157206 wait complete. 2023-10-06T10:44:39.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.367+0000 7fa83eccf700 1 Processor -- start 2023-10-06T10:44:39.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.367+0000 7fa83eccf700 1 -- start start 2023-10-06T10:44:39.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.368+0000 7fa83eccf700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8380b0760 0x7fa83815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:39.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.368+0000 7fa83eccf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8380b49a0 0x7fa838156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:39.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.368+0000 7fa83eccf700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa838159560 0x7fa8380d0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:39.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.368+0000 7fa83eccf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83816da80 con 0x7fa8380b49a0 2023-10-06T10:44:39.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.368+0000 7fa83eccf700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83809cc70 con 0x7fa8380b0760 2023-10-06T10:44:39.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.368+0000 7fa83eccf700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83809c3c0 con 0x7fa838159560 2023-10-06T10:44:39.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.368+0000 7fa82ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8380b49a0 0x7fa838156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:39.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.368+0000 7fa82ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8380b49a0 0x7fa838156870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46488/0 (socket says 172.21.15.92:46488) 2023-10-06T10:44:39.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.368+0000 7fa82ffff700 1 -- 172.21.15.92:0/1371558916 learned_addr learned my addr 172.21.15.92:0/1371558916 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:39.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.368+0000 7fa83d26c700 1 --2- 172.21.15.92:0/1371558916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa838159560 0x7fa8380d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:39.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.368+0000 7fa83ca6b700 1 --2- 172.21.15.92:0/1371558916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8380b0760 0x7fa83815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:39.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.368+0000 7fa82ffff700 1 -- 172.21.15.92:0/1371558916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa838159560 msgr2=0x7fa8380d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:39.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.368+0000 7fa82ffff700 1 --2- 172.21.15.92:0/1371558916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa838159560 0x7fa8380d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:39.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.368+0000 7fa82ffff700 1 -- 172.21.15.92:0/1371558916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8380b0760 msgr2=0x7fa83815ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:39.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.368+0000 7fa82ffff700 1 --2- 172.21.15.92:0/1371558916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8380b0760 0x7fa83815ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:39.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.368+0000 7fa82ffff700 1 -- 172.21.15.92:0/1371558916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa830009ae0 con 0x7fa8380b49a0 2023-10-06T10:44:39.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.368+0000 7fa82ffff700 1 --2- 172.21.15.92:0/1371558916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8380b49a0 0x7fa838156870 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fa82800c790 tx=0x7fa828007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:39.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.368+0000 7fa82cff9700 1 -- 172.21.15.92:0/1371558916 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa828011590 con 0x7fa8380b49a0 2023-10-06T10:44:39.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.368+0000 7fa82cff9700 1 -- 172.21.15.92:0/1371558916 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8280116f0 con 0x7fa8380b49a0 2023-10-06T10:44:39.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.368+0000 7fa82cff9700 1 -- 172.21.15.92:0/1371558916 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa828011ab0 con 0x7fa8380b49a0 2023-10-06T10:44:39.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.369+0000 7fa83eccf700 1 -- 172.21.15.92:0/1371558916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa83809c660 con 0x7fa8380b49a0 2023-10-06T10:44:39.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.369+0000 7fa83eccf700 1 -- 172.21.15.92:0/1371558916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83809c900 con 0x7fa8380b49a0 2023-10-06T10:44:39.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.369+0000 7fa83eccf700 1 -- 172.21.15.92:0/1371558916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa838064a90 con 0x7fa8380b49a0 2023-10-06T10:44:39.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.370+0000 7fa82cff9700 1 -- 172.21.15.92:0/1371558916 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa82801a360 con 0x7fa8380b49a0 2023-10-06T10:44:39.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.370+0000 7fa82cff9700 1 --2- 172.21.15.92:0/1371558916 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa824066f30 0x7fa8240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:39.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.370+0000 7fa82cff9700 1 -- 172.21.15.92:0/1371558916 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa828083820 con 0x7fa8380b49a0 2023-10-06T10:44:39.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.370+0000 7fa83ca6b700 1 --2- 172.21.15.92:0/1371558916 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa824066f30 0x7fa8240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:39.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.371+0000 7fa83ca6b700 1 --2- 172.21.15.92:0/1371558916 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa824066f30 0x7fa8240693e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa834000c00 tx=0x7fa834009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:39.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.374+0000 7fa82cff9700 1 -- 172.21.15.92:0/1371558916 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8280507f0 con 0x7fa8380b49a0 2023-10-06T10:44:39.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.601+0000 7fa83eccf700 1 -- 172.21.15.92:0/1371558916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa83809ce00 con 0x7fa8380b49a0 2023-10-06T10:44:39.606 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T10:44:39.602+0000 7f9e7f3ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T10:44:39.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.606+0000 7fa82cff9700 1 -- 172.21.15.92:0/1371558916 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v93) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa828053e10 con 0x7fa8380b49a0 2023-10-06T10:44:39.611 INFO:tasks.ceph.mds.a.smithi092.stderr: -991> 2023-10-06T10:44:16.533+0000 7fc921715700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2023-10-06T10:43:59.982922+0000 1=0+1) n(v0 rc2023-10-06T10:43:59.982922+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={7866=pAsLsXsFsx/pAsLsXsFsx@4},l=7866 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x56472c3c9080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:43:59.982922+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:43:59.982922+0000 2=0+2)","memory_value.dirstat":"f(v0 m2023-10-06T10:43:59.982922+0000 1=0+1)","memory_value.rstat":"n(v0 rc2023-10-06T10:43:59.982922+0000 2=0+2)","error_str":""},"return_code":-61} 2023-10-06T10:44:39.611 INFO:tasks.ceph.mds.a.smithi092.stderr: -969> 2023-10-06T10:44:16.533+0000 7fc921715700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={7866=pAsLsXsFsx/AsLsXsFsx@2},l=7866 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x56472c3c9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2023-10-06T10:44:39.612 INFO:tasks.ceph.mds.b.smithi124.stderr: -9818> 2023-10-06T10:43:57.765+0000 7f557baa9700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-06T10:44:39.613 INFO:tasks.ceph.mds.c.smithi092.stderr: -1097> 2023-10-06T10:44:16.532+0000 7f13de217700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2023-10-06T10:43:59.981922+0000 1=0+1) n(v2 rc2023-10-06T10:44:00.210915+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={7866=pAsLsXsFsx/pAsLsXsFsx@4},l=7866 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x558f21c5d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:43:59.981922+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:44:00.210915+0000 4=0+4)","memory_value.dirstat":"f(v0 m2023-10-06T10:43:59.981922+0000 1=0+1)","memory_value.rstat":"n(v2 rc2023-10-06T10:44:00.210915+0000 4=0+4)","error_str":""},"return_code":-61} 2023-10-06T10:44:39.613 INFO:tasks.ceph.mds.c.smithi092.stderr: -968> 2023-10-06T10:44:16.534+0000 7f13e4223700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2023-10-06T10:43:59.982922+0000 1=0+1) n(v1 rc2023-10-06T10:44:00.210915+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={7866=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x558f21c5c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:43:59.982922+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:43:59.982922+0000 3=0+3)","memory_value.dirstat":"f(v0 m2023-10-06T10:43:59.982922+0000 1=0+1)","memory_value.rstat":"n(v1 rc2023-10-06T10:44:00.210915+0000 3=0+3)","error_str":""},"return_code":-61} 2023-10-06T10:44:39.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.612+0000 7fa83eccf700 1 -- 172.21.15.92:0/1371558916 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa824066f30 msgr2=0x7fa8240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:39.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.612+0000 7fa83eccf700 1 --2- 172.21.15.92:0/1371558916 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa824066f30 0x7fa8240693e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa834000c00 tx=0x7fa834009040 comp rx=0 tx=0).stop 2023-10-06T10:44:39.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.612+0000 7fa83eccf700 1 -- 172.21.15.92:0/1371558916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8380b49a0 msgr2=0x7fa838156870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:39.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.612+0000 7fa83eccf700 1 --2- 172.21.15.92:0/1371558916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8380b49a0 0x7fa838156870 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fa82800c790 tx=0x7fa828007f20 comp rx=0 tx=0).stop 2023-10-06T10:44:39.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.612+0000 7fa83eccf700 1 -- 172.21.15.92:0/1371558916 shutdown_connections 2023-10-06T10:44:39.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.612+0000 7fa83eccf700 1 --2- 172.21.15.92:0/1371558916 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa824066f30 0x7fa8240693e0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:39.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.612+0000 7fa83eccf700 1 --2- 172.21.15.92:0/1371558916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa838159560 0x7fa8380d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:39.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.612+0000 7fa83eccf700 1 --2- 172.21.15.92:0/1371558916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8380b0760 0x7fa83815ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:39.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.612+0000 7fa83eccf700 1 --2- 172.21.15.92:0/1371558916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8380b49a0 0x7fa838156870 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:39.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.612+0000 7fa83eccf700 1 -- 172.21.15.92:0/1371558916 >> 172.21.15.92:0/1371558916 conn(0x7fa8380997f0 msgr2=0x7fa8380695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:39.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.612+0000 7fa83eccf700 1 -- 172.21.15.92:0/1371558916 shutdown_connections 2023-10-06T10:44:39.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.612+0000 7fa83eccf700 1 -- 172.21.15.92:0/1371558916 wait complete. 2023-10-06T10:44:39.615 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T10:44:39.621 INFO:tasks.ceph.mds.b.smithi124.stderr: -2363> 2023-10-06T10:44:16.532+0000 7f55782a2700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2023-10-06T10:43:59.980922+0000 1=0+1) n(v3 rc2023-10-06T10:44:00.210915+0000 6=0+6) (inest mix dirty) (ifile excl) (iversion lock) caps={7866=pAsLsXsFsx/pAsLsXsFsx@5},l=7866 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55cab959b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:43:59.980922+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:44:00.210915+0000 6=0+6)","memory_value.dirstat":"f(v0 m2023-10-06T10:43:59.980922+0000 1=0+1)","memory_value.rstat":"n(v3 rc2023-10-06T10:44:00.210915+0000 6=0+6)","error_str":""},"return_code":-61} 2023-10-06T10:44:39.621 INFO:tasks.ceph.mds.b.smithi124.stderr: -2264> 2023-10-06T10:44:16.533+0000 7f557e2ae700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2023-10-06T10:43:59.980922+0000 1=0+1) n(v4 rc2023-10-06T10:44:00.210915+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={7866=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x55cab9672000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:43:59.980922+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:44:00.210915+0000 5=0+5)","memory_value.dirstat":"f(v0 m2023-10-06T10:43:59.980922+0000 1=0+1)","memory_value.rstat":"n(v4 rc2023-10-06T10:44:00.210915+0000 5=0+5)","error_str":""},"return_code":-61} 2023-10-06T10:44:39.627 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T10:44:39.633 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T10:44:39.633 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T10:44:39.645 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-06T10:44:39.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.743+0000 7fd5f5173700 1 Processor -- start 2023-10-06T10:44:39.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.743+0000 7fd5f5173700 1 -- start start 2023-10-06T10:44:39.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.744+0000 7fd5f5173700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd5f0142ee0 0x7fd5f006b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:39.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.744+0000 7fd5f5173700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f006b170 con 0x7fd5f0067cc0 2023-10-06T10:44:39.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.744+0000 7fd5f5173700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f01432b0 con 0x7fd5f006bb40 2023-10-06T10:44:39.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.744+0000 7fd5f5173700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f00cb9d0 con 0x7fd5f0142ee0 2023-10-06T10:44:39.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.744+0000 7fd5ef59e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd5f0067cc0 0x7fd5f00c77f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:42976/0 (socket says 172.21.15.92:42976) 2023-10-06T10:44:39.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.744+0000 7fd5ef59e700 1 -- 172.21.15.92:0/2976052808 learned_addr learned my addr 172.21.15.92:0/2976052808 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:39.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.744+0000 7fd5eed9d700 1 --2- 172.21.15.92:0/2976052808 >> v2:172.21.15.124:3300/0 conn(0x7fd5f0142ee0 0x7fd5f006b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:39.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.744+0000 7fd5ee59c700 1 -- 172.21.15.92:0/2976052808 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3163114896 0 0) 0x7fd5f006b170 con 0x7fd5f0067cc0 2023-10-06T10:44:39.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.744+0000 7fd5ee59c700 1 -- 172.21.15.92:0/2976052808 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5dc003570 con 0x7fd5f0067cc0 2023-10-06T10:44:39.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.745+0000 7fd5eed9d700 1 -- 172.21.15.92:0/2976052808 >> v1:172.21.15.124:6789/0 conn(0x7fd5f006bb40 legacy=0x7fd5f00675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:39.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.745+0000 7fd5eed9d700 1 -- 172.21.15.92:0/2976052808 >> v1:172.21.15.92:6789/0 conn(0x7fd5f0067cc0 legacy=0x7fd5f00c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:39.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.745+0000 7fd5ee59c700 1 -- 172.21.15.92:0/2976052808 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4201603675 0 0) 0x7fd5dc003570 con 0x7fd5f0067cc0 2023-10-06T10:44:39.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.745+0000 7fd5eed9d700 1 -- 172.21.15.92:0/2976052808 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f006a840 con 0x7fd5f0142ee0 2023-10-06T10:44:39.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.745+0000 7fd5eed9d700 1 --2- 172.21.15.92:0/2976052808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5f0142ee0 0x7fd5f006b600 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00098e0 tx=0x7fd5e00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4c157c12fce2909 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:39.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.745+0000 7fd5ee59c700 1 -- 172.21.15.92:0/2976052808 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e80017f0 con 0x7fd5f0142ee0 2023-10-06T10:44:39.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.745+0000 7fd5ee59c700 1 -- 172.21.15.92:0/2976052808 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e0011420 con 0x7fd5f0142ee0 2023-10-06T10:44:39.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.745+0000 7fd5ee59c700 1 -- 172.21.15.92:0/2976052808 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e00117c0 con 0x7fd5f0142ee0 2023-10-06T10:44:39.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.745+0000 7fd5f5173700 1 -- 172.21.15.92:0/2976052808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5f0142ee0 msgr2=0x7fd5f006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:39.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.745+0000 7fd5f5173700 1 --2- 172.21.15.92:0/2976052808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5f0142ee0 0x7fd5f006b600 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00098e0 tx=0x7fd5e00091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:39.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.745+0000 7fd5f5173700 1 -- 172.21.15.92:0/2976052808 shutdown_connections 2023-10-06T10:44:39.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.745+0000 7fd5f5173700 1 --2- 172.21.15.92:0/2976052808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5f0142ee0 0x7fd5f006b600 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:39.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.745+0000 7fd5f5173700 1 -- 172.21.15.92:0/2976052808 >> 172.21.15.92:0/2976052808 conn(0x7fd5f00cd1b0 msgr2=0x7fd5f00cd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:39.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.746+0000 7fd5f5173700 1 -- 172.21.15.92:0/2976052808 shutdown_connections 2023-10-06T10:44:39.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.746+0000 7fd5f5173700 1 -- 172.21.15.92:0/2976052808 wait complete. 2023-10-06T10:44:39.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.747+0000 7fd5f5173700 1 Processor -- start 2023-10-06T10:44:39.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.747+0000 7fd5f5173700 1 -- start start 2023-10-06T10:44:39.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.747+0000 7fd5f5173700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5f0067cc0 0x7fd5f015da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:39.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.748+0000 7fd5f5173700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5f006bb40 0x7fd5f0156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:39.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.748+0000 7fd5f5173700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5f0142ee0 0x7fd5f00c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:39.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.748+0000 7fd5f5173700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f015ea10 con 0x7fd5f006bb40 2023-10-06T10:44:39.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.748+0000 7fd5f5173700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f00c5300 con 0x7fd5f0067cc0 2023-10-06T10:44:39.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.748+0000 7fd5f5173700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f00c46f0 con 0x7fd5f0142ee0 2023-10-06T10:44:39.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.748+0000 7fd5e659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5f006bb40 0x7fd5f0156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:39.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.748+0000 7fd5e659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5f006bb40 0x7fd5f0156d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46530/0 (socket says 172.21.15.92:46530) 2023-10-06T10:44:39.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.748+0000 7fd5e659c700 1 -- 172.21.15.92:0/931341984 learned_addr learned my addr 172.21.15.92:0/931341984 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:39.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.748+0000 7fd5eed9d700 1 --2- 172.21.15.92:0/931341984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5f0067cc0 0x7fd5f015da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:39.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.748+0000 7fd5ef59e700 1 --2- 172.21.15.92:0/931341984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5f0142ee0 0x7fd5f00c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:39.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.748+0000 7fd5e659c700 1 -- 172.21.15.92:0/931341984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5f0142ee0 msgr2=0x7fd5f00c2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:39.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.748+0000 7fd5e659c700 1 --2- 172.21.15.92:0/931341984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5f0142ee0 0x7fd5f00c2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:39.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.748+0000 7fd5e659c700 1 -- 172.21.15.92:0/931341984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5f0067cc0 msgr2=0x7fd5f015da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:39.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.748+0000 7fd5e659c700 1 --2- 172.21.15.92:0/931341984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5f0067cc0 0x7fd5f015da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:39.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.748+0000 7fd5e659c700 1 -- 172.21.15.92:0/931341984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e0009420 con 0x7fd5f006bb40 2023-10-06T10:44:39.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.748+0000 7fd5ef59e700 1 --2- 172.21.15.92:0/931341984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5f0142ee0 0x7fd5f00c2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:39.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.748+0000 7fd5e659c700 1 --2- 172.21.15.92:0/931341984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5f006bb40 0x7fd5f0156d30 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800c590 tx=0x7fd5d800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:39.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.748+0000 7fd5e77fe700 1 -- 172.21.15.92:0/931341984 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8011590 con 0x7fd5f006bb40 2023-10-06T10:44:39.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.748+0000 7fd5e77fe700 1 -- 172.21.15.92:0/931341984 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d80116f0 con 0x7fd5f006bb40 2023-10-06T10:44:39.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.748+0000 7fd5f5173700 1 -- 172.21.15.92:0/931341984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f00c49d0 con 0x7fd5f006bb40 2023-10-06T10:44:39.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.748+0000 7fd5e77fe700 1 -- 172.21.15.92:0/931341984 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8011ab0 con 0x7fd5f006bb40 2023-10-06T10:44:39.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.748+0000 7fd5f5173700 1 -- 172.21.15.92:0/931341984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f00c4d60 con 0x7fd5f006bb40 2023-10-06T10:44:39.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.749+0000 7fd5f5173700 1 -- 172.21.15.92:0/931341984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f00c54b0 con 0x7fd5f006bb40 2023-10-06T10:44:39.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.750+0000 7fd5e77fe700 1 -- 172.21.15.92:0/931341984 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd5d801b3b0 con 0x7fd5f006bb40 2023-10-06T10:44:39.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.750+0000 7fd5e77fe700 1 --2- 172.21.15.92:0/931341984 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd5d0066f30 0x7fd5d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:39.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.750+0000 7fd5e77fe700 1 -- 172.21.15.92:0/931341984 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fd5d8083e60 con 0x7fd5f006bb40 2023-10-06T10:44:39.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.750+0000 7fd5eed9d700 1 --2- 172.21.15.92:0/931341984 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd5d0066f30 0x7fd5d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:39.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.750+0000 7fd5eed9d700 1 --2- 172.21.15.92:0/931341984 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd5d0066f30 0x7fd5d00693e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00091e0 tx=0x7fd5e0016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:39.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.754+0000 7fd5e77fe700 1 -- 172.21.15.92:0/931341984 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5f00c54b0 con 0x7fd5f006bb40 2023-10-06T10:44:39.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:39.977+0000 7fd5f5173700 1 -- 172.21.15.92:0/931341984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd5f015e050 con 0x7fd5f006bb40 2023-10-06T10:44:40.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.607+0000 7fd5e77fe700 1 -- 172.21.15.92:0/931341984 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v94) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd5d8017020 con 0x7fd5f006bb40 2023-10-06T10:44:40.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.610+0000 7fd5f5173700 1 -- 172.21.15.92:0/931341984 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd5d0066f30 msgr2=0x7fd5d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.610+0000 7fd5f5173700 1 --2- 172.21.15.92:0/931341984 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd5d0066f30 0x7fd5d00693e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00091e0 tx=0x7fd5e0016000 comp rx=0 tx=0).stop 2023-10-06T10:44:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.610+0000 7fd5f5173700 1 -- 172.21.15.92:0/931341984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5f006bb40 msgr2=0x7fd5f0156d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.610+0000 7fd5f5173700 1 --2- 172.21.15.92:0/931341984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5f006bb40 0x7fd5f0156d30 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800c590 tx=0x7fd5d800c950 comp rx=0 tx=0).stop 2023-10-06T10:44:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.610+0000 7fd5f5173700 1 -- 172.21.15.92:0/931341984 shutdown_connections 2023-10-06T10:44:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.610+0000 7fd5f5173700 1 --2- 172.21.15.92:0/931341984 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd5d0066f30 0x7fd5d00693e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.610+0000 7fd5f5173700 1 --2- 172.21.15.92:0/931341984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5f0142ee0 0x7fd5f00c2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.610+0000 7fd5f5173700 1 --2- 172.21.15.92:0/931341984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5f0067cc0 0x7fd5f015da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.610+0000 7fd5f5173700 1 --2- 172.21.15.92:0/931341984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5f006bb40 0x7fd5f0156d30 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.610+0000 7fd5f5173700 1 -- 172.21.15.92:0/931341984 >> 172.21.15.92:0/931341984 conn(0x7fd5f00cd1b0 msgr2=0x7fd5f01419d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.610+0000 7fd5f5173700 1 -- 172.21.15.92:0/931341984 shutdown_connections 2023-10-06T10:44:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.610+0000 7fd5f5173700 1 -- 172.21.15.92:0/931341984 wait complete. 2023-10-06T10:44:40.626 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T10:44:40.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.783+0000 7f3d0ddea700 1 Processor -- start 2023-10-06T10:44:40.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.783+0000 7f3d0ddea700 1 -- start start 2023-10-06T10:44:40.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.784+0000 7f3d0ddea700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d080b2360 0x7f3d080b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:40.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.784+0000 7f3d0ddea700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d080aede0 0x7f3d080af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:40.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.784+0000 7f3d0ddea700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d080b27d0 con 0x7f3d0806ece0 2023-10-06T10:44:40.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.784+0000 7f3d0ddea700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d0806fb40 con 0x7f3d080aede0 2023-10-06T10:44:40.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.784+0000 7f3d0ddea700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d080b2ce0 con 0x7f3d080b2360 2023-10-06T10:44:40.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.784+0000 7f3d07fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d080aede0 0x7f3d080af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:40.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.784+0000 7f3d07fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d080aede0 0x7f3d080af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46532/0 (socket says 172.21.15.92:46532) 2023-10-06T10:44:40.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.784+0000 7f3d07fff700 1 -- 172.21.15.92:0/2852425922 learned_addr learned my addr 172.21.15.92:0/2852425922 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:40.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.785+0000 7f3d07fff700 1 -- 172.21.15.92:0/2852425922 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d080b2360 msgr2=0x7f3d080b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:40.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.785+0000 7f3d07fff700 1 --2- 172.21.15.92:0/2852425922 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d080b2360 0x7f3d080b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:40.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.785+0000 7f3d07fff700 1 -- 172.21.15.92:0/2852425922 >> v1:172.21.15.124:6789/0 conn(0x7f3d0806ece0 legacy=0x7f3d0806f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:40.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.785+0000 7f3d07fff700 1 -- 172.21.15.92:0/2852425922 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d080b0410 con 0x7f3d080aede0 2023-10-06T10:44:40.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.785+0000 7f3d07fff700 1 --2- 172.21.15.92:0/2852425922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d080aede0 0x7f3d080af240 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f3d000099b0 tx=0x7f3d00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab64101fbf64169 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:40.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.785+0000 7f3d067fc700 1 -- 172.21.15.92:0/2852425922 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0000b820 con 0x7f3d080aede0 2023-10-06T10:44:40.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.785+0000 7f3d067fc700 1 -- 172.21.15.92:0/2852425922 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0000b980 con 0x7f3d080aede0 2023-10-06T10:44:40.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.785+0000 7f3d067fc700 1 -- 172.21.15.92:0/2852425922 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0000bd20 con 0x7f3d080aede0 2023-10-06T10:44:40.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.785+0000 7f3d0ddea700 1 -- 172.21.15.92:0/2852425922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d080aede0 msgr2=0x7f3d080af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:40.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.785+0000 7f3d0ddea700 1 --2- 172.21.15.92:0/2852425922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d080aede0 0x7f3d080af240 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f3d000099b0 tx=0x7f3d00009250 comp rx=0 tx=0).stop 2023-10-06T10:44:40.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.785+0000 7f3d0ddea700 1 -- 172.21.15.92:0/2852425922 shutdown_connections 2023-10-06T10:44:40.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.785+0000 7f3d0ddea700 1 --2- 172.21.15.92:0/2852425922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d080aede0 0x7f3d080af240 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:40.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.785+0000 7f3d0ddea700 1 --2- 172.21.15.92:0/2852425922 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d080b2360 0x7f3d080b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:40.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.785+0000 7f3d0ddea700 1 -- 172.21.15.92:0/2852425922 >> 172.21.15.92:0/2852425922 conn(0x7f3d080682f0 msgr2=0x7f3d080686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:40.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.786+0000 7f3d0ddea700 1 -- 172.21.15.92:0/2852425922 shutdown_connections 2023-10-06T10:44:40.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.786+0000 7f3d0ddea700 1 -- 172.21.15.92:0/2852425922 wait complete. 2023-10-06T10:44:40.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.787+0000 7f3d0ddea700 1 Processor -- start 2023-10-06T10:44:40.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.787+0000 7f3d0ddea700 1 -- start start 2023-10-06T10:44:40.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.787+0000 7f3d0ddea700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d0806ece0 0x7f3d0814b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:40.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.787+0000 7f3d0ddea700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d080aede0 0x7f3d080c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:40.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.787+0000 7f3d0ddea700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d080b2360 0x7f3d080ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:40.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.787+0000 7f3d0ddea700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d081475c0 con 0x7f3d080b2360 2023-10-06T10:44:40.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.787+0000 7f3d0ddea700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d081469b0 con 0x7f3d080aede0 2023-10-06T10:44:40.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.787+0000 7f3d0ddea700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d08146c90 con 0x7f3d0806ece0 2023-10-06T10:44:40.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.788+0000 7f3d07fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d080b2360 0x7f3d080ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:40.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.788+0000 7f3d07fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d080b2360 0x7f3d080ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46542/0 (socket says 172.21.15.92:46542) 2023-10-06T10:44:40.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.788+0000 7f3d07fff700 1 -- 172.21.15.92:0/637035533 learned_addr learned my addr 172.21.15.92:0/637035533 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:40.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.788+0000 7f3d077fe700 1 --2- 172.21.15.92:0/637035533 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d0806ece0 0x7f3d0814b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:40.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.788+0000 7f3d06ffd700 1 --2- 172.21.15.92:0/637035533 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d080aede0 0x7f3d080c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:40.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.788+0000 7f3d07fff700 1 -- 172.21.15.92:0/637035533 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d0806ece0 msgr2=0x7f3d0814b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:40.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.788+0000 7f3d07fff700 1 --2- 172.21.15.92:0/637035533 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d0806ece0 0x7f3d0814b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:40.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.788+0000 7f3d07fff700 1 -- 172.21.15.92:0/637035533 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d080aede0 msgr2=0x7f3d080c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:40.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.788+0000 7f3d07fff700 1 --2- 172.21.15.92:0/637035533 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d080aede0 0x7f3d080c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:40.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.788+0000 7f3d07fff700 1 -- 172.21.15.92:0/637035533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d000094f0 con 0x7f3d080b2360 2023-10-06T10:44:40.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.788+0000 7f3d07fff700 1 --2- 172.21.15.92:0/637035533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d080b2360 0x7f3d080ca300 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009290 tx=0x7f3d00016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:40.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.788+0000 7f3cef7fe700 1 -- 172.21.15.92:0/637035533 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d00026480 con 0x7f3d080b2360 2023-10-06T10:44:40.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.788+0000 7f3cef7fe700 1 -- 172.21.15.92:0/637035533 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0001f070 con 0x7f3d080b2360 2023-10-06T10:44:40.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.788+0000 7f3d0ddea700 1 -- 172.21.15.92:0/637035533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d08146f70 con 0x7f3d080b2360 2023-10-06T10:44:40.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.788+0000 7f3cef7fe700 1 -- 172.21.15.92:0/637035533 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d00026830 con 0x7f3d080b2360 2023-10-06T10:44:40.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.788+0000 7f3d0ddea700 1 -- 172.21.15.92:0/637035533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d08148f10 con 0x7f3d080b2360 2023-10-06T10:44:40.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.789+0000 7f3d0ddea700 1 -- 172.21.15.92:0/637035533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d08147730 con 0x7f3d080b2360 2023-10-06T10:44:40.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.789+0000 7f3cef7fe700 1 -- 172.21.15.92:0/637035533 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d00026c60 con 0x7f3d080b2360 2023-10-06T10:44:40.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.790+0000 7f3cef7fe700 1 --2- 172.21.15.92:0/637035533 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3cf4066f30 0x7f3cf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:40.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.790+0000 7f3cef7fe700 1 -- 172.21.15.92:0/637035533 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f3d00021070 con 0x7f3d080b2360 2023-10-06T10:44:40.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.790+0000 7f3d077fe700 1 --2- 172.21.15.92:0/637035533 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3cf4066f30 0x7f3cf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:40.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.790+0000 7f3d077fe700 1 --2- 172.21.15.92:0/637035533 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3cf4066f30 0x7f3cf40693e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8007560 tx=0x7f3cf8009760 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:40.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.794+0000 7f3cef7fe700 1 -- 172.21.15.92:0/637035533 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d00053ac0 con 0x7f3d080b2360 2023-10-06T10:44:40.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:40.978+0000 7f3d0ddea700 1 -- 172.21.15.92:0/637035533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3d08064a90 con 0x7f3d080b2360 2023-10-06T10:44:41.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.670+0000 7f3cef7fe700 1 -- 172.21.15.92:0/637035533 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v92) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3d000570e0 con 0x7f3d080b2360 2023-10-06T10:44:41.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.673+0000 7f3d0ddea700 1 -- 172.21.15.92:0/637035533 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3cf4066f30 msgr2=0x7f3cf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:41.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.673+0000 7f3d0ddea700 1 --2- 172.21.15.92:0/637035533 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3cf4066f30 0x7f3cf40693e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8007560 tx=0x7f3cf8009760 comp rx=0 tx=0).stop 2023-10-06T10:44:41.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.673+0000 7f3d0ddea700 1 -- 172.21.15.92:0/637035533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d080b2360 msgr2=0x7f3d080ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:41.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.673+0000 7f3d0ddea700 1 --2- 172.21.15.92:0/637035533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d080b2360 0x7f3d080ca300 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009290 tx=0x7f3d00016000 comp rx=0 tx=0).stop 2023-10-06T10:44:41.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.673+0000 7f3d0ddea700 1 -- 172.21.15.92:0/637035533 shutdown_connections 2023-10-06T10:44:41.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.673+0000 7f3d0ddea700 1 --2- 172.21.15.92:0/637035533 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3cf4066f30 0x7f3cf40693e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:41.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.673+0000 7f3d0ddea700 1 --2- 172.21.15.92:0/637035533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d080b2360 0x7f3d080ca300 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:41.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.673+0000 7f3d0ddea700 1 --2- 172.21.15.92:0/637035533 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d0806ece0 0x7f3d0814b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:41.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.673+0000 7f3d0ddea700 1 --2- 172.21.15.92:0/637035533 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d080aede0 0x7f3d080c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:41.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.673+0000 7f3d0ddea700 1 -- 172.21.15.92:0/637035533 >> 172.21.15.92:0/637035533 conn(0x7f3d080682f0 msgr2=0x7f3d080d0820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:41.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.673+0000 7f3d0ddea700 1 -- 172.21.15.92:0/637035533 shutdown_connections 2023-10-06T10:44:41.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.673+0000 7f3d0ddea700 1 -- 172.21.15.92:0/637035533 wait complete. 2023-10-06T10:44:41.676 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-10-06T10:44:41.689 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T10:44:41.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.813+0000 7f04a9081700 1 Processor -- start 2023-10-06T10:44:41.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.813+0000 7f04a9081700 1 -- start start 2023-10-06T10:44:41.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.814+0000 7f04a9081700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04a4153950 0x7f04a415c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:41.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.814+0000 7f04a9081700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f04a4153d20 0x7f04a41577a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:41.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.814+0000 7f04a9081700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a415c400 con 0x7f04a4157ce0 2023-10-06T10:44:41.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.814+0000 7f04a9081700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a4158200 con 0x7f04a4153d20 2023-10-06T10:44:41.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.814+0000 7f04a9081700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a415b7f0 con 0x7f04a4153950 2023-10-06T10:44:41.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.814+0000 7f04a259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f04a4153d20 0x7f04a41577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:41.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.814+0000 7f04a259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f04a4153d20 0x7f04a41577a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46550/0 (socket says 172.21.15.92:46550) 2023-10-06T10:44:41.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.814+0000 7f04a259c700 1 -- 172.21.15.92:0/1915600164 learned_addr learned my addr 172.21.15.92:0/1915600164 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:41.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.814+0000 7f04a2d9d700 1 --2- 172.21.15.92:0/1915600164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04a4153950 0x7f04a415c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:41.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.814+0000 7f04a259c700 1 -- 172.21.15.92:0/1915600164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04a4153950 msgr2=0x7f04a415c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:41.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.814+0000 7f04a259c700 1 --2- 172.21.15.92:0/1915600164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04a4153950 0x7f04a415c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:41.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.814+0000 7f04a259c700 1 -- 172.21.15.92:0/1915600164 >> v1:172.21.15.124:6789/0 conn(0x7f04a4157ce0 legacy=0x7f04a415ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:41.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.814+0000 7f04a259c700 1 -- 172.21.15.92:0/1915600164 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a415bad0 con 0x7f04a4153d20 2023-10-06T10:44:41.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.814+0000 7f04a259c700 1 --2- 172.21.15.92:0/1915600164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04a4153d20 0x7f04a41577a0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f048c0099b0 tx=0x7f048c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3573fcffd1c5a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:41.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.814+0000 7f04a1d9b700 1 -- 172.21.15.92:0/1915600164 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c00b860 con 0x7f04a4153d20 2023-10-06T10:44:41.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.815+0000 7f04a1d9b700 1 -- 172.21.15.92:0/1915600164 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048c00b9c0 con 0x7f04a4153d20 2023-10-06T10:44:41.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.815+0000 7f04a1d9b700 1 -- 172.21.15.92:0/1915600164 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c00bd60 con 0x7f04a4153d20 2023-10-06T10:44:41.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.815+0000 7f04a9081700 1 -- 172.21.15.92:0/1915600164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04a4153d20 msgr2=0x7f04a41577a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:41.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.815+0000 7f04a9081700 1 --2- 172.21.15.92:0/1915600164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04a4153d20 0x7f04a41577a0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f048c0099b0 tx=0x7f048c009250 comp rx=0 tx=0).stop 2023-10-06T10:44:41.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.815+0000 7f04a9081700 1 -- 172.21.15.92:0/1915600164 shutdown_connections 2023-10-06T10:44:41.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.815+0000 7f04a9081700 1 --2- 172.21.15.92:0/1915600164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04a4153950 0x7f04a415c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:41.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.815+0000 7f04a9081700 1 --2- 172.21.15.92:0/1915600164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04a4153d20 0x7f04a41577a0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:41.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.815+0000 7f04a9081700 1 -- 172.21.15.92:0/1915600164 >> 172.21.15.92:0/1915600164 conn(0x7f04a406ece0 msgr2=0x7f04a406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:41.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.815+0000 7f04a9081700 1 -- 172.21.15.92:0/1915600164 shutdown_connections 2023-10-06T10:44:41.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.815+0000 7f04a9081700 1 -- 172.21.15.92:0/1915600164 wait complete. 2023-10-06T10:44:41.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.816+0000 7f04a9081700 1 Processor -- start 2023-10-06T10:44:41.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.816+0000 7f04a9081700 1 -- start start 2023-10-06T10:44:41.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.817+0000 7f04a9081700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04a4153950 0x7f04a416a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:41.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.817+0000 7f04a9081700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04a4153d20 0x7f04a40b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:41.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.817+0000 7f04a9081700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04a4157ce0 0x7f04a41551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:41.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.817+0000 7f04a9081700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a4156160 con 0x7f04a4157ce0 2023-10-06T10:44:41.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.817+0000 7f04a9081700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a40a71f0 con 0x7f04a4153d20 2023-10-06T10:44:41.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.817+0000 7f04a9081700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a4155ba0 con 0x7f04a4153950 2023-10-06T10:44:41.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.817+0000 7f04a359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04a4157ce0 0x7f04a41551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:41.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.817+0000 7f04a359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04a4157ce0 0x7f04a41551c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46566/0 (socket says 172.21.15.92:46566) 2023-10-06T10:44:41.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.817+0000 7f04a359e700 1 -- 172.21.15.92:0/1727478694 learned_addr learned my addr 172.21.15.92:0/1727478694 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:41.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.817+0000 7f04a259c700 1 --2- 172.21.15.92:0/1727478694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04a4153d20 0x7f04a40b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:41.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.817+0000 7f04a2d9d700 1 --2- 172.21.15.92:0/1727478694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04a4153950 0x7f04a416a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:41.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.818+0000 7f04a359e700 1 -- 172.21.15.92:0/1727478694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04a4153950 msgr2=0x7f04a416a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:41.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.818+0000 7f04a359e700 1 --2- 172.21.15.92:0/1727478694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04a4153950 0x7f04a416a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:41.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.818+0000 7f04a359e700 1 -- 172.21.15.92:0/1727478694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04a4153d20 msgr2=0x7f04a40b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:41.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.818+0000 7f04a359e700 1 --2- 172.21.15.92:0/1727478694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04a4153d20 0x7f04a40b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:41.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.818+0000 7f04a359e700 1 -- 172.21.15.92:0/1727478694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c0094f0 con 0x7f04a4157ce0 2023-10-06T10:44:41.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.818+0000 7f04a2d9d700 1 --2- 172.21.15.92:0/1727478694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04a4153950 0x7f04a416a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:41.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.818+0000 7f04a359e700 1 --2- 172.21.15.92:0/1727478694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04a4157ce0 0x7f04a41551c0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f049400c8e0 tx=0x7f0494001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:41.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.818+0000 7f049affd700 1 -- 172.21.15.92:0/1727478694 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049400ce80 con 0x7f04a4157ce0 2023-10-06T10:44:41.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.818+0000 7f049affd700 1 -- 172.21.15.92:0/1727478694 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049400a7e0 con 0x7f04a4157ce0 2023-10-06T10:44:41.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.818+0000 7f04a9081700 1 -- 172.21.15.92:0/1727478694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a4155e80 con 0x7f04a4157ce0 2023-10-06T10:44:41.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.818+0000 7f049affd700 1 -- 172.21.15.92:0/1727478694 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049400aba0 con 0x7f04a4157ce0 2023-10-06T10:44:41.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.818+0000 7f04a9081700 1 -- 172.21.15.92:0/1727478694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a416a8c0 con 0x7f04a4157ce0 2023-10-06T10:44:41.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.819+0000 7f04a9081700 1 -- 172.21.15.92:0/1727478694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a40a7380 con 0x7f04a4157ce0 2023-10-06T10:44:41.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.819+0000 7f049affd700 1 -- 172.21.15.92:0/1727478694 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0494003030 con 0x7f04a4157ce0 2023-10-06T10:44:41.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.820+0000 7f049affd700 1 --2- 172.21.15.92:0/1727478694 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0484066f30 0x7f04840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:41.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.820+0000 7f04a2d9d700 1 --2- 172.21.15.92:0/1727478694 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0484066f30 0x7f04840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:41.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.820+0000 7f049affd700 1 -- 172.21.15.92:0/1727478694 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f04940846a0 con 0x7f04a4157ce0 2023-10-06T10:44:41.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.820+0000 7f04a2d9d700 1 --2- 172.21.15.92:0/1727478694 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0484066f30 0x7f04840693e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f049c006aa0 tx=0x7f049c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:41.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:41.824+0000 7f049affd700 1 -- 172.21.15.92:0/1727478694 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0494051720 con 0x7f04a4157ce0 2023-10-06T10:44:42.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.005+0000 7f04a9081700 1 -- 172.21.15.92:0/1727478694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f04a4064a90 con 0x7f04a4157ce0 2023-10-06T10:44:42.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.676+0000 7f049affd700 1 -- 172.21.15.92:0/1727478694 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v93) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0494054d40 con 0x7f04a4157ce0 2023-10-06T10:44:42.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.679+0000 7f04a9081700 1 -- 172.21.15.92:0/1727478694 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0484066f30 msgr2=0x7f04840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:42.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.679+0000 7f04a9081700 1 --2- 172.21.15.92:0/1727478694 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0484066f30 0x7f04840693e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f049c006aa0 tx=0x7f049c008040 comp rx=0 tx=0).stop 2023-10-06T10:44:42.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.679+0000 7f04a9081700 1 -- 172.21.15.92:0/1727478694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04a4157ce0 msgr2=0x7f04a41551c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:42.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.679+0000 7f04a9081700 1 --2- 172.21.15.92:0/1727478694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04a4157ce0 0x7f04a41551c0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f049400c8e0 tx=0x7f0494001710 comp rx=0 tx=0).stop 2023-10-06T10:44:42.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.679+0000 7f04a9081700 1 -- 172.21.15.92:0/1727478694 shutdown_connections 2023-10-06T10:44:42.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.679+0000 7f04a9081700 1 --2- 172.21.15.92:0/1727478694 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0484066f30 0x7f04840693e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:42.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.679+0000 7f04a9081700 1 --2- 172.21.15.92:0/1727478694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04a4157ce0 0x7f04a41551c0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:42.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.679+0000 7f04a9081700 1 --2- 172.21.15.92:0/1727478694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04a4153950 0x7f04a416a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:42.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.679+0000 7f04a9081700 1 --2- 172.21.15.92:0/1727478694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04a4153d20 0x7f04a40b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:42.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.679+0000 7f04a9081700 1 -- 172.21.15.92:0/1727478694 >> 172.21.15.92:0/1727478694 conn(0x7f04a406ece0 msgr2=0x7f04a40b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:42.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.679+0000 7f04a9081700 1 -- 172.21.15.92:0/1727478694 shutdown_connections 2023-10-06T10:44:42.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.679+0000 7f04a9081700 1 -- 172.21.15.92:0/1727478694 wait complete. 2023-10-06T10:44:42.682 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-10-06T10:44:42.694 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T10:44:42.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.813+0000 7f4bdc39d700 1 Processor -- start 2023-10-06T10:44:42.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.813+0000 7f4bdc39d700 1 -- start start 2023-10-06T10:44:42.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.814+0000 7f4bdc39d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4bd4150940 0x7f4bd4159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:42.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.814+0000 7f4bdc39d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd40cd4d0 0x7f4bd40b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:42.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.814+0000 7f4bdc39d700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd415ac30 con 0x7f4bd415a160 2023-10-06T10:44:42.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.814+0000 7f4bdc39d700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd40cb6a0 con 0x7f4bd4150940 2023-10-06T10:44:42.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.814+0000 7f4bdc39d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd40cb090 con 0x7f4bd40cd4d0 2023-10-06T10:44:42.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.814+0000 7f4bda139700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4bd4150940 0x7f4bd4159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:42.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.814+0000 7f4bda93a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd40cd4d0 0x7f4bd40b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:42.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.815+0000 7f4bda139700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4bd4150940 0x7f4bd4159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:39602/0 (socket says 172.21.15.92:39602) 2023-10-06T10:44:42.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.815+0000 7f4bda139700 1 -- 172.21.15.92:0/1362124125 learned_addr learned my addr 172.21.15.92:0/1362124125 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:42.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.815+0000 7f4bda93a700 1 -- 172.21.15.92:0/1362124125 >> v2:172.21.15.124:3300/0 conn(0x7f4bd4150940 msgr2=0x7f4bd4159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:42.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.815+0000 7f4bda93a700 1 --2- 172.21.15.92:0/1362124125 >> v2:172.21.15.124:3300/0 conn(0x7f4bd4150940 0x7f4bd4159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:42.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.815+0000 7f4bda93a700 1 -- 172.21.15.92:0/1362124125 >> v1:172.21.15.124:6789/0 conn(0x7f4bd415a160 legacy=0x7f4bd40ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:42.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.815+0000 7f4bda93a700 1 -- 172.21.15.92:0/1362124125 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd415a670 con 0x7f4bd40cd4d0 2023-10-06T10:44:42.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.815+0000 7f4bd9137700 1 -- 172.21.15.92:0/1362124125 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115952792 0 0) 0x7f4bc4001110 con 0x7f4bd415a160 2023-10-06T10:44:42.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.815+0000 7f4bda139700 1 --2- 172.21.15.92:0/1362124125 >> v2:172.21.15.124:3300/0 conn(0x7f4bd4150940 0x7f4bd4159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:44:42.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.815+0000 7f4bda93a700 1 --2- 172.21.15.92:0/1362124125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd40cd4d0 0x7f4bd40b2050 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0098e0 tx=0x7f4bcc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b876f3b9b11d6c2c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:42.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.815+0000 7f4bd9137700 1 -- 172.21.15.92:0/1362124125 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc00b640 con 0x7f4bd40cd4d0 2023-10-06T10:44:42.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.815+0000 7f4bd9137700 1 -- 172.21.15.92:0/1362124125 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bcc00b7a0 con 0x7f4bd40cd4d0 2023-10-06T10:44:42.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.815+0000 7f4bd9137700 1 -- 172.21.15.92:0/1362124125 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc00bb40 con 0x7f4bd40cd4d0 2023-10-06T10:44:42.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.816+0000 7f4bdc39d700 1 -- 172.21.15.92:0/1362124125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd40cd4d0 msgr2=0x7f4bd40b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:42.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.816+0000 7f4bdc39d700 1 --2- 172.21.15.92:0/1362124125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd40cd4d0 0x7f4bd40b2050 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0098e0 tx=0x7f4bcc0091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:42.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.816+0000 7f4bdc39d700 1 -- 172.21.15.92:0/1362124125 shutdown_connections 2023-10-06T10:44:42.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.816+0000 7f4bdc39d700 1 --2- 172.21.15.92:0/1362124125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd40cd4d0 0x7f4bd40b2050 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:42.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.816+0000 7f4bdc39d700 1 --2- 172.21.15.92:0/1362124125 >> v2:172.21.15.124:3300/0 conn(0x7f4bd4150940 0x7f4bd4159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:42.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.816+0000 7f4bdc39d700 1 -- 172.21.15.92:0/1362124125 >> 172.21.15.92:0/1362124125 conn(0x7f4bd4099700 msgr2=0x7f4bd406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:42.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.816+0000 7f4bdc39d700 1 -- 172.21.15.92:0/1362124125 shutdown_connections 2023-10-06T10:44:42.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.816+0000 7f4bdc39d700 1 -- 172.21.15.92:0/1362124125 wait complete. 2023-10-06T10:44:42.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.817+0000 7f4bdc39d700 1 Processor -- start 2023-10-06T10:44:42.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.817+0000 7f4bdc39d700 1 -- start start 2023-10-06T10:44:42.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.818+0000 7f4bdc39d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bd40cd4d0 0x7f4bd4150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.818+0000 7f4bdc39d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd415a160 0x7f4bd414fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.818+0000 7f4bdc39d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bd4153b00 0x7f4bd4155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.818+0000 7f4bdc39d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd416aa80 con 0x7f4bd40cd4d0 2023-10-06T10:44:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.818+0000 7f4bdc39d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd4150010 con 0x7f4bd4153b00 2023-10-06T10:44:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.818+0000 7f4bdc39d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd4152d70 con 0x7f4bd415a160 2023-10-06T10:44:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.818+0000 7f4bda139700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bd40cd4d0 0x7f4bd4150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.818+0000 7f4bda139700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bd40cd4d0 0x7f4bd4150d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46576/0 (socket says 172.21.15.92:46576) 2023-10-06T10:44:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.818+0000 7f4bda139700 1 -- 172.21.15.92:0/2208889166 learned_addr learned my addr 172.21.15.92:0/2208889166 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.818+0000 7f4bda93a700 1 --2- 172.21.15.92:0/2208889166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bd4153b00 0x7f4bd4155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.818+0000 7f4bd9938700 1 --2- 172.21.15.92:0/2208889166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd415a160 0x7f4bd414fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.818+0000 7f4bda139700 1 -- 172.21.15.92:0/2208889166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd415a160 msgr2=0x7f4bd414fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:42.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.818+0000 7f4bda139700 1 --2- 172.21.15.92:0/2208889166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd415a160 0x7f4bd414fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:42.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.818+0000 7f4bda139700 1 -- 172.21.15.92:0/2208889166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bd4153b00 msgr2=0x7f4bd4155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:42.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.818+0000 7f4bda139700 1 --2- 172.21.15.92:0/2208889166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bd4153b00 0x7f4bd4155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:42.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.818+0000 7f4bda139700 1 -- 172.21.15.92:0/2208889166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd0009ae0 con 0x7f4bd40cd4d0 2023-10-06T10:44:42.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.819+0000 7f4bda139700 1 --2- 172.21.15.92:0/2208889166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bd40cd4d0 0x7f4bd4150d20 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000c170 tx=0x7f4bd000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:42.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.819+0000 7f4bca7fc700 1 -- 172.21.15.92:0/2208889166 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd000caa0 con 0x7f4bd40cd4d0 2023-10-06T10:44:42.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.819+0000 7f4bca7fc700 1 -- 172.21.15.92:0/2208889166 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd000f040 con 0x7f4bd40cd4d0 2023-10-06T10:44:42.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.819+0000 7f4bdc39d700 1 -- 172.21.15.92:0/2208889166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bcc009420 con 0x7f4bd40cd4d0 2023-10-06T10:44:42.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.819+0000 7f4bca7fc700 1 -- 172.21.15.92:0/2208889166 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd000ce30 con 0x7f4bd40cd4d0 2023-10-06T10:44:42.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.819+0000 7f4bdc39d700 1 -- 172.21.15.92:0/2208889166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd416a1e0 con 0x7f4bd40cd4d0 2023-10-06T10:44:42.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.820+0000 7f4bdc39d700 1 -- 172.21.15.92:0/2208889166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd4064a90 con 0x7f4bd40cd4d0 2023-10-06T10:44:42.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.820+0000 7f4bca7fc700 1 -- 172.21.15.92:0/2208889166 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4bd0014690 con 0x7f4bd40cd4d0 2023-10-06T10:44:42.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.820+0000 7f4bca7fc700 1 --2- 172.21.15.92:0/2208889166 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4bb8066f30 0x7f4bb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:42.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.821+0000 7f4bca7fc700 1 -- 172.21.15.92:0/2208889166 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f4bd0084e70 con 0x7f4bd40cd4d0 2023-10-06T10:44:42.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.821+0000 7f4bd9938700 1 --2- 172.21.15.92:0/2208889166 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4bb8066f30 0x7f4bb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:42.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.821+0000 7f4bd9938700 1 --2- 172.21.15.92:0/2208889166 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4bb8066f30 0x7f4bb80693e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40ca0d0 tx=0x7f4bc4007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:42.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:42.825+0000 7f4bca7fc700 1 -- 172.21.15.92:0/2208889166 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4bd0051ff0 con 0x7f4bd40cd4d0 2023-10-06T10:44:43.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.008+0000 7f4bdc39d700 1 -- 172.21.15.92:0/2208889166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4bd404bb40 con 0x7f4bd40cd4d0 2023-10-06T10:44:43.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.011+0000 7f4bca7fc700 1 -- 172.21.15.92:0/2208889166 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4bd0055610 con 0x7f4bd40cd4d0 2023-10-06T10:44:43.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.014+0000 7f4bdc39d700 1 -- 172.21.15.92:0/2208889166 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4bb8066f30 msgr2=0x7f4bb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:43.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.014+0000 7f4bdc39d700 1 --2- 172.21.15.92:0/2208889166 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4bb8066f30 0x7f4bb80693e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40ca0d0 tx=0x7f4bc4007ec0 comp rx=0 tx=0).stop 2023-10-06T10:44:43.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.014+0000 7f4bdc39d700 1 -- 172.21.15.92:0/2208889166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bd40cd4d0 msgr2=0x7f4bd4150d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:43.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.014+0000 7f4bdc39d700 1 --2- 172.21.15.92:0/2208889166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bd40cd4d0 0x7f4bd4150d20 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000c170 tx=0x7f4bd000b040 comp rx=0 tx=0).stop 2023-10-06T10:44:43.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.014+0000 7f4bdc39d700 1 -- 172.21.15.92:0/2208889166 shutdown_connections 2023-10-06T10:44:43.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.014+0000 7f4bdc39d700 1 --2- 172.21.15.92:0/2208889166 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4bb8066f30 0x7f4bb80693e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:43.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.014+0000 7f4bdc39d700 1 --2- 172.21.15.92:0/2208889166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bd4153b00 0x7f4bd4155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:43.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.014+0000 7f4bdc39d700 1 --2- 172.21.15.92:0/2208889166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bd40cd4d0 0x7f4bd4150d20 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:43.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.014+0000 7f4bdc39d700 1 --2- 172.21.15.92:0/2208889166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd415a160 0x7f4bd414fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:43.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.014+0000 7f4bdc39d700 1 -- 172.21.15.92:0/2208889166 >> 172.21.15.92:0/2208889166 conn(0x7f4bd4099700 msgr2=0x7f4bd40ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:43.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.014+0000 7f4bdc39d700 1 -- 172.21.15.92:0/2208889166 shutdown_connections 2023-10-06T10:44:43.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.014+0000 7f4bdc39d700 1 -- 172.21.15.92:0/2208889166 wait complete. 2023-10-06T10:44:43.028 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0' 2023-10-06T10:44:43.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.141+0000 7f7402599700 1 Processor -- start 2023-10-06T10:44:43.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.141+0000 7f7402599700 1 -- start start 2023-10-06T10:44:43.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.142+0000 7f7402599700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f73fc159560 0x7f73fc0aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:43.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.142+0000 7f7402599700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f73fc0b49a0 0x7f73fc0aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:43.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.142+0000 7f7402599700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73fc0b0760 0x7f73fc0b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:43.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.142+0000 7f7402599700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73fc0b0c60 con 0x7f73fc159560 2023-10-06T10:44:43.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.142+0000 7f7402599700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73fc0b0f40 con 0x7f73fc0b49a0 2023-10-06T10:44:43.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.142+0000 7f7402599700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc0b1220 con 0x7f73fc0b0760 2023-10-06T10:44:43.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.142+0000 7f73fbfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f73fc159560 0x7f73fc0aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:43.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.142+0000 7f73fbfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f73fc159560 0x7f73fc0aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46582/0 (socket says 172.21.15.92:46582) 2023-10-06T10:44:43.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.142+0000 7f73fbfff700 1 -- 172.21.15.92:0/1718466280 learned_addr learned my addr 172.21.15.92:0/1718466280 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:43.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.142+0000 7f73fb7fe700 1 --2- 172.21.15.92:0/1718466280 >> v2:172.21.15.124:3300/0 conn(0x7f73fc0b49a0 0x7f73fc0aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:43.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.142+0000 7f73fbfff700 1 -- 172.21.15.92:0/1718466280 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73fc0b0760 msgr2=0x7f73fc0b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:43.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.142+0000 7f7400b36700 1 --2- 172.21.15.92:0/1718466280 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73fc0b0760 0x7f73fc0b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:43.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.142+0000 7f73fbfff700 1 --2- 172.21.15.92:0/1718466280 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73fc0b0760 0x7f73fc0b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:43.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.142+0000 7f73fbfff700 1 -- 172.21.15.92:0/1718466280 >> v2:172.21.15.124:3300/0 conn(0x7f73fc0b49a0 msgr2=0x7f73fc0aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:43.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.142+0000 7f73fbfff700 1 --2- 172.21.15.92:0/1718466280 >> v2:172.21.15.124:3300/0 conn(0x7f73fc0b49a0 0x7f73fc0aaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:43.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.142+0000 7f73fbfff700 1 -- 172.21.15.92:0/1718466280 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc0b1500 con 0x7f73fc159560 2023-10-06T10:44:43.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.143+0000 7f73fbfff700 1 --2- 172.21.15.92:0/1718466280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73fc159560 0x7f73fc0aa530 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009990 tx=0x7f73ec009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf826a56585b2fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:43.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.143+0000 7f73faffd700 1 -- 172.21.15.92:0/1718466280 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec005c10 con 0x7f73fc159560 2023-10-06T10:44:43.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.143+0000 7f73faffd700 1 -- 172.21.15.92:0/1718466280 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ec005d70 con 0x7f73fc159560 2023-10-06T10:44:43.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.143+0000 7f73faffd700 1 -- 172.21.15.92:0/1718466280 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec00b840 con 0x7f73fc159560 2023-10-06T10:44:43.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.143+0000 7f7402599700 1 -- 172.21.15.92:0/1718466280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73fc159560 msgr2=0x7f73fc0aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:43.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.143+0000 7f7402599700 1 --2- 172.21.15.92:0/1718466280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73fc159560 0x7f73fc0aa530 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009990 tx=0x7f73ec009220 comp rx=0 tx=0).stop 2023-10-06T10:44:43.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.143+0000 7f7402599700 1 -- 172.21.15.92:0/1718466280 shutdown_connections 2023-10-06T10:44:43.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.143+0000 7f7402599700 1 --2- 172.21.15.92:0/1718466280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73fc159560 0x7f73fc0aa530 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:43.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.143+0000 7f7402599700 1 --2- 172.21.15.92:0/1718466280 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73fc0b0760 0x7f73fc0b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:43.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.143+0000 7f7402599700 1 --2- 172.21.15.92:0/1718466280 >> v2:172.21.15.124:3300/0 conn(0x7f73fc0b49a0 0x7f73fc0aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:43.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.143+0000 7f7402599700 1 -- 172.21.15.92:0/1718466280 >> 172.21.15.92:0/1718466280 conn(0x7f73fc0997f0 msgr2=0x7f73fc06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:43.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.143+0000 7f7402599700 1 -- 172.21.15.92:0/1718466280 shutdown_connections 2023-10-06T10:44:43.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.143+0000 7f7402599700 1 -- 172.21.15.92:0/1718466280 wait complete. 2023-10-06T10:44:43.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.144+0000 7f7402599700 1 Processor -- start 2023-10-06T10:44:43.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.145+0000 7f7402599700 1 -- start start 2023-10-06T10:44:43.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.145+0000 7f7402599700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73fc0b0760 0x7f73fc150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:43.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.145+0000 7f7402599700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73fc0b49a0 0x7f73fc14c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:43.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.145+0000 7f7402599700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73fc159560 0x7f73fc15a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:43.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.145+0000 7f7402599700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc15ad80 con 0x7f73fc0b49a0 2023-10-06T10:44:43.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.145+0000 7f7402599700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc153d70 con 0x7f73fc159560 2023-10-06T10:44:43.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.145+0000 7f7402599700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc1548b0 con 0x7f73fc0b0760 2023-10-06T10:44:43.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.145+0000 7f73fb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73fc0b49a0 0x7f73fc14c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:43.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.145+0000 7f73fb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73fc0b49a0 0x7f73fc14c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46584/0 (socket says 172.21.15.92:46584) 2023-10-06T10:44:43.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.145+0000 7f73fb7fe700 1 -- 172.21.15.92:0/1818103365 learned_addr learned my addr 172.21.15.92:0/1818103365 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:43.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.146+0000 7f73fbfff700 1 --2- 172.21.15.92:0/1818103365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73fc0b0760 0x7f73fc150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.146+0000 7f7400b36700 1 --2- 172.21.15.92:0/1818103365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73fc159560 0x7f73fc15a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.146+0000 7f73fb7fe700 1 -- 172.21.15.92:0/1818103365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73fc0b0760 msgr2=0x7f73fc150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.146+0000 7f73fb7fe700 1 --2- 172.21.15.92:0/1818103365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73fc0b0760 0x7f73fc150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.146+0000 7f73fb7fe700 1 -- 172.21.15.92:0/1818103365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73fc159560 msgr2=0x7f73fc15a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.146+0000 7f73fb7fe700 1 --2- 172.21.15.92:0/1818103365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73fc159560 0x7f73fc15a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.146+0000 7f73fb7fe700 1 -- 172.21.15.92:0/1818103365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ec009480 con 0x7f73fc0b49a0 2023-10-06T10:44:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.146+0000 7f73fbfff700 1 --2- 172.21.15.92:0/1818103365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73fc0b0760 0x7f73fc150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.146+0000 7f73fb7fe700 1 --2- 172.21.15.92:0/1818103365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73fc0b49a0 0x7f73fc14c880 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f73f000c880 tx=0x7f73f0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.146+0000 7f73e3fff700 1 -- 172.21.15.92:0/1818103365 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0011600 con 0x7f73fc0b49a0 2023-10-06T10:44:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.146+0000 7f73e3fff700 1 -- 172.21.15.92:0/1818103365 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f0011760 con 0x7f73fc0b49a0 2023-10-06T10:44:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.146+0000 7f7402599700 1 -- 172.21.15.92:0/1818103365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73fc153f80 con 0x7f73fc0b49a0 2023-10-06T10:44:43.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.146+0000 7f73e3fff700 1 -- 172.21.15.92:0/1818103365 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0011b20 con 0x7f73fc0b49a0 2023-10-06T10:44:43.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.146+0000 7f7402599700 1 -- 172.21.15.92:0/1818103365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73fc154540 con 0x7f73fc0b49a0 2023-10-06T10:44:43.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.147+0000 7f7402599700 1 -- 172.21.15.92:0/1818103365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73fc0cd400 con 0x7f73fc0b49a0 2023-10-06T10:44:43.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.148+0000 7f73e3fff700 1 -- 172.21.15.92:0/1818103365 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f73f00193f0 con 0x7f73fc0b49a0 2023-10-06T10:44:43.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.149+0000 7f73e3fff700 1 --2- 172.21.15.92:0/1818103365 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f73e8066f30 0x7f73e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:43.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.149+0000 7f73e3fff700 1 -- 172.21.15.92:0/1818103365 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f73f0084ec0 con 0x7f73fc0b49a0 2023-10-06T10:44:43.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.149+0000 7f73fbfff700 1 --2- 172.21.15.92:0/1818103365 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f73e8066f30 0x7f73e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:43.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.149+0000 7f73fbfff700 1 --2- 172.21.15.92:0/1818103365 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f73e8066f30 0x7f73e80693e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009260 tx=0x7f73ec016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:43.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.152+0000 7f73e3fff700 1 -- 172.21.15.92:0/1818103365 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73f00515a0 con 0x7f73fc0b49a0 2023-10-06T10:44:43.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.416+0000 7f7402599700 1 -- 172.21.15.92:0/1818103365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]} v 0) v1 -- 0x7f73fc064a90 con 0x7f73fc0b49a0 2023-10-06T10:44:43.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.680+0000 7f73e3fff700 1 -- 172.21.15.92:0/1818103365 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]}]=0 v330) v1 ==== 154+0+0 (secure 0 0 0) 0x7f73f0055bf0 con 0x7f73fc0b49a0 2023-10-06T10:44:43.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.682+0000 7f7402599700 1 -- 172.21.15.92:0/1818103365 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f73e8066f30 msgr2=0x7f73e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:43.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.682+0000 7f7402599700 1 --2- 172.21.15.92:0/1818103365 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f73e8066f30 0x7f73e80693e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009260 tx=0x7f73ec016000 comp rx=0 tx=0).stop 2023-10-06T10:44:43.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.682+0000 7f7402599700 1 -- 172.21.15.92:0/1818103365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73fc0b49a0 msgr2=0x7f73fc14c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:43.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.682+0000 7f7402599700 1 --2- 172.21.15.92:0/1818103365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73fc0b49a0 0x7f73fc14c880 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f73f000c880 tx=0x7f73f0008040 comp rx=0 tx=0).stop 2023-10-06T10:44:43.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.682+0000 7f7402599700 1 -- 172.21.15.92:0/1818103365 shutdown_connections 2023-10-06T10:44:43.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.682+0000 7f7402599700 1 --2- 172.21.15.92:0/1818103365 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f73e8066f30 0x7f73e80693e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:43.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.682+0000 7f7402599700 1 --2- 172.21.15.92:0/1818103365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73fc159560 0x7f73fc15a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:43.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.682+0000 7f7402599700 1 --2- 172.21.15.92:0/1818103365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73fc0b0760 0x7f73fc150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:43.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.682+0000 7f7402599700 1 --2- 172.21.15.92:0/1818103365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73fc0b49a0 0x7f73fc14c880 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:43.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.682+0000 7f7402599700 1 -- 172.21.15.92:0/1818103365 >> 172.21.15.92:0/1818103365 conn(0x7f73fc0997f0 msgr2=0x7f73fc0cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:43.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.683+0000 7f7402599700 1 -- 172.21.15.92:0/1818103365 shutdown_connections 2023-10-06T10:44:43.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.683+0000 7f7402599700 1 -- 172.21.15.92:0/1818103365 wait complete. 2023-10-06T10:44:43.696 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2023-10-06T10:44:43.697 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2023-10-06T10:44:43.697 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0' 2023-10-06T10:44:43.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.809+0000 7f8e244ea700 1 Processor -- start 2023-10-06T10:44:43.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.809+0000 7f8e244ea700 1 -- start start 2023-10-06T10:44:43.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.810+0000 7f8e244ea700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8e1c159560 0x7f8e1c0aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:43.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.810+0000 7f8e244ea700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8e1c0b49a0 0x7f8e1c0aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:43.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.810+0000 7f8e244ea700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e1c0b0c60 con 0x7f8e1c0b0760 2023-10-06T10:44:43.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.810+0000 7f8e244ea700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e1c0b0f40 con 0x7f8e1c0b49a0 2023-10-06T10:44:43.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.810+0000 7f8e244ea700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e1c0b1220 con 0x7f8e1c159560 2023-10-06T10:44:43.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.810+0000 7f8e21a85700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8e1c0b49a0 0x7f8e1c0aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:43.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.810+0000 7f8e22a87700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8e1c0b0760 0x7f8e1c0b3890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41648/0 (socket says 172.21.15.92:41648) 2023-10-06T10:44:43.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.810+0000 7f8e22a87700 1 -- 172.21.15.92:0/3839214188 learned_addr learned my addr 172.21.15.92:0/3839214188 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:43.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.810+0000 7f8e22286700 1 --2- 172.21.15.92:0/3839214188 >> v2:172.21.15.124:3300/0 conn(0x7f8e1c159560 0x7f8e1c0aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:43.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.810+0000 7f8e21a85700 1 -- 172.21.15.92:0/3839214188 >> v2:172.21.15.124:3300/0 conn(0x7f8e1c159560 msgr2=0x7f8e1c0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:43.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.810+0000 7f8e21a85700 1 --2- 172.21.15.92:0/3839214188 >> v2:172.21.15.124:3300/0 conn(0x7f8e1c159560 0x7f8e1c0aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:43.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.810+0000 7f8e21a85700 1 -- 172.21.15.92:0/3839214188 >> v1:172.21.15.92:6789/0 conn(0x7f8e1c0b0760 legacy=0x7f8e1c0b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:43.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.810+0000 7f8e21a85700 1 -- 172.21.15.92:0/3839214188 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c0b1500 con 0x7f8e1c0b49a0 2023-10-06T10:44:43.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.811+0000 7f8e21a85700 1 --2- 172.21.15.92:0/3839214188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e1c0b49a0 0x7f8e1c0aaa70 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0098e0 tx=0x7f8e0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed76e365fabeb333 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:43.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.811+0000 7f8e21284700 1 -- 172.21.15.92:0/3839214188 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c009e80 con 0x7f8e1c0b49a0 2023-10-06T10:44:43.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.811+0000 7f8e21284700 1 -- 172.21.15.92:0/3839214188 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c00b820 con 0x7f8e1c0b49a0 2023-10-06T10:44:43.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.811+0000 7f8e21284700 1 -- 172.21.15.92:0/3839214188 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c00bba0 con 0x7f8e1c0b49a0 2023-10-06T10:44:43.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.811+0000 7f8e244ea700 1 -- 172.21.15.92:0/3839214188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e1c0b49a0 msgr2=0x7f8e1c0aaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:43.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.811+0000 7f8e244ea700 1 --2- 172.21.15.92:0/3839214188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e1c0b49a0 0x7f8e1c0aaa70 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0098e0 tx=0x7f8e0c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:43.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.811+0000 7f8e244ea700 1 -- 172.21.15.92:0/3839214188 shutdown_connections 2023-10-06T10:44:43.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.811+0000 7f8e244ea700 1 --2- 172.21.15.92:0/3839214188 >> v2:172.21.15.124:3300/0 conn(0x7f8e1c159560 0x7f8e1c0aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:43.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.811+0000 7f8e244ea700 1 --2- 172.21.15.92:0/3839214188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e1c0b49a0 0x7f8e1c0aaa70 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:43.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.811+0000 7f8e244ea700 1 -- 172.21.15.92:0/3839214188 >> 172.21.15.92:0/3839214188 conn(0x7f8e1c0997f0 msgr2=0x7f8e1c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:43.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.811+0000 7f8e244ea700 1 -- 172.21.15.92:0/3839214188 shutdown_connections 2023-10-06T10:44:43.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.811+0000 7f8e244ea700 1 -- 172.21.15.92:0/3839214188 wait complete. 2023-10-06T10:44:43.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.812+0000 7f8e244ea700 1 Processor -- start 2023-10-06T10:44:43.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.813+0000 7f8e244ea700 1 -- start start 2023-10-06T10:44:43.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.813+0000 7f8e244ea700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e1c0b0760 0x7f8e1c141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:43.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.813+0000 7f8e244ea700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e1c0b49a0 0x7f8e1c153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:43.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.813+0000 7f8e244ea700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e1c159560 0x7f8e1c154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:43.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.813+0000 7f8e244ea700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c140050 con 0x7f8e1c0b49a0 2023-10-06T10:44:43.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.813+0000 7f8e244ea700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c13f440 con 0x7f8e1c0b0760 2023-10-06T10:44:43.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.813+0000 7f8e244ea700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c13f720 con 0x7f8e1c159560 2023-10-06T10:44:43.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.813+0000 7f8e21a85700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e1c0b49a0 0x7f8e1c153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:43.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.813+0000 7f8e21a85700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e1c0b49a0 0x7f8e1c153f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50652/0 (socket says 172.21.15.92:50652) 2023-10-06T10:44:43.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.813+0000 7f8e21a85700 1 -- 172.21.15.92:0/976332601 learned_addr learned my addr 172.21.15.92:0/976332601 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:43.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.814+0000 7f8e22a87700 1 --2- 172.21.15.92:0/976332601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e1c159560 0x7f8e1c154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:43.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.814+0000 7f8e22286700 1 --2- 172.21.15.92:0/976332601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e1c0b0760 0x7f8e1c141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:43.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.814+0000 7f8e21a85700 1 -- 172.21.15.92:0/976332601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e1c159560 msgr2=0x7f8e1c154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:43.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.814+0000 7f8e21a85700 1 --2- 172.21.15.92:0/976332601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e1c159560 0x7f8e1c154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:43.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.814+0000 7f8e21a85700 1 -- 172.21.15.92:0/976332601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e1c0b0760 msgr2=0x7f8e1c141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:43.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.814+0000 7f8e21a85700 1 --2- 172.21.15.92:0/976332601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e1c0b0760 0x7f8e1c141da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:43.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.814+0000 7f8e21a85700 1 -- 172.21.15.92:0/976332601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0c009420 con 0x7f8e1c0b49a0 2023-10-06T10:44:43.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.814+0000 7f8e21a85700 1 --2- 172.21.15.92:0/976332601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e1c0b49a0 0x7f8e1c153f00 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c013000 tx=0x7f8e0c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:43.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.814+0000 7f8e127fc700 1 -- 172.21.15.92:0/976332601 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c00ba60 con 0x7f8e1c0b49a0 2023-10-06T10:44:43.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.814+0000 7f8e127fc700 1 -- 172.21.15.92:0/976332601 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c025420 con 0x7f8e1c0b49a0 2023-10-06T10:44:43.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.814+0000 7f8e127fc700 1 -- 172.21.15.92:0/976332601 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0257e0 con 0x7f8e1c0b49a0 2023-10-06T10:44:43.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.814+0000 7f8e244ea700 1 -- 172.21.15.92:0/976332601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e1c13fa00 con 0x7f8e1c0b49a0 2023-10-06T10:44:43.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.814+0000 7f8e244ea700 1 -- 172.21.15.92:0/976332601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1c154980 con 0x7f8e1c0b49a0 2023-10-06T10:44:43.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.815+0000 7f8e244ea700 1 -- 172.21.15.92:0/976332601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e1c1401c0 con 0x7f8e1c0b49a0 2023-10-06T10:44:43.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.815+0000 7f8e127fc700 1 -- 172.21.15.92:0/976332601 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8e0c023070 con 0x7f8e1c0b49a0 2023-10-06T10:44:43.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.816+0000 7f8e127fc700 1 --2- 172.21.15.92:0/976332601 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8e00066f30 0x7f8e000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:43.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.816+0000 7f8e127fc700 1 -- 172.21.15.92:0/976332601 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f8e0c085c70 con 0x7f8e1c0b49a0 2023-10-06T10:44:43.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.816+0000 7f8e22286700 1 --2- 172.21.15.92:0/976332601 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8e00066f30 0x7f8e000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:43.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.816+0000 7f8e22286700 1 --2- 172.21.15.92:0/976332601 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8e00066f30 0x7f8e000693e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0b4840 tx=0x7f8e18008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:43.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:43.820+0000 7f8e127fc700 1 -- 172.21.15.92:0/976332601 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8e0c052f10 con 0x7f8e1c0b49a0 2023-10-06T10:44:44.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.077+0000 7f8e244ea700 1 -- 172.21.15.92:0/976332601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]} v 0) v1 -- 0x7f8e1c064a90 con 0x7f8e1c0b49a0 2023-10-06T10:44:44.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.687+0000 7f8e127fc700 1 -- 172.21.15.92:0/976332601 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]}]=0 v331) v1 ==== 168+0+0 (secure 0 0 0) 0x7f8e0c0564b0 con 0x7f8e1c0b49a0 2023-10-06T10:44:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.689+0000 7f8e244ea700 1 -- 172.21.15.92:0/976332601 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8e00066f30 msgr2=0x7f8e000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.689+0000 7f8e244ea700 1 --2- 172.21.15.92:0/976332601 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8e00066f30 0x7f8e000693e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0b4840 tx=0x7f8e18008040 comp rx=0 tx=0).stop 2023-10-06T10:44:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.689+0000 7f8e244ea700 1 -- 172.21.15.92:0/976332601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e1c0b49a0 msgr2=0x7f8e1c153f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.689+0000 7f8e244ea700 1 --2- 172.21.15.92:0/976332601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e1c0b49a0 0x7f8e1c153f00 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c013000 tx=0x7f8e0c017000 comp rx=0 tx=0).stop 2023-10-06T10:44:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.689+0000 7f8e244ea700 1 -- 172.21.15.92:0/976332601 shutdown_connections 2023-10-06T10:44:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.689+0000 7f8e244ea700 1 --2- 172.21.15.92:0/976332601 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8e00066f30 0x7f8e000693e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.689+0000 7f8e244ea700 1 --2- 172.21.15.92:0/976332601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e1c159560 0x7f8e1c154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.689+0000 7f8e244ea700 1 --2- 172.21.15.92:0/976332601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e1c0b0760 0x7f8e1c141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.689+0000 7f8e244ea700 1 --2- 172.21.15.92:0/976332601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e1c0b49a0 0x7f8e1c153f00 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.689+0000 7f8e244ea700 1 -- 172.21.15.92:0/976332601 >> 172.21.15.92:0/976332601 conn(0x7f8e1c0997f0 msgr2=0x7f8e1c0d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.689+0000 7f8e244ea700 1 -- 172.21.15.92:0/976332601 shutdown_connections 2023-10-06T10:44:44.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.689+0000 7f8e244ea700 1 -- 172.21.15.92:0/976332601 wait complete. 2023-10-06T10:44:44.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T10:44:44.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.819+0000 7f60d582b700 1 Processor -- start 2023-10-06T10:44:44.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.819+0000 7f60d582b700 1 -- start start 2023-10-06T10:44:44.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.820+0000 7f60d582b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f60d013fe00 0x7f60d00c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:44.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.820+0000 7f60d582b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60d006e340 0x7f60d015a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:44.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.820+0000 7f60d582b700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60d006e7e0 con 0x7f60d00c7ac0 2023-10-06T10:44:44.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.820+0000 7f60d582b700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60d015ade0 con 0x7f60d013fe00 2023-10-06T10:44:44.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.820+0000 7f60d582b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d006ea20 con 0x7f60d006e340 2023-10-06T10:44:44.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.821+0000 7f60c67fc700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f60d013fe00 0x7f60d00c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:44.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.821+0000 7f60cf7fe700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60d006e340 0x7f60d015a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:44.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.821+0000 7f60cf7fe700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60d006e340 0x7f60d015a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:47862/0 (socket says 172.21.15.92:47862) 2023-10-06T10:44:44.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.821+0000 7f60c67fc700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f60d013fe00 0x7f60d00c7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:41668/0 (socket says 172.21.15.92:41668) 2023-10-06T10:44:44.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.821+0000 7f60cf7fe700 1 -- 172.21.15.92:0/1237965551 learned_addr learned my addr 172.21.15.92:0/1237965551 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:44.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.821+0000 7f60ceffd700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f60d00c7ac0 0x7f60d013f6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:50464/0 (socket says 172.21.15.92:50464) 2023-10-06T10:44:44.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.821+0000 7f60cf7fe700 1 -- 172.21.15.92:0/1237965551 >> v2:172.21.15.124:3300/0 conn(0x7f60d013fe00 msgr2=0x7f60d00c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:44.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.821+0000 7f60cf7fe700 1 --2- 172.21.15.92:0/1237965551 >> v2:172.21.15.124:3300/0 conn(0x7f60d013fe00 0x7f60d00c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:44.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.821+0000 7f60cf7fe700 1 -- 172.21.15.92:0/1237965551 >> v1:172.21.15.124:6789/0 conn(0x7f60d00c7ac0 legacy=0x7f60d013f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:44.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.821+0000 7f60cf7fe700 1 -- 172.21.15.92:0/1237965551 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d006ed00 con 0x7f60d006e340 2023-10-06T10:44:44.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.821+0000 7f60c67fc700 1 --2- 172.21.15.92:0/1237965551 >> v2:172.21.15.124:3300/0 conn(0x7f60d013fe00 0x7f60d00c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:44:44.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.821+0000 7f60cf7fe700 1 --2- 172.21.15.92:0/1237965551 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60d006e340 0x7f60d015a8a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009f50 tx=0x7f60c8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51bc9b822419d404 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:44.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.821+0000 7f60ce7fc700 1 -- 172.21.15.92:0/1237965551 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c800e070 con 0x7f60d006e340 2023-10-06T10:44:44.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.821+0000 7f60ce7fc700 1 -- 172.21.15.92:0/1237965551 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c800ba50 con 0x7f60d006e340 2023-10-06T10:44:44.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.821+0000 7f60ce7fc700 1 -- 172.21.15.92:0/1237965551 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c80092e0 con 0x7f60d006e340 2023-10-06T10:44:44.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.822+0000 7f60d582b700 1 -- 172.21.15.92:0/1237965551 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60d006e340 msgr2=0x7f60d015a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:44.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.822+0000 7f60d582b700 1 --2- 172.21.15.92:0/1237965551 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60d006e340 0x7f60d015a8a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009f50 tx=0x7f60c8009840 comp rx=0 tx=0).stop 2023-10-06T10:44:44.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.822+0000 7f60d582b700 1 -- 172.21.15.92:0/1237965551 shutdown_connections 2023-10-06T10:44:44.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.822+0000 7f60d582b700 1 --2- 172.21.15.92:0/1237965551 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60d006e340 0x7f60d015a8a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:44.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.822+0000 7f60d582b700 1 --2- 172.21.15.92:0/1237965551 >> v2:172.21.15.124:3300/0 conn(0x7f60d013fe00 0x7f60d00c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:44.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.822+0000 7f60d582b700 1 -- 172.21.15.92:0/1237965551 >> 172.21.15.92:0/1237965551 conn(0x7f60d00cc0a0 msgr2=0x7f60d00cc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:44.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.822+0000 7f60d582b700 1 -- 172.21.15.92:0/1237965551 shutdown_connections 2023-10-06T10:44:44.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.822+0000 7f60d582b700 1 -- 172.21.15.92:0/1237965551 wait complete. 2023-10-06T10:44:44.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.823+0000 7f60d582b700 1 Processor -- start 2023-10-06T10:44:44.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.823+0000 7f60d582b700 1 -- start start 2023-10-06T10:44:44.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.824+0000 7f60d582b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60d006e340 0x7f60d00b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:44.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.824+0000 7f60d582b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60d00c7ac0 0x7f60d013abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:44.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.824+0000 7f60d582b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60d013fe00 0x7f60d013b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:44.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.824+0000 7f60d582b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d00b17b0 con 0x7f60d00c7ac0 2023-10-06T10:44:44.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.824+0000 7f60d582b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d00b0de0 con 0x7f60d013fe00 2023-10-06T10:44:44.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.824+0000 7f60d582b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d00b07d0 con 0x7f60d006e340 2023-10-06T10:44:44.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.824+0000 7f60c67fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60d00c7ac0 0x7f60d013abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:44.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.824+0000 7f60c67fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60d00c7ac0 0x7f60d013abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50660/0 (socket says 172.21.15.92:50660) 2023-10-06T10:44:44.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.824+0000 7f60c67fc700 1 -- 172.21.15.92:0/3737629984 learned_addr learned my addr 172.21.15.92:0/3737629984 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:44.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.824+0000 7f60cf7fe700 1 --2- 172.21.15.92:0/3737629984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60d013fe00 0x7f60d013b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:44.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.824+0000 7f60ceffd700 1 --2- 172.21.15.92:0/3737629984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60d006e340 0x7f60d00b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:44.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.824+0000 7f60c67fc700 1 -- 172.21.15.92:0/3737629984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60d006e340 msgr2=0x7f60d00b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:44.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.824+0000 7f60c67fc700 1 --2- 172.21.15.92:0/3737629984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60d006e340 0x7f60d00b1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:44.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.824+0000 7f60c67fc700 1 -- 172.21.15.92:0/3737629984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60d013fe00 msgr2=0x7f60d013b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:44.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.824+0000 7f60c67fc700 1 --2- 172.21.15.92:0/3737629984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60d013fe00 0x7f60d013b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:44.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.824+0000 7f60c67fc700 1 -- 172.21.15.92:0/3737629984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b8009420 con 0x7f60d00c7ac0 2023-10-06T10:44:44.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.824+0000 7f60c67fc700 1 --2- 172.21.15.92:0/3737629984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60d00c7ac0 0x7f60d013abc0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f60b800ec80 tx=0x7f60b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:44.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.825+0000 7f60c77fe700 1 -- 172.21.15.92:0/3737629984 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b800c9d0 con 0x7f60d00c7ac0 2023-10-06T10:44:44.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.825+0000 7f60d582b700 1 -- 172.21.15.92:0/3737629984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c8009ae0 con 0x7f60d00c7ac0 2023-10-06T10:44:44.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.825+0000 7f60c77fe700 1 -- 172.21.15.92:0/3737629984 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b800cb30 con 0x7f60d00c7ac0 2023-10-06T10:44:44.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.825+0000 7f60c77fe700 1 -- 172.21.15.92:0/3737629984 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b8013470 con 0x7f60d00c7ac0 2023-10-06T10:44:44.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.825+0000 7f60d582b700 1 -- 172.21.15.92:0/3737629984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d0151b70 con 0x7f60d00c7ac0 2023-10-06T10:44:44.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.826+0000 7f60d582b700 1 -- 172.21.15.92:0/3737629984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d0064a90 con 0x7f60d00c7ac0 2023-10-06T10:44:44.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.826+0000 7f60c77fe700 1 -- 172.21.15.92:0/3737629984 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f60b80137e0 con 0x7f60d00c7ac0 2023-10-06T10:44:44.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.826+0000 7f60c77fe700 1 --2- 172.21.15.92:0/3737629984 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f60b0066f30 0x7f60b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:44.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.826+0000 7f60c77fe700 1 -- 172.21.15.92:0/3737629984 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f60b8084900 con 0x7f60d00c7ac0 2023-10-06T10:44:44.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.826+0000 7f60ceffd700 1 --2- 172.21.15.92:0/3737629984 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f60b0066f30 0x7f60b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:44.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.827+0000 7f60ceffd700 1 --2- 172.21.15.92:0/3737629984 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f60b0066f30 0x7f60b00693e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f60c0007fd0 tx=0x7f60c0007d70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:44.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:44.831+0000 7f60c77fe700 1 -- 172.21.15.92:0/3737629984 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f60b8051b20 con 0x7f60d00c7ac0 2023-10-06T10:44:45.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.014+0000 7f60d582b700 1 -- 172.21.15.92:0/3737629984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f60d00b0f20 con 0x7f60d00c7ac0 2023-10-06T10:44:45.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.019+0000 7f60c77fe700 1 -- 172.21.15.92:0/3737629984 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f60b80170c0 con 0x7f60d00c7ac0 2023-10-06T10:44:45.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.021+0000 7f60d582b700 1 -- 172.21.15.92:0/3737629984 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f60b0066f30 msgr2=0x7f60b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:45.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.021+0000 7f60d582b700 1 --2- 172.21.15.92:0/3737629984 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f60b0066f30 0x7f60b00693e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f60c0007fd0 tx=0x7f60c0007d70 comp rx=0 tx=0).stop 2023-10-06T10:44:45.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.021+0000 7f60d582b700 1 -- 172.21.15.92:0/3737629984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60d00c7ac0 msgr2=0x7f60d013abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:45.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.021+0000 7f60d582b700 1 --2- 172.21.15.92:0/3737629984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60d00c7ac0 0x7f60d013abc0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f60b800ec80 tx=0x7f60b80091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:45.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.022+0000 7f60d582b700 1 -- 172.21.15.92:0/3737629984 shutdown_connections 2023-10-06T10:44:45.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.022+0000 7f60d582b700 1 --2- 172.21.15.92:0/3737629984 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f60b0066f30 0x7f60b00693e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:45.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.022+0000 7f60d582b700 1 --2- 172.21.15.92:0/3737629984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60d013fe00 0x7f60d013b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:45.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.022+0000 7f60d582b700 1 --2- 172.21.15.92:0/3737629984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60d006e340 0x7f60d00b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:45.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.022+0000 7f60d582b700 1 --2- 172.21.15.92:0/3737629984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60d00c7ac0 0x7f60d013abc0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:45.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.022+0000 7f60d582b700 1 -- 172.21.15.92:0/3737629984 >> 172.21.15.92:0/3737629984 conn(0x7f60d00cc0a0 msgr2=0x7f60d013e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:45.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.022+0000 7f60d582b700 1 -- 172.21.15.92:0/3737629984 shutdown_connections 2023-10-06T10:44:45.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.022+0000 7f60d582b700 1 -- 172.21.15.92:0/3737629984 wait complete. 2023-10-06T10:44:45.035 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T10:44:45.036 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:44:45.077 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:44:45.118 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T10:44:45.118 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T10:44:45.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:44:45.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.230+0000 7f87481b1700 1 Processor -- start 2023-10-06T10:44:45.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.231+0000 7f87481b1700 1 -- start start 2023-10-06T10:44:45.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.231+0000 7f87481b1700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f87400b49a0 0x7f87400aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:45.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.231+0000 7f87481b1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f87400b0760 0x7f87400b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:45.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.231+0000 7f87481b1700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87400b0c60 con 0x7f8740159560 2023-10-06T10:44:45.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.231+0000 7f87481b1700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87400b0f40 con 0x7f87400b0760 2023-10-06T10:44:45.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.231+0000 7f87481b1700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87400b1220 con 0x7f87400b49a0 2023-10-06T10:44:45.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.231+0000 7f874674e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f87400b0760 0x7f87400b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:45.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.232+0000 7f874674e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f87400b0760 0x7f87400b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50670/0 (socket says 172.21.15.92:50670) 2023-10-06T10:44:45.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.232+0000 7f874674e700 1 -- 172.21.15.92:0/4166091080 learned_addr learned my addr 172.21.15.92:0/4166091080 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:45.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.232+0000 7f874574c700 1 --2- 172.21.15.92:0/4166091080 >> v2:172.21.15.124:3300/0 conn(0x7f87400b49a0 0x7f87400aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:45.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.232+0000 7f874674e700 1 -- 172.21.15.92:0/4166091080 >> v2:172.21.15.124:3300/0 conn(0x7f87400b49a0 msgr2=0x7f87400aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:45.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.232+0000 7f874674e700 1 --2- 172.21.15.92:0/4166091080 >> v2:172.21.15.124:3300/0 conn(0x7f87400b49a0 0x7f87400aac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:45.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.232+0000 7f874674e700 1 -- 172.21.15.92:0/4166091080 >> v1:172.21.15.124:6789/0 conn(0x7f8740159560 legacy=0x7f87400aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:45.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.232+0000 7f874674e700 1 -- 172.21.15.92:0/4166091080 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87400b1500 con 0x7f87400b0760 2023-10-06T10:44:45.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.232+0000 7f874674e700 1 --2- 172.21.15.92:0/4166091080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87400b0760 0x7f87400b1bc0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f87380099b0 tx=0x7f8738009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4f18468649fd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:45.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.232+0000 7f8744f4b700 1 -- 172.21.15.92:0/4166091080 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738009ea0 con 0x7f87400b0760 2023-10-06T10:44:45.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.232+0000 7f8744f4b700 1 -- 172.21.15.92:0/4166091080 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873800b820 con 0x7f87400b0760 2023-10-06T10:44:45.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.232+0000 7f8744f4b700 1 -- 172.21.15.92:0/4166091080 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873800bbc0 con 0x7f87400b0760 2023-10-06T10:44:45.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.233+0000 7f87481b1700 1 -- 172.21.15.92:0/4166091080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87400b0760 msgr2=0x7f87400b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:45.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.233+0000 7f87481b1700 1 --2- 172.21.15.92:0/4166091080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87400b0760 0x7f87400b1bc0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f87380099b0 tx=0x7f8738009250 comp rx=0 tx=0).stop 2023-10-06T10:44:45.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.233+0000 7f87481b1700 1 -- 172.21.15.92:0/4166091080 shutdown_connections 2023-10-06T10:44:45.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.233+0000 7f87481b1700 1 --2- 172.21.15.92:0/4166091080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87400b0760 0x7f87400b1bc0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:45.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.233+0000 7f87481b1700 1 --2- 172.21.15.92:0/4166091080 >> v2:172.21.15.124:3300/0 conn(0x7f87400b49a0 0x7f87400aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:45.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.233+0000 7f87481b1700 1 -- 172.21.15.92:0/4166091080 >> 172.21.15.92:0/4166091080 conn(0x7f87400997f0 msgr2=0x7f874006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:45.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.233+0000 7f87481b1700 1 -- 172.21.15.92:0/4166091080 shutdown_connections 2023-10-06T10:44:45.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.233+0000 7f87481b1700 1 -- 172.21.15.92:0/4166091080 wait complete. 2023-10-06T10:44:45.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.234+0000 7f87481b1700 1 Processor -- start 2023-10-06T10:44:45.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.234+0000 7f87481b1700 1 -- start start 2023-10-06T10:44:45.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.235+0000 7f87481b1700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87400b0760 0x7f87401579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:45.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.235+0000 7f87481b1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87400b49a0 0x7f87400ad620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:45.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.235+0000 7f87481b1700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8740159560 0x7f87400cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:45.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.235+0000 7f87481b1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87400aec60 con 0x7f87400b49a0 2023-10-06T10:44:45.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.235+0000 7f87481b1700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87400af0f0 con 0x7f8740159560 2023-10-06T10:44:45.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.235+0000 7f87481b1700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87400ae050 con 0x7f87400b0760 2023-10-06T10:44:45.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.235+0000 7f874574c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87400b49a0 0x7f87400ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:45.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.235+0000 7f874574c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87400b49a0 0x7f87400ad620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50684/0 (socket says 172.21.15.92:50684) 2023-10-06T10:44:45.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.235+0000 7f874574c700 1 -- 172.21.15.92:0/994658839 learned_addr learned my addr 172.21.15.92:0/994658839 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:45.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.235+0000 7f874674e700 1 --2- 172.21.15.92:0/994658839 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8740159560 0x7f87400cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:45.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.235+0000 7f8745f4d700 1 --2- 172.21.15.92:0/994658839 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87400b0760 0x7f87401579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:45.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.235+0000 7f874574c700 1 -- 172.21.15.92:0/994658839 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87400b0760 msgr2=0x7f87401579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:45.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.235+0000 7f874574c700 1 --2- 172.21.15.92:0/994658839 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87400b0760 0x7f87401579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:45.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.235+0000 7f874574c700 1 -- 172.21.15.92:0/994658839 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8740159560 msgr2=0x7f87400cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:45.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.235+0000 7f874574c700 1 --2- 172.21.15.92:0/994658839 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8740159560 0x7f87400cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:45.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.235+0000 7f874574c700 1 -- 172.21.15.92:0/994658839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87380094f0 con 0x7f87400b49a0 2023-10-06T10:44:45.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.235+0000 7f874574c700 1 --2- 172.21.15.92:0/994658839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87400b49a0 0x7f87400ad620 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f873000ea50 tx=0x7f87300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:45.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.235+0000 7f8735ffb700 1 -- 172.21.15.92:0/994658839 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873000c920 con 0x7f87400b49a0 2023-10-06T10:44:45.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.235+0000 7f8735ffb700 1 -- 172.21.15.92:0/994658839 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873000ca80 con 0x7f87400b49a0 2023-10-06T10:44:45.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.235+0000 7f8735ffb700 1 -- 172.21.15.92:0/994658839 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873000ce20 con 0x7f87400b49a0 2023-10-06T10:44:45.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.235+0000 7f87481b1700 1 -- 172.21.15.92:0/994658839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87400ae330 con 0x7f87400b49a0 2023-10-06T10:44:45.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.236+0000 7f87481b1700 1 -- 172.21.15.92:0/994658839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87400ae8f0 con 0x7f87400b49a0 2023-10-06T10:44:45.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.236+0000 7f87481b1700 1 -- 172.21.15.92:0/994658839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87400aedd0 con 0x7f87400b49a0 2023-10-06T10:44:45.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.237+0000 7f8735ffb700 1 -- 172.21.15.92:0/994658839 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8730013680 con 0x7f87400b49a0 2023-10-06T10:44:45.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.237+0000 7f8735ffb700 1 --2- 172.21.15.92:0/994658839 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8724066f30 0x7f87240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:45.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.237+0000 7f8735ffb700 1 -- 172.21.15.92:0/994658839 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f8730084970 con 0x7f87400b49a0 2023-10-06T10:44:45.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.237+0000 7f8745f4d700 1 --2- 172.21.15.92:0/994658839 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8724066f30 0x7f87240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:45.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.238+0000 7f8745f4d700 1 --2- 172.21.15.92:0/994658839 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8724066f30 0x7f87240693e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f873c00e160 tx=0x7f873c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:45.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.241+0000 7f8735ffb700 1 -- 172.21.15.92:0/994658839 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8730051b90 con 0x7f87400b49a0 2023-10-06T10:44:45.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.461+0000 7f87481b1700 1 -- 172.21.15.92:0/994658839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8740064a90 con 0x7f87400b49a0 2023-10-06T10:44:45.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.462+0000 7f8735ffb700 1 -- 172.21.15.92:0/994658839 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f8730017de0 con 0x7f87400b49a0 2023-10-06T10:44:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:45.464 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":94,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6662,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/582655905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":582655905},{"type":"v1","addr":"172.21.15.92:6837","nonce":582655905}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2122441982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2122441982},{"type":"v1","addr":"172.21.15.124:6835","nonce":2122441982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78},{"gid":7438,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2543558797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2543558797},{"type":"v1","addr":"172.21.15.124:6834","nonce":2543558797}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8457,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/143003109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":143003109},{"type":"v1","addr":"172.21.15.92:6835","nonce":143003109}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[]} 2023-10-06T10:44:45.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.464+0000 7f87481b1700 1 -- 172.21.15.92:0/994658839 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8724066f30 msgr2=0x7f87240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:45.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.464+0000 7f87481b1700 1 --2- 172.21.15.92:0/994658839 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8724066f30 0x7f87240693e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f873c00e160 tx=0x7f873c009040 comp rx=0 tx=0).stop 2023-10-06T10:44:45.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.464+0000 7f87481b1700 1 -- 172.21.15.92:0/994658839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87400b49a0 msgr2=0x7f87400ad620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:45.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.464+0000 7f87481b1700 1 --2- 172.21.15.92:0/994658839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87400b49a0 0x7f87400ad620 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f873000ea50 tx=0x7f87300091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:45.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.464+0000 7f87481b1700 1 -- 172.21.15.92:0/994658839 shutdown_connections 2023-10-06T10:44:45.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.464+0000 7f87481b1700 1 --2- 172.21.15.92:0/994658839 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8724066f30 0x7f87240693e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:45.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.464+0000 7f87481b1700 1 --2- 172.21.15.92:0/994658839 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8740159560 0x7f87400cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:45.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.464+0000 7f87481b1700 1 --2- 172.21.15.92:0/994658839 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87400b0760 0x7f87401579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:45.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.464+0000 7f87481b1700 1 --2- 172.21.15.92:0/994658839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87400b49a0 0x7f87400ad620 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:45.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.464+0000 7f87481b1700 1 -- 172.21.15.92:0/994658839 >> 172.21.15.92:0/994658839 conn(0x7f87400997f0 msgr2=0x7f8740069a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:45.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.464+0000 7f87481b1700 1 -- 172.21.15.92:0/994658839 shutdown_connections 2023-10-06T10:44:45.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.464+0000 7f87481b1700 1 -- 172.21.15.92:0/994658839 wait complete. 2023-10-06T10:44:45.467 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 94 2023-10-06T10:44:45.478 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T10:44:45.478 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T10:44:45.478 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T10:44:45.478 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:44:45.479 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T10:44:45.479 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T10:44:45.479 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T10:44:45.479 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:44:45.479 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T10:44:45.479 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T10:44:45.480 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T10:44:45.480 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:44:45.480 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T10:44:45.480 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T10:44:45.480 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T10:44:45.480 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:44:45.487 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T10:44:45.487 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T10:44:45.488 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T10:44:45.488 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T10:44:45.493 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T10:44:45.494 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T10:44:45.494 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T10:44:45.495 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T10:44:45.529 INFO:tasks.ceph.mds.a:Started 2023-10-06T10:44:45.529 DEBUG:teuthology.parallel:result is None 2023-10-06T10:44:45.529 INFO:tasks.ceph.mds.c:Started 2023-10-06T10:44:45.530 DEBUG:teuthology.parallel:result is None 2023-10-06T10:44:45.536 INFO:tasks.ceph.mds.b:Started 2023-10-06T10:44:45.536 DEBUG:teuthology.parallel:result is None 2023-10-06T10:44:45.537 INFO:tasks.ceph.mds.d:Started 2023-10-06T10:44:45.537 DEBUG:teuthology.parallel:result is None 2023-10-06T10:44:45.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T10:44:45.602 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T10:44:45.602 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T10:44:45.623 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-06T10:44:45.623 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-06T10:44:45.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.657+0000 7f3bd4dcf700 1 Processor -- start 2023-10-06T10:44:45.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.657+0000 7f3bd4dcf700 1 -- start start 2023-10-06T10:44:45.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.658+0000 7f3bd4dcf700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3bd0159440 0x7f3bd00b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:45.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.658+0000 7f3bd4dcf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3bd00b35d0 0x7f3bd00cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:45.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.658+0000 7f3bd4dcf700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bd00cc170 0x7f3bd00b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:45.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.658+0000 7f3bd4dcf700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd00cb7a0 con 0x7f3bd00b35d0 2023-10-06T10:44:45.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.658+0000 7f3bd4dcf700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd00b0860 con 0x7f3bd0159440 2023-10-06T10:44:45.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.658+0000 7f3bd4dcf700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00cb150 con 0x7f3bd00cc170 2023-10-06T10:44:45.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.658+0000 7f3bceffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3bd00b35d0 0x7f3bd00cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:45.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.658+0000 7f3bceffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3bd00b35d0 0x7f3bd00cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50728/0 (socket says 172.21.15.92:50728) 2023-10-06T10:44:45.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.658+0000 7f3bceffd700 1 -- 172.21.15.92:0/1107424357 learned_addr learned my addr 172.21.15.92:0/1107424357 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:45.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.658+0000 7f3bceffd700 1 -- 172.21.15.92:0/1107424357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bd00cc170 msgr2=0x7f3bd00b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:45.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.658+0000 7f3bcffff700 1 --2- 172.21.15.92:0/1107424357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bd00cc170 0x7f3bd00b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:45.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.658+0000 7f3bcf7fe700 1 --2- 172.21.15.92:0/1107424357 >> v2:172.21.15.124:3300/0 conn(0x7f3bd0159440 0x7f3bd00b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:45.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.658+0000 7f3bceffd700 1 --2- 172.21.15.92:0/1107424357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bd00cc170 0x7f3bd00b2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:45.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.658+0000 7f3bceffd700 1 -- 172.21.15.92:0/1107424357 >> v2:172.21.15.124:3300/0 conn(0x7f3bd0159440 msgr2=0x7f3bd00b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:45.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.658+0000 7f3bceffd700 1 --2- 172.21.15.92:0/1107424357 >> v2:172.21.15.124:3300/0 conn(0x7f3bd0159440 0x7f3bd00b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:45.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.658+0000 7f3bceffd700 1 -- 172.21.15.92:0/1107424357 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd00cb430 con 0x7f3bd00b35d0 2023-10-06T10:44:45.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.659+0000 7f3bceffd700 1 --2- 172.21.15.92:0/1107424357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bd00b35d0 0x7f3bd00cbc30 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f3bd006f9f0 tx=0x7f3bc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcc9d57bdaae833 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:45.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.659+0000 7f3bce7fc700 1 -- 172.21.15.92:0/1107424357 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc400b860 con 0x7f3bd00b35d0 2023-10-06T10:44:45.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.659+0000 7f3bce7fc700 1 -- 172.21.15.92:0/1107424357 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc400b9c0 con 0x7f3bd00b35d0 2023-10-06T10:44:45.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.659+0000 7f3bce7fc700 1 -- 172.21.15.92:0/1107424357 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc400bd60 con 0x7f3bd00b35d0 2023-10-06T10:44:45.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.659+0000 7f3bd4dcf700 1 -- 172.21.15.92:0/1107424357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bd00b35d0 msgr2=0x7f3bd00cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:45.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.659+0000 7f3bd4dcf700 1 --2- 172.21.15.92:0/1107424357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bd00b35d0 0x7f3bd00cbc30 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f3bd006f9f0 tx=0x7f3bc4009250 comp rx=0 tx=0).stop 2023-10-06T10:44:45.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.659+0000 7f3bd4dcf700 1 -- 172.21.15.92:0/1107424357 shutdown_connections 2023-10-06T10:44:45.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.659+0000 7f3bd4dcf700 1 --2- 172.21.15.92:0/1107424357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bd00cc170 0x7f3bd00b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:45.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.659+0000 7f3bd4dcf700 1 --2- 172.21.15.92:0/1107424357 >> v2:172.21.15.124:3300/0 conn(0x7f3bd0159440 0x7f3bd00b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:45.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.659+0000 7f3bd4dcf700 1 --2- 172.21.15.92:0/1107424357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bd00b35d0 0x7f3bd00cbc30 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:45.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.659+0000 7f3bd4dcf700 1 -- 172.21.15.92:0/1107424357 >> 172.21.15.92:0/1107424357 conn(0x7f3bd006ece0 msgr2=0x7f3bd006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:45.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.659+0000 7f3bd4dcf700 1 -- 172.21.15.92:0/1107424357 shutdown_connections 2023-10-06T10:44:45.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.659+0000 7f3bd4dcf700 1 -- 172.21.15.92:0/1107424357 wait complete. 2023-10-06T10:44:45.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.661+0000 7f3bd4dcf700 1 Processor -- start 2023-10-06T10:44:45.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.661+0000 7f3bd4dcf700 1 -- start start 2023-10-06T10:44:45.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.661+0000 7f3bd4dcf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bd00b35d0 0x7f3bd00c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:45.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.661+0000 7f3bd4dcf700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3bd00cc170 0x7f3bd00c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:45.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.661+0000 7f3bd4dcf700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bd0159440 0x7f3bd00cf060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:45.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.661+0000 7f3bd4dcf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00c6340 con 0x7f3bd00b35d0 2023-10-06T10:44:45.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.661+0000 7f3bd4dcf700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00cebd0 con 0x7f3bd00cc170 2023-10-06T10:44:45.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.661+0000 7f3bd4dcf700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd0159c90 con 0x7f3bd0159440 2023-10-06T10:44:45.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.661+0000 7f3bcf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bd00b35d0 0x7f3bd00c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:45.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.661+0000 7f3bcf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bd00b35d0 0x7f3bd00c3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50738/0 (socket says 172.21.15.92:50738) 2023-10-06T10:44:45.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.661+0000 7f3bcf7fe700 1 -- 172.21.15.92:0/1693022949 learned_addr learned my addr 172.21.15.92:0/1693022949 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:45.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.662+0000 7f3bcffff700 1 --2- 172.21.15.92:0/1693022949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bd0159440 0x7f3bd00cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:45.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.662+0000 7f3bceffd700 1 --2- 172.21.15.92:0/1693022949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3bd00cc170 0x7f3bd00c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:45.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.662+0000 7f3bcf7fe700 1 -- 172.21.15.92:0/1693022949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bd0159440 msgr2=0x7f3bd00cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:45.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.662+0000 7f3bcf7fe700 1 --2- 172.21.15.92:0/1693022949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bd0159440 0x7f3bd00cf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:45.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.662+0000 7f3bcf7fe700 1 -- 172.21.15.92:0/1693022949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3bd00cc170 msgr2=0x7f3bd00c5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:45.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.662+0000 7f3bcf7fe700 1 --2- 172.21.15.92:0/1693022949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3bd00cc170 0x7f3bd00c5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:45.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.662+0000 7f3bcf7fe700 1 -- 172.21.15.92:0/1693022949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc40094f0 con 0x7f3bd00b35d0 2023-10-06T10:44:45.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.662+0000 7f3bcf7fe700 1 --2- 172.21.15.92:0/1693022949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bd00b35d0 0x7f3bd00c3d20 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000c8b0 tx=0x7f3bc0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:45.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.662+0000 7f3bb77fe700 1 -- 172.21.15.92:0/1693022949 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0011600 con 0x7f3bd00b35d0 2023-10-06T10:44:45.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.662+0000 7f3bb77fe700 1 -- 172.21.15.92:0/1693022949 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc0011760 con 0x7f3bd00b35d0 2023-10-06T10:44:45.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.662+0000 7f3bb77fe700 1 -- 172.21.15.92:0/1693022949 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0011b00 con 0x7f3bd00b35d0 2023-10-06T10:44:45.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.662+0000 7f3bd4dcf700 1 -- 172.21.15.92:0/1693022949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd00ce8a0 con 0x7f3bd00b35d0 2023-10-06T10:44:45.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.662+0000 7f3bd4dcf700 1 -- 172.21.15.92:0/1693022949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd00c6800 con 0x7f3bd00b35d0 2023-10-06T10:44:45.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.663+0000 7f3bd4dcf700 1 -- 172.21.15.92:0/1693022949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd00ceeb0 con 0x7f3bd00b35d0 2023-10-06T10:44:45.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.663+0000 7f3bb77fe700 1 -- 172.21.15.92:0/1693022949 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3bc0018360 con 0x7f3bd00b35d0 2023-10-06T10:44:45.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.664+0000 7f3bb77fe700 1 --2- 172.21.15.92:0/1693022949 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3bbc066f30 0x7f3bbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:45.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.664+0000 7f3bb77fe700 1 -- 172.21.15.92:0/1693022949 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f3bc0084280 con 0x7f3bd00b35d0 2023-10-06T10:44:45.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.664+0000 7f3bceffd700 1 --2- 172.21.15.92:0/1693022949 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3bbc066f30 0x7f3bbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:45.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.664+0000 7f3bceffd700 1 --2- 172.21.15.92:0/1693022949 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3bbc066f30 0x7f3bbc0693e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3bd006f9f0 tx=0x7f3bc4017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:45.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.668+0000 7f3bb77fe700 1 -- 172.21.15.92:0/1693022949 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3bc0054f30 con 0x7f3bd00b35d0 2023-10-06T10:44:45.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:45.869+0000 7f3bd4dcf700 1 -- 172.21.15.92:0/1693022949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3bd00c6ba0 con 0x7f3bd00b35d0 2023-10-06T10:44:46.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.702+0000 7f3bb77fe700 1 -- 172.21.15.92:0/1693022949 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v98) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3bc0054ac0 con 0x7f3bd00b35d0 2023-10-06T10:44:46.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.704+0000 7f3bd4dcf700 1 -- 172.21.15.92:0/1693022949 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3bbc066f30 msgr2=0x7f3bbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:46.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.704+0000 7f3bd4dcf700 1 --2- 172.21.15.92:0/1693022949 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3bbc066f30 0x7f3bbc0693e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3bd006f9f0 tx=0x7f3bc4017000 comp rx=0 tx=0).stop 2023-10-06T10:44:46.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.704+0000 7f3bd4dcf700 1 -- 172.21.15.92:0/1693022949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bd00b35d0 msgr2=0x7f3bd00c3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:46.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.704+0000 7f3bd4dcf700 1 --2- 172.21.15.92:0/1693022949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bd00b35d0 0x7f3bd00c3d20 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000c8b0 tx=0x7f3bc0008040 comp rx=0 tx=0).stop 2023-10-06T10:44:46.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.705+0000 7f3bd4dcf700 1 -- 172.21.15.92:0/1693022949 shutdown_connections 2023-10-06T10:44:46.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.705+0000 7f3bd4dcf700 1 --2- 172.21.15.92:0/1693022949 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3bbc066f30 0x7f3bbc0693e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:46.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.705+0000 7f3bd4dcf700 1 --2- 172.21.15.92:0/1693022949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bd0159440 0x7f3bd00cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:46.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.705+0000 7f3bd4dcf700 1 --2- 172.21.15.92:0/1693022949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bd00b35d0 0x7f3bd00c3d20 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:46.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.705+0000 7f3bd4dcf700 1 --2- 172.21.15.92:0/1693022949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3bd00cc170 0x7f3bd00c5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:46.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.705+0000 7f3bd4dcf700 1 -- 172.21.15.92:0/1693022949 >> 172.21.15.92:0/1693022949 conn(0x7f3bd006ece0 msgr2=0x7f3bd0152630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:46.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.705+0000 7f3bd4dcf700 1 -- 172.21.15.92:0/1693022949 shutdown_connections 2023-10-06T10:44:46.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.705+0000 7f3bd4dcf700 1 -- 172.21.15.92:0/1693022949 wait complete. 2023-10-06T10:44:46.708 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-10-06T10:44:46.720 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T10:44:46.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.834+0000 7feb5bf68700 1 Processor -- start 2023-10-06T10:44:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.834+0000 7feb5bf68700 1 -- start start 2023-10-06T10:44:46.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.835+0000 7feb5bf68700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7feb54159560 0x7feb540aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:46.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.835+0000 7feb5bf68700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb540b0c60 con 0x7feb540b49a0 2023-10-06T10:44:46.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.835+0000 7feb5bf68700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb540b0f40 con 0x7feb540b0760 2023-10-06T10:44:46.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.835+0000 7feb5bf68700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb540b1220 con 0x7feb54159560 2023-10-06T10:44:46.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.835+0000 7feb59d04700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7feb54159560 0x7feb540aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:46.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.835+0000 7feb59503700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7feb540b49a0 0x7feb540aaa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41668/0 (socket says 172.21.15.92:41668) 2023-10-06T10:44:46.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.835+0000 7feb59d04700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7feb54159560 0x7feb540aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50742/0 (socket says 172.21.15.92:50742) 2023-10-06T10:44:46.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.835+0000 7feb59503700 1 -- 172.21.15.92:0/1149123216 learned_addr learned my addr 172.21.15.92:0/1149123216 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:46.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.835+0000 7feb59d04700 1 -- 172.21.15.92:0/1149123216 >> v1:172.21.15.124:6789/0 conn(0x7feb540b0760 legacy=0x7feb540b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:46.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.835+0000 7feb59d04700 1 -- 172.21.15.92:0/1149123216 >> v1:172.21.15.92:6789/0 conn(0x7feb540b49a0 legacy=0x7feb540aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:46.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.835+0000 7feb59d04700 1 -- 172.21.15.92:0/1149123216 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb540b1500 con 0x7feb54159560 2023-10-06T10:44:46.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.835+0000 7feb59d04700 1 --2- 172.21.15.92:0/1149123216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb54159560 0x7feb540aa530 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7feb50009ee0 tx=0x7feb50009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5a52ce087f7342 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:46.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.835+0000 7feb58d02700 1 -- 172.21.15.92:0/1149123216 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5000e070 con 0x7feb54159560 2023-10-06T10:44:46.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.835+0000 7feb58d02700 1 -- 172.21.15.92:0/1149123216 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5000bc10 con 0x7feb54159560 2023-10-06T10:44:46.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.835+0000 7feb58d02700 1 -- 172.21.15.92:0/1149123216 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb50012430 con 0x7feb54159560 2023-10-06T10:44:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.836+0000 7feb5bf68700 1 -- 172.21.15.92:0/1149123216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb54159560 msgr2=0x7feb540aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.836+0000 7feb5bf68700 1 --2- 172.21.15.92:0/1149123216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb54159560 0x7feb540aa530 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7feb50009ee0 tx=0x7feb50009780 comp rx=0 tx=0).stop 2023-10-06T10:44:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.836+0000 7feb5bf68700 1 -- 172.21.15.92:0/1149123216 shutdown_connections 2023-10-06T10:44:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.836+0000 7feb5bf68700 1 --2- 172.21.15.92:0/1149123216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb54159560 0x7feb540aa530 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.836+0000 7feb5bf68700 1 -- 172.21.15.92:0/1149123216 >> 172.21.15.92:0/1149123216 conn(0x7feb540997f0 msgr2=0x7feb5406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.836+0000 7feb5bf68700 1 -- 172.21.15.92:0/1149123216 shutdown_connections 2023-10-06T10:44:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.836+0000 7feb5bf68700 1 -- 172.21.15.92:0/1149123216 wait complete. 2023-10-06T10:44:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.837+0000 7feb5bf68700 1 Processor -- start 2023-10-06T10:44:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.837+0000 7feb5bf68700 1 -- start start 2023-10-06T10:44:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.838+0000 7feb5bf68700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb540b0760 0x7feb54147440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.838+0000 7feb5bf68700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb540b49a0 0x7feb54069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.838+0000 7feb5bf68700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb54159560 0x7feb54069bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.838+0000 7feb5bf68700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb540ac010 con 0x7feb54159560 2023-10-06T10:44:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.838+0000 7feb5bf68700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb54147980 con 0x7feb540b49a0 2023-10-06T10:44:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.838+0000 7feb5bf68700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb540ad760 con 0x7feb540b0760 2023-10-06T10:44:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.838+0000 7feb5a505700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb54159560 0x7feb54069bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.838+0000 7feb5a505700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb54159560 0x7feb54069bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50756/0 (socket says 172.21.15.92:50756) 2023-10-06T10:44:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.838+0000 7feb5a505700 1 -- 172.21.15.92:0/2768102679 learned_addr learned my addr 172.21.15.92:0/2768102679 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.838+0000 7feb59503700 1 --2- 172.21.15.92:0/2768102679 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb540b49a0 0x7feb54069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.838+0000 7feb5a505700 1 -- 172.21.15.92:0/2768102679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb540b0760 msgr2=0x7feb54147440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.838+0000 7feb5a505700 1 --2- 172.21.15.92:0/2768102679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb540b0760 0x7feb54147440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.838+0000 7feb5a505700 1 -- 172.21.15.92:0/2768102679 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb540b49a0 msgr2=0x7feb54069670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.838+0000 7feb5a505700 1 --2- 172.21.15.92:0/2768102679 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb540b49a0 0x7feb54069670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.838+0000 7feb5a505700 1 -- 172.21.15.92:0/2768102679 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb50009a20 con 0x7feb54159560 2023-10-06T10:44:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.838+0000 7feb5a505700 1 --2- 172.21.15.92:0/2768102679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb54159560 0x7feb54069bb0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7feb4c00c560 tx=0x7feb4c007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.838+0000 7feb49ffb700 1 -- 172.21.15.92:0/2768102679 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c001120 con 0x7feb54159560 2023-10-06T10:44:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.839+0000 7feb49ffb700 1 -- 172.21.15.92:0/2768102679 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4c011590 con 0x7feb54159560 2023-10-06T10:44:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.839+0000 7feb5bf68700 1 -- 172.21.15.92:0/2768102679 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb540ada40 con 0x7feb54159560 2023-10-06T10:44:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.839+0000 7feb49ffb700 1 -- 172.21.15.92:0/2768102679 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c011950 con 0x7feb54159560 2023-10-06T10:44:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.839+0000 7feb5bf68700 1 -- 172.21.15.92:0/2768102679 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb540ae000 con 0x7feb54159560 2023-10-06T10:44:46.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.840+0000 7feb5bf68700 1 -- 172.21.15.92:0/2768102679 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb540ae1f0 con 0x7feb54159560 2023-10-06T10:44:46.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.840+0000 7feb49ffb700 1 -- 172.21.15.92:0/2768102679 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feb4c011cc0 con 0x7feb54159560 2023-10-06T10:44:46.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.840+0000 7feb49ffb700 1 --2- 172.21.15.92:0/2768102679 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7feb38066f30 0x7feb380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:46.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.840+0000 7feb49ffb700 1 -- 172.21.15.92:0/2768102679 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5526+0+0 (secure 0 0 0) 0x7feb4c083990 con 0x7feb54159560 2023-10-06T10:44:46.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.840+0000 7feb59d04700 1 --2- 172.21.15.92:0/2768102679 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7feb38066f30 0x7feb380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:46.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.841+0000 7feb59d04700 1 --2- 172.21.15.92:0/2768102679 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7feb38066f30 0x7feb380693e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7feb540d11c0 tx=0x7feb50019000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:46.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:46.845+0000 7feb49ffb700 1 -- 172.21.15.92:0/2768102679 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feb4c050c10 con 0x7feb54159560 2023-10-06T10:44:47.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.106+0000 7feb5bf68700 1 -- 172.21.15.92:0/2768102679 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7feb54064a90 con 0x7feb54159560 2023-10-06T10:44:47.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.107+0000 7feb49ffb700 1 -- 172.21.15.92:0/2768102679 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7feb4c017070 con 0x7feb54159560 2023-10-06T10:44:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:47.109 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:44:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-10-06T10:44:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:44:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-10-06T10:44:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lc28gCBAAVrjhXIawHuZEox0PE3/Rsg==", 2023-10-06T10:44:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:44:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:44:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:44:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:44:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:44:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:44:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:44:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-10-06T10:44:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lQpdOExAAm44s0n4Jekhy8AJk/o6Waw==", 2023-10-06T10:44:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:44:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:44:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:44:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:44:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:44:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:44:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:44:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-10-06T10:44:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lrCrsDhAAvp++wvvnSpZRGzvRTrGWEw==", 2023-10-06T10:44:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:44:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:44:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:44:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:44:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:44:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:44:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:44:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-10-06T10:44:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9l/DgWGhAAy3JMA5ntn/UaRkuTCyPdkg==", 2023-10-06T10:44:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:44:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:44:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:44:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:44:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:44:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:44:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:44:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-10-06T10:44:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBo4x9lDEIFMBAAc3DY1fYpY2UiY7MX7qSP5A==", 2023-10-06T10:44:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:44:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:44:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:44:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:44:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:44:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:44:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-10-06T10:44:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBr4x9lKzX8LRAApryLtFGkK6urkE8W70l5Cg==", 2023-10-06T10:44:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:44:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:44:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:44:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:44:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:44:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:44:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-10-06T10:44:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBu4x9lCMZpLhAAEBPa40Wnb820p6j+lv+PsA==", 2023-10-06T10:44:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:44:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:44:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:44:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:44:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:44:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:44:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-10-06T10:44:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBx4x9l4AHrLRAAFn6SnNkHnOiVXmmiU8qGyA==", 2023-10-06T10:44:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:44:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:44:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:44:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:44:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:44:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:44:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-10-06T10:44:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB34x9l7V7WMBAAvs5kvBP5n0Th478l/idjeQ==", 2023-10-06T10:44:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:44:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:44:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:44:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:44:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:44:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:44:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-10-06T10:44:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB64x9lfHIlLhAADCuqt+O/zPEETgoj//hwVg==", 2023-10-06T10:44:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:44:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:44:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:44:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:44:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:44:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:44:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-10-06T10:44:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB94x9lUmU7LhAAvxmtGmJxrI8TlOOD7taHOQ==", 2023-10-06T10:44:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:44:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:44:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:44:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:44:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:44:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:44:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-10-06T10:44:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQCA4x9lJE3nLBAAnREk9yNK+FhX4fnG+Za5Ug==", 2023-10-06T10:44:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:44:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T10:44:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T10:44:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:44:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:44:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:44:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-10-06T10:44:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lMePcHRAAMyftXWJWk7rwmWFkq18Rqg==", 2023-10-06T10:44:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:44:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:44:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-10-06T10:44:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T10:44:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:44:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:44:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-10-06T10:44:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lp1s0JhAAKvZ23MYjObxOK83q3UqJBQ==", 2023-10-06T10:44:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:44:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:44:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T10:44:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T10:44:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T10:44:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:44:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:44:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-10-06T10:44:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lc6o9IxAAaOY7S5rYn+OdeMNWK1UYMA==", 2023-10-06T10:44:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:44:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:44:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T10:44:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T10:44:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T10:44:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:44:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:44:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-10-06T10:44:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9lwoyOKxAA811lehuQQ8MQQQzuTFAApg==", 2023-10-06T10:44:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:44:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T10:44:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T10:44:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T10:44:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T10:44:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:44:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:44:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-10-06T10:44:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBk4x9lONCoJhAAepF/sou/5M2GBvvfNAs5wQ==", 2023-10-06T10:44:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:44:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T10:44:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T10:44:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T10:44:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:44:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:44:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:44:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-10-06T10:44:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBl4x9ll8wgBRAAQUrHERQ3Igv1tosa+wryIg==", 2023-10-06T10:44:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:44:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T10:44:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T10:44:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:44:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:44:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:44:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-10-06T10:44:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBk4x9lgYa/OhAAyoI717Ga+Wv72srZ4FK+lQ==", 2023-10-06T10:44:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T10:44:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T10:44:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T10:44:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T10:44:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T10:44:47.126 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:44:47.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.109+0000 7feb5bf68700 1 -- 172.21.15.92:0/2768102679 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7feb38066f30 msgr2=0x7feb380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:47.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.109+0000 7feb5bf68700 1 --2- 172.21.15.92:0/2768102679 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7feb38066f30 0x7feb380693e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7feb540d11c0 tx=0x7feb50019000 comp rx=0 tx=0).stop 2023-10-06T10:44:47.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.109+0000 7feb5bf68700 1 -- 172.21.15.92:0/2768102679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb54159560 msgr2=0x7feb54069bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:47.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.109+0000 7feb5bf68700 1 --2- 172.21.15.92:0/2768102679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb54159560 0x7feb54069bb0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7feb4c00c560 tx=0x7feb4c007ef0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.109+0000 7feb5bf68700 1 -- 172.21.15.92:0/2768102679 shutdown_connections 2023-10-06T10:44:47.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.109+0000 7feb5bf68700 1 --2- 172.21.15.92:0/2768102679 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7feb38066f30 0x7feb380693e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.109+0000 7feb5bf68700 1 --2- 172.21.15.92:0/2768102679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb54159560 0x7feb54069bb0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.109+0000 7feb5bf68700 1 --2- 172.21.15.92:0/2768102679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb540b0760 0x7feb54147440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.109+0000 7feb5bf68700 1 --2- 172.21.15.92:0/2768102679 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb540b49a0 0x7feb54069670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.109+0000 7feb5bf68700 1 -- 172.21.15.92:0/2768102679 >> 172.21.15.92:0/2768102679 conn(0x7feb540997f0 msgr2=0x7feb54158960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:47.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.109+0000 7feb5bf68700 1 -- 172.21.15.92:0/2768102679 shutdown_connections 2023-10-06T10:44:47.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.109+0000 7feb5bf68700 1 -- 172.21.15.92:0/2768102679 wait complete. 2023-10-06T10:44:47.129 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:44:47.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.242+0000 7f82d8bb0700 1 Processor -- start 2023-10-06T10:44:47.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.242+0000 7f82d8bb0700 1 -- start start 2023-10-06T10:44:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.242+0000 7f82d8bb0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f82d4159560 0x7f82d40aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.242+0000 7f82d8bb0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82d40b0760 0x7f82d40b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.242+0000 7f82d8bb0700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d40b0c60 con 0x7f82d40b49a0 2023-10-06T10:44:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.242+0000 7f82d8bb0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82d40b0f40 con 0x7f82d4159560 2023-10-06T10:44:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.242+0000 7f82d8bb0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d40b1220 con 0x7f82d40b0760 2023-10-06T10:44:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.243+0000 7f82d259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f82d4159560 0x7f82d40aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.243+0000 7f82d259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f82d4159560 0x7f82d40aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50766/0 (socket says 172.21.15.92:50766) 2023-10-06T10:44:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.243+0000 7f82d259c700 1 -- 172.21.15.92:0/4199076095 learned_addr learned my addr 172.21.15.92:0/4199076095 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.243+0000 7f82d2d9d700 1 --2- 172.21.15.92:0/4199076095 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82d40b0760 0x7f82d40b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.243+0000 7f82d259c700 1 -- 172.21.15.92:0/4199076095 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82d40b0760 msgr2=0x7f82d40b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.243+0000 7f82d259c700 1 --2- 172.21.15.92:0/4199076095 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82d40b0760 0x7f82d40b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.243+0000 7f82d259c700 1 -- 172.21.15.92:0/4199076095 >> v1:172.21.15.124:6789/0 conn(0x7f82d40b49a0 legacy=0x7f82d40aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.243+0000 7f82d259c700 1 -- 172.21.15.92:0/4199076095 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d40b1500 con 0x7f82d4159560 2023-10-06T10:44:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.243+0000 7f82d259c700 1 --2- 172.21.15.92:0/4199076095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82d4159560 0x7f82d40aa530 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f82c40099c0 tx=0x7f82c4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad480280f01b7b39 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.243+0000 7f82d1d9b700 1 -- 172.21.15.92:0/4199076095 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4005c10 con 0x7f82d4159560 2023-10-06T10:44:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.243+0000 7f82d1d9b700 1 -- 172.21.15.92:0/4199076095 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c4005d70 con 0x7f82d4159560 2023-10-06T10:44:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.243+0000 7f82d1d9b700 1 -- 172.21.15.92:0/4199076095 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c400b840 con 0x7f82d4159560 2023-10-06T10:44:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.244+0000 7f82d8bb0700 1 -- 172.21.15.92:0/4199076095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82d4159560 msgr2=0x7f82d40aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.244+0000 7f82d8bb0700 1 --2- 172.21.15.92:0/4199076095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82d4159560 0x7f82d40aa530 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f82c40099c0 tx=0x7f82c4009220 comp rx=0 tx=0).stop 2023-10-06T10:44:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.244+0000 7f82d8bb0700 1 -- 172.21.15.92:0/4199076095 shutdown_connections 2023-10-06T10:44:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.244+0000 7f82d8bb0700 1 --2- 172.21.15.92:0/4199076095 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82d40b0760 0x7f82d40b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.244+0000 7f82d8bb0700 1 --2- 172.21.15.92:0/4199076095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82d4159560 0x7f82d40aa530 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.244+0000 7f82d8bb0700 1 -- 172.21.15.92:0/4199076095 >> 172.21.15.92:0/4199076095 conn(0x7f82d40997f0 msgr2=0x7f82d406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.244+0000 7f82d8bb0700 1 -- 172.21.15.92:0/4199076095 shutdown_connections 2023-10-06T10:44:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.244+0000 7f82d8bb0700 1 -- 172.21.15.92:0/4199076095 wait complete. 2023-10-06T10:44:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.245+0000 7f82d8bb0700 1 Processor -- start 2023-10-06T10:44:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.245+0000 7f82d8bb0700 1 -- start start 2023-10-06T10:44:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.246+0000 7f82d8bb0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82d40b0760 0x7f82d414a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.246+0000 7f82d8bb0700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82d40b49a0 0x7f82d40ab410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.246+0000 7f82d8bb0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82d4159560 0x7f82d40cec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.246+0000 7f82d8bb0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d40abed0 con 0x7f82d40b0760 2023-10-06T10:44:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.246+0000 7f82d8bb0700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d40ad420 con 0x7f82d40b49a0 2023-10-06T10:44:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.246+0000 7f82d8bb0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d409b9d0 con 0x7f82d4159560 2023-10-06T10:44:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.246+0000 7f82d259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82d40b0760 0x7f82d414a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.246+0000 7f82d259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82d40b0760 0x7f82d414a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50776/0 (socket says 172.21.15.92:50776) 2023-10-06T10:44:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.246+0000 7f82d259c700 1 -- 172.21.15.92:0/1776414173 learned_addr learned my addr 172.21.15.92:0/1776414173 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.246+0000 7f82d2d9d700 1 --2- 172.21.15.92:0/1776414173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82d4159560 0x7f82d40cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.246+0000 7f82c9d9b700 1 --2- 172.21.15.92:0/1776414173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82d40b49a0 0x7f82d40ab410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.246+0000 7f82d259c700 1 -- 172.21.15.92:0/1776414173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82d4159560 msgr2=0x7f82d40cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.246+0000 7f82d259c700 1 --2- 172.21.15.92:0/1776414173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82d4159560 0x7f82d40cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.246+0000 7f82d259c700 1 -- 172.21.15.92:0/1776414173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82d40b49a0 msgr2=0x7f82d40ab410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.246+0000 7f82d259c700 1 --2- 172.21.15.92:0/1776414173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82d40b49a0 0x7f82d40ab410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.246+0000 7f82d259c700 1 -- 172.21.15.92:0/1776414173 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c4009480 con 0x7f82d40b0760 2023-10-06T10:44:47.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.246+0000 7f82d259c700 1 --2- 172.21.15.92:0/1776414173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82d40b0760 0x7f82d414a710 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f82d4067c20 tx=0x7f82c4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:47.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.247+0000 7f82caffd700 1 -- 172.21.15.92:0/1776414173 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4011a30 con 0x7f82d40b0760 2023-10-06T10:44:47.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.247+0000 7f82caffd700 1 -- 172.21.15.92:0/1776414173 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c4011b90 con 0x7f82d40b0760 2023-10-06T10:44:47.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.247+0000 7f82d8bb0700 1 -- 172.21.15.92:0/1776414173 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d409bcb0 con 0x7f82d40b0760 2023-10-06T10:44:47.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.247+0000 7f82caffd700 1 -- 172.21.15.92:0/1776414173 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4025470 con 0x7f82d40b0760 2023-10-06T10:44:47.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.247+0000 7f82d8bb0700 1 -- 172.21.15.92:0/1776414173 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d409c270 con 0x7f82d40b0760 2023-10-06T10:44:47.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.248+0000 7f82d8bb0700 1 -- 172.21.15.92:0/1776414173 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d4064a90 con 0x7f82d40b0760 2023-10-06T10:44:47.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.248+0000 7f82caffd700 1 -- 172.21.15.92:0/1776414173 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f82c4022070 con 0x7f82d40b0760 2023-10-06T10:44:47.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.248+0000 7f82caffd700 1 --2- 172.21.15.92:0/1776414173 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f82b4066f30 0x7f82b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:47.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.248+0000 7f82caffd700 1 -- 172.21.15.92:0/1776414173 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f82c40863a0 con 0x7f82d40b0760 2023-10-06T10:44:47.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.248+0000 7f82c9d9b700 1 --2- 172.21.15.92:0/1776414173 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f82b4066f30 0x7f82b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:47.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.249+0000 7f82c9d9b700 1 --2- 172.21.15.92:0/1776414173 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f82b4066f30 0x7f82b40693e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f82d40ad210 tx=0x7f82bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:47.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.253+0000 7f82caffd700 1 -- 172.21.15.92:0/1776414173 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82c4053740 con 0x7f82d40b0760 2023-10-06T10:44:47.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.438+0000 7f82d8bb0700 1 -- 172.21.15.92:0/1776414173 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82d40b17e0 con 0x7f82d40b0760 2023-10-06T10:44:47.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.439+0000 7f82caffd700 1 -- 172.21.15.92:0/1776414173 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+13856 (secure 0 0 0) 0x7f82c4056d60 con 0x7f82d40b0760 2023-10-06T10:44:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:47.441 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":98,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:44:46.700675+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:44:47.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.441+0000 7f82d8bb0700 1 -- 172.21.15.92:0/1776414173 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f82b4066f30 msgr2=0x7f82b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:47.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.441+0000 7f82d8bb0700 1 --2- 172.21.15.92:0/1776414173 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f82b4066f30 0x7f82b40693e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f82d40ad210 tx=0x7f82bc009040 comp rx=0 tx=0).stop 2023-10-06T10:44:47.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.441+0000 7f82d8bb0700 1 -- 172.21.15.92:0/1776414173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82d40b0760 msgr2=0x7f82d414a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:47.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.441+0000 7f82d8bb0700 1 --2- 172.21.15.92:0/1776414173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82d40b0760 0x7f82d414a710 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f82d4067c20 tx=0x7f82c4016000 comp rx=0 tx=0).stop 2023-10-06T10:44:47.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.441+0000 7f82d8bb0700 1 -- 172.21.15.92:0/1776414173 shutdown_connections 2023-10-06T10:44:47.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.441+0000 7f82d8bb0700 1 --2- 172.21.15.92:0/1776414173 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f82b4066f30 0x7f82b40693e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.441+0000 7f82d8bb0700 1 --2- 172.21.15.92:0/1776414173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82d4159560 0x7f82d40cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.441+0000 7f82d8bb0700 1 --2- 172.21.15.92:0/1776414173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82d40b0760 0x7f82d414a710 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.441+0000 7f82d8bb0700 1 --2- 172.21.15.92:0/1776414173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82d40b49a0 0x7f82d40ab410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.441+0000 7f82d8bb0700 1 -- 172.21.15.92:0/1776414173 >> 172.21.15.92:0/1776414173 conn(0x7f82d40997f0 msgr2=0x7f82d40ccca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:47.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.441+0000 7f82d8bb0700 1 -- 172.21.15.92:0/1776414173 shutdown_connections 2023-10-06T10:44:47.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.441+0000 7f82d8bb0700 1 -- 172.21.15.92:0/1776414173 wait complete. 2023-10-06T10:44:47.455 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T10:38:09.680559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T10:38:10.120786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T10:44:47.455 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T10:44:47.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.571+0000 7f36f12bf700 1 Processor -- start 2023-10-06T10:44:47.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.571+0000 7f36f12bf700 1 -- start start 2023-10-06T10:44:47.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.572+0000 7f36f12bf700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36ec0b2620 0x7f36ec0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:47.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.572+0000 7f36f12bf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f36ec0b29f0 0x7f36ec0cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:47.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.572+0000 7f36f12bf700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ec0cc970 con 0x7f36ec0ceaa0 2023-10-06T10:44:47.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.572+0000 7f36f12bf700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36ec0cbd60 con 0x7f36ec0b29f0 2023-10-06T10:44:47.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.572+0000 7f36f12bf700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec0cc040 con 0x7f36ec0b2620 2023-10-06T10:44:47.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.572+0000 7f36eb7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f36ec0b29f0 0x7f36ec0cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:47.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.572+0000 7f36eb7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f36ec0b29f0 0x7f36ec0cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50788/0 (socket says 172.21.15.92:50788) 2023-10-06T10:44:47.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.572+0000 7f36eb7fe700 1 -- 172.21.15.92:0/539917152 learned_addr learned my addr 172.21.15.92:0/539917152 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:47.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.572+0000 7f36ebfff700 1 --2- 172.21.15.92:0/539917152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36ec0b2620 0x7f36ec0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:47.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.572+0000 7f36eb7fe700 1 -- 172.21.15.92:0/539917152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36ec0b2620 msgr2=0x7f36ec0cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:47.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.572+0000 7f36eb7fe700 1 --2- 172.21.15.92:0/539917152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36ec0b2620 0x7f36ec0cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.572+0000 7f36eb7fe700 1 -- 172.21.15.92:0/539917152 >> v1:172.21.15.124:6789/0 conn(0x7f36ec0ceaa0 legacy=0x7f36ec0cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:47.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.572+0000 7f36eb7fe700 1 -- 172.21.15.92:0/539917152 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ec0cc320 con 0x7f36ec0b29f0 2023-10-06T10:44:47.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.572+0000 7f36eb7fe700 1 --2- 172.21.15.92:0/539917152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec0b29f0 0x7f36ec0cd1f0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f36e00099b0 tx=0x7f36e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4d909b9f6c6240 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:47.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.572+0000 7f36eaffd700 1 -- 172.21.15.92:0/539917152 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e000b820 con 0x7f36ec0b29f0 2023-10-06T10:44:47.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.573+0000 7f36eaffd700 1 -- 172.21.15.92:0/539917152 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e000b980 con 0x7f36ec0b29f0 2023-10-06T10:44:47.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.573+0000 7f36eaffd700 1 -- 172.21.15.92:0/539917152 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e000bd20 con 0x7f36ec0b29f0 2023-10-06T10:44:47.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.573+0000 7f36f12bf700 1 -- 172.21.15.92:0/539917152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec0b29f0 msgr2=0x7f36ec0cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:47.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.573+0000 7f36f12bf700 1 --2- 172.21.15.92:0/539917152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec0b29f0 0x7f36ec0cd1f0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f36e00099b0 tx=0x7f36e0009250 comp rx=0 tx=0).stop 2023-10-06T10:44:47.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.573+0000 7f36f12bf700 1 -- 172.21.15.92:0/539917152 shutdown_connections 2023-10-06T10:44:47.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.573+0000 7f36f12bf700 1 --2- 172.21.15.92:0/539917152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36ec0b2620 0x7f36ec0cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.573+0000 7f36f12bf700 1 --2- 172.21.15.92:0/539917152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec0b29f0 0x7f36ec0cd1f0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.573+0000 7f36f12bf700 1 -- 172.21.15.92:0/539917152 >> 172.21.15.92:0/539917152 conn(0x7f36ec099520 msgr2=0x7f36ec06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:47.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.573+0000 7f36f12bf700 1 -- 172.21.15.92:0/539917152 shutdown_connections 2023-10-06T10:44:47.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.573+0000 7f36f12bf700 1 -- 172.21.15.92:0/539917152 wait complete. 2023-10-06T10:44:47.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.574+0000 7f36f12bf700 1 Processor -- start 2023-10-06T10:44:47.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.574+0000 7f36f12bf700 1 -- start start 2023-10-06T10:44:47.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.575+0000 7f36f12bf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec0b2620 0x7f36ec16a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:47.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.575+0000 7f36f12bf700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36ec0b29f0 0x7f36ec15a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:47.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.575+0000 7f36f12bf700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36ec0ceaa0 0x7f36ec15d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:47.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.575+0000 7f36f12bf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec09a020 con 0x7f36ec0b2620 2023-10-06T10:44:47.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.575+0000 7f36f12bf700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec15a920 con 0x7f36ec0ceaa0 2023-10-06T10:44:47.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.575+0000 7f36f12bf700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec15cf70 con 0x7f36ec0b29f0 2023-10-06T10:44:47.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.575+0000 7f36ebfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec0b2620 0x7f36ec16a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:47.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.575+0000 7f36ebfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec0b2620 0x7f36ec16a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50802/0 (socket says 172.21.15.92:50802) 2023-10-06T10:44:47.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.575+0000 7f36ebfff700 1 -- 172.21.15.92:0/2184227562 learned_addr learned my addr 172.21.15.92:0/2184227562 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:47.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.575+0000 7f36ebfff700 1 -- 172.21.15.92:0/2184227562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36ec0b29f0 msgr2=0x7f36ec15a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:47.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.575+0000 7f36eb7fe700 1 --2- 172.21.15.92:0/2184227562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36ec0b29f0 0x7f36ec15a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:47.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.575+0000 7f36f0abe700 1 --2- 172.21.15.92:0/2184227562 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36ec0ceaa0 0x7f36ec15d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:47.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.575+0000 7f36ebfff700 1 --2- 172.21.15.92:0/2184227562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36ec0b29f0 0x7f36ec15a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.575+0000 7f36ebfff700 1 -- 172.21.15.92:0/2184227562 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36ec0ceaa0 msgr2=0x7f36ec15d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:47.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.575+0000 7f36ebfff700 1 --2- 172.21.15.92:0/2184227562 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36ec0ceaa0 0x7f36ec15d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.575+0000 7f36ebfff700 1 -- 172.21.15.92:0/2184227562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e00094f0 con 0x7f36ec0b2620 2023-10-06T10:44:47.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.575+0000 7f36eb7fe700 1 --2- 172.21.15.92:0/2184227562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36ec0b29f0 0x7f36ec15a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:47.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.576+0000 7f36f0abe700 1 --2- 172.21.15.92:0/2184227562 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36ec0ceaa0 0x7f36ec15d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:47.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.576+0000 7f36ebfff700 1 --2- 172.21.15.92:0/2184227562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec0b2620 0x7f36ec16a640 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00d870 tx=0x7f36dc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:47.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.576+0000 7f36d3fff700 1 -- 172.21.15.92:0/2184227562 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc009630 con 0x7f36ec0b2620 2023-10-06T10:44:47.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.576+0000 7f36d3fff700 1 -- 172.21.15.92:0/2184227562 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36dc009790 con 0x7f36ec0b2620 2023-10-06T10:44:47.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.576+0000 7f36f12bf700 1 -- 172.21.15.92:0/2184227562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ec15d940 con 0x7f36ec0b2620 2023-10-06T10:44:47.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.576+0000 7f36d3fff700 1 -- 172.21.15.92:0/2184227562 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc009b50 con 0x7f36ec0b2620 2023-10-06T10:44:47.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.576+0000 7f36f12bf700 1 -- 172.21.15.92:0/2184227562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ec15c920 con 0x7f36ec0b2620 2023-10-06T10:44:47.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.577+0000 7f36f12bf700 1 -- 172.21.15.92:0/2184227562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ec05f230 con 0x7f36ec0b2620 2023-10-06T10:44:47.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.577+0000 7f36d3fff700 1 -- 172.21.15.92:0/2184227562 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f36dc014410 con 0x7f36ec0b2620 2023-10-06T10:44:47.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.577+0000 7f36d3fff700 1 --2- 172.21.15.92:0/2184227562 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f36d8066f30 0x7f36d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:47.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.578+0000 7f36d3fff700 1 -- 172.21.15.92:0/2184227562 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f36dc084cc0 con 0x7f36ec0b2620 2023-10-06T10:44:47.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.578+0000 7f36eb7fe700 1 --2- 172.21.15.92:0/2184227562 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f36d8066f30 0x7f36d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:47.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.578+0000 7f36eb7fe700 1 --2- 172.21.15.92:0/2184227562 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f36d8066f30 0x7f36d80693e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f36e00099b0 tx=0x7f36e0016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:47.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.582+0000 7f36d3fff700 1 -- 172.21.15.92:0/2184227562 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36dc051fe0 con 0x7f36ec0b2620 2023-10-06T10:44:47.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.765+0000 7f36f12bf700 1 -- 172.21.15.92:0/2184227562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f36ec0cc600 con 0x7f36ec0b2620 2023-10-06T10:44:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.765+0000 7f36d3fff700 1 -- 172.21.15.92:0/2184227562 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v98) v1 ==== 92+0+10 (secure 0 0 0) 0x7f36dc055600 con 0x7f36ec0b2620 2023-10-06T10:44:47.767 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T10:44:47.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.767+0000 7f36f12bf700 1 -- 172.21.15.92:0/2184227562 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f36d8066f30 msgr2=0x7f36d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:47.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.767+0000 7f36f12bf700 1 --2- 172.21.15.92:0/2184227562 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f36d8066f30 0x7f36d80693e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f36e00099b0 tx=0x7f36e0016000 comp rx=0 tx=0).stop 2023-10-06T10:44:47.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.767+0000 7f36f12bf700 1 -- 172.21.15.92:0/2184227562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec0b2620 msgr2=0x7f36ec16a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:47.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.767+0000 7f36f12bf700 1 --2- 172.21.15.92:0/2184227562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec0b2620 0x7f36ec16a640 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00d870 tx=0x7f36dc008040 comp rx=0 tx=0).stop 2023-10-06T10:44:47.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.767+0000 7f36f12bf700 1 -- 172.21.15.92:0/2184227562 shutdown_connections 2023-10-06T10:44:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.767+0000 7f36f12bf700 1 --2- 172.21.15.92:0/2184227562 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f36d8066f30 0x7f36d80693e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.767+0000 7f36f12bf700 1 --2- 172.21.15.92:0/2184227562 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36ec0ceaa0 0x7f36ec15d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.767+0000 7f36f12bf700 1 --2- 172.21.15.92:0/2184227562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec0b2620 0x7f36ec16a640 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.767+0000 7f36f12bf700 1 --2- 172.21.15.92:0/2184227562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36ec0b29f0 0x7f36ec15a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.767+0000 7f36f12bf700 1 -- 172.21.15.92:0/2184227562 >> 172.21.15.92:0/2184227562 conn(0x7f36ec099520 msgr2=0x7f36ec0ccb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.767+0000 7f36f12bf700 1 -- 172.21.15.92:0/2184227562 shutdown_connections 2023-10-06T10:44:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.767+0000 7f36f12bf700 1 -- 172.21.15.92:0/2184227562 wait complete. 2023-10-06T10:44:47.781 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T10:44:47.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.896+0000 7febef59e700 1 Processor -- start 2023-10-06T10:44:47.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.896+0000 7febef59e700 1 -- start start 2023-10-06T10:44:47.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.897+0000 7febef59e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7febf00cd3c0 0x7febf00ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:47.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.897+0000 7febef59e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7febf00b29f0 0x7febf00cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:47.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.897+0000 7febef59e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf00cc970 con 0x7febf00b2620 2023-10-06T10:44:47.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.897+0000 7febef59e700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf00cf520 con 0x7febf00b29f0 2023-10-06T10:44:47.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.897+0000 7febef59e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf00cbd60 con 0x7febf00cd3c0 2023-10-06T10:44:47.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.897+0000 7febedd9b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7febf00cd3c0 0x7febf00ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:47.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.897+0000 7febeed9d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7febf00b29f0 0x7febf00cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:47.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.897+0000 7febee59c700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7febf00b2620 0x7febf00cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:50512/0 (socket says 172.21.15.92:50512) 2023-10-06T10:44:47.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.897+0000 7febee59c700 1 -- 172.21.15.92:0/3241115158 learned_addr learned my addr 172.21.15.92:0/3241115158 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:47.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.897+0000 7febedd9b700 1 --2- 172.21.15.92:0/3241115158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7febf00cd3c0 0x7febf00ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:47988/0 (socket says 172.21.15.92:47988) 2023-10-06T10:44:47.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.897+0000 7febed59a700 1 -- 172.21.15.92:0/3241115158 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3638335801 0 0) 0x7febf00cc970 con 0x7febf00b2620 2023-10-06T10:44:47.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.898+0000 7febed59a700 1 -- 172.21.15.92:0/3241115158 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febd8003570 con 0x7febf00b2620 2023-10-06T10:44:47.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.898+0000 7febedd9b700 1 -- 172.21.15.92:0/3241115158 >> v2:172.21.15.124:3300/0 conn(0x7febf00b29f0 msgr2=0x7febf00cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:47.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.898+0000 7febedd9b700 1 --2- 172.21.15.92:0/3241115158 >> v2:172.21.15.124:3300/0 conn(0x7febf00b29f0 0x7febf00cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.898+0000 7febedd9b700 1 -- 172.21.15.92:0/3241115158 >> v1:172.21.15.124:6789/0 conn(0x7febf00b2620 legacy=0x7febf00cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:47.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.898+0000 7febedd9b700 1 -- 172.21.15.92:0/3241115158 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf00cc040 con 0x7febf00cd3c0 2023-10-06T10:44:47.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.898+0000 7febeed9d700 1 --2- 172.21.15.92:0/3241115158 >> v2:172.21.15.124:3300/0 conn(0x7febf00b29f0 0x7febf00cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:44:47.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.898+0000 7febedd9b700 1 --2- 172.21.15.92:0/3241115158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7febf00cd3c0 0x7febf00ceaa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7febe40098e0 tx=0x7febe40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddbcbd56ad75b416 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:47.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.898+0000 7febed59a700 1 -- 172.21.15.92:0/3241115158 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe4011630 con 0x7febf00cd3c0 2023-10-06T10:44:47.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.898+0000 7febed59a700 1 -- 172.21.15.92:0/3241115158 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe4011790 con 0x7febf00cd3c0 2023-10-06T10:44:47.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.898+0000 7febed59a700 1 -- 172.21.15.92:0/3241115158 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe4011b30 con 0x7febf00cd3c0 2023-10-06T10:44:47.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.898+0000 7febef59e700 1 -- 172.21.15.92:0/3241115158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7febf00cd3c0 msgr2=0x7febf00ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:47.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.898+0000 7febef59e700 1 --2- 172.21.15.92:0/3241115158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7febf00cd3c0 0x7febf00ceaa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7febe40098e0 tx=0x7febe40091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.899+0000 7febef59e700 1 -- 172.21.15.92:0/3241115158 shutdown_connections 2023-10-06T10:44:47.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.899+0000 7febef59e700 1 --2- 172.21.15.92:0/3241115158 >> v2:172.21.15.124:3300/0 conn(0x7febf00b29f0 0x7febf00cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.899+0000 7febef59e700 1 --2- 172.21.15.92:0/3241115158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7febf00cd3c0 0x7febf00ceaa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.899+0000 7febef59e700 1 -- 172.21.15.92:0/3241115158 >> 172.21.15.92:0/3241115158 conn(0x7febf0099520 msgr2=0x7febf006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:47.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.899+0000 7febef59e700 1 -- 172.21.15.92:0/3241115158 shutdown_connections 2023-10-06T10:44:47.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.899+0000 7febef59e700 1 -- 172.21.15.92:0/3241115158 wait complete. 2023-10-06T10:44:47.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.900+0000 7febef59e700 1 Processor -- start 2023-10-06T10:44:47.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.900+0000 7febef59e700 1 -- start start 2023-10-06T10:44:47.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.900+0000 7febef59e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7febf00b2620 0x7febf0155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:47.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.900+0000 7febef59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febf00b29f0 0x7febf0158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:47.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.900+0000 7febef59e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7febf00cd3c0 0x7febf01586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:47.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.900+0000 7febef59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf01558b0 con 0x7febf00b29f0 2023-10-06T10:44:47.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.900+0000 7febef59e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf00b5100 con 0x7febf00b2620 2023-10-06T10:44:47.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.900+0000 7febef59e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf00b3090 con 0x7febf00cd3c0 2023-10-06T10:44:47.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.900+0000 7febedd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febf00b29f0 0x7febf0158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:47.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.900+0000 7febedd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febf00b29f0 0x7febf0158190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50804/0 (socket says 172.21.15.92:50804) 2023-10-06T10:44:47.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.900+0000 7febedd9b700 1 -- 172.21.15.92:0/4171613194 learned_addr learned my addr 172.21.15.92:0/4171613194 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:47.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.900+0000 7febee59c700 1 --2- 172.21.15.92:0/4171613194 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7febf00b2620 0x7febf0155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:47.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.900+0000 7febeed9d700 1 --2- 172.21.15.92:0/4171613194 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7febf00cd3c0 0x7febf01586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:47.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.901+0000 7febedd9b700 1 -- 172.21.15.92:0/4171613194 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7febf00cd3c0 msgr2=0x7febf01586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:47.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.901+0000 7febedd9b700 1 --2- 172.21.15.92:0/4171613194 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7febf00cd3c0 0x7febf01586d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.901+0000 7febedd9b700 1 -- 172.21.15.92:0/4171613194 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7febf00b2620 msgr2=0x7febf0155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:47.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.901+0000 7febedd9b700 1 --2- 172.21.15.92:0/4171613194 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7febf00b2620 0x7febf0155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:47.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.901+0000 7febedd9b700 1 -- 172.21.15.92:0/4171613194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe8009a20 con 0x7febf00b29f0 2023-10-06T10:44:47.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.901+0000 7febedd9b700 1 --2- 172.21.15.92:0/4171613194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febf00b29f0 0x7febf0158190 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7febe4009ed0 tx=0x7febe4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:47.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.901+0000 7febde7fc700 1 -- 172.21.15.92:0/4171613194 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe4011b10 con 0x7febf00b29f0 2023-10-06T10:44:47.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.901+0000 7febde7fc700 1 -- 172.21.15.92:0/4171613194 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe40253f0 con 0x7febf00b29f0 2023-10-06T10:44:47.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.901+0000 7febef59e700 1 -- 172.21.15.92:0/4171613194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe4009420 con 0x7febf00b29f0 2023-10-06T10:44:47.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.901+0000 7febde7fc700 1 -- 172.21.15.92:0/4171613194 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe4025790 con 0x7febf00b29f0 2023-10-06T10:44:47.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.901+0000 7febef59e700 1 -- 172.21.15.92:0/4171613194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf00b3c10 con 0x7febf00b29f0 2023-10-06T10:44:47.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.902+0000 7febef59e700 1 -- 172.21.15.92:0/4171613194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf00b3da0 con 0x7febf00b29f0 2023-10-06T10:44:47.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.902+0000 7febde7fc700 1 -- 172.21.15.92:0/4171613194 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7febe4025b80 con 0x7febf00b29f0 2023-10-06T10:44:47.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.903+0000 7febde7fc700 1 --2- 172.21.15.92:0/4171613194 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7febd0066f30 0x7febd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:47.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.903+0000 7febde7fc700 1 -- 172.21.15.92:0/4171613194 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5526+0+0 (secure 0 0 0) 0x7febe4085b00 con 0x7febf00b29f0 2023-10-06T10:44:47.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.903+0000 7febee59c700 1 --2- 172.21.15.92:0/4171613194 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7febd0066f30 0x7febd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:47.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.903+0000 7febee59c700 1 --2- 172.21.15.92:0/4171613194 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7febd0066f30 0x7febd00693e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7febf015b0e0 tx=0x7febe0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:47.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:47.907+0000 7febde7fc700 1 -- 172.21.15.92:0/4171613194 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7febe4052ee0 con 0x7febf00b29f0 2023-10-06T10:44:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.088+0000 7febef59e700 1 -- 172.21.15.92:0/4171613194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7febf005f230 con 0x7febf00b29f0 2023-10-06T10:44:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.088+0000 7febde7fc700 1 -- 172.21.15.92:0/4171613194 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7febe4056500 con 0x7febf00b29f0 2023-10-06T10:44:48.090 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T10:44:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.090+0000 7febef59e700 1 -- 172.21.15.92:0/4171613194 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7febd0066f30 msgr2=0x7febd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.090+0000 7febef59e700 1 --2- 172.21.15.92:0/4171613194 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7febd0066f30 0x7febd00693e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7febf015b0e0 tx=0x7febe0009040 comp rx=0 tx=0).stop 2023-10-06T10:44:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.090+0000 7febef59e700 1 -- 172.21.15.92:0/4171613194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febf00b29f0 msgr2=0x7febf0158190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.090+0000 7febef59e700 1 --2- 172.21.15.92:0/4171613194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febf00b29f0 0x7febf0158190 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7febe4009ed0 tx=0x7febe4017000 comp rx=0 tx=0).stop 2023-10-06T10:44:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.090+0000 7febef59e700 1 -- 172.21.15.92:0/4171613194 shutdown_connections 2023-10-06T10:44:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.090+0000 7febef59e700 1 --2- 172.21.15.92:0/4171613194 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7febd0066f30 0x7febd00693e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.090+0000 7febef59e700 1 --2- 172.21.15.92:0/4171613194 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7febf00cd3c0 0x7febf01586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.090+0000 7febef59e700 1 --2- 172.21.15.92:0/4171613194 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7febf00b2620 0x7febf0155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.090+0000 7febef59e700 1 --2- 172.21.15.92:0/4171613194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febf00b29f0 0x7febf0158190 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.090+0000 7febef59e700 1 -- 172.21.15.92:0/4171613194 >> 172.21.15.92:0/4171613194 conn(0x7febf0099520 msgr2=0x7febf00cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.090+0000 7febef59e700 1 -- 172.21.15.92:0/4171613194 shutdown_connections 2023-10-06T10:44:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.090+0000 7febef59e700 1 -- 172.21.15.92:0/4171613194 wait complete. 2023-10-06T10:44:48.104 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T10:44:48.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.215+0000 7f74a0d9e700 1 Processor -- start 2023-10-06T10:44:48.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.215+0000 7f74a0d9e700 1 -- start start 2023-10-06T10:44:48.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.216+0000 7f74a0d9e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f749c159560 0x7f749c0aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:48.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.216+0000 7f74a0d9e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f749c0b0760 0x7f749c0b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:48.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.216+0000 7f74a0d9e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749c0b0c60 con 0x7f749c0b49a0 2023-10-06T10:44:48.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.216+0000 7f74a0d9e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f749c0b0f40 con 0x7f749c0b0760 2023-10-06T10:44:48.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.216+0000 7f74a0d9e700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f749c0b1220 con 0x7f749c159560 2023-10-06T10:44:48.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.216+0000 7f749ad9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f749c0b0760 0x7f749c0b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:48.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.216+0000 7f749ad9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f749c0b0760 0x7f749c0b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50806/0 (socket says 172.21.15.92:50806) 2023-10-06T10:44:48.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.216+0000 7f749ad9d700 1 -- 172.21.15.92:0/2717731143 learned_addr learned my addr 172.21.15.92:0/2717731143 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:48.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.216+0000 7f749a59c700 1 --2- 172.21.15.92:0/2717731143 >> v2:172.21.15.124:3300/0 conn(0x7f749c159560 0x7f749c0aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:48.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.216+0000 7f749ad9d700 1 -- 172.21.15.92:0/2717731143 >> v2:172.21.15.124:3300/0 conn(0x7f749c159560 msgr2=0x7f749c0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:48.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.216+0000 7f749ad9d700 1 --2- 172.21.15.92:0/2717731143 >> v2:172.21.15.124:3300/0 conn(0x7f749c159560 0x7f749c0aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:48.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.216+0000 7f749ad9d700 1 -- 172.21.15.92:0/2717731143 >> v1:172.21.15.124:6789/0 conn(0x7f749c0b49a0 legacy=0x7f749c0aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:48.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.216+0000 7f749ad9d700 1 -- 172.21.15.92:0/2717731143 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c0b1500 con 0x7f749c0b0760 2023-10-06T10:44:48.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.217+0000 7f749ad9d700 1 --2- 172.21.15.92:0/2717731143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f749c0b0760 0x7f749c0b1bc0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f7494009f20 tx=0x7f7494009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8957785f7617cf9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:48.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.217+0000 7f7499d9b700 1 -- 172.21.15.92:0/2717731143 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749400e070 con 0x7f749c0b0760 2023-10-06T10:44:48.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.217+0000 7f7499d9b700 1 -- 172.21.15.92:0/2717731143 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749400ba00 con 0x7f749c0b0760 2023-10-06T10:44:48.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.217+0000 7f7499d9b700 1 -- 172.21.15.92:0/2717731143 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74940092e0 con 0x7f749c0b0760 2023-10-06T10:44:48.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.217+0000 7f74a0d9e700 1 -- 172.21.15.92:0/2717731143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f749c0b0760 msgr2=0x7f749c0b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:48.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.217+0000 7f74a0d9e700 1 --2- 172.21.15.92:0/2717731143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f749c0b0760 0x7f749c0b1bc0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f7494009f20 tx=0x7f7494009840 comp rx=0 tx=0).stop 2023-10-06T10:44:48.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.217+0000 7f74a0d9e700 1 -- 172.21.15.92:0/2717731143 shutdown_connections 2023-10-06T10:44:48.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.217+0000 7f74a0d9e700 1 --2- 172.21.15.92:0/2717731143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f749c0b0760 0x7f749c0b1bc0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:48.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.217+0000 7f74a0d9e700 1 --2- 172.21.15.92:0/2717731143 >> v2:172.21.15.124:3300/0 conn(0x7f749c159560 0x7f749c0aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:48.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.217+0000 7f74a0d9e700 1 -- 172.21.15.92:0/2717731143 >> 172.21.15.92:0/2717731143 conn(0x7f749c0997f0 msgr2=0x7f749c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:48.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.217+0000 7f74a0d9e700 1 -- 172.21.15.92:0/2717731143 shutdown_connections 2023-10-06T10:44:48.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.217+0000 7f74a0d9e700 1 -- 172.21.15.92:0/2717731143 wait complete. 2023-10-06T10:44:48.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.219+0000 7f74a0d9e700 1 Processor -- start 2023-10-06T10:44:48.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.219+0000 7f74a0d9e700 1 -- start start 2023-10-06T10:44:48.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.219+0000 7f74a0d9e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f749c0b0760 0x7f749c09c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:48.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.219+0000 7f74a0d9e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f749c0b49a0 0x7f749c0ce250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:48.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.219+0000 7f74a0d9e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f749c159560 0x7f749c0ce790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:48.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.219+0000 7f74a0d9e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c09de50 con 0x7f749c159560 2023-10-06T10:44:48.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.219+0000 7f74a0d9e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c09cdb0 con 0x7f749c0b0760 2023-10-06T10:44:48.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.219+0000 7f74a0d9e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749c09d090 con 0x7f749c0b49a0 2023-10-06T10:44:48.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.219+0000 7f749ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f749c159560 0x7f749c0ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:48.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.219+0000 7f749ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f749c159560 0x7f749c0ce790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50820/0 (socket says 172.21.15.92:50820) 2023-10-06T10:44:48.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.219+0000 7f749ad9d700 1 -- 172.21.15.92:0/3509702066 learned_addr learned my addr 172.21.15.92:0/3509702066 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.220+0000 7f749a59c700 1 --2- 172.21.15.92:0/3509702066 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f749c0b0760 0x7f749c09c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.220+0000 7f7491d9b700 1 --2- 172.21.15.92:0/3509702066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f749c0b49a0 0x7f749c0ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.220+0000 7f749ad9d700 1 -- 172.21.15.92:0/3509702066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f749c0b49a0 msgr2=0x7f749c0ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.220+0000 7f749ad9d700 1 --2- 172.21.15.92:0/3509702066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f749c0b49a0 0x7f749c0ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.220+0000 7f749ad9d700 1 -- 172.21.15.92:0/3509702066 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f749c0b0760 msgr2=0x7f749c09c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.220+0000 7f749ad9d700 1 --2- 172.21.15.92:0/3509702066 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f749c0b0760 0x7f749c09c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.220+0000 7f749ad9d700 1 -- 172.21.15.92:0/3509702066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7494009ae0 con 0x7f749c159560 2023-10-06T10:44:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.220+0000 7f749a59c700 1 --2- 172.21.15.92:0/3509702066 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f749c0b0760 0x7f749c09c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.220+0000 7f749ad9d700 1 --2- 172.21.15.92:0/3509702066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f749c159560 0x7f749c0ce790 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f749c067c20 tx=0x7f7494017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.220+0000 7f7492ffd700 1 -- 172.21.15.92:0/3509702066 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7494012ea0 con 0x7f749c159560 2023-10-06T10:44:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.220+0000 7f7492ffd700 1 -- 172.21.15.92:0/3509702066 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74940092e0 con 0x7f749c159560 2023-10-06T10:44:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.220+0000 7f74a0d9e700 1 -- 172.21.15.92:0/3509702066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749c09d370 con 0x7f749c159560 2023-10-06T10:44:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.220+0000 7f7492ffd700 1 -- 172.21.15.92:0/3509702066 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749401d690 con 0x7f749c159560 2023-10-06T10:44:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.220+0000 7f74a0d9e700 1 -- 172.21.15.92:0/3509702066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749c09d930 con 0x7f749c159560 2023-10-06T10:44:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.221+0000 7f74a0d9e700 1 -- 172.21.15.92:0/3509702066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f749c09db20 con 0x7f749c159560 2023-10-06T10:44:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.221+0000 7f7492ffd700 1 -- 172.21.15.92:0/3509702066 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f749401da00 con 0x7f749c159560 2023-10-06T10:44:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.222+0000 7f7492ffd700 1 --2- 172.21.15.92:0/3509702066 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f747c066f30 0x7f747c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.222+0000 7f7492ffd700 1 -- 172.21.15.92:0/3509702066 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f7494085df0 con 0x7f749c159560 2023-10-06T10:44:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.222+0000 7f749a59c700 1 --2- 172.21.15.92:0/3509702066 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f747c066f30 0x7f747c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.222+0000 7f749a59c700 1 --2- 172.21.15.92:0/3509702066 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f747c066f30 0x7f747c0693e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f749c16d1b0 tx=0x7f748c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:48.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.226+0000 7f7492ffd700 1 -- 172.21.15.92:0/3509702066 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7494053110 con 0x7f749c159560 2023-10-06T10:44:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.409+0000 7f74a0d9e700 1 -- 172.21.15.92:0/3509702066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f749c064a90 con 0x7f749c159560 2023-10-06T10:44:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.410+0000 7f7492ffd700 1 -- 172.21.15.92:0/3509702066 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v98) v1 ==== 84+0+126 (secure 0 0 0) 0x7f749401dcb0 con 0x7f749c159560 2023-10-06T10:44:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:48.412 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T10:44:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:44:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-10-06T10:44:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-10-06T10:44:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T10:44:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T10:44:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-10-06T10:44:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-10-06T10:44:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:44:48.413 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T10:44:48.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.412+0000 7f74a0d9e700 1 -- 172.21.15.92:0/3509702066 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f747c066f30 msgr2=0x7f747c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:48.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.412+0000 7f74a0d9e700 1 --2- 172.21.15.92:0/3509702066 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f747c066f30 0x7f747c0693e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f749c16d1b0 tx=0x7f748c008040 comp rx=0 tx=0).stop 2023-10-06T10:44:48.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.412+0000 7f74a0d9e700 1 -- 172.21.15.92:0/3509702066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f749c159560 msgr2=0x7f749c0ce790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:48.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.412+0000 7f74a0d9e700 1 --2- 172.21.15.92:0/3509702066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f749c159560 0x7f749c0ce790 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f749c067c20 tx=0x7f7494017000 comp rx=0 tx=0).stop 2023-10-06T10:44:48.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.412+0000 7f74a0d9e700 1 -- 172.21.15.92:0/3509702066 shutdown_connections 2023-10-06T10:44:48.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.412+0000 7f74a0d9e700 1 --2- 172.21.15.92:0/3509702066 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f747c066f30 0x7f747c0693e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:48.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.412+0000 7f74a0d9e700 1 --2- 172.21.15.92:0/3509702066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f749c159560 0x7f749c0ce790 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:48.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.412+0000 7f74a0d9e700 1 --2- 172.21.15.92:0/3509702066 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f749c0b0760 0x7f749c09c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:48.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.412+0000 7f74a0d9e700 1 --2- 172.21.15.92:0/3509702066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f749c0b49a0 0x7f749c0ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:48.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.412+0000 7f74a0d9e700 1 -- 172.21.15.92:0/3509702066 >> 172.21.15.92:0/3509702066 conn(0x7f749c0997f0 msgr2=0x7f749c0cda50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:48.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.412+0000 7f74a0d9e700 1 -- 172.21.15.92:0/3509702066 shutdown_connections 2023-10-06T10:44:48.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.412+0000 7f74a0d9e700 1 -- 172.21.15.92:0/3509702066 wait complete. 2023-10-06T10:44:48.426 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T10:44:48.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T10:44:48.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.543+0000 7f68a5612700 1 Processor -- start 2023-10-06T10:44:48.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.543+0000 7f68a5612700 1 -- start start 2023-10-06T10:44:48.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.543+0000 7f68a5612700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f68a006ece0 0x7f68a00b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:48.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.543+0000 7f68a5612700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68a006f0b0 0x7f68a00b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:48.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.543+0000 7f68a5612700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f68a00aede0 0x7f68a00af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:48.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.543+0000 7f68a5612700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68a006fb40 con 0x7f68a006ece0 2023-10-06T10:44:48.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.543+0000 7f68a5612700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68a00b2ce0 con 0x7f68a00aede0 2023-10-06T10:44:48.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.543+0000 7f68a5612700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a00b0410 con 0x7f68a006f0b0 2023-10-06T10:44:48.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.544+0000 7f689effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f68a006ece0 0x7f68a00b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:48.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.544+0000 7f689effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f68a006ece0 0x7f68a00b2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50822/0 (socket says 172.21.15.92:50822) 2023-10-06T10:44:48.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.544+0000 7f689effd700 1 -- 172.21.15.92:0/1378759277 learned_addr learned my addr 172.21.15.92:0/1378759277 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:48.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.544+0000 7f689effd700 1 -- 172.21.15.92:0/1378759277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68a006f0b0 msgr2=0x7f68a00b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:48.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.544+0000 7f689effd700 1 --2- 172.21.15.92:0/1378759277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68a006f0b0 0x7f68a00b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:48.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.544+0000 7f689effd700 1 -- 172.21.15.92:0/1378759277 >> v2:172.21.15.124:3300/0 conn(0x7f68a00aede0 msgr2=0x7f68a00af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:48.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.544+0000 7f689f7fe700 1 --2- 172.21.15.92:0/1378759277 >> v2:172.21.15.124:3300/0 conn(0x7f68a00aede0 0x7f68a00af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:48.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.544+0000 7f689effd700 1 --2- 172.21.15.92:0/1378759277 >> v2:172.21.15.124:3300/0 conn(0x7f68a00aede0 0x7f68a00af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:48.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.544+0000 7f689effd700 1 -- 172.21.15.92:0/1378759277 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a00af780 con 0x7f68a006ece0 2023-10-06T10:44:48.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.544+0000 7f689effd700 1 --2- 172.21.15.92:0/1378759277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68a006ece0 0x7f68a00b2360 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f68a00b1350 tx=0x7f6890009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d14f5bd41f7b40 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:48.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.544+0000 7f689dffb700 1 -- 172.21.15.92:0/1378759277 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689000b860 con 0x7f68a006ece0 2023-10-06T10:44:48.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.544+0000 7f689dffb700 1 -- 172.21.15.92:0/1378759277 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689000b9c0 con 0x7f68a006ece0 2023-10-06T10:44:48.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.544+0000 7f689dffb700 1 -- 172.21.15.92:0/1378759277 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689000bd60 con 0x7f68a006ece0 2023-10-06T10:44:48.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.545+0000 7f68a5612700 1 -- 172.21.15.92:0/1378759277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68a006ece0 msgr2=0x7f68a00b2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:48.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.545+0000 7f68a5612700 1 --2- 172.21.15.92:0/1378759277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68a006ece0 0x7f68a00b2360 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f68a00b1350 tx=0x7f6890009250 comp rx=0 tx=0).stop 2023-10-06T10:44:48.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.545+0000 7f68a5612700 1 -- 172.21.15.92:0/1378759277 shutdown_connections 2023-10-06T10:44:48.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.545+0000 7f68a5612700 1 --2- 172.21.15.92:0/1378759277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68a006ece0 0x7f68a00b2360 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:48.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.545+0000 7f68a5612700 1 --2- 172.21.15.92:0/1378759277 >> v2:172.21.15.124:3300/0 conn(0x7f68a00aede0 0x7f68a00af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:48.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.545+0000 7f68a5612700 1 --2- 172.21.15.92:0/1378759277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68a006f0b0 0x7f68a00b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:48.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.545+0000 7f68a5612700 1 -- 172.21.15.92:0/1378759277 >> 172.21.15.92:0/1378759277 conn(0x7f68a00682f0 msgr2=0x7f68a00686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:48.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.545+0000 7f68a5612700 1 -- 172.21.15.92:0/1378759277 shutdown_connections 2023-10-06T10:44:48.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.545+0000 7f68a5612700 1 -- 172.21.15.92:0/1378759277 wait complete. 2023-10-06T10:44:48.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.546+0000 7f68a5612700 1 Processor -- start 2023-10-06T10:44:48.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.546+0000 7f68a5612700 1 -- start start 2023-10-06T10:44:48.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.547+0000 7f68a5612700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68a006ece0 0x7f68a0155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:48.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.547+0000 7f68a5612700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68a006f0b0 0x7f68a00adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:48.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.547+0000 7f68a5612700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68a00aede0 0x7f68a00cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:48.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.547+0000 7f68a5612700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a00a8b50 con 0x7f68a00aede0 2023-10-06T10:44:48.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.547+0000 7f68a5612700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a0156100 con 0x7f68a006f0b0 2023-10-06T10:44:48.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.547+0000 7f68a5612700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a00ae560 con 0x7f68a006ece0 2023-10-06T10:44:48.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.547+0000 7f689f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68a00aede0 0x7f68a00cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:48.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.547+0000 7f689f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68a00aede0 0x7f68a00cc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50834/0 (socket says 172.21.15.92:50834) 2023-10-06T10:44:48.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.547+0000 7f689f7fe700 1 -- 172.21.15.92:0/999413291 learned_addr learned my addr 172.21.15.92:0/999413291 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:48.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.547+0000 7f689effd700 1 --2- 172.21.15.92:0/999413291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68a006ece0 0x7f68a0155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:48.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.547+0000 7f689e7fc700 1 --2- 172.21.15.92:0/999413291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68a006f0b0 0x7f68a00adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:48.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.547+0000 7f689f7fe700 1 -- 172.21.15.92:0/999413291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68a006ece0 msgr2=0x7f68a0155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:48.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.547+0000 7f689f7fe700 1 --2- 172.21.15.92:0/999413291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68a006ece0 0x7f68a0155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:48.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.547+0000 7f689f7fe700 1 -- 172.21.15.92:0/999413291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68a006f0b0 msgr2=0x7f68a00adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.547+0000 7f689f7fe700 1 --2- 172.21.15.92:0/999413291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68a006f0b0 0x7f68a00adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.547+0000 7f689f7fe700 1 -- 172.21.15.92:0/999413291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68900094f0 con 0x7f68a00aede0 2023-10-06T10:44:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.547+0000 7f689effd700 1 --2- 172.21.15.92:0/999413291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68a006ece0 0x7f68a0155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.547+0000 7f689e7fc700 1 --2- 172.21.15.92:0/999413291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68a006f0b0 0x7f68a00adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.547+0000 7f689f7fe700 1 --2- 172.21.15.92:0/999413291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68a00aede0 0x7f68a00cc970 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f689800ceb0 tx=0x7f6898007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.547+0000 7f6886ffd700 1 -- 172.21.15.92:0/999413291 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689800ae40 con 0x7f68a00aede0 2023-10-06T10:44:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.547+0000 7f6886ffd700 1 -- 172.21.15.92:0/999413291 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6898016430 con 0x7f68a00aede0 2023-10-06T10:44:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.547+0000 7f68a5612700 1 -- 172.21.15.92:0/999413291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a00ae840 con 0x7f68a00aede0 2023-10-06T10:44:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.548+0000 7f6886ffd700 1 -- 172.21.15.92:0/999413291 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68980167d0 con 0x7f68a00aede0 2023-10-06T10:44:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.548+0000 7f68a5612700 1 -- 172.21.15.92:0/999413291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a00a87e0 con 0x7f68a00aede0 2023-10-06T10:44:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.548+0000 7f68a5612700 1 -- 172.21.15.92:0/999413291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68a0064a90 con 0x7f68a00aede0 2023-10-06T10:44:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.549+0000 7f6886ffd700 1 -- 172.21.15.92:0/999413291 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6898003020 con 0x7f68a00aede0 2023-10-06T10:44:48.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.549+0000 7f6886ffd700 1 --2- 172.21.15.92:0/999413291 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f688c066f30 0x7f688c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:48.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.549+0000 7f6886ffd700 1 -- 172.21.15.92:0/999413291 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f6898012070 con 0x7f68a00aede0 2023-10-06T10:44:48.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.549+0000 7f689effd700 1 --2- 172.21.15.92:0/999413291 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f688c066f30 0x7f688c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:48.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.550+0000 7f689effd700 1 --2- 172.21.15.92:0/999413291 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f688c066f30 0x7f688c0693e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f68a00b1350 tx=0x7f6890017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:48.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.553+0000 7f6886ffd700 1 -- 172.21.15.92:0/999413291 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6898056150 con 0x7f68a00aede0 2023-10-06T10:44:48.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:48.739+0000 7f68a5612700 1 -- 172.21.15.92:0/999413291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f68a00b2a00 con 0x7f68a00aede0 2023-10-06T10:44:49.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.715+0000 7f6886ffd700 1 -- 172.21.15.92:0/999413291 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v99) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6898055ce0 con 0x7f68a00aede0 2023-10-06T10:44:49.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.720+0000 7f68a5612700 1 -- 172.21.15.92:0/999413291 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f688c066f30 msgr2=0x7f688c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:49.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.720+0000 7f68a5612700 1 --2- 172.21.15.92:0/999413291 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f688c066f30 0x7f688c0693e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f68a00b1350 tx=0x7f6890017000 comp rx=0 tx=0).stop 2023-10-06T10:44:49.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.720+0000 7f68a5612700 1 -- 172.21.15.92:0/999413291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68a00aede0 msgr2=0x7f68a00cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:49.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.720+0000 7f68a5612700 1 --2- 172.21.15.92:0/999413291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68a00aede0 0x7f68a00cc970 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f689800ceb0 tx=0x7f6898007870 comp rx=0 tx=0).stop 2023-10-06T10:44:49.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.721+0000 7f68a5612700 1 -- 172.21.15.92:0/999413291 shutdown_connections 2023-10-06T10:44:49.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.721+0000 7f68a5612700 1 --2- 172.21.15.92:0/999413291 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f688c066f30 0x7f688c0693e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:49.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.721+0000 7f68a5612700 1 --2- 172.21.15.92:0/999413291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68a00aede0 0x7f68a00cc970 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:49.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.721+0000 7f68a5612700 1 --2- 172.21.15.92:0/999413291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68a006ece0 0x7f68a0155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:49.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.721+0000 7f68a5612700 1 --2- 172.21.15.92:0/999413291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68a006f0b0 0x7f68a00adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:49.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.721+0000 7f68a5612700 1 -- 172.21.15.92:0/999413291 >> 172.21.15.92:0/999413291 conn(0x7f68a00682f0 msgr2=0x7f68a00ce150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:49.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.721+0000 7f68a5612700 1 -- 172.21.15.92:0/999413291 shutdown_connections 2023-10-06T10:44:49.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.721+0000 7f68a5612700 1 -- 172.21.15.92:0/999413291 wait complete. 2023-10-06T10:44:49.725 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-10-06T10:44:49.737 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T10:44:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.858+0000 7f9a94ea6700 1 Processor -- start 2023-10-06T10:44:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.858+0000 7f9a94ea6700 1 -- start start 2023-10-06T10:44:49.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.859+0000 7f9a94ea6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9a901598a0 0x7f9a900ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:49.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.859+0000 7f9a94ea6700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9a900cbc70 0x7f9a900ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:49.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.859+0000 7f9a94ea6700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a900cf5d0 con 0x7f9a901594d0 2023-10-06T10:44:49.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.859+0000 7f9a94ea6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a900cf8b0 con 0x7f9a901598a0 2023-10-06T10:44:49.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.859+0000 7f9a94ea6700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a900cfb90 con 0x7f9a900cbc70 2023-10-06T10:44:49.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.859+0000 7f9a8dd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9a901598a0 0x7f9a900ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:49.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.859+0000 7f9a8dd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9a901598a0 0x7f9a900ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50848/0 (socket says 172.21.15.92:50848) 2023-10-06T10:44:49.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.859+0000 7f9a8dd9b700 1 -- 172.21.15.92:0/872676798 learned_addr learned my addr 172.21.15.92:0/872676798 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:49.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.860+0000 7f9a8ed9d700 1 --2- 172.21.15.92:0/872676798 >> v2:172.21.15.124:3300/0 conn(0x7f9a900cbc70 0x7f9a900ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:49.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.860+0000 7f9a8dd9b700 1 -- 172.21.15.92:0/872676798 >> v2:172.21.15.124:3300/0 conn(0x7f9a900cbc70 msgr2=0x7f9a900ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:49.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.860+0000 7f9a8dd9b700 1 --2- 172.21.15.92:0/872676798 >> v2:172.21.15.124:3300/0 conn(0x7f9a900cbc70 0x7f9a900ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:49.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.860+0000 7f9a8dd9b700 1 -- 172.21.15.92:0/872676798 >> v1:172.21.15.124:6789/0 conn(0x7f9a901594d0 legacy=0x7f9a900cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:49.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.860+0000 7f9a8dd9b700 1 -- 172.21.15.92:0/872676798 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a900cfe70 con 0x7f9a901598a0 2023-10-06T10:44:49.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.860+0000 7f9a8dd9b700 1 --2- 172.21.15.92:0/872676798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a901598a0 0x7f9a900ce7e0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f9a9006f950 tx=0x7f9a84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f319fe0c9287a09c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:49.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.860+0000 7f9a8d59a700 1 -- 172.21.15.92:0/872676798 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a84009e70 con 0x7f9a901598a0 2023-10-06T10:44:49.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.860+0000 7f9a8d59a700 1 -- 172.21.15.92:0/872676798 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8400b860 con 0x7f9a901598a0 2023-10-06T10:44:49.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.860+0000 7f9a8d59a700 1 -- 172.21.15.92:0/872676798 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8400bc00 con 0x7f9a901598a0 2023-10-06T10:44:49.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.860+0000 7f9a94ea6700 1 -- 172.21.15.92:0/872676798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a901598a0 msgr2=0x7f9a900ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:49.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.860+0000 7f9a94ea6700 1 --2- 172.21.15.92:0/872676798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a901598a0 0x7f9a900ce7e0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f9a9006f950 tx=0x7f9a84009250 comp rx=0 tx=0).stop 2023-10-06T10:44:49.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.861+0000 7f9a94ea6700 1 -- 172.21.15.92:0/872676798 shutdown_connections 2023-10-06T10:44:49.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.861+0000 7f9a94ea6700 1 --2- 172.21.15.92:0/872676798 >> v2:172.21.15.124:3300/0 conn(0x7f9a900cbc70 0x7f9a900ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:49.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.861+0000 7f9a94ea6700 1 --2- 172.21.15.92:0/872676798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a901598a0 0x7f9a900ce7e0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:49.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.861+0000 7f9a94ea6700 1 -- 172.21.15.92:0/872676798 >> 172.21.15.92:0/872676798 conn(0x7f9a90089e50 msgr2=0x7f9a90099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:49.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.861+0000 7f9a94ea6700 1 -- 172.21.15.92:0/872676798 shutdown_connections 2023-10-06T10:44:49.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.861+0000 7f9a94ea6700 1 -- 172.21.15.92:0/872676798 wait complete. 2023-10-06T10:44:49.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.862+0000 7f9a94ea6700 1 Processor -- start 2023-10-06T10:44:49.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.862+0000 7f9a94ea6700 1 -- start start 2023-10-06T10:44:49.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.862+0000 7f9a94ea6700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a900cbc70 0x7f9a90169820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:49.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.862+0000 7f9a94ea6700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a901594d0 0x7f9a9014f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:49.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.862+0000 7f9a94ea6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a901598a0 0x7f9a901557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:49.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.862+0000 7f9a94ea6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a90159ca0 con 0x7f9a901598a0 2023-10-06T10:44:49.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.862+0000 7f9a94ea6700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9014e5b0 con 0x7f9a900cbc70 2023-10-06T10:44:49.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.862+0000 7f9a94ea6700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a90159eb0 con 0x7f9a901594d0 2023-10-06T10:44:49.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.862+0000 7f9a8ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a901598a0 0x7f9a901557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:49.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.863+0000 7f9a8ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a901598a0 0x7f9a901557c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50860/0 (socket says 172.21.15.92:50860) 2023-10-06T10:44:49.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.863+0000 7f9a8ed9d700 1 -- 172.21.15.92:0/696798389 learned_addr learned my addr 172.21.15.92:0/696798389 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:49.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.863+0000 7f9a8e59c700 1 --2- 172.21.15.92:0/696798389 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a900cbc70 0x7f9a90169820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:49.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.863+0000 7f9a8dd9b700 1 --2- 172.21.15.92:0/696798389 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a901594d0 0x7f9a9014f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:49.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.863+0000 7f9a8ed9d700 1 -- 172.21.15.92:0/696798389 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a901594d0 msgr2=0x7f9a9014f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:49.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.863+0000 7f9a8ed9d700 1 --2- 172.21.15.92:0/696798389 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a901594d0 0x7f9a9014f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:49.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.863+0000 7f9a8ed9d700 1 -- 172.21.15.92:0/696798389 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a900cbc70 msgr2=0x7f9a90169820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:49.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.863+0000 7f9a8ed9d700 1 --2- 172.21.15.92:0/696798389 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a900cbc70 0x7f9a90169820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:49.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.863+0000 7f9a8ed9d700 1 -- 172.21.15.92:0/696798389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a840094f0 con 0x7f9a901598a0 2023-10-06T10:44:49.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.863+0000 7f9a8ed9d700 1 --2- 172.21.15.92:0/696798389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a901598a0 0x7f9a901557c0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800ea20 tx=0x7f9a880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:49.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.863+0000 7f9a7e7fc700 1 -- 172.21.15.92:0/696798389 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8800ee60 con 0x7f9a901598a0 2023-10-06T10:44:49.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.863+0000 7f9a7e7fc700 1 -- 172.21.15.92:0/696798389 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8800c7f0 con 0x7f9a901598a0 2023-10-06T10:44:49.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.863+0000 7f9a7e7fc700 1 -- 172.21.15.92:0/696798389 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8800cbb0 con 0x7f9a901598a0 2023-10-06T10:44:49.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.863+0000 7f9a94ea6700 1 -- 172.21.15.92:0/696798389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a9015a190 con 0x7f9a901598a0 2023-10-06T10:44:49.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.863+0000 7f9a94ea6700 1 -- 172.21.15.92:0/696798389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9015a750 con 0x7f9a901598a0 2023-10-06T10:44:49.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.864+0000 7f9a94ea6700 1 -- 172.21.15.92:0/696798389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a90064a90 con 0x7f9a901598a0 2023-10-06T10:44:49.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.864+0000 7f9a7e7fc700 1 -- 172.21.15.92:0/696798389 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9a88013470 con 0x7f9a901598a0 2023-10-06T10:44:49.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.865+0000 7f9a7e7fc700 1 --2- 172.21.15.92:0/696798389 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a70066f30 0x7f9a700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:49.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.865+0000 7f9a7e7fc700 1 -- 172.21.15.92:0/696798389 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5896+0+0 (secure 0 0 0) 0x7f9a880847f0 con 0x7f9a901598a0 2023-10-06T10:44:49.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.865+0000 7f9a8e59c700 1 --2- 172.21.15.92:0/696798389 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a70066f30 0x7f9a700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:49.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.865+0000 7f9a8e59c700 1 --2- 172.21.15.92:0/696798389 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a70066f30 0x7f9a700693e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9a80007fd0 tx=0x7f9a800016f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:49.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:49.869+0000 7f9a7e7fc700 1 -- 172.21.15.92:0/696798389 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a880519a0 con 0x7f9a901598a0 2023-10-06T10:44:50.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.053+0000 7f9a94ea6700 1 -- 172.21.15.92:0/696798389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9a900cf260 con 0x7f9a901598a0 2023-10-06T10:44:50.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.723+0000 7f9a7e7fc700 1 -- 172.21.15.92:0/696798389 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v100) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9a88054fc0 con 0x7f9a901598a0 2023-10-06T10:44:50.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.728+0000 7f9a94ea6700 1 -- 172.21.15.92:0/696798389 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a70066f30 msgr2=0x7f9a700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:50.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.728+0000 7f9a94ea6700 1 --2- 172.21.15.92:0/696798389 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a70066f30 0x7f9a700693e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9a80007fd0 tx=0x7f9a800016f0 comp rx=0 tx=0).stop 2023-10-06T10:44:50.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.729+0000 7f9a94ea6700 1 -- 172.21.15.92:0/696798389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a901598a0 msgr2=0x7f9a901557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:50.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.729+0000 7f9a94ea6700 1 --2- 172.21.15.92:0/696798389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a901598a0 0x7f9a901557c0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800ea20 tx=0x7f9a880091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:50.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.729+0000 7f9a94ea6700 1 -- 172.21.15.92:0/696798389 shutdown_connections 2023-10-06T10:44:50.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.729+0000 7f9a94ea6700 1 --2- 172.21.15.92:0/696798389 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9a70066f30 0x7f9a700693e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:50.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.729+0000 7f9a94ea6700 1 --2- 172.21.15.92:0/696798389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a901598a0 0x7f9a901557c0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:50.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.729+0000 7f9a94ea6700 1 --2- 172.21.15.92:0/696798389 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a900cbc70 0x7f9a90169820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:50.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.729+0000 7f9a94ea6700 1 --2- 172.21.15.92:0/696798389 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a901594d0 0x7f9a9014f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:50.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.729+0000 7f9a94ea6700 1 -- 172.21.15.92:0/696798389 >> 172.21.15.92:0/696798389 conn(0x7f9a90089e50 msgr2=0x7f9a90099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:50.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.732+0000 7f9a94ea6700 1 -- 172.21.15.92:0/696798389 shutdown_connections 2023-10-06T10:44:50.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.733+0000 7f9a94ea6700 1 -- 172.21.15.92:0/696798389 wait complete. 2023-10-06T10:44:50.738 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-10-06T10:44:50.754 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T10:44:50.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.869+0000 7fbafef3f700 1 Processor -- start 2023-10-06T10:44:50.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.869+0000 7fbafef3f700 1 -- start start 2023-10-06T10:44:50.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.870+0000 7fbafef3f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbaf81598a0 0x7fbaf80ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:50.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.870+0000 7fbafef3f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaf80cf5d0 con 0x7fbaf81594d0 2023-10-06T10:44:50.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.870+0000 7fbafef3f700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaf80cf8b0 con 0x7fbaf80cbc70 2023-10-06T10:44:50.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.870+0000 7fbafef3f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaf80cfb90 con 0x7fbaf81598a0 2023-10-06T10:44:50.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.870+0000 7fbaf7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbaf81598a0 0x7fbaf80ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:50.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.870+0000 7fbaf7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbaf81598a0 0x7fbaf80ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50866/0 (socket says 172.21.15.92:50866) 2023-10-06T10:44:50.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.870+0000 7fbafccdb700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbaf81594d0 0x7fbaf80cb560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41672/0 (socket says 172.21.15.92:41672) 2023-10-06T10:44:50.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.870+0000 7fbaf7fff700 1 -- 172.21.15.92:0/3521883621 learned_addr learned my addr 172.21.15.92:0/3521883621 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:50.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.870+0000 7fbaf7fff700 1 -- 172.21.15.92:0/3521883621 >> v1:172.21.15.124:6789/0 conn(0x7fbaf80cbc70 legacy=0x7fbaf80ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:50.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.870+0000 7fbaf7fff700 1 -- 172.21.15.92:0/3521883621 >> v1:172.21.15.92:6789/0 conn(0x7fbaf81594d0 legacy=0x7fbaf80cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:50.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.870+0000 7fbaf7fff700 1 -- 172.21.15.92:0/3521883621 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf80cfe70 con 0x7fbaf81598a0 2023-10-06T10:44:50.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.870+0000 7fbaf77fe700 1 -- 172.21.15.92:0/3521883621 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1164838373 0 0) 0x7fbaf80cf5d0 con 0x7fbaf81594d0 2023-10-06T10:44:50.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.871+0000 7fbaf7fff700 1 --2- 172.21.15.92:0/3521883621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaf81598a0 0x7fbaf80ce7e0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fbae00099b0 tx=0x7fbae0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2ee7e3703adf09 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:50.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.871+0000 7fbaf77fe700 1 -- 172.21.15.92:0/3521883621 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae000b820 con 0x7fbaf81598a0 2023-10-06T10:44:50.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.871+0000 7fbaf77fe700 1 -- 172.21.15.92:0/3521883621 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae000b980 con 0x7fbaf81598a0 2023-10-06T10:44:50.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.871+0000 7fbaf77fe700 1 -- 172.21.15.92:0/3521883621 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae000bd20 con 0x7fbaf81598a0 2023-10-06T10:44:50.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.871+0000 7fbafef3f700 1 -- 172.21.15.92:0/3521883621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaf81598a0 msgr2=0x7fbaf80ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:50.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.871+0000 7fbafef3f700 1 --2- 172.21.15.92:0/3521883621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaf81598a0 0x7fbaf80ce7e0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fbae00099b0 tx=0x7fbae0009250 comp rx=0 tx=0).stop 2023-10-06T10:44:50.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.871+0000 7fbafef3f700 1 -- 172.21.15.92:0/3521883621 shutdown_connections 2023-10-06T10:44:50.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.871+0000 7fbafef3f700 1 --2- 172.21.15.92:0/3521883621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaf81598a0 0x7fbaf80ce7e0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:50.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.871+0000 7fbafef3f700 1 -- 172.21.15.92:0/3521883621 >> 172.21.15.92:0/3521883621 conn(0x7fbaf8089e50 msgr2=0x7fbaf8099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:50.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.871+0000 7fbafef3f700 1 -- 172.21.15.92:0/3521883621 shutdown_connections 2023-10-06T10:44:50.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.871+0000 7fbafef3f700 1 -- 172.21.15.92:0/3521883621 wait complete. 2023-10-06T10:44:50.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.872+0000 7fbafef3f700 1 Processor -- start 2023-10-06T10:44:50.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.873+0000 7fbafef3f700 1 -- start start 2023-10-06T10:44:50.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.873+0000 7fbafef3f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbaf80cbc70 0x7fbaf80b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:50.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.873+0000 7fbafef3f700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbaf81594d0 0x7fbaf8169e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:50.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.873+0000 7fbafef3f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaf81598a0 0x7fbaf816a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:50.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.873+0000 7fbafef3f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf81498a0 con 0x7fbaf81598a0 2023-10-06T10:44:50.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.873+0000 7fbafef3f700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf816d4c0 con 0x7fbaf81594d0 2023-10-06T10:44:50.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.873+0000 7fbafef3f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf816a9f0 con 0x7fbaf80cbc70 2023-10-06T10:44:50.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.873+0000 7fbafd4dc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaf81598a0 0x7fbaf816a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:50.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.873+0000 7fbafd4dc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaf81598a0 0x7fbaf816a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50872/0 (socket says 172.21.15.92:50872) 2023-10-06T10:44:50.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.873+0000 7fbafd4dc700 1 -- 172.21.15.92:0/2435832112 learned_addr learned my addr 172.21.15.92:0/2435832112 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:50.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.873+0000 7fbaf7fff700 1 --2- 172.21.15.92:0/2435832112 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbaf81594d0 0x7fbaf8169e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:50.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.873+0000 7fbafccdb700 1 --2- 172.21.15.92:0/2435832112 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbaf80cbc70 0x7fbaf80b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:50.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.874+0000 7fbafd4dc700 1 -- 172.21.15.92:0/2435832112 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbaf80cbc70 msgr2=0x7fbaf80b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:50.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.874+0000 7fbafd4dc700 1 --2- 172.21.15.92:0/2435832112 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbaf80cbc70 0x7fbaf80b4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:50.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.874+0000 7fbafd4dc700 1 -- 172.21.15.92:0/2435832112 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbaf81594d0 msgr2=0x7fbaf8169e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:50.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.874+0000 7fbafd4dc700 1 --2- 172.21.15.92:0/2435832112 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbaf81594d0 0x7fbaf8169e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:50.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.874+0000 7fbafd4dc700 1 -- 172.21.15.92:0/2435832112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae00094f0 con 0x7fbaf81598a0 2023-10-06T10:44:50.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.874+0000 7fbafd4dc700 1 --2- 172.21.15.92:0/2435832112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaf81598a0 0x7fbaf816a3c0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000c4f0 tx=0x7fbaf0007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:50.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.874+0000 7fbaf4ff9700 1 -- 172.21.15.92:0/2435832112 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf000cea0 con 0x7fbaf81598a0 2023-10-06T10:44:50.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.874+0000 7fbaf4ff9700 1 -- 172.21.15.92:0/2435832112 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf0011590 con 0x7fbaf81598a0 2023-10-06T10:44:50.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.874+0000 7fbaf4ff9700 1 -- 172.21.15.92:0/2435832112 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0011950 con 0x7fbaf81598a0 2023-10-06T10:44:50.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.874+0000 7fbafef3f700 1 -- 172.21.15.92:0/2435832112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf816acd0 con 0x7fbaf81598a0 2023-10-06T10:44:50.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.874+0000 7fbafef3f700 1 -- 172.21.15.92:0/2435832112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf816da60 con 0x7fbaf81598a0 2023-10-06T10:44:50.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.875+0000 7fbafef3f700 1 -- 172.21.15.92:0/2435832112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaf8064a90 con 0x7fbaf81598a0 2023-10-06T10:44:50.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.876+0000 7fbaf4ff9700 1 -- 172.21.15.92:0/2435832112 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbaf001c460 con 0x7fbaf81598a0 2023-10-06T10:44:50.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.876+0000 7fbaf4ff9700 1 --2- 172.21.15.92:0/2435832112 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbad8066f30 0x7fbad80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:50.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.876+0000 7fbaf4ff9700 1 -- 172.21.15.92:0/2435832112 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6278+0+0 (secure 0 0 0) 0x7fbaf0084310 con 0x7fbaf81598a0 2023-10-06T10:44:50.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.876+0000 7fbafccdb700 1 --2- 172.21.15.92:0/2435832112 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbad8066f30 0x7fbad80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:50.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.876+0000 7fbafccdb700 1 --2- 172.21.15.92:0/2435832112 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbad8066f30 0x7fbad80693e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbaf815da50 tx=0x7fbae8001b00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:50.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:50.880+0000 7fbaf4ff9700 1 -- 172.21.15.92:0/2435832112 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbaf00512a0 con 0x7fbaf81598a0 2023-10-06T10:44:51.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.104+0000 7fbafef3f700 1 -- 172.21.15.92:0/2435832112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbaf80c66a0 con 0x7fbaf81598a0 2023-10-06T10:44:51.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.726+0000 7fbaf4ff9700 1 -- 172.21.15.92:0/2435832112 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v96) v1 ==== 174+0+0 (secure 0 0 0) 0x7fbaf00548c0 con 0x7fbaf81598a0 2023-10-06T10:44:51.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.730+0000 7fbafef3f700 1 -- 172.21.15.92:0/2435832112 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbad8066f30 msgr2=0x7fbad80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:51.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.730+0000 7fbafef3f700 1 --2- 172.21.15.92:0/2435832112 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbad8066f30 0x7fbad80693e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbaf815da50 tx=0x7fbae8001b00 comp rx=0 tx=0).stop 2023-10-06T10:44:51.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.730+0000 7fbafef3f700 1 -- 172.21.15.92:0/2435832112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaf81598a0 msgr2=0x7fbaf816a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:51.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.730+0000 7fbafef3f700 1 --2- 172.21.15.92:0/2435832112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaf81598a0 0x7fbaf816a3c0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000c4f0 tx=0x7fbaf0007f00 comp rx=0 tx=0).stop 2023-10-06T10:44:51.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.730+0000 7fbafef3f700 1 -- 172.21.15.92:0/2435832112 shutdown_connections 2023-10-06T10:44:51.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.730+0000 7fbafef3f700 1 --2- 172.21.15.92:0/2435832112 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbad8066f30 0x7fbad80693e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:51.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.730+0000 7fbafef3f700 1 --2- 172.21.15.92:0/2435832112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaf81598a0 0x7fbaf816a3c0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:51.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.730+0000 7fbafef3f700 1 --2- 172.21.15.92:0/2435832112 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbaf80cbc70 0x7fbaf80b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:51.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.730+0000 7fbafef3f700 1 --2- 172.21.15.92:0/2435832112 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbaf81594d0 0x7fbaf8169e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:51.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.730+0000 7fbafef3f700 1 -- 172.21.15.92:0/2435832112 >> 172.21.15.92:0/2435832112 conn(0x7fbaf8089e50 msgr2=0x7fbaf815d010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:51.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.733+0000 7fbafef3f700 1 -- 172.21.15.92:0/2435832112 shutdown_connections 2023-10-06T10:44:51.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.733+0000 7fbafef3f700 1 -- 172.21.15.92:0/2435832112 wait complete. 2023-10-06T10:44:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 16 and data pool 17 2023-10-06T10:44:51.761 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:44:51.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.873+0000 7f3171df8700 1 Processor -- start 2023-10-06T10:44:51.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.873+0000 7f3171df8700 1 -- start start 2023-10-06T10:44:51.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.874+0000 7f3171df8700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f316c15a870 0x7f316c0ce980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:51.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.874+0000 7f3171df8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f316c15ac40 0x7f316c0c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:51.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.874+0000 7f3171df8700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316c0cdb40 con 0x7f316c0ceec0 2023-10-06T10:44:51.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.874+0000 7f3171df8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f316c0ccf30 con 0x7f316c15ac40 2023-10-06T10:44:51.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.874+0000 7f3171df8700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f316c0cd210 con 0x7f316c15a870 2023-10-06T10:44:51.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.874+0000 7f316affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f316c15ac40 0x7f316c0c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:51.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.874+0000 7f316affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f316c15ac40 0x7f316c0c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50882/0 (socket says 172.21.15.92:50882) 2023-10-06T10:44:51.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.874+0000 7f316affd700 1 -- 172.21.15.92:0/2786271749 learned_addr learned my addr 172.21.15.92:0/2786271749 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:51.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.875+0000 7f316b7fe700 1 --2- 172.21.15.92:0/2786271749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f316c15a870 0x7f316c0ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:51.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.875+0000 7f316affd700 1 -- 172.21.15.92:0/2786271749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f316c15a870 msgr2=0x7f316c0ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:51.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.875+0000 7f316affd700 1 --2- 172.21.15.92:0/2786271749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f316c15a870 0x7f316c0ce980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:51.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.875+0000 7f316affd700 1 -- 172.21.15.92:0/2786271749 >> v1:172.21.15.124:6789/0 conn(0x7f316c0ceec0 legacy=0x7f316c0cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:51.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.875+0000 7f316affd700 1 -- 172.21.15.92:0/2786271749 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316c0cd4f0 con 0x7f316c15ac40 2023-10-06T10:44:51.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.875+0000 7f316affd700 1 --2- 172.21.15.92:0/2786271749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f316c15ac40 0x7f316c0c8810 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f31600099b0 tx=0x7f3160009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ceb43491f9af05 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:51.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.875+0000 7f316a7fc700 1 -- 172.21.15.92:0/2786271749 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316000b860 con 0x7f316c15ac40 2023-10-06T10:44:51.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.875+0000 7f316a7fc700 1 -- 172.21.15.92:0/2786271749 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316000b9c0 con 0x7f316c15ac40 2023-10-06T10:44:51.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.875+0000 7f316a7fc700 1 -- 172.21.15.92:0/2786271749 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316000bd60 con 0x7f316c15ac40 2023-10-06T10:44:51.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.876+0000 7f3171df8700 1 -- 172.21.15.92:0/2786271749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f316c15ac40 msgr2=0x7f316c0c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:51.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.876+0000 7f3171df8700 1 --2- 172.21.15.92:0/2786271749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f316c15ac40 0x7f316c0c8810 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f31600099b0 tx=0x7f3160009250 comp rx=0 tx=0).stop 2023-10-06T10:44:51.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.876+0000 7f3171df8700 1 -- 172.21.15.92:0/2786271749 shutdown_connections 2023-10-06T10:44:51.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.876+0000 7f3171df8700 1 --2- 172.21.15.92:0/2786271749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f316c15a870 0x7f316c0ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:51.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.876+0000 7f3171df8700 1 --2- 172.21.15.92:0/2786271749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f316c15ac40 0x7f316c0c8810 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:51.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.876+0000 7f3171df8700 1 -- 172.21.15.92:0/2786271749 >> 172.21.15.92:0/2786271749 conn(0x7f316c067d90 msgr2=0x7f316c0ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:51.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.876+0000 7f3171df8700 1 -- 172.21.15.92:0/2786271749 shutdown_connections 2023-10-06T10:44:51.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.876+0000 7f3171df8700 1 -- 172.21.15.92:0/2786271749 wait complete. 2023-10-06T10:44:51.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.877+0000 7f3171df8700 1 Processor -- start 2023-10-06T10:44:51.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.877+0000 7f3171df8700 1 -- start start 2023-10-06T10:44:51.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f3171df8700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f316c0ceec0 0x7f316c0ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:51.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f3171df8700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f316c15a870 0x7f316c156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:51.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f3171df8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f316c15ac40 0x7f316c0af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:51.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f3171df8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c0b1150 con 0x7f316c15ac40 2023-10-06T10:44:51.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f3171df8700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c0ae0f0 con 0x7f316c15a870 2023-10-06T10:44:51.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f3171df8700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f316c0ae3d0 con 0x7f316c0ceec0 2023-10-06T10:44:51.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f316bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f316c15ac40 0x7f316c0af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:51.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f316bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f316c15ac40 0x7f316c0af1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50896/0 (socket says 172.21.15.92:50896) 2023-10-06T10:44:51.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f316bfff700 1 -- 172.21.15.92:0/4156586982 learned_addr learned my addr 172.21.15.92:0/4156586982 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:51.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f316affd700 1 --2- 172.21.15.92:0/4156586982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f316c15a870 0x7f316c156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:51.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f316b7fe700 1 --2- 172.21.15.92:0/4156586982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f316c0ceec0 0x7f316c0ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f316bfff700 1 -- 172.21.15.92:0/4156586982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f316c0ceec0 msgr2=0x7f316c0ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f316bfff700 1 --2- 172.21.15.92:0/4156586982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f316c0ceec0 0x7f316c0ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f316bfff700 1 -- 172.21.15.92:0/4156586982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f316c15a870 msgr2=0x7f316c156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f316bfff700 1 --2- 172.21.15.92:0/4156586982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f316c15a870 0x7f316c156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f316bfff700 1 -- 172.21.15.92:0/4156586982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31600094f0 con 0x7f316c15ac40 2023-10-06T10:44:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f316affd700 1 --2- 172.21.15.92:0/4156586982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f316c15a870 0x7f316c156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f316bfff700 1 --2- 172.21.15.92:0/4156586982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f316c15ac40 0x7f316c0af1f0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f316400c910 tx=0x7f31640016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f31537fe700 1 -- 172.21.15.92:0/4156586982 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164012630 con 0x7f316c15ac40 2023-10-06T10:44:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f31537fe700 1 -- 172.21.15.92:0/4156586982 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31640127b0 con 0x7f316c15ac40 2023-10-06T10:44:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f3171df8700 1 -- 172.21.15.92:0/4156586982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316c0ae6b0 con 0x7f316c15ac40 2023-10-06T10:44:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f31537fe700 1 -- 172.21.15.92:0/4156586982 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164012b70 con 0x7f316c15ac40 2023-10-06T10:44:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.878+0000 7f3171df8700 1 -- 172.21.15.92:0/4156586982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316c0ae990 con 0x7f316c15ac40 2023-10-06T10:44:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.879+0000 7f3171df8700 1 -- 172.21.15.92:0/4156586982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f316c0b12c0 con 0x7f316c15ac40 2023-10-06T10:44:51.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.880+0000 7f31537fe700 1 -- 172.21.15.92:0/4156586982 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f316400a3b0 con 0x7f316c15ac40 2023-10-06T10:44:51.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.880+0000 7f31537fe700 1 --2- 172.21.15.92:0/4156586982 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3158066f30 0x7f31580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:51.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.880+0000 7f31537fe700 1 -- 172.21.15.92:0/4156586982 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3164084a10 con 0x7f316c15ac40 2023-10-06T10:44:51.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.880+0000 7f316b7fe700 1 --2- 172.21.15.92:0/4156586982 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3158066f30 0x7f31580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:51.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.880+0000 7f316b7fe700 1 --2- 172.21.15.92:0/4156586982 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3158066f30 0x7f31580693e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f315c006aa0 tx=0x7f315c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:51.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:51.884+0000 7f31537fe700 1 -- 172.21.15.92:0/4156586982 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f316c0b12c0 con 0x7f316c15ac40 2023-10-06T10:44:52.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.071+0000 7f3171df8700 1 -- 172.21.15.92:0/4156586982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f316c0b1290 con 0x7f316c15ac40 2023-10-06T10:44:52.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.072+0000 7f31537fe700 1 -- 172.21.15.92:0/4156586982 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+17098 (secure 0 0 0) 0x7f3164055470 con 0x7f316c15ac40 2023-10-06T10:44:52.074 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:52.074 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":101,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:44:51.106123+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:44:48.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-06T10:44:50.072416+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:44:52.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.074+0000 7f3171df8700 1 -- 172.21.15.92:0/4156586982 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3158066f30 msgr2=0x7f31580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:52.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.074+0000 7f3171df8700 1 --2- 172.21.15.92:0/4156586982 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3158066f30 0x7f31580693e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f315c006aa0 tx=0x7f315c008040 comp rx=0 tx=0).stop 2023-10-06T10:44:52.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.074+0000 7f3171df8700 1 -- 172.21.15.92:0/4156586982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f316c15ac40 msgr2=0x7f316c0af1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:52.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.074+0000 7f3171df8700 1 --2- 172.21.15.92:0/4156586982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f316c15ac40 0x7f316c0af1f0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f316400c910 tx=0x7f31640016f0 comp rx=0 tx=0).stop 2023-10-06T10:44:52.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.074+0000 7f3171df8700 1 -- 172.21.15.92:0/4156586982 shutdown_connections 2023-10-06T10:44:52.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.074+0000 7f3171df8700 1 --2- 172.21.15.92:0/4156586982 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3158066f30 0x7f31580693e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:52.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.074+0000 7f3171df8700 1 --2- 172.21.15.92:0/4156586982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f316c15ac40 0x7f316c0af1f0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:52.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.074+0000 7f3171df8700 1 --2- 172.21.15.92:0/4156586982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f316c0ceec0 0x7f316c0ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:52.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.074+0000 7f3171df8700 1 --2- 172.21.15.92:0/4156586982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f316c15a870 0x7f316c156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:52.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.074+0000 7f3171df8700 1 -- 172.21.15.92:0/4156586982 >> 172.21.15.92:0/4156586982 conn(0x7f316c067d90 msgr2=0x7f316c06ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:52.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.074+0000 7f3171df8700 1 -- 172.21.15.92:0/4156586982 shutdown_connections 2023-10-06T10:44:52.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.074+0000 7f3171df8700 1 -- 172.21.15.92:0/4156586982 wait complete. 2023-10-06T10:44:52.088 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:44:52.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.201+0000 7fe204c69700 1 Processor -- start 2023-10-06T10:44:52.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.201+0000 7fe204c69700 1 -- start start 2023-10-06T10:44:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.202+0000 7fe204c69700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe2000b0760 0x7fe2000ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.202+0000 7fe204c69700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2000b0c60 con 0x7fe2000b49a0 2023-10-06T10:44:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.202+0000 7fe204c69700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2000b0f40 con 0x7fe200159560 2023-10-06T10:44:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.202+0000 7fe204c69700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2000ab350 con 0x7fe2000b0760 2023-10-06T10:44:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.202+0000 7fe1fed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe2000b0760 0x7fe2000ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.202+0000 7fe1fdd9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe2000b49a0 0x7fe2000aac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41674/0 (socket says 172.21.15.92:41674) 2023-10-06T10:44:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.202+0000 7fe1fed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe2000b0760 0x7fe2000ccf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50902/0 (socket says 172.21.15.92:50902) 2023-10-06T10:44:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.202+0000 7fe1fdd9b700 1 -- 172.21.15.92:0/3155319107 learned_addr learned my addr 172.21.15.92:0/3155319107 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.202+0000 7fe1fed9d700 1 -- 172.21.15.92:0/3155319107 >> v1:172.21.15.124:6789/0 conn(0x7fe200159560 legacy=0x7fe2000aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.202+0000 7fe1fed9d700 1 -- 172.21.15.92:0/3155319107 >> v1:172.21.15.92:6789/0 conn(0x7fe2000b49a0 legacy=0x7fe2000aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.202+0000 7fe1fed9d700 1 -- 172.21.15.92:0/3155319107 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2000b1220 con 0x7fe2000b0760 2023-10-06T10:44:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.202+0000 7fe1fd59a700 1 -- 172.21.15.92:0/3155319107 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1766334679 0 0) 0x7fe2000b0c60 con 0x7fe2000b49a0 2023-10-06T10:44:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.202+0000 7fe1fed9d700 1 --2- 172.21.15.92:0/3155319107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2000b0760 0x7fe2000ccf90 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009e90 tx=0x7fe1f0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eacb52afea3299c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.202+0000 7fe1fd59a700 1 -- 172.21.15.92:0/3155319107 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f000e070 con 0x7fe2000b0760 2023-10-06T10:44:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.202+0000 7fe1fd59a700 1 -- 172.21.15.92:0/3155319107 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f000bbb0 con 0x7fe2000b0760 2023-10-06T10:44:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.202+0000 7fe1fd59a700 1 -- 172.21.15.92:0/3155319107 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0012430 con 0x7fe2000b0760 2023-10-06T10:44:52.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.203+0000 7fe204c69700 1 -- 172.21.15.92:0/3155319107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2000b0760 msgr2=0x7fe2000ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:52.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.203+0000 7fe204c69700 1 --2- 172.21.15.92:0/3155319107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2000b0760 0x7fe2000ccf90 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009e90 tx=0x7fe1f0009780 comp rx=0 tx=0).stop 2023-10-06T10:44:52.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.203+0000 7fe204c69700 1 -- 172.21.15.92:0/3155319107 shutdown_connections 2023-10-06T10:44:52.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.203+0000 7fe204c69700 1 --2- 172.21.15.92:0/3155319107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2000b0760 0x7fe2000ccf90 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:52.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.203+0000 7fe204c69700 1 -- 172.21.15.92:0/3155319107 >> 172.21.15.92:0/3155319107 conn(0x7fe2000997f0 msgr2=0x7fe20006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:52.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.203+0000 7fe204c69700 1 -- 172.21.15.92:0/3155319107 shutdown_connections 2023-10-06T10:44:52.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.203+0000 7fe204c69700 1 -- 172.21.15.92:0/3155319107 wait complete. 2023-10-06T10:44:52.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.204+0000 7fe204c69700 1 Processor -- start 2023-10-06T10:44:52.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.204+0000 7fe204c69700 1 -- start start 2023-10-06T10:44:52.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.205+0000 7fe204c69700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2000b0760 0x7fe20015cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:52.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.205+0000 7fe204c69700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2000b49a0 0x7fe200157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:52.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.205+0000 7fe204c69700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe200159560 0x7fe20015c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:52.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.205+0000 7fe204c69700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe200158010 con 0x7fe2000b49a0 2023-10-06T10:44:52.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.205+0000 7fe204c69700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20009aa00 con 0x7fe200159560 2023-10-06T10:44:52.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.205+0000 7fe204c69700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20009db70 con 0x7fe2000b0760 2023-10-06T10:44:52.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.205+0000 7fe1fdd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2000b49a0 0x7fe200157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:52.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.205+0000 7fe1fdd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2000b49a0 0x7fe200157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50914/0 (socket says 172.21.15.92:50914) 2023-10-06T10:44:52.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.205+0000 7fe1fdd9b700 1 -- 172.21.15.92:0/2493703065 learned_addr learned my addr 172.21.15.92:0/2493703065 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:52.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.205+0000 7fe1fed9d700 1 --2- 172.21.15.92:0/2493703065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe200159560 0x7fe20015c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:52.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.205+0000 7fe1fe59c700 1 --2- 172.21.15.92:0/2493703065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2000b0760 0x7fe20015cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:52.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.205+0000 7fe1fdd9b700 1 -- 172.21.15.92:0/2493703065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2000b0760 msgr2=0x7fe20015cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:52.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.205+0000 7fe1fdd9b700 1 --2- 172.21.15.92:0/2493703065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2000b0760 0x7fe20015cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:52.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.205+0000 7fe1fdd9b700 1 -- 172.21.15.92:0/2493703065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe200159560 msgr2=0x7fe20015c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:52.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.205+0000 7fe1fdd9b700 1 --2- 172.21.15.92:0/2493703065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe200159560 0x7fe20015c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:52.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.205+0000 7fe1fdd9b700 1 -- 172.21.15.92:0/2493703065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f0009a20 con 0x7fe2000b49a0 2023-10-06T10:44:52.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.205+0000 7fe1fe59c700 1 --2- 172.21.15.92:0/2493703065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2000b0760 0x7fe20015cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:52.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.205+0000 7fe1fed9d700 1 --2- 172.21.15.92:0/2493703065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe200159560 0x7fe20015c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:52.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.205+0000 7fe1fdd9b700 1 --2- 172.21.15.92:0/2493703065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2000b49a0 0x7fe200157ad0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400c810 tx=0x7fe1f4007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:52.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.206+0000 7fe1ee7fc700 1 -- 172.21.15.92:0/2493703065 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4005c00 con 0x7fe2000b49a0 2023-10-06T10:44:52.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.206+0000 7fe1ee7fc700 1 -- 172.21.15.92:0/2493703065 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f4005d60 con 0x7fe2000b49a0 2023-10-06T10:44:52.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.206+0000 7fe1ee7fc700 1 -- 172.21.15.92:0/2493703065 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f40072e0 con 0x7fe2000b49a0 2023-10-06T10:44:52.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.206+0000 7fe204c69700 1 -- 172.21.15.92:0/2493703065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20009ac70 con 0x7fe2000b49a0 2023-10-06T10:44:52.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.206+0000 7fe204c69700 1 -- 172.21.15.92:0/2493703065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20015c600 con 0x7fe2000b49a0 2023-10-06T10:44:52.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.206+0000 7fe204c69700 1 -- 172.21.15.92:0/2493703065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe20009ae60 con 0x7fe2000b49a0 2023-10-06T10:44:52.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.207+0000 7fe1ee7fc700 1 -- 172.21.15.92:0/2493703065 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe1f401b3b0 con 0x7fe2000b49a0 2023-10-06T10:44:52.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.207+0000 7fe1ee7fc700 1 --2- 172.21.15.92:0/2493703065 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe1e0066f30 0x7fe1e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:52.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.207+0000 7fe1ee7fc700 1 -- 172.21.15.92:0/2493703065 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe1f4083c20 con 0x7fe2000b49a0 2023-10-06T10:44:52.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.207+0000 7fe1fe59c700 1 --2- 172.21.15.92:0/2493703065 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe1e0066f30 0x7fe1e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:52.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.208+0000 7fe1fe59c700 1 --2- 172.21.15.92:0/2493703065 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe1e0066f30 0x7fe1e00693e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800a920 tx=0x7fe1f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:52.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.211+0000 7fe1ee7fc700 1 -- 172.21.15.92:0/2493703065 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe1f404b5a0 con 0x7fe2000b49a0 2023-10-06T10:44:52.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.394+0000 7fe204c69700 1 -- 172.21.15.92:0/2493703065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe200064a90 con 0x7fe2000b49a0 2023-10-06T10:44:52.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.395+0000 7fe1ee7fc700 1 -- 172.21.15.92:0/2493703065 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+17098 (secure 0 0 0) 0x7fe1f4017090 con 0x7fe2000b49a0 2023-10-06T10:44:52.397 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:52.397 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":101,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:44:51.106123+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:44:48.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-06T10:44:50.072416+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:44:52.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.397+0000 7fe204c69700 1 -- 172.21.15.92:0/2493703065 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe1e0066f30 msgr2=0x7fe1e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:52.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.397+0000 7fe204c69700 1 --2- 172.21.15.92:0/2493703065 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe1e0066f30 0x7fe1e00693e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800a920 tx=0x7fe1f8008040 comp rx=0 tx=0).stop 2023-10-06T10:44:52.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.397+0000 7fe204c69700 1 -- 172.21.15.92:0/2493703065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2000b49a0 msgr2=0x7fe200157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:52.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.397+0000 7fe204c69700 1 --2- 172.21.15.92:0/2493703065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2000b49a0 0x7fe200157ad0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400c810 tx=0x7fe1f4007d90 comp rx=0 tx=0).stop 2023-10-06T10:44:52.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.397+0000 7fe204c69700 1 -- 172.21.15.92:0/2493703065 shutdown_connections 2023-10-06T10:44:52.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.397+0000 7fe204c69700 1 --2- 172.21.15.92:0/2493703065 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe1e0066f30 0x7fe1e00693e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:52.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.397+0000 7fe204c69700 1 --2- 172.21.15.92:0/2493703065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe200159560 0x7fe20015c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:52.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.397+0000 7fe204c69700 1 --2- 172.21.15.92:0/2493703065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2000b0760 0x7fe20015cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:52.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.397+0000 7fe204c69700 1 --2- 172.21.15.92:0/2493703065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2000b49a0 0x7fe200157ad0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:52.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.397+0000 7fe204c69700 1 -- 172.21.15.92:0/2493703065 >> 172.21.15.92:0/2493703065 conn(0x7fe2000997f0 msgr2=0x7fe2000b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:52.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.397+0000 7fe204c69700 1 -- 172.21.15.92:0/2493703065 shutdown_connections 2023-10-06T10:44:52.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.397+0000 7fe204c69700 1 -- 172.21.15.92:0/2493703065 wait complete. 2023-10-06T10:44:52.411 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T10:44:52.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.526+0000 7fce84207700 1 Processor -- start 2023-10-06T10:44:52.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.526+0000 7fce84207700 1 -- start start 2023-10-06T10:44:52.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.527+0000 7fce84207700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce7c157820 0x7fce7c0b1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:52.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.527+0000 7fce84207700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fce7c0c97a0 0x7fce7c0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:52.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.527+0000 7fce84207700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7c0c40e0 con 0x7fce7c157450 2023-10-06T10:44:52.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.527+0000 7fce84207700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7c0cf0a0 con 0x7fce7c157820 2023-10-06T10:44:52.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.527+0000 7fce84207700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7c0c34d0 con 0x7fce7c0c97a0 2023-10-06T10:44:52.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.527+0000 7fce817a2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce7c157820 0x7fce7c0b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:52.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.527+0000 7fce817a2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce7c157820 0x7fce7c0b1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50924/0 (socket says 172.21.15.92:50924) 2023-10-06T10:44:52.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.527+0000 7fce817a2700 1 -- 172.21.15.92:0/3388961234 learned_addr learned my addr 172.21.15.92:0/3388961234 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:52.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.527+0000 7fce827a4700 1 --2- 172.21.15.92:0/3388961234 >> v2:172.21.15.124:3300/0 conn(0x7fce7c0c97a0 0x7fce7c0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:52.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.528+0000 7fce817a2700 1 -- 172.21.15.92:0/3388961234 >> v2:172.21.15.124:3300/0 conn(0x7fce7c0c97a0 msgr2=0x7fce7c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:52.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.528+0000 7fce817a2700 1 --2- 172.21.15.92:0/3388961234 >> v2:172.21.15.124:3300/0 conn(0x7fce7c0c97a0 0x7fce7c0ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:52.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.528+0000 7fce817a2700 1 -- 172.21.15.92:0/3388961234 >> v1:172.21.15.124:6789/0 conn(0x7fce7c157450 legacy=0x7fce7c0c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:52.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.528+0000 7fce817a2700 1 -- 172.21.15.92:0/3388961234 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7c0c37b0 con 0x7fce7c157820 2023-10-06T10:44:52.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.528+0000 7fce817a2700 1 --2- 172.21.15.92:0/3388961234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c157820 0x7fce7c0b1110 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0cf440 tx=0x7fce6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3801e5b4a918c212 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:52.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.528+0000 7fce80fa1700 1 -- 172.21.15.92:0/3388961234 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c009e70 con 0x7fce7c157820 2023-10-06T10:44:52.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.528+0000 7fce80fa1700 1 -- 172.21.15.92:0/3388961234 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c00b860 con 0x7fce7c157820 2023-10-06T10:44:52.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.528+0000 7fce80fa1700 1 -- 172.21.15.92:0/3388961234 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c00bc00 con 0x7fce7c157820 2023-10-06T10:44:52.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.528+0000 7fce84207700 1 -- 172.21.15.92:0/3388961234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c157820 msgr2=0x7fce7c0b1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:52.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.528+0000 7fce84207700 1 --2- 172.21.15.92:0/3388961234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c157820 0x7fce7c0b1110 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0cf440 tx=0x7fce6c009250 comp rx=0 tx=0).stop 2023-10-06T10:44:52.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.529+0000 7fce84207700 1 -- 172.21.15.92:0/3388961234 shutdown_connections 2023-10-06T10:44:52.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.529+0000 7fce84207700 1 --2- 172.21.15.92:0/3388961234 >> v2:172.21.15.124:3300/0 conn(0x7fce7c0c97a0 0x7fce7c0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:52.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.529+0000 7fce84207700 1 --2- 172.21.15.92:0/3388961234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c157820 0x7fce7c0b1110 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:52.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.529+0000 7fce84207700 1 -- 172.21.15.92:0/3388961234 >> 172.21.15.92:0/3388961234 conn(0x7fce7c067d90 msgr2=0x7fce7c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:52.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.529+0000 7fce84207700 1 -- 172.21.15.92:0/3388961234 shutdown_connections 2023-10-06T10:44:52.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.529+0000 7fce84207700 1 -- 172.21.15.92:0/3388961234 wait complete. 2023-10-06T10:44:52.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.530+0000 7fce84207700 1 Processor -- start 2023-10-06T10:44:52.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.530+0000 7fce84207700 1 -- start start 2023-10-06T10:44:52.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.530+0000 7fce84207700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c0c97a0 0x7fce7c0b1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:52.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.530+0000 7fce84207700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c157450 0x7fce7c0c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:52.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.530+0000 7fce84207700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce7c157820 0x7fce7c069610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:52.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.530+0000 7fce84207700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c0c8320 con 0x7fce7c0c97a0 2023-10-06T10:44:52.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.530+0000 7fce84207700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c0af950 con 0x7fce7c157820 2023-10-06T10:44:52.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.530+0000 7fce84207700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c0c8040 con 0x7fce7c157450 2023-10-06T10:44:52.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.530+0000 7fce81fa3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c0c97a0 0x7fce7c0b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:52.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.531+0000 7fce81fa3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c0c97a0 0x7fce7c0b1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50932/0 (socket says 172.21.15.92:50932) 2023-10-06T10:44:52.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.531+0000 7fce81fa3700 1 -- 172.21.15.92:0/3938077292 learned_addr learned my addr 172.21.15.92:0/3938077292 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:52.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.531+0000 7fce827a4700 1 --2- 172.21.15.92:0/3938077292 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce7c157820 0x7fce7c069610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:52.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.531+0000 7fce817a2700 1 --2- 172.21.15.92:0/3938077292 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c157450 0x7fce7c0c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:52.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.531+0000 7fce81fa3700 1 -- 172.21.15.92:0/3938077292 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c157450 msgr2=0x7fce7c0c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:52.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.531+0000 7fce81fa3700 1 --2- 172.21.15.92:0/3938077292 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c157450 0x7fce7c0c7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:52.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.531+0000 7fce81fa3700 1 -- 172.21.15.92:0/3938077292 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce7c157820 msgr2=0x7fce7c069610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:52.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.531+0000 7fce81fa3700 1 --2- 172.21.15.92:0/3938077292 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce7c157820 0x7fce7c069610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:52.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.531+0000 7fce81fa3700 1 -- 172.21.15.92:0/3938077292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c0094f0 con 0x7fce7c0c97a0 2023-10-06T10:44:52.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.531+0000 7fce81fa3700 1 --2- 172.21.15.92:0/3938077292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c0c97a0 0x7fce7c0b1770 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fce7800cdb0 tx=0x7fce78001be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:52.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.531+0000 7fce71ffb700 1 -- 172.21.15.92:0/3938077292 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7800aac0 con 0x7fce7c0c97a0 2023-10-06T10:44:52.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.531+0000 7fce71ffb700 1 -- 172.21.15.92:0/3938077292 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7800ac20 con 0x7fce7c0c97a0 2023-10-06T10:44:52.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.531+0000 7fce71ffb700 1 -- 172.21.15.92:0/3938077292 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce78012420 con 0x7fce7c0c97a0 2023-10-06T10:44:52.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.531+0000 7fce84207700 1 -- 172.21.15.92:0/3938077292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7c0af300 con 0x7fce7c0c97a0 2023-10-06T10:44:52.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.531+0000 7fce84207700 1 -- 172.21.15.92:0/3938077292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7c0c7d50 con 0x7fce7c0c97a0 2023-10-06T10:44:52.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.532+0000 7fce84207700 1 -- 172.21.15.92:0/3938077292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7c064a90 con 0x7fce7c0c97a0 2023-10-06T10:44:52.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.533+0000 7fce71ffb700 1 -- 172.21.15.92:0/3938077292 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fce78012810 con 0x7fce7c0c97a0 2023-10-06T10:44:52.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.533+0000 7fce71ffb700 1 --2- 172.21.15.92:0/3938077292 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fce60066f30 0x7fce600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:52.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.533+0000 7fce71ffb700 1 -- 172.21.15.92:0/3938077292 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fce78084df0 con 0x7fce7c0c97a0 2023-10-06T10:44:52.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.533+0000 7fce817a2700 1 --2- 172.21.15.92:0/3938077292 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fce60066f30 0x7fce600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:52.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.533+0000 7fce817a2700 1 --2- 172.21.15.92:0/3938077292 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fce60066f30 0x7fce600693e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0cf440 tx=0x7fce6c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:52.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.537+0000 7fce71ffb700 1 -- 172.21.15.92:0/3938077292 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce78051dc0 con 0x7fce7c0c97a0 2023-10-06T10:44:52.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:52.767+0000 7fce84207700 1 -- 172.21.15.92:0/3938077292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fce7c04bb40 con 0x7fce7c0c97a0 2023-10-06T10:44:53.511 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:44:53.514 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:44:53.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.749+0000 7fce71ffb700 1 -- 172.21.15.92:0/3938077292 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v99) v1 ==== 118+0+0 (secure 0 0 0) 0x7fce780553e0 con 0x7fce7c0c97a0 2023-10-06T10:44:53.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.752+0000 7fce84207700 1 -- 172.21.15.92:0/3938077292 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fce60066f30 msgr2=0x7fce600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:53.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.752+0000 7fce84207700 1 --2- 172.21.15.92:0/3938077292 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fce60066f30 0x7fce600693e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0cf440 tx=0x7fce6c017000 comp rx=0 tx=0).stop 2023-10-06T10:44:53.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.752+0000 7fce84207700 1 -- 172.21.15.92:0/3938077292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c0c97a0 msgr2=0x7fce7c0b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:53.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.752+0000 7fce84207700 1 --2- 172.21.15.92:0/3938077292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c0c97a0 0x7fce7c0b1770 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fce7800cdb0 tx=0x7fce78001be0 comp rx=0 tx=0).stop 2023-10-06T10:44:53.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.752+0000 7fce84207700 1 -- 172.21.15.92:0/3938077292 shutdown_connections 2023-10-06T10:44:53.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.752+0000 7fce84207700 1 --2- 172.21.15.92:0/3938077292 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fce60066f30 0x7fce600693e0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:53.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.752+0000 7fce84207700 1 --2- 172.21.15.92:0/3938077292 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce7c157820 0x7fce7c069610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:53.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.752+0000 7fce84207700 1 --2- 172.21.15.92:0/3938077292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c0c97a0 0x7fce7c0b1770 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:53.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.752+0000 7fce84207700 1 --2- 172.21.15.92:0/3938077292 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c157450 0x7fce7c0c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:53.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.752+0000 7fce84207700 1 -- 172.21.15.92:0/3938077292 >> 172.21.15.92:0/3938077292 conn(0x7fce7c067d90 msgr2=0x7fce7c06b700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:53.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.752+0000 7fce84207700 1 -- 172.21.15.92:0/3938077292 shutdown_connections 2023-10-06T10:44:53.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.752+0000 7fce84207700 1 -- 172.21.15.92:0/3938077292 wait complete. 2023-10-06T10:44:53.768 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T10:44:53.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.887+0000 7f9e47fff700 1 Processor -- start 2023-10-06T10:44:53.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.887+0000 7f9e47fff700 1 -- start start 2023-10-06T10:44:53.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.888+0000 7f9e47fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e480b2620 0x7f9e480cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:53.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.888+0000 7f9e47fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9e480cf1b0 0x7f9e480cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:53.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.888+0000 7f9e47fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e480cc970 con 0x7f9e480b29f0 2023-10-06T10:44:53.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.888+0000 7f9e47fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e480cbd60 con 0x7f9e480cf1b0 2023-10-06T10:44:53.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.888+0000 7f9e47fff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e480cc040 con 0x7f9e480b2620 2023-10-06T10:44:53.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.888+0000 7f9e477fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9e480cf1b0 0x7f9e480cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:53.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.888+0000 7f9e477fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9e480cf1b0 0x7f9e480cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33706/0 (socket says 172.21.15.92:33706) 2023-10-06T10:44:53.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.888+0000 7f9e467fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9e480b29f0 0x7f9e480ceaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:43898/0 (socket says 172.21.15.92:43898) 2023-10-06T10:44:53.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.888+0000 7f9e477fe700 1 -- 172.21.15.92:0/3656648511 learned_addr learned my addr 172.21.15.92:0/3656648511 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:53.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.888+0000 7f9e46ffd700 1 --2- 172.21.15.92:0/3656648511 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e480b2620 0x7f9e480cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:53.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.888+0000 7f9e477fe700 1 -- 172.21.15.92:0/3656648511 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e480b2620 msgr2=0x7f9e480cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:53.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.888+0000 7f9e477fe700 1 --2- 172.21.15.92:0/3656648511 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e480b2620 0x7f9e480cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:53.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.888+0000 7f9e477fe700 1 -- 172.21.15.92:0/3656648511 >> v1:172.21.15.92:6789/0 conn(0x7f9e480b29f0 legacy=0x7f9e480ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:53.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.889+0000 7f9e477fe700 1 -- 172.21.15.92:0/3656648511 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e480cc320 con 0x7f9e480cf1b0 2023-10-06T10:44:53.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.888+0000 7f9e45ffb700 1 -- 172.21.15.92:0/3656648511 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 995465404 0 0) 0x7f9e480cc970 con 0x7f9e480b29f0 2023-10-06T10:44:53.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.889+0000 7f9e477fe700 1 --2- 172.21.15.92:0/3656648511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e480cf1b0 0x7f9e480cd1f0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f9e400099b0 tx=0x7f9e40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f29a84c73312a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:53.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.889+0000 7f9e45ffb700 1 -- 172.21.15.92:0/3656648511 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4000b860 con 0x7f9e480cf1b0 2023-10-06T10:44:53.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.889+0000 7f9e45ffb700 1 -- 172.21.15.92:0/3656648511 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4000b9c0 con 0x7f9e480cf1b0 2023-10-06T10:44:53.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.889+0000 7f9e45ffb700 1 -- 172.21.15.92:0/3656648511 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4000bd60 con 0x7f9e480cf1b0 2023-10-06T10:44:53.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.889+0000 7f9e47fff700 1 -- 172.21.15.92:0/3656648511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e480cf1b0 msgr2=0x7f9e480cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:53.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.889+0000 7f9e47fff700 1 --2- 172.21.15.92:0/3656648511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e480cf1b0 0x7f9e480cd1f0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f9e400099b0 tx=0x7f9e40009250 comp rx=0 tx=0).stop 2023-10-06T10:44:53.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.889+0000 7f9e47fff700 1 -- 172.21.15.92:0/3656648511 shutdown_connections 2023-10-06T10:44:53.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.889+0000 7f9e47fff700 1 --2- 172.21.15.92:0/3656648511 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e480b2620 0x7f9e480cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:53.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.889+0000 7f9e47fff700 1 --2- 172.21.15.92:0/3656648511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e480cf1b0 0x7f9e480cd1f0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:53.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.889+0000 7f9e47fff700 1 -- 172.21.15.92:0/3656648511 >> 172.21.15.92:0/3656648511 conn(0x7f9e48099520 msgr2=0x7f9e4806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:53.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.889+0000 7f9e47fff700 1 -- 172.21.15.92:0/3656648511 shutdown_connections 2023-10-06T10:44:53.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.889+0000 7f9e47fff700 1 -- 172.21.15.92:0/3656648511 wait complete. 2023-10-06T10:44:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.891+0000 7f9e47fff700 1 Processor -- start 2023-10-06T10:44:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.891+0000 7f9e47fff700 1 -- start start 2023-10-06T10:44:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.891+0000 7f9e47fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e480b2620 0x7f9e4814cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.891+0000 7f9e47fff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e480b29f0 0x7f9e48150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.891+0000 7f9e47fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e480cf1b0 0x7f9e481509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.891+0000 7f9e47fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4814ec80 con 0x7f9e480cf1b0 2023-10-06T10:44:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.891+0000 7f9e47fff700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4814ffc0 con 0x7f9e480b29f0 2023-10-06T10:44:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.891+0000 7f9e47fff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4814ef20 con 0x7f9e480b2620 2023-10-06T10:44:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.891+0000 7f9e477fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e480cf1b0 0x7f9e481509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.891+0000 7f9e477fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e480cf1b0 0x7f9e481509a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33710/0 (socket says 172.21.15.92:33710) 2023-10-06T10:44:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.891+0000 7f9e477fe700 1 -- 172.21.15.92:0/613583794 learned_addr learned my addr 172.21.15.92:0/613583794 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.891+0000 7f9e46ffd700 1 --2- 172.21.15.92:0/613583794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e480b2620 0x7f9e4814cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.891+0000 7f9e467fc700 1 --2- 172.21.15.92:0/613583794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e480b29f0 0x7f9e48150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.892+0000 7f9e477fe700 1 -- 172.21.15.92:0/613583794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e480b2620 msgr2=0x7f9e4814cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.892+0000 7f9e477fe700 1 --2- 172.21.15.92:0/613583794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e480b2620 0x7f9e4814cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:53.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.892+0000 7f9e477fe700 1 -- 172.21.15.92:0/613583794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e480b29f0 msgr2=0x7f9e48150460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:53.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.892+0000 7f9e477fe700 1 --2- 172.21.15.92:0/613583794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e480b29f0 0x7f9e48150460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:53.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.892+0000 7f9e477fe700 1 -- 172.21.15.92:0/613583794 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e400094f0 con 0x7f9e480cf1b0 2023-10-06T10:44:53.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.892+0000 7f9e477fe700 1 --2- 172.21.15.92:0/613583794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e480cf1b0 0x7f9e481509a0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009290 tx=0x7f9e40017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:53.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.892+0000 7f9e2effd700 1 -- 172.21.15.92:0/613583794 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4000b860 con 0x7f9e480cf1b0 2023-10-06T10:44:53.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.892+0000 7f9e2effd700 1 -- 172.21.15.92:0/613583794 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4001a660 con 0x7f9e480cf1b0 2023-10-06T10:44:53.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.892+0000 7f9e47fff700 1 -- 172.21.15.92:0/613583794 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4814f200 con 0x7f9e480cf1b0 2023-10-06T10:44:53.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.892+0000 7f9e2effd700 1 -- 172.21.15.92:0/613583794 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4001aa20 con 0x7f9e480cf1b0 2023-10-06T10:44:53.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.892+0000 7f9e47fff700 1 -- 172.21.15.92:0/613583794 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4814f7c0 con 0x7f9e480cf1b0 2023-10-06T10:44:53.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.893+0000 7f9e47fff700 1 -- 172.21.15.92:0/613583794 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e480998f0 con 0x7f9e480cf1b0 2023-10-06T10:44:53.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.893+0000 7f9e2effd700 1 -- 172.21.15.92:0/613583794 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9e4001b430 con 0x7f9e480cf1b0 2023-10-06T10:44:53.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.894+0000 7f9e2effd700 1 --2- 172.21.15.92:0/613583794 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9e34066f30 0x7f9e340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:53.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.894+0000 7f9e2effd700 1 -- 172.21.15.92:0/613583794 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9e400863a0 con 0x7f9e480cf1b0 2023-10-06T10:44:53.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.894+0000 7f9e46ffd700 1 --2- 172.21.15.92:0/613583794 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9e34066f30 0x7f9e340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:53.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.894+0000 7f9e46ffd700 1 --2- 172.21.15.92:0/613583794 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9e34066f30 0x7f9e340693e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9e380075e0 tx=0x7f9e38009830 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:53.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:53.898+0000 7f9e2effd700 1 -- 172.21.15.92:0/613583794 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e40053420 con 0x7f9e480cf1b0 2023-10-06T10:44:54.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.122+0000 7f9e47fff700 1 -- 172.21.15.92:0/613583794 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9e480cc600 con 0x7f9e480cf1b0 2023-10-06T10:44:54.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.759+0000 7f9e2effd700 1 -- 172.21.15.92:0/613583794 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v100) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9e40056a40 con 0x7f9e480cf1b0 2023-10-06T10:44:54.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.762+0000 7f9e47fff700 1 -- 172.21.15.92:0/613583794 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9e34066f30 msgr2=0x7f9e340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:54.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.762+0000 7f9e47fff700 1 --2- 172.21.15.92:0/613583794 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9e34066f30 0x7f9e340693e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9e380075e0 tx=0x7f9e38009830 comp rx=0 tx=0).stop 2023-10-06T10:44:54.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.762+0000 7f9e47fff700 1 -- 172.21.15.92:0/613583794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e480cf1b0 msgr2=0x7f9e481509a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:54.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.762+0000 7f9e47fff700 1 --2- 172.21.15.92:0/613583794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e480cf1b0 0x7f9e481509a0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009290 tx=0x7f9e40017000 comp rx=0 tx=0).stop 2023-10-06T10:44:54.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.762+0000 7f9e47fff700 1 -- 172.21.15.92:0/613583794 shutdown_connections 2023-10-06T10:44:54.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.762+0000 7f9e47fff700 1 --2- 172.21.15.92:0/613583794 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9e34066f30 0x7f9e340693e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:54.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.762+0000 7f9e47fff700 1 --2- 172.21.15.92:0/613583794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e480cf1b0 0x7f9e481509a0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:54.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.762+0000 7f9e47fff700 1 --2- 172.21.15.92:0/613583794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e480b2620 0x7f9e4814cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:54.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.762+0000 7f9e47fff700 1 --2- 172.21.15.92:0/613583794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e480b29f0 0x7f9e48150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:54.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.762+0000 7f9e47fff700 1 -- 172.21.15.92:0/613583794 >> 172.21.15.92:0/613583794 conn(0x7f9e48099520 msgr2=0x7f9e48068110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:54.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.762+0000 7f9e47fff700 1 -- 172.21.15.92:0/613583794 shutdown_connections 2023-10-06T10:44:54.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.762+0000 7f9e47fff700 1 -- 172.21.15.92:0/613583794 wait complete. 2023-10-06T10:44:54.777 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:44:54.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.892+0000 7ff4bdf97700 1 Processor -- start 2023-10-06T10:44:54.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.892+0000 7ff4bdf97700 1 -- start start 2023-10-06T10:44:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4bdf97700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff4b8159560 0x7ff4b80aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4bdf97700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b80b0c60 con 0x7ff4b80b49a0 2023-10-06T10:44:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4bdf97700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b80b0f40 con 0x7ff4b80b0760 2023-10-06T10:44:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4bdf97700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b80b1220 con 0x7ff4b8159560 2023-10-06T10:44:54.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b6ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff4b80b49a0 0x7ff4b80aaa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:43914/0 (socket says 172.21.15.92:43914) 2023-10-06T10:44:54.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b6ffd700 1 -- 172.21.15.92:0/365460579 learned_addr learned my addr 172.21.15.92:0/365460579 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:54.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b77fe700 1 --2- 172.21.15.92:0/365460579 >> v2:172.21.15.124:3300/0 conn(0x7ff4b8159560 0x7ff4b80aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:54.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b67fc700 1 -- 172.21.15.92:0/365460579 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2219442916 0 0) 0x7ff4b80b0c60 con 0x7ff4b80b49a0 2023-10-06T10:44:54.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b67fc700 1 -- 172.21.15.92:0/365460579 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4a0003570 con 0x7ff4b80b49a0 2023-10-06T10:44:54.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b67fc700 1 -- 172.21.15.92:0/365460579 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4109151488 0 0) 0x7ff4a0003570 con 0x7ff4b80b49a0 2023-10-06T10:44:54.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b67fc700 1 -- 172.21.15.92:0/365460579 >> v2:172.21.15.124:3300/0 conn(0x7ff4b8159560 msgr2=0x7ff4b80aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:54.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b67fc700 1 --2- 172.21.15.92:0/365460579 >> v2:172.21.15.124:3300/0 conn(0x7ff4b8159560 0x7ff4b80aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:54.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b67fc700 1 -- 172.21.15.92:0/365460579 >> v1:172.21.15.124:6789/0 conn(0x7ff4b80b0760 legacy=0x7ff4b80b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:54.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b67fc700 1 -- 172.21.15.92:0/365460579 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b80b1500 con 0x7ff4b80b49a0 2023-10-06T10:44:54.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b67fc700 1 -- 172.21.15.92:0/365460579 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7ff4ac001470 con 0x7ff4b80b49a0 2023-10-06T10:44:54.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b67fc700 1 -- 172.21.15.92:0/365460579 >> v1:172.21.15.92:6789/0 conn(0x7ff4b80b49a0 legacy=0x7ff4b80aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:54.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b67fc700 1 --2- 172.21.15.92:0/365460579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4a0003a60 0x7ff4a0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:54.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b67fc700 1 --2- 172.21.15.92:0/365460579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4a0004810 0x7ff4a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:54.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b67fc700 1 --2- 172.21.15.92:0/365460579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4a00051c0 0x7ff4a000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:54.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b67fc700 1 -- 172.21.15.92:0/365460579 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b80b1220 con 0x7ff4a0004810 2023-10-06T10:44:54.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b67fc700 1 -- 172.21.15.92:0/365460579 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a000c0d0 con 0x7ff4a00051c0 2023-10-06T10:44:54.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b67fc700 1 -- 172.21.15.92:0/365460579 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a000c290 con 0x7ff4a0003a60 2023-10-06T10:44:54.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b6ffd700 1 --2- 172.21.15.92:0/365460579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4a0004810 0x7ff4a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:54.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b77fe700 1 --2- 172.21.15.92:0/365460579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4a0003a60 0x7ff4a0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:54.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b7fff700 1 --2- 172.21.15.92:0/365460579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4a00051c0 0x7ff4a000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:54.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b6ffd700 1 -- 172.21.15.92:0/365460579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4a0003a60 msgr2=0x7ff4a0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:54.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b6ffd700 1 --2- 172.21.15.92:0/365460579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4a0003a60 0x7ff4a0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:54.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b6ffd700 1 -- 172.21.15.92:0/365460579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4a00051c0 msgr2=0x7ff4a000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:54.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b6ffd700 1 --2- 172.21.15.92:0/365460579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4a00051c0 0x7ff4a000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:54.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b6ffd700 1 -- 172.21.15.92:0/365460579 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a0003570 con 0x7ff4a0004810 2023-10-06T10:44:54.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b77fe700 1 --2- 172.21.15.92:0/365460579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4a0003a60 0x7ff4a0003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:54.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b7fff700 1 --2- 172.21.15.92:0/365460579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4a00051c0 0x7ff4a000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:54.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b7fff700 1 -- 172.21.15.92:0/365460579 reap_dead start 2023-10-06T10:44:54.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b6ffd700 1 --2- 172.21.15.92:0/365460579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4a0004810 0x7ff4a0004c80 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8088de0 tx=0x7ff4ac001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e9229d2c1b55a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:54.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b67fc700 1 -- 172.21.15.92:0/365460579 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac00cc30 con 0x7ff4a0004810 2023-10-06T10:44:54.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b67fc700 1 -- 172.21.15.92:0/365460579 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ac00cd90 con 0x7ff4a0004810 2023-10-06T10:44:54.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.894+0000 7ff4b67fc700 1 -- 172.21.15.92:0/365460579 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac005b50 con 0x7ff4a0004810 2023-10-06T10:44:54.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.895+0000 7ff4bdf97700 1 -- 172.21.15.92:0/365460579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4a0004810 msgr2=0x7ff4a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:54.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.895+0000 7ff4bdf97700 1 --2- 172.21.15.92:0/365460579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4a0004810 0x7ff4a0004c80 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8088de0 tx=0x7ff4ac001e50 comp rx=0 tx=0).stop 2023-10-06T10:44:54.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.895+0000 7ff4bdf97700 1 -- 172.21.15.92:0/365460579 shutdown_connections 2023-10-06T10:44:54.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.895+0000 7ff4bdf97700 1 --2- 172.21.15.92:0/365460579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4a0004810 0x7ff4a0004c80 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:54.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.895+0000 7ff4bdf97700 1 -- 172.21.15.92:0/365460579 >> 172.21.15.92:0/365460579 conn(0x7ff4b80997f0 msgr2=0x7ff4b806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:54.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.895+0000 7ff4bdf97700 1 -- 172.21.15.92:0/365460579 shutdown_connections 2023-10-06T10:44:54.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.895+0000 7ff4bdf97700 1 -- 172.21.15.92:0/365460579 wait complete. 2023-10-06T10:44:54.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.896+0000 7ff4bdf97700 1 Processor -- start 2023-10-06T10:44:54.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.896+0000 7ff4bdf97700 1 -- start start 2023-10-06T10:44:54.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff4bdf97700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4a0004810 0x7ff4b8155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:54.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff4bdf97700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4b81599b0 0x7ff4b8159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:54.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff4bdf97700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4b8156410 0x7ff4b8156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:54.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff4bdf97700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b815a360 con 0x7ff4b8156410 2023-10-06T10:44:54.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff4bdf97700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b809dd30 con 0x7ff4b81599b0 2023-10-06T10:44:54.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff4bdf97700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b8156dc0 con 0x7ff4a0004810 2023-10-06T10:44:54.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff4b7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4b8156410 0x7ff4b8156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:54.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff4b7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4b8156410 0x7ff4b8156880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33730/0 (socket says 172.21.15.92:33730) 2023-10-06T10:44:54.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff4b7fff700 1 -- 172.21.15.92:0/2219784596 learned_addr learned my addr 172.21.15.92:0/2219784596 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:54.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff4b6ffd700 1 --2- 172.21.15.92:0/2219784596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4b81599b0 0x7ff4b8159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:54.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff4b77fe700 1 --2- 172.21.15.92:0/2219784596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4a0004810 0x7ff4b8155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:54.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff4b7fff700 1 -- 172.21.15.92:0/2219784596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4a0004810 msgr2=0x7ff4b8155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:54.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff4b7fff700 1 --2- 172.21.15.92:0/2219784596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4a0004810 0x7ff4b8155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:54.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff4b7fff700 1 -- 172.21.15.92:0/2219784596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4b81599b0 msgr2=0x7ff4b8159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:54.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff4b7fff700 1 --2- 172.21.15.92:0/2219784596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4b81599b0 0x7ff4b8159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:54.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff4b7fff700 1 -- 172.21.15.92:0/2219784596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b8169be0 con 0x7ff4b8156410 2023-10-06T10:44:54.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff4b77fe700 1 --2- 172.21.15.92:0/2219784596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4a0004810 0x7ff4b8155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:54.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff4b7fff700 1 --2- 172.21.15.92:0/2219784596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4b8156410 0x7ff4b8156880 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7ff4b80b5610 tx=0x7ff4b0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:54.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff49f7fe700 1 -- 172.21.15.92:0/2219784596 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0014480 con 0x7ff4b8156410 2023-10-06T10:44:54.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff49f7fe700 1 -- 172.21.15.92:0/2219784596 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b00145e0 con 0x7ff4b8156410 2023-10-06T10:44:54.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff4bdf97700 1 -- 172.21.15.92:0/2219784596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b815a550 con 0x7ff4b8156410 2023-10-06T10:44:54.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff49f7fe700 1 -- 172.21.15.92:0/2219784596 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0014960 con 0x7ff4b8156410 2023-10-06T10:44:54.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff4bdf97700 1 -- 172.21.15.92:0/2219784596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b815ab10 con 0x7ff4b8156410 2023-10-06T10:44:54.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.898+0000 7ff4bdf97700 1 -- 172.21.15.92:0/2219784596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b8064a90 con 0x7ff4b8156410 2023-10-06T10:44:54.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.899+0000 7ff49f7fe700 1 -- 172.21.15.92:0/2219784596 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff4b00133e0 con 0x7ff4b8156410 2023-10-06T10:44:54.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.899+0000 7ff49f7fe700 1 --2- 172.21.15.92:0/2219784596 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff4a4066f30 0x7ff4a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:54.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.899+0000 7ff49f7fe700 1 -- 172.21.15.92:0/2219784596 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff4b0083e40 con 0x7ff4b8156410 2023-10-06T10:44:54.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.899+0000 7ff4b77fe700 1 --2- 172.21.15.92:0/2219784596 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff4a4066f30 0x7ff4a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:54.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.899+0000 7ff4b77fe700 1 --2- 172.21.15.92:0/2219784596 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff4a4066f30 0x7ff4a40693e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8088ff0 tx=0x7ff4a800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:54.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:54.903+0000 7ff49f7fe700 1 -- 172.21.15.92:0/2219784596 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4b004d5a0 con 0x7ff4b8156410 2023-10-06T10:44:55.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.128+0000 7ff4bdf97700 1 -- 172.21.15.92:0/2219784596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4b80b17e0 con 0x7ff4b8156410 2023-10-06T10:44:55.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.129+0000 7ff49f7fe700 1 -- 172.21.15.92:0/2219784596 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4607 (secure 0 0 0) 0x7ff4b0019070 con 0x7ff4b8156410 2023-10-06T10:44:55.131 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:55.131 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":100,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2016052382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2016052382},{"type":"v1","addr":"172.21.15.124:6834","nonce":2016052382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":6719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2307521923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2307521923},{"type":"v1","addr":"172.21.15.124:6835","nonce":2307521923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":8517,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1979826278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979826278},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979826278}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:44:51.106486+0000","modified":"2023-10-06T10:44:54.749834+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2298348426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2298348426},{"type":"v1","addr":"172.21.15.92:6837","nonce":2298348426}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T10:44:55.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.131+0000 7ff4bdf97700 1 -- 172.21.15.92:0/2219784596 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff4a4066f30 msgr2=0x7ff4a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:55.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.131+0000 7ff4bdf97700 1 --2- 172.21.15.92:0/2219784596 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff4a4066f30 0x7ff4a40693e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8088ff0 tx=0x7ff4a800b040 comp rx=0 tx=0).stop 2023-10-06T10:44:55.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.131+0000 7ff4bdf97700 1 -- 172.21.15.92:0/2219784596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4b8156410 msgr2=0x7ff4b8156880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:55.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.131+0000 7ff4bdf97700 1 --2- 172.21.15.92:0/2219784596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4b8156410 0x7ff4b8156880 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7ff4b80b5610 tx=0x7ff4b0009040 comp rx=0 tx=0).stop 2023-10-06T10:44:55.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.131+0000 7ff4bdf97700 1 -- 172.21.15.92:0/2219784596 shutdown_connections 2023-10-06T10:44:55.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.131+0000 7ff4bdf97700 1 --2- 172.21.15.92:0/2219784596 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff4a4066f30 0x7ff4a40693e0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.131+0000 7ff4bdf97700 1 --2- 172.21.15.92:0/2219784596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4b8156410 0x7ff4b8156880 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.131+0000 7ff4bdf97700 1 --2- 172.21.15.92:0/2219784596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4a0004810 0x7ff4b8155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.131+0000 7ff4bdf97700 1 --2- 172.21.15.92:0/2219784596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4b81599b0 0x7ff4b8159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.131+0000 7ff4bdf97700 1 -- 172.21.15.92:0/2219784596 >> 172.21.15.92:0/2219784596 conn(0x7ff4b80997f0 msgr2=0x7ff4b80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:55.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.131+0000 7ff4bdf97700 1 -- 172.21.15.92:0/2219784596 shutdown_connections 2023-10-06T10:44:55.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.131+0000 7ff4bdf97700 1 -- 172.21.15.92:0/2219784596 wait complete. 2023-10-06T10:44:55.135 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 100 2023-10-06T10:44:55.145 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:44:55.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfca857700 1 Processor -- start 2023-10-06T10:44:55.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfca857700 1 -- start start 2023-10-06T10:44:55.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfca857700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc4159560 0x7fbfc40aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:55.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfca857700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc40b0c60 con 0x7fbfc40b0760 2023-10-06T10:44:55.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfca857700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc40b0f40 con 0x7fbfc40b49a0 2023-10-06T10:44:55.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfca857700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc40b1220 con 0x7fbfc4159560 2023-10-06T10:44:55.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfc8df4700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbfc40b0760 0x7fbfc40b1bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:43916/0 (socket says 172.21.15.92:43916) 2023-10-06T10:44:55.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfc8df4700 1 -- 172.21.15.92:0/4275712525 learned_addr learned my addr 172.21.15.92:0/4275712525 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:55.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfc2ffd700 1 -- 172.21.15.92:0/4275712525 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3591004103 0 0) 0x7fbfc40b0c60 con 0x7fbfc40b0760 2023-10-06T10:44:55.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfc3fff700 1 --2- 172.21.15.92:0/4275712525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc4159560 0x7fbfc40aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:55.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfc2ffd700 1 -- 172.21.15.92:0/4275712525 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfac003570 con 0x7fbfc40b0760 2023-10-06T10:44:55.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfc2ffd700 1 -- 172.21.15.92:0/4275712525 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1455146728 0 0) 0x7fbfac003570 con 0x7fbfc40b0760 2023-10-06T10:44:55.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfc2ffd700 1 -- 172.21.15.92:0/4275712525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc4159560 msgr2=0x7fbfc40aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:55.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfc2ffd700 1 --2- 172.21.15.92:0/4275712525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc4159560 0x7fbfc40aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfc2ffd700 1 -- 172.21.15.92:0/4275712525 >> v1:172.21.15.124:6789/0 conn(0x7fbfc40b49a0 legacy=0x7fbfc40aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:55.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfc2ffd700 1 -- 172.21.15.92:0/4275712525 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc40b1500 con 0x7fbfc40b0760 2023-10-06T10:44:55.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfc2ffd700 1 -- 172.21.15.92:0/4275712525 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fbfbc0017f0 con 0x7fbfc40b0760 2023-10-06T10:44:55.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfc2ffd700 1 -- 172.21.15.92:0/4275712525 >> v1:172.21.15.92:6789/0 conn(0x7fbfc40b0760 legacy=0x7fbfc40b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:55.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfc2ffd700 1 --2- 172.21.15.92:0/4275712525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc4159560 0x7fbfac003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:55.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfc2ffd700 1 --2- 172.21.15.92:0/4275712525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfac004010 0x7fbfac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:55.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfc2ffd700 1 --2- 172.21.15.92:0/4275712525 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfac004d50 0x7fbfac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:55.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfc2ffd700 1 -- 172.21.15.92:0/4275712525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc40b1220 con 0x7fbfac004010 2023-10-06T10:44:55.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfc2ffd700 1 -- 172.21.15.92:0/4275712525 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfac00bbf0 con 0x7fbfac004d50 2023-10-06T10:44:55.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfc2ffd700 1 -- 172.21.15.92:0/4275712525 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfac00bdb0 con 0x7fbfc4159560 2023-10-06T10:44:55.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.259+0000 7fbfc37fe700 1 --2- 172.21.15.92:0/4275712525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfac004010 0x7fbfac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:55.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.260+0000 7fbfc3fff700 1 --2- 172.21.15.92:0/4275712525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc4159560 0x7fbfac003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:55.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.260+0000 7fbfc8df4700 1 --2- 172.21.15.92:0/4275712525 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfac004d50 0x7fbfac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:55.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.260+0000 7fbfc37fe700 1 -- 172.21.15.92:0/4275712525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc4159560 msgr2=0x7fbfac003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:55.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.260+0000 7fbfc37fe700 1 --2- 172.21.15.92:0/4275712525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc4159560 0x7fbfac003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.260+0000 7fbfc37fe700 1 -- 172.21.15.92:0/4275712525 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfac004d50 msgr2=0x7fbfac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:55.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.260+0000 7fbfc37fe700 1 --2- 172.21.15.92:0/4275712525 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfac004d50 0x7fbfac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.260+0000 7fbfc37fe700 1 -- 172.21.15.92:0/4275712525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfac003570 con 0x7fbfac004010 2023-10-06T10:44:55.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.260+0000 7fbfc37fe700 1 --2- 172.21.15.92:0/4275712525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfac004010 0x7fbfac004810 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40b4db0 tx=0x7fbfb8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff6a4ae53944bc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:55.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.260+0000 7fbfc2ffd700 1 -- 172.21.15.92:0/4275712525 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8012420 con 0x7fbfac004010 2023-10-06T10:44:55.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.260+0000 7fbfc2ffd700 1 -- 172.21.15.92:0/4275712525 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb8012580 con 0x7fbfac004010 2023-10-06T10:44:55.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.260+0000 7fbfc2ffd700 1 -- 172.21.15.92:0/4275712525 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8012920 con 0x7fbfac004010 2023-10-06T10:44:55.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.260+0000 7fbfca857700 1 -- 172.21.15.92:0/4275712525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfac004010 msgr2=0x7fbfac004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:55.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.260+0000 7fbfca857700 1 --2- 172.21.15.92:0/4275712525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfac004010 0x7fbfac004810 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40b4db0 tx=0x7fbfb8007d30 comp rx=0 tx=0).stop 2023-10-06T10:44:55.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.260+0000 7fbfc8df4700 1 -- 172.21.15.92:0/4275712525 reap_dead start 2023-10-06T10:44:55.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.261+0000 7fbfca857700 1 -- 172.21.15.92:0/4275712525 shutdown_connections 2023-10-06T10:44:55.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.261+0000 7fbfca857700 1 -- 172.21.15.92:0/4275712525 >> 172.21.15.92:0/4275712525 conn(0x7fbfc40997f0 msgr2=0x7fbfc406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:55.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.261+0000 7fbfca857700 1 -- 172.21.15.92:0/4275712525 shutdown_connections 2023-10-06T10:44:55.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.261+0000 7fbfca857700 1 -- 172.21.15.92:0/4275712525 wait complete. 2023-10-06T10:44:55.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.262+0000 7fbfca857700 1 Processor -- start 2023-10-06T10:44:55.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.262+0000 7fbfca857700 1 -- start start 2023-10-06T10:44:55.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfca857700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfac004010 0x7fbfc4169ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:55.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfca857700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc409c530 0x7fbfc4156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:55.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfca857700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfc409c9a0 0x7fbfc4156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:55.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfca857700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc415c6b0 con 0x7fbfac004010 2023-10-06T10:44:55.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfca857700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc415a260 con 0x7fbfc409c9a0 2023-10-06T10:44:55.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfca857700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc409b800 con 0x7fbfc409c530 2023-10-06T10:44:55.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfc3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfac004010 0x7fbfc4169ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:55.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfc3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfac004010 0x7fbfc4169ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33754/0 (socket says 172.21.15.92:33754) 2023-10-06T10:44:55.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfc3fff700 1 -- 172.21.15.92:0/1571709429 learned_addr learned my addr 172.21.15.92:0/1571709429 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:55.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfc37fe700 1 --2- 172.21.15.92:0/1571709429 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc409c530 0x7fbfc4156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:55.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfc8df4700 1 --2- 172.21.15.92:0/1571709429 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfc409c9a0 0x7fbfc4156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:55.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfc3fff700 1 -- 172.21.15.92:0/1571709429 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc409c530 msgr2=0x7fbfc4156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:55.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfc3fff700 1 --2- 172.21.15.92:0/1571709429 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc409c530 0x7fbfc4156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfc3fff700 1 -- 172.21.15.92:0/1571709429 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfc409c9a0 msgr2=0x7fbfc4156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:55.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfc3fff700 1 --2- 172.21.15.92:0/1571709429 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfc409c9a0 0x7fbfc4156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfc3fff700 1 -- 172.21.15.92:0/1571709429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc4159930 con 0x7fbfac004010 2023-10-06T10:44:55.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfc37fe700 1 --2- 172.21.15.92:0/1571709429 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc409c530 0x7fbfc4156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:44:55.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfc3fff700 1 --2- 172.21.15.92:0/1571709429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfac004010 0x7fbfc4169ad0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4011b90 tx=0x7fbfb4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:55.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfabfff700 1 -- 172.21.15.92:0/1571709429 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4005c10 con 0x7fbfac004010 2023-10-06T10:44:55.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfabfff700 1 -- 172.21.15.92:0/1571709429 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb4005d70 con 0x7fbfac004010 2023-10-06T10:44:55.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfca857700 1 -- 172.21.15.92:0/1571709429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc4159c10 con 0x7fbfac004010 2023-10-06T10:44:55.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfabfff700 1 -- 172.21.15.92:0/1571709429 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb400fa90 con 0x7fbfac004010 2023-10-06T10:44:55.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.263+0000 7fbfca857700 1 -- 172.21.15.92:0/1571709429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc416a010 con 0x7fbfac004010 2023-10-06T10:44:55.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.264+0000 7fbfca857700 1 -- 172.21.15.92:0/1571709429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc4064a90 con 0x7fbfac004010 2023-10-06T10:44:55.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.265+0000 7fbfabfff700 1 -- 172.21.15.92:0/1571709429 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbfb4006050 con 0x7fbfac004010 2023-10-06T10:44:55.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.265+0000 7fbfabfff700 1 --2- 172.21.15.92:0/1571709429 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfb0066f30 0x7fbfb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:55.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.265+0000 7fbfabfff700 1 -- 172.21.15.92:0/1571709429 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fbfb4085070 con 0x7fbfac004010 2023-10-06T10:44:55.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.265+0000 7fbfc37fe700 1 --2- 172.21.15.92:0/1571709429 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfb0066f30 0x7fbfb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:55.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.265+0000 7fbfc37fe700 1 --2- 172.21.15.92:0/1571709429 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfb0066f30 0x7fbfb00693e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8001370 tx=0x7fbfb8015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:55.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.270+0000 7fbfabfff700 1 -- 172.21.15.92:0/1571709429 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfb4052040 con 0x7fbfac004010 2023-10-06T10:44:55.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.454+0000 7fbfca857700 1 -- 172.21.15.92:0/1571709429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfc40b17e0 con 0x7fbfac004010 2023-10-06T10:44:55.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.455+0000 7fbfabfff700 1 -- 172.21.15.92:0/1571709429 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+17085 (secure 0 0 0) 0x7fbfb4055660 con 0x7fbfac004010 2023-10-06T10:44:55.456 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:55.456 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":102,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:44:52.721783+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:44:48.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-06T10:44:50.072416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:44:55.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.457+0000 7fbfca857700 1 -- 172.21.15.92:0/1571709429 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfb0066f30 msgr2=0x7fbfb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:55.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.457+0000 7fbfca857700 1 --2- 172.21.15.92:0/1571709429 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfb0066f30 0x7fbfb00693e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8001370 tx=0x7fbfb8015000 comp rx=0 tx=0).stop 2023-10-06T10:44:55.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.457+0000 7fbfca857700 1 -- 172.21.15.92:0/1571709429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfac004010 msgr2=0x7fbfc4169ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:55.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.457+0000 7fbfca857700 1 --2- 172.21.15.92:0/1571709429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfac004010 0x7fbfc4169ad0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4011b90 tx=0x7fbfb4009120 comp rx=0 tx=0).stop 2023-10-06T10:44:55.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.457+0000 7fbfca857700 1 -- 172.21.15.92:0/1571709429 shutdown_connections 2023-10-06T10:44:55.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.457+0000 7fbfca857700 1 --2- 172.21.15.92:0/1571709429 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfb0066f30 0x7fbfb00693e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.457+0000 7fbfca857700 1 --2- 172.21.15.92:0/1571709429 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfc409c9a0 0x7fbfc4156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.457+0000 7fbfca857700 1 --2- 172.21.15.92:0/1571709429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfac004010 0x7fbfc4169ad0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.457+0000 7fbfca857700 1 --2- 172.21.15.92:0/1571709429 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc409c530 0x7fbfc4156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.457+0000 7fbfca857700 1 -- 172.21.15.92:0/1571709429 >> 172.21.15.92:0/1571709429 conn(0x7fbfc40997f0 msgr2=0x7fbfc40cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:55.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.457+0000 7fbfca857700 1 -- 172.21.15.92:0/1571709429 shutdown_connections 2023-10-06T10:44:55.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.457+0000 7fbfca857700 1 -- 172.21.15.92:0/1571709429 wait complete. 2023-10-06T10:44:55.470 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T10:44:55.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:44:55.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.583+0000 7f81b2f59700 1 Processor -- start 2023-10-06T10:44:55.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.583+0000 7f81b2f59700 1 -- start start 2023-10-06T10:44:55.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.584+0000 7f81b2f59700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f81ac0b2810 0x7f81ac0cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:55.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.584+0000 7f81b2f59700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ac0cb7a0 con 0x7f81ac0cbc30 2023-10-06T10:44:55.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.584+0000 7f81b2f59700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ac0b0860 con 0x7f81ac159440 2023-10-06T10:44:55.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.584+0000 7f81b2f59700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ac0cb150 con 0x7f81ac0b2810 2023-10-06T10:44:55.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.584+0000 7f81b1756700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f81ac0cbc30 0x7f81ac0b2100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:43918/0 (socket says 172.21.15.92:43918) 2023-10-06T10:44:55.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.584+0000 7f81b1756700 1 -- 172.21.15.92:0/635485067 learned_addr learned my addr 172.21.15.92:0/635485067 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:55.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.584+0000 7f81b2758700 1 --2- 172.21.15.92:0/635485067 >> v2:172.21.15.124:3300/0 conn(0x7f81ac0b2810 0x7f81ac0cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:55.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.584+0000 7f81b0f55700 1 -- 172.21.15.92:0/635485067 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 839137299 0 0) 0x7f81ac0cb7a0 con 0x7f81ac0cbc30 2023-10-06T10:44:55.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.584+0000 7f81b0f55700 1 -- 172.21.15.92:0/635485067 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8194003570 con 0x7f81ac0cbc30 2023-10-06T10:44:55.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.584+0000 7f81b0f55700 1 -- 172.21.15.92:0/635485067 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1702730316 0 0) 0x7f81ac0b0860 con 0x7f81ac159440 2023-10-06T10:44:55.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.584+0000 7f81b0f55700 1 -- 172.21.15.92:0/635485067 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81ac0cb7a0 con 0x7f81ac159440 2023-10-06T10:44:55.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.584+0000 7f81b2758700 1 -- 172.21.15.92:0/635485067 >> v1:172.21.15.124:6789/0 conn(0x7f81ac159440 legacy=0x7f81ac0b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:55.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.584+0000 7f81b0f55700 1 -- 172.21.15.92:0/635485067 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4227733873 0 0) 0x7f8194003570 con 0x7f81ac0cbc30 2023-10-06T10:44:55.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.584+0000 7f81b2758700 1 -- 172.21.15.92:0/635485067 >> v1:172.21.15.92:6789/0 conn(0x7f81ac0cbc30 legacy=0x7f81ac0b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:55.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.584+0000 7f81b2758700 1 -- 172.21.15.92:0/635485067 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac0cb430 con 0x7f81ac0b2810 2023-10-06T10:44:55.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.585+0000 7f81b2758700 1 --2- 172.21.15.92:0/635485067 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f81ac0b2810 0x7f81ac0cc080 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f819c009e60 tx=0x7f819c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f08fba1915f1181 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:55.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.585+0000 7f81b0f55700 1 -- 172.21.15.92:0/635485067 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c00e070 con 0x7f81ac0b2810 2023-10-06T10:44:55.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.585+0000 7f81b0f55700 1 -- 172.21.15.92:0/635485067 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819c00b7a0 con 0x7f81ac0b2810 2023-10-06T10:44:55.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.585+0000 7f81b0f55700 1 -- 172.21.15.92:0/635485067 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a0001470 con 0x7f81ac0b2810 2023-10-06T10:44:55.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.585+0000 7f81b2f59700 1 -- 172.21.15.92:0/635485067 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f81ac0b2810 msgr2=0x7f81ac0cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:55.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.585+0000 7f81b2f59700 1 --2- 172.21.15.92:0/635485067 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f81ac0b2810 0x7f81ac0cc080 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f819c009e60 tx=0x7f819c009780 comp rx=0 tx=0).stop 2023-10-06T10:44:55.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.585+0000 7f81b2f59700 1 -- 172.21.15.92:0/635485067 shutdown_connections 2023-10-06T10:44:55.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.585+0000 7f81b2f59700 1 --2- 172.21.15.92:0/635485067 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f81ac0b2810 0x7f81ac0cc080 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.585+0000 7f81b2f59700 1 -- 172.21.15.92:0/635485067 >> 172.21.15.92:0/635485067 conn(0x7f81ac06ece0 msgr2=0x7f81ac06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:55.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.585+0000 7f81b2f59700 1 -- 172.21.15.92:0/635485067 shutdown_connections 2023-10-06T10:44:55.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.585+0000 7f81b2f59700 1 -- 172.21.15.92:0/635485067 wait complete. 2023-10-06T10:44:55.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.586+0000 7f81b2f59700 1 Processor -- start 2023-10-06T10:44:55.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.586+0000 7f81b2f59700 1 -- start start 2023-10-06T10:44:55.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.587+0000 7f81b2f59700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f81ac0b2810 0x7f81ac154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:55.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.587+0000 7f81b2f59700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f81ac0cbc30 0x7f81ac0c3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:55.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.587+0000 7f81b2f59700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81ac159440 0x7f81ac154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:55.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.587+0000 7f81b2f59700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac15d8b0 con 0x7f81ac159440 2023-10-06T10:44:55.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.587+0000 7f81b2f59700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac0c4510 con 0x7f81ac0b2810 2023-10-06T10:44:55.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.587+0000 7f81b2f59700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac0c47f0 con 0x7f81ac0cbc30 2023-10-06T10:44:55.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.587+0000 7f81b2758700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81ac159440 0x7f81ac154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:55.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.587+0000 7f81b2758700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81ac159440 0x7f81ac154fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33766/0 (socket says 172.21.15.92:33766) 2023-10-06T10:44:55.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.587+0000 7f81b2758700 1 -- 172.21.15.92:0/2672788333 learned_addr learned my addr 172.21.15.92:0/2672788333 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:55.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.587+0000 7f81b1756700 1 --2- 172.21.15.92:0/2672788333 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f81ac0cbc30 0x7f81ac0c3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:55.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.587+0000 7f81b1f57700 1 --2- 172.21.15.92:0/2672788333 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f81ac0b2810 0x7f81ac154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:55.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.588+0000 7f81b2758700 1 -- 172.21.15.92:0/2672788333 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f81ac0cbc30 msgr2=0x7f81ac0c3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:55.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.588+0000 7f81b2758700 1 --2- 172.21.15.92:0/2672788333 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f81ac0cbc30 0x7f81ac0c3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.588+0000 7f81b2758700 1 -- 172.21.15.92:0/2672788333 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f81ac0b2810 msgr2=0x7f81ac154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:55.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.588+0000 7f81b2758700 1 --2- 172.21.15.92:0/2672788333 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f81ac0b2810 0x7f81ac154940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.588+0000 7f81b2758700 1 -- 172.21.15.92:0/2672788333 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819c009a20 con 0x7f81ac159440 2023-10-06T10:44:55.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.588+0000 7f81b2758700 1 --2- 172.21.15.92:0/2672788333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81ac159440 0x7f81ac154fb0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f81ac069150 tx=0x7f819c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:55.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.588+0000 7f81a9ffb700 1 -- 172.21.15.92:0/2672788333 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c012c40 con 0x7f81ac159440 2023-10-06T10:44:55.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.588+0000 7f81a9ffb700 1 -- 172.21.15.92:0/2672788333 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819c012da0 con 0x7f81ac159440 2023-10-06T10:44:55.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.588+0000 7f81b2f59700 1 -- 172.21.15.92:0/2672788333 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ac0c4ad0 con 0x7f81ac159440 2023-10-06T10:44:55.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.588+0000 7f81a9ffb700 1 -- 172.21.15.92:0/2672788333 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c01e680 con 0x7f81ac159440 2023-10-06T10:44:55.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.588+0000 7f81b2f59700 1 -- 172.21.15.92:0/2672788333 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ac0c6710 con 0x7f81ac159440 2023-10-06T10:44:55.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.589+0000 7f81b2f59700 1 -- 172.21.15.92:0/2672788333 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ac15da20 con 0x7f81ac159440 2023-10-06T10:44:55.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.589+0000 7f81a9ffb700 1 -- 172.21.15.92:0/2672788333 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f819c01e9f0 con 0x7f81ac159440 2023-10-06T10:44:55.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.590+0000 7f81a9ffb700 1 --2- 172.21.15.92:0/2672788333 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8198066f30 0x7f81980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:55.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.590+0000 7f81a9ffb700 1 -- 172.21.15.92:0/2672788333 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f819c0869a0 con 0x7f81ac159440 2023-10-06T10:44:55.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.590+0000 7f81b1f57700 1 --2- 172.21.15.92:0/2672788333 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8198066f30 0x7f81980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:55.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.590+0000 7f81b1f57700 1 --2- 172.21.15.92:0/2672788333 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8198066f30 0x7f81980693e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f81a4007fd0 tx=0x7f81a4007d10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:55.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.594+0000 7f81a9ffb700 1 -- 172.21.15.92:0/2672788333 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f819c053970 con 0x7f81ac159440 2023-10-06T10:44:55.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.773+0000 7f81b2f59700 1 -- 172.21.15.92:0/2672788333 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f81ac15d710 con 0x7f8198066f30 2023-10-06T10:44:55.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.782+0000 7f81a9ffb700 1 -- 172.21.15.92:0/2672788333 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+297279 (secure 0 0 0) 0x7f81a4002680 con 0x7f8198066f30 2023-10-06T10:44:55.786 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:55.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.785+0000 7f81b2f59700 1 -- 172.21.15.92:0/2672788333 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8198066f30 msgr2=0x7f81980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:55.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.785+0000 7f81b2f59700 1 --2- 172.21.15.92:0/2672788333 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8198066f30 0x7f81980693e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f81a4007fd0 tx=0x7f81a4007d10 comp rx=0 tx=0).stop 2023-10-06T10:44:55.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.785+0000 7f81b2f59700 1 -- 172.21.15.92:0/2672788333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81ac159440 msgr2=0x7f81ac154fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:55.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.785+0000 7f81b2f59700 1 --2- 172.21.15.92:0/2672788333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81ac159440 0x7f81ac154fb0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f81ac069150 tx=0x7f819c017000 comp rx=0 tx=0).stop 2023-10-06T10:44:55.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.785+0000 7f81b2f59700 1 -- 172.21.15.92:0/2672788333 shutdown_connections 2023-10-06T10:44:55.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.785+0000 7f81b2f59700 1 --2- 172.21.15.92:0/2672788333 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8198066f30 0x7f81980693e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.785+0000 7f81b2f59700 1 --2- 172.21.15.92:0/2672788333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81ac159440 0x7f81ac154fb0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.785+0000 7f81b2f59700 1 --2- 172.21.15.92:0/2672788333 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f81ac0b2810 0x7f81ac154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.785+0000 7f81b2f59700 1 --2- 172.21.15.92:0/2672788333 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f81ac0cbc30 0x7f81ac0c3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.785+0000 7f81b2f59700 1 -- 172.21.15.92:0/2672788333 >> 172.21.15.92:0/2672788333 conn(0x7f81ac06ece0 msgr2=0x7f81ac157100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:55.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.785+0000 7f81b2f59700 1 -- 172.21.15.92:0/2672788333 shutdown_connections 2023-10-06T10:44:55.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.785+0000 7f81b2f59700 1 -- 172.21.15.92:0/2672788333 wait complete. 2023-10-06T10:44:55.789 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:44:55.798 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2023-10-06T10:44:54.168147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2388964,"kb_used_data":10660,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752585756,"statfs":{"total":773094113280,"available":770647814144,"internally_reserved":0,"allocated":10915840,"data_stored":5858455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.452541"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":86,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-06T10:44:45.702353+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:44:45.702353+0000","last_peered":"2023-10-06T10:44:45.702353+0000","last_clean":"2023-10-06T10:44:45.702353+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:44:45.702353+0000","last_undegraded":"2023-10-06T10:44:45.702353+0000","last_fullsized":"2023-10-06T10:44:45.702353+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-06T10:44:46.155898+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:44:46.155898+0000","last_peered":"2023-10-06T10:44:46.155898+0000","last_clean":"2023-10-06T10:44:46.155898+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:44:46.155898+0000","last_undegraded":"2023-10-06T10:44:46.155898+0000","last_fullsized":"2023-10-06T10:44:46.155898+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":80,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-06T10:44:45.964853+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:44:45.964853+0000","last_peered":"2023-10-06T10:44:45.964853+0000","last_clean":"2023-10-06T10:44:45.964853+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:44:45.964853+0000","last_undegraded":"2023-10-06T10:44:45.964853+0000","last_fullsized":"2023-10-06T10:44:45.964853+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":222,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-06T10:44:46.477634+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:44:46.477634+0000","last_peered":"2023-10-06T10:44:46.477634+0000","last_clean":"2023-10-06T10:44:46.477634+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:44:46.477634+0000","last_undegraded":"2023-10-06T10:44:46.477634+0000","last_fullsized":"2023-10-06T10:44:46.477634+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":72,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-06T10:44:46.477558+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:44:46.477558+0000","last_peered":"2023-10-06T10:44:46.477558+0000","last_clean":"2023-10-06T10:44:46.477558+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:44:46.477558+0000","last_undegraded":"2023-10-06T10:44:46.477558+0000","last_fullsized":"2023-10-06T10:44:46.477558+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-06T10:44:46.477560+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:44:46.477560+0000","last_peered":"2023-10-06T10:44:46.477560+0000","last_clean":"2023-10-06T10:44:46.477560+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:44:46.477560+0000","last_undegraded":"2023-10-06T10:44:46.477560+0000","last_fullsized":"2023-10-06T10:44:46.477560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-06T10:44:45.702568+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:44:45.702568+0000","last_peered":"2023-10-06T10:44:45.702568+0000","last_clean":"2023-10-06T10:44:45.702568+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:44:45.702568+0000","last_undegraded":"2023-10-06T10:44:45.702568+0000","last_fullsized":"2023-10-06T10:44:45.702568+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-06T10:44:46.477301+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:44:46.477301+0000","last_peered":"2023-10-06T10:44:46.477301+0000","last_clean":"2023-10-06T10:44:46.477301+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:44:46.477301+0000","last_undegraded":"2023-10-06T10:44:46.477301+0000","last_fullsized":"2023-10-06T10:44:46.477301+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-06T10:44:46.477310+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:44:46.477310+0000","last_peered":"2023-10-06T10:44:46.477310+0000","last_clean":"2023-10-06T10:44:46.477310+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:44:46.477310+0000","last_undegraded":"2023-10-06T10:44:46.477310+0000","last_fullsized":"2023-10-06T10:44:46.477310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574933,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299056,"kb_used_data":1768,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072784,"statfs":{"total":96636764160,"available":96330530816,"internally_reserved":0,"allocated":1810432,"data_stored":1175252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.51000000000000001,"15min":0.48799999999999999},"min":{"1min":0.32800000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63900000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.434},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.53800000000000003,"15min":0.52100000000000002},"min":{"1min":0.29399999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.9700000000000002,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.496,"15min":0.46700000000000003},"min":{"1min":0.32000000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.64800000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.54500000000000004,"15min":0.51300000000000001},"min":{"1min":0.38400000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":3.0059999999999998,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.52200000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56699999999999995,"15min":0.52200000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.497,"5min":0.51900000000000002,"15min":0.499},"min":{"1min":0.371,"5min":0.314,"15min":0.314},"max":{"1min":0.66800000000000004,"5min":0.747,"15min":0.79800000000000004},"last":0.56999999999999995}]},{"osd":3,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55200000000000005,"15min":0.51600000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73099999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.40600000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.71899999999999997,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.67500000000000004}]},{"osd":4,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42199999999999999,"15min":0.46200000000000002},"min":{"1min":0.314,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61899999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47299999999999998,"15min":0.499},"min":{"1min":0.30199999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64900000000000002,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69099999999999995}]},{"osd":5,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.48199999999999998},"min":{"1min":0.33000000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.81100000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.51800000000000002,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85399999999999998,"5min":1.577,"15min":1.577},"last":0.60299999999999998}]},{"osd":6,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.56499999999999995},"min":{"1min":0.39700000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.67200000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.56200000000000006,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69599999999999995,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.63600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298472,"kb_used_data":1184,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073368,"statfs":{"total":96636764160,"available":96331128832,"internally_reserved":0,"allocated":1212416,"data_stored":583531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.495},"min":{"1min":0.34599999999999997,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":0.70399999999999996,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52100000000000002,"15min":0.45500000000000002},"min":{"1min":0.39700000000000002,"5min":0.32500000000000001,"15min":0.32000000000000001},"max":{"1min":0.67400000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.41899999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52000000000000002,"15min":0.46000000000000002},"min":{"1min":0.39300000000000002,"5min":0.32200000000000001,"15min":0.28000000000000003},"max":{"1min":0.70799999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54500000000000004,"15min":0.48899999999999999},"min":{"1min":0.42399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73499999999999999,"5min":0.751,"15min":0.751},"last":0.38300000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52700000000000002,"15min":0.48399999999999999},"min":{"1min":0.39600000000000002,"5min":0.32500000000000001,"15min":0.309},"max":{"1min":0.65400000000000003,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54500000000000004,"15min":0.48699999999999999},"min":{"1min":0.443,"5min":0.34999999999999998,"15min":0.34300000000000003},"max":{"1min":0.66100000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55200000000000005,"15min":0.50600000000000001},"min":{"1min":0.46500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75600000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.55500000000000005,"15min":0.51900000000000002},"min":{"1min":0.40699999999999997,"5min":0.36399999999999999,"15min":0.34499999999999997},"max":{"1min":0.74199999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.52700000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.48699999999999999,"15min":0.48099999999999998},"min":{"1min":0.34999999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.78300000000000003,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.436},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.498,"15min":0.48899999999999999},"min":{"1min":0.377,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79400000000000004,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.51000000000000001,"15min":0.49199999999999999},"min":{"1min":0.34200000000000003,"5min":0.28899999999999998,"15min":0.28299999999999997},"max":{"1min":0.83599999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.38700000000000001,"5min":0.33900000000000002,"15min":0.308},"max":{"1min":0.77800000000000002,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.55900000000000005}]},{"osd":7,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53200000000000003,"15min":0.52000000000000002},"min":{"1min":0.36599999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71899999999999997,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.502},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57399999999999995,"15min":0.54800000000000004},"min":{"1min":0.40000000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.871,"5min":0.879,"15min":0.879},"last":0.54000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299056,"kb_used_data":1768,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072784,"statfs":{"total":96636764160,"available":96330530816,"internally_reserved":0,"allocated":1810432,"data_stored":1175252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.20399999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.35599999999999998,"15min":0.34799999999999998},"min":{"1min":0.23400000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.57099999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.38200000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.33000000000000002,"15min":0.32000000000000001},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.46999999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.372},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38100000000000001,"15min":0.36699999999999999},"min":{"1min":0.27700000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.32400000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39500000000000002,"15min":0.38},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.60999999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.38700000000000001,"15min":0.377},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.26100000000000001}]},{"osd":4,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.40200000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73099999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.746},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.753}]},{"osd":5,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86599999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.55000000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.35699999999999998},"max":{"1min":0.80900000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.76400000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.59899999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.36699999999999999},"max":{"1min":0.92200000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.435,"5min":0.40000000000000002,"15min":0.39800000000000002},"max":{"1min":0.77000000000000002,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.70399999999999996}]},{"osd":7,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.39100000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.84299999999999997,"5min":0.996,"15min":0.996},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.42099999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.67400000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298476,"kb_used_data":1188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073364,"statfs":{"total":96636764160,"available":96331124736,"internally_reserved":0,"allocated":1216512,"data_stored":584884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.33000000000000002,"15min":0.32000000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.433,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.376,"15min":0.34999999999999998},"min":{"1min":0.28399999999999997,"5min":0.25,"15min":0.25},"max":{"1min":0.504,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.32100000000000001,"15min":0.309},"min":{"1min":0.246,"5min":0.191,"15min":0.191},"max":{"1min":0.41799999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35599999999999998,"15min":0.33200000000000002},"min":{"1min":0.26900000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.46000000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.32200000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35399999999999998,"15min":0.34399999999999997},"min":{"1min":0.25600000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.49299999999999999,"5min":0.622,"15min":0.622},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36599999999999999,"15min":0.35299999999999998},"min":{"1min":0.28999999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.47899999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.379}]},{"osd":4,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.41799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78900000000000003,"5min":0.81200000000000006,"15min":0.83299999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56599999999999995,"15min":0.54100000000000004},"min":{"1min":0.38600000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.753,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.51900000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.36899999999999999},"max":{"1min":0.80300000000000005,"5min":0.91900000000000004,"15min":1.0069999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.41299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76700000000000002,"5min":0.80900000000000005,"15min":0.94499999999999995},"last":0.50900000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56999999999999995,"15min":0.58199999999999996},"min":{"1min":0.40300000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.747,"5min":1.046,"15min":1.046},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.42999999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76000000000000001,"5min":0.86399999999999999,"15min":1.075},"last":0.44600000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80700000000000005,"5min":0.83499999999999996,"15min":0.96099999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.44400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76900000000000002,"5min":0.88800000000000001,"15min":0.90100000000000002},"last":0.48299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574931,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298476,"kb_used_data":1188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073364,"statfs":{"total":96636764160,"available":96331124736,"internally_reserved":0,"allocated":1216512,"data_stored":584884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.317,"15min":0.312},"min":{"1min":0.22500000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.45300000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.22600000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.36199999999999999,"15min":0.32900000000000001},"min":{"1min":0.27600000000000002,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.502,"5min":0.622,"15min":0.622},"last":0.32600000000000001}]},{"osd":1,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34499999999999997,"15min":0.32900000000000001},"min":{"1min":0.23499999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.46600000000000003,"5min":0.56599999999999995,"15min":0.60499999999999998},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.378,"15min":0.36799999999999999},"min":{"1min":0.26700000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.55600000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.37}]},{"osd":3,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.38700000000000001,"15min":0.35699999999999998},"min":{"1min":0.28599999999999998,"5min":0.252,"15min":0.20499999999999999},"max":{"1min":0.55000000000000004,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38,"15min":0.36199999999999999},"min":{"1min":0.28499999999999998,"5min":0.25,"15min":0.22800000000000001},"max":{"1min":0.58099999999999996,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.35599999999999998}]},{"osd":4,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.35299999999999998,"5min":0.33400000000000002,"15min":0.315},"max":{"1min":0.78000000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.441,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81100000000000005,"5min":0.82699999999999996,"15min":0.84899999999999998},"last":0.57799999999999996}]},{"osd":5,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73899999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.42799999999999999,"5min":0.36599999999999999,"15min":0.33800000000000002},"max":{"1min":0.80100000000000005,"5min":0.81799999999999995,"15min":0.83399999999999996},"last":0.50600000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.61699999999999999},"min":{"1min":0.42199999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.871,"5min":0.89000000000000001,"15min":0.89100000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.621,"15min":0.622},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79300000000000004,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.61599999999999999}]},{"osd":7,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58999999999999997,"15min":0.60799999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.36799999999999999},"max":{"1min":0.84199999999999997,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60999999999999999,"15min":0.622},"min":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.749,"5min":0.874,"15min":0.874},"last":0.58699999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298476,"kb_used_data":1188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073364,"statfs":{"total":96636764160,"available":96331124736,"internally_reserved":0,"allocated":1216512,"data_stored":584884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.314,"15min":0.41499999999999998},"min":{"1min":0.20599999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.433,"5min":0.52700000000000002,"15min":3.3210000000000002},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34399999999999997,"15min":0.35599999999999998},"min":{"1min":0.21099999999999999,"5min":0.192,"15min":0.192},"max":{"1min":0.436,"5min":0.66800000000000004,"15min":1.218},"last":0.23499999999999999}]},{"osd":1,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.33700000000000002,"15min":0.33900000000000002},"min":{"1min":0.23699999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.40600000000000003,"5min":0.63600000000000001,"15min":0.86299999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.36499999999999999,"15min":0.45600000000000002},"min":{"1min":0.28399999999999997,"5min":0.218,"15min":0.218},"max":{"1min":0.46000000000000002,"5min":0.66000000000000003,"15min":3.359},"last":0.38400000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.36399999999999999,"15min":0.377},"min":{"1min":0.27800000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.55500000000000005,"5min":0.71099999999999997,"15min":1.1100000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.38300000000000001,"15min":0.58899999999999997},"min":{"1min":0.28399999999999997,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46100000000000002,"5min":0.59899999999999998,"15min":7.069},"last":0.40400000000000003}]},{"osd":4,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55900000000000005,"15min":0.57999999999999996},"min":{"1min":0.41599999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.66300000000000003,"5min":0.89400000000000002,"15min":1.165},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.58599999999999997,"15min":0.59299999999999997},"min":{"1min":0.379,"5min":0.375,"15min":0.375},"max":{"1min":0.753,"5min":1.02,"15min":1.02},"last":0.628}]},{"osd":5,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.58199999999999996,"15min":0.67500000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71499999999999997,"5min":0.92500000000000004,"15min":3.4140000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57699999999999996,"15min":0.60099999999999998},"min":{"1min":0.39200000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.96899999999999997,"15min":1.1930000000000001},"last":0.56799999999999995}]},{"osd":6,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.60299999999999998,"15min":0.61299999999999999},"min":{"1min":0.42599999999999999,"5min":0.38,"15min":0.378},"max":{"1min":0.83699999999999997,"5min":1.054,"15min":1.054},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61899999999999999,"15min":0.70599999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.41999999999999998},"max":{"1min":0.80800000000000005,"5min":0.876,"15min":3.3879999999999999},"last":0.502}]},{"osd":7,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85599999999999998,"5min":0.99099999999999999,"15min":1.137},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.436,"5min":0.378,"15min":0.378},"max":{"1min":0.67600000000000005,"5min":0.97899999999999998,"15min":1.014},"last":0.53800000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298476,"kb_used_data":1188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073364,"statfs":{"total":96636764160,"available":96331124736,"internally_reserved":0,"allocated":1216512,"data_stored":584884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.502,"15min":0.48799999999999999},"min":{"1min":0.316,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73799999999999999,"5min":0.74299999999999999,"15min":0.76800000000000002},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52100000000000002,"15min":0.504},"min":{"1min":0.36099999999999999,"5min":0.33100000000000002,"15min":0.28299999999999997},"max":{"1min":0.71399999999999997,"5min":0.73099999999999998,"15min":0.77400000000000002},"last":0.35599999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71199999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.505},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51500000000000001,"15min":0.53300000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70499999999999996,"5min":0.71799999999999997,"15min":0.82899999999999996},"last":0.442}]},{"osd":2,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.32800000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.81000000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.372,"5min":0.36099999999999999,"15min":0.315},"max":{"1min":0.69399999999999995,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.58299999999999996}]},{"osd":3,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55200000000000005,"15min":0.52400000000000002},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.35499999999999998},"max":{"1min":0.73899999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.79500000000000004},"last":0.55400000000000005}]},{"osd":5,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.436,"15min":0.48799999999999999},"min":{"1min":0.30399999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.622,"5min":0.77800000000000002,"15min":0.86899999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46800000000000003,"15min":0.49299999999999999},"min":{"1min":0.29499999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.65200000000000002,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.52900000000000003}]},{"osd":6,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.496,"15min":0.53300000000000003},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53500000000000003,"15min":0.56799999999999995},"min":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69999999999999996,"5min":0.88,"15min":0.91200000000000003},"last":0.54200000000000004}]},{"osd":7,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51700000000000002,"15min":0.56299999999999994},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.67900000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53400000000000003,"15min":0.58299999999999996},"min":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72899999999999998,"5min":0.81299999999999994,"15min":0.85699999999999998},"last":0.72799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574931,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298476,"kb_used_data":1188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073364,"statfs":{"total":96636764160,"available":96331124736,"internally_reserved":0,"allocated":1216512,"data_stored":584884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50900000000000001,"15min":0.47499999999999998},"min":{"1min":0.28599999999999998,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.66400000000000003,"5min":0.88,"15min":0.88},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.50800000000000001,"15min":0.52400000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.64700000000000002,"5min":0.86499999999999999,"15min":1.2549999999999999},"last":0.55600000000000005}]},{"osd":1,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.314,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.59999999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.32300000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64300000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58999999999999997}]},{"osd":2,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53300000000000003,"15min":0.50700000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.30399999999999999},"max":{"1min":0.61899999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.503,"5min":0.54300000000000004,"15min":0.55900000000000005},"min":{"1min":0.35399999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67700000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.47999999999999998}]},{"osd":3,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.54000000000000004,"15min":0.56599999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57899999999999996,"5min":0.79900000000000004,"15min":0.95099999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.51800000000000002,"15min":0.54800000000000004},"min":{"1min":0.33200000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70499999999999996,"5min":0.79100000000000004,"15min":1.3999999999999999},"last":0.57599999999999996}]},{"osd":4,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47399999999999998,"15min":0.48499999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.75700000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.499,"15min":0.50600000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.68999999999999995,"5min":0.70399999999999996,"15min":0.80700000000000005},"last":0.71299999999999997}]},{"osd":6,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.495,"15min":0.51500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.67800000000000005,"5min":0.77200000000000002,"15min":0.85799999999999998},"last":0.62},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78100000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.85499999999999998}]},{"osd":7,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54700000000000004,"15min":0.57399999999999995},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.83299999999999996},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71399999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.73299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:44:55.805 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:44:55.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.951+0000 7f5f4f06f700 1 Processor -- start 2023-10-06T10:44:55.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.951+0000 7f5f4f06f700 1 -- start start 2023-10-06T10:44:55.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.952+0000 7f5f4f06f700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5f480cbc30 0x7f5f480cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:55.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.952+0000 7f5f4f06f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f480cb7a0 con 0x7f5f48159440 2023-10-06T10:44:55.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.952+0000 7f5f4f06f700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f480b0860 con 0x7f5f480b2100 2023-10-06T10:44:55.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.952+0000 7f5f4f06f700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f480cb150 con 0x7f5f480cbc30 2023-10-06T10:44:55.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.952+0000 7f5f4e06d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5f48159440 0x7f5f480b3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:43924/0 (socket says 172.21.15.92:43924) 2023-10-06T10:44:55.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.952+0000 7f5f4e06d700 1 -- 172.21.15.92:0/3063779492 learned_addr learned my addr 172.21.15.92:0/3063779492 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:55.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.952+0000 7f5f4d86c700 1 --2- 172.21.15.92:0/3063779492 >> v2:172.21.15.124:3300/0 conn(0x7f5f480cbc30 0x7f5f480cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:55.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.952+0000 7f5f4d06b700 1 -- 172.21.15.92:0/3063779492 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2397910301 0 0) 0x7f5f480cb7a0 con 0x7f5f48159440 2023-10-06T10:44:55.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.952+0000 7f5f4d06b700 1 -- 172.21.15.92:0/3063779492 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f34003570 con 0x7f5f48159440 2023-10-06T10:44:55.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4d06b700 1 -- 172.21.15.92:0/3063779492 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 493270788 0 0) 0x7f5f34003570 con 0x7f5f48159440 2023-10-06T10:44:55.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4d06b700 1 -- 172.21.15.92:0/3063779492 >> v2:172.21.15.124:3300/0 conn(0x7f5f480cbc30 msgr2=0x7f5f480cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:55.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4d06b700 1 --2- 172.21.15.92:0/3063779492 >> v2:172.21.15.124:3300/0 conn(0x7f5f480cbc30 0x7f5f480cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4d06b700 1 -- 172.21.15.92:0/3063779492 >> v1:172.21.15.124:6789/0 conn(0x7f5f480b2100 legacy=0x7f5f480b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:55.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4d06b700 1 -- 172.21.15.92:0/3063779492 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f480cb430 con 0x7f5f48159440 2023-10-06T10:44:55.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4d06b700 1 -- 172.21.15.92:0/3063779492 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f5f440017f0 con 0x7f5f48159440 2023-10-06T10:44:55.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4d06b700 1 -- 172.21.15.92:0/3063779492 >> v1:172.21.15.92:6789/0 conn(0x7f5f48159440 legacy=0x7f5f480b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:55.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4d06b700 1 --2- 172.21.15.92:0/3063779492 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f34003a60 0x7f5f34003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:55.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4d06b700 1 --2- 172.21.15.92:0/3063779492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f34004810 0x7f5f34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:55.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4d06b700 1 --2- 172.21.15.92:0/3063779492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f340051c0 0x7f5f3400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:55.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4d06b700 1 -- 172.21.15.92:0/3063779492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f480cb150 con 0x7f5f340051c0 2023-10-06T10:44:55.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4d06b700 1 -- 172.21.15.92:0/3063779492 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3400c0d0 con 0x7f5f34003a60 2023-10-06T10:44:55.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4d06b700 1 -- 172.21.15.92:0/3063779492 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3400c290 con 0x7f5f34004810 2023-10-06T10:44:55.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4e86e700 1 --2- 172.21.15.92:0/3063779492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f340051c0 0x7f5f3400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:55.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4e06d700 1 --2- 172.21.15.92:0/3063779492 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f34003a60 0x7f5f34003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:55.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4d86c700 1 --2- 172.21.15.92:0/3063779492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f34004810 0x7f5f34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:55.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4e86e700 1 -- 172.21.15.92:0/3063779492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f34004810 msgr2=0x7f5f34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:55.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4e86e700 1 --2- 172.21.15.92:0/3063779492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f34004810 0x7f5f34004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4e86e700 1 -- 172.21.15.92:0/3063779492 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f34003a60 msgr2=0x7f5f34003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:55.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4e86e700 1 --2- 172.21.15.92:0/3063779492 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f34003a60 0x7f5f34003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4e86e700 1 -- 172.21.15.92:0/3063779492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f34003570 con 0x7f5f340051c0 2023-10-06T10:44:55.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4e86e700 1 -- 172.21.15.92:0/3063779492 reap_dead start 2023-10-06T10:44:55.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4e86e700 1 --2- 172.21.15.92:0/3063779492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f340051c0 0x7f5f3400ba60 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f5f400018e0 tx=0x7f5f40007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb9497c9e2b9ad8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:55.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4d06b700 1 -- 172.21.15.92:0/3063779492 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f40012420 con 0x7f5f340051c0 2023-10-06T10:44:55.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.953+0000 7f5f4d06b700 1 -- 172.21.15.92:0/3063779492 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f400125c0 con 0x7f5f340051c0 2023-10-06T10:44:55.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.954+0000 7f5f4d06b700 1 -- 172.21.15.92:0/3063779492 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f40012980 con 0x7f5f340051c0 2023-10-06T10:44:55.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.954+0000 7f5f4f06f700 1 -- 172.21.15.92:0/3063779492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f340051c0 msgr2=0x7f5f3400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:55.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.954+0000 7f5f4f06f700 1 --2- 172.21.15.92:0/3063779492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f340051c0 0x7f5f3400ba60 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f5f400018e0 tx=0x7f5f40007d30 comp rx=0 tx=0).stop 2023-10-06T10:44:55.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.954+0000 7f5f4f06f700 1 -- 172.21.15.92:0/3063779492 shutdown_connections 2023-10-06T10:44:55.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.954+0000 7f5f4f06f700 1 --2- 172.21.15.92:0/3063779492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f340051c0 0x7f5f3400ba60 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.954+0000 7f5f4f06f700 1 -- 172.21.15.92:0/3063779492 >> 172.21.15.92:0/3063779492 conn(0x7f5f4806ece0 msgr2=0x7f5f4806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:55.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.954+0000 7f5f4f06f700 1 -- 172.21.15.92:0/3063779492 shutdown_connections 2023-10-06T10:44:55.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.954+0000 7f5f4f06f700 1 -- 172.21.15.92:0/3063779492 wait complete. 2023-10-06T10:44:55.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.955+0000 7f5f4f06f700 1 Processor -- start 2023-10-06T10:44:55.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.955+0000 7f5f4f06f700 1 -- start start 2023-10-06T10:44:55.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.956+0000 7f5f4f06f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f340051c0 0x7f5f480c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:55.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.956+0000 7f5f4f06f700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f48159a90 0x7f5f481550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:55.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.956+0000 7f5f4f06f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f48155820 0x7f5f4815a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:55.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.956+0000 7f5f4f06f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4816a7b0 con 0x7f5f340051c0 2023-10-06T10:44:55.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.956+0000 7f5f4f06f700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f481555e0 con 0x7f5f48159a90 2023-10-06T10:44:55.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.956+0000 7f5f4f06f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4816a4d0 con 0x7f5f48155820 2023-10-06T10:44:55.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.956+0000 7f5f4e06d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f340051c0 0x7f5f480c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:55.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.956+0000 7f5f4e06d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f340051c0 0x7f5f480c72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33788/0 (socket says 172.21.15.92:33788) 2023-10-06T10:44:55.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.956+0000 7f5f4e06d700 1 -- 172.21.15.92:0/2453183638 learned_addr learned my addr 172.21.15.92:0/2453183638 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:55.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.956+0000 7f5f4e86e700 1 --2- 172.21.15.92:0/2453183638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f48155820 0x7f5f4815a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:55.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.956+0000 7f5f4d86c700 1 --2- 172.21.15.92:0/2453183638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f48159a90 0x7f5f481550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:55.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.956+0000 7f5f4e06d700 1 -- 172.21.15.92:0/2453183638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f48155820 msgr2=0x7f5f4815a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:55.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.956+0000 7f5f4e06d700 1 --2- 172.21.15.92:0/2453183638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f48155820 0x7f5f4815a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.956+0000 7f5f4e06d700 1 -- 172.21.15.92:0/2453183638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f48159a90 msgr2=0x7f5f481550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:55.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.956+0000 7f5f4e06d700 1 --2- 172.21.15.92:0/2453183638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f48159a90 0x7f5f481550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:55.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.956+0000 7f5f4e06d700 1 -- 172.21.15.92:0/2453183638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4815b530 con 0x7f5f340051c0 2023-10-06T10:44:55.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.956+0000 7f5f4e06d700 1 --2- 172.21.15.92:0/2453183638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f340051c0 0x7f5f480c72d0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f5f44001920 tx=0x7f5f440021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:55.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.957+0000 7f5f3e7fc700 1 -- 172.21.15.92:0/2453183638 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4400ad10 con 0x7f5f340051c0 2023-10-06T10:44:55.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.957+0000 7f5f3e7fc700 1 -- 172.21.15.92:0/2453183638 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4400ae70 con 0x7f5f340051c0 2023-10-06T10:44:55.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.957+0000 7f5f4f06f700 1 -- 172.21.15.92:0/2453183638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f4815b810 con 0x7f5f340051c0 2023-10-06T10:44:55.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.957+0000 7f5f3e7fc700 1 -- 172.21.15.92:0/2453183638 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f44013620 con 0x7f5f340051c0 2023-10-06T10:44:55.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.957+0000 7f5f4f06f700 1 -- 172.21.15.92:0/2453183638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4815ae50 con 0x7f5f340051c0 2023-10-06T10:44:55.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.958+0000 7f5f4f06f700 1 -- 172.21.15.92:0/2453183638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4815b390 con 0x7f5f340051c0 2023-10-06T10:44:55.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.958+0000 7f5f3e7fc700 1 -- 172.21.15.92:0/2453183638 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5f44013990 con 0x7f5f340051c0 2023-10-06T10:44:55.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.958+0000 7f5f3e7fc700 1 --2- 172.21.15.92:0/2453183638 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5f2c066f60 0x7f5f2c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:55.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.958+0000 7f5f3e7fc700 1 -- 172.21.15.92:0/2453183638 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5f44085520 con 0x7f5f340051c0 2023-10-06T10:44:55.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.958+0000 7f5f4d86c700 1 --2- 172.21.15.92:0/2453183638 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5f2c066f60 0x7f5f2c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:55.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.959+0000 7f5f4d86c700 1 --2- 172.21.15.92:0/2453183638 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5f2c066f60 0x7f5f2c069410 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5f38006ed0 tx=0x7f5f38008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:55.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:55.963+0000 7f5f3e7fc700 1 -- 172.21.15.92:0/2453183638 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f4815b390 con 0x7f5f340051c0 2023-10-06T10:44:56.026 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-10-06T10:44:56.025+0000 7f0bb9087700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T10:44:56.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.141+0000 7f5f4f06f700 1 -- 172.21.15.92:0/2453183638 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5f4815b390 con 0x7f5f2c066f60 2023-10-06T10:44:56.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.149+0000 7f5f3e7fc700 1 -- 172.21.15.92:0/2453183638 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+297279 (secure 0 0 0) 0x7f5f4815b390 con 0x7f5f2c066f60 2023-10-06T10:44:56.153 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:56.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.152+0000 7f5f4f06f700 1 -- 172.21.15.92:0/2453183638 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5f2c066f60 msgr2=0x7f5f2c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:56.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.152+0000 7f5f4f06f700 1 --2- 172.21.15.92:0/2453183638 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5f2c066f60 0x7f5f2c069410 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5f38006ed0 tx=0x7f5f38008040 comp rx=0 tx=0).stop 2023-10-06T10:44:56.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.152+0000 7f5f4f06f700 1 -- 172.21.15.92:0/2453183638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f340051c0 msgr2=0x7f5f480c72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:56.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.152+0000 7f5f4f06f700 1 --2- 172.21.15.92:0/2453183638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f340051c0 0x7f5f480c72d0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f5f44001920 tx=0x7f5f440021d0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.152+0000 7f5f4f06f700 1 -- 172.21.15.92:0/2453183638 shutdown_connections 2023-10-06T10:44:56.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.152+0000 7f5f4f06f700 1 --2- 172.21.15.92:0/2453183638 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5f2c066f60 0x7f5f2c069410 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.152+0000 7f5f4f06f700 1 --2- 172.21.15.92:0/2453183638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f48155820 0x7f5f4815a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.152+0000 7f5f4f06f700 1 --2- 172.21.15.92:0/2453183638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f340051c0 0x7f5f480c72d0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.152+0000 7f5f4f06f700 1 --2- 172.21.15.92:0/2453183638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f48159a90 0x7f5f481550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.152+0000 7f5f4f06f700 1 -- 172.21.15.92:0/2453183638 >> 172.21.15.92:0/2453183638 conn(0x7f5f4806ece0 msgr2=0x7f5f48068080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:56.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.152+0000 7f5f4f06f700 1 -- 172.21.15.92:0/2453183638 shutdown_connections 2023-10-06T10:44:56.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.152+0000 7f5f4f06f700 1 -- 172.21.15.92:0/2453183638 wait complete. 2023-10-06T10:44:56.156 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:44:56.165 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2023-10-06T10:44:54.168147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2388964,"kb_used_data":10660,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752585756,"statfs":{"total":773094113280,"available":770647814144,"internally_reserved":0,"allocated":10915840,"data_stored":5858455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.452541"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":86,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-06T10:44:45.702353+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:44:45.702353+0000","last_peered":"2023-10-06T10:44:45.702353+0000","last_clean":"2023-10-06T10:44:45.702353+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:44:45.702353+0000","last_undegraded":"2023-10-06T10:44:45.702353+0000","last_fullsized":"2023-10-06T10:44:45.702353+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-06T10:44:46.155898+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:44:46.155898+0000","last_peered":"2023-10-06T10:44:46.155898+0000","last_clean":"2023-10-06T10:44:46.155898+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:44:46.155898+0000","last_undegraded":"2023-10-06T10:44:46.155898+0000","last_fullsized":"2023-10-06T10:44:46.155898+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":80,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-06T10:44:45.964853+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:44:45.964853+0000","last_peered":"2023-10-06T10:44:45.964853+0000","last_clean":"2023-10-06T10:44:45.964853+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:44:45.964853+0000","last_undegraded":"2023-10-06T10:44:45.964853+0000","last_fullsized":"2023-10-06T10:44:45.964853+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":222,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-06T10:44:46.477634+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:44:46.477634+0000","last_peered":"2023-10-06T10:44:46.477634+0000","last_clean":"2023-10-06T10:44:46.477634+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:44:46.477634+0000","last_undegraded":"2023-10-06T10:44:46.477634+0000","last_fullsized":"2023-10-06T10:44:46.477634+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":72,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-06T10:44:46.477558+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:44:46.477558+0000","last_peered":"2023-10-06T10:44:46.477558+0000","last_clean":"2023-10-06T10:44:46.477558+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:44:46.477558+0000","last_undegraded":"2023-10-06T10:44:46.477558+0000","last_fullsized":"2023-10-06T10:44:46.477558+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-06T10:44:46.477560+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:44:46.477560+0000","last_peered":"2023-10-06T10:44:46.477560+0000","last_clean":"2023-10-06T10:44:46.477560+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:44:46.477560+0000","last_undegraded":"2023-10-06T10:44:46.477560+0000","last_fullsized":"2023-10-06T10:44:46.477560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-06T10:44:45.702568+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:44:45.702568+0000","last_peered":"2023-10-06T10:44:45.702568+0000","last_clean":"2023-10-06T10:44:45.702568+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:44:45.702568+0000","last_undegraded":"2023-10-06T10:44:45.702568+0000","last_fullsized":"2023-10-06T10:44:45.702568+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-06T10:44:46.477301+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:44:46.477301+0000","last_peered":"2023-10-06T10:44:46.477301+0000","last_clean":"2023-10-06T10:44:46.477301+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:44:46.477301+0000","last_undegraded":"2023-10-06T10:44:46.477301+0000","last_fullsized":"2023-10-06T10:44:46.477301+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:50.715731+0000","last_change":"2023-10-06T10:44:50.715731+0000","last_active":"2023-10-06T10:44:50.715731+0000","last_peered":"2023-10-06T10:44:50.715731+0000","last_clean":"2023-10-06T10:44:50.715731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:50.715731+0000","last_undegraded":"2023-10-06T10:44:50.715731+0000","last_fullsized":"2023-10-06T10:44:50.715731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T10:44:49.710036+0000","last_change":"2023-10-06T10:44:49.710036+0000","last_active":"2023-10-06T10:44:49.710036+0000","last_peered":"2023-10-06T10:44:49.710036+0000","last_clean":"2023-10-06T10:44:49.710036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T10:44:49.710036+0000","last_undegraded":"2023-10-06T10:44:49.710036+0000","last_fullsized":"2023-10-06T10:44:49.710036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-06T10:44:46.477310+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:44:46.477310+0000","last_peered":"2023-10-06T10:44:46.477310+0000","last_clean":"2023-10-06T10:44:46.477310+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:44:46.477310+0000","last_undegraded":"2023-10-06T10:44:46.477310+0000","last_fullsized":"2023-10-06T10:44:46.477310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574933,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299056,"kb_used_data":1768,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072784,"statfs":{"total":96636764160,"available":96330530816,"internally_reserved":0,"allocated":1810432,"data_stored":1175252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.51000000000000001,"15min":0.48799999999999999},"min":{"1min":0.32800000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63900000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.434},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.53800000000000003,"15min":0.52100000000000002},"min":{"1min":0.29399999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.9700000000000002,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.496,"15min":0.46700000000000003},"min":{"1min":0.32000000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.64800000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.54500000000000004,"15min":0.51300000000000001},"min":{"1min":0.38400000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":3.0059999999999998,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.52200000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56699999999999995,"15min":0.52200000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.497,"5min":0.51900000000000002,"15min":0.499},"min":{"1min":0.371,"5min":0.314,"15min":0.314},"max":{"1min":0.66800000000000004,"5min":0.747,"15min":0.79800000000000004},"last":0.56999999999999995}]},{"osd":3,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55200000000000005,"15min":0.51600000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73099999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.40600000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.71899999999999997,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.67500000000000004}]},{"osd":4,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42199999999999999,"15min":0.46200000000000002},"min":{"1min":0.314,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61899999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47299999999999998,"15min":0.499},"min":{"1min":0.30199999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64900000000000002,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69099999999999995}]},{"osd":5,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.48199999999999998},"min":{"1min":0.33000000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.81100000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.51800000000000002,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85399999999999998,"5min":1.577,"15min":1.577},"last":0.60299999999999998}]},{"osd":6,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.56499999999999995},"min":{"1min":0.39700000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.67200000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.56200000000000006,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69599999999999995,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.63600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298472,"kb_used_data":1184,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073368,"statfs":{"total":96636764160,"available":96331128832,"internally_reserved":0,"allocated":1212416,"data_stored":583531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.495},"min":{"1min":0.34599999999999997,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":0.70399999999999996,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52100000000000002,"15min":0.45500000000000002},"min":{"1min":0.39700000000000002,"5min":0.32500000000000001,"15min":0.32000000000000001},"max":{"1min":0.67400000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.41899999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52000000000000002,"15min":0.46000000000000002},"min":{"1min":0.39300000000000002,"5min":0.32200000000000001,"15min":0.28000000000000003},"max":{"1min":0.70799999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54500000000000004,"15min":0.48899999999999999},"min":{"1min":0.42399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73499999999999999,"5min":0.751,"15min":0.751},"last":0.38300000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52700000000000002,"15min":0.48399999999999999},"min":{"1min":0.39600000000000002,"5min":0.32500000000000001,"15min":0.309},"max":{"1min":0.65400000000000003,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54500000000000004,"15min":0.48699999999999999},"min":{"1min":0.443,"5min":0.34999999999999998,"15min":0.34300000000000003},"max":{"1min":0.66100000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55200000000000005,"15min":0.50600000000000001},"min":{"1min":0.46500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75600000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.55500000000000005,"15min":0.51900000000000002},"min":{"1min":0.40699999999999997,"5min":0.36399999999999999,"15min":0.34499999999999997},"max":{"1min":0.74199999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.52700000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.48699999999999999,"15min":0.48099999999999998},"min":{"1min":0.34999999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.78300000000000003,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.436},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.498,"15min":0.48899999999999999},"min":{"1min":0.377,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79400000000000004,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.51000000000000001,"15min":0.49199999999999999},"min":{"1min":0.34200000000000003,"5min":0.28899999999999998,"15min":0.28299999999999997},"max":{"1min":0.83599999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.38700000000000001,"5min":0.33900000000000002,"15min":0.308},"max":{"1min":0.77800000000000002,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.55900000000000005}]},{"osd":7,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53200000000000003,"15min":0.52000000000000002},"min":{"1min":0.36599999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71899999999999997,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.502},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57399999999999995,"15min":0.54800000000000004},"min":{"1min":0.40000000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.871,"5min":0.879,"15min":0.879},"last":0.54000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299056,"kb_used_data":1768,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072784,"statfs":{"total":96636764160,"available":96330530816,"internally_reserved":0,"allocated":1810432,"data_stored":1175252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.20399999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.35599999999999998,"15min":0.34799999999999998},"min":{"1min":0.23400000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.57099999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.38200000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.33000000000000002,"15min":0.32000000000000001},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.46999999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.372},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38100000000000001,"15min":0.36699999999999999},"min":{"1min":0.27700000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.32400000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39500000000000002,"15min":0.38},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.60999999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.38700000000000001,"15min":0.377},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.26100000000000001}]},{"osd":4,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.40200000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73099999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.746},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.753}]},{"osd":5,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86599999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.55000000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.35699999999999998},"max":{"1min":0.80900000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.76400000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.59899999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.36699999999999999},"max":{"1min":0.92200000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.435,"5min":0.40000000000000002,"15min":0.39800000000000002},"max":{"1min":0.77000000000000002,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.70399999999999996}]},{"osd":7,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.39100000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.84299999999999997,"5min":0.996,"15min":0.996},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.42099999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.67400000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298476,"kb_used_data":1188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073364,"statfs":{"total":96636764160,"available":96331124736,"internally_reserved":0,"allocated":1216512,"data_stored":584884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.33000000000000002,"15min":0.32000000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.433,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.376,"15min":0.34999999999999998},"min":{"1min":0.28399999999999997,"5min":0.25,"15min":0.25},"max":{"1min":0.504,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.32100000000000001,"15min":0.309},"min":{"1min":0.246,"5min":0.191,"15min":0.191},"max":{"1min":0.41799999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35599999999999998,"15min":0.33200000000000002},"min":{"1min":0.26900000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.46000000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.32200000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35399999999999998,"15min":0.34399999999999997},"min":{"1min":0.25600000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.49299999999999999,"5min":0.622,"15min":0.622},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36599999999999999,"15min":0.35299999999999998},"min":{"1min":0.28999999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.47899999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.379}]},{"osd":4,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.41799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78900000000000003,"5min":0.81200000000000006,"15min":0.83299999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56599999999999995,"15min":0.54100000000000004},"min":{"1min":0.38600000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.753,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.51900000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.36899999999999999},"max":{"1min":0.80300000000000005,"5min":0.91900000000000004,"15min":1.0069999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.41299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76700000000000002,"5min":0.80900000000000005,"15min":0.94499999999999995},"last":0.50900000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56999999999999995,"15min":0.58199999999999996},"min":{"1min":0.40300000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.747,"5min":1.046,"15min":1.046},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.42999999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76000000000000001,"5min":0.86399999999999999,"15min":1.075},"last":0.44600000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80700000000000005,"5min":0.83499999999999996,"15min":0.96099999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.44400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76900000000000002,"5min":0.88800000000000001,"15min":0.90100000000000002},"last":0.48299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574931,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298476,"kb_used_data":1188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073364,"statfs":{"total":96636764160,"available":96331124736,"internally_reserved":0,"allocated":1216512,"data_stored":584884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.317,"15min":0.312},"min":{"1min":0.22500000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.45300000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.22600000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.36199999999999999,"15min":0.32900000000000001},"min":{"1min":0.27600000000000002,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.502,"5min":0.622,"15min":0.622},"last":0.32600000000000001}]},{"osd":1,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34499999999999997,"15min":0.32900000000000001},"min":{"1min":0.23499999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.46600000000000003,"5min":0.56599999999999995,"15min":0.60499999999999998},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.378,"15min":0.36799999999999999},"min":{"1min":0.26700000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.55600000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.37}]},{"osd":3,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.38700000000000001,"15min":0.35699999999999998},"min":{"1min":0.28599999999999998,"5min":0.252,"15min":0.20499999999999999},"max":{"1min":0.55000000000000004,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38,"15min":0.36199999999999999},"min":{"1min":0.28499999999999998,"5min":0.25,"15min":0.22800000000000001},"max":{"1min":0.58099999999999996,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.35599999999999998}]},{"osd":4,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.35299999999999998,"5min":0.33400000000000002,"15min":0.315},"max":{"1min":0.78000000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.441,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81100000000000005,"5min":0.82699999999999996,"15min":0.84899999999999998},"last":0.57799999999999996}]},{"osd":5,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73899999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.42799999999999999,"5min":0.36599999999999999,"15min":0.33800000000000002},"max":{"1min":0.80100000000000005,"5min":0.81799999999999995,"15min":0.83399999999999996},"last":0.50600000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.61699999999999999},"min":{"1min":0.42199999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.871,"5min":0.89000000000000001,"15min":0.89100000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.621,"15min":0.622},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79300000000000004,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.61599999999999999}]},{"osd":7,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58999999999999997,"15min":0.60799999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.36799999999999999},"max":{"1min":0.84199999999999997,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60999999999999999,"15min":0.622},"min":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.749,"5min":0.874,"15min":0.874},"last":0.58699999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298476,"kb_used_data":1188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073364,"statfs":{"total":96636764160,"available":96331124736,"internally_reserved":0,"allocated":1216512,"data_stored":584884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.314,"15min":0.41499999999999998},"min":{"1min":0.20599999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.433,"5min":0.52700000000000002,"15min":3.3210000000000002},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34399999999999997,"15min":0.35599999999999998},"min":{"1min":0.21099999999999999,"5min":0.192,"15min":0.192},"max":{"1min":0.436,"5min":0.66800000000000004,"15min":1.218},"last":0.23499999999999999}]},{"osd":1,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.33700000000000002,"15min":0.33900000000000002},"min":{"1min":0.23699999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.40600000000000003,"5min":0.63600000000000001,"15min":0.86299999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.36499999999999999,"15min":0.45600000000000002},"min":{"1min":0.28399999999999997,"5min":0.218,"15min":0.218},"max":{"1min":0.46000000000000002,"5min":0.66000000000000003,"15min":3.359},"last":0.38400000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.36399999999999999,"15min":0.377},"min":{"1min":0.27800000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.55500000000000005,"5min":0.71099999999999997,"15min":1.1100000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.38300000000000001,"15min":0.58899999999999997},"min":{"1min":0.28399999999999997,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46100000000000002,"5min":0.59899999999999998,"15min":7.069},"last":0.40400000000000003}]},{"osd":4,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55900000000000005,"15min":0.57999999999999996},"min":{"1min":0.41599999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.66300000000000003,"5min":0.89400000000000002,"15min":1.165},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.58599999999999997,"15min":0.59299999999999997},"min":{"1min":0.379,"5min":0.375,"15min":0.375},"max":{"1min":0.753,"5min":1.02,"15min":1.02},"last":0.628}]},{"osd":5,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.58199999999999996,"15min":0.67500000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71499999999999997,"5min":0.92500000000000004,"15min":3.4140000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57699999999999996,"15min":0.60099999999999998},"min":{"1min":0.39200000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.96899999999999997,"15min":1.1930000000000001},"last":0.56799999999999995}]},{"osd":6,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.60299999999999998,"15min":0.61299999999999999},"min":{"1min":0.42599999999999999,"5min":0.38,"15min":0.378},"max":{"1min":0.83699999999999997,"5min":1.054,"15min":1.054},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61899999999999999,"15min":0.70599999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.41999999999999998},"max":{"1min":0.80800000000000005,"5min":0.876,"15min":3.3879999999999999},"last":0.502}]},{"osd":7,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85599999999999998,"5min":0.99099999999999999,"15min":1.137},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.436,"5min":0.378,"15min":0.378},"max":{"1min":0.67600000000000005,"5min":0.97899999999999998,"15min":1.014},"last":0.53800000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298476,"kb_used_data":1188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073364,"statfs":{"total":96636764160,"available":96331124736,"internally_reserved":0,"allocated":1216512,"data_stored":584884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.502,"15min":0.48799999999999999},"min":{"1min":0.316,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73799999999999999,"5min":0.74299999999999999,"15min":0.76800000000000002},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52100000000000002,"15min":0.504},"min":{"1min":0.36099999999999999,"5min":0.33100000000000002,"15min":0.28299999999999997},"max":{"1min":0.71399999999999997,"5min":0.73099999999999998,"15min":0.77400000000000002},"last":0.35599999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71199999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.505},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51500000000000001,"15min":0.53300000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70499999999999996,"5min":0.71799999999999997,"15min":0.82899999999999996},"last":0.442}]},{"osd":2,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.32800000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.81000000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.372,"5min":0.36099999999999999,"15min":0.315},"max":{"1min":0.69399999999999995,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.58299999999999996}]},{"osd":3,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55200000000000005,"15min":0.52400000000000002},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.35499999999999998},"max":{"1min":0.73899999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.79500000000000004},"last":0.55400000000000005}]},{"osd":5,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.436,"15min":0.48799999999999999},"min":{"1min":0.30399999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.622,"5min":0.77800000000000002,"15min":0.86899999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46800000000000003,"15min":0.49299999999999999},"min":{"1min":0.29499999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.65200000000000002,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.52900000000000003}]},{"osd":6,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.496,"15min":0.53300000000000003},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53500000000000003,"15min":0.56799999999999995},"min":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69999999999999996,"5min":0.88,"15min":0.91200000000000003},"last":0.54200000000000004}]},{"osd":7,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51700000000000002,"15min":0.56299999999999994},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.67900000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53400000000000003,"15min":0.58299999999999996},"min":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72899999999999998,"5min":0.81299999999999994,"15min":0.85699999999999998},"last":0.72799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574931,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298476,"kb_used_data":1188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073364,"statfs":{"total":96636764160,"available":96331124736,"internally_reserved":0,"allocated":1216512,"data_stored":584884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50900000000000001,"15min":0.47499999999999998},"min":{"1min":0.28599999999999998,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.66400000000000003,"5min":0.88,"15min":0.88},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.50800000000000001,"15min":0.52400000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.64700000000000002,"5min":0.86499999999999999,"15min":1.2549999999999999},"last":0.55600000000000005}]},{"osd":1,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.314,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.59999999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.32300000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64300000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58999999999999997}]},{"osd":2,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53300000000000003,"15min":0.50700000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.30399999999999999},"max":{"1min":0.61899999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.503,"5min":0.54300000000000004,"15min":0.55900000000000005},"min":{"1min":0.35399999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67700000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.47999999999999998}]},{"osd":3,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.54000000000000004,"15min":0.56599999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57899999999999996,"5min":0.79900000000000004,"15min":0.95099999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.51800000000000002,"15min":0.54800000000000004},"min":{"1min":0.33200000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70499999999999996,"5min":0.79100000000000004,"15min":1.3999999999999999},"last":0.57599999999999996}]},{"osd":4,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47399999999999998,"15min":0.48499999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.75700000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.499,"15min":0.50600000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.68999999999999995,"5min":0.70399999999999996,"15min":0.80700000000000005},"last":0.71299999999999997}]},{"osd":6,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.495,"15min":0.51500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.67800000000000005,"5min":0.77200000000000002,"15min":0.85799999999999998},"last":0.62},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78100000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.85499999999999998}]},{"osd":7,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54700000000000004,"15min":0.57399999999999995},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.83299999999999996},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71399999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.73299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:44:56.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:44:56.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.319+0000 7fb95359e700 1 Processor -- start 2023-10-06T10:44:56.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.319+0000 7fb95359e700 1 -- start start 2023-10-06T10:44:56.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.320+0000 7fb95359e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb954159440 0x7fb9540b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:56.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.320+0000 7fb95359e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb9540b35d0 0x7fb9540cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:56.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.320+0000 7fb95359e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9540cb7a0 con 0x7fb9540cc170 2023-10-06T10:44:56.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.320+0000 7fb95359e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9540b0860 con 0x7fb9540b35d0 2023-10-06T10:44:56.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.320+0000 7fb95359e700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9540cb150 con 0x7fb954159440 2023-10-06T10:44:56.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.320+0000 7fb949d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb9540b35d0 0x7fb9540cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:56.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.320+0000 7fb949d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb9540b35d0 0x7fb9540cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33804/0 (socket says 172.21.15.92:33804) 2023-10-06T10:44:56.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.320+0000 7fb949d9b700 1 -- 172.21.15.92:0/3952580252 learned_addr learned my addr 172.21.15.92:0/3952580252 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:56.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.320+0000 7fb95259c700 1 --2- 172.21.15.92:0/3952580252 >> v2:172.21.15.124:3300/0 conn(0x7fb954159440 0x7fb9540b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:56.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.320+0000 7fb949d9b700 1 -- 172.21.15.92:0/3952580252 >> v2:172.21.15.124:3300/0 conn(0x7fb954159440 msgr2=0x7fb9540b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:56.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.320+0000 7fb949d9b700 1 --2- 172.21.15.92:0/3952580252 >> v2:172.21.15.124:3300/0 conn(0x7fb954159440 0x7fb9540b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.320+0000 7fb949d9b700 1 -- 172.21.15.92:0/3952580252 >> v1:172.21.15.124:6789/0 conn(0x7fb9540cc170 legacy=0x7fb9540b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:56.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.320+0000 7fb949d9b700 1 -- 172.21.15.92:0/3952580252 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9540cb430 con 0x7fb9540b35d0 2023-10-06T10:44:56.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.320+0000 7fb949d9b700 1 --2- 172.21.15.92:0/3952580252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9540b35d0 0x7fb9540cbc30 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fb95406f9f0 tx=0x7fb93c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58bd2c60f465d8a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:56.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.320+0000 7fb951d9b700 1 -- 172.21.15.92:0/3952580252 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c00b860 con 0x7fb9540b35d0 2023-10-06T10:44:56.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.321+0000 7fb951d9b700 1 -- 172.21.15.92:0/3952580252 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93c00b9c0 con 0x7fb9540b35d0 2023-10-06T10:44:56.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.321+0000 7fb951d9b700 1 -- 172.21.15.92:0/3952580252 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c00bd60 con 0x7fb9540b35d0 2023-10-06T10:44:56.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.321+0000 7fb95359e700 1 -- 172.21.15.92:0/3952580252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9540b35d0 msgr2=0x7fb9540cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:56.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.321+0000 7fb95359e700 1 --2- 172.21.15.92:0/3952580252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9540b35d0 0x7fb9540cbc30 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fb95406f9f0 tx=0x7fb93c009250 comp rx=0 tx=0).stop 2023-10-06T10:44:56.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.321+0000 7fb95359e700 1 -- 172.21.15.92:0/3952580252 shutdown_connections 2023-10-06T10:44:56.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.321+0000 7fb95359e700 1 --2- 172.21.15.92:0/3952580252 >> v2:172.21.15.124:3300/0 conn(0x7fb954159440 0x7fb9540b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.321+0000 7fb95359e700 1 --2- 172.21.15.92:0/3952580252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9540b35d0 0x7fb9540cbc30 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.321+0000 7fb95359e700 1 -- 172.21.15.92:0/3952580252 >> 172.21.15.92:0/3952580252 conn(0x7fb95406ece0 msgr2=0x7fb95406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:56.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.321+0000 7fb95359e700 1 -- 172.21.15.92:0/3952580252 shutdown_connections 2023-10-06T10:44:56.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.321+0000 7fb95359e700 1 -- 172.21.15.92:0/3952580252 wait complete. 2023-10-06T10:44:56.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.322+0000 7fb95359e700 1 Processor -- start 2023-10-06T10:44:56.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.322+0000 7fb95359e700 1 -- start start 2023-10-06T10:44:56.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.323+0000 7fb95359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9540b35d0 0x7fb9540c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:56.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.323+0000 7fb95359e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9540cc170 0x7fb954159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:56.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.323+0000 7fb95359e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb954159440 0x7fb954159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:56.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.323+0000 7fb95359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95415a4e0 con 0x7fb9540b35d0 2023-10-06T10:44:56.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.323+0000 7fb95359e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9540c6540 con 0x7fb954159440 2023-10-06T10:44:56.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.323+0000 7fb95359e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9540c54a0 con 0x7fb9540cc170 2023-10-06T10:44:56.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.323+0000 7fb95259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9540b35d0 0x7fb9540c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:56.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.323+0000 7fb95259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9540b35d0 0x7fb9540c4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33820/0 (socket says 172.21.15.92:33820) 2023-10-06T10:44:56.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.323+0000 7fb95259c700 1 -- 172.21.15.92:0/1555852920 learned_addr learned my addr 172.21.15.92:0/1555852920 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:56.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.323+0000 7fb949d9b700 1 --2- 172.21.15.92:0/1555852920 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9540cc170 0x7fb954159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:56.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.323+0000 7fb952d9d700 1 --2- 172.21.15.92:0/1555852920 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb954159440 0x7fb954159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:56.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.323+0000 7fb95259c700 1 -- 172.21.15.92:0/1555852920 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9540cc170 msgr2=0x7fb954159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:56.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.323+0000 7fb95259c700 1 --2- 172.21.15.92:0/1555852920 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9540cc170 0x7fb954159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.323+0000 7fb95259c700 1 -- 172.21.15.92:0/1555852920 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb954159440 msgr2=0x7fb954159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:56.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.323+0000 7fb95259c700 1 --2- 172.21.15.92:0/1555852920 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb954159440 0x7fb954159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.323+0000 7fb95259c700 1 -- 172.21.15.92:0/1555852920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb93c0094f0 con 0x7fb9540b35d0 2023-10-06T10:44:56.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.324+0000 7fb952d9d700 1 --2- 172.21.15.92:0/1555852920 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb954159440 0x7fb954159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:44:56.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.324+0000 7fb95259c700 1 --2- 172.21.15.92:0/1555852920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9540b35d0 0x7fb9540c4ac0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fb94400d860 tx=0x7fb944008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:56.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.324+0000 7fb94affd700 1 -- 172.21.15.92:0/1555852920 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9440125f0 con 0x7fb9540b35d0 2023-10-06T10:44:56.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.324+0000 7fb94affd700 1 -- 172.21.15.92:0/1555852920 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb944012750 con 0x7fb9540b35d0 2023-10-06T10:44:56.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.324+0000 7fb95359e700 1 -- 172.21.15.92:0/1555852920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9540c5780 con 0x7fb9540b35d0 2023-10-06T10:44:56.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.324+0000 7fb94affd700 1 -- 172.21.15.92:0/1555852920 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944012b10 con 0x7fb9540b35d0 2023-10-06T10:44:56.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.324+0000 7fb95359e700 1 -- 172.21.15.92:0/1555852920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9540c5d40 con 0x7fb9540b35d0 2023-10-06T10:44:56.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.325+0000 7fb95359e700 1 -- 172.21.15.92:0/1555852920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9540b5da0 con 0x7fb9540b35d0 2023-10-06T10:44:56.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.325+0000 7fb94affd700 1 -- 172.21.15.92:0/1555852920 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb94401e460 con 0x7fb9540b35d0 2023-10-06T10:44:56.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.325+0000 7fb94affd700 1 --2- 172.21.15.92:0/1555852920 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb934066f30 0x7fb9340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:56.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.326+0000 7fb94affd700 1 -- 172.21.15.92:0/1555852920 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb944084e90 con 0x7fb9540b35d0 2023-10-06T10:44:56.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.326+0000 7fb949d9b700 1 --2- 172.21.15.92:0/1555852920 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb934066f30 0x7fb9340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:56.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.326+0000 7fb949d9b700 1 --2- 172.21.15.92:0/1555852920 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb934066f30 0x7fb9340693e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb95406f9f0 tx=0x7fb93c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:56.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.330+0000 7fb94affd700 1 -- 172.21.15.92:0/1555852920 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9440558f0 con 0x7fb9540b35d0 2023-10-06T10:44:56.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.507+0000 7fb95359e700 1 -- 172.21.15.92:0/1555852920 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb9540b16d0 con 0x7fb934066f30 2023-10-06T10:44:56.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.516+0000 7fb94affd700 1 -- 172.21.15.92:0/1555852920 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312370 (secure 0 0 0) 0x7fb9540b16d0 con 0x7fb934066f30 2023-10-06T10:44:56.520 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:56.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.519+0000 7fb95359e700 1 -- 172.21.15.92:0/1555852920 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb934066f30 msgr2=0x7fb9340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:56.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.519+0000 7fb95359e700 1 --2- 172.21.15.92:0/1555852920 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb934066f30 0x7fb9340693e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb95406f9f0 tx=0x7fb93c017000 comp rx=0 tx=0).stop 2023-10-06T10:44:56.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.519+0000 7fb95359e700 1 -- 172.21.15.92:0/1555852920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9540b35d0 msgr2=0x7fb9540c4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:56.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.519+0000 7fb95359e700 1 --2- 172.21.15.92:0/1555852920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9540b35d0 0x7fb9540c4ac0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fb94400d860 tx=0x7fb944008040 comp rx=0 tx=0).stop 2023-10-06T10:44:56.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.519+0000 7fb95359e700 1 -- 172.21.15.92:0/1555852920 shutdown_connections 2023-10-06T10:44:56.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.519+0000 7fb95359e700 1 --2- 172.21.15.92:0/1555852920 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb934066f30 0x7fb9340693e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.519+0000 7fb95359e700 1 --2- 172.21.15.92:0/1555852920 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb954159440 0x7fb954159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.519+0000 7fb95359e700 1 --2- 172.21.15.92:0/1555852920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9540b35d0 0x7fb9540c4ac0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.519+0000 7fb95359e700 1 --2- 172.21.15.92:0/1555852920 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9540cc170 0x7fb954159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.519+0000 7fb95359e700 1 -- 172.21.15.92:0/1555852920 >> 172.21.15.92:0/1555852920 conn(0x7fb95406ece0 msgr2=0x7fb954157480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:56.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.519+0000 7fb95359e700 1 -- 172.21.15.92:0/1555852920 shutdown_connections 2023-10-06T10:44:56.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.519+0000 7fb95359e700 1 -- 172.21.15.92:0/1555852920 wait complete. 2023-10-06T10:44:56.524 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:44:56.532 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2023-10-06T10:44:56.168451+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2393484,"kb_used_data":11084,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752581236,"statfs":{"total":773094113280,"available":770643185664,"internally_reserved":0,"allocated":11350016,"data_stored":6071633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001108"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.755450+0000","last_change":"2023-10-06T10:44:51.742214+0000","last_active":"2023-10-06T10:44:51.755450+0000","last_peered":"2023-10-06T10:44:51.755450+0000","last_clean":"2023-10-06T10:44:51.755450+0000","last_became_active":"2023-10-06T10:44:51.741939+0000","last_became_peered":"2023-10-06T10:44:51.741939+0000","last_unstale":"2023-10-06T10:44:51.755450+0000","last_undegraded":"2023-10-06T10:44:51.755450+0000","last_fullsized":"2023-10-06T10:44:51.755450+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:13:48.153672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.746271+0000","last_change":"2023-10-06T10:44:51.743338+0000","last_active":"2023-10-06T10:44:51.746271+0000","last_peered":"2023-10-06T10:44:51.746271+0000","last_clean":"2023-10-06T10:44:51.746271+0000","last_became_active":"2023-10-06T10:44:51.743074+0000","last_became_peered":"2023-10-06T10:44:51.743074+0000","last_unstale":"2023-10-06T10:44:51.746271+0000","last_undegraded":"2023-10-06T10:44:51.746271+0000","last_fullsized":"2023-10-06T10:44:51.746271+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:59.356737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745137+0000","last_change":"2023-10-06T10:44:51.740363+0000","last_active":"2023-10-06T10:44:51.745137+0000","last_peered":"2023-10-06T10:44:51.745137+0000","last_clean":"2023-10-06T10:44:51.745137+0000","last_became_active":"2023-10-06T10:44:51.739877+0000","last_became_peered":"2023-10-06T10:44:51.739877+0000","last_unstale":"2023-10-06T10:44:51.745137+0000","last_undegraded":"2023-10-06T10:44:51.745137+0000","last_fullsized":"2023-10-06T10:44:51.745137+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:11.372620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757596+0000","last_change":"2023-10-06T10:44:51.749770+0000","last_active":"2023-10-06T10:44:51.757596+0000","last_peered":"2023-10-06T10:44:51.757596+0000","last_clean":"2023-10-06T10:44:51.757596+0000","last_became_active":"2023-10-06T10:44:51.749314+0000","last_became_peered":"2023-10-06T10:44:51.749314+0000","last_unstale":"2023-10-06T10:44:51.757596+0000","last_undegraded":"2023-10-06T10:44:51.757596+0000","last_fullsized":"2023-10-06T10:44:51.757596+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:05:36.289745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757281+0000","last_change":"2023-10-06T10:44:51.752557+0000","last_active":"2023-10-06T10:44:51.757281+0000","last_peered":"2023-10-06T10:44:51.757281+0000","last_clean":"2023-10-06T10:44:51.757281+0000","last_became_active":"2023-10-06T10:44:51.752087+0000","last_became_peered":"2023-10-06T10:44:51.752087+0000","last_unstale":"2023-10-06T10:44:51.757281+0000","last_undegraded":"2023-10-06T10:44:51.757281+0000","last_fullsized":"2023-10-06T10:44:51.757281+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:30.552215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.751722+0000","last_change":"2023-10-06T10:44:51.749221+0000","last_active":"2023-10-06T10:44:51.751722+0000","last_peered":"2023-10-06T10:44:51.751722+0000","last_clean":"2023-10-06T10:44:51.751722+0000","last_became_active":"2023-10-06T10:44:51.749025+0000","last_became_peered":"2023-10-06T10:44:51.749025+0000","last_unstale":"2023-10-06T10:44:51.751722+0000","last_undegraded":"2023-10-06T10:44:51.751722+0000","last_fullsized":"2023-10-06T10:44:51.751722+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:28.610646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745548+0000","last_change":"2023-10-06T10:44:51.739336+0000","last_active":"2023-10-06T10:44:51.745548+0000","last_peered":"2023-10-06T10:44:51.745548+0000","last_clean":"2023-10-06T10:44:51.745548+0000","last_became_active":"2023-10-06T10:44:51.739055+0000","last_became_peered":"2023-10-06T10:44:51.739055+0000","last_unstale":"2023-10-06T10:44:51.745548+0000","last_undegraded":"2023-10-06T10:44:51.745548+0000","last_fullsized":"2023-10-06T10:44:51.745548+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:30.342288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.747260+0000","last_change":"2023-10-06T10:44:51.743966+0000","last_active":"2023-10-06T10:44:51.747260+0000","last_peered":"2023-10-06T10:44:51.747260+0000","last_clean":"2023-10-06T10:44:51.747260+0000","last_became_active":"2023-10-06T10:44:51.743686+0000","last_became_peered":"2023-10-06T10:44:51.743686+0000","last_unstale":"2023-10-06T10:44:51.747260+0000","last_undegraded":"2023-10-06T10:44:51.747260+0000","last_fullsized":"2023-10-06T10:44:51.747260+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:12:10.059460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.758682+0000","last_change":"2023-10-06T10:44:51.752074+0000","last_active":"2023-10-06T10:44:51.758682+0000","last_peered":"2023-10-06T10:44:51.758682+0000","last_clean":"2023-10-06T10:44:51.758682+0000","last_became_active":"2023-10-06T10:44:51.751397+0000","last_became_peered":"2023-10-06T10:44:51.751397+0000","last_unstale":"2023-10-06T10:44:51.758682+0000","last_undegraded":"2023-10-06T10:44:51.758682+0000","last_fullsized":"2023-10-06T10:44:51.758682+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:09.234346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.758578+0000","last_change":"2023-10-06T10:44:51.754015+0000","last_active":"2023-10-06T10:44:51.758578+0000","last_peered":"2023-10-06T10:44:51.758578+0000","last_clean":"2023-10-06T10:44:51.758578+0000","last_became_active":"2023-10-06T10:44:51.753770+0000","last_became_peered":"2023-10-06T10:44:51.753770+0000","last_unstale":"2023-10-06T10:44:51.758578+0000","last_undegraded":"2023-10-06T10:44:51.758578+0000","last_fullsized":"2023-10-06T10:44:51.758578+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:33:24.214039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745867+0000","last_change":"2023-10-06T10:44:51.741537+0000","last_active":"2023-10-06T10:44:51.745867+0000","last_peered":"2023-10-06T10:44:51.745867+0000","last_clean":"2023-10-06T10:44:51.745867+0000","last_became_active":"2023-10-06T10:44:51.740094+0000","last_became_peered":"2023-10-06T10:44:51.740094+0000","last_unstale":"2023-10-06T10:44:51.745867+0000","last_undegraded":"2023-10-06T10:44:51.745867+0000","last_fullsized":"2023-10-06T10:44:51.745867+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:00.855299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.760219+0000","last_change":"2023-10-06T10:44:51.756108+0000","last_active":"2023-10-06T10:44:51.760219+0000","last_peered":"2023-10-06T10:44:51.760219+0000","last_clean":"2023-10-06T10:44:51.760219+0000","last_became_active":"2023-10-06T10:44:51.755657+0000","last_became_peered":"2023-10-06T10:44:51.755657+0000","last_unstale":"2023-10-06T10:44:51.760219+0000","last_undegraded":"2023-10-06T10:44:51.760219+0000","last_fullsized":"2023-10-06T10:44:51.760219+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:45:43.330258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.754161+0000","last_change":"2023-10-06T10:44:51.746358+0000","last_active":"2023-10-06T10:44:51.754161+0000","last_peered":"2023-10-06T10:44:51.754161+0000","last_clean":"2023-10-06T10:44:51.754161+0000","last_became_active":"2023-10-06T10:44:51.746074+0000","last_became_peered":"2023-10-06T10:44:51.746074+0000","last_unstale":"2023-10-06T10:44:51.754161+0000","last_undegraded":"2023-10-06T10:44:51.754161+0000","last_fullsized":"2023-10-06T10:44:51.754161+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:44:07.998505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.744777+0000","last_change":"2023-10-06T10:44:51.740471+0000","last_active":"2023-10-06T10:44:51.744777+0000","last_peered":"2023-10-06T10:44:51.744777+0000","last_clean":"2023-10-06T10:44:51.744777+0000","last_became_active":"2023-10-06T10:44:51.740180+0000","last_became_peered":"2023-10-06T10:44:51.740180+0000","last_unstale":"2023-10-06T10:44:51.744777+0000","last_undegraded":"2023-10-06T10:44:51.744777+0000","last_fullsized":"2023-10-06T10:44:51.744777+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:45.409013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.748530+0000","last_change":"2023-10-06T10:44:51.745196+0000","last_active":"2023-10-06T10:44:51.748530+0000","last_peered":"2023-10-06T10:44:51.748530+0000","last_clean":"2023-10-06T10:44:51.748530+0000","last_became_active":"2023-10-06T10:44:51.744948+0000","last_became_peered":"2023-10-06T10:44:51.744948+0000","last_unstale":"2023-10-06T10:44:51.748530+0000","last_undegraded":"2023-10-06T10:44:51.748530+0000","last_fullsized":"2023-10-06T10:44:51.748530+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:11.554526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757351+0000","last_change":"2023-10-06T10:44:51.748954+0000","last_active":"2023-10-06T10:44:51.757351+0000","last_peered":"2023-10-06T10:44:51.757351+0000","last_clean":"2023-10-06T10:44:51.757351+0000","last_became_active":"2023-10-06T10:44:51.743005+0000","last_became_peered":"2023-10-06T10:44:51.743005+0000","last_unstale":"2023-10-06T10:44:51.757351+0000","last_undegraded":"2023-10-06T10:44:51.757351+0000","last_fullsized":"2023-10-06T10:44:51.757351+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:07.472732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745914+0000","last_change":"2023-10-06T10:44:51.740909+0000","last_active":"2023-10-06T10:44:51.745914+0000","last_peered":"2023-10-06T10:44:51.745914+0000","last_clean":"2023-10-06T10:44:51.745914+0000","last_became_active":"2023-10-06T10:44:51.740469+0000","last_became_peered":"2023-10-06T10:44:51.740469+0000","last_unstale":"2023-10-06T10:44:51.745914+0000","last_undegraded":"2023-10-06T10:44:51.745914+0000","last_fullsized":"2023-10-06T10:44:51.745914+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:29:51.113126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.744542+0000","last_change":"2023-10-06T10:44:51.739439+0000","last_active":"2023-10-06T10:44:51.744542+0000","last_peered":"2023-10-06T10:44:51.744542+0000","last_clean":"2023-10-06T10:44:51.744542+0000","last_became_active":"2023-10-06T10:44:51.739175+0000","last_became_peered":"2023-10-06T10:44:51.739175+0000","last_unstale":"2023-10-06T10:44:51.744542+0000","last_undegraded":"2023-10-06T10:44:51.744542+0000","last_fullsized":"2023-10-06T10:44:51.744542+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:11.372620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.754171+0000","last_change":"2023-10-06T10:44:51.746412+0000","last_active":"2023-10-06T10:44:51.754171+0000","last_peered":"2023-10-06T10:44:51.754171+0000","last_clean":"2023-10-06T10:44:51.754171+0000","last_became_active":"2023-10-06T10:44:51.746109+0000","last_became_peered":"2023-10-06T10:44:51.746109+0000","last_unstale":"2023-10-06T10:44:51.754171+0000","last_undegraded":"2023-10-06T10:44:51.754171+0000","last_fullsized":"2023-10-06T10:44:51.754171+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:15.138118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.746258+0000","last_change":"2023-10-06T10:44:51.741367+0000","last_active":"2023-10-06T10:44:51.746258+0000","last_peered":"2023-10-06T10:44:51.746258+0000","last_clean":"2023-10-06T10:44:51.746258+0000","last_became_active":"2023-10-06T10:44:51.741058+0000","last_became_peered":"2023-10-06T10:44:51.741058+0000","last_unstale":"2023-10-06T10:44:51.746258+0000","last_undegraded":"2023-10-06T10:44:51.746258+0000","last_fullsized":"2023-10-06T10:44:51.746258+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:21.512208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.753315+0000","last_change":"2023-10-06T10:44:51.747900+0000","last_active":"2023-10-06T10:44:51.753315+0000","last_peered":"2023-10-06T10:44:51.753315+0000","last_clean":"2023-10-06T10:44:51.753315+0000","last_became_active":"2023-10-06T10:44:51.747625+0000","last_became_peered":"2023-10-06T10:44:51.747625+0000","last_unstale":"2023-10-06T10:44:51.753315+0000","last_undegraded":"2023-10-06T10:44:51.753315+0000","last_fullsized":"2023-10-06T10:44:51.753315+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:56:26.690740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.748662+0000","last_change":"2023-10-06T10:44:51.743748+0000","last_active":"2023-10-06T10:44:51.748662+0000","last_peered":"2023-10-06T10:44:51.748662+0000","last_clean":"2023-10-06T10:44:51.748662+0000","last_became_active":"2023-10-06T10:44:51.743511+0000","last_became_peered":"2023-10-06T10:44:51.743511+0000","last_unstale":"2023-10-06T10:44:51.748662+0000","last_undegraded":"2023-10-06T10:44:51.748662+0000","last_fullsized":"2023-10-06T10:44:51.748662+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:58:58.968944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.746207+0000","last_change":"2023-10-06T10:44:51.743049+0000","last_active":"2023-10-06T10:44:51.746207+0000","last_peered":"2023-10-06T10:44:51.746207+0000","last_clean":"2023-10-06T10:44:51.746207+0000","last_became_active":"2023-10-06T10:44:51.740779+0000","last_became_peered":"2023-10-06T10:44:51.740779+0000","last_unstale":"2023-10-06T10:44:51.746207+0000","last_undegraded":"2023-10-06T10:44:51.746207+0000","last_fullsized":"2023-10-06T10:44:51.746207+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:27:22.155683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745690+0000","last_change":"2023-10-06T10:44:51.730584+0000","last_active":"2023-10-06T10:44:51.745690+0000","last_peered":"2023-10-06T10:44:51.745690+0000","last_clean":"2023-10-06T10:44:51.745690+0000","last_became_active":"2023-10-06T10:44:51.729878+0000","last_became_peered":"2023-10-06T10:44:51.729878+0000","last_unstale":"2023-10-06T10:44:51.745690+0000","last_undegraded":"2023-10-06T10:44:51.745690+0000","last_fullsized":"2023-10-06T10:44:51.745690+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:59.356737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745892+0000","last_change":"2023-10-06T10:44:51.740729+0000","last_active":"2023-10-06T10:44:51.745892+0000","last_peered":"2023-10-06T10:44:51.745892+0000","last_clean":"2023-10-06T10:44:51.745892+0000","last_became_active":"2023-10-06T10:44:51.740139+0000","last_became_peered":"2023-10-06T10:44:51.740139+0000","last_unstale":"2023-10-06T10:44:51.745892+0000","last_undegraded":"2023-10-06T10:44:51.745892+0000","last_fullsized":"2023-10-06T10:44:51.745892+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:07:26.376540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757745+0000","last_change":"2023-10-06T10:44:51.751633+0000","last_active":"2023-10-06T10:44:51.757745+0000","last_peered":"2023-10-06T10:44:51.757745+0000","last_clean":"2023-10-06T10:44:51.757745+0000","last_became_active":"2023-10-06T10:44:51.751325+0000","last_became_peered":"2023-10-06T10:44:51.751325+0000","last_unstale":"2023-10-06T10:44:51.757745+0000","last_undegraded":"2023-10-06T10:44:51.757745+0000","last_fullsized":"2023-10-06T10:44:51.757745+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:34.107458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757927+0000","last_change":"2023-10-06T10:44:51.754195+0000","last_active":"2023-10-06T10:44:51.757927+0000","last_peered":"2023-10-06T10:44:51.757927+0000","last_clean":"2023-10-06T10:44:51.757927+0000","last_became_active":"2023-10-06T10:44:51.753939+0000","last_became_peered":"2023-10-06T10:44:51.753939+0000","last_unstale":"2023-10-06T10:44:51.757927+0000","last_undegraded":"2023-10-06T10:44:51.757927+0000","last_fullsized":"2023-10-06T10:44:51.757927+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:14:19.607356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.755601+0000","last_change":"2023-10-06T10:44:51.753173+0000","last_active":"2023-10-06T10:44:51.755601+0000","last_peered":"2023-10-06T10:44:51.755601+0000","last_clean":"2023-10-06T10:44:51.755601+0000","last_became_active":"2023-10-06T10:44:51.752806+0000","last_became_peered":"2023-10-06T10:44:51.752806+0000","last_unstale":"2023-10-06T10:44:51.755601+0000","last_undegraded":"2023-10-06T10:44:51.755601+0000","last_fullsized":"2023-10-06T10:44:51.755601+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:27:22.155683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.746044+0000","last_change":"2023-10-06T10:44:51.741443+0000","last_active":"2023-10-06T10:44:51.746044+0000","last_peered":"2023-10-06T10:44:51.746044+0000","last_clean":"2023-10-06T10:44:51.746044+0000","last_became_active":"2023-10-06T10:44:51.740884+0000","last_became_peered":"2023-10-06T10:44:51.740884+0000","last_unstale":"2023-10-06T10:44:51.746044+0000","last_undegraded":"2023-10-06T10:44:51.746044+0000","last_fullsized":"2023-10-06T10:44:51.746044+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:15.056633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.739324+0000","last_change":"2023-10-06T10:44:51.734925+0000","last_active":"2023-10-06T10:44:51.739324+0000","last_peered":"2023-10-06T10:44:51.739324+0000","last_clean":"2023-10-06T10:44:51.739324+0000","last_became_active":"2023-10-06T10:44:51.734516+0000","last_became_peered":"2023-10-06T10:44:51.734516+0000","last_unstale":"2023-10-06T10:44:51.739324+0000","last_undegraded":"2023-10-06T10:44:51.739324+0000","last_fullsized":"2023-10-06T10:44:51.739324+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:11:42.494648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.755509+0000","last_change":"2023-10-06T10:44:51.743028+0000","last_active":"2023-10-06T10:44:51.755509+0000","last_peered":"2023-10-06T10:44:51.755509+0000","last_clean":"2023-10-06T10:44:51.755509+0000","last_became_active":"2023-10-06T10:44:51.742742+0000","last_became_peered":"2023-10-06T10:44:51.742742+0000","last_unstale":"2023-10-06T10:44:51.755509+0000","last_undegraded":"2023-10-06T10:44:51.755509+0000","last_fullsized":"2023-10-06T10:44:51.755509+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:31:28.825035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.750999+0000","last_change":"2023-10-06T10:44:51.747804+0000","last_active":"2023-10-06T10:44:51.750999+0000","last_peered":"2023-10-06T10:44:51.750999+0000","last_clean":"2023-10-06T10:44:51.750999+0000","last_became_active":"2023-10-06T10:44:51.747553+0000","last_became_peered":"2023-10-06T10:44:51.747553+0000","last_unstale":"2023-10-06T10:44:51.750999+0000","last_undegraded":"2023-10-06T10:44:51.750999+0000","last_fullsized":"2023-10-06T10:44:51.750999+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:00:47.142581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.749647+0000","last_change":"2023-10-06T10:44:51.746945+0000","last_active":"2023-10-06T10:44:51.749647+0000","last_peered":"2023-10-06T10:44:51.749647+0000","last_clean":"2023-10-06T10:44:51.749647+0000","last_became_active":"2023-10-06T10:44:51.746807+0000","last_became_peered":"2023-10-06T10:44:51.746807+0000","last_unstale":"2023-10-06T10:44:51.749647+0000","last_undegraded":"2023-10-06T10:44:51.749647+0000","last_fullsized":"2023-10-06T10:44:51.749647+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:58.636572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"101'4","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.747192+0000","last_change":"2023-10-06T10:44:50.738760+0000","last_active":"2023-10-06T10:44:51.747192+0000","last_peered":"2023-10-06T10:44:51.747192+0000","last_clean":"2023-10-06T10:44:51.747192+0000","last_became_active":"2023-10-06T10:44:50.738480+0000","last_became_peered":"2023-10-06T10:44:50.738480+0000","last_unstale":"2023-10-06T10:44:51.747192+0000","last_undegraded":"2023-10-06T10:44:51.747192+0000","last_fullsized":"2023-10-06T10:44:51.747192+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:29.546521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.744979+0000","last_change":"2023-10-06T10:44:51.741107+0000","last_active":"2023-10-06T10:44:51.744979+0000","last_peered":"2023-10-06T10:44:51.744979+0000","last_clean":"2023-10-06T10:44:51.744979+0000","last_became_active":"2023-10-06T10:44:51.740656+0000","last_became_peered":"2023-10-06T10:44:51.740656+0000","last_unstale":"2023-10-06T10:44:51.744979+0000","last_undegraded":"2023-10-06T10:44:51.744979+0000","last_fullsized":"2023-10-06T10:44:51.744979+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:53.858578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"101'1","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.742828+0000","last_change":"2023-10-06T10:44:50.739987+0000","last_active":"2023-10-06T10:44:51.742828+0000","last_peered":"2023-10-06T10:44:51.742828+0000","last_clean":"2023-10-06T10:44:51.742828+0000","last_became_active":"2023-10-06T10:44:50.739669+0000","last_became_peered":"2023-10-06T10:44:50.739669+0000","last_unstale":"2023-10-06T10:44:51.742828+0000","last_undegraded":"2023-10-06T10:44:51.742828+0000","last_fullsized":"2023-10-06T10:44:51.742828+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:38:47.009118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.749627+0000","last_change":"2023-10-06T10:44:51.746313+0000","last_active":"2023-10-06T10:44:51.749627+0000","last_peered":"2023-10-06T10:44:51.749627+0000","last_clean":"2023-10-06T10:44:51.749627+0000","last_became_active":"2023-10-06T10:44:51.746064+0000","last_became_peered":"2023-10-06T10:44:51.746064+0000","last_unstale":"2023-10-06T10:44:51.749627+0000","last_undegraded":"2023-10-06T10:44:51.749627+0000","last_fullsized":"2023-10-06T10:44:51.749627+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:44:59.326168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"101'2","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.752912+0000","last_change":"2023-10-06T10:44:50.737016+0000","last_active":"2023-10-06T10:44:51.752912+0000","last_peered":"2023-10-06T10:44:51.752912+0000","last_clean":"2023-10-06T10:44:51.752912+0000","last_became_active":"2023-10-06T10:44:50.736567+0000","last_became_peered":"2023-10-06T10:44:50.736567+0000","last_unstale":"2023-10-06T10:44:51.752912+0000","last_undegraded":"2023-10-06T10:44:51.752912+0000","last_fullsized":"2023-10-06T10:44:51.752912+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:08.868364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.754205+0000","last_change":"2023-10-06T10:44:51.746821+0000","last_active":"2023-10-06T10:44:51.754205+0000","last_peered":"2023-10-06T10:44:51.754205+0000","last_clean":"2023-10-06T10:44:51.754205+0000","last_became_active":"2023-10-06T10:44:51.746576+0000","last_became_peered":"2023-10-06T10:44:51.746576+0000","last_unstale":"2023-10-06T10:44:51.754205+0000","last_undegraded":"2023-10-06T10:44:51.754205+0000","last_fullsized":"2023-10-06T10:44:51.754205+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:06:43.616700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"101'2","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.756745+0000","last_change":"2023-10-06T10:44:50.738193+0000","last_active":"2023-10-06T10:44:51.756745+0000","last_peered":"2023-10-06T10:44:51.756745+0000","last_clean":"2023-10-06T10:44:51.756745+0000","last_became_active":"2023-10-06T10:44:50.737952+0000","last_became_peered":"2023-10-06T10:44:50.737952+0000","last_unstale":"2023-10-06T10:44:51.756745+0000","last_undegraded":"2023-10-06T10:44:51.756745+0000","last_fullsized":"2023-10-06T10:44:51.756745+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:31.698962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757718+0000","last_change":"2023-10-06T10:44:51.751538+0000","last_active":"2023-10-06T10:44:51.757718+0000","last_peered":"2023-10-06T10:44:51.757718+0000","last_clean":"2023-10-06T10:44:51.757718+0000","last_became_active":"2023-10-06T10:44:51.751239+0000","last_became_peered":"2023-10-06T10:44:51.751239+0000","last_unstale":"2023-10-06T10:44:51.757718+0000","last_undegraded":"2023-10-06T10:44:51.757718+0000","last_fullsized":"2023-10-06T10:44:51.757718+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:46.027434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"101'1","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.743770+0000","last_change":"2023-10-06T10:44:50.740417+0000","last_active":"2023-10-06T10:44:51.743770+0000","last_peered":"2023-10-06T10:44:51.743770+0000","last_clean":"2023-10-06T10:44:51.743770+0000","last_became_active":"2023-10-06T10:44:50.740148+0000","last_became_peered":"2023-10-06T10:44:50.740148+0000","last_unstale":"2023-10-06T10:44:51.743770+0000","last_undegraded":"2023-10-06T10:44:51.743770+0000","last_fullsized":"2023-10-06T10:44:51.743770+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:02.006450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.750448+0000","last_change":"2023-10-06T10:44:51.744759+0000","last_active":"2023-10-06T10:44:51.750448+0000","last_peered":"2023-10-06T10:44:51.750448+0000","last_clean":"2023-10-06T10:44:51.750448+0000","last_became_active":"2023-10-06T10:44:51.744499+0000","last_became_peered":"2023-10-06T10:44:51.744499+0000","last_unstale":"2023-10-06T10:44:51.750448+0000","last_undegraded":"2023-10-06T10:44:51.750448+0000","last_fullsized":"2023-10-06T10:44:51.750448+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:58:58.968944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"101'1","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.750469+0000","last_change":"2023-10-06T10:44:50.736564+0000","last_active":"2023-10-06T10:44:51.750469+0000","last_peered":"2023-10-06T10:44:51.750469+0000","last_clean":"2023-10-06T10:44:51.750469+0000","last_became_active":"2023-10-06T10:44:50.736297+0000","last_became_peered":"2023-10-06T10:44:50.736297+0000","last_unstale":"2023-10-06T10:44:51.750469+0000","last_undegraded":"2023-10-06T10:44:51.750469+0000","last_fullsized":"2023-10-06T10:44:51.750469+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:04:17.777010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757369+0000","last_change":"2023-10-06T10:44:51.749198+0000","last_active":"2023-10-06T10:44:51.757369+0000","last_peered":"2023-10-06T10:44:51.757369+0000","last_clean":"2023-10-06T10:44:51.757369+0000","last_became_active":"2023-10-06T10:44:51.748962+0000","last_became_peered":"2023-10-06T10:44:51.748962+0000","last_unstale":"2023-10-06T10:44:51.757369+0000","last_undegraded":"2023-10-06T10:44:51.757369+0000","last_fullsized":"2023-10-06T10:44:51.757369+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:39.790377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.732785+0000","last_change":"2023-10-06T10:44:50.732777+0000","last_active":"2023-10-06T10:44:51.732785+0000","last_peered":"2023-10-06T10:44:51.732785+0000","last_clean":"2023-10-06T10:44:51.732785+0000","last_became_active":"2023-10-06T10:44:50.732494+0000","last_became_peered":"2023-10-06T10:44:50.732494+0000","last_unstale":"2023-10-06T10:44:51.732785+0000","last_undegraded":"2023-10-06T10:44:51.732785+0000","last_fullsized":"2023-10-06T10:44:51.732785+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:26:41.299786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745570+0000","last_change":"2023-10-06T10:44:51.739689+0000","last_active":"2023-10-06T10:44:51.745570+0000","last_peered":"2023-10-06T10:44:51.745570+0000","last_clean":"2023-10-06T10:44:51.745570+0000","last_became_active":"2023-10-06T10:44:51.739426+0000","last_became_peered":"2023-10-06T10:44:51.739426+0000","last_unstale":"2023-10-06T10:44:51.745570+0000","last_undegraded":"2023-10-06T10:44:51.745570+0000","last_fullsized":"2023-10-06T10:44:51.745570+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:38:48.014813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.735795+0000","last_change":"2023-10-06T10:44:50.746777+0000","last_active":"2023-10-06T10:44:51.735795+0000","last_peered":"2023-10-06T10:44:51.735795+0000","last_clean":"2023-10-06T10:44:51.735795+0000","last_became_active":"2023-10-06T10:44:50.746550+0000","last_became_peered":"2023-10-06T10:44:50.746550+0000","last_unstale":"2023-10-06T10:44:51.735795+0000","last_undegraded":"2023-10-06T10:44:51.735795+0000","last_fullsized":"2023-10-06T10:44:51.735795+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:41.068289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.758917+0000","last_change":"2023-10-06T10:44:51.756739+0000","last_active":"2023-10-06T10:44:51.758917+0000","last_peered":"2023-10-06T10:44:51.758917+0000","last_clean":"2023-10-06T10:44:51.758917+0000","last_became_active":"2023-10-06T10:44:51.753839+0000","last_became_peered":"2023-10-06T10:44:51.753839+0000","last_unstale":"2023-10-06T10:44:51.758917+0000","last_undegraded":"2023-10-06T10:44:51.758917+0000","last_fullsized":"2023-10-06T10:44:51.758917+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:14:19.607356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"101'1","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.751747+0000","last_change":"2023-10-06T10:44:50.743094+0000","last_active":"2023-10-06T10:44:51.751747+0000","last_peered":"2023-10-06T10:44:51.751747+0000","last_clean":"2023-10-06T10:44:51.751747+0000","last_became_active":"2023-10-06T10:44:50.742703+0000","last_became_peered":"2023-10-06T10:44:50.742703+0000","last_unstale":"2023-10-06T10:44:51.751747+0000","last_undegraded":"2023-10-06T10:44:51.751747+0000","last_fullsized":"2023-10-06T10:44:51.751747+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:25.575241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.748437+0000","last_change":"2023-10-06T10:44:51.744523+0000","last_active":"2023-10-06T10:44:51.748437+0000","last_peered":"2023-10-06T10:44:51.748437+0000","last_clean":"2023-10-06T10:44:51.748437+0000","last_became_active":"2023-10-06T10:44:51.744272+0000","last_became_peered":"2023-10-06T10:44:51.744272+0000","last_unstale":"2023-10-06T10:44:51.748437+0000","last_undegraded":"2023-10-06T10:44:51.748437+0000","last_fullsized":"2023-10-06T10:44:51.748437+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:03.012145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"101'2","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.753032+0000","last_change":"2023-10-06T10:44:50.735822+0000","last_active":"2023-10-06T10:44:51.753032+0000","last_peered":"2023-10-06T10:44:51.753032+0000","last_clean":"2023-10-06T10:44:51.753032+0000","last_became_active":"2023-10-06T10:44:50.735568+0000","last_became_peered":"2023-10-06T10:44:50.735568+0000","last_unstale":"2023-10-06T10:44:51.753032+0000","last_undegraded":"2023-10-06T10:44:51.753032+0000","last_fullsized":"2023-10-06T10:44:51.753032+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:38:19.424287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.756698+0000","last_change":"2023-10-06T10:44:51.754072+0000","last_active":"2023-10-06T10:44:51.756698+0000","last_peered":"2023-10-06T10:44:51.756698+0000","last_clean":"2023-10-06T10:44:51.756698+0000","last_became_active":"2023-10-06T10:44:51.753802+0000","last_became_peered":"2023-10-06T10:44:51.753802+0000","last_unstale":"2023-10-06T10:44:51.756698+0000","last_undegraded":"2023-10-06T10:44:51.756698+0000","last_fullsized":"2023-10-06T10:44:51.756698+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:48:22.355874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"101'1","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.750611+0000","last_change":"2023-10-06T10:44:50.740185+0000","last_active":"2023-10-06T10:44:51.750611+0000","last_peered":"2023-10-06T10:44:51.750611+0000","last_clean":"2023-10-06T10:44:51.750611+0000","last_became_active":"2023-10-06T10:44:50.739930+0000","last_became_peered":"2023-10-06T10:44:50.739930+0000","last_unstale":"2023-10-06T10:44:51.750611+0000","last_undegraded":"2023-10-06T10:44:51.750611+0000","last_fullsized":"2023-10-06T10:44:51.750611+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:47:30.907849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.739759+0000","last_change":"2023-10-06T10:44:51.729671+0000","last_active":"2023-10-06T10:44:51.739759+0000","last_peered":"2023-10-06T10:44:51.739759+0000","last_clean":"2023-10-06T10:44:51.739759+0000","last_became_active":"2023-10-06T10:44:51.729451+0000","last_became_peered":"2023-10-06T10:44:51.729451+0000","last_unstale":"2023-10-06T10:44:51.739759+0000","last_undegraded":"2023-10-06T10:44:51.739759+0000","last_fullsized":"2023-10-06T10:44:51.739759+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:33:51.029950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.725200+0000","last_change":"2023-10-06T10:44:50.743084+0000","last_active":"2023-10-06T10:44:51.725200+0000","last_peered":"2023-10-06T10:44:51.725200+0000","last_clean":"2023-10-06T10:44:51.725200+0000","last_became_active":"2023-10-06T10:44:50.742740+0000","last_became_peered":"2023-10-06T10:44:50.742740+0000","last_unstale":"2023-10-06T10:44:51.725200+0000","last_undegraded":"2023-10-06T10:44:51.725200+0000","last_fullsized":"2023-10-06T10:44:51.725200+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:23.746429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.739337+0000","last_change":"2023-10-06T10:44:51.735032+0000","last_active":"2023-10-06T10:44:51.739337+0000","last_peered":"2023-10-06T10:44:51.739337+0000","last_clean":"2023-10-06T10:44:51.739337+0000","last_became_active":"2023-10-06T10:44:51.734765+0000","last_became_peered":"2023-10-06T10:44:51.734765+0000","last_unstale":"2023-10-06T10:44:51.739337+0000","last_undegraded":"2023-10-06T10:44:51.739337+0000","last_fullsized":"2023-10-06T10:44:51.739337+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:53.858578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"101'2","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757070+0000","last_change":"2023-10-06T10:44:50.735989+0000","last_active":"2023-10-06T10:44:51.757070+0000","last_peered":"2023-10-06T10:44:51.757070+0000","last_clean":"2023-10-06T10:44:51.757070+0000","last_became_active":"2023-10-06T10:44:50.735704+0000","last_became_peered":"2023-10-06T10:44:50.735704+0000","last_unstale":"2023-10-06T10:44:51.757070+0000","last_undegraded":"2023-10-06T10:44:51.757070+0000","last_fullsized":"2023-10-06T10:44:51.757070+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:21.691363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.755630+0000","last_change":"2023-10-06T10:44:51.746428+0000","last_active":"2023-10-06T10:44:51.755630+0000","last_peered":"2023-10-06T10:44:51.755630+0000","last_clean":"2023-10-06T10:44:51.755630+0000","last_became_active":"2023-10-06T10:44:51.746168+0000","last_became_peered":"2023-10-06T10:44:51.746168+0000","last_unstale":"2023-10-06T10:44:51.755630+0000","last_undegraded":"2023-10-06T10:44:51.755630+0000","last_fullsized":"2023-10-06T10:44:51.755630+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:17:28.335605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.733095+0000","last_change":"2023-10-06T10:44:50.737860+0000","last_active":"2023-10-06T10:44:51.733095+0000","last_peered":"2023-10-06T10:44:51.733095+0000","last_clean":"2023-10-06T10:44:51.733095+0000","last_became_active":"2023-10-06T10:44:50.737174+0000","last_became_peered":"2023-10-06T10:44:50.737174+0000","last_unstale":"2023-10-06T10:44:51.733095+0000","last_undegraded":"2023-10-06T10:44:51.733095+0000","last_fullsized":"2023-10-06T10:44:51.733095+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:13.818223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.754190+0000","last_change":"2023-10-06T10:44:51.746749+0000","last_active":"2023-10-06T10:44:51.754190+0000","last_peered":"2023-10-06T10:44:51.754190+0000","last_clean":"2023-10-06T10:44:51.754190+0000","last_became_active":"2023-10-06T10:44:51.746525+0000","last_became_peered":"2023-10-06T10:44:51.746525+0000","last_unstale":"2023-10-06T10:44:51.754190+0000","last_undegraded":"2023-10-06T10:44:51.754190+0000","last_fullsized":"2023-10-06T10:44:51.754190+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:13.902450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.729551+0000","last_change":"2023-10-06T10:44:50.740163+0000","last_active":"2023-10-06T10:44:51.729551+0000","last_peered":"2023-10-06T10:44:51.729551+0000","last_clean":"2023-10-06T10:44:51.729551+0000","last_became_active":"2023-10-06T10:44:50.739892+0000","last_became_peered":"2023-10-06T10:44:50.739892+0000","last_unstale":"2023-10-06T10:44:51.729551+0000","last_undegraded":"2023-10-06T10:44:51.729551+0000","last_fullsized":"2023-10-06T10:44:51.729551+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:31.698962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.747134+0000","last_change":"2023-10-06T10:44:51.743095+0000","last_active":"2023-10-06T10:44:51.747134+0000","last_peered":"2023-10-06T10:44:51.747134+0000","last_clean":"2023-10-06T10:44:51.747134+0000","last_became_active":"2023-10-06T10:44:51.742863+0000","last_became_peered":"2023-10-06T10:44:51.742863+0000","last_unstale":"2023-10-06T10:44:51.747134+0000","last_undegraded":"2023-10-06T10:44:51.747134+0000","last_fullsized":"2023-10-06T10:44:51.747134+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:56:26.690740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.735885+0000","last_change":"2023-10-06T10:44:50.742684+0000","last_active":"2023-10-06T10:44:51.735885+0000","last_peered":"2023-10-06T10:44:51.735885+0000","last_clean":"2023-10-06T10:44:51.735885+0000","last_became_active":"2023-10-06T10:44:50.742434+0000","last_became_peered":"2023-10-06T10:44:50.742434+0000","last_unstale":"2023-10-06T10:44:51.735885+0000","last_undegraded":"2023-10-06T10:44:51.735885+0000","last_fullsized":"2023-10-06T10:44:51.735885+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:57.037993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.753591+0000","last_change":"2023-10-06T10:44:51.751722+0000","last_active":"2023-10-06T10:44:51.753591+0000","last_peered":"2023-10-06T10:44:51.753591+0000","last_clean":"2023-10-06T10:44:51.753591+0000","last_became_active":"2023-10-06T10:44:51.751490+0000","last_became_peered":"2023-10-06T10:44:51.751490+0000","last_unstale":"2023-10-06T10:44:51.753591+0000","last_undegraded":"2023-10-06T10:44:51.753591+0000","last_fullsized":"2023-10-06T10:44:51.753591+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:45.409013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"101'2","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.747535+0000","last_change":"2023-10-06T10:44:50.745009+0000","last_active":"2023-10-06T10:44:51.747535+0000","last_peered":"2023-10-06T10:44:51.747535+0000","last_clean":"2023-10-06T10:44:51.747535+0000","last_became_active":"2023-10-06T10:44:50.744698+0000","last_became_peered":"2023-10-06T10:44:50.744698+0000","last_unstale":"2023-10-06T10:44:51.747535+0000","last_undegraded":"2023-10-06T10:44:51.747535+0000","last_fullsized":"2023-10-06T10:44:51.747535+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:26:41.299786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.758669+0000","last_change":"2023-10-06T10:44:51.751880+0000","last_active":"2023-10-06T10:44:51.758669+0000","last_peered":"2023-10-06T10:44:51.758669+0000","last_clean":"2023-10-06T10:44:51.758669+0000","last_became_active":"2023-10-06T10:44:51.751258+0000","last_became_peered":"2023-10-06T10:44:51.751258+0000","last_unstale":"2023-10-06T10:44:51.758669+0000","last_undegraded":"2023-10-06T10:44:51.758669+0000","last_fullsized":"2023-10-06T10:44:51.758669+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:07.487093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.732945+0000","last_change":"2023-10-06T10:44:50.735743+0000","last_active":"2023-10-06T10:44:51.732945+0000","last_peered":"2023-10-06T10:44:51.732945+0000","last_clean":"2023-10-06T10:44:51.732945+0000","last_became_active":"2023-10-06T10:44:50.735498+0000","last_became_peered":"2023-10-06T10:44:50.735498+0000","last_unstale":"2023-10-06T10:44:51.732945+0000","last_undegraded":"2023-10-06T10:44:51.732945+0000","last_fullsized":"2023-10-06T10:44:51.732945+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:19.608630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757616+0000","last_change":"2023-10-06T10:44:51.749866+0000","last_active":"2023-10-06T10:44:51.757616+0000","last_peered":"2023-10-06T10:44:51.757616+0000","last_clean":"2023-10-06T10:44:51.757616+0000","last_became_active":"2023-10-06T10:44:51.749578+0000","last_became_peered":"2023-10-06T10:44:51.749578+0000","last_unstale":"2023-10-06T10:44:51.757616+0000","last_undegraded":"2023-10-06T10:44:51.757616+0000","last_fullsized":"2023-10-06T10:44:51.757616+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:52.361795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.734752+0000","last_change":"2023-10-06T10:44:50.740894+0000","last_active":"2023-10-06T10:44:51.734752+0000","last_peered":"2023-10-06T10:44:51.734752+0000","last_clean":"2023-10-06T10:44:51.734752+0000","last_became_active":"2023-10-06T10:44:50.740653+0000","last_became_peered":"2023-10-06T10:44:50.740653+0000","last_unstale":"2023-10-06T10:44:51.734752+0000","last_undegraded":"2023-10-06T10:44:51.734752+0000","last_fullsized":"2023-10-06T10:44:51.734752+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:11.169696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.754014+0000","last_change":"2023-10-06T10:44:51.751873+0000","last_active":"2023-10-06T10:44:51.754014+0000","last_peered":"2023-10-06T10:44:51.754014+0000","last_clean":"2023-10-06T10:44:51.754014+0000","last_became_active":"2023-10-06T10:44:51.751589+0000","last_became_peered":"2023-10-06T10:44:51.751589+0000","last_unstale":"2023-10-06T10:44:51.754014+0000","last_undegraded":"2023-10-06T10:44:51.754014+0000","last_fullsized":"2023-10-06T10:44:51.754014+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:12:10.059460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.726919+0000","last_change":"2023-10-06T10:44:50.739163+0000","last_active":"2023-10-06T10:44:51.726919+0000","last_peered":"2023-10-06T10:44:51.726919+0000","last_clean":"2023-10-06T10:44:51.726919+0000","last_became_active":"2023-10-06T10:44:50.738915+0000","last_became_peered":"2023-10-06T10:44:50.738915+0000","last_unstale":"2023-10-06T10:44:51.726919+0000","last_undegraded":"2023-10-06T10:44:51.726919+0000","last_fullsized":"2023-10-06T10:44:51.726919+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:32.689348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.737452+0000","last_change":"2023-10-06T10:44:51.736064+0000","last_active":"2023-10-06T10:44:51.737452+0000","last_peered":"2023-10-06T10:44:51.737452+0000","last_clean":"2023-10-06T10:44:51.737452+0000","last_became_active":"2023-10-06T10:44:51.735730+0000","last_became_peered":"2023-10-06T10:44:51.735730+0000","last_unstale":"2023-10-06T10:44:51.737452+0000","last_undegraded":"2023-10-06T10:44:51.737452+0000","last_fullsized":"2023-10-06T10:44:51.737452+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:42.073984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.725056+0000","last_change":"2023-10-06T10:44:50.738749+0000","last_active":"2023-10-06T10:44:51.725056+0000","last_peered":"2023-10-06T10:44:51.725056+0000","last_clean":"2023-10-06T10:44:51.725056+0000","last_became_active":"2023-10-06T10:44:50.738444+0000","last_became_peered":"2023-10-06T10:44:50.738444+0000","last_unstale":"2023-10-06T10:44:51.725056+0000","last_undegraded":"2023-10-06T10:44:51.725056+0000","last_fullsized":"2023-10-06T10:44:51.725056+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:50:16.152816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.756270+0000","last_change":"2023-10-06T10:44:51.753208+0000","last_active":"2023-10-06T10:44:51.756270+0000","last_peered":"2023-10-06T10:44:51.756270+0000","last_clean":"2023-10-06T10:44:51.756270+0000","last_became_active":"2023-10-06T10:44:51.752963+0000","last_became_peered":"2023-10-06T10:44:51.752963+0000","last_unstale":"2023-10-06T10:44:51.756270+0000","last_undegraded":"2023-10-06T10:44:51.756270+0000","last_fullsized":"2023-10-06T10:44:51.756270+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:58.043687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.732962+0000","last_change":"2023-10-06T10:44:50.740319+0000","last_active":"2023-10-06T10:44:51.732962+0000","last_peered":"2023-10-06T10:44:51.732962+0000","last_clean":"2023-10-06T10:44:51.732962+0000","last_became_active":"2023-10-06T10:44:50.740054+0000","last_became_peered":"2023-10-06T10:44:50.740054+0000","last_unstale":"2023-10-06T10:44:51.732962+0000","last_undegraded":"2023-10-06T10:44:51.732962+0000","last_fullsized":"2023-10-06T10:44:51.732962+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:05:34.133718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.744832+0000","last_change":"2023-10-06T10:44:51.740932+0000","last_active":"2023-10-06T10:44:51.744832+0000","last_peered":"2023-10-06T10:44:51.744832+0000","last_clean":"2023-10-06T10:44:51.744832+0000","last_became_active":"2023-10-06T10:44:51.740255+0000","last_became_peered":"2023-10-06T10:44:51.740255+0000","last_unstale":"2023-10-06T10:44:51.744832+0000","last_undegraded":"2023-10-06T10:44:51.744832+0000","last_fullsized":"2023-10-06T10:44:51.744832+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:33.695043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.737226+0000","last_change":"2023-10-06T10:44:50.737048+0000","last_active":"2023-10-06T10:44:51.737226+0000","last_peered":"2023-10-06T10:44:51.737226+0000","last_clean":"2023-10-06T10:44:51.737226+0000","last_became_active":"2023-10-06T10:44:50.736752+0000","last_became_peered":"2023-10-06T10:44:50.736752+0000","last_unstale":"2023-10-06T10:44:51.737226+0000","last_undegraded":"2023-10-06T10:44:51.737226+0000","last_fullsized":"2023-10-06T10:44:51.737226+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:21.691363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757668+0000","last_change":"2023-10-06T10:44:51.750254+0000","last_active":"2023-10-06T10:44:51.757668+0000","last_peered":"2023-10-06T10:44:51.757668+0000","last_clean":"2023-10-06T10:44:51.757668+0000","last_became_active":"2023-10-06T10:44:51.750002+0000","last_became_peered":"2023-10-06T10:44:51.750002+0000","last_unstale":"2023-10-06T10:44:51.757668+0000","last_undegraded":"2023-10-06T10:44:51.757668+0000","last_fullsized":"2023-10-06T10:44:51.757668+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:31:12.288467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.735995+0000","last_change":"2023-10-06T10:44:50.740969+0000","last_active":"2023-10-06T10:44:51.735995+0000","last_peered":"2023-10-06T10:44:51.735995+0000","last_clean":"2023-10-06T10:44:51.735995+0000","last_became_active":"2023-10-06T10:44:50.740714+0000","last_became_peered":"2023-10-06T10:44:50.740714+0000","last_unstale":"2023-10-06T10:44:51.735995+0000","last_undegraded":"2023-10-06T10:44:51.735995+0000","last_fullsized":"2023-10-06T10:44:51.735995+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:56:16.765456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.733952+0000","last_change":"2023-10-06T10:44:51.730894+0000","last_active":"2023-10-06T10:44:51.733952+0000","last_peered":"2023-10-06T10:44:51.733952+0000","last_clean":"2023-10-06T10:44:51.733952+0000","last_became_active":"2023-10-06T10:44:51.730349+0000","last_became_peered":"2023-10-06T10:44:51.730349+0000","last_unstale":"2023-10-06T10:44:51.733952+0000","last_undegraded":"2023-10-06T10:44:51.733952+0000","last_fullsized":"2023-10-06T10:44:51.733952+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:36:50.152016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"101'6","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.762876+0000","last_change":"2023-10-06T10:44:50.738740+0000","last_active":"2023-10-06T10:44:51.762876+0000","last_peered":"2023-10-06T10:44:51.762876+0000","last_clean":"2023-10-06T10:44:51.762876+0000","last_became_active":"2023-10-06T10:44:50.738498+0000","last_became_peered":"2023-10-06T10:44:50.738498+0000","last_unstale":"2023-10-06T10:44:51.762876+0000","last_undegraded":"2023-10-06T10:44:51.762876+0000","last_fullsized":"2023-10-06T10:44:51.762876+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:42:51.471629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":90,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.732796+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:44:51.732796+0000","last_peered":"2023-10-06T10:44:51.732796+0000","last_clean":"2023-10-06T10:44:51.732796+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:44:51.732796+0000","last_undegraded":"2023-10-06T10:44:51.732796+0000","last_fullsized":"2023-10-06T10:44:51.732796+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.748949+0000","last_change":"2023-10-06T10:44:51.744508+0000","last_active":"2023-10-06T10:44:51.748949+0000","last_peered":"2023-10-06T10:44:51.748949+0000","last_clean":"2023-10-06T10:44:51.748949+0000","last_became_active":"2023-10-06T10:44:51.744214+0000","last_became_peered":"2023-10-06T10:44:51.744214+0000","last_unstale":"2023-10-06T10:44:51.748949+0000","last_undegraded":"2023-10-06T10:44:51.748949+0000","last_fullsized":"2023-10-06T10:44:51.748949+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:58.043687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"101'3","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.752327+0000","last_change":"2023-10-06T10:44:50.740082+0000","last_active":"2023-10-06T10:44:51.752327+0000","last_peered":"2023-10-06T10:44:51.752327+0000","last_clean":"2023-10-06T10:44:51.752327+0000","last_became_active":"2023-10-06T10:44:50.739924+0000","last_became_peered":"2023-10-06T10:44:50.739924+0000","last_unstale":"2023-10-06T10:44:51.752327+0000","last_undegraded":"2023-10-06T10:44:51.752327+0000","last_fullsized":"2023-10-06T10:44:51.752327+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:27:09.636709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.727020+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:44:51.727020+0000","last_peered":"2023-10-06T10:44:51.727020+0000","last_clean":"2023-10-06T10:44:51.727020+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:44:51.727020+0000","last_undegraded":"2023-10-06T10:44:51.727020+0000","last_fullsized":"2023-10-06T10:44:51.727020+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745719+0000","last_change":"2023-10-06T10:44:51.732830+0000","last_active":"2023-10-06T10:44:51.745719+0000","last_peered":"2023-10-06T10:44:51.745719+0000","last_clean":"2023-10-06T10:44:51.745719+0000","last_became_active":"2023-10-06T10:44:51.732580+0000","last_became_peered":"2023-10-06T10:44:51.732580+0000","last_unstale":"2023-10-06T10:44:51.745719+0000","last_undegraded":"2023-10-06T10:44:51.745719+0000","last_fullsized":"2023-10-06T10:44:51.745719+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:12.175391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.731609+0000","last_change":"2023-10-06T10:44:50.741287+0000","last_active":"2023-10-06T10:44:51.731609+0000","last_peered":"2023-10-06T10:44:51.731609+0000","last_clean":"2023-10-06T10:44:51.731609+0000","last_became_active":"2023-10-06T10:44:50.740987+0000","last_became_peered":"2023-10-06T10:44:50.740987+0000","last_unstale":"2023-10-06T10:44:51.731609+0000","last_undegraded":"2023-10-06T10:44:51.731609+0000","last_fullsized":"2023-10-06T10:44:51.731609+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:27:18.042909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":84,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.729061+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:44:51.729061+0000","last_peered":"2023-10-06T10:44:51.729061+0000","last_clean":"2023-10-06T10:44:51.729061+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:44:51.729061+0000","last_undegraded":"2023-10-06T10:44:51.729061+0000","last_fullsized":"2023-10-06T10:44:51.729061+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":226,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.724876+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:44:51.724876+0000","last_peered":"2023-10-06T10:44:51.724876+0000","last_clean":"2023-10-06T10:44:51.724876+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:44:51.724876+0000","last_undegraded":"2023-10-06T10:44:51.724876+0000","last_fullsized":"2023-10-06T10:44:51.724876+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.746427+0000","last_change":"2023-10-06T10:44:51.734173+0000","last_active":"2023-10-06T10:44:51.746427+0000","last_peered":"2023-10-06T10:44:51.746427+0000","last_clean":"2023-10-06T10:44:51.746427+0000","last_became_active":"2023-10-06T10:44:51.733932+0000","last_became_peered":"2023-10-06T10:44:51.733932+0000","last_unstale":"2023-10-06T10:44:51.746427+0000","last_undegraded":"2023-10-06T10:44:51.746427+0000","last_fullsized":"2023-10-06T10:44:51.746427+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:42.073984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"101'2","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.743590+0000","last_change":"2023-10-06T10:44:50.737521+0000","last_active":"2023-10-06T10:44:51.743590+0000","last_peered":"2023-10-06T10:44:51.743590+0000","last_clean":"2023-10-06T10:44:51.743590+0000","last_became_active":"2023-10-06T10:44:50.737276+0000","last_became_peered":"2023-10-06T10:44:50.737276+0000","last_unstale":"2023-10-06T10:44:51.743590+0000","last_undegraded":"2023-10-06T10:44:51.743590+0000","last_fullsized":"2023-10-06T10:44:51.743590+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:14:37.094197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":76,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.726680+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:44:51.726680+0000","last_peered":"2023-10-06T10:44:51.726680+0000","last_clean":"2023-10-06T10:44:51.726680+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:44:51.726680+0000","last_undegraded":"2023-10-06T10:44:51.726680+0000","last_fullsized":"2023-10-06T10:44:51.726680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745715+0000","last_change":"2023-10-06T10:44:51.730648+0000","last_active":"2023-10-06T10:44:51.745715+0000","last_peered":"2023-10-06T10:44:51.745715+0000","last_clean":"2023-10-06T10:44:51.745715+0000","last_became_active":"2023-10-06T10:44:51.729905+0000","last_became_peered":"2023-10-06T10:44:51.729905+0000","last_unstale":"2023-10-06T10:44:51.745715+0000","last_undegraded":"2023-10-06T10:44:51.745715+0000","last_fullsized":"2023-10-06T10:44:51.745715+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:07:26.376540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.725894+0000","last_change":"2023-10-06T10:44:50.742945+0000","last_active":"2023-10-06T10:44:51.725894+0000","last_peered":"2023-10-06T10:44:51.725894+0000","last_clean":"2023-10-06T10:44:51.725894+0000","last_became_active":"2023-10-06T10:44:50.742669+0000","last_became_peered":"2023-10-06T10:44:50.742669+0000","last_unstale":"2023-10-06T10:44:51.725894+0000","last_undegraded":"2023-10-06T10:44:51.725894+0000","last_fullsized":"2023-10-06T10:44:51.725894+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:28.430497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.724961+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:44:51.724961+0000","last_peered":"2023-10-06T10:44:51.724961+0000","last_clean":"2023-10-06T10:44:51.724961+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:44:51.724961+0000","last_undegraded":"2023-10-06T10:44:51.724961+0000","last_fullsized":"2023-10-06T10:44:51.724961+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.746547+0000","last_change":"2023-10-06T10:44:51.741319+0000","last_active":"2023-10-06T10:44:51.746547+0000","last_peered":"2023-10-06T10:44:51.746547+0000","last_clean":"2023-10-06T10:44:51.746547+0000","last_became_active":"2023-10-06T10:44:51.740994+0000","last_became_peered":"2023-10-06T10:44:51.740994+0000","last_unstale":"2023-10-06T10:44:51.746547+0000","last_undegraded":"2023-10-06T10:44:51.746547+0000","last_fullsized":"2023-10-06T10:44:51.746547+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:56:17.771151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"101'8","reported_seq":18,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.760275+0000","last_change":"2023-10-06T10:44:50.738355+0000","last_active":"2023-10-06T10:44:51.760275+0000","last_peered":"2023-10-06T10:44:51.760275+0000","last_clean":"2023-10-06T10:44:51.760275+0000","last_became_active":"2023-10-06T10:44:50.738093+0000","last_became_peered":"2023-10-06T10:44:50.738093+0000","last_unstale":"2023-10-06T10:44:51.760275+0000","last_undegraded":"2023-10-06T10:44:51.760275+0000","last_fullsized":"2023-10-06T10:44:51.760275+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:46.580158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.725988+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:44:51.725988+0000","last_peered":"2023-10-06T10:44:51.725988+0000","last_clean":"2023-10-06T10:44:51.725988+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:44:51.725988+0000","last_undegraded":"2023-10-06T10:44:51.725988+0000","last_fullsized":"2023-10-06T10:44:51.725988+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.753934+0000","last_change":"2023-10-06T10:44:51.752435+0000","last_active":"2023-10-06T10:44:51.753934+0000","last_peered":"2023-10-06T10:44:51.753934+0000","last_clean":"2023-10-06T10:44:51.753934+0000","last_became_active":"2023-10-06T10:44:51.752213+0000","last_became_peered":"2023-10-06T10:44:51.752213+0000","last_unstale":"2023-10-06T10:44:51.753934+0000","last_undegraded":"2023-10-06T10:44:51.753934+0000","last_fullsized":"2023-10-06T10:44:51.753934+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:58.636572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.733702+0000","last_change":"2023-10-06T10:44:50.736915+0000","last_active":"2023-10-06T10:44:51.733702+0000","last_peered":"2023-10-06T10:44:51.733702+0000","last_clean":"2023-10-06T10:44:51.733702+0000","last_became_active":"2023-10-06T10:44:50.736660+0000","last_became_peered":"2023-10-06T10:44:50.736660+0000","last_unstale":"2023-10-06T10:44:51.733702+0000","last_undegraded":"2023-10-06T10:44:51.733702+0000","last_fullsized":"2023-10-06T10:44:51.733702+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:50:16.152816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.733032+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:44:51.733032+0000","last_peered":"2023-10-06T10:44:51.733032+0000","last_clean":"2023-10-06T10:44:51.733032+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:44:51.733032+0000","last_undegraded":"2023-10-06T10:44:51.733032+0000","last_fullsized":"2023-10-06T10:44:51.733032+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.753284+0000","last_change":"2023-10-06T10:44:51.746238+0000","last_active":"2023-10-06T10:44:51.753284+0000","last_peered":"2023-10-06T10:44:51.753284+0000","last_clean":"2023-10-06T10:44:51.753284+0000","last_became_active":"2023-10-06T10:44:51.745970+0000","last_became_peered":"2023-10-06T10:44:51.745970+0000","last_unstale":"2023-10-06T10:44:51.753284+0000","last_undegraded":"2023-10-06T10:44:51.753284+0000","last_fullsized":"2023-10-06T10:44:51.753284+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:48:22.355874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.726787+0000","last_change":"2023-10-06T10:44:50.737172+0000","last_active":"2023-10-06T10:44:51.726787+0000","last_peered":"2023-10-06T10:44:51.726787+0000","last_clean":"2023-10-06T10:44:51.726787+0000","last_became_active":"2023-10-06T10:44:50.736902+0000","last_became_peered":"2023-10-06T10:44:50.736902+0000","last_unstale":"2023-10-06T10:44:51.726787+0000","last_undegraded":"2023-10-06T10:44:51.726787+0000","last_fullsized":"2023-10-06T10:44:51.726787+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:45:13.294923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.733036+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:44:51.733036+0000","last_peered":"2023-10-06T10:44:51.733036+0000","last_clean":"2023-10-06T10:44:51.733036+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:44:51.733036+0000","last_undegraded":"2023-10-06T10:44:51.733036+0000","last_fullsized":"2023-10-06T10:44:51.733036+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299112,"kb_used_data":1824,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072728,"statfs":{"total":96636764160,"available":96330473472,"internally_reserved":0,"allocated":1867776,"data_stored":1203651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.51000000000000001,"15min":0.48799999999999999},"min":{"1min":0.32800000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63900000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.53800000000000003,"15min":0.52100000000000002},"min":{"1min":0.29399999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.9700000000000002,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":0.66100000000000003}]},{"osd":1,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.496,"15min":0.46700000000000003},"min":{"1min":0.32000000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.64800000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.54500000000000004,"15min":0.51300000000000001},"min":{"1min":0.38400000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":3.0059999999999998,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56699999999999995,"15min":0.52200000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.628},{"interface":"front","average":{"1min":0.497,"5min":0.51900000000000002,"15min":0.499},"min":{"1min":0.371,"5min":0.314,"15min":0.314},"max":{"1min":0.66800000000000004,"5min":0.747,"15min":0.79800000000000004},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55200000000000005,"15min":0.51600000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73099999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.40600000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.71899999999999997,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.67600000000000005}]},{"osd":4,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42199999999999999,"15min":0.46200000000000002},"min":{"1min":0.314,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61899999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47299999999999998,"15min":0.499},"min":{"1min":0.30199999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64900000000000002,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.71999999999999997}]},{"osd":5,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.48199999999999998},"min":{"1min":0.33000000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.81100000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.51800000000000002,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85399999999999998,"5min":1.577,"15min":1.577},"last":0.69899999999999995}]},{"osd":6,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.56499999999999995},"min":{"1min":0.39700000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.67200000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.745},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.56200000000000006,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69599999999999995,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.75700000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298512,"kb_used_data":1224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073328,"statfs":{"total":96636764160,"available":96331087872,"internally_reserved":0,"allocated":1253376,"data_stored":604451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.495},"min":{"1min":0.34599999999999997,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":0.70399999999999996,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52100000000000002,"15min":0.45500000000000002},"min":{"1min":0.39700000000000002,"5min":0.32500000000000001,"15min":0.32000000000000001},"max":{"1min":0.67400000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.59899999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52000000000000002,"15min":0.46000000000000002},"min":{"1min":0.39300000000000002,"5min":0.32200000000000001,"15min":0.28000000000000003},"max":{"1min":0.70799999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54500000000000004,"15min":0.48899999999999999},"min":{"1min":0.42399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73499999999999999,"5min":0.751,"15min":0.751},"last":0.621}]},{"osd":2,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52700000000000002,"15min":0.48399999999999999},"min":{"1min":0.39600000000000002,"5min":0.32500000000000001,"15min":0.309},"max":{"1min":0.65400000000000003,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54500000000000004,"15min":0.48699999999999999},"min":{"1min":0.443,"5min":0.34999999999999998,"15min":0.34300000000000003},"max":{"1min":0.66100000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.69699999999999995}]},{"osd":3,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55200000000000005,"15min":0.50600000000000001},"min":{"1min":0.46500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75600000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.55500000000000005,"15min":0.51900000000000002},"min":{"1min":0.40699999999999997,"5min":0.36399999999999999,"15min":0.34499999999999997},"max":{"1min":0.74199999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.79200000000000004}]},{"osd":4,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.48699999999999999,"15min":0.48099999999999998},"min":{"1min":0.34999999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.78300000000000003,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.498,"15min":0.48899999999999999},"min":{"1min":0.377,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79400000000000004,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.71499999999999997}]},{"osd":5,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.51000000000000001,"15min":0.49199999999999999},"min":{"1min":0.34200000000000003,"5min":0.28899999999999998,"15min":0.28299999999999997},"max":{"1min":0.83599999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.38700000000000001,"5min":0.33900000000000002,"15min":0.308},"max":{"1min":0.77800000000000002,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.93100000000000005}]},{"osd":7,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53200000000000003,"15min":0.52000000000000002},"min":{"1min":0.36599999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71899999999999997,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.755},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57399999999999995,"15min":0.54800000000000004},"min":{"1min":0.40000000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.871,"5min":0.879,"15min":0.879},"last":0.76800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303208,"kb_used_data":1824,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068632,"statfs":{"total":96636764160,"available":96326279168,"internally_reserved":0,"allocated":1867776,"data_stored":1203187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.20399999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.35599999999999998,"15min":0.34799999999999998},"min":{"1min":0.23400000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.57099999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.27800000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.33000000000000002,"15min":0.32000000000000001},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.46999999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.373},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38100000000000001,"15min":0.36699999999999999},"min":{"1min":0.27700000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.41299999999999998}]},{"osd":3,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39500000000000002,"15min":0.38},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.60999999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.38700000000000001,"15min":0.377},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.32600000000000001}]},{"osd":4,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.40200000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73099999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.48299999999999998}]},{"osd":5,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86599999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.503},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.55000000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.35699999999999998},"max":{"1min":0.80900000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.49299999999999999}]},{"osd":6,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.59899999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.36699999999999999},"max":{"1min":0.92200000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.435,"5min":0.40000000000000002,"15min":0.39800000000000002},"max":{"1min":0.77000000000000002,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.51900000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.39100000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.84299999999999997,"5min":0.996,"15min":0.996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.42099999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.56999999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298544,"kb_used_data":1256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073296,"statfs":{"total":96636764160,"available":96331055104,"internally_reserved":0,"allocated":1286144,"data_stored":614343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.33000000000000002,"15min":0.32000000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.433,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.376,"15min":0.34999999999999998},"min":{"1min":0.28399999999999997,"5min":0.25,"15min":0.25},"max":{"1min":0.504,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.32100000000000001,"15min":0.309},"min":{"1min":0.246,"5min":0.191,"15min":0.191},"max":{"1min":0.41799999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.248},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35599999999999998,"15min":0.33200000000000002},"min":{"1min":0.26900000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.46000000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.40500000000000003}]},{"osd":3,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35399999999999998,"15min":0.34399999999999997},"min":{"1min":0.25600000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.49299999999999999,"5min":0.622,"15min":0.622},"last":0.31},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36599999999999999,"15min":0.35299999999999998},"min":{"1min":0.28999999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.47899999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.32100000000000001}]},{"osd":4,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.41799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78900000000000003,"5min":0.81200000000000006,"15min":0.83299999999999996},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56599999999999995,"15min":0.54100000000000004},"min":{"1min":0.38600000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.753,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.60399999999999998}]},{"osd":5,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.36899999999999999},"max":{"1min":0.80300000000000005,"5min":0.91900000000000004,"15min":1.0069999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.41299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76700000000000002,"5min":0.80900000000000005,"15min":0.94499999999999995},"last":0.57699999999999996}]},{"osd":6,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56999999999999995,"15min":0.58199999999999996},"min":{"1min":0.40300000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.747,"5min":1.046,"15min":1.046},"last":0.629},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.42999999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76000000000000001,"5min":0.86399999999999999,"15min":1.075},"last":0.53400000000000003}]},{"osd":7,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80700000000000005,"5min":0.83499999999999996,"15min":0.96099999999999997},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.44400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76900000000000002,"5min":0.88800000000000001,"15min":0.90100000000000002},"last":0.54600000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574932,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298512,"kb_used_data":1224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073328,"statfs":{"total":96636764160,"available":96331087872,"internally_reserved":0,"allocated":1253376,"data_stored":605734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.317,"15min":0.312},"min":{"1min":0.22500000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.45300000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.22600000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.36199999999999999,"15min":0.32900000000000001},"min":{"1min":0.27600000000000002,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.502,"5min":0.622,"15min":0.622},"last":0.32600000000000001}]},{"osd":1,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34499999999999997,"15min":0.32900000000000001},"min":{"1min":0.23499999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.46600000000000003,"5min":0.56599999999999995,"15min":0.60499999999999998},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.378,"15min":0.36799999999999999},"min":{"1min":0.26700000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.55600000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.37}]},{"osd":3,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.38700000000000001,"15min":0.35699999999999998},"min":{"1min":0.28599999999999998,"5min":0.252,"15min":0.20499999999999999},"max":{"1min":0.55000000000000004,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38,"15min":0.36199999999999999},"min":{"1min":0.28499999999999998,"5min":0.25,"15min":0.22800000000000001},"max":{"1min":0.58099999999999996,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.35599999999999998}]},{"osd":4,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.35299999999999998,"5min":0.33400000000000002,"15min":0.315},"max":{"1min":0.78000000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.441,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81100000000000005,"5min":0.82699999999999996,"15min":0.84899999999999998},"last":0.57799999999999996}]},{"osd":5,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73899999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.42799999999999999,"5min":0.36599999999999999,"15min":0.33800000000000002},"max":{"1min":0.80100000000000005,"5min":0.81799999999999995,"15min":0.83399999999999996},"last":0.50600000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.61699999999999999},"min":{"1min":0.42199999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.871,"5min":0.89000000000000001,"15min":0.89100000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.621,"15min":0.622},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79300000000000004,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.61599999999999999}]},{"osd":7,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58999999999999997,"15min":0.60799999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.36799999999999999},"max":{"1min":0.84199999999999997,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60999999999999999,"15min":0.622},"min":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.749,"5min":0.874,"15min":0.874},"last":0.58699999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574932,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298540,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073300,"statfs":{"total":96636764160,"available":96331059200,"internally_reserved":0,"allocated":1282048,"data_stored":614301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.314,"15min":0.41499999999999998},"min":{"1min":0.20599999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.433,"5min":0.52700000000000002,"15min":3.3210000000000002},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34399999999999997,"15min":0.35599999999999998},"min":{"1min":0.21099999999999999,"5min":0.192,"15min":0.192},"max":{"1min":0.436,"5min":0.66800000000000004,"15min":1.218},"last":0.35099999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.33700000000000002,"15min":0.33900000000000002},"min":{"1min":0.23699999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.40600000000000003,"5min":0.63600000000000001,"15min":0.86299999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.36499999999999999,"15min":0.45600000000000002},"min":{"1min":0.28399999999999997,"5min":0.218,"15min":0.218},"max":{"1min":0.46000000000000002,"5min":0.66000000000000003,"15min":3.359},"last":0.36499999999999999}]},{"osd":2,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.36399999999999999,"15min":0.377},"min":{"1min":0.27800000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.55500000000000005,"5min":0.71099999999999997,"15min":1.1100000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.38300000000000001,"15min":0.58899999999999997},"min":{"1min":0.28399999999999997,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46100000000000002,"5min":0.59899999999999998,"15min":7.069},"last":0.30599999999999999}]},{"osd":4,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55900000000000005,"15min":0.57999999999999996},"min":{"1min":0.41599999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.66300000000000003,"5min":0.89400000000000002,"15min":1.165},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.58599999999999997,"15min":0.59299999999999997},"min":{"1min":0.379,"5min":0.375,"15min":0.375},"max":{"1min":0.753,"5min":1.02,"15min":1.02},"last":0.53200000000000003}]},{"osd":5,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.58199999999999996,"15min":0.67500000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71499999999999997,"5min":0.92500000000000004,"15min":3.4140000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57699999999999996,"15min":0.60099999999999998},"min":{"1min":0.39200000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.96899999999999997,"15min":1.1930000000000001},"last":0.50800000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.60299999999999998,"15min":0.61299999999999999},"min":{"1min":0.42599999999999999,"5min":0.38,"15min":0.378},"max":{"1min":0.83699999999999997,"5min":1.054,"15min":1.054},"last":0.495},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61899999999999999,"15min":0.70599999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.41999999999999998},"max":{"1min":0.80800000000000005,"5min":0.876,"15min":3.3879999999999999},"last":0.54600000000000004}]},{"osd":7,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85599999999999998,"5min":0.99099999999999999,"15min":1.137},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.436,"5min":0.378,"15min":0.378},"max":{"1min":0.67600000000000005,"5min":0.97899999999999998,"15min":1.014},"last":0.42999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298524,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073316,"statfs":{"total":96636764160,"available":96331075584,"internally_reserved":0,"allocated":1265664,"data_stored":612695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.502,"15min":0.48799999999999999},"min":{"1min":0.316,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73799999999999999,"5min":0.74299999999999999,"15min":0.76800000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52100000000000002,"15min":0.504},"min":{"1min":0.36099999999999999,"5min":0.33100000000000002,"15min":0.28299999999999997},"max":{"1min":0.71399999999999997,"5min":0.73099999999999998,"15min":0.77400000000000002},"last":0.51600000000000001}]},{"osd":1,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71199999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51500000000000001,"15min":0.53300000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70499999999999996,"5min":0.71799999999999997,"15min":0.82899999999999996},"last":0.52900000000000003}]},{"osd":2,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.32800000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.81000000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.372,"5min":0.36099999999999999,"15min":0.315},"max":{"1min":0.69399999999999995,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.501}]},{"osd":3,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55200000000000005,"15min":0.52400000000000002},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.35499999999999998},"max":{"1min":0.73899999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.79500000000000004},"last":0.58999999999999997}]},{"osd":5,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.436,"15min":0.48799999999999999},"min":{"1min":0.30399999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.622,"5min":0.77800000000000002,"15min":0.86899999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46800000000000003,"15min":0.49299999999999999},"min":{"1min":0.29499999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.65200000000000002,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.61799999999999999}]},{"osd":6,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.496,"15min":0.53300000000000003},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53500000000000003,"15min":0.56799999999999995},"min":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69999999999999996,"5min":0.88,"15min":0.91200000000000003},"last":0.60599999999999998}]},{"osd":7,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51700000000000002,"15min":0.56299999999999994},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.67900000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53400000000000003,"15min":0.58299999999999996},"min":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72899999999999998,"5min":0.81299999999999994,"15min":0.85699999999999998},"last":0.63400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298532,"kb_used_data":1244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073308,"statfs":{"total":96636764160,"available":96331067392,"internally_reserved":0,"allocated":1273856,"data_stored":613271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50900000000000001,"15min":0.47499999999999998},"min":{"1min":0.28599999999999998,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.66400000000000003,"5min":0.88,"15min":0.88},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.50800000000000001,"15min":0.52400000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.64700000000000002,"5min":0.86499999999999999,"15min":1.2549999999999999},"last":0.441}]},{"osd":1,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.314,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.59999999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.32300000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64300000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.47599999999999998}]},{"osd":2,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53300000000000003,"15min":0.50700000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.30399999999999999},"max":{"1min":0.61899999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.54300000000000004,"15min":0.55900000000000005},"min":{"1min":0.35399999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67700000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.52800000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.54000000000000004,"15min":0.56599999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57899999999999996,"5min":0.79900000000000004,"15min":0.95099999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.51800000000000002,"15min":0.54800000000000004},"min":{"1min":0.33200000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70499999999999996,"5min":0.79100000000000004,"15min":1.3999999999999999},"last":0.504}]},{"osd":4,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47399999999999998,"15min":0.48499999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.75700000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.499,"15min":0.50600000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.68999999999999995,"5min":0.70399999999999996,"15min":0.80700000000000005},"last":0.60599999999999998}]},{"osd":6,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.495,"15min":0.51500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.67800000000000005,"5min":0.77200000000000002,"15min":0.85799999999999998},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78100000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.55600000000000005}]},{"osd":7,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54700000000000004,"15min":0.57399999999999995},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.83299999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71399999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.58899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:44:56.539 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:44:56.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.685+0000 7f53f9200700 1 Processor -- start 2023-10-06T10:44:56.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.685+0000 7f53f9200700 1 -- start start 2023-10-06T10:44:56.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.685+0000 7f53f9200700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f53f4158d10 0x7f53f406a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:56.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.685+0000 7f53f9200700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53f40cad50 0x7f53f40685c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:56.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.685+0000 7f53f9200700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f53f40cb1c0 0x7f53f4068b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:56.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.685+0000 7f53f9200700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53f40c6db0 con 0x7f53f40cb1c0 2023-10-06T10:44:56.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.685+0000 7f53f9200700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53f40c6fc0 con 0x7f53f4158d10 2023-10-06T10:44:56.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.685+0000 7f53f9200700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f40c72a0 con 0x7f53f40cad50 2023-10-06T10:44:56.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.686+0000 7f53f359e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f53f40cb1c0 0x7f53f4068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:56.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.686+0000 7f53f359e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f53f40cb1c0 0x7f53f4068b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33828/0 (socket says 172.21.15.92:33828) 2023-10-06T10:44:56.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.686+0000 7f53f359e700 1 -- 172.21.15.92:0/3681477920 learned_addr learned my addr 172.21.15.92:0/3681477920 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:56.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.686+0000 7f53f2d9d700 1 --2- 172.21.15.92:0/3681477920 >> v2:172.21.15.124:3300/0 conn(0x7f53f4158d10 0x7f53f406a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:56.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.686+0000 7f53ea59c700 1 --2- 172.21.15.92:0/3681477920 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53f40cad50 0x7f53f40685c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:56.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.686+0000 7f53f359e700 1 -- 172.21.15.92:0/3681477920 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53f40cad50 msgr2=0x7f53f40685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:56.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.686+0000 7f53f359e700 1 --2- 172.21.15.92:0/3681477920 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53f40cad50 0x7f53f40685c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.686+0000 7f53f359e700 1 -- 172.21.15.92:0/3681477920 >> v2:172.21.15.124:3300/0 conn(0x7f53f4158d10 msgr2=0x7f53f406a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:56.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.686+0000 7f53f359e700 1 --2- 172.21.15.92:0/3681477920 >> v2:172.21.15.124:3300/0 conn(0x7f53f4158d10 0x7f53f406a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.686+0000 7f53f359e700 1 -- 172.21.15.92:0/3681477920 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f40c7580 con 0x7f53f40cb1c0 2023-10-06T10:44:56.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.686+0000 7f53f359e700 1 --2- 172.21.15.92:0/3681477920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53f40cb1c0 0x7f53f4068b00 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f53f4065070 tx=0x7f53ec00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8774d3b9f25c5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:56.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.686+0000 7f53f259c700 1 -- 172.21.15.92:0/3681477920 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec010070 con 0x7f53f40cb1c0 2023-10-06T10:44:56.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.686+0000 7f53f259c700 1 -- 172.21.15.92:0/3681477920 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ec00db20 con 0x7f53f40cb1c0 2023-10-06T10:44:56.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.686+0000 7f53f259c700 1 -- 172.21.15.92:0/3681477920 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec016400 con 0x7f53f40cb1c0 2023-10-06T10:44:56.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.687+0000 7f53f9200700 1 -- 172.21.15.92:0/3681477920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53f40cb1c0 msgr2=0x7f53f4068b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:56.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.687+0000 7f53f9200700 1 --2- 172.21.15.92:0/3681477920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53f40cb1c0 0x7f53f4068b00 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f53f4065070 tx=0x7f53ec00ba50 comp rx=0 tx=0).stop 2023-10-06T10:44:56.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.687+0000 7f53f9200700 1 -- 172.21.15.92:0/3681477920 shutdown_connections 2023-10-06T10:44:56.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.687+0000 7f53f9200700 1 --2- 172.21.15.92:0/3681477920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53f40cb1c0 0x7f53f4068b00 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.687+0000 7f53f9200700 1 --2- 172.21.15.92:0/3681477920 >> v2:172.21.15.124:3300/0 conn(0x7f53f4158d10 0x7f53f406a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.687+0000 7f53f9200700 1 --2- 172.21.15.92:0/3681477920 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53f40cad50 0x7f53f40685c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.687+0000 7f53f9200700 1 -- 172.21.15.92:0/3681477920 >> 172.21.15.92:0/3681477920 conn(0x7f53f40c9c80 msgr2=0x7f53f406daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:56.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.687+0000 7f53f9200700 1 -- 172.21.15.92:0/3681477920 shutdown_connections 2023-10-06T10:44:56.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.687+0000 7f53f9200700 1 -- 172.21.15.92:0/3681477920 wait complete. 2023-10-06T10:44:56.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.688+0000 7f53f9200700 1 Processor -- start 2023-10-06T10:44:56.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.688+0000 7f53f9200700 1 -- start start 2023-10-06T10:44:56.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.689+0000 7f53f9200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53f40cad50 0x7f53f4146de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:56.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.689+0000 7f53f9200700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53f40cb1c0 0x7f53f40c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:56.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.689+0000 7f53f9200700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53f4158d10 0x7f53f414f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:56.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.689+0000 7f53f9200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f4166a20 con 0x7f53f40cad50 2023-10-06T10:44:56.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.689+0000 7f53f9200700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f4165e10 con 0x7f53f40cb1c0 2023-10-06T10:44:56.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.689+0000 7f53f9200700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f41660f0 con 0x7f53f4158d10 2023-10-06T10:44:56.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.689+0000 7f53f2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53f40cad50 0x7f53f4146de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:56.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.689+0000 7f53f2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53f40cad50 0x7f53f4146de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33836/0 (socket says 172.21.15.92:33836) 2023-10-06T10:44:56.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.689+0000 7f53f2d9d700 1 -- 172.21.15.92:0/1301940972 learned_addr learned my addr 172.21.15.92:0/1301940972 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:56.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.689+0000 7f53ea59c700 1 --2- 172.21.15.92:0/1301940972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53f40cb1c0 0x7f53f40c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:56.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.689+0000 7f53f359e700 1 --2- 172.21.15.92:0/1301940972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53f4158d10 0x7f53f414f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:56.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.689+0000 7f53f2d9d700 1 -- 172.21.15.92:0/1301940972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53f4158d10 msgr2=0x7f53f414f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:56.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.689+0000 7f53f2d9d700 1 --2- 172.21.15.92:0/1301940972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53f4158d10 0x7f53f414f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.689+0000 7f53f2d9d700 1 -- 172.21.15.92:0/1301940972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53f40cb1c0 msgr2=0x7f53f40c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:56.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.689+0000 7f53f2d9d700 1 --2- 172.21.15.92:0/1301940972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53f40cb1c0 0x7f53f40c48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.689+0000 7f53f2d9d700 1 -- 172.21.15.92:0/1301940972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec00bcf0 con 0x7f53f40cad50 2023-10-06T10:44:56.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.689+0000 7f53f2d9d700 1 --2- 172.21.15.92:0/1301940972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53f40cad50 0x7f53f4146de0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f53e400e930 tx=0x7f53e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:56.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.689+0000 7f53eb7fe700 1 -- 172.21.15.92:0/1301940972 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e400eed0 con 0x7f53f40cad50 2023-10-06T10:44:56.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.689+0000 7f53eb7fe700 1 -- 172.21.15.92:0/1301940972 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e400c850 con 0x7f53f40cad50 2023-10-06T10:44:56.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.689+0000 7f53f9200700 1 -- 172.21.15.92:0/1301940972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f41663d0 con 0x7f53f40cad50 2023-10-06T10:44:56.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.689+0000 7f53eb7fe700 1 -- 172.21.15.92:0/1301940972 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e400cbf0 con 0x7f53f40cad50 2023-10-06T10:44:56.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.690+0000 7f53f9200700 1 -- 172.21.15.92:0/1301940972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f4165a50 con 0x7f53f40cad50 2023-10-06T10:44:56.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.690+0000 7f53f9200700 1 -- 172.21.15.92:0/1301940972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f40abfb0 con 0x7f53f40cad50 2023-10-06T10:44:56.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.691+0000 7f53eb7fe700 1 -- 172.21.15.92:0/1301940972 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f53e4013470 con 0x7f53f40cad50 2023-10-06T10:44:56.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.691+0000 7f53eb7fe700 1 --2- 172.21.15.92:0/1301940972 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f53d4066f30 0x7f53d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:56.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.691+0000 7f53eb7fe700 1 -- 172.21.15.92:0/1301940972 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f53e40849c0 con 0x7f53f40cad50 2023-10-06T10:44:56.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.691+0000 7f53ea59c700 1 --2- 172.21.15.92:0/1301940972 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f53d4066f30 0x7f53d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:56.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.692+0000 7f53ea59c700 1 --2- 172.21.15.92:0/1301940972 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f53d4066f30 0x7f53d40693e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0093f0 tx=0x7f53dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:56.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.696+0000 7f53eb7fe700 1 -- 172.21.15.92:0/1301940972 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53e4051990 con 0x7f53f40cad50 2023-10-06T10:44:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.953+0000 7f53f9200700 1 -- 172.21.15.92:0/1301940972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f53f405f230 con 0x7f53f40cad50 2023-10-06T10:44:56.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.953+0000 7f53eb7fe700 1 -- 172.21.15.92:0/1301940972 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1439 (secure 0 0 0) 0x7f53e4054fb0 con 0x7f53f40cad50 2023-10-06T10:44:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:56.954 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":418,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":23,"data_bytes":592674,"bytes_used":2450927616,"bytes_avail":770643185664,"bytes_total":773094113280,"write_bytes_sec":2218,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":100,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8520}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2023-10-06T10:44:44.166325+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T10:44:56.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.955+0000 7f53f9200700 1 -- 172.21.15.92:0/1301940972 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f53d4066f30 msgr2=0x7f53d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:56.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.955+0000 7f53f9200700 1 --2- 172.21.15.92:0/1301940972 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f53d4066f30 0x7f53d40693e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0093f0 tx=0x7f53dc008040 comp rx=0 tx=0).stop 2023-10-06T10:44:56.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.955+0000 7f53f9200700 1 -- 172.21.15.92:0/1301940972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53f40cad50 msgr2=0x7f53f4146de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:56.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.955+0000 7f53f9200700 1 --2- 172.21.15.92:0/1301940972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53f40cad50 0x7f53f4146de0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f53e400e930 tx=0x7f53e40091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.955+0000 7f53f9200700 1 -- 172.21.15.92:0/1301940972 shutdown_connections 2023-10-06T10:44:56.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.955+0000 7f53f9200700 1 --2- 172.21.15.92:0/1301940972 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f53d4066f30 0x7f53d40693e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.955+0000 7f53f9200700 1 --2- 172.21.15.92:0/1301940972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53f4158d10 0x7f53f414f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.955+0000 7f53f9200700 1 --2- 172.21.15.92:0/1301940972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53f40cad50 0x7f53f4146de0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.955+0000 7f53f9200700 1 --2- 172.21.15.92:0/1301940972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53f40cb1c0 0x7f53f40c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:56.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.955+0000 7f53f9200700 1 -- 172.21.15.92:0/1301940972 >> 172.21.15.92:0/1301940972 conn(0x7f53f40c9c80 msgr2=0x7f53f40707d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:56.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.955+0000 7f53f9200700 1 -- 172.21.15.92:0/1301940972 shutdown_connections 2023-10-06T10:44:56.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:56.955+0000 7f53f9200700 1 -- 172.21.15.92:0/1301940972 wait complete. 2023-10-06T10:44:56.968 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T10:44:56.969 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:44:57.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.082+0000 7fd470595700 1 Processor -- start 2023-10-06T10:44:57.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.082+0000 7fd470595700 1 -- start start 2023-10-06T10:44:57.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.083+0000 7fd470595700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd468159560 0x7fd4680aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:57.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.083+0000 7fd470595700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd4680b0760 0x7fd4680b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:57.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.083+0000 7fd470595700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4680b0c60 con 0x7fd4680b49a0 2023-10-06T10:44:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.083+0000 7fd470595700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4680b0f40 con 0x7fd4680b0760 2023-10-06T10:44:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.083+0000 7fd470595700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4680b1220 con 0x7fd468159560 2023-10-06T10:44:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.083+0000 7fd46eb32700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd4680b0760 0x7fd4680b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.083+0000 7fd46db30700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd4680b49a0 0x7fd4680aaa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:43926/0 (socket says 172.21.15.92:43926) 2023-10-06T10:44:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.083+0000 7fd46eb32700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd4680b0760 0x7fd4680b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33842/0 (socket says 172.21.15.92:33842) 2023-10-06T10:44:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.083+0000 7fd46db30700 1 -- 172.21.15.92:0/3896889259 learned_addr learned my addr 172.21.15.92:0/3896889259 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:44:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.083+0000 7fd46eb32700 1 -- 172.21.15.92:0/3896889259 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd468159560 msgr2=0x7fd4680aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:44:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.083+0000 7fd46eb32700 1 --2- 172.21.15.92:0/3896889259 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd468159560 0x7fd4680aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.083+0000 7fd46eb32700 1 -- 172.21.15.92:0/3896889259 >> v1:172.21.15.92:6789/0 conn(0x7fd4680b49a0 legacy=0x7fd4680aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.084+0000 7fd46eb32700 1 -- 172.21.15.92:0/3896889259 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4680b1500 con 0x7fd4680b0760 2023-10-06T10:44:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.083+0000 7fd46d32f700 1 -- 172.21.15.92:0/3896889259 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3850547776 0 0) 0x7fd4680b0c60 con 0x7fd4680b49a0 2023-10-06T10:44:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.084+0000 7fd46eb32700 1 --2- 172.21.15.92:0/3896889259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4680b0760 0x7fd4680b1bc0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fd460009e60 tx=0x7fd460009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4de69167814a0e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.084+0000 7fd46d32f700 1 -- 172.21.15.92:0/3896889259 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46000e070 con 0x7fd4680b0760 2023-10-06T10:44:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.084+0000 7fd46d32f700 1 -- 172.21.15.92:0/3896889259 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46000bc30 con 0x7fd4680b0760 2023-10-06T10:44:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.084+0000 7fd46d32f700 1 -- 172.21.15.92:0/3896889259 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd460012430 con 0x7fd4680b0760 2023-10-06T10:44:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.084+0000 7fd470595700 1 -- 172.21.15.92:0/3896889259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4680b0760 msgr2=0x7fd4680b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.084+0000 7fd470595700 1 --2- 172.21.15.92:0/3896889259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4680b0760 0x7fd4680b1bc0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fd460009e60 tx=0x7fd460009780 comp rx=0 tx=0).stop 2023-10-06T10:44:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.084+0000 7fd470595700 1 -- 172.21.15.92:0/3896889259 shutdown_connections 2023-10-06T10:44:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.084+0000 7fd470595700 1 --2- 172.21.15.92:0/3896889259 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd468159560 0x7fd4680aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.084+0000 7fd470595700 1 --2- 172.21.15.92:0/3896889259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4680b0760 0x7fd4680b1bc0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.084+0000 7fd470595700 1 -- 172.21.15.92:0/3896889259 >> 172.21.15.92:0/3896889259 conn(0x7fd4680997f0 msgr2=0x7fd46806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.084+0000 7fd470595700 1 -- 172.21.15.92:0/3896889259 shutdown_connections 2023-10-06T10:44:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.084+0000 7fd470595700 1 -- 172.21.15.92:0/3896889259 wait complete. 2023-10-06T10:44:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.085+0000 7fd470595700 1 Processor -- start 2023-10-06T10:44:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.086+0000 7fd470595700 1 -- start start 2023-10-06T10:44:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.086+0000 7fd470595700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4680b0760 0x7fd46815e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.086+0000 7fd470595700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4680b49a0 0x7fd46816a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.086+0000 7fd470595700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd468159560 0x7fd468099f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.086+0000 7fd470595700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46816ae90 con 0x7fd4680b0760 2023-10-06T10:44:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.086+0000 7fd470595700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46815ef00 con 0x7fd468159560 2023-10-06T10:44:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.086+0000 7fd470595700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46815bd70 con 0x7fd4680b49a0 2023-10-06T10:44:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.086+0000 7fd46e331700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4680b0760 0x7fd46815e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.086+0000 7fd46e331700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4680b0760 0x7fd46815e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33846/0 (socket says 172.21.15.92:33846) 2023-10-06T10:44:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.086+0000 7fd46e331700 1 -- 172.21.15.92:0/1655665771 learned_addr learned my addr 172.21.15.92:0/1655665771 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:44:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.087+0000 7fd46e331700 1 -- 172.21.15.92:0/1655665771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4680b49a0 msgr2=0x7fd46816a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.087+0000 7fd46e331700 1 --2- 172.21.15.92:0/1655665771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4680b49a0 0x7fd46816a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.087+0000 7fd46e331700 1 -- 172.21.15.92:0/1655665771 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd468159560 msgr2=0x7fd468099f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.087+0000 7fd46e331700 1 --2- 172.21.15.92:0/1655665771 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd468159560 0x7fd468099f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.087+0000 7fd46e331700 1 -- 172.21.15.92:0/1655665771 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd460009a20 con 0x7fd4680b0760 2023-10-06T10:44:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.087+0000 7fd46e331700 1 --2- 172.21.15.92:0/1655665771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4680b0760 0x7fd46815e870 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fd46400b600 tx=0x7fd4640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:57.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.087+0000 7fd45e7fc700 1 -- 172.21.15.92:0/1655665771 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46400f510 con 0x7fd4680b0760 2023-10-06T10:44:57.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.087+0000 7fd45e7fc700 1 -- 172.21.15.92:0/1655665771 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46400f690 con 0x7fd4680b0760 2023-10-06T10:44:57.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.087+0000 7fd470595700 1 -- 172.21.15.92:0/1655665771 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd46815c050 con 0x7fd4680b0760 2023-10-06T10:44:57.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.087+0000 7fd45e7fc700 1 -- 172.21.15.92:0/1655665771 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46400fa50 con 0x7fd4680b0760 2023-10-06T10:44:57.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.087+0000 7fd470595700 1 -- 172.21.15.92:0/1655665771 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46815c610 con 0x7fd4680b0760 2023-10-06T10:44:57.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.088+0000 7fd470595700 1 -- 172.21.15.92:0/1655665771 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd46815c830 con 0x7fd4680b0760 2023-10-06T10:44:57.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.088+0000 7fd45e7fc700 1 -- 172.21.15.92:0/1655665771 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd46401a400 con 0x7fd4680b0760 2023-10-06T10:44:57.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.089+0000 7fd45e7fc700 1 --2- 172.21.15.92:0/1655665771 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd44c066f30 0x7fd44c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:44:57.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.089+0000 7fd45e7fc700 1 -- 172.21.15.92:0/1655665771 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fd464084440 con 0x7fd4680b0760 2023-10-06T10:44:57.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.089+0000 7fd46db30700 1 --2- 172.21.15.92:0/1655665771 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd44c066f30 0x7fd44c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:44:57.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.089+0000 7fd46db30700 1 --2- 172.21.15.92:0/1655665771 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd44c066f30 0x7fd44c0693e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd4580011e0 tx=0x7fd4580079d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:44:57.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.093+0000 7fd45e7fc700 1 -- 172.21.15.92:0/1655665771 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd4640513f0 con 0x7fd4680b0760 2023-10-06T10:44:57.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.270+0000 7fd470595700 1 -- 172.21.15.92:0/1655665771 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd468064a90 con 0x7fd44c066f30 2023-10-06T10:44:57.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.278+0000 7fd45e7fc700 1 -- 172.21.15.92:0/1655665771 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312370 (secure 0 0 0) 0x7fd468064a90 con 0x7fd44c066f30 2023-10-06T10:44:57.282 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:44:57.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.282+0000 7fd470595700 1 -- 172.21.15.92:0/1655665771 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd44c066f30 msgr2=0x7fd44c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:57.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.282+0000 7fd470595700 1 --2- 172.21.15.92:0/1655665771 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd44c066f30 0x7fd44c0693e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd4580011e0 tx=0x7fd4580079d0 comp rx=0 tx=0).stop 2023-10-06T10:44:57.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.282+0000 7fd470595700 1 -- 172.21.15.92:0/1655665771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4680b0760 msgr2=0x7fd46815e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:44:57.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.282+0000 7fd470595700 1 --2- 172.21.15.92:0/1655665771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4680b0760 0x7fd46815e870 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fd46400b600 tx=0x7fd4640091a0 comp rx=0 tx=0).stop 2023-10-06T10:44:57.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.282+0000 7fd470595700 1 -- 172.21.15.92:0/1655665771 shutdown_connections 2023-10-06T10:44:57.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.282+0000 7fd470595700 1 --2- 172.21.15.92:0/1655665771 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd44c066f30 0x7fd44c0693e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:57.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.282+0000 7fd470595700 1 --2- 172.21.15.92:0/1655665771 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd468159560 0x7fd468099f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:57.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.282+0000 7fd470595700 1 --2- 172.21.15.92:0/1655665771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4680b0760 0x7fd46815e870 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:57.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.282+0000 7fd470595700 1 --2- 172.21.15.92:0/1655665771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4680b49a0 0x7fd46816a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:44:57.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.282+0000 7fd470595700 1 -- 172.21.15.92:0/1655665771 >> 172.21.15.92:0/1655665771 conn(0x7fd4680997f0 msgr2=0x7fd4680683c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:44:57.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.282+0000 7fd470595700 1 -- 172.21.15.92:0/1655665771 shutdown_connections 2023-10-06T10:44:57.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:44:57.282+0000 7fd470595700 1 -- 172.21.15.92:0/1655665771 wait complete. 2023-10-06T10:44:57.287 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:44:57.294 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2023-10-06T10:44:56.168451+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2393484,"kb_used_data":11084,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752581236,"statfs":{"total":773094113280,"available":770643185664,"internally_reserved":0,"allocated":11350016,"data_stored":6071633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001108"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.755450+0000","last_change":"2023-10-06T10:44:51.742214+0000","last_active":"2023-10-06T10:44:51.755450+0000","last_peered":"2023-10-06T10:44:51.755450+0000","last_clean":"2023-10-06T10:44:51.755450+0000","last_became_active":"2023-10-06T10:44:51.741939+0000","last_became_peered":"2023-10-06T10:44:51.741939+0000","last_unstale":"2023-10-06T10:44:51.755450+0000","last_undegraded":"2023-10-06T10:44:51.755450+0000","last_fullsized":"2023-10-06T10:44:51.755450+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:13:48.153672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.746271+0000","last_change":"2023-10-06T10:44:51.743338+0000","last_active":"2023-10-06T10:44:51.746271+0000","last_peered":"2023-10-06T10:44:51.746271+0000","last_clean":"2023-10-06T10:44:51.746271+0000","last_became_active":"2023-10-06T10:44:51.743074+0000","last_became_peered":"2023-10-06T10:44:51.743074+0000","last_unstale":"2023-10-06T10:44:51.746271+0000","last_undegraded":"2023-10-06T10:44:51.746271+0000","last_fullsized":"2023-10-06T10:44:51.746271+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:59.356737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745137+0000","last_change":"2023-10-06T10:44:51.740363+0000","last_active":"2023-10-06T10:44:51.745137+0000","last_peered":"2023-10-06T10:44:51.745137+0000","last_clean":"2023-10-06T10:44:51.745137+0000","last_became_active":"2023-10-06T10:44:51.739877+0000","last_became_peered":"2023-10-06T10:44:51.739877+0000","last_unstale":"2023-10-06T10:44:51.745137+0000","last_undegraded":"2023-10-06T10:44:51.745137+0000","last_fullsized":"2023-10-06T10:44:51.745137+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:11.372620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757596+0000","last_change":"2023-10-06T10:44:51.749770+0000","last_active":"2023-10-06T10:44:51.757596+0000","last_peered":"2023-10-06T10:44:51.757596+0000","last_clean":"2023-10-06T10:44:51.757596+0000","last_became_active":"2023-10-06T10:44:51.749314+0000","last_became_peered":"2023-10-06T10:44:51.749314+0000","last_unstale":"2023-10-06T10:44:51.757596+0000","last_undegraded":"2023-10-06T10:44:51.757596+0000","last_fullsized":"2023-10-06T10:44:51.757596+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:05:36.289745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757281+0000","last_change":"2023-10-06T10:44:51.752557+0000","last_active":"2023-10-06T10:44:51.757281+0000","last_peered":"2023-10-06T10:44:51.757281+0000","last_clean":"2023-10-06T10:44:51.757281+0000","last_became_active":"2023-10-06T10:44:51.752087+0000","last_became_peered":"2023-10-06T10:44:51.752087+0000","last_unstale":"2023-10-06T10:44:51.757281+0000","last_undegraded":"2023-10-06T10:44:51.757281+0000","last_fullsized":"2023-10-06T10:44:51.757281+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:30.552215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.751722+0000","last_change":"2023-10-06T10:44:51.749221+0000","last_active":"2023-10-06T10:44:51.751722+0000","last_peered":"2023-10-06T10:44:51.751722+0000","last_clean":"2023-10-06T10:44:51.751722+0000","last_became_active":"2023-10-06T10:44:51.749025+0000","last_became_peered":"2023-10-06T10:44:51.749025+0000","last_unstale":"2023-10-06T10:44:51.751722+0000","last_undegraded":"2023-10-06T10:44:51.751722+0000","last_fullsized":"2023-10-06T10:44:51.751722+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:28.610646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745548+0000","last_change":"2023-10-06T10:44:51.739336+0000","last_active":"2023-10-06T10:44:51.745548+0000","last_peered":"2023-10-06T10:44:51.745548+0000","last_clean":"2023-10-06T10:44:51.745548+0000","last_became_active":"2023-10-06T10:44:51.739055+0000","last_became_peered":"2023-10-06T10:44:51.739055+0000","last_unstale":"2023-10-06T10:44:51.745548+0000","last_undegraded":"2023-10-06T10:44:51.745548+0000","last_fullsized":"2023-10-06T10:44:51.745548+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:30.342288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.747260+0000","last_change":"2023-10-06T10:44:51.743966+0000","last_active":"2023-10-06T10:44:51.747260+0000","last_peered":"2023-10-06T10:44:51.747260+0000","last_clean":"2023-10-06T10:44:51.747260+0000","last_became_active":"2023-10-06T10:44:51.743686+0000","last_became_peered":"2023-10-06T10:44:51.743686+0000","last_unstale":"2023-10-06T10:44:51.747260+0000","last_undegraded":"2023-10-06T10:44:51.747260+0000","last_fullsized":"2023-10-06T10:44:51.747260+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:12:10.059460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.758682+0000","last_change":"2023-10-06T10:44:51.752074+0000","last_active":"2023-10-06T10:44:51.758682+0000","last_peered":"2023-10-06T10:44:51.758682+0000","last_clean":"2023-10-06T10:44:51.758682+0000","last_became_active":"2023-10-06T10:44:51.751397+0000","last_became_peered":"2023-10-06T10:44:51.751397+0000","last_unstale":"2023-10-06T10:44:51.758682+0000","last_undegraded":"2023-10-06T10:44:51.758682+0000","last_fullsized":"2023-10-06T10:44:51.758682+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:09.234346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.758578+0000","last_change":"2023-10-06T10:44:51.754015+0000","last_active":"2023-10-06T10:44:51.758578+0000","last_peered":"2023-10-06T10:44:51.758578+0000","last_clean":"2023-10-06T10:44:51.758578+0000","last_became_active":"2023-10-06T10:44:51.753770+0000","last_became_peered":"2023-10-06T10:44:51.753770+0000","last_unstale":"2023-10-06T10:44:51.758578+0000","last_undegraded":"2023-10-06T10:44:51.758578+0000","last_fullsized":"2023-10-06T10:44:51.758578+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:33:24.214039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745867+0000","last_change":"2023-10-06T10:44:51.741537+0000","last_active":"2023-10-06T10:44:51.745867+0000","last_peered":"2023-10-06T10:44:51.745867+0000","last_clean":"2023-10-06T10:44:51.745867+0000","last_became_active":"2023-10-06T10:44:51.740094+0000","last_became_peered":"2023-10-06T10:44:51.740094+0000","last_unstale":"2023-10-06T10:44:51.745867+0000","last_undegraded":"2023-10-06T10:44:51.745867+0000","last_fullsized":"2023-10-06T10:44:51.745867+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:00.855299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.760219+0000","last_change":"2023-10-06T10:44:51.756108+0000","last_active":"2023-10-06T10:44:51.760219+0000","last_peered":"2023-10-06T10:44:51.760219+0000","last_clean":"2023-10-06T10:44:51.760219+0000","last_became_active":"2023-10-06T10:44:51.755657+0000","last_became_peered":"2023-10-06T10:44:51.755657+0000","last_unstale":"2023-10-06T10:44:51.760219+0000","last_undegraded":"2023-10-06T10:44:51.760219+0000","last_fullsized":"2023-10-06T10:44:51.760219+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:45:43.330258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.754161+0000","last_change":"2023-10-06T10:44:51.746358+0000","last_active":"2023-10-06T10:44:51.754161+0000","last_peered":"2023-10-06T10:44:51.754161+0000","last_clean":"2023-10-06T10:44:51.754161+0000","last_became_active":"2023-10-06T10:44:51.746074+0000","last_became_peered":"2023-10-06T10:44:51.746074+0000","last_unstale":"2023-10-06T10:44:51.754161+0000","last_undegraded":"2023-10-06T10:44:51.754161+0000","last_fullsized":"2023-10-06T10:44:51.754161+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:44:07.998505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.744777+0000","last_change":"2023-10-06T10:44:51.740471+0000","last_active":"2023-10-06T10:44:51.744777+0000","last_peered":"2023-10-06T10:44:51.744777+0000","last_clean":"2023-10-06T10:44:51.744777+0000","last_became_active":"2023-10-06T10:44:51.740180+0000","last_became_peered":"2023-10-06T10:44:51.740180+0000","last_unstale":"2023-10-06T10:44:51.744777+0000","last_undegraded":"2023-10-06T10:44:51.744777+0000","last_fullsized":"2023-10-06T10:44:51.744777+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:45.409013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.748530+0000","last_change":"2023-10-06T10:44:51.745196+0000","last_active":"2023-10-06T10:44:51.748530+0000","last_peered":"2023-10-06T10:44:51.748530+0000","last_clean":"2023-10-06T10:44:51.748530+0000","last_became_active":"2023-10-06T10:44:51.744948+0000","last_became_peered":"2023-10-06T10:44:51.744948+0000","last_unstale":"2023-10-06T10:44:51.748530+0000","last_undegraded":"2023-10-06T10:44:51.748530+0000","last_fullsized":"2023-10-06T10:44:51.748530+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:11.554526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757351+0000","last_change":"2023-10-06T10:44:51.748954+0000","last_active":"2023-10-06T10:44:51.757351+0000","last_peered":"2023-10-06T10:44:51.757351+0000","last_clean":"2023-10-06T10:44:51.757351+0000","last_became_active":"2023-10-06T10:44:51.743005+0000","last_became_peered":"2023-10-06T10:44:51.743005+0000","last_unstale":"2023-10-06T10:44:51.757351+0000","last_undegraded":"2023-10-06T10:44:51.757351+0000","last_fullsized":"2023-10-06T10:44:51.757351+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:07.472732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745914+0000","last_change":"2023-10-06T10:44:51.740909+0000","last_active":"2023-10-06T10:44:51.745914+0000","last_peered":"2023-10-06T10:44:51.745914+0000","last_clean":"2023-10-06T10:44:51.745914+0000","last_became_active":"2023-10-06T10:44:51.740469+0000","last_became_peered":"2023-10-06T10:44:51.740469+0000","last_unstale":"2023-10-06T10:44:51.745914+0000","last_undegraded":"2023-10-06T10:44:51.745914+0000","last_fullsized":"2023-10-06T10:44:51.745914+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:29:51.113126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.744542+0000","last_change":"2023-10-06T10:44:51.739439+0000","last_active":"2023-10-06T10:44:51.744542+0000","last_peered":"2023-10-06T10:44:51.744542+0000","last_clean":"2023-10-06T10:44:51.744542+0000","last_became_active":"2023-10-06T10:44:51.739175+0000","last_became_peered":"2023-10-06T10:44:51.739175+0000","last_unstale":"2023-10-06T10:44:51.744542+0000","last_undegraded":"2023-10-06T10:44:51.744542+0000","last_fullsized":"2023-10-06T10:44:51.744542+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:11.372620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.754171+0000","last_change":"2023-10-06T10:44:51.746412+0000","last_active":"2023-10-06T10:44:51.754171+0000","last_peered":"2023-10-06T10:44:51.754171+0000","last_clean":"2023-10-06T10:44:51.754171+0000","last_became_active":"2023-10-06T10:44:51.746109+0000","last_became_peered":"2023-10-06T10:44:51.746109+0000","last_unstale":"2023-10-06T10:44:51.754171+0000","last_undegraded":"2023-10-06T10:44:51.754171+0000","last_fullsized":"2023-10-06T10:44:51.754171+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:15.138118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.746258+0000","last_change":"2023-10-06T10:44:51.741367+0000","last_active":"2023-10-06T10:44:51.746258+0000","last_peered":"2023-10-06T10:44:51.746258+0000","last_clean":"2023-10-06T10:44:51.746258+0000","last_became_active":"2023-10-06T10:44:51.741058+0000","last_became_peered":"2023-10-06T10:44:51.741058+0000","last_unstale":"2023-10-06T10:44:51.746258+0000","last_undegraded":"2023-10-06T10:44:51.746258+0000","last_fullsized":"2023-10-06T10:44:51.746258+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:21.512208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.753315+0000","last_change":"2023-10-06T10:44:51.747900+0000","last_active":"2023-10-06T10:44:51.753315+0000","last_peered":"2023-10-06T10:44:51.753315+0000","last_clean":"2023-10-06T10:44:51.753315+0000","last_became_active":"2023-10-06T10:44:51.747625+0000","last_became_peered":"2023-10-06T10:44:51.747625+0000","last_unstale":"2023-10-06T10:44:51.753315+0000","last_undegraded":"2023-10-06T10:44:51.753315+0000","last_fullsized":"2023-10-06T10:44:51.753315+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:56:26.690740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.748662+0000","last_change":"2023-10-06T10:44:51.743748+0000","last_active":"2023-10-06T10:44:51.748662+0000","last_peered":"2023-10-06T10:44:51.748662+0000","last_clean":"2023-10-06T10:44:51.748662+0000","last_became_active":"2023-10-06T10:44:51.743511+0000","last_became_peered":"2023-10-06T10:44:51.743511+0000","last_unstale":"2023-10-06T10:44:51.748662+0000","last_undegraded":"2023-10-06T10:44:51.748662+0000","last_fullsized":"2023-10-06T10:44:51.748662+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:58:58.968944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.746207+0000","last_change":"2023-10-06T10:44:51.743049+0000","last_active":"2023-10-06T10:44:51.746207+0000","last_peered":"2023-10-06T10:44:51.746207+0000","last_clean":"2023-10-06T10:44:51.746207+0000","last_became_active":"2023-10-06T10:44:51.740779+0000","last_became_peered":"2023-10-06T10:44:51.740779+0000","last_unstale":"2023-10-06T10:44:51.746207+0000","last_undegraded":"2023-10-06T10:44:51.746207+0000","last_fullsized":"2023-10-06T10:44:51.746207+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:27:22.155683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745690+0000","last_change":"2023-10-06T10:44:51.730584+0000","last_active":"2023-10-06T10:44:51.745690+0000","last_peered":"2023-10-06T10:44:51.745690+0000","last_clean":"2023-10-06T10:44:51.745690+0000","last_became_active":"2023-10-06T10:44:51.729878+0000","last_became_peered":"2023-10-06T10:44:51.729878+0000","last_unstale":"2023-10-06T10:44:51.745690+0000","last_undegraded":"2023-10-06T10:44:51.745690+0000","last_fullsized":"2023-10-06T10:44:51.745690+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:59.356737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745892+0000","last_change":"2023-10-06T10:44:51.740729+0000","last_active":"2023-10-06T10:44:51.745892+0000","last_peered":"2023-10-06T10:44:51.745892+0000","last_clean":"2023-10-06T10:44:51.745892+0000","last_became_active":"2023-10-06T10:44:51.740139+0000","last_became_peered":"2023-10-06T10:44:51.740139+0000","last_unstale":"2023-10-06T10:44:51.745892+0000","last_undegraded":"2023-10-06T10:44:51.745892+0000","last_fullsized":"2023-10-06T10:44:51.745892+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:07:26.376540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757745+0000","last_change":"2023-10-06T10:44:51.751633+0000","last_active":"2023-10-06T10:44:51.757745+0000","last_peered":"2023-10-06T10:44:51.757745+0000","last_clean":"2023-10-06T10:44:51.757745+0000","last_became_active":"2023-10-06T10:44:51.751325+0000","last_became_peered":"2023-10-06T10:44:51.751325+0000","last_unstale":"2023-10-06T10:44:51.757745+0000","last_undegraded":"2023-10-06T10:44:51.757745+0000","last_fullsized":"2023-10-06T10:44:51.757745+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:34.107458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757927+0000","last_change":"2023-10-06T10:44:51.754195+0000","last_active":"2023-10-06T10:44:51.757927+0000","last_peered":"2023-10-06T10:44:51.757927+0000","last_clean":"2023-10-06T10:44:51.757927+0000","last_became_active":"2023-10-06T10:44:51.753939+0000","last_became_peered":"2023-10-06T10:44:51.753939+0000","last_unstale":"2023-10-06T10:44:51.757927+0000","last_undegraded":"2023-10-06T10:44:51.757927+0000","last_fullsized":"2023-10-06T10:44:51.757927+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:14:19.607356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.755601+0000","last_change":"2023-10-06T10:44:51.753173+0000","last_active":"2023-10-06T10:44:51.755601+0000","last_peered":"2023-10-06T10:44:51.755601+0000","last_clean":"2023-10-06T10:44:51.755601+0000","last_became_active":"2023-10-06T10:44:51.752806+0000","last_became_peered":"2023-10-06T10:44:51.752806+0000","last_unstale":"2023-10-06T10:44:51.755601+0000","last_undegraded":"2023-10-06T10:44:51.755601+0000","last_fullsized":"2023-10-06T10:44:51.755601+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:27:22.155683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.746044+0000","last_change":"2023-10-06T10:44:51.741443+0000","last_active":"2023-10-06T10:44:51.746044+0000","last_peered":"2023-10-06T10:44:51.746044+0000","last_clean":"2023-10-06T10:44:51.746044+0000","last_became_active":"2023-10-06T10:44:51.740884+0000","last_became_peered":"2023-10-06T10:44:51.740884+0000","last_unstale":"2023-10-06T10:44:51.746044+0000","last_undegraded":"2023-10-06T10:44:51.746044+0000","last_fullsized":"2023-10-06T10:44:51.746044+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:15.056633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.739324+0000","last_change":"2023-10-06T10:44:51.734925+0000","last_active":"2023-10-06T10:44:51.739324+0000","last_peered":"2023-10-06T10:44:51.739324+0000","last_clean":"2023-10-06T10:44:51.739324+0000","last_became_active":"2023-10-06T10:44:51.734516+0000","last_became_peered":"2023-10-06T10:44:51.734516+0000","last_unstale":"2023-10-06T10:44:51.739324+0000","last_undegraded":"2023-10-06T10:44:51.739324+0000","last_fullsized":"2023-10-06T10:44:51.739324+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:11:42.494648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.755509+0000","last_change":"2023-10-06T10:44:51.743028+0000","last_active":"2023-10-06T10:44:51.755509+0000","last_peered":"2023-10-06T10:44:51.755509+0000","last_clean":"2023-10-06T10:44:51.755509+0000","last_became_active":"2023-10-06T10:44:51.742742+0000","last_became_peered":"2023-10-06T10:44:51.742742+0000","last_unstale":"2023-10-06T10:44:51.755509+0000","last_undegraded":"2023-10-06T10:44:51.755509+0000","last_fullsized":"2023-10-06T10:44:51.755509+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:31:28.825035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.750999+0000","last_change":"2023-10-06T10:44:51.747804+0000","last_active":"2023-10-06T10:44:51.750999+0000","last_peered":"2023-10-06T10:44:51.750999+0000","last_clean":"2023-10-06T10:44:51.750999+0000","last_became_active":"2023-10-06T10:44:51.747553+0000","last_became_peered":"2023-10-06T10:44:51.747553+0000","last_unstale":"2023-10-06T10:44:51.750999+0000","last_undegraded":"2023-10-06T10:44:51.750999+0000","last_fullsized":"2023-10-06T10:44:51.750999+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:00:47.142581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.749647+0000","last_change":"2023-10-06T10:44:51.746945+0000","last_active":"2023-10-06T10:44:51.749647+0000","last_peered":"2023-10-06T10:44:51.749647+0000","last_clean":"2023-10-06T10:44:51.749647+0000","last_became_active":"2023-10-06T10:44:51.746807+0000","last_became_peered":"2023-10-06T10:44:51.746807+0000","last_unstale":"2023-10-06T10:44:51.749647+0000","last_undegraded":"2023-10-06T10:44:51.749647+0000","last_fullsized":"2023-10-06T10:44:51.749647+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:58.636572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"101'4","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.747192+0000","last_change":"2023-10-06T10:44:50.738760+0000","last_active":"2023-10-06T10:44:51.747192+0000","last_peered":"2023-10-06T10:44:51.747192+0000","last_clean":"2023-10-06T10:44:51.747192+0000","last_became_active":"2023-10-06T10:44:50.738480+0000","last_became_peered":"2023-10-06T10:44:50.738480+0000","last_unstale":"2023-10-06T10:44:51.747192+0000","last_undegraded":"2023-10-06T10:44:51.747192+0000","last_fullsized":"2023-10-06T10:44:51.747192+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:29.546521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.744979+0000","last_change":"2023-10-06T10:44:51.741107+0000","last_active":"2023-10-06T10:44:51.744979+0000","last_peered":"2023-10-06T10:44:51.744979+0000","last_clean":"2023-10-06T10:44:51.744979+0000","last_became_active":"2023-10-06T10:44:51.740656+0000","last_became_peered":"2023-10-06T10:44:51.740656+0000","last_unstale":"2023-10-06T10:44:51.744979+0000","last_undegraded":"2023-10-06T10:44:51.744979+0000","last_fullsized":"2023-10-06T10:44:51.744979+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:53.858578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"101'1","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.742828+0000","last_change":"2023-10-06T10:44:50.739987+0000","last_active":"2023-10-06T10:44:51.742828+0000","last_peered":"2023-10-06T10:44:51.742828+0000","last_clean":"2023-10-06T10:44:51.742828+0000","last_became_active":"2023-10-06T10:44:50.739669+0000","last_became_peered":"2023-10-06T10:44:50.739669+0000","last_unstale":"2023-10-06T10:44:51.742828+0000","last_undegraded":"2023-10-06T10:44:51.742828+0000","last_fullsized":"2023-10-06T10:44:51.742828+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:38:47.009118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.749627+0000","last_change":"2023-10-06T10:44:51.746313+0000","last_active":"2023-10-06T10:44:51.749627+0000","last_peered":"2023-10-06T10:44:51.749627+0000","last_clean":"2023-10-06T10:44:51.749627+0000","last_became_active":"2023-10-06T10:44:51.746064+0000","last_became_peered":"2023-10-06T10:44:51.746064+0000","last_unstale":"2023-10-06T10:44:51.749627+0000","last_undegraded":"2023-10-06T10:44:51.749627+0000","last_fullsized":"2023-10-06T10:44:51.749627+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:44:59.326168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"101'2","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.752912+0000","last_change":"2023-10-06T10:44:50.737016+0000","last_active":"2023-10-06T10:44:51.752912+0000","last_peered":"2023-10-06T10:44:51.752912+0000","last_clean":"2023-10-06T10:44:51.752912+0000","last_became_active":"2023-10-06T10:44:50.736567+0000","last_became_peered":"2023-10-06T10:44:50.736567+0000","last_unstale":"2023-10-06T10:44:51.752912+0000","last_undegraded":"2023-10-06T10:44:51.752912+0000","last_fullsized":"2023-10-06T10:44:51.752912+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:08.868364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.754205+0000","last_change":"2023-10-06T10:44:51.746821+0000","last_active":"2023-10-06T10:44:51.754205+0000","last_peered":"2023-10-06T10:44:51.754205+0000","last_clean":"2023-10-06T10:44:51.754205+0000","last_became_active":"2023-10-06T10:44:51.746576+0000","last_became_peered":"2023-10-06T10:44:51.746576+0000","last_unstale":"2023-10-06T10:44:51.754205+0000","last_undegraded":"2023-10-06T10:44:51.754205+0000","last_fullsized":"2023-10-06T10:44:51.754205+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:06:43.616700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"101'2","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.756745+0000","last_change":"2023-10-06T10:44:50.738193+0000","last_active":"2023-10-06T10:44:51.756745+0000","last_peered":"2023-10-06T10:44:51.756745+0000","last_clean":"2023-10-06T10:44:51.756745+0000","last_became_active":"2023-10-06T10:44:50.737952+0000","last_became_peered":"2023-10-06T10:44:50.737952+0000","last_unstale":"2023-10-06T10:44:51.756745+0000","last_undegraded":"2023-10-06T10:44:51.756745+0000","last_fullsized":"2023-10-06T10:44:51.756745+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:31.698962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757718+0000","last_change":"2023-10-06T10:44:51.751538+0000","last_active":"2023-10-06T10:44:51.757718+0000","last_peered":"2023-10-06T10:44:51.757718+0000","last_clean":"2023-10-06T10:44:51.757718+0000","last_became_active":"2023-10-06T10:44:51.751239+0000","last_became_peered":"2023-10-06T10:44:51.751239+0000","last_unstale":"2023-10-06T10:44:51.757718+0000","last_undegraded":"2023-10-06T10:44:51.757718+0000","last_fullsized":"2023-10-06T10:44:51.757718+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:46.027434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"101'1","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.743770+0000","last_change":"2023-10-06T10:44:50.740417+0000","last_active":"2023-10-06T10:44:51.743770+0000","last_peered":"2023-10-06T10:44:51.743770+0000","last_clean":"2023-10-06T10:44:51.743770+0000","last_became_active":"2023-10-06T10:44:50.740148+0000","last_became_peered":"2023-10-06T10:44:50.740148+0000","last_unstale":"2023-10-06T10:44:51.743770+0000","last_undegraded":"2023-10-06T10:44:51.743770+0000","last_fullsized":"2023-10-06T10:44:51.743770+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:02.006450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.750448+0000","last_change":"2023-10-06T10:44:51.744759+0000","last_active":"2023-10-06T10:44:51.750448+0000","last_peered":"2023-10-06T10:44:51.750448+0000","last_clean":"2023-10-06T10:44:51.750448+0000","last_became_active":"2023-10-06T10:44:51.744499+0000","last_became_peered":"2023-10-06T10:44:51.744499+0000","last_unstale":"2023-10-06T10:44:51.750448+0000","last_undegraded":"2023-10-06T10:44:51.750448+0000","last_fullsized":"2023-10-06T10:44:51.750448+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:58:58.968944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"101'1","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.750469+0000","last_change":"2023-10-06T10:44:50.736564+0000","last_active":"2023-10-06T10:44:51.750469+0000","last_peered":"2023-10-06T10:44:51.750469+0000","last_clean":"2023-10-06T10:44:51.750469+0000","last_became_active":"2023-10-06T10:44:50.736297+0000","last_became_peered":"2023-10-06T10:44:50.736297+0000","last_unstale":"2023-10-06T10:44:51.750469+0000","last_undegraded":"2023-10-06T10:44:51.750469+0000","last_fullsized":"2023-10-06T10:44:51.750469+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:04:17.777010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757369+0000","last_change":"2023-10-06T10:44:51.749198+0000","last_active":"2023-10-06T10:44:51.757369+0000","last_peered":"2023-10-06T10:44:51.757369+0000","last_clean":"2023-10-06T10:44:51.757369+0000","last_became_active":"2023-10-06T10:44:51.748962+0000","last_became_peered":"2023-10-06T10:44:51.748962+0000","last_unstale":"2023-10-06T10:44:51.757369+0000","last_undegraded":"2023-10-06T10:44:51.757369+0000","last_fullsized":"2023-10-06T10:44:51.757369+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:39.790377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.732785+0000","last_change":"2023-10-06T10:44:50.732777+0000","last_active":"2023-10-06T10:44:51.732785+0000","last_peered":"2023-10-06T10:44:51.732785+0000","last_clean":"2023-10-06T10:44:51.732785+0000","last_became_active":"2023-10-06T10:44:50.732494+0000","last_became_peered":"2023-10-06T10:44:50.732494+0000","last_unstale":"2023-10-06T10:44:51.732785+0000","last_undegraded":"2023-10-06T10:44:51.732785+0000","last_fullsized":"2023-10-06T10:44:51.732785+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:26:41.299786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745570+0000","last_change":"2023-10-06T10:44:51.739689+0000","last_active":"2023-10-06T10:44:51.745570+0000","last_peered":"2023-10-06T10:44:51.745570+0000","last_clean":"2023-10-06T10:44:51.745570+0000","last_became_active":"2023-10-06T10:44:51.739426+0000","last_became_peered":"2023-10-06T10:44:51.739426+0000","last_unstale":"2023-10-06T10:44:51.745570+0000","last_undegraded":"2023-10-06T10:44:51.745570+0000","last_fullsized":"2023-10-06T10:44:51.745570+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:38:48.014813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.735795+0000","last_change":"2023-10-06T10:44:50.746777+0000","last_active":"2023-10-06T10:44:51.735795+0000","last_peered":"2023-10-06T10:44:51.735795+0000","last_clean":"2023-10-06T10:44:51.735795+0000","last_became_active":"2023-10-06T10:44:50.746550+0000","last_became_peered":"2023-10-06T10:44:50.746550+0000","last_unstale":"2023-10-06T10:44:51.735795+0000","last_undegraded":"2023-10-06T10:44:51.735795+0000","last_fullsized":"2023-10-06T10:44:51.735795+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:41.068289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.758917+0000","last_change":"2023-10-06T10:44:51.756739+0000","last_active":"2023-10-06T10:44:51.758917+0000","last_peered":"2023-10-06T10:44:51.758917+0000","last_clean":"2023-10-06T10:44:51.758917+0000","last_became_active":"2023-10-06T10:44:51.753839+0000","last_became_peered":"2023-10-06T10:44:51.753839+0000","last_unstale":"2023-10-06T10:44:51.758917+0000","last_undegraded":"2023-10-06T10:44:51.758917+0000","last_fullsized":"2023-10-06T10:44:51.758917+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:14:19.607356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"101'1","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.751747+0000","last_change":"2023-10-06T10:44:50.743094+0000","last_active":"2023-10-06T10:44:51.751747+0000","last_peered":"2023-10-06T10:44:51.751747+0000","last_clean":"2023-10-06T10:44:51.751747+0000","last_became_active":"2023-10-06T10:44:50.742703+0000","last_became_peered":"2023-10-06T10:44:50.742703+0000","last_unstale":"2023-10-06T10:44:51.751747+0000","last_undegraded":"2023-10-06T10:44:51.751747+0000","last_fullsized":"2023-10-06T10:44:51.751747+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:25.575241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.748437+0000","last_change":"2023-10-06T10:44:51.744523+0000","last_active":"2023-10-06T10:44:51.748437+0000","last_peered":"2023-10-06T10:44:51.748437+0000","last_clean":"2023-10-06T10:44:51.748437+0000","last_became_active":"2023-10-06T10:44:51.744272+0000","last_became_peered":"2023-10-06T10:44:51.744272+0000","last_unstale":"2023-10-06T10:44:51.748437+0000","last_undegraded":"2023-10-06T10:44:51.748437+0000","last_fullsized":"2023-10-06T10:44:51.748437+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:03.012145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"101'2","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.753032+0000","last_change":"2023-10-06T10:44:50.735822+0000","last_active":"2023-10-06T10:44:51.753032+0000","last_peered":"2023-10-06T10:44:51.753032+0000","last_clean":"2023-10-06T10:44:51.753032+0000","last_became_active":"2023-10-06T10:44:50.735568+0000","last_became_peered":"2023-10-06T10:44:50.735568+0000","last_unstale":"2023-10-06T10:44:51.753032+0000","last_undegraded":"2023-10-06T10:44:51.753032+0000","last_fullsized":"2023-10-06T10:44:51.753032+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:38:19.424287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.756698+0000","last_change":"2023-10-06T10:44:51.754072+0000","last_active":"2023-10-06T10:44:51.756698+0000","last_peered":"2023-10-06T10:44:51.756698+0000","last_clean":"2023-10-06T10:44:51.756698+0000","last_became_active":"2023-10-06T10:44:51.753802+0000","last_became_peered":"2023-10-06T10:44:51.753802+0000","last_unstale":"2023-10-06T10:44:51.756698+0000","last_undegraded":"2023-10-06T10:44:51.756698+0000","last_fullsized":"2023-10-06T10:44:51.756698+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:48:22.355874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"101'1","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.750611+0000","last_change":"2023-10-06T10:44:50.740185+0000","last_active":"2023-10-06T10:44:51.750611+0000","last_peered":"2023-10-06T10:44:51.750611+0000","last_clean":"2023-10-06T10:44:51.750611+0000","last_became_active":"2023-10-06T10:44:50.739930+0000","last_became_peered":"2023-10-06T10:44:50.739930+0000","last_unstale":"2023-10-06T10:44:51.750611+0000","last_undegraded":"2023-10-06T10:44:51.750611+0000","last_fullsized":"2023-10-06T10:44:51.750611+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:47:30.907849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.739759+0000","last_change":"2023-10-06T10:44:51.729671+0000","last_active":"2023-10-06T10:44:51.739759+0000","last_peered":"2023-10-06T10:44:51.739759+0000","last_clean":"2023-10-06T10:44:51.739759+0000","last_became_active":"2023-10-06T10:44:51.729451+0000","last_became_peered":"2023-10-06T10:44:51.729451+0000","last_unstale":"2023-10-06T10:44:51.739759+0000","last_undegraded":"2023-10-06T10:44:51.739759+0000","last_fullsized":"2023-10-06T10:44:51.739759+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:33:51.029950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.725200+0000","last_change":"2023-10-06T10:44:50.743084+0000","last_active":"2023-10-06T10:44:51.725200+0000","last_peered":"2023-10-06T10:44:51.725200+0000","last_clean":"2023-10-06T10:44:51.725200+0000","last_became_active":"2023-10-06T10:44:50.742740+0000","last_became_peered":"2023-10-06T10:44:50.742740+0000","last_unstale":"2023-10-06T10:44:51.725200+0000","last_undegraded":"2023-10-06T10:44:51.725200+0000","last_fullsized":"2023-10-06T10:44:51.725200+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:23.746429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.739337+0000","last_change":"2023-10-06T10:44:51.735032+0000","last_active":"2023-10-06T10:44:51.739337+0000","last_peered":"2023-10-06T10:44:51.739337+0000","last_clean":"2023-10-06T10:44:51.739337+0000","last_became_active":"2023-10-06T10:44:51.734765+0000","last_became_peered":"2023-10-06T10:44:51.734765+0000","last_unstale":"2023-10-06T10:44:51.739337+0000","last_undegraded":"2023-10-06T10:44:51.739337+0000","last_fullsized":"2023-10-06T10:44:51.739337+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:53.858578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"101'2","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757070+0000","last_change":"2023-10-06T10:44:50.735989+0000","last_active":"2023-10-06T10:44:51.757070+0000","last_peered":"2023-10-06T10:44:51.757070+0000","last_clean":"2023-10-06T10:44:51.757070+0000","last_became_active":"2023-10-06T10:44:50.735704+0000","last_became_peered":"2023-10-06T10:44:50.735704+0000","last_unstale":"2023-10-06T10:44:51.757070+0000","last_undegraded":"2023-10-06T10:44:51.757070+0000","last_fullsized":"2023-10-06T10:44:51.757070+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:21.691363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.755630+0000","last_change":"2023-10-06T10:44:51.746428+0000","last_active":"2023-10-06T10:44:51.755630+0000","last_peered":"2023-10-06T10:44:51.755630+0000","last_clean":"2023-10-06T10:44:51.755630+0000","last_became_active":"2023-10-06T10:44:51.746168+0000","last_became_peered":"2023-10-06T10:44:51.746168+0000","last_unstale":"2023-10-06T10:44:51.755630+0000","last_undegraded":"2023-10-06T10:44:51.755630+0000","last_fullsized":"2023-10-06T10:44:51.755630+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:17:28.335605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.733095+0000","last_change":"2023-10-06T10:44:50.737860+0000","last_active":"2023-10-06T10:44:51.733095+0000","last_peered":"2023-10-06T10:44:51.733095+0000","last_clean":"2023-10-06T10:44:51.733095+0000","last_became_active":"2023-10-06T10:44:50.737174+0000","last_became_peered":"2023-10-06T10:44:50.737174+0000","last_unstale":"2023-10-06T10:44:51.733095+0000","last_undegraded":"2023-10-06T10:44:51.733095+0000","last_fullsized":"2023-10-06T10:44:51.733095+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:13.818223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.754190+0000","last_change":"2023-10-06T10:44:51.746749+0000","last_active":"2023-10-06T10:44:51.754190+0000","last_peered":"2023-10-06T10:44:51.754190+0000","last_clean":"2023-10-06T10:44:51.754190+0000","last_became_active":"2023-10-06T10:44:51.746525+0000","last_became_peered":"2023-10-06T10:44:51.746525+0000","last_unstale":"2023-10-06T10:44:51.754190+0000","last_undegraded":"2023-10-06T10:44:51.754190+0000","last_fullsized":"2023-10-06T10:44:51.754190+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:13.902450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.729551+0000","last_change":"2023-10-06T10:44:50.740163+0000","last_active":"2023-10-06T10:44:51.729551+0000","last_peered":"2023-10-06T10:44:51.729551+0000","last_clean":"2023-10-06T10:44:51.729551+0000","last_became_active":"2023-10-06T10:44:50.739892+0000","last_became_peered":"2023-10-06T10:44:50.739892+0000","last_unstale":"2023-10-06T10:44:51.729551+0000","last_undegraded":"2023-10-06T10:44:51.729551+0000","last_fullsized":"2023-10-06T10:44:51.729551+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:31.698962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.747134+0000","last_change":"2023-10-06T10:44:51.743095+0000","last_active":"2023-10-06T10:44:51.747134+0000","last_peered":"2023-10-06T10:44:51.747134+0000","last_clean":"2023-10-06T10:44:51.747134+0000","last_became_active":"2023-10-06T10:44:51.742863+0000","last_became_peered":"2023-10-06T10:44:51.742863+0000","last_unstale":"2023-10-06T10:44:51.747134+0000","last_undegraded":"2023-10-06T10:44:51.747134+0000","last_fullsized":"2023-10-06T10:44:51.747134+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:56:26.690740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.735885+0000","last_change":"2023-10-06T10:44:50.742684+0000","last_active":"2023-10-06T10:44:51.735885+0000","last_peered":"2023-10-06T10:44:51.735885+0000","last_clean":"2023-10-06T10:44:51.735885+0000","last_became_active":"2023-10-06T10:44:50.742434+0000","last_became_peered":"2023-10-06T10:44:50.742434+0000","last_unstale":"2023-10-06T10:44:51.735885+0000","last_undegraded":"2023-10-06T10:44:51.735885+0000","last_fullsized":"2023-10-06T10:44:51.735885+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:57.037993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.753591+0000","last_change":"2023-10-06T10:44:51.751722+0000","last_active":"2023-10-06T10:44:51.753591+0000","last_peered":"2023-10-06T10:44:51.753591+0000","last_clean":"2023-10-06T10:44:51.753591+0000","last_became_active":"2023-10-06T10:44:51.751490+0000","last_became_peered":"2023-10-06T10:44:51.751490+0000","last_unstale":"2023-10-06T10:44:51.753591+0000","last_undegraded":"2023-10-06T10:44:51.753591+0000","last_fullsized":"2023-10-06T10:44:51.753591+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:45.409013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"101'2","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.747535+0000","last_change":"2023-10-06T10:44:50.745009+0000","last_active":"2023-10-06T10:44:51.747535+0000","last_peered":"2023-10-06T10:44:51.747535+0000","last_clean":"2023-10-06T10:44:51.747535+0000","last_became_active":"2023-10-06T10:44:50.744698+0000","last_became_peered":"2023-10-06T10:44:50.744698+0000","last_unstale":"2023-10-06T10:44:51.747535+0000","last_undegraded":"2023-10-06T10:44:51.747535+0000","last_fullsized":"2023-10-06T10:44:51.747535+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:26:41.299786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.758669+0000","last_change":"2023-10-06T10:44:51.751880+0000","last_active":"2023-10-06T10:44:51.758669+0000","last_peered":"2023-10-06T10:44:51.758669+0000","last_clean":"2023-10-06T10:44:51.758669+0000","last_became_active":"2023-10-06T10:44:51.751258+0000","last_became_peered":"2023-10-06T10:44:51.751258+0000","last_unstale":"2023-10-06T10:44:51.758669+0000","last_undegraded":"2023-10-06T10:44:51.758669+0000","last_fullsized":"2023-10-06T10:44:51.758669+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:07.487093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.732945+0000","last_change":"2023-10-06T10:44:50.735743+0000","last_active":"2023-10-06T10:44:51.732945+0000","last_peered":"2023-10-06T10:44:51.732945+0000","last_clean":"2023-10-06T10:44:51.732945+0000","last_became_active":"2023-10-06T10:44:50.735498+0000","last_became_peered":"2023-10-06T10:44:50.735498+0000","last_unstale":"2023-10-06T10:44:51.732945+0000","last_undegraded":"2023-10-06T10:44:51.732945+0000","last_fullsized":"2023-10-06T10:44:51.732945+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:19.608630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757616+0000","last_change":"2023-10-06T10:44:51.749866+0000","last_active":"2023-10-06T10:44:51.757616+0000","last_peered":"2023-10-06T10:44:51.757616+0000","last_clean":"2023-10-06T10:44:51.757616+0000","last_became_active":"2023-10-06T10:44:51.749578+0000","last_became_peered":"2023-10-06T10:44:51.749578+0000","last_unstale":"2023-10-06T10:44:51.757616+0000","last_undegraded":"2023-10-06T10:44:51.757616+0000","last_fullsized":"2023-10-06T10:44:51.757616+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:52.361795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.734752+0000","last_change":"2023-10-06T10:44:50.740894+0000","last_active":"2023-10-06T10:44:51.734752+0000","last_peered":"2023-10-06T10:44:51.734752+0000","last_clean":"2023-10-06T10:44:51.734752+0000","last_became_active":"2023-10-06T10:44:50.740653+0000","last_became_peered":"2023-10-06T10:44:50.740653+0000","last_unstale":"2023-10-06T10:44:51.734752+0000","last_undegraded":"2023-10-06T10:44:51.734752+0000","last_fullsized":"2023-10-06T10:44:51.734752+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:11.169696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.754014+0000","last_change":"2023-10-06T10:44:51.751873+0000","last_active":"2023-10-06T10:44:51.754014+0000","last_peered":"2023-10-06T10:44:51.754014+0000","last_clean":"2023-10-06T10:44:51.754014+0000","last_became_active":"2023-10-06T10:44:51.751589+0000","last_became_peered":"2023-10-06T10:44:51.751589+0000","last_unstale":"2023-10-06T10:44:51.754014+0000","last_undegraded":"2023-10-06T10:44:51.754014+0000","last_fullsized":"2023-10-06T10:44:51.754014+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:12:10.059460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.726919+0000","last_change":"2023-10-06T10:44:50.739163+0000","last_active":"2023-10-06T10:44:51.726919+0000","last_peered":"2023-10-06T10:44:51.726919+0000","last_clean":"2023-10-06T10:44:51.726919+0000","last_became_active":"2023-10-06T10:44:50.738915+0000","last_became_peered":"2023-10-06T10:44:50.738915+0000","last_unstale":"2023-10-06T10:44:51.726919+0000","last_undegraded":"2023-10-06T10:44:51.726919+0000","last_fullsized":"2023-10-06T10:44:51.726919+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:32.689348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.737452+0000","last_change":"2023-10-06T10:44:51.736064+0000","last_active":"2023-10-06T10:44:51.737452+0000","last_peered":"2023-10-06T10:44:51.737452+0000","last_clean":"2023-10-06T10:44:51.737452+0000","last_became_active":"2023-10-06T10:44:51.735730+0000","last_became_peered":"2023-10-06T10:44:51.735730+0000","last_unstale":"2023-10-06T10:44:51.737452+0000","last_undegraded":"2023-10-06T10:44:51.737452+0000","last_fullsized":"2023-10-06T10:44:51.737452+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:42.073984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.725056+0000","last_change":"2023-10-06T10:44:50.738749+0000","last_active":"2023-10-06T10:44:51.725056+0000","last_peered":"2023-10-06T10:44:51.725056+0000","last_clean":"2023-10-06T10:44:51.725056+0000","last_became_active":"2023-10-06T10:44:50.738444+0000","last_became_peered":"2023-10-06T10:44:50.738444+0000","last_unstale":"2023-10-06T10:44:51.725056+0000","last_undegraded":"2023-10-06T10:44:51.725056+0000","last_fullsized":"2023-10-06T10:44:51.725056+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:50:16.152816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.756270+0000","last_change":"2023-10-06T10:44:51.753208+0000","last_active":"2023-10-06T10:44:51.756270+0000","last_peered":"2023-10-06T10:44:51.756270+0000","last_clean":"2023-10-06T10:44:51.756270+0000","last_became_active":"2023-10-06T10:44:51.752963+0000","last_became_peered":"2023-10-06T10:44:51.752963+0000","last_unstale":"2023-10-06T10:44:51.756270+0000","last_undegraded":"2023-10-06T10:44:51.756270+0000","last_fullsized":"2023-10-06T10:44:51.756270+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:58.043687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.732962+0000","last_change":"2023-10-06T10:44:50.740319+0000","last_active":"2023-10-06T10:44:51.732962+0000","last_peered":"2023-10-06T10:44:51.732962+0000","last_clean":"2023-10-06T10:44:51.732962+0000","last_became_active":"2023-10-06T10:44:50.740054+0000","last_became_peered":"2023-10-06T10:44:50.740054+0000","last_unstale":"2023-10-06T10:44:51.732962+0000","last_undegraded":"2023-10-06T10:44:51.732962+0000","last_fullsized":"2023-10-06T10:44:51.732962+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:05:34.133718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.744832+0000","last_change":"2023-10-06T10:44:51.740932+0000","last_active":"2023-10-06T10:44:51.744832+0000","last_peered":"2023-10-06T10:44:51.744832+0000","last_clean":"2023-10-06T10:44:51.744832+0000","last_became_active":"2023-10-06T10:44:51.740255+0000","last_became_peered":"2023-10-06T10:44:51.740255+0000","last_unstale":"2023-10-06T10:44:51.744832+0000","last_undegraded":"2023-10-06T10:44:51.744832+0000","last_fullsized":"2023-10-06T10:44:51.744832+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:33.695043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.737226+0000","last_change":"2023-10-06T10:44:50.737048+0000","last_active":"2023-10-06T10:44:51.737226+0000","last_peered":"2023-10-06T10:44:51.737226+0000","last_clean":"2023-10-06T10:44:51.737226+0000","last_became_active":"2023-10-06T10:44:50.736752+0000","last_became_peered":"2023-10-06T10:44:50.736752+0000","last_unstale":"2023-10-06T10:44:51.737226+0000","last_undegraded":"2023-10-06T10:44:51.737226+0000","last_fullsized":"2023-10-06T10:44:51.737226+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:21.691363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757668+0000","last_change":"2023-10-06T10:44:51.750254+0000","last_active":"2023-10-06T10:44:51.757668+0000","last_peered":"2023-10-06T10:44:51.757668+0000","last_clean":"2023-10-06T10:44:51.757668+0000","last_became_active":"2023-10-06T10:44:51.750002+0000","last_became_peered":"2023-10-06T10:44:51.750002+0000","last_unstale":"2023-10-06T10:44:51.757668+0000","last_undegraded":"2023-10-06T10:44:51.757668+0000","last_fullsized":"2023-10-06T10:44:51.757668+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:31:12.288467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.735995+0000","last_change":"2023-10-06T10:44:50.740969+0000","last_active":"2023-10-06T10:44:51.735995+0000","last_peered":"2023-10-06T10:44:51.735995+0000","last_clean":"2023-10-06T10:44:51.735995+0000","last_became_active":"2023-10-06T10:44:50.740714+0000","last_became_peered":"2023-10-06T10:44:50.740714+0000","last_unstale":"2023-10-06T10:44:51.735995+0000","last_undegraded":"2023-10-06T10:44:51.735995+0000","last_fullsized":"2023-10-06T10:44:51.735995+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:56:16.765456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.733952+0000","last_change":"2023-10-06T10:44:51.730894+0000","last_active":"2023-10-06T10:44:51.733952+0000","last_peered":"2023-10-06T10:44:51.733952+0000","last_clean":"2023-10-06T10:44:51.733952+0000","last_became_active":"2023-10-06T10:44:51.730349+0000","last_became_peered":"2023-10-06T10:44:51.730349+0000","last_unstale":"2023-10-06T10:44:51.733952+0000","last_undegraded":"2023-10-06T10:44:51.733952+0000","last_fullsized":"2023-10-06T10:44:51.733952+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:36:50.152016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"101'6","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.762876+0000","last_change":"2023-10-06T10:44:50.738740+0000","last_active":"2023-10-06T10:44:51.762876+0000","last_peered":"2023-10-06T10:44:51.762876+0000","last_clean":"2023-10-06T10:44:51.762876+0000","last_became_active":"2023-10-06T10:44:50.738498+0000","last_became_peered":"2023-10-06T10:44:50.738498+0000","last_unstale":"2023-10-06T10:44:51.762876+0000","last_undegraded":"2023-10-06T10:44:51.762876+0000","last_fullsized":"2023-10-06T10:44:51.762876+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:42:51.471629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":90,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.732796+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:44:51.732796+0000","last_peered":"2023-10-06T10:44:51.732796+0000","last_clean":"2023-10-06T10:44:51.732796+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:44:51.732796+0000","last_undegraded":"2023-10-06T10:44:51.732796+0000","last_fullsized":"2023-10-06T10:44:51.732796+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.748949+0000","last_change":"2023-10-06T10:44:51.744508+0000","last_active":"2023-10-06T10:44:51.748949+0000","last_peered":"2023-10-06T10:44:51.748949+0000","last_clean":"2023-10-06T10:44:51.748949+0000","last_became_active":"2023-10-06T10:44:51.744214+0000","last_became_peered":"2023-10-06T10:44:51.744214+0000","last_unstale":"2023-10-06T10:44:51.748949+0000","last_undegraded":"2023-10-06T10:44:51.748949+0000","last_fullsized":"2023-10-06T10:44:51.748949+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:58.043687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"101'3","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.752327+0000","last_change":"2023-10-06T10:44:50.740082+0000","last_active":"2023-10-06T10:44:51.752327+0000","last_peered":"2023-10-06T10:44:51.752327+0000","last_clean":"2023-10-06T10:44:51.752327+0000","last_became_active":"2023-10-06T10:44:50.739924+0000","last_became_peered":"2023-10-06T10:44:50.739924+0000","last_unstale":"2023-10-06T10:44:51.752327+0000","last_undegraded":"2023-10-06T10:44:51.752327+0000","last_fullsized":"2023-10-06T10:44:51.752327+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:27:09.636709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.727020+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:44:51.727020+0000","last_peered":"2023-10-06T10:44:51.727020+0000","last_clean":"2023-10-06T10:44:51.727020+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:44:51.727020+0000","last_undegraded":"2023-10-06T10:44:51.727020+0000","last_fullsized":"2023-10-06T10:44:51.727020+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745719+0000","last_change":"2023-10-06T10:44:51.732830+0000","last_active":"2023-10-06T10:44:51.745719+0000","last_peered":"2023-10-06T10:44:51.745719+0000","last_clean":"2023-10-06T10:44:51.745719+0000","last_became_active":"2023-10-06T10:44:51.732580+0000","last_became_peered":"2023-10-06T10:44:51.732580+0000","last_unstale":"2023-10-06T10:44:51.745719+0000","last_undegraded":"2023-10-06T10:44:51.745719+0000","last_fullsized":"2023-10-06T10:44:51.745719+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:12.175391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.731609+0000","last_change":"2023-10-06T10:44:50.741287+0000","last_active":"2023-10-06T10:44:51.731609+0000","last_peered":"2023-10-06T10:44:51.731609+0000","last_clean":"2023-10-06T10:44:51.731609+0000","last_became_active":"2023-10-06T10:44:50.740987+0000","last_became_peered":"2023-10-06T10:44:50.740987+0000","last_unstale":"2023-10-06T10:44:51.731609+0000","last_undegraded":"2023-10-06T10:44:51.731609+0000","last_fullsized":"2023-10-06T10:44:51.731609+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:27:18.042909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":84,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.729061+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:44:51.729061+0000","last_peered":"2023-10-06T10:44:51.729061+0000","last_clean":"2023-10-06T10:44:51.729061+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:44:51.729061+0000","last_undegraded":"2023-10-06T10:44:51.729061+0000","last_fullsized":"2023-10-06T10:44:51.729061+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":226,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.724876+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:44:51.724876+0000","last_peered":"2023-10-06T10:44:51.724876+0000","last_clean":"2023-10-06T10:44:51.724876+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:44:51.724876+0000","last_undegraded":"2023-10-06T10:44:51.724876+0000","last_fullsized":"2023-10-06T10:44:51.724876+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.746427+0000","last_change":"2023-10-06T10:44:51.734173+0000","last_active":"2023-10-06T10:44:51.746427+0000","last_peered":"2023-10-06T10:44:51.746427+0000","last_clean":"2023-10-06T10:44:51.746427+0000","last_became_active":"2023-10-06T10:44:51.733932+0000","last_became_peered":"2023-10-06T10:44:51.733932+0000","last_unstale":"2023-10-06T10:44:51.746427+0000","last_undegraded":"2023-10-06T10:44:51.746427+0000","last_fullsized":"2023-10-06T10:44:51.746427+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:42.073984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"101'2","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.743590+0000","last_change":"2023-10-06T10:44:50.737521+0000","last_active":"2023-10-06T10:44:51.743590+0000","last_peered":"2023-10-06T10:44:51.743590+0000","last_clean":"2023-10-06T10:44:51.743590+0000","last_became_active":"2023-10-06T10:44:50.737276+0000","last_became_peered":"2023-10-06T10:44:50.737276+0000","last_unstale":"2023-10-06T10:44:51.743590+0000","last_undegraded":"2023-10-06T10:44:51.743590+0000","last_fullsized":"2023-10-06T10:44:51.743590+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:14:37.094197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":76,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.726680+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:44:51.726680+0000","last_peered":"2023-10-06T10:44:51.726680+0000","last_clean":"2023-10-06T10:44:51.726680+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:44:51.726680+0000","last_undegraded":"2023-10-06T10:44:51.726680+0000","last_fullsized":"2023-10-06T10:44:51.726680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745715+0000","last_change":"2023-10-06T10:44:51.730648+0000","last_active":"2023-10-06T10:44:51.745715+0000","last_peered":"2023-10-06T10:44:51.745715+0000","last_clean":"2023-10-06T10:44:51.745715+0000","last_became_active":"2023-10-06T10:44:51.729905+0000","last_became_peered":"2023-10-06T10:44:51.729905+0000","last_unstale":"2023-10-06T10:44:51.745715+0000","last_undegraded":"2023-10-06T10:44:51.745715+0000","last_fullsized":"2023-10-06T10:44:51.745715+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:07:26.376540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.725894+0000","last_change":"2023-10-06T10:44:50.742945+0000","last_active":"2023-10-06T10:44:51.725894+0000","last_peered":"2023-10-06T10:44:51.725894+0000","last_clean":"2023-10-06T10:44:51.725894+0000","last_became_active":"2023-10-06T10:44:50.742669+0000","last_became_peered":"2023-10-06T10:44:50.742669+0000","last_unstale":"2023-10-06T10:44:51.725894+0000","last_undegraded":"2023-10-06T10:44:51.725894+0000","last_fullsized":"2023-10-06T10:44:51.725894+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:28.430497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.724961+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:44:51.724961+0000","last_peered":"2023-10-06T10:44:51.724961+0000","last_clean":"2023-10-06T10:44:51.724961+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:44:51.724961+0000","last_undegraded":"2023-10-06T10:44:51.724961+0000","last_fullsized":"2023-10-06T10:44:51.724961+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.746547+0000","last_change":"2023-10-06T10:44:51.741319+0000","last_active":"2023-10-06T10:44:51.746547+0000","last_peered":"2023-10-06T10:44:51.746547+0000","last_clean":"2023-10-06T10:44:51.746547+0000","last_became_active":"2023-10-06T10:44:51.740994+0000","last_became_peered":"2023-10-06T10:44:51.740994+0000","last_unstale":"2023-10-06T10:44:51.746547+0000","last_undegraded":"2023-10-06T10:44:51.746547+0000","last_fullsized":"2023-10-06T10:44:51.746547+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:56:17.771151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"101'8","reported_seq":18,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.760275+0000","last_change":"2023-10-06T10:44:50.738355+0000","last_active":"2023-10-06T10:44:51.760275+0000","last_peered":"2023-10-06T10:44:51.760275+0000","last_clean":"2023-10-06T10:44:51.760275+0000","last_became_active":"2023-10-06T10:44:50.738093+0000","last_became_peered":"2023-10-06T10:44:50.738093+0000","last_unstale":"2023-10-06T10:44:51.760275+0000","last_undegraded":"2023-10-06T10:44:51.760275+0000","last_fullsized":"2023-10-06T10:44:51.760275+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:46.580158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.725988+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:44:51.725988+0000","last_peered":"2023-10-06T10:44:51.725988+0000","last_clean":"2023-10-06T10:44:51.725988+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:44:51.725988+0000","last_undegraded":"2023-10-06T10:44:51.725988+0000","last_fullsized":"2023-10-06T10:44:51.725988+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.753934+0000","last_change":"2023-10-06T10:44:51.752435+0000","last_active":"2023-10-06T10:44:51.753934+0000","last_peered":"2023-10-06T10:44:51.753934+0000","last_clean":"2023-10-06T10:44:51.753934+0000","last_became_active":"2023-10-06T10:44:51.752213+0000","last_became_peered":"2023-10-06T10:44:51.752213+0000","last_unstale":"2023-10-06T10:44:51.753934+0000","last_undegraded":"2023-10-06T10:44:51.753934+0000","last_fullsized":"2023-10-06T10:44:51.753934+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:58.636572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.733702+0000","last_change":"2023-10-06T10:44:50.736915+0000","last_active":"2023-10-06T10:44:51.733702+0000","last_peered":"2023-10-06T10:44:51.733702+0000","last_clean":"2023-10-06T10:44:51.733702+0000","last_became_active":"2023-10-06T10:44:50.736660+0000","last_became_peered":"2023-10-06T10:44:50.736660+0000","last_unstale":"2023-10-06T10:44:51.733702+0000","last_undegraded":"2023-10-06T10:44:51.733702+0000","last_fullsized":"2023-10-06T10:44:51.733702+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:50:16.152816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.733032+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:44:51.733032+0000","last_peered":"2023-10-06T10:44:51.733032+0000","last_clean":"2023-10-06T10:44:51.733032+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:44:51.733032+0000","last_undegraded":"2023-10-06T10:44:51.733032+0000","last_fullsized":"2023-10-06T10:44:51.733032+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.753284+0000","last_change":"2023-10-06T10:44:51.746238+0000","last_active":"2023-10-06T10:44:51.753284+0000","last_peered":"2023-10-06T10:44:51.753284+0000","last_clean":"2023-10-06T10:44:51.753284+0000","last_became_active":"2023-10-06T10:44:51.745970+0000","last_became_peered":"2023-10-06T10:44:51.745970+0000","last_unstale":"2023-10-06T10:44:51.753284+0000","last_undegraded":"2023-10-06T10:44:51.753284+0000","last_fullsized":"2023-10-06T10:44:51.753284+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:48:22.355874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.726787+0000","last_change":"2023-10-06T10:44:50.737172+0000","last_active":"2023-10-06T10:44:51.726787+0000","last_peered":"2023-10-06T10:44:51.726787+0000","last_clean":"2023-10-06T10:44:51.726787+0000","last_became_active":"2023-10-06T10:44:50.736902+0000","last_became_peered":"2023-10-06T10:44:50.736902+0000","last_unstale":"2023-10-06T10:44:51.726787+0000","last_undegraded":"2023-10-06T10:44:51.726787+0000","last_fullsized":"2023-10-06T10:44:51.726787+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:45:13.294923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.733036+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:44:51.733036+0000","last_peered":"2023-10-06T10:44:51.733036+0000","last_clean":"2023-10-06T10:44:51.733036+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:44:51.733036+0000","last_undegraded":"2023-10-06T10:44:51.733036+0000","last_fullsized":"2023-10-06T10:44:51.733036+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299112,"kb_used_data":1824,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072728,"statfs":{"total":96636764160,"available":96330473472,"internally_reserved":0,"allocated":1867776,"data_stored":1203651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.51000000000000001,"15min":0.48799999999999999},"min":{"1min":0.32800000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63900000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.53800000000000003,"15min":0.52100000000000002},"min":{"1min":0.29399999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.9700000000000002,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":0.66100000000000003}]},{"osd":1,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.496,"15min":0.46700000000000003},"min":{"1min":0.32000000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.64800000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.54500000000000004,"15min":0.51300000000000001},"min":{"1min":0.38400000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":3.0059999999999998,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56699999999999995,"15min":0.52200000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.628},{"interface":"front","average":{"1min":0.497,"5min":0.51900000000000002,"15min":0.499},"min":{"1min":0.371,"5min":0.314,"15min":0.314},"max":{"1min":0.66800000000000004,"5min":0.747,"15min":0.79800000000000004},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55200000000000005,"15min":0.51600000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73099999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.40600000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.71899999999999997,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.67600000000000005}]},{"osd":4,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42199999999999999,"15min":0.46200000000000002},"min":{"1min":0.314,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61899999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47299999999999998,"15min":0.499},"min":{"1min":0.30199999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64900000000000002,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.71999999999999997}]},{"osd":5,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.48199999999999998},"min":{"1min":0.33000000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.81100000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.51800000000000002,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85399999999999998,"5min":1.577,"15min":1.577},"last":0.69899999999999995}]},{"osd":6,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.56499999999999995},"min":{"1min":0.39700000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.67200000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.745},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.56200000000000006,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69599999999999995,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.75700000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298512,"kb_used_data":1224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073328,"statfs":{"total":96636764160,"available":96331087872,"internally_reserved":0,"allocated":1253376,"data_stored":604451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.495},"min":{"1min":0.34599999999999997,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":0.70399999999999996,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52100000000000002,"15min":0.45500000000000002},"min":{"1min":0.39700000000000002,"5min":0.32500000000000001,"15min":0.32000000000000001},"max":{"1min":0.67400000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.59899999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52000000000000002,"15min":0.46000000000000002},"min":{"1min":0.39300000000000002,"5min":0.32200000000000001,"15min":0.28000000000000003},"max":{"1min":0.70799999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54500000000000004,"15min":0.48899999999999999},"min":{"1min":0.42399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73499999999999999,"5min":0.751,"15min":0.751},"last":0.621}]},{"osd":2,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52700000000000002,"15min":0.48399999999999999},"min":{"1min":0.39600000000000002,"5min":0.32500000000000001,"15min":0.309},"max":{"1min":0.65400000000000003,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54500000000000004,"15min":0.48699999999999999},"min":{"1min":0.443,"5min":0.34999999999999998,"15min":0.34300000000000003},"max":{"1min":0.66100000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.69699999999999995}]},{"osd":3,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55200000000000005,"15min":0.50600000000000001},"min":{"1min":0.46500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75600000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.55500000000000005,"15min":0.51900000000000002},"min":{"1min":0.40699999999999997,"5min":0.36399999999999999,"15min":0.34499999999999997},"max":{"1min":0.74199999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.79200000000000004}]},{"osd":4,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.48699999999999999,"15min":0.48099999999999998},"min":{"1min":0.34999999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.78300000000000003,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.498,"15min":0.48899999999999999},"min":{"1min":0.377,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79400000000000004,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.71499999999999997}]},{"osd":5,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.51000000000000001,"15min":0.49199999999999999},"min":{"1min":0.34200000000000003,"5min":0.28899999999999998,"15min":0.28299999999999997},"max":{"1min":0.83599999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.38700000000000001,"5min":0.33900000000000002,"15min":0.308},"max":{"1min":0.77800000000000002,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.93100000000000005}]},{"osd":7,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53200000000000003,"15min":0.52000000000000002},"min":{"1min":0.36599999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71899999999999997,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.755},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57399999999999995,"15min":0.54800000000000004},"min":{"1min":0.40000000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.871,"5min":0.879,"15min":0.879},"last":0.76800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303208,"kb_used_data":1824,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068632,"statfs":{"total":96636764160,"available":96326279168,"internally_reserved":0,"allocated":1867776,"data_stored":1203187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.20399999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.35599999999999998,"15min":0.34799999999999998},"min":{"1min":0.23400000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.57099999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.27800000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.33000000000000002,"15min":0.32000000000000001},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.46999999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.373},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38100000000000001,"15min":0.36699999999999999},"min":{"1min":0.27700000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.41299999999999998}]},{"osd":3,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39500000000000002,"15min":0.38},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.60999999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.38700000000000001,"15min":0.377},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.32600000000000001}]},{"osd":4,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.40200000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73099999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.48299999999999998}]},{"osd":5,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86599999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.503},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.55000000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.35699999999999998},"max":{"1min":0.80900000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.49299999999999999}]},{"osd":6,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.59899999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.36699999999999999},"max":{"1min":0.92200000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.435,"5min":0.40000000000000002,"15min":0.39800000000000002},"max":{"1min":0.77000000000000002,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.51900000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.39100000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.84299999999999997,"5min":0.996,"15min":0.996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.42099999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.56999999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298544,"kb_used_data":1256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073296,"statfs":{"total":96636764160,"available":96331055104,"internally_reserved":0,"allocated":1286144,"data_stored":614343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.33000000000000002,"15min":0.32000000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.433,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.376,"15min":0.34999999999999998},"min":{"1min":0.28399999999999997,"5min":0.25,"15min":0.25},"max":{"1min":0.504,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.32100000000000001,"15min":0.309},"min":{"1min":0.246,"5min":0.191,"15min":0.191},"max":{"1min":0.41799999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.248},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35599999999999998,"15min":0.33200000000000002},"min":{"1min":0.26900000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.46000000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.40500000000000003}]},{"osd":3,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35399999999999998,"15min":0.34399999999999997},"min":{"1min":0.25600000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.49299999999999999,"5min":0.622,"15min":0.622},"last":0.31},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36599999999999999,"15min":0.35299999999999998},"min":{"1min":0.28999999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.47899999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.32100000000000001}]},{"osd":4,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.41799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78900000000000003,"5min":0.81200000000000006,"15min":0.83299999999999996},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56599999999999995,"15min":0.54100000000000004},"min":{"1min":0.38600000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.753,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.60399999999999998}]},{"osd":5,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.36899999999999999},"max":{"1min":0.80300000000000005,"5min":0.91900000000000004,"15min":1.0069999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.41299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76700000000000002,"5min":0.80900000000000005,"15min":0.94499999999999995},"last":0.57699999999999996}]},{"osd":6,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56999999999999995,"15min":0.58199999999999996},"min":{"1min":0.40300000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.747,"5min":1.046,"15min":1.046},"last":0.629},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.42999999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76000000000000001,"5min":0.86399999999999999,"15min":1.075},"last":0.53400000000000003}]},{"osd":7,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80700000000000005,"5min":0.83499999999999996,"15min":0.96099999999999997},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.44400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76900000000000002,"5min":0.88800000000000001,"15min":0.90100000000000002},"last":0.54600000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574932,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298512,"kb_used_data":1224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073328,"statfs":{"total":96636764160,"available":96331087872,"internally_reserved":0,"allocated":1253376,"data_stored":605734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.317,"15min":0.312},"min":{"1min":0.22500000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.45300000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.22600000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.36199999999999999,"15min":0.32900000000000001},"min":{"1min":0.27600000000000002,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.502,"5min":0.622,"15min":0.622},"last":0.32600000000000001}]},{"osd":1,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34499999999999997,"15min":0.32900000000000001},"min":{"1min":0.23499999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.46600000000000003,"5min":0.56599999999999995,"15min":0.60499999999999998},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.378,"15min":0.36799999999999999},"min":{"1min":0.26700000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.55600000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.37}]},{"osd":3,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.38700000000000001,"15min":0.35699999999999998},"min":{"1min":0.28599999999999998,"5min":0.252,"15min":0.20499999999999999},"max":{"1min":0.55000000000000004,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38,"15min":0.36199999999999999},"min":{"1min":0.28499999999999998,"5min":0.25,"15min":0.22800000000000001},"max":{"1min":0.58099999999999996,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.35599999999999998}]},{"osd":4,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.35299999999999998,"5min":0.33400000000000002,"15min":0.315},"max":{"1min":0.78000000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.441,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81100000000000005,"5min":0.82699999999999996,"15min":0.84899999999999998},"last":0.57799999999999996}]},{"osd":5,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73899999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.42799999999999999,"5min":0.36599999999999999,"15min":0.33800000000000002},"max":{"1min":0.80100000000000005,"5min":0.81799999999999995,"15min":0.83399999999999996},"last":0.50600000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.61699999999999999},"min":{"1min":0.42199999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.871,"5min":0.89000000000000001,"15min":0.89100000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.621,"15min":0.622},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79300000000000004,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.61599999999999999}]},{"osd":7,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58999999999999997,"15min":0.60799999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.36799999999999999},"max":{"1min":0.84199999999999997,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60999999999999999,"15min":0.622},"min":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.749,"5min":0.874,"15min":0.874},"last":0.58699999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574932,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298540,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073300,"statfs":{"total":96636764160,"available":96331059200,"internally_reserved":0,"allocated":1282048,"data_stored":614301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.314,"15min":0.41499999999999998},"min":{"1min":0.20599999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.433,"5min":0.52700000000000002,"15min":3.3210000000000002},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34399999999999997,"15min":0.35599999999999998},"min":{"1min":0.21099999999999999,"5min":0.192,"15min":0.192},"max":{"1min":0.436,"5min":0.66800000000000004,"15min":1.218},"last":0.35099999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.33700000000000002,"15min":0.33900000000000002},"min":{"1min":0.23699999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.40600000000000003,"5min":0.63600000000000001,"15min":0.86299999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.36499999999999999,"15min":0.45600000000000002},"min":{"1min":0.28399999999999997,"5min":0.218,"15min":0.218},"max":{"1min":0.46000000000000002,"5min":0.66000000000000003,"15min":3.359},"last":0.36499999999999999}]},{"osd":2,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.36399999999999999,"15min":0.377},"min":{"1min":0.27800000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.55500000000000005,"5min":0.71099999999999997,"15min":1.1100000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.38300000000000001,"15min":0.58899999999999997},"min":{"1min":0.28399999999999997,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46100000000000002,"5min":0.59899999999999998,"15min":7.069},"last":0.30599999999999999}]},{"osd":4,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55900000000000005,"15min":0.57999999999999996},"min":{"1min":0.41599999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.66300000000000003,"5min":0.89400000000000002,"15min":1.165},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.58599999999999997,"15min":0.59299999999999997},"min":{"1min":0.379,"5min":0.375,"15min":0.375},"max":{"1min":0.753,"5min":1.02,"15min":1.02},"last":0.53200000000000003}]},{"osd":5,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.58199999999999996,"15min":0.67500000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71499999999999997,"5min":0.92500000000000004,"15min":3.4140000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57699999999999996,"15min":0.60099999999999998},"min":{"1min":0.39200000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.96899999999999997,"15min":1.1930000000000001},"last":0.50800000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.60299999999999998,"15min":0.61299999999999999},"min":{"1min":0.42599999999999999,"5min":0.38,"15min":0.378},"max":{"1min":0.83699999999999997,"5min":1.054,"15min":1.054},"last":0.495},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61899999999999999,"15min":0.70599999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.41999999999999998},"max":{"1min":0.80800000000000005,"5min":0.876,"15min":3.3879999999999999},"last":0.54600000000000004}]},{"osd":7,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85599999999999998,"5min":0.99099999999999999,"15min":1.137},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.436,"5min":0.378,"15min":0.378},"max":{"1min":0.67600000000000005,"5min":0.97899999999999998,"15min":1.014},"last":0.42999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298524,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073316,"statfs":{"total":96636764160,"available":96331075584,"internally_reserved":0,"allocated":1265664,"data_stored":612695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.502,"15min":0.48799999999999999},"min":{"1min":0.316,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73799999999999999,"5min":0.74299999999999999,"15min":0.76800000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52100000000000002,"15min":0.504},"min":{"1min":0.36099999999999999,"5min":0.33100000000000002,"15min":0.28299999999999997},"max":{"1min":0.71399999999999997,"5min":0.73099999999999998,"15min":0.77400000000000002},"last":0.51600000000000001}]},{"osd":1,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71199999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51500000000000001,"15min":0.53300000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70499999999999996,"5min":0.71799999999999997,"15min":0.82899999999999996},"last":0.52900000000000003}]},{"osd":2,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.32800000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.81000000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.372,"5min":0.36099999999999999,"15min":0.315},"max":{"1min":0.69399999999999995,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.501}]},{"osd":3,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55200000000000005,"15min":0.52400000000000002},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.35499999999999998},"max":{"1min":0.73899999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.79500000000000004},"last":0.58999999999999997}]},{"osd":5,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.436,"15min":0.48799999999999999},"min":{"1min":0.30399999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.622,"5min":0.77800000000000002,"15min":0.86899999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46800000000000003,"15min":0.49299999999999999},"min":{"1min":0.29499999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.65200000000000002,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.61799999999999999}]},{"osd":6,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.496,"15min":0.53300000000000003},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53500000000000003,"15min":0.56799999999999995},"min":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69999999999999996,"5min":0.88,"15min":0.91200000000000003},"last":0.60599999999999998}]},{"osd":7,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51700000000000002,"15min":0.56299999999999994},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.67900000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53400000000000003,"15min":0.58299999999999996},"min":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72899999999999998,"5min":0.81299999999999994,"15min":0.85699999999999998},"last":0.63400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298532,"kb_used_data":1244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073308,"statfs":{"total":96636764160,"available":96331067392,"internally_reserved":0,"allocated":1273856,"data_stored":613271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50900000000000001,"15min":0.47499999999999998},"min":{"1min":0.28599999999999998,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.66400000000000003,"5min":0.88,"15min":0.88},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.50800000000000001,"15min":0.52400000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.64700000000000002,"5min":0.86499999999999999,"15min":1.2549999999999999},"last":0.441}]},{"osd":1,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.314,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.59999999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.32300000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64300000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.47599999999999998}]},{"osd":2,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53300000000000003,"15min":0.50700000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.30399999999999999},"max":{"1min":0.61899999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.54300000000000004,"15min":0.55900000000000005},"min":{"1min":0.35399999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67700000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.52800000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.54000000000000004,"15min":0.56599999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57899999999999996,"5min":0.79900000000000004,"15min":0.95099999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.51800000000000002,"15min":0.54800000000000004},"min":{"1min":0.33200000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70499999999999996,"5min":0.79100000000000004,"15min":1.3999999999999999},"last":0.504}]},{"osd":4,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47399999999999998,"15min":0.48499999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.75700000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.499,"15min":0.50600000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.68999999999999995,"5min":0.70399999999999996,"15min":0.80700000000000005},"last":0.60599999999999998}]},{"osd":6,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.495,"15min":0.51500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.67800000000000005,"5min":0.77200000000000002,"15min":0.85799999999999998},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78100000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.55600000000000005}]},{"osd":7,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54700000000000004,"15min":0.57399999999999995},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.83299999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71399999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.58899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:45:00.304 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:45:00.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.428+0000 7f0d83276700 1 Processor -- start 2023-10-06T10:45:00.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.428+0000 7f0d83276700 1 -- start start 2023-10-06T10:45:00.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.429+0000 7f0d83276700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d7c159560 0x7f0d7c0aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:00.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.429+0000 7f0d83276700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0d7c0b0760 0x7f0d7c0b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:00.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.429+0000 7f0d83276700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d7c0b0c60 con 0x7f0d7c0b49a0 2023-10-06T10:45:00.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.429+0000 7f0d83276700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d7c0b0f40 con 0x7f0d7c0b0760 2023-10-06T10:45:00.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.429+0000 7f0d83276700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c0b1220 con 0x7f0d7c159560 2023-10-06T10:45:00.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.429+0000 7f0d81813700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0d7c0b0760 0x7f0d7c0b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:00.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.429+0000 7f0d80811700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0d7c0b49a0 0x7f0d7c0aaa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:43932/0 (socket says 172.21.15.92:43932) 2023-10-06T10:45:00.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.429+0000 7f0d81813700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0d7c0b0760 0x7f0d7c0b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33858/0 (socket says 172.21.15.92:33858) 2023-10-06T10:45:00.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.429+0000 7f0d80811700 1 -- 172.21.15.92:0/4192586573 learned_addr learned my addr 172.21.15.92:0/4192586573 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:00.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.430+0000 7f0d81012700 1 --2- 172.21.15.92:0/4192586573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d7c159560 0x7f0d7c0aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:00.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.430+0000 7f0d81813700 1 -- 172.21.15.92:0/4192586573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d7c159560 msgr2=0x7f0d7c0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:00.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.430+0000 7f0d81813700 1 --2- 172.21.15.92:0/4192586573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d7c159560 0x7f0d7c0aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:00.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.430+0000 7f0d81813700 1 -- 172.21.15.92:0/4192586573 >> v1:172.21.15.92:6789/0 conn(0x7f0d7c0b49a0 legacy=0x7f0d7c0aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:00.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.430+0000 7f0d73fff700 1 -- 172.21.15.92:0/4192586573 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1169933242 0 0) 0x7f0d7c0b0c60 con 0x7f0d7c0b49a0 2023-10-06T10:45:00.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.430+0000 7f0d81813700 1 -- 172.21.15.92:0/4192586573 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c0b1500 con 0x7f0d7c0b0760 2023-10-06T10:45:00.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.430+0000 7f0d81012700 1 --2- 172.21.15.92:0/4192586573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d7c159560 0x7f0d7c0aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:00.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.430+0000 7f0d81813700 1 --2- 172.21.15.92:0/4192586573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d7c0b0760 0x7f0d7c0b1bc0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f0d740098e0 tx=0x7f0d740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527eca683a3b56ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:00.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.430+0000 7f0d73fff700 1 -- 172.21.15.92:0/4192586573 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d74009e80 con 0x7f0d7c0b0760 2023-10-06T10:45:00.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.430+0000 7f0d73fff700 1 -- 172.21.15.92:0/4192586573 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7400b860 con 0x7f0d7c0b0760 2023-10-06T10:45:00.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.430+0000 7f0d73fff700 1 -- 172.21.15.92:0/4192586573 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7400bbe0 con 0x7f0d7c0b0760 2023-10-06T10:45:00.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.431+0000 7f0d83276700 1 -- 172.21.15.92:0/4192586573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d7c0b0760 msgr2=0x7f0d7c0b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:00.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.431+0000 7f0d83276700 1 --2- 172.21.15.92:0/4192586573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d7c0b0760 0x7f0d7c0b1bc0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f0d740098e0 tx=0x7f0d740091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:00.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.431+0000 7f0d83276700 1 -- 172.21.15.92:0/4192586573 shutdown_connections 2023-10-06T10:45:00.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.431+0000 7f0d83276700 1 --2- 172.21.15.92:0/4192586573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d7c159560 0x7f0d7c0aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:00.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.431+0000 7f0d83276700 1 --2- 172.21.15.92:0/4192586573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d7c0b0760 0x7f0d7c0b1bc0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:00.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.431+0000 7f0d83276700 1 -- 172.21.15.92:0/4192586573 >> 172.21.15.92:0/4192586573 conn(0x7f0d7c0997f0 msgr2=0x7f0d7c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:00.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.431+0000 7f0d83276700 1 -- 172.21.15.92:0/4192586573 shutdown_connections 2023-10-06T10:45:00.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.431+0000 7f0d83276700 1 -- 172.21.15.92:0/4192586573 wait complete. 2023-10-06T10:45:00.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.432+0000 7f0d83276700 1 Processor -- start 2023-10-06T10:45:00.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.432+0000 7f0d83276700 1 -- start start 2023-10-06T10:45:00.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.433+0000 7f0d83276700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d7c0b0760 0x7f0d7c15e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:00.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.433+0000 7f0d83276700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d7c0b49a0 0x7f0d7c15e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:00.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.433+0000 7f0d83276700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d7c159560 0x7f0d7c156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:00.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.433+0000 7f0d83276700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c15c690 con 0x7f0d7c0b0760 2023-10-06T10:45:00.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.433+0000 7f0d83276700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c15efc0 con 0x7f0d7c0b49a0 2023-10-06T10:45:00.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.433+0000 7f0d83276700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c15d170 con 0x7f0d7c159560 2023-10-06T10:45:00.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.433+0000 7f0d81012700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d7c0b0760 0x7f0d7c15e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:00.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.433+0000 7f0d81012700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d7c0b0760 0x7f0d7c15e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33860/0 (socket says 172.21.15.92:33860) 2023-10-06T10:45:00.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.433+0000 7f0d81012700 1 -- 172.21.15.92:0/595696113 learned_addr learned my addr 172.21.15.92:0/595696113 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:00.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.433+0000 7f0d81813700 1 --2- 172.21.15.92:0/595696113 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d7c159560 0x7f0d7c156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:00.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.433+0000 7f0d80811700 1 --2- 172.21.15.92:0/595696113 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d7c0b49a0 0x7f0d7c15e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:00.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.433+0000 7f0d81012700 1 -- 172.21.15.92:0/595696113 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d7c159560 msgr2=0x7f0d7c156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:00.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.433+0000 7f0d81012700 1 --2- 172.21.15.92:0/595696113 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d7c159560 0x7f0d7c156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:00.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.433+0000 7f0d81012700 1 -- 172.21.15.92:0/595696113 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d7c0b49a0 msgr2=0x7f0d7c15e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:00.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.433+0000 7f0d81012700 1 --2- 172.21.15.92:0/595696113 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d7c0b49a0 0x7f0d7c15e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:00.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.433+0000 7f0d81012700 1 -- 172.21.15.92:0/595696113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d74009420 con 0x7f0d7c0b0760 2023-10-06T10:45:00.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.433+0000 7f0d81012700 1 --2- 172.21.15.92:0/595696113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d7c0b0760 0x7f0d7c15e240 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800a330 tx=0x7f0d780078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:00.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.433+0000 7f0d717fa700 1 -- 172.21.15.92:0/595696113 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78015440 con 0x7f0d7c0b0760 2023-10-06T10:45:00.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.433+0000 7f0d717fa700 1 -- 172.21.15.92:0/595696113 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d780155a0 con 0x7f0d7c0b0760 2023-10-06T10:45:00.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.433+0000 7f0d83276700 1 -- 172.21.15.92:0/595696113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7c156fd0 con 0x7f0d7c0b0760 2023-10-06T10:45:00.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.433+0000 7f0d717fa700 1 -- 172.21.15.92:0/595696113 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78015940 con 0x7f0d7c0b0760 2023-10-06T10:45:00.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.433+0000 7f0d83276700 1 -- 172.21.15.92:0/595696113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7c15cb20 con 0x7f0d7c0b0760 2023-10-06T10:45:00.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.434+0000 7f0d83276700 1 -- 172.21.15.92:0/595696113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d7c064a90 con 0x7f0d7c0b0760 2023-10-06T10:45:00.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.435+0000 7f0d717fa700 1 -- 172.21.15.92:0/595696113 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0d78015cf0 con 0x7f0d7c0b0760 2023-10-06T10:45:00.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.435+0000 7f0d717fa700 1 --2- 172.21.15.92:0/595696113 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0d60066f30 0x7f0d600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:00.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.435+0000 7f0d717fa700 1 -- 172.21.15.92:0/595696113 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0d78084eb0 con 0x7f0d7c0b0760 2023-10-06T10:45:00.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.435+0000 7f0d80811700 1 --2- 172.21.15.92:0/595696113 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0d60066f30 0x7f0d600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:00.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.435+0000 7f0d80811700 1 --2- 172.21.15.92:0/595696113 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0d60066f30 0x7f0d600693e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c16cec0 tx=0x7f0d6c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:00.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.439+0000 7f0d717fa700 1 -- 172.21.15.92:0/595696113 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d7804e5a0 con 0x7f0d7c0b0760 2023-10-06T10:45:00.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.618+0000 7f0d83276700 1 -- 172.21.15.92:0/595696113 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d7c0b17e0 con 0x7f0d60066f30 2023-10-06T10:45:00.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.626+0000 7f0d717fa700 1 -- 172.21.15.92:0/595696113 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312262 (secure 0 0 0) 0x7f0d7c0b17e0 con 0x7f0d60066f30 2023-10-06T10:45:00.630 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:00.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.629+0000 7f0d83276700 1 -- 172.21.15.92:0/595696113 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0d60066f30 msgr2=0x7f0d600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:00.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.629+0000 7f0d83276700 1 --2- 172.21.15.92:0/595696113 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0d60066f30 0x7f0d600693e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c16cec0 tx=0x7f0d6c009040 comp rx=0 tx=0).stop 2023-10-06T10:45:00.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.629+0000 7f0d83276700 1 -- 172.21.15.92:0/595696113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d7c0b0760 msgr2=0x7f0d7c15e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:00.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.629+0000 7f0d83276700 1 --2- 172.21.15.92:0/595696113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d7c0b0760 0x7f0d7c15e240 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800a330 tx=0x7f0d780078c0 comp rx=0 tx=0).stop 2023-10-06T10:45:00.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.630+0000 7f0d83276700 1 -- 172.21.15.92:0/595696113 shutdown_connections 2023-10-06T10:45:00.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.630+0000 7f0d83276700 1 --2- 172.21.15.92:0/595696113 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0d60066f30 0x7f0d600693e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:00.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.630+0000 7f0d83276700 1 --2- 172.21.15.92:0/595696113 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d7c159560 0x7f0d7c156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:00.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.630+0000 7f0d83276700 1 --2- 172.21.15.92:0/595696113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d7c0b0760 0x7f0d7c15e240 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:00.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.630+0000 7f0d83276700 1 --2- 172.21.15.92:0/595696113 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d7c0b49a0 0x7f0d7c15e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:00.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.630+0000 7f0d83276700 1 -- 172.21.15.92:0/595696113 >> 172.21.15.92:0/595696113 conn(0x7f0d7c0997f0 msgr2=0x7f0d7c1588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:00.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.630+0000 7f0d83276700 1 -- 172.21.15.92:0/595696113 shutdown_connections 2023-10-06T10:45:00.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.630+0000 7f0d83276700 1 -- 172.21.15.92:0/595696113 wait complete. 2023-10-06T10:45:00.634 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:45:00.642 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2023-10-06T10:45:00.169287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2393508,"kb_used_data":11108,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752581212,"statfs":{"total":773094113280,"available":770643161088,"internally_reserved":0,"allocated":11374592,"data_stored":6085555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.444590"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.755450+0000","last_change":"2023-10-06T10:44:51.742214+0000","last_active":"2023-10-06T10:44:51.755450+0000","last_peered":"2023-10-06T10:44:51.755450+0000","last_clean":"2023-10-06T10:44:51.755450+0000","last_became_active":"2023-10-06T10:44:51.741939+0000","last_became_peered":"2023-10-06T10:44:51.741939+0000","last_unstale":"2023-10-06T10:44:51.755450+0000","last_undegraded":"2023-10-06T10:44:51.755450+0000","last_fullsized":"2023-10-06T10:44:51.755450+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:13:48.153672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.746271+0000","last_change":"2023-10-06T10:44:51.743338+0000","last_active":"2023-10-06T10:44:51.746271+0000","last_peered":"2023-10-06T10:44:51.746271+0000","last_clean":"2023-10-06T10:44:51.746271+0000","last_became_active":"2023-10-06T10:44:51.743074+0000","last_became_peered":"2023-10-06T10:44:51.743074+0000","last_unstale":"2023-10-06T10:44:51.746271+0000","last_undegraded":"2023-10-06T10:44:51.746271+0000","last_fullsized":"2023-10-06T10:44:51.746271+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:59.356737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745137+0000","last_change":"2023-10-06T10:44:51.740363+0000","last_active":"2023-10-06T10:44:51.745137+0000","last_peered":"2023-10-06T10:44:51.745137+0000","last_clean":"2023-10-06T10:44:51.745137+0000","last_became_active":"2023-10-06T10:44:51.739877+0000","last_became_peered":"2023-10-06T10:44:51.739877+0000","last_unstale":"2023-10-06T10:44:51.745137+0000","last_undegraded":"2023-10-06T10:44:51.745137+0000","last_fullsized":"2023-10-06T10:44:51.745137+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:11.372620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757596+0000","last_change":"2023-10-06T10:44:51.749770+0000","last_active":"2023-10-06T10:44:51.757596+0000","last_peered":"2023-10-06T10:44:51.757596+0000","last_clean":"2023-10-06T10:44:51.757596+0000","last_became_active":"2023-10-06T10:44:51.749314+0000","last_became_peered":"2023-10-06T10:44:51.749314+0000","last_unstale":"2023-10-06T10:44:51.757596+0000","last_undegraded":"2023-10-06T10:44:51.757596+0000","last_fullsized":"2023-10-06T10:44:51.757596+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:05:36.289745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757281+0000","last_change":"2023-10-06T10:44:51.752557+0000","last_active":"2023-10-06T10:44:51.757281+0000","last_peered":"2023-10-06T10:44:51.757281+0000","last_clean":"2023-10-06T10:44:51.757281+0000","last_became_active":"2023-10-06T10:44:51.752087+0000","last_became_peered":"2023-10-06T10:44:51.752087+0000","last_unstale":"2023-10-06T10:44:51.757281+0000","last_undegraded":"2023-10-06T10:44:51.757281+0000","last_fullsized":"2023-10-06T10:44:51.757281+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:30.552215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.751722+0000","last_change":"2023-10-06T10:44:51.749221+0000","last_active":"2023-10-06T10:44:51.751722+0000","last_peered":"2023-10-06T10:44:51.751722+0000","last_clean":"2023-10-06T10:44:51.751722+0000","last_became_active":"2023-10-06T10:44:51.749025+0000","last_became_peered":"2023-10-06T10:44:51.749025+0000","last_unstale":"2023-10-06T10:44:51.751722+0000","last_undegraded":"2023-10-06T10:44:51.751722+0000","last_fullsized":"2023-10-06T10:44:51.751722+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:28.610646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745548+0000","last_change":"2023-10-06T10:44:51.739336+0000","last_active":"2023-10-06T10:44:51.745548+0000","last_peered":"2023-10-06T10:44:51.745548+0000","last_clean":"2023-10-06T10:44:51.745548+0000","last_became_active":"2023-10-06T10:44:51.739055+0000","last_became_peered":"2023-10-06T10:44:51.739055+0000","last_unstale":"2023-10-06T10:44:51.745548+0000","last_undegraded":"2023-10-06T10:44:51.745548+0000","last_fullsized":"2023-10-06T10:44:51.745548+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:30.342288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.747260+0000","last_change":"2023-10-06T10:44:51.743966+0000","last_active":"2023-10-06T10:44:51.747260+0000","last_peered":"2023-10-06T10:44:51.747260+0000","last_clean":"2023-10-06T10:44:51.747260+0000","last_became_active":"2023-10-06T10:44:51.743686+0000","last_became_peered":"2023-10-06T10:44:51.743686+0000","last_unstale":"2023-10-06T10:44:51.747260+0000","last_undegraded":"2023-10-06T10:44:51.747260+0000","last_fullsized":"2023-10-06T10:44:51.747260+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:12:10.059460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.758682+0000","last_change":"2023-10-06T10:44:51.752074+0000","last_active":"2023-10-06T10:44:51.758682+0000","last_peered":"2023-10-06T10:44:51.758682+0000","last_clean":"2023-10-06T10:44:51.758682+0000","last_became_active":"2023-10-06T10:44:51.751397+0000","last_became_peered":"2023-10-06T10:44:51.751397+0000","last_unstale":"2023-10-06T10:44:51.758682+0000","last_undegraded":"2023-10-06T10:44:51.758682+0000","last_fullsized":"2023-10-06T10:44:51.758682+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:09.234346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.758578+0000","last_change":"2023-10-06T10:44:51.754015+0000","last_active":"2023-10-06T10:44:51.758578+0000","last_peered":"2023-10-06T10:44:51.758578+0000","last_clean":"2023-10-06T10:44:51.758578+0000","last_became_active":"2023-10-06T10:44:51.753770+0000","last_became_peered":"2023-10-06T10:44:51.753770+0000","last_unstale":"2023-10-06T10:44:51.758578+0000","last_undegraded":"2023-10-06T10:44:51.758578+0000","last_fullsized":"2023-10-06T10:44:51.758578+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:33:24.214039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745867+0000","last_change":"2023-10-06T10:44:51.741537+0000","last_active":"2023-10-06T10:44:51.745867+0000","last_peered":"2023-10-06T10:44:51.745867+0000","last_clean":"2023-10-06T10:44:51.745867+0000","last_became_active":"2023-10-06T10:44:51.740094+0000","last_became_peered":"2023-10-06T10:44:51.740094+0000","last_unstale":"2023-10-06T10:44:51.745867+0000","last_undegraded":"2023-10-06T10:44:51.745867+0000","last_fullsized":"2023-10-06T10:44:51.745867+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:00.855299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.760219+0000","last_change":"2023-10-06T10:44:51.756108+0000","last_active":"2023-10-06T10:44:51.760219+0000","last_peered":"2023-10-06T10:44:51.760219+0000","last_clean":"2023-10-06T10:44:51.760219+0000","last_became_active":"2023-10-06T10:44:51.755657+0000","last_became_peered":"2023-10-06T10:44:51.755657+0000","last_unstale":"2023-10-06T10:44:51.760219+0000","last_undegraded":"2023-10-06T10:44:51.760219+0000","last_fullsized":"2023-10-06T10:44:51.760219+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:45:43.330258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.754161+0000","last_change":"2023-10-06T10:44:51.746358+0000","last_active":"2023-10-06T10:44:51.754161+0000","last_peered":"2023-10-06T10:44:51.754161+0000","last_clean":"2023-10-06T10:44:51.754161+0000","last_became_active":"2023-10-06T10:44:51.746074+0000","last_became_peered":"2023-10-06T10:44:51.746074+0000","last_unstale":"2023-10-06T10:44:51.754161+0000","last_undegraded":"2023-10-06T10:44:51.754161+0000","last_fullsized":"2023-10-06T10:44:51.754161+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:44:07.998505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.744777+0000","last_change":"2023-10-06T10:44:51.740471+0000","last_active":"2023-10-06T10:44:51.744777+0000","last_peered":"2023-10-06T10:44:51.744777+0000","last_clean":"2023-10-06T10:44:51.744777+0000","last_became_active":"2023-10-06T10:44:51.740180+0000","last_became_peered":"2023-10-06T10:44:51.740180+0000","last_unstale":"2023-10-06T10:44:51.744777+0000","last_undegraded":"2023-10-06T10:44:51.744777+0000","last_fullsized":"2023-10-06T10:44:51.744777+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:45.409013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.748530+0000","last_change":"2023-10-06T10:44:51.745196+0000","last_active":"2023-10-06T10:44:51.748530+0000","last_peered":"2023-10-06T10:44:51.748530+0000","last_clean":"2023-10-06T10:44:51.748530+0000","last_became_active":"2023-10-06T10:44:51.744948+0000","last_became_peered":"2023-10-06T10:44:51.744948+0000","last_unstale":"2023-10-06T10:44:51.748530+0000","last_undegraded":"2023-10-06T10:44:51.748530+0000","last_fullsized":"2023-10-06T10:44:51.748530+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:11.554526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757351+0000","last_change":"2023-10-06T10:44:51.748954+0000","last_active":"2023-10-06T10:44:51.757351+0000","last_peered":"2023-10-06T10:44:51.757351+0000","last_clean":"2023-10-06T10:44:51.757351+0000","last_became_active":"2023-10-06T10:44:51.743005+0000","last_became_peered":"2023-10-06T10:44:51.743005+0000","last_unstale":"2023-10-06T10:44:51.757351+0000","last_undegraded":"2023-10-06T10:44:51.757351+0000","last_fullsized":"2023-10-06T10:44:51.757351+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:07.472732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745914+0000","last_change":"2023-10-06T10:44:51.740909+0000","last_active":"2023-10-06T10:44:51.745914+0000","last_peered":"2023-10-06T10:44:51.745914+0000","last_clean":"2023-10-06T10:44:51.745914+0000","last_became_active":"2023-10-06T10:44:51.740469+0000","last_became_peered":"2023-10-06T10:44:51.740469+0000","last_unstale":"2023-10-06T10:44:51.745914+0000","last_undegraded":"2023-10-06T10:44:51.745914+0000","last_fullsized":"2023-10-06T10:44:51.745914+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:29:51.113126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.744542+0000","last_change":"2023-10-06T10:44:51.739439+0000","last_active":"2023-10-06T10:44:51.744542+0000","last_peered":"2023-10-06T10:44:51.744542+0000","last_clean":"2023-10-06T10:44:51.744542+0000","last_became_active":"2023-10-06T10:44:51.739175+0000","last_became_peered":"2023-10-06T10:44:51.739175+0000","last_unstale":"2023-10-06T10:44:51.744542+0000","last_undegraded":"2023-10-06T10:44:51.744542+0000","last_fullsized":"2023-10-06T10:44:51.744542+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:11.372620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.754171+0000","last_change":"2023-10-06T10:44:51.746412+0000","last_active":"2023-10-06T10:44:51.754171+0000","last_peered":"2023-10-06T10:44:51.754171+0000","last_clean":"2023-10-06T10:44:51.754171+0000","last_became_active":"2023-10-06T10:44:51.746109+0000","last_became_peered":"2023-10-06T10:44:51.746109+0000","last_unstale":"2023-10-06T10:44:51.754171+0000","last_undegraded":"2023-10-06T10:44:51.754171+0000","last_fullsized":"2023-10-06T10:44:51.754171+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:15.138118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.746258+0000","last_change":"2023-10-06T10:44:51.741367+0000","last_active":"2023-10-06T10:44:51.746258+0000","last_peered":"2023-10-06T10:44:51.746258+0000","last_clean":"2023-10-06T10:44:51.746258+0000","last_became_active":"2023-10-06T10:44:51.741058+0000","last_became_peered":"2023-10-06T10:44:51.741058+0000","last_unstale":"2023-10-06T10:44:51.746258+0000","last_undegraded":"2023-10-06T10:44:51.746258+0000","last_fullsized":"2023-10-06T10:44:51.746258+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:21.512208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.753315+0000","last_change":"2023-10-06T10:44:51.747900+0000","last_active":"2023-10-06T10:44:51.753315+0000","last_peered":"2023-10-06T10:44:51.753315+0000","last_clean":"2023-10-06T10:44:51.753315+0000","last_became_active":"2023-10-06T10:44:51.747625+0000","last_became_peered":"2023-10-06T10:44:51.747625+0000","last_unstale":"2023-10-06T10:44:51.753315+0000","last_undegraded":"2023-10-06T10:44:51.753315+0000","last_fullsized":"2023-10-06T10:44:51.753315+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:56:26.690740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.748662+0000","last_change":"2023-10-06T10:44:51.743748+0000","last_active":"2023-10-06T10:44:51.748662+0000","last_peered":"2023-10-06T10:44:51.748662+0000","last_clean":"2023-10-06T10:44:51.748662+0000","last_became_active":"2023-10-06T10:44:51.743511+0000","last_became_peered":"2023-10-06T10:44:51.743511+0000","last_unstale":"2023-10-06T10:44:51.748662+0000","last_undegraded":"2023-10-06T10:44:51.748662+0000","last_fullsized":"2023-10-06T10:44:51.748662+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:58:58.968944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.746207+0000","last_change":"2023-10-06T10:44:51.743049+0000","last_active":"2023-10-06T10:44:51.746207+0000","last_peered":"2023-10-06T10:44:51.746207+0000","last_clean":"2023-10-06T10:44:51.746207+0000","last_became_active":"2023-10-06T10:44:51.740779+0000","last_became_peered":"2023-10-06T10:44:51.740779+0000","last_unstale":"2023-10-06T10:44:51.746207+0000","last_undegraded":"2023-10-06T10:44:51.746207+0000","last_fullsized":"2023-10-06T10:44:51.746207+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:27:22.155683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745690+0000","last_change":"2023-10-06T10:44:51.730584+0000","last_active":"2023-10-06T10:44:51.745690+0000","last_peered":"2023-10-06T10:44:51.745690+0000","last_clean":"2023-10-06T10:44:51.745690+0000","last_became_active":"2023-10-06T10:44:51.729878+0000","last_became_peered":"2023-10-06T10:44:51.729878+0000","last_unstale":"2023-10-06T10:44:51.745690+0000","last_undegraded":"2023-10-06T10:44:51.745690+0000","last_fullsized":"2023-10-06T10:44:51.745690+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:59.356737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745892+0000","last_change":"2023-10-06T10:44:51.740729+0000","last_active":"2023-10-06T10:44:51.745892+0000","last_peered":"2023-10-06T10:44:51.745892+0000","last_clean":"2023-10-06T10:44:51.745892+0000","last_became_active":"2023-10-06T10:44:51.740139+0000","last_became_peered":"2023-10-06T10:44:51.740139+0000","last_unstale":"2023-10-06T10:44:51.745892+0000","last_undegraded":"2023-10-06T10:44:51.745892+0000","last_fullsized":"2023-10-06T10:44:51.745892+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:07:26.376540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757745+0000","last_change":"2023-10-06T10:44:51.751633+0000","last_active":"2023-10-06T10:44:51.757745+0000","last_peered":"2023-10-06T10:44:51.757745+0000","last_clean":"2023-10-06T10:44:51.757745+0000","last_became_active":"2023-10-06T10:44:51.751325+0000","last_became_peered":"2023-10-06T10:44:51.751325+0000","last_unstale":"2023-10-06T10:44:51.757745+0000","last_undegraded":"2023-10-06T10:44:51.757745+0000","last_fullsized":"2023-10-06T10:44:51.757745+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:34.107458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757927+0000","last_change":"2023-10-06T10:44:51.754195+0000","last_active":"2023-10-06T10:44:51.757927+0000","last_peered":"2023-10-06T10:44:51.757927+0000","last_clean":"2023-10-06T10:44:51.757927+0000","last_became_active":"2023-10-06T10:44:51.753939+0000","last_became_peered":"2023-10-06T10:44:51.753939+0000","last_unstale":"2023-10-06T10:44:51.757927+0000","last_undegraded":"2023-10-06T10:44:51.757927+0000","last_fullsized":"2023-10-06T10:44:51.757927+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:14:19.607356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.755601+0000","last_change":"2023-10-06T10:44:51.753173+0000","last_active":"2023-10-06T10:44:51.755601+0000","last_peered":"2023-10-06T10:44:51.755601+0000","last_clean":"2023-10-06T10:44:51.755601+0000","last_became_active":"2023-10-06T10:44:51.752806+0000","last_became_peered":"2023-10-06T10:44:51.752806+0000","last_unstale":"2023-10-06T10:44:51.755601+0000","last_undegraded":"2023-10-06T10:44:51.755601+0000","last_fullsized":"2023-10-06T10:44:51.755601+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:27:22.155683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.746044+0000","last_change":"2023-10-06T10:44:51.741443+0000","last_active":"2023-10-06T10:44:51.746044+0000","last_peered":"2023-10-06T10:44:51.746044+0000","last_clean":"2023-10-06T10:44:51.746044+0000","last_became_active":"2023-10-06T10:44:51.740884+0000","last_became_peered":"2023-10-06T10:44:51.740884+0000","last_unstale":"2023-10-06T10:44:51.746044+0000","last_undegraded":"2023-10-06T10:44:51.746044+0000","last_fullsized":"2023-10-06T10:44:51.746044+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:15.056633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.739324+0000","last_change":"2023-10-06T10:44:51.734925+0000","last_active":"2023-10-06T10:44:51.739324+0000","last_peered":"2023-10-06T10:44:51.739324+0000","last_clean":"2023-10-06T10:44:51.739324+0000","last_became_active":"2023-10-06T10:44:51.734516+0000","last_became_peered":"2023-10-06T10:44:51.734516+0000","last_unstale":"2023-10-06T10:44:51.739324+0000","last_undegraded":"2023-10-06T10:44:51.739324+0000","last_fullsized":"2023-10-06T10:44:51.739324+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:11:42.494648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.755509+0000","last_change":"2023-10-06T10:44:51.743028+0000","last_active":"2023-10-06T10:44:51.755509+0000","last_peered":"2023-10-06T10:44:51.755509+0000","last_clean":"2023-10-06T10:44:51.755509+0000","last_became_active":"2023-10-06T10:44:51.742742+0000","last_became_peered":"2023-10-06T10:44:51.742742+0000","last_unstale":"2023-10-06T10:44:51.755509+0000","last_undegraded":"2023-10-06T10:44:51.755509+0000","last_fullsized":"2023-10-06T10:44:51.755509+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:31:28.825035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.750999+0000","last_change":"2023-10-06T10:44:51.747804+0000","last_active":"2023-10-06T10:44:51.750999+0000","last_peered":"2023-10-06T10:44:51.750999+0000","last_clean":"2023-10-06T10:44:51.750999+0000","last_became_active":"2023-10-06T10:44:51.747553+0000","last_became_peered":"2023-10-06T10:44:51.747553+0000","last_unstale":"2023-10-06T10:44:51.750999+0000","last_undegraded":"2023-10-06T10:44:51.750999+0000","last_fullsized":"2023-10-06T10:44:51.750999+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:00:47.142581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.749647+0000","last_change":"2023-10-06T10:44:51.746945+0000","last_active":"2023-10-06T10:44:51.749647+0000","last_peered":"2023-10-06T10:44:51.749647+0000","last_clean":"2023-10-06T10:44:51.749647+0000","last_became_active":"2023-10-06T10:44:51.746807+0000","last_became_peered":"2023-10-06T10:44:51.746807+0000","last_unstale":"2023-10-06T10:44:51.749647+0000","last_undegraded":"2023-10-06T10:44:51.749647+0000","last_fullsized":"2023-10-06T10:44:51.749647+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:58.636572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"101'4","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.747192+0000","last_change":"2023-10-06T10:44:50.738760+0000","last_active":"2023-10-06T10:44:51.747192+0000","last_peered":"2023-10-06T10:44:51.747192+0000","last_clean":"2023-10-06T10:44:51.747192+0000","last_became_active":"2023-10-06T10:44:50.738480+0000","last_became_peered":"2023-10-06T10:44:50.738480+0000","last_unstale":"2023-10-06T10:44:51.747192+0000","last_undegraded":"2023-10-06T10:44:51.747192+0000","last_fullsized":"2023-10-06T10:44:51.747192+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:29.546521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.744979+0000","last_change":"2023-10-06T10:44:51.741107+0000","last_active":"2023-10-06T10:44:51.744979+0000","last_peered":"2023-10-06T10:44:51.744979+0000","last_clean":"2023-10-06T10:44:51.744979+0000","last_became_active":"2023-10-06T10:44:51.740656+0000","last_became_peered":"2023-10-06T10:44:51.740656+0000","last_unstale":"2023-10-06T10:44:51.744979+0000","last_undegraded":"2023-10-06T10:44:51.744979+0000","last_fullsized":"2023-10-06T10:44:51.744979+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:53.858578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"101'1","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.742828+0000","last_change":"2023-10-06T10:44:50.739987+0000","last_active":"2023-10-06T10:44:51.742828+0000","last_peered":"2023-10-06T10:44:51.742828+0000","last_clean":"2023-10-06T10:44:51.742828+0000","last_became_active":"2023-10-06T10:44:50.739669+0000","last_became_peered":"2023-10-06T10:44:50.739669+0000","last_unstale":"2023-10-06T10:44:51.742828+0000","last_undegraded":"2023-10-06T10:44:51.742828+0000","last_fullsized":"2023-10-06T10:44:51.742828+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:38:47.009118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.749627+0000","last_change":"2023-10-06T10:44:51.746313+0000","last_active":"2023-10-06T10:44:51.749627+0000","last_peered":"2023-10-06T10:44:51.749627+0000","last_clean":"2023-10-06T10:44:51.749627+0000","last_became_active":"2023-10-06T10:44:51.746064+0000","last_became_peered":"2023-10-06T10:44:51.746064+0000","last_unstale":"2023-10-06T10:44:51.749627+0000","last_undegraded":"2023-10-06T10:44:51.749627+0000","last_fullsized":"2023-10-06T10:44:51.749627+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:44:59.326168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"101'2","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.752912+0000","last_change":"2023-10-06T10:44:50.737016+0000","last_active":"2023-10-06T10:44:51.752912+0000","last_peered":"2023-10-06T10:44:51.752912+0000","last_clean":"2023-10-06T10:44:51.752912+0000","last_became_active":"2023-10-06T10:44:50.736567+0000","last_became_peered":"2023-10-06T10:44:50.736567+0000","last_unstale":"2023-10-06T10:44:51.752912+0000","last_undegraded":"2023-10-06T10:44:51.752912+0000","last_fullsized":"2023-10-06T10:44:51.752912+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:08.868364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.754205+0000","last_change":"2023-10-06T10:44:51.746821+0000","last_active":"2023-10-06T10:44:51.754205+0000","last_peered":"2023-10-06T10:44:51.754205+0000","last_clean":"2023-10-06T10:44:51.754205+0000","last_became_active":"2023-10-06T10:44:51.746576+0000","last_became_peered":"2023-10-06T10:44:51.746576+0000","last_unstale":"2023-10-06T10:44:51.754205+0000","last_undegraded":"2023-10-06T10:44:51.754205+0000","last_fullsized":"2023-10-06T10:44:51.754205+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:06:43.616700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"101'2","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.756745+0000","last_change":"2023-10-06T10:44:50.738193+0000","last_active":"2023-10-06T10:44:51.756745+0000","last_peered":"2023-10-06T10:44:51.756745+0000","last_clean":"2023-10-06T10:44:51.756745+0000","last_became_active":"2023-10-06T10:44:50.737952+0000","last_became_peered":"2023-10-06T10:44:50.737952+0000","last_unstale":"2023-10-06T10:44:51.756745+0000","last_undegraded":"2023-10-06T10:44:51.756745+0000","last_fullsized":"2023-10-06T10:44:51.756745+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:31.698962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757718+0000","last_change":"2023-10-06T10:44:51.751538+0000","last_active":"2023-10-06T10:44:51.757718+0000","last_peered":"2023-10-06T10:44:51.757718+0000","last_clean":"2023-10-06T10:44:51.757718+0000","last_became_active":"2023-10-06T10:44:51.751239+0000","last_became_peered":"2023-10-06T10:44:51.751239+0000","last_unstale":"2023-10-06T10:44:51.757718+0000","last_undegraded":"2023-10-06T10:44:51.757718+0000","last_fullsized":"2023-10-06T10:44:51.757718+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:46.027434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"101'1","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.743770+0000","last_change":"2023-10-06T10:44:50.740417+0000","last_active":"2023-10-06T10:44:51.743770+0000","last_peered":"2023-10-06T10:44:51.743770+0000","last_clean":"2023-10-06T10:44:51.743770+0000","last_became_active":"2023-10-06T10:44:50.740148+0000","last_became_peered":"2023-10-06T10:44:50.740148+0000","last_unstale":"2023-10-06T10:44:51.743770+0000","last_undegraded":"2023-10-06T10:44:51.743770+0000","last_fullsized":"2023-10-06T10:44:51.743770+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:02.006450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.750448+0000","last_change":"2023-10-06T10:44:51.744759+0000","last_active":"2023-10-06T10:44:51.750448+0000","last_peered":"2023-10-06T10:44:51.750448+0000","last_clean":"2023-10-06T10:44:51.750448+0000","last_became_active":"2023-10-06T10:44:51.744499+0000","last_became_peered":"2023-10-06T10:44:51.744499+0000","last_unstale":"2023-10-06T10:44:51.750448+0000","last_undegraded":"2023-10-06T10:44:51.750448+0000","last_fullsized":"2023-10-06T10:44:51.750448+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:58:58.968944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"101'1","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.750469+0000","last_change":"2023-10-06T10:44:50.736564+0000","last_active":"2023-10-06T10:44:51.750469+0000","last_peered":"2023-10-06T10:44:51.750469+0000","last_clean":"2023-10-06T10:44:51.750469+0000","last_became_active":"2023-10-06T10:44:50.736297+0000","last_became_peered":"2023-10-06T10:44:50.736297+0000","last_unstale":"2023-10-06T10:44:51.750469+0000","last_undegraded":"2023-10-06T10:44:51.750469+0000","last_fullsized":"2023-10-06T10:44:51.750469+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:04:17.777010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757369+0000","last_change":"2023-10-06T10:44:51.749198+0000","last_active":"2023-10-06T10:44:51.757369+0000","last_peered":"2023-10-06T10:44:51.757369+0000","last_clean":"2023-10-06T10:44:51.757369+0000","last_became_active":"2023-10-06T10:44:51.748962+0000","last_became_peered":"2023-10-06T10:44:51.748962+0000","last_unstale":"2023-10-06T10:44:51.757369+0000","last_undegraded":"2023-10-06T10:44:51.757369+0000","last_fullsized":"2023-10-06T10:44:51.757369+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:39.790377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.732785+0000","last_change":"2023-10-06T10:44:50.732777+0000","last_active":"2023-10-06T10:44:51.732785+0000","last_peered":"2023-10-06T10:44:51.732785+0000","last_clean":"2023-10-06T10:44:51.732785+0000","last_became_active":"2023-10-06T10:44:50.732494+0000","last_became_peered":"2023-10-06T10:44:50.732494+0000","last_unstale":"2023-10-06T10:44:51.732785+0000","last_undegraded":"2023-10-06T10:44:51.732785+0000","last_fullsized":"2023-10-06T10:44:51.732785+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:26:41.299786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745570+0000","last_change":"2023-10-06T10:44:51.739689+0000","last_active":"2023-10-06T10:44:51.745570+0000","last_peered":"2023-10-06T10:44:51.745570+0000","last_clean":"2023-10-06T10:44:51.745570+0000","last_became_active":"2023-10-06T10:44:51.739426+0000","last_became_peered":"2023-10-06T10:44:51.739426+0000","last_unstale":"2023-10-06T10:44:51.745570+0000","last_undegraded":"2023-10-06T10:44:51.745570+0000","last_fullsized":"2023-10-06T10:44:51.745570+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:38:48.014813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.735795+0000","last_change":"2023-10-06T10:44:50.746777+0000","last_active":"2023-10-06T10:44:51.735795+0000","last_peered":"2023-10-06T10:44:51.735795+0000","last_clean":"2023-10-06T10:44:51.735795+0000","last_became_active":"2023-10-06T10:44:50.746550+0000","last_became_peered":"2023-10-06T10:44:50.746550+0000","last_unstale":"2023-10-06T10:44:51.735795+0000","last_undegraded":"2023-10-06T10:44:51.735795+0000","last_fullsized":"2023-10-06T10:44:51.735795+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:41.068289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.758917+0000","last_change":"2023-10-06T10:44:51.756739+0000","last_active":"2023-10-06T10:44:51.758917+0000","last_peered":"2023-10-06T10:44:51.758917+0000","last_clean":"2023-10-06T10:44:51.758917+0000","last_became_active":"2023-10-06T10:44:51.753839+0000","last_became_peered":"2023-10-06T10:44:51.753839+0000","last_unstale":"2023-10-06T10:44:51.758917+0000","last_undegraded":"2023-10-06T10:44:51.758917+0000","last_fullsized":"2023-10-06T10:44:51.758917+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:14:19.607356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"101'1","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.751747+0000","last_change":"2023-10-06T10:44:50.743094+0000","last_active":"2023-10-06T10:44:51.751747+0000","last_peered":"2023-10-06T10:44:51.751747+0000","last_clean":"2023-10-06T10:44:51.751747+0000","last_became_active":"2023-10-06T10:44:50.742703+0000","last_became_peered":"2023-10-06T10:44:50.742703+0000","last_unstale":"2023-10-06T10:44:51.751747+0000","last_undegraded":"2023-10-06T10:44:51.751747+0000","last_fullsized":"2023-10-06T10:44:51.751747+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:25.575241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.748437+0000","last_change":"2023-10-06T10:44:51.744523+0000","last_active":"2023-10-06T10:44:51.748437+0000","last_peered":"2023-10-06T10:44:51.748437+0000","last_clean":"2023-10-06T10:44:51.748437+0000","last_became_active":"2023-10-06T10:44:51.744272+0000","last_became_peered":"2023-10-06T10:44:51.744272+0000","last_unstale":"2023-10-06T10:44:51.748437+0000","last_undegraded":"2023-10-06T10:44:51.748437+0000","last_fullsized":"2023-10-06T10:44:51.748437+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:03.012145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"101'2","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.753032+0000","last_change":"2023-10-06T10:44:50.735822+0000","last_active":"2023-10-06T10:44:51.753032+0000","last_peered":"2023-10-06T10:44:51.753032+0000","last_clean":"2023-10-06T10:44:51.753032+0000","last_became_active":"2023-10-06T10:44:50.735568+0000","last_became_peered":"2023-10-06T10:44:50.735568+0000","last_unstale":"2023-10-06T10:44:51.753032+0000","last_undegraded":"2023-10-06T10:44:51.753032+0000","last_fullsized":"2023-10-06T10:44:51.753032+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:38:19.424287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.756698+0000","last_change":"2023-10-06T10:44:51.754072+0000","last_active":"2023-10-06T10:44:51.756698+0000","last_peered":"2023-10-06T10:44:51.756698+0000","last_clean":"2023-10-06T10:44:51.756698+0000","last_became_active":"2023-10-06T10:44:51.753802+0000","last_became_peered":"2023-10-06T10:44:51.753802+0000","last_unstale":"2023-10-06T10:44:51.756698+0000","last_undegraded":"2023-10-06T10:44:51.756698+0000","last_fullsized":"2023-10-06T10:44:51.756698+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:48:22.355874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"101'1","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.750611+0000","last_change":"2023-10-06T10:44:50.740185+0000","last_active":"2023-10-06T10:44:51.750611+0000","last_peered":"2023-10-06T10:44:51.750611+0000","last_clean":"2023-10-06T10:44:51.750611+0000","last_became_active":"2023-10-06T10:44:50.739930+0000","last_became_peered":"2023-10-06T10:44:50.739930+0000","last_unstale":"2023-10-06T10:44:51.750611+0000","last_undegraded":"2023-10-06T10:44:51.750611+0000","last_fullsized":"2023-10-06T10:44:51.750611+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:47:30.907849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.739759+0000","last_change":"2023-10-06T10:44:51.729671+0000","last_active":"2023-10-06T10:44:51.739759+0000","last_peered":"2023-10-06T10:44:51.739759+0000","last_clean":"2023-10-06T10:44:51.739759+0000","last_became_active":"2023-10-06T10:44:51.729451+0000","last_became_peered":"2023-10-06T10:44:51.729451+0000","last_unstale":"2023-10-06T10:44:51.739759+0000","last_undegraded":"2023-10-06T10:44:51.739759+0000","last_fullsized":"2023-10-06T10:44:51.739759+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:33:51.029950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.725200+0000","last_change":"2023-10-06T10:44:50.743084+0000","last_active":"2023-10-06T10:44:51.725200+0000","last_peered":"2023-10-06T10:44:51.725200+0000","last_clean":"2023-10-06T10:44:51.725200+0000","last_became_active":"2023-10-06T10:44:50.742740+0000","last_became_peered":"2023-10-06T10:44:50.742740+0000","last_unstale":"2023-10-06T10:44:51.725200+0000","last_undegraded":"2023-10-06T10:44:51.725200+0000","last_fullsized":"2023-10-06T10:44:51.725200+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:23.746429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.739337+0000","last_change":"2023-10-06T10:44:51.735032+0000","last_active":"2023-10-06T10:44:51.739337+0000","last_peered":"2023-10-06T10:44:51.739337+0000","last_clean":"2023-10-06T10:44:51.739337+0000","last_became_active":"2023-10-06T10:44:51.734765+0000","last_became_peered":"2023-10-06T10:44:51.734765+0000","last_unstale":"2023-10-06T10:44:51.739337+0000","last_undegraded":"2023-10-06T10:44:51.739337+0000","last_fullsized":"2023-10-06T10:44:51.739337+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:53.858578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"101'2","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757070+0000","last_change":"2023-10-06T10:44:50.735989+0000","last_active":"2023-10-06T10:44:51.757070+0000","last_peered":"2023-10-06T10:44:51.757070+0000","last_clean":"2023-10-06T10:44:51.757070+0000","last_became_active":"2023-10-06T10:44:50.735704+0000","last_became_peered":"2023-10-06T10:44:50.735704+0000","last_unstale":"2023-10-06T10:44:51.757070+0000","last_undegraded":"2023-10-06T10:44:51.757070+0000","last_fullsized":"2023-10-06T10:44:51.757070+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:21.691363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.755630+0000","last_change":"2023-10-06T10:44:51.746428+0000","last_active":"2023-10-06T10:44:51.755630+0000","last_peered":"2023-10-06T10:44:51.755630+0000","last_clean":"2023-10-06T10:44:51.755630+0000","last_became_active":"2023-10-06T10:44:51.746168+0000","last_became_peered":"2023-10-06T10:44:51.746168+0000","last_unstale":"2023-10-06T10:44:51.755630+0000","last_undegraded":"2023-10-06T10:44:51.755630+0000","last_fullsized":"2023-10-06T10:44:51.755630+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:17:28.335605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.733095+0000","last_change":"2023-10-06T10:44:50.737860+0000","last_active":"2023-10-06T10:44:51.733095+0000","last_peered":"2023-10-06T10:44:51.733095+0000","last_clean":"2023-10-06T10:44:51.733095+0000","last_became_active":"2023-10-06T10:44:50.737174+0000","last_became_peered":"2023-10-06T10:44:50.737174+0000","last_unstale":"2023-10-06T10:44:51.733095+0000","last_undegraded":"2023-10-06T10:44:51.733095+0000","last_fullsized":"2023-10-06T10:44:51.733095+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:13.818223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.754190+0000","last_change":"2023-10-06T10:44:51.746749+0000","last_active":"2023-10-06T10:44:51.754190+0000","last_peered":"2023-10-06T10:44:51.754190+0000","last_clean":"2023-10-06T10:44:51.754190+0000","last_became_active":"2023-10-06T10:44:51.746525+0000","last_became_peered":"2023-10-06T10:44:51.746525+0000","last_unstale":"2023-10-06T10:44:51.754190+0000","last_undegraded":"2023-10-06T10:44:51.754190+0000","last_fullsized":"2023-10-06T10:44:51.754190+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:13.902450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.729551+0000","last_change":"2023-10-06T10:44:50.740163+0000","last_active":"2023-10-06T10:44:51.729551+0000","last_peered":"2023-10-06T10:44:51.729551+0000","last_clean":"2023-10-06T10:44:51.729551+0000","last_became_active":"2023-10-06T10:44:50.739892+0000","last_became_peered":"2023-10-06T10:44:50.739892+0000","last_unstale":"2023-10-06T10:44:51.729551+0000","last_undegraded":"2023-10-06T10:44:51.729551+0000","last_fullsized":"2023-10-06T10:44:51.729551+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:31.698962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.747134+0000","last_change":"2023-10-06T10:44:51.743095+0000","last_active":"2023-10-06T10:44:51.747134+0000","last_peered":"2023-10-06T10:44:51.747134+0000","last_clean":"2023-10-06T10:44:51.747134+0000","last_became_active":"2023-10-06T10:44:51.742863+0000","last_became_peered":"2023-10-06T10:44:51.742863+0000","last_unstale":"2023-10-06T10:44:51.747134+0000","last_undegraded":"2023-10-06T10:44:51.747134+0000","last_fullsized":"2023-10-06T10:44:51.747134+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:56:26.690740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.735885+0000","last_change":"2023-10-06T10:44:50.742684+0000","last_active":"2023-10-06T10:44:51.735885+0000","last_peered":"2023-10-06T10:44:51.735885+0000","last_clean":"2023-10-06T10:44:51.735885+0000","last_became_active":"2023-10-06T10:44:50.742434+0000","last_became_peered":"2023-10-06T10:44:50.742434+0000","last_unstale":"2023-10-06T10:44:51.735885+0000","last_undegraded":"2023-10-06T10:44:51.735885+0000","last_fullsized":"2023-10-06T10:44:51.735885+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:57.037993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.753591+0000","last_change":"2023-10-06T10:44:51.751722+0000","last_active":"2023-10-06T10:44:51.753591+0000","last_peered":"2023-10-06T10:44:51.753591+0000","last_clean":"2023-10-06T10:44:51.753591+0000","last_became_active":"2023-10-06T10:44:51.751490+0000","last_became_peered":"2023-10-06T10:44:51.751490+0000","last_unstale":"2023-10-06T10:44:51.753591+0000","last_undegraded":"2023-10-06T10:44:51.753591+0000","last_fullsized":"2023-10-06T10:44:51.753591+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:45.409013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"101'2","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.747535+0000","last_change":"2023-10-06T10:44:50.745009+0000","last_active":"2023-10-06T10:44:51.747535+0000","last_peered":"2023-10-06T10:44:51.747535+0000","last_clean":"2023-10-06T10:44:51.747535+0000","last_became_active":"2023-10-06T10:44:50.744698+0000","last_became_peered":"2023-10-06T10:44:50.744698+0000","last_unstale":"2023-10-06T10:44:51.747535+0000","last_undegraded":"2023-10-06T10:44:51.747535+0000","last_fullsized":"2023-10-06T10:44:51.747535+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:26:41.299786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.758669+0000","last_change":"2023-10-06T10:44:51.751880+0000","last_active":"2023-10-06T10:44:51.758669+0000","last_peered":"2023-10-06T10:44:51.758669+0000","last_clean":"2023-10-06T10:44:51.758669+0000","last_became_active":"2023-10-06T10:44:51.751258+0000","last_became_peered":"2023-10-06T10:44:51.751258+0000","last_unstale":"2023-10-06T10:44:51.758669+0000","last_undegraded":"2023-10-06T10:44:51.758669+0000","last_fullsized":"2023-10-06T10:44:51.758669+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:07.487093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"102'1","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T10:44:53.884285+0000","last_change":"2023-10-06T10:44:50.735743+0000","last_active":"2023-10-06T10:44:53.884285+0000","last_peered":"2023-10-06T10:44:53.884285+0000","last_clean":"2023-10-06T10:44:53.884285+0000","last_became_active":"2023-10-06T10:44:50.735498+0000","last_became_peered":"2023-10-06T10:44:50.735498+0000","last_unstale":"2023-10-06T10:44:53.884285+0000","last_undegraded":"2023-10-06T10:44:53.884285+0000","last_fullsized":"2023-10-06T10:44:53.884285+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:19.608630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757616+0000","last_change":"2023-10-06T10:44:51.749866+0000","last_active":"2023-10-06T10:44:51.757616+0000","last_peered":"2023-10-06T10:44:51.757616+0000","last_clean":"2023-10-06T10:44:51.757616+0000","last_became_active":"2023-10-06T10:44:51.749578+0000","last_became_peered":"2023-10-06T10:44:51.749578+0000","last_unstale":"2023-10-06T10:44:51.757616+0000","last_undegraded":"2023-10-06T10:44:51.757616+0000","last_fullsized":"2023-10-06T10:44:51.757616+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:52.361795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.734752+0000","last_change":"2023-10-06T10:44:50.740894+0000","last_active":"2023-10-06T10:44:51.734752+0000","last_peered":"2023-10-06T10:44:51.734752+0000","last_clean":"2023-10-06T10:44:51.734752+0000","last_became_active":"2023-10-06T10:44:50.740653+0000","last_became_peered":"2023-10-06T10:44:50.740653+0000","last_unstale":"2023-10-06T10:44:51.734752+0000","last_undegraded":"2023-10-06T10:44:51.734752+0000","last_fullsized":"2023-10-06T10:44:51.734752+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:11.169696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.754014+0000","last_change":"2023-10-06T10:44:51.751873+0000","last_active":"2023-10-06T10:44:51.754014+0000","last_peered":"2023-10-06T10:44:51.754014+0000","last_clean":"2023-10-06T10:44:51.754014+0000","last_became_active":"2023-10-06T10:44:51.751589+0000","last_became_peered":"2023-10-06T10:44:51.751589+0000","last_unstale":"2023-10-06T10:44:51.754014+0000","last_undegraded":"2023-10-06T10:44:51.754014+0000","last_fullsized":"2023-10-06T10:44:51.754014+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:12:10.059460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.726919+0000","last_change":"2023-10-06T10:44:50.739163+0000","last_active":"2023-10-06T10:44:51.726919+0000","last_peered":"2023-10-06T10:44:51.726919+0000","last_clean":"2023-10-06T10:44:51.726919+0000","last_became_active":"2023-10-06T10:44:50.738915+0000","last_became_peered":"2023-10-06T10:44:50.738915+0000","last_unstale":"2023-10-06T10:44:51.726919+0000","last_undegraded":"2023-10-06T10:44:51.726919+0000","last_fullsized":"2023-10-06T10:44:51.726919+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:32.689348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.737452+0000","last_change":"2023-10-06T10:44:51.736064+0000","last_active":"2023-10-06T10:44:51.737452+0000","last_peered":"2023-10-06T10:44:51.737452+0000","last_clean":"2023-10-06T10:44:51.737452+0000","last_became_active":"2023-10-06T10:44:51.735730+0000","last_became_peered":"2023-10-06T10:44:51.735730+0000","last_unstale":"2023-10-06T10:44:51.737452+0000","last_undegraded":"2023-10-06T10:44:51.737452+0000","last_fullsized":"2023-10-06T10:44:51.737452+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:42.073984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.725056+0000","last_change":"2023-10-06T10:44:50.738749+0000","last_active":"2023-10-06T10:44:51.725056+0000","last_peered":"2023-10-06T10:44:51.725056+0000","last_clean":"2023-10-06T10:44:51.725056+0000","last_became_active":"2023-10-06T10:44:50.738444+0000","last_became_peered":"2023-10-06T10:44:50.738444+0000","last_unstale":"2023-10-06T10:44:51.725056+0000","last_undegraded":"2023-10-06T10:44:51.725056+0000","last_fullsized":"2023-10-06T10:44:51.725056+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:50:16.152816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.756270+0000","last_change":"2023-10-06T10:44:51.753208+0000","last_active":"2023-10-06T10:44:51.756270+0000","last_peered":"2023-10-06T10:44:51.756270+0000","last_clean":"2023-10-06T10:44:51.756270+0000","last_became_active":"2023-10-06T10:44:51.752963+0000","last_became_peered":"2023-10-06T10:44:51.752963+0000","last_unstale":"2023-10-06T10:44:51.756270+0000","last_undegraded":"2023-10-06T10:44:51.756270+0000","last_fullsized":"2023-10-06T10:44:51.756270+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:58.043687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.732962+0000","last_change":"2023-10-06T10:44:50.740319+0000","last_active":"2023-10-06T10:44:51.732962+0000","last_peered":"2023-10-06T10:44:51.732962+0000","last_clean":"2023-10-06T10:44:51.732962+0000","last_became_active":"2023-10-06T10:44:50.740054+0000","last_became_peered":"2023-10-06T10:44:50.740054+0000","last_unstale":"2023-10-06T10:44:51.732962+0000","last_undegraded":"2023-10-06T10:44:51.732962+0000","last_fullsized":"2023-10-06T10:44:51.732962+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:05:34.133718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.744832+0000","last_change":"2023-10-06T10:44:51.740932+0000","last_active":"2023-10-06T10:44:51.744832+0000","last_peered":"2023-10-06T10:44:51.744832+0000","last_clean":"2023-10-06T10:44:51.744832+0000","last_became_active":"2023-10-06T10:44:51.740255+0000","last_became_peered":"2023-10-06T10:44:51.740255+0000","last_unstale":"2023-10-06T10:44:51.744832+0000","last_undegraded":"2023-10-06T10:44:51.744832+0000","last_fullsized":"2023-10-06T10:44:51.744832+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:33.695043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.737226+0000","last_change":"2023-10-06T10:44:50.737048+0000","last_active":"2023-10-06T10:44:51.737226+0000","last_peered":"2023-10-06T10:44:51.737226+0000","last_clean":"2023-10-06T10:44:51.737226+0000","last_became_active":"2023-10-06T10:44:50.736752+0000","last_became_peered":"2023-10-06T10:44:50.736752+0000","last_unstale":"2023-10-06T10:44:51.737226+0000","last_undegraded":"2023-10-06T10:44:51.737226+0000","last_fullsized":"2023-10-06T10:44:51.737226+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:21.691363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.757668+0000","last_change":"2023-10-06T10:44:51.750254+0000","last_active":"2023-10-06T10:44:51.757668+0000","last_peered":"2023-10-06T10:44:51.757668+0000","last_clean":"2023-10-06T10:44:51.757668+0000","last_became_active":"2023-10-06T10:44:51.750002+0000","last_became_peered":"2023-10-06T10:44:51.750002+0000","last_unstale":"2023-10-06T10:44:51.757668+0000","last_undegraded":"2023-10-06T10:44:51.757668+0000","last_fullsized":"2023-10-06T10:44:51.757668+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:31:12.288467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.735995+0000","last_change":"2023-10-06T10:44:50.740969+0000","last_active":"2023-10-06T10:44:51.735995+0000","last_peered":"2023-10-06T10:44:51.735995+0000","last_clean":"2023-10-06T10:44:51.735995+0000","last_became_active":"2023-10-06T10:44:50.740714+0000","last_became_peered":"2023-10-06T10:44:50.740714+0000","last_unstale":"2023-10-06T10:44:51.735995+0000","last_undegraded":"2023-10-06T10:44:51.735995+0000","last_fullsized":"2023-10-06T10:44:51.735995+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:56:16.765456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.733952+0000","last_change":"2023-10-06T10:44:51.730894+0000","last_active":"2023-10-06T10:44:51.733952+0000","last_peered":"2023-10-06T10:44:51.733952+0000","last_clean":"2023-10-06T10:44:51.733952+0000","last_became_active":"2023-10-06T10:44:51.730349+0000","last_became_peered":"2023-10-06T10:44:51.730349+0000","last_unstale":"2023-10-06T10:44:51.733952+0000","last_undegraded":"2023-10-06T10:44:51.733952+0000","last_fullsized":"2023-10-06T10:44:51.733952+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:36:50.152016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"101'6","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.762876+0000","last_change":"2023-10-06T10:44:50.738740+0000","last_active":"2023-10-06T10:44:51.762876+0000","last_peered":"2023-10-06T10:44:51.762876+0000","last_clean":"2023-10-06T10:44:51.762876+0000","last_became_active":"2023-10-06T10:44:50.738498+0000","last_became_peered":"2023-10-06T10:44:50.738498+0000","last_unstale":"2023-10-06T10:44:51.762876+0000","last_undegraded":"2023-10-06T10:44:51.762876+0000","last_fullsized":"2023-10-06T10:44:51.762876+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:42:51.471629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":90,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.732796+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:44:51.732796+0000","last_peered":"2023-10-06T10:44:51.732796+0000","last_clean":"2023-10-06T10:44:51.732796+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:44:51.732796+0000","last_undegraded":"2023-10-06T10:44:51.732796+0000","last_fullsized":"2023-10-06T10:44:51.732796+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.748949+0000","last_change":"2023-10-06T10:44:51.744508+0000","last_active":"2023-10-06T10:44:51.748949+0000","last_peered":"2023-10-06T10:44:51.748949+0000","last_clean":"2023-10-06T10:44:51.748949+0000","last_became_active":"2023-10-06T10:44:51.744214+0000","last_became_peered":"2023-10-06T10:44:51.744214+0000","last_unstale":"2023-10-06T10:44:51.748949+0000","last_undegraded":"2023-10-06T10:44:51.748949+0000","last_fullsized":"2023-10-06T10:44:51.748949+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:58.043687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"101'3","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.752327+0000","last_change":"2023-10-06T10:44:50.740082+0000","last_active":"2023-10-06T10:44:51.752327+0000","last_peered":"2023-10-06T10:44:51.752327+0000","last_clean":"2023-10-06T10:44:51.752327+0000","last_became_active":"2023-10-06T10:44:50.739924+0000","last_became_peered":"2023-10-06T10:44:50.739924+0000","last_unstale":"2023-10-06T10:44:51.752327+0000","last_undegraded":"2023-10-06T10:44:51.752327+0000","last_fullsized":"2023-10-06T10:44:51.752327+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:27:09.636709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.727020+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:44:51.727020+0000","last_peered":"2023-10-06T10:44:51.727020+0000","last_clean":"2023-10-06T10:44:51.727020+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:44:51.727020+0000","last_undegraded":"2023-10-06T10:44:51.727020+0000","last_fullsized":"2023-10-06T10:44:51.727020+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745719+0000","last_change":"2023-10-06T10:44:51.732830+0000","last_active":"2023-10-06T10:44:51.745719+0000","last_peered":"2023-10-06T10:44:51.745719+0000","last_clean":"2023-10-06T10:44:51.745719+0000","last_became_active":"2023-10-06T10:44:51.732580+0000","last_became_peered":"2023-10-06T10:44:51.732580+0000","last_unstale":"2023-10-06T10:44:51.745719+0000","last_undegraded":"2023-10-06T10:44:51.745719+0000","last_fullsized":"2023-10-06T10:44:51.745719+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:12.175391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.731609+0000","last_change":"2023-10-06T10:44:50.741287+0000","last_active":"2023-10-06T10:44:51.731609+0000","last_peered":"2023-10-06T10:44:51.731609+0000","last_clean":"2023-10-06T10:44:51.731609+0000","last_became_active":"2023-10-06T10:44:50.740987+0000","last_became_peered":"2023-10-06T10:44:50.740987+0000","last_unstale":"2023-10-06T10:44:51.731609+0000","last_undegraded":"2023-10-06T10:44:51.731609+0000","last_fullsized":"2023-10-06T10:44:51.731609+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:27:18.042909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":84,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.729061+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:44:51.729061+0000","last_peered":"2023-10-06T10:44:51.729061+0000","last_clean":"2023-10-06T10:44:51.729061+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:44:51.729061+0000","last_undegraded":"2023-10-06T10:44:51.729061+0000","last_fullsized":"2023-10-06T10:44:51.729061+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":226,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.724876+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:44:51.724876+0000","last_peered":"2023-10-06T10:44:51.724876+0000","last_clean":"2023-10-06T10:44:51.724876+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:44:51.724876+0000","last_undegraded":"2023-10-06T10:44:51.724876+0000","last_fullsized":"2023-10-06T10:44:51.724876+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.746427+0000","last_change":"2023-10-06T10:44:51.734173+0000","last_active":"2023-10-06T10:44:51.746427+0000","last_peered":"2023-10-06T10:44:51.746427+0000","last_clean":"2023-10-06T10:44:51.746427+0000","last_became_active":"2023-10-06T10:44:51.733932+0000","last_became_peered":"2023-10-06T10:44:51.733932+0000","last_unstale":"2023-10-06T10:44:51.746427+0000","last_undegraded":"2023-10-06T10:44:51.746427+0000","last_fullsized":"2023-10-06T10:44:51.746427+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:42.073984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"101'2","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.743590+0000","last_change":"2023-10-06T10:44:50.737521+0000","last_active":"2023-10-06T10:44:51.743590+0000","last_peered":"2023-10-06T10:44:51.743590+0000","last_clean":"2023-10-06T10:44:51.743590+0000","last_became_active":"2023-10-06T10:44:50.737276+0000","last_became_peered":"2023-10-06T10:44:50.737276+0000","last_unstale":"2023-10-06T10:44:51.743590+0000","last_undegraded":"2023-10-06T10:44:51.743590+0000","last_fullsized":"2023-10-06T10:44:51.743590+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:14:37.094197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":76,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.726680+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:44:51.726680+0000","last_peered":"2023-10-06T10:44:51.726680+0000","last_clean":"2023-10-06T10:44:51.726680+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:44:51.726680+0000","last_undegraded":"2023-10-06T10:44:51.726680+0000","last_fullsized":"2023-10-06T10:44:51.726680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.745715+0000","last_change":"2023-10-06T10:44:51.730648+0000","last_active":"2023-10-06T10:44:51.745715+0000","last_peered":"2023-10-06T10:44:51.745715+0000","last_clean":"2023-10-06T10:44:51.745715+0000","last_became_active":"2023-10-06T10:44:51.729905+0000","last_became_peered":"2023-10-06T10:44:51.729905+0000","last_unstale":"2023-10-06T10:44:51.745715+0000","last_undegraded":"2023-10-06T10:44:51.745715+0000","last_fullsized":"2023-10-06T10:44:51.745715+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:07:26.376540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.725894+0000","last_change":"2023-10-06T10:44:50.742945+0000","last_active":"2023-10-06T10:44:51.725894+0000","last_peered":"2023-10-06T10:44:51.725894+0000","last_clean":"2023-10-06T10:44:51.725894+0000","last_became_active":"2023-10-06T10:44:50.742669+0000","last_became_peered":"2023-10-06T10:44:50.742669+0000","last_unstale":"2023-10-06T10:44:51.725894+0000","last_undegraded":"2023-10-06T10:44:51.725894+0000","last_fullsized":"2023-10-06T10:44:51.725894+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:28.430497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.724961+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:44:51.724961+0000","last_peered":"2023-10-06T10:44:51.724961+0000","last_clean":"2023-10-06T10:44:51.724961+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:44:51.724961+0000","last_undegraded":"2023-10-06T10:44:51.724961+0000","last_fullsized":"2023-10-06T10:44:51.724961+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.746547+0000","last_change":"2023-10-06T10:44:51.741319+0000","last_active":"2023-10-06T10:44:51.746547+0000","last_peered":"2023-10-06T10:44:51.746547+0000","last_clean":"2023-10-06T10:44:51.746547+0000","last_became_active":"2023-10-06T10:44:51.740994+0000","last_became_peered":"2023-10-06T10:44:51.740994+0000","last_unstale":"2023-10-06T10:44:51.746547+0000","last_undegraded":"2023-10-06T10:44:51.746547+0000","last_fullsized":"2023-10-06T10:44:51.746547+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:56:17.771151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"101'8","reported_seq":18,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.760275+0000","last_change":"2023-10-06T10:44:50.738355+0000","last_active":"2023-10-06T10:44:51.760275+0000","last_peered":"2023-10-06T10:44:51.760275+0000","last_clean":"2023-10-06T10:44:51.760275+0000","last_became_active":"2023-10-06T10:44:50.738093+0000","last_became_peered":"2023-10-06T10:44:50.738093+0000","last_unstale":"2023-10-06T10:44:51.760275+0000","last_undegraded":"2023-10-06T10:44:51.760275+0000","last_fullsized":"2023-10-06T10:44:51.760275+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:46.580158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.725988+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:44:51.725988+0000","last_peered":"2023-10-06T10:44:51.725988+0000","last_clean":"2023-10-06T10:44:51.725988+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:44:51.725988+0000","last_undegraded":"2023-10-06T10:44:51.725988+0000","last_fullsized":"2023-10-06T10:44:51.725988+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.753934+0000","last_change":"2023-10-06T10:44:51.752435+0000","last_active":"2023-10-06T10:44:51.753934+0000","last_peered":"2023-10-06T10:44:51.753934+0000","last_clean":"2023-10-06T10:44:51.753934+0000","last_became_active":"2023-10-06T10:44:51.752213+0000","last_became_peered":"2023-10-06T10:44:51.752213+0000","last_unstale":"2023-10-06T10:44:51.753934+0000","last_undegraded":"2023-10-06T10:44:51.753934+0000","last_fullsized":"2023-10-06T10:44:51.753934+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:58.636572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.733702+0000","last_change":"2023-10-06T10:44:50.736915+0000","last_active":"2023-10-06T10:44:51.733702+0000","last_peered":"2023-10-06T10:44:51.733702+0000","last_clean":"2023-10-06T10:44:51.733702+0000","last_became_active":"2023-10-06T10:44:50.736660+0000","last_became_peered":"2023-10-06T10:44:50.736660+0000","last_unstale":"2023-10-06T10:44:51.733702+0000","last_undegraded":"2023-10-06T10:44:51.733702+0000","last_fullsized":"2023-10-06T10:44:51.733702+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:50:16.152816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.733032+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:44:51.733032+0000","last_peered":"2023-10-06T10:44:51.733032+0000","last_clean":"2023-10-06T10:44:51.733032+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:44:51.733032+0000","last_undegraded":"2023-10-06T10:44:51.733032+0000","last_fullsized":"2023-10-06T10:44:51.733032+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.753284+0000","last_change":"2023-10-06T10:44:51.746238+0000","last_active":"2023-10-06T10:44:51.753284+0000","last_peered":"2023-10-06T10:44:51.753284+0000","last_clean":"2023-10-06T10:44:51.753284+0000","last_became_active":"2023-10-06T10:44:51.745970+0000","last_became_peered":"2023-10-06T10:44:51.745970+0000","last_unstale":"2023-10-06T10:44:51.753284+0000","last_undegraded":"2023-10-06T10:44:51.753284+0000","last_fullsized":"2023-10-06T10:44:51.753284+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:50.715731+0000","last_clean_scrub_stamp":"2023-10-06T10:44:50.715731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:48:22.355874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.726787+0000","last_change":"2023-10-06T10:44:50.737172+0000","last_active":"2023-10-06T10:44:51.726787+0000","last_peered":"2023-10-06T10:44:51.726787+0000","last_clean":"2023-10-06T10:44:51.726787+0000","last_became_active":"2023-10-06T10:44:50.736902+0000","last_became_peered":"2023-10-06T10:44:50.736902+0000","last_unstale":"2023-10-06T10:44:51.726787+0000","last_undegraded":"2023-10-06T10:44:51.726787+0000","last_fullsized":"2023-10-06T10:44:51.726787+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:44:49.710036+0000","last_clean_scrub_stamp":"2023-10-06T10:44:49.710036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:45:13.294923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T10:44:51.733036+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:44:51.733036+0000","last_peered":"2023-10-06T10:44:51.733036+0000","last_clean":"2023-10-06T10:44:51.733036+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:44:51.733036+0000","last_undegraded":"2023-10-06T10:44:51.733036+0000","last_fullsized":"2023-10-06T10:44:51.733036+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574935,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299112,"kb_used_data":1824,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072728,"statfs":{"total":96636764160,"available":96330473472,"internally_reserved":0,"allocated":1867776,"data_stored":1203651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.51000000000000001,"15min":0.48799999999999999},"min":{"1min":0.32800000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63900000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.53800000000000003,"15min":0.52100000000000002},"min":{"1min":0.29399999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.9700000000000002,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":0.51000000000000001}]},{"osd":1,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.496,"15min":0.46700000000000003},"min":{"1min":0.32000000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.64800000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.54500000000000004,"15min":0.51300000000000001},"min":{"1min":0.38400000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":3.0059999999999998,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56699999999999995,"15min":0.52200000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.51900000000000002,"15min":0.499},"min":{"1min":0.371,"5min":0.314,"15min":0.314},"max":{"1min":0.66800000000000004,"5min":0.747,"15min":0.79800000000000004},"last":0.61899999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55200000000000005,"15min":0.51600000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73099999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.40600000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.71899999999999997,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.53800000000000003}]},{"osd":4,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42199999999999999,"15min":0.46200000000000002},"min":{"1min":0.314,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61899999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47299999999999998,"15min":0.499},"min":{"1min":0.30199999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64900000000000002,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.55300000000000005}]},{"osd":5,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.48199999999999998},"min":{"1min":0.33000000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.81100000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.51800000000000002,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85399999999999998,"5min":1.577,"15min":1.577},"last":0.56899999999999995}]},{"osd":6,"last update":"Fri Oct 6 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.56499999999999995},"min":{"1min":0.39700000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.67200000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.56200000000000006,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69599999999999995,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.66800000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298524,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073316,"statfs":{"total":96636764160,"available":96331075584,"internally_reserved":0,"allocated":1265664,"data_stored":611412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.495},"min":{"1min":0.34599999999999997,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":0.70399999999999996,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52100000000000002,"15min":0.45500000000000002},"min":{"1min":0.39700000000000002,"5min":0.32500000000000001,"15min":0.32000000000000001},"max":{"1min":0.67400000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.59899999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52000000000000002,"15min":0.46000000000000002},"min":{"1min":0.39300000000000002,"5min":0.32200000000000001,"15min":0.28000000000000003},"max":{"1min":0.70799999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54500000000000004,"15min":0.48899999999999999},"min":{"1min":0.42399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73499999999999999,"5min":0.751,"15min":0.751},"last":0.621}]},{"osd":2,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52700000000000002,"15min":0.48399999999999999},"min":{"1min":0.39600000000000002,"5min":0.32500000000000001,"15min":0.309},"max":{"1min":0.65400000000000003,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54500000000000004,"15min":0.48699999999999999},"min":{"1min":0.443,"5min":0.34999999999999998,"15min":0.34300000000000003},"max":{"1min":0.66100000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.69699999999999995}]},{"osd":3,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55200000000000005,"15min":0.50600000000000001},"min":{"1min":0.46500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75600000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.55500000000000005,"15min":0.51900000000000002},"min":{"1min":0.40699999999999997,"5min":0.36399999999999999,"15min":0.34499999999999997},"max":{"1min":0.74199999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.79200000000000004}]},{"osd":4,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.48699999999999999,"15min":0.48099999999999998},"min":{"1min":0.34999999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.78300000000000003,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.498,"15min":0.48899999999999999},"min":{"1min":0.377,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79400000000000004,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.71499999999999997}]},{"osd":5,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.51000000000000001,"15min":0.49199999999999999},"min":{"1min":0.34200000000000003,"5min":0.28899999999999998,"15min":0.28299999999999997},"max":{"1min":0.83599999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.38700000000000001,"5min":0.33900000000000002,"15min":0.308},"max":{"1min":0.77800000000000002,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.93100000000000005}]},{"osd":7,"last update":"Fri Oct 6 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53200000000000003,"15min":0.52000000000000002},"min":{"1min":0.36599999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71899999999999997,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.755},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57399999999999995,"15min":0.54800000000000004},"min":{"1min":0.40000000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.871,"5min":0.879,"15min":0.879},"last":0.76800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303208,"kb_used_data":1824,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068632,"statfs":{"total":96636764160,"available":96326279168,"internally_reserved":0,"allocated":1867776,"data_stored":1203187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.20399999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.35599999999999998,"15min":0.34799999999999998},"min":{"1min":0.23400000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.57099999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.315}]},{"osd":2,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.33000000000000002,"15min":0.32000000000000001},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.46999999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.254},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38100000000000001,"15min":0.36699999999999999},"min":{"1min":0.27700000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.29999999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39500000000000002,"15min":0.38},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.60999999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.38700000000000001,"15min":0.377},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.374}]},{"osd":4,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.40200000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73099999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.45600000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86599999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.55000000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.35699999999999998},"max":{"1min":0.80900000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.41099999999999998}]},{"osd":6,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.59899999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.36699999999999999},"max":{"1min":0.92200000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.435,"5min":0.40000000000000002,"15min":0.39800000000000002},"max":{"1min":0.77000000000000002,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.497}]},{"osd":7,"last update":"Fri Oct 6 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.39100000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.84299999999999997,"5min":0.996,"15min":0.996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.42099999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.53300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574935,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298544,"kb_used_data":1256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073296,"statfs":{"total":96636764160,"available":96331055104,"internally_reserved":0,"allocated":1286144,"data_stored":614343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.33000000000000002,"15min":0.32000000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.433,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.376,"15min":0.34999999999999998},"min":{"1min":0.28399999999999997,"5min":0.25,"15min":0.25},"max":{"1min":0.504,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.36699999999999999}]},{"osd":2,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.32100000000000001,"15min":0.309},"min":{"1min":0.246,"5min":0.191,"15min":0.191},"max":{"1min":0.41799999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35599999999999998,"15min":0.33200000000000002},"min":{"1min":0.26900000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.46000000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.28999999999999998}]},{"osd":3,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35399999999999998,"15min":0.34399999999999997},"min":{"1min":0.25600000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.49299999999999999,"5min":0.622,"15min":0.622},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36599999999999999,"15min":0.35299999999999998},"min":{"1min":0.28999999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.47899999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.38500000000000001}]},{"osd":4,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.41799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78900000000000003,"5min":0.81200000000000006,"15min":0.83299999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56599999999999995,"15min":0.54100000000000004},"min":{"1min":0.38600000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.753,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.502}]},{"osd":5,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.36899999999999999},"max":{"1min":0.80300000000000005,"5min":0.91900000000000004,"15min":1.0069999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.41299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76700000000000002,"5min":0.80900000000000005,"15min":0.94499999999999995},"last":0.59499999999999997}]},{"osd":6,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56999999999999995,"15min":0.58199999999999996},"min":{"1min":0.40300000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.747,"5min":1.046,"15min":1.046},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.42999999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76000000000000001,"5min":0.86399999999999999,"15min":1.075},"last":0.55500000000000005}]},{"osd":7,"last update":"Fri Oct 6 10:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80700000000000005,"5min":0.83499999999999996,"15min":0.96099999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.44400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76900000000000002,"5min":0.88800000000000001,"15min":0.90100000000000002},"last":0.57499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298524,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073316,"statfs":{"total":96636764160,"available":96331075584,"internally_reserved":0,"allocated":1265664,"data_stored":612695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.317,"15min":0.312},"min":{"1min":0.22500000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.45300000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.36199999999999999,"15min":0.32900000000000001},"min":{"1min":0.27600000000000002,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.502,"5min":0.622,"15min":0.622},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34499999999999997,"15min":0.32900000000000001},"min":{"1min":0.23499999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.46600000000000003,"5min":0.56599999999999995,"15min":0.60499999999999998},"last":0.437},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.378,"15min":0.36799999999999999},"min":{"1min":0.26700000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.55600000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.47199999999999998}]},{"osd":3,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.38700000000000001,"15min":0.35699999999999998},"min":{"1min":0.28599999999999998,"5min":0.252,"15min":0.20499999999999999},"max":{"1min":0.55000000000000004,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.504},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38,"15min":0.36199999999999999},"min":{"1min":0.28499999999999998,"5min":0.25,"15min":0.22800000000000001},"max":{"1min":0.58099999999999996,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.42699999999999999}]},{"osd":4,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.35299999999999998,"5min":0.33400000000000002,"15min":0.315},"max":{"1min":0.78000000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.441,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81100000000000005,"5min":0.82699999999999996,"15min":0.84899999999999998},"last":0.54900000000000004}]},{"osd":5,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73899999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.42799999999999999,"5min":0.36599999999999999,"15min":0.33800000000000002},"max":{"1min":0.80100000000000005,"5min":0.81799999999999995,"15min":0.83399999999999996},"last":0.58999999999999997}]},{"osd":6,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.61699999999999999},"min":{"1min":0.42199999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.871,"5min":0.89000000000000001,"15min":0.89100000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.621,"15min":0.622},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79300000000000004,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63900000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58999999999999997,"15min":0.60799999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.36799999999999999},"max":{"1min":0.84199999999999997,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60999999999999999,"15min":0.622},"min":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.749,"5min":0.874,"15min":0.874},"last":0.57599999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298540,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073300,"statfs":{"total":96636764160,"available":96331059200,"internally_reserved":0,"allocated":1282048,"data_stored":614301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.314,"15min":0.41499999999999998},"min":{"1min":0.20599999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.433,"5min":0.52700000000000002,"15min":3.3210000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34399999999999997,"15min":0.35599999999999998},"min":{"1min":0.21099999999999999,"5min":0.192,"15min":0.192},"max":{"1min":0.436,"5min":0.66800000000000004,"15min":1.218},"last":0.24099999999999999}]},{"osd":1,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.33700000000000002,"15min":0.33900000000000002},"min":{"1min":0.23699999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.40600000000000003,"5min":0.63600000000000001,"15min":0.86299999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.36499999999999999,"15min":0.45600000000000002},"min":{"1min":0.28399999999999997,"5min":0.218,"15min":0.218},"max":{"1min":0.46000000000000002,"5min":0.66000000000000003,"15min":3.359},"last":0.26900000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.36399999999999999,"15min":0.377},"min":{"1min":0.27800000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.55500000000000005,"5min":0.71099999999999997,"15min":1.1100000000000001},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.377,"5min":0.38300000000000001,"15min":0.58899999999999997},"min":{"1min":0.28399999999999997,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46100000000000002,"5min":0.59899999999999998,"15min":7.069},"last":0.39700000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55900000000000005,"15min":0.57999999999999996},"min":{"1min":0.41599999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.66300000000000003,"5min":0.89400000000000002,"15min":1.165},"last":0.504},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.58599999999999997,"15min":0.59299999999999997},"min":{"1min":0.379,"5min":0.375,"15min":0.375},"max":{"1min":0.753,"5min":1.02,"15min":1.02},"last":0.45400000000000001}]},{"osd":5,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.58199999999999996,"15min":0.67500000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71499999999999997,"5min":0.92500000000000004,"15min":3.4140000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57699999999999996,"15min":0.60099999999999998},"min":{"1min":0.39200000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.96899999999999997,"15min":1.1930000000000001},"last":0.442}]},{"osd":6,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.60299999999999998,"15min":0.61299999999999999},"min":{"1min":0.42599999999999999,"5min":0.38,"15min":0.378},"max":{"1min":0.83699999999999997,"5min":1.054,"15min":1.054},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61899999999999999,"15min":0.70599999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.41999999999999998},"max":{"1min":0.80800000000000005,"5min":0.876,"15min":3.3879999999999999},"last":0.52300000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85599999999999998,"5min":0.99099999999999999,"15min":1.137},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.436,"5min":0.378,"15min":0.378},"max":{"1min":0.67600000000000005,"5min":0.97899999999999998,"15min":1.014},"last":0.48999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574933,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298524,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073316,"statfs":{"total":96636764160,"available":96331075584,"internally_reserved":0,"allocated":1265664,"data_stored":612695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.502,"15min":0.48799999999999999},"min":{"1min":0.316,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73799999999999999,"5min":0.74299999999999999,"15min":0.76800000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52100000000000002,"15min":0.504},"min":{"1min":0.36099999999999999,"5min":0.33100000000000002,"15min":0.28299999999999997},"max":{"1min":0.71399999999999997,"5min":0.73099999999999998,"15min":0.77400000000000002},"last":0.35499999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71199999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51500000000000001,"15min":0.53300000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70499999999999996,"5min":0.71799999999999997,"15min":0.82899999999999996},"last":0.39900000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.32800000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.81000000000000005},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.372,"5min":0.36099999999999999,"15min":0.315},"max":{"1min":0.69399999999999995,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.433}]},{"osd":3,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55200000000000005,"15min":0.52400000000000002},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.35499999999999998},"max":{"1min":0.73899999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.79500000000000004},"last":0.49299999999999999}]},{"osd":5,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.436,"15min":0.48799999999999999},"min":{"1min":0.30399999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.622,"5min":0.77800000000000002,"15min":0.86899999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46800000000000003,"15min":0.49299999999999999},"min":{"1min":0.29499999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.65200000000000002,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.52100000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.496,"15min":0.53300000000000003},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53500000000000003,"15min":0.56799999999999995},"min":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69999999999999996,"5min":0.88,"15min":0.91200000000000003},"last":0.54700000000000004}]},{"osd":7,"last update":"Fri Oct 6 10:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51700000000000002,"15min":0.56299999999999994},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.67900000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53400000000000003,"15min":0.58299999999999996},"min":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72899999999999998,"5min":0.81299999999999994,"15min":0.85699999999999998},"last":0.58899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574933,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298532,"kb_used_data":1244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073308,"statfs":{"total":96636764160,"available":96331067392,"internally_reserved":0,"allocated":1273856,"data_stored":613271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50900000000000001,"15min":0.47499999999999998},"min":{"1min":0.28599999999999998,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.66400000000000003,"5min":0.88,"15min":0.88},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.50800000000000001,"15min":0.52400000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.64700000000000002,"5min":0.86499999999999999,"15min":1.2549999999999999},"last":0.64200000000000002}]},{"osd":1,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.314,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.59999999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.32300000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64300000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.45300000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53300000000000003,"15min":0.50700000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.30399999999999999},"max":{"1min":0.61899999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.503,"5min":0.54300000000000004,"15min":0.55900000000000005},"min":{"1min":0.35399999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67700000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.54000000000000004,"15min":0.56599999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57899999999999996,"5min":0.79900000000000004,"15min":0.95099999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.51800000000000002,"15min":0.54800000000000004},"min":{"1min":0.33200000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70499999999999996,"5min":0.79100000000000004,"15min":1.3999999999999999},"last":0.52600000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47399999999999998,"15min":0.48499999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.75700000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.499,"15min":0.50600000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.68999999999999995,"5min":0.70399999999999996,"15min":0.80700000000000005},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.495,"15min":0.51500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.67800000000000005,"5min":0.77200000000000002,"15min":0.85799999999999998},"last":0.62},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78100000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.629}]},{"osd":7,"last update":"Fri Oct 6 10:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54700000000000004,"15min":0.57399999999999995},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.83299999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71399999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.65600000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:45:00.649 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T10:45:00.650 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T10:45:00.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.804+0000 7ff93fc0d700 1 Processor -- start 2023-10-06T10:45:00.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.804+0000 7ff93fc0d700 1 -- start start 2023-10-06T10:45:00.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.805+0000 7ff93fc0d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff9381594d0 0x7ff9380cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:00.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.805+0000 7ff93fc0d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9381598a0 0x7ff9380cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:00.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.805+0000 7ff93fc0d700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9380cf310 con 0x7ff9380ce7e0 2023-10-06T10:45:00.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.805+0000 7ff93fc0d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9380cf5d0 con 0x7ff9381594d0 2023-10-06T10:45:00.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.805+0000 7ff93fc0d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9380cf8b0 con 0x7ff9381598a0 2023-10-06T10:45:00.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.805+0000 7ff93d9a9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff9381594d0 0x7ff9380cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:00.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.805+0000 7ff93d9a9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff9381594d0 0x7ff9380cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33862/0 (socket says 172.21.15.92:33862) 2023-10-06T10:45:00.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.805+0000 7ff93d9a9700 1 -- 172.21.15.92:0/2668279988 learned_addr learned my addr 172.21.15.92:0/2668279988 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:00.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.805+0000 7ff93d1a8700 1 --2- 172.21.15.92:0/2668279988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9381598a0 0x7ff9380cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:00.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.805+0000 7ff93d9a9700 1 -- 172.21.15.92:0/2668279988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9381598a0 msgr2=0x7ff9380cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:00.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.805+0000 7ff93d9a9700 1 --2- 172.21.15.92:0/2668279988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9381598a0 0x7ff9380cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:00.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.805+0000 7ff93d9a9700 1 -- 172.21.15.92:0/2668279988 >> v1:172.21.15.124:6789/0 conn(0x7ff9380ce7e0 legacy=0x7ff9380cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:00.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.805+0000 7ff93d9a9700 1 -- 172.21.15.92:0/2668279988 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9380cfb90 con 0x7ff9381594d0 2023-10-06T10:45:00.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.805+0000 7ff93d9a9700 1 --2- 172.21.15.92:0/2668279988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9381594d0 0x7ff9380cb560 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7ff930009f20 tx=0x7ff930009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb7a20678ffe331 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:00.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.805+0000 7ff93c9a7700 1 -- 172.21.15.92:0/2668279988 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93000e070 con 0x7ff9381594d0 2023-10-06T10:45:00.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.805+0000 7ff93c9a7700 1 -- 172.21.15.92:0/2668279988 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93000ba40 con 0x7ff9381594d0 2023-10-06T10:45:00.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.805+0000 7ff93c9a7700 1 -- 172.21.15.92:0/2668279988 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9300092e0 con 0x7ff9381594d0 2023-10-06T10:45:00.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.806+0000 7ff93fc0d700 1 -- 172.21.15.92:0/2668279988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9381594d0 msgr2=0x7ff9380cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:00.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.806+0000 7ff93fc0d700 1 --2- 172.21.15.92:0/2668279988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9381594d0 0x7ff9380cb560 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7ff930009f20 tx=0x7ff930009840 comp rx=0 tx=0).stop 2023-10-06T10:45:00.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.806+0000 7ff93fc0d700 1 -- 172.21.15.92:0/2668279988 shutdown_connections 2023-10-06T10:45:00.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.806+0000 7ff93fc0d700 1 --2- 172.21.15.92:0/2668279988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9381594d0 0x7ff9380cb560 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:00.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.806+0000 7ff93fc0d700 1 --2- 172.21.15.92:0/2668279988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9381598a0 0x7ff9380cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:00.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.806+0000 7ff93fc0d700 1 -- 172.21.15.92:0/2668279988 >> 172.21.15.92:0/2668279988 conn(0x7ff938089e50 msgr2=0x7ff938099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:00.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.806+0000 7ff93fc0d700 1 -- 172.21.15.92:0/2668279988 shutdown_connections 2023-10-06T10:45:00.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.806+0000 7ff93fc0d700 1 -- 172.21.15.92:0/2668279988 wait complete. 2023-10-06T10:45:00.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.807+0000 7ff93fc0d700 1 Processor -- start 2023-10-06T10:45:00.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.807+0000 7ff93fc0d700 1 -- start start 2023-10-06T10:45:00.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.808+0000 7ff93fc0d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9380ce7e0 0x7ff93816aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:00.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.808+0000 7ff93fc0d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9381594d0 0x7ff93816b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:00.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.808+0000 7ff93fc0d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9381598a0 0x7ff9380b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:00.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.808+0000 7ff93fc0d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9380b4060 con 0x7ff9381598a0 2023-10-06T10:45:00.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.808+0000 7ff93fc0d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9380b3450 con 0x7ff9381594d0 2023-10-06T10:45:00.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.808+0000 7ff93fc0d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9380b3730 con 0x7ff9380ce7e0 2023-10-06T10:45:00.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.808+0000 7ff93e1aa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9381598a0 0x7ff9380b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:00.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.808+0000 7ff93e1aa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9381598a0 0x7ff9380b5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33870/0 (socket says 172.21.15.92:33870) 2023-10-06T10:45:00.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.808+0000 7ff93e1aa700 1 -- 172.21.15.92:0/3988795146 learned_addr learned my addr 172.21.15.92:0/3988795146 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:00.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.808+0000 7ff93d9a9700 1 --2- 172.21.15.92:0/3988795146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9380ce7e0 0x7ff93816aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:00.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.808+0000 7ff93d1a8700 1 --2- 172.21.15.92:0/3988795146 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9381594d0 0x7ff93816b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:00.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.808+0000 7ff93e1aa700 1 -- 172.21.15.92:0/3988795146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9380ce7e0 msgr2=0x7ff93816aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:00.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.808+0000 7ff93e1aa700 1 --2- 172.21.15.92:0/3988795146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9380ce7e0 0x7ff93816aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:00.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.808+0000 7ff93e1aa700 1 -- 172.21.15.92:0/3988795146 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9381594d0 msgr2=0x7ff93816b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:00.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.808+0000 7ff93e1aa700 1 --2- 172.21.15.92:0/3988795146 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9381594d0 0x7ff93816b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:00.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.808+0000 7ff93e1aa700 1 -- 172.21.15.92:0/3988795146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff930009ae0 con 0x7ff9381598a0 2023-10-06T10:45:00.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.808+0000 7ff93e1aa700 1 --2- 172.21.15.92:0/3988795146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9381598a0 0x7ff9380b5a20 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7ff93400c4f0 tx=0x7ff934007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:00.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.808+0000 7ff92dffb700 1 -- 172.21.15.92:0/3988795146 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff934011590 con 0x7ff9381598a0 2023-10-06T10:45:00.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.808+0000 7ff92dffb700 1 -- 172.21.15.92:0/3988795146 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9340116f0 con 0x7ff9381598a0 2023-10-06T10:45:00.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.808+0000 7ff93fc0d700 1 -- 172.21.15.92:0/3988795146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9380b3a10 con 0x7ff9381598a0 2023-10-06T10:45:00.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.808+0000 7ff92dffb700 1 -- 172.21.15.92:0/3988795146 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff934011ab0 con 0x7ff9381598a0 2023-10-06T10:45:00.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.809+0000 7ff93fc0d700 1 -- 172.21.15.92:0/3988795146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff93816aff0 con 0x7ff9381598a0 2023-10-06T10:45:00.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.809+0000 7ff93fc0d700 1 -- 172.21.15.92:0/3988795146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff938089580 con 0x7ff9381598a0 2023-10-06T10:45:00.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.810+0000 7ff92dffb700 1 -- 172.21.15.92:0/3988795146 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff934012420 con 0x7ff9381598a0 2023-10-06T10:45:00.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.810+0000 7ff92dffb700 1 --2- 172.21.15.92:0/3988795146 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff91c066f30 0x7ff91c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:00.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.810+0000 7ff92dffb700 1 -- 172.21.15.92:0/3988795146 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff934083e40 con 0x7ff9381598a0 2023-10-06T10:45:00.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.810+0000 7ff93d9a9700 1 --2- 172.21.15.92:0/3988795146 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff91c066f30 0x7ff91c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:00.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.811+0000 7ff93d9a9700 1 --2- 172.21.15.92:0/3988795146 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff91c066f30 0x7ff91c0693e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff930009ab0 tx=0x7ff930017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:00.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:00.814+0000 7ff92dffb700 1 -- 172.21.15.92:0/3988795146 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff934050e10 con 0x7ff9381598a0 2023-10-06T10:45:01.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.070+0000 7ff93fc0d700 1 -- 172.21.15.92:0/3988795146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff938064a90 con 0x7ff9381598a0 2023-10-06T10:45:01.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.074+0000 7ff92dffb700 1 -- 172.21.15.92:0/3988795146 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7ff934054430 con 0x7ff9381598a0 2023-10-06T10:45:01.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.077+0000 7ff93fc0d700 1 -- 172.21.15.92:0/3988795146 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff91c066f30 msgr2=0x7ff91c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:01.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.077+0000 7ff93fc0d700 1 --2- 172.21.15.92:0/3988795146 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff91c066f30 0x7ff91c0693e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff930009ab0 tx=0x7ff930017000 comp rx=0 tx=0).stop 2023-10-06T10:45:01.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.077+0000 7ff93fc0d700 1 -- 172.21.15.92:0/3988795146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9381598a0 msgr2=0x7ff9380b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:01.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.077+0000 7ff93fc0d700 1 --2- 172.21.15.92:0/3988795146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9381598a0 0x7ff9380b5a20 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7ff93400c4f0 tx=0x7ff934007f30 comp rx=0 tx=0).stop 2023-10-06T10:45:01.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.077+0000 7ff93fc0d700 1 -- 172.21.15.92:0/3988795146 shutdown_connections 2023-10-06T10:45:01.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.077+0000 7ff93fc0d700 1 --2- 172.21.15.92:0/3988795146 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff91c066f30 0x7ff91c0693e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.077+0000 7ff93fc0d700 1 --2- 172.21.15.92:0/3988795146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9381598a0 0x7ff9380b5a20 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.077+0000 7ff93fc0d700 1 --2- 172.21.15.92:0/3988795146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9380ce7e0 0x7ff93816aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.077+0000 7ff93fc0d700 1 --2- 172.21.15.92:0/3988795146 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9381594d0 0x7ff93816b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.077+0000 7ff93fc0d700 1 -- 172.21.15.92:0/3988795146 >> 172.21.15.92:0/3988795146 conn(0x7ff938089e50 msgr2=0x7ff938069660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:01.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.077+0000 7ff93fc0d700 1 -- 172.21.15.92:0/3988795146 shutdown_connections 2023-10-06T10:45:01.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.077+0000 7ff93fc0d700 1 -- 172.21.15.92:0/3988795146 wait complete. 2023-10-06T10:45:01.079 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-10-06T10:45:01.091 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:45:01.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.205+0000 7f6e01f90700 1 Processor -- start 2023-10-06T10:45:01.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.205+0000 7f6e01f90700 1 -- start start 2023-10-06T10:45:01.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.206+0000 7f6e01f90700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dfc0b49a0 0x7f6dfc0aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:01.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.206+0000 7f6e01f90700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dfc0b0c60 con 0x7f6dfc159560 2023-10-06T10:45:01.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.206+0000 7f6e01f90700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dfc0b0f40 con 0x7f6dfc0b0760 2023-10-06T10:45:01.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.206+0000 7f6e01f90700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc0b1220 con 0x7f6dfc0b49a0 2023-10-06T10:45:01.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.206+0000 7f6dfb7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6dfc159560 0x7f6dfc0aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:43946/0 (socket says 172.21.15.92:43946) 2023-10-06T10:45:01.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.206+0000 7f6dfb7fe700 1 -- 172.21.15.92:0/3796085218 learned_addr learned my addr 172.21.15.92:0/3796085218 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:01.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.206+0000 7f6dfaffd700 1 --2- 172.21.15.92:0/3796085218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dfc0b49a0 0x7f6dfc0aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:01.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfa7fc700 1 -- 172.21.15.92:0/3796085218 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800327651 0 0) 0x7f6dfc0b0c60 con 0x7f6dfc159560 2023-10-06T10:45:01.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfa7fc700 1 -- 172.21.15.92:0/3796085218 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6de4003570 con 0x7f6dfc159560 2023-10-06T10:45:01.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfa7fc700 1 -- 172.21.15.92:0/3796085218 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 435971134 0 0) 0x7f6de4003570 con 0x7f6dfc159560 2023-10-06T10:45:01.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfa7fc700 1 -- 172.21.15.92:0/3796085218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dfc0b49a0 msgr2=0x7f6dfc0aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:01.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfa7fc700 1 --2- 172.21.15.92:0/3796085218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dfc0b49a0 0x7f6dfc0aac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfa7fc700 1 -- 172.21.15.92:0/3796085218 >> v1:172.21.15.124:6789/0 conn(0x7f6dfc0b0760 legacy=0x7f6dfc0b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:01.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfa7fc700 1 -- 172.21.15.92:0/3796085218 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dfc0b1500 con 0x7f6dfc159560 2023-10-06T10:45:01.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfa7fc700 1 -- 172.21.15.92:0/3796085218 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f6dec001970 con 0x7f6dfc159560 2023-10-06T10:45:01.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfa7fc700 1 -- 172.21.15.92:0/3796085218 >> v1:172.21.15.92:6789/0 conn(0x7f6dfc159560 legacy=0x7f6dfc0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:01.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfa7fc700 1 --2- 172.21.15.92:0/3796085218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6de4003a90 0x7f6de4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:01.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfa7fc700 1 --2- 172.21.15.92:0/3796085218 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6de4004810 0x7f6de4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:01.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfa7fc700 1 --2- 172.21.15.92:0/3796085218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dfc0b49a0 0x7f6de40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:01.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfa7fc700 1 -- 172.21.15.92:0/3796085218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc0b1220 con 0x7f6de4003a90 2023-10-06T10:45:01.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfa7fc700 1 -- 172.21.15.92:0/3796085218 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de400bb10 con 0x7f6de4004810 2023-10-06T10:45:01.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfa7fc700 1 -- 172.21.15.92:0/3796085218 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de400bcd0 con 0x7f6dfc0b49a0 2023-10-06T10:45:01.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfb7fe700 1 --2- 172.21.15.92:0/3796085218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6de4003a90 0x7f6de4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:01.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfaffd700 1 --2- 172.21.15.92:0/3796085218 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6de4004810 0x7f6de4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:01.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfbfff700 1 --2- 172.21.15.92:0/3796085218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dfc0b49a0 0x7f6de40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:01.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfb7fe700 1 -- 172.21.15.92:0/3796085218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dfc0b49a0 msgr2=0x7f6de40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:01.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfb7fe700 1 --2- 172.21.15.92:0/3796085218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dfc0b49a0 0x7f6de40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfb7fe700 1 -- 172.21.15.92:0/3796085218 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6de4004810 msgr2=0x7f6de4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:01.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfb7fe700 1 --2- 172.21.15.92:0/3796085218 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6de4004810 0x7f6de4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfb7fe700 1 -- 172.21.15.92:0/3796085218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de4003570 con 0x7f6de4003a90 2023-10-06T10:45:01.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfaffd700 1 --2- 172.21.15.92:0/3796085218 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6de4004810 0x7f6de4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:01.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfbfff700 1 --2- 172.21.15.92:0/3796085218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dfc0b49a0 0x7f6de40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:01.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.207+0000 7f6dfb7fe700 1 --2- 172.21.15.92:0/3796085218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6de4003a90 0x7f6de4003e60 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00c690 tx=0x7f6dec007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a5731ccdbcdb3a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:01.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.208+0000 7f6dfa7fc700 1 -- 172.21.15.92:0/3796085218 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec005b70 con 0x7f6de4003a90 2023-10-06T10:45:01.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.208+0000 7f6dfa7fc700 1 -- 172.21.15.92:0/3796085218 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dec005cf0 con 0x7f6de4003a90 2023-10-06T10:45:01.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.208+0000 7f6dfa7fc700 1 -- 172.21.15.92:0/3796085218 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec007360 con 0x7f6de4003a90 2023-10-06T10:45:01.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.208+0000 7f6e01f90700 1 -- 172.21.15.92:0/3796085218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6de4003a90 msgr2=0x7f6de4003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:01.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.208+0000 7f6e01f90700 1 --2- 172.21.15.92:0/3796085218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6de4003a90 0x7f6de4003e60 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00c690 tx=0x7f6dec007ca0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.208+0000 7f6dfbfff700 1 -- 172.21.15.92:0/3796085218 reap_dead start 2023-10-06T10:45:01.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.208+0000 7f6e01f90700 1 -- 172.21.15.92:0/3796085218 shutdown_connections 2023-10-06T10:45:01.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.208+0000 7f6e01f90700 1 -- 172.21.15.92:0/3796085218 >> 172.21.15.92:0/3796085218 conn(0x7f6dfc0997f0 msgr2=0x7f6dfc06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:01.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.208+0000 7f6e01f90700 1 -- 172.21.15.92:0/3796085218 shutdown_connections 2023-10-06T10:45:01.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.208+0000 7f6e01f90700 1 -- 172.21.15.92:0/3796085218 wait complete. 2023-10-06T10:45:01.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.209+0000 7f6e01f90700 1 Processor -- start 2023-10-06T10:45:01.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.209+0000 7f6e01f90700 1 -- start start 2023-10-06T10:45:01.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.210+0000 7f6e01f90700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6de4003a90 0x7f6dfc1538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:01.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.210+0000 7f6e01f90700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dfc09aa60 0x7f6dfc1552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:01.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.210+0000 7f6e01f90700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6dfc0cf2d0 0x7f6dfc0695d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:01.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.210+0000 7f6e01f90700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc154e40 con 0x7f6de4003a90 2023-10-06T10:45:01.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.210+0000 7f6e01f90700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc154230 con 0x7f6dfc0cf2d0 2023-10-06T10:45:01.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.210+0000 7f6e01f90700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc154510 con 0x7f6dfc09aa60 2023-10-06T10:45:01.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.210+0000 7f6dfb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6de4003a90 0x7f6dfc1538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:01.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.210+0000 7f6dfb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6de4003a90 0x7f6dfc1538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33890/0 (socket says 172.21.15.92:33890) 2023-10-06T10:45:01.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.210+0000 7f6dfb7fe700 1 -- 172.21.15.92:0/2489854993 learned_addr learned my addr 172.21.15.92:0/2489854993 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:01.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.210+0000 7f6dfaffd700 1 --2- 172.21.15.92:0/2489854993 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dfc09aa60 0x7f6dfc1552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:01.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.210+0000 7f6dfbfff700 1 --2- 172.21.15.92:0/2489854993 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6dfc0cf2d0 0x7f6dfc0695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:01.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.210+0000 7f6dfb7fe700 1 -- 172.21.15.92:0/2489854993 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dfc09aa60 msgr2=0x7f6dfc1552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:01.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.210+0000 7f6dfb7fe700 1 --2- 172.21.15.92:0/2489854993 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dfc09aa60 0x7f6dfc1552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.210+0000 7f6dfb7fe700 1 -- 172.21.15.92:0/2489854993 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6dfc0cf2d0 msgr2=0x7f6dfc0695d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:01.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.210+0000 7f6dfb7fe700 1 --2- 172.21.15.92:0/2489854993 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6dfc0cf2d0 0x7f6dfc0695d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.210+0000 7f6dfb7fe700 1 -- 172.21.15.92:0/2489854993 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dfc1547f0 con 0x7f6de4003a90 2023-10-06T10:45:01.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.210+0000 7f6dfaffd700 1 --2- 172.21.15.92:0/2489854993 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dfc09aa60 0x7f6dfc1552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:01.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.211+0000 7f6dfb7fe700 1 --2- 172.21.15.92:0/2489854993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6de4003a90 0x7f6dfc1538f0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f6de4004770 tx=0x7f6dec015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:01.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.211+0000 7f6de37fe700 1 -- 172.21.15.92:0/2489854993 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec007110 con 0x7f6de4003a90 2023-10-06T10:45:01.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.211+0000 7f6de37fe700 1 -- 172.21.15.92:0/2489854993 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dec007270 con 0x7f6de4003a90 2023-10-06T10:45:01.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.211+0000 7f6e01f90700 1 -- 172.21.15.92:0/2489854993 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dfc154ad0 con 0x7f6de4003a90 2023-10-06T10:45:01.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.211+0000 7f6de37fe700 1 -- 172.21.15.92:0/2489854993 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec014630 con 0x7f6de4003a90 2023-10-06T10:45:01.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.211+0000 7f6e01f90700 1 -- 172.21.15.92:0/2489854993 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dfc069b10 con 0x7f6de4003a90 2023-10-06T10:45:01.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.212+0000 7f6e01f90700 1 -- 172.21.15.92:0/2489854993 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dfc064a90 con 0x7f6de4003a90 2023-10-06T10:45:01.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.212+0000 7f6de37fe700 1 -- 172.21.15.92:0/2489854993 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6dec0149e0 con 0x7f6de4003a90 2023-10-06T10:45:01.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.212+0000 7f6de37fe700 1 --2- 172.21.15.92:0/2489854993 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6de8066fb0 0x7f6de8069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:01.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.213+0000 7f6dfaffd700 1 --2- 172.21.15.92:0/2489854993 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6de8066fb0 0x7f6de8069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:01.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.213+0000 7f6de37fe700 1 -- 172.21.15.92:0/2489854993 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6dec086ae0 con 0x7f6de4003a90 2023-10-06T10:45:01.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.213+0000 7f6dfaffd700 1 --2- 172.21.15.92:0/2489854993 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6de8066fb0 0x7f6de8069460 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6df0006dc0 tx=0x7f6df0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:01.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.217+0000 7f6de37fe700 1 -- 172.21.15.92:0/2489854993 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6dec053a10 con 0x7f6de4003a90 2023-10-06T10:45:01.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.443+0000 7f6e01f90700 1 -- 172.21.15.92:0/2489854993 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6dfc064a90 con 0x7f6de4003a90 2023-10-06T10:45:01.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.443+0000 7f6de37fe700 1 -- 172.21.15.92:0/2489854993 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4607 (secure 0 0 0) 0x7f6dec057030 con 0x7f6de4003a90 2023-10-06T10:45:01.444 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:01.445 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":100,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2016052382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2016052382},{"type":"v1","addr":"172.21.15.124:6834","nonce":2016052382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":6719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2307521923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2307521923},{"type":"v1","addr":"172.21.15.124:6835","nonce":2307521923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":8517,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1979826278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979826278},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979826278}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:44:51.106486+0000","modified":"2023-10-06T10:44:54.749834+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2298348426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2298348426},{"type":"v1","addr":"172.21.15.92:6837","nonce":2298348426}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T10:45:01.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.445+0000 7f6e01f90700 1 -- 172.21.15.92:0/2489854993 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6de8066fb0 msgr2=0x7f6de8069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:01.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.445+0000 7f6e01f90700 1 --2- 172.21.15.92:0/2489854993 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6de8066fb0 0x7f6de8069460 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6df0006dc0 tx=0x7f6df0008040 comp rx=0 tx=0).stop 2023-10-06T10:45:01.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.445+0000 7f6e01f90700 1 -- 172.21.15.92:0/2489854993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6de4003a90 msgr2=0x7f6dfc1538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:01.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.445+0000 7f6e01f90700 1 --2- 172.21.15.92:0/2489854993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6de4003a90 0x7f6dfc1538f0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f6de4004770 tx=0x7f6dec015000 comp rx=0 tx=0).stop 2023-10-06T10:45:01.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.445+0000 7f6e01f90700 1 -- 172.21.15.92:0/2489854993 shutdown_connections 2023-10-06T10:45:01.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.445+0000 7f6e01f90700 1 --2- 172.21.15.92:0/2489854993 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6de8066fb0 0x7f6de8069460 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.445+0000 7f6e01f90700 1 --2- 172.21.15.92:0/2489854993 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6dfc0cf2d0 0x7f6dfc0695d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.445+0000 7f6e01f90700 1 --2- 172.21.15.92:0/2489854993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6de4003a90 0x7f6dfc1538f0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.445+0000 7f6e01f90700 1 --2- 172.21.15.92:0/2489854993 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dfc09aa60 0x7f6dfc1552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.445+0000 7f6e01f90700 1 -- 172.21.15.92:0/2489854993 >> 172.21.15.92:0/2489854993 conn(0x7f6dfc0997f0 msgr2=0x7f6dfc0afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:01.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.446+0000 7f6e01f90700 1 -- 172.21.15.92:0/2489854993 shutdown_connections 2023-10-06T10:45:01.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.446+0000 7f6e01f90700 1 -- 172.21.15.92:0/2489854993 wait complete. 2023-10-06T10:45:01.448 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 100 2023-10-06T10:45:01.861 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 100, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T10:44:51.106486+0000', 'modified': '2023-10-06T10:44:54.749834+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8520}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8520': {'gid': 8520, 'name': 'c', 'rank': 0, 'incarnation': 96, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6837/2298348426', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 2298348426}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 2298348426}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T10:45:01.861 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T10:45:01.862 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T10:45:01.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.977+0000 7fb492409700 1 Processor -- start 2023-10-06T10:45:01.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.977+0000 7fb492409700 1 -- start start 2023-10-06T10:45:01.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.978+0000 7fb492409700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb48c0b49a0 0x7fb48c0aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:01.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.978+0000 7fb492409700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb48c0b0760 0x7fb48c0b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:01.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.978+0000 7fb492409700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48c0b0c60 con 0x7fb48c159560 2023-10-06T10:45:01.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.978+0000 7fb492409700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb48c0b0f40 con 0x7fb48c0b0760 2023-10-06T10:45:01.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.978+0000 7fb492409700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0b1220 con 0x7fb48c0b49a0 2023-10-06T10:45:01.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.978+0000 7fb4909a6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb48c0b0760 0x7fb48c0b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:01.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.978+0000 7fb48bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb48c159560 0x7fb48c0aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:43958/0 (socket says 172.21.15.92:43958) 2023-10-06T10:45:01.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.978+0000 7fb4909a6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb48c0b0760 0x7fb48c0b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33904/0 (socket says 172.21.15.92:33904) 2023-10-06T10:45:01.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.978+0000 7fb48bfff700 1 -- 172.21.15.92:0/451918232 learned_addr learned my addr 172.21.15.92:0/451918232 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:01.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.978+0000 7fb48b7fe700 1 --2- 172.21.15.92:0/451918232 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb48c0b49a0 0x7fb48c0aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:01.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.978+0000 7fb4909a6700 1 -- 172.21.15.92:0/451918232 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb48c0b49a0 msgr2=0x7fb48c0aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:01.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.978+0000 7fb4909a6700 1 --2- 172.21.15.92:0/451918232 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb48c0b49a0 0x7fb48c0aac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.978+0000 7fb4909a6700 1 -- 172.21.15.92:0/451918232 >> v1:172.21.15.92:6789/0 conn(0x7fb48c159560 legacy=0x7fb48c0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:01.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.978+0000 7fb4909a6700 1 -- 172.21.15.92:0/451918232 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c0b1500 con 0x7fb48c0b0760 2023-10-06T10:45:01.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.978+0000 7fb48affd700 1 -- 172.21.15.92:0/451918232 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495338300 0 0) 0x7fb48c0b0c60 con 0x7fb48c159560 2023-10-06T10:45:01.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.978+0000 7fb48b7fe700 1 --2- 172.21.15.92:0/451918232 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb48c0b49a0 0x7fb48c0aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:01.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.978+0000 7fb4909a6700 1 --2- 172.21.15.92:0/451918232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb48c0b0760 0x7fb48c0b1bc0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fb484009e60 tx=0x7fb484009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d8c57971b6edcb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:01.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.978+0000 7fb48affd700 1 -- 172.21.15.92:0/451918232 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48400e070 con 0x7fb48c0b0760 2023-10-06T10:45:01.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.978+0000 7fb48affd700 1 -- 172.21.15.92:0/451918232 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48400bc30 con 0x7fb48c0b0760 2023-10-06T10:45:01.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.978+0000 7fb48affd700 1 -- 172.21.15.92:0/451918232 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484012430 con 0x7fb48c0b0760 2023-10-06T10:45:01.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.979+0000 7fb492409700 1 -- 172.21.15.92:0/451918232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb48c0b0760 msgr2=0x7fb48c0b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:01.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.979+0000 7fb492409700 1 --2- 172.21.15.92:0/451918232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb48c0b0760 0x7fb48c0b1bc0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fb484009e60 tx=0x7fb484009780 comp rx=0 tx=0).stop 2023-10-06T10:45:01.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.979+0000 7fb492409700 1 -- 172.21.15.92:0/451918232 shutdown_connections 2023-10-06T10:45:01.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.979+0000 7fb492409700 1 --2- 172.21.15.92:0/451918232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb48c0b0760 0x7fb48c0b1bc0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.979+0000 7fb492409700 1 --2- 172.21.15.92:0/451918232 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb48c0b49a0 0x7fb48c0aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.979+0000 7fb492409700 1 -- 172.21.15.92:0/451918232 >> 172.21.15.92:0/451918232 conn(0x7fb48c0997f0 msgr2=0x7fb48c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:01.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.979+0000 7fb492409700 1 -- 172.21.15.92:0/451918232 shutdown_connections 2023-10-06T10:45:01.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.979+0000 7fb492409700 1 -- 172.21.15.92:0/451918232 wait complete. 2023-10-06T10:45:01.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.980+0000 7fb492409700 1 Processor -- start 2023-10-06T10:45:01.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.980+0000 7fb492409700 1 -- start start 2023-10-06T10:45:01.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.981+0000 7fb492409700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb48c0b0760 0x7fb48c09d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:01.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.981+0000 7fb492409700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb48c0b49a0 0x7fb48c09ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:01.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.981+0000 7fb492409700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb48c159560 0x7fb48c1585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:01.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.981+0000 7fb492409700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c159930 con 0x7fb48c159560 2023-10-06T10:45:01.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.981+0000 7fb492409700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c169c30 con 0x7fb48c0b49a0 2023-10-06T10:45:01.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.981+0000 7fb492409700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c09afd0 con 0x7fb48c0b0760 2023-10-06T10:45:01.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.981+0000 7fb4909a6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb48c159560 0x7fb48c1585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:01.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.981+0000 7fb4909a6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb48c159560 0x7fb48c1585d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33916/0 (socket says 172.21.15.92:33916) 2023-10-06T10:45:01.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.981+0000 7fb4909a6700 1 -- 172.21.15.92:0/1016405276 learned_addr learned my addr 172.21.15.92:0/1016405276 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:01.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.981+0000 7fb48b7fe700 1 --2- 172.21.15.92:0/1016405276 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb48c0b49a0 0x7fb48c09ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:01.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.981+0000 7fb48bfff700 1 --2- 172.21.15.92:0/1016405276 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb48c0b0760 0x7fb48c09d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:01.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.981+0000 7fb4909a6700 1 -- 172.21.15.92:0/1016405276 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb48c0b0760 msgr2=0x7fb48c09d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:01.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.981+0000 7fb4909a6700 1 --2- 172.21.15.92:0/1016405276 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb48c0b0760 0x7fb48c09d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.981+0000 7fb4909a6700 1 -- 172.21.15.92:0/1016405276 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb48c0b49a0 msgr2=0x7fb48c09ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:01.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.981+0000 7fb4909a6700 1 --2- 172.21.15.92:0/1016405276 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb48c0b49a0 0x7fb48c09ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.981+0000 7fb4909a6700 1 -- 172.21.15.92:0/1016405276 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb484009a20 con 0x7fb48c159560 2023-10-06T10:45:01.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.981+0000 7fb4909a6700 1 --2- 172.21.15.92:0/1016405276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb48c159560 0x7fb48c1585d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fb48c067c20 tx=0x7fb484019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:01.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.982+0000 7fb473fff700 1 -- 172.21.15.92:0/1016405276 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484012e50 con 0x7fb48c159560 2023-10-06T10:45:01.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.982+0000 7fb473fff700 1 -- 172.21.15.92:0/1016405276 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48401d630 con 0x7fb48c159560 2023-10-06T10:45:01.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.982+0000 7fb473fff700 1 -- 172.21.15.92:0/1016405276 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48401d9f0 con 0x7fb48c159560 2023-10-06T10:45:01.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.982+0000 7fb492409700 1 -- 172.21.15.92:0/1016405276 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48c09b2b0 con 0x7fb48c159560 2023-10-06T10:45:01.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.982+0000 7fb492409700 1 -- 172.21.15.92:0/1016405276 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48c159e60 con 0x7fb48c159560 2023-10-06T10:45:01.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.983+0000 7fb473fff700 1 -- 172.21.15.92:0/1016405276 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb484024070 con 0x7fb48c159560 2023-10-06T10:45:01.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.983+0000 7fb473fff700 1 --2- 172.21.15.92:0/1016405276 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb478066f30 0x7fb4780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:01.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.984+0000 7fb473fff700 1 -- 172.21.15.92:0/1016405276 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb484086fa0 con 0x7fb48c159560 2023-10-06T10:45:01.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.984+0000 7fb48bfff700 1 --2- 172.21.15.92:0/1016405276 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb478066f30 0x7fb4780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:01.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.984+0000 7fb48bfff700 1 --2- 172.21.15.92:0/1016405276 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb478066f30 0x7fb4780693e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb47c001730 tx=0x7fb47c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:01.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.985+0000 7fb49b701b80 1 Processor -- start 2023-10-06T10:45:01.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.985+0000 7fb49b701b80 1 -- start start 2023-10-06T10:45:01.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.986+0000 7fb49b701b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5625632a3a90 0x5625632a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:01.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.986+0000 7fb49b701b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56256325f9a0 0x56256325c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:01.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.986+0000 7fb49b701b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56256325dbd0 0x56256325cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:01.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.986+0000 7fb49b701b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625631dcc80 con 0x56256325f9a0 2023-10-06T10:45:01.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.986+0000 7fb49b701b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625631b3a70 con 0x56256325dbd0 2023-10-06T10:45:01.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.986+0000 7fb49b701b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5625631b1270 con 0x5625632a3a90 2023-10-06T10:45:01.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.986+0000 7fb48b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56256325f9a0 0x56256325c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:01.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.986+0000 7fb48b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56256325f9a0 0x56256325c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33926/0 (socket says 172.21.15.92:33926) 2023-10-06T10:45:01.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.986+0000 7fb48b7fe700 1 -- 172.21.15.92:0/2467485092 learned_addr learned my addr 172.21.15.92:0/2467485092 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:01.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.987+0000 7fb48bfff700 1 --2- 172.21.15.92:0/2467485092 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5625632a3a90 0x5625632a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:01.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.987+0000 7fb4909a6700 1 --2- 172.21.15.92:0/2467485092 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56256325dbd0 0x56256325cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:01.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.987+0000 7fb48b7fe700 1 -- 172.21.15.92:0/2467485092 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5625632a3a90 msgr2=0x5625632a3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:01.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.987+0000 7fb48b7fe700 1 --2- 172.21.15.92:0/2467485092 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5625632a3a90 0x5625632a3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.987+0000 7fb48b7fe700 1 -- 172.21.15.92:0/2467485092 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56256325dbd0 msgr2=0x56256325cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:01.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.987+0000 7fb48b7fe700 1 --2- 172.21.15.92:0/2467485092 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56256325dbd0 0x56256325cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.987+0000 7fb48b7fe700 1 -- 172.21.15.92:0/2467485092 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562563047270 con 0x56256325f9a0 2023-10-06T10:45:01.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.987+0000 7fb4909a6700 1 --2- 172.21.15.92:0/2467485092 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56256325dbd0 0x56256325cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:01.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.987+0000 7fb48b7fe700 1 --2- 172.21.15.92:0/2467485092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56256325f9a0 0x56256325c580 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fb480009960 tx=0x7fb480008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac834321095ffc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:01.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.987+0000 7fb4717fa700 1 -- 172.21.15.92:0/2467485092 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48000f6d0 con 0x56256325f9a0 2023-10-06T10:45:01.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.987+0000 7fb4717fa700 1 -- 172.21.15.92:0/2467485092 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48000c030 con 0x56256325f9a0 2023-10-06T10:45:01.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.987+0000 7fb4717fa700 1 -- 172.21.15.92:0/2467485092 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48000fa40 con 0x56256325f9a0 2023-10-06T10:45:01.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.988+0000 7fb49b701b80 1 -- 172.21.15.92:0/2467485092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56256325f9a0 msgr2=0x56256325c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:01.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.988+0000 7fb49b701b80 1 --2- 172.21.15.92:0/2467485092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56256325f9a0 0x56256325c580 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fb480009960 tx=0x7fb480008040 comp rx=0 tx=0).stop 2023-10-06T10:45:01.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.988+0000 7fb49b701b80 1 -- 172.21.15.92:0/2467485092 shutdown_connections 2023-10-06T10:45:01.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.988+0000 7fb49b701b80 1 --2- 172.21.15.92:0/2467485092 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56256325dbd0 0x56256325cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.988+0000 7fb49b701b80 1 --2- 172.21.15.92:0/2467485092 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5625632a3a90 0x5625632a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.988+0000 7fb49b701b80 1 --2- 172.21.15.92:0/2467485092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56256325f9a0 0x56256325c580 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.988+0000 7fb49b701b80 1 -- 172.21.15.92:0/2467485092 >> 172.21.15.92:0/2467485092 conn(0x56256325e070 msgr2=0x56256325e440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:01.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.988+0000 7fb49b701b80 1 -- 172.21.15.92:0/2467485092 shutdown_connections 2023-10-06T10:45:01.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.988+0000 7fb49b701b80 1 -- 172.21.15.92:0/2467485092 wait complete. 2023-10-06T10:45:01.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.988+0000 7fb49b701b80 1 -- start start 2023-10-06T10:45:01.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.988+0000 7fb49b701b80 1 Processor -- start 2023-10-06T10:45:01.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.989+0000 7fb49b701b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56256325dbd0 0x562563282c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:01.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.989+0000 7fb49b701b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56256325f9a0 0x56256329f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:01.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.989+0000 7fb49b701b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625632a3a90 0x562563286670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:01.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.989+0000 7fb49b701b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625631ad060 con 0x5625632a3a90 2023-10-06T10:45:01.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.989+0000 7fb49b701b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562563158890 con 0x56256325dbd0 2023-10-06T10:45:01.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.989+0000 7fb49b701b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562563173ea0 con 0x56256325f9a0 2023-10-06T10:45:01.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.989+0000 7fb49b701b80 10 client.0 _ll_register_callbacks cb 0x562563049f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:01.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.989+0000 7fb4909a6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625632a3a90 0x562563286670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:01.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.989+0000 7fb4909a6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625632a3a90 0x562563286670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33936/0 (socket says 172.21.15.92:33936) 2023-10-06T10:45:01.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.989+0000 7fb4909a6700 1 -- 172.21.15.92:0/2949840865 learned_addr learned my addr 172.21.15.92:0/2949840865 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:01.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.989+0000 7fb48bfff700 1 --2- 172.21.15.92:0/2949840865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56256325dbd0 0x562563282c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:01.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.989+0000 7fb48b7fe700 1 --2- 172.21.15.92:0/2949840865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56256325f9a0 0x56256329f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:01.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb4909a6700 1 -- 172.21.15.92:0/2949840865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56256325f9a0 msgr2=0x56256329f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:01.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb4909a6700 1 --2- 172.21.15.92:0/2949840865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56256325f9a0 0x56256329f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb4909a6700 1 -- 172.21.15.92:0/2949840865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56256325dbd0 msgr2=0x562563282c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:01.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb4909a6700 1 --2- 172.21.15.92:0/2949840865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56256325dbd0 0x562563282c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:01.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb4909a6700 1 -- 172.21.15.92:0/2949840865 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48000e040 con 0x5625632a3a90 2023-10-06T10:45:01.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb48bfff700 1 --2- 172.21.15.92:0/2949840865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56256325dbd0 0x562563282c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:01.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb4909a6700 1 --2- 172.21.15.92:0/2949840865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625632a3a90 0x562563286670 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fb484012580 tx=0x7fb484014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f56057833abb722 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:01.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb46a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:01.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb46a7fc700 1 -- 172.21.15.92:0/2949840865 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48400e030 con 0x5625632a3a90 2023-10-06T10:45:01.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb46a7fc700 1 -- 172.21.15.92:0/2949840865 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb48405ebd0 con 0x5625632a3a90 2023-10-06T10:45:01.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb49b701b80 1 -- 172.21.15.92:0/2949840865 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562563251690 con 0x5625632a3a90 2023-10-06T10:45:01.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb46a7fc700 1 -- 172.21.15.92:0/2949840865 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb48405e740 con 0x5625632a3a90 2023-10-06T10:45:01.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb49b701b80 10 client.8700 fetch_fsmap learned FSMap version 100 2023-10-06T10:45:01.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb49b701b80 1 -- 172.21.15.92:0/2949840865 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) v3 -- 0x562562fe9fe0 con 0x5625632a3a90 2023-10-06T10:45:01.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb46a7fc700 1 -- 172.21.15.92:0/2949840865 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562563251690 con 0x5625632a3a90 2023-10-06T10:45:01.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb46a7fc700 1 -- 172.21.15.92:0/2949840865 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 100) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fb48405bbc0 con 0x5625632a3a90 2023-10-06T10:45:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb49b701b80 10 client.8700 fetch_fsmap finished waiting for FSMap version 100 2023-10-06T10:45:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb49b701b80 10 client.8700 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:45:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb49b701b80 20 client.8700 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb49b701b80 1 --2- 172.21.15.92:0/2949840865 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x5625632805f0 0x5625632809c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb49b701b80 4 client.8700 mds_command: new command op to 8520 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb49b701b80 1 -- 172.21.15.92:0/2949840865 --> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56256322e5c0 con 0x5625632805f0 2023-10-06T10:45:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.990+0000 7fb48bfff700 1 --2- 172.21.15.92:0/2949840865 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x5625632805f0 0x5625632809c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.991+0000 7fb46a7fc700 10 client.8700 ms_handle_connect on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.991+0000 7fb48bfff700 1 --2- 172.21.15.92:0/2949840865 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x5625632805f0 0x5625632809c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e07ab073abdbe21 server_cookie=5b31bc924fe0ddfd in_seq=0 out_seq=0 2023-10-06T10:45:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.991+0000 7fb46a7fc700 1 -- 172.21.15.92:0/2949840865 <== mds.0 v2:172.21.15.92:6835/2298348426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x56256322e5c0 con 0x5625632805f0 2023-10-06T10:45:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.991+0000 7fb46a7fc700 10 client.8700 handle_command_reply: tid=0 2023-10-06T10:45:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.991+0000 7fb49b701b80 1 client.8700 shutdown 2023-10-06T10:45:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.991+0000 7fb49b701b80 1 -- 172.21.15.92:0/2949840865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625632a3a90 msgr2=0x562563286670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.991+0000 7fb49b701b80 1 --2- 172.21.15.92:0/2949840865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625632a3a90 0x562563286670 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fb484012580 tx=0x7fb484014040 comp rx=0 tx=0).stop 2023-10-06T10:45:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.991+0000 7fb49b701b80 1 -- 172.21.15.92:0/2949840865 shutdown_connections 2023-10-06T10:45:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.991+0000 7fb49b701b80 1 --2- 172.21.15.92:0/2949840865 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x5625632805f0 0x5625632809c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.991+0000 7fb49b701b80 1 --2- 172.21.15.92:0/2949840865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625632a3a90 0x562563286670 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.991+0000 7fb49b701b80 1 --2- 172.21.15.92:0/2949840865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56256325dbd0 0x562563282c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.991+0000 7fb46a7fc700 0 client.8700 ms_handle_reset on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.991+0000 7fb49b701b80 1 --2- 172.21.15.92:0/2949840865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56256325f9a0 0x56256329f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.991+0000 7fb49b701b80 1 -- 172.21.15.92:0/2949840865 >> 172.21.15.92:0/2949840865 conn(0x56256325e070 msgr2=0x5625632a2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.992+0000 7fb49b701b80 1 -- 172.21.15.92:0/2949840865 shutdown_connections 2023-10-06T10:45:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.992+0000 7fb49b701b80 1 -- 172.21.15.92:0/2949840865 wait complete. 2023-10-06T10:45:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:01.992+0000 7fb49b701b80 20 client.8700 trim_cache size 0 max 16384 2023-10-06T10:45:02.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.008+0000 7fb49b701b80 1 Processor -- start 2023-10-06T10:45:02.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.008+0000 7fb49b701b80 1 -- start start 2023-10-06T10:45:02.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.009+0000 7fb49b701b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56256325dbd0 0x5625632b5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:02.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.009+0000 7fb49b701b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56256325f9a0 0x56256325e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:02.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.009+0000 7fb49b701b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5625632a3a90 0x56256325e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:02.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.009+0000 7fb49b701b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256302fe30 con 0x56256325f9a0 2023-10-06T10:45:02.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.009+0000 7fb49b701b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256329a7c0 con 0x56256325dbd0 2023-10-06T10:45:02.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.009+0000 7fb49b701b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56256325f800 con 0x5625632a3a90 2023-10-06T10:45:02.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.009+0000 7fb48b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56256325f9a0 0x56256325e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:02.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.009+0000 7fb48b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56256325f9a0 0x56256325e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33950/0 (socket says 172.21.15.92:33950) 2023-10-06T10:45:02.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.009+0000 7fb48b7fe700 1 -- 172.21.15.92:0/4199585846 learned_addr learned my addr 172.21.15.92:0/4199585846 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:02.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.009+0000 7fb48bfff700 1 --2- 172.21.15.92:0/4199585846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56256325dbd0 0x5625632b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:02.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.009+0000 7fb4909a6700 1 --2- 172.21.15.92:0/4199585846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5625632a3a90 0x56256325e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:02.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.009+0000 7fb48b7fe700 1 -- 172.21.15.92:0/4199585846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5625632a3a90 msgr2=0x56256325e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:02.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.009+0000 7fb48b7fe700 1 --2- 172.21.15.92:0/4199585846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5625632a3a90 0x56256325e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.009+0000 7fb48b7fe700 1 -- 172.21.15.92:0/4199585846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56256325dbd0 msgr2=0x5625632b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:02.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.009+0000 7fb48b7fe700 1 --2- 172.21.15.92:0/4199585846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56256325dbd0 0x5625632b5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.009+0000 7fb48b7fe700 1 -- 172.21.15.92:0/4199585846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56256317b780 con 0x56256325f9a0 2023-10-06T10:45:02.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.010+0000 7fb48b7fe700 1 --2- 172.21.15.92:0/4199585846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56256325f9a0 0x56256325e440 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fb480006fd0 tx=0x7fb48001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2274dd3ec2ff9635 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:02.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.010+0000 7fb471ffb700 1 -- 172.21.15.92:0/4199585846 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480024070 con 0x56256325f9a0 2023-10-06T10:45:02.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.010+0000 7fb471ffb700 1 -- 172.21.15.92:0/4199585846 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48000fe30 con 0x56256325f9a0 2023-10-06T10:45:02.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.010+0000 7fb471ffb700 1 -- 172.21.15.92:0/4199585846 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48000f6d0 con 0x56256325f9a0 2023-10-06T10:45:02.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.010+0000 7fb49b701b80 1 -- 172.21.15.92:0/4199585846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56256325f9a0 msgr2=0x56256325e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:02.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.010+0000 7fb49b701b80 1 --2- 172.21.15.92:0/4199585846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56256325f9a0 0x56256325e440 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fb480006fd0 tx=0x7fb48001c000 comp rx=0 tx=0).stop 2023-10-06T10:45:02.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.010+0000 7fb49b701b80 1 -- 172.21.15.92:0/4199585846 shutdown_connections 2023-10-06T10:45:02.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.010+0000 7fb49b701b80 1 --2- 172.21.15.92:0/4199585846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5625632a3a90 0x56256325e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.010+0000 7fb49b701b80 1 --2- 172.21.15.92:0/4199585846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56256325dbd0 0x5625632b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.010+0000 7fb49b701b80 1 --2- 172.21.15.92:0/4199585846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56256325f9a0 0x56256325e440 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.010+0000 7fb49b701b80 1 -- 172.21.15.92:0/4199585846 >> 172.21.15.92:0/4199585846 conn(0x56256325e070 msgr2=0x562563284270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:02.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.010+0000 7fb49b701b80 1 -- 172.21.15.92:0/4199585846 shutdown_connections 2023-10-06T10:45:02.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.010+0000 7fb49b701b80 1 -- 172.21.15.92:0/4199585846 wait complete. 2023-10-06T10:45:02.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.011+0000 7fb49b701b80 1 -- start start 2023-10-06T10:45:02.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.011+0000 7fb49b701b80 1 Processor -- start 2023-10-06T10:45:02.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb49b701b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56256325dbd0 0x562563307db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:02.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb49b701b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56256325f9a0 0x56256328ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:02.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb49b701b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5625632a3a90 0x56256328bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:02.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb49b701b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256329de60 con 0x56256325dbd0 2023-10-06T10:45:02.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb49b701b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256327e9d0 con 0x5625632a3a90 2023-10-06T10:45:02.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb49b701b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5625632b4630 con 0x56256325f9a0 2023-10-06T10:45:02.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb49b701b80 10 client.0 _ll_register_callbacks cb 0x562563049f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:02.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb48bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56256325dbd0 0x562563307db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:02.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb48bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56256325dbd0 0x562563307db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33952/0 (socket says 172.21.15.92:33952) 2023-10-06T10:45:02.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb48bfff700 1 -- 172.21.15.92:0/4010140662 learned_addr learned my addr 172.21.15.92:0/4010140662 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:02.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb48b7fe700 1 --2- 172.21.15.92:0/4010140662 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56256325f9a0 0x56256328ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:02.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb4909a6700 1 --2- 172.21.15.92:0/4010140662 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5625632a3a90 0x56256328bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:02.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb48bfff700 1 -- 172.21.15.92:0/4010140662 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56256325f9a0 msgr2=0x56256328ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:02.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb48bfff700 1 --2- 172.21.15.92:0/4010140662 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56256325f9a0 0x56256328ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb48bfff700 1 -- 172.21.15.92:0/4010140662 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5625632a3a90 msgr2=0x56256328bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:02.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb48bfff700 1 --2- 172.21.15.92:0/4010140662 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5625632a3a90 0x56256328bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb48bfff700 1 -- 172.21.15.92:0/4010140662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562563047270 con 0x56256325dbd0 2023-10-06T10:45:02.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb48b7fe700 1 --2- 172.21.15.92:0/4010140662 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56256325f9a0 0x56256328ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:02.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb4909a6700 1 --2- 172.21.15.92:0/4010140662 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5625632a3a90 0x56256328bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:02.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb48bfff700 1 --2- 172.21.15.92:0/4010140662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56256325dbd0 0x562563307db0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x562563281e40 tx=0x7fb47c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f4484051974f38 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:02.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb46a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:02.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb46a7fc700 1 -- 172.21.15.92:0/4010140662 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c029070 con 0x56256325dbd0 2023-10-06T10:45:02.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb46a7fc700 1 -- 172.21.15.92:0/4010140662 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb47c024750 con 0x56256325dbd0 2023-10-06T10:45:02.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb49b701b80 1 -- 172.21.15.92:0/4010140662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb48000c030 con 0x56256325dbd0 2023-10-06T10:45:02.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb46a7fc700 1 -- 172.21.15.92:0/4010140662 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb47c024b20 con 0x56256325dbd0 2023-10-06T10:45:02.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb49b701b80 10 client.8706 fetch_fsmap learned FSMap version 100 2023-10-06T10:45:02.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.012+0000 7fb49b701b80 1 -- 172.21.15.92:0/4010140662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) v3 -- 0x562563162470 con 0x56256325dbd0 2023-10-06T10:45:02.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb46a7fc700 1 -- 172.21.15.92:0/4010140662 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48000c030 con 0x56256325dbd0 2023-10-06T10:45:02.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb46a7fc700 1 -- 172.21.15.92:0/4010140662 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 100) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fb47c024c80 con 0x56256325dbd0 2023-10-06T10:45:02.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb49b701b80 10 client.8706 fetch_fsmap finished waiting for FSMap version 100 2023-10-06T10:45:02.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb49b701b80 10 client.8706 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:45:02.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb49b701b80 20 client.8706 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:02.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb49b701b80 1 --2- 172.21.15.92:0/4010140662 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x56256325cef0 0x56256325d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:02.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb49b701b80 4 client.8706 mds_command: new command op to 8520 tid=0[{"prefix": "status"}] 2023-10-06T10:45:02.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb49b701b80 1 -- 172.21.15.92:0/4010140662 --> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56256314fe00 con 0x56256325cef0 2023-10-06T10:45:02.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb48b7fe700 1 --2- 172.21.15.92:0/4010140662 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x56256325cef0 0x56256325d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:02.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb46a7fc700 10 client.8706 ms_handle_connect on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:02.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb48b7fe700 1 --2- 172.21.15.92:0/4010140662 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x56256325cef0 0x56256325d2c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c23cc5a53edbcf4 server_cookie=7aec498df1f2b361 in_seq=0 out_seq=0 2023-10-06T10:45:02.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb46a7fc700 1 -- 172.21.15.92:0/4010140662 <== mds.0 v2:172.21.15.92:6835/2298348426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x56256314fe00 con 0x56256325cef0 2023-10-06T10:45:02.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb46a7fc700 10 client.8706 handle_command_reply: tid=0 2023-10-06T10:45:02.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb49b701b80 1 client.8706 shutdown 2023-10-06T10:45:02.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb49b701b80 1 -- 172.21.15.92:0/4010140662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56256325dbd0 msgr2=0x562563307db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:02.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb49b701b80 1 --2- 172.21.15.92:0/4010140662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56256325dbd0 0x562563307db0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x562563281e40 tx=0x7fb47c014040 comp rx=0 tx=0).stop 2023-10-06T10:45:02.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb49b701b80 1 -- 172.21.15.92:0/4010140662 shutdown_connections 2023-10-06T10:45:02.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb49b701b80 1 --2- 172.21.15.92:0/4010140662 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x56256325cef0 0x56256325d2c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb49b701b80 1 --2- 172.21.15.92:0/4010140662 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5625632a3a90 0x56256328bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb46a7fc700 0 client.8706 ms_handle_reset on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb49b701b80 1 --2- 172.21.15.92:0/4010140662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56256325dbd0 0x562563307db0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb49b701b80 1 --2- 172.21.15.92:0/4010140662 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56256325f9a0 0x56256328ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.013+0000 7fb49b701b80 1 -- 172.21.15.92:0/4010140662 >> 172.21.15.92:0/4010140662 conn(0x56256325e070 msgr2=0x56256325e440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.014+0000 7fb49b701b80 1 -- 172.21.15.92:0/4010140662 shutdown_connections 2023-10-06T10:45:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.014+0000 7fb49b701b80 1 -- 172.21.15.92:0/4010140662 wait complete. 2023-10-06T10:45:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.014+0000 7fb49b701b80 20 client.8706 trim_cache size 0 max 16384 2023-10-06T10:45:02.022 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:02.022 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:45:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T10:45:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 8520, 2023-10-06T10:45:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:45:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:45:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:45:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 10.291831102, 2023-10-06T10:45:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 100, 2023-10-06T10:45:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 102, 2023-10-06T10:45:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 102, 2023-10-06T10:45:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 16.410478006999998 2023-10-06T10:45:02.023 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:02.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.014+0000 7fb492409700 1 -- 172.21.15.92:0/1016405276 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb478066f30 msgr2=0x7fb4780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:02.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.014+0000 7fb492409700 1 --2- 172.21.15.92:0/1016405276 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb478066f30 0x7fb4780693e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb47c001730 tx=0x7fb47c009040 comp rx=0 tx=0).stop 2023-10-06T10:45:02.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.014+0000 7fb492409700 1 -- 172.21.15.92:0/1016405276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb48c159560 msgr2=0x7fb48c1585d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:02.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.014+0000 7fb492409700 1 --2- 172.21.15.92:0/1016405276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb48c159560 0x7fb48c1585d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fb48c067c20 tx=0x7fb484019000 comp rx=0 tx=0).stop 2023-10-06T10:45:02.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.014+0000 7fb492409700 1 -- 172.21.15.92:0/1016405276 shutdown_connections 2023-10-06T10:45:02.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.014+0000 7fb492409700 1 --2- 172.21.15.92:0/1016405276 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb478066f30 0x7fb4780693e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.014+0000 7fb492409700 1 --2- 172.21.15.92:0/1016405276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb48c159560 0x7fb48c1585d0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.014+0000 7fb492409700 1 --2- 172.21.15.92:0/1016405276 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb48c0b0760 0x7fb48c09d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.014+0000 7fb492409700 1 --2- 172.21.15.92:0/1016405276 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb48c0b49a0 0x7fb48c09ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.014+0000 7fb492409700 1 -- 172.21.15.92:0/1016405276 >> 172.21.15.92:0/1016405276 conn(0x7fb48c0997f0 msgr2=0x7fb48c068d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:02.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.014+0000 7fb492409700 1 -- 172.21.15.92:0/1016405276 shutdown_connections 2023-10-06T10:45:02.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.014+0000 7fb492409700 1 -- 172.21.15.92:0/1016405276 wait complete. 2023-10-06T10:45:02.027 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T10:45:02.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.140+0000 7f658759e700 1 Processor -- start 2023-10-06T10:45:02.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.140+0000 7f658759e700 1 -- start start 2023-10-06T10:45:02.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.141+0000 7f658759e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6588157410 0x7f65880cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:02.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.141+0000 7f658759e700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f658815b660 0x7f65880c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:02.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.141+0000 7f658759e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f658815b1d0 con 0x7f6588157040 2023-10-06T10:45:02.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.141+0000 7f658759e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65880ce4b0 con 0x7f6588157410 2023-10-06T10:45:02.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.141+0000 7f658759e700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65880ce790 con 0x7f658815b660 2023-10-06T10:45:02.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.141+0000 7f6585d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6588157410 0x7f65880cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:02.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.141+0000 7f6585d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6588157410 0x7f65880cde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33956/0 (socket says 172.21.15.92:33956) 2023-10-06T10:45:02.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.141+0000 7f6585d9b700 1 -- 172.21.15.92:0/3595015526 learned_addr learned my addr 172.21.15.92:0/3595015526 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:02.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.141+0000 7f6586d9d700 1 --2- 172.21.15.92:0/3595015526 >> v2:172.21.15.124:3300/0 conn(0x7f658815b660 0x7f65880c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:02.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.141+0000 7f6585d9b700 1 -- 172.21.15.92:0/3595015526 >> v2:172.21.15.124:3300/0 conn(0x7f658815b660 msgr2=0x7f65880c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:02.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.141+0000 7f6585d9b700 1 --2- 172.21.15.92:0/3595015526 >> v2:172.21.15.124:3300/0 conn(0x7f658815b660 0x7f65880c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.141+0000 7f6585d9b700 1 -- 172.21.15.92:0/3595015526 >> v1:172.21.15.124:6789/0 conn(0x7f6588157040 legacy=0x7f658806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:02.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.141+0000 7f6585d9b700 1 -- 172.21.15.92:0/3595015526 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658815ae60 con 0x7f6588157410 2023-10-06T10:45:02.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.141+0000 7f6585d9b700 1 --2- 172.21.15.92:0/3595015526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6588157410 0x7f65880cde60 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f658815b140 tx=0x7f6570009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657e28e1e564e608 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:02.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.142+0000 7f658559a700 1 -- 172.21.15.92:0/3595015526 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570009e70 con 0x7f6588157410 2023-10-06T10:45:02.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.142+0000 7f658559a700 1 -- 172.21.15.92:0/3595015526 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657000b860 con 0x7f6588157410 2023-10-06T10:45:02.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.142+0000 7f658559a700 1 -- 172.21.15.92:0/3595015526 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657000bc00 con 0x7f6588157410 2023-10-06T10:45:02.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.142+0000 7f658759e700 1 -- 172.21.15.92:0/3595015526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6588157410 msgr2=0x7f65880cde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:02.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.142+0000 7f658759e700 1 --2- 172.21.15.92:0/3595015526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6588157410 0x7f65880cde60 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f658815b140 tx=0x7f6570009250 comp rx=0 tx=0).stop 2023-10-06T10:45:02.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.142+0000 7f658759e700 1 -- 172.21.15.92:0/3595015526 shutdown_connections 2023-10-06T10:45:02.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.142+0000 7f658759e700 1 --2- 172.21.15.92:0/3595015526 >> v2:172.21.15.124:3300/0 conn(0x7f658815b660 0x7f65880c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.142+0000 7f658759e700 1 --2- 172.21.15.92:0/3595015526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6588157410 0x7f65880cde60 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.142+0000 7f658759e700 1 -- 172.21.15.92:0/3595015526 >> 172.21.15.92:0/3595015526 conn(0x7f658806b500 msgr2=0x7f658806b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:02.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.142+0000 7f658759e700 1 -- 172.21.15.92:0/3595015526 shutdown_connections 2023-10-06T10:45:02.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.142+0000 7f658759e700 1 -- 172.21.15.92:0/3595015526 wait complete. 2023-10-06T10:45:02.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.143+0000 7f658759e700 1 Processor -- start 2023-10-06T10:45:02.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.143+0000 7f658759e700 1 -- start start 2023-10-06T10:45:02.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.144+0000 7f658759e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6588157040 0x7f6588156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:02.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.144+0000 7f658759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6588157410 0x7f65880c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:02.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.144+0000 7f658759e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f658815b660 0x7f658814ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:02.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.144+0000 7f658759e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65881567a0 con 0x7f6588157410 2023-10-06T10:45:02.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.144+0000 7f658759e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65880c72a0 con 0x7f658815b660 2023-10-06T10:45:02.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.144+0000 7f658759e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65880c7e20 con 0x7f6588157040 2023-10-06T10:45:02.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.144+0000 7f6585d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6588157410 0x7f65880c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:02.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.144+0000 7f6585d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6588157410 0x7f65880c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33960/0 (socket says 172.21.15.92:33960) 2023-10-06T10:45:02.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.144+0000 7f6585d9b700 1 -- 172.21.15.92:0/661781906 learned_addr learned my addr 172.21.15.92:0/661781906 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:02.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.144+0000 7f658659c700 1 --2- 172.21.15.92:0/661781906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6588157040 0x7f6588156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:02.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.144+0000 7f6586d9d700 1 --2- 172.21.15.92:0/661781906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f658815b660 0x7f658814ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:02.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.144+0000 7f6585d9b700 1 -- 172.21.15.92:0/661781906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6588157040 msgr2=0x7f6588156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:02.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.144+0000 7f6585d9b700 1 --2- 172.21.15.92:0/661781906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6588157040 0x7f6588156260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.144+0000 7f6585d9b700 1 -- 172.21.15.92:0/661781906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f658815b660 msgr2=0x7f658814ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:02.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.144+0000 7f6585d9b700 1 --2- 172.21.15.92:0/661781906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f658815b660 0x7f658814ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.144+0000 7f6585d9b700 1 -- 172.21.15.92:0/661781906 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65700094f0 con 0x7f6588157410 2023-10-06T10:45:02.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.145+0000 7f6585d9b700 1 --2- 172.21.15.92:0/661781906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6588157410 0x7f65880c82b0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f6570006fd0 tx=0x7f6570017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:02.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.145+0000 7f657e7fc700 1 -- 172.21.15.92:0/661781906 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657000bbe0 con 0x7f6588157410 2023-10-06T10:45:02.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.145+0000 7f657e7fc700 1 -- 172.21.15.92:0/661781906 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6570011420 con 0x7f6588157410 2023-10-06T10:45:02.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.145+0000 7f658759e700 1 -- 172.21.15.92:0/661781906 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6588069cf0 con 0x7f6588157410 2023-10-06T10:45:02.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.145+0000 7f657e7fc700 1 -- 172.21.15.92:0/661781906 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65700117e0 con 0x7f6588157410 2023-10-06T10:45:02.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.145+0000 7f658759e700 1 -- 172.21.15.92:0/661781906 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65880c74f0 con 0x7f6588157410 2023-10-06T10:45:02.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.146+0000 7f658759e700 1 -- 172.21.15.92:0/661781906 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65880c7fb0 con 0x7f6588157410 2023-10-06T10:45:02.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.146+0000 7f657e7fc700 1 -- 172.21.15.92:0/661781906 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6570011c10 con 0x7f6588157410 2023-10-06T10:45:02.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.146+0000 7f657e7fc700 1 --2- 172.21.15.92:0/661781906 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6568066f30 0x7f65680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:02.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.146+0000 7f657e7fc700 1 -- 172.21.15.92:0/661781906 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6570085f90 con 0x7f6588157410 2023-10-06T10:45:02.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.146+0000 7f658659c700 1 --2- 172.21.15.92:0/661781906 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6568066f30 0x7f65680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:02.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.147+0000 7f658659c700 1 --2- 172.21.15.92:0/661781906 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6568066f30 0x7f65680693e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6580007fd0 tx=0x7f65800016f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:02.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.151+0000 7f657e7fc700 1 -- 172.21.15.92:0/661781906 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f65700b75a0 con 0x7f6588157410 2023-10-06T10:45:02.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.377+0000 7f658759e700 1 -- 172.21.15.92:0/661781906 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f658805f230 con 0x7f6588157410 2023-10-06T10:45:02.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.377+0000 7f657e7fc700 1 -- 172.21.15.92:0/661781906 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6570015060 con 0x7f6588157410 2023-10-06T10:45:02.378 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T10:45:02.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.379+0000 7f658759e700 1 -- 172.21.15.92:0/661781906 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6568066f30 msgr2=0x7f65680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:02.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.379+0000 7f658759e700 1 --2- 172.21.15.92:0/661781906 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6568066f30 0x7f65680693e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6580007fd0 tx=0x7f65800016f0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.379+0000 7f658759e700 1 -- 172.21.15.92:0/661781906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6588157410 msgr2=0x7f65880c82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:02.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.379+0000 7f658759e700 1 --2- 172.21.15.92:0/661781906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6588157410 0x7f65880c82b0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f6570006fd0 tx=0x7f6570017000 comp rx=0 tx=0).stop 2023-10-06T10:45:02.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.379+0000 7f658759e700 1 -- 172.21.15.92:0/661781906 shutdown_connections 2023-10-06T10:45:02.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.379+0000 7f658759e700 1 --2- 172.21.15.92:0/661781906 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6568066f30 0x7f65680693e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.379+0000 7f658759e700 1 --2- 172.21.15.92:0/661781906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f658815b660 0x7f658814ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.379+0000 7f658759e700 1 --2- 172.21.15.92:0/661781906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6588157040 0x7f6588156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.379+0000 7f658759e700 1 --2- 172.21.15.92:0/661781906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6588157410 0x7f65880c82b0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:02.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.379+0000 7f658759e700 1 -- 172.21.15.92:0/661781906 >> 172.21.15.92:0/661781906 conn(0x7f658806b500 msgr2=0x7f65880b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:02.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.380+0000 7f658759e700 1 -- 172.21.15.92:0/661781906 shutdown_connections 2023-10-06T10:45:02.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:02.380+0000 7f658759e700 1 -- 172.21.15.92:0/661781906 wait complete. 2023-10-06T10:45:02.392 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T10:45:02.392 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T10:45:02.392 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T10:45:02.392 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T10:45:02.392 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T10:45:02.392 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T10:45:02.392 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T10:45:02.392 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T10:45:02.393 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T10:45:02.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:45:02.393 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T10:45:02.407 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T10:45:02.408 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T10:45:02.408 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T10:45:02.408 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:45:02.408 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T10:45:02.408 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:45:02.408 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:45:02.408 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:45:02.408 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T10:45:02.408 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:45:02.408 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:45:02.409 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T10:45:02.409 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T10:45:02.409 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:45:02.409 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T10:45:02.409 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T10:45:02.409 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T10:45:02.409 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42107sec preferred_lft 42107sec 2023-10-06T10:45:02.409 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T10:45:02.409 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:45:02.409 INFO:teuthology.orchestra.run.smithi092.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T10:45:02.409 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 92:7f:88:24:4e:ca brd ff:ff:ff:ff:ff:ff 2023-10-06T10:45:02.409 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T10:45:02.410 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:45:02.410 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::9c9d:23ff:fee4:30ae/64 scope link 2023-10-06T10:45:02.410 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:45:02.410 INFO:teuthology.orchestra.run.smithi092.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T10:45:02.410 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 92:7f:88:24:4e:ca brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T10:45:02.410 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::907f:88ff:fe24:4eca/64 scope link 2023-10-06T10:45:02.410 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:45:02.410 INFO:teuthology.orchestra.run.smithi092.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T10:45:02.410 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether b6:32:f9:d9:96:1b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T10:45:02.410 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::b432:f9ff:fed9:961b/64 scope link 2023-10-06T10:45:02.410 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T10:45:02.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:45:02.411 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T10:45:02.470 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T10:45:02.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:45:02.472 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T10:45:02.527 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T10:45:02.528 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T10:45:02.528 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T10:45:02.528 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T10:45:02.528 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T10:45:02.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:45:02.528 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T10:45:02.528 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T10:45:02.528 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T10:45:02.528 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T10:45:02.529 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T10:45:02.529 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T10:45:02.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:45:02.627 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T10:45:02.641 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T10:45:02.642 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T10:45:02.642 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T10:45:02.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:45:02.642 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-10-06T10:45:02.706 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T10:45:02.706 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T10:45:02.761 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T10:45:02.761 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T10:45:02.761 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T10:45:02.815 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T10:45:02.816 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T10:45:02.816 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:45:02.895 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:45:02.895 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:45:02.914 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:45:02.914 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:45:02.915 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-06T10:45:02.915 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@8f307bfb-c4cb-4f66-aade-46fffe0c6e0c.cephfs=/ 2023-10-06T10:45:02.915 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-06T10:45:02.915 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T10:45:02.915 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T10:45:02.915 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T10:45:02.935 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T10:45:02.935 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T10:45:02.935 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-06T10:45:02.936 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@8f307bfb-c4cb-4f66-aade-46fffe0c6e0c.cephfs=/ 2023-10-06T10:45:02.936 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-06T10:45:02.936 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T10:45:02.936 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T10:45:02.936 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T10:45:02.936 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.124:6789,172.21.15.124:6790" 2023-10-06T10:45:02.936 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.124:6789,172.21.15.124:6790:/ 2023-10-06T10:45:02.936 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=8f307bfb-c4cb-4f66-aade-46fffe0c6e0c" will pass to kernel 2023-10-06T10:45:02.937 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T10:45:02.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:45:02.937 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T10:45:03.002 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-10-06T10:45:03.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.158+0000 7fbb220ac700 1 Processor -- start 2023-10-06T10:45:03.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.158+0000 7fbb220ac700 1 -- start start 2023-10-06T10:45:03.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.159+0000 7fbb220ac700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb1c1594d0 0x7fbb1c0cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:03.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.159+0000 7fbb220ac700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbb1c0cbaa0 0x7fbb1c0ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:03.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.159+0000 7fbb220ac700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb1c0cbec0 con 0x7fbb1c1598a0 2023-10-06T10:45:03.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.159+0000 7fbb220ac700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb1c0cf5d0 con 0x7fbb1c0cbaa0 2023-10-06T10:45:03.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.159+0000 7fbb220ac700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c0cf8b0 con 0x7fbb1c1594d0 2023-10-06T10:45:03.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.159+0000 7fbb1b7fe700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb1c1594d0 0x7fbb1c0cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:03.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.160+0000 7fbb1b7fe700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb1c1594d0 0x7fbb1c0cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:41002/0 (socket says 172.21.15.92:41002) 2023-10-06T10:45:03.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.160+0000 7fbb1bfff700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbb1c0cbaa0 0x7fbb1c0ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:03.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.160+0000 7fbb1b7fe700 1 -- 172.21.15.92:0/3903279814 learned_addr learned my addr 172.21.15.92:0/3903279814 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:03.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.160+0000 7fbb1a7fc700 1 -- 172.21.15.92:0/3903279814 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 709491449 0 0) 0x7fbb1c0cbec0 con 0x7fbb1c1598a0 2023-10-06T10:45:03.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.160+0000 7fbb1b7fe700 1 -- 172.21.15.92:0/3903279814 >> v2:172.21.15.124:3300/0 conn(0x7fbb1c0cbaa0 msgr2=0x7fbb1c0ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:03.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.160+0000 7fbb1b7fe700 1 --2- 172.21.15.92:0/3903279814 >> v2:172.21.15.124:3300/0 conn(0x7fbb1c0cbaa0 0x7fbb1c0ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:03.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.160+0000 7fbb1b7fe700 1 -- 172.21.15.92:0/3903279814 >> v1:172.21.15.124:6789/0 conn(0x7fbb1c1598a0 legacy=0x7fbb1c0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:03.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.160+0000 7fbb1b7fe700 1 -- 172.21.15.92:0/3903279814 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c0cfb90 con 0x7fbb1c1594d0 2023-10-06T10:45:03.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.160+0000 7fbb1b7fe700 1 --2- 172.21.15.92:0/3903279814 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb1c1594d0 0x7fbb1c0cb560 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009f20 tx=0x7fbb0c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9523c8ddec240eb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:03.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.160+0000 7fbb1a7fc700 1 -- 172.21.15.92:0/3903279814 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c00e070 con 0x7fbb1c1594d0 2023-10-06T10:45:03.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.160+0000 7fbb1a7fc700 1 -- 172.21.15.92:0/3903279814 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0c00b9d0 con 0x7fbb1c1594d0 2023-10-06T10:45:03.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.160+0000 7fbb1a7fc700 1 -- 172.21.15.92:0/3903279814 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c0092e0 con 0x7fbb1c1594d0 2023-10-06T10:45:03.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.161+0000 7fbb220ac700 1 -- 172.21.15.92:0/3903279814 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb1c1594d0 msgr2=0x7fbb1c0cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:03.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.161+0000 7fbb220ac700 1 --2- 172.21.15.92:0/3903279814 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb1c1594d0 0x7fbb1c0cb560 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009f20 tx=0x7fbb0c009840 comp rx=0 tx=0).stop 2023-10-06T10:45:03.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.161+0000 7fbb220ac700 1 -- 172.21.15.92:0/3903279814 shutdown_connections 2023-10-06T10:45:03.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.161+0000 7fbb220ac700 1 --2- 172.21.15.92:0/3903279814 >> v2:172.21.15.124:3300/0 conn(0x7fbb1c0cbaa0 0x7fbb1c0ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:03.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.161+0000 7fbb220ac700 1 --2- 172.21.15.92:0/3903279814 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb1c1594d0 0x7fbb1c0cb560 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:03.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.161+0000 7fbb220ac700 1 -- 172.21.15.92:0/3903279814 >> 172.21.15.92:0/3903279814 conn(0x7fbb1c089e50 msgr2=0x7fbb1c099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:03.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.161+0000 7fbb220ac700 1 -- 172.21.15.92:0/3903279814 shutdown_connections 2023-10-06T10:45:03.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.161+0000 7fbb220ac700 1 -- 172.21.15.92:0/3903279814 wait complete. 2023-10-06T10:45:03.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.162+0000 7fbb220ac700 1 Processor -- start 2023-10-06T10:45:03.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.162+0000 7fbb220ac700 1 -- start start 2023-10-06T10:45:03.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.163+0000 7fbb220ac700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb1c0cbaa0 0x7fbb1c0b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:03.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.163+0000 7fbb220ac700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb1c1594d0 0x7fbb1c14b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:03.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.163+0000 7fbb220ac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb1c1598a0 0x7fbb1c15cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:03.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.163+0000 7fbb220ac700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c0b5300 con 0x7fbb1c1598a0 2023-10-06T10:45:03.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.163+0000 7fbb220ac700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c0b6210 con 0x7fbb1c1594d0 2023-10-06T10:45:03.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.163+0000 7fbb220ac700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c14caf0 con 0x7fbb1c0cbaa0 2023-10-06T10:45:03.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.163+0000 7fbb1bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb1c1598a0 0x7fbb1c15cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:03.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.163+0000 7fbb1bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb1c1598a0 0x7fbb1c15cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33974/0 (socket says 172.21.15.92:33974) 2023-10-06T10:45:03.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.163+0000 7fbb1bfff700 1 -- 172.21.15.92:0/2594210499 learned_addr learned my addr 172.21.15.92:0/2594210499 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:03.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.163+0000 7fbb1b7fe700 1 --2- 172.21.15.92:0/2594210499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb1c0cbaa0 0x7fbb1c0b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:03.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.163+0000 7fbb1affd700 1 --2- 172.21.15.92:0/2594210499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb1c1594d0 0x7fbb1c14b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:03.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.163+0000 7fbb1bfff700 1 -- 172.21.15.92:0/2594210499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb1c0cbaa0 msgr2=0x7fbb1c0b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:03.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.163+0000 7fbb1bfff700 1 --2- 172.21.15.92:0/2594210499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb1c0cbaa0 0x7fbb1c0b5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:03.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.163+0000 7fbb1bfff700 1 -- 172.21.15.92:0/2594210499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb1c1594d0 msgr2=0x7fbb1c14b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:03.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.163+0000 7fbb1bfff700 1 --2- 172.21.15.92:0/2594210499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb1c1594d0 0x7fbb1c14b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:03.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.163+0000 7fbb1bfff700 1 -- 172.21.15.92:0/2594210499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0c009ae0 con 0x7fbb1c1598a0 2023-10-06T10:45:03.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.163+0000 7fbb1bfff700 1 --2- 172.21.15.92:0/2594210499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb1c1598a0 0x7fbb1c15cfa0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400ea50 tx=0x7fbb140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:03.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.164+0000 7fbb037fe700 1 -- 172.21.15.92:0/2594210499 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb140096b0 con 0x7fbb1c1598a0 2023-10-06T10:45:03.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.164+0000 7fbb037fe700 1 -- 172.21.15.92:0/2594210499 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1400c850 con 0x7fbb1c1598a0 2023-10-06T10:45:03.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.164+0000 7fbb220ac700 1 -- 172.21.15.92:0/2594210499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1c16db00 con 0x7fbb1c1598a0 2023-10-06T10:45:03.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.164+0000 7fbb037fe700 1 -- 172.21.15.92:0/2594210499 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1400cc10 con 0x7fbb1c1598a0 2023-10-06T10:45:03.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.164+0000 7fbb220ac700 1 -- 172.21.15.92:0/2594210499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1c16dff0 con 0x7fbb1c1598a0 2023-10-06T10:45:03.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.165+0000 7fbb220ac700 1 -- 172.21.15.92:0/2594210499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1c064a90 con 0x7fbb1c1598a0 2023-10-06T10:45:03.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.165+0000 7fbb037fe700 1 -- 172.21.15.92:0/2594210499 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbb14013470 con 0x7fbb1c1598a0 2023-10-06T10:45:03.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.165+0000 7fbb037fe700 1 --2- 172.21.15.92:0/2594210499 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbb08066f30 0x7fbb080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:03.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.165+0000 7fbb1b7fe700 1 --2- 172.21.15.92:0/2594210499 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbb08066f30 0x7fbb080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:03.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.166+0000 7fbb037fe700 1 -- 172.21.15.92:0/2594210499 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fbb14084a00 con 0x7fbb1c1598a0 2023-10-06T10:45:03.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.166+0000 7fbb1b7fe700 1 --2- 172.21.15.92:0/2594210499 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbb08066f30 0x7fbb080693e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009880 tx=0x7fbb0c018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:03.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.170+0000 7fbb037fe700 1 -- 172.21.15.92:0/2594210499 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb140519d0 con 0x7fbb1c1598a0 2023-10-06T10:45:03.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:03.400+0000 7fbb220ac700 1 -- 172.21.15.92:0/2594210499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7fbb1c0cfe70 con 0x7fbb1c1598a0 2023-10-06T10:45:04.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.362+0000 7fbb037fe700 1 -- 172.21.15.92:0/2594210499 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v101) v1 ==== 105+0+0 (secure 0 0 0) 0x7fbb140170b0 con 0x7fbb1c1598a0 2023-10-06T10:45:04.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.364+0000 7fbb220ac700 1 -- 172.21.15.92:0/2594210499 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbb08066f30 msgr2=0x7fbb080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:04.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.364+0000 7fbb220ac700 1 --2- 172.21.15.92:0/2594210499 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbb08066f30 0x7fbb080693e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009880 tx=0x7fbb0c018000 comp rx=0 tx=0).stop 2023-10-06T10:45:04.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.364+0000 7fbb220ac700 1 -- 172.21.15.92:0/2594210499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb1c1598a0 msgr2=0x7fbb1c15cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:04.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.364+0000 7fbb220ac700 1 --2- 172.21.15.92:0/2594210499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb1c1598a0 0x7fbb1c15cfa0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400ea50 tx=0x7fbb140091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:04.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.365+0000 7fbb220ac700 1 -- 172.21.15.92:0/2594210499 shutdown_connections 2023-10-06T10:45:04.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.365+0000 7fbb220ac700 1 --2- 172.21.15.92:0/2594210499 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbb08066f30 0x7fbb080693e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:04.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.365+0000 7fbb220ac700 1 --2- 172.21.15.92:0/2594210499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb1c1598a0 0x7fbb1c15cfa0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:04.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.365+0000 7fbb220ac700 1 --2- 172.21.15.92:0/2594210499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb1c0cbaa0 0x7fbb1c0b5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:04.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.365+0000 7fbb220ac700 1 --2- 172.21.15.92:0/2594210499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb1c1594d0 0x7fbb1c14b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:04.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.365+0000 7fbb220ac700 1 -- 172.21.15.92:0/2594210499 >> 172.21.15.92:0/2594210499 conn(0x7fbb1c089e50 msgr2=0x7fbb1c14d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:04.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.366+0000 7fbb220ac700 1 -- 172.21.15.92:0/2594210499 shutdown_connections 2023-10-06T10:45:04.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.366+0000 7fbb220ac700 1 -- 172.21.15.92:0/2594210499 wait complete. 2023-10-06T10:45:04.383 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:45:04.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.495+0000 7f96c7ebb700 1 Processor -- start 2023-10-06T10:45:04.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.496+0000 7f96c7ebb700 1 -- start start 2023-10-06T10:45:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.496+0000 7f96c7ebb700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96c0159650 0x7f96c00ac440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.496+0000 7f96c7ebb700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f96c00ab430 0x7f96c0068a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.496+0000 7f96c7ebb700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c0068f70 con 0x7f96c00685c0 2023-10-06T10:45:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.496+0000 7f96c7ebb700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c00a9f90 con 0x7f96c00ab430 2023-10-06T10:45:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.496+0000 7f96c7ebb700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c00ac980 con 0x7f96c0159650 2023-10-06T10:45:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.496+0000 7f96c5456700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f96c00685c0 0x7f96c015d350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51394/0 (socket says 172.21.15.92:51394) 2023-10-06T10:45:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.497+0000 7f96c5456700 1 -- 172.21.15.92:0/160938221 learned_addr learned my addr 172.21.15.92:0/160938221 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.497+0000 7f96c5c57700 1 --2- 172.21.15.92:0/160938221 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96c0159650 0x7f96c00ac440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.497+0000 7f96c6458700 1 --2- 172.21.15.92:0/160938221 >> v2:172.21.15.124:3300/0 conn(0x7f96c00ab430 0x7f96c0068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.497+0000 7f96c4c55700 1 -- 172.21.15.92:0/160938221 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 874111247 0 0) 0x7f96c0068f70 con 0x7f96c00685c0 2023-10-06T10:45:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.497+0000 7f96c4c55700 1 -- 172.21.15.92:0/160938221 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96ac003570 con 0x7f96c00685c0 2023-10-06T10:45:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.497+0000 7f96c4c55700 1 -- 172.21.15.92:0/160938221 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 102876058 0 0) 0x7f96ac003570 con 0x7f96c00685c0 2023-10-06T10:45:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.497+0000 7f96c4c55700 1 -- 172.21.15.92:0/160938221 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96c0159650 msgr2=0x7f96c00ac440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.497+0000 7f96c4c55700 1 --2- 172.21.15.92:0/160938221 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96c0159650 0x7f96c00ac440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.497+0000 7f96c4c55700 1 -- 172.21.15.92:0/160938221 >> v2:172.21.15.124:3300/0 conn(0x7f96c00ab430 msgr2=0x7f96c0068a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.497+0000 7f96c4c55700 1 --2- 172.21.15.92:0/160938221 >> v2:172.21.15.124:3300/0 conn(0x7f96c00ab430 0x7f96c0068a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.497+0000 7f96c4c55700 1 -- 172.21.15.92:0/160938221 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c00abfa0 con 0x7f96c00685c0 2023-10-06T10:45:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.497+0000 7f96c4c55700 1 -- 172.21.15.92:0/160938221 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f96b0001470 con 0x7f96c00685c0 2023-10-06T10:45:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.497+0000 7f96c4c55700 1 -- 172.21.15.92:0/160938221 >> v1:172.21.15.92:6789/0 conn(0x7f96c00685c0 legacy=0x7f96c015d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.497+0000 7f96c4c55700 1 --2- 172.21.15.92:0/160938221 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96c0159650 0x7f96ac003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:04.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.497+0000 7f96c4c55700 1 --2- 172.21.15.92:0/160938221 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96ac003fe0 0x7f96ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:04.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.497+0000 7f96c4c55700 1 --2- 172.21.15.92:0/160938221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96ac004d50 0x7f96ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:04.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.497+0000 7f96c4c55700 1 -- 172.21.15.92:0/160938221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c00a9f90 con 0x7f96ac004d50 2023-10-06T10:45:04.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.497+0000 7f96c4c55700 1 -- 172.21.15.92:0/160938221 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c00ac980 con 0x7f96ac003fe0 2023-10-06T10:45:04.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.497+0000 7f96c4c55700 1 -- 172.21.15.92:0/160938221 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac00ba90 con 0x7f96c0159650 2023-10-06T10:45:04.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.497+0000 7f96c6458700 1 --2- 172.21.15.92:0/160938221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96ac004d50 0x7f96ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:04.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.497+0000 7f96c5456700 1 --2- 172.21.15.92:0/160938221 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96ac003fe0 0x7f96ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:04.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.497+0000 7f96c5c57700 1 --2- 172.21.15.92:0/160938221 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96c0159650 0x7f96ac003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:04.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.498+0000 7f96c6458700 1 -- 172.21.15.92:0/160938221 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96c0159650 msgr2=0x7f96ac003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:04.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.498+0000 7f96c6458700 1 --2- 172.21.15.92:0/160938221 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96c0159650 0x7f96ac003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:04.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.498+0000 7f96c6458700 1 -- 172.21.15.92:0/160938221 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96ac003fe0 msgr2=0x7f96ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:04.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.498+0000 7f96c6458700 1 --2- 172.21.15.92:0/160938221 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96ac003fe0 0x7f96ac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:04.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.498+0000 7f96c6458700 1 -- 172.21.15.92:0/160938221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ac003570 con 0x7f96ac004d50 2023-10-06T10:45:04.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.498+0000 7f96c6458700 1 --2- 172.21.15.92:0/160938221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96ac004d50 0x7f96ac0051c0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f96b800e210 tx=0x7f96b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6220c2e1213324f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:04.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.498+0000 7f96c4c55700 1 -- 172.21.15.92:0/160938221 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b800ea60 con 0x7f96ac004d50 2023-10-06T10:45:04.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.498+0000 7f96c4c55700 1 -- 172.21.15.92:0/160938221 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b800f040 con 0x7f96ac004d50 2023-10-06T10:45:04.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.498+0000 7f96c4c55700 1 -- 172.21.15.92:0/160938221 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b800ee10 con 0x7f96ac004d50 2023-10-06T10:45:04.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.498+0000 7f96c7ebb700 1 -- 172.21.15.92:0/160938221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96ac004d50 msgr2=0x7f96ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:04.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.498+0000 7f96c7ebb700 1 --2- 172.21.15.92:0/160938221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96ac004d50 0x7f96ac0051c0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f96b800e210 tx=0x7f96b800b040 comp rx=0 tx=0).stop 2023-10-06T10:45:04.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.498+0000 7f96c6458700 1 -- 172.21.15.92:0/160938221 reap_dead start 2023-10-06T10:45:04.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.498+0000 7f96c7ebb700 1 -- 172.21.15.92:0/160938221 shutdown_connections 2023-10-06T10:45:04.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.498+0000 7f96c7ebb700 1 -- 172.21.15.92:0/160938221 >> 172.21.15.92:0/160938221 conn(0x7f96c006de40 msgr2=0x7f96c006e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:04.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.498+0000 7f96c7ebb700 1 -- 172.21.15.92:0/160938221 shutdown_connections 2023-10-06T10:45:04.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.498+0000 7f96c7ebb700 1 -- 172.21.15.92:0/160938221 wait complete. 2023-10-06T10:45:04.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.499+0000 7f96c7ebb700 1 Processor -- start 2023-10-06T10:45:04.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.500+0000 7f96c7ebb700 1 -- start start 2023-10-06T10:45:04.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.500+0000 7f96c7ebb700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96c0151680 0x7f96c016a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:04.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.500+0000 7f96c7ebb700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96c0151a50 0x7f96c016ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:04.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.500+0000 7f96c7ebb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96c01530d0 0x7f96c0153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:04.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.500+0000 7f96c7ebb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c0152130 con 0x7f96c01530d0 2023-10-06T10:45:04.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.500+0000 7f96c7ebb700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c0153a90 con 0x7f96c0151680 2023-10-06T10:45:04.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.500+0000 7f96c7ebb700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c0152c40 con 0x7f96c0151a50 2023-10-06T10:45:04.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.500+0000 7f96c6458700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96c01530d0 0x7f96c0153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:04.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.500+0000 7f96c6458700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96c01530d0 0x7f96c0153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39082/0 (socket says 172.21.15.92:39082) 2023-10-06T10:45:04.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.500+0000 7f96c6458700 1 -- 172.21.15.92:0/3337562941 learned_addr learned my addr 172.21.15.92:0/3337562941 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:04.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.500+0000 7f96c5c57700 1 --2- 172.21.15.92:0/3337562941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96c0151680 0x7f96c016a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:04.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.500+0000 7f96c5456700 1 --2- 172.21.15.92:0/3337562941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96c0151a50 0x7f96c016ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:04.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.501+0000 7f96c6458700 1 -- 172.21.15.92:0/3337562941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96c0151a50 msgr2=0x7f96c016ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:04.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.501+0000 7f96c6458700 1 --2- 172.21.15.92:0/3337562941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96c0151a50 0x7f96c016ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:04.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.501+0000 7f96c6458700 1 -- 172.21.15.92:0/3337562941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96c0151680 msgr2=0x7f96c016a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:04.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.501+0000 7f96c6458700 1 --2- 172.21.15.92:0/3337562941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96c0151680 0x7f96c016a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:04.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.501+0000 7f96c6458700 1 -- 172.21.15.92:0/3337562941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c0152310 con 0x7f96c01530d0 2023-10-06T10:45:04.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.501+0000 7f96c6458700 1 --2- 172.21.15.92:0/3337562941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96c01530d0 0x7f96c0153550 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f96b8016000 tx=0x7f96b801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:04.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.501+0000 7f96b5ffb700 1 -- 172.21.15.92:0/3337562941 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b800eee0 con 0x7f96c01530d0 2023-10-06T10:45:04.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.501+0000 7f96b5ffb700 1 -- 172.21.15.92:0/3337562941 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b8014e10 con 0x7f96c01530d0 2023-10-06T10:45:04.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.501+0000 7f96c7ebb700 1 -- 172.21.15.92:0/3337562941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c01525f0 con 0x7f96c01530d0 2023-10-06T10:45:04.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.501+0000 7f96b5ffb700 1 -- 172.21.15.92:0/3337562941 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b801d880 con 0x7f96c01530d0 2023-10-06T10:45:04.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.501+0000 7f96c7ebb700 1 -- 172.21.15.92:0/3337562941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c0169900 con 0x7f96c01530d0 2023-10-06T10:45:04.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.502+0000 7f96c7ebb700 1 -- 172.21.15.92:0/3337562941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c0152dd0 con 0x7f96c01530d0 2023-10-06T10:45:04.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.502+0000 7f96b5ffb700 1 -- 172.21.15.92:0/3337562941 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f96b8003040 con 0x7f96c01530d0 2023-10-06T10:45:04.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.503+0000 7f96b5ffb700 1 --2- 172.21.15.92:0/3337562941 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f96a4066f30 0x7f96a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:04.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.503+0000 7f96b5ffb700 1 -- 172.21.15.92:0/3337562941 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f96b8085b40 con 0x7f96c01530d0 2023-10-06T10:45:04.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.503+0000 7f96c5c57700 1 --2- 172.21.15.92:0/3337562941 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f96a4066f30 0x7f96a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:04.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.503+0000 7f96c5c57700 1 --2- 172.21.15.92:0/3337562941 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f96a4066f30 0x7f96a40693e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f96bc006e50 tx=0x7f96bc009120 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:04.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.507+0000 7f96b5ffb700 1 -- 172.21.15.92:0/3337562941 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96b8052b10 con 0x7f96c01530d0 2023-10-06T10:45:04.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.733+0000 7f96c7ebb700 1 -- 172.21.15.92:0/3337562941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96c0064a90 con 0x7f96c01530d0 2023-10-06T10:45:04.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.734+0000 7f96b5ffb700 1 -- 172.21.15.92:0/3337562941 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4624 (secure 0 0 0) 0x7f96b8056130 con 0x7f96c01530d0 2023-10-06T10:45:04.735 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:04.735 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":102,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2016052382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2016052382},{"type":"v1","addr":"172.21.15.124:6834","nonce":2016052382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":6719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/2307521923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2307521923},{"type":"v1","addr":"172.21.15.124:6835","nonce":2307521923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:44:51.106486+0000","modified":"2023-10-06T10:45:04.362387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":8520,"mds_1":8517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8517":{"gid":8517,"name":"a","rank":1,"incarnation":102,"state":"up:creating","state_seq":1,"addr":"172.21.15.92:6836/1979826278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979826278},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979826278}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2298348426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2298348426},{"type":"v1","addr":"172.21.15.92:6837","nonce":2298348426}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T10:45:04.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.736+0000 7f96c7ebb700 1 -- 172.21.15.92:0/3337562941 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f96a4066f30 msgr2=0x7f96a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:04.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.736+0000 7f96c7ebb700 1 --2- 172.21.15.92:0/3337562941 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f96a4066f30 0x7f96a40693e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f96bc006e50 tx=0x7f96bc009120 comp rx=0 tx=0).stop 2023-10-06T10:45:04.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.736+0000 7f96c7ebb700 1 -- 172.21.15.92:0/3337562941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96c01530d0 msgr2=0x7f96c0153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:04.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.736+0000 7f96c7ebb700 1 --2- 172.21.15.92:0/3337562941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96c01530d0 0x7f96c0153550 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f96b8016000 tx=0x7f96b801a000 comp rx=0 tx=0).stop 2023-10-06T10:45:04.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.736+0000 7f96c7ebb700 1 -- 172.21.15.92:0/3337562941 shutdown_connections 2023-10-06T10:45:04.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.736+0000 7f96c7ebb700 1 --2- 172.21.15.92:0/3337562941 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f96a4066f30 0x7f96a40693e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:04.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.736+0000 7f96c7ebb700 1 --2- 172.21.15.92:0/3337562941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96c01530d0 0x7f96c0153550 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:04.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.736+0000 7f96c7ebb700 1 --2- 172.21.15.92:0/3337562941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96c0151680 0x7f96c016a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:04.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.736+0000 7f96c7ebb700 1 --2- 172.21.15.92:0/3337562941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96c0151a50 0x7f96c016ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:04.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.736+0000 7f96c7ebb700 1 -- 172.21.15.92:0/3337562941 >> 172.21.15.92:0/3337562941 conn(0x7f96c006de40 msgr2=0x7f96c006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:04.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.736+0000 7f96c7ebb700 1 -- 172.21.15.92:0/3337562941 shutdown_connections 2023-10-06T10:45:04.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:04.736+0000 7f96c7ebb700 1 -- 172.21.15.92:0/3337562941 wait complete. 2023-10-06T10:45:04.740 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 102 2023-10-06T10:45:05.151 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T10:44:51.106486+0000', 'modified': '2023-10-06T10:45:04.362387+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 8520, 'mds_1': 8517}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8517': {'gid': 8517, 'name': 'a', 'rank': 1, 'incarnation': 102, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.92:6836/1979826278', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 1979826278}, {'type': 'v1', 'addr': '172.21.15.92:6836', 'nonce': 1979826278}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_8520': {'gid': 8520, 'name': 'c', 'rank': 0, 'incarnation': 96, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6837/2298348426', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 2298348426}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 2298348426}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T10:45:05.151 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8517:up:creating 2023-10-06T10:45:06.027 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-10-06T10:45:06.025+0000 7f0bb9087700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-06T10:45:06.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:45:06.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.282+0000 7f8a088b9700 1 Processor -- start 2023-10-06T10:45:06.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.282+0000 7f8a088b9700 1 -- start start 2023-10-06T10:45:06.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.283+0000 7f8a088b9700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8a04159440 0x7f8a040b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:06.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.283+0000 7f8a088b9700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a040b2100 0x7f8a040b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:06.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.283+0000 7f8a088b9700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a040cb7a0 con 0x7f8a040b35d0 2023-10-06T10:45:06.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.283+0000 7f8a088b9700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a040b0860 con 0x7f8a04159440 2023-10-06T10:45:06.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.283+0000 7f8a088b9700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a040cb150 con 0x7f8a040b2100 2023-10-06T10:45:06.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.283+0000 7f8a03fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a040b2100 0x7f8a040b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:06.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.283+0000 7f8a03fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a040b2100 0x7f8a040b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:33442/0 (socket says 172.21.15.92:33442) 2023-10-06T10:45:06.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.283+0000 7f8a03fff700 1 -- 172.21.15.92:0/3802438474 learned_addr learned my addr 172.21.15.92:0/3802438474 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:06.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.283+0000 7f8a037fe700 1 --2- 172.21.15.92:0/3802438474 >> v2:172.21.15.124:3300/0 conn(0x7f8a04159440 0x7f8a040b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:06.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.284+0000 7f8a03fff700 1 -- 172.21.15.92:0/3802438474 >> v2:172.21.15.124:3300/0 conn(0x7f8a04159440 msgr2=0x7f8a040b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:06.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.284+0000 7f8a03fff700 1 --2- 172.21.15.92:0/3802438474 >> v2:172.21.15.124:3300/0 conn(0x7f8a04159440 0x7f8a040b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:06.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.284+0000 7f8a03fff700 1 -- 172.21.15.92:0/3802438474 >> v1:172.21.15.124:6789/0 conn(0x7f8a040b35d0 legacy=0x7f8a040cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:06.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.284+0000 7f8a03fff700 1 -- 172.21.15.92:0/3802438474 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a040cb430 con 0x7f8a040b2100 2023-10-06T10:45:06.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.284+0000 7f8a03fff700 1 --2- 172.21.15.92:0/3802438474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a040b2100 0x7f8a040b2560 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0099b0 tx=0x7f89fc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abb968c0043544d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:06.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.284+0000 7f8a027fc700 1 -- 172.21.15.92:0/3802438474 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc009ea0 con 0x7f8a040b2100 2023-10-06T10:45:06.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.284+0000 7f8a027fc700 1 -- 172.21.15.92:0/3802438474 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc00b860 con 0x7f8a040b2100 2023-10-06T10:45:06.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.284+0000 7f8a027fc700 1 -- 172.21.15.92:0/3802438474 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc00bc00 con 0x7f8a040b2100 2023-10-06T10:45:06.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.285+0000 7f8a088b9700 1 -- 172.21.15.92:0/3802438474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a040b2100 msgr2=0x7f8a040b2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:06.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.285+0000 7f8a088b9700 1 --2- 172.21.15.92:0/3802438474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a040b2100 0x7f8a040b2560 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0099b0 tx=0x7f89fc009250 comp rx=0 tx=0).stop 2023-10-06T10:45:06.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.285+0000 7f8a088b9700 1 -- 172.21.15.92:0/3802438474 shutdown_connections 2023-10-06T10:45:06.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.285+0000 7f8a088b9700 1 --2- 172.21.15.92:0/3802438474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a040b2100 0x7f8a040b2560 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:06.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.285+0000 7f8a088b9700 1 --2- 172.21.15.92:0/3802438474 >> v2:172.21.15.124:3300/0 conn(0x7f8a04159440 0x7f8a040b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:06.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.285+0000 7f8a088b9700 1 -- 172.21.15.92:0/3802438474 >> 172.21.15.92:0/3802438474 conn(0x7f8a0406ece0 msgr2=0x7f8a0406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:06.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.285+0000 7f8a088b9700 1 -- 172.21.15.92:0/3802438474 shutdown_connections 2023-10-06T10:45:06.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.285+0000 7f8a088b9700 1 -- 172.21.15.92:0/3802438474 wait complete. 2023-10-06T10:45:06.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.286+0000 7f8a088b9700 1 Processor -- start 2023-10-06T10:45:06.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.286+0000 7f8a088b9700 1 -- start start 2023-10-06T10:45:06.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.287+0000 7f8a088b9700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a040b2100 0x7f8a04155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:06.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.287+0000 7f8a088b9700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a040b35d0 0x7f8a0415ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:06.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.287+0000 7f8a088b9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a04159440 0x7f8a0415f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:06.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.287+0000 7f8a088b9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a040c7470 con 0x7f8a04159440 2023-10-06T10:45:06.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.287+0000 7f8a088b9700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a040c7750 con 0x7f8a040b35d0 2023-10-06T10:45:06.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.287+0000 7f8a088b9700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a040c7a30 con 0x7f8a040b2100 2023-10-06T10:45:06.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.287+0000 7f8a03fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a04159440 0x7f8a0415f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:06.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.287+0000 7f8a03fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a04159440 0x7f8a0415f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39088/0 (socket says 172.21.15.92:39088) 2023-10-06T10:45:06.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.287+0000 7f8a03fff700 1 -- 172.21.15.92:0/4165002974 learned_addr learned my addr 172.21.15.92:0/4165002974 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:06.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.287+0000 7f8a037fe700 1 --2- 172.21.15.92:0/4165002974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a040b2100 0x7f8a04155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:06.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.287+0000 7f8a02ffd700 1 --2- 172.21.15.92:0/4165002974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a040b35d0 0x7f8a0415ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:06.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.287+0000 7f8a03fff700 1 -- 172.21.15.92:0/4165002974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a040b2100 msgr2=0x7f8a04155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:06.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.287+0000 7f8a03fff700 1 --2- 172.21.15.92:0/4165002974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a040b2100 0x7f8a04155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:06.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.287+0000 7f8a03fff700 1 -- 172.21.15.92:0/4165002974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a040b35d0 msgr2=0x7f8a0415ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:06.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.287+0000 7f8a03fff700 1 --2- 172.21.15.92:0/4165002974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a040b35d0 0x7f8a0415ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:06.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.287+0000 7f8a03fff700 1 -- 172.21.15.92:0/4165002974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc0094f0 con 0x7f8a04159440 2023-10-06T10:45:06.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.288+0000 7f8a03fff700 1 --2- 172.21.15.92:0/4165002974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a04159440 0x7f8a0415f460 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f8a040cb290 tx=0x7f89fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:06.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.288+0000 7f89eb7fe700 1 -- 172.21.15.92:0/4165002974 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc00bbe0 con 0x7f8a04159440 2023-10-06T10:45:06.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.288+0000 7f89eb7fe700 1 -- 172.21.15.92:0/4165002974 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc011420 con 0x7f8a04159440 2023-10-06T10:45:06.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.288+0000 7f89eb7fe700 1 -- 172.21.15.92:0/4165002974 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc0117e0 con 0x7f8a04159440 2023-10-06T10:45:06.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.288+0000 7f8a088b9700 1 -- 172.21.15.92:0/4165002974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a040c7d10 con 0x7f8a04159440 2023-10-06T10:45:06.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.288+0000 7f8a088b9700 1 -- 172.21.15.92:0/4165002974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a04155b60 con 0x7f8a04159440 2023-10-06T10:45:06.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.289+0000 7f8a088b9700 1 -- 172.21.15.92:0/4165002974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a040c70f0 con 0x7f8a04159440 2023-10-06T10:45:06.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.289+0000 7f89eb7fe700 1 -- 172.21.15.92:0/4165002974 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f89fc011bd0 con 0x7f8a04159440 2023-10-06T10:45:06.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.290+0000 7f89eb7fe700 1 --2- 172.21.15.92:0/4165002974 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f89f0066fb0 0x7f89f0069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:06.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.290+0000 7f89eb7fe700 1 -- 172.21.15.92:0/4165002974 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f89fc086300 con 0x7f8a04159440 2023-10-06T10:45:06.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.290+0000 7f8a037fe700 1 --2- 172.21.15.92:0/4165002974 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f89f0066fb0 0x7f89f0069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:06.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.290+0000 7f8a037fe700 1 --2- 172.21.15.92:0/4165002974 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f89f0066fb0 0x7f89f0069460 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009fd0 tx=0x7f89f400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:06.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.294+0000 7f89eb7fe700 1 -- 172.21.15.92:0/4165002974 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89fc053230 con 0x7f8a04159440 2023-10-06T10:45:06.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.528+0000 7f8a088b9700 1 -- 172.21.15.92:0/4165002974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a040c8230 con 0x7f8a04159440 2023-10-06T10:45:06.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.529+0000 7f89eb7fe700 1 -- 172.21.15.92:0/4165002974 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4637 (secure 0 0 0) 0x7f8a040c8230 con 0x7f8a04159440 2023-10-06T10:45:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:06.532 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":105,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2016052382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2016052382},{"type":"v1","addr":"172.21.15.124:6834","nonce":2016052382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:44:51.106486+0000","modified":"2023-10-06T10:45:06.368491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8520,"mds_1":8517,"mds_2":6719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6719":{"gid":6719,"name":"d","rank":2,"incarnation":104,"state":"up:active","state_seq":6,"addr":"172.21.15.124:6835/2307521923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2307521923},{"type":"v1","addr":"172.21.15.124:6835","nonce":2307521923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8517":{"gid":8517,"name":"a","rank":1,"incarnation":102,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6836/1979826278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979826278},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979826278}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2298348426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2298348426},{"type":"v1","addr":"172.21.15.92:6837","nonce":2298348426}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T10:45:06.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.531+0000 7f8a088b9700 1 -- 172.21.15.92:0/4165002974 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f89f0066fb0 msgr2=0x7f89f0069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:06.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.531+0000 7f8a088b9700 1 --2- 172.21.15.92:0/4165002974 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f89f0066fb0 0x7f89f0069460 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009fd0 tx=0x7f89f400b040 comp rx=0 tx=0).stop 2023-10-06T10:45:06.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.531+0000 7f8a088b9700 1 -- 172.21.15.92:0/4165002974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a04159440 msgr2=0x7f8a0415f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:06.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.531+0000 7f8a088b9700 1 --2- 172.21.15.92:0/4165002974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a04159440 0x7f8a0415f460 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f8a040cb290 tx=0x7f89fc017000 comp rx=0 tx=0).stop 2023-10-06T10:45:06.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.531+0000 7f8a088b9700 1 -- 172.21.15.92:0/4165002974 shutdown_connections 2023-10-06T10:45:06.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.531+0000 7f8a088b9700 1 --2- 172.21.15.92:0/4165002974 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f89f0066fb0 0x7f89f0069460 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:06.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.531+0000 7f8a088b9700 1 --2- 172.21.15.92:0/4165002974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a04159440 0x7f8a0415f460 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:06.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.531+0000 7f8a088b9700 1 --2- 172.21.15.92:0/4165002974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a040b2100 0x7f8a04155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:06.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.531+0000 7f8a088b9700 1 --2- 172.21.15.92:0/4165002974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a040b35d0 0x7f8a0415ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:06.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.531+0000 7f8a088b9700 1 -- 172.21.15.92:0/4165002974 >> 172.21.15.92:0/4165002974 conn(0x7f8a0406ece0 msgr2=0x7f8a040b4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:06.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.531+0000 7f8a088b9700 1 -- 172.21.15.92:0/4165002974 shutdown_connections 2023-10-06T10:45:06.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:06.531+0000 7f8a088b9700 1 -- 172.21.15.92:0/4165002974 wait complete. 2023-10-06T10:45:06.535 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 105 2023-10-06T10:45:06.947 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T10:44:51.106486+0000', 'modified': '2023-10-06T10:45:06.368491+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 8520, 'mds_1': 8517, 'mds_2': 6719}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6719': {'gid': 6719, 'name': 'd', 'rank': 2, 'incarnation': 104, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.124:6835/2307521923', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6833', 'nonce': 2307521923}, {'type': 'v1', 'addr': '172.21.15.124:6835', 'nonce': 2307521923}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_8517': {'gid': 8517, 'name': 'a', 'rank': 1, 'incarnation': 102, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.92:6836/1979826278', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 1979826278}, {'type': 'v1', 'addr': '172.21.15.92:6836', 'nonce': 1979826278}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_8520': {'gid': 8520, 'name': 'c', 'rank': 0, 'incarnation': 96, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6837/2298348426', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 2298348426}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 2298348426}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T10:45:06.947 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-10-06T10:45:06.948 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T10:45:07.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.062+0000 7f17c5bf5700 1 Processor -- start 2023-10-06T10:45:07.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.062+0000 7f17c5bf5700 1 -- start start 2023-10-06T10:45:07.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.062+0000 7f17c5bf5700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f17c00ad320 0x7f17c00ac230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.063+0000 7f17c5bf5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f17c00ad6f0 0x7f17c00685c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.063+0000 7f17c5bf5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c0068f40 con 0x7f17c0068b00 2023-10-06T10:45:07.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.063+0000 7f17c5bf5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c0159f40 con 0x7f17c00ad6f0 2023-10-06T10:45:07.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.063+0000 7f17c5bf5700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c015a220 con 0x7f17c00ad320 2023-10-06T10:45:07.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.063+0000 7f17b6ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f17c00ad6f0 0x7f17c00685c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.063+0000 7f17bffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f17c0068b00 0x7f17c015d110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51396/0 (socket says 172.21.15.92:51396) 2023-10-06T10:45:07.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.063+0000 7f17bffff700 1 -- 172.21.15.92:0/1074491513 learned_addr learned my addr 172.21.15.92:0/1074491513 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:07.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.063+0000 7f17b6ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f17c00ad6f0 0x7f17c00685c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39096/0 (socket says 172.21.15.92:39096) 2023-10-06T10:45:07.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.063+0000 7f17bf7fe700 1 --2- 172.21.15.92:0/1074491513 >> v2:172.21.15.124:3300/0 conn(0x7f17c00ad320 0x7f17c00ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.063+0000 7f17b6ffd700 1 -- 172.21.15.92:0/1074491513 >> v2:172.21.15.124:3300/0 conn(0x7f17c00ad320 msgr2=0x7f17c00ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.063+0000 7f17b6ffd700 1 --2- 172.21.15.92:0/1074491513 >> v2:172.21.15.124:3300/0 conn(0x7f17c00ad320 0x7f17c00ac230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.063+0000 7f17b6ffd700 1 -- 172.21.15.92:0/1074491513 >> v1:172.21.15.92:6789/0 conn(0x7f17c0068b00 legacy=0x7f17c015d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.063+0000 7f17b6ffd700 1 -- 172.21.15.92:0/1074491513 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c015a500 con 0x7f17c00ad6f0 2023-10-06T10:45:07.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.063+0000 7f17beffd700 1 -- 172.21.15.92:0/1074491513 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3484635364 0 0) 0x7f17c0068f40 con 0x7f17c0068b00 2023-10-06T10:45:07.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.063+0000 7f17bf7fe700 1 --2- 172.21.15.92:0/1074491513 >> v2:172.21.15.124:3300/0 conn(0x7f17c00ad320 0x7f17c00ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:07.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.063+0000 7f17b6ffd700 1 --2- 172.21.15.92:0/1074491513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17c00ad6f0 0x7f17c00685c0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f17c0063db0 tx=0x7f17a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ebe6473909ecea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.063+0000 7f17beffd700 1 -- 172.21.15.92:0/1074491513 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a800b6b0 con 0x7f17c00ad6f0 2023-10-06T10:45:07.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.063+0000 7f17beffd700 1 -- 172.21.15.92:0/1074491513 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a800b810 con 0x7f17c00ad6f0 2023-10-06T10:45:07.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.063+0000 7f17beffd700 1 -- 172.21.15.92:0/1074491513 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a800bbb0 con 0x7f17c00ad6f0 2023-10-06T10:45:07.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.064+0000 7f17c5bf5700 1 -- 172.21.15.92:0/1074491513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17c00ad6f0 msgr2=0x7f17c00685c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.064+0000 7f17c5bf5700 1 --2- 172.21.15.92:0/1074491513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17c00ad6f0 0x7f17c00685c0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f17c0063db0 tx=0x7f17a80091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.064+0000 7f17c5bf5700 1 -- 172.21.15.92:0/1074491513 shutdown_connections 2023-10-06T10:45:07.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.064+0000 7f17c5bf5700 1 --2- 172.21.15.92:0/1074491513 >> v2:172.21.15.124:3300/0 conn(0x7f17c00ad320 0x7f17c00ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.064+0000 7f17c5bf5700 1 --2- 172.21.15.92:0/1074491513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17c00ad6f0 0x7f17c00685c0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.064+0000 7f17c5bf5700 1 -- 172.21.15.92:0/1074491513 >> 172.21.15.92:0/1074491513 conn(0x7f17c006dcd0 msgr2=0x7f17c006e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:07.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.064+0000 7f17c5bf5700 1 -- 172.21.15.92:0/1074491513 shutdown_connections 2023-10-06T10:45:07.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.064+0000 7f17c5bf5700 1 -- 172.21.15.92:0/1074491513 wait complete. 2023-10-06T10:45:07.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.065+0000 7f17c5bf5700 1 Processor -- start 2023-10-06T10:45:07.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.065+0000 7f17c5bf5700 1 -- start start 2023-10-06T10:45:07.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.066+0000 7f17c5bf5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17c0068b00 0x7f17c00cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.066+0000 7f17c5bf5700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17c00ad320 0x7f17c0148120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.066+0000 7f17c5bf5700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17c00ad6f0 0x7f17c00c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.066+0000 7f17c5bf5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c00c8140 con 0x7f17c0068b00 2023-10-06T10:45:07.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.066+0000 7f17c5bf5700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c00c85d0 con 0x7f17c00ad320 2023-10-06T10:45:07.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.066+0000 7f17c5bf5700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c00c7530 con 0x7f17c00ad6f0 2023-10-06T10:45:07.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.066+0000 7f17bf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17c0068b00 0x7f17c00cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.066+0000 7f17bf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17c0068b00 0x7f17c00cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39106/0 (socket says 172.21.15.92:39106) 2023-10-06T10:45:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.066+0000 7f17bf7fe700 1 -- 172.21.15.92:0/2931278864 learned_addr learned my addr 172.21.15.92:0/2931278864 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.066+0000 7f17bf7fe700 1 -- 172.21.15.92:0/2931278864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17c00ad6f0 msgr2=0x7f17c00c6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.066+0000 7f17bf7fe700 1 --2- 172.21.15.92:0/2931278864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17c00ad6f0 0x7f17c00c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.066+0000 7f17bf7fe700 1 -- 172.21.15.92:0/2931278864 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17c00ad320 msgr2=0x7f17c0148120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.066+0000 7f17bf7fe700 1 --2- 172.21.15.92:0/2931278864 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17c00ad320 0x7f17c0148120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.066+0000 7f17bf7fe700 1 -- 172.21.15.92:0/2931278864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a8009420 con 0x7f17c0068b00 2023-10-06T10:45:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.066+0000 7f17bf7fe700 1 --2- 172.21.15.92:0/2931278864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17c0068b00 0x7f17c00cb130 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f17b000c830 tx=0x7f17b0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.067+0000 7f17b7fff700 1 -- 172.21.15.92:0/2931278864 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b000a680 con 0x7f17c0068b00 2023-10-06T10:45:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.067+0000 7f17b7fff700 1 -- 172.21.15.92:0/2931278864 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b000a7e0 con 0x7f17c0068b00 2023-10-06T10:45:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.067+0000 7f17c5bf5700 1 -- 172.21.15.92:0/2931278864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c00c7810 con 0x7f17c0068b00 2023-10-06T10:45:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.067+0000 7f17b7fff700 1 -- 172.21.15.92:0/2931278864 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b000aba0 con 0x7f17c0068b00 2023-10-06T10:45:07.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.067+0000 7f17c5bf5700 1 -- 172.21.15.92:0/2931278864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c00c7dd0 con 0x7f17c0068b00 2023-10-06T10:45:07.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.068+0000 7f17b7fff700 1 -- 172.21.15.92:0/2931278864 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f17b00113f0 con 0x7f17c0068b00 2023-10-06T10:45:07.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.068+0000 7f17b7fff700 1 --2- 172.21.15.92:0/2931278864 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f17a0066f30 0x7f17a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.069+0000 7f17b6ffd700 1 --2- 172.21.15.92:0/2931278864 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f17a0066f30 0x7f17a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.069+0000 7f17b7fff700 1 -- 172.21.15.92:0/2931278864 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f17b0084e00 con 0x7f17c0068b00 2023-10-06T10:45:07.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.069+0000 7f17b6ffd700 1 --2- 172.21.15.92:0/2931278864 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f17a0066f30 0x7f17a00693e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f17c00c17c0 tx=0x7f17a8017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.070+0000 7f17ceeedb80 1 Processor -- start 2023-10-06T10:45:07.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.071+0000 7f17ceeedb80 1 -- start start 2023-10-06T10:45:07.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.071+0000 7f17ceeedb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5620fa845a90 0x5620fa845e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.071+0000 7f17ceeedb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa8019a0 0x5620fa7fe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.071+0000 7f17ceeedb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa7ffbd0 0x5620fa7feac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.071+0000 7f17ceeedb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620fa77ec80 con 0x5620fa8019a0 2023-10-06T10:45:07.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.071+0000 7f17ceeedb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620fa755a70 con 0x5620fa7ffbd0 2023-10-06T10:45:07.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.071+0000 7f17ceeedb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620fa753270 con 0x5620fa845a90 2023-10-06T10:45:07.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.072+0000 7f17b6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa8019a0 0x5620fa7fe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.072+0000 7f17b6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa8019a0 0x5620fa7fe580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39122/0 (socket says 172.21.15.92:39122) 2023-10-06T10:45:07.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.072+0000 7f17b6ffd700 1 -- 172.21.15.92:0/355015453 learned_addr learned my addr 172.21.15.92:0/355015453 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:07.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.072+0000 7f17bf7fe700 1 --2- 172.21.15.92:0/355015453 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5620fa845a90 0x5620fa845e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.072+0000 7f17bffff700 1 --2- 172.21.15.92:0/355015453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa7ffbd0 0x5620fa7feac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.072+0000 7f17b6ffd700 1 -- 172.21.15.92:0/355015453 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5620fa845a90 msgr2=0x5620fa845e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.072+0000 7f17b6ffd700 1 --2- 172.21.15.92:0/355015453 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5620fa845a90 0x5620fa845e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.072+0000 7f17b6ffd700 1 -- 172.21.15.92:0/355015453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa7ffbd0 msgr2=0x5620fa7feac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.072+0000 7f17b6ffd700 1 --2- 172.21.15.92:0/355015453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa7ffbd0 0x5620fa7feac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.072+0000 7f17b6ffd700 1 -- 172.21.15.92:0/355015453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620fa5e9270 con 0x5620fa8019a0 2023-10-06T10:45:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.072+0000 7f17bffff700 1 --2- 172.21.15.92:0/355015453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa7ffbd0 0x5620fa7feac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.072+0000 7f17b6ffd700 1 --2- 172.21.15.92:0/355015453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa8019a0 0x5620fa7fe580 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f17a800ba80 tx=0x7f17a801d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19272f265275337 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.072+0000 7f17b4ff9700 1 -- 172.21.15.92:0/355015453 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a800bb60 con 0x5620fa8019a0 2023-10-06T10:45:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.072+0000 7f17b4ff9700 1 -- 172.21.15.92:0/355015453 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a801cc60 con 0x5620fa8019a0 2023-10-06T10:45:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.072+0000 7f17b4ff9700 1 -- 172.21.15.92:0/355015453 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a801cdf0 con 0x5620fa8019a0 2023-10-06T10:45:07.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.073+0000 7f17ceeedb80 1 -- 172.21.15.92:0/355015453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa8019a0 msgr2=0x5620fa7fe580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.073+0000 7f17ceeedb80 1 --2- 172.21.15.92:0/355015453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa8019a0 0x5620fa7fe580 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f17a800ba80 tx=0x7f17a801d040 comp rx=0 tx=0).stop 2023-10-06T10:45:07.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.073+0000 7f17ceeedb80 1 -- 172.21.15.92:0/355015453 shutdown_connections 2023-10-06T10:45:07.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.073+0000 7f17ceeedb80 1 --2- 172.21.15.92:0/355015453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa7ffbd0 0x5620fa7feac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.073+0000 7f17ceeedb80 1 --2- 172.21.15.92:0/355015453 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5620fa845a90 0x5620fa845e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.073+0000 7f17ceeedb80 1 --2- 172.21.15.92:0/355015453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa8019a0 0x5620fa7fe580 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.073+0000 7f17ceeedb80 1 -- 172.21.15.92:0/355015453 >> 172.21.15.92:0/355015453 conn(0x5620fa800070 msgr2=0x5620fa800440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:07.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.073+0000 7f17ceeedb80 1 -- 172.21.15.92:0/355015453 shutdown_connections 2023-10-06T10:45:07.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.073+0000 7f17ceeedb80 1 -- 172.21.15.92:0/355015453 wait complete. 2023-10-06T10:45:07.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.073+0000 7f17ceeedb80 1 -- start start 2023-10-06T10:45:07.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.073+0000 7f17ceeedb80 1 Processor -- start 2023-10-06T10:45:07.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.074+0000 7f17ceeedb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa7ffbd0 0x5620fa829960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.074+0000 7f17ceeedb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa8019a0 0x5620fa8419c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.074+0000 7f17ceeedb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5620fa845a90 0x5620fa83c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.074+0000 7f17ceeedb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620fa74f060 con 0x5620fa7ffbd0 2023-10-06T10:45:07.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.074+0000 7f17ceeedb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620fa6fa890 con 0x5620fa8019a0 2023-10-06T10:45:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.074+0000 7f17ceeedb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620fa715ea0 con 0x5620fa845a90 2023-10-06T10:45:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.074+0000 7f17ceeedb80 10 client.0 _ll_register_callbacks cb 0x5620fa5ebf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.074+0000 7f17bf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa7ffbd0 0x5620fa829960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.074+0000 7f17bf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa7ffbd0 0x5620fa829960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39132/0 (socket says 172.21.15.92:39132) 2023-10-06T10:45:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.074+0000 7f17bf7fe700 1 -- 172.21.15.92:0/3085122549 learned_addr learned my addr 172.21.15.92:0/3085122549 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f17bffff700 1 --2- 172.21.15.92:0/3085122549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5620fa845a90 0x5620fa83c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f17bf7fe700 1 -- 172.21.15.92:0/3085122549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5620fa845a90 msgr2=0x5620fa83c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f17bf7fe700 1 --2- 172.21.15.92:0/3085122549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5620fa845a90 0x5620fa83c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f17b6ffd700 1 --2- 172.21.15.92:0/3085122549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa8019a0 0x5620fa8419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f17bf7fe700 1 -- 172.21.15.92:0/3085122549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa8019a0 msgr2=0x5620fa8419c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f17bf7fe700 1 --2- 172.21.15.92:0/3085122549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa8019a0 0x5620fa8419c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f17bf7fe700 1 -- 172.21.15.92:0/3085122549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a8021050 con 0x5620fa7ffbd0 2023-10-06T10:45:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f17b6ffd700 1 --2- 172.21.15.92:0/3085122549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa8019a0 0x5620fa8419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f17bf7fe700 1 --2- 172.21.15.92:0/3085122549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa7ffbd0 0x5620fa829960 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f17b0083390 tx=0x7f17b0082f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5197e17051be2a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f179dffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f179dffb700 1 -- 172.21.15.92:0/3085122549 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b005ca30 con 0x5620fa7ffbd0 2023-10-06T10:45:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f179dffb700 1 -- 172.21.15.92:0/3085122549 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f17b0054f20 con 0x5620fa7ffbd0 2023-10-06T10:45:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f17ceeedb80 1 -- 172.21.15.92:0/3085122549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620fa7f3690 con 0x5620fa7ffbd0 2023-10-06T10:45:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f179dffb700 1 -- 172.21.15.92:0/3085122549 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17b0055800 con 0x5620fa7ffbd0 2023-10-06T10:45:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f17ceeedb80 10 client.8739 fetch_fsmap learned FSMap version 105 2023-10-06T10:45:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f17ceeedb80 1 -- 172.21.15.92:0/3085122549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x5620fa58bfe0 con 0x5620fa7ffbd0 2023-10-06T10:45:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f179dffb700 1 -- 172.21.15.92:0/3085122549 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5620fa7f3690 con 0x5620fa7ffbd0 2023-10-06T10:45:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f179dffb700 1 -- 172.21.15.92:0/3085122549 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 105) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f17b005c540 con 0x5620fa7ffbd0 2023-10-06T10:45:07.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f17ceeedb80 10 client.8739 fetch_fsmap finished waiting for FSMap version 105 2023-10-06T10:45:07.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f17ceeedb80 10 client.8739 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T10:45:07.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f17ceeedb80 20 client.8739 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:07.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f17ceeedb80 1 --2- 172.21.15.92:0/3085122549 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x5620fa826cd0 0x5620fa8270a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f17ceeedb80 4 client.8739 mds_command: new command op to 6719 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:07.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.075+0000 7f17ceeedb80 1 -- 172.21.15.92:0/3085122549 --> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620fa7d05c0 con 0x5620fa826cd0 2023-10-06T10:45:07.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.076+0000 7f17b6ffd700 1 --2- 172.21.15.92:0/3085122549 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x5620fa826cd0 0x5620fa8270a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.076+0000 7f179dffb700 10 client.8739 ms_handle_connect on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:07.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.076+0000 7f17b6ffd700 1 --2- 172.21.15.92:0/3085122549 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x5620fa826cd0 0x5620fa8270a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6e7f53d518ec935 server_cookie=3cbbc63a24b7fe02 in_seq=0 out_seq=0 2023-10-06T10:45:07.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.077+0000 7f179dffb700 1 -- 172.21.15.92:0/3085122549 <== mds.2 v2:172.21.15.124:6833/2307521923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5620fa7d05c0 con 0x5620fa826cd0 2023-10-06T10:45:07.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.077+0000 7f179dffb700 10 client.8739 handle_command_reply: tid=0 2023-10-06T10:45:07.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.077+0000 7f17ceeedb80 1 client.8739 shutdown 2023-10-06T10:45:07.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.077+0000 7f17ceeedb80 1 -- 172.21.15.92:0/3085122549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa7ffbd0 msgr2=0x5620fa829960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.077+0000 7f17ceeedb80 1 --2- 172.21.15.92:0/3085122549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa7ffbd0 0x5620fa829960 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f17b0083390 tx=0x7f17b0082f80 comp rx=0 tx=0).stop 2023-10-06T10:45:07.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.077+0000 7f17ceeedb80 1 -- 172.21.15.92:0/3085122549 shutdown_connections 2023-10-06T10:45:07.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.077+0000 7f17ceeedb80 1 --2- 172.21.15.92:0/3085122549 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x5620fa826cd0 0x5620fa8270a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.077+0000 7f17ceeedb80 1 --2- 172.21.15.92:0/3085122549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5620fa845a90 0x5620fa83c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.077+0000 7f179dffb700 0 client.8739 ms_handle_reset on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:07.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.077+0000 7f17ceeedb80 1 --2- 172.21.15.92:0/3085122549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa7ffbd0 0x5620fa829960 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.077+0000 7f17ceeedb80 1 --2- 172.21.15.92:0/3085122549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa8019a0 0x5620fa8419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.077+0000 7f17ceeedb80 1 -- 172.21.15.92:0/3085122549 >> 172.21.15.92:0/3085122549 conn(0x5620fa800070 msgr2=0x5620fa7faa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:07.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.077+0000 7f17ceeedb80 1 -- 172.21.15.92:0/3085122549 shutdown_connections 2023-10-06T10:45:07.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.077+0000 7f17ceeedb80 1 -- 172.21.15.92:0/3085122549 wait complete. 2023-10-06T10:45:07.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.077+0000 7f17ceeedb80 20 client.8739 trim_cache size 0 max 16384 2023-10-06T10:45:07.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.093+0000 7f17ceeedb80 1 Processor -- start 2023-10-06T10:45:07.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.093+0000 7f17ceeedb80 1 -- start start 2023-10-06T10:45:07.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.094+0000 7f17ceeedb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa7ffbd0 0x5620fa8270a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.094+0000 7f17ceeedb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5620fa8019a0 0x5620fa7f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.094+0000 7f17ceeedb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa845a90 0x5620fa8429e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.094+0000 7f17ceeedb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620fa5d1e30 con 0x5620fa7ffbd0 2023-10-06T10:45:07.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.094+0000 7f17ceeedb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620fa7f90c0 con 0x5620fa845a90 2023-10-06T10:45:07.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.094+0000 7f17ceeedb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620fa827d70 con 0x5620fa8019a0 2023-10-06T10:45:07.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.094+0000 7f17bf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa7ffbd0 0x5620fa8270a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.094+0000 7f17bf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa7ffbd0 0x5620fa8270a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39138/0 (socket says 172.21.15.92:39138) 2023-10-06T10:45:07.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.094+0000 7f17bf7fe700 1 -- 172.21.15.92:0/674798102 learned_addr learned my addr 172.21.15.92:0/674798102 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:07.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.094+0000 7f17b6ffd700 1 --2- 172.21.15.92:0/674798102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5620fa8019a0 0x5620fa7f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.094+0000 7f17bffff700 1 --2- 172.21.15.92:0/674798102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa845a90 0x5620fa8429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.094+0000 7f17bf7fe700 1 -- 172.21.15.92:0/674798102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5620fa8019a0 msgr2=0x5620fa7f6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.094+0000 7f17bf7fe700 1 --2- 172.21.15.92:0/674798102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5620fa8019a0 0x5620fa7f6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.094+0000 7f17bf7fe700 1 -- 172.21.15.92:0/674798102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa845a90 msgr2=0x5620fa8429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.094+0000 7f17bf7fe700 1 --2- 172.21.15.92:0/674798102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa845a90 0x5620fa8429e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.094+0000 7f17bf7fe700 1 -- 172.21.15.92:0/674798102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620fa71d780 con 0x5620fa7ffbd0 2023-10-06T10:45:07.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.094+0000 7f17b6ffd700 1 --2- 172.21.15.92:0/674798102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5620fa8019a0 0x5620fa7f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:07.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.094+0000 7f17bffff700 1 --2- 172.21.15.92:0/674798102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa845a90 0x5620fa8429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:07.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.095+0000 7f17bf7fe700 1 --2- 172.21.15.92:0/674798102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa7ffbd0 0x5620fa8270a0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f17b00559c0 tx=0x7f17b006a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0d5e6cbd30788e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.095+0000 7f17b57fa700 1 -- 172.21.15.92:0/674798102 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b0055800 con 0x5620fa7ffbd0 2023-10-06T10:45:07.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.095+0000 7f17b57fa700 1 -- 172.21.15.92:0/674798102 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b0054f20 con 0x5620fa7ffbd0 2023-10-06T10:45:07.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.095+0000 7f17b57fa700 1 -- 172.21.15.92:0/674798102 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b000ad60 con 0x5620fa7ffbd0 2023-10-06T10:45:07.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.095+0000 7f17ceeedb80 1 -- 172.21.15.92:0/674798102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa7ffbd0 msgr2=0x5620fa8270a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.095+0000 7f17ceeedb80 1 --2- 172.21.15.92:0/674798102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa7ffbd0 0x5620fa8270a0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f17b00559c0 tx=0x7f17b006a000 comp rx=0 tx=0).stop 2023-10-06T10:45:07.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.095+0000 7f17ceeedb80 1 -- 172.21.15.92:0/674798102 shutdown_connections 2023-10-06T10:45:07.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.095+0000 7f17ceeedb80 1 --2- 172.21.15.92:0/674798102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa845a90 0x5620fa8429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.095+0000 7f17ceeedb80 1 --2- 172.21.15.92:0/674798102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa7ffbd0 0x5620fa8270a0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.095+0000 7f17ceeedb80 1 --2- 172.21.15.92:0/674798102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5620fa8019a0 0x5620fa7f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.095+0000 7f17ceeedb80 1 -- 172.21.15.92:0/674798102 >> 172.21.15.92:0/674798102 conn(0x5620fa800070 msgr2=0x5620fa800440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:07.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.095+0000 7f17ceeedb80 1 -- 172.21.15.92:0/674798102 shutdown_connections 2023-10-06T10:45:07.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.095+0000 7f17ceeedb80 1 -- 172.21.15.92:0/674798102 wait complete. 2023-10-06T10:45:07.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.096+0000 7f17ceeedb80 1 -- start start 2023-10-06T10:45:07.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.096+0000 7f17ceeedb80 1 Processor -- start 2023-10-06T10:45:07.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17ceeedb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5620fa7ffbd0 0x5620fa8aa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17ceeedb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa8019a0 0x5620fa7fef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17ceeedb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa845a90 0x5620fa829430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17ceeedb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620fa8920e0 con 0x5620fa8019a0 2023-10-06T10:45:07.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17ceeedb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620fa7fb590 con 0x5620fa845a90 2023-10-06T10:45:07.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17ceeedb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620fa8492a0 con 0x5620fa7ffbd0 2023-10-06T10:45:07.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17ceeedb80 10 client.0 _ll_register_callbacks cb 0x5620fa5ebf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:07.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17b6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa8019a0 0x5620fa7fef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17b6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa8019a0 0x5620fa7fef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39142/0 (socket says 172.21.15.92:39142) 2023-10-06T10:45:07.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17b6ffd700 1 -- 172.21.15.92:0/1647326658 learned_addr learned my addr 172.21.15.92:0/1647326658 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:07.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17bf7fe700 1 --2- 172.21.15.92:0/1647326658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5620fa7ffbd0 0x5620fa8aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17bffff700 1 --2- 172.21.15.92:0/1647326658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa845a90 0x5620fa829430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17b6ffd700 1 -- 172.21.15.92:0/1647326658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5620fa7ffbd0 msgr2=0x5620fa8aa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17b6ffd700 1 --2- 172.21.15.92:0/1647326658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5620fa7ffbd0 0x5620fa8aa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17b6ffd700 1 -- 172.21.15.92:0/1647326658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa845a90 msgr2=0x5620fa829430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17b6ffd700 1 --2- 172.21.15.92:0/1647326658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa845a90 0x5620fa829430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17b6ffd700 1 -- 172.21.15.92:0/1647326658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620fa58bfe0 con 0x5620fa8019a0 2023-10-06T10:45:07.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17bf7fe700 1 --2- 172.21.15.92:0/1647326658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5620fa7ffbd0 0x5620fa8aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:07.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17bffff700 1 --2- 172.21.15.92:0/1647326658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa845a90 0x5620fa829430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:07.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17b6ffd700 1 --2- 172.21.15.92:0/1647326658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa8019a0 0x5620fa7fef90 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f17a8003050 tx=0x7f17a8036250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20af7ccc3f971bcb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f179dffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:07.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f179dffb700 1 -- 172.21.15.92:0/1647326658 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8027670 con 0x5620fa8019a0 2023-10-06T10:45:07.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f179dffb700 1 -- 172.21.15.92:0/1647326658 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f17a80277d0 con 0x5620fa8019a0 2023-10-06T10:45:07.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17ceeedb80 1 -- 172.21.15.92:0/1647326658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17a801cc60 con 0x5620fa8019a0 2023-10-06T10:45:07.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f179dffb700 1 -- 172.21.15.92:0/1647326658 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17a8027990 con 0x5620fa8019a0 2023-10-06T10:45:07.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17ceeedb80 10 client.8745 fetch_fsmap learned FSMap version 105 2023-10-06T10:45:07.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f17ceeedb80 1 -- 172.21.15.92:0/1647326658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x5620fa704470 con 0x5620fa8019a0 2023-10-06T10:45:07.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.097+0000 7f179dffb700 1 -- 172.21.15.92:0/1647326658 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a801cc60 con 0x5620fa8019a0 2023-10-06T10:45:07.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.098+0000 7f179dffb700 1 -- 172.21.15.92:0/1647326658 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 105) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f17a8027af0 con 0x5620fa8019a0 2023-10-06T10:45:07.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.098+0000 7f17ceeedb80 10 client.8745 fetch_fsmap finished waiting for FSMap version 105 2023-10-06T10:45:07.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.098+0000 7f17ceeedb80 10 client.8745 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T10:45:07.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.098+0000 7f17ceeedb80 20 client.8745 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:07.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.098+0000 7f17ceeedb80 1 --2- 172.21.15.92:0/1647326658 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x5620fa82dcd0 0x5620fa82e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.098+0000 7f17ceeedb80 4 client.8745 mds_command: new command op to 6719 tid=0[{"prefix": "status"}] 2023-10-06T10:45:07.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.098+0000 7f17ceeedb80 1 -- 172.21.15.92:0/1647326658 --> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5620fa6f1e00 con 0x5620fa82dcd0 2023-10-06T10:45:07.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.098+0000 7f17bf7fe700 1 --2- 172.21.15.92:0/1647326658 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x5620fa82dcd0 0x5620fa82e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.098+0000 7f17bf7fe700 1 --2- 172.21.15.92:0/1647326658 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x5620fa82dcd0 0x5620fa82e0a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=43702dad65ce06c7 server_cookie=d6c77d8e2aeda37e in_seq=0 out_seq=0 2023-10-06T10:45:07.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.098+0000 7f179dffb700 10 client.8745 ms_handle_connect on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:07.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.098+0000 7f179dffb700 1 -- 172.21.15.92:0/1647326658 <== mds.2 v2:172.21.15.124:6833/2307521923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5620fa6f1e00 con 0x5620fa82dcd0 2023-10-06T10:45:07.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.098+0000 7f179dffb700 10 client.8745 handle_command_reply: tid=0 2023-10-06T10:45:07.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17ceeedb80 1 client.8745 shutdown 2023-10-06T10:45:07.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17ceeedb80 1 -- 172.21.15.92:0/1647326658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa8019a0 msgr2=0x5620fa7fef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17ceeedb80 1 --2- 172.21.15.92:0/1647326658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa8019a0 0x5620fa7fef90 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f17a8003050 tx=0x7f17a8036250 comp rx=0 tx=0).stop 2023-10-06T10:45:07.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17ceeedb80 1 -- 172.21.15.92:0/1647326658 shutdown_connections 2023-10-06T10:45:07.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17ceeedb80 1 --2- 172.21.15.92:0/1647326658 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x5620fa82dcd0 0x5620fa82e0a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17ceeedb80 1 --2- 172.21.15.92:0/1647326658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5620fa845a90 0x5620fa829430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f179dffb700 0 client.8745 ms_handle_reset on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:07.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17ceeedb80 1 --2- 172.21.15.92:0/1647326658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5620fa7ffbd0 0x5620fa8aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17ceeedb80 1 --2- 172.21.15.92:0/1647326658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620fa8019a0 0x5620fa7fef90 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17ceeedb80 1 -- 172.21.15.92:0/1647326658 >> 172.21.15.92:0/1647326658 conn(0x5620fa800070 msgr2=0x5620fa844b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:07.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17ceeedb80 1 -- 172.21.15.92:0/1647326658 shutdown_connections 2023-10-06T10:45:07.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17ceeedb80 1 -- 172.21.15.92:0/1647326658 wait complete. 2023-10-06T10:45:07.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17ceeedb80 20 client.8745 trim_cache size 0 max 16384 2023-10-06T10:45:07.109 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:07.109 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:45:07.109 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 2, 2023-10-06T10:45:07.109 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 6719, 2023-10-06T10:45:07.109 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:45:07.109 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:45:07.109 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:45:07.109 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 1.732289199, 2023-10-06T10:45:07.109 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 105, 2023-10-06T10:45:07.110 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 103, 2023-10-06T10:45:07.110 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 102, 2023-10-06T10:45:07.110 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 21.475522868999999 2023-10-06T10:45:07.110 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:07.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17c5bf5700 1 -- 172.21.15.92:0/2931278864 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f17a0066f30 msgr2=0x7f17a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17c5bf5700 1 --2- 172.21.15.92:0/2931278864 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f17a0066f30 0x7f17a00693e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f17c00c17c0 tx=0x7f17a8017000 comp rx=0 tx=0).stop 2023-10-06T10:45:07.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17c5bf5700 1 -- 172.21.15.92:0/2931278864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17c0068b00 msgr2=0x7f17c00cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17c5bf5700 1 --2- 172.21.15.92:0/2931278864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17c0068b00 0x7f17c00cb130 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f17b000c830 tx=0x7f17b0008040 comp rx=0 tx=0).stop 2023-10-06T10:45:07.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17c5bf5700 1 -- 172.21.15.92:0/2931278864 shutdown_connections 2023-10-06T10:45:07.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17c5bf5700 1 --2- 172.21.15.92:0/2931278864 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f17a0066f30 0x7f17a00693e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17c5bf5700 1 --2- 172.21.15.92:0/2931278864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17c00ad6f0 0x7f17c00c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17c5bf5700 1 --2- 172.21.15.92:0/2931278864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17c0068b00 0x7f17c00cb130 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17c5bf5700 1 --2- 172.21.15.92:0/2931278864 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17c00ad320 0x7f17c0148120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17c5bf5700 1 -- 172.21.15.92:0/2931278864 >> 172.21.15.92:0/2931278864 conn(0x7f17c006dcd0 msgr2=0x7f17c006a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:07.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17c5bf5700 1 -- 172.21.15.92:0/2931278864 shutdown_connections 2023-10-06T10:45:07.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.099+0000 7f17c5bf5700 1 -- 172.21.15.92:0/2931278864 wait complete. 2023-10-06T10:45:07.113 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T10:45:07.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.225+0000 7f10e359e700 1 Processor -- start 2023-10-06T10:45:07.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.225+0000 7f10e359e700 1 -- start start 2023-10-06T10:45:07.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.226+0000 7f10e359e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10e40cbc30 0x7f10e40cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.226+0000 7f10e359e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f10e40b2100 0x7f10e40b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.226+0000 7f10e359e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10e40cb7a0 con 0x7f10e4159440 2023-10-06T10:45:07.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.226+0000 7f10e359e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e40b0860 con 0x7f10e40b2100 2023-10-06T10:45:07.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.226+0000 7f10e359e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e40cb150 con 0x7f10e40cbc30 2023-10-06T10:45:07.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.226+0000 7f10e2d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f10e40b2100 0x7f10e40b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.226+0000 7f10e259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f10e4159440 0x7f10e40b3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51398/0 (socket says 172.21.15.92:51398) 2023-10-06T10:45:07.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.226+0000 7f10e2d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f10e40b2100 0x7f10e40b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39146/0 (socket says 172.21.15.92:39146) 2023-10-06T10:45:07.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.226+0000 7f10e259c700 1 -- 172.21.15.92:0/1932083140 learned_addr learned my addr 172.21.15.92:0/1932083140 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:07.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.226+0000 7f10d9d9b700 1 --2- 172.21.15.92:0/1932083140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10e40cbc30 0x7f10e40cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.226+0000 7f10e2d9d700 1 -- 172.21.15.92:0/1932083140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10e40cbc30 msgr2=0x7f10e40cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.226+0000 7f10e2d9d700 1 --2- 172.21.15.92:0/1932083140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10e40cbc30 0x7f10e40cc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.226+0000 7f10e2d9d700 1 -- 172.21.15.92:0/1932083140 >> v1:172.21.15.92:6789/0 conn(0x7f10e4159440 legacy=0x7f10e40b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.226+0000 7f10e1d9b700 1 -- 172.21.15.92:0/1932083140 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459526532 0 0) 0x7f10e40cb7a0 con 0x7f10e4159440 2023-10-06T10:45:07.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.226+0000 7f10e2d9d700 1 -- 172.21.15.92:0/1932083140 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e40cb430 con 0x7f10e40b2100 2023-10-06T10:45:07.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.227+0000 7f10e2d9d700 1 --2- 172.21.15.92:0/1932083140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10e40b2100 0x7f10e40b2580 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009e60 tx=0x7f10dc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7bab8384875129 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.227+0000 7f10e1d9b700 1 -- 172.21.15.92:0/1932083140 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc00e070 con 0x7f10e40b2100 2023-10-06T10:45:07.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.227+0000 7f10e1d9b700 1 -- 172.21.15.92:0/1932083140 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10dc00bc30 con 0x7f10e40b2100 2023-10-06T10:45:07.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.227+0000 7f10e1d9b700 1 -- 172.21.15.92:0/1932083140 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc012430 con 0x7f10e40b2100 2023-10-06T10:45:07.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.227+0000 7f10e359e700 1 -- 172.21.15.92:0/1932083140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10e40b2100 msgr2=0x7f10e40b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.227+0000 7f10e359e700 1 --2- 172.21.15.92:0/1932083140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10e40b2100 0x7f10e40b2580 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009e60 tx=0x7f10dc009780 comp rx=0 tx=0).stop 2023-10-06T10:45:07.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.227+0000 7f10e359e700 1 -- 172.21.15.92:0/1932083140 shutdown_connections 2023-10-06T10:45:07.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.227+0000 7f10e359e700 1 --2- 172.21.15.92:0/1932083140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10e40b2100 0x7f10e40b2580 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.227+0000 7f10e359e700 1 --2- 172.21.15.92:0/1932083140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10e40cbc30 0x7f10e40cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.227+0000 7f10e359e700 1 -- 172.21.15.92:0/1932083140 >> 172.21.15.92:0/1932083140 conn(0x7f10e406ece0 msgr2=0x7f10e406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:07.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.227+0000 7f10e359e700 1 -- 172.21.15.92:0/1932083140 shutdown_connections 2023-10-06T10:45:07.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.227+0000 7f10e359e700 1 -- 172.21.15.92:0/1932083140 wait complete. 2023-10-06T10:45:07.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.229+0000 7f10e359e700 1 Processor -- start 2023-10-06T10:45:07.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.229+0000 7f10e359e700 1 -- start start 2023-10-06T10:45:07.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.229+0000 7f10e359e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10e40b2100 0x7f10e415db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.229+0000 7f10e359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10e40cbc30 0x7f10e415e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.229+0000 7f10e359e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10e4159440 0x7f10e40c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.229+0000 7f10e359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e40c7f90 con 0x7f10e40cbc30 2023-10-06T10:45:07.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.229+0000 7f10e359e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e40c8270 con 0x7f10e4159440 2023-10-06T10:45:07.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.229+0000 7f10e359e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e40c8550 con 0x7f10e40b2100 2023-10-06T10:45:07.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.229+0000 7f10d9d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10e40cbc30 0x7f10e415e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.229+0000 7f10d9d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10e40cbc30 0x7f10e415e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39154/0 (socket says 172.21.15.92:39154) 2023-10-06T10:45:07.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.230+0000 7f10d9d9b700 1 -- 172.21.15.92:0/1994619284 learned_addr learned my addr 172.21.15.92:0/1994619284 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:07.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.230+0000 7f10e259c700 1 --2- 172.21.15.92:0/1994619284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10e40b2100 0x7f10e415db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.230+0000 7f10e2d9d700 1 --2- 172.21.15.92:0/1994619284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10e4159440 0x7f10e40c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.230+0000 7f10d9d9b700 1 -- 172.21.15.92:0/1994619284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10e40b2100 msgr2=0x7f10e415db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.230+0000 7f10d9d9b700 1 --2- 172.21.15.92:0/1994619284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10e40b2100 0x7f10e415db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.230+0000 7f10d9d9b700 1 -- 172.21.15.92:0/1994619284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10e4159440 msgr2=0x7f10e40c72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.230+0000 7f10d9d9b700 1 --2- 172.21.15.92:0/1994619284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10e4159440 0x7f10e40c72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.230+0000 7f10d9d9b700 1 -- 172.21.15.92:0/1994619284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10dc009a20 con 0x7f10e40cbc30 2023-10-06T10:45:07.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.230+0000 7f10d9d9b700 1 --2- 172.21.15.92:0/1994619284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10e40cbc30 0x7f10e415e060 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00c7d0 tx=0x7f10cc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.230+0000 7f10daffd700 1 -- 172.21.15.92:0/1994619284 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc011600 con 0x7f10e40cbc30 2023-10-06T10:45:07.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.230+0000 7f10daffd700 1 -- 172.21.15.92:0/1994619284 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10cc011760 con 0x7f10e40cbc30 2023-10-06T10:45:07.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.230+0000 7f10e359e700 1 -- 172.21.15.92:0/1994619284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e40cdf90 con 0x7f10e40cbc30 2023-10-06T10:45:07.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.230+0000 7f10daffd700 1 -- 172.21.15.92:0/1994619284 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc011b20 con 0x7f10e40cbc30 2023-10-06T10:45:07.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.230+0000 7f10e359e700 1 -- 172.21.15.92:0/1994619284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e4155cb0 con 0x7f10e40cbc30 2023-10-06T10:45:07.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.231+0000 7f10daffd700 1 -- 172.21.15.92:0/1994619284 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f10cc0193f0 con 0x7f10e40cbc30 2023-10-06T10:45:07.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.232+0000 7f10daffd700 1 --2- 172.21.15.92:0/1994619284 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f10c4066f30 0x7f10c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.232+0000 7f10daffd700 1 -- 172.21.15.92:0/1994619284 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f10cc084bb0 con 0x7f10e40cbc30 2023-10-06T10:45:07.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.232+0000 7f10e259c700 1 --2- 172.21.15.92:0/1994619284 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f10c4066f30 0x7f10c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.232+0000 7f10e259c700 1 --2- 172.21.15.92:0/1994619284 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f10c4066f30 0x7f10c40693e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f10d4007160 tx=0x7f10d4007f50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.234+0000 7f10f1eefb80 1 Processor -- start 2023-10-06T10:45:07.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.234+0000 7f10f1eefb80 1 -- start start 2023-10-06T10:45:07.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.234+0000 7f10f1eefb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e23974f950 0x55e23974fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.234+0000 7f10f1eefb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e239799bc0 0x55e239778920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.234+0000 7f10f1eefb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e239778e60 0x55e23974b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.234+0000 7f10f1eefb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2396cfc80 con 0x55e239778e60 2023-10-06T10:45:07.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.234+0000 7f10f1eefb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2396a6a70 con 0x55e239799bc0 2023-10-06T10:45:07.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.234+0000 7f10f1eefb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2396a4270 con 0x55e23974f950 2023-10-06T10:45:07.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.235+0000 7f10e2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e239778e60 0x55e23974b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.235+0000 7f10e2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e239778e60 0x55e23974b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39156/0 (socket says 172.21.15.92:39156) 2023-10-06T10:45:07.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.235+0000 7f10e2d9d700 1 -- 172.21.15.92:0/2737920001 learned_addr learned my addr 172.21.15.92:0/2737920001 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:07.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.235+0000 7f10e259c700 1 --2- 172.21.15.92:0/2737920001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e23974f950 0x55e23974fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.235+0000 7f10d9d9b700 1 --2- 172.21.15.92:0/2737920001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e239799bc0 0x55e239778920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.235+0000 7f10e2d9d700 1 -- 172.21.15.92:0/2737920001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e23974f950 msgr2=0x55e23974fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.235+0000 7f10e2d9d700 1 --2- 172.21.15.92:0/2737920001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e23974f950 0x55e23974fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.235+0000 7f10e2d9d700 1 -- 172.21.15.92:0/2737920001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e239799bc0 msgr2=0x55e239778920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.235+0000 7f10e2d9d700 1 --2- 172.21.15.92:0/2737920001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e239799bc0 0x55e239778920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.235+0000 7f10e2d9d700 1 -- 172.21.15.92:0/2737920001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e23953a270 con 0x55e239778e60 2023-10-06T10:45:07.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.235+0000 7f10e259c700 1 --2- 172.21.15.92:0/2737920001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e23974f950 0x55e23974fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:07.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.235+0000 7f10d9d9b700 1 --2- 172.21.15.92:0/2737920001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e239799bc0 0x55e239778920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:07.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.235+0000 7f10e2d9d700 1 --2- 172.21.15.92:0/2737920001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e239778e60 0x55e23974b990 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f10e4069150 tx=0x7f10dc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1028ccad8ec9c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.235+0000 7f10c37fe700 1 -- 172.21.15.92:0/2737920001 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc0127c0 con 0x55e239778e60 2023-10-06T10:45:07.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.235+0000 7f10c37fe700 1 -- 172.21.15.92:0/2737920001 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10dc012920 con 0x55e239778e60 2023-10-06T10:45:07.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.235+0000 7f10c37fe700 1 -- 172.21.15.92:0/2737920001 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc012ca0 con 0x55e239778e60 2023-10-06T10:45:07.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.236+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2737920001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e239778e60 msgr2=0x55e23974b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.236+0000 7f10f1eefb80 1 --2- 172.21.15.92:0/2737920001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e239778e60 0x55e23974b990 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f10e4069150 tx=0x7f10dc019000 comp rx=0 tx=0).stop 2023-10-06T10:45:07.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.236+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2737920001 shutdown_connections 2023-10-06T10:45:07.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.236+0000 7f10f1eefb80 1 --2- 172.21.15.92:0/2737920001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e239778e60 0x55e23974b990 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.236+0000 7f10f1eefb80 1 --2- 172.21.15.92:0/2737920001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e23974f950 0x55e23974fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.236+0000 7f10f1eefb80 1 --2- 172.21.15.92:0/2737920001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e239799bc0 0x55e239778920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.236+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2737920001 >> 172.21.15.92:0/2737920001 conn(0x55e239751270 msgr2=0x55e239751640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:07.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.236+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2737920001 shutdown_connections 2023-10-06T10:45:07.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.236+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2737920001 wait complete. 2023-10-06T10:45:07.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.236+0000 7f10f1eefb80 1 -- start start 2023-10-06T10:45:07.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.237+0000 7f10f1eefb80 1 Processor -- start 2023-10-06T10:45:07.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.237+0000 7f10f1eefb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e23974f950 0x55e239790860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.237+0000 7f10f1eefb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e239778e60 0x55e239793a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.237+0000 7f10f1eefb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e239799bc0 0x55e2397a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.237+0000 7f10f1eefb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2396a0060 con 0x55e239799bc0 2023-10-06T10:45:07.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.237+0000 7f10f1eefb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e23964b890 con 0x55e23974f950 2023-10-06T10:45:07.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.237+0000 7f10f1eefb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e239666ea0 con 0x55e239778e60 2023-10-06T10:45:07.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.237+0000 7f10f1eefb80 10 client.0 _ll_register_callbacks cb 0x55e23953cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.238+0000 7f10e2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e239799bc0 0x55e2397a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.238+0000 7f10e2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e239799bc0 0x55e2397a5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39168/0 (socket says 172.21.15.92:39168) 2023-10-06T10:45:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.238+0000 7f10e2d9d700 1 -- 172.21.15.92:0/2503430924 learned_addr learned my addr 172.21.15.92:0/2503430924 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.238+0000 7f10e259c700 1 --2- 172.21.15.92:0/2503430924 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e23974f950 0x55e239790860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.238+0000 7f10d9d9b700 1 --2- 172.21.15.92:0/2503430924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e239778e60 0x55e239793a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.238+0000 7f10e2d9d700 1 -- 172.21.15.92:0/2503430924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e239778e60 msgr2=0x55e239793a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.238+0000 7f10e2d9d700 1 --2- 172.21.15.92:0/2503430924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e239778e60 0x55e239793a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.238+0000 7f10e2d9d700 1 -- 172.21.15.92:0/2503430924 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e23974f950 msgr2=0x55e239790860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.238+0000 7f10e2d9d700 1 --2- 172.21.15.92:0/2503430924 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e23974f950 0x55e239790860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.238+0000 7f10e2d9d700 1 -- 172.21.15.92:0/2503430924 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e40cb430 con 0x55e239799bc0 2023-10-06T10:45:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.238+0000 7f10e2d9d700 1 --2- 172.21.15.92:0/2503430924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e239799bc0 0x55e2397a5b20 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00b6c0 tx=0x7f10dc021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327bf02983b95f94 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.238+0000 7f10c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.238+0000 7f10c0ff9700 1 -- 172.21.15.92:0/2503430924 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc027850 con 0x55e239799bc0 2023-10-06T10:45:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.238+0000 7f10c0ff9700 1 -- 172.21.15.92:0/2503430924 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f10dc0279b0 con 0x55e239799bc0 2023-10-06T10:45:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.238+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2503430924 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e239744690 con 0x55e239799bc0 2023-10-06T10:45:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.238+0000 7f10c0ff9700 1 -- 172.21.15.92:0/2503430924 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10dc027d80 con 0x55e239799bc0 2023-10-06T10:45:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.238+0000 7f10f1eefb80 10 client.8760 fetch_fsmap learned FSMap version 105 2023-10-06T10:45:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.238+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2503430924 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x55e2394dcfe0 con 0x55e239799bc0 2023-10-06T10:45:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.238+0000 7f10c0ff9700 1 -- 172.21.15.92:0/2503430924 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e239744690 con 0x55e239799bc0 2023-10-06T10:45:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.238+0000 7f10c0ff9700 1 -- 172.21.15.92:0/2503430924 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 105) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f10dc033030 con 0x55e239799bc0 2023-10-06T10:45:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.239+0000 7f10f1eefb80 10 client.8760 fetch_fsmap finished waiting for FSMap version 105 2023-10-06T10:45:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.239+0000 7f10f1eefb80 10 client.8760 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T10:45:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.239+0000 7f10f1eefb80 20 client.8760 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.239+0000 7f10f1eefb80 1 --2- 172.21.15.92:0/2503430924 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x55e239775370 0x55e239791710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.239+0000 7f10f1eefb80 4 client.8760 mds_command: new command op to 8517 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.239+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2503430924 --> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2397215c0 con 0x55e239775370 2023-10-06T10:45:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.239+0000 7f10e259c700 1 --2- 172.21.15.92:0/2503430924 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x55e239775370 0x55e239791710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.239+0000 7f10c0ff9700 10 client.8760 ms_handle_connect on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.239+0000 7f10e259c700 1 --2- 172.21.15.92:0/2503430924 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x55e239775370 0x55e239791710 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c112dc2a762c448 server_cookie=dbbc5c6267bc8ddb in_seq=0 out_seq=0 2023-10-06T10:45:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.239+0000 7f10c0ff9700 1 -- 172.21.15.92:0/2503430924 <== mds.1 v2:172.21.15.92:6834/1979826278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55e2397215c0 con 0x55e239775370 2023-10-06T10:45:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.239+0000 7f10c0ff9700 10 client.8760 handle_command_reply: tid=0 2023-10-06T10:45:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.239+0000 7f10f1eefb80 1 client.8760 shutdown 2023-10-06T10:45:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.240+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2503430924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e239799bc0 msgr2=0x55e2397a5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.240+0000 7f10f1eefb80 1 --2- 172.21.15.92:0/2503430924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e239799bc0 0x55e2397a5b20 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00b6c0 tx=0x7f10dc021040 comp rx=0 tx=0).stop 2023-10-06T10:45:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.240+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2503430924 shutdown_connections 2023-10-06T10:45:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.240+0000 7f10f1eefb80 1 --2- 172.21.15.92:0/2503430924 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x55e239775370 0x55e239791710 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.240+0000 7f10f1eefb80 1 --2- 172.21.15.92:0/2503430924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e239799bc0 0x55e2397a5b20 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.240+0000 7f10f1eefb80 1 --2- 172.21.15.92:0/2503430924 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e23974f950 0x55e239790860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.240+0000 7f10c0ff9700 0 client.8760 ms_handle_reset on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.240+0000 7f10f1eefb80 1 --2- 172.21.15.92:0/2503430924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e239778e60 0x55e239793a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.240+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2503430924 >> 172.21.15.92:0/2503430924 conn(0x55e239751270 msgr2=0x55e2397461a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.240+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2503430924 shutdown_connections 2023-10-06T10:45:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.240+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2503430924 wait complete. 2023-10-06T10:45:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.240+0000 7f10f1eefb80 20 client.8760 trim_cache size 0 max 16384 2023-10-06T10:45:07.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.256+0000 7f10f1eefb80 1 Processor -- start 2023-10-06T10:45:07.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.256+0000 7f10f1eefb80 1 -- start start 2023-10-06T10:45:07.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.257+0000 7f10f1eefb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e23974f950 0x55e2397782b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.257+0000 7f10f1eefb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e239778e60 0x55e239746280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.257+0000 7f10f1eefb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e239799bc0 0x55e2397a5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.257+0000 7f10f1eefb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e239522e30 con 0x55e23974f950 2023-10-06T10:45:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.257+0000 7f10f1eefb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e23974fdb0 con 0x55e239778e60 2023-10-06T10:45:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.257+0000 7f10f1eefb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e239751780 con 0x55e239799bc0 2023-10-06T10:45:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.257+0000 7f10e259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e23974f950 0x55e2397782b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.257+0000 7f10e259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e23974f950 0x55e2397782b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39180/0 (socket says 172.21.15.92:39180) 2023-10-06T10:45:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.257+0000 7f10e259c700 1 -- 172.21.15.92:0/2933412238 learned_addr learned my addr 172.21.15.92:0/2933412238 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.257+0000 7f10d9d9b700 1 --2- 172.21.15.92:0/2933412238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e239778e60 0x55e239746280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.257+0000 7f10e2d9d700 1 --2- 172.21.15.92:0/2933412238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e239799bc0 0x55e2397a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.258+0000 7f10e259c700 1 -- 172.21.15.92:0/2933412238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e239799bc0 msgr2=0x55e2397a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.258+0000 7f10e259c700 1 --2- 172.21.15.92:0/2933412238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e239799bc0 0x55e2397a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.258+0000 7f10e259c700 1 -- 172.21.15.92:0/2933412238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e239778e60 msgr2=0x55e239746280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.258+0000 7f10e259c700 1 --2- 172.21.15.92:0/2933412238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e239778e60 0x55e239746280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.258+0000 7f10e259c700 1 -- 172.21.15.92:0/2933412238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e23966e780 con 0x55e23974f950 2023-10-06T10:45:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.258+0000 7f10e259c700 1 --2- 172.21.15.92:0/2933412238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e23974f950 0x55e2397782b0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f10d40061c0 tx=0x7f10d4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b96c484602662d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.258+0000 7f10c3fff700 1 -- 172.21.15.92:0/2933412238 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d4005e30 con 0x55e23974f950 2023-10-06T10:45:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.258+0000 7f10c3fff700 1 -- 172.21.15.92:0/2933412238 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d4001360 con 0x55e23974f950 2023-10-06T10:45:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.258+0000 7f10c3fff700 1 -- 172.21.15.92:0/2933412238 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d40016e0 con 0x55e23974f950 2023-10-06T10:45:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.258+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2933412238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e23974f950 msgr2=0x55e2397782b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.258+0000 7f10f1eefb80 1 --2- 172.21.15.92:0/2933412238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e23974f950 0x55e2397782b0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f10d40061c0 tx=0x7f10d4013000 comp rx=0 tx=0).stop 2023-10-06T10:45:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.258+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2933412238 shutdown_connections 2023-10-06T10:45:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.258+0000 7f10f1eefb80 1 --2- 172.21.15.92:0/2933412238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e239799bc0 0x55e2397a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.258+0000 7f10f1eefb80 1 --2- 172.21.15.92:0/2933412238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e23974f950 0x55e2397782b0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.258+0000 7f10f1eefb80 1 --2- 172.21.15.92:0/2933412238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e239778e60 0x55e239746280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.258+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2933412238 >> 172.21.15.92:0/2933412238 conn(0x55e239751270 msgr2=0x55e23974d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.258+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2933412238 shutdown_connections 2023-10-06T10:45:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.259+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2933412238 wait complete. 2023-10-06T10:45:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.259+0000 7f10f1eefb80 1 -- start start 2023-10-06T10:45:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.259+0000 7f10f1eefb80 1 Processor -- start 2023-10-06T10:45:07.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.260+0000 7f10f1eefb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e23974f950 0x55e2397f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.260+0000 7f10f1eefb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e239778e60 0x55e2397a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.260+0000 7f10f1eefb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e239799bc0 0x55e23979a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.260+0000 7f10f1eefb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e239792010 con 0x55e23974f950 2023-10-06T10:45:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.260+0000 7f10f1eefb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2397a9b80 con 0x55e239778e60 2023-10-06T10:45:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.260+0000 7f10f1eefb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2397fc840 con 0x55e239799bc0 2023-10-06T10:45:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.260+0000 7f10f1eefb80 10 client.0 _ll_register_callbacks cb 0x55e23953cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.260+0000 7f10e259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e23974f950 0x55e2397f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.260+0000 7f10e259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e23974f950 0x55e2397f3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39182/0 (socket says 172.21.15.92:39182) 2023-10-06T10:45:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.260+0000 7f10e259c700 1 -- 172.21.15.92:0/2057348458 learned_addr learned my addr 172.21.15.92:0/2057348458 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.260+0000 7f10d9d9b700 1 --2- 172.21.15.92:0/2057348458 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e239778e60 0x55e2397a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.260+0000 7f10e2d9d700 1 --2- 172.21.15.92:0/2057348458 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e239799bc0 0x55e23979a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.260+0000 7f10e259c700 1 -- 172.21.15.92:0/2057348458 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e239799bc0 msgr2=0x55e23979a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.260+0000 7f10e259c700 1 --2- 172.21.15.92:0/2057348458 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e239799bc0 0x55e23979a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.260+0000 7f10e259c700 1 -- 172.21.15.92:0/2057348458 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e239778e60 msgr2=0x55e2397a8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.260+0000 7f10e259c700 1 --2- 172.21.15.92:0/2057348458 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e239778e60 0x55e2397a8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.260+0000 7f10e259c700 1 -- 172.21.15.92:0/2057348458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d4006030 con 0x55e23974f950 2023-10-06T10:45:07.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.260+0000 7f10d9d9b700 1 --2- 172.21.15.92:0/2057348458 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e239778e60 0x55e2397a8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:07.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.260+0000 7f10e259c700 1 --2- 172.21.15.92:0/2057348458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e23974f950 0x55e2397f3e20 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x55e239752610 tx=0x7f10d4025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5e18c3afdfe2cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.260+0000 7f10c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:07.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.261+0000 7f10c0ff9700 1 -- 172.21.15.92:0/2057348458 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d40014d0 con 0x55e23974f950 2023-10-06T10:45:07.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.261+0000 7f10c0ff9700 1 -- 172.21.15.92:0/2057348458 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f10d401b040 con 0x55e23974f950 2023-10-06T10:45:07.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.261+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2057348458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10dc012920 con 0x55e23974f950 2023-10-06T10:45:07.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.261+0000 7f10c0ff9700 1 -- 172.21.15.92:0/2057348458 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10d402c420 con 0x55e23974f950 2023-10-06T10:45:07.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.261+0000 7f10f1eefb80 10 client.8766 fetch_fsmap learned FSMap version 105 2023-10-06T10:45:07.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.261+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2057348458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x55e239655470 con 0x55e23974f950 2023-10-06T10:45:07.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.261+0000 7f10c0ff9700 1 -- 172.21.15.92:0/2057348458 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10dc012920 con 0x55e23974f950 2023-10-06T10:45:07.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.261+0000 7f10c0ff9700 1 -- 172.21.15.92:0/2057348458 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 105) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f10d4004830 con 0x55e23974f950 2023-10-06T10:45:07.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.261+0000 7f10f1eefb80 10 client.8766 fetch_fsmap finished waiting for FSMap version 105 2023-10-06T10:45:07.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.261+0000 7f10f1eefb80 10 client.8766 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T10:45:07.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.261+0000 7f10f1eefb80 20 client.8766 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:07.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.261+0000 7f10f1eefb80 1 --2- 172.21.15.92:0/2057348458 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x55e239793560 0x55e239793930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.261+0000 7f10f1eefb80 4 client.8766 mds_command: new command op to 8517 tid=0[{"prefix": "status"}] 2023-10-06T10:45:07.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.261+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2057348458 --> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e239642e00 con 0x55e239793560 2023-10-06T10:45:07.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.261+0000 7f10d9d9b700 1 --2- 172.21.15.92:0/2057348458 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x55e239793560 0x55e239793930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.261+0000 7f10c0ff9700 10 client.8766 ms_handle_connect on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:07.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.261+0000 7f10d9d9b700 1 --2- 172.21.15.92:0/2057348458 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x55e239793560 0x55e239793930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ed9b8466c3b87efd server_cookie=94c4f30b494f87f8 in_seq=0 out_seq=0 2023-10-06T10:45:07.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.262+0000 7f10c0ff9700 1 -- 172.21.15.92:0/2057348458 <== mds.1 v2:172.21.15.92:6834/1979826278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55e239642e00 con 0x55e239793560 2023-10-06T10:45:07.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.262+0000 7f10c0ff9700 10 client.8766 handle_command_reply: tid=0 2023-10-06T10:45:07.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.262+0000 7f10f1eefb80 1 client.8766 shutdown 2023-10-06T10:45:07.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.262+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2057348458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e23974f950 msgr2=0x55e2397f3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.262+0000 7f10f1eefb80 1 --2- 172.21.15.92:0/2057348458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e23974f950 0x55e2397f3e20 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x55e239752610 tx=0x7f10d4025000 comp rx=0 tx=0).stop 2023-10-06T10:45:07.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.262+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2057348458 shutdown_connections 2023-10-06T10:45:07.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.262+0000 7f10f1eefb80 1 --2- 172.21.15.92:0/2057348458 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55e239799bc0 0x55e23979a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.262+0000 7f10f1eefb80 1 --2- 172.21.15.92:0/2057348458 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x55e239793560 0x55e239793930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.262+0000 7f10f1eefb80 1 --2- 172.21.15.92:0/2057348458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e23974f950 0x55e2397f3e20 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.262+0000 7f10c0ff9700 0 client.8766 ms_handle_reset on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:07.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.262+0000 7f10f1eefb80 1 --2- 172.21.15.92:0/2057348458 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55e239778e60 0x55e2397a8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.262+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2057348458 >> 172.21.15.92:0/2057348458 conn(0x55e239751270 msgr2=0x55e2397912c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:07.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.262+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2057348458 shutdown_connections 2023-10-06T10:45:07.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.262+0000 7f10f1eefb80 1 -- 172.21.15.92:0/2057348458 wait complete. 2023-10-06T10:45:07.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.262+0000 7f10f1eefb80 20 client.8766 trim_cache size 0 max 16384 2023-10-06T10:45:07.272 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:07.272 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:45:07.272 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 1, 2023-10-06T10:45:07.272 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 8517, 2023-10-06T10:45:07.272 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:45:07.272 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:45:07.273 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:45:07.273 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 2.8991152840000001, 2023-10-06T10:45:07.273 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 105, 2023-10-06T10:45:07.273 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 103, 2023-10-06T10:45:07.273 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 102, 2023-10-06T10:45:07.273 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 21.660560202999999 2023-10-06T10:45:07.273 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:07.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.262+0000 7f10e359e700 1 -- 172.21.15.92:0/1994619284 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f10c4066f30 msgr2=0x7f10c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.262+0000 7f10e359e700 1 --2- 172.21.15.92:0/1994619284 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f10c4066f30 0x7f10c40693e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f10d4007160 tx=0x7f10d4007f50 comp rx=0 tx=0).stop 2023-10-06T10:45:07.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.262+0000 7f10e359e700 1 -- 172.21.15.92:0/1994619284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10e40cbc30 msgr2=0x7f10e415e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.262+0000 7f10e359e700 1 --2- 172.21.15.92:0/1994619284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10e40cbc30 0x7f10e415e060 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00c7d0 tx=0x7f10cc008040 comp rx=0 tx=0).stop 2023-10-06T10:45:07.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.262+0000 7f10e359e700 1 -- 172.21.15.92:0/1994619284 shutdown_connections 2023-10-06T10:45:07.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.263+0000 7f10e359e700 1 --2- 172.21.15.92:0/1994619284 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f10c4066f30 0x7f10c40693e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.263+0000 7f10e359e700 1 --2- 172.21.15.92:0/1994619284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10e4159440 0x7f10e40c72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.263+0000 7f10e359e700 1 --2- 172.21.15.92:0/1994619284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10e40b2100 0x7f10e415db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.263+0000 7f10e359e700 1 --2- 172.21.15.92:0/1994619284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10e40cbc30 0x7f10e415e060 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.263+0000 7f10e359e700 1 -- 172.21.15.92:0/1994619284 >> 172.21.15.92:0/1994619284 conn(0x7f10e406ece0 msgr2=0x7f10e40b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:07.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.263+0000 7f10e359e700 1 -- 172.21.15.92:0/1994619284 shutdown_connections 2023-10-06T10:45:07.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.263+0000 7f10e359e700 1 -- 172.21.15.92:0/1994619284 wait complete. 2023-10-06T10:45:07.276 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T10:45:07.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.392+0000 7f53d03d7700 1 Processor -- start 2023-10-06T10:45:07.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.392+0000 7f53d03d7700 1 -- start start 2023-10-06T10:45:07.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.393+0000 7f53d03d7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f53c8159df0 0x7f53c815a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.393+0000 7f53d03d7700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53c815a700 0x7f53c80cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.393+0000 7f53d03d7700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f53c80cd680 0x7f53c806ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.393+0000 7f53d03d7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c815e020 con 0x7f53c8159df0 2023-10-06T10:45:07.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.393+0000 7f53d03d7700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c80cdac0 con 0x7f53c80cd680 2023-10-06T10:45:07.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.393+0000 7f53d03d7700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c815d4a0 con 0x7f53c815a700 2023-10-06T10:45:07.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.393+0000 7f53ce173700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f53c8159df0 0x7f53c815a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.393+0000 7f53ce173700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f53c8159df0 0x7f53c815a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39184/0 (socket says 172.21.15.92:39184) 2023-10-06T10:45:07.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.393+0000 7f53ce173700 1 -- 172.21.15.92:0/1576698753 learned_addr learned my addr 172.21.15.92:0/1576698753 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:07.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.393+0000 7f53ce974700 1 --2- 172.21.15.92:0/1576698753 >> v2:172.21.15.124:3300/0 conn(0x7f53c80cd680 0x7f53c806ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.393+0000 7f53ce173700 1 -- 172.21.15.92:0/1576698753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53c815a700 msgr2=0x7f53c80cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.393+0000 7f53ce173700 1 --2- 172.21.15.92:0/1576698753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53c815a700 0x7f53c80cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.393+0000 7f53ce173700 1 -- 172.21.15.92:0/1576698753 >> v2:172.21.15.124:3300/0 conn(0x7f53c80cd680 msgr2=0x7f53c806ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.393+0000 7f53ce173700 1 --2- 172.21.15.92:0/1576698753 >> v2:172.21.15.124:3300/0 conn(0x7f53c80cd680 0x7f53c806ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.393+0000 7f53ce173700 1 -- 172.21.15.92:0/1576698753 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c815d780 con 0x7f53c8159df0 2023-10-06T10:45:07.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.394+0000 7f53ce173700 1 --2- 172.21.15.92:0/1576698753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53c8159df0 0x7f53c815a1c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f53c8089830 tx=0x7f53c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e65f3f97586257 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.394+0000 7f53cd171700 1 -- 172.21.15.92:0/1576698753 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c400b860 con 0x7f53c8159df0 2023-10-06T10:45:07.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.394+0000 7f53cd171700 1 -- 172.21.15.92:0/1576698753 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c400b9c0 con 0x7f53c8159df0 2023-10-06T10:45:07.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.394+0000 7f53cd171700 1 -- 172.21.15.92:0/1576698753 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c400bd80 con 0x7f53c8159df0 2023-10-06T10:45:07.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.394+0000 7f53d03d7700 1 -- 172.21.15.92:0/1576698753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53c8159df0 msgr2=0x7f53c815a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.394+0000 7f53d03d7700 1 --2- 172.21.15.92:0/1576698753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53c8159df0 0x7f53c815a1c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f53c8089830 tx=0x7f53c4009250 comp rx=0 tx=0).stop 2023-10-06T10:45:07.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.394+0000 7f53d03d7700 1 -- 172.21.15.92:0/1576698753 shutdown_connections 2023-10-06T10:45:07.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.394+0000 7f53d03d7700 1 --2- 172.21.15.92:0/1576698753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53c8159df0 0x7f53c815a1c0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.394+0000 7f53d03d7700 1 --2- 172.21.15.92:0/1576698753 >> v2:172.21.15.124:3300/0 conn(0x7f53c80cd680 0x7f53c806ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.394+0000 7f53d03d7700 1 --2- 172.21.15.92:0/1576698753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53c815a700 0x7f53c80cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.394+0000 7f53d03d7700 1 -- 172.21.15.92:0/1576698753 >> 172.21.15.92:0/1576698753 conn(0x7f53c80707d0 msgr2=0x7f53c8070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:07.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.394+0000 7f53d03d7700 1 -- 172.21.15.92:0/1576698753 shutdown_connections 2023-10-06T10:45:07.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.394+0000 7f53d03d7700 1 -- 172.21.15.92:0/1576698753 wait complete. 2023-10-06T10:45:07.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.395+0000 7f53d03d7700 1 Processor -- start 2023-10-06T10:45:07.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.395+0000 7f53d03d7700 1 -- start start 2023-10-06T10:45:07.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.396+0000 7f53d03d7700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53c80cd680 0x7f53c80c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.396+0000 7f53d03d7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53c8159df0 0x7f53c80b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.396+0000 7f53d03d7700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53c815a700 0x7f53c8158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.396+0000 7f53d03d7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c80c44b0 con 0x7f53c8159df0 2023-10-06T10:45:07.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.396+0000 7f53d03d7700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c80c4c80 con 0x7f53c80cd680 2023-10-06T10:45:07.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.396+0000 7f53d03d7700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c80c46c0 con 0x7f53c815a700 2023-10-06T10:45:07.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.396+0000 7f53cd972700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53c8159df0 0x7f53c80b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.396+0000 7f53cd972700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53c8159df0 0x7f53c80b06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39194/0 (socket says 172.21.15.92:39194) 2023-10-06T10:45:07.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.396+0000 7f53cd972700 1 -- 172.21.15.92:0/3766388831 learned_addr learned my addr 172.21.15.92:0/3766388831 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:07.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.396+0000 7f53ce173700 1 --2- 172.21.15.92:0/3766388831 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53c80cd680 0x7f53c80c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.396+0000 7f53ce974700 1 --2- 172.21.15.92:0/3766388831 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53c815a700 0x7f53c8158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.397+0000 7f53cd972700 1 -- 172.21.15.92:0/3766388831 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53c815a700 msgr2=0x7f53c8158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.397+0000 7f53cd972700 1 --2- 172.21.15.92:0/3766388831 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53c815a700 0x7f53c8158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.397+0000 7f53cd972700 1 -- 172.21.15.92:0/3766388831 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53c80cd680 msgr2=0x7f53c80c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.397+0000 7f53cd972700 1 --2- 172.21.15.92:0/3766388831 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53c80cd680 0x7f53c80c6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.397+0000 7f53cd972700 1 -- 172.21.15.92:0/3766388831 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c40094f0 con 0x7f53c8159df0 2023-10-06T10:45:07.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.397+0000 7f53cd972700 1 --2- 172.21.15.92:0/3766388831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53c8159df0 0x7f53c80b06b0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f53b800b810 tx=0x7f53b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.397+0000 7f53be7fc700 1 -- 172.21.15.92:0/3766388831 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b800f510 con 0x7f53c8159df0 2023-10-06T10:45:07.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.397+0000 7f53be7fc700 1 -- 172.21.15.92:0/3766388831 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b800f690 con 0x7f53c8159df0 2023-10-06T10:45:07.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.397+0000 7f53d03d7700 1 -- 172.21.15.92:0/3766388831 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c80c49a0 con 0x7f53c8159df0 2023-10-06T10:45:07.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.397+0000 7f53be7fc700 1 -- 172.21.15.92:0/3766388831 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b800fa50 con 0x7f53c8159df0 2023-10-06T10:45:07.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.397+0000 7f53d03d7700 1 -- 172.21.15.92:0/3766388831 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c80c6430 con 0x7f53c8159df0 2023-10-06T10:45:07.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.398+0000 7f53be7fc700 1 -- 172.21.15.92:0/3766388831 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f53b801a400 con 0x7f53c8159df0 2023-10-06T10:45:07.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.399+0000 7f53be7fc700 1 --2- 172.21.15.92:0/3766388831 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f53ac066f30 0x7f53ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.399+0000 7f53be7fc700 1 -- 172.21.15.92:0/3766388831 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f53b8084330 con 0x7f53c8159df0 2023-10-06T10:45:07.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.399+0000 7f53ce173700 1 --2- 172.21.15.92:0/3766388831 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f53ac066f30 0x7f53ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.399+0000 7f53ce173700 1 --2- 172.21.15.92:0/3766388831 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f53ac066f30 0x7f53ac0693e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f53c8089830 tx=0x7f53c4017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.401+0000 7f53d96cfb80 1 Processor -- start 2023-10-06T10:45:07.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.401+0000 7f53d96cfb80 1 -- start start 2023-10-06T10:45:07.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.401+0000 7f53d96cfb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5563a1b9aa90 0x5563a1b9ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.401+0000 7f53d96cfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b569a0 0x5563a1b53580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.401+0000 7f53d96cfb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5563a1b54bd0 0x5563a1b53ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.401+0000 7f53d96cfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563a1ad3c80 con 0x5563a1b569a0 2023-10-06T10:45:07.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.401+0000 7f53d96cfb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563a1aaaa70 con 0x5563a1b54bd0 2023-10-06T10:45:07.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.401+0000 7f53d96cfb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563a1aa8270 con 0x5563a1b9aa90 2023-10-06T10:45:07.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.401+0000 7f53cd972700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b569a0 0x5563a1b53580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.401+0000 7f53cd972700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b569a0 0x5563a1b53580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39196/0 (socket says 172.21.15.92:39196) 2023-10-06T10:45:07.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.401+0000 7f53cd972700 1 -- 172.21.15.92:0/146777116 learned_addr learned my addr 172.21.15.92:0/146777116 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:07.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.402+0000 7f53ce974700 1 --2- 172.21.15.92:0/146777116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5563a1b54bd0 0x5563a1b53ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.402+0000 7f53ce173700 1 --2- 172.21.15.92:0/146777116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5563a1b9aa90 0x5563a1b9ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.402+0000 7f53cd972700 1 -- 172.21.15.92:0/146777116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5563a1b9aa90 msgr2=0x5563a1b9ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.402+0000 7f53cd972700 1 --2- 172.21.15.92:0/146777116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5563a1b9aa90 0x5563a1b9ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.402+0000 7f53cd972700 1 -- 172.21.15.92:0/146777116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5563a1b54bd0 msgr2=0x5563a1b53ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.402+0000 7f53cd972700 1 --2- 172.21.15.92:0/146777116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5563a1b54bd0 0x5563a1b53ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.402+0000 7f53cd972700 1 -- 172.21.15.92:0/146777116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563a193e270 con 0x5563a1b569a0 2023-10-06T10:45:07.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.402+0000 7f53ce173700 1 --2- 172.21.15.92:0/146777116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5563a1b9aa90 0x5563a1b9ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:07.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.402+0000 7f53cd972700 1 --2- 172.21.15.92:0/146777116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b569a0 0x5563a1b53580 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f53b8054ef0 tx=0x7f53b8082430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2530e968c7bff1c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.402+0000 7f53ab7fe700 1 -- 172.21.15.92:0/146777116 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b80581b0 con 0x5563a1b569a0 2023-10-06T10:45:07.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.402+0000 7f53ab7fe700 1 -- 172.21.15.92:0/146777116 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b805bf00 con 0x5563a1b569a0 2023-10-06T10:45:07.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.402+0000 7f53ab7fe700 1 -- 172.21.15.92:0/146777116 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b80840d0 con 0x5563a1b569a0 2023-10-06T10:45:07.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.403+0000 7f53d96cfb80 1 -- 172.21.15.92:0/146777116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b569a0 msgr2=0x5563a1b53580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.403+0000 7f53d96cfb80 1 --2- 172.21.15.92:0/146777116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b569a0 0x5563a1b53580 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f53b8054ef0 tx=0x7f53b8082430 comp rx=0 tx=0).stop 2023-10-06T10:45:07.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.403+0000 7f53d96cfb80 1 -- 172.21.15.92:0/146777116 shutdown_connections 2023-10-06T10:45:07.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.403+0000 7f53d96cfb80 1 --2- 172.21.15.92:0/146777116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5563a1b54bd0 0x5563a1b53ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.403+0000 7f53d96cfb80 1 --2- 172.21.15.92:0/146777116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5563a1b9aa90 0x5563a1b9ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.403+0000 7f53d96cfb80 1 --2- 172.21.15.92:0/146777116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b569a0 0x5563a1b53580 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.403+0000 7f53d96cfb80 1 -- 172.21.15.92:0/146777116 >> 172.21.15.92:0/146777116 conn(0x5563a1b55070 msgr2=0x5563a1b55440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:07.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.403+0000 7f53d96cfb80 1 -- 172.21.15.92:0/146777116 shutdown_connections 2023-10-06T10:45:07.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.403+0000 7f53d96cfb80 1 -- 172.21.15.92:0/146777116 wait complete. 2023-10-06T10:45:07.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.403+0000 7f53d96cfb80 1 -- start start 2023-10-06T10:45:07.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.403+0000 7f53d96cfb80 1 Processor -- start 2023-10-06T10:45:07.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.404+0000 7f53d96cfb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5563a1b54bd0 0x5563a1b7a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.404+0000 7f53d96cfb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5563a1b569a0 0x5563a1b95880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.404+0000 7f53d96cfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b9aa90 0x5563a1b9de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.404+0000 7f53d96cfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563a1aa4060 con 0x5563a1b9aa90 2023-10-06T10:45:07.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.404+0000 7f53d96cfb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563a1a4f890 con 0x5563a1b569a0 2023-10-06T10:45:07.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.404+0000 7f53d96cfb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563a1a6aea0 con 0x5563a1b54bd0 2023-10-06T10:45:07.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.404+0000 7f53d96cfb80 10 client.0 _ll_register_callbacks cb 0x5563a1940f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:07.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.404+0000 7f53ce974700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b9aa90 0x5563a1b9de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.404+0000 7f53ce974700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b9aa90 0x5563a1b9de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39202/0 (socket says 172.21.15.92:39202) 2023-10-06T10:45:07.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.404+0000 7f53ce974700 1 -- 172.21.15.92:0/1883933387 learned_addr learned my addr 172.21.15.92:0/1883933387 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:07.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.404+0000 7f53ce173700 1 --2- 172.21.15.92:0/1883933387 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5563a1b54bd0 0x5563a1b7a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.404+0000 7f53cd972700 1 --2- 172.21.15.92:0/1883933387 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5563a1b569a0 0x5563a1b95880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.405+0000 7f53ce974700 1 -- 172.21.15.92:0/1883933387 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5563a1b54bd0 msgr2=0x5563a1b7a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.405+0000 7f53ce974700 1 --2- 172.21.15.92:0/1883933387 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5563a1b54bd0 0x5563a1b7a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.405+0000 7f53ce974700 1 -- 172.21.15.92:0/1883933387 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5563a1b569a0 msgr2=0x5563a1b95880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.405+0000 7f53ce974700 1 --2- 172.21.15.92:0/1883933387 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5563a1b569a0 0x5563a1b95880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.405+0000 7f53ce974700 1 -- 172.21.15.92:0/1883933387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c80c6430 con 0x5563a1b9aa90 2023-10-06T10:45:07.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.405+0000 7f53ce974700 1 --2- 172.21.15.92:0/1883933387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b9aa90 0x5563a1b9de50 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f53c0012f10 tx=0x7f53c0007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2055b6a3393fa1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.405+0000 7f53a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:07.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.405+0000 7f53a8ff9700 1 -- 172.21.15.92:0/1883933387 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c0007c80 con 0x5563a1b9aa90 2023-10-06T10:45:07.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.405+0000 7f53a8ff9700 1 -- 172.21.15.92:0/1883933387 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f53c0010bb0 con 0x5563a1b9aa90 2023-10-06T10:45:07.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.405+0000 7f53d96cfb80 1 -- 172.21.15.92:0/1883933387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5563a1b48690 con 0x5563a1b9aa90 2023-10-06T10:45:07.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.405+0000 7f53a8ff9700 1 -- 172.21.15.92:0/1883933387 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53c001b420 con 0x5563a1b9aa90 2023-10-06T10:45:07.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.405+0000 7f53d96cfb80 10 client.8778 fetch_fsmap learned FSMap version 105 2023-10-06T10:45:07.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.405+0000 7f53d96cfb80 1 -- 172.21.15.92:0/1883933387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x5563a18e0fe0 con 0x5563a1b9aa90 2023-10-06T10:45:07.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.405+0000 7f53a8ff9700 1 -- 172.21.15.92:0/1883933387 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5563a1b48690 con 0x5563a1b9aa90 2023-10-06T10:45:07.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.405+0000 7f53a8ff9700 1 -- 172.21.15.92:0/1883933387 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 105) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f53c001b580 con 0x5563a1b9aa90 2023-10-06T10:45:07.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.405+0000 7f53d96cfb80 10 client.8778 fetch_fsmap finished waiting for FSMap version 105 2023-10-06T10:45:07.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.405+0000 7f53d96cfb80 10 client.8778 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:45:07.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.405+0000 7f53d96cfb80 20 client.8778 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:07.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.405+0000 7f53d96cfb80 1 --2- 172.21.15.92:0/1883933387 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x5563a1b7dea0 0x5563a1b7e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.405+0000 7f53d96cfb80 4 client.8778 mds_command: new command op to 8520 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:07.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.405+0000 7f53d96cfb80 1 -- 172.21.15.92:0/1883933387 --> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5563a1b255c0 con 0x5563a1b7dea0 2023-10-06T10:45:07.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.406+0000 7f53ce173700 1 --2- 172.21.15.92:0/1883933387 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x5563a1b7dea0 0x5563a1b7e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.406+0000 7f53a8ff9700 10 client.8778 ms_handle_connect on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:07.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.406+0000 7f53ce173700 1 --2- 172.21.15.92:0/1883933387 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x5563a1b7dea0 0x5563a1b7e270 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca0417637d2a0cf8 server_cookie=2e24227ad4179b2f in_seq=0 out_seq=0 2023-10-06T10:45:07.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.406+0000 7f53a8ff9700 1 -- 172.21.15.92:0/1883933387 <== mds.0 v2:172.21.15.92:6835/2298348426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5563a1b255c0 con 0x5563a1b7dea0 2023-10-06T10:45:07.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.406+0000 7f53a8ff9700 10 client.8778 handle_command_reply: tid=0 2023-10-06T10:45:07.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.406+0000 7f53d96cfb80 1 client.8778 shutdown 2023-10-06T10:45:07.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.407+0000 7f53d96cfb80 1 -- 172.21.15.92:0/1883933387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b9aa90 msgr2=0x5563a1b9de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.407+0000 7f53d96cfb80 1 --2- 172.21.15.92:0/1883933387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b9aa90 0x5563a1b9de50 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f53c0012f10 tx=0x7f53c0007710 comp rx=0 tx=0).stop 2023-10-06T10:45:07.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.407+0000 7f53d96cfb80 1 -- 172.21.15.92:0/1883933387 shutdown_connections 2023-10-06T10:45:07.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.407+0000 7f53d96cfb80 1 --2- 172.21.15.92:0/1883933387 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x5563a1b7dea0 0x5563a1b7e270 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.407+0000 7f53d96cfb80 1 --2- 172.21.15.92:0/1883933387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b9aa90 0x5563a1b9de50 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.407+0000 7f53a8ff9700 0 client.8778 ms_handle_reset on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:07.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.407+0000 7f53d96cfb80 1 --2- 172.21.15.92:0/1883933387 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5563a1b54bd0 0x5563a1b7a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.407+0000 7f53d96cfb80 1 --2- 172.21.15.92:0/1883933387 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5563a1b569a0 0x5563a1b95880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.407+0000 7f53d96cfb80 1 -- 172.21.15.92:0/1883933387 >> 172.21.15.92:0/1883933387 conn(0x5563a1b55070 msgr2=0x5563a1b983f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:07.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.407+0000 7f53d96cfb80 1 -- 172.21.15.92:0/1883933387 shutdown_connections 2023-10-06T10:45:07.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.407+0000 7f53d96cfb80 1 -- 172.21.15.92:0/1883933387 wait complete. 2023-10-06T10:45:07.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.407+0000 7f53d96cfb80 20 client.8778 trim_cache size 0 max 16384 2023-10-06T10:45:07.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.424+0000 7f53d96cfb80 1 Processor -- start 2023-10-06T10:45:07.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.424+0000 7f53d96cfb80 1 -- start start 2023-10-06T10:45:07.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.424+0000 7f53d96cfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b54bd0 0x5563a1b984c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.424+0000 7f53d96cfb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5563a1b569a0 0x5563a1b53580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.424+0000 7f53d96cfb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5563a1b9aa90 0x5563a1b53ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.424+0000 7f53d96cfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563a1926e30 con 0x5563a1b54bd0 2023-10-06T10:45:07.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.424+0000 7f53d96cfb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563a1b1bb90 con 0x5563a1b9aa90 2023-10-06T10:45:07.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.424+0000 7f53d96cfb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563a1b7e6c0 con 0x5563a1b569a0 2023-10-06T10:45:07.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.424+0000 7f53ce173700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b54bd0 0x5563a1b984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.424+0000 7f53ce173700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b54bd0 0x5563a1b984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39208/0 (socket says 172.21.15.92:39208) 2023-10-06T10:45:07.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.424+0000 7f53ce173700 1 -- 172.21.15.92:0/1168307113 learned_addr learned my addr 172.21.15.92:0/1168307113 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:07.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.425+0000 7f53cd972700 1 --2- 172.21.15.92:0/1168307113 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5563a1b569a0 0x5563a1b53580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.425+0000 7f53ce974700 1 --2- 172.21.15.92:0/1168307113 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5563a1b9aa90 0x5563a1b53ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.425+0000 7f53ce173700 1 -- 172.21.15.92:0/1168307113 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5563a1b569a0 msgr2=0x5563a1b53580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.425+0000 7f53ce173700 1 --2- 172.21.15.92:0/1168307113 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5563a1b569a0 0x5563a1b53580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.425+0000 7f53ce173700 1 -- 172.21.15.92:0/1168307113 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5563a1b9aa90 msgr2=0x5563a1b53ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.425+0000 7f53ce173700 1 --2- 172.21.15.92:0/1168307113 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5563a1b9aa90 0x5563a1b53ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.425+0000 7f53ce173700 1 -- 172.21.15.92:0/1168307113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563a1b4f3b0 con 0x5563a1b54bd0 2023-10-06T10:45:07.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.425+0000 7f53ce974700 1 --2- 172.21.15.92:0/1168307113 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5563a1b9aa90 0x5563a1b53ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:07.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.425+0000 7f53ce173700 1 --2- 172.21.15.92:0/1168307113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b54bd0 0x5563a1b984c0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f53c4006ce0 tx=0x7f53c4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5cffd377cb666d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.425+0000 7f53abfff700 1 -- 172.21.15.92:0/1168307113 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c4020070 con 0x5563a1b54bd0 2023-10-06T10:45:07.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.425+0000 7f53abfff700 1 -- 172.21.15.92:0/1168307113 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c402e040 con 0x5563a1b54bd0 2023-10-06T10:45:07.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.425+0000 7f53abfff700 1 -- 172.21.15.92:0/1168307113 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c400bd80 con 0x5563a1b54bd0 2023-10-06T10:45:07.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.426+0000 7f53d96cfb80 1 -- 172.21.15.92:0/1168307113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b54bd0 msgr2=0x5563a1b984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.426+0000 7f53d96cfb80 1 --2- 172.21.15.92:0/1168307113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b54bd0 0x5563a1b984c0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f53c4006ce0 tx=0x7f53c4014000 comp rx=0 tx=0).stop 2023-10-06T10:45:07.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.426+0000 7f53d96cfb80 1 -- 172.21.15.92:0/1168307113 shutdown_connections 2023-10-06T10:45:07.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.426+0000 7f53d96cfb80 1 --2- 172.21.15.92:0/1168307113 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5563a1b9aa90 0x5563a1b53ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.426+0000 7f53d96cfb80 1 --2- 172.21.15.92:0/1168307113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b54bd0 0x5563a1b984c0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.426+0000 7f53d96cfb80 1 --2- 172.21.15.92:0/1168307113 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5563a1b569a0 0x5563a1b53580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.426+0000 7f53d96cfb80 1 -- 172.21.15.92:0/1168307113 >> 172.21.15.92:0/1168307113 conn(0x5563a1b55070 msgr2=0x5563a1b99350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:07.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.426+0000 7f53d96cfb80 1 -- 172.21.15.92:0/1168307113 shutdown_connections 2023-10-06T10:45:07.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.426+0000 7f53d96cfb80 1 -- 172.21.15.92:0/1168307113 wait complete. 2023-10-06T10:45:07.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.426+0000 7f53d96cfb80 1 -- start start 2023-10-06T10:45:07.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.426+0000 7f53d96cfb80 1 Processor -- start 2023-10-06T10:45:07.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.427+0000 7f53d96cfb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5563a1b54bd0 0x5563a1b53580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.427+0000 7f53d96cfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b569a0 0x5563a1be5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.427+0000 7f53d96cfb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5563a1b9aa90 0x5563a1be5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.427+0000 7f53d96cfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563a1b7c7e0 con 0x5563a1b569a0 2023-10-06T10:45:07.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.427+0000 7f53d96cfb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563a1b90260 con 0x5563a1b54bd0 2023-10-06T10:45:07.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.427+0000 7f53d96cfb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563a1b54240 con 0x5563a1b9aa90 2023-10-06T10:45:07.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.427+0000 7f53d96cfb80 10 client.0 _ll_register_callbacks cb 0x5563a1940f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:07.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.427+0000 7f53cd972700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b569a0 0x5563a1be5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.427+0000 7f53cd972700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b569a0 0x5563a1be5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39216/0 (socket says 172.21.15.92:39216) 2023-10-06T10:45:07.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.427+0000 7f53cd972700 1 -- 172.21.15.92:0/2640235700 learned_addr learned my addr 172.21.15.92:0/2640235700 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:07.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.427+0000 7f53ce173700 1 --2- 172.21.15.92:0/2640235700 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5563a1b54bd0 0x5563a1b53580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.427+0000 7f53ce974700 1 --2- 172.21.15.92:0/2640235700 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5563a1b9aa90 0x5563a1be5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.428+0000 7f53cd972700 1 -- 172.21.15.92:0/2640235700 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5563a1b9aa90 msgr2=0x5563a1be5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.428+0000 7f53cd972700 1 --2- 172.21.15.92:0/2640235700 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5563a1b9aa90 0x5563a1be5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.428+0000 7f53cd972700 1 -- 172.21.15.92:0/2640235700 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5563a1b54bd0 msgr2=0x5563a1b53580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.428+0000 7f53cd972700 1 --2- 172.21.15.92:0/2640235700 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5563a1b54bd0 0x5563a1b53580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.428+0000 7f53cd972700 1 -- 172.21.15.92:0/2640235700 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c815d780 con 0x5563a1b569a0 2023-10-06T10:45:07.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.428+0000 7f53cd972700 1 --2- 172.21.15.92:0/2640235700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b569a0 0x5563a1be5200 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f53b8054ef0 tx=0x7f53b801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2835494463e5e7d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.428+0000 7f53a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:07.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.428+0000 7f53a8ff9700 1 -- 172.21.15.92:0/2640235700 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b805a070 con 0x5563a1b569a0 2023-10-06T10:45:07.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.428+0000 7f53a8ff9700 1 -- 172.21.15.92:0/2640235700 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f53b805a1d0 con 0x5563a1b569a0 2023-10-06T10:45:07.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.428+0000 7f53d96cfb80 1 -- 172.21.15.92:0/2640235700 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53b805bf00 con 0x5563a1b569a0 2023-10-06T10:45:07.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.428+0000 7f53a8ff9700 1 -- 172.21.15.92:0/2640235700 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53b80551c0 con 0x5563a1b569a0 2023-10-06T10:45:07.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.428+0000 7f53d96cfb80 10 client.8784 fetch_fsmap learned FSMap version 105 2023-10-06T10:45:07.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.428+0000 7f53d96cfb80 1 -- 172.21.15.92:0/2640235700 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x5563a1a72780 con 0x5563a1b569a0 2023-10-06T10:45:07.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.428+0000 7f53a8ff9700 1 -- 172.21.15.92:0/2640235700 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b805bf00 con 0x5563a1b569a0 2023-10-06T10:45:07.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.428+0000 7f53a8ff9700 1 -- 172.21.15.92:0/2640235700 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 105) v1 ==== 3124+0+0 (secure 0 0 0) 0x7f53b8054450 con 0x5563a1b569a0 2023-10-06T10:45:07.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.428+0000 7f53d96cfb80 10 client.8784 fetch_fsmap finished waiting for FSMap version 105 2023-10-06T10:45:07.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.428+0000 7f53d96cfb80 10 client.8784 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T10:45:07.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.428+0000 7f53d96cfb80 20 client.8784 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:07.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.428+0000 7f53d96cfb80 1 --2- 172.21.15.92:0/2640235700 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x5563a1b90880 0x5563a1b90c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.428+0000 7f53d96cfb80 4 client.8784 mds_command: new command op to 8520 tid=0[{"prefix": "status"}] 2023-10-06T10:45:07.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.428+0000 7f53d96cfb80 1 -- 172.21.15.92:0/2640235700 --> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5563a1a46e00 con 0x5563a1b90880 2023-10-06T10:45:07.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.428+0000 7f53ce173700 1 --2- 172.21.15.92:0/2640235700 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x5563a1b90880 0x5563a1b90c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.429+0000 7f53a8ff9700 10 client.8784 ms_handle_connect on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:07.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.429+0000 7f53ce173700 1 --2- 172.21.15.92:0/2640235700 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x5563a1b90880 0x5563a1b90c50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f96472ec7ddc0fd0 server_cookie=65514542dff0bce4 in_seq=0 out_seq=0 2023-10-06T10:45:07.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.429+0000 7f53a8ff9700 1 -- 172.21.15.92:0/2640235700 <== mds.0 v2:172.21.15.92:6835/2298348426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x5563a1a46e00 con 0x5563a1b90880 2023-10-06T10:45:07.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.429+0000 7f53a8ff9700 10 client.8784 handle_command_reply: tid=0 2023-10-06T10:45:07.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.429+0000 7f53d96cfb80 1 client.8784 shutdown 2023-10-06T10:45:07.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.429+0000 7f53d96cfb80 1 -- 172.21.15.92:0/2640235700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b569a0 msgr2=0x5563a1be5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.429+0000 7f53d96cfb80 1 --2- 172.21.15.92:0/2640235700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b569a0 0x5563a1be5200 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f53b8054ef0 tx=0x7f53b801c000 comp rx=0 tx=0).stop 2023-10-06T10:45:07.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.429+0000 7f53d96cfb80 1 -- 172.21.15.92:0/2640235700 shutdown_connections 2023-10-06T10:45:07.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.429+0000 7f53d96cfb80 1 --2- 172.21.15.92:0/2640235700 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x5563a1b90880 0x5563a1b90c50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.429+0000 7f53d96cfb80 1 --2- 172.21.15.92:0/2640235700 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5563a1b9aa90 0x5563a1be5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.429+0000 7f53d96cfb80 1 --2- 172.21.15.92:0/2640235700 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5563a1b54bd0 0x5563a1b53580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.429+0000 7f53a8ff9700 0 client.8784 ms_handle_reset on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:07.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.429+0000 7f53d96cfb80 1 --2- 172.21.15.92:0/2640235700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5563a1b569a0 0x5563a1be5200 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.429+0000 7f53d96cfb80 1 -- 172.21.15.92:0/2640235700 >> 172.21.15.92:0/2640235700 conn(0x5563a1b55070 msgr2=0x5563a1b99350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:07.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.429+0000 7f53d96cfb80 1 -- 172.21.15.92:0/2640235700 shutdown_connections 2023-10-06T10:45:07.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.429+0000 7f53d96cfb80 1 -- 172.21.15.92:0/2640235700 wait complete. 2023-10-06T10:45:07.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.429+0000 7f53d96cfb80 20 client.8784 trim_cache size 0 max 16384 2023-10-06T10:45:07.439 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:07.439 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "8f307bfb-c4cb-4f66-aade-46fffe0c6e0c", 2023-10-06T10:45:07.439 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T10:45:07.440 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 8520, 2023-10-06T10:45:07.440 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T10:45:07.440 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T10:45:07.440 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T10:45:07.440 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 15.708259581, 2023-10-06T10:45:07.440 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 105, 2023-10-06T10:45:07.440 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 104, 2023-10-06T10:45:07.440 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 102, 2023-10-06T10:45:07.440 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 21.826906450999999 2023-10-06T10:45:07.440 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:07.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.430+0000 7f53d03d7700 1 -- 172.21.15.92:0/3766388831 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f53ac066f30 msgr2=0x7f53ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.430+0000 7f53d03d7700 1 --2- 172.21.15.92:0/3766388831 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f53ac066f30 0x7f53ac0693e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f53c8089830 tx=0x7f53c4017000 comp rx=0 tx=0).stop 2023-10-06T10:45:07.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.430+0000 7f53d03d7700 1 -- 172.21.15.92:0/3766388831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53c8159df0 msgr2=0x7f53c80b06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.430+0000 7f53d03d7700 1 --2- 172.21.15.92:0/3766388831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53c8159df0 0x7f53c80b06b0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f53b800b810 tx=0x7f53b8009250 comp rx=0 tx=0).stop 2023-10-06T10:45:07.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.430+0000 7f53d03d7700 1 -- 172.21.15.92:0/3766388831 shutdown_connections 2023-10-06T10:45:07.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.430+0000 7f53d03d7700 1 --2- 172.21.15.92:0/3766388831 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f53ac066f30 0x7f53ac0693e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.430+0000 7f53d03d7700 1 --2- 172.21.15.92:0/3766388831 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53c815a700 0x7f53c8158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.430+0000 7f53d03d7700 1 --2- 172.21.15.92:0/3766388831 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53c80cd680 0x7f53c80c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.430+0000 7f53d03d7700 1 --2- 172.21.15.92:0/3766388831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53c8159df0 0x7f53c80b06b0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.430+0000 7f53d03d7700 1 -- 172.21.15.92:0/3766388831 >> 172.21.15.92:0/3766388831 conn(0x7f53c80707d0 msgr2=0x7f53c8063410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:07.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.430+0000 7f53d03d7700 1 -- 172.21.15.92:0/3766388831 shutdown_connections 2023-10-06T10:45:07.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.430+0000 7f53d03d7700 1 -- 172.21.15.92:0/3766388831 wait complete. 2023-10-06T10:45:07.443 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:45:07.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.556+0000 7f1906c4d700 1 Processor -- start 2023-10-06T10:45:07.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.556+0000 7f1906c4d700 1 -- start start 2023-10-06T10:45:07.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.557+0000 7f1906c4d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f19000b25e0 0x7f19000b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.557+0000 7f1906c4d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f19000b29b0 0x7f1900159120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.557+0000 7f1906c4d700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19000cbfa0 con 0x7f1900159660 2023-10-06T10:45:07.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.557+0000 7f1906c4d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19000cb390 con 0x7f19000b29b0 2023-10-06T10:45:07.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.557+0000 7f1906c4d700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19000cb670 con 0x7f19000b25e0 2023-10-06T10:45:07.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.557+0000 7f18fffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f19000b29b0 0x7f1900159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.557+0000 7f18fffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f19000b29b0 0x7f1900159120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39230/0 (socket says 172.21.15.92:39230) 2023-10-06T10:45:07.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.557+0000 7f18fffff700 1 -- 172.21.15.92:0/1725214260 learned_addr learned my addr 172.21.15.92:0/1725214260 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:07.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.558+0000 7f19049e9700 1 --2- 172.21.15.92:0/1725214260 >> v2:172.21.15.124:3300/0 conn(0x7f19000b25e0 0x7f19000b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.558+0000 7f18fffff700 1 -- 172.21.15.92:0/1725214260 >> v2:172.21.15.124:3300/0 conn(0x7f19000b25e0 msgr2=0x7f19000b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:07.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.558+0000 7f18fffff700 1 --2- 172.21.15.92:0/1725214260 >> v2:172.21.15.124:3300/0 conn(0x7f19000b25e0 0x7f19000b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.558+0000 7f18fffff700 1 -- 172.21.15.92:0/1725214260 >> v1:172.21.15.124:6789/0 conn(0x7f1900159660 legacy=0x7f1900068540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.558+0000 7f18fffff700 1 -- 172.21.15.92:0/1725214260 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19000cb950 con 0x7f19000b29b0 2023-10-06T10:45:07.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.558+0000 7f18fffff700 1 --2- 172.21.15.92:0/1725214260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19000b29b0 0x7f1900159120 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f19000d0420 tx=0x7f18f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164bceb2bcafb144 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.558+0000 7f18ff7fe700 1 -- 172.21.15.92:0/1725214260 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f4009e70 con 0x7f19000b29b0 2023-10-06T10:45:07.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.558+0000 7f18ff7fe700 1 -- 172.21.15.92:0/1725214260 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f400b860 con 0x7f19000b29b0 2023-10-06T10:45:07.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.558+0000 7f18ff7fe700 1 -- 172.21.15.92:0/1725214260 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f400bc00 con 0x7f19000b29b0 2023-10-06T10:45:07.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.559+0000 7f1906c4d700 1 -- 172.21.15.92:0/1725214260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19000b29b0 msgr2=0x7f1900159120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.559+0000 7f1906c4d700 1 --2- 172.21.15.92:0/1725214260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19000b29b0 0x7f1900159120 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f19000d0420 tx=0x7f18f4009250 comp rx=0 tx=0).stop 2023-10-06T10:45:07.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.559+0000 7f1906c4d700 1 -- 172.21.15.92:0/1725214260 shutdown_connections 2023-10-06T10:45:07.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.559+0000 7f1906c4d700 1 --2- 172.21.15.92:0/1725214260 >> v2:172.21.15.124:3300/0 conn(0x7f19000b25e0 0x7f19000b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.559+0000 7f1906c4d700 1 --2- 172.21.15.92:0/1725214260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19000b29b0 0x7f1900159120 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.559+0000 7f1906c4d700 1 -- 172.21.15.92:0/1725214260 >> 172.21.15.92:0/1725214260 conn(0x7f190006ece0 msgr2=0x7f190006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:07.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.559+0000 7f1906c4d700 1 -- 172.21.15.92:0/1725214260 shutdown_connections 2023-10-06T10:45:07.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.559+0000 7f1906c4d700 1 -- 172.21.15.92:0/1725214260 wait complete. 2023-10-06T10:45:07.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.560+0000 7f1906c4d700 1 Processor -- start 2023-10-06T10:45:07.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.560+0000 7f1906c4d700 1 -- start start 2023-10-06T10:45:07.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f1906c4d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19000b25e0 0x7f19000c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f1906c4d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19000b29b0 0x7f1900169e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f1906c4d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1900159660 0x7f19000cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f1906c4d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190016ad90 con 0x7f1900159660 2023-10-06T10:45:07.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f1906c4d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190016d920 con 0x7f19000b25e0 2023-10-06T10:45:07.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f1906c4d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190016a7d0 con 0x7f19000b29b0 2023-10-06T10:45:07.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f19051ea700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1900159660 0x7f19000cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f19051ea700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1900159660 0x7f19000cf4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39232/0 (socket says 172.21.15.92:39232) 2023-10-06T10:45:07.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f19051ea700 1 -- 172.21.15.92:0/2586195918 learned_addr learned my addr 172.21.15.92:0/2586195918 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:07.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f18fffff700 1 --2- 172.21.15.92:0/2586195918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19000b29b0 0x7f1900169e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f19049e9700 1 --2- 172.21.15.92:0/2586195918 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19000b25e0 0x7f19000c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f19051ea700 1 -- 172.21.15.92:0/2586195918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19000b29b0 msgr2=0x7f1900169e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f19051ea700 1 --2- 172.21.15.92:0/2586195918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19000b29b0 0x7f1900169e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f19051ea700 1 -- 172.21.15.92:0/2586195918 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19000b25e0 msgr2=0x7f19000c6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f19051ea700 1 --2- 172.21.15.92:0/2586195918 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19000b25e0 0x7f19000c6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f19051ea700 1 -- 172.21.15.92:0/2586195918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f40094f0 con 0x7f1900159660 2023-10-06T10:45:07.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f19049e9700 1 --2- 172.21.15.92:0/2586195918 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19000b25e0 0x7f19000c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:07.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f18fffff700 1 --2- 172.21.15.92:0/2586195918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19000b29b0 0x7f1900169e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:07.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f19051ea700 1 --2- 172.21.15.92:0/2586195918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1900159660 0x7f19000cf4b0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f18f800c4c0 tx=0x7f18f8007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f18fcff9700 1 -- 172.21.15.92:0/2586195918 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f8011590 con 0x7f1900159660 2023-10-06T10:45:07.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f18fcff9700 1 -- 172.21.15.92:0/2586195918 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f8011710 con 0x7f1900159660 2023-10-06T10:45:07.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f1906c4d700 1 -- 172.21.15.92:0/2586195918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190016aab0 con 0x7f1900159660 2023-10-06T10:45:07.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f18fcff9700 1 -- 172.21.15.92:0/2586195918 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f8011ad0 con 0x7f1900159660 2023-10-06T10:45:07.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.561+0000 7f1906c4d700 1 -- 172.21.15.92:0/2586195918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190016a350 con 0x7f1900159660 2023-10-06T10:45:07.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.562+0000 7f1906c4d700 1 -- 172.21.15.92:0/2586195918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f190016a5a0 con 0x7f1900159660 2023-10-06T10:45:07.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.563+0000 7f18fcff9700 1 -- 172.21.15.92:0/2586195918 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f18f801b3b0 con 0x7f1900159660 2023-10-06T10:45:07.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.563+0000 7f18fcff9700 1 --2- 172.21.15.92:0/2586195918 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f18ec066f30 0x7f18ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:07.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.563+0000 7f18fcff9700 1 -- 172.21.15.92:0/2586195918 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f18f8083cf0 con 0x7f1900159660 2023-10-06T10:45:07.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.563+0000 7f19049e9700 1 --2- 172.21.15.92:0/2586195918 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f18ec066f30 0x7f18ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:07.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.564+0000 7f19049e9700 1 --2- 172.21.15.92:0/2586195918 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f18ec066f30 0x7f18ec0693e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009fd0 tx=0x7f18f000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:07.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.567+0000 7f18fcff9700 1 -- 172.21.15.92:0/2586195918 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18f8088070 con 0x7f1900159660 2023-10-06T10:45:07.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.799+0000 7f1906c4d700 1 -- 172.21.15.92:0/2586195918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f190016d410 con 0x7f1900159660 2023-10-06T10:45:07.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.799+0000 7f18fcff9700 1 -- 172.21.15.92:0/2586195918 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4637 (secure 0 0 0) 0x7f190016d410 con 0x7f1900159660 2023-10-06T10:45:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:07.801 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":105,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2016052382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2016052382},{"type":"v1","addr":"172.21.15.124:6834","nonce":2016052382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:44:51.106486+0000","modified":"2023-10-06T10:45:06.368491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8520,"mds_1":8517,"mds_2":6719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6719":{"gid":6719,"name":"d","rank":2,"incarnation":104,"state":"up:active","state_seq":6,"addr":"172.21.15.124:6835/2307521923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2307521923},{"type":"v1","addr":"172.21.15.124:6835","nonce":2307521923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8517":{"gid":8517,"name":"a","rank":1,"incarnation":102,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6836/1979826278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979826278},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979826278}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2298348426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2298348426},{"type":"v1","addr":"172.21.15.92:6837","nonce":2298348426}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T10:45:07.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.801+0000 7f1906c4d700 1 -- 172.21.15.92:0/2586195918 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f18ec066f30 msgr2=0x7f18ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.801+0000 7f1906c4d700 1 --2- 172.21.15.92:0/2586195918 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f18ec066f30 0x7f18ec0693e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009fd0 tx=0x7f18f000b040 comp rx=0 tx=0).stop 2023-10-06T10:45:07.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.801+0000 7f1906c4d700 1 -- 172.21.15.92:0/2586195918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1900159660 msgr2=0x7f19000cf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:07.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.801+0000 7f1906c4d700 1 --2- 172.21.15.92:0/2586195918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1900159660 0x7f19000cf4b0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f18f800c4c0 tx=0x7f18f8007f50 comp rx=0 tx=0).stop 2023-10-06T10:45:07.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.801+0000 7f1906c4d700 1 -- 172.21.15.92:0/2586195918 shutdown_connections 2023-10-06T10:45:07.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.801+0000 7f1906c4d700 1 --2- 172.21.15.92:0/2586195918 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f18ec066f30 0x7f18ec0693e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.801+0000 7f1906c4d700 1 --2- 172.21.15.92:0/2586195918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1900159660 0x7f19000cf4b0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.801+0000 7f1906c4d700 1 --2- 172.21.15.92:0/2586195918 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19000b25e0 0x7f19000c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.801+0000 7f1906c4d700 1 --2- 172.21.15.92:0/2586195918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19000b29b0 0x7f1900169e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:07.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.801+0000 7f1906c4d700 1 -- 172.21.15.92:0/2586195918 >> 172.21.15.92:0/2586195918 conn(0x7f190006ece0 msgr2=0x7f19000b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:07.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.801+0000 7f1906c4d700 1 -- 172.21.15.92:0/2586195918 shutdown_connections 2023-10-06T10:45:07.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:07.801+0000 7f1906c4d700 1 -- 172.21.15.92:0/2586195918 wait complete. 2023-10-06T10:45:07.804 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 105 2023-10-06T10:45:07.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:45:07.815 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2023-10-06T10:45:07.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:45:07.841 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2023-10-06T10:45:07.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:45:07.902 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2023-10-06T10:45:07.958 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:45:07.958 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2023-10-06T10:45:08.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:45:08.013 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2023-10-06T10:45:08.068 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:45:08.238 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:45:08.239 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:45:08.239 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:45:10.240 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:45:10.377 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:45:10.378 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:45:10.378 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:45:12.379 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:45:12.522 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:45:12.522 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:45:12.522 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:45:14.523 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:45:14.673 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-10-06T10:45:14.674 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:45:14.674 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:45:16.675 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-10-06T10:45:16.815 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2023-10-06T10:45:16.816 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2023-10-06T10:45:16.816 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2023-10-06T10:45:16.816 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-10-06T10:45:16.946 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2023-10-06T10:45:16.947 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2023-10-06T10:45:16.947 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:45:18.948 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-10-06T10:45:19.093 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2023-10-06T10:45:19.094 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2023-10-06T10:45:19.094 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:45:21.095 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-10-06T10:45:21.234 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2023-10-06T10:45:21.234 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2023-10-06T10:45:21.235 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:45:23.236 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-10-06T10:45:23.375 INFO:teuthology.orchestra.run.smithi092.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2023-10-06T10:45:23.376 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2023-10-06T10:45:23.376 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:45:23.377 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2023-10-06T10:45:23.520 INFO:teuthology.orchestra.run.smithi124.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-10-06T10:45:23.521 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2023-10-06T10:45:23.521 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2023-10-06T10:45:23.521 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 flush journal 2023-10-06T10:45:23.593 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:45:23.595 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:45:23.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.638+0000 7f47013aa700 1 Processor -- start 2023-10-06T10:45:23.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.638+0000 7f47013aa700 1 -- start start 2023-10-06T10:45:23.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.639+0000 7f47013aa700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46fc153990 0x7f46fc15a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.639+0000 7f47013aa700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f46fc0d09c0 0x7f46fc0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.639+0000 7f47013aa700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46fc0cdbf0 con 0x7f46fc0cca70 2023-10-06T10:45:23.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.639+0000 7f47013aa700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46fc0cb4c0 con 0x7f46fc0d09c0 2023-10-06T10:45:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.639+0000 7f47013aa700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc15a8b0 con 0x7f46fc153990 2023-10-06T10:45:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.639+0000 7f4700ba9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f46fc0d09c0 0x7f46fc0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.639+0000 7f46fb7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f46fc0cca70 0x7f46fc0d02b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34488/0 (socket says 172.21.15.92:34488) 2023-10-06T10:45:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.639+0000 7f46fb7fe700 1 -- 172.21.15.92:0/2966731560 learned_addr learned my addr 172.21.15.92:0/2966731560 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.639+0000 7f46fbfff700 1 --2- 172.21.15.92:0/2966731560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46fc153990 0x7f46fc15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.639+0000 7f4700ba9700 1 -- 172.21.15.92:0/2966731560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46fc153990 msgr2=0x7f46fc15a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.639+0000 7f4700ba9700 1 --2- 172.21.15.92:0/2966731560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46fc153990 0x7f46fc15a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.639+0000 7f4700ba9700 1 -- 172.21.15.92:0/2966731560 >> v1:172.21.15.92:6789/0 conn(0x7f46fc0cca70 legacy=0x7f46fc0d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.639+0000 7f4700ba9700 1 -- 172.21.15.92:0/2966731560 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46fc0ccfe0 con 0x7f46fc0d09c0 2023-10-06T10:45:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.639+0000 7f46faffd700 1 -- 172.21.15.92:0/2966731560 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1759312675 0 0) 0x7f46fc0cdbf0 con 0x7f46fc0cca70 2023-10-06T10:45:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.640+0000 7f4700ba9700 1 --2- 172.21.15.92:0/2966731560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46fc0d09c0 0x7f46fc0ce860 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009e60 tx=0x7f46f4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5cebec1d964385a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.640+0000 7f46faffd700 1 -- 172.21.15.92:0/2966731560 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f400e070 con 0x7f46fc0d09c0 2023-10-06T10:45:23.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.640+0000 7f46faffd700 1 -- 172.21.15.92:0/2966731560 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f400bc30 con 0x7f46fc0d09c0 2023-10-06T10:45:23.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.640+0000 7f46faffd700 1 -- 172.21.15.92:0/2966731560 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f4012430 con 0x7f46fc0d09c0 2023-10-06T10:45:23.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.640+0000 7f47013aa700 1 -- 172.21.15.92:0/2966731560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46fc0d09c0 msgr2=0x7f46fc0ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:23.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.640+0000 7f47013aa700 1 --2- 172.21.15.92:0/2966731560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46fc0d09c0 0x7f46fc0ce860 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009e60 tx=0x7f46f4009780 comp rx=0 tx=0).stop 2023-10-06T10:45:23.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.640+0000 7f47013aa700 1 -- 172.21.15.92:0/2966731560 shutdown_connections 2023-10-06T10:45:23.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.640+0000 7f47013aa700 1 --2- 172.21.15.92:0/2966731560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46fc153990 0x7f46fc15a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.640+0000 7f47013aa700 1 --2- 172.21.15.92:0/2966731560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46fc0d09c0 0x7f46fc0ce860 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.640+0000 7f47013aa700 1 -- 172.21.15.92:0/2966731560 >> 172.21.15.92:0/2966731560 conn(0x7f46fc0680c0 msgr2=0x7f46fc068490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:23.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.640+0000 7f47013aa700 1 -- 172.21.15.92:0/2966731560 shutdown_connections 2023-10-06T10:45:23.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.640+0000 7f47013aa700 1 -- 172.21.15.92:0/2966731560 wait complete. 2023-10-06T10:45:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.641+0000 7f47013aa700 1 Processor -- start 2023-10-06T10:45:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.642+0000 7f47013aa700 1 -- start start 2023-10-06T10:45:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.642+0000 7f47013aa700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46fc0cca70 0x7f46fc0b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.642+0000 7f47013aa700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f46fc0d09c0 0x7f46fc0b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.642+0000 7f47013aa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46fc153990 0x7f46fc1410f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.642+0000 7f47013aa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc0b1140 con 0x7f46fc153990 2023-10-06T10:45:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.642+0000 7f47013aa700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc0b0530 con 0x7f46fc0d09c0 2023-10-06T10:45:23.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.642+0000 7f47013aa700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc0b0810 con 0x7f46fc0cca70 2023-10-06T10:45:23.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.642+0000 7f4700ba9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46fc153990 0x7f46fc1410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.643+0000 7f4700ba9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46fc153990 0x7f46fc1410f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42738/0 (socket says 172.21.15.92:42738) 2023-10-06T10:45:23.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.643+0000 7f4700ba9700 1 -- 172.21.15.92:0/2839599177 learned_addr learned my addr 172.21.15.92:0/2839599177 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:23.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.643+0000 7f46fb7fe700 1 --2- 172.21.15.92:0/2839599177 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f46fc0d09c0 0x7f46fc0b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.643+0000 7f46fbfff700 1 --2- 172.21.15.92:0/2839599177 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46fc0cca70 0x7f46fc0b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.643+0000 7f4700ba9700 1 -- 172.21.15.92:0/2839599177 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46fc0cca70 msgr2=0x7f46fc0b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:23.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.643+0000 7f4700ba9700 1 --2- 172.21.15.92:0/2839599177 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46fc0cca70 0x7f46fc0b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.643+0000 7f4700ba9700 1 -- 172.21.15.92:0/2839599177 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f46fc0d09c0 msgr2=0x7f46fc0b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:23.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.643+0000 7f4700ba9700 1 --2- 172.21.15.92:0/2839599177 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f46fc0d09c0 0x7f46fc0b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.643+0000 7f4700ba9700 1 -- 172.21.15.92:0/2839599177 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f4009a20 con 0x7f46fc153990 2023-10-06T10:45:23.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.643+0000 7f46fb7fe700 1 --2- 172.21.15.92:0/2839599177 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f46fc0d09c0 0x7f46fc0b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:23.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.643+0000 7f46fbfff700 1 --2- 172.21.15.92:0/2839599177 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46fc0cca70 0x7f46fc0b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:23.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.643+0000 7f4700ba9700 1 --2- 172.21.15.92:0/2839599177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46fc153990 0x7f46fc1410f0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0890b0 tx=0x7f46f4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:23.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.643+0000 7f46e3fff700 1 -- 172.21.15.92:0/2839599177 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f401d630 con 0x7f46fc153990 2023-10-06T10:45:23.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.643+0000 7f46e3fff700 1 -- 172.21.15.92:0/2839599177 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f401d7b0 con 0x7f46fc153990 2023-10-06T10:45:23.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.643+0000 7f47013aa700 1 -- 172.21.15.92:0/2839599177 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46fc0b0af0 con 0x7f46fc153990 2023-10-06T10:45:23.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.643+0000 7f46e3fff700 1 -- 172.21.15.92:0/2839599177 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f401db70 con 0x7f46fc153990 2023-10-06T10:45:23.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.643+0000 7f47013aa700 1 -- 172.21.15.92:0/2839599177 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46fc150eb0 con 0x7f46fc153990 2023-10-06T10:45:23.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.644+0000 7f46e3fff700 1 -- 172.21.15.92:0/2839599177 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f46f4024070 con 0x7f46fc153990 2023-10-06T10:45:23.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.645+0000 7f46e3fff700 1 --2- 172.21.15.92:0/2839599177 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f46e8066f30 0x7f46e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.645+0000 7f46e3fff700 1 -- 172.21.15.92:0/2839599177 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f46f40870d0 con 0x7f46fc153990 2023-10-06T10:45:23.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.645+0000 7f46fbfff700 1 --2- 172.21.15.92:0/2839599177 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f46e8066f30 0x7f46e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.645+0000 7f46fbfff700 1 --2- 172.21.15.92:0/2839599177 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f46e8066f30 0x7f46e80693e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f46fc069fe0 tx=0x7f46ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:23.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.647+0000 7f470b904b80 1 Processor -- start 2023-10-06T10:45:23.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.647+0000 7f470b904b80 1 -- start start 2023-10-06T10:45:23.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.648+0000 7f470b904b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2d39a0 0x55f34c2d3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.648+0000 7f470b904b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f34c31dc10 0x55f34c2fc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.648+0000 7f470b904b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c2fceb0 0x55f34c2cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.648+0000 7f470b904b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f34c253cd0 con 0x55f34c2d39a0 2023-10-06T10:45:23.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.648+0000 7f470b904b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f34c22aac0 con 0x55f34c2fceb0 2023-10-06T10:45:23.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.648+0000 7f470b904b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f34c2282c0 con 0x55f34c31dc10 2023-10-06T10:45:23.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.648+0000 7f46fbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2d39a0 0x55f34c2d3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.648+0000 7f46fbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2d39a0 0x55f34c2d3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42752/0 (socket says 172.21.15.92:42752) 2023-10-06T10:45:23.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.648+0000 7f46fbfff700 1 -- 172.21.15.92:0/1887813848 learned_addr learned my addr 172.21.15.92:0/1887813848 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:23.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.648+0000 7f4700ba9700 1 --2- 172.21.15.92:0/1887813848 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c2fceb0 0x55f34c2cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.648+0000 7f46fbfff700 1 -- 172.21.15.92:0/1887813848 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f34c31dc10 msgr2=0x55f34c2fc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:23.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.648+0000 7f46fbfff700 1 --2- 172.21.15.92:0/1887813848 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f34c31dc10 0x55f34c2fc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.648+0000 7f46fbfff700 1 -- 172.21.15.92:0/1887813848 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c2fceb0 msgr2=0x55f34c2cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:23.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.648+0000 7f46fbfff700 1 --2- 172.21.15.92:0/1887813848 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c2fceb0 0x55f34c2cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.648+0000 7f46fbfff700 1 -- 172.21.15.92:0/1887813848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f34c0be2c0 con 0x55f34c2d39a0 2023-10-06T10:45:23.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.648+0000 7f4700ba9700 1 --2- 172.21.15.92:0/1887813848 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c2fceb0 0x55f34c2cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:23.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.648+0000 7f46fbfff700 1 --2- 172.21.15.92:0/1887813848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2d39a0 0x55f34c2d3d70 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0094b0 tx=0x7f46ec015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17faca77c41a48f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:23.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.648+0000 7f46e17fa700 1 -- 172.21.15.92:0/1887813848 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec00ee10 con 0x55f34c2d39a0 2023-10-06T10:45:23.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.648+0000 7f46e17fa700 1 -- 172.21.15.92:0/1887813848 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec0158d0 con 0x55f34c2d39a0 2023-10-06T10:45:23.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.648+0000 7f46e17fa700 1 -- 172.21.15.92:0/1887813848 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec015c50 con 0x55f34c2d39a0 2023-10-06T10:45:23.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.649+0000 7f470b904b80 1 -- 172.21.15.92:0/1887813848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2d39a0 msgr2=0x55f34c2d3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:23.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.649+0000 7f470b904b80 1 --2- 172.21.15.92:0/1887813848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2d39a0 0x55f34c2d3d70 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0094b0 tx=0x7f46ec015000 comp rx=0 tx=0).stop 2023-10-06T10:45:23.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.649+0000 7f470b904b80 1 -- 172.21.15.92:0/1887813848 shutdown_connections 2023-10-06T10:45:23.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.649+0000 7f470b904b80 1 --2- 172.21.15.92:0/1887813848 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c2fceb0 0x55f34c2cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.649+0000 7f470b904b80 1 --2- 172.21.15.92:0/1887813848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2d39a0 0x55f34c2d3d70 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.649+0000 7f470b904b80 1 --2- 172.21.15.92:0/1887813848 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f34c31dc10 0x55f34c2fc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.649+0000 7f470b904b80 1 -- 172.21.15.92:0/1887813848 >> 172.21.15.92:0/1887813848 conn(0x55f34c2d52c0 msgr2=0x55f34c2d5690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:23.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.649+0000 7f470b904b80 1 -- 172.21.15.92:0/1887813848 shutdown_connections 2023-10-06T10:45:23.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.649+0000 7f470b904b80 1 -- 172.21.15.92:0/1887813848 wait complete. 2023-10-06T10:45:23.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.650+0000 7f470b904b80 1 -- start start 2023-10-06T10:45:23.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.650+0000 7f470b904b80 1 Processor -- start 2023-10-06T10:45:23.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f470b904b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2d39a0 0x55f34c2f9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f470b904b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f34c2fceb0 0x55f34c32aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f470b904b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c31dc10 0x55f34c2ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f470b904b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f34c2240b0 con 0x55f34c2d39a0 2023-10-06T10:45:23.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f470b904b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f34c1cf8e0 con 0x55f34c31dc10 2023-10-06T10:45:23.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f470b904b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f34c1eaef0 con 0x55f34c2fceb0 2023-10-06T10:45:23.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f470b904b80 10 client.0 _ll_register_callbacks cb 0x55f34c0c0f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:23.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f46fbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2d39a0 0x55f34c2f9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f46fbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2d39a0 0x55f34c2f9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42760/0 (socket says 172.21.15.92:42760) 2023-10-06T10:45:23.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f46fbfff700 1 -- 172.21.15.92:0/753927488 learned_addr learned my addr 172.21.15.92:0/753927488 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:23.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f4700ba9700 1 --2- 172.21.15.92:0/753927488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c31dc10 0x55f34c2ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f46fb7fe700 1 --2- 172.21.15.92:0/753927488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f34c2fceb0 0x55f34c32aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f46fbfff700 1 -- 172.21.15.92:0/753927488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f34c2fceb0 msgr2=0x55f34c32aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:23.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f46fbfff700 1 --2- 172.21.15.92:0/753927488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f34c2fceb0 0x55f34c32aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f46fbfff700 1 -- 172.21.15.92:0/753927488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c31dc10 msgr2=0x55f34c2ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f46fbfff700 1 --2- 172.21.15.92:0/753927488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c31dc10 0x55f34c2ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f46fbfff700 1 -- 172.21.15.92:0/753927488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ec00e630 con 0x55f34c2d39a0 2023-10-06T10:45:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f46fbfff700 1 --2- 172.21.15.92:0/753927488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2d39a0 0x55f34c2f9420 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f46ec008010 tx=0x7f46ec018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6bf02ac71a30c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f46da7fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f46da7fc700 1 -- 172.21.15.92:0/753927488 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec020830 con 0x55f34c2d39a0 2023-10-06T10:45:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f46da7fc700 1 -- 172.21.15.92:0/753927488 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f46ec020990 con 0x55f34c2d39a0 2023-10-06T10:45:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f470b904b80 1 -- 172.21.15.92:0/753927488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f34c2c86e0 con 0x55f34c2d39a0 2023-10-06T10:45:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f46da7fc700 1 -- 172.21.15.92:0/753927488 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46ec020d60 con 0x55f34c2d39a0 2023-10-06T10:45:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f470b904b80 10 client.8805 fetch_fsmap learned FSMap version 107 2023-10-06T10:45:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.651+0000 7f470b904b80 1 -- 172.21.15.92:0/753927488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x55f34c061030 con 0x55f34c2d39a0 2023-10-06T10:45:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.652+0000 7f46da7fc700 1 -- 172.21.15.92:0/753927488 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f34c2c86e0 con 0x55f34c2d39a0 2023-10-06T10:45:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.652+0000 7f46da7fc700 1 -- 172.21.15.92:0/753927488 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 107) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f46ec031360 con 0x55f34c2d39a0 2023-10-06T10:45:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.652+0000 7f470b904b80 10 client.8805 fetch_fsmap finished waiting for FSMap version 107 2023-10-06T10:45:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.652+0000 7f470b904b80 10 client.8805 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-10-06T10:45:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.652+0000 7f470b904b80 20 client.8805 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.652+0000 7f470b904b80 1 --2- 172.21.15.92:0/753927488 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55f34c314410 0x55f34c317330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.652+0000 7f470b904b80 4 client.8805 mds_command: new command op to 8520 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.652+0000 7f470b904b80 1 -- 172.21.15.92:0/753927488 --> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f34c2a5610 con 0x55f34c314410 2023-10-06T10:45:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.652+0000 7f46fb7fe700 1 --2- 172.21.15.92:0/753927488 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55f34c314410 0x55f34c317330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.652+0000 7f46da7fc700 10 client.8805 ms_handle_connect on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.652+0000 7f46fb7fe700 1 --2- 172.21.15.92:0/753927488 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55f34c314410 0x55f34c317330 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=504b7c6bd67e942e server_cookie=6d4f6a703c6dccbe in_seq=0 out_seq=0 2023-10-06T10:45:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.652+0000 7f46da7fc700 1 -- 172.21.15.92:0/753927488 <== mds.0 v2:172.21.15.92:6835/2298348426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55f34c2a5610 con 0x55f34c314410 2023-10-06T10:45:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.652+0000 7f46da7fc700 10 client.8805 handle_command_reply: tid=0 2023-10-06T10:45:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.653+0000 7f470b904b80 1 client.8805 shutdown 2023-10-06T10:45:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.653+0000 7f470b904b80 1 -- 172.21.15.92:0/753927488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2d39a0 msgr2=0x55f34c2f9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.653+0000 7f470b904b80 1 --2- 172.21.15.92:0/753927488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2d39a0 0x55f34c2f9420 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f46ec008010 tx=0x7f46ec018000 comp rx=0 tx=0).stop 2023-10-06T10:45:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.653+0000 7f470b904b80 1 -- 172.21.15.92:0/753927488 shutdown_connections 2023-10-06T10:45:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.653+0000 7f470b904b80 1 --2- 172.21.15.92:0/753927488 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55f34c314410 0x55f34c317330 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.653+0000 7f470b904b80 1 --2- 172.21.15.92:0/753927488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c31dc10 0x55f34c2ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.653+0000 7f46da7fc700 0 client.8805 ms_handle_reset on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.653+0000 7f470b904b80 1 --2- 172.21.15.92:0/753927488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2d39a0 0x55f34c2f9420 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.653+0000 7f470b904b80 1 --2- 172.21.15.92:0/753927488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f34c2fceb0 0x55f34c32aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.653+0000 7f470b904b80 1 -- 172.21.15.92:0/753927488 >> 172.21.15.92:0/753927488 conn(0x55f34c2d52c0 msgr2=0x55f34c2d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.653+0000 7f470b904b80 1 -- 172.21.15.92:0/753927488 shutdown_connections 2023-10-06T10:45:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.653+0000 7f470b904b80 1 -- 172.21.15.92:0/753927488 wait complete. 2023-10-06T10:45:23.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.653+0000 7f470b904b80 20 client.8805 trim_cache size 0 max 16384 2023-10-06T10:45:23.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.667+0000 7f470b904b80 1 Processor -- start 2023-10-06T10:45:23.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.667+0000 7f470b904b80 1 -- start start 2023-10-06T10:45:23.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.667+0000 7f470b904b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f34c2d39a0 0x55f34c2d1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.667+0000 7f470b904b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2fceb0 0x55f34c32a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.667+0000 7f470b904b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c31dc10 0x55f34c333370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.667+0000 7f470b904b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f34c0a6e80 con 0x55f34c2fceb0 2023-10-06T10:45:23.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.667+0000 7f470b904b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f34c2b3120 con 0x55f34c31dc10 2023-10-06T10:45:23.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.667+0000 7f470b904b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f34c2d5ff0 con 0x55f34c2d39a0 2023-10-06T10:45:23.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.668+0000 7f46fb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2fceb0 0x55f34c32a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.668+0000 7f46fb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2fceb0 0x55f34c32a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42772/0 (socket says 172.21.15.92:42772) 2023-10-06T10:45:23.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.668+0000 7f46fb7fe700 1 -- 172.21.15.92:0/139333634 learned_addr learned my addr 172.21.15.92:0/139333634 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:23.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.668+0000 7f46fbfff700 1 --2- 172.21.15.92:0/139333634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f34c2d39a0 0x55f34c2d1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.668+0000 7f4700ba9700 1 --2- 172.21.15.92:0/139333634 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c31dc10 0x55f34c333370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.668+0000 7f46fb7fe700 1 -- 172.21.15.92:0/139333634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f34c2d39a0 msgr2=0x55f34c2d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:23.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.668+0000 7f46fb7fe700 1 --2- 172.21.15.92:0/139333634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f34c2d39a0 0x55f34c2d1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.668+0000 7f46fb7fe700 1 -- 172.21.15.92:0/139333634 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c31dc10 msgr2=0x55f34c333370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:23.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.668+0000 7f46fb7fe700 1 --2- 172.21.15.92:0/139333634 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c31dc10 0x55f34c333370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.668+0000 7f46fb7fe700 1 -- 172.21.15.92:0/139333634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f34c1f27d0 con 0x55f34c2fceb0 2023-10-06T10:45:23.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.668+0000 7f4700ba9700 1 --2- 172.21.15.92:0/139333634 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c31dc10 0x55f34c333370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:23.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.668+0000 7f46fb7fe700 1 --2- 172.21.15.92:0/139333634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2fceb0 0x55f34c32a210 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0b20c0 tx=0x7f46f000bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c717b1e9ace746b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:23.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.668+0000 7f46e1ffb700 1 -- 172.21.15.92:0/139333634 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0005ce0 con 0x55f34c2fceb0 2023-10-06T10:45:23.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.668+0000 7f46e1ffb700 1 -- 172.21.15.92:0/139333634 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f001a070 con 0x55f34c2fceb0 2023-10-06T10:45:23.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.668+0000 7f46e1ffb700 1 -- 172.21.15.92:0/139333634 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0007330 con 0x55f34c2fceb0 2023-10-06T10:45:23.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.669+0000 7f470b904b80 1 -- 172.21.15.92:0/139333634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2fceb0 msgr2=0x55f34c32a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:23.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.669+0000 7f470b904b80 1 --2- 172.21.15.92:0/139333634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2fceb0 0x55f34c32a210 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0b20c0 tx=0x7f46f000bd10 comp rx=0 tx=0).stop 2023-10-06T10:45:23.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.669+0000 7f470b904b80 1 -- 172.21.15.92:0/139333634 shutdown_connections 2023-10-06T10:45:23.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.669+0000 7f470b904b80 1 --2- 172.21.15.92:0/139333634 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c31dc10 0x55f34c333370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.669+0000 7f470b904b80 1 --2- 172.21.15.92:0/139333634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f34c2d39a0 0x55f34c2d1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.669+0000 7f470b904b80 1 --2- 172.21.15.92:0/139333634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2fceb0 0x55f34c32a210 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.669+0000 7f470b904b80 1 -- 172.21.15.92:0/139333634 >> 172.21.15.92:0/139333634 conn(0x55f34c2d52c0 msgr2=0x55f34c318a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:23.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.669+0000 7f470b904b80 1 -- 172.21.15.92:0/139333634 shutdown_connections 2023-10-06T10:45:23.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.669+0000 7f470b904b80 1 -- 172.21.15.92:0/139333634 wait complete. 2023-10-06T10:45:23.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.669+0000 7f470b904b80 1 -- start start 2023-10-06T10:45:23.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.669+0000 7f470b904b80 1 Processor -- start 2023-10-06T10:45:23.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.670+0000 7f470b904b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2d39a0 0x55f34c2f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.670+0000 7f470b904b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c2fceb0 0x55f34c371fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.670+0000 7f470b904b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f34c31dc10 0x55f34c36da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.670+0000 7f470b904b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f34c2f73b0 con 0x55f34c2d39a0 2023-10-06T10:45:23.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.670+0000 7f470b904b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f34c3705f0 con 0x55f34c2fceb0 2023-10-06T10:45:23.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.670+0000 7f470b904b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f34c36df90 con 0x55f34c31dc10 2023-10-06T10:45:23.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.670+0000 7f470b904b80 10 client.0 _ll_register_callbacks cb 0x55f34c0c0f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:23.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.670+0000 7f46fbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2d39a0 0x55f34c2f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.670+0000 7f46fbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2d39a0 0x55f34c2f6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42786/0 (socket says 172.21.15.92:42786) 2023-10-06T10:45:23.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.670+0000 7f46fbfff700 1 -- 172.21.15.92:0/1836359684 learned_addr learned my addr 172.21.15.92:0/1836359684 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:23.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.670+0000 7f46fb7fe700 1 --2- 172.21.15.92:0/1836359684 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c2fceb0 0x55f34c371fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.670+0000 7f4700ba9700 1 --2- 172.21.15.92:0/1836359684 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f34c31dc10 0x55f34c36da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.670+0000 7f46fbfff700 1 -- 172.21.15.92:0/1836359684 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f34c31dc10 msgr2=0x55f34c36da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:23.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.670+0000 7f46fbfff700 1 --2- 172.21.15.92:0/1836359684 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f34c31dc10 0x55f34c36da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.670+0000 7f46fbfff700 1 -- 172.21.15.92:0/1836359684 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c2fceb0 msgr2=0x55f34c371fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:23.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.670+0000 7f46fbfff700 1 --2- 172.21.15.92:0/1836359684 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c2fceb0 0x55f34c371fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.670+0000 7f46fbfff700 1 -- 172.21.15.92:0/1836359684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f000bac0 con 0x55f34c2d39a0 2023-10-06T10:45:23.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.670+0000 7f46fb7fe700 1 --2- 172.21.15.92:0/1836359684 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c2fceb0 0x55f34c371fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:23.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.671+0000 7f46fbfff700 1 --2- 172.21.15.92:0/1836359684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2d39a0 0x55f34c2f6af0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f46ec015720 tx=0x7f46ec028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13830cd733745276 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:23.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.671+0000 7f46da7fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:23.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.671+0000 7f46da7fc700 1 -- 172.21.15.92:0/1836359684 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec031ab0 con 0x55f34c2d39a0 2023-10-06T10:45:23.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.671+0000 7f46da7fc700 1 -- 172.21.15.92:0/1836359684 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f46ec031870 con 0x55f34c2d39a0 2023-10-06T10:45:23.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.671+0000 7f470b904b80 1 -- 172.21.15.92:0/1836359684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46ec0158d0 con 0x55f34c2d39a0 2023-10-06T10:45:23.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.671+0000 7f46da7fc700 1 -- 172.21.15.92:0/1836359684 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46ec005c10 con 0x55f34c2d39a0 2023-10-06T10:45:23.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.671+0000 7f470b904b80 10 client.8811 fetch_fsmap learned FSMap version 107 2023-10-06T10:45:23.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.671+0000 7f470b904b80 1 -- 172.21.15.92:0/1836359684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x55f34c1d94c0 con 0x55f34c2d39a0 2023-10-06T10:45:23.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.671+0000 7f46da7fc700 1 -- 172.21.15.92:0/1836359684 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec0158d0 con 0x55f34c2d39a0 2023-10-06T10:45:23.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.671+0000 7f46da7fc700 1 -- 172.21.15.92:0/1836359684 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 107) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f46ec006030 con 0x55f34c2d39a0 2023-10-06T10:45:23.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.671+0000 7f470b904b80 10 client.8811 fetch_fsmap finished waiting for FSMap version 107 2023-10-06T10:45:23.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.671+0000 7f470b904b80 10 client.8811 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-10-06T10:45:23.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.671+0000 7f470b904b80 20 client.8811 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:23.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.671+0000 7f470b904b80 1 --2- 172.21.15.92:0/1836359684 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55f34c313100 0x55f34c370a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.671+0000 7f470b904b80 4 client.8811 mds_command: new command op to 8520 tid=0[{"prefix": "flush journal"}] 2023-10-06T10:45:23.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.671+0000 7f470b904b80 1 -- 172.21.15.92:0/1836359684 --> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f34c1c6e50 con 0x55f34c313100 2023-10-06T10:45:23.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.671+0000 7f46fb7fe700 1 --2- 172.21.15.92:0/1836359684 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55f34c313100 0x55f34c370a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.671+0000 7f46da7fc700 10 client.8811 ms_handle_connect on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:23.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.671+0000 7f46fb7fe700 1 --2- 172.21.15.92:0/1836359684 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55f34c313100 0x55f34c370a80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=268f0757561a4eff server_cookie=5e48140823178b44 in_seq=0 out_seq=0 2023-10-06T10:45:23.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.685+0000 7f46da7fc700 1 -- 172.21.15.92:0/1836359684 <== mds.0 v2:172.21.15.92:6835/2298348426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f34c1c6e50 con 0x55f34c313100 2023-10-06T10:45:23.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.685+0000 7f46da7fc700 10 client.8811 handle_command_reply: tid=0 2023-10-06T10:45:23.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.685+0000 7f470b904b80 1 client.8811 shutdown 2023-10-06T10:45:23.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.685+0000 7f470b904b80 1 -- 172.21.15.92:0/1836359684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2d39a0 msgr2=0x55f34c2f6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:23.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.685+0000 7f470b904b80 1 --2- 172.21.15.92:0/1836359684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2d39a0 0x55f34c2f6af0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f46ec015720 tx=0x7f46ec028000 comp rx=0 tx=0).stop 2023-10-06T10:45:23.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.685+0000 7f470b904b80 1 -- 172.21.15.92:0/1836359684 shutdown_connections 2023-10-06T10:45:23.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.685+0000 7f470b904b80 1 --2- 172.21.15.92:0/1836359684 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55f34c313100 0x55f34c370a80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.685+0000 7f470b904b80 1 --2- 172.21.15.92:0/1836359684 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f34c31dc10 0x55f34c36da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.685+0000 7f46da7fc700 0 client.8811 ms_handle_reset on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:23.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.685+0000 7f470b904b80 1 --2- 172.21.15.92:0/1836359684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f34c2d39a0 0x55f34c2f6af0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.685+0000 7f470b904b80 1 --2- 172.21.15.92:0/1836359684 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f34c2fceb0 0x55f34c371fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.685+0000 7f470b904b80 1 -- 172.21.15.92:0/1836359684 >> 172.21.15.92:0/1836359684 conn(0x55f34c2d52c0 msgr2=0x55f34c333370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:23.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.685+0000 7f470b904b80 1 -- 172.21.15.92:0/1836359684 shutdown_connections 2023-10-06T10:45:23.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.685+0000 7f470b904b80 1 -- 172.21.15.92:0/1836359684 wait complete. 2023-10-06T10:45:23.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.685+0000 7f470b904b80 20 client.8811 trim_cache size 0 max 16384 2023-10-06T10:45:23.688 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:23.688 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2023-10-06T10:45:23.688 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:45:23.689 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:23.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.685+0000 7f47013aa700 1 -- 172.21.15.92:0/2839599177 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f46e8066f30 msgr2=0x7f46e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:23.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.685+0000 7f47013aa700 1 --2- 172.21.15.92:0/2839599177 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f46e8066f30 0x7f46e80693e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f46fc069fe0 tx=0x7f46ec008040 comp rx=0 tx=0).stop 2023-10-06T10:45:23.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.686+0000 7f47013aa700 1 -- 172.21.15.92:0/2839599177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46fc153990 msgr2=0x7f46fc1410f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:23.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.686+0000 7f47013aa700 1 --2- 172.21.15.92:0/2839599177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46fc153990 0x7f46fc1410f0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0890b0 tx=0x7f46f4019000 comp rx=0 tx=0).stop 2023-10-06T10:45:23.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.686+0000 7f47013aa700 1 -- 172.21.15.92:0/2839599177 shutdown_connections 2023-10-06T10:45:23.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.686+0000 7f47013aa700 1 --2- 172.21.15.92:0/2839599177 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f46e8066f30 0x7f46e80693e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.686+0000 7f47013aa700 1 --2- 172.21.15.92:0/2839599177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46fc153990 0x7f46fc1410f0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.686+0000 7f47013aa700 1 --2- 172.21.15.92:0/2839599177 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46fc0cca70 0x7f46fc0b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.686+0000 7f47013aa700 1 --2- 172.21.15.92:0/2839599177 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f46fc0d09c0 0x7f46fc0b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.686+0000 7f47013aa700 1 -- 172.21.15.92:0/2839599177 >> 172.21.15.92:0/2839599177 conn(0x7f46fc0680c0 msgr2=0x7f46fc0d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:23.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.686+0000 7f47013aa700 1 -- 172.21.15.92:0/2839599177 shutdown_connections 2023-10-06T10:45:23.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.686+0000 7f47013aa700 1 -- 172.21.15.92:0/2839599177 wait complete. 2023-10-06T10:45:23.699 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:1 flush journal 2023-10-06T10:45:23.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.855+0000 7f62de11f700 1 Processor -- start 2023-10-06T10:45:23.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.855+0000 7f62de11f700 1 -- start start 2023-10-06T10:45:23.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.856+0000 7f62de11f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62d8150940 0x7f62d8159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.856+0000 7f62de11f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62d815a160 0x7f62d80ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.856+0000 7f62de11f700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d815ac30 con 0x7f62d80c5250 2023-10-06T10:45:23.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.856+0000 7f62de11f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d80cb6a0 con 0x7f62d8150940 2023-10-06T10:45:23.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.856+0000 7f62de11f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80cb090 con 0x7f62d815a160 2023-10-06T10:45:23.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.856+0000 7f62d77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62d8150940 0x7f62d8159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.856+0000 7f62d77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62d8150940 0x7f62d8159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42792/0 (socket says 172.21.15.92:42792) 2023-10-06T10:45:23.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.856+0000 7f62d77fe700 1 -- 172.21.15.92:0/3482099174 learned_addr learned my addr 172.21.15.92:0/3482099174 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:23.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.856+0000 7f62d6ffd700 1 --2- 172.21.15.92:0/3482099174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62d815a160 0x7f62d80ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.856+0000 7f62d77fe700 1 -- 172.21.15.92:0/3482099174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62d815a160 msgr2=0x7f62d80ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:23.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.856+0000 7f62d77fe700 1 --2- 172.21.15.92:0/3482099174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62d815a160 0x7f62d80ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.856+0000 7f62d77fe700 1 -- 172.21.15.92:0/3482099174 >> v1:172.21.15.124:6789/0 conn(0x7f62d80c5250 legacy=0x7f62d80b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:23.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.856+0000 7f62d77fe700 1 -- 172.21.15.92:0/3482099174 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d815a670 con 0x7f62d8150940 2023-10-06T10:45:23.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.856+0000 7f62d6ffd700 1 --2- 172.21.15.92:0/3482099174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62d815a160 0x7f62d80ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:23.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.856+0000 7f62d77fe700 1 --2- 172.21.15.92:0/3482099174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d8150940 0x7f62d8159c20 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009f70 tx=0x7f62d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2be6096567857aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:23.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.856+0000 7f62d67fc700 1 -- 172.21.15.92:0/3482099174 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d000e070 con 0x7f62d8150940 2023-10-06T10:45:23.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.856+0000 7f62d67fc700 1 -- 172.21.15.92:0/3482099174 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d000ba90 con 0x7f62d8150940 2023-10-06T10:45:23.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.856+0000 7f62d67fc700 1 -- 172.21.15.92:0/3482099174 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d00092e0 con 0x7f62d8150940 2023-10-06T10:45:23.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.857+0000 7f62de11f700 1 -- 172.21.15.92:0/3482099174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d8150940 msgr2=0x7f62d8159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:23.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.857+0000 7f62de11f700 1 --2- 172.21.15.92:0/3482099174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d8150940 0x7f62d8159c20 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009f70 tx=0x7f62d0009840 comp rx=0 tx=0).stop 2023-10-06T10:45:23.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.857+0000 7f62de11f700 1 -- 172.21.15.92:0/3482099174 shutdown_connections 2023-10-06T10:45:23.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.857+0000 7f62de11f700 1 --2- 172.21.15.92:0/3482099174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d8150940 0x7f62d8159c20 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.857+0000 7f62de11f700 1 --2- 172.21.15.92:0/3482099174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62d815a160 0x7f62d80ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.857+0000 7f62de11f700 1 -- 172.21.15.92:0/3482099174 >> 172.21.15.92:0/3482099174 conn(0x7f62d8099700 msgr2=0x7f62d806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:23.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.857+0000 7f62de11f700 1 -- 172.21.15.92:0/3482099174 shutdown_connections 2023-10-06T10:45:23.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.857+0000 7f62de11f700 1 -- 172.21.15.92:0/3482099174 wait complete. 2023-10-06T10:45:23.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.858+0000 7f62de11f700 1 Processor -- start 2023-10-06T10:45:23.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.858+0000 7f62de11f700 1 -- start start 2023-10-06T10:45:23.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.859+0000 7f62de11f700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62d80c5250 0x7f62d8169c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.859+0000 7f62de11f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62d815a160 0x7f62d8158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.859+0000 7f62de11f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d80b4a30 0x7f62d816c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.859+0000 7f62de11f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d816a150 con 0x7f62d80b4a30 2023-10-06T10:45:23.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.859+0000 7f62de11f700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80b3250 con 0x7f62d80c5250 2023-10-06T10:45:23.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.859+0000 7f62de11f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80b40b0 con 0x7f62d815a160 2023-10-06T10:45:23.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.859+0000 7f62d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d80b4a30 0x7f62d816c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.859+0000 7f62d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d80b4a30 0x7f62d816c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42802/0 (socket says 172.21.15.92:42802) 2023-10-06T10:45:23.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.859+0000 7f62d7fff700 1 -- 172.21.15.92:0/39756587 learned_addr learned my addr 172.21.15.92:0/39756587 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:23.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.859+0000 7f62d77fe700 1 --2- 172.21.15.92:0/39756587 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62d80c5250 0x7f62d8169c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.859+0000 7f62d6ffd700 1 --2- 172.21.15.92:0/39756587 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62d815a160 0x7f62d8158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.859+0000 7f62d7fff700 1 -- 172.21.15.92:0/39756587 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62d815a160 msgr2=0x7f62d8158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:23.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.859+0000 7f62d7fff700 1 --2- 172.21.15.92:0/39756587 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62d815a160 0x7f62d8158af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.859+0000 7f62d7fff700 1 -- 172.21.15.92:0/39756587 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62d80c5250 msgr2=0x7f62d8169c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:23.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.859+0000 7f62d7fff700 1 --2- 172.21.15.92:0/39756587 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62d80c5250 0x7f62d8169c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.859+0000 7f62d7fff700 1 -- 172.21.15.92:0/39756587 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d0009ae0 con 0x7f62d80b4a30 2023-10-06T10:45:23.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.859+0000 7f62d77fe700 1 --2- 172.21.15.92:0/39756587 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62d80c5250 0x7f62d8169c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:23.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.859+0000 7f62d7fff700 1 --2- 172.21.15.92:0/39756587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d80b4a30 0x7f62d816c4e0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f62c800c5b0 tx=0x7f62c8007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:23.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.860+0000 7f62cf7fe700 1 -- 172.21.15.92:0/39756587 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c8011590 con 0x7f62d80b4a30 2023-10-06T10:45:23.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.860+0000 7f62cf7fe700 1 -- 172.21.15.92:0/39756587 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c8011710 con 0x7f62d80b4a30 2023-10-06T10:45:23.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.860+0000 7f62de11f700 1 -- 172.21.15.92:0/39756587 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d80b34a0 con 0x7f62d80b4a30 2023-10-06T10:45:23.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.860+0000 7f62cf7fe700 1 -- 172.21.15.92:0/39756587 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c8011ad0 con 0x7f62d80b4a30 2023-10-06T10:45:23.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.860+0000 7f62de11f700 1 -- 172.21.15.92:0/39756587 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d80b3a60 con 0x7f62d80b4a30 2023-10-06T10:45:23.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.861+0000 7f62cf7fe700 1 -- 172.21.15.92:0/39756587 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f62c801c460 con 0x7f62d80b4a30 2023-10-06T10:45:23.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.862+0000 7f62cf7fe700 1 --2- 172.21.15.92:0/39756587 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f62b8066f30 0x7f62b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.862+0000 7f62cf7fe700 1 -- 172.21.15.92:0/39756587 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f62c8084490 con 0x7f62d80b4a30 2023-10-06T10:45:23.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.862+0000 7f62d77fe700 1 --2- 172.21.15.92:0/39756587 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f62b8066f30 0x7f62b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.862+0000 7f62d77fe700 1 --2- 172.21.15.92:0/39756587 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f62b8066f30 0x7f62b80693e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009880 tx=0x7f62d0017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:23.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.864+0000 7f62e7417b80 1 Processor -- start 2023-10-06T10:45:23.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.864+0000 7f62e7417b80 1 -- start start 2023-10-06T10:45:23.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.864+0000 7f62e7417b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fee16ae0 0x5631fee16eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.864+0000 7f62e7417b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5631fedd29f0 0x5631fedcf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.864+0000 7f62e7417b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fedd0c20 0x5631fedcfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.864+0000 7f62e7417b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631fed4fcd0 con 0x5631fee16ae0 2023-10-06T10:45:23.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.864+0000 7f62e7417b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631fed26ac0 con 0x5631fedd0c20 2023-10-06T10:45:23.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.864+0000 7f62e7417b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5631fed242c0 con 0x5631fedd29f0 2023-10-06T10:45:23.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.865+0000 7f62d77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fee16ae0 0x5631fee16eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.865+0000 7f62d77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fee16ae0 0x5631fee16eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42808/0 (socket says 172.21.15.92:42808) 2023-10-06T10:45:23.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.865+0000 7f62d77fe700 1 -- 172.21.15.92:0/3051350243 learned_addr learned my addr 172.21.15.92:0/3051350243 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:23.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.865+0000 7f62d6ffd700 1 --2- 172.21.15.92:0/3051350243 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5631fedd29f0 0x5631fedcf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.865+0000 7f62d7fff700 1 --2- 172.21.15.92:0/3051350243 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fedd0c20 0x5631fedcfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.865+0000 7f62d77fe700 1 -- 172.21.15.92:0/3051350243 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5631fedd29f0 msgr2=0x5631fedcf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:23.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.865+0000 7f62d77fe700 1 --2- 172.21.15.92:0/3051350243 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5631fedd29f0 0x5631fedcf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.865+0000 7f62d77fe700 1 -- 172.21.15.92:0/3051350243 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fedd0c20 msgr2=0x5631fedcfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:23.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.865+0000 7f62d77fe700 1 --2- 172.21.15.92:0/3051350243 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fedd0c20 0x5631fedcfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.865+0000 7f62d77fe700 1 -- 172.21.15.92:0/3051350243 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631febba2c0 con 0x5631fee16ae0 2023-10-06T10:45:23.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.865+0000 7f62d77fe700 1 --2- 172.21.15.92:0/3051350243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fee16ae0 0x5631fee16eb0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f62d000be70 tx=0x7f62d0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c9d2bd1d2f63b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:23.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.865+0000 7f62ccff9700 1 -- 172.21.15.92:0/3051350243 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0030070 con 0x5631fee16ae0 2023-10-06T10:45:23.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.865+0000 7f62ccff9700 1 -- 172.21.15.92:0/3051350243 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d0012ad0 con 0x5631fee16ae0 2023-10-06T10:45:23.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.865+0000 7f62ccff9700 1 -- 172.21.15.92:0/3051350243 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0012e50 con 0x5631fee16ae0 2023-10-06T10:45:23.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.866+0000 7f62e7417b80 1 -- 172.21.15.92:0/3051350243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fee16ae0 msgr2=0x5631fee16eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:23.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.866+0000 7f62e7417b80 1 --2- 172.21.15.92:0/3051350243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fee16ae0 0x5631fee16eb0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f62d000be70 tx=0x7f62d0020000 comp rx=0 tx=0).stop 2023-10-06T10:45:23.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.866+0000 7f62e7417b80 1 -- 172.21.15.92:0/3051350243 shutdown_connections 2023-10-06T10:45:23.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.866+0000 7f62e7417b80 1 --2- 172.21.15.92:0/3051350243 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fedd0c20 0x5631fedcfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.866+0000 7f62e7417b80 1 --2- 172.21.15.92:0/3051350243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fee16ae0 0x5631fee16eb0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.866+0000 7f62e7417b80 1 --2- 172.21.15.92:0/3051350243 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5631fedd29f0 0x5631fedcf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.866+0000 7f62e7417b80 1 -- 172.21.15.92:0/3051350243 >> 172.21.15.92:0/3051350243 conn(0x5631fedd10c0 msgr2=0x5631fedd1490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:23.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.866+0000 7f62e7417b80 1 -- 172.21.15.92:0/3051350243 shutdown_connections 2023-10-06T10:45:23.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.866+0000 7f62e7417b80 1 -- 172.21.15.92:0/3051350243 wait complete. 2023-10-06T10:45:23.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.867+0000 7f62e7417b80 1 -- start start 2023-10-06T10:45:23.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.867+0000 7f62e7417b80 1 Processor -- start 2023-10-06T10:45:23.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.867+0000 7f62e7417b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fedd0c20 0x5631fedf6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.867+0000 7f62e7417b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5631fedd29f0 0x5631fee136a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.867+0000 7f62e7417b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fee16ae0 0x5631fedf79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.867+0000 7f62e7417b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631fed200b0 con 0x5631fee16ae0 2023-10-06T10:45:23.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.867+0000 7f62e7417b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631feccb8e0 con 0x5631fedd0c20 2023-10-06T10:45:23.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.867+0000 7f62e7417b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5631fece6ef0 con 0x5631fedd29f0 2023-10-06T10:45:23.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.867+0000 7f62e7417b80 10 client.0 _ll_register_callbacks cb 0x5631febbcf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:23.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.867+0000 7f62d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fee16ae0 0x5631fedf79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fee16ae0 0x5631fedf79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42810/0 (socket says 172.21.15.92:42810) 2023-10-06T10:45:23.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62d7fff700 1 -- 172.21.15.92:0/1961583347 learned_addr learned my addr 172.21.15.92:0/1961583347 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:23.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62d77fe700 1 --2- 172.21.15.92:0/1961583347 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fedd0c20 0x5631fedf6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62d6ffd700 1 --2- 172.21.15.92:0/1961583347 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5631fedd29f0 0x5631fee136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62d7fff700 1 -- 172.21.15.92:0/1961583347 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5631fedd29f0 msgr2=0x5631fee136a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:23.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62d7fff700 1 --2- 172.21.15.92:0/1961583347 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5631fedd29f0 0x5631fee136a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62d7fff700 1 -- 172.21.15.92:0/1961583347 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fedd0c20 msgr2=0x5631fedf6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:23.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62d7fff700 1 --2- 172.21.15.92:0/1961583347 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fedd0c20 0x5631fedf6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62d7fff700 1 -- 172.21.15.92:0/1961583347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d815a670 con 0x5631fee16ae0 2023-10-06T10:45:23.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62d77fe700 1 --2- 172.21.15.92:0/1961583347 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fedd0c20 0x5631fedf6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:23.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62d7fff700 1 --2- 172.21.15.92:0/1961583347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fee16ae0 0x5631fedf79f0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f62c8007fd0 tx=0x7f62c8084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5675e99354d60ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:23.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:23.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62b5ffb700 1 -- 172.21.15.92:0/1961583347 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c805c080 con 0x5631fee16ae0 2023-10-06T10:45:23.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62b5ffb700 1 -- 172.21.15.92:0/1961583347 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f62c80840b0 con 0x5631fee16ae0 2023-10-06T10:45:23.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62e7417b80 1 -- 172.21.15.92:0/1961583347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5631fedc46e0 con 0x5631fee16ae0 2023-10-06T10:45:23.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62b5ffb700 1 -- 172.21.15.92:0/1961583347 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62c8084290 con 0x5631fee16ae0 2023-10-06T10:45:23.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62e7417b80 10 client.8823 fetch_fsmap learned FSMap version 107 2023-10-06T10:45:23.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62e7417b80 1 -- 172.21.15.92:0/1961583347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x5631feb5d030 con 0x5631fee16ae0 2023-10-06T10:45:23.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62b5ffb700 1 -- 172.21.15.92:0/1961583347 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5631fedc46e0 con 0x5631fee16ae0 2023-10-06T10:45:23.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62b5ffb700 1 -- 172.21.15.92:0/1961583347 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 107) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f62c80545d0 con 0x5631fee16ae0 2023-10-06T10:45:23.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62e7417b80 10 client.8823 fetch_fsmap finished waiting for FSMap version 107 2023-10-06T10:45:23.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62e7417b80 10 client.8823 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2023-10-06T10:45:23.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62e7417b80 20 client.8823 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:23.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62e7417b80 1 --2- 172.21.15.92:0/1961583347 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5631fee30310 0x5631fee326f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62e7417b80 4 client.8823 mds_command: new command op to 8517 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:23.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.868+0000 7f62e7417b80 1 -- 172.21.15.92:0/1961583347 --> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631feda1610 con 0x5631fee30310 2023-10-06T10:45:23.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.869+0000 7f62d77fe700 1 --2- 172.21.15.92:0/1961583347 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5631fee30310 0x5631fee326f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.869+0000 7f62b5ffb700 10 client.8823 ms_handle_connect on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:23.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.869+0000 7f62d77fe700 1 --2- 172.21.15.92:0/1961583347 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5631fee30310 0x5631fee326f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b52a156ae7285d4 server_cookie=74bc8db43b0613aa in_seq=0 out_seq=0 2023-10-06T10:45:23.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.869+0000 7f62b5ffb700 1 -- 172.21.15.92:0/1961583347 <== mds.1 v2:172.21.15.92:6834/1979826278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5631feda1610 con 0x5631fee30310 2023-10-06T10:45:23.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.869+0000 7f62b5ffb700 10 client.8823 handle_command_reply: tid=0 2023-10-06T10:45:23.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.869+0000 7f62e7417b80 1 client.8823 shutdown 2023-10-06T10:45:23.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.870+0000 7f62e7417b80 1 -- 172.21.15.92:0/1961583347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fee16ae0 msgr2=0x5631fedf79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:23.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.870+0000 7f62e7417b80 1 --2- 172.21.15.92:0/1961583347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fee16ae0 0x5631fedf79f0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f62c8007fd0 tx=0x7f62c8084040 comp rx=0 tx=0).stop 2023-10-06T10:45:23.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.870+0000 7f62e7417b80 1 -- 172.21.15.92:0/1961583347 shutdown_connections 2023-10-06T10:45:23.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.870+0000 7f62e7417b80 1 --2- 172.21.15.92:0/1961583347 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5631fee30310 0x5631fee326f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.870+0000 7f62e7417b80 1 --2- 172.21.15.92:0/1961583347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fee16ae0 0x5631fedf79f0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.870+0000 7f62b5ffb700 0 client.8823 ms_handle_reset on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:23.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.870+0000 7f62e7417b80 1 --2- 172.21.15.92:0/1961583347 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fedd0c20 0x5631fedf6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.870+0000 7f62e7417b80 1 --2- 172.21.15.92:0/1961583347 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5631fedd29f0 0x5631fee136a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.870+0000 7f62e7417b80 1 -- 172.21.15.92:0/1961583347 >> 172.21.15.92:0/1961583347 conn(0x5631fedd10c0 msgr2=0x5631fedcbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:23.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.870+0000 7f62e7417b80 1 -- 172.21.15.92:0/1961583347 shutdown_connections 2023-10-06T10:45:23.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.870+0000 7f62e7417b80 1 -- 172.21.15.92:0/1961583347 wait complete. 2023-10-06T10:45:23.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.870+0000 7f62e7417b80 20 client.8823 trim_cache size 0 max 16384 2023-10-06T10:45:23.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.884+0000 7f62e7417b80 1 Processor -- start 2023-10-06T10:45:23.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.884+0000 7f62e7417b80 1 -- start start 2023-10-06T10:45:23.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.884+0000 7f62e7417b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5631fedd0c20 0x5631fee12ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.884+0000 7f62e7417b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fedd29f0 0x5631fee154a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.884+0000 7f62e7417b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fee16ae0 0x5631fedf6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.884+0000 7f62e7417b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631feba2e80 con 0x5631fedd29f0 2023-10-06T10:45:23.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.884+0000 7f62e7417b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631fedf5cb0 con 0x5631fee16ae0 2023-10-06T10:45:23.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.884+0000 7f62e7417b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5631fedced80 con 0x5631fedd0c20 2023-10-06T10:45:23.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.885+0000 7f62d6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fedd29f0 0x5631fee154a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.885+0000 7f62d6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fedd29f0 0x5631fee154a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42824/0 (socket says 172.21.15.92:42824) 2023-10-06T10:45:23.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.885+0000 7f62d6ffd700 1 -- 172.21.15.92:0/593060244 learned_addr learned my addr 172.21.15.92:0/593060244 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:23.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.885+0000 7f62d7fff700 1 --2- 172.21.15.92:0/593060244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fee16ae0 0x5631fedf6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.885+0000 7f62d77fe700 1 --2- 172.21.15.92:0/593060244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5631fedd0c20 0x5631fee12ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.885+0000 7f62d6ffd700 1 -- 172.21.15.92:0/593060244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5631fedd0c20 msgr2=0x5631fee12ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:23.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.885+0000 7f62d6ffd700 1 --2- 172.21.15.92:0/593060244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5631fedd0c20 0x5631fee12ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.885+0000 7f62d6ffd700 1 -- 172.21.15.92:0/593060244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fee16ae0 msgr2=0x5631fedf6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:23.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.885+0000 7f62d6ffd700 1 --2- 172.21.15.92:0/593060244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fee16ae0 0x5631fedf6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.885+0000 7f62d6ffd700 1 -- 172.21.15.92:0/593060244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631fecee7d0 con 0x5631fedd29f0 2023-10-06T10:45:23.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.885+0000 7f62d7fff700 1 --2- 172.21.15.92:0/593060244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fee16ae0 0x5631fedf6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:23.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.885+0000 7f62d6ffd700 1 --2- 172.21.15.92:0/593060244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fedd29f0 0x5631fee154a0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f62c0006fd0 tx=0x7f62c0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de67704938863f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:23.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.885+0000 7f62cd7fa700 1 -- 172.21.15.92:0/593060244 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0014ad0 con 0x5631fedd29f0 2023-10-06T10:45:23.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.885+0000 7f62cd7fa700 1 -- 172.21.15.92:0/593060244 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c000c040 con 0x5631fedd29f0 2023-10-06T10:45:23.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.885+0000 7f62cd7fa700 1 -- 172.21.15.92:0/593060244 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0014e40 con 0x5631fedd29f0 2023-10-06T10:45:23.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.886+0000 7f62e7417b80 1 -- 172.21.15.92:0/593060244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fedd29f0 msgr2=0x5631fee154a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:23.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.886+0000 7f62e7417b80 1 --2- 172.21.15.92:0/593060244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fedd29f0 0x5631fee154a0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f62c0006fd0 tx=0x7f62c0008040 comp rx=0 tx=0).stop 2023-10-06T10:45:23.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.886+0000 7f62e7417b80 1 -- 172.21.15.92:0/593060244 shutdown_connections 2023-10-06T10:45:23.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.886+0000 7f62e7417b80 1 --2- 172.21.15.92:0/593060244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fee16ae0 0x5631fedf6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.886+0000 7f62e7417b80 1 --2- 172.21.15.92:0/593060244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5631fedd0c20 0x5631fee12ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.886+0000 7f62e7417b80 1 --2- 172.21.15.92:0/593060244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fedd29f0 0x5631fee154a0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.886+0000 7f62e7417b80 1 -- 172.21.15.92:0/593060244 >> 172.21.15.92:0/593060244 conn(0x5631fedd10c0 msgr2=0x5631fee15f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:23.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.886+0000 7f62e7417b80 1 -- 172.21.15.92:0/593060244 shutdown_connections 2023-10-06T10:45:23.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.886+0000 7f62e7417b80 1 -- 172.21.15.92:0/593060244 wait complete. 2023-10-06T10:45:23.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.886+0000 7f62e7417b80 1 -- start start 2023-10-06T10:45:23.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.886+0000 7f62e7417b80 1 Processor -- start 2023-10-06T10:45:23.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.887+0000 7f62e7417b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fedd0c20 0x5631fedff680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.887+0000 7f62e7417b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fedd29f0 0x5631fee64cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.887+0000 7f62e7417b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5631fee16ae0 0x5631fee12880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.887+0000 7f62e7417b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631fee80140 con 0x5631fedd29f0 2023-10-06T10:45:23.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.887+0000 7f62e7417b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631fedcf700 con 0x5631fedd0c20 2023-10-06T10:45:23.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.887+0000 7f62e7417b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5631fee65210 con 0x5631fee16ae0 2023-10-06T10:45:23.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.887+0000 7f62e7417b80 10 client.0 _ll_register_callbacks cb 0x5631febbcf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:23.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.887+0000 7f62d6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fedd29f0 0x5631fee64cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.887+0000 7f62d6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fedd29f0 0x5631fee64cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42836/0 (socket says 172.21.15.92:42836) 2023-10-06T10:45:23.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.887+0000 7f62d6ffd700 1 -- 172.21.15.92:0/2471819876 learned_addr learned my addr 172.21.15.92:0/2471819876 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:23.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.887+0000 7f62d77fe700 1 --2- 172.21.15.92:0/2471819876 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fedd0c20 0x5631fedff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.887+0000 7f62d7fff700 1 --2- 172.21.15.92:0/2471819876 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5631fee16ae0 0x5631fee12880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.887+0000 7f62d6ffd700 1 -- 172.21.15.92:0/2471819876 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5631fee16ae0 msgr2=0x5631fee12880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:23.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.887+0000 7f62d6ffd700 1 --2- 172.21.15.92:0/2471819876 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5631fee16ae0 0x5631fee12880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.887+0000 7f62d6ffd700 1 -- 172.21.15.92:0/2471819876 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fedd0c20 msgr2=0x5631fedff680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:23.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.887+0000 7f62d6ffd700 1 --2- 172.21.15.92:0/2471819876 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fedd0c20 0x5631fedff680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.887+0000 7f62d6ffd700 1 -- 172.21.15.92:0/2471819876 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c000a990 con 0x5631fedd29f0 2023-10-06T10:45:23.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.887+0000 7f62d77fe700 1 --2- 172.21.15.92:0/2471819876 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fedd0c20 0x5631fedff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:23.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.887+0000 7f62d6ffd700 1 --2- 172.21.15.92:0/2471819876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fedd29f0 0x5631fee64cd0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f62d815a5a0 tx=0x7f62c0021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a54989b66af2e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:23.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.887+0000 7f62b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:23.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.888+0000 7f62b5ffb700 1 -- 172.21.15.92:0/2471819876 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c000a800 con 0x5631fedd29f0 2023-10-06T10:45:23.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.888+0000 7f62b5ffb700 1 -- 172.21.15.92:0/2471819876 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f62c0028440 con 0x5631fedd29f0 2023-10-06T10:45:23.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.888+0000 7f62e7417b80 1 -- 172.21.15.92:0/2471819876 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62d0012ad0 con 0x5631fedd29f0 2023-10-06T10:45:23.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.888+0000 7f62b5ffb700 1 -- 172.21.15.92:0/2471819876 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62c0028830 con 0x5631fedd29f0 2023-10-06T10:45:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.888+0000 7f62e7417b80 10 client.8829 fetch_fsmap learned FSMap version 107 2023-10-06T10:45:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.888+0000 7f62e7417b80 1 -- 172.21.15.92:0/2471819876 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x5631fecd54c0 con 0x5631fedd29f0 2023-10-06T10:45:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.888+0000 7f62b5ffb700 1 -- 172.21.15.92:0/2471819876 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d0012ad0 con 0x5631fedd29f0 2023-10-06T10:45:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.888+0000 7f62b5ffb700 1 -- 172.21.15.92:0/2471819876 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 107) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f62c0028990 con 0x5631fedd29f0 2023-10-06T10:45:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.888+0000 7f62e7417b80 10 client.8829 fetch_fsmap finished waiting for FSMap version 107 2023-10-06T10:45:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.888+0000 7f62e7417b80 10 client.8829 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2023-10-06T10:45:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.888+0000 7f62e7417b80 20 client.8829 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.888+0000 7f62e7417b80 1 --2- 172.21.15.92:0/2471819876 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5631fee1d770 0x5631fee12dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.888+0000 7f62e7417b80 4 client.8829 mds_command: new command op to 8517 tid=0[{"prefix": "flush journal"}] 2023-10-06T10:45:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.888+0000 7f62e7417b80 1 -- 172.21.15.92:0/2471819876 --> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5631fecc2e50 con 0x5631fee1d770 2023-10-06T10:45:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.888+0000 7f62d77fe700 1 --2- 172.21.15.92:0/2471819876 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5631fee1d770 0x5631fee12dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.888+0000 7f62b5ffb700 10 client.8829 ms_handle_connect on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.888+0000 7f62d77fe700 1 --2- 172.21.15.92:0/2471819876 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5631fee1d770 0x5631fee12dc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=95fb1935587e0f5a server_cookie=6911019b92bebd77 in_seq=0 out_seq=0 2023-10-06T10:45:23.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.897+0000 7f62b5ffb700 1 -- 172.21.15.92:0/2471819876 <== mds.1 v2:172.21.15.92:6834/1979826278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5631fecc2e50 con 0x5631fee1d770 2023-10-06T10:45:23.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.897+0000 7f62b5ffb700 10 client.8829 handle_command_reply: tid=0 2023-10-06T10:45:23.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.897+0000 7f62e7417b80 1 client.8829 shutdown 2023-10-06T10:45:23.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.898+0000 7f62e7417b80 1 -- 172.21.15.92:0/2471819876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fedd29f0 msgr2=0x5631fee64cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:23.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.898+0000 7f62e7417b80 1 --2- 172.21.15.92:0/2471819876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fedd29f0 0x5631fee64cd0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f62d815a5a0 tx=0x7f62c0021000 comp rx=0 tx=0).stop 2023-10-06T10:45:23.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.898+0000 7f62e7417b80 1 -- 172.21.15.92:0/2471819876 shutdown_connections 2023-10-06T10:45:23.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.898+0000 7f62e7417b80 1 --2- 172.21.15.92:0/2471819876 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5631fee16ae0 0x5631fee12880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.898+0000 7f62e7417b80 1 --2- 172.21.15.92:0/2471819876 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5631fedd0c20 0x5631fedff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.898+0000 7f62e7417b80 1 --2- 172.21.15.92:0/2471819876 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5631fee1d770 0x5631fee12dc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.898+0000 7f62e7417b80 1 --2- 172.21.15.92:0/2471819876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631fedd29f0 0x5631fee64cd0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.898+0000 7f62b5ffb700 0 client.8829 ms_handle_reset on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:23.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.898+0000 7f62e7417b80 1 -- 172.21.15.92:0/2471819876 >> 172.21.15.92:0/2471819876 conn(0x5631fedd10c0 msgr2=0x5631fee15ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:23.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.898+0000 7f62e7417b80 1 -- 172.21.15.92:0/2471819876 shutdown_connections 2023-10-06T10:45:23.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.898+0000 7f62e7417b80 1 -- 172.21.15.92:0/2471819876 wait complete. 2023-10-06T10:45:23.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.898+0000 7f62e7417b80 20 client.8829 trim_cache size 0 max 16384 2023-10-06T10:45:23.901 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:23.901 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2023-10-06T10:45:23.901 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:45:23.901 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:23.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.898+0000 7f62de11f700 1 -- 172.21.15.92:0/39756587 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f62b8066f30 msgr2=0x7f62b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:23.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.898+0000 7f62de11f700 1 --2- 172.21.15.92:0/39756587 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f62b8066f30 0x7f62b80693e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009880 tx=0x7f62d0017000 comp rx=0 tx=0).stop 2023-10-06T10:45:23.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.898+0000 7f62de11f700 1 -- 172.21.15.92:0/39756587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d80b4a30 msgr2=0x7f62d816c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:23.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.898+0000 7f62de11f700 1 --2- 172.21.15.92:0/39756587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d80b4a30 0x7f62d816c4e0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f62c800c5b0 tx=0x7f62c8007f60 comp rx=0 tx=0).stop 2023-10-06T10:45:23.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.898+0000 7f62de11f700 1 -- 172.21.15.92:0/39756587 shutdown_connections 2023-10-06T10:45:23.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.898+0000 7f62de11f700 1 --2- 172.21.15.92:0/39756587 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f62b8066f30 0x7f62b80693e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.898+0000 7f62de11f700 1 --2- 172.21.15.92:0/39756587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d80b4a30 0x7f62d816c4e0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.898+0000 7f62de11f700 1 --2- 172.21.15.92:0/39756587 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62d80c5250 0x7f62d8169c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.898+0000 7f62de11f700 1 --2- 172.21.15.92:0/39756587 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62d815a160 0x7f62d8158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:23.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.898+0000 7f62de11f700 1 -- 172.21.15.92:0/39756587 >> 172.21.15.92:0/39756587 conn(0x7f62d8099700 msgr2=0x7f62d80699c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:23.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.899+0000 7f62de11f700 1 -- 172.21.15.92:0/39756587 shutdown_connections 2023-10-06T10:45:23.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:23.899+0000 7f62de11f700 1 -- 172.21.15.92:0/39756587 wait complete. 2023-10-06T10:45:23.912 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:2 flush journal 2023-10-06T10:45:24.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.025+0000 7f0b728b1700 1 Processor -- start 2023-10-06T10:45:24.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.026+0000 7f0b728b1700 1 -- start start 2023-10-06T10:45:24.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.026+0000 7f0b728b1700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b6c150940 0x7f0b6c159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:24.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.026+0000 7f0b728b1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b6c15ac30 con 0x7f0b6c0cd4d0 2023-10-06T10:45:24.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.026+0000 7f0b728b1700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b6c0cb6a0 con 0x7f0b6c15a160 2023-10-06T10:45:24.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.026+0000 7f0b728b1700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c0cb090 con 0x7f0b6c150940 2023-10-06T10:45:24.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.027+0000 7f0b70e4e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0b6c0cd4d0 0x7f0b6c0b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34502/0 (socket says 172.21.15.92:34502) 2023-10-06T10:45:24.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.027+0000 7f0b70e4e700 1 -- 172.21.15.92:0/1970195624 learned_addr learned my addr 172.21.15.92:0/1970195624 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:24.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.027+0000 7f0b6bfff700 1 --2- 172.21.15.92:0/1970195624 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b6c150940 0x7f0b6c159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.027+0000 7f0b6affd700 1 -- 172.21.15.92:0/1970195624 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2750261028 0 0) 0x7f0b64001490 con 0x7f0b6c0cd4d0 2023-10-06T10:45:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.027+0000 7f0b6affd700 1 -- 172.21.15.92:0/1970195624 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b54003570 con 0x7f0b6c0cd4d0 2023-10-06T10:45:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.027+0000 7f0b6affd700 1 -- 172.21.15.92:0/1970195624 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 879768384 0 0) 0x7f0b6c0cb6a0 con 0x7f0b6c15a160 2023-10-06T10:45:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.027+0000 7f0b6affd700 1 -- 172.21.15.92:0/1970195624 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b64001490 con 0x7f0b6c15a160 2023-10-06T10:45:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.027+0000 7f0b6bfff700 1 -- 172.21.15.92:0/1970195624 >> v1:172.21.15.124:6789/0 conn(0x7f0b6c15a160 legacy=0x7f0b6c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.027+0000 7f0b6bfff700 1 -- 172.21.15.92:0/1970195624 >> v1:172.21.15.92:6789/0 conn(0x7f0b6c0cd4d0 legacy=0x7f0b6c0b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.027+0000 7f0b6bfff700 1 -- 172.21.15.92:0/1970195624 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6c15a670 con 0x7f0b6c150940 2023-10-06T10:45:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.027+0000 7f0b6bfff700 1 --2- 172.21.15.92:0/1970195624 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b6c150940 0x7f0b6c159c20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009900 tx=0x7f0b5c009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=973846568244ff29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.027+0000 7f0b6affd700 1 -- 172.21.15.92:0/1970195624 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c005c10 con 0x7f0b6c150940 2023-10-06T10:45:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.027+0000 7f0b6affd700 1 -- 172.21.15.92:0/1970195624 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5c005d70 con 0x7f0b6c150940 2023-10-06T10:45:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.027+0000 7f0b6affd700 1 -- 172.21.15.92:0/1970195624 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c006240 con 0x7f0b6c150940 2023-10-06T10:45:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.028+0000 7f0b728b1700 1 -- 172.21.15.92:0/1970195624 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b6c150940 msgr2=0x7f0b6c159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:24.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.028+0000 7f0b728b1700 1 --2- 172.21.15.92:0/1970195624 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b6c150940 0x7f0b6c159c20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009900 tx=0x7f0b5c009120 comp rx=0 tx=0).stop 2023-10-06T10:45:24.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.028+0000 7f0b728b1700 1 -- 172.21.15.92:0/1970195624 shutdown_connections 2023-10-06T10:45:24.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.028+0000 7f0b728b1700 1 --2- 172.21.15.92:0/1970195624 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b6c150940 0x7f0b6c159c20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.028+0000 7f0b728b1700 1 -- 172.21.15.92:0/1970195624 >> 172.21.15.92:0/1970195624 conn(0x7f0b6c099700 msgr2=0x7f0b6c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:24.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.028+0000 7f0b728b1700 1 -- 172.21.15.92:0/1970195624 shutdown_connections 2023-10-06T10:45:24.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.028+0000 7f0b728b1700 1 -- 172.21.15.92:0/1970195624 wait complete. 2023-10-06T10:45:24.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.029+0000 7f0b728b1700 1 Processor -- start 2023-10-06T10:45:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.029+0000 7f0b728b1700 1 -- start start 2023-10-06T10:45:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.030+0000 7f0b728b1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b6c0cd4d0 0x7f0b6c14ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.030+0000 7f0b728b1700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b6c15a160 0x7f0b6c152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.030+0000 7f0b728b1700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b6c152cc0 0x7f0b6c153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.030+0000 7f0b728b1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c153ac0 con 0x7f0b6c0cd4d0 2023-10-06T10:45:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.030+0000 7f0b728b1700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c0b5210 con 0x7f0b6c152cc0 2023-10-06T10:45:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.030+0000 7f0b728b1700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c0b6070 con 0x7f0b6c15a160 2023-10-06T10:45:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.030+0000 7f0b6bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b6c0cd4d0 0x7f0b6c14ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.030+0000 7f0b6bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b6c0cd4d0 0x7f0b6c14ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42842/0 (socket says 172.21.15.92:42842) 2023-10-06T10:45:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.030+0000 7f0b6bfff700 1 -- 172.21.15.92:0/3706841555 learned_addr learned my addr 172.21.15.92:0/3706841555 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.030+0000 7f0b6b7fe700 1 --2- 172.21.15.92:0/3706841555 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b6c15a160 0x7f0b6c152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.030+0000 7f0b70e4e700 1 --2- 172.21.15.92:0/3706841555 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b6c152cc0 0x7f0b6c153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.030+0000 7f0b6bfff700 1 -- 172.21.15.92:0/3706841555 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b6c15a160 msgr2=0x7f0b6c152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.030+0000 7f0b6bfff700 1 --2- 172.21.15.92:0/3706841555 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b6c15a160 0x7f0b6c152780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.030+0000 7f0b6bfff700 1 -- 172.21.15.92:0/3706841555 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b6c152cc0 msgr2=0x7f0b6c153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.030+0000 7f0b6bfff700 1 --2- 172.21.15.92:0/3706841555 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b6c152cc0 0x7f0b6c153160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.030+0000 7f0b6bfff700 1 -- 172.21.15.92:0/3706841555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5c0093c0 con 0x7f0b6c0cd4d0 2023-10-06T10:45:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.031+0000 7f0b6bfff700 1 --2- 172.21.15.92:0/3706841555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b6c0cd4d0 0x7f0b6c14ff50 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0d14c0 tx=0x7f0b5c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.031+0000 7f0b53fff700 1 -- 172.21.15.92:0/3706841555 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c01d6b0 con 0x7f0b6c0cd4d0 2023-10-06T10:45:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.031+0000 7f0b53fff700 1 -- 172.21.15.92:0/3706841555 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5c01d850 con 0x7f0b6c0cd4d0 2023-10-06T10:45:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.031+0000 7f0b728b1700 1 -- 172.21.15.92:0/3706841555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b6c0b5460 con 0x7f0b6c0cd4d0 2023-10-06T10:45:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.031+0000 7f0b53fff700 1 -- 172.21.15.92:0/3706841555 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c01dc10 con 0x7f0b6c0cd4d0 2023-10-06T10:45:24.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.031+0000 7f0b728b1700 1 -- 172.21.15.92:0/3706841555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b6c0b5a20 con 0x7f0b6c0cd4d0 2023-10-06T10:45:24.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.032+0000 7f0b53fff700 1 -- 172.21.15.92:0/3706841555 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0b5c01b480 con 0x7f0b6c0cd4d0 2023-10-06T10:45:24.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.032+0000 7f0b53fff700 1 --2- 172.21.15.92:0/3706841555 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0b58066f30 0x7f0b580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:24.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.033+0000 7f0b53fff700 1 -- 172.21.15.92:0/3706841555 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f0b5c0863a0 con 0x7f0b6c0cd4d0 2023-10-06T10:45:24.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.033+0000 7f0b6b7fe700 1 --2- 172.21.15.92:0/3706841555 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0b58066f30 0x7f0b580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:24.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.033+0000 7f0b6b7fe700 1 --2- 172.21.15.92:0/3706841555 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0b58066f30 0x7f0b580693e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0ca830 tx=0x7f0b60007d00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:24.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.034+0000 7f0b7bba9b80 1 Processor -- start 2023-10-06T10:45:24.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.034+0000 7f0b7bba9b80 1 -- start start 2023-10-06T10:45:24.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.035+0000 7f0b7bba9b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cb5226ae0 0x562cb5226eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:24.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.035+0000 7f0b7bba9b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cb51e29f0 0x562cb51df5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:24.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.035+0000 7f0b7bba9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e0c20 0x562cb51dfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:24.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.035+0000 7f0b7bba9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb515fcd0 con 0x562cb51e0c20 2023-10-06T10:45:24.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.035+0000 7f0b7bba9b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb5136ac0 con 0x562cb5226ae0 2023-10-06T10:45:24.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.035+0000 7f0b7bba9b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562cb51342c0 con 0x562cb51e29f0 2023-10-06T10:45:24.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.035+0000 7f0b70e4e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e0c20 0x562cb51dfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:24.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.035+0000 7f0b70e4e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e0c20 0x562cb51dfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42850/0 (socket says 172.21.15.92:42850) 2023-10-06T10:45:24.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.035+0000 7f0b70e4e700 1 -- 172.21.15.92:0/3875584894 learned_addr learned my addr 172.21.15.92:0/3875584894 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:24.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.036+0000 7f0b6b7fe700 1 --2- 172.21.15.92:0/3875584894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cb51e29f0 0x562cb51df5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:24.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.036+0000 7f0b6bfff700 1 --2- 172.21.15.92:0/3875584894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cb5226ae0 0x562cb5226eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:24.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.036+0000 7f0b70e4e700 1 -- 172.21.15.92:0/3875584894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cb51e29f0 msgr2=0x562cb51df5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:24.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.036+0000 7f0b70e4e700 1 --2- 172.21.15.92:0/3875584894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cb51e29f0 0x562cb51df5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.036+0000 7f0b70e4e700 1 -- 172.21.15.92:0/3875584894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cb5226ae0 msgr2=0x562cb5226eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:24.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.036+0000 7f0b70e4e700 1 --2- 172.21.15.92:0/3875584894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cb5226ae0 0x562cb5226eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.036+0000 7f0b70e4e700 1 -- 172.21.15.92:0/3875584894 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cb4fca2c0 con 0x562cb51e0c20 2023-10-06T10:45:24.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.036+0000 7f0b6bfff700 1 --2- 172.21.15.92:0/3875584894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cb5226ae0 0x562cb5226eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:24.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.036+0000 7f0b70e4e700 1 --2- 172.21.15.92:0/3875584894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e0c20 0x562cb51dfb10 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f0b6400f300 tx=0x7f0b64009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddeb6c199e6201a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:24.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.036+0000 7f0b517fa700 1 -- 172.21.15.92:0/3875584894 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b640070d0 con 0x562cb51e0c20 2023-10-06T10:45:24.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.036+0000 7f0b517fa700 1 -- 172.21.15.92:0/3875584894 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b64007230 con 0x562cb51e0c20 2023-10-06T10:45:24.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.036+0000 7f0b517fa700 1 -- 172.21.15.92:0/3875584894 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b640075f0 con 0x562cb51e0c20 2023-10-06T10:45:24.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.037+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/3875584894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e0c20 msgr2=0x562cb51dfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:24.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.037+0000 7f0b7bba9b80 1 --2- 172.21.15.92:0/3875584894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e0c20 0x562cb51dfb10 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f0b6400f300 tx=0x7f0b64009040 comp rx=0 tx=0).stop 2023-10-06T10:45:24.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.037+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/3875584894 shutdown_connections 2023-10-06T10:45:24.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.037+0000 7f0b7bba9b80 1 --2- 172.21.15.92:0/3875584894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e0c20 0x562cb51dfb10 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.037+0000 7f0b7bba9b80 1 --2- 172.21.15.92:0/3875584894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cb5226ae0 0x562cb5226eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.037+0000 7f0b7bba9b80 1 --2- 172.21.15.92:0/3875584894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cb51e29f0 0x562cb51df5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.037+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/3875584894 >> 172.21.15.92:0/3875584894 conn(0x562cb51e10c0 msgr2=0x562cb51e1490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:24.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.037+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/3875584894 shutdown_connections 2023-10-06T10:45:24.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.037+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/3875584894 wait complete. 2023-10-06T10:45:24.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.037+0000 7f0b7bba9b80 1 -- start start 2023-10-06T10:45:24.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.037+0000 7f0b7bba9b80 1 Processor -- start 2023-10-06T10:45:24.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.038+0000 7f0b7bba9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e0c20 0x562cb5206430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:24.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.038+0000 7f0b7bba9b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cb51e29f0 0x562cb5222210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:24.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.038+0000 7f0b7bba9b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cb5226ae0 0x562cb520b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:24.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.038+0000 7f0b7bba9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb51300b0 con 0x562cb51e0c20 2023-10-06T10:45:24.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.038+0000 7f0b7bba9b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb50db8e0 con 0x562cb5226ae0 2023-10-06T10:45:24.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.038+0000 7f0b7bba9b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562cb50f6ef0 con 0x562cb51e29f0 2023-10-06T10:45:24.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.038+0000 7f0b7bba9b80 10 client.0 _ll_register_callbacks cb 0x562cb4fccf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:24.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.038+0000 7f0b6bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e0c20 0x562cb5206430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:24.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.038+0000 7f0b6bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e0c20 0x562cb5206430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42856/0 (socket says 172.21.15.92:42856) 2023-10-06T10:45:24.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.038+0000 7f0b6bfff700 1 -- 172.21.15.92:0/2557639849 learned_addr learned my addr 172.21.15.92:0/2557639849 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:24.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.038+0000 7f0b6b7fe700 1 --2- 172.21.15.92:0/2557639849 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cb51e29f0 0x562cb5222210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:24.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.038+0000 7f0b70e4e700 1 --2- 172.21.15.92:0/2557639849 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cb5226ae0 0x562cb520b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:24.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.038+0000 7f0b6bfff700 1 -- 172.21.15.92:0/2557639849 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cb51e29f0 msgr2=0x562cb5222210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:24.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.038+0000 7f0b6bfff700 1 --2- 172.21.15.92:0/2557639849 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cb51e29f0 0x562cb5222210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.038+0000 7f0b6bfff700 1 -- 172.21.15.92:0/2557639849 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cb5226ae0 msgr2=0x562cb520b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:24.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.038+0000 7f0b6bfff700 1 --2- 172.21.15.92:0/2557639849 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cb5226ae0 0x562cb520b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.038+0000 7f0b6bfff700 1 -- 172.21.15.92:0/2557639849 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b54003570 con 0x562cb51e0c20 2023-10-06T10:45:24.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.039+0000 7f0b6bfff700 1 --2- 172.21.15.92:0/2557639849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e0c20 0x562cb5206430 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009160 tx=0x7f0b5c05c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e9724e2cfab2a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:24.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.039+0000 7f0b4a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:24.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.039+0000 7f0b4a7fc700 1 -- 172.21.15.92:0/2557639849 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c05df70 con 0x562cb51e0c20 2023-10-06T10:45:24.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.039+0000 7f0b4a7fc700 1 -- 172.21.15.92:0/2557639849 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f0b5c05c0b0 con 0x562cb51e0c20 2023-10-06T10:45:24.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.039+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/2557639849 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562cb51d46e0 con 0x562cb51e0c20 2023-10-06T10:45:24.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.039+0000 7f0b4a7fc700 1 -- 172.21.15.92:0/2557639849 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b5c05c270 con 0x562cb51e0c20 2023-10-06T10:45:24.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.039+0000 7f0b7bba9b80 10 client.8841 fetch_fsmap learned FSMap version 107 2023-10-06T10:45:24.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.039+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/2557639849 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x562cb4f6d030 con 0x562cb51e0c20 2023-10-06T10:45:24.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.039+0000 7f0b4a7fc700 1 -- 172.21.15.92:0/2557639849 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562cb51d46e0 con 0x562cb51e0c20 2023-10-06T10:45:24.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.039+0000 7f0b4a7fc700 1 -- 172.21.15.92:0/2557639849 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 107) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f0b5c0564f0 con 0x562cb51e0c20 2023-10-06T10:45:24.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.039+0000 7f0b7bba9b80 10 client.8841 fetch_fsmap finished waiting for FSMap version 107 2023-10-06T10:45:24.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.039+0000 7f0b7bba9b80 10 client.8841 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2023-10-06T10:45:24.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.039+0000 7f0b7bba9b80 20 client.8841 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:24.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.039+0000 7f0b7bba9b80 1 --2- 172.21.15.92:0/2557639849 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x562cb520bb70 0x562cb5207b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:24.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.039+0000 7f0b7bba9b80 4 client.8841 mds_command: new command op to 6719 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:24.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.039+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/2557639849 --> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562cb51b1610 con 0x562cb520bb70 2023-10-06T10:45:24.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.039+0000 7f0b6b7fe700 1 --2- 172.21.15.92:0/2557639849 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x562cb520bb70 0x562cb5207b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:24.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.040+0000 7f0b4a7fc700 10 client.8841 ms_handle_connect on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:24.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.040+0000 7f0b6b7fe700 1 --2- 172.21.15.92:0/2557639849 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x562cb520bb70 0x562cb5207b10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=87cf683ce44daf7c server_cookie=ce32dd6ea206f170 in_seq=0 out_seq=0 2023-10-06T10:45:24.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.040+0000 7f0b4a7fc700 1 -- 172.21.15.92:0/2557639849 <== mds.2 v2:172.21.15.124:6833/2307521923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x562cb51b1610 con 0x562cb520bb70 2023-10-06T10:45:24.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.040+0000 7f0b4a7fc700 10 client.8841 handle_command_reply: tid=0 2023-10-06T10:45:24.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.040+0000 7f0b7bba9b80 1 client.8841 shutdown 2023-10-06T10:45:24.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.040+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/2557639849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e0c20 msgr2=0x562cb5206430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:24.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.040+0000 7f0b7bba9b80 1 --2- 172.21.15.92:0/2557639849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e0c20 0x562cb5206430 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009160 tx=0x7f0b5c05c040 comp rx=0 tx=0).stop 2023-10-06T10:45:24.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.040+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/2557639849 shutdown_connections 2023-10-06T10:45:24.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.040+0000 7f0b7bba9b80 1 --2- 172.21.15.92:0/2557639849 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x562cb520bb70 0x562cb5207b10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.040+0000 7f0b7bba9b80 1 --2- 172.21.15.92:0/2557639849 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cb5226ae0 0x562cb520b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.040+0000 7f0b7bba9b80 1 --2- 172.21.15.92:0/2557639849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e0c20 0x562cb5206430 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.040+0000 7f0b4a7fc700 0 client.8841 ms_handle_reset on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:24.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.040+0000 7f0b7bba9b80 1 --2- 172.21.15.92:0/2557639849 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cb51e29f0 0x562cb5222210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.040+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/2557639849 >> 172.21.15.92:0/2557639849 conn(0x562cb51e10c0 msgr2=0x562cb5224340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:24.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.040+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/2557639849 shutdown_connections 2023-10-06T10:45:24.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.040+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/2557639849 wait complete. 2023-10-06T10:45:24.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.041+0000 7f0b7bba9b80 20 client.8841 trim_cache size 0 max 16384 2023-10-06T10:45:24.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.054+0000 7f0b7bba9b80 1 Processor -- start 2023-10-06T10:45:24.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.054+0000 7f0b7bba9b80 1 -- start start 2023-10-06T10:45:24.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.055+0000 7f0b7bba9b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cb51e10c0 0x562cb520c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:24.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.055+0000 7f0b7bba9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e0c20 0x562cb5205510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:24.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.055+0000 7f0b7bba9b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cb51e29f0 0x562cb5205a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:24.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.055+0000 7f0b7bba9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb4fb2e80 con 0x562cb51e0c20 2023-10-06T10:45:24.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.055+0000 7f0b7bba9b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb5222600 con 0x562cb51e29f0 2023-10-06T10:45:24.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.055+0000 7f0b7bba9b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562cb522aed0 con 0x562cb51e10c0 2023-10-06T10:45:24.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.055+0000 7f0b6b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e0c20 0x562cb5205510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:24.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.055+0000 7f0b6b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e0c20 0x562cb5205510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42860/0 (socket says 172.21.15.92:42860) 2023-10-06T10:45:24.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.055+0000 7f0b6b7fe700 1 -- 172.21.15.92:0/3546119334 learned_addr learned my addr 172.21.15.92:0/3546119334 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:24.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.055+0000 7f0b6bfff700 1 --2- 172.21.15.92:0/3546119334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cb51e10c0 0x562cb520c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:24.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.055+0000 7f0b70e4e700 1 --2- 172.21.15.92:0/3546119334 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cb51e29f0 0x562cb5205a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:24.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.055+0000 7f0b6b7fe700 1 -- 172.21.15.92:0/3546119334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cb51e10c0 msgr2=0x562cb520c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:24.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.055+0000 7f0b6b7fe700 1 --2- 172.21.15.92:0/3546119334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cb51e10c0 0x562cb520c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.055+0000 7f0b6b7fe700 1 -- 172.21.15.92:0/3546119334 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cb51e29f0 msgr2=0x562cb5205a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:24.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.055+0000 7f0b6b7fe700 1 --2- 172.21.15.92:0/3546119334 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cb51e29f0 0x562cb5205a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.055+0000 7f0b6b7fe700 1 -- 172.21.15.92:0/3546119334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cb50fe7d0 con 0x562cb51e0c20 2023-10-06T10:45:24.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.056+0000 7f0b6b7fe700 1 --2- 172.21.15.92:0/3546119334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e0c20 0x562cb5205510 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c15a5a0 tx=0x7f0b60014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5610040a12b3a7c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:24.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.056+0000 7f0b51ffb700 1 -- 172.21.15.92:0/3546119334 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6002c460 con 0x562cb51e0c20 2023-10-06T10:45:24.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.056+0000 7f0b51ffb700 1 -- 172.21.15.92:0/3546119334 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6002c5c0 con 0x562cb51e0c20 2023-10-06T10:45:24.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.056+0000 7f0b51ffb700 1 -- 172.21.15.92:0/3546119334 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6002c940 con 0x562cb51e0c20 2023-10-06T10:45:24.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.056+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/3546119334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e0c20 msgr2=0x562cb5205510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:24.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.056+0000 7f0b7bba9b80 1 --2- 172.21.15.92:0/3546119334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e0c20 0x562cb5205510 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c15a5a0 tx=0x7f0b60014000 comp rx=0 tx=0).stop 2023-10-06T10:45:24.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.056+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/3546119334 shutdown_connections 2023-10-06T10:45:24.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.056+0000 7f0b7bba9b80 1 --2- 172.21.15.92:0/3546119334 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cb51e29f0 0x562cb5205a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.056+0000 7f0b7bba9b80 1 --2- 172.21.15.92:0/3546119334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cb51e10c0 0x562cb520c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.056+0000 7f0b7bba9b80 1 --2- 172.21.15.92:0/3546119334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e0c20 0x562cb5205510 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.056+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/3546119334 >> 172.21.15.92:0/3546119334 conn(0x562cb520f750 msgr2=0x562cb51da790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:24.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.056+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/3546119334 shutdown_connections 2023-10-06T10:45:24.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.056+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/3546119334 wait complete. 2023-10-06T10:45:24.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.057+0000 7f0b7bba9b80 1 -- start start 2023-10-06T10:45:24.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.057+0000 7f0b7bba9b80 1 Processor -- start 2023-10-06T10:45:24.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.058+0000 7f0b7bba9b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cb51e0c20 0x562cb528ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:24.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.058+0000 7f0b7bba9b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cb51e10c0 0x562cb521c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:24.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.058+0000 7f0b7bba9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e29f0 0x562cb521c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:24.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.058+0000 7f0b7bba9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb528feb0 con 0x562cb51e29f0 2023-10-06T10:45:24.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.058+0000 7f0b7bba9b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb51e0780 con 0x562cb51e10c0 2023-10-06T10:45:24.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.058+0000 7f0b7bba9b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562cb5290140 con 0x562cb51e0c20 2023-10-06T10:45:24.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.058+0000 7f0b7bba9b80 10 client.0 _ll_register_callbacks cb 0x562cb4fccf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:24.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.058+0000 7f0b70e4e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e29f0 0x562cb521c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:24.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.058+0000 7f0b70e4e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e29f0 0x562cb521c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42876/0 (socket says 172.21.15.92:42876) 2023-10-06T10:45:24.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.058+0000 7f0b70e4e700 1 -- 172.21.15.92:0/1759694254 learned_addr learned my addr 172.21.15.92:0/1759694254 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:24.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.058+0000 7f0b6bfff700 1 --2- 172.21.15.92:0/1759694254 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cb51e0c20 0x562cb528ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:24.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.058+0000 7f0b6b7fe700 1 --2- 172.21.15.92:0/1759694254 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cb51e10c0 0x562cb521c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:24.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.058+0000 7f0b70e4e700 1 -- 172.21.15.92:0/1759694254 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cb51e0c20 msgr2=0x562cb528ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:24.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.058+0000 7f0b70e4e700 1 --2- 172.21.15.92:0/1759694254 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cb51e0c20 0x562cb528ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.058+0000 7f0b70e4e700 1 -- 172.21.15.92:0/1759694254 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cb51e10c0 msgr2=0x562cb521c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:24.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.058+0000 7f0b70e4e700 1 --2- 172.21.15.92:0/1759694254 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cb51e10c0 0x562cb521c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.058+0000 7f0b70e4e700 1 -- 172.21.15.92:0/1759694254 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b64001490 con 0x562cb51e29f0 2023-10-06T10:45:24.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.058+0000 7f0b70e4e700 1 --2- 172.21.15.92:0/1759694254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e29f0 0x562cb521c7f0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f0b64014000 tx=0x7f0b64015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=accaf2b0ccdd7899 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:24.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.058+0000 7f0b4a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:24.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.058+0000 7f0b4a7fc700 1 -- 172.21.15.92:0/1759694254 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b64001f60 con 0x562cb51e29f0 2023-10-06T10:45:24.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.058+0000 7f0b4a7fc700 1 -- 172.21.15.92:0/1759694254 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f0b6401c630 con 0x562cb51e29f0 2023-10-06T10:45:24.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.058+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/1759694254 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b64007230 con 0x562cb51e29f0 2023-10-06T10:45:24.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.059+0000 7f0b4a7fc700 1 -- 172.21.15.92:0/1759694254 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b6401ca00 con 0x562cb51e29f0 2023-10-06T10:45:24.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.059+0000 7f0b7bba9b80 10 client.8847 fetch_fsmap learned FSMap version 107 2023-10-06T10:45:24.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.059+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/1759694254 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x562cb50e54c0 con 0x562cb51e29f0 2023-10-06T10:45:24.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.059+0000 7f0b4a7fc700 1 -- 172.21.15.92:0/1759694254 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b64007230 con 0x562cb51e29f0 2023-10-06T10:45:24.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.059+0000 7f0b4a7fc700 1 -- 172.21.15.92:0/1759694254 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 107) v1 ==== 3136+0+0 (secure 0 0 0) 0x7f0b64012070 con 0x562cb51e29f0 2023-10-06T10:45:24.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.059+0000 7f0b7bba9b80 10 client.8847 fetch_fsmap finished waiting for FSMap version 107 2023-10-06T10:45:24.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.059+0000 7f0b7bba9b80 10 client.8847 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2023-10-06T10:45:24.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.059+0000 7f0b7bba9b80 20 client.8847 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:24.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.059+0000 7f0b7bba9b80 1 --2- 172.21.15.92:0/1759694254 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x562cb5226ae0 0x562cb51df4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:24.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.059+0000 7f0b7bba9b80 4 client.8847 mds_command: new command op to 6719 tid=0[{"prefix": "flush journal"}] 2023-10-06T10:45:24.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.059+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/1759694254 --> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562cb50d2e50 con 0x562cb5226ae0 2023-10-06T10:45:24.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.059+0000 7f0b6bfff700 1 --2- 172.21.15.92:0/1759694254 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x562cb5226ae0 0x562cb51df4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:24.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.059+0000 7f0b4a7fc700 10 client.8847 ms_handle_connect on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:24.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.059+0000 7f0b6bfff700 1 --2- 172.21.15.92:0/1759694254 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x562cb5226ae0 0x562cb51df4b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3df852aafc16294b server_cookie=6f45da2668d47d03 in_seq=0 out_seq=0 2023-10-06T10:45:24.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.070+0000 7f0b4a7fc700 1 -- 172.21.15.92:0/1759694254 <== mds.2 v2:172.21.15.124:6833/2307521923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562cb50d2e50 con 0x562cb5226ae0 2023-10-06T10:45:24.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.070+0000 7f0b4a7fc700 10 client.8847 handle_command_reply: tid=0 2023-10-06T10:45:24.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.070+0000 7f0b7bba9b80 1 client.8847 shutdown 2023-10-06T10:45:24.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.070+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/1759694254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e29f0 msgr2=0x562cb521c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:24.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.070+0000 7f0b7bba9b80 1 --2- 172.21.15.92:0/1759694254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e29f0 0x562cb521c7f0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f0b64014000 tx=0x7f0b64015000 comp rx=0 tx=0).stop 2023-10-06T10:45:24.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.070+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/1759694254 shutdown_connections 2023-10-06T10:45:24.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.070+0000 7f0b7bba9b80 1 --2- 172.21.15.92:0/1759694254 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x562cb5226ae0 0x562cb51df4b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.070+0000 7f0b7bba9b80 1 --2- 172.21.15.92:0/1759694254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562cb51e29f0 0x562cb521c7f0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.070+0000 7f0b4a7fc700 0 client.8847 ms_handle_reset on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:24.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.070+0000 7f0b7bba9b80 1 --2- 172.21.15.92:0/1759694254 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562cb51e0c20 0x562cb528ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.070+0000 7f0b7bba9b80 1 --2- 172.21.15.92:0/1759694254 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562cb51e10c0 0x562cb521c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.070+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/1759694254 >> 172.21.15.92:0/1759694254 conn(0x562cb520f750 msgr2=0x562cb51dbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:24.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.070+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/1759694254 shutdown_connections 2023-10-06T10:45:24.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.070+0000 7f0b7bba9b80 1 -- 172.21.15.92:0/1759694254 wait complete. 2023-10-06T10:45:24.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.070+0000 7f0b7bba9b80 20 client.8847 trim_cache size 0 max 16384 2023-10-06T10:45:24.074 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2023-10-06T10:45:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:45:24.074 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:24.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.071+0000 7f0b728b1700 1 -- 172.21.15.92:0/3706841555 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0b58066f30 msgr2=0x7f0b580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:24.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.071+0000 7f0b728b1700 1 --2- 172.21.15.92:0/3706841555 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0b58066f30 0x7f0b580693e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0ca830 tx=0x7f0b60007d00 comp rx=0 tx=0).stop 2023-10-06T10:45:24.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.071+0000 7f0b728b1700 1 -- 172.21.15.92:0/3706841555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b6c0cd4d0 msgr2=0x7f0b6c14ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:24.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.071+0000 7f0b728b1700 1 --2- 172.21.15.92:0/3706841555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b6c0cd4d0 0x7f0b6c14ff50 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0d14c0 tx=0x7f0b5c016000 comp rx=0 tx=0).stop 2023-10-06T10:45:24.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.071+0000 7f0b728b1700 1 -- 172.21.15.92:0/3706841555 shutdown_connections 2023-10-06T10:45:24.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.071+0000 7f0b728b1700 1 --2- 172.21.15.92:0/3706841555 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0b58066f30 0x7f0b580693e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.071+0000 7f0b728b1700 1 --2- 172.21.15.92:0/3706841555 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b6c152cc0 0x7f0b6c153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.071+0000 7f0b728b1700 1 --2- 172.21.15.92:0/3706841555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b6c0cd4d0 0x7f0b6c14ff50 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.071+0000 7f0b728b1700 1 --2- 172.21.15.92:0/3706841555 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b6c15a160 0x7f0b6c152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:24.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.071+0000 7f0b728b1700 1 -- 172.21.15.92:0/3706841555 >> 172.21.15.92:0/3706841555 conn(0x7f0b6c099700 msgr2=0x7f0b6c0b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:24.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.071+0000 7f0b728b1700 1 -- 172.21.15.92:0/3706841555 shutdown_connections 2023-10-06T10:45:24.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:24.071+0000 7f0b728b1700 1 -- 172.21.15.92:0/3706841555 wait complete. 2023-10-06T10:45:24.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:45:24.084 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2023-10-06T10:45:24.104 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/ 2023-10-06T10:45:24.104 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4 2023-10-06T10:45:24.104 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4/d5 2023-10-06T10:45:24.104 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4/d5/d6 2023-10-06T10:45:24.104 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4/d5/d6/d7 2023-10-06T10:45:24.104 INFO:teuthology.orchestra.run.smithi092.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2023-10-06T10:45:24.105 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:45:24.105 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:45:24.105 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:45:24.105 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:45:24.105 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2023-10-06T10:45:24.105 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:45:24.209 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627778 2023-10-06T10:45:24.397 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:45:24.397 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:45:24.397 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:45:24.398 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:45:24.398 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2023-10-06T10:45:24.398 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:45:24.501 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627779 2023-10-06T10:45:24.691 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:45:24.691 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:45:24.691 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:45:24.691 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:45:24.691 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2023-10-06T10:45:24.691 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:45:24.793 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627780 2023-10-06T10:45:24.983 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:45:24.983 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:45:24.984 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:45:24.984 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:45:24.984 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2023-10-06T10:45:24.984 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:45:25.087 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627781 2023-10-06T10:45:25.277 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:45:25.277 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:45:25.277 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:45:25.277 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:45:25.278 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2023-10-06T10:45:25.278 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:45:25.381 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627782 2023-10-06T10:45:25.569 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T10:45:25.569 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-10-06T10:45:25.570 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-10-06T10:45:25.570 DEBUG:teuthology.orchestra.run.smithi092:> 2023-10-06T10:45:25.570 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2023-10-06T10:45:25.570 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-10-06T10:45:25.674 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627783 2023-10-06T10:45:25.863 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2023-10-06T10:45:25.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.939+0000 7f259da5a000 1 Processor -- start 2023-10-06T10:45:25.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.939+0000 7f259da5a000 1 -- start start 2023-10-06T10:45:25.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.939+0000 7f259da5a000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3502869c0 0x55f35036a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:25.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.939+0000 7f259da5a000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f350286dc0 0x55f3502c4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:25.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.939+0000 7f259da5a000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f35036cc10 0x55f35029a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:25.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.939+0000 7f259da5a000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35023f320 con 0x55f3502869c0 2023-10-06T10:45:25.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.939+0000 7f259da5a000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3501e3980 con 0x55f350286dc0 2023-10-06T10:45:25.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.939+0000 7f259da5a000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3501e3620 con 0x55f35036cc10 2023-10-06T10:45:25.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.940+0000 7f2596c95700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3502869c0 0x55f35036a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:25.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.940+0000 7f2596c95700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3502869c0 0x55f35036a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42894/0 (socket says 172.21.15.92:42894) 2023-10-06T10:45:25.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.940+0000 7f2596c95700 1 -- 172.21.15.92:0/426848473 learned_addr learned my addr 172.21.15.92:0/426848473 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:25.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.940+0000 7f2596494700 1 --2- 172.21.15.92:0/426848473 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f350286dc0 0x55f3502c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:25.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.940+0000 7f2597496700 1 --2- 172.21.15.92:0/426848473 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f35036cc10 0x55f35029a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:25.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.940+0000 7f2596c95700 1 -- 172.21.15.92:0/426848473 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f35036cc10 msgr2=0x55f35029a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:25.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.940+0000 7f2596c95700 1 --2- 172.21.15.92:0/426848473 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f35036cc10 0x55f35029a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:25.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.940+0000 7f2596c95700 1 -- 172.21.15.92:0/426848473 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f350286dc0 msgr2=0x55f3502c4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:25.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.940+0000 7f2596c95700 1 --2- 172.21.15.92:0/426848473 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f350286dc0 0x55f3502c4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:25.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.940+0000 7f2596c95700 1 -- 172.21.15.92:0/426848473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2590005cf0 con 0x55f3502869c0 2023-10-06T10:45:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.940+0000 7f2596c95700 1 --2- 172.21.15.92:0/426848473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3502869c0 0x55f35036a650 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f25880017b0 tx=0x7f2588006c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae05bcf43d5c3a39 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.940+0000 7f2595492700 1 -- 172.21.15.92:0/426848473 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588006320 con 0x55f3502869c0 2023-10-06T10:45:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.940+0000 7f2595492700 1 -- 172.21.15.92:0/426848473 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2588004520 con 0x55f3502869c0 2023-10-06T10:45:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.940+0000 7f2595492700 1 -- 172.21.15.92:0/426848473 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25880048e0 con 0x55f3502869c0 2023-10-06T10:45:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.940+0000 7f259da5a000 1 -- 172.21.15.92:0/426848473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3502869c0 msgr2=0x55f35036a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.940+0000 7f259da5a000 1 --2- 172.21.15.92:0/426848473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3502869c0 0x55f35036a650 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f25880017b0 tx=0x7f2588006c30 comp rx=0 tx=0).stop 2023-10-06T10:45:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.941+0000 7f259da5a000 1 -- 172.21.15.92:0/426848473 shutdown_connections 2023-10-06T10:45:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.941+0000 7f259da5a000 1 --2- 172.21.15.92:0/426848473 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f35036cc10 0x55f35029a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.941+0000 7f259da5a000 1 --2- 172.21.15.92:0/426848473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3502869c0 0x55f35036a650 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.941+0000 7f259da5a000 1 --2- 172.21.15.92:0/426848473 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f350286dc0 0x55f3502c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.941+0000 7f259da5a000 1 -- 172.21.15.92:0/426848473 >> 172.21.15.92:0/426848473 conn(0x55f350299750 msgr2=0x55f3502c5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:25.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.941+0000 7f259da5a000 1 -- 172.21.15.92:0/426848473 shutdown_connections 2023-10-06T10:45:25.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.941+0000 7f259da5a000 1 -- 172.21.15.92:0/426848473 wait complete. 2023-10-06T10:45:25.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.941+0000 7f259da5a000 1 Processor -- start 2023-10-06T10:45:25.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.941+0000 7f259da5a000 1 -- start start 2023-10-06T10:45:25.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.942+0000 7f259da5a000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3502869c0 0x55f3502d9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:25.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.942+0000 7f259da5a000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f350286dc0 0x55f3502c4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:25.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.942+0000 7f259da5a000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f35036cc10 0x55f3502dcf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:25.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.942+0000 7f259da5a000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3501da400 con 0x55f3502869c0 2023-10-06T10:45:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.942+0000 7f259da5a000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3502c4cc0 con 0x55f35036cc10 2023-10-06T10:45:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.942+0000 7f259da5a000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f350368c40 con 0x55f350286dc0 2023-10-06T10:45:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.942+0000 7f2596c95700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3502869c0 0x55f3502d9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.942+0000 7f2596c95700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3502869c0 0x55f3502d9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42898/0 (socket says 172.21.15.92:42898) 2023-10-06T10:45:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.942+0000 7f2596c95700 1 -- 172.21.15.92:0/2510784286 learned_addr learned my addr 172.21.15.92:0/2510784286 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.942+0000 7f2597496700 1 --2- 172.21.15.92:0/2510784286 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f35036cc10 0x55f3502dcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.942+0000 7f2596494700 1 --2- 172.21.15.92:0/2510784286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f350286dc0 0x55f3502c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.942+0000 7f2596c95700 1 -- 172.21.15.92:0/2510784286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f350286dc0 msgr2=0x55f3502c4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.942+0000 7f2596c95700 1 --2- 172.21.15.92:0/2510784286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f350286dc0 0x55f3502c4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.942+0000 7f2596c95700 1 -- 172.21.15.92:0/2510784286 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f35036cc10 msgr2=0x55f3502dcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.942+0000 7f2596c95700 1 --2- 172.21.15.92:0/2510784286 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f35036cc10 0x55f3502dcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.942+0000 7f2596c95700 1 -- 172.21.15.92:0/2510784286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2588001360 con 0x55f3502869c0 2023-10-06T10:45:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.942+0000 7f2596494700 1 --2- 172.21.15.92:0/2510784286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f350286dc0 0x55f3502c4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.942+0000 7f2596c95700 1 --2- 172.21.15.92:0/2510784286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3502869c0 0x55f3502d9ed0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f2588006550 tx=0x7f2588014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.942+0000 7f25877fe700 1 -- 172.21.15.92:0/2510784286 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588013630 con 0x55f3502869c0 2023-10-06T10:45:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.943+0000 7f25877fe700 1 -- 172.21.15.92:0/2510784286 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25880137b0 con 0x55f3502869c0 2023-10-06T10:45:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.943+0000 7f259da5a000 1 -- 172.21.15.92:0/2510784286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f3501e5ae0 con 0x55f3502869c0 2023-10-06T10:45:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.943+0000 7f25877fe700 1 -- 172.21.15.92:0/2510784286 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588013b70 con 0x55f3502869c0 2023-10-06T10:45:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.943+0000 7f259da5a000 1 -- 172.21.15.92:0/2510784286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3501e5540 con 0x55f3502869c0 2023-10-06T10:45:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.944+0000 7f25877fe700 1 -- 172.21.15.92:0/2510784286 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2588028460 con 0x55f3502869c0 2023-10-06T10:45:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.944+0000 7f25877fe700 1 --2- 172.21.15.92:0/2510784286 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f257c065890 0x7f257c067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.944+0000 7f25877fe700 1 -- 172.21.15.92:0/2510784286 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f2588085160 con 0x55f3502869c0 2023-10-06T10:45:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.944+0000 7f2596494700 1 --2- 172.21.15.92:0/2510784286 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f257c065890 0x7f257c067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.944+0000 7f259da5a000 1 --2- 172.21.15.92:0/2510784286 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x55f350367250 0x55f350369070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.944+0000 7f259da5a000 1 -- 172.21.15.92:0/2510784286 --> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] -- osd_op(unknown.0.0:1 16.12 16:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55f3503695b0 con 0x55f350367250 2023-10-06T10:45:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.945+0000 7f2596494700 1 --2- 172.21.15.92:0/2510784286 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f257c065890 0x7f257c067d40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f258c00f640 tx=0x7f258c003890 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.945+0000 7f2597496700 1 --2- 172.21.15.92:0/2510784286 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x55f350367250 0x55f350369070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.945+0000 7f2597496700 1 --2- 172.21.15.92:0/2510784286 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x55f350367250 0x55f350369070 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.946+0000 7f2597496700 1 -- 172.21.15.92:0/2510784286 <== osd.6 v2:172.21.15.124:6824/90914 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v104'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2590012450 con 0x55f350367250 2023-10-06T10:45:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.946+0000 7f259da5a000 1 -- 172.21.15.92:0/2510784286 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x55f350367250 msgr2=0x55f350369070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.946+0000 7f259da5a000 1 --2- 172.21.15.92:0/2510784286 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x55f350367250 0x55f350369070 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.946+0000 7f259da5a000 1 -- 172.21.15.92:0/2510784286 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f257c065890 msgr2=0x7f257c067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.946+0000 7f259da5a000 1 --2- 172.21.15.92:0/2510784286 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f257c065890 0x7f257c067d40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f258c00f640 tx=0x7f258c003890 comp rx=0 tx=0).stop 2023-10-06T10:45:25.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.946+0000 7f259da5a000 1 -- 172.21.15.92:0/2510784286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3502869c0 msgr2=0x55f3502d9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:25.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.946+0000 7f259da5a000 1 --2- 172.21.15.92:0/2510784286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3502869c0 0x55f3502d9ed0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f2588006550 tx=0x7f2588014000 comp rx=0 tx=0).stop 2023-10-06T10:45:25.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.946+0000 7f2597496700 1 -- 172.21.15.92:0/2510784286 reap_dead start 2023-10-06T10:45:25.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.946+0000 7f259da5a000 1 -- 172.21.15.92:0/2510784286 shutdown_connections 2023-10-06T10:45:25.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.946+0000 7f259da5a000 1 -- 172.21.15.92:0/2510784286 >> 172.21.15.92:0/2510784286 conn(0x55f350299750 msgr2=0x55f3502c5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:25.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.947+0000 7f259da5a000 1 -- 172.21.15.92:0/2510784286 shutdown_connections 2023-10-06T10:45:25.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:25.947+0000 7f259da5a000 1 -- 172.21.15.92:0/2510784286 wait complete. 2023-10-06T10:45:25.954 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2023-10-06T10:45:26.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.030+0000 7f34687cb000 1 Processor -- start 2023-10-06T10:45:26.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.030+0000 7f34687cb000 1 -- start start 2023-10-06T10:45:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.031+0000 7f34687cb000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56302130c990 0x56302124db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.031+0000 7f34687cb000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563021324370 0x56302124e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.031+0000 7f34687cb000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563021324d20 0x56302132c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.031+0000 7f34687cb000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563021197980 con 0x563021324370 2023-10-06T10:45:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.031+0000 7f34687cb000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563021197620 con 0x56302130c990 2023-10-06T10:45:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.031+0000 7f34687cb000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56302118e400 con 0x563021324d20 2023-10-06T10:45:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.031+0000 7f3459205700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563021324370 0x56302124e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.031+0000 7f3459205700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563021324370 0x56302124e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42918/0 (socket says 172.21.15.92:42918) 2023-10-06T10:45:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.031+0000 7f3459205700 1 -- 172.21.15.92:0/2774867397 learned_addr learned my addr 172.21.15.92:0/2774867397 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.031+0000 7f3462207700 1 --2- 172.21.15.92:0/2774867397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563021324d20 0x56302132c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.031+0000 7f3461a06700 1 --2- 172.21.15.92:0/2774867397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56302130c990 0x56302124db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.031+0000 7f3459205700 1 -- 172.21.15.92:0/2774867397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563021324d20 msgr2=0x56302132c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.031+0000 7f3459205700 1 --2- 172.21.15.92:0/2774867397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563021324d20 0x56302132c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.031+0000 7f3459205700 1 -- 172.21.15.92:0/2774867397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56302130c990 msgr2=0x56302124db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.031+0000 7f3459205700 1 --2- 172.21.15.92:0/2774867397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56302130c990 0x56302124db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.031+0000 7f3459205700 1 -- 172.21.15.92:0/2774867397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3454005db0 con 0x563021324370 2023-10-06T10:45:26.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.032+0000 7f3459205700 1 --2- 172.21.15.92:0/2774867397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563021324370 0x56302124e040 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f344c00a870 tx=0x7f344c005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33488de4f3c82e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.032+0000 7f3460a04700 1 -- 172.21.15.92:0/2774867397 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c00f070 con 0x563021324370 2023-10-06T10:45:26.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.032+0000 7f3460a04700 1 -- 172.21.15.92:0/2774867397 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344c008910 con 0x563021324370 2023-10-06T10:45:26.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.032+0000 7f3460a04700 1 -- 172.21.15.92:0/2774867397 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c008cb0 con 0x563021324370 2023-10-06T10:45:26.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.032+0000 7f34687cb000 1 -- 172.21.15.92:0/2774867397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563021324370 msgr2=0x56302124e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.032+0000 7f34687cb000 1 --2- 172.21.15.92:0/2774867397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563021324370 0x56302124e040 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f344c00a870 tx=0x7f344c005d00 comp rx=0 tx=0).stop 2023-10-06T10:45:26.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.032+0000 7f34687cb000 1 -- 172.21.15.92:0/2774867397 shutdown_connections 2023-10-06T10:45:26.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.032+0000 7f34687cb000 1 --2- 172.21.15.92:0/2774867397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563021324d20 0x56302132c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.032+0000 7f34687cb000 1 --2- 172.21.15.92:0/2774867397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56302130c990 0x56302124db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.032+0000 7f34687cb000 1 --2- 172.21.15.92:0/2774867397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563021324370 0x56302124e040 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.032+0000 7f34687cb000 1 -- 172.21.15.92:0/2774867397 >> 172.21.15.92:0/2774867397 conn(0x56302124d520 msgr2=0x56302130abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.032+0000 7f34687cb000 1 -- 172.21.15.92:0/2774867397 shutdown_connections 2023-10-06T10:45:26.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.032+0000 7f34687cb000 1 -- 172.21.15.92:0/2774867397 wait complete. 2023-10-06T10:45:26.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.033+0000 7f34687cb000 1 Processor -- start 2023-10-06T10:45:26.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.033+0000 7f34687cb000 1 -- start start 2023-10-06T10:45:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.033+0000 7f34687cb000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56302130c990 0x56302126e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.033+0000 7f34687cb000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563021324370 0x56302127afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.034+0000 7f34687cb000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563021324d20 0x56302127b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.034+0000 7f34687cb000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630212766b0 con 0x56302130c990 2023-10-06T10:45:26.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.034+0000 7f34687cb000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563021291c20 con 0x563021324370 2023-10-06T10:45:26.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.034+0000 7f34687cb000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56302127d170 con 0x563021324d20 2023-10-06T10:45:26.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.034+0000 7f3461a06700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56302130c990 0x56302126e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.034+0000 7f3461a06700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56302130c990 0x56302126e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42930/0 (socket says 172.21.15.92:42930) 2023-10-06T10:45:26.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.034+0000 7f3461a06700 1 -- 172.21.15.92:0/2774252988 learned_addr learned my addr 172.21.15.92:0/2774252988 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.034+0000 7f3459205700 1 --2- 172.21.15.92:0/2774252988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563021324370 0x56302127afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.034+0000 7f3462207700 1 --2- 172.21.15.92:0/2774252988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563021324d20 0x56302127b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.034+0000 7f3461a06700 1 -- 172.21.15.92:0/2774252988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563021324d20 msgr2=0x56302127b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.034+0000 7f3461a06700 1 --2- 172.21.15.92:0/2774252988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563021324d20 0x56302127b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.034+0000 7f3461a06700 1 -- 172.21.15.92:0/2774252988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563021324370 msgr2=0x56302127afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.034+0000 7f3461a06700 1 --2- 172.21.15.92:0/2774252988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563021324370 0x56302127afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.034+0000 7f3461a06700 1 -- 172.21.15.92:0/2774252988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c00a440 con 0x56302130c990 2023-10-06T10:45:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.034+0000 7f3461a06700 1 --2- 172.21.15.92:0/2774252988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56302130c990 0x56302126e9d0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f345400c010 tx=0x7f3454016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.034+0000 7f345affd700 1 -- 172.21.15.92:0/2774252988 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34540129b0 con 0x56302130c990 2023-10-06T10:45:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.034+0000 7f345affd700 1 -- 172.21.15.92:0/2774252988 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3454012b30 con 0x56302130c990 2023-10-06T10:45:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.034+0000 7f34687cb000 1 -- 172.21.15.92:0/2774252988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5630211a63d0 con 0x56302130c990 2023-10-06T10:45:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.034+0000 7f345affd700 1 -- 172.21.15.92:0/2774252988 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3454026410 con 0x56302130c990 2023-10-06T10:45:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.034+0000 7f34687cb000 1 -- 172.21.15.92:0/2774252988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563021199ae0 con 0x56302130c990 2023-10-06T10:45:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.035+0000 7f345affd700 1 -- 172.21.15.92:0/2774252988 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3454026800 con 0x56302130c990 2023-10-06T10:45:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.036+0000 7f345affd700 1 --2- 172.21.15.92:0/2774252988 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f34440658c0 0x7f3444067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.036+0000 7f3459205700 1 --2- 172.21.15.92:0/2774252988 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f34440658c0 0x7f3444067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.036+0000 7f345affd700 1 -- 172.21.15.92:0/2774252988 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f3454085350 con 0x56302130c990 2023-10-06T10:45:26.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.036+0000 7f34687cb000 1 --2- 172.21.15.92:0/2774252988 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x56302126ef10 0x5630213072b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.036+0000 7f34687cb000 1 -- 172.21.15.92:0/2774252988 --> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] -- osd_op(unknown.0.0:1 16.12 16:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x56302127cc30 con 0x56302126ef10 2023-10-06T10:45:26.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.036+0000 7f3459205700 1 --2- 172.21.15.92:0/2774252988 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f34440658c0 0x7f3444067d70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f344c00bfd0 tx=0x7f344c018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.036+0000 7f3462207700 1 --2- 172.21.15.92:0/2774252988 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x56302126ef10 0x5630213072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.037+0000 7f3462207700 1 --2- 172.21.15.92:0/2774252988 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x56302126ef10 0x5630213072b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.038+0000 7f3462207700 1 -- 172.21.15.92:0/2774252988 <== osd.6 v2:172.21.15.124:6824/90914 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v104'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f345c003970 con 0x56302126ef10 2023-10-06T10:45:26.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.038+0000 7f34687cb000 1 -- 172.21.15.92:0/2774252988 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x56302126ef10 msgr2=0x5630213072b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.038+0000 7f34687cb000 1 --2- 172.21.15.92:0/2774252988 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x56302126ef10 0x5630213072b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.038+0000 7f34687cb000 1 -- 172.21.15.92:0/2774252988 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f34440658c0 msgr2=0x7f3444067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.038+0000 7f34687cb000 1 --2- 172.21.15.92:0/2774252988 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f34440658c0 0x7f3444067d70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f344c00bfd0 tx=0x7f344c018000 comp rx=0 tx=0).stop 2023-10-06T10:45:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.038+0000 7f34687cb000 1 -- 172.21.15.92:0/2774252988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56302130c990 msgr2=0x56302126e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.038+0000 7f34687cb000 1 --2- 172.21.15.92:0/2774252988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56302130c990 0x56302126e9d0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f345400c010 tx=0x7f3454016000 comp rx=0 tx=0).stop 2023-10-06T10:45:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.038+0000 7f3462207700 1 -- 172.21.15.92:0/2774252988 reap_dead start 2023-10-06T10:45:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.038+0000 7f34687cb000 1 -- 172.21.15.92:0/2774252988 shutdown_connections 2023-10-06T10:45:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.038+0000 7f34687cb000 1 -- 172.21.15.92:0/2774252988 >> 172.21.15.92:0/2774252988 conn(0x56302124d520 msgr2=0x56302124d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.038+0000 7f34687cb000 1 -- 172.21.15.92:0/2774252988 shutdown_connections 2023-10-06T10:45:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.038+0000 7f34687cb000 1 -- 172.21.15.92:0/2774252988 wait complete. 2023-10-06T10:45:26.045 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2023-10-06T10:45:26.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.121+0000 7f32756b9000 1 Processor -- start 2023-10-06T10:45:26.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.122+0000 7f32756b9000 1 -- start start 2023-10-06T10:45:26.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.122+0000 7f32756b9000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560c723b94f0 0x560c723b98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.122+0000 7f32756b9000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560c723b9ec0 0x560c723c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.122+0000 7f32756b9000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c723c5240 0x560c723c5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.122+0000 7f32756b9000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7222e980 con 0x560c723c5240 2023-10-06T10:45:26.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.122+0000 7f32756b9000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7222e620 con 0x560c723b94f0 2023-10-06T10:45:26.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.122+0000 7f32756b9000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c72225400 con 0x560c723b9ec0 2023-10-06T10:45:26.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.122+0000 7f326f0f5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c723c5240 0x560c723c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.122+0000 7f326f0f5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c723c5240 0x560c723c5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42950/0 (socket says 172.21.15.92:42950) 2023-10-06T10:45:26.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.122+0000 7f326f0f5700 1 -- 172.21.15.92:0/726907843 learned_addr learned my addr 172.21.15.92:0/726907843 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.122+0000 7f326e8f4700 1 --2- 172.21.15.92:0/726907843 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560c723b94f0 0x560c723b98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.122+0000 7f326e0f3700 1 --2- 172.21.15.92:0/726907843 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560c723b9ec0 0x560c723c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.122+0000 7f326f0f5700 1 -- 172.21.15.92:0/726907843 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560c723b9ec0 msgr2=0x560c723c4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.122+0000 7f326f0f5700 1 --2- 172.21.15.92:0/726907843 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560c723b9ec0 0x560c723c4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.122+0000 7f326f0f5700 1 -- 172.21.15.92:0/726907843 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560c723b94f0 msgr2=0x560c723b98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.122+0000 7f326f0f5700 1 --2- 172.21.15.92:0/726907843 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560c723b94f0 0x560c723b98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.122+0000 7f326f0f5700 1 -- 172.21.15.92:0/726907843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c7223d3d0 con 0x560c723c5240 2023-10-06T10:45:26.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.123+0000 7f326f0f5700 1 --2- 172.21.15.92:0/726907843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c723c5240 0x560c723c5610 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x560c722e6850 tx=0x7f326800c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e017d72cfb94c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.123+0000 7f326d0f1700 1 -- 172.21.15.92:0/726907843 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326800f040 con 0x560c723c5240 2023-10-06T10:45:26.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.123+0000 7f326d0f1700 1 -- 172.21.15.92:0/726907843 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3268007650 con 0x560c723c5240 2023-10-06T10:45:26.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.123+0000 7f326d0f1700 1 -- 172.21.15.92:0/726907843 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32680079f0 con 0x560c723c5240 2023-10-06T10:45:26.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.123+0000 7f32756b9000 1 -- 172.21.15.92:0/726907843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c723c5240 msgr2=0x560c723c5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.123+0000 7f32756b9000 1 --2- 172.21.15.92:0/726907843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c723c5240 0x560c723c5610 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x560c722e6850 tx=0x7f326800c040 comp rx=0 tx=0).stop 2023-10-06T10:45:26.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.123+0000 7f32756b9000 1 -- 172.21.15.92:0/726907843 shutdown_connections 2023-10-06T10:45:26.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.123+0000 7f32756b9000 1 --2- 172.21.15.92:0/726907843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c723c5240 0x560c723c5610 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.123+0000 7f32756b9000 1 --2- 172.21.15.92:0/726907843 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560c723b94f0 0x560c723b98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.123+0000 7f32756b9000 1 --2- 172.21.15.92:0/726907843 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560c723b9ec0 0x560c723c4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.123+0000 7f32756b9000 1 -- 172.21.15.92:0/726907843 >> 172.21.15.92:0/726907843 conn(0x560c722e4750 msgr2=0x560c722e4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.123+0000 7f32756b9000 1 -- 172.21.15.92:0/726907843 shutdown_connections 2023-10-06T10:45:26.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.123+0000 7f32756b9000 1 -- 172.21.15.92:0/726907843 wait complete. 2023-10-06T10:45:26.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.124+0000 7f32756b9000 1 Processor -- start 2023-10-06T10:45:26.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.124+0000 7f32756b9000 1 -- start start 2023-10-06T10:45:26.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.124+0000 7f32756b9000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560c723b94f0 0x560c723b2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.125+0000 7f32756b9000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c723b9ec0 0x560c723a97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.125+0000 7f32756b9000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560c723c5240 0x560c723b0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.125+0000 7f32756b9000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c723b2610 con 0x560c723b9ec0 2023-10-06T10:45:26.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.125+0000 7f32756b9000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c723aaa30 con 0x560c723b94f0 2023-10-06T10:45:26.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.125+0000 7f32756b9000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c723aacd0 con 0x560c723c5240 2023-10-06T10:45:26.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.125+0000 7f326e0f3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c723b9ec0 0x560c723a97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.125+0000 7f326e0f3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c723b9ec0 0x560c723a97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42952/0 (socket says 172.21.15.92:42952) 2023-10-06T10:45:26.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.125+0000 7f326e0f3700 1 -- 172.21.15.92:0/1249903772 learned_addr learned my addr 172.21.15.92:0/1249903772 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.125+0000 7f326e8f4700 1 --2- 172.21.15.92:0/1249903772 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560c723b94f0 0x560c723b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.125+0000 7f326f0f5700 1 --2- 172.21.15.92:0/1249903772 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560c723c5240 0x560c723b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.125+0000 7f326e0f3700 1 -- 172.21.15.92:0/1249903772 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560c723c5240 msgr2=0x560c723b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.125+0000 7f326e0f3700 1 --2- 172.21.15.92:0/1249903772 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x560c723c5240 0x560c723b0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.125+0000 7f326e0f3700 1 -- 172.21.15.92:0/1249903772 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560c723b94f0 msgr2=0x560c723b2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.125+0000 7f326e0f3700 1 --2- 172.21.15.92:0/1249903772 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560c723b94f0 0x560c723b2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.125+0000 7f326e0f3700 1 -- 172.21.15.92:0/1249903772 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3268009430 con 0x560c723b9ec0 2023-10-06T10:45:26.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.125+0000 7f326e8f4700 1 --2- 172.21.15.92:0/1249903772 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x560c723b94f0 0x560c723b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:26.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.125+0000 7f326e0f3700 1 --2- 172.21.15.92:0/1249903772 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c723b9ec0 0x560c723a97c0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f325c00a330 tx=0x7f3258015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.125+0000 7f32677fe700 1 -- 172.21.15.92:0/1249903772 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325800e040 con 0x560c723b9ec0 2023-10-06T10:45:26.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.125+0000 7f32677fe700 1 -- 172.21.15.92:0/1249903772 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3258027430 con 0x560c723b9ec0 2023-10-06T10:45:26.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.125+0000 7f32677fe700 1 -- 172.21.15.92:0/1249903772 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32580277d0 con 0x560c723b9ec0 2023-10-06T10:45:26.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.125+0000 7f32756b9000 1 -- 172.21.15.92:0/1249903772 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560c72230ae0 con 0x560c723b9ec0 2023-10-06T10:45:26.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.125+0000 7f32756b9000 1 -- 172.21.15.92:0/1249903772 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c72230540 con 0x560c723b9ec0 2023-10-06T10:45:26.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.126+0000 7f32677fe700 1 -- 172.21.15.92:0/1249903772 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3258013420 con 0x560c723b9ec0 2023-10-06T10:45:26.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.127+0000 7f32677fe700 1 --2- 172.21.15.92:0/1249903772 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f32500658c0 0x7f3250067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.127+0000 7f32677fe700 1 -- 172.21.15.92:0/1249903772 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f32580847b0 con 0x560c723b9ec0 2023-10-06T10:45:26.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.127+0000 7f326e8f4700 1 --2- 172.21.15.92:0/1249903772 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f32500658c0 0x7f3250067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.127+0000 7f32756b9000 1 --2- 172.21.15.92:0/1249903772 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x560c723b8760 0x560c723b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.127+0000 7f32756b9000 1 -- 172.21.15.92:0/1249903772 --> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] -- osd_op(unknown.0.0:1 16.5 16:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x560c723b4d00 con 0x560c723b8760 2023-10-06T10:45:26.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.127+0000 7f326f0f5700 1 --2- 172.21.15.92:0/1249903772 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x560c723b8760 0x560c723b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.127+0000 7f326e8f4700 1 --2- 172.21.15.92:0/1249903772 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f32500658c0 0x7f3250067d70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3260001f80 tx=0x7f3260005040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.127+0000 7f326f0f5700 1 --2- 172.21.15.92:0/1249903772 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x560c723b8760 0x560c723b47c0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.129+0000 7f326f0f5700 1 -- 172.21.15.92:0/1249903772 <== osd.0 v2:172.21.15.92:6800/90604 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v104'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f326801d040 con 0x560c723b8760 2023-10-06T10:45:26.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.129+0000 7f32756b9000 1 -- 172.21.15.92:0/1249903772 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x560c723b8760 msgr2=0x560c723b47c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.129+0000 7f32756b9000 1 --2- 172.21.15.92:0/1249903772 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x560c723b8760 0x560c723b47c0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.129+0000 7f32756b9000 1 -- 172.21.15.92:0/1249903772 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f32500658c0 msgr2=0x7f3250067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.129+0000 7f32756b9000 1 --2- 172.21.15.92:0/1249903772 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f32500658c0 0x7f3250067d70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3260001f80 tx=0x7f3260005040 comp rx=0 tx=0).stop 2023-10-06T10:45:26.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.129+0000 7f32756b9000 1 -- 172.21.15.92:0/1249903772 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c723b9ec0 msgr2=0x560c723a97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.129+0000 7f32756b9000 1 --2- 172.21.15.92:0/1249903772 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c723b9ec0 0x560c723a97c0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f325c00a330 tx=0x7f3258015000 comp rx=0 tx=0).stop 2023-10-06T10:45:26.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.129+0000 7f326f0f5700 1 -- 172.21.15.92:0/1249903772 reap_dead start 2023-10-06T10:45:26.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.129+0000 7f32756b9000 1 -- 172.21.15.92:0/1249903772 shutdown_connections 2023-10-06T10:45:26.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.129+0000 7f32756b9000 1 -- 172.21.15.92:0/1249903772 >> 172.21.15.92:0/1249903772 conn(0x560c722e4750 msgr2=0x560c722e4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.129+0000 7f32756b9000 1 -- 172.21.15.92:0/1249903772 shutdown_connections 2023-10-06T10:45:26.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.129+0000 7f32756b9000 1 -- 172.21.15.92:0/1249903772 wait complete. 2023-10-06T10:45:26.136 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2023-10-06T10:45:26.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.212+0000 7f17225ed000 1 Processor -- start 2023-10-06T10:45:26.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.212+0000 7f17225ed000 1 -- start start 2023-10-06T10:45:26.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.212+0000 7f17225ed000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771596cfd0 0x557715985980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.212+0000 7f17225ed000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55771596f270 0x5577159fb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.212+0000 7f17225ed000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5577159fdab0 0x5577159fb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.212+0000 7f17225ed000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55771588b980 con 0x55771596cfd0 2023-10-06T10:45:26.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.212+0000 7f17225ed000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55771588b620 con 0x55771596f270 2023-10-06T10:45:26.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.212+0000 7f17225ed000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x557715882400 con 0x5577159fdab0 2023-10-06T10:45:26.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.212+0000 7f171b828700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771596cfd0 0x557715985980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.212+0000 7f171b828700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771596cfd0 0x557715985980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42962/0 (socket says 172.21.15.92:42962) 2023-10-06T10:45:26.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.212+0000 7f171b828700 1 -- 172.21.15.92:0/2123152923 learned_addr learned my addr 172.21.15.92:0/2123152923 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.213+0000 7f171b027700 1 --2- 172.21.15.92:0/2123152923 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55771596f270 0x5577159fb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.213+0000 7f171c029700 1 --2- 172.21.15.92:0/2123152923 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5577159fdab0 0x5577159fb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.213+0000 7f171b828700 1 -- 172.21.15.92:0/2123152923 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5577159fdab0 msgr2=0x5577159fb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.213+0000 7f171b828700 1 --2- 172.21.15.92:0/2123152923 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5577159fdab0 0x5577159fb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.213+0000 7f171b828700 1 -- 172.21.15.92:0/2123152923 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55771596f270 msgr2=0x5577159fb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.213+0000 7f171b828700 1 --2- 172.21.15.92:0/2123152923 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55771596f270 0x5577159fb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.213+0000 7f171b828700 1 -- 172.21.15.92:0/2123152923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1714005db0 con 0x55771596cfd0 2023-10-06T10:45:26.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.213+0000 7f171b027700 1 --2- 172.21.15.92:0/2123152923 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55771596f270 0x5577159fb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:26.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.213+0000 7f171b828700 1 --2- 172.21.15.92:0/2123152923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771596cfd0 0x557715985980 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f170c00a590 tx=0x7f170c005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96d9d201119863b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.213+0000 7f171a025700 1 -- 172.21.15.92:0/2123152923 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c00e510 con 0x55771596cfd0 2023-10-06T10:45:26.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.213+0000 7f171a025700 1 -- 172.21.15.92:0/2123152923 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170c00e690 con 0x55771596cfd0 2023-10-06T10:45:26.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.213+0000 7f171a025700 1 -- 172.21.15.92:0/2123152923 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c00ea50 con 0x55771596cfd0 2023-10-06T10:45:26.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.213+0000 7f17225ed000 1 -- 172.21.15.92:0/2123152923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771596cfd0 msgr2=0x557715985980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.213+0000 7f17225ed000 1 --2- 172.21.15.92:0/2123152923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771596cfd0 0x557715985980 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f170c00a590 tx=0x7f170c005ca0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.213+0000 7f17225ed000 1 -- 172.21.15.92:0/2123152923 shutdown_connections 2023-10-06T10:45:26.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.213+0000 7f17225ed000 1 --2- 172.21.15.92:0/2123152923 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5577159fdab0 0x5577159fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.213+0000 7f17225ed000 1 --2- 172.21.15.92:0/2123152923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771596cfd0 0x557715985980 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.213+0000 7f17225ed000 1 --2- 172.21.15.92:0/2123152923 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55771596f270 0x5577159fb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.213+0000 7f17225ed000 1 -- 172.21.15.92:0/2123152923 >> 172.21.15.92:0/2123152923 conn(0x557715941520 msgr2=0x55771596e270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.214+0000 7f17225ed000 1 -- 172.21.15.92:0/2123152923 shutdown_connections 2023-10-06T10:45:26.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.214+0000 7f17225ed000 1 -- 172.21.15.92:0/2123152923 wait complete. 2023-10-06T10:45:26.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.214+0000 7f17225ed000 1 Processor -- start 2023-10-06T10:45:26.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.214+0000 7f17225ed000 1 -- start start 2023-10-06T10:45:26.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.215+0000 7f17225ed000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771596cfd0 0x5577159f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.215+0000 7f17225ed000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55771596f270 0x557715985640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.215+0000 7f17225ed000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5577159fdab0 0x557715985b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.215+0000 7f17225ed000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577159fb990 con 0x55771596cfd0 2023-10-06T10:45:26.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.215+0000 7f17225ed000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577159f5860 con 0x55771596f270 2023-10-06T10:45:26.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.215+0000 7f17225ed000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5577159f5b00 con 0x5577159fdab0 2023-10-06T10:45:26.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.215+0000 7f171b828700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771596cfd0 0x5577159f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.215+0000 7f171b828700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771596cfd0 0x5577159f5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42968/0 (socket says 172.21.15.92:42968) 2023-10-06T10:45:26.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.215+0000 7f171b828700 1 -- 172.21.15.92:0/4073107223 learned_addr learned my addr 172.21.15.92:0/4073107223 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.215+0000 7f171c029700 1 --2- 172.21.15.92:0/4073107223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5577159fdab0 0x557715985b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.215+0000 7f171b027700 1 --2- 172.21.15.92:0/4073107223 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55771596f270 0x557715985640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.215+0000 7f171b828700 1 -- 172.21.15.92:0/4073107223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5577159fdab0 msgr2=0x557715985b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.215+0000 7f171b828700 1 --2- 172.21.15.92:0/4073107223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5577159fdab0 0x557715985b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.215+0000 7f171b828700 1 -- 172.21.15.92:0/4073107223 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55771596f270 msgr2=0x557715985640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.215+0000 7f171b828700 1 --2- 172.21.15.92:0/4073107223 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55771596f270 0x557715985640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.215+0000 7f171b828700 1 -- 172.21.15.92:0/4073107223 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c00a0e0 con 0x55771596cfd0 2023-10-06T10:45:26.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.215+0000 7f171b828700 1 --2- 172.21.15.92:0/4073107223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771596cfd0 0x5577159f5230 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f170c010000 tx=0x7f170c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.215+0000 7f1718822700 1 -- 172.21.15.92:0/4073107223 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c020070 con 0x55771596cfd0 2023-10-06T10:45:26.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.215+0000 7f1718822700 1 -- 172.21.15.92:0/4073107223 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170c00e510 con 0x55771596cfd0 2023-10-06T10:45:26.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.215+0000 7f1718822700 1 -- 172.21.15.92:0/4073107223 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c00ea30 con 0x55771596cfd0 2023-10-06T10:45:26.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.215+0000 7f17225ed000 1 -- 172.21.15.92:0/4073107223 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55771589a3d0 con 0x55771596cfd0 2023-10-06T10:45:26.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.216+0000 7f17225ed000 1 -- 172.21.15.92:0/4073107223 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55771588dae0 con 0x55771596cfd0 2023-10-06T10:45:26.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.217+0000 7f1718822700 1 -- 172.21.15.92:0/4073107223 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f170c01c640 con 0x55771596cfd0 2023-10-06T10:45:26.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.217+0000 7f1718822700 1 --2- 172.21.15.92:0/4073107223 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f16fc065890 0x7f16fc067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.217+0000 7f1718822700 1 -- 172.21.15.92:0/4073107223 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f170c005e40 con 0x55771596cfd0 2023-10-06T10:45:26.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.217+0000 7f171b027700 1 --2- 172.21.15.92:0/4073107223 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f16fc065890 0x7f16fc067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.217+0000 7f17225ed000 1 --2- 172.21.15.92:0/4073107223 >> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] conn(0x5577159f6780 0x5577159f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.217+0000 7f17225ed000 1 -- 172.21.15.92:0/4073107223 --> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] -- osd_op(unknown.0.0:1 16.9 16:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x5577159f70c0 con 0x5577159f6780 2023-10-06T10:45:26.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.217+0000 7f171c029700 1 --2- 172.21.15.92:0/4073107223 >> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] conn(0x5577159f6780 0x5577159f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.218+0000 7f171b027700 1 --2- 172.21.15.92:0/4073107223 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f16fc065890 0x7f16fc067d40 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f170400aa50 tx=0x7f170400c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.218+0000 7f171c029700 1 --2- 172.21.15.92:0/4073107223 >> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] conn(0x5577159f6780 0x5577159f6b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.219+0000 7f171c029700 1 -- 172.21.15.92:0/4073107223 <== osd.1 v2:172.21.15.92:6816/90606 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v104'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f171401a040 con 0x5577159f6780 2023-10-06T10:45:26.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.219+0000 7f17225ed000 1 -- 172.21.15.92:0/4073107223 >> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] conn(0x5577159f6780 msgr2=0x5577159f6b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.219+0000 7f17225ed000 1 --2- 172.21.15.92:0/4073107223 >> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] conn(0x5577159f6780 0x5577159f6b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.219+0000 7f17225ed000 1 -- 172.21.15.92:0/4073107223 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f16fc065890 msgr2=0x7f16fc067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.219+0000 7f17225ed000 1 --2- 172.21.15.92:0/4073107223 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f16fc065890 0x7f16fc067d40 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f170400aa50 tx=0x7f170400c040 comp rx=0 tx=0).stop 2023-10-06T10:45:26.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.219+0000 7f17225ed000 1 -- 172.21.15.92:0/4073107223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771596cfd0 msgr2=0x5577159f5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.219+0000 7f17225ed000 1 --2- 172.21.15.92:0/4073107223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55771596cfd0 0x5577159f5230 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f170c010000 tx=0x7f170c014000 comp rx=0 tx=0).stop 2023-10-06T10:45:26.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.219+0000 7f171c029700 1 -- 172.21.15.92:0/4073107223 reap_dead start 2023-10-06T10:45:26.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.219+0000 7f17225ed000 1 -- 172.21.15.92:0/4073107223 shutdown_connections 2023-10-06T10:45:26.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.219+0000 7f17225ed000 1 -- 172.21.15.92:0/4073107223 >> 172.21.15.92:0/4073107223 conn(0x557715941520 msgr2=0x55771596e270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.219+0000 7f17225ed000 1 -- 172.21.15.92:0/4073107223 shutdown_connections 2023-10-06T10:45:26.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.219+0000 7f17225ed000 1 -- 172.21.15.92:0/4073107223 wait complete. 2023-10-06T10:45:26.230 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2023-10-06T10:45:26.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.300+0000 7f0689f75000 1 Processor -- start 2023-10-06T10:45:26.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.300+0000 7f0689f75000 1 -- start start 2023-10-06T10:45:26.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.301+0000 7f0689f75000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eca107f270 0x55eca10a6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.301+0000 7f0689f75000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55eca114fc80 0x55eca10aba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.301+0000 7f0689f75000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55eca1150050 0x55eca114ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.301+0000 7f0689f75000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca1023320 con 0x55eca107f270 2023-10-06T10:45:26.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.301+0000 7f0689f75000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca0fc7980 con 0x55eca114fc80 2023-10-06T10:45:26.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.301+0000 7f0689f75000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eca0fc7620 con 0x55eca1150050 2023-10-06T10:45:26.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.301+0000 7f06831b0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eca107f270 0x55eca10a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.301+0000 7f06831b0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eca107f270 0x55eca10a6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42994/0 (socket says 172.21.15.92:42994) 2023-10-06T10:45:26.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.301+0000 7f06831b0700 1 -- 172.21.15.92:0/3361860814 learned_addr learned my addr 172.21.15.92:0/3361860814 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.301+0000 7f06831b0700 1 -- 172.21.15.92:0/3361860814 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55eca1150050 msgr2=0x55eca114ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.301+0000 7f06831b0700 1 --2- 172.21.15.92:0/3361860814 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55eca1150050 0x55eca114ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.301+0000 7f06831b0700 1 -- 172.21.15.92:0/3361860814 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55eca114fc80 msgr2=0x55eca10aba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.301+0000 7f06831b0700 1 --2- 172.21.15.92:0/3361860814 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55eca114fc80 0x55eca10aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.301+0000 7f06831b0700 1 -- 172.21.15.92:0/3361860814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c00a0e0 con 0x55eca107f270 2023-10-06T10:45:26.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.301+0000 7f06831b0700 1 --2- 172.21.15.92:0/3361860814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eca107f270 0x55eca10a6670 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f067400b3d0 tx=0x7f06740017f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c904adac803cd80c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.301+0000 7f06819ad700 1 -- 172.21.15.92:0/3361860814 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067400be40 con 0x55eca107f270 2023-10-06T10:45:26.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.301+0000 7f06819ad700 1 -- 172.21.15.92:0/3361860814 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067400e470 con 0x55eca107f270 2023-10-06T10:45:26.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.301+0000 7f06819ad700 1 -- 172.21.15.92:0/3361860814 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067400e830 con 0x55eca107f270 2023-10-06T10:45:26.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.302+0000 7f0689f75000 1 -- 172.21.15.92:0/3361860814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eca107f270 msgr2=0x55eca10a6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.302+0000 7f0689f75000 1 --2- 172.21.15.92:0/3361860814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eca107f270 0x55eca10a6670 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f067400b3d0 tx=0x7f06740017f0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.302+0000 7f0689f75000 1 -- 172.21.15.92:0/3361860814 shutdown_connections 2023-10-06T10:45:26.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.302+0000 7f0689f75000 1 --2- 172.21.15.92:0/3361860814 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55eca1150050 0x55eca114ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.302+0000 7f0689f75000 1 --2- 172.21.15.92:0/3361860814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eca107f270 0x55eca10a6670 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.302+0000 7f0689f75000 1 --2- 172.21.15.92:0/3361860814 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55eca114fc80 0x55eca10aba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.302+0000 7f0689f75000 1 -- 172.21.15.92:0/3361860814 >> 172.21.15.92:0/3361860814 conn(0x55eca107d750 msgr2=0x55eca114ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.302+0000 7f0689f75000 1 -- 172.21.15.92:0/3361860814 shutdown_connections 2023-10-06T10:45:26.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.302+0000 7f0689f75000 1 -- 172.21.15.92:0/3361860814 wait complete. 2023-10-06T10:45:26.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.302+0000 7f0689f75000 1 Processor -- start 2023-10-06T10:45:26.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.302+0000 7f0689f75000 1 -- start start 2023-10-06T10:45:26.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.303+0000 7f0689f75000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eca107f270 0x55eca1144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.303+0000 7f0689f75000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55eca114fc80 0x55eca107db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.303+0000 7f0689f75000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55eca1150050 0x55eca107e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.303+0000 7f0689f75000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca0fbe400 con 0x55eca107f270 2023-10-06T10:45:26.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.303+0000 7f0689f75000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca1144860 con 0x55eca1150050 2023-10-06T10:45:26.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.303+0000 7f0689f75000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eca11461a0 con 0x55eca114fc80 2023-10-06T10:45:26.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.303+0000 7f06831b0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eca107f270 0x55eca1144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.303+0000 7f06831b0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eca107f270 0x55eca1144320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43010/0 (socket says 172.21.15.92:43010) 2023-10-06T10:45:26.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.303+0000 7f06831b0700 1 -- 172.21.15.92:0/2632102268 learned_addr learned my addr 172.21.15.92:0/2632102268 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.303+0000 7f06829af700 1 --2- 172.21.15.92:0/2632102268 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55eca114fc80 0x55eca107db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.303+0000 7f06839b1700 1 --2- 172.21.15.92:0/2632102268 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55eca1150050 0x55eca107e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.303+0000 7f06831b0700 1 -- 172.21.15.92:0/2632102268 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55eca114fc80 msgr2=0x55eca107db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.304+0000 7f06831b0700 1 --2- 172.21.15.92:0/2632102268 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55eca114fc80 0x55eca107db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.304+0000 7f06831b0700 1 -- 172.21.15.92:0/2632102268 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55eca1150050 msgr2=0x55eca107e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.304+0000 7f06831b0700 1 --2- 172.21.15.92:0/2632102268 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55eca1150050 0x55eca107e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.304+0000 7f06831b0700 1 -- 172.21.15.92:0/2632102268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0674001ad0 con 0x55eca107f270 2023-10-06T10:45:26.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.304+0000 7f06831b0700 1 --2- 172.21.15.92:0/2632102268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eca107f270 0x55eca1144320 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f067400bb20 tx=0x7f0674014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.304+0000 7f066bfff700 1 -- 172.21.15.92:0/2632102268 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067401f070 con 0x55eca107f270 2023-10-06T10:45:26.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.304+0000 7f066bfff700 1 -- 172.21.15.92:0/2632102268 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067401a2e0 con 0x55eca107f270 2023-10-06T10:45:26.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.304+0000 7f0689f75000 1 -- 172.21.15.92:0/2632102268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55eca0fc9ae0 con 0x55eca107f270 2023-10-06T10:45:26.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.304+0000 7f066bfff700 1 -- 172.21.15.92:0/2632102268 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067401a7e0 con 0x55eca107f270 2023-10-06T10:45:26.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.304+0000 7f0689f75000 1 -- 172.21.15.92:0/2632102268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eca0fc9540 con 0x55eca107f270 2023-10-06T10:45:26.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.305+0000 7f066bfff700 1 -- 172.21.15.92:0/2632102268 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f067401ab90 con 0x55eca107f270 2023-10-06T10:45:26.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.305+0000 7f066bfff700 1 --2- 172.21.15.92:0/2632102268 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0670065890 0x7f0670067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.305+0000 7f066bfff700 1 -- 172.21.15.92:0/2632102268 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f06740853e0 con 0x55eca107f270 2023-10-06T10:45:26.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.305+0000 7f06829af700 1 --2- 172.21.15.92:0/2632102268 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0670065890 0x7f0670067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.306+0000 7f0689f75000 1 --2- 172.21.15.92:0/2632102268 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x55eca10c2b70 0x55eca116fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.306+0000 7f0689f75000 1 -- 172.21.15.92:0/2632102268 --> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] -- osd_op(unknown.0.0:1 16.5 16:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55eca114f210 con 0x55eca10c2b70 2023-10-06T10:45:26.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.306+0000 7f06839b1700 1 --2- 172.21.15.92:0/2632102268 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x55eca10c2b70 0x55eca116fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.306+0000 7f06829af700 1 --2- 172.21.15.92:0/2632102268 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0670065890 0x7f0670067d40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f067800c6c0 tx=0x7f0678005830 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.306+0000 7f06839b1700 1 --2- 172.21.15.92:0/2632102268 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x55eca10c2b70 0x55eca116fe90 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.307+0000 7f06839b1700 1 -- 172.21.15.92:0/2632102268 <== osd.0 v2:172.21.15.92:6800/90604 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v104'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f067c00f860 con 0x55eca10c2b70 2023-10-06T10:45:26.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.307+0000 7f0689f75000 1 -- 172.21.15.92:0/2632102268 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x55eca10c2b70 msgr2=0x55eca116fe90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.307+0000 7f0689f75000 1 --2- 172.21.15.92:0/2632102268 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x55eca10c2b70 0x55eca116fe90 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.308+0000 7f0689f75000 1 -- 172.21.15.92:0/2632102268 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0670065890 msgr2=0x7f0670067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.308+0000 7f0689f75000 1 --2- 172.21.15.92:0/2632102268 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0670065890 0x7f0670067d40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f067800c6c0 tx=0x7f0678005830 comp rx=0 tx=0).stop 2023-10-06T10:45:26.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.308+0000 7f0689f75000 1 -- 172.21.15.92:0/2632102268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eca107f270 msgr2=0x55eca1144320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.308+0000 7f0689f75000 1 --2- 172.21.15.92:0/2632102268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eca107f270 0x55eca1144320 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f067400bb20 tx=0x7f0674014000 comp rx=0 tx=0).stop 2023-10-06T10:45:26.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.308+0000 7f06839b1700 1 -- 172.21.15.92:0/2632102268 reap_dead start 2023-10-06T10:45:26.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.308+0000 7f0689f75000 1 -- 172.21.15.92:0/2632102268 shutdown_connections 2023-10-06T10:45:26.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.308+0000 7f0689f75000 1 -- 172.21.15.92:0/2632102268 >> 172.21.15.92:0/2632102268 conn(0x55eca107d750 msgr2=0x55eca11529e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.308+0000 7f0689f75000 1 -- 172.21.15.92:0/2632102268 shutdown_connections 2023-10-06T10:45:26.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.308+0000 7f0689f75000 1 -- 172.21.15.92:0/2632102268 wait complete. 2023-10-06T10:45:26.314 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2023-10-06T10:45:26.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.391+0000 7f689df93000 1 Processor -- start 2023-10-06T10:45:26.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.391+0000 7f689df93000 1 -- start start 2023-10-06T10:45:26.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.391+0000 7f689df93000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c188a2fd0 0x561c188a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.391+0000 7f689df93000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c188a5270 0x561c18956a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.391+0000 7f689df93000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c18933ab0 0x561c18956f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.391+0000 7f689df93000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1881d320 con 0x561c18933ab0 2023-10-06T10:45:26.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.391+0000 7f689df93000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c187c1980 con 0x561c188a2fd0 2023-10-06T10:45:26.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.391+0000 7f689df93000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c187c1620 con 0x561c188a5270 2023-10-06T10:45:26.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.391+0000 7f68979cf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c18933ab0 0x561c18956f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.391+0000 7f68979cf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c18933ab0 0x561c18956f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43024/0 (socket says 172.21.15.92:43024) 2023-10-06T10:45:26.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.391+0000 7f68979cf700 1 -- 172.21.15.92:0/2485209640 learned_addr learned my addr 172.21.15.92:0/2485209640 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.391+0000 7f68971ce700 1 --2- 172.21.15.92:0/2485209640 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c188a2fd0 0x561c188a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.391+0000 7f68969cd700 1 --2- 172.21.15.92:0/2485209640 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c188a5270 0x561c18956a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.392+0000 7f68979cf700 1 -- 172.21.15.92:0/2485209640 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c188a5270 msgr2=0x561c18956a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.392+0000 7f68979cf700 1 --2- 172.21.15.92:0/2485209640 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c188a5270 0x561c18956a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.392+0000 7f68979cf700 1 -- 172.21.15.92:0/2485209640 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c188a2fd0 msgr2=0x561c188a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.392+0000 7f68979cf700 1 --2- 172.21.15.92:0/2485209640 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c188a2fd0 0x561c188a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.392+0000 7f68979cf700 1 -- 172.21.15.92:0/2485209640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6888005cf0 con 0x561c18933ab0 2023-10-06T10:45:26.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.392+0000 7f68969cd700 1 --2- 172.21.15.92:0/2485209640 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c188a5270 0x561c18956a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:26.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.392+0000 7f68979cf700 1 --2- 172.21.15.92:0/2485209640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c18933ab0 0x561c18956f80 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f6890006f30 tx=0x7f68900069a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9934fa1a6580870 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.392+0000 7f68959cb700 1 -- 172.21.15.92:0/2485209640 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890011440 con 0x561c18933ab0 2023-10-06T10:45:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.392+0000 7f68959cb700 1 -- 172.21.15.92:0/2485209640 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68900115a0 con 0x561c18933ab0 2023-10-06T10:45:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.392+0000 7f68959cb700 1 -- 172.21.15.92:0/2485209640 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890011960 con 0x561c18933ab0 2023-10-06T10:45:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.392+0000 7f689df93000 1 -- 172.21.15.92:0/2485209640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c18933ab0 msgr2=0x561c18956f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.392+0000 7f689df93000 1 --2- 172.21.15.92:0/2485209640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c18933ab0 0x561c18956f80 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f6890006f30 tx=0x7f68900069a0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.392+0000 7f689df93000 1 -- 172.21.15.92:0/2485209640 shutdown_connections 2023-10-06T10:45:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.392+0000 7f689df93000 1 --2- 172.21.15.92:0/2485209640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c18933ab0 0x561c18956f80 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.392+0000 7f689df93000 1 --2- 172.21.15.92:0/2485209640 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c188a2fd0 0x561c188a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.392+0000 7f689df93000 1 --2- 172.21.15.92:0/2485209640 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c188a5270 0x561c18956a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.392+0000 7f689df93000 1 -- 172.21.15.92:0/2485209640 >> 172.21.15.92:0/2485209640 conn(0x561c18877520 msgr2=0x561c1894a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.392+0000 7f689df93000 1 -- 172.21.15.92:0/2485209640 shutdown_connections 2023-10-06T10:45:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.392+0000 7f689df93000 1 -- 172.21.15.92:0/2485209640 wait complete. 2023-10-06T10:45:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.393+0000 7f689df93000 1 Processor -- start 2023-10-06T10:45:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.393+0000 7f689df93000 1 -- start start 2023-10-06T10:45:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.394+0000 7f689df93000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c188a2fd0 0x561c18930e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.394+0000 7f689df93000 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c188a5270 0x561c188bb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.394+0000 7f689df93000 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c18933ab0 0x561c1894a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.394+0000 7f689df93000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c187b8400 con 0x561c188a2fd0 2023-10-06T10:45:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.394+0000 7f689df93000 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c18931350 con 0x561c188a5270 2023-10-06T10:45:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.394+0000 7f689df93000 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c188b9350 con 0x561c18933ab0 2023-10-06T10:45:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.394+0000 7f68971ce700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c188a2fd0 0x561c18930e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.394+0000 7f68971ce700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c188a2fd0 0x561c18930e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43040/0 (socket says 172.21.15.92:43040) 2023-10-06T10:45:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.394+0000 7f68971ce700 1 -- 172.21.15.92:0/474101262 learned_addr learned my addr 172.21.15.92:0/474101262 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.394+0000 7f68969cd700 1 --2- 172.21.15.92:0/474101262 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c188a5270 0x561c188bb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.394+0000 7f68979cf700 1 --2- 172.21.15.92:0/474101262 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c18933ab0 0x561c1894a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.394+0000 7f68971ce700 1 -- 172.21.15.92:0/474101262 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c18933ab0 msgr2=0x561c1894a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.394+0000 7f68971ce700 1 --2- 172.21.15.92:0/474101262 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c18933ab0 0x561c1894a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.394+0000 7f68971ce700 1 -- 172.21.15.92:0/474101262 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c188a5270 msgr2=0x561c188bb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.394+0000 7f68971ce700 1 --2- 172.21.15.92:0/474101262 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c188a5270 0x561c188bb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.394+0000 7f68971ce700 1 -- 172.21.15.92:0/474101262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c187c3750 con 0x561c188a2fd0 2023-10-06T10:45:26.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.394+0000 7f68971ce700 1 --2- 172.21.15.92:0/474101262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c188a2fd0 0x561c18930e10 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x561c188a3dd0 tx=0x7f6888013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.394+0000 7f687ffff700 1 -- 172.21.15.92:0/474101262 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688801a050 con 0x561c188a2fd0 2023-10-06T10:45:26.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.394+0000 7f687ffff700 1 -- 172.21.15.92:0/474101262 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688801b040 con 0x561c188a2fd0 2023-10-06T10:45:26.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.394+0000 7f687ffff700 1 -- 172.21.15.92:0/474101262 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68880076e0 con 0x561c188a2fd0 2023-10-06T10:45:26.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.394+0000 7f689df93000 1 -- 172.21.15.92:0/474101262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561c187c3ae0 con 0x561c188a2fd0 2023-10-06T10:45:26.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.394+0000 7f689df93000 1 -- 172.21.15.92:0/474101262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c187c3540 con 0x561c188a2fd0 2023-10-06T10:45:26.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.396+0000 7f687ffff700 1 -- 172.21.15.92:0/474101262 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6888007af0 con 0x561c188a2fd0 2023-10-06T10:45:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.396+0000 7f687ffff700 1 --2- 172.21.15.92:0/474101262 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f68840658c0 0x7f6884067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.396+0000 7f687ffff700 1 -- 172.21.15.92:0/474101262 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f6888084200 con 0x561c188a2fd0 2023-10-06T10:45:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.396+0000 7f68969cd700 1 --2- 172.21.15.92:0/474101262 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f68840658c0 0x7f6884067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.396+0000 7f689df93000 1 --2- 172.21.15.92:0/474101262 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x561c188b8e60 0x561c18926b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.396+0000 7f689df93000 1 -- 172.21.15.92:0/474101262 --> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] -- osd_op(unknown.0.0:1 16.1d 16:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x561c189270a0 con 0x561c188b8e60 2023-10-06T10:45:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.396+0000 7f68979cf700 1 --2- 172.21.15.92:0/474101262 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x561c188b8e60 0x561c18926b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.396+0000 7f68969cd700 1 --2- 172.21.15.92:0/474101262 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f68840658c0 0x7f6884067d70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f688c0015a0 tx=0x7f688c000bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.396+0000 7f68979cf700 1 --2- 172.21.15.92:0/474101262 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x561c188b8e60 0x561c18926b60 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.398+0000 7f68979cf700 1 -- 172.21.15.92:0/474101262 <== osd.2 v2:172.21.15.92:6824/90615 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v104'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f6890000f40 con 0x561c188b8e60 2023-10-06T10:45:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.398+0000 7f689df93000 1 -- 172.21.15.92:0/474101262 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x561c188b8e60 msgr2=0x561c18926b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.398+0000 7f689df93000 1 --2- 172.21.15.92:0/474101262 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x561c188b8e60 0x561c18926b60 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.398+0000 7f689df93000 1 -- 172.21.15.92:0/474101262 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f68840658c0 msgr2=0x7f6884067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.398+0000 7f689df93000 1 --2- 172.21.15.92:0/474101262 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f68840658c0 0x7f6884067d70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f688c0015a0 tx=0x7f688c000bc0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.398+0000 7f689df93000 1 -- 172.21.15.92:0/474101262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c188a2fd0 msgr2=0x561c18930e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.398+0000 7f689df93000 1 --2- 172.21.15.92:0/474101262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c188a2fd0 0x561c18930e10 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x561c188a3dd0 tx=0x7f6888013000 comp rx=0 tx=0).stop 2023-10-06T10:45:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.398+0000 7f68979cf700 1 -- 172.21.15.92:0/474101262 reap_dead start 2023-10-06T10:45:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.398+0000 7f689df93000 1 -- 172.21.15.92:0/474101262 shutdown_connections 2023-10-06T10:45:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.398+0000 7f689df93000 1 -- 172.21.15.92:0/474101262 >> 172.21.15.92:0/474101262 conn(0x561c18877520 msgr2=0x561c188a0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.398+0000 7f689df93000 1 -- 172.21.15.92:0/474101262 shutdown_connections 2023-10-06T10:45:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.398+0000 7f689df93000 1 -- 172.21.15.92:0/474101262 wait complete. 2023-10-06T10:45:26.405 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub start /d1/d2/d3 recursive,force 2023-10-06T10:45:26.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.560+0000 7fe3bb668700 1 Processor -- start 2023-10-06T10:45:26.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.560+0000 7fe3bb668700 1 -- start start 2023-10-06T10:45:26.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.561+0000 7fe3bb668700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe3b40b2360 0x7fe3b40b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.561+0000 7fe3bb668700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe3b40aede0 0x7fe3b40af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.561+0000 7fe3bb668700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b40b2ce0 con 0x7fe3b406ece0 2023-10-06T10:45:26.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.561+0000 7fe3bb668700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b406fb40 con 0x7fe3b40aede0 2023-10-06T10:45:26.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.561+0000 7fe3bb668700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b40b2800 con 0x7fe3b40b2360 2023-10-06T10:45:26.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.561+0000 7fe3b9c05700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe3b40aede0 0x7fe3b40af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.561+0000 7fe3b9c05700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe3b40aede0 0x7fe3b40af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43056/0 (socket says 172.21.15.92:43056) 2023-10-06T10:45:26.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.561+0000 7fe3b9c05700 1 -- 172.21.15.92:0/3998181852 learned_addr learned my addr 172.21.15.92:0/3998181852 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.561+0000 7fe3b8c03700 1 --2- 172.21.15.92:0/3998181852 >> v2:172.21.15.124:3300/0 conn(0x7fe3b40b2360 0x7fe3b40b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.561+0000 7fe3b9c05700 1 -- 172.21.15.92:0/3998181852 >> v2:172.21.15.124:3300/0 conn(0x7fe3b40b2360 msgr2=0x7fe3b40b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.561+0000 7fe3b9c05700 1 --2- 172.21.15.92:0/3998181852 >> v2:172.21.15.124:3300/0 conn(0x7fe3b40b2360 0x7fe3b40b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.561+0000 7fe3b9c05700 1 -- 172.21.15.92:0/3998181852 >> v1:172.21.15.92:6789/0 conn(0x7fe3b406ece0 legacy=0x7fe3b406f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.561+0000 7fe3b9c05700 1 -- 172.21.15.92:0/3998181852 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b40b0410 con 0x7fe3b40aede0 2023-10-06T10:45:26.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.561+0000 7fe3b8c03700 1 --2- 172.21.15.92:0/3998181852 >> v2:172.21.15.124:3300/0 conn(0x7fe3b40b2360 0x7fe3b40b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:26.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.561+0000 7fe3b9c05700 1 --2- 172.21.15.92:0/3998181852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b40aede0 0x7fe3b40af240 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40b0380 tx=0x7fe3ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793b4e4ae436c747 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.561+0000 7fe3abfff700 1 -- 172.21.15.92:0/3998181852 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac00e070 con 0x7fe3b40aede0 2023-10-06T10:45:26.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.562+0000 7fe3abfff700 1 -- 172.21.15.92:0/3998181852 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ac00ba50 con 0x7fe3b40aede0 2023-10-06T10:45:26.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.562+0000 7fe3abfff700 1 -- 172.21.15.92:0/3998181852 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac0092e0 con 0x7fe3b40aede0 2023-10-06T10:45:26.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.562+0000 7fe3bb668700 1 -- 172.21.15.92:0/3998181852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b40aede0 msgr2=0x7fe3b40af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.562+0000 7fe3bb668700 1 --2- 172.21.15.92:0/3998181852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b40aede0 0x7fe3b40af240 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40b0380 tx=0x7fe3ac009840 comp rx=0 tx=0).stop 2023-10-06T10:45:26.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.562+0000 7fe3bb668700 1 -- 172.21.15.92:0/3998181852 shutdown_connections 2023-10-06T10:45:26.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.562+0000 7fe3bb668700 1 --2- 172.21.15.92:0/3998181852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b40aede0 0x7fe3b40af240 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.562+0000 7fe3bb668700 1 --2- 172.21.15.92:0/3998181852 >> v2:172.21.15.124:3300/0 conn(0x7fe3b40b2360 0x7fe3b40b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.562+0000 7fe3bb668700 1 -- 172.21.15.92:0/3998181852 >> 172.21.15.92:0/3998181852 conn(0x7fe3b40682f0 msgr2=0x7fe3b40686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.562+0000 7fe3bb668700 1 -- 172.21.15.92:0/3998181852 shutdown_connections 2023-10-06T10:45:26.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.562+0000 7fe3bb668700 1 -- 172.21.15.92:0/3998181852 wait complete. 2023-10-06T10:45:26.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.563+0000 7fe3bb668700 1 Processor -- start 2023-10-06T10:45:26.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.563+0000 7fe3bb668700 1 -- start start 2023-10-06T10:45:26.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.564+0000 7fe3bb668700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b406ece0 0x7fe3b40c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.564+0000 7fe3bb668700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3b40aede0 0x7fe3b40cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.564+0000 7fe3bb668700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3b40b2360 0x7fe3b40d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.564+0000 7fe3bb668700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b4142e20 con 0x7fe3b406ece0 2023-10-06T10:45:26.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.564+0000 7fe3bb668700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b40ccf40 con 0x7fe3b40b2360 2023-10-06T10:45:26.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.564+0000 7fe3bb668700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b40c6e10 con 0x7fe3b40aede0 2023-10-06T10:45:26.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.564+0000 7fe3b9404700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b406ece0 0x7fe3b40c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.564+0000 7fe3b9404700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b406ece0 0x7fe3b40c68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43064/0 (socket says 172.21.15.92:43064) 2023-10-06T10:45:26.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.564+0000 7fe3b9404700 1 -- 172.21.15.92:0/1972982206 learned_addr learned my addr 172.21.15.92:0/1972982206 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.565+0000 7fe3b9c05700 1 --2- 172.21.15.92:0/1972982206 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3b40b2360 0x7fe3b40d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.565+0000 7fe3b8c03700 1 --2- 172.21.15.92:0/1972982206 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3b40aede0 0x7fe3b40cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.565+0000 7fe3b9404700 1 -- 172.21.15.92:0/1972982206 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3b40aede0 msgr2=0x7fe3b40cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.565+0000 7fe3b9404700 1 --2- 172.21.15.92:0/1972982206 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3b40aede0 0x7fe3b40cc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.565+0000 7fe3b9404700 1 -- 172.21.15.92:0/1972982206 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3b40b2360 msgr2=0x7fe3b40d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.565+0000 7fe3b9404700 1 --2- 172.21.15.92:0/1972982206 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3b40b2360 0x7fe3b40d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.565+0000 7fe3b9404700 1 -- 172.21.15.92:0/1972982206 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ac009ae0 con 0x7fe3b406ece0 2023-10-06T10:45:26.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.565+0000 7fe3b8c03700 1 --2- 172.21.15.92:0/1972982206 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3b40aede0 0x7fe3b40cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:26.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.565+0000 7fe3b9404700 1 --2- 172.21.15.92:0/1972982206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b406ece0 0x7fe3b40c68d0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000c5c0 tx=0x7fe3b0007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.565+0000 7fe3a97fa700 1 -- 172.21.15.92:0/1972982206 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0011590 con 0x7fe3b406ece0 2023-10-06T10:45:26.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.565+0000 7fe3a97fa700 1 -- 172.21.15.92:0/1972982206 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b0011730 con 0x7fe3b406ece0 2023-10-06T10:45:26.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.565+0000 7fe3bb668700 1 -- 172.21.15.92:0/1972982206 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b4150400 con 0x7fe3b406ece0 2023-10-06T10:45:26.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.565+0000 7fe3a97fa700 1 -- 172.21.15.92:0/1972982206 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0011af0 con 0x7fe3b406ece0 2023-10-06T10:45:26.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.565+0000 7fe3bb668700 1 -- 172.21.15.92:0/1972982206 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b4150a40 con 0x7fe3b406ece0 2023-10-06T10:45:26.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.567+0000 7fe3a97fa700 1 -- 172.21.15.92:0/1972982206 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe3b0012420 con 0x7fe3b406ece0 2023-10-06T10:45:26.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.567+0000 7fe3a97fa700 1 --2- 172.21.15.92:0/1972982206 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe398066f30 0x7fe3980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.567+0000 7fe3a97fa700 1 -- 172.21.15.92:0/1972982206 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fe3b0083b20 con 0x7fe3b406ece0 2023-10-06T10:45:26.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.567+0000 7fe3b8c03700 1 --2- 172.21.15.92:0/1972982206 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe398066f30 0x7fe3980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.567+0000 7fe3b8c03700 1 --2- 172.21.15.92:0/1972982206 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe398066f30 0x7fe3980693e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4008b20 tx=0x7fe3a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.569+0000 7fe3c4960b80 1 Processor -- start 2023-10-06T10:45:26.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.569+0000 7fe3c4960b80 1 -- start start 2023-10-06T10:45:26.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.569+0000 7fe3c4960b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562da9f8ab70 0x562da9f8af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.569+0000 7fe3c4960b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f46a80 0x562da9f43660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.569+0000 7fe3c4960b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562da9f44cb0 0x562da9f43ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.569+0000 7fe3c4960b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da9ec3d60 con 0x562da9f46a80 2023-10-06T10:45:26.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.569+0000 7fe3c4960b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da9e9ab50 con 0x562da9f8ab70 2023-10-06T10:45:26.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.569+0000 7fe3c4960b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562da9e98350 con 0x562da9f44cb0 2023-10-06T10:45:26.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.569+0000 7fe3b8c03700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f46a80 0x562da9f43660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.569+0000 7fe3b8c03700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f46a80 0x562da9f43660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43078/0 (socket says 172.21.15.92:43078) 2023-10-06T10:45:26.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.569+0000 7fe3b8c03700 1 -- 172.21.15.92:0/3903371592 learned_addr learned my addr 172.21.15.92:0/3903371592 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.570+0000 7fe3b9404700 1 --2- 172.21.15.92:0/3903371592 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562da9f8ab70 0x562da9f8af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.570+0000 7fe3b9c05700 1 --2- 172.21.15.92:0/3903371592 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562da9f44cb0 0x562da9f43ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.570+0000 7fe3b8c03700 1 -- 172.21.15.92:0/3903371592 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562da9f44cb0 msgr2=0x562da9f43ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.570+0000 7fe3b8c03700 1 --2- 172.21.15.92:0/3903371592 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562da9f44cb0 0x562da9f43ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.570+0000 7fe3b8c03700 1 -- 172.21.15.92:0/3903371592 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562da9f8ab70 msgr2=0x562da9f8af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.570+0000 7fe3b8c03700 1 --2- 172.21.15.92:0/3903371592 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562da9f8ab70 0x562da9f8af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.570+0000 7fe3b8c03700 1 -- 172.21.15.92:0/3903371592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562da9d2e350 con 0x562da9f46a80 2023-10-06T10:45:26.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.570+0000 7fe3b9c05700 1 --2- 172.21.15.92:0/3903371592 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562da9f44cb0 0x562da9f43ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:26.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.570+0000 7fe3b8c03700 1 --2- 172.21.15.92:0/3903371592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f46a80 0x562da9f43660 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400dbf0 tx=0x7fe3a4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d13972c8e07307e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.570+0000 7fe39e7fc700 1 -- 172.21.15.92:0/3903371592 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a400b050 con 0x562da9f46a80 2023-10-06T10:45:26.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.570+0000 7fe39e7fc700 1 -- 172.21.15.92:0/3903371592 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a4017070 con 0x562da9f46a80 2023-10-06T10:45:26.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.570+0000 7fe39e7fc700 1 -- 172.21.15.92:0/3903371592 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4012cf0 con 0x562da9f46a80 2023-10-06T10:45:26.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.571+0000 7fe3c4960b80 1 -- 172.21.15.92:0/3903371592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f46a80 msgr2=0x562da9f43660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.571+0000 7fe3c4960b80 1 --2- 172.21.15.92:0/3903371592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f46a80 0x562da9f43660 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400dbf0 tx=0x7fe3a4014000 comp rx=0 tx=0).stop 2023-10-06T10:45:26.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.571+0000 7fe3c4960b80 1 -- 172.21.15.92:0/3903371592 shutdown_connections 2023-10-06T10:45:26.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.571+0000 7fe3c4960b80 1 --2- 172.21.15.92:0/3903371592 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562da9f44cb0 0x562da9f43ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.571+0000 7fe3c4960b80 1 --2- 172.21.15.92:0/3903371592 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562da9f8ab70 0x562da9f8af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.571+0000 7fe3c4960b80 1 --2- 172.21.15.92:0/3903371592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f46a80 0x562da9f43660 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.571+0000 7fe3c4960b80 1 -- 172.21.15.92:0/3903371592 >> 172.21.15.92:0/3903371592 conn(0x562da9f45150 msgr2=0x562da9f45520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.571+0000 7fe3c4960b80 1 -- 172.21.15.92:0/3903371592 shutdown_connections 2023-10-06T10:45:26.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.571+0000 7fe3c4960b80 1 -- 172.21.15.92:0/3903371592 wait complete. 2023-10-06T10:45:26.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.571+0000 7fe3c4960b80 1 -- start start 2023-10-06T10:45:26.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.572+0000 7fe3c4960b80 1 Processor -- start 2023-10-06T10:45:26.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.572+0000 7fe3c4960b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f44cb0 0x562da9f73ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.572+0000 7fe3c4960b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562da9f46a80 0x562da9f6bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.572+0000 7fe3c4960b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562da9f8ab70 0x562da9f85440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.572+0000 7fe3c4960b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da9e94140 con 0x562da9f44cb0 2023-10-06T10:45:26.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.572+0000 7fe3c4960b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da9e3f970 con 0x562da9f8ab70 2023-10-06T10:45:26.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.572+0000 7fe3c4960b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562da9e5af80 con 0x562da9f46a80 2023-10-06T10:45:26.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.572+0000 7fe3c4960b80 10 client.0 _ll_register_callbacks cb 0x562da9d31010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:26.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.572+0000 7fe3b9404700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f44cb0 0x562da9f73ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.572+0000 7fe3b9404700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f44cb0 0x562da9f73ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43080/0 (socket says 172.21.15.92:43080) 2023-10-06T10:45:26.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.572+0000 7fe3b9404700 1 -- 172.21.15.92:0/3001491716 learned_addr learned my addr 172.21.15.92:0/3001491716 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.572+0000 7fe3b8c03700 1 --2- 172.21.15.92:0/3001491716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562da9f46a80 0x562da9f6bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.572+0000 7fe3b9c05700 1 --2- 172.21.15.92:0/3001491716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562da9f8ab70 0x562da9f85440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3b9404700 1 -- 172.21.15.92:0/3001491716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562da9f46a80 msgr2=0x562da9f6bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3b9404700 1 --2- 172.21.15.92:0/3001491716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562da9f46a80 0x562da9f6bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3b9404700 1 -- 172.21.15.92:0/3001491716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562da9f8ab70 msgr2=0x562da9f85440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3b9404700 1 --2- 172.21.15.92:0/3001491716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562da9f8ab70 0x562da9f85440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3b9404700 1 -- 172.21.15.92:0/3001491716 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a400d7f0 con 0x562da9f44cb0 2023-10-06T10:45:26.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3b8c03700 1 --2- 172.21.15.92:0/3001491716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562da9f46a80 0x562da9f6bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:26.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3b9404700 1 --2- 172.21.15.92:0/3001491716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f44cb0 0x562da9f73ab0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000c590 tx=0x7fe3b0081cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d034aacda2a3edf5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3937fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:26.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3937fe700 1 -- 172.21.15.92:0/3001491716 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b00529e0 con 0x562da9f44cb0 2023-10-06T10:45:26.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3937fe700 1 -- 172.21.15.92:0/3001491716 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fe3b0053d10 con 0x562da9f44cb0 2023-10-06T10:45:26.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3c4960b80 1 -- 172.21.15.92:0/3001491716 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562da9f38770 con 0x562da9f44cb0 2023-10-06T10:45:26.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3937fe700 1 -- 172.21.15.92:0/3001491716 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3b005b280 con 0x562da9f44cb0 2023-10-06T10:45:26.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3c4960b80 10 client.8922 fetch_fsmap learned FSMap version 108 2023-10-06T10:45:26.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3c4960b80 1 -- 172.21.15.92:0/3001491716 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x562da9cd10c0 con 0x562da9f44cb0 2023-10-06T10:45:26.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3937fe700 1 -- 172.21.15.92:0/3001491716 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562da9f38770 con 0x562da9f44cb0 2023-10-06T10:45:26.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3937fe700 1 -- 172.21.15.92:0/3001491716 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 108) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fe3b0050610 con 0x562da9f44cb0 2023-10-06T10:45:26.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3c4960b80 10 client.8922 fetch_fsmap finished waiting for FSMap version 108 2023-10-06T10:45:26.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3c4960b80 10 client.8922 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-10-06T10:45:26.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3c4960b80 20 client.8922 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:26.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3c4960b80 1 --2- 172.21.15.92:0/3001491716 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x562da9f6d750 0x562da9f6db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3c4960b80 4 client.8922 mds_command: new command op to 8520 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:26.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3c4960b80 1 -- 172.21.15.92:0/3001491716 --> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562da9f156a0 con 0x562da9f6d750 2023-10-06T10:45:26.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.573+0000 7fe3b8c03700 1 --2- 172.21.15.92:0/3001491716 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x562da9f6d750 0x562da9f6db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.574+0000 7fe3937fe700 10 client.8922 ms_handle_connect on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:26.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.574+0000 7fe3b8c03700 1 --2- 172.21.15.92:0/3001491716 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x562da9f6d750 0x562da9f6db50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b21250fad8bf20c server_cookie=234ee44b13a10632 in_seq=0 out_seq=0 2023-10-06T10:45:26.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.574+0000 7fe3937fe700 1 -- 172.21.15.92:0/3001491716 <== mds.0 v2:172.21.15.92:6835/2298348426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x562da9f156a0 con 0x562da9f6d750 2023-10-06T10:45:26.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.574+0000 7fe3937fe700 10 client.8922 handle_command_reply: tid=0 2023-10-06T10:45:26.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.574+0000 7fe3c4960b80 1 client.8922 shutdown 2023-10-06T10:45:26.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.574+0000 7fe3c4960b80 1 -- 172.21.15.92:0/3001491716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f44cb0 msgr2=0x562da9f73ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.574+0000 7fe3c4960b80 1 --2- 172.21.15.92:0/3001491716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f44cb0 0x562da9f73ab0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000c590 tx=0x7fe3b0081cc0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.574+0000 7fe3c4960b80 1 -- 172.21.15.92:0/3001491716 shutdown_connections 2023-10-06T10:45:26.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.574+0000 7fe3c4960b80 1 --2- 172.21.15.92:0/3001491716 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x562da9f6d750 0x562da9f6db50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.574+0000 7fe3c4960b80 1 --2- 172.21.15.92:0/3001491716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562da9f8ab70 0x562da9f85440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.574+0000 7fe3937fe700 0 client.8922 ms_handle_reset on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:26.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.574+0000 7fe3c4960b80 1 --2- 172.21.15.92:0/3001491716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f44cb0 0x562da9f73ab0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.574+0000 7fe3c4960b80 1 --2- 172.21.15.92:0/3001491716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562da9f46a80 0x562da9f6bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.574+0000 7fe3c4960b80 1 -- 172.21.15.92:0/3001491716 >> 172.21.15.92:0/3001491716 conn(0x562da9f45150 msgr2=0x562da9f88570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.575+0000 7fe3c4960b80 1 -- 172.21.15.92:0/3001491716 shutdown_connections 2023-10-06T10:45:26.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.575+0000 7fe3c4960b80 1 -- 172.21.15.92:0/3001491716 wait complete. 2023-10-06T10:45:26.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.575+0000 7fe3c4960b80 20 client.8922 trim_cache size 0 max 16384 2023-10-06T10:45:26.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.591+0000 7fe3c4960b80 1 Processor -- start 2023-10-06T10:45:26.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.591+0000 7fe3c4960b80 1 -- start start 2023-10-06T10:45:26.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.591+0000 7fe3c4960b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562da9f44cb0 0x562da9f3e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.591+0000 7fe3c4960b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f46a80 0x562da9f3ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.591+0000 7fe3c4960b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562da9f8ab70 0x562da9f83ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.591+0000 7fe3c4960b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da9d16f10 con 0x562da9f46a80 2023-10-06T10:45:26.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.591+0000 7fe3c4960b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da9f45f80 con 0x562da9f44cb0 2023-10-06T10:45:26.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.591+0000 7fe3c4960b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562da9f44920 con 0x562da9f8ab70 2023-10-06T10:45:26.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.591+0000 7fe3b8c03700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f46a80 0x562da9f3ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.591+0000 7fe3b8c03700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f46a80 0x562da9f3ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43088/0 (socket says 172.21.15.92:43088) 2023-10-06T10:45:26.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.592+0000 7fe3b8c03700 1 -- 172.21.15.92:0/1813906244 learned_addr learned my addr 172.21.15.92:0/1813906244 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.592+0000 7fe3b8c03700 1 -- 172.21.15.92:0/1813906244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562da9f8ab70 msgr2=0x562da9f83ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.592+0000 7fe3b8c03700 1 --2- 172.21.15.92:0/1813906244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562da9f8ab70 0x562da9f83ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.592+0000 7fe3b8c03700 1 -- 172.21.15.92:0/1813906244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562da9f44cb0 msgr2=0x562da9f3e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.592+0000 7fe3b8c03700 1 --2- 172.21.15.92:0/1813906244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562da9f44cb0 0x562da9f3e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.592+0000 7fe3b8c03700 1 -- 172.21.15.92:0/1813906244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562da9f3c380 con 0x562da9f46a80 2023-10-06T10:45:26.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.592+0000 7fe3b8c03700 1 --2- 172.21.15.92:0/1813906244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f46a80 0x562da9f3ed60 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4020040 tx=0x7fe3a4015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5633791941da6ce4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.592+0000 7fe39effd700 1 -- 172.21.15.92:0/1813906244 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a400b040 con 0x562da9f46a80 2023-10-06T10:45:26.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.592+0000 7fe39effd700 1 -- 172.21.15.92:0/1813906244 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a400c040 con 0x562da9f46a80 2023-10-06T10:45:26.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.592+0000 7fe39effd700 1 -- 172.21.15.92:0/1813906244 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4012ae0 con 0x562da9f46a80 2023-10-06T10:45:26.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.592+0000 7fe3c4960b80 1 -- 172.21.15.92:0/1813906244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f46a80 msgr2=0x562da9f3ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.592+0000 7fe3c4960b80 1 --2- 172.21.15.92:0/1813906244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f46a80 0x562da9f3ed60 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4020040 tx=0x7fe3a4015040 comp rx=0 tx=0).stop 2023-10-06T10:45:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.592+0000 7fe3c4960b80 1 -- 172.21.15.92:0/1813906244 shutdown_connections 2023-10-06T10:45:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.592+0000 7fe3c4960b80 1 --2- 172.21.15.92:0/1813906244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562da9f8ab70 0x562da9f83ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.592+0000 7fe3c4960b80 1 --2- 172.21.15.92:0/1813906244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562da9f44cb0 0x562da9f3e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.592+0000 7fe3c4960b80 1 --2- 172.21.15.92:0/1813906244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f46a80 0x562da9f3ed60 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.592+0000 7fe3c4960b80 1 -- 172.21.15.92:0/1813906244 >> 172.21.15.92:0/1813906244 conn(0x562da9f45150 msgr2=0x562da9f6eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.593+0000 7fe3c4960b80 1 -- 172.21.15.92:0/1813906244 shutdown_connections 2023-10-06T10:45:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.593+0000 7fe3c4960b80 1 -- 172.21.15.92:0/1813906244 wait complete. 2023-10-06T10:45:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.593+0000 7fe3c4960b80 1 -- start start 2023-10-06T10:45:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.593+0000 7fe3c4960b80 1 Processor -- start 2023-10-06T10:45:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.594+0000 7fe3c4960b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562da9f44cb0 0x562da9f83ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.594+0000 7fe3c4960b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562da9f46a80 0x562da9f69110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.594+0000 7fe3c4960b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f8ab70 0x562da9f69650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.594+0000 7fe3c4960b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da9f94410 con 0x562da9f8ab70 2023-10-06T10:45:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.594+0000 7fe3c4960b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da9fa4180 con 0x562da9f44cb0 2023-10-06T10:45:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.594+0000 7fe3c4960b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562da9f80df0 con 0x562da9f46a80 2023-10-06T10:45:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.594+0000 7fe3c4960b80 10 client.0 _ll_register_callbacks cb 0x562da9d31010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.594+0000 7fe3b9c05700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f8ab70 0x562da9f69650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.594+0000 7fe3b9c05700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f8ab70 0x562da9f69650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43104/0 (socket says 172.21.15.92:43104) 2023-10-06T10:45:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.594+0000 7fe3b9c05700 1 -- 172.21.15.92:0/2604665259 learned_addr learned my addr 172.21.15.92:0/2604665259 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.594+0000 7fe3b9c05700 1 -- 172.21.15.92:0/2604665259 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562da9f46a80 msgr2=0x562da9f69110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.594+0000 7fe3b9c05700 1 --2- 172.21.15.92:0/2604665259 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562da9f46a80 0x562da9f69110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.594+0000 7fe3b9c05700 1 -- 172.21.15.92:0/2604665259 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562da9f44cb0 msgr2=0x562da9f83ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.594+0000 7fe3b9c05700 1 --2- 172.21.15.92:0/2604665259 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562da9f44cb0 0x562da9f83ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.594+0000 7fe3b9c05700 1 -- 172.21.15.92:0/2604665259 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562da9d2e350 con 0x562da9f8ab70 2023-10-06T10:45:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.595+0000 7fe3b9c05700 1 --2- 172.21.15.92:0/2604665259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f8ab70 0x562da9f69650 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac012000 tx=0x7fe3ac018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215fa2aa15e2a8ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.595+0000 7fe3937fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.595+0000 7fe3937fe700 1 -- 172.21.15.92:0/2604665259 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac023070 con 0x562da9f8ab70 2023-10-06T10:45:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.595+0000 7fe3937fe700 1 -- 172.21.15.92:0/2604665259 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fe3ac00be60 con 0x562da9f8ab70 2023-10-06T10:45:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.595+0000 7fe3c4960b80 1 -- 172.21.15.92:0/2604665259 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3a4017070 con 0x562da9f8ab70 2023-10-06T10:45:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.595+0000 7fe3937fe700 1 -- 172.21.15.92:0/2604665259 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3ac0032f0 con 0x562da9f8ab70 2023-10-06T10:45:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.595+0000 7fe3c4960b80 10 client.8928 fetch_fsmap learned FSMap version 108 2023-10-06T10:45:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.595+0000 7fe3c4960b80 1 -- 172.21.15.92:0/2604665259 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x562da9e62860 con 0x562da9f8ab70 2023-10-06T10:45:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.595+0000 7fe3937fe700 1 -- 172.21.15.92:0/2604665259 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a4017070 con 0x562da9f8ab70 2023-10-06T10:45:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.595+0000 7fe3937fe700 1 -- 172.21.15.92:0/2604665259 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 108) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fe3ac00b050 con 0x562da9f8ab70 2023-10-06T10:45:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.595+0000 7fe3c4960b80 10 client.8928 fetch_fsmap finished waiting for FSMap version 108 2023-10-06T10:45:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.595+0000 7fe3c4960b80 10 client.8928 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-10-06T10:45:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.595+0000 7fe3c4960b80 20 client.8928 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.595+0000 7fe3c4960b80 1 --2- 172.21.15.92:0/2604665259 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x562da9f88690 0x562da9f6a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.595+0000 7fe3c4960b80 4 client.8928 mds_command: new command op to 8520 tid=0[{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2023-10-06T10:45:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.595+0000 7fe3c4960b80 1 -- 172.21.15.92:0/2604665259 --> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) v1 -- 0x562da9e36ee0 con 0x562da9f88690 2023-10-06T10:45:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.595+0000 7fe3b9404700 1 --2- 172.21.15.92:0/2604665259 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x562da9f88690 0x562da9f6a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.595+0000 7fe3937fe700 10 client.8928 ms_handle_connect on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.595+0000 7fe3b9404700 1 --2- 172.21.15.92:0/2604665259 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x562da9f88690 0x562da9f6a7b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1363ce3a2d61a04e server_cookie=c89de2ac48a4e7fd in_seq=0 out_seq=0 2023-10-06T10:45:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.596+0000 7fe3937fe700 1 -- 172.21.15.92:0/2604665259 <== mds.0 v2:172.21.15.92:6835/2298348426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562da9e36ee0 con 0x562da9f88690 2023-10-06T10:45:26.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.596+0000 7fe3937fe700 10 client.8928 handle_command_reply: tid=0 2023-10-06T10:45:26.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.596+0000 7fe3c4960b80 1 client.8928 shutdown 2023-10-06T10:45:26.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.596+0000 7fe3c4960b80 1 -- 172.21.15.92:0/2604665259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f8ab70 msgr2=0x562da9f69650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.596+0000 7fe3c4960b80 1 --2- 172.21.15.92:0/2604665259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f8ab70 0x562da9f69650 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac012000 tx=0x7fe3ac018000 comp rx=0 tx=0).stop 2023-10-06T10:45:26.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.596+0000 7fe3c4960b80 1 -- 172.21.15.92:0/2604665259 shutdown_connections 2023-10-06T10:45:26.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.596+0000 7fe3c4960b80 1 --2- 172.21.15.92:0/2604665259 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x562da9f88690 0x562da9f6a7b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.596+0000 7fe3c4960b80 1 --2- 172.21.15.92:0/2604665259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562da9f8ab70 0x562da9f69650 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.596+0000 7fe3c4960b80 1 --2- 172.21.15.92:0/2604665259 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562da9f44cb0 0x562da9f83ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.596+0000 7fe3c4960b80 1 --2- 172.21.15.92:0/2604665259 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562da9f46a80 0x562da9f69110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.596+0000 7fe3c4960b80 1 -- 172.21.15.92:0/2604665259 >> 172.21.15.92:0/2604665259 conn(0x562da9f45150 msgr2=0x562da9f6e080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.596+0000 7fe3937fe700 0 client.8928 ms_handle_reset on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:26.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.596+0000 7fe3c4960b80 1 -- 172.21.15.92:0/2604665259 shutdown_connections 2023-10-06T10:45:26.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.597+0000 7fe3c4960b80 1 -- 172.21.15.92:0/2604665259 wait complete. 2023-10-06T10:45:26.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.597+0000 7fe3c4960b80 20 client.8928 trim_cache size 0 max 16384 2023-10-06T10:45:26.605 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-10-06T10:45:26.597+0000 7f0bb5880700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2023-10-06T10:45:07.835939+0000 1=0+1) n(v3 rc2023-10-06T10:45:08.064932+0000 6=0+6) (inest mix dirty) (ifile excl) (iversion lock) caps={7633=pAsLsXsFsx/pAsLsXsFsx@5},l=7633 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x5632d5309700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(16)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:45:07.835939+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:45:08.064932+0000 6=0+6)","memory_value.dirstat":"f(v0 m2023-10-06T10:45:07.835939+0000 1=0+1)","memory_value.rstat":"n(v3 rc2023-10-06T10:45:08.064932+0000 6=0+6)","error_str":""},"return_code":-61} 2023-10-06T10:45:26.605 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-10-06T10:45:26.597+0000 7fa288f3b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2023-10-06T10:45:07.836939+0000 1=0+1) n(v2 rc2023-10-06T10:45:08.064932+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={7633=pAsLsXsFsx/pAsLsXsFsx@4},l=7633 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x56246ef3e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(16)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:45:07.836939+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:45:08.064932+0000 4=0+4)","memory_value.dirstat":"f(v0 m2023-10-06T10:45:07.836939+0000 1=0+1)","memory_value.rstat":"n(v2 rc2023-10-06T10:45:08.064932+0000 4=0+4)","error_str":""},"return_code":-61} 2023-10-06T10:45:26.606 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:26.606 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0, 2023-10-06T10:45:26.606 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_tag": "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8", 2023-10-06T10:45:26.606 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": "asynchronous" 2023-10-06T10:45:26.606 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:26.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.597+0000 7fe3bb668700 1 -- 172.21.15.92:0/1972982206 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe398066f30 msgr2=0x7fe3980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.597+0000 7fe3bb668700 1 --2- 172.21.15.92:0/1972982206 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe398066f30 0x7fe3980693e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4008b20 tx=0x7fe3a4008040 comp rx=0 tx=0).stop 2023-10-06T10:45:26.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.597+0000 7fe3bb668700 1 -- 172.21.15.92:0/1972982206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b406ece0 msgr2=0x7fe3b40c68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.597+0000 7fe3bb668700 1 --2- 172.21.15.92:0/1972982206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b406ece0 0x7fe3b40c68d0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000c5c0 tx=0x7fe3b0007f30 comp rx=0 tx=0).stop 2023-10-06T10:45:26.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.597+0000 7fe3bb668700 1 -- 172.21.15.92:0/1972982206 shutdown_connections 2023-10-06T10:45:26.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.597+0000 7fe3bb668700 1 --2- 172.21.15.92:0/1972982206 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe398066f30 0x7fe3980693e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.597+0000 7fe3bb668700 1 --2- 172.21.15.92:0/1972982206 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3b40b2360 0x7fe3b40d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.597+0000 7fe3bb668700 1 --2- 172.21.15.92:0/1972982206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b406ece0 0x7fe3b40c68d0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.597+0000 7fe3bb668700 1 --2- 172.21.15.92:0/1972982206 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3b40aede0 0x7fe3b40cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.597+0000 7fe3bb668700 1 -- 172.21.15.92:0/1972982206 >> 172.21.15.92:0/1972982206 conn(0x7fe3b40682f0 msgr2=0x7fe3b40b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.597+0000 7fe3bb668700 1 -- 172.21.15.92:0/1972982206 shutdown_connections 2023-10-06T10:45:26.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.597+0000 7fe3bb668700 1 -- 172.21.15.92:0/1972982206 wait complete. 2023-10-06T10:45:26.608 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-10-06T10:45:26.598+0000 7f0bbb88c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2023-10-06T10:45:07.836939+0000 1=0+1) n(v4 rc2023-10-06T10:45:08.064932+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={7633=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x5632d5386000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(16)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:45:07.836939+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:45:08.064932+0000 5=0+5)","memory_value.dirstat":"f(v0 m2023-10-06T10:45:07.836939+0000 1=0+1)","memory_value.rstat":"n(v4 rc2023-10-06T10:45:08.064932+0000 5=0+5)","error_str":""},"return_code":-61} 2023-10-06T10:45:26.608 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-10-06T10:45:26.598+0000 7fa28ef47700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2023-10-06T10:45:07.837939+0000 1=0+1) n(v1 rc2023-10-06T10:45:08.064932+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={7633=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x56246ef3e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(16)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:45:07.837939+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:45:07.837939+0000 3=0+3)","memory_value.dirstat":"f(v0 m2023-10-06T10:45:07.837939+0000 1=0+1)","memory_value.rstat":"n(v1 rc2023-10-06T10:45:08.064932+0000 3=0+3)","error_str":""},"return_code":-61} 2023-10-06T10:45:26.608 INFO:tasks.ceph.mds.d.smithi124.stderr:2023-10-06T10:45:26.598+0000 7f8a4bed5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2023-10-06T10:45:07.837939+0000 1=0+1) n(v0 rc2023-10-06T10:45:07.837939+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={7633=pAsLsXsFsx/pAsLsXsFsx@4},l=7633 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55ecb6337080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(16)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:45:07.837939+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:45:07.837939+0000 2=0+2)","memory_value.dirstat":"f(v0 m2023-10-06T10:45:07.837939+0000 1=0+1)","memory_value.rstat":"n(v0 rc2023-10-06T10:45:07.837939+0000 2=0+2)","error_str":""},"return_code":-61} 2023-10-06T10:45:26.609 INFO:tasks.ceph.mds.d.smithi124.stderr:2023-10-06T10:45:26.599+0000 7f8a4bed5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={7633=pAsLsXsFsx/AsLsXsFsx@2},l=7633 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55ecb6337600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(16)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2023-10-06T10:45:26.611 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub pause 2023-10-06T10:45:26.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.726+0000 7f8edb1ab700 1 Processor -- start 2023-10-06T10:45:26.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.726+0000 7f8edb1ab700 1 -- start start 2023-10-06T10:45:26.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.727+0000 7f8edb1ab700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8ed40cd3c0 0x7f8ed40ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.727+0000 7f8edb1ab700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ed40b29f0 0x7f8ed40cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.727+0000 7f8edb1ab700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed40cc970 con 0x7f8ed40b2620 2023-10-06T10:45:26.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.727+0000 7f8edb1ab700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed40cf520 con 0x7f8ed40cd3c0 2023-10-06T10:45:26.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.727+0000 7f8edb1ab700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed40cbd60 con 0x7f8ed40b29f0 2023-10-06T10:45:26.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.727+0000 7f8ed99a8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8ed40cd3c0 0x7f8ed40ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.727+0000 7f8eda1a9700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8ed40b2620 0x7f8ed40cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34554/0 (socket says 172.21.15.92:34554) 2023-10-06T10:45:26.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.727+0000 7f8eda1a9700 1 -- 172.21.15.92:0/2007113059 learned_addr learned my addr 172.21.15.92:0/2007113059 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:26.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.727+0000 7f8eda9aa700 1 --2- 172.21.15.92:0/2007113059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ed40b29f0 0x7f8ed40cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.727+0000 7f8ed99a8700 1 -- 172.21.15.92:0/2007113059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ed40b29f0 msgr2=0x7f8ed40cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.727+0000 7f8ed99a8700 1 --2- 172.21.15.92:0/2007113059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ed40b29f0 0x7f8ed40cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.727+0000 7f8ed99a8700 1 -- 172.21.15.92:0/2007113059 >> v1:172.21.15.92:6789/0 conn(0x7f8ed40b2620 legacy=0x7f8ed40cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.727+0000 7f8ed91a7700 1 -- 172.21.15.92:0/2007113059 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3591922937 0 0) 0x7f8ed40cc970 con 0x7f8ed40b2620 2023-10-06T10:45:26.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.727+0000 7f8ed99a8700 1 -- 172.21.15.92:0/2007113059 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed40cc040 con 0x7f8ed40cd3c0 2023-10-06T10:45:26.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.728+0000 7f8ed99a8700 1 --2- 172.21.15.92:0/2007113059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ed40cd3c0 0x7f8ed40ceaa0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40098e0 tx=0x7f8ec40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23912da9927f640 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.728+0000 7f8ed91a7700 1 -- 172.21.15.92:0/2007113059 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4009e80 con 0x7f8ed40cd3c0 2023-10-06T10:45:26.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.728+0000 7f8ed91a7700 1 -- 172.21.15.92:0/2007113059 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec400b860 con 0x7f8ed40cd3c0 2023-10-06T10:45:26.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.728+0000 7f8ed91a7700 1 -- 172.21.15.92:0/2007113059 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec400bbe0 con 0x7f8ed40cd3c0 2023-10-06T10:45:26.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.728+0000 7f8edb1ab700 1 -- 172.21.15.92:0/2007113059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ed40cd3c0 msgr2=0x7f8ed40ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.728+0000 7f8edb1ab700 1 --2- 172.21.15.92:0/2007113059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ed40cd3c0 0x7f8ed40ceaa0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40098e0 tx=0x7f8ec40091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.728+0000 7f8edb1ab700 1 -- 172.21.15.92:0/2007113059 shutdown_connections 2023-10-06T10:45:26.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.728+0000 7f8edb1ab700 1 --2- 172.21.15.92:0/2007113059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ed40b29f0 0x7f8ed40cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.728+0000 7f8edb1ab700 1 --2- 172.21.15.92:0/2007113059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ed40cd3c0 0x7f8ed40ceaa0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.728+0000 7f8edb1ab700 1 -- 172.21.15.92:0/2007113059 >> 172.21.15.92:0/2007113059 conn(0x7f8ed4099520 msgr2=0x7f8ed406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.728+0000 7f8edb1ab700 1 -- 172.21.15.92:0/2007113059 shutdown_connections 2023-10-06T10:45:26.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.728+0000 7f8edb1ab700 1 -- 172.21.15.92:0/2007113059 wait complete. 2023-10-06T10:45:26.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.729+0000 7f8edb1ab700 1 Processor -- start 2023-10-06T10:45:26.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.729+0000 7f8edb1ab700 1 -- start start 2023-10-06T10:45:26.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.730+0000 7f8edb1ab700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ed40b2620 0x7f8ed415ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.730+0000 7f8edb1ab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ed40b29f0 0x7f8ed40b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.730+0000 7f8edb1ab700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ed40cd3c0 0x7f8ed4153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.730+0000 7f8edb1ab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed40b4980 con 0x7f8ed40b29f0 2023-10-06T10:45:26.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.730+0000 7f8edb1ab700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed40b3d70 con 0x7f8ed40cd3c0 2023-10-06T10:45:26.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.730+0000 7f8edb1ab700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed40b4050 con 0x7f8ed40b2620 2023-10-06T10:45:26.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.730+0000 7f8ed99a8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ed40b29f0 0x7f8ed40b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.730+0000 7f8ed99a8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ed40b29f0 0x7f8ed40b3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43112/0 (socket says 172.21.15.92:43112) 2023-10-06T10:45:26.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.730+0000 7f8ed99a8700 1 -- 172.21.15.92:0/2397328539 learned_addr learned my addr 172.21.15.92:0/2397328539 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.730+0000 7f8eda1a9700 1 --2- 172.21.15.92:0/2397328539 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ed40b2620 0x7f8ed415ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.730+0000 7f8eda9aa700 1 --2- 172.21.15.92:0/2397328539 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ed40cd3c0 0x7f8ed4153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.730+0000 7f8ed99a8700 1 -- 172.21.15.92:0/2397328539 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ed40b2620 msgr2=0x7f8ed415ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.730+0000 7f8ed99a8700 1 --2- 172.21.15.92:0/2397328539 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ed40b2620 0x7f8ed415ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.731+0000 7f8ed99a8700 1 -- 172.21.15.92:0/2397328539 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ed40cd3c0 msgr2=0x7f8ed4153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.731+0000 7f8ed99a8700 1 --2- 172.21.15.92:0/2397328539 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ed40cd3c0 0x7f8ed4153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.731+0000 7f8ed99a8700 1 -- 172.21.15.92:0/2397328539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec4009420 con 0x7f8ed40b29f0 2023-10-06T10:45:26.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.731+0000 7f8eda9aa700 1 --2- 172.21.15.92:0/2397328539 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ed40cd3c0 0x7f8ed4153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:26.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.731+0000 7f8ed99a8700 1 --2- 172.21.15.92:0/2397328539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ed40b29f0 0x7f8ed40b3540 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4013000 tx=0x7f8ec4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.731+0000 7f8eca7fc700 1 -- 172.21.15.92:0/2397328539 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec400bbe0 con 0x7f8ed40b29f0 2023-10-06T10:45:26.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.731+0000 7f8eca7fc700 1 -- 172.21.15.92:0/2397328539 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec4011420 con 0x7f8ed40b29f0 2023-10-06T10:45:26.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.731+0000 7f8edb1ab700 1 -- 172.21.15.92:0/2397328539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed40b4330 con 0x7f8ed40b29f0 2023-10-06T10:45:26.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.731+0000 7f8eca7fc700 1 -- 172.21.15.92:0/2397328539 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec40117e0 con 0x7f8ed40b29f0 2023-10-06T10:45:26.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.731+0000 7f8edb1ab700 1 -- 172.21.15.92:0/2397328539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed40b3a80 con 0x7f8ed40b29f0 2023-10-06T10:45:26.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.732+0000 7f8eca7fc700 1 -- 172.21.15.92:0/2397328539 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8ec4011bd0 con 0x7f8ed40b29f0 2023-10-06T10:45:26.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.733+0000 7f8eca7fc700 1 --2- 172.21.15.92:0/2397328539 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8eb8066f30 0x7f8eb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.733+0000 7f8eca7fc700 1 -- 172.21.15.92:0/2397328539 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f8ec4085d00 con 0x7f8ed40b29f0 2023-10-06T10:45:26.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.733+0000 7f8eda1a9700 1 --2- 172.21.15.92:0/2397328539 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8eb8066f30 0x7f8eb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.733+0000 7f8eda1a9700 1 --2- 172.21.15.92:0/2397328539 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8eb8066f30 0x7f8eb80693e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4069d20 tx=0x7f8ed0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.735+0000 7f8ee5705b80 1 Processor -- start 2023-10-06T10:45:26.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.735+0000 7f8ee5705b80 1 -- start start 2023-10-06T10:45:26.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.735+0000 7f8ee5705b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556622004990 0x556622004d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.735+0000 7f8ee5705b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55662204ec00 0x55662202d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.735+0000 7f8ee5705b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662202dea0 0x5566220009d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.735+0000 7f8ee5705b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556621f84cc0 con 0x55662202dea0 2023-10-06T10:45:26.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.735+0000 7f8ee5705b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x556621f5bab0 con 0x556622004990 2023-10-06T10:45:26.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.735+0000 7f8ee5705b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x556621f592b0 con 0x55662204ec00 2023-10-06T10:45:26.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.736+0000 7f8eda9aa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662202dea0 0x5566220009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.736+0000 7f8eda9aa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662202dea0 0x5566220009d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43120/0 (socket says 172.21.15.92:43120) 2023-10-06T10:45:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.736+0000 7f8eda9aa700 1 -- 172.21.15.92:0/1523102374 learned_addr learned my addr 172.21.15.92:0/1523102374 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.736+0000 7f8eda1a9700 1 --2- 172.21.15.92:0/1523102374 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556622004990 0x556622004d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.736+0000 7f8ed99a8700 1 --2- 172.21.15.92:0/1523102374 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55662204ec00 0x55662202d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.736+0000 7f8eda9aa700 1 -- 172.21.15.92:0/1523102374 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55662204ec00 msgr2=0x55662202d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.736+0000 7f8eda9aa700 1 --2- 172.21.15.92:0/1523102374 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55662204ec00 0x55662202d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.736+0000 7f8eda9aa700 1 -- 172.21.15.92:0/1523102374 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556622004990 msgr2=0x556622004d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.736+0000 7f8eda9aa700 1 --2- 172.21.15.92:0/1523102374 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556622004990 0x556622004d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.736+0000 7f8eda9aa700 1 -- 172.21.15.92:0/1523102374 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556621def2b0 con 0x55662202dea0 2023-10-06T10:45:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.736+0000 7f8eda9aa700 1 --2- 172.21.15.92:0/1523102374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662202dea0 0x5566220009d0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc007470 tx=0x7f8ecc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b538ebc68327824 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.736+0000 7f8eb77fe700 1 -- 172.21.15.92:0/1523102374 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc0145e0 con 0x55662202dea0 2023-10-06T10:45:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.736+0000 7f8eb77fe700 1 -- 172.21.15.92:0/1523102374 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ecc00f050 con 0x55662202dea0 2023-10-06T10:45:26.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.736+0000 7f8eb77fe700 1 -- 172.21.15.92:0/1523102374 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc014950 con 0x55662202dea0 2023-10-06T10:45:26.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.737+0000 7f8ee5705b80 1 -- 172.21.15.92:0/1523102374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662202dea0 msgr2=0x5566220009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.737+0000 7f8ee5705b80 1 --2- 172.21.15.92:0/1523102374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662202dea0 0x5566220009d0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc007470 tx=0x7f8ecc00b040 comp rx=0 tx=0).stop 2023-10-06T10:45:26.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.737+0000 7f8ee5705b80 1 -- 172.21.15.92:0/1523102374 shutdown_connections 2023-10-06T10:45:26.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.737+0000 7f8ee5705b80 1 --2- 172.21.15.92:0/1523102374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662202dea0 0x5566220009d0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.737+0000 7f8ee5705b80 1 --2- 172.21.15.92:0/1523102374 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556622004990 0x556622004d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.737+0000 7f8ee5705b80 1 --2- 172.21.15.92:0/1523102374 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55662204ec00 0x55662202d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.737+0000 7f8ee5705b80 1 -- 172.21.15.92:0/1523102374 >> 172.21.15.92:0/1523102374 conn(0x5566220062b0 msgr2=0x556622006680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.737+0000 7f8ee5705b80 1 -- 172.21.15.92:0/1523102374 shutdown_connections 2023-10-06T10:45:26.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.737+0000 7f8ee5705b80 1 -- 172.21.15.92:0/1523102374 wait complete. 2023-10-06T10:45:26.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.737+0000 7f8ee5705b80 1 -- start start 2023-10-06T10:45:26.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.738+0000 7f8ee5705b80 1 Processor -- start 2023-10-06T10:45:26.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.738+0000 7f8ee5705b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556622004990 0x55662204cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.738+0000 7f8ee5705b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662202dea0 0x5566220450b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.738+0000 7f8ee5705b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55662204ec00 0x5566220455f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.738+0000 7f8ee5705b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556621f550a0 con 0x55662202dea0 2023-10-06T10:45:26.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.738+0000 7f8ee5705b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x556621f008d0 con 0x55662204ec00 2023-10-06T10:45:26.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.738+0000 7f8ee5705b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x556621f1bee0 con 0x556622004990 2023-10-06T10:45:26.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.739+0000 7f8ee5705b80 10 client.0 _ll_register_callbacks cb 0x556621df1f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:26.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.739+0000 7f8ed99a8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662202dea0 0x5566220450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.739+0000 7f8ed99a8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662202dea0 0x5566220450b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43136/0 (socket says 172.21.15.92:43136) 2023-10-06T10:45:26.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.739+0000 7f8ed99a8700 1 -- 172.21.15.92:0/3668353405 learned_addr learned my addr 172.21.15.92:0/3668353405 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.739+0000 7f8eda1a9700 1 --2- 172.21.15.92:0/3668353405 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556622004990 0x55662204cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.739+0000 7f8eda9aa700 1 --2- 172.21.15.92:0/3668353405 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55662204ec00 0x5566220455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.739+0000 7f8ed99a8700 1 -- 172.21.15.92:0/3668353405 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556622004990 msgr2=0x55662204cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.739+0000 7f8ed99a8700 1 --2- 172.21.15.92:0/3668353405 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556622004990 0x55662204cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.739+0000 7f8ed99a8700 1 -- 172.21.15.92:0/3668353405 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55662204ec00 msgr2=0x5566220455f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.739+0000 7f8ed99a8700 1 --2- 172.21.15.92:0/3668353405 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55662204ec00 0x5566220455f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.739+0000 7f8ed99a8700 1 -- 172.21.15.92:0/3668353405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ecc009d60 con 0x55662202dea0 2023-10-06T10:45:26.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.739+0000 7f8ed99a8700 1 --2- 172.21.15.92:0/3668353405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662202dea0 0x5566220450b0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4052f80 tx=0x7f8ec4083ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28eccc88011d6497 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.739+0000 7f8eb4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:26.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.739+0000 7f8eb4ff9700 1 -- 172.21.15.92:0/3668353405 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4056a80 con 0x55662202dea0 2023-10-06T10:45:26.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.739+0000 7f8eb4ff9700 1 -- 172.21.15.92:0/3668353405 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f8ec4059c20 con 0x55662202dea0 2023-10-06T10:45:26.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.739+0000 7f8ee5705b80 1 -- 172.21.15.92:0/3668353405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556621ff96d0 con 0x55662202dea0 2023-10-06T10:45:26.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.739+0000 7f8eb4ff9700 1 -- 172.21.15.92:0/3668353405 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ec405d970 con 0x55662202dea0 2023-10-06T10:45:26.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.739+0000 7f8ee5705b80 10 client.8943 fetch_fsmap learned FSMap version 108 2023-10-06T10:45:26.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.739+0000 7f8ee5705b80 1 -- 172.21.15.92:0/3668353405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x556621d92020 con 0x55662202dea0 2023-10-06T10:45:26.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.739+0000 7f8eb4ff9700 1 -- 172.21.15.92:0/3668353405 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556621ff96d0 con 0x55662202dea0 2023-10-06T10:45:26.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.739+0000 7f8eb4ff9700 1 -- 172.21.15.92:0/3668353405 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 108) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f8ec405d4e0 con 0x55662202dea0 2023-10-06T10:45:26.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.740+0000 7f8ee5705b80 10 client.8943 fetch_fsmap finished waiting for FSMap version 108 2023-10-06T10:45:26.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.740+0000 7f8ee5705b80 10 client.8943 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-10-06T10:45:26.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.740+0000 7f8ee5705b80 20 client.8943 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:26.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.740+0000 7f8ee5705b80 1 --2- 172.21.15.92:0/3668353405 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x556622043810 0x55662205e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.740+0000 7f8ee5705b80 4 client.8943 mds_command: new command op to 8520 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:26.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.740+0000 7f8ee5705b80 1 -- 172.21.15.92:0/3668353405 --> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556621fd6600 con 0x556622043810 2023-10-06T10:45:26.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.740+0000 7f8eda1a9700 1 --2- 172.21.15.92:0/3668353405 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x556622043810 0x55662205e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.740+0000 7f8eb4ff9700 10 client.8943 ms_handle_connect on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:26.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.740+0000 7f8eda1a9700 1 --2- 172.21.15.92:0/3668353405 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x556622043810 0x55662205e320 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efbf149194c62e9d server_cookie=26415eb0025a196a in_seq=0 out_seq=0 2023-10-06T10:45:26.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.741+0000 7f8eb4ff9700 1 -- 172.21.15.92:0/3668353405 <== mds.0 v2:172.21.15.92:6835/2298348426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x556621fd6600 con 0x556622043810 2023-10-06T10:45:26.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.741+0000 7f8eb4ff9700 10 client.8943 handle_command_reply: tid=0 2023-10-06T10:45:26.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.741+0000 7f8ee5705b80 1 client.8943 shutdown 2023-10-06T10:45:26.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.741+0000 7f8ee5705b80 1 -- 172.21.15.92:0/3668353405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662202dea0 msgr2=0x5566220450b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.741+0000 7f8ee5705b80 1 --2- 172.21.15.92:0/3668353405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662202dea0 0x5566220450b0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4052f80 tx=0x7f8ec4083ea0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.741+0000 7f8ee5705b80 1 -- 172.21.15.92:0/3668353405 shutdown_connections 2023-10-06T10:45:26.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.741+0000 7f8ee5705b80 1 --2- 172.21.15.92:0/3668353405 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x556622043810 0x55662205e320 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.741+0000 7f8ee5705b80 1 --2- 172.21.15.92:0/3668353405 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55662204ec00 0x5566220455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.741+0000 7f8eb4ff9700 0 client.8943 ms_handle_reset on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:26.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.741+0000 7f8ee5705b80 1 --2- 172.21.15.92:0/3668353405 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556622004990 0x55662204cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.741+0000 7f8ee5705b80 1 --2- 172.21.15.92:0/3668353405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662202dea0 0x5566220450b0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.741+0000 7f8ee5705b80 1 -- 172.21.15.92:0/3668353405 >> 172.21.15.92:0/3668353405 conn(0x5566220062b0 msgr2=0x556621ffb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.741+0000 7f8ee5705b80 1 -- 172.21.15.92:0/3668353405 shutdown_connections 2023-10-06T10:45:26.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.741+0000 7f8ee5705b80 1 -- 172.21.15.92:0/3668353405 wait complete. 2023-10-06T10:45:26.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.741+0000 7f8ee5705b80 20 client.8943 trim_cache size 0 max 16384 2023-10-06T10:45:26.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.757+0000 7f8ee5705b80 1 Processor -- start 2023-10-06T10:45:26.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.757+0000 7f8ee5705b80 1 -- start start 2023-10-06T10:45:26.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.757+0000 7f8ee5705b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556622004990 0x55662202d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.757+0000 7f8ee5705b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55662202dea0 0x5566220450d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.757+0000 7f8ee5705b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556622043810 0x556622041460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.757+0000 7f8ee5705b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556621dd7e70 con 0x556622043810 2023-10-06T10:45:26.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.757+0000 7f8ee5705b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566220443f0 con 0x556622004990 2023-10-06T10:45:26.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.757+0000 7f8ee5705b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55662204de20 con 0x55662202dea0 2023-10-06T10:45:26.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.757+0000 7f8eda9aa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556622043810 0x556622041460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.757+0000 7f8eda9aa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556622043810 0x556622041460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43144/0 (socket says 172.21.15.92:43144) 2023-10-06T10:45:26.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.757+0000 7f8eda9aa700 1 -- 172.21.15.92:0/2622152606 learned_addr learned my addr 172.21.15.92:0/2622152606 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.758+0000 7f8eda1a9700 1 --2- 172.21.15.92:0/2622152606 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556622004990 0x55662202d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.758+0000 7f8eda9aa700 1 -- 172.21.15.92:0/2622152606 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55662202dea0 msgr2=0x5566220450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.758+0000 7f8eda9aa700 1 --2- 172.21.15.92:0/2622152606 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55662202dea0 0x5566220450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.758+0000 7f8eda9aa700 1 -- 172.21.15.92:0/2622152606 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556622004990 msgr2=0x55662202d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.758+0000 7f8eda9aa700 1 --2- 172.21.15.92:0/2622152606 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556622004990 0x55662202d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.758+0000 7f8eda9aa700 1 -- 172.21.15.92:0/2622152606 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556621f237c0 con 0x556622043810 2023-10-06T10:45:26.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.758+0000 7f8eda9aa700 1 --2- 172.21.15.92:0/2622152606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556622043810 0x556622041460 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc007440 tx=0x7f8ecc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e6ba39700009ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.758+0000 7f8eb7fff700 1 -- 172.21.15.92:0/2622152606 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc024070 con 0x556622043810 2023-10-06T10:45:26.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.758+0000 7f8eb7fff700 1 -- 172.21.15.92:0/2622152606 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ecc000fc0 con 0x556622043810 2023-10-06T10:45:26.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.758+0000 7f8eb7fff700 1 -- 172.21.15.92:0/2622152606 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc014d00 con 0x556622043810 2023-10-06T10:45:26.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.758+0000 7f8ee5705b80 1 -- 172.21.15.92:0/2622152606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556622043810 msgr2=0x556622041460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.758+0000 7f8ee5705b80 1 --2- 172.21.15.92:0/2622152606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556622043810 0x556622041460 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc007440 tx=0x7f8ecc01c000 comp rx=0 tx=0).stop 2023-10-06T10:45:26.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.758+0000 7f8ee5705b80 1 -- 172.21.15.92:0/2622152606 shutdown_connections 2023-10-06T10:45:26.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.758+0000 7f8ee5705b80 1 --2- 172.21.15.92:0/2622152606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556622043810 0x556622041460 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.758+0000 7f8ee5705b80 1 --2- 172.21.15.92:0/2622152606 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556622004990 0x55662202d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.758+0000 7f8ee5705b80 1 --2- 172.21.15.92:0/2622152606 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55662202dea0 0x5566220450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.758+0000 7f8ee5705b80 1 -- 172.21.15.92:0/2622152606 >> 172.21.15.92:0/2622152606 conn(0x5566220062b0 msgr2=0x556622007b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.759+0000 7f8ee5705b80 1 -- 172.21.15.92:0/2622152606 shutdown_connections 2023-10-06T10:45:26.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.759+0000 7f8ee5705b80 1 -- 172.21.15.92:0/2622152606 wait complete. 2023-10-06T10:45:26.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.759+0000 7f8ee5705b80 1 -- start start 2023-10-06T10:45:26.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.759+0000 7f8ee5705b80 1 Processor -- start 2023-10-06T10:45:26.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.760+0000 7f8ee5705b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556622004990 0x556622004d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.760+0000 7f8ee5705b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55662202dea0 0x5566220052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.760+0000 7f8ee5705b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556622043810 0x556622041800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.760+0000 7f8ee5705b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566220b0b50 con 0x556622004990 2023-10-06T10:45:26.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.760+0000 7f8ee5705b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566220b45e0 con 0x55662202dea0 2023-10-06T10:45:26.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.760+0000 7f8ee5705b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566220b4820 con 0x556622043810 2023-10-06T10:45:26.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.760+0000 7f8ee5705b80 10 client.0 _ll_register_callbacks cb 0x556621df1f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:26.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.760+0000 7f8eda1a9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556622004990 0x556622004d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.760+0000 7f8eda1a9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556622004990 0x556622004d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43150/0 (socket says 172.21.15.92:43150) 2023-10-06T10:45:26.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.760+0000 7f8eda1a9700 1 -- 172.21.15.92:0/2555431184 learned_addr learned my addr 172.21.15.92:0/2555431184 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.760+0000 7f8eda1a9700 1 -- 172.21.15.92:0/2555431184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556622043810 msgr2=0x556622041800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.760+0000 7f8eda1a9700 1 --2- 172.21.15.92:0/2555431184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556622043810 0x556622041800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.760+0000 7f8eda1a9700 1 -- 172.21.15.92:0/2555431184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55662202dea0 msgr2=0x5566220052a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.760+0000 7f8eda1a9700 1 --2- 172.21.15.92:0/2555431184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55662202dea0 0x5566220052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.760+0000 7f8eda1a9700 1 -- 172.21.15.92:0/2555431184 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556621def2b0 con 0x556622004990 2023-10-06T10:45:26.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.761+0000 7f8eda1a9700 1 --2- 172.21.15.92:0/2555431184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556622004990 0x556622004d60 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000f4b0 tx=0x7f8ed0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d94f30a259c1f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.761+0000 7f8eb4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:26.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.761+0000 7f8eb4ff9700 1 -- 172.21.15.92:0/2555431184 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0028070 con 0x556622004990 2023-10-06T10:45:26.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.761+0000 7f8eb4ff9700 1 -- 172.21.15.92:0/2555431184 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f8ed001c040 con 0x556622004990 2023-10-06T10:45:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.761+0000 7f8ee5705b80 1 -- 172.21.15.92:0/2555431184 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ecc00f050 con 0x556622004990 2023-10-06T10:45:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.761+0000 7f8eb4ff9700 1 -- 172.21.15.92:0/2555431184 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ed0016ae0 con 0x556622004990 2023-10-06T10:45:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.761+0000 7f8ee5705b80 10 client.8949 fetch_fsmap learned FSMap version 108 2023-10-06T10:45:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.761+0000 7f8ee5705b80 1 -- 172.21.15.92:0/2555431184 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x556621f0a4b0 con 0x556622004990 2023-10-06T10:45:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.761+0000 7f8eb4ff9700 1 -- 172.21.15.92:0/2555431184 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ecc00f050 con 0x556622004990 2023-10-06T10:45:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.761+0000 7f8eb4ff9700 1 -- 172.21.15.92:0/2555431184 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 108) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f8ed0016c40 con 0x556622004990 2023-10-06T10:45:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.761+0000 7f8ee5705b80 10 client.8949 fetch_fsmap finished waiting for FSMap version 108 2023-10-06T10:45:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.761+0000 7f8ee5705b80 10 client.8949 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-10-06T10:45:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.761+0000 7f8ee5705b80 20 client.8949 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.761+0000 7f8ee5705b80 1 --2- 172.21.15.92:0/2555431184 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55662204ec00 0x556622041d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.761+0000 7f8ee5705b80 4 client.8949 mds_command: new command op to 8520 tid=0[{"prefix": "scrub pause"}] 2023-10-06T10:45:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.761+0000 7f8ee5705b80 1 -- 172.21.15.92:0/2555431184 --> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x556621ef7e40 con 0x55662204ec00 2023-10-06T10:45:26.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.761+0000 7f8ed99a8700 1 --2- 172.21.15.92:0/2555431184 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55662204ec00 0x556622041d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.762+0000 7f8eb4ff9700 10 client.8949 ms_handle_connect on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:26.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.762+0000 7f8ed99a8700 1 --2- 172.21.15.92:0/2555431184 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55662204ec00 0x556622041d40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e7c7e8c24beeb73 server_cookie=46771c29d1d8af5f in_seq=0 out_seq=0 2023-10-06T10:45:26.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.762+0000 7f8eb4ff9700 1 -- 172.21.15.92:0/2555431184 <== mds.0 v2:172.21.15.92:6835/2298348426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x556621ef7e40 con 0x55662204ec00 2023-10-06T10:45:26.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.762+0000 7f8eb4ff9700 10 client.8949 handle_command_reply: tid=0 2023-10-06T10:45:26.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.762+0000 7f8ee5705b80 1 client.8949 shutdown 2023-10-06T10:45:26.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.762+0000 7f8ee5705b80 1 -- 172.21.15.92:0/2555431184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556622004990 msgr2=0x556622004d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.762+0000 7f8ee5705b80 1 --2- 172.21.15.92:0/2555431184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556622004990 0x556622004d60 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000f4b0 tx=0x7f8ed0014000 comp rx=0 tx=0).stop 2023-10-06T10:45:26.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.762+0000 7f8ee5705b80 1 -- 172.21.15.92:0/2555431184 shutdown_connections 2023-10-06T10:45:26.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.762+0000 7f8ee5705b80 1 --2- 172.21.15.92:0/2555431184 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55662204ec00 0x556622041d40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.762+0000 7f8ee5705b80 1 --2- 172.21.15.92:0/2555431184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556622043810 0x556622041800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.762+0000 7f8ee5705b80 1 --2- 172.21.15.92:0/2555431184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556622004990 0x556622004d60 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.762+0000 7f8eb4ff9700 0 client.8949 ms_handle_reset on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:26.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.762+0000 7f8ee5705b80 1 --2- 172.21.15.92:0/2555431184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55662202dea0 0x5566220052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.762+0000 7f8ee5705b80 1 -- 172.21.15.92:0/2555431184 >> 172.21.15.92:0/2555431184 conn(0x5566220062b0 msgr2=0x5566220450d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.762+0000 7f8ee5705b80 1 -- 172.21.15.92:0/2555431184 shutdown_connections 2023-10-06T10:45:26.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.762+0000 7f8ee5705b80 1 -- 172.21.15.92:0/2555431184 wait complete. 2023-10-06T10:45:26.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.762+0000 7f8ee5705b80 20 client.8949 trim_cache size 0 max 16384 2023-10-06T10:45:26.771 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:26.771 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:45:26.771 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:26.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.763+0000 7f8edb1ab700 1 -- 172.21.15.92:0/2397328539 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8eb8066f30 msgr2=0x7f8eb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.763+0000 7f8edb1ab700 1 --2- 172.21.15.92:0/2397328539 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8eb8066f30 0x7f8eb80693e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4069d20 tx=0x7f8ed0009040 comp rx=0 tx=0).stop 2023-10-06T10:45:26.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.763+0000 7f8edb1ab700 1 -- 172.21.15.92:0/2397328539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ed40b29f0 msgr2=0x7f8ed40b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.763+0000 7f8edb1ab700 1 --2- 172.21.15.92:0/2397328539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ed40b29f0 0x7f8ed40b3540 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4013000 tx=0x7f8ec4017000 comp rx=0 tx=0).stop 2023-10-06T10:45:26.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.763+0000 7f8edb1ab700 1 -- 172.21.15.92:0/2397328539 shutdown_connections 2023-10-06T10:45:26.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.763+0000 7f8edb1ab700 1 --2- 172.21.15.92:0/2397328539 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8eb8066f30 0x7f8eb80693e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.763+0000 7f8edb1ab700 1 --2- 172.21.15.92:0/2397328539 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ed40cd3c0 0x7f8ed4153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.763+0000 7f8edb1ab700 1 --2- 172.21.15.92:0/2397328539 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ed40b2620 0x7f8ed415ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.763+0000 7f8edb1ab700 1 --2- 172.21.15.92:0/2397328539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ed40b29f0 0x7f8ed40b3540 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.763+0000 7f8edb1ab700 1 -- 172.21.15.92:0/2397328539 >> 172.21.15.92:0/2397328539 conn(0x7f8ed4099520 msgr2=0x7f8ed40c6040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.763+0000 7f8edb1ab700 1 -- 172.21.15.92:0/2397328539 shutdown_connections 2023-10-06T10:45:26.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.763+0000 7f8edb1ab700 1 -- 172.21.15.92:0/2397328539 wait complete. 2023-10-06T10:45:26.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:1 scrub status 2023-10-06T10:45:26.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.889+0000 7fd4959d1700 1 Processor -- start 2023-10-06T10:45:26.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.889+0000 7fd4959d1700 1 -- start start 2023-10-06T10:45:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.890+0000 7fd4959d1700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd49006bfb0 0x7fd490159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.890+0000 7fd4959d1700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd49006a420 0x7fd49015a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.890+0000 7fd4959d1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd49015d130 con 0x7fd49006a050 2023-10-06T10:45:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.890+0000 7fd4959d1700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd49015a920 con 0x7fd49006a420 2023-10-06T10:45:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.890+0000 7fd4959d1700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49015d380 con 0x7fd49006bfb0 2023-10-06T10:45:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.890+0000 7fd48effd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd49006a050 0x7fd49006b8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34568/0 (socket says 172.21.15.92:34568) 2023-10-06T10:45:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.890+0000 7fd48effd700 1 -- 172.21.15.92:0/719985027 learned_addr learned my addr 172.21.15.92:0/719985027 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48f7fe700 1 --2- 172.21.15.92:0/719985027 >> v2:172.21.15.124:3300/0 conn(0x7fd49006a420 0x7fd49015a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48e7fc700 1 --2- 172.21.15.92:0/719985027 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd49006bfb0 0x7fd490159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48dffb700 1 -- 172.21.15.92:0/719985027 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1309141806 0 0) 0x7fd49015d130 con 0x7fd49006a050 2023-10-06T10:45:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48dffb700 1 -- 172.21.15.92:0/719985027 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd47c003570 con 0x7fd49006a050 2023-10-06T10:45:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48dffb700 1 -- 172.21.15.92:0/719985027 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3253789589 0 0) 0x7fd47c003570 con 0x7fd49006a050 2023-10-06T10:45:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48dffb700 1 -- 172.21.15.92:0/719985027 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd49006bfb0 msgr2=0x7fd490159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48dffb700 1 --2- 172.21.15.92:0/719985027 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd49006bfb0 0x7fd490159ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48dffb700 1 -- 172.21.15.92:0/719985027 >> v2:172.21.15.124:3300/0 conn(0x7fd49006a420 msgr2=0x7fd49015a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48dffb700 1 --2- 172.21.15.92:0/719985027 >> v2:172.21.15.124:3300/0 conn(0x7fd49006a420 0x7fd49015a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48dffb700 1 -- 172.21.15.92:0/719985027 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49015d660 con 0x7fd49006a050 2023-10-06T10:45:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48dffb700 1 -- 172.21.15.92:0/719985027 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fd4800017f0 con 0x7fd49006a050 2023-10-06T10:45:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48dffb700 1 -- 172.21.15.92:0/719985027 >> v1:172.21.15.92:6789/0 conn(0x7fd49006a050 legacy=0x7fd49006b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48dffb700 1 --2- 172.21.15.92:0/719985027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd47c003a60 0x7fd47c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48dffb700 1 --2- 172.21.15.92:0/719985027 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd47c004810 0x7fd47c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48dffb700 1 --2- 172.21.15.92:0/719985027 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd49006bfb0 0x7fd47c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48dffb700 1 -- 172.21.15.92:0/719985027 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49015a920 con 0x7fd47c003a60 2023-10-06T10:45:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48dffb700 1 -- 172.21.15.92:0/719985027 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49015d380 con 0x7fd47c004810 2023-10-06T10:45:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48dffb700 1 -- 172.21.15.92:0/719985027 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c00bb10 con 0x7fd49006bfb0 2023-10-06T10:45:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48effd700 1 --2- 172.21.15.92:0/719985027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd47c003a60 0x7fd47c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48e7fc700 1 --2- 172.21.15.92:0/719985027 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd47c004810 0x7fd47c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48f7fe700 1 --2- 172.21.15.92:0/719985027 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd49006bfb0 0x7fd47c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48effd700 1 -- 172.21.15.92:0/719985027 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd49006bfb0 msgr2=0x7fd47c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48effd700 1 --2- 172.21.15.92:0/719985027 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd49006bfb0 0x7fd47c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48effd700 1 -- 172.21.15.92:0/719985027 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd47c004810 msgr2=0x7fd47c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48effd700 1 --2- 172.21.15.92:0/719985027 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd47c004810 0x7fd47c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.891+0000 7fd48effd700 1 -- 172.21.15.92:0/719985027 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c003570 con 0x7fd47c003a60 2023-10-06T10:45:26.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.892+0000 7fd48effd700 1 --2- 172.21.15.92:0/719985027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd47c003a60 0x7fd47c003e70 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fd48000a970 tx=0x7fd480002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29b99c0c0aaf930 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.892+0000 7fd48dffb700 1 -- 172.21.15.92:0/719985027 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480012440 con 0x7fd47c003a60 2023-10-06T10:45:26.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.892+0000 7fd48dffb700 1 -- 172.21.15.92:0/719985027 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4800125c0 con 0x7fd47c003a60 2023-10-06T10:45:26.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.892+0000 7fd48dffb700 1 -- 172.21.15.92:0/719985027 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480012980 con 0x7fd47c003a60 2023-10-06T10:45:26.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.892+0000 7fd4959d1700 1 -- 172.21.15.92:0/719985027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd47c003a60 msgr2=0x7fd47c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.892+0000 7fd4959d1700 1 --2- 172.21.15.92:0/719985027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd47c003a60 0x7fd47c003e70 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fd48000a970 tx=0x7fd480002210 comp rx=0 tx=0).stop 2023-10-06T10:45:26.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.892+0000 7fd48f7fe700 1 -- 172.21.15.92:0/719985027 reap_dead start 2023-10-06T10:45:26.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.892+0000 7fd4959d1700 1 -- 172.21.15.92:0/719985027 shutdown_connections 2023-10-06T10:45:26.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.892+0000 7fd4959d1700 1 -- 172.21.15.92:0/719985027 >> 172.21.15.92:0/719985027 conn(0x7fd4900ce2f0 msgr2=0x7fd4900ce6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.892+0000 7fd4959d1700 1 -- 172.21.15.92:0/719985027 shutdown_connections 2023-10-06T10:45:26.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.892+0000 7fd4959d1700 1 -- 172.21.15.92:0/719985027 wait complete. 2023-10-06T10:45:26.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.893+0000 7fd4959d1700 1 Processor -- start 2023-10-06T10:45:26.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.894+0000 7fd4959d1700 1 -- start start 2023-10-06T10:45:26.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.894+0000 7fd4959d1700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd47c003a60 0x7fd4900b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.894+0000 7fd4959d1700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4900b1a60 0x7fd490157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.894+0000 7fd4959d1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4900af8c0 0x7fd4900afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.894+0000 7fd4959d1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4900b0270 con 0x7fd4900af8c0 2023-10-06T10:45:26.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.894+0000 7fd4959d1700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd490157e60 con 0x7fd4900b1a60 2023-10-06T10:45:26.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.894+0000 7fd4959d1700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4900b21b0 con 0x7fd47c003a60 2023-10-06T10:45:26.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.894+0000 7fd48f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4900af8c0 0x7fd4900afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.894+0000 7fd48f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4900af8c0 0x7fd4900afd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43166/0 (socket says 172.21.15.92:43166) 2023-10-06T10:45:26.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.894+0000 7fd48f7fe700 1 -- 172.21.15.92:0/3797401277 learned_addr learned my addr 172.21.15.92:0/3797401277 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.894+0000 7fd48effd700 1 --2- 172.21.15.92:0/3797401277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd47c003a60 0x7fd4900b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.894+0000 7fd48e7fc700 1 --2- 172.21.15.92:0/3797401277 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4900b1a60 0x7fd490157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.895+0000 7fd48f7fe700 1 -- 172.21.15.92:0/3797401277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd47c003a60 msgr2=0x7fd4900b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.895+0000 7fd48f7fe700 1 --2- 172.21.15.92:0/3797401277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd47c003a60 0x7fd4900b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.895+0000 7fd48f7fe700 1 -- 172.21.15.92:0/3797401277 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4900b1a60 msgr2=0x7fd490157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.895+0000 7fd48f7fe700 1 --2- 172.21.15.92:0/3797401277 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4900b1a60 0x7fd490157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.895+0000 7fd48f7fe700 1 -- 172.21.15.92:0/3797401277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4900b2490 con 0x7fd4900af8c0 2023-10-06T10:45:26.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.895+0000 7fd48f7fe700 1 --2- 172.21.15.92:0/3797401277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4900af8c0 0x7fd4900afd30 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fd4900705a0 tx=0x7fd4880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.895+0000 7fd486ffd700 1 -- 172.21.15.92:0/3797401277 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488013750 con 0x7fd4900af8c0 2023-10-06T10:45:26.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.895+0000 7fd486ffd700 1 -- 172.21.15.92:0/3797401277 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4880138b0 con 0x7fd4900af8c0 2023-10-06T10:45:26.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.895+0000 7fd4959d1700 1 -- 172.21.15.92:0/3797401277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4900b2770 con 0x7fd4900af8c0 2023-10-06T10:45:26.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.895+0000 7fd486ffd700 1 -- 172.21.15.92:0/3797401277 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488013c50 con 0x7fd4900af8c0 2023-10-06T10:45:26.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.895+0000 7fd4959d1700 1 -- 172.21.15.92:0/3797401277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4900b1ed0 con 0x7fd4900af8c0 2023-10-06T10:45:26.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.897+0000 7fd486ffd700 1 -- 172.21.15.92:0/3797401277 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd488014650 con 0x7fd4900af8c0 2023-10-06T10:45:26.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.897+0000 7fd486ffd700 1 --2- 172.21.15.92:0/3797401277 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd470066f30 0x7fd4700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.897+0000 7fd486ffd700 1 -- 172.21.15.92:0/3797401277 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fd488084cf0 con 0x7fd4900af8c0 2023-10-06T10:45:26.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.897+0000 7fd48effd700 1 --2- 172.21.15.92:0/3797401277 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd470066f30 0x7fd4700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.897+0000 7fd48effd700 1 --2- 172.21.15.92:0/3797401277 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd470066f30 0x7fd4700693e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd48000a430 tx=0x7fd480015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.899+0000 7fd49ecc9b80 1 Processor -- start 2023-10-06T10:45:26.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.899+0000 7fd49ecc9b80 1 -- start start 2023-10-06T10:45:26.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.899+0000 7fd49ecc9b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7eebae0 0x5626a7eebeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.899+0000 7fd49ecc9b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5626a7ea79f0 0x5626a7ea45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.899+0000 7fd49ecc9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7ea5c20 0x5626a7ea4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.899+0000 7fd49ecc9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a7e24cd0 con 0x5626a7ea5c20 2023-10-06T10:45:26.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.899+0000 7fd49ecc9b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a7dfbac0 con 0x5626a7eebae0 2023-10-06T10:45:26.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.899+0000 7fd49ecc9b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626a7df92c0 con 0x5626a7ea79f0 2023-10-06T10:45:26.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.900+0000 7fd48f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7ea5c20 0x5626a7ea4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.900+0000 7fd48f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7ea5c20 0x5626a7ea4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43176/0 (socket says 172.21.15.92:43176) 2023-10-06T10:45:26.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.900+0000 7fd48f7fe700 1 -- 172.21.15.92:0/3103995035 learned_addr learned my addr 172.21.15.92:0/3103995035 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.900+0000 7fd48effd700 1 --2- 172.21.15.92:0/3103995035 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7eebae0 0x5626a7eebeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.900+0000 7fd48e7fc700 1 --2- 172.21.15.92:0/3103995035 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5626a7ea79f0 0x5626a7ea45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.900+0000 7fd48f7fe700 1 -- 172.21.15.92:0/3103995035 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5626a7ea79f0 msgr2=0x5626a7ea45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.900+0000 7fd48f7fe700 1 --2- 172.21.15.92:0/3103995035 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5626a7ea79f0 0x5626a7ea45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.900+0000 7fd48f7fe700 1 -- 172.21.15.92:0/3103995035 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7eebae0 msgr2=0x5626a7eebeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.900+0000 7fd48f7fe700 1 --2- 172.21.15.92:0/3103995035 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7eebae0 0x5626a7eebeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.900+0000 7fd48f7fe700 1 -- 172.21.15.92:0/3103995035 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626a7c8f2c0 con 0x5626a7ea5c20 2023-10-06T10:45:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.900+0000 7fd48effd700 1 --2- 172.21.15.92:0/3103995035 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7eebae0 0x5626a7eebeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.900+0000 7fd48f7fe700 1 --2- 172.21.15.92:0/3103995035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7ea5c20 0x5626a7ea4b10 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fd488082e90 tx=0x7fd488082df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fb84f1e7bfabd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.900+0000 7fd46ffff700 1 -- 172.21.15.92:0/3103995035 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48805c8c0 con 0x5626a7ea5c20 2023-10-06T10:45:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.900+0000 7fd46ffff700 1 -- 172.21.15.92:0/3103995035 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd488006c50 con 0x5626a7ea5c20 2023-10-06T10:45:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.900+0000 7fd46ffff700 1 -- 172.21.15.92:0/3103995035 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488053cc0 con 0x5626a7ea5c20 2023-10-06T10:45:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.901+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/3103995035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7ea5c20 msgr2=0x5626a7ea4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.901+0000 7fd49ecc9b80 1 --2- 172.21.15.92:0/3103995035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7ea5c20 0x5626a7ea4b10 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fd488082e90 tx=0x7fd488082df0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.901+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/3103995035 shutdown_connections 2023-10-06T10:45:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.901+0000 7fd49ecc9b80 1 --2- 172.21.15.92:0/3103995035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7ea5c20 0x5626a7ea4b10 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.901+0000 7fd49ecc9b80 1 --2- 172.21.15.92:0/3103995035 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7eebae0 0x5626a7eebeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.901+0000 7fd49ecc9b80 1 --2- 172.21.15.92:0/3103995035 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5626a7ea79f0 0x5626a7ea45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.901+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/3103995035 >> 172.21.15.92:0/3103995035 conn(0x5626a7ea60c0 msgr2=0x5626a7ea6490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.901+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/3103995035 shutdown_connections 2023-10-06T10:45:26.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.901+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/3103995035 wait complete. 2023-10-06T10:45:26.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.901+0000 7fd49ecc9b80 1 -- start start 2023-10-06T10:45:26.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.901+0000 7fd49ecc9b80 1 Processor -- start 2023-10-06T10:45:26.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.902+0000 7fd49ecc9b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5626a7ea5c20 0x5626a7ee8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.902+0000 7fd49ecc9b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7ea79f0 0x5626a7ed4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.902+0000 7fd49ecc9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7eebae0 0x5626a7ee5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.902+0000 7fd49ecc9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a7df50b0 con 0x5626a7eebae0 2023-10-06T10:45:26.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.902+0000 7fd49ecc9b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a7da08e0 con 0x5626a7ea79f0 2023-10-06T10:45:26.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.902+0000 7fd49ecc9b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626a7dbbef0 con 0x5626a7ea5c20 2023-10-06T10:45:26.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.902+0000 7fd49ecc9b80 10 client.0 _ll_register_callbacks cb 0x5626a7c91f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:26.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.902+0000 7fd48f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7eebae0 0x5626a7ee5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.902+0000 7fd48f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7eebae0 0x5626a7ee5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43184/0 (socket says 172.21.15.92:43184) 2023-10-06T10:45:26.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.902+0000 7fd48f7fe700 1 -- 172.21.15.92:0/2676070595 learned_addr learned my addr 172.21.15.92:0/2676070595 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd48e7fc700 1 --2- 172.21.15.92:0/2676070595 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7ea79f0 0x5626a7ed4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd48effd700 1 --2- 172.21.15.92:0/2676070595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5626a7ea5c20 0x5626a7ee8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd48f7fe700 1 -- 172.21.15.92:0/2676070595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5626a7ea5c20 msgr2=0x5626a7ee8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd48f7fe700 1 --2- 172.21.15.92:0/2676070595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5626a7ea5c20 0x5626a7ee8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd48f7fe700 1 -- 172.21.15.92:0/2676070595 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7ea79f0 msgr2=0x5626a7ed4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd48f7fe700 1 --2- 172.21.15.92:0/2676070595 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7ea79f0 0x5626a7ed4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd48f7fe700 1 -- 172.21.15.92:0/2676070595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4900b1ed0 con 0x5626a7eebae0 2023-10-06T10:45:26.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd48e7fc700 1 --2- 172.21.15.92:0/2676070595 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7ea79f0 0x5626a7ed4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:26.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd48f7fe700 1 --2- 172.21.15.92:0/2676070595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7eebae0 0x5626a7ee5f20 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fd488006b90 tx=0x7fd488069000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7e7aba8b604aa2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd46d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:26.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd46d7fa700 1 -- 172.21.15.92:0/2676070595 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488053cc0 con 0x5626a7eebae0 2023-10-06T10:45:26.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd46d7fa700 1 -- 172.21.15.92:0/2676070595 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fd48805c430 con 0x5626a7eebae0 2023-10-06T10:45:26.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/2676070595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626a7e996e0 con 0x5626a7eebae0 2023-10-06T10:45:26.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd46d7fa700 1 -- 172.21.15.92:0/2676070595 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd48805cb10 con 0x5626a7eebae0 2023-10-06T10:45:26.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd49ecc9b80 10 client.8961 fetch_fsmap learned FSMap version 108 2023-10-06T10:45:26.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/2676070595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x5626a7c32030 con 0x5626a7eebae0 2023-10-06T10:45:26.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd46d7fa700 1 -- 172.21.15.92:0/2676070595 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626a7e996e0 con 0x5626a7eebae0 2023-10-06T10:45:26.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd46d7fa700 1 -- 172.21.15.92:0/2676070595 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 108) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fd48805cc70 con 0x5626a7eebae0 2023-10-06T10:45:26.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd49ecc9b80 10 client.8961 fetch_fsmap finished waiting for FSMap version 108 2023-10-06T10:45:26.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd49ecc9b80 10 client.8961 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2023-10-06T10:45:26.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd49ecc9b80 20 client.8961 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:26.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd49ecc9b80 1 --2- 172.21.15.92:0/2676070595 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5626a7ece930 0x5626a7eced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd49ecc9b80 4 client.8961 mds_command: new command op to 8517 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:26.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.903+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/2676070595 --> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626a7e76610 con 0x5626a7ece930 2023-10-06T10:45:26.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.904+0000 7fd48effd700 1 --2- 172.21.15.92:0/2676070595 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5626a7ece930 0x5626a7eced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.904+0000 7fd46d7fa700 10 client.8961 ms_handle_connect on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:26.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.904+0000 7fd48effd700 1 --2- 172.21.15.92:0/2676070595 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5626a7ece930 0x5626a7eced30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=583b7e1bf2bc2932 server_cookie=33d60c55042fc80a in_seq=0 out_seq=0 2023-10-06T10:45:26.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.904+0000 7fd46d7fa700 1 -- 172.21.15.92:0/2676070595 <== mds.1 v2:172.21.15.92:6834/1979826278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5626a7e76610 con 0x5626a7ece930 2023-10-06T10:45:26.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.904+0000 7fd46d7fa700 10 client.8961 handle_command_reply: tid=0 2023-10-06T10:45:26.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.904+0000 7fd49ecc9b80 1 client.8961 shutdown 2023-10-06T10:45:26.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.905+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/2676070595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7eebae0 msgr2=0x5626a7ee5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.905+0000 7fd49ecc9b80 1 --2- 172.21.15.92:0/2676070595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7eebae0 0x5626a7ee5f20 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fd488006b90 tx=0x7fd488069000 comp rx=0 tx=0).stop 2023-10-06T10:45:26.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.905+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/2676070595 shutdown_connections 2023-10-06T10:45:26.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.905+0000 7fd49ecc9b80 1 --2- 172.21.15.92:0/2676070595 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5626a7ece930 0x5626a7eced30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.905+0000 7fd49ecc9b80 1 --2- 172.21.15.92:0/2676070595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7eebae0 0x5626a7ee5f20 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.905+0000 7fd46d7fa700 0 client.8961 ms_handle_reset on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:26.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.905+0000 7fd49ecc9b80 1 --2- 172.21.15.92:0/2676070595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5626a7ea5c20 0x5626a7ee8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.905+0000 7fd49ecc9b80 1 --2- 172.21.15.92:0/2676070595 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7ea79f0 0x5626a7ed4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.905+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/2676070595 >> 172.21.15.92:0/2676070595 conn(0x5626a7ea60c0 msgr2=0x5626a7eeaec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.905+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/2676070595 shutdown_connections 2023-10-06T10:45:26.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.905+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/2676070595 wait complete. 2023-10-06T10:45:26.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.905+0000 7fd49ecc9b80 20 client.8961 trim_cache size 0 max 16384 2023-10-06T10:45:26.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.920+0000 7fd49ecc9b80 1 Processor -- start 2023-10-06T10:45:26.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.921+0000 7fd49ecc9b80 1 -- start start 2023-10-06T10:45:26.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.921+0000 7fd49ecc9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7ea5c20 0x5626a7ea1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.921+0000 7fd49ecc9b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7ea79f0 0x5626a7e9f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.921+0000 7fd49ecc9b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5626a7eebae0 0x5626a7eced00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.921+0000 7fd49ecc9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a7c77e80 con 0x5626a7ea5c20 2023-10-06T10:45:26.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.921+0000 7fd49ecc9b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a7e9f110 con 0x5626a7ea79f0 2023-10-06T10:45:26.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.921+0000 7fd49ecc9b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626a7eea7d0 con 0x5626a7eebae0 2023-10-06T10:45:26.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.921+0000 7fd48effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7ea5c20 0x5626a7ea1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.921+0000 7fd48effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7ea5c20 0x5626a7ea1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43186/0 (socket says 172.21.15.92:43186) 2023-10-06T10:45:26.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.921+0000 7fd48effd700 1 -- 172.21.15.92:0/1230840488 learned_addr learned my addr 172.21.15.92:0/1230840488 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.922+0000 7fd48e7fc700 1 --2- 172.21.15.92:0/1230840488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7ea79f0 0x5626a7e9f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.922+0000 7fd48f7fe700 1 --2- 172.21.15.92:0/1230840488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5626a7eebae0 0x5626a7eced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.922+0000 7fd48effd700 1 -- 172.21.15.92:0/1230840488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5626a7eebae0 msgr2=0x5626a7eced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.922+0000 7fd48effd700 1 --2- 172.21.15.92:0/1230840488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5626a7eebae0 0x5626a7eced00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.922+0000 7fd48effd700 1 -- 172.21.15.92:0/1230840488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7ea79f0 msgr2=0x5626a7e9f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.922+0000 7fd48effd700 1 --2- 172.21.15.92:0/1230840488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7ea79f0 0x5626a7e9f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.922+0000 7fd48effd700 1 -- 172.21.15.92:0/1230840488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626a7ecd5e0 con 0x5626a7ea5c20 2023-10-06T10:45:26.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.922+0000 7fd48e7fc700 1 --2- 172.21.15.92:0/1230840488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7ea79f0 0x5626a7e9f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:26.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.922+0000 7fd48effd700 1 --2- 172.21.15.92:0/1230840488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7ea5c20 0x5626a7ea1ee0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fd480002250 tx=0x7fd480035250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e5d09aa2301592 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.922+0000 7fd484ff9700 1 -- 172.21.15.92:0/1230840488 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480032070 con 0x5626a7ea5c20 2023-10-06T10:45:26.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.922+0000 7fd484ff9700 1 -- 172.21.15.92:0/1230840488 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd480012870 con 0x5626a7ea5c20 2023-10-06T10:45:26.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.922+0000 7fd484ff9700 1 -- 172.21.15.92:0/1230840488 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4800017b0 con 0x5626a7ea5c20 2023-10-06T10:45:26.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.922+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/1230840488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7ea5c20 msgr2=0x5626a7ea1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.922+0000 7fd49ecc9b80 1 --2- 172.21.15.92:0/1230840488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7ea5c20 0x5626a7ea1ee0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fd480002250 tx=0x7fd480035250 comp rx=0 tx=0).stop 2023-10-06T10:45:26.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.922+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/1230840488 shutdown_connections 2023-10-06T10:45:26.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.922+0000 7fd49ecc9b80 1 --2- 172.21.15.92:0/1230840488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5626a7eebae0 0x5626a7eced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.922+0000 7fd49ecc9b80 1 --2- 172.21.15.92:0/1230840488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7ea5c20 0x5626a7ea1ee0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.922+0000 7fd49ecc9b80 1 --2- 172.21.15.92:0/1230840488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7ea79f0 0x5626a7e9f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.922+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/1230840488 >> 172.21.15.92:0/1230840488 conn(0x5626a7ea60c0 msgr2=0x5626a7ee8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.923+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/1230840488 shutdown_connections 2023-10-06T10:45:26.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.923+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/1230840488 wait complete. 2023-10-06T10:45:26.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.923+0000 7fd49ecc9b80 1 -- start start 2023-10-06T10:45:26.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.923+0000 7fd49ecc9b80 1 Processor -- start 2023-10-06T10:45:26.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.924+0000 7fd49ecc9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7ea5c20 0x5626a7ea6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.924+0000 7fd49ecc9b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5626a7ea79f0 0x5626a7ea69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.924+0000 7fd49ecc9b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7eebae0 0x5626a7ecb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.924+0000 7fd49ecc9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a7f51670 con 0x5626a7ea5c20 2023-10-06T10:45:26.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.924+0000 7fd49ecc9b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a7f51800 con 0x5626a7eebae0 2023-10-06T10:45:26.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.924+0000 7fd49ecc9b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626a7ea2100 con 0x5626a7ea79f0 2023-10-06T10:45:26.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.924+0000 7fd49ecc9b80 10 client.0 _ll_register_callbacks cb 0x5626a7c91f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:26.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.924+0000 7fd48effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7ea5c20 0x5626a7ea6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.924+0000 7fd48effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7ea5c20 0x5626a7ea6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43188/0 (socket says 172.21.15.92:43188) 2023-10-06T10:45:26.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.924+0000 7fd48effd700 1 -- 172.21.15.92:0/623205773 learned_addr learned my addr 172.21.15.92:0/623205773 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:26.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.924+0000 7fd48f7fe700 1 --2- 172.21.15.92:0/623205773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7eebae0 0x5626a7ecb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.924+0000 7fd48e7fc700 1 --2- 172.21.15.92:0/623205773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5626a7ea79f0 0x5626a7ea69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.924+0000 7fd48effd700 1 -- 172.21.15.92:0/623205773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5626a7ea79f0 msgr2=0x5626a7ea69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.924+0000 7fd48effd700 1 --2- 172.21.15.92:0/623205773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5626a7ea79f0 0x5626a7ea69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.924+0000 7fd48effd700 1 -- 172.21.15.92:0/623205773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7eebae0 msgr2=0x5626a7ecb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:26.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.924+0000 7fd48effd700 1 --2- 172.21.15.92:0/623205773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7eebae0 0x5626a7ecb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.924+0000 7fd48effd700 1 -- 172.21.15.92:0/623205773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd47c003570 con 0x5626a7ea5c20 2023-10-06T10:45:26.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.924+0000 7fd48effd700 1 --2- 172.21.15.92:0/623205773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7ea5c20 0x5626a7ea6490 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fd480002250 tx=0x7fd480032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdac382ef48c20ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:26.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.924+0000 7fd46d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:26.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.925+0000 7fd46d7fa700 1 -- 172.21.15.92:0/623205773 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48003d070 con 0x5626a7ea5c20 2023-10-06T10:45:26.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.925+0000 7fd46d7fa700 1 -- 172.21.15.92:0/623205773 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fd48002d710 con 0x5626a7ea5c20 2023-10-06T10:45:26.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.925+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/623205773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd488006c50 con 0x5626a7ea5c20 2023-10-06T10:45:26.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.925+0000 7fd46d7fa700 1 -- 172.21.15.92:0/623205773 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd48002dae0 con 0x5626a7ea5c20 2023-10-06T10:45:26.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.925+0000 7fd49ecc9b80 10 client.8967 fetch_fsmap learned FSMap version 108 2023-10-06T10:45:26.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.925+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/623205773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x5626a7dc37d0 con 0x5626a7ea5c20 2023-10-06T10:45:26.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.925+0000 7fd46d7fa700 1 -- 172.21.15.92:0/623205773 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd488006c50 con 0x5626a7ea5c20 2023-10-06T10:45:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.925+0000 7fd46d7fa700 1 -- 172.21.15.92:0/623205773 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 108) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fd48002dc40 con 0x5626a7ea5c20 2023-10-06T10:45:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.925+0000 7fd49ecc9b80 10 client.8967 fetch_fsmap finished waiting for FSMap version 108 2023-10-06T10:45:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.925+0000 7fd49ecc9b80 10 client.8967 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2023-10-06T10:45:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.925+0000 7fd49ecc9b80 20 client.8967 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.925+0000 7fd49ecc9b80 1 --2- 172.21.15.92:0/623205773 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5626a7f39a10 0x5626a7ecbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.925+0000 7fd49ecc9b80 4 client.8967 mds_command: new command op to 8517 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:45:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.925+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/623205773 --> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5626a7d97e50 con 0x5626a7f39a10 2023-10-06T10:45:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.925+0000 7fd48e7fc700 1 --2- 172.21.15.92:0/623205773 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5626a7f39a10 0x5626a7ecbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.925+0000 7fd46d7fa700 10 client.8967 ms_handle_connect on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.925+0000 7fd48e7fc700 1 --2- 172.21.15.92:0/623205773 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5626a7f39a10 0x5626a7ecbca0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db95a1b50b03cf0c server_cookie=395f282a471019ef in_seq=0 out_seq=0 2023-10-06T10:45:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd46d7fa700 1 -- 172.21.15.92:0/623205773 <== mds.1 v2:172.21.15.92:6834/1979826278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5626a7d97e50 con 0x5626a7f39a10 2023-10-06T10:45:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd46d7fa700 10 client.8967 handle_command_reply: tid=0 2023-10-06T10:45:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd49ecc9b80 1 client.8967 shutdown 2023-10-06T10:45:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/623205773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7ea5c20 msgr2=0x5626a7ea6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd49ecc9b80 1 --2- 172.21.15.92:0/623205773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7ea5c20 0x5626a7ea6490 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fd480002250 tx=0x7fd480032000 comp rx=0 tx=0).stop 2023-10-06T10:45:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/623205773 shutdown_connections 2023-10-06T10:45:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd49ecc9b80 1 --2- 172.21.15.92:0/623205773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5626a7eebae0 0x5626a7ecb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd49ecc9b80 1 --2- 172.21.15.92:0/623205773 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5626a7f39a10 0x5626a7ecbca0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd49ecc9b80 1 --2- 172.21.15.92:0/623205773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626a7ea5c20 0x5626a7ea6490 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd46d7fa700 0 client.8967 ms_handle_reset on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd49ecc9b80 1 --2- 172.21.15.92:0/623205773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5626a7ea79f0 0x5626a7ea69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/623205773 >> 172.21.15.92:0/623205773 conn(0x5626a7ea60c0 msgr2=0x5626a7ea0820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/623205773 shutdown_connections 2023-10-06T10:45:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd49ecc9b80 1 -- 172.21.15.92:0/623205773 wait complete. 2023-10-06T10:45:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd49ecc9b80 20 client.8967 trim_cache size 0 max 16384 2023-10-06T10:45:26.936 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "PAUSED (0 inodes in the stack)", 2023-10-06T10:45:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:45:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8": { 2023-10-06T10:45:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:45:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8", 2023-10-06T10:45:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:45:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:26.937 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:26.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd4959d1700 1 -- 172.21.15.92:0/3797401277 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd470066f30 msgr2=0x7fd4700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd4959d1700 1 --2- 172.21.15.92:0/3797401277 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd470066f30 0x7fd4700693e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd48000a430 tx=0x7fd480015000 comp rx=0 tx=0).stop 2023-10-06T10:45:26.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd4959d1700 1 -- 172.21.15.92:0/3797401277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4900af8c0 msgr2=0x7fd4900afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:26.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd4959d1700 1 --2- 172.21.15.92:0/3797401277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4900af8c0 0x7fd4900afd30 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fd4900705a0 tx=0x7fd4880091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd4959d1700 1 -- 172.21.15.92:0/3797401277 shutdown_connections 2023-10-06T10:45:26.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd4959d1700 1 --2- 172.21.15.92:0/3797401277 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd470066f30 0x7fd4700693e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd4959d1700 1 --2- 172.21.15.92:0/3797401277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4900af8c0 0x7fd4900afd30 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd4959d1700 1 --2- 172.21.15.92:0/3797401277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd47c003a60 0x7fd4900b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd4959d1700 1 --2- 172.21.15.92:0/3797401277 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4900b1a60 0x7fd490157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:26.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.926+0000 7fd4959d1700 1 -- 172.21.15.92:0/3797401277 >> 172.21.15.92:0/3797401277 conn(0x7fd4900ce2f0 msgr2=0x7fd49006b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:26.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.927+0000 7fd4959d1700 1 -- 172.21.15.92:0/3797401277 shutdown_connections 2023-10-06T10:45:26.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:26.927+0000 7fd4959d1700 1 -- 172.21.15.92:0/3797401277 wait complete. 2023-10-06T10:45:26.940 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:45:26.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:2 scrub status 2023-10-06T10:45:27.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.053+0000 7fe052c01700 1 Processor -- start 2023-10-06T10:45:27.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.054+0000 7fe052c01700 1 -- start start 2023-10-06T10:45:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.054+0000 7fe052c01700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe04c0b2620 0x7fe04c0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.054+0000 7fe052c01700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe04c0cf1b0 0x7fe04c0cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.054+0000 7fe052c01700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe04c0cc970 con 0x7fe04c0b29f0 2023-10-06T10:45:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.055+0000 7fe052c01700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe04c0cbd60 con 0x7fe04c0b2620 2023-10-06T10:45:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.055+0000 7fe052c01700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c0cc040 con 0x7fe04c0cf1b0 2023-10-06T10:45:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.055+0000 7fe051bff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe04c0b2620 0x7fe04c0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.055+0000 7fe051bff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe04c0b2620 0x7fe04c0cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43190/0 (socket says 172.21.15.92:43190) 2023-10-06T10:45:27.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.055+0000 7fe051bff700 1 -- 172.21.15.92:0/3283910777 learned_addr learned my addr 172.21.15.92:0/3283910777 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:27.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.055+0000 7fe052400700 1 --2- 172.21.15.92:0/3283910777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe04c0cf1b0 0x7fe04c0cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.055+0000 7fe051bff700 1 -- 172.21.15.92:0/3283910777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe04c0cf1b0 msgr2=0x7fe04c0cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:27.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.055+0000 7fe051bff700 1 --2- 172.21.15.92:0/3283910777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe04c0cf1b0 0x7fe04c0cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.055+0000 7fe051bff700 1 -- 172.21.15.92:0/3283910777 >> v1:172.21.15.124:6789/0 conn(0x7fe04c0b29f0 legacy=0x7fe04c0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.055+0000 7fe051bff700 1 -- 172.21.15.92:0/3283910777 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c0cc320 con 0x7fe04c0b2620 2023-10-06T10:45:27.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.055+0000 7fe051bff700 1 --2- 172.21.15.92:0/3283910777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe04c0b2620 0x7fe04c0cccb0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fe048009990 tx=0x7fe048009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc281545867ff14 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:27.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.055+0000 7fe050bfd700 1 -- 172.21.15.92:0/3283910777 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe048009e80 con 0x7fe04c0b2620 2023-10-06T10:45:27.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.055+0000 7fe050bfd700 1 -- 172.21.15.92:0/3283910777 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe048005c10 con 0x7fe04c0b2620 2023-10-06T10:45:27.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.055+0000 7fe050bfd700 1 -- 172.21.15.92:0/3283910777 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04800b650 con 0x7fe04c0b2620 2023-10-06T10:45:27.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.056+0000 7fe052c01700 1 -- 172.21.15.92:0/3283910777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe04c0b2620 msgr2=0x7fe04c0cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.056+0000 7fe052c01700 1 --2- 172.21.15.92:0/3283910777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe04c0b2620 0x7fe04c0cccb0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fe048009990 tx=0x7fe048009220 comp rx=0 tx=0).stop 2023-10-06T10:45:27.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.056+0000 7fe052c01700 1 -- 172.21.15.92:0/3283910777 shutdown_connections 2023-10-06T10:45:27.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.056+0000 7fe052c01700 1 --2- 172.21.15.92:0/3283910777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe04c0cf1b0 0x7fe04c0cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.056+0000 7fe052c01700 1 --2- 172.21.15.92:0/3283910777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe04c0b2620 0x7fe04c0cccb0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.056+0000 7fe052c01700 1 -- 172.21.15.92:0/3283910777 >> 172.21.15.92:0/3283910777 conn(0x7fe04c099520 msgr2=0x7fe04c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:27.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.056+0000 7fe052c01700 1 -- 172.21.15.92:0/3283910777 shutdown_connections 2023-10-06T10:45:27.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.056+0000 7fe052c01700 1 -- 172.21.15.92:0/3283910777 wait complete. 2023-10-06T10:45:27.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.057+0000 7fe052c01700 1 Processor -- start 2023-10-06T10:45:27.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.057+0000 7fe052c01700 1 -- start start 2023-10-06T10:45:27.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.058+0000 7fe052c01700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe04c0b2620 0x7fe04c09db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.058+0000 7fe052c01700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe04c0b29f0 0x7fe04c09cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.058+0000 7fe052c01700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe04c0cf1b0 0x7fe04c0b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.058+0000 7fe052c01700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c09d300 con 0x7fe04c0b2620 2023-10-06T10:45:27.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.058+0000 7fe052c01700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c16a340 con 0x7fe04c0cf1b0 2023-10-06T10:45:27.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.058+0000 7fe052c01700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c169730 con 0x7fe04c0b29f0 2023-10-06T10:45:27.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.058+0000 7fe051bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe04c0b2620 0x7fe04c09db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.058+0000 7fe051bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe04c0b2620 0x7fe04c09db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43200/0 (socket says 172.21.15.92:43200) 2023-10-06T10:45:27.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.058+0000 7fe051bff700 1 -- 172.21.15.92:0/3061430479 learned_addr learned my addr 172.21.15.92:0/3061430479 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:27.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.058+0000 7fe0513fe700 1 --2- 172.21.15.92:0/3061430479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe04c0b29f0 0x7fe04c09cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.058+0000 7fe052400700 1 --2- 172.21.15.92:0/3061430479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe04c0cf1b0 0x7fe04c0b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.058+0000 7fe051bff700 1 -- 172.21.15.92:0/3061430479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe04c0b29f0 msgr2=0x7fe04c09cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.058+0000 7fe051bff700 1 --2- 172.21.15.92:0/3061430479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe04c0b29f0 0x7fe04c09cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.058+0000 7fe051bff700 1 -- 172.21.15.92:0/3061430479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe04c0cf1b0 msgr2=0x7fe04c0b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.058+0000 7fe051bff700 1 --2- 172.21.15.92:0/3061430479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe04c0cf1b0 0x7fe04c0b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.058+0000 7fe051bff700 1 -- 172.21.15.92:0/3061430479 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe048009480 con 0x7fe04c0b2620 2023-10-06T10:45:27.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.058+0000 7fe051bff700 1 --2- 172.21.15.92:0/3061430479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe04c0b2620 0x7fe04c09db60 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fe04800b3c0 tx=0x7fe048016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:27.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.058+0000 7fe041ffb700 1 -- 172.21.15.92:0/3061430479 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe048011ae0 con 0x7fe04c0b2620 2023-10-06T10:45:27.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.058+0000 7fe041ffb700 1 -- 172.21.15.92:0/3061430479 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe048020070 con 0x7fe04c0b2620 2023-10-06T10:45:27.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.058+0000 7fe052c01700 1 -- 172.21.15.92:0/3061430479 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe04c169a10 con 0x7fe04c0b2620 2023-10-06T10:45:27.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.058+0000 7fe041ffb700 1 -- 172.21.15.92:0/3061430479 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe048011e90 con 0x7fe04c0b2620 2023-10-06T10:45:27.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.059+0000 7fe052c01700 1 -- 172.21.15.92:0/3061430479 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe04c169fd0 con 0x7fe04c0b2620 2023-10-06T10:45:27.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.060+0000 7fe041ffb700 1 -- 172.21.15.92:0/3061430479 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe048022030 con 0x7fe04c0b2620 2023-10-06T10:45:27.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.060+0000 7fe041ffb700 1 --2- 172.21.15.92:0/3061430479 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe030066fb0 0x7fe030069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.061+0000 7fe041ffb700 1 -- 172.21.15.92:0/3061430479 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fe0480864f0 con 0x7fe04c0b2620 2023-10-06T10:45:27.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.061+0000 7fe0513fe700 1 --2- 172.21.15.92:0/3061430479 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe030066fb0 0x7fe030069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.061+0000 7fe0513fe700 1 --2- 172.21.15.92:0/3061430479 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe030066fb0 0x7fe030069460 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe04c0b2e10 tx=0x7fe03c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:27.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.063+0000 7fe05d15bb80 1 Processor -- start 2023-10-06T10:45:27.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.063+0000 7fe05d15bb80 1 -- start start 2023-10-06T10:45:27.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.063+0000 7fe05d15bb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5566bcdcb9a0 0x5566bcdcbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.063+0000 7fe05d15bb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566bce15c10 0x5566bcdf4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.063+0000 7fe05d15bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bcdf4eb0 0x5566bcdc79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.063+0000 7fe05d15bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566bcd4bcd0 con 0x5566bcdf4eb0 2023-10-06T10:45:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.063+0000 7fe05d15bb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566bcd22ac0 con 0x5566bce15c10 2023-10-06T10:45:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.063+0000 7fe05d15bb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566bcd202c0 con 0x5566bcdcb9a0 2023-10-06T10:45:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.063+0000 7fe052400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bcdf4eb0 0x5566bcdc79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.063+0000 7fe052400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bcdf4eb0 0x5566bcdc79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43214/0 (socket says 172.21.15.92:43214) 2023-10-06T10:45:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.063+0000 7fe052400700 1 -- 172.21.15.92:0/1590207032 learned_addr learned my addr 172.21.15.92:0/1590207032 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.064+0000 7fe051bff700 1 --2- 172.21.15.92:0/1590207032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5566bcdcb9a0 0x5566bcdcbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.064+0000 7fe0513fe700 1 --2- 172.21.15.92:0/1590207032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566bce15c10 0x5566bcdf4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.064+0000 7fe052400700 1 -- 172.21.15.92:0/1590207032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5566bcdcb9a0 msgr2=0x5566bcdcbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.064+0000 7fe052400700 1 --2- 172.21.15.92:0/1590207032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5566bcdcb9a0 0x5566bcdcbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.064+0000 7fe052400700 1 -- 172.21.15.92:0/1590207032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566bce15c10 msgr2=0x5566bcdf4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.064+0000 7fe052400700 1 --2- 172.21.15.92:0/1590207032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566bce15c10 0x5566bcdf4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.064+0000 7fe052400700 1 -- 172.21.15.92:0/1590207032 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566bcbb62c0 con 0x5566bcdf4eb0 2023-10-06T10:45:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.064+0000 7fe051bff700 1 --2- 172.21.15.92:0/1590207032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5566bcdcb9a0 0x5566bcdcbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.064+0000 7fe0513fe700 1 --2- 172.21.15.92:0/1590207032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566bce15c10 0x5566bcdf4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.064+0000 7fe052400700 1 --2- 172.21.15.92:0/1590207032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bcdf4eb0 0x5566bcdc79e0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fe0440075e0 tx=0x7fe04400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8721d8fb622134 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.064+0000 7fe036ffd700 1 -- 172.21.15.92:0/1590207032 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04400e030 con 0x5566bcdf4eb0 2023-10-06T10:45:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.064+0000 7fe036ffd700 1 -- 172.21.15.92:0/1590207032 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe044015600 con 0x5566bcdf4eb0 2023-10-06T10:45:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.064+0000 7fe036ffd700 1 -- 172.21.15.92:0/1590207032 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0440159c0 con 0x5566bcdf4eb0 2023-10-06T10:45:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.065+0000 7fe05d15bb80 1 -- 172.21.15.92:0/1590207032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bcdf4eb0 msgr2=0x5566bcdc79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.065+0000 7fe05d15bb80 1 --2- 172.21.15.92:0/1590207032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bcdf4eb0 0x5566bcdc79e0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fe0440075e0 tx=0x7fe04400b040 comp rx=0 tx=0).stop 2023-10-06T10:45:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.065+0000 7fe05d15bb80 1 -- 172.21.15.92:0/1590207032 shutdown_connections 2023-10-06T10:45:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.065+0000 7fe05d15bb80 1 --2- 172.21.15.92:0/1590207032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bcdf4eb0 0x5566bcdc79e0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.065+0000 7fe05d15bb80 1 --2- 172.21.15.92:0/1590207032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5566bcdcb9a0 0x5566bcdcbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.065+0000 7fe05d15bb80 1 --2- 172.21.15.92:0/1590207032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566bce15c10 0x5566bcdf4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.065+0000 7fe05d15bb80 1 -- 172.21.15.92:0/1590207032 >> 172.21.15.92:0/1590207032 conn(0x5566bcdcd2c0 msgr2=0x5566bcdcd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.065+0000 7fe05d15bb80 1 -- 172.21.15.92:0/1590207032 shutdown_connections 2023-10-06T10:45:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.065+0000 7fe05d15bb80 1 -- 172.21.15.92:0/1590207032 wait complete. 2023-10-06T10:45:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.065+0000 7fe05d15bb80 1 -- start start 2023-10-06T10:45:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.065+0000 7fe05d15bb80 1 Processor -- start 2023-10-06T10:45:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.066+0000 7fe05d15bb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566bcdcb9a0 0x5566bce0bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.066+0000 7fe05d15bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bcdf4eb0 0x5566bce09b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.066+0000 7fe05d15bb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5566bce15c10 0x5566bce0a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.066+0000 7fe05d15bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566bcd1c0b0 con 0x5566bcdf4eb0 2023-10-06T10:45:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.066+0000 7fe05d15bb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566bccc78e0 con 0x5566bcdcb9a0 2023-10-06T10:45:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.066+0000 7fe05d15bb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566bcce2ef0 con 0x5566bce15c10 2023-10-06T10:45:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.066+0000 7fe05d15bb80 10 client.0 _ll_register_callbacks cb 0x5566bcbb8f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.066+0000 7fe0513fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bcdf4eb0 0x5566bce09b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.066+0000 7fe0513fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bcdf4eb0 0x5566bce09b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43220/0 (socket says 172.21.15.92:43220) 2023-10-06T10:45:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.066+0000 7fe0513fe700 1 -- 172.21.15.92:0/2662341489 learned_addr learned my addr 172.21.15.92:0/2662341489 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.066+0000 7fe051bff700 1 --2- 172.21.15.92:0/2662341489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566bcdcb9a0 0x5566bce0bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe052400700 1 --2- 172.21.15.92:0/2662341489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5566bce15c10 0x5566bce0a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe0513fe700 1 -- 172.21.15.92:0/2662341489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5566bce15c10 msgr2=0x5566bce0a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe0513fe700 1 --2- 172.21.15.92:0/2662341489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5566bce15c10 0x5566bce0a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe0513fe700 1 -- 172.21.15.92:0/2662341489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566bcdcb9a0 msgr2=0x5566bce0bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe0513fe700 1 --2- 172.21.15.92:0/2662341489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566bcdcb9a0 0x5566bce0bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe0513fe700 1 -- 172.21.15.92:0/2662341489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe044007340 con 0x5566bcdf4eb0 2023-10-06T10:45:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe051bff700 1 --2- 172.21.15.92:0/2662341489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566bcdcb9a0 0x5566bce0bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe0513fe700 1 --2- 172.21.15.92:0/2662341489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bcdf4eb0 0x5566bce09b30 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fe03c003f10 tx=0x7fe03c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e02a9b3e9a3a84 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe02bfff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe02bfff700 1 -- 172.21.15.92:0/2662341489 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c00eee0 con 0x5566bcdf4eb0 2023-10-06T10:45:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe02bfff700 1 -- 172.21.15.92:0/2662341489 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fe03c01f420 con 0x5566bcdf4eb0 2023-10-06T10:45:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe05d15bb80 1 -- 172.21.15.92:0/2662341489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5566bcdc06e0 con 0x5566bcdf4eb0 2023-10-06T10:45:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe02bfff700 1 -- 172.21.15.92:0/2662341489 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe03c013e50 con 0x5566bcdf4eb0 2023-10-06T10:45:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe05d15bb80 10 client.8979 fetch_fsmap learned FSMap version 108 2023-10-06T10:45:27.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe05d15bb80 1 -- 172.21.15.92:0/2662341489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x5566bcb59030 con 0x5566bcdf4eb0 2023-10-06T10:45:27.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe02bfff700 1 -- 172.21.15.92:0/2662341489 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5566bcdc06e0 con 0x5566bcdf4eb0 2023-10-06T10:45:27.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe02bfff700 1 -- 172.21.15.92:0/2662341489 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 108) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fe03c01f7f0 con 0x5566bcdf4eb0 2023-10-06T10:45:27.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe05d15bb80 10 client.8979 fetch_fsmap finished waiting for FSMap version 108 2023-10-06T10:45:27.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe05d15bb80 10 client.8979 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2023-10-06T10:45:27.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe05d15bb80 20 client.8979 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:27.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe05d15bb80 1 --2- 172.21.15.92:0/2662341489 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x5566bce22a60 0x5566bcdf2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe05d15bb80 4 client.8979 mds_command: new command op to 6719 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:27.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.067+0000 7fe05d15bb80 1 -- 172.21.15.92:0/2662341489 --> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566bcd9d610 con 0x5566bce22a60 2023-10-06T10:45:27.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.068+0000 7fe051bff700 1 --2- 172.21.15.92:0/2662341489 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x5566bce22a60 0x5566bcdf2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.068+0000 7fe051bff700 1 --2- 172.21.15.92:0/2662341489 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x5566bce22a60 0x5566bcdf2f40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=346182f78a353b12 server_cookie=682c887d66b95fe1 in_seq=0 out_seq=0 2023-10-06T10:45:27.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.068+0000 7fe02bfff700 10 client.8979 ms_handle_connect on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.068+0000 7fe02bfff700 1 -- 172.21.15.92:0/2662341489 <== mds.2 v2:172.21.15.124:6833/2307521923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5566bcd9d610 con 0x5566bce22a60 2023-10-06T10:45:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.069+0000 7fe02bfff700 10 client.8979 handle_command_reply: tid=0 2023-10-06T10:45:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.069+0000 7fe05d15bb80 1 client.8979 shutdown 2023-10-06T10:45:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.069+0000 7fe05d15bb80 1 -- 172.21.15.92:0/2662341489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bcdf4eb0 msgr2=0x5566bce09b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.069+0000 7fe05d15bb80 1 --2- 172.21.15.92:0/2662341489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bcdf4eb0 0x5566bce09b30 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fe03c003f10 tx=0x7fe03c014000 comp rx=0 tx=0).stop 2023-10-06T10:45:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.069+0000 7fe05d15bb80 1 -- 172.21.15.92:0/2662341489 shutdown_connections 2023-10-06T10:45:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.069+0000 7fe05d15bb80 1 --2- 172.21.15.92:0/2662341489 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x5566bce22a60 0x5566bcdf2f40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.069+0000 7fe05d15bb80 1 --2- 172.21.15.92:0/2662341489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5566bce15c10 0x5566bce0a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.069+0000 7fe02bfff700 0 client.8979 ms_handle_reset on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.069+0000 7fe05d15bb80 1 --2- 172.21.15.92:0/2662341489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566bcdcb9a0 0x5566bce0bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.069+0000 7fe05d15bb80 1 --2- 172.21.15.92:0/2662341489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bcdf4eb0 0x5566bce09b30 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.069+0000 7fe05d15bb80 1 -- 172.21.15.92:0/2662341489 >> 172.21.15.92:0/2662341489 conn(0x5566bcdcd2c0 msgr2=0x5566bcdc21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.069+0000 7fe05d15bb80 1 -- 172.21.15.92:0/2662341489 shutdown_connections 2023-10-06T10:45:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.069+0000 7fe05d15bb80 1 -- 172.21.15.92:0/2662341489 wait complete. 2023-10-06T10:45:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.069+0000 7fe05d15bb80 20 client.8979 trim_cache size 0 max 16384 2023-10-06T10:45:27.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.085+0000 7fe05d15bb80 1 Processor -- start 2023-10-06T10:45:27.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.085+0000 7fe05d15bb80 1 -- start start 2023-10-06T10:45:27.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.085+0000 7fe05d15bb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5566bcdcb9a0 0x5566bcdf2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.085+0000 7fe05d15bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bcdf4eb0 0x5566bcdf3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.085+0000 7fe05d15bb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566bce15c10 0x5566bcdfb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.085+0000 7fe05d15bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566bcb9ee80 con 0x5566bcdf4eb0 2023-10-06T10:45:27.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.085+0000 7fe05d15bb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566bce10af0 con 0x5566bce15c10 2023-10-06T10:45:27.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.085+0000 7fe05d15bb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566bcdce3b0 con 0x5566bcdcb9a0 2023-10-06T10:45:27.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.085+0000 7fe0513fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bcdf4eb0 0x5566bcdf3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.086+0000 7fe0513fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bcdf4eb0 0x5566bcdf3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43228/0 (socket says 172.21.15.92:43228) 2023-10-06T10:45:27.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.086+0000 7fe0513fe700 1 -- 172.21.15.92:0/595692578 learned_addr learned my addr 172.21.15.92:0/595692578 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:27.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.086+0000 7fe051bff700 1 --2- 172.21.15.92:0/595692578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5566bcdcb9a0 0x5566bcdf2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.086+0000 7fe052400700 1 --2- 172.21.15.92:0/595692578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566bce15c10 0x5566bcdfb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.086+0000 7fe0513fe700 1 -- 172.21.15.92:0/595692578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5566bcdcb9a0 msgr2=0x5566bcdf2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:27.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.086+0000 7fe0513fe700 1 --2- 172.21.15.92:0/595692578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5566bcdcb9a0 0x5566bcdf2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.086+0000 7fe0513fe700 1 -- 172.21.15.92:0/595692578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566bce15c10 msgr2=0x5566bcdfb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:27.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.086+0000 7fe0513fe700 1 --2- 172.21.15.92:0/595692578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566bce15c10 0x5566bcdfb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.086+0000 7fe0513fe700 1 -- 172.21.15.92:0/595692578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566bccea7d0 con 0x5566bcdf4eb0 2023-10-06T10:45:27.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.086+0000 7fe0513fe700 1 --2- 172.21.15.92:0/595692578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bcdf4eb0 0x5566bcdf3480 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fe03c0035d0 tx=0x7fe03c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba1ba0edef9392c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:27.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.086+0000 7fe0377fe700 1 -- 172.21.15.92:0/595692578 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c035070 con 0x5566bcdf4eb0 2023-10-06T10:45:27.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.086+0000 7fe0377fe700 1 -- 172.21.15.92:0/595692578 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03c013a90 con 0x5566bcdf4eb0 2023-10-06T10:45:27.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.086+0000 7fe0377fe700 1 -- 172.21.15.92:0/595692578 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c01f420 con 0x5566bcdf4eb0 2023-10-06T10:45:27.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.086+0000 7fe05d15bb80 1 -- 172.21.15.92:0/595692578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bcdf4eb0 msgr2=0x5566bcdf3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.086+0000 7fe05d15bb80 1 --2- 172.21.15.92:0/595692578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bcdf4eb0 0x5566bcdf3480 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fe03c0035d0 tx=0x7fe03c01f000 comp rx=0 tx=0).stop 2023-10-06T10:45:27.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.087+0000 7fe05d15bb80 1 -- 172.21.15.92:0/595692578 shutdown_connections 2023-10-06T10:45:27.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.087+0000 7fe05d15bb80 1 --2- 172.21.15.92:0/595692578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566bce15c10 0x5566bcdfb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.087+0000 7fe05d15bb80 1 --2- 172.21.15.92:0/595692578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5566bcdcb9a0 0x5566bcdf2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.087+0000 7fe05d15bb80 1 --2- 172.21.15.92:0/595692578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bcdf4eb0 0x5566bcdf3480 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.087+0000 7fe05d15bb80 1 -- 172.21.15.92:0/595692578 >> 172.21.15.92:0/595692578 conn(0x5566bcdcd2c0 msgr2=0x5566bce0f880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:27.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.087+0000 7fe05d15bb80 1 -- 172.21.15.92:0/595692578 shutdown_connections 2023-10-06T10:45:27.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.087+0000 7fe05d15bb80 1 -- 172.21.15.92:0/595692578 wait complete. 2023-10-06T10:45:27.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.087+0000 7fe05d15bb80 1 -- start start 2023-10-06T10:45:27.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.087+0000 7fe05d15bb80 1 Processor -- start 2023-10-06T10:45:27.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.088+0000 7fe05d15bb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5566bcdcb9a0 0x5566bcdc9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.088+0000 7fe05d15bb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566bcdf4eb0 0x5566bce31c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.088+0000 7fe05d15bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bce15c10 0x5566bcdfb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.088+0000 7fe05d15bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566bcdf1b20 con 0x5566bce15c10 2023-10-06T10:45:27.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.088+0000 7fe05d15bb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566bce30220 con 0x5566bcdf4eb0 2023-10-06T10:45:27.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.088+0000 7fe05d15bb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566bce30500 con 0x5566bcdcb9a0 2023-10-06T10:45:27.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.088+0000 7fe05d15bb80 10 client.0 _ll_register_callbacks cb 0x5566bcbb8f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:27.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.088+0000 7fe052400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bce15c10 0x5566bcdfb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.088+0000 7fe052400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bce15c10 0x5566bcdfb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43244/0 (socket says 172.21.15.92:43244) 2023-10-06T10:45:27.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.088+0000 7fe052400700 1 -- 172.21.15.92:0/1611483450 learned_addr learned my addr 172.21.15.92:0/1611483450 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.088+0000 7fe051bff700 1 --2- 172.21.15.92:0/1611483450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5566bcdcb9a0 0x5566bcdc9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.088+0000 7fe0513fe700 1 --2- 172.21.15.92:0/1611483450 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566bcdf4eb0 0x5566bce31c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.088+0000 7fe052400700 1 -- 172.21.15.92:0/1611483450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5566bcdcb9a0 msgr2=0x5566bcdc9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.088+0000 7fe052400700 1 --2- 172.21.15.92:0/1611483450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5566bcdcb9a0 0x5566bcdc9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.088+0000 7fe052400700 1 -- 172.21.15.92:0/1611483450 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566bcdf4eb0 msgr2=0x5566bce31c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.088+0000 7fe052400700 1 --2- 172.21.15.92:0/1611483450 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566bcdf4eb0 0x5566bce31c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.088+0000 7fe052400700 1 -- 172.21.15.92:0/1611483450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566bcb59030 con 0x5566bce15c10 2023-10-06T10:45:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.089+0000 7fe052400700 1 --2- 172.21.15.92:0/1611483450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bce15c10 0x5566bcdfb2b0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fe0440070b0 tx=0x7fe04401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a726ad591f5d2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.089+0000 7fe02bfff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.089+0000 7fe02bfff700 1 -- 172.21.15.92:0/1611483450 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04400e040 con 0x5566bce15c10 2023-10-06T10:45:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.089+0000 7fe02bfff700 1 -- 172.21.15.92:0/1611483450 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fe04400f070 con 0x5566bce15c10 2023-10-06T10:45:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.089+0000 7fe05d15bb80 1 -- 172.21.15.92:0/1611483450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5566bcdc91d0 con 0x5566bce15c10 2023-10-06T10:45:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.089+0000 7fe02bfff700 1 -- 172.21.15.92:0/1611483450 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe044001040 con 0x5566bce15c10 2023-10-06T10:45:27.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.089+0000 7fe05d15bb80 10 client.8985 fetch_fsmap learned FSMap version 108 2023-10-06T10:45:27.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.089+0000 7fe05d15bb80 1 -- 172.21.15.92:0/1611483450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x5566bccd14c0 con 0x5566bce15c10 2023-10-06T10:45:27.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.089+0000 7fe02bfff700 1 -- 172.21.15.92:0/1611483450 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5566bcdc91d0 con 0x5566bce15c10 2023-10-06T10:45:27.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.089+0000 7fe02bfff700 1 -- 172.21.15.92:0/1611483450 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 108) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fe044027070 con 0x5566bce15c10 2023-10-06T10:45:27.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.089+0000 7fe05d15bb80 10 client.8985 fetch_fsmap finished waiting for FSMap version 108 2023-10-06T10:45:27.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.089+0000 7fe05d15bb80 10 client.8985 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2023-10-06T10:45:27.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.089+0000 7fe05d15bb80 20 client.8985 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:27.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.089+0000 7fe05d15bb80 1 --2- 172.21.15.92:0/1611483450 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x5566bce77650 0x5566bce77a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.089+0000 7fe05d15bb80 4 client.8985 mds_command: new command op to 6719 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:45:27.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.089+0000 7fe05d15bb80 1 -- 172.21.15.92:0/1611483450 --> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fe044015600 con 0x5566bce77650 2023-10-06T10:45:27.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.089+0000 7fe051bff700 1 --2- 172.21.15.92:0/1611483450 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x5566bce77650 0x5566bce77a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.090+0000 7fe02bfff700 10 client.8985 ms_handle_connect on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:27.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.090+0000 7fe051bff700 1 --2- 172.21.15.92:0/1611483450 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x5566bce77650 0x5566bce77a50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed346d91e53d27a7 server_cookie=5bac973551457a99 in_seq=0 out_seq=0 2023-10-06T10:45:27.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.090+0000 7fe02bfff700 1 -- 172.21.15.92:0/1611483450 <== mds.2 v2:172.21.15.124:6833/2307521923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x7fe044015600 con 0x5566bce77650 2023-10-06T10:45:27.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.090+0000 7fe02bfff700 10 client.8985 handle_command_reply: tid=0 2023-10-06T10:45:27.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.090+0000 7fe05d15bb80 1 client.8985 shutdown 2023-10-06T10:45:27.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.090+0000 7fe05d15bb80 1 -- 172.21.15.92:0/1611483450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bce15c10 msgr2=0x5566bcdfb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.090+0000 7fe05d15bb80 1 --2- 172.21.15.92:0/1611483450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bce15c10 0x5566bcdfb2b0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fe0440070b0 tx=0x7fe04401b000 comp rx=0 tx=0).stop 2023-10-06T10:45:27.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.090+0000 7fe05d15bb80 1 -- 172.21.15.92:0/1611483450 shutdown_connections 2023-10-06T10:45:27.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.090+0000 7fe05d15bb80 1 --2- 172.21.15.92:0/1611483450 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x5566bce77650 0x5566bce77a50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.090+0000 7fe05d15bb80 1 --2- 172.21.15.92:0/1611483450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566bce15c10 0x5566bcdfb2b0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.090+0000 7fe02bfff700 0 client.8985 ms_handle_reset on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:27.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.090+0000 7fe05d15bb80 1 --2- 172.21.15.92:0/1611483450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5566bcdcb9a0 0x5566bcdc9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.090+0000 7fe05d15bb80 1 --2- 172.21.15.92:0/1611483450 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5566bcdf4eb0 0x5566bce31c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.090+0000 7fe05d15bb80 1 -- 172.21.15.92:0/1611483450 >> 172.21.15.92:0/1611483450 conn(0x5566bcdcd2c0 msgr2=0x5566bcdf2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:27.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.090+0000 7fe05d15bb80 1 -- 172.21.15.92:0/1611483450 shutdown_connections 2023-10-06T10:45:27.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.090+0000 7fe05d15bb80 1 -- 172.21.15.92:0/1611483450 wait complete. 2023-10-06T10:45:27.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.090+0000 7fe05d15bb80 20 client.8985 trim_cache size 0 max 16384 2023-10-06T10:45:27.099 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:27.099 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "PAUSED (0 inodes in the stack)", 2023-10-06T10:45:27.099 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:45:27.099 INFO:teuthology.orchestra.run.smithi092.stdout: "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8": { 2023-10-06T10:45:27.099 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:45:27.100 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8", 2023-10-06T10:45:27.100 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:45:27.100 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:27.100 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:27.100 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:27.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.091+0000 7fe052c01700 1 -- 172.21.15.92:0/3061430479 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe030066fb0 msgr2=0x7fe030069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.091+0000 7fe052c01700 1 --2- 172.21.15.92:0/3061430479 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe030066fb0 0x7fe030069460 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe04c0b2e10 tx=0x7fe03c009040 comp rx=0 tx=0).stop 2023-10-06T10:45:27.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.091+0000 7fe052c01700 1 -- 172.21.15.92:0/3061430479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe04c0b2620 msgr2=0x7fe04c09db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.091+0000 7fe052c01700 1 --2- 172.21.15.92:0/3061430479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe04c0b2620 0x7fe04c09db60 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fe04800b3c0 tx=0x7fe048016000 comp rx=0 tx=0).stop 2023-10-06T10:45:27.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.091+0000 7fe052c01700 1 -- 172.21.15.92:0/3061430479 shutdown_connections 2023-10-06T10:45:27.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.091+0000 7fe052c01700 1 --2- 172.21.15.92:0/3061430479 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe030066fb0 0x7fe030069460 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.091+0000 7fe052c01700 1 --2- 172.21.15.92:0/3061430479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe04c0cf1b0 0x7fe04c0b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.091+0000 7fe052c01700 1 --2- 172.21.15.92:0/3061430479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe04c0b2620 0x7fe04c09db60 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.091+0000 7fe052c01700 1 --2- 172.21.15.92:0/3061430479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe04c0b29f0 0x7fe04c09cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.091+0000 7fe052c01700 1 -- 172.21.15.92:0/3061430479 >> 172.21.15.92:0/3061430479 conn(0x7fe04c099520 msgr2=0x7fe04c0cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:27.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.091+0000 7fe052c01700 1 -- 172.21.15.92:0/3061430479 shutdown_connections 2023-10-06T10:45:27.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.091+0000 7fe052c01700 1 -- 172.21.15.92:0/3061430479 wait complete. 2023-10-06T10:45:27.104 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:45:27.104 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub status 2023-10-06T10:45:27.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.218+0000 7fa4a156b700 1 Processor -- start 2023-10-06T10:45:27.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.218+0000 7fa4a156b700 1 -- start start 2023-10-06T10:45:27.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.219+0000 7fa4a156b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa49c150940 0x7fa49c159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.219+0000 7fa4a156b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa49c0cd4d0 0x7fa49c0b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.219+0000 7fa4a156b700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa49c15ac30 con 0x7fa49c15a160 2023-10-06T10:45:27.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.219+0000 7fa4a156b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa49c0cb6a0 con 0x7fa49c150940 2023-10-06T10:45:27.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.219+0000 7fa4a156b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c0cb090 con 0x7fa49c0cd4d0 2023-10-06T10:45:27.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.219+0000 7fa49affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa49c150940 0x7fa49c159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.219+0000 7fa49affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa49c150940 0x7fa49c159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43254/0 (socket says 172.21.15.92:43254) 2023-10-06T10:45:27.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.219+0000 7fa49affd700 1 -- 172.21.15.92:0/3045132213 learned_addr learned my addr 172.21.15.92:0/3045132213 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:27.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.219+0000 7fa49b7fe700 1 --2- 172.21.15.92:0/3045132213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa49c0cd4d0 0x7fa49c0b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.219+0000 7fa49affd700 1 -- 172.21.15.92:0/3045132213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa49c0cd4d0 msgr2=0x7fa49c0b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:27.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.219+0000 7fa49affd700 1 --2- 172.21.15.92:0/3045132213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa49c0cd4d0 0x7fa49c0b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.219+0000 7fa49affd700 1 -- 172.21.15.92:0/3045132213 >> v1:172.21.15.124:6789/0 conn(0x7fa49c15a160 legacy=0x7fa49c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:27.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.220+0000 7fa49affd700 1 -- 172.21.15.92:0/3045132213 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c15a670 con 0x7fa49c150940 2023-10-06T10:45:27.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.220+0000 7fa49affd700 1 --2- 172.21.15.92:0/3045132213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa49c150940 0x7fa49c159c20 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009990 tx=0x7fa48c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526eaaa4f5511dd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:27.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.220+0000 7fa49a7fc700 1 -- 172.21.15.92:0/3045132213 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c005c10 con 0x7fa49c150940 2023-10-06T10:45:27.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.220+0000 7fa49a7fc700 1 -- 172.21.15.92:0/3045132213 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48c005d70 con 0x7fa49c150940 2023-10-06T10:45:27.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.220+0000 7fa49a7fc700 1 -- 172.21.15.92:0/3045132213 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c00b880 con 0x7fa49c150940 2023-10-06T10:45:27.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.220+0000 7fa4a156b700 1 -- 172.21.15.92:0/3045132213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa49c150940 msgr2=0x7fa49c159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.220+0000 7fa4a156b700 1 --2- 172.21.15.92:0/3045132213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa49c150940 0x7fa49c159c20 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009990 tx=0x7fa48c009220 comp rx=0 tx=0).stop 2023-10-06T10:45:27.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.220+0000 7fa4a156b700 1 -- 172.21.15.92:0/3045132213 shutdown_connections 2023-10-06T10:45:27.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.220+0000 7fa4a156b700 1 --2- 172.21.15.92:0/3045132213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa49c0cd4d0 0x7fa49c0b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.220+0000 7fa4a156b700 1 --2- 172.21.15.92:0/3045132213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa49c150940 0x7fa49c159c20 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.220+0000 7fa4a156b700 1 -- 172.21.15.92:0/3045132213 >> 172.21.15.92:0/3045132213 conn(0x7fa49c099700 msgr2=0x7fa49c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:27.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.220+0000 7fa4a156b700 1 -- 172.21.15.92:0/3045132213 shutdown_connections 2023-10-06T10:45:27.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.220+0000 7fa4a156b700 1 -- 172.21.15.92:0/3045132213 wait complete. 2023-10-06T10:45:27.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.222+0000 7fa4a156b700 1 Processor -- start 2023-10-06T10:45:27.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.222+0000 7fa4a156b700 1 -- start start 2023-10-06T10:45:27.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.222+0000 7fa4a156b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa49c0cd4d0 0x7fa49c0b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.222+0000 7fa4a156b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa49c15a160 0x7fa49c0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.222+0000 7fa4a156b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa49c069490 0x7fa49c157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.222+0000 7fa4a156b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c0b5a30 con 0x7fa49c069490 2023-10-06T10:45:27.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.222+0000 7fa4a156b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c0b3910 con 0x7fa49c0cd4d0 2023-10-06T10:45:27.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.222+0000 7fa4a156b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c0b3bf0 con 0x7fa49c15a160 2023-10-06T10:45:27.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.222+0000 7fa49b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa49c069490 0x7fa49c157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.223+0000 7fa49b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa49c069490 0x7fa49c157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43260/0 (socket says 172.21.15.92:43260) 2023-10-06T10:45:27.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.223+0000 7fa49b7fe700 1 -- 172.21.15.92:0/2311795224 learned_addr learned my addr 172.21.15.92:0/2311795224 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:27.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.223+0000 7fa49affd700 1 --2- 172.21.15.92:0/2311795224 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa49c0cd4d0 0x7fa49c0b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.223+0000 7fa4927fc700 1 --2- 172.21.15.92:0/2311795224 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa49c15a160 0x7fa49c0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.223+0000 7fa49b7fe700 1 -- 172.21.15.92:0/2311795224 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa49c15a160 msgr2=0x7fa49c0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.223+0000 7fa49b7fe700 1 --2- 172.21.15.92:0/2311795224 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa49c15a160 0x7fa49c0b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.223+0000 7fa49b7fe700 1 -- 172.21.15.92:0/2311795224 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa49c0cd4d0 msgr2=0x7fa49c0b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.223+0000 7fa49b7fe700 1 --2- 172.21.15.92:0/2311795224 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa49c0cd4d0 0x7fa49c0b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.223+0000 7fa49b7fe700 1 -- 172.21.15.92:0/2311795224 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c009480 con 0x7fa49c069490 2023-10-06T10:45:27.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.223+0000 7fa49b7fe700 1 --2- 172.21.15.92:0/2311795224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa49c069490 0x7fa49c157840 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fa494007f30 tx=0x7fa4940098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:27.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.223+0000 7fa4937fe700 1 -- 172.21.15.92:0/2311795224 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49400eb90 con 0x7fa49c069490 2023-10-06T10:45:27.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.223+0000 7fa4937fe700 1 -- 172.21.15.92:0/2311795224 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49400ecf0 con 0x7fa49c069490 2023-10-06T10:45:27.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.223+0000 7fa4a156b700 1 -- 172.21.15.92:0/2311795224 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49c0b3ed0 con 0x7fa49c069490 2023-10-06T10:45:27.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.223+0000 7fa4937fe700 1 -- 172.21.15.92:0/2311795224 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494014670 con 0x7fa49c069490 2023-10-06T10:45:27.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.223+0000 7fa4a156b700 1 -- 172.21.15.92:0/2311795224 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49c1518e0 con 0x7fa49c069490 2023-10-06T10:45:27.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.225+0000 7fa4937fe700 1 -- 172.21.15.92:0/2311795224 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa4940149e0 con 0x7fa49c069490 2023-10-06T10:45:27.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.225+0000 7fa4937fe700 1 --2- 172.21.15.92:0/2311795224 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa47c066f30 0x7fa47c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.225+0000 7fa4937fe700 1 -- 172.21.15.92:0/2311795224 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fa494084e30 con 0x7fa49c069490 2023-10-06T10:45:27.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.225+0000 7fa49affd700 1 --2- 172.21.15.92:0/2311795224 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa47c066f30 0x7fa47c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.226+0000 7fa49affd700 1 --2- 172.21.15.92:0/2311795224 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa47c066f30 0x7fa47c0693e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009260 tx=0x7fa48c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:27.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.227+0000 7fa4aa863b80 1 Processor -- start 2023-10-06T10:45:27.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.227+0000 7fa4aa863b80 1 -- start start 2023-10-06T10:45:27.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.228+0000 7fa4aa863b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2eeb6ae0 0x556c2eeb6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.228+0000 7fa4aa863b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556c2ee729f0 0x556c2ee6f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.228+0000 7fa4aa863b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556c2ee70c20 0x556c2ee6fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.228+0000 7fa4aa863b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c2edefcd0 con 0x556c2eeb6ae0 2023-10-06T10:45:27.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.228+0000 7fa4aa863b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c2edc6ac0 con 0x556c2ee729f0 2023-10-06T10:45:27.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.228+0000 7fa4aa863b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c2edc42c0 con 0x556c2ee70c20 2023-10-06T10:45:27.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.228+0000 7fa49affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2eeb6ae0 0x556c2eeb6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.228+0000 7fa49affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2eeb6ae0 0x556c2eeb6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43262/0 (socket says 172.21.15.92:43262) 2023-10-06T10:45:27.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.228+0000 7fa49affd700 1 -- 172.21.15.92:0/4095874191 learned_addr learned my addr 172.21.15.92:0/4095874191 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:27.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.228+0000 7fa49b7fe700 1 --2- 172.21.15.92:0/4095874191 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556c2ee70c20 0x556c2ee6fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.228+0000 7fa4927fc700 1 --2- 172.21.15.92:0/4095874191 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556c2ee729f0 0x556c2ee6f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.228+0000 7fa49affd700 1 -- 172.21.15.92:0/4095874191 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556c2ee70c20 msgr2=0x556c2ee6fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:27.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.228+0000 7fa49affd700 1 --2- 172.21.15.92:0/4095874191 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556c2ee70c20 0x556c2ee6fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.228+0000 7fa49affd700 1 -- 172.21.15.92:0/4095874191 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556c2ee729f0 msgr2=0x556c2ee6f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:27.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.228+0000 7fa49affd700 1 --2- 172.21.15.92:0/4095874191 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556c2ee729f0 0x556c2ee6f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.228+0000 7fa49affd700 1 -- 172.21.15.92:0/4095874191 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c2ec5a2c0 con 0x556c2eeb6ae0 2023-10-06T10:45:27.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.228+0000 7fa49affd700 1 --2- 172.21.15.92:0/4095874191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2eeb6ae0 0x556c2eeb6eb0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0d14c0 tx=0x7fa48c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2cf49ddc2bbbf5f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:27.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.228+0000 7fa47bfff700 1 -- 172.21.15.92:0/4095874191 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c021070 con 0x556c2eeb6ae0 2023-10-06T10:45:27.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.228+0000 7fa47bfff700 1 -- 172.21.15.92:0/4095874191 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48c02a810 con 0x556c2eeb6ae0 2023-10-06T10:45:27.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.228+0000 7fa47bfff700 1 -- 172.21.15.92:0/4095874191 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c02a9c0 con 0x556c2eeb6ae0 2023-10-06T10:45:27.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.229+0000 7fa4aa863b80 1 -- 172.21.15.92:0/4095874191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2eeb6ae0 msgr2=0x556c2eeb6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.229+0000 7fa4aa863b80 1 --2- 172.21.15.92:0/4095874191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2eeb6ae0 0x556c2eeb6eb0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0d14c0 tx=0x7fa48c021000 comp rx=0 tx=0).stop 2023-10-06T10:45:27.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.229+0000 7fa4aa863b80 1 -- 172.21.15.92:0/4095874191 shutdown_connections 2023-10-06T10:45:27.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.229+0000 7fa4aa863b80 1 --2- 172.21.15.92:0/4095874191 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556c2ee70c20 0x556c2ee6fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.229+0000 7fa4aa863b80 1 --2- 172.21.15.92:0/4095874191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2eeb6ae0 0x556c2eeb6eb0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.229+0000 7fa4aa863b80 1 --2- 172.21.15.92:0/4095874191 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556c2ee729f0 0x556c2ee6f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.229+0000 7fa4aa863b80 1 -- 172.21.15.92:0/4095874191 >> 172.21.15.92:0/4095874191 conn(0x556c2ee710c0 msgr2=0x556c2ee71490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:27.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.229+0000 7fa4aa863b80 1 -- 172.21.15.92:0/4095874191 shutdown_connections 2023-10-06T10:45:27.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.229+0000 7fa4aa863b80 1 -- 172.21.15.92:0/4095874191 wait complete. 2023-10-06T10:45:27.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.230+0000 7fa4aa863b80 1 -- start start 2023-10-06T10:45:27.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.230+0000 7fa4aa863b80 1 Processor -- start 2023-10-06T10:45:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa4aa863b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2ee70c20 0x556c2ee9a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa4aa863b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556c2ee729f0 0x556c2ee97a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa4aa863b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556c2eeb6ae0 0x556c2ee97f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa4aa863b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c2edc00b0 con 0x556c2ee70c20 2023-10-06T10:45:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa4aa863b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c2ed6b8e0 con 0x556c2ee729f0 2023-10-06T10:45:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa4aa863b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c2ed86ef0 con 0x556c2eeb6ae0 2023-10-06T10:45:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa4aa863b80 10 client.0 _ll_register_callbacks cb 0x556c2ec5cf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa49affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2ee70c20 0x556c2ee9a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa49affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2ee70c20 0x556c2ee9a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43272/0 (socket says 172.21.15.92:43272) 2023-10-06T10:45:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa49affd700 1 -- 172.21.15.92:0/3208469629 learned_addr learned my addr 172.21.15.92:0/3208469629 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa4927fc700 1 --2- 172.21.15.92:0/3208469629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556c2ee729f0 0x556c2ee97a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa49b7fe700 1 --2- 172.21.15.92:0/3208469629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556c2eeb6ae0 0x556c2ee97f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa49affd700 1 -- 172.21.15.92:0/3208469629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556c2eeb6ae0 msgr2=0x556c2ee97f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa49affd700 1 --2- 172.21.15.92:0/3208469629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556c2eeb6ae0 0x556c2ee97f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa49affd700 1 -- 172.21.15.92:0/3208469629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556c2ee729f0 msgr2=0x556c2ee97a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa49affd700 1 --2- 172.21.15.92:0/3208469629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556c2ee729f0 0x556c2ee97a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa49affd700 1 -- 172.21.15.92:0/3208469629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49c15a670 con 0x556c2ee70c20 2023-10-06T10:45:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa4927fc700 1 --2- 172.21.15.92:0/3208469629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556c2ee729f0 0x556c2ee97a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa49b7fe700 1 --2- 172.21.15.92:0/3208469629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556c2eeb6ae0 0x556c2ee97f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa49affd700 1 --2- 172.21.15.92:0/3208469629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2ee70c20 0x556c2ee9a080 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009990 tx=0x7fa48c02c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495ccd9fb20d6074 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa4797fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa4797fa700 1 -- 172.21.15.92:0/3208469629 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c021040 con 0x556c2ee70c20 2023-10-06T10:45:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa4797fa700 1 -- 172.21.15.92:0/3208469629 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fa48c03b070 con 0x556c2ee70c20 2023-10-06T10:45:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.231+0000 7fa4aa863b80 1 -- 172.21.15.92:0/3208469629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c2ee646e0 con 0x556c2ee70c20 2023-10-06T10:45:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.232+0000 7fa4797fa700 1 -- 172.21.15.92:0/3208469629 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa48c033810 con 0x556c2ee70c20 2023-10-06T10:45:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.232+0000 7fa4aa863b80 10 client.8997 fetch_fsmap learned FSMap version 108 2023-10-06T10:45:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.232+0000 7fa4aa863b80 1 -- 172.21.15.92:0/3208469629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x556c2ebfd030 con 0x556c2ee70c20 2023-10-06T10:45:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.232+0000 7fa4797fa700 1 -- 172.21.15.92:0/3208469629 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556c2ee646e0 con 0x556c2ee70c20 2023-10-06T10:45:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.232+0000 7fa4797fa700 1 -- 172.21.15.92:0/3208469629 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 108) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fa48c033970 con 0x556c2ee70c20 2023-10-06T10:45:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.232+0000 7fa4aa863b80 10 client.8997 fetch_fsmap finished waiting for FSMap version 108 2023-10-06T10:45:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.232+0000 7fa4aa863b80 10 client.8997 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-10-06T10:45:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.232+0000 7fa4aa863b80 20 client.8997 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.232+0000 7fa4aa863b80 1 --2- 172.21.15.92:0/3208469629 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x556c2eeb1410 0x556c2eeb17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.232+0000 7fa4aa863b80 4 client.8997 mds_command: new command op to 8520 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.232+0000 7fa4aa863b80 1 -- 172.21.15.92:0/3208469629 --> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c2ee41610 con 0x556c2eeb1410 2023-10-06T10:45:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.232+0000 7fa4927fc700 1 --2- 172.21.15.92:0/3208469629 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x556c2eeb1410 0x556c2eeb17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.232+0000 7fa4797fa700 10 client.8997 ms_handle_connect on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.232+0000 7fa4927fc700 1 --2- 172.21.15.92:0/3208469629 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x556c2eeb1410 0x556c2eeb17e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24f3bfb6e9907774 server_cookie=aa3b0d6cecf36450 in_seq=0 out_seq=0 2023-10-06T10:45:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.233+0000 7fa4797fa700 1 -- 172.21.15.92:0/3208469629 <== mds.0 v2:172.21.15.92:6835/2298348426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x556c2ee41610 con 0x556c2eeb1410 2023-10-06T10:45:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.233+0000 7fa4797fa700 10 client.8997 handle_command_reply: tid=0 2023-10-06T10:45:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.233+0000 7fa4aa863b80 1 client.8997 shutdown 2023-10-06T10:45:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.233+0000 7fa4aa863b80 1 -- 172.21.15.92:0/3208469629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2ee70c20 msgr2=0x556c2ee9a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.233+0000 7fa4aa863b80 1 --2- 172.21.15.92:0/3208469629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2ee70c20 0x556c2ee9a080 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009990 tx=0x7fa48c02c040 comp rx=0 tx=0).stop 2023-10-06T10:45:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.233+0000 7fa4aa863b80 1 -- 172.21.15.92:0/3208469629 shutdown_connections 2023-10-06T10:45:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.233+0000 7fa4aa863b80 1 --2- 172.21.15.92:0/3208469629 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x556c2eeb1410 0x556c2eeb17e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.233+0000 7fa4aa863b80 1 --2- 172.21.15.92:0/3208469629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556c2eeb6ae0 0x556c2ee97f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.233+0000 7fa4aa863b80 1 --2- 172.21.15.92:0/3208469629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2ee70c20 0x556c2ee9a080 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.233+0000 7fa4797fa700 0 client.8997 ms_handle_reset on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.233+0000 7fa4aa863b80 1 --2- 172.21.15.92:0/3208469629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556c2ee729f0 0x556c2ee97a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.233+0000 7fa4aa863b80 1 -- 172.21.15.92:0/3208469629 >> 172.21.15.92:0/3208469629 conn(0x556c2ee710c0 msgr2=0x556c2eeb4740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.233+0000 7fa4aa863b80 1 -- 172.21.15.92:0/3208469629 shutdown_connections 2023-10-06T10:45:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.233+0000 7fa4aa863b80 1 -- 172.21.15.92:0/3208469629 wait complete. 2023-10-06T10:45:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.233+0000 7fa4aa863b80 20 client.8997 trim_cache size 0 max 16384 2023-10-06T10:45:27.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.249+0000 7fa4aa863b80 1 Processor -- start 2023-10-06T10:45:27.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.249+0000 7fa4aa863b80 1 -- start start 2023-10-06T10:45:27.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.250+0000 7fa4aa863b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556c2ee70c20 0x556c2ee914e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.250+0000 7fa4aa863b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556c2ee729f0 0x556c2ee6c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.250+0000 7fa4aa863b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2eeb6ae0 0x556c2ee97a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.250+0000 7fa4aa863b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c2ec42e80 con 0x556c2eeb6ae0 2023-10-06T10:45:27.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.250+0000 7fa4aa863b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c2ee987e0 con 0x556c2ee70c20 2023-10-06T10:45:27.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.250+0000 7fa4aa863b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c2eeb1dd0 con 0x556c2ee729f0 2023-10-06T10:45:27.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.250+0000 7fa49b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2eeb6ae0 0x556c2ee97a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.250+0000 7fa49b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2eeb6ae0 0x556c2ee97a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43276/0 (socket says 172.21.15.92:43276) 2023-10-06T10:45:27.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.250+0000 7fa49b7fe700 1 -- 172.21.15.92:0/3235630282 learned_addr learned my addr 172.21.15.92:0/3235630282 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:27.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.250+0000 7fa49affd700 1 --2- 172.21.15.92:0/3235630282 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556c2ee70c20 0x556c2ee914e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.250+0000 7fa4927fc700 1 --2- 172.21.15.92:0/3235630282 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556c2ee729f0 0x556c2ee6c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.250+0000 7fa49b7fe700 1 -- 172.21.15.92:0/3235630282 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556c2ee729f0 msgr2=0x556c2ee6c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:27.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.250+0000 7fa49b7fe700 1 --2- 172.21.15.92:0/3235630282 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556c2ee729f0 0x556c2ee6c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.250+0000 7fa49b7fe700 1 -- 172.21.15.92:0/3235630282 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556c2ee70c20 msgr2=0x556c2ee914e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.250+0000 7fa49b7fe700 1 --2- 172.21.15.92:0/3235630282 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556c2ee70c20 0x556c2ee914e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.250+0000 7fa49b7fe700 1 -- 172.21.15.92:0/3235630282 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c2ee682f0 con 0x556c2eeb6ae0 2023-10-06T10:45:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.250+0000 7fa49b7fe700 1 --2- 172.21.15.92:0/3235630282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2eeb6ae0 0x556c2ee97a50 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fa49400c040 tx=0x7fa494082fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12df56b3be1b682 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.250+0000 7fa490ff9700 1 -- 172.21.15.92:0/3235630282 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494055a80 con 0x556c2eeb6ae0 2023-10-06T10:45:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.250+0000 7fa490ff9700 1 -- 172.21.15.92:0/3235630282 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49406e030 con 0x556c2eeb6ae0 2023-10-06T10:45:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.250+0000 7fa490ff9700 1 -- 172.21.15.92:0/3235630282 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494058cd0 con 0x556c2eeb6ae0 2023-10-06T10:45:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.251+0000 7fa4aa863b80 1 -- 172.21.15.92:0/3235630282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2eeb6ae0 msgr2=0x556c2ee97a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.251+0000 7fa4aa863b80 1 --2- 172.21.15.92:0/3235630282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2eeb6ae0 0x556c2ee97a50 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fa49400c040 tx=0x7fa494082fd0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.251+0000 7fa4aa863b80 1 -- 172.21.15.92:0/3235630282 shutdown_connections 2023-10-06T10:45:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.251+0000 7fa4aa863b80 1 --2- 172.21.15.92:0/3235630282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2eeb6ae0 0x556c2ee97a50 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.251+0000 7fa4aa863b80 1 --2- 172.21.15.92:0/3235630282 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556c2ee70c20 0x556c2ee914e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.251+0000 7fa4aa863b80 1 --2- 172.21.15.92:0/3235630282 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556c2ee729f0 0x556c2ee6c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.251+0000 7fa4aa863b80 1 -- 172.21.15.92:0/3235630282 >> 172.21.15.92:0/3235630282 conn(0x556c2ee710c0 msgr2=0x556c2eeb51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:27.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.251+0000 7fa4aa863b80 1 -- 172.21.15.92:0/3235630282 shutdown_connections 2023-10-06T10:45:27.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.251+0000 7fa4aa863b80 1 -- 172.21.15.92:0/3235630282 wait complete. 2023-10-06T10:45:27.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.251+0000 7fa4aa863b80 1 -- start start 2023-10-06T10:45:27.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.252+0000 7fa4aa863b80 1 Processor -- start 2023-10-06T10:45:27.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.252+0000 7fa4aa863b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2ee70c20 0x556c2ee95720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.252+0000 7fa4aa863b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556c2ee729f0 0x556c2ef1b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.252+0000 7fa4aa863b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556c2eeb6ae0 0x556c2ee9a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.252+0000 7fa4aa863b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c2ee978f0 con 0x556c2ee70c20 2023-10-06T10:45:27.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.252+0000 7fa4aa863b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c2ee95d20 con 0x556c2eeb6ae0 2023-10-06T10:45:27.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.252+0000 7fa4aa863b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c2ee99e90 con 0x556c2ee729f0 2023-10-06T10:45:27.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.252+0000 7fa4aa863b80 10 client.0 _ll_register_callbacks cb 0x556c2ec5cf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:27.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.253+0000 7fa49affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2ee70c20 0x556c2ee95720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.253+0000 7fa49affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2ee70c20 0x556c2ee95720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43290/0 (socket says 172.21.15.92:43290) 2023-10-06T10:45:27.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.253+0000 7fa49affd700 1 -- 172.21.15.92:0/504854509 learned_addr learned my addr 172.21.15.92:0/504854509 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:27.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.253+0000 7fa49b7fe700 1 --2- 172.21.15.92:0/504854509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556c2eeb6ae0 0x556c2ee9a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.253+0000 7fa4927fc700 1 --2- 172.21.15.92:0/504854509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556c2ee729f0 0x556c2ef1b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.253+0000 7fa49affd700 1 -- 172.21.15.92:0/504854509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556c2ee729f0 msgr2=0x556c2ef1b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:27.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.253+0000 7fa49affd700 1 --2- 172.21.15.92:0/504854509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556c2ee729f0 0x556c2ef1b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.253+0000 7fa49affd700 1 -- 172.21.15.92:0/504854509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556c2eeb6ae0 msgr2=0x556c2ee9a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:27.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.253+0000 7fa49affd700 1 --2- 172.21.15.92:0/504854509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556c2eeb6ae0 0x556c2ee9a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.253+0000 7fa49affd700 1 -- 172.21.15.92:0/504854509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49c1518e0 con 0x556c2ee70c20 2023-10-06T10:45:27.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.253+0000 7fa49affd700 1 --2- 172.21.15.92:0/504854509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2ee70c20 0x556c2ee95720 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fa48c03fe90 tx=0x7fa48c038000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02cb32b2c9e8383 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:27.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.253+0000 7fa4797fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:27.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.253+0000 7fa4797fa700 1 -- 172.21.15.92:0/504854509 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c021040 con 0x556c2ee70c20 2023-10-06T10:45:27.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.253+0000 7fa4797fa700 1 -- 172.21.15.92:0/504854509 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fa48c03b050 con 0x556c2ee70c20 2023-10-06T10:45:27.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.253+0000 7fa4aa863b80 1 -- 172.21.15.92:0/504854509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa48c02a810 con 0x556c2ee70c20 2023-10-06T10:45:27.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.253+0000 7fa4797fa700 1 -- 172.21.15.92:0/504854509 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa48c03f630 con 0x556c2ee70c20 2023-10-06T10:45:27.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.253+0000 7fa4aa863b80 10 client.9003 fetch_fsmap learned FSMap version 108 2023-10-06T10:45:27.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.253+0000 7fa4aa863b80 1 -- 172.21.15.92:0/504854509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x556c2ed8e7d0 con 0x556c2ee70c20 2023-10-06T10:45:27.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.253+0000 7fa4797fa700 1 -- 172.21.15.92:0/504854509 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48c02a810 con 0x556c2ee70c20 2023-10-06T10:45:27.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4797fa700 1 -- 172.21.15.92:0/504854509 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 108) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fa48c03f790 con 0x556c2ee70c20 2023-10-06T10:45:27.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4aa863b80 10 client.9003 fetch_fsmap finished waiting for FSMap version 108 2023-10-06T10:45:27.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4aa863b80 10 client.9003 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-10-06T10:45:27.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4aa863b80 20 client.9003 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:27.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4aa863b80 1 --2- 172.21.15.92:0/504854509 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x556c2eeb48f0 0x556c2ee96e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4aa863b80 4 client.9003 mds_command: new command op to 8520 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:45:27.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4aa863b80 1 -- 172.21.15.92:0/504854509 --> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556c2ed62e50 con 0x556c2eeb48f0 2023-10-06T10:45:27.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4927fc700 1 --2- 172.21.15.92:0/504854509 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x556c2eeb48f0 0x556c2ee96e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4797fa700 10 client.9003 ms_handle_connect on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:27.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4927fc700 1 --2- 172.21.15.92:0/504854509 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x556c2eeb48f0 0x556c2ee96e20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81456e1be56cb9f server_cookie=357de12a609c8ff4 in_seq=0 out_seq=0 2023-10-06T10:45:27.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4797fa700 1 -- 172.21.15.92:0/504854509 <== mds.0 v2:172.21.15.92:6835/2298348426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x556c2ed62e50 con 0x556c2eeb48f0 2023-10-06T10:45:27.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4797fa700 10 client.9003 handle_command_reply: tid=0 2023-10-06T10:45:27.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4aa863b80 1 client.9003 shutdown 2023-10-06T10:45:27.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4aa863b80 1 -- 172.21.15.92:0/504854509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2ee70c20 msgr2=0x556c2ee95720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4aa863b80 1 --2- 172.21.15.92:0/504854509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2ee70c20 0x556c2ee95720 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fa48c03fe90 tx=0x7fa48c038000 comp rx=0 tx=0).stop 2023-10-06T10:45:27.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4aa863b80 1 -- 172.21.15.92:0/504854509 shutdown_connections 2023-10-06T10:45:27.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4aa863b80 1 --2- 172.21.15.92:0/504854509 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x556c2eeb48f0 0x556c2ee96e20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4aa863b80 1 --2- 172.21.15.92:0/504854509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x556c2eeb6ae0 0x556c2ee9a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4797fa700 0 client.9003 ms_handle_reset on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:27.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4aa863b80 1 --2- 172.21.15.92:0/504854509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c2ee70c20 0x556c2ee95720 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4aa863b80 1 --2- 172.21.15.92:0/504854509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x556c2ee729f0 0x556c2ef1b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4aa863b80 1 -- 172.21.15.92:0/504854509 >> 172.21.15.92:0/504854509 conn(0x556c2ee710c0 msgr2=0x556c2ee6c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:27.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4aa863b80 1 -- 172.21.15.92:0/504854509 shutdown_connections 2023-10-06T10:45:27.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.254+0000 7fa4aa863b80 1 -- 172.21.15.92:0/504854509 wait complete. 2023-10-06T10:45:27.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.255+0000 7fa4aa863b80 20 client.9003 trim_cache size 0 max 16384 2023-10-06T10:45:27.264 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:27.264 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "PAUSED (0 inodes in the stack)", 2023-10-06T10:45:27.264 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:45:27.265 INFO:teuthology.orchestra.run.smithi092.stdout: "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8": { 2023-10-06T10:45:27.265 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:45:27.265 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8", 2023-10-06T10:45:27.265 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:45:27.265 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:27.265 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:27.265 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:27.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.255+0000 7fa4a156b700 1 -- 172.21.15.92:0/2311795224 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa47c066f30 msgr2=0x7fa47c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.255+0000 7fa4a156b700 1 --2- 172.21.15.92:0/2311795224 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa47c066f30 0x7fa47c0693e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009260 tx=0x7fa48c016000 comp rx=0 tx=0).stop 2023-10-06T10:45:27.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.255+0000 7fa4a156b700 1 -- 172.21.15.92:0/2311795224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa49c069490 msgr2=0x7fa49c157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.255+0000 7fa4a156b700 1 --2- 172.21.15.92:0/2311795224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa49c069490 0x7fa49c157840 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fa494007f30 tx=0x7fa4940098e0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.255+0000 7fa4a156b700 1 -- 172.21.15.92:0/2311795224 shutdown_connections 2023-10-06T10:45:27.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.255+0000 7fa4a156b700 1 --2- 172.21.15.92:0/2311795224 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa47c066f30 0x7fa47c0693e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.255+0000 7fa4a156b700 1 --2- 172.21.15.92:0/2311795224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa49c069490 0x7fa49c157840 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.255+0000 7fa4a156b700 1 --2- 172.21.15.92:0/2311795224 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa49c0cd4d0 0x7fa49c0b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.255+0000 7fa4a156b700 1 --2- 172.21.15.92:0/2311795224 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa49c15a160 0x7fa49c0b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.255+0000 7fa4a156b700 1 -- 172.21.15.92:0/2311795224 >> 172.21.15.92:0/2311795224 conn(0x7fa49c099700 msgr2=0x7fa49c15f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:27.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.255+0000 7fa4a156b700 1 -- 172.21.15.92:0/2311795224 shutdown_connections 2023-10-06T10:45:27.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.255+0000 7fa4a156b700 1 -- 172.21.15.92:0/2311795224 wait complete. 2023-10-06T10:45:27.268 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:45:27.268 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:45:27.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.380+0000 7f0730952700 1 Processor -- start 2023-10-06T10:45:27.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.380+0000 7f0730952700 1 -- start start 2023-10-06T10:45:27.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.381+0000 7f0730952700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f072c0ad210 0x7f072c0cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.381+0000 7f0730952700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f072c0cecf0 con 0x7f072c15db10 2023-10-06T10:45:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.381+0000 7f0730952700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f072c0cefd0 con 0x7f072c0ad5e0 2023-10-06T10:45:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.381+0000 7f0730952700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f072c0cca90 con 0x7f072c0ad210 2023-10-06T10:45:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.381+0000 7f072a59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f072c0ad210 0x7f072c0cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.381+0000 7f072ad9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f072c15db10 0x7f072c0ca0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34574/0 (socket says 172.21.15.92:34574) 2023-10-06T10:45:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.381+0000 7f072ad9d700 1 -- 172.21.15.92:0/3927637004 learned_addr learned my addr 172.21.15.92:0/3927637004 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.381+0000 7f072a59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f072c0ad210 0x7f072c0cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43302/0 (socket says 172.21.15.92:43302) 2023-10-06T10:45:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.382+0000 7f072a59c700 1 -- 172.21.15.92:0/3927637004 >> v1:172.21.15.124:6789/0 conn(0x7f072c0ad5e0 legacy=0x7f072c15d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.382+0000 7f072a59c700 1 -- 172.21.15.92:0/3927637004 >> v1:172.21.15.92:6789/0 conn(0x7f072c15db10 legacy=0x7f072c0ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.382+0000 7f072a59c700 1 -- 172.21.15.92:0/3927637004 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c0ccd70 con 0x7f072c0ad210 2023-10-06T10:45:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.382+0000 7f072959a700 1 -- 172.21.15.92:0/3927637004 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2076704678 0 0) 0x7f072c0cecf0 con 0x7f072c15db10 2023-10-06T10:45:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.382+0000 7f072a59c700 1 --2- 172.21.15.92:0/3927637004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f072c0ad210 0x7f072c0cd290 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f071c009eb0 tx=0x7f071c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d505e29999565e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.382+0000 7f072959a700 1 -- 172.21.15.92:0/3927637004 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c00e070 con 0x7f072c0ad210 2023-10-06T10:45:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.382+0000 7f072959a700 1 -- 172.21.15.92:0/3927637004 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071c00bc30 con 0x7f072c0ad210 2023-10-06T10:45:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.382+0000 7f072959a700 1 -- 172.21.15.92:0/3927637004 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c012430 con 0x7f072c0ad210 2023-10-06T10:45:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.382+0000 7f0730952700 1 -- 172.21.15.92:0/3927637004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f072c0ad210 msgr2=0x7f072c0cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.382+0000 7f0730952700 1 --2- 172.21.15.92:0/3927637004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f072c0ad210 0x7f072c0cd290 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f071c009eb0 tx=0x7f071c009780 comp rx=0 tx=0).stop 2023-10-06T10:45:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.383+0000 7f0730952700 1 -- 172.21.15.92:0/3927637004 shutdown_connections 2023-10-06T10:45:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.383+0000 7f0730952700 1 --2- 172.21.15.92:0/3927637004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f072c0ad210 0x7f072c0cd290 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.383+0000 7f0730952700 1 -- 172.21.15.92:0/3927637004 >> 172.21.15.92:0/3927637004 conn(0x7f072c067d90 msgr2=0x7f072c0ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.383+0000 7f0730952700 1 -- 172.21.15.92:0/3927637004 shutdown_connections 2023-10-06T10:45:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.383+0000 7f0730952700 1 -- 172.21.15.92:0/3927637004 wait complete. 2023-10-06T10:45:27.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.384+0000 7f0730952700 1 Processor -- start 2023-10-06T10:45:27.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.384+0000 7f0730952700 1 -- start start 2023-10-06T10:45:27.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.384+0000 7f0730952700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f072c0ad210 0x7f072c0c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.384+0000 7f0730952700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f072c0ad5e0 0x7f072c16e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.384+0000 7f0730952700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f072c15db10 0x7f072c0b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.384+0000 7f0730952700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c0b3590 con 0x7f072c15db10 2023-10-06T10:45:27.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.384+0000 7f0730952700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c16e770 con 0x7f072c0ad5e0 2023-10-06T10:45:27.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.384+0000 7f0730952700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072c0b2c60 con 0x7f072c0ad210 2023-10-06T10:45:27.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.384+0000 7f072ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f072c15db10 0x7f072c0b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.384+0000 7f072ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f072c15db10 0x7f072c0b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43314/0 (socket says 172.21.15.92:43314) 2023-10-06T10:45:27.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.384+0000 7f072ad9d700 1 -- 172.21.15.92:0/3828683866 learned_addr learned my addr 172.21.15.92:0/3828683866 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:27.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.384+0000 7f072a59c700 1 --2- 172.21.15.92:0/3828683866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f072c0ad210 0x7f072c0c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.384+0000 7f0729d9b700 1 --2- 172.21.15.92:0/3828683866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f072c0ad5e0 0x7f072c16e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.385+0000 7f072ad9d700 1 -- 172.21.15.92:0/3828683866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f072c0ad210 msgr2=0x7f072c0c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.385+0000 7f072ad9d700 1 --2- 172.21.15.92:0/3828683866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f072c0ad210 0x7f072c0c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.385+0000 7f072ad9d700 1 -- 172.21.15.92:0/3828683866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f072c0ad5e0 msgr2=0x7f072c16e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.385+0000 7f072ad9d700 1 --2- 172.21.15.92:0/3828683866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f072c0ad5e0 0x7f072c16e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.385+0000 7f072ad9d700 1 -- 172.21.15.92:0/3828683866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c009a20 con 0x7f072c15db10 2023-10-06T10:45:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.385+0000 7f072ad9d700 1 --2- 172.21.15.92:0/3828683866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f072c15db10 0x7f072c0b3a20 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f072400c510 tx=0x7f0724007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.385+0000 7f071a7fc700 1 -- 172.21.15.92:0/3828683866 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072400ce40 con 0x7f072c15db10 2023-10-06T10:45:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.385+0000 7f071a7fc700 1 -- 172.21.15.92:0/3828683866 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0724005b30 con 0x7f072c15db10 2023-10-06T10:45:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.385+0000 7f0730952700 1 -- 172.21.15.92:0/3828683866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f072c0b2f40 con 0x7f072c15db10 2023-10-06T10:45:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.385+0000 7f071a7fc700 1 -- 172.21.15.92:0/3828683866 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07240070d0 con 0x7f072c15db10 2023-10-06T10:45:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.385+0000 7f0730952700 1 -- 172.21.15.92:0/3828683866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072c0b2910 con 0x7f072c15db10 2023-10-06T10:45:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.386+0000 7f0730952700 1 -- 172.21.15.92:0/3828683866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f072c0b3740 con 0x7f072c15db10 2023-10-06T10:45:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.386+0000 7f071a7fc700 1 -- 172.21.15.92:0/3828683866 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f07240074c0 con 0x7f072c15db10 2023-10-06T10:45:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.387+0000 7f071a7fc700 1 --2- 172.21.15.92:0/3828683866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f070c066f30 0x7f070c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.387+0000 7f071a7fc700 1 -- 172.21.15.92:0/3828683866 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f0724083b50 con 0x7f072c15db10 2023-10-06T10:45:27.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.387+0000 7f072a59c700 1 --2- 172.21.15.92:0/3828683866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f070c066f30 0x7f070c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.387+0000 7f072a59c700 1 --2- 172.21.15.92:0/3828683866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f070c066f30 0x7f070c0693e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f071c009eb0 tx=0x7f071c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:27.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.391+0000 7f071a7fc700 1 -- 172.21.15.92:0/3828683866 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f072c0b3740 con 0x7f072c15db10 2023-10-06T10:45:27.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.617+0000 7f0730952700 1 -- 172.21.15.92:0/3828683866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f072c0b3740 con 0x7f072c15db10 2023-10-06T10:45:27.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.617+0000 7f071a7fc700 1 -- 172.21.15.92:0/3828683866 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+4643 (secure 0 0 0) 0x7f072c0b3740 con 0x7f072c15db10 2023-10-06T10:45:27.619 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:27.619 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":108,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2016052382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2016052382},{"type":"v1","addr":"172.21.15.124:6834","nonce":2016052382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:44:51.106486+0000","modified":"2023-10-06T10:45:26.429733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8520,"mds_1":8517,"mds_2":6719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6719":{"gid":6719,"name":"d","rank":2,"incarnation":104,"state":"up:active","state_seq":6,"addr":"172.21.15.124:6835/2307521923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2307521923},{"type":"v1","addr":"172.21.15.124:6835","nonce":2307521923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8517":{"gid":8517,"name":"a","rank":1,"incarnation":102,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6836/1979826278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979826278},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979826278}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2298348426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2298348426},{"type":"v1","addr":"172.21.15.92:6837","nonce":2298348426}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T10:45:27.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.619+0000 7f0730952700 1 -- 172.21.15.92:0/3828683866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f070c066f30 msgr2=0x7f070c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.619+0000 7f0730952700 1 --2- 172.21.15.92:0/3828683866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f070c066f30 0x7f070c0693e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f071c009eb0 tx=0x7f071c017000 comp rx=0 tx=0).stop 2023-10-06T10:45:27.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.619+0000 7f0730952700 1 -- 172.21.15.92:0/3828683866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f072c15db10 msgr2=0x7f072c0b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.619+0000 7f0730952700 1 --2- 172.21.15.92:0/3828683866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f072c15db10 0x7f072c0b3a20 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f072400c510 tx=0x7f0724007f00 comp rx=0 tx=0).stop 2023-10-06T10:45:27.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.619+0000 7f0730952700 1 -- 172.21.15.92:0/3828683866 shutdown_connections 2023-10-06T10:45:27.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.619+0000 7f0730952700 1 --2- 172.21.15.92:0/3828683866 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f070c066f30 0x7f070c0693e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.619+0000 7f0730952700 1 --2- 172.21.15.92:0/3828683866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f072c15db10 0x7f072c0b3a20 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.619+0000 7f0730952700 1 --2- 172.21.15.92:0/3828683866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f072c0ad210 0x7f072c0c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.619+0000 7f0730952700 1 --2- 172.21.15.92:0/3828683866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f072c0ad5e0 0x7f072c16e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.619+0000 7f0730952700 1 -- 172.21.15.92:0/3828683866 >> 172.21.15.92:0/3828683866 conn(0x7f072c067d90 msgr2=0x7f072c06b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:27.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.620+0000 7f0730952700 1 -- 172.21.15.92:0/3828683866 shutdown_connections 2023-10-06T10:45:27.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.620+0000 7f0730952700 1 -- 172.21.15.92:0/3828683866 wait complete. 2023-10-06T10:45:27.623 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 108 2023-10-06T10:45:27.633 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2023-10-06T10:45:27.633 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:45:27.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.744+0000 7f3400e82700 1 Processor -- start 2023-10-06T10:45:27.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.744+0000 7f3400e82700 1 -- start start 2023-10-06T10:45:27.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.745+0000 7f3400e82700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33fc0b49a0 0x7f33fc0aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.745+0000 7f3400e82700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33fc0b0c60 con 0x7f33fc0b0760 2023-10-06T10:45:27.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.745+0000 7f3400e82700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33fc0b0f40 con 0x7f33fc159560 2023-10-06T10:45:27.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.745+0000 7f3400e82700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc0b1220 con 0x7f33fc0b49a0 2023-10-06T10:45:27.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.745+0000 7f33fad9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f33fc0b0760 0x7f33fc0b1bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34584/0 (socket says 172.21.15.92:34584) 2023-10-06T10:45:27.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.745+0000 7f33fad9d700 1 -- 172.21.15.92:0/330310776 learned_addr learned my addr 172.21.15.92:0/330310776 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:27.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.745+0000 7f33f1d9b700 1 --2- 172.21.15.92:0/330310776 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33fc0b49a0 0x7f33fc0aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.745+0000 7f33f9d9b700 1 -- 172.21.15.92:0/330310776 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4194556244 0 0) 0x7f33fc0b0c60 con 0x7f33fc0b0760 2023-10-06T10:45:27.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.745+0000 7f33f9d9b700 1 -- 172.21.15.92:0/330310776 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33e8003570 con 0x7f33fc0b0760 2023-10-06T10:45:27.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33f9d9b700 1 -- 172.21.15.92:0/330310776 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 680104213 0 0) 0x7f33e8003570 con 0x7f33fc0b0760 2023-10-06T10:45:27.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33f9d9b700 1 -- 172.21.15.92:0/330310776 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33fc0b49a0 msgr2=0x7f33fc0aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:27.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33f9d9b700 1 --2- 172.21.15.92:0/330310776 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33fc0b49a0 0x7f33fc0aac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33f9d9b700 1 -- 172.21.15.92:0/330310776 >> v1:172.21.15.124:6789/0 conn(0x7f33fc159560 legacy=0x7f33fc0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33f9d9b700 1 -- 172.21.15.92:0/330310776 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33fc0b1500 con 0x7f33fc0b0760 2023-10-06T10:45:27.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33f9d9b700 1 -- 172.21.15.92:0/330310776 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f33f4001470 con 0x7f33fc0b0760 2023-10-06T10:45:27.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33f9d9b700 1 -- 172.21.15.92:0/330310776 >> v1:172.21.15.92:6789/0 conn(0x7f33fc0b0760 legacy=0x7f33fc0b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33f9d9b700 1 --2- 172.21.15.92:0/330310776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33e8003a90 0x7f33e8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33f9d9b700 1 --2- 172.21.15.92:0/330310776 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33fc0b49a0 0x7f33e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33f9d9b700 1 --2- 172.21.15.92:0/330310776 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33e8004d50 0x7f33e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33f9d9b700 1 -- 172.21.15.92:0/330310776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc0b1220 con 0x7f33e8003a90 2023-10-06T10:45:27.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33f9d9b700 1 -- 172.21.15.92:0/330310776 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e800bb70 con 0x7f33e8004d50 2023-10-06T10:45:27.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33f9d9b700 1 -- 172.21.15.92:0/330310776 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e800bd30 con 0x7f33fc0b49a0 2023-10-06T10:45:27.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33fa59c700 1 --2- 172.21.15.92:0/330310776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33e8003a90 0x7f33e8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33f1d9b700 1 --2- 172.21.15.92:0/330310776 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33fc0b49a0 0x7f33e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33fad9d700 1 --2- 172.21.15.92:0/330310776 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33e8004d50 0x7f33e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33fa59c700 1 -- 172.21.15.92:0/330310776 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33fc0b49a0 msgr2=0x7f33e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:27.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33fa59c700 1 --2- 172.21.15.92:0/330310776 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33fc0b49a0 0x7f33e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33fa59c700 1 -- 172.21.15.92:0/330310776 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33e8004d50 msgr2=0x7f33e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:27.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33fa59c700 1 --2- 172.21.15.92:0/330310776 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33e8004d50 0x7f33e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33fa59c700 1 -- 172.21.15.92:0/330310776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e8003570 con 0x7f33e8003a90 2023-10-06T10:45:27.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33fad9d700 1 --2- 172.21.15.92:0/330310776 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33e8004d50 0x7f33e80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:27.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33fa59c700 1 --2- 172.21.15.92:0/330310776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33e8003a90 0x7f33e8003ea0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00c2d0 tx=0x7f33ec001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801db6943d07137 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:27.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33f9d9b700 1 -- 172.21.15.92:0/330310776 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec001d80 con 0x7f33e8003a90 2023-10-06T10:45:27.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33f9d9b700 1 -- 172.21.15.92:0/330310776 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ec00cba0 con 0x7f33e8003a90 2023-10-06T10:45:27.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.746+0000 7f33f9d9b700 1 -- 172.21.15.92:0/330310776 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec012400 con 0x7f33e8003a90 2023-10-06T10:45:27.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.747+0000 7f3400e82700 1 -- 172.21.15.92:0/330310776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33e8003a90 msgr2=0x7f33e8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.747+0000 7f3400e82700 1 --2- 172.21.15.92:0/330310776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33e8003a90 0x7f33e8003ea0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00c2d0 tx=0x7f33ec001a70 comp rx=0 tx=0).stop 2023-10-06T10:45:27.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.747+0000 7f33fad9d700 1 -- 172.21.15.92:0/330310776 reap_dead start 2023-10-06T10:45:27.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.747+0000 7f3400e82700 1 -- 172.21.15.92:0/330310776 shutdown_connections 2023-10-06T10:45:27.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.747+0000 7f3400e82700 1 -- 172.21.15.92:0/330310776 >> 172.21.15.92:0/330310776 conn(0x7f33fc0997f0 msgr2=0x7f33fc06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:27.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.747+0000 7f3400e82700 1 -- 172.21.15.92:0/330310776 shutdown_connections 2023-10-06T10:45:27.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.747+0000 7f3400e82700 1 -- 172.21.15.92:0/330310776 wait complete. 2023-10-06T10:45:27.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.748+0000 7f3400e82700 1 Processor -- start 2023-10-06T10:45:27.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.748+0000 7f3400e82700 1 -- start start 2023-10-06T10:45:27.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.749+0000 7f3400e82700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33e8003a90 0x7f33fc0cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.749+0000 7f3400e82700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33fc15a900 0x7f33fc15ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.749+0000 7f3400e82700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33fc09cb40 0x7f33fc09cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.749+0000 7f3400e82700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc0af720 con 0x7f33e8003a90 2023-10-06T10:45:27.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.749+0000 7f3400e82700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc09d520 con 0x7f33fc15a900 2023-10-06T10:45:27.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.749+0000 7f3400e82700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc15c060 con 0x7f33fc09cb40 2023-10-06T10:45:27.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.749+0000 7f33fa59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33e8003a90 0x7f33fc0cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.749+0000 7f33fa59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33e8003a90 0x7f33fc0cc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43340/0 (socket says 172.21.15.92:43340) 2023-10-06T10:45:27.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.749+0000 7f33fa59c700 1 -- 172.21.15.92:0/1276335144 learned_addr learned my addr 172.21.15.92:0/1276335144 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:27.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.749+0000 7f33f1d9b700 1 --2- 172.21.15.92:0/1276335144 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33fc15a900 0x7f33fc15ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.749+0000 7f33fa59c700 1 -- 172.21.15.92:0/1276335144 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33fc09cb40 msgr2=0x7f33fc09cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.749+0000 7f33fad9d700 1 --2- 172.21.15.92:0/1276335144 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33fc09cb40 0x7f33fc09cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.749+0000 7f33fa59c700 1 --2- 172.21.15.92:0/1276335144 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33fc09cb40 0x7f33fc09cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.749+0000 7f33fa59c700 1 -- 172.21.15.92:0/1276335144 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33fc15a900 msgr2=0x7f33fc15ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:27.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.749+0000 7f33fa59c700 1 --2- 172.21.15.92:0/1276335144 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33fc15a900 0x7f33fc15ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:27.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.750+0000 7f33fa59c700 1 -- 172.21.15.92:0/1276335144 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33fc0ccb50 con 0x7f33e8003a90 2023-10-06T10:45:27.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.750+0000 7f33f1d9b700 1 --2- 172.21.15.92:0/1276335144 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33fc15a900 0x7f33fc15ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:27.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.750+0000 7f33fa59c700 1 --2- 172.21.15.92:0/1276335144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33e8003a90 0x7f33fc0cc610 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00c8c0 tx=0x7f33ec016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:27.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.750+0000 7f33f2ffd700 1 -- 172.21.15.92:0/1276335144 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec027470 con 0x7f33e8003a90 2023-10-06T10:45:27.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.750+0000 7f33f2ffd700 1 -- 172.21.15.92:0/1276335144 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ec0275d0 con 0x7f33e8003a90 2023-10-06T10:45:27.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.750+0000 7f3400e82700 1 -- 172.21.15.92:0/1276335144 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33fc15b450 con 0x7f33e8003a90 2023-10-06T10:45:27.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.750+0000 7f33f2ffd700 1 -- 172.21.15.92:0/1276335144 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec027990 con 0x7f33e8003a90 2023-10-06T10:45:27.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.750+0000 7f3400e82700 1 -- 172.21.15.92:0/1276335144 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33fc15ba10 con 0x7f33e8003a90 2023-10-06T10:45:27.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.751+0000 7f3400e82700 1 -- 172.21.15.92:0/1276335144 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33fc064a90 con 0x7f33e8003a90 2023-10-06T10:45:27.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.751+0000 7f33f2ffd700 1 -- 172.21.15.92:0/1276335144 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f33ec024070 con 0x7f33e8003a90 2023-10-06T10:45:27.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.751+0000 7f33f2ffd700 1 --2- 172.21.15.92:0/1276335144 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f33dc066f30 0x7f33dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:27.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.751+0000 7f33f2ffd700 1 -- 172.21.15.92:0/1276335144 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f33ec086f20 con 0x7f33e8003a90 2023-10-06T10:45:27.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.752+0000 7f33f1d9b700 1 --2- 172.21.15.92:0/1276335144 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f33dc066f30 0x7f33dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:27.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.752+0000 7f33f1d9b700 1 --2- 172.21.15.92:0/1276335144 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f33dc066f30 0x7f33dc0693e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f33e4006dc0 tx=0x7f33e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:27.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:27.756+0000 7f33f2ffd700 1 -- 172.21.15.92:0/1276335144 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f33ec053eb0 con 0x7f33e8003a90 2023-10-06T10:45:28.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:28.013+0000 7f3400e82700 1 -- 172.21.15.92:0/1276335144 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f33fc0b17e0 con 0x7f33e8003a90 2023-10-06T10:45:28.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:28.014+0000 7f33f2ffd700 1 -- 172.21.15.92:0/1276335144 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1603 (secure 0 0 0) 0x7f33ec0574d0 con 0x7f33e8003a90 2023-10-06T10:45:28.016 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:28.016 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":449,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":698400,"bytes_used":2455613440,"bytes_avail":770638499840,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":10152,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":108,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6719},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":8517},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8520}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2023-10-06T10:44:44.166325+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T10:45:28.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:28.015+0000 7f3400e82700 1 -- 172.21.15.92:0/1276335144 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f33dc066f30 msgr2=0x7f33dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:28.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:28.015+0000 7f3400e82700 1 --2- 172.21.15.92:0/1276335144 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f33dc066f30 0x7f33dc0693e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f33e4006dc0 tx=0x7f33e4008040 comp rx=0 tx=0).stop 2023-10-06T10:45:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:28.015+0000 7f3400e82700 1 -- 172.21.15.92:0/1276335144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33e8003a90 msgr2=0x7f33fc0cc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:28.015+0000 7f3400e82700 1 --2- 172.21.15.92:0/1276335144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33e8003a90 0x7f33fc0cc610 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00c8c0 tx=0x7f33ec016000 comp rx=0 tx=0).stop 2023-10-06T10:45:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:28.015+0000 7f3400e82700 1 -- 172.21.15.92:0/1276335144 shutdown_connections 2023-10-06T10:45:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:28.016+0000 7f3400e82700 1 --2- 172.21.15.92:0/1276335144 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f33dc066f30 0x7f33dc0693e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:28.016+0000 7f3400e82700 1 --2- 172.21.15.92:0/1276335144 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33fc09cb40 0x7f33fc09cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:28.016+0000 7f3400e82700 1 --2- 172.21.15.92:0/1276335144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33e8003a90 0x7f33fc0cc610 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:28.016+0000 7f3400e82700 1 --2- 172.21.15.92:0/1276335144 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33fc15a900 0x7f33fc15ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:28.016+0000 7f3400e82700 1 -- 172.21.15.92:0/1276335144 >> 172.21.15.92:0/1276335144 conn(0x7f33fc0997f0 msgr2=0x7f33fc157620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:28.016+0000 7f3400e82700 1 -- 172.21.15.92:0/1276335144 shutdown_connections 2023-10-06T10:45:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:28.016+0000 7f3400e82700 1 -- 172.21.15.92:0/1276335144 wait complete. 2023-10-06T10:45:28.029 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2023-10-06T10:45:29.030 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:45:29.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.145+0000 7fc26d870700 1 Processor -- start 2023-10-06T10:45:29.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.145+0000 7fc26d870700 1 -- start start 2023-10-06T10:45:29.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.146+0000 7fc26d870700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc2680cb5f0 0x7fc2680691a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:29.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.146+0000 7fc26d870700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc26815a1e0 0x7fc2680ca670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:29.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.146+0000 7fc26d870700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc268068d10 con 0x7fc2680696e0 2023-10-06T10:45:29.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.146+0000 7fc26d870700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc268154020 con 0x7fc2680cb5f0 2023-10-06T10:45:29.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.146+0000 7fc26d870700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc26815a670 con 0x7fc26815a1e0 2023-10-06T10:45:29.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.146+0000 7fc2677fe700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc26815a1e0 0x7fc2680ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:29.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.147+0000 7fc266ffd700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc2680cb5f0 0x7fc2680691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:29.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.147+0000 7fc2677fe700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc26815a1e0 0x7fc2680ca670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:51784/0 (socket says 172.21.15.92:51784) 2023-10-06T10:45:29.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.147+0000 7fc25e7fc700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fc2680696e0 0x7fc268159ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:60848/0 (socket says 172.21.15.92:60848) 2023-10-06T10:45:29.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.147+0000 7fc2677fe700 1 -- 172.21.15.92:0/1113569507 learned_addr learned my addr 172.21.15.92:0/1113569507 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:29.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.147+0000 7fc266ffd700 1 -- 172.21.15.92:0/1113569507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc26815a1e0 msgr2=0x7fc2680ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:29.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.147+0000 7fc266ffd700 1 --2- 172.21.15.92:0/1113569507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc26815a1e0 0x7fc2680ca670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:29.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.147+0000 7fc266ffd700 1 -- 172.21.15.92:0/1113569507 >> v1:172.21.15.124:6789/0 conn(0x7fc2680696e0 legacy=0x7fc268159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:29.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.147+0000 7fc266ffd700 1 -- 172.21.15.92:0/1113569507 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26815a950 con 0x7fc2680cb5f0 2023-10-06T10:45:29.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.147+0000 7fc2667fc700 1 -- 172.21.15.92:0/1113569507 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2889514549 0 0) 0x7fc268068d10 con 0x7fc2680696e0 2023-10-06T10:45:29.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.147+0000 7fc2677fe700 1 --2- 172.21.15.92:0/1113569507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc26815a1e0 0x7fc2680ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:45:29.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.147+0000 7fc266ffd700 1 --2- 172.21.15.92:0/1113569507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2680cb5f0 0x7fc2680691a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc2580098e0 tx=0x7fc2580091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12f9ada140062a4d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:29.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.147+0000 7fc2667fc700 1 -- 172.21.15.92:0/1113569507 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25800b640 con 0x7fc2680cb5f0 2023-10-06T10:45:29.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.147+0000 7fc2667fc700 1 -- 172.21.15.92:0/1113569507 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25800b7a0 con 0x7fc2680cb5f0 2023-10-06T10:45:29.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.147+0000 7fc2667fc700 1 -- 172.21.15.92:0/1113569507 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25800bb40 con 0x7fc2680cb5f0 2023-10-06T10:45:29.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.148+0000 7fc26d870700 1 -- 172.21.15.92:0/1113569507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2680cb5f0 msgr2=0x7fc2680691a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:29.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.148+0000 7fc26d870700 1 --2- 172.21.15.92:0/1113569507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2680cb5f0 0x7fc2680691a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc2580098e0 tx=0x7fc2580091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:29.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.148+0000 7fc26d870700 1 -- 172.21.15.92:0/1113569507 shutdown_connections 2023-10-06T10:45:29.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.148+0000 7fc26d870700 1 --2- 172.21.15.92:0/1113569507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc26815a1e0 0x7fc2680ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:29.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.148+0000 7fc26d870700 1 --2- 172.21.15.92:0/1113569507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2680cb5f0 0x7fc2680691a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:29.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.148+0000 7fc26d870700 1 -- 172.21.15.92:0/1113569507 >> 172.21.15.92:0/1113569507 conn(0x7fc2680707d0 msgr2=0x7fc268070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:29.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.148+0000 7fc26d870700 1 -- 172.21.15.92:0/1113569507 shutdown_connections 2023-10-06T10:45:29.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.148+0000 7fc26d870700 1 -- 172.21.15.92:0/1113569507 wait complete. 2023-10-06T10:45:29.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.149+0000 7fc26d870700 1 Processor -- start 2023-10-06T10:45:29.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.149+0000 7fc26d870700 1 -- start start 2023-10-06T10:45:29.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.150+0000 7fc26d870700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2680696e0 0x7fc2680b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:29.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.150+0000 7fc26d870700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2680cb5f0 0x7fc268152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:29.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.150+0000 7fc26d870700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc26815a1e0 0x7fc2680b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:29.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.150+0000 7fc26d870700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26816a900 con 0x7fc2680cb5f0 2023-10-06T10:45:29.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.150+0000 7fc26d870700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2680b3130 con 0x7fc26815a1e0 2023-10-06T10:45:29.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.150+0000 7fc26d870700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc268156b70 con 0x7fc2680696e0 2023-10-06T10:45:29.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.150+0000 7fc25e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2680cb5f0 0x7fc268152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:29.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.150+0000 7fc25e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2680cb5f0 0x7fc268152350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43344/0 (socket says 172.21.15.92:43344) 2023-10-06T10:45:29.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.150+0000 7fc25e7fc700 1 -- 172.21.15.92:0/3461934048 learned_addr learned my addr 172.21.15.92:0/3461934048 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:29.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.150+0000 7fc266ffd700 1 --2- 172.21.15.92:0/3461934048 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2680696e0 0x7fc2680b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:29.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.150+0000 7fc2677fe700 1 --2- 172.21.15.92:0/3461934048 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc26815a1e0 0x7fc2680b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:29.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.150+0000 7fc25e7fc700 1 -- 172.21.15.92:0/3461934048 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2680696e0 msgr2=0x7fc2680b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:29.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.150+0000 7fc25e7fc700 1 --2- 172.21.15.92:0/3461934048 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2680696e0 0x7fc2680b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:29.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.150+0000 7fc25e7fc700 1 -- 172.21.15.92:0/3461934048 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc26815a1e0 msgr2=0x7fc2680b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:29.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.150+0000 7fc25e7fc700 1 --2- 172.21.15.92:0/3461934048 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc26815a1e0 0x7fc2680b35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:29.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.150+0000 7fc25e7fc700 1 -- 172.21.15.92:0/3461934048 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc260009ae0 con 0x7fc2680cb5f0 2023-10-06T10:45:29.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.150+0000 7fc2677fe700 1 --2- 172.21.15.92:0/3461934048 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc26815a1e0 0x7fc2680b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:29.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.151+0000 7fc25e7fc700 1 --2- 172.21.15.92:0/3461934048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2680cb5f0 0x7fc268152350 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fc25000c5e0 tx=0x7fc250001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:29.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.151+0000 7fc25f7fe700 1 -- 172.21.15.92:0/3461934048 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc250005a70 con 0x7fc2680cb5f0 2023-10-06T10:45:29.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.151+0000 7fc25f7fe700 1 -- 172.21.15.92:0/3461934048 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc250005bd0 con 0x7fc2680cb5f0 2023-10-06T10:45:29.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.151+0000 7fc26d870700 1 -- 172.21.15.92:0/3461934048 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc258009420 con 0x7fc2680cb5f0 2023-10-06T10:45:29.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.151+0000 7fc25f7fe700 1 -- 172.21.15.92:0/3461934048 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2500070d0 con 0x7fc2680cb5f0 2023-10-06T10:45:29.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.151+0000 7fc26d870700 1 -- 172.21.15.92:0/3461934048 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc268152890 con 0x7fc2680cb5f0 2023-10-06T10:45:29.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.152+0000 7fc26d870700 1 -- 172.21.15.92:0/3461934048 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc268064a90 con 0x7fc2680cb5f0 2023-10-06T10:45:29.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.152+0000 7fc25f7fe700 1 -- 172.21.15.92:0/3461934048 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc250007480 con 0x7fc2680cb5f0 2023-10-06T10:45:29.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.152+0000 7fc25f7fe700 1 --2- 172.21.15.92:0/3461934048 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc248066fb0 0x7fc248069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:29.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.152+0000 7fc25f7fe700 1 -- 172.21.15.92:0/3461934048 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fc250083d00 con 0x7fc2680cb5f0 2023-10-06T10:45:29.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.153+0000 7fc266ffd700 1 --2- 172.21.15.92:0/3461934048 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc248066fb0 0x7fc248069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:29.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.153+0000 7fc266ffd700 1 --2- 172.21.15.92:0/3461934048 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc248066fb0 0x7fc248069460 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc2580093f0 tx=0x7fc258017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:29.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.158+0000 7fc25f7fe700 1 -- 172.21.15.92:0/3461934048 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc25004b5a0 con 0x7fc2680cb5f0 2023-10-06T10:45:29.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.380+0000 7fc26d870700 1 -- 172.21.15.92:0/3461934048 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2680cc690 con 0x7fc2680cb5f0 2023-10-06T10:45:29.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.380+0000 7fc25f7fe700 1 -- 172.21.15.92:0/3461934048 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+4643 (secure 0 0 0) 0x7fc250017070 con 0x7fc2680cb5f0 2023-10-06T10:45:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:29.383 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":108,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2016052382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2016052382},{"type":"v1","addr":"172.21.15.124:6834","nonce":2016052382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:44:51.106486+0000","modified":"2023-10-06T10:45:26.429733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8520,"mds_1":8517,"mds_2":6719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6719":{"gid":6719,"name":"d","rank":2,"incarnation":104,"state":"up:active","state_seq":6,"addr":"172.21.15.124:6835/2307521923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2307521923},{"type":"v1","addr":"172.21.15.124:6835","nonce":2307521923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8517":{"gid":8517,"name":"a","rank":1,"incarnation":102,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6836/1979826278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979826278},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979826278}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2298348426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2298348426},{"type":"v1","addr":"172.21.15.92:6837","nonce":2298348426}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T10:45:29.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.382+0000 7fc26d870700 1 -- 172.21.15.92:0/3461934048 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc248066fb0 msgr2=0x7fc248069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:29.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.382+0000 7fc26d870700 1 --2- 172.21.15.92:0/3461934048 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc248066fb0 0x7fc248069460 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc2580093f0 tx=0x7fc258017000 comp rx=0 tx=0).stop 2023-10-06T10:45:29.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.382+0000 7fc26d870700 1 -- 172.21.15.92:0/3461934048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2680cb5f0 msgr2=0x7fc268152350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:29.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.382+0000 7fc26d870700 1 --2- 172.21.15.92:0/3461934048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2680cb5f0 0x7fc268152350 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fc25000c5e0 tx=0x7fc250001040 comp rx=0 tx=0).stop 2023-10-06T10:45:29.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.383+0000 7fc26d870700 1 -- 172.21.15.92:0/3461934048 shutdown_connections 2023-10-06T10:45:29.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.383+0000 7fc26d870700 1 --2- 172.21.15.92:0/3461934048 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc248066fb0 0x7fc248069460 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:29.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.383+0000 7fc26d870700 1 --2- 172.21.15.92:0/3461934048 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc26815a1e0 0x7fc2680b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:29.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.383+0000 7fc26d870700 1 --2- 172.21.15.92:0/3461934048 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2680696e0 0x7fc2680b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:29.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.383+0000 7fc26d870700 1 --2- 172.21.15.92:0/3461934048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2680cb5f0 0x7fc268152350 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:29.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.383+0000 7fc26d870700 1 -- 172.21.15.92:0/3461934048 >> 172.21.15.92:0/3461934048 conn(0x7fc2680707d0 msgr2=0x7fc2680ce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.383+0000 7fc26d870700 1 -- 172.21.15.92:0/3461934048 shutdown_connections 2023-10-06T10:45:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.383+0000 7fc26d870700 1 -- 172.21.15.92:0/3461934048 wait complete. 2023-10-06T10:45:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 108 2023-10-06T10:45:29.396 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2023-10-06T10:45:29.396 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:45:29.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.510+0000 7fb215611700 1 Processor -- start 2023-10-06T10:45:29.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.510+0000 7fb215611700 1 -- start start 2023-10-06T10:45:29.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.510+0000 7fb215611700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb2100b49a0 0x7fb2100aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:29.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.510+0000 7fb215611700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb2100b0760 0x7fb2100b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:29.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.510+0000 7fb215611700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2100b0c60 con 0x7fb210159560 2023-10-06T10:45:29.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.510+0000 7fb215611700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2100b0f40 con 0x7fb2100b0760 2023-10-06T10:45:29.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.510+0000 7fb215611700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2100b1220 con 0x7fb2100b49a0 2023-10-06T10:45:29.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.510+0000 7fb20f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb2100b0760 0x7fb2100b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:29.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.510+0000 7fb20effd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb210159560 0x7fb2100aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34592/0 (socket says 172.21.15.92:34592) 2023-10-06T10:45:29.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.510+0000 7fb20effd700 1 -- 172.21.15.92:0/3489415213 learned_addr learned my addr 172.21.15.92:0/3489415213 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:29.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.510+0000 7fb20f7fe700 1 -- 172.21.15.92:0/3489415213 >> v2:172.21.15.124:3300/0 conn(0x7fb2100b49a0 msgr2=0x7fb2100aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:29.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.510+0000 7fb20f7fe700 1 --2- 172.21.15.92:0/3489415213 >> v2:172.21.15.124:3300/0 conn(0x7fb2100b49a0 0x7fb2100aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:29.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.510+0000 7fb20f7fe700 1 -- 172.21.15.92:0/3489415213 >> v1:172.21.15.92:6789/0 conn(0x7fb210159560 legacy=0x7fb2100aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:29.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.510+0000 7fb20f7fe700 1 -- 172.21.15.92:0/3489415213 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2100b1500 con 0x7fb2100b0760 2023-10-06T10:45:29.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.510+0000 7fb20e7fc700 1 -- 172.21.15.92:0/3489415213 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890741037 0 0) 0x7fb2100b0c60 con 0x7fb210159560 2023-10-06T10:45:29.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.510+0000 7fb20f7fe700 1 --2- 172.21.15.92:0/3489415213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2100b0760 0x7fb2100b1bc0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fb208009e60 tx=0x7fb208009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27be3c39d6038fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:29.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.511+0000 7fb20e7fc700 1 -- 172.21.15.92:0/3489415213 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20800e070 con 0x7fb2100b0760 2023-10-06T10:45:29.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.511+0000 7fb20e7fc700 1 -- 172.21.15.92:0/3489415213 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20800bbf0 con 0x7fb2100b0760 2023-10-06T10:45:29.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.511+0000 7fb20e7fc700 1 -- 172.21.15.92:0/3489415213 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208012430 con 0x7fb2100b0760 2023-10-06T10:45:29.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.511+0000 7fb215611700 1 -- 172.21.15.92:0/3489415213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2100b0760 msgr2=0x7fb2100b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:29.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.511+0000 7fb215611700 1 --2- 172.21.15.92:0/3489415213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2100b0760 0x7fb2100b1bc0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fb208009e60 tx=0x7fb208009780 comp rx=0 tx=0).stop 2023-10-06T10:45:29.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.511+0000 7fb215611700 1 -- 172.21.15.92:0/3489415213 shutdown_connections 2023-10-06T10:45:29.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.511+0000 7fb215611700 1 --2- 172.21.15.92:0/3489415213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2100b0760 0x7fb2100b1bc0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:29.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.511+0000 7fb215611700 1 --2- 172.21.15.92:0/3489415213 >> v2:172.21.15.124:3300/0 conn(0x7fb2100b49a0 0x7fb2100aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:29.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.511+0000 7fb215611700 1 -- 172.21.15.92:0/3489415213 >> 172.21.15.92:0/3489415213 conn(0x7fb2100997f0 msgr2=0x7fb21006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:29.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.511+0000 7fb215611700 1 -- 172.21.15.92:0/3489415213 shutdown_connections 2023-10-06T10:45:29.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.511+0000 7fb215611700 1 -- 172.21.15.92:0/3489415213 wait complete. 2023-10-06T10:45:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.512+0000 7fb215611700 1 Processor -- start 2023-10-06T10:45:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.512+0000 7fb215611700 1 -- start start 2023-10-06T10:45:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.513+0000 7fb215611700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2100b0760 0x7fb2100ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.513+0000 7fb215611700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2100b49a0 0x7fb21015c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.513+0000 7fb215611700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb210159560 0x7fb21015fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.513+0000 7fb215611700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21015cd10 con 0x7fb2100b49a0 2023-10-06T10:45:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.513+0000 7fb215611700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21016db30 con 0x7fb2100b0760 2023-10-06T10:45:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.513+0000 7fb215611700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb21016d200 con 0x7fb210159560 2023-10-06T10:45:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.513+0000 7fb2067fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2100b49a0 0x7fb21015c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.513+0000 7fb2067fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2100b49a0 0x7fb21015c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43354/0 (socket says 172.21.15.92:43354) 2023-10-06T10:45:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.513+0000 7fb2067fc700 1 -- 172.21.15.92:0/2676350719 learned_addr learned my addr 172.21.15.92:0/2676350719 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.513+0000 7fb20effd700 1 --2- 172.21.15.92:0/2676350719 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2100b0760 0x7fb2100ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.513+0000 7fb20f7fe700 1 --2- 172.21.15.92:0/2676350719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb210159560 0x7fb21015fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.513+0000 7fb2067fc700 1 -- 172.21.15.92:0/2676350719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb210159560 msgr2=0x7fb21015fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.513+0000 7fb2067fc700 1 --2- 172.21.15.92:0/2676350719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb210159560 0x7fb21015fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.513+0000 7fb2067fc700 1 -- 172.21.15.92:0/2676350719 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2100b0760 msgr2=0x7fb2100ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.513+0000 7fb2067fc700 1 --2- 172.21.15.92:0/2676350719 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2100b0760 0x7fb2100ad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.513+0000 7fb2067fc700 1 -- 172.21.15.92:0/2676350719 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb208009a20 con 0x7fb2100b49a0 2023-10-06T10:45:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.514+0000 7fb2067fc700 1 --2- 172.21.15.92:0/2676350719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2100b49a0 0x7fb21015c430 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800b810 tx=0x7fb1f8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.514+0000 7fb2077fe700 1 -- 172.21.15.92:0/2676350719 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f800f510 con 0x7fb2100b49a0 2023-10-06T10:45:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.514+0000 7fb2077fe700 1 -- 172.21.15.92:0/2676350719 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f800f690 con 0x7fb2100b49a0 2023-10-06T10:45:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.514+0000 7fb215611700 1 -- 172.21.15.92:0/2676350719 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb21016d4e0 con 0x7fb2100b49a0 2023-10-06T10:45:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.514+0000 7fb2077fe700 1 -- 172.21.15.92:0/2676350719 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f800fa50 con 0x7fb2100b49a0 2023-10-06T10:45:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.514+0000 7fb215611700 1 -- 172.21.15.92:0/2676350719 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb21015c970 con 0x7fb2100b49a0 2023-10-06T10:45:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.515+0000 7fb215611700 1 -- 172.21.15.92:0/2676350719 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb21016dec0 con 0x7fb2100b49a0 2023-10-06T10:45:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.515+0000 7fb2077fe700 1 -- 172.21.15.92:0/2676350719 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb1f801a400 con 0x7fb2100b49a0 2023-10-06T10:45:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.515+0000 7fb2077fe700 1 --2- 172.21.15.92:0/2676350719 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb1f0066f30 0x7fb1f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:29.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.516+0000 7fb2077fe700 1 -- 172.21.15.92:0/2676350719 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fb1f8084330 con 0x7fb2100b49a0 2023-10-06T10:45:29.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.516+0000 7fb20effd700 1 --2- 172.21.15.92:0/2676350719 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb1f0066f30 0x7fb1f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:29.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.516+0000 7fb20effd700 1 --2- 172.21.15.92:0/2676350719 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb1f0066f30 0x7fb1f00693e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb200002970 tx=0x7fb200009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:29.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.520+0000 7fb2077fe700 1 -- 172.21.15.92:0/2676350719 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb21016dec0 con 0x7fb2100b49a0 2023-10-06T10:45:29.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.780+0000 7fb215611700 1 -- 172.21.15.92:0/2676350719 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb21016dec0 con 0x7fb2100b49a0 2023-10-06T10:45:29.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.780+0000 7fb2077fe700 1 -- 172.21.15.92:0/2676350719 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1602 (secure 0 0 0) 0x7fb21016dec0 con 0x7fb2100b49a0 2023-10-06T10:45:29.781 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:29.781 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":451,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":698400,"bytes_used":2455613440,"bytes_avail":770638499840,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":9554,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":108,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6719},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":8517},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8520}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2023-10-06T10:44:44.166325+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T10:45:29.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.782+0000 7fb215611700 1 -- 172.21.15.92:0/2676350719 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb1f0066f30 msgr2=0x7fb1f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:29.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.782+0000 7fb215611700 1 --2- 172.21.15.92:0/2676350719 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb1f0066f30 0x7fb1f00693e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb200002970 tx=0x7fb200009040 comp rx=0 tx=0).stop 2023-10-06T10:45:29.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.782+0000 7fb215611700 1 -- 172.21.15.92:0/2676350719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2100b49a0 msgr2=0x7fb21015c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:29.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.782+0000 7fb215611700 1 --2- 172.21.15.92:0/2676350719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2100b49a0 0x7fb21015c430 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800b810 tx=0x7fb1f8009220 comp rx=0 tx=0).stop 2023-10-06T10:45:29.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.782+0000 7fb215611700 1 -- 172.21.15.92:0/2676350719 shutdown_connections 2023-10-06T10:45:29.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.782+0000 7fb215611700 1 --2- 172.21.15.92:0/2676350719 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb1f0066f30 0x7fb1f00693e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:29.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.782+0000 7fb215611700 1 --2- 172.21.15.92:0/2676350719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb210159560 0x7fb21015fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:29.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.782+0000 7fb215611700 1 --2- 172.21.15.92:0/2676350719 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2100b0760 0x7fb2100ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:29.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.782+0000 7fb215611700 1 --2- 172.21.15.92:0/2676350719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2100b49a0 0x7fb21015c430 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:29.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.782+0000 7fb215611700 1 -- 172.21.15.92:0/2676350719 >> 172.21.15.92:0/2676350719 conn(0x7fb2100997f0 msgr2=0x7fb2100683c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:29.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.783+0000 7fb215611700 1 -- 172.21.15.92:0/2676350719 shutdown_connections 2023-10-06T10:45:29.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:29.783+0000 7fb215611700 1 -- 172.21.15.92:0/2676350719 wait complete. 2023-10-06T10:45:29.795 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2023-10-06T10:45:30.796 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:45:30.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.915+0000 7f974a2e2700 1 Processor -- start 2023-10-06T10:45:30.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.915+0000 7f974a2e2700 1 -- start start 2023-10-06T10:45:30.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.916+0000 7f974a2e2700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9744159440 0x7f97440b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:30.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.916+0000 7f974a2e2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97440cb7a0 con 0x7f97440b2100 2023-10-06T10:45:30.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.916+0000 7f974a2e2700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97440b0860 con 0x7f97440b35d0 2023-10-06T10:45:30.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.916+0000 7f974a2e2700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97440cb150 con 0x7f9744159440 2023-10-06T10:45:30.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.916+0000 7f9749ae1700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f97440b2100 0x7f97440b2580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34596/0 (socket says 172.21.15.92:34596) 2023-10-06T10:45:30.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.916+0000 7f9749ae1700 1 -- 172.21.15.92:0/2069135567 learned_addr learned my addr 172.21.15.92:0/2069135567 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:30.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.916+0000 7f97492e0700 1 --2- 172.21.15.92:0/2069135567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9744159440 0x7f97440b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:30.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.916+0000 7f973bfff700 1 -- 172.21.15.92:0/2069135567 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 986357985 0 0) 0x7f97440cb7a0 con 0x7f97440b2100 2023-10-06T10:45:30.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.916+0000 7f973bfff700 1 -- 172.21.15.92:0/2069135567 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9730003570 con 0x7f97440b2100 2023-10-06T10:45:30.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.916+0000 7f973bfff700 1 -- 172.21.15.92:0/2069135567 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 828604015 0 0) 0x7f9730003570 con 0x7f97440b2100 2023-10-06T10:45:30.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.916+0000 7f973bfff700 1 -- 172.21.15.92:0/2069135567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9744159440 msgr2=0x7f97440b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:30.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.916+0000 7f973bfff700 1 --2- 172.21.15.92:0/2069135567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9744159440 0x7f97440b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:30.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.916+0000 7f973bfff700 1 -- 172.21.15.92:0/2069135567 >> v1:172.21.15.124:6789/0 conn(0x7f97440b35d0 legacy=0x7f97440cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:30.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.916+0000 7f973bfff700 1 -- 172.21.15.92:0/2069135567 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97440cb430 con 0x7f97440b2100 2023-10-06T10:45:30.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.916+0000 7f973bfff700 1 -- 172.21.15.92:0/2069135567 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f97400017f0 con 0x7f97440b2100 2023-10-06T10:45:30.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.916+0000 7f973bfff700 1 -- 172.21.15.92:0/2069135567 >> v1:172.21.15.92:6789/0 conn(0x7f97440b2100 legacy=0x7f97440b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:30.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.917+0000 7f973bfff700 1 --2- 172.21.15.92:0/2069135567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9730003a90 0x7f9730003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:30.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.917+0000 7f973bfff700 1 --2- 172.21.15.92:0/2069135567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9744159440 0x7f9730004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:30.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.917+0000 7f973bfff700 1 --2- 172.21.15.92:0/2069135567 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9730004d50 0x7f97300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:30.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.917+0000 7f973bfff700 1 -- 172.21.15.92:0/2069135567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97440cb150 con 0x7f9730003a90 2023-10-06T10:45:30.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.917+0000 7f973bfff700 1 -- 172.21.15.92:0/2069135567 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973000bb70 con 0x7f9730004d50 2023-10-06T10:45:30.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.917+0000 7f973bfff700 1 -- 172.21.15.92:0/2069135567 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973000bd30 con 0x7f9744159440 2023-10-06T10:45:30.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.917+0000 7f97492e0700 1 --2- 172.21.15.92:0/2069135567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9730003a90 0x7f9730003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:30.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.917+0000 7f9748adf700 1 --2- 172.21.15.92:0/2069135567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9744159440 0x7f9730004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:30.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.917+0000 7f9749ae1700 1 --2- 172.21.15.92:0/2069135567 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9730004d50 0x7f97300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:30.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.917+0000 7f97492e0700 1 -- 172.21.15.92:0/2069135567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9744159440 msgr2=0x7f9730004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:30.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.917+0000 7f97492e0700 1 --2- 172.21.15.92:0/2069135567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9744159440 0x7f9730004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:30.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.917+0000 7f97492e0700 1 -- 172.21.15.92:0/2069135567 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9730004d50 msgr2=0x7f97300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:30.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.917+0000 7f97492e0700 1 --2- 172.21.15.92:0/2069135567 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9730004d50 0x7f97300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:30.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.917+0000 7f97492e0700 1 -- 172.21.15.92:0/2069135567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9730003570 con 0x7f9730003a90 2023-10-06T10:45:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.917+0000 7f97492e0700 1 --2- 172.21.15.92:0/2069135567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9730003a90 0x7f9730003e60 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f973c009c00 tx=0x7f973c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddab06f798045f8e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.917+0000 7f973bfff700 1 -- 172.21.15.92:0/2069135567 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973c00e710 con 0x7f9730003a90 2023-10-06T10:45:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.917+0000 7f973bfff700 1 -- 172.21.15.92:0/2069135567 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973c00e870 con 0x7f9730003a90 2023-10-06T10:45:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.917+0000 7f973bfff700 1 -- 172.21.15.92:0/2069135567 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973c00ec30 con 0x7f9730003a90 2023-10-06T10:45:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.918+0000 7f974a2e2700 1 -- 172.21.15.92:0/2069135567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9730003a90 msgr2=0x7f9730003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.918+0000 7f974a2e2700 1 --2- 172.21.15.92:0/2069135567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9730003a90 0x7f9730003e60 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f973c009c00 tx=0x7f973c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.918+0000 7f9749ae1700 1 -- 172.21.15.92:0/2069135567 reap_dead start 2023-10-06T10:45:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.918+0000 7f974a2e2700 1 -- 172.21.15.92:0/2069135567 shutdown_connections 2023-10-06T10:45:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.918+0000 7f974a2e2700 1 -- 172.21.15.92:0/2069135567 >> 172.21.15.92:0/2069135567 conn(0x7f974406ece0 msgr2=0x7f974406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.918+0000 7f974a2e2700 1 -- 172.21.15.92:0/2069135567 shutdown_connections 2023-10-06T10:45:30.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.918+0000 7f974a2e2700 1 -- 172.21.15.92:0/2069135567 wait complete. 2023-10-06T10:45:30.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.919+0000 7f974a2e2700 1 Processor -- start 2023-10-06T10:45:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.919+0000 7f974a2e2700 1 -- start start 2023-10-06T10:45:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.920+0000 7f974a2e2700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9730003a90 0x7f97441698f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.920+0000 7f974a2e2700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f974415d160 0x7f9744168800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.920+0000 7f974a2e2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97441575b0 0x7f9744157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.920+0000 7f974a2e2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974416a0a0 con 0x7f97441575b0 2023-10-06T10:45:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.920+0000 7f974a2e2700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9744157f60 con 0x7f9730003a90 2023-10-06T10:45:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.920+0000 7f974a2e2700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9744169e30 con 0x7f974415d160 2023-10-06T10:45:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.920+0000 7f9749ae1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97441575b0 0x7f9744157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.920+0000 7f9749ae1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97441575b0 0x7f9744157a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43366/0 (socket says 172.21.15.92:43366) 2023-10-06T10:45:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.920+0000 7f9749ae1700 1 -- 172.21.15.92:0/3176129404 learned_addr learned my addr 172.21.15.92:0/3176129404 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.920+0000 7f9749ae1700 1 -- 172.21.15.92:0/3176129404 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f974415d160 msgr2=0x7f9744168800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.920+0000 7f9749ae1700 1 --2- 172.21.15.92:0/3176129404 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f974415d160 0x7f9744168800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.920+0000 7f9749ae1700 1 -- 172.21.15.92:0/3176129404 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9730003a90 msgr2=0x7f97441698f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.920+0000 7f9749ae1700 1 --2- 172.21.15.92:0/3176129404 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9730003a90 0x7f97441698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.920+0000 7f9749ae1700 1 -- 172.21.15.92:0/3176129404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9744168d40 con 0x7f97441575b0 2023-10-06T10:45:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.920+0000 7f9749ae1700 1 --2- 172.21.15.92:0/3176129404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97441575b0 0x7f9744157a20 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f974406f9f0 tx=0x7f97400021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.920+0000 7f97397fa700 1 -- 172.21.15.92:0/3176129404 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974000ad60 con 0x7f97441575b0 2023-10-06T10:45:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.920+0000 7f97397fa700 1 -- 172.21.15.92:0/3176129404 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97400133f0 con 0x7f97441575b0 2023-10-06T10:45:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.920+0000 7f974a2e2700 1 -- 172.21.15.92:0/3176129404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974415d720 con 0x7f97441575b0 2023-10-06T10:45:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.920+0000 7f97397fa700 1 -- 172.21.15.92:0/3176129404 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97400137b0 con 0x7f97441575b0 2023-10-06T10:45:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.921+0000 7f974a2e2700 1 -- 172.21.15.92:0/3176129404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974415dce0 con 0x7f97441575b0 2023-10-06T10:45:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.922+0000 7f974a2e2700 1 -- 172.21.15.92:0/3176129404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f974405f230 con 0x7f97441575b0 2023-10-06T10:45:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.922+0000 7f97397fa700 1 -- 172.21.15.92:0/3176129404 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9740013ba0 con 0x7f97441575b0 2023-10-06T10:45:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.922+0000 7f97397fa700 1 --2- 172.21.15.92:0/3176129404 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9728066f30 0x7f97280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:30.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.922+0000 7f97397fa700 1 -- 172.21.15.92:0/3176129404 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f9740085430 con 0x7f97441575b0 2023-10-06T10:45:30.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.922+0000 7f97492e0700 1 --2- 172.21.15.92:0/3176129404 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9728066f30 0x7f97280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:30.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.923+0000 7f97492e0700 1 --2- 172.21.15.92:0/3176129404 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9728066f30 0x7f97280693e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f974416c430 tx=0x7f973c01a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:30.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:30.926+0000 7f97397fa700 1 -- 172.21.15.92:0/3176129404 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97400523c0 con 0x7f97441575b0 2023-10-06T10:45:31.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.152+0000 7f974a2e2700 1 -- 172.21.15.92:0/3176129404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9744159810 con 0x7f97441575b0 2023-10-06T10:45:31.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.153+0000 7f97397fa700 1 -- 172.21.15.92:0/3176129404 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+4643 (secure 0 0 0) 0x7f97400559e0 con 0x7f97441575b0 2023-10-06T10:45:31.154 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:31.154 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":108,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2016052382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2016052382},{"type":"v1","addr":"172.21.15.124:6834","nonce":2016052382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:44:51.106486+0000","modified":"2023-10-06T10:45:26.429733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8520,"mds_1":8517,"mds_2":6719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6719":{"gid":6719,"name":"d","rank":2,"incarnation":104,"state":"up:active","state_seq":6,"addr":"172.21.15.124:6835/2307521923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2307521923},{"type":"v1","addr":"172.21.15.124:6835","nonce":2307521923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8517":{"gid":8517,"name":"a","rank":1,"incarnation":102,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6836/1979826278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979826278},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979826278}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2298348426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2298348426},{"type":"v1","addr":"172.21.15.92:6837","nonce":2298348426}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T10:45:31.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.155+0000 7f974a2e2700 1 -- 172.21.15.92:0/3176129404 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9728066f30 msgr2=0x7f97280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:31.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.155+0000 7f974a2e2700 1 --2- 172.21.15.92:0/3176129404 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9728066f30 0x7f97280693e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f974416c430 tx=0x7f973c01a000 comp rx=0 tx=0).stop 2023-10-06T10:45:31.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.155+0000 7f974a2e2700 1 -- 172.21.15.92:0/3176129404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97441575b0 msgr2=0x7f9744157a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:31.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.155+0000 7f974a2e2700 1 --2- 172.21.15.92:0/3176129404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97441575b0 0x7f9744157a20 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f974406f9f0 tx=0x7f97400021f0 comp rx=0 tx=0).stop 2023-10-06T10:45:31.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.155+0000 7f974a2e2700 1 -- 172.21.15.92:0/3176129404 shutdown_connections 2023-10-06T10:45:31.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.155+0000 7f974a2e2700 1 --2- 172.21.15.92:0/3176129404 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9728066f30 0x7f97280693e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:31.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.155+0000 7f974a2e2700 1 --2- 172.21.15.92:0/3176129404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97441575b0 0x7f9744157a20 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:31.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.155+0000 7f974a2e2700 1 --2- 172.21.15.92:0/3176129404 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9730003a90 0x7f97441698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:31.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.155+0000 7f974a2e2700 1 --2- 172.21.15.92:0/3176129404 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f974415d160 0x7f9744168800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:31.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.155+0000 7f974a2e2700 1 -- 172.21.15.92:0/3176129404 >> 172.21.15.92:0/3176129404 conn(0x7f974406ece0 msgr2=0x7f9744155590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:31.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.155+0000 7f974a2e2700 1 -- 172.21.15.92:0/3176129404 shutdown_connections 2023-10-06T10:45:31.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.155+0000 7f974a2e2700 1 -- 172.21.15.92:0/3176129404 wait complete. 2023-10-06T10:45:31.157 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 108 2023-10-06T10:45:31.168 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2023-10-06T10:45:31.168 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:45:31.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.280+0000 7fb397fff700 1 Processor -- start 2023-10-06T10:45:31.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.280+0000 7fb397fff700 1 -- start start 2023-10-06T10:45:31.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.281+0000 7fb397fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb398159440 0x7fb3980b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:31.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.281+0000 7fb397fff700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb3980b2100 0x7fb3980b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:31.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.281+0000 7fb397fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3980cb7a0 con 0x7fb3980b35d0 2023-10-06T10:45:31.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.281+0000 7fb397fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3980b0860 con 0x7fb398159440 2023-10-06T10:45:31.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.281+0000 7fb397fff700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3980cb150 con 0x7fb3980b2100 2023-10-06T10:45:31.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.281+0000 7fb396ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb398159440 0x7fb3980b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:31.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.281+0000 7fb38e7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb3980b35d0 0x7fb3980cbc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34600/0 (socket says 172.21.15.92:34600) 2023-10-06T10:45:31.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.281+0000 7fb38e7fc700 1 -- 172.21.15.92:0/1187593930 learned_addr learned my addr 172.21.15.92:0/1187593930 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:31.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.281+0000 7fb3977fe700 1 --2- 172.21.15.92:0/1187593930 >> v2:172.21.15.124:3300/0 conn(0x7fb3980b2100 0x7fb3980b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:31.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.282+0000 7fb396ffd700 1 -- 172.21.15.92:0/1187593930 >> v2:172.21.15.124:3300/0 conn(0x7fb3980b2100 msgr2=0x7fb3980b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:31.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.282+0000 7fb396ffd700 1 --2- 172.21.15.92:0/1187593930 >> v2:172.21.15.124:3300/0 conn(0x7fb3980b2100 0x7fb3980b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:31.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.282+0000 7fb396ffd700 1 -- 172.21.15.92:0/1187593930 >> v1:172.21.15.92:6789/0 conn(0x7fb3980b35d0 legacy=0x7fb3980cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:31.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.282+0000 7fb396ffd700 1 -- 172.21.15.92:0/1187593930 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3980cb430 con 0x7fb398159440 2023-10-06T10:45:31.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.282+0000 7fb3967fc700 1 -- 172.21.15.92:0/1187593930 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 222582668 0 0) 0x7fb3980cb7a0 con 0x7fb3980b35d0 2023-10-06T10:45:31.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.282+0000 7fb396ffd700 1 --2- 172.21.15.92:0/1187593930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb398159440 0x7fb3980b3090 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fb3980ca7f0 tx=0x7fb3880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8000697036151937 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:31.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.282+0000 7fb3967fc700 1 -- 172.21.15.92:0/1187593930 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38800b6b0 con 0x7fb398159440 2023-10-06T10:45:31.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.282+0000 7fb3967fc700 1 -- 172.21.15.92:0/1187593930 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38800b810 con 0x7fb398159440 2023-10-06T10:45:31.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.282+0000 7fb3967fc700 1 -- 172.21.15.92:0/1187593930 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38800bbb0 con 0x7fb398159440 2023-10-06T10:45:31.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.282+0000 7fb397fff700 1 -- 172.21.15.92:0/1187593930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb398159440 msgr2=0x7fb3980b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:31.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.282+0000 7fb397fff700 1 --2- 172.21.15.92:0/1187593930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb398159440 0x7fb3980b3090 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fb3980ca7f0 tx=0x7fb3880091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:31.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.282+0000 7fb397fff700 1 -- 172.21.15.92:0/1187593930 shutdown_connections 2023-10-06T10:45:31.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.282+0000 7fb397fff700 1 --2- 172.21.15.92:0/1187593930 >> v2:172.21.15.124:3300/0 conn(0x7fb3980b2100 0x7fb3980b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:31.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.282+0000 7fb397fff700 1 --2- 172.21.15.92:0/1187593930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb398159440 0x7fb3980b3090 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:31.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.282+0000 7fb397fff700 1 -- 172.21.15.92:0/1187593930 >> 172.21.15.92:0/1187593930 conn(0x7fb39806ece0 msgr2=0x7fb39806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:31.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.282+0000 7fb397fff700 1 -- 172.21.15.92:0/1187593930 shutdown_connections 2023-10-06T10:45:31.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.283+0000 7fb397fff700 1 -- 172.21.15.92:0/1187593930 wait complete. 2023-10-06T10:45:31.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.284+0000 7fb397fff700 1 Processor -- start 2023-10-06T10:45:31.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.284+0000 7fb397fff700 1 -- start start 2023-10-06T10:45:31.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.284+0000 7fb397fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3980b2100 0x7fb3980c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:31.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.284+0000 7fb397fff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3980b35d0 0x7fb3980cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:31.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.284+0000 7fb397fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb398159440 0x7fb3980cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:31.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.284+0000 7fb397fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb398168f10 con 0x7fb3980b2100 2023-10-06T10:45:31.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.284+0000 7fb397fff700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb398169d30 con 0x7fb3980b35d0 2023-10-06T10:45:31.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.284+0000 7fb397fff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb398169120 con 0x7fb398159440 2023-10-06T10:45:31.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.285+0000 7fb396ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3980b2100 0x7fb3980c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:31.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.285+0000 7fb396ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3980b2100 0x7fb3980c6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43382/0 (socket says 172.21.15.92:43382) 2023-10-06T10:45:31.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.285+0000 7fb396ffd700 1 -- 172.21.15.92:0/2870088545 learned_addr learned my addr 172.21.15.92:0/2870088545 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:31.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.285+0000 7fb38e7fc700 1 --2- 172.21.15.92:0/2870088545 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3980b35d0 0x7fb3980cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:31.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.285+0000 7fb3977fe700 1 --2- 172.21.15.92:0/2870088545 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb398159440 0x7fb3980cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:31.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.285+0000 7fb396ffd700 1 -- 172.21.15.92:0/2870088545 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb398159440 msgr2=0x7fb3980cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:31.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.285+0000 7fb396ffd700 1 --2- 172.21.15.92:0/2870088545 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb398159440 0x7fb3980cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:31.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.285+0000 7fb396ffd700 1 -- 172.21.15.92:0/2870088545 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3980b35d0 msgr2=0x7fb3980cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:31.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.285+0000 7fb396ffd700 1 --2- 172.21.15.92:0/2870088545 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3980b35d0 0x7fb3980cf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:31.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.285+0000 7fb396ffd700 1 -- 172.21.15.92:0/2870088545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb388009420 con 0x7fb3980b2100 2023-10-06T10:45:31.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.285+0000 7fb3977fe700 1 --2- 172.21.15.92:0/2870088545 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb398159440 0x7fb3980cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:31.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.285+0000 7fb396ffd700 1 --2- 172.21.15.92:0/2870088545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3980b2100 0x7fb3980c6240 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fb398069150 tx=0x7fb388017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:31.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.285+0000 7fb38f7fe700 1 -- 172.21.15.92:0/2870088545 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38800bb90 con 0x7fb3980b2100 2023-10-06T10:45:31.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.285+0000 7fb38f7fe700 1 -- 172.21.15.92:0/2870088545 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb388011420 con 0x7fb3980b2100 2023-10-06T10:45:31.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.285+0000 7fb397fff700 1 -- 172.21.15.92:0/2870088545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb398169400 con 0x7fb3980b2100 2023-10-06T10:45:31.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.285+0000 7fb38f7fe700 1 -- 172.21.15.92:0/2870088545 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3880117c0 con 0x7fb3980b2100 2023-10-06T10:45:31.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.285+0000 7fb397fff700 1 -- 172.21.15.92:0/2870088545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3981699c0 con 0x7fb3980b2100 2023-10-06T10:45:31.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.286+0000 7fb397fff700 1 -- 172.21.15.92:0/2870088545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb39805f230 con 0x7fb3980b2100 2023-10-06T10:45:31.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.287+0000 7fb38f7fe700 1 -- 172.21.15.92:0/2870088545 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb388011bf0 con 0x7fb3980b2100 2023-10-06T10:45:31.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.287+0000 7fb38f7fe700 1 --2- 172.21.15.92:0/2870088545 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb378066f30 0x7fb3780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:31.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.287+0000 7fb38e7fc700 1 --2- 172.21.15.92:0/2870088545 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb378066f30 0x7fb3780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:31.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.287+0000 7fb38f7fe700 1 -- 172.21.15.92:0/2870088545 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fb388086080 con 0x7fb3980b2100 2023-10-06T10:45:31.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.287+0000 7fb38e7fc700 1 --2- 172.21.15.92:0/2870088545 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb378066f30 0x7fb3780693e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb3980c61e0 tx=0x7fb380007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:31.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.291+0000 7fb38f7fe700 1 -- 172.21.15.92:0/2870088545 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb388053010 con 0x7fb3980b2100 2023-10-06T10:45:31.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.551+0000 7fb397fff700 1 -- 172.21.15.92:0/2870088545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb398159810 con 0x7fb3980b2100 2023-10-06T10:45:31.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.552+0000 7fb38f7fe700 1 -- 172.21.15.92:0/2870088545 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1602 (secure 0 0 0) 0x7fb388056630 con 0x7fb3980b2100 2023-10-06T10:45:31.553 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:31.553 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":453,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":698974,"bytes_used":2455638016,"bytes_avail":770638475264,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":9811,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":108,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6719},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":8517},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8520}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2023-10-06T10:44:44.166325+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T10:45:31.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.554+0000 7fb397fff700 1 -- 172.21.15.92:0/2870088545 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb378066f30 msgr2=0x7fb3780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:31.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.554+0000 7fb397fff700 1 --2- 172.21.15.92:0/2870088545 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb378066f30 0x7fb3780693e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb3980c61e0 tx=0x7fb380007d30 comp rx=0 tx=0).stop 2023-10-06T10:45:31.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.554+0000 7fb397fff700 1 -- 172.21.15.92:0/2870088545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3980b2100 msgr2=0x7fb3980c6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:31.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.554+0000 7fb397fff700 1 --2- 172.21.15.92:0/2870088545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3980b2100 0x7fb3980c6240 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fb398069150 tx=0x7fb388017000 comp rx=0 tx=0).stop 2023-10-06T10:45:31.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.554+0000 7fb397fff700 1 -- 172.21.15.92:0/2870088545 shutdown_connections 2023-10-06T10:45:31.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.554+0000 7fb397fff700 1 --2- 172.21.15.92:0/2870088545 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb378066f30 0x7fb3780693e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:31.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.554+0000 7fb397fff700 1 --2- 172.21.15.92:0/2870088545 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb398159440 0x7fb3980cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:31.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.554+0000 7fb397fff700 1 --2- 172.21.15.92:0/2870088545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3980b2100 0x7fb3980c6240 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:31.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.554+0000 7fb397fff700 1 --2- 172.21.15.92:0/2870088545 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3980b35d0 0x7fb3980cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:31.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.554+0000 7fb397fff700 1 -- 172.21.15.92:0/2870088545 >> 172.21.15.92:0/2870088545 conn(0x7fb39806ece0 msgr2=0x7fb3981573f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:31.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.554+0000 7fb397fff700 1 -- 172.21.15.92:0/2870088545 shutdown_connections 2023-10-06T10:45:31.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:31.554+0000 7fb397fff700 1 -- 172.21.15.92:0/2870088545 wait complete. 2023-10-06T10:45:31.566 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2023-10-06T10:45:32.445 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T10:45:32.444+0000 7f9e81bcf700 -1 log_channel(cluster) log [ERR] : Health check failed: 2 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T10:45:32.567 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:45:32.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.684+0000 7f4bda20f700 1 Processor -- start 2023-10-06T10:45:32.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.684+0000 7f4bda20f700 1 -- start start 2023-10-06T10:45:32.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.684+0000 7f4bda20f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd4159ad0 0x7f4bd4159f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:32.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.684+0000 7f4bda20f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd4068d10 con 0x7f4bd40cb5f0 2023-10-06T10:45:32.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.684+0000 7f4bda20f700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd415a670 con 0x7f4bd40ca670 2023-10-06T10:45:32.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.684+0000 7f4bda20f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd4154020 con 0x7f4bd4159ad0 2023-10-06T10:45:32.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.685+0000 7f4bd37fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4bd40cb5f0 0x7f4bd40691a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34602/0 (socket says 172.21.15.92:34602) 2023-10-06T10:45:32.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.685+0000 7f4bd37fe700 1 -- 172.21.15.92:0/57070669 learned_addr learned my addr 172.21.15.92:0/57070669 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:32.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.685+0000 7f4bd2ffd700 1 --2- 172.21.15.92:0/57070669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd4159ad0 0x7f4bd4159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:32.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.685+0000 7f4bd27fc700 1 -- 172.21.15.92:0/57070669 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1340189932 0 0) 0x7f4bd4068d10 con 0x7f4bd40cb5f0 2023-10-06T10:45:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.685+0000 7f4bd27fc700 1 -- 172.21.15.92:0/57070669 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bbc003570 con 0x7f4bd40cb5f0 2023-10-06T10:45:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.685+0000 7f4bd27fc700 1 -- 172.21.15.92:0/57070669 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626809370 0 0) 0x7f4bd415a670 con 0x7f4bd40ca670 2023-10-06T10:45:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.685+0000 7f4bd27fc700 1 -- 172.21.15.92:0/57070669 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bd4068d10 con 0x7f4bd40ca670 2023-10-06T10:45:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.685+0000 7f4bd2ffd700 1 -- 172.21.15.92:0/57070669 >> v1:172.21.15.124:6789/0 conn(0x7f4bd40ca670 legacy=0x7f4bd40ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.685+0000 7f4bd2ffd700 1 -- 172.21.15.92:0/57070669 >> v1:172.21.15.92:6789/0 conn(0x7f4bd40cb5f0 legacy=0x7f4bd40691a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.685+0000 7f4bd27fc700 1 -- 172.21.15.92:0/57070669 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1004448941 0 0) 0x7f4bbc003570 con 0x7f4bd40cb5f0 2023-10-06T10:45:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.685+0000 7f4bd2ffd700 1 -- 172.21.15.92:0/57070669 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd415a950 con 0x7f4bd4159ad0 2023-10-06T10:45:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.685+0000 7f4bd2ffd700 1 --2- 172.21.15.92:0/57070669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd4159ad0 0x7f4bd4159f20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009910 tx=0x7f4bc80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61d9a8da2a5409a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.685+0000 7f4bd27fc700 1 -- 172.21.15.92:0/57070669 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc40017f0 con 0x7f4bd4159ad0 2023-10-06T10:45:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.686+0000 7f4bd27fc700 1 -- 172.21.15.92:0/57070669 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc8011420 con 0x7f4bd4159ad0 2023-10-06T10:45:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.686+0000 7f4bd27fc700 1 -- 172.21.15.92:0/57070669 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc80117a0 con 0x7f4bd4159ad0 2023-10-06T10:45:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.686+0000 7f4bda20f700 1 -- 172.21.15.92:0/57070669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd4159ad0 msgr2=0x7f4bd4159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:32.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.686+0000 7f4bda20f700 1 --2- 172.21.15.92:0/57070669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd4159ad0 0x7f4bd4159f20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009910 tx=0x7f4bc80091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:32.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.686+0000 7f4bda20f700 1 -- 172.21.15.92:0/57070669 shutdown_connections 2023-10-06T10:45:32.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.686+0000 7f4bda20f700 1 --2- 172.21.15.92:0/57070669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd4159ad0 0x7f4bd4159f20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:32.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.686+0000 7f4bda20f700 1 -- 172.21.15.92:0/57070669 >> 172.21.15.92:0/57070669 conn(0x7f4bd40707d0 msgr2=0x7f4bd4070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:32.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.686+0000 7f4bda20f700 1 -- 172.21.15.92:0/57070669 shutdown_connections 2023-10-06T10:45:32.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.686+0000 7f4bda20f700 1 -- 172.21.15.92:0/57070669 wait complete. 2023-10-06T10:45:32.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.687+0000 7f4bda20f700 1 Processor -- start 2023-10-06T10:45:32.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.687+0000 7f4bda20f700 1 -- start start 2023-10-06T10:45:32.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.688+0000 7f4bda20f700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bd40ca670 0x7f4bd4143590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:32.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.688+0000 7f4bda20f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bd40cb5f0 0x7f4bd40c3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:32.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.688+0000 7f4bda20f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd40c3c60 0x7f4bd40694d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:32.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.688+0000 7f4bda20f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd40c52e0 con 0x7f4bd40cb5f0 2023-10-06T10:45:32.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.688+0000 7f4bda20f700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd40c41c0 con 0x7f4bd40ca670 2023-10-06T10:45:32.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.688+0000 7f4bda20f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd40c44a0 con 0x7f4bd40c3c60 2023-10-06T10:45:32.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.688+0000 7f4bd2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bd40cb5f0 0x7f4bd40c3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:32.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.688+0000 7f4bd2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bd40cb5f0 0x7f4bd40c3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43386/0 (socket says 172.21.15.92:43386) 2023-10-06T10:45:32.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.688+0000 7f4bd2ffd700 1 -- 172.21.15.92:0/3563011636 learned_addr learned my addr 172.21.15.92:0/3563011636 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:32.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.688+0000 7f4bd3fff700 1 --2- 172.21.15.92:0/3563011636 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd40c3c60 0x7f4bd40694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:32.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.688+0000 7f4bd37fe700 1 --2- 172.21.15.92:0/3563011636 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bd40ca670 0x7f4bd4143590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:32.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.688+0000 7f4bd2ffd700 1 -- 172.21.15.92:0/3563011636 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd40c3c60 msgr2=0x7f4bd40694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:32.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.688+0000 7f4bd2ffd700 1 --2- 172.21.15.92:0/3563011636 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd40c3c60 0x7f4bd40694d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:32.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.688+0000 7f4bd2ffd700 1 -- 172.21.15.92:0/3563011636 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bd40ca670 msgr2=0x7f4bd4143590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:32.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.688+0000 7f4bd2ffd700 1 --2- 172.21.15.92:0/3563011636 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bd40ca670 0x7f4bd4143590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:32.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.688+0000 7f4bd2ffd700 1 -- 172.21.15.92:0/3563011636 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc8009420 con 0x7f4bd40cb5f0 2023-10-06T10:45:32.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.688+0000 7f4bd3fff700 1 --2- 172.21.15.92:0/3563011636 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd40c3c60 0x7f4bd40694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:32.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.688+0000 7f4bd37fe700 1 --2- 172.21.15.92:0/3563011636 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bd40ca670 0x7f4bd4143590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:32.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.689+0000 7f4bd2ffd700 1 --2- 172.21.15.92:0/3563011636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bd40cb5f0 0x7f4bd40c3720 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009860 tx=0x7f4bc8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:32.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.689+0000 7f4bbb7fe700 1 -- 172.21.15.92:0/3563011636 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc801cc50 con 0x7f4bd40cb5f0 2023-10-06T10:45:32.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.689+0000 7f4bbb7fe700 1 -- 172.21.15.92:0/3563011636 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc801cdb0 con 0x7f4bd40cb5f0 2023-10-06T10:45:32.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.689+0000 7f4bbb7fe700 1 -- 172.21.15.92:0/3563011636 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc801a6a0 con 0x7f4bd40cb5f0 2023-10-06T10:45:32.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.689+0000 7f4bda20f700 1 -- 172.21.15.92:0/3563011636 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd40c4780 con 0x7f4bd40cb5f0 2023-10-06T10:45:32.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.689+0000 7f4bda20f700 1 -- 172.21.15.92:0/3563011636 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd40c8850 con 0x7f4bd40cb5f0 2023-10-06T10:45:32.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.690+0000 7f4bda20f700 1 -- 172.21.15.92:0/3563011636 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd40c5450 con 0x7f4bd40cb5f0 2023-10-06T10:45:32.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.690+0000 7f4bbb7fe700 1 -- 172.21.15.92:0/3563011636 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4bc801aad0 con 0x7f4bd40cb5f0 2023-10-06T10:45:32.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.690+0000 7f4bbb7fe700 1 --2- 172.21.15.92:0/3563011636 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4bc0066f30 0x7f4bc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:32.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.690+0000 7f4bbb7fe700 1 -- 172.21.15.92:0/3563011636 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f4bc8086160 con 0x7f4bd40cb5f0 2023-10-06T10:45:32.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.690+0000 7f4bd37fe700 1 --2- 172.21.15.92:0/3563011636 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4bc0066f30 0x7f4bc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:32.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.691+0000 7f4bd37fe700 1 --2- 172.21.15.92:0/3563011636 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4bc0066f30 0x7f4bc00693e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4002200 tx=0x7f4bc4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:32.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.695+0000 7f4bbb7fe700 1 -- 172.21.15.92:0/3563011636 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4bd40c5450 con 0x7f4bd40cb5f0 2023-10-06T10:45:32.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.920+0000 7f4bda20f700 1 -- 172.21.15.92:0/3563011636 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4bd40c5450 con 0x7f4bd40cb5f0 2023-10-06T10:45:32.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.921+0000 7f4bbb7fe700 1 -- 172.21.15.92:0/3563011636 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4645 (secure 0 0 0) 0x7f4bd40c5450 con 0x7f4bd40cb5f0 2023-10-06T10:45:32.922 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:32.922 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":109,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2016052382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2016052382},{"type":"v1","addr":"172.21.15.124:6834","nonce":2016052382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:44:51.106486+0000","modified":"2023-10-06T10:45:32.445190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8520,"mds_1":8517,"mds_2":6719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6719":{"gid":6719,"name":"d","rank":2,"incarnation":104,"state":"up:active","state_seq":6,"addr":"172.21.15.124:6835/2307521923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2307521923},{"type":"v1","addr":"172.21.15.124:6835","nonce":2307521923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8517":{"gid":8517,"name":"a","rank":1,"incarnation":102,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6836/1979826278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979826278},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979826278}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6837/2298348426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2298348426},{"type":"v1","addr":"172.21.15.92:6837","nonce":2298348426}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T10:45:32.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.923+0000 7f4bda20f700 1 -- 172.21.15.92:0/3563011636 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4bc0066f30 msgr2=0x7f4bc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:32.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.923+0000 7f4bda20f700 1 --2- 172.21.15.92:0/3563011636 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4bc0066f30 0x7f4bc00693e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4002200 tx=0x7f4bc4009040 comp rx=0 tx=0).stop 2023-10-06T10:45:32.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.923+0000 7f4bda20f700 1 -- 172.21.15.92:0/3563011636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bd40cb5f0 msgr2=0x7f4bd40c3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:32.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.923+0000 7f4bda20f700 1 --2- 172.21.15.92:0/3563011636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bd40cb5f0 0x7f4bd40c3720 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009860 tx=0x7f4bc8016000 comp rx=0 tx=0).stop 2023-10-06T10:45:32.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.923+0000 7f4bda20f700 1 -- 172.21.15.92:0/3563011636 shutdown_connections 2023-10-06T10:45:32.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.923+0000 7f4bda20f700 1 --2- 172.21.15.92:0/3563011636 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4bc0066f30 0x7f4bc00693e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:32.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.923+0000 7f4bda20f700 1 --2- 172.21.15.92:0/3563011636 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd40c3c60 0x7f4bd40694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:32.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.923+0000 7f4bda20f700 1 --2- 172.21.15.92:0/3563011636 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bd40ca670 0x7f4bd4143590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:32.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.923+0000 7f4bda20f700 1 --2- 172.21.15.92:0/3563011636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bd40cb5f0 0x7f4bd40c3720 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:32.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.923+0000 7f4bda20f700 1 -- 172.21.15.92:0/3563011636 >> 172.21.15.92:0/3563011636 conn(0x7f4bd40707d0 msgr2=0x7f4bd415d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:32.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.923+0000 7f4bda20f700 1 -- 172.21.15.92:0/3563011636 shutdown_connections 2023-10-06T10:45:32.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:32.923+0000 7f4bda20f700 1 -- 172.21.15.92:0/3563011636 wait complete. 2023-10-06T10:45:32.925 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 109 2023-10-06T10:45:32.935 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2023-10-06T10:45:32.936 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:45:33.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.047+0000 7f56e6649700 1 Processor -- start 2023-10-06T10:45:33.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.047+0000 7f56e6649700 1 -- start start 2023-10-06T10:45:33.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.048+0000 7f56e6649700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56e00ca370 0x7f56e0158270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.048+0000 7f56e6649700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e0157de0 con 0x7f56e00adb60 2023-10-06T10:45:33.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.048+0000 7f56e6649700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e00cb5d0 con 0x7f56e00ce8a0 2023-10-06T10:45:33.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.048+0000 7f56e6649700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e00cbb90 con 0x7f56e00ca370 2023-10-06T10:45:33.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.048+0000 7f56df7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f56e00adb60 0x7f56e00ce190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34610/0 (socket says 172.21.15.92:34610) 2023-10-06T10:45:33.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.048+0000 7f56df7fe700 1 -- 172.21.15.92:0/2170115287 learned_addr learned my addr 172.21.15.92:0/2170115287 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:33.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56dffff700 1 --2- 172.21.15.92:0/2170115287 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56e00ca370 0x7f56e0158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56deffd700 1 -- 172.21.15.92:0/2170115287 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1595448727 0 0) 0x7f56e0157de0 con 0x7f56e00adb60 2023-10-06T10:45:33.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56deffd700 1 -- 172.21.15.92:0/2170115287 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56c8003570 con 0x7f56e00adb60 2023-10-06T10:45:33.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56deffd700 1 -- 172.21.15.92:0/2170115287 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 624152576 0 0) 0x7f56c8003570 con 0x7f56e00adb60 2023-10-06T10:45:33.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56deffd700 1 -- 172.21.15.92:0/2170115287 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56e00ca370 msgr2=0x7f56e0158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56deffd700 1 --2- 172.21.15.92:0/2170115287 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56e00ca370 0x7f56e0158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56deffd700 1 -- 172.21.15.92:0/2170115287 >> v1:172.21.15.124:6789/0 conn(0x7f56e00ce8a0 legacy=0x7f56e015aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56deffd700 1 -- 172.21.15.92:0/2170115287 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e00cb8b0 con 0x7f56e00adb60 2023-10-06T10:45:33.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56deffd700 1 -- 172.21.15.92:0/2170115287 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f56d4001470 con 0x7f56e00adb60 2023-10-06T10:45:33.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56deffd700 1 -- 172.21.15.92:0/2170115287 >> v1:172.21.15.92:6789/0 conn(0x7f56e00adb60 legacy=0x7f56e00ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56deffd700 1 --2- 172.21.15.92:0/2170115287 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56c8003a90 0x7f56c8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56deffd700 1 --2- 172.21.15.92:0/2170115287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c8004810 0x7f56c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56deffd700 1 --2- 172.21.15.92:0/2170115287 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56e00ca370 0x7f56c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56deffd700 1 -- 172.21.15.92:0/2170115287 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e00cbb90 con 0x7f56c8004810 2023-10-06T10:45:33.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56deffd700 1 -- 172.21.15.92:0/2170115287 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c800bb10 con 0x7f56c8003a90 2023-10-06T10:45:33.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56deffd700 1 -- 172.21.15.92:0/2170115287 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c800bcd0 con 0x7f56e00ca370 2023-10-06T10:45:33.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56df7fe700 1 --2- 172.21.15.92:0/2170115287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c8004810 0x7f56c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56dffff700 1 --2- 172.21.15.92:0/2170115287 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56c8003a90 0x7f56c8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56e4be6700 1 --2- 172.21.15.92:0/2170115287 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56e00ca370 0x7f56c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56df7fe700 1 -- 172.21.15.92:0/2170115287 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56e00ca370 msgr2=0x7f56c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56df7fe700 1 --2- 172.21.15.92:0/2170115287 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56e00ca370 0x7f56c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56df7fe700 1 -- 172.21.15.92:0/2170115287 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56c8003a90 msgr2=0x7f56c8003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56df7fe700 1 --2- 172.21.15.92:0/2170115287 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56c8003a90 0x7f56c8003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.049+0000 7f56df7fe700 1 -- 172.21.15.92:0/2170115287 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c8003570 con 0x7f56c8004810 2023-10-06T10:45:33.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.050+0000 7f56df7fe700 1 --2- 172.21.15.92:0/2170115287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c8004810 0x7f56c8004c80 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f56d400c3c0 tx=0x7f56d4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae17fc009918e18 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.050+0000 7f56deffd700 1 -- 172.21.15.92:0/2170115287 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d400cc90 con 0x7f56c8004810 2023-10-06T10:45:33.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.050+0000 7f56deffd700 1 -- 172.21.15.92:0/2170115287 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d400cdf0 con 0x7f56c8004810 2023-10-06T10:45:33.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.050+0000 7f56deffd700 1 -- 172.21.15.92:0/2170115287 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4005b70 con 0x7f56c8004810 2023-10-06T10:45:33.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.050+0000 7f56e6649700 1 -- 172.21.15.92:0/2170115287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c8004810 msgr2=0x7f56c8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.050+0000 7f56e6649700 1 --2- 172.21.15.92:0/2170115287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c8004810 0x7f56c8004c80 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f56d400c3c0 tx=0x7f56d4001e70 comp rx=0 tx=0).stop 2023-10-06T10:45:33.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.050+0000 7f56e4be6700 1 -- 172.21.15.92:0/2170115287 reap_dead start 2023-10-06T10:45:33.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.050+0000 7f56e6649700 1 -- 172.21.15.92:0/2170115287 shutdown_connections 2023-10-06T10:45:33.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.050+0000 7f56e6649700 1 -- 172.21.15.92:0/2170115287 >> 172.21.15.92:0/2170115287 conn(0x7f56e00707d0 msgr2=0x7f56e0070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.050+0000 7f56e6649700 1 -- 172.21.15.92:0/2170115287 shutdown_connections 2023-10-06T10:45:33.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.050+0000 7f56e6649700 1 -- 172.21.15.92:0/2170115287 wait complete. 2023-10-06T10:45:33.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.051+0000 7f56e6649700 1 Processor -- start 2023-10-06T10:45:33.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.052+0000 7f56e6649700 1 -- start start 2023-10-06T10:45:33.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.052+0000 7f56e6649700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c8004810 0x7f56e00c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.052+0000 7f56e6649700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56e00c1fe0 0x7f56e0148550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.052+0000 7f56e6649700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56e016b190 0x7f56e016b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.052+0000 7f56e6649700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e00c6e40 con 0x7f56c8004810 2023-10-06T10:45:33.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.052+0000 7f56e6649700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e00c30d0 con 0x7f56e00c1fe0 2023-10-06T10:45:33.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.052+0000 7f56e6649700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e016bb20 con 0x7f56e016b190 2023-10-06T10:45:33.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.052+0000 7f56dffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c8004810 0x7f56e00c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.052+0000 7f56dffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c8004810 0x7f56e00c52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43400/0 (socket says 172.21.15.92:43400) 2023-10-06T10:45:33.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.052+0000 7f56dffff700 1 -- 172.21.15.92:0/2936009664 learned_addr learned my addr 172.21.15.92:0/2936009664 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:33.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.053+0000 7f56df7fe700 1 --2- 172.21.15.92:0/2936009664 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56e00c1fe0 0x7f56e0148550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.053+0000 7f56e4be6700 1 --2- 172.21.15.92:0/2936009664 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56e016b190 0x7f56e016b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.053+0000 7f56dffff700 1 -- 172.21.15.92:0/2936009664 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56e016b190 msgr2=0x7f56e016b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.053+0000 7f56dffff700 1 --2- 172.21.15.92:0/2936009664 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56e016b190 0x7f56e016b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.053+0000 7f56dffff700 1 -- 172.21.15.92:0/2936009664 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56e00c1fe0 msgr2=0x7f56e0148550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.053+0000 7f56dffff700 1 --2- 172.21.15.92:0/2936009664 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56e00c1fe0 0x7f56e0148550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.053+0000 7f56dffff700 1 -- 172.21.15.92:0/2936009664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e00c27e0 con 0x7f56c8004810 2023-10-06T10:45:33.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.053+0000 7f56dffff700 1 --2- 172.21.15.92:0/2936009664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c8004810 0x7f56e00c52e0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f56d0011d70 tx=0x7f56d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.053+0000 7f56c7fff700 1 -- 172.21.15.92:0/2936009664 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d000fa10 con 0x7f56c8004810 2023-10-06T10:45:33.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.053+0000 7f56c7fff700 1 -- 172.21.15.92:0/2936009664 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d000fb70 con 0x7f56c8004810 2023-10-06T10:45:33.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.053+0000 7f56c7fff700 1 -- 172.21.15.92:0/2936009664 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d0014440 con 0x7f56c8004810 2023-10-06T10:45:33.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.053+0000 7f56e6649700 1 -- 172.21.15.92:0/2936009664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e00c2a80 con 0x7f56c8004810 2023-10-06T10:45:33.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.053+0000 7f56e6649700 1 -- 172.21.15.92:0/2936009664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e00c2450 con 0x7f56c8004810 2023-10-06T10:45:33.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.054+0000 7f56e6649700 1 -- 172.21.15.92:0/2936009664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e0064a90 con 0x7f56c8004810 2023-10-06T10:45:33.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.055+0000 7f56c7fff700 1 -- 172.21.15.92:0/2936009664 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f56d0027480 con 0x7f56c8004810 2023-10-06T10:45:33.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.055+0000 7f56c7fff700 1 --2- 172.21.15.92:0/2936009664 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f56cc066f30 0x7f56cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.055+0000 7f56c7fff700 1 -- 172.21.15.92:0/2936009664 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f56d0018030 con 0x7f56c8004810 2023-10-06T10:45:33.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.055+0000 7f56df7fe700 1 --2- 172.21.15.92:0/2936009664 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f56cc066f30 0x7f56cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.056+0000 7f56df7fe700 1 --2- 172.21.15.92:0/2936009664 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f56cc066f30 0x7f56cc0693e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f56e01576d0 tx=0x7f56d4015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.059+0000 7f56c7fff700 1 -- 172.21.15.92:0/2936009664 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f56d0052710 con 0x7f56c8004810 2023-10-06T10:45:33.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.316+0000 7f56e6649700 1 -- 172.21.15.92:0/2936009664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f56e00c33b0 con 0x7f56c8004810 2023-10-06T10:45:33.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.317+0000 7f56c7fff700 1 -- 172.21.15.92:0/2936009664 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1972 (secure 0 0 0) 0x7f56d00227d0 con 0x7f56c8004810 2023-10-06T10:45:33.318 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:33.318 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"2 MDSs report damaged metadata","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":454,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":698974,"bytes_used":2455638016,"bytes_avail":770638475264,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6739,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":109,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6719},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":8517},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8520}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2023-10-06T10:45:32.177411+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2023-10-06T10:45:33.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.319+0000 7f56e6649700 1 -- 172.21.15.92:0/2936009664 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f56cc066f30 msgr2=0x7f56cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.319+0000 7f56e6649700 1 --2- 172.21.15.92:0/2936009664 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f56cc066f30 0x7f56cc0693e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f56e01576d0 tx=0x7f56d4015000 comp rx=0 tx=0).stop 2023-10-06T10:45:33.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.319+0000 7f56e6649700 1 -- 172.21.15.92:0/2936009664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c8004810 msgr2=0x7f56e00c52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.319+0000 7f56e6649700 1 --2- 172.21.15.92:0/2936009664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c8004810 0x7f56e00c52e0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f56d0011d70 tx=0x7f56d00091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.319+0000 7f56e6649700 1 -- 172.21.15.92:0/2936009664 shutdown_connections 2023-10-06T10:45:33.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.319+0000 7f56e6649700 1 --2- 172.21.15.92:0/2936009664 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f56cc066f30 0x7f56cc0693e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.319+0000 7f56e6649700 1 --2- 172.21.15.92:0/2936009664 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56e016b190 0x7f56e016b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.319+0000 7f56e6649700 1 --2- 172.21.15.92:0/2936009664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c8004810 0x7f56e00c52e0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.319+0000 7f56e6649700 1 --2- 172.21.15.92:0/2936009664 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56e00c1fe0 0x7f56e0148550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.319+0000 7f56e6649700 1 -- 172.21.15.92:0/2936009664 >> 172.21.15.92:0/2936009664 conn(0x7f56e00707d0 msgr2=0x7f56e0070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.319+0000 7f56e6649700 1 -- 172.21.15.92:0/2936009664 shutdown_connections 2023-10-06T10:45:33.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.319+0000 7f56e6649700 1 -- 172.21.15.92:0/2936009664 wait complete. 2023-10-06T10:45:33.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2023-10-06T10:45:33.332 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub abort 2023-10-06T10:45:33.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.447+0000 7fe30d7f4700 1 Processor -- start 2023-10-06T10:45:33.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.448+0000 7fe30d7f4700 1 -- start start 2023-10-06T10:45:33.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.448+0000 7fe30d7f4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe308067bd0 0x7fe3081584d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.448+0000 7fe30d7f4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe308141180 con 0x7fe3081405f0 2023-10-06T10:45:33.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.448+0000 7fe30d7f4700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe308157a60 con 0x7fe308067800 2023-10-06T10:45:33.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.448+0000 7fe30d7f4700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3080c9930 con 0x7fe308067bd0 2023-10-06T10:45:33.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.448+0000 7fe3077fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe3081405f0 0x7fe308140a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45492/0 (socket says 172.21.15.92:45492) 2023-10-06T10:45:33.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.449+0000 7fe3077fe700 1 -- 172.21.15.92:0/3002140015 learned_addr learned my addr 172.21.15.92:0/3002140015 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:33.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.449+0000 7fe2fe7fc700 1 --2- 172.21.15.92:0/3002140015 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe308067bd0 0x7fe3081584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.449+0000 7fe3067fc700 1 -- 172.21.15.92:0/3002140015 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3238007352 0 0) 0x7fe308141180 con 0x7fe3081405f0 2023-10-06T10:45:33.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.449+0000 7fe3067fc700 1 -- 172.21.15.92:0/3002140015 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2f4003570 con 0x7fe3081405f0 2023-10-06T10:45:33.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.449+0000 7fe3067fc700 1 -- 172.21.15.92:0/3002140015 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3352125401 0 0) 0x7fe308157a60 con 0x7fe308067800 2023-10-06T10:45:33.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.449+0000 7fe3067fc700 1 -- 172.21.15.92:0/3002140015 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe308141180 con 0x7fe308067800 2023-10-06T10:45:33.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.449+0000 7fe2fe7fc700 1 -- 172.21.15.92:0/3002140015 >> v1:172.21.15.124:6789/0 conn(0x7fe308067800 legacy=0x7fe30815af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.449+0000 7fe2fe7fc700 1 -- 172.21.15.92:0/3002140015 >> v1:172.21.15.92:6789/0 conn(0x7fe3081405f0 legacy=0x7fe308140a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.449+0000 7fe2fe7fc700 1 -- 172.21.15.92:0/3002140015 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30815ba80 con 0x7fe308067bd0 2023-10-06T10:45:33.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.449+0000 7fe2fe7fc700 1 --2- 172.21.15.92:0/3002140015 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe308067bd0 0x7fe3081584d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009940 tx=0x7fe2f00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5156aa6adc435df6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.449+0000 7fe3067fc700 1 -- 172.21.15.92:0/3002140015 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0011630 con 0x7fe308067bd0 2023-10-06T10:45:33.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.449+0000 7fe3067fc700 1 -- 172.21.15.92:0/3002140015 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f0011790 con 0x7fe308067bd0 2023-10-06T10:45:33.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.449+0000 7fe3067fc700 1 -- 172.21.15.92:0/3002140015 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0011b30 con 0x7fe308067bd0 2023-10-06T10:45:33.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.450+0000 7fe30d7f4700 1 -- 172.21.15.92:0/3002140015 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe308067bd0 msgr2=0x7fe3081584d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.450+0000 7fe30d7f4700 1 --2- 172.21.15.92:0/3002140015 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe308067bd0 0x7fe3081584d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009940 tx=0x7fe2f00091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.450+0000 7fe30d7f4700 1 -- 172.21.15.92:0/3002140015 shutdown_connections 2023-10-06T10:45:33.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.450+0000 7fe30d7f4700 1 --2- 172.21.15.92:0/3002140015 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe308067bd0 0x7fe3081584d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.450+0000 7fe30d7f4700 1 -- 172.21.15.92:0/3002140015 >> 172.21.15.92:0/3002140015 conn(0x7fe3080cd1a0 msgr2=0x7fe3080cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.450+0000 7fe30d7f4700 1 -- 172.21.15.92:0/3002140015 shutdown_connections 2023-10-06T10:45:33.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.450+0000 7fe30d7f4700 1 -- 172.21.15.92:0/3002140015 wait complete. 2023-10-06T10:45:33.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.451+0000 7fe30d7f4700 1 Processor -- start 2023-10-06T10:45:33.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.451+0000 7fe30d7f4700 1 -- start start 2023-10-06T10:45:33.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.452+0000 7fe30d7f4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe308067800 0x7fe30815d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.452+0000 7fe30d7f4700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe308067bd0 0x7fe308149c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.452+0000 7fe30d7f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3081405f0 0x7fe308159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.452+0000 7fe30d7f4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30814a1b0 con 0x7fe3081405f0 2023-10-06T10:45:33.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.452+0000 7fe30d7f4700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3081571f0 con 0x7fe308067bd0 2023-10-06T10:45:33.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.452+0000 7fe30d7f4700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30815d7b0 con 0x7fe308067800 2023-10-06T10:45:33.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.452+0000 7fe3077fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3081405f0 0x7fe308159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.452+0000 7fe3077fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3081405f0 0x7fe308159fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54858/0 (socket says 172.21.15.92:54858) 2023-10-06T10:45:33.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.452+0000 7fe3077fe700 1 -- 172.21.15.92:0/3418369773 learned_addr learned my addr 172.21.15.92:0/3418369773 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:33.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.452+0000 7fe2fe7fc700 1 --2- 172.21.15.92:0/3418369773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe308067bd0 0x7fe308149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.452+0000 7fe306ffd700 1 --2- 172.21.15.92:0/3418369773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe308067800 0x7fe30815d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.452+0000 7fe3077fe700 1 -- 172.21.15.92:0/3418369773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe308067800 msgr2=0x7fe30815d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.452+0000 7fe3077fe700 1 --2- 172.21.15.92:0/3418369773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe308067800 0x7fe30815d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.452+0000 7fe3077fe700 1 -- 172.21.15.92:0/3418369773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe308067bd0 msgr2=0x7fe308149c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.452+0000 7fe3077fe700 1 --2- 172.21.15.92:0/3418369773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe308067bd0 0x7fe308149c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.452+0000 7fe3077fe700 1 -- 172.21.15.92:0/3418369773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f0009420 con 0x7fe3081405f0 2023-10-06T10:45:33.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.453+0000 7fe3077fe700 1 --2- 172.21.15.92:0/3418369773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3081405f0 0x7fe308159fd0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fe30000cbc0 tx=0x7fe300001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.455+0000 7fe2ff7fe700 1 -- 172.21.15.92:0/3418369773 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30000a7e0 con 0x7fe3081405f0 2023-10-06T10:45:33.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.455+0000 7fe30d7f4700 1 -- 172.21.15.92:0/3418369773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe308168460 con 0x7fe3081405f0 2023-10-06T10:45:33.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.455+0000 7fe30d7f4700 1 -- 172.21.15.92:0/3418369773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe308168a20 con 0x7fe3081405f0 2023-10-06T10:45:33.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.455+0000 7fe2ff7fe700 1 -- 172.21.15.92:0/3418369773 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30000ab50 con 0x7fe3081405f0 2023-10-06T10:45:33.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.455+0000 7fe2ff7fe700 1 -- 172.21.15.92:0/3418369773 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30000acc0 con 0x7fe3081405f0 2023-10-06T10:45:33.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.456+0000 7fe2ff7fe700 1 -- 172.21.15.92:0/3418369773 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe300007300 con 0x7fe3081405f0 2023-10-06T10:45:33.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.456+0000 7fe2ff7fe700 1 --2- 172.21.15.92:0/3418369773 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe2e8066f30 0x7fe2e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.456+0000 7fe2ff7fe700 1 -- 172.21.15.92:0/3418369773 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fe300084b00 con 0x7fe3081405f0 2023-10-06T10:45:33.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.456+0000 7fe306ffd700 1 --2- 172.21.15.92:0/3418369773 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe2e8066f30 0x7fe2e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.457+0000 7fe306ffd700 1 --2- 172.21.15.92:0/3418369773 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe2e8066f30 0x7fe2e80693e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8007fd0 tx=0x7fe2f8007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.458+0000 7fe316aecb80 1 Processor -- start 2023-10-06T10:45:33.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.458+0000 7fe316aecb80 1 -- start start 2023-10-06T10:45:33.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.459+0000 7fe316aecb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8e841f0 0x55b1e8e9e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.459+0000 7fe316aecb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b1e8ea1a40 0x55b1e8e9ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.459+0000 7fe316aecb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b1e8e58c10 0x55b1e8e575c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.459+0000 7fe316aecb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8dd7cc0 con 0x55b1e8e841f0 2023-10-06T10:45:33.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.459+0000 7fe316aecb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8daeab0 con 0x55b1e8ea1a40 2023-10-06T10:45:33.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.459+0000 7fe316aecb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8dac2b0 con 0x55b1e8e58c10 2023-10-06T10:45:33.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.459+0000 7fe306ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8e841f0 0x55b1e8e9e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.459+0000 7fe306ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8e841f0 0x55b1e8e9e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54868/0 (socket says 172.21.15.92:54868) 2023-10-06T10:45:33.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.459+0000 7fe306ffd700 1 -- 172.21.15.92:0/2123414430 learned_addr learned my addr 172.21.15.92:0/2123414430 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:33.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.459+0000 7fe3077fe700 1 --2- 172.21.15.92:0/2123414430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b1e8e58c10 0x55b1e8e575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.459+0000 7fe2fe7fc700 1 --2- 172.21.15.92:0/2123414430 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b1e8ea1a40 0x55b1e8e9ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.460+0000 7fe306ffd700 1 -- 172.21.15.92:0/2123414430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b1e8e58c10 msgr2=0x55b1e8e575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.460+0000 7fe306ffd700 1 --2- 172.21.15.92:0/2123414430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b1e8e58c10 0x55b1e8e575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.460+0000 7fe306ffd700 1 -- 172.21.15.92:0/2123414430 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b1e8ea1a40 msgr2=0x55b1e8e9ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.460+0000 7fe306ffd700 1 --2- 172.21.15.92:0/2123414430 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b1e8ea1a40 0x55b1e8e9ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.460+0000 7fe306ffd700 1 -- 172.21.15.92:0/2123414430 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1e8c422b0 con 0x55b1e8e841f0 2023-10-06T10:45:33.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.460+0000 7fe2fe7fc700 1 --2- 172.21.15.92:0/2123414430 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b1e8ea1a40 0x55b1e8e9ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:33.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.460+0000 7fe306ffd700 1 --2- 172.21.15.92:0/2123414430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8e841f0 0x55b1e8e9e900 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8007fa0 tx=0x7fe2f8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a955bb4cd1a3047f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.460+0000 7fe2e7fff700 1 -- 172.21.15.92:0/2123414430 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f800fed0 con 0x55b1e8e841f0 2023-10-06T10:45:33.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.460+0000 7fe2e7fff700 1 -- 172.21.15.92:0/2123414430 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f8014660 con 0x55b1e8e841f0 2023-10-06T10:45:33.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.460+0000 7fe2e7fff700 1 -- 172.21.15.92:0/2123414430 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8014a00 con 0x55b1e8e841f0 2023-10-06T10:45:33.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.460+0000 7fe316aecb80 1 -- 172.21.15.92:0/2123414430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8e841f0 msgr2=0x55b1e8e9e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.460+0000 7fe316aecb80 1 --2- 172.21.15.92:0/2123414430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8e841f0 0x55b1e8e9e900 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8007fa0 tx=0x7fe2f8015000 comp rx=0 tx=0).stop 2023-10-06T10:45:33.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.461+0000 7fe316aecb80 1 -- 172.21.15.92:0/2123414430 shutdown_connections 2023-10-06T10:45:33.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.461+0000 7fe316aecb80 1 --2- 172.21.15.92:0/2123414430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b1e8e58c10 0x55b1e8e575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.461+0000 7fe316aecb80 1 --2- 172.21.15.92:0/2123414430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8e841f0 0x55b1e8e9e900 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.461+0000 7fe316aecb80 1 --2- 172.21.15.92:0/2123414430 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b1e8ea1a40 0x55b1e8e9ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.461+0000 7fe316aecb80 1 -- 172.21.15.92:0/2123414430 >> 172.21.15.92:0/2123414430 conn(0x55b1e8e590b0 msgr2=0x55b1e8e59480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.461+0000 7fe316aecb80 1 -- 172.21.15.92:0/2123414430 shutdown_connections 2023-10-06T10:45:33.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.461+0000 7fe316aecb80 1 -- 172.21.15.92:0/2123414430 wait complete. 2023-10-06T10:45:33.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.461+0000 7fe316aecb80 1 -- start start 2023-10-06T10:45:33.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.461+0000 7fe316aecb80 1 Processor -- start 2023-10-06T10:45:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.462+0000 7fe316aecb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b1e8e58c10 0x55b1e8e542d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.462+0000 7fe316aecb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b1e8e841f0 0x55b1e8e52e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.462+0000 7fe316aecb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8ea1a40 0x55b1e8e874d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.462+0000 7fe316aecb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8da80a0 con 0x55b1e8ea1a40 2023-10-06T10:45:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.462+0000 7fe316aecb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8d538d0 con 0x55b1e8e58c10 2023-10-06T10:45:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.462+0000 7fe316aecb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8d6eee0 con 0x55b1e8e841f0 2023-10-06T10:45:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.462+0000 7fe316aecb80 10 client.0 _ll_register_callbacks cb 0x55b1e8c44f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.462+0000 7fe3077fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8ea1a40 0x55b1e8e874d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.462+0000 7fe3077fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8ea1a40 0x55b1e8e874d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54870/0 (socket says 172.21.15.92:54870) 2023-10-06T10:45:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.462+0000 7fe3077fe700 1 -- 172.21.15.92:0/476900896 learned_addr learned my addr 172.21.15.92:0/476900896 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.462+0000 7fe2fe7fc700 1 --2- 172.21.15.92:0/476900896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b1e8e841f0 0x55b1e8e52e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.462+0000 7fe306ffd700 1 --2- 172.21.15.92:0/476900896 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b1e8e58c10 0x55b1e8e542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.463+0000 7fe3077fe700 1 -- 172.21.15.92:0/476900896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b1e8e841f0 msgr2=0x55b1e8e52e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.463+0000 7fe3077fe700 1 --2- 172.21.15.92:0/476900896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b1e8e841f0 0x55b1e8e52e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.463+0000 7fe3077fe700 1 -- 172.21.15.92:0/476900896 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b1e8e58c10 msgr2=0x55b1e8e542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.463+0000 7fe3077fe700 1 --2- 172.21.15.92:0/476900896 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b1e8e58c10 0x55b1e8e542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.463+0000 7fe3077fe700 1 -- 172.21.15.92:0/476900896 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe308141180 con 0x55b1e8ea1a40 2023-10-06T10:45:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.463+0000 7fe3077fe700 1 --2- 172.21.15.92:0/476900896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8ea1a40 0x55b1e8e874d0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fe300001cd0 tx=0x7fe300082ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2671e13b65b959 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.463+0000 7fe2e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.463+0000 7fe2e57fa700 1 -- 172.21.15.92:0/476900896 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe300058a00 con 0x55b1e8ea1a40 2023-10-06T10:45:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.463+0000 7fe2e57fa700 1 -- 172.21.15.92:0/476900896 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fe30005c6f0 con 0x55b1e8ea1a40 2023-10-06T10:45:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.463+0000 7fe316aecb80 1 -- 172.21.15.92:0/476900896 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1e8e4c6d0 con 0x55b1e8ea1a40 2023-10-06T10:45:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.463+0000 7fe2e57fa700 1 -- 172.21.15.92:0/476900896 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe300054c40 con 0x55b1e8ea1a40 2023-10-06T10:45:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.463+0000 7fe316aecb80 10 client.9069 fetch_fsmap learned FSMap version 109 2023-10-06T10:45:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.463+0000 7fe316aecb80 1 -- 172.21.15.92:0/476900896 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x55b1e8be5020 con 0x55b1e8ea1a40 2023-10-06T10:45:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.463+0000 7fe2e57fa700 1 -- 172.21.15.92:0/476900896 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b1e8e4c6d0 con 0x55b1e8ea1a40 2023-10-06T10:45:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.463+0000 7fe2e57fa700 1 -- 172.21.15.92:0/476900896 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 109) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fe30005c260 con 0x55b1e8ea1a40 2023-10-06T10:45:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.463+0000 7fe316aecb80 10 client.9069 fetch_fsmap finished waiting for FSMap version 109 2023-10-06T10:45:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.463+0000 7fe316aecb80 10 client.9069 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-10-06T10:45:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.463+0000 7fe316aecb80 20 client.9069 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.463+0000 7fe316aecb80 1 --2- 172.21.15.92:0/476900896 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55b1e8eb2470 0x55b1e8e80880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.463+0000 7fe316aecb80 4 client.9069 mds_command: new command op to 8520 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.463+0000 7fe316aecb80 1 -- 172.21.15.92:0/476900896 --> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1e8e29600 con 0x55b1e8eb2470 2023-10-06T10:45:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.463+0000 7fe306ffd700 1 --2- 172.21.15.92:0/476900896 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55b1e8eb2470 0x55b1e8e80880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.464+0000 7fe2e57fa700 10 client.9069 ms_handle_connect on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.464+0000 7fe306ffd700 1 --2- 172.21.15.92:0/476900896 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55b1e8eb2470 0x55b1e8e80880 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a373b64bf5be4ad2 server_cookie=9980c2ef42fb50bb in_seq=0 out_seq=0 2023-10-06T10:45:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.464+0000 7fe2e57fa700 1 -- 172.21.15.92:0/476900896 <== mds.0 v2:172.21.15.92:6835/2298348426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55b1e8e29600 con 0x55b1e8eb2470 2023-10-06T10:45:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.464+0000 7fe2e57fa700 10 client.9069 handle_command_reply: tid=0 2023-10-06T10:45:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.464+0000 7fe316aecb80 1 client.9069 shutdown 2023-10-06T10:45:33.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.464+0000 7fe316aecb80 1 -- 172.21.15.92:0/476900896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8ea1a40 msgr2=0x55b1e8e874d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.464+0000 7fe316aecb80 1 --2- 172.21.15.92:0/476900896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8ea1a40 0x55b1e8e874d0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fe300001cd0 tx=0x7fe300082ca0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.464+0000 7fe316aecb80 1 -- 172.21.15.92:0/476900896 shutdown_connections 2023-10-06T10:45:33.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.464+0000 7fe316aecb80 1 --2- 172.21.15.92:0/476900896 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55b1e8eb2470 0x55b1e8e80880 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.464+0000 7fe316aecb80 1 --2- 172.21.15.92:0/476900896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8ea1a40 0x55b1e8e874d0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.464+0000 7fe2e57fa700 0 client.9069 ms_handle_reset on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:33.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.464+0000 7fe316aecb80 1 --2- 172.21.15.92:0/476900896 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b1e8e58c10 0x55b1e8e542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.464+0000 7fe316aecb80 1 --2- 172.21.15.92:0/476900896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b1e8e841f0 0x55b1e8e52e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.464+0000 7fe316aecb80 1 -- 172.21.15.92:0/476900896 >> 172.21.15.92:0/476900896 conn(0x55b1e8e590b0 msgr2=0x55b1e8e575c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.465+0000 7fe316aecb80 1 -- 172.21.15.92:0/476900896 shutdown_connections 2023-10-06T10:45:33.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.465+0000 7fe316aecb80 1 -- 172.21.15.92:0/476900896 wait complete. 2023-10-06T10:45:33.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.465+0000 7fe316aecb80 20 client.9069 trim_cache size 0 max 16384 2023-10-06T10:45:33.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.480+0000 7fe316aecb80 1 Processor -- start 2023-10-06T10:45:33.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.480+0000 7fe316aecb80 1 -- start start 2023-10-06T10:45:33.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.481+0000 7fe316aecb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8e58c10 0x55b1e8ea1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.481+0000 7fe316aecb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b1e8e841f0 0x55b1e8e94690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.481+0000 7fe316aecb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b1e8ea1a40 0x55b1e8e94bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.481+0000 7fe316aecb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8c2ae70 con 0x55b1e8e58c10 2023-10-06T10:45:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.481+0000 7fe316aecb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8e83ea0 con 0x55b1e8e841f0 2023-10-06T10:45:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.481+0000 7fe316aecb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8e96470 con 0x55b1e8ea1a40 2023-10-06T10:45:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.481+0000 7fe306ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8e58c10 0x55b1e8ea1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.481+0000 7fe306ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8e58c10 0x55b1e8ea1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54880/0 (socket says 172.21.15.92:54880) 2023-10-06T10:45:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.481+0000 7fe306ffd700 1 -- 172.21.15.92:0/905324803 learned_addr learned my addr 172.21.15.92:0/905324803 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.481+0000 7fe306ffd700 1 -- 172.21.15.92:0/905324803 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b1e8ea1a40 msgr2=0x55b1e8e94bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.481+0000 7fe306ffd700 1 --2- 172.21.15.92:0/905324803 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b1e8ea1a40 0x55b1e8e94bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.481+0000 7fe306ffd700 1 -- 172.21.15.92:0/905324803 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b1e8e841f0 msgr2=0x55b1e8e94690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.481+0000 7fe306ffd700 1 --2- 172.21.15.92:0/905324803 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b1e8e841f0 0x55b1e8e94690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.481+0000 7fe306ffd700 1 -- 172.21.15.92:0/905324803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1e8d767c0 con 0x55b1e8e58c10 2023-10-06T10:45:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.481+0000 7fe306ffd700 1 --2- 172.21.15.92:0/905324803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8e58c10 0x55b1e8ea1030 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8023040 tx=0x7fe2f8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62bdeba23a7bb05b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.481+0000 7fe2fcff9700 1 -- 172.21.15.92:0/905324803 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8032070 con 0x55b1e8e58c10 2023-10-06T10:45:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.481+0000 7fe2fcff9700 1 -- 172.21.15.92:0/905324803 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f800fe50 con 0x55b1e8e58c10 2023-10-06T10:45:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.481+0000 7fe2fcff9700 1 -- 172.21.15.92:0/905324803 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8000c30 con 0x55b1e8e58c10 2023-10-06T10:45:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.482+0000 7fe316aecb80 1 -- 172.21.15.92:0/905324803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8e58c10 msgr2=0x55b1e8ea1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.482+0000 7fe316aecb80 1 --2- 172.21.15.92:0/905324803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8e58c10 0x55b1e8ea1030 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8023040 tx=0x7fe2f8019000 comp rx=0 tx=0).stop 2023-10-06T10:45:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.482+0000 7fe316aecb80 1 -- 172.21.15.92:0/905324803 shutdown_connections 2023-10-06T10:45:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.482+0000 7fe316aecb80 1 --2- 172.21.15.92:0/905324803 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b1e8ea1a40 0x55b1e8e94bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.482+0000 7fe316aecb80 1 --2- 172.21.15.92:0/905324803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8e58c10 0x55b1e8ea1030 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.482+0000 7fe316aecb80 1 --2- 172.21.15.92:0/905324803 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b1e8e841f0 0x55b1e8e94690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.482+0000 7fe316aecb80 1 -- 172.21.15.92:0/905324803 >> 172.21.15.92:0/905324803 conn(0x55b1e8e590b0 msgr2=0x55b1e8e59480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.482+0000 7fe316aecb80 1 -- 172.21.15.92:0/905324803 shutdown_connections 2023-10-06T10:45:33.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.482+0000 7fe316aecb80 1 -- 172.21.15.92:0/905324803 wait complete. 2023-10-06T10:45:33.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.482+0000 7fe316aecb80 1 -- start start 2023-10-06T10:45:33.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.482+0000 7fe316aecb80 1 Processor -- start 2023-10-06T10:45:33.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.483+0000 7fe316aecb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b1e8e58c10 0x55b1e8eedab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.483+0000 7fe316aecb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8e841f0 0x55b1e8e575c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.483+0000 7fe316aecb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b1e8ea1a40 0x55b1e8e57b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.483+0000 7fe316aecb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8e9d920 con 0x55b1e8e841f0 2023-10-06T10:45:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.483+0000 7fe316aecb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8f07c70 con 0x55b1e8ea1a40 2023-10-06T10:45:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.483+0000 7fe316aecb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8e87ea0 con 0x55b1e8e58c10 2023-10-06T10:45:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.483+0000 7fe316aecb80 10 client.0 _ll_register_callbacks cb 0x55b1e8c44f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.483+0000 7fe2fe7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8e841f0 0x55b1e8e575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.483+0000 7fe2fe7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8e841f0 0x55b1e8e575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54888/0 (socket says 172.21.15.92:54888) 2023-10-06T10:45:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.483+0000 7fe2fe7fc700 1 -- 172.21.15.92:0/1993590798 learned_addr learned my addr 172.21.15.92:0/1993590798 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe306ffd700 1 --2- 172.21.15.92:0/1993590798 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b1e8e58c10 0x55b1e8eedab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe3077fe700 1 --2- 172.21.15.92:0/1993590798 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b1e8ea1a40 0x55b1e8e57b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe2fe7fc700 1 -- 172.21.15.92:0/1993590798 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b1e8e58c10 msgr2=0x55b1e8eedab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe2fe7fc700 1 --2- 172.21.15.92:0/1993590798 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b1e8e58c10 0x55b1e8eedab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe2fe7fc700 1 -- 172.21.15.92:0/1993590798 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b1e8ea1a40 msgr2=0x55b1e8e57b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe2fe7fc700 1 --2- 172.21.15.92:0/1993590798 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b1e8ea1a40 0x55b1e8e57b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe2fe7fc700 1 -- 172.21.15.92:0/1993590798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b1e8c422b0 con 0x55b1e8e841f0 2023-10-06T10:45:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe2fe7fc700 1 --2- 172.21.15.92:0/1993590798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8e841f0 0x55b1e8e575c0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fe30806d340 tx=0x7fe2f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c3c4583a127d95 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe2e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe2e57fa700 1 -- 172.21.15.92:0/1993590798 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f001f040 con 0x55b1e8e841f0 2023-10-06T10:45:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe2e57fa700 1 -- 172.21.15.92:0/1993590798 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fe2f0026700 con 0x55b1e8e841f0 2023-10-06T10:45:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe316aecb80 1 -- 172.21.15.92:0/1993590798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1e8e56a50 con 0x55b1e8e841f0 2023-10-06T10:45:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe2e57fa700 1 -- 172.21.15.92:0/1993590798 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2f0026ad0 con 0x55b1e8e841f0 2023-10-06T10:45:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe316aecb80 10 client.9075 fetch_fsmap learned FSMap version 109 2023-10-06T10:45:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe316aecb80 1 -- 172.21.15.92:0/1993590798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x55b1e8d5d4b0 con 0x55b1e8e841f0 2023-10-06T10:45:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe2e57fa700 1 -- 172.21.15.92:0/1993590798 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b1e8e56a50 con 0x55b1e8e841f0 2023-10-06T10:45:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe2e57fa700 1 -- 172.21.15.92:0/1993590798 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 109) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fe2f0026c30 con 0x55b1e8e841f0 2023-10-06T10:45:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe316aecb80 10 client.9075 fetch_fsmap finished waiting for FSMap version 109 2023-10-06T10:45:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe316aecb80 10 client.9075 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-10-06T10:45:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe316aecb80 20 client.9075 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe316aecb80 1 --2- 172.21.15.92:0/1993590798 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55b1e8e87950 0x55b1e8e9af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe316aecb80 4 client.9075 mds_command: new command op to 8520 tid=0[{"prefix": "scrub abort"}] 2023-10-06T10:45:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.484+0000 7fe316aecb80 1 -- 172.21.15.92:0/1993590798 --> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x7fe2f8014660 con 0x55b1e8e87950 2023-10-06T10:45:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.485+0000 7fe306ffd700 1 --2- 172.21.15.92:0/1993590798 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55b1e8e87950 0x55b1e8e9af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.485+0000 7fe2e57fa700 10 client.9075 ms_handle_connect on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:33.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.485+0000 7fe306ffd700 1 --2- 172.21.15.92:0/1993590798 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55b1e8e87950 0x55b1e8e9af20 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a82e608658d9746 server_cookie=d0d25b8b81aa3f83 in_seq=0 out_seq=0 2023-10-06T10:45:33.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.485+0000 7fe2e57fa700 1 -- 172.21.15.92:0/1993590798 <== mds.0 v2:172.21.15.92:6835/2298348426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x7fe2f8014660 con 0x55b1e8e87950 2023-10-06T10:45:33.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.485+0000 7fe2e57fa700 10 client.9075 handle_command_reply: tid=0 2023-10-06T10:45:33.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.485+0000 7fe316aecb80 1 client.9075 shutdown 2023-10-06T10:45:33.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.485+0000 7fe316aecb80 1 -- 172.21.15.92:0/1993590798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8e841f0 msgr2=0x55b1e8e575c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.485+0000 7fe316aecb80 1 --2- 172.21.15.92:0/1993590798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8e841f0 0x55b1e8e575c0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fe30806d340 tx=0x7fe2f0017000 comp rx=0 tx=0).stop 2023-10-06T10:45:33.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.485+0000 7fe316aecb80 1 -- 172.21.15.92:0/1993590798 shutdown_connections 2023-10-06T10:45:33.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.485+0000 7fe316aecb80 1 --2- 172.21.15.92:0/1993590798 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55b1e8e87950 0x55b1e8e9af20 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.485+0000 7fe316aecb80 1 --2- 172.21.15.92:0/1993590798 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b1e8ea1a40 0x55b1e8e57b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.485+0000 7fe2e57fa700 0 client.9075 ms_handle_reset on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:33.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.485+0000 7fe316aecb80 1 --2- 172.21.15.92:0/1993590798 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b1e8e58c10 0x55b1e8eedab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.485+0000 7fe316aecb80 1 --2- 172.21.15.92:0/1993590798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1e8e841f0 0x55b1e8e575c0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.485+0000 7fe316aecb80 1 -- 172.21.15.92:0/1993590798 >> 172.21.15.92:0/1993590798 conn(0x55b1e8e590b0 msgr2=0x55b1e8e54be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.485+0000 7fe316aecb80 1 -- 172.21.15.92:0/1993590798 shutdown_connections 2023-10-06T10:45:33.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.485+0000 7fe316aecb80 1 -- 172.21.15.92:0/1993590798 wait complete. 2023-10-06T10:45:33.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.486+0000 7fe316aecb80 20 client.9075 trim_cache size 0 max 16384 2023-10-06T10:45:33.494 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:33.494 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:45:33.494 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:33.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.486+0000 7fe30d7f4700 1 -- 172.21.15.92:0/3418369773 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe2e8066f30 msgr2=0x7fe2e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.486+0000 7fe30d7f4700 1 --2- 172.21.15.92:0/3418369773 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe2e8066f30 0x7fe2e80693e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8007fd0 tx=0x7fe2f8007d50 comp rx=0 tx=0).stop 2023-10-06T10:45:33.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.486+0000 7fe30d7f4700 1 -- 172.21.15.92:0/3418369773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3081405f0 msgr2=0x7fe308159fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.486+0000 7fe30d7f4700 1 --2- 172.21.15.92:0/3418369773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3081405f0 0x7fe308159fd0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fe30000cbc0 tx=0x7fe300001a40 comp rx=0 tx=0).stop 2023-10-06T10:45:33.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.486+0000 7fe30d7f4700 1 -- 172.21.15.92:0/3418369773 shutdown_connections 2023-10-06T10:45:33.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.486+0000 7fe30d7f4700 1 --2- 172.21.15.92:0/3418369773 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe2e8066f30 0x7fe2e80693e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.486+0000 7fe30d7f4700 1 --2- 172.21.15.92:0/3418369773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3081405f0 0x7fe308159fd0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.486+0000 7fe30d7f4700 1 --2- 172.21.15.92:0/3418369773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe308067800 0x7fe30815d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.486+0000 7fe30d7f4700 1 --2- 172.21.15.92:0/3418369773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe308067bd0 0x7fe308149c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.486+0000 7fe30d7f4700 1 -- 172.21.15.92:0/3418369773 >> 172.21.15.92:0/3418369773 conn(0x7fe3080cd1a0 msgr2=0x7fe3080c27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.486+0000 7fe30d7f4700 1 -- 172.21.15.92:0/3418369773 shutdown_connections 2023-10-06T10:45:33.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.486+0000 7fe30d7f4700 1 -- 172.21.15.92:0/3418369773 wait complete. 2023-10-06T10:45:33.498 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:1 scrub status 2023-10-06T10:45:33.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.613+0000 7f6498c09700 1 Processor -- start 2023-10-06T10:45:33.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.613+0000 7f6498c09700 1 -- start start 2023-10-06T10:45:33.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.614+0000 7f6498c09700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f64940cd3c0 0x7f64940ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.614+0000 7f6498c09700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64940b29f0 0x7f64940cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.614+0000 7f6498c09700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64940cc970 con 0x7f64940b2620 2023-10-06T10:45:33.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.614+0000 7f6498c09700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64940cf520 con 0x7f64940cd3c0 2023-10-06T10:45:33.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.614+0000 7f6498c09700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64940cbd60 con 0x7f64940b29f0 2023-10-06T10:45:33.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.614+0000 7f6492ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f64940cd3c0 0x7f64940ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.614+0000 7f64937fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f64940b2620 0x7f64940cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45502/0 (socket says 172.21.15.92:45502) 2023-10-06T10:45:33.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.614+0000 7f64937fe700 1 -- 172.21.15.92:0/333106578 learned_addr learned my addr 172.21.15.92:0/333106578 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.615+0000 7f6493fff700 1 --2- 172.21.15.92:0/333106578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64940b29f0 0x7f64940cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.615+0000 7f6492ffd700 1 -- 172.21.15.92:0/333106578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64940b29f0 msgr2=0x7f64940cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.615+0000 7f6492ffd700 1 --2- 172.21.15.92:0/333106578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64940b29f0 0x7f64940cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.615+0000 7f64927fc700 1 -- 172.21.15.92:0/333106578 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970834312 0 0) 0x7f64940cc970 con 0x7f64940b2620 2023-10-06T10:45:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.615+0000 7f6492ffd700 1 -- 172.21.15.92:0/333106578 >> v1:172.21.15.92:6789/0 conn(0x7f64940b2620 legacy=0x7f64940cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.615+0000 7f6492ffd700 1 -- 172.21.15.92:0/333106578 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64940cc040 con 0x7f64940cd3c0 2023-10-06T10:45:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.615+0000 7f6493fff700 1 --2- 172.21.15.92:0/333106578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64940b29f0 0x7f64940cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.615+0000 7f6492ffd700 1 --2- 172.21.15.92:0/333106578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64940cd3c0 0x7f64940ceaa0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f64880098e0 tx=0x7f64880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af28fbae9aba0f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.615+0000 7f64927fc700 1 -- 172.21.15.92:0/333106578 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6488009e80 con 0x7f64940cd3c0 2023-10-06T10:45:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.615+0000 7f64927fc700 1 -- 172.21.15.92:0/333106578 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648800b860 con 0x7f64940cd3c0 2023-10-06T10:45:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.615+0000 7f64927fc700 1 -- 172.21.15.92:0/333106578 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648800bbe0 con 0x7f64940cd3c0 2023-10-06T10:45:33.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.615+0000 7f6498c09700 1 -- 172.21.15.92:0/333106578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64940cd3c0 msgr2=0x7f64940ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.615+0000 7f6498c09700 1 --2- 172.21.15.92:0/333106578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64940cd3c0 0x7f64940ceaa0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f64880098e0 tx=0x7f64880091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.615+0000 7f6498c09700 1 -- 172.21.15.92:0/333106578 shutdown_connections 2023-10-06T10:45:33.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.615+0000 7f6498c09700 1 --2- 172.21.15.92:0/333106578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64940b29f0 0x7f64940cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.615+0000 7f6498c09700 1 --2- 172.21.15.92:0/333106578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64940cd3c0 0x7f64940ceaa0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.615+0000 7f6498c09700 1 -- 172.21.15.92:0/333106578 >> 172.21.15.92:0/333106578 conn(0x7f6494099520 msgr2=0x7f649406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.615+0000 7f6498c09700 1 -- 172.21.15.92:0/333106578 shutdown_connections 2023-10-06T10:45:33.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.615+0000 7f6498c09700 1 -- 172.21.15.92:0/333106578 wait complete. 2023-10-06T10:45:33.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.617+0000 7f6498c09700 1 Processor -- start 2023-10-06T10:45:33.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.617+0000 7f6498c09700 1 -- start start 2023-10-06T10:45:33.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.617+0000 7f6498c09700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64940b2620 0x7f64941532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.617+0000 7f6498c09700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64940b29f0 0x7f649415bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.617+0000 7f6498c09700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64940cd3c0 0x7f6494068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.617+0000 7f6498c09700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649415ab60 con 0x7f64940b2620 2023-10-06T10:45:33.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.617+0000 7f6498c09700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649415ae40 con 0x7f64940b29f0 2023-10-06T10:45:33.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.617+0000 7f6498c09700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649415b120 con 0x7f64940cd3c0 2023-10-06T10:45:33.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.617+0000 7f64937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64940b2620 0x7f64941532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.617+0000 7f64937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64940b2620 0x7f64941532f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54900/0 (socket says 172.21.15.92:54900) 2023-10-06T10:45:33.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.617+0000 7f64937fe700 1 -- 172.21.15.92:0/895389781 learned_addr learned my addr 172.21.15.92:0/895389781 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:33.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.618+0000 7f6493fff700 1 --2- 172.21.15.92:0/895389781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64940cd3c0 0x7f6494068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.618+0000 7f6492ffd700 1 --2- 172.21.15.92:0/895389781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64940b29f0 0x7f649415bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.618+0000 7f64937fe700 1 -- 172.21.15.92:0/895389781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64940cd3c0 msgr2=0x7f6494068d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.618+0000 7f64937fe700 1 --2- 172.21.15.92:0/895389781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64940cd3c0 0x7f6494068d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.618+0000 7f64937fe700 1 -- 172.21.15.92:0/895389781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64940b29f0 msgr2=0x7f649415bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.618+0000 7f64937fe700 1 --2- 172.21.15.92:0/895389781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64940b29f0 0x7f649415bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.618+0000 7f64937fe700 1 -- 172.21.15.92:0/895389781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6488009420 con 0x7f64940b2620 2023-10-06T10:45:33.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.618+0000 7f6492ffd700 1 --2- 172.21.15.92:0/895389781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64940b29f0 0x7f649415bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:33.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.618+0000 7f64937fe700 1 --2- 172.21.15.92:0/895389781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64940b2620 0x7f64941532f0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f648400c8e0 tx=0x7f64840016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.618+0000 7f647b7fe700 1 -- 172.21.15.92:0/895389781 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6484005d80 con 0x7f64940b2620 2023-10-06T10:45:33.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.618+0000 7f647b7fe700 1 -- 172.21.15.92:0/895389781 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64840070f0 con 0x7f64940b2620 2023-10-06T10:45:33.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.618+0000 7f6498c09700 1 -- 172.21.15.92:0/895389781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f649415b400 con 0x7f64940b2620 2023-10-06T10:45:33.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.618+0000 7f647b7fe700 1 -- 172.21.15.92:0/895389781 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64840074b0 con 0x7f64940b2620 2023-10-06T10:45:33.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.618+0000 7f6498c09700 1 -- 172.21.15.92:0/895389781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64940692d0 con 0x7f64940b2620 2023-10-06T10:45:33.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.620+0000 7f647b7fe700 1 -- 172.21.15.92:0/895389781 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f64840078e0 con 0x7f64940b2620 2023-10-06T10:45:33.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.620+0000 7f647b7fe700 1 --2- 172.21.15.92:0/895389781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6480066f30 0x7f64800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.620+0000 7f6492ffd700 1 --2- 172.21.15.92:0/895389781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6480066f30 0x7f64800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.620+0000 7f647b7fe700 1 -- 172.21.15.92:0/895389781 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f6484084500 con 0x7f64940b2620 2023-10-06T10:45:33.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.621+0000 7f6492ffd700 1 --2- 172.21.15.92:0/895389781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6480066f30 0x7f64800693e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f64940cd7c0 tx=0x7f6488017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.622+0000 7f64a3163b80 1 Processor -- start 2023-10-06T10:45:33.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.622+0000 7f64a3163b80 1 -- start start 2023-10-06T10:45:33.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.623+0000 7f64a3163b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d8ce3a79a0 0x55d8ce3a7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.623+0000 7f64a3163b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3f1c10 0x55d8ce3d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.623+0000 7f64a3163b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d8ce3d0eb0 0x55d8ce3a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.623+0000 7f64a3163b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ce327cd0 con 0x55d8ce3f1c10 2023-10-06T10:45:33.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.623+0000 7f64a3163b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ce2feac0 con 0x55d8ce3a79a0 2023-10-06T10:45:33.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.623+0000 7f64a3163b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8ce2fc2c0 con 0x55d8ce3d0eb0 2023-10-06T10:45:33.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.623+0000 7f6492ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3f1c10 0x55d8ce3d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.623+0000 7f6492ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3f1c10 0x55d8ce3d0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54912/0 (socket says 172.21.15.92:54912) 2023-10-06T10:45:33.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.623+0000 7f6492ffd700 1 -- 172.21.15.92:0/2970369405 learned_addr learned my addr 172.21.15.92:0/2970369405 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:33.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.623+0000 7f6493fff700 1 --2- 172.21.15.92:0/2970369405 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d8ce3d0eb0 0x55d8ce3a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.623+0000 7f64937fe700 1 --2- 172.21.15.92:0/2970369405 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d8ce3a79a0 0x55d8ce3a7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.623+0000 7f6492ffd700 1 -- 172.21.15.92:0/2970369405 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d8ce3d0eb0 msgr2=0x55d8ce3a39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.623+0000 7f6492ffd700 1 --2- 172.21.15.92:0/2970369405 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d8ce3d0eb0 0x55d8ce3a39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.623+0000 7f6492ffd700 1 -- 172.21.15.92:0/2970369405 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d8ce3a79a0 msgr2=0x55d8ce3a7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.623+0000 7f6492ffd700 1 --2- 172.21.15.92:0/2970369405 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d8ce3a79a0 0x55d8ce3a7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.623+0000 7f6492ffd700 1 -- 172.21.15.92:0/2970369405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8ce1922c0 con 0x55d8ce3f1c10 2023-10-06T10:45:33.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.623+0000 7f6493fff700 1 --2- 172.21.15.92:0/2970369405 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d8ce3d0eb0 0x55d8ce3a39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:33.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.623+0000 7f6492ffd700 1 --2- 172.21.15.92:0/2970369405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3f1c10 0x55d8ce3d0970 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f648800b3c0 tx=0x7f6488020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1386f8273600479a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.623+0000 7f6478ff9700 1 -- 172.21.15.92:0/2970369405 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6488020070 con 0x55d8ce3f1c10 2023-10-06T10:45:33.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.624+0000 7f6478ff9700 1 -- 172.21.15.92:0/2970369405 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6488029d80 con 0x55d8ce3f1c10 2023-10-06T10:45:33.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.624+0000 7f6478ff9700 1 -- 172.21.15.92:0/2970369405 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648801a420 con 0x55d8ce3f1c10 2023-10-06T10:45:33.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.624+0000 7f64a3163b80 1 -- 172.21.15.92:0/2970369405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3f1c10 msgr2=0x55d8ce3d0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.624+0000 7f64a3163b80 1 --2- 172.21.15.92:0/2970369405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3f1c10 0x55d8ce3d0970 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f648800b3c0 tx=0x7f6488020000 comp rx=0 tx=0).stop 2023-10-06T10:45:33.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.624+0000 7f64a3163b80 1 -- 172.21.15.92:0/2970369405 shutdown_connections 2023-10-06T10:45:33.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.624+0000 7f64a3163b80 1 --2- 172.21.15.92:0/2970369405 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d8ce3d0eb0 0x55d8ce3a39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.624+0000 7f64a3163b80 1 --2- 172.21.15.92:0/2970369405 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d8ce3a79a0 0x55d8ce3a7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.624+0000 7f64a3163b80 1 --2- 172.21.15.92:0/2970369405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3f1c10 0x55d8ce3d0970 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.624+0000 7f64a3163b80 1 -- 172.21.15.92:0/2970369405 >> 172.21.15.92:0/2970369405 conn(0x55d8ce3a92c0 msgr2=0x55d8ce3a9690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.624+0000 7f64a3163b80 1 -- 172.21.15.92:0/2970369405 shutdown_connections 2023-10-06T10:45:33.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.624+0000 7f64a3163b80 1 -- 172.21.15.92:0/2970369405 wait complete. 2023-10-06T10:45:33.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.625+0000 7f64a3163b80 1 -- start start 2023-10-06T10:45:33.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.625+0000 7f64a3163b80 1 Processor -- start 2023-10-06T10:45:33.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f64a3163b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3a79a0 0x55d8ce3e5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f64a3163b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d8ce3d0eb0 0x55d8ce39e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f64a3163b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d8ce3f1c10 0x55d8ce39e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f64a3163b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ce2f80b0 con 0x55d8ce3a79a0 2023-10-06T10:45:33.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f64a3163b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ce2a38e0 con 0x55d8ce3d0eb0 2023-10-06T10:45:33.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f64a3163b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8ce2beef0 con 0x55d8ce3f1c10 2023-10-06T10:45:33.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f64a3163b80 10 client.0 _ll_register_callbacks cb 0x55d8ce194f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:33.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f64937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3a79a0 0x55d8ce3e5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f64937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3a79a0 0x55d8ce3e5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54918/0 (socket says 172.21.15.92:54918) 2023-10-06T10:45:33.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f64937fe700 1 -- 172.21.15.92:0/1715571865 learned_addr learned my addr 172.21.15.92:0/1715571865 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:33.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f64937fe700 1 -- 172.21.15.92:0/1715571865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d8ce3f1c10 msgr2=0x55d8ce39e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f6493fff700 1 --2- 172.21.15.92:0/1715571865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d8ce3f1c10 0x55d8ce39e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f64937fe700 1 --2- 172.21.15.92:0/1715571865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d8ce3f1c10 0x55d8ce39e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f64937fe700 1 -- 172.21.15.92:0/1715571865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d8ce3d0eb0 msgr2=0x55d8ce39e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f64937fe700 1 --2- 172.21.15.92:0/1715571865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d8ce3d0eb0 0x55d8ce39e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f64937fe700 1 -- 172.21.15.92:0/1715571865 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64940cc040 con 0x55d8ce3a79a0 2023-10-06T10:45:33.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f64937fe700 1 --2- 172.21.15.92:0/1715571865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3a79a0 0x55d8ce3e5b30 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f6484055210 tx=0x7f6484014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432e381d12145ae4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f6471ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:33.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f6471ffb700 1 -- 172.21.15.92:0/1715571865 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648405c170 con 0x55d8ce3a79a0 2023-10-06T10:45:33.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f6471ffb700 1 -- 172.21.15.92:0/1715571865 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f6484053620 con 0x55d8ce3a79a0 2023-10-06T10:45:33.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f64a3163b80 1 -- 172.21.15.92:0/1715571865 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8ce39c6e0 con 0x55d8ce3a79a0 2023-10-06T10:45:33.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f6471ffb700 1 -- 172.21.15.92:0/1715571865 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6484054980 con 0x55d8ce3a79a0 2023-10-06T10:45:33.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f64a3163b80 10 client.9090 fetch_fsmap learned FSMap version 109 2023-10-06T10:45:33.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f64a3163b80 1 -- 172.21.15.92:0/1715571865 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x55d8ce135030 con 0x55d8ce3a79a0 2023-10-06T10:45:33.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.626+0000 7f6471ffb700 1 -- 172.21.15.92:0/1715571865 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d8ce39c6e0 con 0x55d8ce3a79a0 2023-10-06T10:45:33.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.627+0000 7f6471ffb700 1 -- 172.21.15.92:0/1715571865 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 109) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f648405bce0 con 0x55d8ce3a79a0 2023-10-06T10:45:33.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.627+0000 7f64a3163b80 10 client.9090 fetch_fsmap finished waiting for FSMap version 109 2023-10-06T10:45:33.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.627+0000 7f64a3163b80 10 client.9090 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2023-10-06T10:45:33.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.627+0000 7f64a3163b80 20 client.9090 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:33.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.627+0000 7f64a3163b80 1 --2- 172.21.15.92:0/1715571865 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x55d8ce3cfc30 0x55d8ce3fdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.627+0000 7f64a3163b80 4 client.9090 mds_command: new command op to 8517 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:33.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.627+0000 7f64a3163b80 1 -- 172.21.15.92:0/1715571865 --> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8ce379610 con 0x55d8ce3cfc30 2023-10-06T10:45:33.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.627+0000 7f6492ffd700 1 --2- 172.21.15.92:0/1715571865 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x55d8ce3cfc30 0x55d8ce3fdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.627+0000 7f6471ffb700 10 client.9090 ms_handle_connect on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.627+0000 7f6492ffd700 1 --2- 172.21.15.92:0/1715571865 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x55d8ce3cfc30 0x55d8ce3fdb40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef7c7953ec14d612 server_cookie=83187880eabee8a8 in_seq=0 out_seq=0 2023-10-06T10:45:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.628+0000 7f6471ffb700 1 -- 172.21.15.92:0/1715571865 <== mds.1 v2:172.21.15.92:6834/1979826278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55d8ce379610 con 0x55d8ce3cfc30 2023-10-06T10:45:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.628+0000 7f6471ffb700 10 client.9090 handle_command_reply: tid=0 2023-10-06T10:45:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.628+0000 7f64a3163b80 1 client.9090 shutdown 2023-10-06T10:45:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.628+0000 7f64a3163b80 1 -- 172.21.15.92:0/1715571865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3a79a0 msgr2=0x55d8ce3e5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.628+0000 7f64a3163b80 1 --2- 172.21.15.92:0/1715571865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3a79a0 0x55d8ce3e5b30 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f6484055210 tx=0x7f6484014040 comp rx=0 tx=0).stop 2023-10-06T10:45:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.628+0000 7f64a3163b80 1 -- 172.21.15.92:0/1715571865 shutdown_connections 2023-10-06T10:45:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.628+0000 7f64a3163b80 1 --2- 172.21.15.92:0/1715571865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d8ce3f1c10 0x55d8ce39e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.628+0000 7f64a3163b80 1 --2- 172.21.15.92:0/1715571865 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x55d8ce3cfc30 0x55d8ce3fdb40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.628+0000 7f64a3163b80 1 --2- 172.21.15.92:0/1715571865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3a79a0 0x55d8ce3e5b30 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.628+0000 7f6471ffb700 0 client.9090 ms_handle_reset on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.628+0000 7f64a3163b80 1 --2- 172.21.15.92:0/1715571865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d8ce3d0eb0 0x55d8ce39e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.628+0000 7f64a3163b80 1 -- 172.21.15.92:0/1715571865 >> 172.21.15.92:0/1715571865 conn(0x55d8ce3a92c0 msgr2=0x55d8ce3ed4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.628+0000 7f64a3163b80 1 -- 172.21.15.92:0/1715571865 shutdown_connections 2023-10-06T10:45:33.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.628+0000 7f64a3163b80 1 -- 172.21.15.92:0/1715571865 wait complete. 2023-10-06T10:45:33.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.628+0000 7f64a3163b80 20 client.9090 trim_cache size 0 max 16384 2023-10-06T10:45:33.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.644+0000 7f64a3163b80 1 Processor -- start 2023-10-06T10:45:33.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.644+0000 7f64a3163b80 1 -- start start 2023-10-06T10:45:33.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.645+0000 7f64a3163b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d8ce3a79a0 0x55d8ce3d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.645+0000 7f64a3163b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d8ce3d0eb0 0x55d8ce3eaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.645+0000 7f64a3163b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3f1c10 0x55d8ce3a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.645+0000 7f64a3163b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ce17ae80 con 0x55d8ce3f1c10 2023-10-06T10:45:33.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.645+0000 7f64a3163b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ce39c450 con 0x55d8ce3d0eb0 2023-10-06T10:45:33.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.645+0000 7f64a3163b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8ce3e5bf0 con 0x55d8ce3a79a0 2023-10-06T10:45:33.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.645+0000 7f6493fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3f1c10 0x55d8ce3a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.645+0000 7f6493fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3f1c10 0x55d8ce3a3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54924/0 (socket says 172.21.15.92:54924) 2023-10-06T10:45:33.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.645+0000 7f6493fff700 1 -- 172.21.15.92:0/2247242273 learned_addr learned my addr 172.21.15.92:0/2247242273 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:33.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.645+0000 7f64937fe700 1 --2- 172.21.15.92:0/2247242273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d8ce3a79a0 0x55d8ce3d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.645+0000 7f6492ffd700 1 --2- 172.21.15.92:0/2247242273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d8ce3d0eb0 0x55d8ce3eaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.645+0000 7f6493fff700 1 -- 172.21.15.92:0/2247242273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d8ce3a79a0 msgr2=0x55d8ce3d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.645+0000 7f6493fff700 1 --2- 172.21.15.92:0/2247242273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d8ce3a79a0 0x55d8ce3d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.645+0000 7f6493fff700 1 -- 172.21.15.92:0/2247242273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d8ce3d0eb0 msgr2=0x55d8ce3eaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.645+0000 7f6493fff700 1 --2- 172.21.15.92:0/2247242273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d8ce3d0eb0 0x55d8ce3eaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.645+0000 7f6493fff700 1 -- 172.21.15.92:0/2247242273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8ce3a8d20 con 0x55d8ce3f1c10 2023-10-06T10:45:33.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.645+0000 7f6492ffd700 1 --2- 172.21.15.92:0/2247242273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d8ce3d0eb0 0x55d8ce3eaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:33.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.645+0000 7f6493fff700 1 --2- 172.21.15.92:0/2247242273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3f1c10 0x55d8ce3a3fb0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f6494089490 tx=0x7f648c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36b4b30a48148ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.645+0000 7f64797fa700 1 -- 172.21.15.92:0/2247242273 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c012920 con 0x55d8ce3f1c10 2023-10-06T10:45:33.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.645+0000 7f64797fa700 1 -- 172.21.15.92:0/2247242273 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648c00c040 con 0x55d8ce3f1c10 2023-10-06T10:45:33.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.645+0000 7f64797fa700 1 -- 172.21.15.92:0/2247242273 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c012cd0 con 0x55d8ce3f1c10 2023-10-06T10:45:33.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.646+0000 7f64a3163b80 1 -- 172.21.15.92:0/2247242273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3f1c10 msgr2=0x55d8ce3a3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.646+0000 7f64a3163b80 1 --2- 172.21.15.92:0/2247242273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3f1c10 0x55d8ce3a3fb0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f6494089490 tx=0x7f648c008040 comp rx=0 tx=0).stop 2023-10-06T10:45:33.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.646+0000 7f64a3163b80 1 -- 172.21.15.92:0/2247242273 shutdown_connections 2023-10-06T10:45:33.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.646+0000 7f64a3163b80 1 --2- 172.21.15.92:0/2247242273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3f1c10 0x55d8ce3a3fb0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.646+0000 7f64a3163b80 1 --2- 172.21.15.92:0/2247242273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d8ce3a79a0 0x55d8ce3d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.646+0000 7f64a3163b80 1 --2- 172.21.15.92:0/2247242273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d8ce3d0eb0 0x55d8ce3eaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.646+0000 7f64a3163b80 1 -- 172.21.15.92:0/2247242273 >> 172.21.15.92:0/2247242273 conn(0x55d8ce3a92c0 msgr2=0x55d8ce3e89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.646+0000 7f64a3163b80 1 -- 172.21.15.92:0/2247242273 shutdown_connections 2023-10-06T10:45:33.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.646+0000 7f64a3163b80 1 -- 172.21.15.92:0/2247242273 wait complete. 2023-10-06T10:45:33.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.646+0000 7f64a3163b80 1 -- start start 2023-10-06T10:45:33.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.646+0000 7f64a3163b80 1 Processor -- start 2023-10-06T10:45:33.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.647+0000 7f64a3163b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d8ce3a79a0 0x55d8ce453270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.647+0000 7f64a3163b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3d0eb0 0x55d8ce3a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.647+0000 7f64a3163b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d8ce3f1c10 0x55d8ce3ee110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.647+0000 7f64a3163b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ce3a4230 con 0x55d8ce3d0eb0 2023-10-06T10:45:33.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.647+0000 7f64a3163b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ce457720 con 0x55d8ce3f1c10 2023-10-06T10:45:33.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.647+0000 7f64a3163b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8ce3eef00 con 0x55d8ce3a79a0 2023-10-06T10:45:33.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.647+0000 7f64a3163b80 10 client.0 _ll_register_callbacks cb 0x55d8ce194f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:33.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.647+0000 7f6492ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3d0eb0 0x55d8ce3a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.647+0000 7f6492ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3d0eb0 0x55d8ce3a3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54934/0 (socket says 172.21.15.92:54934) 2023-10-06T10:45:33.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.647+0000 7f6492ffd700 1 -- 172.21.15.92:0/3068335966 learned_addr learned my addr 172.21.15.92:0/3068335966 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:33.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.647+0000 7f6493fff700 1 --2- 172.21.15.92:0/3068335966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d8ce3f1c10 0x55d8ce3ee110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.647+0000 7f64937fe700 1 --2- 172.21.15.92:0/3068335966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d8ce3a79a0 0x55d8ce453270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.648+0000 7f6492ffd700 1 -- 172.21.15.92:0/3068335966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d8ce3a79a0 msgr2=0x55d8ce453270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.648+0000 7f6492ffd700 1 --2- 172.21.15.92:0/3068335966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d8ce3a79a0 0x55d8ce453270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.648+0000 7f6492ffd700 1 -- 172.21.15.92:0/3068335966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d8ce3f1c10 msgr2=0x55d8ce3ee110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.648+0000 7f6492ffd700 1 --2- 172.21.15.92:0/3068335966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d8ce3f1c10 0x55d8ce3ee110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.648+0000 7f6492ffd700 1 -- 172.21.15.92:0/3068335966 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648c00a850 con 0x55d8ce3d0eb0 2023-10-06T10:45:33.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.648+0000 7f6492ffd700 1 --2- 172.21.15.92:0/3068335966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3d0eb0 0x55d8ce3a3c60 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f6488006c60 tx=0x7f6488014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1e12a0823b3e1e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.648+0000 7f6471ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:33.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.648+0000 7f6471ffb700 1 -- 172.21.15.92:0/3068335966 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6488020040 con 0x55d8ce3d0eb0 2023-10-06T10:45:33.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.648+0000 7f6471ffb700 1 -- 172.21.15.92:0/3068335966 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f6488029250 con 0x55d8ce3d0eb0 2023-10-06T10:45:33.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.648+0000 7f64a3163b80 1 -- 172.21.15.92:0/3068335966 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6488029d80 con 0x55d8ce3d0eb0 2023-10-06T10:45:33.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.648+0000 7f6471ffb700 1 -- 172.21.15.92:0/3068335966 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f648801a4a0 con 0x55d8ce3d0eb0 2023-10-06T10:45:33.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.648+0000 7f64a3163b80 10 client.9096 fetch_fsmap learned FSMap version 109 2023-10-06T10:45:33.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.648+0000 7f64a3163b80 1 -- 172.21.15.92:0/3068335966 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x55d8ce2c67d0 con 0x55d8ce3d0eb0 2023-10-06T10:45:33.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.648+0000 7f6471ffb700 1 -- 172.21.15.92:0/3068335966 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6488029d80 con 0x55d8ce3d0eb0 2023-10-06T10:45:33.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.648+0000 7f6471ffb700 1 -- 172.21.15.92:0/3068335966 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 109) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f64880298f0 con 0x55d8ce3d0eb0 2023-10-06T10:45:33.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.648+0000 7f64a3163b80 10 client.9096 fetch_fsmap finished waiting for FSMap version 109 2023-10-06T10:45:33.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.648+0000 7f64a3163b80 10 client.9096 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2023-10-06T10:45:33.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.648+0000 7f64a3163b80 20 client.9096 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:33.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.648+0000 7f64a3163b80 1 --2- 172.21.15.92:0/3068335966 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x55d8ce3cc980 0x55d8ce3ee650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.648+0000 7f64a3163b80 4 client.9096 mds_command: new command op to 8517 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:45:33.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.648+0000 7f64a3163b80 1 -- 172.21.15.92:0/3068335966 --> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d8ce29ae50 con 0x55d8ce3cc980 2023-10-06T10:45:33.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.648+0000 7f64937fe700 1 --2- 172.21.15.92:0/3068335966 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x55d8ce3cc980 0x55d8ce3ee650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.649+0000 7f6471ffb700 10 client.9096 ms_handle_connect on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:33.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.649+0000 7f64937fe700 1 --2- 172.21.15.92:0/3068335966 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x55d8ce3cc980 0x55d8ce3ee650 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e76115b59049080 server_cookie=1a21504a19b7b17e in_seq=0 out_seq=0 2023-10-06T10:45:33.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.649+0000 7f6471ffb700 1 -- 172.21.15.92:0/3068335966 <== mds.1 v2:172.21.15.92:6834/1979826278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55d8ce29ae50 con 0x55d8ce3cc980 2023-10-06T10:45:33.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.649+0000 7f6471ffb700 10 client.9096 handle_command_reply: tid=0 2023-10-06T10:45:33.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.649+0000 7f64a3163b80 1 client.9096 shutdown 2023-10-06T10:45:33.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.649+0000 7f64a3163b80 1 -- 172.21.15.92:0/3068335966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3d0eb0 msgr2=0x55d8ce3a3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.649+0000 7f64a3163b80 1 --2- 172.21.15.92:0/3068335966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3d0eb0 0x55d8ce3a3c60 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f6488006c60 tx=0x7f6488014000 comp rx=0 tx=0).stop 2023-10-06T10:45:33.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.649+0000 7f64a3163b80 1 -- 172.21.15.92:0/3068335966 shutdown_connections 2023-10-06T10:45:33.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.649+0000 7f64a3163b80 1 --2- 172.21.15.92:0/3068335966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d8ce3f1c10 0x55d8ce3ee110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.649+0000 7f64a3163b80 1 --2- 172.21.15.92:0/3068335966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d8ce3a79a0 0x55d8ce453270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.649+0000 7f64a3163b80 1 --2- 172.21.15.92:0/3068335966 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x55d8ce3cc980 0x55d8ce3ee650 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.649+0000 7f64a3163b80 1 --2- 172.21.15.92:0/3068335966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8ce3d0eb0 0x55d8ce3a3c60 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.649+0000 7f6471ffb700 0 client.9096 ms_handle_reset on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:33.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.649+0000 7f64a3163b80 1 -- 172.21.15.92:0/3068335966 >> 172.21.15.92:0/3068335966 conn(0x55d8ce3a92c0 msgr2=0x55d8ce39e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.649+0000 7f64a3163b80 1 -- 172.21.15.92:0/3068335966 shutdown_connections 2023-10-06T10:45:33.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.649+0000 7f64a3163b80 1 -- 172.21.15.92:0/3068335966 wait complete. 2023-10-06T10:45:33.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.649+0000 7f64a3163b80 20 client.9096 trim_cache size 0 max 16384 2023-10-06T10:45:33.658 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:33.658 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "PAUSED (0 inodes in the stack)", 2023-10-06T10:45:33.658 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:45:33.658 INFO:teuthology.orchestra.run.smithi092.stdout: "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8": { 2023-10-06T10:45:33.659 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:45:33.659 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8", 2023-10-06T10:45:33.659 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:45:33.659 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:33.659 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:33.659 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:33.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.650+0000 7f6498c09700 1 -- 172.21.15.92:0/895389781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6480066f30 msgr2=0x7f64800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.650+0000 7f6498c09700 1 --2- 172.21.15.92:0/895389781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6480066f30 0x7f64800693e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f64940cd7c0 tx=0x7f6488017000 comp rx=0 tx=0).stop 2023-10-06T10:45:33.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.650+0000 7f6498c09700 1 -- 172.21.15.92:0/895389781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64940b2620 msgr2=0x7f64941532f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.650+0000 7f6498c09700 1 --2- 172.21.15.92:0/895389781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64940b2620 0x7f64941532f0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f648400c8e0 tx=0x7f64840016f0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.650+0000 7f6498c09700 1 -- 172.21.15.92:0/895389781 shutdown_connections 2023-10-06T10:45:33.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.650+0000 7f6498c09700 1 --2- 172.21.15.92:0/895389781 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6480066f30 0x7f64800693e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.650+0000 7f6498c09700 1 --2- 172.21.15.92:0/895389781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64940cd3c0 0x7f6494068d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.650+0000 7f6498c09700 1 --2- 172.21.15.92:0/895389781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64940b2620 0x7f64941532f0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.650+0000 7f6498c09700 1 --2- 172.21.15.92:0/895389781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64940b29f0 0x7f649415bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.650+0000 7f6498c09700 1 -- 172.21.15.92:0/895389781 >> 172.21.15.92:0/895389781 conn(0x7f6494099520 msgr2=0x7f6494068110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.650+0000 7f6498c09700 1 -- 172.21.15.92:0/895389781 shutdown_connections 2023-10-06T10:45:33.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.650+0000 7f6498c09700 1 -- 172.21.15.92:0/895389781 wait complete. 2023-10-06T10:45:33.662 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:45:33.662 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:2 scrub status 2023-10-06T10:45:33.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.778+0000 7f41ac475700 1 Processor -- start 2023-10-06T10:45:33.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.778+0000 7f41ac475700 1 -- start start 2023-10-06T10:45:33.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.779+0000 7f41ac475700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f41a4150940 0x7f41a4159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.779+0000 7f41ac475700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41a415a160 0x7f41a40ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.779+0000 7f41ac475700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f41a40c5250 0x7f41a40cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.779+0000 7f41ac475700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a415ac30 con 0x7f41a4150940 2023-10-06T10:45:33.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.779+0000 7f41ac475700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a40cb6a0 con 0x7f41a40c5250 2023-10-06T10:45:33.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.779+0000 7f41ac475700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a40cb090 con 0x7f41a415a160 2023-10-06T10:45:33.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.779+0000 7f41aa211700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f41a4150940 0x7f41a4159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.779+0000 7f41aa211700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f41a4150940 0x7f41a4159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54948/0 (socket says 172.21.15.92:54948) 2023-10-06T10:45:33.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.779+0000 7f41aa211700 1 -- 172.21.15.92:0/3725638704 learned_addr learned my addr 172.21.15.92:0/3725638704 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:33.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.779+0000 7f41aaa12700 1 --2- 172.21.15.92:0/3725638704 >> v2:172.21.15.124:3300/0 conn(0x7f41a40c5250 0x7f41a40cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.779+0000 7f41a9a10700 1 --2- 172.21.15.92:0/3725638704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41a415a160 0x7f41a40ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.780+0000 7f41aa211700 1 -- 172.21.15.92:0/3725638704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41a415a160 msgr2=0x7f41a40ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.780+0000 7f41aa211700 1 --2- 172.21.15.92:0/3725638704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41a415a160 0x7f41a40ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.780+0000 7f41aa211700 1 -- 172.21.15.92:0/3725638704 >> v2:172.21.15.124:3300/0 conn(0x7f41a40c5250 msgr2=0x7f41a40cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.780+0000 7f41aa211700 1 --2- 172.21.15.92:0/3725638704 >> v2:172.21.15.124:3300/0 conn(0x7f41a40c5250 0x7f41a40cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.780+0000 7f41aa211700 1 -- 172.21.15.92:0/3725638704 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a415a670 con 0x7f41a4150940 2023-10-06T10:45:33.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.780+0000 7f41aa211700 1 --2- 172.21.15.92:0/3725638704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41a4150940 0x7f41a4159c20 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f41a4071190 tx=0x7f41a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7883a6b5b45f1679 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.780+0000 7f41a920f700 1 -- 172.21.15.92:0/3725638704 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a000e070 con 0x7f41a4150940 2023-10-06T10:45:33.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.780+0000 7f41a920f700 1 -- 172.21.15.92:0/3725638704 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a000bad0 con 0x7f41a4150940 2023-10-06T10:45:33.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.780+0000 7f41a920f700 1 -- 172.21.15.92:0/3725638704 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a00092e0 con 0x7f41a4150940 2023-10-06T10:45:33.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.780+0000 7f41ac475700 1 -- 172.21.15.92:0/3725638704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41a4150940 msgr2=0x7f41a4159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.780+0000 7f41ac475700 1 --2- 172.21.15.92:0/3725638704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41a4150940 0x7f41a4159c20 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f41a4071190 tx=0x7f41a0009840 comp rx=0 tx=0).stop 2023-10-06T10:45:33.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.780+0000 7f41ac475700 1 -- 172.21.15.92:0/3725638704 shutdown_connections 2023-10-06T10:45:33.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.780+0000 7f41ac475700 1 --2- 172.21.15.92:0/3725638704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41a4150940 0x7f41a4159c20 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.780+0000 7f41ac475700 1 --2- 172.21.15.92:0/3725638704 >> v2:172.21.15.124:3300/0 conn(0x7f41a40c5250 0x7f41a40cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.780+0000 7f41ac475700 1 --2- 172.21.15.92:0/3725638704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41a415a160 0x7f41a40ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.780+0000 7f41ac475700 1 -- 172.21.15.92:0/3725638704 >> 172.21.15.92:0/3725638704 conn(0x7f41a4099700 msgr2=0x7f41a406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.780+0000 7f41ac475700 1 -- 172.21.15.92:0/3725638704 shutdown_connections 2023-10-06T10:45:33.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.781+0000 7f41ac475700 1 -- 172.21.15.92:0/3725638704 wait complete. 2023-10-06T10:45:33.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.782+0000 7f41ac475700 1 Processor -- start 2023-10-06T10:45:33.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.782+0000 7f41ac475700 1 -- start start 2023-10-06T10:45:33.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.782+0000 7f41ac475700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41a40c5250 0x7f41a40b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.782+0000 7f41ac475700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41a415a160 0x7f41a40c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.782+0000 7f41ac475700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41a40b4770 0x7f41a40c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.782+0000 7f41ac475700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a40c92e0 con 0x7f41a40b4770 2023-10-06T10:45:33.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.782+0000 7f41ac475700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a40c86d0 con 0x7f41a415a160 2023-10-06T10:45:33.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.782+0000 7f41ac475700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a40c89b0 con 0x7f41a40c5250 2023-10-06T10:45:33.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.782+0000 7f41aaa12700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41a40b4770 0x7f41a40c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.782+0000 7f41aaa12700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41a40b4770 0x7f41a40c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54954/0 (socket says 172.21.15.92:54954) 2023-10-06T10:45:33.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.782+0000 7f41aaa12700 1 -- 172.21.15.92:0/62209967 learned_addr learned my addr 172.21.15.92:0/62209967 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.782+0000 7f41a9a10700 1 --2- 172.21.15.92:0/62209967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41a415a160 0x7f41a40c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.782+0000 7f41aa211700 1 --2- 172.21.15.92:0/62209967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41a40c5250 0x7f41a40b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.783+0000 7f41aaa12700 1 -- 172.21.15.92:0/62209967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41a40c5250 msgr2=0x7f41a40b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.783+0000 7f41aaa12700 1 --2- 172.21.15.92:0/62209967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41a40c5250 0x7f41a40b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.783+0000 7f41aaa12700 1 -- 172.21.15.92:0/62209967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41a415a160 msgr2=0x7f41a40c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.783+0000 7f41aaa12700 1 --2- 172.21.15.92:0/62209967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41a415a160 0x7f41a40c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.783+0000 7f41aaa12700 1 -- 172.21.15.92:0/62209967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a0009ae0 con 0x7f41a40b4770 2023-10-06T10:45:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.783+0000 7f41aaa12700 1 --2- 172.21.15.92:0/62209967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41a40b4770 0x7f41a40c9cb0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f419c00d890 tx=0x7f419c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.783+0000 7f419a7fc700 1 -- 172.21.15.92:0/62209967 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c0125f0 con 0x7f41a40b4770 2023-10-06T10:45:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.783+0000 7f419a7fc700 1 -- 172.21.15.92:0/62209967 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419c012770 con 0x7f41a40b4770 2023-10-06T10:45:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.783+0000 7f41ac475700 1 -- 172.21.15.92:0/62209967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a40c8c90 con 0x7f41a40b4770 2023-10-06T10:45:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.783+0000 7f419a7fc700 1 -- 172.21.15.92:0/62209967 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c012b30 con 0x7f41a40b4770 2023-10-06T10:45:33.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.783+0000 7f41ac475700 1 -- 172.21.15.92:0/62209967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a414fd20 con 0x7f41a40b4770 2023-10-06T10:45:33.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.784+0000 7f419a7fc700 1 -- 172.21.15.92:0/62209967 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f419c01e460 con 0x7f41a40b4770 2023-10-06T10:45:33.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.785+0000 7f419a7fc700 1 --2- 172.21.15.92:0/62209967 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4188066f30 0x7f41880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.785+0000 7f419a7fc700 1 -- 172.21.15.92:0/62209967 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f419c084cd0 con 0x7f41a40b4770 2023-10-06T10:45:33.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.785+0000 7f41aa211700 1 --2- 172.21.15.92:0/62209967 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4188066f30 0x7f41880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.785+0000 7f41aa211700 1 --2- 172.21.15.92:0/62209967 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4188066f30 0x7f41880693e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f41a0009fd0 tx=0x7f41a0018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.787+0000 7f41b576db80 1 Processor -- start 2023-10-06T10:45:33.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.787+0000 7f41b576db80 1 -- start start 2023-10-06T10:45:33.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.788+0000 7f41b576db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563573222ae0 0x563573222eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.788+0000 7f41b576db80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635731de9f0 0x5635731db5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.788+0000 7f41b576db80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635731dcc20 0x5635731dbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.788+0000 7f41b576db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357315bcd0 con 0x563573222ae0 2023-10-06T10:45:33.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.788+0000 7f41b576db80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563573132ac0 con 0x5635731dcc20 2023-10-06T10:45:33.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.788+0000 7f41b576db80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635731302c0 con 0x5635731de9f0 2023-10-06T10:45:33.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.788+0000 7f41aa211700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563573222ae0 0x563573222eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.788+0000 7f41aa211700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563573222ae0 0x563573222eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54958/0 (socket says 172.21.15.92:54958) 2023-10-06T10:45:33.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.788+0000 7f41aa211700 1 -- 172.21.15.92:0/170668072 learned_addr learned my addr 172.21.15.92:0/170668072 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:33.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.788+0000 7f41a9a10700 1 --2- 172.21.15.92:0/170668072 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635731de9f0 0x5635731db5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.788+0000 7f41aaa12700 1 --2- 172.21.15.92:0/170668072 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635731dcc20 0x5635731dbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.788+0000 7f41aa211700 1 -- 172.21.15.92:0/170668072 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635731de9f0 msgr2=0x5635731db5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.788+0000 7f41aa211700 1 --2- 172.21.15.92:0/170668072 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635731de9f0 0x5635731db5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.788+0000 7f41aa211700 1 -- 172.21.15.92:0/170668072 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635731dcc20 msgr2=0x5635731dbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.788+0000 7f41aa211700 1 --2- 172.21.15.92:0/170668072 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635731dcc20 0x5635731dbb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.788+0000 7f41aa211700 1 -- 172.21.15.92:0/170668072 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563572fc62c0 con 0x563573222ae0 2023-10-06T10:45:33.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.788+0000 7f41a9a10700 1 --2- 172.21.15.92:0/170668072 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635731de9f0 0x5635731db5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:33.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.788+0000 7f41aa211700 1 --2- 172.21.15.92:0/170668072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563573222ae0 0x563573222eb0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f41a000bc40 tx=0x7f41a0020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bbb35a8b094107c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.788+0000 7f41877fe700 1 -- 172.21.15.92:0/170668072 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a000bea0 con 0x563573222ae0 2023-10-06T10:45:33.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.788+0000 7f41877fe700 1 -- 172.21.15.92:0/170668072 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a0007aa0 con 0x563573222ae0 2023-10-06T10:45:33.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.788+0000 7f41877fe700 1 -- 172.21.15.92:0/170668072 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a0007e40 con 0x563573222ae0 2023-10-06T10:45:33.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.789+0000 7f41b576db80 1 -- 172.21.15.92:0/170668072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563573222ae0 msgr2=0x563573222eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.789+0000 7f41b576db80 1 --2- 172.21.15.92:0/170668072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563573222ae0 0x563573222eb0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f41a000bc40 tx=0x7f41a0020040 comp rx=0 tx=0).stop 2023-10-06T10:45:33.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.789+0000 7f41b576db80 1 -- 172.21.15.92:0/170668072 shutdown_connections 2023-10-06T10:45:33.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.789+0000 7f41b576db80 1 --2- 172.21.15.92:0/170668072 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635731dcc20 0x5635731dbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.789+0000 7f41b576db80 1 --2- 172.21.15.92:0/170668072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563573222ae0 0x563573222eb0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.789+0000 7f41b576db80 1 --2- 172.21.15.92:0/170668072 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635731de9f0 0x5635731db5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.789+0000 7f41b576db80 1 -- 172.21.15.92:0/170668072 >> 172.21.15.92:0/170668072 conn(0x5635731dd0c0 msgr2=0x5635731dd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.789+0000 7f41b576db80 1 -- 172.21.15.92:0/170668072 shutdown_connections 2023-10-06T10:45:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.789+0000 7f41b576db80 1 -- 172.21.15.92:0/170668072 wait complete. 2023-10-06T10:45:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.789+0000 7f41b576db80 1 -- start start 2023-10-06T10:45:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.790+0000 7f41b576db80 1 Processor -- start 2023-10-06T10:45:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.790+0000 7f41b576db80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635731dcc20 0x56357321c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.790+0000 7f41b576db80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635731de9f0 0x563573203bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.790+0000 7f41b576db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563573222ae0 0x563573204130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.790+0000 7f41b576db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357312c0b0 con 0x563573222ae0 2023-10-06T10:45:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.790+0000 7f41b576db80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635730d78e0 con 0x5635731de9f0 2023-10-06T10:45:33.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.790+0000 7f41b576db80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635730f2ef0 con 0x5635731dcc20 2023-10-06T10:45:33.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.791+0000 7f41b576db80 10 client.0 _ll_register_callbacks cb 0x563572fc8f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:33.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.791+0000 7f41aaa12700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563573222ae0 0x563573204130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.791+0000 7f41aaa12700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563573222ae0 0x563573204130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54970/0 (socket says 172.21.15.92:54970) 2023-10-06T10:45:33.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.791+0000 7f41aaa12700 1 -- 172.21.15.92:0/1946111577 learned_addr learned my addr 172.21.15.92:0/1946111577 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:33.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.791+0000 7f41aa211700 1 --2- 172.21.15.92:0/1946111577 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635731dcc20 0x56357321c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.791+0000 7f41a9a10700 1 --2- 172.21.15.92:0/1946111577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635731de9f0 0x563573203bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.791+0000 7f41aaa12700 1 -- 172.21.15.92:0/1946111577 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635731dcc20 msgr2=0x56357321c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.791+0000 7f41aaa12700 1 --2- 172.21.15.92:0/1946111577 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635731dcc20 0x56357321c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.791+0000 7f41aaa12700 1 -- 172.21.15.92:0/1946111577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635731de9f0 msgr2=0x563573203bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.791+0000 7f41aaa12700 1 --2- 172.21.15.92:0/1946111577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635731de9f0 0x563573203bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.791+0000 7f41aaa12700 1 -- 172.21.15.92:0/1946111577 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a415a670 con 0x563573222ae0 2023-10-06T10:45:33.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.791+0000 7f41aaa12700 1 --2- 172.21.15.92:0/1946111577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563573222ae0 0x563573204130 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f419c006fd0 tx=0x7f419c082e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175c070d58364c04 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.791+0000 7f4184ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:33.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.791+0000 7f4184ff9700 1 -- 172.21.15.92:0/1946111577 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c055460 con 0x563573222ae0 2023-10-06T10:45:33.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.791+0000 7f4184ff9700 1 -- 172.21.15.92:0/1946111577 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f419c055920 con 0x563573222ae0 2023-10-06T10:45:33.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.791+0000 7f41b576db80 1 -- 172.21.15.92:0/1946111577 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635731d06e0 con 0x563573222ae0 2023-10-06T10:45:33.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.791+0000 7f4184ff9700 1 -- 172.21.15.92:0/1946111577 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f419c058b70 con 0x563573222ae0 2023-10-06T10:45:33.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.791+0000 7f41b576db80 10 client.9108 fetch_fsmap learned FSMap version 109 2023-10-06T10:45:33.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.791+0000 7f41b576db80 1 -- 172.21.15.92:0/1946111577 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x563572f69030 con 0x563573222ae0 2023-10-06T10:45:33.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.791+0000 7f4184ff9700 1 -- 172.21.15.92:0/1946111577 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5635731d06e0 con 0x563573222ae0 2023-10-06T10:45:33.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.791+0000 7f4184ff9700 1 -- 172.21.15.92:0/1946111577 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 109) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f419c05c430 con 0x563573222ae0 2023-10-06T10:45:33.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.792+0000 7f41b576db80 10 client.9108 fetch_fsmap finished waiting for FSMap version 109 2023-10-06T10:45:33.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.792+0000 7f41b576db80 10 client.9108 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2023-10-06T10:45:33.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.792+0000 7f41b576db80 20 client.9108 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:33.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.792+0000 7f41b576db80 1 --2- 172.21.15.92:0/1946111577 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x56357321c280 0x563573201720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.792+0000 7f41b576db80 4 client.9108 mds_command: new command op to 6719 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:33.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.792+0000 7f41b576db80 1 -- 172.21.15.92:0/1946111577 --> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635731ad610 con 0x56357321c280 2023-10-06T10:45:33.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.792+0000 7f41aa211700 1 --2- 172.21.15.92:0/1946111577 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x56357321c280 0x563573201720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.792+0000 7f4184ff9700 10 client.9108 ms_handle_connect on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:33.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.792+0000 7f41aa211700 1 --2- 172.21.15.92:0/1946111577 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x56357321c280 0x563573201720 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e8920285f1855c5 server_cookie=58d6f5fe2e05b8cb in_seq=0 out_seq=0 2023-10-06T10:45:33.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.793+0000 7f4184ff9700 1 -- 172.21.15.92:0/1946111577 <== mds.2 v2:172.21.15.124:6833/2307521923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5635731ad610 con 0x56357321c280 2023-10-06T10:45:33.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.793+0000 7f4184ff9700 10 client.9108 handle_command_reply: tid=0 2023-10-06T10:45:33.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.793+0000 7f41b576db80 1 client.9108 shutdown 2023-10-06T10:45:33.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.793+0000 7f41b576db80 1 -- 172.21.15.92:0/1946111577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563573222ae0 msgr2=0x563573204130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.793+0000 7f41b576db80 1 --2- 172.21.15.92:0/1946111577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563573222ae0 0x563573204130 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f419c006fd0 tx=0x7f419c082e70 comp rx=0 tx=0).stop 2023-10-06T10:45:33.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.793+0000 7f41b576db80 1 -- 172.21.15.92:0/1946111577 shutdown_connections 2023-10-06T10:45:33.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.793+0000 7f41b576db80 1 --2- 172.21.15.92:0/1946111577 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x56357321c280 0x563573201720 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.793+0000 7f41b576db80 1 --2- 172.21.15.92:0/1946111577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563573222ae0 0x563573204130 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.793+0000 7f4184ff9700 0 client.9108 ms_handle_reset on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:33.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.793+0000 7f41b576db80 1 --2- 172.21.15.92:0/1946111577 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635731dcc20 0x56357321c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.793+0000 7f41b576db80 1 --2- 172.21.15.92:0/1946111577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5635731de9f0 0x563573203bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.793+0000 7f41b576db80 1 -- 172.21.15.92:0/1946111577 >> 172.21.15.92:0/1946111577 conn(0x5635731dd0c0 msgr2=0x5635731d8840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.793+0000 7f41b576db80 1 -- 172.21.15.92:0/1946111577 shutdown_connections 2023-10-06T10:45:33.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.793+0000 7f41b576db80 1 -- 172.21.15.92:0/1946111577 wait complete. 2023-10-06T10:45:33.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.793+0000 7f41b576db80 20 client.9108 trim_cache size 0 max 16384 2023-10-06T10:45:33.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.809+0000 7f41b576db80 1 Processor -- start 2023-10-06T10:45:33.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.809+0000 7f41b576db80 1 -- start start 2023-10-06T10:45:33.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.809+0000 7f41b576db80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635731dcc20 0x563573220490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.809+0000 7f41b576db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635731de9f0 0x563573201720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.809+0000 7f41b576db80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563573222ae0 0x563573201c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.809+0000 7f41b576db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563572faee80 con 0x5635731de9f0 2023-10-06T10:45:33.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.809+0000 7f41b576db80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635731d6ce0 con 0x563573222ae0 2023-10-06T10:45:33.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.809+0000 7f41b576db80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563573220300 con 0x5635731dcc20 2023-10-06T10:45:33.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.810+0000 7f41a9a10700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635731de9f0 0x563573201720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.810+0000 7f41a9a10700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635731de9f0 0x563573201720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54980/0 (socket says 172.21.15.92:54980) 2023-10-06T10:45:33.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.810+0000 7f41a9a10700 1 -- 172.21.15.92:0/2368613573 learned_addr learned my addr 172.21.15.92:0/2368613573 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:33.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.810+0000 7f41aaa12700 1 --2- 172.21.15.92:0/2368613573 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563573222ae0 0x563573201c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.810+0000 7f41aa211700 1 --2- 172.21.15.92:0/2368613573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635731dcc20 0x563573220490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.810+0000 7f41a9a10700 1 -- 172.21.15.92:0/2368613573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635731dcc20 msgr2=0x563573220490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.810+0000 7f41a9a10700 1 --2- 172.21.15.92:0/2368613573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635731dcc20 0x563573220490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.810+0000 7f41a9a10700 1 -- 172.21.15.92:0/2368613573 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563573222ae0 msgr2=0x563573201c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.810+0000 7f41a9a10700 1 --2- 172.21.15.92:0/2368613573 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563573222ae0 0x563573201c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.810+0000 7f41a9a10700 1 -- 172.21.15.92:0/2368613573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635730fa7d0 con 0x5635731de9f0 2023-10-06T10:45:33.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.810+0000 7f41aaa12700 1 --2- 172.21.15.92:0/2368613573 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563573222ae0 0x563573201c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:33.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.810+0000 7f41aa211700 1 --2- 172.21.15.92:0/2368613573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635731dcc20 0x563573220490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:33.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.810+0000 7f41a9a10700 1 --2- 172.21.15.92:0/2368613573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635731de9f0 0x563573201720 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f419400bb10 tx=0x7f4194008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858dbe8fa37ed884 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.810+0000 7f4187fff700 1 -- 172.21.15.92:0/2368613573 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4194014870 con 0x5635731de9f0 2023-10-06T10:45:33.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.810+0000 7f4187fff700 1 -- 172.21.15.92:0/2368613573 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41940149d0 con 0x5635731de9f0 2023-10-06T10:45:33.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.810+0000 7f4187fff700 1 -- 172.21.15.92:0/2368613573 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4194014d70 con 0x5635731de9f0 2023-10-06T10:45:33.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.811+0000 7f41b576db80 1 -- 172.21.15.92:0/2368613573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635731de9f0 msgr2=0x563573201720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.811+0000 7f41b576db80 1 --2- 172.21.15.92:0/2368613573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635731de9f0 0x563573201720 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f419400bb10 tx=0x7f4194008040 comp rx=0 tx=0).stop 2023-10-06T10:45:33.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.811+0000 7f41b576db80 1 -- 172.21.15.92:0/2368613573 shutdown_connections 2023-10-06T10:45:33.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.811+0000 7f41b576db80 1 --2- 172.21.15.92:0/2368613573 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563573222ae0 0x563573201c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.811+0000 7f41b576db80 1 --2- 172.21.15.92:0/2368613573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635731dcc20 0x563573220490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.811+0000 7f41b576db80 1 --2- 172.21.15.92:0/2368613573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635731de9f0 0x563573201720 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.811+0000 7f41b576db80 1 -- 172.21.15.92:0/2368613573 >> 172.21.15.92:0/2368613573 conn(0x5635731dd0c0 msgr2=0x5635731d8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.811+0000 7f41b576db80 1 -- 172.21.15.92:0/2368613573 shutdown_connections 2023-10-06T10:45:33.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.811+0000 7f41b576db80 1 -- 172.21.15.92:0/2368613573 wait complete. 2023-10-06T10:45:33.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.811+0000 7f41b576db80 1 -- start start 2023-10-06T10:45:33.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.811+0000 7f41b576db80 1 Processor -- start 2023-10-06T10:45:33.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.812+0000 7f41b576db80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635731dcc20 0x56357328acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.812+0000 7f41b576db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635731de9f0 0x563573227eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.812+0000 7f41b576db80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563573222ae0 0x563573203150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.812+0000 7f41b576db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563573228880 con 0x5635731de9f0 2023-10-06T10:45:33.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.812+0000 7f41b576db80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635731db8f0 con 0x563573222ae0 2023-10-06T10:45:33.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.812+0000 7f41b576db80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563573202cc0 con 0x5635731dcc20 2023-10-06T10:45:33.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.812+0000 7f41b576db80 10 client.0 _ll_register_callbacks cb 0x563572fc8f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:33.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.812+0000 7f41a9a10700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635731de9f0 0x563573227eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.812+0000 7f41a9a10700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635731de9f0 0x563573227eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54982/0 (socket says 172.21.15.92:54982) 2023-10-06T10:45:33.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.812+0000 7f41a9a10700 1 -- 172.21.15.92:0/3635596619 learned_addr learned my addr 172.21.15.92:0/3635596619 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:33.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.812+0000 7f41aa211700 1 --2- 172.21.15.92:0/3635596619 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635731dcc20 0x56357328acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.812+0000 7f41aaa12700 1 --2- 172.21.15.92:0/3635596619 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563573222ae0 0x563573203150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f41a9a10700 1 -- 172.21.15.92:0/3635596619 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635731dcc20 msgr2=0x56357328acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f41a9a10700 1 --2- 172.21.15.92:0/3635596619 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635731dcc20 0x56357328acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f41a9a10700 1 -- 172.21.15.92:0/3635596619 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563573222ae0 msgr2=0x563573203150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f41a9a10700 1 --2- 172.21.15.92:0/3635596619 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563573222ae0 0x563573203150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f41a9a10700 1 -- 172.21.15.92:0/3635596619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419400e040 con 0x5635731de9f0 2023-10-06T10:45:33.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f41aaa12700 1 --2- 172.21.15.92:0/3635596619 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563573222ae0 0x563573203150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:33.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f41a9a10700 1 --2- 172.21.15.92:0/3635596619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635731de9f0 0x563573227eb0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f419401b000 tx=0x7f4194021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b6396bb212a958 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f4184ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:33.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f4184ff9700 1 -- 172.21.15.92:0/3635596619 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419401f880 con 0x5635731de9f0 2023-10-06T10:45:33.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f4184ff9700 1 -- 172.21.15.92:0/3635596619 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f419401f9e0 con 0x5635731de9f0 2023-10-06T10:45:33.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f41b576db80 1 -- 172.21.15.92:0/3635596619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41a0007aa0 con 0x5635731de9f0 2023-10-06T10:45:33.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f4184ff9700 1 -- 172.21.15.92:0/3635596619 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f419401fdb0 con 0x5635731de9f0 2023-10-06T10:45:33.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f41b576db80 10 client.9114 fetch_fsmap learned FSMap version 109 2023-10-06T10:45:33.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f41b576db80 1 -- 172.21.15.92:0/3635596619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x5635730e14c0 con 0x5635731de9f0 2023-10-06T10:45:33.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f4184ff9700 1 -- 172.21.15.92:0/3635596619 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a0007aa0 con 0x5635731de9f0 2023-10-06T10:45:33.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f4184ff9700 1 -- 172.21.15.92:0/3635596619 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 109) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f4194029070 con 0x5635731de9f0 2023-10-06T10:45:33.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f41b576db80 10 client.9114 fetch_fsmap finished waiting for FSMap version 109 2023-10-06T10:45:33.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f41b576db80 10 client.9114 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2023-10-06T10:45:33.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f41b576db80 20 client.9114 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:33.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f41b576db80 1 --2- 172.21.15.92:0/3635596619 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x563573203690 0x563573201720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f41b576db80 4 client.9114 mds_command: new command op to 6719 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:45:33.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f41b576db80 1 -- 172.21.15.92:0/3635596619 --> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5635730cee50 con 0x563573203690 2023-10-06T10:45:33.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.813+0000 7f41aa211700 1 --2- 172.21.15.92:0/3635596619 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x563573203690 0x563573201720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.814+0000 7f4184ff9700 10 client.9114 ms_handle_connect on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:33.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.814+0000 7f41aa211700 1 --2- 172.21.15.92:0/3635596619 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x563573203690 0x563573201720 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d2ba7db8d8416b1f server_cookie=48fb103611dfa311 in_seq=0 out_seq=0 2023-10-06T10:45:33.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.814+0000 7f4184ff9700 1 -- 172.21.15.92:0/3635596619 <== mds.2 v2:172.21.15.124:6833/2307521923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5635730cee50 con 0x563573203690 2023-10-06T10:45:33.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.814+0000 7f4184ff9700 10 client.9114 handle_command_reply: tid=0 2023-10-06T10:45:33.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.814+0000 7f41b576db80 1 client.9114 shutdown 2023-10-06T10:45:33.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.814+0000 7f41b576db80 1 -- 172.21.15.92:0/3635596619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635731de9f0 msgr2=0x563573227eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.814+0000 7f41b576db80 1 --2- 172.21.15.92:0/3635596619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635731de9f0 0x563573227eb0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f419401b000 tx=0x7f4194021000 comp rx=0 tx=0).stop 2023-10-06T10:45:33.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.814+0000 7f41b576db80 1 -- 172.21.15.92:0/3635596619 shutdown_connections 2023-10-06T10:45:33.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.814+0000 7f41b576db80 1 --2- 172.21.15.92:0/3635596619 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x563573203690 0x563573201720 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.814+0000 7f41b576db80 1 --2- 172.21.15.92:0/3635596619 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563573222ae0 0x563573203150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.814+0000 7f4184ff9700 0 client.9114 ms_handle_reset on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:33.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.814+0000 7f41b576db80 1 --2- 172.21.15.92:0/3635596619 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5635731dcc20 0x56357328acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.814+0000 7f41b576db80 1 --2- 172.21.15.92:0/3635596619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5635731de9f0 0x563573227eb0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.814+0000 7f41b576db80 1 -- 172.21.15.92:0/3635596619 >> 172.21.15.92:0/3635596619 conn(0x5635731dd0c0 msgr2=0x563573221400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.814+0000 7f41b576db80 1 -- 172.21.15.92:0/3635596619 shutdown_connections 2023-10-06T10:45:33.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.814+0000 7f41b576db80 1 -- 172.21.15.92:0/3635596619 wait complete. 2023-10-06T10:45:33.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.814+0000 7f41b576db80 20 client.9114 trim_cache size 0 max 16384 2023-10-06T10:45:33.824 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:33.824 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "PAUSED (0 inodes in the stack)", 2023-10-06T10:45:33.824 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:45:33.824 INFO:teuthology.orchestra.run.smithi092.stdout: "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8": { 2023-10-06T10:45:33.824 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:45:33.824 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8", 2023-10-06T10:45:33.824 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:45:33.824 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:33.824 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:33.825 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:33.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.815+0000 7f41ac475700 1 -- 172.21.15.92:0/62209967 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4188066f30 msgr2=0x7f41880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.815+0000 7f41ac475700 1 --2- 172.21.15.92:0/62209967 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4188066f30 0x7f41880693e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f41a0009fd0 tx=0x7f41a0018000 comp rx=0 tx=0).stop 2023-10-06T10:45:33.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.815+0000 7f41ac475700 1 -- 172.21.15.92:0/62209967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41a40b4770 msgr2=0x7f41a40c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.815+0000 7f41ac475700 1 --2- 172.21.15.92:0/62209967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41a40b4770 0x7f41a40c9cb0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f419c00d890 tx=0x7f419c008040 comp rx=0 tx=0).stop 2023-10-06T10:45:33.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.815+0000 7f41ac475700 1 -- 172.21.15.92:0/62209967 shutdown_connections 2023-10-06T10:45:33.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.815+0000 7f41ac475700 1 --2- 172.21.15.92:0/62209967 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4188066f30 0x7f41880693e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.815+0000 7f41ac475700 1 --2- 172.21.15.92:0/62209967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41a40b4770 0x7f41a40c9cb0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.815+0000 7f41ac475700 1 --2- 172.21.15.92:0/62209967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41a40c5250 0x7f41a40b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.815+0000 7f41ac475700 1 --2- 172.21.15.92:0/62209967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41a415a160 0x7f41a40c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.815+0000 7f41ac475700 1 -- 172.21.15.92:0/62209967 >> 172.21.15.92:0/62209967 conn(0x7f41a4099700 msgr2=0x7f41a40ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.815+0000 7f41ac475700 1 -- 172.21.15.92:0/62209967 shutdown_connections 2023-10-06T10:45:33.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.815+0000 7f41ac475700 1 -- 172.21.15.92:0/62209967 wait complete. 2023-10-06T10:45:33.827 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:45:33.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub status 2023-10-06T10:45:33.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.943+0000 7fd8cf764700 1 Processor -- start 2023-10-06T10:45:33.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.943+0000 7fd8cf764700 1 -- start start 2023-10-06T10:45:33.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.944+0000 7fd8cf764700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd8c8150940 0x7fd8c8159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.944+0000 7fd8cf764700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8c80cd4d0 0x7fd8c80b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.944+0000 7fd8cf764700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c815ac30 con 0x7fd8c815a160 2023-10-06T10:45:33.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.944+0000 7fd8cf764700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c80cb6a0 con 0x7fd8c8150940 2023-10-06T10:45:33.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.944+0000 7fd8cf764700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c80cb090 con 0x7fd8c80cd4d0 2023-10-06T10:45:33.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.944+0000 7fd8cd500700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd8c8150940 0x7fd8c8159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.944+0000 7fd8cccff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd8c815a160 0x7fd8c80ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45514/0 (socket says 172.21.15.92:45514) 2023-10-06T10:45:33.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.944+0000 7fd8cccff700 1 -- 172.21.15.92:0/1496890877 learned_addr learned my addr 172.21.15.92:0/1496890877 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:33.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.944+0000 7fd8cdd01700 1 --2- 172.21.15.92:0/1496890877 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8c80cd4d0 0x7fd8c80b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.944+0000 7fd8cd500700 1 -- 172.21.15.92:0/1496890877 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8c80cd4d0 msgr2=0x7fd8c80b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.944+0000 7fd8cd500700 1 --2- 172.21.15.92:0/1496890877 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8c80cd4d0 0x7fd8c80b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.944+0000 7fd8cd500700 1 -- 172.21.15.92:0/1496890877 >> v1:172.21.15.92:6789/0 conn(0x7fd8c815a160 legacy=0x7fd8c80ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.944+0000 7fd8bffff700 1 -- 172.21.15.92:0/1496890877 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1638532390 0 0) 0x7fd8b8001110 con 0x7fd8c815a160 2023-10-06T10:45:33.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.944+0000 7fd8cd500700 1 -- 172.21.15.92:0/1496890877 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c815a670 con 0x7fd8c8150940 2023-10-06T10:45:33.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.944+0000 7fd8cd500700 1 --2- 172.21.15.92:0/1496890877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8c8150940 0x7fd8c8159c20 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fd8c00098d0 tx=0x7fd8c0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ddd64a40c80bc58 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.944+0000 7fd8bffff700 1 -- 172.21.15.92:0/1496890877 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0009e70 con 0x7fd8c8150940 2023-10-06T10:45:33.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.944+0000 7fd8bffff700 1 -- 172.21.15.92:0/1496890877 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c0005c10 con 0x7fd8c8150940 2023-10-06T10:45:33.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.944+0000 7fd8bffff700 1 -- 172.21.15.92:0/1496890877 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c000b650 con 0x7fd8c8150940 2023-10-06T10:45:33.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.945+0000 7fd8cf764700 1 -- 172.21.15.92:0/1496890877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8c8150940 msgr2=0x7fd8c8159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.945+0000 7fd8cf764700 1 --2- 172.21.15.92:0/1496890877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8c8150940 0x7fd8c8159c20 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fd8c00098d0 tx=0x7fd8c0009120 comp rx=0 tx=0).stop 2023-10-06T10:45:33.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.945+0000 7fd8cf764700 1 -- 172.21.15.92:0/1496890877 shutdown_connections 2023-10-06T10:45:33.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.945+0000 7fd8cf764700 1 --2- 172.21.15.92:0/1496890877 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8c80cd4d0 0x7fd8c80b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.945+0000 7fd8cf764700 1 --2- 172.21.15.92:0/1496890877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8c8150940 0x7fd8c8159c20 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.945+0000 7fd8cf764700 1 -- 172.21.15.92:0/1496890877 >> 172.21.15.92:0/1496890877 conn(0x7fd8c8099700 msgr2=0x7fd8c806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.945+0000 7fd8cf764700 1 -- 172.21.15.92:0/1496890877 shutdown_connections 2023-10-06T10:45:33.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.945+0000 7fd8cf764700 1 -- 172.21.15.92:0/1496890877 wait complete. 2023-10-06T10:45:33.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.946+0000 7fd8cf764700 1 Processor -- start 2023-10-06T10:45:33.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.946+0000 7fd8cf764700 1 -- start start 2023-10-06T10:45:33.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.947+0000 7fd8cf764700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8c80cd4d0 0x7fd8c8147390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.947+0000 7fd8cf764700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8c815a160 0x7fd8c80b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.947+0000 7fd8cf764700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8c80b5640 0x7fd8c8155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.947+0000 7fd8cf764700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c80b5ae0 con 0x7fd8c80b5640 2023-10-06T10:45:33.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.947+0000 7fd8cf764700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c816d100 con 0x7fd8c80cd4d0 2023-10-06T10:45:33.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.947+0000 7fd8cf764700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c816d3e0 con 0x7fd8c815a160 2023-10-06T10:45:33.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.947+0000 7fd8cdd01700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8c80b5640 0x7fd8c8155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.947+0000 7fd8cdd01700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8c80b5640 0x7fd8c8155380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54986/0 (socket says 172.21.15.92:54986) 2023-10-06T10:45:33.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.947+0000 7fd8cdd01700 1 -- 172.21.15.92:0/1777936820 learned_addr learned my addr 172.21.15.92:0/1777936820 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:33.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.947+0000 7fd8cccff700 1 --2- 172.21.15.92:0/1777936820 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8c815a160 0x7fd8c80b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.947+0000 7fd8cdd01700 1 -- 172.21.15.92:0/1777936820 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8c815a160 msgr2=0x7fd8c80b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.947+0000 7fd8cdd01700 1 --2- 172.21.15.92:0/1777936820 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8c815a160 0x7fd8c80b5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.947+0000 7fd8cdd01700 1 -- 172.21.15.92:0/1777936820 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8c80cd4d0 msgr2=0x7fd8c8147390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.947+0000 7fd8cdd01700 1 --2- 172.21.15.92:0/1777936820 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8c80cd4d0 0x7fd8c8147390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.947+0000 7fd8cdd01700 1 -- 172.21.15.92:0/1777936820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c00093c0 con 0x7fd8c80b5640 2023-10-06T10:45:33.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.948+0000 7fd8cdd01700 1 --2- 172.21.15.92:0/1777936820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8c80b5640 0x7fd8c8155380 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fd8c400ce80 tx=0x7fd8c4007840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.948+0000 7fd8bd7fa700 1 -- 172.21.15.92:0/1777936820 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c400ae40 con 0x7fd8c80b5640 2023-10-06T10:45:33.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.948+0000 7fd8bd7fa700 1 -- 172.21.15.92:0/1777936820 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c4016430 con 0x7fd8c80b5640 2023-10-06T10:45:33.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.948+0000 7fd8cf764700 1 -- 172.21.15.92:0/1777936820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c816d6c0 con 0x7fd8c80b5640 2023-10-06T10:45:33.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.948+0000 7fd8bd7fa700 1 -- 172.21.15.92:0/1777936820 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c40167d0 con 0x7fd8c80b5640 2023-10-06T10:45:33.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.948+0000 7fd8cf764700 1 -- 172.21.15.92:0/1777936820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c8152870 con 0x7fd8c80b5640 2023-10-06T10:45:33.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.949+0000 7fd8bd7fa700 1 -- 172.21.15.92:0/1777936820 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd8c4003020 con 0x7fd8c80b5640 2023-10-06T10:45:33.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.950+0000 7fd8bd7fa700 1 --2- 172.21.15.92:0/1777936820 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd8ac066f30 0x7fd8ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.950+0000 7fd8bd7fa700 1 -- 172.21.15.92:0/1777936820 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fd8c4012070 con 0x7fd8c80b5640 2023-10-06T10:45:33.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.950+0000 7fd8cd500700 1 --2- 172.21.15.92:0/1777936820 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd8ac066f30 0x7fd8ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.950+0000 7fd8cd500700 1 --2- 172.21.15.92:0/1777936820 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd8ac066f30 0x7fd8ac0693e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009fd0 tx=0x7fd8c0016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.952+0000 7fd8d8a5cb80 1 Processor -- start 2023-10-06T10:45:33.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.952+0000 7fd8d8a5cb80 1 -- start start 2023-10-06T10:45:33.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.952+0000 7fd8d8a5cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ca2abe0ae0 0x55ca2abe0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.952+0000 7fd8d8a5cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ca2ab9c9f0 0x55ca2ab995d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.952+0000 7fd8d8a5cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2ab9ac20 0x55ca2ab99b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.952+0000 7fd8d8a5cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2ab19cd0 con 0x55ca2ab9ac20 2023-10-06T10:45:33.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.952+0000 7fd8d8a5cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2aaf0ac0 con 0x55ca2ab9c9f0 2023-10-06T10:45:33.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.952+0000 7fd8d8a5cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca2aaee2c0 con 0x55ca2abe0ae0 2023-10-06T10:45:33.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.952+0000 7fd8cdd01700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2ab9ac20 0x55ca2ab99b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.952+0000 7fd8cdd01700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2ab9ac20 0x55ca2ab99b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54998/0 (socket says 172.21.15.92:54998) 2023-10-06T10:45:33.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.952+0000 7fd8cdd01700 1 -- 172.21.15.92:0/2736865493 learned_addr learned my addr 172.21.15.92:0/2736865493 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:33.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.952+0000 7fd8cd500700 1 --2- 172.21.15.92:0/2736865493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ca2abe0ae0 0x55ca2abe0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.952+0000 7fd8cccff700 1 --2- 172.21.15.92:0/2736865493 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ca2ab9c9f0 0x55ca2ab995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.953+0000 7fd8cdd01700 1 -- 172.21.15.92:0/2736865493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ca2abe0ae0 msgr2=0x55ca2abe0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.953+0000 7fd8cdd01700 1 --2- 172.21.15.92:0/2736865493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ca2abe0ae0 0x55ca2abe0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.953+0000 7fd8cdd01700 1 -- 172.21.15.92:0/2736865493 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ca2ab9c9f0 msgr2=0x55ca2ab995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.953+0000 7fd8cdd01700 1 --2- 172.21.15.92:0/2736865493 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ca2ab9c9f0 0x55ca2ab995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.953+0000 7fd8cdd01700 1 -- 172.21.15.92:0/2736865493 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca2a9842c0 con 0x55ca2ab9ac20 2023-10-06T10:45:33.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.953+0000 7fd8cdd01700 1 --2- 172.21.15.92:0/2736865493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2ab9ac20 0x55ca2ab99b10 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4083990 tx=0x7fd8c4083920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792c120013f4f889 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.953+0000 7fd8b27fc700 1 -- 172.21.15.92:0/2736865493 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4054510 con 0x55ca2ab9ac20 2023-10-06T10:45:33.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.953+0000 7fd8b27fc700 1 -- 172.21.15.92:0/2736865493 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c4054770 con 0x55ca2ab9ac20 2023-10-06T10:45:33.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.953+0000 7fd8b27fc700 1 -- 172.21.15.92:0/2736865493 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4056860 con 0x55ca2ab9ac20 2023-10-06T10:45:33.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.953+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/2736865493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2ab9ac20 msgr2=0x55ca2ab99b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.953+0000 7fd8d8a5cb80 1 --2- 172.21.15.92:0/2736865493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2ab9ac20 0x55ca2ab99b10 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4083990 tx=0x7fd8c4083920 comp rx=0 tx=0).stop 2023-10-06T10:45:33.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.953+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/2736865493 shutdown_connections 2023-10-06T10:45:33.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.953+0000 7fd8d8a5cb80 1 --2- 172.21.15.92:0/2736865493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2ab9ac20 0x55ca2ab99b10 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.953+0000 7fd8d8a5cb80 1 --2- 172.21.15.92:0/2736865493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ca2abe0ae0 0x55ca2abe0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.953+0000 7fd8d8a5cb80 1 --2- 172.21.15.92:0/2736865493 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ca2ab9c9f0 0x55ca2ab995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.953+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/2736865493 >> 172.21.15.92:0/2736865493 conn(0x55ca2ab9b0c0 msgr2=0x55ca2ab9b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.954+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/2736865493 shutdown_connections 2023-10-06T10:45:33.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.954+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/2736865493 wait complete. 2023-10-06T10:45:33.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.954+0000 7fd8d8a5cb80 1 -- start start 2023-10-06T10:45:33.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.954+0000 7fd8d8a5cb80 1 Processor -- start 2023-10-06T10:45:33.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.955+0000 7fd8d8a5cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ca2ab9ac20 0x55ca2abdb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.955+0000 7fd8d8a5cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ca2ab9c9f0 0x55ca2abc1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.955+0000 7fd8d8a5cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2abe0ae0 0x55ca2abc2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.955+0000 7fd8d8a5cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2aaea0b0 con 0x55ca2abe0ae0 2023-10-06T10:45:33.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.955+0000 7fd8d8a5cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2aa958e0 con 0x55ca2ab9c9f0 2023-10-06T10:45:33.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.955+0000 7fd8d8a5cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca2aab0ef0 con 0x55ca2ab9ac20 2023-10-06T10:45:33.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.955+0000 7fd8d8a5cb80 10 client.0 _ll_register_callbacks cb 0x55ca2a986f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:33.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.955+0000 7fd8cdd01700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2abe0ae0 0x55ca2abc2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.955+0000 7fd8cdd01700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2abe0ae0 0x55ca2abc2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55014/0 (socket says 172.21.15.92:55014) 2023-10-06T10:45:33.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.955+0000 7fd8cdd01700 1 -- 172.21.15.92:0/2954760310 learned_addr learned my addr 172.21.15.92:0/2954760310 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:33.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.955+0000 7fd8cd500700 1 --2- 172.21.15.92:0/2954760310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ca2ab9ac20 0x55ca2abdb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.955+0000 7fd8cccff700 1 --2- 172.21.15.92:0/2954760310 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ca2ab9c9f0 0x55ca2abc1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.955+0000 7fd8cdd01700 1 -- 172.21.15.92:0/2954760310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ca2ab9ac20 msgr2=0x55ca2abdb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.955+0000 7fd8cdd01700 1 --2- 172.21.15.92:0/2954760310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ca2ab9ac20 0x55ca2abdb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.955+0000 7fd8cdd01700 1 -- 172.21.15.92:0/2954760310 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ca2ab9c9f0 msgr2=0x55ca2abc1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.955+0000 7fd8cdd01700 1 --2- 172.21.15.92:0/2954760310 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ca2ab9c9f0 0x55ca2abc1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.956+0000 7fd8cdd01700 1 -- 172.21.15.92:0/2954760310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c8152870 con 0x55ca2abe0ae0 2023-10-06T10:45:33.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.956+0000 7fd8cccff700 1 --2- 172.21.15.92:0/2954760310 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ca2ab9c9f0 0x55ca2abc1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:33.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.956+0000 7fd8cdd01700 1 --2- 172.21.15.92:0/2954760310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2abe0ae0 0x55ca2abc2130 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fd8c40528d0 tx=0x7fd8c4071000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5b540c426b8190 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.956+0000 7fd8a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:33.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.956+0000 7fd8a77fe700 1 -- 172.21.15.92:0/2954760310 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4056840 con 0x55ca2abe0ae0 2023-10-06T10:45:33.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.956+0000 7fd8a77fe700 1 -- 172.21.15.92:0/2954760310 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fd8c405cee0 con 0x55ca2abe0ae0 2023-10-06T10:45:33.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.956+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/2954760310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca2ab8e6e0 con 0x55ca2abe0ae0 2023-10-06T10:45:33.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.956+0000 7fd8a77fe700 1 -- 172.21.15.92:0/2954760310 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8c4016b40 con 0x55ca2abe0ae0 2023-10-06T10:45:33.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.956+0000 7fd8d8a5cb80 10 client.9129 fetch_fsmap learned FSMap version 109 2023-10-06T10:45:33.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.956+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/2954760310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x55ca2a927030 con 0x55ca2abe0ae0 2023-10-06T10:45:33.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.956+0000 7fd8a77fe700 1 -- 172.21.15.92:0/2954760310 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca2ab8e6e0 con 0x55ca2abe0ae0 2023-10-06T10:45:33.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.956+0000 7fd8a77fe700 1 -- 172.21.15.92:0/2954760310 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 109) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fd8c4016ca0 con 0x55ca2abe0ae0 2023-10-06T10:45:33.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.956+0000 7fd8d8a5cb80 10 client.9129 fetch_fsmap finished waiting for FSMap version 109 2023-10-06T10:45:33.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.956+0000 7fd8d8a5cb80 10 client.9129 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-10-06T10:45:33.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.956+0000 7fd8d8a5cb80 20 client.9129 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:33.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.956+0000 7fd8d8a5cb80 1 --2- 172.21.15.92:0/2954760310 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55ca2abc0520 0x55ca2abc54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.956+0000 7fd8d8a5cb80 4 client.9129 mds_command: new command op to 8520 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:33.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.956+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/2954760310 --> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca2ab6b610 con 0x55ca2abc0520 2023-10-06T10:45:33.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.956+0000 7fd8cd500700 1 --2- 172.21.15.92:0/2954760310 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55ca2abc0520 0x55ca2abc54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.957+0000 7fd8cd500700 1 --2- 172.21.15.92:0/2954760310 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55ca2abc0520 0x55ca2abc54b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a1bb41ad92ede3 server_cookie=d9e99edf09d77979 in_seq=0 out_seq=0 2023-10-06T10:45:33.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.957+0000 7fd8a77fe700 10 client.9129 ms_handle_connect on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:33.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.957+0000 7fd8a77fe700 1 -- 172.21.15.92:0/2954760310 <== mds.0 v2:172.21.15.92:6835/2298348426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55ca2ab6b610 con 0x55ca2abc0520 2023-10-06T10:45:33.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.957+0000 7fd8a77fe700 10 client.9129 handle_command_reply: tid=0 2023-10-06T10:45:33.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.957+0000 7fd8d8a5cb80 1 client.9129 shutdown 2023-10-06T10:45:33.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.957+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/2954760310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2abe0ae0 msgr2=0x55ca2abc2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.957+0000 7fd8d8a5cb80 1 --2- 172.21.15.92:0/2954760310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2abe0ae0 0x55ca2abc2130 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fd8c40528d0 tx=0x7fd8c4071000 comp rx=0 tx=0).stop 2023-10-06T10:45:33.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.957+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/2954760310 shutdown_connections 2023-10-06T10:45:33.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.957+0000 7fd8d8a5cb80 1 --2- 172.21.15.92:0/2954760310 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55ca2abc0520 0x55ca2abc54b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.957+0000 7fd8d8a5cb80 1 --2- 172.21.15.92:0/2954760310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2abe0ae0 0x55ca2abc2130 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.957+0000 7fd8d8a5cb80 1 --2- 172.21.15.92:0/2954760310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ca2ab9ac20 0x55ca2abdb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.957+0000 7fd8d8a5cb80 1 --2- 172.21.15.92:0/2954760310 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ca2ab9c9f0 0x55ca2abc1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.957+0000 7fd8a77fe700 0 client.9129 ms_handle_reset on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:33.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.957+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/2954760310 >> 172.21.15.92:0/2954760310 conn(0x55ca2ab9b0c0 msgr2=0x55ca2ab95ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.957+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/2954760310 shutdown_connections 2023-10-06T10:45:33.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.957+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/2954760310 wait complete. 2023-10-06T10:45:33.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.957+0000 7fd8d8a5cb80 20 client.9129 trim_cache size 0 max 16384 2023-10-06T10:45:33.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.973+0000 7fd8d8a5cb80 1 Processor -- start 2023-10-06T10:45:33.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.973+0000 7fd8d8a5cb80 1 -- start start 2023-10-06T10:45:33.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.974+0000 7fd8d8a5cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ca2ab9ac20 0x55ca2abdce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.974+0000 7fd8d8a5cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2ab9c9f0 0x55ca2abf9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.974+0000 7fd8d8a5cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ca2abe0ae0 0x55ca2abc8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.974+0000 7fd8d8a5cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2a96ce80 con 0x55ca2ab9c9f0 2023-10-06T10:45:33.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.974+0000 7fd8d8a5cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2abdfb30 con 0x55ca2ab9ac20 2023-10-06T10:45:33.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.974+0000 7fd8d8a5cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca2ab79080 con 0x55ca2abe0ae0 2023-10-06T10:45:33.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.974+0000 7fd8cccff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2ab9c9f0 0x55ca2abf9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.974+0000 7fd8cccff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2ab9c9f0 0x55ca2abf9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55030/0 (socket says 172.21.15.92:55030) 2023-10-06T10:45:33.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.974+0000 7fd8cccff700 1 -- 172.21.15.92:0/1674672802 learned_addr learned my addr 172.21.15.92:0/1674672802 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:33.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.974+0000 7fd8cd500700 1 --2- 172.21.15.92:0/1674672802 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ca2ab9ac20 0x55ca2abdce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.974+0000 7fd8cdd01700 1 --2- 172.21.15.92:0/1674672802 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ca2abe0ae0 0x55ca2abc8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.974+0000 7fd8cccff700 1 -- 172.21.15.92:0/1674672802 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ca2abe0ae0 msgr2=0x55ca2abc8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.974+0000 7fd8cccff700 1 --2- 172.21.15.92:0/1674672802 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ca2abe0ae0 0x55ca2abc8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.974+0000 7fd8cccff700 1 -- 172.21.15.92:0/1674672802 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ca2ab9ac20 msgr2=0x55ca2abdce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.974+0000 7fd8cccff700 1 --2- 172.21.15.92:0/1674672802 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ca2ab9ac20 0x55ca2abdce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.974+0000 7fd8cccff700 1 -- 172.21.15.92:0/1674672802 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca2aab87d0 con 0x55ca2ab9c9f0 2023-10-06T10:45:33.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.974+0000 7fd8cd500700 1 --2- 172.21.15.92:0/1674672802 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ca2ab9ac20 0x55ca2abdce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:33.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.974+0000 7fd8cdd01700 1 --2- 172.21.15.92:0/1674672802 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ca2abe0ae0 0x55ca2abc8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:33.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.974+0000 7fd8cccff700 1 --2- 172.21.15.92:0/1674672802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2ab9c9f0 0x55ca2abf9670 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fd8c815a5a0 tx=0x7fd8b8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c66777844c1a187 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.975+0000 7fd8b2ffd700 1 -- 172.21.15.92:0/1674672802 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8015680 con 0x55ca2ab9c9f0 2023-10-06T10:45:33.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.975+0000 7fd8b2ffd700 1 -- 172.21.15.92:0/1674672802 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b800d070 con 0x55ca2ab9c9f0 2023-10-06T10:45:33.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.975+0000 7fd8b2ffd700 1 -- 172.21.15.92:0/1674672802 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8015a10 con 0x55ca2ab9c9f0 2023-10-06T10:45:33.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.975+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/1674672802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2ab9c9f0 msgr2=0x55ca2abf9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.975+0000 7fd8d8a5cb80 1 --2- 172.21.15.92:0/1674672802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2ab9c9f0 0x55ca2abf9670 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fd8c815a5a0 tx=0x7fd8b8009040 comp rx=0 tx=0).stop 2023-10-06T10:45:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.975+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/1674672802 shutdown_connections 2023-10-06T10:45:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.975+0000 7fd8d8a5cb80 1 --2- 172.21.15.92:0/1674672802 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ca2abe0ae0 0x55ca2abc8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.975+0000 7fd8d8a5cb80 1 --2- 172.21.15.92:0/1674672802 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ca2ab9ac20 0x55ca2abdce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.975+0000 7fd8d8a5cb80 1 --2- 172.21.15.92:0/1674672802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2ab9c9f0 0x55ca2abf9670 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.975+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/1674672802 >> 172.21.15.92:0/1674672802 conn(0x55ca2ab9b0c0 msgr2=0x55ca2ab96a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.975+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/1674672802 shutdown_connections 2023-10-06T10:45:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.975+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/1674672802 wait complete. 2023-10-06T10:45:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.976+0000 7fd8d8a5cb80 1 -- start start 2023-10-06T10:45:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.976+0000 7fd8d8a5cb80 1 Processor -- start 2023-10-06T10:45:33.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8d8a5cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2ab9ac20 0x55ca2abdce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8d8a5cb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ca2ab9c9f0 0x55ca2ac2e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8d8a5cb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ca2abe0ae0 0x55ca2abe5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8d8a5cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2ac2ec30 con 0x55ca2ab9ac20 2023-10-06T10:45:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8d8a5cb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2abc0970 con 0x55ca2ab9c9f0 2023-10-06T10:45:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8d8a5cb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca2abe69a0 con 0x55ca2abe0ae0 2023-10-06T10:45:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8d8a5cb80 10 client.0 _ll_register_callbacks cb 0x55ca2a986f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8cd500700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2ab9ac20 0x55ca2abdce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8cd500700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2ab9ac20 0x55ca2abdce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55038/0 (socket says 172.21.15.92:55038) 2023-10-06T10:45:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8cd500700 1 -- 172.21.15.92:0/3468741099 learned_addr learned my addr 172.21.15.92:0/3468741099 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8cccff700 1 --2- 172.21.15.92:0/3468741099 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ca2ab9c9f0 0x55ca2ac2e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8cdd01700 1 --2- 172.21.15.92:0/3468741099 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ca2abe0ae0 0x55ca2abe5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8cd500700 1 -- 172.21.15.92:0/3468741099 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ca2abe0ae0 msgr2=0x55ca2abe5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8cd500700 1 --2- 172.21.15.92:0/3468741099 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ca2abe0ae0 0x55ca2abe5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8cd500700 1 -- 172.21.15.92:0/3468741099 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ca2ab9c9f0 msgr2=0x55ca2ac2e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8cd500700 1 --2- 172.21.15.92:0/3468741099 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ca2ab9c9f0 0x55ca2ac2e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8cd500700 1 -- 172.21.15.92:0/3468741099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b8005e10 con 0x55ca2ab9ac20 2023-10-06T10:45:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8cd500700 1 --2- 172.21.15.92:0/3468741099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2ab9ac20 0x55ca2abdce20 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80d14c0 tx=0x7fd8c0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d03b6c729c06a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8a77fe700 1 -- 172.21.15.92:0/3468741099 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0030040 con 0x55ca2ab9ac20 2023-10-06T10:45:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8a77fe700 1 -- 172.21.15.92:0/3468741099 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fd8c0006460 con 0x55ca2ab9ac20 2023-10-06T10:45:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/3468741099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8c4054770 con 0x55ca2ab9ac20 2023-10-06T10:45:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8a77fe700 1 -- 172.21.15.92:0/3468741099 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8c000bed0 con 0x55ca2ab9ac20 2023-10-06T10:45:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8d8a5cb80 10 client.9135 fetch_fsmap learned FSMap version 109 2023-10-06T10:45:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.977+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/3468741099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x55ca2aa9f4c0 con 0x55ca2ab9ac20 2023-10-06T10:45:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8a77fe700 1 -- 172.21.15.92:0/3468741099 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c4054770 con 0x55ca2ab9ac20 2023-10-06T10:45:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8a77fe700 1 -- 172.21.15.92:0/3468741099 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 109) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fd8c0036490 con 0x55ca2ab9ac20 2023-10-06T10:45:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8d8a5cb80 10 client.9135 fetch_fsmap finished waiting for FSMap version 109 2023-10-06T10:45:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8d8a5cb80 10 client.9135 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-10-06T10:45:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8d8a5cb80 20 client.9135 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8d8a5cb80 1 --2- 172.21.15.92:0/3468741099 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55ca2abdb6e0 0x55ca2abc1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8d8a5cb80 4 client.9135 mds_command: new command op to 8520 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:45:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/3468741099 --> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ca2aa8ce50 con 0x55ca2abdb6e0 2023-10-06T10:45:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8cccff700 1 --2- 172.21.15.92:0/3468741099 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55ca2abdb6e0 0x55ca2abc1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8a77fe700 10 client.9135 ms_handle_connect on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8cccff700 1 --2- 172.21.15.92:0/3468741099 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55ca2abdb6e0 0x55ca2abc1380 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6540d8c873aef390 server_cookie=276cc8b390268a20 in_seq=0 out_seq=0 2023-10-06T10:45:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8a77fe700 1 -- 172.21.15.92:0/3468741099 <== mds.0 v2:172.21.15.92:6835/2298348426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55ca2aa8ce50 con 0x55ca2abdb6e0 2023-10-06T10:45:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8a77fe700 10 client.9135 handle_command_reply: tid=0 2023-10-06T10:45:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8d8a5cb80 1 client.9135 shutdown 2023-10-06T10:45:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/3468741099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2ab9ac20 msgr2=0x55ca2abdce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8d8a5cb80 1 --2- 172.21.15.92:0/3468741099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2ab9ac20 0x55ca2abdce20 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80d14c0 tx=0x7fd8c0013000 comp rx=0 tx=0).stop 2023-10-06T10:45:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/3468741099 shutdown_connections 2023-10-06T10:45:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8d8a5cb80 1 --2- 172.21.15.92:0/3468741099 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55ca2abdb6e0 0x55ca2abc1380 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8d8a5cb80 1 --2- 172.21.15.92:0/3468741099 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55ca2abe0ae0 0x55ca2abe5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8d8a5cb80 1 --2- 172.21.15.92:0/3468741099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca2ab9ac20 0x55ca2abdce20 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8a77fe700 0 client.9135 ms_handle_reset on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8d8a5cb80 1 --2- 172.21.15.92:0/3468741099 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55ca2ab9c9f0 0x55ca2ac2e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.978+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/3468741099 >> 172.21.15.92:0/3468741099 conn(0x55ca2ab9b0c0 msgr2=0x55ca2abc8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.979+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/3468741099 shutdown_connections 2023-10-06T10:45:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.979+0000 7fd8d8a5cb80 1 -- 172.21.15.92:0/3468741099 wait complete. 2023-10-06T10:45:33.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.979+0000 7fd8d8a5cb80 20 client.9135 trim_cache size 0 max 16384 2023-10-06T10:45:33.988 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:33.988 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "PAUSED (0 inodes in the stack)", 2023-10-06T10:45:33.988 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:45:33.988 INFO:teuthology.orchestra.run.smithi092.stdout: "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8": { 2023-10-06T10:45:33.988 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:45:33.988 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8", 2023-10-06T10:45:33.988 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:45:33.988 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:33.988 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:33.989 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:33.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.979+0000 7fd8cf764700 1 -- 172.21.15.92:0/1777936820 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd8ac066f30 msgr2=0x7fd8ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.979+0000 7fd8cf764700 1 --2- 172.21.15.92:0/1777936820 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd8ac066f30 0x7fd8ac0693e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009fd0 tx=0x7fd8c0016000 comp rx=0 tx=0).stop 2023-10-06T10:45:33.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.979+0000 7fd8cf764700 1 -- 172.21.15.92:0/1777936820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8c80b5640 msgr2=0x7fd8c8155380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:33.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.979+0000 7fd8cf764700 1 --2- 172.21.15.92:0/1777936820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8c80b5640 0x7fd8c8155380 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fd8c400ce80 tx=0x7fd8c4007840 comp rx=0 tx=0).stop 2023-10-06T10:45:33.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.979+0000 7fd8cf764700 1 -- 172.21.15.92:0/1777936820 shutdown_connections 2023-10-06T10:45:33.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.979+0000 7fd8cf764700 1 --2- 172.21.15.92:0/1777936820 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd8ac066f30 0x7fd8ac0693e0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.979+0000 7fd8cf764700 1 --2- 172.21.15.92:0/1777936820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8c80b5640 0x7fd8c8155380 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.979+0000 7fd8cf764700 1 --2- 172.21.15.92:0/1777936820 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8c80cd4d0 0x7fd8c8147390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.979+0000 7fd8cf764700 1 --2- 172.21.15.92:0/1777936820 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8c815a160 0x7fd8c80b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:33.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.979+0000 7fd8cf764700 1 -- 172.21.15.92:0/1777936820 >> 172.21.15.92:0/1777936820 conn(0x7fd8c8099700 msgr2=0x7fd8c80c5250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:33.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.979+0000 7fd8cf764700 1 -- 172.21.15.92:0/1777936820 shutdown_connections 2023-10-06T10:45:33.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:33.979+0000 7fd8cf764700 1 -- 172.21.15.92:0/1777936820 wait complete. 2023-10-06T10:45:33.991 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:45:33.992 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:1 scrub status 2023-10-06T10:45:34.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.107+0000 7ff7cecdf700 1 Processor -- start 2023-10-06T10:45:34.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.107+0000 7ff7cecdf700 1 -- start start 2023-10-06T10:45:34.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.108+0000 7ff7cecdf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff7c8150940 0x7ff7c8159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.108+0000 7ff7cecdf700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff7c80cd4d0 0x7ff7c80b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.108+0000 7ff7cecdf700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7c815ac30 con 0x7ff7c815a160 2023-10-06T10:45:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.108+0000 7ff7cecdf700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c80cb6a0 con 0x7ff7c8150940 2023-10-06T10:45:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.108+0000 7ff7cecdf700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c80cb090 con 0x7ff7c80cd4d0 2023-10-06T10:45:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.108+0000 7ff7cca7b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff7c8150940 0x7ff7c8159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.108+0000 7ff7bffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff7c815a160 0x7ff7c80ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45526/0 (socket says 172.21.15.92:45526) 2023-10-06T10:45:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.108+0000 7ff7bffff700 1 -- 172.21.15.92:0/3359039706 learned_addr learned my addr 172.21.15.92:0/3359039706 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.108+0000 7ff7cd27c700 1 --2- 172.21.15.92:0/3359039706 >> v2:172.21.15.124:3300/0 conn(0x7ff7c80cd4d0 0x7ff7c80b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.109+0000 7ff7cca7b700 1 -- 172.21.15.92:0/3359039706 >> v2:172.21.15.124:3300/0 conn(0x7ff7c80cd4d0 msgr2=0x7ff7c80b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.109+0000 7ff7cca7b700 1 --2- 172.21.15.92:0/3359039706 >> v2:172.21.15.124:3300/0 conn(0x7ff7c80cd4d0 0x7ff7c80b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.109+0000 7ff7cca7b700 1 -- 172.21.15.92:0/3359039706 >> v1:172.21.15.92:6789/0 conn(0x7ff7c815a160 legacy=0x7ff7c80ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.109+0000 7ff7bf7fe700 1 -- 172.21.15.92:0/3359039706 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513609569 0 0) 0x7ff7b8001110 con 0x7ff7c815a160 2023-10-06T10:45:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.109+0000 7ff7cca7b700 1 -- 172.21.15.92:0/3359039706 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c815a670 con 0x7ff7c8150940 2023-10-06T10:45:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.109+0000 7ff7cca7b700 1 --2- 172.21.15.92:0/3359039706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c8150940 0x7ff7c8159c20 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80d0460 tx=0x7ff7c4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32e4d90fd176a7e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.109+0000 7ff7bf7fe700 1 -- 172.21.15.92:0/3359039706 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c400e070 con 0x7ff7c8150940 2023-10-06T10:45:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.109+0000 7ff7bf7fe700 1 -- 172.21.15.92:0/3359039706 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c400bc30 con 0x7ff7c8150940 2023-10-06T10:45:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.109+0000 7ff7bf7fe700 1 -- 172.21.15.92:0/3359039706 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4012430 con 0x7ff7c8150940 2023-10-06T10:45:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.109+0000 7ff7cecdf700 1 -- 172.21.15.92:0/3359039706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c8150940 msgr2=0x7ff7c8159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.109+0000 7ff7cecdf700 1 --2- 172.21.15.92:0/3359039706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c8150940 0x7ff7c8159c20 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80d0460 tx=0x7ff7c4009780 comp rx=0 tx=0).stop 2023-10-06T10:45:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.109+0000 7ff7cecdf700 1 -- 172.21.15.92:0/3359039706 shutdown_connections 2023-10-06T10:45:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.109+0000 7ff7cecdf700 1 --2- 172.21.15.92:0/3359039706 >> v2:172.21.15.124:3300/0 conn(0x7ff7c80cd4d0 0x7ff7c80b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.109+0000 7ff7cecdf700 1 --2- 172.21.15.92:0/3359039706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c8150940 0x7ff7c8159c20 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.109+0000 7ff7cecdf700 1 -- 172.21.15.92:0/3359039706 >> 172.21.15.92:0/3359039706 conn(0x7ff7c8099700 msgr2=0x7ff7c806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.109+0000 7ff7cecdf700 1 -- 172.21.15.92:0/3359039706 shutdown_connections 2023-10-06T10:45:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.109+0000 7ff7cecdf700 1 -- 172.21.15.92:0/3359039706 wait complete. 2023-10-06T10:45:34.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.111+0000 7ff7cecdf700 1 Processor -- start 2023-10-06T10:45:34.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.111+0000 7ff7cecdf700 1 -- start start 2023-10-06T10:45:34.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.111+0000 7ff7cecdf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c80cd4d0 0x7ff7c816c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.111+0000 7ff7cecdf700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7c815a160 0x7ff7c816cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.111+0000 7ff7cecdf700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7c8156e00 0x7ff7c816a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.111+0000 7ff7cecdf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c81437e0 con 0x7ff7c80cd4d0 2023-10-06T10:45:34.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.111+0000 7ff7cecdf700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c8153060 con 0x7ff7c815a160 2023-10-06T10:45:34.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.111+0000 7ff7cecdf700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c816d650 con 0x7ff7c8156e00 2023-10-06T10:45:34.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.112+0000 7ff7cca7b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c80cd4d0 0x7ff7c816c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.112+0000 7ff7cca7b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c80cd4d0 0x7ff7c816c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55054/0 (socket says 172.21.15.92:55054) 2023-10-06T10:45:34.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.112+0000 7ff7cca7b700 1 -- 172.21.15.92:0/544290291 learned_addr learned my addr 172.21.15.92:0/544290291 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:34.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.112+0000 7ff7cd27c700 1 --2- 172.21.15.92:0/544290291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7c8156e00 0x7ff7c816a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.112+0000 7ff7bffff700 1 --2- 172.21.15.92:0/544290291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7c815a160 0x7ff7c816cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.112+0000 7ff7cca7b700 1 -- 172.21.15.92:0/544290291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7c8156e00 msgr2=0x7ff7c816a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.112+0000 7ff7cca7b700 1 --2- 172.21.15.92:0/544290291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7c8156e00 0x7ff7c816a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.112+0000 7ff7cca7b700 1 -- 172.21.15.92:0/544290291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7c815a160 msgr2=0x7ff7c816cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.112+0000 7ff7cca7b700 1 --2- 172.21.15.92:0/544290291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7c815a160 0x7ff7c816cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.112+0000 7ff7cca7b700 1 -- 172.21.15.92:0/544290291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c4009a20 con 0x7ff7c80cd4d0 2023-10-06T10:45:34.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.112+0000 7ff7cca7b700 1 --2- 172.21.15.92:0/544290291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c80cd4d0 0x7ff7c816c730 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4013000 tx=0x7ff7c4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.112+0000 7ff7bcff9700 1 -- 172.21.15.92:0/544290291 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4001090 con 0x7ff7c80cd4d0 2023-10-06T10:45:34.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.112+0000 7ff7bcff9700 1 -- 172.21.15.92:0/544290291 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c4027470 con 0x7ff7c80cd4d0 2023-10-06T10:45:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.112+0000 7ff7bcff9700 1 -- 172.21.15.92:0/544290291 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4027830 con 0x7ff7c80cd4d0 2023-10-06T10:45:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.112+0000 7ff7cecdf700 1 -- 172.21.15.92:0/544290291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c816d930 con 0x7ff7c80cd4d0 2023-10-06T10:45:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.112+0000 7ff7cecdf700 1 -- 172.21.15.92:0/544290291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c816d2b0 con 0x7ff7c80cd4d0 2023-10-06T10:45:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.113+0000 7ff7bcff9700 1 -- 172.21.15.92:0/544290291 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff7c4024070 con 0x7ff7c80cd4d0 2023-10-06T10:45:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.114+0000 7ff7bcff9700 1 --2- 172.21.15.92:0/544290291 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff7b4066f30 0x7ff7b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.114+0000 7ff7bcff9700 1 -- 172.21.15.92:0/544290291 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7ff7c4086e20 con 0x7ff7c80cd4d0 2023-10-06T10:45:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.114+0000 7ff7bffff700 1 --2- 172.21.15.92:0/544290291 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff7b4066f30 0x7ff7b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.115+0000 7ff7bffff700 1 --2- 172.21.15.92:0/544290291 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff7b4066f30 0x7ff7b40693e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80cac60 tx=0x7ff7b8007f50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.116+0000 7ff7d7fd7b80 1 Processor -- start 2023-10-06T10:45:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.116+0000 7ff7d7fd7b80 1 -- start start 2023-10-06T10:45:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.117+0000 7ff7d7fd7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c7131ae0 0x5623c7131eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.117+0000 7ff7d7fd7b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5623c70ed9f0 0x5623c70ea5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.117+0000 7ff7d7fd7b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623c70ebc20 0x5623c70eab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.117+0000 7ff7d7fd7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623c706acd0 con 0x5623c7131ae0 2023-10-06T10:45:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.117+0000 7ff7d7fd7b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623c7041ac0 con 0x5623c70ebc20 2023-10-06T10:45:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.117+0000 7ff7d7fd7b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623c703f2c0 con 0x5623c70ed9f0 2023-10-06T10:45:34.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.117+0000 7ff7cca7b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c7131ae0 0x5623c7131eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.117+0000 7ff7cca7b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c7131ae0 0x5623c7131eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55058/0 (socket says 172.21.15.92:55058) 2023-10-06T10:45:34.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.117+0000 7ff7cca7b700 1 -- 172.21.15.92:0/2846631219 learned_addr learned my addr 172.21.15.92:0/2846631219 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:34.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.117+0000 7ff7bffff700 1 --2- 172.21.15.92:0/2846631219 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5623c70ed9f0 0x5623c70ea5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.117+0000 7ff7cd27c700 1 --2- 172.21.15.92:0/2846631219 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623c70ebc20 0x5623c70eab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.117+0000 7ff7cca7b700 1 -- 172.21.15.92:0/2846631219 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5623c70ed9f0 msgr2=0x5623c70ea5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.117+0000 7ff7cca7b700 1 --2- 172.21.15.92:0/2846631219 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5623c70ed9f0 0x5623c70ea5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.117+0000 7ff7cca7b700 1 -- 172.21.15.92:0/2846631219 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623c70ebc20 msgr2=0x5623c70eab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.117+0000 7ff7cca7b700 1 --2- 172.21.15.92:0/2846631219 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623c70ebc20 0x5623c70eab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.117+0000 7ff7cca7b700 1 -- 172.21.15.92:0/2846631219 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623c6ed52c0 con 0x5623c7131ae0 2023-10-06T10:45:34.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.117+0000 7ff7cca7b700 1 --2- 172.21.15.92:0/2846631219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c7131ae0 0x5623c7131eb0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400b920 tx=0x7ff7c4084fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f698fb5d90e70f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.117+0000 7ff7adffb700 1 -- 172.21.15.92:0/2846631219 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c400e040 con 0x5623c7131ae0 2023-10-06T10:45:34.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.117+0000 7ff7adffb700 1 -- 172.21.15.92:0/2846631219 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c4057a50 con 0x5623c7131ae0 2023-10-06T10:45:34.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.117+0000 7ff7adffb700 1 -- 172.21.15.92:0/2846631219 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c405aca0 con 0x5623c7131ae0 2023-10-06T10:45:34.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.118+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/2846631219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c7131ae0 msgr2=0x5623c7131eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.118+0000 7ff7d7fd7b80 1 --2- 172.21.15.92:0/2846631219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c7131ae0 0x5623c7131eb0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400b920 tx=0x7ff7c4084fa0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.118+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/2846631219 shutdown_connections 2023-10-06T10:45:34.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.118+0000 7ff7d7fd7b80 1 --2- 172.21.15.92:0/2846631219 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623c70ebc20 0x5623c70eab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.118+0000 7ff7d7fd7b80 1 --2- 172.21.15.92:0/2846631219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c7131ae0 0x5623c7131eb0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.118+0000 7ff7d7fd7b80 1 --2- 172.21.15.92:0/2846631219 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5623c70ed9f0 0x5623c70ea5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.118+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/2846631219 >> 172.21.15.92:0/2846631219 conn(0x5623c70ec0c0 msgr2=0x5623c70ec490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:34.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.118+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/2846631219 shutdown_connections 2023-10-06T10:45:34.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.118+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/2846631219 wait complete. 2023-10-06T10:45:34.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.119+0000 7ff7d7fd7b80 1 -- start start 2023-10-06T10:45:34.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.119+0000 7ff7d7fd7b80 1 Processor -- start 2023-10-06T10:45:34.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7d7fd7b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623c70ebc20 0x5623c712b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7d7fd7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c70ed9f0 0x5623c7111bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7d7fd7b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5623c7131ae0 0x5623c7112130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7d7fd7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623c703b0b0 con 0x5623c70ed9f0 2023-10-06T10:45:34.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7d7fd7b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623c6fe68e0 con 0x5623c70ebc20 2023-10-06T10:45:34.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7d7fd7b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623c7001ef0 con 0x5623c7131ae0 2023-10-06T10:45:34.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7d7fd7b80 10 client.0 _ll_register_callbacks cb 0x5623c6ed7f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:34.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7bffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c70ed9f0 0x5623c7111bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7bffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c70ed9f0 0x5623c7111bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55064/0 (socket says 172.21.15.92:55064) 2023-10-06T10:45:34.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7bffff700 1 -- 172.21.15.92:0/322537665 learned_addr learned my addr 172.21.15.92:0/322537665 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:34.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7cca7b700 1 --2- 172.21.15.92:0/322537665 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623c70ebc20 0x5623c712b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7cd27c700 1 --2- 172.21.15.92:0/322537665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5623c7131ae0 0x5623c7112130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7bffff700 1 -- 172.21.15.92:0/322537665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5623c7131ae0 msgr2=0x5623c7112130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7bffff700 1 --2- 172.21.15.92:0/322537665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5623c7131ae0 0x5623c7112130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7bffff700 1 -- 172.21.15.92:0/322537665 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623c70ebc20 msgr2=0x5623c712b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7bffff700 1 --2- 172.21.15.92:0/322537665 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623c70ebc20 0x5623c712b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7bffff700 1 -- 172.21.15.92:0/322537665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c816d2b0 con 0x5623c70ed9f0 2023-10-06T10:45:34.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7cd27c700 1 --2- 172.21.15.92:0/322537665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5623c7131ae0 0x5623c7112130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:34.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7bffff700 1 --2- 172.21.15.92:0/322537665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c70ed9f0 0x5623c7111bf0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80066b0 tx=0x7ff7b8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc6cf916bb0dd66 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:34.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7a6ffd700 1 -- 172.21.15.92:0/322537665 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8006790 con 0x5623c70ed9f0 2023-10-06T10:45:34.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7a6ffd700 1 -- 172.21.15.92:0/322537665 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7ff7b801b070 con 0x5623c70ed9f0 2023-10-06T10:45:34.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/322537665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623c70df6e0 con 0x5623c70ed9f0 2023-10-06T10:45:34.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7a6ffd700 1 -- 172.21.15.92:0/322537665 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7b8005e10 con 0x5623c70ed9f0 2023-10-06T10:45:34.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.120+0000 7ff7d7fd7b80 10 client.9150 fetch_fsmap learned FSMap version 109 2023-10-06T10:45:34.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.121+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/322537665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x5623c6e78030 con 0x5623c70ed9f0 2023-10-06T10:45:34.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.121+0000 7ff7a6ffd700 1 -- 172.21.15.92:0/322537665 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5623c70df6e0 con 0x5623c70ed9f0 2023-10-06T10:45:34.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.121+0000 7ff7a6ffd700 1 -- 172.21.15.92:0/322537665 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 109) v1 ==== 3140+0+0 (secure 0 0 0) 0x7ff7b80019a0 con 0x5623c70ed9f0 2023-10-06T10:45:34.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.121+0000 7ff7d7fd7b80 10 client.9150 fetch_fsmap finished waiting for FSMap version 109 2023-10-06T10:45:34.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.121+0000 7ff7d7fd7b80 10 client.9150 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2023-10-06T10:45:34.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.121+0000 7ff7d7fd7b80 20 client.9150 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:34.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.121+0000 7ff7d7fd7b80 1 --2- 172.21.15.92:0/322537665 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5623c7115690 0x5623c71413e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.121+0000 7ff7d7fd7b80 4 client.9150 mds_command: new command op to 8517 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:34.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.121+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/322537665 --> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623c70bc610 con 0x5623c7115690 2023-10-06T10:45:34.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.121+0000 7ff7cca7b700 1 --2- 172.21.15.92:0/322537665 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5623c7115690 0x5623c71413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.121+0000 7ff7a6ffd700 10 client.9150 ms_handle_connect on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:34.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.121+0000 7ff7cca7b700 1 --2- 172.21.15.92:0/322537665 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5623c7115690 0x5623c71413e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d4896ae8c1718cda server_cookie=ed973bed9165a0ff in_seq=0 out_seq=0 2023-10-06T10:45:34.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.122+0000 7ff7a6ffd700 1 -- 172.21.15.92:0/322537665 <== mds.1 v2:172.21.15.92:6834/1979826278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5623c70bc610 con 0x5623c7115690 2023-10-06T10:45:34.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.122+0000 7ff7a6ffd700 10 client.9150 handle_command_reply: tid=0 2023-10-06T10:45:34.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.122+0000 7ff7d7fd7b80 1 client.9150 shutdown 2023-10-06T10:45:34.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.122+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/322537665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c70ed9f0 msgr2=0x5623c7111bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.122+0000 7ff7d7fd7b80 1 --2- 172.21.15.92:0/322537665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c70ed9f0 0x5623c7111bf0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80066b0 tx=0x7ff7b8014000 comp rx=0 tx=0).stop 2023-10-06T10:45:34.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.122+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/322537665 shutdown_connections 2023-10-06T10:45:34.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.122+0000 7ff7d7fd7b80 1 --2- 172.21.15.92:0/322537665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5623c7131ae0 0x5623c7112130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.122+0000 7ff7d7fd7b80 1 --2- 172.21.15.92:0/322537665 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623c70ebc20 0x5623c712b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.122+0000 7ff7d7fd7b80 1 --2- 172.21.15.92:0/322537665 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5623c7115690 0x5623c71413e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.122+0000 7ff7d7fd7b80 1 --2- 172.21.15.92:0/322537665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c70ed9f0 0x5623c7111bf0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.122+0000 7ff7a6ffd700 0 client.9150 ms_handle_reset on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:34.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.122+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/322537665 >> 172.21.15.92:0/322537665 conn(0x5623c70ec0c0 msgr2=0x5623c7131000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:34.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.122+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/322537665 shutdown_connections 2023-10-06T10:45:34.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.122+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/322537665 wait complete. 2023-10-06T10:45:34.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.122+0000 7ff7d7fd7b80 20 client.9150 trim_cache size 0 max 16384 2023-10-06T10:45:34.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.138+0000 7ff7d7fd7b80 1 Processor -- start 2023-10-06T10:45:34.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.138+0000 7ff7d7fd7b80 1 -- start start 2023-10-06T10:45:34.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.139+0000 7ff7d7fd7b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5623c70ebc20 0x5623c712ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.139+0000 7ff7d7fd7b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623c70ed9f0 0x5623c7111af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.139+0000 7ff7d7fd7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c7131ae0 0x5623c7112030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.139+0000 7ff7d7fd7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623c6ebde80 con 0x5623c7131ae0 2023-10-06T10:45:34.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.139+0000 7ff7d7fd7b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623c70e99e0 con 0x5623c70ed9f0 2023-10-06T10:45:34.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.139+0000 7ff7d7fd7b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623c71310b0 con 0x5623c70ebc20 2023-10-06T10:45:34.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.139+0000 7ff7cd27c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c7131ae0 0x5623c7112030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.139+0000 7ff7cd27c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c7131ae0 0x5623c7112030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55072/0 (socket says 172.21.15.92:55072) 2023-10-06T10:45:34.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.139+0000 7ff7cd27c700 1 -- 172.21.15.92:0/1559729130 learned_addr learned my addr 172.21.15.92:0/1559729130 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:34.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.139+0000 7ff7bffff700 1 --2- 172.21.15.92:0/1559729130 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623c70ed9f0 0x5623c7111af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.139+0000 7ff7cd27c700 1 -- 172.21.15.92:0/1559729130 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5623c70ebc20 msgr2=0x5623c712ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.139+0000 7ff7cd27c700 1 --2- 172.21.15.92:0/1559729130 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5623c70ebc20 0x5623c712ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.139+0000 7ff7cd27c700 1 -- 172.21.15.92:0/1559729130 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623c70ed9f0 msgr2=0x5623c7111af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.139+0000 7ff7cd27c700 1 --2- 172.21.15.92:0/1559729130 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623c70ed9f0 0x5623c7111af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.139+0000 7ff7cd27c700 1 -- 172.21.15.92:0/1559729130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623c70097d0 con 0x5623c7131ae0 2023-10-06T10:45:34.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.139+0000 7ff7cd27c700 1 --2- 172.21.15.92:0/1559729130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c7131ae0 0x5623c7112030 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0006b00 tx=0x7ff7c0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bca6f2bdf686759 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.139+0000 7ff7ae7fc700 1 -- 172.21.15.92:0/1559729130 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0014aa0 con 0x5623c7131ae0 2023-10-06T10:45:34.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.139+0000 7ff7ae7fc700 1 -- 172.21.15.92:0/1559729130 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c0014c20 con 0x5623c7131ae0 2023-10-06T10:45:34.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.139+0000 7ff7ae7fc700 1 -- 172.21.15.92:0/1559729130 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0016430 con 0x5623c7131ae0 2023-10-06T10:45:34.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.140+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/1559729130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c7131ae0 msgr2=0x5623c7112030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.140+0000 7ff7d7fd7b80 1 --2- 172.21.15.92:0/1559729130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c7131ae0 0x5623c7112030 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0006b00 tx=0x7ff7c0008040 comp rx=0 tx=0).stop 2023-10-06T10:45:34.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.140+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/1559729130 shutdown_connections 2023-10-06T10:45:34.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.140+0000 7ff7d7fd7b80 1 --2- 172.21.15.92:0/1559729130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c7131ae0 0x5623c7112030 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.140+0000 7ff7d7fd7b80 1 --2- 172.21.15.92:0/1559729130 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5623c70ebc20 0x5623c712ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.140+0000 7ff7d7fd7b80 1 --2- 172.21.15.92:0/1559729130 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623c70ed9f0 0x5623c7111af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.140+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/1559729130 >> 172.21.15.92:0/1559729130 conn(0x5623c70ec0c0 msgr2=0x5623c712b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:34.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.140+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/1559729130 shutdown_connections 2023-10-06T10:45:34.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.140+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/1559729130 wait complete. 2023-10-06T10:45:34.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.140+0000 7ff7d7fd7b80 1 -- start start 2023-10-06T10:45:34.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.141+0000 7ff7d7fd7b80 1 Processor -- start 2023-10-06T10:45:34.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.141+0000 7ff7d7fd7b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623c70ebc20 0x5623c7196820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.141+0000 7ff7d7fd7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c70ed9f0 0x5623c70e7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.141+0000 7ff7d7fd7b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5623c7131ae0 0x5623c70e7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.141+0000 7ff7d7fd7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623c714af40 con 0x5623c70ed9f0 2023-10-06T10:45:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.141+0000 7ff7d7fd7b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623c70e5d20 con 0x5623c70ebc20 2023-10-06T10:45:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.141+0000 7ff7d7fd7b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623c70eb890 con 0x5623c7131ae0 2023-10-06T10:45:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.141+0000 7ff7d7fd7b80 10 client.0 _ll_register_callbacks cb 0x5623c6ed7f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.141+0000 7ff7bffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c70ed9f0 0x5623c70e7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.141+0000 7ff7bffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c70ed9f0 0x5623c70e7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55088/0 (socket says 172.21.15.92:55088) 2023-10-06T10:45:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.141+0000 7ff7bffff700 1 -- 172.21.15.92:0/3219014256 learned_addr learned my addr 172.21.15.92:0/3219014256 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.141+0000 7ff7cd27c700 1 --2- 172.21.15.92:0/3219014256 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5623c7131ae0 0x5623c70e7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.141+0000 7ff7cca7b700 1 --2- 172.21.15.92:0/3219014256 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623c70ebc20 0x5623c7196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.142+0000 7ff7bffff700 1 -- 172.21.15.92:0/3219014256 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5623c7131ae0 msgr2=0x5623c70e7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.142+0000 7ff7bffff700 1 --2- 172.21.15.92:0/3219014256 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5623c7131ae0 0x5623c70e7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.142+0000 7ff7bffff700 1 -- 172.21.15.92:0/3219014256 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623c70ebc20 msgr2=0x5623c7196820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.142+0000 7ff7bffff700 1 --2- 172.21.15.92:0/3219014256 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623c70ebc20 0x5623c7196820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.142+0000 7ff7bffff700 1 -- 172.21.15.92:0/3219014256 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c000e040 con 0x5623c70ed9f0 2023-10-06T10:45:34.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.142+0000 7ff7bffff700 1 --2- 172.21.15.92:0/3219014256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c70ed9f0 0x5623c70e7880 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8025040 tx=0x7ff7b8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81849875ca5a82dc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.142+0000 7ff7a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:34.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.142+0000 7ff7a6ffd700 1 -- 172.21.15.92:0/3219014256 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b80015e0 con 0x5623c70ed9f0 2023-10-06T10:45:34.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.142+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/3219014256 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7c4057a50 con 0x5623c70ed9f0 2023-10-06T10:45:34.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.142+0000 7ff7a6ffd700 1 -- 172.21.15.92:0/3219014256 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7ff7b8006d40 con 0x5623c70ed9f0 2023-10-06T10:45:34.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.142+0000 7ff7a6ffd700 1 -- 172.21.15.92:0/3219014256 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7b8005e10 con 0x5623c70ed9f0 2023-10-06T10:45:34.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.142+0000 7ff7d7fd7b80 10 client.9156 fetch_fsmap learned FSMap version 109 2023-10-06T10:45:34.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.142+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/3219014256 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x5623c6ff04c0 con 0x5623c70ed9f0 2023-10-06T10:45:34.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.143+0000 7ff7a6ffd700 1 -- 172.21.15.92:0/3219014256 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c4057a50 con 0x5623c70ed9f0 2023-10-06T10:45:34.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.143+0000 7ff7a6ffd700 1 -- 172.21.15.92:0/3219014256 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 109) v1 ==== 3140+0+0 (secure 0 0 0) 0x7ff7b803c020 con 0x5623c70ed9f0 2023-10-06T10:45:34.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.143+0000 7ff7d7fd7b80 10 client.9156 fetch_fsmap finished waiting for FSMap version 109 2023-10-06T10:45:34.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.143+0000 7ff7d7fd7b80 10 client.9156 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2023-10-06T10:45:34.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.143+0000 7ff7d7fd7b80 20 client.9156 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:34.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.143+0000 7ff7d7fd7b80 1 --2- 172.21.15.92:0/3219014256 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5623c70e5790 0x5623c71197d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.143+0000 7ff7d7fd7b80 4 client.9156 mds_command: new command op to 8517 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:45:34.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.143+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/3219014256 --> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5623c6fdde50 con 0x5623c70e5790 2023-10-06T10:45:34.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.143+0000 7ff7cca7b700 1 --2- 172.21.15.92:0/3219014256 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5623c70e5790 0x5623c71197d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.143+0000 7ff7a6ffd700 10 client.9156 ms_handle_connect on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:34.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.143+0000 7ff7cca7b700 1 --2- 172.21.15.92:0/3219014256 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5623c70e5790 0x5623c71197d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=887a70ae1b13495 server_cookie=afb470fb1ffbdaf9 in_seq=0 out_seq=0 2023-10-06T10:45:34.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.143+0000 7ff7a6ffd700 1 -- 172.21.15.92:0/3219014256 <== mds.1 v2:172.21.15.92:6834/1979826278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5623c6fdde50 con 0x5623c70e5790 2023-10-06T10:45:34.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.143+0000 7ff7a6ffd700 10 client.9156 handle_command_reply: tid=0 2023-10-06T10:45:34.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.143+0000 7ff7d7fd7b80 1 client.9156 shutdown 2023-10-06T10:45:34.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/3219014256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c70ed9f0 msgr2=0x5623c70e7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7d7fd7b80 1 --2- 172.21.15.92:0/3219014256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c70ed9f0 0x5623c70e7880 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8025040 tx=0x7ff7b8019000 comp rx=0 tx=0).stop 2023-10-06T10:45:34.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/3219014256 shutdown_connections 2023-10-06T10:45:34.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7d7fd7b80 1 --2- 172.21.15.92:0/3219014256 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5623c7131ae0 0x5623c70e7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7d7fd7b80 1 --2- 172.21.15.92:0/3219014256 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5623c70ebc20 0x5623c7196820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7d7fd7b80 1 --2- 172.21.15.92:0/3219014256 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5623c70e5790 0x5623c71197d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7d7fd7b80 1 --2- 172.21.15.92:0/3219014256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623c70ed9f0 0x5623c70e7880 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7a6ffd700 0 client.9156 ms_handle_reset on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:34.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/3219014256 >> 172.21.15.92:0/3219014256 conn(0x5623c70ec0c0 msgr2=0x5623c712db70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:34.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/3219014256 shutdown_connections 2023-10-06T10:45:34.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7d7fd7b80 1 -- 172.21.15.92:0/3219014256 wait complete. 2023-10-06T10:45:34.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7d7fd7b80 20 client.9156 trim_cache size 0 max 16384 2023-10-06T10:45:34.152 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:34.152 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "PAUSED (0 inodes in the stack)", 2023-10-06T10:45:34.152 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:45:34.152 INFO:teuthology.orchestra.run.smithi092.stdout: "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8": { 2023-10-06T10:45:34.152 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:45:34.152 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8", 2023-10-06T10:45:34.152 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:45:34.152 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:34.152 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:34.152 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:34.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7cecdf700 1 -- 172.21.15.92:0/544290291 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff7b4066f30 msgr2=0x7ff7b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7cecdf700 1 --2- 172.21.15.92:0/544290291 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff7b4066f30 0x7ff7b40693e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80cac60 tx=0x7ff7b8007f50 comp rx=0 tx=0).stop 2023-10-06T10:45:34.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7cecdf700 1 -- 172.21.15.92:0/544290291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c80cd4d0 msgr2=0x7ff7c816c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7cecdf700 1 --2- 172.21.15.92:0/544290291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c80cd4d0 0x7ff7c816c730 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4013000 tx=0x7ff7c4019000 comp rx=0 tx=0).stop 2023-10-06T10:45:34.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7cecdf700 1 -- 172.21.15.92:0/544290291 shutdown_connections 2023-10-06T10:45:34.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7cecdf700 1 --2- 172.21.15.92:0/544290291 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff7b4066f30 0x7ff7b40693e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7cecdf700 1 --2- 172.21.15.92:0/544290291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7c8156e00 0x7ff7c816a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7cecdf700 1 --2- 172.21.15.92:0/544290291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c80cd4d0 0x7ff7c816c730 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7cecdf700 1 --2- 172.21.15.92:0/544290291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7c815a160 0x7ff7c816cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7cecdf700 1 -- 172.21.15.92:0/544290291 >> 172.21.15.92:0/544290291 conn(0x7ff7c8099700 msgr2=0x7ff7c8159830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:34.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7cecdf700 1 -- 172.21.15.92:0/544290291 shutdown_connections 2023-10-06T10:45:34.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.144+0000 7ff7cecdf700 1 -- 172.21.15.92:0/544290291 wait complete. 2023-10-06T10:45:34.156 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:45:34.157 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:2 scrub status 2023-10-06T10:45:34.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.272+0000 7f1d7b77a700 1 Processor -- start 2023-10-06T10:45:34.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.272+0000 7f1d7b77a700 1 -- start start 2023-10-06T10:45:34.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.273+0000 7f1d7b77a700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1d740c5250 0x7f1d740ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.273+0000 7f1d7b77a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d7415ac30 con 0x7f1d740cd300 2023-10-06T10:45:34.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.273+0000 7f1d7b77a700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d740cb6a0 con 0x7f1d74150940 2023-10-06T10:45:34.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.273+0000 7f1d7b77a700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d740cb090 con 0x7f1d740c5250 2023-10-06T10:45:34.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.273+0000 7f1d79d17700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1d740cd300 0x7f1d740b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45536/0 (socket says 172.21.15.92:45536) 2023-10-06T10:45:34.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.273+0000 7f1d79d17700 1 -- 172.21.15.92:0/185362494 learned_addr learned my addr 172.21.15.92:0/185362494 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:34.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.273+0000 7f1d78d15700 1 --2- 172.21.15.92:0/185362494 >> v2:172.21.15.124:3300/0 conn(0x7f1d740c5250 0x7f1d740ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.273+0000 7f1d73fff700 1 -- 172.21.15.92:0/185362494 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1134467151 0 0) 0x7f1d64001110 con 0x7f1d740cd300 2023-10-06T10:45:34.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.273+0000 7f1d73fff700 1 -- 172.21.15.92:0/185362494 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d5c003570 con 0x7f1d740cd300 2023-10-06T10:45:34.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.273+0000 7f1d73fff700 1 -- 172.21.15.92:0/185362494 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1846557594 0 0) 0x7f1d5c003570 con 0x7f1d740cd300 2023-10-06T10:45:34.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.273+0000 7f1d73fff700 1 -- 172.21.15.92:0/185362494 >> v2:172.21.15.124:3300/0 conn(0x7f1d740c5250 msgr2=0x7f1d740ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.273+0000 7f1d73fff700 1 --2- 172.21.15.92:0/185362494 >> v2:172.21.15.124:3300/0 conn(0x7f1d740c5250 0x7f1d740ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.273+0000 7f1d73fff700 1 -- 172.21.15.92:0/185362494 >> v1:172.21.15.124:6789/0 conn(0x7f1d74150940 legacy=0x7f1d74159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.273+0000 7f1d73fff700 1 -- 172.21.15.92:0/185362494 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d740cd700 con 0x7f1d740cd300 2023-10-06T10:45:34.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.273+0000 7f1d73fff700 1 -- 172.21.15.92:0/185362494 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f1d64001600 con 0x7f1d740cd300 2023-10-06T10:45:34.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.273+0000 7f1d73fff700 1 -- 172.21.15.92:0/185362494 >> v1:172.21.15.92:6789/0 conn(0x7f1d740cd300 legacy=0x7f1d740b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.273+0000 7f1d73fff700 1 --2- 172.21.15.92:0/185362494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d5c003a60 0x7f1d5c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.273+0000 7f1d73fff700 1 --2- 172.21.15.92:0/185362494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d5c004810 0x7f1d5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.273+0000 7f1d73fff700 1 --2- 172.21.15.92:0/185362494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d5c0051c0 0x7f1d5c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.274+0000 7f1d73fff700 1 -- 172.21.15.92:0/185362494 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d740cb090 con 0x7f1d5c003a60 2023-10-06T10:45:34.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.274+0000 7f1d73fff700 1 -- 172.21.15.92:0/185362494 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c00c0d0 con 0x7f1d5c0051c0 2023-10-06T10:45:34.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.274+0000 7f1d73fff700 1 -- 172.21.15.92:0/185362494 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c00c290 con 0x7f1d5c004810 2023-10-06T10:45:34.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.274+0000 7f1d79516700 1 --2- 172.21.15.92:0/185362494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d5c003a60 0x7f1d5c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.274+0000 7f1d78d15700 1 --2- 172.21.15.92:0/185362494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d5c004810 0x7f1d5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.274+0000 7f1d79d17700 1 --2- 172.21.15.92:0/185362494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d5c0051c0 0x7f1d5c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.274+0000 7f1d79516700 1 -- 172.21.15.92:0/185362494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d5c004810 msgr2=0x7f1d5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.274+0000 7f1d79516700 1 --2- 172.21.15.92:0/185362494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d5c004810 0x7f1d5c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.274+0000 7f1d79516700 1 -- 172.21.15.92:0/185362494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d5c0051c0 msgr2=0x7f1d5c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.274+0000 7f1d79516700 1 --2- 172.21.15.92:0/185362494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d5c0051c0 0x7f1d5c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.274+0000 7f1d79d17700 1 -- 172.21.15.92:0/185362494 reap_dead start 2023-10-06T10:45:34.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.274+0000 7f1d79516700 1 -- 172.21.15.92:0/185362494 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5c003570 con 0x7f1d5c003a60 2023-10-06T10:45:34.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.274+0000 7f1d79516700 1 --2- 172.21.15.92:0/185362494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d5c003a60 0x7f1d5c003e70 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00c2d0 tx=0x7f1d6c001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7da594d40e2d6e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.274+0000 7f1d73fff700 1 -- 172.21.15.92:0/185362494 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c001d80 con 0x7f1d5c003a60 2023-10-06T10:45:34.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.274+0000 7f1d73fff700 1 -- 172.21.15.92:0/185362494 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c00cba0 con 0x7f1d5c003a60 2023-10-06T10:45:34.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.274+0000 7f1d73fff700 1 -- 172.21.15.92:0/185362494 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c012400 con 0x7f1d5c003a60 2023-10-06T10:45:34.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.275+0000 7f1d7b77a700 1 -- 172.21.15.92:0/185362494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d5c003a60 msgr2=0x7f1d5c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.275+0000 7f1d7b77a700 1 --2- 172.21.15.92:0/185362494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d5c003a60 0x7f1d5c003e70 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00c2d0 tx=0x7f1d6c001a70 comp rx=0 tx=0).stop 2023-10-06T10:45:34.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.275+0000 7f1d7b77a700 1 -- 172.21.15.92:0/185362494 shutdown_connections 2023-10-06T10:45:34.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.275+0000 7f1d7b77a700 1 --2- 172.21.15.92:0/185362494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d5c003a60 0x7f1d5c003e70 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.275+0000 7f1d7b77a700 1 -- 172.21.15.92:0/185362494 >> 172.21.15.92:0/185362494 conn(0x7f1d74099700 msgr2=0x7f1d7406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:34.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.275+0000 7f1d7b77a700 1 -- 172.21.15.92:0/185362494 shutdown_connections 2023-10-06T10:45:34.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.275+0000 7f1d7b77a700 1 -- 172.21.15.92:0/185362494 wait complete. 2023-10-06T10:45:34.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.276+0000 7f1d7b77a700 1 Processor -- start 2023-10-06T10:45:34.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.276+0000 7f1d7b77a700 1 -- start start 2023-10-06T10:45:34.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.277+0000 7f1d7b77a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d5c003a60 0x7f1d740b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.277+0000 7f1d7b77a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d7416dcf0 0x7f1d7414e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.277+0000 7f1d7b77a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d7409af80 0x7f1d740c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.277+0000 7f1d7b77a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d740b3b50 con 0x7f1d7416dcf0 2023-10-06T10:45:34.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.277+0000 7f1d7b77a700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d740b3e30 con 0x7f1d7409af80 2023-10-06T10:45:34.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.277+0000 7f1d7b77a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d740b4110 con 0x7f1d5c003a60 2023-10-06T10:45:34.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.277+0000 7f1d78d15700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d7416dcf0 0x7f1d7414e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.277+0000 7f1d78d15700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d7416dcf0 0x7f1d7414e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55106/0 (socket says 172.21.15.92:55106) 2023-10-06T10:45:34.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.277+0000 7f1d78d15700 1 -- 172.21.15.92:0/2676064397 learned_addr learned my addr 172.21.15.92:0/2676064397 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:34.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.277+0000 7f1d78d15700 1 -- 172.21.15.92:0/2676064397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d5c003a60 msgr2=0x7f1d740b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.277+0000 7f1d78d15700 1 --2- 172.21.15.92:0/2676064397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d5c003a60 0x7f1d740b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.277+0000 7f1d78d15700 1 -- 172.21.15.92:0/2676064397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d7409af80 msgr2=0x7f1d740c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.277+0000 7f1d78d15700 1 --2- 172.21.15.92:0/2676064397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d7409af80 0x7f1d740c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.277+0000 7f1d78d15700 1 -- 172.21.15.92:0/2676064397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d740b46a0 con 0x7f1d7416dcf0 2023-10-06T10:45:34.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.277+0000 7f1d78d15700 1 --2- 172.21.15.92:0/2676064397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d7416dcf0 0x7f1d7414e0b0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f1d68011d40 tx=0x7f1d680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.277+0000 7f1d717fa700 1 -- 172.21.15.92:0/2676064397 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6800fac0 con 0x7f1d7416dcf0 2023-10-06T10:45:34.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.277+0000 7f1d717fa700 1 -- 172.21.15.92:0/2676064397 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6800fc20 con 0x7f1d7416dcf0 2023-10-06T10:45:34.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.277+0000 7f1d7b77a700 1 -- 172.21.15.92:0/2676064397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d740b4980 con 0x7f1d7416dcf0 2023-10-06T10:45:34.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.277+0000 7f1d717fa700 1 -- 172.21.15.92:0/2676064397 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d68014440 con 0x7f1d7416dcf0 2023-10-06T10:45:34.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.277+0000 7f1d7b77a700 1 -- 172.21.15.92:0/2676064397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7414e5f0 con 0x7f1d7416dcf0 2023-10-06T10:45:34.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.279+0000 7f1d717fa700 1 -- 172.21.15.92:0/2676064397 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1d68014a80 con 0x7f1d7416dcf0 2023-10-06T10:45:34.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.279+0000 7f1d717fa700 1 --2- 172.21.15.92:0/2676064397 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1d60066f30 0x7f1d600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.279+0000 7f1d717fa700 1 -- 172.21.15.92:0/2676064397 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f1d68018030 con 0x7f1d7416dcf0 2023-10-06T10:45:34.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.279+0000 7f1d79516700 1 --2- 172.21.15.92:0/2676064397 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1d60066f30 0x7f1d600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.280+0000 7f1d79516700 1 --2- 172.21.15.92:0/2676064397 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1d60066f30 0x7f1d600693e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0021d0 tx=0x7f1d6c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.281+0000 7f1d84a72b80 1 Processor -- start 2023-10-06T10:45:34.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.281+0000 7f1d84a72b80 1 -- start start 2023-10-06T10:45:34.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.282+0000 7f1d84a72b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f687ae0 0x561c5f687eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.282+0000 7f1d84a72b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c5f6439f0 0x561c5f6405d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.282+0000 7f1d84a72b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c5f641c20 0x561c5f640b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.282+0000 7f1d84a72b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5f5c0cd0 con 0x561c5f687ae0 2023-10-06T10:45:34.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.282+0000 7f1d84a72b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5f597ac0 con 0x561c5f6439f0 2023-10-06T10:45:34.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.282+0000 7f1d84a72b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c5f5952c0 con 0x561c5f641c20 2023-10-06T10:45:34.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.282+0000 7f1d79516700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f687ae0 0x561c5f687eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.282+0000 7f1d79516700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f687ae0 0x561c5f687eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55108/0 (socket says 172.21.15.92:55108) 2023-10-06T10:45:34.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.282+0000 7f1d79516700 1 -- 172.21.15.92:0/1500484020 learned_addr learned my addr 172.21.15.92:0/1500484020 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:34.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.282+0000 7f1d79d17700 1 --2- 172.21.15.92:0/1500484020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c5f641c20 0x561c5f640b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.282+0000 7f1d78d15700 1 --2- 172.21.15.92:0/1500484020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c5f6439f0 0x561c5f6405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.282+0000 7f1d79516700 1 -- 172.21.15.92:0/1500484020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c5f641c20 msgr2=0x561c5f640b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.282+0000 7f1d79516700 1 --2- 172.21.15.92:0/1500484020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c5f641c20 0x561c5f640b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.282+0000 7f1d79516700 1 -- 172.21.15.92:0/1500484020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c5f6439f0 msgr2=0x561c5f6405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.282+0000 7f1d79516700 1 --2- 172.21.15.92:0/1500484020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c5f6439f0 0x561c5f6405d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.282+0000 7f1d79516700 1 -- 172.21.15.92:0/1500484020 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c5f42b2c0 con 0x561c5f687ae0 2023-10-06T10:45:34.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.282+0000 7f1d78d15700 1 --2- 172.21.15.92:0/1500484020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c5f6439f0 0x561c5f6405d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:34.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.282+0000 7f1d79516700 1 --2- 172.21.15.92:0/1500484020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f687ae0 0x561c5f687eb0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c000fc0 tx=0x7f1d6c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74aecbac2cc1a3a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.282+0000 7f1d5a7fc700 1 -- 172.21.15.92:0/1500484020 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c012980 con 0x561c5f687ae0 2023-10-06T10:45:34.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.282+0000 7f1d5a7fc700 1 -- 172.21.15.92:0/1500484020 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c012ae0 con 0x561c5f687ae0 2023-10-06T10:45:34.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.282+0000 7f1d5a7fc700 1 -- 172.21.15.92:0/1500484020 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c012e80 con 0x561c5f687ae0 2023-10-06T10:45:34.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.283+0000 7f1d84a72b80 1 -- 172.21.15.92:0/1500484020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f687ae0 msgr2=0x561c5f687eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.283+0000 7f1d84a72b80 1 --2- 172.21.15.92:0/1500484020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f687ae0 0x561c5f687eb0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c000fc0 tx=0x7f1d6c01e000 comp rx=0 tx=0).stop 2023-10-06T10:45:34.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.283+0000 7f1d84a72b80 1 -- 172.21.15.92:0/1500484020 shutdown_connections 2023-10-06T10:45:34.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.283+0000 7f1d84a72b80 1 --2- 172.21.15.92:0/1500484020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c5f641c20 0x561c5f640b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.283+0000 7f1d84a72b80 1 --2- 172.21.15.92:0/1500484020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f687ae0 0x561c5f687eb0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.283+0000 7f1d84a72b80 1 --2- 172.21.15.92:0/1500484020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c5f6439f0 0x561c5f6405d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.283+0000 7f1d84a72b80 1 -- 172.21.15.92:0/1500484020 >> 172.21.15.92:0/1500484020 conn(0x561c5f6420c0 msgr2=0x561c5f642490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:34.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.283+0000 7f1d84a72b80 1 -- 172.21.15.92:0/1500484020 shutdown_connections 2023-10-06T10:45:34.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.283+0000 7f1d84a72b80 1 -- 172.21.15.92:0/1500484020 wait complete. 2023-10-06T10:45:34.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.284+0000 7f1d84a72b80 1 -- start start 2023-10-06T10:45:34.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.284+0000 7f1d84a72b80 1 Processor -- start 2023-10-06T10:45:34.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d84a72b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f641c20 0x561c5f66d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d84a72b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c5f6439f0 0x561c5f668950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d84a72b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c5f687ae0 0x561c5f668e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d84a72b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5f5910b0 con 0x561c5f641c20 2023-10-06T10:45:34.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d84a72b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5f53c8e0 con 0x561c5f687ae0 2023-10-06T10:45:34.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d84a72b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c5f557ef0 con 0x561c5f6439f0 2023-10-06T10:45:34.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d84a72b80 10 client.0 _ll_register_callbacks cb 0x561c5f42df80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:34.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d79516700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f641c20 0x561c5f66d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d79516700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f641c20 0x561c5f66d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55112/0 (socket says 172.21.15.92:55112) 2023-10-06T10:45:34.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d79516700 1 -- 172.21.15.92:0/1770697895 learned_addr learned my addr 172.21.15.92:0/1770697895 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:34.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d78d15700 1 --2- 172.21.15.92:0/1770697895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c5f6439f0 0x561c5f668950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d79516700 1 -- 172.21.15.92:0/1770697895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c5f6439f0 msgr2=0x561c5f668950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d79516700 1 --2- 172.21.15.92:0/1770697895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c5f6439f0 0x561c5f668950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d79516700 1 -- 172.21.15.92:0/1770697895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c5f687ae0 msgr2=0x561c5f668e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d79d17700 1 --2- 172.21.15.92:0/1770697895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c5f687ae0 0x561c5f668e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d79516700 1 --2- 172.21.15.92:0/1770697895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c5f687ae0 0x561c5f668e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d79516700 1 -- 172.21.15.92:0/1770697895 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d5c003570 con 0x561c5f641c20 2023-10-06T10:45:34.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d78d15700 1 --2- 172.21.15.92:0/1770697895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c5f6439f0 0x561c5f668950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:34.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d79516700 1 --2- 172.21.15.92:0/1770697895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f641c20 0x561c5f66d070 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c012880 tx=0x7f1d6c02a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a53582a023fb99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d537fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:34.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d537fe700 1 -- 172.21.15.92:0/1770697895 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c0125e0 con 0x561c5f641c20 2023-10-06T10:45:34.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d537fe700 1 -- 172.21.15.92:0/1770697895 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f1d6c02a870 con 0x561c5f641c20 2023-10-06T10:45:34.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d84a72b80 1 -- 172.21.15.92:0/1770697895 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c5f6356e0 con 0x561c5f641c20 2023-10-06T10:45:34.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d537fe700 1 -- 172.21.15.92:0/1770697895 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d6c02ac40 con 0x561c5f641c20 2023-10-06T10:45:34.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d84a72b80 10 client.9168 fetch_fsmap learned FSMap version 109 2023-10-06T10:45:34.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.285+0000 7f1d84a72b80 1 -- 172.21.15.92:0/1770697895 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x561c5f3ce030 con 0x561c5f641c20 2023-10-06T10:45:34.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.286+0000 7f1d537fe700 1 -- 172.21.15.92:0/1770697895 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561c5f6356e0 con 0x561c5f641c20 2023-10-06T10:45:34.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.286+0000 7f1d537fe700 1 -- 172.21.15.92:0/1770697895 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 109) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f1d6c03b440 con 0x561c5f641c20 2023-10-06T10:45:34.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.286+0000 7f1d84a72b80 10 client.9168 fetch_fsmap finished waiting for FSMap version 109 2023-10-06T10:45:34.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.286+0000 7f1d84a72b80 10 client.9168 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2023-10-06T10:45:34.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.286+0000 7f1d84a72b80 20 client.9168 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:34.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.286+0000 7f1d84a72b80 1 --2- 172.21.15.92:0/1770697895 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x561c5f6819b0 0x561c5f682410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.286+0000 7f1d84a72b80 4 client.9168 mds_command: new command op to 6719 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:34.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.286+0000 7f1d84a72b80 1 -- 172.21.15.92:0/1770697895 --> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c5f612610 con 0x561c5f6819b0 2023-10-06T10:45:34.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.286+0000 7f1d78d15700 1 --2- 172.21.15.92:0/1770697895 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x561c5f6819b0 0x561c5f682410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.286+0000 7f1d537fe700 10 client.9168 ms_handle_connect on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:34.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.286+0000 7f1d78d15700 1 --2- 172.21.15.92:0/1770697895 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x561c5f6819b0 0x561c5f682410 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc8c4dfdcc7b5a9c server_cookie=dae36519a27af888 in_seq=0 out_seq=0 2023-10-06T10:45:34.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.287+0000 7f1d537fe700 1 -- 172.21.15.92:0/1770697895 <== mds.2 v2:172.21.15.124:6833/2307521923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x561c5f612610 con 0x561c5f6819b0 2023-10-06T10:45:34.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.287+0000 7f1d537fe700 10 client.9168 handle_command_reply: tid=0 2023-10-06T10:45:34.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.287+0000 7f1d84a72b80 1 client.9168 shutdown 2023-10-06T10:45:34.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.287+0000 7f1d84a72b80 1 -- 172.21.15.92:0/1770697895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f641c20 msgr2=0x561c5f66d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.287+0000 7f1d84a72b80 1 --2- 172.21.15.92:0/1770697895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f641c20 0x561c5f66d070 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c012880 tx=0x7f1d6c02a000 comp rx=0 tx=0).stop 2023-10-06T10:45:34.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.287+0000 7f1d84a72b80 1 -- 172.21.15.92:0/1770697895 shutdown_connections 2023-10-06T10:45:34.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.287+0000 7f1d84a72b80 1 --2- 172.21.15.92:0/1770697895 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x561c5f6819b0 0x561c5f682410 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.287+0000 7f1d84a72b80 1 --2- 172.21.15.92:0/1770697895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c5f687ae0 0x561c5f668e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.287+0000 7f1d84a72b80 1 --2- 172.21.15.92:0/1770697895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f641c20 0x561c5f66d070 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.287+0000 7f1d537fe700 0 client.9168 ms_handle_reset on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:34.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.287+0000 7f1d84a72b80 1 --2- 172.21.15.92:0/1770697895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c5f6439f0 0x561c5f668950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.287+0000 7f1d84a72b80 1 -- 172.21.15.92:0/1770697895 >> 172.21.15.92:0/1770697895 conn(0x561c5f6420c0 msgr2=0x561c5f63b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:34.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.287+0000 7f1d84a72b80 1 -- 172.21.15.92:0/1770697895 shutdown_connections 2023-10-06T10:45:34.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.287+0000 7f1d84a72b80 1 -- 172.21.15.92:0/1770697895 wait complete. 2023-10-06T10:45:34.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.287+0000 7f1d84a72b80 20 client.9168 trim_cache size 0 max 16384 2023-10-06T10:45:34.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.303+0000 7f1d84a72b80 1 Processor -- start 2023-10-06T10:45:34.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.303+0000 7f1d84a72b80 1 -- start start 2023-10-06T10:45:34.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.304+0000 7f1d84a72b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c5f641c20 0x561c5f682d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.304+0000 7f1d84a72b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f6439f0 0x561c5f687eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.304+0000 7f1d84a72b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c5f6819b0 0x561c5f668b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.304+0000 7f1d84a72b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5f413e80 con 0x561c5f6439f0 2023-10-06T10:45:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.304+0000 7f1d84a72b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5f6825b0 con 0x561c5f6819b0 2023-10-06T10:45:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.304+0000 7f1d84a72b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c5f683270 con 0x561c5f641c20 2023-10-06T10:45:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.304+0000 7f1d78d15700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f6439f0 0x561c5f687eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.304+0000 7f1d78d15700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f6439f0 0x561c5f687eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55116/0 (socket says 172.21.15.92:55116) 2023-10-06T10:45:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.304+0000 7f1d78d15700 1 -- 172.21.15.92:0/1992671955 learned_addr learned my addr 172.21.15.92:0/1992671955 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.304+0000 7f1d79516700 1 --2- 172.21.15.92:0/1992671955 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c5f641c20 0x561c5f682d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.304+0000 7f1d78d15700 1 -- 172.21.15.92:0/1992671955 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c5f641c20 msgr2=0x561c5f682d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.304+0000 7f1d78d15700 1 --2- 172.21.15.92:0/1992671955 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c5f641c20 0x561c5f682d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.304+0000 7f1d79d17700 1 --2- 172.21.15.92:0/1992671955 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c5f6819b0 0x561c5f668b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.304+0000 7f1d78d15700 1 -- 172.21.15.92:0/1992671955 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c5f6819b0 msgr2=0x561c5f668b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.304+0000 7f1d78d15700 1 --2- 172.21.15.92:0/1992671955 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c5f6819b0 0x561c5f668b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.304+0000 7f1d78d15700 1 -- 172.21.15.92:0/1992671955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c5f55f7d0 con 0x561c5f6439f0 2023-10-06T10:45:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.304+0000 7f1d78d15700 1 --2- 172.21.15.92:0/1992671955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f6439f0 0x561c5f687eb0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f1d68055e20 tx=0x7f1d68085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56d2502cb8109c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.304+0000 7f1d5affd700 1 -- 172.21.15.92:0/1992671955 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d68058040 con 0x561c5f6439f0 2023-10-06T10:45:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.304+0000 7f1d5affd700 1 -- 172.21.15.92:0/1992671955 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d68059370 con 0x561c5f6439f0 2023-10-06T10:45:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.304+0000 7f1d5affd700 1 -- 172.21.15.92:0/1992671955 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6805d0c0 con 0x561c5f6439f0 2023-10-06T10:45:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.305+0000 7f1d84a72b80 1 -- 172.21.15.92:0/1992671955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f6439f0 msgr2=0x561c5f687eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.305+0000 7f1d84a72b80 1 --2- 172.21.15.92:0/1992671955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f6439f0 0x561c5f687eb0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f1d68055e20 tx=0x7f1d68085040 comp rx=0 tx=0).stop 2023-10-06T10:45:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.305+0000 7f1d84a72b80 1 -- 172.21.15.92:0/1992671955 shutdown_connections 2023-10-06T10:45:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.305+0000 7f1d84a72b80 1 --2- 172.21.15.92:0/1992671955 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c5f6819b0 0x561c5f668b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.305+0000 7f1d84a72b80 1 --2- 172.21.15.92:0/1992671955 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c5f641c20 0x561c5f682d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.305+0000 7f1d84a72b80 1 --2- 172.21.15.92:0/1992671955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f6439f0 0x561c5f687eb0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.305+0000 7f1d84a72b80 1 -- 172.21.15.92:0/1992671955 >> 172.21.15.92:0/1992671955 conn(0x561c5f6420c0 msgr2=0x561c5f642490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:34.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.305+0000 7f1d84a72b80 1 -- 172.21.15.92:0/1992671955 shutdown_connections 2023-10-06T10:45:34.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.305+0000 7f1d84a72b80 1 -- 172.21.15.92:0/1992671955 wait complete. 2023-10-06T10:45:34.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.305+0000 7f1d84a72b80 1 -- start start 2023-10-06T10:45:34.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.306+0000 7f1d84a72b80 1 Processor -- start 2023-10-06T10:45:34.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.306+0000 7f1d84a72b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f641c20 0x561c5f68afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.306+0000 7f1d84a72b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c5f6439f0 0x561c5f66f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.306+0000 7f1d84a72b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c5f6819b0 0x561c5f66fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.306+0000 7f1d84a72b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5f6ed8d0 con 0x561c5f641c20 2023-10-06T10:45:34.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.306+0000 7f1d84a72b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5f6683e0 con 0x561c5f6439f0 2023-10-06T10:45:34.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.306+0000 7f1d84a72b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c5f666e00 con 0x561c5f6819b0 2023-10-06T10:45:34.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.306+0000 7f1d84a72b80 10 client.0 _ll_register_callbacks cb 0x561c5f42df80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:34.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.306+0000 7f1d79516700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f641c20 0x561c5f68afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.306+0000 7f1d79516700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f641c20 0x561c5f68afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55128/0 (socket says 172.21.15.92:55128) 2023-10-06T10:45:34.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.306+0000 7f1d79516700 1 -- 172.21.15.92:0/2165323327 learned_addr learned my addr 172.21.15.92:0/2165323327 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:34.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.306+0000 7f1d78d15700 1 --2- 172.21.15.92:0/2165323327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c5f6439f0 0x561c5f66f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.306+0000 7f1d79d17700 1 --2- 172.21.15.92:0/2165323327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c5f6819b0 0x561c5f66fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.307+0000 7f1d79516700 1 -- 172.21.15.92:0/2165323327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c5f6819b0 msgr2=0x561c5f66fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.307+0000 7f1d79516700 1 --2- 172.21.15.92:0/2165323327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c5f6819b0 0x561c5f66fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.307+0000 7f1d79516700 1 -- 172.21.15.92:0/2165323327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c5f6439f0 msgr2=0x561c5f66f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.307+0000 7f1d79516700 1 --2- 172.21.15.92:0/2165323327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c5f6439f0 0x561c5f66f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.307+0000 7f1d79516700 1 -- 172.21.15.92:0/2165323327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7414e5f0 con 0x561c5f641c20 2023-10-06T10:45:34.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.307+0000 7f1d79516700 1 --2- 172.21.15.92:0/2165323327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f641c20 0x561c5f68afa0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c001da0 tx=0x7f1d6c035070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e7e84714e7e0e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.307+0000 7f1d537fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:34.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.307+0000 7f1d537fe700 1 -- 172.21.15.92:0/2165323327 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c01db60 con 0x561c5f641c20 2023-10-06T10:45:34.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.307+0000 7f1d537fe700 1 -- 172.21.15.92:0/2165323327 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f1d6c01dcc0 con 0x561c5f641c20 2023-10-06T10:45:34.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.307+0000 7f1d84a72b80 1 -- 172.21.15.92:0/2165323327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d6c012ae0 con 0x561c5f641c20 2023-10-06T10:45:34.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.307+0000 7f1d537fe700 1 -- 172.21.15.92:0/2165323327 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d6c01de80 con 0x561c5f641c20 2023-10-06T10:45:34.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.307+0000 7f1d84a72b80 10 client.9174 fetch_fsmap learned FSMap version 109 2023-10-06T10:45:34.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.307+0000 7f1d84a72b80 1 -- 172.21.15.92:0/2165323327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x561c5f5464c0 con 0x561c5f641c20 2023-10-06T10:45:34.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.307+0000 7f1d537fe700 1 -- 172.21.15.92:0/2165323327 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c012ae0 con 0x561c5f641c20 2023-10-06T10:45:34.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.307+0000 7f1d537fe700 1 -- 172.21.15.92:0/2165323327 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 109) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f1d6c01d420 con 0x561c5f641c20 2023-10-06T10:45:34.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.307+0000 7f1d84a72b80 10 client.9174 fetch_fsmap finished waiting for FSMap version 109 2023-10-06T10:45:34.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.307+0000 7f1d84a72b80 10 client.9174 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2023-10-06T10:45:34.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.307+0000 7f1d84a72b80 20 client.9174 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:34.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.307+0000 7f1d84a72b80 1 --2- 172.21.15.92:0/2165323327 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x561c5f687ae0 0x561c5f6ec6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.307+0000 7f1d84a72b80 4 client.9174 mds_command: new command op to 6719 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:45:34.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.307+0000 7f1d84a72b80 1 -- 172.21.15.92:0/2165323327 --> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561c5f533e50 con 0x561c5f687ae0 2023-10-06T10:45:34.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.307+0000 7f1d78d15700 1 --2- 172.21.15.92:0/2165323327 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x561c5f687ae0 0x561c5f6ec6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.308+0000 7f1d537fe700 10 client.9174 ms_handle_connect on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:34.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.308+0000 7f1d78d15700 1 --2- 172.21.15.92:0/2165323327 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x561c5f687ae0 0x561c5f6ec6a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=30be3057e9b7b6bc server_cookie=e13f2fc7f1ba9f59 in_seq=0 out_seq=0 2023-10-06T10:45:34.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.308+0000 7f1d537fe700 1 -- 172.21.15.92:0/2165323327 <== mds.2 v2:172.21.15.124:6833/2307521923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x561c5f533e50 con 0x561c5f687ae0 2023-10-06T10:45:34.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.308+0000 7f1d537fe700 10 client.9174 handle_command_reply: tid=0 2023-10-06T10:45:34.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.308+0000 7f1d84a72b80 1 client.9174 shutdown 2023-10-06T10:45:34.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.308+0000 7f1d84a72b80 1 -- 172.21.15.92:0/2165323327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f641c20 msgr2=0x561c5f68afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.308+0000 7f1d84a72b80 1 --2- 172.21.15.92:0/2165323327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f641c20 0x561c5f68afa0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c001da0 tx=0x7f1d6c035070 comp rx=0 tx=0).stop 2023-10-06T10:45:34.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.308+0000 7f1d84a72b80 1 -- 172.21.15.92:0/2165323327 shutdown_connections 2023-10-06T10:45:34.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.308+0000 7f1d84a72b80 1 --2- 172.21.15.92:0/2165323327 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x561c5f687ae0 0x561c5f6ec6a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.308+0000 7f1d84a72b80 1 --2- 172.21.15.92:0/2165323327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561c5f6819b0 0x561c5f66fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.308+0000 7f1d537fe700 0 client.9174 ms_handle_reset on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:34.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.308+0000 7f1d84a72b80 1 --2- 172.21.15.92:0/2165323327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c5f641c20 0x561c5f68afa0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.308+0000 7f1d84a72b80 1 --2- 172.21.15.92:0/2165323327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561c5f6439f0 0x561c5f66f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.308+0000 7f1d84a72b80 1 -- 172.21.15.92:0/2165323327 >> 172.21.15.92:0/2165323327 conn(0x561c5f6420c0 msgr2=0x561c5f66a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:34.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.308+0000 7f1d84a72b80 1 -- 172.21.15.92:0/2165323327 shutdown_connections 2023-10-06T10:45:34.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.308+0000 7f1d84a72b80 1 -- 172.21.15.92:0/2165323327 wait complete. 2023-10-06T10:45:34.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.308+0000 7f1d84a72b80 20 client.9174 trim_cache size 0 max 16384 2023-10-06T10:45:34.317 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:34.317 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "PAUSED (0 inodes in the stack)", 2023-10-06T10:45:34.317 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:45:34.317 INFO:teuthology.orchestra.run.smithi092.stdout: "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8": { 2023-10-06T10:45:34.317 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:45:34.317 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8", 2023-10-06T10:45:34.317 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:45:34.317 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:34.318 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:34.318 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:34.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.309+0000 7f1d7b77a700 1 -- 172.21.15.92:0/2676064397 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1d60066f30 msgr2=0x7f1d600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.309+0000 7f1d7b77a700 1 --2- 172.21.15.92:0/2676064397 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1d60066f30 0x7f1d600693e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0021d0 tx=0x7f1d6c016000 comp rx=0 tx=0).stop 2023-10-06T10:45:34.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.309+0000 7f1d7b77a700 1 -- 172.21.15.92:0/2676064397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d7416dcf0 msgr2=0x7f1d7414e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.309+0000 7f1d7b77a700 1 --2- 172.21.15.92:0/2676064397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d7416dcf0 0x7f1d7414e0b0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f1d68011d40 tx=0x7f1d680091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.309+0000 7f1d7b77a700 1 -- 172.21.15.92:0/2676064397 shutdown_connections 2023-10-06T10:45:34.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.309+0000 7f1d7b77a700 1 --2- 172.21.15.92:0/2676064397 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1d60066f30 0x7f1d600693e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.309+0000 7f1d7b77a700 1 --2- 172.21.15.92:0/2676064397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d7409af80 0x7f1d740c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.309+0000 7f1d7b77a700 1 --2- 172.21.15.92:0/2676064397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d5c003a60 0x7f1d740b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.309+0000 7f1d7b77a700 1 --2- 172.21.15.92:0/2676064397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d7416dcf0 0x7f1d7414e0b0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.309+0000 7f1d7b77a700 1 -- 172.21.15.92:0/2676064397 >> 172.21.15.92:0/2676064397 conn(0x7f1d74099700 msgr2=0x7f1d740ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:34.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.309+0000 7f1d7b77a700 1 -- 172.21.15.92:0/2676064397 shutdown_connections 2023-10-06T10:45:34.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.309+0000 7f1d7b77a700 1 -- 172.21.15.92:0/2676064397 wait complete. 2023-10-06T10:45:34.321 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:45:34.321 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub status 2023-10-06T10:45:34.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.437+0000 7ff77b199700 1 Processor -- start 2023-10-06T10:45:34.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.437+0000 7ff77b199700 1 -- start start 2023-10-06T10:45:34.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.437+0000 7ff77b199700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff774150940 0x7ff774159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.437+0000 7ff77b199700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7740cd4d0 0x7ff7740b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.437+0000 7ff77b199700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77415ac30 con 0x7ff77415a160 2023-10-06T10:45:34.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.437+0000 7ff77b199700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7740cb6a0 con 0x7ff774150940 2023-10-06T10:45:34.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.437+0000 7ff77b199700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740cb090 con 0x7ff7740cd4d0 2023-10-06T10:45:34.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.438+0000 7ff778f35700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff774150940 0x7ff774159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.438+0000 7ff778f35700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff774150940 0x7ff774159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55132/0 (socket says 172.21.15.92:55132) 2023-10-06T10:45:34.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.438+0000 7ff76bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff77415a160 0x7ff7740ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45548/0 (socket says 172.21.15.92:45548) 2023-10-06T10:45:34.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.438+0000 7ff778f35700 1 -- 172.21.15.92:0/1437341359 learned_addr learned my addr 172.21.15.92:0/1437341359 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:34.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.438+0000 7ff779736700 1 --2- 172.21.15.92:0/1437341359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7740cd4d0 0x7ff7740b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.438+0000 7ff778f35700 1 -- 172.21.15.92:0/1437341359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7740cd4d0 msgr2=0x7ff7740b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.438+0000 7ff778f35700 1 --2- 172.21.15.92:0/1437341359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7740cd4d0 0x7ff7740b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.438+0000 7ff778f35700 1 -- 172.21.15.92:0/1437341359 >> v1:172.21.15.92:6789/0 conn(0x7ff77415a160 legacy=0x7ff7740ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.438+0000 7ff778f35700 1 -- 172.21.15.92:0/1437341359 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77415a670 con 0x7ff774150940 2023-10-06T10:45:34.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.438+0000 7ff76b7fe700 1 -- 172.21.15.92:0/1437341359 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3849684572 0 0) 0x7ff764001110 con 0x7ff77415a160 2023-10-06T10:45:34.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.438+0000 7ff779736700 1 --2- 172.21.15.92:0/1437341359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7740cd4d0 0x7ff7740b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:34.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.438+0000 7ff778f35700 1 --2- 172.21.15.92:0/1437341359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff774150940 0x7ff774159c20 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7ff770009990 tx=0x7ff770009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23b3a8d3418cdf3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.438+0000 7ff76b7fe700 1 -- 172.21.15.92:0/1437341359 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770005c10 con 0x7ff774150940 2023-10-06T10:45:34.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.438+0000 7ff76b7fe700 1 -- 172.21.15.92:0/1437341359 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff770005d70 con 0x7ff774150940 2023-10-06T10:45:34.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.438+0000 7ff76b7fe700 1 -- 172.21.15.92:0/1437341359 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77000b880 con 0x7ff774150940 2023-10-06T10:45:34.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.439+0000 7ff77b199700 1 -- 172.21.15.92:0/1437341359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff774150940 msgr2=0x7ff774159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.439+0000 7ff77b199700 1 --2- 172.21.15.92:0/1437341359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff774150940 0x7ff774159c20 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7ff770009990 tx=0x7ff770009220 comp rx=0 tx=0).stop 2023-10-06T10:45:34.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.439+0000 7ff77b199700 1 -- 172.21.15.92:0/1437341359 shutdown_connections 2023-10-06T10:45:34.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.439+0000 7ff77b199700 1 --2- 172.21.15.92:0/1437341359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7740cd4d0 0x7ff7740b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.439+0000 7ff77b199700 1 --2- 172.21.15.92:0/1437341359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff774150940 0x7ff774159c20 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.439+0000 7ff77b199700 1 -- 172.21.15.92:0/1437341359 >> 172.21.15.92:0/1437341359 conn(0x7ff774099700 msgr2=0x7ff77406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:34.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.439+0000 7ff77b199700 1 -- 172.21.15.92:0/1437341359 shutdown_connections 2023-10-06T10:45:34.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.439+0000 7ff77b199700 1 -- 172.21.15.92:0/1437341359 wait complete. 2023-10-06T10:45:34.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.440+0000 7ff77b199700 1 Processor -- start 2023-10-06T10:45:34.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.440+0000 7ff77b199700 1 -- start start 2023-10-06T10:45:34.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff77b199700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7740cd4d0 0x7ff7741568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff77b199700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff77415a160 0x7ff7741500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff77b199700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7740b5a50 0x7ff77415f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff77b199700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77415c260 con 0x7ff77415a160 2023-10-06T10:45:34.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff77b199700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff774156eb0 con 0x7ff7740cd4d0 2023-10-06T10:45:34.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff77b199700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff774157190 con 0x7ff7740b5a50 2023-10-06T10:45:34.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff76bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff77415a160 0x7ff7741500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff76bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff77415a160 0x7ff7741500c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55148/0 (socket says 172.21.15.92:55148) 2023-10-06T10:45:34.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff76bfff700 1 -- 172.21.15.92:0/2871963226 learned_addr learned my addr 172.21.15.92:0/2871963226 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:34.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff778f35700 1 --2- 172.21.15.92:0/2871963226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7740cd4d0 0x7ff7741568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff779736700 1 --2- 172.21.15.92:0/2871963226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7740b5a50 0x7ff77415f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff76bfff700 1 -- 172.21.15.92:0/2871963226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7740b5a50 msgr2=0x7ff77415f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff76bfff700 1 --2- 172.21.15.92:0/2871963226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7740b5a50 0x7ff77415f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff76bfff700 1 -- 172.21.15.92:0/2871963226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7740cd4d0 msgr2=0x7ff7741568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff76bfff700 1 --2- 172.21.15.92:0/2871963226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7740cd4d0 0x7ff7741568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff76bfff700 1 -- 172.21.15.92:0/2871963226 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff770009480 con 0x7ff77415a160 2023-10-06T10:45:34.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff778f35700 1 --2- 172.21.15.92:0/2871963226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7740cd4d0 0x7ff7741568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:34.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff779736700 1 --2- 172.21.15.92:0/2871963226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7740b5a50 0x7ff77415f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:34.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff76bfff700 1 --2- 172.21.15.92:0/2871963226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff77415a160 0x7ff7741500c0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7ff76400c510 tx=0x7ff76400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff768ff9700 1 -- 172.21.15.92:0/2871963226 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff764005a70 con 0x7ff77415a160 2023-10-06T10:45:34.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff768ff9700 1 -- 172.21.15.92:0/2871963226 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff764005bf0 con 0x7ff77415a160 2023-10-06T10:45:34.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff768ff9700 1 -- 172.21.15.92:0/2871963226 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7640070d0 con 0x7ff77415a160 2023-10-06T10:45:34.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff77b199700 1 -- 172.21.15.92:0/2871963226 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77415bef0 con 0x7ff77415a160 2023-10-06T10:45:34.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.441+0000 7ff77b199700 1 -- 172.21.15.92:0/2871963226 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7740b5ed0 con 0x7ff77415a160 2023-10-06T10:45:34.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.443+0000 7ff768ff9700 1 -- 172.21.15.92:0/2871963226 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff764007440 con 0x7ff77415a160 2023-10-06T10:45:34.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.443+0000 7ff768ff9700 1 --2- 172.21.15.92:0/2871963226 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff760066f30 0x7ff7600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.444+0000 7ff768ff9700 1 -- 172.21.15.92:0/2871963226 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7ff764083c70 con 0x7ff77415a160 2023-10-06T10:45:34.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.444+0000 7ff778f35700 1 --2- 172.21.15.92:0/2871963226 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff760066f30 0x7ff7600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.444+0000 7ff778f35700 1 --2- 172.21.15.92:0/2871963226 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff760066f30 0x7ff7600693e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff770009260 tx=0x7ff770016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.445+0000 7ff784491b80 1 Processor -- start 2023-10-06T10:45:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.445+0000 7ff784491b80 1 -- start start 2023-10-06T10:45:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.446+0000 7ff784491b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56318e8a9ae0 0x56318e8a9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.446+0000 7ff784491b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56318e8659f0 0x56318e8625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.446+0000 7ff784491b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e863c20 0x56318e862b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.446+0000 7ff784491b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318e7e2cd0 con 0x56318e863c20 2023-10-06T10:45:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.446+0000 7ff784491b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318e7b9ac0 con 0x56318e8a9ae0 2023-10-06T10:45:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.446+0000 7ff784491b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56318e7b72c0 con 0x56318e8659f0 2023-10-06T10:45:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.446+0000 7ff779736700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e863c20 0x56318e862b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.446+0000 7ff779736700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e863c20 0x56318e862b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55156/0 (socket says 172.21.15.92:55156) 2023-10-06T10:45:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.446+0000 7ff779736700 1 -- 172.21.15.92:0/1173835635 learned_addr learned my addr 172.21.15.92:0/1173835635 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.446+0000 7ff778f35700 1 --2- 172.21.15.92:0/1173835635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56318e8a9ae0 0x56318e8a9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.446+0000 7ff76bfff700 1 --2- 172.21.15.92:0/1173835635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56318e8659f0 0x56318e8625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.446+0000 7ff779736700 1 -- 172.21.15.92:0/1173835635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56318e8659f0 msgr2=0x56318e8625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.446+0000 7ff779736700 1 --2- 172.21.15.92:0/1173835635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56318e8659f0 0x56318e8625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.446+0000 7ff779736700 1 -- 172.21.15.92:0/1173835635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56318e8a9ae0 msgr2=0x56318e8a9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.446+0000 7ff779736700 1 --2- 172.21.15.92:0/1173835635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56318e8a9ae0 0x56318e8a9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.446+0000 7ff779736700 1 -- 172.21.15.92:0/1173835635 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56318e64d2c0 con 0x56318e863c20 2023-10-06T10:45:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.447+0000 7ff778f35700 1 --2- 172.21.15.92:0/1173835635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56318e8a9ae0 0x56318e8a9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.447+0000 7ff779736700 1 --2- 172.21.15.92:0/1173835635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e863c20 0x56318e862b10 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0076c0 tx=0x7ff76c009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a600d2f26b944c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.447+0000 7ff759ffb700 1 -- 172.21.15.92:0/1173835635 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c013580 con 0x56318e863c20 2023-10-06T10:45:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.447+0000 7ff759ffb700 1 -- 172.21.15.92:0/1173835635 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c00f050 con 0x56318e863c20 2023-10-06T10:45:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.447+0000 7ff759ffb700 1 -- 172.21.15.92:0/1173835635 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c013930 con 0x56318e863c20 2023-10-06T10:45:34.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.447+0000 7ff784491b80 1 -- 172.21.15.92:0/1173835635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e863c20 msgr2=0x56318e862b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.447+0000 7ff784491b80 1 --2- 172.21.15.92:0/1173835635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e863c20 0x56318e862b10 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0076c0 tx=0x7ff76c009cb0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.447+0000 7ff784491b80 1 -- 172.21.15.92:0/1173835635 shutdown_connections 2023-10-06T10:45:34.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.447+0000 7ff784491b80 1 --2- 172.21.15.92:0/1173835635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e863c20 0x56318e862b10 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.447+0000 7ff784491b80 1 --2- 172.21.15.92:0/1173835635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56318e8a9ae0 0x56318e8a9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.447+0000 7ff784491b80 1 --2- 172.21.15.92:0/1173835635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56318e8659f0 0x56318e8625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.447+0000 7ff784491b80 1 -- 172.21.15.92:0/1173835635 >> 172.21.15.92:0/1173835635 conn(0x56318e8640c0 msgr2=0x56318e864490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:34.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.447+0000 7ff784491b80 1 -- 172.21.15.92:0/1173835635 shutdown_connections 2023-10-06T10:45:34.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.447+0000 7ff784491b80 1 -- 172.21.15.92:0/1173835635 wait complete. 2023-10-06T10:45:34.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.448+0000 7ff784491b80 1 -- start start 2023-10-06T10:45:34.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.448+0000 7ff784491b80 1 Processor -- start 2023-10-06T10:45:34.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.449+0000 7ff784491b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e863c20 0x56318e88da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.449+0000 7ff784491b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56318e8659f0 0x56318e8a6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.449+0000 7ff784491b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56318e8a9ae0 0x56318e88c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.449+0000 7ff784491b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318e7b30b0 con 0x56318e863c20 2023-10-06T10:45:34.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.449+0000 7ff784491b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318e75e8e0 con 0x56318e8a9ae0 2023-10-06T10:45:34.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.449+0000 7ff784491b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56318e779ef0 con 0x56318e8659f0 2023-10-06T10:45:34.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.449+0000 7ff784491b80 10 client.0 _ll_register_callbacks cb 0x56318e64ff80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:34.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.449+0000 7ff778f35700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e863c20 0x56318e88da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.449+0000 7ff778f35700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e863c20 0x56318e88da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55162/0 (socket says 172.21.15.92:55162) 2023-10-06T10:45:34.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.449+0000 7ff778f35700 1 -- 172.21.15.92:0/4077962290 learned_addr learned my addr 172.21.15.92:0/4077962290 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:34.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.449+0000 7ff76bfff700 1 --2- 172.21.15.92:0/4077962290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56318e8659f0 0x56318e8a6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.449+0000 7ff779736700 1 --2- 172.21.15.92:0/4077962290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56318e8a9ae0 0x56318e88c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.449+0000 7ff778f35700 1 -- 172.21.15.92:0/4077962290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56318e8659f0 msgr2=0x56318e8a6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.449+0000 7ff778f35700 1 --2- 172.21.15.92:0/4077962290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56318e8659f0 0x56318e8a6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.449+0000 7ff778f35700 1 -- 172.21.15.92:0/4077962290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56318e8a9ae0 msgr2=0x56318e88c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.449+0000 7ff778f35700 1 --2- 172.21.15.92:0/4077962290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56318e8a9ae0 0x56318e88c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.449+0000 7ff778f35700 1 -- 172.21.15.92:0/4077962290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff76c007340 con 0x56318e863c20 2023-10-06T10:45:34.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.449+0000 7ff779736700 1 --2- 172.21.15.92:0/4077962290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56318e8a9ae0 0x56318e88c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:34.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.450+0000 7ff778f35700 1 --2- 172.21.15.92:0/4077962290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e863c20 0x56318e88da60 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7ff770012000 tx=0x7ff770022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451e95b721753965 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.450+0000 7ff752ffd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:34.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.450+0000 7ff752ffd700 1 -- 172.21.15.92:0/4077962290 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770030070 con 0x56318e863c20 2023-10-06T10:45:34.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.450+0000 7ff752ffd700 1 -- 172.21.15.92:0/4077962290 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7ff77002baa0 con 0x56318e863c20 2023-10-06T10:45:34.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.450+0000 7ff784491b80 1 -- 172.21.15.92:0/4077962290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56318e8576e0 con 0x56318e863c20 2023-10-06T10:45:34.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.450+0000 7ff752ffd700 1 -- 172.21.15.92:0/4077962290 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff770022070 con 0x56318e863c20 2023-10-06T10:45:34.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.450+0000 7ff784491b80 10 client.9189 fetch_fsmap learned FSMap version 110 2023-10-06T10:45:34.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.450+0000 7ff784491b80 1 -- 172.21.15.92:0/4077962290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) v3 -- 0x56318e5f0030 con 0x56318e863c20 2023-10-06T10:45:34.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.450+0000 7ff752ffd700 1 -- 172.21.15.92:0/4077962290 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56318e8576e0 con 0x56318e863c20 2023-10-06T10:45:34.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.450+0000 7ff752ffd700 1 -- 172.21.15.92:0/4077962290 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 110) v1 ==== 3140+0+0 (secure 0 0 0) 0x7ff770011420 con 0x56318e863c20 2023-10-06T10:45:34.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.450+0000 7ff784491b80 10 client.9189 fetch_fsmap finished waiting for FSMap version 110 2023-10-06T10:45:34.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.450+0000 7ff784491b80 10 client.9189 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-10-06T10:45:34.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.450+0000 7ff784491b80 20 client.9189 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:34.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.450+0000 7ff784491b80 1 --2- 172.21.15.92:0/4077962290 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x56318e88cc00 0x56318e88ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.450+0000 7ff784491b80 4 client.9189 mds_command: new command op to 8520 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:34.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.450+0000 7ff784491b80 1 -- 172.21.15.92:0/4077962290 --> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56318e834610 con 0x56318e88cc00 2023-10-06T10:45:34.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.450+0000 7ff76bfff700 1 --2- 172.21.15.92:0/4077962290 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x56318e88cc00 0x56318e88ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.450+0000 7ff76bfff700 1 --2- 172.21.15.92:0/4077962290 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x56318e88cc00 0x56318e88ac60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb18a0f8af80c015 server_cookie=48e11049bc65a319 in_seq=0 out_seq=0 2023-10-06T10:45:34.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.450+0000 7ff752ffd700 10 client.9189 ms_handle_connect on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:34.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.451+0000 7ff752ffd700 1 -- 172.21.15.92:0/4077962290 <== mds.0 v2:172.21.15.92:6835/2298348426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x56318e834610 con 0x56318e88cc00 2023-10-06T10:45:34.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.451+0000 7ff752ffd700 10 client.9189 handle_command_reply: tid=0 2023-10-06T10:45:34.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.451+0000 7ff784491b80 1 client.9189 shutdown 2023-10-06T10:45:34.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.451+0000 7ff784491b80 1 -- 172.21.15.92:0/4077962290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e863c20 msgr2=0x56318e88da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.451+0000 7ff784491b80 1 --2- 172.21.15.92:0/4077962290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e863c20 0x56318e88da60 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7ff770012000 tx=0x7ff770022000 comp rx=0 tx=0).stop 2023-10-06T10:45:34.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.451+0000 7ff784491b80 1 -- 172.21.15.92:0/4077962290 shutdown_connections 2023-10-06T10:45:34.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.451+0000 7ff784491b80 1 --2- 172.21.15.92:0/4077962290 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x56318e88cc00 0x56318e88ac60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.451+0000 7ff784491b80 1 --2- 172.21.15.92:0/4077962290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56318e8a9ae0 0x56318e88c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.451+0000 7ff752ffd700 0 client.9189 ms_handle_reset on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:34.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.451+0000 7ff784491b80 1 --2- 172.21.15.92:0/4077962290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e863c20 0x56318e88da60 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.451+0000 7ff784491b80 1 --2- 172.21.15.92:0/4077962290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56318e8659f0 0x56318e8a6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.451+0000 7ff784491b80 1 -- 172.21.15.92:0/4077962290 >> 172.21.15.92:0/4077962290 conn(0x56318e8640c0 msgr2=0x56318e8a9020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:34.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.451+0000 7ff784491b80 1 -- 172.21.15.92:0/4077962290 shutdown_connections 2023-10-06T10:45:34.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.451+0000 7ff784491b80 1 -- 172.21.15.92:0/4077962290 wait complete. 2023-10-06T10:45:34.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.451+0000 7ff784491b80 20 client.9189 trim_cache size 0 max 16384 2023-10-06T10:45:34.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.467+0000 7ff784491b80 1 Processor -- start 2023-10-06T10:45:34.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.467+0000 7ff784491b80 1 -- start start 2023-10-06T10:45:34.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.468+0000 7ff784491b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e863c20 0x56318e8a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.468+0000 7ff784491b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56318e8659f0 0x56318e8a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.468+0000 7ff784491b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56318e8a9ae0 0x56318e85e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.468+0000 7ff784491b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318e635e80 con 0x56318e863c20 2023-10-06T10:45:34.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.468+0000 7ff784491b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318e85d110 con 0x56318e8a9ae0 2023-10-06T10:45:34.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.468+0000 7ff784491b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56318e85d3a0 con 0x56318e8659f0 2023-10-06T10:45:34.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.468+0000 7ff778f35700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e863c20 0x56318e8a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.468+0000 7ff778f35700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e863c20 0x56318e8a4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55166/0 (socket says 172.21.15.92:55166) 2023-10-06T10:45:34.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.468+0000 7ff778f35700 1 -- 172.21.15.92:0/4273069299 learned_addr learned my addr 172.21.15.92:0/4273069299 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:34.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.468+0000 7ff778f35700 1 -- 172.21.15.92:0/4273069299 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56318e8659f0 msgr2=0x56318e8a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.468+0000 7ff778f35700 1 --2- 172.21.15.92:0/4273069299 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56318e8659f0 0x56318e8a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.468+0000 7ff778f35700 1 -- 172.21.15.92:0/4273069299 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56318e8a9ae0 msgr2=0x56318e85e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.468+0000 7ff778f35700 1 --2- 172.21.15.92:0/4273069299 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56318e8a9ae0 0x56318e85e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.468+0000 7ff778f35700 1 -- 172.21.15.92:0/4273069299 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56318e7817d0 con 0x56318e863c20 2023-10-06T10:45:34.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.469+0000 7ff778f35700 1 --2- 172.21.15.92:0/4273069299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e863c20 0x56318e8a4a40 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7ff770007fd0 tx=0x7ff770030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8dce6d4b9fd2782 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.469+0000 7ff75a7fc700 1 -- 172.21.15.92:0/4273069299 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770022070 con 0x56318e863c20 2023-10-06T10:45:34.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.469+0000 7ff75a7fc700 1 -- 172.21.15.92:0/4273069299 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77002be90 con 0x56318e863c20 2023-10-06T10:45:34.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.469+0000 7ff75a7fc700 1 -- 172.21.15.92:0/4273069299 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770011980 con 0x56318e863c20 2023-10-06T10:45:34.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.469+0000 7ff784491b80 1 -- 172.21.15.92:0/4273069299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e863c20 msgr2=0x56318e8a4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.469+0000 7ff784491b80 1 --2- 172.21.15.92:0/4273069299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e863c20 0x56318e8a4a40 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7ff770007fd0 tx=0x7ff770030000 comp rx=0 tx=0).stop 2023-10-06T10:45:34.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.469+0000 7ff784491b80 1 -- 172.21.15.92:0/4273069299 shutdown_connections 2023-10-06T10:45:34.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.469+0000 7ff784491b80 1 --2- 172.21.15.92:0/4273069299 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56318e8a9ae0 0x56318e85e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.469+0000 7ff784491b80 1 --2- 172.21.15.92:0/4273069299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e863c20 0x56318e8a4a40 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.469+0000 7ff784491b80 1 --2- 172.21.15.92:0/4273069299 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56318e8659f0 0x56318e8a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.469+0000 7ff784491b80 1 -- 172.21.15.92:0/4273069299 >> 172.21.15.92:0/4273069299 conn(0x56318e8640c0 msgr2=0x56318e8bc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:34.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.469+0000 7ff784491b80 1 -- 172.21.15.92:0/4273069299 shutdown_connections 2023-10-06T10:45:34.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.469+0000 7ff784491b80 1 -- 172.21.15.92:0/4273069299 wait complete. 2023-10-06T10:45:34.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.470+0000 7ff784491b80 1 -- start start 2023-10-06T10:45:34.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.470+0000 7ff784491b80 1 Processor -- start 2023-10-06T10:45:34.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff784491b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56318e863c20 0x56318e85e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff784491b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e8659f0 0x56318e90dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff784491b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56318e8a9ae0 0x56318e90e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff784491b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318e85f010 con 0x56318e8659f0 2023-10-06T10:45:34.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff784491b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318e90ea80 con 0x56318e8a9ae0 2023-10-06T10:45:34.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff784491b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56318e85e7f0 con 0x56318e863c20 2023-10-06T10:45:34.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff784491b80 10 client.0 _ll_register_callbacks cb 0x56318e64ff80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:34.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff76bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e8659f0 0x56318e90dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff76bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e8659f0 0x56318e90dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55180/0 (socket says 172.21.15.92:55180) 2023-10-06T10:45:34.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff76bfff700 1 -- 172.21.15.92:0/2455352320 learned_addr learned my addr 172.21.15.92:0/2455352320 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:34.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff779736700 1 --2- 172.21.15.92:0/2455352320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56318e8a9ae0 0x56318e90e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff778f35700 1 --2- 172.21.15.92:0/2455352320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56318e863c20 0x56318e85e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff76bfff700 1 -- 172.21.15.92:0/2455352320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56318e863c20 msgr2=0x56318e85e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff76bfff700 1 --2- 172.21.15.92:0/2455352320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56318e863c20 0x56318e85e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff76bfff700 1 -- 172.21.15.92:0/2455352320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56318e8a9ae0 msgr2=0x56318e90e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff76bfff700 1 --2- 172.21.15.92:0/2455352320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56318e8a9ae0 0x56318e90e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff76bfff700 1 -- 172.21.15.92:0/2455352320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56318e5f0030 con 0x56318e8659f0 2023-10-06T10:45:34.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff779736700 1 --2- 172.21.15.92:0/2455352320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56318e8a9ae0 0x56318e90e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:34.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff76bfff700 1 --2- 172.21.15.92:0/2455352320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e8659f0 0x56318e90dcf0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7ff764081e10 tx=0x7ff764014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3a9fcfc6ba6648 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff752ffd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:34.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff752ffd700 1 -- 172.21.15.92:0/2455352320 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff764052bc0 con 0x56318e8659f0 2023-10-06T10:45:34.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff752ffd700 1 -- 172.21.15.92:0/2455352320 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7ff76406ae40 con 0x56318e8659f0 2023-10-06T10:45:34.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.471+0000 7ff784491b80 1 -- 172.21.15.92:0/2455352320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff76c00f050 con 0x56318e8659f0 2023-10-06T10:45:34.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.472+0000 7ff752ffd700 1 -- 172.21.15.92:0/2455352320 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff764054c90 con 0x56318e8659f0 2023-10-06T10:45:34.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.472+0000 7ff784491b80 10 client.9195 fetch_fsmap learned FSMap version 110 2023-10-06T10:45:34.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.472+0000 7ff784491b80 1 -- 172.21.15.92:0/2455352320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) v3 -- 0x56318e7684c0 con 0x56318e8659f0 2023-10-06T10:45:34.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.472+0000 7ff752ffd700 1 -- 172.21.15.92:0/2455352320 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c00f050 con 0x56318e8659f0 2023-10-06T10:45:34.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.472+0000 7ff752ffd700 1 -- 172.21.15.92:0/2455352320 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 110) v1 ==== 3140+0+0 (secure 0 0 0) 0x7ff764053d10 con 0x56318e8659f0 2023-10-06T10:45:34.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.472+0000 7ff784491b80 10 client.9195 fetch_fsmap finished waiting for FSMap version 110 2023-10-06T10:45:34.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.472+0000 7ff784491b80 10 client.9195 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-10-06T10:45:34.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.472+0000 7ff784491b80 20 client.9195 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:34.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.472+0000 7ff784491b80 1 --2- 172.21.15.92:0/2455352320 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x56318e8a4a40 0x56318e8a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.472+0000 7ff784491b80 4 client.9195 mds_command: new command op to 8520 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:45:34.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.472+0000 7ff784491b80 1 -- 172.21.15.92:0/2455352320 --> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56318e755e50 con 0x56318e8a4a40 2023-10-06T10:45:34.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.472+0000 7ff778f35700 1 --2- 172.21.15.92:0/2455352320 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x56318e8a4a40 0x56318e8a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.472+0000 7ff752ffd700 10 client.9195 ms_handle_connect on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.472+0000 7ff778f35700 1 --2- 172.21.15.92:0/2455352320 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x56318e8a4a40 0x56318e8a4e40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e80d04923d1dc85d server_cookie=38379e21b5ffd8cb in_seq=0 out_seq=0 2023-10-06T10:45:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.472+0000 7ff752ffd700 1 -- 172.21.15.92:0/2455352320 <== mds.0 v2:172.21.15.92:6835/2298348426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x56318e755e50 con 0x56318e8a4a40 2023-10-06T10:45:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.472+0000 7ff752ffd700 10 client.9195 handle_command_reply: tid=0 2023-10-06T10:45:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff784491b80 1 client.9195 shutdown 2023-10-06T10:45:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff784491b80 1 -- 172.21.15.92:0/2455352320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e8659f0 msgr2=0x56318e90dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff784491b80 1 --2- 172.21.15.92:0/2455352320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e8659f0 0x56318e90dcf0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7ff764081e10 tx=0x7ff764014040 comp rx=0 tx=0).stop 2023-10-06T10:45:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff784491b80 1 -- 172.21.15.92:0/2455352320 shutdown_connections 2023-10-06T10:45:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff784491b80 1 --2- 172.21.15.92:0/2455352320 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x56318e8a4a40 0x56318e8a4e40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff784491b80 1 --2- 172.21.15.92:0/2455352320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56318e8a9ae0 0x56318e90e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff752ffd700 0 client.9195 ms_handle_reset on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff784491b80 1 --2- 172.21.15.92:0/2455352320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56318e863c20 0x56318e85e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff784491b80 1 --2- 172.21.15.92:0/2455352320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56318e8659f0 0x56318e90dcf0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff784491b80 1 -- 172.21.15.92:0/2455352320 >> 172.21.15.92:0/2455352320 conn(0x56318e8640c0 msgr2=0x56318e8a9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:34.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff784491b80 1 -- 172.21.15.92:0/2455352320 shutdown_connections 2023-10-06T10:45:34.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff784491b80 1 -- 172.21.15.92:0/2455352320 wait complete. 2023-10-06T10:45:34.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff784491b80 20 client.9195 trim_cache size 0 max 16384 2023-10-06T10:45:34.481 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:34.481 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "PAUSED (0 inodes in the stack)", 2023-10-06T10:45:34.481 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:45:34.481 INFO:teuthology.orchestra.run.smithi092.stdout: "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8": { 2023-10-06T10:45:34.481 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:45:34.482 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8", 2023-10-06T10:45:34.482 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:45:34.482 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:34.482 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:34.482 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:34.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff77b199700 1 -- 172.21.15.92:0/2871963226 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff760066f30 msgr2=0x7ff7600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff77b199700 1 --2- 172.21.15.92:0/2871963226 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff760066f30 0x7ff7600693e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff770009260 tx=0x7ff770016000 comp rx=0 tx=0).stop 2023-10-06T10:45:34.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff77b199700 1 -- 172.21.15.92:0/2871963226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff77415a160 msgr2=0x7ff7741500c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff77b199700 1 --2- 172.21.15.92:0/2871963226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff77415a160 0x7ff7741500c0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7ff76400c510 tx=0x7ff76400c8d0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff77b199700 1 -- 172.21.15.92:0/2871963226 shutdown_connections 2023-10-06T10:45:34.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff77b199700 1 --2- 172.21.15.92:0/2871963226 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff760066f30 0x7ff7600693e0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff77b199700 1 --2- 172.21.15.92:0/2871963226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7740b5a50 0x7ff77415f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff77b199700 1 --2- 172.21.15.92:0/2871963226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7740cd4d0 0x7ff7741568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff77b199700 1 --2- 172.21.15.92:0/2871963226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff77415a160 0x7ff7741500c0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.473+0000 7ff77b199700 1 -- 172.21.15.92:0/2871963226 >> 172.21.15.92:0/2871963226 conn(0x7ff774099700 msgr2=0x7ff774159850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:34.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.474+0000 7ff77b199700 1 -- 172.21.15.92:0/2871963226 shutdown_connections 2023-10-06T10:45:34.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.474+0000 7ff77b199700 1 -- 172.21.15.92:0/2871963226 wait complete. 2023-10-06T10:45:34.486 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:45:34.486 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:45:34.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.599+0000 7f44957a3700 1 Processor -- start 2023-10-06T10:45:34.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.599+0000 7f44957a3700 1 -- start start 2023-10-06T10:45:34.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.600+0000 7f44957a3700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4490069a60 0x7f449006cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.600+0000 7f44957a3700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f449006d120 0x7f44900b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.600+0000 7f44957a3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4490159f60 con 0x7f4490069690 2023-10-06T10:45:34.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.600+0000 7f44957a3700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4490159980 con 0x7f4490069a60 2023-10-06T10:45:34.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.600+0000 7f44957a3700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449006d610 con 0x7f449006d120 2023-10-06T10:45:34.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.600+0000 7f448effd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4490069690 0x7f449015a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45560/0 (socket says 172.21.15.92:45560) 2023-10-06T10:45:34.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.600+0000 7f448effd700 1 -- 172.21.15.92:0/3406680639 learned_addr learned my addr 172.21.15.92:0/3406680639 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:34.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.600+0000 7f448e7fc700 1 --2- 172.21.15.92:0/3406680639 >> v2:172.21.15.124:3300/0 conn(0x7f4490069a60 0x7f449006cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.600+0000 7f448f7fe700 1 --2- 172.21.15.92:0/3406680639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f449006d120 0x7f44900b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.600+0000 7f448dffb700 1 -- 172.21.15.92:0/3406680639 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1038824427 0 0) 0x7f4490159f60 con 0x7f4490069690 2023-10-06T10:45:34.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.600+0000 7f448dffb700 1 -- 172.21.15.92:0/3406680639 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4478003570 con 0x7f4490069690 2023-10-06T10:45:34.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.600+0000 7f448e7fc700 1 -- 172.21.15.92:0/3406680639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f449006d120 msgr2=0x7f44900b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.600+0000 7f448e7fc700 1 --2- 172.21.15.92:0/3406680639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f449006d120 0x7f44900b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.600+0000 7f448e7fc700 1 -- 172.21.15.92:0/3406680639 >> v1:172.21.15.92:6789/0 conn(0x7f4490069690 legacy=0x7f449015a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.600+0000 7f448e7fc700 1 -- 172.21.15.92:0/3406680639 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449006d8f0 con 0x7f4490069a60 2023-10-06T10:45:34.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.601+0000 7f448e7fc700 1 --2- 172.21.15.92:0/3406680639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4490069a60 0x7f449006cbe0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f44840098e0 tx=0x7f44840091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f414f83f492f076 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.601+0000 7f448dffb700 1 -- 172.21.15.92:0/3406680639 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4484011630 con 0x7f4490069a60 2023-10-06T10:45:34.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.601+0000 7f448dffb700 1 -- 172.21.15.92:0/3406680639 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4484011790 con 0x7f4490069a60 2023-10-06T10:45:34.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.601+0000 7f448dffb700 1 -- 172.21.15.92:0/3406680639 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4484011b30 con 0x7f4490069a60 2023-10-06T10:45:34.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.601+0000 7f44957a3700 1 -- 172.21.15.92:0/3406680639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4490069a60 msgr2=0x7f449006cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.601+0000 7f44957a3700 1 --2- 172.21.15.92:0/3406680639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4490069a60 0x7f449006cbe0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f44840098e0 tx=0x7f44840091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.601+0000 7f44957a3700 1 -- 172.21.15.92:0/3406680639 shutdown_connections 2023-10-06T10:45:34.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.601+0000 7f44957a3700 1 --2- 172.21.15.92:0/3406680639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f449006d120 0x7f44900b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.601+0000 7f44957a3700 1 --2- 172.21.15.92:0/3406680639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4490069a60 0x7f449006cbe0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.601+0000 7f44957a3700 1 -- 172.21.15.92:0/3406680639 >> 172.21.15.92:0/3406680639 conn(0x7f449006b500 msgr2=0x7f449006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:34.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.601+0000 7f44957a3700 1 -- 172.21.15.92:0/3406680639 shutdown_connections 2023-10-06T10:45:34.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.601+0000 7f44957a3700 1 -- 172.21.15.92:0/3406680639 wait complete. 2023-10-06T10:45:34.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.603+0000 7f44957a3700 1 Processor -- start 2023-10-06T10:45:34.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.603+0000 7f44957a3700 1 -- start start 2023-10-06T10:45:34.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.603+0000 7f44957a3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4490069690 0x7f449016d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.603+0000 7f44957a3700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4490069a60 0x7f44900ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.603+0000 7f44957a3700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f449006d120 0x7f449016abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.603+0000 7f44957a3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4490169bc0 con 0x7f4490069690 2023-10-06T10:45:34.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.603+0000 7f44957a3700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449016d600 con 0x7f449006d120 2023-10-06T10:45:34.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.603+0000 7f44957a3700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449016d8e0 con 0x7f4490069a60 2023-10-06T10:45:34.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.603+0000 7f448effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4490069690 0x7f449016d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.603+0000 7f448effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4490069690 0x7f449016d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55184/0 (socket says 172.21.15.92:55184) 2023-10-06T10:45:34.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.603+0000 7f448effd700 1 -- 172.21.15.92:0/2724517161 learned_addr learned my addr 172.21.15.92:0/2724517161 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:34.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.603+0000 7f448f7fe700 1 --2- 172.21.15.92:0/2724517161 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f449006d120 0x7f449016abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.603+0000 7f448e7fc700 1 --2- 172.21.15.92:0/2724517161 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4490069a60 0x7f44900ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.604+0000 7f448effd700 1 -- 172.21.15.92:0/2724517161 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4490069a60 msgr2=0x7f44900ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.604+0000 7f448effd700 1 --2- 172.21.15.92:0/2724517161 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4490069a60 0x7f44900ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.604+0000 7f448effd700 1 -- 172.21.15.92:0/2724517161 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f449006d120 msgr2=0x7f449016abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.604+0000 7f448effd700 1 --2- 172.21.15.92:0/2724517161 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f449006d120 0x7f449016abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.604+0000 7f448effd700 1 -- 172.21.15.92:0/2724517161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4484009420 con 0x7f4490069690 2023-10-06T10:45:34.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.604+0000 7f448effd700 1 --2- 172.21.15.92:0/2724517161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4490069690 0x7f449016d020 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f448000cc40 tx=0x7f4480001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.604+0000 7f4476ffd700 1 -- 172.21.15.92:0/2724517161 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448000a8b0 con 0x7f4490069690 2023-10-06T10:45:34.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.604+0000 7f4476ffd700 1 -- 172.21.15.92:0/2724517161 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448000aa30 con 0x7f4490069690 2023-10-06T10:45:34.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.604+0000 7f44957a3700 1 -- 172.21.15.92:0/2724517161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449016dbc0 con 0x7f4490069690 2023-10-06T10:45:34.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.604+0000 7f4476ffd700 1 -- 172.21.15.92:0/2724517161 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448000adf0 con 0x7f4490069690 2023-10-06T10:45:34.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.604+0000 7f44957a3700 1 -- 172.21.15.92:0/2724517161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44900ce600 con 0x7f4490069690 2023-10-06T10:45:34.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.605+0000 7f44957a3700 1 -- 172.21.15.92:0/2724517161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f449005f230 con 0x7f4490069690 2023-10-06T10:45:34.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.605+0000 7f4476ffd700 1 -- 172.21.15.92:0/2724517161 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4480007300 con 0x7f4490069690 2023-10-06T10:45:34.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.606+0000 7f4476ffd700 1 --2- 172.21.15.92:0/2724517161 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f447c066f30 0x7f447c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.606+0000 7f4476ffd700 1 -- 172.21.15.92:0/2724517161 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f4480084b30 con 0x7f4490069690 2023-10-06T10:45:34.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.606+0000 7f448e7fc700 1 --2- 172.21.15.92:0/2724517161 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f447c066f30 0x7f447c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.606+0000 7f448e7fc700 1 --2- 172.21.15.92:0/2724517161 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f447c066f30 0x7f447c0693e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f44840091e0 tx=0x7f4484017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.610+0000 7f4476ffd700 1 -- 172.21.15.92:0/2724517161 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4480051b40 con 0x7f4490069690 2023-10-06T10:45:34.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.833+0000 7f44957a3700 1 -- 172.21.15.92:0/2724517161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44901591d0 con 0x7f4490069690 2023-10-06T10:45:34.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.834+0000 7f4476ffd700 1 -- 172.21.15.92:0/2724517161 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4646 (secure 0 0 0) 0x7f4480055160 con 0x7f4490069690 2023-10-06T10:45:34.835 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:34.835 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":110,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2016052382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2016052382},{"type":"v1","addr":"172.21.15.124:6834","nonce":2016052382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:44:51.106486+0000","modified":"2023-10-06T10:45:34.353086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8520,"mds_1":8517,"mds_2":6719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6719":{"gid":6719,"name":"d","rank":2,"incarnation":104,"state":"up:active","state_seq":13,"addr":"172.21.15.124:6835/2307521923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2307521923},{"type":"v1","addr":"172.21.15.124:6835","nonce":2307521923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8517":{"gid":8517,"name":"a","rank":1,"incarnation":102,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6836/1979826278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979826278},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979826278}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6837/2298348426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2298348426},{"type":"v1","addr":"172.21.15.92:6837","nonce":2298348426}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T10:45:34.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.836+0000 7f44957a3700 1 -- 172.21.15.92:0/2724517161 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f447c066f30 msgr2=0x7f447c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.836+0000 7f44957a3700 1 --2- 172.21.15.92:0/2724517161 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f447c066f30 0x7f447c0693e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f44840091e0 tx=0x7f4484017000 comp rx=0 tx=0).stop 2023-10-06T10:45:34.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.836+0000 7f44957a3700 1 -- 172.21.15.92:0/2724517161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4490069690 msgr2=0x7f449016d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.836+0000 7f44957a3700 1 --2- 172.21.15.92:0/2724517161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4490069690 0x7f449016d020 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f448000cc40 tx=0x7f4480001a70 comp rx=0 tx=0).stop 2023-10-06T10:45:34.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.836+0000 7f44957a3700 1 -- 172.21.15.92:0/2724517161 shutdown_connections 2023-10-06T10:45:34.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.836+0000 7f44957a3700 1 --2- 172.21.15.92:0/2724517161 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f447c066f30 0x7f447c0693e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.836+0000 7f44957a3700 1 --2- 172.21.15.92:0/2724517161 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f449006d120 0x7f449016abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.836+0000 7f44957a3700 1 --2- 172.21.15.92:0/2724517161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4490069690 0x7f449016d020 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.836+0000 7f44957a3700 1 --2- 172.21.15.92:0/2724517161 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4490069a60 0x7f44900ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.836+0000 7f44957a3700 1 -- 172.21.15.92:0/2724517161 >> 172.21.15.92:0/2724517161 conn(0x7f449006b500 msgr2=0x7f44901577b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:34.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.836+0000 7f44957a3700 1 -- 172.21.15.92:0/2724517161 shutdown_connections 2023-10-06T10:45:34.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.836+0000 7f44957a3700 1 -- 172.21.15.92:0/2724517161 wait complete. 2023-10-06T10:45:34.838 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 110 2023-10-06T10:45:34.848 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2023-10-06T10:45:34.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:45:34.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.961+0000 7fbfc8d3b700 1 Processor -- start 2023-10-06T10:45:34.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.961+0000 7fbfc8d3b700 1 -- start start 2023-10-06T10:45:34.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.962+0000 7fbfc8d3b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbfc4159470 0x7fbfc40ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.962+0000 7fbfc8d3b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc40c6220 0x7fbfc40cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.962+0000 7fbfc8d3b700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc40cd5d0 con 0x7fbfc40cd160 2023-10-06T10:45:34.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.962+0000 7fbfc8d3b700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc40cc800 con 0x7fbfc4159470 2023-10-06T10:45:34.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.962+0000 7fbfc8d3b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc40cbc30 con 0x7fbfc40c6220 2023-10-06T10:45:34.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.962+0000 7fbfc3fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc40c6220 0x7fbfc40cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.962+0000 7fbfc37fe700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbfc4159470 0x7fbfc40ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.962+0000 7fbfc2ffd700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fbfc40cd160 0x7fbfc40ce980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:40740/0 (socket says 172.21.15.92:40740) 2023-10-06T10:45:34.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.963+0000 7fbfc37fe700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbfc4159470 0x7fbfc40ccc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.92:60526/0 (socket says 172.21.15.92:60526) 2023-10-06T10:45:34.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.963+0000 7fbfc3fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc40c6220 0x7fbfc40cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:48190/0 (socket says 172.21.15.92:48190) 2023-10-06T10:45:34.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.963+0000 7fbfc37fe700 1 -- 172.21.15.92:0/193804488 learned_addr learned my addr 172.21.15.92:0/193804488 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:34.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.963+0000 7fbfc37fe700 1 -- 172.21.15.92:0/193804488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc40c6220 msgr2=0x7fbfc40cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:34.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.963+0000 7fbfc37fe700 1 --2- 172.21.15.92:0/193804488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc40c6220 0x7fbfc40cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.963+0000 7fbfc37fe700 1 -- 172.21.15.92:0/193804488 >> v1:172.21.15.124:6789/0 conn(0x7fbfc40cd160 legacy=0x7fbfc40ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.963+0000 7fbfc37fe700 1 -- 172.21.15.92:0/193804488 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc40cbed0 con 0x7fbfc4159470 2023-10-06T10:45:34.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.963+0000 7fbfc27fc700 1 -- 172.21.15.92:0/193804488 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4222726351 0 0) 0x7fbfc40cd5d0 con 0x7fbfc40cd160 2023-10-06T10:45:34.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.963+0000 7fbfc37fe700 1 --2- 172.21.15.92:0/193804488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfc4159470 0x7fbfc40ccc20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009990 tx=0x7fbfb4009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d1b0fb7e2de64c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.963+0000 7fbfc27fc700 1 -- 172.21.15.92:0/193804488 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4009e80 con 0x7fbfc4159470 2023-10-06T10:45:34.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.963+0000 7fbfc27fc700 1 -- 172.21.15.92:0/193804488 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb4005c10 con 0x7fbfc4159470 2023-10-06T10:45:34.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.963+0000 7fbfc27fc700 1 -- 172.21.15.92:0/193804488 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb400b650 con 0x7fbfc4159470 2023-10-06T10:45:34.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.964+0000 7fbfc8d3b700 1 -- 172.21.15.92:0/193804488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfc4159470 msgr2=0x7fbfc40ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.964+0000 7fbfc8d3b700 1 --2- 172.21.15.92:0/193804488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfc4159470 0x7fbfc40ccc20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009990 tx=0x7fbfb4009220 comp rx=0 tx=0).stop 2023-10-06T10:45:34.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.964+0000 7fbfc8d3b700 1 -- 172.21.15.92:0/193804488 shutdown_connections 2023-10-06T10:45:34.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.964+0000 7fbfc8d3b700 1 --2- 172.21.15.92:0/193804488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc40c6220 0x7fbfc40cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.964+0000 7fbfc8d3b700 1 --2- 172.21.15.92:0/193804488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfc4159470 0x7fbfc40ccc20 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.964+0000 7fbfc8d3b700 1 -- 172.21.15.92:0/193804488 >> 172.21.15.92:0/193804488 conn(0x7fbfc40995c0 msgr2=0x7fbfc406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:34.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.964+0000 7fbfc8d3b700 1 -- 172.21.15.92:0/193804488 shutdown_connections 2023-10-06T10:45:34.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.964+0000 7fbfc8d3b700 1 -- 172.21.15.92:0/193804488 wait complete. 2023-10-06T10:45:34.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.965+0000 7fbfc8d3b700 1 Processor -- start 2023-10-06T10:45:34.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.965+0000 7fbfc8d3b700 1 -- start start 2023-10-06T10:45:34.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.966+0000 7fbfc8d3b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfc40c6220 0x7fbfc414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.966+0000 7fbfc8d3b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc40cd160 0x7fbfc415d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.966+0000 7fbfc8d3b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfc4159470 0x7fbfc415be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.966+0000 7fbfc8d3b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc415e3b0 con 0x7fbfc4159470 2023-10-06T10:45:34.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.966+0000 7fbfc8d3b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc415e690 con 0x7fbfc40c6220 2023-10-06T10:45:34.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.966+0000 7fbfc8d3b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc4159840 con 0x7fbfc40cd160 2023-10-06T10:45:34.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.966+0000 7fbfc3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfc4159470 0x7fbfc415be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.966+0000 7fbfc3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfc4159470 0x7fbfc415be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55192/0 (socket says 172.21.15.92:55192) 2023-10-06T10:45:34.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.966+0000 7fbfc3fff700 1 -- 172.21.15.92:0/3812243601 learned_addr learned my addr 172.21.15.92:0/3812243601 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:34.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.966+0000 7fbfc2ffd700 1 --2- 172.21.15.92:0/3812243601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc40cd160 0x7fbfc415d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.966+0000 7fbfc37fe700 1 --2- 172.21.15.92:0/3812243601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfc40c6220 0x7fbfc414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.966+0000 7fbfc3fff700 1 -- 172.21.15.92:0/3812243601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc40cd160 msgr2=0x7fbfc415d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.966+0000 7fbfc3fff700 1 --2- 172.21.15.92:0/3812243601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc40cd160 0x7fbfc415d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.966+0000 7fbfc3fff700 1 -- 172.21.15.92:0/3812243601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfc40c6220 msgr2=0x7fbfc414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:34.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.966+0000 7fbfc3fff700 1 --2- 172.21.15.92:0/3812243601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfc40c6220 0x7fbfc414afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:34.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.966+0000 7fbfc3fff700 1 -- 172.21.15.92:0/3812243601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb4009480 con 0x7fbfc4159470 2023-10-06T10:45:34.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.966+0000 7fbfc37fe700 1 --2- 172.21.15.92:0/3812243601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfc40c6220 0x7fbfc414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:34.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.966+0000 7fbfc3fff700 1 --2- 172.21.15.92:0/3812243601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfc4159470 0x7fbfc415be20 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009df0 tx=0x7fbfbc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.966+0000 7fbfab7fe700 1 -- 172.21.15.92:0/3812243601 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc00cb50 con 0x7fbfc4159470 2023-10-06T10:45:34.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.967+0000 7fbfc8d3b700 1 -- 172.21.15.92:0/3812243601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc4159b20 con 0x7fbfc4159470 2023-10-06T10:45:34.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.967+0000 7fbfc8d3b700 1 -- 172.21.15.92:0/3812243601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc41486d0 con 0x7fbfc4159470 2023-10-06T10:45:34.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.967+0000 7fbfab7fe700 1 -- 172.21.15.92:0/3812243601 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc00f040 con 0x7fbfc4159470 2023-10-06T10:45:34.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.967+0000 7fbfab7fe700 1 -- 172.21.15.92:0/3812243601 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc00cec0 con 0x7fbfc4159470 2023-10-06T10:45:34.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.967+0000 7fbfc8d3b700 1 -- 172.21.15.92:0/3812243601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa40052b0 con 0x7fbfc4159470 2023-10-06T10:45:34.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.968+0000 7fbfab7fe700 1 -- 172.21.15.92:0/3812243601 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbfbc014690 con 0x7fbfc4159470 2023-10-06T10:45:34.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.968+0000 7fbfab7fe700 1 --2- 172.21.15.92:0/3812243601 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfb0066f30 0x7fbfb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:34.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.968+0000 7fbfab7fe700 1 -- 172.21.15.92:0/3812243601 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fbfbc085270 con 0x7fbfc4159470 2023-10-06T10:45:34.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.968+0000 7fbfc37fe700 1 --2- 172.21.15.92:0/3812243601 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfb0066f30 0x7fbfb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:34.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.969+0000 7fbfc37fe700 1 --2- 172.21.15.92:0/3812243601 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfb0066f30 0x7fbfb00693e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009370 tx=0x7fbfb4016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:34.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:34.972+0000 7fbfab7fe700 1 -- 172.21.15.92:0/3812243601 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfbc055bf0 con 0x7fbfc4159470 2023-10-06T10:45:35.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.232+0000 7fbfc8d3b700 1 -- 172.21.15.92:0/3812243601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fbfa4005ae0 con 0x7fbfc4159470 2023-10-06T10:45:35.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.232+0000 7fbfab7fe700 1 -- 172.21.15.92:0/3812243601 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1972 (secure 0 0 0) 0x7fbfbc055780 con 0x7fbfc4159470 2023-10-06T10:45:35.233 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:35.233 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":456,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":698974,"bytes_used":2455638016,"bytes_avail":770638475264,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6739,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":110,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6719},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":8517},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8520}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2023-10-06T10:45:32.177411+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2023-10-06T10:45:35.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.234+0000 7fbfc8d3b700 1 -- 172.21.15.92:0/3812243601 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfb0066f30 msgr2=0x7fbfb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.234+0000 7fbfc8d3b700 1 --2- 172.21.15.92:0/3812243601 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfb0066f30 0x7fbfb00693e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009370 tx=0x7fbfb4016000 comp rx=0 tx=0).stop 2023-10-06T10:45:35.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.234+0000 7fbfc8d3b700 1 -- 172.21.15.92:0/3812243601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfc4159470 msgr2=0x7fbfc415be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.234+0000 7fbfc8d3b700 1 --2- 172.21.15.92:0/3812243601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfc4159470 0x7fbfc415be20 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009df0 tx=0x7fbfbc00b040 comp rx=0 tx=0).stop 2023-10-06T10:45:35.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.234+0000 7fbfc8d3b700 1 -- 172.21.15.92:0/3812243601 shutdown_connections 2023-10-06T10:45:35.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.234+0000 7fbfc8d3b700 1 --2- 172.21.15.92:0/3812243601 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfb0066f30 0x7fbfb00693e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.234+0000 7fbfc8d3b700 1 --2- 172.21.15.92:0/3812243601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfc4159470 0x7fbfc415be20 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.234+0000 7fbfc8d3b700 1 --2- 172.21.15.92:0/3812243601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfc40c6220 0x7fbfc414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.234+0000 7fbfc8d3b700 1 --2- 172.21.15.92:0/3812243601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfc40cd160 0x7fbfc415d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.234+0000 7fbfc8d3b700 1 -- 172.21.15.92:0/3812243601 >> 172.21.15.92:0/3812243601 conn(0x7fbfc40995c0 msgr2=0x7fbfc40694e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.234+0000 7fbfc8d3b700 1 -- 172.21.15.92:0/3812243601 shutdown_connections 2023-10-06T10:45:35.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.234+0000 7fbfc8d3b700 1 -- 172.21.15.92:0/3812243601 wait complete. 2023-10-06T10:45:35.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2023-10-06T10:45:35.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub resume 2023-10-06T10:45:35.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.362+0000 7f98a8501700 1 Processor -- start 2023-10-06T10:45:35.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.362+0000 7f98a8501700 1 -- start start 2023-10-06T10:45:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.363+0000 7f98a8501700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98a0150940 0x7f98a0159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.363+0000 7f98a8501700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f98a015a160 0x7f98a00ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.363+0000 7f98a8501700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a015ac30 con 0x7f98a00c5250 2023-10-06T10:45:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.363+0000 7f98a8501700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a00cb6a0 con 0x7f98a015a160 2023-10-06T10:45:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.363+0000 7f98a8501700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00cb090 con 0x7f98a0150940 2023-10-06T10:45:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.363+0000 7f98a629d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98a0150940 0x7f98a0159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.364+0000 7f98a629d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98a0150940 0x7f98a0159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:48208/0 (socket says 172.21.15.92:48208) 2023-10-06T10:45:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.364+0000 7f98a629d700 1 -- 172.21.15.92:0/2454037487 learned_addr learned my addr 172.21.15.92:0/2454037487 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.364+0000 7f98a5a9c700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f98a015a160 0x7f98a00ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.364+0000 7f98a5a9c700 1 -- 172.21.15.92:0/2454037487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98a0150940 msgr2=0x7f98a0159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.364+0000 7f98a5a9c700 1 --2- 172.21.15.92:0/2454037487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98a0150940 0x7f98a0159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.364+0000 7f98a5a9c700 1 -- 172.21.15.92:0/2454037487 >> v1:172.21.15.124:6789/0 conn(0x7f98a00c5250 legacy=0x7f98a00b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.364+0000 7f98a5a9c700 1 -- 172.21.15.92:0/2454037487 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a015a670 con 0x7f98a015a160 2023-10-06T10:45:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.364+0000 7f98a529b700 1 -- 172.21.15.92:0/2454037487 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972584466 0 0) 0x7f9898001490 con 0x7f98a00c5250 2023-10-06T10:45:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.364+0000 7f98a5a9c700 1 --2- 172.21.15.92:0/2454037487 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98a015a160 0x7f98a00ccdc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f98900098e0 tx=0x7f98900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc4aaa9322e21ad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.364+0000 7f98a529b700 1 -- 172.21.15.92:0/2454037487 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989000b640 con 0x7f98a015a160 2023-10-06T10:45:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.364+0000 7f98a529b700 1 -- 172.21.15.92:0/2454037487 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989000b7a0 con 0x7f98a015a160 2023-10-06T10:45:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.364+0000 7f98a529b700 1 -- 172.21.15.92:0/2454037487 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989000bb40 con 0x7f98a015a160 2023-10-06T10:45:35.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.365+0000 7f98a8501700 1 -- 172.21.15.92:0/2454037487 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98a015a160 msgr2=0x7f98a00ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.365+0000 7f98a8501700 1 --2- 172.21.15.92:0/2454037487 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98a015a160 0x7f98a00ccdc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f98900098e0 tx=0x7f98900091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.365+0000 7f98a8501700 1 -- 172.21.15.92:0/2454037487 shutdown_connections 2023-10-06T10:45:35.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.365+0000 7f98a8501700 1 --2- 172.21.15.92:0/2454037487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98a0150940 0x7f98a0159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.365+0000 7f98a8501700 1 --2- 172.21.15.92:0/2454037487 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98a015a160 0x7f98a00ccdc0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.365+0000 7f98a8501700 1 -- 172.21.15.92:0/2454037487 >> 172.21.15.92:0/2454037487 conn(0x7f98a0099700 msgr2=0x7f98a006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.365+0000 7f98a8501700 1 -- 172.21.15.92:0/2454037487 shutdown_connections 2023-10-06T10:45:35.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.365+0000 7f98a8501700 1 -- 172.21.15.92:0/2454037487 wait complete. 2023-10-06T10:45:35.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.366+0000 7f98a8501700 1 Processor -- start 2023-10-06T10:45:35.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.366+0000 7f98a8501700 1 -- start start 2023-10-06T10:45:35.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.367+0000 7f98a8501700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98a00c5250 0x7f98a014ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.367+0000 7f98a8501700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a015a160 0x7f98a00c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.367+0000 7f98a8501700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98a00b4730 0x7f98a0151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.367+0000 7f98a8501700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a0153720 con 0x7f98a015a160 2023-10-06T10:45:35.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.367+0000 7f98a8501700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a014f350 con 0x7f98a00c5250 2023-10-06T10:45:35.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.367+0000 7f98a8501700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a01539a0 con 0x7f98a00b4730 2023-10-06T10:45:35.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.367+0000 7f98a5a9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a015a160 0x7f98a00c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.367+0000 7f98a5a9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a015a160 0x7f98a00c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55194/0 (socket says 172.21.15.92:55194) 2023-10-06T10:45:35.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.367+0000 7f98a5a9c700 1 -- 172.21.15.92:0/1523091688 learned_addr learned my addr 172.21.15.92:0/1523091688 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:35.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.367+0000 7f98a629d700 1 --2- 172.21.15.92:0/1523091688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98a00c5250 0x7f98a014ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.367+0000 7f98a6a9e700 1 --2- 172.21.15.92:0/1523091688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98a00b4730 0x7f98a0151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.367+0000 7f98a5a9c700 1 -- 172.21.15.92:0/1523091688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98a00b4730 msgr2=0x7f98a0151710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.367+0000 7f98a5a9c700 1 --2- 172.21.15.92:0/1523091688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98a00b4730 0x7f98a0151710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.367+0000 7f98a5a9c700 1 -- 172.21.15.92:0/1523091688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98a00c5250 msgr2=0x7f98a014ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.367+0000 7f98a5a9c700 1 --2- 172.21.15.92:0/1523091688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98a00c5250 0x7f98a014ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.367+0000 7f98a5a9c700 1 -- 172.21.15.92:0/1523091688 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9890009420 con 0x7f98a015a160 2023-10-06T10:45:35.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.367+0000 7f98a629d700 1 --2- 172.21.15.92:0/1523091688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98a00c5250 0x7f98a014ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:35.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.367+0000 7f98a6a9e700 1 --2- 172.21.15.92:0/1523091688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98a00b4730 0x7f98a0151710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:35.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.367+0000 7f98a5a9c700 1 --2- 172.21.15.92:0/1523091688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a015a160 0x7f98a00c9e80 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f98900093f0 tx=0x7f9890017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.367+0000 7f98967fc700 1 -- 172.21.15.92:0/1523091688 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989000bb20 con 0x7f98a015a160 2023-10-06T10:45:35.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.368+0000 7f98967fc700 1 -- 172.21.15.92:0/1523091688 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9890011420 con 0x7f98a015a160 2023-10-06T10:45:35.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.368+0000 7f98a8501700 1 -- 172.21.15.92:0/1523091688 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a0153c80 con 0x7f98a015a160 2023-10-06T10:45:35.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.368+0000 7f98967fc700 1 -- 172.21.15.92:0/1523091688 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98900117a0 con 0x7f98a015a160 2023-10-06T10:45:35.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.368+0000 7f98a8501700 1 -- 172.21.15.92:0/1523091688 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a00c8860 con 0x7f98a015a160 2023-10-06T10:45:35.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.369+0000 7f98967fc700 1 -- 172.21.15.92:0/1523091688 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9890011bd0 con 0x7f98a015a160 2023-10-06T10:45:35.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.369+0000 7f98967fc700 1 --2- 172.21.15.92:0/1523091688 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9884066f30 0x7f98840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.369+0000 7f98967fc700 1 -- 172.21.15.92:0/1523091688 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f98900862f0 con 0x7f98a015a160 2023-10-06T10:45:35.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.369+0000 7f98a629d700 1 --2- 172.21.15.92:0/1523091688 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9884066f30 0x7f98840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.370+0000 7f98a629d700 1 --2- 172.21.15.92:0/1523091688 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9884066f30 0x7f98840693e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f989c006f60 tx=0x7f989c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.371+0000 7f98b17f9b80 1 Processor -- start 2023-10-06T10:45:35.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.371+0000 7f98b17f9b80 1 -- start start 2023-10-06T10:45:35.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.372+0000 7f98b17f9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236589ae0 0x55d236589eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.372+0000 7f98b17f9b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d2365459f0 0x55d2365425d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.372+0000 7f98b17f9b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d236543c20 0x55d236542b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.372+0000 7f98b17f9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2364c2cd0 con 0x55d236589ae0 2023-10-06T10:45:35.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.372+0000 7f98b17f9b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d236499ac0 con 0x55d2365459f0 2023-10-06T10:45:35.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.372+0000 7f98b17f9b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2364972c0 con 0x55d236543c20 2023-10-06T10:45:35.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.372+0000 7f98a629d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236589ae0 0x55d236589eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.372+0000 7f98a629d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236589ae0 0x55d236589eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55196/0 (socket says 172.21.15.92:55196) 2023-10-06T10:45:35.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.372+0000 7f98a629d700 1 -- 172.21.15.92:0/2854348584 learned_addr learned my addr 172.21.15.92:0/2854348584 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:35.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.372+0000 7f98a6a9e700 1 --2- 172.21.15.92:0/2854348584 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d236543c20 0x55d236542b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.372+0000 7f98a5a9c700 1 --2- 172.21.15.92:0/2854348584 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d2365459f0 0x55d2365425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.372+0000 7f98a629d700 1 -- 172.21.15.92:0/2854348584 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d236543c20 msgr2=0x55d236542b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.372+0000 7f98a629d700 1 --2- 172.21.15.92:0/2854348584 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d236543c20 0x55d236542b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.372+0000 7f98a629d700 1 -- 172.21.15.92:0/2854348584 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d2365459f0 msgr2=0x55d2365425d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.372+0000 7f98a629d700 1 --2- 172.21.15.92:0/2854348584 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d2365459f0 0x55d2365425d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.372+0000 7f98a629d700 1 -- 172.21.15.92:0/2854348584 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d23632d2c0 con 0x55d236589ae0 2023-10-06T10:45:35.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.373+0000 7f98a629d700 1 --2- 172.21.15.92:0/2854348584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236589ae0 0x55d236589eb0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f989c010b80 tx=0x7f989c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a640a9faa5af09a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.373+0000 7f98837fe700 1 -- 172.21.15.92:0/2854348584 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c015b10 con 0x55d236589ae0 2023-10-06T10:45:35.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.373+0000 7f98837fe700 1 -- 172.21.15.92:0/2854348584 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c01a070 con 0x55d236589ae0 2023-10-06T10:45:35.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.373+0000 7f98837fe700 1 -- 172.21.15.92:0/2854348584 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c015ea0 con 0x55d236589ae0 2023-10-06T10:45:35.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.373+0000 7f98b17f9b80 1 -- 172.21.15.92:0/2854348584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236589ae0 msgr2=0x55d236589eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.373+0000 7f98b17f9b80 1 --2- 172.21.15.92:0/2854348584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236589ae0 0x55d236589eb0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f989c010b80 tx=0x7f989c017000 comp rx=0 tx=0).stop 2023-10-06T10:45:35.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.373+0000 7f98b17f9b80 1 -- 172.21.15.92:0/2854348584 shutdown_connections 2023-10-06T10:45:35.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.373+0000 7f98b17f9b80 1 --2- 172.21.15.92:0/2854348584 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d236543c20 0x55d236542b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.373+0000 7f98b17f9b80 1 --2- 172.21.15.92:0/2854348584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236589ae0 0x55d236589eb0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.373+0000 7f98b17f9b80 1 --2- 172.21.15.92:0/2854348584 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d2365459f0 0x55d2365425d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.373+0000 7f98b17f9b80 1 -- 172.21.15.92:0/2854348584 >> 172.21.15.92:0/2854348584 conn(0x55d2365440c0 msgr2=0x55d236544490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.373+0000 7f98b17f9b80 1 -- 172.21.15.92:0/2854348584 shutdown_connections 2023-10-06T10:45:35.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.373+0000 7f98b17f9b80 1 -- 172.21.15.92:0/2854348584 wait complete. 2023-10-06T10:45:35.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.374+0000 7f98b17f9b80 1 -- start start 2023-10-06T10:45:35.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.374+0000 7f98b17f9b80 1 Processor -- start 2023-10-06T10:45:35.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.375+0000 7f98b17f9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236543c20 0x55d23656abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.375+0000 7f98b17f9b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d2365459f0 0x55d23653d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.375+0000 7f98b17f9b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d236589ae0 0x55d23653dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.375+0000 7f98b17f9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2364930b0 con 0x55d236543c20 2023-10-06T10:45:35.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.375+0000 7f98b17f9b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d23643e8e0 con 0x55d2365459f0 2023-10-06T10:45:35.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.375+0000 7f98b17f9b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d236459ef0 con 0x55d236589ae0 2023-10-06T10:45:35.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.375+0000 7f98b17f9b80 10 client.0 _ll_register_callbacks cb 0x55d23632ff80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:35.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.375+0000 7f98a629d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236543c20 0x55d23656abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.375+0000 7f98a629d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236543c20 0x55d23656abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55206/0 (socket says 172.21.15.92:55206) 2023-10-06T10:45:35.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.375+0000 7f98a629d700 1 -- 172.21.15.92:0/865420885 learned_addr learned my addr 172.21.15.92:0/865420885 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:35.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.375+0000 7f98a6a9e700 1 --2- 172.21.15.92:0/865420885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d236589ae0 0x55d23653dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.375+0000 7f98a5a9c700 1 --2- 172.21.15.92:0/865420885 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d2365459f0 0x55d23653d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.375+0000 7f98a629d700 1 -- 172.21.15.92:0/865420885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d236589ae0 msgr2=0x55d23653dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.375+0000 7f98a629d700 1 --2- 172.21.15.92:0/865420885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d236589ae0 0x55d23653dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.375+0000 7f98a629d700 1 -- 172.21.15.92:0/865420885 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d2365459f0 msgr2=0x55d23653d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.375+0000 7f98a629d700 1 --2- 172.21.15.92:0/865420885 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d2365459f0 0x55d23653d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.375+0000 7f98a629d700 1 -- 172.21.15.92:0/865420885 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f989c00e050 con 0x55d236543c20 2023-10-06T10:45:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.375+0000 7f98a629d700 1 --2- 172.21.15.92:0/865420885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236543c20 0x55d23656abf0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f989c011010 tx=0x7f989c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a59430c4b462b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.376+0000 7f9880ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.376+0000 7f9880ff9700 1 -- 172.21.15.92:0/865420885 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c021d10 con 0x55d236543c20 2023-10-06T10:45:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.376+0000 7f9880ff9700 1 -- 172.21.15.92:0/865420885 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f989c016420 con 0x55d236543c20 2023-10-06T10:45:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.376+0000 7f98b17f9b80 1 -- 172.21.15.92:0/865420885 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2365376e0 con 0x55d236543c20 2023-10-06T10:45:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.376+0000 7f9880ff9700 1 -- 172.21.15.92:0/865420885 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) v2 ==== 24+0+0 (secure 0 0 0) 0x7f989c021e70 con 0x55d236543c20 2023-10-06T10:45:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.376+0000 7f98b17f9b80 10 client.9213 fetch_fsmap learned FSMap version 110 2023-10-06T10:45:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.376+0000 7f98b17f9b80 1 -- 172.21.15.92:0/865420885 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) v3 -- 0x55d2362d0030 con 0x55d236543c20 2023-10-06T10:45:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.376+0000 7f9880ff9700 1 -- 172.21.15.92:0/865420885 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d2365376e0 con 0x55d236543c20 2023-10-06T10:45:35.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.376+0000 7f9880ff9700 1 -- 172.21.15.92:0/865420885 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 110) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f989c016810 con 0x55d236543c20 2023-10-06T10:45:35.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.376+0000 7f98b17f9b80 10 client.9213 fetch_fsmap finished waiting for FSMap version 110 2023-10-06T10:45:35.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.376+0000 7f98b17f9b80 10 client.9213 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-10-06T10:45:35.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.376+0000 7f98b17f9b80 20 client.9213 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:35.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.376+0000 7f98b17f9b80 1 --2- 172.21.15.92:0/865420885 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55d23656b130 0x55d236599c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.376+0000 7f98b17f9b80 4 client.9213 mds_command: new command op to 8520 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:35.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.376+0000 7f98b17f9b80 1 -- 172.21.15.92:0/865420885 --> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d236514610 con 0x55d23656b130 2023-10-06T10:45:35.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.376+0000 7f98a5a9c700 1 --2- 172.21.15.92:0/865420885 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55d23656b130 0x55d236599c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.376+0000 7f9880ff9700 10 client.9213 ms_handle_connect on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:35.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.376+0000 7f98a5a9c700 1 --2- 172.21.15.92:0/865420885 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55d23656b130 0x55d236599c10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=720a010b956eed0b server_cookie=dae3f1acea92d406 in_seq=0 out_seq=0 2023-10-06T10:45:35.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.377+0000 7f9880ff9700 1 -- 172.21.15.92:0/865420885 <== mds.0 v2:172.21.15.92:6835/2298348426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55d236514610 con 0x55d23656b130 2023-10-06T10:45:35.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.377+0000 7f9880ff9700 10 client.9213 handle_command_reply: tid=0 2023-10-06T10:45:35.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.377+0000 7f98b17f9b80 1 client.9213 shutdown 2023-10-06T10:45:35.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.377+0000 7f98b17f9b80 1 -- 172.21.15.92:0/865420885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236543c20 msgr2=0x55d23656abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.377+0000 7f98b17f9b80 1 --2- 172.21.15.92:0/865420885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236543c20 0x55d23656abf0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f989c011010 tx=0x7f989c01a000 comp rx=0 tx=0).stop 2023-10-06T10:45:35.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.377+0000 7f98b17f9b80 1 -- 172.21.15.92:0/865420885 shutdown_connections 2023-10-06T10:45:35.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.377+0000 7f98b17f9b80 1 --2- 172.21.15.92:0/865420885 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55d23656b130 0x55d236599c10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.377+0000 7f98b17f9b80 1 --2- 172.21.15.92:0/865420885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d236589ae0 0x55d23653dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.377+0000 7f98b17f9b80 1 --2- 172.21.15.92:0/865420885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236543c20 0x55d23656abf0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.377+0000 7f9880ff9700 0 client.9213 ms_handle_reset on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:35.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.377+0000 7f98b17f9b80 1 --2- 172.21.15.92:0/865420885 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d2365459f0 0x55d23653d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.377+0000 7f98b17f9b80 1 -- 172.21.15.92:0/865420885 >> 172.21.15.92:0/865420885 conn(0x55d2365440c0 msgr2=0x55d23653eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.377+0000 7f98b17f9b80 1 -- 172.21.15.92:0/865420885 shutdown_connections 2023-10-06T10:45:35.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.377+0000 7f98b17f9b80 1 -- 172.21.15.92:0/865420885 wait complete. 2023-10-06T10:45:35.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.377+0000 7f98b17f9b80 20 client.9213 trim_cache size 0 max 16384 2023-10-06T10:45:35.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.393+0000 7f98b17f9b80 1 Processor -- start 2023-10-06T10:45:35.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.393+0000 7f98b17f9b80 1 -- start start 2023-10-06T10:45:35.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.394+0000 7f98b17f9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236543c20 0x55d2365862c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.394+0000 7f98b17f9b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d2365459f0 0x55d236586b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.394+0000 7f98b17f9b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d236589ae0 0x55d236587320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.394+0000 7f98b17f9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d236315e80 con 0x55d236543c20 2023-10-06T10:45:35.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.394+0000 7f98b17f9b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d23653d3a0 con 0x55d2365459f0 2023-10-06T10:45:35.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.394+0000 7f98b17f9b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d23653d110 con 0x55d236589ae0 2023-10-06T10:45:35.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.394+0000 7f98a629d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236543c20 0x55d2365862c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.394+0000 7f98a629d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236543c20 0x55d2365862c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55222/0 (socket says 172.21.15.92:55222) 2023-10-06T10:45:35.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.394+0000 7f98a629d700 1 -- 172.21.15.92:0/2521726725 learned_addr learned my addr 172.21.15.92:0/2521726725 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:35.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.394+0000 7f98a629d700 1 -- 172.21.15.92:0/2521726725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d236589ae0 msgr2=0x55d236587320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.394+0000 7f98a629d700 1 --2- 172.21.15.92:0/2521726725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d236589ae0 0x55d236587320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.394+0000 7f98a629d700 1 -- 172.21.15.92:0/2521726725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d2365459f0 msgr2=0x55d236586b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.394+0000 7f98a629d700 1 --2- 172.21.15.92:0/2521726725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d2365459f0 0x55d236586b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.394+0000 7f98a629d700 1 -- 172.21.15.92:0/2521726725 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2364617d0 con 0x55d236543c20 2023-10-06T10:45:35.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.394+0000 7f98a629d700 1 --2- 172.21.15.92:0/2521726725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236543c20 0x55d2365862c0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f989c017fd0 tx=0x7f989c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da0339feabc2911 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.394+0000 7f9883fff700 1 -- 172.21.15.92:0/2521726725 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c009a90 con 0x55d236543c20 2023-10-06T10:45:35.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.394+0000 7f9883fff700 1 -- 172.21.15.92:0/2521726725 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c016420 con 0x55d236543c20 2023-10-06T10:45:35.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.394+0000 7f9883fff700 1 -- 172.21.15.92:0/2521726725 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c021900 con 0x55d236543c20 2023-10-06T10:45:35.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.395+0000 7f98b17f9b80 1 -- 172.21.15.92:0/2521726725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236543c20 msgr2=0x55d2365862c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.395+0000 7f98b17f9b80 1 --2- 172.21.15.92:0/2521726725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236543c20 0x55d2365862c0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f989c017fd0 tx=0x7f989c02c000 comp rx=0 tx=0).stop 2023-10-06T10:45:35.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.395+0000 7f98b17f9b80 1 -- 172.21.15.92:0/2521726725 shutdown_connections 2023-10-06T10:45:35.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.395+0000 7f98b17f9b80 1 --2- 172.21.15.92:0/2521726725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d236589ae0 0x55d236587320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.395+0000 7f98b17f9b80 1 --2- 172.21.15.92:0/2521726725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236543c20 0x55d2365862c0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.395+0000 7f98b17f9b80 1 --2- 172.21.15.92:0/2521726725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d2365459f0 0x55d236586b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.395+0000 7f98b17f9b80 1 -- 172.21.15.92:0/2521726725 >> 172.21.15.92:0/2521726725 conn(0x55d2365440c0 msgr2=0x55d236568180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.395+0000 7f98b17f9b80 1 -- 172.21.15.92:0/2521726725 shutdown_connections 2023-10-06T10:45:35.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.395+0000 7f98b17f9b80 1 -- 172.21.15.92:0/2521726725 wait complete. 2023-10-06T10:45:35.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.395+0000 7f98b17f9b80 1 -- start start 2023-10-06T10:45:35.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.395+0000 7f98b17f9b80 1 Processor -- start 2023-10-06T10:45:35.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.396+0000 7f98b17f9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236543c20 0x55d23656a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.396+0000 7f98b17f9b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d2365459f0 0x55d23658a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.396+0000 7f98b17f9b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d236589ae0 0x55d2365ee400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.396+0000 7f98b17f9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d236583440 con 0x55d236543c20 2023-10-06T10:45:35.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.396+0000 7f98b17f9b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d23656a960 con 0x55d2365459f0 2023-10-06T10:45:35.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.396+0000 7f98b17f9b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d23657f650 con 0x55d236589ae0 2023-10-06T10:45:35.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.396+0000 7f98b17f9b80 10 client.0 _ll_register_callbacks cb 0x55d23632ff80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:35.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.396+0000 7f98a629d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236543c20 0x55d23656a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.396+0000 7f98a629d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236543c20 0x55d23656a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55236/0 (socket says 172.21.15.92:55236) 2023-10-06T10:45:35.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.396+0000 7f98a629d700 1 -- 172.21.15.92:0/2270569474 learned_addr learned my addr 172.21.15.92:0/2270569474 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:35.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f98a5a9c700 1 --2- 172.21.15.92:0/2270569474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d2365459f0 0x55d23658a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f98a629d700 1 -- 172.21.15.92:0/2270569474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d236589ae0 msgr2=0x55d2365ee400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f98a629d700 1 --2- 172.21.15.92:0/2270569474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d236589ae0 0x55d2365ee400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f98a629d700 1 -- 172.21.15.92:0/2270569474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d2365459f0 msgr2=0x55d23658a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f98a629d700 1 --2- 172.21.15.92:0/2270569474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d2365459f0 0x55d23658a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f98a629d700 1 -- 172.21.15.92:0/2270569474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2362d0030 con 0x55d236543c20 2023-10-06T10:45:35.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f98a629d700 1 --2- 172.21.15.92:0/2270569474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236543c20 0x55d23656a3f0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f989c011010 tx=0x7f989c03a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9339c537208ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f9880ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:35.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f9880ff9700 1 -- 172.21.15.92:0/2270569474 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c005a00 con 0x55d236543c20 2023-10-06T10:45:35.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f9880ff9700 1 -- 172.21.15.92:0/2270569474 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f989c047030 con 0x55d236543c20 2023-10-06T10:45:35.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f98b17f9b80 1 -- 172.21.15.92:0/2270569474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f989c01a070 con 0x55d236543c20 2023-10-06T10:45:35.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f9880ff9700 1 -- 172.21.15.92:0/2270569474 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) v2 ==== 24+0+0 (secure 0 0 0) 0x7f989c005b60 con 0x55d236543c20 2023-10-06T10:45:35.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f98b17f9b80 10 client.9219 fetch_fsmap learned FSMap version 110 2023-10-06T10:45:35.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f98b17f9b80 1 -- 172.21.15.92:0/2270569474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) v3 -- 0x55d2364484c0 con 0x55d236543c20 2023-10-06T10:45:35.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f9880ff9700 1 -- 172.21.15.92:0/2270569474 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c04f020 con 0x55d236543c20 2023-10-06T10:45:35.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f9880ff9700 1 -- 172.21.15.92:0/2270569474 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 110) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f989c015470 con 0x55d236543c20 2023-10-06T10:45:35.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f98b17f9b80 10 client.9219 fetch_fsmap finished waiting for FSMap version 110 2023-10-06T10:45:35.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f98b17f9b80 10 client.9219 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-10-06T10:45:35.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f98b17f9b80 20 client.9219 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:35.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f98b17f9b80 1 --2- 172.21.15.92:0/2270569474 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55d236582c00 0x55d23657fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f98b17f9b80 4 client.9219 mds_command: new command op to 8520 tid=0[{"prefix": "scrub resume"}] 2023-10-06T10:45:35.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f98b17f9b80 1 -- 172.21.15.92:0/2270569474 --> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x55d236435e50 con 0x55d236582c00 2023-10-06T10:45:35.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.397+0000 7f98a5a9c700 1 --2- 172.21.15.92:0/2270569474 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55d236582c00 0x55d23657fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.398+0000 7f98a5a9c700 1 --2- 172.21.15.92:0/2270569474 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55d236582c00 0x55d23657fae0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa5d71edb0538bbf server_cookie=cf2adaab504c5d11 in_seq=0 out_seq=0 2023-10-06T10:45:35.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.398+0000 7f9880ff9700 10 client.9219 ms_handle_connect on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:35.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.398+0000 7f9880ff9700 1 -- 172.21.15.92:0/2270569474 <== mds.0 v2:172.21.15.92:6835/2298348426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55d236435e50 con 0x55d236582c00 2023-10-06T10:45:35.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.398+0000 7f9880ff9700 10 client.9219 handle_command_reply: tid=0 2023-10-06T10:45:35.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.398+0000 7f98b17f9b80 1 client.9219 shutdown 2023-10-06T10:45:35.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.398+0000 7f98b17f9b80 1 -- 172.21.15.92:0/2270569474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236543c20 msgr2=0x55d23656a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.398+0000 7f98b17f9b80 1 --2- 172.21.15.92:0/2270569474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236543c20 0x55d23656a3f0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f989c011010 tx=0x7f989c03a000 comp rx=0 tx=0).stop 2023-10-06T10:45:35.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.398+0000 7f98b17f9b80 1 -- 172.21.15.92:0/2270569474 shutdown_connections 2023-10-06T10:45:35.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.398+0000 7f98b17f9b80 1 --2- 172.21.15.92:0/2270569474 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x55d236582c00 0x55d23657fae0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.398+0000 7f98b17f9b80 1 --2- 172.21.15.92:0/2270569474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55d236589ae0 0x55d2365ee400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.398+0000 7f98b17f9b80 1 --2- 172.21.15.92:0/2270569474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d236543c20 0x55d23656a3f0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.398+0000 7f9880ff9700 0 client.9219 ms_handle_reset on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:35.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.398+0000 7f98b17f9b80 1 --2- 172.21.15.92:0/2270569474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55d2365459f0 0x55d23658a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.398+0000 7f98b17f9b80 1 -- 172.21.15.92:0/2270569474 >> 172.21.15.92:0/2270569474 conn(0x55d2365440c0 msgr2=0x55d236591e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.398+0000 7f98b17f9b80 1 -- 172.21.15.92:0/2270569474 shutdown_connections 2023-10-06T10:45:35.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.398+0000 7f98b17f9b80 1 -- 172.21.15.92:0/2270569474 wait complete. 2023-10-06T10:45:35.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.398+0000 7f98b17f9b80 20 client.9219 trim_cache size 0 max 16384 2023-10-06T10:45:35.406 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:35.407 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0 2023-10-06T10:45:35.407 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:35.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.399+0000 7f98a8501700 1 -- 172.21.15.92:0/1523091688 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9884066f30 msgr2=0x7f98840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.399+0000 7f98a8501700 1 --2- 172.21.15.92:0/1523091688 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9884066f30 0x7f98840693e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f989c006f60 tx=0x7f989c008040 comp rx=0 tx=0).stop 2023-10-06T10:45:35.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.399+0000 7f98a8501700 1 -- 172.21.15.92:0/1523091688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a015a160 msgr2=0x7f98a00c9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.399+0000 7f98a8501700 1 --2- 172.21.15.92:0/1523091688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a015a160 0x7f98a00c9e80 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f98900093f0 tx=0x7f9890017000 comp rx=0 tx=0).stop 2023-10-06T10:45:35.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.399+0000 7f98a8501700 1 -- 172.21.15.92:0/1523091688 shutdown_connections 2023-10-06T10:45:35.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.399+0000 7f98a8501700 1 --2- 172.21.15.92:0/1523091688 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9884066f30 0x7f98840693e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.399+0000 7f98a8501700 1 --2- 172.21.15.92:0/1523091688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98a00b4730 0x7f98a0151710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.399+0000 7f98a8501700 1 --2- 172.21.15.92:0/1523091688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98a00c5250 0x7f98a014ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.399+0000 7f98a8501700 1 --2- 172.21.15.92:0/1523091688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a015a160 0x7f98a00c9e80 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.399+0000 7f98a8501700 1 -- 172.21.15.92:0/1523091688 >> 172.21.15.92:0/1523091688 conn(0x7f98a0099700 msgr2=0x7f98a00cebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.399+0000 7f98a8501700 1 -- 172.21.15.92:0/1523091688 shutdown_connections 2023-10-06T10:45:35.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.399+0000 7f98a8501700 1 -- 172.21.15.92:0/1523091688 wait complete. 2023-10-06T10:45:35.411 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:1 scrub status 2023-10-06T10:45:35.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.527+0000 7f10672f6700 1 Processor -- start 2023-10-06T10:45:35.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.527+0000 7f10672f6700 1 -- start start 2023-10-06T10:45:35.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.528+0000 7f10672f6700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10600cd4d0 0x7f10600b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.528+0000 7f10672f6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f106015ac30 con 0x7f10600c5250 2023-10-06T10:45:35.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.528+0000 7f10672f6700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10600cb6a0 con 0x7f1060150940 2023-10-06T10:45:35.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.528+0000 7f10672f6700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10600cb090 con 0x7f10600cd4d0 2023-10-06T10:45:35.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.528+0000 7f1064891700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f10600c5250 0x7f10600ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45570/0 (socket says 172.21.15.92:45570) 2023-10-06T10:45:35.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.528+0000 7f1064891700 1 -- 172.21.15.92:0/2453426952 learned_addr learned my addr 172.21.15.92:0/2453426952 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:35.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.528+0000 7f1065893700 1 --2- 172.21.15.92:0/2453426952 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10600cd4d0 0x7f10600b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.528+0000 7f1057fff700 1 -- 172.21.15.92:0/2453426952 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362721735 0 0) 0x7f1050001110 con 0x7f10600c5250 2023-10-06T10:45:35.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.528+0000 7f1057fff700 1 -- 172.21.15.92:0/2453426952 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f104c003570 con 0x7f10600c5250 2023-10-06T10:45:35.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.528+0000 7f1057fff700 1 -- 172.21.15.92:0/2453426952 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2840793156 0 0) 0x7f10600cb6a0 con 0x7f1060150940 2023-10-06T10:45:35.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.528+0000 7f1057fff700 1 -- 172.21.15.92:0/2453426952 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1050001110 con 0x7f1060150940 2023-10-06T10:45:35.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.529+0000 7f1065893700 1 -- 172.21.15.92:0/2453426952 >> v1:172.21.15.124:6789/0 conn(0x7f1060150940 legacy=0x7f1060159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.529+0000 7f1065893700 1 -- 172.21.15.92:0/2453426952 >> v1:172.21.15.92:6789/0 conn(0x7f10600c5250 legacy=0x7f10600ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.529+0000 7f1065893700 1 -- 172.21.15.92:0/2453426952 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106015a670 con 0x7f10600cd4d0 2023-10-06T10:45:35.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.529+0000 7f1065893700 1 --2- 172.21.15.92:0/2453426952 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10600cd4d0 0x7f10600b2050 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1058009e90 tx=0x7f1058009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5082228cdd4d7cf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.529+0000 7f1057fff700 1 -- 172.21.15.92:0/2453426952 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105800e070 con 0x7f10600cd4d0 2023-10-06T10:45:35.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.529+0000 7f1057fff700 1 -- 172.21.15.92:0/2453426952 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105800bb80 con 0x7f10600cd4d0 2023-10-06T10:45:35.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.529+0000 7f1057fff700 1 -- 172.21.15.92:0/2453426952 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058012430 con 0x7f10600cd4d0 2023-10-06T10:45:35.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.530+0000 7f10672f6700 1 -- 172.21.15.92:0/2453426952 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10600cd4d0 msgr2=0x7f10600b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.530+0000 7f10672f6700 1 --2- 172.21.15.92:0/2453426952 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10600cd4d0 0x7f10600b2050 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1058009e90 tx=0x7f1058009780 comp rx=0 tx=0).stop 2023-10-06T10:45:35.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.530+0000 7f10672f6700 1 -- 172.21.15.92:0/2453426952 shutdown_connections 2023-10-06T10:45:35.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.530+0000 7f10672f6700 1 --2- 172.21.15.92:0/2453426952 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10600cd4d0 0x7f10600b2050 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.530+0000 7f10672f6700 1 -- 172.21.15.92:0/2453426952 >> 172.21.15.92:0/2453426952 conn(0x7f1060099700 msgr2=0x7f106006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.530+0000 7f10672f6700 1 -- 172.21.15.92:0/2453426952 shutdown_connections 2023-10-06T10:45:35.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.530+0000 7f10672f6700 1 -- 172.21.15.92:0/2453426952 wait complete. 2023-10-06T10:45:35.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.531+0000 7f10672f6700 1 Processor -- start 2023-10-06T10:45:35.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.531+0000 7f10672f6700 1 -- start start 2023-10-06T10:45:35.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.531+0000 7f10672f6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10600c5250 0x7f10601543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.532+0000 7f10672f6700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10600cd4d0 0x7f10600b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.532+0000 7f10672f6700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f106014de80 0x7f10600cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.532+0000 7f10672f6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10600b3840 con 0x7f10600c5250 2023-10-06T10:45:35.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.532+0000 7f10672f6700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1060156760 con 0x7f106014de80 2023-10-06T10:45:35.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.532+0000 7f10672f6700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1060156a00 con 0x7f10600cd4d0 2023-10-06T10:45:35.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.532+0000 7f1065092700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10600c5250 0x7f10601543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.532+0000 7f1065092700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10600c5250 0x7f10601543b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55238/0 (socket says 172.21.15.92:55238) 2023-10-06T10:45:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.532+0000 7f1065092700 1 -- 172.21.15.92:0/2703039957 learned_addr learned my addr 172.21.15.92:0/2703039957 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.532+0000 7f1064891700 1 --2- 172.21.15.92:0/2703039957 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10600cd4d0 0x7f10600b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.532+0000 7f1065893700 1 --2- 172.21.15.92:0/2703039957 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f106014de80 0x7f10600cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.532+0000 7f1065092700 1 -- 172.21.15.92:0/2703039957 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10600cd4d0 msgr2=0x7f10600b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.532+0000 7f1065092700 1 --2- 172.21.15.92:0/2703039957 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10600cd4d0 0x7f10600b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.532+0000 7f1065092700 1 -- 172.21.15.92:0/2703039957 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f106014de80 msgr2=0x7f10600cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.532+0000 7f1065092700 1 --2- 172.21.15.92:0/2703039957 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f106014de80 0x7f10600cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.532+0000 7f1065092700 1 -- 172.21.15.92:0/2703039957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1058009a20 con 0x7f10600c5250 2023-10-06T10:45:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.532+0000 7f1065092700 1 --2- 172.21.15.92:0/2703039957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10600c5250 0x7f10601543b0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f105c00c560 tx=0x7f105c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.532+0000 7f10557fa700 1 -- 172.21.15.92:0/2703039957 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c010510 con 0x7f10600c5250 2023-10-06T10:45:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.532+0000 7f10557fa700 1 -- 172.21.15.92:0/2703039957 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105c010670 con 0x7f10600c5250 2023-10-06T10:45:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.532+0000 7f10672f6700 1 -- 172.21.15.92:0/2703039957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1060156ce0 con 0x7f10600c5250 2023-10-06T10:45:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.532+0000 7f10557fa700 1 -- 172.21.15.92:0/2703039957 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c010a30 con 0x7f10600c5250 2023-10-06T10:45:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.532+0000 7f10672f6700 1 -- 172.21.15.92:0/2703039957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10601572a0 con 0x7f10600c5250 2023-10-06T10:45:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.534+0000 7f10557fa700 1 -- 172.21.15.92:0/2703039957 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f105c002050 con 0x7f10600c5250 2023-10-06T10:45:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.534+0000 7f10557fa700 1 --2- 172.21.15.92:0/2703039957 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1044066f30 0x7f10440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.534+0000 7f10557fa700 1 -- 172.21.15.92:0/2703039957 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f105c0842a0 con 0x7f10600c5250 2023-10-06T10:45:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.534+0000 7f1064891700 1 --2- 172.21.15.92:0/2703039957 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1044066f30 0x7f10440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.534+0000 7f1064891700 1 --2- 172.21.15.92:0/2703039957 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1044066f30 0x7f10440693e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f10600b23f0 tx=0x7f1050009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.536+0000 7f10705eeb80 1 Processor -- start 2023-10-06T10:45:35.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.536+0000 7f10705eeb80 1 -- start start 2023-10-06T10:45:35.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.537+0000 7f10705eeb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642e9ef7ae0 0x5642e9ef7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.537+0000 7f10705eeb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5642e9eb39f0 0x5642e9eb05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.537+0000 7f10705eeb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9eb1c20 0x5642e9eb0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.537+0000 7f10705eeb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642e9e30cd0 con 0x5642e9eb1c20 2023-10-06T10:45:35.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.537+0000 7f10705eeb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642e9e07ac0 con 0x5642e9ef7ae0 2023-10-06T10:45:35.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.537+0000 7f10705eeb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642e9e052c0 con 0x5642e9eb39f0 2023-10-06T10:45:35.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.537+0000 7f1065893700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9eb1c20 0x5642e9eb0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.537+0000 7f1065893700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9eb1c20 0x5642e9eb0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55250/0 (socket says 172.21.15.92:55250) 2023-10-06T10:45:35.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.537+0000 7f1065893700 1 -- 172.21.15.92:0/3472114264 learned_addr learned my addr 172.21.15.92:0/3472114264 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:35.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.537+0000 7f1065092700 1 --2- 172.21.15.92:0/3472114264 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642e9ef7ae0 0x5642e9ef7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.537+0000 7f1064891700 1 --2- 172.21.15.92:0/3472114264 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5642e9eb39f0 0x5642e9eb05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.537+0000 7f1065893700 1 -- 172.21.15.92:0/3472114264 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5642e9eb39f0 msgr2=0x5642e9eb05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.537+0000 7f1065893700 1 --2- 172.21.15.92:0/3472114264 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5642e9eb39f0 0x5642e9eb05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.537+0000 7f1065893700 1 -- 172.21.15.92:0/3472114264 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642e9ef7ae0 msgr2=0x5642e9ef7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.537+0000 7f1065893700 1 --2- 172.21.15.92:0/3472114264 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642e9ef7ae0 0x5642e9ef7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.537+0000 7f1065893700 1 -- 172.21.15.92:0/3472114264 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642e9c9b2c0 con 0x5642e9eb1c20 2023-10-06T10:45:35.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.537+0000 7f1065893700 1 --2- 172.21.15.92:0/3472114264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9eb1c20 0x5642e9eb0b10 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f10600d14c0 tx=0x7f1058019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc138cf4decbf286 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.537+0000 7f104a7fc700 1 -- 172.21.15.92:0/3472114264 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058021070 con 0x5642e9eb1c20 2023-10-06T10:45:35.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.537+0000 7f104a7fc700 1 -- 172.21.15.92:0/3472114264 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105800bcf0 con 0x5642e9eb1c20 2023-10-06T10:45:35.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.537+0000 7f104a7fc700 1 -- 172.21.15.92:0/3472114264 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058012a40 con 0x5642e9eb1c20 2023-10-06T10:45:35.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.538+0000 7f10705eeb80 1 -- 172.21.15.92:0/3472114264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9eb1c20 msgr2=0x5642e9eb0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.538+0000 7f10705eeb80 1 --2- 172.21.15.92:0/3472114264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9eb1c20 0x5642e9eb0b10 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f10600d14c0 tx=0x7f1058019000 comp rx=0 tx=0).stop 2023-10-06T10:45:35.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.538+0000 7f10705eeb80 1 -- 172.21.15.92:0/3472114264 shutdown_connections 2023-10-06T10:45:35.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.538+0000 7f10705eeb80 1 --2- 172.21.15.92:0/3472114264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9eb1c20 0x5642e9eb0b10 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.538+0000 7f10705eeb80 1 --2- 172.21.15.92:0/3472114264 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642e9ef7ae0 0x5642e9ef7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.538+0000 7f10705eeb80 1 --2- 172.21.15.92:0/3472114264 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5642e9eb39f0 0x5642e9eb05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.538+0000 7f10705eeb80 1 -- 172.21.15.92:0/3472114264 >> 172.21.15.92:0/3472114264 conn(0x5642e9eb20c0 msgr2=0x5642e9eb2490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.538+0000 7f10705eeb80 1 -- 172.21.15.92:0/3472114264 shutdown_connections 2023-10-06T10:45:35.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.538+0000 7f10705eeb80 1 -- 172.21.15.92:0/3472114264 wait complete. 2023-10-06T10:45:35.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.539+0000 7f10705eeb80 1 -- start start 2023-10-06T10:45:35.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.539+0000 7f10705eeb80 1 Processor -- start 2023-10-06T10:45:35.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.539+0000 7f10705eeb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642e9eb1c20 0x5642e9ed8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f10705eeb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5642e9eb39f0 0x5642e9edc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f10705eeb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9ef7ae0 0x5642e9eda6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f10705eeb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642e9e010b0 con 0x5642e9ef7ae0 2023-10-06T10:45:35.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f10705eeb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642e9dac8e0 con 0x5642e9eb1c20 2023-10-06T10:45:35.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f10705eeb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642e9dc7ef0 con 0x5642e9eb39f0 2023-10-06T10:45:35.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f10705eeb80 10 client.0 _ll_register_callbacks cb 0x5642e9c9df80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:35.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f1065893700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9ef7ae0 0x5642e9eda6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f1065893700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9ef7ae0 0x5642e9eda6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55260/0 (socket says 172.21.15.92:55260) 2023-10-06T10:45:35.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f1065893700 1 -- 172.21.15.92:0/1903511038 learned_addr learned my addr 172.21.15.92:0/1903511038 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:35.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f1065092700 1 --2- 172.21.15.92:0/1903511038 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642e9eb1c20 0x5642e9ed8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f1064891700 1 --2- 172.21.15.92:0/1903511038 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5642e9eb39f0 0x5642e9edc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f1065893700 1 -- 172.21.15.92:0/1903511038 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5642e9eb39f0 msgr2=0x5642e9edc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f1065893700 1 --2- 172.21.15.92:0/1903511038 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5642e9eb39f0 0x5642e9edc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f1065893700 1 -- 172.21.15.92:0/1903511038 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642e9eb1c20 msgr2=0x5642e9ed8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f1065893700 1 --2- 172.21.15.92:0/1903511038 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642e9eb1c20 0x5642e9ed8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f1065893700 1 -- 172.21.15.92:0/1903511038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106015a670 con 0x5642e9ef7ae0 2023-10-06T10:45:35.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f1065893700 1 --2- 172.21.15.92:0/1903511038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9ef7ae0 0x5642e9eda6c0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f105800b610 tx=0x7f1058021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643b50e75398d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f103f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:35.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f103f7fe700 1 -- 172.21.15.92:0/1903511038 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058025640 con 0x5642e9ef7ae0 2023-10-06T10:45:35.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f103f7fe700 1 -- 172.21.15.92:0/1903511038 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f10580257a0 con 0x5642e9ef7ae0 2023-10-06T10:45:35.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f10705eeb80 1 -- 172.21.15.92:0/1903511038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5642e9ea56e0 con 0x5642e9ef7ae0 2023-10-06T10:45:35.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f103f7fe700 1 -- 172.21.15.92:0/1903511038 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1058025b70 con 0x5642e9ef7ae0 2023-10-06T10:45:35.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f10705eeb80 10 client.9231 fetch_fsmap learned FSMap version 110 2023-10-06T10:45:35.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.540+0000 7f10705eeb80 1 -- 172.21.15.92:0/1903511038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) v3 -- 0x5642e9c3e030 con 0x5642e9ef7ae0 2023-10-06T10:45:35.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.541+0000 7f103f7fe700 1 -- 172.21.15.92:0/1903511038 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5642e9ea56e0 con 0x5642e9ef7ae0 2023-10-06T10:45:35.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.541+0000 7f103f7fe700 1 -- 172.21.15.92:0/1903511038 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 110) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f1058025cd0 con 0x5642e9ef7ae0 2023-10-06T10:45:35.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.541+0000 7f10705eeb80 10 client.9231 fetch_fsmap finished waiting for FSMap version 110 2023-10-06T10:45:35.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.541+0000 7f10705eeb80 10 client.9231 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2023-10-06T10:45:35.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.541+0000 7f10705eeb80 20 client.9231 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:35.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.541+0000 7f10705eeb80 1 --2- 172.21.15.92:0/1903511038 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5642e9ed7320 0x5642e9edac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.541+0000 7f10705eeb80 4 client.9231 mds_command: new command op to 8517 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:35.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.541+0000 7f10705eeb80 1 -- 172.21.15.92:0/1903511038 --> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642e9e82610 con 0x5642e9ed7320 2023-10-06T10:45:35.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.541+0000 7f1065092700 1 --2- 172.21.15.92:0/1903511038 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5642e9ed7320 0x5642e9edac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.541+0000 7f103f7fe700 10 client.9231 ms_handle_connect on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:35.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.541+0000 7f1065092700 1 --2- 172.21.15.92:0/1903511038 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5642e9ed7320 0x5642e9edac00 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=67bfac2892822fdf server_cookie=16eb92242be14537 in_seq=0 out_seq=0 2023-10-06T10:45:35.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.542+0000 7f103f7fe700 1 -- 172.21.15.92:0/1903511038 <== mds.1 v2:172.21.15.92:6834/1979826278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5642e9e82610 con 0x5642e9ed7320 2023-10-06T10:45:35.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.542+0000 7f103f7fe700 10 client.9231 handle_command_reply: tid=0 2023-10-06T10:45:35.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.542+0000 7f10705eeb80 1 client.9231 shutdown 2023-10-06T10:45:35.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.542+0000 7f10705eeb80 1 -- 172.21.15.92:0/1903511038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9ef7ae0 msgr2=0x5642e9eda6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.542+0000 7f10705eeb80 1 --2- 172.21.15.92:0/1903511038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9ef7ae0 0x5642e9eda6c0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f105800b610 tx=0x7f1058021000 comp rx=0 tx=0).stop 2023-10-06T10:45:35.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.542+0000 7f10705eeb80 1 -- 172.21.15.92:0/1903511038 shutdown_connections 2023-10-06T10:45:35.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.542+0000 7f10705eeb80 1 --2- 172.21.15.92:0/1903511038 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5642e9ed7320 0x5642e9edac00 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.542+0000 7f10705eeb80 1 --2- 172.21.15.92:0/1903511038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9ef7ae0 0x5642e9eda6c0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.542+0000 7f10705eeb80 1 --2- 172.21.15.92:0/1903511038 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642e9eb1c20 0x5642e9ed8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.542+0000 7f10705eeb80 1 --2- 172.21.15.92:0/1903511038 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5642e9eb39f0 0x5642e9edc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.542+0000 7f103f7fe700 0 client.9231 ms_handle_reset on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:35.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.542+0000 7f10705eeb80 1 -- 172.21.15.92:0/1903511038 >> 172.21.15.92:0/1903511038 conn(0x5642e9eb20c0 msgr2=0x5642e9ef6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.542+0000 7f10705eeb80 1 -- 172.21.15.92:0/1903511038 shutdown_connections 2023-10-06T10:45:35.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.542+0000 7f10705eeb80 1 -- 172.21.15.92:0/1903511038 wait complete. 2023-10-06T10:45:35.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.542+0000 7f10705eeb80 20 client.9231 trim_cache size 0 max 16384 2023-10-06T10:45:35.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.558+0000 7f10705eeb80 1 Processor -- start 2023-10-06T10:45:35.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.558+0000 7f10705eeb80 1 -- start start 2023-10-06T10:45:35.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.558+0000 7f10705eeb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5642e9eb1c20 0x5642e9ef6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.559+0000 7f10705eeb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9eb39f0 0x5642e9edb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.559+0000 7f10705eeb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642e9ef7ae0 0x5642e9ead930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.559+0000 7f10705eeb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642e9c83e80 con 0x5642e9eb39f0 2023-10-06T10:45:35.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.559+0000 7f10705eeb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642e9f19550 con 0x5642e9ef7ae0 2023-10-06T10:45:35.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.559+0000 7f10705eeb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642e9ef3390 con 0x5642e9eb1c20 2023-10-06T10:45:35.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.559+0000 7f1064891700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9eb39f0 0x5642e9edb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.559+0000 7f1064891700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9eb39f0 0x5642e9edb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55272/0 (socket says 172.21.15.92:55272) 2023-10-06T10:45:35.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.559+0000 7f1064891700 1 -- 172.21.15.92:0/4086610214 learned_addr learned my addr 172.21.15.92:0/4086610214 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:35.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.559+0000 7f1065092700 1 --2- 172.21.15.92:0/4086610214 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5642e9eb1c20 0x5642e9ef6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.559+0000 7f1065893700 1 --2- 172.21.15.92:0/4086610214 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642e9ef7ae0 0x5642e9ead930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.559+0000 7f1064891700 1 -- 172.21.15.92:0/4086610214 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5642e9eb1c20 msgr2=0x5642e9ef6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.559+0000 7f1064891700 1 --2- 172.21.15.92:0/4086610214 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5642e9eb1c20 0x5642e9ef6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.559+0000 7f1064891700 1 -- 172.21.15.92:0/4086610214 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642e9ef7ae0 msgr2=0x5642e9ead930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.559+0000 7f1064891700 1 --2- 172.21.15.92:0/4086610214 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642e9ef7ae0 0x5642e9ead930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.559+0000 7f1064891700 1 -- 172.21.15.92:0/4086610214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642e9dcf7d0 con 0x5642e9eb39f0 2023-10-06T10:45:35.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.559+0000 7f1064891700 1 --2- 172.21.15.92:0/4086610214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9eb39f0 0x5642e9edb360 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f1050005d10 tx=0x7f1050017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6acc32435cc0c7ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.559+0000 7f104affd700 1 -- 172.21.15.92:0/4086610214 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050005df0 con 0x5642e9eb39f0 2023-10-06T10:45:35.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.559+0000 7f104affd700 1 -- 172.21.15.92:0/4086610214 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1050004830 con 0x5642e9eb39f0 2023-10-06T10:45:35.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.559+0000 7f104affd700 1 -- 172.21.15.92:0/4086610214 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050004bb0 con 0x5642e9eb39f0 2023-10-06T10:45:35.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.560+0000 7f10705eeb80 1 -- 172.21.15.92:0/4086610214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9eb39f0 msgr2=0x5642e9edb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.560+0000 7f10705eeb80 1 --2- 172.21.15.92:0/4086610214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9eb39f0 0x5642e9edb360 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f1050005d10 tx=0x7f1050017040 comp rx=0 tx=0).stop 2023-10-06T10:45:35.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.560+0000 7f10705eeb80 1 -- 172.21.15.92:0/4086610214 shutdown_connections 2023-10-06T10:45:35.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.560+0000 7f10705eeb80 1 --2- 172.21.15.92:0/4086610214 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642e9ef7ae0 0x5642e9ead930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.560+0000 7f10705eeb80 1 --2- 172.21.15.92:0/4086610214 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5642e9eb1c20 0x5642e9ef6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.560+0000 7f10705eeb80 1 --2- 172.21.15.92:0/4086610214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9eb39f0 0x5642e9edb360 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.560+0000 7f10705eeb80 1 -- 172.21.15.92:0/4086610214 >> 172.21.15.92:0/4086610214 conn(0x5642e9eb20c0 msgr2=0x5642e9eae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.560+0000 7f10705eeb80 1 -- 172.21.15.92:0/4086610214 shutdown_connections 2023-10-06T10:45:35.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.560+0000 7f10705eeb80 1 -- 172.21.15.92:0/4086610214 wait complete. 2023-10-06T10:45:35.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.560+0000 7f10705eeb80 1 -- start start 2023-10-06T10:45:35.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.560+0000 7f10705eeb80 1 Processor -- start 2023-10-06T10:45:35.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.561+0000 7f10705eeb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9eb1c20 0x5642e9f425f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.561+0000 7f10705eeb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642e9eb39f0 0x5642e9eac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.561+0000 7f10705eeb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5642e9ef7ae0 0x5642e9eed2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.561+0000 7f10705eeb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642e9eedbe0 con 0x5642e9eb1c20 2023-10-06T10:45:35.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.561+0000 7f10705eeb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642e9ef2210 con 0x5642e9eb39f0 2023-10-06T10:45:35.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.561+0000 7f10705eeb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642e9ef48d0 con 0x5642e9ef7ae0 2023-10-06T10:45:35.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.561+0000 7f10705eeb80 10 client.0 _ll_register_callbacks cb 0x5642e9c9df80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:35.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.561+0000 7f1065092700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9eb1c20 0x5642e9f425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.561+0000 7f1065092700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9eb1c20 0x5642e9f425f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55282/0 (socket says 172.21.15.92:55282) 2023-10-06T10:45:35.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.561+0000 7f1065092700 1 -- 172.21.15.92:0/2985305301 learned_addr learned my addr 172.21.15.92:0/2985305301 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:35.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.561+0000 7f1064891700 1 --2- 172.21.15.92:0/2985305301 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642e9eb39f0 0x5642e9eac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.561+0000 7f1065893700 1 --2- 172.21.15.92:0/2985305301 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5642e9ef7ae0 0x5642e9eed2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.561+0000 7f1065092700 1 -- 172.21.15.92:0/2985305301 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5642e9ef7ae0 msgr2=0x5642e9eed2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.561+0000 7f1065092700 1 --2- 172.21.15.92:0/2985305301 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5642e9ef7ae0 0x5642e9eed2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.561+0000 7f1065092700 1 -- 172.21.15.92:0/2985305301 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642e9eb39f0 msgr2=0x5642e9eac1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.561+0000 7f1065092700 1 --2- 172.21.15.92:0/2985305301 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642e9eb39f0 0x5642e9eac1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.561+0000 7f1065092700 1 -- 172.21.15.92:0/2985305301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105001d040 con 0x5642e9eb1c20 2023-10-06T10:45:35.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.562+0000 7f1065092700 1 --2- 172.21.15.92:0/2985305301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9eb1c20 0x5642e9f425f0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f105c01a000 tx=0x7f105c057040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a149b919590c8f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.562+0000 7f103f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:35.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.562+0000 7f103f7fe700 1 -- 172.21.15.92:0/2985305301 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c084130 con 0x5642e9eb1c20 2023-10-06T10:45:35.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.562+0000 7f103f7fe700 1 -- 172.21.15.92:0/2985305301 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f105c053290 con 0x5642e9eb1c20 2023-10-06T10:45:35.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.562+0000 7f10705eeb80 1 -- 172.21.15.92:0/2985305301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f105800bcf0 con 0x5642e9eb1c20 2023-10-06T10:45:35.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.562+0000 7f103f7fe700 1 -- 172.21.15.92:0/2985305301 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) v2 ==== 24+0+0 (secure 0 0 0) 0x7f105c010de0 con 0x5642e9eb1c20 2023-10-06T10:45:35.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.562+0000 7f10705eeb80 10 client.9237 fetch_fsmap learned FSMap version 110 2023-10-06T10:45:35.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.562+0000 7f10705eeb80 1 -- 172.21.15.92:0/2985305301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) v3 -- 0x5642e9db64c0 con 0x5642e9eb1c20 2023-10-06T10:45:35.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.562+0000 7f103f7fe700 1 -- 172.21.15.92:0/2985305301 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105800bcf0 con 0x5642e9eb1c20 2023-10-06T10:45:35.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.562+0000 7f103f7fe700 1 -- 172.21.15.92:0/2985305301 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 110) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f105c05a0b0 con 0x5642e9eb1c20 2023-10-06T10:45:35.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.562+0000 7f10705eeb80 10 client.9237 fetch_fsmap finished waiting for FSMap version 110 2023-10-06T10:45:35.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.562+0000 7f10705eeb80 10 client.9237 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2023-10-06T10:45:35.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.562+0000 7f10705eeb80 20 client.9237 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:35.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.562+0000 7f10705eeb80 1 --2- 172.21.15.92:0/2985305301 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5642e9ef2980 0x5642e9ef2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.562+0000 7f10705eeb80 4 client.9237 mds_command: new command op to 8517 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:45:35.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.562+0000 7f10705eeb80 1 -- 172.21.15.92:0/2985305301 --> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5642e9da3e50 con 0x5642e9ef2980 2023-10-06T10:45:35.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.562+0000 7f1064891700 1 --2- 172.21.15.92:0/2985305301 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5642e9ef2980 0x5642e9ef2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.562+0000 7f103f7fe700 10 client.9237 ms_handle_connect on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:35.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.562+0000 7f1064891700 1 --2- 172.21.15.92:0/2985305301 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5642e9ef2980 0x5642e9ef2d50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf5d45c68badd202 server_cookie=dda44eb025c6845c in_seq=0 out_seq=0 2023-10-06T10:45:35.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.563+0000 7f103f7fe700 1 -- 172.21.15.92:0/2985305301 <== mds.1 v2:172.21.15.92:6834/1979826278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5642e9da3e50 con 0x5642e9ef2980 2023-10-06T10:45:35.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.563+0000 7f103f7fe700 10 client.9237 handle_command_reply: tid=0 2023-10-06T10:45:35.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.563+0000 7f10705eeb80 1 client.9237 shutdown 2023-10-06T10:45:35.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.563+0000 7f10705eeb80 1 -- 172.21.15.92:0/2985305301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9eb1c20 msgr2=0x5642e9f425f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.563+0000 7f10705eeb80 1 --2- 172.21.15.92:0/2985305301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9eb1c20 0x5642e9f425f0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f105c01a000 tx=0x7f105c057040 comp rx=0 tx=0).stop 2023-10-06T10:45:35.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.563+0000 7f10705eeb80 1 -- 172.21.15.92:0/2985305301 shutdown_connections 2023-10-06T10:45:35.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.563+0000 7f10705eeb80 1 --2- 172.21.15.92:0/2985305301 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5642e9ef7ae0 0x5642e9eed2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.563+0000 7f10705eeb80 1 --2- 172.21.15.92:0/2985305301 >> [v2:172.21.15.92:6834/1979826278,v1:172.21.15.92:6836/1979826278] conn(0x5642e9ef2980 0x5642e9ef2d50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.563+0000 7f10705eeb80 1 --2- 172.21.15.92:0/2985305301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642e9eb1c20 0x5642e9f425f0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.563+0000 7f10705eeb80 1 --2- 172.21.15.92:0/2985305301 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642e9eb39f0 0x5642e9eac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.563+0000 7f103f7fe700 0 client.9237 ms_handle_reset on v2:172.21.15.92:6834/1979826278 2023-10-06T10:45:35.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.563+0000 7f10705eeb80 1 -- 172.21.15.92:0/2985305301 >> 172.21.15.92:0/2985305301 conn(0x5642e9eb20c0 msgr2=0x5642e9ed9840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.563+0000 7f10705eeb80 1 -- 172.21.15.92:0/2985305301 shutdown_connections 2023-10-06T10:45:35.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.563+0000 7f10705eeb80 1 -- 172.21.15.92:0/2985305301 wait complete. 2023-10-06T10:45:35.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.563+0000 7f10705eeb80 20 client.9237 trim_cache size 0 max 16384 2023-10-06T10:45:35.572 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:35.572 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T10:45:35.572 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:45:35.573 INFO:teuthology.orchestra.run.smithi092.stdout: "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8": { 2023-10-06T10:45:35.573 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:45:35.573 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8", 2023-10-06T10:45:35.573 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:45:35.573 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:35.573 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:35.573 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:35.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.563+0000 7f10672f6700 1 -- 172.21.15.92:0/2703039957 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1044066f30 msgr2=0x7f10440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.563+0000 7f10672f6700 1 --2- 172.21.15.92:0/2703039957 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1044066f30 0x7f10440693e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f10600b23f0 tx=0x7f1050009040 comp rx=0 tx=0).stop 2023-10-06T10:45:35.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.563+0000 7f10672f6700 1 -- 172.21.15.92:0/2703039957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10600c5250 msgr2=0x7f10601543b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.563+0000 7f10672f6700 1 --2- 172.21.15.92:0/2703039957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10600c5250 0x7f10601543b0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f105c00c560 tx=0x7f105c007e00 comp rx=0 tx=0).stop 2023-10-06T10:45:35.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.564+0000 7f10672f6700 1 -- 172.21.15.92:0/2703039957 shutdown_connections 2023-10-06T10:45:35.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.564+0000 7f10672f6700 1 --2- 172.21.15.92:0/2703039957 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f1044066f30 0x7f10440693e0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.564+0000 7f10672f6700 1 --2- 172.21.15.92:0/2703039957 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f106014de80 0x7f10600cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.564+0000 7f10672f6700 1 --2- 172.21.15.92:0/2703039957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10600c5250 0x7f10601543b0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.564+0000 7f10672f6700 1 --2- 172.21.15.92:0/2703039957 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10600cd4d0 0x7f10600b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.564+0000 7f10672f6700 1 -- 172.21.15.92:0/2703039957 >> 172.21.15.92:0/2703039957 conn(0x7f1060099700 msgr2=0x7f1060159e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.564+0000 7f10672f6700 1 -- 172.21.15.92:0/2703039957 shutdown_connections 2023-10-06T10:45:35.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.564+0000 7f10672f6700 1 -- 172.21.15.92:0/2703039957 wait complete. 2023-10-06T10:45:35.576 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:45:35.576 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:2 scrub status 2023-10-06T10:45:35.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.692+0000 7f396d2c3700 1 Processor -- start 2023-10-06T10:45:35.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.692+0000 7f396d2c3700 1 -- start start 2023-10-06T10:45:35.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.693+0000 7f396d2c3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f39680cd4d0 0x7f39680b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.693+0000 7f396d2c3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f396815ac30 con 0x7f39680c5250 2023-10-06T10:45:35.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.693+0000 7f396d2c3700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39680cb6a0 con 0x7f3968150940 2023-10-06T10:45:35.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.693+0000 7f396d2c3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39680cb090 con 0x7f39680cd4d0 2023-10-06T10:45:35.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.693+0000 7f39677fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f39680cd4d0 0x7f39680b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.693+0000 7f39667fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f39680c5250 0x7f39680ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45572/0 (socket says 172.21.15.92:45572) 2023-10-06T10:45:35.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.693+0000 7f39677fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f39680cd4d0 0x7f39680b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55298/0 (socket says 172.21.15.92:55298) 2023-10-06T10:45:35.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.693+0000 7f39667fc700 1 -- 172.21.15.92:0/222452436 learned_addr learned my addr 172.21.15.92:0/222452436 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:35.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.693+0000 7f39677fe700 1 -- 172.21.15.92:0/222452436 >> v1:172.21.15.124:6789/0 conn(0x7f3968150940 legacy=0x7f3968159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.693+0000 7f39677fe700 1 -- 172.21.15.92:0/222452436 >> v1:172.21.15.92:6789/0 conn(0x7f39680c5250 legacy=0x7f39680ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.693+0000 7f39677fe700 1 -- 172.21.15.92:0/222452436 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f396815a670 con 0x7f39680cd4d0 2023-10-06T10:45:35.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.693+0000 7f3965ffb700 1 -- 172.21.15.92:0/222452436 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83794384 0 0) 0x7f395c001110 con 0x7f39680c5250 2023-10-06T10:45:35.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.693+0000 7f39677fe700 1 --2- 172.21.15.92:0/222452436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39680cd4d0 0x7f39680b2050 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f39580098e0 tx=0x7f39580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160b609131b4b6d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.693+0000 7f3965ffb700 1 -- 172.21.15.92:0/222452436 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3958009e80 con 0x7f39680cd4d0 2023-10-06T10:45:35.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.693+0000 7f3965ffb700 1 -- 172.21.15.92:0/222452436 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395800b820 con 0x7f39680cd4d0 2023-10-06T10:45:35.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.693+0000 7f3965ffb700 1 -- 172.21.15.92:0/222452436 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395800bba0 con 0x7f39680cd4d0 2023-10-06T10:45:35.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.694+0000 7f396d2c3700 1 -- 172.21.15.92:0/222452436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39680cd4d0 msgr2=0x7f39680b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.694+0000 7f396d2c3700 1 --2- 172.21.15.92:0/222452436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39680cd4d0 0x7f39680b2050 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f39580098e0 tx=0x7f39580091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.694+0000 7f396d2c3700 1 -- 172.21.15.92:0/222452436 shutdown_connections 2023-10-06T10:45:35.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.694+0000 7f396d2c3700 1 --2- 172.21.15.92:0/222452436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39680cd4d0 0x7f39680b2050 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.694+0000 7f396d2c3700 1 -- 172.21.15.92:0/222452436 >> 172.21.15.92:0/222452436 conn(0x7f3968099700 msgr2=0x7f396806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.694+0000 7f396d2c3700 1 -- 172.21.15.92:0/222452436 shutdown_connections 2023-10-06T10:45:35.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.694+0000 7f396d2c3700 1 -- 172.21.15.92:0/222452436 wait complete. 2023-10-06T10:45:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.695+0000 7f396d2c3700 1 Processor -- start 2023-10-06T10:45:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.695+0000 7f396d2c3700 1 -- start start 2023-10-06T10:45:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.696+0000 7f396d2c3700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39680c5250 0x7f39680b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.696+0000 7f396d2c3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39680cd4d0 0x7f39680b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.696+0000 7f396d2c3700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f396809d370 0x7f396809d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.696+0000 7f396d2c3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396815c130 con 0x7f39680cd4d0 2023-10-06T10:45:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.696+0000 7f396d2c3700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396809dd30 con 0x7f396809d370 2023-10-06T10:45:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.696+0000 7f396d2c3700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39680c9f30 con 0x7f39680c5250 2023-10-06T10:45:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.696+0000 7f39667fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39680cd4d0 0x7f39680b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.696+0000 7f39667fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39680cd4d0 0x7f39680b3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55310/0 (socket says 172.21.15.92:55310) 2023-10-06T10:45:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.696+0000 7f39667fc700 1 -- 172.21.15.92:0/2971061475 learned_addr learned my addr 172.21.15.92:0/2971061475 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.696+0000 7f39677fe700 1 --2- 172.21.15.92:0/2971061475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f396809d370 0x7f396809d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.696+0000 7f3966ffd700 1 --2- 172.21.15.92:0/2971061475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39680c5250 0x7f39680b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.696+0000 7f39667fc700 1 -- 172.21.15.92:0/2971061475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39680c5250 msgr2=0x7f39680b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.696+0000 7f39667fc700 1 --2- 172.21.15.92:0/2971061475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39680c5250 0x7f39680b5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.696+0000 7f39667fc700 1 -- 172.21.15.92:0/2971061475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f396809d370 msgr2=0x7f396809d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.696+0000 7f39667fc700 1 --2- 172.21.15.92:0/2971061475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f396809d370 0x7f396809d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.696+0000 7f39667fc700 1 -- 172.21.15.92:0/2971061475 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3958009420 con 0x7f39680cd4d0 2023-10-06T10:45:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.697+0000 7f39667fc700 1 --2- 172.21.15.92:0/2971061475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39680cd4d0 0x7f39680b3c10 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f395c00c5b0 tx=0x7f395c001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.697+0000 7f394effd700 1 -- 172.21.15.92:0/2971061475 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c005b30 con 0x7f39680cd4d0 2023-10-06T10:45:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.697+0000 7f394effd700 1 -- 172.21.15.92:0/2971061475 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395c005c90 con 0x7f39680cd4d0 2023-10-06T10:45:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.697+0000 7f396d2c3700 1 -- 172.21.15.92:0/2971061475 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39680c93b0 con 0x7f39680cd4d0 2023-10-06T10:45:35.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.697+0000 7f394effd700 1 -- 172.21.15.92:0/2971061475 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c007300 con 0x7f39680cd4d0 2023-10-06T10:45:35.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.697+0000 7f396d2c3700 1 -- 172.21.15.92:0/2971061475 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39680c9970 con 0x7f39680cd4d0 2023-10-06T10:45:35.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.698+0000 7f394effd700 1 -- 172.21.15.92:0/2971061475 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f395c01b3b0 con 0x7f39680cd4d0 2023-10-06T10:45:35.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.699+0000 7f394effd700 1 --2- 172.21.15.92:0/2971061475 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3954066f30 0x7f39540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.699+0000 7f3966ffd700 1 --2- 172.21.15.92:0/2971061475 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3954066f30 0x7f39540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.699+0000 7f394effd700 1 -- 172.21.15.92:0/2971061475 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f395c083bf0 con 0x7f39680cd4d0 2023-10-06T10:45:35.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.699+0000 7f3966ffd700 1 --2- 172.21.15.92:0/2971061475 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3954066f30 0x7f39540693e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f396000e280 tx=0x7f3960009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.701+0000 7f39765bbb80 1 Processor -- start 2023-10-06T10:45:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.701+0000 7f39765bbb80 1 -- start start 2023-10-06T10:45:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.701+0000 7f39765bbb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56530c294ae0 0x56530c294eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.701+0000 7f39765bbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c24d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.701+0000 7f39765bbb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56530c24ec20 0x56530c24db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.702+0000 7f39765bbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56530c1cdcd0 con 0x56530c2509f0 2023-10-06T10:45:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.702+0000 7f39765bbb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56530c1a4ac0 con 0x56530c24ec20 2023-10-06T10:45:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.702+0000 7f39765bbb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56530c1a22c0 con 0x56530c294ae0 2023-10-06T10:45:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.702+0000 7f39667fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c24d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.702+0000 7f39667fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c24d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55318/0 (socket says 172.21.15.92:55318) 2023-10-06T10:45:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.702+0000 7f39667fc700 1 -- 172.21.15.92:0/1409855925 learned_addr learned my addr 172.21.15.92:0/1409855925 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.702+0000 7f3966ffd700 1 --2- 172.21.15.92:0/1409855925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56530c294ae0 0x56530c294eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.702+0000 7f39677fe700 1 --2- 172.21.15.92:0/1409855925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56530c24ec20 0x56530c24db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.702+0000 7f39667fc700 1 -- 172.21.15.92:0/1409855925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56530c294ae0 msgr2=0x56530c294eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.702+0000 7f39667fc700 1 --2- 172.21.15.92:0/1409855925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56530c294ae0 0x56530c294eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.702+0000 7f39667fc700 1 -- 172.21.15.92:0/1409855925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56530c24ec20 msgr2=0x56530c24db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.702+0000 7f39667fc700 1 --2- 172.21.15.92:0/1409855925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56530c24ec20 0x56530c24db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.702+0000 7f39667fc700 1 -- 172.21.15.92:0/1409855925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56530c0382c0 con 0x56530c2509f0 2023-10-06T10:45:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.702+0000 7f39667fc700 1 --2- 172.21.15.92:0/1409855925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c24d5d0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f395c081e40 tx=0x7f395c081d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533315f63102970c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.702+0000 7f3947fff700 1 -- 172.21.15.92:0/1409855925 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c057a70 con 0x56530c2509f0 2023-10-06T10:45:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.702+0000 7f3947fff700 1 -- 172.21.15.92:0/1409855925 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395c05b7c0 con 0x56530c2509f0 2023-10-06T10:45:35.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.702+0000 7f3947fff700 1 -- 172.21.15.92:0/1409855925 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c005e00 con 0x56530c2509f0 2023-10-06T10:45:35.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.703+0000 7f39765bbb80 1 -- 172.21.15.92:0/1409855925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 msgr2=0x56530c24d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.703+0000 7f39765bbb80 1 --2- 172.21.15.92:0/1409855925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c24d5d0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f395c081e40 tx=0x7f395c081d70 comp rx=0 tx=0).stop 2023-10-06T10:45:35.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.703+0000 7f39765bbb80 1 -- 172.21.15.92:0/1409855925 shutdown_connections 2023-10-06T10:45:35.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.703+0000 7f39765bbb80 1 --2- 172.21.15.92:0/1409855925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56530c24ec20 0x56530c24db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.703+0000 7f39765bbb80 1 --2- 172.21.15.92:0/1409855925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56530c294ae0 0x56530c294eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.703+0000 7f39765bbb80 1 --2- 172.21.15.92:0/1409855925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c24d5d0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.703+0000 7f39765bbb80 1 -- 172.21.15.92:0/1409855925 >> 172.21.15.92:0/1409855925 conn(0x56530c24f0c0 msgr2=0x56530c24f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.703+0000 7f39765bbb80 1 -- 172.21.15.92:0/1409855925 shutdown_connections 2023-10-06T10:45:35.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.703+0000 7f39765bbb80 1 -- 172.21.15.92:0/1409855925 wait complete. 2023-10-06T10:45:35.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.704+0000 7f39765bbb80 1 -- start start 2023-10-06T10:45:35.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.704+0000 7f39765bbb80 1 Processor -- start 2023-10-06T10:45:35.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.704+0000 7f39765bbb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56530c24ec20 0x56530c274370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39765bbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c27a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39765bbb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56530c294ae0 0x56530c279930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39765bbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56530c19e0b0 con 0x56530c2509f0 2023-10-06T10:45:35.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39765bbb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56530c1498e0 con 0x56530c294ae0 2023-10-06T10:45:35.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39765bbb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56530c164ef0 con 0x56530c24ec20 2023-10-06T10:45:35.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39765bbb80 10 client.0 _ll_register_callbacks cb 0x56530c03af80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:35.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39667fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c27a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39667fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c27a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55334/0 (socket says 172.21.15.92:55334) 2023-10-06T10:45:35.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39667fc700 1 -- 172.21.15.92:0/2558481173 learned_addr learned my addr 172.21.15.92:0/2558481173 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:35.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39667fc700 1 -- 172.21.15.92:0/2558481173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56530c24ec20 msgr2=0x56530c274370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39677fe700 1 --2- 172.21.15.92:0/2558481173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56530c294ae0 0x56530c279930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f3966ffd700 1 --2- 172.21.15.92:0/2558481173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56530c24ec20 0x56530c274370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39667fc700 1 --2- 172.21.15.92:0/2558481173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56530c24ec20 0x56530c274370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39667fc700 1 -- 172.21.15.92:0/2558481173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56530c294ae0 msgr2=0x56530c279930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39667fc700 1 --2- 172.21.15.92:0/2558481173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56530c294ae0 0x56530c279930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39667fc700 1 -- 172.21.15.92:0/2558481173 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39680c9970 con 0x56530c2509f0 2023-10-06T10:45:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39667fc700 1 --2- 172.21.15.92:0/2558481173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c27a940 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f395c0070a0 tx=0x7f395c069000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a5fb26bb8049a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39457fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39457fa700 1 -- 172.21.15.92:0/2558481173 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c005e00 con 0x56530c2509f0 2023-10-06T10:45:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39457fa700 1 -- 172.21.15.92:0/2558481173 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f395c06ae40 con 0x56530c2509f0 2023-10-06T10:45:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39765bbb80 1 -- 172.21.15.92:0/2558481173 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56530c2426e0 con 0x56530c2509f0 2023-10-06T10:45:35.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39457fa700 1 -- 172.21.15.92:0/2558481173 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) v2 ==== 24+0+0 (secure 0 0 0) 0x7f395c0702d0 con 0x56530c2509f0 2023-10-06T10:45:35.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39765bbb80 10 client.9252 fetch_fsmap learned FSMap version 110 2023-10-06T10:45:35.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39765bbb80 1 -- 172.21.15.92:0/2558481173 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) v3 -- 0x56530bfdb030 con 0x56530c2509f0 2023-10-06T10:45:35.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.705+0000 7f39457fa700 1 -- 172.21.15.92:0/2558481173 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56530c2426e0 con 0x56530c2509f0 2023-10-06T10:45:35.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.706+0000 7f39457fa700 1 -- 172.21.15.92:0/2558481173 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 110) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f395c053d10 con 0x56530c2509f0 2023-10-06T10:45:35.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.706+0000 7f39765bbb80 10 client.9252 fetch_fsmap finished waiting for FSMap version 110 2023-10-06T10:45:35.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.706+0000 7f39765bbb80 10 client.9252 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2023-10-06T10:45:35.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.706+0000 7f39765bbb80 20 client.9252 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:35.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.706+0000 7f39765bbb80 1 --2- 172.21.15.92:0/2558481173 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x56530c28fbb0 0x56530c274e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.706+0000 7f39765bbb80 4 client.9252 mds_command: new command op to 6719 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:35.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.706+0000 7f39765bbb80 1 -- 172.21.15.92:0/2558481173 --> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56530c21f610 con 0x56530c28fbb0 2023-10-06T10:45:35.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.706+0000 7f3966ffd700 1 --2- 172.21.15.92:0/2558481173 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x56530c28fbb0 0x56530c274e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.706+0000 7f39457fa700 10 client.9252 ms_handle_connect on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.706+0000 7f3966ffd700 1 --2- 172.21.15.92:0/2558481173 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x56530c28fbb0 0x56530c274e30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6bfcfeafada2393 server_cookie=186add203e0be798 in_seq=0 out_seq=0 2023-10-06T10:45:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.707+0000 7f39457fa700 1 -- 172.21.15.92:0/2558481173 <== mds.2 v2:172.21.15.124:6833/2307521923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x56530c21f610 con 0x56530c28fbb0 2023-10-06T10:45:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.707+0000 7f39457fa700 10 client.9252 handle_command_reply: tid=0 2023-10-06T10:45:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.707+0000 7f39765bbb80 1 client.9252 shutdown 2023-10-06T10:45:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.707+0000 7f39765bbb80 1 -- 172.21.15.92:0/2558481173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 msgr2=0x56530c27a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.707+0000 7f39765bbb80 1 --2- 172.21.15.92:0/2558481173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c27a940 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f395c0070a0 tx=0x7f395c069000 comp rx=0 tx=0).stop 2023-10-06T10:45:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.707+0000 7f39765bbb80 1 -- 172.21.15.92:0/2558481173 shutdown_connections 2023-10-06T10:45:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.707+0000 7f39765bbb80 1 --2- 172.21.15.92:0/2558481173 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x56530c28fbb0 0x56530c274e30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.707+0000 7f39765bbb80 1 --2- 172.21.15.92:0/2558481173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56530c294ae0 0x56530c279930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.707+0000 7f39765bbb80 1 --2- 172.21.15.92:0/2558481173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56530c24ec20 0x56530c274370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.707+0000 7f39457fa700 0 client.9252 ms_handle_reset on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:35.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.707+0000 7f39765bbb80 1 --2- 172.21.15.92:0/2558481173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c27a940 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.707+0000 7f39765bbb80 1 -- 172.21.15.92:0/2558481173 >> 172.21.15.92:0/2558481173 conn(0x56530c24f0c0 msgr2=0x56530c24a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.707+0000 7f39765bbb80 1 -- 172.21.15.92:0/2558481173 shutdown_connections 2023-10-06T10:45:35.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.707+0000 7f39765bbb80 1 -- 172.21.15.92:0/2558481173 wait complete. 2023-10-06T10:45:35.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.707+0000 7f39765bbb80 20 client.9252 trim_cache size 0 max 16384 2023-10-06T10:45:35.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.723+0000 7f39765bbb80 1 Processor -- start 2023-10-06T10:45:35.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.723+0000 7f39765bbb80 1 -- start start 2023-10-06T10:45:35.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.724+0000 7f39765bbb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56530c24ec20 0x56530c2736b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.724+0000 7f39765bbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c2a30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.724+0000 7f39765bbb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56530c294ae0 0x56530c2932d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.724+0000 7f39765bbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56530c020e80 con 0x56530c2509f0 2023-10-06T10:45:35.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.724+0000 7f39765bbb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56530c273d80 con 0x56530c294ae0 2023-10-06T10:45:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.724+0000 7f39765bbb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56530c250320 con 0x56530c24ec20 2023-10-06T10:45:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.724+0000 7f39667fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c2a30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.724+0000 7f39667fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c2a30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55350/0 (socket says 172.21.15.92:55350) 2023-10-06T10:45:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.724+0000 7f39667fc700 1 -- 172.21.15.92:0/3266235903 learned_addr learned my addr 172.21.15.92:0/3266235903 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.724+0000 7f39667fc700 1 -- 172.21.15.92:0/3266235903 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56530c24ec20 msgr2=0x56530c2736b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.724+0000 7f39667fc700 1 --2- 172.21.15.92:0/3266235903 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56530c24ec20 0x56530c2736b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.724+0000 7f39667fc700 1 -- 172.21.15.92:0/3266235903 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56530c294ae0 msgr2=0x56530c2932d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.724+0000 7f39667fc700 1 --2- 172.21.15.92:0/3266235903 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56530c294ae0 0x56530c2932d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.724+0000 7f39667fc700 1 -- 172.21.15.92:0/3266235903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56530c16c7d0 con 0x56530c2509f0 2023-10-06T10:45:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.724+0000 7f39667fc700 1 --2- 172.21.15.92:0/3266235903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c2a30c0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f395c001eb0 tx=0x7f395c070000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64e7a5cbb53867 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.724+0000 7f394cff9700 1 -- 172.21.15.92:0/3266235903 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c00e040 con 0x56530c2509f0 2023-10-06T10:45:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.724+0000 7f394cff9700 1 -- 172.21.15.92:0/3266235903 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395c005e00 con 0x56530c2509f0 2023-10-06T10:45:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.724+0000 7f394cff9700 1 -- 172.21.15.92:0/3266235903 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c053d10 con 0x56530c2509f0 2023-10-06T10:45:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.725+0000 7f39765bbb80 1 -- 172.21.15.92:0/3266235903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 msgr2=0x56530c2a30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.725+0000 7f39765bbb80 1 --2- 172.21.15.92:0/3266235903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c2a30c0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f395c001eb0 tx=0x7f395c070000 comp rx=0 tx=0).stop 2023-10-06T10:45:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.725+0000 7f39765bbb80 1 -- 172.21.15.92:0/3266235903 shutdown_connections 2023-10-06T10:45:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.725+0000 7f39765bbb80 1 --2- 172.21.15.92:0/3266235903 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56530c294ae0 0x56530c2932d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.725+0000 7f39765bbb80 1 --2- 172.21.15.92:0/3266235903 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56530c24ec20 0x56530c2736b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.725+0000 7f39765bbb80 1 --2- 172.21.15.92:0/3266235903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c2a30c0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.725+0000 7f39765bbb80 1 -- 172.21.15.92:0/3266235903 >> 172.21.15.92:0/3266235903 conn(0x56530c24f0c0 msgr2=0x56530c28ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.725+0000 7f39765bbb80 1 -- 172.21.15.92:0/3266235903 shutdown_connections 2023-10-06T10:45:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.725+0000 7f39765bbb80 1 -- 172.21.15.92:0/3266235903 wait complete. 2023-10-06T10:45:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.725+0000 7f39765bbb80 1 -- start start 2023-10-06T10:45:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.726+0000 7f39765bbb80 1 Processor -- start 2023-10-06T10:45:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.726+0000 7f39765bbb80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56530c24ec20 0x56530c2932d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.726+0000 7f39765bbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c293810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.726+0000 7f39765bbb80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56530c294ae0 0x56530c2f9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.726+0000 7f39765bbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56530c2a3430 con 0x56530c2509f0 2023-10-06T10:45:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.726+0000 7f39765bbb80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56530c292b70 con 0x56530c24ec20 2023-10-06T10:45:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.726+0000 7f39765bbb80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56530c292240 con 0x56530c294ae0 2023-10-06T10:45:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.726+0000 7f39765bbb80 10 client.0 _ll_register_callbacks cb 0x56530c03af80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.726+0000 7f39667fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c293810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39667fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c293810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55352/0 (socket says 172.21.15.92:55352) 2023-10-06T10:45:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39667fc700 1 -- 172.21.15.92:0/2562588428 learned_addr learned my addr 172.21.15.92:0/2562588428 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39677fe700 1 --2- 172.21.15.92:0/2562588428 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56530c294ae0 0x56530c2f9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f3966ffd700 1 --2- 172.21.15.92:0/2562588428 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56530c24ec20 0x56530c2932d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39667fc700 1 -- 172.21.15.92:0/2562588428 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56530c294ae0 msgr2=0x56530c2f9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39667fc700 1 --2- 172.21.15.92:0/2562588428 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56530c294ae0 0x56530c2f9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39667fc700 1 -- 172.21.15.92:0/2562588428 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56530c24ec20 msgr2=0x56530c2932d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39667fc700 1 --2- 172.21.15.92:0/2562588428 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56530c24ec20 0x56530c2932d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39667fc700 1 -- 172.21.15.92:0/2562588428 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56530bfdb030 con 0x56530c2509f0 2023-10-06T10:45:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39677fe700 1 --2- 172.21.15.92:0/2562588428 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56530c294ae0 0x56530c2f9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f3966ffd700 1 --2- 172.21.15.92:0/2562588428 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56530c24ec20 0x56530c2932d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:35.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39667fc700 1 --2- 172.21.15.92:0/2562588428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c293810 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f395c082180 tx=0x7f395c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8befcb9a2b13d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39457fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:35.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39457fa700 1 -- 172.21.15.92:0/2562588428 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c00e040 con 0x56530c2509f0 2023-10-06T10:45:35.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39457fa700 1 -- 172.21.15.92:0/2562588428 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f395c06ae40 con 0x56530c2509f0 2023-10-06T10:45:35.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39765bbb80 1 -- 172.21.15.92:0/2562588428 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f395c05b7c0 con 0x56530c2509f0 2023-10-06T10:45:35.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39457fa700 1 -- 172.21.15.92:0/2562588428 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) v2 ==== 24+0+0 (secure 0 0 0) 0x7f395c053d10 con 0x56530c2509f0 2023-10-06T10:45:35.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39765bbb80 10 client.9258 fetch_fsmap learned FSMap version 110 2023-10-06T10:45:35.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39765bbb80 1 -- 172.21.15.92:0/2562588428 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) v3 -- 0x56530c1534c0 con 0x56530c2509f0 2023-10-06T10:45:35.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39457fa700 1 -- 172.21.15.92:0/2562588428 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395c05b7c0 con 0x56530c2509f0 2023-10-06T10:45:35.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39457fa700 1 -- 172.21.15.92:0/2562588428 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 110) v1 ==== 3140+0+0 (secure 0 0 0) 0x7f395c07c120 con 0x56530c2509f0 2023-10-06T10:45:35.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39765bbb80 10 client.9258 fetch_fsmap finished waiting for FSMap version 110 2023-10-06T10:45:35.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39765bbb80 10 client.9258 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2023-10-06T10:45:35.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39765bbb80 20 client.9258 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:35.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39765bbb80 1 --2- 172.21.15.92:0/2562588428 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x56530c290e60 0x56530c2f97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39765bbb80 4 client.9258 mds_command: new command op to 6719 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:45:35.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.727+0000 7f39765bbb80 1 -- 172.21.15.92:0/2562588428 --> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56530c140e50 con 0x56530c290e60 2023-10-06T10:45:35.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.728+0000 7f3966ffd700 1 --2- 172.21.15.92:0/2562588428 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x56530c290e60 0x56530c2f97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.728+0000 7f39457fa700 10 client.9258 ms_handle_connect on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:35.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.728+0000 7f3966ffd700 1 --2- 172.21.15.92:0/2562588428 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x56530c290e60 0x56530c2f97b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=563aee8b951bb5df server_cookie=8431cc8eb2145fd5 in_seq=0 out_seq=0 2023-10-06T10:45:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.728+0000 7f39457fa700 1 -- 172.21.15.92:0/2562588428 <== mds.2 v2:172.21.15.124:6833/2307521923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x56530c140e50 con 0x56530c290e60 2023-10-06T10:45:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.728+0000 7f39457fa700 10 client.9258 handle_command_reply: tid=0 2023-10-06T10:45:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.728+0000 7f39765bbb80 1 client.9258 shutdown 2023-10-06T10:45:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.728+0000 7f39765bbb80 1 -- 172.21.15.92:0/2562588428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 msgr2=0x56530c293810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.728+0000 7f39765bbb80 1 --2- 172.21.15.92:0/2562588428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c293810 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f395c082180 tx=0x7f395c009040 comp rx=0 tx=0).stop 2023-10-06T10:45:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.728+0000 7f39765bbb80 1 -- 172.21.15.92:0/2562588428 shutdown_connections 2023-10-06T10:45:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.728+0000 7f39765bbb80 1 --2- 172.21.15.92:0/2562588428 >> [v2:172.21.15.124:6833/2307521923,v1:172.21.15.124:6835/2307521923] conn(0x56530c290e60 0x56530c2f97b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.728+0000 7f39765bbb80 1 --2- 172.21.15.92:0/2562588428 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56530c294ae0 0x56530c2f9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.728+0000 7f39765bbb80 1 --2- 172.21.15.92:0/2562588428 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56530c24ec20 0x56530c2932d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.728+0000 7f39457fa700 0 client.9258 ms_handle_reset on v2:172.21.15.124:6833/2307521923 2023-10-06T10:45:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.728+0000 7f39765bbb80 1 --2- 172.21.15.92:0/2562588428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56530c2509f0 0x56530c293810 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.728+0000 7f39765bbb80 1 -- 172.21.15.92:0/2562588428 >> 172.21.15.92:0/2562588428 conn(0x56530c24f0c0 msgr2=0x56530c2794a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.728+0000 7f39765bbb80 1 -- 172.21.15.92:0/2562588428 shutdown_connections 2023-10-06T10:45:35.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.728+0000 7f39765bbb80 1 -- 172.21.15.92:0/2562588428 wait complete. 2023-10-06T10:45:35.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.729+0000 7f39765bbb80 20 client.9258 trim_cache size 0 max 16384 2023-10-06T10:45:35.737 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:35.737 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T10:45:35.737 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:45:35.737 INFO:teuthology.orchestra.run.smithi092.stdout: "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8": { 2023-10-06T10:45:35.737 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:45:35.738 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8", 2023-10-06T10:45:35.738 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:45:35.738 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:35.738 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:35.738 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:35.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.729+0000 7f396d2c3700 1 -- 172.21.15.92:0/2971061475 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3954066f30 msgr2=0x7f39540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.729+0000 7f396d2c3700 1 --2- 172.21.15.92:0/2971061475 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3954066f30 0x7f39540693e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f396000e280 tx=0x7f3960009040 comp rx=0 tx=0).stop 2023-10-06T10:45:35.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.729+0000 7f396d2c3700 1 -- 172.21.15.92:0/2971061475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39680cd4d0 msgr2=0x7f39680b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.729+0000 7f396d2c3700 1 --2- 172.21.15.92:0/2971061475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39680cd4d0 0x7f39680b3c10 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f395c00c5b0 tx=0x7f395c001070 comp rx=0 tx=0).stop 2023-10-06T10:45:35.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.729+0000 7f396d2c3700 1 -- 172.21.15.92:0/2971061475 shutdown_connections 2023-10-06T10:45:35.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.729+0000 7f396d2c3700 1 --2- 172.21.15.92:0/2971061475 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3954066f30 0x7f39540693e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.729+0000 7f396d2c3700 1 --2- 172.21.15.92:0/2971061475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f396809d370 0x7f396809d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.729+0000 7f396d2c3700 1 --2- 172.21.15.92:0/2971061475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39680c5250 0x7f39680b5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.729+0000 7f396d2c3700 1 --2- 172.21.15.92:0/2971061475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39680cd4d0 0x7f39680b3c10 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.729+0000 7f396d2c3700 1 -- 172.21.15.92:0/2971061475 >> 172.21.15.92:0/2971061475 conn(0x7f3968099700 msgr2=0x7f39680b2860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.729+0000 7f396d2c3700 1 -- 172.21.15.92:0/2971061475 shutdown_connections 2023-10-06T10:45:35.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.729+0000 7f396d2c3700 1 -- 172.21.15.92:0/2971061475 wait complete. 2023-10-06T10:45:35.741 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:45:35.741 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub status 2023-10-06T10:45:35.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.858+0000 7fe4e9b2f700 1 Processor -- start 2023-10-06T10:45:35.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.858+0000 7fe4e9b2f700 1 -- start start 2023-10-06T10:45:35.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.859+0000 7fe4e9b2f700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe4e40cd3c0 0x7fe4e40ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.859+0000 7fe4e9b2f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4e40cc970 con 0x7fe4e40b29f0 2023-10-06T10:45:35.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.859+0000 7fe4e9b2f700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4e40cbd60 con 0x7fe4e40b2620 2023-10-06T10:45:35.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.859+0000 7fe4e9b2f700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4e40cc040 con 0x7fe4e40cd3c0 2023-10-06T10:45:35.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.859+0000 7fe4e932e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe4e40b29f0 0x7fe4e40c5350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45588/0 (socket says 172.21.15.92:45588) 2023-10-06T10:45:35.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.859+0000 7fe4e932e700 1 -- 172.21.15.92:0/2592161348 learned_addr learned my addr 172.21.15.92:0/2592161348 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:35.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.859+0000 7fe4e37fe700 1 -- 172.21.15.92:0/2592161348 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83530483 0 0) 0x7fe4e40cc970 con 0x7fe4e40b29f0 2023-10-06T10:45:35.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.859+0000 7fe4e37fe700 1 -- 172.21.15.92:0/2592161348 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4d4003570 con 0x7fe4e40b29f0 2023-10-06T10:45:35.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.859+0000 7fe4e3fff700 1 --2- 172.21.15.92:0/2592161348 >> v2:172.21.15.124:3300/0 conn(0x7fe4e40cd3c0 0x7fe4e40ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.859+0000 7fe4e37fe700 1 -- 172.21.15.92:0/2592161348 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2679491232 0 0) 0x7fe4d4003570 con 0x7fe4e40b29f0 2023-10-06T10:45:35.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.859+0000 7fe4e37fe700 1 -- 172.21.15.92:0/2592161348 >> v2:172.21.15.124:3300/0 conn(0x7fe4e40cd3c0 msgr2=0x7fe4e40ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.859+0000 7fe4e37fe700 1 --2- 172.21.15.92:0/2592161348 >> v2:172.21.15.124:3300/0 conn(0x7fe4e40cd3c0 0x7fe4e40ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.859+0000 7fe4e37fe700 1 -- 172.21.15.92:0/2592161348 >> v1:172.21.15.124:6789/0 conn(0x7fe4e40b2620 legacy=0x7fe4e40cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.859+0000 7fe4e37fe700 1 -- 172.21.15.92:0/2592161348 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e40cc320 con 0x7fe4e40b29f0 2023-10-06T10:45:35.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.859+0000 7fe4e37fe700 1 -- 172.21.15.92:0/2592161348 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fe4dc0017f0 con 0x7fe4e40b29f0 2023-10-06T10:45:35.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.859+0000 7fe4e37fe700 1 -- 172.21.15.92:0/2592161348 >> v1:172.21.15.92:6789/0 conn(0x7fe4e40b29f0 legacy=0x7fe4e40c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.859+0000 7fe4e37fe700 1 --2- 172.21.15.92:0/2592161348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4d4003a60 0x7fe4d4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.860+0000 7fe4e37fe700 1 --2- 172.21.15.92:0/2592161348 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4d4004810 0x7fe4d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.860+0000 7fe4e37fe700 1 --2- 172.21.15.92:0/2592161348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4d40051c0 0x7fe4d400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.860+0000 7fe4e37fe700 1 -- 172.21.15.92:0/2592161348 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e40cc040 con 0x7fe4d40051c0 2023-10-06T10:45:35.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.860+0000 7fe4e37fe700 1 -- 172.21.15.92:0/2592161348 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d400c0d0 con 0x7fe4d4004810 2023-10-06T10:45:35.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.860+0000 7fe4e37fe700 1 -- 172.21.15.92:0/2592161348 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d400c290 con 0x7fe4d4003a60 2023-10-06T10:45:35.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.860+0000 7fe4e932e700 1 --2- 172.21.15.92:0/2592161348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4d40051c0 0x7fe4d400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.860+0000 7fe4e3fff700 1 --2- 172.21.15.92:0/2592161348 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4d4004810 0x7fe4d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.860+0000 7fe4e8b2d700 1 --2- 172.21.15.92:0/2592161348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4d4003a60 0x7fe4d4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.860+0000 7fe4e932e700 1 -- 172.21.15.92:0/2592161348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4d4003a60 msgr2=0x7fe4d4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.860+0000 7fe4e932e700 1 --2- 172.21.15.92:0/2592161348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4d4003a60 0x7fe4d4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.860+0000 7fe4e932e700 1 -- 172.21.15.92:0/2592161348 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4d4004810 msgr2=0x7fe4d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.860+0000 7fe4e932e700 1 --2- 172.21.15.92:0/2592161348 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4d4004810 0x7fe4d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.860+0000 7fe4e932e700 1 -- 172.21.15.92:0/2592161348 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d4003570 con 0x7fe4d40051c0 2023-10-06T10:45:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.860+0000 7fe4e932e700 1 -- 172.21.15.92:0/2592161348 reap_dead start 2023-10-06T10:45:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.860+0000 7fe4e932e700 1 --2- 172.21.15.92:0/2592161348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4d40051c0 0x7fe4d400ba60 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00a900 tx=0x7fe4dc0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981e4391f555a5c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.860+0000 7fe4e37fe700 1 -- 172.21.15.92:0/2592161348 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc0070d0 con 0x7fe4d40051c0 2023-10-06T10:45:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.860+0000 7fe4e37fe700 1 -- 172.21.15.92:0/2592161348 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4dc007230 con 0x7fe4d40051c0 2023-10-06T10:45:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.860+0000 7fe4e37fe700 1 -- 172.21.15.92:0/2592161348 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc0075d0 con 0x7fe4d40051c0 2023-10-06T10:45:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.861+0000 7fe4e9b2f700 1 -- 172.21.15.92:0/2592161348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4d40051c0 msgr2=0x7fe4d400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.861+0000 7fe4e9b2f700 1 --2- 172.21.15.92:0/2592161348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4d40051c0 0x7fe4d400ba60 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00a900 tx=0x7fe4dc0021d0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.861+0000 7fe4e9b2f700 1 -- 172.21.15.92:0/2592161348 shutdown_connections 2023-10-06T10:45:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.861+0000 7fe4e9b2f700 1 --2- 172.21.15.92:0/2592161348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4d40051c0 0x7fe4d400ba60 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.861+0000 7fe4e9b2f700 1 -- 172.21.15.92:0/2592161348 >> 172.21.15.92:0/2592161348 conn(0x7fe4e4099520 msgr2=0x7fe4e406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.861+0000 7fe4e9b2f700 1 -- 172.21.15.92:0/2592161348 shutdown_connections 2023-10-06T10:45:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.861+0000 7fe4e9b2f700 1 -- 172.21.15.92:0/2592161348 wait complete. 2023-10-06T10:45:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.862+0000 7fe4e9b2f700 1 Processor -- start 2023-10-06T10:45:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.862+0000 7fe4e9b2f700 1 -- start start 2023-10-06T10:45:35.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.863+0000 7fe4e9b2f700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4d40051c0 0x7fe4e4152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.863+0000 7fe4e9b2f700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4e415b810 0x7fe4e415bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.863+0000 7fe4e9b2f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4e40c55e0 0x7fe4e40c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.863+0000 7fe4e9b2f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e40c8270 con 0x7fe4e40c55e0 2023-10-06T10:45:35.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.863+0000 7fe4e9b2f700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e40c5f70 con 0x7fe4e415b810 2023-10-06T10:45:35.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.863+0000 7fe4e9b2f700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e40c7380 con 0x7fe4d40051c0 2023-10-06T10:45:35.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.863+0000 7fe4e932e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4e40c55e0 0x7fe4e40c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.863+0000 7fe4e932e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4e40c55e0 0x7fe4e40c5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55360/0 (socket says 172.21.15.92:55360) 2023-10-06T10:45:35.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.863+0000 7fe4e932e700 1 -- 172.21.15.92:0/55493145 learned_addr learned my addr 172.21.15.92:0/55493145 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:35.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.863+0000 7fe4e3fff700 1 --2- 172.21.15.92:0/55493145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4e415b810 0x7fe4e415bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.863+0000 7fe4e8b2d700 1 --2- 172.21.15.92:0/55493145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4d40051c0 0x7fe4e4152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.863+0000 7fe4e932e700 1 -- 172.21.15.92:0/55493145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4d40051c0 msgr2=0x7fe4e4152f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.863+0000 7fe4e932e700 1 --2- 172.21.15.92:0/55493145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4d40051c0 0x7fe4e4152f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.863+0000 7fe4e932e700 1 -- 172.21.15.92:0/55493145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4e415b810 msgr2=0x7fe4e415bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.863+0000 7fe4e932e700 1 --2- 172.21.15.92:0/55493145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4e415b810 0x7fe4e415bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.863+0000 7fe4e932e700 1 -- 172.21.15.92:0/55493145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e40c7660 con 0x7fe4e40c55e0 2023-10-06T10:45:35.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.863+0000 7fe4e8b2d700 1 --2- 172.21.15.92:0/55493145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4d40051c0 0x7fe4e4152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:35.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.863+0000 7fe4e932e700 1 --2- 172.21.15.92:0/55493145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4e40c55e0 0x7fe4e40c5a30 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc000f80 tx=0x7fe4dc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.863+0000 7fe4e0ff9700 1 -- 172.21.15.92:0/55493145 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc01e430 con 0x7fe4e40c55e0 2023-10-06T10:45:35.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.863+0000 7fe4e0ff9700 1 -- 172.21.15.92:0/55493145 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4dc01e5f0 con 0x7fe4e40c55e0 2023-10-06T10:45:35.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.864+0000 7fe4e0ff9700 1 -- 172.21.15.92:0/55493145 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc01e9b0 con 0x7fe4e40c55e0 2023-10-06T10:45:35.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.864+0000 7fe4e9b2f700 1 -- 172.21.15.92:0/55493145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e40c7940 con 0x7fe4e40c55e0 2023-10-06T10:45:35.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.864+0000 7fe4e9b2f700 1 -- 172.21.15.92:0/55493145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e40c7f00 con 0x7fe4e40c55e0 2023-10-06T10:45:35.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.865+0000 7fe4e0ff9700 1 -- 172.21.15.92:0/55493145 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe4dc029460 con 0x7fe4e40c55e0 2023-10-06T10:45:35.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.865+0000 7fe4e0ff9700 1 --2- 172.21.15.92:0/55493145 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe4c8066f30 0x7fe4c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.866+0000 7fe4e0ff9700 1 -- 172.21.15.92:0/55493145 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fe4dc086e50 con 0x7fe4e40c55e0 2023-10-06T10:45:35.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.866+0000 7fe4e8b2d700 1 --2- 172.21.15.92:0/55493145 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe4c8066f30 0x7fe4c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.866+0000 7fe4e8b2d700 1 --2- 172.21.15.92:0/55493145 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe4c8066f30 0x7fe4c80693e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8007c10 tx=0x7fe4d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.867+0000 7fe4f4089b80 1 Processor -- start 2023-10-06T10:45:35.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.867+0000 7fe4f4089b80 1 -- start start 2023-10-06T10:45:35.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.868+0000 7fe4f4089b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563104e629a0 0x563104e62d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.868+0000 7fe4f4089b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104eacc10 0x563104e8b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.868+0000 7fe4f4089b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563104e8beb0 0x563104e5e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.868+0000 7fe4f4089b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563104de2cd0 con 0x563104eacc10 2023-10-06T10:45:35.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.868+0000 7fe4f4089b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563104db9ac0 con 0x563104e8beb0 2023-10-06T10:45:35.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.868+0000 7fe4f4089b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563104db72c0 con 0x563104e629a0 2023-10-06T10:45:35.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.868+0000 7fe4e3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104eacc10 0x563104e8b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.868+0000 7fe4e3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104eacc10 0x563104e8b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55376/0 (socket says 172.21.15.92:55376) 2023-10-06T10:45:35.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.868+0000 7fe4e3fff700 1 -- 172.21.15.92:0/2432753621 learned_addr learned my addr 172.21.15.92:0/2432753621 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:35.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.868+0000 7fe4e932e700 1 --2- 172.21.15.92:0/2432753621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563104e8beb0 0x563104e5e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.868+0000 7fe4e8b2d700 1 --2- 172.21.15.92:0/2432753621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563104e629a0 0x563104e62d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.868+0000 7fe4e3fff700 1 -- 172.21.15.92:0/2432753621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563104e629a0 msgr2=0x563104e62d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.868+0000 7fe4e3fff700 1 --2- 172.21.15.92:0/2432753621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563104e629a0 0x563104e62d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.868+0000 7fe4e3fff700 1 -- 172.21.15.92:0/2432753621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563104e8beb0 msgr2=0x563104e5e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.868+0000 7fe4e3fff700 1 --2- 172.21.15.92:0/2432753621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563104e8beb0 0x563104e5e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.868+0000 7fe4e3fff700 1 -- 172.21.15.92:0/2432753621 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563104c4d2c0 con 0x563104eacc10 2023-10-06T10:45:35.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.868+0000 7fe4e932e700 1 --2- 172.21.15.92:0/2432753621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563104e8beb0 0x563104e5e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:35.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.868+0000 7fe4e3fff700 1 --2- 172.21.15.92:0/2432753621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104eacc10 0x563104e8b970 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009bc0 tx=0x7fe4d0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ec138eac42f24d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.869+0000 7fe4cdffb700 1 -- 172.21.15.92:0/2432753621 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0014bb0 con 0x563104eacc10 2023-10-06T10:45:35.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.869+0000 7fe4cdffb700 1 -- 172.21.15.92:0/2432753621 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d0014d10 con 0x563104eacc10 2023-10-06T10:45:35.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.869+0000 7fe4cdffb700 1 -- 172.21.15.92:0/2432753621 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d001f640 con 0x563104eacc10 2023-10-06T10:45:35.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.869+0000 7fe4f4089b80 1 -- 172.21.15.92:0/2432753621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104eacc10 msgr2=0x563104e8b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.869+0000 7fe4f4089b80 1 --2- 172.21.15.92:0/2432753621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104eacc10 0x563104e8b970 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009bc0 tx=0x7fe4d0008040 comp rx=0 tx=0).stop 2023-10-06T10:45:35.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.869+0000 7fe4f4089b80 1 -- 172.21.15.92:0/2432753621 shutdown_connections 2023-10-06T10:45:35.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.869+0000 7fe4f4089b80 1 --2- 172.21.15.92:0/2432753621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563104e8beb0 0x563104e5e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.869+0000 7fe4f4089b80 1 --2- 172.21.15.92:0/2432753621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563104e629a0 0x563104e62d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.869+0000 7fe4f4089b80 1 --2- 172.21.15.92:0/2432753621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104eacc10 0x563104e8b970 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.869+0000 7fe4f4089b80 1 -- 172.21.15.92:0/2432753621 >> 172.21.15.92:0/2432753621 conn(0x563104e642c0 msgr2=0x563104e64690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.869+0000 7fe4f4089b80 1 -- 172.21.15.92:0/2432753621 shutdown_connections 2023-10-06T10:45:35.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.869+0000 7fe4f4089b80 1 -- 172.21.15.92:0/2432753621 wait complete. 2023-10-06T10:45:35.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.870+0000 7fe4f4089b80 1 -- start start 2023-10-06T10:45:35.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.870+0000 7fe4f4089b80 1 Processor -- start 2023-10-06T10:45:35.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.871+0000 7fe4f4089b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563104e629a0 0x563104ea6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.871+0000 7fe4f4089b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104e8beb0 0x563104e88500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.871+0000 7fe4f4089b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563104eacc10 0x563104e5eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.871+0000 7fe4f4089b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563104db30b0 con 0x563104e8beb0 2023-10-06T10:45:35.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.871+0000 7fe4f4089b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563104d5e8e0 con 0x563104eacc10 2023-10-06T10:45:35.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.871+0000 7fe4f4089b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563104d79ef0 con 0x563104e629a0 2023-10-06T10:45:35.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.871+0000 7fe4f4089b80 10 client.0 _ll_register_callbacks cb 0x563104c4ff80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:35.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.871+0000 7fe4e3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104e8beb0 0x563104e88500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.871+0000 7fe4e3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104e8beb0 0x563104e88500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55382/0 (socket says 172.21.15.92:55382) 2023-10-06T10:45:35.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.871+0000 7fe4e3fff700 1 -- 172.21.15.92:0/1930774357 learned_addr learned my addr 172.21.15.92:0/1930774357 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:35.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.871+0000 7fe4e8b2d700 1 --2- 172.21.15.92:0/1930774357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563104e629a0 0x563104ea6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.871+0000 7fe4e932e700 1 --2- 172.21.15.92:0/1930774357 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563104eacc10 0x563104e5eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.871+0000 7fe4e3fff700 1 -- 172.21.15.92:0/1930774357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563104e629a0 msgr2=0x563104ea6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.871+0000 7fe4e3fff700 1 --2- 172.21.15.92:0/1930774357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563104e629a0 0x563104ea6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.871+0000 7fe4e3fff700 1 -- 172.21.15.92:0/1930774357 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563104eacc10 msgr2=0x563104e5eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.871+0000 7fe4e3fff700 1 --2- 172.21.15.92:0/1930774357 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563104eacc10 0x563104e5eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.871+0000 7fe4e3fff700 1 -- 172.21.15.92:0/1930774357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d00098f0 con 0x563104e8beb0 2023-10-06T10:45:35.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.871+0000 7fe4e3fff700 1 --2- 172.21.15.92:0/1930774357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104e8beb0 0x563104e88500 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0014970 tx=0x7fe4d001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc65ab2b1c1eb0a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.871+0000 7fe4c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:35.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.872+0000 7fe4c2ffd700 1 -- 172.21.15.92:0/1930774357 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0024070 con 0x563104e8beb0 2023-10-06T10:45:35.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.872+0000 7fe4c2ffd700 1 -- 172.21.15.92:0/1930774357 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fe4d001a030 con 0x563104e8beb0 2023-10-06T10:45:35.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.872+0000 7fe4f4089b80 1 -- 172.21.15.92:0/1930774357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563104e576e0 con 0x563104e8beb0 2023-10-06T10:45:35.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.872+0000 7fe4c2ffd700 1 -- 172.21.15.92:0/1930774357 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4d001fbc0 con 0x563104e8beb0 2023-10-06T10:45:35.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.872+0000 7fe4f4089b80 10 client.9270 fetch_fsmap learned FSMap version 110 2023-10-06T10:45:35.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.872+0000 7fe4f4089b80 1 -- 172.21.15.92:0/1930774357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) v3 -- 0x563104bf0030 con 0x563104e8beb0 2023-10-06T10:45:35.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.872+0000 7fe4c2ffd700 1 -- 172.21.15.92:0/1930774357 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563104e576e0 con 0x563104e8beb0 2023-10-06T10:45:35.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.872+0000 7fe4c2ffd700 1 -- 172.21.15.92:0/1930774357 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 110) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fe4d0011070 con 0x563104e8beb0 2023-10-06T10:45:35.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.872+0000 7fe4f4089b80 10 client.9270 fetch_fsmap finished waiting for FSMap version 110 2023-10-06T10:45:35.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.872+0000 7fe4f4089b80 10 client.9270 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-10-06T10:45:35.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.872+0000 7fe4f4089b80 20 client.9270 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:35.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.872+0000 7fe4f4089b80 1 --2- 172.21.15.92:0/1930774357 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x563104e8a060 0x563104ea4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.872+0000 7fe4f4089b80 4 client.9270 mds_command: new command op to 8520 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T10:45:35.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.872+0000 7fe4f4089b80 1 -- 172.21.15.92:0/1930774357 --> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563104e34610 con 0x563104e8a060 2023-10-06T10:45:35.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.872+0000 7fe4e8b2d700 1 --2- 172.21.15.92:0/1930774357 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x563104e8a060 0x563104ea4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.872+0000 7fe4c2ffd700 10 client.9270 ms_handle_connect on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:35.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.872+0000 7fe4e8b2d700 1 --2- 172.21.15.92:0/1930774357 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x563104e8a060 0x563104ea4cd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24f8c8ae7e9bf72a server_cookie=c6f6e0a15852ae1d in_seq=0 out_seq=0 2023-10-06T10:45:35.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.873+0000 7fe4c2ffd700 1 -- 172.21.15.92:0/1930774357 <== mds.0 v2:172.21.15.92:6835/2298348426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x563104e34610 con 0x563104e8a060 2023-10-06T10:45:35.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.873+0000 7fe4c2ffd700 10 client.9270 handle_command_reply: tid=0 2023-10-06T10:45:35.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.873+0000 7fe4f4089b80 1 client.9270 shutdown 2023-10-06T10:45:35.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.873+0000 7fe4f4089b80 1 -- 172.21.15.92:0/1930774357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104e8beb0 msgr2=0x563104e88500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.873+0000 7fe4f4089b80 1 --2- 172.21.15.92:0/1930774357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104e8beb0 0x563104e88500 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0014970 tx=0x7fe4d001e000 comp rx=0 tx=0).stop 2023-10-06T10:45:35.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.873+0000 7fe4f4089b80 1 -- 172.21.15.92:0/1930774357 shutdown_connections 2023-10-06T10:45:35.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.873+0000 7fe4f4089b80 1 --2- 172.21.15.92:0/1930774357 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x563104e8a060 0x563104ea4cd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.873+0000 7fe4f4089b80 1 --2- 172.21.15.92:0/1930774357 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563104eacc10 0x563104e5eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.873+0000 7fe4c2ffd700 0 client.9270 ms_handle_reset on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:35.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.873+0000 7fe4f4089b80 1 --2- 172.21.15.92:0/1930774357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563104e629a0 0x563104ea6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.873+0000 7fe4f4089b80 1 --2- 172.21.15.92:0/1930774357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104e8beb0 0x563104e88500 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.873+0000 7fe4f4089b80 1 -- 172.21.15.92:0/1930774357 >> 172.21.15.92:0/1930774357 conn(0x563104e642c0 msgr2=0x563104e591f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.873+0000 7fe4f4089b80 1 -- 172.21.15.92:0/1930774357 shutdown_connections 2023-10-06T10:45:35.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.873+0000 7fe4f4089b80 1 -- 172.21.15.92:0/1930774357 wait complete. 2023-10-06T10:45:35.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.873+0000 7fe4f4089b80 20 client.9270 trim_cache size 0 max 16384 2023-10-06T10:45:35.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.889+0000 7fe4f4089b80 1 Processor -- start 2023-10-06T10:45:35.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.889+0000 7fe4f4089b80 1 -- start start 2023-10-06T10:45:35.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.890+0000 7fe4f4089b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104e629a0 0x563104e92340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.890+0000 7fe4f4089b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563104e8beb0 0x563104e62d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.890+0000 7fe4f4089b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563104eacc10 0x563104e5ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.890+0000 7fe4f4089b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563104c35e80 con 0x563104e629a0 2023-10-06T10:45:35.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.890+0000 7fe4f4089b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563104e64730 con 0x563104eacc10 2023-10-06T10:45:35.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.890+0000 7fe4f4089b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563104ea80b0 con 0x563104e8beb0 2023-10-06T10:45:35.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.890+0000 7fe4e8b2d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104e629a0 0x563104e92340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.890+0000 7fe4e8b2d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104e629a0 0x563104e92340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55384/0 (socket says 172.21.15.92:55384) 2023-10-06T10:45:35.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.890+0000 7fe4e8b2d700 1 -- 172.21.15.92:0/3807087528 learned_addr learned my addr 172.21.15.92:0/3807087528 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:35.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.890+0000 7fe4e8b2d700 1 -- 172.21.15.92:0/3807087528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563104e8beb0 msgr2=0x563104e62d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.890+0000 7fe4e8b2d700 1 --2- 172.21.15.92:0/3807087528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563104e8beb0 0x563104e62d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.890+0000 7fe4e8b2d700 1 -- 172.21.15.92:0/3807087528 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563104eacc10 msgr2=0x563104e5ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.890+0000 7fe4e8b2d700 1 --2- 172.21.15.92:0/3807087528 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563104eacc10 0x563104e5ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.890+0000 7fe4e8b2d700 1 -- 172.21.15.92:0/3807087528 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563104d817d0 con 0x563104e629a0 2023-10-06T10:45:35.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.890+0000 7fe4e8b2d700 1 --2- 172.21.15.92:0/3807087528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104e629a0 0x563104e92340 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8000c00 tx=0x7fe4d8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c973c55579752a58 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.890+0000 7fe4ce7fc700 1 -- 172.21.15.92:0/3807087528 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8025970 con 0x563104e629a0 2023-10-06T10:45:35.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.891+0000 7fe4ce7fc700 1 -- 172.21.15.92:0/3807087528 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d8025ad0 con 0x563104e629a0 2023-10-06T10:45:35.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.891+0000 7fe4ce7fc700 1 -- 172.21.15.92:0/3807087528 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8025c60 con 0x563104e629a0 2023-10-06T10:45:35.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.891+0000 7fe4f4089b80 1 -- 172.21.15.92:0/3807087528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104e629a0 msgr2=0x563104e92340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.891+0000 7fe4f4089b80 1 --2- 172.21.15.92:0/3807087528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104e629a0 0x563104e92340 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8000c00 tx=0x7fe4d8015000 comp rx=0 tx=0).stop 2023-10-06T10:45:35.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.891+0000 7fe4f4089b80 1 -- 172.21.15.92:0/3807087528 shutdown_connections 2023-10-06T10:45:35.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.891+0000 7fe4f4089b80 1 --2- 172.21.15.92:0/3807087528 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563104eacc10 0x563104e5ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.891+0000 7fe4f4089b80 1 --2- 172.21.15.92:0/3807087528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104e629a0 0x563104e92340 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.891+0000 7fe4f4089b80 1 --2- 172.21.15.92:0/3807087528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563104e8beb0 0x563104e62d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.891+0000 7fe4f4089b80 1 -- 172.21.15.92:0/3807087528 >> 172.21.15.92:0/3807087528 conn(0x563104e642c0 msgr2=0x563104ea84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.891+0000 7fe4f4089b80 1 -- 172.21.15.92:0/3807087528 shutdown_connections 2023-10-06T10:45:35.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.891+0000 7fe4f4089b80 1 -- 172.21.15.92:0/3807087528 wait complete. 2023-10-06T10:45:35.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.892+0000 7fe4f4089b80 1 -- start start 2023-10-06T10:45:35.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.892+0000 7fe4f4089b80 1 Processor -- start 2023-10-06T10:45:35.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4f4089b80 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563104e629a0 0x563104ebbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4f4089b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104e8beb0 0x563104e5ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4f4089b80 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563104eacc10 0x563104e591f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4f4089b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563104e5f2b0 con 0x563104e8beb0 2023-10-06T10:45:35.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4f4089b80 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563104ef6370 con 0x563104eacc10 2023-10-06T10:45:35.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4f4089b80 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563104ea01b0 con 0x563104e629a0 2023-10-06T10:45:35.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4f4089b80 10 client.0 _ll_register_callbacks cb 0x563104c4ff80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T10:45:35.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4e3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104e8beb0 0x563104e5ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4e3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104e8beb0 0x563104e5ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55388/0 (socket says 172.21.15.92:55388) 2023-10-06T10:45:35.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4e3fff700 1 -- 172.21.15.92:0/4096269320 learned_addr learned my addr 172.21.15.92:0/4096269320 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:35.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4e932e700 1 --2- 172.21.15.92:0/4096269320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563104eacc10 0x563104e591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4e8b2d700 1 --2- 172.21.15.92:0/4096269320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563104e629a0 0x563104ebbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4e3fff700 1 -- 172.21.15.92:0/4096269320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563104e629a0 msgr2=0x563104ebbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4e3fff700 1 --2- 172.21.15.92:0/4096269320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563104e629a0 0x563104ebbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4e3fff700 1 -- 172.21.15.92:0/4096269320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563104eacc10 msgr2=0x563104e591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:35.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4e3fff700 1 --2- 172.21.15.92:0/4096269320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563104eacc10 0x563104e591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4e3fff700 1 -- 172.21.15.92:0/4096269320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d800f040 con 0x563104e8beb0 2023-10-06T10:45:35.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4e3fff700 1 --2- 172.21.15.92:0/4096269320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104e8beb0 0x563104e5ea40 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0014760 tx=0x7fe4d0021050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5d915e08717cbf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:35.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T10:45:35.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4c2ffd700 1 -- 172.21.15.92:0/4096269320 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0024040 con 0x563104e8beb0 2023-10-06T10:45:35.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4c2ffd700 1 -- 172.21.15.92:0/4096269320 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fe4d001fe40 con 0x563104e8beb0 2023-10-06T10:45:35.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4f4089b80 1 -- 172.21.15.92:0/4096269320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4d0014d10 con 0x563104e8beb0 2023-10-06T10:45:35.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.893+0000 7fe4c2ffd700 1 -- 172.21.15.92:0/4096269320 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4d0014bb0 con 0x563104e8beb0 2023-10-06T10:45:35.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.894+0000 7fe4f4089b80 10 client.9276 fetch_fsmap learned FSMap version 110 2023-10-06T10:45:35.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.894+0000 7fe4f4089b80 1 -- 172.21.15.92:0/4096269320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) v3 -- 0x563104d684c0 con 0x563104e8beb0 2023-10-06T10:45:35.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.894+0000 7fe4c2ffd700 1 -- 172.21.15.92:0/4096269320 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d0014d10 con 0x563104e8beb0 2023-10-06T10:45:35.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.894+0000 7fe4c2ffd700 1 -- 172.21.15.92:0/4096269320 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 110) v1 ==== 3140+0+0 (secure 0 0 0) 0x7fe4d00286d0 con 0x563104e8beb0 2023-10-06T10:45:35.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.894+0000 7fe4f4089b80 10 client.9276 fetch_fsmap finished waiting for FSMap version 110 2023-10-06T10:45:35.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.894+0000 7fe4f4089b80 10 client.9276 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-10-06T10:45:35.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.894+0000 7fe4f4089b80 20 client.9276 populate_metadata read hostname 'smithi092' 2023-10-06T10:45:35.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.894+0000 7fe4f4089b80 1 --2- 172.21.15.92:0/4096269320 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x563104e59730 0x563104e88e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:35.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.894+0000 7fe4f4089b80 4 client.9276 mds_command: new command op to 8520 tid=0[{"prefix": "scrub status"}] 2023-10-06T10:45:35.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.894+0000 7fe4f4089b80 1 -- 172.21.15.92:0/4096269320 --> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563104d55e50 con 0x563104e59730 2023-10-06T10:45:35.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.894+0000 7fe4e8b2d700 1 --2- 172.21.15.92:0/4096269320 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x563104e59730 0x563104e88e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:35.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.894+0000 7fe4c2ffd700 10 client.9276 ms_handle_connect on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:35.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.894+0000 7fe4e8b2d700 1 --2- 172.21.15.92:0/4096269320 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x563104e59730 0x563104e88e90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2abc5ca03dd95562 server_cookie=33349ed43a7d5ec0 in_seq=0 out_seq=0 2023-10-06T10:45:35.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.894+0000 7fe4c2ffd700 1 -- 172.21.15.92:0/4096269320 <== mds.0 v2:172.21.15.92:6835/2298348426 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x563104d55e50 con 0x563104e59730 2023-10-06T10:45:35.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.894+0000 7fe4c2ffd700 10 client.9276 handle_command_reply: tid=0 2023-10-06T10:45:35.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.894+0000 7fe4f4089b80 1 client.9276 shutdown 2023-10-06T10:45:35.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.895+0000 7fe4f4089b80 1 -- 172.21.15.92:0/4096269320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104e8beb0 msgr2=0x563104e5ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.895+0000 7fe4f4089b80 1 --2- 172.21.15.92:0/4096269320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104e8beb0 0x563104e5ea40 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0014760 tx=0x7fe4d0021050 comp rx=0 tx=0).stop 2023-10-06T10:45:35.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.895+0000 7fe4f4089b80 1 -- 172.21.15.92:0/4096269320 shutdown_connections 2023-10-06T10:45:35.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.895+0000 7fe4f4089b80 1 --2- 172.21.15.92:0/4096269320 >> [v2:172.21.15.92:6835/2298348426,v1:172.21.15.92:6837/2298348426] conn(0x563104e59730 0x563104e88e90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.895+0000 7fe4f4089b80 1 --2- 172.21.15.92:0/4096269320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563104eacc10 0x563104e591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.895+0000 7fe4f4089b80 1 --2- 172.21.15.92:0/4096269320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563104e629a0 0x563104ebbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.895+0000 7fe4f4089b80 1 --2- 172.21.15.92:0/4096269320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563104e8beb0 0x563104e5ea40 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.895+0000 7fe4c2ffd700 0 client.9276 ms_handle_reset on v2:172.21.15.92:6835/2298348426 2023-10-06T10:45:35.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.895+0000 7fe4f4089b80 1 -- 172.21.15.92:0/4096269320 >> 172.21.15.92:0/4096269320 conn(0x563104e642c0 msgr2=0x563104e92300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.895+0000 7fe4f4089b80 1 -- 172.21.15.92:0/4096269320 shutdown_connections 2023-10-06T10:45:35.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.895+0000 7fe4f4089b80 1 -- 172.21.15.92:0/4096269320 wait complete. 2023-10-06T10:45:35.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.895+0000 7fe4f4089b80 20 client.9276 trim_cache size 0 max 16384 2023-10-06T10:45:35.903 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:35.903 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T10:45:35.903 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-10-06T10:45:35.903 INFO:teuthology.orchestra.run.smithi092.stdout: "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8": { 2023-10-06T10:45:35.903 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/d1/d2/d3", 2023-10-06T10:45:35.903 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "b16d26e0-23aa-4424-8d5a-ff9c6b9c47d8", 2023-10-06T10:45:35.904 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2023-10-06T10:45:35.904 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:35.904 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T10:45:35.904 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:35.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.895+0000 7fe4e9b2f700 1 -- 172.21.15.92:0/55493145 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe4c8066f30 msgr2=0x7fe4c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.895+0000 7fe4e9b2f700 1 --2- 172.21.15.92:0/55493145 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe4c8066f30 0x7fe4c80693e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8007c10 tx=0x7fe4d8009040 comp rx=0 tx=0).stop 2023-10-06T10:45:35.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.895+0000 7fe4e9b2f700 1 -- 172.21.15.92:0/55493145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4e40c55e0 msgr2=0x7fe4e40c5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:35.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.895+0000 7fe4e9b2f700 1 --2- 172.21.15.92:0/55493145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4e40c55e0 0x7fe4e40c5a30 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc000f80 tx=0x7fe4dc016000 comp rx=0 tx=0).stop 2023-10-06T10:45:35.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.895+0000 7fe4e9b2f700 1 -- 172.21.15.92:0/55493145 shutdown_connections 2023-10-06T10:45:35.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.895+0000 7fe4e9b2f700 1 --2- 172.21.15.92:0/55493145 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fe4c8066f30 0x7fe4c80693e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.895+0000 7fe4e9b2f700 1 --2- 172.21.15.92:0/55493145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4e40c55e0 0x7fe4e40c5a30 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.895+0000 7fe4e9b2f700 1 --2- 172.21.15.92:0/55493145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4d40051c0 0x7fe4e4152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.895+0000 7fe4e9b2f700 1 --2- 172.21.15.92:0/55493145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4e415b810 0x7fe4e415bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:35.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.895+0000 7fe4e9b2f700 1 -- 172.21.15.92:0/55493145 >> 172.21.15.92:0/55493145 conn(0x7fe4e4099520 msgr2=0x7fe4e415d120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:35.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.896+0000 7fe4e9b2f700 1 -- 172.21.15.92:0/55493145 shutdown_connections 2023-10-06T10:45:35.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:35.896+0000 7fe4e9b2f700 1 -- 172.21.15.92:0/55493145 wait complete. 2023-10-06T10:45:35.908 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T10:45:35.908 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:45:36.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.022+0000 7f9b04425700 1 Processor -- start 2023-10-06T10:45:36.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.022+0000 7f9b04425700 1 -- start start 2023-10-06T10:45:36.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.023+0000 7f9b04425700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9afc0b0760 0x7f9afc0ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:36.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.023+0000 7f9b04425700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9afc0b0c60 con 0x7f9afc159560 2023-10-06T10:45:36.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.023+0000 7f9b04425700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9afc0b0f40 con 0x7f9afc0b49a0 2023-10-06T10:45:36.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.023+0000 7f9b04425700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9afc0ab350 con 0x7f9afc0b0760 2023-10-06T10:45:36.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.023+0000 7f9b021c1700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9afc159560 0x7f9afc0aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45596/0 (socket says 172.21.15.92:45596) 2023-10-06T10:45:36.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.023+0000 7f9b021c1700 1 -- 172.21.15.92:0/46478676 learned_addr learned my addr 172.21.15.92:0/46478676 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:36.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.023+0000 7f9b011bf700 1 -- 172.21.15.92:0/46478676 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 168853209 0 0) 0x7f9afc0b0c60 con 0x7f9afc159560 2023-10-06T10:45:36.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.023+0000 7f9b029c2700 1 --2- 172.21.15.92:0/46478676 >> v2:172.21.15.124:3300/0 conn(0x7f9afc0b0760 0x7f9afc0ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:36.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.023+0000 7f9b011bf700 1 -- 172.21.15.92:0/46478676 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ae8003570 con 0x7f9afc159560 2023-10-06T10:45:36.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.023+0000 7f9b011bf700 1 -- 172.21.15.92:0/46478676 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3497180207 0 0) 0x7f9ae8003570 con 0x7f9afc159560 2023-10-06T10:45:36.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.023+0000 7f9b011bf700 1 -- 172.21.15.92:0/46478676 >> v2:172.21.15.124:3300/0 conn(0x7f9afc0b0760 msgr2=0x7f9afc0ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:36.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.023+0000 7f9b011bf700 1 --2- 172.21.15.92:0/46478676 >> v2:172.21.15.124:3300/0 conn(0x7f9afc0b0760 0x7f9afc0ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:36.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.023+0000 7f9b011bf700 1 -- 172.21.15.92:0/46478676 >> v1:172.21.15.124:6789/0 conn(0x7f9afc0b49a0 legacy=0x7f9afc0aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:36.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.023+0000 7f9b011bf700 1 -- 172.21.15.92:0/46478676 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc0b1220 con 0x7f9afc159560 2023-10-06T10:45:36.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.023+0000 7f9b011bf700 1 -- 172.21.15.92:0/46478676 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f9af40015f0 con 0x7f9afc159560 2023-10-06T10:45:36.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.023+0000 7f9b011bf700 1 -- 172.21.15.92:0/46478676 >> v1:172.21.15.92:6789/0 conn(0x7f9afc159560 legacy=0x7f9afc0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:36.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.023+0000 7f9b011bf700 1 --2- 172.21.15.92:0/46478676 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ae8003a60 0x7f9ae8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:36.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.023+0000 7f9b011bf700 1 --2- 172.21.15.92:0/46478676 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ae8004810 0x7f9ae8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:36.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.023+0000 7f9b011bf700 1 --2- 172.21.15.92:0/46478676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae80051c0 0x7f9ae800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:36.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.023+0000 7f9b011bf700 1 -- 172.21.15.92:0/46478676 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc0ab350 con 0x7f9ae80051c0 2023-10-06T10:45:36.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.023+0000 7f9b011bf700 1 -- 172.21.15.92:0/46478676 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae800c0d0 con 0x7f9ae8004810 2023-10-06T10:45:36.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.023+0000 7f9b011bf700 1 -- 172.21.15.92:0/46478676 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae800c290 con 0x7f9ae8003a60 2023-10-06T10:45:36.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.024+0000 7f9b029c2700 1 --2- 172.21.15.92:0/46478676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae80051c0 0x7f9ae800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:36.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.024+0000 7f9b021c1700 1 --2- 172.21.15.92:0/46478676 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ae8003a60 0x7f9ae8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:36.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.024+0000 7f9b019c0700 1 --2- 172.21.15.92:0/46478676 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ae8004810 0x7f9ae8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:36.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.024+0000 7f9b029c2700 1 -- 172.21.15.92:0/46478676 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ae8003a60 msgr2=0x7f9ae8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:36.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.024+0000 7f9b029c2700 1 --2- 172.21.15.92:0/46478676 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ae8003a60 0x7f9ae8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:36.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.024+0000 7f9b029c2700 1 -- 172.21.15.92:0/46478676 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ae8004810 msgr2=0x7f9ae8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:36.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.024+0000 7f9b029c2700 1 --2- 172.21.15.92:0/46478676 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ae8004810 0x7f9ae8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:36.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.024+0000 7f9b029c2700 1 -- 172.21.15.92:0/46478676 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae8003570 con 0x7f9ae80051c0 2023-10-06T10:45:36.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.024+0000 7f9b029c2700 1 -- 172.21.15.92:0/46478676 reap_dead start 2023-10-06T10:45:36.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.024+0000 7f9b029c2700 1 --2- 172.21.15.92:0/46478676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae80051c0 0x7f9ae800ba60 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f9af800e210 tx=0x7f9af800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a1076147557c1f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:36.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.024+0000 7f9b011bf700 1 -- 172.21.15.92:0/46478676 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af800eac0 con 0x7f9ae80051c0 2023-10-06T10:45:36.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.024+0000 7f9b011bf700 1 -- 172.21.15.92:0/46478676 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af800f040 con 0x7f9ae80051c0 2023-10-06T10:45:36.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.024+0000 7f9b011bf700 1 -- 172.21.15.92:0/46478676 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af800ee70 con 0x7f9ae80051c0 2023-10-06T10:45:36.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.025+0000 7f9b04425700 1 -- 172.21.15.92:0/46478676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae80051c0 msgr2=0x7f9ae800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:36.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.025+0000 7f9b04425700 1 --2- 172.21.15.92:0/46478676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae80051c0 0x7f9ae800ba60 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f9af800e210 tx=0x7f9af800b040 comp rx=0 tx=0).stop 2023-10-06T10:45:36.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.025+0000 7f9b04425700 1 -- 172.21.15.92:0/46478676 shutdown_connections 2023-10-06T10:45:36.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.025+0000 7f9b04425700 1 --2- 172.21.15.92:0/46478676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae80051c0 0x7f9ae800ba60 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:36.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.025+0000 7f9b04425700 1 -- 172.21.15.92:0/46478676 >> 172.21.15.92:0/46478676 conn(0x7f9afc0997f0 msgr2=0x7f9afc06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:36.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.025+0000 7f9b04425700 1 -- 172.21.15.92:0/46478676 shutdown_connections 2023-10-06T10:45:36.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.025+0000 7f9b04425700 1 -- 172.21.15.92:0/46478676 wait complete. 2023-10-06T10:45:36.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.026+0000 7f9b04425700 1 Processor -- start 2023-10-06T10:45:36.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.026+0000 7f9b04425700 1 -- start start 2023-10-06T10:45:36.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.027+0000 7f9b04425700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ae80051c0 0x7f9afc150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:36.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.027+0000 7f9b04425700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9afc153190 0x7f9afc169540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:36.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.027+0000 7f9b04425700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9afc169a80 0x7f9afc0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:36.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.027+0000 7f9b04425700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc140a50 con 0x7f9afc153190 2023-10-06T10:45:36.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.027+0000 7f9b04425700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc153ca0 con 0x7f9afc169a80 2023-10-06T10:45:36.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.027+0000 7f9b04425700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc1536e0 con 0x7f9ae80051c0 2023-10-06T10:45:36.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.027+0000 7f9b019c0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9afc153190 0x7f9afc169540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:36.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.027+0000 7f9b019c0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9afc153190 0x7f9afc169540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55402/0 (socket says 172.21.15.92:55402) 2023-10-06T10:45:36.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.027+0000 7f9b019c0700 1 -- 172.21.15.92:0/378283431 learned_addr learned my addr 172.21.15.92:0/378283431 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:36.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.027+0000 7f9b021c1700 1 --2- 172.21.15.92:0/378283431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ae80051c0 0x7f9afc150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:36.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.027+0000 7f9b029c2700 1 --2- 172.21.15.92:0/378283431 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9afc169a80 0x7f9afc0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:36.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.027+0000 7f9b019c0700 1 -- 172.21.15.92:0/378283431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ae80051c0 msgr2=0x7f9afc150b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:36.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.027+0000 7f9b019c0700 1 --2- 172.21.15.92:0/378283431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ae80051c0 0x7f9afc150b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:36.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.027+0000 7f9b019c0700 1 -- 172.21.15.92:0/378283431 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9afc169a80 msgr2=0x7f9afc0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:36.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.027+0000 7f9b019c0700 1 --2- 172.21.15.92:0/378283431 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9afc169a80 0x7f9afc0cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:36.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.027+0000 7f9b019c0700 1 -- 172.21.15.92:0/378283431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc1539c0 con 0x7f9afc153190 2023-10-06T10:45:36.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.027+0000 7f9b019c0700 1 --2- 172.21.15.92:0/378283431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9afc153190 0x7f9afc169540 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f9aec007a80 tx=0x7f9aec007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:36.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.027+0000 7f9af27fc700 1 -- 172.21.15.92:0/378283431 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec002e30 con 0x7f9afc153190 2023-10-06T10:45:36.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.027+0000 7f9af27fc700 1 -- 172.21.15.92:0/378283431 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aec00f8d0 con 0x7f9afc153190 2023-10-06T10:45:36.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.027+0000 7f9b04425700 1 -- 172.21.15.92:0/378283431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9afc1406e0 con 0x7f9afc153190 2023-10-06T10:45:36.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.027+0000 7f9af27fc700 1 -- 172.21.15.92:0/378283431 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec00fc90 con 0x7f9afc153190 2023-10-06T10:45:36.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.027+0000 7f9b04425700 1 -- 172.21.15.92:0/378283431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc0cdd30 con 0x7f9afc153190 2023-10-06T10:45:36.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.028+0000 7f9b04425700 1 -- 172.21.15.92:0/378283431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9afc064a90 con 0x7f9afc153190 2023-10-06T10:45:36.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.029+0000 7f9af27fc700 1 -- 172.21.15.92:0/378283431 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9aec01c400 con 0x7f9afc153190 2023-10-06T10:45:36.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.029+0000 7f9af27fc700 1 --2- 172.21.15.92:0/378283431 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9ae0066f30 0x7f9ae00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:36.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.029+0000 7f9b021c1700 1 --2- 172.21.15.92:0/378283431 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9ae0066f30 0x7f9ae00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:36.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.029+0000 7f9af27fc700 1 -- 172.21.15.92:0/378283431 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f9aec084930 con 0x7f9afc153190 2023-10-06T10:45:36.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.030+0000 7f9b021c1700 1 --2- 172.21.15.92:0/378283431 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9ae0066f30 0x7f9ae00693e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9af4002e70 tx=0x7f9af4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:36.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.033+0000 7f9af27fc700 1 -- 172.21.15.92:0/378283431 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9aec0518d0 con 0x7f9afc153190 2023-10-06T10:45:36.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.258+0000 7f9b04425700 1 -- 172.21.15.92:0/378283431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9afc0b1500 con 0x7f9afc153190 2023-10-06T10:45:36.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.258+0000 7f9af27fc700 1 -- 172.21.15.92:0/378283431 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4646 (secure 0 0 0) 0x7f9aec054ef0 con 0x7f9afc153190 2023-10-06T10:45:36.259 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:36.259 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":110,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2016052382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2016052382},{"type":"v1","addr":"172.21.15.124:6834","nonce":2016052382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:44:51.106486+0000","modified":"2023-10-06T10:45:34.353086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8520,"mds_1":8517,"mds_2":6719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6719":{"gid":6719,"name":"d","rank":2,"incarnation":104,"state":"up:active","state_seq":13,"addr":"172.21.15.124:6835/2307521923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2307521923},{"type":"v1","addr":"172.21.15.124:6835","nonce":2307521923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8517":{"gid":8517,"name":"a","rank":1,"incarnation":102,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6836/1979826278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979826278},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979826278}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6837/2298348426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2298348426},{"type":"v1","addr":"172.21.15.92:6837","nonce":2298348426}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T10:45:36.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.260+0000 7f9b04425700 1 -- 172.21.15.92:0/378283431 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9ae0066f30 msgr2=0x7f9ae00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:36.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.260+0000 7f9b04425700 1 --2- 172.21.15.92:0/378283431 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9ae0066f30 0x7f9ae00693e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9af4002e70 tx=0x7f9af4009040 comp rx=0 tx=0).stop 2023-10-06T10:45:36.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.260+0000 7f9b04425700 1 -- 172.21.15.92:0/378283431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9afc153190 msgr2=0x7f9afc169540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:36.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.260+0000 7f9b04425700 1 --2- 172.21.15.92:0/378283431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9afc153190 0x7f9afc169540 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f9aec007a80 tx=0x7f9aec007d30 comp rx=0 tx=0).stop 2023-10-06T10:45:36.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.260+0000 7f9b04425700 1 -- 172.21.15.92:0/378283431 shutdown_connections 2023-10-06T10:45:36.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.260+0000 7f9b04425700 1 --2- 172.21.15.92:0/378283431 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9ae0066f30 0x7f9ae00693e0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:36.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.260+0000 7f9b04425700 1 --2- 172.21.15.92:0/378283431 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9afc169a80 0x7f9afc0cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:36.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.260+0000 7f9b04425700 1 --2- 172.21.15.92:0/378283431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ae80051c0 0x7f9afc150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:36.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.260+0000 7f9b04425700 1 --2- 172.21.15.92:0/378283431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9afc153190 0x7f9afc169540 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:36.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.260+0000 7f9b04425700 1 -- 172.21.15.92:0/378283431 >> 172.21.15.92:0/378283431 conn(0x7f9afc0997f0 msgr2=0x7f9afc068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:36.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.260+0000 7f9b04425700 1 -- 172.21.15.92:0/378283431 shutdown_connections 2023-10-06T10:45:36.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.260+0000 7f9b04425700 1 -- 172.21.15.92:0/378283431 wait complete. 2023-10-06T10:45:36.263 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 110 2023-10-06T10:45:36.273 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2023-10-06T10:45:36.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:45:36.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.388+0000 7f7665c39700 1 Processor -- start 2023-10-06T10:45:36.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.388+0000 7f7665c39700 1 -- start start 2023-10-06T10:45:36.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.389+0000 7f7665c39700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f76600b49a0 0x7f76600aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:36.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.389+0000 7f7665c39700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76600b0760 0x7f76600b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:36.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.389+0000 7f7665c39700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76600b0c60 con 0x7f7660159560 2023-10-06T10:45:36.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.389+0000 7f7665c39700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76600b0f40 con 0x7f76600b49a0 2023-10-06T10:45:36.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.389+0000 7f7665c39700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76600b1220 con 0x7f76600b0760 2023-10-06T10:45:36.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.389+0000 7f765ffff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76600b0760 0x7f76600b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:36.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.389+0000 7f765effd700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f76600b49a0 0x7f76600aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:36.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.389+0000 7f765ffff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76600b0760 0x7f76600b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:48412/0 (socket says 172.21.15.92:48412) 2023-10-06T10:45:36.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.389+0000 7f765f7fe700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f7660159560 0x7f76600aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:40794/0 (socket says 172.21.15.92:40794) 2023-10-06T10:45:36.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.389+0000 7f765ffff700 1 -- 172.21.15.92:0/978031224 learned_addr learned my addr 172.21.15.92:0/978031224 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:36.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.390+0000 7f765ffff700 1 -- 172.21.15.92:0/978031224 >> v2:172.21.15.124:3300/0 conn(0x7f76600b49a0 msgr2=0x7f76600aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:36.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.390+0000 7f765ffff700 1 --2- 172.21.15.92:0/978031224 >> v2:172.21.15.124:3300/0 conn(0x7f76600b49a0 0x7f76600aac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:36.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.390+0000 7f765ffff700 1 -- 172.21.15.92:0/978031224 >> v1:172.21.15.124:6789/0 conn(0x7f7660159560 legacy=0x7f76600aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:36.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.390+0000 7f765ffff700 1 -- 172.21.15.92:0/978031224 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76600b1500 con 0x7f76600b0760 2023-10-06T10:45:36.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.390+0000 7f765effd700 1 --2- 172.21.15.92:0/978031224 >> v2:172.21.15.124:3300/0 conn(0x7f76600b49a0 0x7f76600aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:45:36.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.390+0000 7f765ffff700 1 --2- 172.21.15.92:0/978031224 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76600b0760 0x7f76600b1bc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7658009f50 tx=0x7f7658009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bccc261a0a85205d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:36.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.390+0000 7f765e7fc700 1 -- 172.21.15.92:0/978031224 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765800e070 con 0x7f76600b0760 2023-10-06T10:45:36.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.390+0000 7f765e7fc700 1 -- 172.21.15.92:0/978031224 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765800b9c0 con 0x7f76600b0760 2023-10-06T10:45:36.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.390+0000 7f765e7fc700 1 -- 172.21.15.92:0/978031224 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76580092e0 con 0x7f76600b0760 2023-10-06T10:45:36.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.391+0000 7f7665c39700 1 -- 172.21.15.92:0/978031224 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76600b0760 msgr2=0x7f76600b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:36.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.391+0000 7f7665c39700 1 --2- 172.21.15.92:0/978031224 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76600b0760 0x7f76600b1bc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7658009f50 tx=0x7f7658009840 comp rx=0 tx=0).stop 2023-10-06T10:45:36.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.391+0000 7f7665c39700 1 -- 172.21.15.92:0/978031224 shutdown_connections 2023-10-06T10:45:36.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.391+0000 7f7665c39700 1 --2- 172.21.15.92:0/978031224 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76600b0760 0x7f76600b1bc0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:36.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.391+0000 7f7665c39700 1 --2- 172.21.15.92:0/978031224 >> v2:172.21.15.124:3300/0 conn(0x7f76600b49a0 0x7f76600aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:36.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.391+0000 7f7665c39700 1 -- 172.21.15.92:0/978031224 >> 172.21.15.92:0/978031224 conn(0x7f76600997f0 msgr2=0x7f766006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:36.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.391+0000 7f7665c39700 1 -- 172.21.15.92:0/978031224 shutdown_connections 2023-10-06T10:45:36.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.391+0000 7f7665c39700 1 -- 172.21.15.92:0/978031224 wait complete. 2023-10-06T10:45:36.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.392+0000 7f7665c39700 1 Processor -- start 2023-10-06T10:45:36.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.392+0000 7f7665c39700 1 -- start start 2023-10-06T10:45:36.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.393+0000 7f7665c39700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76600b0760 0x7f76601575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:36.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.393+0000 7f7665c39700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76600b49a0 0x7f766009c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:36.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.393+0000 7f7665c39700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7660159560 0x7f766009cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.393+0000 7f7665c39700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766009de90 con 0x7f76600b0760 2023-10-06T10:45:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.393+0000 7f7665c39700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76601580b0 con 0x7f76600b49a0 2023-10-06T10:45:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.393+0000 7f7665c39700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7660147420 con 0x7f7660159560 2023-10-06T10:45:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.393+0000 7f765f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76600b0760 0x7f76601575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.393+0000 7f765f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76600b0760 0x7f76601575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55404/0 (socket says 172.21.15.92:55404) 2023-10-06T10:45:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.393+0000 7f765f7fe700 1 -- 172.21.15.92:0/3850959765 learned_addr learned my addr 172.21.15.92:0/3850959765 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.393+0000 7f765ffff700 1 --2- 172.21.15.92:0/3850959765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7660159560 0x7f766009cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.393+0000 7f765effd700 1 --2- 172.21.15.92:0/3850959765 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76600b49a0 0x7f766009c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.393+0000 7f765f7fe700 1 -- 172.21.15.92:0/3850959765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7660159560 msgr2=0x7f766009cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.393+0000 7f765f7fe700 1 --2- 172.21.15.92:0/3850959765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7660159560 0x7f766009cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.393+0000 7f765f7fe700 1 -- 172.21.15.92:0/3850959765 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76600b49a0 msgr2=0x7f766009c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.393+0000 7f765f7fe700 1 --2- 172.21.15.92:0/3850959765 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76600b49a0 0x7f766009c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.393+0000 7f765f7fe700 1 -- 172.21.15.92:0/3850959765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7654009420 con 0x7f76600b0760 2023-10-06T10:45:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.393+0000 7f765f7fe700 1 --2- 172.21.15.92:0/3850959765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76600b0760 0x7f76601575e0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f765000c4a0 tx=0x7f765000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.393+0000 7f76477fe700 1 -- 172.21.15.92:0/3850959765 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7650005ac0 con 0x7f76600b0760 2023-10-06T10:45:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.394+0000 7f76477fe700 1 -- 172.21.15.92:0/3850959765 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7650005c40 con 0x7f76600b0760 2023-10-06T10:45:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.394+0000 7f7665c39700 1 -- 172.21.15.92:0/3850959765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7658009ae0 con 0x7f76600b0760 2023-10-06T10:45:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.394+0000 7f76477fe700 1 -- 172.21.15.92:0/3850959765 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76500070d0 con 0x7f76600b0760 2023-10-06T10:45:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.394+0000 7f7665c39700 1 -- 172.21.15.92:0/3850959765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f766009d210 con 0x7f76600b0760 2023-10-06T10:45:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.394+0000 7f7665c39700 1 -- 172.21.15.92:0/3850959765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7660064a90 con 0x7f76600b0760 2023-10-06T10:45:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.395+0000 7f76477fe700 1 -- 172.21.15.92:0/3850959765 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7650007440 con 0x7f76600b0760 2023-10-06T10:45:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.395+0000 7f76477fe700 1 --2- 172.21.15.92:0/3850959765 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f764c066f30 0x7f764c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.395+0000 7f76477fe700 1 -- 172.21.15.92:0/3850959765 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f7650083bc0 con 0x7f76600b0760 2023-10-06T10:45:36.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.395+0000 7f765effd700 1 --2- 172.21.15.92:0/3850959765 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f764c066f30 0x7f764c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:36.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.396+0000 7f765effd700 1 --2- 172.21.15.92:0/3850959765 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f764c066f30 0x7f764c0693e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f76600ccda0 tx=0x7f76540091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:36.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.399+0000 7f76477fe700 1 -- 172.21.15.92:0/3850959765 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7650050b50 con 0x7f76600b0760 2023-10-06T10:45:36.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.658+0000 7f7665c39700 1 -- 172.21.15.92:0/3850959765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f766004bb40 con 0x7f76600b0760 2023-10-06T10:45:36.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.659+0000 7f76477fe700 1 -- 172.21.15.92:0/3850959765 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1978 (secure 0 0 0) 0x7f7650017080 con 0x7f76600b0760 2023-10-06T10:45:36.660 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:36.660 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":458,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":704416,"bytes_used":2455654400,"bytes_avail":770638458880,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":7422,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":110,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6719},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":8517},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8520}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2023-10-06T10:45:36.178415+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2023-10-06T10:45:36.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.661+0000 7f7665c39700 1 -- 172.21.15.92:0/3850959765 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f764c066f30 msgr2=0x7f764c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:36.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.661+0000 7f7665c39700 1 --2- 172.21.15.92:0/3850959765 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f764c066f30 0x7f764c0693e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f76600ccda0 tx=0x7f76540091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:36.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.661+0000 7f7665c39700 1 -- 172.21.15.92:0/3850959765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76600b0760 msgr2=0x7f76601575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:36.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.661+0000 7f7665c39700 1 --2- 172.21.15.92:0/3850959765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76600b0760 0x7f76601575e0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f765000c4a0 tx=0x7f765000c7b0 comp rx=0 tx=0).stop 2023-10-06T10:45:36.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.661+0000 7f7665c39700 1 -- 172.21.15.92:0/3850959765 shutdown_connections 2023-10-06T10:45:36.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.661+0000 7f7665c39700 1 --2- 172.21.15.92:0/3850959765 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f764c066f30 0x7f764c0693e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:36.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.661+0000 7f7665c39700 1 --2- 172.21.15.92:0/3850959765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7660159560 0x7f766009cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:36.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.661+0000 7f7665c39700 1 --2- 172.21.15.92:0/3850959765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76600b0760 0x7f76601575e0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:36.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.661+0000 7f7665c39700 1 --2- 172.21.15.92:0/3850959765 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76600b49a0 0x7f766009c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:36.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.661+0000 7f7665c39700 1 -- 172.21.15.92:0/3850959765 >> 172.21.15.92:0/3850959765 conn(0x7f76600997f0 msgr2=0x7f76600b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:36.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.661+0000 7f7665c39700 1 -- 172.21.15.92:0/3850959765 shutdown_connections 2023-10-06T10:45:36.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:36.661+0000 7f7665c39700 1 -- 172.21.15.92:0/3850959765 wait complete. 2023-10-06T10:45:36.673 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2023-10-06T10:45:37.675 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:45:37.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.795+0000 7ff6ecb3e700 1 Processor -- start 2023-10-06T10:45:37.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.795+0000 7ff6ecb3e700 1 -- start start 2023-10-06T10:45:37.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.796+0000 7ff6ecb3e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e80b49a0 0x7ff6e80aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:37.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.796+0000 7ff6ecb3e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff6e80b0760 0x7ff6e80b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:37.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.796+0000 7ff6ecb3e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e80b0c60 con 0x7ff6e8159560 2023-10-06T10:45:37.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.796+0000 7ff6ecb3e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e80b0f40 con 0x7ff6e80b0760 2023-10-06T10:45:37.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.796+0000 7ff6ecb3e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80b1220 con 0x7ff6e80b49a0 2023-10-06T10:45:37.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.796+0000 7ff6e6d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff6e80b0760 0x7ff6e80b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:37.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.796+0000 7ff6e659c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff6e8159560 0x7ff6e80aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45602/0 (socket says 172.21.15.92:45602) 2023-10-06T10:45:37.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.796+0000 7ff6e659c700 1 -- 172.21.15.92:0/3275523766 learned_addr learned my addr 172.21.15.92:0/3275523766 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:37.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.796+0000 7ff6e6d9d700 1 --2- 172.21.15.92:0/3275523766 >> v2:172.21.15.92:3300/0 conn(0x7ff6e80b0760 0x7ff6e80b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55414/0 (socket says 172.21.15.92:55414) 2023-10-06T10:45:37.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.796+0000 7ff6ddd9b700 1 --2- 172.21.15.92:0/3275523766 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e80b49a0 0x7ff6e80aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:37.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.796+0000 7ff6e6d9d700 1 -- 172.21.15.92:0/3275523766 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e80b49a0 msgr2=0x7ff6e80aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:37.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.796+0000 7ff6e6d9d700 1 --2- 172.21.15.92:0/3275523766 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e80b49a0 0x7ff6e80aac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:37.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.796+0000 7ff6e6d9d700 1 -- 172.21.15.92:0/3275523766 >> v1:172.21.15.92:6789/0 conn(0x7ff6e8159560 legacy=0x7ff6e80aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:37.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.796+0000 7ff6e6d9d700 1 -- 172.21.15.92:0/3275523766 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e80b1500 con 0x7ff6e80b0760 2023-10-06T10:45:37.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.796+0000 7ff6e5d9b700 1 -- 172.21.15.92:0/3275523766 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288970647 0 0) 0x7ff6e80b0c60 con 0x7ff6e8159560 2023-10-06T10:45:37.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.796+0000 7ff6ddd9b700 1 --2- 172.21.15.92:0/3275523766 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e80b49a0 0x7ff6e80aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:37.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.797+0000 7ff6e6d9d700 1 --2- 172.21.15.92:0/3275523766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6e80b0760 0x7ff6e80b1bc0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009e60 tx=0x7ff6e0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1d0801997e1a9e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:37.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.797+0000 7ff6e5d9b700 1 -- 172.21.15.92:0/3275523766 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e000e070 con 0x7ff6e80b0760 2023-10-06T10:45:37.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.797+0000 7ff6e5d9b700 1 -- 172.21.15.92:0/3275523766 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e000bc30 con 0x7ff6e80b0760 2023-10-06T10:45:37.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.797+0000 7ff6e5d9b700 1 -- 172.21.15.92:0/3275523766 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0012430 con 0x7ff6e80b0760 2023-10-06T10:45:37.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.797+0000 7ff6ecb3e700 1 -- 172.21.15.92:0/3275523766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6e80b0760 msgr2=0x7ff6e80b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:37.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.797+0000 7ff6ecb3e700 1 --2- 172.21.15.92:0/3275523766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6e80b0760 0x7ff6e80b1bc0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009e60 tx=0x7ff6e0009780 comp rx=0 tx=0).stop 2023-10-06T10:45:37.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.797+0000 7ff6ecb3e700 1 -- 172.21.15.92:0/3275523766 shutdown_connections 2023-10-06T10:45:37.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.797+0000 7ff6ecb3e700 1 --2- 172.21.15.92:0/3275523766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6e80b0760 0x7ff6e80b1bc0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:37.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.797+0000 7ff6ecb3e700 1 --2- 172.21.15.92:0/3275523766 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e80b49a0 0x7ff6e80aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:37.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.797+0000 7ff6ecb3e700 1 -- 172.21.15.92:0/3275523766 >> 172.21.15.92:0/3275523766 conn(0x7ff6e80997f0 msgr2=0x7ff6e806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:37.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.797+0000 7ff6ecb3e700 1 -- 172.21.15.92:0/3275523766 shutdown_connections 2023-10-06T10:45:37.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.797+0000 7ff6ecb3e700 1 -- 172.21.15.92:0/3275523766 wait complete. 2023-10-06T10:45:37.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.799+0000 7ff6ecb3e700 1 Processor -- start 2023-10-06T10:45:37.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.799+0000 7ff6ecb3e700 1 -- start start 2023-10-06T10:45:37.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.799+0000 7ff6ecb3e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6e80b0760 0x7ff6e8152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:37.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.799+0000 7ff6ecb3e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e80b49a0 0x7ff6e8151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:37.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.799+0000 7ff6ecb3e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6e8159560 0x7ff6e814fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:37.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.799+0000 7ff6ecb3e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e8150820 con 0x7ff6e8159560 2023-10-06T10:45:37.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.799+0000 7ff6ecb3e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e815d660 con 0x7ff6e80b0760 2023-10-06T10:45:37.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.799+0000 7ff6ecb3e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e815e480 con 0x7ff6e80b49a0 2023-10-06T10:45:37.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.799+0000 7ff6e6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6e8159560 0x7ff6e814fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:37.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.799+0000 7ff6e6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6e8159560 0x7ff6e814fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55416/0 (socket says 172.21.15.92:55416) 2023-10-06T10:45:37.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.799+0000 7ff6e6d9d700 1 -- 172.21.15.92:0/3073013241 learned_addr learned my addr 172.21.15.92:0/3073013241 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:37.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.800+0000 7ff6ddd9b700 1 --2- 172.21.15.92:0/3073013241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e80b49a0 0x7ff6e8151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:37.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.800+0000 7ff6e659c700 1 --2- 172.21.15.92:0/3073013241 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6e80b0760 0x7ff6e8152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:37.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.800+0000 7ff6e6d9d700 1 -- 172.21.15.92:0/3073013241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e80b49a0 msgr2=0x7ff6e8151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:37.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.800+0000 7ff6e6d9d700 1 --2- 172.21.15.92:0/3073013241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e80b49a0 0x7ff6e8151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:37.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.800+0000 7ff6e6d9d700 1 -- 172.21.15.92:0/3073013241 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6e80b0760 msgr2=0x7ff6e8152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:37.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.800+0000 7ff6e6d9d700 1 --2- 172.21.15.92:0/3073013241 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6e80b0760 0x7ff6e8152bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:37.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.800+0000 7ff6e6d9d700 1 -- 172.21.15.92:0/3073013241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e0009a20 con 0x7ff6e8159560 2023-10-06T10:45:37.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.800+0000 7ff6ddd9b700 1 --2- 172.21.15.92:0/3073013241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e80b49a0 0x7ff6e8151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:37.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.800+0000 7ff6e659c700 1 --2- 172.21.15.92:0/3073013241 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6e80b0760 0x7ff6e8152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:37.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.800+0000 7ff6e6d9d700 1 --2- 172.21.15.92:0/3073013241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6e8159560 0x7ff6e814fcc0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8067c20 tx=0x7ff6e0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:37.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.800+0000 7ff6deffd700 1 -- 172.21.15.92:0/3073013241 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e001d630 con 0x7ff6e8159560 2023-10-06T10:45:37.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.800+0000 7ff6deffd700 1 -- 172.21.15.92:0/3073013241 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e001d7b0 con 0x7ff6e8159560 2023-10-06T10:45:37.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.800+0000 7ff6ecb3e700 1 -- 172.21.15.92:0/3073013241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e815d870 con 0x7ff6e8159560 2023-10-06T10:45:37.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.800+0000 7ff6deffd700 1 -- 172.21.15.92:0/3073013241 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e001db70 con 0x7ff6e8159560 2023-10-06T10:45:37.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.800+0000 7ff6ecb3e700 1 -- 172.21.15.92:0/3073013241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e815de30 con 0x7ff6e8159560 2023-10-06T10:45:37.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.801+0000 7ff6ecb3e700 1 -- 172.21.15.92:0/3073013241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e8064a90 con 0x7ff6e8159560 2023-10-06T10:45:37.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.801+0000 7ff6deffd700 1 -- 172.21.15.92:0/3073013241 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff6e0024070 con 0x7ff6e8159560 2023-10-06T10:45:37.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.802+0000 7ff6deffd700 1 --2- 172.21.15.92:0/3073013241 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff6c8066f30 0x7ff6c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:37.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.802+0000 7ff6deffd700 1 -- 172.21.15.92:0/3073013241 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7ff6e00871a0 con 0x7ff6e8159560 2023-10-06T10:45:37.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.802+0000 7ff6e659c700 1 --2- 172.21.15.92:0/3073013241 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff6c8066f30 0x7ff6c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:37.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.802+0000 7ff6e659c700 1 --2- 172.21.15.92:0/3073013241 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff6c8066f30 0x7ff6c80693e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8001810 tx=0x7ff6d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:37.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:37.806+0000 7ff6deffd700 1 -- 172.21.15.92:0/3073013241 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6e0054130 con 0x7ff6e8159560 2023-10-06T10:45:38.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.028+0000 7ff6ecb3e700 1 -- 172.21.15.92:0/3073013241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6e815e5c0 con 0x7ff6e8159560 2023-10-06T10:45:38.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.028+0000 7ff6deffd700 1 -- 172.21.15.92:0/3073013241 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4646 (secure 0 0 0) 0x7ff6e0057750 con 0x7ff6e8159560 2023-10-06T10:45:38.029 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:38.029 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":110,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2016052382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2016052382},{"type":"v1","addr":"172.21.15.124:6834","nonce":2016052382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:44:51.106486+0000","modified":"2023-10-06T10:45:34.353086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8520,"mds_1":8517,"mds_2":6719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6719":{"gid":6719,"name":"d","rank":2,"incarnation":104,"state":"up:active","state_seq":13,"addr":"172.21.15.124:6835/2307521923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2307521923},{"type":"v1","addr":"172.21.15.124:6835","nonce":2307521923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8517":{"gid":8517,"name":"a","rank":1,"incarnation":102,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6836/1979826278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979826278},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979826278}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6837/2298348426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2298348426},{"type":"v1","addr":"172.21.15.92:6837","nonce":2298348426}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T10:45:38.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.030+0000 7ff6ecb3e700 1 -- 172.21.15.92:0/3073013241 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff6c8066f30 msgr2=0x7ff6c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:38.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.030+0000 7ff6ecb3e700 1 --2- 172.21.15.92:0/3073013241 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff6c8066f30 0x7ff6c80693e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8001810 tx=0x7ff6d8009040 comp rx=0 tx=0).stop 2023-10-06T10:45:38.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.030+0000 7ff6ecb3e700 1 -- 172.21.15.92:0/3073013241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6e8159560 msgr2=0x7ff6e814fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:38.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.030+0000 7ff6ecb3e700 1 --2- 172.21.15.92:0/3073013241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6e8159560 0x7ff6e814fcc0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8067c20 tx=0x7ff6e0019000 comp rx=0 tx=0).stop 2023-10-06T10:45:38.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.030+0000 7ff6ecb3e700 1 -- 172.21.15.92:0/3073013241 shutdown_connections 2023-10-06T10:45:38.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.030+0000 7ff6ecb3e700 1 --2- 172.21.15.92:0/3073013241 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff6c8066f30 0x7ff6c80693e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:38.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.030+0000 7ff6ecb3e700 1 --2- 172.21.15.92:0/3073013241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6e8159560 0x7ff6e814fcc0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:38.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.030+0000 7ff6ecb3e700 1 --2- 172.21.15.92:0/3073013241 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6e80b0760 0x7ff6e8152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:38.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.030+0000 7ff6ecb3e700 1 --2- 172.21.15.92:0/3073013241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e80b49a0 0x7ff6e8151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:38.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.030+0000 7ff6ecb3e700 1 -- 172.21.15.92:0/3073013241 >> 172.21.15.92:0/3073013241 conn(0x7ff6e80997f0 msgr2=0x7ff6e80b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:38.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.030+0000 7ff6ecb3e700 1 -- 172.21.15.92:0/3073013241 shutdown_connections 2023-10-06T10:45:38.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.030+0000 7ff6ecb3e700 1 -- 172.21.15.92:0/3073013241 wait complete. 2023-10-06T10:45:38.033 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 110 2023-10-06T10:45:38.043 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2023-10-06T10:45:38.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:45:38.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.155+0000 7f9f526fa700 1 Processor -- start 2023-10-06T10:45:38.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.156+0000 7f9f526fa700 1 -- start start 2023-10-06T10:45:38.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.156+0000 7f9f526fa700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9f4c159ad0 0x7f9f4c159f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:38.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.156+0000 7f9f526fa700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f4c0ca670 0x7f9f4c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:38.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.156+0000 7f9f526fa700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f4c0cafd0 con 0x7f9f4c0cb5f0 2023-10-06T10:45:38.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.156+0000 7f9f526fa700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f4c15a490 con 0x7f9f4c0ca670 2023-10-06T10:45:38.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.156+0000 7f9f526fa700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f4c154020 con 0x7f9f4c159ad0 2023-10-06T10:45:38.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.156+0000 7f9f50c97700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f4c0ca670 0x7f9f4c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:38.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.157+0000 7f9f50c97700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f4c0ca670 0x7f9f4c0caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55422/0 (socket says 172.21.15.92:55422) 2023-10-06T10:45:38.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.157+0000 7f9f50c97700 1 -- 172.21.15.92:0/59982637 learned_addr learned my addr 172.21.15.92:0/59982637 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:38.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.157+0000 7f9f4b7fe700 1 --2- 172.21.15.92:0/59982637 >> v2:172.21.15.124:3300/0 conn(0x7f9f4c159ad0 0x7f9f4c159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:38.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.157+0000 7f9f50c97700 1 -- 172.21.15.92:0/59982637 >> v2:172.21.15.124:3300/0 conn(0x7f9f4c159ad0 msgr2=0x7f9f4c159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:38.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.157+0000 7f9f50c97700 1 --2- 172.21.15.92:0/59982637 >> v2:172.21.15.124:3300/0 conn(0x7f9f4c159ad0 0x7f9f4c159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:38.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.157+0000 7f9f50c97700 1 -- 172.21.15.92:0/59982637 >> v1:172.21.15.124:6789/0 conn(0x7f9f4c0cb5f0 legacy=0x7f9f4c0691a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:38.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.157+0000 7f9f50c97700 1 -- 172.21.15.92:0/59982637 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4c068d10 con 0x7f9f4c0ca670 2023-10-06T10:45:38.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.157+0000 7f9f50c97700 1 --2- 172.21.15.92:0/59982637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f4c0ca670 0x7f9f4c0caa90 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c06f2c0 tx=0x7f9f44009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38739106749efc95 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:38.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.157+0000 7f9f4affd700 1 -- 172.21.15.92:0/59982637 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4400e070 con 0x7f9f4c0ca670 2023-10-06T10:45:38.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.157+0000 7f9f4affd700 1 -- 172.21.15.92:0/59982637 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4400b990 con 0x7f9f4c0ca670 2023-10-06T10:45:38.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.157+0000 7f9f4affd700 1 -- 172.21.15.92:0/59982637 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f440092e0 con 0x7f9f4c0ca670 2023-10-06T10:45:38.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.157+0000 7f9f526fa700 1 -- 172.21.15.92:0/59982637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f4c0ca670 msgr2=0x7f9f4c0caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:38.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.158+0000 7f9f526fa700 1 --2- 172.21.15.92:0/59982637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f4c0ca670 0x7f9f4c0caa90 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c06f2c0 tx=0x7f9f44009840 comp rx=0 tx=0).stop 2023-10-06T10:45:38.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.158+0000 7f9f526fa700 1 -- 172.21.15.92:0/59982637 shutdown_connections 2023-10-06T10:45:38.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.158+0000 7f9f526fa700 1 --2- 172.21.15.92:0/59982637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f4c0ca670 0x7f9f4c0caa90 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:38.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.158+0000 7f9f526fa700 1 --2- 172.21.15.92:0/59982637 >> v2:172.21.15.124:3300/0 conn(0x7f9f4c159ad0 0x7f9f4c159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:38.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.158+0000 7f9f526fa700 1 -- 172.21.15.92:0/59982637 >> 172.21.15.92:0/59982637 conn(0x7f9f4c0707d0 msgr2=0x7f9f4c070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:38.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.158+0000 7f9f526fa700 1 -- 172.21.15.92:0/59982637 shutdown_connections 2023-10-06T10:45:38.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.158+0000 7f9f526fa700 1 -- 172.21.15.92:0/59982637 wait complete. 2023-10-06T10:45:38.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.159+0000 7f9f526fa700 1 Processor -- start 2023-10-06T10:45:38.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.159+0000 7f9f526fa700 1 -- start start 2023-10-06T10:45:38.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.160+0000 7f9f526fa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f4c0ca670 0x7f9f4c1518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:38.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.160+0000 7f9f526fa700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f4c0cb5f0 0x7f9f4c155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:38.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.160+0000 7f9f526fa700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f4c159ad0 0x7f9f4c069a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.160+0000 7f9f526fa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c156900 con 0x7f9f4c0ca670 2023-10-06T10:45:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.160+0000 7f9f526fa700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c155cf0 con 0x7f9f4c0cb5f0 2023-10-06T10:45:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.160+0000 7f9f526fa700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c155fd0 con 0x7f9f4c159ad0 2023-10-06T10:45:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.160+0000 7f9f4bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f4c0ca670 0x7f9f4c1518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.160+0000 7f9f4bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f4c0ca670 0x7f9f4c1518b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55430/0 (socket says 172.21.15.92:55430) 2023-10-06T10:45:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.160+0000 7f9f4bfff700 1 -- 172.21.15.92:0/2455327102 learned_addr learned my addr 172.21.15.92:0/2455327102 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.160+0000 7f9f4b7fe700 1 --2- 172.21.15.92:0/2455327102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f4c0cb5f0 0x7f9f4c155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.160+0000 7f9f50c97700 1 --2- 172.21.15.92:0/2455327102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f4c159ad0 0x7f9f4c069a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.160+0000 7f9f4bfff700 1 -- 172.21.15.92:0/2455327102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f4c159ad0 msgr2=0x7f9f4c069a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.160+0000 7f9f4bfff700 1 --2- 172.21.15.92:0/2455327102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f4c159ad0 0x7f9f4c069a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.160+0000 7f9f4bfff700 1 -- 172.21.15.92:0/2455327102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f4c0cb5f0 msgr2=0x7f9f4c155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.160+0000 7f9f4bfff700 1 --2- 172.21.15.92:0/2455327102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f4c0cb5f0 0x7f9f4c155380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.160+0000 7f9f4bfff700 1 -- 172.21.15.92:0/2455327102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f44009ae0 con 0x7f9f4c0ca670 2023-10-06T10:45:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.160+0000 7f9f4bfff700 1 --2- 172.21.15.92:0/2455327102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f4c0ca670 0x7f9f4c1518b0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00c590 tx=0x7f9f3c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.160+0000 7f9f33fff700 1 -- 172.21.15.92:0/2455327102 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c011590 con 0x7f9f4c0ca670 2023-10-06T10:45:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.160+0000 7f9f33fff700 1 -- 172.21.15.92:0/2455327102 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3c0116f0 con 0x7f9f4c0ca670 2023-10-06T10:45:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.160+0000 7f9f526fa700 1 -- 172.21.15.92:0/2455327102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f4c1562b0 con 0x7f9f4c0ca670 2023-10-06T10:45:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.160+0000 7f9f33fff700 1 -- 172.21.15.92:0/2455327102 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c011ab0 con 0x7f9f4c0ca670 2023-10-06T10:45:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.160+0000 7f9f526fa700 1 -- 172.21.15.92:0/2455327102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4c151df0 con 0x7f9f4c0ca670 2023-10-06T10:45:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.161+0000 7f9f526fa700 1 -- 172.21.15.92:0/2455327102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f4c156750 con 0x7f9f4c0ca670 2023-10-06T10:45:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.162+0000 7f9f33fff700 1 -- 172.21.15.92:0/2455327102 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f3c01b3b0 con 0x7f9f4c0ca670 2023-10-06T10:45:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.162+0000 7f9f33fff700 1 --2- 172.21.15.92:0/2455327102 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f38066f30 0x7f9f380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.162+0000 7f9f4b7fe700 1 --2- 172.21.15.92:0/2455327102 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f38066f30 0x7f9f380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:38.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.162+0000 7f9f33fff700 1 -- 172.21.15.92:0/2455327102 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f9f3c083c40 con 0x7f9f4c0ca670 2023-10-06T10:45:38.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.162+0000 7f9f4b7fe700 1 --2- 172.21.15.92:0/2455327102 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f38066f30 0x7f9f380693e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9f400093f0 tx=0x7f9f40008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:38.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.166+0000 7f9f33fff700 1 -- 172.21.15.92:0/2455327102 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f3c050bd0 con 0x7f9f4c0ca670 2023-10-06T10:45:38.352 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T10:45:38.352+0000 7f9e81bcf700 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T10:45:38.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.423+0000 7f9f526fa700 1 -- 172.21.15.92:0/2455327102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f9f4c0cafd0 con 0x7f9f4c0ca670 2023-10-06T10:45:38.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.424+0000 7f9f33fff700 1 -- 172.21.15.92:0/2455327102 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1958 (secure 0 0 0) 0x7f9f4c0cafd0 con 0x7f9f4c0ca670 2023-10-06T10:45:38.425 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:38.425 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":459,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":704416,"bytes_used":2455654400,"bytes_avail":770638458880,"bytes_total":773094113280,"write_bytes_sec":938,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":110,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6719},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":8517},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8520}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2023-10-06T10:45:36.178415+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2023-10-06T10:45:38.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.426+0000 7f9f526fa700 1 -- 172.21.15.92:0/2455327102 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f38066f30 msgr2=0x7f9f380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:38.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.426+0000 7f9f526fa700 1 --2- 172.21.15.92:0/2455327102 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f38066f30 0x7f9f380693e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9f400093f0 tx=0x7f9f40008040 comp rx=0 tx=0).stop 2023-10-06T10:45:38.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.426+0000 7f9f526fa700 1 -- 172.21.15.92:0/2455327102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f4c0ca670 msgr2=0x7f9f4c1518b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:38.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.426+0000 7f9f526fa700 1 --2- 172.21.15.92:0/2455327102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f4c0ca670 0x7f9f4c1518b0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00c590 tx=0x7f9f3c00c950 comp rx=0 tx=0).stop 2023-10-06T10:45:38.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.426+0000 7f9f526fa700 1 -- 172.21.15.92:0/2455327102 shutdown_connections 2023-10-06T10:45:38.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.426+0000 7f9f526fa700 1 --2- 172.21.15.92:0/2455327102 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9f38066f30 0x7f9f380693e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:38.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.426+0000 7f9f526fa700 1 --2- 172.21.15.92:0/2455327102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f4c159ad0 0x7f9f4c069a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:38.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.426+0000 7f9f526fa700 1 --2- 172.21.15.92:0/2455327102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f4c0ca670 0x7f9f4c1518b0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:38.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.426+0000 7f9f526fa700 1 --2- 172.21.15.92:0/2455327102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f4c0cb5f0 0x7f9f4c155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:38.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.426+0000 7f9f526fa700 1 -- 172.21.15.92:0/2455327102 >> 172.21.15.92:0/2455327102 conn(0x7f9f4c0707d0 msgr2=0x7f9f4c152820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:38.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.426+0000 7f9f526fa700 1 -- 172.21.15.92:0/2455327102 shutdown_connections 2023-10-06T10:45:38.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:38.426+0000 7f9f526fa700 1 -- 172.21.15.92:0/2455327102 wait complete. 2023-10-06T10:45:38.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2023-10-06T10:45:39.440 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:45:39.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.557+0000 7ff1eb450700 1 Processor -- start 2023-10-06T10:45:39.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.557+0000 7ff1eb450700 1 -- start start 2023-10-06T10:45:39.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.558+0000 7ff1eb450700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff1e4159440 0x7ff1e40b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:39.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.558+0000 7ff1eb450700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff1e40b35d0 0x7ff1e40cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:39.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.558+0000 7ff1eb450700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e40cb7a0 con 0x7ff1e40cc170 2023-10-06T10:45:39.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.558+0000 7ff1eb450700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e40b0860 con 0x7ff1e40b35d0 2023-10-06T10:45:39.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.558+0000 7ff1eb450700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e40cb150 con 0x7ff1e4159440 2023-10-06T10:45:39.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.558+0000 7ff1e9c4d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff1e40b35d0 0x7ff1e40cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:39.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.558+0000 7ff1e9c4d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff1e40b35d0 0x7ff1e40cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55446/0 (socket says 172.21.15.92:55446) 2023-10-06T10:45:39.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.558+0000 7ff1e9c4d700 1 -- 172.21.15.92:0/3424465926 learned_addr learned my addr 172.21.15.92:0/3424465926 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:39.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.558+0000 7ff1ea44e700 1 --2- 172.21.15.92:0/3424465926 >> v2:172.21.15.124:3300/0 conn(0x7ff1e4159440 0x7ff1e40b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:39.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.558+0000 7ff1e9c4d700 1 -- 172.21.15.92:0/3424465926 >> v2:172.21.15.124:3300/0 conn(0x7ff1e4159440 msgr2=0x7ff1e40b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:39.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.558+0000 7ff1e9c4d700 1 --2- 172.21.15.92:0/3424465926 >> v2:172.21.15.124:3300/0 conn(0x7ff1e4159440 0x7ff1e40b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:39.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.558+0000 7ff1e9c4d700 1 -- 172.21.15.92:0/3424465926 >> v1:172.21.15.124:6789/0 conn(0x7ff1e40cc170 legacy=0x7ff1e40b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:39.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.559+0000 7ff1e9c4d700 1 -- 172.21.15.92:0/3424465926 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e40cb430 con 0x7ff1e40b35d0 2023-10-06T10:45:39.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.559+0000 7ff1ea44e700 1 --2- 172.21.15.92:0/3424465926 >> v2:172.21.15.124:3300/0 conn(0x7ff1e4159440 0x7ff1e40b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:39.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.559+0000 7ff1e9c4d700 1 --2- 172.21.15.92:0/3424465926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1e40b35d0 0x7ff1e40cbc30 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7ff1e406f9f0 tx=0x7ff1d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71eff64cbaa51d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:39.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.559+0000 7ff1e944c700 1 -- 172.21.15.92:0/3424465926 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d400b860 con 0x7ff1e40b35d0 2023-10-06T10:45:39.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.559+0000 7ff1e944c700 1 -- 172.21.15.92:0/3424465926 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d400b9c0 con 0x7ff1e40b35d0 2023-10-06T10:45:39.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.559+0000 7ff1e944c700 1 -- 172.21.15.92:0/3424465926 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d400bd60 con 0x7ff1e40b35d0 2023-10-06T10:45:39.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.559+0000 7ff1eb450700 1 -- 172.21.15.92:0/3424465926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1e40b35d0 msgr2=0x7ff1e40cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:39.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.559+0000 7ff1eb450700 1 --2- 172.21.15.92:0/3424465926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1e40b35d0 0x7ff1e40cbc30 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7ff1e406f9f0 tx=0x7ff1d4009250 comp rx=0 tx=0).stop 2023-10-06T10:45:39.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.559+0000 7ff1eb450700 1 -- 172.21.15.92:0/3424465926 shutdown_connections 2023-10-06T10:45:39.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.559+0000 7ff1eb450700 1 --2- 172.21.15.92:0/3424465926 >> v2:172.21.15.124:3300/0 conn(0x7ff1e4159440 0x7ff1e40b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:39.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.559+0000 7ff1eb450700 1 --2- 172.21.15.92:0/3424465926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1e40b35d0 0x7ff1e40cbc30 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:39.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.559+0000 7ff1eb450700 1 -- 172.21.15.92:0/3424465926 >> 172.21.15.92:0/3424465926 conn(0x7ff1e406ece0 msgr2=0x7ff1e406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:39.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.559+0000 7ff1eb450700 1 -- 172.21.15.92:0/3424465926 shutdown_connections 2023-10-06T10:45:39.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.559+0000 7ff1eb450700 1 -- 172.21.15.92:0/3424465926 wait complete. 2023-10-06T10:45:39.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.560+0000 7ff1eb450700 1 Processor -- start 2023-10-06T10:45:39.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.560+0000 7ff1eb450700 1 -- start start 2023-10-06T10:45:39.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.561+0000 7ff1eb450700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1e40b35d0 0x7ff1e40b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:39.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.561+0000 7ff1eb450700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1e40cc170 0x7ff1e40c4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:39.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.561+0000 7ff1eb450700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1e4159440 0x7ff1e40c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:39.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.561+0000 7ff1eb450700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e40b05a0 con 0x7ff1e4159440 2023-10-06T10:45:39.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.561+0000 7ff1eb450700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e415f370 con 0x7ff1e40b35d0 2023-10-06T10:45:39.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.561+0000 7ff1eb450700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e415f020 con 0x7ff1e40cc170 2023-10-06T10:45:39.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.561+0000 7ff1eac4f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1e4159440 0x7ff1e40c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:39.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.561+0000 7ff1eac4f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1e4159440 0x7ff1e40c4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55458/0 (socket says 172.21.15.92:55458) 2023-10-06T10:45:39.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.561+0000 7ff1eac4f700 1 -- 172.21.15.92:0/58117313 learned_addr learned my addr 172.21.15.92:0/58117313 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:39.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.561+0000 7ff1ea44e700 1 --2- 172.21.15.92:0/58117313 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1e40b35d0 0x7ff1e40b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:39.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.561+0000 7ff1e9c4d700 1 --2- 172.21.15.92:0/58117313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1e40cc170 0x7ff1e40c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:39.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.562+0000 7ff1eac4f700 1 -- 172.21.15.92:0/58117313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1e40cc170 msgr2=0x7ff1e40c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:39.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.562+0000 7ff1eac4f700 1 --2- 172.21.15.92:0/58117313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1e40cc170 0x7ff1e40c4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:39.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.562+0000 7ff1eac4f700 1 -- 172.21.15.92:0/58117313 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1e40b35d0 msgr2=0x7ff1e40b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:39.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.562+0000 7ff1eac4f700 1 --2- 172.21.15.92:0/58117313 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1e40b35d0 0x7ff1e40b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:39.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.562+0000 7ff1eac4f700 1 -- 172.21.15.92:0/58117313 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d40094f0 con 0x7ff1e4159440 2023-10-06T10:45:39.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.562+0000 7ff1eac4f700 1 --2- 172.21.15.92:0/58117313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1e4159440 0x7ff1e40c4c70 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00c560 tx=0x7ff1dc007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:39.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.562+0000 7ff1da7fc700 1 -- 172.21.15.92:0/58117313 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc001120 con 0x7ff1e4159440 2023-10-06T10:45:39.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.562+0000 7ff1da7fc700 1 -- 172.21.15.92:0/58117313 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc011590 con 0x7ff1e4159440 2023-10-06T10:45:39.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.562+0000 7ff1eb450700 1 -- 172.21.15.92:0/58117313 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e40c52e0 con 0x7ff1e4159440 2023-10-06T10:45:39.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.562+0000 7ff1da7fc700 1 -- 172.21.15.92:0/58117313 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc011950 con 0x7ff1e4159440 2023-10-06T10:45:39.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.562+0000 7ff1eb450700 1 -- 172.21.15.92:0/58117313 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e416d460 con 0x7ff1e4159440 2023-10-06T10:45:39.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.563+0000 7ff1eb450700 1 -- 172.21.15.92:0/58117313 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e40c5800 con 0x7ff1e4159440 2023-10-06T10:45:39.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.563+0000 7ff1da7fc700 1 -- 172.21.15.92:0/58117313 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff1dc011cc0 con 0x7ff1e4159440 2023-10-06T10:45:39.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.564+0000 7ff1da7fc700 1 --2- 172.21.15.92:0/58117313 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff1c8066f60 0x7ff1c8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:39.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.564+0000 7ff1ea44e700 1 --2- 172.21.15.92:0/58117313 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff1c8066f60 0x7ff1c8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:39.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.564+0000 7ff1da7fc700 1 -- 172.21.15.92:0/58117313 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7ff1dc083ac0 con 0x7ff1e4159440 2023-10-06T10:45:39.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.564+0000 7ff1ea44e700 1 --2- 172.21.15.92:0/58117313 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff1c8066f60 0x7ff1c8069410 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00076a0 tx=0x7ff1e0009930 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:39.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.568+0000 7ff1da7fc700 1 -- 172.21.15.92:0/58117313 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1dc050a50 con 0x7ff1e4159440 2023-10-06T10:45:39.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.794+0000 7ff1eb450700 1 -- 172.21.15.92:0/58117313 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1e415f4e0 con 0x7ff1e4159440 2023-10-06T10:45:39.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.794+0000 7ff1da7fc700 1 -- 172.21.15.92:0/58117313 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4646 (secure 0 0 0) 0x7ff1dc017070 con 0x7ff1e4159440 2023-10-06T10:45:39.795 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:39.795 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":110,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2016052382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2016052382},{"type":"v1","addr":"172.21.15.124:6834","nonce":2016052382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:44:51.106486+0000","modified":"2023-10-06T10:45:34.353086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8520,"mds_1":8517,"mds_2":6719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6719":{"gid":6719,"name":"d","rank":2,"incarnation":104,"state":"up:active","state_seq":13,"addr":"172.21.15.124:6835/2307521923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2307521923},{"type":"v1","addr":"172.21.15.124:6835","nonce":2307521923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8517":{"gid":8517,"name":"a","rank":1,"incarnation":102,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6836/1979826278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979826278},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979826278}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6837/2298348426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2298348426},{"type":"v1","addr":"172.21.15.92:6837","nonce":2298348426}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T10:45:39.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.796+0000 7ff1eb450700 1 -- 172.21.15.92:0/58117313 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff1c8066f60 msgr2=0x7ff1c8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:39.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.796+0000 7ff1eb450700 1 --2- 172.21.15.92:0/58117313 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff1c8066f60 0x7ff1c8069410 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00076a0 tx=0x7ff1e0009930 comp rx=0 tx=0).stop 2023-10-06T10:45:39.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.796+0000 7ff1eb450700 1 -- 172.21.15.92:0/58117313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1e4159440 msgr2=0x7ff1e40c4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:39.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.796+0000 7ff1eb450700 1 --2- 172.21.15.92:0/58117313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1e4159440 0x7ff1e40c4c70 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00c560 tx=0x7ff1dc007ef0 comp rx=0 tx=0).stop 2023-10-06T10:45:39.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.796+0000 7ff1eb450700 1 -- 172.21.15.92:0/58117313 shutdown_connections 2023-10-06T10:45:39.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.796+0000 7ff1eb450700 1 --2- 172.21.15.92:0/58117313 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff1c8066f60 0x7ff1c8069410 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:39.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.796+0000 7ff1eb450700 1 --2- 172.21.15.92:0/58117313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1e4159440 0x7ff1e40c4c70 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:39.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.796+0000 7ff1eb450700 1 --2- 172.21.15.92:0/58117313 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1e40b35d0 0x7ff1e40b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:39.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.796+0000 7ff1eb450700 1 --2- 172.21.15.92:0/58117313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1e40cc170 0x7ff1e40c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:39.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.796+0000 7ff1eb450700 1 -- 172.21.15.92:0/58117313 >> 172.21.15.92:0/58117313 conn(0x7ff1e406ece0 msgr2=0x7ff1e40b4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:39.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.796+0000 7ff1eb450700 1 -- 172.21.15.92:0/58117313 shutdown_connections 2023-10-06T10:45:39.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.796+0000 7ff1eb450700 1 -- 172.21.15.92:0/58117313 wait complete. 2023-10-06T10:45:39.799 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 110 2023-10-06T10:45:39.809 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2023-10-06T10:45:39.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:45:39.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.922+0000 7fb3c8042700 1 Processor -- start 2023-10-06T10:45:39.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.923+0000 7fb3c8042700 1 -- start start 2023-10-06T10:45:39.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.923+0000 7fb3c8042700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb3c0159560 0x7fb3c00aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:39.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.923+0000 7fb3c8042700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb3c00b49a0 0x7fb3c00aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:39.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.923+0000 7fb3c8042700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3c00b0c60 con 0x7fb3c00b0760 2023-10-06T10:45:39.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.923+0000 7fb3c8042700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3c00b0f40 con 0x7fb3c00b49a0 2023-10-06T10:45:39.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.923+0000 7fb3c8042700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3c00b1220 con 0x7fb3c0159560 2023-10-06T10:45:39.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.924+0000 7fb3c55dd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb3c00b49a0 0x7fb3c00aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:39.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.924+0000 7fb3c55dd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb3c00b49a0 0x7fb3c00aaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55468/0 (socket says 172.21.15.92:55468) 2023-10-06T10:45:39.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.924+0000 7fb3c65df700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb3c00b0760 0x7fb3c00b3890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45618/0 (socket says 172.21.15.92:45618) 2023-10-06T10:45:39.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.924+0000 7fb3c55dd700 1 -- 172.21.15.92:0/297075190 learned_addr learned my addr 172.21.15.92:0/297075190 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:39.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.924+0000 7fb3c5dde700 1 --2- 172.21.15.92:0/297075190 >> v2:172.21.15.124:3300/0 conn(0x7fb3c0159560 0x7fb3c00aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:39.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.924+0000 7fb3c55dd700 1 -- 172.21.15.92:0/297075190 >> v2:172.21.15.124:3300/0 conn(0x7fb3c0159560 msgr2=0x7fb3c00aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:39.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.924+0000 7fb3c55dd700 1 --2- 172.21.15.92:0/297075190 >> v2:172.21.15.124:3300/0 conn(0x7fb3c0159560 0x7fb3c00aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:39.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.924+0000 7fb3c55dd700 1 -- 172.21.15.92:0/297075190 >> v1:172.21.15.92:6789/0 conn(0x7fb3c00b0760 legacy=0x7fb3c00b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:39.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.924+0000 7fb3c4ddc700 1 -- 172.21.15.92:0/297075190 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 5791146 0 0) 0x7fb3c00b0c60 con 0x7fb3c00b0760 2023-10-06T10:45:39.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.924+0000 7fb3c55dd700 1 -- 172.21.15.92:0/297075190 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c00b1500 con 0x7fb3c00b49a0 2023-10-06T10:45:39.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.924+0000 7fb3c5dde700 1 --2- 172.21.15.92:0/297075190 >> v2:172.21.15.124:3300/0 conn(0x7fb3c0159560 0x7fb3c00aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:39.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.924+0000 7fb3c55dd700 1 --2- 172.21.15.92:0/297075190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3c00b49a0 0x7fb3c00aaa70 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fb3b00099b0 tx=0x7fb3b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e042c6d37f0f6bcf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.924+0000 7fb3c4ddc700 1 -- 172.21.15.92:0/297075190 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b000b820 con 0x7fb3c00b49a0 2023-10-06T10:45:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.924+0000 7fb3c4ddc700 1 -- 172.21.15.92:0/297075190 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b000b980 con 0x7fb3c00b49a0 2023-10-06T10:45:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.924+0000 7fb3c4ddc700 1 -- 172.21.15.92:0/297075190 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b000bd20 con 0x7fb3c00b49a0 2023-10-06T10:45:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.925+0000 7fb3c8042700 1 -- 172.21.15.92:0/297075190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3c00b49a0 msgr2=0x7fb3c00aaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.925+0000 7fb3c8042700 1 --2- 172.21.15.92:0/297075190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3c00b49a0 0x7fb3c00aaa70 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fb3b00099b0 tx=0x7fb3b0009250 comp rx=0 tx=0).stop 2023-10-06T10:45:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.925+0000 7fb3c8042700 1 -- 172.21.15.92:0/297075190 shutdown_connections 2023-10-06T10:45:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.925+0000 7fb3c8042700 1 --2- 172.21.15.92:0/297075190 >> v2:172.21.15.124:3300/0 conn(0x7fb3c0159560 0x7fb3c00aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.925+0000 7fb3c8042700 1 --2- 172.21.15.92:0/297075190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3c00b49a0 0x7fb3c00aaa70 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.925+0000 7fb3c8042700 1 -- 172.21.15.92:0/297075190 >> 172.21.15.92:0/297075190 conn(0x7fb3c00997f0 msgr2=0x7fb3c006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.925+0000 7fb3c8042700 1 -- 172.21.15.92:0/297075190 shutdown_connections 2023-10-06T10:45:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.925+0000 7fb3c8042700 1 -- 172.21.15.92:0/297075190 wait complete. 2023-10-06T10:45:39.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.926+0000 7fb3c8042700 1 Processor -- start 2023-10-06T10:45:39.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.926+0000 7fb3c8042700 1 -- start start 2023-10-06T10:45:39.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.927+0000 7fb3c8042700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3c00b0760 0x7fb3c015aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:39.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.927+0000 7fb3c8042700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3c00b49a0 0x7fb3c015afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:39.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.927+0000 7fb3c8042700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3c0159560 0x7fb3c009ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:39.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.927+0000 7fb3c8042700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c015c350 con 0x7fb3c00b0760 2023-10-06T10:45:39.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.927+0000 7fb3c8042700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c015b7d0 con 0x7fb3c00b49a0 2023-10-06T10:45:39.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.927+0000 7fb3c8042700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c015bab0 con 0x7fb3c0159560 2023-10-06T10:45:39.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.927+0000 7fb3c5dde700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3c00b0760 0x7fb3c015aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:39.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.927+0000 7fb3c5dde700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3c00b0760 0x7fb3c015aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55472/0 (socket says 172.21.15.92:55472) 2023-10-06T10:45:39.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.927+0000 7fb3c5dde700 1 -- 172.21.15.92:0/4192581517 learned_addr learned my addr 172.21.15.92:0/4192581517 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:39.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.927+0000 7fb3c5dde700 1 -- 172.21.15.92:0/4192581517 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3c0159560 msgr2=0x7fb3c009ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:39.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.927+0000 7fb3c55dd700 1 --2- 172.21.15.92:0/4192581517 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3c00b49a0 0x7fb3c015afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:39.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.927+0000 7fb3c5dde700 1 --2- 172.21.15.92:0/4192581517 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3c0159560 0x7fb3c009ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:39.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.927+0000 7fb3c5dde700 1 -- 172.21.15.92:0/4192581517 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3c00b49a0 msgr2=0x7fb3c015afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:39.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.927+0000 7fb3c5dde700 1 --2- 172.21.15.92:0/4192581517 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3c00b49a0 0x7fb3c015afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:39.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.927+0000 7fb3c5dde700 1 -- 172.21.15.92:0/4192581517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b00094f0 con 0x7fb3c00b0760 2023-10-06T10:45:39.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.927+0000 7fb3c5dde700 1 --2- 172.21.15.92:0/4192581517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3c00b0760 0x7fb3c015aa60 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00d890 tx=0x7fb3bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:39.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.927+0000 7fb3b5ffb700 1 -- 172.21.15.92:0/4192581517 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc005a00 con 0x7fb3c00b0760 2023-10-06T10:45:39.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.927+0000 7fb3b5ffb700 1 -- 172.21.15.92:0/4192581517 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3bc005b60 con 0x7fb3c00b0760 2023-10-06T10:45:39.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.927+0000 7fb3c8042700 1 -- 172.21.15.92:0/4192581517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c015bd90 con 0x7fb3c00b0760 2023-10-06T10:45:39.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.927+0000 7fb3b5ffb700 1 -- 172.21.15.92:0/4192581517 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc0125f0 con 0x7fb3c00b0760 2023-10-06T10:45:39.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.928+0000 7fb3c8042700 1 -- 172.21.15.92:0/4192581517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c015b4e0 con 0x7fb3c00b0760 2023-10-06T10:45:39.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.928+0000 7fb3c8042700 1 -- 172.21.15.92:0/4192581517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c0064a90 con 0x7fb3c00b0760 2023-10-06T10:45:39.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.929+0000 7fb3b5ffb700 1 -- 172.21.15.92:0/4192581517 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb3bc0129a0 con 0x7fb3c00b0760 2023-10-06T10:45:39.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.929+0000 7fb3b5ffb700 1 --2- 172.21.15.92:0/4192581517 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb3a4066f30 0x7fb3a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:39.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.929+0000 7fb3b5ffb700 1 -- 172.21.15.92:0/4192581517 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fb3bc084a60 con 0x7fb3c00b0760 2023-10-06T10:45:39.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.929+0000 7fb3c55dd700 1 --2- 172.21.15.92:0/4192581517 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb3a4066f30 0x7fb3a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:39.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.929+0000 7fb3c55dd700 1 --2- 172.21.15.92:0/4192581517 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb3a4066f30 0x7fb3a40693e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0088de0 tx=0x7fb3b0017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:39.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:39.933+0000 7fb3b5ffb700 1 -- 172.21.15.92:0/4192581517 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3bc051aa0 con 0x7fb3c00b0760 2023-10-06T10:45:40.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:40.194+0000 7fb3c8042700 1 -- 172.21.15.92:0/4192581517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb3c00b17e0 con 0x7fb3c00b0760 2023-10-06T10:45:40.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:40.194+0000 7fb3b5ffb700 1 -- 172.21.15.92:0/4192581517 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1958 (secure 0 0 0) 0x7fb3bc0550c0 con 0x7fb3c00b0760 2023-10-06T10:45:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:40.196 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":461,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":704416,"bytes_used":2455654400,"bytes_avail":770638458880,"bytes_total":773094113280,"write_bytes_sec":938,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":110,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6719},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":8517},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8520}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2023-10-06T10:45:36.178415+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2023-10-06T10:45:40.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:40.196+0000 7fb3c8042700 1 -- 172.21.15.92:0/4192581517 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb3a4066f30 msgr2=0x7fb3a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:40.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:40.196+0000 7fb3c8042700 1 --2- 172.21.15.92:0/4192581517 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb3a4066f30 0x7fb3a40693e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0088de0 tx=0x7fb3b0017000 comp rx=0 tx=0).stop 2023-10-06T10:45:40.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:40.196+0000 7fb3c8042700 1 -- 172.21.15.92:0/4192581517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3c00b0760 msgr2=0x7fb3c015aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:40.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:40.196+0000 7fb3c8042700 1 --2- 172.21.15.92:0/4192581517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3c00b0760 0x7fb3c015aa60 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00d890 tx=0x7fb3bc008040 comp rx=0 tx=0).stop 2023-10-06T10:45:40.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:40.196+0000 7fb3c8042700 1 -- 172.21.15.92:0/4192581517 shutdown_connections 2023-10-06T10:45:40.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:40.196+0000 7fb3c8042700 1 --2- 172.21.15.92:0/4192581517 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb3a4066f30 0x7fb3a40693e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:40.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:40.196+0000 7fb3c8042700 1 --2- 172.21.15.92:0/4192581517 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3c0159560 0x7fb3c009ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:40.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:40.196+0000 7fb3c8042700 1 --2- 172.21.15.92:0/4192581517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3c00b0760 0x7fb3c015aa60 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:40.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:40.196+0000 7fb3c8042700 1 --2- 172.21.15.92:0/4192581517 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3c00b49a0 0x7fb3c015afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:40.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:40.196+0000 7fb3c8042700 1 -- 172.21.15.92:0/4192581517 >> 172.21.15.92:0/4192581517 conn(0x7fb3c00997f0 msgr2=0x7fb3c01574a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:40.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:40.196+0000 7fb3c8042700 1 -- 172.21.15.92:0/4192581517 shutdown_connections 2023-10-06T10:45:40.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:40.196+0000 7fb3c8042700 1 -- 172.21.15.92:0/4192581517 wait complete. 2023-10-06T10:45:40.210 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2023-10-06T10:45:41.211 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:45:41.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.331+0000 7f142ee64700 1 Processor -- start 2023-10-06T10:45:41.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.331+0000 7f142ee64700 1 -- start start 2023-10-06T10:45:41.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.332+0000 7f142ee64700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1428159560 0x7f14280aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:41.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.332+0000 7f142ee64700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14280b0c60 con 0x7f14280b0760 2023-10-06T10:45:41.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.332+0000 7f142ee64700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14280b0f40 con 0x7f14280b49a0 2023-10-06T10:45:41.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.332+0000 7f142ee64700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14280b1220 con 0x7f1428159560 2023-10-06T10:45:41.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.332+0000 7f142cc00700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1428159560 0x7f14280aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:41.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.332+0000 7f142cc00700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1428159560 0x7f14280aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55482/0 (socket says 172.21.15.92:55482) 2023-10-06T10:45:41.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.332+0000 7f142cc00700 1 -- 172.21.15.92:0/1953811490 learned_addr learned my addr 172.21.15.92:0/1953811490 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:41.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.332+0000 7f142cc00700 1 -- 172.21.15.92:0/1953811490 >> v1:172.21.15.124:6789/0 conn(0x7f14280b49a0 legacy=0x7f14280aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:41.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.332+0000 7f142cc00700 1 -- 172.21.15.92:0/1953811490 >> v1:172.21.15.92:6789/0 conn(0x7f14280b0760 legacy=0x7f14280b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:41.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.332+0000 7f142cc00700 1 -- 172.21.15.92:0/1953811490 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14280b1500 con 0x7f1428159560 2023-10-06T10:45:41.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.332+0000 7f142cc00700 1 --2- 172.21.15.92:0/1953811490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1428159560 0x7f14280aa530 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f141c0099c0 tx=0x7f141c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2de664da2a69347 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:41.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.332+0000 7f14277fe700 1 -- 172.21.15.92:0/1953811490 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c005c10 con 0x7f1428159560 2023-10-06T10:45:41.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.332+0000 7f14277fe700 1 -- 172.21.15.92:0/1953811490 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141c005d70 con 0x7f1428159560 2023-10-06T10:45:41.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.332+0000 7f14277fe700 1 -- 172.21.15.92:0/1953811490 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c00b840 con 0x7f1428159560 2023-10-06T10:45:41.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.333+0000 7f142ee64700 1 -- 172.21.15.92:0/1953811490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1428159560 msgr2=0x7f14280aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:41.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.333+0000 7f142ee64700 1 --2- 172.21.15.92:0/1953811490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1428159560 0x7f14280aa530 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f141c0099c0 tx=0x7f141c009220 comp rx=0 tx=0).stop 2023-10-06T10:45:41.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.333+0000 7f142ee64700 1 -- 172.21.15.92:0/1953811490 shutdown_connections 2023-10-06T10:45:41.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.333+0000 7f142ee64700 1 --2- 172.21.15.92:0/1953811490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1428159560 0x7f14280aa530 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:41.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.333+0000 7f142ee64700 1 -- 172.21.15.92:0/1953811490 >> 172.21.15.92:0/1953811490 conn(0x7f14280997f0 msgr2=0x7f142806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:41.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.333+0000 7f142ee64700 1 -- 172.21.15.92:0/1953811490 shutdown_connections 2023-10-06T10:45:41.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.333+0000 7f142ee64700 1 -- 172.21.15.92:0/1953811490 wait complete. 2023-10-06T10:45:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.334+0000 7f142ee64700 1 Processor -- start 2023-10-06T10:45:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.334+0000 7f142ee64700 1 -- start start 2023-10-06T10:45:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.335+0000 7f142ee64700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14280b0760 0x7f142814a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.335+0000 7f142ee64700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14280b49a0 0x7f14280b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.335+0000 7f142ee64700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1428159560 0x7f14280b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.335+0000 7f142ee64700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142809af40 con 0x7f1428159560 2023-10-06T10:45:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.335+0000 7f142ee64700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142809bbb0 con 0x7f14280b0760 2023-10-06T10:45:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.335+0000 7f142ee64700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f142809be90 con 0x7f14280b49a0 2023-10-06T10:45:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.335+0000 7f142d401700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1428159560 0x7f14280b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.335+0000 7f142d401700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1428159560 0x7f14280b2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55486/0 (socket says 172.21.15.92:55486) 2023-10-06T10:45:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.335+0000 7f142d401700 1 -- 172.21.15.92:0/4208950974 learned_addr learned my addr 172.21.15.92:0/4208950974 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.335+0000 7f142cc00700 1 --2- 172.21.15.92:0/4208950974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14280b0760 0x7f142814a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.335+0000 7f1427fff700 1 --2- 172.21.15.92:0/4208950974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14280b49a0 0x7f14280b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.335+0000 7f142d401700 1 -- 172.21.15.92:0/4208950974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14280b49a0 msgr2=0x7f14280b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.335+0000 7f142d401700 1 --2- 172.21.15.92:0/4208950974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14280b49a0 0x7f14280b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.335+0000 7f142d401700 1 -- 172.21.15.92:0/4208950974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14280b0760 msgr2=0x7f142814a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.335+0000 7f142d401700 1 --2- 172.21.15.92:0/4208950974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14280b0760 0x7f142814a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.335+0000 7f142d401700 1 -- 172.21.15.92:0/4208950974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c009480 con 0x7f1428159560 2023-10-06T10:45:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.336+0000 7f142d401700 1 --2- 172.21.15.92:0/4208950974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1428159560 0x7f14280b2090 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f142000c8e0 tx=0x7f1420001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.336+0000 7f1424ff9700 1 -- 172.21.15.92:0/4208950974 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1420012630 con 0x7f1428159560 2023-10-06T10:45:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.336+0000 7f1424ff9700 1 -- 172.21.15.92:0/4208950974 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1420012790 con 0x7f1428159560 2023-10-06T10:45:41.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.336+0000 7f142ee64700 1 -- 172.21.15.92:0/4208950974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142809c170 con 0x7f1428159560 2023-10-06T10:45:41.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.336+0000 7f1424ff9700 1 -- 172.21.15.92:0/4208950974 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1420012b50 con 0x7f1428159560 2023-10-06T10:45:41.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.336+0000 7f142ee64700 1 -- 172.21.15.92:0/4208950974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142814ace0 con 0x7f1428159560 2023-10-06T10:45:41.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.337+0000 7f142ee64700 1 -- 172.21.15.92:0/4208950974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1428064a90 con 0x7f1428159560 2023-10-06T10:45:41.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.337+0000 7f1424ff9700 1 -- 172.21.15.92:0/4208950974 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f142000a3b0 con 0x7f1428159560 2023-10-06T10:45:41.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.338+0000 7f1424ff9700 1 --2- 172.21.15.92:0/4208950974 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f140c066f30 0x7f140c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:41.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.338+0000 7f1424ff9700 1 -- 172.21.15.92:0/4208950974 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f1420084b40 con 0x7f1428159560 2023-10-06T10:45:41.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.338+0000 7f142cc00700 1 --2- 172.21.15.92:0/4208950974 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f140c066f30 0x7f140c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:41.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.338+0000 7f142cc00700 1 --2- 172.21.15.92:0/4208950974 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f140c066f30 0x7f140c0693e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f141c009260 tx=0x7f141c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:41.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.342+0000 7f1424ff9700 1 -- 172.21.15.92:0/4208950974 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1420051a30 con 0x7f1428159560 2023-10-06T10:45:41.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.570+0000 7f142ee64700 1 -- 172.21.15.92:0/4208950974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14280b17e0 con 0x7f1428159560 2023-10-06T10:45:41.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.571+0000 7f1424ff9700 1 -- 172.21.15.92:0/4208950974 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4646 (secure 0 0 0) 0x7f1420055050 con 0x7f1428159560 2023-10-06T10:45:41.573 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:41.573 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":110,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2016052382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2016052382},{"type":"v1","addr":"172.21.15.124:6834","nonce":2016052382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:44:51.106486+0000","modified":"2023-10-06T10:45:34.353086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8520,"mds_1":8517,"mds_2":6719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6719":{"gid":6719,"name":"d","rank":2,"incarnation":104,"state":"up:active","state_seq":13,"addr":"172.21.15.124:6835/2307521923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2307521923},{"type":"v1","addr":"172.21.15.124:6835","nonce":2307521923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8517":{"gid":8517,"name":"a","rank":1,"incarnation":102,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6836/1979826278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979826278},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979826278}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6837/2298348426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2298348426},{"type":"v1","addr":"172.21.15.92:6837","nonce":2298348426}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T10:45:41.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.572+0000 7f142ee64700 1 -- 172.21.15.92:0/4208950974 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f140c066f30 msgr2=0x7f140c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:41.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.572+0000 7f142ee64700 1 --2- 172.21.15.92:0/4208950974 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f140c066f30 0x7f140c0693e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f141c009260 tx=0x7f141c016000 comp rx=0 tx=0).stop 2023-10-06T10:45:41.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.572+0000 7f142ee64700 1 -- 172.21.15.92:0/4208950974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1428159560 msgr2=0x7f14280b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:41.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.572+0000 7f142ee64700 1 --2- 172.21.15.92:0/4208950974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1428159560 0x7f14280b2090 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f142000c8e0 tx=0x7f1420001710 comp rx=0 tx=0).stop 2023-10-06T10:45:41.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.573+0000 7f142ee64700 1 -- 172.21.15.92:0/4208950974 shutdown_connections 2023-10-06T10:45:41.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.573+0000 7f142ee64700 1 --2- 172.21.15.92:0/4208950974 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f140c066f30 0x7f140c0693e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:41.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.573+0000 7f142ee64700 1 --2- 172.21.15.92:0/4208950974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1428159560 0x7f14280b2090 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:41.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.573+0000 7f142ee64700 1 --2- 172.21.15.92:0/4208950974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14280b0760 0x7f142814a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:41.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.573+0000 7f142ee64700 1 --2- 172.21.15.92:0/4208950974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14280b49a0 0x7f14280b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:41.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.573+0000 7f142ee64700 1 -- 172.21.15.92:0/4208950974 >> 172.21.15.92:0/4208950974 conn(0x7f14280997f0 msgr2=0x7f14280aa660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:41.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.573+0000 7f142ee64700 1 -- 172.21.15.92:0/4208950974 shutdown_connections 2023-10-06T10:45:41.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.573+0000 7f142ee64700 1 -- 172.21.15.92:0/4208950974 wait complete. 2023-10-06T10:45:41.578 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 110 2023-10-06T10:45:41.586 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2023-10-06T10:45:41.587 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:45:41.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.698+0000 7f2aa9c1c700 1 Processor -- start 2023-10-06T10:45:41.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.698+0000 7f2aa9c1c700 1 -- start start 2023-10-06T10:45:41.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.699+0000 7f2aa9c1c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2aa4159470 0x7f2aa40ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:41.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.699+0000 7f2aa9c1c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa40cd5d0 con 0x7f2aa40c6220 2023-10-06T10:45:41.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.699+0000 7f2aa9c1c700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa40cc800 con 0x7f2aa40cd160 2023-10-06T10:45:41.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.699+0000 7f2aa9c1c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa40cbc30 con 0x7f2aa4159470 2023-10-06T10:45:41.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.699+0000 7f2aa941b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2aa40c6220 0x7f2aa4163030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45640/0 (socket says 172.21.15.92:45640) 2023-10-06T10:45:41.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.699+0000 7f2aa941b700 1 -- 172.21.15.92:0/3887220314 learned_addr learned my addr 172.21.15.92:0/3887220314 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:41.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.699+0000 7f2aa8c1a700 1 --2- 172.21.15.92:0/3887220314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2aa4159470 0x7f2aa40ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.699+0000 7f2aa37fe700 1 -- 172.21.15.92:0/3887220314 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799686951 0 0) 0x7f2aa40cd5d0 con 0x7f2aa40c6220 2023-10-06T10:45:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.700+0000 7f2aa37fe700 1 -- 172.21.15.92:0/3887220314 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a8c003570 con 0x7f2aa40c6220 2023-10-06T10:45:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.700+0000 7f2aa37fe700 1 -- 172.21.15.92:0/3887220314 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1908694068 0 0) 0x7f2aa40cc800 con 0x7f2aa40cd160 2023-10-06T10:45:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.700+0000 7f2aa37fe700 1 -- 172.21.15.92:0/3887220314 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2aa40cd5d0 con 0x7f2aa40cd160 2023-10-06T10:45:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.700+0000 7f2aa37fe700 1 -- 172.21.15.92:0/3887220314 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1271211606 0 0) 0x7f2a8c003570 con 0x7f2aa40c6220 2023-10-06T10:45:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.700+0000 7f2aa8c1a700 1 -- 172.21.15.92:0/3887220314 >> v1:172.21.15.124:6789/0 conn(0x7f2aa40cd160 legacy=0x7f2aa40ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.700+0000 7f2aa8c1a700 1 -- 172.21.15.92:0/3887220314 >> v1:172.21.15.92:6789/0 conn(0x7f2aa40c6220 legacy=0x7f2aa4163030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.700+0000 7f2aa8c1a700 1 -- 172.21.15.92:0/3887220314 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa40cbed0 con 0x7f2aa4159470 2023-10-06T10:45:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.700+0000 7f2aa8c1a700 1 --2- 172.21.15.92:0/3887220314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2aa4159470 0x7f2aa40ccc20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2a94009900 tx=0x7f2a94009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6d5d33524e5e4a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.700+0000 7f2aa37fe700 1 -- 172.21.15.92:0/3887220314 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0017f0 con 0x7f2aa4159470 2023-10-06T10:45:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.700+0000 7f2aa37fe700 1 -- 172.21.15.92:0/3887220314 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a94011420 con 0x7f2aa4159470 2023-10-06T10:45:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.700+0000 7f2aa37fe700 1 -- 172.21.15.92:0/3887220314 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a940117e0 con 0x7f2aa4159470 2023-10-06T10:45:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.701+0000 7f2aa9c1c700 1 -- 172.21.15.92:0/3887220314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2aa4159470 msgr2=0x7f2aa40ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.701+0000 7f2aa9c1c700 1 --2- 172.21.15.92:0/3887220314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2aa4159470 0x7f2aa40ccc20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2a94009900 tx=0x7f2a94009120 comp rx=0 tx=0).stop 2023-10-06T10:45:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.701+0000 7f2aa9c1c700 1 -- 172.21.15.92:0/3887220314 shutdown_connections 2023-10-06T10:45:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.701+0000 7f2aa9c1c700 1 --2- 172.21.15.92:0/3887220314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2aa4159470 0x7f2aa40ccc20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.701+0000 7f2aa9c1c700 1 -- 172.21.15.92:0/3887220314 >> 172.21.15.92:0/3887220314 conn(0x7f2aa40995c0 msgr2=0x7f2aa406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.701+0000 7f2aa9c1c700 1 -- 172.21.15.92:0/3887220314 shutdown_connections 2023-10-06T10:45:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.701+0000 7f2aa9c1c700 1 -- 172.21.15.92:0/3887220314 wait complete. 2023-10-06T10:45:41.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.702+0000 7f2aa9c1c700 1 Processor -- start 2023-10-06T10:45:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.702+0000 7f2aa9c1c700 1 -- start start 2023-10-06T10:45:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.703+0000 7f2aa9c1c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa40c6220 0x7f2aa4149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.703+0000 7f2aa9c1c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2aa40cd160 0x7f2aa415c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.703+0000 7f2aa9c1c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2aa4159470 0x7f2aa40b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.703+0000 7f2aa9c1c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa4156440 con 0x7f2aa40c6220 2023-10-06T10:45:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.703+0000 7f2aa9c1c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa416d760 con 0x7f2aa40cd160 2023-10-06T10:45:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.703+0000 7f2aa9c1c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa416ce30 con 0x7f2aa4159470 2023-10-06T10:45:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.703+0000 7f2aa8c1a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa40c6220 0x7f2aa4149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.703+0000 7f2aa8c1a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa40c6220 0x7f2aa4149db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55488/0 (socket says 172.21.15.92:55488) 2023-10-06T10:45:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.703+0000 7f2aa8c1a700 1 -- 172.21.15.92:0/1093151082 learned_addr learned my addr 172.21.15.92:0/1093151082 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.703+0000 7f2aa8c1a700 1 -- 172.21.15.92:0/1093151082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2aa4159470 msgr2=0x7f2aa40b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.703+0000 7f2aa8c1a700 1 --2- 172.21.15.92:0/1093151082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2aa4159470 0x7f2aa40b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.703+0000 7f2aa8c1a700 1 -- 172.21.15.92:0/1093151082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2aa40cd160 msgr2=0x7f2aa415c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.703+0000 7f2aa8c1a700 1 --2- 172.21.15.92:0/1093151082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2aa40cd160 0x7f2aa415c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.703+0000 7f2aa8c1a700 1 -- 172.21.15.92:0/1093151082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a940093c0 con 0x7f2aa40c6220 2023-10-06T10:45:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.703+0000 7f2aa8c1a700 1 --2- 172.21.15.92:0/1093151082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa40c6220 0x7f2aa4149db0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f2a94002000 tx=0x7f2a94016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.703+0000 7f2aa0ff9700 1 -- 172.21.15.92:0/1093151082 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9401cc10 con 0x7f2aa40c6220 2023-10-06T10:45:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.703+0000 7f2aa0ff9700 1 -- 172.21.15.92:0/1093151082 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9401cdb0 con 0x7f2aa40c6220 2023-10-06T10:45:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.703+0000 7f2aa9c1c700 1 -- 172.21.15.92:0/1093151082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa416d110 con 0x7f2aa40c6220 2023-10-06T10:45:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.704+0000 7f2aa0ff9700 1 -- 172.21.15.92:0/1093151082 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9401a6a0 con 0x7f2aa40c6220 2023-10-06T10:45:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.704+0000 7f2aa9c1c700 1 -- 172.21.15.92:0/1093151082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa4159d40 con 0x7f2aa40c6220 2023-10-06T10:45:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.704+0000 7f2aa9c1c700 1 -- 172.21.15.92:0/1093151082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aa4070b40 con 0x7f2aa40c6220 2023-10-06T10:45:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.705+0000 7f2aa0ff9700 1 -- 172.21.15.92:0/1093151082 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2a9401aa10 con 0x7f2aa40c6220 2023-10-06T10:45:41.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.705+0000 7f2aa0ff9700 1 --2- 172.21.15.92:0/1093151082 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2a90066f30 0x7f2a900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:41.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.705+0000 7f2aa0ff9700 1 -- 172.21.15.92:0/1093151082 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f2a94086040 con 0x7f2aa40c6220 2023-10-06T10:45:41.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.705+0000 7f2aa3fff700 1 --2- 172.21.15.92:0/1093151082 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2a90066f30 0x7f2a900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:41.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.706+0000 7f2aa3fff700 1 --2- 172.21.15.92:0/1093151082 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2a90066f30 0x7f2a900693e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2aa415a190 tx=0x7f2a980079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:41.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.709+0000 7f2aa0ff9700 1 -- 172.21.15.92:0/1093151082 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a94052fd0 con 0x7f2aa40c6220 2023-10-06T10:45:41.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.966+0000 7f2aa9c1c700 1 -- 172.21.15.92:0/1093151082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2aa405f230 con 0x7f2aa40c6220 2023-10-06T10:45:41.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.966+0000 7f2aa0ff9700 1 -- 172.21.15.92:0/1093151082 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1958 (secure 0 0 0) 0x7f2a940565f0 con 0x7f2aa40c6220 2023-10-06T10:45:41.968 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:41.968 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":463,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":704416,"bytes_used":2455654400,"bytes_avail":770638458880,"bytes_total":773094113280,"write_bytes_sec":938,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":110,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6719},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":8517},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8520}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2023-10-06T10:45:36.178415+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2023-10-06T10:45:41.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.968+0000 7f2aa9c1c700 1 -- 172.21.15.92:0/1093151082 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2a90066f30 msgr2=0x7f2a900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:41.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.968+0000 7f2aa9c1c700 1 --2- 172.21.15.92:0/1093151082 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2a90066f30 0x7f2a900693e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2aa415a190 tx=0x7f2a980079c0 comp rx=0 tx=0).stop 2023-10-06T10:45:41.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.968+0000 7f2aa9c1c700 1 -- 172.21.15.92:0/1093151082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa40c6220 msgr2=0x7f2aa4149db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:41.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.968+0000 7f2aa9c1c700 1 --2- 172.21.15.92:0/1093151082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa40c6220 0x7f2aa4149db0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f2a94002000 tx=0x7f2a94016000 comp rx=0 tx=0).stop 2023-10-06T10:45:41.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.968+0000 7f2aa9c1c700 1 -- 172.21.15.92:0/1093151082 shutdown_connections 2023-10-06T10:45:41.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.968+0000 7f2aa9c1c700 1 --2- 172.21.15.92:0/1093151082 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2a90066f30 0x7f2a900693e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:41.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.968+0000 7f2aa9c1c700 1 --2- 172.21.15.92:0/1093151082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2aa4159470 0x7f2aa40b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:41.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.968+0000 7f2aa9c1c700 1 --2- 172.21.15.92:0/1093151082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa40c6220 0x7f2aa4149db0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:41.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.968+0000 7f2aa9c1c700 1 --2- 172.21.15.92:0/1093151082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2aa40cd160 0x7f2aa415c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:41.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.968+0000 7f2aa9c1c700 1 -- 172.21.15.92:0/1093151082 >> 172.21.15.92:0/1093151082 conn(0x7f2aa40995c0 msgr2=0x7f2aa4158000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:41.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.968+0000 7f2aa9c1c700 1 -- 172.21.15.92:0/1093151082 shutdown_connections 2023-10-06T10:45:41.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:41.968+0000 7f2aa9c1c700 1 -- 172.21.15.92:0/1093151082 wait complete. 2023-10-06T10:45:41.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2023-10-06T10:45:42.983 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:45:43.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.100+0000 7f7d23a71700 1 Processor -- start 2023-10-06T10:45:43.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.100+0000 7f7d23a71700 1 -- start start 2023-10-06T10:45:43.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.101+0000 7f7d23a71700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d1c0b49a0 0x7f7d1c0aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:43.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.101+0000 7f7d23a71700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7d1c0b0760 0x7f7d1c0b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:43.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.101+0000 7f7d23a71700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d1c0b0c60 con 0x7f7d1c159560 2023-10-06T10:45:43.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.101+0000 7f7d23a71700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1c0b0f40 con 0x7f7d1c0b0760 2023-10-06T10:45:43.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.101+0000 7f7d23a71700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c0b1220 con 0x7f7d1c0b49a0 2023-10-06T10:45:43.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.101+0000 7f7d2200e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7d1c0b0760 0x7f7d1c0b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:43.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.101+0000 7f7d2200e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7d1c0b0760 0x7f7d1c0b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55492/0 (socket says 172.21.15.92:55492) 2023-10-06T10:45:43.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.101+0000 7f7d2200e700 1 -- 172.21.15.92:0/2378115509 learned_addr learned my addr 172.21.15.92:0/2378115509 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:43.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.101+0000 7f7d2100c700 1 --2- 172.21.15.92:0/2378115509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d1c0b49a0 0x7f7d1c0aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:43.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.102+0000 7f7d2200e700 1 -- 172.21.15.92:0/2378115509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d1c0b49a0 msgr2=0x7f7d1c0aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:43.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.102+0000 7f7d2200e700 1 --2- 172.21.15.92:0/2378115509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d1c0b49a0 0x7f7d1c0aac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:43.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.102+0000 7f7d2200e700 1 -- 172.21.15.92:0/2378115509 >> v1:172.21.15.124:6789/0 conn(0x7f7d1c159560 legacy=0x7f7d1c0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:43.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.102+0000 7f7d2200e700 1 -- 172.21.15.92:0/2378115509 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c0b1500 con 0x7f7d1c0b0760 2023-10-06T10:45:43.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.102+0000 7f7d2200e700 1 --2- 172.21.15.92:0/2378115509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d1c0b0760 0x7f7d1c0b1bc0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f7d140099b0 tx=0x7f7d14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62874b47b6a61a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:43.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.102+0000 7f7d2080b700 1 -- 172.21.15.92:0/2378115509 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1400b820 con 0x7f7d1c0b0760 2023-10-06T10:45:43.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.102+0000 7f7d2080b700 1 -- 172.21.15.92:0/2378115509 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1400b980 con 0x7f7d1c0b0760 2023-10-06T10:45:43.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.102+0000 7f7d2080b700 1 -- 172.21.15.92:0/2378115509 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1400bd20 con 0x7f7d1c0b0760 2023-10-06T10:45:43.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.102+0000 7f7d23a71700 1 -- 172.21.15.92:0/2378115509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d1c0b0760 msgr2=0x7f7d1c0b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:43.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.102+0000 7f7d23a71700 1 --2- 172.21.15.92:0/2378115509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d1c0b0760 0x7f7d1c0b1bc0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f7d140099b0 tx=0x7f7d14009250 comp rx=0 tx=0).stop 2023-10-06T10:45:43.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.102+0000 7f7d23a71700 1 -- 172.21.15.92:0/2378115509 shutdown_connections 2023-10-06T10:45:43.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.102+0000 7f7d23a71700 1 --2- 172.21.15.92:0/2378115509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d1c0b0760 0x7f7d1c0b1bc0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:43.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.102+0000 7f7d23a71700 1 --2- 172.21.15.92:0/2378115509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d1c0b49a0 0x7f7d1c0aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:43.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.102+0000 7f7d23a71700 1 -- 172.21.15.92:0/2378115509 >> 172.21.15.92:0/2378115509 conn(0x7f7d1c0997f0 msgr2=0x7f7d1c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:43.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.103+0000 7f7d23a71700 1 -- 172.21.15.92:0/2378115509 shutdown_connections 2023-10-06T10:45:43.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.103+0000 7f7d23a71700 1 -- 172.21.15.92:0/2378115509 wait complete. 2023-10-06T10:45:43.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.104+0000 7f7d23a71700 1 Processor -- start 2023-10-06T10:45:43.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.104+0000 7f7d23a71700 1 -- start start 2023-10-06T10:45:43.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.104+0000 7f7d23a71700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d1c0b0760 0x7f7d1c0aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:43.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.104+0000 7f7d23a71700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d1c0b49a0 0x7f7d1c1695d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:43.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.104+0000 7f7d23a71700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d1c159560 0x7f7d1c15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:43.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.104+0000 7f7d23a71700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c15cea0 con 0x7f7d1c0b0760 2023-10-06T10:45:43.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.104+0000 7f7d23a71700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c0ab200 con 0x7f7d1c0b49a0 2023-10-06T10:45:43.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.104+0000 7f7d23a71700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c0ac020 con 0x7f7d1c159560 2023-10-06T10:45:43.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.104+0000 7f7d2180d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d1c0b0760 0x7f7d1c0aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:43.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.104+0000 7f7d2180d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d1c0b0760 0x7f7d1c0aebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55498/0 (socket says 172.21.15.92:55498) 2023-10-06T10:45:43.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.104+0000 7f7d2180d700 1 -- 172.21.15.92:0/1849667411 learned_addr learned my addr 172.21.15.92:0/1849667411 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:43.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.104+0000 7f7d2200e700 1 --2- 172.21.15.92:0/1849667411 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d1c159560 0x7f7d1c15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:43.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.104+0000 7f7d2100c700 1 --2- 172.21.15.92:0/1849667411 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d1c0b49a0 0x7f7d1c1695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:43.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.105+0000 7f7d2180d700 1 -- 172.21.15.92:0/1849667411 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d1c159560 msgr2=0x7f7d1c15baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:43.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.105+0000 7f7d2180d700 1 --2- 172.21.15.92:0/1849667411 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d1c159560 0x7f7d1c15baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:43.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.105+0000 7f7d2180d700 1 -- 172.21.15.92:0/1849667411 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d1c0b49a0 msgr2=0x7f7d1c1695d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:43.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.105+0000 7f7d2180d700 1 --2- 172.21.15.92:0/1849667411 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d1c0b49a0 0x7f7d1c1695d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:43.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.105+0000 7f7d2180d700 1 -- 172.21.15.92:0/1849667411 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d140094f0 con 0x7f7d1c0b0760 2023-10-06T10:45:43.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.105+0000 7f7d2180d700 1 --2- 172.21.15.92:0/1849667411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d1c0b0760 0x7f7d1c0aebd0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800bc30 tx=0x7f7d18009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:43.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.105+0000 7f7d11ffb700 1 -- 172.21.15.92:0/1849667411 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1800c840 con 0x7f7d1c0b0760 2023-10-06T10:45:43.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.105+0000 7f7d11ffb700 1 -- 172.21.15.92:0/1849667411 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1800c9e0 con 0x7f7d1c0b0760 2023-10-06T10:45:43.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.105+0000 7f7d23a71700 1 -- 172.21.15.92:0/1849667411 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1c0ab410 con 0x7f7d1c0b0760 2023-10-06T10:45:43.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.105+0000 7f7d11ffb700 1 -- 172.21.15.92:0/1849667411 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1800cda0 con 0x7f7d1c0b0760 2023-10-06T10:45:43.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.105+0000 7f7d23a71700 1 -- 172.21.15.92:0/1849667411 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1c0ab9d0 con 0x7f7d1c0b0760 2023-10-06T10:45:43.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.106+0000 7f7d23a71700 1 -- 172.21.15.92:0/1849667411 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1c064a90 con 0x7f7d1c0b0760 2023-10-06T10:45:43.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.107+0000 7f7d11ffb700 1 -- 172.21.15.92:0/1849667411 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7d18014620 con 0x7f7d1c0b0760 2023-10-06T10:45:43.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.107+0000 7f7d11ffb700 1 --2- 172.21.15.92:0/1849667411 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7d00066f30 0x7f7d000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:43.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.107+0000 7f7d11ffb700 1 -- 172.21.15.92:0/1849667411 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f7d18084810 con 0x7f7d1c0b0760 2023-10-06T10:45:43.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.107+0000 7f7d2100c700 1 --2- 172.21.15.92:0/1849667411 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7d00066f30 0x7f7d000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:43.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.107+0000 7f7d2100c700 1 --2- 172.21.15.92:0/1849667411 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7d00066f30 0x7f7d000693e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0aa780 tx=0x7f7d0c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:43.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.111+0000 7f7d11ffb700 1 -- 172.21.15.92:0/1849667411 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d1804c5a0 con 0x7f7d1c0b0760 2023-10-06T10:45:43.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.334+0000 7f7d23a71700 1 -- 172.21.15.92:0/1849667411 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d1c0b17e0 con 0x7f7d1c0b0760 2023-10-06T10:45:43.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.334+0000 7f7d11ffb700 1 -- 172.21.15.92:0/1849667411 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4646 (secure 0 0 0) 0x7f7d180517a0 con 0x7f7d1c0b0760 2023-10-06T10:45:43.337 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:43.337 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":110,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2016052382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2016052382},{"type":"v1","addr":"172.21.15.124:6834","nonce":2016052382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:44:51.106486+0000","modified":"2023-10-06T10:45:34.353086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8520,"mds_1":8517,"mds_2":6719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6719":{"gid":6719,"name":"d","rank":2,"incarnation":104,"state":"up:active","state_seq":13,"addr":"172.21.15.124:6835/2307521923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2307521923},{"type":"v1","addr":"172.21.15.124:6835","nonce":2307521923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8517":{"gid":8517,"name":"a","rank":1,"incarnation":102,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6836/1979826278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979826278},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979826278}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6837/2298348426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2298348426},{"type":"v1","addr":"172.21.15.92:6837","nonce":2298348426}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T10:45:43.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.337+0000 7f7d23a71700 1 -- 172.21.15.92:0/1849667411 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7d00066f30 msgr2=0x7f7d000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:43.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.337+0000 7f7d23a71700 1 --2- 172.21.15.92:0/1849667411 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7d00066f30 0x7f7d000693e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0aa780 tx=0x7f7d0c008040 comp rx=0 tx=0).stop 2023-10-06T10:45:43.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.337+0000 7f7d23a71700 1 -- 172.21.15.92:0/1849667411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d1c0b0760 msgr2=0x7f7d1c0aebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:43.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.337+0000 7f7d23a71700 1 --2- 172.21.15.92:0/1849667411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d1c0b0760 0x7f7d1c0aebd0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800bc30 tx=0x7f7d18009820 comp rx=0 tx=0).stop 2023-10-06T10:45:43.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.337+0000 7f7d23a71700 1 -- 172.21.15.92:0/1849667411 shutdown_connections 2023-10-06T10:45:43.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.337+0000 7f7d23a71700 1 --2- 172.21.15.92:0/1849667411 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7d00066f30 0x7f7d000693e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:43.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.337+0000 7f7d23a71700 1 --2- 172.21.15.92:0/1849667411 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d1c159560 0x7f7d1c15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:43.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.337+0000 7f7d23a71700 1 --2- 172.21.15.92:0/1849667411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d1c0b0760 0x7f7d1c0aebd0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:43.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.337+0000 7f7d23a71700 1 --2- 172.21.15.92:0/1849667411 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d1c0b49a0 0x7f7d1c1695d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:43.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.337+0000 7f7d23a71700 1 -- 172.21.15.92:0/1849667411 >> 172.21.15.92:0/1849667411 conn(0x7f7d1c0997f0 msgr2=0x7f7d1c069490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:43.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.337+0000 7f7d23a71700 1 -- 172.21.15.92:0/1849667411 shutdown_connections 2023-10-06T10:45:43.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.337+0000 7f7d23a71700 1 -- 172.21.15.92:0/1849667411 wait complete. 2023-10-06T10:45:43.340 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 110 2023-10-06T10:45:43.350 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2023-10-06T10:45:43.351 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:45:43.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.463+0000 7f8c81ea0700 1 Processor -- start 2023-10-06T10:45:43.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.463+0000 7f8c81ea0700 1 -- start start 2023-10-06T10:45:43.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.464+0000 7f8c81ea0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8c7c0b49a0 0x7f8c7c0aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:43.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.464+0000 7f8c81ea0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8c7c0b0760 0x7f8c7c0b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:43.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.464+0000 7f8c81ea0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c7c0b0c60 con 0x7f8c7c159560 2023-10-06T10:45:43.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.464+0000 7f8c81ea0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c7c0b0f40 con 0x7f8c7c0b0760 2023-10-06T10:45:43.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.464+0000 7f8c81ea0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c0b1220 con 0x7f8c7c0b49a0 2023-10-06T10:45:43.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.464+0000 7f8c7bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8c7c0b0760 0x7f8c7c0b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:43.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.464+0000 7f8c7b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8c7c159560 0x7f8c7c0aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58754/0 (socket says 172.21.15.92:58754) 2023-10-06T10:45:43.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.464+0000 7f8c7b7fe700 1 -- 172.21.15.92:0/3129634860 learned_addr learned my addr 172.21.15.92:0/3129634860 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:43.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.465+0000 7f8c7affd700 1 --2- 172.21.15.92:0/3129634860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8c7c0b49a0 0x7f8c7c0aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:43.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.465+0000 7f8c7bfff700 1 -- 172.21.15.92:0/3129634860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8c7c0b49a0 msgr2=0x7f8c7c0aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:43.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.465+0000 7f8c7a7fc700 1 -- 172.21.15.92:0/3129634860 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2872791724 0 0) 0x7f8c7c0b0c60 con 0x7f8c7c159560 2023-10-06T10:45:43.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.465+0000 7f8c7bfff700 1 --2- 172.21.15.92:0/3129634860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8c7c0b49a0 0x7f8c7c0aac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:43.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.465+0000 7f8c7bfff700 1 -- 172.21.15.92:0/3129634860 >> v1:172.21.15.92:6789/0 conn(0x7f8c7c159560 legacy=0x7f8c7c0aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:43.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.465+0000 7f8c7bfff700 1 -- 172.21.15.92:0/3129634860 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7c0b1500 con 0x7f8c7c0b0760 2023-10-06T10:45:43.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.465+0000 7f8c7affd700 1 --2- 172.21.15.92:0/3129634860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8c7c0b49a0 0x7f8c7c0aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:43.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.465+0000 7f8c7bfff700 1 --2- 172.21.15.92:0/3129634860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c7c0b0760 0x7f8c7c0b1bc0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009e60 tx=0x7f8c74009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6aea268eb9a1a0d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:43.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.465+0000 7f8c7a7fc700 1 -- 172.21.15.92:0/3129634860 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7400e070 con 0x7f8c7c0b0760 2023-10-06T10:45:43.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.465+0000 7f8c7a7fc700 1 -- 172.21.15.92:0/3129634860 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7400bc30 con 0x7f8c7c0b0760 2023-10-06T10:45:43.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.465+0000 7f8c7a7fc700 1 -- 172.21.15.92:0/3129634860 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74012430 con 0x7f8c7c0b0760 2023-10-06T10:45:43.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.465+0000 7f8c81ea0700 1 -- 172.21.15.92:0/3129634860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c7c0b0760 msgr2=0x7f8c7c0b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:43.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.465+0000 7f8c81ea0700 1 --2- 172.21.15.92:0/3129634860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c7c0b0760 0x7f8c7c0b1bc0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009e60 tx=0x7f8c74009780 comp rx=0 tx=0).stop 2023-10-06T10:45:43.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.465+0000 7f8c81ea0700 1 -- 172.21.15.92:0/3129634860 shutdown_connections 2023-10-06T10:45:43.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.465+0000 7f8c81ea0700 1 --2- 172.21.15.92:0/3129634860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c7c0b0760 0x7f8c7c0b1bc0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:43.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.465+0000 7f8c81ea0700 1 --2- 172.21.15.92:0/3129634860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8c7c0b49a0 0x7f8c7c0aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:43.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.465+0000 7f8c81ea0700 1 -- 172.21.15.92:0/3129634860 >> 172.21.15.92:0/3129634860 conn(0x7f8c7c0997f0 msgr2=0x7f8c7c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:43.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.465+0000 7f8c81ea0700 1 -- 172.21.15.92:0/3129634860 shutdown_connections 2023-10-06T10:45:43.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.465+0000 7f8c81ea0700 1 -- 172.21.15.92:0/3129634860 wait complete. 2023-10-06T10:45:43.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.467+0000 7f8c81ea0700 1 Processor -- start 2023-10-06T10:45:43.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.467+0000 7f8c81ea0700 1 -- start start 2023-10-06T10:45:43.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.467+0000 7f8c81ea0700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8c7c0b0760 0x7f8c7c152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:43.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.467+0000 7f8c81ea0700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8c7c0b49a0 0x7f8c7c151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:43.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.467+0000 7f8c81ea0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c7c159560 0x7f8c7c14fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:43.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.467+0000 7f8c81ea0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c150820 con 0x7f8c7c159560 2023-10-06T10:45:43.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.467+0000 7f8c81ea0700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c15d660 con 0x7f8c7c0b49a0 2023-10-06T10:45:43.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.467+0000 7f8c81ea0700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c15e480 con 0x7f8c7c0b0760 2023-10-06T10:45:43.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.468+0000 7f8c7bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c7c159560 0x7f8c7c14fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:43.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.468+0000 7f8c7bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c7c159560 0x7f8c7c14fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47140/0 (socket says 172.21.15.92:47140) 2023-10-06T10:45:43.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.468+0000 7f8c7bfff700 1 -- 172.21.15.92:0/3550697898 learned_addr learned my addr 172.21.15.92:0/3550697898 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:43.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.468+0000 7f8c7b7fe700 1 --2- 172.21.15.92:0/3550697898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8c7c0b0760 0x7f8c7c152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:43.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.468+0000 7f8c7affd700 1 --2- 172.21.15.92:0/3550697898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8c7c0b49a0 0x7f8c7c151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:43.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.468+0000 7f8c7bfff700 1 -- 172.21.15.92:0/3550697898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8c7c0b0760 msgr2=0x7f8c7c152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:43.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.468+0000 7f8c7bfff700 1 --2- 172.21.15.92:0/3550697898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8c7c0b0760 0x7f8c7c152bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:43.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.468+0000 7f8c7bfff700 1 -- 172.21.15.92:0/3550697898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8c7c0b49a0 msgr2=0x7f8c7c151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:43.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.468+0000 7f8c7bfff700 1 --2- 172.21.15.92:0/3550697898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8c7c0b49a0 0x7f8c7c151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:43.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.468+0000 7f8c7bfff700 1 -- 172.21.15.92:0/3550697898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c74009a20 con 0x7f8c7c159560 2023-10-06T10:45:43.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.468+0000 7f8c7affd700 1 --2- 172.21.15.92:0/3550697898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8c7c0b49a0 0x7f8c7c151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:43.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.468+0000 7f8c7bfff700 1 --2- 172.21.15.92:0/3550697898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c7c159560 0x7f8c7c14fcc0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c067c20 tx=0x7f8c74019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:43.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.468+0000 7f8c637fe700 1 -- 172.21.15.92:0/3550697898 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7401d630 con 0x7f8c7c159560 2023-10-06T10:45:43.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.468+0000 7f8c637fe700 1 -- 172.21.15.92:0/3550697898 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7401d7b0 con 0x7f8c7c159560 2023-10-06T10:45:43.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.468+0000 7f8c637fe700 1 -- 172.21.15.92:0/3550697898 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7401db70 con 0x7f8c7c159560 2023-10-06T10:45:43.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.468+0000 7f8c81ea0700 1 -- 172.21.15.92:0/3550697898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c7c15d870 con 0x7f8c7c159560 2023-10-06T10:45:43.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.468+0000 7f8c81ea0700 1 -- 172.21.15.92:0/3550697898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c7c15de30 con 0x7f8c7c159560 2023-10-06T10:45:43.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.469+0000 7f8c81ea0700 1 -- 172.21.15.92:0/3550697898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c7c064a90 con 0x7f8c7c159560 2023-10-06T10:45:43.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.469+0000 7f8c637fe700 1 -- 172.21.15.92:0/3550697898 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c74024070 con 0x7f8c7c159560 2023-10-06T10:45:43.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.470+0000 7f8c637fe700 1 --2- 172.21.15.92:0/3550697898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8c68066f30 0x7f8c680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:43.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.470+0000 7f8c637fe700 1 -- 172.21.15.92:0/3550697898 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f8c74087120 con 0x7f8c7c159560 2023-10-06T10:45:43.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.470+0000 7f8c7b7fe700 1 --2- 172.21.15.92:0/3550697898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8c68066f30 0x7f8c680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:43.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.470+0000 7f8c7b7fe700 1 --2- 172.21.15.92:0/3550697898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8c68066f30 0x7f8c680693e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c002800 tx=0x7f8c6c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:43.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.474+0000 7f8c637fe700 1 -- 172.21.15.92:0/3550697898 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c740540b0 con 0x7f8c7c159560 2023-10-06T10:45:43.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.731+0000 7f8c81ea0700 1 -- 172.21.15.92:0/3550697898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f8c7c15e5c0 con 0x7f8c7c159560 2023-10-06T10:45:43.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.732+0000 7f8c637fe700 1 -- 172.21.15.92:0/3550697898 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1970 (secure 0 0 0) 0x7f8c740576d0 con 0x7f8c7c159560 2023-10-06T10:45:43.734 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:43.734 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":465,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":704416,"bytes_used":2455654400,"bytes_avail":770638458880,"bytes_total":773094113280,"write_bytes_sec":682,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":110,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6719},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":8517},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8520}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2023-10-06T10:45:42.179532+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2023-10-06T10:45:43.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.734+0000 7f8c81ea0700 1 -- 172.21.15.92:0/3550697898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8c68066f30 msgr2=0x7f8c680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:43.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.734+0000 7f8c81ea0700 1 --2- 172.21.15.92:0/3550697898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8c68066f30 0x7f8c680693e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c002800 tx=0x7f8c6c009040 comp rx=0 tx=0).stop 2023-10-06T10:45:43.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.734+0000 7f8c81ea0700 1 -- 172.21.15.92:0/3550697898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c7c159560 msgr2=0x7f8c7c14fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:43.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.734+0000 7f8c81ea0700 1 --2- 172.21.15.92:0/3550697898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c7c159560 0x7f8c7c14fcc0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c067c20 tx=0x7f8c74019000 comp rx=0 tx=0).stop 2023-10-06T10:45:43.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.734+0000 7f8c81ea0700 1 -- 172.21.15.92:0/3550697898 shutdown_connections 2023-10-06T10:45:43.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.734+0000 7f8c81ea0700 1 --2- 172.21.15.92:0/3550697898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8c68066f30 0x7f8c680693e0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:43.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.734+0000 7f8c81ea0700 1 --2- 172.21.15.92:0/3550697898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c7c159560 0x7f8c7c14fcc0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:43.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.734+0000 7f8c81ea0700 1 --2- 172.21.15.92:0/3550697898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8c7c0b0760 0x7f8c7c152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:43.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.734+0000 7f8c81ea0700 1 --2- 172.21.15.92:0/3550697898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8c7c0b49a0 0x7f8c7c151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:43.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.734+0000 7f8c81ea0700 1 -- 172.21.15.92:0/3550697898 >> 172.21.15.92:0/3550697898 conn(0x7f8c7c0997f0 msgr2=0x7f8c7c0b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:43.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.734+0000 7f8c81ea0700 1 -- 172.21.15.92:0/3550697898 shutdown_connections 2023-10-06T10:45:43.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:43.734+0000 7f8c81ea0700 1 -- 172.21.15.92:0/3550697898 wait complete. 2023-10-06T10:45:43.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]', 'd': 'idle+waiting paths [/d1/d2/d3]'} 2023-10-06T10:45:44.749 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:45:44.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.864+0000 7f7966045700 1 Processor -- start 2023-10-06T10:45:44.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.864+0000 7f7966045700 1 -- start start 2023-10-06T10:45:44.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.865+0000 7f7966045700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f79600ca370 0x7f7960158270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:44.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.865+0000 7f7966045700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f79600adb60 0x7f79600ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:44.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.865+0000 7f7966045700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7960157de0 con 0x7f79600ce6d0 2023-10-06T10:45:44.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.865+0000 7f7966045700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79600cbb90 con 0x7f79600adb60 2023-10-06T10:45:44.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.865+0000 7f7966045700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79600ceb10 con 0x7f79600ca370 2023-10-06T10:45:44.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.865+0000 7f795effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f79600adb60 0x7f79600ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:44.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.865+0000 7f795effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f79600adb60 0x7f79600ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47152/0 (socket says 172.21.15.92:47152) 2023-10-06T10:45:44.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.865+0000 7f795effd700 1 -- 172.21.15.92:0/3287986486 learned_addr learned my addr 172.21.15.92:0/3287986486 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:44.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.866+0000 7f795effd700 1 -- 172.21.15.92:0/3287986486 >> v2:172.21.15.124:3300/0 conn(0x7f79600ca370 msgr2=0x7f7960158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:44.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.866+0000 7f795f7fe700 1 --2- 172.21.15.92:0/3287986486 >> v2:172.21.15.124:3300/0 conn(0x7f79600ca370 0x7f7960158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:44.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.866+0000 7f795effd700 1 --2- 172.21.15.92:0/3287986486 >> v2:172.21.15.124:3300/0 conn(0x7f79600ca370 0x7f7960158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:44.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.866+0000 7f795effd700 1 -- 172.21.15.92:0/3287986486 >> v1:172.21.15.124:6789/0 conn(0x7f79600ce6d0 legacy=0x7f796015aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:44.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.866+0000 7f795effd700 1 -- 172.21.15.92:0/3287986486 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79600cb5d0 con 0x7f79600adb60 2023-10-06T10:45:44.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.866+0000 7f795effd700 1 --2- 172.21.15.92:0/3287986486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79600adb60 0x7f79600ce190 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f796006f2c0 tx=0x7f7954009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed3bc60a29b2cd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:44.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.866+0000 7f795e7fc700 1 -- 172.21.15.92:0/3287986486 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795400b820 con 0x7f79600adb60 2023-10-06T10:45:44.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.866+0000 7f795e7fc700 1 -- 172.21.15.92:0/3287986486 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795400b980 con 0x7f79600adb60 2023-10-06T10:45:44.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.866+0000 7f795e7fc700 1 -- 172.21.15.92:0/3287986486 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795400bd20 con 0x7f79600adb60 2023-10-06T10:45:44.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.867+0000 7f7966045700 1 -- 172.21.15.92:0/3287986486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79600adb60 msgr2=0x7f79600ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:44.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.867+0000 7f7966045700 1 --2- 172.21.15.92:0/3287986486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79600adb60 0x7f79600ce190 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f796006f2c0 tx=0x7f7954009250 comp rx=0 tx=0).stop 2023-10-06T10:45:44.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.867+0000 7f7966045700 1 -- 172.21.15.92:0/3287986486 shutdown_connections 2023-10-06T10:45:44.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.867+0000 7f7966045700 1 --2- 172.21.15.92:0/3287986486 >> v2:172.21.15.124:3300/0 conn(0x7f79600ca370 0x7f7960158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:44.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.867+0000 7f7966045700 1 --2- 172.21.15.92:0/3287986486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79600adb60 0x7f79600ce190 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:44.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.867+0000 7f7966045700 1 -- 172.21.15.92:0/3287986486 >> 172.21.15.92:0/3287986486 conn(0x7f79600707d0 msgr2=0x7f7960070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:44.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.867+0000 7f7966045700 1 -- 172.21.15.92:0/3287986486 shutdown_connections 2023-10-06T10:45:44.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.867+0000 7f7966045700 1 -- 172.21.15.92:0/3287986486 wait complete. 2023-10-06T10:45:44.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.868+0000 7f7966045700 1 Processor -- start 2023-10-06T10:45:44.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.868+0000 7f7966045700 1 -- start start 2023-10-06T10:45:44.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f7966045700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79600adb60 0x7f79600c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:44.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f7966045700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79600ca370 0x7f79600c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:44.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f7966045700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79600ce6d0 0x7f7960149560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:44.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f7966045700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7960168f70 con 0x7f79600adb60 2023-10-06T10:45:44.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f7966045700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7960169250 con 0x7f79600ca370 2023-10-06T10:45:44.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f7966045700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7960169530 con 0x7f79600ce6d0 2023-10-06T10:45:44.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f795f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79600adb60 0x7f79600c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:44.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f795f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79600adb60 0x7f79600c76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47158/0 (socket says 172.21.15.92:47158) 2023-10-06T10:45:44.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f795f7fe700 1 -- 172.21.15.92:0/3994596919 learned_addr learned my addr 172.21.15.92:0/3994596919 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:44.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f795f7fe700 1 -- 172.21.15.92:0/3994596919 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79600ce6d0 msgr2=0x7f7960149560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:44.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f795ffff700 1 --2- 172.21.15.92:0/3994596919 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79600ce6d0 0x7f7960149560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:44.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f795effd700 1 --2- 172.21.15.92:0/3994596919 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79600ca370 0x7f79600c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:44.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f795f7fe700 1 --2- 172.21.15.92:0/3994596919 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79600ce6d0 0x7f7960149560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:44.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f795f7fe700 1 -- 172.21.15.92:0/3994596919 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79600ca370 msgr2=0x7f79600c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:44.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f795f7fe700 1 --2- 172.21.15.92:0/3994596919 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79600ca370 0x7f79600c5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:44.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f795f7fe700 1 -- 172.21.15.92:0/3994596919 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79540094f0 con 0x7f79600adb60 2023-10-06T10:45:44.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f795effd700 1 --2- 172.21.15.92:0/3994596919 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79600ca370 0x7f79600c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:44.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f795ffff700 1 --2- 172.21.15.92:0/3994596919 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79600ce6d0 0x7f7960149560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:44.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f795f7fe700 1 --2- 172.21.15.92:0/3994596919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79600adb60 0x7f79600c76c0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f795000ce10 tx=0x7f7950007810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:44.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f79477fe700 1 -- 172.21.15.92:0/3994596919 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795000ab90 con 0x7f79600adb60 2023-10-06T10:45:44.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f79477fe700 1 -- 172.21.15.92:0/3994596919 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795000acf0 con 0x7f79600adb60 2023-10-06T10:45:44.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f7966045700 1 -- 172.21.15.92:0/3994596919 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7960169810 con 0x7f79600adb60 2023-10-06T10:45:44.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.869+0000 7f79477fe700 1 -- 172.21.15.92:0/3994596919 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7950017660 con 0x7f79600adb60 2023-10-06T10:45:44.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.870+0000 7f7966045700 1 -- 172.21.15.92:0/3994596919 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7960169dd0 con 0x7f79600adb60 2023-10-06T10:45:44.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.870+0000 7f7966045700 1 -- 172.21.15.92:0/3994596919 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7960169ff0 con 0x7f79600adb60 2023-10-06T10:45:44.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.871+0000 7f79477fe700 1 -- 172.21.15.92:0/3994596919 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7950003020 con 0x7f79600adb60 2023-10-06T10:45:44.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.871+0000 7f79477fe700 1 --2- 172.21.15.92:0/3994596919 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f794c066f30 0x7f794c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:44.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.871+0000 7f79477fe700 1 -- 172.21.15.92:0/3994596919 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f7950013070 con 0x7f79600adb60 2023-10-06T10:45:44.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.871+0000 7f795effd700 1 --2- 172.21.15.92:0/3994596919 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f794c066f30 0x7f794c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:44.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.871+0000 7f795effd700 1 --2- 172.21.15.92:0/3994596919 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f794c066f30 0x7f794c0693e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f796006f2c0 tx=0x7f7954016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:44.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:44.875+0000 7f79477fe700 1 -- 172.21.15.92:0/3994596919 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7950051d30 con 0x7f79600adb60 2023-10-06T10:45:45.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.099+0000 7f7966045700 1 -- 172.21.15.92:0/3994596919 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7960064a90 con 0x7f79600adb60 2023-10-06T10:45:45.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.100+0000 7f79477fe700 1 -- 172.21.15.92:0/3994596919 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4646 (secure 0 0 0) 0x7f795001c020 con 0x7f79600adb60 2023-10-06T10:45:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:45.102 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":110,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2016052382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2016052382},{"type":"v1","addr":"172.21.15.124:6834","nonce":2016052382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:44:51.106486+0000","modified":"2023-10-06T10:45:34.353086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8520,"mds_1":8517,"mds_2":6719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6719":{"gid":6719,"name":"d","rank":2,"incarnation":104,"state":"up:active","state_seq":13,"addr":"172.21.15.124:6835/2307521923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2307521923},{"type":"v1","addr":"172.21.15.124:6835","nonce":2307521923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8517":{"gid":8517,"name":"a","rank":1,"incarnation":102,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6836/1979826278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979826278},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979826278}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6837/2298348426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2298348426},{"type":"v1","addr":"172.21.15.92:6837","nonce":2298348426}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T10:45:45.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.102+0000 7f7966045700 1 -- 172.21.15.92:0/3994596919 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f794c066f30 msgr2=0x7f794c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:45.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.102+0000 7f7966045700 1 --2- 172.21.15.92:0/3994596919 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f794c066f30 0x7f794c0693e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f796006f2c0 tx=0x7f7954016000 comp rx=0 tx=0).stop 2023-10-06T10:45:45.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.102+0000 7f7966045700 1 -- 172.21.15.92:0/3994596919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79600adb60 msgr2=0x7f79600c76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:45.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.102+0000 7f7966045700 1 --2- 172.21.15.92:0/3994596919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79600adb60 0x7f79600c76c0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f795000ce10 tx=0x7f7950007810 comp rx=0 tx=0).stop 2023-10-06T10:45:45.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.102+0000 7f7966045700 1 -- 172.21.15.92:0/3994596919 shutdown_connections 2023-10-06T10:45:45.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.102+0000 7f7966045700 1 --2- 172.21.15.92:0/3994596919 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f794c066f30 0x7f794c0693e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:45.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.102+0000 7f7966045700 1 --2- 172.21.15.92:0/3994596919 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79600ce6d0 0x7f7960149560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:45.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.102+0000 7f7966045700 1 --2- 172.21.15.92:0/3994596919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79600adb60 0x7f79600c76c0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:45.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.102+0000 7f7966045700 1 --2- 172.21.15.92:0/3994596919 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79600ca370 0x7f79600c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:45.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.102+0000 7f7966045700 1 -- 172.21.15.92:0/3994596919 >> 172.21.15.92:0/3994596919 conn(0x7f79600707d0 msgr2=0x7f796015a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:45.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.102+0000 7f7966045700 1 -- 172.21.15.92:0/3994596919 shutdown_connections 2023-10-06T10:45:45.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.102+0000 7f7966045700 1 -- 172.21.15.92:0/3994596919 wait complete. 2023-10-06T10:45:45.107 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 110 2023-10-06T10:45:45.115 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2023-10-06T10:45:45.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:45:45.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.226+0000 7fbc19056700 1 Processor -- start 2023-10-06T10:45:45.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.226+0000 7fbc19056700 1 -- start start 2023-10-06T10:45:45.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.227+0000 7fbc19056700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc140b49a0 0x7fbc140aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:45.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.227+0000 7fbc19056700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbc140b0760 0x7fbc140b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:45.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.227+0000 7fbc19056700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc140b0c60 con 0x7fbc14159560 2023-10-06T10:45:45.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.227+0000 7fbc19056700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc140b0f40 con 0x7fbc140b0760 2023-10-06T10:45:45.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.227+0000 7fbc19056700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc140b1220 con 0x7fbc140b49a0 2023-10-06T10:45:45.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.227+0000 7fbc1359e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbc140b0760 0x7fbc140b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:45.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.227+0000 7fbc1359e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbc140b0760 0x7fbc140b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47170/0 (socket says 172.21.15.92:47170) 2023-10-06T10:45:45.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.227+0000 7fbc1359e700 1 -- 172.21.15.92:0/527028527 learned_addr learned my addr 172.21.15.92:0/527028527 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:45.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.227+0000 7fbc12d9d700 1 --1- 172.21.15.92:0/527028527 >> v1:172.21.15.92:6789/0 conn(0x7fbc14159560 0x7fbc140aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58766/0 (socket says 172.21.15.92:58766) 2023-10-06T10:45:45.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.227+0000 7fbc0a59c700 1 --2- 172.21.15.92:0/527028527 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc140b49a0 0x7fbc140aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:45.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.227+0000 7fbc1359e700 1 -- 172.21.15.92:0/527028527 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc140b49a0 msgr2=0x7fbc140aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:45.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.227+0000 7fbc1359e700 1 --2- 172.21.15.92:0/527028527 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc140b49a0 0x7fbc140aac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:45.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.227+0000 7fbc1359e700 1 -- 172.21.15.92:0/527028527 >> v1:172.21.15.92:6789/0 conn(0x7fbc14159560 legacy=0x7fbc140aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:45.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.227+0000 7fbc1359e700 1 -- 172.21.15.92:0/527028527 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc140b1500 con 0x7fbc140b0760 2023-10-06T10:45:45.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.227+0000 7fbc1259c700 1 -- 172.21.15.92:0/527028527 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3632731357 0 0) 0x7fbc140b0c60 con 0x7fbc14159560 2023-10-06T10:45:45.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.227+0000 7fbc0a59c700 1 --2- 172.21.15.92:0/527028527 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc140b49a0 0x7fbc140aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:45.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.228+0000 7fbc1359e700 1 --2- 172.21.15.92:0/527028527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc140b0760 0x7fbc140b1bc0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009f20 tx=0x7fbc0c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e622ce2e58d401 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:45.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.228+0000 7fbc1259c700 1 -- 172.21.15.92:0/527028527 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c00e070 con 0x7fbc140b0760 2023-10-06T10:45:45.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.228+0000 7fbc1259c700 1 -- 172.21.15.92:0/527028527 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0c00ba80 con 0x7fbc140b0760 2023-10-06T10:45:45.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.228+0000 7fbc1259c700 1 -- 172.21.15.92:0/527028527 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c0092e0 con 0x7fbc140b0760 2023-10-06T10:45:45.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.228+0000 7fbc19056700 1 -- 172.21.15.92:0/527028527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc140b0760 msgr2=0x7fbc140b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:45.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.228+0000 7fbc19056700 1 --2- 172.21.15.92:0/527028527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc140b0760 0x7fbc140b1bc0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009f20 tx=0x7fbc0c009840 comp rx=0 tx=0).stop 2023-10-06T10:45:45.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.228+0000 7fbc19056700 1 -- 172.21.15.92:0/527028527 shutdown_connections 2023-10-06T10:45:45.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.228+0000 7fbc19056700 1 --2- 172.21.15.92:0/527028527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc140b0760 0x7fbc140b1bc0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:45.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.228+0000 7fbc19056700 1 --2- 172.21.15.92:0/527028527 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc140b49a0 0x7fbc140aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:45.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.228+0000 7fbc19056700 1 -- 172.21.15.92:0/527028527 >> 172.21.15.92:0/527028527 conn(0x7fbc140997f0 msgr2=0x7fbc1406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:45.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.228+0000 7fbc19056700 1 -- 172.21.15.92:0/527028527 shutdown_connections 2023-10-06T10:45:45.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.228+0000 7fbc19056700 1 -- 172.21.15.92:0/527028527 wait complete. 2023-10-06T10:45:45.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.229+0000 7fbc19056700 1 Processor -- start 2023-10-06T10:45:45.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.230+0000 7fbc19056700 1 -- start start 2023-10-06T10:45:45.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.230+0000 7fbc19056700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc140b0760 0x7fbc1409a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:45.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.230+0000 7fbc19056700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbc140b49a0 0x7fbc1409d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:45.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.230+0000 7fbc19056700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc14159560 0x7fbc14158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:45.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.230+0000 7fbc19056700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1409af40 con 0x7fbc140b0760 2023-10-06T10:45:45.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.230+0000 7fbc19056700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1409b220 con 0x7fbc140b49a0 2023-10-06T10:45:45.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.230+0000 7fbc19056700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1409b500 con 0x7fbc14159560 2023-10-06T10:45:45.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.230+0000 7fbc12d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc140b0760 0x7fbc1409a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:45.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.230+0000 7fbc12d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc140b0760 0x7fbc1409a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47180/0 (socket says 172.21.15.92:47180) 2023-10-06T10:45:45.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.230+0000 7fbc12d9d700 1 -- 172.21.15.92:0/893732962 learned_addr learned my addr 172.21.15.92:0/893732962 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:45.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.231+0000 7fbc1359e700 1 --2- 172.21.15.92:0/893732962 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc14159560 0x7fbc14158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:45.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.231+0000 7fbc12d9d700 1 -- 172.21.15.92:0/893732962 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc14159560 msgr2=0x7fbc14158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:45.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.231+0000 7fbc12d9d700 1 --2- 172.21.15.92:0/893732962 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc14159560 0x7fbc14158950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:45.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.231+0000 7fbc12d9d700 1 -- 172.21.15.92:0/893732962 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbc140b49a0 msgr2=0x7fbc1409d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:45.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.231+0000 7fbc12d9d700 1 --2- 172.21.15.92:0/893732962 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbc140b49a0 0x7fbc1409d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:45.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.231+0000 7fbc12d9d700 1 -- 172.21.15.92:0/893732962 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0c009ae0 con 0x7fbc140b0760 2023-10-06T10:45:45.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.231+0000 7fbc12d9d700 1 --2- 172.21.15.92:0/893732962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc140b0760 0x7fbc1409a960 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400c5b0 tx=0x7fbc04007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:45.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.231+0000 7fbc0b7fe700 1 -- 172.21.15.92:0/893732962 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc040072e0 con 0x7fbc140b0760 2023-10-06T10:45:45.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.231+0000 7fbc0b7fe700 1 -- 172.21.15.92:0/893732962 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc04007460 con 0x7fbc140b0760 2023-10-06T10:45:45.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.231+0000 7fbc19056700 1 -- 172.21.15.92:0/893732962 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1409b7e0 con 0x7fbc140b0760 2023-10-06T10:45:45.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.231+0000 7fbc0b7fe700 1 -- 172.21.15.92:0/893732962 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc040057f0 con 0x7fbc140b0760 2023-10-06T10:45:45.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.231+0000 7fbc19056700 1 -- 172.21.15.92:0/893732962 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1409de80 con 0x7fbc140b0760 2023-10-06T10:45:45.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.232+0000 7fbc19056700 1 -- 172.21.15.92:0/893732962 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc14064a90 con 0x7fbc140b0760 2023-10-06T10:45:45.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.232+0000 7fbc0b7fe700 1 -- 172.21.15.92:0/893732962 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbc04005bc0 con 0x7fbc140b0760 2023-10-06T10:45:45.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.233+0000 7fbc0b7fe700 1 --2- 172.21.15.92:0/893732962 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbbf4066f30 0x7fbbf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:45.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.233+0000 7fbc0b7fe700 1 -- 172.21.15.92:0/893732962 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fbc04083d40 con 0x7fbc140b0760 2023-10-06T10:45:45.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.233+0000 7fbc0a59c700 1 --2- 172.21.15.92:0/893732962 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbbf4066f30 0x7fbbf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:45.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.233+0000 7fbc0a59c700 1 --2- 172.21.15.92:0/893732962 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbbf4066f30 0x7fbbf40693e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc006fd0 tx=0x7fbbfc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:45.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.237+0000 7fbc0b7fe700 1 -- 172.21.15.92:0/893732962 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc04050cd0 con 0x7fbc140b0760 2023-10-06T10:45:45.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.499+0000 7fbc19056700 1 -- 172.21.15.92:0/893732962 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fbc140b17e0 con 0x7fbc140b0760 2023-10-06T10:45:45.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.499+0000 7fbc0b7fe700 1 -- 172.21.15.92:0/893732962 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1970 (secure 0 0 0) 0x7fbc04017080 con 0x7fbc140b0760 2023-10-06T10:45:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:45.501 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":466,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":704416,"bytes_used":2455654400,"bytes_avail":770638458880,"bytes_total":773094113280,"write_bytes_sec":682,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":110,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6719},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":8517},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8520}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2023-10-06T10:45:42.179532+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2023-10-06T10:45:45.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.501+0000 7fbc19056700 1 -- 172.21.15.92:0/893732962 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbbf4066f30 msgr2=0x7fbbf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:45.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.501+0000 7fbc19056700 1 --2- 172.21.15.92:0/893732962 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbbf4066f30 0x7fbbf40693e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc006fd0 tx=0x7fbbfc008040 comp rx=0 tx=0).stop 2023-10-06T10:45:45.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.501+0000 7fbc19056700 1 -- 172.21.15.92:0/893732962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc140b0760 msgr2=0x7fbc1409a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:45.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.501+0000 7fbc19056700 1 --2- 172.21.15.92:0/893732962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc140b0760 0x7fbc1409a960 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400c5b0 tx=0x7fbc04007f40 comp rx=0 tx=0).stop 2023-10-06T10:45:45.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.501+0000 7fbc19056700 1 -- 172.21.15.92:0/893732962 shutdown_connections 2023-10-06T10:45:45.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.501+0000 7fbc19056700 1 --2- 172.21.15.92:0/893732962 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbbf4066f30 0x7fbbf40693e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:45.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.501+0000 7fbc19056700 1 --2- 172.21.15.92:0/893732962 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc14159560 0x7fbc14158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:45.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.501+0000 7fbc19056700 1 --2- 172.21.15.92:0/893732962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc140b0760 0x7fbc1409a960 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:45.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.501+0000 7fbc19056700 1 --2- 172.21.15.92:0/893732962 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbc140b49a0 0x7fbc1409d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:45.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.501+0000 7fbc19056700 1 -- 172.21.15.92:0/893732962 >> 172.21.15.92:0/893732962 conn(0x7fbc140997f0 msgr2=0x7fbc140b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:45.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.501+0000 7fbc19056700 1 -- 172.21.15.92:0/893732962 shutdown_connections 2023-10-06T10:45:45.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:45.501+0000 7fbc19056700 1 -- 172.21.15.92:0/893732962 wait complete. 2023-10-06T10:45:45.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]', 'd': 'idle+waiting paths [/d1/d2/d3]'} 2023-10-06T10:45:46.516 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:45:46.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.633+0000 7f8f4eb30700 1 Processor -- start 2023-10-06T10:45:46.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.633+0000 7f8f4eb30700 1 -- start start 2023-10-06T10:45:46.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.634+0000 7f8f4eb30700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8f480b49a0 0x7f8f480aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:46.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.634+0000 7f8f4eb30700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f480b0760 0x7f8f480b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:46.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.634+0000 7f8f4eb30700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f480b0c60 con 0x7f8f48159560 2023-10-06T10:45:46.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.634+0000 7f8f4eb30700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f480b0f40 con 0x7f8f480b49a0 2023-10-06T10:45:46.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.634+0000 7f8f4eb30700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f480b1220 con 0x7f8f480b0760 2023-10-06T10:45:46.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.634+0000 7f8f47fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8f480b49a0 0x7f8f480aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:46.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.634+0000 7f8f47fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8f480b49a0 0x7f8f480aac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47196/0 (socket says 172.21.15.92:47196) 2023-10-06T10:45:46.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.634+0000 7f8f47fff700 1 -- 172.21.15.92:0/888515244 learned_addr learned my addr 172.21.15.92:0/888515244 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:46.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.634+0000 7f8f4d0cd700 1 --2- 172.21.15.92:0/888515244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f480b0760 0x7f8f480b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:46.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.634+0000 7f8f47fff700 1 -- 172.21.15.92:0/888515244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f480b0760 msgr2=0x7f8f480b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:46.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.634+0000 7f8f47fff700 1 --2- 172.21.15.92:0/888515244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f480b0760 0x7f8f480b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:46.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.634+0000 7f8f47fff700 1 -- 172.21.15.92:0/888515244 >> v1:172.21.15.124:6789/0 conn(0x7f8f48159560 legacy=0x7f8f480aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:46.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.634+0000 7f8f47fff700 1 -- 172.21.15.92:0/888515244 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f480b1500 con 0x7f8f480b49a0 2023-10-06T10:45:46.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.634+0000 7f8f47fff700 1 --2- 172.21.15.92:0/888515244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f480b49a0 0x7f8f480aac40 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009f50 tx=0x7f8f38009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222e0d82f23a5594 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:46.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.634+0000 7f8f477fe700 1 -- 172.21.15.92:0/888515244 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3800e070 con 0x7f8f480b49a0 2023-10-06T10:45:46.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.634+0000 7f8f477fe700 1 -- 172.21.15.92:0/888515244 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3800ba70 con 0x7f8f480b49a0 2023-10-06T10:45:46.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.634+0000 7f8f477fe700 1 -- 172.21.15.92:0/888515244 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f380092e0 con 0x7f8f480b49a0 2023-10-06T10:45:46.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.635+0000 7f8f4eb30700 1 -- 172.21.15.92:0/888515244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f480b49a0 msgr2=0x7f8f480aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:46.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.635+0000 7f8f4eb30700 1 --2- 172.21.15.92:0/888515244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f480b49a0 0x7f8f480aac40 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009f50 tx=0x7f8f38009840 comp rx=0 tx=0).stop 2023-10-06T10:45:46.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.635+0000 7f8f4eb30700 1 -- 172.21.15.92:0/888515244 shutdown_connections 2023-10-06T10:45:46.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.635+0000 7f8f4eb30700 1 --2- 172.21.15.92:0/888515244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f480b0760 0x7f8f480b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:46.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.635+0000 7f8f4eb30700 1 --2- 172.21.15.92:0/888515244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f480b49a0 0x7f8f480aac40 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:46.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.635+0000 7f8f4eb30700 1 -- 172.21.15.92:0/888515244 >> 172.21.15.92:0/888515244 conn(0x7f8f480997f0 msgr2=0x7f8f4806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:46.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.635+0000 7f8f4eb30700 1 -- 172.21.15.92:0/888515244 shutdown_connections 2023-10-06T10:45:46.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.635+0000 7f8f4eb30700 1 -- 172.21.15.92:0/888515244 wait complete. 2023-10-06T10:45:46.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.636+0000 7f8f4eb30700 1 Processor -- start 2023-10-06T10:45:46.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.636+0000 7f8f4eb30700 1 -- start start 2023-10-06T10:45:46.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.637+0000 7f8f4eb30700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f480b0760 0x7f8f4809a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:46.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.637+0000 7f8f4eb30700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f480b49a0 0x7f8f480ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:46.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.637+0000 7f8f4eb30700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f48159560 0x7f8f480ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:46.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.637+0000 7f8f4eb30700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4816cfe0 con 0x7f8f480b0760 2023-10-06T10:45:46.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.637+0000 7f8f4eb30700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4816d230 con 0x7f8f48159560 2023-10-06T10:45:46.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.637+0000 7f8f4eb30700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4816d510 con 0x7f8f480b49a0 2023-10-06T10:45:46.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.637+0000 7f8f4c8cc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f480b0760 0x7f8f4809a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:46.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.637+0000 7f8f4c8cc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f480b0760 0x7f8f4809a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47206/0 (socket says 172.21.15.92:47206) 2023-10-06T10:45:46.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.637+0000 7f8f4c8cc700 1 -- 172.21.15.92:0/4045298122 learned_addr learned my addr 172.21.15.92:0/4045298122 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:46.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.637+0000 7f8f4c8cc700 1 -- 172.21.15.92:0/4045298122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f480b49a0 msgr2=0x7f8f480ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:46.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.637+0000 7f8f47fff700 1 --2- 172.21.15.92:0/4045298122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f480b49a0 0x7f8f480ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:46.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.637+0000 7f8f4d0cd700 1 --2- 172.21.15.92:0/4045298122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f48159560 0x7f8f480ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:46.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.637+0000 7f8f4c8cc700 1 --2- 172.21.15.92:0/4045298122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f480b49a0 0x7f8f480ae970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:46.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.637+0000 7f8f4c8cc700 1 -- 172.21.15.92:0/4045298122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f48159560 msgr2=0x7f8f480ab3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:46.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.637+0000 7f8f4c8cc700 1 --2- 172.21.15.92:0/4045298122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f48159560 0x7f8f480ab3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:46.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.637+0000 7f8f4c8cc700 1 -- 172.21.15.92:0/4045298122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f38009ae0 con 0x7f8f480b0760 2023-10-06T10:45:46.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.637+0000 7f8f4d0cd700 1 --2- 172.21.15.92:0/4045298122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f48159560 0x7f8f480ab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:46.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.637+0000 7f8f47fff700 1 --2- 172.21.15.92:0/4045298122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f480b49a0 0x7f8f480ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:46.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.638+0000 7f8f4c8cc700 1 --2- 172.21.15.92:0/4045298122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f480b0760 0x7f8f4809a050 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000c540 tx=0x7f8f40001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:46.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.638+0000 7f8f44ff9700 1 -- 172.21.15.92:0/4045298122 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f40011590 con 0x7f8f480b0760 2023-10-06T10:45:46.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.638+0000 7f8f44ff9700 1 -- 172.21.15.92:0/4045298122 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f400116f0 con 0x7f8f480b0760 2023-10-06T10:45:46.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.638+0000 7f8f4eb30700 1 -- 172.21.15.92:0/4045298122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4816d7f0 con 0x7f8f480b0760 2023-10-06T10:45:46.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.638+0000 7f8f44ff9700 1 -- 172.21.15.92:0/4045298122 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f40011ab0 con 0x7f8f480b0760 2023-10-06T10:45:46.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.638+0000 7f8f4eb30700 1 -- 172.21.15.92:0/4045298122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4816ddb0 con 0x7f8f480b0760 2023-10-06T10:45:46.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.639+0000 7f8f4eb30700 1 -- 172.21.15.92:0/4045298122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f48169a70 con 0x7f8f480b0760 2023-10-06T10:45:46.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.639+0000 7f8f44ff9700 1 -- 172.21.15.92:0/4045298122 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8f4001b3b0 con 0x7f8f480b0760 2023-10-06T10:45:46.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.639+0000 7f8f44ff9700 1 --2- 172.21.15.92:0/4045298122 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8f28066f60 0x7f8f28069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:46.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.639+0000 7f8f44ff9700 1 -- 172.21.15.92:0/4045298122 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f8f40083c30 con 0x7f8f480b0760 2023-10-06T10:45:46.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.639+0000 7f8f47fff700 1 --2- 172.21.15.92:0/4045298122 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8f28066f60 0x7f8f28069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:46.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.640+0000 7f8f47fff700 1 --2- 172.21.15.92:0/4045298122 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8f28066f60 0x7f8f28069410 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009f50 tx=0x7f8f38018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:46.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.644+0000 7f8f44ff9700 1 -- 172.21.15.92:0/4045298122 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f48169a70 con 0x7f8f480b0760 2023-10-06T10:45:46.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.870+0000 7f8f4eb30700 1 -- 172.21.15.92:0/4045298122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f48169a70 con 0x7f8f480b0760 2023-10-06T10:45:46.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.870+0000 7f8f44ff9700 1 -- 172.21.15.92:0/4045298122 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4646 (secure 0 0 0) 0x7f8f48169a70 con 0x7f8f480b0760 2023-10-06T10:45:46.872 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:46.872 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":110,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2016052382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2016052382},{"type":"v1","addr":"172.21.15.124:6834","nonce":2016052382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:44:51.106486+0000","modified":"2023-10-06T10:45:34.353086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8520,"mds_1":8517,"mds_2":6719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6719":{"gid":6719,"name":"d","rank":2,"incarnation":104,"state":"up:active","state_seq":13,"addr":"172.21.15.124:6835/2307521923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2307521923},{"type":"v1","addr":"172.21.15.124:6835","nonce":2307521923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8517":{"gid":8517,"name":"a","rank":1,"incarnation":102,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6836/1979826278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979826278},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979826278}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6837/2298348426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2298348426},{"type":"v1","addr":"172.21.15.92:6837","nonce":2298348426}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T10:45:46.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.872+0000 7f8f4eb30700 1 -- 172.21.15.92:0/4045298122 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8f28066f60 msgr2=0x7f8f28069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:46.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.872+0000 7f8f4eb30700 1 --2- 172.21.15.92:0/4045298122 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8f28066f60 0x7f8f28069410 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009f50 tx=0x7f8f38018000 comp rx=0 tx=0).stop 2023-10-06T10:45:46.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.873+0000 7f8f4eb30700 1 -- 172.21.15.92:0/4045298122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f480b0760 msgr2=0x7f8f4809a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:46.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.873+0000 7f8f4eb30700 1 --2- 172.21.15.92:0/4045298122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f480b0760 0x7f8f4809a050 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000c540 tx=0x7f8f40001070 comp rx=0 tx=0).stop 2023-10-06T10:45:46.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.873+0000 7f8f4eb30700 1 -- 172.21.15.92:0/4045298122 shutdown_connections 2023-10-06T10:45:46.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.873+0000 7f8f4eb30700 1 --2- 172.21.15.92:0/4045298122 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8f28066f60 0x7f8f28069410 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:46.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.873+0000 7f8f4eb30700 1 --2- 172.21.15.92:0/4045298122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f48159560 0x7f8f480ab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:46.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.873+0000 7f8f4eb30700 1 --2- 172.21.15.92:0/4045298122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f480b0760 0x7f8f4809a050 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:46.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.873+0000 7f8f4eb30700 1 --2- 172.21.15.92:0/4045298122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f480b49a0 0x7f8f480ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:46.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.873+0000 7f8f4eb30700 1 -- 172.21.15.92:0/4045298122 >> 172.21.15.92:0/4045298122 conn(0x7f8f480997f0 msgr2=0x7f8f480b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:46.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.873+0000 7f8f4eb30700 1 -- 172.21.15.92:0/4045298122 shutdown_connections 2023-10-06T10:45:46.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.873+0000 7f8f4eb30700 1 -- 172.21.15.92:0/4045298122 wait complete. 2023-10-06T10:45:46.876 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 110 2023-10-06T10:45:46.886 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2023-10-06T10:45:46.886 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T10:45:46.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.996+0000 7f8327e22700 1 Processor -- start 2023-10-06T10:45:46.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.997+0000 7f8327e22700 1 -- start start 2023-10-06T10:45:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.997+0000 7f8327e22700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8320159560 0x7f83200aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.997+0000 7f8327e22700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f83200b0760 0x7f83200b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.997+0000 7f8327e22700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83200b0c60 con 0x7f83200b49a0 2023-10-06T10:45:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.997+0000 7f8327e22700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83200b0f40 con 0x7f83200b0760 2023-10-06T10:45:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.997+0000 7f8327e22700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83200b1220 con 0x7f8320159560 2023-10-06T10:45:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.998+0000 7f83263bf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f83200b0760 0x7f83200b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.998+0000 7f83263bf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f83200b0760 0x7f83200b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47208/0 (socket says 172.21.15.92:47208) 2023-10-06T10:45:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.998+0000 7f83263bf700 1 -- 172.21.15.92:0/203932938 learned_addr learned my addr 172.21.15.92:0/203932938 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.998+0000 7f8325bbe700 1 --2- 172.21.15.92:0/203932938 >> v2:172.21.15.124:3300/0 conn(0x7f8320159560 0x7f83200aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.998+0000 7f83263bf700 1 -- 172.21.15.92:0/203932938 >> v2:172.21.15.124:3300/0 conn(0x7f8320159560 msgr2=0x7f83200aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.998+0000 7f83263bf700 1 --2- 172.21.15.92:0/203932938 >> v2:172.21.15.124:3300/0 conn(0x7f8320159560 0x7f83200aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.998+0000 7f83263bf700 1 -- 172.21.15.92:0/203932938 >> v1:172.21.15.124:6789/0 conn(0x7f83200b49a0 legacy=0x7f83200aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.998+0000 7f83263bf700 1 -- 172.21.15.92:0/203932938 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83200b1500 con 0x7f83200b0760 2023-10-06T10:45:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.998+0000 7f83263bf700 1 --2- 172.21.15.92:0/203932938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83200b0760 0x7f83200b1bc0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f83180099b0 tx=0x7f8318009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755e12a990bfe221 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.998+0000 7f8324bbc700 1 -- 172.21.15.92:0/203932938 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8318009ea0 con 0x7f83200b0760 2023-10-06T10:45:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.998+0000 7f8324bbc700 1 -- 172.21.15.92:0/203932938 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831800b820 con 0x7f83200b0760 2023-10-06T10:45:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.998+0000 7f8324bbc700 1 -- 172.21.15.92:0/203932938 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831800bbc0 con 0x7f83200b0760 2023-10-06T10:45:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.999+0000 7f8327e22700 1 -- 172.21.15.92:0/203932938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83200b0760 msgr2=0x7f83200b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.999+0000 7f8327e22700 1 --2- 172.21.15.92:0/203932938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83200b0760 0x7f83200b1bc0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f83180099b0 tx=0x7f8318009250 comp rx=0 tx=0).stop 2023-10-06T10:45:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.999+0000 7f8327e22700 1 -- 172.21.15.92:0/203932938 shutdown_connections 2023-10-06T10:45:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.999+0000 7f8327e22700 1 --2- 172.21.15.92:0/203932938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83200b0760 0x7f83200b1bc0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.999+0000 7f8327e22700 1 --2- 172.21.15.92:0/203932938 >> v2:172.21.15.124:3300/0 conn(0x7f8320159560 0x7f83200aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.999+0000 7f8327e22700 1 -- 172.21.15.92:0/203932938 >> 172.21.15.92:0/203932938 conn(0x7f83200997f0 msgr2=0x7f832006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.999+0000 7f8327e22700 1 -- 172.21.15.92:0/203932938 shutdown_connections 2023-10-06T10:45:47.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:46.999+0000 7f8327e22700 1 -- 172.21.15.92:0/203932938 wait complete. 2023-10-06T10:45:47.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.000+0000 7f8327e22700 1 Processor -- start 2023-10-06T10:45:47.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.000+0000 7f8327e22700 1 -- start start 2023-10-06T10:45:47.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.001+0000 7f8327e22700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83200b0760 0x7f83200aab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:47.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.001+0000 7f8327e22700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83200b49a0 0x7f83200aea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:47.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.001+0000 7f8327e22700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8320159560 0x7f83200acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:47.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.001+0000 7f8327e22700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83201599d0 con 0x7f83200b0760 2023-10-06T10:45:47.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.001+0000 7f8327e22700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83200abcb0 con 0x7f8320159560 2023-10-06T10:45:47.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.001+0000 7f8327e22700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83200abf50 con 0x7f83200b49a0 2023-10-06T10:45:47.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.001+0000 7f8325bbe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83200b0760 0x7f83200aab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:47.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.001+0000 7f8325bbe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83200b0760 0x7f83200aab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47222/0 (socket says 172.21.15.92:47222) 2023-10-06T10:45:47.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.001+0000 7f8325bbe700 1 -- 172.21.15.92:0/3222944645 learned_addr learned my addr 172.21.15.92:0/3222944645 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:47.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.001+0000 7f83253bd700 1 --2- 172.21.15.92:0/3222944645 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83200b49a0 0x7f83200aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:47.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.001+0000 7f83263bf700 1 --2- 172.21.15.92:0/3222944645 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8320159560 0x7f83200acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:47.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.001+0000 7f8325bbe700 1 -- 172.21.15.92:0/3222944645 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83200b49a0 msgr2=0x7f83200aea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:47.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.001+0000 7f8325bbe700 1 --2- 172.21.15.92:0/3222944645 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83200b49a0 0x7f83200aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.001+0000 7f8325bbe700 1 -- 172.21.15.92:0/3222944645 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8320159560 msgr2=0x7f83200acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:47.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.001+0000 7f8325bbe700 1 --2- 172.21.15.92:0/3222944645 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8320159560 0x7f83200acbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.001+0000 7f8325bbe700 1 -- 172.21.15.92:0/3222944645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83180094f0 con 0x7f83200b0760 2023-10-06T10:45:47.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.001+0000 7f8325bbe700 1 --2- 172.21.15.92:0/3222944645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83200b0760 0x7f83200aab30 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f831c009ea0 tx=0x7f831c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:47.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.001+0000 7f8315ffb700 1 -- 172.21.15.92:0/3222944645 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c00cc00 con 0x7f83200b0760 2023-10-06T10:45:47.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.001+0000 7f8315ffb700 1 -- 172.21.15.92:0/3222944645 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c00f040 con 0x7f83200b0760 2023-10-06T10:45:47.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.001+0000 7f8315ffb700 1 -- 172.21.15.92:0/3222944645 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c014480 con 0x7f83200b0760 2023-10-06T10:45:47.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.001+0000 7f8327e22700 1 -- 172.21.15.92:0/3222944645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8320159c40 con 0x7f83200b0760 2023-10-06T10:45:47.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.001+0000 7f8327e22700 1 -- 172.21.15.92:0/3222944645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83200aef70 con 0x7f83200b0760 2023-10-06T10:45:47.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.002+0000 7f8327e22700 1 -- 172.21.15.92:0/3222944645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8320064a90 con 0x7f83200b0760 2023-10-06T10:45:47.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.003+0000 7f8315ffb700 1 -- 172.21.15.92:0/3222944645 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f831c0147f0 con 0x7f83200b0760 2023-10-06T10:45:47.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.003+0000 7f8315ffb700 1 --2- 172.21.15.92:0/3222944645 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8304066f60 0x7f8304069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:47.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.003+0000 7f8315ffb700 1 -- 172.21.15.92:0/3222944645 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f831c084fa0 con 0x7f83200b0760 2023-10-06T10:45:47.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.003+0000 7f83253bd700 1 --2- 172.21.15.92:0/3222944645 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8304066f60 0x7f8304069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:47.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.003+0000 7f83253bd700 1 --2- 172.21.15.92:0/3222944645 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8304066f60 0x7f8304069410 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8320099f20 tx=0x7f8310009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:47.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.007+0000 7f8315ffb700 1 -- 172.21.15.92:0/3222944645 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f831c051f30 con 0x7f83200b0760 2023-10-06T10:45:47.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.266+0000 7f8327e22700 1 -- 172.21.15.92:0/3222944645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f83200b17e0 con 0x7f83200b0760 2023-10-06T10:45:47.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.266+0000 7f8315ffb700 1 -- 172.21.15.92:0/3222944645 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1701 (secure 0 0 0) 0x7f831c055550 con 0x7f83200b0760 2023-10-06T10:45:47.268 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:47.268 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":468,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1696588689,"num_in_osds":8,"osd_in_since":1696588680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":70,"data_bytes":710079,"bytes_used":2455662592,"bytes_avail":770638450688,"bytes_total":773094113280,"write_bytes_sec":1279,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":110,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6719},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":8517},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8520}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":28,"modified":"2023-10-06T10:45:46.180740+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T10:45:47.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.268+0000 7f8327e22700 1 -- 172.21.15.92:0/3222944645 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8304066f60 msgr2=0x7f8304069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:47.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.268+0000 7f8327e22700 1 --2- 172.21.15.92:0/3222944645 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8304066f60 0x7f8304069410 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8320099f20 tx=0x7f8310009040 comp rx=0 tx=0).stop 2023-10-06T10:45:47.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.268+0000 7f8327e22700 1 -- 172.21.15.92:0/3222944645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83200b0760 msgr2=0x7f83200aab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:47.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.268+0000 7f8327e22700 1 --2- 172.21.15.92:0/3222944645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83200b0760 0x7f83200aab30 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f831c009ea0 tx=0x7f831c00b040 comp rx=0 tx=0).stop 2023-10-06T10:45:47.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.268+0000 7f8327e22700 1 -- 172.21.15.92:0/3222944645 shutdown_connections 2023-10-06T10:45:47.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.268+0000 7f8327e22700 1 --2- 172.21.15.92:0/3222944645 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8304066f60 0x7f8304069410 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.268+0000 7f8327e22700 1 --2- 172.21.15.92:0/3222944645 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8320159560 0x7f83200acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.268+0000 7f8327e22700 1 --2- 172.21.15.92:0/3222944645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83200b0760 0x7f83200aab30 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.268+0000 7f8327e22700 1 --2- 172.21.15.92:0/3222944645 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83200b49a0 0x7f83200aea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.268+0000 7f8327e22700 1 -- 172.21.15.92:0/3222944645 >> 172.21.15.92:0/3222944645 conn(0x7f83200997f0 msgr2=0x7f83200cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:47.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.268+0000 7f8327e22700 1 -- 172.21.15.92:0/3222944645 shutdown_connections 2023-10-06T10:45:47.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.268+0000 7f8327e22700 1 -- 172.21.15.92:0/3222944645 wait complete. 2023-10-06T10:45:47.282 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2023-10-06T10:45:47.282 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T10:45:47.282 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:45:47.318 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T10:45:47.358 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T10:45:47.358 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T10:45:47.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T10:45:47.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T10:45:47.398 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T10:45:47.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:45:47.398 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T10:45:47.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:45:47.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.606+0000 7f3573fff700 1 Processor -- start 2023-10-06T10:45:47.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.606+0000 7f3573fff700 1 -- start start 2023-10-06T10:45:47.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.607+0000 7f3573fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35740cbc30 0x7f35740cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:47.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.607+0000 7f3573fff700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f35740b2100 0x7f35740b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:47.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.607+0000 7f3573fff700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35740cb7a0 con 0x7f3574159440 2023-10-06T10:45:47.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.607+0000 7f3573fff700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35740b0860 con 0x7f35740b2100 2023-10-06T10:45:47.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.607+0000 7f3573fff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35740cb150 con 0x7f35740cbc30 2023-10-06T10:45:47.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.607+0000 7f35737fe700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f35740b2100 0x7f35740b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:47.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.607+0000 7f356a7fc700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35740cbc30 0x7f35740cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:47.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.607+0000 7f3572ffd700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f3574159440 0x7f35740b3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:51240/0 (socket says 172.21.15.92:51240) 2023-10-06T10:45:47.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.607+0000 7f3572ffd700 1 -- 172.21.15.92:0/2903063852 learned_addr learned my addr 172.21.15.92:0/2903063852 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:47.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.607+0000 7f35727fc700 1 -- 172.21.15.92:0/2903063852 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3033109790 0 0) 0x7f35740cb7a0 con 0x7f3574159440 2023-10-06T10:45:47.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.607+0000 7f35727fc700 1 -- 172.21.15.92:0/2903063852 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3560003570 con 0x7f3574159440 2023-10-06T10:45:47.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.608+0000 7f356a7fc700 1 -- 172.21.15.92:0/2903063852 >> v2:172.21.15.124:3300/0 conn(0x7f35740b2100 msgr2=0x7f35740b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:47.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.608+0000 7f356a7fc700 1 --2- 172.21.15.92:0/2903063852 >> v2:172.21.15.124:3300/0 conn(0x7f35740b2100 0x7f35740b2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.608+0000 7f356a7fc700 1 -- 172.21.15.92:0/2903063852 >> v1:172.21.15.124:6789/0 conn(0x7f3574159440 legacy=0x7f35740b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:47.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.608+0000 7f356a7fc700 1 -- 172.21.15.92:0/2903063852 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35740cb430 con 0x7f35740cbc30 2023-10-06T10:45:47.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.608+0000 7f35737fe700 1 --2- 172.21.15.92:0/2903063852 >> v2:172.21.15.124:3300/0 conn(0x7f35740b2100 0x7f35740b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:45:47.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.608+0000 7f356a7fc700 1 --2- 172.21.15.92:0/2903063852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35740cbc30 0x7f35740cc080 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f355c0098e0 tx=0x7f355c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62a71134021a4278 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:47.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.608+0000 7f35727fc700 1 -- 172.21.15.92:0/2903063852 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c011630 con 0x7f35740cbc30 2023-10-06T10:45:47.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.608+0000 7f35727fc700 1 -- 172.21.15.92:0/2903063852 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355c011790 con 0x7f35740cbc30 2023-10-06T10:45:47.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.608+0000 7f35727fc700 1 -- 172.21.15.92:0/2903063852 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c011b30 con 0x7f35740cbc30 2023-10-06T10:45:47.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.608+0000 7f3573fff700 1 -- 172.21.15.92:0/2903063852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35740cbc30 msgr2=0x7f35740cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:47.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.608+0000 7f3573fff700 1 --2- 172.21.15.92:0/2903063852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35740cbc30 0x7f35740cc080 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f355c0098e0 tx=0x7f355c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.608+0000 7f3573fff700 1 -- 172.21.15.92:0/2903063852 shutdown_connections 2023-10-06T10:45:47.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.608+0000 7f3573fff700 1 --2- 172.21.15.92:0/2903063852 >> v2:172.21.15.124:3300/0 conn(0x7f35740b2100 0x7f35740b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.608+0000 7f3573fff700 1 --2- 172.21.15.92:0/2903063852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35740cbc30 0x7f35740cc080 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.608+0000 7f3573fff700 1 -- 172.21.15.92:0/2903063852 >> 172.21.15.92:0/2903063852 conn(0x7f357406ece0 msgr2=0x7f357406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:47.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.608+0000 7f3573fff700 1 -- 172.21.15.92:0/2903063852 shutdown_connections 2023-10-06T10:45:47.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.608+0000 7f3573fff700 1 -- 172.21.15.92:0/2903063852 wait complete. 2023-10-06T10:45:47.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.610+0000 7f3573fff700 1 Processor -- start 2023-10-06T10:45:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.610+0000 7f3573fff700 1 -- start start 2023-10-06T10:45:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.610+0000 7f3573fff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35740b2100 0x7f35740c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.610+0000 7f3573fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35740cbc30 0x7f3574153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.610+0000 7f3573fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3574159440 0x7f35740c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.610+0000 7f3573fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35740c7d20 con 0x7f35740cbc30 2023-10-06T10:45:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.610+0000 7f3573fff700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35740c8000 con 0x7f35740b2100 2023-10-06T10:45:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.610+0000 7f3573fff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35740c82e0 con 0x7f3574159440 2023-10-06T10:45:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.611+0000 7f356a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35740cbc30 0x7f3574153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.611+0000 7f356a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35740cbc30 0x7f3574153e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47236/0 (socket says 172.21.15.92:47236) 2023-10-06T10:45:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.611+0000 7f356a7fc700 1 -- 172.21.15.92:0/3719449444 learned_addr learned my addr 172.21.15.92:0/3719449444 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.611+0000 7f3572ffd700 1 --2- 172.21.15.92:0/3719449444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35740b2100 0x7f35740c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.611+0000 7f35737fe700 1 --2- 172.21.15.92:0/3719449444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3574159440 0x7f35740c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.611+0000 7f356a7fc700 1 -- 172.21.15.92:0/3719449444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3574159440 msgr2=0x7f35740c72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.611+0000 7f356a7fc700 1 --2- 172.21.15.92:0/3719449444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3574159440 0x7f35740c72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.611+0000 7f356a7fc700 1 -- 172.21.15.92:0/3719449444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35740b2100 msgr2=0x7f35740c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:47.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.611+0000 7f356a7fc700 1 --2- 172.21.15.92:0/3719449444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35740b2100 0x7f35740c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.611+0000 7f356a7fc700 1 -- 172.21.15.92:0/3719449444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c009a20 con 0x7f35740cbc30 2023-10-06T10:45:47.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.611+0000 7f356a7fc700 1 --2- 172.21.15.92:0/3719449444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35740cbc30 0x7f3574153e90 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f35740cb290 tx=0x7f355c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:47.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.611+0000 7f356b7fe700 1 -- 172.21.15.92:0/3719449444 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c011b10 con 0x7f35740cbc30 2023-10-06T10:45:47.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.611+0000 7f356b7fe700 1 -- 172.21.15.92:0/3719449444 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355c000cc0 con 0x7f35740cbc30 2023-10-06T10:45:47.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.611+0000 7f3573fff700 1 -- 172.21.15.92:0/3719449444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355c009420 con 0x7f35740cbc30 2023-10-06T10:45:47.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.611+0000 7f356b7fe700 1 -- 172.21.15.92:0/3719449444 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c023600 con 0x7f35740cbc30 2023-10-06T10:45:47.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.611+0000 7f3573fff700 1 -- 172.21.15.92:0/3719449444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35740c88a0 con 0x7f35740cbc30 2023-10-06T10:45:47.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.612+0000 7f3573fff700 1 -- 172.21.15.92:0/3719449444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35740cd800 con 0x7f35740cbc30 2023-10-06T10:45:47.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.613+0000 7f356b7fe700 1 -- 172.21.15.92:0/3719449444 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f355c02c3b0 con 0x7f35740cbc30 2023-10-06T10:45:47.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.613+0000 7f356b7fe700 1 --2- 172.21.15.92:0/3719449444 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3554066f30 0x7f35540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:47.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.613+0000 7f356b7fe700 1 -- 172.21.15.92:0/3719449444 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f355c085b70 con 0x7f35740cbc30 2023-10-06T10:45:47.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.613+0000 7f3572ffd700 1 --2- 172.21.15.92:0/3719449444 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3554066f30 0x7f35540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:47.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.613+0000 7f3572ffd700 1 --2- 172.21.15.92:0/3719449444 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3554066f30 0x7f35540693e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3564007fd0 tx=0x7f3564009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:47.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.617+0000 7f356b7fe700 1 -- 172.21.15.92:0/3719449444 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f355c052c40 con 0x7f35740cbc30 2023-10-06T10:45:47.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.844+0000 7f3573fff700 1 -- 172.21.15.92:0/3719449444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f357405f230 con 0x7f35740cbc30 2023-10-06T10:45:47.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.844+0000 7f356b7fe700 1 -- 172.21.15.92:0/3719449444 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4646 (secure 0 0 0) 0x7f355c015090 con 0x7f35740cbc30 2023-10-06T10:45:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:47.847 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":110,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2016052382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2016052382},{"type":"v1","addr":"172.21.15.124:6834","nonce":2016052382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:44:51.106486+0000","modified":"2023-10-06T10:45:34.353086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8520,"mds_1":8517,"mds_2":6719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6719":{"gid":6719,"name":"d","rank":2,"incarnation":104,"state":"up:active","state_seq":13,"addr":"172.21.15.124:6835/2307521923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2307521923},{"type":"v1","addr":"172.21.15.124:6835","nonce":2307521923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8517":{"gid":8517,"name":"a","rank":1,"incarnation":102,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6836/1979826278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979826278},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979826278}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6837/2298348426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2298348426},{"type":"v1","addr":"172.21.15.92:6837","nonce":2298348426}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T10:45:47.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.846+0000 7f3573fff700 1 -- 172.21.15.92:0/3719449444 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3554066f30 msgr2=0x7f35540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:47.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.846+0000 7f3573fff700 1 --2- 172.21.15.92:0/3719449444 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3554066f30 0x7f35540693e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3564007fd0 tx=0x7f3564009040 comp rx=0 tx=0).stop 2023-10-06T10:45:47.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.846+0000 7f3573fff700 1 -- 172.21.15.92:0/3719449444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35740cbc30 msgr2=0x7f3574153e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:47.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.846+0000 7f3573fff700 1 --2- 172.21.15.92:0/3719449444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35740cbc30 0x7f3574153e90 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f35740cb290 tx=0x7f355c017000 comp rx=0 tx=0).stop 2023-10-06T10:45:47.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.846+0000 7f3573fff700 1 -- 172.21.15.92:0/3719449444 shutdown_connections 2023-10-06T10:45:47.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.846+0000 7f3573fff700 1 --2- 172.21.15.92:0/3719449444 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3554066f30 0x7f35540693e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.846+0000 7f3573fff700 1 --2- 172.21.15.92:0/3719449444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3574159440 0x7f35740c72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.846+0000 7f3573fff700 1 --2- 172.21.15.92:0/3719449444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35740b2100 0x7f35740c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.846+0000 7f3573fff700 1 --2- 172.21.15.92:0/3719449444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35740cbc30 0x7f3574153e90 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.846+0000 7f3573fff700 1 -- 172.21.15.92:0/3719449444 >> 172.21.15.92:0/3719449444 conn(0x7f357406ece0 msgr2=0x7f3574155df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:47.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.846+0000 7f3573fff700 1 -- 172.21.15.92:0/3719449444 shutdown_connections 2023-10-06T10:45:47.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.846+0000 7f3573fff700 1 -- 172.21.15.92:0/3719449444 wait complete. 2023-10-06T10:45:47.850 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 110 2023-10-06T10:45:47.860 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:45:47.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.972+0000 7f2b5cf91700 1 Processor -- start 2023-10-06T10:45:47.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.972+0000 7f2b5cf91700 1 -- start start 2023-10-06T10:45:47.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.973+0000 7f2b5cf91700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2b580b2810 0x7f2b580cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:47.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.973+0000 7f2b5cf91700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b580cb7a0 con 0x7f2b580cbc30 2023-10-06T10:45:47.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.973+0000 7f2b5cf91700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b580b0860 con 0x7f2b58159440 2023-10-06T10:45:47.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.973+0000 7f2b5cf91700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b580cb150 con 0x7f2b580b2810 2023-10-06T10:45:47.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.973+0000 7f2b57fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2b580b2810 0x7f2b580cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.973+0000 7f2b56ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2b580cbc30 0x7f2b580b2100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58770/0 (socket says 172.21.15.92:58770) 2023-10-06T10:45:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.973+0000 7f2b56ffd700 1 -- 172.21.15.92:0/3899184557 learned_addr learned my addr 172.21.15.92:0/3899184557 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.973+0000 7f2b57fff700 1 -- 172.21.15.92:0/3899184557 >> v1:172.21.15.124:6789/0 conn(0x7f2b58159440 legacy=0x7f2b580b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.973+0000 7f2b57fff700 1 -- 172.21.15.92:0/3899184557 >> v1:172.21.15.92:6789/0 conn(0x7f2b580cbc30 legacy=0x7f2b580b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.973+0000 7f2b57fff700 1 -- 172.21.15.92:0/3899184557 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b580cb430 con 0x7f2b580b2810 2023-10-06T10:45:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.973+0000 7f2b567fc700 1 -- 172.21.15.92:0/3899184557 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901522500 0 0) 0x7f2b580cb7a0 con 0x7f2b580cbc30 2023-10-06T10:45:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.973+0000 7f2b57fff700 1 --2- 172.21.15.92:0/3899184557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b580b2810 0x7f2b580cc080 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f2b500098e0 tx=0x7f2b500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca6313f9342d87 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.974+0000 7f2b567fc700 1 -- 172.21.15.92:0/3899184557 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b50009e80 con 0x7f2b580b2810 2023-10-06T10:45:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.974+0000 7f2b567fc700 1 -- 172.21.15.92:0/3899184557 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5000b860 con 0x7f2b580b2810 2023-10-06T10:45:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.974+0000 7f2b567fc700 1 -- 172.21.15.92:0/3899184557 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5000bbe0 con 0x7f2b580b2810 2023-10-06T10:45:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.974+0000 7f2b5cf91700 1 -- 172.21.15.92:0/3899184557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b580b2810 msgr2=0x7f2b580cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.974+0000 7f2b5cf91700 1 --2- 172.21.15.92:0/3899184557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b580b2810 0x7f2b580cc080 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f2b500098e0 tx=0x7f2b500091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.974+0000 7f2b5cf91700 1 -- 172.21.15.92:0/3899184557 shutdown_connections 2023-10-06T10:45:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.974+0000 7f2b5cf91700 1 --2- 172.21.15.92:0/3899184557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b580b2810 0x7f2b580cc080 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.974+0000 7f2b5cf91700 1 -- 172.21.15.92:0/3899184557 >> 172.21.15.92:0/3899184557 conn(0x7f2b5806ece0 msgr2=0x7f2b5806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.974+0000 7f2b5cf91700 1 -- 172.21.15.92:0/3899184557 shutdown_connections 2023-10-06T10:45:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.974+0000 7f2b5cf91700 1 -- 172.21.15.92:0/3899184557 wait complete. 2023-10-06T10:45:47.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.975+0000 7f2b5cf91700 1 Processor -- start 2023-10-06T10:45:47.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.975+0000 7f2b5cf91700 1 -- start start 2023-10-06T10:45:47.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.976+0000 7f2b5cf91700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b580b2810 0x7f2b58153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:47.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.976+0000 7f2b5cf91700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b580cbc30 0x7f2b58154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:47.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.976+0000 7f2b5cf91700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b58159440 0x7f2b581545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:47.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.976+0000 7f2b5cf91700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b58169b60 con 0x7f2b580cbc30 2023-10-06T10:45:47.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.976+0000 7f2b5cf91700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b580c9670 con 0x7f2b580b2810 2023-10-06T10:45:47.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.976+0000 7f2b5cf91700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b58169290 con 0x7f2b58159440 2023-10-06T10:45:47.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.976+0000 7f2b56ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b580cbc30 0x7f2b58154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:47.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.976+0000 7f2b56ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b580cbc30 0x7f2b58154090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47244/0 (socket says 172.21.15.92:47244) 2023-10-06T10:45:47.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.976+0000 7f2b56ffd700 1 -- 172.21.15.92:0/2626516244 learned_addr learned my addr 172.21.15.92:0/2626516244 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:47.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.976+0000 7f2b56ffd700 1 -- 172.21.15.92:0/2626516244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b58159440 msgr2=0x7f2b581545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:47.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.976+0000 7f2b577fe700 1 --2- 172.21.15.92:0/2626516244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b580b2810 0x7f2b58153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:47.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.976+0000 7f2b56ffd700 1 --2- 172.21.15.92:0/2626516244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b58159440 0x7f2b581545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.976+0000 7f2b56ffd700 1 -- 172.21.15.92:0/2626516244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b580b2810 msgr2=0x7f2b58153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:47.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.976+0000 7f2b56ffd700 1 --2- 172.21.15.92:0/2626516244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b580b2810 0x7f2b58153890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:47.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.977+0000 7f2b56ffd700 1 -- 172.21.15.92:0/2626516244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b50009420 con 0x7f2b580cbc30 2023-10-06T10:45:47.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.977+0000 7f2b56ffd700 1 --2- 172.21.15.92:0/2626516244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b580cbc30 0x7f2b58154090 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c00c810 tx=0x7f2b4c007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:47.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.977+0000 7f2b3f7fe700 1 -- 172.21.15.92:0/2626516244 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c005c00 con 0x7f2b580cbc30 2023-10-06T10:45:47.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.977+0000 7f2b3f7fe700 1 -- 172.21.15.92:0/2626516244 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4c005d60 con 0x7f2b580cbc30 2023-10-06T10:45:47.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.977+0000 7f2b5cf91700 1 -- 172.21.15.92:0/2626516244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b580c9f30 con 0x7f2b580cbc30 2023-10-06T10:45:47.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.977+0000 7f2b3f7fe700 1 -- 172.21.15.92:0/2626516244 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c007340 con 0x7f2b580cbc30 2023-10-06T10:45:47.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.977+0000 7f2b5cf91700 1 -- 172.21.15.92:0/2626516244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b581697f0 con 0x7f2b580cbc30 2023-10-06T10:45:47.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.978+0000 7f2b5cf91700 1 -- 172.21.15.92:0/2626516244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b5805f230 con 0x7f2b580cbc30 2023-10-06T10:45:47.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.978+0000 7f2b3f7fe700 1 -- 172.21.15.92:0/2626516244 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2b4c01b400 con 0x7f2b580cbc30 2023-10-06T10:45:47.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.978+0000 7f2b3f7fe700 1 --2- 172.21.15.92:0/2626516244 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2b44066f30 0x7f2b440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:47.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.979+0000 7f2b3f7fe700 1 -- 172.21.15.92:0/2626516244 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f2b4c0842c0 con 0x7f2b580cbc30 2023-10-06T10:45:47.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.979+0000 7f2b577fe700 1 --2- 172.21.15.92:0/2626516244 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2b44066f30 0x7f2b440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:47.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.979+0000 7f2b577fe700 1 --2- 172.21.15.92:0/2626516244 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2b44066f30 0x7f2b440693e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2b58154f10 tx=0x7f2b48007420 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:47.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:47.983+0000 7f2b3f7fe700 1 -- 172.21.15.92:0/2626516244 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b4c0512d0 con 0x7f2b580cbc30 2023-10-06T10:45:48.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.166+0000 7f2b5cf91700 1 -- 172.21.15.92:0/2626516244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b58169ca0 con 0x7f2b580cbc30 2023-10-06T10:45:48.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.167+0000 7f2b3f7fe700 1 -- 172.21.15.92:0/2626516244 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+17181 (secure 0 0 0) 0x7f2b4c0548f0 con 0x7f2b580cbc30 2023-10-06T10:45:48.169 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:48.169 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":104,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:45:07.376972+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:44:48.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-06T10:44:50.072416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.4","mappings":[{"from":0,"to":5}]},{"pgid":"17.5","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:45:48.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.169+0000 7f2b5cf91700 1 -- 172.21.15.92:0/2626516244 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2b44066f30 msgr2=0x7f2b440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:48.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.169+0000 7f2b5cf91700 1 --2- 172.21.15.92:0/2626516244 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2b44066f30 0x7f2b440693e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2b58154f10 tx=0x7f2b48007420 comp rx=0 tx=0).stop 2023-10-06T10:45:48.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.169+0000 7f2b5cf91700 1 -- 172.21.15.92:0/2626516244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b580cbc30 msgr2=0x7f2b58154090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:48.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.169+0000 7f2b5cf91700 1 --2- 172.21.15.92:0/2626516244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b580cbc30 0x7f2b58154090 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c00c810 tx=0x7f2b4c007d90 comp rx=0 tx=0).stop 2023-10-06T10:45:48.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.169+0000 7f2b5cf91700 1 -- 172.21.15.92:0/2626516244 shutdown_connections 2023-10-06T10:45:48.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.169+0000 7f2b5cf91700 1 --2- 172.21.15.92:0/2626516244 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2b44066f30 0x7f2b440693e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.169+0000 7f2b5cf91700 1 --2- 172.21.15.92:0/2626516244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b58159440 0x7f2b581545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.169+0000 7f2b5cf91700 1 --2- 172.21.15.92:0/2626516244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b580b2810 0x7f2b58153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.169+0000 7f2b5cf91700 1 --2- 172.21.15.92:0/2626516244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b580cbc30 0x7f2b58154090 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.169+0000 7f2b5cf91700 1 -- 172.21.15.92:0/2626516244 >> 172.21.15.92:0/2626516244 conn(0x7f2b5806ece0 msgr2=0x7f2b58155dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:48.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.169+0000 7f2b5cf91700 1 -- 172.21.15.92:0/2626516244 shutdown_connections 2023-10-06T10:45:48.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.169+0000 7f2b5cf91700 1 -- 172.21.15.92:0/2626516244 wait complete. 2023-10-06T10:45:48.183 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T10:38:09.680559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T10:38:10.120786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T10:44:48.758515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T10:44:50.072416+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T10:45:48.183 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T10:45:48.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.298+0000 7fbfdb60b700 1 Processor -- start 2023-10-06T10:45:48.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.299+0000 7fbfdb60b700 1 -- start start 2023-10-06T10:45:48.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.299+0000 7fbfdb60b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbfd4159530 0x7fbfd4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:48.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.299+0000 7fbfdb60b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfd406a2f0 0x7fbfd4155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:48.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.299+0000 7fbfdb60b700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfd406a750 con 0x7fbfd4155800 2023-10-06T10:45:48.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.299+0000 7fbfdb60b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd406f020 con 0x7fbfd4159530 2023-10-06T10:45:48.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.299+0000 7fbfdb60b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd41599c0 con 0x7fbfd406a2f0 2023-10-06T10:45:48.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.299+0000 7fbfd8ba6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbfd4159530 0x7fbfd4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:48.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.299+0000 7fbfd8ba6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbfd4159530 0x7fbfd4069db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47258/0 (socket says 172.21.15.92:47258) 2023-10-06T10:45:48.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.299+0000 7fbfd8ba6700 1 -- 172.21.15.92:0/1579783757 learned_addr learned my addr 172.21.15.92:0/1579783757 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:48.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.300+0000 7fbfd9ba8700 1 --2- 172.21.15.92:0/1579783757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfd406a2f0 0x7fbfd4155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:48.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.300+0000 7fbfd8ba6700 1 -- 172.21.15.92:0/1579783757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfd406a2f0 msgr2=0x7fbfd4155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:48.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.300+0000 7fbfd8ba6700 1 --2- 172.21.15.92:0/1579783757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfd406a2f0 0x7fbfd4155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.300+0000 7fbfd8ba6700 1 -- 172.21.15.92:0/1579783757 >> v1:172.21.15.124:6789/0 conn(0x7fbfd4155800 legacy=0x7fbfd406b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:48.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.300+0000 7fbfd8ba6700 1 -- 172.21.15.92:0/1579783757 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd4159ca0 con 0x7fbfd4159530 2023-10-06T10:45:48.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.300+0000 7fbfd8ba6700 1 --2- 172.21.15.92:0/1579783757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd4159530 0x7fbfd4069db0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40099e0 tx=0x7fbfc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3521392b5cca3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:48.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.300+0000 7fbfcbfff700 1 -- 172.21.15.92:0/1579783757 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc400b820 con 0x7fbfd4159530 2023-10-06T10:45:48.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.300+0000 7fbfcbfff700 1 -- 172.21.15.92:0/1579783757 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc400b980 con 0x7fbfd4159530 2023-10-06T10:45:48.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.300+0000 7fbfcbfff700 1 -- 172.21.15.92:0/1579783757 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc400bd20 con 0x7fbfd4159530 2023-10-06T10:45:48.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.301+0000 7fbfdb60b700 1 -- 172.21.15.92:0/1579783757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd4159530 msgr2=0x7fbfd4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:48.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.301+0000 7fbfdb60b700 1 --2- 172.21.15.92:0/1579783757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd4159530 0x7fbfd4069db0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40099e0 tx=0x7fbfc4009250 comp rx=0 tx=0).stop 2023-10-06T10:45:48.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.301+0000 7fbfdb60b700 1 -- 172.21.15.92:0/1579783757 shutdown_connections 2023-10-06T10:45:48.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.301+0000 7fbfdb60b700 1 --2- 172.21.15.92:0/1579783757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfd406a2f0 0x7fbfd4155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.301+0000 7fbfdb60b700 1 --2- 172.21.15.92:0/1579783757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd4159530 0x7fbfd4069db0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.301+0000 7fbfdb60b700 1 -- 172.21.15.92:0/1579783757 >> 172.21.15.92:0/1579783757 conn(0x7fbfd40cc0a0 msgr2=0x7fbfd40cc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:48.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.301+0000 7fbfdb60b700 1 -- 172.21.15.92:0/1579783757 shutdown_connections 2023-10-06T10:45:48.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.301+0000 7fbfdb60b700 1 -- 172.21.15.92:0/1579783757 wait complete. 2023-10-06T10:45:48.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.302+0000 7fbfdb60b700 1 Processor -- start 2023-10-06T10:45:48.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.302+0000 7fbfdb60b700 1 -- start start 2023-10-06T10:45:48.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.303+0000 7fbfdb60b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd406a2f0 0x7fbfd414a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:48.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.303+0000 7fbfdb60b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfd4155800 0x7fbfd4169ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:48.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.303+0000 7fbfdb60b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfd4159530 0x7fbfd414df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:48.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.303+0000 7fbfdb60b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd41535d0 con 0x7fbfd406a2f0 2023-10-06T10:45:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.303+0000 7fbfdb60b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd414d320 con 0x7fbfd4159530 2023-10-06T10:45:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.303+0000 7fbfdb60b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd414d600 con 0x7fbfd4155800 2023-10-06T10:45:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.303+0000 7fbfd93a7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd406a2f0 0x7fbfd414a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.303+0000 7fbfd93a7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd406a2f0 0x7fbfd414a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47262/0 (socket says 172.21.15.92:47262) 2023-10-06T10:45:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.303+0000 7fbfd93a7700 1 -- 172.21.15.92:0/419305710 learned_addr learned my addr 172.21.15.92:0/419305710 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.303+0000 7fbfd8ba6700 1 --2- 172.21.15.92:0/419305710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfd4155800 0x7fbfd4169ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.303+0000 7fbfd9ba8700 1 --2- 172.21.15.92:0/419305710 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfd4159530 0x7fbfd414df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.303+0000 7fbfd93a7700 1 -- 172.21.15.92:0/419305710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfd4155800 msgr2=0x7fbfd4169ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.303+0000 7fbfd93a7700 1 --2- 172.21.15.92:0/419305710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfd4155800 0x7fbfd4169ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.303+0000 7fbfd93a7700 1 -- 172.21.15.92:0/419305710 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfd4159530 msgr2=0x7fbfd414df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.303+0000 7fbfd93a7700 1 --2- 172.21.15.92:0/419305710 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfd4159530 0x7fbfd414df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.303+0000 7fbfd93a7700 1 -- 172.21.15.92:0/419305710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc40094f0 con 0x7fbfd406a2f0 2023-10-06T10:45:48.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.303+0000 7fbfd9ba8700 1 --2- 172.21.15.92:0/419305710 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfd4159530 0x7fbfd414df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:48.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.303+0000 7fbfd8ba6700 1 --2- 172.21.15.92:0/419305710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfd4155800 0x7fbfd4169ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:48.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.303+0000 7fbfd93a7700 1 --2- 172.21.15.92:0/419305710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd406a2f0 0x7fbfd414a200 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000cb10 tx=0x7fbfd00017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:48.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.304+0000 7fbfc97fa700 1 -- 172.21.15.92:0/419305710 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd000a760 con 0x7fbfd406a2f0 2023-10-06T10:45:48.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.304+0000 7fbfc97fa700 1 -- 172.21.15.92:0/419305710 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd000a8e0 con 0x7fbfd406a2f0 2023-10-06T10:45:48.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.304+0000 7fbfc97fa700 1 -- 172.21.15.92:0/419305710 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd000aca0 con 0x7fbfd406a2f0 2023-10-06T10:45:48.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.304+0000 7fbfdb60b700 1 -- 172.21.15.92:0/419305710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd414d8e0 con 0x7fbfd406a2f0 2023-10-06T10:45:48.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.304+0000 7fbfdb60b700 1 -- 172.21.15.92:0/419305710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd416a010 con 0x7fbfd406a2f0 2023-10-06T10:45:48.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.305+0000 7fbfdb60b700 1 -- 172.21.15.92:0/419305710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd40ccc50 con 0x7fbfd406a2f0 2023-10-06T10:45:48.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.305+0000 7fbfc97fa700 1 -- 172.21.15.92:0/419305710 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbfd0003030 con 0x7fbfd406a2f0 2023-10-06T10:45:48.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.305+0000 7fbfc97fa700 1 --2- 172.21.15.92:0/419305710 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfb8066f30 0x7fbfb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:48.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.305+0000 7fbfc97fa700 1 -- 172.21.15.92:0/419305710 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fbfd00847e0 con 0x7fbfd406a2f0 2023-10-06T10:45:48.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.305+0000 7fbfd8ba6700 1 --2- 172.21.15.92:0/419305710 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfb8066f30 0x7fbfb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:48.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.306+0000 7fbfd8ba6700 1 --2- 172.21.15.92:0/419305710 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfb8066f30 0x7fbfb80693e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40099e0 tx=0x7fbfc4017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:48.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.309+0000 7fbfc97fa700 1 -- 172.21.15.92:0/419305710 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfd0051770 con 0x7fbfd406a2f0 2023-10-06T10:45:48.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.493+0000 7fbfdb60b700 1 -- 172.21.15.92:0/419305710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbfd406a750 con 0x7fbfd406a2f0 2023-10-06T10:45:48.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.493+0000 7fbfc97fa700 1 -- 172.21.15.92:0/419305710 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v104) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbfd406a750 con 0x7fbfd406a2f0 2023-10-06T10:45:48.495 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T10:45:48.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.495+0000 7fbfdb60b700 1 -- 172.21.15.92:0/419305710 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfb8066f30 msgr2=0x7fbfb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:48.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.495+0000 7fbfdb60b700 1 --2- 172.21.15.92:0/419305710 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfb8066f30 0x7fbfb80693e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40099e0 tx=0x7fbfc4017000 comp rx=0 tx=0).stop 2023-10-06T10:45:48.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.495+0000 7fbfdb60b700 1 -- 172.21.15.92:0/419305710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd406a2f0 msgr2=0x7fbfd414a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:48.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.495+0000 7fbfdb60b700 1 --2- 172.21.15.92:0/419305710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd406a2f0 0x7fbfd414a200 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000cb10 tx=0x7fbfd00017b0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.496+0000 7fbfdb60b700 1 -- 172.21.15.92:0/419305710 shutdown_connections 2023-10-06T10:45:48.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.496+0000 7fbfdb60b700 1 --2- 172.21.15.92:0/419305710 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fbfb8066f30 0x7fbfb80693e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.496+0000 7fbfdb60b700 1 --2- 172.21.15.92:0/419305710 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbfd4159530 0x7fbfd414df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.496+0000 7fbfdb60b700 1 --2- 172.21.15.92:0/419305710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd406a2f0 0x7fbfd414a200 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.496+0000 7fbfdb60b700 1 --2- 172.21.15.92:0/419305710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbfd4155800 0x7fbfd4169ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.496+0000 7fbfdb60b700 1 -- 172.21.15.92:0/419305710 >> 172.21.15.92:0/419305710 conn(0x7fbfd40cc0a0 msgr2=0x7fbfd40c11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:48.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.496+0000 7fbfdb60b700 1 -- 172.21.15.92:0/419305710 shutdown_connections 2023-10-06T10:45:48.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.496+0000 7fbfdb60b700 1 -- 172.21.15.92:0/419305710 wait complete. 2023-10-06T10:45:48.509 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T10:45:48.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.625+0000 7f84cb1ec700 1 Processor -- start 2023-10-06T10:45:48.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.625+0000 7f84cb1ec700 1 -- start start 2023-10-06T10:45:48.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.626+0000 7f84cb1ec700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84c40cbc70 0x7f84c40ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:48.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.626+0000 7f84cb1ec700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c40cf5d0 con 0x7f84c41598a0 2023-10-06T10:45:48.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.626+0000 7f84cb1ec700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c40cf8b0 con 0x7f84c41594d0 2023-10-06T10:45:48.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.626+0000 7f84cb1ec700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c40cfb90 con 0x7f84c40cbc70 2023-10-06T10:45:48.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.626+0000 7f84c3fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f84c41598a0 0x7f84c40ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58772/0 (socket says 172.21.15.92:58772) 2023-10-06T10:45:48.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.626+0000 7f84c3fff700 1 -- 172.21.15.92:0/1176349725 learned_addr learned my addr 172.21.15.92:0/1176349725 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:48.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.626+0000 7f84c37fe700 1 -- 172.21.15.92:0/1176349725 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 600605830 0 0) 0x7f84c40cf5d0 con 0x7f84c41598a0 2023-10-06T10:45:48.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.626+0000 7f84c37fe700 1 -- 172.21.15.92:0/1176349725 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84ac003570 con 0x7f84c41598a0 2023-10-06T10:45:48.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.626+0000 7f84c9789700 1 --2- 172.21.15.92:0/1176349725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84c40cbc70 0x7f84c40ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:48.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.626+0000 7f84c37fe700 1 -- 172.21.15.92:0/1176349725 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3127867385 0 0) 0x7f84ac003570 con 0x7f84c41598a0 2023-10-06T10:45:48.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.626+0000 7f84c37fe700 1 -- 172.21.15.92:0/1176349725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84c40cbc70 msgr2=0x7f84c40ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:48.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.626+0000 7f84c37fe700 1 --2- 172.21.15.92:0/1176349725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84c40cbc70 0x7f84c40ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c37fe700 1 -- 172.21.15.92:0/1176349725 >> v1:172.21.15.124:6789/0 conn(0x7f84c41594d0 legacy=0x7f84c40cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:48.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c37fe700 1 -- 172.21.15.92:0/1176349725 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c40cfe70 con 0x7f84c41598a0 2023-10-06T10:45:48.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c37fe700 1 -- 172.21.15.92:0/1176349725 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f84b8001470 con 0x7f84c41598a0 2023-10-06T10:45:48.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c37fe700 1 -- 172.21.15.92:0/1176349725 >> v1:172.21.15.92:6789/0 conn(0x7f84c41598a0 legacy=0x7f84c40ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:48.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c37fe700 1 --2- 172.21.15.92:0/1176349725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84ac003a90 0x7f84ac003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:48.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c37fe700 1 --2- 172.21.15.92:0/1176349725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84c40cbc70 0x7f84ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:48.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c37fe700 1 --2- 172.21.15.92:0/1176349725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84ac004da0 0x7f84ac0051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:48.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c37fe700 1 -- 172.21.15.92:0/1176349725 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c40cfb90 con 0x7f84ac003a90 2023-10-06T10:45:48.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c37fe700 1 -- 172.21.15.92:0/1176349725 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac00bc20 con 0x7f84ac004da0 2023-10-06T10:45:48.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c37fe700 1 -- 172.21.15.92:0/1176349725 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac00bde0 con 0x7f84c40cbc70 2023-10-06T10:45:48.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c8f88700 1 --2- 172.21.15.92:0/1176349725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84ac003a90 0x7f84ac003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:48.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c3fff700 1 --2- 172.21.15.92:0/1176349725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84c40cbc70 0x7f84ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:48.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c9789700 1 --2- 172.21.15.92:0/1176349725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84ac004da0 0x7f84ac0051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:48.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c8f88700 1 -- 172.21.15.92:0/1176349725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84c40cbc70 msgr2=0x7f84ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:48.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c8f88700 1 --2- 172.21.15.92:0/1176349725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84c40cbc70 0x7f84ac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c8f88700 1 -- 172.21.15.92:0/1176349725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84ac004da0 msgr2=0x7f84ac0051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:48.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c8f88700 1 --2- 172.21.15.92:0/1176349725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84ac004da0 0x7f84ac0051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c8f88700 1 -- 172.21.15.92:0/1176349725 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ac003570 con 0x7f84ac003a90 2023-10-06T10:45:48.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c3fff700 1 --2- 172.21.15.92:0/1176349725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84c40cbc70 0x7f84ac004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:48.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c8f88700 1 --2- 172.21.15.92:0/1176349725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84ac003a90 0x7f84ac003ea0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f84b4001f90 tx=0x7f84b4001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b3200aef2f4947 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:48.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c37fe700 1 -- 172.21.15.92:0/1176349725 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b4001950 con 0x7f84ac003a90 2023-10-06T10:45:48.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c37fe700 1 -- 172.21.15.92:0/1176349725 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b400c8b0 con 0x7f84ac003a90 2023-10-06T10:45:48.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.627+0000 7f84c37fe700 1 -- 172.21.15.92:0/1176349725 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b400cc50 con 0x7f84ac003a90 2023-10-06T10:45:48.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.628+0000 7f84cb1ec700 1 -- 172.21.15.92:0/1176349725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84ac003a90 msgr2=0x7f84ac003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:48.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.628+0000 7f84cb1ec700 1 --2- 172.21.15.92:0/1176349725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84ac003a90 0x7f84ac003ea0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f84b4001f90 tx=0x7f84b4001640 comp rx=0 tx=0).stop 2023-10-06T10:45:48.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.628+0000 7f84c9789700 1 -- 172.21.15.92:0/1176349725 reap_dead start 2023-10-06T10:45:48.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.628+0000 7f84cb1ec700 1 -- 172.21.15.92:0/1176349725 shutdown_connections 2023-10-06T10:45:48.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.628+0000 7f84cb1ec700 1 -- 172.21.15.92:0/1176349725 >> 172.21.15.92:0/1176349725 conn(0x7f84c4089e50 msgr2=0x7f84c4099770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:48.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.628+0000 7f84cb1ec700 1 -- 172.21.15.92:0/1176349725 shutdown_connections 2023-10-06T10:45:48.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.628+0000 7f84cb1ec700 1 -- 172.21.15.92:0/1176349725 wait complete. 2023-10-06T10:45:48.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.629+0000 7f84cb1ec700 1 Processor -- start 2023-10-06T10:45:48.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.629+0000 7f84cb1ec700 1 -- start start 2023-10-06T10:45:48.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.630+0000 7f84cb1ec700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84ac003a90 0x7f84c40b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:48.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.630+0000 7f84cb1ec700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c40b5770 0x7f84c40c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:48.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.630+0000 7f84cb1ec700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84c4149250 0x7f84c415bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:48.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.630+0000 7f84cb1ec700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c40b4760 con 0x7f84c40b5770 2023-10-06T10:45:48.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.630+0000 7f84cb1ec700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c40c8af0 con 0x7f84c4149250 2023-10-06T10:45:48.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.630+0000 7f84cb1ec700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c40b49b0 con 0x7f84ac003a90 2023-10-06T10:45:48.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.630+0000 7f84c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c40b5770 0x7f84c40c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:48.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.630+0000 7f84c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c40b5770 0x7f84c40c85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47276/0 (socket says 172.21.15.92:47276) 2023-10-06T10:45:48.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.630+0000 7f84c3fff700 1 -- 172.21.15.92:0/2613613042 learned_addr learned my addr 172.21.15.92:0/2613613042 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:48.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.630+0000 7f84c9789700 1 --2- 172.21.15.92:0/2613613042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84c4149250 0x7f84c415bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:48.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.630+0000 7f84c8f88700 1 --2- 172.21.15.92:0/2613613042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84ac003a90 0x7f84c40b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:48.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.630+0000 7f84c3fff700 1 -- 172.21.15.92:0/2613613042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84ac003a90 msgr2=0x7f84c40b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:48.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.630+0000 7f84c3fff700 1 --2- 172.21.15.92:0/2613613042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84ac003a90 0x7f84c40b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.630+0000 7f84c3fff700 1 -- 172.21.15.92:0/2613613042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84c4149250 msgr2=0x7f84c415bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:48.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.630+0000 7f84c3fff700 1 --2- 172.21.15.92:0/2613613042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84c4149250 0x7f84c415bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.630+0000 7f84c3fff700 1 -- 172.21.15.92:0/2613613042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c40b4c90 con 0x7f84c40b5770 2023-10-06T10:45:48.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.630+0000 7f84c8f88700 1 --2- 172.21.15.92:0/2613613042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84ac003a90 0x7f84c40b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:48.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.630+0000 7f84c3fff700 1 --2- 172.21.15.92:0/2613613042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c40b5770 0x7f84c40c85b0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f84b800eb90 tx=0x7f84b8001430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:48.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.630+0000 7f84c0ff9700 1 -- 172.21.15.92:0/2613613042 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8005e90 con 0x7f84c40b5770 2023-10-06T10:45:48.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.631+0000 7f84c0ff9700 1 -- 172.21.15.92:0/2613613042 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b800c7e0 con 0x7f84c40b5770 2023-10-06T10:45:48.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.631+0000 7f84c0ff9700 1 -- 172.21.15.92:0/2613613042 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b800cb60 con 0x7f84c40b5770 2023-10-06T10:45:48.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.631+0000 7f84cb1ec700 1 -- 172.21.15.92:0/2613613042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c40b4f70 con 0x7f84c40b5770 2023-10-06T10:45:48.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.631+0000 7f84cb1ec700 1 -- 172.21.15.92:0/2613613042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c41496f0 con 0x7f84c40b5770 2023-10-06T10:45:48.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.631+0000 7f84cb1ec700 1 -- 172.21.15.92:0/2613613042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c4064a90 con 0x7f84c40b5770 2023-10-06T10:45:48.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.632+0000 7f84c0ff9700 1 -- 172.21.15.92:0/2613613042 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f84b801d460 con 0x7f84c40b5770 2023-10-06T10:45:48.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.632+0000 7f84c0ff9700 1 --2- 172.21.15.92:0/2613613042 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f84b0066f30 0x7f84b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:48.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.632+0000 7f84c0ff9700 1 -- 172.21.15.92:0/2613613042 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f84b8084800 con 0x7f84c40b5770 2023-10-06T10:45:48.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.632+0000 7f84c8f88700 1 --2- 172.21.15.92:0/2613613042 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f84b0066f30 0x7f84b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:48.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.633+0000 7f84c8f88700 1 --2- 172.21.15.92:0/2613613042 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f84b0066f30 0x7f84b00693e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f84b4004290 tx=0x7f84b4015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:48.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.637+0000 7f84c0ff9700 1 -- 172.21.15.92:0/2613613042 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f84b8051790 con 0x7f84c40b5770 2023-10-06T10:45:48.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.819+0000 7f84cb1ec700 1 -- 172.21.15.92:0/2613613042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f84c40c66a0 con 0x7f84c40b5770 2023-10-06T10:45:48.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.819+0000 7f84c0ff9700 1 -- 172.21.15.92:0/2613613042 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) v1 ==== 93+0+10 (secure 0 0 0) 0x7f84b8082e60 con 0x7f84c40b5770 2023-10-06T10:45:48.821 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T10:45:48.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.822+0000 7f84cb1ec700 1 -- 172.21.15.92:0/2613613042 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f84b0066f30 msgr2=0x7f84b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:48.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.822+0000 7f84cb1ec700 1 --2- 172.21.15.92:0/2613613042 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f84b0066f30 0x7f84b00693e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f84b4004290 tx=0x7f84b4015000 comp rx=0 tx=0).stop 2023-10-06T10:45:48.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.822+0000 7f84cb1ec700 1 -- 172.21.15.92:0/2613613042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c40b5770 msgr2=0x7f84c40c85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:48.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.822+0000 7f84cb1ec700 1 --2- 172.21.15.92:0/2613613042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c40b5770 0x7f84c40c85b0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f84b800eb90 tx=0x7f84b8001430 comp rx=0 tx=0).stop 2023-10-06T10:45:48.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.822+0000 7f84cb1ec700 1 -- 172.21.15.92:0/2613613042 shutdown_connections 2023-10-06T10:45:48.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.822+0000 7f84cb1ec700 1 --2- 172.21.15.92:0/2613613042 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f84b0066f30 0x7f84b00693e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.822+0000 7f84cb1ec700 1 --2- 172.21.15.92:0/2613613042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84c4149250 0x7f84c415bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.822+0000 7f84cb1ec700 1 --2- 172.21.15.92:0/2613613042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84ac003a90 0x7f84c40b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.822+0000 7f84cb1ec700 1 --2- 172.21.15.92:0/2613613042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c40b5770 0x7f84c40c85b0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.822+0000 7f84cb1ec700 1 -- 172.21.15.92:0/2613613042 >> 172.21.15.92:0/2613613042 conn(0x7f84c4089e50 msgr2=0x7f84c40d0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:48.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.822+0000 7f84cb1ec700 1 -- 172.21.15.92:0/2613613042 shutdown_connections 2023-10-06T10:45:48.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.822+0000 7f84cb1ec700 1 -- 172.21.15.92:0/2613613042 wait complete. 2023-10-06T10:45:48.835 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T10:45:48.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.951+0000 7f13ca0a4700 1 Processor -- start 2023-10-06T10:45:48.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.951+0000 7f13ca0a4700 1 -- start start 2023-10-06T10:45:48.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.951+0000 7f13ca0a4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13c40c7ac0 0x7f13c40c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:48.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.951+0000 7f13ca0a4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f13c413fe00 0x7f13c415a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:48.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.951+0000 7f13ca0a4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c406e790 con 0x7f13c406e340 2023-10-06T10:45:48.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.951+0000 7f13ca0a4700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13c415ade0 con 0x7f13c413fe00 2023-10-06T10:45:48.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.951+0000 7f13ca0a4700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c406ea20 con 0x7f13c40c7ac0 2023-10-06T10:45:48.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.951+0000 7f13c3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f13c413fe00 0x7f13c415a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:48.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.952+0000 7f13c2ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f13c406e340 0x7f13c413f6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58774/0 (socket says 172.21.15.92:58774) 2023-10-06T10:45:48.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.952+0000 7f13c2ffd700 1 -- 172.21.15.92:0/2365088948 learned_addr learned my addr 172.21.15.92:0/2365088948 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:48.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.952+0000 7f13c37fe700 1 --2- 172.21.15.92:0/2365088948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13c40c7ac0 0x7f13c40c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:48.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.952+0000 7f13c3fff700 1 -- 172.21.15.92:0/2365088948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13c40c7ac0 msgr2=0x7f13c40c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:48.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.952+0000 7f13c3fff700 1 --2- 172.21.15.92:0/2365088948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13c40c7ac0 0x7f13c40c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.952+0000 7f13c3fff700 1 -- 172.21.15.92:0/2365088948 >> v1:172.21.15.92:6789/0 conn(0x7f13c406e340 legacy=0x7f13c413f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:48.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.952+0000 7f13c3fff700 1 -- 172.21.15.92:0/2365088948 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c406ed00 con 0x7f13c413fe00 2023-10-06T10:45:48.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.952+0000 7f13c27fc700 1 -- 172.21.15.92:0/2365088948 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376405482 0 0) 0x7f13c406e790 con 0x7f13c406e340 2023-10-06T10:45:48.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.952+0000 7f13c3fff700 1 --2- 172.21.15.92:0/2365088948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c413fe00 0x7f13c415a8a0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0098e0 tx=0x7f13bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0827bdc4b26a77 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:48.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.952+0000 7f13c27fc700 1 -- 172.21.15.92:0/2365088948 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc0108a0 con 0x7f13c413fe00 2023-10-06T10:45:48.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.952+0000 7f13c27fc700 1 -- 172.21.15.92:0/2365088948 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13bc010a00 con 0x7f13c413fe00 2023-10-06T10:45:48.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.952+0000 7f13c27fc700 1 -- 172.21.15.92:0/2365088948 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc010da0 con 0x7f13c413fe00 2023-10-06T10:45:48.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.953+0000 7f13ca0a4700 1 -- 172.21.15.92:0/2365088948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c413fe00 msgr2=0x7f13c415a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:48.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.953+0000 7f13ca0a4700 1 --2- 172.21.15.92:0/2365088948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c413fe00 0x7f13c415a8a0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0098e0 tx=0x7f13bc0091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.953+0000 7f13ca0a4700 1 -- 172.21.15.92:0/2365088948 shutdown_connections 2023-10-06T10:45:48.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.953+0000 7f13ca0a4700 1 --2- 172.21.15.92:0/2365088948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13c40c7ac0 0x7f13c40c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.953+0000 7f13ca0a4700 1 --2- 172.21.15.92:0/2365088948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c413fe00 0x7f13c415a8a0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.953+0000 7f13ca0a4700 1 -- 172.21.15.92:0/2365088948 >> 172.21.15.92:0/2365088948 conn(0x7f13c40cc0a0 msgr2=0x7f13c40cc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:48.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.953+0000 7f13ca0a4700 1 -- 172.21.15.92:0/2365088948 shutdown_connections 2023-10-06T10:45:48.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.953+0000 7f13ca0a4700 1 -- 172.21.15.92:0/2365088948 wait complete. 2023-10-06T10:45:48.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.954+0000 7f13ca0a4700 1 Processor -- start 2023-10-06T10:45:48.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.954+0000 7f13ca0a4700 1 -- start start 2023-10-06T10:45:48.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13ca0a4700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13c406e340 0x7f13c40af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:48.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13ca0a4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c40c7ac0 0x7f13c413d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:48.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13ca0a4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13c413fe00 0x7f13c413d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:48.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13ca0a4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c40b00a0 con 0x7f13c40c7ac0 2023-10-06T10:45:48.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13ca0a4700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c40b0380 con 0x7f13c406e340 2023-10-06T10:45:48.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13ca0a4700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c40b0660 con 0x7f13c413fe00 2023-10-06T10:45:48.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13c2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c40c7ac0 0x7f13c413d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:48.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13c2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c40c7ac0 0x7f13c413d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47294/0 (socket says 172.21.15.92:47294) 2023-10-06T10:45:48.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13c2ffd700 1 -- 172.21.15.92:0/2576065418 learned_addr learned my addr 172.21.15.92:0/2576065418 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:48.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13c37fe700 1 --2- 172.21.15.92:0/2576065418 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13c406e340 0x7f13c40af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:48.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13c3fff700 1 --2- 172.21.15.92:0/2576065418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13c413fe00 0x7f13c413d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:48.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13c2ffd700 1 -- 172.21.15.92:0/2576065418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13c413fe00 msgr2=0x7f13c413d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:48.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13c2ffd700 1 --2- 172.21.15.92:0/2576065418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13c413fe00 0x7f13c413d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13c2ffd700 1 -- 172.21.15.92:0/2576065418 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13c406e340 msgr2=0x7f13c40af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:48.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13c2ffd700 1 --2- 172.21.15.92:0/2576065418 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13c406e340 0x7f13c40af800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:48.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13c2ffd700 1 -- 172.21.15.92:0/2576065418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13bc009420 con 0x7f13c40c7ac0 2023-10-06T10:45:48.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13c37fe700 1 --2- 172.21.15.92:0/2576065418 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13c406e340 0x7f13c40af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:48.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13c2ffd700 1 --2- 172.21.15.92:0/2576065418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c40c7ac0 0x7f13c413d300 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f13b800c5b0 tx=0x7f13b8001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:48.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13ab7fe700 1 -- 172.21.15.92:0/2576065418 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b8005b30 con 0x7f13c40c7ac0 2023-10-06T10:45:48.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13ab7fe700 1 -- 172.21.15.92:0/2576065418 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b8005c90 con 0x7f13c40c7ac0 2023-10-06T10:45:48.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13ca0a4700 1 -- 172.21.15.92:0/2576065418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c40b0940 con 0x7f13c40c7ac0 2023-10-06T10:45:48.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13ab7fe700 1 -- 172.21.15.92:0/2576065418 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b8007300 con 0x7f13c40c7ac0 2023-10-06T10:45:48.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.955+0000 7f13ca0a4700 1 -- 172.21.15.92:0/2576065418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c40afd40 con 0x7f13c40c7ac0 2023-10-06T10:45:48.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.956+0000 7f13ca0a4700 1 -- 172.21.15.92:0/2576065418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c4064a90 con 0x7f13c40c7ac0 2023-10-06T10:45:48.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.957+0000 7f13ab7fe700 1 -- 172.21.15.92:0/2576065418 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f13b801b3b0 con 0x7f13c40c7ac0 2023-10-06T10:45:48.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.957+0000 7f13ab7fe700 1 --2- 172.21.15.92:0/2576065418 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f13b0066f30 0x7f13b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:48.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.957+0000 7f13ab7fe700 1 -- 172.21.15.92:0/2576065418 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f13b8083d60 con 0x7f13c40c7ac0 2023-10-06T10:45:48.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.957+0000 7f13c37fe700 1 --2- 172.21.15.92:0/2576065418 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f13b0066f30 0x7f13b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:48.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.957+0000 7f13c37fe700 1 --2- 172.21.15.92:0/2576065418 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f13b0066f30 0x7f13b00693e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f13b4007720 tx=0x7f13b4009780 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:48.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:48.961+0000 7f13ab7fe700 1 -- 172.21.15.92:0/2576065418 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f13b8050cf0 con 0x7f13c40c7ac0 2023-10-06T10:45:49.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.142+0000 7f13ca0a4700 1 -- 172.21.15.92:0/2576065418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f13c404bb40 con 0x7f13c40c7ac0 2023-10-06T10:45:49.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.143+0000 7f13ab7fe700 1 -- 172.21.15.92:0/2576065418 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v104) v1 ==== 104+0+11 (secure 0 0 0) 0x7f13b80170a0 con 0x7f13c40c7ac0 2023-10-06T10:45:49.145 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T10:45:49.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.145+0000 7f13ca0a4700 1 -- 172.21.15.92:0/2576065418 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f13b0066f30 msgr2=0x7f13b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:49.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.145+0000 7f13ca0a4700 1 --2- 172.21.15.92:0/2576065418 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f13b0066f30 0x7f13b00693e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f13b4007720 tx=0x7f13b4009780 comp rx=0 tx=0).stop 2023-10-06T10:45:49.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.145+0000 7f13ca0a4700 1 -- 172.21.15.92:0/2576065418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c40c7ac0 msgr2=0x7f13c413d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:49.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.145+0000 7f13ca0a4700 1 --2- 172.21.15.92:0/2576065418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c40c7ac0 0x7f13c413d300 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f13b800c5b0 tx=0x7f13b8001070 comp rx=0 tx=0).stop 2023-10-06T10:45:49.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.145+0000 7f13ca0a4700 1 -- 172.21.15.92:0/2576065418 shutdown_connections 2023-10-06T10:45:49.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.145+0000 7f13ca0a4700 1 --2- 172.21.15.92:0/2576065418 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f13b0066f30 0x7f13b00693e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.145+0000 7f13ca0a4700 1 --2- 172.21.15.92:0/2576065418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13c413fe00 0x7f13c413d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.145+0000 7f13ca0a4700 1 --2- 172.21.15.92:0/2576065418 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13c406e340 0x7f13c40af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.145+0000 7f13ca0a4700 1 --2- 172.21.15.92:0/2576065418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c40c7ac0 0x7f13c413d300 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.145+0000 7f13ca0a4700 1 -- 172.21.15.92:0/2576065418 >> 172.21.15.92:0/2576065418 conn(0x7f13c40cc0a0 msgr2=0x7f13c413c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:49.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.145+0000 7f13ca0a4700 1 -- 172.21.15.92:0/2576065418 shutdown_connections 2023-10-06T10:45:49.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.145+0000 7f13ca0a4700 1 -- 172.21.15.92:0/2576065418 wait complete. 2023-10-06T10:45:49.158 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T10:45:49.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.274+0000 7f4497226700 1 Processor -- start 2023-10-06T10:45:49.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.274+0000 7f4497226700 1 -- start start 2023-10-06T10:45:49.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.275+0000 7f4497226700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44900b2620 0x7f44900cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:49.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.275+0000 7f4497226700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f44900cf1b0 0x7f44900cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:49.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.275+0000 7f4497226700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44900cc970 con 0x7f44900b29f0 2023-10-06T10:45:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.275+0000 7f4497226700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44900cbd60 con 0x7f44900cf1b0 2023-10-06T10:45:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.275+0000 7f4497226700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44900cc040 con 0x7f44900b2620 2023-10-06T10:45:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.275+0000 7f4496a25700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f44900cf1b0 0x7f44900cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.275+0000 7f4496a25700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f44900cf1b0 0x7f44900cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47306/0 (socket says 172.21.15.92:47306) 2023-10-06T10:45:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.275+0000 7f4496a25700 1 -- 172.21.15.92:0/2473458477 learned_addr learned my addr 172.21.15.92:0/2473458477 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.275+0000 7f4496224700 1 --2- 172.21.15.92:0/2473458477 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44900b2620 0x7f44900cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.275+0000 7f4496a25700 1 -- 172.21.15.92:0/2473458477 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44900b2620 msgr2=0x7f44900cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.275+0000 7f4496a25700 1 --2- 172.21.15.92:0/2473458477 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44900b2620 0x7f44900cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.275+0000 7f4496a25700 1 -- 172.21.15.92:0/2473458477 >> v1:172.21.15.124:6789/0 conn(0x7f44900b29f0 legacy=0x7f44900ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:49.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.275+0000 7f4496a25700 1 -- 172.21.15.92:0/2473458477 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44900cc320 con 0x7f44900cf1b0 2023-10-06T10:45:49.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.275+0000 7f4496224700 1 --2- 172.21.15.92:0/2473458477 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44900b2620 0x7f44900cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:49.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.275+0000 7f4496a25700 1 --2- 172.21.15.92:0/2473458477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44900cf1b0 0x7f44900cd1f0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f4488009f20 tx=0x7f4488009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9d0f72184f2430 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:49.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.275+0000 7f4495222700 1 -- 172.21.15.92:0/2473458477 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448800e070 con 0x7f44900cf1b0 2023-10-06T10:45:49.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.276+0000 7f4495222700 1 -- 172.21.15.92:0/2473458477 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448800ba80 con 0x7f44900cf1b0 2023-10-06T10:45:49.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.276+0000 7f4495222700 1 -- 172.21.15.92:0/2473458477 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44880092e0 con 0x7f44900cf1b0 2023-10-06T10:45:49.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.276+0000 7f4497226700 1 -- 172.21.15.92:0/2473458477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44900cf1b0 msgr2=0x7f44900cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:49.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.276+0000 7f4497226700 1 --2- 172.21.15.92:0/2473458477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44900cf1b0 0x7f44900cd1f0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f4488009f20 tx=0x7f4488009840 comp rx=0 tx=0).stop 2023-10-06T10:45:49.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.276+0000 7f4497226700 1 -- 172.21.15.92:0/2473458477 shutdown_connections 2023-10-06T10:45:49.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.276+0000 7f4497226700 1 --2- 172.21.15.92:0/2473458477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44900cf1b0 0x7f44900cd1f0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.276+0000 7f4497226700 1 --2- 172.21.15.92:0/2473458477 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44900b2620 0x7f44900cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.276+0000 7f4497226700 1 -- 172.21.15.92:0/2473458477 >> 172.21.15.92:0/2473458477 conn(0x7f4490099520 msgr2=0x7f449006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:49.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.276+0000 7f4497226700 1 -- 172.21.15.92:0/2473458477 shutdown_connections 2023-10-06T10:45:49.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.276+0000 7f4497226700 1 -- 172.21.15.92:0/2473458477 wait complete. 2023-10-06T10:45:49.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.277+0000 7f4497226700 1 Processor -- start 2023-10-06T10:45:49.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.277+0000 7f4497226700 1 -- start start 2023-10-06T10:45:49.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.278+0000 7f4497226700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44900b2620 0x7f44900b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:49.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.278+0000 7f4497226700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44900b29f0 0x7f4490155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:49.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.278+0000 7f4497226700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44900cf1b0 0x7f4490158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:49.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.278+0000 7f4497226700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44900ca290 con 0x7f44900cf1b0 2023-10-06T10:45:49.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.278+0000 7f4497226700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44900b5360 con 0x7f44900b29f0 2023-10-06T10:45:49.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.278+0000 7f4497226700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44900c9680 con 0x7f44900b2620 2023-10-06T10:45:49.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.278+0000 7f4496a25700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44900cf1b0 0x7f4490158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:49.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.278+0000 7f4496a25700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44900cf1b0 0x7f4490158140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47314/0 (socket says 172.21.15.92:47314) 2023-10-06T10:45:49.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.278+0000 7f4496a25700 1 -- 172.21.15.92:0/2395606817 learned_addr learned my addr 172.21.15.92:0/2395606817 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:49.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.278+0000 7f4496224700 1 --2- 172.21.15.92:0/2395606817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44900b2620 0x7f44900b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:49.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.278+0000 7f4495a23700 1 --2- 172.21.15.92:0/2395606817 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44900b29f0 0x7f4490155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:49.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.278+0000 7f4496a25700 1 -- 172.21.15.92:0/2395606817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44900b2620 msgr2=0x7f44900b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:49.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.278+0000 7f4496a25700 1 --2- 172.21.15.92:0/2395606817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44900b2620 0x7f44900b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.278+0000 7f4496a25700 1 -- 172.21.15.92:0/2395606817 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44900b29f0 msgr2=0x7f4490155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:49.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.278+0000 7f4496a25700 1 --2- 172.21.15.92:0/2395606817 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44900b29f0 0x7f4490155060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.278+0000 7f4496a25700 1 -- 172.21.15.92:0/2395606817 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4488009ae0 con 0x7f44900cf1b0 2023-10-06T10:45:49.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.278+0000 7f4495a23700 1 --2- 172.21.15.92:0/2395606817 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44900b29f0 0x7f4490155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:49.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.278+0000 7f4496a25700 1 --2- 172.21.15.92:0/2395606817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44900cf1b0 0x7f4490158140 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f4488007940 tx=0x7f4488018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:49.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.279+0000 7f44867fc700 1 -- 172.21.15.92:0/2395606817 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44880092e0 con 0x7f44900cf1b0 2023-10-06T10:45:49.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.279+0000 7f44867fc700 1 -- 172.21.15.92:0/2395606817 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4488007de0 con 0x7f44900cf1b0 2023-10-06T10:45:49.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.279+0000 7f4497226700 1 -- 172.21.15.92:0/2395606817 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44900c9960 con 0x7f44900cf1b0 2023-10-06T10:45:49.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.279+0000 7f44867fc700 1 -- 172.21.15.92:0/2395606817 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448801e680 con 0x7f44900cf1b0 2023-10-06T10:45:49.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.279+0000 7f4497226700 1 -- 172.21.15.92:0/2395606817 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44900c9f20 con 0x7f44900cf1b0 2023-10-06T10:45:49.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.280+0000 7f4497226700 1 -- 172.21.15.92:0/2395606817 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f449005f230 con 0x7f44900cf1b0 2023-10-06T10:45:49.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.280+0000 7f44867fc700 1 -- 172.21.15.92:0/2395606817 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4488003040 con 0x7f44900cf1b0 2023-10-06T10:45:49.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.280+0000 7f44867fc700 1 --2- 172.21.15.92:0/2395606817 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4474066f30 0x7f44740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:49.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.280+0000 7f44867fc700 1 -- 172.21.15.92:0/2395606817 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f44880869f0 con 0x7f44900cf1b0 2023-10-06T10:45:49.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.280+0000 7f4496224700 1 --2- 172.21.15.92:0/2395606817 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4474066f30 0x7f44740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:49.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.281+0000 7f4496224700 1 --2- 172.21.15.92:0/2395606817 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4474066f30 0x7f44740693e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f448c006ea0 tx=0x7f448c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:49.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.285+0000 7f44867fc700 1 -- 172.21.15.92:0/2395606817 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4488053a30 con 0x7f44900cf1b0 2023-10-06T10:45:49.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.468+0000 7f4497226700 1 -- 172.21.15.92:0/2395606817 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f44900cc600 con 0x7f44900cf1b0 2023-10-06T10:45:49.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.468+0000 7f44867fc700 1 -- 172.21.15.92:0/2395606817 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v104) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4488057050 con 0x7f44900cf1b0 2023-10-06T10:45:49.471 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T10:45:49.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.470+0000 7f4497226700 1 -- 172.21.15.92:0/2395606817 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4474066f30 msgr2=0x7f44740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:49.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.470+0000 7f4497226700 1 --2- 172.21.15.92:0/2395606817 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4474066f30 0x7f44740693e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f448c006ea0 tx=0x7f448c008040 comp rx=0 tx=0).stop 2023-10-06T10:45:49.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.470+0000 7f4497226700 1 -- 172.21.15.92:0/2395606817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44900cf1b0 msgr2=0x7f4490158140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:49.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.470+0000 7f4497226700 1 --2- 172.21.15.92:0/2395606817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44900cf1b0 0x7f4490158140 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f4488007940 tx=0x7f4488018000 comp rx=0 tx=0).stop 2023-10-06T10:45:49.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.470+0000 7f4497226700 1 -- 172.21.15.92:0/2395606817 shutdown_connections 2023-10-06T10:45:49.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.470+0000 7f4497226700 1 --2- 172.21.15.92:0/2395606817 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4474066f30 0x7f44740693e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.470+0000 7f4497226700 1 --2- 172.21.15.92:0/2395606817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44900cf1b0 0x7f4490158140 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.470+0000 7f4497226700 1 --2- 172.21.15.92:0/2395606817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44900b2620 0x7f44900b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.470+0000 7f4497226700 1 --2- 172.21.15.92:0/2395606817 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44900b29f0 0x7f4490155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.470+0000 7f4497226700 1 -- 172.21.15.92:0/2395606817 >> 172.21.15.92:0/2395606817 conn(0x7f4490099520 msgr2=0x7f44900cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:49.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.470+0000 7f4497226700 1 -- 172.21.15.92:0/2395606817 shutdown_connections 2023-10-06T10:45:49.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.470+0000 7f4497226700 1 -- 172.21.15.92:0/2395606817 wait complete. 2023-10-06T10:45:49.483 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:45:49.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.595+0000 7f7f179fd700 1 Processor -- start 2023-10-06T10:45:49.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.595+0000 7f7f179fd700 1 -- start start 2023-10-06T10:45:49.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.596+0000 7f7f179fd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f100b0760 0x7f7f100ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:49.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.596+0000 7f7f179fd700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f100b0c60 con 0x7f7f10159560 2023-10-06T10:45:49.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.596+0000 7f7f179fd700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f100b0f40 con 0x7f7f100b49a0 2023-10-06T10:45:49.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.596+0000 7f7f179fd700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f100ab350 con 0x7f7f100b0760 2023-10-06T10:45:49.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.596+0000 7f7f15799700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7f10159560 0x7f7f100aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58790/0 (socket says 172.21.15.92:58790) 2023-10-06T10:45:49.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.596+0000 7f7f15799700 1 -- 172.21.15.92:0/541366516 learned_addr learned my addr 172.21.15.92:0/541366516 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:49.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.596+0000 7f7f07fff700 1 -- 172.21.15.92:0/541366516 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3621729589 0 0) 0x7f7f100b0c60 con 0x7f7f10159560 2023-10-06T10:45:49.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.596+0000 7f7f15f9a700 1 --2- 172.21.15.92:0/541366516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f100b0760 0x7f7f100ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:49.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f07fff700 1 -- 172.21.15.92:0/541366516 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7efc003570 con 0x7f7f10159560 2023-10-06T10:45:49.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f07fff700 1 -- 172.21.15.92:0/541366516 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3838175645 0 0) 0x7f7efc003570 con 0x7f7f10159560 2023-10-06T10:45:49.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f07fff700 1 -- 172.21.15.92:0/541366516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f100b0760 msgr2=0x7f7f100ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:49.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f07fff700 1 --2- 172.21.15.92:0/541366516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f100b0760 0x7f7f100ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f07fff700 1 -- 172.21.15.92:0/541366516 >> v1:172.21.15.124:6789/0 conn(0x7f7f100b49a0 legacy=0x7f7f100aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:49.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f07fff700 1 -- 172.21.15.92:0/541366516 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f100b1220 con 0x7f7f10159560 2023-10-06T10:45:49.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f07fff700 1 -- 172.21.15.92:0/541366516 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f7f0c001970 con 0x7f7f10159560 2023-10-06T10:45:49.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f07fff700 1 -- 172.21.15.92:0/541366516 >> v1:172.21.15.92:6789/0 conn(0x7f7f10159560 legacy=0x7f7f100aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:49.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f07fff700 1 --2- 172.21.15.92:0/541366516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7efc003a90 0x7f7efc003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:49.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f07fff700 1 --2- 172.21.15.92:0/541366516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f100b0760 0x7f7efc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:49.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f07fff700 1 --2- 172.21.15.92:0/541366516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7efc004d50 0x7f7efc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:49.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f07fff700 1 -- 172.21.15.92:0/541366516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f100ab350 con 0x7f7efc004d50 2023-10-06T10:45:49.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f07fff700 1 -- 172.21.15.92:0/541366516 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc00bb10 con 0x7f7efc003a90 2023-10-06T10:45:49.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f07fff700 1 -- 172.21.15.92:0/541366516 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc00bcd0 con 0x7f7f100b0760 2023-10-06T10:45:49.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f15f9a700 1 --2- 172.21.15.92:0/541366516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7efc004d50 0x7f7efc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:49.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f15799700 1 --2- 172.21.15.92:0/541366516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7efc003a90 0x7f7efc003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:49.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f14f98700 1 --2- 172.21.15.92:0/541366516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f100b0760 0x7f7efc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:49.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f15f9a700 1 -- 172.21.15.92:0/541366516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f100b0760 msgr2=0x7f7efc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:49.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f15f9a700 1 --2- 172.21.15.92:0/541366516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f100b0760 0x7f7efc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f15f9a700 1 -- 172.21.15.92:0/541366516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7efc003a90 msgr2=0x7f7efc003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:49.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f15f9a700 1 --2- 172.21.15.92:0/541366516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7efc003a90 0x7f7efc003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f15f9a700 1 -- 172.21.15.92:0/541366516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc003570 con 0x7f7efc004d50 2023-10-06T10:45:49.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.597+0000 7f7f15f9a700 1 --2- 172.21.15.92:0/541366516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7efc004d50 0x7f7efc0051c0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f7f100b64b0 tx=0x7f7f080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43691df246b6c9b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:49.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.598+0000 7f7f07fff700 1 -- 172.21.15.92:0/541366516 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0800e710 con 0x7f7efc004d50 2023-10-06T10:45:49.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.598+0000 7f7f07fff700 1 -- 172.21.15.92:0/541366516 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0800e870 con 0x7f7efc004d50 2023-10-06T10:45:49.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.598+0000 7f7f07fff700 1 -- 172.21.15.92:0/541366516 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0800ec30 con 0x7f7efc004d50 2023-10-06T10:45:49.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.598+0000 7f7f179fd700 1 -- 172.21.15.92:0/541366516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7efc004d50 msgr2=0x7f7efc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:49.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.598+0000 7f7f179fd700 1 --2- 172.21.15.92:0/541366516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7efc004d50 0x7f7efc0051c0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f7f100b64b0 tx=0x7f7f080091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.598+0000 7f7f15f9a700 1 -- 172.21.15.92:0/541366516 reap_dead start 2023-10-06T10:45:49.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.598+0000 7f7f179fd700 1 -- 172.21.15.92:0/541366516 shutdown_connections 2023-10-06T10:45:49.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.598+0000 7f7f179fd700 1 -- 172.21.15.92:0/541366516 >> 172.21.15.92:0/541366516 conn(0x7f7f100997f0 msgr2=0x7f7f1006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:49.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.598+0000 7f7f179fd700 1 -- 172.21.15.92:0/541366516 shutdown_connections 2023-10-06T10:45:49.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.598+0000 7f7f179fd700 1 -- 172.21.15.92:0/541366516 wait complete. 2023-10-06T10:45:49.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.599+0000 7f7f179fd700 1 Processor -- start 2023-10-06T10:45:49.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.599+0000 7f7f179fd700 1 -- start start 2023-10-06T10:45:49.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.600+0000 7f7f179fd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f100ae420 0x7f7f100ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:49.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.600+0000 7f7f179fd700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f100cea80 0x7f7f100ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:49.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.600+0000 7f7f179fd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f1014a660 0x7f7f1014aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:49.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.600+0000 7f7f179fd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f100cf430 con 0x7f7f100ae420 2023-10-06T10:45:49.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.600+0000 7f7f179fd700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f100ad120 con 0x7f7f100cea80 2023-10-06T10:45:49.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.600+0000 7f7f179fd700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f100ac510 con 0x7f7f1014a660 2023-10-06T10:45:49.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.600+0000 7f7f15799700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f100ae420 0x7f7f100ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:49.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.600+0000 7f7f15799700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f100ae420 0x7f7f100ad5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47342/0 (socket says 172.21.15.92:47342) 2023-10-06T10:45:49.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.600+0000 7f7f15799700 1 -- 172.21.15.92:0/1032868008 learned_addr learned my addr 172.21.15.92:0/1032868008 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:49.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.600+0000 7f7f15f9a700 1 --2- 172.21.15.92:0/1032868008 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f1014a660 0x7f7f1014aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:49.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.600+0000 7f7f14f98700 1 --2- 172.21.15.92:0/1032868008 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f100cea80 0x7f7f100ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:49.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.600+0000 7f7f15799700 1 -- 172.21.15.92:0/1032868008 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f1014a660 msgr2=0x7f7f1014aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:49.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.600+0000 7f7f15799700 1 --2- 172.21.15.92:0/1032868008 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f1014a660 0x7f7f1014aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.600+0000 7f7f15799700 1 -- 172.21.15.92:0/1032868008 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f100cea80 msgr2=0x7f7f100ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:49.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.600+0000 7f7f15799700 1 --2- 172.21.15.92:0/1032868008 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f100cea80 0x7f7f100ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.600+0000 7f7f15799700 1 -- 172.21.15.92:0/1032868008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f100ac7f0 con 0x7f7f100ae420 2023-10-06T10:45:49.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.600+0000 7f7f14f98700 1 --2- 172.21.15.92:0/1032868008 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f100cea80 0x7f7f100ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:49.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.601+0000 7f7f15799700 1 --2- 172.21.15.92:0/1032868008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f100ae420 0x7f7f100ad5b0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00ff80 tx=0x7f7f0c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:49.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.601+0000 7f7f057fa700 1 -- 172.21.15.92:0/1032868008 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c005e50 con 0x7f7f100ae420 2023-10-06T10:45:49.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.601+0000 7f7f057fa700 1 -- 172.21.15.92:0/1032868008 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0c00d970 con 0x7f7f100ae420 2023-10-06T10:45:49.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.601+0000 7f7f179fd700 1 -- 172.21.15.92:0/1032868008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f100acad0 con 0x7f7f100ae420 2023-10-06T10:45:49.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.601+0000 7f7f057fa700 1 -- 172.21.15.92:0/1032868008 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c00dd30 con 0x7f7f100ae420 2023-10-06T10:45:49.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.601+0000 7f7f179fd700 1 -- 172.21.15.92:0/1032868008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f100adaf0 con 0x7f7f100ae420 2023-10-06T10:45:49.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.602+0000 7f7f179fd700 1 -- 172.21.15.92:0/1032868008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f10064a90 con 0x7f7f100ae420 2023-10-06T10:45:49.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.602+0000 7f7f057fa700 1 -- 172.21.15.92:0/1032868008 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7f0c0072e0 con 0x7f7f100ae420 2023-10-06T10:45:49.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.603+0000 7f7f057fa700 1 --2- 172.21.15.92:0/1032868008 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7ef4066f30 0x7f7ef40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:49.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.603+0000 7f7f14f98700 1 --2- 172.21.15.92:0/1032868008 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7ef4066f30 0x7f7ef40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:49.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.603+0000 7f7f057fa700 1 -- 172.21.15.92:0/1032868008 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f7f0c084d90 con 0x7f7f100ae420 2023-10-06T10:45:49.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.603+0000 7f7f14f98700 1 --2- 172.21.15.92:0/1032868008 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7ef4066f30 0x7f7ef40693e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7f00007e20 tx=0x7f7f00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:49.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.607+0000 7f7f057fa700 1 -- 172.21.15.92:0/1032868008 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f0c051d20 con 0x7f7f100ae420 2023-10-06T10:45:49.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.831+0000 7f7f179fd700 1 -- 172.21.15.92:0/1032868008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f100b1500 con 0x7f7f100ae420 2023-10-06T10:45:49.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.831+0000 7f7f057fa700 1 -- 172.21.15.92:0/1032868008 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4646 (secure 0 0 0) 0x7f7f0c055340 con 0x7f7f100ae420 2023-10-06T10:45:49.833 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:49.834 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":110,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2016052382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2016052382},{"type":"v1","addr":"172.21.15.124:6834","nonce":2016052382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:44:51.106486+0000","modified":"2023-10-06T10:45:34.353086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8520,"mds_1":8517,"mds_2":6719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6719":{"gid":6719,"name":"d","rank":2,"incarnation":104,"state":"up:active","state_seq":13,"addr":"172.21.15.124:6835/2307521923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2307521923},{"type":"v1","addr":"172.21.15.124:6835","nonce":2307521923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8517":{"gid":8517,"name":"a","rank":1,"incarnation":102,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6836/1979826278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979826278},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979826278}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6837/2298348426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2298348426},{"type":"v1","addr":"172.21.15.92:6837","nonce":2298348426}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T10:45:49.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.834+0000 7f7f179fd700 1 -- 172.21.15.92:0/1032868008 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7ef4066f30 msgr2=0x7f7ef40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:49.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.834+0000 7f7f179fd700 1 --2- 172.21.15.92:0/1032868008 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7ef4066f30 0x7f7ef40693e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7f00007e20 tx=0x7f7f00009040 comp rx=0 tx=0).stop 2023-10-06T10:45:49.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.834+0000 7f7f179fd700 1 -- 172.21.15.92:0/1032868008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f100ae420 msgr2=0x7f7f100ad5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:49.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.834+0000 7f7f179fd700 1 --2- 172.21.15.92:0/1032868008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f100ae420 0x7f7f100ad5b0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00ff80 tx=0x7f7f0c009040 comp rx=0 tx=0).stop 2023-10-06T10:45:49.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.834+0000 7f7f179fd700 1 -- 172.21.15.92:0/1032868008 shutdown_connections 2023-10-06T10:45:49.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.834+0000 7f7f179fd700 1 --2- 172.21.15.92:0/1032868008 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7ef4066f30 0x7f7ef40693e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.834+0000 7f7f179fd700 1 --2- 172.21.15.92:0/1032868008 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f1014a660 0x7f7f1014aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.834+0000 7f7f179fd700 1 --2- 172.21.15.92:0/1032868008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f100ae420 0x7f7f100ad5b0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.834+0000 7f7f179fd700 1 --2- 172.21.15.92:0/1032868008 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f100cea80 0x7f7f100ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.834+0000 7f7f179fd700 1 -- 172.21.15.92:0/1032868008 >> 172.21.15.92:0/1032868008 conn(0x7f7f100997f0 msgr2=0x7f7f100683c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:49.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.834+0000 7f7f179fd700 1 -- 172.21.15.92:0/1032868008 shutdown_connections 2023-10-06T10:45:49.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.834+0000 7f7f179fd700 1 -- 172.21.15.92:0/1032868008 wait complete. 2023-10-06T10:45:49.837 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 110 2023-10-06T10:45:49.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:45:49.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.960+0000 7fa530db1700 1 Processor -- start 2023-10-06T10:45:49.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.960+0000 7fa530db1700 1 -- start start 2023-10-06T10:45:49.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.961+0000 7fa530db1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa52c159440 0x7fa52c0b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:49.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.961+0000 7fa530db1700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa52c0b35d0 0x7fa52c0cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:49.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.961+0000 7fa530db1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa52c0cb7a0 con 0x7fa52c0cc170 2023-10-06T10:45:49.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.961+0000 7fa530db1700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52c0b0860 con 0x7fa52c159440 2023-10-06T10:45:49.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.961+0000 7fa530db1700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52c0cb150 con 0x7fa52c0b35d0 2023-10-06T10:45:49.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.961+0000 7fa52b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa52c159440 0x7fa52c0b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:49.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.961+0000 7fa52bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa52c0cc170 0x7fa52c0b2100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58804/0 (socket says 172.21.15.92:58804) 2023-10-06T10:45:49.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.961+0000 7fa52bfff700 1 -- 172.21.15.92:0/330556607 learned_addr learned my addr 172.21.15.92:0/330556607 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:49.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.961+0000 7fa52affd700 1 --2- 172.21.15.92:0/330556607 >> v2:172.21.15.124:3300/0 conn(0x7fa52c0b35d0 0x7fa52c0cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:49.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.961+0000 7fa52b7fe700 1 -- 172.21.15.92:0/330556607 >> v2:172.21.15.124:3300/0 conn(0x7fa52c0b35d0 msgr2=0x7fa52c0cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:49.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.961+0000 7fa52b7fe700 1 --2- 172.21.15.92:0/330556607 >> v2:172.21.15.124:3300/0 conn(0x7fa52c0b35d0 0x7fa52c0cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.961+0000 7fa52b7fe700 1 -- 172.21.15.92:0/330556607 >> v1:172.21.15.92:6789/0 conn(0x7fa52c0cc170 legacy=0x7fa52c0b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:49.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.962+0000 7fa52b7fe700 1 -- 172.21.15.92:0/330556607 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c0cb430 con 0x7fa52c159440 2023-10-06T10:45:49.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.961+0000 7fa52a7fc700 1 -- 172.21.15.92:0/330556607 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355909941 0 0) 0x7fa52c0cb7a0 con 0x7fa52c0cc170 2023-10-06T10:45:49.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.962+0000 7fa52b7fe700 1 --2- 172.21.15.92:0/330556607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa52c159440 0x7fa52c0b3090 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0d0220 tx=0x7fa51c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39720099f17dad5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:49.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.962+0000 7fa52a7fc700 1 -- 172.21.15.92:0/330556607 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c00e070 con 0x7fa52c159440 2023-10-06T10:45:49.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.962+0000 7fa52a7fc700 1 -- 172.21.15.92:0/330556607 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c00bc30 con 0x7fa52c159440 2023-10-06T10:45:49.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.962+0000 7fa52a7fc700 1 -- 172.21.15.92:0/330556607 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c012430 con 0x7fa52c159440 2023-10-06T10:45:49.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.962+0000 7fa530db1700 1 -- 172.21.15.92:0/330556607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa52c159440 msgr2=0x7fa52c0b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:49.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.962+0000 7fa530db1700 1 --2- 172.21.15.92:0/330556607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa52c159440 0x7fa52c0b3090 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0d0220 tx=0x7fa51c009780 comp rx=0 tx=0).stop 2023-10-06T10:45:49.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.962+0000 7fa530db1700 1 -- 172.21.15.92:0/330556607 shutdown_connections 2023-10-06T10:45:49.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.962+0000 7fa530db1700 1 --2- 172.21.15.92:0/330556607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa52c159440 0x7fa52c0b3090 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.962+0000 7fa530db1700 1 --2- 172.21.15.92:0/330556607 >> v2:172.21.15.124:3300/0 conn(0x7fa52c0b35d0 0x7fa52c0cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.962+0000 7fa530db1700 1 -- 172.21.15.92:0/330556607 >> 172.21.15.92:0/330556607 conn(0x7fa52c06ece0 msgr2=0x7fa52c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:49.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.962+0000 7fa530db1700 1 -- 172.21.15.92:0/330556607 shutdown_connections 2023-10-06T10:45:49.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.962+0000 7fa530db1700 1 -- 172.21.15.92:0/330556607 wait complete. 2023-10-06T10:45:49.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.964+0000 7fa530db1700 1 Processor -- start 2023-10-06T10:45:49.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.964+0000 7fa530db1700 1 -- start start 2023-10-06T10:45:49.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.964+0000 7fa530db1700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa52c0b35d0 0x7fa52c1664c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:49.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.964+0000 7fa530db1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa52c0cc170 0x7fa52c15cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:49.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.964+0000 7fa530db1700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa52c159440 0x7fa52c15d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:49.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.964+0000 7fa530db1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c15b8b0 con 0x7fa52c0cc170 2023-10-06T10:45:49.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.964+0000 7fa530db1700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c15bb90 con 0x7fa52c0b35d0 2023-10-06T10:45:49.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.964+0000 7fa530db1700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c15be70 con 0x7fa52c159440 2023-10-06T10:45:49.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.964+0000 7fa52affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa52c0cc170 0x7fa52c15cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:49.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.964+0000 7fa52affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa52c0cc170 0x7fa52c15cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47352/0 (socket says 172.21.15.92:47352) 2023-10-06T10:45:49.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.964+0000 7fa52affd700 1 -- 172.21.15.92:0/2152422958 learned_addr learned my addr 172.21.15.92:0/2152422958 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:49.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.965+0000 7fa52bfff700 1 --2- 172.21.15.92:0/2152422958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa52c159440 0x7fa52c15d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:49.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.965+0000 7fa52b7fe700 1 --2- 172.21.15.92:0/2152422958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa52c0b35d0 0x7fa52c1664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:49.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.965+0000 7fa52affd700 1 -- 172.21.15.92:0/2152422958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa52c159440 msgr2=0x7fa52c15d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.965+0000 7fa52affd700 1 --2- 172.21.15.92:0/2152422958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa52c159440 0x7fa52c15d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.965+0000 7fa52affd700 1 -- 172.21.15.92:0/2152422958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa52c0b35d0 msgr2=0x7fa52c1664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.965+0000 7fa52affd700 1 --2- 172.21.15.92:0/2152422958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa52c0b35d0 0x7fa52c1664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.965+0000 7fa52affd700 1 -- 172.21.15.92:0/2152422958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c009a20 con 0x7fa52c0cc170 2023-10-06T10:45:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.965+0000 7fa52b7fe700 1 --2- 172.21.15.92:0/2152422958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa52c0b35d0 0x7fa52c1664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.965+0000 7fa52affd700 1 --2- 172.21.15.92:0/2152422958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa52c0cc170 0x7fa52c15cf10 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fa52000c7d0 tx=0x7fa520008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.965+0000 7fa5137fe700 1 -- 172.21.15.92:0/2152422958 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa520011600 con 0x7fa52c0cc170 2023-10-06T10:45:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.965+0000 7fa5137fe700 1 -- 172.21.15.92:0/2152422958 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa520011760 con 0x7fa52c0cc170 2023-10-06T10:45:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.965+0000 7fa530db1700 1 -- 172.21.15.92:0/2152422958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52c15c150 con 0x7fa52c0cc170 2023-10-06T10:45:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.965+0000 7fa5137fe700 1 -- 172.21.15.92:0/2152422958 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa520011b20 con 0x7fa52c0cc170 2023-10-06T10:45:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.965+0000 7fa530db1700 1 -- 172.21.15.92:0/2152422958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52c166bc0 con 0x7fa52c0cc170 2023-10-06T10:45:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.966+0000 7fa530db1700 1 -- 172.21.15.92:0/2152422958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa52c05f230 con 0x7fa52c0cc170 2023-10-06T10:45:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.966+0000 7fa5137fe700 1 -- 172.21.15.92:0/2152422958 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa520018350 con 0x7fa52c0cc170 2023-10-06T10:45:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.967+0000 7fa5137fe700 1 --2- 172.21.15.92:0/2152422958 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa518066f30 0x7fa5180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.967+0000 7fa5137fe700 1 -- 172.21.15.92:0/2152422958 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fa5200846d0 con 0x7fa52c0cc170 2023-10-06T10:45:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.967+0000 7fa52b7fe700 1 --2- 172.21.15.92:0/2152422958 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa518066f30 0x7fa5180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.967+0000 7fa52b7fe700 1 --2- 172.21.15.92:0/2152422958 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa518066f30 0x7fa5180693e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0d0220 tx=0x7fa51c019000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:49.971+0000 7fa5137fe700 1 -- 172.21.15.92:0/2152422958 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa520051660 con 0x7fa52c0cc170 2023-10-06T10:45:50.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.156+0000 7fa530db1700 1 -- 172.21.15.92:0/2152422958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa52c159810 con 0x7fa52c0cc170 2023-10-06T10:45:50.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.157+0000 7fa5137fe700 1 -- 172.21.15.92:0/2152422958 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+17181 (secure 0 0 0) 0x7fa520054c80 con 0x7fa52c0cc170 2023-10-06T10:45:50.159 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:50.160 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":104,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:45:07.376972+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:44:48.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-06T10:44:50.072416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.4","mappings":[{"from":0,"to":5}]},{"pgid":"17.5","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:45:50.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.159+0000 7fa530db1700 1 -- 172.21.15.92:0/2152422958 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa518066f30 msgr2=0x7fa5180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:50.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.159+0000 7fa530db1700 1 --2- 172.21.15.92:0/2152422958 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa518066f30 0x7fa5180693e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0d0220 tx=0x7fa51c019000 comp rx=0 tx=0).stop 2023-10-06T10:45:50.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.159+0000 7fa530db1700 1 -- 172.21.15.92:0/2152422958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa52c0cc170 msgr2=0x7fa52c15cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:50.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.159+0000 7fa530db1700 1 --2- 172.21.15.92:0/2152422958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa52c0cc170 0x7fa52c15cf10 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fa52000c7d0 tx=0x7fa520008040 comp rx=0 tx=0).stop 2023-10-06T10:45:50.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.159+0000 7fa530db1700 1 -- 172.21.15.92:0/2152422958 shutdown_connections 2023-10-06T10:45:50.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.160+0000 7fa530db1700 1 --2- 172.21.15.92:0/2152422958 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa518066f30 0x7fa5180693e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.160+0000 7fa530db1700 1 --2- 172.21.15.92:0/2152422958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa52c159440 0x7fa52c15d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.160+0000 7fa530db1700 1 --2- 172.21.15.92:0/2152422958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa52c0b35d0 0x7fa52c1664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.160+0000 7fa530db1700 1 --2- 172.21.15.92:0/2152422958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa52c0cc170 0x7fa52c15cf10 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.160+0000 7fa530db1700 1 -- 172.21.15.92:0/2152422958 >> 172.21.15.92:0/2152422958 conn(0x7fa52c06ece0 msgr2=0x7fa52c1573f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:50.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.160+0000 7fa530db1700 1 -- 172.21.15.92:0/2152422958 shutdown_connections 2023-10-06T10:45:50.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.160+0000 7fa530db1700 1 -- 172.21.15.92:0/2152422958 wait complete. 2023-10-06T10:45:50.174 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T10:45:50.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:45:50.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.288+0000 7fc96e1be700 1 Processor -- start 2023-10-06T10:45:50.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.289+0000 7fc96e1be700 1 -- start start 2023-10-06T10:45:50.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.289+0000 7fc96e1be700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc9680adb60 0x7fc9680ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:50.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.289+0000 7fc96e1be700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9680ce6d0 0x7fc96815aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:50.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.289+0000 7fc96e1be700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc968157de0 con 0x7fc9680ca370 2023-10-06T10:45:50.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.289+0000 7fc96e1be700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9680cbb90 con 0x7fc9680adb60 2023-10-06T10:45:50.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.289+0000 7fc96e1be700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9680ceb10 con 0x7fc9680ce6d0 2023-10-06T10:45:50.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.290+0000 7fc967fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9680ce6d0 0x7fc96815aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:50.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.290+0000 7fc95effd700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc9680adb60 0x7fc9680ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:50.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.290+0000 7fc9677fe700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fc9680ca370 0x7fc968158270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:51292/0 (socket says 172.21.15.92:51292) 2023-10-06T10:45:50.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.290+0000 7fc9677fe700 1 -- 172.21.15.92:0/1682093032 learned_addr learned my addr 172.21.15.92:0/1682093032 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:50.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.290+0000 7fc966ffd700 1 -- 172.21.15.92:0/1682093032 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964398466 0 0) 0x7fc968157de0 con 0x7fc9680ca370 2023-10-06T10:45:50.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.290+0000 7fc967fff700 1 -- 172.21.15.92:0/1682093032 >> v2:172.21.15.124:3300/0 conn(0x7fc9680adb60 msgr2=0x7fc9680ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:50.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.290+0000 7fc967fff700 1 --2- 172.21.15.92:0/1682093032 >> v2:172.21.15.124:3300/0 conn(0x7fc9680adb60 0x7fc9680ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.290+0000 7fc967fff700 1 -- 172.21.15.92:0/1682093032 >> v1:172.21.15.124:6789/0 conn(0x7fc9680ca370 legacy=0x7fc968158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:50.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.290+0000 7fc967fff700 1 -- 172.21.15.92:0/1682093032 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9680cb5d0 con 0x7fc9680ce6d0 2023-10-06T10:45:50.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.290+0000 7fc967fff700 1 --2- 172.21.15.92:0/1682093032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9680ce6d0 0x7fc96815aed0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc960009e60 tx=0x7fc960009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85edba0bdcedfebe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:50.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.290+0000 7fc966ffd700 1 -- 172.21.15.92:0/1682093032 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96000e070 con 0x7fc9680ce6d0 2023-10-06T10:45:50.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.290+0000 7fc966ffd700 1 -- 172.21.15.92:0/1682093032 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96000bb60 con 0x7fc9680ce6d0 2023-10-06T10:45:50.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.290+0000 7fc966ffd700 1 -- 172.21.15.92:0/1682093032 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc960012430 con 0x7fc9680ce6d0 2023-10-06T10:45:50.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.291+0000 7fc96e1be700 1 -- 172.21.15.92:0/1682093032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9680ce6d0 msgr2=0x7fc96815aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:50.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.291+0000 7fc96e1be700 1 --2- 172.21.15.92:0/1682093032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9680ce6d0 0x7fc96815aed0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc960009e60 tx=0x7fc960009780 comp rx=0 tx=0).stop 2023-10-06T10:45:50.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.291+0000 7fc96e1be700 1 -- 172.21.15.92:0/1682093032 shutdown_connections 2023-10-06T10:45:50.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.291+0000 7fc96e1be700 1 --2- 172.21.15.92:0/1682093032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9680ce6d0 0x7fc96815aed0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.291+0000 7fc96e1be700 1 --2- 172.21.15.92:0/1682093032 >> v2:172.21.15.124:3300/0 conn(0x7fc9680adb60 0x7fc9680ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.291+0000 7fc96e1be700 1 -- 172.21.15.92:0/1682093032 >> 172.21.15.92:0/1682093032 conn(0x7fc9680707d0 msgr2=0x7fc968070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:50.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.291+0000 7fc96e1be700 1 -- 172.21.15.92:0/1682093032 shutdown_connections 2023-10-06T10:45:50.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.291+0000 7fc96e1be700 1 -- 172.21.15.92:0/1682093032 wait complete. 2023-10-06T10:45:50.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.292+0000 7fc96e1be700 1 Processor -- start 2023-10-06T10:45:50.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.292+0000 7fc96e1be700 1 -- start start 2023-10-06T10:45:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.293+0000 7fc96e1be700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9680adb60 0x7fc9681455b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.293+0000 7fc96e1be700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9680ca370 0x7fc9681554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.293+0000 7fc96e1be700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9680ce6d0 0x7fc9681559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.293+0000 7fc96e1be700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9680c5070 con 0x7fc9680adb60 2023-10-06T10:45:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.293+0000 7fc96e1be700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9680c4d90 con 0x7fc9680ca370 2023-10-06T10:45:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.293+0000 7fc96e1be700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9680b1430 con 0x7fc9680ce6d0 2023-10-06T10:45:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.293+0000 7fc9677fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9680adb60 0x7fc9681455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.293+0000 7fc9677fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9680adb60 0x7fc9681455b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47360/0 (socket says 172.21.15.92:47360) 2023-10-06T10:45:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.293+0000 7fc9677fe700 1 -- 172.21.15.92:0/1874212926 learned_addr learned my addr 172.21.15.92:0/1874212926 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.293+0000 7fc967fff700 1 --2- 172.21.15.92:0/1874212926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9680ce6d0 0x7fc9681559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.293+0000 7fc95effd700 1 --2- 172.21.15.92:0/1874212926 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9680ca370 0x7fc9681554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.293+0000 7fc9677fe700 1 -- 172.21.15.92:0/1874212926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9680ce6d0 msgr2=0x7fc9681559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.293+0000 7fc9677fe700 1 --2- 172.21.15.92:0/1874212926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9680ce6d0 0x7fc9681559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.293+0000 7fc9677fe700 1 -- 172.21.15.92:0/1874212926 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9680ca370 msgr2=0x7fc9681554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.294+0000 7fc9677fe700 1 --2- 172.21.15.92:0/1874212926 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9680ca370 0x7fc9681554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.294+0000 7fc9677fe700 1 -- 172.21.15.92:0/1874212926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc960009a20 con 0x7fc9680adb60 2023-10-06T10:45:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.294+0000 7fc9677fe700 1 --2- 172.21.15.92:0/1874212926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9680adb60 0x7fc9681455b0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fc95800c7e0 tx=0x7fc958007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.294+0000 7fc95ffff700 1 -- 172.21.15.92:0/1874212926 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958005b30 con 0x7fc9680adb60 2023-10-06T10:45:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.294+0000 7fc95ffff700 1 -- 172.21.15.92:0/1874212926 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc958005c90 con 0x7fc9680adb60 2023-10-06T10:45:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.294+0000 7fc96e1be700 1 -- 172.21.15.92:0/1874212926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9680b1710 con 0x7fc9680adb60 2023-10-06T10:45:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.294+0000 7fc95ffff700 1 -- 172.21.15.92:0/1874212926 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958007300 con 0x7fc9680adb60 2023-10-06T10:45:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.294+0000 7fc96e1be700 1 -- 172.21.15.92:0/1874212926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc968145af0 con 0x7fc9680adb60 2023-10-06T10:45:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.295+0000 7fc96e1be700 1 -- 172.21.15.92:0/1874212926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc968064a90 con 0x7fc9680adb60 2023-10-06T10:45:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.295+0000 7fc95ffff700 1 -- 172.21.15.92:0/1874212926 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc95801b3b0 con 0x7fc9680adb60 2023-10-06T10:45:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.296+0000 7fc95ffff700 1 --2- 172.21.15.92:0/1874212926 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc948066f30 0x7fc9480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.296+0000 7fc95effd700 1 --2- 172.21.15.92:0/1874212926 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc948066f30 0x7fc9480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.296+0000 7fc95ffff700 1 -- 172.21.15.92:0/1874212926 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fc958083c60 con 0x7fc9680adb60 2023-10-06T10:45:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.296+0000 7fc95effd700 1 --2- 172.21.15.92:0/1874212926 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc948066f30 0x7fc9480693e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc9680ad660 tx=0x7fc9500091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.300+0000 7fc95ffff700 1 -- 172.21.15.92:0/1874212926 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc958050bf0 con 0x7fc9680adb60 2023-10-06T10:45:50.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.525+0000 7fc96e1be700 1 -- 172.21.15.92:0/1874212926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9680cb8b0 con 0x7fc9680adb60 2023-10-06T10:45:50.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.525+0000 7fc95ffff700 1 -- 172.21.15.92:0/1874212926 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4646 (secure 0 0 0) 0x7fc958017020 con 0x7fc9680adb60 2023-10-06T10:45:50.528 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:50.528 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":110,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2016052382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2016052382},{"type":"v1","addr":"172.21.15.124:6834","nonce":2016052382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:44:51.106486+0000","modified":"2023-10-06T10:45:34.353086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8520,"mds_1":8517,"mds_2":6719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6719":{"gid":6719,"name":"d","rank":2,"incarnation":104,"state":"up:active","state_seq":13,"addr":"172.21.15.124:6835/2307521923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2307521923},{"type":"v1","addr":"172.21.15.124:6835","nonce":2307521923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8517":{"gid":8517,"name":"a","rank":1,"incarnation":102,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6836/1979826278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979826278},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979826278}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6837/2298348426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2298348426},{"type":"v1","addr":"172.21.15.92:6837","nonce":2298348426}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T10:45:50.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.528+0000 7fc96e1be700 1 -- 172.21.15.92:0/1874212926 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc948066f30 msgr2=0x7fc9480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:50.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.528+0000 7fc96e1be700 1 --2- 172.21.15.92:0/1874212926 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc948066f30 0x7fc9480693e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc9680ad660 tx=0x7fc9500091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.528+0000 7fc96e1be700 1 -- 172.21.15.92:0/1874212926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9680adb60 msgr2=0x7fc9681455b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:50.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.528+0000 7fc96e1be700 1 --2- 172.21.15.92:0/1874212926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9680adb60 0x7fc9681455b0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fc95800c7e0 tx=0x7fc958007df0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.528+0000 7fc96e1be700 1 -- 172.21.15.92:0/1874212926 shutdown_connections 2023-10-06T10:45:50.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.528+0000 7fc96e1be700 1 --2- 172.21.15.92:0/1874212926 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fc948066f30 0x7fc9480693e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.528+0000 7fc96e1be700 1 --2- 172.21.15.92:0/1874212926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9680ce6d0 0x7fc9681559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.528+0000 7fc96e1be700 1 --2- 172.21.15.92:0/1874212926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9680adb60 0x7fc9681455b0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.528+0000 7fc96e1be700 1 --2- 172.21.15.92:0/1874212926 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9680ca370 0x7fc9681554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.528+0000 7fc96e1be700 1 -- 172.21.15.92:0/1874212926 >> 172.21.15.92:0/1874212926 conn(0x7fc9680707d0 msgr2=0x7fc9680cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:50.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.528+0000 7fc96e1be700 1 -- 172.21.15.92:0/1874212926 shutdown_connections 2023-10-06T10:45:50.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.528+0000 7fc96e1be700 1 -- 172.21.15.92:0/1874212926 wait complete. 2023-10-06T10:45:50.531 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 110 2023-10-06T10:45:50.541 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:45:50.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.657+0000 7fa59ddcc700 1 Processor -- start 2023-10-06T10:45:50.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.657+0000 7fa59ddcc700 1 -- start start 2023-10-06T10:45:50.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.658+0000 7fa59ddcc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa5980b49a0 0x7fa5980aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:50.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.658+0000 7fa59ddcc700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa5980b0760 0x7fa5980b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:50.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.658+0000 7fa59ddcc700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5980b0c60 con 0x7fa598159560 2023-10-06T10:45:50.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.658+0000 7fa59ddcc700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5980b0f40 con 0x7fa5980b49a0 2023-10-06T10:45:50.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.658+0000 7fa59ddcc700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5980b1220 con 0x7fa5980b0760 2023-10-06T10:45:50.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.658+0000 7fa5977fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa598159560 0x7fa5980aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58814/0 (socket says 172.21.15.92:58814) 2023-10-06T10:45:50.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.658+0000 7fa596ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa5980b49a0 0x7fa5980aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:50.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.658+0000 7fa5977fe700 1 -- 172.21.15.92:0/3860412712 learned_addr learned my addr 172.21.15.92:0/3860412712 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:50.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.658+0000 7fa596ffd700 1 -- 172.21.15.92:0/3860412712 >> v2:172.21.15.124:3300/0 conn(0x7fa5980b0760 msgr2=0x7fa5980b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:50.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.658+0000 7fa596ffd700 1 --2- 172.21.15.92:0/3860412712 >> v2:172.21.15.124:3300/0 conn(0x7fa5980b0760 0x7fa5980b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.658+0000 7fa596ffd700 1 -- 172.21.15.92:0/3860412712 >> v1:172.21.15.92:6789/0 conn(0x7fa598159560 legacy=0x7fa5980aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:50.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.658+0000 7fa596ffd700 1 -- 172.21.15.92:0/3860412712 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5980b1500 con 0x7fa5980b49a0 2023-10-06T10:45:50.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.658+0000 7fa5967fc700 1 -- 172.21.15.92:0/3860412712 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093189383 0 0) 0x7fa5980b0c60 con 0x7fa598159560 2023-10-06T10:45:50.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.659+0000 7fa596ffd700 1 --2- 172.21.15.92:0/3860412712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5980b49a0 0x7fa5980aac40 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0098e0 tx=0x7fa58c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e36a67f133589d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:50.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.659+0000 7fa5967fc700 1 -- 172.21.15.92:0/3860412712 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c009e80 con 0x7fa5980b49a0 2023-10-06T10:45:50.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.659+0000 7fa5967fc700 1 -- 172.21.15.92:0/3860412712 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c00b820 con 0x7fa5980b49a0 2023-10-06T10:45:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.659+0000 7fa5967fc700 1 -- 172.21.15.92:0/3860412712 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c00bba0 con 0x7fa5980b49a0 2023-10-06T10:45:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.659+0000 7fa59ddcc700 1 -- 172.21.15.92:0/3860412712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5980b49a0 msgr2=0x7fa5980aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.659+0000 7fa59ddcc700 1 --2- 172.21.15.92:0/3860412712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5980b49a0 0x7fa5980aac40 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0098e0 tx=0x7fa58c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.659+0000 7fa59ddcc700 1 -- 172.21.15.92:0/3860412712 shutdown_connections 2023-10-06T10:45:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.659+0000 7fa59ddcc700 1 --2- 172.21.15.92:0/3860412712 >> v2:172.21.15.124:3300/0 conn(0x7fa5980b0760 0x7fa5980b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.659+0000 7fa59ddcc700 1 --2- 172.21.15.92:0/3860412712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5980b49a0 0x7fa5980aac40 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.659+0000 7fa59ddcc700 1 -- 172.21.15.92:0/3860412712 >> 172.21.15.92:0/3860412712 conn(0x7fa5980997f0 msgr2=0x7fa59806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.659+0000 7fa59ddcc700 1 -- 172.21.15.92:0/3860412712 shutdown_connections 2023-10-06T10:45:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.659+0000 7fa59ddcc700 1 -- 172.21.15.92:0/3860412712 wait complete. 2023-10-06T10:45:50.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.661+0000 7fa59ddcc700 1 Processor -- start 2023-10-06T10:45:50.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.661+0000 7fa59ddcc700 1 -- start start 2023-10-06T10:45:50.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.661+0000 7fa59ddcc700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5980b0760 0x7fa59815c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:50.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.661+0000 7fa59ddcc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5980b49a0 0x7fa598159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:50.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.661+0000 7fa59ddcc700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa598159560 0x7fa598159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:50.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.661+0000 7fa59ddcc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59809dfc0 con 0x7fa5980b49a0 2023-10-06T10:45:50.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.661+0000 7fa59ddcc700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59809a840 con 0x7fa5980b0760 2023-10-06T10:45:50.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.661+0000 7fa59ddcc700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa59809dce0 con 0x7fa598159560 2023-10-06T10:45:50.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.661+0000 7fa596ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5980b49a0 0x7fa598159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:50.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.661+0000 7fa596ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5980b49a0 0x7fa598159930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47370/0 (socket says 172.21.15.92:47370) 2023-10-06T10:45:50.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.661+0000 7fa596ffd700 1 -- 172.21.15.92:0/3976176531 learned_addr learned my addr 172.21.15.92:0/3976176531 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:50.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.662+0000 7fa5977fe700 1 --2- 172.21.15.92:0/3976176531 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5980b0760 0x7fa59815c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:50.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.662+0000 7fa597fff700 1 --2- 172.21.15.92:0/3976176531 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa598159560 0x7fa598159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:50.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.662+0000 7fa596ffd700 1 -- 172.21.15.92:0/3976176531 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa598159560 msgr2=0x7fa598159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:50.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.662+0000 7fa596ffd700 1 --2- 172.21.15.92:0/3976176531 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa598159560 0x7fa598159e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.662+0000 7fa596ffd700 1 -- 172.21.15.92:0/3976176531 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5980b0760 msgr2=0x7fa59815c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:50.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.662+0000 7fa596ffd700 1 --2- 172.21.15.92:0/3976176531 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5980b0760 0x7fa59815c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.662+0000 7fa596ffd700 1 -- 172.21.15.92:0/3976176531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58c009420 con 0x7fa5980b49a0 2023-10-06T10:45:50.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.662+0000 7fa596ffd700 1 --2- 172.21.15.92:0/3976176531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5980b49a0 0x7fa598159930 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fa58c013000 tx=0x7fa58c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:50.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.662+0000 7fa57f7fe700 1 -- 172.21.15.92:0/3976176531 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c011420 con 0x7fa5980b49a0 2023-10-06T10:45:50.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.662+0000 7fa57f7fe700 1 -- 172.21.15.92:0/3976176531 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c0115a0 con 0x7fa5980b49a0 2023-10-06T10:45:50.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.662+0000 7fa59ddcc700 1 -- 172.21.15.92:0/3976176531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59809a1f0 con 0x7fa5980b49a0 2023-10-06T10:45:50.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.662+0000 7fa57f7fe700 1 -- 172.21.15.92:0/3976176531 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c011960 con 0x7fa5980b49a0 2023-10-06T10:45:50.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.662+0000 7fa59ddcc700 1 -- 172.21.15.92:0/3976176531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59815cc60 con 0x7fa5980b49a0 2023-10-06T10:45:50.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.663+0000 7fa59ddcc700 1 -- 172.21.15.92:0/3976176531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa59809a050 con 0x7fa5980b49a0 2023-10-06T10:45:50.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.663+0000 7fa57f7fe700 1 -- 172.21.15.92:0/3976176531 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa58c011d50 con 0x7fa5980b49a0 2023-10-06T10:45:50.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.664+0000 7fa57f7fe700 1 --2- 172.21.15.92:0/3976176531 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa584066f30 0x7fa5840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:50.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.664+0000 7fa57f7fe700 1 -- 172.21.15.92:0/3976176531 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fa58c086340 con 0x7fa5980b49a0 2023-10-06T10:45:50.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.664+0000 7fa5977fe700 1 --2- 172.21.15.92:0/3976176531 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa584066f30 0x7fa5840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:50.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.664+0000 7fa5977fe700 1 --2- 172.21.15.92:0/3976176531 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa584066f30 0x7fa5840693e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa588001e30 tx=0x7fa588009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:50.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.668+0000 7fa57f7fe700 1 -- 172.21.15.92:0/3976176531 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa59809a050 con 0x7fa5980b49a0 2023-10-06T10:45:50.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.894+0000 7fa59ddcc700 1 -- 172.21.15.92:0/3976176531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa59809a050 con 0x7fa5980b49a0 2023-10-06T10:45:50.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.895+0000 7fa57f7fe700 1 -- 172.21.15.92:0/3976176531 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4646 (secure 0 0 0) 0x7fa59809a050 con 0x7fa5980b49a0 2023-10-06T10:45:50.897 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:50.897 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":110,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2016052382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2016052382},{"type":"v1","addr":"172.21.15.124:6834","nonce":2016052382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T10:44:51.106486+0000","modified":"2023-10-06T10:45:34.353086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8520,"mds_1":8517,"mds_2":6719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6719":{"gid":6719,"name":"d","rank":2,"incarnation":104,"state":"up:active","state_seq":13,"addr":"172.21.15.124:6835/2307521923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":2307521923},{"type":"v1","addr":"172.21.15.124:6835","nonce":2307521923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8517":{"gid":8517,"name":"a","rank":1,"incarnation":102,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6836/1979826278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979826278},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979826278}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8520":{"gid":8520,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":13,"addr":"172.21.15.92:6837/2298348426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2298348426},{"type":"v1","addr":"172.21.15.92:6837","nonce":2298348426}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T10:45:50.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.897+0000 7fa59ddcc700 1 -- 172.21.15.92:0/3976176531 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa584066f30 msgr2=0x7fa5840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:50.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.897+0000 7fa59ddcc700 1 --2- 172.21.15.92:0/3976176531 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa584066f30 0x7fa5840693e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa588001e30 tx=0x7fa588009040 comp rx=0 tx=0).stop 2023-10-06T10:45:50.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.897+0000 7fa59ddcc700 1 -- 172.21.15.92:0/3976176531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5980b49a0 msgr2=0x7fa598159930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:50.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.897+0000 7fa59ddcc700 1 --2- 172.21.15.92:0/3976176531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5980b49a0 0x7fa598159930 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fa58c013000 tx=0x7fa58c017000 comp rx=0 tx=0).stop 2023-10-06T10:45:50.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.897+0000 7fa59ddcc700 1 -- 172.21.15.92:0/3976176531 shutdown_connections 2023-10-06T10:45:50.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.897+0000 7fa59ddcc700 1 --2- 172.21.15.92:0/3976176531 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa584066f30 0x7fa5840693e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.897+0000 7fa59ddcc700 1 --2- 172.21.15.92:0/3976176531 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa598159560 0x7fa598159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.897+0000 7fa59ddcc700 1 --2- 172.21.15.92:0/3976176531 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5980b0760 0x7fa59815c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.897+0000 7fa59ddcc700 1 --2- 172.21.15.92:0/3976176531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5980b49a0 0x7fa598159930 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:50.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.897+0000 7fa59ddcc700 1 -- 172.21.15.92:0/3976176531 >> 172.21.15.92:0/3976176531 conn(0x7fa5980997f0 msgr2=0x7fa5981588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:50.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.897+0000 7fa59ddcc700 1 -- 172.21.15.92:0/3976176531 shutdown_connections 2023-10-06T10:45:50.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:50.897+0000 7fa59ddcc700 1 -- 172.21.15.92:0/3976176531 wait complete. 2023-10-06T10:45:50.900 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 110 2023-10-06T10:45:50.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:45:51.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.022+0000 7f3c9809b700 1 Processor -- start 2023-10-06T10:45:51.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.022+0000 7f3c9809b700 1 -- start start 2023-10-06T10:45:51.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.023+0000 7f3c9809b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c900b29b0 0x7f3c90159120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:51.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.023+0000 7f3c9809b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3c90159660 0x7f3c900cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:51.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.023+0000 7f3c9809b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c900b0da0 con 0x7f3c900b25e0 2023-10-06T10:45:51.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.023+0000 7f3c9809b700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c900cbfa0 con 0x7f3c90159660 2023-10-06T10:45:51.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.023+0000 7f3c9809b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c900cb390 con 0x7f3c900b29b0 2023-10-06T10:45:51.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.023+0000 7f3c95e37700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3c900b25e0 0x7f3c900b0690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58828/0 (socket says 172.21.15.92:58828) 2023-10-06T10:45:51.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.023+0000 7f3c95e37700 1 -- 172.21.15.92:0/1203568388 learned_addr learned my addr 172.21.15.92:0/1203568388 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:51.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.023+0000 7f3c95636700 1 --2- 172.21.15.92:0/1203568388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c900b29b0 0x7f3c90159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:51.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.023+0000 7f3c96638700 1 --2- 172.21.15.92:0/1203568388 >> v2:172.21.15.124:3300/0 conn(0x7f3c90159660 0x7f3c900cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:51.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c94e35700 1 -- 172.21.15.92:0/1203568388 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232449980 0 0) 0x7f3c8c001290 con 0x7f3c900b25e0 2023-10-06T10:45:51.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c94e35700 1 -- 172.21.15.92:0/1203568388 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c7c003570 con 0x7f3c900b25e0 2023-10-06T10:45:51.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c94e35700 1 -- 172.21.15.92:0/1203568388 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1971872242 0 0) 0x7f3c7c003570 con 0x7f3c900b25e0 2023-10-06T10:45:51.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c94e35700 1 -- 172.21.15.92:0/1203568388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c900b29b0 msgr2=0x7f3c90159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:51.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c94e35700 1 --2- 172.21.15.92:0/1203568388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c900b29b0 0x7f3c90159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c94e35700 1 -- 172.21.15.92:0/1203568388 >> v2:172.21.15.124:3300/0 conn(0x7f3c90159660 msgr2=0x7f3c900cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c94e35700 1 --2- 172.21.15.92:0/1203568388 >> v2:172.21.15.124:3300/0 conn(0x7f3c90159660 0x7f3c900cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c94e35700 1 -- 172.21.15.92:0/1203568388 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c900cb670 con 0x7f3c900b25e0 2023-10-06T10:45:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c94e35700 1 -- 172.21.15.92:0/1203568388 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f3c8c001780 con 0x7f3c900b25e0 2023-10-06T10:45:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c94e35700 1 -- 172.21.15.92:0/1203568388 >> v1:172.21.15.92:6789/0 conn(0x7f3c900b25e0 legacy=0x7f3c900b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c94e35700 1 --2- 172.21.15.92:0/1203568388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c7c003a60 0x7f3c7c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c94e35700 1 --2- 172.21.15.92:0/1203568388 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c7c004810 0x7f3c7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c94e35700 1 --2- 172.21.15.92:0/1203568388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c7c0051c0 0x7f3c7c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c94e35700 1 -- 172.21.15.92:0/1203568388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c900cbfa0 con 0x7f3c7c003a60 2023-10-06T10:45:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c94e35700 1 -- 172.21.15.92:0/1203568388 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c900cb390 con 0x7f3c7c004810 2023-10-06T10:45:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c94e35700 1 -- 172.21.15.92:0/1203568388 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c00bfa0 con 0x7f3c7c0051c0 2023-10-06T10:45:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c95636700 1 --2- 172.21.15.92:0/1203568388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c900b29b0 0x7f3c90159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:45:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c96638700 1 --2- 172.21.15.92:0/1203568388 >> v2:172.21.15.124:3300/0 conn(0x7f3c90159660 0x7f3c900cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:45:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c95e37700 1 --2- 172.21.15.92:0/1203568388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c7c003a60 0x7f3c7c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c96638700 1 --2- 172.21.15.92:0/1203568388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c7c0051c0 0x7f3c7c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c95636700 1 --2- 172.21.15.92:0/1203568388 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c7c004810 0x7f3c7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c95e37700 1 -- 172.21.15.92:0/1203568388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c7c0051c0 msgr2=0x7f3c7c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c95e37700 1 --2- 172.21.15.92:0/1203568388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c7c0051c0 0x7f3c7c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c95e37700 1 -- 172.21.15.92:0/1203568388 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c7c004810 msgr2=0x7f3c7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c95e37700 1 --2- 172.21.15.92:0/1203568388 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c7c004810 0x7f3c7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c95e37700 1 -- 172.21.15.92:0/1203568388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7c003570 con 0x7f3c7c003a60 2023-10-06T10:45:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c96638700 1 --2- 172.21.15.92:0/1203568388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c7c0051c0 0x7f3c7c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.024+0000 7f3c95e37700 1 --2- 172.21.15.92:0/1203568388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c7c003a60 0x7f3c7c003e70 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c00c620 tx=0x7f3c8c007ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949998400d512678 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.025+0000 7f3c94e35700 1 -- 172.21.15.92:0/1203568388 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c00cde0 con 0x7f3c7c003a60 2023-10-06T10:45:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.025+0000 7f3c94e35700 1 -- 172.21.15.92:0/1203568388 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c8c0070d0 con 0x7f3c7c003a60 2023-10-06T10:45:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.025+0000 7f3c94e35700 1 -- 172.21.15.92:0/1203568388 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c007490 con 0x7f3c7c003a60 2023-10-06T10:45:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.025+0000 7f3c9809b700 1 -- 172.21.15.92:0/1203568388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c7c003a60 msgr2=0x7f3c7c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.025+0000 7f3c9809b700 1 --2- 172.21.15.92:0/1203568388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c7c003a60 0x7f3c7c003e70 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c00c620 tx=0x7f3c8c007ab0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.025+0000 7f3c96638700 1 -- 172.21.15.92:0/1203568388 reap_dead start 2023-10-06T10:45:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.025+0000 7f3c9809b700 1 -- 172.21.15.92:0/1203568388 shutdown_connections 2023-10-06T10:45:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.025+0000 7f3c9809b700 1 -- 172.21.15.92:0/1203568388 >> 172.21.15.92:0/1203568388 conn(0x7f3c9006ece0 msgr2=0x7f3c9006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.025+0000 7f3c9809b700 1 -- 172.21.15.92:0/1203568388 shutdown_connections 2023-10-06T10:45:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.025+0000 7f3c9809b700 1 -- 172.21.15.92:0/1203568388 wait complete. 2023-10-06T10:45:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.026+0000 7f3c9809b700 1 Processor -- start 2023-10-06T10:45:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.026+0000 7f3c9809b700 1 -- start start 2023-10-06T10:45:51.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.027+0000 7f3c9809b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c7c003a60 0x7f3c90153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:51.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.027+0000 7f3c9809b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c9015c050 0x7f3c9016ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:51.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.027+0000 7f3c9809b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c9015a020 0x7f3c900ce200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:51.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.027+0000 7f3c9809b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9016a120 con 0x7f3c9015a020 2023-10-06T10:45:51.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.027+0000 7f3c9809b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c900c5d60 con 0x7f3c9015c050 2023-10-06T10:45:51.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.027+0000 7f3c9809b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c90153db0 con 0x7f3c7c003a60 2023-10-06T10:45:51.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.027+0000 7f3c96638700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c9015a020 0x7f3c900ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:51.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.027+0000 7f3c96638700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c9015a020 0x7f3c900ce200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47378/0 (socket says 172.21.15.92:47378) 2023-10-06T10:45:51.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.027+0000 7f3c96638700 1 -- 172.21.15.92:0/187445519 learned_addr learned my addr 172.21.15.92:0/187445519 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.027+0000 7f3c95e37700 1 --2- 172.21.15.92:0/187445519 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c7c003a60 0x7f3c90153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.027+0000 7f3c95636700 1 --2- 172.21.15.92:0/187445519 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c9015c050 0x7f3c9016ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.027+0000 7f3c96638700 1 -- 172.21.15.92:0/187445519 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c7c003a60 msgr2=0x7f3c90153870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.027+0000 7f3c96638700 1 --2- 172.21.15.92:0/187445519 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c7c003a60 0x7f3c90153870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.027+0000 7f3c96638700 1 -- 172.21.15.92:0/187445519 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c9015c050 msgr2=0x7f3c9016ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.027+0000 7f3c96638700 1 --2- 172.21.15.92:0/187445519 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c9015c050 0x7f3c9016ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.027+0000 7f3c96638700 1 -- 172.21.15.92:0/187445519 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9015c4c0 con 0x7f3c9015a020 2023-10-06T10:45:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.027+0000 7f3c95636700 1 --2- 172.21.15.92:0/187445519 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c9015c050 0x7f3c9016ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.027+0000 7f3c96638700 1 --2- 172.21.15.92:0/187445519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c9015a020 0x7f3c900ce200 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f3c9015d1c0 tx=0x7f3c8800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.028+0000 7f3c85ffb700 1 -- 172.21.15.92:0/187445519 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8800e070 con 0x7f3c9015a020 2023-10-06T10:45:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.028+0000 7f3c85ffb700 1 -- 172.21.15.92:0/187445519 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c88011d00 con 0x7f3c9015a020 2023-10-06T10:45:51.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.028+0000 7f3c9809b700 1 -- 172.21.15.92:0/187445519 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c9015c7a0 con 0x7f3c9015a020 2023-10-06T10:45:51.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.028+0000 7f3c85ffb700 1 -- 172.21.15.92:0/187445519 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c880166a0 con 0x7f3c9015a020 2023-10-06T10:45:51.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.028+0000 7f3c9809b700 1 -- 172.21.15.92:0/187445519 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c900c59f0 con 0x7f3c9015a020 2023-10-06T10:45:51.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.029+0000 7f3c9809b700 1 -- 172.21.15.92:0/187445519 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c900c5570 con 0x7f3c9015a020 2023-10-06T10:45:51.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.029+0000 7f3c85ffb700 1 -- 172.21.15.92:0/187445519 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c88016a50 con 0x7f3c9015a020 2023-10-06T10:45:51.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.030+0000 7f3c85ffb700 1 --2- 172.21.15.92:0/187445519 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c74066f30 0x7f3c740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:51.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.030+0000 7f3c85ffb700 1 -- 172.21.15.92:0/187445519 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f3c880852f0 con 0x7f3c9015a020 2023-10-06T10:45:51.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.030+0000 7f3c95e37700 1 --2- 172.21.15.92:0/187445519 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c74066f30 0x7f3c740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:51.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.030+0000 7f3c95e37700 1 --2- 172.21.15.92:0/187445519 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c74066f30 0x7f3c740693e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c000ef0 tx=0x7f3c8c015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:51.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.034+0000 7f3c85ffb700 1 -- 172.21.15.92:0/187445519 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c900c5570 con 0x7f3c9015a020 2023-10-06T10:45:51.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.218+0000 7f3c9809b700 1 -- 172.21.15.92:0/187445519 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c900c5570 con 0x7f3c9015a020 2023-10-06T10:45:51.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.219+0000 7f3c85ffb700 1 -- 172.21.15.92:0/187445519 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+17181 (secure 0 0 0) 0x7f3c900c5570 con 0x7f3c9015a020 2023-10-06T10:45:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:51.221 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":104,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:45:07.376972+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-06T10:44:48.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-06T10:44:50.072416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.4","mappings":[{"from":0,"to":5}]},{"pgid":"17.5","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:45:51.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.221+0000 7f3c9809b700 1 -- 172.21.15.92:0/187445519 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c74066f30 msgr2=0x7f3c740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:51.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.221+0000 7f3c9809b700 1 --2- 172.21.15.92:0/187445519 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c74066f30 0x7f3c740693e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c000ef0 tx=0x7f3c8c015000 comp rx=0 tx=0).stop 2023-10-06T10:45:51.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.221+0000 7f3c9809b700 1 -- 172.21.15.92:0/187445519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c9015a020 msgr2=0x7f3c900ce200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:51.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.221+0000 7f3c9809b700 1 --2- 172.21.15.92:0/187445519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c9015a020 0x7f3c900ce200 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f3c9015d1c0 tx=0x7f3c8800b040 comp rx=0 tx=0).stop 2023-10-06T10:45:51.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.221+0000 7f3c9809b700 1 -- 172.21.15.92:0/187445519 shutdown_connections 2023-10-06T10:45:51.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.221+0000 7f3c9809b700 1 --2- 172.21.15.92:0/187445519 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3c74066f30 0x7f3c740693e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.221+0000 7f3c9809b700 1 --2- 172.21.15.92:0/187445519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c9015a020 0x7f3c900ce200 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.221+0000 7f3c9809b700 1 --2- 172.21.15.92:0/187445519 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c7c003a60 0x7f3c90153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.221+0000 7f3c9809b700 1 --2- 172.21.15.92:0/187445519 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c9015c050 0x7f3c9016ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.221+0000 7f3c9809b700 1 -- 172.21.15.92:0/187445519 >> 172.21.15.92:0/187445519 conn(0x7f3c9006ece0 msgr2=0x7f3c900b4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:51.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.221+0000 7f3c9809b700 1 -- 172.21.15.92:0/187445519 shutdown_connections 2023-10-06T10:45:51.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.221+0000 7f3c9809b700 1 -- 172.21.15.92:0/187445519 wait complete. 2023-10-06T10:45:51.236 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T10:45:51.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.349+0000 7fb83eb96700 1 Processor -- start 2023-10-06T10:45:51.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.349+0000 7fb83eb96700 1 -- start start 2023-10-06T10:45:51.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.350+0000 7fb83eb96700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb838159560 0x7fb8380aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:51.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.350+0000 7fb83eb96700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb8380b49a0 0x7fb8380aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:51.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.350+0000 7fb83eb96700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8380b0760 0x7fb8380b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:51.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.350+0000 7fb83eb96700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8380b0c60 con 0x7fb838159560 2023-10-06T10:45:51.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.350+0000 7fb83eb96700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8380b0f40 con 0x7fb8380b49a0 2023-10-06T10:45:51.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.350+0000 7fb83eb96700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8380b1220 con 0x7fb8380b0760 2023-10-06T10:45:51.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.350+0000 7fb83c932700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb838159560 0x7fb8380aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:51.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.350+0000 7fb83c932700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb838159560 0x7fb8380aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47394/0 (socket says 172.21.15.92:47394) 2023-10-06T10:45:51.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.350+0000 7fb83c932700 1 -- 172.21.15.92:0/3370983980 learned_addr learned my addr 172.21.15.92:0/3370983980 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:51.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.350+0000 7fb82ffff700 1 --2- 172.21.15.92:0/3370983980 >> v2:172.21.15.124:3300/0 conn(0x7fb8380b49a0 0x7fb8380aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:51.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.350+0000 7fb83d133700 1 --2- 172.21.15.92:0/3370983980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8380b0760 0x7fb8380b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:51.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.351+0000 7fb83c932700 1 -- 172.21.15.92:0/3370983980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8380b0760 msgr2=0x7fb8380b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:51.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.351+0000 7fb83c932700 1 --2- 172.21.15.92:0/3370983980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8380b0760 0x7fb8380b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.351+0000 7fb83c932700 1 -- 172.21.15.92:0/3370983980 >> v2:172.21.15.124:3300/0 conn(0x7fb8380b49a0 msgr2=0x7fb8380aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:51.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.351+0000 7fb83c932700 1 --2- 172.21.15.92:0/3370983980 >> v2:172.21.15.124:3300/0 conn(0x7fb8380b49a0 0x7fb8380aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.351+0000 7fb83c932700 1 -- 172.21.15.92:0/3370983980 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8380b1500 con 0x7fb838159560 2023-10-06T10:45:51.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.351+0000 7fb83c932700 1 --2- 172.21.15.92:0/3370983980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb838159560 0x7fb8380aa530 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fb834009f70 tx=0x7fb834009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5c723ebf316029 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.351+0000 7fb82f7fe700 1 -- 172.21.15.92:0/3370983980 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83400e070 con 0x7fb838159560 2023-10-06T10:45:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.351+0000 7fb82f7fe700 1 -- 172.21.15.92:0/3370983980 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83400ba90 con 0x7fb838159560 2023-10-06T10:45:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.351+0000 7fb82f7fe700 1 -- 172.21.15.92:0/3370983980 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8340092e0 con 0x7fb838159560 2023-10-06T10:45:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.351+0000 7fb83eb96700 1 -- 172.21.15.92:0/3370983980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb838159560 msgr2=0x7fb8380aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.351+0000 7fb83eb96700 1 --2- 172.21.15.92:0/3370983980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb838159560 0x7fb8380aa530 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fb834009f70 tx=0x7fb834009840 comp rx=0 tx=0).stop 2023-10-06T10:45:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.351+0000 7fb83eb96700 1 -- 172.21.15.92:0/3370983980 shutdown_connections 2023-10-06T10:45:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.351+0000 7fb83eb96700 1 --2- 172.21.15.92:0/3370983980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb838159560 0x7fb8380aa530 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.351+0000 7fb83eb96700 1 --2- 172.21.15.92:0/3370983980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8380b0760 0x7fb8380b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.351+0000 7fb83eb96700 1 --2- 172.21.15.92:0/3370983980 >> v2:172.21.15.124:3300/0 conn(0x7fb8380b49a0 0x7fb8380aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.351+0000 7fb83eb96700 1 -- 172.21.15.92:0/3370983980 >> 172.21.15.92:0/3370983980 conn(0x7fb8380997f0 msgr2=0x7fb83806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:51.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.351+0000 7fb83eb96700 1 -- 172.21.15.92:0/3370983980 shutdown_connections 2023-10-06T10:45:51.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.351+0000 7fb83eb96700 1 -- 172.21.15.92:0/3370983980 wait complete. 2023-10-06T10:45:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.352+0000 7fb83eb96700 1 Processor -- start 2023-10-06T10:45:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.353+0000 7fb83eb96700 1 -- start start 2023-10-06T10:45:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.353+0000 7fb83eb96700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8380b0760 0x7fb83815c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.353+0000 7fb83eb96700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8380b49a0 0x7fb838146b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.353+0000 7fb83eb96700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb838159560 0x7fb83815b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.353+0000 7fb83eb96700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8380af9e0 con 0x7fb838159560 2023-10-06T10:45:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.353+0000 7fb83eb96700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8381442c0 con 0x7fb8380b0760 2023-10-06T10:45:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.353+0000 7fb83eb96700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb83809a9e0 con 0x7fb8380b49a0 2023-10-06T10:45:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.353+0000 7fb83d133700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb838159560 0x7fb83815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.354+0000 7fb83d133700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb838159560 0x7fb83815b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47402/0 (socket says 172.21.15.92:47402) 2023-10-06T10:45:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.354+0000 7fb83d133700 1 -- 172.21.15.92:0/2743094396 learned_addr learned my addr 172.21.15.92:0/2743094396 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.354+0000 7fb83c932700 1 --2- 172.21.15.92:0/2743094396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8380b0760 0x7fb83815c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.354+0000 7fb82ffff700 1 --2- 172.21.15.92:0/2743094396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8380b49a0 0x7fb838146b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.354+0000 7fb83d133700 1 -- 172.21.15.92:0/2743094396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8380b49a0 msgr2=0x7fb838146b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.354+0000 7fb83d133700 1 --2- 172.21.15.92:0/2743094396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8380b49a0 0x7fb838146b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.354+0000 7fb83d133700 1 -- 172.21.15.92:0/2743094396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8380b0760 msgr2=0x7fb83815c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.354+0000 7fb83d133700 1 --2- 172.21.15.92:0/2743094396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8380b0760 0x7fb83815c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.354+0000 7fb83d133700 1 -- 172.21.15.92:0/2743094396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb834009ae0 con 0x7fb838159560 2023-10-06T10:45:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.354+0000 7fb83d133700 1 --2- 172.21.15.92:0/2743094396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb838159560 0x7fb83815b810 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fb83000d890 tx=0x7fb830008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:51.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.354+0000 7fb82cff9700 1 -- 172.21.15.92:0/2743094396 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8300125f0 con 0x7fb838159560 2023-10-06T10:45:51.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.354+0000 7fb82cff9700 1 -- 172.21.15.92:0/2743094396 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb830012750 con 0x7fb838159560 2023-10-06T10:45:51.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.354+0000 7fb83eb96700 1 -- 172.21.15.92:0/2743094396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb83809a3d0 con 0x7fb838159560 2023-10-06T10:45:51.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.354+0000 7fb82cff9700 1 -- 172.21.15.92:0/2743094396 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb830012b10 con 0x7fb838159560 2023-10-06T10:45:51.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.354+0000 7fb83eb96700 1 -- 172.21.15.92:0/2743094396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb83815b110 con 0x7fb838159560 2023-10-06T10:45:51.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.355+0000 7fb83eb96700 1 -- 172.21.15.92:0/2743094396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb838064a90 con 0x7fb838159560 2023-10-06T10:45:51.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.355+0000 7fb82cff9700 1 -- 172.21.15.92:0/2743094396 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb830013480 con 0x7fb838159560 2023-10-06T10:45:51.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.356+0000 7fb82cff9700 1 --2- 172.21.15.92:0/2743094396 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb824066f30 0x7fb8240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:51.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.356+0000 7fb82cff9700 1 -- 172.21.15.92:0/2743094396 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6436+0+0 (secure 0 0 0) 0x7fb830084a80 con 0x7fb838159560 2023-10-06T10:45:51.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.356+0000 7fb83c932700 1 --2- 172.21.15.92:0/2743094396 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb824066f30 0x7fb8240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:51.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.356+0000 7fb83c932700 1 --2- 172.21.15.92:0/2743094396 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb824066f30 0x7fb8240693e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb834009880 tx=0x7fb834018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:51.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.360+0000 7fb82cff9700 1 -- 172.21.15.92:0/2743094396 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb830051a10 con 0x7fb838159560 2023-10-06T10:45:51.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.588+0000 7fb83eb96700 1 -- 172.21.15.92:0/2743094396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb8380b17e0 con 0x7fb838159560 2023-10-06T10:45:51.590 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T10:45:51.589+0000 7f9e7f3ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T10:45:51.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.598+0000 7fb82cff9700 1 -- 172.21.15.92:0/2743094396 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v111) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb830055030 con 0x7fb838159560 2023-10-06T10:45:51.602 INFO:tasks.ceph.mds.d.smithi124.stderr: -1134> 2023-10-06T10:45:26.598+0000 7f8a4bed5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2023-10-06T10:45:07.837939+0000 1=0+1) n(v0 rc2023-10-06T10:45:07.837939+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={7633=pAsLsXsFsx/pAsLsXsFsx@4},l=7633 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55ecb6337080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(16)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:45:07.837939+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:45:07.837939+0000 2=0+2)","memory_value.dirstat":"f(v0 m2023-10-06T10:45:07.837939+0000 1=0+1)","memory_value.rstat":"n(v0 rc2023-10-06T10:45:07.837939+0000 2=0+2)","error_str":""},"return_code":-61} 2023-10-06T10:45:51.602 INFO:tasks.ceph.mds.d.smithi124.stderr: -1108> 2023-10-06T10:45:26.599+0000 7f8a4bed5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={7633=pAsLsXsFsx/AsLsXsFsx@2},l=7633 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55ecb6337600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(16)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2023-10-06T10:45:51.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.600+0000 7fb83eb96700 1 -- 172.21.15.92:0/2743094396 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb824066f30 msgr2=0x7fb8240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:51.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.600+0000 7fb83eb96700 1 --2- 172.21.15.92:0/2743094396 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb824066f30 0x7fb8240693e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb834009880 tx=0x7fb834018000 comp rx=0 tx=0).stop 2023-10-06T10:45:51.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.600+0000 7fb83eb96700 1 -- 172.21.15.92:0/2743094396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb838159560 msgr2=0x7fb83815b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:51.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.600+0000 7fb83eb96700 1 --2- 172.21.15.92:0/2743094396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb838159560 0x7fb83815b810 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fb83000d890 tx=0x7fb830008040 comp rx=0 tx=0).stop 2023-10-06T10:45:51.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.601+0000 7fb83eb96700 1 -- 172.21.15.92:0/2743094396 shutdown_connections 2023-10-06T10:45:51.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.601+0000 7fb83eb96700 1 --2- 172.21.15.92:0/2743094396 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fb824066f30 0x7fb8240693e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.601+0000 7fb83eb96700 1 --2- 172.21.15.92:0/2743094396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb838159560 0x7fb83815b810 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.601+0000 7fb83eb96700 1 --2- 172.21.15.92:0/2743094396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8380b0760 0x7fb83815c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.601+0000 7fb83eb96700 1 --2- 172.21.15.92:0/2743094396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8380b49a0 0x7fb838146b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.601+0000 7fb83eb96700 1 -- 172.21.15.92:0/2743094396 >> 172.21.15.92:0/2743094396 conn(0x7fb8380997f0 msgr2=0x7fb8380697f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:51.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.601+0000 7fb83eb96700 1 -- 172.21.15.92:0/2743094396 shutdown_connections 2023-10-06T10:45:51.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.601+0000 7fb83eb96700 1 -- 172.21.15.92:0/2743094396 wait complete. 2023-10-06T10:45:51.605 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T10:45:51.605 INFO:tasks.ceph.mds.a.smithi092.stderr: -1218> 2023-10-06T10:45:26.597+0000 7fa288f3b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2023-10-06T10:45:07.836939+0000 1=0+1) n(v2 rc2023-10-06T10:45:08.064932+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={7633=pAsLsXsFsx/pAsLsXsFsx@4},l=7633 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x56246ef3e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(16)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:45:07.836939+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:45:08.064932+0000 4=0+4)","memory_value.dirstat":"f(v0 m2023-10-06T10:45:07.836939+0000 1=0+1)","memory_value.rstat":"n(v2 rc2023-10-06T10:45:08.064932+0000 4=0+4)","error_str":""},"return_code":-61} 2023-10-06T10:45:51.605 INFO:tasks.ceph.mds.a.smithi092.stderr: -1096> 2023-10-06T10:45:26.598+0000 7fa28ef47700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2023-10-06T10:45:07.837939+0000 1=0+1) n(v1 rc2023-10-06T10:45:08.064932+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={7633=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x56246ef3e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(16)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:45:07.837939+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:45:07.837939+0000 3=0+3)","memory_value.dirstat":"f(v0 m2023-10-06T10:45:07.837939+0000 1=0+1)","memory_value.rstat":"n(v1 rc2023-10-06T10:45:08.064932+0000 3=0+3)","error_str":""},"return_code":-61} 2023-10-06T10:45:51.608 INFO:tasks.ceph.mds.c.smithi092.stderr: -2790> 2023-10-06T10:45:26.597+0000 7f0bb5880700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2023-10-06T10:45:07.835939+0000 1=0+1) n(v3 rc2023-10-06T10:45:08.064932+0000 6=0+6) (inest mix dirty) (ifile excl) (iversion lock) caps={7633=pAsLsXsFsx/pAsLsXsFsx@5},l=7633 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x5632d5309700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(16)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:45:07.835939+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:45:08.064932+0000 6=0+6)","memory_value.dirstat":"f(v0 m2023-10-06T10:45:07.835939+0000 1=0+1)","memory_value.rstat":"n(v3 rc2023-10-06T10:45:08.064932+0000 6=0+6)","error_str":""},"return_code":-61} 2023-10-06T10:45:51.608 INFO:tasks.ceph.mds.c.smithi092.stderr: -2707> 2023-10-06T10:45:26.598+0000 7f0bbb88c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2023-10-06T10:45:07.836939+0000 1=0+1) n(v4 rc2023-10-06T10:45:08.064932+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={7633=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x5632d5386000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(16)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T10:45:07.836939+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-10-06T10:45:08.064932+0000 5=0+5)","memory_value.dirstat":"f(v0 m2023-10-06T10:45:07.836939+0000 1=0+1)","memory_value.rstat":"n(v4 rc2023-10-06T10:45:08.064932+0000 5=0+5)","error_str":""},"return_code":-61} 2023-10-06T10:45:51.617 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T10:45:51.624 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-06T10:45:51.627 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T10:45:51.632 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T10:45:51.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.734+0000 7f572c77b700 1 Processor -- start 2023-10-06T10:45:51.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.734+0000 7f572c77b700 1 -- start start 2023-10-06T10:45:51.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.735+0000 7f572c77b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5724150940 0x7f5724159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:51.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.735+0000 7f572c77b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f57240cd4d0 0x7f57240b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:51.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.735+0000 7f572c77b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f572415ac30 con 0x7f572415a160 2023-10-06T10:45:51.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.735+0000 7f572c77b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57240cb6a0 con 0x7f57240cd4d0 2023-10-06T10:45:51.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.735+0000 7f572c77b700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57240cb090 con 0x7f5724150940 2023-10-06T10:45:51.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.735+0000 7f572ad18700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f57240cd4d0 0x7f57240b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:51.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.736+0000 7f5729d16700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f572415a160 0x7f57240ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58842/0 (socket says 172.21.15.92:58842) 2023-10-06T10:45:51.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.736+0000 7f5729d16700 1 -- 172.21.15.92:0/2949019026 learned_addr learned my addr 172.21.15.92:0/2949019026 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:51.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.736+0000 7f572a517700 1 --2- 172.21.15.92:0/2949019026 >> v2:172.21.15.124:3300/0 conn(0x7f5724150940 0x7f5724159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.736+0000 7f572ad18700 1 -- 172.21.15.92:0/2949019026 >> v2:172.21.15.124:3300/0 conn(0x7f5724150940 msgr2=0x7f5724159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.736+0000 7f572ad18700 1 --2- 172.21.15.92:0/2949019026 >> v2:172.21.15.124:3300/0 conn(0x7f5724150940 0x7f5724159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.736+0000 7f572ad18700 1 -- 172.21.15.92:0/2949019026 >> v1:172.21.15.92:6789/0 conn(0x7f572415a160 legacy=0x7f57240ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.736+0000 7f572ad18700 1 -- 172.21.15.92:0/2949019026 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f572415a670 con 0x7f57240cd4d0 2023-10-06T10:45:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.736+0000 7f5729515700 1 -- 172.21.15.92:0/2949019026 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155251905 0 0) 0x7f5714001110 con 0x7f572415a160 2023-10-06T10:45:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.736+0000 7f572a517700 1 --2- 172.21.15.92:0/2949019026 >> v2:172.21.15.124:3300/0 conn(0x7f5724150940 0x7f5724159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.736+0000 7f572ad18700 1 --2- 172.21.15.92:0/2949019026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57240cd4d0 0x7f57240b2050 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f572408ab60 tx=0x7f571c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cc107003677001 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.736+0000 7f5729515700 1 -- 172.21.15.92:0/2949019026 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c00e070 con 0x7f57240cd4d0 2023-10-06T10:45:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.736+0000 7f5729515700 1 -- 172.21.15.92:0/2949019026 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571c00bc30 con 0x7f57240cd4d0 2023-10-06T10:45:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.736+0000 7f5729515700 1 -- 172.21.15.92:0/2949019026 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c012430 con 0x7f57240cd4d0 2023-10-06T10:45:51.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.737+0000 7f572c77b700 1 -- 172.21.15.92:0/2949019026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57240cd4d0 msgr2=0x7f57240b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.737+0000 7f572c77b700 1 --2- 172.21.15.92:0/2949019026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57240cd4d0 0x7f57240b2050 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f572408ab60 tx=0x7f571c009780 comp rx=0 tx=0).stop 2023-10-06T10:45:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.737+0000 7f572c77b700 1 -- 172.21.15.92:0/2949019026 shutdown_connections 2023-10-06T10:45:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.737+0000 7f572c77b700 1 --2- 172.21.15.92:0/2949019026 >> v2:172.21.15.124:3300/0 conn(0x7f5724150940 0x7f5724159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.737+0000 7f572c77b700 1 --2- 172.21.15.92:0/2949019026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57240cd4d0 0x7f57240b2050 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.737+0000 7f572c77b700 1 -- 172.21.15.92:0/2949019026 >> 172.21.15.92:0/2949019026 conn(0x7f5724099700 msgr2=0x7f572406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.737+0000 7f572c77b700 1 -- 172.21.15.92:0/2949019026 shutdown_connections 2023-10-06T10:45:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.737+0000 7f572c77b700 1 -- 172.21.15.92:0/2949019026 wait complete. 2023-10-06T10:45:51.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.738+0000 7f572c77b700 1 Processor -- start 2023-10-06T10:45:51.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.738+0000 7f572c77b700 1 -- start start 2023-10-06T10:45:51.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.738+0000 7f572c77b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57240cd4d0 0x7f572414dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:51.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.739+0000 7f572c77b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f572415a160 0x7f5724156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:51.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.739+0000 7f572c77b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57240b4170 0x7f5724156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:51.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.739+0000 7f572c77b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57241571e0 con 0x7f572415a160 2023-10-06T10:45:51.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.739+0000 7f572c77b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57241546a0 con 0x7f57240cd4d0 2023-10-06T10:45:51.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.739+0000 7f572c77b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57240b4ba0 con 0x7f57240b4170 2023-10-06T10:45:51.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.739+0000 7f5729d16700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f572415a160 0x7f5724156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:51.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.739+0000 7f5729d16700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f572415a160 0x7f5724156760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47454/0 (socket says 172.21.15.92:47454) 2023-10-06T10:45:51.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.739+0000 7f5729d16700 1 -- 172.21.15.92:0/23140898 learned_addr learned my addr 172.21.15.92:0/23140898 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:51.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.739+0000 7f572a517700 1 --2- 172.21.15.92:0/23140898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57240cd4d0 0x7f572414dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:51.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.739+0000 7f572ad18700 1 --2- 172.21.15.92:0/23140898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57240b4170 0x7f5724156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:51.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.739+0000 7f5729d16700 1 -- 172.21.15.92:0/23140898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57240b4170 msgr2=0x7f5724156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:51.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.739+0000 7f5729d16700 1 --2- 172.21.15.92:0/23140898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57240b4170 0x7f5724156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.739+0000 7f5729d16700 1 -- 172.21.15.92:0/23140898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57240cd4d0 msgr2=0x7f572414dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:51.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.739+0000 7f5729d16700 1 --2- 172.21.15.92:0/23140898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57240cd4d0 0x7f572414dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.739+0000 7f5729d16700 1 -- 172.21.15.92:0/23140898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c009a20 con 0x7f572415a160 2023-10-06T10:45:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.739+0000 7f5729d16700 1 --2- 172.21.15.92:0/23140898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f572415a160 0x7f5724156760 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f571400c5e0 tx=0x7f5714000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.739+0000 7f571a7fc700 1 -- 172.21.15.92:0/23140898 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5714005b30 con 0x7f572415a160 2023-10-06T10:45:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.739+0000 7f571a7fc700 1 -- 172.21.15.92:0/23140898 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5714005c90 con 0x7f572415a160 2023-10-06T10:45:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.739+0000 7f572c77b700 1 -- 172.21.15.92:0/23140898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57240b45e0 con 0x7f572415a160 2023-10-06T10:45:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.739+0000 7f571a7fc700 1 -- 172.21.15.92:0/23140898 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5714007300 con 0x7f572415a160 2023-10-06T10:45:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.739+0000 7f572c77b700 1 -- 172.21.15.92:0/23140898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5724154330 con 0x7f572415a160 2023-10-06T10:45:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.740+0000 7f572c77b700 1 -- 172.21.15.92:0/23140898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5724064a90 con 0x7f572415a160 2023-10-06T10:45:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.741+0000 7f571a7fc700 1 -- 172.21.15.92:0/23140898 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f571401b3b0 con 0x7f572415a160 2023-10-06T10:45:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.741+0000 7f571a7fc700 1 --2- 172.21.15.92:0/23140898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5708066f30 0x7f57080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.741+0000 7f571a7fc700 1 -- 172.21.15.92:0/23140898 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f5714083b90 con 0x7f572415a160 2023-10-06T10:45:51.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.741+0000 7f572a517700 1 --2- 172.21.15.92:0/23140898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5708066f30 0x7f57080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:51.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.741+0000 7f572a517700 1 --2- 172.21.15.92:0/23140898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5708066f30 0x7f57080693e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f572000ad40 tx=0x7f5720008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:51.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.745+0000 7f571a7fc700 1 -- 172.21.15.92:0/23140898 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5714050a20 con 0x7f572415a160 2023-10-06T10:45:51.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:51.974+0000 7f572c77b700 1 -- 172.21.15.92:0/23140898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f572415ac30 con 0x7f572415a160 2023-10-06T10:45:52.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.597+0000 7f571a7fc700 1 -- 172.21.15.92:0/23140898 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v112) v1 ==== 104+0+0 (secure 0 0 0) 0x7f572415ac30 con 0x7f572415a160 2023-10-06T10:45:52.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.599+0000 7f572c77b700 1 -- 172.21.15.92:0/23140898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5708066f30 msgr2=0x7f57080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:52.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.599+0000 7f572c77b700 1 --2- 172.21.15.92:0/23140898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5708066f30 0x7f57080693e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f572000ad40 tx=0x7f5720008040 comp rx=0 tx=0).stop 2023-10-06T10:45:52.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.599+0000 7f572c77b700 1 -- 172.21.15.92:0/23140898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f572415a160 msgr2=0x7f5724156760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:52.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.599+0000 7f572c77b700 1 --2- 172.21.15.92:0/23140898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f572415a160 0x7f5724156760 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f571400c5e0 tx=0x7f5714000fe0 comp rx=0 tx=0).stop 2023-10-06T10:45:52.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.601+0000 7f572c77b700 1 -- 172.21.15.92:0/23140898 shutdown_connections 2023-10-06T10:45:52.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.601+0000 7f572c77b700 1 --2- 172.21.15.92:0/23140898 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5708066f30 0x7f57080693e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:52.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.601+0000 7f572c77b700 1 --2- 172.21.15.92:0/23140898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57240b4170 0x7f5724156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:52.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.601+0000 7f572c77b700 1 --2- 172.21.15.92:0/23140898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57240cd4d0 0x7f572414dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:52.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.601+0000 7f572c77b700 1 --2- 172.21.15.92:0/23140898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f572415a160 0x7f5724156760 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:52.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.601+0000 7f572c77b700 1 -- 172.21.15.92:0/23140898 >> 172.21.15.92:0/23140898 conn(0x7f5724099700 msgr2=0x7f57240cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:52.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.601+0000 7f572c77b700 1 -- 172.21.15.92:0/23140898 shutdown_connections 2023-10-06T10:45:52.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.601+0000 7f572c77b700 1 -- 172.21.15.92:0/23140898 wait complete. 2023-10-06T10:45:52.615 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T10:45:52.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.773+0000 7f7b2afc4700 1 Processor -- start 2023-10-06T10:45:52.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.773+0000 7f7b2afc4700 1 -- start start 2023-10-06T10:45:52.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.773+0000 7f7b2afc4700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7b240b2360 0x7f7b240b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:52.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.773+0000 7f7b2afc4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b240aede0 0x7f7b240af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:52.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.773+0000 7f7b2afc4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b240b2ce0 con 0x7f7b2406ece0 2023-10-06T10:45:52.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.773+0000 7f7b2afc4700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b2406fb40 con 0x7f7b240b2360 2023-10-06T10:45:52.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.773+0000 7f7b2afc4700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b240b2800 con 0x7f7b240aede0 2023-10-06T10:45:52.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b28d60700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7b2406ece0 0x7f7b2406f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58848/0 (socket says 172.21.15.92:58848) 2023-10-06T10:45:52.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b28d60700 1 -- 172.21.15.92:0/2025847078 learned_addr learned my addr 172.21.15.92:0/2025847078 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:52.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b1bfff700 1 --2- 172.21.15.92:0/2025847078 >> v2:172.21.15.124:3300/0 conn(0x7f7b240b2360 0x7f7b240b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:52.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b29561700 1 --2- 172.21.15.92:0/2025847078 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b240aede0 0x7f7b240af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:52.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b1b7fe700 1 -- 172.21.15.92:0/2025847078 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2501288358 0 0) 0x7f7b240b2ce0 con 0x7f7b2406ece0 2023-10-06T10:45:52.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b1b7fe700 1 -- 172.21.15.92:0/2025847078 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b0c003570 con 0x7f7b2406ece0 2023-10-06T10:45:52.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b1b7fe700 1 -- 172.21.15.92:0/2025847078 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2216831379 0 0) 0x7f7b0c003570 con 0x7f7b2406ece0 2023-10-06T10:45:52.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b1b7fe700 1 -- 172.21.15.92:0/2025847078 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b240aede0 msgr2=0x7f7b240af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:52.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b1b7fe700 1 --2- 172.21.15.92:0/2025847078 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b240aede0 0x7f7b240af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:52.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b1b7fe700 1 -- 172.21.15.92:0/2025847078 >> v2:172.21.15.124:3300/0 conn(0x7f7b240b2360 msgr2=0x7f7b240b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:52.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b1b7fe700 1 --2- 172.21.15.92:0/2025847078 >> v2:172.21.15.124:3300/0 conn(0x7f7b240b2360 0x7f7b240b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:52.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b1b7fe700 1 -- 172.21.15.92:0/2025847078 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b240b0410 con 0x7f7b2406ece0 2023-10-06T10:45:52.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b1b7fe700 1 -- 172.21.15.92:0/2025847078 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f7b20001470 con 0x7f7b2406ece0 2023-10-06T10:45:52.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b1b7fe700 1 -- 172.21.15.92:0/2025847078 >> v1:172.21.15.92:6789/0 conn(0x7f7b2406ece0 legacy=0x7f7b2406f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:52.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b1b7fe700 1 --2- 172.21.15.92:0/2025847078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c003a90 0x7f7b0c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:52.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b1b7fe700 1 --2- 172.21.15.92:0/2025847078 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b240aede0 0x7f7b0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:52.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b1b7fe700 1 --2- 172.21.15.92:0/2025847078 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b0c004d50 0x7f7b0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:52.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b1b7fe700 1 -- 172.21.15.92:0/2025847078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b240b2800 con 0x7f7b0c003a90 2023-10-06T10:45:52.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b1b7fe700 1 -- 172.21.15.92:0/2025847078 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c00ba60 con 0x7f7b0c004d50 2023-10-06T10:45:52.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b1b7fe700 1 -- 172.21.15.92:0/2025847078 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c00bba0 con 0x7f7b240aede0 2023-10-06T10:45:52.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b28d60700 1 --2- 172.21.15.92:0/2025847078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c003a90 0x7f7b0c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:52.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b1bfff700 1 --2- 172.21.15.92:0/2025847078 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b240aede0 0x7f7b0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:52.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.774+0000 7f7b29561700 1 --2- 172.21.15.92:0/2025847078 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b0c004d50 0x7f7b0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:52.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.775+0000 7f7b28d60700 1 -- 172.21.15.92:0/2025847078 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b240aede0 msgr2=0x7f7b0c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:52.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.775+0000 7f7b28d60700 1 --2- 172.21.15.92:0/2025847078 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b240aede0 0x7f7b0c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:52.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.775+0000 7f7b28d60700 1 -- 172.21.15.92:0/2025847078 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b0c004d50 msgr2=0x7f7b0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:52.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.775+0000 7f7b28d60700 1 --2- 172.21.15.92:0/2025847078 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b0c004d50 0x7f7b0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:52.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.775+0000 7f7b28d60700 1 -- 172.21.15.92:0/2025847078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c003570 con 0x7f7b0c003a90 2023-10-06T10:45:52.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.775+0000 7f7b28d60700 1 --2- 172.21.15.92:0/2025847078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c003a90 0x7f7b0c003ea0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000c3c0 tx=0x7f7b20001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5cce53f06c7547 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:52.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.775+0000 7f7b1b7fe700 1 -- 172.21.15.92:0/2025847078 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2000cc30 con 0x7f7b0c003a90 2023-10-06T10:45:52.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.775+0000 7f7b1b7fe700 1 -- 172.21.15.92:0/2025847078 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2000cd90 con 0x7f7b0c003a90 2023-10-06T10:45:52.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.775+0000 7f7b1b7fe700 1 -- 172.21.15.92:0/2025847078 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20005b50 con 0x7f7b0c003a90 2023-10-06T10:45:52.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.775+0000 7f7b2afc4700 1 -- 172.21.15.92:0/2025847078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c003a90 msgr2=0x7f7b0c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:52.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.775+0000 7f7b2afc4700 1 --2- 172.21.15.92:0/2025847078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c003a90 0x7f7b0c003ea0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000c3c0 tx=0x7f7b20001e50 comp rx=0 tx=0).stop 2023-10-06T10:45:52.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.775+0000 7f7b29561700 1 -- 172.21.15.92:0/2025847078 reap_dead start 2023-10-06T10:45:52.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.775+0000 7f7b2afc4700 1 -- 172.21.15.92:0/2025847078 shutdown_connections 2023-10-06T10:45:52.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.775+0000 7f7b2afc4700 1 -- 172.21.15.92:0/2025847078 >> 172.21.15.92:0/2025847078 conn(0x7f7b240682f0 msgr2=0x7f7b240686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:52.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.775+0000 7f7b2afc4700 1 -- 172.21.15.92:0/2025847078 shutdown_connections 2023-10-06T10:45:52.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.775+0000 7f7b2afc4700 1 -- 172.21.15.92:0/2025847078 wait complete. 2023-10-06T10:45:52.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.776+0000 7f7b2afc4700 1 Processor -- start 2023-10-06T10:45:52.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.777+0000 7f7b2afc4700 1 -- start start 2023-10-06T10:45:52.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.777+0000 7f7b2afc4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c003a90 0x7f7b24156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:52.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.777+0000 7f7b2afc4700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b240ae2d0 0x7f7b240ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:52.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.777+0000 7f7b2afc4700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b240b40d0 0x7f7b240b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:52.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.777+0000 7f7b2afc4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b240ca620 con 0x7f7b0c003a90 2023-10-06T10:45:52.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.777+0000 7f7b2afc4700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b240b4a40 con 0x7f7b240ae2d0 2023-10-06T10:45:52.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.777+0000 7f7b2afc4700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b240c9a10 con 0x7f7b240b40d0 2023-10-06T10:45:52.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.777+0000 7f7b28d60700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c003a90 0x7f7b24156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:52.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.777+0000 7f7b28d60700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c003a90 0x7f7b24156860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47472/0 (socket says 172.21.15.92:47472) 2023-10-06T10:45:52.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.778+0000 7f7b28d60700 1 -- 172.21.15.92:0/1671597130 learned_addr learned my addr 172.21.15.92:0/1671597130 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:52.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.778+0000 7f7b1bfff700 1 --2- 172.21.15.92:0/1671597130 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b240ae2d0 0x7f7b240ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:52.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.778+0000 7f7b29561700 1 --2- 172.21.15.92:0/1671597130 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b240b40d0 0x7f7b240b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:52.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.778+0000 7f7b28d60700 1 -- 172.21.15.92:0/1671597130 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b240b40d0 msgr2=0x7f7b240b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:52.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.778+0000 7f7b28d60700 1 --2- 172.21.15.92:0/1671597130 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b240b40d0 0x7f7b240b4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:52.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.778+0000 7f7b28d60700 1 -- 172.21.15.92:0/1671597130 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b240ae2d0 msgr2=0x7f7b240ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:52.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.778+0000 7f7b28d60700 1 --2- 172.21.15.92:0/1671597130 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b240ae2d0 0x7f7b240ae740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:52.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.778+0000 7f7b28d60700 1 -- 172.21.15.92:0/1671597130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b240c9cf0 con 0x7f7b0c003a90 2023-10-06T10:45:52.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.778+0000 7f7b28d60700 1 --2- 172.21.15.92:0/1671597130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c003a90 0x7f7b24156860 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c004760 tx=0x7f7b20015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:52.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.778+0000 7f7b18ff9700 1 -- 172.21.15.92:0/1671597130 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2000cc30 con 0x7f7b0c003a90 2023-10-06T10:45:52.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.778+0000 7f7b18ff9700 1 -- 172.21.15.92:0/1671597130 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b200072e0 con 0x7f7b0c003a90 2023-10-06T10:45:52.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.778+0000 7f7b2afc4700 1 -- 172.21.15.92:0/1671597130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b240c9fd0 con 0x7f7b0c003a90 2023-10-06T10:45:52.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.778+0000 7f7b18ff9700 1 -- 172.21.15.92:0/1671597130 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20007680 con 0x7f7b0c003a90 2023-10-06T10:45:52.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.778+0000 7f7b2afc4700 1 -- 172.21.15.92:0/1671597130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b240c96c0 con 0x7f7b0c003a90 2023-10-06T10:45:52.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.779+0000 7f7b2afc4700 1 -- 172.21.15.92:0/1671597130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b24064a90 con 0x7f7b0c003a90 2023-10-06T10:45:52.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.779+0000 7f7b18ff9700 1 -- 172.21.15.92:0/1671597130 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7b20019030 con 0x7f7b0c003a90 2023-10-06T10:45:52.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.780+0000 7f7b18ff9700 1 --2- 172.21.15.92:0/1671597130 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7b10066f30 0x7f7b100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:52.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.780+0000 7f7b18ff9700 1 -- 172.21.15.92:0/1671597130 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f7b20085bb0 con 0x7f7b0c003a90 2023-10-06T10:45:52.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.780+0000 7f7b1bfff700 1 --2- 172.21.15.92:0/1671597130 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7b10066f30 0x7f7b100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:52.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.780+0000 7f7b1bfff700 1 --2- 172.21.15.92:0/1671597130 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7b10066f30 0x7f7b100693e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7b240b0e00 tx=0x7f7b140091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:52.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.784+0000 7f7b18ff9700 1 -- 172.21.15.92:0/1671597130 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b20052a40 con 0x7f7b0c003a90 2023-10-06T10:45:52.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:52.970+0000 7f7b2afc4700 1 -- 172.21.15.92:0/1671597130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7b240b2a00 con 0x7f7b0c003a90 2023-10-06T10:45:53.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.601+0000 7f7b18ff9700 1 -- 172.21.15.92:0/1671597130 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v106) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7b2002b0d0 con 0x7f7b0c003a90 2023-10-06T10:45:53.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.604+0000 7f7b2afc4700 1 -- 172.21.15.92:0/1671597130 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7b10066f30 msgr2=0x7f7b100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:53.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.604+0000 7f7b2afc4700 1 --2- 172.21.15.92:0/1671597130 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7b10066f30 0x7f7b100693e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7b240b0e00 tx=0x7f7b140091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:53.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.604+0000 7f7b2afc4700 1 -- 172.21.15.92:0/1671597130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c003a90 msgr2=0x7f7b24156860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:53.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.604+0000 7f7b2afc4700 1 --2- 172.21.15.92:0/1671597130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c003a90 0x7f7b24156860 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c004760 tx=0x7f7b20015000 comp rx=0 tx=0).stop 2023-10-06T10:45:53.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.604+0000 7f7b2afc4700 1 -- 172.21.15.92:0/1671597130 shutdown_connections 2023-10-06T10:45:53.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.604+0000 7f7b2afc4700 1 --2- 172.21.15.92:0/1671597130 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7b10066f30 0x7f7b100693e0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:53.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.604+0000 7f7b2afc4700 1 --2- 172.21.15.92:0/1671597130 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b240b40d0 0x7f7b240b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:53.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.604+0000 7f7b2afc4700 1 --2- 172.21.15.92:0/1671597130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c003a90 0x7f7b24156860 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:53.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.604+0000 7f7b2afc4700 1 --2- 172.21.15.92:0/1671597130 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b240ae2d0 0x7f7b240ae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:53.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.604+0000 7f7b2afc4700 1 -- 172.21.15.92:0/1671597130 >> 172.21.15.92:0/1671597130 conn(0x7f7b240682f0 msgr2=0x7f7b2415c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:53.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.605+0000 7f7b2afc4700 1 -- 172.21.15.92:0/1671597130 shutdown_connections 2023-10-06T10:45:53.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.605+0000 7f7b2afc4700 1 -- 172.21.15.92:0/1671597130 wait complete. 2023-10-06T10:45:53.608 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-10-06T10:45:53.620 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T10:45:53.645 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:45:53.665 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:45:53.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.742+0000 7f68e749b700 1 Processor -- start 2023-10-06T10:45:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.742+0000 7f68e749b700 1 -- start start 2023-10-06T10:45:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.743+0000 7f68e749b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f68e0153990 0x7f68e015a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.743+0000 7f68e749b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f68e00cca70 0x7f68e00d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.743+0000 7f68e749b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e00cdbf0 con 0x7f68e00d07f0 2023-10-06T10:45:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.743+0000 7f68e749b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e00cb4c0 con 0x7f68e00cca70 2023-10-06T10:45:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.743+0000 7f68e749b700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e015a8b0 con 0x7f68e0153990 2023-10-06T10:45:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.743+0000 7f68e5c98700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f68e00cca70 0x7f68e00d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.743+0000 7f68e6c9a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f68e00d07f0 0x7f68e015cf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35334/0 (socket says 172.21.15.92:35334) 2023-10-06T10:45:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.743+0000 7f68e5c98700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f68e00cca70 0x7f68e00d02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37858/0 (socket says 172.21.15.92:37858) 2023-10-06T10:45:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.743+0000 7f68e6c9a700 1 -- 172.21.15.92:0/2013186322 learned_addr learned my addr 172.21.15.92:0/2013186322 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.743+0000 7f68e6499700 1 --2- 172.21.15.92:0/2013186322 >> v2:172.21.15.124:3300/0 conn(0x7f68e0153990 0x7f68e015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.743+0000 7f68e5c98700 1 -- 172.21.15.92:0/2013186322 >> v2:172.21.15.124:3300/0 conn(0x7f68e0153990 msgr2=0x7f68e015a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.743+0000 7f68e5c98700 1 --2- 172.21.15.92:0/2013186322 >> v2:172.21.15.124:3300/0 conn(0x7f68e0153990 0x7f68e015a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.743+0000 7f68e5c98700 1 -- 172.21.15.92:0/2013186322 >> v1:172.21.15.92:6789/0 conn(0x7f68e00d07f0 legacy=0x7f68e015cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.743+0000 7f68e5c98700 1 -- 172.21.15.92:0/2013186322 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e00d0bf0 con 0x7f68e00cca70 2023-10-06T10:45:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.743+0000 7f68e5497700 1 -- 172.21.15.92:0/2013186322 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592192829 0 0) 0x7f68e00cdbf0 con 0x7f68e00d07f0 2023-10-06T10:45:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.743+0000 7f68e5c98700 1 --2- 172.21.15.92:0/2013186322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e00cca70 0x7f68e00d02b0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f68e00d0280 tx=0x7f68d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04ae36fa80b84ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.744+0000 7f68e5497700 1 -- 172.21.15.92:0/2013186322 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d000b6b0 con 0x7f68e00cca70 2023-10-06T10:45:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.744+0000 7f68e5497700 1 -- 172.21.15.92:0/2013186322 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d000b810 con 0x7f68e00cca70 2023-10-06T10:45:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.744+0000 7f68e5497700 1 -- 172.21.15.92:0/2013186322 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d000bbb0 con 0x7f68e00cca70 2023-10-06T10:45:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.744+0000 7f68e749b700 1 -- 172.21.15.92:0/2013186322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e00cca70 msgr2=0x7f68e00d02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.744+0000 7f68e749b700 1 --2- 172.21.15.92:0/2013186322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e00cca70 0x7f68e00d02b0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f68e00d0280 tx=0x7f68d00091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.744+0000 7f68e749b700 1 -- 172.21.15.92:0/2013186322 shutdown_connections 2023-10-06T10:45:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.744+0000 7f68e749b700 1 --2- 172.21.15.92:0/2013186322 >> v2:172.21.15.124:3300/0 conn(0x7f68e0153990 0x7f68e015a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.744+0000 7f68e749b700 1 --2- 172.21.15.92:0/2013186322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e00cca70 0x7f68e00d02b0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.744+0000 7f68e749b700 1 -- 172.21.15.92:0/2013186322 >> 172.21.15.92:0/2013186322 conn(0x7f68e00680c0 msgr2=0x7f68e0068490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:53.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.744+0000 7f68e749b700 1 -- 172.21.15.92:0/2013186322 shutdown_connections 2023-10-06T10:45:53.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.744+0000 7f68e749b700 1 -- 172.21.15.92:0/2013186322 wait complete. 2023-10-06T10:45:53.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.745+0000 7f68e749b700 1 Processor -- start 2023-10-06T10:45:53.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.745+0000 7f68e749b700 1 -- start start 2023-10-06T10:45:53.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.746+0000 7f68e749b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68e00cca70 0x7f68e00c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:53.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.746+0000 7f68e749b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e00d07f0 0x7f68e0156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:53.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.746+0000 7f68e749b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68e0153990 0x7f68e0150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:53.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.746+0000 7f68e749b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e01572e0 con 0x7f68e00d07f0 2023-10-06T10:45:53.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.746+0000 7f68e749b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e0169520 con 0x7f68e00cca70 2023-10-06T10:45:53.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.746+0000 7f68e749b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e01697c0 con 0x7f68e0153990 2023-10-06T10:45:53.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.746+0000 7f68e5c98700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e00d07f0 0x7f68e0156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:53.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.746+0000 7f68e5c98700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e00d07f0 0x7f68e0156da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37866/0 (socket says 172.21.15.92:37866) 2023-10-06T10:45:53.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.746+0000 7f68e5c98700 1 -- 172.21.15.92:0/4213863860 learned_addr learned my addr 172.21.15.92:0/4213863860 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:53.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.746+0000 7f68e6499700 1 --2- 172.21.15.92:0/4213863860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68e00cca70 0x7f68e00c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:53.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.746+0000 7f68e6c9a700 1 --2- 172.21.15.92:0/4213863860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68e0153990 0x7f68e0150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:53.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.746+0000 7f68e5c98700 1 -- 172.21.15.92:0/4213863860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68e0153990 msgr2=0x7f68e0150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:53.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.746+0000 7f68e5c98700 1 --2- 172.21.15.92:0/4213863860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68e0153990 0x7f68e0150c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:53.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.746+0000 7f68e5c98700 1 -- 172.21.15.92:0/4213863860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68e00cca70 msgr2=0x7f68e00c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:53.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.746+0000 7f68e5c98700 1 --2- 172.21.15.92:0/4213863860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68e00cca70 0x7f68e00c73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:53.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.746+0000 7f68e5c98700 1 -- 172.21.15.92:0/4213863860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d0009420 con 0x7f68e00d07f0 2023-10-06T10:45:53.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.746+0000 7f68e6c9a700 1 --2- 172.21.15.92:0/4213863860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68e0153990 0x7f68e0150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:53.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.746+0000 7f68e5c98700 1 --2- 172.21.15.92:0/4213863860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e00d07f0 0x7f68e0156da0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f68e00890b0 tx=0x7f68d0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:53.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.747+0000 7f68d67fc700 1 -- 172.21.15.92:0/4213863860 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d000bb90 con 0x7f68e00d07f0 2023-10-06T10:45:53.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.747+0000 7f68d67fc700 1 -- 172.21.15.92:0/4213863860 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d0011420 con 0x7f68e00d07f0 2023-10-06T10:45:53.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.747+0000 7f68e749b700 1 -- 172.21.15.92:0/4213863860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e00c9fa0 con 0x7f68e00d07f0 2023-10-06T10:45:53.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.747+0000 7f68d67fc700 1 -- 172.21.15.92:0/4213863860 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d00117e0 con 0x7f68e00d07f0 2023-10-06T10:45:53.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.747+0000 7f68e749b700 1 -- 172.21.15.92:0/4213863860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e01511d0 con 0x7f68e00d07f0 2023-10-06T10:45:53.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.747+0000 7f68e749b700 1 -- 172.21.15.92:0/4213863860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e0169930 con 0x7f68e00d07f0 2023-10-06T10:45:53.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.748+0000 7f68d67fc700 1 -- 172.21.15.92:0/4213863860 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f68d0011b90 con 0x7f68e00d07f0 2023-10-06T10:45:53.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.748+0000 7f68d67fc700 1 --2- 172.21.15.92:0/4213863860 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f68c4066f30 0x7f68c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:53.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.748+0000 7f68d67fc700 1 -- 172.21.15.92:0/4213863860 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6256+0+0 (secure 0 0 0) 0x7f68d0085c70 con 0x7f68e00d07f0 2023-10-06T10:45:53.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.748+0000 7f68e6499700 1 --2- 172.21.15.92:0/4213863860 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f68c4066f30 0x7f68c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:53.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.749+0000 7f68e6499700 1 --2- 172.21.15.92:0/4213863860 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f68c4066f30 0x7f68c40693e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009fd0 tx=0x7f68d8009880 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:53.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.753+0000 7f68d67fc700 1 -- 172.21.15.92:0/4213863860 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68d0052d40 con 0x7f68e00d07f0 2023-10-06T10:45:53.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:53.933+0000 7f68e749b700 1 -- 172.21.15.92:0/4213863860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f68e005f230 con 0x7f68e00d07f0 2023-10-06T10:45:54.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.607+0000 7f68d67fc700 1 -- 172.21.15.92:0/4213863860 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v107) v1 ==== 169+0+0 (secure 0 0 0) 0x7f68d0056360 con 0x7f68e00d07f0 2023-10-06T10:45:54.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.611+0000 7f68e749b700 1 -- 172.21.15.92:0/4213863860 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f68c4066f30 msgr2=0x7f68c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:54.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.611+0000 7f68e749b700 1 --2- 172.21.15.92:0/4213863860 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f68c4066f30 0x7f68c40693e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009fd0 tx=0x7f68d8009880 comp rx=0 tx=0).stop 2023-10-06T10:45:54.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.611+0000 7f68e749b700 1 -- 172.21.15.92:0/4213863860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e00d07f0 msgr2=0x7f68e0156da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:54.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.611+0000 7f68e749b700 1 --2- 172.21.15.92:0/4213863860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e00d07f0 0x7f68e0156da0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f68e00890b0 tx=0x7f68d0017000 comp rx=0 tx=0).stop 2023-10-06T10:45:54.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.611+0000 7f68e749b700 1 -- 172.21.15.92:0/4213863860 shutdown_connections 2023-10-06T10:45:54.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.611+0000 7f68e749b700 1 --2- 172.21.15.92:0/4213863860 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f68c4066f30 0x7f68c40693e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:54.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.611+0000 7f68e749b700 1 --2- 172.21.15.92:0/4213863860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68e0153990 0x7f68e0150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:54.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.611+0000 7f68e749b700 1 --2- 172.21.15.92:0/4213863860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68e00cca70 0x7f68e00c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:54.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.611+0000 7f68e749b700 1 --2- 172.21.15.92:0/4213863860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e00d07f0 0x7f68e0156da0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:54.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.611+0000 7f68e749b700 1 -- 172.21.15.92:0/4213863860 >> 172.21.15.92:0/4213863860 conn(0x7f68e00680c0 msgr2=0x7f68e00ce860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:54.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.611+0000 7f68e749b700 1 -- 172.21.15.92:0/4213863860 shutdown_connections 2023-10-06T10:45:54.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.611+0000 7f68e749b700 1 -- 172.21.15.92:0/4213863860 wait complete. 2023-10-06T10:45:54.615 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-10-06T10:45:54.626 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T10:45:54.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.741+0000 7f9135d19700 1 Processor -- start 2023-10-06T10:45:54.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.742+0000 7f9135d19700 1 -- start start 2023-10-06T10:45:54.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.742+0000 7f9135d19700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9130157520 0x7f91300afff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:54.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.742+0000 7f9135d19700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91300cd2c0 con 0x7f91300afc20 2023-10-06T10:45:54.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.742+0000 7f9135d19700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91300cc6b0 con 0x7f91300cdfc0 2023-10-06T10:45:54.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.742+0000 7f9135d19700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91300cc990 con 0x7f9130157520 2023-10-06T10:45:54.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.742+0000 7f9135518700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9130157520 0x7f91300afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:54.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.742+0000 7f9134d17700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f91300afc20 0x7f91300cbcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35344/0 (socket says 172.21.15.92:35344) 2023-10-06T10:45:54.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.743+0000 7f9134d17700 1 -- 172.21.15.92:0/1648014560 learned_addr learned my addr 172.21.15.92:0/1648014560 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:54.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.743+0000 7f9135518700 1 -- 172.21.15.92:0/1648014560 >> v1:172.21.15.124:6789/0 conn(0x7f91300cdfc0 legacy=0x7f9130156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:54.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.743+0000 7f9135518700 1 -- 172.21.15.92:0/1648014560 >> v1:172.21.15.92:6789/0 conn(0x7f91300afc20 legacy=0x7f91300cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:54.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.743+0000 7f9135518700 1 -- 172.21.15.92:0/1648014560 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91300ccc70 con 0x7f9130157520 2023-10-06T10:45:54.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.743+0000 7f9135518700 1 --2- 172.21.15.92:0/1648014560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9130157520 0x7f91300afff0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f9128009e60 tx=0x7f9128009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcf20fb07c15e53 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:54.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.743+0000 7f912f7fe700 1 -- 172.21.15.92:0/1648014560 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912800e070 con 0x7f9130157520 2023-10-06T10:45:54.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.743+0000 7f912f7fe700 1 -- 172.21.15.92:0/1648014560 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912800bbf0 con 0x7f9130157520 2023-10-06T10:45:54.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.743+0000 7f912f7fe700 1 -- 172.21.15.92:0/1648014560 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9128012430 con 0x7f9130157520 2023-10-06T10:45:54.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.744+0000 7f9135d19700 1 -- 172.21.15.92:0/1648014560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9130157520 msgr2=0x7f91300afff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:54.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.744+0000 7f9135d19700 1 --2- 172.21.15.92:0/1648014560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9130157520 0x7f91300afff0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f9128009e60 tx=0x7f9128009780 comp rx=0 tx=0).stop 2023-10-06T10:45:54.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.744+0000 7f9135d19700 1 -- 172.21.15.92:0/1648014560 shutdown_connections 2023-10-06T10:45:54.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.744+0000 7f9135d19700 1 --2- 172.21.15.92:0/1648014560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9130157520 0x7f91300afff0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:54.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.744+0000 7f9135d19700 1 -- 172.21.15.92:0/1648014560 >> 172.21.15.92:0/1648014560 conn(0x7f91300675b0 msgr2=0x7f91300679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:54.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.744+0000 7f9135d19700 1 -- 172.21.15.92:0/1648014560 shutdown_connections 2023-10-06T10:45:54.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.744+0000 7f9135d19700 1 -- 172.21.15.92:0/1648014560 wait complete. 2023-10-06T10:45:54.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.745+0000 7f9135d19700 1 Processor -- start 2023-10-06T10:45:54.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.745+0000 7f9135d19700 1 -- start start 2023-10-06T10:45:54.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.746+0000 7f9135d19700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91300afc20 0x7f9130153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:54.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.746+0000 7f9135d19700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91300cdfc0 0x7f91301579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:54.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.746+0000 7f9135d19700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9130157520 0x7f91300b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:54.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.746+0000 7f9135d19700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91300c2e30 con 0x7f91300cdfc0 2023-10-06T10:45:54.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.746+0000 7f9135d19700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91300c3110 con 0x7f9130157520 2023-10-06T10:45:54.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.746+0000 7f9135d19700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91301542f0 con 0x7f91300afc20 2023-10-06T10:45:54.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.746+0000 7f912ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91300cdfc0 0x7f91301579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:54.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.746+0000 7f912ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91300cdfc0 0x7f91301579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37882/0 (socket says 172.21.15.92:37882) 2023-10-06T10:45:54.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.746+0000 7f912ffff700 1 -- 172.21.15.92:0/647828529 learned_addr learned my addr 172.21.15.92:0/647828529 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:54.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.746+0000 7f9135518700 1 --2- 172.21.15.92:0/647828529 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9130157520 0x7f91300b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:54.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.746+0000 7f912ffff700 1 -- 172.21.15.92:0/647828529 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91300afc20 msgr2=0x7f9130153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:54.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.746+0000 7f9134d17700 1 --2- 172.21.15.92:0/647828529 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91300afc20 0x7f9130153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:54.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.746+0000 7f912ffff700 1 --2- 172.21.15.92:0/647828529 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91300afc20 0x7f9130153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:54.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.746+0000 7f912ffff700 1 -- 172.21.15.92:0/647828529 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9130157520 msgr2=0x7f91300b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:54.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.746+0000 7f912ffff700 1 --2- 172.21.15.92:0/647828529 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9130157520 0x7f91300b21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:54.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.746+0000 7f912ffff700 1 -- 172.21.15.92:0/647828529 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9128009a20 con 0x7f91300cdfc0 2023-10-06T10:45:54.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.746+0000 7f912ffff700 1 --2- 172.21.15.92:0/647828529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91300cdfc0 0x7f91301579a0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f911c00c600 tx=0x7f911c007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:54.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.746+0000 7f912cff9700 1 -- 172.21.15.92:0/647828529 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c011590 con 0x7f91300cdfc0 2023-10-06T10:45:54.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.746+0000 7f912cff9700 1 -- 172.21.15.92:0/647828529 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c011710 con 0x7f91300cdfc0 2023-10-06T10:45:54.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.746+0000 7f9135d19700 1 -- 172.21.15.92:0/647828529 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91301582a0 con 0x7f91300cdfc0 2023-10-06T10:45:54.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.746+0000 7f912cff9700 1 -- 172.21.15.92:0/647828529 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c011ad0 con 0x7f91300cdfc0 2023-10-06T10:45:54.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.746+0000 7f9135d19700 1 -- 172.21.15.92:0/647828529 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9130158860 con 0x7f91300cdfc0 2023-10-06T10:45:54.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.747+0000 7f9135d19700 1 -- 172.21.15.92:0/647828529 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913005f230 con 0x7f91300cdfc0 2023-10-06T10:45:54.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.748+0000 7f912cff9700 1 -- 172.21.15.92:0/647828529 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f911c01c460 con 0x7f91300cdfc0 2023-10-06T10:45:54.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.748+0000 7f912cff9700 1 --2- 172.21.15.92:0/647828529 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9114066f30 0x7f91140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:54.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.748+0000 7f912cff9700 1 -- 172.21.15.92:0/647828529 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f911c084360 con 0x7f91300cdfc0 2023-10-06T10:45:54.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.748+0000 7f9134d17700 1 --2- 172.21.15.92:0/647828529 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9114066f30 0x7f91140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:54.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.749+0000 7f9134d17700 1 --2- 172.21.15.92:0/647828529 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9114066f30 0x7f91140693e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f91240058d0 tx=0x7f9124009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:54.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.752+0000 7f912cff9700 1 -- 172.21.15.92:0/647828529 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f911c051580 con 0x7f91300cdfc0 2023-10-06T10:45:54.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.935+0000 7f9135d19700 1 -- 172.21.15.92:0/647828529 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f91300ccf50 con 0x7f91300cdfc0 2023-10-06T10:45:54.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.939+0000 7f912cff9700 1 -- 172.21.15.92:0/647828529 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f911c054ba0 con 0x7f91300cdfc0 2023-10-06T10:45:54.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.941+0000 7f9135d19700 1 -- 172.21.15.92:0/647828529 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9114066f30 msgr2=0x7f91140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:54.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.941+0000 7f9135d19700 1 --2- 172.21.15.92:0/647828529 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9114066f30 0x7f91140693e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f91240058d0 tx=0x7f9124009040 comp rx=0 tx=0).stop 2023-10-06T10:45:54.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.941+0000 7f9135d19700 1 -- 172.21.15.92:0/647828529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91300cdfc0 msgr2=0x7f91301579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:54.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.941+0000 7f9135d19700 1 --2- 172.21.15.92:0/647828529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91300cdfc0 0x7f91301579a0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f911c00c600 tx=0x7f911c007f60 comp rx=0 tx=0).stop 2023-10-06T10:45:54.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.941+0000 7f9135d19700 1 -- 172.21.15.92:0/647828529 shutdown_connections 2023-10-06T10:45:54.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.941+0000 7f9135d19700 1 --2- 172.21.15.92:0/647828529 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9114066f30 0x7f91140693e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:54.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.941+0000 7f9135d19700 1 --2- 172.21.15.92:0/647828529 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9130157520 0x7f91300b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:54.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.941+0000 7f9135d19700 1 --2- 172.21.15.92:0/647828529 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91300afc20 0x7f9130153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:54.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.941+0000 7f9135d19700 1 --2- 172.21.15.92:0/647828529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91300cdfc0 0x7f91301579a0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:54.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.941+0000 7f9135d19700 1 -- 172.21.15.92:0/647828529 >> 172.21.15.92:0/647828529 conn(0x7f91300675b0 msgr2=0x7f91300ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:54.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.941+0000 7f9135d19700 1 -- 172.21.15.92:0/647828529 shutdown_connections 2023-10-06T10:45:54.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:54.941+0000 7f9135d19700 1 -- 172.21.15.92:0/647828529 wait complete. 2023-10-06T10:45:54.955 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0' 2023-10-06T10:45:55.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.067+0000 7f38de588700 1 Processor -- start 2023-10-06T10:45:55.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.067+0000 7f38de588700 1 -- start start 2023-10-06T10:45:55.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.067+0000 7f38de588700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f38d80b49a0 0x7f38d80aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:55.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.067+0000 7f38de588700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d80b0c60 con 0x7f38d8159560 2023-10-06T10:45:55.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.067+0000 7f38de588700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d80b0f40 con 0x7f38d80b0760 2023-10-06T10:45:55.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.067+0000 7f38de588700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d80b1220 con 0x7f38d80b49a0 2023-10-06T10:45:55.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.067+0000 7f38d7fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f38d8159560 0x7f38d80aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35350/0 (socket says 172.21.15.92:35350) 2023-10-06T10:45:55.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.067+0000 7f38d77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f38d80b49a0 0x7f38d80aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:55.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.068+0000 7f38d7fff700 1 -- 172.21.15.92:0/129128643 learned_addr learned my addr 172.21.15.92:0/129128643 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:55.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.068+0000 7f38d6ffd700 1 -- 172.21.15.92:0/129128643 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 675891824 0 0) 0x7f38d80b0c60 con 0x7f38d8159560 2023-10-06T10:45:55.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.068+0000 7f38d6ffd700 1 -- 172.21.15.92:0/129128643 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38c0003570 con 0x7f38d8159560 2023-10-06T10:45:55.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.068+0000 7f38d77fe700 1 -- 172.21.15.92:0/129128643 >> v1:172.21.15.124:6789/0 conn(0x7f38d80b0760 legacy=0x7f38d80b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:55.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.068+0000 7f38d77fe700 1 -- 172.21.15.92:0/129128643 >> v1:172.21.15.92:6789/0 conn(0x7f38d8159560 legacy=0x7f38d80aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:55.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.068+0000 7f38d77fe700 1 -- 172.21.15.92:0/129128643 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d80b1500 con 0x7f38d80b49a0 2023-10-06T10:45:55.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.068+0000 7f38d77fe700 1 --2- 172.21.15.92:0/129128643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d80b49a0 0x7f38d80aac40 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009910 tx=0x7f38cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dce0da56e8dff57 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:55.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.068+0000 7f38d6ffd700 1 -- 172.21.15.92:0/129128643 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc00b640 con 0x7f38d80b49a0 2023-10-06T10:45:55.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.068+0000 7f38d6ffd700 1 -- 172.21.15.92:0/129128643 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38cc00b7a0 con 0x7f38d80b49a0 2023-10-06T10:45:55.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.068+0000 7f38d6ffd700 1 -- 172.21.15.92:0/129128643 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc00bb60 con 0x7f38d80b49a0 2023-10-06T10:45:55.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.069+0000 7f38de588700 1 -- 172.21.15.92:0/129128643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d80b49a0 msgr2=0x7f38d80aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:55.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.069+0000 7f38de588700 1 --2- 172.21.15.92:0/129128643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d80b49a0 0x7f38d80aac40 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009910 tx=0x7f38cc0091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:55.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.069+0000 7f38de588700 1 -- 172.21.15.92:0/129128643 shutdown_connections 2023-10-06T10:45:55.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.069+0000 7f38de588700 1 --2- 172.21.15.92:0/129128643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d80b49a0 0x7f38d80aac40 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:55.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.069+0000 7f38de588700 1 -- 172.21.15.92:0/129128643 >> 172.21.15.92:0/129128643 conn(0x7f38d80997f0 msgr2=0x7f38d806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:55.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.069+0000 7f38de588700 1 -- 172.21.15.92:0/129128643 shutdown_connections 2023-10-06T10:45:55.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.069+0000 7f38de588700 1 -- 172.21.15.92:0/129128643 wait complete. 2023-10-06T10:45:55.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.070+0000 7f38de588700 1 Processor -- start 2023-10-06T10:45:55.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.070+0000 7f38de588700 1 -- start start 2023-10-06T10:45:55.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.071+0000 7f38de588700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d80b0760 0x7f38d80ab290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:55.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.071+0000 7f38de588700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38d80b49a0 0x7f38d809d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:55.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.071+0000 7f38de588700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38d8159560 0x7f38d809ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:55.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.071+0000 7f38de588700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d809e090 con 0x7f38d80b0760 2023-10-06T10:45:55.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.071+0000 7f38de588700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d80ab7d0 con 0x7f38d80b49a0 2023-10-06T10:45:55.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.071+0000 7f38de588700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d809ddb0 con 0x7f38d8159560 2023-10-06T10:45:55.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.071+0000 7f38d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d80b0760 0x7f38d80ab290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:55.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.071+0000 7f38d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d80b0760 0x7f38d80ab290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37908/0 (socket says 172.21.15.92:37908) 2023-10-06T10:45:55.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.071+0000 7f38d7fff700 1 -- 172.21.15.92:0/307126889 learned_addr learned my addr 172.21.15.92:0/307126889 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:55.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.071+0000 7f38d77fe700 1 --2- 172.21.15.92:0/307126889 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38d80b49a0 0x7f38d809d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:55.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.071+0000 7f38dcb25700 1 --2- 172.21.15.92:0/307126889 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38d8159560 0x7f38d809ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:55.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.072+0000 7f38d7fff700 1 -- 172.21.15.92:0/307126889 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38d8159560 msgr2=0x7f38d809ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:55.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.072+0000 7f38d7fff700 1 --2- 172.21.15.92:0/307126889 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38d8159560 0x7f38d809ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:55.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.072+0000 7f38d7fff700 1 -- 172.21.15.92:0/307126889 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38d80b49a0 msgr2=0x7f38d809d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:55.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.072+0000 7f38d7fff700 1 --2- 172.21.15.92:0/307126889 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38d80b49a0 0x7f38d809d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:55.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.072+0000 7f38d7fff700 1 -- 172.21.15.92:0/307126889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc009420 con 0x7f38d80b0760 2023-10-06T10:45:55.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.073+0000 7f38d7fff700 1 --2- 172.21.15.92:0/307126889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d80b0760 0x7f38d80ab290 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f38c800c790 tx=0x7f38c8007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:55.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.073+0000 7f38bffff700 1 -- 172.21.15.92:0/307126889 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c80072e0 con 0x7f38d80b0760 2023-10-06T10:45:55.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.073+0000 7f38bffff700 1 -- 172.21.15.92:0/307126889 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c8007440 con 0x7f38d80b0760 2023-10-06T10:45:55.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.073+0000 7f38de588700 1 -- 172.21.15.92:0/307126889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d80ada10 con 0x7f38d80b0760 2023-10-06T10:45:55.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.073+0000 7f38bffff700 1 -- 172.21.15.92:0/307126889 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c80057f0 con 0x7f38d80b0760 2023-10-06T10:45:55.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.073+0000 7f38de588700 1 -- 172.21.15.92:0/307126889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d80adfd0 con 0x7f38d80b0760 2023-10-06T10:45:55.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.074+0000 7f38de588700 1 -- 172.21.15.92:0/307126889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d80ad870 con 0x7f38d80b0760 2023-10-06T10:45:55.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.075+0000 7f38bffff700 1 -- 172.21.15.92:0/307126889 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f38c8005be0 con 0x7f38d80b0760 2023-10-06T10:45:55.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.076+0000 7f38bffff700 1 --2- 172.21.15.92:0/307126889 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f38c4066f30 0x7f38c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:55.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.076+0000 7f38bffff700 1 -- 172.21.15.92:0/307126889 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f38c8083be0 con 0x7f38d80b0760 2023-10-06T10:45:55.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.076+0000 7f38d77fe700 1 --2- 172.21.15.92:0/307126889 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f38c4066f30 0x7f38c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:55.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.077+0000 7f38d77fe700 1 --2- 172.21.15.92:0/307126889 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f38c4066f30 0x7f38c40693e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009910 tx=0x7f38cc017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:55.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.080+0000 7f38bffff700 1 -- 172.21.15.92:0/307126889 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38c8050e90 con 0x7f38d80b0760 2023-10-06T10:45:55.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.335+0000 7f38de588700 1 -- 172.21.15.92:0/307126889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]} v 0) v1 -- 0x7f38d80ad870 con 0x7f38d80b0760 2023-10-06T10:45:55.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.607+0000 7f38bffff700 1 -- 172.21.15.92:0/307126889 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]}]=0 v391) v1 ==== 165+0+0 (secure 0 0 0) 0x7f38d80ad870 con 0x7f38d80b0760 2023-10-06T10:45:55.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.609+0000 7f38de588700 1 -- 172.21.15.92:0/307126889 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f38c4066f30 msgr2=0x7f38c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:55.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.609+0000 7f38de588700 1 --2- 172.21.15.92:0/307126889 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f38c4066f30 0x7f38c40693e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009910 tx=0x7f38cc017000 comp rx=0 tx=0).stop 2023-10-06T10:45:55.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.609+0000 7f38de588700 1 -- 172.21.15.92:0/307126889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d80b0760 msgr2=0x7f38d80ab290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:55.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.609+0000 7f38de588700 1 --2- 172.21.15.92:0/307126889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d80b0760 0x7f38d80ab290 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f38c800c790 tx=0x7f38c8007c80 comp rx=0 tx=0).stop 2023-10-06T10:45:55.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.609+0000 7f38de588700 1 -- 172.21.15.92:0/307126889 shutdown_connections 2023-10-06T10:45:55.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.609+0000 7f38de588700 1 --2- 172.21.15.92:0/307126889 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f38c4066f30 0x7f38c40693e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:55.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.609+0000 7f38de588700 1 --2- 172.21.15.92:0/307126889 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38d8159560 0x7f38d809ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:55.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.609+0000 7f38de588700 1 --2- 172.21.15.92:0/307126889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d80b0760 0x7f38d80ab290 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:55.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.609+0000 7f38de588700 1 --2- 172.21.15.92:0/307126889 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38d80b49a0 0x7f38d809d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:55.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.609+0000 7f38de588700 1 -- 172.21.15.92:0/307126889 >> 172.21.15.92:0/307126889 conn(0x7f38d80997f0 msgr2=0x7f38d80688b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:55.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.610+0000 7f38de588700 1 -- 172.21.15.92:0/307126889 shutdown_connections 2023-10-06T10:45:55.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:55.610+0000 7f38de588700 1 -- 172.21.15.92:0/307126889 wait complete. 2023-10-06T10:45:55.623 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_with_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2023-10-06T10:45:55.624 INFO:tasks.cephfs_test_runner: 2023-10-06T10:45:55.624 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-06T10:45:55.624 INFO:tasks.cephfs_test_runner:Ran 6 tests in 418.414s 2023-10-06T10:45:55.624 INFO:tasks.cephfs_test_runner: 2023-10-06T10:45:55.624 INFO:tasks.cephfs_test_runner:OK 2023-10-06T10:45:55.624 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-06T10:45:55.635 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-06T10:45:55.685 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-06T10:45:55.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:45:55.685 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T10:45:55.700 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T10:45:55.700 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T10:45:55.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:45:55.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T10:45:55.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:45:55.772 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T10:45:55.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:45:55.856 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2023-10-06T10:45:55.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:45:55.959 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T10:45:55.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:45:55.974 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2023-10-06T10:45:56.042 INFO:teuthology.orchestra.run.smithi092.stderr:Cannot find device "ceph-brx" 2023-10-06T10:45:56.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T10:45:56.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:45:56.043 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2023-10-06T10:45:56.060 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T10:45:56.060 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T10:45:56.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:45:56.060 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T10:45:56.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:45:56.139 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T10:45:56.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:45:56.222 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link delete ceph-brx 2023-10-06T10:45:56.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:45:56.319 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2023-10-06T10:45:56.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T10:45:56.334 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link delete ceph-brx 2023-10-06T10:45:56.402 INFO:teuthology.orchestra.run.smithi124.stderr:Cannot find device "ceph-brx" 2023-10-06T10:45:56.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T10:45:56.403 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-06T10:45:56.412 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-06T10:45:56.412 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:45:56.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.527+0000 7f5aebfff700 1 Processor -- start 2023-10-06T10:45:56.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.527+0000 7f5aebfff700 1 -- start start 2023-10-06T10:45:56.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.528+0000 7f5aebfff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5aec06a590 0x7f5aec1536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:56.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.528+0000 7f5aebfff700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5aec06a960 0x7f5aec153c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:56.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.528+0000 7f5aebfff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aec0c9cc0 con 0x7f5aec06df50 2023-10-06T10:45:56.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.528+0000 7f5aebfff700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aec06f560 con 0x7f5aec06a960 2023-10-06T10:45:56.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.528+0000 7f5aebfff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec06eac0 con 0x7f5aec06a590 2023-10-06T10:45:56.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.528+0000 7f5aeb7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5aec06df50 0x7f5aec06e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35356/0 (socket says 172.21.15.92:35356) 2023-10-06T10:45:56.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.528+0000 7f5aeb7fe700 1 -- 172.21.15.92:0/1753303634 learned_addr learned my addr 172.21.15.92:0/1753303634 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:56.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.528+0000 7f5aeaffd700 1 --2- 172.21.15.92:0/1753303634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5aec06a590 0x7f5aec1536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:56.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.528+0000 7f5ae27fc700 1 --2- 172.21.15.92:0/1753303634 >> v2:172.21.15.124:3300/0 conn(0x7f5aec06a960 0x7f5aec153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:56.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.528+0000 7f5aea7fc700 1 -- 172.21.15.92:0/1753303634 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902686975 0 0) 0x7f5aec0c9cc0 con 0x7f5aec06df50 2023-10-06T10:45:56.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.528+0000 7f5aea7fc700 1 -- 172.21.15.92:0/1753303634 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ad8003570 con 0x7f5aec06df50 2023-10-06T10:45:56.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.528+0000 7f5aea7fc700 1 -- 172.21.15.92:0/1753303634 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 948551673 0 0) 0x7f5ad8003570 con 0x7f5aec06df50 2023-10-06T10:45:56.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.528+0000 7f5aea7fc700 1 -- 172.21.15.92:0/1753303634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5aec06a590 msgr2=0x7f5aec1536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:56.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.528+0000 7f5aea7fc700 1 --2- 172.21.15.92:0/1753303634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5aec06a590 0x7f5aec1536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:56.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.528+0000 7f5aea7fc700 1 -- 172.21.15.92:0/1753303634 >> v2:172.21.15.124:3300/0 conn(0x7f5aec06a960 msgr2=0x7f5aec153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:56.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.528+0000 7f5aea7fc700 1 --2- 172.21.15.92:0/1753303634 >> v2:172.21.15.124:3300/0 conn(0x7f5aec06a960 0x7f5aec153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:56.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.528+0000 7f5aea7fc700 1 -- 172.21.15.92:0/1753303634 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec0c6710 con 0x7f5aec06df50 2023-10-06T10:45:56.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.528+0000 7f5aea7fc700 1 -- 172.21.15.92:0/1753303634 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f5ae40017f0 con 0x7f5aec06df50 2023-10-06T10:45:56.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.529+0000 7f5aea7fc700 1 -- 172.21.15.92:0/1753303634 >> v1:172.21.15.92:6789/0 conn(0x7f5aec06df50 legacy=0x7f5aec06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:56.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.529+0000 7f5aea7fc700 1 --2- 172.21.15.92:0/1753303634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ad8003a90 0x7f5ad8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:56.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.529+0000 7f5aea7fc700 1 --2- 172.21.15.92:0/1753303634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5aec06a590 0x7f5ad8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:56.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.529+0000 7f5aea7fc700 1 --2- 172.21.15.92:0/1753303634 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ad8004d50 0x7f5ad80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:56.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.529+0000 7f5aea7fc700 1 -- 172.21.15.92:0/1753303634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec06f560 con 0x7f5ad8003a90 2023-10-06T10:45:56.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.529+0000 7f5aea7fc700 1 -- 172.21.15.92:0/1753303634 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec06eac0 con 0x7f5ad8004d50 2023-10-06T10:45:56.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.529+0000 7f5aea7fc700 1 -- 172.21.15.92:0/1753303634 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad800ba60 con 0x7f5aec06a590 2023-10-06T10:45:56.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.529+0000 7f5aeaffd700 1 --2- 172.21.15.92:0/1753303634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ad8003a90 0x7f5ad8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:56.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.529+0000 7f5ae27fc700 1 --2- 172.21.15.92:0/1753303634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5aec06a590 0x7f5ad8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:56.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.529+0000 7f5aeb7fe700 1 --2- 172.21.15.92:0/1753303634 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ad8004d50 0x7f5ad80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:56.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.529+0000 7f5aeaffd700 1 -- 172.21.15.92:0/1753303634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5aec06a590 msgr2=0x7f5ad8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:56.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.529+0000 7f5aeaffd700 1 --2- 172.21.15.92:0/1753303634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5aec06a590 0x7f5ad8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:56.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.529+0000 7f5aeaffd700 1 -- 172.21.15.92:0/1753303634 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ad8004d50 msgr2=0x7f5ad80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:56.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.529+0000 7f5aeaffd700 1 --2- 172.21.15.92:0/1753303634 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ad8004d50 0x7f5ad80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:56.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.529+0000 7f5aeaffd700 1 -- 172.21.15.92:0/1753303634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad8003570 con 0x7f5ad8003a90 2023-10-06T10:45:56.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.529+0000 7f5aeaffd700 1 --2- 172.21.15.92:0/1753303634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ad8003a90 0x7f5ad8003ea0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009c00 tx=0x7f5adc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b3afbfe0e276b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:56.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.529+0000 7f5aea7fc700 1 -- 172.21.15.92:0/1753303634 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc00e6b0 con 0x7f5ad8003a90 2023-10-06T10:45:56.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.529+0000 7f5aea7fc700 1 -- 172.21.15.92:0/1753303634 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5adc00e810 con 0x7f5ad8003a90 2023-10-06T10:45:56.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.529+0000 7f5aea7fc700 1 -- 172.21.15.92:0/1753303634 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc00ebd0 con 0x7f5ad8003a90 2023-10-06T10:45:56.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.530+0000 7f5aebfff700 1 -- 172.21.15.92:0/1753303634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ad8003a90 msgr2=0x7f5ad8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:56.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.530+0000 7f5aebfff700 1 --2- 172.21.15.92:0/1753303634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ad8003a90 0x7f5ad8003ea0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009c00 tx=0x7f5adc0091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:56.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.530+0000 7f5aeb7fe700 1 -- 172.21.15.92:0/1753303634 reap_dead start 2023-10-06T10:45:56.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.530+0000 7f5aebfff700 1 -- 172.21.15.92:0/1753303634 shutdown_connections 2023-10-06T10:45:56.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.530+0000 7f5aebfff700 1 -- 172.21.15.92:0/1753303634 >> 172.21.15.92:0/1753303634 conn(0x7f5aec068080 msgr2=0x7f5aec0cbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:56.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.530+0000 7f5aebfff700 1 -- 172.21.15.92:0/1753303634 shutdown_connections 2023-10-06T10:45:56.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.530+0000 7f5aebfff700 1 -- 172.21.15.92:0/1753303634 wait complete. 2023-10-06T10:45:56.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.531+0000 7f5aebfff700 1 Processor -- start 2023-10-06T10:45:56.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.531+0000 7f5aebfff700 1 -- start start 2023-10-06T10:45:56.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5aebfff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ad8003a90 0x7f5aec1566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:56.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5aebfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5aec15fa70 0x7f5aec0c2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:56.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5aebfff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5aec145c30 0x7f5aec15de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:56.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5aebfff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec160660 con 0x7f5aec15fa70 2023-10-06T10:45:56.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5aebfff700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec0b0110 con 0x7f5aec145c30 2023-10-06T10:45:56.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5aebfff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec0c29b0 con 0x7f5ad8003a90 2023-10-06T10:45:56.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5ae27fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5aec15fa70 0x7f5aec0c2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:56.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5ae27fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5aec15fa70 0x7f5aec0c2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37920/0 (socket says 172.21.15.92:37920) 2023-10-06T10:45:56.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5ae27fc700 1 -- 172.21.15.92:0/4262873932 learned_addr learned my addr 172.21.15.92:0/4262873932 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:56.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5aeaffd700 1 --2- 172.21.15.92:0/4262873932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ad8003a90 0x7f5aec1566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:56.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5aeb7fe700 1 --2- 172.21.15.92:0/4262873932 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5aec145c30 0x7f5aec15de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:56.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5ae27fc700 1 -- 172.21.15.92:0/4262873932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ad8003a90 msgr2=0x7f5aec1566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:56.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5ae27fc700 1 --2- 172.21.15.92:0/4262873932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ad8003a90 0x7f5aec1566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:56.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5ae27fc700 1 -- 172.21.15.92:0/4262873932 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5aec145c30 msgr2=0x7f5aec15de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:56.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5ae27fc700 1 --2- 172.21.15.92:0/4262873932 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5aec145c30 0x7f5aec15de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:56.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5ae27fc700 1 -- 172.21.15.92:0/4262873932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec1600a0 con 0x7f5aec15fa70 2023-10-06T10:45:56.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5aeb7fe700 1 --2- 172.21.15.92:0/4262873932 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5aec145c30 0x7f5aec15de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:56.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5aeaffd700 1 --2- 172.21.15.92:0/4262873932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ad8003a90 0x7f5aec1566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:56.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5ae27fc700 1 --2- 172.21.15.92:0/4262873932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5aec15fa70 0x7f5aec0c2470 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4011da0 tx=0x7f5ad40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:56.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5ae37fe700 1 -- 172.21.15.92:0/4262873932 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4009e20 con 0x7f5aec15fa70 2023-10-06T10:45:56.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5ae37fe700 1 -- 172.21.15.92:0/4262873932 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad400fcd0 con 0x7f5aec15fa70 2023-10-06T10:45:56.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5aebfff700 1 -- 172.21.15.92:0/4262873932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aec160380 con 0x7f5aec15fa70 2023-10-06T10:45:56.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5ae37fe700 1 -- 172.21.15.92:0/4262873932 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4014650 con 0x7f5aec15fa70 2023-10-06T10:45:56.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.532+0000 7f5aebfff700 1 -- 172.21.15.92:0/4262873932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aec1460a0 con 0x7f5aec15fa70 2023-10-06T10:45:56.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.533+0000 7f5aebfff700 1 -- 172.21.15.92:0/4262873932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aec0afc00 con 0x7f5aec15fa70 2023-10-06T10:45:56.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.534+0000 7f5ae37fe700 1 -- 172.21.15.92:0/4262873932 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ad4014a80 con 0x7f5aec15fa70 2023-10-06T10:45:56.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.534+0000 7f5ae37fe700 1 --2- 172.21.15.92:0/4262873932 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5acc066f30 0x7f5acc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:56.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.534+0000 7f5ae37fe700 1 -- 172.21.15.92:0/4262873932 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f5ad4018030 con 0x7f5aec15fa70 2023-10-06T10:45:56.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.534+0000 7f5aeaffd700 1 --2- 172.21.15.92:0/4262873932 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5acc066f30 0x7f5acc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:56.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.534+0000 7f5aeaffd700 1 --2- 172.21.15.92:0/4262873932 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5acc066f30 0x7f5acc0693e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0091e0 tx=0x7f5adc01a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:56.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.538+0000 7f5ae37fe700 1 -- 172.21.15.92:0/4262873932 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5aec0afc00 con 0x7f5aec15fa70 2023-10-06T10:45:56.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.723+0000 7f5aebfff700 1 -- 172.21.15.92:0/4262873932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5aec1607a0 con 0x7f5aec15fa70 2023-10-06T10:45:56.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.724+0000 7f5ae37fe700 1 -- 172.21.15.92:0/4262873932 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+14256 (secure 0 0 0) 0x7f5ad4022090 con 0x7f5aec15fa70 2023-10-06T10:45:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:56.726 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":107,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:45:54.601348+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6835/2298348426":"2023-10-07T10:45:51.590092+0000","172.21.15.124:6835/2307521923":"2023-10-07T10:45:51.590072+0000","172.21.15.124:6833/2307521923":"2023-10-07T10:45:51.590072+0000","172.21.15.92:6837/2298348426":"2023-10-07T10:45:51.590092+0000","172.21.15.92:6836/1979826278":"2023-10-07T10:45:51.590086+0000","172.21.15.92:6834/1979826278":"2023-10-07T10:45:51.590086+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:45:56.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.726+0000 7f5aebfff700 1 -- 172.21.15.92:0/4262873932 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5acc066f30 msgr2=0x7f5acc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.726+0000 7f5aebfff700 1 --2- 172.21.15.92:0/4262873932 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5acc066f30 0x7f5acc0693e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0091e0 tx=0x7f5adc01a000 comp rx=0 tx=0).stop 2023-10-06T10:45:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.726+0000 7f5aebfff700 1 -- 172.21.15.92:0/4262873932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5aec15fa70 msgr2=0x7f5aec0c2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.726+0000 7f5aebfff700 1 --2- 172.21.15.92:0/4262873932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5aec15fa70 0x7f5aec0c2470 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4011da0 tx=0x7f5ad40091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.726+0000 7f5aebfff700 1 -- 172.21.15.92:0/4262873932 shutdown_connections 2023-10-06T10:45:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.726+0000 7f5aebfff700 1 --2- 172.21.15.92:0/4262873932 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5acc066f30 0x7f5acc0693e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.726+0000 7f5aebfff700 1 --2- 172.21.15.92:0/4262873932 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5aec145c30 0x7f5aec15de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.726+0000 7f5aebfff700 1 --2- 172.21.15.92:0/4262873932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ad8003a90 0x7f5aec1566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.726+0000 7f5aebfff700 1 --2- 172.21.15.92:0/4262873932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5aec15fa70 0x7f5aec0c2470 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.726+0000 7f5aebfff700 1 -- 172.21.15.92:0/4262873932 >> 172.21.15.92:0/4262873932 conn(0x7f5aec068080 msgr2=0x7f5aec146e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.726+0000 7f5aebfff700 1 -- 172.21.15.92:0/4262873932 shutdown_connections 2023-10-06T10:45:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.726+0000 7f5aebfff700 1 -- 172.21.15.92:0/4262873932 wait complete. 2023-10-06T10:45:56.740 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T10:45:56.740 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:45:56.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.852+0000 7f6f0d3a3700 1 Processor -- start 2023-10-06T10:45:56.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.852+0000 7f6f0d3a3700 1 -- start start 2023-10-06T10:45:56.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.853+0000 7f6f0d3a3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6f08159390 0x7f6f080ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:56.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.853+0000 7f6f0d3a3700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6f08159760 0x7f6f080aad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:56.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.853+0000 7f6f0d3a3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f080686c0 con 0x7f6f080ab240 2023-10-06T10:45:56.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.853+0000 7f6f0d3a3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f08068c80 con 0x7f6f08159390 2023-10-06T10:45:56.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.853+0000 7f6f0d3a3700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f080ab660 con 0x7f6f08159760 2023-10-06T10:45:56.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.853+0000 7f6f07fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6f08159390 0x7f6f080ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:56.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.853+0000 7f6f07fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6f08159390 0x7f6f080ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37924/0 (socket says 172.21.15.92:37924) 2023-10-06T10:45:56.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.853+0000 7f6f07fff700 1 -- 172.21.15.92:0/952496437 learned_addr learned my addr 172.21.15.92:0/952496437 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:56.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.853+0000 7f6f0cba2700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6f080ab240 0x7f6f08069110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35370/0 (socket says 172.21.15.92:35370) 2023-10-06T10:45:56.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.854+0000 7f6f077fe700 1 --2- 172.21.15.92:0/952496437 >> v2:172.21.15.124:3300/0 conn(0x7f6f08159760 0x7f6f080aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:56.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.854+0000 7f6f07fff700 1 -- 172.21.15.92:0/952496437 >> v2:172.21.15.124:3300/0 conn(0x7f6f08159760 msgr2=0x7f6f080aad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:56.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.854+0000 7f6f07fff700 1 --2- 172.21.15.92:0/952496437 >> v2:172.21.15.124:3300/0 conn(0x7f6f08159760 0x7f6f080aad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:56.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.854+0000 7f6f07fff700 1 -- 172.21.15.92:0/952496437 >> v1:172.21.15.92:6789/0 conn(0x7f6f080ab240 legacy=0x7f6f08069110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:56.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.854+0000 7f6f07fff700 1 -- 172.21.15.92:0/952496437 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f080ace20 con 0x7f6f08159390 2023-10-06T10:45:56.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.854+0000 7f6f06ffd700 1 -- 172.21.15.92:0/952496437 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262066385 0 0) 0x7f6f080686c0 con 0x7f6f080ab240 2023-10-06T10:45:56.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.854+0000 7f6f07fff700 1 --2- 172.21.15.92:0/952496437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f08159390 0x7f6f080ac8e0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f6f0806ee70 tx=0x7f6ef8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1a8d822218123f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:56.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.854+0000 7f6f06ffd700 1 -- 172.21.15.92:0/952496437 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef800b860 con 0x7f6f08159390 2023-10-06T10:45:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.854+0000 7f6f06ffd700 1 -- 172.21.15.92:0/952496437 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef800b9c0 con 0x7f6f08159390 2023-10-06T10:45:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.854+0000 7f6f06ffd700 1 -- 172.21.15.92:0/952496437 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef800bd60 con 0x7f6f08159390 2023-10-06T10:45:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.854+0000 7f6f0d3a3700 1 -- 172.21.15.92:0/952496437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f08159390 msgr2=0x7f6f080ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.854+0000 7f6f0d3a3700 1 --2- 172.21.15.92:0/952496437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f08159390 0x7f6f080ac8e0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f6f0806ee70 tx=0x7f6ef8009250 comp rx=0 tx=0).stop 2023-10-06T10:45:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.855+0000 7f6f0d3a3700 1 -- 172.21.15.92:0/952496437 shutdown_connections 2023-10-06T10:45:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.855+0000 7f6f0d3a3700 1 --2- 172.21.15.92:0/952496437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f08159390 0x7f6f080ac8e0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.855+0000 7f6f0d3a3700 1 --2- 172.21.15.92:0/952496437 >> v2:172.21.15.124:3300/0 conn(0x7f6f08159760 0x7f6f080aad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.855+0000 7f6f0d3a3700 1 -- 172.21.15.92:0/952496437 >> 172.21.15.92:0/952496437 conn(0x7f6f08097410 msgr2=0x7f6f08067d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.855+0000 7f6f0d3a3700 1 -- 172.21.15.92:0/952496437 shutdown_connections 2023-10-06T10:45:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.855+0000 7f6f0d3a3700 1 -- 172.21.15.92:0/952496437 wait complete. 2023-10-06T10:45:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.856+0000 7f6f0d3a3700 1 Processor -- start 2023-10-06T10:45:56.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.856+0000 7f6f0d3a3700 1 -- start start 2023-10-06T10:45:56.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.856+0000 7f6f0d3a3700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f080ab240 0x7f6f080b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:56.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.857+0000 7f6f0d3a3700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f08159390 0x7f6f08155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:56.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.857+0000 7f6f0d3a3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f08159760 0x7f6f0809a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:56.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.857+0000 7f6f0d3a3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0809bd80 con 0x7f6f08159760 2023-10-06T10:45:56.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.857+0000 7f6f0d3a3700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f081562b0 con 0x7f6f080ab240 2023-10-06T10:45:56.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.857+0000 7f6f0d3a3700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0809b200 con 0x7f6f08159390 2023-10-06T10:45:56.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.857+0000 7f6f0cba2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f08159760 0x7f6f0809a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:56.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.857+0000 7f6f0cba2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f08159760 0x7f6f0809a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37930/0 (socket says 172.21.15.92:37930) 2023-10-06T10:45:56.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.857+0000 7f6f0cba2700 1 -- 172.21.15.92:0/3415613872 learned_addr learned my addr 172.21.15.92:0/3415613872 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:56.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.857+0000 7f6f07fff700 1 --2- 172.21.15.92:0/3415613872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f080ab240 0x7f6f080b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:56.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.857+0000 7f6f077fe700 1 --2- 172.21.15.92:0/3415613872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f08159390 0x7f6f08155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:56.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.857+0000 7f6f0cba2700 1 -- 172.21.15.92:0/3415613872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f08159390 msgr2=0x7f6f08155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:56.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.857+0000 7f6f0cba2700 1 --2- 172.21.15.92:0/3415613872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f08159390 0x7f6f08155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:56.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.857+0000 7f6f0cba2700 1 -- 172.21.15.92:0/3415613872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f080ab240 msgr2=0x7f6f080b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:56.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.857+0000 7f6f0cba2700 1 --2- 172.21.15.92:0/3415613872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f080ab240 0x7f6f080b31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:56.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.857+0000 7f6f0cba2700 1 -- 172.21.15.92:0/3415613872 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef80094f0 con 0x7f6f08159760 2023-10-06T10:45:56.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.857+0000 7f6f0cba2700 1 --2- 172.21.15.92:0/3415613872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f08159760 0x7f6f0809a9a0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000ccc0 tx=0x7f6f00001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:56.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.857+0000 7f6eeffff700 1 -- 172.21.15.92:0/3415613872 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f00005df0 con 0x7f6f08159760 2023-10-06T10:45:56.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.857+0000 7f6eeffff700 1 -- 172.21.15.92:0/3415613872 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0000a600 con 0x7f6f08159760 2023-10-06T10:45:56.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.857+0000 7f6f0d3a3700 1 -- 172.21.15.92:0/3415613872 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f0809b4e0 con 0x7f6f08159760 2023-10-06T10:45:56.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.857+0000 7f6eeffff700 1 -- 172.21.15.92:0/3415613872 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0000a9c0 con 0x7f6f08159760 2023-10-06T10:45:56.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.857+0000 7f6f0d3a3700 1 -- 172.21.15.92:0/3415613872 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f0809baa0 con 0x7f6f08159760 2023-10-06T10:45:56.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.858+0000 7f6f0d3a3700 1 -- 172.21.15.92:0/3415613872 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f0805f230 con 0x7f6f08159760 2023-10-06T10:45:56.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.859+0000 7f6eeffff700 1 -- 172.21.15.92:0/3415613872 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6f000070f0 con 0x7f6f08159760 2023-10-06T10:45:56.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.859+0000 7f6eeffff700 1 --2- 172.21.15.92:0/3415613872 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6ef4066f60 0x7f6ef4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:56.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.859+0000 7f6eeffff700 1 -- 172.21.15.92:0/3415613872 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f6f00084510 con 0x7f6f08159760 2023-10-06T10:45:56.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.859+0000 7f6f07fff700 1 --2- 172.21.15.92:0/3415613872 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6ef4066f60 0x7f6ef4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:56.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.859+0000 7f6f07fff700 1 --2- 172.21.15.92:0/3415613872 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6ef4066f60 0x7f6ef4069410 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6f0806ee70 tx=0x7f6ef8017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:56.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:56.863+0000 7f6eeffff700 1 -- 172.21.15.92:0/3415613872 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f00051860 con 0x7f6f08159760 2023-10-06T10:45:57.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.042+0000 7f6f0d3a3700 1 -- 172.21.15.92:0/3415613872 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6f0806bac0 con 0x7f6ef4066f60 2023-10-06T10:45:57.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.045+0000 7f6eeffff700 1 -- 172.21.15.92:0/3415613872 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73209 (secure 0 0 0) 0x7f6f0806bac0 con 0x7f6ef4066f60 2023-10-06T10:45:57.048 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:57.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.047+0000 7f6f0d3a3700 1 -- 172.21.15.92:0/3415613872 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6ef4066f60 msgr2=0x7f6ef4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:57.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.047+0000 7f6f0d3a3700 1 --2- 172.21.15.92:0/3415613872 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6ef4066f60 0x7f6ef4069410 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6f0806ee70 tx=0x7f6ef8017000 comp rx=0 tx=0).stop 2023-10-06T10:45:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.047+0000 7f6f0d3a3700 1 -- 172.21.15.92:0/3415613872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f08159760 msgr2=0x7f6f0809a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.047+0000 7f6f0d3a3700 1 --2- 172.21.15.92:0/3415613872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f08159760 0x7f6f0809a9a0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000ccc0 tx=0x7f6f00001a70 comp rx=0 tx=0).stop 2023-10-06T10:45:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.047+0000 7f6f0d3a3700 1 -- 172.21.15.92:0/3415613872 shutdown_connections 2023-10-06T10:45:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.047+0000 7f6f0d3a3700 1 --2- 172.21.15.92:0/3415613872 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6ef4066f60 0x7f6ef4069410 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.047+0000 7f6f0d3a3700 1 --2- 172.21.15.92:0/3415613872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f08159760 0x7f6f0809a9a0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.047+0000 7f6f0d3a3700 1 --2- 172.21.15.92:0/3415613872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f080ab240 0x7f6f080b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.047+0000 7f6f0d3a3700 1 --2- 172.21.15.92:0/3415613872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f08159390 0x7f6f08155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.047+0000 7f6f0d3a3700 1 -- 172.21.15.92:0/3415613872 >> 172.21.15.92:0/3415613872 conn(0x7f6f08097410 msgr2=0x7f6f08067d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.047+0000 7f6f0d3a3700 1 -- 172.21.15.92:0/3415613872 shutdown_connections 2023-10-06T10:45:57.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.047+0000 7f6f0d3a3700 1 -- 172.21.15.92:0/3415613872 wait complete. 2023-10-06T10:45:57.051 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:45:57.061 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2023-10-06T10:45:56.183005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":206,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2398132,"kb_used_data":11636,"kb_used_omap":0,"kb_used_meta":2386432,"kb_avail":752576588,"statfs":{"total":773094113280,"available":770638426112,"internally_reserved":0,"allocated":11915264,"data_stored":6415006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2443706368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-120150,"num_objects":-68,"num_object_clones":0,"num_object_copies":-136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-68,"num_whiteouts":0,"num_read":-1,"num_read_kb":0,"num_write":-246,"num_write_kb":-199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001639"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.597891+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:45:51.597891+0000","last_peered":"2023-10-06T10:45:51.597891+0000","last_clean":"2023-10-06T10:45:51.597891+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:45:51.597891+0000","last_undegraded":"2023-10-06T10:45:51.597891+0000","last_fullsized":"2023-10-06T10:45:51.597891+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.695321+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:45:51.695321+0000","last_peered":"2023-10-06T10:45:51.695321+0000","last_clean":"2023-10-06T10:45:51.695321+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:45:51.695321+0000","last_undegraded":"2023-10-06T10:45:51.695321+0000","last_fullsized":"2023-10-06T10:45:51.695321+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":88,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.695604+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:45:51.695604+0000","last_peered":"2023-10-06T10:45:51.695604+0000","last_clean":"2023-10-06T10:45:51.695604+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:45:51.695604+0000","last_undegraded":"2023-10-06T10:45:51.695604+0000","last_fullsized":"2023-10-06T10:45:51.695604+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":230,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.694648+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:45:51.694648+0000","last_peered":"2023-10-06T10:45:51.694648+0000","last_clean":"2023-10-06T10:45:51.694648+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:45:51.694648+0000","last_undegraded":"2023-10-06T10:45:51.694648+0000","last_fullsized":"2023-10-06T10:45:51.694648+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":80,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.694820+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:45:51.694820+0000","last_peered":"2023-10-06T10:45:51.694820+0000","last_clean":"2023-10-06T10:45:51.694820+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:45:51.694820+0000","last_undegraded":"2023-10-06T10:45:51.694820+0000","last_fullsized":"2023-10-06T10:45:51.694820+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.694804+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:45:51.694804+0000","last_peered":"2023-10-06T10:45:51.694804+0000","last_clean":"2023-10-06T10:45:51.694804+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:45:51.694804+0000","last_undegraded":"2023-10-06T10:45:51.694804+0000","last_fullsized":"2023-10-06T10:45:51.694804+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.597840+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:45:51.597840+0000","last_peered":"2023-10-06T10:45:51.597840+0000","last_clean":"2023-10-06T10:45:51.597840+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:45:51.597840+0000","last_undegraded":"2023-10-06T10:45:51.597840+0000","last_fullsized":"2023-10-06T10:45:51.597840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.695597+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:45:51.695597+0000","last_peered":"2023-10-06T10:45:51.695597+0000","last_clean":"2023-10-06T10:45:51.695597+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:45:51.695597+0000","last_undegraded":"2023-10-06T10:45:51.695597+0000","last_fullsized":"2023-10-06T10:45:51.695597+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.695686+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:45:51.695686+0000","last_peered":"2023-10-06T10:45:51.695686+0000","last_clean":"2023-10-06T10:45:51.695686+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:45:51.695686+0000","last_undegraded":"2023-10-06T10:45:51.695686+0000","last_fullsized":"2023-10-06T10:45:51.695686+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574946,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299148,"kb_used_data":1860,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072692,"statfs":{"total":96636764160,"available":96330436608,"internally_reserved":0,"allocated":1904640,"data_stored":1224125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52300000000000002,"15min":0.49399999999999999},"min":{"1min":0.434,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.78400000000000003,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53700000000000003,"15min":0.52100000000000002},"min":{"1min":0.38900000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66100000000000003,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":0.60899999999999999}]},{"osd":1,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.495,"15min":0.47099999999999997},"min":{"1min":0.39700000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.64700000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.623},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54700000000000004,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.748,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.64700000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56999999999999995,"15min":0.52500000000000002},"min":{"1min":0.40999999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69899999999999995,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51800000000000002,"15min":0.501},"min":{"1min":0.35599999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.67600000000000005,"5min":0.747,"15min":0.79800000000000004},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56200000000000006,"15min":0.52200000000000002},"min":{"1min":0.36699999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76200000000000001,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.42699999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.66500000000000004}]},{"osd":4,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.44,"15min":0.46300000000000002},"min":{"1min":0.30599999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70099999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.48999999999999999,"15min":0.501},"min":{"1min":0.311,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.75700000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.78300000000000003}]},{"osd":5,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.48399999999999999,"15min":0.48499999999999999},"min":{"1min":0.36099999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.77700000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.34000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.746,"5min":1.577,"15min":1.577},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.55200000000000005,"15min":0.56699999999999995},"min":{"1min":0.438,"5min":0.308,"15min":0.308},"max":{"1min":0.77300000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.57899999999999996,"15min":0.57799999999999996},"min":{"1min":0.37,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80700000000000005,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.80400000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574944,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298564,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073276,"statfs":{"total":96636764160,"available":96331034624,"internally_reserved":0,"allocated":1306624,"data_stored":631908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51200000000000001,"15min":0.497},"min":{"1min":0.38100000000000001,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.77000000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51600000000000001,"15min":0.46100000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.70099999999999996,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.52400000000000002}]},{"osd":1,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51400000000000001,"15min":0.46500000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.28000000000000003},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.80500000000000005},"last":0.622},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.495},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.751,"15min":0.751},"last":0.53900000000000003}]},{"osd":2,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51800000000000002,"15min":0.48699999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54200000000000004,"15min":0.49199999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.73899999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.64700000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55200000000000005,"15min":0.51100000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.76900000000000002,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55000000000000004,"15min":0.52400000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79400000000000004},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.5,"15min":0.48999999999999999},"min":{"1min":0.32300000000000001,"5min":0.308,"15min":0.26700000000000002},"max":{"1min":0.72499999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.40100000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51100000000000001,"15min":0.495},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.75700000000000001,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55400000000000005,"15min":0.54300000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.308},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.42599999999999999}]},{"osd":7,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53200000000000003,"15min":0.52100000000000002},"min":{"1min":0.34899999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.755,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57899999999999996,"15min":0.55200000000000005},"min":{"1min":0.38200000000000001,"5min":0.33000000000000002,"15min":0.32300000000000001},"max":{"1min":0.79100000000000004,"5min":0.879,"15min":0.879},"last":0.46700000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303288,"kb_used_data":1904,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068552,"statfs":{"total":96636764160,"available":96326197248,"internally_reserved":0,"allocated":1949696,"data_stored":1246951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32500000000000001,"15min":0.32700000000000001},"min":{"1min":0.217,"5min":0.188,"15min":0.188},"max":{"1min":0.42599999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.246,"5min":0.189,"15min":0.189},"max":{"1min":0.53200000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.32700000000000001,"15min":0.32200000000000001},"min":{"1min":0.22900000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.48099999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.373,"15min":0.36699999999999999},"min":{"1min":0.28599999999999998,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.54000000000000004,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.503}]},{"osd":3,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39200000000000002,"15min":0.38200000000000001},"min":{"1min":0.30499999999999999,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.54800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.371,"5min":0.38,"15min":0.377},"min":{"1min":0.26100000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.45800000000000002,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.51900000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38200000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.746,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.45300000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.753,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.59399999999999997}]},{"osd":5,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.77500000000000002,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56999999999999995,"15min":0.55200000000000005},"min":{"1min":0.41099999999999998,"5min":0.36799999999999999,"15min":0.35699999999999998},"max":{"1min":0.76800000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.61899999999999999}]},{"osd":6,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.39100000000000001,"5min":0.38700000000000001,"15min":0.36699999999999999},"max":{"1min":0.81299999999999994,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.46500000000000002,"5min":0.40000000000000002,"15min":0.39800000000000002},"max":{"1min":0.74399999999999999,"5min":0.93500000000000005,"15min":0.94099999999999995},"last":0.65200000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.80200000000000005,"5min":0.996,"15min":0.996},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.41299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.73399999999999999,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.68600000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574946,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298696,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073144,"statfs":{"total":96636764160,"available":96330899456,"internally_reserved":0,"allocated":1441792,"data_stored":733555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.317,"15min":0.31900000000000001},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.41399999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.251},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.37,"15min":0.35199999999999998},"min":{"1min":0.27000000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.57299999999999995,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.312}]},{"osd":2,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.314,"15min":0.309},"min":{"1min":0.22600000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.46999999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34799999999999998,"15min":0.33400000000000002},"min":{"1min":0.247,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.61399999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.28100000000000003}]},{"osd":3,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34399999999999997,"15min":0.34499999999999997},"min":{"1min":0.24299999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.51400000000000001,"5min":0.622,"15min":0.622},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36099999999999999,"15min":0.35399999999999998},"min":{"1min":0.28399999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54200000000000004,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.29799999999999999}]},{"osd":4,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54500000000000004,"15min":0.55600000000000005},"min":{"1min":0.435,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.74099999999999999,"5min":0.78900000000000003,"15min":0.83299999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.54100000000000004},"min":{"1min":0.42699999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69699999999999995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.35199999999999998}]},{"osd":5,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.435,"5min":0.377,"15min":0.36899999999999999},"max":{"1min":0.76400000000000001,"5min":0.85699999999999998,"15min":1.0069999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70499999999999996,"5min":0.77500000000000002,"15min":0.94499999999999995},"last":0.435}]},{"osd":6,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56200000000000006,"15min":0.57899999999999996},"min":{"1min":0.40899999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.68600000000000005,"5min":1.046,"15min":1.046},"last":0.376},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.68899999999999995,"5min":0.80000000000000004,"15min":1.075},"last":0.55700000000000005}]},{"osd":7,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.40600000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73399999999999999,"5min":0.83499999999999996,"15min":0.96099999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.60399999999999998},"min":{"1min":0.46100000000000002,"5min":0.41899999999999998,"15min":0.41499999999999998},"max":{"1min":0.76200000000000001,"5min":0.77200000000000002,"15min":0.90100000000000002},"last":0.497}]}]},{"osd":2,"up_from":13,"seq":55834574944,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298580,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073260,"statfs":{"total":96636764160,"available":96331018240,"internally_reserved":0,"allocated":1323008,"data_stored":633551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.315,"15min":0.312},"min":{"1min":0.22600000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.45700000000000002,"5min":0.496,"15min":0.57999999999999996},"last":0.249},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.35699999999999998,"15min":0.33100000000000002},"min":{"1min":0.26600000000000001,"5min":0.20200000000000001,"15min":0.20100000000000001},"max":{"1min":0.51700000000000002,"5min":0.55600000000000005,"15min":0.622},"last":0.35399999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33000000000000002},"min":{"1min":0.253,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.47299999999999998,"5min":0.56599999999999995,"15min":0.60499999999999998},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.376,"15min":0.36899999999999999},"min":{"1min":0.29599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47999999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.27400000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.38700000000000001,"15min":0.35999999999999999},"min":{"1min":0.309,"5min":0.25700000000000001,"15min":0.20499999999999999},"max":{"1min":0.504,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.378,"15min":0.36299999999999999},"min":{"1min":0.27500000000000002,"5min":0.25,"15min":0.22800000000000001},"max":{"1min":0.50900000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.314}]},{"osd":4,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.40799999999999997,"5min":0.33400000000000002,"15min":0.315},"max":{"1min":0.66300000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.45700000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75,"5min":0.82399999999999995,"15min":0.84899999999999998},"last":0.499}]},{"osd":5,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.51000000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72199999999999998,"5min":0.83799999999999997,"15min":0.91200000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.41699999999999998,"5min":0.36599999999999999,"15min":0.33800000000000002},"max":{"1min":0.69499999999999995,"5min":0.81799999999999995,"15min":0.83399999999999996},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59299999999999997,"15min":0.61399999999999999},"min":{"1min":0.42999999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.70899999999999996,"5min":0.88700000000000001,"15min":0.89100000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61299999999999999,"15min":0.62},"min":{"1min":0.46800000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.748,"5min":0.80400000000000005,"15min":0.93799999999999994},"last":0.52100000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57799999999999996,"15min":0.60299999999999998},"min":{"1min":0.433,"5min":0.38500000000000001,"15min":0.36799999999999999},"max":{"1min":0.64300000000000002,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60799999999999998,"15min":0.621},"min":{"1min":0.438,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.70999999999999996,"5min":0.81100000000000005,"15min":0.874},"last":0.56699999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574944,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298576,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073264,"statfs":{"total":96636764160,"available":96331022336,"internally_reserved":0,"allocated":1318912,"data_stored":640063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.31,"15min":0.40500000000000003},"min":{"1min":0.214,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.436,"5min":0.52700000000000002,"15min":3.3210000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.33000000000000002,"15min":0.35399999999999998},"min":{"1min":0.22500000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":1.218},"last":0.45900000000000002}]},{"osd":1,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.33300000000000002,"15min":0.33800000000000002},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.20100000000000001},"max":{"1min":0.496,"5min":0.63600000000000001,"15min":0.86299999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35199999999999998,"15min":0.44500000000000001},"min":{"1min":0.26900000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.47299999999999998,"5min":0.66000000000000003,"15min":3.359},"last":0.42999999999999999}]},{"osd":2,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.35299999999999998,"15min":0.374},"min":{"1min":0.28100000000000003,"5min":0.255,"15min":0.215},"max":{"1min":0.51700000000000002,"5min":0.71099999999999997,"15min":1.1100000000000001},"last":0.498},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.56699999999999995},"min":{"1min":0.28100000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.54200000000000004,"5min":0.59899999999999998,"15min":7.069},"last":0.44500000000000001}]},{"osd":4,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54800000000000004,"15min":0.57799999999999996},"min":{"1min":0.43099999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73799999999999999,"5min":0.84699999999999998,"15min":1.165},"last":0.622},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57799999999999996,"15min":0.59099999999999997},"min":{"1min":0.441,"5min":0.378,"15min":0.375},"max":{"1min":0.73599999999999999,"5min":1.02,"15min":1.02},"last":0.67500000000000004}]},{"osd":5,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.66500000000000004},"min":{"1min":0.41699999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72699999999999998,"5min":0.84499999999999997,"15min":3.4140000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57199999999999995,"15min":0.59799999999999998},"min":{"1min":0.39800000000000002,"5min":0.39200000000000002,"15min":0.34599999999999997},"max":{"1min":0.71299999999999997,"5min":0.83299999999999996,"15min":1.1930000000000001},"last":0.66200000000000003}]},{"osd":6,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59599999999999997,"15min":0.61099999999999999},"min":{"1min":0.41899999999999998,"5min":0.41699999999999998,"15min":0.378},"max":{"1min":0.79900000000000004,"5min":1.054,"15min":1.054},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60899999999999999,"15min":0.69499999999999995},"min":{"1min":0.45100000000000001,"5min":0.42099999999999999,"15min":0.41999999999999998},"max":{"1min":0.754,"5min":0.876,"15min":3.3879999999999999},"last":0.70699999999999996}]},{"osd":7,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58299999999999996,"15min":0.60199999999999998},"min":{"1min":0.42999999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.70499999999999996,"5min":0.86699999999999999,"15min":1.137},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59199999999999997,"15min":0.60799999999999998},"min":{"1min":0.42999999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":0.97899999999999998,"15min":1.014},"last":0.60199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574944,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302660,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069180,"statfs":{"total":96636764160,"available":96326840320,"internally_reserved":0,"allocated":1306624,"data_stored":633203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.30299999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.625,"5min":0.74299999999999999,"15min":0.76800000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.51900000000000002,"15min":0.504},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.28299999999999997},"max":{"1min":0.67100000000000004,"5min":0.73099999999999998,"15min":0.77400000000000002},"last":0.61499999999999999}]},{"osd":1,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.41999999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67600000000000005,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51800000000000002,"15min":0.53100000000000003},"min":{"1min":0.29499999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69499999999999995,"5min":0.71799999999999997,"15min":0.82899999999999996},"last":0.54600000000000004}]},{"osd":2,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52600000000000002,"15min":0.51400000000000001},"min":{"1min":0.38100000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67800000000000005,"5min":0.77500000000000002,"15min":0.81000000000000005},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.52800000000000002,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.372,"15min":0.315},"max":{"1min":0.60799999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.42699999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55100000000000005,"15min":0.52600000000000002},"min":{"1min":0.41399999999999998,"5min":0.36599999999999999,"15min":0.35499999999999998},"max":{"1min":0.72799999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54700000000000004,"15min":0.53800000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.751,"5min":0.78800000000000003,"15min":0.79500000000000004},"last":0.66300000000000003}]},{"osd":5,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.44800000000000001,"15min":0.48599999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72399999999999998,"5min":0.77800000000000002,"15min":0.86899999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48199999999999998,"15min":0.49399999999999999},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.255},"max":{"1min":0.70699999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.751}]},{"osd":6,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51000000000000001,"15min":0.53200000000000003},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.312},"max":{"1min":0.78300000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72399999999999998,"5min":0.88,"15min":0.91200000000000003},"last":0.70499999999999996}]},{"osd":7,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.33200000000000002,"15min":0.28999999999999998},"max":{"1min":0.74199999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.58099999999999996},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79900000000000004,"5min":0.81299999999999994,"15min":0.85699999999999998},"last":0.77700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298620,"kb_used_data":1332,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073220,"statfs":{"total":96636764160,"available":96330977280,"internally_reserved":0,"allocated":1363968,"data_stored":671650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.50700000000000001,"15min":0.47999999999999998},"min":{"1min":0.34999999999999998,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.67500000000000004,"5min":0.88,"15min":0.88},"last":0.434},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.503,"15min":0.52500000000000002},"min":{"1min":0.39300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.73699999999999999,"5min":0.86499999999999999,"15min":1.2549999999999999},"last":0.44900000000000001}]},{"osd":1,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.495,"15min":0.48999999999999999},"min":{"1min":0.36899999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.81599999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51400000000000001,"15min":0.51700000000000002},"min":{"1min":0.38500000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.628,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.38500000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52700000000000002,"15min":0.51000000000000001},"min":{"1min":0.33000000000000002,"5min":0.307,"15min":0.30399999999999999},"max":{"1min":0.76500000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54000000000000004,"15min":0.55800000000000005},"min":{"1min":0.47599999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77700000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.36299999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.56599999999999995},"min":{"1min":0.442,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69399999999999995,"5min":0.79900000000000004,"15min":0.95099999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51500000000000001,"15min":0.54700000000000004},"min":{"1min":0.42399999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70499999999999996,"5min":0.79100000000000004,"15min":1.3999999999999999},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.28899999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.504,"15min":0.50700000000000001},"min":{"1min":0.33500000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.80700000000000005},"last":0.52200000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.498,"15min":0.51500000000000001},"min":{"1min":0.314,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.85799999999999998},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.437,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.50700000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55600000000000005,"15min":0.57299999999999995},"min":{"1min":0.41999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.83299999999999996},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.34699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.55700000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:45:57.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:45:57.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.176+0000 7f272e76b700 1 Processor -- start 2023-10-06T10:45:57.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.176+0000 7f272e76b700 1 -- start start 2023-10-06T10:45:57.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.177+0000 7f272e76b700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f27280b25e0 0x7f27280b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:57.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.177+0000 7f272e76b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2728159830 0x7f27280cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:57.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.177+0000 7f272e76b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27280cbfa0 con 0x7f27280b29b0 2023-10-06T10:45:57.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.177+0000 7f272e76b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27280cb390 con 0x7f2728159830 2023-10-06T10:45:57.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.177+0000 7f272e76b700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27280cb670 con 0x7f27280b25e0 2023-10-06T10:45:57.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.177+0000 7f272cd08700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2728159830 0x7f27280cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:57.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.177+0000 7f27277fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f27280b29b0 0x7f2728159120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35380/0 (socket says 172.21.15.92:35380) 2023-10-06T10:45:57.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.177+0000 7f272cd08700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2728159830 0x7f27280cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37940/0 (socket says 172.21.15.92:37940) 2023-10-06T10:45:57.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.177+0000 7f27277fe700 1 -- 172.21.15.92:0/2318856529 learned_addr learned my addr 172.21.15.92:0/2318856529 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:57.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.178+0000 7f2727fff700 1 --2- 172.21.15.92:0/2318856529 >> v2:172.21.15.124:3300/0 conn(0x7f27280b25e0 0x7f27280b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:57.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.178+0000 7f272cd08700 1 -- 172.21.15.92:0/2318856529 >> v2:172.21.15.124:3300/0 conn(0x7f27280b25e0 msgr2=0x7f27280b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:57.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.178+0000 7f272cd08700 1 --2- 172.21.15.92:0/2318856529 >> v2:172.21.15.124:3300/0 conn(0x7f27280b25e0 0x7f27280b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.178+0000 7f272cd08700 1 -- 172.21.15.92:0/2318856529 >> v1:172.21.15.92:6789/0 conn(0x7f27280b29b0 legacy=0x7f2728159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:57.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.178+0000 7f272cd08700 1 -- 172.21.15.92:0/2318856529 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27280cb950 con 0x7f2728159830 2023-10-06T10:45:57.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.178+0000 7f2726ffd700 1 -- 172.21.15.92:0/2318856529 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663382870 0 0) 0x7f27280cbfa0 con 0x7f27280b29b0 2023-10-06T10:45:57.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.178+0000 7f272cd08700 1 --2- 172.21.15.92:0/2318856529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2728159830 0x7f27280cc430 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f27280d0420 tx=0x7f2720009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8c2359f49c0746 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:57.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.178+0000 7f2726ffd700 1 -- 172.21.15.92:0/2318856529 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272000e070 con 0x7f2728159830 2023-10-06T10:45:57.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.178+0000 7f2726ffd700 1 -- 172.21.15.92:0/2318856529 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f272000bb60 con 0x7f2728159830 2023-10-06T10:45:57.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.178+0000 7f2726ffd700 1 -- 172.21.15.92:0/2318856529 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2720012430 con 0x7f2728159830 2023-10-06T10:45:57.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.179+0000 7f272e76b700 1 -- 172.21.15.92:0/2318856529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2728159830 msgr2=0x7f27280cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:57.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.179+0000 7f272e76b700 1 --2- 172.21.15.92:0/2318856529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2728159830 0x7f27280cc430 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f27280d0420 tx=0x7f2720009780 comp rx=0 tx=0).stop 2023-10-06T10:45:57.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.179+0000 7f272e76b700 1 -- 172.21.15.92:0/2318856529 shutdown_connections 2023-10-06T10:45:57.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.179+0000 7f272e76b700 1 --2- 172.21.15.92:0/2318856529 >> v2:172.21.15.124:3300/0 conn(0x7f27280b25e0 0x7f27280b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.179+0000 7f272e76b700 1 --2- 172.21.15.92:0/2318856529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2728159830 0x7f27280cc430 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.179+0000 7f272e76b700 1 -- 172.21.15.92:0/2318856529 >> 172.21.15.92:0/2318856529 conn(0x7f272806ece0 msgr2=0x7f272806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:57.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.179+0000 7f272e76b700 1 -- 172.21.15.92:0/2318856529 shutdown_connections 2023-10-06T10:45:57.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.179+0000 7f272e76b700 1 -- 172.21.15.92:0/2318856529 wait complete. 2023-10-06T10:45:57.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.180+0000 7f272e76b700 1 Processor -- start 2023-10-06T10:45:57.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.180+0000 7f272e76b700 1 -- start start 2023-10-06T10:45:57.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f272e76b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27280b25e0 0x7f272815e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:57.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f272e76b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27280b29b0 0x7f27280c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:57.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f272e76b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2728159830 0x7f272816d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:57.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f272e76b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27280c6fa0 con 0x7f27280b29b0 2023-10-06T10:45:57.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f272e76b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272815a500 con 0x7f27280b25e0 2023-10-06T10:45:57.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f272e76b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27280c69e0 con 0x7f2728159830 2023-10-06T10:45:57.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f27277fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27280b29b0 0x7f27280c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:57.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f27277fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27280b29b0 0x7f27280c3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37942/0 (socket says 172.21.15.92:37942) 2023-10-06T10:45:57.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f27277fe700 1 -- 172.21.15.92:0/971275599 learned_addr learned my addr 172.21.15.92:0/971275599 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:57.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f2727fff700 1 --2- 172.21.15.92:0/971275599 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27280b25e0 0x7f272815e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:57.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f272cd08700 1 --2- 172.21.15.92:0/971275599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2728159830 0x7f272816d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:57.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f27277fe700 1 -- 172.21.15.92:0/971275599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2728159830 msgr2=0x7f272816d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:57.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f27277fe700 1 --2- 172.21.15.92:0/971275599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2728159830 0x7f272816d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f27277fe700 1 -- 172.21.15.92:0/971275599 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27280b25e0 msgr2=0x7f272815e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:57.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f27277fe700 1 --2- 172.21.15.92:0/971275599 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27280b25e0 0x7f272815e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f27277fe700 1 -- 172.21.15.92:0/971275599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2720009a20 con 0x7f27280b29b0 2023-10-06T10:45:57.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f272cd08700 1 --2- 172.21.15.92:0/971275599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2728159830 0x7f272816d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:57.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f27277fe700 1 --2- 172.21.15.92:0/971275599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27280b29b0 0x7f27280c3f20 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f271c00c840 tx=0x7f271c007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:57.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f270ffff700 1 -- 172.21.15.92:0/971275599 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c005c00 con 0x7f27280b29b0 2023-10-06T10:45:57.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f270ffff700 1 -- 172.21.15.92:0/971275599 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271c005d60 con 0x7f27280b29b0 2023-10-06T10:45:57.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f272e76b700 1 -- 172.21.15.92:0/971275599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27280c6cc0 con 0x7f27280b29b0 2023-10-06T10:45:57.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f270ffff700 1 -- 172.21.15.92:0/971275599 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c007320 con 0x7f27280b29b0 2023-10-06T10:45:57.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.181+0000 7f272e76b700 1 -- 172.21.15.92:0/971275599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27280c4460 con 0x7f27280b29b0 2023-10-06T10:45:57.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.182+0000 7f272e76b700 1 -- 172.21.15.92:0/971275599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2728064a90 con 0x7f27280b29b0 2023-10-06T10:45:57.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.183+0000 7f270ffff700 1 -- 172.21.15.92:0/971275599 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f271c01b3b0 con 0x7f27280b29b0 2023-10-06T10:45:57.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.183+0000 7f270ffff700 1 --2- 172.21.15.92:0/971275599 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2714067000 0x7f27140694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:57.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.183+0000 7f270ffff700 1 -- 172.21.15.92:0/971275599 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f271c083a90 con 0x7f27280b29b0 2023-10-06T10:45:57.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.183+0000 7f2727fff700 1 --2- 172.21.15.92:0/971275599 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2714067000 0x7f27140694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:57.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.183+0000 7f2727fff700 1 --2- 172.21.15.92:0/971275599 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2714067000 0x7f27140694b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2718009da0 tx=0x7f2718009120 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:57.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.187+0000 7f270ffff700 1 -- 172.21.15.92:0/971275599 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f271c050d30 con 0x7f27280b29b0 2023-10-06T10:45:57.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.365+0000 7f272e76b700 1 -- 172.21.15.92:0/971275599 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f272815a640 con 0x7f2714067000 2023-10-06T10:45:57.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.368+0000 7f270ffff700 1 -- 172.21.15.92:0/971275599 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73209 (secure 0 0 0) 0x7f272815a640 con 0x7f2714067000 2023-10-06T10:45:57.371 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:57.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.370+0000 7f272e76b700 1 -- 172.21.15.92:0/971275599 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2714067000 msgr2=0x7f27140694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:57.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.370+0000 7f272e76b700 1 --2- 172.21.15.92:0/971275599 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2714067000 0x7f27140694b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2718009da0 tx=0x7f2718009120 comp rx=0 tx=0).stop 2023-10-06T10:45:57.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.370+0000 7f272e76b700 1 -- 172.21.15.92:0/971275599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27280b29b0 msgr2=0x7f27280c3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:57.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.370+0000 7f272e76b700 1 --2- 172.21.15.92:0/971275599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27280b29b0 0x7f27280c3f20 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f271c00c840 tx=0x7f271c007d90 comp rx=0 tx=0).stop 2023-10-06T10:45:57.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.370+0000 7f272e76b700 1 -- 172.21.15.92:0/971275599 shutdown_connections 2023-10-06T10:45:57.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.370+0000 7f272e76b700 1 --2- 172.21.15.92:0/971275599 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f2714067000 0x7f27140694b0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.370+0000 7f272e76b700 1 --2- 172.21.15.92:0/971275599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2728159830 0x7f272816d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.370+0000 7f272e76b700 1 --2- 172.21.15.92:0/971275599 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27280b25e0 0x7f272815e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.370+0000 7f272e76b700 1 --2- 172.21.15.92:0/971275599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27280b29b0 0x7f27280c3f20 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.370+0000 7f272e76b700 1 -- 172.21.15.92:0/971275599 >> 172.21.15.92:0/971275599 conn(0x7f272806ece0 msgr2=0x7f2728154160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:57.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.370+0000 7f272e76b700 1 -- 172.21.15.92:0/971275599 shutdown_connections 2023-10-06T10:45:57.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.370+0000 7f272e76b700 1 -- 172.21.15.92:0/971275599 wait complete. 2023-10-06T10:45:57.373 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:45:57.384 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2023-10-06T10:45:56.183005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":206,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2398132,"kb_used_data":11636,"kb_used_omap":0,"kb_used_meta":2386432,"kb_avail":752576588,"statfs":{"total":773094113280,"available":770638426112,"internally_reserved":0,"allocated":11915264,"data_stored":6415006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2443706368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-120150,"num_objects":-68,"num_object_clones":0,"num_object_copies":-136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-68,"num_whiteouts":0,"num_read":-1,"num_read_kb":0,"num_write":-246,"num_write_kb":-199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001639"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.597891+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:45:51.597891+0000","last_peered":"2023-10-06T10:45:51.597891+0000","last_clean":"2023-10-06T10:45:51.597891+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:45:51.597891+0000","last_undegraded":"2023-10-06T10:45:51.597891+0000","last_fullsized":"2023-10-06T10:45:51.597891+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.695321+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:45:51.695321+0000","last_peered":"2023-10-06T10:45:51.695321+0000","last_clean":"2023-10-06T10:45:51.695321+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:45:51.695321+0000","last_undegraded":"2023-10-06T10:45:51.695321+0000","last_fullsized":"2023-10-06T10:45:51.695321+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":88,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.695604+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:45:51.695604+0000","last_peered":"2023-10-06T10:45:51.695604+0000","last_clean":"2023-10-06T10:45:51.695604+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:45:51.695604+0000","last_undegraded":"2023-10-06T10:45:51.695604+0000","last_fullsized":"2023-10-06T10:45:51.695604+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":230,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.694648+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:45:51.694648+0000","last_peered":"2023-10-06T10:45:51.694648+0000","last_clean":"2023-10-06T10:45:51.694648+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:45:51.694648+0000","last_undegraded":"2023-10-06T10:45:51.694648+0000","last_fullsized":"2023-10-06T10:45:51.694648+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":80,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.694820+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:45:51.694820+0000","last_peered":"2023-10-06T10:45:51.694820+0000","last_clean":"2023-10-06T10:45:51.694820+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:45:51.694820+0000","last_undegraded":"2023-10-06T10:45:51.694820+0000","last_fullsized":"2023-10-06T10:45:51.694820+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.694804+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:45:51.694804+0000","last_peered":"2023-10-06T10:45:51.694804+0000","last_clean":"2023-10-06T10:45:51.694804+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:45:51.694804+0000","last_undegraded":"2023-10-06T10:45:51.694804+0000","last_fullsized":"2023-10-06T10:45:51.694804+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.597840+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:45:51.597840+0000","last_peered":"2023-10-06T10:45:51.597840+0000","last_clean":"2023-10-06T10:45:51.597840+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:45:51.597840+0000","last_undegraded":"2023-10-06T10:45:51.597840+0000","last_fullsized":"2023-10-06T10:45:51.597840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.695597+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:45:51.695597+0000","last_peered":"2023-10-06T10:45:51.695597+0000","last_clean":"2023-10-06T10:45:51.695597+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:45:51.695597+0000","last_undegraded":"2023-10-06T10:45:51.695597+0000","last_fullsized":"2023-10-06T10:45:51.695597+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.695686+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:45:51.695686+0000","last_peered":"2023-10-06T10:45:51.695686+0000","last_clean":"2023-10-06T10:45:51.695686+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:45:51.695686+0000","last_undegraded":"2023-10-06T10:45:51.695686+0000","last_fullsized":"2023-10-06T10:45:51.695686+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574946,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299148,"kb_used_data":1860,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072692,"statfs":{"total":96636764160,"available":96330436608,"internally_reserved":0,"allocated":1904640,"data_stored":1224125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52300000000000002,"15min":0.49399999999999999},"min":{"1min":0.434,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.78400000000000003,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53700000000000003,"15min":0.52100000000000002},"min":{"1min":0.38900000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66100000000000003,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":0.60899999999999999}]},{"osd":1,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.495,"15min":0.47099999999999997},"min":{"1min":0.39700000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.64700000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.623},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54700000000000004,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.748,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.64700000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56999999999999995,"15min":0.52500000000000002},"min":{"1min":0.40999999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69899999999999995,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51800000000000002,"15min":0.501},"min":{"1min":0.35599999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.67600000000000005,"5min":0.747,"15min":0.79800000000000004},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56200000000000006,"15min":0.52200000000000002},"min":{"1min":0.36699999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76200000000000001,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.42699999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.66500000000000004}]},{"osd":4,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.44,"15min":0.46300000000000002},"min":{"1min":0.30599999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70099999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.48999999999999999,"15min":0.501},"min":{"1min":0.311,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.75700000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.78300000000000003}]},{"osd":5,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.48399999999999999,"15min":0.48499999999999999},"min":{"1min":0.36099999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.77700000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.34000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.746,"5min":1.577,"15min":1.577},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.55200000000000005,"15min":0.56699999999999995},"min":{"1min":0.438,"5min":0.308,"15min":0.308},"max":{"1min":0.77300000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.57899999999999996,"15min":0.57799999999999996},"min":{"1min":0.37,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80700000000000005,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.80400000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574944,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298564,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073276,"statfs":{"total":96636764160,"available":96331034624,"internally_reserved":0,"allocated":1306624,"data_stored":631908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51200000000000001,"15min":0.497},"min":{"1min":0.38100000000000001,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.77000000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51600000000000001,"15min":0.46100000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.70099999999999996,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.52400000000000002}]},{"osd":1,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51400000000000001,"15min":0.46500000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.28000000000000003},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.80500000000000005},"last":0.622},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.495},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.751,"15min":0.751},"last":0.53900000000000003}]},{"osd":2,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51800000000000002,"15min":0.48699999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54200000000000004,"15min":0.49199999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.73899999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.64700000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55200000000000005,"15min":0.51100000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.76900000000000002,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55000000000000004,"15min":0.52400000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79400000000000004},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.5,"15min":0.48999999999999999},"min":{"1min":0.32300000000000001,"5min":0.308,"15min":0.26700000000000002},"max":{"1min":0.72499999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.40100000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51100000000000001,"15min":0.495},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.75700000000000001,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55400000000000005,"15min":0.54300000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.308},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.42599999999999999}]},{"osd":7,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53200000000000003,"15min":0.52100000000000002},"min":{"1min":0.34899999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.755,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57899999999999996,"15min":0.55200000000000005},"min":{"1min":0.38200000000000001,"5min":0.33000000000000002,"15min":0.32300000000000001},"max":{"1min":0.79100000000000004,"5min":0.879,"15min":0.879},"last":0.46700000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303288,"kb_used_data":1904,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068552,"statfs":{"total":96636764160,"available":96326197248,"internally_reserved":0,"allocated":1949696,"data_stored":1246951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32500000000000001,"15min":0.32700000000000001},"min":{"1min":0.217,"5min":0.188,"15min":0.188},"max":{"1min":0.42599999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.246,"5min":0.189,"15min":0.189},"max":{"1min":0.53200000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.32700000000000001,"15min":0.32200000000000001},"min":{"1min":0.22900000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.48099999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.373,"15min":0.36699999999999999},"min":{"1min":0.28599999999999998,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.54000000000000004,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.503}]},{"osd":3,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39200000000000002,"15min":0.38200000000000001},"min":{"1min":0.30499999999999999,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.54800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.371,"5min":0.38,"15min":0.377},"min":{"1min":0.26100000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.45800000000000002,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.51900000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38200000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.746,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.45300000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.753,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.59399999999999997}]},{"osd":5,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.77500000000000002,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56999999999999995,"15min":0.55200000000000005},"min":{"1min":0.41099999999999998,"5min":0.36799999999999999,"15min":0.35699999999999998},"max":{"1min":0.76800000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.61899999999999999}]},{"osd":6,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.39100000000000001,"5min":0.38700000000000001,"15min":0.36699999999999999},"max":{"1min":0.81299999999999994,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.46500000000000002,"5min":0.40000000000000002,"15min":0.39800000000000002},"max":{"1min":0.74399999999999999,"5min":0.93500000000000005,"15min":0.94099999999999995},"last":0.65200000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.80200000000000005,"5min":0.996,"15min":0.996},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.41299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.73399999999999999,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.68600000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574946,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298696,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073144,"statfs":{"total":96636764160,"available":96330899456,"internally_reserved":0,"allocated":1441792,"data_stored":733555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.317,"15min":0.31900000000000001},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.41399999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.251},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.37,"15min":0.35199999999999998},"min":{"1min":0.27000000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.57299999999999995,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.312}]},{"osd":2,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.314,"15min":0.309},"min":{"1min":0.22600000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.46999999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34799999999999998,"15min":0.33400000000000002},"min":{"1min":0.247,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.61399999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.28100000000000003}]},{"osd":3,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34399999999999997,"15min":0.34499999999999997},"min":{"1min":0.24299999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.51400000000000001,"5min":0.622,"15min":0.622},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36099999999999999,"15min":0.35399999999999998},"min":{"1min":0.28399999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54200000000000004,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.29799999999999999}]},{"osd":4,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54500000000000004,"15min":0.55600000000000005},"min":{"1min":0.435,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.74099999999999999,"5min":0.78900000000000003,"15min":0.83299999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.54100000000000004},"min":{"1min":0.42699999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69699999999999995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.35199999999999998}]},{"osd":5,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.435,"5min":0.377,"15min":0.36899999999999999},"max":{"1min":0.76400000000000001,"5min":0.85699999999999998,"15min":1.0069999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70499999999999996,"5min":0.77500000000000002,"15min":0.94499999999999995},"last":0.435}]},{"osd":6,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56200000000000006,"15min":0.57899999999999996},"min":{"1min":0.40899999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.68600000000000005,"5min":1.046,"15min":1.046},"last":0.376},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.68899999999999995,"5min":0.80000000000000004,"15min":1.075},"last":0.55700000000000005}]},{"osd":7,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.40600000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73399999999999999,"5min":0.83499999999999996,"15min":0.96099999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.60399999999999998},"min":{"1min":0.46100000000000002,"5min":0.41899999999999998,"15min":0.41499999999999998},"max":{"1min":0.76200000000000001,"5min":0.77200000000000002,"15min":0.90100000000000002},"last":0.497}]}]},{"osd":2,"up_from":13,"seq":55834574944,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298580,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073260,"statfs":{"total":96636764160,"available":96331018240,"internally_reserved":0,"allocated":1323008,"data_stored":633551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.315,"15min":0.312},"min":{"1min":0.22600000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.45700000000000002,"5min":0.496,"15min":0.57999999999999996},"last":0.249},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.35699999999999998,"15min":0.33100000000000002},"min":{"1min":0.26600000000000001,"5min":0.20200000000000001,"15min":0.20100000000000001},"max":{"1min":0.51700000000000002,"5min":0.55600000000000005,"15min":0.622},"last":0.35399999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33000000000000002},"min":{"1min":0.253,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.47299999999999998,"5min":0.56599999999999995,"15min":0.60499999999999998},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.376,"15min":0.36899999999999999},"min":{"1min":0.29599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47999999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.27400000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.38700000000000001,"15min":0.35999999999999999},"min":{"1min":0.309,"5min":0.25700000000000001,"15min":0.20499999999999999},"max":{"1min":0.504,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.378,"15min":0.36299999999999999},"min":{"1min":0.27500000000000002,"5min":0.25,"15min":0.22800000000000001},"max":{"1min":0.50900000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.314}]},{"osd":4,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.40799999999999997,"5min":0.33400000000000002,"15min":0.315},"max":{"1min":0.66300000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.45700000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75,"5min":0.82399999999999995,"15min":0.84899999999999998},"last":0.499}]},{"osd":5,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.51000000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72199999999999998,"5min":0.83799999999999997,"15min":0.91200000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.41699999999999998,"5min":0.36599999999999999,"15min":0.33800000000000002},"max":{"1min":0.69499999999999995,"5min":0.81799999999999995,"15min":0.83399999999999996},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59299999999999997,"15min":0.61399999999999999},"min":{"1min":0.42999999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.70899999999999996,"5min":0.88700000000000001,"15min":0.89100000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61299999999999999,"15min":0.62},"min":{"1min":0.46800000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.748,"5min":0.80400000000000005,"15min":0.93799999999999994},"last":0.52100000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57799999999999996,"15min":0.60299999999999998},"min":{"1min":0.433,"5min":0.38500000000000001,"15min":0.36799999999999999},"max":{"1min":0.64300000000000002,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60799999999999998,"15min":0.621},"min":{"1min":0.438,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.70999999999999996,"5min":0.81100000000000005,"15min":0.874},"last":0.56699999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574944,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298576,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073264,"statfs":{"total":96636764160,"available":96331022336,"internally_reserved":0,"allocated":1318912,"data_stored":640063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.31,"15min":0.40500000000000003},"min":{"1min":0.214,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.436,"5min":0.52700000000000002,"15min":3.3210000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.33000000000000002,"15min":0.35399999999999998},"min":{"1min":0.22500000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":1.218},"last":0.45900000000000002}]},{"osd":1,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.33300000000000002,"15min":0.33800000000000002},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.20100000000000001},"max":{"1min":0.496,"5min":0.63600000000000001,"15min":0.86299999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35199999999999998,"15min":0.44500000000000001},"min":{"1min":0.26900000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.47299999999999998,"5min":0.66000000000000003,"15min":3.359},"last":0.42999999999999999}]},{"osd":2,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.35299999999999998,"15min":0.374},"min":{"1min":0.28100000000000003,"5min":0.255,"15min":0.215},"max":{"1min":0.51700000000000002,"5min":0.71099999999999997,"15min":1.1100000000000001},"last":0.498},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.56699999999999995},"min":{"1min":0.28100000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.54200000000000004,"5min":0.59899999999999998,"15min":7.069},"last":0.44500000000000001}]},{"osd":4,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54800000000000004,"15min":0.57799999999999996},"min":{"1min":0.43099999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73799999999999999,"5min":0.84699999999999998,"15min":1.165},"last":0.622},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57799999999999996,"15min":0.59099999999999997},"min":{"1min":0.441,"5min":0.378,"15min":0.375},"max":{"1min":0.73599999999999999,"5min":1.02,"15min":1.02},"last":0.67500000000000004}]},{"osd":5,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.66500000000000004},"min":{"1min":0.41699999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72699999999999998,"5min":0.84499999999999997,"15min":3.4140000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57199999999999995,"15min":0.59799999999999998},"min":{"1min":0.39800000000000002,"5min":0.39200000000000002,"15min":0.34599999999999997},"max":{"1min":0.71299999999999997,"5min":0.83299999999999996,"15min":1.1930000000000001},"last":0.66200000000000003}]},{"osd":6,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59599999999999997,"15min":0.61099999999999999},"min":{"1min":0.41899999999999998,"5min":0.41699999999999998,"15min":0.378},"max":{"1min":0.79900000000000004,"5min":1.054,"15min":1.054},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60899999999999999,"15min":0.69499999999999995},"min":{"1min":0.45100000000000001,"5min":0.42099999999999999,"15min":0.41999999999999998},"max":{"1min":0.754,"5min":0.876,"15min":3.3879999999999999},"last":0.70699999999999996}]},{"osd":7,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58299999999999996,"15min":0.60199999999999998},"min":{"1min":0.42999999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.70499999999999996,"5min":0.86699999999999999,"15min":1.137},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59199999999999997,"15min":0.60799999999999998},"min":{"1min":0.42999999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":0.97899999999999998,"15min":1.014},"last":0.60199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574944,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302660,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069180,"statfs":{"total":96636764160,"available":96326840320,"internally_reserved":0,"allocated":1306624,"data_stored":633203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.30299999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.625,"5min":0.74299999999999999,"15min":0.76800000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.51900000000000002,"15min":0.504},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.28299999999999997},"max":{"1min":0.67100000000000004,"5min":0.73099999999999998,"15min":0.77400000000000002},"last":0.61499999999999999}]},{"osd":1,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.41999999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67600000000000005,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51800000000000002,"15min":0.53100000000000003},"min":{"1min":0.29499999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69499999999999995,"5min":0.71799999999999997,"15min":0.82899999999999996},"last":0.54600000000000004}]},{"osd":2,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52600000000000002,"15min":0.51400000000000001},"min":{"1min":0.38100000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67800000000000005,"5min":0.77500000000000002,"15min":0.81000000000000005},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.52800000000000002,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.372,"15min":0.315},"max":{"1min":0.60799999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.42699999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55100000000000005,"15min":0.52600000000000002},"min":{"1min":0.41399999999999998,"5min":0.36599999999999999,"15min":0.35499999999999998},"max":{"1min":0.72799999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54700000000000004,"15min":0.53800000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.751,"5min":0.78800000000000003,"15min":0.79500000000000004},"last":0.66300000000000003}]},{"osd":5,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.44800000000000001,"15min":0.48599999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72399999999999998,"5min":0.77800000000000002,"15min":0.86899999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48199999999999998,"15min":0.49399999999999999},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.255},"max":{"1min":0.70699999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.751}]},{"osd":6,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51000000000000001,"15min":0.53200000000000003},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.312},"max":{"1min":0.78300000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72399999999999998,"5min":0.88,"15min":0.91200000000000003},"last":0.70499999999999996}]},{"osd":7,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.33200000000000002,"15min":0.28999999999999998},"max":{"1min":0.74199999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.58099999999999996},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79900000000000004,"5min":0.81299999999999994,"15min":0.85699999999999998},"last":0.77700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298620,"kb_used_data":1332,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073220,"statfs":{"total":96636764160,"available":96330977280,"internally_reserved":0,"allocated":1363968,"data_stored":671650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.50700000000000001,"15min":0.47999999999999998},"min":{"1min":0.34999999999999998,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.67500000000000004,"5min":0.88,"15min":0.88},"last":0.434},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.503,"15min":0.52500000000000002},"min":{"1min":0.39300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.73699999999999999,"5min":0.86499999999999999,"15min":1.2549999999999999},"last":0.44900000000000001}]},{"osd":1,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.495,"15min":0.48999999999999999},"min":{"1min":0.36899999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.81599999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51400000000000001,"15min":0.51700000000000002},"min":{"1min":0.38500000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.628,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.38500000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52700000000000002,"15min":0.51000000000000001},"min":{"1min":0.33000000000000002,"5min":0.307,"15min":0.30399999999999999},"max":{"1min":0.76500000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54000000000000004,"15min":0.55800000000000005},"min":{"1min":0.47599999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77700000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.36299999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.56599999999999995},"min":{"1min":0.442,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69399999999999995,"5min":0.79900000000000004,"15min":0.95099999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51500000000000001,"15min":0.54700000000000004},"min":{"1min":0.42399999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70499999999999996,"5min":0.79100000000000004,"15min":1.3999999999999999},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.28899999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.504,"15min":0.50700000000000001},"min":{"1min":0.33500000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.80700000000000005},"last":0.52200000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.498,"15min":0.51500000000000001},"min":{"1min":0.314,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.85799999999999998},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.437,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.50700000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55600000000000005,"15min":0.57299999999999995},"min":{"1min":0.41999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.83299999999999996},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.34699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.55700000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:45:57.386 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T10:45:57.386 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:45:57.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.499+0000 7fd47d049700 1 Processor -- start 2023-10-06T10:45:57.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.499+0000 7fd47d049700 1 -- start start 2023-10-06T10:45:57.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.499+0000 7fd47d049700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd478159560 0x7fd4780aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:57.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.499+0000 7fd47d049700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd4780b49a0 0x7fd4780aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:57.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.499+0000 7fd47d049700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4780b0c60 con 0x7fd4780b0760 2023-10-06T10:45:57.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.499+0000 7fd47d049700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4780b0f40 con 0x7fd478159560 2023-10-06T10:45:57.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.499+0000 7fd47d049700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4780b1220 con 0x7fd4780b49a0 2023-10-06T10:45:57.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.500+0000 7fd476d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd478159560 0x7fd4780aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:57.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.500+0000 7fd476d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd478159560 0x7fd4780aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37954/0 (socket says 172.21.15.92:37954) 2023-10-06T10:45:57.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.500+0000 7fd476d9d700 1 -- 172.21.15.92:0/1044897432 learned_addr learned my addr 172.21.15.92:0/1044897432 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:57.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.500+0000 7fd46e59c700 1 --2- 172.21.15.92:0/1044897432 >> v2:172.21.15.124:3300/0 conn(0x7fd4780b49a0 0x7fd4780aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:57.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.500+0000 7fd476d9d700 1 -- 172.21.15.92:0/1044897432 >> v2:172.21.15.124:3300/0 conn(0x7fd4780b49a0 msgr2=0x7fd4780aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:57.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.500+0000 7fd476d9d700 1 --2- 172.21.15.92:0/1044897432 >> v2:172.21.15.124:3300/0 conn(0x7fd4780b49a0 0x7fd4780aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.500+0000 7fd476d9d700 1 -- 172.21.15.92:0/1044897432 >> v1:172.21.15.124:6789/0 conn(0x7fd4780b0760 legacy=0x7fd4780b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:57.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.500+0000 7fd476d9d700 1 -- 172.21.15.92:0/1044897432 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4780b1500 con 0x7fd478159560 2023-10-06T10:45:57.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.500+0000 7fd476d9d700 1 --2- 172.21.15.92:0/1044897432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd478159560 0x7fd4780aa530 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fd468009990 tx=0x7fd468009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3347246fba369046 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:57.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.500+0000 7fd47659c700 1 -- 172.21.15.92:0/1044897432 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd468005c10 con 0x7fd478159560 2023-10-06T10:45:57.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.500+0000 7fd47659c700 1 -- 172.21.15.92:0/1044897432 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd468005d70 con 0x7fd478159560 2023-10-06T10:45:57.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.500+0000 7fd47659c700 1 -- 172.21.15.92:0/1044897432 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46800b840 con 0x7fd478159560 2023-10-06T10:45:57.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.501+0000 7fd47d049700 1 -- 172.21.15.92:0/1044897432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd478159560 msgr2=0x7fd4780aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:57.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.501+0000 7fd47d049700 1 --2- 172.21.15.92:0/1044897432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd478159560 0x7fd4780aa530 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fd468009990 tx=0x7fd468009220 comp rx=0 tx=0).stop 2023-10-06T10:45:57.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.501+0000 7fd47d049700 1 -- 172.21.15.92:0/1044897432 shutdown_connections 2023-10-06T10:45:57.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.501+0000 7fd47d049700 1 --2- 172.21.15.92:0/1044897432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd478159560 0x7fd4780aa530 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.501+0000 7fd47d049700 1 --2- 172.21.15.92:0/1044897432 >> v2:172.21.15.124:3300/0 conn(0x7fd4780b49a0 0x7fd4780aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.501+0000 7fd47d049700 1 -- 172.21.15.92:0/1044897432 >> 172.21.15.92:0/1044897432 conn(0x7fd4780997f0 msgr2=0x7fd47806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:57.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.501+0000 7fd47d049700 1 -- 172.21.15.92:0/1044897432 shutdown_connections 2023-10-06T10:45:57.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.501+0000 7fd47d049700 1 -- 172.21.15.92:0/1044897432 wait complete. 2023-10-06T10:45:57.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.502+0000 7fd47d049700 1 Processor -- start 2023-10-06T10:45:57.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.502+0000 7fd47d049700 1 -- start start 2023-10-06T10:45:57.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.503+0000 7fd47d049700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4780b0760 0x7fd478156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:57.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.503+0000 7fd47d049700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4780b49a0 0x7fd4781585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:57.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.503+0000 7fd47d049700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd478159560 0x7fd47815b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:57.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.503+0000 7fd47d049700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd478158000 con 0x7fd4780b49a0 2023-10-06T10:45:57.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.503+0000 7fd47d049700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4781573f0 con 0x7fd478159560 2023-10-06T10:45:57.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.503+0000 7fd47d049700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4781576d0 con 0x7fd4780b0760 2023-10-06T10:45:57.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.503+0000 7fd46e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4780b49a0 0x7fd4781585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:57.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.503+0000 7fd46e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4780b49a0 0x7fd4781585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37960/0 (socket says 172.21.15.92:37960) 2023-10-06T10:45:57.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.503+0000 7fd46e59c700 1 -- 172.21.15.92:0/2697050875 learned_addr learned my addr 172.21.15.92:0/2697050875 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:57.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.503+0000 7fd476d9d700 1 --2- 172.21.15.92:0/2697050875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4780b0760 0x7fd478156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:57.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.503+0000 7fd47759e700 1 --2- 172.21.15.92:0/2697050875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd478159560 0x7fd47815b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:57.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.503+0000 7fd46e59c700 1 -- 172.21.15.92:0/2697050875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4780b0760 msgr2=0x7fd478156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:57.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.503+0000 7fd46e59c700 1 --2- 172.21.15.92:0/2697050875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4780b0760 0x7fd478156c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.503+0000 7fd46e59c700 1 -- 172.21.15.92:0/2697050875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd478159560 msgr2=0x7fd47815b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:57.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.503+0000 7fd46e59c700 1 --2- 172.21.15.92:0/2697050875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd478159560 0x7fd47815b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.503+0000 7fd46e59c700 1 -- 172.21.15.92:0/2697050875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd468009480 con 0x7fd4780b49a0 2023-10-06T10:45:57.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.503+0000 7fd476d9d700 1 --2- 172.21.15.92:0/2697050875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4780b0760 0x7fd478156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:45:57.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.503+0000 7fd46e59c700 1 --2- 172.21.15.92:0/2697050875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4780b49a0 0x7fd4781585f0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fd46000d890 tx=0x7fd460008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:57.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.503+0000 7fd46f7fe700 1 -- 172.21.15.92:0/2697050875 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd460012630 con 0x7fd4780b49a0 2023-10-06T10:45:57.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.503+0000 7fd46f7fe700 1 -- 172.21.15.92:0/2697050875 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd460012790 con 0x7fd4780b49a0 2023-10-06T10:45:57.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.503+0000 7fd47d049700 1 -- 172.21.15.92:0/2697050875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4781579b0 con 0x7fd4780b49a0 2023-10-06T10:45:57.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.503+0000 7fd46f7fe700 1 -- 172.21.15.92:0/2697050875 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd460012b50 con 0x7fd4780b49a0 2023-10-06T10:45:57.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.504+0000 7fd47d049700 1 -- 172.21.15.92:0/2697050875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd478158b30 con 0x7fd4780b49a0 2023-10-06T10:45:57.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.504+0000 7fd47d049700 1 -- 172.21.15.92:0/2697050875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd478158170 con 0x7fd4780b49a0 2023-10-06T10:45:57.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.505+0000 7fd46f7fe700 1 -- 172.21.15.92:0/2697050875 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd46001d400 con 0x7fd4780b49a0 2023-10-06T10:45:57.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.505+0000 7fd46f7fe700 1 --2- 172.21.15.92:0/2697050875 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd458066f30 0x7fd4580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:57.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.505+0000 7fd46f7fe700 1 -- 172.21.15.92:0/2697050875 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7fd4600849f0 con 0x7fd4780b49a0 2023-10-06T10:45:57.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.505+0000 7fd476d9d700 1 --2- 172.21.15.92:0/2697050875 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd458066f30 0x7fd4580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:57.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.506+0000 7fd476d9d700 1 --2- 172.21.15.92:0/2697050875 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd458066f30 0x7fd4580693e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd468009260 tx=0x7fd468016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:57.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.509+0000 7fd46f7fe700 1 -- 172.21.15.92:0/2697050875 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd460051cc0 con 0x7fd4780b49a0 2023-10-06T10:45:57.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.694+0000 7fd47d049700 1 -- 172.21.15.92:0/2697050875 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd478064a90 con 0x7fd458066f30 2023-10-06T10:45:57.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.697+0000 7fd46f7fe700 1 -- 172.21.15.92:0/2697050875 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73209 (secure 0 0 0) 0x7fd478064a90 con 0x7fd458066f30 2023-10-06T10:45:57.699 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:57.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.699+0000 7fd47d049700 1 -- 172.21.15.92:0/2697050875 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd458066f30 msgr2=0x7fd4580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:57.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.699+0000 7fd47d049700 1 --2- 172.21.15.92:0/2697050875 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd458066f30 0x7fd4580693e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd468009260 tx=0x7fd468016000 comp rx=0 tx=0).stop 2023-10-06T10:45:57.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.699+0000 7fd47d049700 1 -- 172.21.15.92:0/2697050875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4780b49a0 msgr2=0x7fd4781585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:57.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.699+0000 7fd47d049700 1 --2- 172.21.15.92:0/2697050875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4780b49a0 0x7fd4781585f0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fd46000d890 tx=0x7fd460008040 comp rx=0 tx=0).stop 2023-10-06T10:45:57.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.699+0000 7fd47d049700 1 -- 172.21.15.92:0/2697050875 shutdown_connections 2023-10-06T10:45:57.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.699+0000 7fd47d049700 1 --2- 172.21.15.92:0/2697050875 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd458066f30 0x7fd4580693e0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.699+0000 7fd47d049700 1 --2- 172.21.15.92:0/2697050875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd478159560 0x7fd47815b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.699+0000 7fd47d049700 1 --2- 172.21.15.92:0/2697050875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4780b0760 0x7fd478156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.699+0000 7fd47d049700 1 --2- 172.21.15.92:0/2697050875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4780b49a0 0x7fd4781585f0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.699+0000 7fd47d049700 1 -- 172.21.15.92:0/2697050875 >> 172.21.15.92:0/2697050875 conn(0x7fd4780997f0 msgr2=0x7fd4780695e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:57.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.699+0000 7fd47d049700 1 -- 172.21.15.92:0/2697050875 shutdown_connections 2023-10-06T10:45:57.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.699+0000 7fd47d049700 1 -- 172.21.15.92:0/2697050875 wait complete. 2023-10-06T10:45:57.702 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:45:57.713 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2023-10-06T10:45:56.183005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":206,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2398132,"kb_used_data":11636,"kb_used_omap":0,"kb_used_meta":2386432,"kb_avail":752576588,"statfs":{"total":773094113280,"available":770638426112,"internally_reserved":0,"allocated":11915264,"data_stored":6415006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2443706368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-120150,"num_objects":-68,"num_object_clones":0,"num_object_copies":-136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-68,"num_whiteouts":0,"num_read":-1,"num_read_kb":0,"num_write":-246,"num_write_kb":-199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001639"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.597891+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:45:51.597891+0000","last_peered":"2023-10-06T10:45:51.597891+0000","last_clean":"2023-10-06T10:45:51.597891+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:45:51.597891+0000","last_undegraded":"2023-10-06T10:45:51.597891+0000","last_fullsized":"2023-10-06T10:45:51.597891+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.695321+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:45:51.695321+0000","last_peered":"2023-10-06T10:45:51.695321+0000","last_clean":"2023-10-06T10:45:51.695321+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:45:51.695321+0000","last_undegraded":"2023-10-06T10:45:51.695321+0000","last_fullsized":"2023-10-06T10:45:51.695321+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":88,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.695604+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:45:51.695604+0000","last_peered":"2023-10-06T10:45:51.695604+0000","last_clean":"2023-10-06T10:45:51.695604+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:45:51.695604+0000","last_undegraded":"2023-10-06T10:45:51.695604+0000","last_fullsized":"2023-10-06T10:45:51.695604+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":230,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.694648+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:45:51.694648+0000","last_peered":"2023-10-06T10:45:51.694648+0000","last_clean":"2023-10-06T10:45:51.694648+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:45:51.694648+0000","last_undegraded":"2023-10-06T10:45:51.694648+0000","last_fullsized":"2023-10-06T10:45:51.694648+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":80,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.694820+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:45:51.694820+0000","last_peered":"2023-10-06T10:45:51.694820+0000","last_clean":"2023-10-06T10:45:51.694820+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:45:51.694820+0000","last_undegraded":"2023-10-06T10:45:51.694820+0000","last_fullsized":"2023-10-06T10:45:51.694820+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.694804+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:45:51.694804+0000","last_peered":"2023-10-06T10:45:51.694804+0000","last_clean":"2023-10-06T10:45:51.694804+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:45:51.694804+0000","last_undegraded":"2023-10-06T10:45:51.694804+0000","last_fullsized":"2023-10-06T10:45:51.694804+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.597840+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:45:51.597840+0000","last_peered":"2023-10-06T10:45:51.597840+0000","last_clean":"2023-10-06T10:45:51.597840+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:45:51.597840+0000","last_undegraded":"2023-10-06T10:45:51.597840+0000","last_fullsized":"2023-10-06T10:45:51.597840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.695597+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:45:51.695597+0000","last_peered":"2023-10-06T10:45:51.695597+0000","last_clean":"2023-10-06T10:45:51.695597+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:45:51.695597+0000","last_undegraded":"2023-10-06T10:45:51.695597+0000","last_fullsized":"2023-10-06T10:45:51.695597+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T10:45:51.695686+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:45:51.695686+0000","last_peered":"2023-10-06T10:45:51.695686+0000","last_clean":"2023-10-06T10:45:51.695686+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:45:51.695686+0000","last_undegraded":"2023-10-06T10:45:51.695686+0000","last_fullsized":"2023-10-06T10:45:51.695686+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574946,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299148,"kb_used_data":1860,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072692,"statfs":{"total":96636764160,"available":96330436608,"internally_reserved":0,"allocated":1904640,"data_stored":1224125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52300000000000002,"15min":0.49399999999999999},"min":{"1min":0.434,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.78400000000000003,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53700000000000003,"15min":0.52100000000000002},"min":{"1min":0.38900000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66100000000000003,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":0.60899999999999999}]},{"osd":1,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.495,"15min":0.47099999999999997},"min":{"1min":0.39700000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.64700000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.623},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54700000000000004,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.748,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.64700000000000002}]},{"osd":2,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56999999999999995,"15min":0.52500000000000002},"min":{"1min":0.40999999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69899999999999995,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51800000000000002,"15min":0.501},"min":{"1min":0.35599999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.67600000000000005,"5min":0.747,"15min":0.79800000000000004},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56200000000000006,"15min":0.52200000000000002},"min":{"1min":0.36699999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76200000000000001,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.42699999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.66500000000000004}]},{"osd":4,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.44,"15min":0.46300000000000002},"min":{"1min":0.30599999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70099999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.48999999999999999,"15min":0.501},"min":{"1min":0.311,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.75700000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.78300000000000003}]},{"osd":5,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.48399999999999999,"15min":0.48499999999999999},"min":{"1min":0.36099999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.77700000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.34000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.746,"5min":1.577,"15min":1.577},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.55200000000000005,"15min":0.56699999999999995},"min":{"1min":0.438,"5min":0.308,"15min":0.308},"max":{"1min":0.77300000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.57899999999999996,"15min":0.57799999999999996},"min":{"1min":0.37,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80700000000000005,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.80400000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574944,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298564,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073276,"statfs":{"total":96636764160,"available":96331034624,"internally_reserved":0,"allocated":1306624,"data_stored":631908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51200000000000001,"15min":0.497},"min":{"1min":0.38100000000000001,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.77000000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51600000000000001,"15min":0.46100000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.70099999999999996,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.52400000000000002}]},{"osd":1,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51400000000000001,"15min":0.46500000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.28000000000000003},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.80500000000000005},"last":0.622},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.495},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.751,"15min":0.751},"last":0.53900000000000003}]},{"osd":2,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51800000000000002,"15min":0.48699999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54200000000000004,"15min":0.49199999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.73899999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.64700000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55200000000000005,"15min":0.51100000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.76900000000000002,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55000000000000004,"15min":0.52400000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79400000000000004},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.5,"15min":0.48999999999999999},"min":{"1min":0.32300000000000001,"5min":0.308,"15min":0.26700000000000002},"max":{"1min":0.72499999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.40100000000000002}]},{"osd":5,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51100000000000001,"15min":0.495},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.75700000000000001,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55400000000000005,"15min":0.54300000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.308},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.42599999999999999}]},{"osd":7,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53200000000000003,"15min":0.52100000000000002},"min":{"1min":0.34899999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.755,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57899999999999996,"15min":0.55200000000000005},"min":{"1min":0.38200000000000001,"5min":0.33000000000000002,"15min":0.32300000000000001},"max":{"1min":0.79100000000000004,"5min":0.879,"15min":0.879},"last":0.46700000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303288,"kb_used_data":1904,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068552,"statfs":{"total":96636764160,"available":96326197248,"internally_reserved":0,"allocated":1949696,"data_stored":1246951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32500000000000001,"15min":0.32700000000000001},"min":{"1min":0.217,"5min":0.188,"15min":0.188},"max":{"1min":0.42599999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.246,"5min":0.189,"15min":0.189},"max":{"1min":0.53200000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.32700000000000001,"15min":0.32200000000000001},"min":{"1min":0.22900000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.48099999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.373,"15min":0.36699999999999999},"min":{"1min":0.28599999999999998,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.54000000000000004,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.503}]},{"osd":3,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39200000000000002,"15min":0.38200000000000001},"min":{"1min":0.30499999999999999,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.54800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.371,"5min":0.38,"15min":0.377},"min":{"1min":0.26100000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.45800000000000002,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.51900000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38200000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.746,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.45300000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.753,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.59399999999999997}]},{"osd":5,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.77500000000000002,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56999999999999995,"15min":0.55200000000000005},"min":{"1min":0.41099999999999998,"5min":0.36799999999999999,"15min":0.35699999999999998},"max":{"1min":0.76800000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.61899999999999999}]},{"osd":6,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.39100000000000001,"5min":0.38700000000000001,"15min":0.36699999999999999},"max":{"1min":0.81299999999999994,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.46500000000000002,"5min":0.40000000000000002,"15min":0.39800000000000002},"max":{"1min":0.74399999999999999,"5min":0.93500000000000005,"15min":0.94099999999999995},"last":0.65200000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.80200000000000005,"5min":0.996,"15min":0.996},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.41299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.73399999999999999,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.68600000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574946,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298696,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073144,"statfs":{"total":96636764160,"available":96330899456,"internally_reserved":0,"allocated":1441792,"data_stored":733555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.317,"15min":0.31900000000000001},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.41399999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.251},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.37,"15min":0.35199999999999998},"min":{"1min":0.27000000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.57299999999999995,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.312}]},{"osd":2,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.314,"15min":0.309},"min":{"1min":0.22600000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.46999999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34799999999999998,"15min":0.33400000000000002},"min":{"1min":0.247,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.61399999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.28100000000000003}]},{"osd":3,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34399999999999997,"15min":0.34499999999999997},"min":{"1min":0.24299999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.51400000000000001,"5min":0.622,"15min":0.622},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36099999999999999,"15min":0.35399999999999998},"min":{"1min":0.28399999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54200000000000004,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.29799999999999999}]},{"osd":4,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54500000000000004,"15min":0.55600000000000005},"min":{"1min":0.435,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.74099999999999999,"5min":0.78900000000000003,"15min":0.83299999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.54100000000000004},"min":{"1min":0.42699999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69699999999999995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.35199999999999998}]},{"osd":5,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.435,"5min":0.377,"15min":0.36899999999999999},"max":{"1min":0.76400000000000001,"5min":0.85699999999999998,"15min":1.0069999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70499999999999996,"5min":0.77500000000000002,"15min":0.94499999999999995},"last":0.435}]},{"osd":6,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56200000000000006,"15min":0.57899999999999996},"min":{"1min":0.40899999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.68600000000000005,"5min":1.046,"15min":1.046},"last":0.376},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.68899999999999995,"5min":0.80000000000000004,"15min":1.075},"last":0.55700000000000005}]},{"osd":7,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.40600000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73399999999999999,"5min":0.83499999999999996,"15min":0.96099999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.60399999999999998},"min":{"1min":0.46100000000000002,"5min":0.41899999999999998,"15min":0.41499999999999998},"max":{"1min":0.76200000000000001,"5min":0.77200000000000002,"15min":0.90100000000000002},"last":0.497}]}]},{"osd":2,"up_from":13,"seq":55834574944,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298580,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073260,"statfs":{"total":96636764160,"available":96331018240,"internally_reserved":0,"allocated":1323008,"data_stored":633551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.315,"15min":0.312},"min":{"1min":0.22600000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.45700000000000002,"5min":0.496,"15min":0.57999999999999996},"last":0.249},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.35699999999999998,"15min":0.33100000000000002},"min":{"1min":0.26600000000000001,"5min":0.20200000000000001,"15min":0.20100000000000001},"max":{"1min":0.51700000000000002,"5min":0.55600000000000005,"15min":0.622},"last":0.35399999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33000000000000002},"min":{"1min":0.253,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.47299999999999998,"5min":0.56599999999999995,"15min":0.60499999999999998},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.376,"15min":0.36899999999999999},"min":{"1min":0.29599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47999999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.27400000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.38700000000000001,"15min":0.35999999999999999},"min":{"1min":0.309,"5min":0.25700000000000001,"15min":0.20499999999999999},"max":{"1min":0.504,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.378,"15min":0.36299999999999999},"min":{"1min":0.27500000000000002,"5min":0.25,"15min":0.22800000000000001},"max":{"1min":0.50900000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.314}]},{"osd":4,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.40799999999999997,"5min":0.33400000000000002,"15min":0.315},"max":{"1min":0.66300000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.45700000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75,"5min":0.82399999999999995,"15min":0.84899999999999998},"last":0.499}]},{"osd":5,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.51000000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72199999999999998,"5min":0.83799999999999997,"15min":0.91200000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.41699999999999998,"5min":0.36599999999999999,"15min":0.33800000000000002},"max":{"1min":0.69499999999999995,"5min":0.81799999999999995,"15min":0.83399999999999996},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59299999999999997,"15min":0.61399999999999999},"min":{"1min":0.42999999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.70899999999999996,"5min":0.88700000000000001,"15min":0.89100000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61299999999999999,"15min":0.62},"min":{"1min":0.46800000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.748,"5min":0.80400000000000005,"15min":0.93799999999999994},"last":0.52100000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57799999999999996,"15min":0.60299999999999998},"min":{"1min":0.433,"5min":0.38500000000000001,"15min":0.36799999999999999},"max":{"1min":0.64300000000000002,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60799999999999998,"15min":0.621},"min":{"1min":0.438,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.70999999999999996,"5min":0.81100000000000005,"15min":0.874},"last":0.56699999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574944,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298576,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073264,"statfs":{"total":96636764160,"available":96331022336,"internally_reserved":0,"allocated":1318912,"data_stored":640063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.31,"15min":0.40500000000000003},"min":{"1min":0.214,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.436,"5min":0.52700000000000002,"15min":3.3210000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.33000000000000002,"15min":0.35399999999999998},"min":{"1min":0.22500000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":1.218},"last":0.45900000000000002}]},{"osd":1,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.33300000000000002,"15min":0.33800000000000002},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.20100000000000001},"max":{"1min":0.496,"5min":0.63600000000000001,"15min":0.86299999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35199999999999998,"15min":0.44500000000000001},"min":{"1min":0.26900000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.47299999999999998,"5min":0.66000000000000003,"15min":3.359},"last":0.42999999999999999}]},{"osd":2,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.35299999999999998,"15min":0.374},"min":{"1min":0.28100000000000003,"5min":0.255,"15min":0.215},"max":{"1min":0.51700000000000002,"5min":0.71099999999999997,"15min":1.1100000000000001},"last":0.498},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.56699999999999995},"min":{"1min":0.28100000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.54200000000000004,"5min":0.59899999999999998,"15min":7.069},"last":0.44500000000000001}]},{"osd":4,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54800000000000004,"15min":0.57799999999999996},"min":{"1min":0.43099999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73799999999999999,"5min":0.84699999999999998,"15min":1.165},"last":0.622},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57799999999999996,"15min":0.59099999999999997},"min":{"1min":0.441,"5min":0.378,"15min":0.375},"max":{"1min":0.73599999999999999,"5min":1.02,"15min":1.02},"last":0.67500000000000004}]},{"osd":5,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.66500000000000004},"min":{"1min":0.41699999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72699999999999998,"5min":0.84499999999999997,"15min":3.4140000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57199999999999995,"15min":0.59799999999999998},"min":{"1min":0.39800000000000002,"5min":0.39200000000000002,"15min":0.34599999999999997},"max":{"1min":0.71299999999999997,"5min":0.83299999999999996,"15min":1.1930000000000001},"last":0.66200000000000003}]},{"osd":6,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59599999999999997,"15min":0.61099999999999999},"min":{"1min":0.41899999999999998,"5min":0.41699999999999998,"15min":0.378},"max":{"1min":0.79900000000000004,"5min":1.054,"15min":1.054},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60899999999999999,"15min":0.69499999999999995},"min":{"1min":0.45100000000000001,"5min":0.42099999999999999,"15min":0.41999999999999998},"max":{"1min":0.754,"5min":0.876,"15min":3.3879999999999999},"last":0.70699999999999996}]},{"osd":7,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58299999999999996,"15min":0.60199999999999998},"min":{"1min":0.42999999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.70499999999999996,"5min":0.86699999999999999,"15min":1.137},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59199999999999997,"15min":0.60799999999999998},"min":{"1min":0.42999999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":0.97899999999999998,"15min":1.014},"last":0.60199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574944,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302660,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069180,"statfs":{"total":96636764160,"available":96326840320,"internally_reserved":0,"allocated":1306624,"data_stored":633203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.30299999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.625,"5min":0.74299999999999999,"15min":0.76800000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.51900000000000002,"15min":0.504},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.28299999999999997},"max":{"1min":0.67100000000000004,"5min":0.73099999999999998,"15min":0.77400000000000002},"last":0.61499999999999999}]},{"osd":1,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.41999999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67600000000000005,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51800000000000002,"15min":0.53100000000000003},"min":{"1min":0.29499999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69499999999999995,"5min":0.71799999999999997,"15min":0.82899999999999996},"last":0.54600000000000004}]},{"osd":2,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52600000000000002,"15min":0.51400000000000001},"min":{"1min":0.38100000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67800000000000005,"5min":0.77500000000000002,"15min":0.81000000000000005},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.52800000000000002,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.372,"15min":0.315},"max":{"1min":0.60799999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.42699999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55100000000000005,"15min":0.52600000000000002},"min":{"1min":0.41399999999999998,"5min":0.36599999999999999,"15min":0.35499999999999998},"max":{"1min":0.72799999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54700000000000004,"15min":0.53800000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.751,"5min":0.78800000000000003,"15min":0.79500000000000004},"last":0.66300000000000003}]},{"osd":5,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.44800000000000001,"15min":0.48599999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72399999999999998,"5min":0.77800000000000002,"15min":0.86899999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48199999999999998,"15min":0.49399999999999999},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.255},"max":{"1min":0.70699999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.751}]},{"osd":6,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51000000000000001,"15min":0.53200000000000003},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.312},"max":{"1min":0.78300000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72399999999999998,"5min":0.88,"15min":0.91200000000000003},"last":0.70499999999999996}]},{"osd":7,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.33200000000000002,"15min":0.28999999999999998},"max":{"1min":0.74199999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.58099999999999996},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79900000000000004,"5min":0.81299999999999994,"15min":0.85699999999999998},"last":0.77700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298620,"kb_used_data":1332,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073220,"statfs":{"total":96636764160,"available":96330977280,"internally_reserved":0,"allocated":1363968,"data_stored":671650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.50700000000000001,"15min":0.47999999999999998},"min":{"1min":0.34999999999999998,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.67500000000000004,"5min":0.88,"15min":0.88},"last":0.434},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.503,"15min":0.52500000000000002},"min":{"1min":0.39300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.73699999999999999,"5min":0.86499999999999999,"15min":1.2549999999999999},"last":0.44900000000000001}]},{"osd":1,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.495,"15min":0.48999999999999999},"min":{"1min":0.36899999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.81599999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51400000000000001,"15min":0.51700000000000002},"min":{"1min":0.38500000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.628,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.38500000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52700000000000002,"15min":0.51000000000000001},"min":{"1min":0.33000000000000002,"5min":0.307,"15min":0.30399999999999999},"max":{"1min":0.76500000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54000000000000004,"15min":0.55800000000000005},"min":{"1min":0.47599999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77700000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.36299999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.56599999999999995},"min":{"1min":0.442,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69399999999999995,"5min":0.79900000000000004,"15min":0.95099999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51500000000000001,"15min":0.54700000000000004},"min":{"1min":0.42399999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70499999999999996,"5min":0.79100000000000004,"15min":1.3999999999999999},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.28899999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.504,"15min":0.50700000000000001},"min":{"1min":0.33500000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.80700000000000005},"last":0.52200000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.498,"15min":0.51500000000000001},"min":{"1min":0.314,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.85799999999999998},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.437,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.50700000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55600000000000005,"15min":0.57299999999999995},"min":{"1min":0.41999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.83299999999999996},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.34699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.55700000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:45:57.715 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T10:45:57.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.828+0000 7f8abb7cb700 1 Processor -- start 2023-10-06T10:45:57.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.828+0000 7f8abb7cb700 1 -- start start 2023-10-06T10:45:57.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.829+0000 7f8abb7cb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8ab40adb60 0x7f8ab40ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:57.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.829+0000 7f8abb7cb700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ab40ce6d0 0x7f8ab415aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:57.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.829+0000 7f8abb7cb700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab4157de0 con 0x7f8ab40ca370 2023-10-06T10:45:57.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.829+0000 7f8abb7cb700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab40cbb90 con 0x7f8ab40adb60 2023-10-06T10:45:57.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.829+0000 7f8abb7cb700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab40ceb10 con 0x7f8ab40ce6d0 2023-10-06T10:45:57.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.829+0000 7f8ab8d66700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8ab40adb60 0x7f8ab40ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:57.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.829+0000 7f8ab8d66700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8ab40adb60 0x7f8ab40ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37974/0 (socket says 172.21.15.92:37974) 2023-10-06T10:45:57.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.829+0000 7f8ab8d66700 1 -- 172.21.15.92:0/3696545553 learned_addr learned my addr 172.21.15.92:0/3696545553 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:57.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.829+0000 7f8ab9567700 1 --1- 172.21.15.92:0/3696545553 >> v1:172.21.15.92:6789/0 conn(0x7f8ab40ca370 0x7f8ab4158270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35394/0 (socket says 172.21.15.92:35394) 2023-10-06T10:45:57.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.829+0000 7f8ab9d68700 1 --2- 172.21.15.92:0/3696545553 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ab40ce6d0 0x7f8ab415aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:57.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.829+0000 7f8ab8d66700 1 -- 172.21.15.92:0/3696545553 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ab40ce6d0 msgr2=0x7f8ab415aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:57.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.829+0000 7f8ab8d66700 1 --2- 172.21.15.92:0/3696545553 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ab40ce6d0 0x7f8ab415aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.829+0000 7f8ab8d66700 1 -- 172.21.15.92:0/3696545553 >> v1:172.21.15.92:6789/0 conn(0x7f8ab40ca370 legacy=0x7f8ab4158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:57.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.829+0000 7f8ab8d66700 1 -- 172.21.15.92:0/3696545553 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab40cb5d0 con 0x7f8ab40adb60 2023-10-06T10:45:57.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.829+0000 7f8aabfff700 1 -- 172.21.15.92:0/3696545553 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853058566 0 0) 0x7f8ab4157de0 con 0x7f8ab40ca370 2023-10-06T10:45:57.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.829+0000 7f8ab8d66700 1 --2- 172.21.15.92:0/3696545553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ab40adb60 0x7f8ab40ce190 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40099b0 tx=0x7f8aa4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2e8837a92a8666 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:57.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.829+0000 7f8aabfff700 1 -- 172.21.15.92:0/3696545553 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa400b860 con 0x7f8ab40adb60 2023-10-06T10:45:57.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.829+0000 7f8aabfff700 1 -- 172.21.15.92:0/3696545553 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa400b9c0 con 0x7f8ab40adb60 2023-10-06T10:45:57.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.829+0000 7f8aabfff700 1 -- 172.21.15.92:0/3696545553 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa400bd60 con 0x7f8ab40adb60 2023-10-06T10:45:57.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.830+0000 7f8abb7cb700 1 -- 172.21.15.92:0/3696545553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ab40adb60 msgr2=0x7f8ab40ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:57.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.830+0000 7f8abb7cb700 1 --2- 172.21.15.92:0/3696545553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ab40adb60 0x7f8ab40ce190 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40099b0 tx=0x7f8aa4009250 comp rx=0 tx=0).stop 2023-10-06T10:45:57.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.830+0000 7f8abb7cb700 1 -- 172.21.15.92:0/3696545553 shutdown_connections 2023-10-06T10:45:57.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.830+0000 7f8abb7cb700 1 --2- 172.21.15.92:0/3696545553 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ab40ce6d0 0x7f8ab415aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.830+0000 7f8abb7cb700 1 --2- 172.21.15.92:0/3696545553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ab40adb60 0x7f8ab40ce190 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.830+0000 7f8abb7cb700 1 -- 172.21.15.92:0/3696545553 >> 172.21.15.92:0/3696545553 conn(0x7f8ab40707d0 msgr2=0x7f8ab4070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:57.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.830+0000 7f8abb7cb700 1 -- 172.21.15.92:0/3696545553 shutdown_connections 2023-10-06T10:45:57.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.830+0000 7f8abb7cb700 1 -- 172.21.15.92:0/3696545553 wait complete. 2023-10-06T10:45:57.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.831+0000 7f8abb7cb700 1 Processor -- start 2023-10-06T10:45:57.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.831+0000 7f8abb7cb700 1 -- start start 2023-10-06T10:45:57.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.832+0000 7f8abb7cb700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ab40adb60 0x7f8ab415acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:57.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.832+0000 7f8abb7cb700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ab40ca370 0x7f8ab40b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:57.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.832+0000 7f8abb7cb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ab40ce6d0 0x7f8ab40b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:57.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.832+0000 7f8abb7cb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab416c040 con 0x7f8ab40ce6d0 2023-10-06T10:45:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.832+0000 7f8abb7cb700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab415bfe0 con 0x7f8ab40ca370 2023-10-06T10:45:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.832+0000 7f8abb7cb700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab40b2900 con 0x7f8ab40adb60 2023-10-06T10:45:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.832+0000 7f8ab9d68700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ab40ce6d0 0x7f8ab40b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.832+0000 7f8ab9d68700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ab40ce6d0 0x7f8ab40b2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37978/0 (socket says 172.21.15.92:37978) 2023-10-06T10:45:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.832+0000 7f8ab9d68700 1 -- 172.21.15.92:0/2778725773 learned_addr learned my addr 172.21.15.92:0/2778725773 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.832+0000 7f8ab9567700 1 --2- 172.21.15.92:0/2778725773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ab40adb60 0x7f8ab415acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.832+0000 7f8ab9d68700 1 -- 172.21.15.92:0/2778725773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ab40adb60 msgr2=0x7f8ab415acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.832+0000 7f8ab8d66700 1 --2- 172.21.15.92:0/2778725773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ab40ca370 0x7f8ab40b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.832+0000 7f8ab9d68700 1 --2- 172.21.15.92:0/2778725773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ab40adb60 0x7f8ab415acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.832+0000 7f8ab9d68700 1 -- 172.21.15.92:0/2778725773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ab40ca370 msgr2=0x7f8ab40b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.832+0000 7f8ab9d68700 1 --2- 172.21.15.92:0/2778725773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ab40ca370 0x7f8ab40b1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:57.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.832+0000 7f8ab9d68700 1 -- 172.21.15.92:0/2778725773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa40094f0 con 0x7f8ab40ce6d0 2023-10-06T10:45:57.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.833+0000 7f8ab9d68700 1 --2- 172.21.15.92:0/2778725773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ab40ce6d0 0x7f8ab40b2060 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00d8c0 tx=0x7f8aac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:57.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.833+0000 7f8aa97fa700 1 -- 172.21.15.92:0/2778725773 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac00b570 con 0x7f8ab40ce6d0 2023-10-06T10:45:57.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.833+0000 7f8aa97fa700 1 -- 172.21.15.92:0/2778725773 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aac00b6f0 con 0x7f8ab40ce6d0 2023-10-06T10:45:57.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.833+0000 7f8abb7cb700 1 -- 172.21.15.92:0/2778725773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab415b3d0 con 0x7f8ab40ce6d0 2023-10-06T10:45:57.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.833+0000 7f8aa97fa700 1 -- 172.21.15.92:0/2778725773 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac00bab0 con 0x7f8ab40ce6d0 2023-10-06T10:45:57.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.833+0000 7f8abb7cb700 1 -- 172.21.15.92:0/2778725773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab415b990 con 0x7f8ab40ce6d0 2023-10-06T10:45:57.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.834+0000 7f8abb7cb700 1 -- 172.21.15.92:0/2778725773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab4064a90 con 0x7f8ab40ce6d0 2023-10-06T10:45:57.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.834+0000 7f8aa97fa700 1 -- 172.21.15.92:0/2778725773 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8aac01e460 con 0x7f8ab40ce6d0 2023-10-06T10:45:57.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.834+0000 7f8aa97fa700 1 --2- 172.21.15.92:0/2778725773 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8a98066fb0 0x7f8a98069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:57.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.835+0000 7f8aa97fa700 1 -- 172.21.15.92:0/2778725773 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f8aac084c40 con 0x7f8ab40ce6d0 2023-10-06T10:45:57.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.835+0000 7f8ab9567700 1 --2- 172.21.15.92:0/2778725773 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8a98066fb0 0x7f8a98069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:57.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.835+0000 7f8ab9567700 1 --2- 172.21.15.92:0/2778725773 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8a98066fb0 0x7f8a98069460 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8ab41588a0 tx=0x7f8ab0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:57.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:57.839+0000 7f8aa97fa700 1 -- 172.21.15.92:0/2778725773 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8aac051dc0 con 0x7f8ab40ce6d0 2023-10-06T10:45:58.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:58.023+0000 7f8abb7cb700 1 -- 172.21.15.92:0/2778725773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ab40cb8b0 con 0x7f8ab40ce6d0 2023-10-06T10:45:58.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:58.023+0000 7f8aa97fa700 1 -- 172.21.15.92:0/2778725773 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+14256 (secure 0 0 0) 0x7f8aac0553e0 con 0x7f8ab40ce6d0 2023-10-06T10:45:58.025 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:45:58.026 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":107,"fsid":"8f307bfb-c4cb-4f66-aade-46fffe0c6e0c","created":"2023-10-06T10:37:58.530463+0000","modified":"2023-10-06T10:45:54.601348+0000","last_up_change":"2023-10-06T10:38:09.254652+0000","last_in_change":"2023-10-06T10:38:00.935065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T10:38:09.680559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T10:38:10.120786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"07b2c439-9b78-4ecf-baec-34cf38431e7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90604},{"type":"v1","addr":"172.21.15.92:6801","nonce":90604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90604},{"type":"v1","addr":"172.21.15.92:6803","nonce":90604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90604},{"type":"v1","addr":"172.21.15.92:6807","nonce":90604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90604},{"type":"v1","addr":"172.21.15.92:6805","nonce":90604}]},"public_addr":"172.21.15.92:6801/90604","cluster_addr":"172.21.15.92:6803/90604","heartbeat_back_addr":"172.21.15.92:6807/90604","heartbeat_front_addr":"172.21.15.92:6805/90604","state":["exists","up"]},{"osd":1,"uuid":"c8a0b0c4-3d96-4679-b5c3-905d147b24fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":90606},{"type":"v1","addr":"172.21.15.92:6817","nonce":90606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":90606},{"type":"v1","addr":"172.21.15.92:6819","nonce":90606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90606},{"type":"v1","addr":"172.21.15.92:6823","nonce":90606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90606},{"type":"v1","addr":"172.21.15.92:6821","nonce":90606}]},"public_addr":"172.21.15.92:6817/90606","cluster_addr":"172.21.15.92:6819/90606","heartbeat_back_addr":"172.21.15.92:6823/90606","heartbeat_front_addr":"172.21.15.92:6821/90606","state":["exists","up"]},{"osd":2,"uuid":"d5138725-f0d1-4c57-99de-6dbda923cdd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":90615},{"type":"v1","addr":"172.21.15.92:6825","nonce":90615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90615},{"type":"v1","addr":"172.21.15.92:6827","nonce":90615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90615},{"type":"v1","addr":"172.21.15.92:6831","nonce":90615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":90615},{"type":"v1","addr":"172.21.15.92:6829","nonce":90615}]},"public_addr":"172.21.15.92:6825/90615","cluster_addr":"172.21.15.92:6827/90615","heartbeat_back_addr":"172.21.15.92:6831/90615","heartbeat_front_addr":"172.21.15.92:6829/90615","state":["exists","up"]},{"osd":3,"uuid":"c3210bd7-36b2-4ea4-969b-8bb51bb9ca76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90605},{"type":"v1","addr":"172.21.15.92:6809","nonce":90605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90605},{"type":"v1","addr":"172.21.15.92:6811","nonce":90605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":90605},{"type":"v1","addr":"172.21.15.92:6815","nonce":90605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90605},{"type":"v1","addr":"172.21.15.92:6813","nonce":90605}]},"public_addr":"172.21.15.92:6809/90605","cluster_addr":"172.21.15.92:6811/90605","heartbeat_back_addr":"172.21.15.92:6815/90605","heartbeat_front_addr":"172.21.15.92:6813/90605","state":["exists","up"]},{"osd":4,"uuid":"5d42097a-3896-4e09-b7e3-5a6702775f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":90883},{"type":"v1","addr":"172.21.15.124:6801","nonce":90883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":90883},{"type":"v1","addr":"172.21.15.124:6803","nonce":90883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":90883},{"type":"v1","addr":"172.21.15.124:6807","nonce":90883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":90883},{"type":"v1","addr":"172.21.15.124:6805","nonce":90883}]},"public_addr":"172.21.15.124:6801/90883","cluster_addr":"172.21.15.124:6803/90883","heartbeat_back_addr":"172.21.15.124:6807/90883","heartbeat_front_addr":"172.21.15.124:6805/90883","state":["exists","up"]},{"osd":5,"uuid":"ac00cda9-44d6-432e-b24e-86fa42706227","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":90907},{"type":"v1","addr":"172.21.15.124:6817","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":90907},{"type":"v1","addr":"172.21.15.124:6819","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":90907},{"type":"v1","addr":"172.21.15.124:6823","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":90907},{"type":"v1","addr":"172.21.15.124:6821","nonce":90907}]},"public_addr":"172.21.15.124:6817/90907","cluster_addr":"172.21.15.124:6819/90907","heartbeat_back_addr":"172.21.15.124:6823/90907","heartbeat_front_addr":"172.21.15.124:6821/90907","state":["exists","up"]},{"osd":6,"uuid":"a03ebfe4-29aa-4a8f-9a44-ed1fa2fedf01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":90914},{"type":"v1","addr":"172.21.15.124:6825","nonce":90914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":90914},{"type":"v1","addr":"172.21.15.124:6827","nonce":90914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":90914},{"type":"v1","addr":"172.21.15.124:6831","nonce":90914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":90914},{"type":"v1","addr":"172.21.15.124:6829","nonce":90914}]},"public_addr":"172.21.15.124:6825/90914","cluster_addr":"172.21.15.124:6827/90914","heartbeat_back_addr":"172.21.15.124:6831/90914","heartbeat_front_addr":"172.21.15.124:6829/90914","state":["exists","up"]},{"osd":7,"uuid":"3c94185e-03aa-4150-9860-9550f0651c4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":90884},{"type":"v1","addr":"172.21.15.124:6809","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":90884},{"type":"v1","addr":"172.21.15.124:6811","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":90884},{"type":"v1","addr":"172.21.15.124:6815","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":90884},{"type":"v1","addr":"172.21.15.124:6813","nonce":90884}]},"public_addr":"172.21.15.124:6809/90884","cluster_addr":"172.21.15.124:6811/90884","heartbeat_back_addr":"172.21.15.124:6815/90884","heartbeat_front_addr":"172.21.15.124:6813/90884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.028449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.126712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.127825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.065569+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:05.832876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.125944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.169670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T10:38:06.154918+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6835/2298348426":"2023-10-07T10:45:51.590092+0000","172.21.15.124:6835/2307521923":"2023-10-07T10:45:51.590072+0000","172.21.15.124:6833/2307521923":"2023-10-07T10:45:51.590072+0000","172.21.15.92:6837/2298348426":"2023-10-07T10:45:51.590092+0000","172.21.15.92:6836/1979826278":"2023-10-07T10:45:51.590086+0000","172.21.15.92:6834/1979826278":"2023-10-07T10:45:51.590086+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T10:45:58.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:58.026+0000 7f8abb7cb700 1 -- 172.21.15.92:0/2778725773 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8a98066fb0 msgr2=0x7f8a98069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:58.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:58.026+0000 7f8abb7cb700 1 --2- 172.21.15.92:0/2778725773 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8a98066fb0 0x7f8a98069460 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8ab41588a0 tx=0x7f8ab0009040 comp rx=0 tx=0).stop 2023-10-06T10:45:58.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:58.026+0000 7f8abb7cb700 1 -- 172.21.15.92:0/2778725773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ab40ce6d0 msgr2=0x7f8ab40b2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:58.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:58.026+0000 7f8abb7cb700 1 --2- 172.21.15.92:0/2778725773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ab40ce6d0 0x7f8ab40b2060 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00d8c0 tx=0x7f8aac008040 comp rx=0 tx=0).stop 2023-10-06T10:45:58.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:58.026+0000 7f8abb7cb700 1 -- 172.21.15.92:0/2778725773 shutdown_connections 2023-10-06T10:45:58.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:58.026+0000 7f8abb7cb700 1 --2- 172.21.15.92:0/2778725773 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8a98066fb0 0x7f8a98069460 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:58.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:58.026+0000 7f8abb7cb700 1 --2- 172.21.15.92:0/2778725773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ab40ce6d0 0x7f8ab40b2060 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:58.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:58.026+0000 7f8abb7cb700 1 --2- 172.21.15.92:0/2778725773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ab40adb60 0x7f8ab415acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:58.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:58.026+0000 7f8abb7cb700 1 --2- 172.21.15.92:0/2778725773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ab40ca370 0x7f8ab40b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:58.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:58.026+0000 7f8abb7cb700 1 -- 172.21.15.92:0/2778725773 >> 172.21.15.92:0/2778725773 conn(0x7f8ab40707d0 msgr2=0x7f8ab40ca740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:58.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:58.026+0000 7f8abb7cb700 1 -- 172.21.15.92:0/2778725773 shutdown_connections 2023-10-06T10:45:58.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:58.026+0000 7f8abb7cb700 1 -- 172.21.15.92:0/2778725773 wait complete. 2023-10-06T10:45:59.041 INFO:tasks.ceph:Scrubbing osd.0 2023-10-06T10:45:59.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-06T10:45:59.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.166+0000 7fd58234b700 1 Processor -- start 2023-10-06T10:45:59.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.167+0000 7fd58234b700 1 -- start start 2023-10-06T10:45:59.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.167+0000 7fd58234b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd57c06ece0 0x7fd57c0b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.167+0000 7fd58234b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd57c06f0b0 0x7fd57c0b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.167+0000 7fd58234b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd57c0b2ce0 con 0x7fd57c0aede0 2023-10-06T10:45:59.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.167+0000 7fd58234b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd57c06fb40 con 0x7fd57c06f0b0 2023-10-06T10:45:59.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.167+0000 7fd58234b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c0b0410 con 0x7fd57c06ece0 2023-10-06T10:45:59.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.167+0000 7fd57b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd57c06f0b0 0x7fd57c0b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.167+0000 7fd5808e8700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd57c0aede0 0x7fd57c0af240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35400/0 (socket says 172.21.15.92:35400) 2023-10-06T10:45:59.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.168+0000 7fd5808e8700 1 -- 172.21.15.92:0/2569165364 learned_addr learned my addr 172.21.15.92:0/2569165364 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:59.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.168+0000 7fd57b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd57c06f0b0 0x7fd57c0b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37994/0 (socket says 172.21.15.92:37994) 2023-10-06T10:45:59.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.168+0000 7fd57bfff700 1 --2- 172.21.15.92:0/2569165364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd57c06ece0 0x7fd57c0b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.168+0000 7fd57b7fe700 1 -- 172.21.15.92:0/2569165364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd57c06ece0 msgr2=0x7fd57c0b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:59.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.168+0000 7fd57b7fe700 1 --2- 172.21.15.92:0/2569165364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd57c06ece0 0x7fd57c0b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.168+0000 7fd57b7fe700 1 -- 172.21.15.92:0/2569165364 >> v1:172.21.15.92:6789/0 conn(0x7fd57c0aede0 legacy=0x7fd57c0af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.168+0000 7fd57b7fe700 1 -- 172.21.15.92:0/2569165364 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57c0b2a00 con 0x7fd57c06f0b0 2023-10-06T10:45:59.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.168+0000 7fd57bfff700 1 --2- 172.21.15.92:0/2569165364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd57c06ece0 0x7fd57c0b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:59.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.168+0000 7fd57b7fe700 1 --2- 172.21.15.92:0/2569165364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c06f0b0 0x7fd57c0b08a0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fd570009910 tx=0x7fd5700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80c5416029b60ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:59.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.168+0000 7fd57affd700 1 -- 172.21.15.92:0/2569165364 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57000b820 con 0x7fd57c06f0b0 2023-10-06T10:45:59.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.168+0000 7fd57affd700 1 -- 172.21.15.92:0/2569165364 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57000b980 con 0x7fd57c06f0b0 2023-10-06T10:45:59.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.168+0000 7fd57affd700 1 -- 172.21.15.92:0/2569165364 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57000bd20 con 0x7fd57c06f0b0 2023-10-06T10:45:59.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.169+0000 7fd58234b700 1 -- 172.21.15.92:0/2569165364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c06f0b0 msgr2=0x7fd57c0b08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.169+0000 7fd58234b700 1 --2- 172.21.15.92:0/2569165364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c06f0b0 0x7fd57c0b08a0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fd570009910 tx=0x7fd5700091a0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.169+0000 7fd58234b700 1 -- 172.21.15.92:0/2569165364 shutdown_connections 2023-10-06T10:45:59.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.169+0000 7fd58234b700 1 --2- 172.21.15.92:0/2569165364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd57c06ece0 0x7fd57c0b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.169+0000 7fd58234b700 1 --2- 172.21.15.92:0/2569165364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c06f0b0 0x7fd57c0b08a0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.169+0000 7fd58234b700 1 -- 172.21.15.92:0/2569165364 >> 172.21.15.92:0/2569165364 conn(0x7fd57c0682f0 msgr2=0x7fd57c0686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:59.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.169+0000 7fd58234b700 1 -- 172.21.15.92:0/2569165364 shutdown_connections 2023-10-06T10:45:59.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.169+0000 7fd58234b700 1 -- 172.21.15.92:0/2569165364 wait complete. 2023-10-06T10:45:59.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.170+0000 7fd58234b700 1 Processor -- start 2023-10-06T10:45:59.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.170+0000 7fd58234b700 1 -- start start 2023-10-06T10:45:59.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.171+0000 7fd58234b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd57c06ece0 0x7fd57c148840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.171+0000 7fd58234b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c06f0b0 0x7fd57c0c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.171+0000 7fd58234b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd57c0aede0 0x7fd57c15b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.171+0000 7fd58234b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c1584a0 con 0x7fd57c06f0b0 2023-10-06T10:45:59.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.171+0000 7fd58234b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c157890 con 0x7fd57c0aede0 2023-10-06T10:45:59.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.171+0000 7fd58234b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c157b70 con 0x7fd57c06ece0 2023-10-06T10:45:59.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.171+0000 7fd57b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c06f0b0 0x7fd57c0c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.171+0000 7fd57b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c06f0b0 0x7fd57c0c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38000/0 (socket says 172.21.15.92:38000) 2023-10-06T10:45:59.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.171+0000 7fd57b7fe700 1 -- 172.21.15.92:0/4240354276 learned_addr learned my addr 172.21.15.92:0/4240354276 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:59.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.171+0000 7fd57bfff700 1 --2- 172.21.15.92:0/4240354276 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd57c06ece0 0x7fd57c148840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.171+0000 7fd57b7fe700 1 -- 172.21.15.92:0/4240354276 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd57c06ece0 msgr2=0x7fd57c148840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.171+0000 7fd57b7fe700 1 --2- 172.21.15.92:0/4240354276 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd57c06ece0 0x7fd57c148840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.171+0000 7fd57b7fe700 1 -- 172.21.15.92:0/4240354276 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd57c0aede0 msgr2=0x7fd57c15b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.171+0000 7fd5808e8700 1 --2- 172.21.15.92:0/4240354276 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd57c0aede0 0x7fd57c15b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.171+0000 7fd57b7fe700 1 --2- 172.21.15.92:0/4240354276 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd57c0aede0 0x7fd57c15b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.171+0000 7fd57b7fe700 1 -- 172.21.15.92:0/4240354276 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd570009420 con 0x7fd57c06f0b0 2023-10-06T10:45:59.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.171+0000 7fd57b7fe700 1 --2- 172.21.15.92:0/4240354276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c06f0b0 0x7fd57c0c9b80 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fd57c089380 tx=0x7fd570017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:59.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.171+0000 7fd563fff700 1 -- 172.21.15.92:0/4240354276 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570024440 con 0x7fd57c06f0b0 2023-10-06T10:45:59.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.171+0000 7fd563fff700 1 -- 172.21.15.92:0/4240354276 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5700245c0 con 0x7fd57c06f0b0 2023-10-06T10:45:59.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.171+0000 7fd58234b700 1 -- 172.21.15.92:0/4240354276 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57c157e50 con 0x7fd57c06f0b0 2023-10-06T10:45:59.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.171+0000 7fd563fff700 1 -- 172.21.15.92:0/4240354276 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570024980 con 0x7fd57c06f0b0 2023-10-06T10:45:59.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.171+0000 7fd58234b700 1 -- 172.21.15.92:0/4240354276 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57c148d80 con 0x7fd57c06f0b0 2023-10-06T10:45:59.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.172+0000 7fd58234b700 1 -- 172.21.15.92:0/4240354276 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd57c064a90 con 0x7fd57c06f0b0 2023-10-06T10:45:59.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.173+0000 7fd563fff700 1 -- 172.21.15.92:0/4240354276 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd57001a420 con 0x7fd57c06f0b0 2023-10-06T10:45:59.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.173+0000 7fd563fff700 1 --2- 172.21.15.92:0/4240354276 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd568066f30 0x7fd5680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.173+0000 7fd563fff700 1 -- 172.21.15.92:0/4240354276 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7fd570086500 con 0x7fd57c06f0b0 2023-10-06T10:45:59.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.173+0000 7fd57bfff700 1 --2- 172.21.15.92:0/4240354276 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd568066f30 0x7fd5680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.173+0000 7fd563fff700 1 --2- 172.21.15.92:0/4240354276 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x7fd56806d990 0x7fd56806fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.173+0000 7fd563fff700 1 -- 172.21.15.92:0/4240354276 --> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd568070490 con 0x7fd56806d990 2023-10-06T10:45:59.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.173+0000 7fd563fff700 1 -- 172.21.15.92:0/4240354276 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd570088b40 con 0x7fd57c06f0b0 2023-10-06T10:45:59.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.173+0000 7fd5808e8700 1 --2- 172.21.15.92:0/4240354276 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x7fd56806d990 0x7fd56806fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.174+0000 7fd57bfff700 1 --2- 172.21.15.92:0/4240354276 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd568066f30 0x7fd5680693e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd56c006fd0 tx=0x7fd56c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:59.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.174+0000 7fd5808e8700 1 --2- 172.21.15.92:0/4240354276 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x7fd56806d990 0x7fd56806fdc0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:59.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.174+0000 7fd563fff700 1 -- 172.21.15.92:0/4240354276 <== osd.0 v2:172.21.15.92:6800/90604 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fd568070490 con 0x7fd56806d990 2023-10-06T10:45:59.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.188+0000 7fd58234b700 1 -- 172.21.15.92:0/4240354276 --> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd57c0cd470 con 0x7fd56806d990 2023-10-06T10:45:59.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.188+0000 7fd563fff700 1 -- 172.21.15.92:0/4240354276 <== osd.0 v2:172.21.15.92:6800/90604 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd57c0cd470 con 0x7fd56806d990 2023-10-06T10:45:59.190 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:59.190 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T10:45:59.190 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:59.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.188+0000 7fd58234b700 1 -- 172.21.15.92:0/4240354276 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x7fd56806d990 msgr2=0x7fd56806fdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.188+0000 7fd58234b700 1 --2- 172.21.15.92:0/4240354276 >> [v2:172.21.15.92:6800/90604,v1:172.21.15.92:6801/90604] conn(0x7fd56806d990 0x7fd56806fdc0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.188+0000 7fd58234b700 1 -- 172.21.15.92:0/4240354276 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd568066f30 msgr2=0x7fd5680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.188+0000 7fd58234b700 1 --2- 172.21.15.92:0/4240354276 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd568066f30 0x7fd5680693e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd56c006fd0 tx=0x7fd56c008040 comp rx=0 tx=0).stop 2023-10-06T10:45:59.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.188+0000 7fd58234b700 1 -- 172.21.15.92:0/4240354276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c06f0b0 msgr2=0x7fd57c0c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.188+0000 7fd58234b700 1 --2- 172.21.15.92:0/4240354276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c06f0b0 0x7fd57c0c9b80 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fd57c089380 tx=0x7fd570017000 comp rx=0 tx=0).stop 2023-10-06T10:45:59.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.188+0000 7fd5808e8700 1 -- 172.21.15.92:0/4240354276 reap_dead start 2023-10-06T10:45:59.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.188+0000 7fd58234b700 1 -- 172.21.15.92:0/4240354276 shutdown_connections 2023-10-06T10:45:59.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.188+0000 7fd58234b700 1 -- 172.21.15.92:0/4240354276 >> 172.21.15.92:0/4240354276 conn(0x7fd57c0682f0 msgr2=0x7fd57c0b5580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:59.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.188+0000 7fd58234b700 1 -- 172.21.15.92:0/4240354276 shutdown_connections 2023-10-06T10:45:59.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.188+0000 7fd58234b700 1 -- 172.21.15.92:0/4240354276 wait complete. 2023-10-06T10:45:59.201 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-06T10:45:59.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.315+0000 7f046339d700 1 Processor -- start 2023-10-06T10:45:59.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.315+0000 7f046339d700 1 -- start start 2023-10-06T10:45:59.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.316+0000 7f046339d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f045c0cd330 0x7f045c0ce980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.316+0000 7f046339d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f045c0c6220 0x7f045c0ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.316+0000 7f046339d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f045c0cc800 con 0x7f045c159470 2023-10-06T10:45:59.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.316+0000 7f046339d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f045c0cbc30 con 0x7f045c0c6220 2023-10-06T10:45:59.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.316+0000 7f046339d700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f045c0cf400 con 0x7f045c0cd330 2023-10-06T10:45:59.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.316+0000 7f0462b9c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f045c0c6220 0x7f045c0ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.316+0000 7f046239b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f045c159470 0x7f045c0ccc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35404/0 (socket says 172.21.15.92:35404) 2023-10-06T10:45:59.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.316+0000 7f046239b700 1 -- 172.21.15.92:0/1487771813 learned_addr learned my addr 172.21.15.92:0/1487771813 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:59.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.316+0000 7f0462b9c700 1 -- 172.21.15.92:0/1487771813 >> v2:172.21.15.124:3300/0 conn(0x7f045c0cd330 msgr2=0x7f045c0ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:59.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.316+0000 7f0462b9c700 1 --2- 172.21.15.92:0/1487771813 >> v2:172.21.15.124:3300/0 conn(0x7f045c0cd330 0x7f045c0ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.316+0000 7f0462b9c700 1 -- 172.21.15.92:0/1487771813 >> v1:172.21.15.92:6789/0 conn(0x7f045c159470 legacy=0x7f045c0ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.316+0000 7f0461399700 1 -- 172.21.15.92:0/1487771813 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3789446860 0 0) 0x7f045c0cc800 con 0x7f045c159470 2023-10-06T10:45:59.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.317+0000 7f0462b9c700 1 -- 172.21.15.92:0/1487771813 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045c0cbed0 con 0x7f045c0c6220 2023-10-06T10:45:59.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.317+0000 7f0462b9c700 1 --2- 172.21.15.92:0/1487771813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f045c0c6220 0x7f045c0ceec0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f045c06a6f0 tx=0x7f0454009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8a4d58fb92bae1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:59.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.317+0000 7f0461399700 1 -- 172.21.15.92:0/1487771813 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045400e070 con 0x7f045c0c6220 2023-10-06T10:45:59.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.317+0000 7f0461399700 1 -- 172.21.15.92:0/1487771813 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045400bc30 con 0x7f045c0c6220 2023-10-06T10:45:59.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.317+0000 7f0461399700 1 -- 172.21.15.92:0/1487771813 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0454012430 con 0x7f045c0c6220 2023-10-06T10:45:59.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.317+0000 7f046339d700 1 -- 172.21.15.92:0/1487771813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f045c0c6220 msgr2=0x7f045c0ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.317+0000 7f046339d700 1 --2- 172.21.15.92:0/1487771813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f045c0c6220 0x7f045c0ceec0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f045c06a6f0 tx=0x7f0454009780 comp rx=0 tx=0).stop 2023-10-06T10:45:59.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.317+0000 7f046339d700 1 -- 172.21.15.92:0/1487771813 shutdown_connections 2023-10-06T10:45:59.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.317+0000 7f046339d700 1 --2- 172.21.15.92:0/1487771813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f045c0c6220 0x7f045c0ceec0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.317+0000 7f046339d700 1 --2- 172.21.15.92:0/1487771813 >> v2:172.21.15.124:3300/0 conn(0x7f045c0cd330 0x7f045c0ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.317+0000 7f046339d700 1 -- 172.21.15.92:0/1487771813 >> 172.21.15.92:0/1487771813 conn(0x7f045c0995c0 msgr2=0x7f045c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:59.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.317+0000 7f046339d700 1 -- 172.21.15.92:0/1487771813 shutdown_connections 2023-10-06T10:45:59.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.317+0000 7f046339d700 1 -- 172.21.15.92:0/1487771813 wait complete. 2023-10-06T10:45:59.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.318+0000 7f046339d700 1 Processor -- start 2023-10-06T10:45:59.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.318+0000 7f046339d700 1 -- start start 2023-10-06T10:45:59.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.319+0000 7f046339d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f045c0c6220 0x7f045c15bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.319+0000 7f046339d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f045c0cd330 0x7f045c1549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.319+0000 7f046339d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f045c159470 0x7f045c154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.319+0000 7f046339d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045c155470 con 0x7f045c159470 2023-10-06T10:45:59.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.319+0000 7f046339d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045c15a810 con 0x7f045c0cd330 2023-10-06T10:45:59.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.319+0000 7f046339d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f045c15aaf0 con 0x7f045c0c6220 2023-10-06T10:45:59.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.319+0000 7f0462b9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f045c159470 0x7f045c154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.319+0000 7f0462b9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f045c159470 0x7f045c154f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38006/0 (socket says 172.21.15.92:38006) 2023-10-06T10:45:59.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.319+0000 7f0462b9c700 1 -- 172.21.15.92:0/1106170998 learned_addr learned my addr 172.21.15.92:0/1106170998 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:59.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.319+0000 7f0462b9c700 1 -- 172.21.15.92:0/1106170998 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f045c0c6220 msgr2=0x7f045c15bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.319+0000 7f0462b9c700 1 --2- 172.21.15.92:0/1106170998 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f045c0c6220 0x7f045c15bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.319+0000 7f0462b9c700 1 -- 172.21.15.92:0/1106170998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f045c0cd330 msgr2=0x7f045c1549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.319+0000 7f0461b9a700 1 --2- 172.21.15.92:0/1106170998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f045c0cd330 0x7f045c1549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.320+0000 7f0462b9c700 1 --2- 172.21.15.92:0/1106170998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f045c0cd330 0x7f045c1549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.320+0000 7f0462b9c700 1 -- 172.21.15.92:0/1106170998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0454009a20 con 0x7f045c159470 2023-10-06T10:45:59.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.320+0000 7f0462b9c700 1 --2- 172.21.15.92:0/1106170998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f045c159470 0x7f045c154f30 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f045400b6c0 tx=0x7f0454019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:59.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.320+0000 7f04527fc700 1 -- 172.21.15.92:0/1106170998 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04540070a0 con 0x7f045c159470 2023-10-06T10:45:59.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.320+0000 7f04527fc700 1 -- 172.21.15.92:0/1106170998 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045401b600 con 0x7f045c159470 2023-10-06T10:45:59.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.320+0000 7f046339d700 1 -- 172.21.15.92:0/1106170998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f045c15add0 con 0x7f045c159470 2023-10-06T10:45:59.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.320+0000 7f04527fc700 1 -- 172.21.15.92:0/1106170998 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045401b9c0 con 0x7f045c159470 2023-10-06T10:45:59.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.320+0000 7f046339d700 1 -- 172.21.15.92:0/1106170998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f045c155e60 con 0x7f045c159470 2023-10-06T10:45:59.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.321+0000 7f046339d700 1 -- 172.21.15.92:0/1106170998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f045c156110 con 0x7f045c159470 2023-10-06T10:45:59.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.321+0000 7f04527fc700 1 -- 172.21.15.92:0/1106170998 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0454003050 con 0x7f045c159470 2023-10-06T10:45:59.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.322+0000 7f04527fc700 1 --2- 172.21.15.92:0/1106170998 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0440066f30 0x7f04400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.322+0000 7f04527fc700 1 -- 172.21.15.92:0/1106170998 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f04540862f0 con 0x7f045c159470 2023-10-06T10:45:59.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.322+0000 7f046239b700 1 --2- 172.21.15.92:0/1106170998 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0440066f30 0x7f04400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.322+0000 7f046239b700 1 --2- 172.21.15.92:0/1106170998 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0440066f30 0x7f04400693e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0458001640 tx=0x7f0458007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:59.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.326+0000 7f04527fc700 1 -- 172.21.15.92:0/1106170998 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f045c156110 con 0x7f045c159470 2023-10-06T10:45:59.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.510+0000 7f046339d700 1 -- 172.21.15.92:0/1106170998 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f045c156110 con 0x7f0440066f30 2023-10-06T10:45:59.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.510+0000 7f04527fc700 1 -- 172.21.15.92:0/1106170998 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f045c156110 con 0x7f0440066f30 2023-10-06T10:45:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.512+0000 7f046339d700 1 -- 172.21.15.92:0/1106170998 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0440066f30 msgr2=0x7f04400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.512+0000 7f046339d700 1 --2- 172.21.15.92:0/1106170998 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0440066f30 0x7f04400693e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0458001640 tx=0x7f0458007a40 comp rx=0 tx=0).stop 2023-10-06T10:45:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.512+0000 7f046339d700 1 -- 172.21.15.92:0/1106170998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f045c159470 msgr2=0x7f045c154f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.512+0000 7f046339d700 1 --2- 172.21.15.92:0/1106170998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f045c159470 0x7f045c154f30 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f045400b6c0 tx=0x7f0454019000 comp rx=0 tx=0).stop 2023-10-06T10:45:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.513+0000 7f046339d700 1 -- 172.21.15.92:0/1106170998 shutdown_connections 2023-10-06T10:45:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.513+0000 7f046339d700 1 --2- 172.21.15.92:0/1106170998 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f0440066f30 0x7f04400693e0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.513+0000 7f046339d700 1 --2- 172.21.15.92:0/1106170998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f045c159470 0x7f045c154f30 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.513+0000 7f046339d700 1 --2- 172.21.15.92:0/1106170998 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f045c0c6220 0x7f045c15bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.513+0000 7f046339d700 1 --2- 172.21.15.92:0/1106170998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f045c0cd330 0x7f045c1549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.513+0000 7f046339d700 1 -- 172.21.15.92:0/1106170998 >> 172.21.15.92:0/1106170998 conn(0x7f045c0995c0 msgr2=0x7f045c0cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:59.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.513+0000 7f046339d700 1 -- 172.21.15.92:0/1106170998 shutdown_connections 2023-10-06T10:45:59.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.513+0000 7f046339d700 1 -- 172.21.15.92:0/1106170998 wait complete. 2023-10-06T10:45:59.516 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 0 to deep-scrub 2023-10-06T10:45:59.526 INFO:tasks.ceph:Scrubbing osd.1 2023-10-06T10:45:59.526 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-06T10:45:59.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.643+0000 7f6137fff700 1 Processor -- start 2023-10-06T10:45:59.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.643+0000 7f6137fff700 1 -- start start 2023-10-06T10:45:59.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.644+0000 7f6137fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6138153990 0x7f613815a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.644+0000 7f6137fff700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f61380cca70 0x7f61380d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.644+0000 7f6137fff700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61380cdbf0 con 0x7f61380d07f0 2023-10-06T10:45:59.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.644+0000 7f6137fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61380cb4c0 con 0x7f6138153990 2023-10-06T10:45:59.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.644+0000 7f6137fff700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f613815a8b0 con 0x7f61380cca70 2023-10-06T10:45:59.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.644+0000 7f6136ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6138153990 0x7f613815a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.644+0000 7f6136ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6138153990 0x7f613815a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38018/0 (socket says 172.21.15.92:38018) 2023-10-06T10:45:59.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.644+0000 7f6136ffd700 1 -- 172.21.15.92:0/938281489 learned_addr learned my addr 172.21.15.92:0/938281489 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:59.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.644+0000 7f61367fc700 1 --2- 172.21.15.92:0/938281489 >> v2:172.21.15.124:3300/0 conn(0x7f61380cca70 0x7f61380d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.644+0000 7f6136ffd700 1 -- 172.21.15.92:0/938281489 >> v2:172.21.15.124:3300/0 conn(0x7f61380cca70 msgr2=0x7f61380d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:59.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.644+0000 7f6136ffd700 1 --2- 172.21.15.92:0/938281489 >> v2:172.21.15.124:3300/0 conn(0x7f61380cca70 0x7f61380d02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.644+0000 7f6136ffd700 1 -- 172.21.15.92:0/938281489 >> v1:172.21.15.124:6789/0 conn(0x7f61380d07f0 legacy=0x7f613815cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:59.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.644+0000 7f6136ffd700 1 -- 172.21.15.92:0/938281489 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61380d0bf0 con 0x7f6138153990 2023-10-06T10:45:59.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.644+0000 7f61367fc700 1 --2- 172.21.15.92:0/938281489 >> v2:172.21.15.124:3300/0 conn(0x7f61380cca70 0x7f61380d02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:59.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.645+0000 7f6136ffd700 1 --2- 172.21.15.92:0/938281489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6138153990 0x7f613815a370 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f61380d0250 tx=0x7f6128009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c94738d7a43b9e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:59.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.645+0000 7f6135ffb700 1 -- 172.21.15.92:0/938281489 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612800b820 con 0x7f6138153990 2023-10-06T10:45:59.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.645+0000 7f6135ffb700 1 -- 172.21.15.92:0/938281489 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f612800b980 con 0x7f6138153990 2023-10-06T10:45:59.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.645+0000 7f6135ffb700 1 -- 172.21.15.92:0/938281489 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612800bd20 con 0x7f6138153990 2023-10-06T10:45:59.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.645+0000 7f6137fff700 1 -- 172.21.15.92:0/938281489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6138153990 msgr2=0x7f613815a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.645+0000 7f6137fff700 1 --2- 172.21.15.92:0/938281489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6138153990 0x7f613815a370 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f61380d0250 tx=0x7f6128009250 comp rx=0 tx=0).stop 2023-10-06T10:45:59.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.645+0000 7f6137fff700 1 -- 172.21.15.92:0/938281489 shutdown_connections 2023-10-06T10:45:59.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.645+0000 7f6137fff700 1 --2- 172.21.15.92:0/938281489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6138153990 0x7f613815a370 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.645+0000 7f6137fff700 1 --2- 172.21.15.92:0/938281489 >> v2:172.21.15.124:3300/0 conn(0x7f61380cca70 0x7f61380d02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.645+0000 7f6137fff700 1 -- 172.21.15.92:0/938281489 >> 172.21.15.92:0/938281489 conn(0x7f61380680c0 msgr2=0x7f6138068490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:59.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.645+0000 7f6137fff700 1 -- 172.21.15.92:0/938281489 shutdown_connections 2023-10-06T10:45:59.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.645+0000 7f6137fff700 1 -- 172.21.15.92:0/938281489 wait complete. 2023-10-06T10:45:59.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.646+0000 7f6137fff700 1 Processor -- start 2023-10-06T10:45:59.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.647+0000 7f6137fff700 1 -- start start 2023-10-06T10:45:59.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.647+0000 7f6137fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61380cca70 0x7f6138151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.647+0000 7f6137fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61380d07f0 0x7f61380b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.647+0000 7f6137fff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6138153990 0x7f61380b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.647+0000 7f6137fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61381695b0 con 0x7f61380cca70 2023-10-06T10:45:59.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.647+0000 7f6137fff700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613816a3d0 con 0x7f6138153990 2023-10-06T10:45:59.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.647+0000 7f6137fff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61381697c0 con 0x7f61380d07f0 2023-10-06T10:45:59.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.647+0000 7f6136ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61380cca70 0x7f6138151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.647+0000 7f6136ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61380cca70 0x7f6138151670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38020/0 (socket says 172.21.15.92:38020) 2023-10-06T10:45:59.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.647+0000 7f6136ffd700 1 -- 172.21.15.92:0/2731274054 learned_addr learned my addr 172.21.15.92:0/2731274054 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:59.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.647+0000 7f61367fc700 1 --2- 172.21.15.92:0/2731274054 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61380d07f0 0x7f61380b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.647+0000 7f61377fe700 1 --2- 172.21.15.92:0/2731274054 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6138153990 0x7f61380b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.648+0000 7f6136ffd700 1 -- 172.21.15.92:0/2731274054 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61380d07f0 msgr2=0x7f61380b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.648+0000 7f6136ffd700 1 --2- 172.21.15.92:0/2731274054 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61380d07f0 0x7f61380b5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.648+0000 7f6136ffd700 1 -- 172.21.15.92:0/2731274054 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6138153990 msgr2=0x7f61380b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.648+0000 7f6136ffd700 1 --2- 172.21.15.92:0/2731274054 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6138153990 0x7f61380b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.648+0000 7f6136ffd700 1 -- 172.21.15.92:0/2731274054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61280094f0 con 0x7f61380cca70 2023-10-06T10:45:59.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.648+0000 7f61377fe700 1 --2- 172.21.15.92:0/2731274054 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6138153990 0x7f61380b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:45:59.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.648+0000 7f6136ffd700 1 --2- 172.21.15.92:0/2731274054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61380cca70 0x7f6138151670 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f61280030c0 tx=0x7f6128016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:59.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.648+0000 7f611effd700 1 -- 172.21.15.92:0/2731274054 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6128011e60 con 0x7f61380cca70 2023-10-06T10:45:59.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.648+0000 7f611effd700 1 -- 172.21.15.92:0/2731274054 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6128020070 con 0x7f61380cca70 2023-10-06T10:45:59.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.648+0000 7f6137fff700 1 -- 172.21.15.92:0/2731274054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6138169aa0 con 0x7f61380cca70 2023-10-06T10:45:59.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.648+0000 7f611effd700 1 -- 172.21.15.92:0/2731274054 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612801b6a0 con 0x7f61380cca70 2023-10-06T10:45:59.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.648+0000 7f6137fff700 1 -- 172.21.15.92:0/2731274054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f613816a060 con 0x7f61380cca70 2023-10-06T10:45:59.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.649+0000 7f6137fff700 1 -- 172.21.15.92:0/2731274054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f613805f230 con 0x7f61380cca70 2023-10-06T10:45:59.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.649+0000 7f611effd700 1 -- 172.21.15.92:0/2731274054 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6128022030 con 0x7f61380cca70 2023-10-06T10:45:59.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.649+0000 7f611effd700 1 --2- 172.21.15.92:0/2731274054 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6124066f30 0x7f61240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.650+0000 7f611effd700 1 -- 172.21.15.92:0/2731274054 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f6128086500 con 0x7f61380cca70 2023-10-06T10:45:59.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.650+0000 7f61367fc700 1 --2- 172.21.15.92:0/2731274054 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6124066f30 0x7f61240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.650+0000 7f611effd700 1 --2- 172.21.15.92:0/2731274054 >> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] conn(0x7f612406d990 0x7f612406fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.650+0000 7f611effd700 1 -- 172.21.15.92:0/2731274054 --> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6124070490 con 0x7f612406d990 2023-10-06T10:45:59.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.650+0000 7f611effd700 1 -- 172.21.15.92:0/2731274054 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6128088b40 con 0x7f61380cca70 2023-10-06T10:45:59.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.650+0000 7f61377fe700 1 --2- 172.21.15.92:0/2731274054 >> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] conn(0x7f612406d990 0x7f612406fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.650+0000 7f61367fc700 1 --2- 172.21.15.92:0/2731274054 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6124066f30 0x7f61240693e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f61380b58f0 tx=0x7f612c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:59.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.650+0000 7f61377fe700 1 --2- 172.21.15.92:0/2731274054 >> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] conn(0x7f612406d990 0x7f612406fdc0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:59.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.651+0000 7f611effd700 1 -- 172.21.15.92:0/2731274054 <== osd.1 v2:172.21.15.92:6816/90606 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f6124070490 con 0x7f612406d990 2023-10-06T10:45:59.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.664+0000 7f6137fff700 1 -- 172.21.15.92:0/2731274054 --> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f61380ccfe0 con 0x7f612406d990 2023-10-06T10:45:59.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.664+0000 7f611effd700 1 -- 172.21.15.92:0/2731274054 <== osd.1 v2:172.21.15.92:6816/90606 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f61380ccfe0 con 0x7f612406d990 2023-10-06T10:45:59.666 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:45:59.667 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T10:45:59.667 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:45:59.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.665+0000 7f6137fff700 1 -- 172.21.15.92:0/2731274054 >> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] conn(0x7f612406d990 msgr2=0x7f612406fdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.665+0000 7f6137fff700 1 --2- 172.21.15.92:0/2731274054 >> [v2:172.21.15.92:6816/90606,v1:172.21.15.92:6817/90606] conn(0x7f612406d990 0x7f612406fdc0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.665+0000 7f6137fff700 1 -- 172.21.15.92:0/2731274054 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6124066f30 msgr2=0x7f61240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.665+0000 7f6137fff700 1 --2- 172.21.15.92:0/2731274054 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6124066f30 0x7f61240693e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f61380b58f0 tx=0x7f612c008040 comp rx=0 tx=0).stop 2023-10-06T10:45:59.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.665+0000 7f6137fff700 1 -- 172.21.15.92:0/2731274054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61380cca70 msgr2=0x7f6138151670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.665+0000 7f6137fff700 1 --2- 172.21.15.92:0/2731274054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61380cca70 0x7f6138151670 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f61280030c0 tx=0x7f6128016000 comp rx=0 tx=0).stop 2023-10-06T10:45:59.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.665+0000 7f61377fe700 1 -- 172.21.15.92:0/2731274054 reap_dead start 2023-10-06T10:45:59.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.665+0000 7f6137fff700 1 -- 172.21.15.92:0/2731274054 shutdown_connections 2023-10-06T10:45:59.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.665+0000 7f6137fff700 1 -- 172.21.15.92:0/2731274054 >> 172.21.15.92:0/2731274054 conn(0x7f61380680c0 msgr2=0x7f6138157570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:59.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.665+0000 7f6137fff700 1 -- 172.21.15.92:0/2731274054 shutdown_connections 2023-10-06T10:45:59.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.665+0000 7f6137fff700 1 -- 172.21.15.92:0/2731274054 wait complete. 2023-10-06T10:45:59.678 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-06T10:45:59.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.791+0000 7f8f72416700 1 Processor -- start 2023-10-06T10:45:59.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.791+0000 7f8f72416700 1 -- start start 2023-10-06T10:45:59.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.792+0000 7f8f72416700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8f6c159560 0x7f8f6c0aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.792+0000 7f8f72416700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f6c0b0760 0x7f8f6c0b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.792+0000 7f8f72416700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6c0b0c60 con 0x7f8f6c0b49a0 2023-10-06T10:45:59.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.792+0000 7f8f72416700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f6c0b0f40 con 0x7f8f6c159560 2023-10-06T10:45:59.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.792+0000 7f8f72416700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0b1220 con 0x7f8f6c0b0760 2023-10-06T10:45:59.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.793+0000 7f8f6bfff700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8f6c159560 0x7f8f6c0aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.793+0000 7f8f6b7fe700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f8f6c0b49a0 0x7f8f6c0aaa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:49156/0 (socket says 172.21.15.92:49156) 2023-10-06T10:45:59.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.793+0000 7f8f709b3700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f6c0b0760 0x7f8f6c0b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.793+0000 7f8f6b7fe700 1 -- 172.21.15.92:0/3358099438 learned_addr learned my addr 172.21.15.92:0/3358099438 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:45:59.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.793+0000 7f8f6bfff700 1 -- 172.21.15.92:0/3358099438 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f6c0b0760 msgr2=0x7f8f6c0b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:45:59.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.793+0000 7f8f6bfff700 1 --2- 172.21.15.92:0/3358099438 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f6c0b0760 0x7f8f6c0b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.793+0000 7f8f6bfff700 1 -- 172.21.15.92:0/3358099438 >> v1:172.21.15.124:6789/0 conn(0x7f8f6c0b49a0 legacy=0x7f8f6c0aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.793+0000 7f8f6bfff700 1 -- 172.21.15.92:0/3358099438 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6c0b1500 con 0x7f8f6c159560 2023-10-06T10:45:59.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.793+0000 7f8f6affd700 1 -- 172.21.15.92:0/3358099438 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3566004919 0 0) 0x7f8f6c0b0c60 con 0x7f8f6c0b49a0 2023-10-06T10:45:59.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.793+0000 7f8f6bfff700 1 --2- 172.21.15.92:0/3358099438 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f6c159560 0x7f8f6c0aa530 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009900 tx=0x7f8f5c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dddb9a0f8153eb0a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:59.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.793+0000 7f8f6affd700 1 -- 172.21.15.92:0/3358099438 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c005c10 con 0x7f8f6c159560 2023-10-06T10:45:59.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.793+0000 7f8f6affd700 1 -- 172.21.15.92:0/3358099438 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5c005d70 con 0x7f8f6c159560 2023-10-06T10:45:59.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.793+0000 7f8f6affd700 1 -- 172.21.15.92:0/3358099438 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c006260 con 0x7f8f6c159560 2023-10-06T10:45:59.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.794+0000 7f8f72416700 1 -- 172.21.15.92:0/3358099438 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f6c159560 msgr2=0x7f8f6c0aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.794+0000 7f8f72416700 1 --2- 172.21.15.92:0/3358099438 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f6c159560 0x7f8f6c0aa530 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009900 tx=0x7f8f5c009120 comp rx=0 tx=0).stop 2023-10-06T10:45:59.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.794+0000 7f8f72416700 1 -- 172.21.15.92:0/3358099438 shutdown_connections 2023-10-06T10:45:59.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.794+0000 7f8f72416700 1 --2- 172.21.15.92:0/3358099438 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f6c0b0760 0x7f8f6c0b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.794+0000 7f8f72416700 1 --2- 172.21.15.92:0/3358099438 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f6c159560 0x7f8f6c0aa530 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.794+0000 7f8f72416700 1 -- 172.21.15.92:0/3358099438 >> 172.21.15.92:0/3358099438 conn(0x7f8f6c0997f0 msgr2=0x7f8f6c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:59.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.794+0000 7f8f72416700 1 -- 172.21.15.92:0/3358099438 shutdown_connections 2023-10-06T10:45:59.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.794+0000 7f8f72416700 1 -- 172.21.15.92:0/3358099438 wait complete. 2023-10-06T10:45:59.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.795+0000 7f8f72416700 1 Processor -- start 2023-10-06T10:45:59.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.795+0000 7f8f72416700 1 -- start start 2023-10-06T10:45:59.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.796+0000 7f8f72416700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f6c0b0760 0x7f8f6c1473b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.796+0000 7f8f72416700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f6c0b49a0 0x7f8f6c09a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.796+0000 7f8f72416700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f6c159560 0x7f8f6c15ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.796+0000 7f8f72416700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0ae040 con 0x7f8f6c0b49a0 2023-10-06T10:45:59.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.796+0000 7f8f72416700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c1478f0 con 0x7f8f6c0b0760 2023-10-06T10:45:59.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.796+0000 7f8f72416700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0ad430 con 0x7f8f6c159560 2023-10-06T10:45:59.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.796+0000 7f8f6b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f6c0b49a0 0x7f8f6c09a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.796+0000 7f8f6b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f6c0b49a0 0x7f8f6c09a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38036/0 (socket says 172.21.15.92:38036) 2023-10-06T10:45:59.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.796+0000 7f8f6b7fe700 1 -- 172.21.15.92:0/1172813934 learned_addr learned my addr 172.21.15.92:0/1172813934 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:45:59.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.796+0000 7f8f6bfff700 1 --2- 172.21.15.92:0/1172813934 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f6c0b0760 0x7f8f6c1473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.796+0000 7f8f709b3700 1 --2- 172.21.15.92:0/1172813934 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f6c159560 0x7f8f6c15ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.796+0000 7f8f6b7fe700 1 -- 172.21.15.92:0/1172813934 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f6c159560 msgr2=0x7f8f6c15ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.796+0000 7f8f6b7fe700 1 --2- 172.21.15.92:0/1172813934 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f6c159560 0x7f8f6c15ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.796+0000 7f8f6b7fe700 1 -- 172.21.15.92:0/1172813934 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f6c0b0760 msgr2=0x7f8f6c1473b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.796+0000 7f8f6b7fe700 1 --2- 172.21.15.92:0/1172813934 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f6c0b0760 0x7f8f6c1473b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.796+0000 7f8f6b7fe700 1 -- 172.21.15.92:0/1172813934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5c0093c0 con 0x7f8f6c0b49a0 2023-10-06T10:45:59.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.796+0000 7f8f6b7fe700 1 --2- 172.21.15.92:0/1172813934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f6c0b49a0 0x7f8f6c09a8e0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000c7e0 tx=0x7f8f60007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:59.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.797+0000 7f8f53fff700 1 -- 172.21.15.92:0/1172813934 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f60005c00 con 0x7f8f6c0b49a0 2023-10-06T10:45:59.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.797+0000 7f8f53fff700 1 -- 172.21.15.92:0/1172813934 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f60005d60 con 0x7f8f6c0b49a0 2023-10-06T10:45:59.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.797+0000 7f8f53fff700 1 -- 172.21.15.92:0/1172813934 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f600072e0 con 0x7f8f6c0b49a0 2023-10-06T10:45:59.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.797+0000 7f8f72416700 1 -- 172.21.15.92:0/1172813934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6c0ad710 con 0x7f8f6c0b49a0 2023-10-06T10:45:59.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.797+0000 7f8f72416700 1 -- 172.21.15.92:0/1172813934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6c0adcd0 con 0x7f8f6c0b49a0 2023-10-06T10:45:59.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.798+0000 7f8f72416700 1 -- 172.21.15.92:0/1172813934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f6c064a90 con 0x7f8f6c0b49a0 2023-10-06T10:45:59.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.798+0000 7f8f53fff700 1 -- 172.21.15.92:0/1172813934 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8f6001b3b0 con 0x7f8f6c0b49a0 2023-10-06T10:45:59.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.798+0000 7f8f53fff700 1 --2- 172.21.15.92:0/1172813934 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8f58066f30 0x7f8f580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:45:59.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.799+0000 7f8f53fff700 1 -- 172.21.15.92:0/1172813934 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f8f600839f0 con 0x7f8f6c0b49a0 2023-10-06T10:45:59.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.799+0000 7f8f6bfff700 1 --2- 172.21.15.92:0/1172813934 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8f58066f30 0x7f8f580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:45:59.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.799+0000 7f8f6bfff700 1 --2- 172.21.15.92:0/1172813934 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8f58066f30 0x7f8f580693e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009160 tx=0x7f8f5c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:45:59.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.803+0000 7f8f53fff700 1 -- 172.21.15.92:0/1172813934 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f6004b5a0 con 0x7f8f6c0b49a0 2023-10-06T10:45:59.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.985+0000 7f8f72416700 1 -- 172.21.15.92:0/1172813934 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f6c0ae180 con 0x7f8f58066f30 2023-10-06T10:45:59.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.985+0000 7f8f53fff700 1 -- 172.21.15.92:0/1172813934 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8f6c0ae180 con 0x7f8f58066f30 2023-10-06T10:45:59.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.987+0000 7f8f72416700 1 -- 172.21.15.92:0/1172813934 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8f58066f30 msgr2=0x7f8f580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.987+0000 7f8f72416700 1 --2- 172.21.15.92:0/1172813934 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8f58066f30 0x7f8f580693e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009160 tx=0x7f8f5c016000 comp rx=0 tx=0).stop 2023-10-06T10:45:59.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.987+0000 7f8f72416700 1 -- 172.21.15.92:0/1172813934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f6c0b49a0 msgr2=0x7f8f6c09a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:45:59.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.987+0000 7f8f72416700 1 --2- 172.21.15.92:0/1172813934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f6c0b49a0 0x7f8f6c09a8e0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000c7e0 tx=0x7f8f60007d90 comp rx=0 tx=0).stop 2023-10-06T10:45:59.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.987+0000 7f8f72416700 1 -- 172.21.15.92:0/1172813934 shutdown_connections 2023-10-06T10:45:59.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.987+0000 7f8f72416700 1 --2- 172.21.15.92:0/1172813934 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f8f58066f30 0x7f8f580693e0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.987+0000 7f8f72416700 1 --2- 172.21.15.92:0/1172813934 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f6c159560 0x7f8f6c15ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.987+0000 7f8f72416700 1 --2- 172.21.15.92:0/1172813934 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f6c0b0760 0x7f8f6c1473b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.987+0000 7f8f72416700 1 --2- 172.21.15.92:0/1172813934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f6c0b49a0 0x7f8f6c09a8e0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:45:59.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.987+0000 7f8f72416700 1 -- 172.21.15.92:0/1172813934 >> 172.21.15.92:0/1172813934 conn(0x7f8f6c0997f0 msgr2=0x7f8f6c0cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:45:59.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.987+0000 7f8f72416700 1 -- 172.21.15.92:0/1172813934 shutdown_connections 2023-10-06T10:45:59.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:45:59.987+0000 7f8f72416700 1 -- 172.21.15.92:0/1172813934 wait complete. 2023-10-06T10:45:59.990 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 1 to deep-scrub 2023-10-06T10:46:00.001 INFO:tasks.ceph:Scrubbing osd.2 2023-10-06T10:46:00.001 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-06T10:46:00.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.116+0000 7f7ba3b98700 1 Processor -- start 2023-10-06T10:46:00.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.116+0000 7f7ba3b98700 1 -- start start 2023-10-06T10:46:00.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.117+0000 7f7ba3b98700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7b9c0a7be0 0x7f7b9c15a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.117+0000 7f7ba3b98700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b9c0ae570 0x7f7b9c0675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.117+0000 7f7ba3b98700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b9c0c9a50 con 0x7f7b9c067af0 2023-10-06T10:46:00.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.117+0000 7f7ba3b98700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b9c067f70 con 0x7f7b9c0a7be0 2023-10-06T10:46:00.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.117+0000 7f7ba3b98700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c0c9440 con 0x7f7b9c0ae570 2023-10-06T10:46:00.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.117+0000 7f7ba1934700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7b9c0a7be0 0x7f7b9c15a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.117+0000 7f7ba2135700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7b9c067af0 0x7f7b9c06e440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35420/0 (socket says 172.21.15.92:35420) 2023-10-06T10:46:00.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.117+0000 7f7ba2135700 1 -- 172.21.15.92:0/3922879975 learned_addr learned my addr 172.21.15.92:0/3922879975 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:46:00.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.117+0000 7f7ba1133700 1 --2- 172.21.15.92:0/3922879975 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b9c0ae570 0x7f7b9c0675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.117+0000 7f7ba1934700 1 -- 172.21.15.92:0/3922879975 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b9c0ae570 msgr2=0x7f7b9c0675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:00.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.117+0000 7f7ba1934700 1 --2- 172.21.15.92:0/3922879975 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b9c0ae570 0x7f7b9c0675b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.117+0000 7f7ba1934700 1 -- 172.21.15.92:0/3922879975 >> v1:172.21.15.92:6789/0 conn(0x7f7b9c067af0 legacy=0x7f7b9c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.118+0000 7f7ba1934700 1 -- 172.21.15.92:0/3922879975 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9c0ad0d0 con 0x7f7b9c0a7be0 2023-10-06T10:46:00.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.117+0000 7f7ba0932700 1 -- 172.21.15.92:0/3922879975 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 720296875 0 0) 0x7f7b9c0c9a50 con 0x7f7b9c067af0 2023-10-06T10:46:00.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.118+0000 7f7ba1133700 1 --2- 172.21.15.92:0/3922879975 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b9c0ae570 0x7f7b9c0675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:46:00.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.118+0000 7f7ba1934700 1 --2- 172.21.15.92:0/3922879975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b9c0a7be0 0x7f7b9c15a660 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f7b980098e0 tx=0x7f7b980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905d9f83a3b2c623 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:00.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.118+0000 7f7ba0932700 1 -- 172.21.15.92:0/3922879975 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b98009e80 con 0x7f7b9c0a7be0 2023-10-06T10:46:00.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.118+0000 7f7ba0932700 1 -- 172.21.15.92:0/3922879975 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9800b860 con 0x7f7b9c0a7be0 2023-10-06T10:46:00.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.118+0000 7f7ba0932700 1 -- 172.21.15.92:0/3922879975 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9800bbe0 con 0x7f7b9c0a7be0 2023-10-06T10:46:00.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.118+0000 7f7ba3b98700 1 -- 172.21.15.92:0/3922879975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b9c0a7be0 msgr2=0x7f7b9c15a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.118+0000 7f7ba3b98700 1 --2- 172.21.15.92:0/3922879975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b9c0a7be0 0x7f7b9c15a660 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f7b980098e0 tx=0x7f7b980091a0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.118+0000 7f7ba3b98700 1 -- 172.21.15.92:0/3922879975 shutdown_connections 2023-10-06T10:46:00.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.118+0000 7f7ba3b98700 1 --2- 172.21.15.92:0/3922879975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b9c0a7be0 0x7f7b9c15a660 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.118+0000 7f7ba3b98700 1 --2- 172.21.15.92:0/3922879975 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b9c0ae570 0x7f7b9c0675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.118+0000 7f7ba3b98700 1 -- 172.21.15.92:0/3922879975 >> 172.21.15.92:0/3922879975 conn(0x7f7b9c0cd1e0 msgr2=0x7f7b9c0cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:00.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.118+0000 7f7ba3b98700 1 -- 172.21.15.92:0/3922879975 shutdown_connections 2023-10-06T10:46:00.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.118+0000 7f7ba3b98700 1 -- 172.21.15.92:0/3922879975 wait complete. 2023-10-06T10:46:00.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.119+0000 7f7ba3b98700 1 Processor -- start 2023-10-06T10:46:00.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.120+0000 7f7ba3b98700 1 -- start start 2023-10-06T10:46:00.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.120+0000 7f7ba3b98700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b9c067af0 0x7f7b9c0b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.120+0000 7f7ba3b98700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b9c0a7be0 0x7f7b9c15c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.120+0000 7f7ba3b98700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b9c0ae570 0x7f7b9c06ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.120+0000 7f7ba3b98700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c14ef60 con 0x7f7b9c067af0 2023-10-06T10:46:00.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.120+0000 7f7ba3b98700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c150a10 con 0x7f7b9c0a7be0 2023-10-06T10:46:00.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.120+0000 7f7ba3b98700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c15ad90 con 0x7f7b9c0ae570 2023-10-06T10:46:00.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.120+0000 7f7ba1934700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b9c067af0 0x7f7b9c0b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.120+0000 7f7ba1934700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b9c067af0 0x7f7b9c0b0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38062/0 (socket says 172.21.15.92:38062) 2023-10-06T10:46:00.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.120+0000 7f7ba1934700 1 -- 172.21.15.92:0/2428074612 learned_addr learned my addr 172.21.15.92:0/2428074612 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:46:00.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.121+0000 7f7ba1133700 1 --2- 172.21.15.92:0/2428074612 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b9c0a7be0 0x7f7b9c15c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.121+0000 7f7ba2135700 1 --2- 172.21.15.92:0/2428074612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b9c0ae570 0x7f7b9c06ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.121+0000 7f7ba1934700 1 -- 172.21.15.92:0/2428074612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b9c0ae570 msgr2=0x7f7b9c06ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.121+0000 7f7ba1934700 1 --2- 172.21.15.92:0/2428074612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b9c0ae570 0x7f7b9c06ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.121+0000 7f7ba1934700 1 -- 172.21.15.92:0/2428074612 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b9c0a7be0 msgr2=0x7f7b9c15c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.121+0000 7f7ba1934700 1 --2- 172.21.15.92:0/2428074612 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b9c0a7be0 0x7f7b9c15c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.121+0000 7f7ba1934700 1 -- 172.21.15.92:0/2428074612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b98009420 con 0x7f7b9c067af0 2023-10-06T10:46:00.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.121+0000 7f7ba1133700 1 --2- 172.21.15.92:0/2428074612 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b9c0a7be0 0x7f7b9c15c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:46:00.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.121+0000 7f7ba2135700 1 --2- 172.21.15.92:0/2428074612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b9c0ae570 0x7f7b9c06ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:46:00.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.121+0000 7f7ba1934700 1 --2- 172.21.15.92:0/2428074612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b9c067af0 0x7f7b9c0b0990 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0ac6f0 tx=0x7f7b98017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:00.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.121+0000 7f7b91ffb700 1 -- 172.21.15.92:0/2428074612 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9800bbe0 con 0x7f7b9c067af0 2023-10-06T10:46:00.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.121+0000 7f7b91ffb700 1 -- 172.21.15.92:0/2428074612 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b98011420 con 0x7f7b9c067af0 2023-10-06T10:46:00.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.121+0000 7f7b91ffb700 1 -- 172.21.15.92:0/2428074612 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b980117a0 con 0x7f7b9c067af0 2023-10-06T10:46:00.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.121+0000 7f7ba3b98700 1 -- 172.21.15.92:0/2428074612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9c0cc0b0 con 0x7f7b9c067af0 2023-10-06T10:46:00.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.121+0000 7f7ba3b98700 1 -- 172.21.15.92:0/2428074612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9c0cb4a0 con 0x7f7b9c067af0 2023-10-06T10:46:00.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.122+0000 7f7ba3b98700 1 -- 172.21.15.92:0/2428074612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7b9c064a90 con 0x7f7b9c067af0 2023-10-06T10:46:00.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.123+0000 7f7b91ffb700 1 -- 172.21.15.92:0/2428074612 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7b98011bd0 con 0x7f7b9c067af0 2023-10-06T10:46:00.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.123+0000 7f7b91ffb700 1 --2- 172.21.15.92:0/2428074612 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7b80066f30 0x7f7b800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.123+0000 7f7b91ffb700 1 -- 172.21.15.92:0/2428074612 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f7b98086ac0 con 0x7f7b9c067af0 2023-10-06T10:46:00.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.123+0000 7f7ba1133700 1 --2- 172.21.15.92:0/2428074612 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7b80066f30 0x7f7b800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.123+0000 7f7b91ffb700 1 --2- 172.21.15.92:0/2428074612 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x7f7b8006d990 0x7f7b8006fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.123+0000 7f7b91ffb700 1 -- 172.21.15.92:0/2428074612 --> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b80070490 con 0x7f7b8006d990 2023-10-06T10:46:00.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.123+0000 7f7b91ffb700 1 -- 172.21.15.92:0/2428074612 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b98086e90 con 0x7f7b9c067af0 2023-10-06T10:46:00.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.123+0000 7f7ba2135700 1 --2- 172.21.15.92:0/2428074612 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x7f7b8006d990 0x7f7b8006fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.123+0000 7f7ba1133700 1 --2- 172.21.15.92:0/2428074612 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7b80066f30 0x7f7b800693e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c006f00 tx=0x7f7b8c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:00.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.123+0000 7f7ba2135700 1 --2- 172.21.15.92:0/2428074612 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x7f7b8006d990 0x7f7b8006fdc0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:00.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.124+0000 7f7b91ffb700 1 -- 172.21.15.92:0/2428074612 <== osd.2 v2:172.21.15.92:6824/90615 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f7b80070490 con 0x7f7b8006d990 2023-10-06T10:46:00.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.138+0000 7f7ba3b98700 1 -- 172.21.15.92:0/2428074612 --> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7b9c0afb30 con 0x7f7b8006d990 2023-10-06T10:46:00.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.138+0000 7f7b91ffb700 1 -- 172.21.15.92:0/2428074612 <== osd.2 v2:172.21.15.92:6824/90615 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7b9c0afb30 con 0x7f7b8006d990 2023-10-06T10:46:00.140 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:46:00.140 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T10:46:00.140 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:46:00.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.138+0000 7f7ba3b98700 1 -- 172.21.15.92:0/2428074612 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x7f7b8006d990 msgr2=0x7f7b8006fdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.138+0000 7f7ba3b98700 1 --2- 172.21.15.92:0/2428074612 >> [v2:172.21.15.92:6824/90615,v1:172.21.15.92:6825/90615] conn(0x7f7b8006d990 0x7f7b8006fdc0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.138+0000 7f7ba3b98700 1 -- 172.21.15.92:0/2428074612 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7b80066f30 msgr2=0x7f7b800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.138+0000 7f7ba3b98700 1 --2- 172.21.15.92:0/2428074612 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7b80066f30 0x7f7b800693e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c006f00 tx=0x7f7b8c008040 comp rx=0 tx=0).stop 2023-10-06T10:46:00.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.138+0000 7f7ba3b98700 1 -- 172.21.15.92:0/2428074612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b9c067af0 msgr2=0x7f7b9c0b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.138+0000 7f7ba3b98700 1 --2- 172.21.15.92:0/2428074612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b9c067af0 0x7f7b9c0b0990 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0ac6f0 tx=0x7f7b98017000 comp rx=0 tx=0).stop 2023-10-06T10:46:00.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.138+0000 7f7ba2135700 1 -- 172.21.15.92:0/2428074612 reap_dead start 2023-10-06T10:46:00.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.138+0000 7f7ba3b98700 1 -- 172.21.15.92:0/2428074612 shutdown_connections 2023-10-06T10:46:00.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.138+0000 7f7ba3b98700 1 -- 172.21.15.92:0/2428074612 >> 172.21.15.92:0/2428074612 conn(0x7f7b9c0cd1e0 msgr2=0x7f7b9c0ca4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:00.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.138+0000 7f7ba3b98700 1 -- 172.21.15.92:0/2428074612 shutdown_connections 2023-10-06T10:46:00.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.138+0000 7f7ba3b98700 1 -- 172.21.15.92:0/2428074612 wait complete. 2023-10-06T10:46:00.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-06T10:46:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.265+0000 7f9bed440700 1 Processor -- start 2023-10-06T10:46:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.265+0000 7f9bed440700 1 -- start start 2023-10-06T10:46:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.266+0000 7f9bed440700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9be8159470 0x7f9be80ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.266+0000 7f9bed440700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9be80c6220 0x7f9be80cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.266+0000 7f9bed440700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be80cd5d0 con 0x7f9be80cd160 2023-10-06T10:46:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.266+0000 7f9bed440700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be80cc800 con 0x7f9be8159470 2023-10-06T10:46:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.266+0000 7f9bed440700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be80cbc30 con 0x7f9be80c6220 2023-10-06T10:46:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.266+0000 7f9be7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9be8159470 0x7f9be80ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.266+0000 7f9be77fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9be80cd160 0x7f9be80ce980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35436/0 (socket says 172.21.15.92:35436) 2023-10-06T10:46:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.266+0000 7f9be7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9be8159470 0x7f9be80ccc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38076/0 (socket says 172.21.15.92:38076) 2023-10-06T10:46:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.266+0000 7f9be77fe700 1 -- 172.21.15.92:0/2370146710 learned_addr learned my addr 172.21.15.92:0/2370146710 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:46:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.266+0000 7f9becc3f700 1 --2- 172.21.15.92:0/2370146710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9be80c6220 0x7f9be80cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.266+0000 7f9be7fff700 1 -- 172.21.15.92:0/2370146710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9be80c6220 msgr2=0x7f9be80cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.266+0000 7f9be7fff700 1 --2- 172.21.15.92:0/2370146710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9be80c6220 0x7f9be80cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.266+0000 7f9be7fff700 1 -- 172.21.15.92:0/2370146710 >> v1:172.21.15.92:6789/0 conn(0x7f9be80cd160 legacy=0x7f9be80ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.266+0000 7f9be7fff700 1 -- 172.21.15.92:0/2370146710 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be80cbed0 con 0x7f9be8159470 2023-10-06T10:46:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.266+0000 7f9be6ffd700 1 -- 172.21.15.92:0/2370146710 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3311893202 0 0) 0x7f9be80cd5d0 con 0x7f9be80cd160 2023-10-06T10:46:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.267+0000 7f9be7fff700 1 --2- 172.21.15.92:0/2370146710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9be8159470 0x7f9be80ccc20 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80098d0 tx=0x7f9bd8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57c77d5074f5982 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.267+0000 7f9be6ffd700 1 -- 172.21.15.92:0/2370146710 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8005c10 con 0x7f9be8159470 2023-10-06T10:46:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.267+0000 7f9be6ffd700 1 -- 172.21.15.92:0/2370146710 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd8005d70 con 0x7f9be8159470 2023-10-06T10:46:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.267+0000 7f9be6ffd700 1 -- 172.21.15.92:0/2370146710 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8006260 con 0x7f9be8159470 2023-10-06T10:46:00.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.267+0000 7f9bed440700 1 -- 172.21.15.92:0/2370146710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9be8159470 msgr2=0x7f9be80ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.267+0000 7f9bed440700 1 --2- 172.21.15.92:0/2370146710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9be8159470 0x7f9be80ccc20 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80098d0 tx=0x7f9bd8009120 comp rx=0 tx=0).stop 2023-10-06T10:46:00.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.267+0000 7f9bed440700 1 -- 172.21.15.92:0/2370146710 shutdown_connections 2023-10-06T10:46:00.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.267+0000 7f9bed440700 1 --2- 172.21.15.92:0/2370146710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9be80c6220 0x7f9be80cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.267+0000 7f9bed440700 1 --2- 172.21.15.92:0/2370146710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9be8159470 0x7f9be80ccc20 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.267+0000 7f9bed440700 1 -- 172.21.15.92:0/2370146710 >> 172.21.15.92:0/2370146710 conn(0x7f9be80995c0 msgr2=0x7f9be806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:00.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.267+0000 7f9bed440700 1 -- 172.21.15.92:0/2370146710 shutdown_connections 2023-10-06T10:46:00.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.267+0000 7f9bed440700 1 -- 172.21.15.92:0/2370146710 wait complete. 2023-10-06T10:46:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.268+0000 7f9bed440700 1 Processor -- start 2023-10-06T10:46:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.269+0000 7f9bed440700 1 -- start start 2023-10-06T10:46:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.269+0000 7f9bed440700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9be80c6220 0x7f9be81562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.269+0000 7f9bed440700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9be80cd160 0x7f9be8153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.269+0000 7f9bed440700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9be8159470 0x7f9be80ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.269+0000 7f9bed440700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be8156a80 con 0x7f9be80c6220 2023-10-06T10:46:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.269+0000 7f9bed440700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be8169030 con 0x7f9be8159470 2023-10-06T10:46:00.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.269+0000 7f9bed440700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be8169b70 con 0x7f9be80cd160 2023-10-06T10:46:00.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.269+0000 7f9be7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9be80c6220 0x7f9be81562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.269+0000 7f9be7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9be80c6220 0x7f9be81562a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38080/0 (socket says 172.21.15.92:38080) 2023-10-06T10:46:00.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.269+0000 7f9be7fff700 1 -- 172.21.15.92:0/1891017135 learned_addr learned my addr 172.21.15.92:0/1891017135 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:46:00.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.270+0000 7f9be7fff700 1 -- 172.21.15.92:0/1891017135 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9be80cd160 msgr2=0x7f9be8153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.270+0000 7f9be7fff700 1 --2- 172.21.15.92:0/1891017135 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9be80cd160 0x7f9be8153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.270+0000 7f9be7fff700 1 -- 172.21.15.92:0/1891017135 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9be8159470 msgr2=0x7f9be80ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.270+0000 7f9be7fff700 1 --2- 172.21.15.92:0/1891017135 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9be8159470 0x7f9be80ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.270+0000 7f9be7fff700 1 -- 172.21.15.92:0/1891017135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd80093c0 con 0x7f9be80c6220 2023-10-06T10:46:00.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.270+0000 7f9be7fff700 1 --2- 172.21.15.92:0/1891017135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9be80c6220 0x7f9be81562a0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009f70 tx=0x7f9bd8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:00.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.270+0000 7f9bcffff700 1 -- 172.21.15.92:0/1891017135 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd801d6b0 con 0x7f9be80c6220 2023-10-06T10:46:00.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.270+0000 7f9bcffff700 1 -- 172.21.15.92:0/1891017135 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd801d810 con 0x7f9be80c6220 2023-10-06T10:46:00.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.270+0000 7f9bed440700 1 -- 172.21.15.92:0/1891017135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be816a000 con 0x7f9be80c6220 2023-10-06T10:46:00.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.270+0000 7f9bcffff700 1 -- 172.21.15.92:0/1891017135 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd801dbd0 con 0x7f9be80c6220 2023-10-06T10:46:00.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.270+0000 7f9bed440700 1 -- 172.21.15.92:0/1891017135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be8156830 con 0x7f9be80c6220 2023-10-06T10:46:00.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.271+0000 7f9bed440700 1 -- 172.21.15.92:0/1891017135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be805f230 con 0x7f9be80c6220 2023-10-06T10:46:00.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.271+0000 7f9bcffff700 1 -- 172.21.15.92:0/1891017135 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9bd801b480 con 0x7f9be80c6220 2023-10-06T10:46:00.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.272+0000 7f9bcffff700 1 --2- 172.21.15.92:0/1891017135 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9bd4066f30 0x7f9bd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.272+0000 7f9bcffff700 1 -- 172.21.15.92:0/1891017135 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f9bd80861d0 con 0x7f9be80c6220 2023-10-06T10:46:00.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.272+0000 7f9be77fe700 1 --2- 172.21.15.92:0/1891017135 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9bd4066f30 0x7f9bd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.272+0000 7f9be77fe700 1 --2- 172.21.15.92:0/1891017135 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9bd4066f30 0x7f9bd40693e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9be8156010 tx=0x7f9bdc007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:00.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.276+0000 7f9bcffff700 1 -- 172.21.15.92:0/1891017135 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bd80533f0 con 0x7f9be80c6220 2023-10-06T10:46:00.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.458+0000 7f9bed440700 1 -- 172.21.15.92:0/1891017135 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9be80cc1b0 con 0x7f9bd4066f30 2023-10-06T10:46:00.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.459+0000 7f9bcffff700 1 -- 172.21.15.92:0/1891017135 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9be80cc1b0 con 0x7f9bd4066f30 2023-10-06T10:46:00.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.460+0000 7f9bed440700 1 -- 172.21.15.92:0/1891017135 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9bd4066f30 msgr2=0x7f9bd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.460+0000 7f9bed440700 1 --2- 172.21.15.92:0/1891017135 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9bd4066f30 0x7f9bd40693e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9be8156010 tx=0x7f9bdc007a40 comp rx=0 tx=0).stop 2023-10-06T10:46:00.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.460+0000 7f9bed440700 1 -- 172.21.15.92:0/1891017135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9be80c6220 msgr2=0x7f9be81562a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.460+0000 7f9bed440700 1 --2- 172.21.15.92:0/1891017135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9be80c6220 0x7f9be81562a0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009f70 tx=0x7f9bd8016000 comp rx=0 tx=0).stop 2023-10-06T10:46:00.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.460+0000 7f9bed440700 1 -- 172.21.15.92:0/1891017135 shutdown_connections 2023-10-06T10:46:00.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.461+0000 7f9bed440700 1 --2- 172.21.15.92:0/1891017135 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f9bd4066f30 0x7f9bd40693e0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.461+0000 7f9bed440700 1 --2- 172.21.15.92:0/1891017135 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9be8159470 0x7f9be80ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.461+0000 7f9bed440700 1 --2- 172.21.15.92:0/1891017135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9be80c6220 0x7f9be81562a0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.461+0000 7f9bed440700 1 --2- 172.21.15.92:0/1891017135 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9be80cd160 0x7f9be8153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.461+0000 7f9bed440700 1 -- 172.21.15.92:0/1891017135 >> 172.21.15.92:0/1891017135 conn(0x7f9be80995c0 msgr2=0x7f9be8154d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:00.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.461+0000 7f9bed440700 1 -- 172.21.15.92:0/1891017135 shutdown_connections 2023-10-06T10:46:00.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.461+0000 7f9bed440700 1 -- 172.21.15.92:0/1891017135 wait complete. 2023-10-06T10:46:00.465 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 2 to deep-scrub 2023-10-06T10:46:00.474 INFO:tasks.ceph:Scrubbing osd.3 2023-10-06T10:46:00.474 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-06T10:46:00.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.590+0000 7f4547e58700 1 Processor -- start 2023-10-06T10:46:00.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.590+0000 7f4547e58700 1 -- start start 2023-10-06T10:46:00.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.591+0000 7f4547e58700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45400b2360 0x7f45400b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.591+0000 7f4547e58700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45400b27d0 con 0x7f454006ece0 2023-10-06T10:46:00.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.591+0000 7f4547e58700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45400b2ce0 con 0x7f45400aede0 2023-10-06T10:46:00.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.591+0000 7f4547e58700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454006fb40 con 0x7f45400b2360 2023-10-06T10:46:00.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.591+0000 7f4545bf4700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f454006ece0 0x7f454006f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35446/0 (socket says 172.21.15.92:35446) 2023-10-06T10:46:00.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.591+0000 7f4545bf4700 1 -- 172.21.15.92:0/650565691 learned_addr learned my addr 172.21.15.92:0/650565691 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:46:00.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f45453f3700 1 --2- 172.21.15.92:0/650565691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45400b2360 0x7f45400b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f4544bf2700 1 -- 172.21.15.92:0/650565691 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4151707950 0 0) 0x7f45400b27d0 con 0x7f454006ece0 2023-10-06T10:46:00.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f4544bf2700 1 -- 172.21.15.92:0/650565691 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f452c003570 con 0x7f454006ece0 2023-10-06T10:46:00.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f4544bf2700 1 -- 172.21.15.92:0/650565691 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2149847879 0 0) 0x7f452c003570 con 0x7f454006ece0 2023-10-06T10:46:00.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f4544bf2700 1 -- 172.21.15.92:0/650565691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45400b2360 msgr2=0x7f45400b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:00.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f4544bf2700 1 --2- 172.21.15.92:0/650565691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45400b2360 0x7f45400b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f4544bf2700 1 -- 172.21.15.92:0/650565691 >> v1:172.21.15.124:6789/0 conn(0x7f45400aede0 legacy=0x7f45400af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f4544bf2700 1 -- 172.21.15.92:0/650565691 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45400b0410 con 0x7f454006ece0 2023-10-06T10:46:00.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f4544bf2700 1 -- 172.21.15.92:0/650565691 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f453c001470 con 0x7f454006ece0 2023-10-06T10:46:00.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f4544bf2700 1 -- 172.21.15.92:0/650565691 >> v1:172.21.15.92:6789/0 conn(0x7f454006ece0 legacy=0x7f454006f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f4544bf2700 1 --2- 172.21.15.92:0/650565691 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f452c003a90 0x7f452c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f4544bf2700 1 --2- 172.21.15.92:0/650565691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f452c004810 0x7f452c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f4544bf2700 1 --2- 172.21.15.92:0/650565691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45400b2360 0x7f452c0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f4544bf2700 1 -- 172.21.15.92:0/650565691 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c00bb60 con 0x7f452c004810 2023-10-06T10:46:00.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f4544bf2700 1 -- 172.21.15.92:0/650565691 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c00bcd0 con 0x7f452c003a90 2023-10-06T10:46:00.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f4544bf2700 1 -- 172.21.15.92:0/650565691 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452c00be90 con 0x7f45400b2360 2023-10-06T10:46:00.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f45453f3700 1 --2- 172.21.15.92:0/650565691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f452c004810 0x7f452c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f4545bf4700 1 --2- 172.21.15.92:0/650565691 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f452c003a90 0x7f452c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f45463f5700 1 --2- 172.21.15.92:0/650565691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45400b2360 0x7f452c0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f45453f3700 1 -- 172.21.15.92:0/650565691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45400b2360 msgr2=0x7f452c0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:00.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f45453f3700 1 --2- 172.21.15.92:0/650565691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45400b2360 0x7f452c0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f45453f3700 1 -- 172.21.15.92:0/650565691 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f452c003a90 msgr2=0x7f452c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:00.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f45453f3700 1 --2- 172.21.15.92:0/650565691 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f452c003a90 0x7f452c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.592+0000 7f45453f3700 1 -- 172.21.15.92:0/650565691 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c003570 con 0x7f452c004810 2023-10-06T10:46:00.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.593+0000 7f45453f3700 1 --2- 172.21.15.92:0/650565691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f452c004810 0x7f452c004c80 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f4530009c00 tx=0x7f45300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa66b4c5f0356638 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:00.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.593+0000 7f4544bf2700 1 -- 172.21.15.92:0/650565691 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453000e710 con 0x7f452c004810 2023-10-06T10:46:00.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.593+0000 7f4544bf2700 1 -- 172.21.15.92:0/650565691 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453000e870 con 0x7f452c004810 2023-10-06T10:46:00.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.593+0000 7f4544bf2700 1 -- 172.21.15.92:0/650565691 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453000ec30 con 0x7f452c004810 2023-10-06T10:46:00.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.593+0000 7f4547e58700 1 -- 172.21.15.92:0/650565691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f452c004810 msgr2=0x7f452c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.593+0000 7f4547e58700 1 --2- 172.21.15.92:0/650565691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f452c004810 0x7f452c004c80 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f4530009c00 tx=0x7f45300091a0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.593+0000 7f45463f5700 1 -- 172.21.15.92:0/650565691 reap_dead start 2023-10-06T10:46:00.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.593+0000 7f4547e58700 1 -- 172.21.15.92:0/650565691 shutdown_connections 2023-10-06T10:46:00.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.593+0000 7f4547e58700 1 -- 172.21.15.92:0/650565691 >> 172.21.15.92:0/650565691 conn(0x7f45400682f0 msgr2=0x7f45400686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:00.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.593+0000 7f4547e58700 1 -- 172.21.15.92:0/650565691 shutdown_connections 2023-10-06T10:46:00.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.593+0000 7f4547e58700 1 -- 172.21.15.92:0/650565691 wait complete. 2023-10-06T10:46:00.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.595+0000 7f4547e58700 1 Processor -- start 2023-10-06T10:46:00.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.595+0000 7f4547e58700 1 -- start start 2023-10-06T10:46:00.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.595+0000 7f4547e58700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f452c004810 0x7f45401566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.595+0000 7f4547e58700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45400ca1f0 0x7f45401576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.595+0000 7f4547e58700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4540157bf0 0x7f45400c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.595+0000 7f4547e58700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4540156c30 con 0x7f45400ca1f0 2023-10-06T10:46:00.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.595+0000 7f4547e58700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45400c74a0 con 0x7f4540157bf0 2023-10-06T10:46:00.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.595+0000 7f4547e58700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45400c8020 con 0x7f452c004810 2023-10-06T10:46:00.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.595+0000 7f45453f3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45400ca1f0 0x7f45401576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.595+0000 7f45453f3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45400ca1f0 0x7f45401576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38088/0 (socket says 172.21.15.92:38088) 2023-10-06T10:46:00.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.595+0000 7f45453f3700 1 -- 172.21.15.92:0/2246007599 learned_addr learned my addr 172.21.15.92:0/2246007599 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:46:00.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.595+0000 7f4545bf4700 1 --2- 172.21.15.92:0/2246007599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f452c004810 0x7f45401566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.595+0000 7f45463f5700 1 --2- 172.21.15.92:0/2246007599 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4540157bf0 0x7f45400c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.596+0000 7f45453f3700 1 -- 172.21.15.92:0/2246007599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f452c004810 msgr2=0x7f45401566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.596+0000 7f45453f3700 1 --2- 172.21.15.92:0/2246007599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f452c004810 0x7f45401566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.596+0000 7f45453f3700 1 -- 172.21.15.92:0/2246007599 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4540157bf0 msgr2=0x7f45400c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.596+0000 7f45453f3700 1 --2- 172.21.15.92:0/2246007599 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4540157bf0 0x7f45400c84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.596+0000 7f45453f3700 1 -- 172.21.15.92:0/2246007599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45400c76f0 con 0x7f45400ca1f0 2023-10-06T10:46:00.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.596+0000 7f45453f3700 1 --2- 172.21.15.92:0/2246007599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45400ca1f0 0x7f45401576b0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f453000e4b0 tx=0x7f453001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:00.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.596+0000 7f4535ffb700 1 -- 172.21.15.92:0/2246007599 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4530022070 con 0x7f45400ca1f0 2023-10-06T10:46:00.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.596+0000 7f4535ffb700 1 -- 172.21.15.92:0/2246007599 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453000e9e0 con 0x7f45400ca1f0 2023-10-06T10:46:00.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.596+0000 7f4547e58700 1 -- 172.21.15.92:0/2246007599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45400c79d0 con 0x7f45400ca1f0 2023-10-06T10:46:00.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.596+0000 7f4535ffb700 1 -- 172.21.15.92:0/2246007599 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45300276a0 con 0x7f45400ca1f0 2023-10-06T10:46:00.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.596+0000 7f4547e58700 1 -- 172.21.15.92:0/2246007599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45400ca660 con 0x7f45400ca1f0 2023-10-06T10:46:00.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.597+0000 7f4547e58700 1 -- 172.21.15.92:0/2246007599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4540064a90 con 0x7f45400ca1f0 2023-10-06T10:46:00.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.598+0000 7f4535ffb700 1 -- 172.21.15.92:0/2246007599 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4530027ad0 con 0x7f45400ca1f0 2023-10-06T10:46:00.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.598+0000 7f4535ffb700 1 --2- 172.21.15.92:0/2246007599 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4524066f30 0x7f45240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.598+0000 7f4535ffb700 1 -- 172.21.15.92:0/2246007599 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f4530086d20 con 0x7f45400ca1f0 2023-10-06T10:46:00.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.598+0000 7f4545bf4700 1 --2- 172.21.15.92:0/2246007599 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4524066f30 0x7f45240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.598+0000 7f4535ffb700 1 --2- 172.21.15.92:0/2246007599 >> [v2:172.21.15.92:6808/90605,v1:172.21.15.92:6809/90605] conn(0x7f452406d990 0x7f452406fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.598+0000 7f4535ffb700 1 -- 172.21.15.92:0/2246007599 --> [v2:172.21.15.92:6808/90605,v1:172.21.15.92:6809/90605] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4524070490 con 0x7f452406d990 2023-10-06T10:46:00.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.598+0000 7f4535ffb700 1 -- 172.21.15.92:0/2246007599 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4530087010 con 0x7f45400ca1f0 2023-10-06T10:46:00.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.598+0000 7f45463f5700 1 --2- 172.21.15.92:0/2246007599 >> [v2:172.21.15.92:6808/90605,v1:172.21.15.92:6809/90605] conn(0x7f452406d990 0x7f452406fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.599+0000 7f4545bf4700 1 --2- 172.21.15.92:0/2246007599 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4524066f30 0x7f45240693e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f453c001050 tx=0x7f453c001e50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:00.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.599+0000 7f45463f5700 1 --2- 172.21.15.92:0/2246007599 >> [v2:172.21.15.92:6808/90605,v1:172.21.15.92:6809/90605] conn(0x7f452406d990 0x7f452406fdc0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:00.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.599+0000 7f4535ffb700 1 -- 172.21.15.92:0/2246007599 <== osd.3 v2:172.21.15.92:6808/90605 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f4524070490 con 0x7f452406d990 2023-10-06T10:46:00.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.613+0000 7f4547e58700 1 -- 172.21.15.92:0/2246007599 --> [v2:172.21.15.92:6808/90605,v1:172.21.15.92:6809/90605] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f45400c7e70 con 0x7f452406d990 2023-10-06T10:46:00.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.613+0000 7f4535ffb700 1 -- 172.21.15.92:0/2246007599 <== osd.3 v2:172.21.15.92:6808/90605 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f45400c7e70 con 0x7f452406d990 2023-10-06T10:46:00.615 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:46:00.615 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T10:46:00.615 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:46:00.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.613+0000 7f4547e58700 1 -- 172.21.15.92:0/2246007599 >> [v2:172.21.15.92:6808/90605,v1:172.21.15.92:6809/90605] conn(0x7f452406d990 msgr2=0x7f452406fdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.613+0000 7f4547e58700 1 --2- 172.21.15.92:0/2246007599 >> [v2:172.21.15.92:6808/90605,v1:172.21.15.92:6809/90605] conn(0x7f452406d990 0x7f452406fdc0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.614+0000 7f4547e58700 1 -- 172.21.15.92:0/2246007599 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4524066f30 msgr2=0x7f45240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.614+0000 7f4547e58700 1 --2- 172.21.15.92:0/2246007599 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4524066f30 0x7f45240693e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f453c001050 tx=0x7f453c001e50 comp rx=0 tx=0).stop 2023-10-06T10:46:00.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.614+0000 7f4547e58700 1 -- 172.21.15.92:0/2246007599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45400ca1f0 msgr2=0x7f45401576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.614+0000 7f4547e58700 1 --2- 172.21.15.92:0/2246007599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45400ca1f0 0x7f45401576b0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f453000e4b0 tx=0x7f453001a000 comp rx=0 tx=0).stop 2023-10-06T10:46:00.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.614+0000 7f45463f5700 1 -- 172.21.15.92:0/2246007599 reap_dead start 2023-10-06T10:46:00.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.614+0000 7f4547e58700 1 -- 172.21.15.92:0/2246007599 shutdown_connections 2023-10-06T10:46:00.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.614+0000 7f4547e58700 1 -- 172.21.15.92:0/2246007599 >> 172.21.15.92:0/2246007599 conn(0x7f45400682f0 msgr2=0x7f45400ce420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:00.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.614+0000 7f4547e58700 1 -- 172.21.15.92:0/2246007599 shutdown_connections 2023-10-06T10:46:00.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.614+0000 7f4547e58700 1 -- 172.21.15.92:0/2246007599 wait complete. 2023-10-06T10:46:00.627 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-06T10:46:00.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.740+0000 7ff42ce7c700 1 Processor -- start 2023-10-06T10:46:00.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.740+0000 7ff42ce7c700 1 -- start start 2023-10-06T10:46:00.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.741+0000 7ff42ce7c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4280b49a0 0x7ff4280aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.741+0000 7ff42ce7c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4280b0c60 con 0x7ff428159560 2023-10-06T10:46:00.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.741+0000 7ff42ce7c700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4280b0f40 con 0x7ff4280b0760 2023-10-06T10:46:00.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.741+0000 7ff42ce7c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4280b1220 con 0x7ff4280b49a0 2023-10-06T10:46:00.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.741+0000 7ff42659c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff428159560 0x7ff4280aa530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35448/0 (socket says 172.21.15.92:35448) 2023-10-06T10:46:00.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.741+0000 7ff42659c700 1 -- 172.21.15.92:0/603748118 learned_addr learned my addr 172.21.15.92:0/603748118 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:46:00.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.741+0000 7ff41dd9b700 1 --2- 172.21.15.92:0/603748118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4280b49a0 0x7ff4280aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.741+0000 7ff425d9b700 1 -- 172.21.15.92:0/603748118 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 710905761 0 0) 0x7ff4280b0c60 con 0x7ff428159560 2023-10-06T10:46:00.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.741+0000 7ff425d9b700 1 -- 172.21.15.92:0/603748118 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff414003570 con 0x7ff428159560 2023-10-06T10:46:00.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.741+0000 7ff425d9b700 1 -- 172.21.15.92:0/603748118 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2782532504 0 0) 0x7ff414003570 con 0x7ff428159560 2023-10-06T10:46:00.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.741+0000 7ff425d9b700 1 -- 172.21.15.92:0/603748118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4280b49a0 msgr2=0x7ff4280aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:00.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.741+0000 7ff425d9b700 1 --2- 172.21.15.92:0/603748118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4280b49a0 0x7ff4280aac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.741+0000 7ff425d9b700 1 -- 172.21.15.92:0/603748118 >> v1:172.21.15.124:6789/0 conn(0x7ff4280b0760 legacy=0x7ff4280b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.742+0000 7ff425d9b700 1 -- 172.21.15.92:0/603748118 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4280b1500 con 0x7ff428159560 2023-10-06T10:46:00.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.742+0000 7ff425d9b700 1 -- 172.21.15.92:0/603748118 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7ff4180015f0 con 0x7ff428159560 2023-10-06T10:46:00.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.742+0000 7ff41dd9b700 1 --2- 172.21.15.92:0/603748118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4280b49a0 0x7ff4280aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:46:00.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.742+0000 7ff425d9b700 1 -- 172.21.15.92:0/603748118 >> v1:172.21.15.92:6789/0 conn(0x7ff428159560 legacy=0x7ff4280aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.742+0000 7ff425d9b700 1 --2- 172.21.15.92:0/603748118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff414003a90 0x7ff414003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.742+0000 7ff425d9b700 1 --2- 172.21.15.92:0/603748118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4280b49a0 0x7ff414004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.742+0000 7ff425d9b700 1 --2- 172.21.15.92:0/603748118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff414004d50 0x7ff4140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.742+0000 7ff425d9b700 1 -- 172.21.15.92:0/603748118 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4280b1220 con 0x7ff414004d50 2023-10-06T10:46:00.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.742+0000 7ff425d9b700 1 -- 172.21.15.92:0/603748118 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41400bb10 con 0x7ff414003a90 2023-10-06T10:46:00.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.742+0000 7ff425d9b700 1 -- 172.21.15.92:0/603748118 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41400bc80 con 0x7ff4280b49a0 2023-10-06T10:46:00.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.742+0000 7ff426d9d700 1 --2- 172.21.15.92:0/603748118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff414004d50 0x7ff4140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.742+0000 7ff41dd9b700 1 --2- 172.21.15.92:0/603748118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4280b49a0 0x7ff414004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.742+0000 7ff42659c700 1 --2- 172.21.15.92:0/603748118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff414003a90 0x7ff414003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.742+0000 7ff426d9d700 1 -- 172.21.15.92:0/603748118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4280b49a0 msgr2=0x7ff414004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:00.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.742+0000 7ff426d9d700 1 --2- 172.21.15.92:0/603748118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4280b49a0 0x7ff414004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.742+0000 7ff426d9d700 1 -- 172.21.15.92:0/603748118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff414003a90 msgr2=0x7ff414003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:00.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.742+0000 7ff426d9d700 1 --2- 172.21.15.92:0/603748118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff414003a90 0x7ff414003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.742+0000 7ff426d9d700 1 -- 172.21.15.92:0/603748118 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff410009420 con 0x7ff414004d50 2023-10-06T10:46:00.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.742+0000 7ff426d9d700 1 --2- 172.21.15.92:0/603748118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff414004d50 0x7ff4140051c0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7ff4200020a0 tx=0x7ff4200016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57788739760ae371 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:00.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.742+0000 7ff425d9b700 1 -- 172.21.15.92:0/603748118 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4200019e0 con 0x7ff414004d50 2023-10-06T10:46:00.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.742+0000 7ff425d9b700 1 -- 172.21.15.92:0/603748118 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42000c8b0 con 0x7ff414004d50 2023-10-06T10:46:00.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.742+0000 7ff425d9b700 1 -- 172.21.15.92:0/603748118 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42000cc50 con 0x7ff414004d50 2023-10-06T10:46:00.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.743+0000 7ff42ce7c700 1 -- 172.21.15.92:0/603748118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff414004d50 msgr2=0x7ff4140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.743+0000 7ff42ce7c700 1 --2- 172.21.15.92:0/603748118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff414004d50 0x7ff4140051c0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7ff4200020a0 tx=0x7ff4200016d0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.743+0000 7ff426d9d700 1 -- 172.21.15.92:0/603748118 reap_dead start 2023-10-06T10:46:00.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.743+0000 7ff42ce7c700 1 -- 172.21.15.92:0/603748118 shutdown_connections 2023-10-06T10:46:00.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.743+0000 7ff42ce7c700 1 -- 172.21.15.92:0/603748118 >> 172.21.15.92:0/603748118 conn(0x7ff4280997f0 msgr2=0x7ff42806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:00.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.743+0000 7ff42ce7c700 1 -- 172.21.15.92:0/603748118 shutdown_connections 2023-10-06T10:46:00.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.743+0000 7ff42ce7c700 1 -- 172.21.15.92:0/603748118 wait complete. 2023-10-06T10:46:00.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.744+0000 7ff42ce7c700 1 Processor -- start 2023-10-06T10:46:00.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.744+0000 7ff42ce7c700 1 -- start start 2023-10-06T10:46:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.745+0000 7ff42ce7c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff414004d50 0x7ff42809c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.745+0000 7ff42ce7c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff42816c9a0 0x7ff42816ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.745+0000 7ff42ce7c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff42815a340 0x7ff42815a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.745+0000 7ff42ce7c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4280af9e0 con 0x7ff414004d50 2023-10-06T10:46:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.745+0000 7ff42ce7c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42815b2e0 con 0x7ff42816c9a0 2023-10-06T10:46:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.745+0000 7ff42ce7c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42816d350 con 0x7ff42815a340 2023-10-06T10:46:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.745+0000 7ff42659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff414004d50 0x7ff42809c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.745+0000 7ff42659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff414004d50 0x7ff42809c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38108/0 (socket says 172.21.15.92:38108) 2023-10-06T10:46:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.745+0000 7ff42659c700 1 -- 172.21.15.92:0/3538109090 learned_addr learned my addr 172.21.15.92:0/3538109090 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:46:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.745+0000 7ff426d9d700 1 --2- 172.21.15.92:0/3538109090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff42815a340 0x7ff42815a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.745+0000 7ff41dd9b700 1 --2- 172.21.15.92:0/3538109090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff42816c9a0 0x7ff42816ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.745+0000 7ff42659c700 1 -- 172.21.15.92:0/3538109090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff42815a340 msgr2=0x7ff42815a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.745+0000 7ff42659c700 1 --2- 172.21.15.92:0/3538109090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff42815a340 0x7ff42815a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.745+0000 7ff42659c700 1 -- 172.21.15.92:0/3538109090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff42816c9a0 msgr2=0x7ff42816ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.745+0000 7ff42659c700 1 --2- 172.21.15.92:0/3538109090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff42816c9a0 0x7ff42816ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.745+0000 7ff42659c700 1 -- 172.21.15.92:0/3538109090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42815b000 con 0x7ff414004d50 2023-10-06T10:46:00.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.745+0000 7ff41dd9b700 1 --2- 172.21.15.92:0/3538109090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff42816c9a0 0x7ff42816ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T10:46:00.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.745+0000 7ff42659c700 1 --2- 172.21.15.92:0/3538109090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff414004d50 0x7ff42809c890 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7ff41800fc60 tx=0x7ff418009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:00.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.745+0000 7ff41effd700 1 -- 172.21.15.92:0/3538109090 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4180010a0 con 0x7ff414004d50 2023-10-06T10:46:00.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.745+0000 7ff41effd700 1 -- 172.21.15.92:0/3538109090 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41800d9d0 con 0x7ff414004d50 2023-10-06T10:46:00.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.745+0000 7ff42ce7c700 1 -- 172.21.15.92:0/3538109090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff42816a9d0 con 0x7ff414004d50 2023-10-06T10:46:00.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.745+0000 7ff41effd700 1 -- 172.21.15.92:0/3538109090 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41800dd70 con 0x7ff414004d50 2023-10-06T10:46:00.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.746+0000 7ff42ce7c700 1 -- 172.21.15.92:0/3538109090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff42816af90 con 0x7ff414004d50 2023-10-06T10:46:00.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.747+0000 7ff42ce7c700 1 -- 172.21.15.92:0/3538109090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff428064a90 con 0x7ff414004d50 2023-10-06T10:46:00.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.747+0000 7ff41effd700 1 -- 172.21.15.92:0/3538109090 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff4180072e0 con 0x7ff414004d50 2023-10-06T10:46:00.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.747+0000 7ff41effd700 1 --2- 172.21.15.92:0/3538109090 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff408066f30 0x7ff4080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:00.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.747+0000 7ff41dd9b700 1 --2- 172.21.15.92:0/3538109090 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff408066f30 0x7ff4080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:00.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.747+0000 7ff41effd700 1 -- 172.21.15.92:0/3538109090 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7ff4180846b0 con 0x7ff414004d50 2023-10-06T10:46:00.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.748+0000 7ff41dd9b700 1 --2- 172.21.15.92:0/3538109090 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff408066f30 0x7ff4080693e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff4100093f0 tx=0x7ff410008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:00.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.752+0000 7ff41effd700 1 -- 172.21.15.92:0/3538109090 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4180518d0 con 0x7ff414004d50 2023-10-06T10:46:00.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.935+0000 7ff42ce7c700 1 -- 172.21.15.92:0/3538109090 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4280b17e0 con 0x7ff408066f30 2023-10-06T10:46:00.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.935+0000 7ff41effd700 1 -- 172.21.15.92:0/3538109090 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff4280b17e0 con 0x7ff408066f30 2023-10-06T10:46:00.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.937+0000 7ff42ce7c700 1 -- 172.21.15.92:0/3538109090 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff408066f30 msgr2=0x7ff4080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.937+0000 7ff42ce7c700 1 --2- 172.21.15.92:0/3538109090 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff408066f30 0x7ff4080693e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff4100093f0 tx=0x7ff410008040 comp rx=0 tx=0).stop 2023-10-06T10:46:00.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.937+0000 7ff42ce7c700 1 -- 172.21.15.92:0/3538109090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff414004d50 msgr2=0x7ff42809c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:00.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.937+0000 7ff42ce7c700 1 --2- 172.21.15.92:0/3538109090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff414004d50 0x7ff42809c890 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7ff41800fc60 tx=0x7ff418009040 comp rx=0 tx=0).stop 2023-10-06T10:46:00.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.937+0000 7ff42ce7c700 1 -- 172.21.15.92:0/3538109090 shutdown_connections 2023-10-06T10:46:00.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.937+0000 7ff42ce7c700 1 --2- 172.21.15.92:0/3538109090 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7ff408066f30 0x7ff4080693e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.937+0000 7ff42ce7c700 1 --2- 172.21.15.92:0/3538109090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff42815a340 0x7ff42815a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.937+0000 7ff42ce7c700 1 --2- 172.21.15.92:0/3538109090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff414004d50 0x7ff42809c890 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.937+0000 7ff42ce7c700 1 --2- 172.21.15.92:0/3538109090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff42816c9a0 0x7ff42816ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:00.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.937+0000 7ff42ce7c700 1 -- 172.21.15.92:0/3538109090 >> 172.21.15.92:0/3538109090 conn(0x7ff4280997f0 msgr2=0x7ff4280afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:00.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.937+0000 7ff42ce7c700 1 -- 172.21.15.92:0/3538109090 shutdown_connections 2023-10-06T10:46:00.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:00.938+0000 7ff42ce7c700 1 -- 172.21.15.92:0/3538109090 wait complete. 2023-10-06T10:46:00.941 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 3 to deep-scrub 2023-10-06T10:46:00.951 INFO:tasks.ceph:Scrubbing osd.4 2023-10-06T10:46:00.951 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-06T10:46:01.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.067+0000 7f5d0ae57700 1 Processor -- start 2023-10-06T10:46:01.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.067+0000 7f5d0ae57700 1 -- start start 2023-10-06T10:46:01.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.068+0000 7f5d0ae57700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5d0406ece0 0x7f5d040b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.068+0000 7f5d0ae57700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5d0406f0b0 0x7f5d040b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.068+0000 7f5d0ae57700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d040b2ce0 con 0x7f5d040aede0 2023-10-06T10:46:01.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.068+0000 7f5d0ae57700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0406fb40 con 0x7f5d0406f0b0 2023-10-06T10:46:01.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.068+0000 7f5d0ae57700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d040b0410 con 0x7f5d0406ece0 2023-10-06T10:46:01.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.068+0000 7f5d03fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5d0406f0b0 0x7f5d040b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.068+0000 7f5d03fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5d0406f0b0 0x7f5d040b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38114/0 (socket says 172.21.15.92:38114) 2023-10-06T10:46:01.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.068+0000 7f5d093f4700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5d040aede0 0x7f5d040af240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35452/0 (socket says 172.21.15.92:35452) 2023-10-06T10:46:01.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.068+0000 7f5d03fff700 1 -- 172.21.15.92:0/2486815618 learned_addr learned my addr 172.21.15.92:0/2486815618 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:46:01.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.068+0000 7f5d08bf3700 1 --2- 172.21.15.92:0/2486815618 >> v2:172.21.15.124:3300/0 conn(0x7f5d0406ece0 0x7f5d040b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.068+0000 7f5d03fff700 1 -- 172.21.15.92:0/2486815618 >> v2:172.21.15.124:3300/0 conn(0x7f5d0406ece0 msgr2=0x7f5d040b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:01.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.068+0000 7f5d03fff700 1 --2- 172.21.15.92:0/2486815618 >> v2:172.21.15.124:3300/0 conn(0x7f5d0406ece0 0x7f5d040b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.068+0000 7f5d03fff700 1 -- 172.21.15.92:0/2486815618 >> v1:172.21.15.92:6789/0 conn(0x7f5d040aede0 legacy=0x7f5d040af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.068+0000 7f5d03fff700 1 -- 172.21.15.92:0/2486815618 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d040b2a00 con 0x7f5d0406f0b0 2023-10-06T10:46:01.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.068+0000 7f5d037fe700 1 -- 172.21.15.92:0/2486815618 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763978869 0 0) 0x7f5d040b2ce0 con 0x7f5d040aede0 2023-10-06T10:46:01.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.068+0000 7f5d03fff700 1 --2- 172.21.15.92:0/2486815618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d0406f0b0 0x7f5d040b08a0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00099b0 tx=0x7f5cf0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c54526fb6b4a1c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:01.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.069+0000 7f5d037fe700 1 -- 172.21.15.92:0/2486815618 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf000b820 con 0x7f5d0406f0b0 2023-10-06T10:46:01.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.069+0000 7f5d037fe700 1 -- 172.21.15.92:0/2486815618 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf000b980 con 0x7f5d0406f0b0 2023-10-06T10:46:01.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.069+0000 7f5d037fe700 1 -- 172.21.15.92:0/2486815618 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf000bd20 con 0x7f5d0406f0b0 2023-10-06T10:46:01.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.069+0000 7f5d0ae57700 1 -- 172.21.15.92:0/2486815618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d0406f0b0 msgr2=0x7f5d040b08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.069+0000 7f5d0ae57700 1 --2- 172.21.15.92:0/2486815618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d0406f0b0 0x7f5d040b08a0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00099b0 tx=0x7f5cf0009250 comp rx=0 tx=0).stop 2023-10-06T10:46:01.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.069+0000 7f5d0ae57700 1 -- 172.21.15.92:0/2486815618 shutdown_connections 2023-10-06T10:46:01.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.069+0000 7f5d0ae57700 1 --2- 172.21.15.92:0/2486815618 >> v2:172.21.15.124:3300/0 conn(0x7f5d0406ece0 0x7f5d040b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.069+0000 7f5d0ae57700 1 --2- 172.21.15.92:0/2486815618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d0406f0b0 0x7f5d040b08a0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.069+0000 7f5d0ae57700 1 -- 172.21.15.92:0/2486815618 >> 172.21.15.92:0/2486815618 conn(0x7f5d040682f0 msgr2=0x7f5d040686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:01.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.069+0000 7f5d0ae57700 1 -- 172.21.15.92:0/2486815618 shutdown_connections 2023-10-06T10:46:01.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.069+0000 7f5d0ae57700 1 -- 172.21.15.92:0/2486815618 wait complete. 2023-10-06T10:46:01.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.070+0000 7f5d0ae57700 1 Processor -- start 2023-10-06T10:46:01.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.070+0000 7f5d0ae57700 1 -- start start 2023-10-06T10:46:01.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.071+0000 7f5d0ae57700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d0406ece0 0x7f5d0415bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.071+0000 7f5d0ae57700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d0406f0b0 0x7f5d0415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.071+0000 7f5d0ae57700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d040aede0 0x7f5d04155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.071+0000 7f5d0ae57700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0415b4f0 con 0x7f5d0406ece0 2023-10-06T10:46:01.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.071+0000 7f5d0ae57700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d04159f10 con 0x7f5d0406f0b0 2023-10-06T10:46:01.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.071+0000 7f5d0ae57700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0415a1f0 con 0x7f5d040aede0 2023-10-06T10:46:01.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.071+0000 7f5d08bf3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d0406ece0 0x7f5d0415bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.071+0000 7f5d08bf3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d0406ece0 0x7f5d0415bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38116/0 (socket says 172.21.15.92:38116) 2023-10-06T10:46:01.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.071+0000 7f5d08bf3700 1 -- 172.21.15.92:0/3747141509 learned_addr learned my addr 172.21.15.92:0/3747141509 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:46:01.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.071+0000 7f5d093f4700 1 --2- 172.21.15.92:0/3747141509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d040aede0 0x7f5d04155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.071+0000 7f5d03fff700 1 --2- 172.21.15.92:0/3747141509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d0406f0b0 0x7f5d0415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.072+0000 7f5d08bf3700 1 -- 172.21.15.92:0/3747141509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d040aede0 msgr2=0x7f5d04155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.072+0000 7f5d08bf3700 1 --2- 172.21.15.92:0/3747141509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d040aede0 0x7f5d04155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.072+0000 7f5d08bf3700 1 -- 172.21.15.92:0/3747141509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d0406f0b0 msgr2=0x7f5d0415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.072+0000 7f5d08bf3700 1 --2- 172.21.15.92:0/3747141509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d0406f0b0 0x7f5d0415afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.072+0000 7f5d08bf3700 1 -- 172.21.15.92:0/3747141509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf00094f0 con 0x7f5d0406ece0 2023-10-06T10:46:01.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.072+0000 7f5d08bf3700 1 --2- 172.21.15.92:0/3747141509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d0406ece0 0x7f5d0415bec0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800d860 tx=0x7f5cf8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:01.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.072+0000 7f5d00ff9700 1 -- 172.21.15.92:0/3747141509 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8009610 con 0x7f5d0406ece0 2023-10-06T10:46:01.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.072+0000 7f5d00ff9700 1 -- 172.21.15.92:0/3747141509 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf8009770 con 0x7f5d0406ece0 2023-10-06T10:46:01.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.072+0000 7f5d00ff9700 1 -- 172.21.15.92:0/3747141509 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8009b30 con 0x7f5d0406ece0 2023-10-06T10:46:01.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.072+0000 7f5d0ae57700 1 -- 172.21.15.92:0/3747141509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0415a4d0 con 0x7f5d0406ece0 2023-10-06T10:46:01.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.072+0000 7f5d0ae57700 1 -- 172.21.15.92:0/3747141509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0415aa90 con 0x7f5d0406ece0 2023-10-06T10:46:01.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.073+0000 7f5d0ae57700 1 -- 172.21.15.92:0/3747141509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5d04064a90 con 0x7f5d0406ece0 2023-10-06T10:46:01.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.073+0000 7f5d00ff9700 1 -- 172.21.15.92:0/3747141509 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5cf801e460 con 0x7f5d0406ece0 2023-10-06T10:46:01.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.074+0000 7f5d00ff9700 1 --2- 172.21.15.92:0/3747141509 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5ce8066f30 0x7f5ce80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.074+0000 7f5d00ff9700 1 -- 172.21.15.92:0/3747141509 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f5cf8085640 con 0x7f5d0406ece0 2023-10-06T10:46:01.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.074+0000 7f5d03fff700 1 --2- 172.21.15.92:0/3747141509 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5ce8066f30 0x7f5ce80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.074+0000 7f5d00ff9700 1 --2- 172.21.15.92:0/3747141509 >> [v2:172.21.15.124:6800/90883,v1:172.21.15.124:6801/90883] conn(0x7f5ce806d990 0x7f5ce806fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.074+0000 7f5d00ff9700 1 -- 172.21.15.92:0/3747141509 --> [v2:172.21.15.124:6800/90883,v1:172.21.15.124:6801/90883] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5ce8070490 con 0x7f5ce806d990 2023-10-06T10:46:01.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.074+0000 7f5d00ff9700 1 -- 172.21.15.92:0/3747141509 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cf8085a10 con 0x7f5d0406ece0 2023-10-06T10:46:01.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.074+0000 7f5d093f4700 1 --2- 172.21.15.92:0/3747141509 >> [v2:172.21.15.124:6800/90883,v1:172.21.15.124:6801/90883] conn(0x7f5ce806d990 0x7f5ce806fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.074+0000 7f5d03fff700 1 --2- 172.21.15.92:0/3747141509 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5ce8066f30 0x7f5ce80693e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00099b0 tx=0x7f5cf0017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:01.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.074+0000 7f5d093f4700 1 --2- 172.21.15.92:0/3747141509 >> [v2:172.21.15.124:6800/90883,v1:172.21.15.124:6801/90883] conn(0x7f5ce806d990 0x7f5ce806fdc0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:01.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.075+0000 7f5d00ff9700 1 -- 172.21.15.92:0/3747141509 <== osd.4 v2:172.21.15.124:6800/90883 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f5ce8070490 con 0x7f5ce806d990 2023-10-06T10:46:01.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.089+0000 7f5d0ae57700 1 -- 172.21.15.92:0/3747141509 --> [v2:172.21.15.124:6800/90883,v1:172.21.15.124:6801/90883] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5d0404bb40 con 0x7f5ce806d990 2023-10-06T10:46:01.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.089+0000 7f5d00ff9700 1 -- 172.21.15.92:0/3747141509 <== osd.4 v2:172.21.15.124:6800/90883 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f5d0404bb40 con 0x7f5ce806d990 2023-10-06T10:46:01.091 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:46:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T10:46:01.091 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:46:01.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.089+0000 7f5d0ae57700 1 -- 172.21.15.92:0/3747141509 >> [v2:172.21.15.124:6800/90883,v1:172.21.15.124:6801/90883] conn(0x7f5ce806d990 msgr2=0x7f5ce806fdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.089+0000 7f5d0ae57700 1 --2- 172.21.15.92:0/3747141509 >> [v2:172.21.15.124:6800/90883,v1:172.21.15.124:6801/90883] conn(0x7f5ce806d990 0x7f5ce806fdc0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.090+0000 7f5d0ae57700 1 -- 172.21.15.92:0/3747141509 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5ce8066f30 msgr2=0x7f5ce80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.090+0000 7f5d0ae57700 1 --2- 172.21.15.92:0/3747141509 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f5ce8066f30 0x7f5ce80693e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00099b0 tx=0x7f5cf0017000 comp rx=0 tx=0).stop 2023-10-06T10:46:01.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.090+0000 7f5d0ae57700 1 -- 172.21.15.92:0/3747141509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d0406ece0 msgr2=0x7f5d0415bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.090+0000 7f5d0ae57700 1 --2- 172.21.15.92:0/3747141509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d0406ece0 0x7f5d0415bec0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800d860 tx=0x7f5cf8008040 comp rx=0 tx=0).stop 2023-10-06T10:46:01.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.090+0000 7f5d093f4700 1 -- 172.21.15.92:0/3747141509 reap_dead start 2023-10-06T10:46:01.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.090+0000 7f5d0ae57700 1 -- 172.21.15.92:0/3747141509 shutdown_connections 2023-10-06T10:46:01.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.090+0000 7f5d0ae57700 1 -- 172.21.15.92:0/3747141509 >> 172.21.15.92:0/3747141509 conn(0x7f5d040682f0 msgr2=0x7f5d040d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:01.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.090+0000 7f5d0ae57700 1 -- 172.21.15.92:0/3747141509 shutdown_connections 2023-10-06T10:46:01.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.090+0000 7f5d0ae57700 1 -- 172.21.15.92:0/3747141509 wait complete. 2023-10-06T10:46:01.103 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-06T10:46:01.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.216+0000 7fd7b7fff700 1 Processor -- start 2023-10-06T10:46:01.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.216+0000 7fd7b7fff700 1 -- start start 2023-10-06T10:46:01.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.217+0000 7fd7b7fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b8159440 0x7fd7b80b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.217+0000 7fd7b7fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b80cb7a0 con 0x7fd7b80b2100 2023-10-06T10:46:01.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.217+0000 7fd7b7fff700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b80b0860 con 0x7fd7b80b35d0 2023-10-06T10:46:01.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.217+0000 7fd7b7fff700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b80cb150 con 0x7fd7b8159440 2023-10-06T10:46:01.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.217+0000 7fd7b77fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd7b80b2100 0x7fd7b80b2580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35454/0 (socket says 172.21.15.92:35454) 2023-10-06T10:46:01.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.217+0000 7fd7b77fe700 1 -- 172.21.15.92:0/597272749 learned_addr learned my addr 172.21.15.92:0/597272749 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:46:01.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.217+0000 7fd7b6ffd700 1 --2- 172.21.15.92:0/597272749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b8159440 0x7fd7b80b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.217+0000 7fd7b67fc700 1 -- 172.21.15.92:0/597272749 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2748942013 0 0) 0x7fd7b80cb7a0 con 0x7fd7b80b2100 2023-10-06T10:46:01.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.217+0000 7fd7b67fc700 1 -- 172.21.15.92:0/597272749 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7a4003570 con 0x7fd7b80b2100 2023-10-06T10:46:01.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.218+0000 7fd7b6ffd700 1 -- 172.21.15.92:0/597272749 >> v1:172.21.15.124:6789/0 conn(0x7fd7b80b35d0 legacy=0x7fd7b80cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.218+0000 7fd7b6ffd700 1 -- 172.21.15.92:0/597272749 >> v1:172.21.15.92:6789/0 conn(0x7fd7b80b2100 legacy=0x7fd7b80b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.218+0000 7fd7b6ffd700 1 -- 172.21.15.92:0/597272749 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b80cb430 con 0x7fd7b8159440 2023-10-06T10:46:01.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.218+0000 7fd7b6ffd700 1 --2- 172.21.15.92:0/597272749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b8159440 0x7fd7b80b3090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009910 tx=0x7fd7a80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8a861ae0aae666 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:01.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.218+0000 7fd7b67fc700 1 -- 172.21.15.92:0/597272749 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a800b680 con 0x7fd7b8159440 2023-10-06T10:46:01.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.218+0000 7fd7b67fc700 1 -- 172.21.15.92:0/597272749 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a800b7e0 con 0x7fd7b8159440 2023-10-06T10:46:01.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.218+0000 7fd7b67fc700 1 -- 172.21.15.92:0/597272749 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a800bb60 con 0x7fd7b8159440 2023-10-06T10:46:01.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.218+0000 7fd7b7fff700 1 -- 172.21.15.92:0/597272749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b8159440 msgr2=0x7fd7b80b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.218+0000 7fd7b7fff700 1 --2- 172.21.15.92:0/597272749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b8159440 0x7fd7b80b3090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009910 tx=0x7fd7a80091a0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.218+0000 7fd7b7fff700 1 -- 172.21.15.92:0/597272749 shutdown_connections 2023-10-06T10:46:01.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.218+0000 7fd7b7fff700 1 --2- 172.21.15.92:0/597272749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b8159440 0x7fd7b80b3090 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.218+0000 7fd7b7fff700 1 -- 172.21.15.92:0/597272749 >> 172.21.15.92:0/597272749 conn(0x7fd7b806ece0 msgr2=0x7fd7b806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:01.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.219+0000 7fd7b7fff700 1 -- 172.21.15.92:0/597272749 shutdown_connections 2023-10-06T10:46:01.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.219+0000 7fd7b7fff700 1 -- 172.21.15.92:0/597272749 wait complete. 2023-10-06T10:46:01.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.220+0000 7fd7b7fff700 1 Processor -- start 2023-10-06T10:46:01.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.220+0000 7fd7b7fff700 1 -- start start 2023-10-06T10:46:01.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.220+0000 7fd7b7fff700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b80b2100 0x7fd7b80c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.220+0000 7fd7b7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7b80b35d0 0x7fd7b8159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.221+0000 7fd7b6ffd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b80b2100 0x7fd7b80c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.221+0000 7fd7b7fff700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd7b8159440 0x7fd7b8155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.221+0000 7fd7ae7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7b80b35d0 0x7fd7b8159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.221+0000 7fd7b7fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b815a300 con 0x7fd7b80b35d0 2023-10-06T10:46:01.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.221+0000 7fd7b6ffd700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b80b2100 0x7fd7b80c9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.92:53700/0 (socket says 172.21.15.92:53700) 2023-10-06T10:46:01.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.221+0000 7fd7b6ffd700 1 -- 172.21.15.92:0/409853464 learned_addr learned my addr 172.21.15.92:0/409853464 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:46:01.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.221+0000 7fd7b7fff700 1 -- 172.21.15.92:0/409853464 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b815a5e0 con 0x7fd7b8159440 2023-10-06T10:46:01.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.221+0000 7fd7b7fff700 1 -- 172.21.15.92:0/409853464 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b815a8c0 con 0x7fd7b80b2100 2023-10-06T10:46:01.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.221+0000 7fd7b77fe700 1 --2- 172.21.15.92:0/409853464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd7b8159440 0x7fd7b8155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.221+0000 7fd7ae7fc700 1 -- 172.21.15.92:0/409853464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b80b2100 msgr2=0x7fd7b80c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.221+0000 7fd7ae7fc700 1 --2- 172.21.15.92:0/409853464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b80b2100 0x7fd7b80c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.221+0000 7fd7ae7fc700 1 -- 172.21.15.92:0/409853464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd7b8159440 msgr2=0x7fd7b8155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.221+0000 7fd7ae7fc700 1 --2- 172.21.15.92:0/409853464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd7b8159440 0x7fd7b8155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.221+0000 7fd7ae7fc700 1 -- 172.21.15.92:0/409853464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a8009420 con 0x7fd7b80b35d0 2023-10-06T10:46:01.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.221+0000 7fd7ae7fc700 1 --2- 172.21.15.92:0/409853464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7b80b35d0 0x7fd7b8159ab0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000c590 tx=0x7fd7a0007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:01.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.221+0000 7fd7af7fe700 1 -- 172.21.15.92:0/409853464 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0011590 con 0x7fd7b80b35d0 2023-10-06T10:46:01.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.221+0000 7fd7af7fe700 1 -- 172.21.15.92:0/409853464 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a0011710 con 0x7fd7b80b35d0 2023-10-06T10:46:01.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.221+0000 7fd7b7fff700 1 -- 172.21.15.92:0/409853464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b815aba0 con 0x7fd7b80b35d0 2023-10-06T10:46:01.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.221+0000 7fd7af7fe700 1 -- 172.21.15.92:0/409853464 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0011ad0 con 0x7fd7b80b35d0 2023-10-06T10:46:01.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.221+0000 7fd7b7fff700 1 -- 172.21.15.92:0/409853464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b8159ff0 con 0x7fd7b80b35d0 2023-10-06T10:46:01.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.222+0000 7fd7b7fff700 1 -- 172.21.15.92:0/409853464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b815b0a0 con 0x7fd7b80b35d0 2023-10-06T10:46:01.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.223+0000 7fd7af7fe700 1 -- 172.21.15.92:0/409853464 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd7a001b3b0 con 0x7fd7b80b35d0 2023-10-06T10:46:01.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.223+0000 7fd7af7fe700 1 --2- 172.21.15.92:0/409853464 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd798066f30 0x7fd7980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.223+0000 7fd7af7fe700 1 -- 172.21.15.92:0/409853464 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7fd7a00839b0 con 0x7fd7b80b35d0 2023-10-06T10:46:01.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.223+0000 7fd7b6ffd700 1 --2- 172.21.15.92:0/409853464 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd798066f30 0x7fd7980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.223+0000 7fd7b6ffd700 1 --2- 172.21.15.92:0/409853464 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd798066f30 0x7fd7980693e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009860 tx=0x7fd7a8017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:01.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.227+0000 7fd7af7fe700 1 -- 172.21.15.92:0/409853464 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd7a0050bd0 con 0x7fd7b80b35d0 2023-10-06T10:46:01.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.410+0000 7fd7b7fff700 1 -- 172.21.15.92:0/409853464 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7b805f230 con 0x7fd798066f30 2023-10-06T10:46:01.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.410+0000 7fd7af7fe700 1 -- 172.21.15.92:0/409853464 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd7b805f230 con 0x7fd798066f30 2023-10-06T10:46:01.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.412+0000 7fd7b7fff700 1 -- 172.21.15.92:0/409853464 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd798066f30 msgr2=0x7fd7980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.412+0000 7fd7b7fff700 1 --2- 172.21.15.92:0/409853464 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd798066f30 0x7fd7980693e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009860 tx=0x7fd7a8017000 comp rx=0 tx=0).stop 2023-10-06T10:46:01.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.412+0000 7fd7b7fff700 1 -- 172.21.15.92:0/409853464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7b80b35d0 msgr2=0x7fd7b8159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.412+0000 7fd7b7fff700 1 --2- 172.21.15.92:0/409853464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7b80b35d0 0x7fd7b8159ab0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000c590 tx=0x7fd7a0007e80 comp rx=0 tx=0).stop 2023-10-06T10:46:01.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.412+0000 7fd7b7fff700 1 -- 172.21.15.92:0/409853464 shutdown_connections 2023-10-06T10:46:01.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.412+0000 7fd7b7fff700 1 --2- 172.21.15.92:0/409853464 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fd798066f30 0x7fd7980693e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.412+0000 7fd7b7fff700 1 --2- 172.21.15.92:0/409853464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd7b8159440 0x7fd7b8155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.412+0000 7fd7b7fff700 1 --2- 172.21.15.92:0/409853464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b80b2100 0x7fd7b80c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.412+0000 7fd7b7fff700 1 --2- 172.21.15.92:0/409853464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7b80b35d0 0x7fd7b8159ab0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.412+0000 7fd7b7fff700 1 -- 172.21.15.92:0/409853464 >> 172.21.15.92:0/409853464 conn(0x7fd7b806ece0 msgr2=0x7fd7b81581c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:01.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.412+0000 7fd7b7fff700 1 -- 172.21.15.92:0/409853464 shutdown_connections 2023-10-06T10:46:01.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.412+0000 7fd7b7fff700 1 -- 172.21.15.92:0/409853464 wait complete. 2023-10-06T10:46:01.415 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 4 to deep-scrub 2023-10-06T10:46:01.426 INFO:tasks.ceph:Scrubbing osd.5 2023-10-06T10:46:01.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-06T10:46:01.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.541+0000 7fab1ecb2700 1 Processor -- start 2023-10-06T10:46:01.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.541+0000 7fab1ecb2700 1 -- start start 2023-10-06T10:46:01.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.542+0000 7fab1ecb2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fab1806ece0 0x7fab180b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.542+0000 7fab1ecb2700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab1806f0b0 0x7fab180b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.542+0000 7fab1ecb2700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab180b2ce0 con 0x7fab180aede0 2023-10-06T10:46:01.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.542+0000 7fab1ecb2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab1806fb40 con 0x7fab1806ece0 2023-10-06T10:46:01.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.542+0000 7fab1ecb2700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab180b0410 con 0x7fab1806f0b0 2023-10-06T10:46:01.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.542+0000 7fab1ca4e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fab1806ece0 0x7fab180b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.542+0000 7fab1ca4e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fab1806ece0 0x7fab180b2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38136/0 (socket says 172.21.15.92:38136) 2023-10-06T10:46:01.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.542+0000 7fab1ca4e700 1 -- 172.21.15.92:0/2272127174 learned_addr learned my addr 172.21.15.92:0/2272127174 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:46:01.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.542+0000 7fab17fff700 1 --2- 172.21.15.92:0/2272127174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab1806f0b0 0x7fab180b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.543+0000 7fab1ca4e700 1 -- 172.21.15.92:0/2272127174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab1806f0b0 msgr2=0x7fab180b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:01.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.543+0000 7fab1ca4e700 1 --2- 172.21.15.92:0/2272127174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab1806f0b0 0x7fab180b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.543+0000 7fab1ca4e700 1 -- 172.21.15.92:0/2272127174 >> v1:172.21.15.124:6789/0 conn(0x7fab180aede0 legacy=0x7fab180af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:01.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.543+0000 7fab1ca4e700 1 -- 172.21.15.92:0/2272127174 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab180b2a00 con 0x7fab1806ece0 2023-10-06T10:46:01.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.543+0000 7fab1ca4e700 1 --2- 172.21.15.92:0/2272127174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1806ece0 0x7fab180b2360 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fab08009f50 tx=0x7fab08009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac3fed62811e836 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:01.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.543+0000 7fab177fe700 1 -- 172.21.15.92:0/2272127174 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0800e070 con 0x7fab1806ece0 2023-10-06T10:46:01.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.543+0000 7fab177fe700 1 -- 172.21.15.92:0/2272127174 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0800ba30 con 0x7fab1806ece0 2023-10-06T10:46:01.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.543+0000 7fab177fe700 1 -- 172.21.15.92:0/2272127174 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab080092e0 con 0x7fab1806ece0 2023-10-06T10:46:01.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.543+0000 7fab1ecb2700 1 -- 172.21.15.92:0/2272127174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1806ece0 msgr2=0x7fab180b2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.543+0000 7fab1ecb2700 1 --2- 172.21.15.92:0/2272127174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1806ece0 0x7fab180b2360 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fab08009f50 tx=0x7fab08009840 comp rx=0 tx=0).stop 2023-10-06T10:46:01.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.543+0000 7fab1ecb2700 1 -- 172.21.15.92:0/2272127174 shutdown_connections 2023-10-06T10:46:01.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.543+0000 7fab1ecb2700 1 --2- 172.21.15.92:0/2272127174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1806ece0 0x7fab180b2360 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.543+0000 7fab1ecb2700 1 --2- 172.21.15.92:0/2272127174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab1806f0b0 0x7fab180b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.543+0000 7fab1ecb2700 1 -- 172.21.15.92:0/2272127174 >> 172.21.15.92:0/2272127174 conn(0x7fab180682f0 msgr2=0x7fab180686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:01.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.543+0000 7fab1ecb2700 1 -- 172.21.15.92:0/2272127174 shutdown_connections 2023-10-06T10:46:01.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.543+0000 7fab1ecb2700 1 -- 172.21.15.92:0/2272127174 wait complete. 2023-10-06T10:46:01.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.544+0000 7fab1ecb2700 1 Processor -- start 2023-10-06T10:46:01.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.545+0000 7fab1ecb2700 1 -- start start 2023-10-06T10:46:01.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.545+0000 7fab1ecb2700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab1806ece0 0x7fab18169410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.545+0000 7fab1ecb2700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab1806f0b0 0x7fab180adc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.545+0000 7fab1ecb2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab180aede0 0x7fab180ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.545+0000 7fab1ecb2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab180a8dc0 con 0x7fab180aede0 2023-10-06T10:46:01.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.545+0000 7fab1ecb2700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab180ab420 con 0x7fab1806ece0 2023-10-06T10:46:01.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.545+0000 7fab1ecb2700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab180ae790 con 0x7fab1806f0b0 2023-10-06T10:46:01.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.546+0000 7fab1d24f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab180aede0 0x7fab180ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.546+0000 7fab1d24f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab180aede0 0x7fab180ae180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38146/0 (socket says 172.21.15.92:38146) 2023-10-06T10:46:01.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.546+0000 7fab1d24f700 1 -- 172.21.15.92:0/2387023336 learned_addr learned my addr 172.21.15.92:0/2387023336 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:46:01.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.546+0000 7fab1ca4e700 1 --2- 172.21.15.92:0/2387023336 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab1806ece0 0x7fab18169410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.546+0000 7fab17fff700 1 --2- 172.21.15.92:0/2387023336 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab1806f0b0 0x7fab180adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.546+0000 7fab1d24f700 1 -- 172.21.15.92:0/2387023336 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab1806f0b0 msgr2=0x7fab180adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.546+0000 7fab1d24f700 1 --2- 172.21.15.92:0/2387023336 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab1806f0b0 0x7fab180adc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.546+0000 7fab1d24f700 1 -- 172.21.15.92:0/2387023336 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab1806ece0 msgr2=0x7fab18169410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.546+0000 7fab1d24f700 1 --2- 172.21.15.92:0/2387023336 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab1806ece0 0x7fab18169410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.546+0000 7fab1d24f700 1 -- 172.21.15.92:0/2387023336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab08009ae0 con 0x7fab180aede0 2023-10-06T10:46:01.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.546+0000 7fab1d24f700 1 --2- 172.21.15.92:0/2387023336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab180aede0 0x7fab180ae180 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fab1000c5b0 tx=0x7fab10007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:01.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.546+0000 7fab14ff9700 1 -- 172.21.15.92:0/2387023336 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab10011590 con 0x7fab180aede0 2023-10-06T10:46:01.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.546+0000 7fab14ff9700 1 -- 172.21.15.92:0/2387023336 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab10011710 con 0x7fab180aede0 2023-10-06T10:46:01.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.546+0000 7fab1ecb2700 1 -- 172.21.15.92:0/2387023336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab180aea70 con 0x7fab180aede0 2023-10-06T10:46:01.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.546+0000 7fab14ff9700 1 -- 172.21.15.92:0/2387023336 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab10011ad0 con 0x7fab180aede0 2023-10-06T10:46:01.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.546+0000 7fab1ecb2700 1 -- 172.21.15.92:0/2387023336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab180a8a50 con 0x7fab180aede0 2023-10-06T10:46:01.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.547+0000 7fab1ecb2700 1 -- 172.21.15.92:0/2387023336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fab18064a90 con 0x7fab180aede0 2023-10-06T10:46:01.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.547+0000 7fab14ff9700 1 -- 172.21.15.92:0/2387023336 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fab1001c460 con 0x7fab180aede0 2023-10-06T10:46:01.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.548+0000 7fab14ff9700 1 --2- 172.21.15.92:0/2387023336 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fab04066f30 0x7fab040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.548+0000 7fab14ff9700 1 -- 172.21.15.92:0/2387023336 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7fab100840d0 con 0x7fab180aede0 2023-10-06T10:46:01.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.548+0000 7fab1ca4e700 1 --2- 172.21.15.92:0/2387023336 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fab04066f30 0x7fab040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.548+0000 7fab14ff9700 1 --2- 172.21.15.92:0/2387023336 >> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] conn(0x7fab0406d990 0x7fab0406fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.548+0000 7fab14ff9700 1 -- 172.21.15.92:0/2387023336 --> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab04070490 con 0x7fab0406d990 2023-10-06T10:46:01.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.548+0000 7fab14ff9700 1 -- 172.21.15.92:0/2387023336 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab10086710 con 0x7fab180aede0 2023-10-06T10:46:01.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.548+0000 7fab1ca4e700 1 --2- 172.21.15.92:0/2387023336 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fab04066f30 0x7fab040693e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fab08009880 tx=0x7fab08017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:01.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.548+0000 7fab17fff700 1 --2- 172.21.15.92:0/2387023336 >> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] conn(0x7fab0406d990 0x7fab0406fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.549+0000 7fab17fff700 1 --2- 172.21.15.92:0/2387023336 >> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] conn(0x7fab0406d990 0x7fab0406fdc0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:01.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.550+0000 7fab14ff9700 1 -- 172.21.15.92:0/2387023336 <== osd.5 v2:172.21.15.124:6816/90907 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fab04070490 con 0x7fab0406d990 2023-10-06T10:46:01.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.563+0000 7fab1ecb2700 1 -- 172.21.15.92:0/2387023336 --> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fab180d1480 con 0x7fab0406d990 2023-10-06T10:46:01.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.564+0000 7fab14ff9700 1 -- 172.21.15.92:0/2387023336 <== osd.5 v2:172.21.15.124:6816/90907 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fab180d1480 con 0x7fab0406d990 2023-10-06T10:46:01.566 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:46:01.566 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T10:46:01.566 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:46:01.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.564+0000 7fab1ecb2700 1 -- 172.21.15.92:0/2387023336 >> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] conn(0x7fab0406d990 msgr2=0x7fab0406fdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.564+0000 7fab1ecb2700 1 --2- 172.21.15.92:0/2387023336 >> [v2:172.21.15.124:6816/90907,v1:172.21.15.124:6817/90907] conn(0x7fab0406d990 0x7fab0406fdc0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.564+0000 7fab1ecb2700 1 -- 172.21.15.92:0/2387023336 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fab04066f30 msgr2=0x7fab040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.564+0000 7fab1ecb2700 1 --2- 172.21.15.92:0/2387023336 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fab04066f30 0x7fab040693e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fab08009880 tx=0x7fab08017000 comp rx=0 tx=0).stop 2023-10-06T10:46:01.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.564+0000 7fab1ecb2700 1 -- 172.21.15.92:0/2387023336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab180aede0 msgr2=0x7fab180ae180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.564+0000 7fab1ecb2700 1 --2- 172.21.15.92:0/2387023336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab180aede0 0x7fab180ae180 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fab1000c5b0 tx=0x7fab10007f60 comp rx=0 tx=0).stop 2023-10-06T10:46:01.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.564+0000 7fab1d24f700 1 -- 172.21.15.92:0/2387023336 reap_dead start 2023-10-06T10:46:01.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.564+0000 7fab1ecb2700 1 -- 172.21.15.92:0/2387023336 shutdown_connections 2023-10-06T10:46:01.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.564+0000 7fab1ecb2700 1 -- 172.21.15.92:0/2387023336 >> 172.21.15.92:0/2387023336 conn(0x7fab180682f0 msgr2=0x7fab180b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:01.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.564+0000 7fab1ecb2700 1 -- 172.21.15.92:0/2387023336 shutdown_connections 2023-10-06T10:46:01.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.565+0000 7fab1ecb2700 1 -- 172.21.15.92:0/2387023336 wait complete. 2023-10-06T10:46:01.578 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-06T10:46:01.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.692+0000 7f241304a700 1 Processor -- start 2023-10-06T10:46:01.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.692+0000 7f241304a700 1 -- start start 2023-10-06T10:46:01.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.693+0000 7f241304a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f240c159440 0x7f240c0b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.693+0000 7f241304a700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f240c0b2100 0x7f240c0b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.693+0000 7f241304a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f240c0cb7a0 con 0x7f240c0b35d0 2023-10-06T10:46:01.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.693+0000 7f241304a700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f240c0b0860 con 0x7f240c0b2100 2023-10-06T10:46:01.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.693+0000 7f241304a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f240c0cb150 con 0x7f240c159440 2023-10-06T10:46:01.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.693+0000 7f2411847700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f240c0b35d0 0x7f240c0cbc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35470/0 (socket says 172.21.15.92:35470) 2023-10-06T10:46:01.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.693+0000 7f2411847700 1 -- 172.21.15.92:0/761411315 learned_addr learned my addr 172.21.15.92:0/761411315 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:46:01.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.693+0000 7f2412849700 1 --2- 172.21.15.92:0/761411315 >> v2:172.21.15.124:3300/0 conn(0x7f240c0b2100 0x7f240c0b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.693+0000 7f2412048700 1 --2- 172.21.15.92:0/761411315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f240c159440 0x7f240c0b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.693+0000 7f2411046700 1 -- 172.21.15.92:0/761411315 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2620381071 0 0) 0x7f240c0cb7a0 con 0x7f240c0b35d0 2023-10-06T10:46:01.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.693+0000 7f2411046700 1 -- 172.21.15.92:0/761411315 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23f8003570 con 0x7f240c0b35d0 2023-10-06T10:46:01.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411046700 1 -- 172.21.15.92:0/761411315 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2259682028 0 0) 0x7f23f8003570 con 0x7f240c0b35d0 2023-10-06T10:46:01.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411046700 1 -- 172.21.15.92:0/761411315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f240c159440 msgr2=0x7f240c0b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:01.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411046700 1 --2- 172.21.15.92:0/761411315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f240c159440 0x7f240c0b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411046700 1 -- 172.21.15.92:0/761411315 >> v2:172.21.15.124:3300/0 conn(0x7f240c0b2100 msgr2=0x7f240c0b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:01.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411046700 1 --2- 172.21.15.92:0/761411315 >> v2:172.21.15.124:3300/0 conn(0x7f240c0b2100 0x7f240c0b2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411046700 1 -- 172.21.15.92:0/761411315 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240c0cb430 con 0x7f240c0b35d0 2023-10-06T10:46:01.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411046700 1 -- 172.21.15.92:0/761411315 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f23fc001470 con 0x7f240c0b35d0 2023-10-06T10:46:01.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411046700 1 -- 172.21.15.92:0/761411315 >> v1:172.21.15.92:6789/0 conn(0x7f240c0b35d0 legacy=0x7f240c0cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411046700 1 --2- 172.21.15.92:0/761411315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f240c159440 0x7f23f8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411046700 1 --2- 172.21.15.92:0/761411315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23f8003fe0 0x7f23f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411046700 1 --2- 172.21.15.92:0/761411315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23f8004d50 0x7f23f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411046700 1 -- 172.21.15.92:0/761411315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c0b0860 con 0x7f23f8003fe0 2023-10-06T10:46:01.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411046700 1 -- 172.21.15.92:0/761411315 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c0cb150 con 0x7f23f8004d50 2023-10-06T10:46:01.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411046700 1 -- 172.21.15.92:0/761411315 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f800ba90 con 0x7f240c159440 2023-10-06T10:46:01.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411847700 1 --2- 172.21.15.92:0/761411315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23f8003fe0 0x7f23f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2412849700 1 --2- 172.21.15.92:0/761411315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23f8004d50 0x7f23f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2412048700 1 --2- 172.21.15.92:0/761411315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f240c159440 0x7f23f8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411847700 1 -- 172.21.15.92:0/761411315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f240c159440 msgr2=0x7f23f8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:01.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411847700 1 --2- 172.21.15.92:0/761411315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f240c159440 0x7f23f8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411847700 1 -- 172.21.15.92:0/761411315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23f8004d50 msgr2=0x7f23f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:01.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411847700 1 --2- 172.21.15.92:0/761411315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23f8004d50 0x7f23f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411847700 1 -- 172.21.15.92:0/761411315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f8003570 con 0x7f23f8003fe0 2023-10-06T10:46:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2412048700 1 --2- 172.21.15.92:0/761411315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f240c159440 0x7f23f8003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:46:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411847700 1 --2- 172.21.15.92:0/761411315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23f8003fe0 0x7f23f8004810 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00c3c0 tx=0x7f23fc001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c17e05156cb12b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411046700 1 -- 172.21.15.92:0/761411315 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc00cc30 con 0x7f23f8003fe0 2023-10-06T10:46:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411046700 1 -- 172.21.15.92:0/761411315 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23fc00cdb0 con 0x7f23f8003fe0 2023-10-06T10:46:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.694+0000 7f2411046700 1 -- 172.21.15.92:0/761411315 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc005b70 con 0x7f23f8003fe0 2023-10-06T10:46:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.695+0000 7f241304a700 1 -- 172.21.15.92:0/761411315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23f8003fe0 msgr2=0x7f23f8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.695+0000 7f241304a700 1 --2- 172.21.15.92:0/761411315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23f8003fe0 0x7f23f8004810 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00c3c0 tx=0x7f23fc001e90 comp rx=0 tx=0).stop 2023-10-06T10:46:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.695+0000 7f2412849700 1 -- 172.21.15.92:0/761411315 reap_dead start 2023-10-06T10:46:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.695+0000 7f241304a700 1 -- 172.21.15.92:0/761411315 shutdown_connections 2023-10-06T10:46:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.695+0000 7f241304a700 1 -- 172.21.15.92:0/761411315 >> 172.21.15.92:0/761411315 conn(0x7f240c06ece0 msgr2=0x7f240c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.695+0000 7f241304a700 1 -- 172.21.15.92:0/761411315 shutdown_connections 2023-10-06T10:46:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.695+0000 7f241304a700 1 -- 172.21.15.92:0/761411315 wait complete. 2023-10-06T10:46:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.696+0000 7f241304a700 1 Processor -- start 2023-10-06T10:46:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.696+0000 7f241304a700 1 -- start start 2023-10-06T10:46:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.697+0000 7f241304a700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23f8003fe0 0x7f240c154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.697+0000 7f241304a700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f240c0c5510 0x7f240c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.697+0000 7f241304a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f240c157a20 0x7f240c157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.697+0000 7f241304a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c1583d0 con 0x7f240c157a20 2023-10-06T10:46:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.697+0000 7f241304a700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c0cddf0 con 0x7f23f8003fe0 2023-10-06T10:46:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.697+0000 7f241304a700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f240c0c5fb0 con 0x7f240c0c5510 2023-10-06T10:46:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.697+0000 7f2412849700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f240c157a20 0x7f240c157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.697+0000 7f2412849700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f240c157a20 0x7f240c157e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38160/0 (socket says 172.21.15.92:38160) 2023-10-06T10:46:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.697+0000 7f2412849700 1 -- 172.21.15.92:0/1464284042 learned_addr learned my addr 172.21.15.92:0/1464284042 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:46:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.697+0000 7f2412048700 1 --2- 172.21.15.92:0/1464284042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23f8003fe0 0x7f240c154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.697+0000 7f2411847700 1 --2- 172.21.15.92:0/1464284042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f240c0c5510 0x7f240c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.697+0000 7f2412849700 1 -- 172.21.15.92:0/1464284042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f240c0c5510 msgr2=0x7f240c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.697+0000 7f2412849700 1 --2- 172.21.15.92:0/1464284042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f240c0c5510 0x7f240c0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.697+0000 7f2412849700 1 -- 172.21.15.92:0/1464284042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23f8003fe0 msgr2=0x7f240c154840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.697+0000 7f2412849700 1 --2- 172.21.15.92:0/1464284042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23f8003fe0 0x7f240c154840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.697+0000 7f2412849700 1 -- 172.21.15.92:0/1464284042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240c0c6290 con 0x7f240c157a20 2023-10-06T10:46:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.697+0000 7f2412048700 1 --2- 172.21.15.92:0/1464284042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23f8003fe0 0x7f240c154840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:46:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.697+0000 7f2411847700 1 --2- 172.21.15.92:0/1464284042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f240c0c5510 0x7f240c0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:46:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.697+0000 7f2412849700 1 --2- 172.21.15.92:0/1464284042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f240c157a20 0x7f240c157e90 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f240c0ca7f0 tx=0x7f240400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.697+0000 7f24027fc700 1 -- 172.21.15.92:0/1464284042 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240400e040 con 0x7f240c157a20 2023-10-06T10:46:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.698+0000 7f24027fc700 1 -- 172.21.15.92:0/1464284042 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240400f040 con 0x7f240c157a20 2023-10-06T10:46:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.698+0000 7f241304a700 1 -- 172.21.15.92:0/1464284042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f240c0c6570 con 0x7f240c157a20 2023-10-06T10:46:01.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.698+0000 7f24027fc700 1 -- 172.21.15.92:0/1464284042 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240400fe20 con 0x7f240c157a20 2023-10-06T10:46:01.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.698+0000 7f241304a700 1 -- 172.21.15.92:0/1464284042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240c0af3d0 con 0x7f240c157a20 2023-10-06T10:46:01.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.699+0000 7f241304a700 1 -- 172.21.15.92:0/1464284042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f240c0cdf80 con 0x7f240c157a20 2023-10-06T10:46:01.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.699+0000 7f24027fc700 1 -- 172.21.15.92:0/1464284042 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2404015640 con 0x7f240c157a20 2023-10-06T10:46:01.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.699+0000 7f24027fc700 1 --2- 172.21.15.92:0/1464284042 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f23f0066f30 0x7f23f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:01.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.699+0000 7f2412048700 1 --2- 172.21.15.92:0/1464284042 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f23f0066f30 0x7f23f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:01.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.699+0000 7f24027fc700 1 -- 172.21.15.92:0/1464284042 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f2404084f80 con 0x7f240c157a20 2023-10-06T10:46:01.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.700+0000 7f2412048700 1 --2- 172.21.15.92:0/1464284042 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f23f0066f30 0x7f23f00693e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2408006dc0 tx=0x7f2408008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:01.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.704+0000 7f24027fc700 1 -- 172.21.15.92:0/1464284042 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24040521a0 con 0x7f240c157a20 2023-10-06T10:46:01.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.887+0000 7f241304a700 1 -- 172.21.15.92:0/1464284042 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f240c05f230 con 0x7f23f0066f30 2023-10-06T10:46:01.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.887+0000 7f24027fc700 1 -- 172.21.15.92:0/1464284042 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f240c05f230 con 0x7f23f0066f30 2023-10-06T10:46:01.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.889+0000 7f241304a700 1 -- 172.21.15.92:0/1464284042 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f23f0066f30 msgr2=0x7f23f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.889+0000 7f241304a700 1 --2- 172.21.15.92:0/1464284042 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f23f0066f30 0x7f23f00693e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2408006dc0 tx=0x7f2408008040 comp rx=0 tx=0).stop 2023-10-06T10:46:01.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.889+0000 7f241304a700 1 -- 172.21.15.92:0/1464284042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f240c157a20 msgr2=0x7f240c157e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:01.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.889+0000 7f241304a700 1 --2- 172.21.15.92:0/1464284042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f240c157a20 0x7f240c157e90 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f240c0ca7f0 tx=0x7f240400b040 comp rx=0 tx=0).stop 2023-10-06T10:46:01.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.889+0000 7f241304a700 1 -- 172.21.15.92:0/1464284042 shutdown_connections 2023-10-06T10:46:01.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.889+0000 7f241304a700 1 --2- 172.21.15.92:0/1464284042 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f23f0066f30 0x7f23f00693e0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.889+0000 7f241304a700 1 --2- 172.21.15.92:0/1464284042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f240c157a20 0x7f240c157e90 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.889+0000 7f241304a700 1 --2- 172.21.15.92:0/1464284042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23f8003fe0 0x7f240c154840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.889+0000 7f241304a700 1 --2- 172.21.15.92:0/1464284042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f240c0c5510 0x7f240c0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:01.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.889+0000 7f241304a700 1 -- 172.21.15.92:0/1464284042 >> 172.21.15.92:0/1464284042 conn(0x7f240c06ece0 msgr2=0x7f240c0b55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:01.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.889+0000 7f241304a700 1 -- 172.21.15.92:0/1464284042 shutdown_connections 2023-10-06T10:46:01.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:01.889+0000 7f241304a700 1 -- 172.21.15.92:0/1464284042 wait complete. 2023-10-06T10:46:01.892 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 5 to deep-scrub 2023-10-06T10:46:01.903 INFO:tasks.ceph:Scrubbing osd.6 2023-10-06T10:46:01.903 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-06T10:46:02.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.018+0000 7f759240d700 1 Processor -- start 2023-10-06T10:46:02.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.018+0000 7f759240d700 1 -- start start 2023-10-06T10:46:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.018+0000 7f759240d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f758c06ece0 0x7f758c0b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.018+0000 7f759240d700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f758c06f0b0 0x7f758c0b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.018+0000 7f759240d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f758c0aede0 0x7f758c0af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.018+0000 7f759240d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758c06fb40 con 0x7f758c06ece0 2023-10-06T10:46:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.018+0000 7f759240d700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758c0b2ce0 con 0x7f758c06f0b0 2023-10-06T10:46:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.018+0000 7f759240d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c0b0410 con 0x7f758c0aede0 2023-10-06T10:46:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.019+0000 7f758bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f758c06ece0 0x7f758c0b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.019+0000 7f758bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f758c06ece0 0x7f758c0b2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38172/0 (socket says 172.21.15.92:38172) 2023-10-06T10:46:02.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.019+0000 7f758bfff700 1 -- 172.21.15.92:0/608028275 learned_addr learned my addr 172.21.15.92:0/608028275 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:46:02.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.019+0000 7f75909aa700 1 --2- 172.21.15.92:0/608028275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f758c0aede0 0x7f758c0af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.019+0000 7f758b7fe700 1 --2- 172.21.15.92:0/608028275 >> v2:172.21.15.124:3300/0 conn(0x7f758c06f0b0 0x7f758c0b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.019+0000 7f758bfff700 1 -- 172.21.15.92:0/608028275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f758c0aede0 msgr2=0x7f758c0af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:02.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.019+0000 7f758bfff700 1 --2- 172.21.15.92:0/608028275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f758c0aede0 0x7f758c0af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.019+0000 7f758bfff700 1 -- 172.21.15.92:0/608028275 >> v2:172.21.15.124:3300/0 conn(0x7f758c06f0b0 msgr2=0x7f758c0b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:02.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.019+0000 7f758bfff700 1 --2- 172.21.15.92:0/608028275 >> v2:172.21.15.124:3300/0 conn(0x7f758c06f0b0 0x7f758c0b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.019+0000 7f758bfff700 1 -- 172.21.15.92:0/608028275 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c0af780 con 0x7f758c06ece0 2023-10-06T10:46:02.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.019+0000 7f758b7fe700 1 --2- 172.21.15.92:0/608028275 >> v2:172.21.15.124:3300/0 conn(0x7f758c06f0b0 0x7f758c0b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:46:02.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.019+0000 7f758bfff700 1 --2- 172.21.15.92:0/608028275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f758c06ece0 0x7f758c0b2360 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f758c0b18e0 tx=0x7f757c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246f2fff6db4057 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:02.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.019+0000 7f758affd700 1 -- 172.21.15.92:0/608028275 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c009e70 con 0x7f758c06ece0 2023-10-06T10:46:02.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.019+0000 7f758affd700 1 -- 172.21.15.92:0/608028275 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757c00b860 con 0x7f758c06ece0 2023-10-06T10:46:02.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.019+0000 7f758affd700 1 -- 172.21.15.92:0/608028275 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c00bc00 con 0x7f758c06ece0 2023-10-06T10:46:02.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.020+0000 7f759240d700 1 -- 172.21.15.92:0/608028275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f758c06ece0 msgr2=0x7f758c0b2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.020+0000 7f759240d700 1 --2- 172.21.15.92:0/608028275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f758c06ece0 0x7f758c0b2360 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f758c0b18e0 tx=0x7f757c009250 comp rx=0 tx=0).stop 2023-10-06T10:46:02.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.020+0000 7f759240d700 1 -- 172.21.15.92:0/608028275 shutdown_connections 2023-10-06T10:46:02.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.020+0000 7f759240d700 1 --2- 172.21.15.92:0/608028275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f758c06ece0 0x7f758c0b2360 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.020+0000 7f759240d700 1 --2- 172.21.15.92:0/608028275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f758c0aede0 0x7f758c0af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.020+0000 7f759240d700 1 --2- 172.21.15.92:0/608028275 >> v2:172.21.15.124:3300/0 conn(0x7f758c06f0b0 0x7f758c0b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.020+0000 7f759240d700 1 -- 172.21.15.92:0/608028275 >> 172.21.15.92:0/608028275 conn(0x7f758c0682f0 msgr2=0x7f758c0686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:02.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.020+0000 7f759240d700 1 -- 172.21.15.92:0/608028275 shutdown_connections 2023-10-06T10:46:02.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.020+0000 7f759240d700 1 -- 172.21.15.92:0/608028275 wait complete. 2023-10-06T10:46:02.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.021+0000 7f759240d700 1 Processor -- start 2023-10-06T10:46:02.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.021+0000 7f759240d700 1 -- start start 2023-10-06T10:46:02.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.022+0000 7f759240d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f758c06ece0 0x7f758c155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.022+0000 7f759240d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f758c06f0b0 0x7f758c0aa330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.022+0000 7f759240d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f758c0aede0 0x7f758c0b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.022+0000 7f759240d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c0ad980 con 0x7f758c06ece0 2023-10-06T10:46:02.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.022+0000 7f759240d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c16cf50 con 0x7f758c0aede0 2023-10-06T10:46:02.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.022+0000 7f759240d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c0a94e0 con 0x7f758c06f0b0 2023-10-06T10:46:02.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.022+0000 7f758bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f758c06ece0 0x7f758c155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.022+0000 7f758bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f758c06ece0 0x7f758c155910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38174/0 (socket says 172.21.15.92:38174) 2023-10-06T10:46:02.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.022+0000 7f758bfff700 1 -- 172.21.15.92:0/1744177797 learned_addr learned my addr 172.21.15.92:0/1744177797 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:46:02.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.022+0000 7f75909aa700 1 --2- 172.21.15.92:0/1744177797 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f758c0aede0 0x7f758c0b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.022+0000 7f758b7fe700 1 --2- 172.21.15.92:0/1744177797 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f758c06f0b0 0x7f758c0aa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.022+0000 7f758bfff700 1 -- 172.21.15.92:0/1744177797 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f758c06f0b0 msgr2=0x7f758c0aa330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.022+0000 7f758bfff700 1 --2- 172.21.15.92:0/1744177797 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f758c06f0b0 0x7f758c0aa330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.022+0000 7f758bfff700 1 -- 172.21.15.92:0/1744177797 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f758c0aede0 msgr2=0x7f758c0b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.022+0000 7f758bfff700 1 --2- 172.21.15.92:0/1744177797 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f758c0aede0 0x7f758c0b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.022+0000 7f758bfff700 1 -- 172.21.15.92:0/1744177797 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c0094f0 con 0x7f758c06ece0 2023-10-06T10:46:02.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.022+0000 7f75909aa700 1 --2- 172.21.15.92:0/1744177797 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f758c0aede0 0x7f758c0b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:46:02.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.022+0000 7f758bfff700 1 --2- 172.21.15.92:0/1744177797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f758c06ece0 0x7f758c155910 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f757c009290 tx=0x7f757c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:02.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.023+0000 7f7573fff700 1 -- 172.21.15.92:0/1744177797 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c00bbe0 con 0x7f758c06ece0 2023-10-06T10:46:02.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.023+0000 7f7573fff700 1 -- 172.21.15.92:0/1744177797 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757c01f070 con 0x7f758c06ece0 2023-10-06T10:46:02.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.023+0000 7f759240d700 1 -- 172.21.15.92:0/1744177797 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758c16d1a0 con 0x7f758c06ece0 2023-10-06T10:46:02.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.023+0000 7f7573fff700 1 -- 172.21.15.92:0/1744177797 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c026690 con 0x7f758c06ece0 2023-10-06T10:46:02.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.023+0000 7f759240d700 1 -- 172.21.15.92:0/1744177797 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758c16d760 con 0x7f758c06ece0 2023-10-06T10:46:02.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.023+0000 7f759240d700 1 -- 172.21.15.92:0/1744177797 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f758c064a90 con 0x7f758c06ece0 2023-10-06T10:46:02.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.024+0000 7f7573fff700 1 -- 172.21.15.92:0/1744177797 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f757c01a430 con 0x7f758c06ece0 2023-10-06T10:46:02.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.024+0000 7f7573fff700 1 --2- 172.21.15.92:0/1744177797 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7578066f60 0x7f7578069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.024+0000 7f7573fff700 1 -- 172.21.15.92:0/1744177797 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f757c021070 con 0x7f758c06ece0 2023-10-06T10:46:02.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.024+0000 7f758b7fe700 1 --2- 172.21.15.92:0/1744177797 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7578066f60 0x7f7578069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.024+0000 7f7573fff700 1 --2- 172.21.15.92:0/1744177797 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x7f757806d9c0 0x7f757806fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.024+0000 7f7573fff700 1 -- 172.21.15.92:0/1744177797 --> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f75780704c0 con 0x7f757806d9c0 2023-10-06T10:46:02.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.024+0000 7f7573fff700 1 -- 172.21.15.92:0/1744177797 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f757c088790 con 0x7f758c06ece0 2023-10-06T10:46:02.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.025+0000 7f75909aa700 1 --2- 172.21.15.92:0/1744177797 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x7f757806d9c0 0x7f757806fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.025+0000 7f758b7fe700 1 --2- 172.21.15.92:0/1744177797 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7578066f60 0x7f7578069410 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7580006f00 tx=0x7f7580008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:02.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.025+0000 7f75909aa700 1 --2- 172.21.15.92:0/1744177797 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x7f757806d9c0 0x7f757806fdf0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:02.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.026+0000 7f7573fff700 1 -- 172.21.15.92:0/1744177797 <== osd.6 v2:172.21.15.124:6824/90914 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f75780704c0 con 0x7f757806d9c0 2023-10-06T10:46:02.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.039+0000 7f759240d700 1 -- 172.21.15.92:0/1744177797 --> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f758c156b20 con 0x7f757806d9c0 2023-10-06T10:46:02.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.041+0000 7f7573fff700 1 -- 172.21.15.92:0/1744177797 <== osd.6 v2:172.21.15.124:6824/90914 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f758c156b20 con 0x7f757806d9c0 2023-10-06T10:46:02.042 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:46:02.042 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T10:46:02.042 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:46:02.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.041+0000 7f759240d700 1 -- 172.21.15.92:0/1744177797 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x7f757806d9c0 msgr2=0x7f757806fdf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.041+0000 7f759240d700 1 --2- 172.21.15.92:0/1744177797 >> [v2:172.21.15.124:6824/90914,v1:172.21.15.124:6825/90914] conn(0x7f757806d9c0 0x7f757806fdf0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.041+0000 7f759240d700 1 -- 172.21.15.92:0/1744177797 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7578066f60 msgr2=0x7f7578069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.041+0000 7f759240d700 1 --2- 172.21.15.92:0/1744177797 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7578066f60 0x7f7578069410 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7580006f00 tx=0x7f7580008040 comp rx=0 tx=0).stop 2023-10-06T10:46:02.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.041+0000 7f759240d700 1 -- 172.21.15.92:0/1744177797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f758c06ece0 msgr2=0x7f758c155910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.041+0000 7f759240d700 1 --2- 172.21.15.92:0/1744177797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f758c06ece0 0x7f758c155910 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f757c009290 tx=0x7f757c017000 comp rx=0 tx=0).stop 2023-10-06T10:46:02.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.041+0000 7f75909aa700 1 -- 172.21.15.92:0/1744177797 reap_dead start 2023-10-06T10:46:02.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.041+0000 7f759240d700 1 -- 172.21.15.92:0/1744177797 shutdown_connections 2023-10-06T10:46:02.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.041+0000 7f759240d700 1 -- 172.21.15.92:0/1744177797 >> 172.21.15.92:0/1744177797 conn(0x7f758c0682f0 msgr2=0x7f758c0d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:02.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.041+0000 7f759240d700 1 -- 172.21.15.92:0/1744177797 shutdown_connections 2023-10-06T10:46:02.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.041+0000 7f759240d700 1 -- 172.21.15.92:0/1744177797 wait complete. 2023-10-06T10:46:02.053 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-06T10:46:02.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.167+0000 7f798a3f1700 1 Processor -- start 2023-10-06T10:46:02.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.167+0000 7f798a3f1700 1 -- start start 2023-10-06T10:46:02.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.168+0000 7f798a3f1700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f79840b1830 0x7f79840c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.168+0000 7f798a3f1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f798415ad50 con 0x7f7984156010 2023-10-06T10:46:02.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.168+0000 7f798a3f1700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7984159e60 con 0x7f79841563e0 2023-10-06T10:46:02.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.168+0000 7f798a3f1700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f798415a140 con 0x7f79840b1830 2023-10-06T10:46:02.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.168+0000 7f79893ef700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7984156010 0x7f79840b07b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35478/0 (socket says 172.21.15.92:35478) 2023-10-06T10:46:02.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.168+0000 7f79893ef700 1 -- 172.21.15.92:0/1706944339 learned_addr learned my addr 172.21.15.92:0/1706944339 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:46:02.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.168+0000 7f7989bf0700 1 --2- 172.21.15.92:0/1706944339 >> v2:172.21.15.124:3300/0 conn(0x7f79840b1830 0x7f79840c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.168+0000 7f797bfff700 1 -- 172.21.15.92:0/1706944339 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897468842 0 0) 0x7f798415ad50 con 0x7f7984156010 2023-10-06T10:46:02.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.168+0000 7f797bfff700 1 -- 172.21.15.92:0/1706944339 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7970003570 con 0x7f7984156010 2023-10-06T10:46:02.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.168+0000 7f797bfff700 1 -- 172.21.15.92:0/1706944339 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 947780030 0 0) 0x7f7984159e60 con 0x7f79841563e0 2023-10-06T10:46:02.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.168+0000 7f797bfff700 1 -- 172.21.15.92:0/1706944339 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f798415ad50 con 0x7f79841563e0 2023-10-06T10:46:02.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.168+0000 7f7989bf0700 1 -- 172.21.15.92:0/1706944339 >> v1:172.21.15.124:6789/0 conn(0x7f79841563e0 legacy=0x7f79840b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.168+0000 7f7989bf0700 1 -- 172.21.15.92:0/1706944339 >> v1:172.21.15.92:6789/0 conn(0x7f7984156010 legacy=0x7f79840b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.168+0000 7f797bfff700 1 -- 172.21.15.92:0/1706944339 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 472534503 0 0) 0x7f7970003570 con 0x7f7984156010 2023-10-06T10:46:02.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.169+0000 7f7989bf0700 1 -- 172.21.15.92:0/1706944339 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798415a420 con 0x7f79840b1830 2023-10-06T10:46:02.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.169+0000 7f7989bf0700 1 --2- 172.21.15.92:0/1706944339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79840b1830 0x7f79840c9c00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f797c009e60 tx=0x7f797c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67c08c01854eef9a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:02.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.169+0000 7f797bfff700 1 -- 172.21.15.92:0/1706944339 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c00e070 con 0x7f79840b1830 2023-10-06T10:46:02.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.169+0000 7f797bfff700 1 -- 172.21.15.92:0/1706944339 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797c00bab0 con 0x7f79840b1830 2023-10-06T10:46:02.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.169+0000 7f797bfff700 1 -- 172.21.15.92:0/1706944339 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c00be70 con 0x7f79840b1830 2023-10-06T10:46:02.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.169+0000 7f798a3f1700 1 -- 172.21.15.92:0/1706944339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79840b1830 msgr2=0x7f79840c9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.169+0000 7f798a3f1700 1 --2- 172.21.15.92:0/1706944339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79840b1830 0x7f79840c9c00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f797c009e60 tx=0x7f797c009780 comp rx=0 tx=0).stop 2023-10-06T10:46:02.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.169+0000 7f798a3f1700 1 -- 172.21.15.92:0/1706944339 shutdown_connections 2023-10-06T10:46:02.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.170+0000 7f798a3f1700 1 --2- 172.21.15.92:0/1706944339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79840b1830 0x7f79840c9c00 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.170+0000 7f798a3f1700 1 -- 172.21.15.92:0/1706944339 >> 172.21.15.92:0/1706944339 conn(0x7f7984069190 msgr2=0x7f79840aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:02.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.170+0000 7f798a3f1700 1 -- 172.21.15.92:0/1706944339 shutdown_connections 2023-10-06T10:46:02.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.170+0000 7f798a3f1700 1 -- 172.21.15.92:0/1706944339 wait complete. 2023-10-06T10:46:02.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.171+0000 7f798a3f1700 1 Processor -- start 2023-10-06T10:46:02.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.171+0000 7f798a3f1700 1 -- start start 2023-10-06T10:46:02.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.172+0000 7f798a3f1700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79840b1830 0x7f7984159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.172+0000 7f798a3f1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7984156010 0x7f798413f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.172+0000 7f798a3f1700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79841563e0 0x7f798413fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.172+0000 7f798a3f1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798416a930 con 0x7f7984156010 2023-10-06T10:46:02.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.172+0000 7f798a3f1700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79840c84d0 con 0x7f79840b1830 2023-10-06T10:46:02.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.172+0000 7f798a3f1700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79840c7900 con 0x7f79841563e0 2023-10-06T10:46:02.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.172+0000 7f7988bee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7984156010 0x7f798413f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.172+0000 7f7988bee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7984156010 0x7f798413f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38178/0 (socket says 172.21.15.92:38178) 2023-10-06T10:46:02.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.172+0000 7f7988bee700 1 -- 172.21.15.92:0/3106614145 learned_addr learned my addr 172.21.15.92:0/3106614145 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:46:02.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.172+0000 7f79893ef700 1 --2- 172.21.15.92:0/3106614145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79840b1830 0x7f7984159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.172+0000 7f7989bf0700 1 --2- 172.21.15.92:0/3106614145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79841563e0 0x7f798413fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.172+0000 7f7988bee700 1 -- 172.21.15.92:0/3106614145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79841563e0 msgr2=0x7f798413fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.172+0000 7f7988bee700 1 --2- 172.21.15.92:0/3106614145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79841563e0 0x7f798413fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.172+0000 7f7988bee700 1 -- 172.21.15.92:0/3106614145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79840b1830 msgr2=0x7f7984159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.172+0000 7f7988bee700 1 --2- 172.21.15.92:0/3106614145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79840b1830 0x7f7984159510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.172+0000 7f7988bee700 1 -- 172.21.15.92:0/3106614145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797c009a20 con 0x7f7984156010 2023-10-06T10:46:02.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.172+0000 7f7988bee700 1 --2- 172.21.15.92:0/3106614145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7984156010 0x7f798413f550 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f797400c790 tx=0x7f7974007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:02.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.172+0000 7f79797fa700 1 -- 172.21.15.92:0/3106614145 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7974010510 con 0x7f7984156010 2023-10-06T10:46:02.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.172+0000 7f79797fa700 1 -- 172.21.15.92:0/3106614145 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7974010670 con 0x7f7984156010 2023-10-06T10:46:02.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.172+0000 7f798a3f1700 1 -- 172.21.15.92:0/3106614145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79840c7ba0 con 0x7f7984156010 2023-10-06T10:46:02.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.172+0000 7f79797fa700 1 -- 172.21.15.92:0/3106614145 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7974010a10 con 0x7f7984156010 2023-10-06T10:46:02.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.172+0000 7f798a3f1700 1 -- 172.21.15.92:0/3106614145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79840c8160 con 0x7f7984156010 2023-10-06T10:46:02.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.173+0000 7f798a3f1700 1 -- 172.21.15.92:0/3106614145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79840c8640 con 0x7f7984156010 2023-10-06T10:46:02.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.174+0000 7f79797fa700 1 -- 172.21.15.92:0/3106614145 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f797401b3b0 con 0x7f7984156010 2023-10-06T10:46:02.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.174+0000 7f79797fa700 1 --2- 172.21.15.92:0/3106614145 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7968066f30 0x7f79680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.174+0000 7f79797fa700 1 -- 172.21.15.92:0/3106614145 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f7974083df0 con 0x7f7984156010 2023-10-06T10:46:02.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.174+0000 7f79893ef700 1 --2- 172.21.15.92:0/3106614145 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7968066f30 0x7f79680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.175+0000 7f79893ef700 1 --2- 172.21.15.92:0/3106614145 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7968066f30 0x7f79680693e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f79800011e0 tx=0x7f7980007af0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:02.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.178+0000 7f79797fa700 1 -- 172.21.15.92:0/3106614145 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7974051010 con 0x7f7984156010 2023-10-06T10:46:02.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.361+0000 7f798a3f1700 1 -- 172.21.15.92:0/3106614145 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f798405f230 con 0x7f7968066f30 2023-10-06T10:46:02.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.361+0000 7f79797fa700 1 -- 172.21.15.92:0/3106614145 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f798405f230 con 0x7f7968066f30 2023-10-06T10:46:02.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.364+0000 7f798a3f1700 1 -- 172.21.15.92:0/3106614145 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7968066f30 msgr2=0x7f79680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.364+0000 7f798a3f1700 1 --2- 172.21.15.92:0/3106614145 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7968066f30 0x7f79680693e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f79800011e0 tx=0x7f7980007af0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.364+0000 7f798a3f1700 1 -- 172.21.15.92:0/3106614145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7984156010 msgr2=0x7f798413f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.364+0000 7f798a3f1700 1 --2- 172.21.15.92:0/3106614145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7984156010 0x7f798413f550 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f797400c790 tx=0x7f7974007e10 comp rx=0 tx=0).stop 2023-10-06T10:46:02.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.364+0000 7f798a3f1700 1 -- 172.21.15.92:0/3106614145 shutdown_connections 2023-10-06T10:46:02.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.364+0000 7f798a3f1700 1 --2- 172.21.15.92:0/3106614145 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f7968066f30 0x7f79680693e0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.364+0000 7f798a3f1700 1 --2- 172.21.15.92:0/3106614145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79841563e0 0x7f798413fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.364+0000 7f798a3f1700 1 --2- 172.21.15.92:0/3106614145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79840b1830 0x7f7984159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.364+0000 7f798a3f1700 1 --2- 172.21.15.92:0/3106614145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7984156010 0x7f798413f550 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.364+0000 7f798a3f1700 1 -- 172.21.15.92:0/3106614145 >> 172.21.15.92:0/3106614145 conn(0x7f7984069190 msgr2=0x7f7984151de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:02.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.364+0000 7f798a3f1700 1 -- 172.21.15.92:0/3106614145 shutdown_connections 2023-10-06T10:46:02.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.364+0000 7f798a3f1700 1 -- 172.21.15.92:0/3106614145 wait complete. 2023-10-06T10:46:02.367 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 6 to deep-scrub 2023-10-06T10:46:02.377 INFO:tasks.ceph:Scrubbing osd.7 2023-10-06T10:46:02.377 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-06T10:46:02.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.493+0000 7f3d7b59e700 1 Processor -- start 2023-10-06T10:46:02.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.493+0000 7f3d7b59e700 1 -- start start 2023-10-06T10:46:02.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.494+0000 7f3d7b59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d7c0d09c0 0x7f3d7c1642f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.494+0000 7f3d7b59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d7c0cdbf0 con 0x7f3d7c153990 2023-10-06T10:46:02.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.494+0000 7f3d7b59e700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d7c0cb4c0 con 0x7f3d7c0cca70 2023-10-06T10:46:02.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.494+0000 7f3d7b59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d7c0ccfe0 con 0x7f3d7c0d09c0 2023-10-06T10:46:02.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.494+0000 7f3d7ad9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d7c0d09c0 0x7f3d7c1642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.494+0000 7f3d7ad9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d7c0d09c0 0x7f3d7c1642f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38180/0 (socket says 172.21.15.92:38180) 2023-10-06T10:46:02.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.494+0000 7f3d7ad9d700 1 -- 172.21.15.92:0/7648098 learned_addr learned my addr 172.21.15.92:0/7648098 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:46:02.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.494+0000 7f3d7a59c700 1 --1- 172.21.15.92:0/7648098 >> v1:172.21.15.92:6789/0 conn(0x7f3d7c153990 0x7f3d7c15a370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35484/0 (socket says 172.21.15.92:35484) 2023-10-06T10:46:02.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.494+0000 7f3d7ad9d700 1 -- 172.21.15.92:0/7648098 >> v1:172.21.15.124:6789/0 conn(0x7f3d7c0cca70 legacy=0x7f3d7c0d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:02.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.494+0000 7f3d7ad9d700 1 -- 172.21.15.92:0/7648098 >> v1:172.21.15.92:6789/0 conn(0x7f3d7c153990 legacy=0x7f3d7c15a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.494+0000 7f3d7ad9d700 1 -- 172.21.15.92:0/7648098 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c0cd2c0 con 0x7f3d7c0d09c0 2023-10-06T10:46:02.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.494+0000 7f3d7959a700 1 -- 172.21.15.92:0/7648098 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1253044772 0 0) 0x7f3d7c0cdbf0 con 0x7f3d7c153990 2023-10-06T10:46:02.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.494+0000 7f3d7ad9d700 1 --2- 172.21.15.92:0/7648098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d7c0d09c0 0x7f3d7c1642f0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009f20 tx=0x7f3d74009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1818131348ec224 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:02.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.495+0000 7f3d7959a700 1 -- 172.21.15.92:0/7648098 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7400e070 con 0x7f3d7c0d09c0 2023-10-06T10:46:02.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.495+0000 7f3d7959a700 1 -- 172.21.15.92:0/7648098 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7400ba40 con 0x7f3d7c0d09c0 2023-10-06T10:46:02.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.495+0000 7f3d7959a700 1 -- 172.21.15.92:0/7648098 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d740092e0 con 0x7f3d7c0d09c0 2023-10-06T10:46:02.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.495+0000 7f3d7b59e700 1 -- 172.21.15.92:0/7648098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d7c0d09c0 msgr2=0x7f3d7c1642f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.495+0000 7f3d7b59e700 1 --2- 172.21.15.92:0/7648098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d7c0d09c0 0x7f3d7c1642f0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009f20 tx=0x7f3d74009840 comp rx=0 tx=0).stop 2023-10-06T10:46:02.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.495+0000 7f3d7b59e700 1 -- 172.21.15.92:0/7648098 shutdown_connections 2023-10-06T10:46:02.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.495+0000 7f3d7b59e700 1 --2- 172.21.15.92:0/7648098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d7c0d09c0 0x7f3d7c1642f0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.495+0000 7f3d7b59e700 1 -- 172.21.15.92:0/7648098 >> 172.21.15.92:0/7648098 conn(0x7f3d7c0680c0 msgr2=0x7f3d7c068490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:02.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.495+0000 7f3d7b59e700 1 -- 172.21.15.92:0/7648098 shutdown_connections 2023-10-06T10:46:02.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.495+0000 7f3d7b59e700 1 -- 172.21.15.92:0/7648098 wait complete. 2023-10-06T10:46:02.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.496+0000 7f3d7b59e700 1 Processor -- start 2023-10-06T10:46:02.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.496+0000 7f3d7b59e700 1 -- start start 2023-10-06T10:46:02.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.497+0000 7f3d7b59e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d7c0cca70 0x7f3d7c1510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.497+0000 7f3d7b59e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d7c0d09c0 0x7f3d7c15dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.497+0000 7f3d7b59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d7c153990 0x7f3d7c15e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.497+0000 7f3d7b59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0c8df0 con 0x7f3d7c153990 2023-10-06T10:46:02.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.497+0000 7f3d7b59e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0c9280 con 0x7f3d7c0cca70 2023-10-06T10:46:02.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.497+0000 7f3d7b59e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0c81e0 con 0x7f3d7c0d09c0 2023-10-06T10:46:02.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.497+0000 7f3d7ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d7c153990 0x7f3d7c15e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.497+0000 7f3d7ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d7c153990 0x7f3d7c15e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38184/0 (socket says 172.21.15.92:38184) 2023-10-06T10:46:02.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.497+0000 7f3d7ad9d700 1 -- 172.21.15.92:0/3860883257 learned_addr learned my addr 172.21.15.92:0/3860883257 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:46:02.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.497+0000 7f3d7a59c700 1 --2- 172.21.15.92:0/3860883257 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d7c0cca70 0x7f3d7c1510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.497+0000 7f3d79d9b700 1 --2- 172.21.15.92:0/3860883257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d7c0d09c0 0x7f3d7c15dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.497+0000 7f3d7ad9d700 1 -- 172.21.15.92:0/3860883257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d7c0d09c0 msgr2=0x7f3d7c15dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.497+0000 7f3d7ad9d700 1 --2- 172.21.15.92:0/3860883257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d7c0d09c0 0x7f3d7c15dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.497+0000 7f3d7ad9d700 1 -- 172.21.15.92:0/3860883257 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d7c0cca70 msgr2=0x7f3d7c1510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.497+0000 7f3d7ad9d700 1 --2- 172.21.15.92:0/3860883257 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d7c0cca70 0x7f3d7c1510f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.497+0000 7f3d7ad9d700 1 -- 172.21.15.92:0/3860883257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d74009ae0 con 0x7f3d7c153990 2023-10-06T10:46:02.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.498+0000 7f3d7ad9d700 1 --2- 172.21.15.92:0/3860883257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d7c153990 0x7f3d7c15e260 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f3d74003000 tx=0x7f3d74017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:02.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.498+0000 7f3d6a7fc700 1 -- 172.21.15.92:0/3860883257 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74012e60 con 0x7f3d7c153990 2023-10-06T10:46:02.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.498+0000 7f3d6a7fc700 1 -- 172.21.15.92:0/3860883257 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d740092e0 con 0x7f3d7c153990 2023-10-06T10:46:02.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.498+0000 7f3d7b59e700 1 -- 172.21.15.92:0/3860883257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7c0c84c0 con 0x7f3d7c153990 2023-10-06T10:46:02.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.498+0000 7f3d6a7fc700 1 -- 172.21.15.92:0/3860883257 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7401d650 con 0x7f3d7c153990 2023-10-06T10:46:02.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.498+0000 7f3d7b59e700 1 -- 172.21.15.92:0/3860883257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7c0c8a80 con 0x7f3d7c153990 2023-10-06T10:46:02.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.499+0000 7f3d7b59e700 1 -- 172.21.15.92:0/3860883257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3d7c05f230 con 0x7f3d7c153990 2023-10-06T10:46:02.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.499+0000 7f3d6a7fc700 1 -- 172.21.15.92:0/3860883257 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d7401da00 con 0x7f3d7c153990 2023-10-06T10:46:02.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.499+0000 7f3d6a7fc700 1 --2- 172.21.15.92:0/3860883257 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3d5c066f30 0x7f3d5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.499+0000 7f3d6a7fc700 1 -- 172.21.15.92:0/3860883257 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f3d74085ea0 con 0x7f3d7c153990 2023-10-06T10:46:02.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.499+0000 7f3d7a59c700 1 --2- 172.21.15.92:0/3860883257 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3d5c066f30 0x7f3d5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.500+0000 7f3d6a7fc700 1 --2- 172.21.15.92:0/3860883257 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x7f3d5c06d990 0x7f3d5c06fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.500+0000 7f3d6a7fc700 1 -- 172.21.15.92:0/3860883257 --> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d5c070490 con 0x7f3d5c06d990 2023-10-06T10:46:02.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.500+0000 7f3d6a7fc700 1 -- 172.21.15.92:0/3860883257 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d740884e0 con 0x7f3d7c153990 2023-10-06T10:46:02.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.500+0000 7f3d7a59c700 1 --2- 172.21.15.92:0/3860883257 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3d5c066f30 0x7f3d5c0693e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0011e0 tx=0x7f3d6c007bf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:02.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.500+0000 7f3d79d9b700 1 --2- 172.21.15.92:0/3860883257 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x7f3d5c06d990 0x7f3d5c06fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.500+0000 7f3d79d9b700 1 --2- 172.21.15.92:0/3860883257 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x7f3d5c06d990 0x7f3d5c06fdc0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:02.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.501+0000 7f3d6a7fc700 1 -- 172.21.15.92:0/3860883257 <== osd.7 v2:172.21.15.124:6808/90884 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f3d5c070490 con 0x7f3d5c06d990 2023-10-06T10:46:02.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.514+0000 7f3d7b59e700 1 -- 172.21.15.92:0/3860883257 --> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3d7c0cd5a0 con 0x7f3d5c06d990 2023-10-06T10:46:02.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.515+0000 7f3d6a7fc700 1 -- 172.21.15.92:0/3860883257 <== osd.7 v2:172.21.15.124:6808/90884 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3d7c0cd5a0 con 0x7f3d5c06d990 2023-10-06T10:46:02.517 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T10:46:02.517 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T10:46:02.517 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T10:46:02.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.515+0000 7f3d7b59e700 1 -- 172.21.15.92:0/3860883257 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x7f3d5c06d990 msgr2=0x7f3d5c06fdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.515+0000 7f3d7b59e700 1 --2- 172.21.15.92:0/3860883257 >> [v2:172.21.15.124:6808/90884,v1:172.21.15.124:6809/90884] conn(0x7f3d5c06d990 0x7f3d5c06fdc0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.515+0000 7f3d7b59e700 1 -- 172.21.15.92:0/3860883257 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3d5c066f30 msgr2=0x7f3d5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.515+0000 7f3d7b59e700 1 --2- 172.21.15.92:0/3860883257 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f3d5c066f30 0x7f3d5c0693e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0011e0 tx=0x7f3d6c007bf0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.515+0000 7f3d7b59e700 1 -- 172.21.15.92:0/3860883257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d7c153990 msgr2=0x7f3d7c15e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.515+0000 7f3d7b59e700 1 --2- 172.21.15.92:0/3860883257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d7c153990 0x7f3d7c15e260 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f3d74003000 tx=0x7f3d74017000 comp rx=0 tx=0).stop 2023-10-06T10:46:02.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.515+0000 7f3d7ad9d700 1 -- 172.21.15.92:0/3860883257 reap_dead start 2023-10-06T10:46:02.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.515+0000 7f3d7b59e700 1 -- 172.21.15.92:0/3860883257 shutdown_connections 2023-10-06T10:46:02.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.515+0000 7f3d7b59e700 1 -- 172.21.15.92:0/3860883257 >> 172.21.15.92:0/3860883257 conn(0x7f3d7c0680c0 msgr2=0x7f3d7c152f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:02.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.515+0000 7f3d7b59e700 1 -- 172.21.15.92:0/3860883257 shutdown_connections 2023-10-06T10:46:02.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.515+0000 7f3d7b59e700 1 -- 172.21.15.92:0/3860883257 wait complete. 2023-10-06T10:46:02.528 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-06T10:46:02.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.643+0000 7efe187ce700 1 Processor -- start 2023-10-06T10:46:02.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.643+0000 7efe187ce700 1 -- start start 2023-10-06T10:46:02.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.644+0000 7efe187ce700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efe1015a870 0x7efe100ce980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.644+0000 7efe187ce700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe1015ac40 0x7efe100c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.644+0000 7efe187ce700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe100cdb40 con 0x7efe100ceec0 2023-10-06T10:46:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.644+0000 7efe187ce700 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe100ccf30 con 0x7efe1015a870 2023-10-06T10:46:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.644+0000 7efe187ce700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe100cd210 con 0x7efe1015ac40 2023-10-06T10:46:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.644+0000 7efe1656a700 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efe1015a870 0x7efe100ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.644+0000 7efe15d69700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe1015ac40 0x7efe100c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.644+0000 7efe16d6b700 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7efe100ceec0 0x7efe100cbda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.92:49234/0 (socket says 172.21.15.92:49234) 2023-10-06T10:46:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.644+0000 7efe16d6b700 1 -- 172.21.15.92:0/1982261311 learned_addr learned my addr 172.21.15.92:0/1982261311 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:46:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.645+0000 7efe15d69700 1 -- 172.21.15.92:0/1982261311 >> v2:172.21.15.124:3300/0 conn(0x7efe1015a870 msgr2=0x7efe100ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.645+0000 7efe15d69700 1 --2- 172.21.15.92:0/1982261311 >> v2:172.21.15.124:3300/0 conn(0x7efe1015a870 0x7efe100ce980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.645+0000 7efe15d69700 1 -- 172.21.15.92:0/1982261311 >> v1:172.21.15.124:6789/0 conn(0x7efe100ceec0 legacy=0x7efe100cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.645+0000 7efe15d69700 1 -- 172.21.15.92:0/1982261311 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe100cd4f0 con 0x7efe1015ac40 2023-10-06T10:46:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.645+0000 7efe1656a700 1 --2- 172.21.15.92:0/1982261311 >> v2:172.21.15.124:3300/0 conn(0x7efe1015a870 0x7efe100ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T10:46:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.645+0000 7efe15d69700 1 --2- 172.21.15.92:0/1982261311 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe1015ac40 0x7efe100c8810 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efe000098e0 tx=0x7efe000091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdf8108ffa31dd63 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.645+0000 7efe15568700 1 -- 172.21.15.92:0/1982261311 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0000b640 con 0x7efe1015ac40 2023-10-06T10:46:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.645+0000 7efe15568700 1 -- 172.21.15.92:0/1982261311 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0000b7a0 con 0x7efe1015ac40 2023-10-06T10:46:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.645+0000 7efe15568700 1 -- 172.21.15.92:0/1982261311 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0000bb40 con 0x7efe1015ac40 2023-10-06T10:46:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.646+0000 7efe187ce700 1 -- 172.21.15.92:0/1982261311 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe1015ac40 msgr2=0x7efe100c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.646+0000 7efe187ce700 1 --2- 172.21.15.92:0/1982261311 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe1015ac40 0x7efe100c8810 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efe000098e0 tx=0x7efe000091a0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.646+0000 7efe187ce700 1 -- 172.21.15.92:0/1982261311 shutdown_connections 2023-10-06T10:46:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.646+0000 7efe187ce700 1 --2- 172.21.15.92:0/1982261311 >> v2:172.21.15.124:3300/0 conn(0x7efe1015a870 0x7efe100ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.646+0000 7efe187ce700 1 --2- 172.21.15.92:0/1982261311 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe1015ac40 0x7efe100c8810 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.646+0000 7efe187ce700 1 -- 172.21.15.92:0/1982261311 >> 172.21.15.92:0/1982261311 conn(0x7efe10067d90 msgr2=0x7efe100ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:02.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.646+0000 7efe187ce700 1 -- 172.21.15.92:0/1982261311 shutdown_connections 2023-10-06T10:46:02.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.646+0000 7efe187ce700 1 -- 172.21.15.92:0/1982261311 wait complete. 2023-10-06T10:46:02.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.647+0000 7efe187ce700 1 Processor -- start 2023-10-06T10:46:02.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.647+0000 7efe187ce700 1 -- start start 2023-10-06T10:46:02.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.648+0000 7efe187ce700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe100ceec0 0x7efe100af030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.648+0000 7efe187ce700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe1015a870 0x7efe100af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.648+0000 7efe187ce700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe1015ac40 0x7efe100b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.648+0000 7efe187ce700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1016b380 con 0x7efe1015ac40 2023-10-06T10:46:02.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.648+0000 7efe187ce700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe100b1350 con 0x7efe100ceec0 2023-10-06T10:46:02.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.648+0000 7efe187ce700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe10168310 con 0x7efe1015a870 2023-10-06T10:46:02.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.648+0000 7efe16d6b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe1015ac40 0x7efe100b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.648+0000 7efe16d6b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe1015ac40 0x7efe100b23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38192/0 (socket says 172.21.15.92:38192) 2023-10-06T10:46:02.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.648+0000 7efe16d6b700 1 -- 172.21.15.92:0/2183670339 learned_addr learned my addr 172.21.15.92:0/2183670339 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:46:02.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.648+0000 7efe1656a700 1 --2- 172.21.15.92:0/2183670339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe100ceec0 0x7efe100af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.648+0000 7efe15d69700 1 --2- 172.21.15.92:0/2183670339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe1015a870 0x7efe100af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.648+0000 7efe16d6b700 1 -- 172.21.15.92:0/2183670339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe1015a870 msgr2=0x7efe100af680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.648+0000 7efe16d6b700 1 --2- 172.21.15.92:0/2183670339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe1015a870 0x7efe100af680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.648+0000 7efe16d6b700 1 -- 172.21.15.92:0/2183670339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe100ceec0 msgr2=0x7efe100af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.648+0000 7efe16d6b700 1 --2- 172.21.15.92:0/2183670339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe100ceec0 0x7efe100af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.648+0000 7efe16d6b700 1 -- 172.21.15.92:0/2183670339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0c0093c0 con 0x7efe1015ac40 2023-10-06T10:46:02.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.648+0000 7efe16d6b700 1 --2- 172.21.15.92:0/2183670339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe1015ac40 0x7efe100b23b0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7efe0800cc70 tx=0x7efe08001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:02.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.648+0000 7efe067fc700 1 -- 172.21.15.92:0/2183670339 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0800a830 con 0x7efe1015ac40 2023-10-06T10:46:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.648+0000 7efe067fc700 1 -- 172.21.15.92:0/2183670339 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0800a990 con 0x7efe1015ac40 2023-10-06T10:46:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.648+0000 7efe187ce700 1 -- 172.21.15.92:0/2183670339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe00009420 con 0x7efe1015ac40 2023-10-06T10:46:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.648+0000 7efe067fc700 1 -- 172.21.15.92:0/2183670339 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0800ad50 con 0x7efe1015ac40 2023-10-06T10:46:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.648+0000 7efe187ce700 1 -- 172.21.15.92:0/2183670339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe101688d0 con 0x7efe1015ac40 2023-10-06T10:46:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.649+0000 7efe187ce700 1 -- 172.21.15.92:0/2183670339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe10064a90 con 0x7efe1015ac40 2023-10-06T10:46:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.650+0000 7efe067fc700 1 -- 172.21.15.92:0/2183670339 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efe08007300 con 0x7efe1015ac40 2023-10-06T10:46:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.650+0000 7efe067fc700 1 --2- 172.21.15.92:0/2183670339 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efdf4066f30 0x7efdf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.650+0000 7efe1656a700 1 --2- 172.21.15.92:0/2183670339 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efdf4066f30 0x7efdf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.650+0000 7efe067fc700 1 -- 172.21.15.92:0/2183670339 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7efe080847f0 con 0x7efe1015ac40 2023-10-06T10:46:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.651+0000 7efe1656a700 1 --2- 172.21.15.92:0/2183670339 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efdf4066f30 0x7efdf40693e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009f30 tx=0x7efe0c009120 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.654+0000 7efe067fc700 1 -- 172.21.15.92:0/2183670339 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe08051a90 con 0x7efe1015ac40 2023-10-06T10:46:02.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.838+0000 7efe187ce700 1 -- 172.21.15.92:0/2183670339 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7efe100cd7d0 con 0x7efdf4066f30 2023-10-06T10:46:02.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.838+0000 7efe067fc700 1 -- 172.21.15.92:0/2183670339 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7efe100cd7d0 con 0x7efdf4066f30 2023-10-06T10:46:02.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.840+0000 7efe187ce700 1 -- 172.21.15.92:0/2183670339 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efdf4066f30 msgr2=0x7efdf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.840+0000 7efe187ce700 1 --2- 172.21.15.92:0/2183670339 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efdf4066f30 0x7efdf40693e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009f30 tx=0x7efe0c009120 comp rx=0 tx=0).stop 2023-10-06T10:46:02.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.840+0000 7efe187ce700 1 -- 172.21.15.92:0/2183670339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe1015ac40 msgr2=0x7efe100b23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.840+0000 7efe187ce700 1 --2- 172.21.15.92:0/2183670339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe1015ac40 0x7efe100b23b0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7efe0800cc70 tx=0x7efe08001a20 comp rx=0 tx=0).stop 2023-10-06T10:46:02.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.840+0000 7efe187ce700 1 -- 172.21.15.92:0/2183670339 shutdown_connections 2023-10-06T10:46:02.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.840+0000 7efe187ce700 1 --2- 172.21.15.92:0/2183670339 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7efdf4066f30 0x7efdf40693e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.840+0000 7efe187ce700 1 --2- 172.21.15.92:0/2183670339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe1015ac40 0x7efe100b23b0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.840+0000 7efe187ce700 1 --2- 172.21.15.92:0/2183670339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe100ceec0 0x7efe100af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.840+0000 7efe187ce700 1 --2- 172.21.15.92:0/2183670339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe1015a870 0x7efe100af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.840+0000 7efe187ce700 1 -- 172.21.15.92:0/2183670339 >> 172.21.15.92:0/2183670339 conn(0x7efe10067d90 msgr2=0x7efe100c96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:02.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.840+0000 7efe187ce700 1 -- 172.21.15.92:0/2183670339 shutdown_connections 2023-10-06T10:46:02.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.840+0000 7efe187ce700 1 -- 172.21.15.92:0/2183670339 wait complete. 2023-10-06T10:46:02.844 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 7 to deep-scrub 2023-10-06T10:46:02.854 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:46:02.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.967+0000 7fa2e264b700 1 Processor -- start 2023-10-06T10:46:02.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.967+0000 7fa2e264b700 1 -- start start 2023-10-06T10:46:02.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.968+0000 7fa2e264b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa2dc157040 0x7fa2dc0cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.968+0000 7fa2e264b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2dc15b660 0x7fa2dc0c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.968+0000 7fa2e264b700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2dc15b1d0 con 0x7fa2dc157410 2023-10-06T10:46:02.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.968+0000 7fa2e264b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2dc0ce4b0 con 0x7fa2dc157040 2023-10-06T10:46:02.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.968+0000 7fa2e264b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc0ce790 con 0x7fa2dc15b660 2023-10-06T10:46:02.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.968+0000 7fa2e1649700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa2dc157040 0x7fa2dc0cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.968+0000 7fa2e1649700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa2dc157040 0x7fa2dc0cde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38206/0 (socket says 172.21.15.92:38206) 2023-10-06T10:46:02.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.968+0000 7fa2e1649700 1 -- 172.21.15.92:0/2527609691 learned_addr learned my addr 172.21.15.92:0/2527609691 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:46:02.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.968+0000 7fa2e1e4a700 1 --2- 172.21.15.92:0/2527609691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2dc15b660 0x7fa2dc0c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.969+0000 7fa2e1649700 1 -- 172.21.15.92:0/2527609691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2dc15b660 msgr2=0x7fa2dc0c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:02.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.969+0000 7fa2e1649700 1 --2- 172.21.15.92:0/2527609691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2dc15b660 0x7fa2dc0c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.969+0000 7fa2e1649700 1 -- 172.21.15.92:0/2527609691 >> v1:172.21.15.124:6789/0 conn(0x7fa2dc157410 legacy=0x7fa2dc06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.969+0000 7fa2e1649700 1 -- 172.21.15.92:0/2527609691 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2dc15ae60 con 0x7fa2dc157040 2023-10-06T10:46:02.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.969+0000 7fa2e1e4a700 1 --2- 172.21.15.92:0/2527609691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2dc15b660 0x7fa2dc0c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T10:46:02.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.969+0000 7fa2e1649700 1 --2- 172.21.15.92:0/2527609691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2dc157040 0x7fa2dc0cde60 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009f20 tx=0x7fa2d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa60b6255d2daa4d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:02.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.969+0000 7fa2d3fff700 1 -- 172.21.15.92:0/2527609691 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d800e070 con 0x7fa2dc157040 2023-10-06T10:46:02.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.969+0000 7fa2d3fff700 1 -- 172.21.15.92:0/2527609691 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d800ba80 con 0x7fa2dc157040 2023-10-06T10:46:02.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.969+0000 7fa2d3fff700 1 -- 172.21.15.92:0/2527609691 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d80092e0 con 0x7fa2dc157040 2023-10-06T10:46:02.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.969+0000 7fa2e264b700 1 -- 172.21.15.92:0/2527609691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2dc157040 msgr2=0x7fa2dc0cde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.969+0000 7fa2e264b700 1 --2- 172.21.15.92:0/2527609691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2dc157040 0x7fa2dc0cde60 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009f20 tx=0x7fa2d8009840 comp rx=0 tx=0).stop 2023-10-06T10:46:02.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.970+0000 7fa2e264b700 1 -- 172.21.15.92:0/2527609691 shutdown_connections 2023-10-06T10:46:02.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.970+0000 7fa2e264b700 1 --2- 172.21.15.92:0/2527609691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2dc15b660 0x7fa2dc0c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.970+0000 7fa2e264b700 1 --2- 172.21.15.92:0/2527609691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2dc157040 0x7fa2dc0cde60 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.970+0000 7fa2e264b700 1 -- 172.21.15.92:0/2527609691 >> 172.21.15.92:0/2527609691 conn(0x7fa2dc06b500 msgr2=0x7fa2dc06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:02.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.970+0000 7fa2e264b700 1 -- 172.21.15.92:0/2527609691 shutdown_connections 2023-10-06T10:46:02.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.970+0000 7fa2e264b700 1 -- 172.21.15.92:0/2527609691 wait complete. 2023-10-06T10:46:02.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.971+0000 7fa2e264b700 1 Processor -- start 2023-10-06T10:46:02.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.971+0000 7fa2e264b700 1 -- start start 2023-10-06T10:46:02.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.972+0000 7fa2e264b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2dc157040 0x7fa2dc1480c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.972+0000 7fa2e264b700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2dc157410 0x7fa2dc135280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.972+0000 7fa2e264b700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2dc15b660 0x7fa2dc0b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.972+0000 7fa2e264b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc159b20 con 0x7fa2dc157040 2023-10-06T10:46:02.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.972+0000 7fa2e264b700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc148600 con 0x7fa2dc157410 2023-10-06T10:46:02.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.972+0000 7fa2e264b700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc158530 con 0x7fa2dc15b660 2023-10-06T10:46:02.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.972+0000 7fa2e1649700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2dc157040 0x7fa2dc1480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.972+0000 7fa2e1649700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2dc157040 0x7fa2dc1480c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38214/0 (socket says 172.21.15.92:38214) 2023-10-06T10:46:02.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.972+0000 7fa2e1649700 1 -- 172.21.15.92:0/1898546299 learned_addr learned my addr 172.21.15.92:0/1898546299 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:46:02.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.972+0000 7fa2e1e4a700 1 --2- 172.21.15.92:0/1898546299 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2dc15b660 0x7fa2dc0b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.972+0000 7fa2e0e48700 1 --2- 172.21.15.92:0/1898546299 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2dc157410 0x7fa2dc135280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.972+0000 7fa2e1649700 1 -- 172.21.15.92:0/1898546299 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2dc15b660 msgr2=0x7fa2dc0b24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.972+0000 7fa2e1649700 1 --2- 172.21.15.92:0/1898546299 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2dc15b660 0x7fa2dc0b24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.972+0000 7fa2e1649700 1 -- 172.21.15.92:0/1898546299 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2dc157410 msgr2=0x7fa2dc135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:02.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.972+0000 7fa2e1649700 1 --2- 172.21.15.92:0/1898546299 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2dc157410 0x7fa2dc135280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:02.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.972+0000 7fa2e1649700 1 -- 172.21.15.92:0/1898546299 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d8009ae0 con 0x7fa2dc157040 2023-10-06T10:46:02.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.972+0000 7fa2e1649700 1 --2- 172.21.15.92:0/1898546299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2dc157040 0x7fa2dc1480c0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800b6c0 tx=0x7fa2d8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:02.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.972+0000 7fa2d17fa700 1 -- 172.21.15.92:0/1898546299 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8008bb0 con 0x7fa2dc157040 2023-10-06T10:46:02.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.972+0000 7fa2d17fa700 1 -- 172.21.15.92:0/1898546299 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d8008d10 con 0x7fa2dc157040 2023-10-06T10:46:02.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.972+0000 7fa2d17fa700 1 -- 172.21.15.92:0/1898546299 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d80266b0 con 0x7fa2dc157040 2023-10-06T10:46:02.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.972+0000 7fa2e264b700 1 -- 172.21.15.92:0/1898546299 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2dc158810 con 0x7fa2dc157040 2023-10-06T10:46:02.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.972+0000 7fa2e264b700 1 -- 172.21.15.92:0/1898546299 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2dc1357c0 con 0x7fa2dc157040 2023-10-06T10:46:02.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.973+0000 7fa2e264b700 1 -- 172.21.15.92:0/1898546299 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2dc05f230 con 0x7fa2dc157040 2023-10-06T10:46:02.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.974+0000 7fa2d17fa700 1 -- 172.21.15.92:0/1898546299 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa2d8003040 con 0x7fa2dc157040 2023-10-06T10:46:02.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.974+0000 7fa2d17fa700 1 --2- 172.21.15.92:0/1898546299 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa2c0066f30 0x7fa2c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:02.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.974+0000 7fa2d17fa700 1 -- 172.21.15.92:0/1898546299 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7fa2d80864b0 con 0x7fa2dc157040 2023-10-06T10:46:02.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.974+0000 7fa2e0e48700 1 --2- 172.21.15.92:0/1898546299 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa2c0066f30 0x7fa2c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:02.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.974+0000 7fa2e0e48700 1 --2- 172.21.15.92:0/1898546299 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa2c0066f30 0x7fa2c00693e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc14b0f0 tx=0x7fa2cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:02.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:02.979+0000 7fa2d17fa700 1 -- 172.21.15.92:0/1898546299 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa2d8053780 con 0x7fa2dc157040 2023-10-06T10:46:03.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:03.156+0000 7fa2e264b700 1 -- 172.21.15.92:0/1898546299 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa2dc159630 con 0x7fa2c0066f30 2023-10-06T10:46:03.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:03.158+0000 7fa2d17fa700 1 -- 172.21.15.92:0/1898546299 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73153 (secure 0 0 0) 0x7fa2cc015430 con 0x7fa2c0066f30 2023-10-06T10:46:03.161 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:46:03.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:03.161+0000 7fa2e264b700 1 -- 172.21.15.92:0/1898546299 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa2c0066f30 msgr2=0x7fa2c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:03.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:03.161+0000 7fa2e264b700 1 --2- 172.21.15.92:0/1898546299 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa2c0066f30 0x7fa2c00693e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc14b0f0 tx=0x7fa2cc009040 comp rx=0 tx=0).stop 2023-10-06T10:46:03.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:03.161+0000 7fa2e264b700 1 -- 172.21.15.92:0/1898546299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2dc157040 msgr2=0x7fa2dc1480c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:03.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:03.161+0000 7fa2e264b700 1 --2- 172.21.15.92:0/1898546299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2dc157040 0x7fa2dc1480c0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800b6c0 tx=0x7fa2d8018000 comp rx=0 tx=0).stop 2023-10-06T10:46:03.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:03.161+0000 7fa2e264b700 1 -- 172.21.15.92:0/1898546299 shutdown_connections 2023-10-06T10:46:03.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:03.161+0000 7fa2e264b700 1 --2- 172.21.15.92:0/1898546299 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fa2c0066f30 0x7fa2c00693e0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:03.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:03.161+0000 7fa2e264b700 1 --2- 172.21.15.92:0/1898546299 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2dc15b660 0x7fa2dc0b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:03.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:03.161+0000 7fa2e264b700 1 --2- 172.21.15.92:0/1898546299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2dc157040 0x7fa2dc1480c0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:03.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:03.161+0000 7fa2e264b700 1 --2- 172.21.15.92:0/1898546299 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2dc157410 0x7fa2dc135280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:03.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:03.161+0000 7fa2e264b700 1 -- 172.21.15.92:0/1898546299 >> 172.21.15.92:0/1898546299 conn(0x7fa2dc06b500 msgr2=0x7fa2dc0cbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:03.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:03.161+0000 7fa2e264b700 1 -- 172.21.15.92:0/1898546299 shutdown_connections 2023-10-06T10:46:03.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:03.161+0000 7fa2e264b700 1 -- 172.21.15.92:0/1898546299 wait complete. 2023-10-06T10:46:03.164 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:46:03.175 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2023-10-06T10:46:02.184005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2398084,"kb_used_data":11588,"kb_used_omap":0,"kb_used_meta":2386432,"kb_avail":752576636,"statfs":{"total":773094113280,"available":770638475264,"internally_reserved":0,"allocated":11866112,"data_stored":6351343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2443706368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.582764"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T10:45:53.608005+0000","last_change":"2023-10-06T10:38:11.905147+0000","last_active":"2023-10-06T10:45:53.608005+0000","last_peered":"2023-10-06T10:45:53.608005+0000","last_clean":"2023-10-06T10:45:53.608005+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:45:53.608005+0000","last_undegraded":"2023-10-06T10:45:53.608005+0000","last_fullsized":"2023-10-06T10:45:53.608005+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":85,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T10:45:53.603568+0000","last_change":"2023-10-06T10:38:11.320765+0000","last_active":"2023-10-06T10:45:53.603568+0000","last_peered":"2023-10-06T10:45:53.603568+0000","last_clean":"2023-10-06T10:45:53.603568+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:45:53.603568+0000","last_undegraded":"2023-10-06T10:45:53.603568+0000","last_fullsized":"2023-10-06T10:45:53.603568+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":89,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T10:45:53.998430+0000","last_change":"2023-10-06T10:38:11.331084+0000","last_active":"2023-10-06T10:45:53.998430+0000","last_peered":"2023-10-06T10:45:53.998430+0000","last_clean":"2023-10-06T10:45:53.998430+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:45:53.998430+0000","last_undegraded":"2023-10-06T10:45:53.998430+0000","last_fullsized":"2023-10-06T10:45:53.998430+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":231,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T10:45:53.995327+0000","last_change":"2023-10-06T10:38:11.324413+0000","last_active":"2023-10-06T10:45:53.995327+0000","last_peered":"2023-10-06T10:45:53.995327+0000","last_clean":"2023-10-06T10:45:53.995327+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:45:53.995327+0000","last_undegraded":"2023-10-06T10:45:53.995327+0000","last_fullsized":"2023-10-06T10:45:53.995327+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:39.255782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":81,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T10:45:53.995633+0000","last_change":"2023-10-06T10:38:11.526096+0000","last_active":"2023-10-06T10:45:53.995633+0000","last_peered":"2023-10-06T10:45:53.995633+0000","last_clean":"2023-10-06T10:45:53.995633+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:45:53.995633+0000","last_undegraded":"2023-10-06T10:45:53.995633+0000","last_fullsized":"2023-10-06T10:45:53.995633+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:04.222600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":81,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T10:45:53.995426+0000","last_change":"2023-10-06T10:38:11.526055+0000","last_active":"2023-10-06T10:45:53.995426+0000","last_peered":"2023-10-06T10:45:53.995426+0000","last_clean":"2023-10-06T10:45:53.995426+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:45:53.995426+0000","last_undegraded":"2023-10-06T10:45:53.995426+0000","last_fullsized":"2023-10-06T10:45:53.995426+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:35.538015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T10:45:53.609331+0000","last_change":"2023-10-06T10:38:11.526057+0000","last_active":"2023-10-06T10:45:53.609331+0000","last_peered":"2023-10-06T10:45:53.609331+0000","last_clean":"2023-10-06T10:45:53.609331+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:45:53.609331+0000","last_undegraded":"2023-10-06T10:45:53.609331+0000","last_fullsized":"2023-10-06T10:45:53.609331+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T10:45:53.997531+0000","last_change":"2023-10-06T10:38:11.526051+0000","last_active":"2023-10-06T10:45:53.997531+0000","last_peered":"2023-10-06T10:45:53.997531+0000","last_clean":"2023-10-06T10:45:53.997531+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:45:53.997531+0000","last_undegraded":"2023-10-06T10:45:53.997531+0000","last_fullsized":"2023-10-06T10:45:53.997531+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:19.762229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T10:45:53.998067+0000","last_change":"2023-10-06T10:38:11.330798+0000","last_active":"2023-10-06T10:45:53.998067+0000","last_peered":"2023-10-06T10:45:53.998067+0000","last_clean":"2023-10-06T10:45:53.998067+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:45:53.998067+0000","last_undegraded":"2023-10-06T10:45:53.998067+0000","last_fullsized":"2023-10-06T10:45:53.998067+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:38:10.304423+0000","last_clean_scrub_stamp":"2023-10-06T10:38:10.304423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:48.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574947,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299172,"kb_used_data":1884,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072668,"statfs":{"total":96636764160,"available":96330412032,"internally_reserved":0,"allocated":1929216,"data_stored":1236863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52300000000000002,"15min":0.49399999999999999},"min":{"1min":0.434,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.78400000000000003,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53700000000000003,"15min":0.52100000000000002},"min":{"1min":0.38900000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66100000000000003,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":0.58999999999999997}]},{"osd":1,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.495,"15min":0.47099999999999997},"min":{"1min":0.39700000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.64700000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54700000000000004,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.748,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.55000000000000004}]},{"osd":2,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56999999999999995,"15min":0.52500000000000002},"min":{"1min":0.40999999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69899999999999995,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51800000000000002,"15min":0.501},"min":{"1min":0.35599999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.67600000000000005,"5min":0.747,"15min":0.79800000000000004},"last":0.69499999999999995}]},{"osd":3,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56200000000000006,"15min":0.52200000000000002},"min":{"1min":0.36699999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76200000000000001,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.42699999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.755}]},{"osd":4,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.44,"15min":0.46300000000000002},"min":{"1min":0.30599999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70099999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.48999999999999999,"15min":0.501},"min":{"1min":0.311,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.75700000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.36799999999999999}]},{"osd":5,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.48399999999999999,"15min":0.48499999999999999},"min":{"1min":0.36099999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.77700000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.34000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.746,"5min":1.577,"15min":1.577},"last":0.39600000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.55200000000000005,"15min":0.56699999999999995},"min":{"1min":0.438,"5min":0.308,"15min":0.308},"max":{"1min":0.77300000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.57899999999999996,"15min":0.57799999999999996},"min":{"1min":0.37,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80700000000000005,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.41199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298588,"kb_used_data":1300,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073252,"statfs":{"total":96636764160,"available":96331010048,"internally_reserved":0,"allocated":1331200,"data_stored":645142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51200000000000001,"15min":0.497},"min":{"1min":0.38100000000000001,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.77000000000000002},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51600000000000001,"15min":0.46100000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.70099999999999996,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.61699999999999999}]},{"osd":1,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51400000000000001,"15min":0.46500000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.28000000000000003},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.80500000000000005},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.495},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.751,"15min":0.751},"last":0.58399999999999996}]},{"osd":2,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51800000000000002,"15min":0.48699999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54200000000000004,"15min":0.49199999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.73899999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.59699999999999998}]},{"osd":3,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55200000000000005,"15min":0.51100000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.76900000000000002,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55000000000000004,"15min":0.52400000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79400000000000004},"last":0.50800000000000001}]},{"osd":4,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.5,"15min":0.48999999999999999},"min":{"1min":0.32300000000000001,"5min":0.308,"15min":0.26700000000000002},"max":{"1min":0.72499999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.48899999999999999}]},{"osd":5,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51100000000000001,"15min":0.495},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.75700000000000001,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55400000000000005,"15min":0.54300000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.308},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.46000000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53200000000000003,"15min":0.52100000000000002},"min":{"1min":0.34899999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.755,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57899999999999996,"15min":0.55200000000000005},"min":{"1min":0.38200000000000001,"5min":0.33000000000000002,"15min":0.32300000000000001},"max":{"1min":0.79100000000000004,"5min":0.879,"15min":0.879},"last":0.52300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303268,"kb_used_data":1884,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068572,"statfs":{"total":96636764160,"available":96326217728,"internally_reserved":0,"allocated":1929216,"data_stored":1236863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32500000000000001,"15min":0.32700000000000001},"min":{"1min":0.217,"5min":0.188,"15min":0.188},"max":{"1min":0.42599999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.246,"5min":0.189,"15min":0.189},"max":{"1min":0.53200000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.28299999999999997}]},{"osd":2,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.32700000000000001,"15min":0.32200000000000001},"min":{"1min":0.22900000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.48099999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.373,"15min":0.36699999999999999},"min":{"1min":0.28599999999999998,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.54000000000000004,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.32000000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39200000000000002,"15min":0.38200000000000001},"min":{"1min":0.30499999999999999,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.54800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.38,"15min":0.377},"min":{"1min":0.26100000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.45800000000000002,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.36399999999999999}]},{"osd":4,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38200000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.746,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.498},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.45300000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.753,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.48899999999999999}]},{"osd":5,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.77500000000000002,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56999999999999995,"15min":0.55200000000000005},"min":{"1min":0.41099999999999998,"5min":0.36799999999999999,"15min":0.35699999999999998},"max":{"1min":0.76800000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.45200000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.39100000000000001,"5min":0.38700000000000001,"15min":0.36699999999999999},"max":{"1min":0.81299999999999994,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.46500000000000002,"5min":0.40000000000000002,"15min":0.39800000000000002},"max":{"1min":0.74399999999999999,"5min":0.93500000000000005,"15min":0.94099999999999995},"last":0.42499999999999999}]},{"osd":7,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.80200000000000005,"5min":0.996,"15min":0.996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.41299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.73399999999999999,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.435}]}]},{"osd":0,"up_from":13,"seq":55834574947,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298592,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073248,"statfs":{"total":96636764160,"available":96331005952,"internally_reserved":0,"allocated":1335296,"data_stored":646495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.317,"15min":0.31900000000000001},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.41399999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.37,"15min":0.35199999999999998},"min":{"1min":0.27000000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.57299999999999995,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.41999999999999998}]},{"osd":2,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.314,"15min":0.309},"min":{"1min":0.22600000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.46999999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34799999999999998,"15min":0.33400000000000002},"min":{"1min":0.247,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.61399999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.48499999999999999}]},{"osd":3,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34399999999999997,"15min":0.34499999999999997},"min":{"1min":0.24299999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.51400000000000001,"5min":0.622,"15min":0.622},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36099999999999999,"15min":0.35399999999999998},"min":{"1min":0.28399999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54200000000000004,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.437}]},{"osd":4,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54500000000000004,"15min":0.55600000000000005},"min":{"1min":0.435,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.74099999999999999,"5min":0.78900000000000003,"15min":0.83299999999999996},"last":0.499},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.54100000000000004},"min":{"1min":0.42699999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69699999999999995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.54200000000000004}]},{"osd":5,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.435,"5min":0.377,"15min":0.36899999999999999},"max":{"1min":0.76400000000000001,"5min":0.85699999999999998,"15min":1.0069999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70499999999999996,"5min":0.77500000000000002,"15min":0.94499999999999995},"last":0.52200000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56200000000000006,"15min":0.57899999999999996},"min":{"1min":0.40899999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.68600000000000005,"5min":1.046,"15min":1.046},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.68899999999999995,"5min":0.80000000000000004,"15min":1.075},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.40600000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73399999999999999,"5min":0.83499999999999996,"15min":0.96099999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.60399999999999998},"min":{"1min":0.46100000000000002,"5min":0.41899999999999998,"15min":0.41499999999999998},"max":{"1min":0.76200000000000001,"5min":0.77200000000000002,"15min":0.90100000000000002},"last":0.55100000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574945,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298592,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073248,"statfs":{"total":96636764160,"available":96331005952,"internally_reserved":0,"allocated":1335296,"data_stored":646495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.315,"15min":0.312},"min":{"1min":0.22600000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.45700000000000002,"5min":0.496,"15min":0.57999999999999996},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.35699999999999998,"15min":0.33100000000000002},"min":{"1min":0.26600000000000001,"5min":0.20200000000000001,"15min":0.20100000000000001},"max":{"1min":0.51700000000000002,"5min":0.55600000000000005,"15min":0.622},"last":0.34899999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33000000000000002},"min":{"1min":0.253,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.47299999999999998,"5min":0.56599999999999995,"15min":0.60499999999999998},"last":0.312},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.376,"15min":0.36899999999999999},"min":{"1min":0.29599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47999999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.39200000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.38700000000000001,"15min":0.35999999999999999},"min":{"1min":0.309,"5min":0.25700000000000001,"15min":0.20499999999999999},"max":{"1min":0.504,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.378,"15min":0.36299999999999999},"min":{"1min":0.27500000000000002,"5min":0.25,"15min":0.22800000000000001},"max":{"1min":0.50900000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.375}]},{"osd":4,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.40799999999999997,"5min":0.33400000000000002,"15min":0.315},"max":{"1min":0.66300000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.45700000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75,"5min":0.82399999999999995,"15min":0.84899999999999998},"last":0.503}]},{"osd":5,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.51000000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72199999999999998,"5min":0.83799999999999997,"15min":0.91200000000000003},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.41699999999999998,"5min":0.36599999999999999,"15min":0.33800000000000002},"max":{"1min":0.69499999999999995,"5min":0.81799999999999995,"15min":0.83399999999999996},"last":0.51600000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59299999999999997,"15min":0.61399999999999999},"min":{"1min":0.42999999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.70899999999999996,"5min":0.88700000000000001,"15min":0.89100000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61299999999999999,"15min":0.62},"min":{"1min":0.46800000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.748,"5min":0.80400000000000005,"15min":0.93799999999999994},"last":0.44800000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57799999999999996,"15min":0.60299999999999998},"min":{"1min":0.433,"5min":0.38500000000000001,"15min":0.36799999999999999},"max":{"1min":0.64300000000000002,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60799999999999998,"15min":0.621},"min":{"1min":0.438,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.70999999999999996,"5min":0.81100000000000005,"15min":0.874},"last":0.48999999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574945,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298592,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073248,"statfs":{"total":96636764160,"available":96331005952,"internally_reserved":0,"allocated":1335296,"data_stored":646495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.31,"15min":0.40500000000000003},"min":{"1min":0.214,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.436,"5min":0.52700000000000002,"15min":3.3210000000000002},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.33000000000000002,"15min":0.35399999999999998},"min":{"1min":0.22500000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":1.218},"last":0.36899999999999999}]},{"osd":1,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.33300000000000002,"15min":0.33800000000000002},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.20100000000000001},"max":{"1min":0.496,"5min":0.63600000000000001,"15min":0.86299999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35199999999999998,"15min":0.44500000000000001},"min":{"1min":0.26900000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.47299999999999998,"5min":0.66000000000000003,"15min":3.359},"last":0.38400000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.35299999999999998,"15min":0.374},"min":{"1min":0.28100000000000003,"5min":0.255,"15min":0.215},"max":{"1min":0.51700000000000002,"5min":0.71099999999999997,"15min":1.1100000000000001},"last":0.317},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.56699999999999995},"min":{"1min":0.28100000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.54200000000000004,"5min":0.59899999999999998,"15min":7.069},"last":0.39900000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54800000000000004,"15min":0.57799999999999996},"min":{"1min":0.43099999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73799999999999999,"5min":0.84699999999999998,"15min":1.165},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57799999999999996,"15min":0.59099999999999997},"min":{"1min":0.441,"5min":0.378,"15min":0.375},"max":{"1min":0.73599999999999999,"5min":1.02,"15min":1.02},"last":0.47399999999999998}]},{"osd":5,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.66500000000000004},"min":{"1min":0.41699999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72699999999999998,"5min":0.84499999999999997,"15min":3.4140000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57199999999999995,"15min":0.59799999999999998},"min":{"1min":0.39800000000000002,"5min":0.39200000000000002,"15min":0.34599999999999997},"max":{"1min":0.71299999999999997,"5min":0.83299999999999996,"15min":1.1930000000000001},"last":0.503}]},{"osd":6,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59599999999999997,"15min":0.61099999999999999},"min":{"1min":0.41899999999999998,"5min":0.41699999999999998,"15min":0.378},"max":{"1min":0.79900000000000004,"5min":1.054,"15min":1.054},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60899999999999999,"15min":0.69499999999999995},"min":{"1min":0.45100000000000001,"5min":0.42099999999999999,"15min":0.41999999999999998},"max":{"1min":0.754,"5min":0.876,"15min":3.3879999999999999},"last":0.52400000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58299999999999996,"15min":0.60199999999999998},"min":{"1min":0.42999999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.70499999999999996,"5min":0.86699999999999999,"15min":1.137},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59199999999999997,"15min":0.60799999999999998},"min":{"1min":0.42999999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":0.97899999999999998,"15min":1.014},"last":0.441}]}]},{"osd":4,"up_from":13,"seq":55834574945,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302688,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069152,"statfs":{"total":96636764160,"available":96326811648,"internally_reserved":0,"allocated":1335296,"data_stored":646495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.30299999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.625,"5min":0.74299999999999999,"15min":0.76800000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.51900000000000002,"15min":0.504},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.28299999999999997},"max":{"1min":0.67100000000000004,"5min":0.73099999999999998,"15min":0.77400000000000002},"last":0.64700000000000002}]},{"osd":1,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.41999999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67600000000000005,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51800000000000002,"15min":0.53100000000000003},"min":{"1min":0.29499999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69499999999999995,"5min":0.71799999999999997,"15min":0.82899999999999996},"last":0.56699999999999995}]},{"osd":2,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52600000000000002,"15min":0.51400000000000001},"min":{"1min":0.38100000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67800000000000005,"5min":0.77500000000000002,"15min":0.81000000000000005},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.52800000000000002,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.372,"15min":0.315},"max":{"1min":0.60799999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.46200000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55100000000000005,"15min":0.52600000000000002},"min":{"1min":0.41399999999999998,"5min":0.36599999999999999,"15min":0.35499999999999998},"max":{"1min":0.72799999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54700000000000004,"15min":0.53800000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.751,"5min":0.78800000000000003,"15min":0.79500000000000004},"last":0.63300000000000001}]},{"osd":5,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.44800000000000001,"15min":0.48599999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72399999999999998,"5min":0.77800000000000002,"15min":0.86899999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48199999999999998,"15min":0.49399999999999999},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.255},"max":{"1min":0.70699999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.38400000000000001}]},{"osd":6,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51000000000000001,"15min":0.53200000000000003},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.312},"max":{"1min":0.78300000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.372},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72399999999999998,"5min":0.88,"15min":0.91200000000000003},"last":0.50600000000000001}]},{"osd":7,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.33200000000000002,"15min":0.28999999999999998},"max":{"1min":0.74199999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.436},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.58099999999999996},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79900000000000004,"5min":0.81299999999999994,"15min":0.85699999999999998},"last":0.55200000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574945,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298592,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073248,"statfs":{"total":96636764160,"available":96331005952,"internally_reserved":0,"allocated":1335296,"data_stored":646495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.50700000000000001,"15min":0.47999999999999998},"min":{"1min":0.34999999999999998,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.67500000000000004,"5min":0.88,"15min":0.88},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.503,"15min":0.52500000000000002},"min":{"1min":0.39300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.73699999999999999,"5min":0.86499999999999999,"15min":1.2549999999999999},"last":0.53100000000000003}]},{"osd":1,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.495,"15min":0.48999999999999999},"min":{"1min":0.36899999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.81599999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51400000000000001,"15min":0.51700000000000002},"min":{"1min":0.38500000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.628,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.626}]},{"osd":2,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52700000000000002,"15min":0.51000000000000001},"min":{"1min":0.33000000000000002,"5min":0.307,"15min":0.30399999999999999},"max":{"1min":0.76500000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54000000000000004,"15min":0.55800000000000005},"min":{"1min":0.47599999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77700000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.58299999999999996}]},{"osd":3,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.56599999999999995},"min":{"1min":0.442,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69399999999999995,"5min":0.79900000000000004,"15min":0.95099999999999996},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51500000000000001,"15min":0.54700000000000004},"min":{"1min":0.42399999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70499999999999996,"5min":0.79100000000000004,"15min":1.3999999999999999},"last":0.54700000000000004}]},{"osd":4,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.28899999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.504,"15min":0.50700000000000001},"min":{"1min":0.33500000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.80700000000000005},"last":0.375}]},{"osd":6,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.498,"15min":0.51500000000000001},"min":{"1min":0.314,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.85799999999999998},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.437,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.47999999999999998}]},{"osd":7,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55600000000000005,"15min":0.57299999999999995},"min":{"1min":0.41999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.83299999999999996},"last":0.438},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.34699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.46200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:46:03.179 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2023-10-06T10:38:10.304423+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=10, tm_min=38, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=10, tm_min=45, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T10:46:03.179 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2023-10-06T10:38:10.304423+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=10, tm_min=38, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=10, tm_min=45, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T10:46:03.179 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2023-10-06T10:38:10.304423+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=10, tm_min=38, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=10, tm_min=45, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T10:46:03.180 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-06T10:38:10.304423+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=10, tm_min=38, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=10, tm_min=45, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T10:46:03.180 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2023-10-06T10:38:10.304423+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=10, tm_min=38, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=10, tm_min=45, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T10:46:03.180 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-06T10:38:10.304423+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=10, tm_min=38, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=10, tm_min=45, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T10:46:03.180 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2023-10-06T10:38:10.304423+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=10, tm_min=38, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=10, tm_min=45, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T10:46:03.180 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2023-10-06T10:38:10.304423+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=10, tm_min=38, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=10, tm_min=45, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T10:46:03.180 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2023-10-06T10:38:10.304423+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=10, tm_min=38, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=10, tm_min=45, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T10:46:03.180 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T10:46:23.182 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T10:46:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.306+0000 7f445359e700 1 Processor -- start 2023-10-06T10:46:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.306+0000 7f445359e700 1 -- start start 2023-10-06T10:46:23.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.307+0000 7f445359e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f44540cbc30 0x7f44540cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:23.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.307+0000 7f445359e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44540b2100 0x7f44540b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:23.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.307+0000 7f445359e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44540cb7a0 con 0x7f4454159440 2023-10-06T10:46:23.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.307+0000 7f445359e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44540b0860 con 0x7f44540cbc30 2023-10-06T10:46:23.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.307+0000 7f445359e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44540cb150 con 0x7f44540b2100 2023-10-06T10:46:23.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.307+0000 7f4449d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f44540cbc30 0x7f44540cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:23.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.307+0000 7f445259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4454159440 0x7f44540b3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59226/0 (socket says 172.21.15.92:59226) 2023-10-06T10:46:23.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.307+0000 7f4449d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f44540cbc30 0x7f44540cc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34132/0 (socket says 172.21.15.92:34132) 2023-10-06T10:46:23.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.307+0000 7f445259c700 1 -- 172.21.15.92:0/84811286 learned_addr learned my addr 172.21.15.92:0/84811286 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:46:23.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.307+0000 7f4452d9d700 1 --2- 172.21.15.92:0/84811286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44540b2100 0x7f44540b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:23.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.307+0000 7f4449d9b700 1 -- 172.21.15.92:0/84811286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44540b2100 msgr2=0x7f44540b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:23.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.307+0000 7f4449d9b700 1 --2- 172.21.15.92:0/84811286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44540b2100 0x7f44540b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:23.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.307+0000 7f4449d9b700 1 -- 172.21.15.92:0/84811286 >> v1:172.21.15.92:6789/0 conn(0x7f4454159440 legacy=0x7f44540b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:23.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.307+0000 7f4449d9b700 1 -- 172.21.15.92:0/84811286 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44540cb430 con 0x7f44540cbc30 2023-10-06T10:46:23.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.307+0000 7f4451d9b700 1 -- 172.21.15.92:0/84811286 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2123080452 0 0) 0x7f44540cb7a0 con 0x7f4454159440 2023-10-06T10:46:23.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.307+0000 7f4449d9b700 1 --2- 172.21.15.92:0/84811286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44540cbc30 0x7f44540cc080 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f443c0098e0 tx=0x7f443c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a35893338b0c8fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:23.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.308+0000 7f4451d9b700 1 -- 172.21.15.92:0/84811286 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c009e80 con 0x7f44540cbc30 2023-10-06T10:46:23.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.308+0000 7f4451d9b700 1 -- 172.21.15.92:0/84811286 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443c00b860 con 0x7f44540cbc30 2023-10-06T10:46:23.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.308+0000 7f4451d9b700 1 -- 172.21.15.92:0/84811286 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c00bbe0 con 0x7f44540cbc30 2023-10-06T10:46:23.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.308+0000 7f445359e700 1 -- 172.21.15.92:0/84811286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44540cbc30 msgr2=0x7f44540cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:23.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.308+0000 7f445359e700 1 --2- 172.21.15.92:0/84811286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44540cbc30 0x7f44540cc080 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f443c0098e0 tx=0x7f443c0091a0 comp rx=0 tx=0).stop 2023-10-06T10:46:23.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.308+0000 7f445359e700 1 -- 172.21.15.92:0/84811286 shutdown_connections 2023-10-06T10:46:23.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.308+0000 7f445359e700 1 --2- 172.21.15.92:0/84811286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44540b2100 0x7f44540b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:23.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.308+0000 7f445359e700 1 --2- 172.21.15.92:0/84811286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44540cbc30 0x7f44540cc080 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:23.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.308+0000 7f445359e700 1 -- 172.21.15.92:0/84811286 >> 172.21.15.92:0/84811286 conn(0x7f445406ece0 msgr2=0x7f445406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:23.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.308+0000 7f445359e700 1 -- 172.21.15.92:0/84811286 shutdown_connections 2023-10-06T10:46:23.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.308+0000 7f445359e700 1 -- 172.21.15.92:0/84811286 wait complete. 2023-10-06T10:46:23.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.309+0000 7f445359e700 1 Processor -- start 2023-10-06T10:46:23.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.309+0000 7f445359e700 1 -- start start 2023-10-06T10:46:23.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.310+0000 7f445359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44540b2100 0x7f44540ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:23.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.310+0000 7f445359e700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44540cbc30 0x7f445415ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:23.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.310+0000 7f445359e700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4454159440 0x7f4454155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:23.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.310+0000 7f445359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445415efd0 con 0x7f44540b2100 2023-10-06T10:46:23.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.310+0000 7f445359e700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4454155cd0 con 0x7f4454159440 2023-10-06T10:46:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.310+0000 7f445359e700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44540cf200 con 0x7f44540cbc30 2023-10-06T10:46:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.310+0000 7f445259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44540b2100 0x7f44540ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.310+0000 7f445259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44540b2100 0x7f44540ca190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34138/0 (socket says 172.21.15.92:34138) 2023-10-06T10:46:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.310+0000 7f445259c700 1 -- 172.21.15.92:0/2892638924 learned_addr learned my addr 172.21.15.92:0/2892638924 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:46:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.310+0000 7f4449d9b700 1 --2- 172.21.15.92:0/2892638924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44540cbc30 0x7f445415ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.310+0000 7f4452d9d700 1 --2- 172.21.15.92:0/2892638924 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4454159440 0x7f4454155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.311+0000 7f445259c700 1 -- 172.21.15.92:0/2892638924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44540cbc30 msgr2=0x7f445415ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.311+0000 7f445259c700 1 --2- 172.21.15.92:0/2892638924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44540cbc30 0x7f445415ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.311+0000 7f445259c700 1 -- 172.21.15.92:0/2892638924 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4454159440 msgr2=0x7f4454155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.311+0000 7f445259c700 1 --2- 172.21.15.92:0/2892638924 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4454159440 0x7f4454155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.311+0000 7f445259c700 1 -- 172.21.15.92:0/2892638924 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443c009420 con 0x7f44540b2100 2023-10-06T10:46:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.311+0000 7f445259c700 1 --2- 172.21.15.92:0/2892638924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44540b2100 0x7f44540ca190 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f444400c5e0 tx=0x7f4444001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.311+0000 7f444affd700 1 -- 172.21.15.92:0/2892638924 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4444005b30 con 0x7f44540b2100 2023-10-06T10:46:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.311+0000 7f444affd700 1 -- 172.21.15.92:0/2892638924 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4444005c90 con 0x7f44540b2100 2023-10-06T10:46:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.311+0000 7f445359e700 1 -- 172.21.15.92:0/2892638924 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445415f1e0 con 0x7f44540b2100 2023-10-06T10:46:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.311+0000 7f444affd700 1 -- 172.21.15.92:0/2892638924 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44440072e0 con 0x7f44540b2100 2023-10-06T10:46:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.311+0000 7f445359e700 1 -- 172.21.15.92:0/2892638924 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445415f7a0 con 0x7f44540b2100 2023-10-06T10:46:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.312+0000 7f445359e700 1 -- 172.21.15.92:0/2892638924 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445405f230 con 0x7f44540b2100 2023-10-06T10:46:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.312+0000 7f444affd700 1 -- 172.21.15.92:0/2892638924 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f444401b3b0 con 0x7f44540b2100 2023-10-06T10:46:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.313+0000 7f444affd700 1 --2- 172.21.15.92:0/2892638924 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4434066f30 0x7f44340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.313+0000 7f444affd700 1 -- 172.21.15.92:0/2892638924 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f44440839e0 con 0x7f44540b2100 2023-10-06T10:46:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.313+0000 7f4449d9b700 1 --2- 172.21.15.92:0/2892638924 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4434066f30 0x7f44340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.313+0000 7f4449d9b700 1 --2- 172.21.15.92:0/2892638924 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4434066f30 0x7f44340693e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f443c0091e0 tx=0x7f443c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.317+0000 7f444affd700 1 -- 172.21.15.92:0/2892638924 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f444404b5a0 con 0x7f44540b2100 2023-10-06T10:46:23.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.496+0000 7f445359e700 1 -- 172.21.15.92:0/2892638924 --> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f44540cf3c0 con 0x7f4434066f30 2023-10-06T10:46:23.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.499+0000 7f444affd700 1 -- 172.21.15.92:0/2892638924 <== mgr.4107 v2:172.21.15.92:6832/90373 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73207 (secure 0 0 0) 0x7f44540cf3c0 con 0x7f4434066f30 2023-10-06T10:46:23.502 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:46:23.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.501+0000 7f445359e700 1 -- 172.21.15.92:0/2892638924 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4434066f30 msgr2=0x7f44340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:23.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.501+0000 7f445359e700 1 --2- 172.21.15.92:0/2892638924 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4434066f30 0x7f44340693e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f443c0091e0 tx=0x7f443c017000 comp rx=0 tx=0).stop 2023-10-06T10:46:23.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.501+0000 7f445359e700 1 -- 172.21.15.92:0/2892638924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44540b2100 msgr2=0x7f44540ca190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.501+0000 7f445359e700 1 --2- 172.21.15.92:0/2892638924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44540b2100 0x7f44540ca190 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f444400c5e0 tx=0x7f4444001070 comp rx=0 tx=0).stop 2023-10-06T10:46:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.501+0000 7f445359e700 1 -- 172.21.15.92:0/2892638924 shutdown_connections 2023-10-06T10:46:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.501+0000 7f445359e700 1 --2- 172.21.15.92:0/2892638924 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f4434066f30 0x7f44340693e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.501+0000 7f445359e700 1 --2- 172.21.15.92:0/2892638924 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4454159440 0x7f4454155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.501+0000 7f445359e700 1 --2- 172.21.15.92:0/2892638924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44540b2100 0x7f44540ca190 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.501+0000 7f445359e700 1 --2- 172.21.15.92:0/2892638924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44540cbc30 0x7f445415ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.501+0000 7f445359e700 1 -- 172.21.15.92:0/2892638924 >> 172.21.15.92:0/2892638924 conn(0x7f445406ece0 msgr2=0x7f4454158370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.501+0000 7f445359e700 1 -- 172.21.15.92:0/2892638924 shutdown_connections 2023-10-06T10:46:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.501+0000 7f445359e700 1 -- 172.21.15.92:0/2892638924 wait complete. 2023-10-06T10:46:23.505 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T10:46:23.515 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2023-10-06T10:46:22.187798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2398084,"kb_used_data":11588,"kb_used_omap":0,"kb_used_meta":2386432,"kb_avail":752576636,"statfs":{"total":773094113280,"available":770638475264,"internally_reserved":0,"allocated":11866112,"data_stored":6351343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2443706368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002418"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T10:46:03.374819+0000","last_change":"2023-10-06T10:46:03.374784+0000","last_active":"2023-10-06T10:46:03.374819+0000","last_peered":"2023-10-06T10:46:03.374819+0000","last_clean":"2023-10-06T10:46:03.374819+0000","last_became_active":"2023-10-06T10:38:11.904838+0000","last_became_peered":"2023-10-06T10:38:11.904838+0000","last_unstale":"2023-10-06T10:46:03.374819+0000","last_undegraded":"2023-10-06T10:46:03.374819+0000","last_fullsized":"2023-10-06T10:46:03.374819+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:46:03.374681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:46:03.374681+0000","last_clean_scrub_stamp":"2023-10-06T10:46:03.374681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:21.074043+0000","scrub_duration":0.0025067710000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T10:46:01.578189+0000","last_change":"2023-10-06T10:46:01.578165+0000","last_active":"2023-10-06T10:46:01.578189+0000","last_peered":"2023-10-06T10:46:01.578189+0000","last_clean":"2023-10-06T10:46:01.578189+0000","last_became_active":"2023-10-06T10:38:11.320470+0000","last_became_peered":"2023-10-06T10:38:11.320470+0000","last_unstale":"2023-10-06T10:46:01.578189+0000","last_undegraded":"2023-10-06T10:46:01.578189+0000","last_fullsized":"2023-10-06T10:46:01.578189+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:46:01.578067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:46:01.578067+0000","last_clean_scrub_stamp":"2023-10-06T10:46:01.578067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:10:56.925414+0000","scrub_duration":0.0023407570000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":100,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T10:46:02.910223+0000","last_change":"2023-10-06T10:46:02.910189+0000","last_active":"2023-10-06T10:46:02.910223+0000","last_peered":"2023-10-06T10:46:02.910223+0000","last_clean":"2023-10-06T10:46:02.910223+0000","last_became_active":"2023-10-06T10:38:11.330820+0000","last_became_peered":"2023-10-06T10:38:11.330820+0000","last_unstale":"2023-10-06T10:46:02.910223+0000","last_undegraded":"2023-10-06T10:46:02.910223+0000","last_fullsized":"2023-10-06T10:46:02.910223+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:46:02.910055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:46:02.910055+0000","last_clean_scrub_stamp":"2023-10-06T10:46:02.910055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:14:09.336180+0000","scrub_duration":0.0021868289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"107'119","reported_seq":244,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T10:46:08.643650+0000","last_change":"2023-10-06T10:46:08.643605+0000","last_active":"2023-10-06T10:46:08.643650+0000","last_peered":"2023-10-06T10:46:08.643650+0000","last_clean":"2023-10-06T10:46:08.643650+0000","last_became_active":"2023-10-06T10:38:11.323569+0000","last_became_peered":"2023-10-06T10:38:11.323569+0000","last_unstale":"2023-10-06T10:46:08.643650+0000","last_undegraded":"2023-10-06T10:46:08.643650+0000","last_fullsized":"2023-10-06T10:46:08.643650+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'119","last_scrub_stamp":"2023-10-06T10:46:08.643499+0000","last_deep_scrub":"107'119","last_deep_scrub_stamp":"2023-10-06T10:46:08.643499+0000","last_clean_scrub_stamp":"2023-10-06T10:46:08.643499+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:28.863602+0000","scrub_duration":0.0052765149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":92,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T10:46:07.611612+0000","last_change":"2023-10-06T10:46:07.611587+0000","last_active":"2023-10-06T10:46:07.611612+0000","last_peered":"2023-10-06T10:46:07.611612+0000","last_clean":"2023-10-06T10:46:07.611612+0000","last_became_active":"2023-10-06T10:38:11.525833+0000","last_became_peered":"2023-10-06T10:38:11.525833+0000","last_unstale":"2023-10-06T10:46:07.611612+0000","last_undegraded":"2023-10-06T10:46:07.611612+0000","last_fullsized":"2023-10-06T10:46:07.611612+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:46:07.611469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:46:07.611469+0000","last_clean_scrub_stamp":"2023-10-06T10:46:07.611469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:47.255365+0000","scrub_duration":0.0023834329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T10:46:05.558947+0000","last_change":"2023-10-06T10:46:05.558903+0000","last_active":"2023-10-06T10:46:05.558947+0000","last_peered":"2023-10-06T10:46:05.558947+0000","last_clean":"2023-10-06T10:46:05.558947+0000","last_became_active":"2023-10-06T10:38:11.525786+0000","last_became_peered":"2023-10-06T10:38:11.525786+0000","last_unstale":"2023-10-06T10:46:05.558947+0000","last_undegraded":"2023-10-06T10:46:05.558947+0000","last_fullsized":"2023-10-06T10:46:05.558947+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:46:05.558801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:46:05.558801+0000","last_clean_scrub_stamp":"2023-10-06T10:46:05.558801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:15:38.776593+0000","scrub_duration":0.002412721,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":111,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T10:46:04.476394+0000","last_change":"2023-10-06T10:46:04.476366+0000","last_active":"2023-10-06T10:46:04.476394+0000","last_peered":"2023-10-06T10:46:04.476394+0000","last_clean":"2023-10-06T10:46:04.476394+0000","last_became_active":"2023-10-06T10:38:11.525806+0000","last_became_peered":"2023-10-06T10:38:11.525806+0000","last_unstale":"2023-10-06T10:46:04.476394+0000","last_undegraded":"2023-10-06T10:46:04.476394+0000","last_fullsized":"2023-10-06T10:46:04.476394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:46:04.476257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:46:04.476257+0000","last_clean_scrub_stamp":"2023-10-06T10:46:04.476257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:50:29.909019+0000","scrub_duration":0.0020530409999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T10:46:04.324197+0000","last_change":"2023-10-06T10:46:04.324171+0000","last_active":"2023-10-06T10:46:04.324197+0000","last_peered":"2023-10-06T10:46:04.324197+0000","last_clean":"2023-10-06T10:46:04.324197+0000","last_became_active":"2023-10-06T10:38:11.525820+0000","last_became_peered":"2023-10-06T10:38:11.525820+0000","last_unstale":"2023-10-06T10:46:04.324197+0000","last_undegraded":"2023-10-06T10:46:04.324197+0000","last_fullsized":"2023-10-06T10:46:04.324197+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:46:04.324074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:46:04.324074+0000","last_clean_scrub_stamp":"2023-10-06T10:46:04.324074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:15:37.541865+0000","scrub_duration":0.001901252,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-06T10:46:06.345418+0000","last_change":"2023-10-06T10:46:06.345388+0000","last_active":"2023-10-06T10:46:06.345418+0000","last_peered":"2023-10-06T10:46:06.345418+0000","last_clean":"2023-10-06T10:46:06.345418+0000","last_became_active":"2023-10-06T10:38:11.330521+0000","last_became_peered":"2023-10-06T10:38:11.330521+0000","last_unstale":"2023-10-06T10:46:06.345418+0000","last_undegraded":"2023-10-06T10:46:06.345418+0000","last_fullsized":"2023-10-06T10:46:06.345418+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T10:46:06.345287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T10:46:06.345287+0000","last_clean_scrub_stamp":"2023-10-06T10:46:06.345287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:45.989184+0000","scrub_duration":0.0021939239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574951,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299172,"kb_used_data":1884,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072668,"statfs":{"total":96636764160,"available":96330412032,"internally_reserved":0,"allocated":1929216,"data_stored":1236863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52300000000000002,"15min":0.49399999999999999},"min":{"1min":0.434,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.78400000000000003,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53700000000000003,"15min":0.52100000000000002},"min":{"1min":0.38900000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66100000000000003,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":0.53500000000000003}]},{"osd":1,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.495,"15min":0.47099999999999997},"min":{"1min":0.39700000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.64700000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54700000000000004,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.748,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.56999999999999995}]},{"osd":2,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56999999999999995,"15min":0.52500000000000002},"min":{"1min":0.40999999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69899999999999995,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51800000000000002,"15min":0.501},"min":{"1min":0.35599999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.67600000000000005,"5min":0.747,"15min":0.79800000000000004},"last":0.66400000000000003}]},{"osd":3,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56200000000000006,"15min":0.52200000000000002},"min":{"1min":0.36699999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76200000000000001,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.42699999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.44,"15min":0.46300000000000002},"min":{"1min":0.30599999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70099999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.48999999999999999,"15min":0.501},"min":{"1min":0.311,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.75700000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.47999999999999998}]},{"osd":5,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.48399999999999999,"15min":0.48499999999999999},"min":{"1min":0.36099999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.77700000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.627},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.34000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.746,"5min":1.577,"15min":1.577},"last":0.505}]},{"osd":6,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.55200000000000005,"15min":0.56699999999999995},"min":{"1min":0.438,"5min":0.308,"15min":0.308},"max":{"1min":0.77300000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.57899999999999996,"15min":0.57799999999999996},"min":{"1min":0.37,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80700000000000005,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.68799999999999994}]}]},{"osd":6,"up_from":13,"seq":55834574949,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298588,"kb_used_data":1300,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073252,"statfs":{"total":96636764160,"available":96331010048,"internally_reserved":0,"allocated":1331200,"data_stored":645142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51200000000000001,"15min":0.497},"min":{"1min":0.38100000000000001,"5min":0.29299999999999998,"15min":0.27000000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.77000000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51600000000000001,"15min":0.46100000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.70099999999999996,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.68000000000000005}]},{"osd":1,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51400000000000001,"15min":0.46500000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.28000000000000003},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.80500000000000005},"last":0.623},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.495},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.751,"15min":0.751},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51800000000000002,"15min":0.48699999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54200000000000004,"15min":0.49199999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.73899999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55200000000000005,"15min":0.51100000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.76900000000000002,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55000000000000004,"15min":0.52400000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79400000000000004},"last":0.69099999999999995}]},{"osd":4,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.502,"5min":0.5,"15min":0.48999999999999999},"min":{"1min":0.32300000000000001,"5min":0.308,"15min":0.26700000000000002},"max":{"1min":0.72499999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.39000000000000001}]},{"osd":5,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51100000000000001,"15min":0.495},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.75700000000000001,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55400000000000005,"15min":0.54300000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.308},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.48999999999999999}]},{"osd":7,"last update":"Fri Oct 6 10:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53200000000000003,"15min":0.52100000000000002},"min":{"1min":0.34899999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.755,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57899999999999996,"15min":0.55200000000000005},"min":{"1min":0.38200000000000001,"5min":0.33000000000000002,"15min":0.32300000000000001},"max":{"1min":0.79100000000000004,"5min":0.879,"15min":0.879},"last":0.52400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303268,"kb_used_data":1884,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068572,"statfs":{"total":96636764160,"available":96326217728,"internally_reserved":0,"allocated":1929216,"data_stored":1236863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32500000000000001,"15min":0.32700000000000001},"min":{"1min":0.217,"5min":0.188,"15min":0.188},"max":{"1min":0.42599999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.248},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.246,"5min":0.189,"15min":0.189},"max":{"1min":0.53200000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.316}]},{"osd":2,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.32700000000000001,"15min":0.32200000000000001},"min":{"1min":0.22900000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.48099999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.373,"15min":0.36699999999999999},"min":{"1min":0.28599999999999998,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.54000000000000004,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.33900000000000002}]},{"osd":3,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39200000000000002,"15min":0.38200000000000001},"min":{"1min":0.30499999999999999,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.54800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.38},{"interface":"front","average":{"1min":0.371,"5min":0.38,"15min":0.377},"min":{"1min":0.26100000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.45800000000000002,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.372}]},{"osd":4,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38200000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.746,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.45300000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.753,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.434}]},{"osd":5,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.77500000000000002,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56999999999999995,"15min":0.55200000000000005},"min":{"1min":0.41099999999999998,"5min":0.36799999999999999,"15min":0.35699999999999998},"max":{"1min":0.76800000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.47899999999999998}]},{"osd":6,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.39100000000000001,"5min":0.38700000000000001,"15min":0.36699999999999999},"max":{"1min":0.81299999999999994,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.46500000000000002,"5min":0.40000000000000002,"15min":0.39800000000000002},"max":{"1min":0.74399999999999999,"5min":0.93500000000000005,"15min":0.94099999999999995},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Oct 6 10:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.80200000000000005,"5min":0.996,"15min":0.996},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.41299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.73399999999999999,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.50800000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574951,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298592,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073248,"statfs":{"total":96636764160,"available":96331005952,"internally_reserved":0,"allocated":1335296,"data_stored":646495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.317,"15min":0.31900000000000001},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.41399999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.5},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.37,"15min":0.35199999999999998},"min":{"1min":0.27000000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.57299999999999995,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.58299999999999996}]},{"osd":2,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.314,"15min":0.309},"min":{"1min":0.22600000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.46999999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34799999999999998,"15min":0.33400000000000002},"min":{"1min":0.247,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.61399999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.53500000000000003}]},{"osd":3,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34399999999999997,"15min":0.34499999999999997},"min":{"1min":0.24299999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.51400000000000001,"5min":0.622,"15min":0.622},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36099999999999999,"15min":0.35399999999999998},"min":{"1min":0.28399999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54200000000000004,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54500000000000004,"15min":0.55600000000000005},"min":{"1min":0.435,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.74099999999999999,"5min":0.78900000000000003,"15min":0.83299999999999996},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.54100000000000004},"min":{"1min":0.42699999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69699999999999995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.65400000000000003}]},{"osd":5,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.435,"5min":0.377,"15min":0.36899999999999999},"max":{"1min":0.76400000000000001,"5min":0.85699999999999998,"15min":1.0069999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70499999999999996,"5min":0.77500000000000002,"15min":0.94499999999999995},"last":0.77600000000000002}]},{"osd":6,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56200000000000006,"15min":0.57899999999999996},"min":{"1min":0.40899999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.68600000000000005,"5min":1.046,"15min":1.046},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.68899999999999995,"5min":0.80000000000000004,"15min":1.075},"last":0.81100000000000005}]},{"osd":7,"last update":"Fri Oct 6 10:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.40600000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73399999999999999,"5min":0.83499999999999996,"15min":0.96099999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.60399999999999998},"min":{"1min":0.46100000000000002,"5min":0.41899999999999998,"15min":0.41499999999999998},"max":{"1min":0.76200000000000001,"5min":0.77200000000000002,"15min":0.90100000000000002},"last":0.83899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574949,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298592,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073248,"statfs":{"total":96636764160,"available":96331005952,"internally_reserved":0,"allocated":1335296,"data_stored":646495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.315,"15min":0.312},"min":{"1min":0.22600000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.45700000000000002,"5min":0.496,"15min":0.57999999999999996},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.35699999999999998,"15min":0.33100000000000002},"min":{"1min":0.26600000000000001,"5min":0.20200000000000001,"15min":0.20100000000000001},"max":{"1min":0.51700000000000002,"5min":0.55600000000000005,"15min":0.622},"last":0.34799999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33000000000000002},"min":{"1min":0.253,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.47299999999999998,"5min":0.56599999999999995,"15min":0.60499999999999998},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.376,"15min":0.36899999999999999},"min":{"1min":0.29599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47999999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.31900000000000001}]},{"osd":3,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.38700000000000001,"15min":0.35999999999999999},"min":{"1min":0.309,"5min":0.25700000000000001,"15min":0.20499999999999999},"max":{"1min":0.504,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.378,"15min":0.36299999999999999},"min":{"1min":0.27500000000000002,"5min":0.25,"15min":0.22800000000000001},"max":{"1min":0.50900000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.377}]},{"osd":4,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.40799999999999997,"5min":0.33400000000000002,"15min":0.315},"max":{"1min":0.66300000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.44},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.45700000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75,"5min":0.82399999999999995,"15min":0.84899999999999998},"last":0.54100000000000004}]},{"osd":5,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.51000000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72199999999999998,"5min":0.83799999999999997,"15min":0.91200000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.41699999999999998,"5min":0.36599999999999999,"15min":0.33800000000000002},"max":{"1min":0.69499999999999995,"5min":0.81799999999999995,"15min":0.83399999999999996},"last":0.496}]},{"osd":6,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59299999999999997,"15min":0.61399999999999999},"min":{"1min":0.42999999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.70899999999999996,"5min":0.88700000000000001,"15min":0.89100000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61299999999999999,"15min":0.62},"min":{"1min":0.46800000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.748,"5min":0.80400000000000005,"15min":0.93799999999999994},"last":0.57099999999999995}]},{"osd":7,"last update":"Fri Oct 6 10:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57799999999999996,"15min":0.60299999999999998},"min":{"1min":0.433,"5min":0.38500000000000001,"15min":0.36799999999999999},"max":{"1min":0.64300000000000002,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60799999999999998,"15min":0.621},"min":{"1min":0.438,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.70999999999999996,"5min":0.81100000000000005,"15min":0.874},"last":0.46500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574949,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298592,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073248,"statfs":{"total":96636764160,"available":96331005952,"internally_reserved":0,"allocated":1335296,"data_stored":646495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.31,"15min":0.40500000000000003},"min":{"1min":0.214,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.436,"5min":0.52700000000000002,"15min":3.3210000000000002},"last":0.377},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.33000000000000002,"15min":0.35399999999999998},"min":{"1min":0.22500000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":1.218},"last":0.307}]},{"osd":1,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.33300000000000002,"15min":0.33800000000000002},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.20100000000000001},"max":{"1min":0.496,"5min":0.63600000000000001,"15min":0.86299999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35199999999999998,"15min":0.44500000000000001},"min":{"1min":0.26900000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.47299999999999998,"5min":0.66000000000000003,"15min":3.359},"last":0.42999999999999999}]},{"osd":2,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.35299999999999998,"15min":0.374},"min":{"1min":0.28100000000000003,"5min":0.255,"15min":0.215},"max":{"1min":0.51700000000000002,"5min":0.71099999999999997,"15min":1.1100000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.56699999999999995},"min":{"1min":0.28100000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.54200000000000004,"5min":0.59899999999999998,"15min":7.069},"last":0.39500000000000002}]},{"osd":4,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54800000000000004,"15min":0.57799999999999996},"min":{"1min":0.43099999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73799999999999999,"5min":0.84699999999999998,"15min":1.165},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57799999999999996,"15min":0.59099999999999997},"min":{"1min":0.441,"5min":0.378,"15min":0.375},"max":{"1min":0.73599999999999999,"5min":1.02,"15min":1.02},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.66500000000000004},"min":{"1min":0.41699999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72699999999999998,"5min":0.84499999999999997,"15min":3.4140000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57199999999999995,"15min":0.59799999999999998},"min":{"1min":0.39800000000000002,"5min":0.39200000000000002,"15min":0.34599999999999997},"max":{"1min":0.71299999999999997,"5min":0.83299999999999996,"15min":1.1930000000000001},"last":0.57799999999999996}]},{"osd":6,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59599999999999997,"15min":0.61099999999999999},"min":{"1min":0.41899999999999998,"5min":0.41699999999999998,"15min":0.378},"max":{"1min":0.79900000000000004,"5min":1.054,"15min":1.054},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60899999999999999,"15min":0.69499999999999995},"min":{"1min":0.45100000000000001,"5min":0.42099999999999999,"15min":0.41999999999999998},"max":{"1min":0.754,"5min":0.876,"15min":3.3879999999999999},"last":0.67200000000000004}]},{"osd":7,"last update":"Fri Oct 6 10:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58299999999999996,"15min":0.60199999999999998},"min":{"1min":0.42999999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.70499999999999996,"5min":0.86699999999999999,"15min":1.137},"last":0.622},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59199999999999997,"15min":0.60799999999999998},"min":{"1min":0.42999999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":0.97899999999999998,"15min":1.014},"last":0.55700000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574949,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302688,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069152,"statfs":{"total":96636764160,"available":96326811648,"internally_reserved":0,"allocated":1335296,"data_stored":646495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.30299999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.625,"5min":0.74299999999999999,"15min":0.76800000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.51900000000000002,"15min":0.504},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.28299999999999997},"max":{"1min":0.67100000000000004,"5min":0.73099999999999998,"15min":0.77400000000000002},"last":0.60099999999999998}]},{"osd":1,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.41999999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67600000000000005,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51800000000000002,"15min":0.53100000000000003},"min":{"1min":0.29499999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69499999999999995,"5min":0.71799999999999997,"15min":0.82899999999999996},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52600000000000002,"15min":0.51400000000000001},"min":{"1min":0.38100000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67800000000000005,"5min":0.77500000000000002,"15min":0.81000000000000005},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.52800000000000002,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.372,"15min":0.315},"max":{"1min":0.60799999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.53400000000000003}]},{"osd":3,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55100000000000005,"15min":0.52600000000000002},"min":{"1min":0.41399999999999998,"5min":0.36599999999999999,"15min":0.35499999999999998},"max":{"1min":0.72799999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54700000000000004,"15min":0.53800000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.751,"5min":0.78800000000000003,"15min":0.79500000000000004},"last":0.58599999999999997}]},{"osd":5,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.44800000000000001,"15min":0.48599999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72399999999999998,"5min":0.77800000000000002,"15min":0.86899999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48199999999999998,"15min":0.49399999999999999},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.255},"max":{"1min":0.70699999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.625}]},{"osd":6,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51000000000000001,"15min":0.53200000000000003},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.312},"max":{"1min":0.78300000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72399999999999998,"5min":0.88,"15min":0.91200000000000003},"last":0.65200000000000002}]},{"osd":7,"last update":"Fri Oct 6 10:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.33200000000000002,"15min":0.28999999999999998},"max":{"1min":0.74199999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.58099999999999996},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79900000000000004,"5min":0.81299999999999994,"15min":0.85699999999999998},"last":0.69999999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574949,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298592,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073248,"statfs":{"total":96636764160,"available":96331005952,"internally_reserved":0,"allocated":1335296,"data_stored":646495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.50700000000000001,"15min":0.47999999999999998},"min":{"1min":0.34999999999999998,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.67500000000000004,"5min":0.88,"15min":0.88},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.503,"15min":0.52500000000000002},"min":{"1min":0.39300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.73699999999999999,"5min":0.86499999999999999,"15min":1.2549999999999999},"last":0.63900000000000001}]},{"osd":1,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.495,"15min":0.48999999999999999},"min":{"1min":0.36899999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.81599999999999995},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51400000000000001,"15min":0.51700000000000002},"min":{"1min":0.38500000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.628,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54300000000000004}]},{"osd":2,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52700000000000002,"15min":0.51000000000000001},"min":{"1min":0.33000000000000002,"5min":0.307,"15min":0.30399999999999999},"max":{"1min":0.76500000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54000000000000004,"15min":0.55800000000000005},"min":{"1min":0.47599999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77700000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.53300000000000003}]},{"osd":3,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.56599999999999995},"min":{"1min":0.442,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69399999999999995,"5min":0.79900000000000004,"15min":0.95099999999999996},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51500000000000001,"15min":0.54700000000000004},"min":{"1min":0.42399999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70499999999999996,"5min":0.79100000000000004,"15min":1.3999999999999999},"last":0.69899999999999995}]},{"osd":4,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.28899999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.379},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.504,"15min":0.50700000000000001},"min":{"1min":0.33500000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.80700000000000005},"last":0.41299999999999998}]},{"osd":6,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.498,"15min":0.51500000000000001},"min":{"1min":0.314,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.85799999999999998},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.437,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.43099999999999999}]},{"osd":7,"last update":"Fri Oct 6 10:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55600000000000005,"15min":0.57299999999999995},"min":{"1min":0.41999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.83299999999999996},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.34699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.39600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T10:46:23.517 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-06T10:46:23.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.629+0000 7fcc45a8c700 1 Processor -- start 2023-10-06T10:46:23.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.629+0000 7fcc45a8c700 1 -- start start 2023-10-06T10:46:23.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.630+0000 7fcc45a8c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc400b29f0 0x7fcc40164320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:23.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.630+0000 7fcc45a8c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc400cc970 con 0x7fcc400b2620 2023-10-06T10:46:23.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.630+0000 7fcc45a8c700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc400cbd60 con 0x7fcc400cd3c0 2023-10-06T10:46:23.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.630+0000 7fcc45a8c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc400cc040 con 0x7fcc400b29f0 2023-10-06T10:46:23.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.630+0000 7fcc44a8a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcc400b2620 0x7fcc400cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58360/0 (socket says 172.21.15.92:58360) 2023-10-06T10:46:23.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.630+0000 7fcc44a8a700 1 -- 172.21.15.92:0/1267809966 learned_addr learned my addr 172.21.15.92:0/1267809966 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:46:23.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.630+0000 7fcc4528b700 1 --2- 172.21.15.92:0/1267809966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc400b29f0 0x7fcc40164320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:23.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.630+0000 7fcc377fe700 1 -- 172.21.15.92:0/1267809966 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426431309 0 0) 0x7fcc400cc970 con 0x7fcc400b2620 2023-10-06T10:46:23.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.630+0000 7fcc377fe700 1 -- 172.21.15.92:0/1267809966 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc28003570 con 0x7fcc400b2620 2023-10-06T10:46:23.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc377fe700 1 -- 172.21.15.92:0/1267809966 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4165398539 0 0) 0x7fcc28003570 con 0x7fcc400b2620 2023-10-06T10:46:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc377fe700 1 -- 172.21.15.92:0/1267809966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc400b29f0 msgr2=0x7fcc40164320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc377fe700 1 --2- 172.21.15.92:0/1267809966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc400b29f0 0x7fcc40164320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc377fe700 1 -- 172.21.15.92:0/1267809966 >> v1:172.21.15.124:6789/0 conn(0x7fcc400cd3c0 legacy=0x7fcc400ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc377fe700 1 -- 172.21.15.92:0/1267809966 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc400cc320 con 0x7fcc400b2620 2023-10-06T10:46:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc377fe700 1 -- 172.21.15.92:0/1267809966 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7fcc3c0015f0 con 0x7fcc400b2620 2023-10-06T10:46:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc377fe700 1 -- 172.21.15.92:0/1267809966 >> v1:172.21.15.92:6789/0 conn(0x7fcc400b2620 legacy=0x7fcc400cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc377fe700 1 --2- 172.21.15.92:0/1267809966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc28003a90 0x7fcc28003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc377fe700 1 --2- 172.21.15.92:0/1267809966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc400b29f0 0x7fcc28004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc377fe700 1 --2- 172.21.15.92:0/1267809966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc28004d50 0x7fcc280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc377fe700 1 -- 172.21.15.92:0/1267809966 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc400cc040 con 0x7fcc28004d50 2023-10-06T10:46:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc377fe700 1 -- 172.21.15.92:0/1267809966 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2800bb70 con 0x7fcc28003a90 2023-10-06T10:46:23.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc377fe700 1 -- 172.21.15.92:0/1267809966 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2800bd30 con 0x7fcc400b29f0 2023-10-06T10:46:23.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc4528b700 1 --2- 172.21.15.92:0/1267809966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc28004d50 0x7fcc280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:23.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc44a8a700 1 --2- 172.21.15.92:0/1267809966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc28003a90 0x7fcc28003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:23.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc37fff700 1 --2- 172.21.15.92:0/1267809966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc400b29f0 0x7fcc28004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:23.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc4528b700 1 -- 172.21.15.92:0/1267809966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc400b29f0 msgr2=0x7fcc28004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:23.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc4528b700 1 --2- 172.21.15.92:0/1267809966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc400b29f0 0x7fcc28004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:23.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc4528b700 1 -- 172.21.15.92:0/1267809966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc28003a90 msgr2=0x7fcc28003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:23.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc4528b700 1 --2- 172.21.15.92:0/1267809966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc28003a90 0x7fcc28003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:23.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc4528b700 1 -- 172.21.15.92:0/1267809966 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc28003570 con 0x7fcc28004d50 2023-10-06T10:46:23.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc4528b700 1 --2- 172.21.15.92:0/1267809966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc28004d50 0x7fcc280051c0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fcc40089490 tx=0x7fcc3800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68434975120bce03 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:23.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc377fe700 1 -- 172.21.15.92:0/1267809966 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3800ea60 con 0x7fcc28004d50 2023-10-06T10:46:23.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc377fe700 1 -- 172.21.15.92:0/1267809966 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3800f040 con 0x7fcc28004d50 2023-10-06T10:46:23.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.631+0000 7fcc377fe700 1 -- 172.21.15.92:0/1267809966 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3800ee10 con 0x7fcc28004d50 2023-10-06T10:46:23.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.632+0000 7fcc45a8c700 1 -- 172.21.15.92:0/1267809966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc28004d50 msgr2=0x7fcc280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:23.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.632+0000 7fcc45a8c700 1 --2- 172.21.15.92:0/1267809966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc28004d50 0x7fcc280051c0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fcc40089490 tx=0x7fcc3800b040 comp rx=0 tx=0).stop 2023-10-06T10:46:23.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.632+0000 7fcc4528b700 1 -- 172.21.15.92:0/1267809966 reap_dead start 2023-10-06T10:46:23.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.632+0000 7fcc45a8c700 1 -- 172.21.15.92:0/1267809966 shutdown_connections 2023-10-06T10:46:23.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.632+0000 7fcc45a8c700 1 -- 172.21.15.92:0/1267809966 >> 172.21.15.92:0/1267809966 conn(0x7fcc40099520 msgr2=0x7fcc4006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:23.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.632+0000 7fcc45a8c700 1 -- 172.21.15.92:0/1267809966 shutdown_connections 2023-10-06T10:46:23.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.632+0000 7fcc45a8c700 1 -- 172.21.15.92:0/1267809966 wait complete. 2023-10-06T10:46:23.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.633+0000 7fcc45a8c700 1 Processor -- start 2023-10-06T10:46:23.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.633+0000 7fcc45a8c700 1 -- start start 2023-10-06T10:46:23.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.634+0000 7fcc45a8c700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc400c7380 0x7fcc40155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:23.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.634+0000 7fcc45a8c700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc400c7750 0x7fcc400b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:23.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.634+0000 7fcc45a8c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc400c92c0 0x7fcc400c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:23.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.634+0000 7fcc45a8c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc401511f0 con 0x7fcc400c92c0 2023-10-06T10:46:23.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.634+0000 7fcc45a8c700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc40156450 con 0x7fcc400c7380 2023-10-06T10:46:23.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.634+0000 7fcc45a8c700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc401558d0 con 0x7fcc400c7750 2023-10-06T10:46:23.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.634+0000 7fcc4528b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc400c92c0 0x7fcc400c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:23.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.634+0000 7fcc4528b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc400c92c0 0x7fcc400c9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45342/0 (socket says 172.21.15.92:45342) 2023-10-06T10:46:23.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.634+0000 7fcc4528b700 1 -- 172.21.15.92:0/564851788 learned_addr learned my addr 172.21.15.92:0/564851788 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:46:23.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.634+0000 7fcc44a8a700 1 --2- 172.21.15.92:0/564851788 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc400c7380 0x7fcc40155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:23.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.634+0000 7fcc37fff700 1 --2- 172.21.15.92:0/564851788 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc400c7750 0x7fcc400b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:23.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.634+0000 7fcc4528b700 1 -- 172.21.15.92:0/564851788 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc400c7750 msgr2=0x7fcc400b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:23.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.634+0000 7fcc4528b700 1 --2- 172.21.15.92:0/564851788 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc400c7750 0x7fcc400b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:23.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.634+0000 7fcc4528b700 1 -- 172.21.15.92:0/564851788 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc400c7380 msgr2=0x7fcc40155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:23.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.634+0000 7fcc4528b700 1 --2- 172.21.15.92:0/564851788 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc400c7380 0x7fcc40155200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:23.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.634+0000 7fcc4528b700 1 -- 172.21.15.92:0/564851788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc40155bb0 con 0x7fcc400c92c0 2023-10-06T10:46:23.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.635+0000 7fcc4528b700 1 --2- 172.21.15.92:0/564851788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc400c92c0 0x7fcc400c9730 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fcc38016000 tx=0x7fcc3801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:23.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.635+0000 7fcc34ff9700 1 -- 172.21.15.92:0/564851788 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3800eee0 con 0x7fcc400c92c0 2023-10-06T10:46:23.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.635+0000 7fcc34ff9700 1 -- 172.21.15.92:0/564851788 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc38022070 con 0x7fcc400c92c0 2023-10-06T10:46:23.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.635+0000 7fcc45a8c700 1 -- 172.21.15.92:0/564851788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc40155e90 con 0x7fcc400c92c0 2023-10-06T10:46:23.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.635+0000 7fcc34ff9700 1 -- 172.21.15.92:0/564851788 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc380276a0 con 0x7fcc400c92c0 2023-10-06T10:46:23.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.635+0000 7fcc45a8c700 1 -- 172.21.15.92:0/564851788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc400c7bc0 con 0x7fcc400c92c0 2023-10-06T10:46:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.636+0000 7fcc45a8c700 1 -- 172.21.15.92:0/564851788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4005f230 con 0x7fcc400c92c0 2023-10-06T10:46:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.636+0000 7fcc34ff9700 1 -- 172.21.15.92:0/564851788 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcc38024030 con 0x7fcc400c92c0 2023-10-06T10:46:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.636+0000 7fcc34ff9700 1 --2- 172.21.15.92:0/564851788 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcc2c066f30 0x7fcc2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.637+0000 7fcc34ff9700 1 -- 172.21.15.92:0/564851788 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7fcc38086160 con 0x7fcc400c92c0 2023-10-06T10:46:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.637+0000 7fcc44a8a700 1 --2- 172.21.15.92:0/564851788 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcc2c066f30 0x7fcc2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.637+0000 7fcc44a8a700 1 --2- 172.21.15.92:0/564851788 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcc2c066f30 0x7fcc2c0693e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c007a30 tx=0x7fcc3c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.641+0000 7fcc34ff9700 1 -- 172.21.15.92:0/564851788 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc38053440 con 0x7fcc400c92c0 2023-10-06T10:46:23.755 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:46:23.757 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:46:23.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.830+0000 7fcc45a8c700 1 -- 172.21.15.92:0/564851788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fcc400cc600 con 0x7fcc400c92c0 2023-10-06T10:46:23.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.833+0000 7fcc34ff9700 1 -- 172.21.15.92:0/564851788 <== mon.0 v2:172.21.15.92:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fcc38053440 con 0x7fcc400c92c0 2023-10-06T10:46:23.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.834+0000 7fcc34ff9700 1 -- 172.21.15.92:0/564851788 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v22)=0 v22) v1 ==== 123+0+0 (secure 0 0 0) 0x7fcc38056a60 con 0x7fcc400c92c0 2023-10-06T10:46:23.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.836+0000 7fcc45a8c700 1 -- 172.21.15.92:0/564851788 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcc2c066f30 msgr2=0x7fcc2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:23.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.836+0000 7fcc45a8c700 1 --2- 172.21.15.92:0/564851788 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcc2c066f30 0x7fcc2c0693e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c007a30 tx=0x7fcc3c009040 comp rx=0 tx=0).stop 2023-10-06T10:46:23.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.836+0000 7fcc45a8c700 1 -- 172.21.15.92:0/564851788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc400c92c0 msgr2=0x7fcc400c9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:23.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.836+0000 7fcc45a8c700 1 --2- 172.21.15.92:0/564851788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc400c92c0 0x7fcc400c9730 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fcc38016000 tx=0x7fcc3801a000 comp rx=0 tx=0).stop 2023-10-06T10:46:23.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.837+0000 7fcc45a8c700 1 -- 172.21.15.92:0/564851788 shutdown_connections 2023-10-06T10:46:23.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.837+0000 7fcc45a8c700 1 --2- 172.21.15.92:0/564851788 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7fcc2c066f30 0x7fcc2c0693e0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:23.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.837+0000 7fcc45a8c700 1 --2- 172.21.15.92:0/564851788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc400c92c0 0x7fcc400c9730 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:23.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.837+0000 7fcc45a8c700 1 --2- 172.21.15.92:0/564851788 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc400c7380 0x7fcc40155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:23.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.837+0000 7fcc45a8c700 1 --2- 172.21.15.92:0/564851788 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc400c7750 0x7fcc400b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:23.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.837+0000 7fcc45a8c700 1 -- 172.21.15.92:0/564851788 >> 172.21.15.92:0/564851788 conn(0x7fcc40099520 msgr2=0x7fcc400c5350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:23.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.837+0000 7fcc45a8c700 1 -- 172.21.15.92:0/564851788 shutdown_connections 2023-10-06T10:46:23.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:23.837+0000 7fcc45a8c700 1 -- 172.21.15.92:0/564851788 wait complete. 2023-10-06T10:46:23.851 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T10:46:23.851 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T10:46:24.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.003+0000 7f6bba71d700 1 Processor -- start 2023-10-06T10:46:24.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.003+0000 7f6bba71d700 1 -- start start 2023-10-06T10:46:24.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.004+0000 7f6bba71d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb4159560 0x7f6bb40aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:24.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.004+0000 7f6bba71d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb40b0c60 con 0x7f6bb40b49a0 2023-10-06T10:46:24.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.004+0000 7f6bba71d700 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb40b0f40 con 0x7f6bb40b0760 2023-10-06T10:46:24.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.004+0000 7f6bba71d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb40b1220 con 0x7f6bb4159560 2023-10-06T10:46:24.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.004+0000 7f6bb37fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6bb40b49a0 0x7f6bb40aaa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58368/0 (socket says 172.21.15.92:58368) 2023-10-06T10:46:24.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.004+0000 7f6bb37fe700 1 -- 172.21.15.92:0/964149498 learned_addr learned my addr 172.21.15.92:0/964149498 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T10:46:24.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.004+0000 7f6bb3fff700 1 --2- 172.21.15.92:0/964149498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb4159560 0x7f6bb40aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:24.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb2ffd700 1 -- 172.21.15.92:0/964149498 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360295593 0 0) 0x7f6bb40b0c60 con 0x7f6bb40b49a0 2023-10-06T10:46:24.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb2ffd700 1 -- 172.21.15.92:0/964149498 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b9c003570 con 0x7f6bb40b49a0 2023-10-06T10:46:24.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb2ffd700 1 -- 172.21.15.92:0/964149498 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3964921266 0 0) 0x7f6bb40b0f40 con 0x7f6bb40b0760 2023-10-06T10:46:24.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb2ffd700 1 -- 172.21.15.92:0/964149498 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bb40b0c60 con 0x7f6bb40b0760 2023-10-06T10:46:24.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb2ffd700 1 -- 172.21.15.92:0/964149498 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2867343379 0 0) 0x7f6b9c003570 con 0x7f6bb40b49a0 2023-10-06T10:46:24.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb2ffd700 1 -- 172.21.15.92:0/964149498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb4159560 msgr2=0x7f6bb40aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:24.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb2ffd700 1 --2- 172.21.15.92:0/964149498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb4159560 0x7f6bb40aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:24.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb2ffd700 1 -- 172.21.15.92:0/964149498 >> v1:172.21.15.124:6789/0 conn(0x7f6bb40b0760 legacy=0x7f6bb40b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:24.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb2ffd700 1 -- 172.21.15.92:0/964149498 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb40b1500 con 0x7f6bb40b49a0 2023-10-06T10:46:24.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb2ffd700 1 -- 172.21.15.92:0/964149498 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2885415442 0 0) 0x7f6ba8001470 con 0x7f6bb40b49a0 2023-10-06T10:46:24.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb2ffd700 1 -- 172.21.15.92:0/964149498 >> v1:172.21.15.92:6789/0 conn(0x7f6bb40b49a0 legacy=0x7f6bb40aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:24.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb2ffd700 1 --2- 172.21.15.92:0/964149498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b9c003c20 0x7f6b9c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:24.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb2ffd700 1 --2- 172.21.15.92:0/964149498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb4159560 0x7f6b9c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:24.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb2ffd700 1 --2- 172.21.15.92:0/964149498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b9c004d50 0x7f6b9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:24.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb2ffd700 1 -- 172.21.15.92:0/964149498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb40b1220 con 0x7f6b9c003c20 2023-10-06T10:46:24.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb2ffd700 1 -- 172.21.15.92:0/964149498 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c00bb40 con 0x7f6b9c004d50 2023-10-06T10:46:24.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb2ffd700 1 -- 172.21.15.92:0/964149498 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c00bd00 con 0x7f6bb4159560 2023-10-06T10:46:24.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb3fff700 1 --2- 172.21.15.92:0/964149498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b9c003c20 0x7f6b9c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:24.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb37fe700 1 --2- 172.21.15.92:0/964149498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb4159560 0x7f6b9c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:24.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb8cba700 1 --2- 172.21.15.92:0/964149498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b9c004d50 0x7f6b9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:24.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb3fff700 1 -- 172.21.15.92:0/964149498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb4159560 msgr2=0x7f6b9c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:24.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb3fff700 1 --2- 172.21.15.92:0/964149498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb4159560 0x7f6b9c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:24.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb3fff700 1 -- 172.21.15.92:0/964149498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b9c004d50 msgr2=0x7f6b9c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T10:46:24.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb3fff700 1 --2- 172.21.15.92:0/964149498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b9c004d50 0x7f6b9c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:24.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb3fff700 1 -- 172.21.15.92:0/964149498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9c003570 con 0x7f6b9c003c20 2023-10-06T10:46:24.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb3fff700 1 --2- 172.21.15.92:0/964149498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b9c003c20 0x7f6b9c004010 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009bf0 tx=0x7f6ba4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e0cc79259ea756 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:24.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.005+0000 7f6bb2ffd700 1 -- 172.21.15.92:0/964149498 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4005a00 con 0x7f6b9c003c20 2023-10-06T10:46:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.006+0000 7f6bb2ffd700 1 -- 172.21.15.92:0/964149498 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6ba4005b60 con 0x7f6b9c003c20 2023-10-06T10:46:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.006+0000 7f6bb2ffd700 1 -- 172.21.15.92:0/964149498 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba400e710 con 0x7f6b9c003c20 2023-10-06T10:46:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.006+0000 7f6bba71d700 1 -- 172.21.15.92:0/964149498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b9c003c20 msgr2=0x7f6b9c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.006+0000 7f6bba71d700 1 --2- 172.21.15.92:0/964149498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b9c003c20 0x7f6b9c004010 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009bf0 tx=0x7f6ba4009120 comp rx=0 tx=0).stop 2023-10-06T10:46:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.006+0000 7f6bb8cba700 1 -- 172.21.15.92:0/964149498 reap_dead start 2023-10-06T10:46:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.006+0000 7f6bba71d700 1 -- 172.21.15.92:0/964149498 shutdown_connections 2023-10-06T10:46:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.006+0000 7f6bba71d700 1 -- 172.21.15.92:0/964149498 >> 172.21.15.92:0/964149498 conn(0x7f6bb40997f0 msgr2=0x7f6bb406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.006+0000 7f6bba71d700 1 -- 172.21.15.92:0/964149498 shutdown_connections 2023-10-06T10:46:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.006+0000 7f6bba71d700 1 -- 172.21.15.92:0/964149498 wait complete. 2023-10-06T10:46:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.007+0000 7f6bba71d700 1 Processor -- start 2023-10-06T10:46:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.007+0000 7f6bba71d700 1 -- start start 2023-10-06T10:46:24.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.008+0000 7f6bba71d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b9c003c20 0x7f6bb4158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:24.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.008+0000 7f6bba71d700 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb409d8f0 0x7f6bb40ac040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:24.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.008+0000 7f6bba71d700 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bb409cf60 0x7f6bb40acb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:24.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.008+0000 7f6bba71d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb40ab310 con 0x7f6b9c003c20 2023-10-06T10:46:24.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.008+0000 7f6bba71d700 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb40abbb0 con 0x7f6bb409cf60 2023-10-06T10:46:24.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.008+0000 7f6bba71d700 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb409ddb0 con 0x7f6bb409d8f0 2023-10-06T10:46:24.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.008+0000 7f6bb3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b9c003c20 0x7f6bb4158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:24.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.008+0000 7f6bb3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b9c003c20 0x7f6bb4158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45360/0 (socket says 172.21.15.92:45360) 2023-10-06T10:46:24.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.008+0000 7f6bb3fff700 1 -- 172.21.15.92:0/3946975278 learned_addr learned my addr 172.21.15.92:0/3946975278 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T10:46:24.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.008+0000 7f6bb8cba700 1 --2- 172.21.15.92:0/3946975278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bb409cf60 0x7f6bb40acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:24.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.008+0000 7f6bb37fe700 1 --2- 172.21.15.92:0/3946975278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb409d8f0 0x7f6bb40ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:24.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.008+0000 7f6bb3fff700 1 -- 172.21.15.92:0/3946975278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb409d8f0 msgr2=0x7f6bb40ac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:24.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.008+0000 7f6bb3fff700 1 --2- 172.21.15.92:0/3946975278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb409d8f0 0x7f6bb40ac040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:24.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.008+0000 7f6bb3fff700 1 -- 172.21.15.92:0/3946975278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bb409cf60 msgr2=0x7f6bb40acb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:24.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.008+0000 7f6bb3fff700 1 --2- 172.21.15.92:0/3946975278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bb409cf60 0x7f6bb40acb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:24.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.008+0000 7f6bb3fff700 1 -- 172.21.15.92:0/3946975278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb40ab560 con 0x7f6b9c003c20 2023-10-06T10:46:24.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.009+0000 7f6bb3fff700 1 --2- 172.21.15.92:0/3946975278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b9c003c20 0x7f6bb4158010 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4000e10 tx=0x7f6ba4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:24.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.009+0000 7f6b9bfff700 1 -- 172.21.15.92:0/3946975278 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4006240 con 0x7f6b9c003c20 2023-10-06T10:46:24.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.009+0000 7f6b9bfff700 1 -- 172.21.15.92:0/3946975278 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6ba40063c0 con 0x7f6b9c003c20 2023-10-06T10:46:24.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.009+0000 7f6bba71d700 1 -- 172.21.15.92:0/3946975278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb40ab840 con 0x7f6b9c003c20 2023-10-06T10:46:24.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.009+0000 7f6b9bfff700 1 -- 172.21.15.92:0/3946975278 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba40067a0 con 0x7f6b9c003c20 2023-10-06T10:46:24.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.009+0000 7f6bba71d700 1 -- 172.21.15.92:0/3946975278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb40af700 con 0x7f6b9c003c20 2023-10-06T10:46:24.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.010+0000 7f6bba71d700 1 -- 172.21.15.92:0/3946975278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb4064a90 con 0x7f6b9c003c20 2023-10-06T10:46:24.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.010+0000 7f6b9bfff700 1 -- 172.21.15.92:0/3946975278 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ba4013680 con 0x7f6b9c003c20 2023-10-06T10:46:24.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.010+0000 7f6b9bfff700 1 --2- 172.21.15.92:0/3946975278 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6ba0066f30 0x7f6ba00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T10:46:24.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.010+0000 7f6b9bfff700 1 -- 172.21.15.92:0/3946975278 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f6ba4085de0 con 0x7f6b9c003c20 2023-10-06T10:46:24.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.010+0000 7f6bb37fe700 1 --2- 172.21.15.92:0/3946975278 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6ba0066f30 0x7f6ba00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T10:46:24.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.011+0000 7f6bb37fe700 1 --2- 172.21.15.92:0/3946975278 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6ba0066f30 0x7f6ba00693e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40ce160 tx=0x7f6ba8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T10:46:24.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.015+0000 7f6b9bfff700 1 -- 172.21.15.92:0/3946975278 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ba4053080 con 0x7f6b9c003c20 2023-10-06T10:46:24.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.238+0000 7f6bba71d700 1 -- 172.21.15.92:0/3946975278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bb40b17e0 con 0x7f6b9c003c20 2023-10-06T10:46:24.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.239+0000 7f6b9bfff700 1 -- 172.21.15.92:0/3946975278 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+3536 (secure 0 0 0) 0x7f6ba402b090 con 0x7f6b9c003c20 2023-10-06T10:46:24.241 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T10:46:24.241 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":112,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6834/2016052382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2016052382},{"type":"v1","addr":"172.21.15.124:6834","nonce":2016052382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":7262,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/4121585966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6833","nonce":4121585966},{"type":"v1","addr":"172.21.15.124:6835","nonce":4121585966}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":112},{"gid":9465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/100707971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":100707971},{"type":"v1","addr":"172.21.15.92:6835","nonce":100707971}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":112},{"gid":9474,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2254155169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2254155169},{"type":"v1","addr":"172.21.15.92:6837","nonce":2254155169}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":112}],"filesystems":[]} 2023-10-06T10:46:24.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.241+0000 7f6bba71d700 1 -- 172.21.15.92:0/3946975278 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6ba0066f30 msgr2=0x7f6ba00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:24.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.241+0000 7f6bba71d700 1 --2- 172.21.15.92:0/3946975278 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6ba0066f30 0x7f6ba00693e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40ce160 tx=0x7f6ba8009040 comp rx=0 tx=0).stop 2023-10-06T10:46:24.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.241+0000 7f6bba71d700 1 -- 172.21.15.92:0/3946975278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b9c003c20 msgr2=0x7f6bb4158010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T10:46:24.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.241+0000 7f6bba71d700 1 --2- 172.21.15.92:0/3946975278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b9c003c20 0x7f6bb4158010 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4000e10 tx=0x7f6ba4009120 comp rx=0 tx=0).stop 2023-10-06T10:46:24.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.241+0000 7f6bba71d700 1 -- 172.21.15.92:0/3946975278 shutdown_connections 2023-10-06T10:46:24.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.241+0000 7f6bba71d700 1 --2- 172.21.15.92:0/3946975278 >> [v2:172.21.15.92:6832/90373,v1:172.21.15.92:6833/90373] conn(0x7f6ba0066f30 0x7f6ba00693e0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:24.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.241+0000 7f6bba71d700 1 --2- 172.21.15.92:0/3946975278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bb409cf60 0x7f6bb40acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:24.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.241+0000 7f6bba71d700 1 --2- 172.21.15.92:0/3946975278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b9c003c20 0x7f6bb4158010 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:24.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.241+0000 7f6bba71d700 1 --2- 172.21.15.92:0/3946975278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bb409d8f0 0x7f6bb40ac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T10:46:24.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.241+0000 7f6bba71d700 1 -- 172.21.15.92:0/3946975278 >> 172.21.15.92:0/3946975278 conn(0x7f6bb40997f0 msgr2=0x7f6bb4069bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T10:46:24.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.241+0000 7f6bba71d700 1 -- 172.21.15.92:0/3946975278 shutdown_connections 2023-10-06T10:46:24.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T10:46:24.241+0000 7f6bba71d700 1 -- 172.21.15.92:0/3946975278 wait complete. 2023-10-06T10:46:24.245 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 112 2023-10-06T10:46:24.254 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-06T10:46:24.255 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-06T10:46:24.255 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T10:46:24.255 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:46:24.263 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T10:46:24.263 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T10:46:24.263 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:46:24.312 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T10:46:24.313 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T10:46:24.313 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:46:24.323 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T10:46:24.323 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T10:46:24.323 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:46:24.374 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T10:46:24.374 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-06T10:46:24.375 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-06T10:46:24.375 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:46:24.401 INFO:tasks.ceph.osd.0:Stopped 2023-10-06T10:46:24.402 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-06T10:46:24.402 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:46:24.466 INFO:tasks.ceph.osd.1:Stopped 2023-10-06T10:46:24.466 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-06T10:46:24.466 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:46:24.528 INFO:tasks.ceph.osd.2:Stopped 2023-10-06T10:46:24.529 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-06T10:46:24.529 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:46:24.597 INFO:tasks.ceph.osd.3:Stopped 2023-10-06T10:46:24.597 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-06T10:46:24.598 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:46:24.627 INFO:tasks.ceph.osd.4:Stopped 2023-10-06T10:46:24.627 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-06T10:46:24.627 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:46:24.699 INFO:tasks.ceph.osd.5:Stopped 2023-10-06T10:46:24.699 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-06T10:46:24.699 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:46:24.775 INFO:tasks.ceph.osd.6:Stopped 2023-10-06T10:46:24.775 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-06T10:46:24.775 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:46:24.845 INFO:tasks.ceph.osd.7:Stopped 2023-10-06T10:46:24.845 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-06T10:46:24.845 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-06T10:46:24.845 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:46:24.870 INFO:tasks.ceph.mgr.y:Stopped 2023-10-06T10:46:24.870 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-06T10:46:24.870 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:46:24.910 INFO:tasks.ceph.mgr.x:Stopped 2023-10-06T10:46:24.910 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-06T10:46:24.911 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-06T10:46:24.911 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:46:24.921 INFO:tasks.ceph.mon.a:Stopped 2023-10-06T10:46:24.921 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-06T10:46:24.922 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:46:24.962 INFO:tasks.ceph.mon.b:Stopped 2023-10-06T10:46:24.962 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-06T10:46:24.962 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T10:46:25.013 INFO:tasks.ceph.mon.c:Stopped 2023-10-06T10:46:25.014 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-06T10:46:25.014 DEBUG:teuthology.orchestra.run.smithi092:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Scrub error on inode' | head -n 1 2023-10-06T10:46:25.042 INFO:teuthology.orchestra.run.smithi092.stdout:1696588696.3345454 mon.a (mon.0) 213 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T10:46:25.042 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-06T10:46:25.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Scrub error on inode' | head -n 1 2023-10-06T10:46:25.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Scrub error on inode' | head -n 1 2023-10-06T10:46:25.178 DEBUG:teuthology.orchestra.run.smithi092:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Scrub error on inode' | head -n 1 2023-10-06T10:46:25.246 INFO:teuthology.orchestra.run.smithi092.stdout:1696588696.3345454 mon.a (mon.0) 213 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T10:46:25.246 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T10:46:25.246 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-06T10:46:25.567 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T10:46:25.567 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-06T10:46:25.687 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T10:46:25.687 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-06T10:46:25.779 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T10:46:25.779 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-06T10:46:25.849 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi124.front.sepia.ceph.com 2023-10-06T10:46:25.849 DEBUG:teuthology.orchestra.run.smithi124:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-06T10:46:26.068 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi124.front.sepia.ceph.com 2023-10-06T10:46:26.069 DEBUG:teuthology.orchestra.run.smithi124:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-06T10:46:26.150 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi124.front.sepia.ceph.com 2023-10-06T10:46:26.151 DEBUG:teuthology.orchestra.run.smithi124:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-06T10:46:26.222 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi124.front.sepia.ceph.com 2023-10-06T10:46:26.222 DEBUG:teuthology.orchestra.run.smithi124:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-06T10:46:26.302 INFO:tasks.ceph:Archiving mon data... 2023-10-06T10:46:26.713 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412360/data/mon.a.tgz 2023-10-06T10:46:26.713 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2023-10-06T10:46:26.730 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.mu6zxnRRJF 2023-10-06T10:46:26.730 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.mu6zxnRRJF 2023-10-06T10:46:26.885 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.mu6zxnRRJF 2023-10-06T10:46:26.967 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.mu6zxnRRJF is 447KB 2023-10-06T10:46:27.034 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.mu6zxnRRJF 2023-10-06T10:46:27.048 DEBUG:teuthology.misc:Transferring archived files from smithi124:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412360/data/mon.b.tgz 2023-10-06T10:46:27.048 DEBUG:teuthology.orchestra.run.smithi124:> mktemp 2023-10-06T10:46:27.065 INFO:teuthology.orchestra.run.smithi124.stdout:/tmp/tmp.eNQ6HADmce 2023-10-06T10:46:27.065 DEBUG:teuthology.orchestra.run.smithi124:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.eNQ6HADmce 2023-10-06T10:46:27.236 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 0666 /tmp/tmp.eNQ6HADmce 2023-10-06T10:46:27.317 DEBUG:teuthology.orchestra.remote:smithi124:/tmp/tmp.eNQ6HADmce is 463KB 2023-10-06T10:46:27.415 DEBUG:teuthology.orchestra.run.smithi124:> rm -fr /tmp/tmp.eNQ6HADmce 2023-10-06T10:46:27.429 DEBUG:teuthology.misc:Transferring archived files from smithi124:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412360/data/mon.c.tgz 2023-10-06T10:46:27.429 DEBUG:teuthology.orchestra.run.smithi124:> mktemp 2023-10-06T10:46:27.482 INFO:teuthology.orchestra.run.smithi124.stdout:/tmp/tmp.G5ihqTOmjx 2023-10-06T10:46:27.483 DEBUG:teuthology.orchestra.run.smithi124:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.G5ihqTOmjx 2023-10-06T10:46:27.637 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 0666 /tmp/tmp.G5ihqTOmjx 2023-10-06T10:46:27.717 DEBUG:teuthology.orchestra.remote:smithi124:/tmp/tmp.G5ihqTOmjx is 464KB 2023-10-06T10:46:27.813 DEBUG:teuthology.orchestra.run.smithi124:> rm -fr /tmp/tmp.G5ihqTOmjx 2023-10-06T10:46:27.827 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-06T10:46:27.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T10:46:27.829 DEBUG:teuthology.orchestra.run.smithi124:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T10:46:27.890 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-06T10:46:27.890 DEBUG:teuthology.orchestra.run.smithi092:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T10:46:27.895 DEBUG:teuthology.orchestra.run.smithi124:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T10:46:27.918 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T10:46:27.955 INFO:teuthology.orchestra.run.smithi124.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T10:46:27.961 INFO:tasks.ceph:Archiving crash dumps... 2023-10-06T10:46:27.963 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412360/remote/smithi092/crash 2023-10-06T10:46:27.963 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T10:46:27.988 DEBUG:teuthology.misc:Transferring archived files from smithi124:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412360/remote/smithi124/crash 2023-10-06T10:46:27.989 DEBUG:teuthology.orchestra.run.smithi124:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T10:46:28.024 INFO:tasks.ceph:Shutting down logrotate 2023-10-06T10:46:28.025 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:46:28.033 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T10:46:28.111 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T10:46:28.135 DEBUG:teuthology.orchestra.run.smithi124:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T10:46:28.168 INFO:tasks.ceph:Compressing logs... 2023-10-06T10:46:28.168 DEBUG:teuthology.orchestra.run.smithi092:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T10:46:28.178 DEBUG:teuthology.orchestra.run.smithi124:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T10:46:34.053 INFO:tasks.ceph:Archiving logs... 2023-10-06T10:46:34.054 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412360/remote/smithi092/log 2023-10-06T10:46:34.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T10:46:35.263 DEBUG:teuthology.misc:Transferring archived files from smithi124:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412360/remote/smithi124/log 2023-10-06T10:46:35.264 DEBUG:teuthology.orchestra.run.smithi124:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T10:46:35.950 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-06T10:46:35.959 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-06T10:46:35.960 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T10:46:35.962 DEBUG:teuthology.orchestra.run.smithi124:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T10:46:36.045 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-06T10:46:36.058 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-06T10:46:36.058 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T10:46:36.061 DEBUG:teuthology.orchestra.run.smithi124:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T10:46:36.072 INFO:teuthology.orchestra.run.smithi092.stderr:bash: ntpq: command not found 2023-10-06T10:46:36.074 INFO:teuthology.orchestra.run.smithi124.stderr:bash: ntpq: command not found 2023-10-06T10:46:36.075 INFO:teuthology.orchestra.run.smithi092.stdout:210 Number of sources = 4 2023-10-06T10:46:36.076 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T10:46:36.076 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2023-10-06T10:46:36.076 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 22 -554us[ -554us] +/- 73ms 2023-10-06T10:46:36.076 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv02.front.sepia.ceph.com 3 6 377 24 -225us[ -232us] +/- 49ms 2023-10-06T10:46:36.076 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 +830us[ +830us] +/- 68ms 2023-10-06T10:46:36.076 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T10:46:36.078 INFO:teuthology.orchestra.run.smithi124.stdout:210 Number of sources = 4 2023-10-06T10:46:36.078 INFO:teuthology.orchestra.run.smithi124.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T10:46:36.078 INFO:teuthology.orchestra.run.smithi124.stdout:=============================================================================== 2023-10-06T10:46:36.079 INFO:teuthology.orchestra.run.smithi124.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 23 -557us[ -557us] +/- 73ms 2023-10-06T10:46:36.079 INFO:teuthology.orchestra.run.smithi124.stdout:^* hv02.front.sepia.ceph.com 3 6 377 26 -224us[ -211us] +/- 49ms 2023-10-06T10:46:36.079 INFO:teuthology.orchestra.run.smithi124.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 +836us[ +836us] +/- 68ms 2023-10-06T10:46:36.079 INFO:teuthology.orchestra.run.smithi124.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T10:46:36.079 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-06T10:46:36.121 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-06T10:46:36.121 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-06T10:46:36.154 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T10:46:36.261 DEBUG:teuthology.orchestra.run.smithi124:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T10:46:36.351 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T10:46:36.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T10:46:36.382 DEBUG:teuthology.orchestra.run.smithi124:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T10:46:36.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T10:46:36.412 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-06T10:46:36.422 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-06T10:46:36.446 INFO:teuthology.task.internal:Duration was 1175.280420 seconds 2023-10-06T10:46:36.446 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-06T10:46:36.480 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-06T10:46:36.480 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T10:46:36.482 DEBUG:teuthology.orchestra.run.smithi124:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T10:46:36.524 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T10:46:36.525 INFO:teuthology.orchestra.run.smithi124.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T10:46:36.660 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-06T10:46:36.661 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T10:46:36.661 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T10:46:36.719 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi124.front.sepia.ceph.com 2023-10-06T10:46:36.720 DEBUG:teuthology.orchestra.run.smithi124:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T10:46:36.738 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-06T10:46:36.738 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T10:46:36.762 DEBUG:teuthology.orchestra.run.smithi124:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T10:46:36.817 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-06T10:46:36.817 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T10:46:36.846 DEBUG:teuthology.orchestra.run.smithi124:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T10:46:36.935 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-06T10:46:36.945 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-06T10:46:36.945 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T10:46:36.969 DEBUG:teuthology.orchestra.run.smithi124:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T10:46:36.994 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-06T10:46:37.006 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T10:46:37.012 DEBUG:teuthology.orchestra.run.smithi124:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T10:46:37.033 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2023-10-06T10:46:37.058 INFO:teuthology.orchestra.run.smithi124.stdout:kernel.core_pattern = core 2023-10-06T10:46:37.071 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T10:46:37.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T10:46:37.101 DEBUG:teuthology.orchestra.run.smithi124:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T10:46:37.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T10:46:37.125 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-06T10:46:37.136 INFO:teuthology.task.internal:Transferring archived files... 2023-10-06T10:46:37.137 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412360/remote/smithi092 2023-10-06T10:46:37.137 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T10:46:37.178 DEBUG:teuthology.misc:Transferring archived files from smithi124:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412360/remote/smithi124 2023-10-06T10:46:37.179 DEBUG:teuthology.orchestra.run.smithi124:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T10:46:37.214 INFO:teuthology.task.internal:Removing archive directory... 2023-10-06T10:46:37.214 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T10:46:37.216 DEBUG:teuthology.orchestra.run.smithi124:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T10:46:37.263 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-06T10:46:37.275 INFO:teuthology.task.internal:Not uploading archives. 2023-10-06T10:46:37.275 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-06T10:46:37.297 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-06T10:46:37.297 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T10:46:37.299 DEBUG:teuthology.orchestra.run.smithi124:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T10:46:37.311 INFO:teuthology.orchestra.run.smithi092.stdout: 263007 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 10:46 /home/ubuntu/cephtest 2023-10-06T10:46:37.317 INFO:teuthology.orchestra.run.smithi124.stdout: 262908 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 10:46 /home/ubuntu/cephtest 2023-10-06T10:46:37.319 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-06T10:46:37.331 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-06T10:46:37.397 INFO:teuthology.nuke:Checking targets against current locks 2023-10-06T10:46:37.422 DEBUG:teuthology.nuke:shortname: smithi092 2023-10-06T10:46:37.422 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T10:46:37.446 DEBUG:teuthology.nuke:shortname: smithi124 2023-10-06T10:46:37.447 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T10:46:37.450 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412360', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 10:19:41.323619', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF53OE4i5vcG3W2Q2lwwvr0uEhnNAo2G09ujsL4/0+SF1JifRhTX0X3iB1Zpi5s16tvOQdqI//q3UQa2B/6tRpQ='} 2023-10-06T10:46:37.472 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi124.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412360', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 10:19:41.321956', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ/sH53ZqpmGXnu83Xkf3Q5s6Bg6EwuiibrtBaWjsLHpp52l0b1kwbr7opnFzmdmy/p6yF4gbYp7YLTzdj6jNUo='} 2023-10-06T10:46:37.506 INFO:teuthology.orchestra.console.smithi092:Power off 2023-10-06T10:46:37.506 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T10:46:37.529 DEBUG:teuthology.orchestra.console.smithi092:power off output: Chassis Power Control: Down/Off 2023-10-06T10:46:37.529 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T10:46:37.544 INFO:teuthology.orchestra.console.smithi124:Power off 2023-10-06T10:46:37.544 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T10:46:37.562 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2023-10-06T10:46:37.566 DEBUG:teuthology.orchestra.console.smithi124:power off output: Chassis Power Control: Down/Off 2023-10-06T10:46:37.566 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T10:46:37.587 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is on 2023-10-06T10:46:41.563 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T10:46:41.588 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T10:46:41.674 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2023-10-06T10:46:41.699 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is on 2023-10-06T10:46:45.678 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T10:46:45.700 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T10:46:45.791 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is off 2023-10-06T10:46:45.811 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is on 2023-10-06T10:46:45.892 INFO:teuthology.orchestra.console.smithi092:Power off completed 2023-10-06T10:46:49.813 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T10:46:49.924 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is off 2023-10-06T10:46:50.024 INFO:teuthology.orchestra.console.smithi124:Power off completed 2023-10-06T10:46:50.129 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/multimds_misc} duration: 1175.2804200649261 failure_reason: '"1696588696.3345454 mon.a (mon.0) 213 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-10-06T10:46:50.130 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T10:46:50.212 INFO:teuthology.run:FAIL